diff --git a/.gitattributes b/.gitattributes index 1f032400db4244b1483fdd19f541e70d7b534a8e..7030d5791d6b4ddd9a394d7e3d00a080ca25495f 100644 --- a/.gitattributes +++ b/.gitattributes @@ -49,3 +49,9 @@ paper_results/figures/Fig1_editable.pptx filter=lfs diff=lfs merge=lfs -text paper_results/figures/flops_performance.png filter=lfs diff=lfs merge=lfs -text paper_results/figures/latency_performance.png filter=lfs diff=lfs merge=lfs -text paper_results/figures/method.pdf filter=lfs diff=lfs merge=lfs -text +releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact.png filter=lfs diff=lfs merge=lfs -text +releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact.png filter=lfs diff=lfs merge=lfs -text +releases/20260913_lr/paper_data/reproduction/construction_image_path.jpg filter=lfs diff=lfs merge=lfs -text +releases/20260913_lr/paper_data/reproduction/nlvr2_image2.jpg filter=lfs diff=lfs merge=lfs -text +releases/20260913_lr/paper_data/test_results.json filter=lfs diff=lfs merge=lfs -text +releases/20260913_lr/paper_data/validation_manifest.json filter=lfs diff=lfs merge=lfs -text diff --git a/releases/20260913_lr/LATEST_RELEASE.md b/releases/20260913_lr/LATEST_RELEASE.md new file mode 100644 index 0000000000000000000000000000000000000000..0031fbe521c8a6cd0b96c75107506352acf03d12 --- /dev/null +++ b/releases/20260913_lr/LATEST_RELEASE.md @@ -0,0 +1,50 @@ +# Latest release: 20260913_lr + +The release contains the latest paper results and the full experiment record. Download artifacts from `nohi191212/ModelCollaboration` on Hugging Face under `releases/20260913_lr/`. + +## Weights and records + +`full_budget_20260913_weights.tar.gz` contains exactly **80 router checkpoints**: 16 model pairs × five seeds for the main table. This release publishes no other router weights. + +The study-specific `_records.tar.gz` archives retain configurations, training histories, validation/test outputs, curves and selection records for the experiments. Extract them into the same artifact directory; paths under `outputs/` are preserved. Earlier fixed-answer CSS/OvA adaptations are separate from the corrected K-class implementations. + +Previously released backbone and specialist weights remain in the root-level `weights.tar`. External models and datasets are linked in `MODELS_AND_DATA.md`. + +## Data + +Eight `training_inputs_.tar.gz` files contain the train/validation/test arrays and example records for the latest 8M frozen-backbone setup. They contain image/text features, the selected specialist hidden state, confidence, output features, validity masks and labels. `input_manifest.json` records dimensions, sample counts and hidden-layer names. Set `ICASSP_DATA_ROOT` to the directory containing `training_inputs/` and this manifest. + +`paired_outcomes.tar.gz` provides the paired correctness and task-metric sufficient statistics used for result replay. `paper_replay_runs.tar.gz` contains the 400 training runs used by the latest Tables 1 and 3, with raw logits and configurations. These two files are sufficient for `scripts/replay_latest_results.py`. + +`additional_analysis_records.tar.gz` contains scoring explorations and cost records. `experiment_sources.tar.gz` preserves original experiment source snapshots, also included in the GitHub repository. + +The latest `paper/data/` directory stores table and figure summaries. Main weights were selected over the full 0–100% validation curve. Table 3 retains its earlier training selection setting, as specified in the supplement and run configurations. + +## Training examples + +Install the dependencies in `requirements.txt` and a PyTorch build for your GPU. Commands below are run from the repository root. They start training only when invoked. + +For normalized LR, download/extract a specialist's input archive and `input_manifest.json`, then run: + +```bash +export ICASSP_DATA_ROOT="$PWD/artifacts/releases/20260913_lr" +python code/experiments/full_budget_train.py --source configs/full_budget/full_cub_MiniCPM_s2026 --output outputs/my_lr_run +``` + +For post-hoc deferral, extract its records archive, choose a saved `config.json`, and run: + +```bash +python code/experiments/train_ltd.py --root "$PWD" --config "$ICASSP_DATA_ROOT/outputs/ltd_fixed_backbone_20260913/runs/cub_MiniCPM_posthoc_c0_s2026/config.json" --output outputs/my_posthoc_run --device cuda +``` + +Use `train_three.py` for a configuration from `three_state_20260913`, and `train_native.py` for a configuration from `ltd_kclass_native_20260913`. Configurations are supplied in the records and weight archives. The original objective experiment sources are in `experiments/source_snapshots/`; the five-seed objective trainer is also in `code/objectives/`. + +To evaluate a newly trained post-hoc/three-state/native run, use its corresponding `evaluate_ltd.py`, `evaluate_three.py` or `evaluate_native.py` with `--help` for arguments. Input paths are resolved through `ICASSP_DATA_ROOT`. + +Earlier architecture studies use other feature sizes or hidden layers. Their original configurations and source snapshots are supplied; regenerating those inputs from images uses the original feature-preparation scripts and the upstream datasets. The cached inputs are the latest common 8M setup. + +## Verification + +`scripts/replay_latest_results.py` recomputes the latest Tables 1 and 3 using the saved per-example outputs. `verification_latest.json` records 80 main runs, 320 Table 3 runs and 19,392 checked curve points. `weight_reload_verification.json` checks all 80 main weights against saved logits using the released cache loader (128 validation inputs per run). + +Original code is MIT licensed. Derived data retain the licenses of their source datasets; consult `MODELS_AND_DATA.md` before redistribution. diff --git a/releases/20260913_lr/MODELS_AND_DATA.md b/releases/20260913_lr/MODELS_AND_DATA.md new file mode 100644 index 0000000000000000000000000000000000000000..2c7c42716b5bf694380a2bd45282075380f4c8ec --- /dev/null +++ b/releases/20260913_lr/MODELS_AND_DATA.md @@ -0,0 +1,51 @@ +# Upstream models and datasets + +### Vision-language models and router teacher + +| Model | Hugging Face download | Role | +|---|---|---| +| MiniCPM-V-4.5 | [openbmb/MiniCPM-V-4_5](https://huggingface.co/openbmb/MiniCPM-V-4_5) | VLM in the main comparisons | +| Qwen3.8-27B-FP8 | [Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8) | FP8 VLM in the main comparisons | +| SigLIP2 Base, patch 16, resolution 224 | [google/siglip2-base-patch16-224](https://huggingface.co/google/siglip2-base-patch16-224) | Teacher for the compact image/text encoders | + +```bash +hf download openbmb/MiniCPM-V-4_5 --local-dir models/MiniCPM-V-4_5 +hf download Qwen/Qwen3.8-27B-FP8 --local-dir models/Qwen3.8-27B-FP8 +hf download google/siglip2-base-patch16-224 --local-dir models/siglip2-base-patch16-224 +``` + +Follow each model card for serving dependencies. Preserve the Qwen FP8 variant and the task prompts in [`prompts/`](prompts/) to match the experiments. + +### Specialist models + +| Specialist / task | Download or implementation | Checkpoint note | +|---|---|---| +| InstanceVG / gRefCOCO | [Hugging Face weights](https://huggingface.co/Dmmm997/InstanceVG/tree/main/gres/InstanceVG-grefcoco), [implementation](https://github.com/Dmmm1997/InstanceVG) | Use the gRefCOCO checkpoint | +| ViLT / NLVR2 | [dandelin/vilt-b32-finetuned-nlvr2](https://huggingface.co/dandelin/vilt-b32-finetuned-nlvr2) | Task-finetuned upstream weights | +| BEiT-3 / NLVR2 | [Raghavan/beit3_base_patch16_224_nlvr2](https://huggingface.co/Raghavan/beit3_base_patch16_224_nlvr2), [implementation](https://github.com/microsoft/unilm/tree/master/beit3) | Task-finetuned upstream weights; preserve the recorded input processing | +| Grounding DINO / gRefCOCO | [Implementation and checkpoints](https://github.com/IDEA-Research/GroundingDINO) | Match the experiment's architecture and checkpoint | +| ResNet-50 / CUB-200-2011 | [Torchvision ResNet-50](https://docs.pytorch.org/vision/stable/models/generated/torchvision.models.resnet50.html) | Initialization only; the CUB-trained checkpoint is also required | +| GLSim / CUB-200-2011 | Paper checkpoint: `cub_vit_b16_16_2.pth` | Exact public download URL pending | +| YOLO26x / ConstructionSite | [Our trained weights](https://huggingface.co/nohi191212/ModelCollaboration/resolve/main/weights.tar) | `specialists/yolo26x/best.pt` inside the archive | +| RT-DETR-X / ConstructionSite | [Our trained weights](https://huggingface.co/nohi191212/ModelCollaboration/resolve/main/weights.tar) | `specialists/rtdetr_x_fp32/best.pt`; recorded inference uses FP32 | + +### Datasets + +| Dataset | Official download / access instructions | Required contents | +|---|---|---| +| CUB-200-2011 | [Caltech dataset record](https://data.caltech.edu/records/65de6-vp158) | Images, classes, bounding boxes and original split annotations | +| gRefCOCO | [Official annotations](https://github.com/henghuiding/gRefCOCO), [COCO images](https://cocodataset.org/#download) | Expressions, annotations and corresponding COCO images; follow upstream image-version instructions | +| NLVR2 | [Official dataset page](https://lil.nlp.cornell.edu/nlvr/), [download instructions](https://github.com/lil-lab/nlvr/tree/master/nlvr2) | Paired images, statements and labels | +| ConstructionSite (ConstructionSite10k) | [LouisChen15/ConstructionSite](https://huggingface.co/datasets/LouisChen15/ConstructionSite) | Images and annotations; accept the dataset's access conditions | + +For ConstructionSite, after access is granted: + +```bash +hf auth login +hf download LouisChen15/ConstructionSite --repo-type dataset --local-dir data/constructionsite10k +``` + +Dataset and model licenses remain those of their authors. ConstructionSite is distributed under CC BY-NC 4.0. Original dataset splits are not automatically the router train/validation/test splits: follow the split definitions, exclusions and evaluation rules in [`paper/supplementary.pdf`](paper/supplementary.pdf). + + +Upstream models and datasets retain their original licenses. diff --git a/releases/20260913_lr/REPRODUCE.md b/releases/20260913_lr/REPRODUCE.md new file mode 100644 index 0000000000000000000000000000000000000000..0031fbe521c8a6cd0b96c75107506352acf03d12 --- /dev/null +++ b/releases/20260913_lr/REPRODUCE.md @@ -0,0 +1,50 @@ +# Latest release: 20260913_lr + +The release contains the latest paper results and the full experiment record. Download artifacts from `nohi191212/ModelCollaboration` on Hugging Face under `releases/20260913_lr/`. + +## Weights and records + +`full_budget_20260913_weights.tar.gz` contains exactly **80 router checkpoints**: 16 model pairs × five seeds for the main table. This release publishes no other router weights. + +The study-specific `_records.tar.gz` archives retain configurations, training histories, validation/test outputs, curves and selection records for the experiments. Extract them into the same artifact directory; paths under `outputs/` are preserved. Earlier fixed-answer CSS/OvA adaptations are separate from the corrected K-class implementations. + +Previously released backbone and specialist weights remain in the root-level `weights.tar`. External models and datasets are linked in `MODELS_AND_DATA.md`. + +## Data + +Eight `training_inputs_.tar.gz` files contain the train/validation/test arrays and example records for the latest 8M frozen-backbone setup. They contain image/text features, the selected specialist hidden state, confidence, output features, validity masks and labels. `input_manifest.json` records dimensions, sample counts and hidden-layer names. Set `ICASSP_DATA_ROOT` to the directory containing `training_inputs/` and this manifest. + +`paired_outcomes.tar.gz` provides the paired correctness and task-metric sufficient statistics used for result replay. `paper_replay_runs.tar.gz` contains the 400 training runs used by the latest Tables 1 and 3, with raw logits and configurations. These two files are sufficient for `scripts/replay_latest_results.py`. + +`additional_analysis_records.tar.gz` contains scoring explorations and cost records. `experiment_sources.tar.gz` preserves original experiment source snapshots, also included in the GitHub repository. + +The latest `paper/data/` directory stores table and figure summaries. Main weights were selected over the full 0–100% validation curve. Table 3 retains its earlier training selection setting, as specified in the supplement and run configurations. + +## Training examples + +Install the dependencies in `requirements.txt` and a PyTorch build for your GPU. Commands below are run from the repository root. They start training only when invoked. + +For normalized LR, download/extract a specialist's input archive and `input_manifest.json`, then run: + +```bash +export ICASSP_DATA_ROOT="$PWD/artifacts/releases/20260913_lr" +python code/experiments/full_budget_train.py --source configs/full_budget/full_cub_MiniCPM_s2026 --output outputs/my_lr_run +``` + +For post-hoc deferral, extract its records archive, choose a saved `config.json`, and run: + +```bash +python code/experiments/train_ltd.py --root "$PWD" --config "$ICASSP_DATA_ROOT/outputs/ltd_fixed_backbone_20260913/runs/cub_MiniCPM_posthoc_c0_s2026/config.json" --output outputs/my_posthoc_run --device cuda +``` + +Use `train_three.py` for a configuration from `three_state_20260913`, and `train_native.py` for a configuration from `ltd_kclass_native_20260913`. Configurations are supplied in the records and weight archives. The original objective experiment sources are in `experiments/source_snapshots/`; the five-seed objective trainer is also in `code/objectives/`. + +To evaluate a newly trained post-hoc/three-state/native run, use its corresponding `evaluate_ltd.py`, `evaluate_three.py` or `evaluate_native.py` with `--help` for arguments. Input paths are resolved through `ICASSP_DATA_ROOT`. + +Earlier architecture studies use other feature sizes or hidden layers. Their original configurations and source snapshots are supplied; regenerating those inputs from images uses the original feature-preparation scripts and the upstream datasets. The cached inputs are the latest common 8M setup. + +## Verification + +`scripts/replay_latest_results.py` recomputes the latest Tables 1 and 3 using the saved per-example outputs. `verification_latest.json` records 80 main runs, 320 Table 3 runs and 19,392 checked curve points. `weight_reload_verification.json` checks all 80 main weights against saved logits using the released cache loader (128 validation inputs per run). + +Original code is MIT licensed. Derived data retain the licenses of their source datasets; consult `MODELS_AND_DATA.md` before redistribution. diff --git a/releases/20260913_lr/appendix.pdf b/releases/20260913_lr/appendix.pdf new file mode 100644 index 0000000000000000000000000000000000000000..d6326edded3b107992be403712fef1f27ffbbdf5 --- /dev/null +++ b/releases/20260913_lr/appendix.pdf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5bfda42c8b67bb5b356d1d316a9f48f9c4466019482dc232eb0017049236119 +size 1090203 diff --git a/releases/20260913_lr/main.pdf b/releases/20260913_lr/main.pdf new file mode 100644 index 0000000000000000000000000000000000000000..66f642496889ee4cfcd5f201111c6fe30cc00fd6 --- /dev/null +++ b/releases/20260913_lr/main.pdf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c5a140a24780ff19e84d870579fc094ab7b8c76296e123f1ab64d2eb52feba38 +size 4826693 diff --git a/releases/20260913_lr/paired_outcomes.tar.gz b/releases/20260913_lr/paired_outcomes.tar.gz new file mode 100644 index 0000000000000000000000000000000000000000..7c55e5c5950aae2c961523e134a2e98be5b48c17 --- /dev/null +++ b/releases/20260913_lr/paired_outcomes.tar.gz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5bc2d10e03e7124ff1a073120cfc1e7951cbb4d8ef016b5a305c30a4c78143fd +size 2668397 diff --git a/releases/20260913_lr/paper_data/budget_diagnostics.json b/releases/20260913_lr/paper_data/budget_diagnostics.json new file mode 100644 index 0000000000000000000000000000000000000000..6ce33a5058450ef99752f69ef16d7453017d6edd --- /dev/null +++ b/releases/20260913_lr/paper_data/budget_diagnostics.json @@ -0,0 +1 @@ +[{"key": ["cub", "cub", "MiniCPM"], "areas": {"confidence": {"validation_fixed_A25": 0.9058333333333335, "test_fixed_A25": 0.6472903003106663, "test_ranked_A25_diagnostic": 0.7895926820849155, "selected_validation_actual_calls": 0.0, "selected_test_actual_calls": 0.0}, "learned": {"validation_fixed_A25": 0.9740333333333333, "test_fixed_A25": 0.8063444943044529, "test_ranked_A25_diagnostic": 0.8089368312046945, "selected_validation_actual_calls": 0.004, "selected_test_actual_calls": 0.0059371763893683125}}}, {"key": ["cub", "cub", "Qwen3.8"], "areas": {"confidence": {"validation_fixed_A25": 0.9171666666666667, "test_fixed_A25": 0.7316189161201244, "test_ranked_A25_diagnostic": 0.8101484294097343, "selected_validation_actual_calls": 0.0, "selected_test_actual_calls": 0.0}, "learned": {"validation_fixed_A25": 0.9834833333333335, "test_fixed_A25": 0.8129651363479462, "test_ranked_A25_diagnostic": 0.8117915084570246, "selected_validation_actual_calls": 0.0, "selected_test_actual_calls": 0.0}}}, {"key": ["cub", "glsim", "MiniCPM"], "areas": {"confidence": {"validation_fixed_A25": 0.9005000000000001, "test_fixed_A25": 0.7253365550569554, "test_ranked_A25_diagnostic": 0.8519848118743528, "selected_validation_actual_calls": 0.0, "selected_test_actual_calls": 0.0}, "learned": {"validation_fixed_A25": 0.9781666666666666, "test_fixed_A25": 0.8896064894718676, "test_ranked_A25_diagnostic": 0.8920262340352089, "selected_validation_actual_calls": 0.004, "selected_test_actual_calls": 0.008491542975491888}}}, {"key": ["cub", "glsim", "Qwen3.8"], "areas": {"confidence": {"validation_fixed_A25": 0.9213333333333334, "test_fixed_A25": 0.7860545391784605, "test_ranked_A25_diagnostic": 0.8691232309285468, "selected_validation_actual_calls": 0.0, "selected_test_actual_calls": 0.0}, "learned": {"validation_fixed_A25": 0.9858, "test_fixed_A25": 0.8977666551605109, "test_ranked_A25_diagnostic": 0.897169485674836, "selected_validation_actual_calls": 0.0, "selected_test_actual_calls": 0.0}}}, {"key": ["grefcoco", "groundingdino", "MiniCPM"], "areas": {"confidence": {"validation_fixed_A25": 0.5566729917773561, "test_fixed_A25": 0.27538212857669514, "test_ranked_A25_diagnostic": 0.2786603522105323, "selected_validation_actual_calls": 1.0, "selected_test_actual_calls": 1.0}, "learned": {"validation_fixed_A25": 0.6126923887834703, "test_fixed_A25": 0.37841703768822854, "test_ranked_A25_diagnostic": 0.3486317102912402, "selected_validation_actual_calls": 0.4579520697167756, "selected_test_actual_calls": 0.5818676800045374}}}, {"key": ["grefcoco", "groundingdino", "Qwen3.8"], "areas": {"confidence": {"validation_fixed_A25": 0.5648886077728582, "test_fixed_A25": 0.28625187873975555, "test_ranked_A25_diagnostic": 0.29324220854720245, "selected_validation_actual_calls": 1.0, "selected_test_actual_calls": 1.0}, "learned": {"validation_fixed_A25": 0.647486119895987, "test_fixed_A25": 0.42089272041516607, "test_ranked_A25_diagnostic": 0.3745852593369821, "selected_validation_actual_calls": 0.7759786351816713, "selected_test_actual_calls": 0.8846666477611093}}}, {"key": ["grefcoco", "instancevg", "MiniCPM"], "areas": {"confidence": {"validation_fixed_A25": 0.7465176751704266, "test_fixed_A25": 0.6639253608598249, "test_ranked_A25_diagnostic": 0.6590023537418825, "selected_validation_actual_calls": 0.6499402628434886, "selected_test_actual_calls": 0.5209426310864078}, "learned": {"validation_fixed_A25": 0.7538997821350764, "test_fixed_A25": 0.678765278053484, "test_ranked_A25_diagnostic": 0.6785259336982106, "selected_validation_actual_calls": 0.14395951929158762, "selected_test_actual_calls": 0.19038085245157815}}}, {"key": ["grefcoco", "instancevg", "Qwen3.8"], "areas": {"confidence": {"validation_fixed_A25": 0.7477616136060159, "test_fixed_A25": 0.6640728242066757, "test_ranked_A25_diagnostic": 0.6611405722712191, "selected_validation_actual_calls": 0.7099585353854804, "selected_test_actual_calls": 0.6838612710206166}, "learned": {"validation_fixed_A25": 0.7825750228406776, "test_fixed_A25": 0.7278393783852765, "test_ranked_A25_diagnostic": 0.7134446870657629, "selected_validation_actual_calls": 0.45595614589921996, "selected_test_actual_calls": 0.6200153135014037}}}, {"key": ["nlvr2", "nlvr2", "MiniCPM"], "areas": {"confidence": {"validation_fixed_A25": 0.8284302492122602, "test_fixed_A25": 0.8339025615268711, "test_ranked_A25_diagnostic": 0.835882901628758, "selected_validation_actual_calls": 0.8587797192781438, "selected_test_actual_calls": 0.8580038745784602}, "learned": {"validation_fixed_A25": 0.8353480378115155, "test_fixed_A25": 0.8422214249838561, "test_ranked_A25_diagnostic": 0.8411638085671236, "selected_validation_actual_calls": 0.9139215124606132, "selected_test_actual_calls": 0.9110425486116093}}}, {"key": ["nlvr2", "nlvr2", "Qwen3.8"], "areas": {"confidence": {"validation_fixed_A25": 0.8312804354053279, "test_fixed_A25": 0.8343976465523427, "test_ranked_A25_diagnostic": 0.8361340317141422, "selected_validation_actual_calls": 0.8587797192781438, "selected_test_actual_calls": 0.8580038745784602}, "learned": {"validation_fixed_A25": 0.8401432254368376, "test_fixed_A25": 0.8425816172777499, "test_ranked_A25_diagnostic": 0.8422458204778647, "selected_validation_actual_calls": 0.8419077628186766, "selected_test_actual_calls": 0.8434096290449882}}}, {"key": ["nlvr2", "vilt", "MiniCPM"], "areas": {"confidence": {"validation_fixed_A25": 0.7926095674591808, "test_fixed_A25": 0.7987371744277822, "test_ranked_A25_diagnostic": 0.8005237856066586, "selected_validation_actual_calls": 0.9598968776854769, "selected_test_actual_calls": 0.9593169261677549}, "learned": {"validation_fixed_A25": 0.7940532798625036, "test_fixed_A25": 0.8014909951926528, "test_ranked_A25_diagnostic": 0.8024137188778073, "selected_validation_actual_calls": 0.9819535949584648, "selected_test_actual_calls": 0.9805984071177442}}}, {"key": ["nlvr2", "vilt", "Qwen3.8"], "areas": {"confidence": {"validation_fixed_A25": 0.7940561443712404, "test_fixed_A25": 0.7988735021884195, "test_ranked_A25_diagnostic": 0.8006242376408124, "selected_validation_actual_calls": 0.9598968776854769, "selected_test_actual_calls": 0.9593169261677549}, "learned": {"validation_fixed_A25": 0.7962102549412775, "test_fixed_A25": 0.8036521489560164, "test_ranked_A25_diagnostic": 0.8044127143574658, "selected_validation_actual_calls": 0.9959896877685477, "selected_test_actual_calls": 0.9961684724115664}}}, {"key": ["construction", "yolo26x", "MiniCPM"], "areas": {"confidence": {"validation_fixed_A25": 0.1745164052515915, "test_fixed_A25": 0.17697501725814355, "test_ranked_A25_diagnostic": 0.1716572124279524, "selected_validation_actual_calls": 0.9386590584878745, "selected_test_actual_calls": 0.9254327563249002}, "learned": {"validation_fixed_A25": 0.2280241031947047, "test_fixed_A25": 0.21383535478219448, "test_ranked_A25_diagnostic": 0.21318248319590255, "selected_validation_actual_calls": 0.4573466476462197, "selected_test_actual_calls": 0.4531957390146471}}}, {"key": ["construction", "yolo26x", "Qwen3.8"], "areas": {"confidence": {"validation_fixed_A25": 0.17942553980644182, "test_fixed_A25": 0.18055274894913717, "test_ranked_A25_diagnostic": 0.1793307821359281, "selected_validation_actual_calls": 0.9286733238231099, "selected_test_actual_calls": 0.9221038615179761}, "learned": {"validation_fixed_A25": 0.2325252201898847, "test_fixed_A25": 0.21172533739102792, "test_ranked_A25_diagnostic": 0.21128777332230012, "selected_validation_actual_calls": 0.45135520684736086, "selected_test_actual_calls": 0.44766977363515315}}}, {"key": ["construction", "rtdetr_x_fp32", "MiniCPM"], "areas": {"confidence": {"validation_fixed_A25": 0.08994758708890332, "test_fixed_A25": 0.08325267500178912, "test_ranked_A25_diagnostic": 0.08357421552875581, "selected_validation_actual_calls": 1.0, "selected_test_actual_calls": 1.0}, "learned": {"validation_fixed_A25": 0.10056264714432415, "test_fixed_A25": 0.10905069389769234, "test_ranked_A25_diagnostic": 0.1142703643498825, "selected_validation_actual_calls": 0.7728958630527817, "selected_test_actual_calls": 0.7843541944074567}}}, {"key": ["construction", "rtdetr_x_fp32", "Qwen3.8"], "areas": {"confidence": {"validation_fixed_A25": 0.10053453664183931, "test_fixed_A25": 0.08910633276678805, "test_ranked_A25_diagnostic": 0.08907503728424503, "selected_validation_actual_calls": 1.0, "selected_test_actual_calls": 1.0}, "learned": {"validation_fixed_A25": 0.11153717985565076, "test_fixed_A25": 0.11890339192252425, "test_ranked_A25_diagnostic": 0.12154998969679998, "selected_validation_actual_calls": 0.8308131241084166, "selected_test_actual_calls": 0.8356857523302264}}}] \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/comparisons.json b/releases/20260913_lr/paper_data/comparisons.json new file mode 100644 index 0000000000000000000000000000000000000000..252c5a252c8914910bb85432e19920c97ef18024 --- /dev/null +++ b/releases/20260913_lr/paper_data/comparisons.json @@ -0,0 +1,11558 @@ +[ + { + "baseline": "original", + "candidate": "score_gain", + "counts": { + "total": 48, + "performance_wins": 31, + "performance_ties": 0, + "performance_losses": 17, + "call_less": 21, + "call_ties": 0, + "call_more": 27, + "both_performance_win_and_call_less": 19, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.23610378506203042, + "performance_delta": 0.00636626667272841, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7952729693741678, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23330554571941767, + "performance_delta": 0.0008511922008775652, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7902796271637816, + "call_delta": 0.009320905459387463 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22951409007632145, + "performance_delta": -0.003254425173411568, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.8009320905459387, + "call_delta": 0.019640479360852225 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2687941772737477, + "performance_delta": 0.0004707717644751308, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8062583222370173, + "call_delta": -0.00998668442077233 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2699197889196931, + "performance_delta": 0.003992751149164753, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.831225033288948, + "call_delta": -0.007989347536617841 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2687146609089476, + "performance_delta": 0.00037410281989919136, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8225699067909454, + "call_delta": -0.006990679094540542 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2668710925054341, + "performance_delta": -0.0004626043850421979, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.39580559254327563, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.25610142784250745, + "performance_delta": -0.002510607023462952, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.46804260985352863, + "call_delta": 0.10852197070572572 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2553143923177106, + "performance_delta": -0.010255108607445773, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5402796271637816, + "call_delta": 0.08588548601864182 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3034889605344151, + "performance_delta": 0.005719274933867702, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.46937416777629826, + "call_delta": -0.021637816245006658 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2998980207837558, + "performance_delta": 0.000508977878148853, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.4284287616511318, + "call_delta": -0.025965379494008012 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3033337683996216, + "performance_delta": 0.0003620662089585114, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.45272969374167776, + "call_delta": -0.001997336884154488 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0024162927166033743, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.004832585433206766, + "call_delta": -0.0372799447704522 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8134276838108389, + "performance_delta": 0.005522954780807776, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.010700724887814981, + "call_delta": -0.04470141525716258 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.81429064549534, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.0075940628236106315, + "call_delta": -0.03676216775975147 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0008629616845011734, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.0075940628236106315, + "call_delta": -0.010182947877114256 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0012081463583016872, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.0069036934760096655, + "call_delta": -0.006213324128408699 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8151536071798412, + "performance_delta": 0.0008629616845011734, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.007766655160510874, + "call_delta": -0.0006903693476009659 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8998964445978599, + "performance_delta": 0.0006903693476010275, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.027614773904038662, + "call_delta": 0.001208146358301694 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8995512599240594, + "performance_delta": 0.0010355540214014303, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.016396272005522954, + "call_delta": -0.007594062823610633 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8988608905764585, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.015533310321021747, + "call_delta": -0.00655850880220918 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0020711080428028605, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.00845702450811184, + "call_delta": -0.010873317224715224 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.009492578529513289, + "call_delta": -0.00742147048671039 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.9011045909561616, + "performance_delta": 0.0008629616845012844, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.011045909561615464, + "call_delta": -0.004487400759406283 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7514108272126592, + "performance_delta": 0.002070158523097887, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.8495023112043786, + "call_delta": -0.00944332586563812 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.752006352267249, + "performance_delta": 0.039389728610725205, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8824263392224144, + "call_delta": 0.19391430110881092 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7502197771034795, + "performance_delta": 0.09576610044522582, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9653177551541275, + "call_delta": 0.4029435952698296 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7459376683776197, + "performance_delta": -0.012619459490117113, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.978617814706633, + "call_delta": 0.4110824376825568 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.968380455434875, + "call_delta": 0.35853444119899036 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7482063352522474, + "performance_delta": -0.012194084451124354, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.96633865524771, + "call_delta": 0.36423446672149273 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9067231111430006, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9916050800028701, + "call_delta": 0.06845088613044414 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.0010762717945038691, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9057185908014638, + "performance_delta": -0.0018655377771399362, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9883045131663917, + "call_delta": 0.12004018081366152 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9060055966133315, + "performance_delta": -0.001004520341536863, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9787615699217909, + "call_delta": 0.12850685226375824 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9049293248188276, + "performance_delta": 0.0011480232474707641, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9672813374470833, + "call_delta": 0.10303508646050086 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9067948625959675, + "performance_delta": -0.00035875726483469705, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9913180741910024, + "call_delta": 0.0025113008538423243 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.906364353878166, + "performance_delta": -0.0006457630767022771, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9926813517973739, + "call_delta": 0.002009040683073837 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.989811293678697, + "call_delta": 0.0025830523068092193 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9039965559302576, + "performance_delta": 0.004018081366147674, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9788333213747579, + "call_delta": 0.017148597259094567 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9023462725120184, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9800530960751955, + "call_delta": 0.010619215039104568 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9036377986654229, + "performance_delta": 0.0033005668364783913, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9882327617134247, + "call_delta": 0.017722608882829838 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "checkpoint_perf", + "counts": { + "total": 48, + "performance_wins": 12, + "performance_ties": 10, + "performance_losses": 26, + "call_less": 20, + "call_ties": 8, + "call_more": 20, + "both_performance_win_and_call_less": 7, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2305421865004371, + "performance_delta": 0.0008046681111350817, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7816245006657789, + "call_delta": -0.00898801597869514 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.22838091389772372, + "performance_delta": -0.0040734396208163826, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.8049267643142477, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.23035654564059277, + "performance_delta": -0.002411969609140252, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.774300932090546, + "call_delta": -0.006990679094540542 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.26792218076116586, + "performance_delta": -0.00040122474810672104, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8215712383488681, + "call_delta": 0.005326231691078487 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.26876666264349774, + "performance_delta": 0.002839624872969404, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8125832223701731, + "call_delta": -0.026631158455392767 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26833406705021806, + "performance_delta": -6.491038830325202e-06, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8229027962716379, + "call_delta": -0.0066577896138481085 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2624005680779394, + "performance_delta": -0.0049331288125369355, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.37350199733688416, + "call_delta": 0.0016644474034620549 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.25991507502469235, + "performance_delta": 0.0013030401587219576, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.37283621837549935, + "call_delta": 0.013315579227696439 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2613858995371947, + "performance_delta": -0.004183601387961644, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.3808255659121172, + "call_delta": -0.07356857523302263 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29875083466112995, + "performance_delta": 0.0009811490605825535, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.43275632490013316, + "call_delta": -0.058255659121171754 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29645395142312325, + "performance_delta": -0.0029350914824837204, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.524300932090546, + "call_delta": 0.06990679094540614 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3014472023134628, + "performance_delta": -0.0015244998772003249, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4490679094540613, + "call_delta": -0.0056591211717709755 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8129099068001381, + "performance_delta": 0.0010355540214014303, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.05074214704867104, + "call_delta": 0.00862961684501208 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8130824991370383, + "performance_delta": 0.005177770107007262, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.044183638246461855, + "call_delta": -0.011218501898515705 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.8151536071798412, + "performance_delta": 0.0, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.0443562305833621, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8123921297894373, + "performance_delta": -0.0010355540214015413, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.020711080428028994, + "call_delta": 0.002934069727304107 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8127373144632378, + "performance_delta": -0.00034518467380051376, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.02347255781843286, + "call_delta": 0.010355540214014497 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8113565757680359, + "performance_delta": -0.002934069727304145, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.03641698308595098, + "call_delta": 0.027959958577839145 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8967897825336555, + "performance_delta": -0.0024162927166033743, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.038488091128753885, + "call_delta": 0.012081463583016917 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8979979288919572, + "performance_delta": -0.0005177770107007706, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.0277873662409389, + "call_delta": 0.003797031411805315 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8966171901967552, + "performance_delta": -0.003106662064204402, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.03141180531584398, + "call_delta": 0.00931998619261305 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.9009319986192613, + "performance_delta": 0.001380738695201944, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.018985157059026578, + "call_delta": -0.000345184673800486 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.9005868139454608, + "performance_delta": -0.00034518467380051376, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.015015533310321022, + "call_delta": -0.0018985157059026574 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8992060752502589, + "performance_delta": -0.0010355540214014303, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.015360717984121504, + "call_delta": -0.000172592336900243 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.445622890848765, + "performance_delta": -0.010974676006011985, + "baseline_call": 0.407140628987891, + "candidate_call": 0.4267929557893543, + "call_delta": 0.019652326801463282 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4702379264384766, + "performance_delta": 0.007061225647279068, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.47905736891359213, + "call_delta": 0.023452343816464827 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.44219153220089047, + "performance_delta": -0.004140317046195718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.4057227121912486, + "call_delta": 0.0015880668122394415 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.6876612880356181, + "performance_delta": -0.06167938065394318, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.6266057907721975, + "call_delta": -0.23233984629781923 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.6622805773757195, + "performance_delta": -0.050336046280804236, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.5790488614128123, + "call_delta": -0.1094631767007912 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.6775940787794572, + "performance_delta": 0.023140402121203496, + "baseline_call": 0.562374159884298, + "candidate_call": 0.6149505147037971, + "call_delta": 0.05257635481949918 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6657119360235941, + "performance_delta": -0.008904517482914098, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.322916371267334, + "call_delta": 0.09006607492272353 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6712134531945665, + "performance_delta": 0.0, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.27691915038425546, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6739925701159856, + "performance_delta": 0.0, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.2715877832288801, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7582735445084082, + "performance_delta": -0.00028358335932854306, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.537475540935258, + "call_delta": -0.030059836088818237 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7583586195162068, + "performance_delta": 0.0, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.6098460142358847, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7604004197033718, + "performance_delta": 0.0, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6021041885262173, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9080863887493722, + "performance_delta": 0.0, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9231541938724259, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9079428858434383, + "performance_delta": -7.175145296700602e-05, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9026332783238861, + "call_delta": -0.0317141422113798 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.907297122766736, + "performance_delta": -0.00028700581186769103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8648202626103179, + "call_delta": -0.0034440697424122924 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9072253713137691, + "performance_delta": -0.0009327688885699681, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.8425773121905719, + "call_delta": 0.01528305948195452 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9070101169548683, + "performance_delta": 0.0, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.8502547176580326, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9037813015713568, + "performance_delta": 0.0, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8642462509865825, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9071536198608022, + "performance_delta": 0.0, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9778288010332209, + "call_delta": -0.010977972303939154 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9070818684078352, + "performance_delta": 7.1751452966895e-05, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9764655234268494, + "call_delta": -0.01420678768745065 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9047858219128937, + "performance_delta": -0.0023677979479085343, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9235129511372605, + "call_delta": -0.0637152902346273 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9013417521704814, + "performance_delta": 0.0013632776063714491, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9797660902633278, + "call_delta": 0.018081366147664535 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9002654803759776, + "performance_delta": 0.00021525435890079603, + "baseline_call": 0.969433881036091, + "candidate_call": 0.966563822917414, + "call_delta": -0.0028700581186769103 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9003372318289445, + "performance_delta": 0.0, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9903853053024324, + "call_delta": 0.019875152471837576 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "both", + "counts": { + "total": 48, + "performance_wins": 26, + "performance_ties": 2, + "performance_losses": 20, + "call_less": 24, + "call_ties": 1, + "call_more": 23, + "both_performance_win_and_call_less": 16, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22850906574549645, + "performance_delta": -0.0012284526438055554, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7906125166444741, + "call_delta": 0.0 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23004315338506715, + "performance_delta": -0.002411200133472957, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7926098535286284, + "call_delta": 0.011651131824234273 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2308450652014655, + "performance_delta": -0.001923450048267522, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7766311584553928, + "call_delta": -0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2672954581839896, + "performance_delta": 0.0013684204134612465, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8215712383488681, + "call_delta": -0.017643142476697737 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.267918819975632, + "performance_delta": -0.0004217381134163922, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8149134487350199, + "call_delta": -0.01464713715046606 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2675187098549581, + "performance_delta": 0.00018501296448181215, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.4144474034620506, + "call_delta": 0.042609853528628505 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.26086905351963807, + "performance_delta": -0.004700447405518282, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.39980026631158455, + "call_delta": -0.05459387483355527 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3029107097560361, + "performance_delta": 0.005141024155488727, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4337549933422104, + "call_delta": -0.05725699067909451 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29937410202312603, + "performance_delta": -1.4940882480940498e-05, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.44274300932090543, + "call_delta": -0.011651131824234384 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3009613147999525, + "performance_delta": -0.002010387390710633, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4477363515312916, + "call_delta": -0.006990679094540653 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8125647221263376, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.017604418363824648, + "call_delta": -0.024508111839834312 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8134276838108389, + "performance_delta": 0.005522954780807776, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.010700724887814981, + "call_delta": -0.04470141525716258 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.81429064549534, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.006558508802209182, + "call_delta": -0.03779772178115291 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0008629616845011734, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.007766655160510874, + "call_delta": -0.010010355540214014 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0012081463583016872, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.005695547117707974, + "call_delta": -0.00742147048671039 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.00638591646530894, + "call_delta": -0.0020711080428028996 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8998964445978599, + "performance_delta": 0.0006903693476010275, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.027614773904038662, + "call_delta": 0.001208146358301694 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.901449775629962, + "performance_delta": 0.002934069727304034, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.010873317224715223, + "call_delta": -0.013117017604418364 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.901449775629962, + "performance_delta": 0.0017259233690023468, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.010355540214014497, + "call_delta": -0.01173627890921643 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0020711080428028605, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.01156368657231619, + "call_delta": -0.0077666551605108745 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.006731101139109423, + "call_delta": -0.010182947877114256 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.001380738695201944, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.007076285812909907, + "call_delta": -0.008457024508111841 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7511272438533307, + "performance_delta": 0.001786575163769455, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.852763519836656, + "call_delta": -0.006182117233360707 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7511839605251964, + "performance_delta": 0.09673028386694271, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8652411876471089, + "call_delta": 0.30286702776281094 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9705640473017043, + "call_delta": 0.3607180330658196 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7550406942120637, + "performance_delta": -0.005359725491308143, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9094518333664181, + "call_delta": 0.30734764484020083 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.002009040683073837, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9890937791490277, + "call_delta": 0.06593958527660182 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0015785319652723562, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.98973954222573, + "call_delta": 0.12147520987299987 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9052163306306953, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9674248403530171, + "call_delta": 0.1171701226949845 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9075123771256368, + "performance_delta": 0.0037310755542799834, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9586711630910526, + "call_delta": 0.09442491210447013 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9071536198608022, + "performance_delta": 0.0, + "baseline_call": 0.98880677333716, + "candidate_call": 0.99253784889144, + "call_delta": 0.0037310755542799834 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.00035875726483458603, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9929683576092416, + "call_delta": 0.0022960464949415282 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.989811293678697, + "call_delta": 0.0025830523068092193 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9034225443065222, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9885197675252924, + "call_delta": 0.026835043409629056 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9037095501183899, + "performance_delta": 0.0036593241013130884, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9783310612039894, + "call_delta": 0.008897180167898422 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9026332783238861, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9682141063356533, + "call_delta": -0.0022960464949415282 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "score_only_gain", + "counts": { + "total": 48, + "performance_wins": 20, + "performance_ties": 1, + "performance_losses": 27, + "call_less": 17, + "call_ties": 0, + "call_more": 31, + "both_performance_win_and_call_less": 10, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22725460966786967, + "performance_delta": -0.0024829087214323364, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.8768308921438083, + "call_delta": 0.0862183754993342 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23264549293538322, + "performance_delta": 0.0001911394168431224, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7882822902796272, + "call_delta": 0.007323568575232975 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.229420185942548, + "performance_delta": -0.0033483293071850107, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7959387483355526, + "call_delta": 0.01464713715046606 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.270279514537055, + "performance_delta": 0.0043524767665266695, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.801930758988016, + "call_delta": -0.03728362183754985 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26989568494530464, + "performance_delta": 0.0015551268562562504, + "baseline_call": 0.829560585885486, + "candidate_call": 0.800266311584554, + "call_delta": -0.02929427430093201 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2668710925054341, + "performance_delta": -0.0004626043850421979, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.39580559254327563, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2565642588848189, + "performance_delta": -0.00900524204033748, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.46571238348868177, + "call_delta": 0.011318242343541951 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29895985090664157, + "performance_delta": 0.0011901653060941708, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4497336884154461, + "call_delta": -0.04127829560585883 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2991984700035601, + "performance_delta": -0.0001905729020468816, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.453728362183755, + "call_delta": -0.0006657789613848109 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3033337683996216, + "performance_delta": 0.0003620662089585114, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.45272969374167776, + "call_delta": -0.001997336884154488 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8125647221263376, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.026406627545736968, + "call_delta": -0.015705902657921992 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8111839834311356, + "performance_delta": 0.0032792544011045477, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.02347255781843286, + "call_delta": -0.0319295823265447 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.81429064549534, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.006558508802209182, + "call_delta": -0.03779772178115291 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8134276838108389, + "performance_delta": 0.0, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.01570590265792199, + "call_delta": -0.0020711080428028987 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8118743527787367, + "performance_delta": -0.0012081463583016872, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.0298584742837418, + "call_delta": 0.016741456679323436 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8127373144632378, + "performance_delta": -0.001553331032102201, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.032274767000345185, + "call_delta": 0.023817742492233347 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.9005868139454608, + "performance_delta": 0.001380738695201944, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.021574042112530202, + "call_delta": -0.004832585433206766 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8973075595443563, + "performance_delta": -0.0012081463583016872, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.02347255781843286, + "call_delta": -0.0005177770107007255 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8992060752502589, + "performance_delta": -0.0005177770107007706, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.020711080428028994, + "call_delta": -0.0013807386952019336 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.8992060752502589, + "performance_delta": -0.00034518467380051376, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.022091819123230928, + "call_delta": 0.0027614773904038638 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.8990334829133586, + "performance_delta": -0.0018985157059027147, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.02002071108042803, + "call_delta": 0.00310666206420435 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8988608905764585, + "performance_delta": -0.001380738695201833, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.02105626510182948, + "call_delta": 0.005522954780807733 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.511414230212971, + "performance_delta": 0.05481666335819407, + "baseline_call": 0.407140628987891, + "candidate_call": 0.7920199642684967, + "call_delta": 0.38487933528060575 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.503984346198565, + "performance_delta": 0.04080764540736748, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.597339988089499, + "call_delta": 0.14173496299237165 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5101097467600602, + "performance_delta": 0.06377789751297397, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.7076255565323427, + "call_delta": 0.30349091115333354 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7498794770722854, + "performance_delta": 0.0005388083827241319, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.846070952556504, + "call_delta": -0.012874684513512702 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7510138105095993, + "performance_delta": 0.09656013385134554, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8730964467005076, + "call_delta": 0.3107222868162096 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6809403624195333, + "performance_delta": 0.006323908913025034, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.1506961971471514, + "call_delta": -0.08215409919745909 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6708164364915067, + "performance_delta": -0.00039701670305980485, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.00870600913138417, + "call_delta": -0.26821314125287127 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6797493123103536, + "performance_delta": 0.0057567421943680586, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.11924680259762357, + "call_delta": -0.15234098063125653 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7474406601820606, + "performance_delta": -0.010917959334146188, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9800073731673425, + "call_delta": 0.37016135893145785 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7482063352522474, + "performance_delta": -0.012194084451124354, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.96633865524771, + "call_delta": 0.36423446672149273 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9068666140489345, + "performance_delta": -0.0012197747004376591, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9918203343617709, + "call_delta": 0.06866614048934494 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9055750878955299, + "performance_delta": -0.0024395494008754293, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9716581760780656, + "call_delta": 0.03731075554279972 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9057185908014638, + "performance_delta": -0.0018655377771399362, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9883045131663917, + "call_delta": 0.12004018081366152 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9060055966133315, + "performance_delta": -0.001004520341536863, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9787615699217909, + "call_delta": 0.12850685226375824 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9060773480662984, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9367152184831743, + "call_delta": 0.07246896749659182 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.0010762717945038691, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9899547965846308, + "call_delta": 0.0011480232474707641 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.906364353878166, + "performance_delta": -0.0006457630767022771, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9926813517973739, + "call_delta": 0.002009040683073837 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9059338451603645, + "performance_delta": -0.0012197747004377701, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9891655306019946, + "call_delta": 0.0019372892301068312 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9033507928535552, + "performance_delta": 0.0033723182894452863, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9880892588074909, + "call_delta": 0.026404534691827575 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9023462725120184, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9800530960751955, + "call_delta": 0.010619215039104568 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9033507928535552, + "performance_delta": 0.0030135610246107003, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.987012987012987, + "call_delta": 0.01650283418239218 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.25", + "counts": { + "total": 48, + "performance_wins": 17, + "performance_ties": 0, + "performance_losses": 31, + "call_less": 5, + "call_ties": 0, + "call_more": 43, + "both_performance_win_and_call_less": 1, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2211131170053352, + "performance_delta": -0.008624401383966807, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.988681757656458, + "call_delta": 0.19806924101198398 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.2222578228236749, + "performance_delta": -0.010196530694865202, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.9880159786950732, + "call_delta": 0.207057256990679 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22347202518399215, + "performance_delta": -0.009296490065740876, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.988681757656458, + "call_delta": 0.20739014647137155 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2707502617838614, + "performance_delta": 0.002426856274588829, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.9880159786950732, + "call_delta": 0.17177097203728353 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.27188089881706984, + "performance_delta": 0.005953861046541498, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.9883488681757656, + "call_delta": 0.14913448735019974 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27098320199585757, + "performance_delta": 0.0026426439068091834, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9873501997336884, + "call_delta": 0.15778961384820245 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2587284729338324, + "performance_delta": -0.008605223956643926, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.5702396804260985, + "call_delta": 0.1984021304926764 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2591386172712909, + "performance_delta": 0.000526582405320497, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.5509320905459387, + "call_delta": 0.1914114513981358 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.25650251356280634, + "performance_delta": -0.00906698736235001, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5622503328894807, + "call_delta": 0.10785619174434086 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.2712680480298825, + "performance_delta": -0.02650163757066487, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.6844207723035952, + "call_delta": 0.1934087882822903 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.27348865863056027, + "performance_delta": -0.025900384275046706, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.6474700399467377, + "call_delta": 0.19307589880159787 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2704420670349152, + "performance_delta": -0.032529635155747916, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.5572569906790945, + "call_delta": 0.10252996005326226 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.804280289955126, + "performance_delta": -0.007594062823610637, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.10631687953054884, + "call_delta": 0.06420434932688988 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8034173282706247, + "performance_delta": -0.004487400759406346, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.1102865032792544, + "call_delta": 0.054884363134276845 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.8030721435968243, + "performance_delta": -0.012081463583016872, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.11408353469105972, + "call_delta": 0.06972730410769762 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8203313772868485, + "performance_delta": 0.006903693476009609, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.10976872626855368, + "call_delta": 0.09199171556782879 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8198136002761477, + "performance_delta": 0.006731101139109352, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.10631687953054884, + "call_delta": 0.09319986192613047 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8194684156023473, + "performance_delta": 0.005177770107007262, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.10976872626855368, + "call_delta": 0.10131170176044184 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8779772178115292, + "performance_delta": -0.02122885743872971, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.09009319986192613, + "call_delta": 0.06368657231618916 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8765964791163272, + "performance_delta": -0.021919226786330737, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.09199171556782879, + "call_delta": 0.0680013807386952 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8781498101484294, + "performance_delta": -0.021574042112530223, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.08940283051432517, + "call_delta": 0.06731101139109424 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.8893683120469451, + "performance_delta": -0.010182947877114268, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.08940283051432517, + "call_delta": 0.0700724887814981 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.8881601656886434, + "performance_delta": -0.012771832930617899, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.09354504659993096, + "call_delta": 0.07663099758370728 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8885053503624439, + "performance_delta": -0.011736278909216358, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.09112875388332758, + "call_delta": 0.07559544356230584 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.4992201457618467, + "performance_delta": 0.04262257890706972, + "baseline_call": 0.407140628987891, + "candidate_call": 0.9824178317216345, + "call_delta": 0.5752772027337436 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4999858208320336, + "performance_delta": 0.03680912004083603, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.9854238153305164, + "call_delta": 0.5298187902333891 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4995604457930409, + "performance_delta": 0.05322859654595469, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.984941723619658, + "call_delta": 0.5808070782406489 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7393018177693333, + "performance_delta": -0.010038850920227937, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.9836088818308142, + "call_delta": 0.12466324476079749 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7390182344100048, + "performance_delta": 0.02640161075348102, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.9834954484870828, + "call_delta": 0.2949834103734793 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7399540594957887, + "performance_delta": 0.08550038283753503, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9844879902447324, + "call_delta": 0.4221138303604345 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6695403113745285, + "performance_delta": -0.005076142131979711, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.019595610129597596, + "call_delta": -0.2132546862150129 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6694835947026628, + "performance_delta": -0.0017298584919036575, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.05478830502226129, + "call_delta": -0.22213084536199418 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6695403113745285, + "performance_delta": -0.004452258741457049, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.017383659926835493, + "call_delta": -0.2542041233020446 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7470436434790007, + "performance_delta": -0.011513484388736117, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9068712247965289, + "call_delta": 0.33933584777245274 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7490287269943, + "performance_delta": -0.009329892521906746, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9408445112440802, + "call_delta": 0.3309984970081955 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7492272353458299, + "performance_delta": -0.011173184357541888, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.938831069392848, + "call_delta": 0.3367268808666307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.0009327688885699681, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9870847384659539, + "call_delta": 0.06393054459352798 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9050728277247614, + "performance_delta": -0.0029418095716439163, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9550835904427064, + "call_delta": 0.020736169907440538 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0012915261534045541, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9650570424051087, + "call_delta": 0.09679271005237855 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9090191576379422, + "performance_delta": 0.0008610174356030731, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9368587213891081, + "call_delta": 0.10956446868049075 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9069383655019014, + "performance_delta": -7.1751452966895e-05, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9668508287292817, + "call_delta": 0.11659611107124912 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9096649207146444, + "performance_delta": 0.005883619143287611, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8986151969577384, + "call_delta": 0.03436894597115592 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9070101169548683, + "performance_delta": -0.00014350290593390103, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9903135538494654, + "call_delta": 0.0015067805123053502 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9068666140489345, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9901700509435316, + "call_delta": -0.0005022601707684871 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9904570567553993, + "call_delta": 0.0032288153835114963 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9027767812298199, + "performance_delta": 0.0027983066657099043, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9891655306019946, + "call_delta": 0.027480806486331333 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9030637870416875, + "performance_delta": 0.0030135610246107003, + "baseline_call": 0.969433881036091, + "candidate_call": 0.988017507354524, + "call_delta": 0.018583626318433022 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9031355384946546, + "performance_delta": 0.0027983066657100153, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9648417880462079, + "call_delta": -0.005668364784386926 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.5", + "counts": { + "total": 48, + "performance_wins": 16, + "performance_ties": 1, + "performance_losses": 31, + "call_less": 4, + "call_ties": 0, + "call_more": 44, + "both_performance_win_and_call_less": 1, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22427095268610697, + "performance_delta": -0.0054665657031950365, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.9853528628495339, + "call_delta": 0.19474034620505987 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.22607173521804688, + "performance_delta": -0.00638261830049322, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.9833555259653795, + "call_delta": 0.20239680426098527 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2255853947574774, + "performance_delta": -0.0071831204922556124, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9810252996005326, + "call_delta": 0.19973368841544614 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.27125656976472395, + "performance_delta": 0.0029331642554513704, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.9856857523302264, + "call_delta": 0.16944074567243672 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.27047490574776584, + "performance_delta": 0.004547867977237496, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.9846870838881492, + "call_delta": 0.1454727030625833 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2711912470021881, + "performance_delta": 0.0028506889131397206, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9866844207723036, + "call_delta": 0.15712383488681758 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.26009300196494534, + "performance_delta": -0.007240694925530977, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.5699067909454061, + "call_delta": 0.19806924101198398 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2610246960878656, + "performance_delta": 0.002412661221895207, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.5223035952063915, + "call_delta": 0.16278295605858856 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2584417288296328, + "performance_delta": -0.007127772095523566, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5529294274300932, + "call_delta": 0.0985352862849534 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.2741835645844183, + "performance_delta": -0.023586121016129113, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.6231691078561917, + "call_delta": 0.1321571238348868 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2747498073529785, + "performance_delta": -0.02463923555262848, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.6195073235685752, + "call_delta": 0.16511318242343537 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2746117403663106, + "performance_delta": -0.02835996182435252, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.6088548601864181, + "call_delta": 0.15412782956058585 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8068691750086296, + "performance_delta": -0.0050051777701070055, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.08715913013462202, + "call_delta": 0.04504659993096306 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8068691750086296, + "performance_delta": -0.0010355540214014303, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.09043838453572661, + "call_delta": 0.03503624439074905 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.8080773213669313, + "performance_delta": -0.007076285812909866, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.09354504659993096, + "call_delta": 0.04918881601656886 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8225750776665516, + "performance_delta": 0.009147393855712727, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.09941318605453918, + "call_delta": 0.08163617535381429 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8210217466344494, + "performance_delta": 0.00793924749741104, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.10027614773904038, + "call_delta": 0.08715913013462201 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8217121159820504, + "performance_delta": 0.00742147048671038, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.0949257852951329, + "call_delta": 0.08646876078702107 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8843631342768381, + "performance_delta": -0.01484294097342076, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.08025543665861236, + "call_delta": 0.05384880911287539 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8803935105281325, + "performance_delta": -0.01812219537452542, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.08940283051432517, + "call_delta": 0.06541249568519159 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8793579565067311, + "performance_delta": -0.020365895754228536, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.09199171556782879, + "call_delta": 0.06989989644459786 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.8936831204694512, + "performance_delta": -0.005868139454608179, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.08232654470141526, + "call_delta": 0.0629962029685882 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.8929927511218502, + "performance_delta": -0.00793924749741115, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.08111839834311356, + "call_delta": 0.06420434932688988 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8933379357956507, + "performance_delta": -0.006903693476009609, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.08232654470141526, + "call_delta": 0.06679323438039352 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.5032753878002438, + "performance_delta": 0.04667782094546685, + "baseline_call": 0.407140628987891, + "candidate_call": 0.9798088648158126, + "call_delta": 0.5726682358279216 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.5033321044721096, + "performance_delta": 0.04015540368091208, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.9834954484870828, + "call_delta": 0.5278904233899555 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5037574795111023, + "performance_delta": 0.05742563026401609, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.9825879817372316, + "call_delta": 0.5784533363582225 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7408048095737743, + "performance_delta": -0.008535859115787026, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.9809431982531265, + "call_delta": 0.12199756118310978 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7410033179253042, + "performance_delta": 0.028386694268780377, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.9824178317216345, + "call_delta": 0.29390579360803104 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7419391430110881, + "performance_delta": 0.08748546635283438, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9827581317528288, + "call_delta": 0.4203839718685308 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.005104500467912554, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.019680685137396136, + "call_delta": -0.21316961120721437 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6695403113745285, + "performance_delta": -0.001673141820037971, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.019935910160791766, + "call_delta": -0.2569832402234637 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.669625386382327, + "performance_delta": -0.0043671837336585195, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.016929926551909932, + "call_delta": -0.25465785667697016 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7471287184867992, + "performance_delta": -0.011428409380937588, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9701103139267788, + "call_delta": 0.40257493690270263 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7488018603068372, + "performance_delta": -0.009556759209369603, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.959107279584834, + "call_delta": 0.3492612653489493 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7490287269943, + "performance_delta": -0.01137169270907179, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9377818109633327, + "call_delta": 0.3356776224371154 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.0014350290593384551, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.987012987012987, + "call_delta": 0.06385879314056109 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9075123771256368, + "performance_delta": -0.0005022601707684871, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9893807849608954, + "call_delta": 0.05503336442562956 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.0006457630767022771, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9877305015426563, + "call_delta": 0.11946616918992614 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080863887493722, + "performance_delta": -7.1751452966895e-05, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9687163665064218, + "call_delta": 0.14142211379780445 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9065796082370668, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9649852909521418, + "call_delta": 0.11473057329410918 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9091626605438761, + "performance_delta": 0.0053813589725192346, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9574513883906149, + "call_delta": 0.09320513740403247 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9070101169548683, + "performance_delta": -0.00014350290593390103, + "baseline_call": 0.98880677333716, + "candidate_call": 0.990744062567267, + "call_delta": 0.0019372892301069422 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9070101169548683, + "performance_delta": 0.0, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9913898256439693, + "call_delta": 0.0007175145296691721 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9070818684078352, + "performance_delta": -7.175145296700602e-05, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9910310683791347, + "call_delta": 0.0038028270072468784 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9029202841357538, + "performance_delta": 0.0029418095716438053, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9893807849608954, + "call_delta": 0.02769606084523213 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9034942957594891, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9801965989811293, + "call_delta": 0.010762717945038358 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9028485326827869, + "performance_delta": 0.0025113008538423243, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9671378345411494, + "call_delta": -0.0033723182894453974 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.75", + "counts": { + "total": 48, + "performance_wins": 23, + "performance_ties": 0, + "performance_losses": 25, + "call_less": 10, + "call_ties": 0, + "call_more": 38, + "both_performance_win_and_call_less": 7, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2319989174313225, + "performance_delta": 0.0022613990420204955, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7762982689747004, + "call_delta": -0.014314247669773628 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23674301754245702, + "performance_delta": 0.004288664023916922, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7596537949400799, + "call_delta": -0.02130492676431428 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.23714462398254227, + "performance_delta": 0.004376108732809247, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.8578561917443409, + "call_delta": 0.07656458055925441 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2698761268543865, + "performance_delta": 0.0015527213451139166, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8032623169107856, + "call_delta": -0.012982689747004006 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2684151526427204, + "performance_delta": 0.002488114872192082, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8169107856191744, + "call_delta": -0.02230359520639147 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2696999750942859, + "performance_delta": 0.0013594170052375065, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8199067909454061, + "call_delta": -0.009653794940079896 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.26372894434686617, + "performance_delta": -0.0036047525436101457, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.3848202396804261, + "call_delta": 0.012982689747004006 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2621221451028754, + "performance_delta": 0.0035101102369050063, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.3725033288948069, + "call_delta": 0.012982689747004006 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.25716044964994544, + "performance_delta": -0.00840905127521091, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.40346205059920104, + "call_delta": -0.05093209054593878 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29493861915351943, + "performance_delta": -0.0028310664470279634, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.5512649800266312, + "call_delta": 0.06025299600532624 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2927064243318904, + "performance_delta": -0.006682618573716548, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5825565912117177, + "call_delta": 0.12816245006657784 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2929922062010103, + "performance_delta": -0.009979495989652809, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.5808921438082557, + "call_delta": 0.12616511318242346 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8120469451156369, + "performance_delta": 0.00017259233690025688, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.06247842595788747, + "call_delta": 0.020365895754228508 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.810666206420435, + "performance_delta": 0.002761477390403888, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.06644804970659303, + "call_delta": 0.011045909561615469 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.8137728684846393, + "performance_delta": -0.001380738695201944, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.06247842595788747, + "call_delta": 0.01812219537452537 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.8198136002761477, + "performance_delta": 0.0063859164653088385, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.07611322057300655, + "call_delta": 0.058336209872281664 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.8196410079392475, + "performance_delta": 0.006558508802209206, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.08836727649292371, + "call_delta": 0.07525025888850534 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8203313772868485, + "performance_delta": 0.006040731791508436, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.09837763203313774, + "call_delta": 0.0899206075250259 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8945460821539524, + "performance_delta": -0.004659993096306492, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.050914739385571284, + "call_delta": 0.024508111839834316 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.8924749741111495, + "performance_delta": -0.006040731791508436, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.06127027959958578, + "call_delta": 0.0372799447704522 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8988608905764585, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.054366586123576116, + "call_delta": 0.03227476700034519 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.8976527442181567, + "performance_delta": -0.0018985157059027147, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.07611322057300655, + "call_delta": 0.05678287884017949 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.8966171901967552, + "performance_delta": -0.004314808422506089, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.07076285812909906, + "call_delta": 0.05384880911287539 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.897134967207456, + "performance_delta": -0.003106662064204291, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.06989989644459786, + "call_delta": 0.054366586123576116 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48044692737430167, + "performance_delta": 0.023849360519524687, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5296202818818592, + "call_delta": 0.12247965289396817 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4890111448260216, + "performance_delta": 0.025834444034824045, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.550860675495562, + "call_delta": 0.09525565039843464 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4808723024132944, + "performance_delta": 0.034540453166208185, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5036156878314381, + "call_delta": 0.09948104245242889 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7464481184244108, + "performance_delta": -0.002892550265150451, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.9825879817372316, + "call_delta": 0.12364234466721491 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7467033434478065, + "performance_delta": 0.03408671979128275, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.9870402404786887, + "call_delta": 0.29852820236508515 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7468734934634036, + "performance_delta": 0.09241981680514988, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9880611405722712, + "call_delta": 0.4256869806879733 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.671865694921022, + "performance_delta": -0.002750758585486235, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.02064486855911295, + "call_delta": -0.21220542778549753 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6721492782803505, + "performance_delta": 0.0009358250857840478, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.018659785043813628, + "call_delta": -0.25825936534044186 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6705612114681111, + "performance_delta": -0.0034313586478744718, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.016306043161387288, + "call_delta": -0.2552817400674928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7471287184867992, + "performance_delta": -0.011428409380937588, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9700252389189802, + "call_delta": 0.402489861894904 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7486600686271729, + "performance_delta": -0.009698550889033819, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9587386212177069, + "call_delta": 0.34889260698182223 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7493690270254941, + "performance_delta": -0.011031392677877672, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9490117119927403, + "call_delta": 0.346907523466523 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.987945755901557, + "call_delta": 0.06479156202913106 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9885915189782594, + "call_delta": 0.054244098442993494 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9881610102604578, + "call_delta": 0.11989667790772762 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9084451460142068, + "performance_delta": 0.00028700581186769103, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9588146659969864, + "call_delta": 0.13152041328836905 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0007175145296691721, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.966563822917414, + "call_delta": 0.11630910525938143 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9085168974671737, + "performance_delta": 0.0047355958958168465, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9580254000143503, + "call_delta": 0.09377914902776785 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9915333285499032, + "call_delta": 0.0027265552127431203 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9067231111430006, + "performance_delta": -0.00028700581186769103, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9921073401736386, + "call_delta": 0.0014350290593384551 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9068666140489345, + "performance_delta": -0.00028700581186769103, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9906723111143001, + "call_delta": 0.0034440697424122924 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9031355384946546, + "performance_delta": 0.0031570639305446013, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9893090335079285, + "call_delta": 0.027624309392265234 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9039965559302576, + "performance_delta": 0.003946329913180779, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9791920786395925, + "call_delta": 0.009758197603501495 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.902705029776853, + "performance_delta": 0.0023677979479084232, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9555858506134749, + "call_delta": -0.014924302217119934 + } + ], + "rule": "historical" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_1", + "counts": { + "total": 48, + "performance_wins": 26, + "performance_ties": 2, + "performance_losses": 20, + "call_less": 24, + "call_ties": 1, + "call_more": 23, + "both_performance_win_and_call_less": 16, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22850906574549645, + "performance_delta": -0.0012284526438055554, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7906125166444741, + "call_delta": 0.0 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23004315338506715, + "performance_delta": -0.002411200133472957, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7926098535286284, + "call_delta": 0.011651131824234273 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2308450652014655, + "performance_delta": -0.001923450048267522, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7766311584553928, + "call_delta": -0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2672954581839896, + "performance_delta": 0.0013684204134612465, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8215712383488681, + "call_delta": -0.017643142476697737 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.267918819975632, + "performance_delta": -0.0004217381134163922, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8149134487350199, + "call_delta": -0.01464713715046606 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2675187098549581, + "performance_delta": 0.00018501296448181215, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.4144474034620506, + "call_delta": 0.042609853528628505 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.26086905351963807, + "performance_delta": -0.004700447405518282, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.39980026631158455, + "call_delta": -0.05459387483355527 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3029107097560361, + "performance_delta": 0.005141024155488727, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4337549933422104, + "call_delta": -0.05725699067909451 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29937410202312603, + "performance_delta": -1.4940882480940498e-05, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.44274300932090543, + "call_delta": -0.011651131824234384 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3009613147999525, + "performance_delta": -0.002010387390710633, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4477363515312916, + "call_delta": -0.006990679094540653 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8118743527787367, + "candidate_metric": 0.8125647221263376, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.04211253020365896, + "candidate_call": 0.017604418363824648, + "call_delta": -0.024508111839834312 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8079047290300311, + "candidate_metric": 0.8134276838108389, + "performance_delta": 0.005522954780807776, + "baseline_call": 0.05540214014497756, + "candidate_call": 0.010700724887814981, + "call_delta": -0.04470141525716258 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.81429064549534, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.006558508802209182, + "call_delta": -0.03779772178115291 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8134276838108389, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0008629616845011734, + "baseline_call": 0.017777010700724887, + "candidate_call": 0.007766655160510874, + "call_delta": -0.010010355540214014 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.8130824991370383, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0012081463583016872, + "baseline_call": 0.013117017604418364, + "candidate_call": 0.005695547117707974, + "call_delta": -0.00742147048671039 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.00638591646530894, + "call_delta": -0.0020711080428028996 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8992060752502589, + "candidate_metric": 0.8998964445978599, + "performance_delta": 0.0006903693476010275, + "baseline_call": 0.026406627545736968, + "candidate_call": 0.027614773904038662, + "call_delta": 0.001208146358301694 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.898515705902658, + "candidate_metric": 0.901449775629962, + "performance_delta": 0.002934069727304034, + "baseline_call": 0.023990334829133587, + "candidate_call": 0.010873317224715223, + "call_delta": -0.013117017604418364 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.901449775629962, + "performance_delta": 0.0017259233690023468, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.010355540214014497, + "call_delta": -0.01173627890921643 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.8995512599240594, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0020711080428028605, + "baseline_call": 0.019330341732827064, + "candidate_call": 0.01156368657231619, + "call_delta": -0.0077666551605108745 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9009319986192613, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0006903693476009165, + "baseline_call": 0.01691404901622368, + "candidate_call": 0.006731101139109423, + "call_delta": -0.010182947877114256 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.001380738695201944, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.007076285812909907, + "call_delta": -0.008457024508111841 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7511272438533307, + "performance_delta": 0.001786575163769455, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.852763519836656, + "call_delta": -0.006182117233360707 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7511839605251964, + "performance_delta": 0.09673028386694271, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8652411876471089, + "call_delta": 0.30286702776281094 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9705640473017043, + "call_delta": 0.3607180330658196 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7550406942120637, + "performance_delta": -0.005359725491308143, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9094518333664181, + "call_delta": 0.30734764484020083 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.002009040683073837, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9890937791490277, + "call_delta": 0.06593958527660182 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0015785319652723562, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.98973954222573, + "call_delta": 0.12147520987299987 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9052163306306953, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9674248403530171, + "call_delta": 0.1171701226949845 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9075123771256368, + "performance_delta": 0.0037310755542799834, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9586711630910526, + "call_delta": 0.09442491210447013 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9071536198608022, + "performance_delta": 0.0, + "baseline_call": 0.98880677333716, + "candidate_call": 0.99253784889144, + "call_delta": 0.0037310755542799834 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.00035875726483458603, + "baseline_call": 0.9906723111143001, + "candidate_call": 0.9929683576092416, + "call_delta": 0.0022960464949415282 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.989811293678697, + "call_delta": 0.0025830523068092193 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9034225443065222, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9885197675252924, + "call_delta": 0.026835043409629056 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9000502260170768, + "candidate_metric": 0.9037095501183899, + "performance_delta": 0.0036593241013130884, + "baseline_call": 0.969433881036091, + "candidate_call": 0.9783310612039894, + "call_delta": 0.008897180167898422 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.9026332783238861, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9682141063356533, + "call_delta": -0.0022960464949415282 + } + ], + "rule": "historical" + }, + { + "rule": "historical", + "baseline": "original", + "candidate": "confidence", + "counts": { + "total": 48, + "performance_wins": 12, + "performance_ties": 0, + "performance_losses": 36, + "call_less": 12, + "call_ties": 0, + "call_more": 36, + "both_performance_win_and_call_less": 3, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.799275112185019, + "performance_delta": -0.01587849499482219, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.113565757680359, + "call_delta": 0.0692095270969969 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.799275112185019, + "performance_delta": -0.01587849499482219, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.113565757680359, + "call_delta": 0.0692095270969969 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.8151536071798412, + "candidate_metric": 0.799275112185019, + "performance_delta": -0.01587849499482219, + "baseline_call": 0.0443562305833621, + "candidate_call": 0.113565757680359, + "call_delta": 0.0692095270969969 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8158439765274422, + "performance_delta": 0.001553331032102201, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.113565757680359, + "call_delta": 0.10510873317224716 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8158439765274422, + "performance_delta": 0.001553331032102201, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.113565757680359, + "call_delta": 0.10510873317224716 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8158439765274422, + "performance_delta": 0.001553331032102201, + "baseline_call": 0.00845702450811184, + "candidate_call": 0.113565757680359, + "call_delta": 0.10510873317224716 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8769416637901277, + "performance_delta": -0.02278218847083191, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.08698653779772178, + "call_delta": 0.06489471867449086 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8769416637901277, + "performance_delta": -0.02278218847083191, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.08698653779772178, + "call_delta": 0.06489471867449086 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.8997238522609596, + "candidate_metric": 0.8769416637901277, + "performance_delta": -0.02278218847083191, + "baseline_call": 0.022091819123230928, + "candidate_call": 0.08698653779772178, + "call_delta": 0.06489471867449086 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8878149810148429, + "performance_delta": -0.012426648256817385, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.08698653779772178, + "call_delta": 0.07145322747670003 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8878149810148429, + "performance_delta": -0.012426648256817385, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.08698653779772178, + "call_delta": 0.07145322747670003 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9002416292716603, + "candidate_metric": 0.8878149810148429, + "performance_delta": -0.012426648256817385, + "baseline_call": 0.015533310321021747, + "candidate_call": 0.08698653779772178, + "call_delta": 0.07145322747670003 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4968380455434875, + "performance_delta": 0.050506196296401296, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.9873805405098829, + "call_delta": 0.5832458951308737 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4968380455434875, + "performance_delta": 0.050506196296401296, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.9873805405098829, + "call_delta": 0.5832458951308737 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4968380455434875, + "performance_delta": 0.050506196296401296, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.9873805405098829, + "call_delta": 0.5832458951308737 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7387062927147435, + "performance_delta": 0.08425261605648982, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9873805405098829, + "call_delta": 0.42500638062558493 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7387062927147435, + "performance_delta": 0.08425261605648982, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9873805405098829, + "call_delta": 0.42500638062558493 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7387062927147435, + "performance_delta": 0.08425261605648982, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9873805405098829, + "call_delta": 0.42500638062558493 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.21609512872414097, + "performance_delta": -0.016673386525592054, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9790279627163782, + "call_delta": 0.19773635153129165 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.21609512872414097, + "performance_delta": -0.016673386525592054, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9790279627163782, + "call_delta": 0.19773635153129165 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.21609512872414097, + "performance_delta": -0.016673386525592054, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9790279627163782, + "call_delta": 0.19773635153129165 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26818431283513144, + "performance_delta": -0.00015624525391694233, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9860186418109188, + "call_delta": 0.15645805592543283 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26818431283513144, + "performance_delta": -0.00015624525391694233, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9860186418109188, + "call_delta": 0.15645805592543283 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26818431283513144, + "performance_delta": -0.00015624525391694233, + "baseline_call": 0.829560585885486, + "candidate_call": 0.9860186418109188, + "call_delta": 0.15645805592543283 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9593169261677549, + "call_delta": -0.027911315204132925 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9593169261677549, + "call_delta": -0.027911315204132925 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9872282413718878, + "candidate_call": 0.9593169261677549, + "call_delta": -0.027911315204132925 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9593169261677549, + "call_delta": -0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9593169261677549, + "call_delta": -0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9003372318289445, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9705101528305948, + "candidate_call": 0.9593169261677549, + "call_delta": -0.01119322666283995 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + } + ] + }, + { + "baseline": "original", + "candidate": "score_gain", + "counts": { + "total": 48, + "performance_wins": 22, + "performance_ties": 12, + "performance_losses": 14, + "call_less": 12, + "call_ties": 12, + "call_more": 24, + "both_performance_win_and_call_less": 12, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.23610378506203042, + "performance_delta": 0.00636626667272841, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7952729693741678, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23330554571941767, + "performance_delta": 0.0008511922008775652, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7902796271637816, + "call_delta": 0.009320905459387463 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22951409007632145, + "performance_delta": -0.003254425173411568, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.8009320905459387, + "call_delta": 0.019640479360852225 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2687941772737477, + "performance_delta": 0.0004707717644751308, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8062583222370173, + "call_delta": -0.00998668442077233 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2699197889196931, + "performance_delta": 0.003992751149164753, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.831225033288948, + "call_delta": -0.007989347536617841 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2687146609089476, + "performance_delta": 0.00037410281989919136, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8225699067909454, + "call_delta": -0.006990679094540542 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2668710925054341, + "performance_delta": -0.0004626043850421979, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.39580559254327563, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.25610142784250745, + "performance_delta": -0.002510607023462952, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.46804260985352863, + "call_delta": 0.10852197070572572 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2553143923177106, + "performance_delta": -0.010255108607445773, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5402796271637816, + "call_delta": 0.08588548601864182 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3034889605344151, + "performance_delta": 0.005719274933867702, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.46937416777629826, + "call_delta": -0.021637816245006658 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2998980207837558, + "performance_delta": 0.000508977878148853, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.4284287616511318, + "call_delta": -0.025965379494008012 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3033337683996216, + "performance_delta": 0.0003620662089585114, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.45272969374167776, + "call_delta": -0.001997336884154488 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8134276838108389, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0, + "candidate_call": 0.010700724887814981, + "call_delta": 0.010700724887814981 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8998964445978599, + "performance_delta": -0.0017259233690023468, + "baseline_call": 0.0, + "candidate_call": 0.027614773904038662, + "call_delta": 0.027614773904038662 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7514108272126592, + "performance_delta": 0.002070158523097887, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.8495023112043786, + "call_delta": -0.00944332586563812 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.752006352267249, + "performance_delta": 0.039389728610725205, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8824263392224144, + "call_delta": 0.19391430110881092 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7502197771034795, + "performance_delta": 0.09576610044522582, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9653177551541275, + "call_delta": 0.4029435952698296 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7447182599325072, + "performance_delta": -0.013838867935229593, + "baseline_call": 0.5675353770240762, + "candidate_call": 1.0, + "call_delta": 0.4324646229759238 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.968380455434875, + "call_delta": 0.35853444119899036 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7482063352522474, + "performance_delta": -0.012194084451124354, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.96633865524771, + "call_delta": 0.36423446672149273 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9231541938724259, + "candidate_call": 1.0, + "call_delta": 0.07684580612757408 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.0006457630767023881, + "baseline_call": 0.9343474205352659, + "candidate_call": 1.0, + "call_delta": 0.06565257946473413 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.8682643323527302, + "candidate_call": 1.0, + "call_delta": 0.13173566764726985 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9060055966133315, + "performance_delta": -0.001004520341536863, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9787615699217909, + "call_delta": 0.12850685226375824 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9049293248188276, + "performance_delta": 0.0011480232474707641, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9672813374470833, + "call_delta": 0.10303508646050086 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9039965559302576, + "performance_delta": 0.004018081366147674, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9788333213747579, + "call_delta": 0.017148597259094567 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9023462725120184, + "performance_delta": 0.0017220348712061462, + "baseline_call": 1.0, + "candidate_call": 0.9800530960751955, + "call_delta": -0.01994690392480447 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9036377986654229, + "performance_delta": 0.0030135610246107003, + "baseline_call": 1.0, + "candidate_call": 0.9882327617134247, + "call_delta": -0.011767238286575332 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "checkpoint_perf", + "counts": { + "total": 48, + "performance_wins": 7, + "performance_ties": 18, + "performance_losses": 23, + "call_less": 17, + "call_ties": 17, + "call_more": 14, + "both_performance_win_and_call_less": 3, + "both_performance_tie_and_call_less": 1 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2305421865004371, + "performance_delta": 0.0008046681111350817, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7816245006657789, + "call_delta": -0.00898801597869514 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.22838091389772372, + "performance_delta": -0.0040734396208163826, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.8049267643142477, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.23035654564059277, + "performance_delta": -0.002411969609140252, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.774300932090546, + "call_delta": -0.006990679094540542 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.26792218076116586, + "performance_delta": -0.00040122474810672104, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8215712383488681, + "call_delta": 0.005326231691078487 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.26876666264349774, + "performance_delta": 0.002839624872969404, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8125832223701731, + "call_delta": -0.026631158455392767 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26833406705021806, + "performance_delta": -6.491038830325202e-06, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8229027962716379, + "call_delta": -0.0066577896138481085 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2624005680779394, + "performance_delta": -0.0049331288125369355, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.37350199733688416, + "call_delta": 0.0016644474034620549 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.25991507502469235, + "performance_delta": 0.0013030401587219576, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.37283621837549935, + "call_delta": 0.013315579227696439 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2613858995371947, + "performance_delta": -0.004183601387961644, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.3808255659121172, + "call_delta": -0.07356857523302263 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29875083466112995, + "performance_delta": 0.0009811490605825535, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.43275632490013316, + "call_delta": -0.058255659121171754 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29645395142312325, + "performance_delta": -0.0029350914824837204, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.524300932090546, + "call_delta": 0.06990679094540614 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3014472023134628, + "performance_delta": -0.0015244998772003249, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4490679094540613, + "call_delta": -0.0056591211717709755 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8967897825336555, + "performance_delta": -0.004832585433206749, + "baseline_call": 0.0, + "candidate_call": 0.038488091128753885, + "call_delta": 0.038488091128753885 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8979979288919572, + "performance_delta": -0.0036244390749050615, + "baseline_call": 0.0, + "candidate_call": 0.0277873662409389, + "call_delta": 0.0277873662409389 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.445622890848765, + "performance_delta": -0.010974676006011985, + "baseline_call": 0.407140628987891, + "candidate_call": 0.4267929557893543, + "call_delta": 0.019652326801463282 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4702379264384766, + "performance_delta": 0.007061225647279068, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.47905736891359213, + "call_delta": 0.023452343816464827 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.44219153220089047, + "performance_delta": -0.004140317046195718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.4057227121912486, + "call_delta": 0.0015880668122394415 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.6876612880356181, + "performance_delta": -0.06167938065394318, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.6266057907721975, + "call_delta": -0.23233984629781923 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.6622805773757195, + "performance_delta": -0.050336046280804236, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.5790488614128123, + "call_delta": -0.1094631767007912 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.6775940787794572, + "performance_delta": 0.023140402121203496, + "baseline_call": 0.562374159884298, + "candidate_call": 0.6149505147037971, + "call_delta": 0.05257635481949918 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6657119360235941, + "performance_delta": -0.008904517482914098, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.322916371267334, + "call_delta": 0.09006607492272353 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6712134531945665, + "performance_delta": 0.0, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.27691915038425546, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6739925701159856, + "performance_delta": 0.0, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.2715877832288801, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7582735445084082, + "performance_delta": -0.00028358335932854306, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.537475540935258, + "call_delta": -0.030059836088818237 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7583586195162068, + "performance_delta": 0.0, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.6098460142358847, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7604004197033718, + "performance_delta": 0.0, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6021041885262173, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9080863887493722, + "performance_delta": 0.0, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9231541938724259, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9079428858434383, + "performance_delta": -7.175145296700602e-05, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9026332783238861, + "call_delta": -0.0317141422113798 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.907297122766736, + "performance_delta": -0.00028700581186769103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8648202626103179, + "call_delta": -0.0034440697424122924 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9072253713137691, + "performance_delta": -0.0009327688885699681, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.8425773121905719, + "call_delta": 0.01528305948195452 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9070101169548683, + "performance_delta": 0.0, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.8502547176580326, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9037813015713568, + "performance_delta": 0.0, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8642462509865825, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9071536198608022, + "performance_delta": 0.0, + "baseline_call": 0.98880677333716, + "candidate_call": 0.9778288010332209, + "call_delta": -0.010977972303939154 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9070818684078352, + "performance_delta": -0.00028700581186769103, + "baseline_call": 1.0, + "candidate_call": 0.9764655234268494, + "call_delta": -0.023534476573150553 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9047858219128937, + "performance_delta": -0.0025830523068092193, + "baseline_call": 1.0, + "candidate_call": 0.9235129511372605, + "call_delta": -0.07648704886273949 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9013417521704814, + "performance_delta": 0.0013632776063714491, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9797660902633278, + "call_delta": 0.018081366147664535 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9002654803759776, + "performance_delta": -0.00035875726483458603, + "baseline_call": 1.0, + "candidate_call": 0.966563822917414, + "call_delta": -0.03343617708258595 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9003372318289445, + "performance_delta": -0.00028700581186769103, + "baseline_call": 1.0, + "candidate_call": 0.9903853053024324, + "call_delta": -0.009614694697567594 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "both", + "counts": { + "total": 48, + "performance_wins": 17, + "performance_ties": 10, + "performance_losses": 21, + "call_less": 14, + "call_ties": 11, + "call_more": 23, + "both_performance_win_and_call_less": 8, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22850906574549645, + "performance_delta": -0.0012284526438055554, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7906125166444741, + "call_delta": 0.0 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23004315338506715, + "performance_delta": -0.002411200133472957, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7926098535286284, + "call_delta": 0.011651131824234273 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2308450652014655, + "performance_delta": -0.001923450048267522, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7766311584553928, + "call_delta": -0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2672954581839896, + "performance_delta": 0.0013684204134612465, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8215712383488681, + "call_delta": -0.017643142476697737 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.267918819975632, + "performance_delta": -0.0004217381134163922, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8149134487350199, + "call_delta": -0.01464713715046606 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2675187098549581, + "performance_delta": 0.00018501296448181215, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.4144474034620506, + "call_delta": 0.042609853528628505 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.26086905351963807, + "performance_delta": -0.004700447405518282, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.39980026631158455, + "call_delta": -0.05459387483355527 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3029107097560361, + "performance_delta": 0.005141024155488727, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4337549933422104, + "call_delta": -0.05725699067909451 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29937410202312603, + "performance_delta": -1.4940882480940498e-05, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.44274300932090543, + "call_delta": -0.011651131824234384 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3009613147999525, + "performance_delta": -0.002010387390710633, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4477363515312916, + "call_delta": -0.006990679094540653 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8125647221263376, + "performance_delta": -0.0017259233690024578, + "baseline_call": 0.0, + "candidate_call": 0.017604418363824648, + "call_delta": 0.017604418363824648 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8134276838108389, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0, + "candidate_call": 0.010700724887814981, + "call_delta": 0.010700724887814981 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8998964445978599, + "performance_delta": -0.0017259233690023468, + "baseline_call": 0.0, + "candidate_call": 0.027614773904038662, + "call_delta": 0.027614773904038662 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.901449775629962, + "performance_delta": -0.00017259233690025688, + "baseline_call": 0.0, + "candidate_call": 0.010873317224715223, + "call_delta": 0.010873317224715223 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7511272438533307, + "performance_delta": 0.001786575163769455, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.852763519836656, + "call_delta": -0.006182117233360707 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7511839605251964, + "performance_delta": 0.09673028386694271, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8652411876471089, + "call_delta": 0.30286702776281094 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9705640473017043, + "call_delta": 0.3607180330658196 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7550406942120637, + "performance_delta": -0.005359725491308143, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9094518333664181, + "call_delta": 0.30734764484020083 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.002009040683073837, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9890937791490277, + "call_delta": 0.06593958527660182 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0015785319652723562, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.98973954222573, + "call_delta": 0.12147520987299987 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9052163306306953, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9674248403530171, + "call_delta": 0.1171701226949845 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9075123771256368, + "performance_delta": 0.0037310755542799834, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9586711630910526, + "call_delta": 0.09442491210447013 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9034225443065222, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9885197675252924, + "call_delta": 0.026835043409629056 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9037095501183899, + "performance_delta": 0.0030853124775777063, + "baseline_call": 1.0, + "candidate_call": 0.9783310612039894, + "call_delta": -0.021668938796010617 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9026332783238861, + "performance_delta": 0.002009040683073837, + "baseline_call": 1.0, + "candidate_call": 0.9682141063356533, + "call_delta": -0.0317858936643467 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "score_only_gain", + "counts": { + "total": 48, + "performance_wins": 18, + "performance_ties": 13, + "performance_losses": 17, + "call_less": 12, + "call_ties": 13, + "call_more": 23, + "both_performance_win_and_call_less": 9, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22725460966786967, + "performance_delta": -0.0024829087214323364, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.8768308921438083, + "call_delta": 0.0862183754993342 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23264549293538322, + "performance_delta": 0.0001911394168431224, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7882822902796272, + "call_delta": 0.007323568575232975 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.229420185942548, + "performance_delta": -0.0033483293071850107, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7959387483355526, + "call_delta": 0.01464713715046606 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.270279514537055, + "performance_delta": 0.0043524767665266695, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.801930758988016, + "call_delta": -0.03728362183754985 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.26989568494530464, + "performance_delta": 0.0015551268562562504, + "baseline_call": 0.829560585885486, + "candidate_call": 0.800266311584554, + "call_delta": -0.02929427430093201 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2668710925054341, + "performance_delta": -0.0004626043850421979, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.39580559254327563, + "call_delta": 0.023968042609853524 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2565642588848189, + "performance_delta": -0.00900524204033748, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.46571238348868177, + "call_delta": 0.011318242343541951 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29895985090664157, + "performance_delta": 0.0011901653060941708, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4497336884154461, + "call_delta": -0.04127829560585883 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2991984700035601, + "performance_delta": -0.0001905729020468816, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.453728362183755, + "call_delta": -0.0006657789613848109 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3033337683996216, + "performance_delta": 0.0003620662089585114, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.45272969374167776, + "call_delta": -0.001997336884154488 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9005868139454608, + "performance_delta": -0.0010355540214014303, + "baseline_call": 0.0, + "candidate_call": 0.021574042112530202, + "call_delta": 0.021574042112530202 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.511414230212971, + "performance_delta": 0.05481666335819407, + "baseline_call": 0.407140628987891, + "candidate_call": 0.7920199642684967, + "call_delta": 0.38487933528060575 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.503984346198565, + "performance_delta": 0.04080764540736748, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.597339988089499, + "call_delta": 0.14173496299237165 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5101097467600602, + "performance_delta": 0.06377789751297397, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.7076255565323427, + "call_delta": 0.30349091115333354 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7498794770722854, + "performance_delta": 0.0005388083827241319, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.846070952556504, + "call_delta": -0.012874684513512702 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7510138105095993, + "performance_delta": 0.09656013385134554, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8730964467005076, + "call_delta": 0.3107222868162096 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6809403624195333, + "performance_delta": 0.006323908913025034, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.1506961971471514, + "call_delta": -0.08215409919745909 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.0017015001559708143, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.0, + "call_delta": -0.27691915038425546 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6797493123103536, + "performance_delta": 0.0057567421943680586, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.11924680259762357, + "call_delta": -0.15234098063125653 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7474406601820606, + "performance_delta": -0.010917959334146188, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9800073731673425, + "call_delta": 0.37016135893145785 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7482063352522474, + "performance_delta": -0.012194084451124354, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.96633865524771, + "call_delta": 0.36423446672149273 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9231541938724259, + "candidate_call": 1.0, + "call_delta": 0.07684580612757408 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9055750878955299, + "performance_delta": -0.0024395494008754293, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9716581760780656, + "call_delta": 0.03731075554279972 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.8682643323527302, + "candidate_call": 1.0, + "call_delta": 0.13173566764726985 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9060055966133315, + "performance_delta": -0.001004520341536863, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9787615699217909, + "call_delta": 0.12850685226375824 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9060773480662984, + "performance_delta": 0.0022960464949415282, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9367152184831743, + "call_delta": 0.07246896749659182 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9033507928535552, + "performance_delta": 0.0033723182894452863, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9880892588074909, + "call_delta": 0.026404534691827575 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9023462725120184, + "performance_delta": 0.0017220348712061462, + "baseline_call": 1.0, + "candidate_call": 0.9800530960751955, + "call_delta": -0.01994690392480447 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9033507928535552, + "performance_delta": 0.0027265552127430093, + "baseline_call": 1.0, + "candidate_call": 0.987012987012987, + "call_delta": -0.012987012987012991 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.25", + "counts": { + "total": 48, + "performance_wins": 15, + "performance_ties": 14, + "performance_losses": 19, + "call_less": 5, + "call_ties": 14, + "call_more": 29, + "both_performance_win_and_call_less": 2, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2211131170053352, + "performance_delta": -0.008624401383966807, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.988681757656458, + "call_delta": 0.19806924101198398 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.2222578228236749, + "performance_delta": -0.010196530694865202, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.9880159786950732, + "call_delta": 0.207057256990679 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22347202518399215, + "performance_delta": -0.009296490065740876, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.988681757656458, + "call_delta": 0.20739014647137155 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.0016866851695898633, + "baseline_call": 0.8162450066577897, + "candidate_call": 1.0, + "call_delta": 0.18375499334221035 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.004083052908334106, + "baseline_call": 0.8392143808255659, + "candidate_call": 1.0, + "call_delta": 0.16078561917443412 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2587284729338324, + "performance_delta": -0.008605223956643926, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.5702396804260985, + "call_delta": 0.1984021304926764 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2591386172712909, + "performance_delta": 0.000526582405320497, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.5509320905459387, + "call_delta": 0.1914114513981358 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.25650251356280634, + "performance_delta": -0.00906698736235001, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5622503328894807, + "call_delta": 0.10785619174434086 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.2712680480298825, + "performance_delta": -0.02650163757066487, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.6844207723035952, + "call_delta": 0.1934087882822903 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.27348865863056027, + "performance_delta": -0.025900384275046706, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.6474700399467377, + "call_delta": 0.19307589880159787 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2704420670349152, + "performance_delta": -0.032529635155747916, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.5572569906790945, + "call_delta": 0.10252996005326226 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.043473328985055126, + "baseline_call": 0.407140628987891, + "candidate_call": 1.0, + "call_delta": 0.5928593710121091 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.03689419504863456, + "baseline_call": 0.4556050250971273, + "candidate_call": 1.0, + "call_delta": 0.5443949749028727 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7447182599325072, + "performance_delta": -0.004622408757054108, + "baseline_call": 0.8589456370700167, + "candidate_call": 1.0, + "call_delta": 0.1410543629299833 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.03210163627598339, + "baseline_call": 0.6885120381136035, + "candidate_call": 1.0, + "call_delta": 0.3114879618863965 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.005104500467912554, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.0, + "call_delta": -0.2328502963446105 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.0017015001559708143, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.0, + "call_delta": -0.27691915038425546 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.004480617077389892, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.0, + "call_delta": -0.2715877832288801 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7470436434790007, + "performance_delta": -0.011513484388736117, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9068712247965289, + "call_delta": 0.33933584777245274 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7490287269943, + "performance_delta": -0.009329892521906746, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9408445112440802, + "call_delta": 0.3309984970081955 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7492272353458299, + "performance_delta": -0.011173184357541888, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.938831069392848, + "call_delta": 0.3367268808666307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.0007175145296692831, + "baseline_call": 0.9231541938724259, + "candidate_call": 1.0, + "call_delta": 0.07684580612757408 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9050728277247614, + "performance_delta": -0.0029418095716439163, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9550835904427064, + "call_delta": 0.020736169907440538 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0012915261534045541, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9650570424051087, + "call_delta": 0.09679271005237855 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9090191576379422, + "performance_delta": 0.0008610174356030731, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9368587213891081, + "call_delta": 0.10956446868049075 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9069383655019014, + "performance_delta": -7.1751452966895e-05, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9668508287292817, + "call_delta": 0.11659611107124912 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9096649207146444, + "performance_delta": 0.005883619143287611, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8986151969577384, + "call_delta": 0.03436894597115592 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9027767812298199, + "performance_delta": 0.0027983066657099043, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9891655306019946, + "call_delta": 0.027480806486331333 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9030637870416875, + "performance_delta": 0.0024395494008753182, + "baseline_call": 1.0, + "candidate_call": 0.988017507354524, + "call_delta": -0.011982492645476017 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9031355384946546, + "performance_delta": 0.0025113008538423243, + "baseline_call": 1.0, + "candidate_call": 0.9648417880462079, + "call_delta": -0.035158211953792096 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.5", + "counts": { + "total": 48, + "performance_wins": 14, + "performance_ties": 14, + "performance_losses": 20, + "call_less": 5, + "call_ties": 14, + "call_more": 29, + "both_performance_win_and_call_less": 2, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22427095268610697, + "performance_delta": -0.0054665657031950365, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.9853528628495339, + "call_delta": 0.19474034620505987 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.22607173521804688, + "performance_delta": -0.00638261830049322, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.9833555259653795, + "call_delta": 0.20239680426098527 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2255853947574774, + "performance_delta": -0.0071831204922556124, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.9810252996005326, + "call_delta": 0.19973368841544614 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.0016866851695898633, + "baseline_call": 0.8162450066577897, + "candidate_call": 1.0, + "call_delta": 0.18375499334221035 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.004083052908334106, + "baseline_call": 0.8392143808255659, + "candidate_call": 1.0, + "call_delta": 0.16078561917443412 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.26009300196494534, + "performance_delta": -0.007240694925530977, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.5699067909454061, + "call_delta": 0.19806924101198398 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2610246960878656, + "performance_delta": 0.002412661221895207, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.5223035952063915, + "call_delta": 0.16278295605858856 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2584417288296328, + "performance_delta": -0.007127772095523566, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5529294274300932, + "call_delta": 0.0985352862849534 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.2741835645844183, + "performance_delta": -0.023586121016129113, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.6231691078561917, + "call_delta": 0.1321571238348868 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2747498073529785, + "performance_delta": -0.02463923555262848, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.6195073235685752, + "call_delta": 0.16511318242343537 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2746117403663106, + "performance_delta": -0.02835996182435252, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.6088548601864181, + "call_delta": 0.15412782956058585 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.043473328985055126, + "baseline_call": 0.407140628987891, + "candidate_call": 1.0, + "call_delta": 0.5928593710121091 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.03689419504863456, + "baseline_call": 0.4556050250971273, + "candidate_call": 1.0, + "call_delta": 0.5443949749028727 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7447182599325072, + "performance_delta": -0.004622408757054108, + "baseline_call": 0.8589456370700167, + "candidate_call": 1.0, + "call_delta": 0.1410543629299833 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.03210163627598339, + "baseline_call": 0.6885120381136035, + "candidate_call": 1.0, + "call_delta": 0.3114879618863965 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6695119530385957, + "performance_delta": -0.005104500467912554, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.019680685137396136, + "call_delta": -0.21316961120721437 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6695403113745285, + "performance_delta": -0.001673141820037971, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.019935910160791766, + "call_delta": -0.2569832402234637 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.669625386382327, + "performance_delta": -0.0043671837336585195, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.016929926551909932, + "call_delta": -0.25465785667697016 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7471287184867992, + "performance_delta": -0.011428409380937588, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9701103139267788, + "call_delta": 0.40257493690270263 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7488018603068372, + "performance_delta": -0.009556759209369603, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.959107279584834, + "call_delta": 0.3492612653489493 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7490287269943, + "performance_delta": -0.01137169270907179, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9377818109633327, + "call_delta": 0.3356776224371154 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.0014350290593384551, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.987012987012987, + "call_delta": 0.06385879314056109 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9075123771256368, + "performance_delta": -0.0005022601707684871, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9893807849608954, + "call_delta": 0.05503336442562956 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9069383655019014, + "performance_delta": -0.0006457630767022771, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9877305015426563, + "call_delta": 0.11946616918992614 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080863887493722, + "performance_delta": -7.1751452966895e-05, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9687163665064218, + "call_delta": 0.14142211379780445 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9065796082370668, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9649852909521418, + "call_delta": 0.11473057329410918 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9091626605438761, + "performance_delta": 0.0053813589725192346, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9574513883906149, + "call_delta": 0.09320513740403247 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9029202841357538, + "performance_delta": 0.0029418095716438053, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9893807849608954, + "call_delta": 0.02769606084523213 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9034942957594891, + "performance_delta": 0.0028700581186769103, + "baseline_call": 1.0, + "candidate_call": 0.9801965989811293, + "call_delta": -0.01980340101887068 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9028485326827869, + "performance_delta": 0.0022242950419746332, + "baseline_call": 1.0, + "candidate_call": 0.9671378345411494, + "call_delta": -0.03286216545885057 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_0.75", + "counts": { + "total": 48, + "performance_wins": 20, + "performance_ties": 11, + "performance_losses": 17, + "call_less": 11, + "call_ties": 11, + "call_more": 26, + "both_performance_win_and_call_less": 8, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.2319989174313225, + "performance_delta": 0.0022613990420204955, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7762982689747004, + "call_delta": -0.014314247669773628 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23674301754245702, + "performance_delta": 0.004288664023916922, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7596537949400799, + "call_delta": -0.02130492676431428 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.23714462398254227, + "performance_delta": 0.004376108732809247, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.8578561917443409, + "call_delta": 0.07656458055925441 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2698761268543865, + "performance_delta": 0.0015527213451139166, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8032623169107856, + "call_delta": -0.012982689747004006 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2684151526427204, + "performance_delta": 0.002488114872192082, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8169107856191744, + "call_delta": -0.02230359520639147 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.2696999750942859, + "performance_delta": 0.0013594170052375065, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8199067909454061, + "call_delta": -0.009653794940079896 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.26372894434686617, + "performance_delta": -0.0036047525436101457, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.3848202396804261, + "call_delta": 0.012982689747004006 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2621221451028754, + "performance_delta": 0.0035101102369050063, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.3725033288948069, + "call_delta": 0.012982689747004006 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.25716044964994544, + "performance_delta": -0.00840905127521091, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.40346205059920104, + "call_delta": -0.05093209054593878 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.29493861915351943, + "performance_delta": -0.0028310664470279634, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.5512649800266312, + "call_delta": 0.06025299600532624 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.2927064243318904, + "performance_delta": -0.006682618573716548, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.5825565912117177, + "call_delta": 0.12816245006657784 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2929922062010103, + "performance_delta": -0.009979495989652809, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.5808921438082557, + "call_delta": 0.12616511318242346 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8120469451156369, + "performance_delta": -0.0022437003797031174, + "baseline_call": 0.0, + "candidate_call": 0.06247842595788747, + "call_delta": 0.06247842595788747 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8198136002761477, + "performance_delta": 0.005522954780807665, + "baseline_call": 0.0, + "candidate_call": 0.07611322057300655, + "call_delta": 0.07611322057300655 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8945460821539524, + "performance_delta": -0.007076285812909866, + "baseline_call": 0.0, + "candidate_call": 0.050914739385571284, + "call_delta": 0.050914739385571284 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48044692737430167, + "performance_delta": 0.023849360519524687, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5296202818818592, + "call_delta": 0.12247965289396817 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.4890111448260216, + "performance_delta": 0.025834444034824045, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.550860675495562, + "call_delta": 0.09525565039843464 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.4808723024132944, + "performance_delta": 0.034540453166208185, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5036156878314381, + "call_delta": 0.09948104245242889 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7464481184244108, + "performance_delta": -0.002892550265150451, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.9825879817372316, + "call_delta": 0.12364234466721491 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7467033434478065, + "performance_delta": 0.03408671979128275, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.9870402404786887, + "call_delta": 0.29852820236508515 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7468734934634036, + "performance_delta": 0.09241981680514988, + "baseline_call": 0.562374159884298, + "candidate_call": 0.9880611405722712, + "call_delta": 0.4256869806879733 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.671865694921022, + "performance_delta": -0.002750758585486235, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.02064486855911295, + "call_delta": -0.21220542778549753 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6721492782803505, + "performance_delta": 0.0009358250857840478, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.018659785043813628, + "call_delta": -0.25825936534044186 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6705612114681111, + "performance_delta": -0.0034313586478744718, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.016306043161387288, + "call_delta": -0.2552817400674928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7471287184867992, + "performance_delta": -0.011428409380937588, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.9700252389189802, + "call_delta": 0.402489861894904 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7486600686271729, + "performance_delta": -0.009698550889033819, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9587386212177069, + "call_delta": 0.34889260698182223 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7493690270254941, + "performance_delta": -0.011031392677877672, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9490117119927403, + "call_delta": 0.346907523466523 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.987945755901557, + "call_delta": 0.06479156202913106 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.9066513596900337, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9885915189782594, + "call_delta": 0.054244098442993494 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9073688742197029, + "performance_delta": -0.00021525435890079603, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.9881610102604578, + "call_delta": 0.11989667790772762 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9084451460142068, + "performance_delta": 0.00028700581186769103, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.9588146659969864, + "call_delta": 0.13152041328836905 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9062926024251992, + "performance_delta": -0.0007175145296691721, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.966563822917414, + "call_delta": 0.11630910525938143 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9085168974671737, + "performance_delta": 0.0047355958958168465, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9580254000143503, + "call_delta": 0.09377914902776785 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9031355384946546, + "performance_delta": 0.0031570639305446013, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9893090335079285, + "call_delta": 0.027624309392265234 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9039965559302576, + "performance_delta": 0.0033723182894453974, + "baseline_call": 1.0, + "candidate_call": 0.9791920786395925, + "call_delta": -0.020807921360407544 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.902705029776853, + "performance_delta": 0.002080792136040732, + "baseline_call": 1.0, + "candidate_call": 0.9555858506134749, + "call_delta": -0.044414149386525104 + } + ], + "rule": "endpoint" + }, + { + "baseline": "original", + "candidate": "fusion_alpha_1", + "counts": { + "total": 48, + "performance_wins": 17, + "performance_ties": 10, + "performance_losses": 21, + "call_less": 14, + "call_ties": 11, + "call_more": 23, + "both_performance_win_and_call_less": 8, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.229737518389302, + "candidate_metric": 0.22850906574549645, + "performance_delta": -0.0012284526438055554, + "baseline_call": 0.7906125166444741, + "candidate_call": 0.7906125166444741, + "call_delta": 0.0 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.2324543535185401, + "candidate_metric": 0.23004315338506715, + "performance_delta": -0.002411200133472957, + "baseline_call": 0.7809587217043942, + "candidate_call": 0.7926098535286284, + "call_delta": 0.011651131824234273 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.2308450652014655, + "performance_delta": -0.001923450048267522, + "baseline_call": 0.7812916111850865, + "candidate_call": 0.7766311584553928, + "call_delta": -0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683234055092726, + "candidate_metric": 0.2678997047385957, + "performance_delta": -0.0004237007706768825, + "baseline_call": 0.8162450066577897, + "candidate_call": 0.8209054593874834, + "call_delta": 0.0046604527296937315 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.26592703777052834, + "candidate_metric": 0.2672954581839896, + "performance_delta": 0.0013684204134612465, + "baseline_call": 0.8392143808255659, + "candidate_call": 0.8215712383488681, + "call_delta": -0.017643142476697737 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.267918819975632, + "performance_delta": -0.0004217381134163922, + "baseline_call": 0.829560585885486, + "candidate_call": 0.8149134487350199, + "call_delta": -0.01464713715046606 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2673336968904763, + "candidate_metric": 0.2675187098549581, + "performance_delta": 0.00018501296448181215, + "baseline_call": 0.3718375499334221, + "candidate_call": 0.4144474034620506, + "call_delta": 0.042609853528628505 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.2586120348659704, + "candidate_metric": 0.2578686516978949, + "performance_delta": -0.0007433831680754843, + "baseline_call": 0.3595206391478029, + "candidate_call": 0.35918774966711053, + "call_delta": -0.00033288948069237767 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.26086905351963807, + "performance_delta": -0.004700447405518282, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.39980026631158455, + "call_delta": -0.05459387483355527 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.2977696856005474, + "candidate_metric": 0.3029107097560361, + "performance_delta": 0.005141024155488727, + "baseline_call": 0.4910119840213049, + "candidate_call": 0.4337549933422104, + "call_delta": -0.05725699067909451 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.299389042905607, + "candidate_metric": 0.29937410202312603, + "performance_delta": -1.4940882480940498e-05, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.44274300932090543, + "call_delta": -0.011651131824234384 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.3009613147999525, + "performance_delta": -0.002010387390710633, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.4477363515312916, + "call_delta": -0.006990679094540653 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8125647221263376, + "performance_delta": -0.0017259233690024578, + "baseline_call": 0.0, + "candidate_call": 0.017604418363824648, + "call_delta": 0.017604418363824648 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.8134276838108389, + "performance_delta": -0.0008629616845011734, + "baseline_call": 0.0, + "candidate_call": 0.010700724887814981, + "call_delta": 0.010700724887814981 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.8998964445978599, + "performance_delta": -0.0017259233690023468, + "baseline_call": 0.0, + "candidate_call": 0.027614773904038662, + "call_delta": 0.027614773904038662 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.901449775629962, + "performance_delta": -0.00017259233690025688, + "baseline_call": 0.0, + "candidate_call": 0.010873317224715223, + "call_delta": 0.010873317224715223 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.456597566854777, + "candidate_metric": 0.48586336953747555, + "performance_delta": 0.029265802682698572, + "baseline_call": 0.407140628987891, + "candidate_call": 0.5045798712531548, + "call_delta": 0.09743924226526385 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.46317670079119755, + "candidate_metric": 0.49794402064486853, + "performance_delta": 0.034767319853670986, + "baseline_call": 0.4556050250971273, + "candidate_call": 0.5430054164421632, + "call_delta": 0.08740039134503585 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.49649774551229336, + "performance_delta": 0.05016589626520718, + "baseline_call": 0.40413464537900917, + "candidate_call": 0.5379576326461163, + "call_delta": 0.13382298726710712 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7493406686895613, + "candidate_metric": 0.7511272438533307, + "performance_delta": 0.001786575163769455, + "baseline_call": 0.8589456370700167, + "candidate_call": 0.852763519836656, + "call_delta": -0.006182117233360707 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.7126166236565238, + "candidate_metric": 0.7536511357513541, + "performance_delta": 0.04103451209483033, + "baseline_call": 0.6885120381136035, + "candidate_call": 0.8933726568924936, + "call_delta": 0.20486061877889006 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7511839605251964, + "performance_delta": 0.09673028386694271, + "baseline_call": 0.562374159884298, + "candidate_call": 0.8652411876471089, + "call_delta": 0.30286702776281094 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6746164535065082, + "candidate_metric": 0.6800045373337492, + "performance_delta": 0.005388083827240986, + "baseline_call": 0.2328502963446105, + "candidate_call": 0.12287666959702805, + "call_delta": -0.10997362674758245 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6712134531945665, + "candidate_metric": 0.6805717040524062, + "performance_delta": 0.0093582508578397, + "baseline_call": 0.27691915038425546, + "candidate_call": 0.09480191702350906, + "call_delta": -0.18211723336074642 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6810254374273318, + "performance_delta": 0.007032867311346225, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.12109009443325866, + "call_delta": -0.15049768879562145 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7585571278677368, + "candidate_metric": 0.7519496355953833, + "performance_delta": -0.006607492272353466, + "baseline_call": 0.5675353770240762, + "candidate_call": 0.8643620792331906, + "call_delta": 0.29682670220911445 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7583586195162068, + "candidate_metric": 0.7473555851742619, + "performance_delta": -0.011003034341944828, + "baseline_call": 0.6098460142358847, + "candidate_call": 0.9705640473017043, + "call_delta": 0.3607180330658196 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7550406942120637, + "performance_delta": -0.005359725491308143, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.9094518333664181, + "call_delta": 0.30734764484020083 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080863887493722, + "candidate_metric": 0.9060773480662984, + "performance_delta": -0.002009040683073837, + "baseline_call": 0.9231541938724259, + "candidate_call": 0.9890937791490277, + "call_delta": 0.06593958527660182 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9080146372964053, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0015785319652723562, + "baseline_call": 0.9343474205352659, + "candidate_call": 0.9886632704312263, + "call_delta": 0.05431584989596039 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9071536198608022, + "performance_delta": -0.00043050871780148103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.98973954222573, + "call_delta": 0.12147520987299987 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9081581402023391, + "candidate_metric": 0.9080146372964053, + "performance_delta": -0.00014350290593379, + "baseline_call": 0.8272942527086173, + "candidate_call": 0.948051948051948, + "call_delta": 0.1207576953433307 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9070101169548683, + "candidate_metric": 0.9052163306306953, + "performance_delta": -0.0017937863241730412, + "baseline_call": 0.8502547176580326, + "candidate_call": 0.9674248403530171, + "call_delta": 0.1171701226949845 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9075123771256368, + "performance_delta": 0.0037310755542799834, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.9586711630910526, + "call_delta": 0.09442491210447013 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9071536198608022, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.000215254358900685, + "baseline_call": 0.98880677333716, + "candidate_call": 1.0, + "call_delta": 0.01119322666283995 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.9073688742197029, + "performance_delta": 0.0, + "baseline_call": 1.0, + "candidate_call": 1.0, + "call_delta": 0.0 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.89997847456411, + "candidate_metric": 0.9034225443065222, + "performance_delta": 0.0034440697424122924, + "baseline_call": 0.9616847241156633, + "candidate_call": 0.9885197675252924, + "call_delta": 0.026835043409629056 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9037095501183899, + "performance_delta": 0.0030853124775777063, + "baseline_call": 1.0, + "candidate_call": 0.9783310612039894, + "call_delta": -0.021668938796010617 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.9026332783238861, + "performance_delta": 0.002009040683073837, + "baseline_call": 1.0, + "candidate_call": 0.9682141063356533, + "call_delta": -0.0317858936643467 + } + ], + "rule": "endpoint" + }, + { + "rule": "endpoint", + "baseline": "original", + "candidate": "confidence", + "counts": { + "total": 48, + "performance_wins": 12, + "performance_ties": 12, + "performance_losses": 24, + "call_less": 12, + "call_ties": 12, + "call_more": 24, + "both_performance_win_and_call_less": 3, + "both_performance_tie_and_call_less": 0 + }, + "pairs": [ + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|cub|MiniCPM", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|cub|Qwen3.8", + "baseline_metric": 0.81429064549534, + "candidate_metric": 0.81429064549534, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "cub|glsim|MiniCPM", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "cub|glsim|Qwen3.8", + "baseline_metric": 0.9016223679668622, + "candidate_metric": 0.9016223679668622, + "performance_delta": 0.0, + "baseline_call": 0.0, + "candidate_call": 0.0, + "call_delta": 0.0 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|groundingdino|MiniCPM", + "baseline_metric": 0.4463318492470862, + "candidate_metric": 0.5000708958398321, + "performance_delta": 0.05373904659274592, + "baseline_call": 0.40413464537900917, + "candidate_call": 1.0, + "call_delta": 0.5958653546209909 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|groundingdino|Qwen3.8", + "baseline_metric": 0.6544536766582537, + "candidate_metric": 0.7447182599325072, + "performance_delta": 0.09026458327425346, + "baseline_call": 0.562374159884298, + "candidate_call": 1.0, + "call_delta": 0.43762584011570205 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "grefcoco|instancevg|MiniCPM", + "baseline_metric": 0.6739925701159856, + "candidate_metric": 0.6187788900547316, + "performance_delta": -0.055213680061253934, + "baseline_call": 0.2715877832288801, + "candidate_call": 0.5209426310864078, + "call_delta": 0.2493548478575277 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "grefcoco|instancevg|Qwen3.8", + "baseline_metric": 0.7604004197033718, + "candidate_metric": 0.7243286163967898, + "performance_delta": -0.036071803306581995, + "baseline_call": 0.6021041885262173, + "candidate_call": 0.6838612710206166, + "call_delta": 0.08175708249439928 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|nlvr2|MiniCPM", + "baseline_metric": 0.9075841285786037, + "candidate_metric": 0.9077276314845376, + "performance_delta": 0.00014350290593390103, + "baseline_call": 0.8682643323527302, + "candidate_call": 0.8580038745784602, + "call_delta": -0.010260457774269982 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|nlvr2|Qwen3.8", + "baseline_metric": 0.9037813015713568, + "candidate_metric": 0.9024180239649853, + "performance_delta": -0.0013632776063715601, + "baseline_call": 0.8642462509865825, + "candidate_call": 0.8580038745784602, + "call_delta": -0.006242376408122308 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22361115727328065, + "performance_delta": -0.009157357976452368, + "baseline_call": 0.7812916111850865, + "candidate_call": 1.0, + "call_delta": 0.2187083888149135 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22361115727328065, + "performance_delta": -0.009157357976452368, + "baseline_call": 0.7812916111850865, + "candidate_call": 1.0, + "call_delta": 0.2187083888149135 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|MiniCPM", + "baseline_metric": 0.23276851524973302, + "candidate_metric": 0.22361115727328065, + "performance_delta": -0.009157357976452368, + "baseline_call": 0.7812916111850865, + "candidate_call": 1.0, + "call_delta": 0.2187083888149135 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|rtdetr_x_fp32|Qwen3.8", + "baseline_metric": 0.2683405580890484, + "candidate_metric": 0.27001009067886245, + "performance_delta": 0.00166953258981406, + "baseline_call": 0.829560585885486, + "candidate_call": 1.0, + "call_delta": 0.17043941411451402 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0009327688885699681, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0009327688885699681, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "nlvr2|vilt|MiniCPM", + "baseline_metric": 0.9073688742197029, + "candidate_metric": 0.906436105331133, + "performance_delta": -0.0009327688885699681, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0010045203415369741, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0010045203415369741, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "nlvr2|vilt|Qwen3.8", + "baseline_metric": 0.9006242376408122, + "candidate_metric": 0.8996197172992753, + "performance_delta": -0.0010045203415369741, + "baseline_call": 1.0, + "candidate_call": 0.9593169261677549, + "call_delta": -0.04068307383224512 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 42, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 43, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "seed": 44, + "pair_key": "construction|yolo26x|MiniCPM", + "baseline_metric": 0.26556950092515635, + "candidate_metric": 0.2523047077304935, + "performance_delta": -0.013264793194662838, + "baseline_call": 0.4543941411451398, + "candidate_call": 0.9254327563249002, + "call_delta": 0.47103861517976037 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 42, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 43, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + }, + { + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "seed": 44, + "pair_key": "construction|yolo26x|Qwen3.8", + "baseline_metric": 0.3029717021906631, + "candidate_metric": 0.2597490004762105, + "performance_delta": -0.04322270171445258, + "baseline_call": 0.45472703062583225, + "candidate_call": 0.9221038615179761, + "call_delta": 0.4673768308921438 + } + ] + } +] \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/construction_router.json b/releases/20260913_lr/paper_data/cost_20260911/construction_router.json new file mode 100644 index 0000000000000000000000000000000000000000..73a5a64e91b2c4e80d9fb67960fe15135a1dcffc --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/construction_router.json @@ -0,0 +1,27 @@ +{ + "component": "router", + "expert": "yolo26x", + "task": "construction", + "endpoint": "Qwen3.8", + "condition": "both", + "seed": 42, + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_method_improvement_20260911_r1/conditions/both_yolo26x_Qwen3.8_seed42/best.pt", + "sample_count": 1024, + "batch_size": 256, + "row_count_loaded": 3004, + "mean_latency_ms_per_sample": 0.004675843752920628, + "median_latency_ms_per_sample": 0.004634750075638294, + "latency_ms_per_sample": [ + 0.004889749921858311, + 0.004717750009149313, + 0.004544124938547611, + 0.0045517501421272755 + ], + "flops_counted_per_batch": 0, + "gflops_counted_per_sample": 0.0, + "operators": {}, + "flops_scope": "GPU FlopCounterMode for the router forward; multiply-add counted as 2 FLOPs.", + "latency_scope": "A800 CUDA forward latency on cached router features, excluding feature loading and model load.", + "completed": true +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/cost_summary.json b/releases/20260913_lr/paper_data/cost_20260911/cost_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..f8a7e179097a3d5324bcaf6ebb212f67ef4a3d31 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/cost_summary.json @@ -0,0 +1,1036 @@ +{ + "status": "complete", + "protocol": { + "test_results": "C:\\Users\\csh10\\Documents\\CodexProjects\\ICASSP-COLLA\\dse_results\\method_improvement_20260911\\test_results.json", + "curves_use": "validation-calibrated thresholds transferred unchanged to test", + "cost_uses": "actual test call fraction", + "flop_counting": "multiply-add counted as 2 FLOPs; unsupported operations excluded", + "large_generation_flops": "mean prefill + mean completion tokens * one-token decode probe", + "seed_aggregation": "mean and population standard deviation across three seeds for learned/original methods" + }, + "components": { + "grefcoco|instancevg|MiniCPM": { + "pair_key": "grefcoco|instancevg|MiniCPM", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "label": "gRefCOCO / InstanceVG + MiniCPM", + "specialist_gflops_per_input": 7.502680064, + "specialist_forward_latency_ms": 11.246967077255249, + "specialist_end_to_end_latency_ms": 16.31702448321448, + "specialist_sample_count": 64, + "router_gflops_per_input": 0.000954368, + "router_flop_count": 954368, + "router_flop_breakdown": { + "image": 393728, + "text": 196608, + "hidden": 196608, + "confidence": 2048, + "output": 512, + "head_input_to_hidden": 163840, + "head_hidden_to_classes": 1024 + }, + "router_measured_gflops_per_input": 0.0, + "router_latency_ms": 0.001830718741985038, + "router_sample_count": 1024, + "large_prefill_gflops": 7467.756340960001, + "large_decode_one_token_gflops": 15.445344384, + "large_mean_completion_tokens": 169.5, + "large_completion_token_min": 5, + "large_completion_token_max": 512, + "large_estimated_full_generation_gflops": 10085.742214048001, + "large_batch_wall_latency_ms_per_input": 1447.4320754343353, + "large_measured_sample_count": 30, + "large_flops_probe_count": 4, + "flops_formula": "specialist + router + actual_call_fraction * (prefill + mean_completion_tokens * one_token_decode)", + "latency_formula": "specialist end-to-end + router + actual_call_fraction * large-model batch-wall latency", + "specialist_flops_scope": "GPU FlopCounterMode dense products, convolutions and supported attention; multiply-add counted as 2 FLOPs. Elementwise, normalization, custom ops and postprocessing may be missing.", + "specialist_latency_scope": "A800 wall-clock CUDA forward latency and end-to-end latency including image read/preprocess, excluding model load; batch processing uses the recorded batch size.", + "large_flops_scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "large_latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path." + }, + "construction|yolo26x|Qwen3.8": { + "pair_key": "construction|yolo26x|Qwen3.8", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "label": "Construction / YOLO26x + Qwen3.8", + "specialist_gflops_per_input": 194.4179712, + "specialist_forward_latency_ms": 32.08068132400513, + "specialist_end_to_end_latency_ms": 31.79218670447881, + "specialist_sample_count": 64, + "router_gflops_per_input": 0.000726528, + "router_flop_count": 726528, + "router_flop_breakdown": { + "image": 393728, + "hidden": 196608, + "confidence": 2048, + "output": 2048, + "head_input_to_hidden": 131072, + "head_hidden_to_classes": 1024 + }, + "router_measured_gflops_per_input": 0.0, + "router_latency_ms": 0.004675843752920628, + "router_sample_count": 1024, + "large_prefill_gflops": 87588.607720992, + "large_decode_one_token_gflops": 51.905495232, + "large_mean_completion_tokens": 137.03333333333333, + "large_completion_token_min": 110, + "large_completion_token_max": 226, + "large_estimated_full_generation_gflops": 94701.3907509504, + "large_batch_wall_latency_ms_per_input": 2599.870914062194, + "large_measured_sample_count": 30, + "large_flops_probe_count": 4, + "flops_formula": "specialist + router + actual_call_fraction * (prefill + mean_completion_tokens * one_token_decode)", + "latency_formula": "specialist end-to-end + router + actual_call_fraction * large-model batch-wall latency", + "specialist_flops_scope": "GPU FlopCounterMode around the real Ultralytics predict path; multiply-add counted as 2 FLOPs. Elementwise, postprocessing and unsupported custom ops may be missing.", + "specialist_latency_scope": "A800 wall-clock latency for real Ultralytics prediction including image loading/preprocess, forward and result postprocessing, excluding model load; batch processing uses the recorded batch size.", + "large_flops_scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "large_latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path." + } + }, + "selected_points": [ + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "seed_count": 3, + "selected_nominal_budget_mean": 0.04666666666666667, + "selected_nominal_budgets": [ + 0.05, + 0.04, + 0.05 + ], + "test_metric": 0.6805338929378291, + "test_metric_std": 0.0004176374109666599, + "actual_test_call_fraction": 0.11292289368459858, + "actual_test_call_fraction_std": 0.012834207132185534, + "flops_gflops_per_input": 1146.4148301992102, + "latency_ms_per_input": 179.7670735719058 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "seed_count": 3, + "selected_nominal_budget_mean": 0.19333333333333333, + "selected_nominal_budgets": [ + 0.16, + 0.2, + 0.22 + ], + "test_metric": 0.6732741589390202, + "test_metric_std": 0.0014792314998359972, + "actual_test_call_fraction": 0.26045240998591535, + "actual_test_call_fraction_std": 0.019638624816705554, + "flops_gflops_per_input": 2634.3595005774837, + "latency_ms_per_input": 393.3060275397443 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "seed_count": 1, + "selected_nominal_budget_mean": 0.65, + "selected_nominal_budgets": [ + 0.65 + ], + "test_metric": 0.6187788900547316, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.5209426310864078, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 5261.596719877418, + "latency_ms_per_input": 770.347928897579 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "seed_count": 3, + "selected_nominal_budget_mean": 0.44333333333333336, + "selected_nominal_budgets": [ + 0.44, + 0.44, + 0.45 + ], + "test_metric": 0.3010820421930382, + "test_metric_std": 0.001446335574639307, + "actual_test_call_fraction": 0.4414114513981358, + "actual_test_call_fraction_std": 0.005785002395799842, + "flops_gflops_per_input": 41996.69703852701, + "latency_ms_per_input": 1179.4096561722229 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "seed_count": 3, + "selected_nominal_budget_mean": 0.47333333333333333, + "selected_nominal_budgets": [ + 0.5, + 0.46, + 0.46 + ], + "test_metric": 0.3000434768989391, + "test_metric_std": 0.002173546570267329, + "actual_test_call_fraction": 0.46671105193075896, + "actual_test_call_fraction_std": 0.01718389127843443, + "flops_gflops_per_input": 44392.60439440991, + "latency_ms_per_input": 1245.185351734382 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "seed_count": 1, + "selected_nominal_budget_mean": 0.93, + "selected_nominal_budgets": [ + 0.93 + ], + "test_metric": 0.2597490004762105, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.9221038615179761, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 87518.93680030212, + "latency_ms_per_input": 2429.147871853251 + } + ], + "fixed_budget_points": [ + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.0, + "seed_count": 3, + "test_metric": 0.6695119530385957, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 7.503634432, + "latency_ms_per_input": 16.318855201956467 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.01, + "seed_count": 3, + "test_metric": 0.6743328701471798, + "test_metric_std": 0.0002504540415259137, + "actual_test_call_fraction": 0.021268751949635595, + "actual_test_call_fraction_std": 0.0024468276041848553, + "flops_gflops_per_input": 222.01478381055546, + "latency_ms_per_input": 47.10392897831558 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.05, + "seed_count": 3, + "test_metric": 0.6805811568310505, + "test_metric_std": 0.0004271568320976906, + "actual_test_call_fraction": 0.11976670542305912, + "actual_test_call_fraction_std": 0.0032185843726325893, + "flops_gflops_per_input": 1215.4397511547988, + "latency_ms_per_input": 189.67302620038757 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.1, + "seed_count": 3, + "test_metric": 0.6804866290446077, + "test_metric_std": 0.0005385595679416328, + "actual_test_call_fraction": 0.172031118547297, + "actual_test_call_fraction_std": 0.0033259813564282133, + "flops_gflops_per_input": 1742.5651488943693, + "latency_ms_per_input": 265.32221416016074 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.25, + "seed_count": 3, + "test_metric": 0.6794184650578038, + "test_metric_std": 0.00038210745789084077, + "actual_test_call_fraction": 0.24530905859777485, + "actual_test_call_fraction_std": 0.009454413833556307, + "flops_gflops_per_input": 2481.627562219953, + "latency_ms_per_input": 371.38705501097667 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.5, + "seed_count": 3, + "test_metric": 0.664568149807636, + "test_metric_std": 0.0016062493473305785, + "actual_test_call_fraction": 0.4310372534006371, + "actual_test_call_fraction_std": 0.0088737075287505, + "flops_gflops_per_input": 4354.834256882111, + "latency_ms_per_input": 640.2160014811561 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.75, + "seed_count": 3, + "test_metric": 0.6150355897115957, + "test_metric_std": 0.0013848240058531283, + "actual_test_call_fraction": 0.7057350008034863, + "actual_test_call_fraction_std": 0.006014701362190238, + "flops_gflops_per_input": 7125.364923966921, + "latency_ms_per_input": 1037.8223321215987 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.99, + "seed_count": 3, + "test_metric": 0.5069147075782926, + "test_metric_std": 9.357773432602366e-05, + "actual_test_call_fraction": 0.988051687793627, + "actual_test_call_fraction_std": 0.00048144255369636903, + "flops_gflops_per_input": 9972.73825167356, + "latency_ms_per_input": 1446.4565603014837 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 1.0, + "seed_count": 3, + "test_metric": 0.5000708958398321, + "test_metric_std": 0.0, + "actual_test_call_fraction": 1.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 10093.24584848, + "latency_ms_per_input": 1463.7509306362917 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.0, + "seed_count": 3, + "test_metric": 0.6695119530385957, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 7.503634432, + "latency_ms_per_input": 16.318855201956467 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.01, + "seed_count": 3, + "test_metric": 0.6742194368034484, + "test_metric_std": 0.0005415378002401663, + "actual_test_call_fraction": 0.04492905689627466, + "actual_test_call_fraction_std": 0.0027758676387843045, + "flops_gflops_per_input": 460.64652020812184, + "latency_ms_per_input": 81.35061327263864 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.05, + "seed_count": 3, + "test_metric": 0.679891103990018, + "test_metric_std": 0.0007177890142290943, + "actual_test_call_fraction": 0.10489748461560276, + "actual_test_call_fraction_std": 0.007840998690258661, + "flops_gflops_per_input": 1065.4726231670354, + "latency_ms_per_input": 168.15083906695963 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.1, + "seed_count": 3, + "test_metric": 0.680127423456125, + "test_metric_std": 0.0007907632127485806, + "actual_test_call_fraction": 0.16706840975904869, + "actual_test_call_fraction_std": 0.004046760924054268, + "flops_gflops_per_input": 1692.5125473727062, + "latency_ms_per_input": 258.1390302790103 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.25, + "seed_count": 3, + "test_metric": 0.6658726332605469, + "test_metric_std": 0.0048175982262131, + "actual_test_call_fraction": 0.3117620924670807, + "actual_test_call_fraction_std": 0.009206880232216604, + "flops_gflops_per_input": 3151.855731167172, + "latency_ms_per_input": 467.5733077433342 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.5, + "seed_count": 3, + "test_metric": 0.6273242019491629, + "test_metric_std": 0.009832598413097358, + "actual_test_call_fraction": 0.5310665570144345, + "actual_test_call_fraction_std": 0.009248673211785196, + "flops_gflops_per_input": 5363.704026981612, + "latency_ms_per_input": 785.0016240151261 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.75, + "seed_count": 3, + "test_metric": 0.5814404144098159, + "test_metric_std": 0.00858033252822142, + "actual_test_call_fraction": 0.7394719677849304, + "actual_test_call_fraction_std": 0.008502993761525655, + "flops_gflops_per_input": 7465.627276025616, + "latency_ms_per_input": 1086.65430025841 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.99, + "seed_count": 3, + "test_metric": 0.5100152189736173, + "test_metric_std": 0.0036056620177345906, + "actual_test_call_fraction": 0.978079006323909, + "actual_test_call_fraction_std": 0.009338895613008017, + "flops_gflops_per_input": 9872.15635718717, + "latency_ms_per_input": 1432.0217812641245 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "original", + "method_label": "原方法", + "nominal_budget": 1.0, + "seed_count": 3, + "test_metric": 0.5000708958398321, + "test_metric_std": 0.0, + "actual_test_call_fraction": 1.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 10093.24584848, + "latency_ms_per_input": 1463.7509306362917 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.0, + "seed_count": 1, + "test_metric": 0.6695119530385957, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 7.503634432, + "latency_ms_per_input": 16.318855201956467 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.01, + "seed_count": 1, + "test_metric": 0.6686044862887446, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.002467175226157729, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 32.38692775991243, + "latency_ms_per_input": 19.889923760014124 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.05, + "seed_count": 1, + "test_metric": 0.6662791027422511, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.012534384482318578, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 133.9222051324287, + "latency_ms_per_input": 34.46152534749078 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.1, + "seed_count": 1, + "test_metric": 0.6645208859144145, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.0345971698380739, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 356.4417707544502, + "latency_ms_per_input": 66.39590854483396 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.25, + "seed_count": 1, + "test_metric": 0.6596999688058305, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.12477667810452883, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 1265.9690441195253, + "latency_ms_per_input": 196.92462135659662 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.5, + "seed_count": 1, + "test_metric": 0.6527521765022829, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.2345234381646485, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 2372.8465749128713, + "latency_ms_per_input": 355.77560204260965 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.75, + "seed_count": 1, + "test_metric": 0.567762243711539, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.7478093185491875, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 7549.715646582009, + "latency_ms_per_input": 1098.722049178743 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.99, + "seed_count": 1, + "test_metric": 0.5019425460114001, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.9908118991577575, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 10000.577031948465, + "latency_ms_per_input": 1450.4517787649047 + }, + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "pair_label": "gRefCOCO / InstanceVG + MiniCPM", + "task": "grefcoco", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 1.0, + "seed_count": 1, + "test_metric": 0.5000708958398321, + "test_metric_std": 0.0, + "actual_test_call_fraction": 1.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 10093.24584848, + "latency_ms_per_input": 1463.7509306362917 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.0, + "seed_count": 3, + "test_metric": 0.18129624718318374, + "test_metric_std": 2.7755575615628914e-17, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 194.418697728, + "latency_ms_per_input": 31.79686254823173 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.01, + "seed_count": 3, + "test_metric": 0.1834834493716783, + "test_metric_std": 0.0012311365629078506, + "actual_test_call_fraction": 0.01187305814469596, + "actual_test_call_fraction_std": 0.0025254786233335643, + "flops_gflops_per_input": 1318.8138164976065, + "latency_ms_per_input": 62.665281079596 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.05, + "seed_count": 3, + "test_metric": 0.19244312207391248, + "test_metric_std": 0.0014149660525805208, + "actual_test_call_fraction": 0.04804704837993786, + "actual_test_call_fraction_std": 0.0025254786233335656, + "flops_gflops_per_input": 4744.541000786314, + "latency_ms_per_input": 156.71298613777122 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.1, + "seed_count": 3, + "test_metric": 0.2058996264309502, + "test_metric_std": 0.000756529123691676, + "actual_test_call_fraction": 0.10707944962272525, + "actual_test_call_fraction_std": 0.0037002512207130658, + "flops_gflops_per_input": 10334.991497846413, + "latency_ms_per_input": 310.1896091161431 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.25, + "seed_count": 3, + "test_metric": 0.25617136079673564, + "test_metric_std": 0.0020077364088740926, + "actual_test_call_fraction": 0.26498002663115844, + "actual_test_call_fraction_std": 0.008293336414138841, + "flops_gflops_per_input": 25288.39574092258, + "latency_ms_per_input": 720.7107265940061 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.5, + "seed_count": 3, + "test_metric": 0.29980106360416753, + "test_metric_std": 0.002221016752728669, + "actual_test_call_fraction": 0.49511762094984463, + "actual_test_call_fraction_std": 0.00440231567351916, + "flops_gflops_per_input": 47082.74598698018, + "latency_ms_per_input": 1319.038764295403 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.75, + "seed_count": 3, + "test_metric": 0.28788636673929996, + "test_metric_std": 0.0011284209510194398, + "actual_test_call_fraction": 0.7496671105193076, + "actual_test_call_fraction_std": 0.005429263125932131, + "flops_gflops_per_input": 71188.93666415288, + "latency_ms_per_input": 1980.8345784164278 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 0.99, + "seed_count": 3, + "test_metric": 0.27239176103160984, + "test_metric_std": 0.0005578690368974412, + "actual_test_call_fraction": 0.9862405681313805, + "actual_test_call_fraction_std": 0.0023116585281845767, + "flops_gflops_per_input": 93592.7721147772, + "latency_ms_per_input": 2595.8950299011817 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "both", + "method_label": "两项同时改", + "nominal_budget": 1.0, + "seed_count": 3, + "test_metric": 0.27001009067886245, + "test_metric_std": 0.0, + "actual_test_call_fraction": 1.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 94895.80944867841, + "latency_ms_per_input": 2631.667776610426 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.0, + "seed_count": 3, + "test_metric": 0.18129624718318374, + "test_metric_std": 2.7755575615628914e-17, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 194.418697728, + "latency_ms_per_input": 31.79686254823173 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.01, + "seed_count": 3, + "test_metric": 0.18300903853620068, + "test_metric_std": 0.000992883110506064, + "actual_test_call_fraction": 0.010319573901464712, + "actual_test_call_fraction_std": 0.0032954377285658006, + "flops_gflops_per_input": 1171.6966981539194, + "latency_ms_per_input": 58.626422580165155 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.05, + "seed_count": 3, + "test_metric": 0.19493527207894204, + "test_metric_std": 0.0032238119689520237, + "actual_test_call_fraction": 0.049822458943630715, + "actual_test_call_fraction_std": 0.011397380483895993, + "flops_gflops_per_input": 4912.674850321956, + "latency_ms_per_input": 161.32882442283505 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.1, + "seed_count": 3, + "test_metric": 0.20346296196880112, + "test_metric_std": 0.004110692938093332, + "actual_test_call_fraction": 0.09576120727918332, + "actual_test_call_fraction_std": 0.006259345218007303, + "flops_gflops_per_input": 9263.138207056696, + "latency_ms_per_input": 280.76364004886125 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.25, + "seed_count": 3, + "test_metric": 0.24461408052105227, + "test_metric_std": 0.0022654085644492608, + "actual_test_call_fraction": 0.2533288948069241, + "actual_test_call_fraction_std": 0.0017823322685540256, + "flops_gflops_per_input": 24185.01735334493, + "latency_ms_per_input": 690.419287848275 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.5, + "seed_count": 3, + "test_metric": 0.29849806777867033, + "test_metric_std": 0.0015269090065487179, + "actual_test_call_fraction": 0.4957833999112295, + "actual_test_call_fraction_std": 0.00518804692745715, + "flops_gflops_per_input": 47145.796180556055, + "latency_ms_per_input": 1320.7697036523023 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.75, + "seed_count": 3, + "test_metric": 0.28532383356176916, + "test_metric_std": 0.0029913445567370696, + "actual_test_call_fraction": 0.7654238792720817, + "actual_test_call_fraction_std": 0.006553405708048556, + "flops_gflops_per_input": 72681.1245787817, + "latency_ms_per_input": 2021.8001431963692 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 0.99, + "seed_count": 3, + "test_metric": 0.27053969734098926, + "test_metric_std": 0.0014164533171887024, + "actual_test_call_fraction": 0.9897913892587661, + "actual_test_call_fraction_std": 0.0027895705925380728, + "flops_gflops_per_input": 93929.03981384847, + "latency_ms_per_input": 2605.126706471309 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "original", + "method_label": "原方法", + "nominal_budget": 1.0, + "seed_count": 3, + "test_metric": 0.27001009067886245, + "test_metric_std": 0.0, + "actual_test_call_fraction": 1.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 94895.80944867841, + "latency_ms_per_input": 2631.667776610426 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.0, + "seed_count": 1, + "test_metric": 0.18129624718318377, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 194.418697728, + "latency_ms_per_input": 31.79686254823173 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.01, + "seed_count": 1, + "test_metric": 0.18129624718318377, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 194.418697728, + "latency_ms_per_input": 31.79686254823173 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.05, + "seed_count": 1, + "test_metric": 0.18129624718318377, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 194.418697728, + "latency_ms_per_input": 31.79686254823173 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.1, + "seed_count": 1, + "test_metric": 0.18129624718318377, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 194.418697728, + "latency_ms_per_input": 31.79686254823173 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.25, + "seed_count": 1, + "test_metric": 0.18026802885144627, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.24800266311584554, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 23680.615804738005, + "latency_ms_per_input": 676.5717729930834 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.5, + "seed_count": 1, + "test_metric": 0.18015785955296285, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.5013315579227696, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 47671.21446035493, + "latency_ms_per_input": 1335.1941982931266 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.75, + "seed_count": 1, + "test_metric": 0.22103445955201576, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.7486684420772304, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 71094.36137378908, + "latency_ms_per_input": 1978.2381693810794 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 0.99, + "seed_count": 1, + "test_metric": 0.2649306125590882, + "test_metric_std": 0.0, + "actual_test_call_fraction": 0.987017310252996, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 93666.33067394902, + "latency_ms_per_input": 2597.9144591508966 + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "pair_label": "Construction / YOLO26x + Qwen3.8", + "task": "construction", + "method": "confidence", + "method_label": "置信度路由", + "nominal_budget": 1.0, + "seed_count": 1, + "test_metric": 0.27001009067886245, + "test_metric_std": 0.0, + "actual_test_call_fraction": 1.0, + "actual_test_call_fraction_std": 0.0, + "flops_gflops_per_input": 94895.80944867841, + "latency_ms_per_input": 2631.667776610426 + } + ], + "comparisons": { + "grefcoco|instancevg|MiniCPM": { + "both_vs_confidence_metric_delta_percentage_points": 6.175500288309744, + "both_vs_confidence_call_delta_percentage_points": -40.80197374018093, + "both_vs_confidence_flops_relative_percent": -78.21165529717908, + "both_vs_confidence_latency_relative_percent": -76.66417123634449 + }, + "construction|yolo26x|Qwen3.8": { + "both_vs_confidence_metric_delta_percentage_points": 4.1333041716827665, + "both_vs_confidence_call_delta_percentage_points": -48.069241011984026, + "both_vs_confidence_flops_relative_percent": -52.01415993620476, + "both_vs_confidence_latency_relative_percent": -51.44759733081112 + } + } +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/fixed_budget_cost_performance.csv b/releases/20260913_lr/paper_data/cost_20260911/fixed_budget_cost_performance.csv new file mode 100644 index 0000000000000000000000000000000000000000..170b88740875639fc901833b21cbbe4e834b3814 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/fixed_budget_cost_performance.csv @@ -0,0 +1,55 @@ +pair_key,pair_label,task,method,method_label,nominal_budget,seed_count,test_metric,test_metric_std,actual_test_call_fraction,actual_test_call_fraction_std,flops_gflops_per_input,latency_ms_per_input +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.0,3,0.6695119530385957,0.0,0.0,0.0,7.503634432,16.318855201956467 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.01,3,0.6743328701471798,0.0002504540415259137,0.021268751949635595,0.0024468276041848553,222.01478381055546,47.10392897831558 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.05,3,0.6805811568310505,0.0004271568320976906,0.11976670542305912,0.0032185843726325893,1215.4397511547988,189.67302620038757 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.1,3,0.6804866290446077,0.0005385595679416328,0.172031118547297,0.0033259813564282133,1742.5651488943693,265.32221416016074 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.25,3,0.6794184650578038,0.00038210745789084077,0.24530905859777485,0.009454413833556307,2481.627562219953,371.38705501097667 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.5,3,0.664568149807636,0.0016062493473305785,0.4310372534006371,0.0088737075287505,4354.834256882111,640.2160014811561 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.75,3,0.6150355897115957,0.0013848240058531283,0.7057350008034863,0.006014701362190238,7125.364923966921,1037.8223321215987 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.99,3,0.5069147075782926,9.357773432602366e-05,0.988051687793627,0.00048144255369636903,9972.73825167356,1446.4565603014837 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,1.0,3,0.5000708958398321,0.0,1.0,0.0,10093.24584848,1463.7509306362917 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.0,3,0.6695119530385957,0.0,0.0,0.0,7.503634432,16.318855201956467 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.01,3,0.6742194368034484,0.0005415378002401663,0.04492905689627466,0.0027758676387843045,460.64652020812184,81.35061327263864 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.05,3,0.679891103990018,0.0007177890142290943,0.10489748461560276,0.007840998690258661,1065.4726231670354,168.15083906695963 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.1,3,0.680127423456125,0.0007907632127485806,0.16706840975904869,0.004046760924054268,1692.5125473727062,258.1390302790103 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.25,3,0.6658726332605469,0.0048175982262131,0.3117620924670807,0.009206880232216604,3151.855731167172,467.5733077433342 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.5,3,0.6273242019491629,0.009832598413097358,0.5310665570144345,0.009248673211785196,5363.704026981612,785.0016240151261 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.75,3,0.5814404144098159,0.00858033252822142,0.7394719677849304,0.008502993761525655,7465.627276025616,1086.65430025841 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.99,3,0.5100152189736173,0.0036056620177345906,0.978079006323909,0.009338895613008017,9872.15635718717,1432.0217812641245 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,1.0,3,0.5000708958398321,0.0,1.0,0.0,10093.24584848,1463.7509306362917 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.0,1,0.6695119530385957,0.0,0.0,0.0,7.503634432,16.318855201956467 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.01,1,0.6686044862887446,0.0,0.002467175226157729,0.0,32.38692775991243,19.889923760014124 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.05,1,0.6662791027422511,0.0,0.012534384482318578,0.0,133.9222051324287,34.46152534749078 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.1,1,0.6645208859144145,0.0,0.0345971698380739,0.0,356.4417707544502,66.39590854483396 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.25,1,0.6596999688058305,0.0,0.12477667810452883,0.0,1265.9690441195253,196.92462135659662 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.5,1,0.6527521765022829,0.0,0.2345234381646485,0.0,2372.8465749128713,355.77560204260965 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.75,1,0.567762243711539,0.0,0.7478093185491875,0.0,7549.715646582009,1098.722049178743 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.99,1,0.5019425460114001,0.0,0.9908118991577575,0.0,10000.577031948465,1450.4517787649047 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,1.0,1,0.5000708958398321,0.0,1.0,0.0,10093.24584848,1463.7509306362917 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.0,3,0.18129624718318374,2.7755575615628914e-17,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.01,3,0.1834834493716783,0.0012311365629078506,0.01187305814469596,0.0025254786233335643,1318.8138164976065,62.665281079596 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.05,3,0.19244312207391248,0.0014149660525805208,0.04804704837993786,0.0025254786233335656,4744.541000786314,156.71298613777122 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.1,3,0.2058996264309502,0.000756529123691676,0.10707944962272525,0.0037002512207130658,10334.991497846413,310.1896091161431 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.25,3,0.25617136079673564,0.0020077364088740926,0.26498002663115844,0.008293336414138841,25288.39574092258,720.7107265940061 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.5,3,0.29980106360416753,0.002221016752728669,0.49511762094984463,0.00440231567351916,47082.74598698018,1319.038764295403 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.75,3,0.28788636673929996,0.0011284209510194398,0.7496671105193076,0.005429263125932131,71188.93666415288,1980.8345784164278 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.99,3,0.27239176103160984,0.0005578690368974412,0.9862405681313805,0.0023116585281845767,93592.7721147772,2595.8950299011817 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,1.0,3,0.27001009067886245,0.0,1.0,0.0,94895.80944867841,2631.667776610426 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.0,3,0.18129624718318374,2.7755575615628914e-17,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.01,3,0.18300903853620068,0.000992883110506064,0.010319573901464712,0.0032954377285658006,1171.6966981539194,58.626422580165155 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.05,3,0.19493527207894204,0.0032238119689520237,0.049822458943630715,0.011397380483895993,4912.674850321956,161.32882442283505 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.1,3,0.20346296196880112,0.004110692938093332,0.09576120727918332,0.006259345218007303,9263.138207056696,280.76364004886125 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.25,3,0.24461408052105227,0.0022654085644492608,0.2533288948069241,0.0017823322685540256,24185.01735334493,690.419287848275 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.5,3,0.29849806777867033,0.0015269090065487179,0.4957833999112295,0.00518804692745715,47145.796180556055,1320.7697036523023 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.75,3,0.28532383356176916,0.0029913445567370696,0.7654238792720817,0.006553405708048556,72681.1245787817,2021.8001431963692 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.99,3,0.27053969734098926,0.0014164533171887024,0.9897913892587661,0.0027895705925380728,93929.03981384847,2605.126706471309 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,1.0,3,0.27001009067886245,0.0,1.0,0.0,94895.80944867841,2631.667776610426 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.0,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.01,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.05,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.1,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.25,1,0.18026802885144627,0.0,0.24800266311584554,0.0,23680.615804738005,676.5717729930834 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.5,1,0.18015785955296285,0.0,0.5013315579227696,0.0,47671.21446035493,1335.1941982931266 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.75,1,0.22103445955201576,0.0,0.7486684420772304,0.0,71094.36137378908,1978.2381693810794 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.99,1,0.2649306125590882,0.0,0.987017310252996,0.0,93666.33067394902,2597.9144591508966 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,1.0,1,0.27001009067886245,0.0,1.0,0.0,94895.80944867841,2631.667776610426 diff --git a/releases/20260913_lr/paper_data/cost_20260911/full_cost_performance_curves.csv b/releases/20260913_lr/paper_data/cost_20260911/full_cost_performance_curves.csv new file mode 100644 index 0000000000000000000000000000000000000000..b3d0c5c8a11be3acb61ff79972ca9c824ce73f90 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/full_cost_performance_curves.csv @@ -0,0 +1,607 @@ +pair_key,pair_label,task,method,method_label,nominal_budget,seed_count,test_metric,test_metric_std,actual_test_call_fraction,actual_test_call_fraction_std,flops_gflops_per_input,latency_ms_per_input +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.0,3,0.6695119530385957,0.0,0.0,0.0,7.503634432,16.318855201956467 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.01,3,0.6743328701471798,0.0002504540415259137,0.021268751949635595,0.0024468276041848553,222.01478381055546,47.10392897831558 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.02,3,0.6771403454045317,0.0005474456601563245,0.041365359347380164,0.003542736992048041,424.70398540113723,76.19240313322202 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.03,3,0.679257767820851,0.0002151415870599417,0.06490277817164355,0.00620057607312886,662.0963240467386,110.26121811239277 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.04,3,0.6804582707086748,0.00033393872203453485,0.09923527020767754,0.005261539209768673,1008.3649882880334,159.95516831494223 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.05,3,0.6805811568310505,0.0004271568320976906,0.11976670542305912,0.0032185843726325893,1215.4397511547988,189.67302620038757 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.06,3,0.6805811568310505,0.00040657938287739023,0.13569463743867508,0.004358151286070409,1376.0847674671834,212.72762589512757 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.07,3,0.6804866290446077,0.00044175940581907595,0.14781309966064524,0.003803002168080138,1498.3084534686539,230.26827682014644 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.08,3,0.6805433457164733,0.0005109749312968245,0.15775742279443042,0.0039413666334665,1598.604333089205,244.6620090924708 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.09,3,0.6805338929378291,0.0005230727327891268,0.16548034294680922,0.0037416051429164958,1676.495714885774,255.8404114370421 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.1,3,0.6804866290446077,0.0005385595679416328,0.172031118547297,0.0033259813564282133,1742.5651488943693,265.32221416016074 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.11,3,0.6804488179300305,0.000551024837810731,0.17804308576506064,0.003015095594116757,1803.200300452041,274.02412834761157 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.12,3,0.6804488179300305,0.0005748346537167685,0.18348788626416734,0.0026700376856177714,1858.115154692951,281.9051072343595 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.13,3,0.6804204595940977,0.0005158480995053015,0.1884222367164828,0.00207569760609485,1907.881741348776,289.0472443504748 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.14,3,0.6803542901435877,0.0006083654410081598,0.1931864371532012,0.0011580329018036773,1955.9322388095723,295.9431008763792 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.15,3,0.680363742922232,0.0005766969738883526,0.19768595978787964,0.0009296460579564395,2001.3132641892132,302.45585426195566 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.16,3,0.680297573471722,0.0005291867775393541,0.2019208046205182,0.001092522041836435,2044.024817487699,308.58550450720406 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.17,3,0.680193592906635,0.0004758420232217985,0.20723326621860494,0.002194470247856415,2097.6049356880317,316.27493182378794 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.18,3,0.6801179706774807,0.0004435761074081306,0.21170443051735055,0.0030486933096992973,2142.6999462018343,322.7466384443292 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.19,3,0.6800234428910379,0.0004383073143472105,0.2164969892900018,0.00403774197749049,2191.036458528469,329.6835417352688 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.2,3,0.6798816512113736,0.00040260404085582066,0.221355717513163,0.004883419260253461,2240.0403388753925,336.7162208112904 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.21,3,0.6797776706462865,0.00045609044727959696,0.22576071236139864,0.006488174074638593,2284.4679813689067,343.0921516467497 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.22,3,0.6795697095161123,0.00042965972558070103,0.23054381835540558,0.007590304608337386,2332.7091554069284,350.01537268267754 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.23,3,0.679560256737468,0.00035369059039917175,0.2358090160602709,0.008359420906117757,2385.8125821641975,357.6363887242029 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.24,3,0.6795035400656023,0.0003945332077003918,0.24052595260376786,0.009161288573368712,2433.38638818193,364.4638339750487 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.25,3,0.6794184650578038,0.00038210745789084077,0.24530905859777485,0.009454413833556307,2481.627562219953,371.38705501097667 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.26,3,0.6792199567062739,0.0004769673902406576,0.249666789552789,0.01028280716757621,2525.5785132704027,377.6945745713773 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.27,3,0.6791065233625425,0.00046328260045348395,0.25479964835663443,0.011050682323431758,2577.3472039870944,385.1240390427387 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.28,3,0.6788229400032141,0.00046328260045348395,0.2601215627333655,0.011400435106655959,2631.02266047604,392.82714861433436 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.29,3,0.678321942735067,0.0005327208992846384,0.2659255688209549,0.012015377815299292,2689.5603696842322,401.2280531915274 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.3,3,0.6781423399408256,0.00058865846078289,0.27142708599192733,0.012355349420345482,2745.0472536568186,409.19112560834566 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.31,3,0.677924926032007,0.000523072732789084,0.27730671430867104,0.012028979317988072,2804.3476691739124,417.7014882256325 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.32,3,0.6774711926570816,0.0006136302400861127,0.28365898155762886,0.01241046520670959,2868.414999121641,426.89596359350503 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.33,3,0.6768662148238475,0.0005813266733411556,0.29009632381438527,0.013141762694036838,2933.3403736668843,436.2135792564831 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.34,3,0.6765637259072305,0.00058865846078289,0.2966849105294502,0.013581856831012882,2999.79116082993,445.7501109996486 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.35,3,0.6760721814177278,0.0007381645730786406,0.30406753065063474,0.014055293304190442,3074.2503642364413,456.43595216379805 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.36,3,0.6758642202875534,0.0010195863623812392,0.3104859673501026,0.014256325875966572,3138.985062204459,465.72620331675273 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.37,3,0.6754010341339837,0.0010440935274166046,0.31819943472383705,0.014529772546897555,3216.781105712415,476.8909234063122 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.38,3,0.6746920757356625,0.0011032652166457105,0.32562931873824313,0.014599179327613425,3291.7170005619905,487.6451758455204 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.39,3,0.6740776451237842,0.0013327412235750052,0.33336169166926616,0.014639885449032641,3369.703720647172,498.8372604451033 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.4,3,0.6736428173061472,0.001351780397401897,0.3405458034389209,0.014481317516906796,3442.1608199928173,509.2357742540069 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.41,3,0.6731134617020674,0.0013643474125036833,0.348580665286561,0.014614442124489846,3523.1983653136053,520.8656910139648 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.42,3,0.6722532588454376,0.001422509710237129,0.35671005492064395,0.014547989099783849,3605.1892935205196,532.6324303240399 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.43,3,0.6714592254393178,0.001458242388452423,0.3657374585259337,0.013594877008401294,3696.23735914564,545.6989838602278 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.44,3,0.6707408142623524,0.0014262736338482374,0.3746608815661363,0.012079460205687865,3786.2367035960197,558.6150325912869 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.45,3,0.6699373280775883,0.0014342703248619284,0.3841420185463517,0.0118463419778686,3881.86100707455,572.3383343680373 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.46,3,0.6692472752365558,0.0014790502695860163,0.39315996937299724,0.01098682111367372,3972.813734411058,585.3912056492136 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.47,3,0.6682358279216176,0.0015388589437112046,0.4021401090850656,0.010223755863623863,4063.385108593114,598.389347910343 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.48,3,0.6672338333853235,0.001474694061998076,0.41078940154458404,0.010181612407033868,4150.619642673726,610.9086112460623 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.49,3,0.665976613825634,0.0015931788394659891,0.42075263023565773,0.009642362446211063,4251.106198871502,625.32970802841 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.5,3,0.664568149807636,0.0016062493473305785,0.4310372534006371,0.0088737075287505,4354.834256882111,640.2160014811561 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.51,3,0.6633676469198121,0.0020042120030806976,0.44035769314389966,0.009190035929571216,4448.837809454225,653.7067049227073 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.52,3,0.6618646551153712,0.0020936997808162646,0.45062341075159046,0.009219595028175877,4552.3751908876075,668.56563386543 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.53,3,0.6604845494333059,0.002308994774316215,0.46022743385418147,0.0075127649438487005,4649.238892118102,682.4668049573326 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.54,3,0.6591611604231064,0.0022260842136846678,0.470351359782208,0.006670581494901897,4751.346199222294,697.1205000748795 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.55,3,0.6576014519467998,0.0024106253622611174,0.4805320023821002,0.006491808860058663,4854.025536058162,711.8562887224967 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.56,3,0.656334779608466,0.0024757258577969956,0.4908355311043681,0.005158636890377099,4957.944270645996,726.7699466852663 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.57,3,0.6546143738952065,0.0023265744453588003,0.5011485126052804,0.005109749312968089,5061.958343522443,741.6972869030476 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.58,3,0.6530074015256785,0.002758786605113626,0.5124067719706208,0.005671525393117397,5175.506245260158,757.9928526220003 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.59,3,0.6513342597056405,0.0026360544008023792,0.5232112979610356,0.00454624605624975,5284.477909144463,773.6316701003906 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.6,3,0.6497650984506896,0.00287911216792696,0.5344128406545104,0.004390019397162448,5397.453781150503,789.8451422892732 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.61,3,0.6485835011201543,0.002901156172953934,0.5444611443533827,0.005016692132993845,5498.798381945794,804.3893793667263 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.62,3,0.6468252842923177,0.003072138516620979,0.5550482564349791,0.004845948325466897,5605.577265192009,819.7135049798475 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.63,3,0.6448023896624413,0.00259188916077078,0.5661269130060781,0.004568030217447332,5717.313739446082,835.7491078535775 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.64,3,0.6427794950325648,0.0027303971085773995,0.5773095501422643,0.003800463761860775,5830.098934874895,851.9352155324365 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.65,3,0.6406715253948898,0.0028274081038884478,0.5881046233540349,0.00425603944707499,5938.975260470589,867.5603507558153 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.66,3,0.6382894251765306,0.002835928119495768,0.5990414882454699,0.004466415402750962,6049.281660395474,883.3907198043698 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.67,3,0.6364650388981841,0.0026804586879219344,0.6101295975952131,0.005978746116874079,6161.11347293816,899.4400049331116 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.68,3,0.6344421442683077,0.0025285431940637567,0.6208868596924065,0.006286872452124139,6269.608445379403,915.0104111364434 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.69,3,0.6319938745994385,0.0026412355899017264,0.6330242274716653,0.0058356803797160505,6392.022807958099,932.5784265714857 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.7,3,0.6292242104566638,0.0025190901336351124,0.6451426896936354,0.0062416404627669364,6514.246493959568,950.1190774965045 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.71,3,0.6266341491081304,0.002463859092390738,0.6572706046942499,0.006014701362190216,6636.565518249652,967.6734106765351 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.72,3,0.6240629933168855,0.001985668601086228,0.668954239098583,0.006375364870457525,6754.403642974938,984.584677871015 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.73,3,0.621132631937158,0.0018631807980439294,0.6817911124975188,0.005965280050983444,6883.873038910974,1003.1651801769244 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.74,3,0.6183157039011618,0.0016373238631933423,0.6936732552533816,0.005451049532285977,7003.713267697125,1020.36377472665 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.75,3,0.6150355897115957,0.0013848240058531283,0.7057350008034863,0.006014701362190238,7125.364923966921,1037.8223321215987 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.76,3,0.6122564727901766,0.001265896126137565,0.7173997296505309,0.005155570083727779,7243.012372114983,1054.7062348060554 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.77,3,0.608418644660598,0.0011947205342415093,0.7293953057501253,0.005822343836658303,7363.996660364488,1072.0690164159219 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.78,3,0.6055166416168033,0.001000030714379607,0.7408615262456398,0.0056449474110625755,7479.642004451681,1088.6655917451321 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.79,3,0.6020663774116403,0.000768063714214944,0.7519212772594503,0.005908659652184152,7591.187802128529,1104.673830108839 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.8,3,0.5983703409617257,0.0007836119738748615,0.7647770562156747,0.005375149505067936,7720.847874641791,1123.2816969247717 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.81,3,0.5947215684050327,0.000990333653117727,0.7770278573386648,0.005883714995308605,7844.406296683838,1141.0138994199547 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.82,3,0.5916021514524195,0.0006166806163325103,0.7887682084148636,0.004924906745427358,7962.816451140801,1158.007260144505 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.83,3,0.5881046233540349,0.0005558683956928728,0.800527465048351,0.004752764604088509,8081.41728217499,1175.0279853790785 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.84,3,0.5840493813156378,0.0002497394778970151,0.81213547722353,0.00399710608049565,8198.492700591376,1191.8297945334648 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.85,3,0.5802871754152134,0.0006021650004682148,0.8231952282373403,0.00311901589985948,8310.03849826822,1207.8380328971712 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.86,3,0.5761468583690176,0.0010293550775858228,0.8354554821389747,0.002533521028340118,8433.692258598883,1225.5839176473658 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.87,3,0.572479180255036,0.0008323814371154528,0.8468555331839794,0.0031484423058575906,8548.67023466579,1242.0847171914945 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.88,3,0.5682348826437532,0.0011296762607777792,0.8575655313879516,0.0023733440561952085,8656.68851566397,1257.5867121197678 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.89,3,0.5638582461314504,0.0014653941241585177,0.8685591129512521,0.002135742322974502,8767.566945320528,1273.4991746983926 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.9,3,0.5592736484889733,0.0015491601715829182,0.8805546890508466,0.0021221435423951097,8888.551233570033,1290.861956308259 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.91,3,0.5544621841590335,0.0012354604492409806,0.8920020039890725,0.0015331579383471344,9004.005901080001,1307.4311671274459 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.92,3,0.5496696253863823,0.0007570489578377954,0.9039975800886669,0.0013462164011598249,9124.990189329506,1324.7939487373123 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.93,3,0.5436103942753973,0.0001675036643381324,0.917515053549991,0.0021790243850193644,9261.323942045656,1344.3595734040653 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.94,3,0.5380427076539148,9.261793731991073e-05,0.929728043558404,0.0024346364982378798,9384.501010933254,1362.0370468792012 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.95,3,0.5325411904829425,0.0007963930945495145,0.9418937696735955,0.0008472769698284989,9507.201388377787,1379.6461090792786 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.96,3,0.5270207677546815,0.0009698200779801628,0.9528778984582518,0.0010717133384227106,9617.984479745734,1395.5448894028916 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.97,3,0.5204983504901266,0.000893575120142663,0.9650719829093762,0.0010717133384227106,9740.970872056107,1413.1949983680042 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.98,3,0.5137963304313303,0.0006302955157216679,0.9764342228398037,0.0008143666182094168,9855.567494968562,1429.6410688920857 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,0.99,3,0.5069147075782926,9.357773432602366e-05,0.988051687793627,0.00048144255369636903,9972.73825167356,1446.4565603014837 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,both,两项同时改,1.0,3,0.5000708958398321,0.0,1.0,0.0,10093.24584848,1463.7509306362917 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.0,3,0.6695119530385957,0.0,0.0,0.0,7.503634432,16.318855201956467 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.01,3,0.6742194368034484,0.0005415378002401663,0.04492905689627466,0.0027758676387843045,460.64652020812184,81.35061327263864 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.02,3,0.6771970620763973,0.0011912751540770635,0.06431670589569805,0.005721922375358464,656.1853501527518,109.41291830166644 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.03,3,0.6786338844303282,0.00135336590911576,0.07992324343740843,0.007672920053500877,813.5888646523051,132.00232132600817 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.04,3,0.679626426187978,0.001070461969913116,0.0931665863180482,0.008142261602234385,947.1578069986856,151.1711605974211 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.05,3,0.679891103990018,0.0007177890142290943,0.10489748461560276,0.007840998690258661,1065.4726231670354,168.15083906695963 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.06,3,0.6803259318076549,0.0005137652588341501,0.11747913299114275,0.006737053101355148,1192.3678853105275,186.3619204875525 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.07,3,0.6808174762971575,0.0005367314018122495,0.13036327028329978,0.005832586567250194,1322.3139726896259,205.01083406852027 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.08,3,0.6805433457164733,0.0006225930408671896,0.14230212971102854,0.005903076729771085,1442.7262312074547,222.2915221483165 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.09,3,0.680363742922232,0.0006775736962238823,0.15481760863605856,0.00505475350641574,1568.9541253306581,240.40682778382737 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.1,3,0.680127423456125,0.0007907632127485806,0.16706840975904869,0.004046760924054268,1692.5125473727062,258.1390302790103 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.11,3,0.6798343873181523,0.0008798704045388216,0.17844955524676478,0.004109339347644479,1807.2998468623864,274.6124653131153 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.12,3,0.6795129928442466,0.0009377804365471553,0.18981179517719235,0.004338382597917462,1921.8964697748415,291.0585358371969 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.13,3,0.6786716955449054,0.0011238872811125834,0.20089045174829143,0.004868612323082673,2033.6329440289157,307.0941387109271 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.14,3,0.6780667177116714,0.00145732296103388,0.21069298320241236,0.005245721543421001,2132.498749320277,321.2826371580757 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.15,3,0.6772065148550416,0.001465394124158557,0.22071292856535182,0.007056725601839535,2233.55733524973,335.7858274704938 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.16,3,0.6763935758916334,0.0013835329147595995,0.22986321829301723,0.007171812629403647,2325.8447986268147,349.03025032183405 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.17,3,0.6753915813553394,0.0015173402884336763,0.23860703853897855,0.007874669574867588,2414.0327155935543,361.6863362076706 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.18,3,0.6739264006654757,0.0019317464938165153,0.24883494503209222,0.00780619823882189,2517.1887438724866,376.49053613034647 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.19,3,0.6731512728166443,0.002259233871584527,0.256992693002108,0.008544172047533749,2599.4656869452388,388.2983222054566 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.2,3,0.6717428087986463,0.003125197842375181,0.26705990225826876,0.009143363531876508,2701.0009643177546,402.86992379293315 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.21,3,0.6707786253769296,0.0030076180790647306,0.2755390447021902,0.00934283681814668,2786.5194092033394,415.1429065384417 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.22,3,0.6695686697104614,0.0034814982850518003,0.28362117044305174,0.009380119469322839,2868.0336459671903,426.8412345734582 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.23,3,0.6684626946090804,0.0037012426905678997,0.2929794213008914,0.009999342092610693,2962.4185516937546,440.3866670350562 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.24,3,0.667148758377525,0.004307626146057513,0.3025834444034824,0.009073249470922487,3059.282252924249,454.2878381269588 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.25,3,0.6658726332605469,0.0048175982262131,0.3117620924670807,0.009206880232216604,3151.855731167172,467.5733077433342 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.26,3,0.6642845664483076,0.005205400677898473,0.32189547117375156,0.009123660155824689,3254.0583765599777,482.2406851158929 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.27,3,0.6627532163079337,0.0054874825203285495,0.33114974146650406,0.00947656027835034,3347.394561111802,495.635612772362 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.28,3,0.661127338381117,0.005933574738293955,0.3402716728582367,0.008895752791723036,3439.3960096230494,508.83898885866716 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.29,3,0.6596527049126091,0.006356162512517477,0.34870355140893666,0.009462972695712816,3524.43776306557,521.0435603291171 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.3,3,0.6582347881159667,0.00679647553569934,0.3571354299596366,0.0098068284728935,3609.47951650809,533.248131799567 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.31,3,0.657109907457297,0.006940837493572802,0.3657941751977994,0.009915122212193869,3696.8093888773155,545.781077390298 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.32,3,0.6557770656684533,0.007285842201549398,0.37517133161292765,0.009565950445108471,3791.384971181106,559.3538743619196 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.33,3,0.6542646210853681,0.007736936190344077,0.38488878805925,0.011016190526590496,3889.392731874952,573.4192325139627 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.34,3,0.6526576487158401,0.008230282001722029,0.3938689277713184,0.011599726490191186,3979.964106057009,586.4173747750921 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.35,3,0.6512113735832648,0.008659787457800234,0.4029719536057624,0.011492114314915383,4071.7748779910303,599.5933863513737 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.36,3,0.6496894762215354,0.008775916115160294,0.41243418502868917,0.011254135420690394,4167.2085048923345,613.2893236181006 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.37,3,0.6481675788598058,0.009146998240623951,0.4212347219465162,0.011367578473858324,4255.96845159075,626.0275030340075 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.38,3,0.6462203064590836,0.009393541527385984,0.4305173505752016,0.012010022181898157,4349.590651008412,639.4634774555118 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.39,3,0.6448496535556627,0.009432982703099675,0.4394029625008271,0.011652534976922176,4439.208642304343,652.324797166524 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.4,3,0.6437247728969929,0.009427098373150046,0.4468517520725217,0.011414064837074227,4514.335213731143,663.1064141157556 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.41,3,0.641947650511868,0.00963591050967979,0.45572791121950296,0.011504245208283616,4603.85786673846,675.9540515717562 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.42,3,0.6401799809053871,0.009651347174920779,0.4641314314342701,0.011176142957454109,4688.613605315143,688.1175762771709 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.43,3,0.6385919140931478,0.009629630556149547,0.4725349516490372,0.011309916840396294,4773.369343891825,700.2811009825856 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.44,3,0.6370416583954853,0.009636188699480854,0.4808723024132944,0.010511039305674751,4857.457714448219,712.3488499029185 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.45,3,0.6352456304530717,0.009815598936038833,0.4893892559717929,0.010125854252442038,4943.3575124882545,724.6765616684738 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.46,3,0.6336670164194765,0.00999477470733724,0.49671515942111183,0.009524470782921032,5017.24468616309,735.2803093025531 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.47,3,0.6320127801567271,0.009797393520568215,0.5050808685213019,0.009887109410395993,5101.619071585322,747.389104987921 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.48,3,0.6303679966726219,0.00981029934431753,0.5133047859418277,0.009329169408993173,5184.563382678364,759.2926668481134 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.49,3,0.6287515715244495,0.009895673277692182,0.5226157729064459,0.009526572035391971,5278.471596961865,772.7696880346527 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.5,3,0.6273242019491629,0.009832598413097358,0.5310665570144345,0.009248673211785196,5363.704026981612,785.0016240151261 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.51,3,0.6256037962359036,0.009883746878345902,0.5397158494739529,0.008402908792330719,5450.938561062223,797.5208873508454 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.52,3,0.6237888627362013,0.009800293342150409,0.5482517085897399,0.0084842466726669,5537.029035679483,809.8759636264241 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.53,3,0.6221818903666733,0.010030917064038556,0.5556248759322803,0.008700449615085806,5611.392900797382,820.5481225355619 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.54,3,0.6202818818591725,0.010121370442643416,0.5646428267589257,0.009470184090957644,5702.3456281338895,833.6009938167382 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.55,3,0.6188734178411744,0.010075589007489926,0.5723751996899488,0.009764120943604904,5780.33234821907,844.7930784163211 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.56,3,0.6167276370889222,0.009989552281372877,0.581081208821333,0.008945855207442318,5868.13891203136,857.3944352821109 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.57,3,0.6151490230553271,0.009972865147611568,0.5888986567601546,0.009190502623313012,5946.983676714056,868.7096601767993 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.58,3,0.613239561769182,0.010023841647160636,0.596838990821352,0.008852657156197262,6027.067839148716,880.2027543866401 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.59,3,0.6116514949569426,0.010029536234948755,0.6040325553696508,0.009023596945745508,6099.620276782974,890.6149504505552 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.6,3,0.6101485031525017,0.009905312309579351,0.611764928300674,0.008638836428393179,6177.606996868156,901.8070350501383 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.61,3,0.6082674002022895,0.00980816777353647,0.6198848651561126,0.008487658314180385,6259.502586786458,913.5600921052016 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.62,3,0.6065848056036072,0.00980405818230916,0.6282411214776583,0.008715943762864817,6343.781633920076,925.6552055355578 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.63,3,0.6048265887757707,0.009541923380768483,0.6364272277836069,0.009228749291161274,6426.344591858667,937.5040383757031 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.64,3,0.6030211080547128,0.009399732052994915,0.6452372174800783,0.008786631057525985,6515.199876845697,950.2559000466218 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.65,3,0.6013007023414533,0.00913597241018234,0.6528656098460143,0.009029298911157406,6592.137875756138,961.2974798410759 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.66,3,0.5992872604902212,0.00919936530599135,0.6610800744878956,0.008388391721092287,6674.986848560566,973.1873594462562 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.67,3,0.5975290436623846,0.00907101365783639,0.6695025002599514,0.0077602188887146,6759.933263714474,985.3782486616946 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.68,3,0.5953927156887767,0.009062912838653894,0.6785204510865969,0.007426503817004564,6850.885991050982,998.4311199428707 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.69,3,0.5937006683114502,0.009178360873760371,0.68631899346813,0.006291433155867443,6929.5400791564525,1009.7189803275359 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.7,3,0.5916494153456409,0.009196888113421205,0.6952991331801983,0.00628921715217643,7020.111453338509,1022.7171225886652 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.71,3,0.5898533874032271,0.009341593413559118,0.703286731134617,0.006278196564291221,7100.672307216233,1034.2786280735645 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.72,3,0.5876508899791094,0.009419474553195561,0.7131743375965366,0.006639014245866606,7200.3961571051095,1048.5902668158187 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.73,3,0.5857508814716086,0.009109644299865004,0.7214644244675723,0.006817845676298493,7284.0078362184395,1060.589604461093 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.74,3,0.5836334590552893,0.008991257258145258,0.7299246613542051,0.006704426152341965,7369.335604526797,1072.8352226965778 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.75,3,0.5814404144098159,0.00858033252822142,0.7394719677849304,0.008502993761525655,7465.627276025616,1086.65430025841 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.76,3,0.579408067001295,0.008342825576559049,0.7475068296325705,0.008574332003549545,7546.664821346404,1098.284217018368 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.77,3,0.57736626681413,0.00804285381163401,0.7562790082144647,0.008681212876348115,7635.1387531789815,1110.9813496692398 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.78,3,0.5751165054967907,0.008107916657113915,0.7657601451946799,0.008773349914837794,7730.76305665751,1124.70465144599 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.79,3,0.5726398774919889,0.007660145947342854,0.7751940182816739,0.009274876423397391,7825.910668692976,1138.3595419476817 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.8,3,0.5707965856563537,0.007802305398239622,0.7841363468791651,0.008943627510790869,7916.100689720582,1151.3029551887644 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.81,3,0.5685184660030815,0.00763102158917731,0.7934095227292063,0.00891706217729894,8009.627550849633,1164.725247355257 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.82,3,0.566259251907098,0.0072902804900308715,0.8023045874334761,0.009254448918612828,8099.340880434175,1177.600249321281 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.83,3,0.5641229239334903,0.007223395091058391,0.8105757687472233,0.008296621479466769,8182.76188297028,1189.5722224565316 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.84,3,0.5616179375927555,0.006707184432545703,0.8201703390711699,0.009488602960963334,8279.530245912161,1203.4597112934223 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.85,3,0.5591318568093091,0.006475077472665963,0.8296136649368081,0.009342664664304827,8374.77319623624,1217.1282840501258 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.86,3,0.5566930399190843,0.006185946301883619,0.8385654463129436,0.010218877812758589,8465.058555552458,1230.08537954622 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.87,3,0.5539611868908866,0.005829061910237554,0.8484341472175746,0.010905598250638093,8564.591728864109,1244.3696537784508 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.88,3,0.5513900310996417,0.004896083101297839,0.8582461314503398,0.011959584413082348,8663.552872444081,1258.5718344806112 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.89,3,0.5482138974751628,0.00409738429460064,0.8676800045373337,0.013450603171652009,8758.700484479548,1272.2267249823028 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.9,3,0.5453213472100124,0.003690387058675928,0.8773029331972134,0.01223524083671025,8855.75486228727,1286.1552605842292 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.91,3,0.5420128746845135,0.0033024983771880426,0.8877955174923668,0.014059259757785809,8961.580362647355,1301.3425636472327 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.92,3,0.5385342521434175,0.0030980607738457146,0.8988363629488889,0.015067883344047338,9072.93548374698,1317.3234375009163 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.93,3,0.5348193101362145,0.0029220861371181467,0.9099811889704977,0.015344844384073209,9185.33932602134,1333.4548161597281 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.94,3,0.5316242709544471,0.003351753236698738,0.9201334732344573,0.015695193946690233,9287.732647991372,1348.1495580423104 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.95,3,0.5275879344733384,0.003908949709419728,0.9317320326309918,0.015784368548069225,9404.712728119144,1364.9376849416847 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.96,3,0.5239864258098668,0.0038559227014723627,0.9413927724054485,0.015118412573123127,9502.148459081314,1378.9209495636576 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.97,3,0.5198272032063825,0.003925373748507694,0.9532938207186001,0.013209405010335662,9622.179364444692,1396.146908623407 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.98,3,0.5152520583425498,0.003849103678203556,0.9653366607114161,0.011483791694188549,9743.640344137262,1413.578101508332 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,0.99,3,0.5100152189736173,0.0036056620177345906,0.978079006323909,0.009338895613008017,9872.15635718717,1432.0217812641245 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,original,原方法,1.0,3,0.5000708958398321,0.0,1.0,0.0,10093.24584848,1463.7509306362917 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.0,1,0.6695119530385957,0.0,0.0,0.0,7.503634432,16.318855201956467 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.01,1,0.6686044862887446,0.0,0.002467175226157729,0.0,32.38692775991243,19.889923760014124 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.02,1,0.6678388112185577,0.0,0.004849275444516916,0.0,56.41217649031064,23.337852022966345 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.03,1,0.667214927828035,0.0,0.007600034030003119,0.0,84.1556184766038,27.319388231375456 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.04,1,0.6666761194453109,0.0,0.009811984232765221,0.0,106.46477801197358,30.521035904116808 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.05,1,0.6662791027422511,0.0,0.012534384482318578,0.0,133.9222051324287,34.46152534749078 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.06,1,0.6658537277032583,0.0,0.01607917647392451,0.0,169.67406336218795,39.592370976883956 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.07,1,0.6654850693361314,0.0,0.019709043473328986,0.0,206.2839661894614,44.84635690138258 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.08,1,0.6652865609846014,0.0,0.02353741882426339,0.0,244.89597307760135,50.38767018112722 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.09,1,0.6648611859456087,0.0,0.028868785979638715,0.0,298.66676785515926,58.10446200773457 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.1,1,0.6645208859144145,0.0,0.0345971698380739,0.0,356.4417707544502,66.39590854483396 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.11,1,0.6642089442191532,0.0,0.03981510364971783,0.0,409.0685060686558,73.94851331130073 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.12,1,0.6640387942035562,0.0,0.045855429203414344,0.0,469.9896724921656,82.69147426378672 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.13,1,0.6637268525082948,0.0,0.05317187987408899,0.0,543.7815078783887,93.28153964285426 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.14,1,0.6633581941411678,0.0,0.05960922213084536,0.0,608.7068824236314,102.59915530583227 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.15,1,0.6630178941099736,0.0,0.0675211978561098,0.0,688.505029992454,114.05120275063786 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.16,1,0.6626492357428466,0.0,0.07551824858917279,0.0,769.1612221587909,125.6263904905489 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.17,1,0.6621671440319882,0.0,0.08212574086152624,0.0,835.8026858990621,135.19028674373777 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.18,1,0.6619969940163911,0.0,0.08856308311828262,0.0,900.7280604443049,144.50790240671583 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.19,1,0.6615716189773984,0.0,0.09480191702350906,0.0,963.6513309286811,153.5381907144478 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.2,1,0.6612029606102714,0.0,0.10007656750701868,0.0,1016.8500959745629,161.17288901098487 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.21,1,0.6608910189150101,0.0,0.10554972634205825,0.0,1072.0509650813112,169.09491466276796 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.22,1,0.6605790772197487,0.0,0.11005870175538099,0.0,1117.5273287495647,175.62135030335608 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.23,1,0.6602671355244875,0.0,0.11527663556702493,0.0,1170.1540640637704,183.17395506982288 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.24,1,0.6599835521651589,0.0,0.12029606102713893,0.0,1220.7786953171096,190.4392324810436 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.25,1,0.6596999688058305,0.0,0.12477667810452883,0.0,1265.9690441195253,196.92462135659662 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.26,1,0.659217877094972,0.0,0.12914386183818732,0.0,1310.0153334585884,203.24582317200898 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.27,1,0.6587925020559794,0.0,0.1322916371267334,0.0,1341.7629835666146,207.80201409091015 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.28,1,0.6582253353373224,0.0,0.13626180415733205,0.0,1381.8050647839452,213.54856119583053 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.29,1,0.6578283186342625,0.0,0.14023197118793068,0.0,1421.8471460012754,219.29510830075088 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.3,1,0.6573178685874713,0.0,0.14326631313274538,0.0,1452.4507366459495,223.68711215951146 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.31,1,0.6569208518844114,0.0,0.14766185520233674,0.0,1496.783040850851,230.04936073995904 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.32,1,0.6565805518532173,0.0,0.15066783881121856,0.0,1527.1006166296868,234.40031783368445 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.33,1,0.6561268184782917,0.0,0.15452457249808582,0.0,1565.998638383665,239.98267787846424 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.34,1,0.6558715934548961,0.0,0.15806936448969175,0.0,1601.750496613424,245.11352350785742 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.35,1,0.6555596517596347,0.0,0.16201117318435754,0.0,1641.5065629649162,250.8190238477426 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.36,1,0.6551626350565749,0.0,0.1657544735274934,0.0,1679.260525255542,256.2371968323818 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.37,1,0.6545954683379179,0.0,0.17012165726115192,0.0,1723.3068145946054,262.5583986477942 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.38,1,0.6540850182911266,0.0,0.17536794940872868,0.0,1776.219564774649,270.1520501792961 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.39,1,0.6534044182287383,0.0,0.17965005813458867,0.0,1819.4078095161983,276.35011169960313 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.4,1,0.653092476533477,0.0,0.18401724186824717,0.0,1863.4540988552617,282.6713135150155 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.41,1,0.6527521765022829,0.0,0.18858293395343562,0.0,1909.5024922551916,289.27984268567394 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.42,1,0.6523835181351558,0.0,0.19238295096843716,0.0,1947.8284842774933,294.78010920038344 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.43,1,0.6517596347446332,0.0,0.19689192638175992,0.0,1993.3048479457473,301.3065448409716 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.44,1,0.6511641096900433,0.0,0.20182627683407536,0.0,2043.071434601572,308.4486819570869 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.45,1,0.6505402262995208,0.0,0.20718600232538353,0.0,2097.1282442449683,316.2065205487294 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.46,1,0.6500864929245952,0.0,0.21189348609023623,0.0,2144.6067119740887,323.02028354456354 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.47,1,0.6500581345886624,0.0,0.217139778237813,0.0,2197.5194621541323,330.61393507606545 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.48,1,0.6507670929869835,0.0,0.22198905368232993,0.0,2246.428004212443,337.63293189707537 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.49,1,0.6515611263931033,0.0,0.22763236253296656,0.0,2303.34496251422,345.80123813906926 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.5,1,0.6527521765022829,0.0,0.2345234381646485,0.0,2372.8465749128713,355.77560204260965 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.51,1,0.6537447182599325,0.0,0.24382497235062248,0.0,2466.65945090776,369.2389409741374 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.52,1,0.6545387516660522,0.0,0.252644414825738,0.0,2555.6100741834007,382.0044849000676 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.53,1,0.6538581516036639,0.0,0.26401610753480986,0.0,2670.3020353844686,398.46423767916093 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.54,1,0.6537163599239997,0.0,0.2760116836344043,0.0,2791.2863236339736,415.8270192890275 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.55,1,0.6533477015568726,0.0,0.2893401015228426,0.0,2925.7133105778685,435.11899885554584 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.56,1,0.6525536681507529,0.0,0.3036043444970649,0.0,3069.57878809442,455.76552166822404 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.57,1,0.6509088846666478,0.0,0.3175850041119587,0.0,3210.58411695259,476.0015768305508 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.58,1,0.6498312679011996,0.0,0.33462836400760004,0.0,3382.4790513212724,500.6706826166732 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.59,1,0.6465984176048549,0.0,0.35637920766809406,0.0,3601.8524534190756,532.1535513986298 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.6,1,0.6442730340583614,0.0,0.37968975980489467,0.0,3836.9566731379723,565.8939922575194 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.61,1,0.6405580920511584,0.0,0.4046450954258004,0.0,4088.649755075477,602.0151454884474 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.62,1,0.6357371749425744,0.0,0.43141536454640844,0.0,4358.647788426619,640.7632916816248 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.63,1,0.630604316138729,0.0,0.4610214672603012,0.0,4657.247308361568,683.6161143783166 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.64,1,0.6244221989053682,0.0,0.4916484700677764,0.0,4966.143363466687,727.9466206162738 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.65,1,0.6187788900547316,0.0,0.5209426310864078,0.0,5261.596719877418,770.347928897579 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.66,1,0.6118310977511839,0.0,0.5529308340186598,0.0,5584.221488542766,816.6486798572232 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.67,1,0.6045146470805093,0.0,0.5888041289737118,0.0,5946.030293827928,868.5728376266821 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.68,1,0.5983608881830814,0.0,0.6175027649377535,0.0,6235.47733805606,910.1121638422494 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.69,1,0.5925474293168477,0.0,0.6421461588633979,0.0,6484.024256469345,945.7818026577909 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.7,1,0.5877548705441965,0.0,0.665910444375124,0.0,6723.704714041652,980.1789917572427 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.71,1,0.5829339534356124,0.0,0.6838612710206166,0.0,6904.7521241171535,1006.1615940244899 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.72,1,0.5779996029832969,0.0,0.7029464311034228,0.0,7097.240128826176,1033.7860668931426 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.73,1,0.5745966026713553,0.0,0.7180614241556306,0.0,7249.68605231787,1055.6639926568753 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.74,1,0.5710234523438165,0.0,0.7352465757309361,0.0,7423.011061015743,1080.5383322681735 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.75,1,0.567762243711539,0.0,0.7478093185491875,0.0,7549.715646582009,1098.722049178743 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.76,1,0.565153276805717,0.0,0.759804894648782,0.0,7670.699934831514,1116.0848307886095 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.77,1,0.5620622181890367,0.0,0.7729064458497575,0.0,7802.838802848705,1135.0484362348468 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.78,1,0.5588860845645578,0.0,0.7855259053398747,0.0,7930.115418146647,1153.3142466754864 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.79,1,0.5562771176587358,0.0,0.7964722230099538,0.0,8040.517156360144,1169.1582979790526 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.8,1,0.5536114340810481,0.0,0.8082125740861527,0.0,8158.927310817107,1186.1516587036028 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.81,1,0.5504920171284349,0.0,0.8197544168108215,0.0,8275.335361213203,1202.857692072907 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.82,1,0.5469188668008961,0.0,0.8333380597226555,0.0,8412.33648194964,1222.5190925247418 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.83,1,0.5450188582933954,0.0,0.8426112355726966,0.0,8505.86334307869,1235.9413846912344 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.84,1,0.5423531747157077,0.0,0.8529903865241187,0.0,8610.544783975425,1250.9645006940975 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.85,1,0.5395456994583557,0.0,0.8635396874911381,0.0,8716.94231406719,1266.2338972871717 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.86,1,0.5378725576383178,0.0,0.8710262881774098,0.0,8792.450238648442,1277.07024325645 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.87,1,0.535774040779287,0.0,0.8809517057539064,0.0,8892.555441691766,1291.436611018751 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.88,1,0.5330516405297338,0.0,0.8913024983693957,0.0,8996.950867722664,1306.4186802565791 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.89,1,0.5304710319598446,0.0,0.900915974250631,0.0,9093.90990724177,1320.3335336034934 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.9,1,0.5277486317102913,0.0,0.910954825170859,0.0,9195.15916974845,1334.864088425935 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.91,1,0.5245724980858123,0.0,0.921901142840938,0.0,9305.560907961944,1350.708139729501 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.92,1,0.5217366644925275,0.0,0.9303235686129938,0.0,9390.507323115853,1362.8990289449393 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.93,1,0.5187306808836457,0.0,0.9393698777755721,0.0,9481.746065318199,1375.9929469911506 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.94,1,0.515526188923234,0.0,0.9490117119927403,0.0,9578.991119703143,1389.9488471031002 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.95,1,0.5122366219550237,0.0,0.9587953378895726,0.0,9677.66624841728,1404.1099810402252 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.96,1,0.5097694467288659,0.0,0.9673595553412926,0.0,9764.042737900378,1416.5061040808393 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.97,1,0.5075574965261038,0.0,0.9746192893401016,0.0,9837.262543554925,1427.0140759298365 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.98,1,0.5046365879250205,0.0,0.9830984317840229,0.0,9922.780988440509,1439.287058675345 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,0.99,1,0.5019425460114001,0.0,0.9908118991577575,0.0,10000.577031948465,1450.4517787649047 +grefcoco|instancevg|MiniCPM,gRefCOCO / InstanceVG + MiniCPM,grefcoco,confidence,置信度路由,1.0,1,0.5000708958398321,0.0,1.0,0.0,10093.24584848,1463.7509306362917 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.0,3,0.18129624718318374,2.7755575615628914e-17,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.01,3,0.1834834493716783,0.0012311365629078506,0.01187305814469596,0.0025254786233335643,1318.8138164976065,62.665281079596 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.02,3,0.18516926468318548,0.0015624361327920705,0.02041722148246782,0.004089109137303213,2127.957967387884,84.87900282646561 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.03,3,0.18663764287485787,0.0018019747907258071,0.03151353750554816,0.0009545411969643387,3178.7945269856473,113.72799210811446 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.04,3,0.18706369336710219,0.0011714382116068973,0.03817132711939636,0.00041518612813736623,3809.296462744305,131.03738567710377 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.05,3,0.19244312207391248,0.0014149660525805208,0.04804704837993786,0.0025254786233335656,4744.541000786314,156.71298613777122 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.06,3,0.19381392328321423,0.003949261489595185,0.059698180204172224,0.006387855288337441,5847.9193883639655,187.00442488350254 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.07,3,0.19647974599905252,0.002543160302569749,0.06924101198402131,0.002052068576221363,6751.638829618042,211.81455566572055 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.08,3,0.20111251826239193,0.0016692561651361969,0.08843763870395027,0.003932544417813882,8569.58607772217,261.723307122973 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.09,3,0.20211540664478245,0.0014043232403395474,0.09565024411895251,0.0015455379801580266,9252.629841460719,280.47515015604483 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.1,3,0.2058996264309502,0.000756529123691676,0.10707944962272525,0.0037002512207130658,10334.991497846413,310.1896091161431 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.11,3,0.2097488558515801,0.0014923128238356481,0.11873058144695962,0.004940042008333793,11438.369885424067,340.4810478618744 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.12,3,0.21357479992379627,0.002043397662673137,0.1318242343541944,0.0034056833673375042,12678.357025749425,374.52285521422004 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.13,3,0.21757582639483944,0.0005091408775510048,0.13903683976919665,0.0037102204832002903,13361.400789487974,393.27469824729184 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.14,3,0.2208265597787713,0.0022365367656948204,0.1484687083888149,0.002940000288391433,14254.61186514607,417.7963391366933 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.15,3,0.22299694778849402,0.0017828789549076917,0.1553484243231247,0.0034201142937022692,14906.130532096682,435.68271249131556 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.16,3,0.2255777502717166,0.0014946748752493809,0.16533510874389704,0.0020580600300700597,15851.883435734671,461.6468028447996 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.17,3,0.22875870242149624,0.0029675820167744103,0.17587660896582336,0.0005658033193067903,16850.178167352547,489.053342662366 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.18,3,0.23119107161987315,0.0037577528930060286,0.18519751442521085,0.0012844914450499617,17732.880877414667,513.286493658951 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.19,3,0.23272355305340456,0.003899840382608578,0.19485130936529074,0.0014969748738606365,18647.10868426472,538.3851143339855 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.2,3,0.23435202613894154,0.00439571270563422,0.2049489569462938,0.0024662794907781742,19603.36995349868,564.6376945802858 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.21,3,0.2390742822705132,0.007755915980427258,0.2177097203728362,0.007778509617541884,20811.83199703611,597.814032254182 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.22,3,0.24247271253490546,0.007731319003581467,0.22614292055037724,0.00997311303534013,21610.46778233041,619.7392641082351 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.23,3,0.24464526821569152,0.007707556599008443,0.23424323124722593,0.009003756225078026,22377.57847083678,640.7990262838389 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.24,3,0.24994120439814113,0.0057830070708975645,0.25233022636484687,0.008981849243163802,24090.442062981132,687.8228788129264 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.25,3,0.25617136079673564,0.0020077364088740926,0.26498002663115844,0.008293336414138841,25288.39574092258,720.7107265940061 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.26,3,0.2574242353399269,0.002315956397572623,0.2714158899245451,0.011021719966124286,25897.880945489287,737.4431403773626 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.27,3,0.26032390146245593,0.002379685300559487,0.2817354638260097,0.00953379561651602,26875.158945915202,764.2727004092958 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.28,3,0.2628052136572033,0.001830304310467188,0.2939414114513981,0.007625034417492007,28031.079161472742,796.0065886191096 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.29,3,0.26385315219544514,0.002608335850383775,0.300821127385708,0.008974992345269047,28682.59782842336,813.8929619737321 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.3,3,0.26634968880720944,0.001741898001346996,0.3129161118508655,0.007778509617541895,29828.009678384915,845.3383602907292 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.31,3,0.2690146179614465,0.0012560400241288774,0.3235685752330226,0.003805243719370242,30836.81277559877,873.033390001112 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.32,3,0.2724482524370387,0.001967022198843412,0.3308921438082557,0.004540007222698347,31530.3649049333,892.0737229270005 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.33,3,0.2785792932806596,0.003228203810863451,0.34309809143364406,0.0069527075889128055,32686.285120490837,923.8076111368141 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.34,3,0.280240582539615,0.003751257110681198,0.35286284953395475,0.00124555838441209,33611.021292936864,949.1947217046651 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.35,3,0.28366301748133643,0.002918594811022553,0.3615179760319574,0.0009415536367330725,34430.67380942312,971.6969333443511 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.36,3,0.28804573870453565,0.003850453698457738,0.3707279183311141,0.00433467043600191,35302.86815388926,995.6415944481198 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.37,3,0.2918619157168322,0.003446497810500408,0.37893919218819355,0.00416074861501391,36080.48720799161,1016.98984651654 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.38,3,0.2924785593949612,0.0035256332780609946,0.383710608078118,0.006271136773610646,36532.34692861864,1029.3949119076487 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.39,3,0.2940271449000534,0.004408540165257409,0.3965823346648913,0.007756316962850123,37751.31733775205,1062.8597394743615 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.4,3,0.29720887304775023,0.0016301503743588201,0.4050155348424323,0.0030550155121917685,38549.95312304635,1084.7849713284147 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.41,3,0.29763961568174857,0.0019073673088544577,0.41344873501997337,0.002122850251854073,39348.58890834065,1106.7102031824677 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.42,3,0.2997354669008654,0.001493745228754507,0.42532179316466934,0.004402315673519171,40472.98402711026,1137.578621713832 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.43,3,0.2990037319893076,0.0015550709692697656,0.43153573013759433,0.003866234218860266,41061.452500485,1153.7340557115554 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.44,3,0.30076478793578976,0.001539412121817168,0.438526409232135,0.0036902550262573925,41723.4795330316,1171.9089189589943 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.45,3,0.2997684358624232,0.0008510987206140805,0.44795827785175324,0.0005658033193067849,42616.69060868969,1196.4305598483957 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.46,3,0.3001253059883045,0.0011488907279669142,0.4539502885042166,0.0021969584857105553,43184.142350872484,1212.009014060486 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.47,3,0.30082512975049697,0.0029890971348924835,0.4629383044829117,0.005258764141621297,44035.31996414667,1235.3766953786217 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.48,3,0.300731861770386,0.0023331902978894745,0.47203728362183756,0.005898816626720836,44897.00594301684,1259.0328665895736 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.49,3,0.30024162465381776,0.002588021422925145,0.4823568575233022,0.004058886186900106,45874.28394344275,1285.862426621507 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.5,3,0.29980106360416753,0.002221016752728669,0.49511762094984463,0.00440231567351916,47082.74598698018,1319.038764295403 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.51,3,0.300001834137088,0.0018514239054487809,0.5088770528184642,0.0023116585281846283,48385.783320881405,1354.8115110046476 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.52,3,0.3002467938073421,0.0006176309726740444,0.5199733688415445,0.0025928365391806253,49436.61988047917,1383.6605002862966 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.53,3,0.2997991298687485,0.0008969949407747574,0.5315135375055481,0.004726040709952734,50529.48990246084,1413.6634491392113 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.54,3,0.3007437474424171,0.003838910143448746,0.5430537061695517,0.00441906519124274,51622.35992444252,1443.6663979921263 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.55,3,0.30024301841980733,0.0035732909218718816,0.5548158011540169,0.003588762012604599,52736.246677616145,1474.246326630674 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.56,3,0.2997639514216622,0.0038812246830032806,0.5624722592099423,0.00611607358167803,53461.3239037386,1494.1521292350117 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.57,3,0.29853753336894806,0.0035064754246926957,0.5763426542387927,0.004932558981556364,54774.86960323581,1530.2133658370728 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.58,3,0.29866536548639894,0.0030097879435426163,0.5889924545051043,0.0060309268715939696,55972.82328117726,1563.1012136181525 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.59,3,0.2979213023866331,0.0028219662091492214,0.6000887705281847,0.008082811296704103,57023.65984077502,1591.9502028998013 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.6,3,0.296181514312352,0.0030278910707881817,0.6119618286728806,0.009198571582806334,58148.05495954463,1622.8186214311656 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.61,3,0.295425969846723,0.0024041729399304087,0.624167776298269,0.007338684315928307,59303.975175102176,1654.5525096409795 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.62,3,0.2960016708262807,0.0011822362382635915,0.6344873501997337,0.001438246937063454,60281.25317552809,1681.3820696729128 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.63,3,0.2956470818737702,0.0016490449406111047,0.6456946293830449,0.003710220483200315,61342.598100721836,1710.5195488473782 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.64,3,0.29404411164987093,0.0016104093348727044,0.6541278295605859,0.004219519161524997,62141.23388601613,1732.4447807014312 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.65,3,0.2930877570799049,0.0017077002503879724,0.6640035508211274,0.006176174012461514,63076.478424058136,1758.1203811620987 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.66,3,0.2931103060052791,0.0023825255128999425,0.6742121615623612,0.006910074335082253,64043.24805888807,1784.6614513012153 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.67,3,0.2911080923222205,0.0021816454930482197,0.6840878828229028,0.010861927513107179,64978.49259693009,1810.337051761883 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.68,3,0.29092854977655214,0.0013345094998848492,0.6919662671992898,0.007365480148813142,65724.5865542445,1830.8198341518537 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.69,3,0.2896057055401419,0.0019433334937609774,0.7003994673768309,0.009132746712628604,66523.22233953881,1852.745066005907 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.7,3,0.28901527116547426,0.001802837339194139,0.7075011096316023,0.0065534057080485555,67195.75773768137,1871.2084191461622 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.71,3,0.28835089710053935,0.0021525756584588945,0.7126054150022192,0.006984511754041831,67679.14255509635,1884.4789542157205 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.72,3,0.2889521855225583,0.001324148010058386,0.7203728362183756,0.006144193608480728,68414.72814681478,1904.673246712875 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.73,3,0.2893751337558484,0.0011370340700424628,0.7332445628051487,0.007223693208504127,69633.69855594818,1938.1380742795875 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.74,3,0.2893596502813577,0.001299151835177392,0.7395694629383045,0.007203210105245204,70232.67539491892,1954.5819981701275 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.75,3,0.28788636673929996,0.0011284209510194398,0.7496671105193076,0.005429263125932131,71188.93666415288,1980.8345784164278 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.76,3,0.28837566257562763,0.001277295112344517,0.757878384376387,0.00464722405489835,71966.55571825521,2002.1828304848477 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.77,3,0.2867700993449886,0.0027059959624612722,0.770861074123391,0.0023116585281846105,73196.0344929846,2035.9361479443771 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.78,3,0.2887400928282857,0.0019716435845878837,0.7779627163781625,0.004104137152442735,73868.56989112718,2054.3995010846324 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.79,3,0.28928209765123375,0.00225110075102362,0.7877274744784731,0.007274648647618171,74793.3060635732,2079.7866116524833 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.8,3,0.28985471644589866,0.004827324933946651,0.7950510430537062,0.007455205074230275,75486.85819290773,2098.8269445783717 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.81,3,0.2886876082446482,0.005146306163708364,0.8039280958721703,0.008248676227272878,76327.52744058592,2121.9061360036903 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.82,3,0.28743690856703363,0.005693203323036328,0.8125832223701731,0.0059611078644311085,77147.17995707219,2144.4083476433766 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.83,3,0.28644382983111544,0.005806966387956601,0.8249001331557922,0.00222480444471411,78313.6085382257,2176.430725746007 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.84,3,0.28480770451609944,0.005177287421963846,0.8373280071016422,0.0005658033193067777,79490.5454849752,2208.7415937414535 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.85,3,0.2831555945783542,0.004881627617365338,0.8462050599201065,0.002940000288391392,80331.21473265342,2231.8207851667726 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.86,3,0.2817864618218226,0.005205046028156837,0.8584110075454948,0.007073847070798583,81487.13494821095,2263.5546733765864 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.87,3,0.28077685358886717,0.003976619583586016,0.8698402130492676,0.007237316404403161,82569.49660459664,2293.269132336685 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.88,3,0.2807792209572976,0.0047050100149973245,0.8802707501109631,0.0045182585600315826,83557.28297061854,2320.3871822614346 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.89,3,0.2797249967658659,0.003993909763816949,0.891034176653351,0.004160748615013883,84576.59443342837,2348.370701864634 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.9,3,0.2808861418053256,0.0034796247435544114,0.9010208610741234,0.004623317056369212,85522.34733706636,2374.334792218118 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.91,3,0.28060995842168185,0.00195870708983316,0.9079005770084332,0.005572516435636824,86173.86600401698,2392.22116557274 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.92,3,0.27872311422141527,0.001933219388154891,0.9185530403905903,0.005981727462025553,87182.66910123083,2419.916195283123 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.93,3,0.2776252989388338,0.00209501538736219,0.9297603195739015,0.006010476060342179,88244.01402642457,2449.053674457589 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.94,3,0.27702282067436085,0.002048066268398205,0.9395250776742121,0.0029815865163214566,89168.75019887059,2474.4407850254393 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.95,3,0.2764570604735362,0.001106373667494808,0.9451841988459831,0.0013680457174809137,89704.67684426546,2489.1537695590805 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.96,3,0.27465070840546985,9.533193830567857e-05,0.957723035952064,0.006120098638765064,90892.12215661095,2521.753127447344 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.97,3,0.2736502964972301,0.00033280989483324634,0.9670439414114513,0.0019026218596851095,91774.82486667305,2545.9862784439288 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.98,3,0.27212054745108016,0.00047581600622207385,0.9783621837549935,0.002718031228121553,92846.67815746278,2575.412247511211 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,0.99,3,0.27239176103160984,0.0005578690368974412,0.9862405681313805,0.0023116585281845767,93592.7721147772,2595.8950299011817 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,both,两项同时改,1.0,3,0.27001009067886245,0.0,1.0,0.0,94895.80944867841,2631.667776610426 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.0,3,0.18129624718318374,2.7755575615628914e-17,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.01,3,0.18300903853620068,0.000992883110506064,0.010319573901464712,0.0032954377285658006,1171.6966981539194,58.626422580165155 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.02,3,0.1863051480658938,0.0010484972619040074,0.016977363515312916,0.003339972690477677,1802.1986339125776,75.93581614915446 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.03,3,0.18984164754521382,0.0011730932549335858,0.026742121615623615,0.003979232373954973,2726.9348063586094,101.32292671700546 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.04,3,0.1923009691917478,0.0012207053006872883,0.03650687971593431,0.004834225679270296,3651.6709788046405,126.71003728485644 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.05,3,0.19493527207894204,0.0032238119689520237,0.049822458943630715,0.011397380483895993,4912.674850321956,161.32882442283505 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.06,3,0.19653781676071322,0.0037018743838214347,0.05670217487794052,0.01068477862158076,5564.193517272569,179.21519777745732 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.07,3,0.19845342291677406,0.002558550208561758,0.07012871726586772,0.007921246470295046,6835.705754385861,214.1224748082524 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.08,3,0.1996517807186753,0.00326377452572563,0.08055925432756324,0.005616533837036735,7823.492120407758,241.24052473300233 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.09,3,0.20131030497544633,0.003862950889574192,0.0883266755437195,0.007612105142539945,8559.077712126194,261.43481723015657 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.1,3,0.20346296196880112,0.004110692938093332,0.09576120727918332,0.006259345218007303,9263.138207056696,280.76364004886125 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.11,3,0.20570676605778512,0.0038511319162046987,0.10497114957833999,0.007177524025209153,10135.332551522839,304.7083011526298 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.12,3,0.2070376175899661,0.0032282329917381024,0.11562361296049711,0.004558952193558395,11144.135648736692,332.4033308630127 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.13,3,0.20882521553556344,0.004240270628896633,0.125277407900577,0.003321489025098921,12058.363455586745,357.5019515380472 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.14,3,0.2104535116001729,0.003047249002743018,0.13792720816688858,0.004089109137303215,13256.317133528195,390.38979931912684 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.15,3,0.21628026754255258,0.0012410430113859144,0.1513537505548158,0.0037002512207130593,14527.82937064149,425.297076349922 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.16,3,0.21953898964292348,0.0026751029505782367,0.1615623612960497,0.006200051062893467,15494.59900547143,451.83814648903893 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.17,3,0.22275747006798005,0.0018712171302913713,0.1767643142476698,0.0032954377285657993,16934.24509212037,491.36126180489794 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.18,3,0.22506270243381263,0.0016242401220143224,0.1870838881491345,0.004374257289042843,17911.523092546286,518.1908218368312 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.19,3,0.2279934806251012,0.0034383753330345097,0.19773635153129163,0.0037760096598254663,18920.32618976014,545.8858515472142 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.2,3,0.23281283097861405,0.0029141009403282343,0.21116289391921883,0.002829016596533941,20191.838426873433,580.7931285780093 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.21,3,0.2346923902449591,0.004269125091948503,0.21704394141145142,0.003834254891959823,20748.781803460246,596.0830928972832 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.22,3,0.23677784454531806,0.0038297567611664286,0.22625388371060806,0.0038852954727056407,21620.976147926387,620.0277540010517 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.23,3,0.24022916831367214,0.0033764440693607152,0.23679538393253438,0.0035369235353992913,22619.270879544263,647.434293818618 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.24,3,0.2427389920879772,0.002567039306571362,0.24522858411007545,0.0013947852962690268,23417.906664838563,669.3595256726712 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.25,3,0.24461408052105227,0.0022654085644492608,0.2533288948069241,0.0017823322685540256,24185.01735334493,690.419287848275 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.26,3,0.24875717867839975,0.0012150962427443486,0.2663115845539281,0.001087212491248644,25414.496128074315,724.1726053078041 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.27,3,0.25108136960988475,0.001884551609379602,0.27774079005770086,0.000565803319306803,26496.85778446001,753.8870642679024 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.28,3,0.25617362028487206,0.002583411595586948,0.28750554815801155,0.003505452507530237,27421.593956906043,779.2741748357533 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.29,3,0.2586135395304076,0.0021069512821710393,0.2999334221038615,0.0030630656328988486,28598.530903655534,811.5850428312 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.3,3,0.26281783580408685,0.003646149962582769,0.3118064802485575,0.006282906199241932,29722.926022425145,842.4534613625644 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.31,3,0.2659933335809095,0.0005472898637729304,0.32146027518863735,0.0030910759603160498,30637.153829275194,867.5520820375988 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.32,3,0.26795817858559673,0.000877323726982746,0.3276742121615624,0.002525478623333576,31225.622302649947,883.7075160353222 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.33,3,0.2697964580656605,0.00109146610981483,0.33732800710164224,0.0008303722562747448,32139.850109499996,908.8061367103567 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.34,3,0.2766676309946044,0.0038824825596419704,0.35153129161118507,0.0036768844930716396,33484.920905785126,945.7328429908672 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.35,3,0.27948418600108166,0.004511307928265581,0.3626276076342654,0.0025833215106131385,34535.75746538289,974.581832272516 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.36,3,0.28193304237414524,0.005383147607894484,0.37006213936972926,0.004368623986912911,35239.8179603134,993.9106550912207 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.37,3,0.28457928152218565,0.004320314330230671,0.3801597869507323,0.0036567370291353883,36196.07922954736,1020.1632353375211 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.38,3,0.28754612977164984,0.0031303927363197253,0.39092321349312026,0.0014969748738606341,37215.390692357185,1048.1467549407205 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.39,3,0.29211408749364837,0.0012621062792047405,0.4020195295162006,0.0012256281643572137,38266.22725195495,1076.9957442223695 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.4,3,0.29533548537843374,0.00299091721268992,0.4090102086107413,0.000565803319306803,38928.25428450155,1095.1706074698084 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.41,3,0.2976678659864959,0.0024569192378858322,0.413115845539281,0.0016533138510598571,39317.06381155272,1105.8447335040184 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.42,3,0.29926803271898383,0.0016044579796451672,0.4177762982689747,0.0009800000961304744,39758.41516658378,1117.9613090023108 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.43,3,0.30127482256635135,0.0014336142909526564,0.427208166888593,0.0017474495947651613,40651.62624224187,1142.4829498917122 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.44,3,0.3021047906767967,0.0003143158714052115,0.43586329338659563,0.00031385121224435753,41471.27875872813,1164.9851615313983 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.45,3,0.30240862593775775,0.00030672094867666285,0.44462938304482913,0.002075930640686835,42301.439640810364,1187.775863063901 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.46,3,0.3008577463264692,0.0015321281332227141,0.45528184642698627,0.0010290300150350249,43310.242738024215,1215.470892774284 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.47,3,0.3007178428135277,0.001611341013705818,0.4620505992010653,0.0015133357408994423,43951.253039378855,1233.06877623609 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.48,3,0.29962982001813376,0.0008195800307813963,0.46881935197514424,0.0018501256103565405,44592.26334073349,1250.6666596978955 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.49,3,0.30052237095071205,0.0018602790035998225,0.48102529960053264,0.0016974099579203656,45748.18355629103,1282.4005479077093 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.5,3,0.29849806777867033,0.0015269090065487179,0.4957833999112295,0.00518804692745715,47145.796180556055,1320.7697036523023 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.51,3,0.29827759512733504,0.00160137319563178,0.5075454948956947,0.0009545411969643582,48259.68293372968,1351.3496322908502 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.52,3,0.29763241873670115,0.00010078147354254689,0.5186418109187749,0.001245558384412085,49310.51949332744,1380.1986215724987 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.53,3,0.29801476087234113,0.0011742841718185815,0.534398579671549,0.0036701809606172545,50802.707407956266,1421.16418635244 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.54,3,0.2979694068577388,0.001202989512666543,0.5444962272525521,0.00015692560612219186,51758.96867719023,1447.4167665987404 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.55,3,0.2977270437091987,0.0012574341467251767,0.5544829116733245,0.0016607445125494896,52704.72158082822,1473.3808569522248 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.56,3,0.29609515932266106,0.00043402663006813217,0.563138038171327,0.004468937098447037,53524.37409731446,1495.8830685919104 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.57,3,0.2949258133108787,0.00032069248798376903,0.5781180648024855,0.0033546862977876166,54943.003452771445,1534.8292041221364 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.58,3,0.29574963421908845,0.0014111706990447396,0.5896582334664892,0.001496974873860648,56035.87347475313,1564.8321529750515 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.59,3,0.29622362807340946,0.0010731007721218823,0.6001997336884154,0.004490924621581925,57034.168206370996,1592.2386927926177 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.6,3,0.29525164414394867,0.0018070223722285175,0.609520639147803,0.0031345865272582123,57916.87091643312,1616.471843789203 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.61,3,0.2946903258163905,0.0011178432101274494,0.6233910341766533,0.0018097543753106938,59230.41661593032,1652.5330803912639 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.62,3,0.29352857709136265,0.001563132670522377,0.634820239680426,0.0034056833673375146,60312.77827231601,1682.247539351362 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.63,3,0.29314010821341197,0.0010629859658179415,0.6443630714602752,0.006468305211624347,61216.4977135701,1707.0576701335801 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.64,3,0.29135756898718546,0.0018490687911767016,0.6543497558810475,0.005505830550401814,62162.25061720808,1733.0217604870643 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.65,3,0.29002854350984486,0.0014350769484040525,0.6655570350643586,0.0041250843975679415,63223.59554240182,1762.1592396615295 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.66,3,0.29051736842744474,0.002613056805357834,0.6766533510874391,0.004526426554454301,64274.4321019996,1791.0082289431787 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.67,3,0.2897036098368808,0.002773767083577822,0.6845317354638261,0.007380510052514833,65020.526059314005,1811.4910113331491 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.68,3,0.2885789653876438,0.0023913071877969057,0.6974034620505991,0.00498964206744721,66239.4964684474,1844.9558388998614 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.69,3,0.2894905448762066,0.0036888322640739215,0.7076120727918331,0.002906302896961553,67206.26610327736,1871.4969090389789 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.7,3,0.2899145779502814,0.004586119846080865,0.7148246782068353,0.0014969748738606341,67889.3098670159,1890.2487520720504 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.71,3,0.28784978500457364,0.004735495702149088,0.727474478473147,0.0016607445125494896,69087.26354495736,1923.1365998531303 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.72,3,0.287461816607299,0.003191611555537852,0.7354638260097648,0.0006277024244887674,69843.86586786775,1943.9078721359174 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.73,3,0.28597281154624166,0.0016500495088830277,0.7446737683089214,0.0023222868992960636,70716.06021233388,1967.8525332396857 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.74,3,0.28630420585088484,0.0031615309093860314,0.7581003106968486,0.0049325589815563155,71987.57244944717,2002.7598102704808 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.75,3,0.28532383356176916,0.0029913445567370696,0.7654238792720817,0.006553405708048556,72681.1245787817,2021.8001431963692 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.76,3,0.28488007994545494,0.0022731942118462767,0.7760763426542386,0.009016055510773158,73689.92767599554,2049.4951729067516 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.77,3,0.28404447541777583,0.0014287448901881493,0.7891699955614736,0.002003492020114048,74929.91481632092,2083.5369802590976 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.78,3,0.28340097339207143,0.0008750672685543647,0.7986018641810918,0.002837707912237987,75823.125891979,2108.0586211484992 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.79,3,0.28179742094153154,0.0005289088741643894,0.8048158011540169,0.001368045717480914,76411.59436535375,2124.2140551462226 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.8,3,0.28013367081884827,0.0007247353638171571,0.8124722592099424,0.0023116585281846105,77136.67159147622,2144.1198577505606 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.81,3,0.2781888539103711,8.025423339638847e-05,0.8191300488237904,0.0005658033193067777,77767.17352723486,2161.429251319549 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.82,3,0.27770839725276386,0.0002773807579299807,0.8262316910785619,0.0011847623448769545,78439.70892537743,2179.8926044598047 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.83,3,0.2773234169518653,0.0005471202130380938,0.8369951176209498,0.005651500922435365,79459.02038818727,2207.876124063004 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.84,3,0.27742453433880515,0.0003494196082334972,0.8468708388814914,0.003676884493071685,80394.26492622928,2233.551724523672 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.85,3,0.27697393267513615,0.00034541476375886957,0.8579671549045716,0.0035054525075302184,81445.10148582704,2262.4007138053203 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.86,3,0.27658947811127477,0.00024441240230851225,0.8669551708832667,0.003856668263866561,82296.27909910123,2285.768395123456 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.87,3,0.2761395782158422,9.707962667947815e-05,0.8754993342210385,0.0021573704055951725,83105.42324999151,2307.9821168703256 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.88,3,0.27576917838734344,1.2278371232447543e-05,0.8841544607190412,0.0016533138510598272,83925.07576647776,2330.484328510012 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.89,3,0.2746915354514406,0.0003353981100208156,0.9009098979138926,0.005598968402815291,85511.8389714704,2374.046302325302 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.9,3,0.27479930216523635,0.0014741533261943386,0.9128939192188193,0.003960623235147524,86646.74245583596,2405.2032107494824 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.91,3,0.2747705463699966,0.0015181309875469695,0.9196626719928984,0.0013407729664441483,87287.75275719061,2422.8010942112883 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.92,3,0.2749000202760476,0.0006634607327990483,0.9282068353306702,0.0006840228587404539,88096.89690808087,2445.0148159581577 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.93,3,0.27363351631012667,0.00047104825594728914,0.9390812250332891,0.0009415536367330988,89126.7167364867,2473.286825454174 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.94,3,0.2731463916013074,0.0003553599231972337,0.9459609409675988,0.0015455379801579937,89778.2354034373,2491.173198808796 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.95,3,0.2722572862405767,0.0010281933856878484,0.9542831779849091,0.003941926246855662,90566.36282313564,2512.809940770033 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.96,3,0.2716243010726274,0.0011175756892329853,0.9620505992010653,0.003595617342658609,91301.94841485407,2533.004233267187 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.97,3,0.2714097002752531,0.0015154316158404004,0.9725920994229916,0.006055376675384957,92300.24314647194,2560.410773084753 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.98,3,0.2713742683212467,0.0019811518016786263,0.9816910785619175,0.005707865579055362,93161.92912534211,2584.0669442957055 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,0.99,3,0.27053969734098926,0.0014164533171887024,0.9897913892587661,0.0027895705925380728,93929.03981384847,2605.126706471309 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,original,原方法,1.0,3,0.27001009067886245,0.0,1.0,0.0,94895.80944867841,2631.667776610426 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.0,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.01,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.02,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.03,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.04,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.05,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.06,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.07,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.08,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.09,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.1,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.11,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.12,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.13,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.14,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.15,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.16,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.17,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.18,1,0.18129624718318377,0.0,0.0,0.0,194.418697728,31.79686254823173 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.19,1,0.1784481207469488,0.0,0.17942743009320905,0.0,17186.44586642383,498.2850192324936 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.2,1,0.1780928651788194,0.0,0.19340878828229027,0.0,18510.49993151701,534.6347457273712 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.21,1,0.17829989216687975,0.0,0.20306258322237017,0.0,19424.727738367063,559.7333664024056 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.22,1,0.1779854683890517,0.0,0.20838881491344874,0.0,19929.129286973992,573.5808812575971 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.23,1,0.17917657728013692,0.0,0.21537949400798934,0.0,20591.15631952058,591.7557445050359 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.24,1,0.17803687432776102,0.0,0.23135818908122505,0.0,22104.360965341362,633.2982890706102 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.25,1,0.18026802885144627,0.0,0.24800266311584554,0.0,23680.615804738005,676.5717729930834 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.26,1,0.18119827871024874,0.0,0.2589880159786951,0.0,24720.943998739793,705.1322723819159 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.27,1,0.17963665085316138,0.0,0.27430093209054596,0.0,26171.098450984704,744.9438775905912 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.28,1,0.17924095839307458,0.0,0.281624500665779,0.0,26864.65058031923,763.9842105164795 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.29,1,0.1789157174618718,0.0,0.28761651131824234,0.0,27432.102322502018,779.5626647285699 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.3,1,0.1787206845800119,0.0,0.2969374167776298,0.0,28314.805032564138,803.7958157251549 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.31,1,0.17813242189706796,0.0,0.30459387483355527,0.0,29039.882258686597,823.7016183294926 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.32,1,0.17772000842137498,0.0,0.3129161118508655,0.0,29828.009678384915,845.3383602907292 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.33,1,0.17791259734473341,0.0,0.31990679094540614,0.0,30490.036710931512,863.5132235381681 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.34,1,0.1789653934599337,0.0,0.3322237017310253,0.0,31656.465292085024,895.5356016407982 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.35,1,0.18042682200794885,0.0,0.34054593874833555,0.0,32444.59271178335,917.1723436020349 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.36,1,0.17962877325912951,0.0,0.3488681757656458,0.0,33232.72013148167,938.8090855632715 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.37,1,0.18158304685510065,0.0,0.3581890812250333,0.0,34115.42284154379,963.0422365598565 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.38,1,0.18069239723097547,0.0,0.37017310252996005,0.0,35250.32632590937,994.1991449840373 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.39,1,0.1812845803851709,0.0,0.37683089214380827,0.0,35880.82826166803,1011.5085385530266 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.4,1,0.1824781664689562,0.0,0.38348868175765644,0.0,36511.33019742669,1028.817932122016 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.41,1,0.18220594501399923,0.0,0.4021304926764314,0.0,38276.73561755093,1077.2842341151859 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.42,1,0.18162498930772586,0.0,0.41311584553928093,0.0,39317.06381155271,1105.8447335040182 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.43,1,0.18066434967152634,0.0,0.42676431424766975,0.0,40609.592779857965,1141.3289903204463 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.44,1,0.1803913789289444,0.0,0.4324234354194407,0.0,41145.51942525282,1156.0419748540871 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.45,1,0.17959468902865128,0.0,0.44107856191744343,0.0,41965.17194173908,1178.5441864937734 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.46,1,0.17863383514615866,0.0,0.45772303595206393,0.0,43541.42678113572,1221.8176704162468 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.47,1,0.17957308538379646,0.0,0.4717043941411451,0.0,44865.4808462289,1258.1673969111241 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.48,1,0.17910245145856155,0.0,0.4826897470039947,0.0,45905.80904023069,1286.7278962999567 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.49,1,0.17871833907402393,0.0,0.4896804260985353,0.0,46567.83607277728,1304.9027595473954 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.5,1,0.18015785955296285,0.0,0.5013315579227696,0.0,47671.21446035493,1335.1941982931266 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.51,1,0.1810878167493375,0.0,0.5116511318242344,0.0,48648.49246078086,1362.02375832506 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.52,1,0.18177962418520882,0.0,0.5183089214380826,0.0,49278.994396539514,1379.3331518940495 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.53,1,0.18246323539519374,0.0,0.5252996005326231,0.0,49941.0214290861,1397.5080151414882 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.54,1,0.1839876166906015,0.0,0.5396138482023968,0.0,51296.60059096721,1434.723211314815 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.55,1,0.1846674213907964,0.0,0.5452729693741678,0.0,51832.527236362075,1449.4361958484562 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.56,1,0.18676294233791663,0.0,0.559254327563249,0.0,53156.581301455255,1485.7859223433336 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.57,1,0.18938663221175195,0.0,0.572237017310253,0.0,54386.060076184636,1519.5392398028628 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.58,1,0.1909571002676625,0.0,0.5862183754993342,0.0,55710.114141277816,1555.8889662977404 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.59,1,0.1934300101790686,0.0,0.5998668442077231,0.0,57002.64310958307,1591.3732231141685 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.6,1,0.1973701682130997,0.0,0.6191744340878829,0.0,58831.098723283176,1641.5704644642374 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.61,1,0.1986727924998457,0.0,0.6264980026631158,0.0,59524.6508526177,1660.6107973901255 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.62,1,0.20186289191698192,0.0,0.6384820239680427,0.0,60659.55433698329,1691.7677058143065 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.63,1,0.20246031058520178,0.0,0.6461384820239681,0.0,61384.631563105744,1711.6735084186441 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.64,1,0.20349944137622245,0.0,0.653462050599201,0.0,62078.18369244026,1730.7138413445323 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.65,1,0.20442990090722246,0.0,0.6597869507323568,0.0,62677.160531410984,1747.157765235072 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.66,1,0.2044993942500819,0.0,0.6661118508655126,0.0,63276.13737038171,1763.601689125612 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.67,1,0.2064379542733177,0.0,0.6734354194407457,0.0,63969.68949971624,1782.6420220515004 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.68,1,0.2065831206227621,0.0,0.6820905459387483,0.0,64789.342016202485,1805.1442336911862 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.69,1,0.20802802394105854,0.0,0.6927430093209055,0.0,65798.14511341635,1832.8392634015693 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.7,1,0.21169680596278434,0.0,0.7090545938748336,0.0,67342.87485602507,1875.2472776455932 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.71,1,0.2139280527103987,0.0,0.7170439414114514,0.0,68099.47717893546,1896.0185499283803 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.72,1,0.2146900231025185,0.0,0.7243675099866844,0.0,68793.02930826998,1915.0588828542684 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.73,1,0.21653561577692484,0.0,0.7330226364846871,0.0,69612.68182475623,1937.5610944939547 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.74,1,0.21721689648133474,0.0,0.7363515312916112,0.0,69927.93279263556,1946.2157912784494 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.75,1,0.22103445955201576,0.0,0.7486684420772304,0.0,71094.36137378908,1978.2381693810794 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.76,1,0.22340134248367294,0.0,0.757323568575233,0.0,71914.01389027532,2000.7403810207654 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.77,1,0.2269322568854726,0.0,0.7639813581890812,0.0,72544.51582603398,2018.0497745897546 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.78,1,0.22757452798038888,0.0,0.7733022636484687,0.0,73427.2185360961,2042.2829255863396 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.79,1,0.227821050912831,0.0,0.784953395472703,0.0,74530.59692367376,2072.574364332071 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.8,1,0.22991865671924006,0.0,0.7939414114513982,0.0,75381.77453694795,2095.9420456502066 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.81,1,0.23486191981046528,0.0,0.8039280958721704,0.0,76327.52744058594,2121.906136003691 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.82,1,0.23803916047544685,0.0,0.8122503328894807,0.0,77115.65486028425,2143.5428779649274 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.83,1,0.24022947987632814,0.0,0.8215712383488681,0.0,77998.35757034637,2167.776028961512 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.84,1,0.24528627405740036,0.0,0.8388814913448736,0.0,79637.6626033189,2212.7804522408846 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.85,1,0.249537136326177,0.0,0.8535286284953395,0.0,81024.76686198794,2250.861118092661 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.86,1,0.25211074301800224,0.0,0.8638482023968043,0.0,82002.04486241385,2277.6906781245943 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.87,1,0.2540042317315216,0.0,0.8745006657789614,0.0,83010.8479596277,2305.3857078349774 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.88,1,0.2540996517958595,0.0,0.8764980026631158,0.0,83199.9985403553,2310.5785259056743 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.89,1,0.25436194678023316,0.0,0.8841544607190412,0.0,83925.07576647776,2330.484328510012 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.9,1,0.2534339447803836,0.0,0.8891478029294274,0.0,84397.95221829675,2343.4663736867537 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.91,1,0.2563240880787248,0.0,0.8981358189081226,0.0,85249.12983157096,2366.8340550048897 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.92,1,0.25551845731687656,0.0,0.9054593874833555,0.0,85942.68196090547,2385.8743879307776 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.93,1,0.2597490004762105,0.0,0.9221038615179761,0.0,87518.93680030212,2429.147871853251 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.94,1,0.26306323674307786,0.0,0.9254327563249002,0.0,87834.18776818145,2437.8025686377455 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.95,1,0.26218185491449153,0.0,0.9364181091877497,0.0,88874.51596218323,2466.363068026578 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.96,1,0.26357052137555514,0.0,0.953728362183755,0.0,90513.82099515574,2511.3674913059504 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.97,1,0.26400290307863034,0.0,0.96271637816245,0.0,91364.99860842993,2534.7351726240854 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.98,1,0.2647475724296899,0.0,0.9700399467376831,0.0,92058.55073776445,2553.775505549974 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,0.99,1,0.2649306125590882,0.0,0.987017310252996,0.0,93666.33067394902,2597.9144591508966 +construction|yolo26x|Qwen3.8,Construction / YOLO26x + Qwen3.8,construction,confidence,置信度路由,1.0,1,0.27001009067886245,0.0,1.0,0.0,94895.80944867841,2631.667776610426 diff --git a/releases/20260913_lr/paper_data/cost_20260911/grefcoco_router.json b/releases/20260913_lr/paper_data/cost_20260911/grefcoco_router.json new file mode 100644 index 0000000000000000000000000000000000000000..80b451c5f84b44c4671e69f113349972a947e4a8 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/grefcoco_router.json @@ -0,0 +1,27 @@ +{ + "component": "router", + "expert": "instancevg", + "task": "grefcoco", + "endpoint": "MiniCPM", + "condition": "both", + "seed": 42, + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_method_improvement_20260911_r1/conditions/both_instancevg_MiniCPM_seed42/best.pt", + "sample_count": 1024, + "batch_size": 256, + "row_count_loaded": 35263, + "mean_latency_ms_per_sample": 0.001830718741985038, + "median_latency_ms_per_sample": 0.0018171250121667981, + "latency_ms_per_sample": [ + 0.001949499943293631, + 0.0018388750031590462, + 0.00179537502117455, + 0.0017391250003129244 + ], + "flops_counted_per_batch": 0, + "gflops_counted_per_sample": 0.0, + "operators": {}, + "flops_scope": "GPU FlopCounterMode for the router forward; multiply-add counted as 2 FLOPs.", + "latency_scope": "A800 CUDA forward latency on cached router features, excluding feature loading and model load.", + "completed": true +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/instancevg_specialist.json b/releases/20260913_lr/paper_data/cost_20260911/instancevg_specialist.json new file mode 100644 index 0000000000000000000000000000000000000000..1a616c8b255c9b0393deb223230ee7ad395a142a --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/instancevg_specialist.json @@ -0,0 +1,134 @@ +{ + "component": "specialist", + "expert": "instancevg", + "task": "grefcoco", + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/models/grefcoco/instancevg/InstanceVG-grefcoco.pth", + "sample_count": 64, + "batch_size": 16, + "sample_ids": [ + "testA-30844-84861", + "testB-44353-122077", + "testB-36889-101643", + "testA-49055-135087", + "testA-81245-223633", + "testB-37669-103833", + "testA-5038-13943", + "testB-79646-219274", + "testA-469-1309", + "testB-46644-128438", + "testA-24562-67515", + "testA-47070-129599", + "testA-54554-150141", + "testB-12382-34099", + "testB-23014-63279", + "testB-79647-219275", + "testB-69479-191564", + "testA-15582-42797", + "testB-56692-156000", + "testB-42726-117660", + "testA-80455-221494", + "testB-33115-91174", + "testA-62940-173455", + "testB-31494-86670", + "testA-54586-150233", + "testA-81445-224177", + "testB-8417-23141", + "testA-24438-67190", + "testA-59285-163255", + "testA-73258-201893", + "testB-945-2633", + "testA-54307-149426", + "testB-60557-166822", + "testA-81622-224665", + "testA-69466-191525", + "testA-60731-167321", + "testA-7857-21644", + "testA-44142-121499", + "testB-66162-182407", + "testA-31764-87406", + "testB-38637-106482", + "testB-52921-145709", + "testB-65796-181394", + "testB-2936-8113", + "testA-79438-218690", + "testA-57397-158016", + "testB-23920-65743", + "testB-15419-42358", + "testA-10078-27739", + "testA-43890-120793", + "testA-55116-151689", + "testB-27487-75582", + "testA-26856-73830", + "testB-50845-140107", + "testA-29199-80374", + "testB-65935-181778", + "testA-17736-48684", + "testB-69477-191553", + "testB-22290-61227", + "testA-58476-160950", + "testB-36265-99859", + "testA-5734-15837", + "testB-67952-187353", + "testB-47054-129555" + ], + "model_load_seconds_excluded_from_latency": true, + "forward_latency_ms_per_sample": [ + 11.21629810333252, + 11.368107795715332, + 11.16999626159668, + 11.233466148376465 + ], + "end_to_end_latency_ms_per_sample": [ + 16.296884561597835, + 16.604527561867144, + 15.993722627172247, + 16.3729631822207 + ], + "mean_forward_latency_ms_per_sample": 11.246967077255249, + "median_forward_latency_ms_per_sample": 11.224882125854492, + "mean_end_to_end_latency_ms_per_sample": 16.31702448321448, + "median_end_to_end_latency_ms_per_sample": 16.334923871909268, + "flop_profiles": [ + { + "batch_size": 16, + "flops_counted": 120042881024, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 120010113024, + "aten.baddbmm": 32768000 + } + }, + { + "batch_size": 16, + "flops_counted": 120042881024, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 120010113024, + "aten.baddbmm": 32768000 + } + }, + { + "batch_size": 16, + "flops_counted": 120042881024, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 120010113024, + "aten.baddbmm": 32768000 + } + }, + { + "batch_size": 16, + "flops_counted": 120042881024, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 120010113024, + "aten.baddbmm": 32768000 + } + } + ], + "mean_counted_gflops_per_sample": 7.502680064, + "flops_scope": "GPU FlopCounterMode dense products, convolutions and supported attention; multiply-add counted as 2 FLOPs. Elementwise, normalization, custom ops and postprocessing may be missing.", + "latency_scope": "A800 wall-clock CUDA forward latency and end-to-end latency including image read/preprocess, excluding model load; batch processing uses the recorded batch size.", + "completed": true +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/make_cost_analysis_20260911.py b/releases/20260913_lr/paper_data/cost_20260911/make_cost_analysis_20260911.py new file mode 100644 index 0000000000000000000000000000000000000000..29a38156111022bf1eab8143a59b4140bbad7851 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/make_cost_analysis_20260911.py @@ -0,0 +1,457 @@ +"""Build the A800 cost--performance supplement for two representative pairs. + +The script reads the frozen test curves and the measured A800 component records. +It does not select a test-set operating point: the selected points are copied from +the validation-locked records, and all curve values retain their nominal budget. +""" + +from __future__ import annotations + +import argparse +import csv +import json +from pathlib import Path + +import matplotlib.pyplot as plt +import numpy as np + + +PAIR_FILES = { + "grefcoco|instancevg|MiniCPM": { + "label": "gRefCOCO / InstanceVG + MiniCPM", + "specialist": "instancevg_specialist.json", + "router": "grefcoco_router.json", + "large_latency": "minicpm_grefcoco.json", + "large_flops": "minicpm_grefcoco_flops.json", + }, + "construction|yolo26x|Qwen3.8": { + "label": "Construction / YOLO26x + Qwen3.8", + "specialist": "yolo26x_specialist.json", + "router": "construction_router.json", + "large_latency": "qwen_construction.json", + "large_flops": "qwen_construction_flops.json", + }, +} + +METHOD_LABELS = { + "both": "两项同时改", + "original": "原方法", + "confidence": "置信度路由", +} +PLOT_LABELS = {"both": "Two changes", "original": "Original", "confidence": "Confidence"} +PLOT_METHODS = ("both", "original", "confidence") +PLOT_COLORS = {"both": "#2166ac", "original": "#d6604d", "confidence": "#252525"} +PLOT_STYLES = {"both": "-", "original": "--", "confidence": ":"} + + +def read_json(path: Path): + return json.loads(path.read_text(encoding="utf-8")) + + +def write_json(path: Path, value) -> None: + path.write_text(json.dumps(value, ensure_ascii=False, indent=2), encoding="utf-8") + + +def mean_std(values): + values = np.asarray(values, dtype=float) + return float(values.mean()), float(values.std(ddof=0)) + + +def router_flops(architecture: dict) -> tuple[int, dict[str, int]]: + d = int(architecture["representation_dim"]) + branch_dims = architecture["input_dimensions"] + branch_costs = {} + for branch in architecture["branches"]: + input_key = "image_flat_plus_valid" if branch == "image" else branch + branch_costs[branch] = 2 * int(branch_dims[input_key]) * d + head_input = len(architecture["branches"]) * d + branch_costs["head_input_to_hidden"] = 2 * head_input * int(architecture["head_hidden_dim"]) + branch_costs["head_hidden_to_classes"] = 2 * int(architecture["head_hidden_dim"]) * int(architecture["class_count"]) + return sum(branch_costs.values()), branch_costs + + +def build_cost_components(root: Path, architectures: list[dict]) -> dict[str, dict]: + components = {} + for architecture in architectures: + key = architecture["pair_key"] + files = PAIR_FILES[key] + specialist = read_json(root / files["specialist"]) + router = read_json(root / files["router"]) + large_latency = read_json(root / files["large_latency"]) + large_flops = read_json(root / files["large_flops"]) + router_flop_count, router_breakdown = router_flops(architecture) + completion_tokens = [ + output["completion_tokens"] + for batch in large_latency["batches"] + for output in batch["outputs"] + ] + prefill = float(large_flops["mean_prefill_gflops_counted"]) + decode = float(large_flops["mean_decode_one_token_gflops_counted"]) + mean_tokens = float(large_latency["mean_completion_tokens"]) + large_estimated = prefill + decode * mean_tokens + components[key] = { + "pair_key": key, + "task": architecture["task"], + "expert": architecture["expert"], + "endpoint": architecture["endpoint"], + "label": PAIR_FILES[key]["label"], + "specialist_gflops_per_input": float(specialist["mean_counted_gflops_per_sample"]), + "specialist_forward_latency_ms": float(specialist["mean_forward_latency_ms_per_sample"]), + "specialist_end_to_end_latency_ms": float(specialist["mean_end_to_end_latency_ms_per_sample"]), + "specialist_sample_count": int(specialist["sample_count"]), + "router_gflops_per_input": router_flop_count / 1e9, + "router_flop_count": router_flop_count, + "router_flop_breakdown": router_breakdown, + "router_measured_gflops_per_input": float(router["gflops_counted_per_sample"]), + "router_latency_ms": float(router["mean_latency_ms_per_sample"]), + "router_sample_count": int(router["sample_count"]), + "large_prefill_gflops": prefill, + "large_decode_one_token_gflops": decode, + "large_mean_completion_tokens": mean_tokens, + "large_completion_token_min": int(min(completion_tokens)), + "large_completion_token_max": int(max(completion_tokens)), + "large_estimated_full_generation_gflops": large_estimated, + "large_batch_wall_latency_ms_per_input": float(large_latency["mean_batch_wall_ms_per_sample"]), + "large_measured_sample_count": int(large_latency["measured_count"]), + "large_flops_probe_count": int(large_flops["sample_count"]), + "flops_formula": "specialist + router + actual_call_fraction * (prefill + mean_completion_tokens * one_token_decode)", + "latency_formula": "specialist end-to-end + router + actual_call_fraction * large-model batch-wall latency", + "specialist_flops_scope": specialist["flops_scope"], + "specialist_latency_scope": specialist["latency_scope"], + "large_flops_scope": large_flops["scope"], + "large_latency_scope": large_latency["latency_scope"], + } + return components + + +def load_relevant_groups(results: dict, pair_key: str) -> dict[str, list[dict]]: + task, expert, endpoint = pair_key.split("|") + groups = { + method: [] + for method in PLOT_METHODS + } + for group in results["groups"]: + if (group["task"], group["expert"], group["endpoint"]) != (task, expert, endpoint): + continue + if group["method"] in groups: + groups[group["method"]].append(group) + for method, entries in groups.items(): + if not entries: + raise ValueError(f"missing {method} group for {pair_key}") + return groups + + +def curve_rows(groups: dict[str, list[dict]], pair_key: str, components: dict) -> list[dict]: + rows = [] + for method, entries in groups.items(): + per_budget = {} + for group in entries: + curve = group["deployments"]["endpoint"]["test_curve"] + for point in curve: + budget = round(float(point["budget"]), 10) + per_budget.setdefault(budget, []).append(point) + for budget in sorted(per_budget): + points = per_budget[budget] + metrics = [float(point["metric"]) for point in points] + calls = [float(point["actual_fraction"]) for point in points] + metric_mean, metric_std = mean_std(metrics) + call_mean, call_std = mean_std(calls) + component = components[pair_key] + rows.append({ + "pair_key": pair_key, + "pair_label": component["label"], + "task": component["task"], + "method": method, + "method_label": METHOD_LABELS[method], + "nominal_budget": budget, + "seed_count": len(points), + "test_metric": metric_mean, + "test_metric_std": metric_std, + "actual_test_call_fraction": call_mean, + "actual_test_call_fraction_std": call_std, + "flops_gflops_per_input": component["specialist_gflops_per_input"] + component["router_gflops_per_input"] + call_mean * component["large_estimated_full_generation_gflops"], + "latency_ms_per_input": component["specialist_end_to_end_latency_ms"] + component["router_latency_ms"] + call_mean * component["large_batch_wall_latency_ms_per_input"], + }) + return rows + + +def selected_rows(groups: dict[str, list[dict]], pair_key: str, components: dict) -> list[dict]: + rows = [] + component = components[pair_key] + for method, entries in groups.items(): + points = [entry["deployments"]["endpoint"]["test_point"] for entry in entries] + metrics = [float(point["metric"]) for point in points] + calls = [float(point["actual_fraction"]) for point in points] + budgets = [float(point["budget"]) for point in points] + metric_mean, metric_std = mean_std(metrics) + call_mean, call_std = mean_std(calls) + rows.append({ + "pair_key": pair_key, + "pair_label": component["label"], + "task": component["task"], + "method": method, + "method_label": METHOD_LABELS[method], + "seed_count": len(points), + "selected_nominal_budget_mean": float(np.mean(budgets)), + "selected_nominal_budgets": budgets, + "test_metric": metric_mean, + "test_metric_std": metric_std, + "actual_test_call_fraction": call_mean, + "actual_test_call_fraction_std": call_std, + "flops_gflops_per_input": component["specialist_gflops_per_input"] + component["router_gflops_per_input"] + call_mean * component["large_estimated_full_generation_gflops"], + "latency_ms_per_input": component["specialist_end_to_end_latency_ms"] + component["router_latency_ms"] + call_mean * component["large_batch_wall_latency_ms_per_input"], + }) + return rows + + +def fixed_budget_rows(curves: list[dict], budgets=(0.0, 0.01, 0.05, 0.10, 0.25, 0.50, 0.75, 0.99, 1.0)) -> list[dict]: + wanted = {round(float(budget), 10) for budget in budgets} + return [row for row in curves if round(float(row["nominal_budget"]), 10) in wanted] + + +def write_csv(path: Path, rows: list[dict]) -> None: + if not rows: + raise ValueError(f"no rows for {path}") + fields = list(rows[0]) + with path.open("w", encoding="utf-8", newline="") as handle: + writer = csv.DictWriter(handle, fieldnames=fields) + writer.writeheader() + writer.writerows(rows) + + +def make_plot(curves: list[dict], selected: list[dict], out_path: Path, cost_field: str, x_label: str, log_x: bool) -> None: + fig, axes = plt.subplots(1, 2, figsize=(7.2, 2.9), constrained_layout=True) + pair_keys = list(PAIR_FILES) + for index, (ax, pair_key) in enumerate(zip(axes, pair_keys)): + pair_rows = [row for row in curves if row["pair_key"] == pair_key] + all_y = [] + for method in PLOT_METHODS: + method_rows = [row for row in pair_rows if row["method"] == method] + x = np.asarray([row[cost_field] for row in method_rows], dtype=float) + y = 100 * np.asarray([row["test_metric"] for row in method_rows], dtype=float) + y_std = 100 * np.asarray([row["test_metric_std"] for row in method_rows], dtype=float) + all_y.extend((y - y_std).tolist()) + all_y.extend((y + y_std).tolist()) + label = PLOT_LABELS[method] + ax.plot(x, y, PLOT_STYLES[method], color=PLOT_COLORS[method], linewidth=1.25, label=label) + if method in ("both", "original") and np.any(y_std > 0): + ax.fill_between(x, y - y_std, y + y_std, color=PLOT_COLORS[method], alpha=0.10, linewidth=0) + selected_row = next(row for row in selected if row["pair_key"] == pair_key and row["method"] == method) + ax.scatter( + [selected_row[cost_field]], + [100 * selected_row["test_metric"]], + color=PLOT_COLORS[method], + marker="*", + s=42, + edgecolor="white", + linewidth=0.45, + zorder=5, + ) + y_min, y_max = min(all_y), max(all_y) + y_pad = max(0.05 * (y_max - y_min), 0.2) + ax.set_ylim(y_min - y_pad, y_max + 2.0 * y_pad) + task_label = "gRefCOCO" if PAIR_FILES[pair_key]["label"].startswith("gRefCOCO") else "Construction" + ax.text(0.02, 0.98, f"({'ab'[index]}) {task_label}", transform=ax.transAxes, ha="left", va="top", fontsize=8.5) + ax.set_xlabel(x_label) + ax.set_ylabel("Test performance (%)") + ax.grid(True, color="#d9d9d9", linewidth=0.45, alpha=0.65) + ax.spines["top"].set_visible(False) + ax.spines["right"].set_visible(False) + if log_x: + ax.set_xscale("log") + ax.tick_params(labelsize=8) + handles, labels = axes[0].get_legend_handles_labels() + fig.legend(handles, labels, loc="lower center", bbox_to_anchor=(0.5, -0.045), ncol=3, frameon=False, fontsize=8) + fig.savefig(out_path, format="pdf", bbox_inches="tight", pad_inches=0.04) + fig.savefig(out_path.with_suffix(".png"), format="png", dpi=300, bbox_inches="tight", pad_inches=0.04) + plt.close(fig) + + +def fmt(value: float, digits=3) -> str: + return f"{value:.{digits}f}" + + +def pct(value: float, digits=2) -> str: + return f"{100 * value:.{digits}f}%" + + +def report_markdown(components: dict, selected: list[dict], fixed: list[dict], root: Path, summary: dict) -> str: + selected_by_key = {(row["pair_key"], row["method"]): row for row in selected} + lines = [ + "# A800成本—性能补充报告(2026-09-11)", + "", + "## 1. 做了什么", + "", + "- 在A800上补测了正文中两个代表性模型对:gRefCOCO的InstanceVG + MiniCPM,以及Construction的YOLO26x + Qwen3.8。", + "- 分别实测了小模型端、路由器端和大模型端的延迟与可计数运算量;大模型端延迟采用批处理均摊后的真实墙钟时间。", + "- 从已经锁定的测试曲线中提取代表性调用预算,按测试集实际调用率换算每个输入的成本,并生成完整曲线。没有用测试集重新挑选模型、阈值或预算。", + "", + "## 2. 为什么做", + "", + "正文原有成本图只是代理估算,缺少InstanceVG、完整解码和真实延迟。此次补测用于回答一个更直接的问题:两项同时改相对置信度路由的性能提升,是否值得它增加的计算和时间成本。", + "", + "## 3. 原始实测成本", + "", + "| 模型对 | 小模型端到端延迟(毫秒/样本) | 小模型可计数运算量(GFLOPs/样本) | 路由器延迟(毫秒/样本) | 路由器稠密运算量(GFLOPs/样本) | 大模型调用延迟(毫秒/样本) | 大模型完整生成估算(GFLOPs/调用) |", + "|---|---:|---:|---:|---:|---:|---:|", + ] + for key, component in components.items(): + lines.append( + f"| {component['label']} | {fmt(component['specialist_end_to_end_latency_ms'])} | {fmt(component['specialist_gflops_per_input'])} | {fmt(component['router_latency_ms'], 6)} | {fmt(component['router_gflops_per_input'], 6)} | {fmt(component['large_batch_wall_latency_ms_per_input'])} | {fmt(component['large_estimated_full_generation_gflops'])} |" + ) + lines.extend([ + "", + "说明:GFLOPs按乘加计2次运算,只覆盖计数器能识别的稠密乘法、卷积和注意力等算子;路由器的计数器返回0,因此按已加载检查点的线性层结构精确计算了稠密乘加量。大模型完整生成GFLOPs为“预填充GFLOPs + 平均生成长度 × 单Token解码GFLOPs”的估算,不把未计数算子冒充成完整硬件计数。", + "", + "## 4. 验证集锁定点在测试集上的结果", + "", + "下表的均值和标准差只在三个随机种子之间统计。成本使用对应的平均实际调用率;星号表示验证集选出的运行点。", + "", + "| 模型对 | 方法 | 测试性能 | 实际调用率 | 估算GFLOPs/输入 | 端到端延迟(毫秒/输入) |", + "|---|---|---:|---:|---:|---:|", + ]) + for key in components: + for method in ("both", "original", "confidence"): + row = selected_by_key[(key, method)] + metric = f"{100 * row['test_metric']:.2f}±{100 * row['test_metric_std']:.2f}%" + calls = f"{100 * row['actual_test_call_fraction']:.2f}±{100 * row['actual_test_call_fraction_std']:.2f}%" + lines.append(f"| {row['pair_label']} | {row['method_label']}* | {metric} | {calls} | {fmt(row['flops_gflops_per_input'])} | {fmt(row['latency_ms_per_input'])} |") + lines.extend([ + "", + "### 两项同时改相对置信度路由", + "", + "| 模型对 | 性能变化 | 调用率变化 | 估算GFLOPs变化 | 实际延迟变化 |", + "|---|---:|---:|---:|---:|", + ]) + for key in components: + learned = selected_by_key[(key, "both")] + confidence = selected_by_key[(key, "confidence")] + metric_delta = 100 * (learned["test_metric"] - confidence["test_metric"]) + call_delta = 100 * (learned["actual_test_call_fraction"] - confidence["actual_test_call_fraction"]) + flop_delta = 100 * (learned["flops_gflops_per_input"] / confidence["flops_gflops_per_input"] - 1) + latency_delta = 100 * (learned["latency_ms_per_input"] / confidence["latency_ms_per_input"] - 1) + lines.append(f"| {learned['pair_label']} | {metric_delta:+.2f} 个百分点 | {call_delta:+.2f} 个百分点 | {flop_delta:+.1f}% | {latency_delta:+.1f}% |") + summary[key]["both_vs_confidence"] = { + "metric_delta_percentage_points": metric_delta, + "call_delta_percentage_points": call_delta, + "flops_relative_percent": flop_delta, + "latency_relative_percent": latency_delta, + } + lines.extend([ + "", + "直白地说:两项同时改在这两个代表性模型对上都提高了测试性能,同时减少了实际调用,因此计算量和延迟也一起下降。原因不是路由器本身更快,而是它让大模型少被调用;路由器自身成本相对很小。", + "", + "## 5. 固定名义调用预算的代表点", + "", + "名义预算是验证阶段设定的阈值预算;阈值原样转到测试集后,实际调用率会有偏差。完整的99个阈值点和端点均在CSV中。", + "", + "| 模型对 | 名义预算 | 两项同时改:性能 / 实际调用 | 置信度:性能 / 实际调用 | 两项同时改:GFLOPs/输入 | 两项同时改:毫秒/输入 |", + "|---|---:|---:|---:|---:|---:|", + ]) + for key in components: + rows = {(row["nominal_budget"], row["method"]): row for row in fixed if row["pair_key"] == key} + for budget in (0.25, 0.50): + learned = rows[(budget, "both")] + confidence = rows[(budget, "confidence")] + lines.append( + f"| {components[key]['label']} | {100 * budget:.0f}% | {100 * learned['test_metric']:.2f}% / {100 * learned['actual_test_call_fraction']:.2f}% | {100 * confidence['test_metric']:.2f}% / {100 * confidence['actual_test_call_fraction']:.2f}% | {fmt(learned['flops_gflops_per_input'])} | {fmt(learned['latency_ms_per_input'])} |" + ) + lines.extend([ + "", + "## 6. 结果怎么理解", + "", + "1. gRefCOCO的InstanceVG + MiniCPM是这两组中相对置信度提升更明显的一组;Construction的YOLO26x + Qwen3.8也有稳定的正向变化。", + "2. 这里的成本是按真实测试调用率换算的平均每输入成本,不是把名义预算当成实际调用率。", + "3. 大模型延迟是A800、批量4下的均摊值,模型加载和预热没有计入;实际系统的并发度、批量大小和请求排队会改变墙钟延迟。", + "4. InstanceVG使用了远端已有的MMCV等价算子兼容路径,因为原生detrex扩展不可用;这不改变本次模型输出链路,但应在复现实验时保留该环境说明。", + "5. 本补充不修改正文,也不覆盖旧实验结果;它是针对两个代表性模型对的成本实测和成本—性能换算。", + "", + "## 7. 文件在哪里", + "", + f"- 原始A800测量记录:`{root.as_posix()}`目录下的8个JSON文件。", + f"- 完整成本—性能曲线数据:`{(root / 'full_cost_performance_curves.csv').as_posix()}`。", + f"- 固定预算代表点:`{(root / 'fixed_budget_cost_performance.csv').as_posix()}`。", + f"- FLOPs—性能图:`{(root / 'flops_performance.pdf').as_posix()}`;实际延迟—性能图:`{(root / 'latency_performance.pdf').as_posix()}`。", + f"- 可复现脚本:`{(root / 'make_cost_analysis_20260911.py').as_posix()}`;插入论文的LaTeX片段:`{(root / 'latex_includes.tex').as_posix()}`。", + ]) + return "\n".join(lines) + "\n" + + +def latex_includes() -> str: + return r"""% A800 cost--performance supplement (generated 2026-09-11) +\begin{figure*}[t] + \centering + \includegraphics[width=0.98\textwidth]{dse_results/method_improvement_20260911/cost_analysis_20260911/flops_performance.pdf} + \caption{A800 counted-FLOPs--performance curves for the two representative model pairs. Each point uses a validation-calibrated nominal budget transferred unchanged to test; the horizontal coordinate uses the resulting actual test call rate. The star marks the validation-selected operating point. Large-model full-generation FLOPs are estimated from measured prefill FLOPs, one-token decode FLOPs and the measured mean completion length; unsupported operations are not included.} + \label{fig:a800-flops-performance} +\end{figure*} + +\begin{figure*}[t] + \centering + \includegraphics[width=0.98\textwidth]{dse_results/method_improvement_20260911/cost_analysis_20260911/latency_performance.pdf} + \caption{A800 end-to-end latency--performance curves. Specialist and router latency are always paid; the large-model term is weighted by the actual test call rate and uses the measured batch-wall latency per input. Model loading and warm-up are excluded.} + \label{fig:a800-latency-performance} +\end{figure*} +""" + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--experiment-dir", type=Path, default=Path(__file__).resolve().parent.parent) + parser.add_argument("--output-dir", type=Path, default=Path(__file__).resolve().parent) + args = parser.parse_args() + root = args.output_dir.resolve() + experiment_dir = args.experiment_dir.resolve() + root.mkdir(parents=True, exist_ok=True) + + architectures = read_json(root / "router_architecture.json") + architecture_keys = {item["pair_key"] for item in architectures} + if architecture_keys != set(PAIR_FILES): + raise ValueError("router architecture keys do not match selected pairs") + results = read_json(experiment_dir / "test_results.json") + components = build_cost_components(root, architectures) + all_curves = [] + all_selected = [] + for pair_key in PAIR_FILES: + groups = load_relevant_groups(results, pair_key) + all_curves.extend(curve_rows(groups, pair_key, components)) + all_selected.extend(selected_rows(groups, pair_key, components)) + all_fixed = fixed_budget_rows(all_curves) + + write_csv(root / "full_cost_performance_curves.csv", all_curves) + write_csv(root / "fixed_budget_cost_performance.csv", all_fixed) + make_plot(all_curves, all_selected, root / "flops_performance.pdf", "flops_gflops_per_input", "Estimated GFLOPs / input", True) + make_plot(all_curves, all_selected, root / "latency_performance.pdf", "latency_ms_per_input", "End-to-end latency (ms / input)", False) + (root / "latex_includes.tex").write_text(latex_includes(), encoding="utf-8") + + summary = { + "status": "complete", + "protocol": { + "test_results": str((experiment_dir / "test_results.json").resolve()), + "curves_use": "validation-calibrated thresholds transferred unchanged to test", + "cost_uses": "actual test call fraction", + "flop_counting": "multiply-add counted as 2 FLOPs; unsupported operations excluded", + "large_generation_flops": "mean prefill + mean completion tokens * one-token decode probe", + "seed_aggregation": "mean and population standard deviation across three seeds for learned/original methods", + }, + "components": components, + "selected_points": all_selected, + "fixed_budget_points": all_fixed, + } + for key in components: + summary.setdefault("comparisons", {}) + learned = next(row for row in all_selected if row["pair_key"] == key and row["method"] == "both") + confidence = next(row for row in all_selected if row["pair_key"] == key and row["method"] == "confidence") + summary["comparisons"][key] = { + "both_vs_confidence_metric_delta_percentage_points": 100 * (learned["test_metric"] - confidence["test_metric"]), + "both_vs_confidence_call_delta_percentage_points": 100 * (learned["actual_test_call_fraction"] - confidence["actual_test_call_fraction"]), + "both_vs_confidence_flops_relative_percent": 100 * (learned["flops_gflops_per_input"] / confidence["flops_gflops_per_input"] - 1), + "both_vs_confidence_latency_relative_percent": 100 * (learned["latency_ms_per_input"] / confidence["latency_ms_per_input"] - 1), + } + write_json(root / "cost_summary.json", summary) + (root / "成本性能补充报告.md").write_text(report_markdown(components, all_selected, all_fixed, root, summary["comparisons"]), encoding="utf-8") + print(json.dumps({"status": "complete", "output_dir": str(root), "curve_rows": len(all_curves), "fixed_rows": len(all_fixed)}, ensure_ascii=False)) + + +if __name__ == "__main__": + main() diff --git a/releases/20260913_lr/paper_data/cost_20260911/minicpm_grefcoco.json b/releases/20260913_lr/paper_data/cost_20260911/minicpm_grefcoco.json new file mode 100644 index 0000000000000000000000000000000000000000..b5eaa052393dd3863740062808986f2109077a65 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/minicpm_grefcoco.json @@ -0,0 +1,385 @@ +{ + "status": "complete", + "model_kind": "MiniCPM", + "task": "grefcoco", + "expert": "instancevg", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/MiniCPM-V-4_5", + "device": "cuda:0", + "gpu_memory_utilization": 0.75, + "sample_count_requested": 32, + "warmup_count": 2, + "measured_count": 30, + "batch_size": 4, + "sample_ids": [ + "testA-24562-67519", + "testB-37093-102233", + "testB-23966-65871", + "testB-66119-182290", + "testA-81450-224192", + "testB-33122-91196", + "testB-80267-220980", + "testA-47140-129791", + "testA-24439-67195", + "testA-54310-149436", + "testA-79491-218834", + "testA-10078-27741", + "testB-57040-156982", + "testA-26858-73835", + "testB-44430-122263", + "testA-60760-167399", + "testB-3182-8806", + "testB-947-2650", + "testA-73301-202004", + "testA-29200-80379", + "testB-42734-117684", + "testB-46676-128533", + "testB-8468-23291", + "testA-54557-150151", + "testA-58479-160960", + "testA-81251-223648", + "testA-5038-13944", + "testB-37760-104089", + "testA-62943-173467", + "testB-12467-34323", + "testA-80534-221720", + "testB-53121-146261" + ], + "batches": [ + { + "sample_ids": [ + "testB-23966-65871", + "testB-66119-182290", + "testA-81450-224192", + "testB-33122-91196" + ], + "batch_size": 4, + "generation_latency_seconds": 6.051569456933066, + "batch_wall_latency_seconds": 6.090657560969703, + "amortized_generation_ms_per_sample": 1512.8923642332666, + "amortized_batch_wall_ms_per_sample": 1522.6643902424257, + "outputs": [ + { + "finish_reason": "length", + "stop_reason": null, + "prompt_tokens": 528, + "completion_tokens": 512, + "text_characters": 528 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 526, + "completion_tokens": 87, + "text_characters": 140 + }, + { + "finish_reason": "length", + "stop_reason": null, + "prompt_tokens": 523, + "completion_tokens": 512, + "text_characters": 528 + }, + { + "finish_reason": "length", + "stop_reason": null, + "prompt_tokens": 527, + "completion_tokens": 512, + "text_characters": 528 + } + ] + }, + { + "sample_ids": [ + "testB-80267-220980", + "testA-47140-129791", + "testA-24439-67195", + "testA-54310-149436" + ], + "batch_size": 4, + "generation_latency_seconds": 6.01708423695527, + "batch_wall_latency_seconds": 6.064165806979872, + "amortized_generation_ms_per_sample": 1504.2710592388175, + "amortized_batch_wall_ms_per_sample": 1516.041451744968, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 525, + "completion_tokens": 5, + "text_characters": 14 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 524, + "completion_tokens": 29, + "text_characters": 52 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 524, + "completion_tokens": 33, + "text_characters": 56 + }, + { + "finish_reason": "length", + "stop_reason": null, + "prompt_tokens": 522, + "completion_tokens": 512, + "text_characters": 528 + } + ] + }, + { + "sample_ids": [ + "testA-79491-218834", + "testA-10078-27741", + "testB-57040-156982", + "testA-26858-73835" + ], + "batch_size": 4, + "generation_latency_seconds": 6.010764242964797, + "batch_wall_latency_seconds": 6.046397175989114, + "amortized_generation_ms_per_sample": 1502.6910607411992, + "amortized_batch_wall_ms_per_sample": 1511.5992939972784, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 524, + "completion_tokens": 62, + "text_characters": 100 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 529, + "completion_tokens": 87, + "text_characters": 140 + }, + { + "finish_reason": "length", + "stop_reason": null, + "prompt_tokens": 528, + "completion_tokens": 512, + "text_characters": 1690 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 521, + "completion_tokens": 5, + "text_characters": 14 + } + ] + }, + { + "sample_ids": [ + "testB-44430-122263", + "testA-60760-167399", + "testB-3182-8806", + "testB-947-2650" + ], + "batch_size": 4, + "generation_latency_seconds": 2.593448459985666, + "batch_wall_latency_seconds": 2.6335477660177276, + "amortized_generation_ms_per_sample": 648.3621149964165, + "amortized_batch_wall_ms_per_sample": 658.3869415044319, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 525, + "completion_tokens": 6, + "text_characters": 14 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 392, + "completion_tokens": 33, + "text_characters": 56 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 392, + "completion_tokens": 33, + "text_characters": 56 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 528, + "completion_tokens": 208, + "text_characters": 321 + } + ] + }, + { + "sample_ids": [ + "testA-73301-202004", + "testA-29200-80379", + "testB-42734-117684", + "testB-46676-128533" + ], + "batch_size": 4, + "generation_latency_seconds": 6.017314173979685, + "batch_wall_latency_seconds": 6.065163187915459, + "amortized_generation_ms_per_sample": 1504.3285434949212, + "amortized_batch_wall_ms_per_sample": 1516.2907969788648, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 525, + "completion_tokens": 61, + "text_characters": 99 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 520, + "completion_tokens": 5, + "text_characters": 14 + }, + { + "finish_reason": "length", + "stop_reason": null, + "prompt_tokens": 523, + "completion_tokens": 512, + "text_characters": 528 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 528, + "completion_tokens": 5, + "text_characters": 14 + } + ] + }, + { + "sample_ids": [ + "testB-8468-23291", + "testA-54557-150151", + "testA-58479-160960", + "testA-81251-223648" + ], + "batch_size": 4, + "generation_latency_seconds": 1.6802827849751338, + "batch_wall_latency_seconds": 1.716626849025488, + "amortized_generation_ms_per_sample": 420.07069624378346, + "amortized_batch_wall_ms_per_sample": 429.156712256372, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 539, + "completion_tokens": 62, + "text_characters": 100 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 522, + "completion_tokens": 30, + "text_characters": 53 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 529, + "completion_tokens": 6, + "text_characters": 14 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 525, + "completion_tokens": 122, + "text_characters": 190 + } + ] + }, + { + "sample_ids": [ + "testA-5038-13944", + "testB-37760-104089", + "testA-62943-173467", + "testB-12467-34323" + ], + "batch_size": 4, + "generation_latency_seconds": 5.998426596052013, + "batch_wall_latency_seconds": 6.038230134989135, + "amortized_generation_ms_per_sample": 1499.6066490130033, + "amortized_batch_wall_ms_per_sample": 1509.5575337472837, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 522, + "completion_tokens": 39, + "text_characters": 64 + }, + { + "finish_reason": "length", + "stop_reason": null, + "prompt_tokens": 536, + "completion_tokens": 512, + "text_characters": 528 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 538, + "completion_tokens": 33, + "text_characters": 56 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 522, + "completion_tokens": 5, + "text_characters": 14 + } + ] + }, + { + "sample_ids": [ + "testA-80534-221720", + "testB-53121-146261" + ], + "batch_size": 2, + "generation_latency_seconds": 5.806203712942079, + "batch_wall_latency_seconds": 5.831518966006115, + "amortized_generation_ms_per_sample": 2903.1018564710394, + "amortized_batch_wall_ms_per_sample": 2915.7594830030575, + "outputs": [ + { + "finish_reason": "length", + "stop_reason": null, + "prompt_tokens": 390, + "completion_tokens": 512, + "text_characters": 528 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 523, + "completion_tokens": 33, + "text_characters": 56 + } + ] + } + ], + "mean_generation_ms_per_sample": 1436.915543054056, + "median_generation_ms_per_sample": 1503.4810599900084, + "mean_batch_wall_ms_per_sample": 1447.4320754343353, + "median_batch_wall_ms_per_sample": 1513.8203728711233, + "mean_prompt_tokens": 512.6666666666666, + "mean_completion_tokens": 169.5, + "latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.", + "flops_scope": "This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/minicpm_grefcoco_flops.json b/releases/20260913_lr/paper_data/cost_20260911/minicpm_grefcoco_flops.json new file mode 100644 index 0000000000000000000000000000000000000000..70e2cf17ccbbee36e612e639c7c0575d9a63fe99 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/minicpm_grefcoco_flops.json @@ -0,0 +1,117 @@ +{ + "status": "complete", + "model_kind": "MiniCPM", + "task": "grefcoco", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/MiniCPM-V-4_5", + "sample_count": 4, + "records": [ + { + "sample_id": "testB-37890-104442", + "prompt_tokens_in_reference_tensor": 528, + "input_shapes": { + "input_ids": [ + 1, + 528 + ], + "attention_mask": [ + 1, + 528 + ] + }, + "prefill_flops_counted": 7500407080960, + "decode_one_token_flops_counted": 15446671488, + "prefill_gflops_counted": 7500.40708096, + "decode_one_token_gflops_counted": 15.446671488, + "prefill_operators": { + "aten.bmm": 164433561600, + "aten.mm": 7335973519360 + }, + "decode_operators": { + "aten.bmm": 312017024, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testA-73305-202016", + "prompt_tokens_in_reference_tensor": 525, + "input_shapes": { + "input_ids": [ + 1, + 525 + ], + "attention_mask": [ + 1, + 525 + ] + }, + "prefill_flops_counted": 7456869222016, + "decode_one_token_flops_counted": 15444902016, + "prefill_gflops_counted": 7456.869222016, + "decode_one_token_gflops_counted": 15.444902016, + "prefill_operators": { + "aten.bmm": 162570307200, + "aten.mm": 7294298914816 + }, + "decode_operators": { + "aten.bmm": 310247552, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testB-24036-66087", + "prompt_tokens_in_reference_tensor": 522, + "input_shapes": { + "input_ids": [ + 1, + 522 + ], + "attention_mask": [ + 1, + 522 + ] + }, + "prefill_flops_counted": 7413341979904, + "decode_one_token_flops_counted": 15443132544, + "prefill_gflops_counted": 7413.341979904, + "decode_one_token_gflops_counted": 15.443132544, + "prefill_operators": { + "aten.bmm": 160717669632, + "aten.mm": 7252624310272 + }, + "decode_operators": { + "aten.bmm": 308478080, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testA-5038-13945", + "prompt_tokens_in_reference_tensor": 528, + "input_shapes": { + "input_ids": [ + 1, + 528 + ], + "attention_mask": [ + 1, + 528 + ] + }, + "prefill_flops_counted": 7500407080960, + "decode_one_token_flops_counted": 15446671488, + "prefill_gflops_counted": 7500.40708096, + "decode_one_token_gflops_counted": 15.446671488, + "prefill_operators": { + "aten.bmm": 164433561600, + "aten.mm": 7335973519360 + }, + "decode_operators": { + "aten.bmm": 312017024, + "aten.mm": 15134654464 + } + } + ], + "mean_prefill_gflops_counted": 7467.756340960001, + "mean_decode_one_token_gflops_counted": 15.445344384, + "scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "answer_generation": "not performed; one known-token decode probe only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/qwen_construction.json b/releases/20260913_lr/paper_data/cost_20260911/qwen_construction.json new file mode 100644 index 0000000000000000000000000000000000000000..8698bf1c5f0ab0813fee5ae200e538c50e334b40 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/qwen_construction.json @@ -0,0 +1,385 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "construction", + "expert": "yolo26x", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "device": "cuda:0", + "gpu_memory_utilization": 0.75, + "sample_count_requested": 32, + "warmup_count": 2, + "measured_count": 30, + "batch_size": 4, + "sample_ids": [ + "0000772", + "0003400", + "0003077", + "0004213", + "0002162", + "0003303", + "0004535", + "0001299", + "0000762", + "0001453", + "0002112", + "0000382", + "0003936", + "0000833", + "0003551", + "0001683", + "0002571", + "0002523", + "0001939", + "0000904", + "0003516", + "0003633", + "0002690", + "0001464", + "0001595", + "0002168", + "0000183", + "0003404", + "0001727", + "0002796", + "0002153", + "0003848" + ], + "batches": [ + { + "sample_ids": [ + "0003077", + "0004213", + "0002162", + "0003303" + ], + "batch_size": 4, + "generation_latency_seconds": 4.881686039967462, + "batch_wall_latency_seconds": 4.988335099071264, + "amortized_generation_ms_per_sample": 1220.4215099918656, + "amortized_batch_wall_ms_per_sample": 1247.083774767816, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + } + ] + }, + { + "sample_ids": [ + "0004535", + "0001299", + "0000762", + "0001453" + ], + "batch_size": 4, + "generation_latency_seconds": 7.0890667490893975, + "batch_wall_latency_seconds": 7.216836072970182, + "amortized_generation_ms_per_sample": 1772.2666872723494, + "amortized_batch_wall_ms_per_sample": 1804.2090182425454, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1694, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1531, + "completion_tokens": 144, + "text_characters": 435 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1531, + "completion_tokens": 226, + "text_characters": 616 + } + ] + }, + { + "sample_ids": [ + "0002112", + "0000382", + "0003936", + "0000833" + ], + "batch_size": 4, + "generation_latency_seconds": 5.4999597480054945, + "batch_wall_latency_seconds": 5.588348102988675, + "amortized_generation_ms_per_sample": 1374.9899370013736, + "amortized_batch_wall_ms_per_sample": 1397.0870257471688, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1405, + "completion_tokens": 153, + "text_characters": 480 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1531, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1683, + "completion_tokens": 139, + "text_characters": 409 + } + ] + }, + { + "sample_ids": [ + "0003551", + "0001683", + "0002571", + "0002523" + ], + "batch_size": 4, + "generation_latency_seconds": 4.797023626044393, + "batch_wall_latency_seconds": 4.88577020005323, + "amortized_generation_ms_per_sample": 1199.2559065110981, + "amortized_batch_wall_ms_per_sample": 1221.4425500133075, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1694, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + } + ] + }, + { + "sample_ids": [ + "0001939", + "0000904", + "0003516", + "0003633" + ], + "batch_size": 4, + "generation_latency_seconds": 6.3565617019776255, + "batch_wall_latency_seconds": 6.429374794010073, + "amortized_generation_ms_per_sample": 1589.1404254944064, + "amortized_batch_wall_ms_per_sample": 1607.3436985025182, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 147, + "text_characters": 465 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1683, + "completion_tokens": 185, + "text_characters": 574 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1405, + "completion_tokens": 141, + "text_characters": 357 + } + ] + }, + { + "sample_ids": [ + "0002690", + "0001464", + "0001595", + "0002168" + ], + "batch_size": 4, + "generation_latency_seconds": 11.480005273944698, + "batch_wall_latency_seconds": 11.721151808975264, + "amortized_generation_ms_per_sample": 2870.0013184861746, + "amortized_batch_wall_ms_per_sample": 2930.287952243816, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 177, + "text_characters": 545 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 155, + "text_characters": 481 + } + ] + }, + { + "sample_ids": [ + "0000183", + "0003404", + "0001727", + "0002796" + ], + "batch_size": 4, + "generation_latency_seconds": 13.841306244954467, + "batch_wall_latency_seconds": 13.994719217997044, + "amortized_generation_ms_per_sample": 3460.3265612386167, + "amortized_batch_wall_ms_per_sample": 3498.679804499261, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 157, + "text_characters": 510 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 150, + "text_characters": 462 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1531, + "completion_tokens": 169, + "text_characters": 508 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 187, + "text_characters": 511 + } + ] + }, + { + "sample_ids": [ + "0002153", + "0003848" + ], + "batch_size": 2, + "generation_latency_seconds": 14.003285999991931, + "batch_wall_latency_seconds": 14.185666976962239, + "amortized_generation_ms_per_sample": 7001.6429999959655, + "amortized_batch_wall_ms_per_sample": 7092.833488481119, + "outputs": [ + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 221, + "text_characters": 616 + }, + { + "finish_reason": "stop", + "stop_reason": null, + "prompt_tokens": 1705, + "completion_tokens": 110, + "text_characters": 345 + } + ] + } + ], + "mean_generation_ms_per_sample": 2561.0056682489812, + "median_generation_ms_per_sample": 1680.7035563833779, + "mean_batch_wall_ms_per_sample": 2599.870914062194, + "median_batch_wall_ms_per_sample": 1705.7763583725318, + "mean_prompt_tokens": 1659.6, + "mean_completion_tokens": 137.03333333333333, + "latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.", + "flops_scope": "This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/qwen_construction_flops.json b/releases/20260913_lr/paper_data/cost_20260911/qwen_construction_flops.json new file mode 100644 index 0000000000000000000000000000000000000000..68d9892127d3f92669d84b7ea8099750bf25f4b7 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/qwen_construction_flops.json @@ -0,0 +1,177 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "construction", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "sample_count": 4, + "records": [ + { + "sample_id": "0003439", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0001957", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003113", + "prompt_tokens_in_reference_tensor": 1531, + "input_shapes": { + "input_ids": [ + 1, + 1531 + ], + "attention_mask": [ + 1, + 1531 + ], + "mm_token_type_ids": [ + 1, + 1531 + ], + "pixel_values": [ + 3192, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 80104234744896, + "decode_one_token_flops_counted": 51854180544, + "prefill_gflops_counted": 80104.234744896, + "decode_one_token_gflops_counted": 51.854180544, + "prefill_operators": { + "aten.convolution": 17328242688, + "aten.addmm": 2695817134080, + "aten.bmm": 2827143609408, + "aten.mm": 74563945758720 + }, + "decode_operators": { + "aten.bmm": 602407104, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0000184", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + } + ], + "mean_prefill_gflops_counted": 87588.607720992, + "mean_decode_one_token_gflops_counted": 51.905495232, + "scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "answer_generation": "not performed; one known-token decode probe only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_20260911/router_architecture.json b/releases/20260913_lr/paper_data/cost_20260911/router_architecture.json new file mode 100644 index 0000000000000000000000000000000000000000..5377c707132233fbd1efbd77580e85cd50785a6c --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/router_architecture.json @@ -0,0 +1,37 @@ +[ + { + "pair_key": "grefcoco|instancevg|MiniCPM", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "branches": ["image", "text", "hidden", "confidence", "output"], + "input_dimensions": { + "image_flat_plus_valid": 1538, + "text": 768, + "hidden": 768, + "confidence": 8, + "output": 2 + }, + "representation_dim": 128, + "head_hidden_dim": 128, + "class_count": 4, + "source_checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_method_improvement_20260911_r1/conditions/both_instancevg_MiniCPM_seed42/best.pt" + }, + { + "pair_key": "construction|yolo26x|Qwen3.8", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "branches": ["image", "hidden", "confidence", "output"], + "input_dimensions": { + "image_flat_plus_valid": 1538, + "hidden": 768, + "confidence": 8, + "output": 8 + }, + "representation_dim": 128, + "head_hidden_dim": 128, + "class_count": 4, + "source_checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_method_improvement_20260911_r1/conditions/both_yolo26x_Qwen3.8_seed42/best.pt" + } +] diff --git a/releases/20260913_lr/paper_data/cost_20260911/yolo26x_specialist.json b/releases/20260913_lr/paper_data/cost_20260911/yolo26x_specialist.json new file mode 100644 index 0000000000000000000000000000000000000000..d0e7d0bf409a559841a9f529e6886cfefaf50cf9 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_20260911/yolo26x_specialist.json @@ -0,0 +1,134 @@ +{ + "component": "specialist", + "expert": "yolo26x", + "task": "construction", + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/models/construction/yolo26x.pt", + "sample_count": 64, + "batch_size": 16, + "sample_ids": [ + "0000944", + "0003511", + "0003365", + "0001351", + "0002149", + "0003368", + "0000176", + "0004487", + "0000013", + "0003583", + "0000758", + "0001291", + "0001451", + "0002755", + "0003088", + "0004538", + "0004324", + "0000516", + "0003893", + "0003475", + "0002137", + "0003272", + "0001711", + "0003262", + "0001470", + "0002145", + "0002662", + "0000756", + "0001631", + "0001924", + "0002496", + "0001443", + "0004053", + "0002168", + "0001881", + "0001666", + "0000304", + "0001252", + "0004844", + "0000949", + "0003430", + "0003807", + "0004199", + "0002538", + "0002091", + "0001569", + "0003048", + "0002854", + "0000380", + "0001234", + "0001484", + "0003180", + "0000824", + "0003784", + "0000894", + "0004168", + "0000564", + "0004296", + "0003054", + "0001583", + "0003386", + "0000205", + "0004214", + "0003625" + ], + "model_load_seconds_excluded_from_latency": true, + "forward_latency_ms_per_sample": [ + 28.76109504699707, + 28.29251480102539, + 28.306396484375, + 42.96271896362305 + ], + "end_to_end_latency_ms_per_sample": [ + 28.19202906539431, + 27.162021433468908, + 28.371096690534614, + 43.443599628517404 + ], + "mean_forward_latency_ms_per_sample": 32.08068132400513, + "median_forward_latency_ms_per_sample": 28.533745765686035, + "mean_end_to_end_latency_ms_per_sample": 31.79218670447881, + "median_end_to_end_latency_ms_per_sample": 28.28156287796446, + "flop_profiles": [ + { + "batch_size": 16, + "flops_counted": 3110687539200, + "gflops_counted_per_sample": 194.4179712, + "operators": { + "aten.convolution": 3101840179200, + "aten.bmm": 8847360000 + } + }, + { + "batch_size": 16, + "flops_counted": 3110687539200, + "gflops_counted_per_sample": 194.4179712, + "operators": { + "aten.convolution": 3101840179200, + "aten.bmm": 8847360000 + } + }, + { + "batch_size": 16, + "flops_counted": 3110687539200, + "gflops_counted_per_sample": 194.4179712, + "operators": { + "aten.convolution": 3101840179200, + "aten.bmm": 8847360000 + } + }, + { + "batch_size": 16, + "flops_counted": 3110687539200, + "gflops_counted_per_sample": 194.4179712, + "operators": { + "aten.convolution": 3101840179200, + "aten.bmm": 8847360000 + } + } + ], + "mean_counted_gflops_per_sample": 194.4179712, + "flops_scope": "GPU FlopCounterMode around the real Ultralytics predict path; multiply-add counted as 2 FLOPs. Elementwise, postprocessing and unsupported custom ops may be missing.", + "latency_scope": "A800 wall-clock latency for real Ultralytics prediction including image loading/preprocess, forward and result postprocessing, excluding model load; batch processing uses the recorded batch size.", + "completed": true +} \ No newline at end of file diff --git "a/releases/20260913_lr/paper_data/cost_20260911/\346\210\220\346\234\254\346\200\247\350\203\275\350\241\245\345\205\205\346\212\245\345\221\212.md" "b/releases/20260913_lr/paper_data/cost_20260911/\346\210\220\346\234\254\346\200\247\350\203\275\350\241\245\345\205\205\346\212\245\345\221\212.md" new file mode 100644 index 0000000000000000000000000000000000000000..0d109141be177a7cadfda2c2ddd811da6bf4fc24 --- /dev/null +++ "b/releases/20260913_lr/paper_data/cost_20260911/\346\210\220\346\234\254\346\200\247\350\203\275\350\241\245\345\205\205\346\212\245\345\221\212.md" @@ -0,0 +1,69 @@ +# A800成本—性能补充报告(2026-09-11) + +## 1. 做了什么 + +- 在A800上补测了正文中两个代表性模型对:gRefCOCO的InstanceVG + MiniCPM,以及Construction的YOLO26x + Qwen3.8。 +- 分别实测了小模型端、路由器端和大模型端的延迟与可计数运算量;大模型端延迟采用批处理均摊后的真实墙钟时间。 +- 从已经锁定的测试曲线中提取代表性调用预算,按测试集实际调用率换算每个输入的成本,并生成完整曲线。没有用测试集重新挑选模型、阈值或预算。 + +## 2. 为什么做 + +正文原有成本图只是代理估算,缺少InstanceVG、完整解码和真实延迟。此次补测用于回答一个更直接的问题:两项同时改相对置信度路由的性能提升,是否值得它增加的计算和时间成本。 + +## 3. 原始实测成本 + +| 模型对 | 小模型端到端延迟(毫秒/样本) | 小模型可计数运算量(GFLOPs/样本) | 路由器延迟(毫秒/样本) | 路由器稠密运算量(GFLOPs/样本) | 大模型调用延迟(毫秒/样本) | 大模型完整生成估算(GFLOPs/调用) | +|---|---:|---:|---:|---:|---:|---:| +| gRefCOCO / InstanceVG + MiniCPM | 16.317 | 7.503 | 0.001831 | 0.000954 | 1447.432 | 10085.742 | +| Construction / YOLO26x + Qwen3.8 | 31.792 | 194.418 | 0.004676 | 0.000727 | 2599.871 | 94701.391 | + +说明:GFLOPs按乘加计2次运算,只覆盖计数器能识别的稠密乘法、卷积和注意力等算子;路由器的计数器返回0,因此按已加载检查点的线性层结构精确计算了稠密乘加量。大模型完整生成GFLOPs为“预填充GFLOPs + 平均生成长度 × 单Token解码GFLOPs”的估算,不把未计数算子冒充成完整硬件计数。 + +## 4. 验证集锁定点在测试集上的结果 + +下表的均值和标准差只在三个随机种子之间统计。成本使用对应的平均实际调用率;星号表示验证集选出的运行点。 + +| 模型对 | 方法 | 测试性能 | 实际调用率 | 估算GFLOPs/输入 | 端到端延迟(毫秒/输入) | +|---|---|---:|---:|---:|---:| +| gRefCOCO / InstanceVG + MiniCPM | 两项同时改* | 68.05±0.04% | 11.29±1.28% | 1146.415 | 179.767 | +| gRefCOCO / InstanceVG + MiniCPM | 原方法* | 67.33±0.15% | 26.05±1.96% | 2634.360 | 393.306 | +| gRefCOCO / InstanceVG + MiniCPM | 置信度路由* | 61.88±0.00% | 52.09±0.00% | 5261.597 | 770.348 | +| Construction / YOLO26x + Qwen3.8 | 两项同时改* | 30.11±0.14% | 44.14±0.58% | 41996.697 | 1179.410 | +| Construction / YOLO26x + Qwen3.8 | 原方法* | 30.00±0.22% | 46.67±1.72% | 44392.604 | 1245.185 | +| Construction / YOLO26x + Qwen3.8 | 置信度路由* | 25.97±0.00% | 92.21±0.00% | 87518.937 | 2429.148 | + +### 两项同时改相对置信度路由 + +| 模型对 | 性能变化 | 调用率变化 | 估算GFLOPs变化 | 实际延迟变化 | +|---|---:|---:|---:|---:| +| gRefCOCO / InstanceVG + MiniCPM | +6.18 个百分点 | -40.80 个百分点 | -78.2% | -76.7% | +| Construction / YOLO26x + Qwen3.8 | +4.13 个百分点 | -48.07 个百分点 | -52.0% | -51.4% | + +直白地说:两项同时改在这两个代表性模型对上都提高了测试性能,同时减少了实际调用,因此计算量和延迟也一起下降。原因不是路由器本身更快,而是它让大模型少被调用;路由器自身成本相对很小。 + +## 5. 固定名义调用预算的代表点 + +名义预算是验证阶段设定的阈值预算;阈值原样转到测试集后,实际调用率会有偏差。完整的99个阈值点和端点均在CSV中。 + +| 模型对 | 名义预算 | 两项同时改:性能 / 实际调用 | 置信度:性能 / 实际调用 | 两项同时改:GFLOPs/输入 | 两项同时改:毫秒/输入 | +|---|---:|---:|---:|---:|---:| +| gRefCOCO / InstanceVG + MiniCPM | 25% | 67.94% / 24.53% | 65.97% / 12.48% | 2481.628 | 371.387 | +| gRefCOCO / InstanceVG + MiniCPM | 50% | 66.46% / 43.10% | 65.28% / 23.45% | 4354.834 | 640.216 | +| Construction / YOLO26x + Qwen3.8 | 25% | 25.62% / 26.50% | 18.03% / 24.80% | 25288.396 | 720.711 | +| Construction / YOLO26x + Qwen3.8 | 50% | 29.98% / 49.51% | 18.02% / 50.13% | 47082.746 | 1319.039 | + +## 6. 结果怎么理解 + +1. gRefCOCO的InstanceVG + MiniCPM是这两组中相对置信度提升更明显的一组;Construction的YOLO26x + Qwen3.8也有稳定的正向变化。 +2. 这里的成本是按真实测试调用率换算的平均每输入成本,不是把名义预算当成实际调用率。 +3. 大模型延迟是A800、批量4下的均摊值,模型加载和预热没有计入;实际系统的并发度、批量大小和请求排队会改变墙钟延迟。 +4. InstanceVG使用了远端已有的MMCV等价算子兼容路径,因为原生detrex扩展不可用;这不改变本次模型输出链路,但应在复现实验时保留该环境说明。 +5. 本补充不修改正文,也不覆盖旧实验结果;它是针对两个代表性模型对的成本实测和成本—性能换算。 + +## 7. 文件在哪里 + +- 原始A800测量记录:`C:/Users/csh10/Documents/CodexProjects/ICASSP-COLLA/dse_results/method_improvement_20260911/cost_analysis_20260911`目录下的8个JSON文件。 +- 完整成本—性能曲线数据:`C:/Users/csh10/Documents/CodexProjects/ICASSP-COLLA/dse_results/method_improvement_20260911/cost_analysis_20260911/full_cost_performance_curves.csv`。 +- 固定预算代表点:`C:/Users/csh10/Documents/CodexProjects/ICASSP-COLLA/dse_results/method_improvement_20260911/cost_analysis_20260911/fixed_budget_cost_performance.csv`。 +- FLOPs—性能图:`C:/Users/csh10/Documents/CodexProjects/ICASSP-COLLA/dse_results/method_improvement_20260911/cost_analysis_20260911/flops_performance.pdf`;实际延迟—性能图:`C:/Users/csh10/Documents/CodexProjects/ICASSP-COLLA/dse_results/method_improvement_20260911/cost_analysis_20260911/latency_performance.pdf`。 +- 可复现脚本:`C:/Users/csh10/Documents/CodexProjects/ICASSP-COLLA/dse_results/method_improvement_20260911/cost_analysis_20260911/make_cost_analysis_20260911.py`;插入论文的LaTeX片段:`C:/Users/csh10/Documents/CodexProjects/ICASSP-COLLA/dse_results/method_improvement_20260911/cost_analysis_20260911/latex_includes.tex`。 diff --git a/releases/20260913_lr/paper_data/cost_components.json b/releases/20260913_lr/paper_data/cost_components.json new file mode 100644 index 0000000000000000000000000000000000000000..3ecee6e5711d902b83cdc60ad2db5d2fcf81d5a6 --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_components.json @@ -0,0 +1,41 @@ +{ + "components": { + "grefcoco|instancevg|MiniCPM": { + "specialist_gflops": 7.502680064, + "router_gflops": 3.642171392, + "large_gflops": 12916.147709832, + "specialist_ms": 46.08042127620138, + "router_ms": 13.180577338061994, + "large_ms": 3667.173011115665, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + }, + "construction|yolo26x|Qwen3.8": { + "specialist_gflops": 135.97853184000002, + "router_gflops": 3.183754752, + "large_gflops": 96874.712551641, + "specialist_ms": 46.30809425361804, + "router_ms": 25.261267359383055, + "large_ms": 9822.475872695577, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + } + }, + "protocol": { + "method": "original main-table logit contrast; original checkpoints and validation thresholds; seed 42", + "large_timing": "64 measured examples; batch size one; same full task prompt; two separate warmup examples", + "cost": "serial component estimates, not a timed integrated service; includes active distilled encoders and routing head only for learned", + "flops": "multiply-add=2; supported operators; mean prefill (including visual embedding) plus average generated length times one-token decode probe", + "scope_limits": "mean call cost extrapolated across policies; no policy-specific input/output-length costs; specialist-feature extraction overhead is not separately measured; confidence threshold overhead not separately timed" + }, + "source_files": [ + "dse_results/main_table_cost_20260912/cost_summary.json", + "dse_results/main_table_cost_qwen_20260912/cost_summary.json" + ] +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/cost_five_seeds.json b/releases/20260913_lr/paper_data/cost_five_seeds.json new file mode 100644 index 0000000000000000000000000000000000000000..8c760e5a9bec5dd838fbde947b196f12e617256a --- /dev/null +++ b/releases/20260913_lr/paper_data/cost_five_seeds.json @@ -0,0 +1,3299 @@ +{ + "components": { + "grefcoco|instancevg|MiniCPM": { + "specialist_gflops": 7.502680064, + "router_gflops": 3.642171392, + "large_gflops": 12916.147709832, + "specialist_ms": 46.08042127620138, + "router_ms": 13.180577338061994, + "large_ms": 3667.173011115665, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + }, + "construction|yolo26x|Qwen3.8": { + "specialist_gflops": 135.97853184000002, + "router_gflops": 3.183754752, + "large_gflops": 96874.712551641, + "specialist_ms": 46.30809425361804, + "router_ms": 25.261267359383055, + "large_ms": 9822.475872695577, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + } + }, + "selected": [ + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6791367722542041, + "calls": 0.19038085245157815, + "gflops": 2470.132062844315, + "ms": 757.4205225578843 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6187788900547316, + "calls": 0.5209426310864078, + "gflops": 6736.074651524562, + "ms": 1956.4671783358606 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2979678330127048, + "calls": 0.44766977363515315, + "gflops": 43507.04292555566, + "ms": 4468.794912079384 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2597490004762105, + "calls": 0.9221038615179761, + "gflops": 89464.5250591521, + "ms": 9103.651026133362 + } + ], + "curves": [ + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6695119530385957, + "calls": 0.0, + "gflops": 11.144851456, + "ms": 59.26099861426337 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6736239117488585, + "calls": 0.020332926863851627, + "gflops": 273.76793820271877, + "ms": 133.82535924636875 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6766298953577403, + "calls": 0.04462467742392876, + "gflops": 587.5237765670691, + "ms": 222.90741129303743 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6786603522105323, + "calls": 0.07091285483367836, + "gflops": 927.0657590136638, + "ms": 319.31070600149167 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6797663273119132, + "calls": 0.09045174829141027, + "gflops": 1179.4329930003992, + "ms": 390.9632087567506 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6804072257039957, + "calls": 0.11004735842100785, + "gflops": 1432.5327878985618, + "ms": 462.8237013603555 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6805717040524062, + "calls": 0.12548563650285, + "gflops": 1631.9358679890966, + "ms": 519.4385380801856 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6804015540368091, + "calls": 0.13746986926807134, + "gflops": 1786.725988573704, + "ms": 563.3867930357333 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6803391656977569, + "calls": 0.15023112043785272, + "gflops": 1951.5521936448667, + "ms": 610.1845089136239 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6802767773587046, + "calls": 0.16403595836996285, + "gflops": 2129.857519486293, + "ms": 660.8092380010838 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6802881206930778, + "calls": 0.1779372146442447, + "gflops": 2309.4081988771463, + "ms": 711.7875498307326 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6802824490258912, + "calls": 0.18960383404701814, + "gflops": 2460.09597835776, + "ms": 754.5710616355417 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6801293140118538, + "calls": 0.2002041800187165, + "gflops": 2597.011612703539, + "ms": 793.4443642914426 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6800839406743613, + "calls": 0.2103564642826759, + "gflops": 2728.1400158490414, + "ms": 830.6745471454087 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6799818506650028, + "calls": 0.21922695176247053, + "gflops": 2842.7125423962843, + "ms": 863.2041594267511 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6798684173212717, + "calls": 0.2273317641720784, + "gflops": 2947.395496639259, + "ms": 892.9259087554203 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6796925956384878, + "calls": 0.2348580665286561, + "gflops": 3044.606329585673, + "ms": 920.5261616309583 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6796018489635027, + "calls": 0.24202705385247994, + "gflops": 3137.202028790095, + "ms": 946.8160784619154 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6794316989479057, + "calls": 0.24939454952783371, + "gflops": 3232.361691184513, + "ms": 973.8339597620842 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6793693106088534, + "calls": 0.25641040183762015, + "gflops": 3322.9794759280803, + "ms": 999.5623040025065 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6792615489323086, + "calls": 0.2633184924708618, + "gflops": 3412.2053949399365, + "ms": 1024.895467531071 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6791027422510847, + "calls": 0.2692510563480135, + "gflops": 3488.8312662752414, + "ms": 1046.6512056680817 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6789496072370473, + "calls": 0.27534242690638916, + "gflops": 3567.508308162543, + "ms": 1068.9893153804614 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6787681138870771, + "calls": 0.2815245441397499, + "gflops": 3647.3574475081286, + "ms": 1091.660208850195 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6786490088761592, + "calls": 0.28715650965601336, + "gflops": 3720.1007460128676, + "ms": 1112.3136007909704 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6784902021949353, + "calls": 0.29289056518163514, + "gflops": 3794.162654158177, + "ms": 1133.3413744587692 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6783540821824576, + "calls": 0.2986983523806823, + "gflops": 3869.1768914883414, + "ms": 1154.639534929418 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6781215438278082, + "calls": 0.30438703456881144, + "gflops": 3942.6527509045077, + "ms": 1175.5009167185397 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6777301987919349, + "calls": 0.31002467175226156, + "gflops": 4015.469305500391, + "ms": 1196.17510764415 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6775146754388452, + "calls": 0.31561126393103256, + "gflops": 4087.626555275989, + "ms": 1216.662107706249 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6773048237529421, + "calls": 0.3206477043927063, + "gflops": 4152.677964210741, + "ms": 1235.1316062393896 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6772084054107704, + "calls": 0.3267560899526416, + "gflops": 4231.57477437147, + "ms": 1257.5321129062731 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6769191503842554, + "calls": 0.3326829821626067, + "gflops": 4308.127389615633, + "ms": 1279.2670520584488 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6765788503530613, + "calls": 0.33824688767263134, + "gflops": 4379.991615026658, + "ms": 1299.6708561812088 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6762782519921732, + "calls": 0.3451209483027536, + "gflops": 4468.777997491658, + "ms": 1324.8792258007659 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6760173553015909, + "calls": 0.3511272438533307, + "gflops": 4546.356198011819, + "ms": 1346.9053507406265 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6754955619204265, + "calls": 0.35775175112724383, + "gflops": 4631.919312466537, + "ms": 1371.19856502746 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6750021268751949, + "calls": 0.36507387346510506, + "gflops": 4726.492926131814, + "ms": 1398.0500544489519 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6744406318237245, + "calls": 0.3725207724810708, + "gflops": 4822.678173802229, + "ms": 1425.3591215368053 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6740152567847318, + "calls": 0.37905453307999887, + "gflops": 4907.069190798665, + "ms": 1449.3195520662853 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6733346567223435, + "calls": 0.38662053710688254, + "gflops": 5004.792816383078, + "ms": 1477.0653978356654 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6728752516802314, + "calls": 0.39437937781810967, + "gflops": 5105.007149066346, + "ms": 1505.5184090894231 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6721946516178432, + "calls": 0.40177523182939623, + "gflops": 5200.533091916476, + "ms": 1532.6402853137647 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6716388282335591, + "calls": 0.4089045174829141, + "gflops": 5292.6159984829, + "ms": 1558.7846092508796 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6709298698352381, + "calls": 0.4166690298613277, + "gflops": 5392.903587257309, + "ms": 1587.2584194894714 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.670351359782208, + "calls": 0.42493832061934605, + "gflops": 5499.710968143422, + "ms": 1617.5833393783444 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6695743413776479, + "calls": 0.43286731134617024, + "gflops": 5602.122983560971, + "ms": 1646.6603201771404 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6687065762981028, + "calls": 0.4409891387573377, + "gflops": 5707.025706077373, + "ms": 1676.4444664603132 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6676913478717068, + "calls": 0.4495646995434308, + "gflops": 5817.788915885195, + "ms": 1707.8925315302556 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6667838811218558, + "calls": 0.45795309531236705, + "gflops": 5926.134674685305, + "ms": 1738.6542301006555 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.665910444375124, + "calls": 0.46621671440319884, + "gflops": 6032.868799480276, + "ms": 1768.9583510046941 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6649065592831013, + "calls": 0.47530839690326976, + "gflops": 6150.298313582086, + "ms": 1802.2991236945866 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6638005841817202, + "calls": 0.4846496327595496, + "gflops": 6270.951095694176, + "ms": 1836.555051717202 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6626946090803392, + "calls": 0.4937072852565011, + "gflops": 6387.94107324913, + "ms": 1869.771030498087 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6615659473102118, + "calls": 0.5040467345376174, + "gflops": 6521.486927402344, + "ms": 1907.687579851596 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6605280322150696, + "calls": 0.5130249836939569, + "gflops": 6637.4513196812995, + "ms": 1940.612372844796 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6591214587528004, + "calls": 0.5229674162720132, + "gflops": 6765.8692474545205, + "ms": 1977.0729932598813 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6579020503076879, + "calls": 0.5324504438079574, + "gflops": 6888.35343184518, + "ms": 2011.8488959033625 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6562969684938889, + "calls": 0.5418654113376628, + "gflops": 7009.9585431421265, + "ms": 2046.3752107288285 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6548733800300598, + "calls": 0.5520120239344355, + "gflops": 7141.013690196485, + "ms": 2083.58459459796 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6530811331991039, + "calls": 0.5628449082607834, + "gflops": 7280.932824279119, + "ms": 2123.3106556320804 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6512435130306553, + "calls": 0.5729631625216232, + "gflops": 7411.621690877762, + "ms": 2160.416044577038 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6493945495278337, + "calls": 0.5823667867169554, + "gflops": 7533.080289992424, + "ms": 2194.9007614328348 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.647755437710915, + "calls": 0.5923942943028104, + "gflops": 7662.597059132788, + "ms": 2231.67336662044 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6456966225221904, + "calls": 0.6029662819385759, + "gflops": 7799.146413022853, + "ms": 2270.4426743521676 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6438760173553015, + "calls": 0.6129314011853785, + "gflops": 7927.857365160644, + "ms": 2306.986490706591 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.641459887133823, + "calls": 0.6240138388679352, + "gflops": 8070.999767253555, + "ms": 2347.6277070734345 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6394010719450983, + "calls": 0.6343873181521709, + "gflops": 8204.985157953626, + "ms": 2385.669050335951 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6372458384142019, + "calls": 0.6447324391004736, + "gflops": 8338.604268197982, + "ms": 2423.6063986742943 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6348240365255367, + "calls": 0.655871593454896, + "gflops": 8482.479231202318, + "ms": 2464.4556048894833 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6324419363071774, + "calls": 0.6665116410969005, + "gflops": 8619.907658186099, + "ms": 2503.4745004392275 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6297138643904374, + "calls": 0.6780421404871962, + "gflops": 8768.837291479285, + "ms": 2545.7588366080054 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6270595241471231, + "calls": 0.687848453052775, + "gflops": 8895.497073065082, + "ms": 2581.7202813870604 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.624359810566316, + "calls": 0.6985792473697643, + "gflops": 9034.097597507143, + "ms": 2621.0719606941566 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.62159203697927, + "calls": 0.7093157133539403, + "gflops": 9172.771378040346, + "ms": 2660.4444389860882 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6185803817032016, + "calls": 0.7205569577177211, + "gflops": 9317.964950685257, + "ms": 2701.6680269283015 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6154042480787226, + "calls": 0.7306752119785611, + "gflops": 9448.653817283903, + "ms": 2738.77341587326 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6121713977823781, + "calls": 0.7409692879221847, + "gflops": 9581.613622707973, + "ms": 2776.5235733480918 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6087967558063693, + "calls": 0.7520970989422342, + "gflops": 9725.342073530028, + "ms": 2817.3311815936127 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6053710688256813, + "calls": 0.7639338683606047, + "gflops": 9878.227535744925, + "ms": 2860.73866294346 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.6012931401185378, + "calls": 0.7769787028897144, + "gflops": 10046.716545373221, + "ms": 2908.576328063081 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5976405864503871, + "calls": 0.7888154723080849, + "gflops": 10199.602007588117, + "ms": 2951.9838094129286 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5939540027791168, + "calls": 0.799671043303179, + "gflops": 10339.81416603532, + "ms": 2991.793066386387 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5896775657204435, + "calls": 0.8114284093809374, + "gflops": 10491.674042974217, + "ms": 3034.90936194855 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5853273969883448, + "calls": 0.823225477129002, + "gflops": 10644.046712551113, + "ms": 3078.1712504045554 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5808467799109548, + "calls": 0.8346992598474321, + "gflops": 10792.243784932874, + "ms": 3120.2475967249875 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5766554178600799, + "calls": 0.8458951308737204, + "gflops": 10936.851308848642, + "ms": 3161.304792788524 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5724016674701529, + "calls": 0.8569038368828519, + "gflops": 11079.0413817567, + "ms": 3201.6756223525176 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5677111987068598, + "calls": 0.868638516291864, + "gflops": 11230.608234331025, + "ms": 3244.7087219753416 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5631284916201118, + "calls": 0.8793125939369878, + "gflops": 11368.476197861659, + "ms": 3283.852411434093 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5585287695318039, + "calls": 0.8898675665711936, + "gflops": 11504.805783478296, + "ms": 3322.5593222113166 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5539857641153618, + "calls": 0.9006267192241161, + "gflops": 11643.772588376074, + "ms": 3362.0149964425877 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5498227604004197, + "calls": 0.9102855684428437, + "gflops": 11768.527711592154, + "ms": 3397.435667615941 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5444630349091115, + "calls": 0.9222641295408784, + "gflops": 11923.244576085619, + "ms": 3441.363123586654 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.539069279414684, + "calls": 0.9331140288687859, + "gflops": 12063.38347844168, + "ms": 3481.151581575278 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5336868672546294, + "calls": 0.9439185548592007, + "gflops": 12202.936332068595, + "ms": 3520.7736476852256 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5275557950259478, + "calls": 0.9553469642401383, + "gflops": 12350.547355721215, + "ms": 3562.683602126981 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.520874571080169, + "calls": 0.966695970280464, + "gflops": 12497.132794097837, + "ms": 3604.3023707810516 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5142840938093752, + "calls": 0.977761392961461, + "gflops": 12640.05542791732, + "ms": 3644.8811901933914 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5079318265604174, + "calls": 0.9880441255707115, + "gflops": 12772.868721159106, + "ms": 3682.5897496985535 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "metric": 0.5000708958398321, + "calls": 1.0, + "gflops": 12927.292561287999, + "ms": 3726.434009729928 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6695119530385957, + "calls": 0.0, + "gflops": 7.502680064, + "ms": 46.08042127620138 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6686044862887446, + "calls": 0.002467175226157729, + "gflops": 39.3690797110914, + "ms": 55.12797967926019 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6678388112185577, + "calls": 0.004849275444516916, + "gflops": 70.13663799104171, + "ms": 63.86355330979973 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.667214927828035, + "calls": 0.007600034030003119, + "gflops": 105.66584219527005, + "ms": 73.95106095458944 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6666761194453109, + "calls": 0.009811984232765221, + "gflops": 134.23571774093818, + "ms": 82.06266504009045 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6662791027422511, + "calls": 0.012534384482318578, + "gflops": 169.39864148945287, + "ms": 92.04617776070705 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6658537277032583, + "calls": 0.01607917647392451, + "gflops": 215.18369845366465, + "ms": 105.04554328234329 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6654850693361314, + "calls": 0.019709043473328986, + "gflops": 262.0675967850175, + "ms": 118.35689357649878 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6652865609846014, + "calls": 0.02353741882426339, + "gflops": 311.5154583063662, + "ms": 132.3962083398659 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6648611859456087, + "calls": 0.028868785979638715, + "gflops": 380.3761839805407, + "ms": 151.94725408440678 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6645208859144145, + "calls": 0.0345971698380739, + "gflops": 454.3648360347069, + "ms": 172.9542287673709 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6642089442191532, + "calls": 0.03981510364971783, + "gflops": 521.7604398860266, + "ms": 192.08929481521943 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6640387942035562, + "calls": 0.045855429203414344, + "gflops": 599.7781769530435, + "ms": 214.24021366408755 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6637268525082948, + "calls": 0.05317187987408899, + "gflops": 694.2785345271766, + "ms": 241.0709041007447 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6633581941411678, + "calls": 0.05960922213084536, + "gflops": 777.4241979741852, + "ms": 264.6777518880361 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6630178941099736, + "calls": 0.0675211978561098, + "gflops": 879.6164451183058, + "ms": 293.69233573232816 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6626492357428466, + "calls": 0.07551824858917279, + "gflops": 982.9075336295676, + "ms": 323.01890434913946 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6621671440319882, + "calls": 0.08212574086152624, + "gflops": 1068.2508798108586, + "ms": 347.24972168146934 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6619969940163911, + "calls": 0.08856308311828262, + "gflops": 1151.396543257867, + "ms": 370.85656946876077 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6615716189773984, + "calls": 0.09480191702350906, + "gflops": 1231.9782435148798, + "ms": 393.7354527868405 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6612029606102714, + "calls": 0.10007656750701868, + "gflops": 1300.1064082776268, + "ms": 413.0785086830352 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6608910189150101, + "calls": 0.10554972634205825, + "gflops": 1370.79853623037, + "ms": 433.14952904844154 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6605790772197487, + "calls": 0.11005870175538099, + "gflops": 1429.0371286888474, + "ms": 449.68472199196276 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6602671355244875, + "calls": 0.11527663556702493, + "gflops": 1496.432732540167, + "ms": 468.8197880398113 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6599835521651589, + "calls": 0.12029606102713893, + "gflops": 1561.264373201491, + "ms": 487.2268896184482 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6596999688058305, + "calls": 0.12477667810452883, + "gflops": 1619.1366852042547, + "ms": 503.6580876377964 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.659217877094972, + "calls": 0.12914386183818732, + "gflops": 1675.5438753841634, + "ms": 519.6733059604521 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6587925020559794, + "calls": 0.1322916371267334, + "gflops": 1716.2010059683835, + "ms": 531.2167425436651 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6582253353373224, + "calls": 0.13626180415733205, + "gflops": 1767.4802697683008, + "ms": 545.7760319278977 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6578283186342625, + "calls": 0.14023197118793068, + "gflops": 1818.7595335682179, + "ms": 560.3353213121303 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6573178685874713, + "calls": 0.14326631313274538, + "gflops": 1857.9515423295834, + "ms": 571.462778198651 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6569208518844114, + "calls": 0.14766185520233674, + "gflops": 1914.725012965206, + "ms": 587.5819914454798 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6565805518532173, + "calls": 0.15066783881121856, + "gflops": 1953.5507412708575, + "ms": 598.6054534078274 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6561268184782917, + "calls": 0.15452457249808582, + "gflops": 2003.36488324792, + "ms": 612.7487630953676 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6558715934548961, + "calls": 0.15806936448969175, + "gflops": 2049.1499402121317, + "ms": 625.7481286170038 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6555596517596347, + "calls": 0.16201117318435754, + "gflops": 2100.0629235563347, + "ms": 640.2034230770632 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6551626350565749, + "calls": 0.1657544735274934, + "gflops": 2148.4119437105423, + "ms": 653.930753067911 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6545954683379179, + "calls": 0.17012165726115192, + "gflops": 2204.8191338904517, + "ms": 669.9459713905669 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6540850182911266, + "calls": 0.17536794940872868, + "gflops": 2272.581018197485, + "ms": 689.1850323625886 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6534044182287383, + "calls": 0.17965005813458867, + "gflops": 2327.889367010253, + "ms": 704.8882659127252 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.653092476533477, + "calls": 0.18401724186824717, + "gflops": 2384.2965571901614, + "ms": 720.9034842353809 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6527521765022829, + "calls": 0.18858293395343562, + "gflops": 2443.2677105600665, + "ms": 737.6466670272484 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6523835181351558, + "calls": 0.19238295096843716, + "gflops": 2492.3492916257014, + "ms": 751.5819868664424 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6517596347446332, + "calls": 0.19689192638175992, + "gflops": 2550.5878840841788, + "ms": 768.1171798099638 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6511641096900433, + "calls": 0.20182627683407536, + "gflops": 2614.3206833783615, + "ms": 786.2122966160813 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6505402262995208, + "calls": 0.20718600232538353, + "gflops": 2683.5476895082497, + "ms": 805.8673372847952 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6500864929245952, + "calls": 0.21189348609023623, + "gflops": 2744.350245156723, + "ms": 823.1304946975282 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6500581345886624, + "calls": 0.217139778237813, + "gflops": 2812.1121294637564, + "ms": 842.3695556695498 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6507670929869835, + "calls": 0.22198905368232993, + "gflops": 2874.7460873907985, + "ms": 860.1526877031482 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6515611263931033, + "calls": 0.22763236253296656, + "gflops": 2947.635898077823, + "ms": 880.8476776135931 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6527521765022829, + "calls": 0.2345234381646485, + "gflops": 3036.6420488162507, + "ms": 906.1184441876538 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6537447182599325, + "calls": 0.24382497235062248, + "gflops": 3156.782038290343, + "ms": 940.2287793164273 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6545387516660522, + "calls": 0.252644414825738, + "gflops": 3270.6952600173017, + "ms": 972.5712007342581 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6538581516036639, + "calls": 0.26401610753480986, + "gflops": 3417.573722758493, + "ms": 1014.2731653276672 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6537163599239997, + "calls": 0.2760116836344043, + "gflops": 3572.5103555253854, + "ms": 1058.2630182528842 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6533477015568726, + "calls": 0.2893401015228426, + "gflops": 3744.6621697108217, + "ms": 1107.1406326142362 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6525536681507529, + "calls": 0.3036043444970649, + "gflops": 3928.90123893481, + "ms": 1159.4500794733005 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6509088846666478, + "calls": 0.3175850041119587, + "gflops": 4109.477503601662, + "ms": 1210.719577090634 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6498312679011996, + "calls": 0.33462836400760004, + "gflops": 4329.612057485591, + "ms": 1273.2205265186608 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6465984176048549, + "calls": 0.35637920766809406, + "gflops": 4610.549167017995, + "ms": 1352.9846333594207 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6442730340583614, + "calls": 0.37968975980489467, + "gflops": 4911.631701614652, + "ms": 1438.4684610297004 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6405580920511584, + "calls": 0.4046450954258004, + "gflops": 5233.958502642702, + "ms": 1529.9839943020195 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6357371749425744, + "calls": 0.43141536454640844, + "gflops": 5579.72725283643, + "ms": 1628.1552027214163 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.630604316138729, + "calls": 0.4610214672603012, + "gflops": 5962.124048601527, + "ms": 1736.725903558122 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6244221989053682, + "calls": 0.4916484700677764, + "gflops": 6357.706940772317, + "ms": 1849.0404216650588 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6187788900547316, + "calls": 0.5209426310864078, + "gflops": 6736.074651524562, + "ms": 1956.4671783358606 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6118310977511839, + "calls": 0.5529308340186598, + "gflops": 7149.2390055696105, + "ms": 2073.773452803106 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.6045146470805093, + "calls": 0.5888041289737118, + "gflops": 7612.583782047433, + "ms": 2205.3270318820646 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5983608881830814, + "calls": 0.6175027649377535, + "gflops": 7983.2596032296915, + "ms": 2310.5698951452314 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5925474293168477, + "calls": 0.6421461588633979, + "gflops": 8301.557319244892, + "ms": 2400.941484251646 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5877548705441965, + "calls": 0.665910444375124, + "gflops": 8608.500341132967, + "ms": 2488.0892307086956 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5829339534356124, + "calls": 0.6838612710206166, + "gflops": 8840.355869599738, + "ms": 2553.9180177102617 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5779996029832969, + "calls": 0.7029464311034228, + "gflops": 9086.862616295053, + "ms": 2623.9066016787506 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5745966026713553, + "calls": 0.7180614241556306, + "gflops": 9282.090099190453, + "ms": 2679.3358962630077 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5710234523438165, + "calls": 0.7352465757309361, + "gflops": 9504.05605535295, + "ms": 2742.3568203119 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.567762243711539, + "calls": 0.7478093185491875, + "gflops": 9666.318297234116, + "ms": 2788.426571720579 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.565153276805717, + "calls": 0.759804894648782, + "gflops": 9821.254930001009, + "ms": 2832.416424645796 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5620622181890367, + "calls": 0.7729064458497575, + "gflops": 9990.476500540737, + "ms": 2880.4620796137633 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5588860845645578, + "calls": 0.7855259053398747, + "gflops": 10153.47130333333, + "ms": 2926.739820870788 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5562771176587358, + "calls": 0.7964722230099538, + "gflops": 10294.855559238817, + "ms": 2966.881861601601 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5536114340810481, + "calls": 0.8082125740861527, + "gflops": 10446.495667904286, + "ms": 3009.93576020926 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5504920171284349, + "calls": 0.8197544168108215, + "gflops": 10595.57181337976, + "ms": 3052.2616943477074 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5469188668008961, + "calls": 0.8333380597226555, + "gflops": 10771.02015166662, + "ms": 3102.075263026618 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5450188582933954, + "calls": 0.8426112355726966, + "gflops": 10890.793860684997, + "ms": 3136.0816032312177 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5423531747157077, + "calls": 0.8529903865241187, + "gflops": 11024.85250747621, + "ms": 3174.1437454785687 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5395456994583557, + "calls": 0.8635396874911381, + "gflops": 11161.108837001704, + "ms": 3212.829857270958 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5378725576383178, + "calls": 0.8710262881774098, + "gflops": 11257.806877310119, + "ms": 3240.284517252654 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.535774040779287, + "calls": 0.8809517057539064, + "gflops": 11386.005036809911, + "ms": 3276.6827407132355 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5330516405297338, + "calls": 0.8913024983693957, + "gflops": 11519.69740314541, + "ms": 3314.640888036413 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5304710319598446, + "calls": 0.900915974250631, + "gflops": 11643.866477632353, + "ms": 3349.8951673310903 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5277486317102913, + "calls": 0.910954825170859, + "gflops": 11773.529758955001, + "ms": 3386.7093704883646 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5245724980858123, + "calls": 0.921901142840938, + "gflops": 11914.914014860486, + "ms": 3426.851411219177 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5217366644925275, + "calls": 0.9303235686129938, + "gflops": 12023.699310207454, + "ms": 3457.737903698585 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5187306808836457, + "calls": 0.9393698777755721, + "gflops": 12140.542775580121, + "ms": 3490.9122845098004 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.515526188923234, + "calls": 0.9490117119927403, + "gflops": 12265.078130522777, + "ms": 3526.270558728651 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5122366219550237, + "calls": 0.9587953378895726, + "gflops": 12391.444887744003, + "ms": 3562.1488075683665 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5097694467288659, + "calls": 0.9673595553412926, + "gflops": 12502.061585369538, + "ms": 3593.55527466864 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5075574965261038, + "calls": 0.9746192893401016, + "gflops": 12595.829382032243, + "ms": 3620.177975256951 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5046365879250205, + "calls": 0.9830984317840229, + "gflops": 12705.347238290638, + "ms": 3651.2724575847046 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5019425460114001, + "calls": 0.9908118991577575, + "gflops": 12804.975522244764, + "ms": 3679.559076959785 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "metric": 0.5000708958398321, + "calls": 1.0, + "gflops": 12923.650389896, + "ms": 3713.253432391866 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.18129624718318374, + "calls": 0.0, + "gflops": 139.16228659200002, + "ms": 71.56936161300109 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.18372855448704165, + "calls": 0.007789613848202396, + "gflops": 893.7788890248893, + "ms": 148.08265569458447 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.18685908224889786, + "calls": 0.018908122503328895, + "gflops": 1970.8812189932014, + "ms": 257.29393869992145 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.1884468119835047, + "calls": 0.029227696404793613, + "gflops": 2970.5869743530125, + "ms": 358.6577043635576 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.19016333859244455, + "calls": 0.037150466045272966, + "gflops": 3738.1030058873175, + "ms": 436.47891800209106 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.19203936879648048, + "calls": 0.047203728362183756, + "gflops": 4712.009903044295, + "ms": 535.226844551827 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.1973276972772499, + "calls": 0.06245006657789613, + "gflops": 6188.99453515653, + "ms": 684.9836338226183 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.19856778453770563, + "calls": 0.07423435419440745, + "gflops": 7330.594010631928, + "ms": 800.734514612706 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.19964444666304987, + "calls": 0.08342210386151798, + "gflops": 8220.654618629695, + "ms": 890.980964042266 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2008447299915772, + "calls": 0.09094540612516644, + "gflops": 8949.47236285975, + "ms": 964.878419009949 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2053760837500663, + "calls": 0.10252996005326231, + "gflops": 10071.722694683021, + "ms": 1078.6674204646115 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2084699431938867, + "calls": 0.11284953395472704, + "gflops": 11071.428450042833, + "ms": 1180.0311861282478 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.21117730070554924, + "calls": 0.12343541944074565, + "gflops": 12096.933063605475, + "ms": 1284.0107909057838 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.21355089822347736, + "calls": 0.1318908122503329, + "gflops": 12916.04681154545, + "ms": 1367.0636827721182 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.21604151501630087, + "calls": 0.14500665778961386, + "gflops": 14186.640578035016, + "ms": 1495.8937591317074 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.21788898058414213, + "calls": 0.1533288948069241, + "gflops": 14992.854896873574, + "ms": 1577.6387314410913 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2206453715306817, + "calls": 0.16624500665778957, + "gflops": 16244.09951971101, + "ms": 1704.5069284652548 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.22256047317491415, + "calls": 0.17569906790945405, + "gflops": 17159.958985911613, + "ms": 1797.3692170087152 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.22392279063908965, + "calls": 0.18202396804260984, + "gflops": 17772.681868228916, + "ms": 1859.495395963847 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.22639256671969005, + "calls": 0.1914780292942743, + "gflops": 18688.541334429516, + "ms": 1952.3576845073073 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2303544739053073, + "calls": 0.2051264980026631, + "gflops": 20010.73281732475, + "ms": 2086.419439094697 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2328940374950383, + "calls": 0.2138482023968043, + "gflops": 20855.645423467562, + "ms": 2172.088170074932 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.23741897056590364, + "calls": 0.22656458055925435, + "gflops": 22087.540902652876, + "ms": 2296.9944877636703 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.24026689104972293, + "calls": 0.23794940079893476, + "gflops": 23190.44209082402, + "ms": 2408.8216098829075 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2431042580703525, + "calls": 0.2474034620505992, + "gflops": 24106.30155702462, + "ms": 2501.6838984263677 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2446393126551028, + "calls": 0.2549267643142477, + "gflops": 24835.11930125468, + "ms": 2575.581353394051 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.24855675020919854, + "calls": 0.26198402130492676, + "gflops": 25518.78904362977, + "ms": 2644.901089912408 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.25234723459722813, + "calls": 0.2728362183754993, + "gflops": 26570.09251539525, + "ms": 2751.4965338038446 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2555614615399192, + "calls": 0.28169107856191744, + "gflops": 27427.904550639476, + "ms": 2838.4731843410295 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.25818509905301673, + "calls": 0.2905459387483355, + "gflops": 28285.716585883696, + "ms": 2925.4498348782136 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2606012166289425, + "calls": 0.30099866844207723, + "gflops": 29298.321770344926, + "ms": 3028.1215200988004 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2653611798288512, + "calls": 0.31178428761651134, + "gflops": 30343.1755275597, + "ms": 3134.063004211762 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.26884911383364996, + "calls": 0.3237017310252996, + "gflops": 31497.674432136508, + "ms": 3251.1218045588 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2729730863348169, + "calls": 0.336884154460719, + "gflops": 32774.717913176784, + "ms": 3380.6058406968637 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.27765278557626005, + "calls": 0.34800266311584555, + "gflops": 33851.8202431451, + "ms": 3489.817123702201 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2796074774091736, + "calls": 0.3575233022636485, + "gflops": 34774.129423896404, + "ms": 3583.333372024136 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28204715333825625, + "calls": 0.3648468708388815, + "gflops": 35483.59802447434, + "ms": 3655.268947656394 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2865153302629788, + "calls": 0.37490013315579224, + "gflops": 36457.504921631305, + "ms": 3754.0168742061296 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2895286620478625, + "calls": 0.3859520639147802, + "gflops": 37528.15753704891, + "ms": 3862.574197432991 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2924523439388038, + "calls": 0.39387483355525965, + "gflops": 38295.673568583225, + "ms": 3940.3954110715254 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29493220967499056, + "calls": 0.403595206391478, + "gflops": 39237.33189298665, + "ms": 4035.8735387288853 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29679244853296216, + "calls": 0.4097869507323568, + "gflops": 39837.15534620254, + "ms": 4096.691798127067 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29824032060818284, + "calls": 0.4171105193075899, + "gflops": 40546.62394678048, + "ms": 4168.627373759326 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.30060258544213647, + "calls": 0.42676431424766975, + "gflops": 41481.8325566332, + "ms": 4263.45154163821 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.30092311829652235, + "calls": 0.4334886817576565, + "gflops": 42133.253726254756, + "ms": 4329.501479264193 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29908337970039717, + "calls": 0.44560585885486015, + "gflops": 43307.101774483686, + "ms": 4448.5221589466555 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2977899223131896, + "calls": 0.4533288948069242, + "gflops": 44055.26866236588, + "ms": 4524.381493249765 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2982768381293317, + "calls": 0.45952063914780295, + "gflops": 44655.09211558176, + "ms": 4585.199752647946 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29713930300206126, + "calls": 0.46677762982689747, + "gflops": 45358.11100160898, + "ms": 4656.481368501729 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2981419006299502, + "calls": 0.4787616511318243, + "gflops": 46519.05962073651, + "ms": 4774.194128627242 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.30134517763453006, + "calls": 0.4900133155792277, + "gflops": 47609.06137980623, + "ms": 4884.7133311895295 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.3013386133731364, + "calls": 0.5019973368841545, + "gflops": 48770.009998933754, + "ms": 5002.426091315042 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.3006032949818893, + "calls": 0.5176431424766977, + "gflops": 50285.69291835024, + "ms": 5156.106639256684 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29926650757486717, + "calls": 0.5346205059920106, + "gflops": 51930.370128780894, + "ms": 5322.8663827678265 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29911586063303675, + "calls": 0.5474700399467377, + "gflops": 53175.16503706763, + "ms": 5449.080620013516 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29813893282180814, + "calls": 0.5575898801597869, + "gflops": 54155.52164877531, + "ms": 5548.482506341727 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2974509950363013, + "calls": 0.5675765645805592, + "gflops": 55122.97883138157, + "ms": 5646.576473112987 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29724357969129245, + "calls": 0.5804260985352863, + "gflops": 56367.773739668315, + "ms": 5772.790710358676 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29665775151403084, + "calls": 0.5938082556591212, + "gflops": 57664.16636436072, + "ms": 5904.236625832166 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2975661409457673, + "calls": 0.603595206391478, + "gflops": 58612.274403314856, + "ms": 6000.368713268001 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29631335243812956, + "calls": 0.6135153129161118, + "gflops": 59573.28187137042, + "ms": 6097.808720260787 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2960348088956203, + "calls": 0.6214380825565912, + "gflops": 60340.79790290472, + "ms": 6175.62993389932 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2951989613231927, + "calls": 0.6318242343541943, + "gflops": 61346.95337281523, + "ms": 6277.647659341431 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29386485972911613, + "calls": 0.6431424766977363, + "gflops": 62443.404846435675, + "ms": 6388.820821682193 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.292479005789999, + "calls": 0.6534620505992011, + "gflops": 63443.110601795495, + "ms": 6490.184587345831 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2926539927366717, + "calls": 0.6655126498002664, + "gflops": 64610.50893547373, + "ms": 6608.551307249818 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29278795045342987, + "calls": 0.6742343541944075, + "gflops": 65455.42154161653, + "ms": 6694.2200382300525 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29144368638516527, + "calls": 0.6844207723035952, + "gflops": 66442.22786787493, + "ms": 6794.275884336738 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2913839507485529, + "calls": 0.6950732356857523, + "gflops": 67474.18219598828, + "ms": 6898.909448892749 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2916685120367808, + "calls": 0.70219707057257, + "gflops": 68164.30165291409, + "ms": 6968.883145189583 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2908903102338221, + "calls": 0.70745672436751, + "gflops": 68673.82910242004, + "ms": 7020.545967689112 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29068952549932253, + "calls": 0.7203728362183754, + "gflops": 69925.07372525749, + "ms": 7147.4141647132765 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28982721462710204, + "calls": 0.7292942743009322, + "gflops": 70789.33547505244, + "ms": 7235.044775028938 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.289887284031958, + "calls": 0.7372836218375498, + "gflops": 71563.30122113743, + "ms": 7313.519948445944 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2914457139433469, + "calls": 0.7460719041278296, + "gflops": 72414.66354183097, + "ms": 7399.842639204656 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29113442803741185, + "calls": 0.7552596537949402, + "gflops": 73304.72414982873, + "ms": 7490.089088634216 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.29031315276962183, + "calls": 0.7674434087882822, + "gflops": 74485.02191260837, + "ms": 7609.763728095152 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.289990968297447, + "calls": 0.7755659121171771, + "gflops": 75271.8870877948, + "ms": 7689.5468210691115 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2901920651624007, + "calls": 0.7833555259653795, + "gflops": 76026.50369022769, + "ms": 7766.060115150694 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2901181158873772, + "calls": 0.7904793608521972, + "gflops": 76716.62314715351, + "ms": 7836.033811447528 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28973901353907683, + "calls": 0.7987350199733687, + "gflops": 77516.38775144133, + "ms": 7917.124823978435 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28888237446044834, + "calls": 0.8127829560585885, + "gflops": 78877.27752164083, + "ms": 8055.110337236676 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2890360783246395, + "calls": 0.8225699067909453, + "gflops": 79825.38556059496, + "ms": 8151.242424672511 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28767900194586976, + "calls": 0.8346205059920105, + "gflops": 80992.78389427319, + "ms": 8269.6091445765 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28869706841582, + "calls": 0.8463382157123837, + "gflops": 82127.9336551979, + "ms": 8384.706065588114 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2875418385011496, + "calls": 0.8567243675099867, + "gflops": 83134.08912510841, + "ms": 8486.723791030225 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28776751513017684, + "calls": 0.86491344873502, + "gflops": 83927.40401484555, + "ms": 8567.160843782658 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2880879394318149, + "calls": 0.8767643142476697, + "gflops": 85075.45320487165, + "ms": 8683.565684351219 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.287273877029594, + "calls": 0.8856857523302264, + "gflops": 85939.71495466659, + "ms": 8771.196294666881 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28547632945289336, + "calls": 0.8951398135818909, + "gflops": 86855.5744208672, + "ms": 8864.05858321034 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.284269379519687, + "calls": 0.9029960053262316, + "gflops": 87616.64073785077, + "ms": 8941.225837070398 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.28250852235623786, + "calls": 0.9131824234354193, + "gflops": 88603.44706410918, + "ms": 9041.281683177083 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2814829677111451, + "calls": 0.9243675099866845, + "gflops": 89686.99910862821, + "ms": 9151.146925960897 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2800886018148173, + "calls": 0.9328229027962717, + "gflops": 90506.11285656819, + "ms": 9234.199817827232 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.27854520316604037, + "calls": 0.9400798934753661, + "gflops": 91209.13174259539, + "ms": 9305.481433681012 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.27746458817138653, + "calls": 0.9482689747003995, + "gflops": 92002.44663233255, + "ms": 9385.918486433447 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2760878894837333, + "calls": 0.9603861517976032, + "gflops": 93176.29468056148, + "ms": 9504.939166115912 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.27468746115763226, + "calls": 0.9677762982689748, + "gflops": 93892.21299569013, + "ms": 9577.528701526644 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.2732948963509506, + "calls": 0.9782290279627164, + "gflops": 94904.81818015134, + "ms": 9680.20038674723 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.27348968306795485, + "calls": 0.9849533954727031, + "gflops": 95556.2393497729, + "ms": 9746.250324373212 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "metric": 0.27001009067886245, + "calls": 1.0, + "gflops": 97013.87483823301, + "ms": 9894.045234308578 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18129624718318377, + "calls": 0.0, + "gflops": 135.97853184000002, + "ms": 46.30809425361804 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1784481207469488, + "calls": 0.17942743009320905, + "gflops": 17517.959245999285, + "ms": 1808.7296972439362 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1780928651788194, + "calls": 0.19340878828229027, + "gflops": 18872.39930164806, + "ms": 1946.0612507237013 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17829989216687975, + "calls": 0.20306258322237017, + "gflops": 19807.60791150079, + "ms": 2040.8854186025867 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1779854683890517, + "calls": 0.20838881491344874, + "gflops": 20323.585075557465, + "ms": 2093.2022008805925 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17917657728013692, + "calls": 0.21537949400798934, + "gflops": 21000.805103381852, + "ms": 2161.867977620475 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17803687432776102, + "calls": 0.23135818908122505, + "gflops": 22548.736595551884, + "ms": 2318.8183244544925 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18026802885144627, + "calls": 0.24800266311584554, + "gflops": 24161.165233228996, + "ms": 2482.3082690732604 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18119827871024874, + "calls": 0.2589880159786951, + "gflops": 25225.368134095894, + "ms": 2590.211632521647 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17963665085316138, + "calls": 0.27430093209054596, + "gflops": 26708.802480758837, + "ms": 2740.6223815709136 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17924095839307458, + "calls": 0.281624500665779, + "gflops": 27418.271081336767, + "ms": 2812.5579572031716 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1789157174618718, + "calls": 0.28761651131824234, + "gflops": 27998.745390900527, + "ms": 2871.4143372659278 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1787206845800119, + "calls": 0.2969374167776298, + "gflops": 28901.70542799971, + "ms": 2962.9687062524376 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17813242189706796, + "calls": 0.30459387483355527, + "gflops": 29643.422601331185, + "ms": 3038.174080777071 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17772000842137498, + "calls": 0.3129161118508655, + "gflops": 30449.636920169738, + "ms": 3119.919053086455 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17791259734473341, + "calls": 0.31990679094540614, + "gflops": 31126.85694799413, + "ms": 3188.5848298263377 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1789653934599337, + "calls": 0.3322237017310253, + "gflops": 32320.05413987519, + "ms": 3309.5673888442257 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18042682200794885, + "calls": 0.34054593874833555, + "gflops": 33126.268458713756, + "ms": 3391.31236115361 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17962877325912951, + "calls": 0.3488681757656458, + "gflops": 33932.48277755231, + "ms": 3473.057333462994 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18158304685510065, + "calls": 0.3581890812250333, + "gflops": 34835.44281465149, + "ms": 3564.611702449504 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18069239723097547, + "calls": 0.37017310252996005, + "gflops": 35996.39143377901, + "ms": 3682.3244625750167 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1812845803851709, + "calls": 0.37683089214380827, + "gflops": 36641.362888849864, + "ms": 3747.720440422524 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1824781664689562, + "calls": 0.38348868175765644, + "gflops": 37286.3343439207, + "ms": 3813.116418270031 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18220594501399923, + "calls": 0.4021304926764314, + "gflops": 39092.254418119075, + "ms": 3996.225156243051 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18162498930772586, + "calls": 0.41311584553928093, + "gflops": 40156.457318985966, + "ms": 4104.128519691438 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18066434967152634, + "calls": 0.42676431424766975, + "gflops": 41478.6488018812, + "ms": 4238.190274278827 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1803913789289444, + "calls": 0.4324234354194407, + "gflops": 42026.874538691416, + "ms": 4293.776855449209 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17959468902865128, + "calls": 0.44107856191744343, + "gflops": 42865.33743028352, + "ms": 4378.791626650968 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17863383514615866, + "calls": 0.45772303595206393, + "gflops": 44477.766067960634, + "ms": 4542.281571269737 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17957308538379646, + "calls": 0.4717043941411451, + "gflops": 45832.20612360941, + "ms": 4679.6131247495005 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17910245145856155, + "calls": 0.4826897470039947, + "gflops": 46896.40902447631, + "ms": 4787.516488197888 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.17871833907402393, + "calls": 0.4896804260985353, + "gflops": 47573.62905230069, + "ms": 4856.182264937771 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18015785955296285, + "calls": 0.5013315579227696, + "gflops": 48702.32909867467, + "ms": 4970.625226170908 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1810878167493375, + "calls": 0.5116511318242344, + "gflops": 49702.034854034486, + "ms": 5071.988991834544 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18177962418520882, + "calls": 0.5183089214380826, + "gflops": 50347.00630910533, + "ms": 5137.384969682052 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18246323539519374, + "calls": 0.5252996005326231, + "gflops": 51024.226336929714, + "ms": 5206.050746421934 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1839876166906015, + "calls": 0.5396138482023968, + "gflops": 52410.914965332035, + "ms": 5346.652098794074 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1846674213907964, + "calls": 0.5452729693741678, + "gflops": 52959.14070214226, + "ms": 5402.2386799644555 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18676294233791663, + "calls": 0.559254327563249, + "gflops": 54313.58075779103, + "ms": 5539.57023344422 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.18938663221175195, + "calls": 0.572237017310253, + "gflops": 55571.275095179175, + "ms": 5667.092390246859 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1909571002676625, + "calls": 0.5862183754993342, + "gflops": 56925.71515082795, + "ms": 5804.423943726624 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1934300101790686, + "calls": 0.5998668442077231, + "gflops": 58247.90663372319, + "ms": 5938.485698314014 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1973701682130997, + "calls": 0.6191744340878829, + "gflops": 60118.32385342864, + "ms": 6128.134034071785 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.1986727924998457, + "calls": 0.6264980026631158, + "gflops": 60827.792454006565, + "ms": 6200.069609704043 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.20186289191698192, + "calls": 0.6384820239680427, + "gflops": 61988.74107313409, + "ms": 6317.782369829556 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.20246031058520178, + "calls": 0.6461384820239681, + "gflops": 62730.45824646557, + "ms": 6392.987744354189 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.20349944137622245, + "calls": 0.653462050599201, + "gflops": 63439.92684704349, + "ms": 6464.923319986447 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.20442990090722246, + "calls": 0.6597869507323568, + "gflops": 64052.64972936079, + "ms": 6527.049498941578 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2044993942500819, + "calls": 0.6661118508655126, + "gflops": 64665.372611678096, + "ms": 6589.17567789671 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2064379542733177, + "calls": 0.6734354194407457, + "gflops": 65374.84121225603, + "ms": 6661.1112535289685 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2065831206227621, + "calls": 0.6820905459387483, + "gflops": 66213.30410384812, + "ms": 6746.126024730727 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.20802802394105854, + "calls": 0.6927430093209055, + "gflops": 67245.25843196148, + "ms": 6850.759589286739 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.21169680596278434, + "calls": 0.7090545938748336, + "gflops": 68825.43849688505, + "ms": 7010.979735013132 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2139280527103987, + "calls": 0.7170439414114514, + "gflops": 69599.40424297006, + "ms": 7089.454908430141 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2146900231025185, + "calls": 0.7243675099866844, + "gflops": 70308.87284354799, + "ms": 7161.390484062398 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.21653561577692484, + "calls": 0.7330226364846871, + "gflops": 71147.33573514009, + "ms": 7246.405255264158 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.21721689648133474, + "calls": 0.7363515312916112, + "gflops": 71469.82146267552, + "ms": 7279.1032441879115 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.22103445955201576, + "calls": 0.7486684420772304, + "gflops": 72663.01865455658, + "ms": 7400.0858032057995 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.22340134248367294, + "calls": 0.757323568575233, + "gflops": 73501.48154614867, + "ms": 7485.100574407558 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2269322568854726, + "calls": 0.7639813581890812, + "gflops": 74146.45300121952, + "ms": 7550.496552255066 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.22757452798038888, + "calls": 0.7733022636484687, + "gflops": 75049.4130383187, + "ms": 7642.050921241575 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.227821050912831, + "calls": 0.784953395472703, + "gflops": 76178.11308469268, + "ms": 7756.493882474713 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.22991865671924006, + "calls": 0.7939414114513982, + "gflops": 77048.82454903833, + "ms": 7844.7784525688485 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.23486191981046528, + "calls": 0.8039280958721704, + "gflops": 78016.28173164459, + "ms": 7942.8724193401085 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.23803916047544685, + "calls": 0.8122503328894807, + "gflops": 78822.49605048315, + "ms": 8024.617391649493 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.24022947987632814, + "calls": 0.8215712383488681, + "gflops": 79725.45608758234, + "ms": 8116.171760636003 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.24528627405740036, + "calls": 0.8388814913448736, + "gflops": 81402.38187076653, + "ms": 8286.201303039523 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.249537136326177, + "calls": 0.8535286284953395, + "gflops": 82821.3190719224, + "ms": 8430.072454304036 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.25211074301800224, + "calls": 0.8638482023968043, + "gflops": 83821.0248272822, + "ms": 8531.436219967674 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2540042317315216, + "calls": 0.8745006657789614, + "gflops": 84852.97915539556, + "ms": 8636.069784523685 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2540996517958595, + "calls": 0.8764980026631158, + "gflops": 85046.4705919168, + "ms": 8655.688577877936 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.25436194678023316, + "calls": 0.8841544607190412, + "gflops": 85788.18776524828, + "ms": 8730.89395240257 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2534339447803836, + "calls": 0.8891478029294274, + "gflops": 86271.91635655142, + "ms": 8779.9409357882 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2563240880787248, + "calls": 0.8981358189081226, + "gflops": 87142.62782089706, + "ms": 8868.225505882336 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.25551845731687656, + "calls": 0.9054593874833555, + "gflops": 87852.09642147498, + "ms": 8940.161081514594 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2597490004762105, + "calls": 0.9221038615179761, + "gflops": 89464.5250591521, + "ms": 9103.651026133362 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.26306323674307786, + "calls": 0.9254327563249002, + "gflops": 89787.01078668753, + "ms": 9136.349015057116 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.26218185491449153, + "calls": 0.9364181091877497, + "gflops": 90851.21368755442, + "ms": 9244.252378505502 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.26357052137555514, + "calls": 0.953728362183755, + "gflops": 92528.13947073862, + "ms": 9414.281920909021 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.26400290307863034, + "calls": 0.96271637816245, + "gflops": 93398.85093508427, + "ms": 9502.566491003154 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2647475724296899, + "calls": 0.9700399467376831, + "gflops": 94108.3195356622, + "ms": 9574.502066635414 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.2649306125590882, + "calls": 0.987017310252996, + "gflops": 95752.99674609285, + "ms": 9741.261810146556 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "metric": 0.27001009067886245, + "calls": 1.0, + "gflops": 97010.691083481, + "ms": 9868.783966949195 + } + ], + "protocol": "Five-seed mean call rates times existing measured mean component costs; not integrated latency measurements." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/earlier_results.json b/releases/20260913_lr/paper_data/earlier_results.json new file mode 100644 index 0000000000000000000000000000000000000000..28fc0dede625517749a11d9342fa6c5f9b30cfd7 --- /dev/null +++ b/releases/20260913_lr/paper_data/earlier_results.json @@ -0,0 +1,6448 @@ +{ + "status": "complete", + "planned": 264, + "completed": 264, + "missing": [], + "rows": [ + { + "id": "formal_0205", + "tag": "baseline_25pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12161748372078302, + "fixed25_actual_fraction": 0.26897470039946736, + "fixed25_rescue": 1103, + "fixed25_harm": 518, + "rank25_metric": 0.12103479129839005, + "validation_selection": 0.13942018347651794, + "best_epoch": 1, + "seconds": 3.9248127937316895 + }, + { + "id": "formal_0206", + "tag": "baseline_50pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11138731664705415, + "fixed25_actual_fraction": 0.25798934753661784, + "fixed25_rescue": 1337, + "fixed25_harm": 464, + "rank25_metric": 0.11157434949576527, + "validation_selection": 0.10286119045745137, + "best_epoch": 3, + "seconds": 5.573187828063965 + }, + { + "id": "extra_0031", + "tag": "representation_32", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12677122948456399, + "fixed25_actual_fraction": 0.25099866844207724, + "fixed25_rescue": 1352, + "fixed25_harm": 445, + "rank25_metric": 0.12664773557636813, + "validation_selection": 0.10657501019916582, + "best_epoch": 16, + "seconds": 17.047492742538452 + }, + { + "id": "extra_0032", + "tag": "representation_64", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.1199038046999191, + "fixed25_actual_fraction": 0.24500665778961384, + "fixed25_rescue": 1268, + "fixed25_harm": 454, + "rank25_metric": 0.12123578062564436, + "validation_selection": 0.10492587324457393, + "best_epoch": 1, + "seconds": 6.62996768951416 + }, + { + "id": "extra_0033", + "tag": "student_8M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12852176195071466, + "fixed25_actual_fraction": 0.28129161118508655, + "fixed25_rescue": 1430, + "fixed25_harm": 528, + "rank25_metric": 0.12791702188360424, + "validation_selection": 0.1101086951271347, + "best_epoch": 16, + "seconds": 17.678598165512085 + }, + { + "id": "formal_0114", + "tag": "baseline_100pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11409050364241882, + "fixed25_actual_fraction": 0.2333555259653795, + "fixed25_rescue": 1155, + "fixed25_harm": 428, + "rank25_metric": 0.1142018021781044, + "validation_selection": 0.09196748903785351, + "best_epoch": 1, + "seconds": 7.1226747035980225 + }, + { + "id": "formal_0115", + "tag": "drop_image", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12791159721178413, + "fixed25_actual_fraction": 0.2626498002663116, + "fixed25_rescue": 1369, + "fixed25_harm": 464, + "rank25_metric": 0.12685189507799977, + "validation_selection": 0.12058376950960199, + "best_epoch": 47, + "seconds": 33.72947406768799 + }, + { + "id": "formal_0116", + "tag": "drop_text", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11259143129519858, + "fixed25_actual_fraction": 0.25, + "fixed25_rescue": 1237, + "fixed25_harm": 472, + "rank25_metric": 0.11259143129519858, + "validation_selection": 0.10775056216618559, + "best_epoch": 1, + "seconds": 7.603894233703613 + }, + { + "id": "formal_0117", + "tag": "drop_hidden", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12396779300064459, + "fixed25_actual_fraction": 0.2589880159786951, + "fixed25_rescue": 1318, + "fixed25_harm": 473, + "rank25_metric": 0.12250988658190227, + "validation_selection": 0.09268882908524566, + "best_epoch": 4, + "seconds": 9.375170469284058 + }, + { + "id": "formal_0118", + "tag": "drop_confidence", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.1079733386177777, + "fixed25_actual_fraction": 0.24866844207723035, + "fixed25_rescue": 1260, + "fixed25_harm": 450, + "rank25_metric": 0.1079857737928408, + "validation_selection": 0.0975193305968221, + "best_epoch": 1, + "seconds": 7.336633205413818 + }, + { + "id": "formal_0119", + "tag": "hidden_position_feature_blocks__layer_03", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_03", + "fixed25_metric": 0.12865126241054672, + "fixed25_actual_fraction": 0.25, + "fixed25_rescue": 1285, + "fixed25_harm": 410, + "rank25_metric": 0.12865126241054672, + "validation_selection": 0.11612297444582934, + "best_epoch": 15, + "seconds": 17.645463705062866 + }, + { + "id": "formal_0120", + "tag": "hidden_position_feature_blocks__layer_24", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_24", + "fixed25_metric": 0.13130687829759294, + "fixed25_actual_fraction": 0.28395472703062585, + "fixed25_rescue": 1454, + "fixed25_harm": 484, + "rank25_metric": 0.12813467068447604, + "validation_selection": 0.11472043231101983, + "best_epoch": 5, + "seconds": 10.408715963363647 + }, + { + "id": "formal_0121", + "tag": "hidden_multi_layer", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_03", + "fixed25_metric": 0.1319786802729061, + "fixed25_actual_fraction": 0.28961384820239683, + "fixed25_rescue": 1468, + "fixed25_harm": 495, + "rank25_metric": 0.1291292294447022, + "validation_selection": 0.12972548686634056, + "best_epoch": 26, + "seconds": 25.749829053878784 + }, + { + "id": "formal_0122", + "tag": "representation_128", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12735166754382224, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1334, + "fixed25_harm": 491, + "rank25_metric": 0.12825332409803333, + "validation_selection": 0.10683973009120012, + "best_epoch": 1, + "seconds": 7.299476146697998 + }, + { + "id": "formal_0123", + "tag": "representation_256", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.1296190849296716, + "fixed25_actual_fraction": 0.25798934753661784, + "fixed25_rescue": 1349, + "fixed25_harm": 478, + "rank25_metric": 0.12942150404041414, + "validation_selection": 0.1047735090992197, + "best_epoch": 4, + "seconds": 9.053397178649902 + }, + { + "id": "formal_0124", + "tag": "student_1M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12038804662706637, + "fixed25_actual_fraction": 0.24167776298268975, + "fixed25_rescue": 1274, + "fixed25_harm": 442, + "rank25_metric": 0.12037820583822222, + "validation_selection": 0.1010803735167908, + "best_epoch": 5, + "seconds": 11.015799045562744 + }, + { + "id": "formal_0125", + "tag": "student_2M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12899655423992204, + "fixed25_actual_fraction": 0.27529960053262315, + "fixed25_rescue": 1369, + "fixed25_harm": 532, + "rank25_metric": 0.12740185163408788, + "validation_selection": 0.11096710446690258, + "best_epoch": 16, + "seconds": 17.74562120437622 + }, + { + "id": "formal_0126", + "tag": "target_error", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12363065005052852, + "fixed25_actual_fraction": 0.26531291611185087, + "fixed25_rescue": 1430, + "fixed25_harm": 491, + "rank25_metric": 0.12410125357669675, + "validation_selection": 0.1136359842515143, + "best_epoch": 36, + "seconds": 33.95516061782837 + }, + { + "id": "formal_0127", + "tag": "target_four_state", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12288572047662566, + "fixed25_actual_fraction": 0.26198402130492676, + "fixed25_rescue": 1220, + "fixed25_harm": 355, + "rank25_metric": 0.12364678175576603, + "validation_selection": 0.13440378191215316, + "best_epoch": 15, + "seconds": 17.694798231124878 + }, + { + "id": "formal_0128", + "tag": "fusion_attention", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.1205106128632194, + "fixed25_actual_fraction": 0.25798934753661784, + "fixed25_rescue": 1335, + "fixed25_harm": 490, + "rank25_metric": 0.11993843825140044, + "validation_selection": 0.11337000715277953, + "best_epoch": 16, + "seconds": 24.939371585845947 + }, + { + "id": "formal_0137", + "tag": "target_error", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.13298608758280392, + "fixed25_actual_fraction": 0.27729693741677763, + "fixed25_rescue": 1315, + "fixed25_harm": 506, + "rank25_metric": 0.13302267602819143, + "validation_selection": 0.12208124766901975, + "best_epoch": 13, + "seconds": 13.368404150009155 + }, + { + "id": "formal_0207", + "tag": "baseline_25pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14471609214350373, + "fixed25_actual_fraction": 0.2892809587217044, + "fixed25_rescue": 1383, + "fixed25_harm": 258, + "rank25_metric": 0.14346432207238047, + "validation_selection": 0.16233096000534752, + "best_epoch": 17, + "seconds": 8.028136253356934 + }, + { + "id": "formal_0208", + "tag": "baseline_50pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1499641421353766, + "fixed25_actual_fraction": 0.26797603195739017, + "fixed25_rescue": 1348, + "fixed25_harm": 225, + "rank25_metric": 0.14991013725904712, + "validation_selection": 0.14960536227871302, + "best_epoch": 14, + "seconds": 7.761091709136963 + }, + { + "id": "extra_0034", + "tag": "representation_32", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 32, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.15246154351186944, + "fixed25_actual_fraction": 0.26531291611185087, + "fixed25_rescue": 1381, + "fixed25_harm": 233, + "rank25_metric": 0.15246107875534035, + "validation_selection": 0.13959259383410275, + "best_epoch": 11, + "seconds": 9.594881772994995 + }, + { + "id": "extra_0035", + "tag": "representation_64", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 64, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14746508696753746, + "fixed25_actual_fraction": 0.27463382157123833, + "fixed25_rescue": 1305, + "fixed25_harm": 253, + "rank25_metric": 0.14258824887550908, + "validation_selection": 0.13878028268730078, + "best_epoch": 1, + "seconds": 6.531559228897095 + }, + { + "id": "extra_0036", + "tag": "student_8M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 10.990516424179077 + }, + { + "id": "formal_0126", + "tag": "target_error", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.13996613214172965, + "fixed25_actual_fraction": 0.26564580559254325, + "fixed25_rescue": 1525, + "fixed25_harm": 248, + "rank25_metric": 0.13802590609803905, + "validation_selection": 0.13929873372091645, + "best_epoch": 30, + "seconds": 33.95516061782837 + }, + { + "id": "formal_0129", + "tag": "baseline_100pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 13.48578929901123 + }, + { + "id": "formal_0130", + "tag": "hidden_position_feature_blocks__layer_03", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_03", + "fixed25_metric": 0.15010748535267285, + "fixed25_actual_fraction": 0.2513315579227696, + "fixed25_rescue": 1192, + "fixed25_harm": 205, + "rank25_metric": 0.14998631841175578, + "validation_selection": 0.14027716877702798, + "best_epoch": 12, + "seconds": 11.686585903167725 + }, + { + "id": "formal_0131", + "tag": "hidden_position_feature_blocks__layer_24", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_24", + "fixed25_metric": 0.13107435550044824, + "fixed25_actual_fraction": 0.2643142476697736, + "fixed25_rescue": 1339, + "fixed25_harm": 193, + "rank25_metric": 0.12841698578626595, + "validation_selection": 0.1314715107927409, + "best_epoch": 1, + "seconds": 6.488433599472046 + }, + { + "id": "formal_0132", + "tag": "hidden_multi_layer", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_03", + "fixed25_metric": 0.1465080909816311, + "fixed25_actual_fraction": 0.27463382157123833, + "fixed25_rescue": 1540, + "fixed25_harm": 254, + "rank25_metric": 0.14337516260382116, + "validation_selection": 0.1327068555595483, + "best_epoch": 9, + "seconds": 10.790199518203735 + }, + { + "id": "formal_0133", + "tag": "representation_128", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 128, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14803489232686926, + "fixed25_actual_fraction": 0.2739680426098535, + "fixed25_rescue": 1403, + "fixed25_harm": 224, + "rank25_metric": 0.14798656495937493, + "validation_selection": 0.14100279346596564, + "best_epoch": 10, + "seconds": 10.514626741409302 + }, + { + "id": "formal_0134", + "tag": "representation_256", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 256, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14730603556730637, + "fixed25_actual_fraction": 0.2626498002663116, + "fixed25_rescue": 1281, + "fixed25_harm": 261, + "rank25_metric": 0.1447121436747074, + "validation_selection": 0.14000847459984167, + "best_epoch": 1, + "seconds": 5.760014295578003 + }, + { + "id": "formal_0135", + "tag": "student_1M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 12.58092188835144 + }, + { + "id": "formal_0136", + "tag": "student_2M", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 13.341734886169434 + }, + { + "id": "formal_0137", + "tag": "target_error", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1550677773614836, + "fixed25_actual_fraction": 0.2872836218375499, + "fixed25_rescue": 1438, + "fixed25_harm": 248, + "rank25_metric": 0.15400963778422433, + "validation_selection": 0.14514986541925773, + "best_epoch": 10, + "seconds": 13.368404150009155 + }, + { + "id": "formal_0138", + "tag": "target_four_state", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1008636659774502, + "fixed25_actual_fraction": 0.26131824234354195, + "fixed25_rescue": 873, + "fixed25_harm": 87, + "rank25_metric": 0.10092160765502416, + "validation_selection": 0.11724983142173177, + "best_epoch": 1, + "seconds": 5.6207404136657715 + }, + { + "id": "formal_0213", + "tag": "baseline_25pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.245059255252362, + "best_epoch": 2, + "seconds": 2.557858467102051 + }, + { + "id": "formal_0214", + "tag": "baseline_50pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.2411747600492345, + "best_epoch": 1, + "seconds": 3.123697280883789 + }, + { + "id": "extra_0043", + "tag": "representation_32", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.24346726209631106, + "fixed25_actual_fraction": 0.4374167776298269, + "fixed25_rescue": 1092, + "fixed25_harm": 880, + "rank25_metric": 0.2235506221639314, + "validation_selection": 0.2322722938123662, + "best_epoch": 8, + "seconds": 7.180579900741577 + }, + { + "id": "extra_0044", + "tag": "representation_64", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.2235506221639314, + "validation_selection": 0.23309404497704123, + "best_epoch": 4, + "seconds": 6.816771030426025 + }, + { + "id": "extra_0045", + "tag": "student_8M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 7.2654266357421875 + }, + { + "id": "formal_0169", + "tag": "baseline_100pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 9.216264009475708 + }, + { + "id": "formal_0170", + "tag": "representation_128", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.2235506221639314, + "validation_selection": 0.23309404497704123, + "best_epoch": 4, + "seconds": 8.15360426902771 + }, + { + "id": "formal_0171", + "tag": "representation_256", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.2235506221639314, + "validation_selection": 0.23309404497704123, + "best_epoch": 4, + "seconds": 9.3523588180542 + }, + { + "id": "formal_0172", + "tag": "student_1M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 8.9949951171875 + }, + { + "id": "formal_0173", + "tag": "student_2M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 9.508620262145996 + }, + { + "id": "formal_0174", + "tag": "target_error", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.23040572580934265, + "best_epoch": 4, + "seconds": 9.888505697250366 + }, + { + "id": "formal_0175", + "tag": "target_four_state", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22322830961852883, + "validation_selection": 0.22881953100143768, + "best_epoch": 3, + "seconds": 6.8639256954193115 + }, + { + "id": "formal_0176", + "tag": "harm_cost_2", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.23040572580934265, + "best_epoch": 7, + "seconds": 10.785712957382202 + }, + { + "id": "formal_0182", + "tag": "target_error", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.23040572580934265, + "best_epoch": 4, + "seconds": 9.399006128311157 + }, + { + "id": "formal_0215", + "tag": "baseline_25pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 7, + "seconds": 3.354280948638916 + }, + { + "id": "formal_0216", + "tag": "baseline_50pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 14, + "seconds": 4.801910877227783 + }, + { + "id": "extra_0046", + "tag": "representation_32", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2917660075719453, + "fixed25_actual_fraction": 0.43209054593874835, + "fixed25_rescue": 1103, + "fixed25_harm": 315, + "rank25_metric": 0.2631043176189476, + "validation_selection": 0.24807135884361062, + "best_epoch": 8, + "seconds": 9.370010614395142 + }, + { + "id": "extra_0047", + "tag": "representation_64", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 3, + "seconds": 5.145432233810425 + }, + { + "id": "extra_0048", + "tag": "student_8M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 7.741400241851807 + }, + { + "id": "formal_0174", + "tag": "target_error", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.249028639438933, + "best_epoch": 4, + "seconds": 9.888505697250366 + }, + { + "id": "formal_0177", + "tag": "baseline_100pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 11.173577785491943 + }, + { + "id": "formal_0178", + "tag": "representation_128", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 13.545645475387573 + }, + { + "id": "formal_0179", + "tag": "representation_256", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 12.355992555618286 + }, + { + "id": "formal_0180", + "tag": "student_1M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 15.147192239761353 + }, + { + "id": "formal_0181", + "tag": "student_2M", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 11.001330137252808 + }, + { + "id": "formal_0182", + "tag": "target_error", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.249028639438933, + "best_epoch": 4, + "seconds": 9.399006128311157 + }, + { + "id": "formal_0183", + "tag": "target_four_state", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.24575590452405222, + "fixed25_actual_fraction": 0.21837549933422104, + "fixed25_rescue": 716, + "fixed25_harm": 113, + "rank25_metric": 0.26408669760452147, + "validation_selection": 0.23738117267347428, + "best_epoch": 4, + "seconds": 8.231167554855347 + }, + { + "id": "formal_0184", + "tag": "harm_cost_2", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.24575590452405222, + "fixed25_actual_fraction": 0.21837549933422104, + "fixed25_rescue": 716, + "fixed25_harm": 113, + "rank25_metric": 0.26408669760452147, + "validation_selection": 0.23758094807145608, + "best_epoch": 13, + "seconds": 10.565741539001465 + }, + { + "id": "formal_0185", + "tag": "baseline_25pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7849499482222989, + "fixed25_actual_fraction": 0.2587159130134622, + "fixed25_rescue": 31, + "fixed25_harm": 201, + "rank25_metric": 0.7884017949603037, + "validation_selection": 0.9741666666666666, + "best_epoch": 12, + "seconds": 5.802937984466553 + }, + { + "id": "formal_0186", + "tag": "baseline_50pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7904729030031067, + "fixed25_actual_fraction": 0.2549188816016569, + "fixed25_rescue": 38, + "fixed25_harm": 176, + "rank25_metric": 0.7930617880566103, + "validation_selection": 0.9731666666666667, + "best_epoch": 7, + "seconds": 7.1614015102386475 + }, + { + "id": "extra_0001", + "tag": "representation_32", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7916810493614084, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 5.997061252593994 + }, + { + "id": "extra_0002", + "tag": "representation_64", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7846047635484984, + "fixed25_actual_fraction": 0.2613047980669658, + "fixed25_rescue": 34, + "fixed25_harm": 206, + "rank25_metric": 0.7889195719710045, + "validation_selection": 0.9743333333333333, + "best_epoch": 12, + "seconds": 8.659805059432983 + }, + { + "id": "extra_0003", + "tag": "student_8M", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 7.315621376037598 + }, + { + "id": "formal_0001", + "tag": "baseline_100pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 2.9200518131256104 + }, + { + "id": "formal_0002", + "tag": "representation_128", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7846047635484984, + "fixed25_actual_fraction": 0.2613047980669658, + "fixed25_rescue": 34, + "fixed25_harm": 206, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744999999999999, + "best_epoch": 9, + "seconds": 3.735602378845215 + }, + { + "id": "formal_0003", + "tag": "representation_256", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7877114256127028, + "fixed25_actual_fraction": 0.2542285122540559, + "fixed25_rescue": 30, + "fixed25_harm": 184, + "rank25_metric": 0.7908180876769072, + "validation_selection": 0.9745833333333332, + "best_epoch": 16, + "seconds": 4.4355103969573975 + }, + { + "id": "formal_0004", + "tag": "student_1M", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 2.9487767219543457 + }, + { + "id": "formal_0005", + "tag": "student_2M", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 3.1687443256378174 + }, + { + "id": "formal_0006", + "tag": "target_error", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6755264066275457, + "fixed25_actual_fraction": 0.25716258198136005, + "fixed25_rescue": 89, + "fixed25_harm": 893, + "rank25_metric": 0.6810493614083535, + "validation_selection": 0.8948333333333334, + "best_epoch": 8, + "seconds": 3.2323741912841797 + }, + { + "id": "formal_0007", + "tag": "target_four_state", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7863306869175009, + "fixed25_actual_fraction": 0.2569899896444598, + "fixed25_rescue": 35, + "fixed25_harm": 197, + "rank25_metric": 0.7906454953400069, + "validation_selection": 0.9722500000000001, + "best_epoch": 2, + "seconds": 2.0133934020996094 + }, + { + "id": "formal_0008", + "tag": "harm_cost_2", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9745833333333332, + "best_epoch": 7, + "seconds": 3.1982953548431396 + }, + { + "id": "formal_0016", + "tag": "target_error", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.5395236451501554, + "fixed25_actual_fraction": 0.5980324473593372, + "fixed25_rescue": 227, + "fixed25_harm": 1819, + "rank25_metric": 0.7454263030721436, + "validation_selection": 0.9059999999999999, + "best_epoch": 2, + "seconds": 2.198946475982666 + }, + { + "id": "formal_0187", + "tag": "baseline_25pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6874352778736624, + "fixed25_actual_fraction": 0.6039005868139454, + "fixed25_rescue": 358, + "fixed25_harm": 1093, + "rank25_metric": 0.8060062133241284, + "validation_selection": 0.9316666666666666, + "best_epoch": 5, + "seconds": 4.623678922653198 + }, + { + "id": "formal_0188", + "tag": "baseline_50pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.694338971349672, + "fixed25_actual_fraction": 0.5997583707283397, + "fixed25_rescue": 357, + "fixed25_harm": 1052, + "rank25_metric": 0.8096306523990335, + "validation_selection": 0.9334166666666667, + "best_epoch": 3, + "seconds": 4.7386627197265625 + }, + { + "id": "extra_0004", + "tag": "representation_32", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7309285467725233, + "fixed25_actual_fraction": 0.5091473938557128, + "fixed25_rescue": 334, + "fixed25_harm": 817, + "rank25_metric": 0.7766655160510874, + "validation_selection": 0.9439166666666667, + "best_epoch": 16, + "seconds": 9.204903364181519 + }, + { + "id": "extra_0005", + "tag": "representation_64", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7074559889540905, + "fixed25_actual_fraction": 0.5857783914394201, + "fixed25_rescue": 352, + "fixed25_harm": 971, + "rank25_metric": 0.8108387987573352, + "validation_selection": 0.9390833333333334, + "best_epoch": 2, + "seconds": 5.327153205871582 + }, + { + "id": "extra_0006", + "tag": "student_8M", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 5.070952415466309 + }, + { + "id": "formal_0006", + "tag": "target_error", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7433551950293407, + "fixed25_actual_fraction": 0.2604418363824646, + "fixed25_rescue": 132, + "fixed25_harm": 543, + "rank25_metric": 0.7474974111149465, + "validation_selection": 0.9173333333333333, + "best_epoch": 1, + "seconds": 3.2323741912841797 + }, + { + "id": "formal_0009", + "tag": "baseline_100pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 2.053396463394165 + }, + { + "id": "formal_0010", + "tag": "drop_confidence", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.8065239903348291, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 48, + "fixed25_harm": 93, + "rank25_metric": 0.8065239903348291, + "validation_selection": 0.9790833333333333, + "best_epoch": 1, + "seconds": 2.09718656539917 + }, + { + "id": "formal_0011", + "tag": "drop_output", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6950293406972731, + "fixed25_actual_fraction": 0.5472903003106662, + "fixed25_rescue": 349, + "fixed25_harm": 1040, + "rank25_metric": 0.7894373489817053, + "validation_selection": 0.9210833333333334, + "best_epoch": 1, + "seconds": 1.6060831546783447 + }, + { + "id": "formal_0012", + "tag": "representation_128", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7711425612702796, + "fixed25_actual_fraction": 0.381429064549534, + "fixed25_rescue": 227, + "fixed25_harm": 477, + "rank25_metric": 0.7897825336555057, + "validation_selection": 0.9603333333333334, + "best_epoch": 50, + "seconds": 9.383041143417358 + }, + { + "id": "formal_0013", + "tag": "representation_256", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6919226786330687, + "fixed25_actual_fraction": 0.5954435623058336, + "fixed25_rescue": 357, + "fixed25_harm": 1066, + "rank25_metric": 0.8098032447359337, + "validation_selection": 0.931, + "best_epoch": 2, + "seconds": 2.16186261177063 + }, + { + "id": "formal_0014", + "tag": "student_1M", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 1.4905903339385986 + }, + { + "id": "formal_0015", + "tag": "student_2M", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 2.0411505699157715 + }, + { + "id": "formal_0016", + "tag": "target_error", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6732827062478426, + "fixed25_actual_fraction": 0.5995857783914395, + "fixed25_rescue": 357, + "fixed25_harm": 1174, + "rank25_metric": 0.7889195719710045, + "validation_selection": 0.9179166666666667, + "best_epoch": 1, + "seconds": 2.198946475982666 + }, + { + "id": "formal_0017", + "tag": "target_four_state", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 5.897979497909546 + }, + { + "id": "formal_0018", + "tag": "harm_cost_2", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6846738004832585, + "fixed25_actual_fraction": 0.6052813255091474, + "fixed25_rescue": 358, + "fixed25_harm": 1109, + "rank25_metric": 0.8096306523990335, + "validation_selection": 0.9307500000000001, + "best_epoch": 1, + "seconds": 5.417069911956787 + }, + { + "id": "formal_0189", + "tag": "baseline_25pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8674490852606145, + "fixed25_actual_fraction": 0.24818778046254747, + "fixed25_rescue": 11, + "fixed25_harm": 209, + "rank25_metric": 0.8674490852606145, + "validation_selection": 0.9786666666666667, + "best_epoch": 12, + "seconds": 6.889333963394165 + }, + { + "id": "formal_0190", + "tag": "baseline_50pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.870037970314118, + "fixed25_actual_fraction": 0.24991370383154987, + "fixed25_rescue": 13, + "fixed25_harm": 196, + "rank25_metric": 0.870037970314118, + "validation_selection": 0.9773333333333334, + "best_epoch": 6, + "seconds": 5.409059524536133 + }, + { + "id": "extra_0007", + "tag": "representation_32", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8690024162927166, + "fixed25_actual_fraction": 0.24853296513634796, + "fixed25_rescue": 11, + "fixed25_harm": 200, + "rank25_metric": 0.8686572316189162, + "validation_selection": 0.9784166666666667, + "best_epoch": 15, + "seconds": 10.294649839401245 + }, + { + "id": "extra_0008", + "tag": "representation_64", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8679668622713151, + "fixed25_actual_fraction": 0.24974111149464964, + "fixed25_rescue": 13, + "fixed25_harm": 208, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9781666666666666, + "best_epoch": 5, + "seconds": 5.710233926773071 + }, + { + "id": "extra_0009", + "tag": "student_8M", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 7.065940618515015 + }, + { + "id": "formal_0019", + "tag": "baseline_100pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 7.6912243366241455 + }, + { + "id": "formal_0020", + "tag": "representation_128", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8679668622713151, + "fixed25_actual_fraction": 0.24974111149464964, + "fixed25_rescue": 13, + "fixed25_harm": 208, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9786666666666667, + "best_epoch": 9, + "seconds": 7.554404973983765 + }, + { + "id": "formal_0021", + "tag": "representation_256", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.870037970314118, + "fixed25_actual_fraction": 0.24991370383154987, + "fixed25_rescue": 13, + "fixed25_harm": 196, + "rank25_metric": 0.870037970314118, + "validation_selection": 0.9788333333333333, + "best_epoch": 16, + "seconds": 9.600196123123169 + }, + { + "id": "formal_0022", + "tag": "student_1M", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 7.301284313201904 + }, + { + "id": "formal_0023", + "tag": "student_2M", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 6.240914821624756 + }, + { + "id": "formal_0024", + "tag": "target_error", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.7664825681739731, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 46, + "fixed25_harm": 829, + "rank25_metric": 0.7664825681739731, + "validation_selection": 0.9097500000000001, + "best_epoch": 16, + "seconds": 11.102000713348389 + }, + { + "id": "formal_0025", + "tag": "target_four_state", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8683120469451157, + "fixed25_actual_fraction": 0.25008629616845013, + "fixed25_rescue": 16, + "fixed25_harm": 209, + "rank25_metric": 0.8683120469451157, + "validation_selection": 0.9762500000000001, + "best_epoch": 2, + "seconds": 3.639625310897827 + }, + { + "id": "formal_0026", + "tag": "harm_cost_2", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.870037970314118, + "fixed25_actual_fraction": 0.24991370383154987, + "fixed25_rescue": 13, + "fixed25_harm": 196, + "rank25_metric": 0.870037970314118, + "validation_selection": 0.9783333333333333, + "best_epoch": 6, + "seconds": 7.290773391723633 + }, + { + "id": "formal_0038", + "tag": "target_error", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.7580255436658613, + "fixed25_actual_fraction": 0.25232999654815325, + "fixed25_rescue": 49, + "fixed25_harm": 881, + "rank25_metric": 0.7597514670348636, + "validation_selection": 0.9071666666666667, + "best_epoch": 4, + "seconds": 9.048983573913574 + }, + { + "id": "formal_0191", + "tag": "baseline_25pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8797031411805316, + "fixed25_actual_fraction": 0.23662409389023129, + "fixed25_rescue": 18, + "fixed25_harm": 145, + "rank25_metric": 0.8776320331377286, + "validation_selection": 0.98025, + "best_epoch": 18, + "seconds": 7.610384702682495 + }, + { + "id": "formal_0192", + "tag": "baseline_50pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.876423886779427, + "fixed25_actual_fraction": 0.24525371073524335, + "fixed25_rescue": 8, + "fixed25_harm": 154, + "rank25_metric": 0.8762512944425268, + "validation_selection": 0.981, + "best_epoch": 25, + "seconds": 10.510464906692505 + }, + { + "id": "extra_0010", + "tag": "representation_32", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8836727649292372, + "fixed25_actual_fraction": 0.24370037970314118, + "fixed25_rescue": 5, + "fixed25_harm": 109, + "rank25_metric": 0.882119433897135, + "validation_selection": 0.9845833333333334, + "best_epoch": 3, + "seconds": 5.182578086853027 + }, + { + "id": "extra_0011", + "tag": "representation_64", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8855712806351398, + "fixed25_actual_fraction": 0.23662409389023129, + "fixed25_rescue": 5, + "fixed25_harm": 98, + "rank25_metric": 0.8843631342768381, + "validation_selection": 0.9846666666666666, + "best_epoch": 3, + "seconds": 5.3754966259002686 + }, + { + "id": "extra_0012", + "tag": "student_8M", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.870728339661719, + "fixed25_actual_fraction": 0.24853296513634796, + "fixed25_rescue": 14, + "fixed25_harm": 193, + "rank25_metric": 0.8705557473248188, + "validation_selection": 0.977, + "best_epoch": 1, + "seconds": 4.12035346031189 + }, + { + "id": "formal_0024", + "tag": "target_error", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8075595443562306, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 56, + "fixed25_harm": 601, + "rank25_metric": 0.8070417673455299, + "validation_selection": 0.9256666666666666, + "best_epoch": 16, + "seconds": 11.102000713348389 + }, + { + "id": "formal_0027", + "tag": "baseline_100pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8684846392820159, + "fixed25_actual_fraction": 0.2507766655160511, + "fixed25_rescue": 17, + "fixed25_harm": 209, + "rank25_metric": 0.8684846392820159, + "validation_selection": 0.97525, + "best_epoch": 1, + "seconds": 3.935011863708496 + }, + { + "id": "formal_0028", + "tag": "drop_image", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8788401794960303, + "fixed25_actual_fraction": 0.25181221953745253, + "fixed25_rescue": 8, + "fixed25_harm": 140, + "rank25_metric": 0.8793579565067311, + "validation_selection": 0.9799166666666667, + "best_epoch": 23, + "seconds": 13.344428062438965 + }, + { + "id": "formal_0029", + "tag": "drop_text", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8733172247152227, + "fixed25_actual_fraction": 0.2430100103555402, + "fixed25_rescue": 17, + "fixed25_harm": 181, + "rank25_metric": 0.8722816706938212, + "validation_selection": 0.9743333333333333, + "best_epoch": 5, + "seconds": 6.613906621932983 + }, + { + "id": "formal_0030", + "tag": "drop_hidden", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8372454263030722, + "fixed25_actual_fraction": 0.26561960648947186, + "fixed25_rescue": 27, + "fixed25_harm": 400, + "rank25_metric": 0.843631342768381, + "validation_selection": 0.9608333333333333, + "best_epoch": 41, + "seconds": 18.277963161468506 + }, + { + "id": "formal_0031", + "tag": "hidden_position_aggregator__layer_01", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8731446323783224, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 22, + "fixed25_harm": 187, + "rank25_metric": 0.8727994477045219, + "validation_selection": 0.9765, + "best_epoch": 1, + "seconds": 5.343691825866699 + }, + { + "id": "formal_0032", + "tag": "hidden_position_global_local_encoder__layer_06", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_06", + "fixed25_metric": 0.8419054193993787, + "fixed25_actual_fraction": 0.2575077666551605, + "fixed25_rescue": 36, + "fixed25_harm": 382, + "rank25_metric": 0.8439765274421815, + "validation_selection": 0.9591666666666667, + "best_epoch": 1, + "seconds": 5.476740121841431 + }, + { + "id": "formal_0033", + "tag": "hidden_multi_layer", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8738350017259233, + "fixed25_actual_fraction": 0.24767000345184673, + "fixed25_rescue": 13, + "fixed25_harm": 174, + "rank25_metric": 0.8733172247152227, + "validation_selection": 0.9786666666666667, + "best_epoch": 2, + "seconds": 6.360007286071777 + }, + { + "id": "formal_0034", + "tag": "representation_128", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8810838798757336, + "fixed25_actual_fraction": 0.25112185018985156, + "fixed25_rescue": 7, + "fixed25_harm": 126, + "rank25_metric": 0.8809112875388333, + "validation_selection": 0.9837499999999999, + "best_epoch": 2, + "seconds": 6.371859073638916 + }, + { + "id": "formal_0035", + "tag": "representation_256", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8783224024853297, + "fixed25_actual_fraction": 0.2474974111149465, + "fixed25_rescue": 16, + "fixed25_harm": 151, + "rank25_metric": 0.8779772178115292, + "validation_selection": 0.9819166666666667, + "best_epoch": 1, + "seconds": 3.1990182399749756 + }, + { + "id": "formal_0036", + "tag": "student_1M", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.865723161891612, + "fixed25_actual_fraction": 0.25560925094925785, + "fixed25_rescue": 16, + "fixed25_harm": 224, + "rank25_metric": 0.8662409389023127, + "validation_selection": 0.97575, + "best_epoch": 1, + "seconds": 5.600154161453247 + }, + { + "id": "formal_0037", + "tag": "student_2M", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8662409389023127, + "fixed25_actual_fraction": 0.2659647911632724, + "fixed25_rescue": 19, + "fixed25_harm": 224, + "rank25_metric": 0.8683120469451157, + "validation_selection": 0.97625, + "best_epoch": 1, + "seconds": 5.593548774719238 + }, + { + "id": "formal_0038", + "tag": "target_error", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8282706247842596, + "fixed25_actual_fraction": 0.24197445633413878, + "fixed25_rescue": 40, + "fixed25_harm": 465, + "rank25_metric": 0.8260269244045564, + "validation_selection": 0.94, + "best_epoch": 7, + "seconds": 9.048983573913574 + }, + { + "id": "formal_0039", + "tag": "target_four_state", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8684846392820159, + "fixed25_actual_fraction": 0.2507766655160511, + "fixed25_rescue": 17, + "fixed25_harm": 209, + "rank25_metric": 0.8684846392820159, + "validation_selection": 0.97525, + "best_epoch": 1, + "seconds": 5.580618619918823 + }, + { + "id": "formal_0040", + "tag": "fusion_attention", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8669313082499137, + "fixed25_actual_fraction": 0.2597514670348636, + "fixed25_rescue": 14, + "fixed25_harm": 215, + "rank25_metric": 0.8684846392820159, + "validation_selection": 0.9769166666666667, + "best_epoch": 1, + "seconds": 7.3831634521484375 + }, + { + "id": "formal_0041", + "tag": "harm_cost_2", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8717638936831205, + "fixed25_actual_fraction": 0.2599240593717639, + "fixed25_rescue": 13, + "fixed25_harm": 186, + "rank25_metric": 0.8729720400414221, + "validation_selection": 0.9796666666666667, + "best_epoch": 1, + "seconds": 3.963697910308838 + }, + { + "id": "formal_0193", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.38158976831239544, + "fixed25_actual_fraction": 0.24836230609987806, + "fixed25_rescue": 4083, + "fixed25_harm": 38, + "rank25_metric": 0.38201514335138814, + "validation_selection": 0.6237472766884531, + "best_epoch": 1, + "seconds": 28.353870153427124 + }, + { + "id": "formal_0194", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3658792502055979, + "fixed25_actual_fraction": 0.22034427019822478, + "fixed25_rescue": 3503, + "fixed25_harm": 12, + "rank25_metric": 0.3758330261180274, + "validation_selection": 0.6238526951999438, + "best_epoch": 2, + "seconds": 56.78019833564758 + }, + { + "id": "extra_0013", + "tag": "representation_32", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3701613589314579, + "fixed25_actual_fraction": 0.2273487791736381, + "fixed25_rescue": 3680, + "fixed25_harm": 38, + "rank25_metric": 0.37838527635198366, + "validation_selection": 0.6219270503900485, + "best_epoch": 2, + "seconds": 128.10148692131042 + }, + { + "id": "extra_0014", + "tag": "representation_64", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3667583586195162, + "fixed25_actual_fraction": 0.22235771204945695, + "fixed25_rescue": 3554, + "fixed25_harm": 32, + "rank25_metric": 0.37668377619601284, + "validation_selection": 0.6208728652751423, + "best_epoch": 2, + "seconds": 127.02403688430786 + }, + { + "id": "extra_0015", + "tag": "student_8M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37294047585287693, + "fixed25_actual_fraction": 0.2379264384765902, + "fixed25_rescue": 3803, + "fixed25_harm": 63, + "rank25_metric": 0.378130051328588, + "validation_selection": 0.6220605805046033, + "best_epoch": 1, + "seconds": 114.63445687294006 + }, + { + "id": "formal_0042", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3693106088534725, + "fixed25_actual_fraction": 0.22853982928281769, + "fixed25_rescue": 3642, + "fixed25_harm": 30, + "rank25_metric": 0.3758330261180274, + "validation_selection": 0.6221730269168599, + "best_epoch": 9, + "seconds": 476.6986792087555 + }, + { + "id": "formal_0043", + "tag": "drop_image", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.364376258401157, + "fixed25_actual_fraction": 0.21804724498766412, + "fixed25_rescue": 3447, + "fixed25_harm": 9, + "rank25_metric": 0.3735643592433996, + "validation_selection": 0.6235926628716002, + "best_epoch": 1, + "seconds": 195.11448216438293 + }, + { + "id": "formal_0044", + "tag": "drop_text", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37495391770410913, + "fixed25_actual_fraction": 0.2399115219918895, + "fixed25_rescue": 3854, + "fixed25_harm": 43, + "rank25_metric": 0.3789240847347078, + "validation_selection": 0.6211891208096142, + "best_epoch": 1, + "seconds": 261.2588014602661 + }, + { + "id": "formal_0045", + "tag": "drop_hidden", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.36732552533817314, + "fixed25_actual_fraction": 0.22170547032300145, + "fixed25_rescue": 3589, + "fixed25_harm": 47, + "rank25_metric": 0.37725094291466976, + "validation_selection": 0.6216881017640031, + "best_epoch": 7, + "seconds": 358.06045365333557 + }, + { + "id": "formal_0046", + "tag": "drop_confidence", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.36247624989365623, + "fixed25_actual_fraction": 0.2210248702606131, + "fixed25_rescue": 3395, + "fixed25_harm": 24, + "rank25_metric": 0.372741967501347, + "validation_selection": 0.6210907301988895, + "best_epoch": 1, + "seconds": 263.48469948768616 + }, + { + "id": "formal_0047", + "tag": "hidden_position_multimodal_encoder__layer_01", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_01", + "fixed25_metric": 0.3823270850466495, + "fixed25_actual_fraction": 0.24632050591271304, + "fixed25_rescue": 4165, + "fixed25_harm": 94, + "rank25_metric": 0.38365992683549327, + "validation_selection": 0.6220535526038372, + "best_epoch": 7, + "seconds": 426.6365387439728 + }, + { + "id": "formal_0048", + "tag": "hidden_position_text_encoder__layer_06", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_06", + "fixed25_metric": 0.3687434421348155, + "fixed25_actual_fraction": 0.22797266256416074, + "fixed25_rescue": 3629, + "fixed25_harm": 37, + "rank25_metric": 0.3761449678132887, + "validation_selection": 0.6237964719938154, + "best_epoch": 3, + "seconds": 326.0554850101471 + }, + { + "id": "formal_0049", + "tag": "hidden_multi_layer", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_01", + "fixed25_metric": 0.3881121855769503, + "fixed25_actual_fraction": 0.2539205399427162, + "fixed25_rescue": 4332, + "fixed25_harm": 57, + "rank25_metric": 0.38638232708504666, + "validation_selection": 0.6241478670321174, + "best_epoch": 3, + "seconds": 315.55904817581177 + }, + { + "id": "formal_0050", + "tag": "representation_128", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3645180500808213, + "fixed25_actual_fraction": 0.21955023679210503, + "fixed25_rescue": 3463, + "fixed25_harm": 20, + "rank25_metric": 0.374046450954258, + "validation_selection": 0.6219903014969428, + "best_epoch": 4, + "seconds": 351.69083309173584 + }, + { + "id": "formal_0051", + "tag": "representation_256", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3698777755721294, + "fixed25_actual_fraction": 0.2289652043218104, + "fixed25_rescue": 3671, + "fixed25_harm": 39, + "rank25_metric": 0.3778748263051924, + "validation_selection": 0.621856771382388, + "best_epoch": 3, + "seconds": 296.8873279094696 + }, + { + "id": "formal_0052", + "tag": "student_1M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3665314919320534, + "fixed25_actual_fraction": 0.22278308708844966, + "fixed25_rescue": 3530, + "fixed25_harm": 16, + "rank25_metric": 0.3759464594617588, + "validation_selection": 0.6230093471080188, + "best_epoch": 1, + "seconds": 240.49552273750305 + }, + { + "id": "formal_0053", + "tag": "student_2M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37197629243116015, + "fixed25_actual_fraction": 0.23001446275132575, + "fixed25_rescue": 3743, + "fixed25_harm": 37, + "rank25_metric": 0.38022856818761874, + "validation_selection": 0.6234872443601096, + "best_epoch": 3, + "seconds": 289.043110370636 + }, + { + "id": "formal_0054", + "tag": "target_error", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37489720103224344, + "fixed25_actual_fraction": 0.24345631398349546, + "fixed25_rescue": 3872, + "fixed25_harm": 63, + "rank25_metric": 0.37705243456313986, + "validation_selection": 0.6214210415348935, + "best_epoch": 1, + "seconds": 351.4256999492645 + }, + { + "id": "formal_0055", + "tag": "target_four_state", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37872557638317783, + "fixed25_actual_fraction": 0.23265178799308056, + "fixed25_rescue": 4019, + "fixed25_harm": 75, + "rank25_metric": 0.38530471031959845, + "validation_selection": 0.6250544662309369, + "best_epoch": 4, + "seconds": 317.04318404197693 + }, + { + "id": "formal_0056", + "tag": "fusion_concat", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.36426282505742563, + "fixed25_actual_fraction": 0.21901142840938093, + "fixed25_rescue": 3456, + "fixed25_harm": 22, + "rank25_metric": 0.37401809261832514, + "validation_selection": 0.6226157846651205, + "best_epoch": 4, + "seconds": 275.1700701713562 + }, + { + "id": "formal_0069", + "tag": "target_error", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.37370615092306386, + "fixed25_actual_fraction": 0.23596971329722372, + "fixed25_rescue": 3814, + "fixed25_harm": 47, + "rank25_metric": 0.379065876414372, + "validation_selection": 0.6227985100850376, + "best_epoch": 1, + "seconds": 256.6061637401581 + }, + { + "id": "formal_0195", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4562572668235828, + "fixed25_actual_fraction": 0.24856081445140799, + "fixed25_rescue": 6732, + "fixed25_harm": 54, + "rank25_metric": 0.45705130022970253, + "validation_selection": 0.6656265373532926, + "best_epoch": 1, + "seconds": 18.048375368118286 + }, + { + "id": "formal_0196", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.434648214842753, + "fixed25_actual_fraction": 0.22008904517482913, + "fixed25_rescue": 5930, + "fixed25_harm": 14, + "rank25_metric": 0.4536766582536937, + "validation_selection": 0.6660130718954248, + "best_epoch": 2, + "seconds": 33.53883171081543 + }, + { + "id": "extra_0016", + "tag": "representation_32", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4426169072398832, + "fixed25_actual_fraction": 0.2276607208688994, + "fixed25_rescue": 6227, + "fixed25_harm": 30, + "rank25_metric": 0.4566826418625755, + "validation_selection": 0.6649307751774545, + "best_epoch": 2, + "seconds": 85.932044506073 + }, + { + "id": "extra_0017", + "tag": "representation_64", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.44343929898193574, + "fixed25_actual_fraction": 0.22851147094688484, + "fixed25_rescue": 6257, + "fixed25_harm": 31, + "rank25_metric": 0.45719309190936674, + "validation_selection": 0.6650010541851149, + "best_epoch": 2, + "seconds": 86.59067559242249 + }, + { + "id": "extra_0018", + "tag": "student_8M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.42920341434364634, + "fixed25_actual_fraction": 0.20959646087967557, + "fixed25_rescue": 5739, + "fixed25_harm": 15, + "rank25_metric": 0.4557468167767915, + "validation_selection": 0.6653032539180547, + "best_epoch": 2, + "seconds": 87.00826907157898 + }, + { + "id": "formal_0054", + "tag": "target_error", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4345631398349545, + "fixed25_actual_fraction": 0.2188696367297167, + "fixed25_rescue": 5928, + "fixed25_harm": 15, + "rank25_metric": 0.4539318832770893, + "validation_selection": 0.6653594771241831, + "best_epoch": 4, + "seconds": 351.4256999492645 + }, + { + "id": "formal_0057", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43033774778096023, + "fixed25_actual_fraction": 0.2126591611604231, + "fixed25_rescue": 5784, + "fixed25_harm": 20, + "rank25_metric": 0.4544139749879477, + "validation_selection": 0.6650924168950734, + "best_epoch": 5, + "seconds": 281.1101198196411 + }, + { + "id": "formal_0058", + "tag": "drop_image", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43430791481155884, + "fixed25_actual_fraction": 0.21804724498766412, + "fixed25_rescue": 5914, + "fixed25_harm": 10, + "rank25_metric": 0.4541871083004849, + "validation_selection": 0.6667510014758591, + "best_epoch": 1, + "seconds": 166.40395164489746 + }, + { + "id": "formal_0059", + "tag": "drop_text", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.441709440490032, + "fixed25_actual_fraction": 0.22870997929841477, + "fixed25_rescue": 6177, + "fixed25_harm": 12, + "rank25_metric": 0.4553214417377988, + "validation_selection": 0.6643755710169372, + "best_epoch": 2, + "seconds": 181.88760209083557 + }, + { + "id": "formal_0060", + "tag": "drop_hidden", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.44636020758301903, + "fixed25_actual_fraction": 0.23795479681252305, + "fixed25_rescue": 6388, + "fixed25_harm": 59, + "rank25_metric": 0.4537333749255594, + "validation_selection": 0.662126642771804, + "best_epoch": 1, + "seconds": 106.5097143650055 + }, + { + "id": "formal_0061", + "tag": "drop_confidence", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4283526642656609, + "fixed25_actual_fraction": 0.21180841108243767, + "fixed25_rescue": 5705, + "fixed25_harm": 11, + "rank25_metric": 0.45384680826929075, + "validation_selection": 0.6646215475437487, + "best_epoch": 2, + "seconds": 186.71321535110474 + }, + { + "id": "formal_0062", + "tag": "hidden_position_multimodal_encoder__layer_01", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_01", + "fixed25_metric": 0.45812891699515074, + "fixed25_actual_fraction": 0.2401383886793523, + "fixed25_rescue": 6809, + "fixed25_harm": 65, + "rank25_metric": 0.4648782009471684, + "validation_selection": 0.6651205284981376, + "best_epoch": 7, + "seconds": 283.8257784843445 + }, + { + "id": "formal_0063", + "tag": "hidden_position_text_encoder__layer_06", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_06", + "fixed25_metric": 0.44017809034965827, + "fixed25_actual_fraction": 0.22984431273572867, + "fixed25_rescue": 6150, + "fixed25_harm": 39, + "rank25_metric": 0.452684116496044, + "validation_selection": 0.6649096914751563, + "best_epoch": 7, + "seconds": 320.17189478874207 + }, + { + "id": "formal_0064", + "tag": "hidden_multi_layer", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_01", + "fixed25_metric": 0.4681394095794459, + "fixed25_actual_fraction": 0.25074440631823725, + "fixed25_rescue": 7146, + "fixed25_harm": 49, + "rank25_metric": 0.4675155261889232, + "validation_selection": 0.6663012158268324, + "best_epoch": 3, + "seconds": 237.93119263648987 + }, + { + "id": "formal_0065", + "tag": "representation_128", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43632135666279104, + "fixed25_actual_fraction": 0.22091143691688173, + "fixed25_rescue": 5996, + "fixed25_harm": 21, + "rank25_metric": 0.4532796415506338, + "validation_selection": 0.6649026635743903, + "best_epoch": 4, + "seconds": 234.7888114452362 + }, + { + "id": "formal_0066", + "tag": "representation_256", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43663329835805237, + "fixed25_actual_fraction": 0.21872784505005247, + "fixed25_rescue": 6003, + "fixed25_harm": 17, + "rank25_metric": 0.4554348750815302, + "validation_selection": 0.6652259470096282, + "best_epoch": 4, + "seconds": 231.40922713279724 + }, + { + "id": "formal_0067", + "tag": "student_1M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43731389842044066, + "fixed25_actual_fraction": 0.22224427870572555, + "fixed25_rescue": 6021, + "fixed25_harm": 11, + "rank25_metric": 0.4541020332926864, + "validation_selection": 0.6657179000632512, + "best_epoch": 4, + "seconds": 229.5956084728241 + }, + { + "id": "formal_0068", + "tag": "student_2M", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4318123812494683, + "fixed25_actual_fraction": 0.21399200294926693, + "fixed25_rescue": 5830, + "fixed25_harm": 14, + "rank25_metric": 0.4548109916910076, + "validation_selection": 0.6654157003303113, + "best_epoch": 2, + "seconds": 199.25073266029358 + }, + { + "id": "formal_0069", + "tag": "target_error", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4454810991691008, + "fixed25_actual_fraction": 0.23177267957916228, + "fixed25_rescue": 6322, + "fixed25_harm": 24, + "rank25_metric": 0.4573065252530982, + "validation_selection": 0.6649659146812846, + "best_epoch": 5, + "seconds": 256.6061637401581 + }, + { + "id": "formal_0070", + "tag": "target_four_state", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.438561665201486, + "fixed25_actual_fraction": 0.21575021977710349, + "fixed25_rescue": 6099, + "fixed25_harm": 45, + "rank25_metric": 0.4611632589399654, + "validation_selection": 0.6686977299880525, + "best_epoch": 7, + "seconds": 314.07090616226196 + }, + { + "id": "formal_0071", + "tag": "fusion_attention", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4318123812494683, + "fixed25_actual_fraction": 0.2146158863397896, + "fixed25_rescue": 5831, + "fixed25_harm": 15, + "rank25_metric": 0.4541587499645521, + "validation_selection": 0.665190807505798, + "best_epoch": 4, + "seconds": 340.09505891799927 + }, + { + "id": "formal_0197", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6523268014632901, + "fixed25_actual_fraction": 0.3479284235601055, + "fixed25_rescue": 1394, + "fixed25_harm": 2000, + "rank25_metric": 0.6652582026486685, + "validation_selection": 0.753925082577834, + "best_epoch": 13, + "seconds": 28.585034132003784 + }, + { + "id": "formal_0198", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6567223435328815, + "fixed25_actual_fraction": 0.3123670703003148, + "fixed25_rescue": 1366, + "fixed25_harm": 1817, + "rank25_metric": 0.662762669086578, + "validation_selection": 0.7576779815868999, + "best_epoch": 13, + "seconds": 44.9037389755249 + }, + { + "id": "extra_0019", + "tag": "representation_32", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6564104018376201, + "fixed25_actual_fraction": 0.32365368800158806, + "fixed25_rescue": 1502, + "fixed25_harm": 1964, + "rank25_metric": 0.6659671610469897, + "validation_selection": 0.7572914470447678, + "best_epoch": 3, + "seconds": 78.71963858604431 + }, + { + "id": "extra_0020", + "tag": "representation_64", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6560417434704932, + "fixed25_actual_fraction": 0.318464112525877, + "fixed25_rescue": 1439, + "fixed25_harm": 1914, + "rank25_metric": 0.6641522275472875, + "validation_selection": 0.7562864572352239, + "best_epoch": 3, + "seconds": 77.79833817481995 + }, + { + "id": "extra_0021", + "tag": "student_8M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 79.3776524066925 + }, + { + "id": "formal_0072", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 163.15981340408325 + }, + { + "id": "formal_0073", + "tag": "hidden_position_joint_encoder__layer_02", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "joint_encoder__layer_02", + "fixed25_metric": 0.6435924339959731, + "fixed25_actual_fraction": 0.28193857584436943, + "fixed25_rescue": 877, + "fixed25_harm": 1791, + "rank25_metric": 0.6475626010265718, + "validation_selection": 0.7462927823459133, + "best_epoch": 7, + "seconds": 218.39065027236938 + }, + { + "id": "formal_0074", + "tag": "hidden_position_lateral_features__layer_02", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_02", + "fixed25_metric": 0.6621671440319882, + "fixed25_actual_fraction": 0.2838669426878031, + "fixed25_rescue": 1361, + "fixed25_harm": 1620, + "rank25_metric": 0.665740294359527, + "validation_selection": 0.7524351676154333, + "best_epoch": 6, + "seconds": 191.70505738258362 + }, + { + "id": "formal_0075", + "tag": "hidden_multi_layer", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "joint_encoder__layer_02", + "fixed25_metric": 0.6581686186654567, + "fixed25_actual_fraction": 0.30692226980120807, + "fixed25_rescue": 1428, + "fixed25_harm": 1828, + "rank25_metric": 0.6646059609222131, + "validation_selection": 0.7494834492936959, + "best_epoch": 3, + "seconds": 145.08960509300232 + }, + { + "id": "formal_0076", + "tag": "representation_128", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6579984686498597, + "fixed25_actual_fraction": 0.3205626293849077, + "fixed25_rescue": 1415, + "fixed25_harm": 1821, + "rank25_metric": 0.6648895442815416, + "validation_selection": 0.7587883899079345, + "best_epoch": 3, + "seconds": 146.03866457939148 + }, + { + "id": "formal_0077", + "tag": "representation_256", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6572327935796728, + "fixed25_actual_fraction": 0.32413577971244645, + "fixed25_rescue": 1438, + "fixed25_harm": 1871, + "rank25_metric": 0.6655701443439299, + "validation_selection": 0.7598074355190104, + "best_epoch": 3, + "seconds": 144.2073380947113 + }, + { + "id": "formal_0078", + "tag": "student_1M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 143.30026388168335 + }, + { + "id": "formal_0079", + "tag": "student_2M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 163.39957666397095 + }, + { + "id": "formal_0080", + "tag": "target_error", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6521566514476931, + "fixed25_actual_fraction": 0.30961631171482856, + "fixed25_rescue": 1379, + "fixed25_harm": 1991, + "rank25_metric": 0.6618835606726597, + "validation_selection": 0.7524070560123691, + "best_epoch": 7, + "seconds": 200.08106350898743 + }, + { + "id": "formal_0081", + "tag": "target_four_state", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6556730851033661, + "fixed25_actual_fraction": 0.31480588719053965, + "fixed25_rescue": 1509, + "fixed25_harm": 1997, + "rank25_metric": 0.6647477526018774, + "validation_selection": 0.7561529271206691, + "best_epoch": 7, + "seconds": 189.60577154159546 + }, + { + "id": "formal_0082", + "tag": "harm_cost_2", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.659416385446502, + "fixed25_actual_fraction": 0.30834018659785045, + "fixed25_rescue": 1264, + "fixed25_harm": 1620, + "rank25_metric": 0.6656835776876613, + "validation_selection": 0.7544029798299249, + "best_epoch": 3, + "seconds": 147.022536277771 + }, + { + "id": "formal_0088", + "tag": "target_error", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.6762612369906134, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 493, + "fixed25_harm": 255, + "rank25_metric": 0.6638686441879591, + "validation_selection": 0.7442125237191651, + "best_epoch": 2, + "seconds": 150.90026092529297 + }, + { + "id": "formal_0199", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 14.837297677993774 + }, + { + "id": "formal_0200", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7416555596517597, + "fixed25_actual_fraction": 0.4823752942177353, + "fixed25_rescue": 3418, + "fixed25_harm": 874, + "rank25_metric": 0.7207838244051838, + "validation_selection": 0.8144985592803429, + "best_epoch": 1, + "seconds": 21.554462432861328 + }, + { + "id": "extra_0022", + "tag": "representation_32", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 70.20556235313416 + }, + { + "id": "extra_0023", + "tag": "representation_64", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 71.17191433906555 + }, + { + "id": "extra_0024", + "tag": "student_8M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 71.64453220367432 + }, + { + "id": "formal_0080", + "tag": "target_error", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.7439525848623203, + "fixed25_actual_fraction": 0.3155715622607265, + "fixed25_rescue": 3506, + "fixed25_harm": 881, + "rank25_metric": 0.7360689674729887, + "validation_selection": 0.7762878628153771, + "best_epoch": 5, + "seconds": 200.08106350898743 + }, + { + "id": "formal_0083", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 121.36263680458069 + }, + { + "id": "formal_0084", + "tag": "representation_128", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 118.60358738899231 + }, + { + "id": "formal_0085", + "tag": "representation_256", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 122.39959692955017 + }, + { + "id": "formal_0086", + "tag": "student_1M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 120.20404839515686 + }, + { + "id": "formal_0087", + "tag": "student_2M", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 117.77066564559937 + }, + { + "id": "formal_0088", + "tag": "target_error", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7162748489918611, + "validation_selection": 0.7588446131140628, + "best_epoch": 1, + "seconds": 150.90026092529297 + }, + { + "id": "formal_0089", + "tag": "target_four_state", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 129.93259263038635 + }, + { + "id": "formal_0090", + "tag": "harm_cost_2", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 129.64483284950256 + }, + { + "id": "formal_0201", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.863815742268781, + "fixed25_actual_fraction": 0.253928392049939, + "fixed25_rescue": 1100, + "fixed25_harm": 211, + "rank25_metric": 0.8630264762861448, + "validation_selection": 0.8331853337152678, + "best_epoch": 2, + "seconds": 11.065314054489136 + }, + { + "id": "formal_0202", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.868336083805697, + "fixed25_actual_fraction": 0.24388318863456984, + "fixed25_rescue": 1152, + "fixed25_harm": 200, + "rank25_metric": 0.8689818468823993, + "validation_selection": 0.8330850759094816, + "best_epoch": 3, + "seconds": 16.297266960144043 + }, + { + "id": "extra_0025", + "tag": "representation_32", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8694841070531678, + "fixed25_actual_fraction": 0.2631125780297051, + "fixed25_rescue": 1175, + "fixed25_harm": 207, + "rank25_metric": 0.8678338236349286, + "validation_selection": 0.8335147522199943, + "best_epoch": 6, + "seconds": 62.19458746910095 + }, + { + "id": "extra_0026", + "tag": "representation_64", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8696276099591017, + "fixed25_actual_fraction": 0.26296907512377127, + "fixed25_rescue": 1176, + "fixed25_harm": 206, + "rank25_metric": 0.8669728061993255, + "validation_selection": 0.833887138355772, + "best_epoch": 6, + "seconds": 62.55568766593933 + }, + { + "id": "extra_0027", + "tag": "student_8M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 61.909870862960815 + }, + { + "id": "formal_0091", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 82.51407408714294 + }, + { + "id": "formal_0092", + "tag": "drop_hidden", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8648920140632848, + "fixed25_actual_fraction": 0.23871708402095143, + "fixed25_rescue": 1173, + "fixed25_harm": 269, + "rank25_metric": 0.8675468178230609, + "validation_selection": 0.8284445717559438, + "best_epoch": 2, + "seconds": 54.881943464279175 + }, + { + "id": "formal_0093", + "tag": "drop_confidence", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8552055679127503, + "fixed25_actual_fraction": 0.2546459065796082, + "fixed25_rescue": 946, + "fixed25_harm": 177, + "rank25_metric": 0.8551338164597833, + "validation_selection": 0.827427671154397, + "best_epoch": 6, + "seconds": 73.42077493667603 + }, + { + "id": "formal_0094", + "tag": "hidden_position_multimodal_encoder__layer_02", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_02", + "fixed25_metric": 0.8604434239793356, + "fixed25_actual_fraction": 0.243739685728636, + "fixed25_rescue": 1079, + "fixed25_harm": 237, + "rank25_metric": 0.8617349501327402, + "validation_selection": 0.8337439129189345, + "best_epoch": 21, + "seconds": 164.92656469345093 + }, + { + "id": "formal_0095", + "tag": "hidden_position_multimodal_encoder__layer_08", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_08", + "fixed25_metric": 0.8613761928679056, + "fixed25_actual_fraction": 0.2468967496591806, + "fixed25_rescue": 1098, + "fixed25_harm": 243, + "rank25_metric": 0.8616631986797733, + "validation_selection": 0.8295474076195932, + "best_epoch": 1, + "seconds": 62.42424654960632 + }, + { + "id": "formal_0096", + "tag": "hidden_multi_layer", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_02", + "fixed25_metric": 0.8651072684221855, + "fixed25_actual_fraction": 0.2501255650426921, + "fixed25_rescue": 1132, + "fixed25_harm": 225, + "rank25_metric": 0.8650355169692187, + "validation_selection": 0.8327413348610714, + "best_epoch": 1, + "seconds": 56.55902671813965 + }, + { + "id": "formal_0097", + "tag": "representation_128", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8690535983353663, + "fixed25_actual_fraction": 0.2578747219631198, + "fixed25_rescue": 1175, + "fixed25_harm": 213, + "rank25_metric": 0.8676903207289948, + "validation_selection": 0.8337582354626182, + "best_epoch": 6, + "seconds": 81.54244780540466 + }, + { + "id": "formal_0098", + "tag": "representation_256", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8663270431226232, + "fixed25_actual_fraction": 0.24933629906005597, + "fixed25_rescue": 1131, + "fixed25_harm": 207, + "rank25_metric": 0.8664705460285571, + "validation_selection": 0.8330134631910627, + "best_epoch": 3, + "seconds": 64.1797993183136 + }, + { + "id": "formal_0099", + "tag": "student_1M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 85.27223777770996 + }, + { + "id": "formal_0100", + "tag": "student_2M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 90.53286790847778 + }, + { + "id": "formal_0101", + "tag": "target_error", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8702016215828371, + "fixed25_actual_fraction": 0.2504125708545598, + "fixed25_rescue": 1230, + "fixed25_harm": 252, + "rank25_metric": 0.8702016215828371, + "validation_selection": 0.8348324262389, + "best_epoch": 6, + "seconds": 89.40919137001038 + }, + { + "id": "formal_0102", + "tag": "target_four_state", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8702016215828371, + "fixed25_actual_fraction": 0.2523498600846667, + "fixed25_rescue": 1206, + "fixed25_harm": 228, + "rank25_metric": 0.8700581186769032, + "validation_selection": 0.8349470065883701, + "best_epoch": 11, + "seconds": 110.4758677482605 + }, + { + "id": "formal_0103", + "tag": "harm_cost_2", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8582908803903279, + "fixed25_actual_fraction": 0.2504843223075267, + "fixed25_rescue": 985, + "fixed25_harm": 173, + "rank25_metric": 0.8582908803903279, + "validation_selection": 0.8309510169006016, + "best_epoch": 13, + "seconds": 114.95233988761902 + }, + { + "id": "formal_0111", + "tag": "target_error", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8696276099591017, + "fixed25_actual_fraction": 0.24926454760708905, + "fixed25_rescue": 1235, + "fixed25_harm": 265, + "rank25_metric": 0.8697711128650355, + "validation_selection": 0.8327986250358064, + "best_epoch": 7, + "seconds": 75.30136489868164 + }, + { + "id": "formal_0203", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8717801535481093, + "fixed25_actual_fraction": 0.25299562316136903, + "fixed25_rescue": 1180, + "fixed25_harm": 180, + "rank25_metric": 0.8714213962832748, + "validation_selection": 0.8382268690919508, + "best_epoch": 3, + "seconds": 12.40796971321106 + }, + { + "id": "formal_0204", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.872067159359977, + "fixed25_actual_fraction": 0.25407189495587285, + "fixed25_rescue": 1178, + "fixed25_harm": 174, + "rank25_metric": 0.8703451244887709, + "validation_selection": 0.8385276425093097, + "best_epoch": 11, + "seconds": 27.56421136856079 + }, + { + "id": "extra_0028", + "tag": "representation_32", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8714213962832748, + "fixed25_actual_fraction": 0.2545024036736744, + "fixed25_rescue": 1163, + "fixed25_harm": 168, + "rank25_metric": 0.8705603788476717, + "validation_selection": 0.838269836723002, + "best_epoch": 1, + "seconds": 50.37446331977844 + }, + { + "id": "extra_0029", + "tag": "representation_64", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.871206141924374, + "fixed25_actual_fraction": 0.2544306522207075, + "fixed25_rescue": 1158, + "fixed25_harm": 166, + "rank25_metric": 0.8703451244887709, + "validation_selection": 0.8382268690919508, + "best_epoch": 7, + "seconds": 70.82060599327087 + }, + { + "id": "extra_0030", + "tag": "student_8M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 70.37148523330688 + }, + { + "id": "formal_0101", + "tag": "target_error", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.2504125708545598, + "fixed25_rescue": 1205, + "fixed25_harm": 219, + "rank25_metric": 0.8707038817536055, + "validation_selection": 0.8368375823546261, + "best_epoch": 6, + "seconds": 89.40919137001038 + }, + { + "id": "formal_0104", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 91.79227209091187 + }, + { + "id": "formal_0105", + "tag": "drop_confidence", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8000287005811868, + "fixed25_actual_fraction": 0.0, + "fixed25_rescue": 0, + "fixed25_harm": 0, + "rank25_metric": 0.8355456697998135, + "validation_selection": 0.8181323403036379, + "best_epoch": 1, + "seconds": 47.980828046798706 + }, + { + "id": "formal_0106", + "tag": "drop_output", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8656812800459209, + "fixed25_actual_fraction": 0.23871708402095143, + "fixed25_rescue": 1186, + "fixed25_harm": 271, + "rank25_metric": 0.8671880605582263, + "validation_selection": 0.8313806932111143, + "best_epoch": 1, + "seconds": 45.731327533721924 + }, + { + "id": "formal_0107", + "tag": "representation_128", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.2530673746143359, + "fixed25_rescue": 1155, + "fixed25_harm": 167, + "rank25_metric": 0.8704168759417378, + "validation_selection": 0.8383128043540533, + "best_epoch": 8, + "seconds": 93.71329402923584 + }, + { + "id": "formal_0108", + "tag": "representation_256", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8710626390184402, + "fixed25_actual_fraction": 0.2497668077778575, + "fixed25_rescue": 1179, + "fixed25_harm": 189, + "rank25_metric": 0.8711343904714071, + "validation_selection": 0.838384417072472, + "best_epoch": 5, + "seconds": 75.97537636756897 + }, + { + "id": "formal_0109", + "tag": "student_1M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 83.97415852546692 + }, + { + "id": "formal_0110", + "tag": "student_2M", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 77.26504230499268 + }, + { + "id": "formal_0111", + "tag": "target_error", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8708473846595394, + "fixed25_actual_fraction": 0.24926454760708905, + "fixed25_rescue": 1183, + "fixed25_harm": 196, + "rank25_metric": 0.8709908875654732, + "validation_selection": 0.8376109997135491, + "best_epoch": 7, + "seconds": 75.30136489868164 + }, + { + "id": "formal_0112", + "tag": "target_four_state", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8713496448303079, + "fixed25_actual_fraction": 0.2545024036736744, + "fixed25_rescue": 1162, + "fixed25_harm": 168, + "rank25_metric": 0.8704168759417378, + "validation_selection": 0.8381839014608995, + "best_epoch": 4, + "seconds": 66.18311429023743 + }, + { + "id": "formal_0113", + "tag": "harm_cost_2", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8713496448303079, + "fixed25_actual_fraction": 0.2547176580325752, + "fixed25_rescue": 1160, + "fixed25_harm": 166, + "rank25_metric": 0.8700581186769032, + "validation_selection": 0.8382411916356345, + "best_epoch": 4, + "seconds": 65.56374073028564 + }, + { + "id": "formal_0209", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8323886058692689, + "fixed25_actual_fraction": 0.24072612470402527, + "fixed25_rescue": 1189, + "fixed25_harm": 239, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7950730449727872, + "best_epoch": 4, + "seconds": 18.022375106811523 + }, + { + "id": "formal_0210", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8337518834756403, + "fixed25_actual_fraction": 0.24704025256511444, + "fixed25_rescue": 1213, + "fixed25_harm": 244, + "rank25_metric": 0.8344693980053096, + "validation_selection": 0.7959610426811802, + "best_epoch": 8, + "seconds": 38.606887102127075 + }, + { + "id": "extra_0037", + "tag": "representation_32", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 32, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8343258950993757, + "fixed25_actual_fraction": 0.24761426418884983, + "fixed25_rescue": 1213, + "fixed25_harm": 236, + "rank25_metric": 0.8349716581760781, + "validation_selection": 0.7954311085648811, + "best_epoch": 10, + "seconds": 90.23428177833557 + }, + { + "id": "extra_0038", + "tag": "representation_64", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 64, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8354021668938796, + "fixed25_actual_fraction": 0.24826002726555213, + "fixed25_rescue": 1229, + "fixed25_harm": 237, + "rank25_metric": 0.8360479299705819, + "validation_selection": 0.7952162704096247, + "best_epoch": 5, + "seconds": 69.96001935005188 + }, + { + "id": "extra_0039", + "tag": "student_4M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8341823921934419, + "fixed25_actual_fraction": 0.24632273803544522, + "fixed25_rescue": 1212, + "fixed25_harm": 237, + "rank25_metric": 0.835043409629045, + "validation_selection": 0.7951876253222573, + "best_epoch": 3, + "seconds": 61.948747873306274 + }, + { + "id": "formal_0139", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.24208940231039677, + "fixed25_rescue": 1196, + "fixed25_harm": 241, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952162704096247, + "best_epoch": 3, + "seconds": 136.20692539215088 + }, + { + "id": "formal_0140", + "tag": "drop_image", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.834110640740475, + "fixed25_actual_fraction": 0.2450312118820406, + "fixed25_rescue": 1213, + "fixed25_harm": 239, + "rank25_metric": 0.8351151610820119, + "validation_selection": 0.7956459467201376, + "best_epoch": 3, + "seconds": 78.08365845680237 + }, + { + "id": "formal_0141", + "tag": "drop_text", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.832245102963335, + "fixed25_actual_fraction": 0.24201765085742985, + "fixed25_rescue": 1189, + "fixed25_harm": 241, + "rank25_metric": 0.8338236349286073, + "validation_selection": 0.7942280148954455, + "best_epoch": 3, + "seconds": 116.7338719367981 + }, + { + "id": "formal_0142", + "tag": "drop_hidden", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8328908660400373, + "fixed25_actual_fraction": 0.24194589940446293, + "fixed25_rescue": 1198, + "fixed25_harm": 241, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7951589802348897, + "best_epoch": 2, + "seconds": 108.56272602081299 + }, + { + "id": "formal_0143", + "tag": "drop_confidence", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8183970725407189, + "fixed25_actual_fraction": 0.2401521130802899, + "fixed25_rescue": 968, + "fixed25_harm": 213, + "rank25_metric": 0.8196885986941236, + "validation_selection": 0.7879260956745918, + "best_epoch": 9, + "seconds": 167.05382919311523 + }, + { + "id": "formal_0144", + "tag": "hidden_position_ordered_image_text__layer_02", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_02", + "fixed25_metric": 0.833249623304872, + "fixed25_actual_fraction": 0.241874147951496, + "fixed25_rescue": 1199, + "fixed25_harm": 237, + "rank25_metric": 0.8349716581760781, + "validation_selection": 0.7944428530507018, + "best_epoch": 3, + "seconds": 115.09392309188843 + }, + { + "id": "formal_0145", + "tag": "hidden_position_ordered_image_text__layer_08", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_08", + "fixed25_metric": 0.834038889287508, + "fixed25_actual_fraction": 0.244672454617206, + "fixed25_rescue": 1215, + "fixed25_harm": 242, + "rank25_metric": 0.8351151610820119, + "validation_selection": 0.7938699513033515, + "best_epoch": 2, + "seconds": 116.08835816383362 + }, + { + "id": "formal_0146", + "tag": "hidden_multi_layer", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_02", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.24208940231039677, + "fixed25_rescue": 1193, + "fixed25_harm": 238, + "rank25_metric": 0.834110640740475, + "validation_selection": 0.7944714981380693, + "best_epoch": 3, + "seconds": 122.4584412574768 + }, + { + "id": "formal_0147", + "tag": "representation_128", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 128, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8329626174930043, + "fixed25_actual_fraction": 0.2450312118820406, + "fixed25_rescue": 1194, + "fixed25_harm": 236, + "rank25_metric": 0.8342541436464088, + "validation_selection": 0.7950157547980522, + "best_epoch": 8, + "seconds": 162.51085686683655 + }, + { + "id": "formal_0148", + "tag": "representation_256", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 256, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8347564038171773, + "fixed25_actual_fraction": 0.25163234555499747, + "fixed25_rescue": 1224, + "fixed25_harm": 241, + "rank25_metric": 0.8344693980053096, + "validation_selection": 0.7955456889143512, + "best_epoch": 5, + "seconds": 153.2538034915924 + }, + { + "id": "formal_0149", + "tag": "student_1M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8339671378345411, + "fixed25_actual_fraction": 0.24287866829303292, + "fixed25_rescue": 1208, + "fixed25_harm": 236, + "rank25_metric": 0.8352586639879458, + "validation_selection": 0.7945287883128044, + "best_epoch": 3, + "seconds": 137.92107677459717 + }, + { + "id": "formal_0150", + "tag": "student_2M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8330343689459712, + "fixed25_actual_fraction": 0.24510296333500753, + "fixed25_rescue": 1202, + "fixed25_harm": 243, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952878831280435, + "best_epoch": 3, + "seconds": 123.86622095108032 + }, + { + "id": "formal_0151", + "tag": "target_error", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8310253282628973, + "fixed25_actual_fraction": 0.24029561598622373, + "fixed25_rescue": 1177, + "fixed25_harm": 246, + "rank25_metric": 0.8339671378345411, + "validation_selection": 0.7920509882555142, + "best_epoch": 4, + "seconds": 133.07174134254456 + }, + { + "id": "formal_0152", + "tag": "target_four_state", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.24208940231039677, + "fixed25_rescue": 1196, + "fixed25_harm": 241, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952162704096247, + "best_epoch": 3, + "seconds": 134.1348536014557 + }, + { + "id": "formal_0153", + "tag": "fusion_concat", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8328908660400373, + "fixed25_actual_fraction": 0.24366793427566907, + "fixed25_rescue": 1196, + "fixed25_harm": 239, + "rank25_metric": 0.8348999067231111, + "validation_selection": 0.7947149813806932, + "best_epoch": 3, + "seconds": 95.99445247650146 + }, + { + "id": "formal_0166", + "tag": "target_error", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8317428427925665, + "fixed25_actual_fraction": 0.2414436392336945, + "fixed25_rescue": 1188, + "fixed25_harm": 247, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7920223431681467, + "best_epoch": 3, + "seconds": 106.3988299369812 + }, + { + "id": "formal_0211", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8336083805697065, + "fixed25_actual_fraction": 0.24431369735237138, + "fixed25_rescue": 1234, + "fixed25_harm": 267, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7945717559438557, + "best_epoch": 2, + "seconds": 12.189603567123413 + }, + { + "id": "formal_0212", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8308818253569635, + "fixed25_actual_fraction": 0.2361340317141422, + "fixed25_rescue": 1178, + "fixed25_harm": 249, + "rank25_metric": 0.834110640740475, + "validation_selection": 0.7955027212832999, + "best_epoch": 3, + "seconds": 18.873005151748657 + }, + { + "id": "extra_0040", + "tag": "representation_32", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 32, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8328191145870704, + "fixed25_actual_fraction": 0.23677979479084452, + "fixed25_rescue": 1210, + "fixed25_harm": 254, + "rank25_metric": 0.8352586639879458, + "validation_selection": 0.7952735605843597, + "best_epoch": 2, + "seconds": 48.64926338195801 + }, + { + "id": "extra_0041", + "tag": "representation_64", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 64, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.2382865753031499, + "fixed25_rescue": 1207, + "fixed25_harm": 252, + "rank25_metric": 0.8351151610820119, + "validation_selection": 0.794786594099112, + "best_epoch": 2, + "seconds": 49.1362726688385 + }, + { + "id": "extra_0042", + "tag": "student_4M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8335366291167396, + "fixed25_actual_fraction": 0.24072612470402527, + "fixed25_rescue": 1222, + "fixed25_harm": 256, + "rank25_metric": 0.8348999067231111, + "validation_selection": 0.7951160126038385, + "best_epoch": 1, + "seconds": 45.84835124015808 + }, + { + "id": "formal_0151", + "tag": "target_error", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8307383224510296, + "fixed25_actual_fraction": 0.24029561598622373, + "fixed25_rescue": 1214, + "fixed25_harm": 287, + "rank25_metric": 0.8330343689459712, + "validation_selection": 0.7934116299054712, + "best_epoch": 4, + "seconds": 133.07174134254456 + }, + { + "id": "formal_0154", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8329626174930043, + "fixed25_actual_fraction": 0.24094137906292604, + "fixed25_rescue": 1225, + "fixed25_harm": 267, + "rank25_metric": 0.8348281552701442, + "validation_selection": 0.7948438842738471, + "best_epoch": 2, + "seconds": 78.18392181396484 + }, + { + "id": "formal_0155", + "tag": "drop_image", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8321733515103681, + "fixed25_actual_fraction": 0.23692329769677836, + "fixed25_rescue": 1202, + "fixed25_harm": 255, + "rank25_metric": 0.8348281552701442, + "validation_selection": 0.7950300773417358, + "best_epoch": 1, + "seconds": 70.36687850952148 + }, + { + "id": "formal_0156", + "tag": "drop_text", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8335366291167396, + "fixed25_actual_fraction": 0.24302217119896677, + "fixed25_rescue": 1222, + "fixed25_harm": 256, + "rank25_metric": 0.8354021668938796, + "validation_selection": 0.7942136923517618, + "best_epoch": 5, + "seconds": 98.69216632843018 + }, + { + "id": "formal_0157", + "tag": "drop_hidden", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8338953863815742, + "fixed25_actual_fraction": 0.2443854488053383, + "fixed25_rescue": 1243, + "fixed25_harm": 272, + "rank25_metric": 0.8348281552701442, + "validation_selection": 0.7966342022343168, + "best_epoch": 3, + "seconds": 74.8990888595581 + }, + { + "id": "formal_0158", + "tag": "drop_confidence", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8091411351079859, + "fixed25_actual_fraction": 0.2415871421396283, + "fixed25_rescue": 854, + "fixed25_harm": 228, + "rank25_metric": 0.8104326612613906, + "validation_selection": 0.7869378401604126, + "best_epoch": 7, + "seconds": 135.98869681358337 + }, + { + "id": "formal_0159", + "tag": "hidden_position_ordered_image_text__layer_02", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_02", + "fixed25_metric": 0.8321016000574012, + "fixed25_actual_fraction": 0.24194589940446293, + "fixed25_rescue": 1211, + "fixed25_harm": 265, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7966055571469493, + "best_epoch": 3, + "seconds": 94.14239645004272 + }, + { + "id": "formal_0160", + "tag": "hidden_position_ordered_image_text__layer_08", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_08", + "fixed25_metric": 0.8323886058692689, + "fixed25_actual_fraction": 0.24122838487479373, + "fixed25_rescue": 1219, + "fixed25_harm": 269, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952162704096248, + "best_epoch": 1, + "seconds": 75.29347610473633 + }, + { + "id": "formal_0161", + "tag": "hidden_multi_layer", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_02", + "fixed25_metric": 0.8319580971514673, + "fixed25_actual_fraction": 0.23993685872138912, + "fixed25_rescue": 1221, + "fixed25_harm": 277, + "rank25_metric": 0.8336083805697065, + "validation_selection": 0.7959610426811802, + "best_epoch": 1, + "seconds": 71.51151084899902 + }, + { + "id": "formal_0162", + "tag": "representation_128", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 128, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8326038602281696, + "fixed25_actual_fraction": 0.2398651072684222, + "fixed25_rescue": 1214, + "fixed25_harm": 261, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7945717559438557, + "best_epoch": 1, + "seconds": 71.7733223438263 + }, + { + "id": "formal_0163", + "tag": "representation_256", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 256, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8323168544163019, + "fixed25_actual_fraction": 0.24065437325105835, + "fixed25_rescue": 1209, + "fixed25_harm": 260, + "rank25_metric": 0.8342541436464088, + "validation_selection": 0.7948295617301633, + "best_epoch": 2, + "seconds": 87.87156558036804 + }, + { + "id": "formal_0164", + "tag": "student_1M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "1M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8338236349286073, + "fixed25_actual_fraction": 0.24165889359259524, + "fixed25_rescue": 1226, + "fixed25_harm": 256, + "rank25_metric": 0.8364784386883835, + "validation_selection": 0.7949298195359495, + "best_epoch": 5, + "seconds": 109.21437549591064 + }, + { + "id": "formal_0165", + "tag": "student_8M", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8307383224510296, + "fixed25_actual_fraction": 0.23649278897897683, + "fixed25_rescue": 1181, + "fixed25_harm": 254, + "rank25_metric": 0.8338953863815742, + "validation_selection": 0.7947006588370095, + "best_epoch": 9, + "seconds": 120.51619029045105 + }, + { + "id": "formal_0166", + "tag": "target_error", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8300208079213605, + "fixed25_actual_fraction": 0.2402238645332568, + "fixed25_rescue": 1210, + "fixed25_harm": 293, + "rank25_metric": 0.8324603573222358, + "validation_selection": 0.7937124033228301, + "best_epoch": 4, + "seconds": 106.3988299369812 + }, + { + "id": "formal_0167", + "tag": "target_four_state", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8328908660400373, + "fixed25_actual_fraction": 0.23785606658534836, + "fixed25_rescue": 1190, + "fixed25_harm": 233, + "rank25_metric": 0.8354021668938796, + "validation_selection": 0.7952449154969923, + "best_epoch": 5, + "seconds": 106.75910758972168 + }, + { + "id": "formal_0168", + "tag": "fusion_attention", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8330343689459712, + "fixed25_actual_fraction": 0.23886058692688528, + "fixed25_rescue": 1206, + "fixed25_harm": 247, + "rank25_metric": 0.8355456697998135, + "validation_selection": 0.7952305929533086, + "best_epoch": 5, + "seconds": 144.87321782112122 + } + ] +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/fig2_audit/remote_counts.json b/releases/20260913_lr/paper_data/fig2_audit/remote_counts.json new file mode 100644 index 0000000000000000000000000000000000000000..dae151bf5c6e2b28f2183715490865b03df05c2c --- /dev/null +++ b/releases/20260913_lr/paper_data/fig2_audit/remote_counts.json @@ -0,0 +1,74 @@ +[ + { + "expert": "yolo26x", + "endpoint": "MiniCPM", + "images": 3004, + "counts": { + "both_correct": 8176, + "negative": 11580, + "both_correct_negative": 7940, + "rescuable": 1198, + "harmful": 2109, + "positive": 436, + "both_wrong": 533, + "both_correct_positive": 236, + "null_predictions": 112, + "metric_vs_state_difference": 103, + "difference_null": 103 + } + }, + { + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "images": 3004, + "counts": { + "both_correct": 9508, + "negative": 11580, + "both_correct_negative": 9198, + "rescuable": 1282, + "positive": 436, + "both_correct_positive": 310, + "both_wrong": 449, + "harmful": 777, + "null_predictions": 36, + "metric_vs_state_difference": 34, + "difference_null": 34 + } + }, + { + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "images": 3004, + "counts": { + "rescuable": 3314, + "negative": 11580, + "both_correct": 6060, + "both_correct_negative": 5846, + "harmful": 1945, + "positive": 436, + "both_wrong": 697, + "both_correct_positive": 214, + "null_predictions": 112, + "metric_vs_state_difference": 103, + "difference_null": 103 + } + }, + { + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "images": 3004, + "counts": { + "rescuable": 3461, + "negative": 11580, + "both_correct": 7329, + "both_correct_negative": 7036, + "positive": 436, + "both_correct_positive": 293, + "both_wrong": 550, + "harmful": 676, + "null_predictions": 36, + "metric_vs_state_difference": 34, + "difference_null": 34 + } + } +] diff --git a/releases/20260913_lr/paper_data/fig2_audit/remote_counts.py b/releases/20260913_lr/paper_data/fig2_audit/remote_counts.py new file mode 100644 index 0000000000000000000000000000000000000000..52742ccfc7c3ad4861522325458586f293c04fef --- /dev/null +++ b/releases/20260913_lr/paper_data/fig2_audit/remote_counts.py @@ -0,0 +1,25 @@ +from pathlib import Path +import json +from collections import Counter + +base=Path('/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_full_20260910/data') +out=[] +for expert in ['yolo26x','rtdetr_x_fp32']: + rows=[json.loads(s) for s in (base/expert/'test/records.jsonl').read_text().splitlines()] + for endpoint in ['MiniCPM','Qwen3.8']: + c=Counter() + for r in rows: + for event,s in r['small_label']['events'].items(): + l=r['large_labels'][endpoint]['events'][event] + assert s['target']==l['target'] + state={(True,True):'both_correct',(False,True):'rescuable',(True,False):'harmful',(False,False):'both_wrong'}[(bool(s['correct']),bool(l['correct']))] + c[state]+=1 + c['positive' if s['target'] else 'negative']+=1 + if state=='both_correct':c['both_correct_positive' if s['target'] else 'both_correct_negative']+=1 + if l['prediction'] is None:c['null_predictions']+=1 + score_correct=(l['prediction'] is True)==bool(l['target']) + if score_correct!=bool(l['correct']): + c['metric_vs_state_difference']+=1 + c['difference_null' if l['prediction'] is None else 'difference_other']+=1 + out.append(dict(expert=expert,endpoint=endpoint,images=len(rows),counts=dict(c))) +print(json.dumps(out,indent=2)) diff --git a/releases/20260913_lr/paper_data/five_seed_counts.json b/releases/20260913_lr/paper_data/five_seed_counts.json new file mode 100644 index 0000000000000000000000000000000000000000..d9969fd1102902e35ce53bea247ab4599de6c382 --- /dev/null +++ b/releases/20260913_lr/paper_data/five_seed_counts.json @@ -0,0 +1,7 @@ +{ + "better_metric": 11, + "fewer_calls": 9, + "both": 8, + "above_endpoints": 10, + "better_ranked_area": 16 +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/full_budget_results.json b/releases/20260913_lr/paper_data/full_budget_results.json new file mode 100644 index 0000000000000000000000000000000000000000..40b2917982680d05fa79922e2fc9b21123591992 --- /dev/null +++ b/releases/20260913_lr/paper_data/full_budget_results.json @@ -0,0 +1 @@ +{"protocol": {"pairs": 16, "seeds": [2026, 2027, 2028, 2029, 2030], "runs": 80, "gpu_ids": [0, 1], "concurrent_runs": 10, "training": "Same data, frozen backbone, paired cross-entropy, optimizer, random seeds, maximum 50 epochs and patience 8 as original. Only validation selection changes.", "selection": "At each epoch, evaluate normalized score with alpha in [0,.25,.5,.75,1], beta=1 on validation exact-call curve at 101 points including endpoints. Maximize full-curve mean; same-epoch ties prefer smaller alpha; epoch ties retain earlier epoch. Early stopping follows this metric.", "threshold": "Select maximum validation task score over budgets 0..100% inclusive; ties fewer calls then lower budget. Never uses test to select.", "caveat": "Exploratory revision; test previously inspected. Historical comparison changes model selection and scoring jointly; same-checkpoint score controls are additionally reported.", "time": 1789272780.0844343}, "pairs": [{"expert": "cub", "endpoint": "MiniCPM", "task": "cub", "small": 81.429064549534, "large": 26.561960648947185, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.0, "validation_area100": 0.7167333333333333, "training_seconds": 4.1882862279890105, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.004982936196029186, "operator": "ge"}, {"budget": 0.02, "threshold": -0.007442648988217115, "operator": "ge"}, {"budget": 0.03, "threshold": -0.008495921269059181, "operator": "ge"}, {"budget": 0.04, "threshold": -0.009911535307765007, "operator": "ge"}, {"budget": 0.05, "threshold": -0.011113252490758896, "operator": "ge"}, {"budget": 0.06, "threshold": -0.014889698475599289, "operator": "ge"}, {"budget": 0.07, "threshold": -0.022369058802723885, "operator": "ge"}, {"budget": 0.08, "threshold": -0.03006061539053917, "operator": "ge"}, {"budget": 0.09, "threshold": -0.03601733595132828, "operator": "ge"}, {"budget": 0.1, "threshold": -0.04350412264466286, "operator": "ge"}, {"budget": 0.11, "threshold": -0.05446135997772217, "operator": "ge"}, {"budget": 0.12, "threshold": -0.07404012233018875, "operator": "ge"}, {"budget": 0.13, "threshold": -0.09383945167064667, "operator": "ge"}, {"budget": 0.14, "threshold": -0.10576465725898743, "operator": "ge"}, {"budget": 0.15, "threshold": -0.12814979255199432, "operator": "ge"}, {"budget": 0.16, "threshold": -0.16552507877349854, "operator": "ge"}, {"budget": 0.17, "threshold": -0.20467445254325867, "operator": "ge"}, {"budget": 0.18, "threshold": -0.22682896256446838, "operator": "ge"}, {"budget": 0.19, "threshold": -0.24548551440238953, "operator": "ge"}, {"budget": 0.2, "threshold": -0.2673734724521637, "operator": "ge"}, {"budget": 0.21, "threshold": -0.3145397901535034, "operator": "ge"}, {"budget": 0.22, "threshold": -0.3506848216056824, "operator": "ge"}, {"budget": 0.23, "threshold": -0.37523525953292847, "operator": "ge"}, {"budget": 0.24, "threshold": -0.40892019867897034, "operator": "ge"}, {"budget": 0.25, "threshold": -0.44742876291275024, "operator": "ge"}, {"budget": 0.26, "threshold": -0.4835400879383087, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5130662322044373, "operator": "ge"}, {"budget": 0.28, "threshold": -0.5726482272148132, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6036837100982666, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6394438743591309, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6597222089767456, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6965112686157227, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7220051288604736, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7446874380111694, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7761076092720032, "operator": "ge"}, {"budget": 0.36, "threshold": -0.787558913230896, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8019205927848816, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8184149265289307, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8369619846343994, "operator": "ge"}, {"budget": 0.4, "threshold": -0.851140558719635, "operator": "ge"}, {"budget": 0.41, "threshold": -0.8581717610359192, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8659921884536743, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8749133348464966, "operator": "ge"}, {"budget": 0.44, "threshold": -0.883246123790741, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8899792432785034, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9014853835105896, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9081348180770874, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9156812429428101, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9221369028091431, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9308832287788391, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9360386729240417, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9423516988754272, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9475168585777283, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9521111249923706, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9543133974075317, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9577895998954773, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9610756635665894, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9623329043388367, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9644604921340942, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9655663967132568, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9677255749702454, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9697721600532532, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9724923372268677, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9737414121627808, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9753139019012451, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9767133593559265, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9779720306396484, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9798229336738586, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9804249405860901, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9810692071914673, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9819605946540833, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9833666682243347, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9842023849487305, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9849138855934143, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9858112931251526, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9864581823348999, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9873479008674622, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9881471395492554, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9892186522483826, "operator": "ge"}, {"budget": 0.8, "threshold": -0.990036129951477, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9912050366401672, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9917094111442566, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9927935004234314, "operator": "ge"}, {"budget": 0.84, "threshold": -0.993672788143158, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9942247271537781, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9950496554374695, "operator": "ge"}, {"budget": 0.87, "threshold": -0.995774507522583, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9961745142936707, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9965148568153381, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9967954158782959, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9972493648529053, "operator": "ge"}, {"budget": 0.92, "threshold": -0.997543215751648, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9978970885276794, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9982203245162964, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9984103441238403, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9985606670379639, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9987450838088989, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9988723993301392, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9991393089294434, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.0, 97.66666666666667, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.08333333333333, 96.91666666666666, 96.5, 96.16666666666667, 95.83333333333334, 95.41666666666667, 95.16666666666667, 94.91666666666667, 94.41666666666667, 94.08333333333333, 93.58333333333333, 92.91666666666667, 92.16666666666666, 91.66666666666666, 90.83333333333333, 90.08333333333334, 89.25, 88.33333333333333, 87.58333333333333, 86.83333333333334, 86.0, 85.08333333333333, 84.25, 83.58333333333333, 82.83333333333334, 81.91666666666667, 81.08333333333333, 80.33333333333333, 79.5, 78.66666666666666, 77.83333333333333, 77.08333333333334, 76.33333333333333, 75.5, 74.5, 73.66666666666667, 72.66666666666667, 71.75, 70.91666666666667, 70.08333333333334, 69.25, 68.16666666666666, 67.16666666666666, 66.25000000000001, 65.25000000000001, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.666666666666664, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.91666666666667, 54.0, 53.0, 52.08333333333333, 51.083333333333336, 50.16666666666667, 49.16666666666667, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.166666666666664, 42.25, 41.25, 40.25, 39.24999999999999, 38.33333333333333, 37.416666666666664, 36.416666666666664, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 8, 13, 16, 18, 18, 21, 23, 28, 32, 36, 41, 44, 47, 53, 57, 64, 72, 81, 87, 97, 106, 116, 127, 136, 145, 155, 166, 176, 184, 193, 204, 214, 224, 234, 244, 254, 263, 272, 282, 294, 305, 317, 328, 338, 348, 358, 371, 383, 394, 406, 416, 428, 440, 450, 462, 473, 485, 495, 507, 519, 530, 541, 553, 564, 576, 587, 599, 611, 623, 635, 646, 659, 671, 682, 694, 706, 718, 729, 740, 752, 762, 774, 786, 797, 809, 820, 832, 844, 856], "exact_score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.0, 97.66666666666667, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.08333333333333, 96.91666666666666, 96.5, 96.16666666666667, 95.83333333333334, 95.41666666666667, 95.16666666666667, 94.91666666666667, 94.41666666666667, 94.08333333333333, 93.58333333333333, 92.91666666666667, 92.16666666666666, 91.66666666666666, 90.83333333333333, 90.08333333333334, 89.25, 88.33333333333333, 87.58333333333333, 86.83333333333334, 86.0, 85.08333333333333, 84.25, 83.58333333333333, 82.75, 81.91666666666667, 81.08333333333333, 80.33333333333333, 79.5, 78.66666666666666, 77.83333333333333, 77.08333333333334, 76.33333333333333, 75.5, 74.5, 73.66666666666667, 72.66666666666667, 71.75, 70.91666666666667, 70.08333333333334, 69.16666666666667, 68.16666666666666, 67.16666666666666, 66.25000000000001, 65.25000000000001, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.666666666666664, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.91666666666667, 54.0, 53.0, 52.08333333333333, 51.083333333333336, 50.16666666666667, 49.16666666666667, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.166666666666664, 42.25, 41.25, 40.25, 39.24999999999999, 38.33333333333333, 37.416666666666664, 36.416666666666664, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.67333333333333, "threshold_area100": 71.67583333333332, "threshold_area25": 97.45000000000002}, "test": {"score": [81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.32550914739386, 81.23921297894374, 81.1356575768036, 81.11839834311357, 81.03210217466345, 80.84225060407319, 80.70417673455299, 80.53158439765275, 80.32447359337246, 80.11736278909216, 79.68588194684156, 79.23714187090093, 78.89195719710045, 78.51225405591991, 77.83914394200899, 77.09699689333793, 76.8553676216776, 75.38833275802554, 74.81877804625475, 74.09389023127373, 73.67966862271315, 72.7821884708319, 71.95374525371074, 71.17707973765965, 69.74456334138765, 69.29582326544703, 68.46738004832585, 67.79426993441491, 66.75871591301346, 65.91301346220229, 65.4987918536417, 65.11908871246118, 64.48049706593028, 63.65205384880912, 63.13427683810839, 62.02968588194684, 61.32205730065585, 60.71798412150501, 59.82050396962375, 58.81946841560235, 58.23265447014153, 57.26613738350017, 56.454953400069044, 55.678287884017955, 55.31584397652745, 54.48740075940628, 53.71073524335519, 53.227476700034515, 52.53710735243355, 52.226441146013116, 51.363479461511915, 50.58681394546083, 49.620296858819465, 49.119779081808765, 48.4984466689679, 47.79081808767691, 47.25578184328616, 46.23748705557473, 46.03037625129445, 45.75422851225406, 45.23645150155333, 44.21815671384191, 43.8557128063514, 43.372454263030725, 42.751121850189854, 42.31964100793925, 41.56023472557819, 41.042457714877465, 39.98964445978599, 39.26475664480497, 38.26372109078357, 37.83224024853297, 37.159130134622025, 36.34794615119089, 35.761132205730064, 34.89817052122886, 34.22506040731791, 33.53469105971695, 32.982395581636176, 32.68898860890577, 31.72247152226441, 31.11839834311356, 30.531584397652743, 29.720400414221615, 29.237141870900928, 28.719364860200216, 28.098032447359344, 27.683810838798763, 27.06247842595789, 26.561960648947192], "calls": [0.0, 1.3634794615119088, 1.9675526406627546, 2.2609596133931653, 2.8132550914739385, 3.1066620642043494, 4.176734552985847, 5.591991715567829, 7.00724887814981, 8.025543665861235, 9.147393855712806, 10.40731791508457, 12.133241284086987, 13.462202278218847, 14.169830859509839, 15.447014152571626, 17.138419054193992, 19.15774939592682, 20.141525716258197, 20.987228167069382, 21.763893683120468, 23.62789092164308, 25.008629616845013, 26.078702105626512, 27.32136693130825, 29.27166033828098, 30.9285467725233, 32.22298929927511, 35.1397997928892, 36.29616845012082, 37.97031411805316, 38.88505350362444, 40.85260614428719, 42.30238177424923, 43.872972040041425, 46.427338626164996, 47.39385571280635, 48.74007594062824, 49.913703831549874, 51.76044183638246, 53.03762512944425, 53.86606834656541, 54.60821539523645, 55.47117707973766, 56.61028650327925, 57.42147048671039, 59.16465308940283, 60.04487400759406, 60.90783569209527, 62.16775975146703, 63.634794615119084, 64.51501553331032, 65.77493959268207, 66.82775284777355, 67.88056610286503, 68.32930617880567, 69.26130479806697, 70.34863652053849, 70.88367276492924, 71.67759751467034, 72.04004142216085, 73.04107697618227, 74.04211253020367, 75.25025888850536, 75.87159130134621, 76.56196064894719, 77.33862616499827, 77.94269934414912, 79.08180876769072, 79.30617880566103, 79.59958577839143, 80.13462202278218, 81.25647221263375, 81.65343458750431, 82.20573006558509, 82.861580945806, 83.3620987228167, 84.12150500517777, 84.7083189506386, 85.79565067311012, 86.55505695547117, 87.62512944425268, 88.09112875388332, 88.86779426993442, 89.7134967207456, 90.38660683465655, 91.31860545391784, 92.07801173627891, 92.82015878494995, 93.38971349672074, 93.70037970314118, 94.70141525716258, 95.32274767000345, 95.94408008284432, 96.77252329996548, 97.27304107697618, 97.82533655505695, 98.44666896789782, 98.8608905764584, 99.48222298929927, 100.0], "rescued": [0, 6, 6, 6, 6, 6, 6, 8, 10, 12, 12, 12, 13, 13, 14, 17, 24, 34, 37, 38, 40, 49, 56, 60, 67, 76, 80, 91, 100, 104, 109, 112, 125, 130, 135, 143, 147, 151, 155, 163, 165, 169, 174, 175, 176, 179, 186, 188, 189, 192, 197, 201, 206, 209, 213, 213, 213, 216, 216, 216, 216, 217, 219, 221, 222, 223, 223, 224, 225, 225, 225, 225, 225, 225, 225, 225, 225, 225, 226, 226, 226, 227, 227, 228, 228, 230, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 8, 8, 8, 8, 8, 9, 10, 10, 13, 15, 18, 24, 30, 32, 40, 58, 76, 89, 102, 116, 150, 183, 207, 236, 284, 331, 356, 450, 487, 534, 561, 626, 679, 729, 820, 850, 902, 945, 1013, 1064, 1092, 1119, 1157, 1206, 1239, 1310, 1353, 1389, 1444, 1507, 1545, 1606, 1656, 1705, 1726, 1774, 1822, 1850, 1890, 1908, 1959, 2006, 2064, 2094, 2131, 2172, 2204, 2264, 2276, 2292, 2322, 2381, 2402, 2430, 2466, 2491, 2535, 2566, 2627, 2669, 2728, 2753, 2793, 2840, 2876, 2927, 2967, 3007, 3039, 3056, 3112, 3147, 3181, 3228, 3256, 3286, 3322, 3346, 3382, 3411], "exact_score": [81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.25647221263375, 81.18743527787366, 81.11839834311357, 81.0666206420435, 81.01484294097342, 80.84225060407319, 80.70417673455299, 80.66965826717293, 80.60062133241284, 80.32447359337246, 79.96202968588196, 79.7894373489817, 79.54780807732136, 79.23714187090093, 78.90921643079048, 78.56403175698999, 78.32240248532966, 77.92544011045909, 77.51121850189851, 77.11425612702796, 76.9244045564377, 76.52744218156714, 75.99240593717639, 75.44011045909562, 74.99137038315499, 74.62892647566449, 74.0766309975837, 73.59337245426303, 73.09285467725233, 72.69589230238178, 72.143596824301, 71.57404211253021, 71.09078356920952, 70.59026579219882, 70.03797031411806, 69.48567483603728, 68.95063859164652, 68.32930617880567, 67.75975146703487, 67.19019675526407, 66.60338280980325, 65.94753192958233, 65.44701415257163, 64.87745944080083, 64.08353469105973, 63.375906109768735, 62.7890921643079, 62.115982050396966, 61.37383500172593, 60.68346565412496, 59.97583707283397, 59.285467725233, 58.62961684501209, 57.818432861580945, 57.05902657921988, 56.316879530548846, 55.591991715567836, 54.74628926475664, 53.96962374870555, 53.141180531584396, 52.26095961339317, 51.39799792889196, 50.62133241284087, 49.86192613047981, 48.9989644459786, 48.08422506040732, 47.204004142216085, 46.30652399033483, 45.3572661373835, 44.45978598550225, 43.56230583362099, 42.64756644804971, 41.69830859509838, 40.766309975837075, 39.81705212288574, 38.90231273731447, 37.93579565067311, 37.03831549879186, 36.10631687953055, 35.2088367276493, 34.29409734207801, 33.36209872281671, 32.41284086986538, 31.446323783224027, 30.479806696582678, 29.49603037625129, 28.546772523299968, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 63.19187811561338, "threshold_area100": 58.43501898515706, "threshold_area25": 80.711080428029}, "selected_budget": 1, "selected": {"val": {"score": 98.66666666666667, "calls": 1.0, "rescued": 1, "harmed": 0}, "test": {"score": 81.39454608215395, "calls": 1.3634794615119088, "rescued": 6, "harmed": 8}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.004982936196029186, "operator": "ge"}, {"budget": 0.02, "threshold": -0.007442648988217115, "operator": "ge"}, {"budget": 0.03, "threshold": -0.008495921269059181, "operator": "ge"}, {"budget": 0.04, "threshold": -0.009911535307765007, "operator": "ge"}, {"budget": 0.05, "threshold": -0.011113252490758896, "operator": "ge"}, {"budget": 0.06, "threshold": -0.014889698475599289, "operator": "ge"}, {"budget": 0.07, "threshold": -0.022369058802723885, "operator": "ge"}, {"budget": 0.08, "threshold": -0.03006061539053917, "operator": "ge"}, {"budget": 0.09, "threshold": -0.03601733595132828, "operator": "ge"}, {"budget": 0.1, "threshold": -0.04350412264466286, "operator": "ge"}, {"budget": 0.11, "threshold": -0.05446135997772217, "operator": "ge"}, {"budget": 0.12, "threshold": -0.07404012233018875, "operator": "ge"}, {"budget": 0.13, "threshold": -0.09383945167064667, "operator": "ge"}, {"budget": 0.14, "threshold": -0.10576465725898743, "operator": "ge"}, {"budget": 0.15, "threshold": -0.12814979255199432, "operator": "ge"}, {"budget": 0.16, "threshold": -0.16552507877349854, "operator": "ge"}, {"budget": 0.17, "threshold": -0.20467445254325867, "operator": "ge"}, {"budget": 0.18, "threshold": -0.22682896256446838, "operator": "ge"}, {"budget": 0.19, "threshold": -0.24548551440238953, "operator": "ge"}, {"budget": 0.2, "threshold": -0.2673734724521637, "operator": "ge"}, {"budget": 0.21, "threshold": -0.3145397901535034, "operator": "ge"}, {"budget": 0.22, "threshold": -0.3506848216056824, "operator": "ge"}, {"budget": 0.23, "threshold": -0.37523525953292847, "operator": "ge"}, {"budget": 0.24, "threshold": -0.40892019867897034, "operator": "ge"}, {"budget": 0.25, "threshold": -0.44742876291275024, "operator": "ge"}, {"budget": 0.26, "threshold": -0.4835400879383087, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5130662322044373, "operator": "ge"}, {"budget": 0.28, "threshold": -0.5726482272148132, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6036837100982666, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6394438743591309, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6597222089767456, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6965112686157227, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7220051288604736, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7446874380111694, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7761076092720032, "operator": "ge"}, {"budget": 0.36, "threshold": -0.787558913230896, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8019205927848816, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8184149265289307, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8369619846343994, "operator": "ge"}, {"budget": 0.4, "threshold": -0.851140558719635, "operator": "ge"}, {"budget": 0.41, "threshold": -0.8581717610359192, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8659921884536743, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8749133348464966, "operator": "ge"}, {"budget": 0.44, "threshold": -0.883246123790741, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8899792432785034, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9014853835105896, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9081348180770874, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9156812429428101, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9221369028091431, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9308832287788391, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9360386729240417, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9423516988754272, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9475168585777283, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9521111249923706, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9543133974075317, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9577895998954773, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9610756635665894, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9623329043388367, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9644604921340942, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9655663967132568, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9677255749702454, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9697721600532532, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9724923372268677, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9737414121627808, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9753139019012451, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9767133593559265, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9779720306396484, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9798229336738586, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9804249405860901, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9810692071914673, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9819605946540833, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9833666682243347, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9842023849487305, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9849138855934143, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9858112931251526, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9864581823348999, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9873479008674622, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9881471395492554, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9892186522483826, "operator": "ge"}, {"budget": 0.8, "threshold": -0.990036129951477, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9912050366401672, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9917094111442566, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9927935004234314, "operator": "ge"}, {"budget": 0.84, "threshold": -0.993672788143158, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9942247271537781, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9950496554374695, "operator": "ge"}, {"budget": 0.87, "threshold": -0.995774507522583, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9961745142936707, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9965148568153381, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9967954158782959, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9972493648529053, "operator": "ge"}, {"budget": 0.92, "threshold": -0.997543215751648, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9978970885276794, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9982203245162964, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9984103441238403, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9985606670379639, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9987450838088989, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9988723993301392, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9991393089294434, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.0, 97.66666666666667, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.08333333333333, 96.91666666666666, 96.5, 96.16666666666667, 95.83333333333334, 95.41666666666667, 95.16666666666667, 94.91666666666667, 94.41666666666667, 94.08333333333333, 93.58333333333333, 92.91666666666667, 92.16666666666666, 91.66666666666666, 90.83333333333333, 90.08333333333334, 89.25, 88.33333333333333, 87.58333333333333, 86.83333333333334, 86.0, 85.08333333333333, 84.25, 83.58333333333333, 82.83333333333334, 81.91666666666667, 81.08333333333333, 80.33333333333333, 79.5, 78.66666666666666, 77.83333333333333, 77.08333333333334, 76.33333333333333, 75.5, 74.5, 73.66666666666667, 72.66666666666667, 71.75, 70.91666666666667, 70.08333333333334, 69.25, 68.16666666666666, 67.16666666666666, 66.25000000000001, 65.25000000000001, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.666666666666664, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.91666666666667, 54.0, 53.0, 52.08333333333333, 51.083333333333336, 50.16666666666667, 49.16666666666667, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.166666666666664, 42.25, 41.25, 40.25, 39.24999999999999, 38.33333333333333, 37.416666666666664, 36.416666666666664, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 8, 13, 16, 18, 18, 21, 23, 28, 32, 36, 41, 44, 47, 53, 57, 64, 72, 81, 87, 97, 106, 116, 127, 136, 145, 155, 166, 176, 184, 193, 204, 214, 224, 234, 244, 254, 263, 272, 282, 294, 305, 317, 328, 338, 348, 358, 371, 383, 394, 406, 416, 428, 440, 450, 462, 473, 485, 495, 507, 519, 530, 541, 553, 564, 576, 587, 599, 611, 623, 635, 646, 659, 671, 682, 694, 706, 718, 729, 740, 752, 762, 774, 786, 797, 809, 820, 832, 844, 856], "exact_score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.0, 97.66666666666667, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.08333333333333, 96.91666666666666, 96.5, 96.16666666666667, 95.83333333333334, 95.41666666666667, 95.16666666666667, 94.91666666666667, 94.41666666666667, 94.08333333333333, 93.58333333333333, 92.91666666666667, 92.16666666666666, 91.66666666666666, 90.83333333333333, 90.08333333333334, 89.25, 88.33333333333333, 87.58333333333333, 86.83333333333334, 86.0, 85.08333333333333, 84.25, 83.58333333333333, 82.75, 81.91666666666667, 81.08333333333333, 80.33333333333333, 79.5, 78.66666666666666, 77.83333333333333, 77.08333333333334, 76.33333333333333, 75.5, 74.5, 73.66666666666667, 72.66666666666667, 71.75, 70.91666666666667, 70.08333333333334, 69.16666666666667, 68.16666666666666, 67.16666666666666, 66.25000000000001, 65.25000000000001, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.666666666666664, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.91666666666667, 54.0, 53.0, 52.08333333333333, 51.083333333333336, 50.16666666666667, 49.16666666666667, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.166666666666664, 42.25, 41.25, 40.25, 39.24999999999999, 38.33333333333333, 37.416666666666664, 36.416666666666664, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.67333333333333, "threshold_area100": 71.67583333333332, "threshold_area25": 97.45000000000002}, "test": {"score": [81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.32550914739386, 81.23921297894374, 81.1356575768036, 81.11839834311357, 81.03210217466345, 80.84225060407319, 80.70417673455299, 80.53158439765275, 80.32447359337246, 80.11736278909216, 79.68588194684156, 79.23714187090093, 78.89195719710045, 78.51225405591991, 77.83914394200899, 77.09699689333793, 76.8553676216776, 75.38833275802554, 74.81877804625475, 74.09389023127373, 73.67966862271315, 72.7821884708319, 71.95374525371074, 71.17707973765965, 69.74456334138765, 69.29582326544703, 68.46738004832585, 67.79426993441491, 66.75871591301346, 65.91301346220229, 65.4987918536417, 65.11908871246118, 64.48049706593028, 63.65205384880912, 63.13427683810839, 62.02968588194684, 61.32205730065585, 60.71798412150501, 59.82050396962375, 58.81946841560235, 58.23265447014153, 57.26613738350017, 56.454953400069044, 55.678287884017955, 55.31584397652745, 54.48740075940628, 53.71073524335519, 53.227476700034515, 52.53710735243355, 52.226441146013116, 51.363479461511915, 50.58681394546083, 49.620296858819465, 49.119779081808765, 48.4984466689679, 47.79081808767691, 47.25578184328616, 46.23748705557473, 46.03037625129445, 45.75422851225406, 45.23645150155333, 44.21815671384191, 43.8557128063514, 43.372454263030725, 42.751121850189854, 42.31964100793925, 41.56023472557819, 41.042457714877465, 39.98964445978599, 39.26475664480497, 38.26372109078357, 37.83224024853297, 37.159130134622025, 36.34794615119089, 35.761132205730064, 34.89817052122886, 34.22506040731791, 33.53469105971695, 32.982395581636176, 32.68898860890577, 31.72247152226441, 31.11839834311356, 30.531584397652743, 29.720400414221615, 29.237141870900928, 28.719364860200216, 28.098032447359344, 27.683810838798763, 27.06247842595789, 26.561960648947192], "calls": [0.0, 1.3634794615119088, 1.9675526406627546, 2.2609596133931653, 2.8132550914739385, 3.1066620642043494, 4.176734552985847, 5.591991715567829, 7.00724887814981, 8.025543665861235, 9.147393855712806, 10.40731791508457, 12.133241284086987, 13.462202278218847, 14.169830859509839, 15.447014152571626, 17.138419054193992, 19.15774939592682, 20.141525716258197, 20.987228167069382, 21.763893683120468, 23.62789092164308, 25.008629616845013, 26.078702105626512, 27.32136693130825, 29.27166033828098, 30.9285467725233, 32.22298929927511, 35.1397997928892, 36.29616845012082, 37.97031411805316, 38.88505350362444, 40.85260614428719, 42.30238177424923, 43.872972040041425, 46.427338626164996, 47.39385571280635, 48.74007594062824, 49.913703831549874, 51.76044183638246, 53.03762512944425, 53.86606834656541, 54.60821539523645, 55.47117707973766, 56.61028650327925, 57.42147048671039, 59.16465308940283, 60.04487400759406, 60.90783569209527, 62.16775975146703, 63.634794615119084, 64.51501553331032, 65.77493959268207, 66.82775284777355, 67.88056610286503, 68.32930617880567, 69.26130479806697, 70.34863652053849, 70.88367276492924, 71.67759751467034, 72.04004142216085, 73.04107697618227, 74.04211253020367, 75.25025888850536, 75.87159130134621, 76.56196064894719, 77.33862616499827, 77.94269934414912, 79.08180876769072, 79.30617880566103, 79.59958577839143, 80.13462202278218, 81.25647221263375, 81.65343458750431, 82.20573006558509, 82.861580945806, 83.3620987228167, 84.12150500517777, 84.7083189506386, 85.79565067311012, 86.55505695547117, 87.62512944425268, 88.09112875388332, 88.86779426993442, 89.7134967207456, 90.38660683465655, 91.31860545391784, 92.07801173627891, 92.82015878494995, 93.38971349672074, 93.70037970314118, 94.70141525716258, 95.32274767000345, 95.94408008284432, 96.77252329996548, 97.27304107697618, 97.82533655505695, 98.44666896789782, 98.8608905764584, 99.48222298929927, 100.0], "rescued": [0, 6, 6, 6, 6, 6, 6, 8, 10, 12, 12, 12, 13, 13, 14, 17, 24, 34, 37, 38, 40, 49, 56, 60, 67, 76, 80, 91, 100, 104, 109, 112, 125, 130, 135, 143, 147, 151, 155, 163, 165, 169, 174, 175, 176, 179, 186, 188, 189, 192, 197, 201, 206, 209, 213, 213, 213, 216, 216, 216, 216, 217, 219, 221, 222, 223, 223, 224, 225, 225, 225, 225, 225, 225, 225, 225, 225, 225, 226, 226, 226, 227, 227, 228, 228, 230, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 8, 8, 8, 8, 8, 9, 10, 10, 13, 15, 18, 24, 30, 32, 40, 58, 76, 89, 102, 116, 150, 183, 207, 236, 284, 331, 356, 450, 487, 534, 561, 626, 679, 729, 820, 850, 902, 945, 1013, 1064, 1092, 1119, 1157, 1206, 1239, 1310, 1353, 1389, 1444, 1507, 1545, 1606, 1656, 1705, 1726, 1774, 1822, 1850, 1890, 1908, 1959, 2006, 2064, 2094, 2131, 2172, 2204, 2264, 2276, 2292, 2322, 2381, 2402, 2430, 2466, 2491, 2535, 2566, 2627, 2669, 2728, 2753, 2793, 2840, 2876, 2927, 2967, 3007, 3039, 3056, 3112, 3147, 3181, 3228, 3256, 3286, 3322, 3346, 3382, 3411], "exact_score": [81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.25647221263375, 81.18743527787366, 81.11839834311357, 81.0666206420435, 81.01484294097342, 80.84225060407319, 80.70417673455299, 80.66965826717293, 80.60062133241284, 80.32447359337246, 79.96202968588196, 79.7894373489817, 79.54780807732136, 79.23714187090093, 78.90921643079048, 78.56403175698999, 78.32240248532966, 77.92544011045909, 77.51121850189851, 77.11425612702796, 76.9244045564377, 76.52744218156714, 75.99240593717639, 75.44011045909562, 74.99137038315499, 74.62892647566449, 74.0766309975837, 73.59337245426303, 73.09285467725233, 72.69589230238178, 72.143596824301, 71.57404211253021, 71.09078356920952, 70.59026579219882, 70.03797031411806, 69.48567483603728, 68.95063859164652, 68.32930617880567, 67.75975146703487, 67.19019675526407, 66.60338280980325, 65.94753192958233, 65.44701415257163, 64.87745944080083, 64.08353469105973, 63.375906109768735, 62.7890921643079, 62.115982050396966, 61.37383500172593, 60.68346565412496, 59.97583707283397, 59.285467725233, 58.62961684501209, 57.818432861580945, 57.05902657921988, 56.316879530548846, 55.591991715567836, 54.74628926475664, 53.96962374870555, 53.141180531584396, 52.26095961339317, 51.39799792889196, 50.62133241284087, 49.86192613047981, 48.9989644459786, 48.08422506040732, 47.204004142216085, 46.30652399033483, 45.3572661373835, 44.45978598550225, 43.56230583362099, 42.64756644804971, 41.69830859509838, 40.766309975837075, 39.81705212288574, 38.90231273731447, 37.93579565067311, 37.03831549879186, 36.10631687953055, 35.2088367276493, 34.29409734207801, 33.36209872281671, 32.41284086986538, 31.446323783224027, 30.479806696582678, 29.49603037625129, 28.546772523299968, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 63.19187811561338, "threshold_area100": 58.43501898515706, "threshold_area25": 80.711080428029}, "selected_budget": 1, "selected": {"val": {"score": 98.66666666666667, "calls": 1.0, "rescued": 1, "harmed": 0}, "test": {"score": 81.39454608215395, "calls": 1.3634794615119088, "rescued": 6, "harmed": 8}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.1720302104949951, "operator": "ge"}, {"budget": 0.02, "threshold": -1.8163822889328003, "operator": "ge"}, {"budget": 0.03, "threshold": -2.3195700645446777, "operator": "ge"}, {"budget": 0.04, "threshold": -2.5348222255706787, "operator": "ge"}, {"budget": 0.05, "threshold": -2.865732431411743, "operator": "ge"}, {"budget": 0.06, "threshold": -3.357342481613159, "operator": "ge"}, {"budget": 0.07, "threshold": -3.7136318683624268, "operator": "ge"}, {"budget": 0.08, "threshold": -3.9441003799438477, "operator": "ge"}, {"budget": 0.09, "threshold": -4.231906890869141, "operator": "ge"}, {"budget": 0.1, "threshold": -4.417206764221191, "operator": "ge"}, {"budget": 0.11, "threshold": -4.68219518661499, "operator": "ge"}, {"budget": 0.12, "threshold": -4.947386264801025, "operator": "ge"}, {"budget": 0.13, "threshold": -5.204185485839844, "operator": "ge"}, {"budget": 0.14, "threshold": -5.494235992431641, "operator": "ge"}, {"budget": 0.15, "threshold": -5.794378757476807, "operator": "ge"}, {"budget": 0.16, "threshold": -6.0317559242248535, "operator": "ge"}, {"budget": 0.17, "threshold": -6.228052139282227, "operator": "ge"}, {"budget": 0.18, "threshold": -6.54228401184082, "operator": "ge"}, {"budget": 0.19, "threshold": -6.803375244140625, "operator": "ge"}, {"budget": 0.2, "threshold": -6.989691734313965, "operator": "ge"}, {"budget": 0.21, "threshold": -7.244710445404053, "operator": "ge"}, {"budget": 0.22, "threshold": -7.411199569702148, "operator": "ge"}, {"budget": 0.23, "threshold": -7.5584869384765625, "operator": "ge"}, {"budget": 0.24, "threshold": -7.700355529785156, "operator": "ge"}, {"budget": 0.25, "threshold": -7.867853164672852, "operator": "ge"}, {"budget": 0.26, "threshold": -8.07803726196289, "operator": "ge"}, {"budget": 0.27, "threshold": -8.218085289001465, "operator": "ge"}, {"budget": 0.28, "threshold": -8.407442092895508, "operator": "ge"}, {"budget": 0.29, "threshold": -8.580204010009766, "operator": "ge"}, {"budget": 0.3, "threshold": -8.669848442077637, "operator": "ge"}, {"budget": 0.31, "threshold": -8.79475212097168, "operator": "ge"}, {"budget": 0.32, "threshold": -8.92697811126709, "operator": "ge"}, {"budget": 0.33, "threshold": -9.001590728759766, "operator": "ge"}, {"budget": 0.34, "threshold": -9.05352783203125, "operator": "ge"}, {"budget": 0.35, "threshold": -9.135518074035645, "operator": "ge"}, {"budget": 0.36, "threshold": -9.23332405090332, "operator": "ge"}, {"budget": 0.37, "threshold": -9.317520141601562, "operator": "ge"}, {"budget": 0.38, "threshold": -9.436118125915527, "operator": "ge"}, {"budget": 0.39, "threshold": -9.555011749267578, "operator": "ge"}, {"budget": 0.4, "threshold": -9.649856567382812, "operator": "ge"}, {"budget": 0.41, "threshold": -9.742595672607422, "operator": "ge"}, {"budget": 0.42, "threshold": -9.839241981506348, "operator": "ge"}, {"budget": 0.43, "threshold": -9.966024398803711, "operator": "ge"}, {"budget": 0.44, "threshold": -10.083967208862305, "operator": "ge"}, {"budget": 0.45, "threshold": -10.185546875, "operator": "ge"}, {"budget": 0.46, "threshold": -10.249189376831055, "operator": "ge"}, {"budget": 0.47, "threshold": -10.375177383422852, "operator": "ge"}, {"budget": 0.48, "threshold": -10.479530334472656, "operator": "ge"}, {"budget": 0.49, "threshold": -10.585382461547852, "operator": "ge"}, {"budget": 0.5, "threshold": -10.618694305419922, "operator": "ge"}, {"budget": 0.51, "threshold": -10.695332527160645, "operator": "ge"}, {"budget": 0.52, "threshold": -10.796576499938965, "operator": "ge"}, {"budget": 0.53, "threshold": -10.841650009155273, "operator": "ge"}, {"budget": 0.54, "threshold": -10.913297653198242, "operator": "ge"}, {"budget": 0.55, "threshold": -10.961624145507812, "operator": "ge"}, {"budget": 0.56, "threshold": -11.040328025817871, "operator": "ge"}, {"budget": 0.57, "threshold": -11.102496147155762, "operator": "ge"}, {"budget": 0.58, "threshold": -11.220667839050293, "operator": "ge"}, {"budget": 0.59, "threshold": -11.29136848449707, "operator": "ge"}, {"budget": 0.6, "threshold": -11.354547500610352, "operator": "ge"}, {"budget": 0.61, "threshold": -11.46363639831543, "operator": "ge"}, {"budget": 0.62, "threshold": -11.511838912963867, "operator": "ge"}, {"budget": 0.63, "threshold": -11.576386451721191, "operator": "ge"}, {"budget": 0.64, "threshold": -11.647958755493164, "operator": "ge"}, {"budget": 0.65, "threshold": -11.728675842285156, "operator": "ge"}, {"budget": 0.66, "threshold": -11.830423355102539, "operator": "ge"}, {"budget": 0.67, "threshold": -11.90757942199707, "operator": "ge"}, {"budget": 0.68, "threshold": -11.986227035522461, "operator": "ge"}, {"budget": 0.69, "threshold": -12.039335250854492, "operator": "ge"}, {"budget": 0.7, "threshold": -12.099372863769531, "operator": "ge"}, {"budget": 0.71, "threshold": -12.190771102905273, "operator": "ge"}, {"budget": 0.72, "threshold": -12.249063491821289, "operator": "ge"}, {"budget": 0.73, "threshold": -12.305184364318848, "operator": "ge"}, {"budget": 0.74, "threshold": -12.379453659057617, "operator": "ge"}, {"budget": 0.75, "threshold": -12.456527709960938, "operator": "ge"}, {"budget": 0.76, "threshold": -12.515621185302734, "operator": "ge"}, {"budget": 0.77, "threshold": -12.592628479003906, "operator": "ge"}, {"budget": 0.78, "threshold": -12.6636962890625, "operator": "ge"}, {"budget": 0.79, "threshold": -12.721109390258789, "operator": "ge"}, {"budget": 0.8, "threshold": -12.77702522277832, "operator": "ge"}, {"budget": 0.81, "threshold": -12.850393295288086, "operator": "ge"}, {"budget": 0.82, "threshold": -12.938549995422363, "operator": "ge"}, {"budget": 0.83, "threshold": -12.986394882202148, "operator": "ge"}, {"budget": 0.84, "threshold": -13.080953598022461, "operator": "ge"}, {"budget": 0.85, "threshold": -13.151510238647461, "operator": "ge"}, {"budget": 0.86, "threshold": -13.226458549499512, "operator": "ge"}, {"budget": 0.87, "threshold": -13.289103507995605, "operator": "ge"}, {"budget": 0.88, "threshold": -13.387083053588867, "operator": "ge"}, {"budget": 0.89, "threshold": -13.459146499633789, "operator": "ge"}, {"budget": 0.9, "threshold": -13.609857559204102, "operator": "ge"}, {"budget": 0.91, "threshold": -13.70678997039795, "operator": "ge"}, {"budget": 0.92, "threshold": -13.805699348449707, "operator": "ge"}, {"budget": 0.93, "threshold": -13.905977249145508, "operator": "ge"}, {"budget": 0.94, "threshold": -14.121179580688477, "operator": "ge"}, {"budget": 0.95, "threshold": -14.24481201171875, "operator": "ge"}, {"budget": 0.96, "threshold": -14.413885116577148, "operator": "ge"}, {"budget": 0.97, "threshold": -14.593155860900879, "operator": "ge"}, {"budget": 0.98, "threshold": -14.739706039428711, "operator": "ge"}, {"budget": 0.99, "threshold": -14.916051864624023, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.25, 97.83333333333334, 97.25, 96.66666666666667, 96.08333333333333, 95.58333333333333, 95.16666666666667, 94.5, 94.0, 93.5, 92.83333333333333, 92.08333333333334, 91.58333333333333, 90.91666666666667, 90.25, 89.58333333333334, 88.66666666666667, 88.08333333333334, 87.5, 86.66666666666667, 86.25, 85.83333333333334, 85.41666666666667, 85.16666666666667, 85.0, 84.75, 84.33333333333334, 84.08333333333333, 83.66666666666667, 83.16666666666667, 82.83333333333334, 82.33333333333334, 81.91666666666667, 81.75, 81.41666666666667, 81.08333333333333, 80.58333333333334, 80.08333333333333, 79.83333333333333, 79.41666666666667, 78.75, 78.08333333333334, 77.41666666666667, 77.0, 76.33333333333333, 75.5, 75.0, 74.16666666666667, 73.41666666666666, 72.41666666666666, 71.75, 70.91666666666667, 70.25, 69.66666666666667, 68.91666666666667, 68.08333333333334, 67.33333333333333, 66.41666666666667, 65.66666666666667, 64.83333333333333, 63.916666666666664, 63.0, 62.16666666666666, 61.33333333333333, 60.33333333333333, 59.41666666666667, 58.41666666666667, 57.49999999999999, 56.75, 55.75, 54.75, 53.91666666666667, 53.0, 52.08333333333333, 51.16666666666667, 50.16666666666667, 49.41666666666667, 48.66666666666667, 47.75, 46.833333333333336, 46.166666666666664, 45.25, 44.25, 43.25, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.583333333333336, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 3, 3, 3, 3, 3, 4, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 7, 12, 19, 26, 33, 40, 46, 54, 61, 67, 75, 84, 90, 98, 106, 114, 125, 132, 139, 149, 154, 159, 164, 167, 169, 172, 177, 180, 185, 191, 195, 201, 206, 208, 212, 216, 222, 228, 231, 236, 244, 252, 260, 265, 273, 283, 289, 299, 308, 320, 328, 338, 346, 353, 362, 372, 381, 392, 401, 411, 422, 433, 443, 453, 465, 476, 488, 499, 508, 520, 532, 542, 553, 564, 575, 587, 596, 605, 616, 627, 635, 646, 658, 670, 680, 692, 704, 716, 728, 740, 750, 762, 774, 786, 797, 809, 821, 832, 844, 856], "exact_score": [98.58333333333333, 98.25, 97.83333333333334, 97.25, 96.66666666666667, 96.08333333333333, 95.58333333333333, 95.16666666666667, 94.5, 94.0, 93.5, 92.83333333333333, 92.08333333333334, 91.58333333333333, 90.91666666666667, 90.25, 89.58333333333334, 88.66666666666667, 88.08333333333334, 87.5, 86.66666666666667, 86.25, 85.83333333333334, 85.41666666666667, 85.16666666666667, 85.0, 84.75, 84.33333333333334, 84.08333333333333, 83.66666666666667, 83.16666666666667, 82.83333333333334, 82.33333333333334, 81.91666666666667, 81.75, 81.41666666666667, 81.08333333333333, 80.58333333333334, 80.08333333333333, 79.83333333333333, 79.41666666666667, 78.75, 78.08333333333334, 77.41666666666667, 77.0, 76.33333333333333, 75.5, 75.0, 74.16666666666667, 73.41666666666666, 72.41666666666666, 71.75, 70.91666666666667, 70.25, 69.66666666666667, 68.91666666666667, 68.08333333333334, 67.25, 66.41666666666667, 65.66666666666667, 64.83333333333333, 63.916666666666664, 63.0, 62.16666666666666, 61.33333333333333, 60.33333333333333, 59.41666666666667, 58.41666666666667, 57.49999999999999, 56.666666666666664, 55.75, 54.75, 53.91666666666667, 53.0, 52.08333333333333, 51.16666666666667, 50.16666666666667, 49.41666666666667, 48.66666666666667, 47.75, 46.833333333333336, 46.166666666666664, 45.166666666666664, 44.25, 43.25, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.583333333333336, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 68.39333333333332, "threshold_area100": 68.39583333333333, "threshold_area25": 91.65833333333333}, "test": {"score": [81.429064549534, 80.94580600621333, 79.996548153262, 78.70210562651019, 78.02899551259924, 76.9244045564377, 74.87055574732481, 73.558853986883, 72.55781843286158, 71.41870900932, 70.6420434932689, 69.33034173282707, 68.17397307559546, 66.91404901622369, 65.51605108733173, 64.39420089748016, 63.738350017259236, 63.15153607179842, 61.839834311356576, 60.994131860545394, 60.44183638246462, 59.630652399033494, 59.04383845357266, 58.7849499482223, 58.439765274421816, 57.852951328961, 57.317915084570245, 56.98998964445978, 56.43769416637902, 56.040731791508456, 55.79910251984812, 55.57473248187781, 55.212288574387294, 55.07421470486711, 54.918881601656885, 54.71177079737659, 54.48740075940628, 54.29754918881602, 54.09043838453573, 53.693476009665176, 53.48636520538488, 53.21021746634449, 52.778736624093895, 52.05384880911288, 51.58784949948223, 51.27718329306179, 51.070072488781506, 50.48325854332067, 49.82740766309976, 49.27511218501899, 49.22333448394892, 48.87814981014843, 48.4294097342078, 48.17052122885744, 47.84259578874698, 47.53192958232655, 47.10044874007595, 46.80704176734553, 46.341042457714885, 46.03037625129445, 45.857783914394204, 45.18467380048326, 44.97756299620297, 44.68415602347256, 44.21815671384191, 43.872972040041425, 43.372454263030725, 42.9237141870901, 42.49223334483949, 42.23334483948913, 41.905419399378665, 41.02519848118744, 40.54193993786676, 40.075940628236104, 39.62720055229548, 39.17846047635486, 38.781498101484296, 38.12564722126338, 37.60787021056265, 37.17638936831205, 36.7794269934415, 36.227131515360725, 35.58853986882983, 35.29513289609941, 34.829133586468764, 34.39765274421816, 33.914394200897476, 33.482913358646876, 32.80980324473594, 32.58543320676562, 31.912323092854677, 31.411805315843978, 31.01484294097342, 30.617880566102862, 29.685881946841562, 29.168104936140836, 28.546772523299968, 28.063513979979295, 27.614773904038657, 27.114256127027957, 26.561960648947192], "calls": [0.0, 3.5208836727649295, 7.4387297204004135, 12.27131515360718, 14.60131170176044, 18.691750086296167, 24.887814981014845, 28.460476354849845, 31.066620642043492, 33.707283396617186, 35.43320676561961, 37.6423886779427, 39.799792889195714, 41.905419399378665, 44.20089748015188, 46.565412495685194, 47.894373489817056, 48.9299275112185, 50.77666551605109, 52.08836727649292, 53.072143596824304, 54.48740075940628, 55.41939937866759, 56.16154642733863, 56.83465654124957, 57.749395926820846, 58.92302381774249, 59.578874697963414, 60.648947186744905, 61.66724197445633, 62.2885743872972, 62.96168450120815, 63.94546082153953, 64.34242319641008, 64.70486710390058, 65.34345875043148, 65.94753192958233, 66.51708664135313, 67.138419054194, 68.12219537452538, 68.6399723852261, 69.05419399378667, 69.86537797721782, 70.86641353123922, 71.55678287884018, 72.143596824301, 72.48878149810149, 73.31722471522264, 74.16292716603382, 74.9223334483949, 75.00862961684501, 75.40559199171557, 76.0787021056265, 76.38936831204694, 76.78633068691751, 77.11425612702796, 77.66655160510874, 78.13255091473938, 78.77114256127028, 79.08180876769072, 79.27166033828098, 80.10010355540214, 80.3589920607525, 80.65239903348291, 81.17017604418363, 81.58439765274423, 82.15395236451502, 82.67172937521575, 83.17224715222645, 83.46565412495686, 83.82809803244736, 84.72557818432863, 85.27787366240939, 85.76113220573006, 86.24439074905074, 86.76216775975146, 87.17638936831204, 87.849499482223, 88.38453572661373, 88.88505350362443, 89.31653434587504, 89.86882982395582, 90.52468070417673, 90.8526061442872, 91.33586468760787, 91.80186399723851, 92.31964100793924, 92.76838108387989, 93.44149119779081, 93.66586123576113, 94.39074905074214, 94.90852606144287, 95.3572661373835, 95.78874697963411, 96.77252329996548, 97.30755954435622, 97.96341042457715, 98.46392820158785, 98.91266827752848, 99.41318605453918, 100.0], "rescued": [0, 31, 57, 78, 93, 116, 141, 159, 167, 177, 185, 191, 197, 200, 206, 216, 219, 219, 221, 222, 222, 222, 222, 223, 225, 225, 225, 226, 226, 226, 226, 226, 226, 226, 226, 226, 226, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 228, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 59, 140, 236, 290, 377, 521, 615, 681, 757, 810, 892, 965, 1041, 1128, 1203, 1244, 1278, 1356, 1406, 1438, 1485, 1519, 1535, 1557, 1591, 1622, 1642, 1674, 1697, 1711, 1724, 1745, 1753, 1762, 1774, 1787, 1799, 1811, 1834, 1846, 1862, 1887, 1929, 1956, 1974, 1986, 2020, 2058, 2090, 2093, 2113, 2139, 2154, 2173, 2191, 2216, 2234, 2262, 2280, 2290, 2329, 2341, 2358, 2385, 2405, 2434, 2460, 2485, 2500, 2519, 2570, 2598, 2625, 2651, 2677, 2700, 2738, 2768, 2795, 2818, 2850, 2887, 2904, 2931, 2956, 2984, 3009, 3048, 3061, 3100, 3129, 3153, 3176, 3230, 3260, 3296, 3324, 3350, 3379, 3411], "exact_score": [81.429064549534, 81.27373144632378, 81.10113910942354, 80.98032447359338, 81.03210217466345, 80.73869520193303, 80.4970659302727, 80.08284432171212, 79.82395581636176, 79.59958577839144, 79.27166033828098, 78.96099413186055, 78.78840179496031, 78.54677252329996, 78.18432861580946, 77.92544011045909, 77.83914394200899, 77.47670003451846, 77.14877459440801, 76.82084915429755, 76.47566448049706, 76.14773904038661, 75.75077666551606, 75.37107352433551, 75.16396272005524, 74.85329651363479, 74.47359337245426, 74.14566793234381, 73.81774249223335, 73.33448394891266, 72.98929927511219, 72.5750776665516, 72.09181912323092, 71.67759751467035, 71.24611667241975, 70.76285812909907, 70.43493268898861, 69.7790818087677, 69.10597169485675, 68.62271315153608, 68.0704176734553, 67.51812219537453, 66.84501208146358, 66.25819813600276, 65.61960648947188, 65.15360717984122, 64.70486710390058, 64.18709009319986, 63.66931308249914, 63.099758370728345, 62.44390749050742, 61.66724197445633, 61.04590956161546, 60.493614083534695, 59.803244735933724, 59.31998619261305, 58.83672764929238, 58.33620987228167, 57.73213669313083, 57.248878149810146, 56.748360372799446, 56.265101829478766, 55.902657921988265, 55.55747324818778, 55.195029340697275, 54.81532619951675, 54.47014152571625, 54.15947531929582, 53.79703141180532, 53.26199516741457, 52.67518122195375, 51.96755264066275, 51.31170176044184, 50.742147048671036, 49.94822229892993, 49.22333448394892, 48.51570590265793, 47.63548498446669, 46.910597169485676, 46.11667241974457, 45.270969968933386, 44.373489817052125, 43.51052813255092, 42.64756644804971, 41.750086296168455, 40.83534691059717, 39.86882982395582, 38.97134967207456, 38.004832585433206, 37.072833966171906, 36.10631687953055, 35.157059026579226, 34.22506040731791, 33.258543320676566, 32.257507766655166, 31.32550914739386, 30.4280289955126, 29.461511908871252, 28.512254055919918, 27.52847773558854, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 60.462938282958255, "threshold_area100": 49.90861235761132, "threshold_area25": 68.4086986537798}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.3469364643096924, "operator": "ge"}, {"budget": 0.02, "threshold": -0.6042323112487793, "operator": "ge"}, {"budget": 0.03, "threshold": -0.9766881465911865, "operator": "ge"}, {"budget": 0.04, "threshold": -1.2009623050689697, "operator": "ge"}, {"budget": 0.05, "threshold": -1.5288630723953247, "operator": "ge"}, {"budget": 0.06, "threshold": -1.8335810899734497, "operator": "ge"}, {"budget": 0.07, "threshold": -2.1084043979644775, "operator": "ge"}, {"budget": 0.08, "threshold": -2.5442607402801514, "operator": "ge"}, {"budget": 0.09, "threshold": -2.8055386543273926, "operator": "ge"}, {"budget": 0.1, "threshold": -2.9910998344421387, "operator": "ge"}, {"budget": 0.11, "threshold": -3.206728458404541, "operator": "ge"}, {"budget": 0.12, "threshold": -3.438002586364746, "operator": "ge"}, {"budget": 0.13, "threshold": -3.739912509918213, "operator": "ge"}, {"budget": 0.14, "threshold": -4.0155768394470215, "operator": "ge"}, {"budget": 0.15, "threshold": -4.218285083770752, "operator": "ge"}, {"budget": 0.16, "threshold": -4.427378177642822, "operator": "ge"}, {"budget": 0.17, "threshold": -4.783847332000732, "operator": "ge"}, {"budget": 0.18, "threshold": -4.9623918533325195, "operator": "ge"}, {"budget": 0.19, "threshold": -5.363668918609619, "operator": "ge"}, {"budget": 0.2, "threshold": -5.8514838218688965, "operator": "ge"}, {"budget": 0.21, "threshold": -6.110718727111816, "operator": "ge"}, {"budget": 0.22, "threshold": -6.334627151489258, "operator": "ge"}, {"budget": 0.23, "threshold": -6.550974369049072, "operator": "ge"}, {"budget": 0.24, "threshold": -6.763053894042969, "operator": "ge"}, {"budget": 0.25, "threshold": -6.852327346801758, "operator": "ge"}, {"budget": 0.26, "threshold": -7.00369930267334, "operator": "ge"}, {"budget": 0.27, "threshold": -7.128650665283203, "operator": "ge"}, {"budget": 0.28, "threshold": -7.282268524169922, "operator": "ge"}, {"budget": 0.29, "threshold": -7.432925224304199, "operator": "ge"}, {"budget": 0.3, "threshold": -7.583016395568848, "operator": "ge"}, {"budget": 0.31, "threshold": -7.710378170013428, "operator": "ge"}, {"budget": 0.32, "threshold": -7.810883045196533, "operator": "ge"}, {"budget": 0.33, "threshold": -7.8950371742248535, "operator": "ge"}, {"budget": 0.34, "threshold": -7.996467590332031, "operator": "ge"}, {"budget": 0.35, "threshold": -8.059907913208008, "operator": "ge"}, {"budget": 0.36, "threshold": -8.195082664489746, "operator": "ge"}, {"budget": 0.37, "threshold": -8.282519340515137, "operator": "ge"}, {"budget": 0.38, "threshold": -8.397651672363281, "operator": "ge"}, {"budget": 0.39, "threshold": -8.475408554077148, "operator": "ge"}, {"budget": 0.4, "threshold": -8.577108383178711, "operator": "ge"}, {"budget": 0.41, "threshold": -8.65925121307373, "operator": "ge"}, {"budget": 0.42, "threshold": -8.739517211914062, "operator": "ge"}, {"budget": 0.43, "threshold": -8.798608779907227, "operator": "ge"}, {"budget": 0.44, "threshold": -8.919940948486328, "operator": "ge"}, {"budget": 0.45, "threshold": -8.994279861450195, "operator": "ge"}, {"budget": 0.46, "threshold": -9.074981689453125, "operator": "ge"}, {"budget": 0.47, "threshold": -9.163585662841797, "operator": "ge"}, {"budget": 0.48, "threshold": -9.221552848815918, "operator": "ge"}, {"budget": 0.49, "threshold": -9.328619003295898, "operator": "ge"}, {"budget": 0.5, "threshold": -9.440502166748047, "operator": "ge"}, {"budget": 0.51, "threshold": -9.519222259521484, "operator": "ge"}, {"budget": 0.52, "threshold": -9.606337547302246, "operator": "ge"}, {"budget": 0.53, "threshold": -9.653589248657227, "operator": "ge"}, {"budget": 0.54, "threshold": -9.799467086791992, "operator": "ge"}, {"budget": 0.55, "threshold": -9.877481460571289, "operator": "ge"}, {"budget": 0.56, "threshold": -9.932550430297852, "operator": "ge"}, {"budget": 0.57, "threshold": -10.011258125305176, "operator": "ge"}, {"budget": 0.58, "threshold": -10.066974639892578, "operator": "ge"}, {"budget": 0.59, "threshold": -10.14468765258789, "operator": "ge"}, {"budget": 0.6, "threshold": -10.200340270996094, "operator": "ge"}, {"budget": 0.61, "threshold": -10.265619277954102, "operator": "ge"}, {"budget": 0.62, "threshold": -10.373769760131836, "operator": "ge"}, {"budget": 0.63, "threshold": -10.443353652954102, "operator": "ge"}, {"budget": 0.64, "threshold": -10.54179573059082, "operator": "ge"}, {"budget": 0.65, "threshold": -10.589288711547852, "operator": "ge"}, {"budget": 0.66, "threshold": -10.675439834594727, "operator": "ge"}, {"budget": 0.67, "threshold": -10.72926139831543, "operator": "ge"}, {"budget": 0.68, "threshold": -10.823211669921875, "operator": "ge"}, {"budget": 0.69, "threshold": -10.900558471679688, "operator": "ge"}, {"budget": 0.7, "threshold": -10.96751594543457, "operator": "ge"}, {"budget": 0.71, "threshold": -11.046006202697754, "operator": "ge"}, {"budget": 0.72, "threshold": -11.120063781738281, "operator": "ge"}, {"budget": 0.73, "threshold": -11.18687629699707, "operator": "ge"}, {"budget": 0.74, "threshold": -11.251760482788086, "operator": "ge"}, {"budget": 0.75, "threshold": -11.293464660644531, "operator": "ge"}, {"budget": 0.76, "threshold": -11.345220565795898, "operator": "ge"}, {"budget": 0.77, "threshold": -11.4285888671875, "operator": "ge"}, {"budget": 0.78, "threshold": -11.488786697387695, "operator": "ge"}, {"budget": 0.79, "threshold": -11.527498245239258, "operator": "ge"}, {"budget": 0.8, "threshold": -11.57530403137207, "operator": "ge"}, {"budget": 0.81, "threshold": -11.612988471984863, "operator": "ge"}, {"budget": 0.82, "threshold": -11.6696195602417, "operator": "ge"}, {"budget": 0.83, "threshold": -11.757308959960938, "operator": "ge"}, {"budget": 0.84, "threshold": -11.834697723388672, "operator": "ge"}, {"budget": 0.85, "threshold": -11.894087791442871, "operator": "ge"}, {"budget": 0.86, "threshold": -11.942621231079102, "operator": "ge"}, {"budget": 0.87, "threshold": -12.01115608215332, "operator": "ge"}, {"budget": 0.88, "threshold": -12.084212303161621, "operator": "ge"}, {"budget": 0.89, "threshold": -12.125463485717773, "operator": "ge"}, {"budget": 0.9, "threshold": -12.173398971557617, "operator": "ge"}, {"budget": 0.91, "threshold": -12.232025146484375, "operator": "ge"}, {"budget": 0.92, "threshold": -12.332958221435547, "operator": "ge"}, {"budget": 0.93, "threshold": -12.4004487991333, "operator": "ge"}, {"budget": 0.94, "threshold": -12.459961891174316, "operator": "ge"}, {"budget": 0.95, "threshold": -12.622629165649414, "operator": "ge"}, {"budget": 0.96, "threshold": -12.696675300598145, "operator": "ge"}, {"budget": 0.97, "threshold": -12.862691879272461, "operator": "ge"}, {"budget": 0.98, "threshold": -13.077056884765625, "operator": "ge"}, {"budget": 0.99, "threshold": -13.36268424987793, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.5, 97.91666666666666, 97.5, 96.66666666666667, 96.25, 95.58333333333333, 94.91666666666667, 94.58333333333333, 93.91666666666667, 93.33333333333333, 92.58333333333333, 91.83333333333333, 91.0, 90.33333333333333, 89.75, 89.0, 88.16666666666667, 87.5, 86.91666666666666, 86.08333333333333, 85.66666666666667, 85.25, 84.83333333333334, 84.25, 83.91666666666666, 83.5, 83.25, 82.91666666666666, 82.75, 82.33333333333334, 82.0, 81.41666666666667, 80.83333333333333, 80.5, 80.33333333333333, 79.91666666666667, 79.58333333333334, 79.16666666666667, 78.58333333333334, 78.08333333333334, 77.91666666666667, 77.33333333333333, 76.58333333333334, 75.91666666666667, 75.41666666666667, 74.66666666666667, 73.83333333333333, 73.33333333333334, 72.83333333333333, 72.08333333333333, 71.16666666666667, 70.5, 69.58333333333333, 68.83333333333333, 68.08333333333334, 67.41666666666667, 66.75, 65.91666666666667, 65.0, 64.25, 63.25000000000001, 62.33333333333333, 61.66666666666667, 60.66666666666667, 60.00000000000001, 59.08333333333333, 58.08333333333333, 57.08333333333333, 56.25, 55.333333333333336, 54.49999999999999, 53.666666666666664, 52.75000000000001, 51.916666666666664, 51.083333333333336, 50.16666666666667, 49.16666666666667, 48.25000000000001, 47.5, 46.58333333333333, 45.666666666666664, 44.833333333333336, 43.833333333333336, 42.91666666666667, 42.166666666666664, 41.333333333333336, 40.416666666666664, 39.5, 38.5, 37.5, 36.5, 35.5, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 3, 3, 3, 3, 3, 3, 3, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 4, 11, 16, 26, 31, 39, 47, 53, 61, 68, 77, 87, 97, 105, 112, 121, 131, 139, 146, 156, 161, 166, 171, 178, 182, 187, 190, 194, 196, 201, 205, 212, 219, 223, 225, 230, 234, 239, 246, 252, 254, 261, 270, 278, 284, 293, 303, 309, 315, 324, 335, 343, 354, 363, 372, 380, 388, 398, 409, 418, 430, 441, 449, 461, 469, 480, 492, 504, 514, 525, 535, 545, 556, 566, 576, 587, 599, 610, 619, 630, 641, 651, 663, 674, 683, 693, 704, 715, 727, 739, 751, 763, 774, 785, 797, 809, 820, 832, 844, 856], "exact_score": [98.58333333333333, 98.5, 97.91666666666666, 97.5, 96.66666666666667, 96.25, 95.58333333333333, 94.91666666666667, 94.58333333333333, 93.91666666666667, 93.33333333333333, 92.58333333333333, 91.83333333333333, 91.0, 90.33333333333333, 89.75, 89.0, 88.16666666666667, 87.5, 86.91666666666666, 86.08333333333333, 85.66666666666667, 85.25, 84.83333333333334, 84.25, 83.91666666666666, 83.5, 83.25, 82.91666666666666, 82.75, 82.33333333333334, 82.0, 81.41666666666667, 80.83333333333333, 80.5, 80.33333333333333, 79.91666666666667, 79.58333333333334, 79.16666666666667, 78.58333333333334, 78.08333333333334, 77.91666666666667, 77.33333333333333, 76.58333333333334, 75.91666666666667, 75.41666666666667, 74.66666666666667, 73.83333333333333, 73.33333333333334, 72.83333333333333, 72.08333333333333, 71.16666666666667, 70.5, 69.58333333333333, 68.83333333333333, 68.08333333333334, 67.41666666666667, 66.75, 65.91666666666667, 65.0, 64.25, 63.25000000000001, 62.33333333333333, 61.66666666666667, 60.66666666666667, 60.00000000000001, 59.08333333333333, 58.08333333333333, 57.08333333333333, 56.166666666666664, 55.333333333333336, 54.49999999999999, 53.666666666666664, 52.75000000000001, 51.916666666666664, 51.083333333333336, 50.16666666666667, 49.16666666666667, 48.25000000000001, 47.5, 46.58333333333333, 45.666666666666664, 44.75, 43.833333333333336, 42.91666666666667, 42.166666666666664, 41.333333333333336, 40.416666666666664, 39.5, 38.5, 37.5, 36.5, 35.5, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 67.91666666666667, "threshold_area100": 67.91833333333332, "threshold_area25": 91.33333333333334}, "test": {"score": [81.429064549534, 81.0666206420435, 80.30721435968243, 78.58129099068002, 77.47670003451846, 76.04418363824647, 74.68070417673457, 73.26544701415257, 71.21159820503969, 69.69278564031757, 68.58819468415602, 67.88056610286503, 66.81049361408353, 65.41249568519159, 64.23886779426994, 63.2896099413186, 62.616499827407665, 61.51190887124611, 60.87331722471523, 59.90680013807387, 58.54332067656196, 57.870210562651025, 57.3524335519503, 56.88643424231964, 56.29962029685882, 56.12702795995859, 55.91991715567829, 55.64376941663791, 55.33310321021747, 54.98791853641698, 54.59095616154643, 54.193993786675875, 53.93510528132551, 53.727994477045215, 53.503624439074905, 53.26199516741457, 52.83051432516397, 52.6406627545737, 52.20918191232309, 52.019330341732825, 51.74318260269244, 51.449775629962026, 51.1391094235416, 50.84570245081119, 50.46599930963065, 50.27614773904039, 50.0, 49.79288919571972, 49.58577839143942, 49.22333448394892, 48.602002071108046, 48.22229892992752, 47.859855022437, 47.60096651708665, 46.94511563686572, 46.59993096306524, 46.289264756644805, 45.96133931653435, 45.63341387642389, 45.13289609941319, 44.75319295823266, 44.56334138764239, 44.06282361063169, 43.82119433897135, 43.32067656196065, 43.13082499137039, 42.73386261649983, 42.45771487745944, 41.69830859509838, 41.180531584397656, 40.749050742147055, 40.283051432516395, 39.59268208491543, 39.29927511218502, 38.62616499827408, 38.47083189506386, 38.09112875388333, 37.19364860200207, 36.744908526061444, 36.382464618570935, 36.0717984121505, 35.70935450466, 35.2088367276493, 34.38039351052814, 33.79357956506731, 33.344839489126684, 32.982395581636176, 32.53365550569555, 31.843286158094582, 31.567138419054196, 31.135657576803588, 30.73869520193303, 30.100103555402146, 29.77217811529168, 29.461511908871252, 28.719364860200216, 28.391439420089746, 27.718329306178813, 27.131515360717984, 26.820849154297555, 26.561960648947192], "calls": [0.0, 3.8488091128753883, 7.266137383500173, 13.444943044528824, 16.22367966862271, 20.711080428028996, 24.801518812564723, 28.49499482222989, 33.22402485329651, 36.106316879530546, 37.93579565067311, 39.67897825336555, 41.50845702450811, 43.76941663790127, 45.857783914394204, 47.30755954435623, 48.3603727994477, 50.05177770107008, 51.05281325509148, 52.6406627545737, 54.67725232999655, 55.850880220918185, 56.76561960648947, 57.71487745944081, 58.95754228512254, 59.38902312737314, 59.99309630652399, 60.54539178460476, 61.201242664825685, 62.046945115636866, 62.89264756644805, 63.686572316189164, 64.46323783224025, 65.08457024508112, 65.68864342423197, 66.15464273386262, 67.05212288574387, 67.51812219537453, 68.48463928201588, 69.01967552640663, 69.76182257507767, 70.34863652053849, 70.90093199861927, 71.2806351397998, 72.143596824301, 72.50604073179151, 72.98929927511219, 73.38626164998274, 73.73144632378322, 74.33551950293406, 75.14670348636521, 75.64722126337591, 76.2167759751467, 76.51018294787711, 77.40766309975837, 77.856403175699, 78.28788401794961, 78.77114256127028, 79.15084570245081, 79.7204004142216, 80.2209181912323, 80.4280289955126, 81.01484294097342, 81.3600276147739, 81.91232309285468, 82.11943389713497, 82.58543320676563, 82.94787711425613, 83.77632033137729, 84.34587504314808, 84.88091128753882, 85.4332067656196, 86.14083534691059, 86.53779772178115, 87.22816706938212, 87.43527787366241, 87.849499482223, 88.78149810148429, 89.28201587849499, 89.67897825336554, 90.02416292716603, 90.40386606834656, 90.92164307904729, 91.83638246461857, 92.45771487745944, 92.94097342078011, 93.30341732827063, 93.76941663790129, 94.47704521919226, 94.77045219192269, 95.20193303417328, 95.59889540904383, 96.27200552295479, 96.61719019675526, 96.9278564031757, 97.68726268553675, 98.0669658267173, 98.77459440800828, 99.41318605453918, 99.72385226095962, 100.0], "rescued": [0, 26, 48, 78, 86, 109, 131, 151, 173, 184, 191, 200, 206, 207, 214, 216, 217, 219, 220, 222, 223, 223, 223, 224, 224, 224, 226, 226, 226, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 228, 228, 228, 229, 229, 229, 230, 230, 230, 230, 230, 230, 230, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232], "harmed": [0, 47, 113, 243, 315, 421, 522, 624, 765, 864, 935, 985, 1053, 1135, 1210, 1267, 1307, 1373, 1411, 1469, 1549, 1588, 1618, 1646, 1680, 1690, 1704, 1720, 1738, 1759, 1782, 1805, 1820, 1832, 1845, 1859, 1884, 1895, 1920, 1931, 1947, 1964, 1982, 1999, 2021, 2032, 2048, 2060, 2072, 2093, 2129, 2151, 2172, 2187, 2225, 2245, 2263, 2282, 2301, 2330, 2353, 2364, 2393, 2408, 2437, 2448, 2472, 2488, 2532, 2562, 2587, 2614, 2654, 2672, 2711, 2720, 2742, 2794, 2820, 2841, 2859, 2880, 2909, 2957, 2991, 3017, 3038, 3064, 3104, 3120, 3145, 3168, 3205, 3224, 3242, 3285, 3305, 3344, 3378, 3396, 3411], "exact_score": [81.429064549534, 81.46358301691406, 81.23921297894374, 81.18743527787366, 81.01484294097342, 80.7904729030031, 80.63513979979288, 80.35899206075251, 80.01380738695202, 79.68588194684156, 79.39247497411115, 79.20262340352089, 79.04729030031066, 78.68484639282016, 78.42595788746979, 77.99447704521918, 77.52847773558854, 77.20055229547809, 76.95892302381775, 76.63099758370728, 76.33759061097687, 75.88885053503624, 75.66448049706594, 75.25025888850536, 74.97411114946496, 74.64618570935451, 74.18018639972385, 73.81774249223335, 73.40352088367277, 73.04107697618227, 72.50604073179151, 72.12633759061097, 71.69485674836038, 71.21159820503969, 70.72833966171902, 70.22782188470832, 69.76182257507767, 69.15774939592683, 68.60545391784605, 68.1394546082154, 67.72523299965482, 67.10390058681395, 66.46530894028305, 65.82671729375215, 65.27442181567137, 64.82568173973075, 64.15257162581982, 63.4967207455989, 62.85812909906801, 62.15050051777702, 61.52916810493614, 60.925094925785295, 60.2865032792544, 59.630652399033494, 58.92302381774249, 58.33620987228167, 57.818432861580945, 57.21435968243011, 56.748360372799446, 56.29962029685882, 55.91991715567829, 55.41939937866759, 55.02243700379703, 54.52191922678633, 54.073179150845704, 53.74525371073524, 53.348291335864694, 52.847773558853994, 52.46807041767345, 52.019330341732825, 51.656886434242324, 51.08733172247153, 50.569554711770806, 50.0, 49.395926820849155, 48.72281670693822, 47.980669658267175, 47.23852260959613, 46.51363479461512, 45.771487745944086, 44.89126682775285, 44.080082844321716, 43.251639627200554, 42.42319641007939, 41.542975491888164, 40.680013807386956, 39.74801518812565, 38.86779426993442, 37.953054884363134, 37.021056265101834, 36.10631687953055, 35.157059026579226, 34.24231964100794, 33.29306178805661, 32.30928546772523, 31.342768381083886, 30.35899206075251, 29.39247497411115, 28.460476354849852, 27.52847773558854, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 60.06935685911259, "threshold_area100": 48.932775284777364, "threshold_area25": 67.04867103900588}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.66666666666667, "calls": 1.0}, "test": {"score": 81.39454608215395, "calls": 1.3634794615119088}}, "old_normalized_interior": {"val": {"score": 98.66666666666667, "calls": 1.0}, "test": {"score": 81.39454608215395, "calls": 1.3634794615119088}}, "old_normalized_endpoints": {"val": {"score": 98.66666666666667, "calls": 1.0}, "test": {"score": 81.39454608215395, "calls": 1.3634794615119088}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.0, 97.66666666666667, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.08333333333333, 96.91666666666666, 96.5, 96.16666666666667, 95.83333333333334, 95.41666666666667, 95.16666666666667, 94.91666666666667, 94.41666666666667, 94.08333333333333, 93.58333333333333, 92.91666666666667, 92.16666666666666, 91.66666666666666, 90.83333333333333, 90.08333333333334, 89.25, 88.33333333333333, 87.58333333333333, 86.83333333333334, 86.0, 85.08333333333333, 84.25, 83.58333333333333, 82.83333333333334, 81.91666666666667, 81.08333333333333, 80.33333333333333, 79.5, 78.66666666666666, 77.83333333333333, 77.08333333333334, 76.33333333333333, 75.5, 74.5, 73.66666666666667, 72.66666666666667, 71.75, 70.91666666666667, 70.08333333333334, 69.25, 68.16666666666666, 67.16666666666666, 66.25000000000001, 65.25000000000001, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.666666666666664, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.91666666666667, 54.0, 53.0, 52.08333333333333, 51.083333333333336, 50.16666666666667, 49.16666666666667, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.166666666666664, 42.25, 41.25, 40.25, 39.24999999999999, 38.33333333333333, 37.416666666666664, 36.416666666666664, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.0, 97.66666666666667, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.08333333333333, 96.91666666666666, 96.5, 96.16666666666667, 95.83333333333334, 95.41666666666667, 95.16666666666667, 94.91666666666667, 94.41666666666667, 94.08333333333333, 93.58333333333333, 92.91666666666667, 92.16666666666666, 91.66666666666666, 90.83333333333333, 90.08333333333334, 89.25, 88.33333333333333, 87.58333333333333, 86.83333333333334, 86.0, 85.08333333333333, 84.25, 83.58333333333333, 82.75, 81.91666666666667, 81.08333333333333, 80.33333333333333, 79.5, 78.66666666666666, 77.83333333333333, 77.08333333333334, 76.33333333333333, 75.5, 74.5, 73.66666666666667, 72.66666666666667, 71.75, 70.91666666666667, 70.08333333333334, 69.16666666666667, 68.16666666666666, 67.16666666666666, 66.25000000000001, 65.25000000000001, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.666666666666664, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.91666666666667, 54.0, 53.0, 52.08333333333333, 51.083333333333336, 50.16666666666667, 49.16666666666667, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.166666666666664, 42.25, 41.25, 40.25, 39.24999999999999, 38.33333333333333, 37.416666666666664, 36.416666666666664, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.45000000000002, "exact_area100": 71.67333333333333}, "test": {"score": [81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.32550914739386, 81.23921297894374, 81.1356575768036, 81.11839834311357, 81.03210217466345, 80.84225060407319, 80.70417673455299, 80.53158439765275, 80.32447359337246, 80.11736278909216, 79.68588194684156, 79.23714187090093, 78.89195719710045, 78.51225405591991, 77.83914394200899, 77.09699689333793, 76.8553676216776, 75.38833275802554, 74.81877804625475, 74.09389023127373, 73.67966862271315, 72.7821884708319, 71.95374525371074, 71.17707973765965, 69.74456334138765, 69.29582326544703, 68.46738004832585, 67.79426993441491, 66.75871591301346, 65.91301346220229, 65.4987918536417, 65.11908871246118, 64.48049706593028, 63.65205384880912, 63.13427683810839, 62.02968588194684, 61.32205730065585, 60.71798412150501, 59.82050396962375, 58.81946841560235, 58.23265447014153, 57.26613738350017, 56.454953400069044, 55.678287884017955, 55.31584397652745, 54.48740075940628, 53.71073524335519, 53.227476700034515, 52.53710735243355, 52.226441146013116, 51.363479461511915, 50.58681394546083, 49.620296858819465, 49.119779081808765, 48.4984466689679, 47.79081808767691, 47.25578184328616, 46.23748705557473, 46.03037625129445, 45.75422851225406, 45.23645150155333, 44.21815671384191, 43.8557128063514, 43.372454263030725, 42.751121850189854, 42.31964100793925, 41.56023472557819, 41.042457714877465, 39.98964445978599, 39.26475664480497, 38.26372109078357, 37.83224024853297, 37.159130134622025, 36.34794615119089, 35.761132205730064, 34.89817052122886, 34.22506040731791, 33.53469105971695, 32.982395581636176, 32.68898860890577, 31.72247152226441, 31.11839834311356, 30.531584397652743, 29.720400414221615, 29.237141870900928, 28.719364860200216, 28.098032447359344, 27.683810838798763, 27.06247842595789, 26.561960648947192], "calls": [0.0, 1.3634794615119088, 1.9675526406627546, 2.2609596133931653, 2.8132550914739385, 3.1066620642043494, 4.176734552985847, 5.591991715567829, 7.00724887814981, 8.025543665861235, 9.147393855712806, 10.40731791508457, 12.133241284086987, 13.462202278218847, 14.169830859509839, 15.447014152571626, 17.138419054193992, 19.15774939592682, 20.141525716258197, 20.987228167069382, 21.763893683120468, 23.62789092164308, 25.008629616845013, 26.078702105626512, 27.32136693130825, 29.27166033828098, 30.9285467725233, 32.22298929927511, 35.1397997928892, 36.29616845012082, 37.97031411805316, 38.88505350362444, 40.85260614428719, 42.30238177424923, 43.872972040041425, 46.427338626164996, 47.39385571280635, 48.74007594062824, 49.913703831549874, 51.76044183638246, 53.03762512944425, 53.86606834656541, 54.60821539523645, 55.47117707973766, 56.61028650327925, 57.42147048671039, 59.16465308940283, 60.04487400759406, 60.90783569209527, 62.16775975146703, 63.634794615119084, 64.51501553331032, 65.77493959268207, 66.82775284777355, 67.88056610286503, 68.32930617880567, 69.26130479806697, 70.34863652053849, 70.88367276492924, 71.67759751467034, 72.04004142216085, 73.04107697618227, 74.04211253020367, 75.25025888850536, 75.87159130134621, 76.56196064894719, 77.33862616499827, 77.94269934414912, 79.08180876769072, 79.30617880566103, 79.59958577839143, 80.13462202278218, 81.25647221263375, 81.65343458750431, 82.20573006558509, 82.861580945806, 83.3620987228167, 84.12150500517777, 84.7083189506386, 85.79565067311012, 86.55505695547117, 87.62512944425268, 88.09112875388332, 88.86779426993442, 89.7134967207456, 90.38660683465655, 91.31860545391784, 92.07801173627891, 92.82015878494995, 93.38971349672074, 93.70037970314118, 94.70141525716258, 95.32274767000345, 95.94408008284432, 96.77252329996548, 97.27304107697618, 97.82533655505695, 98.44666896789782, 98.8608905764584, 99.48222298929927, 100.0], "exact_score": [81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.25647221263375, 81.18743527787366, 81.11839834311357, 81.0666206420435, 81.01484294097342, 80.84225060407319, 80.70417673455299, 80.66965826717293, 80.60062133241284, 80.32447359337246, 79.96202968588196, 79.7894373489817, 79.54780807732136, 79.23714187090093, 78.90921643079048, 78.56403175698999, 78.32240248532966, 77.92544011045909, 77.51121850189851, 77.11425612702796, 76.9244045564377, 76.52744218156714, 75.99240593717639, 75.44011045909562, 74.99137038315499, 74.62892647566449, 74.0766309975837, 73.59337245426303, 73.09285467725233, 72.69589230238178, 72.143596824301, 71.57404211253021, 71.09078356920952, 70.59026579219882, 70.03797031411806, 69.48567483603728, 68.95063859164652, 68.32930617880567, 67.75975146703487, 67.19019675526407, 66.60338280980325, 65.94753192958233, 65.44701415257163, 64.87745944080083, 64.08353469105973, 63.375906109768735, 62.7890921643079, 62.115982050396966, 61.37383500172593, 60.68346565412496, 59.97583707283397, 59.285467725233, 58.62961684501209, 57.818432861580945, 57.05902657921988, 56.316879530548846, 55.591991715567836, 54.74628926475664, 53.96962374870555, 53.141180531584396, 52.26095961339317, 51.39799792889196, 50.62133241284087, 49.86192613047981, 48.9989644459786, 48.08422506040732, 47.204004142216085, 46.30652399033483, 45.3572661373835, 44.45978598550225, 43.56230583362099, 42.64756644804971, 41.69830859509838, 40.766309975837075, 39.81705212288574, 38.90231273731447, 37.93579565067311, 37.03831549879186, 36.10631687953055, 35.2088367276493, 34.29409734207801, 33.36209872281671, 32.41284086986538, 31.446323783224027, 30.479806696582678, 29.49603037625129, 28.546772523299968, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 80.711080428029, "exact_area100": 63.19187811561338}}, "old_normalized_endpoint_budget": 1}, {"seed": 2027, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.0, "validation_area100": 0.7156166666666667, "training_seconds": 3.31173656496685, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0061280992813408375, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0075284517370164394, "operator": "ge"}, {"budget": 0.03, "threshold": -0.00957958772778511, "operator": "ge"}, {"budget": 0.04, "threshold": -0.01133684255182743, "operator": "ge"}, {"budget": 0.05, "threshold": -0.012673596851527691, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01567986235022545, "operator": "ge"}, {"budget": 0.07, "threshold": -0.022006699815392494, "operator": "ge"}, {"budget": 0.08, "threshold": -0.028553253039717674, "operator": "ge"}, {"budget": 0.09, "threshold": -0.0374443382024765, "operator": "ge"}, {"budget": 0.1, "threshold": -0.04497264325618744, "operator": "ge"}, {"budget": 0.11, "threshold": -0.060270339250564575, "operator": "ge"}, {"budget": 0.12, "threshold": -0.0833895280957222, "operator": "ge"}, {"budget": 0.13, "threshold": -0.10451038926839828, "operator": "ge"}, {"budget": 0.14, "threshold": -0.12435751408338547, "operator": "ge"}, {"budget": 0.15, "threshold": -0.13938559591770172, "operator": "ge"}, {"budget": 0.16, "threshold": -0.14907951653003693, "operator": "ge"}, {"budget": 0.17, "threshold": -0.18306884169578552, "operator": "ge"}, {"budget": 0.18, "threshold": -0.2087668627500534, "operator": "ge"}, {"budget": 0.19, "threshold": -0.24214644730091095, "operator": "ge"}, {"budget": 0.2, "threshold": -0.2866281569004059, "operator": "ge"}, {"budget": 0.21, "threshold": -0.32901450991630554, "operator": "ge"}, {"budget": 0.22, "threshold": -0.36170750856399536, "operator": "ge"}, {"budget": 0.23, "threshold": -0.3933005928993225, "operator": "ge"}, {"budget": 0.24, "threshold": -0.4291616380214691, "operator": "ge"}, {"budget": 0.25, "threshold": -0.4785645604133606, "operator": "ge"}, {"budget": 0.26, "threshold": -0.5161387324333191, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5669921636581421, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6123486161231995, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6507254838943481, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6782835721969604, "operator": "ge"}, {"budget": 0.31, "threshold": -0.701304018497467, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7246511578559875, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7430075407028198, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7607767581939697, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7891996502876282, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7991636395454407, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8097347021102905, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8294148445129395, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8433073163032532, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8509774804115295, "operator": "ge"}, {"budget": 0.41, "threshold": -0.8616909384727478, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8751422166824341, "operator": "ge"}, {"budget": 0.43, "threshold": -0.886139988899231, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8923884034156799, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8956731557846069, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9055857062339783, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9144119620323181, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9213765263557434, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9294031858444214, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9386687874794006, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9422948360443115, "operator": "ge"}, {"budget": 0.52, "threshold": -0.946887731552124, "operator": "ge"}, {"budget": 0.53, "threshold": -0.949507474899292, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9517558217048645, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9543506503105164, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9571070075035095, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9595099687576294, "operator": "ge"}, {"budget": 0.58, "threshold": -0.960839033126831, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9630143642425537, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9656237959861755, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9691079258918762, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9701248407363892, "operator": "ge"}, {"budget": 0.63, "threshold": -0.97266685962677, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9753293991088867, "operator": "ge"}, {"budget": 0.65, "threshold": -0.976718008518219, "operator": "ge"}, {"budget": 0.66, "threshold": -0.978922426700592, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9798735976219177, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9814716577529907, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9828217625617981, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9844800233840942, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9852387309074402, "operator": "ge"}, {"budget": 0.72, "threshold": -0.98598712682724, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9868913888931274, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9874170422554016, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9884153604507446, "operator": "ge"}, {"budget": 0.76, "threshold": -0.989288866519928, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9898726940155029, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9910945892333984, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9918593764305115, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9925138354301453, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9928067922592163, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9935107827186584, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9938969016075134, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9945120811462402, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9950332641601562, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9954079389572144, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9957066774368286, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9960741400718689, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9965130090713501, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9969208836555481, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9970928430557251, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9973589181900024, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9976683259010315, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9978731870651245, "operator": "ge"}, {"budget": 0.95, "threshold": -0.998117208480835, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9982621669769287, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9984968900680542, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9987394213676453, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9989650249481201, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.25, 98.25, 98.16666666666667, 98.0, 97.91666666666666, 97.5, 97.25, 97.08333333333333, 96.75, 96.41666666666666, 96.5, 96.08333333333333, 95.41666666666667, 95.0, 94.83333333333334, 94.25, 93.5, 93.08333333333333, 92.58333333333333, 92.0, 91.25, 90.33333333333333, 89.58333333333334, 88.75, 88.16666666666667, 87.41666666666667, 86.91666666666666, 86.0, 85.16666666666667, 84.5, 83.58333333333333, 82.75, 81.83333333333334, 81.0, 80.0, 79.16666666666667, 78.33333333333333, 77.66666666666666, 76.91666666666667, 76.0, 75.08333333333333, 74.16666666666667, 73.25, 72.33333333333334, 71.41666666666666, 70.41666666666666, 69.58333333333333, 68.91666666666667, 67.91666666666667, 67.0, 66.08333333333334, 65.16666666666666, 64.25, 63.5, 62.5, 61.5, 60.5, 59.66666666666667, 58.666666666666664, 57.75, 56.666666666666664, 55.666666666666664, 54.666666666666664, 53.91666666666667, 52.916666666666664, 51.916666666666664, 50.916666666666664, 50.0, 49.083333333333336, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 4, 5, 7, 8, 13, 16, 18, 22, 26, 26, 31, 39, 44, 47, 54, 63, 69, 76, 83, 92, 103, 112, 122, 129, 138, 144, 155, 165, 173, 184, 194, 205, 215, 227, 237, 247, 256, 265, 276, 287, 298, 309, 320, 331, 343, 353, 362, 374, 385, 396, 407, 418, 427, 439, 451, 463, 473, 485, 496, 509, 521, 533, 542, 554, 566, 578, 589, 600, 612, 623, 635, 646, 659, 670, 682, 693, 704, 716, 727, 739, 751, 763, 775, 787, 799, 810, 822, 833, 844, 856], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.25, 98.25, 98.16666666666667, 98.0, 97.91666666666666, 97.5, 97.25, 97.08333333333333, 96.75, 96.41666666666666, 96.5, 96.08333333333333, 95.41666666666667, 95.0, 94.83333333333334, 94.25, 93.5, 93.08333333333333, 92.58333333333333, 92.0, 91.25, 90.33333333333333, 89.58333333333334, 88.75, 88.16666666666667, 87.41666666666667, 86.91666666666666, 86.0, 85.16666666666667, 84.5, 83.58333333333333, 82.66666666666667, 81.83333333333334, 81.0, 80.0, 79.16666666666667, 78.33333333333333, 77.66666666666666, 76.91666666666667, 76.0, 75.08333333333333, 74.16666666666667, 73.25, 72.33333333333334, 71.41666666666666, 70.41666666666666, 69.58333333333333, 68.83333333333333, 67.91666666666667, 67.0, 66.08333333333334, 65.16666666666666, 64.25, 63.5, 62.5, 61.5, 60.5, 59.66666666666667, 58.666666666666664, 57.666666666666664, 56.666666666666664, 55.666666666666664, 54.666666666666664, 53.91666666666667, 52.916666666666664, 51.916666666666664, 50.916666666666664, 50.0, 49.083333333333336, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.56166666666667, "threshold_area100": 71.56500000000001, "threshold_area25": 97.44999999999999}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.39454608215395, 81.429064549534, 81.429064549534, 81.36002761477391, 81.41180531584398, 81.30824991370383, 81.25647221263375, 81.10113910942354, 81.04936140835348, 80.82499137038316, 80.68691750086296, 80.44528822920263, 79.97928891957199, 79.72040041422161, 79.28891957197101, 78.84017949603039, 78.49499482222988, 77.82188470831896, 77.07973765964792, 76.18225750776666, 75.45736969278565, 74.7842595788747, 74.23196410079392, 73.558853986883, 73.09285467725233, 72.43700379703142, 71.60856057991025, 70.2105626510183, 69.79634104245773, 69.26130479806697, 67.91508457024509, 67.01760441836383, 66.5688643424232, 65.77493959268207, 64.5667932343804, 63.75560925094926, 63.18605453917846, 62.87538833275803, 61.75353814290645, 60.925094925785295, 60.303762512944424, 59.285467725233, 58.025543665861235, 57.33517431826027, 56.16154642733863, 55.712806351397994, 55.16051087331723, 54.694511563686575, 53.952364515015525, 53.227476700034515, 52.951328960994125, 52.41629271660339, 51.60510873317226, 50.58681394546083, 50.20711080428028, 49.27511218501899, 48.41215050051778, 47.94615119088713, 46.96237487055575, 46.51363479461512, 45.702450811183986, 44.97756299620297, 44.080082844321716, 43.665861235761135, 43.23438039351053, 42.7166033828098, 42.267863306869174, 41.750086296168455, 41.31860545391785, 40.95616154642734, 39.972385226095966, 39.282015878494995, 38.557128063513986, 38.31549879185364, 37.60787021056265, 37.12461166724197, 36.382464618570935, 35.5022437003797, 34.77735588539869, 34.5012081463583, 33.74180186399724, 32.72350707628581, 32.03313772868485, 31.687953054884368, 31.10113910942355, 30.46254746289265, 30.031066620642044, 29.444252675181225, 29.133586468760786, 28.61580945806006, 28.011736278909215, 27.33862616499828, 26.561960648947192], "calls": [0.0, 0.6731101139109423, 1.1045909561615463, 2.0365895754228514, 2.6406627545736967, 3.089402830514325, 3.917846047635485, 5.471177079737659, 6.765619606489472, 7.870210562651018, 8.715913013462202, 10.52813255091474, 12.15050051777701, 13.410424577148774, 14.60131170176044, 15.67138419054194, 16.17190196755264, 17.86330686917501, 18.74352778736624, 19.986192613047983, 22.195374525371072, 23.783224024853297, 24.887814981014845, 25.957887469796344, 27.338626164998274, 29.219882637210908, 30.669658267172938, 32.982395581636176, 35.01898515705902, 36.90024162927166, 38.298239558163615, 39.575422851225404, 40.88712461166724, 42.06075250258888, 43.54504659993096, 45.96133931653434, 46.89333793579565, 47.70452191922679, 49.98274076630997, 51.43251639627201, 52.17466344494305, 53.31377286848464, 54.918881601656885, 56.19606489471868, 57.093545046599935, 57.54228512254056, 59.11287538833275, 60.510873317224714, 61.52916810493614, 63.16879530548844, 64.94649637556093, 65.80945806006213, 67.19019675526407, 67.7079737659648, 68.3638246461857, 69.0024162927166, 69.98619261304798, 70.81463583016914, 71.15982050396963, 71.79841215050051, 72.71315153607179, 73.9212978943735, 74.31826026924404, 75.47462892647566, 76.51018294787711, 77.06247842595789, 78.13255091473938, 78.66758715913014, 79.4960303762513, 80.37625129444253, 81.41180531584398, 81.82602692440456, 82.34380393510529, 82.89609941318605, 83.39661719019675, 83.96617190196756, 84.5012081463583, 84.86365205384881, 85.88194684156024, 86.60683465654125, 87.38350017259233, 87.62512944425268, 88.40179496030376, 88.90231273731446, 89.7134967207456, 90.61097687262686, 91.38764238867795, 91.71556782878841, 92.50949257852952, 93.56230583362098, 94.26993441491199, 94.66689678978253, 95.28822920262341, 95.97859855022438, 96.427338626165, 97.0659302726959, 97.37659647911633, 97.94615119088712, 98.55022437003798, 99.22333448394892, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 1, 3, 6, 6, 6, 11, 12, 13, 15, 17, 20, 21, 26, 34, 47, 48, 51, 56, 65, 69, 81, 95, 107, 113, 114, 125, 128, 131, 136, 140, 142, 152, 156, 159, 162, 164, 168, 173, 174, 177, 186, 189, 196, 203, 205, 207, 207, 208, 210, 212, 212, 213, 214, 215, 218, 218, 220, 222, 223, 224, 224, 224, 225, 226, 226, 226, 226, 226, 226, 228, 228, 228, 228, 228, 228, 229, 229, 230, 230, 230, 230, 230, 230, 230, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 0, 0, 0, 0, 0, 1, 3, 5, 6, 6, 10, 12, 19, 23, 34, 39, 55, 64, 83, 118, 146, 172, 201, 226, 274, 321, 385, 441, 492, 530, 570, 608, 649, 700, 786, 814, 847, 935, 991, 1020, 1069, 1141, 1192, 1230, 1249, 1317, 1374, 1413, 1479, 1559, 1601, 1671, 1697, 1730, 1759, 1804, 1846, 1863, 1895, 1943, 2005, 2027, 2083, 2135, 2163, 2221, 2247, 2294, 2337, 2390, 2414, 2439, 2469, 2495, 2525, 2552, 2573, 2630, 2670, 2712, 2726, 2768, 2796, 2840, 2891, 2933, 2949, 2993, 3052, 3092, 3113, 3147, 3185, 3210, 3244, 3262, 3292, 3327, 3366, 3411], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.429064549534, 81.429064549534, 81.37728684846392, 81.37728684846392, 81.39454608215395, 81.32550914739386, 81.27373144632378, 81.23921297894374, 81.04936140835348, 80.9285467725233, 80.82499137038316, 80.60062133241284, 80.44528822920263, 80.18639972385226, 80.06558508802209, 79.89299275112185, 79.70314118053159, 79.25440110459095, 78.84017949603039, 78.59855022437004, 78.28788401794961, 77.92544011045909, 77.47670003451846, 76.9244045564377, 76.57921988263722, 76.18225750776666, 75.83707283396618, 75.45736969278565, 75.12944425267519, 74.73248187780463, 74.37003797031412, 73.86952019330342, 73.38626164998274, 73.04107697618227, 72.47152226441146, 71.93648602002072, 71.45322747670004, 70.79737659647913, 70.19330341732827, 69.7100448740076, 69.08871246116672, 68.5018985157059, 67.89782533655506, 67.27649292371419, 66.68967897825337, 66.03382809803244, 65.23990334829134, 64.54953400069037, 63.91094235415948, 63.20331377286848, 62.58198136002762, 61.822575077666556, 61.32205730065585, 60.66620642043493, 59.95857783914394, 59.38902312737315, 58.64687607870211, 58.00828443217121, 57.21435968243011, 56.316879530548846, 55.45391784604764, 54.694511563686575, 53.952364515015525, 53.08940283051433, 52.295478080773215, 51.380738695201934, 50.517777010700726, 49.620296858819465, 48.80911287538833, 48.01518812564722, 47.08318950638592, 46.18570935450466, 45.32274767000345, 44.45978598550225, 43.51052813255092, 42.63030721435969, 41.73282706247843, 40.83534691059717, 39.86882982395582, 38.90231273731447, 37.953054884363134, 37.03831549879186, 36.10631687953055, 35.1397997928892, 34.22506040731791, 33.27580255436659, 32.29202623403521, 31.39454608215395, 30.445288229202628, 29.513289609941317, 28.564031756989994, 27.56299620296859, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 63.206904730102394, "threshold_area100": 58.21599930963066, "threshold_area25": 80.712806351398}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0061280992813408375, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0075284517370164394, "operator": "ge"}, {"budget": 0.03, "threshold": -0.00957958772778511, "operator": "ge"}, {"budget": 0.04, "threshold": -0.01133684255182743, "operator": "ge"}, {"budget": 0.05, "threshold": -0.012673596851527691, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01567986235022545, "operator": "ge"}, {"budget": 0.07, "threshold": -0.022006699815392494, "operator": "ge"}, {"budget": 0.08, "threshold": -0.028553253039717674, "operator": "ge"}, {"budget": 0.09, "threshold": -0.0374443382024765, "operator": "ge"}, {"budget": 0.1, "threshold": -0.04497264325618744, "operator": "ge"}, {"budget": 0.11, "threshold": -0.060270339250564575, "operator": "ge"}, {"budget": 0.12, "threshold": -0.0833895280957222, "operator": "ge"}, {"budget": 0.13, "threshold": -0.10451038926839828, "operator": "ge"}, {"budget": 0.14, "threshold": -0.12435751408338547, "operator": "ge"}, {"budget": 0.15, "threshold": -0.13938559591770172, "operator": "ge"}, {"budget": 0.16, "threshold": -0.14907951653003693, "operator": "ge"}, {"budget": 0.17, "threshold": -0.18306884169578552, "operator": "ge"}, {"budget": 0.18, "threshold": -0.2087668627500534, "operator": "ge"}, {"budget": 0.19, "threshold": -0.24214644730091095, "operator": "ge"}, {"budget": 0.2, "threshold": -0.2866281569004059, "operator": "ge"}, {"budget": 0.21, "threshold": -0.32901450991630554, "operator": "ge"}, {"budget": 0.22, "threshold": -0.36170750856399536, "operator": "ge"}, {"budget": 0.23, "threshold": -0.3933005928993225, "operator": "ge"}, {"budget": 0.24, "threshold": -0.4291616380214691, "operator": "ge"}, {"budget": 0.25, "threshold": -0.4785645604133606, "operator": "ge"}, {"budget": 0.26, "threshold": -0.5161387324333191, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5669921636581421, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6123486161231995, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6507254838943481, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6782835721969604, "operator": "ge"}, {"budget": 0.31, "threshold": -0.701304018497467, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7246511578559875, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7430075407028198, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7607767581939697, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7891996502876282, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7991636395454407, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8097347021102905, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8294148445129395, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8433073163032532, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8509774804115295, "operator": "ge"}, {"budget": 0.41, "threshold": -0.8616909384727478, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8751422166824341, "operator": "ge"}, {"budget": 0.43, "threshold": -0.886139988899231, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8923884034156799, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8956731557846069, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9055857062339783, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9144119620323181, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9213765263557434, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9294031858444214, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9386687874794006, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9422948360443115, "operator": "ge"}, {"budget": 0.52, "threshold": -0.946887731552124, "operator": "ge"}, {"budget": 0.53, "threshold": -0.949507474899292, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9517558217048645, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9543506503105164, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9571070075035095, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9595099687576294, "operator": "ge"}, {"budget": 0.58, "threshold": -0.960839033126831, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9630143642425537, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9656237959861755, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9691079258918762, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9701248407363892, "operator": "ge"}, {"budget": 0.63, "threshold": -0.97266685962677, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9753293991088867, "operator": "ge"}, {"budget": 0.65, "threshold": -0.976718008518219, "operator": "ge"}, {"budget": 0.66, "threshold": -0.978922426700592, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9798735976219177, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9814716577529907, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9828217625617981, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9844800233840942, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9852387309074402, "operator": "ge"}, {"budget": 0.72, "threshold": -0.98598712682724, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9868913888931274, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9874170422554016, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9884153604507446, "operator": "ge"}, {"budget": 0.76, "threshold": -0.989288866519928, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9898726940155029, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9910945892333984, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9918593764305115, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9925138354301453, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9928067922592163, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9935107827186584, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9938969016075134, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9945120811462402, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9950332641601562, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9954079389572144, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9957066774368286, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9960741400718689, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9965130090713501, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9969208836555481, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9970928430557251, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9973589181900024, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9976683259010315, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9978731870651245, "operator": "ge"}, {"budget": 0.95, "threshold": -0.998117208480835, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9982621669769287, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9984968900680542, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9987394213676453, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9989650249481201, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.25, 98.25, 98.16666666666667, 98.0, 97.91666666666666, 97.5, 97.25, 97.08333333333333, 96.75, 96.41666666666666, 96.5, 96.08333333333333, 95.41666666666667, 95.0, 94.83333333333334, 94.25, 93.5, 93.08333333333333, 92.58333333333333, 92.0, 91.25, 90.33333333333333, 89.58333333333334, 88.75, 88.16666666666667, 87.41666666666667, 86.91666666666666, 86.0, 85.16666666666667, 84.5, 83.58333333333333, 82.75, 81.83333333333334, 81.0, 80.0, 79.16666666666667, 78.33333333333333, 77.66666666666666, 76.91666666666667, 76.0, 75.08333333333333, 74.16666666666667, 73.25, 72.33333333333334, 71.41666666666666, 70.41666666666666, 69.58333333333333, 68.91666666666667, 67.91666666666667, 67.0, 66.08333333333334, 65.16666666666666, 64.25, 63.5, 62.5, 61.5, 60.5, 59.66666666666667, 58.666666666666664, 57.75, 56.666666666666664, 55.666666666666664, 54.666666666666664, 53.91666666666667, 52.916666666666664, 51.916666666666664, 50.916666666666664, 50.0, 49.083333333333336, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 4, 5, 7, 8, 13, 16, 18, 22, 26, 26, 31, 39, 44, 47, 54, 63, 69, 76, 83, 92, 103, 112, 122, 129, 138, 144, 155, 165, 173, 184, 194, 205, 215, 227, 237, 247, 256, 265, 276, 287, 298, 309, 320, 331, 343, 353, 362, 374, 385, 396, 407, 418, 427, 439, 451, 463, 473, 485, 496, 509, 521, 533, 542, 554, 566, 578, 589, 600, 612, 623, 635, 646, 659, 670, 682, 693, 704, 716, 727, 739, 751, 763, 775, 787, 799, 810, 822, 833, 844, 856], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.25, 98.25, 98.16666666666667, 98.0, 97.91666666666666, 97.5, 97.25, 97.08333333333333, 96.75, 96.41666666666666, 96.5, 96.08333333333333, 95.41666666666667, 95.0, 94.83333333333334, 94.25, 93.5, 93.08333333333333, 92.58333333333333, 92.0, 91.25, 90.33333333333333, 89.58333333333334, 88.75, 88.16666666666667, 87.41666666666667, 86.91666666666666, 86.0, 85.16666666666667, 84.5, 83.58333333333333, 82.66666666666667, 81.83333333333334, 81.0, 80.0, 79.16666666666667, 78.33333333333333, 77.66666666666666, 76.91666666666667, 76.0, 75.08333333333333, 74.16666666666667, 73.25, 72.33333333333334, 71.41666666666666, 70.41666666666666, 69.58333333333333, 68.83333333333333, 67.91666666666667, 67.0, 66.08333333333334, 65.16666666666666, 64.25, 63.5, 62.5, 61.5, 60.5, 59.66666666666667, 58.666666666666664, 57.666666666666664, 56.666666666666664, 55.666666666666664, 54.666666666666664, 53.91666666666667, 52.916666666666664, 51.916666666666664, 50.916666666666664, 50.0, 49.083333333333336, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.56166666666667, "threshold_area100": 71.56500000000001, "threshold_area25": 97.44999999999999}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.39454608215395, 81.429064549534, 81.429064549534, 81.36002761477391, 81.41180531584398, 81.30824991370383, 81.25647221263375, 81.10113910942354, 81.04936140835348, 80.82499137038316, 80.68691750086296, 80.44528822920263, 79.97928891957199, 79.72040041422161, 79.28891957197101, 78.84017949603039, 78.49499482222988, 77.82188470831896, 77.07973765964792, 76.18225750776666, 75.45736969278565, 74.7842595788747, 74.23196410079392, 73.558853986883, 73.09285467725233, 72.43700379703142, 71.60856057991025, 70.2105626510183, 69.79634104245773, 69.26130479806697, 67.91508457024509, 67.01760441836383, 66.5688643424232, 65.77493959268207, 64.5667932343804, 63.75560925094926, 63.18605453917846, 62.87538833275803, 61.75353814290645, 60.925094925785295, 60.303762512944424, 59.285467725233, 58.025543665861235, 57.33517431826027, 56.16154642733863, 55.712806351397994, 55.16051087331723, 54.694511563686575, 53.952364515015525, 53.227476700034515, 52.951328960994125, 52.41629271660339, 51.60510873317226, 50.58681394546083, 50.20711080428028, 49.27511218501899, 48.41215050051778, 47.94615119088713, 46.96237487055575, 46.51363479461512, 45.702450811183986, 44.97756299620297, 44.080082844321716, 43.665861235761135, 43.23438039351053, 42.7166033828098, 42.267863306869174, 41.750086296168455, 41.31860545391785, 40.95616154642734, 39.972385226095966, 39.282015878494995, 38.557128063513986, 38.31549879185364, 37.60787021056265, 37.12461166724197, 36.382464618570935, 35.5022437003797, 34.77735588539869, 34.5012081463583, 33.74180186399724, 32.72350707628581, 32.03313772868485, 31.687953054884368, 31.10113910942355, 30.46254746289265, 30.031066620642044, 29.444252675181225, 29.133586468760786, 28.61580945806006, 28.011736278909215, 27.33862616499828, 26.561960648947192], "calls": [0.0, 0.6731101139109423, 1.1045909561615463, 2.0365895754228514, 2.6406627545736967, 3.089402830514325, 3.917846047635485, 5.471177079737659, 6.765619606489472, 7.870210562651018, 8.715913013462202, 10.52813255091474, 12.15050051777701, 13.410424577148774, 14.60131170176044, 15.67138419054194, 16.17190196755264, 17.86330686917501, 18.74352778736624, 19.986192613047983, 22.195374525371072, 23.783224024853297, 24.887814981014845, 25.957887469796344, 27.338626164998274, 29.219882637210908, 30.669658267172938, 32.982395581636176, 35.01898515705902, 36.90024162927166, 38.298239558163615, 39.575422851225404, 40.88712461166724, 42.06075250258888, 43.54504659993096, 45.96133931653434, 46.89333793579565, 47.70452191922679, 49.98274076630997, 51.43251639627201, 52.17466344494305, 53.31377286848464, 54.918881601656885, 56.19606489471868, 57.093545046599935, 57.54228512254056, 59.11287538833275, 60.510873317224714, 61.52916810493614, 63.16879530548844, 64.94649637556093, 65.80945806006213, 67.19019675526407, 67.7079737659648, 68.3638246461857, 69.0024162927166, 69.98619261304798, 70.81463583016914, 71.15982050396963, 71.79841215050051, 72.71315153607179, 73.9212978943735, 74.31826026924404, 75.47462892647566, 76.51018294787711, 77.06247842595789, 78.13255091473938, 78.66758715913014, 79.4960303762513, 80.37625129444253, 81.41180531584398, 81.82602692440456, 82.34380393510529, 82.89609941318605, 83.39661719019675, 83.96617190196756, 84.5012081463583, 84.86365205384881, 85.88194684156024, 86.60683465654125, 87.38350017259233, 87.62512944425268, 88.40179496030376, 88.90231273731446, 89.7134967207456, 90.61097687262686, 91.38764238867795, 91.71556782878841, 92.50949257852952, 93.56230583362098, 94.26993441491199, 94.66689678978253, 95.28822920262341, 95.97859855022438, 96.427338626165, 97.0659302726959, 97.37659647911633, 97.94615119088712, 98.55022437003798, 99.22333448394892, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 1, 3, 6, 6, 6, 11, 12, 13, 15, 17, 20, 21, 26, 34, 47, 48, 51, 56, 65, 69, 81, 95, 107, 113, 114, 125, 128, 131, 136, 140, 142, 152, 156, 159, 162, 164, 168, 173, 174, 177, 186, 189, 196, 203, 205, 207, 207, 208, 210, 212, 212, 213, 214, 215, 218, 218, 220, 222, 223, 224, 224, 224, 225, 226, 226, 226, 226, 226, 226, 228, 228, 228, 228, 228, 228, 229, 229, 230, 230, 230, 230, 230, 230, 230, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 0, 0, 0, 0, 0, 1, 3, 5, 6, 6, 10, 12, 19, 23, 34, 39, 55, 64, 83, 118, 146, 172, 201, 226, 274, 321, 385, 441, 492, 530, 570, 608, 649, 700, 786, 814, 847, 935, 991, 1020, 1069, 1141, 1192, 1230, 1249, 1317, 1374, 1413, 1479, 1559, 1601, 1671, 1697, 1730, 1759, 1804, 1846, 1863, 1895, 1943, 2005, 2027, 2083, 2135, 2163, 2221, 2247, 2294, 2337, 2390, 2414, 2439, 2469, 2495, 2525, 2552, 2573, 2630, 2670, 2712, 2726, 2768, 2796, 2840, 2891, 2933, 2949, 2993, 3052, 3092, 3113, 3147, 3185, 3210, 3244, 3262, 3292, 3327, 3366, 3411], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.429064549534, 81.429064549534, 81.37728684846392, 81.37728684846392, 81.39454608215395, 81.32550914739386, 81.27373144632378, 81.23921297894374, 81.04936140835348, 80.9285467725233, 80.82499137038316, 80.60062133241284, 80.44528822920263, 80.18639972385226, 80.06558508802209, 79.89299275112185, 79.70314118053159, 79.25440110459095, 78.84017949603039, 78.59855022437004, 78.28788401794961, 77.92544011045909, 77.47670003451846, 76.9244045564377, 76.57921988263722, 76.18225750776666, 75.83707283396618, 75.45736969278565, 75.12944425267519, 74.73248187780463, 74.37003797031412, 73.86952019330342, 73.38626164998274, 73.04107697618227, 72.47152226441146, 71.93648602002072, 71.45322747670004, 70.79737659647913, 70.19330341732827, 69.7100448740076, 69.08871246116672, 68.5018985157059, 67.89782533655506, 67.27649292371419, 66.68967897825337, 66.03382809803244, 65.23990334829134, 64.54953400069037, 63.91094235415948, 63.20331377286848, 62.58198136002762, 61.822575077666556, 61.32205730065585, 60.66620642043493, 59.95857783914394, 59.38902312737315, 58.64687607870211, 58.00828443217121, 57.21435968243011, 56.316879530548846, 55.45391784604764, 54.694511563686575, 53.952364515015525, 53.08940283051433, 52.295478080773215, 51.380738695201934, 50.517777010700726, 49.620296858819465, 48.80911287538833, 48.01518812564722, 47.08318950638592, 46.18570935450466, 45.32274767000345, 44.45978598550225, 43.51052813255092, 42.63030721435969, 41.73282706247843, 40.83534691059717, 39.86882982395582, 38.90231273731447, 37.953054884363134, 37.03831549879186, 36.10631687953055, 35.1397997928892, 34.22506040731791, 33.27580255436659, 32.29202623403521, 31.39454608215395, 30.445288229202628, 29.513289609941317, 28.564031756989994, 27.56299620296859, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 63.206904730102394, "threshold_area100": 58.21599930963066, "threshold_area25": 80.712806351398}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.2670059204101562, "operator": "ge"}, {"budget": 0.02, "threshold": -2.762105703353882, "operator": "ge"}, {"budget": 0.03, "threshold": -3.2467668056488037, "operator": "ge"}, {"budget": 0.04, "threshold": -3.7816720008850098, "operator": "ge"}, {"budget": 0.05, "threshold": -4.157200336456299, "operator": "ge"}, {"budget": 0.06, "threshold": -4.569339275360107, "operator": "ge"}, {"budget": 0.07, "threshold": -4.936811923980713, "operator": "ge"}, {"budget": 0.08, "threshold": -5.098090171813965, "operator": "ge"}, {"budget": 0.09, "threshold": -5.255184650421143, "operator": "ge"}, {"budget": 0.1, "threshold": -5.717022895812988, "operator": "ge"}, {"budget": 0.11, "threshold": -5.9153571128845215, "operator": "ge"}, {"budget": 0.12, "threshold": -6.225737571716309, "operator": "ge"}, {"budget": 0.13, "threshold": -6.551548957824707, "operator": "ge"}, {"budget": 0.14, "threshold": -6.746844291687012, "operator": "ge"}, {"budget": 0.15, "threshold": -7.079832077026367, "operator": "ge"}, {"budget": 0.16, "threshold": -7.375344753265381, "operator": "ge"}, {"budget": 0.17, "threshold": -7.694469451904297, "operator": "ge"}, {"budget": 0.18, "threshold": -8.012914657592773, "operator": "ge"}, {"budget": 0.19, "threshold": -8.469160079956055, "operator": "ge"}, {"budget": 0.2, "threshold": -8.781855583190918, "operator": "ge"}, {"budget": 0.21, "threshold": -9.258280754089355, "operator": "ge"}, {"budget": 0.22, "threshold": -9.545408248901367, "operator": "ge"}, {"budget": 0.23, "threshold": -9.816880226135254, "operator": "ge"}, {"budget": 0.24, "threshold": -10.105500221252441, "operator": "ge"}, {"budget": 0.25, "threshold": -10.426362991333008, "operator": "ge"}, {"budget": 0.26, "threshold": -10.662906646728516, "operator": "ge"}, {"budget": 0.27, "threshold": -10.886289596557617, "operator": "ge"}, {"budget": 0.28, "threshold": -11.013819694519043, "operator": "ge"}, {"budget": 0.29, "threshold": -11.049652099609375, "operator": "ge"}, {"budget": 0.3, "threshold": -11.150790214538574, "operator": "ge"}, {"budget": 0.31, "threshold": -11.205633163452148, "operator": "ge"}, {"budget": 0.32, "threshold": -11.293389320373535, "operator": "ge"}, {"budget": 0.33, "threshold": -11.430155754089355, "operator": "ge"}, {"budget": 0.34, "threshold": -11.547002792358398, "operator": "ge"}, {"budget": 0.35, "threshold": -11.624927520751953, "operator": "ge"}, {"budget": 0.36, "threshold": -11.739851951599121, "operator": "ge"}, {"budget": 0.37, "threshold": -11.79393482208252, "operator": "ge"}, {"budget": 0.38, "threshold": -11.865988731384277, "operator": "ge"}, {"budget": 0.39, "threshold": -11.932682991027832, "operator": "ge"}, {"budget": 0.4, "threshold": -12.044357299804688, "operator": "ge"}, {"budget": 0.41, "threshold": -12.092586517333984, "operator": "ge"}, {"budget": 0.42, "threshold": -12.203695297241211, "operator": "ge"}, {"budget": 0.43, "threshold": -12.286503791809082, "operator": "ge"}, {"budget": 0.44, "threshold": -12.330665588378906, "operator": "ge"}, {"budget": 0.45, "threshold": -12.406221389770508, "operator": "ge"}, {"budget": 0.46, "threshold": -12.488428115844727, "operator": "ge"}, {"budget": 0.47, "threshold": -12.566469192504883, "operator": "ge"}, {"budget": 0.48, "threshold": -12.610992431640625, "operator": "ge"}, {"budget": 0.49, "threshold": -12.655670166015625, "operator": "ge"}, {"budget": 0.5, "threshold": -12.747575759887695, "operator": "ge"}, {"budget": 0.51, "threshold": -12.777390480041504, "operator": "ge"}, {"budget": 0.52, "threshold": -12.844160079956055, "operator": "ge"}, {"budget": 0.53, "threshold": -12.889076232910156, "operator": "ge"}, {"budget": 0.54, "threshold": -12.97193717956543, "operator": "ge"}, {"budget": 0.55, "threshold": -13.020915985107422, "operator": "ge"}, {"budget": 0.56, "threshold": -13.0966796875, "operator": "ge"}, {"budget": 0.57, "threshold": -13.140222549438477, "operator": "ge"}, {"budget": 0.58, "threshold": -13.177453994750977, "operator": "ge"}, {"budget": 0.59, "threshold": -13.227304458618164, "operator": "ge"}, {"budget": 0.6, "threshold": -13.28622055053711, "operator": "ge"}, {"budget": 0.61, "threshold": -13.372936248779297, "operator": "ge"}, {"budget": 0.62, "threshold": -13.398265838623047, "operator": "ge"}, {"budget": 0.63, "threshold": -13.45861530303955, "operator": "ge"}, {"budget": 0.64, "threshold": -13.541536331176758, "operator": "ge"}, {"budget": 0.65, "threshold": -13.578591346740723, "operator": "ge"}, {"budget": 0.66, "threshold": -13.637454986572266, "operator": "ge"}, {"budget": 0.67, "threshold": -13.750665664672852, "operator": "ge"}, {"budget": 0.68, "threshold": -13.813132286071777, "operator": "ge"}, {"budget": 0.69, "threshold": -13.863805770874023, "operator": "ge"}, {"budget": 0.7, "threshold": -13.920650482177734, "operator": "ge"}, {"budget": 0.71, "threshold": -13.978830337524414, "operator": "ge"}, {"budget": 0.72, "threshold": -14.070486068725586, "operator": "ge"}, {"budget": 0.73, "threshold": -14.142193794250488, "operator": "ge"}, {"budget": 0.74, "threshold": -14.171789169311523, "operator": "ge"}, {"budget": 0.75, "threshold": -14.22890853881836, "operator": "ge"}, {"budget": 0.76, "threshold": -14.307597160339355, "operator": "ge"}, {"budget": 0.77, "threshold": -14.369194030761719, "operator": "ge"}, {"budget": 0.78, "threshold": -14.427787780761719, "operator": "ge"}, {"budget": 0.79, "threshold": -14.517303466796875, "operator": "ge"}, {"budget": 0.8, "threshold": -14.562019348144531, "operator": "ge"}, {"budget": 0.81, "threshold": -14.603699684143066, "operator": "ge"}, {"budget": 0.82, "threshold": -14.650510787963867, "operator": "ge"}, {"budget": 0.83, "threshold": -14.707229614257812, "operator": "ge"}, {"budget": 0.84, "threshold": -14.76105785369873, "operator": "ge"}, {"budget": 0.85, "threshold": -14.813211441040039, "operator": "ge"}, {"budget": 0.86, "threshold": -14.907234191894531, "operator": "ge"}, {"budget": 0.87, "threshold": -14.9609375, "operator": "ge"}, {"budget": 0.88, "threshold": -15.003854751586914, "operator": "ge"}, {"budget": 0.89, "threshold": -15.104240417480469, "operator": "ge"}, {"budget": 0.9, "threshold": -15.1685791015625, "operator": "ge"}, {"budget": 0.91, "threshold": -15.248035430908203, "operator": "ge"}, {"budget": 0.92, "threshold": -15.3065185546875, "operator": "ge"}, {"budget": 0.93, "threshold": -15.377525329589844, "operator": "ge"}, {"budget": 0.94, "threshold": -15.465913772583008, "operator": "ge"}, {"budget": 0.95, "threshold": -15.581015586853027, "operator": "ge"}, {"budget": 0.96, "threshold": -15.672099113464355, "operator": "ge"}, {"budget": 0.97, "threshold": -15.853094100952148, "operator": "ge"}, {"budget": 0.98, "threshold": -16.01482582092285, "operator": "ge"}, {"budget": 0.99, "threshold": -16.309051513671875, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.16666666666667, 97.75, 97.16666666666667, 96.41666666666666, 96.08333333333333, 95.33333333333334, 94.75, 94.33333333333334, 93.91666666666667, 93.25, 92.41666666666667, 91.75, 90.83333333333333, 90.0, 89.33333333333333, 88.66666666666667, 88.0, 87.33333333333333, 86.41666666666666, 85.58333333333333, 85.25, 84.75, 84.16666666666667, 83.75, 82.91666666666666, 82.33333333333334, 81.83333333333334, 81.33333333333333, 81.16666666666667, 80.83333333333333, 80.5, 80.08333333333333, 79.75, 79.5, 79.08333333333333, 78.41666666666667, 77.91666666666667, 77.16666666666667, 76.66666666666666, 76.08333333333334, 75.58333333333334, 74.83333333333333, 74.41666666666666, 74.0, 73.58333333333333, 72.91666666666667, 72.41666666666666, 71.75, 71.0, 70.33333333333334, 69.5, 68.66666666666667, 68.0, 67.58333333333333, 67.0, 66.25000000000001, 65.41666666666667, 64.33333333333333, 64.0, 63.083333333333336, 62.33333333333333, 61.583333333333336, 60.83333333333334, 59.916666666666664, 59.00000000000001, 58.33333333333333, 57.41666666666667, 56.49999999999999, 55.75, 54.75, 53.833333333333336, 53.0, 52.33333333333333, 51.5, 50.75000000000001, 49.833333333333336, 48.833333333333336, 48.0, 47.16666666666667, 46.41666666666667, 45.666666666666664, 44.833333333333336, 43.91666666666667, 42.91666666666667, 42.00000000000001, 41.083333333333336, 40.16666666666667, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.5, 35.5, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 2, 3, 3, 3, 3, 3, 3, 4, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 7, 13, 20, 29, 33, 42, 49, 55, 61, 69, 79, 88, 99, 109, 117, 125, 133, 141, 152, 162, 166, 172, 179, 184, 194, 201, 207, 213, 215, 219, 223, 228, 232, 235, 240, 248, 254, 263, 269, 276, 282, 291, 296, 301, 306, 314, 320, 328, 337, 345, 355, 365, 373, 378, 385, 394, 404, 417, 421, 432, 441, 450, 459, 470, 481, 489, 500, 511, 520, 532, 543, 553, 561, 571, 580, 591, 603, 613, 623, 632, 641, 651, 662, 674, 685, 696, 707, 716, 728, 740, 751, 763, 774, 786, 798, 810, 820, 832, 844, 856], "exact_score": [98.58333333333333, 98.16666666666667, 97.75, 97.16666666666667, 96.41666666666666, 96.08333333333333, 95.33333333333334, 94.75, 94.33333333333334, 93.91666666666667, 93.25, 92.41666666666667, 91.75, 90.83333333333333, 90.0, 89.33333333333333, 88.66666666666667, 88.0, 87.33333333333333, 86.41666666666666, 85.58333333333333, 85.25, 84.75, 84.16666666666667, 83.75, 82.91666666666666, 82.33333333333334, 81.83333333333334, 81.33333333333333, 81.16666666666667, 80.83333333333333, 80.5, 80.08333333333333, 79.75, 79.5, 79.08333333333333, 78.41666666666667, 77.91666666666667, 77.16666666666667, 76.66666666666666, 76.08333333333334, 75.5, 74.83333333333333, 74.41666666666666, 74.0, 73.58333333333333, 72.91666666666667, 72.41666666666666, 71.75, 71.0, 70.33333333333334, 69.5, 68.66666666666667, 68.0, 67.58333333333333, 67.0, 66.25000000000001, 65.33333333333333, 64.33333333333333, 64.0, 63.083333333333336, 62.33333333333333, 61.583333333333336, 60.83333333333334, 59.916666666666664, 59.00000000000001, 58.33333333333333, 57.41666666666667, 56.49999999999999, 55.666666666666664, 54.75, 53.833333333333336, 53.0, 52.33333333333333, 51.5, 50.75000000000001, 49.833333333333336, 48.833333333333336, 48.0, 47.16666666666667, 46.41666666666667, 45.666666666666664, 44.75, 43.91666666666667, 42.91666666666667, 42.00000000000001, 41.083333333333336, 40.16666666666667, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.5, 35.5, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 67.16, "threshold_area100": 67.16333333333333, "threshold_area25": 91.0}, "test": {"score": [81.429064549534, 80.73869520193303, 79.96202968588196, 78.75388332758025, 77.14877459440801, 75.47462892647566, 73.54159475319297, 71.90196755264067, 71.24611667241975, 70.31411805315844, 68.0013807386952, 66.96582671729375, 65.63686572316189, 64.35968243010011, 63.72109078356921, 62.892647566448055, 61.891612012426656, 60.890576458405256, 60.11391094235417, 59.00931998619261, 58.26717293752157, 57.33517431826027, 56.92095270969969, 56.523990334829136, 55.83362098722817, 55.229547808077314, 54.83258543320677, 54.418363824646185, 54.17673455298585, 54.12495685191578, 53.81429064549535, 53.74525371073524, 53.503624439074905, 53.10666206420436, 52.86503279254402, 52.623403520883684, 52.32999654815327, 52.157404211253024, 51.77770107007249, 51.449775629962026, 51.08733172247153, 50.84570245081119, 50.53503624439075, 49.98274076630998, 49.79288919571972, 49.58577839143942, 49.257852951328964, 48.8608905764584, 48.688298239558165, 48.55022437003797, 47.9979288919572, 47.87711425612703, 47.54918881601657, 47.273041076976185, 46.789782533655504, 46.54815326199517, 46.202968588194686, 46.04763548498447, 45.82326544701416, 45.59889540904384, 45.288229202623405, 44.73593372454263, 44.649637556092515, 44.26993441491198, 43.76941663790128, 43.56230583362099, 43.04452882292026, 42.45771487745944, 41.92267863306869, 41.50845702450812, 41.11149464963756, 40.66275457369693, 40.09319986192613, 39.36831204694512, 39.057645840524685, 38.557128063513986, 37.90127718329306, 37.31446323783224, 36.79668622713152, 35.881946841560236, 35.467725232999655, 35.036244390749054, 34.5012081463583, 34.0697273041077, 33.620987228167074, 33.29306178805661, 32.481877804625476, 31.981360027614773, 31.87780462547463, 31.066620642043496, 30.63513979979289, 30.23817742492233, 30.031066620642044, 29.754918881601654, 29.375215740421122, 28.874697963410423, 28.512254055919918, 27.80462547462893, 27.390403866068347, 26.87262685536762, 26.561960648947192], "calls": [0.0, 4.263030721435968, 8.198136002761478, 12.651018294787711, 18.001380738695204, 22.609596133931653, 27.804625474628924, 31.774249223334483, 33.206765619606486, 34.673800483258546, 38.97134967207456, 40.86986537797721, 43.13082499137038, 45.04659993096307, 46.03037625129444, 47.60096651708664, 49.13703831549879, 50.517777010700726, 51.605108733172244, 53.31377286848464, 54.48740075940628, 56.092509492578536, 56.851915774939584, 57.54228512254056, 58.50880220918191, 59.7859855022437, 60.528132550914734, 61.44287193648602, 61.9951674145668, 62.133241284086985, 62.80635139799793, 63.03072143596824, 63.4967207455989, 64.32516396272005, 65.10182947877115, 65.63686572316189, 66.25819813600276, 66.65516051087332, 67.44908526061442, 68.15671384190541, 69.03693476009664, 69.55471177079737, 70.33137728684846, 71.22885743872972, 71.59130134622022, 72.05730065585088, 72.6441146013117, 73.31722471522264, 73.54159475319297, 73.85226095961339, 74.87055574732481, 75.09492578529513, 75.56092509492579, 75.90610976872627, 76.61373835001726, 77.02795995857784, 77.59751467034863, 77.82188470831896, 78.18432861580946, 78.51225405591991, 79.01277183293061, 79.7204004142216, 79.82395581636176, 80.30721435968243, 81.01484294097342, 81.27373144632378, 81.91232309285468, 82.58543320676563, 83.24128408698654, 83.72454263030721, 84.19054193993787, 84.7083189506386, 85.41594753192958, 86.17535381429065, 86.48602002071108, 87.05557473248187, 87.83224024853297, 88.4880911287539, 89.04038660683466, 90.04142216085606, 90.45564376941664, 90.95616154642734, 91.54297549188816, 92.00897480151882, 92.50949257852952, 92.85467725232999, 93.68312046945115, 94.2181567138419, 94.3562305833621, 95.23645150155333, 95.719710044874, 96.16845012081463, 96.39282015878496, 96.70348636520538, 97.10044874007593, 97.63548498446669, 98.03244735933725, 98.74007594062823, 99.17155678287884, 99.68933379357956, 100.0], "rescued": [0, 30, 58, 82, 109, 133, 155, 173, 178, 181, 192, 204, 209, 212, 214, 218, 220, 222, 222, 223, 223, 224, 226, 226, 226, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 229, 229, 229, 229, 229, 229, 229, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 231, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 70, 143, 237, 357, 478, 612, 725, 768, 825, 970, 1042, 1124, 1201, 1240, 1292, 1352, 1412, 1457, 1522, 1565, 1620, 1646, 1669, 1709, 1745, 1768, 1792, 1806, 1809, 1827, 1831, 1845, 1868, 1882, 1896, 1913, 1923, 1945, 1964, 1985, 1999, 2018, 2050, 2061, 2073, 2092, 2115, 2125, 2133, 2165, 2172, 2191, 2207, 2235, 2249, 2269, 2278, 2291, 2304, 2322, 2354, 2359, 2381, 2411, 2423, 2453, 2487, 2518, 2542, 2565, 2592, 2625, 2667, 2685, 2714, 2752, 2786, 2816, 2869, 2893, 2919, 2950, 2975, 3001, 3020, 3067, 3097, 3103, 3150, 3175, 3198, 3210, 3226, 3248, 3277, 3298, 3339, 3363, 3393, 3411], "exact_score": [81.429064549534, 81.34276838108389, 81.25647221263375, 81.11839834311357, 80.84225060407319, 80.68691750086296, 80.4280289955126, 80.20365895754229, 79.9275112185019, 79.70314118053159, 79.44425267518123, 79.18536416983086, 78.97825336555057, 78.65032792544011, 78.39143942008975, 78.06351397997929, 77.82188470831896, 77.44218156713842, 77.14877459440801, 76.78633068691751, 76.47566448049706, 76.07870210562652, 75.69899896444599, 75.33655505695546, 74.88781498101484, 74.5598895409044, 74.33551950293406, 73.95581636175355, 73.54159475319297, 73.02381774249224, 72.67863306869175, 72.2816706938212, 71.79841215050052, 71.40144977562997, 70.7801173627891, 70.17604418363825, 69.65826717293753, 69.03693476009666, 68.58819468415602, 68.01863997238523, 67.5698998964446, 66.91404901622369, 66.39627200552296, 65.74042112530203, 65.0673110113911, 64.39420089748016, 63.738350017259236, 63.18605453917846, 62.66827752847773, 62.02968588194684, 61.23576113220574, 60.56265101829479, 59.87228167069383, 59.216430790472906, 58.61235761132206, 57.95650673110114, 57.38695201933035, 56.83465654124957, 56.23058336209872, 55.60925094925786, 55.07421470486711, 54.59095616154643, 54.17673455298585, 53.74525371073524, 53.192958232654476, 52.88229202623403, 52.46807041767345, 51.96755264066275, 51.48429409734207, 51.10459095616156, 50.673110113910944, 50.15533310321022, 49.63755609250949, 49.06800138073869, 48.4984466689679, 47.894373489817056, 47.23852260959613, 46.58267172937522, 45.926820849154296, 45.288229202623405, 44.54608215395236, 43.7866758715913, 42.95823265447014, 42.12978943734898, 41.30134622022783, 40.472903003106666, 39.558163617535385, 38.62616499827408, 37.76320331377287, 36.84846392820159, 35.91646530894029, 35.001725923369, 34.08698653779773, 33.15498791853642, 32.205730065585094, 31.29099068001381, 30.410769761822575, 29.461511908871252, 28.546772523299968, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 59.81765473644727, "threshold_area100": 48.61330686917501, "threshold_area25": 66.59302726958924}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.5071883201599121, "operator": "ge"}, {"budget": 0.02, "threshold": -0.7419226169586182, "operator": "ge"}, {"budget": 0.03, "threshold": -1.1173410415649414, "operator": "ge"}, {"budget": 0.04, "threshold": -1.4384939670562744, "operator": "ge"}, {"budget": 0.05, "threshold": -1.6941320896148682, "operator": "ge"}, {"budget": 0.06, "threshold": -2.0641865730285645, "operator": "ge"}, {"budget": 0.07, "threshold": -2.3049516677856445, "operator": "ge"}, {"budget": 0.08, "threshold": -2.607316493988037, "operator": "ge"}, {"budget": 0.09, "threshold": -2.8466429710388184, "operator": "ge"}, {"budget": 0.1, "threshold": -3.1891891956329346, "operator": "ge"}, {"budget": 0.11, "threshold": -3.4842448234558105, "operator": "ge"}, {"budget": 0.12, "threshold": -3.8570590019226074, "operator": "ge"}, {"budget": 0.13, "threshold": -4.118746280670166, "operator": "ge"}, {"budget": 0.14, "threshold": -4.324134826660156, "operator": "ge"}, {"budget": 0.15, "threshold": -4.523721694946289, "operator": "ge"}, {"budget": 0.16, "threshold": -4.7868876457214355, "operator": "ge"}, {"budget": 0.17, "threshold": -5.076382637023926, "operator": "ge"}, {"budget": 0.18, "threshold": -5.4992265701293945, "operator": "ge"}, {"budget": 0.19, "threshold": -5.890434265136719, "operator": "ge"}, {"budget": 0.2, "threshold": -6.253354072570801, "operator": "ge"}, {"budget": 0.21, "threshold": -6.60682487487793, "operator": "ge"}, {"budget": 0.22, "threshold": -6.924437522888184, "operator": "ge"}, {"budget": 0.23, "threshold": -7.1952667236328125, "operator": "ge"}, {"budget": 0.24, "threshold": -7.417975425720215, "operator": "ge"}, {"budget": 0.25, "threshold": -7.575943946838379, "operator": "ge"}, {"budget": 0.26, "threshold": -7.667542457580566, "operator": "ge"}, {"budget": 0.27, "threshold": -7.778276443481445, "operator": "ge"}, {"budget": 0.28, "threshold": -7.955372333526611, "operator": "ge"}, {"budget": 0.29, "threshold": -8.05909538269043, "operator": "ge"}, {"budget": 0.3, "threshold": -8.246744155883789, "operator": "ge"}, {"budget": 0.31, "threshold": -8.377540588378906, "operator": "ge"}, {"budget": 0.32, "threshold": -8.465188026428223, "operator": "ge"}, {"budget": 0.33, "threshold": -8.559769630432129, "operator": "ge"}, {"budget": 0.34, "threshold": -8.722491264343262, "operator": "ge"}, {"budget": 0.35, "threshold": -8.801969528198242, "operator": "ge"}, {"budget": 0.36, "threshold": -8.941728591918945, "operator": "ge"}, {"budget": 0.37, "threshold": -9.009220123291016, "operator": "ge"}, {"budget": 0.38, "threshold": -9.093622207641602, "operator": "ge"}, {"budget": 0.39, "threshold": -9.16795825958252, "operator": "ge"}, {"budget": 0.4, "threshold": -9.26661491394043, "operator": "ge"}, {"budget": 0.41, "threshold": -9.387859344482422, "operator": "ge"}, {"budget": 0.42, "threshold": -9.51795768737793, "operator": "ge"}, {"budget": 0.43, "threshold": -9.56310749053955, "operator": "ge"}, {"budget": 0.44, "threshold": -9.695483207702637, "operator": "ge"}, {"budget": 0.45, "threshold": -9.751996994018555, "operator": "ge"}, {"budget": 0.46, "threshold": -9.798810958862305, "operator": "ge"}, {"budget": 0.47, "threshold": -9.889972686767578, "operator": "ge"}, {"budget": 0.48, "threshold": -9.934978485107422, "operator": "ge"}, {"budget": 0.49, "threshold": -9.977211952209473, "operator": "ge"}, {"budget": 0.5, "threshold": -10.079236030578613, "operator": "ge"}, {"budget": 0.51, "threshold": -10.133999824523926, "operator": "ge"}, {"budget": 0.52, "threshold": -10.189330101013184, "operator": "ge"}, {"budget": 0.53, "threshold": -10.263818740844727, "operator": "ge"}, {"budget": 0.54, "threshold": -10.32396411895752, "operator": "ge"}, {"budget": 0.55, "threshold": -10.394054412841797, "operator": "ge"}, {"budget": 0.56, "threshold": -10.447187423706055, "operator": "ge"}, {"budget": 0.57, "threshold": -10.506073951721191, "operator": "ge"}, {"budget": 0.58, "threshold": -10.54587173461914, "operator": "ge"}, {"budget": 0.59, "threshold": -10.620207786560059, "operator": "ge"}, {"budget": 0.6, "threshold": -10.71352767944336, "operator": "ge"}, {"budget": 0.61, "threshold": -10.771476745605469, "operator": "ge"}, {"budget": 0.62, "threshold": -10.835723876953125, "operator": "ge"}, {"budget": 0.63, "threshold": -10.929681777954102, "operator": "ge"}, {"budget": 0.64, "threshold": -10.976640701293945, "operator": "ge"}, {"budget": 0.65, "threshold": -11.068807601928711, "operator": "ge"}, {"budget": 0.66, "threshold": -11.13797378540039, "operator": "ge"}, {"budget": 0.67, "threshold": -11.216729164123535, "operator": "ge"}, {"budget": 0.68, "threshold": -11.27798843383789, "operator": "ge"}, {"budget": 0.69, "threshold": -11.333824157714844, "operator": "ge"}, {"budget": 0.7, "threshold": -11.395925521850586, "operator": "ge"}, {"budget": 0.71, "threshold": -11.436956405639648, "operator": "ge"}, {"budget": 0.72, "threshold": -11.487555503845215, "operator": "ge"}, {"budget": 0.73, "threshold": -11.552515029907227, "operator": "ge"}, {"budget": 0.74, "threshold": -11.614090919494629, "operator": "ge"}, {"budget": 0.75, "threshold": -11.673184394836426, "operator": "ge"}, {"budget": 0.76, "threshold": -11.731666564941406, "operator": "ge"}, {"budget": 0.77, "threshold": -11.778457641601562, "operator": "ge"}, {"budget": 0.78, "threshold": -11.822383880615234, "operator": "ge"}, {"budget": 0.79, "threshold": -11.896312713623047, "operator": "ge"}, {"budget": 0.8, "threshold": -11.954233169555664, "operator": "ge"}, {"budget": 0.81, "threshold": -12.02332592010498, "operator": "ge"}, {"budget": 0.82, "threshold": -12.093273162841797, "operator": "ge"}, {"budget": 0.83, "threshold": -12.16953182220459, "operator": "ge"}, {"budget": 0.84, "threshold": -12.224645614624023, "operator": "ge"}, {"budget": 0.85, "threshold": -12.281761169433594, "operator": "ge"}, {"budget": 0.86, "threshold": -12.339937210083008, "operator": "ge"}, {"budget": 0.87, "threshold": -12.410703659057617, "operator": "ge"}, {"budget": 0.88, "threshold": -12.466614723205566, "operator": "ge"}, {"budget": 0.89, "threshold": -12.525653839111328, "operator": "ge"}, {"budget": 0.9, "threshold": -12.607173919677734, "operator": "ge"}, {"budget": 0.91, "threshold": -12.701955795288086, "operator": "ge"}, {"budget": 0.92, "threshold": -12.783645629882812, "operator": "ge"}, {"budget": 0.93, "threshold": -12.850566864013672, "operator": "ge"}, {"budget": 0.94, "threshold": -12.925821304321289, "operator": "ge"}, {"budget": 0.95, "threshold": -13.052862167358398, "operator": "ge"}, {"budget": 0.96, "threshold": -13.191192626953125, "operator": "ge"}, {"budget": 0.97, "threshold": -13.329259872436523, "operator": "ge"}, {"budget": 0.98, "threshold": -13.487983703613281, "operator": "ge"}, {"budget": 0.99, "threshold": -13.647964477539062, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.41666666666666, 97.75, 97.25, 96.58333333333333, 96.0, 95.41666666666667, 95.0, 94.25, 93.83333333333333, 93.33333333333333, 92.58333333333333, 91.83333333333333, 91.08333333333334, 90.41666666666667, 89.58333333333334, 88.83333333333333, 88.16666666666667, 87.41666666666667, 86.75, 86.25, 85.91666666666666, 85.08333333333333, 84.41666666666667, 83.75, 83.66666666666667, 83.25, 83.0, 82.91666666666666, 82.5, 81.91666666666667, 81.58333333333333, 80.83333333333333, 80.41666666666667, 80.0, 79.66666666666666, 79.5, 78.83333333333333, 78.33333333333333, 77.83333333333333, 77.33333333333333, 76.91666666666667, 76.16666666666667, 75.58333333333334, 75.5, 74.83333333333333, 73.91666666666666, 73.08333333333333, 72.50000000000001, 72.0, 71.33333333333334, 70.75, 70.0, 69.16666666666667, 68.41666666666666, 67.41666666666667, 66.5, 65.91666666666667, 65.0, 64.0, 63.16666666666666, 62.25000000000001, 61.583333333333336, 60.916666666666664, 60.083333333333336, 59.33333333333333, 58.41666666666667, 57.666666666666664, 56.91666666666666, 56.166666666666664, 55.25, 54.58333333333334, 53.666666666666664, 52.83333333333333, 51.916666666666664, 51.0, 50.0, 49.25, 48.33333333333334, 47.5, 46.5, 45.666666666666664, 44.833333333333336, 43.91666666666667, 43.00000000000001, 42.166666666666664, 41.166666666666664, 40.25, 39.5, 38.583333333333336, 37.583333333333336, 36.583333333333336, 35.583333333333336, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 2, 3, 3, 3, 3, 3, 3, 4, 4, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 3, 12, 19, 27, 34, 41, 46, 55, 61, 67, 77, 87, 96, 104, 114, 123, 131, 140, 148, 154, 158, 168, 176, 184, 185, 190, 193, 194, 199, 206, 210, 219, 224, 229, 233, 235, 243, 249, 255, 261, 266, 275, 282, 283, 291, 302, 312, 319, 325, 333, 340, 349, 359, 368, 380, 391, 398, 409, 421, 431, 442, 450, 458, 468, 477, 488, 497, 506, 515, 526, 534, 545, 555, 566, 577, 589, 598, 609, 619, 631, 641, 651, 662, 673, 683, 695, 706, 715, 726, 738, 750, 762, 773, 785, 797, 809, 821, 832, 844, 856], "exact_score": [98.58333333333333, 98.41666666666666, 97.75, 97.25, 96.58333333333333, 96.0, 95.41666666666667, 95.0, 94.25, 93.83333333333333, 93.33333333333333, 92.58333333333333, 91.83333333333333, 91.08333333333334, 90.41666666666667, 89.58333333333334, 88.83333333333333, 88.16666666666667, 87.41666666666667, 86.75, 86.25, 85.91666666666666, 85.08333333333333, 84.41666666666667, 83.75, 83.66666666666667, 83.25, 83.0, 82.91666666666666, 82.5, 81.91666666666667, 81.58333333333333, 80.83333333333333, 80.41666666666667, 80.0, 79.66666666666666, 79.5, 78.83333333333333, 78.33333333333333, 77.83333333333333, 77.33333333333333, 76.83333333333333, 76.16666666666667, 75.58333333333334, 75.5, 74.83333333333333, 73.91666666666666, 73.08333333333333, 72.50000000000001, 72.0, 71.33333333333334, 70.75, 70.0, 69.16666666666667, 68.41666666666666, 67.41666666666667, 66.5, 65.83333333333333, 65.0, 64.0, 63.16666666666666, 62.25000000000001, 61.583333333333336, 60.916666666666664, 60.083333333333336, 59.33333333333333, 58.41666666666667, 57.666666666666664, 56.91666666666666, 56.08333333333333, 55.25, 54.58333333333334, 53.666666666666664, 52.83333333333333, 51.916666666666664, 51.0, 50.0, 49.25, 48.33333333333334, 47.5, 46.5, 45.666666666666664, 44.75, 43.91666666666667, 43.00000000000001, 42.166666666666664, 41.166666666666664, 40.25, 39.5, 38.583333333333336, 37.583333333333336, 36.583333333333336, 35.583333333333336, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 67.61333333333333, "threshold_area100": 67.61666666666667, "threshold_area25": 91.25833333333333}, "test": {"score": [81.429064549534, 80.98032447359338, 79.96202968588196, 78.54677252329996, 77.23507076285813, 76.28581290990681, 74.61166724197446, 72.98929927511219, 71.47048671039006, 70.48671039005868, 68.69175008629618, 67.25923369002416, 65.41249568519159, 64.37694166379013, 63.72109078356921, 62.94442526751812, 61.94338971349672, 61.09768726268554, 60.16568864342423, 59.302726958923024, 58.49154297549188, 57.57680358992061, 56.869175008629625, 56.403175698998965, 56.092509492578536, 55.55747324818778, 55.41939937866759, 55.16051087331723, 54.729030031066614, 54.48740075940628, 54.142216085605796, 53.93510528132551, 53.74525371073524, 53.572661373835004, 53.15843976527442, 52.951328960994125, 52.53710735243355, 52.24370037970314, 52.0020711080428, 51.86399723852262, 51.570590265792205, 51.08733172247153, 50.58681394546083, 50.5005177770107, 49.896444597859855, 49.74111149464964, 49.603037625129446, 49.171556782878845, 48.98170521228858, 48.84363134276838, 48.25681739730756, 47.91163272350708, 47.60096651708665, 47.08318950638592, 46.772523299965485, 46.496375560925095, 46.25474628926476, 45.89230238177425, 45.63341387642389, 45.11563686572316, 44.787711425612706, 44.54608215395236, 44.166379012771834, 43.54504659993096, 43.18260269244046, 42.664825681739735, 42.3541594753193, 41.57749395926821, 41.11149464963756, 40.61097687262686, 40.2140144977563, 40.04142216085606, 39.59268208491543, 39.02312737314463, 38.574387297204005, 38.004832585433206, 37.43527787366241, 37.021056265101834, 36.7794269934415, 36.33068691750086, 35.89920607525026, 35.31239212978944, 34.77735588539869, 34.190541939937866, 33.65550569554712, 33.258543320676566, 32.74076630997584, 32.257507766655166, 31.87780462547463, 31.567138419054196, 31.04936140835347, 30.497065930272694, 30.117362789092162, 29.73765964791163, 29.288919571971007, 28.564031756989994, 27.977217811529165, 27.597514670348634, 27.26958923023818, 27.027959958577842, 26.561960648947192], "calls": [0.0, 5.022437003797031, 7.973765964791163, 13.134276838108388, 17.397307559544355, 20.417673455298583, 25.198481187435277, 28.201587849499482, 31.774249223334483, 34.53572661373835, 38.160165688643424, 40.628236106316876, 43.4242319641008, 45.18467380048326, 46.15119088712461, 47.39385571280635, 48.826372109078356, 50.189851570590264, 51.74318260269244, 53.192958232654476, 54.59095616154642, 56.10976872626855, 57.248878149810146, 58.18087676907145, 58.92302381774249, 59.803244735933724, 60.16568864342423, 60.68346565412496, 61.52916810493614, 62.08146358301691, 63.06523990334829, 63.7901277183293, 64.37694166379013, 64.77390403866067, 65.7921988263721, 66.25819813600276, 67.138419054194, 67.6389368312047, 68.15671384190541, 68.55367621677597, 69.278564031757, 70.22782188470832, 71.12530203658957, 71.33241284086986, 72.48878149810149, 72.7821884708319, 73.0755954435623, 73.73144632378322, 74.12840869865379, 74.35277873662409, 75.16396272005524, 75.59544356230583, 76.00966517086641, 76.70003451846738, 77.13151536071798, 77.51121850189851, 77.89092164307905, 78.33966171901967, 78.66758715913014, 79.34069727304107, 79.80669658267173, 80.10010355540214, 80.58336209872282, 81.30824991370383, 81.68795305488436, 82.37832240248532, 82.82706247842596, 83.65550569554712, 84.17328270624785, 84.76009665170866, 85.22609596133933, 85.48498446668968, 85.9337245426303, 86.64135312392129, 87.15913013462202, 87.83224024853297, 88.43631342768381, 88.86779426993442, 89.14394200897479, 89.69623748705557, 90.17949603037624, 90.83534691059717, 91.40490162236796, 92.02623403520883, 92.56127027959958, 92.97549188816016, 93.51052813255092, 94.01104590956162, 94.39074905074214, 94.7186744908526, 95.25371073524336, 95.84052468070418, 96.27200552295479, 96.68622713151535, 97.15222644114601, 97.89437348981706, 98.53296513634795, 98.92992751121851, 99.27511218501898, 99.53400069036935, 100.0], "rescued": [0, 30, 47, 76, 95, 105, 137, 140, 160, 177, 193, 199, 207, 211, 215, 215, 215, 216, 222, 222, 223, 223, 223, 224, 225, 225, 225, 225, 225, 225, 226, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 228, 228, 228, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 56, 132, 243, 338, 403, 532, 629, 737, 811, 931, 1020, 1135, 1199, 1241, 1286, 1344, 1394, 1454, 1504, 1552, 1605, 1646, 1674, 1693, 1724, 1732, 1747, 1772, 1786, 1807, 1820, 1831, 1841, 1865, 1877, 1901, 1918, 1932, 1940, 1957, 1985, 2014, 2019, 2054, 2063, 2071, 2096, 2107, 2115, 2149, 2169, 2187, 2217, 2235, 2251, 2265, 2286, 2301, 2331, 2350, 2364, 2386, 2423, 2444, 2474, 2493, 2538, 2565, 2594, 2617, 2627, 2653, 2686, 2712, 2745, 2778, 2802, 2816, 2844, 2869, 2903, 2934, 2968, 2999, 3022, 3052, 3080, 3102, 3120, 3150, 3182, 3205, 3227, 3253, 3295, 3329, 3351, 3370, 3384, 3411], "exact_score": [81.429064549534, 81.41180531584398, 81.2909906800138, 81.20469451156369, 81.15291681049361, 80.9975837072834, 80.4970659302727, 80.30721435968243, 79.96202968588196, 79.58232654470142, 79.30617880566103, 79.03003106662064, 78.87469796341043, 78.56403175698999, 78.33966171901969, 78.06351397997929, 77.68381083879876, 77.37314463237833, 77.09699689333793, 76.78633068691751, 76.47566448049706, 76.06144287193649, 75.66448049706594, 75.45736969278565, 75.02588885053504, 74.73248187780463, 74.24922333448394, 73.73144632378322, 73.12737314463237, 72.60959613393166, 72.16085605799103, 71.7811529168105, 71.40144977562997, 71.02174663444943, 70.67656196064895, 70.33137728684846, 69.72730410769762, 69.278564031757, 68.7780462547463, 68.1394546082154, 67.60441836382465, 67.05212288574387, 66.43079047290301, 65.68864342423197, 65.01553331032103, 64.46323783224025, 63.7901277183293, 63.20331377286848, 62.4956851915775, 61.822575077666556, 61.23576113220574, 60.64894718674492, 60.06213324128409, 59.423541594753196, 58.83672764929238, 58.2844321712116, 57.6458405246807, 57.024508111839836, 56.48947186744908, 56.05799102519848, 55.5056955471177, 54.97065930272696, 54.504659993096304, 54.193993786675875, 53.88332758025545, 53.46910597169485, 53.08940283051433, 52.60614428719366, 52.071108042802905, 51.69140490162236, 51.20814635830169, 50.690369347600964, 50.20711080428028, 49.63755609250949, 49.05074214704867, 48.3603727994477, 47.618225750776666, 46.841560234725584, 46.18570935450466, 45.39178460476355, 44.63237832240249, 43.76941663790128, 42.95823265447014, 42.181567138419055, 41.266827752847774, 40.45564376941664, 39.54090438384536, 38.72972040041422, 37.866758715913015, 36.90024162927166, 36.089057645840526, 35.174318260269246, 34.22506040731791, 33.24128408698654, 32.274767000345186, 31.29099068001381, 30.376251294442525, 29.4269934414912, 28.460476354849852, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 59.92367212627205, "threshold_area100": 48.65645495340007, "threshold_area25": 66.98136002761478}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.429064549534, "calls": 0.6731101139109423}}, "old_normalized_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.429064549534, "calls": 0.6731101139109423}}, "old_normalized_endpoints": {"val": {"score": 98.58333333333333, "calls": 0.0}, "test": {"score": 81.429064549534, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.25, 98.25, 98.16666666666667, 98.0, 97.91666666666666, 97.5, 97.25, 97.08333333333333, 96.75, 96.41666666666666, 96.5, 96.08333333333333, 95.41666666666667, 95.0, 94.83333333333334, 94.25, 93.5, 93.08333333333333, 92.58333333333333, 92.0, 91.25, 90.33333333333333, 89.58333333333334, 88.75, 88.16666666666667, 87.41666666666667, 86.91666666666666, 86.0, 85.16666666666667, 84.5, 83.58333333333333, 82.75, 81.83333333333334, 81.0, 80.0, 79.16666666666667, 78.33333333333333, 77.66666666666666, 76.91666666666667, 76.0, 75.08333333333333, 74.16666666666667, 73.25, 72.33333333333334, 71.41666666666666, 70.41666666666666, 69.58333333333333, 68.91666666666667, 67.91666666666667, 67.0, 66.08333333333334, 65.16666666666666, 64.25, 63.5, 62.5, 61.5, 60.5, 59.66666666666667, 58.666666666666664, 57.75, 56.666666666666664, 55.666666666666664, 54.666666666666664, 53.91666666666667, 52.916666666666664, 51.916666666666664, 50.916666666666664, 50.0, 49.083333333333336, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.25, 98.25, 98.16666666666667, 98.0, 97.91666666666666, 97.5, 97.25, 97.08333333333333, 96.75, 96.41666666666666, 96.5, 96.08333333333333, 95.41666666666667, 95.0, 94.83333333333334, 94.25, 93.5, 93.08333333333333, 92.58333333333333, 92.0, 91.25, 90.33333333333333, 89.58333333333334, 88.75, 88.16666666666667, 87.41666666666667, 86.91666666666666, 86.0, 85.16666666666667, 84.5, 83.58333333333333, 82.66666666666667, 81.83333333333334, 81.0, 80.0, 79.16666666666667, 78.33333333333333, 77.66666666666666, 76.91666666666667, 76.0, 75.08333333333333, 74.16666666666667, 73.25, 72.33333333333334, 71.41666666666666, 70.41666666666666, 69.58333333333333, 68.83333333333333, 67.91666666666667, 67.0, 66.08333333333334, 65.16666666666666, 64.25, 63.5, 62.5, 61.5, 60.5, 59.66666666666667, 58.666666666666664, 57.666666666666664, 56.666666666666664, 55.666666666666664, 54.666666666666664, 53.91666666666667, 52.916666666666664, 51.916666666666664, 50.916666666666664, 50.0, 49.083333333333336, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.44999999999999, "exact_area100": 71.56166666666667}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.39454608215395, 81.429064549534, 81.429064549534, 81.36002761477391, 81.41180531584398, 81.30824991370383, 81.25647221263375, 81.10113910942354, 81.04936140835348, 80.82499137038316, 80.68691750086296, 80.44528822920263, 79.97928891957199, 79.72040041422161, 79.28891957197101, 78.84017949603039, 78.49499482222988, 77.82188470831896, 77.07973765964792, 76.18225750776666, 75.45736969278565, 74.7842595788747, 74.23196410079392, 73.558853986883, 73.09285467725233, 72.43700379703142, 71.60856057991025, 70.2105626510183, 69.79634104245773, 69.26130479806697, 67.91508457024509, 67.01760441836383, 66.5688643424232, 65.77493959268207, 64.5667932343804, 63.75560925094926, 63.18605453917846, 62.87538833275803, 61.75353814290645, 60.925094925785295, 60.303762512944424, 59.285467725233, 58.025543665861235, 57.33517431826027, 56.16154642733863, 55.712806351397994, 55.16051087331723, 54.694511563686575, 53.952364515015525, 53.227476700034515, 52.951328960994125, 52.41629271660339, 51.60510873317226, 50.58681394546083, 50.20711080428028, 49.27511218501899, 48.41215050051778, 47.94615119088713, 46.96237487055575, 46.51363479461512, 45.702450811183986, 44.97756299620297, 44.080082844321716, 43.665861235761135, 43.23438039351053, 42.7166033828098, 42.267863306869174, 41.750086296168455, 41.31860545391785, 40.95616154642734, 39.972385226095966, 39.282015878494995, 38.557128063513986, 38.31549879185364, 37.60787021056265, 37.12461166724197, 36.382464618570935, 35.5022437003797, 34.77735588539869, 34.5012081463583, 33.74180186399724, 32.72350707628581, 32.03313772868485, 31.687953054884368, 31.10113910942355, 30.46254746289265, 30.031066620642044, 29.444252675181225, 29.133586468760786, 28.61580945806006, 28.011736278909215, 27.33862616499828, 26.561960648947192], "calls": [0.0, 0.6731101139109423, 1.1045909561615463, 2.0365895754228514, 2.6406627545736967, 3.089402830514325, 3.917846047635485, 5.471177079737659, 6.765619606489472, 7.870210562651018, 8.715913013462202, 10.52813255091474, 12.15050051777701, 13.410424577148774, 14.60131170176044, 15.67138419054194, 16.17190196755264, 17.86330686917501, 18.74352778736624, 19.986192613047983, 22.195374525371072, 23.783224024853297, 24.887814981014845, 25.957887469796344, 27.338626164998274, 29.219882637210908, 30.669658267172938, 32.982395581636176, 35.01898515705902, 36.90024162927166, 38.298239558163615, 39.575422851225404, 40.88712461166724, 42.06075250258888, 43.54504659993096, 45.96133931653434, 46.89333793579565, 47.70452191922679, 49.98274076630997, 51.43251639627201, 52.17466344494305, 53.31377286848464, 54.918881601656885, 56.19606489471868, 57.093545046599935, 57.54228512254056, 59.11287538833275, 60.510873317224714, 61.52916810493614, 63.16879530548844, 64.94649637556093, 65.80945806006213, 67.19019675526407, 67.7079737659648, 68.3638246461857, 69.0024162927166, 69.98619261304798, 70.81463583016914, 71.15982050396963, 71.79841215050051, 72.71315153607179, 73.9212978943735, 74.31826026924404, 75.47462892647566, 76.51018294787711, 77.06247842595789, 78.13255091473938, 78.66758715913014, 79.4960303762513, 80.37625129444253, 81.41180531584398, 81.82602692440456, 82.34380393510529, 82.89609941318605, 83.39661719019675, 83.96617190196756, 84.5012081463583, 84.86365205384881, 85.88194684156024, 86.60683465654125, 87.38350017259233, 87.62512944425268, 88.40179496030376, 88.90231273731446, 89.7134967207456, 90.61097687262686, 91.38764238867795, 91.71556782878841, 92.50949257852952, 93.56230583362098, 94.26993441491199, 94.66689678978253, 95.28822920262341, 95.97859855022438, 96.427338626165, 97.0659302726959, 97.37659647911633, 97.94615119088712, 98.55022437003798, 99.22333448394892, 100.0], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.429064549534, 81.429064549534, 81.37728684846392, 81.37728684846392, 81.39454608215395, 81.32550914739386, 81.27373144632378, 81.23921297894374, 81.04936140835348, 80.9285467725233, 80.82499137038316, 80.60062133241284, 80.44528822920263, 80.18639972385226, 80.06558508802209, 79.89299275112185, 79.70314118053159, 79.25440110459095, 78.84017949603039, 78.59855022437004, 78.28788401794961, 77.92544011045909, 77.47670003451846, 76.9244045564377, 76.57921988263722, 76.18225750776666, 75.83707283396618, 75.45736969278565, 75.12944425267519, 74.73248187780463, 74.37003797031412, 73.86952019330342, 73.38626164998274, 73.04107697618227, 72.47152226441146, 71.93648602002072, 71.45322747670004, 70.79737659647913, 70.19330341732827, 69.7100448740076, 69.08871246116672, 68.5018985157059, 67.89782533655506, 67.27649292371419, 66.68967897825337, 66.03382809803244, 65.23990334829134, 64.54953400069037, 63.91094235415948, 63.20331377286848, 62.58198136002762, 61.822575077666556, 61.32205730065585, 60.66620642043493, 59.95857783914394, 59.38902312737315, 58.64687607870211, 58.00828443217121, 57.21435968243011, 56.316879530548846, 55.45391784604764, 54.694511563686575, 53.952364515015525, 53.08940283051433, 52.295478080773215, 51.380738695201934, 50.517777010700726, 49.620296858819465, 48.80911287538833, 48.01518812564722, 47.08318950638592, 46.18570935450466, 45.32274767000345, 44.45978598550225, 43.51052813255092, 42.63030721435969, 41.73282706247843, 40.83534691059717, 39.86882982395582, 38.90231273731447, 37.953054884363134, 37.03831549879186, 36.10631687953055, 35.1397997928892, 34.22506040731791, 33.27580255436659, 32.29202623403521, 31.39454608215395, 30.445288229202628, 29.513289609941317, 28.564031756989994, 27.56299620296859, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 80.712806351398, "exact_area100": 63.206904730102394}}, "old_normalized_endpoint_budget": 0}, {"seed": 2028, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.25, "validation_area100": 0.7168249999999999, "training_seconds": 3.4497901590075344, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.01179835725550838, "operator": "ge"}, {"budget": 0.02, "threshold": -0.014533522871909572, "operator": "ge"}, {"budget": 0.03, "threshold": -0.017304103523839015, "operator": "ge"}, {"budget": 0.04, "threshold": -0.021346742514084555, "operator": "ge"}, {"budget": 0.05, "threshold": -0.024336744729242316, "operator": "ge"}, {"budget": 0.06, "threshold": -0.0308659752736493, "operator": "ge"}, {"budget": 0.07, "threshold": -0.0408423469469797, "operator": "ge"}, {"budget": 0.08, "threshold": -0.04839185939282904, "operator": "ge"}, {"budget": 0.09, "threshold": -0.05798189670698013, "operator": "ge"}, {"budget": 0.1, "threshold": -0.06603599800136048, "operator": "ge"}, {"budget": 0.11, "threshold": -0.08036929771433263, "operator": "ge"}, {"budget": 0.12, "threshold": -0.09864868698229494, "operator": "ge"}, {"budget": 0.13, "threshold": -0.12482637331456285, "operator": "ge"}, {"budget": 0.14, "threshold": -0.15684194896835071, "operator": "ge"}, {"budget": 0.15, "threshold": -0.1784685082364398, "operator": "ge"}, {"budget": 0.16, "threshold": -0.2139165821093056, "operator": "ge"}, {"budget": 0.17, "threshold": -0.23983913607532734, "operator": "ge"}, {"budget": 0.18, "threshold": -0.29389559537847804, "operator": "ge"}, {"budget": 0.19, "threshold": -0.35149730718645567, "operator": "ge"}, {"budget": 0.2, "threshold": -0.38560151355096994, "operator": "ge"}, {"budget": 0.21, "threshold": -0.433390566996234, "operator": "ge"}, {"budget": 0.22, "threshold": -0.4965540893287694, "operator": "ge"}, {"budget": 0.23, "threshold": -0.5411856623455242, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5953951609750348, "operator": "ge"}, {"budget": 0.25, "threshold": -0.6320675347592546, "operator": "ge"}, {"budget": 0.26, "threshold": -0.6578226107285622, "operator": "ge"}, {"budget": 0.27, "threshold": -0.696861532140154, "operator": "ge"}, {"budget": 0.28, "threshold": -0.7133649100036981, "operator": "ge"}, {"budget": 0.29, "threshold": -0.7405943128943695, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7645436999920158, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8005623502925919, "operator": "ge"}, {"budget": 0.32, "threshold": -0.8222804826284887, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8413586300411672, "operator": "ge"}, {"budget": 0.34, "threshold": -0.8579251265173008, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8727108298675155, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8854381592252072, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9008823988753775, "operator": "ge"}, {"budget": 0.38, "threshold": -0.907228263835764, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9198831982370881, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9255601995199628, "operator": "ge"}, {"budget": 0.41, "threshold": -0.931131899586946, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9381149382966757, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9425226113942892, "operator": "ge"}, {"budget": 0.44, "threshold": -0.950111975057705, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9553942827663738, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9592002068111299, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9630444740674866, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9654792653845279, "operator": "ge"}, {"budget": 0.49, "threshold": -0.967493300067489, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9691599665961491, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9714908722115986, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9733939579497831, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9753460683973438, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9766384364470242, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9783221804978945, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9805655197830258, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9820071138028872, "operator": "ge"}, {"budget": 0.58, "threshold": -0.983018588347588, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9840955955054168, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9851309442325976, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9864905764144944, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9870601758599422, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9882817684201834, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9889571050125745, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9897336771092945, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9903810776757175, "operator": "ge"}, {"budget": 0.67, "threshold": -0.990793951908875, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9911760195393088, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9915720389484458, "operator": "ge"}, {"budget": 0.7, "threshold": -0.992068558417655, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9925863842733139, "operator": "ge"}, {"budget": 0.72, "threshold": -0.993478276497567, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9940762575143848, "operator": "ge"}, {"budget": 0.74, "threshold": -0.994659824152615, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9952069681237655, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9956169406644777, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9959626503983041, "operator": "ge"}, {"budget": 0.78, "threshold": -0.996221177365996, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9965991483781115, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9969750625762551, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9972612995120375, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9974020811085864, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9976771840644971, "operator": "ge"}, {"budget": 0.84, "threshold": -0.997972964998184, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9981322519617335, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9983263242070639, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9984632924285705, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9985891629170458, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9986682898989427, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9988219650213799, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9989325459599627, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9990589255279932, "operator": "ge"}, {"budget": 0.93, "threshold": -0.999157861063165, "operator": "ge"}, {"budget": 0.94, "threshold": -0.999214745745321, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9993073948333763, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9993910448949025, "operator": "ge"}, {"budget": 0.97, "threshold": -0.999488624355106, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9995815537911509, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9997015796121921, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.41666666666666, 98.16666666666667, 98.0, 97.83333333333334, 97.66666666666667, 97.5, 96.91666666666666, 96.75, 96.75, 96.41666666666666, 96.33333333333334, 96.25, 96.25, 95.75, 95.33333333333334, 94.91666666666667, 94.33333333333334, 93.66666666666667, 93.25, 92.75, 92.16666666666666, 91.58333333333333, 90.75, 90.0, 89.41666666666667, 88.5, 88.0, 87.16666666666667, 86.41666666666666, 85.5, 84.58333333333333, 83.83333333333334, 83.0, 82.0, 81.08333333333333, 80.33333333333333, 79.58333333333334, 78.83333333333333, 77.91666666666667, 77.0, 76.0, 75.25, 74.33333333333334, 73.58333333333333, 72.75, 72.08333333333333, 71.08333333333333, 70.08333333333334, 69.16666666666667, 68.33333333333333, 67.33333333333333, 66.33333333333333, 65.41666666666667, 64.5, 63.583333333333336, 62.66666666666667, 61.66666666666667, 60.74999999999999, 59.833333333333336, 58.833333333333336, 58.08333333333333, 57.00000000000001, 56.08333333333333, 55.08333333333333, 54.08333333333333, 53.08333333333333, 52.08333333333333, 51.16666666666667, 50.16666666666667, 49.16666666666667, 48.25000000000001, 47.25, 46.25, 45.333333333333336, 44.25, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.416666666666664, 33.41666666666667, 32.41666666666667, 31.41666666666667, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 2, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 5, 7, 9, 11, 13, 20, 22, 22, 26, 27, 28, 28, 34, 39, 44, 51, 60, 66, 72, 79, 87, 97, 106, 113, 124, 131, 141, 150, 161, 172, 181, 191, 203, 214, 223, 232, 241, 252, 263, 275, 284, 295, 305, 315, 324, 336, 348, 359, 369, 381, 393, 404, 415, 426, 437, 449, 460, 471, 483, 492, 505, 516, 528, 540, 552, 564, 575, 587, 599, 610, 622, 634, 645, 658, 670, 682, 693, 704, 716, 728, 740, 752, 764, 776, 788, 800, 812, 822, 833, 844, 856], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.41666666666666, 98.16666666666667, 98.0, 97.83333333333334, 97.66666666666667, 97.5, 96.91666666666666, 96.75, 96.75, 96.41666666666666, 96.33333333333334, 96.25, 96.25, 95.75, 95.33333333333334, 94.91666666666667, 94.33333333333334, 93.66666666666667, 93.25, 92.75, 92.16666666666666, 91.58333333333333, 90.75, 90.0, 89.41666666666667, 88.5, 88.0, 87.16666666666667, 86.41666666666666, 85.5, 84.58333333333333, 83.83333333333334, 82.91666666666666, 82.0, 81.08333333333333, 80.33333333333333, 79.58333333333334, 78.83333333333333, 77.91666666666667, 77.0, 76.0, 75.25, 74.33333333333334, 73.58333333333333, 72.75, 72.08333333333333, 71.08333333333333, 70.08333333333334, 69.08333333333334, 68.33333333333333, 67.33333333333333, 66.33333333333333, 65.41666666666667, 64.5, 63.583333333333336, 62.66666666666667, 61.66666666666667, 60.74999999999999, 59.833333333333336, 58.833333333333336, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.08333333333333, 54.08333333333333, 53.08333333333333, 52.08333333333333, 51.16666666666667, 50.16666666666667, 49.16666666666667, 48.25000000000001, 47.25, 46.25, 45.25, 44.25, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.416666666666664, 33.41666666666667, 32.41666666666667, 31.41666666666667, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.68249999999999, "threshold_area100": 71.68583333333333, "threshold_area25": 97.275}, "test": {"score": [81.429064549534, 81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.41180531584398, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.36002761477391, 81.32550914739386, 81.23921297894374, 81.17017604418363, 81.0666206420435, 80.94580600621333, 80.89402830514325, 80.53158439765275, 80.22091819123231, 79.996548153262, 79.66862271315154, 79.13358646876078, 78.87469796341043, 78.47773558853987, 77.99447704521918, 77.63203313772868, 76.70003451846739, 76.25129444252676, 75.80255436658612, 74.99137038315499, 73.99033482913359, 72.98929927511219, 72.17811529168105, 71.24611667241975, 70.40041422160857, 69.72730410769762, 68.34656541249569, 67.65619606489473, 66.2927166033828, 65.7921988263721, 65.01553331032103, 64.0662754573697, 63.41042457714878, 62.37487055574733, 61.339316534345876, 60.510873317224714, 59.56161546427339, 58.871246116672424, 58.19813600276148, 57.749395926820846, 57.024508111839836, 56.3513979979289, 55.66102865032793, 55.212288574387294, 54.26303072143597, 53.52088367276493, 52.81325509147394, 52.20918191232309, 51.69140490162236, 50.880220918191235, 49.77562996202969, 49.29237141870901, 48.187780462547465, 47.54918881601657, 46.75526406627546, 46.23748705557473, 45.719710044874006, 45.40904383845357, 44.8567483603728, 44.54608215395236, 43.99378667587159, 43.01001035554022, 42.2851225405592, 41.819123230928554, 41.042457714877465, 40.40386606834657, 39.851570590265794, 39.420089748015194, 38.67794269934415, 37.97031411805316, 37.43527787366241, 37.141870900932, 36.36520538488091, 35.5712806351398, 35.08802209181913, 34.41491197790818, 33.77632033137729, 33.29306178805661, 32.89609941318606, 32.30928546772523, 31.72247152226441, 30.997583707283393, 30.307214359682433, 29.944770452191925, 29.323438039351057, 28.857438729720396, 28.20158784949949, 27.683810838798763, 27.09699689333793, 26.561960648947192], "calls": [0.0, 0.535036244390749, 1.173627890921643, 1.8467380048325857, 2.6924404556437693, 3.3137728684846395, 4.884363134276838, 6.4204349326889885, 7.4387297204004135, 8.612357611322057, 9.31998619261305, 10.493614083534691, 11.667241974456335, 13.065239903348292, 14.325163962720055, 15.25716258198136, 16.448049706593025, 17.224715222644114, 19.019675526406626, 20.538488091128755, 21.384190541939937, 22.644114601311703, 24.456334138764237, 25.681739730755954, 27.32136693130825, 28.581290990680014, 29.616845012081463, 31.653434587504314, 32.61995167414567, 34.03520883672765, 35.605799102519846, 38.26372109078357, 40.00690369347601, 41.542975491888164, 43.07904729030031, 44.75319295823266, 45.926820849154296, 48.32585433206766, 49.326889886089056, 51.656886434242324, 52.62340352088367, 53.96962374870555, 55.26406627545737, 56.23058336209872, 58.19813600276148, 59.66517086641353, 60.94235415947532, 62.150500517777004, 62.96168450120815, 63.876423886779435, 64.51501553331032, 65.41249568519157, 66.32723507076285, 67.31101139109424, 67.94960303762512, 69.10597169485675, 70.08974801518812, 71.0044874007594, 71.66033828098033, 72.4197445633414, 73.43803935105281, 74.7152226441146, 75.26751812219537, 76.51018294787711, 77.2178115291681, 78.13255091473938, 78.80566102865033, 79.37521574042113, 79.80669658267173, 80.37625129444253, 80.80773213669313, 81.429064549534, 82.60269244045564, 83.39661719019675, 83.89713496720745, 84.72557818432863, 85.38142906454954, 85.95098377632033, 86.39972385226096, 87.15913013462202, 87.90127718329306, 88.52260959613393, 88.83327580255437, 89.66171901967553, 90.45564376941664, 90.97342078011737, 91.73282706247844, 92.44045564376941, 92.95823265447014, 93.37245426303072, 94.02830514325164, 94.63237832240249, 95.4263030721436, 96.11667241974456, 96.47911632723508, 97.13496720745599, 97.61822575077666, 98.27407663099758, 98.79185364169831, 99.46496375560925, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 4, 5, 5, 5, 5, 8, 11, 12, 13, 14, 16, 21, 23, 24, 27, 40, 46, 55, 59, 65, 74, 78, 86, 93, 104, 110, 115, 120, 126, 130, 144, 144, 153, 158, 165, 168, 168, 182, 187, 191, 196, 198, 198, 199, 201, 203, 205, 205, 207, 210, 211, 211, 212, 214, 216, 217, 218, 218, 218, 220, 220, 221, 221, 224, 224, 225, 225, 226, 226, 226, 226, 226, 226, 226, 227, 227, 228, 228, 228, 228, 229, 229, 229, 230, 230, 231, 231, 231, 231, 231, 231, 231, 232, 232], "harmed": [0, 0, 1, 1, 1, 2, 2, 3, 5, 6, 7, 9, 14, 22, 27, 34, 42, 47, 73, 93, 107, 129, 173, 194, 226, 258, 285, 348, 378, 412, 466, 535, 599, 651, 710, 765, 808, 902, 942, 1030, 1064, 1116, 1174, 1212, 1286, 1351, 1403, 1463, 1505, 1544, 1571, 1615, 1656, 1698, 1724, 1781, 1827, 1869, 1904, 1935, 1984, 2050, 2079, 2144, 2181, 2227, 2259, 2289, 2308, 2340, 2361, 2393, 2451, 2493, 2521, 2566, 2603, 2635, 2660, 2703, 2744, 2776, 2793, 2839, 2885, 2913, 2952, 2990, 3018, 3041, 3076, 3110, 3153, 3193, 3214, 3250, 3277, 3315, 3345, 3380, 3411], "exact_score": [81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.429064549534, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.34276838108389, 81.2909906800138, 81.23921297894374, 81.17017604418363, 81.08387987573352, 80.96306523990336, 80.89402830514325, 80.70417673455299, 80.53158439765275, 80.30721435968243, 80.06558508802209, 79.91025198481188, 79.54780807732136, 79.30617880566103, 78.97825336555057, 78.78840179496031, 78.51225405591991, 78.23610631687953, 77.77010700724888, 77.44218156713842, 76.9244045564377, 76.52744218156714, 76.13047980669658, 75.81981360027615, 75.33655505695546, 74.83603727994476, 74.49085260614429, 74.0766309975837, 73.558853986883, 72.98929927511219, 72.4197445633414, 71.9192267863307, 71.33241284086986, 70.7801173627891, 70.26234035208837, 69.69278564031757, 69.05419399378667, 68.51915774939593, 67.82878840179495, 67.39730755954436, 66.74145667932343, 66.2064204349327, 65.63686572316189, 64.998274076631, 64.29064549534, 63.61753538142907, 63.099758370728345, 62.53020365895754, 61.80531584397653, 61.114946496375566, 60.45909561615465, 59.66517086641353, 58.8539868829824, 58.12909906800139, 57.3524335519503, 56.55850880220918, 55.88539868829825, 55.17777010700725, 54.383845357266146, 53.624439074905084, 52.81325509147394, 51.96755264066275, 51.294442526751816, 50.4314808422506, 49.55125992405937, 48.63652053848809, 47.75629962029686, 46.89333793579566, 46.04763548498447, 45.219192267863306, 44.373489817052125, 43.579565067311016, 42.64756644804971, 41.73282706247843, 40.7835692095271, 39.81705212288574, 38.850535036244395, 37.90127718329306, 36.98653779772179, 36.037279944770454, 35.08802209181913, 34.173282706247846, 33.258543320676566, 32.34380393510528, 31.411805315843978, 30.4280289955126, 29.461511908871252, 28.477735588539865, 27.52847773558854, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 63.01273537227816, "threshold_area100": 57.909993096306536, "threshold_area25": 80.71280635139799}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0026866754051297903, "operator": "ge"}, {"budget": 0.02, "threshold": -0.003547861473634839, "operator": "ge"}, {"budget": 0.03, "threshold": -0.004479917697608471, "operator": "ge"}, {"budget": 0.04, "threshold": -0.005928820930421352, "operator": "ge"}, {"budget": 0.05, "threshold": -0.007231207098811865, "operator": "ge"}, {"budget": 0.06, "threshold": -0.009832097217440605, "operator": "ge"}, {"budget": 0.07, "threshold": -0.014234333299100399, "operator": "ge"}, {"budget": 0.08, "threshold": -0.018372630700469017, "operator": "ge"}, {"budget": 0.09, "threshold": -0.023244865238666534, "operator": "ge"}, {"budget": 0.1, "threshold": -0.02991613931953907, "operator": "ge"}, {"budget": 0.11, "threshold": -0.03643512725830078, "operator": "ge"}, {"budget": 0.12, "threshold": -0.0478639155626297, "operator": "ge"}, {"budget": 0.13, "threshold": -0.06544563919305801, "operator": "ge"}, {"budget": 0.14, "threshold": -0.08458636701107025, "operator": "ge"}, {"budget": 0.15, "threshold": -0.10048224776983261, "operator": "ge"}, {"budget": 0.16, "threshold": -0.127938911318779, "operator": "ge"}, {"budget": 0.17, "threshold": -0.14905540645122528, "operator": "ge"}, {"budget": 0.18, "threshold": -0.19924531877040863, "operator": "ge"}, {"budget": 0.19, "threshold": -0.2506515085697174, "operator": "ge"}, {"budget": 0.2, "threshold": -0.29002973437309265, "operator": "ge"}, {"budget": 0.21, "threshold": -0.3280066251754761, "operator": "ge"}, {"budget": 0.22, "threshold": -0.39535462856292725, "operator": "ge"}, {"budget": 0.23, "threshold": -0.44462817907333374, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5087196230888367, "operator": "ge"}, {"budget": 0.25, "threshold": -0.562032163143158, "operator": "ge"}, {"budget": 0.26, "threshold": -0.6048153638839722, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6400593519210815, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6704739332199097, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6888324618339539, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7365207672119141, "operator": "ge"}, {"budget": 0.31, "threshold": -0.7689455151557922, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7857252359390259, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8099433183670044, "operator": "ge"}, {"budget": 0.34, "threshold": -0.8266644477844238, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8482482433319092, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8650665879249573, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8779292106628418, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8931683301925659, "operator": "ge"}, {"budget": 0.39, "threshold": -0.901085615158081, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9102094173431396, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9192646741867065, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9243045449256897, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9309571385383606, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9382821321487427, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9437500834465027, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9509193301200867, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9538823962211609, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9570133686065674, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9589731097221375, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9619607925415039, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9651693105697632, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9665360450744629, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9683898091316223, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9711535573005676, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9732761383056641, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9752033948898315, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9767856597900391, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9783799648284912, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9794123768806458, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9810121059417725, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9825423955917358, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9840716123580933, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9852109551429749, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9861676692962646, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9868257641792297, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9875472187995911, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9880587458610535, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9883893132209778, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9892231822013855, "operator": "ge"}, {"budget": 0.7, "threshold": -0.98981112241745, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9909319877624512, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9914804697036743, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9921431541442871, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9931932687759399, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9937430620193481, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9944360852241516, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9947589635848999, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9950308799743652, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9956059455871582, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9960572719573975, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9964202642440796, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9966936707496643, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9969604015350342, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9973244071006775, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9975545406341553, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9978289008140564, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9980463981628418, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9981430768966675, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9982336163520813, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9984338283538818, "operator": "ge"}, {"budget": 0.91, "threshold": -0.998583197593689, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9987532496452332, "operator": "ge"}, {"budget": 0.93, "threshold": -0.998878002166748, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9989762902259827, "operator": "ge"}, {"budget": 0.95, "threshold": -0.999086320400238, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9991893172264099, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9993206858634949, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9994428753852844, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9996052384376526, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.41666666666666, 98.16666666666667, 98.0, 97.83333333333334, 97.75, 97.5, 96.91666666666666, 96.75, 96.75, 96.41666666666666, 96.33333333333334, 96.25, 96.25, 95.75, 95.5, 94.83333333333334, 94.33333333333334, 93.58333333333333, 93.25, 92.58333333333333, 92.16666666666666, 91.66666666666666, 90.83333333333333, 90.25, 89.41666666666667, 88.58333333333334, 87.75, 87.08333333333333, 86.08333333333333, 85.5, 84.83333333333334, 83.91666666666666, 83.08333333333333, 82.0, 81.16666666666667, 80.16666666666666, 79.5, 78.83333333333333, 77.83333333333333, 76.91666666666667, 76.16666666666667, 75.25, 74.41666666666666, 73.58333333333333, 72.75, 71.75, 70.75, 69.91666666666667, 69.0, 68.33333333333333, 67.33333333333333, 66.33333333333333, 65.41666666666667, 64.58333333333333, 63.66666666666667, 62.66666666666667, 61.66666666666667, 60.83333333333334, 59.833333333333336, 59.00000000000001, 58.08333333333333, 57.08333333333333, 56.08333333333333, 55.08333333333333, 54.08333333333333, 53.08333333333333, 52.08333333333333, 51.16666666666667, 50.16666666666667, 49.16666666666667, 48.25000000000001, 47.25, 46.25, 45.333333333333336, 44.25, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.416666666666664, 33.41666666666667, 32.41666666666667, 31.41666666666667, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 5, 7, 9, 10, 13, 20, 22, 22, 26, 27, 28, 28, 34, 37, 45, 51, 60, 64, 72, 79, 86, 96, 103, 113, 123, 133, 141, 153, 161, 169, 180, 190, 203, 213, 225, 233, 241, 253, 264, 273, 284, 294, 304, 314, 326, 338, 349, 360, 369, 381, 393, 404, 414, 425, 437, 449, 459, 471, 481, 492, 504, 516, 528, 540, 552, 564, 575, 587, 599, 610, 622, 634, 645, 658, 670, 682, 693, 704, 716, 728, 740, 752, 764, 776, 788, 800, 812, 822, 833, 844, 856], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.41666666666666, 98.16666666666667, 98.0, 97.83333333333334, 97.75, 97.5, 96.91666666666666, 96.75, 96.75, 96.41666666666666, 96.33333333333334, 96.25, 96.25, 95.75, 95.5, 94.83333333333334, 94.33333333333334, 93.58333333333333, 93.25, 92.58333333333333, 92.16666666666666, 91.66666666666666, 90.83333333333333, 90.25, 89.41666666666667, 88.58333333333334, 87.75, 87.08333333333333, 86.08333333333333, 85.5, 84.83333333333334, 83.91666666666666, 83.0, 82.0, 81.16666666666667, 80.16666666666666, 79.5, 78.83333333333333, 77.83333333333333, 76.91666666666667, 76.16666666666667, 75.25, 74.41666666666666, 73.58333333333333, 72.75, 71.75, 70.75, 69.91666666666667, 68.91666666666667, 68.33333333333333, 67.33333333333333, 66.33333333333333, 65.41666666666667, 64.58333333333333, 63.66666666666667, 62.66666666666667, 61.66666666666667, 60.83333333333334, 59.833333333333336, 59.00000000000001, 58.00000000000001, 57.08333333333333, 56.08333333333333, 55.08333333333333, 54.08333333333333, 53.08333333333333, 52.08333333333333, 51.16666666666667, 50.16666666666667, 49.16666666666667, 48.25000000000001, 47.25, 46.25, 45.25, 44.25, 43.25, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.416666666666664, 33.41666666666667, 32.41666666666667, 31.41666666666667, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.67833333333333, "threshold_area100": 71.68166666666667, "threshold_area25": 97.275}, "test": {"score": [81.429064549534, 81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.22195374525371, 81.15291681049361, 81.04936140835348, 80.9975837072834, 80.96306523990336, 80.63513979979288, 80.39351052813255, 80.15188125647221, 79.9275112185019, 79.44425267518123, 79.21988263721092, 78.73662409389023, 78.25336555056955, 77.66655160510874, 77.30410769761824, 76.82084915429755, 76.54470141525717, 75.18122195374526, 74.43907490507422, 73.71418709009319, 73.00655850880221, 72.17811529168105, 71.33241284086986, 70.24508111839835, 69.31308249913704, 67.94960303762512, 67.41456679323439, 66.55160510873317, 65.4987918536417, 65.05005177770107, 64.03175698998965, 62.96168450120815, 62.202278218847084, 60.75250258888505, 60.26924404556438, 59.630652399033494, 59.302726958923024, 58.47428374180187, 57.61132205730066, 57.19710044874008, 56.679323438039354, 55.816361753538146, 55.03969623748706, 54.47014152571625, 53.624439074905084, 52.89955125992406, 52.41629271660339, 51.39799792889196, 50.379703141180535, 49.4994822229893, 48.774594408008284, 48.08422506040732, 47.566448049706594, 47.04867103900587, 46.565412495685194, 46.23748705557473, 45.47808077321367, 45.02934069727304, 44.06282361063169, 43.579565067311016, 42.9237141870901, 41.93993786675872, 41.37038315498792, 40.6454953400069, 40.17949603037626, 39.90334829133587, 38.97134967207456, 38.19468415602347, 37.60787021056265, 37.10735243355195, 36.520538488091134, 35.84742837418019, 35.22609596133932, 34.38039351052814, 33.60372799447705, 33.36209872281671, 33.03417328270625, 32.39558163617536, 31.860545391784605, 31.04936140835347, 30.4280289955126, 29.910251984811875, 29.409734207801176, 28.874697963410423, 28.23610631687953, 27.701070072488786, 27.114256127027957, 26.561960648947192], "calls": [0.0, 0.535036244390749, 1.156368657231619, 1.794960303762513, 2.5888850535036245, 3.296513634794615, 4.470141525716258, 5.97169485674836, 6.972730410769762, 8.025543665861235, 8.940283051432516, 9.665170866413533, 10.942354159475318, 12.581981360027614, 13.738350017259235, 14.791163272350708, 15.9993096306524, 16.741456679323438, 18.32930617880566, 19.934414911977907, 20.831895063859164, 21.763893683120468, 23.248187780462548, 24.076630997583706, 25.785295132896096, 26.99344149119779, 28.788401794960304, 29.806696582671727, 31.18743527787366, 31.929582326544704, 34.62202278218847, 36.41698308595098, 37.91853641698309, 39.62720055229548, 41.042457714877465, 42.87193648602002, 44.511563686572316, 46.13393165343459, 48.13600276147739, 49.15429754918881, 50.793924749741116, 52.571625819813605, 53.38280980324473, 54.79806696582672, 56.38591646530894, 57.73213669313082, 59.59613393165344, 60.45909561615465, 61.54642733862617, 62.02968588194684, 63.16879530548844, 64.4287193648602, 64.9292371418709, 65.68864342423197, 66.7932343803935, 67.67345529858474, 68.45012081463582, 69.48567483603728, 70.43493268898861, 71.19433897134967, 72.40248532965137, 73.48981705212289, 74.5598895409044, 75.47462892647566, 76.32033137728685, 76.95892302381773, 77.70107007248878, 78.23610631687953, 78.59855022437003, 79.4269934414912, 80.06558508802209, 81.20469451156369, 81.7915084570245, 82.55091473938558, 83.6382464618571, 84.25957887469797, 85.0707628581291, 85.5712806351398, 85.86468760787021, 86.8484639282016, 87.65964791163272, 88.26372109078356, 88.8505350362444, 89.45460821539524, 90.1449775629962, 90.8526061442872, 91.75008629616845, 92.63030721435969, 92.88919571971005, 93.21712115982051, 93.9247497411115, 94.49430445288229, 95.37452537107353, 95.9958577839144, 96.51363479461511, 97.04867103900587, 97.60096651708665, 98.23955816361753, 98.77459440800828, 99.44770452191922, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 3, 4, 5, 9, 10, 11, 13, 15, 18, 21, 22, 24, 29, 32, 41, 47, 53, 57, 66, 72, 82, 90, 95, 106, 109, 117, 121, 127, 132, 137, 147, 155, 156, 157, 164, 171, 172, 180, 187, 188, 192, 194, 195, 197, 199, 199, 202, 204, 206, 207, 209, 209, 211, 212, 214, 215, 218, 219, 219, 220, 222, 222, 222, 223, 225, 225, 226, 226, 226, 226, 226, 226, 227, 227, 227, 228, 228, 229, 229, 229, 230, 230, 231, 231, 231, 231, 231, 231, 231, 232, 232], "harmed": [0, 0, 1, 1, 1, 2, 2, 3, 4, 6, 8, 9, 12, 21, 26, 33, 38, 42, 64, 81, 96, 111, 144, 160, 197, 231, 271, 296, 333, 355, 444, 495, 542, 594, 645, 702, 769, 829, 913, 949, 1009, 1078, 1105, 1165, 1234, 1285, 1370, 1406, 1450, 1470, 1522, 1574, 1599, 1631, 1683, 1728, 1764, 1815, 1859, 1888, 1949, 2008, 2061, 2104, 2146, 2177, 2210, 2239, 2258, 2303, 2331, 2387, 2415, 2454, 2513, 2546, 2589, 2616, 2632, 2686, 2731, 2765, 2795, 2829, 2868, 2905, 2954, 3000, 3014, 3033, 3071, 3102, 3150, 3186, 3216, 3245, 3276, 3313, 3344, 3379, 3411], "exact_score": [81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.30824991370383, 81.27373144632378, 81.17017604418363, 81.11839834311357, 81.04936140835348, 80.9975837072834, 80.91128753883328, 80.72143596824301, 80.54884363134278, 80.39351052813255, 80.10010355540214, 79.80669658267173, 79.51328960994132, 79.23714187090093, 78.94373489817053, 78.65032792544011, 78.25336555056955, 78.02899551259924, 77.59751467034863, 77.21781152916812, 76.8553676216776, 76.49292371418709, 75.80255436658612, 75.37107352433551, 75.11218501898516, 74.5771487745944, 74.18018639972385, 73.69692785640318, 73.23092854677252, 72.7821884708319, 72.22989299275112, 71.69485674836038, 71.26337590610977, 70.59026579219882, 70.003451846738, 69.39937866758716, 68.7090093199862, 68.03589920607524, 67.5008629616845, 66.93130824991371, 66.39627200552296, 65.89575422851226, 65.22264411460131, 64.54953400069037, 63.94546082153953, 63.25509147393856, 62.616499827407665, 61.96064894718675, 61.23576113220574, 60.510873317224714, 59.94131860545392, 59.33724542630308, 58.577839143942015, 57.92198826372109, 57.19710044874008, 56.42043493268899, 55.64376941663791, 54.79806696582672, 54.073179150845704, 53.24473593372454, 52.519848118743525, 51.794960303762515, 50.880220918191235, 49.98274076630998, 49.119779081808765, 48.37763203313773, 47.566448049706594, 46.772523299965485, 45.857783914394204, 45.081118398343115, 44.26993441491198, 43.40697273041077, 42.50949257852952, 41.6465308940283, 40.73179150845703, 39.79979288919572, 38.83327580255437, 37.88401794960304, 36.96927856403176, 36.00276147739041, 35.08802209181913, 34.15602347255782, 33.258543320676566, 32.34380393510528, 31.429064549534004, 30.4280289955126, 29.461511908871252, 28.477735588539865, 27.52847773558854, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 62.83600248742673, "threshold_area100": 58.27813255091474, "threshold_area25": 80.75595443562308}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.852957010269165, "operator": "ge"}, {"budget": 0.02, "threshold": -3.4612574577331543, "operator": "ge"}, {"budget": 0.03, "threshold": -3.8497390747070312, "operator": "ge"}, {"budget": 0.04, "threshold": -4.2455339431762695, "operator": "ge"}, {"budget": 0.05, "threshold": -4.523426055908203, "operator": "ge"}, {"budget": 0.06, "threshold": -4.949087619781494, "operator": "ge"}, {"budget": 0.07, "threshold": -5.250169277191162, "operator": "ge"}, {"budget": 0.08, "threshold": -5.552947044372559, "operator": "ge"}, {"budget": 0.09, "threshold": -5.956336975097656, "operator": "ge"}, {"budget": 0.1, "threshold": -6.101198196411133, "operator": "ge"}, {"budget": 0.11, "threshold": -6.4375, "operator": "ge"}, {"budget": 0.12, "threshold": -6.70873498916626, "operator": "ge"}, {"budget": 0.13, "threshold": -7.054481506347656, "operator": "ge"}, {"budget": 0.14, "threshold": -7.475530624389648, "operator": "ge"}, {"budget": 0.15, "threshold": -7.5779829025268555, "operator": "ge"}, {"budget": 0.16, "threshold": -7.879614353179932, "operator": "ge"}, {"budget": 0.17, "threshold": -8.137227058410645, "operator": "ge"}, {"budget": 0.18, "threshold": -8.474802017211914, "operator": "ge"}, {"budget": 0.19, "threshold": -8.687814712524414, "operator": "ge"}, {"budget": 0.2, "threshold": -8.997812271118164, "operator": "ge"}, {"budget": 0.21, "threshold": -9.157302856445312, "operator": "ge"}, {"budget": 0.22, "threshold": -9.308221817016602, "operator": "ge"}, {"budget": 0.23, "threshold": -9.506471633911133, "operator": "ge"}, {"budget": 0.24, "threshold": -9.679539680480957, "operator": "ge"}, {"budget": 0.25, "threshold": -9.846654891967773, "operator": "ge"}, {"budget": 0.26, "threshold": -9.948704719543457, "operator": "ge"}, {"budget": 0.27, "threshold": -10.072346687316895, "operator": "ge"}, {"budget": 0.28, "threshold": -10.217041015625, "operator": "ge"}, {"budget": 0.29, "threshold": -10.308636665344238, "operator": "ge"}, {"budget": 0.3, "threshold": -10.369559288024902, "operator": "ge"}, {"budget": 0.31, "threshold": -10.456890106201172, "operator": "ge"}, {"budget": 0.32, "threshold": -10.544360160827637, "operator": "ge"}, {"budget": 0.33, "threshold": -10.636173248291016, "operator": "ge"}, {"budget": 0.34, "threshold": -10.702492713928223, "operator": "ge"}, {"budget": 0.35, "threshold": -10.785628318786621, "operator": "ge"}, {"budget": 0.36, "threshold": -10.84520149230957, "operator": "ge"}, {"budget": 0.37, "threshold": -10.906278610229492, "operator": "ge"}, {"budget": 0.38, "threshold": -10.956941604614258, "operator": "ge"}, {"budget": 0.39, "threshold": -11.055719375610352, "operator": "ge"}, {"budget": 0.4, "threshold": -11.092029571533203, "operator": "ge"}, {"budget": 0.41, "threshold": -11.147771835327148, "operator": "ge"}, {"budget": 0.42, "threshold": -11.208105087280273, "operator": "ge"}, {"budget": 0.43, "threshold": -11.26070785522461, "operator": "ge"}, {"budget": 0.44, "threshold": -11.316946029663086, "operator": "ge"}, {"budget": 0.45, "threshold": -11.375157356262207, "operator": "ge"}, {"budget": 0.46, "threshold": -11.447214126586914, "operator": "ge"}, {"budget": 0.47, "threshold": -11.529314994812012, "operator": "ge"}, {"budget": 0.48, "threshold": -11.637934684753418, "operator": "ge"}, {"budget": 0.49, "threshold": -11.721536636352539, "operator": "ge"}, {"budget": 0.5, "threshold": -11.816019058227539, "operator": "ge"}, {"budget": 0.51, "threshold": -11.90927505493164, "operator": "ge"}, {"budget": 0.52, "threshold": -11.989368438720703, "operator": "ge"}, {"budget": 0.53, "threshold": -12.043185234069824, "operator": "ge"}, {"budget": 0.54, "threshold": -12.125543594360352, "operator": "ge"}, {"budget": 0.55, "threshold": -12.194652557373047, "operator": "ge"}, {"budget": 0.56, "threshold": -12.287054061889648, "operator": "ge"}, {"budget": 0.57, "threshold": -12.316354751586914, "operator": "ge"}, {"budget": 0.58, "threshold": -12.384103775024414, "operator": "ge"}, {"budget": 0.59, "threshold": -12.430612564086914, "operator": "ge"}, {"budget": 0.6, "threshold": -12.505615234375, "operator": "ge"}, {"budget": 0.61, "threshold": -12.571977615356445, "operator": "ge"}, {"budget": 0.62, "threshold": -12.630266189575195, "operator": "ge"}, {"budget": 0.63, "threshold": -12.70709228515625, "operator": "ge"}, {"budget": 0.64, "threshold": -12.776128768920898, "operator": "ge"}, {"budget": 0.65, "threshold": -12.8450927734375, "operator": "ge"}, {"budget": 0.66, "threshold": -12.881105422973633, "operator": "ge"}, {"budget": 0.67, "threshold": -12.911334991455078, "operator": "ge"}, {"budget": 0.68, "threshold": -12.960491180419922, "operator": "ge"}, {"budget": 0.69, "threshold": -13.022256851196289, "operator": "ge"}, {"budget": 0.7, "threshold": -13.062459945678711, "operator": "ge"}, {"budget": 0.71, "threshold": -13.087949752807617, "operator": "ge"}, {"budget": 0.72, "threshold": -13.125743865966797, "operator": "ge"}, {"budget": 0.73, "threshold": -13.183586120605469, "operator": "ge"}, {"budget": 0.74, "threshold": -13.255237579345703, "operator": "ge"}, {"budget": 0.75, "threshold": -13.334959030151367, "operator": "ge"}, {"budget": 0.76, "threshold": -13.41484260559082, "operator": "ge"}, {"budget": 0.77, "threshold": -13.487625122070312, "operator": "ge"}, {"budget": 0.78, "threshold": -13.580303192138672, "operator": "ge"}, {"budget": 0.79, "threshold": -13.653177261352539, "operator": "ge"}, {"budget": 0.8, "threshold": -13.707620620727539, "operator": "ge"}, {"budget": 0.81, "threshold": -13.748434066772461, "operator": "ge"}, {"budget": 0.82, "threshold": -13.829544067382812, "operator": "ge"}, {"budget": 0.83, "threshold": -13.908063888549805, "operator": "ge"}, {"budget": 0.84, "threshold": -14.042869567871094, "operator": "ge"}, {"budget": 0.85, "threshold": -14.098867416381836, "operator": "ge"}, {"budget": 0.86, "threshold": -14.205782890319824, "operator": "ge"}, {"budget": 0.87, "threshold": -14.235618591308594, "operator": "ge"}, {"budget": 0.88, "threshold": -14.326226234436035, "operator": "ge"}, {"budget": 0.89, "threshold": -14.436708450317383, "operator": "ge"}, {"budget": 0.9, "threshold": -14.560700416564941, "operator": "ge"}, {"budget": 0.91, "threshold": -14.613402366638184, "operator": "ge"}, {"budget": 0.92, "threshold": -14.712873458862305, "operator": "ge"}, {"budget": 0.93, "threshold": -14.825215339660645, "operator": "ge"}, {"budget": 0.94, "threshold": -14.920564651489258, "operator": "ge"}, {"budget": 0.95, "threshold": -15.103133201599121, "operator": "ge"}, {"budget": 0.96, "threshold": -15.208525657653809, "operator": "ge"}, {"budget": 0.97, "threshold": -15.381217956542969, "operator": "ge"}, {"budget": 0.98, "threshold": -15.608622550964355, "operator": "ge"}, {"budget": 0.99, "threshold": -15.984237670898438, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 97.91666666666666, 97.16666666666667, 96.58333333333333, 96.25, 95.58333333333333, 95.08333333333333, 94.25, 93.75, 93.08333333333333, 92.66666666666666, 92.0, 91.16666666666666, 90.66666666666667, 89.91666666666667, 89.16666666666667, 88.41666666666667, 87.83333333333333, 87.0, 86.33333333333333, 85.75, 85.25, 84.58333333333333, 84.0, 83.25, 82.91666666666666, 82.41666666666667, 82.08333333333333, 81.75, 81.25, 80.58333333333334, 80.16666666666666, 79.75, 79.5, 79.0, 78.5, 78.33333333333333, 77.83333333333333, 77.33333333333333, 76.83333333333333, 76.58333333333334, 75.91666666666667, 75.25, 74.75, 74.25, 73.91666666666666, 73.41666666666666, 72.83333333333333, 72.0, 71.33333333333334, 70.91666666666667, 70.08333333333334, 69.25, 68.58333333333333, 67.91666666666667, 67.0, 66.5, 65.83333333333333, 65.16666666666666, 64.25, 63.583333333333336, 62.83333333333334, 62.0, 61.083333333333336, 60.16666666666667, 59.25, 58.25, 57.41666666666667, 56.58333333333334, 55.666666666666664, 54.75, 53.91666666666667, 52.916666666666664, 51.916666666666664, 50.916666666666664, 50.083333333333336, 49.16666666666667, 48.5, 47.75, 47.083333333333336, 46.08333333333333, 45.25, 44.25, 43.25, 42.25, 41.333333333333336, 40.33333333333333, 39.33333333333333, 38.33333333333333, 37.33333333333333, 36.33333333333333, 35.333333333333336, 34.5, 33.58333333333333, 32.666666666666664, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 2, 2, 2, 2, 4, 4, 4, 4, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 2, 10, 19, 26, 32, 40, 46, 56, 63, 71, 77, 85, 95, 101, 110, 119, 128, 135, 145, 153, 160, 166, 174, 181, 190, 194, 200, 204, 208, 214, 222, 227, 232, 235, 241, 247, 249, 255, 261, 267, 270, 278, 286, 292, 298, 302, 308, 315, 325, 333, 338, 348, 358, 366, 374, 385, 391, 399, 407, 418, 426, 435, 445, 456, 467, 478, 490, 500, 510, 521, 532, 542, 554, 566, 578, 588, 599, 607, 616, 624, 636, 646, 658, 670, 682, 693, 705, 717, 729, 741, 753, 765, 775, 786, 797, 808, 820, 832, 844, 856], "exact_score": [98.58333333333333, 98.58333333333333, 97.91666666666666, 97.16666666666667, 96.58333333333333, 96.25, 95.58333333333333, 95.08333333333333, 94.25, 93.75, 93.08333333333333, 92.66666666666666, 92.0, 91.16666666666666, 90.66666666666667, 89.91666666666667, 89.16666666666667, 88.41666666666667, 87.83333333333333, 87.0, 86.33333333333333, 85.75, 85.25, 84.58333333333333, 84.0, 83.25, 82.91666666666666, 82.41666666666667, 82.08333333333333, 81.75, 81.25, 80.58333333333334, 80.16666666666666, 79.75, 79.5, 79.0, 78.5, 78.33333333333333, 77.83333333333333, 77.33333333333333, 76.83333333333333, 76.5, 75.91666666666667, 75.25, 74.75, 74.25, 73.91666666666666, 73.41666666666666, 72.83333333333333, 72.0, 71.33333333333334, 70.91666666666667, 70.08333333333334, 69.25, 68.58333333333333, 67.91666666666667, 67.0, 66.41666666666667, 65.83333333333333, 65.16666666666666, 64.25, 63.583333333333336, 62.83333333333334, 62.0, 61.083333333333336, 60.16666666666667, 59.25, 58.25, 57.41666666666667, 56.49999999999999, 55.666666666666664, 54.75, 53.91666666666667, 52.916666666666664, 51.916666666666664, 50.916666666666664, 50.083333333333336, 49.16666666666667, 48.5, 47.75, 47.083333333333336, 46.08333333333333, 45.25, 44.25, 43.25, 42.25, 41.333333333333336, 40.33333333333333, 39.33333333333333, 38.33333333333333, 37.33333333333333, 36.33333333333333, 35.333333333333336, 34.5, 33.58333333333333, 32.666666666666664, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 67.68166666666666, "threshold_area100": 67.68416666666668, "threshold_area25": 91.3}, "test": {"score": [81.429064549534, 81.25647221263375, 80.01380738695202, 79.28891957197101, 77.95995857783915, 76.94166379012772, 75.18122195374526, 73.81774249223335, 72.35070762858129, 70.46945115636866, 69.86537797721782, 68.3638246461857, 67.3627890921643, 65.84397652744218, 64.15257162581982, 63.65205384880912, 62.616499827407665, 61.59820503969624, 60.54539178460476, 59.924059371763896, 58.90576458405247, 58.47428374180187, 58.06006213324129, 57.50776665516051, 57.093545046599935, 56.541249568519156, 56.178805661028655, 55.712806351397994, 55.195029340697275, 54.95340006903693, 54.81532619951675, 54.47014152571625, 54.193993786675875, 53.90058681394546, 53.641698308595096, 53.40006903693476, 53.227476700034515, 52.96858819468415, 52.67518122195375, 52.226441146013116, 52.03658957542285, 51.86399723852262, 51.67414566793235, 51.46703486365205, 51.27718329306179, 51.0010355540214, 50.75940628236106, 50.27614773904039, 49.82740766309976, 49.48222298929928, 48.87814981014843, 48.4984466689679, 48.10148429409735, 47.687262685536766, 47.186744908526066, 46.910597169485676, 46.289264756644805, 46.13393165343459, 45.771487745944086, 45.270969968933386, 44.61511908871246, 44.11460131170176, 43.63134276838109, 43.18260269244046, 42.7856403175699, 42.2851225405592, 41.888160165688646, 41.6465308940283, 41.30134622022783, 40.714532274767, 40.265792198826375, 40.075940628236104, 39.782533655505695, 39.420089748015194, 38.988608905764586, 38.41905419399379, 37.76320331377287, 37.41801863997239, 36.7103900586814, 36.1408353469106, 35.761132205730064, 35.536762167759754, 34.94994822229893, 34.51846738004833, 33.4311356575768, 32.96513634794616, 32.39558163617536, 32.274767000345186, 31.912323092854677, 31.360027614773912, 30.686917500862965, 30.410769761822575, 30.013807386952017, 29.685881946841562, 29.323438039351057, 28.822920262340357, 28.408698653779773, 27.942699344149126, 27.442181567138423, 26.855367621677594, 26.561960648947192], "calls": [0.0, 4.055919917155678, 9.095616154642734, 12.56472212633759, 16.983085950983774, 19.968933379357956, 24.905074214704868, 28.23610631687953, 31.34276838108388, 34.760096651708665, 36.08905764584052, 38.729720400414216, 40.80082844321712, 43.320676561960646, 46.32378322402486, 47.186744908526066, 48.774594408008284, 50.24162927166034, 52.071108042802905, 53.05488436313428, 54.6427338626165, 55.350362443907485, 56.10976872626855, 57.093545046599935, 57.97376596479117, 58.83672764929238, 59.61339316534345, 60.424577148774596, 61.39109423541594, 61.9951674145668, 62.53020365895754, 63.20331377286848, 63.84190541939938, 64.4977562996203, 64.98101484294098, 65.65412495685192, 66.12012426648258, 66.75871591301346, 67.32827062478425, 68.2084915429755, 68.53641698308594, 68.84708318950639, 69.34760096651709, 69.81360027614774, 70.2795995857784, 70.71108042802899, 71.26337590610977, 71.98826372109079, 72.74767000345184, 73.38626164998274, 74.40455643769417, 74.97411114946496, 75.54366586123577, 76.02692440455644, 76.78633068691751, 77.23507076285813, 77.99447704521918, 78.18432861580946, 78.77114256127028, 79.34069727304107, 80.13462202278218, 80.77321366931308, 81.37728684846392, 81.91232309285468, 82.4301001035554, 83.05143251639628, 83.51743182602692, 83.7935795650673, 84.17328270624785, 84.89817052122885, 85.38142906454954, 85.58853986882983, 85.9337245426303, 86.33068691750087, 86.79668622713152, 87.41801863997239, 88.14290645495339, 88.4880911287539, 89.24749741111495, 89.86882982395582, 90.24853296513635, 90.49016223679669, 91.11149464963756, 91.57749395926821, 92.73386261649983, 93.21712115982051, 93.80393510528133, 93.94200897480151, 94.33897134967208, 94.96030376251295, 95.68519157749395, 95.9958577839144, 96.39282015878496, 96.73800483258543, 97.13496720745599, 97.68726268553675, 98.10148429409735, 98.58474283741802, 99.10251984811875, 99.70659302726959, 100.0], "rescued": [0, 32, 60, 85, 108, 121, 141, 156, 168, 178, 185, 196, 203, 209, 217, 219, 220, 221, 223, 223, 224, 225, 225, 225, 226, 226, 226, 226, 226, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 229, 229, 229, 229, 229, 229, 229, 230, 230, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 42, 142, 209, 309, 381, 503, 597, 694, 813, 855, 953, 1018, 1112, 1218, 1249, 1310, 1370, 1433, 1469, 1529, 1555, 1579, 1611, 1636, 1668, 1689, 1716, 1746, 1761, 1769, 1789, 1805, 1822, 1837, 1851, 1861, 1876, 1893, 1919, 1930, 1940, 1952, 1964, 1975, 1991, 2005, 2033, 2059, 2079, 2114, 2136, 2159, 2183, 2212, 2228, 2264, 2273, 2294, 2323, 2361, 2390, 2418, 2445, 2468, 2497, 2520, 2534, 2554, 2588, 2615, 2626, 2644, 2665, 2690, 2723, 2761, 2781, 2822, 2855, 2877, 2890, 2924, 2949, 3013, 3040, 3073, 3080, 3101, 3133, 3172, 3188, 3211, 3230, 3251, 3280, 3304, 3331, 3360, 3394, 3411], "exact_score": [81.429064549534, 81.41180531584398, 81.44632378322403, 81.30824991370383, 81.23921297894374, 80.98032447359338, 80.72143596824301, 80.41076976182258, 80.25543665861235, 80.06558508802209, 79.8584742837418, 79.56506731101139, 79.4269934414912, 79.15084570245081, 78.82292026234036, 78.40869865377978, 78.23610631687953, 77.95995857783915, 77.63203313772868, 77.28684846392821, 76.94166379012772, 76.56196064894719, 76.21677597514672, 75.88885053503624, 75.57818432861582, 75.14670348636521, 74.81877804625475, 74.37003797031412, 73.90403866068347, 73.36900241629272, 73.09285467725233, 72.59233690024163, 72.02278218847084, 71.36693130824992, 70.90093199861927, 70.34863652053849, 69.89989644459786, 69.39937866758716, 68.72626855367622, 68.17397307559546, 67.7079737659648, 67.24197445633415, 66.68967897825337, 66.05108733172247, 65.53331032102174, 64.94649637556093, 64.34242319641008, 63.738350017259236, 63.099758370728345, 62.46116672419745, 61.73627890921644, 61.114946496375566, 60.61442871936487, 59.94131860545392, 59.26820849154297, 58.664135312392126, 58.12909906800139, 57.55954435623059, 57.07628581290991, 56.454953400069044, 55.97169485674836, 55.384880911287546, 54.95340006903693, 54.57369692785641, 54.12495685191578, 53.641698308595096, 53.24473593372454, 52.86503279254402, 52.36451501553332, 51.794960303762515, 51.41525716258199, 50.880220918191235, 50.27614773904039, 49.672074559889545, 49.06800138073869, 48.48118743527788, 47.72178115291681, 47.065930272695894, 46.289264756644805, 45.564376941663795, 44.75319295823266, 43.97652744218157, 43.13082499137039, 42.2851225405592, 41.45667932343804, 40.66275457369693, 39.73075595443562, 38.83327580255437, 37.90127718329306, 36.93476009665171, 36.02002071108043, 35.0707628581291, 34.1387642388678, 33.18950638591647, 32.22298929927511, 31.32550914739386, 30.410769761822575, 29.444252675181225, 28.512254055919918, 27.52847773558854, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 60.22313800154397, "threshold_area100": 49.05583362098724, "threshold_area25": 67.67000345184675}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.068418025970459, "operator": "ge"}, {"budget": 0.02, "threshold": -1.4335250854492188, "operator": "ge"}, {"budget": 0.03, "threshold": -1.7821903228759766, "operator": "ge"}, {"budget": 0.04, "threshold": -2.2312395572662354, "operator": "ge"}, {"budget": 0.05, "threshold": -2.651010751724243, "operator": "ge"}, {"budget": 0.06, "threshold": -2.893944263458252, "operator": "ge"}, {"budget": 0.07, "threshold": -3.1673450469970703, "operator": "ge"}, {"budget": 0.08, "threshold": -3.4184906482696533, "operator": "ge"}, {"budget": 0.09, "threshold": -3.737623691558838, "operator": "ge"}, {"budget": 0.1, "threshold": -4.074032783508301, "operator": "ge"}, {"budget": 0.11, "threshold": -4.418152332305908, "operator": "ge"}, {"budget": 0.12, "threshold": -4.651919364929199, "operator": "ge"}, {"budget": 0.13, "threshold": -4.914571762084961, "operator": "ge"}, {"budget": 0.14, "threshold": -5.159577369689941, "operator": "ge"}, {"budget": 0.15, "threshold": -5.553363800048828, "operator": "ge"}, {"budget": 0.16, "threshold": -5.783990859985352, "operator": "ge"}, {"budget": 0.17, "threshold": -6.052553176879883, "operator": "ge"}, {"budget": 0.18, "threshold": -6.32862663269043, "operator": "ge"}, {"budget": 0.19, "threshold": -6.469913482666016, "operator": "ge"}, {"budget": 0.2, "threshold": -6.678528785705566, "operator": "ge"}, {"budget": 0.21, "threshold": -6.817497253417969, "operator": "ge"}, {"budget": 0.22, "threshold": -7.017492294311523, "operator": "ge"}, {"budget": 0.23, "threshold": -7.165578842163086, "operator": "ge"}, {"budget": 0.24, "threshold": -7.263388156890869, "operator": "ge"}, {"budget": 0.25, "threshold": -7.482306480407715, "operator": "ge"}, {"budget": 0.26, "threshold": -7.645999431610107, "operator": "ge"}, {"budget": 0.27, "threshold": -7.754989147186279, "operator": "ge"}, {"budget": 0.28, "threshold": -7.876125335693359, "operator": "ge"}, {"budget": 0.29, "threshold": -7.984524726867676, "operator": "ge"}, {"budget": 0.3, "threshold": -8.087646484375, "operator": "ge"}, {"budget": 0.31, "threshold": -8.219573974609375, "operator": "ge"}, {"budget": 0.32, "threshold": -8.344242095947266, "operator": "ge"}, {"budget": 0.33, "threshold": -8.43078899383545, "operator": "ge"}, {"budget": 0.34, "threshold": -8.492520332336426, "operator": "ge"}, {"budget": 0.35, "threshold": -8.598609924316406, "operator": "ge"}, {"budget": 0.36, "threshold": -8.67156982421875, "operator": "ge"}, {"budget": 0.37, "threshold": -8.80344295501709, "operator": "ge"}, {"budget": 0.38, "threshold": -8.897350311279297, "operator": "ge"}, {"budget": 0.39, "threshold": -9.008764266967773, "operator": "ge"}, {"budget": 0.4, "threshold": -9.188684463500977, "operator": "ge"}, {"budget": 0.41, "threshold": -9.301719665527344, "operator": "ge"}, {"budget": 0.42, "threshold": -9.359724044799805, "operator": "ge"}, {"budget": 0.43, "threshold": -9.436110496520996, "operator": "ge"}, {"budget": 0.44, "threshold": -9.542603492736816, "operator": "ge"}, {"budget": 0.45, "threshold": -9.611976623535156, "operator": "ge"}, {"budget": 0.46, "threshold": -9.71461296081543, "operator": "ge"}, {"budget": 0.47, "threshold": -9.80970287322998, "operator": "ge"}, {"budget": 0.48, "threshold": -9.877546310424805, "operator": "ge"}, {"budget": 0.49, "threshold": -9.949859619140625, "operator": "ge"}, {"budget": 0.5, "threshold": -10.082374572753906, "operator": "ge"}, {"budget": 0.51, "threshold": -10.163858413696289, "operator": "ge"}, {"budget": 0.52, "threshold": -10.299131393432617, "operator": "ge"}, {"budget": 0.53, "threshold": -10.418089866638184, "operator": "ge"}, {"budget": 0.54, "threshold": -10.522787094116211, "operator": "ge"}, {"budget": 0.55, "threshold": -10.62254810333252, "operator": "ge"}, {"budget": 0.56, "threshold": -10.696805953979492, "operator": "ge"}, {"budget": 0.57, "threshold": -10.803836822509766, "operator": "ge"}, {"budget": 0.58, "threshold": -10.89113998413086, "operator": "ge"}, {"budget": 0.59, "threshold": -10.931695938110352, "operator": "ge"}, {"budget": 0.6, "threshold": -11.003084182739258, "operator": "ge"}, {"budget": 0.61, "threshold": -11.070130348205566, "operator": "ge"}, {"budget": 0.62, "threshold": -11.147669792175293, "operator": "ge"}, {"budget": 0.63, "threshold": -11.233407020568848, "operator": "ge"}, {"budget": 0.64, "threshold": -11.309122085571289, "operator": "ge"}, {"budget": 0.65, "threshold": -11.366363525390625, "operator": "ge"}, {"budget": 0.66, "threshold": -11.417692184448242, "operator": "ge"}, {"budget": 0.67, "threshold": -11.45999813079834, "operator": "ge"}, {"budget": 0.68, "threshold": -11.523513793945312, "operator": "ge"}, {"budget": 0.69, "threshold": -11.585577011108398, "operator": "ge"}, {"budget": 0.7, "threshold": -11.661155700683594, "operator": "ge"}, {"budget": 0.71, "threshold": -11.732612609863281, "operator": "ge"}, {"budget": 0.72, "threshold": -11.798595428466797, "operator": "ge"}, {"budget": 0.73, "threshold": -11.846940994262695, "operator": "ge"}, {"budget": 0.74, "threshold": -11.900733947753906, "operator": "ge"}, {"budget": 0.75, "threshold": -11.964866638183594, "operator": "ge"}, {"budget": 0.76, "threshold": -12.03292179107666, "operator": "ge"}, {"budget": 0.77, "threshold": -12.10163688659668, "operator": "ge"}, {"budget": 0.78, "threshold": -12.174083709716797, "operator": "ge"}, {"budget": 0.79, "threshold": -12.222182273864746, "operator": "ge"}, {"budget": 0.8, "threshold": -12.30774211883545, "operator": "ge"}, {"budget": 0.81, "threshold": -12.374980926513672, "operator": "ge"}, {"budget": 0.82, "threshold": -12.450539588928223, "operator": "ge"}, {"budget": 0.83, "threshold": -12.54262924194336, "operator": "ge"}, {"budget": 0.84, "threshold": -12.611824035644531, "operator": "ge"}, {"budget": 0.85, "threshold": -12.712468147277832, "operator": "ge"}, {"budget": 0.86, "threshold": -12.784867286682129, "operator": "ge"}, {"budget": 0.87, "threshold": -12.895426750183105, "operator": "ge"}, {"budget": 0.88, "threshold": -12.959096908569336, "operator": "ge"}, {"budget": 0.89, "threshold": -13.036327362060547, "operator": "ge"}, {"budget": 0.9, "threshold": -13.134617805480957, "operator": "ge"}, {"budget": 0.91, "threshold": -13.258975982666016, "operator": "ge"}, {"budget": 0.92, "threshold": -13.383543968200684, "operator": "ge"}, {"budget": 0.93, "threshold": -13.468987464904785, "operator": "ge"}, {"budget": 0.94, "threshold": -13.603887557983398, "operator": "ge"}, {"budget": 0.95, "threshold": -13.734949111938477, "operator": "ge"}, {"budget": 0.96, "threshold": -13.916865348815918, "operator": "ge"}, {"budget": 0.97, "threshold": -13.987752914428711, "operator": "ge"}, {"budget": 0.98, "threshold": -14.206903457641602, "operator": "ge"}, {"budget": 0.99, "threshold": -14.557823181152344, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.5, 97.91666666666666, 97.41666666666666, 96.91666666666666, 96.41666666666666, 95.91666666666667, 95.25, 94.66666666666667, 94.0, 93.25, 92.75, 92.16666666666666, 91.58333333333333, 90.91666666666667, 90.08333333333334, 89.58333333333334, 88.91666666666667, 88.08333333333334, 87.5, 87.0, 86.75, 86.33333333333333, 85.91666666666666, 85.58333333333333, 85.25, 85.0, 84.75, 84.41666666666667, 84.0, 83.41666666666667, 82.91666666666666, 82.16666666666667, 82.16666666666667, 81.91666666666667, 81.33333333333333, 80.58333333333334, 79.91666666666667, 79.58333333333334, 79.08333333333333, 78.41666666666667, 77.75, 77.16666666666667, 76.41666666666667, 75.91666666666667, 75.41666666666667, 74.75, 74.25, 73.5, 72.58333333333333, 71.83333333333333, 71.25, 70.75, 69.91666666666667, 69.16666666666667, 68.41666666666666, 67.66666666666667, 66.83333333333333, 66.0, 65.41666666666667, 64.58333333333333, 63.66666666666667, 62.74999999999999, 62.0, 61.16666666666667, 60.16666666666667, 59.166666666666664, 58.33333333333333, 57.49999999999999, 56.58333333333334, 55.666666666666664, 54.91666666666667, 54.08333333333333, 53.16666666666667, 52.33333333333333, 51.33333333333334, 50.5, 49.583333333333336, 48.75, 47.75, 46.833333333333336, 46.0, 45.08333333333333, 44.08333333333333, 43.166666666666664, 42.166666666666664, 41.166666666666664, 40.25, 39.24999999999999, 38.33333333333333, 37.416666666666664, 36.5, 35.583333333333336, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 2, 2, 2, 3, 4, 4, 4, 4, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 1, 10, 16, 22, 29, 36, 44, 51, 59, 69, 76, 83, 90, 98, 108, 114, 122, 132, 139, 145, 148, 153, 158, 162, 166, 169, 172, 176, 181, 188, 194, 203, 203, 206, 213, 222, 230, 234, 240, 248, 256, 263, 272, 278, 284, 292, 298, 307, 318, 327, 334, 340, 350, 359, 368, 377, 387, 397, 404, 414, 425, 436, 445, 455, 467, 479, 489, 499, 510, 521, 530, 540, 551, 561, 573, 583, 594, 604, 616, 627, 637, 648, 660, 671, 683, 695, 706, 718, 729, 740, 751, 762, 773, 785, 797, 809, 820, 832, 844, 856], "exact_score": [98.58333333333333, 98.5, 97.91666666666666, 97.41666666666666, 96.91666666666666, 96.41666666666666, 95.91666666666667, 95.25, 94.66666666666667, 94.0, 93.25, 92.75, 92.16666666666666, 91.58333333333333, 90.91666666666667, 90.08333333333334, 89.58333333333334, 88.91666666666667, 88.08333333333334, 87.5, 87.0, 86.75, 86.33333333333333, 85.91666666666666, 85.58333333333333, 85.25, 85.0, 84.75, 84.41666666666667, 84.0, 83.41666666666667, 82.91666666666666, 82.16666666666667, 82.16666666666667, 81.91666666666667, 81.33333333333333, 80.58333333333334, 79.91666666666667, 79.58333333333334, 79.08333333333333, 78.41666666666667, 77.66666666666666, 77.16666666666667, 76.41666666666667, 75.91666666666667, 75.41666666666667, 74.75, 74.25, 73.5, 72.58333333333333, 71.83333333333333, 71.25, 70.75, 69.91666666666667, 69.16666666666667, 68.41666666666666, 67.66666666666667, 66.83333333333333, 66.0, 65.41666666666667, 64.58333333333333, 63.66666666666667, 62.74999999999999, 62.0, 61.16666666666667, 60.16666666666667, 59.166666666666664, 58.33333333333333, 57.49999999999999, 56.49999999999999, 55.666666666666664, 54.91666666666667, 54.08333333333333, 53.16666666666667, 52.33333333333333, 51.33333333333334, 50.5, 49.583333333333336, 48.75, 47.75, 46.833333333333336, 46.0, 44.99999999999999, 44.08333333333333, 43.166666666666664, 42.166666666666664, 41.166666666666664, 40.25, 39.24999999999999, 38.33333333333333, 37.416666666666664, 36.5, 35.583333333333336, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 68.28, "threshold_area100": 68.2825, "threshold_area25": 91.73333333333335}, "test": {"score": [81.429064549534, 80.9285467725233, 80.10010355540214, 79.15084570245081, 77.44218156713842, 75.66448049706594, 74.5598895409044, 72.95478080773215, 71.95374525371074, 70.40041422160857, 68.7780462547463, 67.38004832585433, 66.17190196755264, 65.36071798412151, 64.34242319641008, 62.892647566448055, 62.09872281670694, 61.13220573006559, 60.32102174663445, 59.924059371763896, 59.49257852951328, 59.09561615464274, 58.56057991025199, 58.215395236451506, 58.00828443217121, 57.42147048671039, 57.05902657921988, 56.78287884017949, 56.43769416637902, 56.16154642733863, 55.8508802209182, 55.5056955471177, 55.229547808077314, 54.98791853641698, 54.86710390058682, 54.53917846047636, 54.332067656196074, 53.952364515015525, 53.53814290645496, 53.10666206420436, 52.27821884708319, 51.86399723852262, 51.67414566793235, 51.363479461511915, 51.070072488781506, 50.65585088022092, 50.4314808422506, 50.05177770107008, 49.810148429409736, 49.620296858819465, 48.9989644459786, 48.619261304798066, 48.08422506040732, 47.480151881256475, 47.065930272695894, 46.703486365205386, 46.289264756644805, 45.66793234380393, 45.02934069727304, 44.804970659302725, 44.45978598550225, 44.13186054539179, 43.76941663790128, 43.11356575768036, 42.50949257852952, 42.07801173627891, 41.73282706247843, 41.2840869865378, 40.92164307904729, 40.38660683465654, 39.92060752502589, 39.420089748015194, 38.88505350362444, 38.52260959613393, 38.24646185709354, 37.780462547462896, 37.26268553676217, 36.727649292371424, 36.1408353469106, 35.743872972040045, 35.1397997928892, 34.6392820158785, 34.27683810838799, 33.67276492923714, 33.10321021746635, 32.4991370383155, 32.257507766655166, 31.77424922333449, 31.567138419054196, 31.066620642043496, 30.566102865032796, 29.996548153261994, 29.47877114256128, 29.168104936140836, 28.667587159130136, 28.33966171901968, 27.908180876769073, 27.77010700724888, 27.286848463928205, 26.855367621677594, 26.561960648947192], "calls": [0.0, 5.747324818778046, 9.458060062133242, 13.341387642388677, 18.70900931998619, 23.317224715222647, 26.42388677942699, 29.634104245771486, 32.067656196064895, 35.450465999309635, 38.229202623403516, 40.973420780117365, 43.04452882292026, 44.64963755609251, 46.082153952364514, 48.4984466689679, 49.74111149464964, 51.294442526751816, 52.76147739040387, 53.503624439074905, 54.6427338626165, 55.47117707973766, 56.36865723161891, 57.16258198136003, 57.818432861580945, 59.216430790472906, 60.148429409734206, 60.80428028995512, 61.46013117017605, 62.18501898515706, 62.823610631687956, 63.548498446668965, 64.30790472903003, 64.7911632723507, 65.15360717984122, 65.7921988263721, 66.0683465654125, 66.84501208146358, 67.41456679323437, 68.17397307559544, 69.33034173282707, 70.05522954780807, 70.46945115636866, 70.95270969968934, 71.60856057991025, 72.12633759061097, 72.59233690024163, 73.16189161201243, 73.50707628581291, 73.76596479116327, 74.61166724197446, 75.11218501898516, 75.88885053503624, 76.61373835001726, 77.18329306178806, 77.63203313772868, 78.11529168104936, 78.96099413186055, 79.73765964791163, 80.04832585433206, 80.53158439765275, 80.91128753883328, 81.3600276147739, 82.0676561960649, 82.80980324473593, 83.31032102174663, 83.69002416292717, 84.15602347255782, 84.60476354849845, 85.17431826026925, 85.72661373835003, 86.38246461857094, 86.95201933034173, 87.31446323783224, 87.59061097687263, 88.0738695201933, 88.60890576458405, 89.23023817742492, 89.86882982395582, 90.31756989989644, 90.92164307904729, 91.45667932343804, 91.83638246461857, 92.45771487745944, 93.04452882292026, 93.73489817052123, 93.99378667587159, 94.51156368657232, 94.75319295823266, 95.27096996893339, 95.77148774594409, 96.37556092509493, 96.96237487055575, 97.27304107697618, 97.82533655505695, 98.17052122885744, 98.6365205384881, 98.77459440800828, 99.25785295132896, 99.70659302726959, 100.0], "rescued": [0, 31, 50, 70, 96, 116, 139, 149, 164, 180, 189, 200, 205, 211, 212, 218, 219, 220, 221, 222, 222, 222, 223, 224, 224, 224, 225, 226, 226, 226, 226, 226, 226, 226, 226, 226, 226, 226, 226, 226, 226, 226, 226, 227, 228, 228, 228, 228, 228, 228, 228, 228, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 230, 230, 230, 230, 230, 231, 231, 231, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 60, 127, 202, 327, 450, 537, 640, 713, 819, 922, 1014, 1089, 1142, 1202, 1292, 1339, 1396, 1444, 1468, 1493, 1516, 1548, 1569, 1581, 1615, 1637, 1654, 1674, 1690, 1708, 1728, 1744, 1758, 1765, 1784, 1796, 1818, 1842, 1867, 1915, 1939, 1950, 1969, 1987, 2011, 2024, 2046, 2060, 2071, 2107, 2129, 2161, 2196, 2220, 2241, 2265, 2301, 2338, 2351, 2371, 2390, 2411, 2449, 2484, 2510, 2530, 2556, 2577, 2608, 2636, 2665, 2696, 2717, 2733, 2760, 2790, 2821, 2856, 2879, 2914, 2943, 2964, 2999, 3032, 3067, 3081, 3109, 3121, 3150, 3179, 3212, 3242, 3260, 3289, 3308, 3333, 3341, 3369, 3394, 3411], "exact_score": [81.429064549534, 81.44632378322403, 81.36002761477391, 81.17017604418363, 81.03210217466345, 81.01484294097342, 80.87676907145323, 80.73869520193303, 80.54884363134278, 80.22091819123231, 79.96202968588196, 79.7894373489817, 79.44425267518123, 79.21988263721092, 79.03003106662064, 78.73662409389023, 78.46047635484985, 78.09803244735933, 77.70107007248879, 77.26958923023818, 77.02795995857784, 76.76907145322748, 76.33759061097687, 75.81981360027615, 75.52640662754574, 75.02588885053504, 74.66344494304454, 74.283741801864, 73.76596479116327, 73.2827062478426, 72.76492923714187, 72.4197445633414, 71.98826372109079, 71.69485674836038, 71.21159820503969, 70.69382119433898, 70.003451846738, 69.48567483603728, 68.91612012426648, 68.48463928201588, 67.863306869175, 67.3627890921643, 66.77597514670349, 66.2064204349327, 65.75768035899206, 65.15360717984122, 64.39420089748016, 63.70383154987918, 63.13427683810839, 62.65101829478772, 61.9951674145668, 61.2875388332758, 60.75250258888505, 60.23472557818433, 59.716948567483605, 59.33724542630308, 58.76769071453227, 58.249913703831545, 57.95650673110114, 57.47324818778047, 57.11080428028996, 56.679323438039354, 56.23058336209872, 55.73006558508802, 55.3503624439075, 54.93614083534691, 54.383845357266146, 53.848809112875394, 53.15843976527442, 52.53710735243355, 51.91577493959269, 51.32896099413187, 50.75940628236106, 50.15533310321022, 49.44770452191923, 48.74007594062824, 48.01518812564722, 47.204004142216085, 46.37556092509493, 45.650673110113914, 44.83948912668278, 44.04556437694166, 43.165343458750435, 42.33690024162927, 41.43942008974802, 40.576458405246804, 39.7134967207456, 38.850535036244395, 37.866758715913015, 36.952019330341734, 36.037279944770454, 35.0707628581291, 34.121505005177774, 33.15498791853642, 32.257507766655166, 31.32550914739386, 30.341732827062483, 29.444252675181225, 28.49499482222989, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 60.45744833340859, "threshold_area100": 48.95331377286848, "threshold_area25": 67.25060407317915}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.75, "calls": 1.0}, "test": {"score": 81.30824991370383, "calls": 1.9330341732827063}}, "old_normalized_interior": {"val": {"score": 98.75, "calls": 1.0}, "test": {"score": 81.30824991370383, "calls": 1.9330341732827063}}, "old_normalized_endpoints": {"val": {"score": 98.75, "calls": 1.0}, "test": {"score": 81.30824991370383, "calls": 1.9330341732827063}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.5, 98.5, 98.33333333333334, 98.16666666666667, 97.75, 97.75, 97.25, 97.08333333333333, 96.91666666666666, 96.41666666666666, 96.16666666666667, 95.66666666666667, 95.0, 94.41666666666667, 93.66666666666667, 93.16666666666666, 92.58333333333333, 92.25, 91.75, 91.41666666666667, 90.58333333333334, 90.08333333333334, 89.33333333333333, 88.58333333333334, 87.75, 87.16666666666667, 86.58333333333333, 85.75, 85.0, 84.25, 83.58333333333333, 82.75, 81.83333333333334, 80.91666666666667, 80.0, 79.16666666666667, 78.33333333333333, 77.41666666666667, 76.41666666666667, 75.5, 74.58333333333333, 73.66666666666667, 72.83333333333333, 72.0, 71.08333333333333, 70.33333333333334, 69.5, 68.66666666666667, 67.58333333333333, 66.75, 65.83333333333333, 64.83333333333333, 63.916666666666664, 62.916666666666664, 61.916666666666664, 61.16666666666667, 60.16666666666667, 59.33333333333333, 58.41666666666667, 57.58333333333334, 56.49999999999999, 55.49999999999999, 54.49999999999999, 53.91666666666667, 53.0, 52.0, 51.0, 50.0, 49.16666666666667, 48.16666666666667, 47.25, 46.25, 45.333333333333336, 44.25, 43.25, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.5, 38.583333333333336, 37.583333333333336, 36.66666666666667, 35.75000000000001, 34.75, 33.75, 32.75, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.5, 98.5, 98.33333333333334, 98.16666666666667, 97.75, 97.75, 97.25, 97.08333333333333, 96.91666666666666, 96.41666666666666, 96.16666666666667, 95.66666666666667, 95.0, 94.41666666666667, 93.66666666666667, 93.16666666666666, 92.58333333333333, 92.25, 91.75, 91.41666666666667, 90.58333333333334, 90.08333333333334, 89.33333333333333, 88.58333333333334, 87.75, 87.16666666666667, 86.58333333333333, 85.75, 85.0, 84.25, 83.58333333333333, 82.66666666666667, 81.83333333333334, 80.91666666666667, 80.0, 79.16666666666667, 78.33333333333333, 77.41666666666667, 76.41666666666667, 75.5, 74.58333333333333, 73.66666666666667, 72.83333333333333, 72.0, 71.08333333333333, 70.33333333333334, 69.5, 68.58333333333333, 67.58333333333333, 66.75, 65.83333333333333, 64.83333333333333, 63.916666666666664, 62.916666666666664, 61.916666666666664, 61.16666666666667, 60.16666666666667, 59.33333333333333, 58.41666666666667, 57.49999999999999, 56.49999999999999, 55.49999999999999, 54.49999999999999, 53.91666666666667, 53.0, 52.0, 51.0, 50.0, 49.16666666666667, 48.16666666666667, 47.25, 46.25, 45.25, 44.25, 43.25, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.5, 38.583333333333336, 37.583333333333336, 36.66666666666667, 35.75000000000001, 34.75, 33.75, 32.75, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.40833333333335, "exact_area100": 71.42916666666666}, "test": {"score": [81.429064549534, 81.30824991370383, 81.2909906800138, 81.30824991370383, 81.30824991370383, 81.34276838108389, 81.32550914739386, 81.27373144632378, 81.2909906800138, 81.2909906800138, 81.20469451156369, 81.0666206420435, 80.9975837072834, 80.84225060407319, 80.66965826717293, 80.35899206075251, 80.34173282706249, 80.10010355540214, 79.97928891957199, 79.53054884363134, 79.15084570245081, 78.9264756644805, 78.58129099068002, 78.23610631687953, 77.77010700724888, 77.11425612702796, 76.52744218156714, 76.30307214359682, 75.68173973075596, 74.95685191577495, 74.49085260614429, 74.0766309975837, 73.36900241629272, 72.45426303072145, 72.04004142216085, 71.2806351397998, 70.55574732481877, 69.6410079392475, 68.95063859164652, 68.05315843976527, 67.39730755954436, 66.89678978253366, 66.43079047290301, 66.2064204349327, 65.2916810493614, 64.54953400069037, 63.7901277183293, 63.2896099413186, 62.547462892647566, 61.64998274076632, 60.63168795305489, 59.993096306523995, 59.16465308940283, 58.526061442871935, 57.83569209527097, 56.98998964445978, 56.368657231618926, 55.91991715567829, 55.03969623748706, 54.29754918881602, 53.848809112875394, 53.192958232654476, 52.623403520883684, 51.74318260269244, 51.46703486365205, 50.48325854332067, 50.086296168450126, 49.395926820849155, 48.91266827752848, 48.44666896789783, 47.54918881601657, 46.89333793579566, 45.82326544701416, 45.34000690369348, 44.47704521919227, 43.9247497411115, 43.441491197790825, 42.388677942699346, 41.7846047635485, 41.059716948567484, 40.283051432516395, 39.45460821539524, 38.81601656886434, 37.43527787366241, 37.03831549879186, 36.43424231964101, 35.743872972040045, 35.01898515705903, 34.51846738004833, 33.828098032447365, 33.37935795650673, 32.89609941318606, 32.51639627200553, 31.739730755954437, 30.54884363134277, 29.651363479461512, 29.27166033828098, 28.52951328960994, 27.977217811529165, 27.09699689333793, 26.561960648947192], "calls": [0.0, 1.9330341732827063, 2.5716258198136, 3.19295823265447, 4.055919917155678, 5.591991715567829, 6.2651018294787715, 7.62858129099068, 8.853986882982396, 9.49257852951329, 10.234725578184328, 11.891612012426648, 12.961684501208145, 14.998274076630997, 15.878494994822228, 17.362789092164306, 18.053158439765273, 19.261304798066966, 19.986192613047983, 21.384190541939937, 22.661373835001726, 23.438039351052815, 24.525371073524337, 25.371073524335518, 26.907145322747674, 28.184328615809456, 29.823955816361753, 30.583362098722816, 31.843286158094582, 33.24128408698654, 34.36313427683811, 35.1397997928892, 36.175353814290645, 37.55609250949258, 38.40179496030377, 39.66171901967553, 41.00793924749741, 42.80289955125993, 43.872972040041425, 45.18467380048326, 46.20296858819468, 47.065930272695894, 47.94615119088713, 48.343113565757676, 49.75837072833966, 50.70762858129099, 51.74318260269244, 52.36451501553331, 53.572661373835004, 54.901622367966866, 56.47221263375906, 57.38695201933034, 58.47428374180187, 59.38902312737314, 60.49361408353469, 61.7880566102865, 62.56472212633759, 63.25509147393855, 64.51501553331032, 65.60234725578185, 66.25819813600276, 67.08664135312392, 67.75975146703487, 69.10597169485675, 69.52019330341733, 70.91819123230928, 71.48774594408009, 72.36796686227132, 73.00655850880221, 73.61063168795306, 74.80151881256472, 75.66448049706594, 76.99344149119779, 77.66655160510874, 78.7193648602002, 79.51328960994132, 80.15188125647221, 81.46358301691406, 82.20573006558509, 83.12046945115637, 84.0697273041077, 85.08802209181913, 85.8474283741802, 87.46979634104247, 87.98757335174318, 88.76423886779426, 89.60994131860545, 90.3520883672765, 90.97342078011737, 91.80186399723851, 92.40593717638936, 92.90645495340006, 93.33793579565068, 94.25267518122196, 95.49534000690369, 96.4963755609251, 96.9968933379358, 97.84259578874698, 98.4984466689679, 99.44770452191922, 100.0], "exact_score": [81.429064549534, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.34276838108389, 81.34276838108389, 81.27373144632378, 81.2909906800138, 81.2909906800138, 81.23921297894374, 81.15291681049361, 81.0666206420435, 80.9975837072834, 80.9285467725233, 80.84225060407319, 80.66965826717293, 80.39351052813255, 80.34173282706249, 80.16914049016223, 79.97928891957199, 79.63410424577148, 79.4269934414912, 79.11632723507076, 78.73662409389023, 78.42595788746979, 78.14981014842941, 77.75284777355886, 77.18329306178806, 76.87262685536763, 76.47566448049706, 76.09596133931653, 75.57818432861582, 75.14670348636521, 74.66344494304454, 74.12840869865379, 73.48981705212289, 72.86848463928202, 72.2816706938212, 71.71211598205039, 71.05626510182948, 70.55574732481877, 70.08974801518812, 69.45115636865722, 68.8643424231964, 68.19123230928547, 67.51812219537453, 66.94856748360372, 66.39627200552296, 65.72316189161201, 65.10182947877115, 64.37694166379013, 63.600276147739045, 62.97894373489817, 62.271315153607176, 61.64998274076632, 60.94235415947532, 60.26924404556438, 59.527096996893334, 58.7849499482223, 58.18087676907145, 57.47324818778047, 56.817397307559546, 56.07525025888851, 55.402140144977565, 54.67725232999655, 54.004142216085604, 53.29651363479462, 52.46807041767345, 51.81221953745254, 51.10459095616156, 50.4314808422506, 49.74111149464964, 48.9299275112185, 48.118743527787366, 47.37659647911633, 46.634449430445294, 45.82326544701416, 45.081118398343115, 44.25267518122196, 43.56230583362099, 42.751121850189854, 41.93993786675872, 41.1460131170176, 40.352088367276494, 39.506385916465305, 38.67794269934415, 37.76320331377287, 37.03831549879186, 36.27890921643079, 35.36416983085951, 34.5012081463583, 33.67276492923714, 32.82706247842596, 31.94684156023473, 31.04936140835347, 30.100103555402146, 29.27166033828098, 28.374180186399723, 27.511218501898515, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 80.26579219882639, "exact_area100": 61.66475168806268}}, "old_normalized_endpoint_budget": 1}, {"seed": 2029, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.0, "validation_area100": 0.7172166666666666, "training_seconds": 3.3971830509835854, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.006374697666615248, "operator": "ge"}, {"budget": 0.02, "threshold": -0.008000615052878857, "operator": "ge"}, {"budget": 0.03, "threshold": -0.01090380921959877, "operator": "ge"}, {"budget": 0.04, "threshold": -0.012897567823529243, "operator": "ge"}, {"budget": 0.05, "threshold": -0.018178042024374008, "operator": "ge"}, {"budget": 0.06, "threshold": -0.0229185800999403, "operator": "ge"}, {"budget": 0.07, "threshold": -0.03192461282014847, "operator": "ge"}, {"budget": 0.08, "threshold": -0.03905806317925453, "operator": "ge"}, {"budget": 0.09, "threshold": -0.05093434080481529, "operator": "ge"}, {"budget": 0.1, "threshold": -0.06854777783155441, "operator": "ge"}, {"budget": 0.11, "threshold": -0.09115777909755707, "operator": "ge"}, {"budget": 0.12, "threshold": -0.12664136290550232, "operator": "ge"}, {"budget": 0.13, "threshold": -0.16334937512874603, "operator": "ge"}, {"budget": 0.14, "threshold": -0.18040838837623596, "operator": "ge"}, {"budget": 0.15, "threshold": -0.20965366065502167, "operator": "ge"}, {"budget": 0.16, "threshold": -0.22695674002170563, "operator": "ge"}, {"budget": 0.17, "threshold": -0.24349530041217804, "operator": "ge"}, {"budget": 0.18, "threshold": -0.2722572386264801, "operator": "ge"}, {"budget": 0.19, "threshold": -0.30108189582824707, "operator": "ge"}, {"budget": 0.2, "threshold": -0.32632365822792053, "operator": "ge"}, {"budget": 0.21, "threshold": -0.36973875761032104, "operator": "ge"}, {"budget": 0.22, "threshold": -0.4238666594028473, "operator": "ge"}, {"budget": 0.23, "threshold": -0.465654194355011, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5233580470085144, "operator": "ge"}, {"budget": 0.25, "threshold": -0.5518055558204651, "operator": "ge"}, {"budget": 0.26, "threshold": -0.5899313688278198, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6385111212730408, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6671701669692993, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6991006731987, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7283256649971008, "operator": "ge"}, {"budget": 0.31, "threshold": -0.7502700090408325, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7730559706687927, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7973757386207581, "operator": "ge"}, {"budget": 0.34, "threshold": -0.812284529209137, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8362254500389099, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8537514209747314, "operator": "ge"}, {"budget": 0.37, "threshold": -0.870746374130249, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8842661380767822, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8919141292572021, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8997867107391357, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9053520560264587, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9122543334960938, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9199572205543518, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9269464015960693, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9311970472335815, "operator": "ge"}, {"budget": 0.46, "threshold": -0.93575119972229, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9388840794563293, "operator": "ge"}, {"budget": 0.48, "threshold": -0.944952130317688, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9483853578567505, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9511979818344116, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9536014795303345, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9573624134063721, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9596214890480042, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9623753428459167, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9653403162956238, "operator": "ge"}, {"budget": 0.56, "threshold": -0.968255877494812, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9697988033294678, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9721333384513855, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9737838506698608, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9754050374031067, "operator": "ge"}, {"budget": 0.61, "threshold": -0.977423906326294, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9788259267807007, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9806696176528931, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9828256368637085, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9846146702766418, "operator": "ge"}, {"budget": 0.66, "threshold": -0.985964298248291, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9869765043258667, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9879208207130432, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9885963201522827, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9892610907554626, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9902253150939941, "operator": "ge"}, {"budget": 0.72, "threshold": -0.990734875202179, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9915135502815247, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9922994375228882, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9927964210510254, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9932020306587219, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9939956068992615, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9944846630096436, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9949446320533752, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9957988262176514, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9961876273155212, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9963974356651306, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9966326951980591, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9969083666801453, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9972094893455505, "operator": "ge"}, {"budget": 0.86, "threshold": -0.997473955154419, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9979202747344971, "operator": "ge"}, {"budget": 0.88, "threshold": -0.998117983341217, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9984232187271118, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9986497759819031, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9987273216247559, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9988301396369934, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9989836812019348, "operator": "gt"}, {"budget": 0.94, "threshold": -0.9991183876991272, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9992983937263489, "operator": "ge"}, {"budget": 0.96, "threshold": -0.999452531337738, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9995492100715637, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9996384382247925, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9997320175170898, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.41666666666666, 98.25, 98.25, 98.25, 98.08333333333333, 97.75, 97.58333333333333, 97.33333333333334, 97.16666666666667, 96.75, 96.5, 96.33333333333334, 96.16666666666667, 95.75, 95.5, 95.33333333333334, 94.75, 94.25, 93.33333333333333, 93.25, 92.58333333333333, 92.0, 91.5, 90.91666666666667, 90.16666666666666, 89.66666666666667, 88.83333333333333, 88.16666666666667, 87.58333333333333, 86.66666666666667, 85.75, 84.75, 83.91666666666666, 83.25, 82.41666666666667, 81.5, 80.5, 79.66666666666666, 78.83333333333333, 77.83333333333333, 76.83333333333333, 76.08333333333334, 75.33333333333333, 74.41666666666666, 73.5, 72.66666666666667, 71.75, 70.75, 70.0, 69.16666666666667, 68.25, 67.33333333333333, 66.41666666666667, 65.41666666666667, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.5, 60.5, 59.66666666666667, 58.833333333333336, 58.08333333333333, 57.00000000000001, 56.08333333333333, 55.16666666666667, 54.16666666666667, 53.16666666666667, 52.16666666666667, 51.24999999999999, 50.24999999999999, 49.333333333333336, 48.33333333333334, 47.333333333333336, 46.333333333333336, 45.41666666666667, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.5, 38.5, 37.5, 36.583333333333336, 35.583333333333336, 34.66666666666667, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 92.91666666666667, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 1, 2, 4, 4, 4, 6, 10, 12, 15, 17, 22, 26, 28, 30, 35, 39, 41, 48, 55, 66, 68, 76, 83, 89, 96, 105, 111, 121, 129, 136, 147, 158, 170, 180, 188, 198, 209, 221, 231, 241, 253, 265, 274, 284, 295, 306, 316, 327, 339, 348, 358, 369, 380, 391, 403, 415, 427, 439, 450, 462, 472, 482, 492, 505, 516, 527, 539, 551, 563, 574, 586, 597, 609, 621, 633, 644, 657, 669, 681, 692, 704, 715, 727, 739, 750, 762, 773, 786, 798, 810, 821, 833, 844, 856], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.41666666666666, 98.25, 98.25, 98.25, 98.08333333333333, 97.75, 97.58333333333333, 97.33333333333334, 97.16666666666667, 96.75, 96.5, 96.33333333333334, 96.16666666666667, 95.75, 95.5, 95.33333333333334, 94.75, 94.25, 93.33333333333333, 93.25, 92.58333333333333, 92.0, 91.5, 90.91666666666667, 90.16666666666666, 89.66666666666667, 88.83333333333333, 88.16666666666667, 87.58333333333333, 86.66666666666667, 85.75, 84.75, 83.91666666666666, 83.25, 82.41666666666667, 81.5, 80.5, 79.66666666666666, 78.83333333333333, 77.83333333333333, 76.83333333333333, 76.08333333333334, 75.33333333333333, 74.41666666666666, 73.5, 72.66666666666667, 71.75, 70.75, 70.0, 69.16666666666667, 68.25, 67.33333333333333, 66.41666666666667, 65.41666666666667, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.5, 60.5, 59.66666666666667, 58.833333333333336, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.16666666666667, 54.16666666666667, 53.16666666666667, 52.16666666666667, 51.24999999999999, 50.24999999999999, 49.333333333333336, 48.33333333333334, 47.333333333333336, 46.333333333333336, 45.333333333333336, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.5, 38.5, 37.5, 36.583333333333336, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.72166666666666, "threshold_area100": 71.72416666666666, "threshold_area25": 97.35000000000001}, "test": {"score": [81.429064549534, 81.44632378322403, 81.41180531584398, 81.39454608215395, 81.41180531584398, 81.30824991370383, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.27373144632378, 81.15291681049361, 81.10113910942354, 80.89402830514325, 80.58336209872282, 80.47980669658268, 80.32447359337246, 80.18639972385226, 80.08284432171212, 79.94477045219193, 79.75491888160165, 79.54780807732136, 79.25440110459095, 78.75388332758025, 78.23610631687953, 77.47670003451846, 77.06247842595789, 76.28581290990681, 75.66448049706594, 75.18122195374526, 74.66344494304454, 73.81774249223335, 73.21366931308249, 72.6441146013117, 71.76389368312047, 70.98722816706939, 70.17604418363825, 69.17500862961685, 68.0704176734553, 67.17293752157406, 66.48256817397308, 65.87849499482223, 65.41249568519159, 64.77390403866067, 63.63479461511909, 62.73731446323784, 62.21953745253711, 61.615464273386266, 61.18398343113566, 60.303762512944424, 59.68243010010356, 59.04383845357266, 58.664135312392126, 57.73213669313083, 57.024508111839836, 56.23058336209872, 55.54021401449776, 54.40110459095616, 53.88332758025545, 53.26199516741457, 52.70969968933379, 52.122885743872985, 51.0010355540214, 50.569554711770806, 49.58577839143942, 48.58474283741802, 47.30755954435623, 46.703486365205386, 46.15119088712461, 45.46082153952365, 45.081118398343115, 44.68415602347256, 44.04556437694166, 43.803935105281326, 43.251639627200554, 42.63030721435969, 41.95719710044874, 41.56023472557819, 40.766309975837075, 40.2140144977563, 39.558163617535385, 37.76320331377287, 36.98653779772179, 36.62409389023127, 36.158094580600626, 35.58853986882983, 35.01898515705903, 34.58750431480843, 33.65550569554712, 33.37935795650673, 32.68898860890577, 31.79150845702451, 31.515360717984127, 31.256472212633756, 30.85950983776321, 30.289955125992407, 29.58232654470142, 28.909216430790472, 28.391439420089746, 27.87366240938902, 27.37314463237832, 26.561960648947192], "calls": [0.0, 1.0182947877114257, 1.6223679668622715, 2.7959958577839146, 3.503624439074905, 5.229547808077322, 6.196064894718675, 7.4387297204004135, 8.180876769071453, 9.458060062133242, 11.011391094235416, 12.478425957887469, 14.60131170176044, 16.75871591301346, 17.725232999654818, 19.071453227476702, 19.796341042457716, 20.331377286848465, 21.401449775629963, 22.35070762858129, 23.144632378322402, 24.456334138764237, 26.009665170866413, 27.20055229547808, 29.09906800138074, 30.013807386952017, 31.687953054884368, 33.569209527097, 34.72557818432861, 36.175353814290645, 37.72868484639282, 39.04038660683465, 40.196755264066276, 41.767345529858474, 43.13082499137038, 44.68415602347256, 46.25474628926476, 48.30859509837763, 49.58577839143942, 50.56955471177079, 51.39799792889196, 52.31273731446324, 53.48636520538488, 54.93614083534691, 56.403175698998965, 57.30065585088022, 58.26717293752157, 58.940283051432516, 60.16568864342423, 61.011391094235414, 61.839834311356576, 62.39212978943735, 63.65205384880911, 64.53227476700035, 65.61960648947188, 66.46530894028305, 68.03589920607524, 68.67449085260614, 69.67552640662754, 70.33137728684846, 71.21159820503969, 72.52329996548154, 73.24818778046254, 74.31826026924404, 75.56092509492579, 76.95892302381773, 77.68381083879876, 78.30514325163963, 79.04729030031066, 79.53054884363134, 79.996548153262, 80.80773213669313, 81.11839834311357, 81.73973075595443, 82.46461857093546, 83.2067656196065, 83.6382464618571, 84.53572661373835, 85.19157749395927, 85.91646530894027, 87.90127718329306, 88.76423886779426, 89.2129789437349, 89.74801518812565, 90.38660683465655, 91.02519848118743, 91.49119779081809, 92.50949257852952, 92.85467725232999, 93.61408353469106, 94.54608215395236, 94.82222989299275, 95.08111839834311, 95.54711770797375, 96.16845012081463, 96.87607870210563, 97.54918881601657, 98.0669658267173, 98.60200207110805, 99.18881601656886, 100.0], "rescued": [0, 4, 4, 5, 6, 7, 10, 10, 10, 12, 12, 17, 21, 26, 27, 31, 35, 35, 42, 45, 46, 56, 63, 64, 67, 68, 73, 82, 88, 98, 101, 106, 111, 114, 121, 128, 132, 142, 142, 142, 144, 148, 155, 155, 162, 163, 166, 169, 172, 173, 176, 177, 180, 181, 185, 186, 191, 192, 198, 200, 204, 205, 211, 211, 213, 213, 213, 213, 214, 215, 215, 219, 219, 219, 219, 220, 220, 221, 222, 222, 224, 225, 227, 228, 228, 228, 228, 229, 229, 230, 230, 230, 230, 230, 231, 231, 231, 231, 231, 232, 232], "harmed": [0, 3, 5, 7, 7, 14, 14, 15, 17, 21, 28, 36, 52, 75, 82, 95, 107, 113, 128, 142, 155, 182, 218, 249, 296, 321, 371, 416, 450, 490, 542, 582, 620, 674, 726, 780, 842, 916, 968, 1008, 1045, 1076, 1120, 1186, 1245, 1276, 1314, 1342, 1396, 1433, 1473, 1496, 1553, 1595, 1645, 1686, 1757, 1788, 1830, 1864, 1902, 1968, 1999, 2056, 2116, 2190, 2225, 2257, 2298, 2321, 2344, 2385, 2399, 2431, 2467, 2507, 2530, 2577, 2610, 2648, 2754, 2800, 2823, 2851, 2884, 2917, 2942, 2997, 3013, 3054, 3106, 3122, 3137, 3160, 3194, 3235, 3274, 3304, 3334, 3364, 3411], "exact_score": [81.429064549534, 81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.32550914739386, 81.32550914739386, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.22195374525371, 81.15291681049361, 81.11839834311357, 81.08387987573352, 80.96306523990336, 80.84225060407319, 80.68691750086296, 80.56610286503279, 80.4280289955126, 80.30721435968243, 80.15188125647221, 80.01380738695202, 79.77217811529168, 79.56506731101139, 79.40973420780118, 79.08180876769072, 78.73662409389023, 78.32240248532966, 77.856403175699, 77.49395926820849, 77.07973765964792, 76.57921988263722, 76.14773904038661, 75.81981360027615, 75.40559199171557, 75.11218501898516, 74.7842595788747, 74.37003797031412, 73.74870555747324, 73.23092854677252, 72.7821884708319, 72.19537452537108, 71.62581981360027, 71.09078356920952, 70.53848809112876, 70.0724887814981, 69.38211943389715, 68.82982395581637, 68.26026924404557, 67.55264066275457, 66.8622713151536, 66.17190196755264, 65.60234725578185, 65.05005177770107, 64.37694166379013, 63.600276147739045, 62.97894373489817, 62.37487055574733, 61.7880566102865, 61.13220573006559, 60.40731791508457, 59.68243010010356, 58.940283051432516, 58.18087676907145, 57.455988954090444, 56.748360372799446, 55.93717638936832, 55.05695547117708, 54.43562305833621, 53.693476009665176, 53.003106662064205, 52.27821884708319, 51.43251639627201, 50.793924749741116, 49.896444597859855, 49.08526061442872, 48.20503969623749, 47.273041076976185, 46.41007939247498, 45.51259924059372, 44.68415602347256, 43.872972040041425, 42.99275112185019, 42.16430790472903, 41.249568519157755, 40.36934760096652, 39.506385916465305, 38.574387297204005, 37.69416637901277, 36.76216775975147, 35.93372454263031, 35.036244390749054, 34.1387642388678, 33.258543320676566, 32.30928546772523, 31.342768381083886, 30.445288229202628, 29.461511908871252, 28.460476354849852, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 62.69335465288566, "threshold_area100": 58.076544701415266, "threshold_area25": 80.31584397652745}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.006374697666615248, "operator": "ge"}, {"budget": 0.02, "threshold": -0.008000615052878857, "operator": "ge"}, {"budget": 0.03, "threshold": -0.01090380921959877, "operator": "ge"}, {"budget": 0.04, "threshold": -0.012897567823529243, "operator": "ge"}, {"budget": 0.05, "threshold": -0.018178042024374008, "operator": "ge"}, {"budget": 0.06, "threshold": -0.0229185800999403, "operator": "ge"}, {"budget": 0.07, "threshold": -0.03192461282014847, "operator": "ge"}, {"budget": 0.08, "threshold": -0.03905806317925453, "operator": "ge"}, {"budget": 0.09, "threshold": -0.05093434080481529, "operator": "ge"}, {"budget": 0.1, "threshold": -0.06854777783155441, "operator": "ge"}, {"budget": 0.11, "threshold": -0.09115777909755707, "operator": "ge"}, {"budget": 0.12, "threshold": -0.12664136290550232, "operator": "ge"}, {"budget": 0.13, "threshold": -0.16334937512874603, "operator": "ge"}, {"budget": 0.14, "threshold": -0.18040838837623596, "operator": "ge"}, {"budget": 0.15, "threshold": -0.20965366065502167, "operator": "ge"}, {"budget": 0.16, "threshold": -0.22695674002170563, "operator": "ge"}, {"budget": 0.17, "threshold": -0.24349530041217804, "operator": "ge"}, {"budget": 0.18, "threshold": -0.2722572386264801, "operator": "ge"}, {"budget": 0.19, "threshold": -0.30108189582824707, "operator": "ge"}, {"budget": 0.2, "threshold": -0.32632365822792053, "operator": "ge"}, {"budget": 0.21, "threshold": -0.36973875761032104, "operator": "ge"}, {"budget": 0.22, "threshold": -0.4238666594028473, "operator": "ge"}, {"budget": 0.23, "threshold": -0.465654194355011, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5233580470085144, "operator": "ge"}, {"budget": 0.25, "threshold": -0.5518055558204651, "operator": "ge"}, {"budget": 0.26, "threshold": -0.5899313688278198, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6385111212730408, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6671701669692993, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6991006731987, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7283256649971008, "operator": "ge"}, {"budget": 0.31, "threshold": -0.7502700090408325, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7730559706687927, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7973757386207581, "operator": "ge"}, {"budget": 0.34, "threshold": -0.812284529209137, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8362254500389099, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8537514209747314, "operator": "ge"}, {"budget": 0.37, "threshold": -0.870746374130249, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8842661380767822, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8919141292572021, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8997867107391357, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9053520560264587, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9122543334960938, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9199572205543518, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9269464015960693, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9311970472335815, "operator": "ge"}, {"budget": 0.46, "threshold": -0.93575119972229, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9388840794563293, "operator": "ge"}, {"budget": 0.48, "threshold": -0.944952130317688, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9483853578567505, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9511979818344116, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9536014795303345, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9573624134063721, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9596214890480042, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9623753428459167, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9653403162956238, "operator": "ge"}, {"budget": 0.56, "threshold": -0.968255877494812, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9697988033294678, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9721333384513855, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9737838506698608, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9754050374031067, "operator": "ge"}, {"budget": 0.61, "threshold": -0.977423906326294, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9788259267807007, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9806696176528931, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9828256368637085, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9846146702766418, "operator": "ge"}, {"budget": 0.66, "threshold": -0.985964298248291, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9869765043258667, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9879208207130432, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9885963201522827, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9892610907554626, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9902253150939941, "operator": "ge"}, {"budget": 0.72, "threshold": -0.990734875202179, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9915135502815247, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9922994375228882, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9927964210510254, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9932020306587219, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9939956068992615, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9944846630096436, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9949446320533752, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9957988262176514, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9961876273155212, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9963974356651306, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9966326951980591, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9969083666801453, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9972094893455505, "operator": "ge"}, {"budget": 0.86, "threshold": -0.997473955154419, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9979202747344971, "operator": "ge"}, {"budget": 0.88, "threshold": -0.998117983341217, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9984232187271118, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9986497759819031, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9987273216247559, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9988301396369934, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9989836812019348, "operator": "gt"}, {"budget": 0.94, "threshold": -0.9991183876991272, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9992983937263489, "operator": "ge"}, {"budget": 0.96, "threshold": -0.999452531337738, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9995492100715637, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9996384382247925, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9997320175170898, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.41666666666666, 98.25, 98.25, 98.25, 98.08333333333333, 97.75, 97.58333333333333, 97.33333333333334, 97.16666666666667, 96.75, 96.5, 96.33333333333334, 96.16666666666667, 95.75, 95.5, 95.33333333333334, 94.75, 94.25, 93.33333333333333, 93.25, 92.58333333333333, 92.0, 91.5, 90.91666666666667, 90.16666666666666, 89.66666666666667, 88.83333333333333, 88.16666666666667, 87.58333333333333, 86.66666666666667, 85.75, 84.75, 83.91666666666666, 83.25, 82.41666666666667, 81.5, 80.5, 79.66666666666666, 78.83333333333333, 77.83333333333333, 76.83333333333333, 76.08333333333334, 75.33333333333333, 74.41666666666666, 73.5, 72.66666666666667, 71.75, 70.75, 70.0, 69.16666666666667, 68.25, 67.33333333333333, 66.41666666666667, 65.41666666666667, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.5, 60.5, 59.66666666666667, 58.833333333333336, 58.08333333333333, 57.00000000000001, 56.08333333333333, 55.16666666666667, 54.16666666666667, 53.16666666666667, 52.16666666666667, 51.24999999999999, 50.24999999999999, 49.333333333333336, 48.33333333333334, 47.333333333333336, 46.333333333333336, 45.41666666666667, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.5, 38.5, 37.5, 36.583333333333336, 35.583333333333336, 34.66666666666667, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 92.91666666666667, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 1, 2, 4, 4, 4, 6, 10, 12, 15, 17, 22, 26, 28, 30, 35, 39, 41, 48, 55, 66, 68, 76, 83, 89, 96, 105, 111, 121, 129, 136, 147, 158, 170, 180, 188, 198, 209, 221, 231, 241, 253, 265, 274, 284, 295, 306, 316, 327, 339, 348, 358, 369, 380, 391, 403, 415, 427, 439, 450, 462, 472, 482, 492, 505, 516, 527, 539, 551, 563, 574, 586, 597, 609, 621, 633, 644, 657, 669, 681, 692, 704, 715, 727, 739, 750, 762, 773, 786, 798, 810, 821, 833, 844, 856], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.41666666666666, 98.25, 98.25, 98.25, 98.08333333333333, 97.75, 97.58333333333333, 97.33333333333334, 97.16666666666667, 96.75, 96.5, 96.33333333333334, 96.16666666666667, 95.75, 95.5, 95.33333333333334, 94.75, 94.25, 93.33333333333333, 93.25, 92.58333333333333, 92.0, 91.5, 90.91666666666667, 90.16666666666666, 89.66666666666667, 88.83333333333333, 88.16666666666667, 87.58333333333333, 86.66666666666667, 85.75, 84.75, 83.91666666666666, 83.25, 82.41666666666667, 81.5, 80.5, 79.66666666666666, 78.83333333333333, 77.83333333333333, 76.83333333333333, 76.08333333333334, 75.33333333333333, 74.41666666666666, 73.5, 72.66666666666667, 71.75, 70.75, 70.0, 69.16666666666667, 68.25, 67.33333333333333, 66.41666666666667, 65.41666666666667, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.5, 60.5, 59.66666666666667, 58.833333333333336, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.16666666666667, 54.16666666666667, 53.16666666666667, 52.16666666666667, 51.24999999999999, 50.24999999999999, 49.333333333333336, 48.33333333333334, 47.333333333333336, 46.333333333333336, 45.333333333333336, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.5, 38.5, 37.5, 36.583333333333336, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.72166666666666, "threshold_area100": 71.72416666666666, "threshold_area25": 97.35000000000001}, "test": {"score": [81.429064549534, 81.44632378322403, 81.41180531584398, 81.39454608215395, 81.41180531584398, 81.30824991370383, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.27373144632378, 81.15291681049361, 81.10113910942354, 80.89402830514325, 80.58336209872282, 80.47980669658268, 80.32447359337246, 80.18639972385226, 80.08284432171212, 79.94477045219193, 79.75491888160165, 79.54780807732136, 79.25440110459095, 78.75388332758025, 78.23610631687953, 77.47670003451846, 77.06247842595789, 76.28581290990681, 75.66448049706594, 75.18122195374526, 74.66344494304454, 73.81774249223335, 73.21366931308249, 72.6441146013117, 71.76389368312047, 70.98722816706939, 70.17604418363825, 69.17500862961685, 68.0704176734553, 67.17293752157406, 66.48256817397308, 65.87849499482223, 65.41249568519159, 64.77390403866067, 63.63479461511909, 62.73731446323784, 62.21953745253711, 61.615464273386266, 61.18398343113566, 60.303762512944424, 59.68243010010356, 59.04383845357266, 58.664135312392126, 57.73213669313083, 57.024508111839836, 56.23058336209872, 55.54021401449776, 54.40110459095616, 53.88332758025545, 53.26199516741457, 52.70969968933379, 52.122885743872985, 51.0010355540214, 50.569554711770806, 49.58577839143942, 48.58474283741802, 47.30755954435623, 46.703486365205386, 46.15119088712461, 45.46082153952365, 45.081118398343115, 44.68415602347256, 44.04556437694166, 43.803935105281326, 43.251639627200554, 42.63030721435969, 41.95719710044874, 41.56023472557819, 40.766309975837075, 40.2140144977563, 39.558163617535385, 37.76320331377287, 36.98653779772179, 36.62409389023127, 36.158094580600626, 35.58853986882983, 35.01898515705903, 34.58750431480843, 33.65550569554712, 33.37935795650673, 32.68898860890577, 31.79150845702451, 31.515360717984127, 31.256472212633756, 30.85950983776321, 30.289955125992407, 29.58232654470142, 28.909216430790472, 28.391439420089746, 27.87366240938902, 27.37314463237832, 26.561960648947192], "calls": [0.0, 1.0182947877114257, 1.6223679668622715, 2.7959958577839146, 3.503624439074905, 5.229547808077322, 6.196064894718675, 7.4387297204004135, 8.180876769071453, 9.458060062133242, 11.011391094235416, 12.478425957887469, 14.60131170176044, 16.75871591301346, 17.725232999654818, 19.071453227476702, 19.796341042457716, 20.331377286848465, 21.401449775629963, 22.35070762858129, 23.144632378322402, 24.456334138764237, 26.009665170866413, 27.20055229547808, 29.09906800138074, 30.013807386952017, 31.687953054884368, 33.569209527097, 34.72557818432861, 36.175353814290645, 37.72868484639282, 39.04038660683465, 40.196755264066276, 41.767345529858474, 43.13082499137038, 44.68415602347256, 46.25474628926476, 48.30859509837763, 49.58577839143942, 50.56955471177079, 51.39799792889196, 52.31273731446324, 53.48636520538488, 54.93614083534691, 56.403175698998965, 57.30065585088022, 58.26717293752157, 58.940283051432516, 60.16568864342423, 61.011391094235414, 61.839834311356576, 62.39212978943735, 63.65205384880911, 64.53227476700035, 65.61960648947188, 66.46530894028305, 68.03589920607524, 68.67449085260614, 69.67552640662754, 70.33137728684846, 71.21159820503969, 72.52329996548154, 73.24818778046254, 74.31826026924404, 75.56092509492579, 76.95892302381773, 77.68381083879876, 78.30514325163963, 79.04729030031066, 79.53054884363134, 79.996548153262, 80.80773213669313, 81.11839834311357, 81.73973075595443, 82.46461857093546, 83.2067656196065, 83.6382464618571, 84.53572661373835, 85.19157749395927, 85.91646530894027, 87.90127718329306, 88.76423886779426, 89.2129789437349, 89.74801518812565, 90.38660683465655, 91.02519848118743, 91.49119779081809, 92.50949257852952, 92.85467725232999, 93.61408353469106, 94.54608215395236, 94.82222989299275, 95.08111839834311, 95.54711770797375, 96.16845012081463, 96.87607870210563, 97.54918881601657, 98.0669658267173, 98.60200207110805, 99.18881601656886, 100.0], "rescued": [0, 4, 4, 5, 6, 7, 10, 10, 10, 12, 12, 17, 21, 26, 27, 31, 35, 35, 42, 45, 46, 56, 63, 64, 67, 68, 73, 82, 88, 98, 101, 106, 111, 114, 121, 128, 132, 142, 142, 142, 144, 148, 155, 155, 162, 163, 166, 169, 172, 173, 176, 177, 180, 181, 185, 186, 191, 192, 198, 200, 204, 205, 211, 211, 213, 213, 213, 213, 214, 215, 215, 219, 219, 219, 219, 220, 220, 221, 222, 222, 224, 225, 227, 228, 228, 228, 228, 229, 229, 230, 230, 230, 230, 230, 231, 231, 231, 231, 231, 232, 232], "harmed": [0, 3, 5, 7, 7, 14, 14, 15, 17, 21, 28, 36, 52, 75, 82, 95, 107, 113, 128, 142, 155, 182, 218, 249, 296, 321, 371, 416, 450, 490, 542, 582, 620, 674, 726, 780, 842, 916, 968, 1008, 1045, 1076, 1120, 1186, 1245, 1276, 1314, 1342, 1396, 1433, 1473, 1496, 1553, 1595, 1645, 1686, 1757, 1788, 1830, 1864, 1902, 1968, 1999, 2056, 2116, 2190, 2225, 2257, 2298, 2321, 2344, 2385, 2399, 2431, 2467, 2507, 2530, 2577, 2610, 2648, 2754, 2800, 2823, 2851, 2884, 2917, 2942, 2997, 3013, 3054, 3106, 3122, 3137, 3160, 3194, 3235, 3274, 3304, 3334, 3364, 3411], "exact_score": [81.429064549534, 81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.32550914739386, 81.32550914739386, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.22195374525371, 81.15291681049361, 81.11839834311357, 81.08387987573352, 80.96306523990336, 80.84225060407319, 80.68691750086296, 80.56610286503279, 80.4280289955126, 80.30721435968243, 80.15188125647221, 80.01380738695202, 79.77217811529168, 79.56506731101139, 79.40973420780118, 79.08180876769072, 78.73662409389023, 78.32240248532966, 77.856403175699, 77.49395926820849, 77.07973765964792, 76.57921988263722, 76.14773904038661, 75.81981360027615, 75.40559199171557, 75.11218501898516, 74.7842595788747, 74.37003797031412, 73.74870555747324, 73.23092854677252, 72.7821884708319, 72.19537452537108, 71.62581981360027, 71.09078356920952, 70.53848809112876, 70.0724887814981, 69.38211943389715, 68.82982395581637, 68.26026924404557, 67.55264066275457, 66.8622713151536, 66.17190196755264, 65.60234725578185, 65.05005177770107, 64.37694166379013, 63.600276147739045, 62.97894373489817, 62.37487055574733, 61.7880566102865, 61.13220573006559, 60.40731791508457, 59.68243010010356, 58.940283051432516, 58.18087676907145, 57.455988954090444, 56.748360372799446, 55.93717638936832, 55.05695547117708, 54.43562305833621, 53.693476009665176, 53.003106662064205, 52.27821884708319, 51.43251639627201, 50.793924749741116, 49.896444597859855, 49.08526061442872, 48.20503969623749, 47.273041076976185, 46.41007939247498, 45.51259924059372, 44.68415602347256, 43.872972040041425, 42.99275112185019, 42.16430790472903, 41.249568519157755, 40.36934760096652, 39.506385916465305, 38.574387297204005, 37.69416637901277, 36.76216775975147, 35.93372454263031, 35.036244390749054, 34.1387642388678, 33.258543320676566, 32.30928546772523, 31.342768381083886, 30.445288229202628, 29.461511908871252, 28.460476354849852, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 62.69335465288566, "threshold_area100": 58.076544701415266, "threshold_area25": 80.31584397652745}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.230358362197876, "operator": "ge"}, {"budget": 0.02, "threshold": -2.7401976585388184, "operator": "ge"}, {"budget": 0.03, "threshold": -3.7350120544433594, "operator": "ge"}, {"budget": 0.04, "threshold": -4.253167152404785, "operator": "ge"}, {"budget": 0.05, "threshold": -4.692059516906738, "operator": "ge"}, {"budget": 0.06, "threshold": -5.074877738952637, "operator": "ge"}, {"budget": 0.07, "threshold": -5.3259782791137695, "operator": "ge"}, {"budget": 0.08, "threshold": -5.653398513793945, "operator": "ge"}, {"budget": 0.09, "threshold": -5.925279140472412, "operator": "ge"}, {"budget": 0.1, "threshold": -6.3245110511779785, "operator": "ge"}, {"budget": 0.11, "threshold": -6.5330071449279785, "operator": "ge"}, {"budget": 0.12, "threshold": -6.934679985046387, "operator": "ge"}, {"budget": 0.13, "threshold": -7.200185775756836, "operator": "ge"}, {"budget": 0.14, "threshold": -7.525809288024902, "operator": "ge"}, {"budget": 0.15, "threshold": -7.756378173828125, "operator": "ge"}, {"budget": 0.16, "threshold": -7.945965766906738, "operator": "ge"}, {"budget": 0.17, "threshold": -8.121847152709961, "operator": "ge"}, {"budget": 0.18, "threshold": -8.297152519226074, "operator": "ge"}, {"budget": 0.19, "threshold": -8.450115203857422, "operator": "ge"}, {"budget": 0.2, "threshold": -8.544468879699707, "operator": "ge"}, {"budget": 0.21, "threshold": -8.758308410644531, "operator": "ge"}, {"budget": 0.22, "threshold": -8.85645866394043, "operator": "ge"}, {"budget": 0.23, "threshold": -8.988407135009766, "operator": "ge"}, {"budget": 0.24, "threshold": -9.06076431274414, "operator": "ge"}, {"budget": 0.25, "threshold": -9.169647216796875, "operator": "ge"}, {"budget": 0.26, "threshold": -9.274831771850586, "operator": "ge"}, {"budget": 0.27, "threshold": -9.386558532714844, "operator": "ge"}, {"budget": 0.28, "threshold": -9.51878833770752, "operator": "ge"}, {"budget": 0.29, "threshold": -9.591014862060547, "operator": "ge"}, {"budget": 0.3, "threshold": -9.645881652832031, "operator": "ge"}, {"budget": 0.31, "threshold": -9.744115829467773, "operator": "ge"}, {"budget": 0.32, "threshold": -9.843992233276367, "operator": "ge"}, {"budget": 0.33, "threshold": -9.941534042358398, "operator": "ge"}, {"budget": 0.34, "threshold": -9.993708610534668, "operator": "ge"}, {"budget": 0.35, "threshold": -10.058650970458984, "operator": "ge"}, {"budget": 0.36, "threshold": -10.104426383972168, "operator": "ge"}, {"budget": 0.37, "threshold": -10.215034484863281, "operator": "ge"}, {"budget": 0.38, "threshold": -10.267372131347656, "operator": "ge"}, {"budget": 0.39, "threshold": -10.387877464294434, "operator": "ge"}, {"budget": 0.4, "threshold": -10.448406219482422, "operator": "ge"}, {"budget": 0.41, "threshold": -10.497751235961914, "operator": "ge"}, {"budget": 0.42, "threshold": -10.543270111083984, "operator": "ge"}, {"budget": 0.43, "threshold": -10.63748550415039, "operator": "ge"}, {"budget": 0.44, "threshold": -10.70160961151123, "operator": "ge"}, {"budget": 0.45, "threshold": -10.783834457397461, "operator": "ge"}, {"budget": 0.46, "threshold": -10.82375431060791, "operator": "ge"}, {"budget": 0.47, "threshold": -10.873178482055664, "operator": "ge"}, {"budget": 0.48, "threshold": -10.961721420288086, "operator": "ge"}, {"budget": 0.49, "threshold": -11.005008697509766, "operator": "ge"}, {"budget": 0.5, "threshold": -11.112348556518555, "operator": "ge"}, {"budget": 0.51, "threshold": -11.19715690612793, "operator": "ge"}, {"budget": 0.52, "threshold": -11.257482528686523, "operator": "ge"}, {"budget": 0.53, "threshold": -11.3438720703125, "operator": "ge"}, {"budget": 0.54, "threshold": -11.435173988342285, "operator": "ge"}, {"budget": 0.55, "threshold": -11.50180435180664, "operator": "ge"}, {"budget": 0.56, "threshold": -11.555580139160156, "operator": "ge"}, {"budget": 0.57, "threshold": -11.618733406066895, "operator": "ge"}, {"budget": 0.58, "threshold": -11.70974063873291, "operator": "ge"}, {"budget": 0.59, "threshold": -11.752419471740723, "operator": "ge"}, {"budget": 0.6, "threshold": -11.831430435180664, "operator": "ge"}, {"budget": 0.61, "threshold": -11.919034957885742, "operator": "ge"}, {"budget": 0.62, "threshold": -12.002429962158203, "operator": "ge"}, {"budget": 0.63, "threshold": -12.084551811218262, "operator": "ge"}, {"budget": 0.64, "threshold": -12.118209838867188, "operator": "ge"}, {"budget": 0.65, "threshold": -12.177262306213379, "operator": "ge"}, {"budget": 0.66, "threshold": -12.262048721313477, "operator": "ge"}, {"budget": 0.67, "threshold": -12.372838973999023, "operator": "ge"}, {"budget": 0.68, "threshold": -12.506131172180176, "operator": "ge"}, {"budget": 0.69, "threshold": -12.563255310058594, "operator": "ge"}, {"budget": 0.7, "threshold": -12.623400688171387, "operator": "ge"}, {"budget": 0.71, "threshold": -12.74046516418457, "operator": "ge"}, {"budget": 0.72, "threshold": -12.798604011535645, "operator": "ge"}, {"budget": 0.73, "threshold": -12.880267143249512, "operator": "ge"}, {"budget": 0.74, "threshold": -12.995708465576172, "operator": "ge"}, {"budget": 0.75, "threshold": -13.077566146850586, "operator": "ge"}, {"budget": 0.76, "threshold": -13.164397239685059, "operator": "ge"}, {"budget": 0.77, "threshold": -13.197675704956055, "operator": "ge"}, {"budget": 0.78, "threshold": -13.245747566223145, "operator": "ge"}, {"budget": 0.79, "threshold": -13.311421394348145, "operator": "ge"}, {"budget": 0.8, "threshold": -13.411325454711914, "operator": "ge"}, {"budget": 0.81, "threshold": -13.509528160095215, "operator": "ge"}, {"budget": 0.82, "threshold": -13.604700088500977, "operator": "ge"}, {"budget": 0.83, "threshold": -13.675140380859375, "operator": "ge"}, {"budget": 0.84, "threshold": -13.739232063293457, "operator": "ge"}, {"budget": 0.85, "threshold": -13.801328659057617, "operator": "ge"}, {"budget": 0.86, "threshold": -13.901971817016602, "operator": "ge"}, {"budget": 0.87, "threshold": -13.987235069274902, "operator": "ge"}, {"budget": 0.88, "threshold": -14.091081619262695, "operator": "ge"}, {"budget": 0.89, "threshold": -14.232654571533203, "operator": "ge"}, {"budget": 0.9, "threshold": -14.361896514892578, "operator": "ge"}, {"budget": 0.91, "threshold": -14.474992752075195, "operator": "ge"}, {"budget": 0.92, "threshold": -14.596799850463867, "operator": "ge"}, {"budget": 0.93, "threshold": -14.671646118164062, "operator": "ge"}, {"budget": 0.94, "threshold": -14.744136810302734, "operator": "ge"}, {"budget": 0.95, "threshold": -14.842597007751465, "operator": "ge"}, {"budget": 0.96, "threshold": -14.955503463745117, "operator": "ge"}, {"budget": 0.97, "threshold": -15.17534065246582, "operator": "ge"}, {"budget": 0.98, "threshold": -15.366878509521484, "operator": "ge"}, {"budget": 0.99, "threshold": -15.984682083129883, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.16666666666667, 97.66666666666667, 97.08333333333333, 96.58333333333333, 95.83333333333334, 95.41666666666667, 94.75, 94.0, 93.5, 93.0, 92.41666666666667, 91.75, 91.16666666666666, 90.75, 90.0, 89.5, 88.91666666666667, 88.58333333333334, 87.91666666666667, 87.41666666666667, 86.75, 86.33333333333333, 85.75, 85.08333333333333, 84.66666666666667, 84.25, 83.58333333333333, 82.91666666666666, 82.33333333333334, 81.91666666666667, 81.16666666666667, 80.83333333333333, 80.33333333333333, 79.83333333333333, 79.25, 78.83333333333333, 78.25, 77.75, 77.08333333333334, 76.58333333333334, 76.0, 75.41666666666667, 74.83333333333333, 74.5, 74.0, 73.58333333333333, 72.75, 72.16666666666667, 71.41666666666666, 70.91666666666667, 70.08333333333334, 69.33333333333334, 68.75, 68.0, 67.16666666666666, 66.66666666666667, 66.0, 65.16666666666666, 64.25, 63.5, 62.83333333333334, 62.16666666666666, 61.5, 60.66666666666667, 59.916666666666664, 59.166666666666664, 58.33333333333333, 57.58333333333334, 56.75, 55.666666666666664, 54.75, 54.0, 53.08333333333333, 52.25, 51.33333333333334, 50.416666666666664, 49.5, 48.583333333333336, 47.583333333333336, 46.666666666666664, 45.833333333333336, 45.25, 44.25, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.583333333333336, 39.583333333333336, 38.583333333333336, 37.583333333333336, 36.583333333333336, 35.583333333333336, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 7, 14, 21, 27, 36, 42, 50, 59, 65, 71, 79, 87, 95, 100, 109, 115, 122, 126, 134, 140, 148, 153, 160, 168, 173, 178, 186, 194, 201, 206, 215, 219, 225, 231, 238, 243, 250, 256, 264, 270, 277, 284, 291, 295, 301, 306, 316, 323, 332, 338, 348, 357, 364, 373, 383, 389, 397, 407, 418, 427, 435, 443, 451, 461, 470, 479, 489, 498, 508, 521, 532, 541, 552, 562, 573, 584, 595, 606, 618, 629, 639, 646, 658, 669, 681, 692, 702, 714, 726, 738, 750, 762, 774, 785, 797, 809, 821, 832, 844, 856], "exact_score": [98.58333333333333, 98.16666666666667, 97.66666666666667, 97.08333333333333, 96.58333333333333, 95.83333333333334, 95.41666666666667, 94.75, 94.0, 93.5, 93.0, 92.41666666666667, 91.75, 91.16666666666666, 90.75, 90.0, 89.5, 88.91666666666667, 88.58333333333334, 87.91666666666667, 87.41666666666667, 86.75, 86.33333333333333, 85.75, 85.08333333333333, 84.66666666666667, 84.25, 83.58333333333333, 82.91666666666666, 82.33333333333334, 81.91666666666667, 81.16666666666667, 80.83333333333333, 80.33333333333333, 79.83333333333333, 79.25, 78.83333333333333, 78.25, 77.75, 77.08333333333334, 76.58333333333334, 75.91666666666667, 75.41666666666667, 74.83333333333333, 74.5, 74.0, 73.58333333333333, 72.75, 72.16666666666667, 71.41666666666666, 70.91666666666667, 70.08333333333334, 69.33333333333334, 68.75, 68.0, 67.16666666666666, 66.66666666666667, 65.91666666666667, 65.16666666666666, 64.25, 63.5, 62.83333333333334, 62.16666666666666, 61.5, 60.66666666666667, 59.916666666666664, 59.166666666666664, 58.33333333333333, 57.58333333333334, 56.666666666666664, 55.666666666666664, 54.75, 54.0, 53.08333333333333, 52.25, 51.33333333333334, 50.416666666666664, 49.5, 48.583333333333336, 47.583333333333336, 46.666666666666664, 45.833333333333336, 45.166666666666664, 44.25, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.583333333333336, 39.583333333333336, 38.583333333333336, 37.583333333333336, 36.583333333333336, 35.583333333333336, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 67.70666666666666, "threshold_area100": 67.71, "threshold_area25": 91.575}, "test": {"score": [81.429064549534, 80.68691750086296, 80.18639972385226, 78.06351397997929, 76.64825681739731, 75.0604073179151, 73.83500172592336, 72.98929927511219, 71.67759751467035, 70.45219192267864, 68.7090093199862, 68.03589920607524, 66.3617535381429, 65.60234725578185, 64.4977562996203, 63.51397997928891, 62.823610631687956, 62.23679668622714, 61.615464273386266, 60.994131860545394, 60.59716948567484, 59.7859855022437, 59.40628236106317, 59.04383845357266, 58.83672764929238, 58.45702450811184, 58.00828443217121, 57.680358992060754, 57.14532274767, 56.88643424231964, 56.73110113910942, 56.28236106316879, 55.816361753538146, 55.367621677597526, 55.16051087331723, 54.97065930272696, 54.83258543320677, 54.29754918881602, 54.03866068346566, 53.36555056955472, 52.951328960994125, 52.74421815671384, 52.519848118743525, 52.226441146013116, 51.98481187435278, 51.622367966862285, 51.41525716258199, 51.242664825681736, 50.72488781498101, 50.55229547808078, 49.94822229892993, 49.326889886089056, 48.895409043838455, 48.44666896789783, 47.894373489817056, 47.618225750776666, 47.342078011736284, 46.97963410424577, 46.54815326199517, 46.341042457714885, 45.90956161546427, 45.32274767000345, 44.9257852951329, 44.45978598550225, 44.18363824646186, 43.8557128063514, 43.372454263030725, 42.7856403175699, 42.12978943734898, 41.750086296168455, 41.31860545391785, 40.680013807386956, 40.43838453572662, 40.02416292716603, 39.23023817742492, 38.79875733517432, 38.142906454953405, 38.004832585433206, 37.79772178115292, 37.29720400414222, 36.7103900586814, 36.037279944770454, 35.27787366240939, 34.79461511908871, 34.466689678978256, 34.121505005177774, 33.569209527097, 33.10321021746635, 32.58543320676562, 31.79150845702451, 31.152916810493615, 30.617880566102862, 30.04832585433207, 29.823955816361757, 29.513289609941317, 29.20262340352089, 28.909216430790472, 28.270624784259578, 27.787366240938905, 26.83810838798758, 26.561960648947192], "calls": [0.0, 4.953400069036935, 7.4387297204004135, 13.945460821539523, 17.82878840179496, 21.867449085260617, 25.81981360027615, 28.02899551259924, 30.876769071453225, 33.24128408698654, 36.744908526061444, 38.31549879185364, 41.42216085605799, 42.99275112185019, 45.098377632033134, 46.77252329996548, 47.94615119088713, 48.9299275112185, 50.01725923369003, 51.03555402140145, 51.6396272005523, 52.96858819468415, 53.641698308595096, 54.6427338626165, 55.10873317224715, 55.97169485674836, 56.955471177079744, 57.628581290990674, 58.76769071453227, 59.38902312737314, 59.855022437003804, 60.75250258888505, 61.59820503969624, 62.51294442526751, 63.03072143596824, 63.53123921297894, 63.962720055229546, 64.9292371418709, 65.25716258198136, 66.46530894028305, 67.2074559889541, 67.6389368312047, 68.0704176734553, 68.69175008629617, 69.24404556437695, 69.98619261304798, 70.26234035208837, 70.62478425957887, 71.47048671039006, 71.79841215050051, 72.62685536762167, 73.48981705212289, 74.14566793234381, 74.87055574732481, 75.81981360027615, 76.2167759751467, 76.61373835001726, 77.11425612702796, 77.73558853986883, 78.02899551259924, 78.65032792544011, 79.37521574042113, 79.82395581636176, 80.4970659302727, 80.7904729030031, 81.1356575768036, 81.67069382119433, 82.37832240248532, 83.2930617880566, 83.69002416292717, 84.19054193993787, 84.9326889886089, 85.24335519502934, 85.77839143942009, 86.64135312392129, 87.0728339661719, 87.78046254746289, 87.91853641698309, 88.12564722126338, 88.69520193303417, 89.31653434587504, 90.02416292716603, 90.81808767690714, 91.33586468760787, 91.69830859509838, 92.06075250258888, 92.63030721435969, 93.11356575768036, 93.64860200207112, 94.44252675181221, 95.15015533310321, 95.70245081118398, 96.32378322402485, 96.5654124956852, 96.87607870210563, 97.18674490852605, 97.5664480497066, 98.23955816361753, 98.7228167069382, 99.72385226095962, 100.0], "rescued": [0, 43, 61, 94, 105, 117, 143, 156, 168, 178, 193, 198, 205, 210, 212, 216, 217, 218, 218, 218, 218, 219, 220, 222, 222, 222, 223, 223, 224, 224, 224, 224, 224, 224, 224, 225, 225, 225, 225, 225, 225, 225, 225, 225, 226, 226, 226, 226, 226, 226, 227, 227, 227, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 229, 230, 230, 231, 231, 231, 231, 232, 232, 232, 232, 232], "harmed": [0, 86, 133, 289, 382, 486, 583, 645, 733, 814, 930, 974, 1078, 1127, 1193, 1254, 1295, 1330, 1366, 1402, 1425, 1473, 1496, 1519, 1531, 1553, 1580, 1599, 1631, 1646, 1655, 1681, 1708, 1734, 1746, 1758, 1766, 1797, 1812, 1851, 1875, 1887, 1900, 1917, 1932, 1953, 1965, 1975, 2005, 2015, 2051, 2087, 2112, 2139, 2171, 2187, 2203, 2224, 2249, 2261, 2286, 2320, 2343, 2370, 2386, 2405, 2433, 2467, 2506, 2528, 2553, 2590, 2604, 2628, 2674, 2699, 2737, 2745, 2757, 2786, 2820, 2859, 2903, 2931, 2950, 2970, 3002, 3029, 3059, 3105, 3143, 3174, 3208, 3221, 3239, 3257, 3275, 3312, 3340, 3395, 3411], "exact_score": [81.429064549534, 81.48084225060408, 81.2909906800138, 81.08387987573352, 80.87676907145323, 80.68691750086296, 80.47980669658268, 80.35899206075251, 80.01380738695202, 79.73765964791163, 79.40973420780118, 79.01277183293061, 78.75388332758025, 78.33966171901969, 78.04625474628926, 77.63203313772868, 77.28684846392821, 76.80358992060754, 76.57921988263722, 76.26855367621678, 75.81981360027615, 75.50914739385571, 75.04314808422507, 74.66344494304454, 74.31826026924405, 74.12840869865379, 73.76596479116327, 73.47255781843286, 73.00655850880221, 72.62685536762167, 72.143596824301, 71.59130134622022, 71.17707973765965, 70.60752502588885, 70.17604418363825, 69.52019330341733, 69.10597169485675, 68.60545391784605, 68.19123230928547, 67.67345529858476, 67.22471522264412, 66.58612357611322, 66.12012426648258, 65.60234725578185, 65.03279254401104, 64.48049706593028, 64.01449775629962, 63.35864687607871, 62.771832930617876, 62.16775975146703, 61.63272350707629, 61.02865032792544, 60.39005868139456, 59.7859855022437, 59.33724542630308, 58.8539868829824, 58.439765274421816, 57.99102519848118, 57.594062823610635, 57.05902657921988, 56.679323438039354, 56.12702795995859, 55.64376941663791, 55.16051087331723, 54.81532619951675, 54.24577148774594, 53.624439074905084, 53.05488436313428, 52.53710735243355, 52.157404211253024, 51.622367966862285, 51.0010355540214, 50.4314808422506, 49.689333793579564, 48.9989644459786, 48.3603727994477, 47.79081808767691, 47.10044874007595, 46.358301691404904, 45.63341387642389, 44.804970659302725, 43.97652744218157, 43.096306523990336, 42.30238177424923, 41.50845702450812, 40.62823610631688, 39.851570590265794, 38.88505350362444, 37.93579565067311, 37.021056265101834, 36.0717984121505, 35.1397997928892, 34.190541939937866, 33.22402485329651, 32.24024853296514, 31.29099068001381, 30.341732827062483, 29.39247497411115, 28.512254055919918, 27.52847773558854, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 59.97288507066158, "threshold_area100": 49.77071108042803, "threshold_area25": 67.5647221263376}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.5030670166015625, "operator": "ge"}, {"budget": 0.02, "threshold": -0.866295337677002, "operator": "ge"}, {"budget": 0.03, "threshold": -1.473695158958435, "operator": "ge"}, {"budget": 0.04, "threshold": -2.125162124633789, "operator": "ge"}, {"budget": 0.05, "threshold": -2.4762558937072754, "operator": "ge"}, {"budget": 0.06, "threshold": -3.1187286376953125, "operator": "ge"}, {"budget": 0.07, "threshold": -3.631312847137451, "operator": "ge"}, {"budget": 0.08, "threshold": -3.9097580909729004, "operator": "ge"}, {"budget": 0.09, "threshold": -4.579003810882568, "operator": "ge"}, {"budget": 0.1, "threshold": -4.82643461227417, "operator": "ge"}, {"budget": 0.11, "threshold": -5.130178451538086, "operator": "ge"}, {"budget": 0.12, "threshold": -5.463098526000977, "operator": "ge"}, {"budget": 0.13, "threshold": -5.685878753662109, "operator": "ge"}, {"budget": 0.14, "threshold": -5.94412899017334, "operator": "ge"}, {"budget": 0.15, "threshold": -6.144252300262451, "operator": "ge"}, {"budget": 0.16, "threshold": -6.282859802246094, "operator": "ge"}, {"budget": 0.17, "threshold": -6.5142130851745605, "operator": "ge"}, {"budget": 0.18, "threshold": -6.785749435424805, "operator": "ge"}, {"budget": 0.19, "threshold": -6.933772087097168, "operator": "ge"}, {"budget": 0.2, "threshold": -7.152909755706787, "operator": "ge"}, {"budget": 0.21, "threshold": -7.291565895080566, "operator": "ge"}, {"budget": 0.22, "threshold": -7.449728965759277, "operator": "ge"}, {"budget": 0.23, "threshold": -7.551186561584473, "operator": "ge"}, {"budget": 0.24, "threshold": -7.794648170471191, "operator": "ge"}, {"budget": 0.25, "threshold": -7.865233421325684, "operator": "ge"}, {"budget": 0.26, "threshold": -8.0368013381958, "operator": "ge"}, {"budget": 0.27, "threshold": -8.155923843383789, "operator": "ge"}, {"budget": 0.28, "threshold": -8.314676284790039, "operator": "ge"}, {"budget": 0.29, "threshold": -8.37991714477539, "operator": "ge"}, {"budget": 0.3, "threshold": -8.468722343444824, "operator": "ge"}, {"budget": 0.31, "threshold": -8.55954360961914, "operator": "ge"}, {"budget": 0.32, "threshold": -8.645238876342773, "operator": "ge"}, {"budget": 0.33, "threshold": -8.767606735229492, "operator": "ge"}, {"budget": 0.34, "threshold": -8.859392166137695, "operator": "ge"}, {"budget": 0.35, "threshold": -8.971811294555664, "operator": "ge"}, {"budget": 0.36, "threshold": -9.081278800964355, "operator": "ge"}, {"budget": 0.37, "threshold": -9.140656471252441, "operator": "ge"}, {"budget": 0.38, "threshold": -9.239570617675781, "operator": "ge"}, {"budget": 0.39, "threshold": -9.303388595581055, "operator": "ge"}, {"budget": 0.4, "threshold": -9.408723831176758, "operator": "ge"}, {"budget": 0.41, "threshold": -9.460538864135742, "operator": "ge"}, {"budget": 0.42, "threshold": -9.547945022583008, "operator": "ge"}, {"budget": 0.43, "threshold": -9.617995262145996, "operator": "ge"}, {"budget": 0.44, "threshold": -9.757527351379395, "operator": "ge"}, {"budget": 0.45, "threshold": -9.81039047241211, "operator": "ge"}, {"budget": 0.46, "threshold": -9.862960815429688, "operator": "ge"}, {"budget": 0.47, "threshold": -9.960746765136719, "operator": "ge"}, {"budget": 0.48, "threshold": -10.001860618591309, "operator": "ge"}, {"budget": 0.49, "threshold": -10.096572875976562, "operator": "ge"}, {"budget": 0.5, "threshold": -10.192108154296875, "operator": "ge"}, {"budget": 0.51, "threshold": -10.25231647491455, "operator": "ge"}, {"budget": 0.52, "threshold": -10.35395336151123, "operator": "ge"}, {"budget": 0.53, "threshold": -10.432418823242188, "operator": "ge"}, {"budget": 0.54, "threshold": -10.480474472045898, "operator": "ge"}, {"budget": 0.55, "threshold": -10.55540943145752, "operator": "ge"}, {"budget": 0.56, "threshold": -10.637195587158203, "operator": "ge"}, {"budget": 0.57, "threshold": -10.695018768310547, "operator": "ge"}, {"budget": 0.58, "threshold": -10.832136154174805, "operator": "ge"}, {"budget": 0.59, "threshold": -10.89107894897461, "operator": "ge"}, {"budget": 0.6, "threshold": -10.966773986816406, "operator": "ge"}, {"budget": 0.61, "threshold": -11.015024185180664, "operator": "ge"}, {"budget": 0.62, "threshold": -11.140975952148438, "operator": "ge"}, {"budget": 0.63, "threshold": -11.233526229858398, "operator": "ge"}, {"budget": 0.64, "threshold": -11.311803817749023, "operator": "ge"}, {"budget": 0.65, "threshold": -11.39772891998291, "operator": "ge"}, {"budget": 0.66, "threshold": -11.488396644592285, "operator": "ge"}, {"budget": 0.67, "threshold": -11.567441940307617, "operator": "ge"}, {"budget": 0.68, "threshold": -11.646181106567383, "operator": "ge"}, {"budget": 0.69, "threshold": -11.735523223876953, "operator": "ge"}, {"budget": 0.7, "threshold": -11.807767868041992, "operator": "ge"}, {"budget": 0.71, "threshold": -11.904312133789062, "operator": "ge"}, {"budget": 0.72, "threshold": -11.993539810180664, "operator": "ge"}, {"budget": 0.73, "threshold": -12.05343246459961, "operator": "ge"}, {"budget": 0.74, "threshold": -12.116811752319336, "operator": "ge"}, {"budget": 0.75, "threshold": -12.192956924438477, "operator": "ge"}, {"budget": 0.76, "threshold": -12.282928466796875, "operator": "ge"}, {"budget": 0.77, "threshold": -12.316728591918945, "operator": "ge"}, {"budget": 0.78, "threshold": -12.444620132446289, "operator": "ge"}, {"budget": 0.79, "threshold": -12.515989303588867, "operator": "ge"}, {"budget": 0.8, "threshold": -12.623359680175781, "operator": "ge"}, {"budget": 0.81, "threshold": -12.688867568969727, "operator": "ge"}, {"budget": 0.82, "threshold": -12.743379592895508, "operator": "ge"}, {"budget": 0.83, "threshold": -12.802005767822266, "operator": "ge"}, {"budget": 0.84, "threshold": -12.865030288696289, "operator": "ge"}, {"budget": 0.85, "threshold": -12.937553405761719, "operator": "ge"}, {"budget": 0.86, "threshold": -13.044262886047363, "operator": "ge"}, {"budget": 0.87, "threshold": -13.119827270507812, "operator": "ge"}, {"budget": 0.88, "threshold": -13.218955039978027, "operator": "ge"}, {"budget": 0.89, "threshold": -13.281183242797852, "operator": "ge"}, {"budget": 0.9, "threshold": -13.3582763671875, "operator": "ge"}, {"budget": 0.91, "threshold": -13.450209617614746, "operator": "ge"}, {"budget": 0.92, "threshold": -13.526220321655273, "operator": "ge"}, {"budget": 0.93, "threshold": -13.685338020324707, "operator": "ge"}, {"budget": 0.94, "threshold": -13.831117630004883, "operator": "ge"}, {"budget": 0.95, "threshold": -13.966889381408691, "operator": "ge"}, {"budget": 0.96, "threshold": -14.122100830078125, "operator": "ge"}, {"budget": 0.97, "threshold": -14.226219177246094, "operator": "ge"}, {"budget": 0.98, "threshold": -14.519306182861328, "operator": "ge"}, {"budget": 0.99, "threshold": -14.952789306640625, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.16666666666667, 97.75, 97.25, 96.58333333333333, 95.91666666666667, 95.33333333333334, 94.75, 94.16666666666667, 93.58333333333333, 92.91666666666667, 92.41666666666667, 91.75, 91.08333333333334, 90.75, 90.25, 89.41666666666667, 89.0, 88.5, 88.16666666666667, 87.66666666666667, 87.25, 87.08333333333333, 86.75, 86.16666666666667, 85.83333333333334, 85.33333333333333, 84.91666666666666, 84.41666666666667, 83.83333333333334, 83.0, 82.58333333333333, 82.08333333333333, 81.5, 80.91666666666667, 80.33333333333333, 79.58333333333334, 79.08333333333333, 78.58333333333334, 78.08333333333334, 77.75, 77.25, 76.5, 76.0, 75.25, 74.41666666666666, 73.58333333333333, 72.83333333333333, 72.33333333333334, 71.83333333333333, 71.25, 70.5, 69.75, 69.08333333333334, 68.33333333333333, 67.41666666666667, 66.58333333333333, 65.83333333333333, 65.16666666666666, 64.58333333333333, 63.583333333333336, 62.74999999999999, 61.916666666666664, 61.16666666666667, 60.5, 59.833333333333336, 58.916666666666664, 58.33333333333333, 57.41666666666667, 56.75, 55.75, 54.75, 53.833333333333336, 53.08333333333333, 52.25, 51.33333333333334, 50.66666666666666, 49.666666666666664, 48.833333333333336, 48.0, 47.0, 46.0, 45.08333333333333, 44.00000000000001, 43.00000000000001, 42.08333333333334, 41.083333333333336, 40.25, 39.24999999999999, 38.416666666666664, 37.5, 36.5, 35.5, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 6, 12, 19, 27, 35, 42, 50, 57, 64, 72, 78, 87, 95, 99, 106, 116, 121, 127, 131, 137, 142, 144, 148, 155, 159, 165, 170, 176, 183, 193, 198, 204, 211, 218, 225, 234, 240, 246, 252, 256, 262, 271, 277, 286, 296, 306, 315, 321, 327, 334, 343, 352, 360, 369, 380, 390, 399, 407, 414, 426, 436, 446, 455, 463, 471, 482, 489, 500, 508, 520, 532, 543, 552, 562, 573, 581, 593, 603, 613, 625, 637, 648, 661, 673, 684, 696, 706, 718, 728, 739, 751, 763, 774, 785, 797, 808, 820, 832, 844, 856], "exact_score": [98.58333333333333, 98.16666666666667, 97.75, 97.25, 96.58333333333333, 95.91666666666667, 95.33333333333334, 94.75, 94.16666666666667, 93.58333333333333, 92.91666666666667, 92.41666666666667, 91.75, 91.08333333333334, 90.75, 90.25, 89.41666666666667, 89.0, 88.5, 88.16666666666667, 87.66666666666667, 87.25, 87.08333333333333, 86.75, 86.16666666666667, 85.83333333333334, 85.33333333333333, 84.91666666666666, 84.41666666666667, 83.83333333333334, 83.0, 82.58333333333333, 82.08333333333333, 81.5, 80.91666666666667, 80.33333333333333, 79.58333333333334, 79.08333333333333, 78.58333333333334, 78.08333333333334, 77.75, 77.25, 76.5, 76.0, 75.25, 74.41666666666666, 73.58333333333333, 72.83333333333333, 72.33333333333334, 71.83333333333333, 71.25, 70.5, 69.75, 69.08333333333334, 68.33333333333333, 67.41666666666667, 66.58333333333333, 65.83333333333333, 65.16666666666666, 64.58333333333333, 63.583333333333336, 62.74999999999999, 61.916666666666664, 61.16666666666667, 60.5, 59.833333333333336, 58.916666666666664, 58.33333333333333, 57.41666666666667, 56.666666666666664, 55.75, 54.75, 53.833333333333336, 53.08333333333333, 52.25, 51.33333333333334, 50.66666666666666, 49.666666666666664, 48.833333333333336, 48.0, 47.0, 46.0, 44.99999999999999, 44.00000000000001, 43.00000000000001, 42.08333333333334, 41.083333333333336, 40.25, 39.24999999999999, 38.416666666666664, 37.5, 36.5, 35.5, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 67.98916666666668, "threshold_area100": 67.99083333333333, "threshold_area25": 91.79166666666667}, "test": {"score": [81.429064549534, 80.15188125647221, 79.27166033828098, 77.66655160510874, 75.87159130134621, 75.09492578529513, 73.11011391094236, 71.43596824301002, 70.9181912323093, 68.53641698308596, 67.7079737659648, 66.82775284777355, 65.55056955471177, 65.03279254401104, 63.94546082153953, 63.44494304452882, 63.08249913703832, 62.46116672419745, 61.71901967552641, 61.25302036589575, 60.59716948567484, 59.95857783914394, 59.527096996893334, 59.31998619261305, 58.664135312392126, 58.50880220918191, 58.077321366931315, 57.73213669313083, 57.16258198136003, 56.955471177079744, 56.541249568519156, 56.178805661028655, 55.93717638936832, 55.57473248187781, 55.1432516396272, 54.79806696582672, 54.29754918881602, 53.98688298239558, 53.451846738004825, 53.15843976527442, 52.83051432516397, 52.60614428719366, 52.27821884708319, 52.019330341732825, 51.31170176044184, 51.10459095616156, 50.89748015188126, 50.44874007594062, 50.27614773904039, 49.86192613047981, 49.395926820849155, 49.15429754918882, 48.74007594062824, 48.34311356575768, 48.03244735933725, 47.73904038660683, 47.497411114946495, 47.15222644114602, 46.47911632723507, 46.08215395236452, 45.68519157749396, 45.4263030721436, 44.770452191922686, 44.166379012771834, 43.527787366240936, 43.01001035554022, 42.61304798066966, 42.371418709009326, 41.819123230928554, 41.266827752847774, 40.680013807386956, 39.90334829133587, 39.26475664480497, 38.81601656886434, 38.38453572661374, 38.02209181912323, 37.452537107352434, 37.24542630307214, 36.7103900586814, 36.313427683810836, 35.70935450466, 35.24335519502934, 34.94994822229893, 34.483948912668275, 34.035208836727655, 33.58646876078703, 32.844321712115985, 32.37832240248533, 31.912323092854677, 31.549879185364173, 31.066620642043496, 30.63513979979289, 30.376251294442525, 29.66862271315154, 29.099068001380747, 28.52951328960994, 28.218847083189512, 27.87366240938902, 27.26958923023818, 26.786330686917502, 26.561960648947192], "calls": [0.0, 6.368657231618916, 9.147393855712806, 14.22160856057991, 19.295823265447016, 21.867449085260617, 26.803589920607525, 30.531584397652743, 32.430100103555404, 37.090093199861926, 38.48809112875388, 40.2140144977563, 42.49223334483949, 43.71763893683121, 45.46082153952364, 46.634449430445294, 47.53192958232654, 48.774594408008284, 50.172592336900244, 51.104590956161545, 52.55436658612358, 53.58992060752502, 54.45288229202624, 55.10873317224715, 56.541249568519156, 57.04176734552986, 58.04280289955126, 58.73317224715222, 60.13117017604418, 60.648947186744905, 61.373835001725915, 62.02968588194684, 62.4956851915775, 63.375906109768735, 64.11805315843976, 64.82568173973075, 65.74042112530203, 66.2064204349327, 67.08664135312392, 67.48360372799448, 68.08767690714532, 68.51915774939593, 68.98515705902658, 69.48567483603728, 70.72833966171902, 71.12530203658957, 71.50500517777012, 72.143596824301, 72.38522609596134, 72.98929927511219, 73.61063168795306, 74.00759406282361, 74.66344494304454, 75.2157404211253, 75.59544356230583, 76.00966517086641, 76.45840524680703, 76.90714532274767, 77.83914394200897, 78.3569209527097, 78.87469796341043, 79.20262340352089, 79.996548153262, 80.70417673455299, 81.429064549534, 82.03313772868485, 82.53365550569555, 82.89609941318605, 83.46565412495686, 84.0697273041077, 84.67380048325855, 85.51950293406972, 86.27890921643079, 86.76216775975146, 87.29720400414222, 87.72868484639282, 88.38453572661373, 88.60890576458405, 89.31653434587504, 89.78253365550569, 90.45564376941664, 90.92164307904729, 91.2150500517777, 91.68104936140836, 92.12978943734899, 92.61304798066966, 93.35519502934069, 93.83845357266138, 94.30445288229203, 94.7186744908526, 95.2191922678633, 95.68519157749395, 95.96133931653435, 96.66896789782534, 97.32481877804625, 97.9288919571971, 98.25681739730756, 98.6365205384881, 99.29237141870901, 99.77562996202968, 100.0], "rescued": [0, 38, 52, 76, 102, 117, 138, 157, 172, 190, 192, 198, 204, 208, 210, 211, 214, 217, 217, 219, 219, 219, 219, 219, 219, 219, 219, 219, 221, 222, 222, 222, 222, 224, 224, 225, 225, 225, 225, 225, 225, 225, 225, 225, 225, 225, 225, 226, 226, 226, 227, 227, 227, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 229, 229, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 112, 177, 294, 424, 484, 620, 736, 781, 937, 987, 1044, 1124, 1158, 1223, 1253, 1277, 1316, 1359, 1388, 1426, 1463, 1488, 1500, 1538, 1547, 1572, 1592, 1627, 1640, 1664, 1685, 1699, 1722, 1747, 1768, 1797, 1815, 1846, 1863, 1882, 1895, 1914, 1929, 1970, 1982, 1994, 2021, 2031, 2055, 2083, 2097, 2121, 2145, 2163, 2180, 2194, 2214, 2253, 2276, 2299, 2314, 2352, 2387, 2424, 2454, 2477, 2491, 2523, 2555, 2589, 2634, 2671, 2697, 2722, 2743, 2777, 2789, 2822, 2845, 2880, 2907, 2924, 2951, 2977, 3003, 3046, 3073, 3100, 3121, 3149, 3174, 3189, 3230, 3264, 3297, 3315, 3335, 3370, 3398, 3411], "exact_score": [81.429064549534, 81.30824991370383, 81.17017604418363, 80.94580600621333, 80.72143596824301, 80.44528822920263, 80.2899551259924, 79.97928891957199, 79.65136347946151, 79.3579565067311, 79.03003106662064, 78.63306869175008, 78.30514325163963, 77.95995857783915, 77.66655160510874, 77.49395926820849, 77.13151536071798, 76.78633068691751, 76.38936831204694, 75.94062823610632, 75.69899896444599, 75.35381429064549, 75.0776665516051, 74.59440800828443, 74.26648256817397, 73.9212978943735, 73.50707628581291, 73.00655850880221, 72.47152226441146, 72.09181912323092, 71.69485674836038, 71.22885743872972, 71.02174663444943, 70.60752502588885, 70.03797031411806, 69.48567483603728, 69.08871246116672, 68.570935450466, 67.98412150500518, 67.5008629616845, 66.96582671729375, 66.37901277183293, 65.84397652744218, 65.32619951674145, 64.80842250604073, 64.27338626164997, 63.72109078356921, 63.25509147393856, 62.90990680013808, 62.323092854677256, 61.80531584397653, 61.32205730065585, 60.890576458405256, 60.2865032792544, 59.7859855022437, 59.38902312737315, 58.940283051432516, 58.526061442871935, 58.094580600621335, 57.594062823610635, 57.2833966171902, 56.748360372799446, 56.19606489471868, 55.712806351397994, 55.229547808077314, 54.65999309630652, 54.142216085605796, 53.48636520538488, 52.89955125992406, 52.27821884708319, 51.70866413531239, 51.19088712461166, 50.55229547808078, 49.86192613047981, 49.15429754918882, 48.4984466689679, 47.75629962029686, 47.08318950638592, 46.37556092509493, 45.564376941663795, 44.770452191922686, 43.89023127373145, 43.04452882292026, 42.267863306869174, 41.335864687607874, 40.42112530203659, 39.489126682775286, 38.62616499827408, 37.780462547462896, 36.96927856403176, 36.1408353469106, 35.174318260269246, 34.173282706247846, 33.20676561960649, 32.22298929927511, 31.273731446323783, 30.341732827062483, 29.3579565067311, 28.477735588539865, 27.52847773558854, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 59.95278107201229, "threshold_area100": 49.385657576803595, "threshold_area25": 67.3627890921643}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.37728684846392, "calls": 1.3462202278218847}}, "old_normalized_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.37728684846392, "calls": 1.3462202278218847}}, "old_normalized_endpoints": {"val": {"score": 98.58333333333333, "calls": 0.0}, "test": {"score": 81.429064549534, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.33333333333334, 98.08333333333333, 97.91666666666666, 97.75, 97.58333333333333, 97.41666666666666, 97.33333333333334, 97.08333333333333, 96.91666666666666, 96.91666666666666, 96.66666666666667, 96.25, 96.08333333333333, 95.5, 95.08333333333333, 94.75, 94.58333333333333, 94.16666666666667, 93.58333333333333, 92.91666666666667, 92.33333333333333, 91.5, 90.83333333333333, 90.0, 89.33333333333333, 88.33333333333333, 87.83333333333333, 86.91666666666666, 86.5, 85.58333333333333, 84.75, 84.08333333333333, 83.16666666666667, 82.33333333333334, 81.41666666666667, 80.5, 79.58333333333334, 78.75, 77.83333333333333, 77.0, 76.08333333333334, 75.33333333333333, 74.41666666666666, 73.41666666666666, 72.66666666666667, 71.66666666666667, 70.66666666666667, 69.83333333333334, 68.91666666666667, 67.91666666666667, 66.91666666666667, 66.0, 65.08333333333334, 64.16666666666666, 63.25000000000001, 62.416666666666664, 61.5, 60.583333333333336, 59.583333333333336, 58.666666666666664, 57.833333333333336, 56.75, 55.833333333333336, 54.91666666666667, 54.0, 53.0, 52.0, 51.083333333333336, 50.083333333333336, 49.16666666666667, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.166666666666664, 42.25, 41.333333333333336, 40.33333333333333, 39.33333333333333, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.33333333333334, 98.08333333333333, 97.91666666666666, 97.75, 97.58333333333333, 97.41666666666666, 97.33333333333334, 97.08333333333333, 96.91666666666666, 96.91666666666666, 96.66666666666667, 96.25, 96.08333333333333, 95.5, 95.08333333333333, 94.75, 94.58333333333333, 94.16666666666667, 93.58333333333333, 92.91666666666667, 92.33333333333333, 91.5, 90.83333333333333, 90.0, 89.33333333333333, 88.33333333333333, 87.83333333333333, 86.91666666666666, 86.5, 85.58333333333333, 84.75, 84.08333333333333, 83.08333333333333, 82.33333333333334, 81.41666666666667, 80.5, 79.58333333333334, 78.75, 77.83333333333333, 77.0, 76.08333333333334, 75.33333333333333, 74.41666666666666, 73.41666666666666, 72.66666666666667, 71.66666666666667, 70.66666666666667, 69.83333333333334, 68.83333333333333, 67.91666666666667, 66.91666666666667, 66.0, 65.08333333333334, 64.16666666666666, 63.25000000000001, 62.416666666666664, 61.5, 60.583333333333336, 59.583333333333336, 58.666666666666664, 57.75, 56.75, 55.833333333333336, 54.91666666666667, 54.0, 53.0, 52.0, 51.083333333333336, 50.083333333333336, 49.16666666666667, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.166666666666664, 42.25, 41.333333333333336, 40.33333333333333, 39.33333333333333, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.36666666666667, "exact_area100": 71.65416666666667}, "test": {"score": [81.429064549534, 81.37728684846392, 81.37728684846392, 81.37728684846392, 81.41180531584398, 81.429064549534, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.20469451156369, 81.20469451156369, 81.11839834311357, 81.0666206420435, 80.91128753883328, 80.65239903348291, 80.54884363134278, 80.2899551259924, 80.15188125647221, 79.7894373489817, 79.32343803935106, 78.89195719710045, 78.32240248532966, 77.78736624093891, 77.44218156713842, 76.66551605108734, 75.94062823610632, 75.19848118743528, 74.37003797031412, 73.90403866068347, 73.26544701415257, 72.81670693821194, 72.10907835692095, 71.17707973765965, 70.46945115636866, 69.72730410769762, 69.12323092854677, 67.94960303762512, 67.2074559889541, 66.37901277183293, 65.60234725578185, 64.61857093545046, 63.4967207455989, 62.90990680013808, 62.547462892647566, 62.01242664825681, 61.477390403866075, 60.73524335519503, 59.69968933379358, 58.8539868829824, 57.62858129099069, 56.57576803589921, 56.19606489471868, 55.45391784604764, 54.6427338626165, 54.03866068346566, 53.572661373835004, 51.846738004832595, 51.12185018985157, 50.742147048671036, 50.13807386952019, 49.395926820849155, 48.63652053848809, 47.9979288919572, 47.4283741801864, 46.703486365205386, 45.96133931653435, 45.44356230583362, 44.8567483603728, 43.8557128063514, 43.32067656196065, 42.57852951328962, 41.95719710044874, 41.612012426648256, 41.09423541594753, 40.50742147048671, 40.00690369347601, 39.36831204694512, 38.919571971004494, 38.45357266137383, 37.90127718329306, 36.29616845012082, 35.536762167759754, 34.86365205384881, 33.89713496720746, 33.258543320676566, 32.481877804625476, 31.77424922333449, 31.4981014842941, 31.083879875733523, 30.531584397652743, 29.96202968588195, 29.651363479461512, 29.150845702450813, 28.322402485329658, 27.75284777355885, 27.18329306178806, 26.561960648947192], "calls": [0.0, 1.3462202278218847, 1.6568864342423197, 2.554366586123576, 2.98584742837418, 3.762512944425268, 4.815326199516742, 5.67828788401795, 7.059026579219882, 8.267172937521575, 9.49257852951329, 10.165688643424232, 11.73627890921643, 13.462202278218847, 14.497756299620296, 15.343458750431479, 16.189161201242666, 17.32827062478426, 19.019675526406626, 19.761822575077666, 20.987228167069382, 21.660338280980323, 23.041076976182257, 24.439074905074214, 26.182257507766654, 27.994477045219192, 29.49603037625129, 30.600621332412842, 32.430100103555404, 33.914394200897476, 35.70935450466, 37.090093199861926, 38.280980324473596, 39.26475664480497, 40.30031066620642, 41.80186399723853, 43.49326889886089, 44.804970659302725, 45.89230238177425, 46.996893337935795, 48.94718674490853, 50.08629616845012, 51.225405591991716, 52.41629271660339, 54.228512254055914, 55.83362098722817, 56.80013807386952, 57.35243355195029, 58.23265447014152, 59.04383845357266, 60.217466344494305, 61.615464273386266, 62.75457369692785, 64.44597859855023, 65.75768035899206, 66.32723507076285, 67.2074559889541, 68.4328615809458, 69.10597169485675, 69.67552640662754, 71.6430790472903, 72.50604073179151, 72.93752157404211, 73.71418709009319, 74.5771487745944, 75.47462892647566, 76.28581290990681, 76.95892302381773, 77.7183293061788, 78.63306869175008, 79.2198826372109, 79.996548153262, 81.18743527787366, 81.73973075595443, 82.65447014152572, 83.44839489126683, 83.84535726613738, 84.38039351052812, 85.001725923369, 85.58853986882983, 86.27890921643079, 86.77942699344149, 87.26268553676216, 87.83224024853297, 89.62720055229548, 90.49016223679669, 91.18053158439766, 92.23334483948913, 92.94097342078011, 93.7866758715913, 94.54608215395236, 94.82222989299275, 95.30548843631342, 95.87504314808423, 96.4963755609251, 96.82430100103555, 97.34207801173628, 98.20503969623748, 98.77459440800828, 99.3614083534691, 100.0], "exact_score": [81.429064549534, 81.37728684846392, 81.37728684846392, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.2909906800138, 81.30824991370383, 81.23921297894374, 81.22195374525371, 81.17017604418363, 81.11839834311357, 80.96306523990336, 80.87676907145323, 80.63513979979288, 80.47980669658268, 80.2899551259924, 80.08284432171212, 79.7894373489817, 79.47877114256127, 79.23714187090093, 78.9955125992406, 78.61580945806006, 78.32240248532966, 78.02899551259924, 77.68381083879876, 77.28684846392821, 76.78633068691751, 76.423886779427, 75.87159130134621, 75.49188816016569, 75.00862961684501, 74.43907490507422, 74.02485329651364, 73.45529858474285, 72.88574387297204, 72.47152226441146, 71.98826372109079, 71.43596824301002, 70.93545046599931, 70.38315498791854, 69.62374870555747, 69.12323092854677, 68.53641698308596, 67.89782533655506, 67.25923369002416, 66.55160510873317, 65.87849499482223, 65.25716258198136, 64.7911632723507, 64.04901622367967, 63.35864687607871, 62.823610631687956, 62.13324128408699, 61.52916810493614, 60.90783569209527, 60.14842940973422, 59.423541594753196, 58.715913013462206, 57.95650673110114, 57.21435968243011, 56.42043493268899, 55.66102865032793, 55.005177770107004, 54.12495685191578, 53.279254401104595, 52.43355195029341, 51.570590265792205, 50.70762858129099, 49.87918536416983, 49.06800138073869, 48.23955816361754, 47.39385571280636, 46.47911632723507, 45.61615464273386, 44.8567483603728, 44.011045909561616, 43.096306523990336, 42.2851225405592, 41.49119779081809, 40.52468070417674, 39.66171901967553, 38.72972040041422, 37.74594408008284, 36.90024162927166, 35.950983776320335, 35.053503624439074, 34.1387642388678, 33.20676561960649, 32.274767000345186, 31.32550914739386, 30.410769761822575, 29.47877114256128, 28.512254055919918, 27.54573696927857, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 80.81463583016915, "exact_area100": 62.93401878962788}}, "old_normalized_endpoint_budget": 0}, {"seed": 2030, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.0, "validation_area100": 0.7170000000000001, "training_seconds": 3.356659989920445, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.004842729773372412, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0063249715603888035, "operator": "ge"}, {"budget": 0.03, "threshold": -0.007143797352910042, "operator": "ge"}, {"budget": 0.04, "threshold": -0.007921483367681503, "operator": "ge"}, {"budget": 0.05, "threshold": -0.009030326269567013, "operator": "ge"}, {"budget": 0.06, "threshold": -0.010685857385396957, "operator": "ge"}, {"budget": 0.07, "threshold": -0.015300276689231396, "operator": "ge"}, {"budget": 0.08, "threshold": -0.018539929762482643, "operator": "ge"}, {"budget": 0.09, "threshold": -0.021706044673919678, "operator": "ge"}, {"budget": 0.1, "threshold": -0.025133026763796806, "operator": "ge"}, {"budget": 0.11, "threshold": -0.029159247875213623, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03566305339336395, "operator": "ge"}, {"budget": 0.13, "threshold": -0.04552208259701729, "operator": "ge"}, {"budget": 0.14, "threshold": -0.057093583047389984, "operator": "ge"}, {"budget": 0.15, "threshold": -0.07223709672689438, "operator": "ge"}, {"budget": 0.16, "threshold": -0.09025540947914124, "operator": "ge"}, {"budget": 0.17, "threshold": -0.10614888370037079, "operator": "ge"}, {"budget": 0.18, "threshold": -0.11900870501995087, "operator": "ge"}, {"budget": 0.19, "threshold": -0.14532308280467987, "operator": "ge"}, {"budget": 0.2, "threshold": -0.1645021289587021, "operator": "ge"}, {"budget": 0.21, "threshold": -0.1965615600347519, "operator": "ge"}, {"budget": 0.22, "threshold": -0.22182922065258026, "operator": "ge"}, {"budget": 0.23, "threshold": -0.26034075021743774, "operator": "ge"}, {"budget": 0.24, "threshold": -0.3018578886985779, "operator": "ge"}, {"budget": 0.25, "threshold": -0.34069493412971497, "operator": "ge"}, {"budget": 0.26, "threshold": -0.3878975212574005, "operator": "ge"}, {"budget": 0.27, "threshold": -0.4436900019645691, "operator": "ge"}, {"budget": 0.28, "threshold": -0.47693032026290894, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5193653702735901, "operator": "ge"}, {"budget": 0.3, "threshold": -0.5532676577568054, "operator": "ge"}, {"budget": 0.31, "threshold": -0.5833015441894531, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6091939210891724, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6396224498748779, "operator": "ge"}, {"budget": 0.34, "threshold": -0.6672426462173462, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6815017461776733, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7100384831428528, "operator": "ge"}, {"budget": 0.37, "threshold": -0.7273280024528503, "operator": "ge"}, {"budget": 0.38, "threshold": -0.7472882270812988, "operator": "ge"}, {"budget": 0.39, "threshold": -0.7580146789550781, "operator": "ge"}, {"budget": 0.4, "threshold": -0.7743940353393555, "operator": "ge"}, {"budget": 0.41, "threshold": -0.7978382706642151, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8112537264823914, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8250505328178406, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8327509760856628, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8495388031005859, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8626042604446411, "operator": "ge"}, {"budget": 0.47, "threshold": -0.8735563158988953, "operator": "ge"}, {"budget": 0.48, "threshold": -0.8819729089736938, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8910600543022156, "operator": "ge"}, {"budget": 0.5, "threshold": -0.8982272148132324, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9042551517486572, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9119252562522888, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9170188903808594, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9225958585739136, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9320340156555176, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9355973601341248, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9391952753067017, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9426664113998413, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9458372592926025, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9484013319015503, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9509693384170532, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9539373517036438, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9565995931625366, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9599572420120239, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9631677865982056, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9661461114883423, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9688633680343628, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9716160893440247, "operator": "ge"}, {"budget": 0.69, "threshold": -0.97446608543396, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9761160016059875, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9783990383148193, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9798417687416077, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9818354845046997, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9828813076019287, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9847743511199951, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9860324263572693, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9870374202728271, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9880881905555725, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9889624118804932, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9899786114692688, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9905115962028503, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9911103844642639, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9916425347328186, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9923301935195923, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9930607080459595, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9935264587402344, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9939806461334229, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9943426251411438, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9949426651000977, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9954419732093811, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9958906769752502, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9966928958892822, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9968719482421875, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9972443580627441, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9977071285247803, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9979973435401917, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9983287453651428, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9984469413757324, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9987004995346069, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.08333333333333, 97.91666666666666, 97.66666666666667, 97.5, 97.25, 97.08333333333333, 97.0, 96.83333333333334, 96.75, 96.5, 96.25, 96.16666666666667, 95.75, 95.16666666666667, 94.5, 93.91666666666667, 93.25, 92.75, 92.25, 91.58333333333333, 91.0, 90.25, 89.66666666666667, 88.75, 87.83333333333333, 87.0, 86.33333333333333, 85.66666666666667, 84.91666666666666, 83.91666666666666, 83.25, 82.16666666666667, 81.41666666666667, 80.58333333333334, 79.66666666666666, 79.0, 78.16666666666667, 77.16666666666667, 76.16666666666667, 75.41666666666667, 74.41666666666666, 73.41666666666666, 72.50000000000001, 71.50000000000001, 70.5, 69.75, 68.91666666666667, 68.0, 67.0, 66.0, 65.25000000000001, 64.33333333333333, 63.416666666666664, 62.583333333333336, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.75, 57.91666666666666, 56.833333333333336, 55.833333333333336, 54.833333333333336, 53.833333333333336, 52.83333333333333, 52.0, 51.0, 50.0, 49.0, 48.0, 47.0, 46.08333333333333, 45.333333333333336, 44.25, 43.25, 42.333333333333336, 41.333333333333336, 40.33333333333333, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.58333333333333, 29.58333333333334, 28.666666666666664, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 7, 9, 12, 14, 17, 19, 21, 24, 25, 28, 31, 32, 37, 44, 52, 59, 68, 74, 80, 88, 95, 104, 111, 122, 133, 143, 151, 159, 168, 180, 188, 201, 210, 220, 231, 240, 250, 262, 274, 283, 295, 307, 318, 330, 342, 351, 361, 372, 384, 396, 406, 417, 428, 438, 450, 462, 473, 484, 494, 507, 519, 531, 543, 555, 565, 577, 589, 601, 613, 625, 636, 645, 658, 670, 681, 693, 705, 716, 728, 740, 752, 763, 775, 786, 798, 810, 822, 834, 845, 856], "exact_score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.08333333333333, 97.91666666666666, 97.66666666666667, 97.5, 97.25, 97.08333333333333, 97.0, 96.83333333333334, 96.75, 96.5, 96.25, 96.16666666666667, 95.75, 95.16666666666667, 94.5, 93.91666666666667, 93.25, 92.75, 92.25, 91.58333333333333, 91.0, 90.25, 89.66666666666667, 88.75, 87.83333333333333, 87.0, 86.33333333333333, 85.66666666666667, 84.91666666666666, 83.91666666666666, 83.16666666666667, 82.16666666666667, 81.41666666666667, 80.58333333333334, 79.66666666666666, 79.0, 78.16666666666667, 77.16666666666667, 76.16666666666667, 75.41666666666667, 74.41666666666666, 73.41666666666666, 72.50000000000001, 71.50000000000001, 70.5, 69.75, 68.83333333333333, 68.0, 67.0, 66.0, 65.25000000000001, 64.33333333333333, 63.416666666666664, 62.583333333333336, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.75, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.833333333333336, 53.833333333333336, 52.83333333333333, 52.0, 51.0, 50.0, 49.0, 48.0, 47.0, 46.08333333333333, 45.25, 44.25, 43.25, 42.333333333333336, 41.333333333333336, 40.33333333333333, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.58333333333333, 29.58333333333334, 28.666666666666664, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.7, "threshold_area100": 71.70333333333333, "threshold_area25": 97.49166666666667}, "test": {"score": [81.429064549534, 81.46358301691406, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.46358301691406, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.34276838108389, 81.30824991370383, 81.23921297894374, 81.17017604418363, 81.08387987573352, 80.94580600621333, 80.80773213669313, 80.63513979979288, 80.34173282706249, 79.996548153262, 79.7894373489817, 79.47877114256127, 78.9264756644805, 78.33966171901969, 77.89092164307905, 77.25232999654816, 76.30307214359682, 75.64722126337591, 74.85329651363479, 74.12840869865379, 73.43803935105282, 72.85122540559199, 71.9192267863307, 70.86641353123922, 70.33137728684846, 69.31308249913704, 68.69175008629618, 67.67345529858476, 67.34552985847428, 66.4998274076631, 65.30894028305143, 64.35968243010011, 63.4276838108388, 62.927166033828094, 61.75353814290645, 60.890576458405256, 60.26924404556438, 59.49257852951328, 58.59509837763204, 58.025543665861235, 57.455988954090444, 56.57576803589921, 55.902657921988265, 55.26406627545737, 53.60717984121506, 52.86503279254402, 52.39903348291336, 52.08836727649293, 51.31170176044184, 50.44874007594062, 49.98274076630998, 49.20607525025889, 48.3603727994477, 47.67000345184674, 46.68622713151536, 46.09941318605454, 45.61615464273386, 44.89126682775285, 44.20089748015189, 43.63134276838109, 43.06178805661029, 42.57852951328962, 41.974456334138765, 41.69830859509838, 41.07697618225751, 40.283051432516395, 39.76527442181567, 39.02312737314463, 38.280980324473596, 37.31446323783224, 37.021056265101834, 36.382464618570935, 36.05453917846047, 35.450465999309635, 34.79461511908871, 34.31135657576804, 33.690024162927166, 33.31032102174664, 32.654470141525714, 32.10217466344494, 31.532619951674146, 30.600621332412846, 30.35899206075251, 29.754918881601654, 28.9264756644805, 28.391439420089746, 27.666551605108737, 27.442181567138423, 27.114256127027957, 26.561960648947192], "calls": [0.0, 1.6051087331722471, 2.312737314463238, 2.7959958577839146, 3.3137728684846395, 3.8833275802554366, 4.798066965826718, 6.869175008629617, 7.8529513289609945, 8.733172247152227, 9.475319295823265, 10.217466344494303, 11.460131170176044, 12.909906800138074, 13.962720055229546, 15.4987918536417, 16.82775284777356, 17.949603037625128, 18.77804625474629, 20.417673455298583, 21.539523645150155, 23.38626164998274, 24.456334138764237, 25.975146703486363, 27.52847773558854, 28.89195719710045, 30.72143596824301, 32.861580945806004, 34.41491197790818, 36.27890921643079, 37.866758715913015, 39.299275112185015, 40.68001380738695, 42.474974111149464, 44.33897134967207, 45.4263030721436, 47.10044874007594, 48.25681739730756, 49.82740766309976, 50.46599930963065, 51.794960303762515, 53.727994477045215, 55.125992405937176, 56.351397997928885, 57.179841215050054, 58.87124611667242, 60.303762512944424, 61.58094580600621, 62.633759061097685, 63.84190541939938, 64.77390403866067, 65.51605108733172, 66.65516051087332, 67.44908526061442, 68.3638246461857, 70.22782188470832, 71.03900586813945, 71.74663444943045, 72.16085605799103, 72.97204004142216, 73.9212978943735, 74.5598895409044, 75.45736969278563, 76.40662754573697, 77.25232999654816, 78.32240248532966, 78.9955125992406, 79.58232654470142, 80.4280289955126, 81.18743527787366, 81.80876769071453, 82.4301001035554, 82.98239558163617, 83.62098722816707, 83.9316534345875, 84.67380048325855, 85.51950293406972, 86.0717984121505, 86.86572316189162, 87.64238867794269, 88.67794269934414, 89.00586813945462, 89.6444597859855, 90.02416292716603, 90.66275457369693, 91.37038315498792, 91.85364169830859, 92.47497411114946, 92.88919571971005, 93.61408353469106, 94.2181567138419, 94.80497065930273, 95.8577839143942, 96.11667241974456, 96.78978253365551, 97.61822575077666, 98.15326199516741, 98.89540904383846, 99.11977908180877, 99.44770452191922, 100.0], "rescued": [0, 17, 17, 17, 17, 17, 17, 19, 19, 19, 19, 20, 21, 21, 21, 24, 27, 30, 32, 35, 38, 49, 52, 61, 67, 69, 78, 85, 93, 103, 110, 114, 120, 127, 130, 135, 138, 143, 145, 149, 153, 159, 163, 165, 171, 177, 185, 196, 199, 201, 204, 205, 207, 208, 210, 212, 213, 217, 219, 219, 220, 220, 221, 222, 222, 222, 223, 223, 224, 224, 224, 225, 225, 225, 225, 226, 226, 226, 226, 226, 227, 228, 228, 229, 229, 229, 229, 229, 230, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 15, 15, 16, 16, 16, 16, 17, 20, 20, 20, 25, 28, 32, 36, 44, 55, 66, 78, 98, 121, 144, 165, 206, 246, 274, 320, 382, 428, 484, 533, 577, 617, 678, 742, 778, 840, 881, 942, 965, 1018, 1093, 1152, 1208, 1243, 1317, 1375, 1422, 1470, 1524, 1560, 1594, 1647, 1687, 1726, 1824, 1868, 1899, 1919, 1964, 2015, 2042, 2088, 2138, 2178, 2235, 2270, 2298, 2341, 2381, 2414, 2448, 2476, 2511, 2527, 2564, 2610, 2640, 2683, 2726, 2783, 2801, 2838, 2858, 2893, 2931, 2959, 2995, 3018, 3057, 3089, 3122, 3176, 3190, 3226, 3274, 3305, 3347, 3360, 3379, 3411], "exact_score": [81.429064549534, 81.39454608215395, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.429064549534, 81.46358301691406, 81.41180531584398, 81.41180531584398, 81.36002761477391, 81.34276838108389, 81.27373144632378, 81.20469451156369, 81.17017604418363, 81.10113910942354, 81.03210217466345, 80.9285467725233, 80.82499137038316, 80.63513979979288, 80.37625129444253, 80.20365895754229, 79.87573351743183, 79.77217811529168, 79.65136347946151, 79.25440110459095, 78.90921643079048, 78.59855022437004, 78.14981014842941, 77.90818087676908, 77.56299620296859, 77.18329306178806, 76.73455298584743, 76.23403520883673, 75.7852951328961, 75.35381429064549, 75.00862961684501, 74.59440800828443, 74.11114946496376, 73.67966862271315, 73.12737314463237, 72.67863306869175, 72.17811529168105, 71.59130134622022, 71.09078356920952, 70.59026579219882, 69.96893337935795, 69.36486020020712, 68.91612012426648, 68.2084915429755, 67.60441836382465, 66.96582671729375, 66.39627200552296, 65.70590265792198, 65.11908871246118, 64.41146013117019, 63.772868484639275, 63.099758370728345, 62.40938902312738, 61.66724197445633, 61.04590956161546, 60.424577148774596, 59.95857783914394, 59.31998619261305, 58.526061442871935, 57.818432861580945, 57.11080428028996, 56.24784259578875, 55.5056955471177, 54.729030031066614, 53.81429064549535, 52.916810493614086, 52.19192267863306, 51.294442526751816, 50.396962374870554, 49.620296858819465, 48.757335174318264, 47.84259578874698, 46.996893337935795, 46.09941318605454, 45.288229202623405, 44.39074905074215, 43.441491197790825, 42.56127027959959, 41.681049361408355, 40.7835692095271, 39.83431135657577, 38.90231273731447, 37.97031411805316, 37.03831549879186, 36.089057645840526, 35.157059026579226, 34.173282706247846, 33.24128408698654, 32.29202623403521, 31.39454608215395, 30.46254746289265, 29.54780807732137, 28.546772523299968, 27.56299620296859, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 63.12111942782035, "threshold_area100": 57.807473248187776, "threshold_area25": 80.71971004487402}, "selected_budget": 1, "selected": {"val": {"score": 98.66666666666667, "calls": 1.0, "rescued": 1, "harmed": 0}, "test": {"score": 81.46358301691406, "calls": 1.6051087331722471, "rescued": 17, "harmed": 15}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.004842729773372412, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0063249715603888035, "operator": "ge"}, {"budget": 0.03, "threshold": -0.007143797352910042, "operator": "ge"}, {"budget": 0.04, "threshold": -0.007921483367681503, "operator": "ge"}, {"budget": 0.05, "threshold": -0.009030326269567013, "operator": "ge"}, {"budget": 0.06, "threshold": -0.010685857385396957, "operator": "ge"}, {"budget": 0.07, "threshold": -0.015300276689231396, "operator": "ge"}, {"budget": 0.08, "threshold": -0.018539929762482643, "operator": "ge"}, {"budget": 0.09, "threshold": -0.021706044673919678, "operator": "ge"}, {"budget": 0.1, "threshold": -0.025133026763796806, "operator": "ge"}, {"budget": 0.11, "threshold": -0.029159247875213623, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03566305339336395, "operator": "ge"}, {"budget": 0.13, "threshold": -0.04552208259701729, "operator": "ge"}, {"budget": 0.14, "threshold": -0.057093583047389984, "operator": "ge"}, {"budget": 0.15, "threshold": -0.07223709672689438, "operator": "ge"}, {"budget": 0.16, "threshold": -0.09025540947914124, "operator": "ge"}, {"budget": 0.17, "threshold": -0.10614888370037079, "operator": "ge"}, {"budget": 0.18, "threshold": -0.11900870501995087, "operator": "ge"}, {"budget": 0.19, "threshold": -0.14532308280467987, "operator": "ge"}, {"budget": 0.2, "threshold": -0.1645021289587021, "operator": "ge"}, {"budget": 0.21, "threshold": -0.1965615600347519, "operator": "ge"}, {"budget": 0.22, "threshold": -0.22182922065258026, "operator": "ge"}, {"budget": 0.23, "threshold": -0.26034075021743774, "operator": "ge"}, {"budget": 0.24, "threshold": -0.3018578886985779, "operator": "ge"}, {"budget": 0.25, "threshold": -0.34069493412971497, "operator": "ge"}, {"budget": 0.26, "threshold": -0.3878975212574005, "operator": "ge"}, {"budget": 0.27, "threshold": -0.4436900019645691, "operator": "ge"}, {"budget": 0.28, "threshold": -0.47693032026290894, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5193653702735901, "operator": "ge"}, {"budget": 0.3, "threshold": -0.5532676577568054, "operator": "ge"}, {"budget": 0.31, "threshold": -0.5833015441894531, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6091939210891724, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6396224498748779, "operator": "ge"}, {"budget": 0.34, "threshold": -0.6672426462173462, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6815017461776733, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7100384831428528, "operator": "ge"}, {"budget": 0.37, "threshold": -0.7273280024528503, "operator": "ge"}, {"budget": 0.38, "threshold": -0.7472882270812988, "operator": "ge"}, {"budget": 0.39, "threshold": -0.7580146789550781, "operator": "ge"}, {"budget": 0.4, "threshold": -0.7743940353393555, "operator": "ge"}, {"budget": 0.41, "threshold": -0.7978382706642151, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8112537264823914, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8250505328178406, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8327509760856628, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8495388031005859, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8626042604446411, "operator": "ge"}, {"budget": 0.47, "threshold": -0.8735563158988953, "operator": "ge"}, {"budget": 0.48, "threshold": -0.8819729089736938, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8910600543022156, "operator": "ge"}, {"budget": 0.5, "threshold": -0.8982272148132324, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9042551517486572, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9119252562522888, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9170188903808594, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9225958585739136, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9320340156555176, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9355973601341248, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9391952753067017, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9426664113998413, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9458372592926025, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9484013319015503, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9509693384170532, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9539373517036438, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9565995931625366, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9599572420120239, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9631677865982056, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9661461114883423, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9688633680343628, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9716160893440247, "operator": "ge"}, {"budget": 0.69, "threshold": -0.97446608543396, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9761160016059875, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9783990383148193, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9798417687416077, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9818354845046997, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9828813076019287, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9847743511199951, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9860324263572693, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9870374202728271, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9880881905555725, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9889624118804932, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9899786114692688, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9905115962028503, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9911103844642639, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9916425347328186, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9923301935195923, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9930607080459595, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9935264587402344, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9939806461334229, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9943426251411438, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9949426651000977, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9954419732093811, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9958906769752502, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9966928958892822, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9968719482421875, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9972443580627441, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9977071285247803, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9979973435401917, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9983287453651428, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9984469413757324, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9987004995346069, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.08333333333333, 97.91666666666666, 97.66666666666667, 97.5, 97.25, 97.08333333333333, 97.0, 96.83333333333334, 96.75, 96.5, 96.25, 96.16666666666667, 95.75, 95.16666666666667, 94.5, 93.91666666666667, 93.25, 92.75, 92.25, 91.58333333333333, 91.0, 90.25, 89.66666666666667, 88.75, 87.83333333333333, 87.0, 86.33333333333333, 85.66666666666667, 84.91666666666666, 83.91666666666666, 83.25, 82.16666666666667, 81.41666666666667, 80.58333333333334, 79.66666666666666, 79.0, 78.16666666666667, 77.16666666666667, 76.16666666666667, 75.41666666666667, 74.41666666666666, 73.41666666666666, 72.50000000000001, 71.50000000000001, 70.5, 69.75, 68.91666666666667, 68.0, 67.0, 66.0, 65.25000000000001, 64.33333333333333, 63.416666666666664, 62.583333333333336, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.75, 57.91666666666666, 56.833333333333336, 55.833333333333336, 54.833333333333336, 53.833333333333336, 52.83333333333333, 52.0, 51.0, 50.0, 49.0, 48.0, 47.0, 46.08333333333333, 45.333333333333336, 44.25, 43.25, 42.333333333333336, 41.333333333333336, 40.33333333333333, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.58333333333333, 29.58333333333334, 28.666666666666664, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 7, 9, 12, 14, 17, 19, 21, 24, 25, 28, 31, 32, 37, 44, 52, 59, 68, 74, 80, 88, 95, 104, 111, 122, 133, 143, 151, 159, 168, 180, 188, 201, 210, 220, 231, 240, 250, 262, 274, 283, 295, 307, 318, 330, 342, 351, 361, 372, 384, 396, 406, 417, 428, 438, 450, 462, 473, 484, 494, 507, 519, 531, 543, 555, 565, 577, 589, 601, 613, 625, 636, 645, 658, 670, 681, 693, 705, 716, 728, 740, 752, 763, 775, 786, 798, 810, 822, 834, 845, 856], "exact_score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.08333333333333, 97.91666666666666, 97.66666666666667, 97.5, 97.25, 97.08333333333333, 97.0, 96.83333333333334, 96.75, 96.5, 96.25, 96.16666666666667, 95.75, 95.16666666666667, 94.5, 93.91666666666667, 93.25, 92.75, 92.25, 91.58333333333333, 91.0, 90.25, 89.66666666666667, 88.75, 87.83333333333333, 87.0, 86.33333333333333, 85.66666666666667, 84.91666666666666, 83.91666666666666, 83.16666666666667, 82.16666666666667, 81.41666666666667, 80.58333333333334, 79.66666666666666, 79.0, 78.16666666666667, 77.16666666666667, 76.16666666666667, 75.41666666666667, 74.41666666666666, 73.41666666666666, 72.50000000000001, 71.50000000000001, 70.5, 69.75, 68.83333333333333, 68.0, 67.0, 66.0, 65.25000000000001, 64.33333333333333, 63.416666666666664, 62.583333333333336, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.75, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.833333333333336, 53.833333333333336, 52.83333333333333, 52.0, 51.0, 50.0, 49.0, 48.0, 47.0, 46.08333333333333, 45.25, 44.25, 43.25, 42.333333333333336, 41.333333333333336, 40.33333333333333, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.58333333333333, 29.58333333333334, 28.666666666666664, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.7, "threshold_area100": 71.70333333333333, "threshold_area25": 97.49166666666667}, "test": {"score": [81.429064549534, 81.46358301691406, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.46358301691406, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.34276838108389, 81.30824991370383, 81.23921297894374, 81.17017604418363, 81.08387987573352, 80.94580600621333, 80.80773213669313, 80.63513979979288, 80.34173282706249, 79.996548153262, 79.7894373489817, 79.47877114256127, 78.9264756644805, 78.33966171901969, 77.89092164307905, 77.25232999654816, 76.30307214359682, 75.64722126337591, 74.85329651363479, 74.12840869865379, 73.43803935105282, 72.85122540559199, 71.9192267863307, 70.86641353123922, 70.33137728684846, 69.31308249913704, 68.69175008629618, 67.67345529858476, 67.34552985847428, 66.4998274076631, 65.30894028305143, 64.35968243010011, 63.4276838108388, 62.927166033828094, 61.75353814290645, 60.890576458405256, 60.26924404556438, 59.49257852951328, 58.59509837763204, 58.025543665861235, 57.455988954090444, 56.57576803589921, 55.902657921988265, 55.26406627545737, 53.60717984121506, 52.86503279254402, 52.39903348291336, 52.08836727649293, 51.31170176044184, 50.44874007594062, 49.98274076630998, 49.20607525025889, 48.3603727994477, 47.67000345184674, 46.68622713151536, 46.09941318605454, 45.61615464273386, 44.89126682775285, 44.20089748015189, 43.63134276838109, 43.06178805661029, 42.57852951328962, 41.974456334138765, 41.69830859509838, 41.07697618225751, 40.283051432516395, 39.76527442181567, 39.02312737314463, 38.280980324473596, 37.31446323783224, 37.021056265101834, 36.382464618570935, 36.05453917846047, 35.450465999309635, 34.79461511908871, 34.31135657576804, 33.690024162927166, 33.31032102174664, 32.654470141525714, 32.10217466344494, 31.532619951674146, 30.600621332412846, 30.35899206075251, 29.754918881601654, 28.9264756644805, 28.391439420089746, 27.666551605108737, 27.442181567138423, 27.114256127027957, 26.561960648947192], "calls": [0.0, 1.6051087331722471, 2.312737314463238, 2.7959958577839146, 3.3137728684846395, 3.8833275802554366, 4.798066965826718, 6.869175008629617, 7.8529513289609945, 8.733172247152227, 9.475319295823265, 10.217466344494303, 11.460131170176044, 12.909906800138074, 13.962720055229546, 15.4987918536417, 16.82775284777356, 17.949603037625128, 18.77804625474629, 20.417673455298583, 21.539523645150155, 23.38626164998274, 24.456334138764237, 25.975146703486363, 27.52847773558854, 28.89195719710045, 30.72143596824301, 32.861580945806004, 34.41491197790818, 36.27890921643079, 37.866758715913015, 39.299275112185015, 40.68001380738695, 42.474974111149464, 44.33897134967207, 45.4263030721436, 47.10044874007594, 48.25681739730756, 49.82740766309976, 50.46599930963065, 51.794960303762515, 53.727994477045215, 55.125992405937176, 56.351397997928885, 57.179841215050054, 58.87124611667242, 60.303762512944424, 61.58094580600621, 62.633759061097685, 63.84190541939938, 64.77390403866067, 65.51605108733172, 66.65516051087332, 67.44908526061442, 68.3638246461857, 70.22782188470832, 71.03900586813945, 71.74663444943045, 72.16085605799103, 72.97204004142216, 73.9212978943735, 74.5598895409044, 75.45736969278563, 76.40662754573697, 77.25232999654816, 78.32240248532966, 78.9955125992406, 79.58232654470142, 80.4280289955126, 81.18743527787366, 81.80876769071453, 82.4301001035554, 82.98239558163617, 83.62098722816707, 83.9316534345875, 84.67380048325855, 85.51950293406972, 86.0717984121505, 86.86572316189162, 87.64238867794269, 88.67794269934414, 89.00586813945462, 89.6444597859855, 90.02416292716603, 90.66275457369693, 91.37038315498792, 91.85364169830859, 92.47497411114946, 92.88919571971005, 93.61408353469106, 94.2181567138419, 94.80497065930273, 95.8577839143942, 96.11667241974456, 96.78978253365551, 97.61822575077666, 98.15326199516741, 98.89540904383846, 99.11977908180877, 99.44770452191922, 100.0], "rescued": [0, 17, 17, 17, 17, 17, 17, 19, 19, 19, 19, 20, 21, 21, 21, 24, 27, 30, 32, 35, 38, 49, 52, 61, 67, 69, 78, 85, 93, 103, 110, 114, 120, 127, 130, 135, 138, 143, 145, 149, 153, 159, 163, 165, 171, 177, 185, 196, 199, 201, 204, 205, 207, 208, 210, 212, 213, 217, 219, 219, 220, 220, 221, 222, 222, 222, 223, 223, 224, 224, 224, 225, 225, 225, 225, 226, 226, 226, 226, 226, 227, 228, 228, 229, 229, 229, 229, 229, 230, 231, 231, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 15, 15, 16, 16, 16, 16, 17, 20, 20, 20, 25, 28, 32, 36, 44, 55, 66, 78, 98, 121, 144, 165, 206, 246, 274, 320, 382, 428, 484, 533, 577, 617, 678, 742, 778, 840, 881, 942, 965, 1018, 1093, 1152, 1208, 1243, 1317, 1375, 1422, 1470, 1524, 1560, 1594, 1647, 1687, 1726, 1824, 1868, 1899, 1919, 1964, 2015, 2042, 2088, 2138, 2178, 2235, 2270, 2298, 2341, 2381, 2414, 2448, 2476, 2511, 2527, 2564, 2610, 2640, 2683, 2726, 2783, 2801, 2838, 2858, 2893, 2931, 2959, 2995, 3018, 3057, 3089, 3122, 3176, 3190, 3226, 3274, 3305, 3347, 3360, 3379, 3411], "exact_score": [81.429064549534, 81.39454608215395, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.429064549534, 81.46358301691406, 81.41180531584398, 81.41180531584398, 81.36002761477391, 81.34276838108389, 81.27373144632378, 81.20469451156369, 81.17017604418363, 81.10113910942354, 81.03210217466345, 80.9285467725233, 80.82499137038316, 80.63513979979288, 80.37625129444253, 80.20365895754229, 79.87573351743183, 79.77217811529168, 79.65136347946151, 79.25440110459095, 78.90921643079048, 78.59855022437004, 78.14981014842941, 77.90818087676908, 77.56299620296859, 77.18329306178806, 76.73455298584743, 76.23403520883673, 75.7852951328961, 75.35381429064549, 75.00862961684501, 74.59440800828443, 74.11114946496376, 73.67966862271315, 73.12737314463237, 72.67863306869175, 72.17811529168105, 71.59130134622022, 71.09078356920952, 70.59026579219882, 69.96893337935795, 69.36486020020712, 68.91612012426648, 68.2084915429755, 67.60441836382465, 66.96582671729375, 66.39627200552296, 65.70590265792198, 65.11908871246118, 64.41146013117019, 63.772868484639275, 63.099758370728345, 62.40938902312738, 61.66724197445633, 61.04590956161546, 60.424577148774596, 59.95857783914394, 59.31998619261305, 58.526061442871935, 57.818432861580945, 57.11080428028996, 56.24784259578875, 55.5056955471177, 54.729030031066614, 53.81429064549535, 52.916810493614086, 52.19192267863306, 51.294442526751816, 50.396962374870554, 49.620296858819465, 48.757335174318264, 47.84259578874698, 46.996893337935795, 46.09941318605454, 45.288229202623405, 44.39074905074215, 43.441491197790825, 42.56127027959959, 41.681049361408355, 40.7835692095271, 39.83431135657577, 38.90231273731447, 37.97031411805316, 37.03831549879186, 36.089057645840526, 35.157059026579226, 34.173282706247846, 33.24128408698654, 32.29202623403521, 31.39454608215395, 30.46254746289265, 29.54780807732137, 28.546772523299968, 27.56299620296859, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 63.12111942782035, "threshold_area100": 57.807473248187776, "threshold_area25": 80.71971004487402}, "selected_budget": 1, "selected": {"val": {"score": 98.66666666666667, "calls": 1.0, "rescued": 1, "harmed": 0}, "test": {"score": 81.46358301691406, "calls": 1.6051087331722471, "rescued": 17, "harmed": 15}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.2544234991073608, "operator": "ge"}, {"budget": 0.02, "threshold": -1.936328649520874, "operator": "ge"}, {"budget": 0.03, "threshold": -2.8112034797668457, "operator": "ge"}, {"budget": 0.04, "threshold": -3.2183549404144287, "operator": "ge"}, {"budget": 0.05, "threshold": -3.6870334148406982, "operator": "ge"}, {"budget": 0.06, "threshold": -4.061175346374512, "operator": "ge"}, {"budget": 0.07, "threshold": -4.389405250549316, "operator": "ge"}, {"budget": 0.08, "threshold": -4.626847267150879, "operator": "ge"}, {"budget": 0.09, "threshold": -4.925370693206787, "operator": "ge"}, {"budget": 0.1, "threshold": -5.2924652099609375, "operator": "ge"}, {"budget": 0.11, "threshold": -5.763829231262207, "operator": "ge"}, {"budget": 0.12, "threshold": -6.0098185539245605, "operator": "ge"}, {"budget": 0.13, "threshold": -6.228329658508301, "operator": "ge"}, {"budget": 0.14, "threshold": -6.6207685470581055, "operator": "ge"}, {"budget": 0.15, "threshold": -6.868588924407959, "operator": "ge"}, {"budget": 0.16, "threshold": -7.260344982147217, "operator": "ge"}, {"budget": 0.17, "threshold": -7.540545463562012, "operator": "ge"}, {"budget": 0.18, "threshold": -8.024834632873535, "operator": "ge"}, {"budget": 0.19, "threshold": -8.346846580505371, "operator": "ge"}, {"budget": 0.2, "threshold": -8.631322860717773, "operator": "ge"}, {"budget": 0.21, "threshold": -8.971366882324219, "operator": "ge"}, {"budget": 0.22, "threshold": -9.246245384216309, "operator": "ge"}, {"budget": 0.23, "threshold": -9.505003929138184, "operator": "ge"}, {"budget": 0.24, "threshold": -9.90048599243164, "operator": "ge"}, {"budget": 0.25, "threshold": -10.139074325561523, "operator": "ge"}, {"budget": 0.26, "threshold": -10.242990493774414, "operator": "ge"}, {"budget": 0.27, "threshold": -10.3602933883667, "operator": "ge"}, {"budget": 0.28, "threshold": -10.548548698425293, "operator": "ge"}, {"budget": 0.29, "threshold": -10.737157821655273, "operator": "ge"}, {"budget": 0.3, "threshold": -10.867067337036133, "operator": "ge"}, {"budget": 0.31, "threshold": -10.978067398071289, "operator": "ge"}, {"budget": 0.32, "threshold": -11.0880126953125, "operator": "ge"}, {"budget": 0.33, "threshold": -11.181827545166016, "operator": "ge"}, {"budget": 0.34, "threshold": -11.232829093933105, "operator": "ge"}, {"budget": 0.35, "threshold": -11.321667671203613, "operator": "ge"}, {"budget": 0.36, "threshold": -11.442927360534668, "operator": "ge"}, {"budget": 0.37, "threshold": -11.513103485107422, "operator": "ge"}, {"budget": 0.38, "threshold": -11.590832710266113, "operator": "ge"}, {"budget": 0.39, "threshold": -11.712173461914062, "operator": "ge"}, {"budget": 0.4, "threshold": -11.789525985717773, "operator": "ge"}, {"budget": 0.41, "threshold": -11.849559783935547, "operator": "ge"}, {"budget": 0.42, "threshold": -11.937789916992188, "operator": "ge"}, {"budget": 0.43, "threshold": -12.039793014526367, "operator": "ge"}, {"budget": 0.44, "threshold": -12.118639945983887, "operator": "ge"}, {"budget": 0.45, "threshold": -12.217530250549316, "operator": "ge"}, {"budget": 0.46, "threshold": -12.278141021728516, "operator": "ge"}, {"budget": 0.47, "threshold": -12.339448928833008, "operator": "ge"}, {"budget": 0.48, "threshold": -12.472339630126953, "operator": "ge"}, {"budget": 0.49, "threshold": -12.575360298156738, "operator": "ge"}, {"budget": 0.5, "threshold": -12.655588150024414, "operator": "ge"}, {"budget": 0.51, "threshold": -12.716165542602539, "operator": "ge"}, {"budget": 0.52, "threshold": -12.802321434020996, "operator": "ge"}, {"budget": 0.53, "threshold": -12.894624710083008, "operator": "ge"}, {"budget": 0.54, "threshold": -12.965235710144043, "operator": "ge"}, {"budget": 0.55, "threshold": -13.048943519592285, "operator": "ge"}, {"budget": 0.56, "threshold": -13.10671615600586, "operator": "ge"}, {"budget": 0.57, "threshold": -13.195165634155273, "operator": "ge"}, {"budget": 0.58, "threshold": -13.234576225280762, "operator": "ge"}, {"budget": 0.59, "threshold": -13.312433242797852, "operator": "ge"}, {"budget": 0.6, "threshold": -13.359681129455566, "operator": "ge"}, {"budget": 0.61, "threshold": -13.400371551513672, "operator": "ge"}, {"budget": 0.62, "threshold": -13.471892356872559, "operator": "ge"}, {"budget": 0.63, "threshold": -13.52037239074707, "operator": "ge"}, {"budget": 0.64, "threshold": -13.560891151428223, "operator": "ge"}, {"budget": 0.65, "threshold": -13.635615348815918, "operator": "ge"}, {"budget": 0.66, "threshold": -13.665932655334473, "operator": "ge"}, {"budget": 0.67, "threshold": -13.71776008605957, "operator": "ge"}, {"budget": 0.68, "threshold": -13.748178482055664, "operator": "ge"}, {"budget": 0.69, "threshold": -13.808197975158691, "operator": "ge"}, {"budget": 0.7, "threshold": -13.886240005493164, "operator": "ge"}, {"budget": 0.71, "threshold": -13.950531005859375, "operator": "ge"}, {"budget": 0.72, "threshold": -13.977849960327148, "operator": "ge"}, {"budget": 0.73, "threshold": -14.037473678588867, "operator": "ge"}, {"budget": 0.74, "threshold": -14.096827507019043, "operator": "ge"}, {"budget": 0.75, "threshold": -14.134664535522461, "operator": "ge"}, {"budget": 0.76, "threshold": -14.17977237701416, "operator": "ge"}, {"budget": 0.77, "threshold": -14.23458194732666, "operator": "ge"}, {"budget": 0.78, "threshold": -14.32077407836914, "operator": "ge"}, {"budget": 0.79, "threshold": -14.373430252075195, "operator": "ge"}, {"budget": 0.8, "threshold": -14.436912536621094, "operator": "ge"}, {"budget": 0.81, "threshold": -14.504549026489258, "operator": "ge"}, {"budget": 0.82, "threshold": -14.5410795211792, "operator": "ge"}, {"budget": 0.83, "threshold": -14.579818725585938, "operator": "ge"}, {"budget": 0.84, "threshold": -14.61104965209961, "operator": "ge"}, {"budget": 0.85, "threshold": -14.694195747375488, "operator": "ge"}, {"budget": 0.86, "threshold": -14.75511360168457, "operator": "ge"}, {"budget": 0.87, "threshold": -14.817962646484375, "operator": "ge"}, {"budget": 0.88, "threshold": -14.873027801513672, "operator": "ge"}, {"budget": 0.89, "threshold": -14.955135345458984, "operator": "ge"}, {"budget": 0.9, "threshold": -15.028950691223145, "operator": "ge"}, {"budget": 0.91, "threshold": -15.096561431884766, "operator": "ge"}, {"budget": 0.92, "threshold": -15.15115737915039, "operator": "ge"}, {"budget": 0.93, "threshold": -15.229963302612305, "operator": "ge"}, {"budget": 0.94, "threshold": -15.288959503173828, "operator": "ge"}, {"budget": 0.95, "threshold": -15.372575759887695, "operator": "ge"}, {"budget": 0.96, "threshold": -15.488283157348633, "operator": "ge"}, {"budget": 0.97, "threshold": -15.637356758117676, "operator": "ge"}, {"budget": 0.98, "threshold": -15.735271453857422, "operator": "ge"}, {"budget": 0.99, "threshold": -16.093284606933594, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.33333333333334, 97.75, 97.25, 96.83333333333334, 96.08333333333333, 95.5, 94.83333333333334, 94.16666666666667, 93.5, 93.0, 92.25, 91.66666666666666, 90.91666666666667, 90.16666666666666, 89.5, 88.75, 88.0, 87.33333333333333, 86.5, 85.83333333333334, 85.25, 84.66666666666667, 83.91666666666666, 83.33333333333334, 82.91666666666666, 82.58333333333333, 82.25, 81.83333333333334, 81.25, 80.91666666666667, 80.41666666666667, 80.0, 79.58333333333334, 79.5, 79.08333333333333, 78.58333333333334, 78.08333333333334, 77.5, 76.83333333333333, 76.58333333333334, 76.08333333333334, 75.5, 74.91666666666667, 74.58333333333333, 73.91666666666666, 73.33333333333334, 72.58333333333333, 72.08333333333333, 71.50000000000001, 70.66666666666667, 69.75, 68.91666666666667, 68.16666666666666, 67.41666666666667, 66.58333333333333, 65.91666666666667, 65.33333333333333, 64.5, 63.83333333333334, 62.916666666666664, 62.0, 61.25000000000001, 60.5, 59.583333333333336, 58.916666666666664, 58.25, 57.49999999999999, 56.666666666666664, 55.91666666666666, 55.08333333333333, 54.16666666666667, 53.333333333333336, 52.416666666666664, 51.5, 50.66666666666666, 49.833333333333336, 49.0, 48.25000000000001, 47.25, 46.41666666666667, 45.58333333333333, 44.833333333333336, 43.91666666666667, 43.08333333333333, 42.25, 41.333333333333336, 40.33333333333333, 39.33333333333333, 38.416666666666664, 37.416666666666664, 36.5, 35.5, 34.5, 33.5, 32.58333333333333, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 3, 3, 3, 3, 3, 4, 4, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 6, 13, 19, 24, 33, 41, 49, 58, 66, 73, 82, 89, 98, 107, 115, 124, 133, 141, 151, 159, 166, 173, 182, 189, 194, 198, 202, 207, 214, 218, 224, 229, 234, 235, 240, 246, 252, 259, 267, 270, 276, 283, 290, 294, 302, 309, 318, 324, 331, 341, 352, 362, 371, 380, 390, 398, 405, 415, 423, 434, 445, 454, 463, 474, 482, 490, 499, 509, 518, 528, 539, 549, 560, 571, 581, 591, 601, 610, 622, 632, 642, 651, 662, 672, 682, 693, 705, 717, 728, 740, 751, 763, 775, 787, 798, 809, 820, 832, 844, 856], "exact_score": [98.58333333333333, 98.33333333333334, 97.75, 97.25, 96.83333333333334, 96.08333333333333, 95.5, 94.83333333333334, 94.16666666666667, 93.5, 93.0, 92.25, 91.66666666666666, 90.91666666666667, 90.16666666666666, 89.5, 88.75, 88.0, 87.33333333333333, 86.5, 85.83333333333334, 85.25, 84.66666666666667, 83.91666666666666, 83.33333333333334, 82.91666666666666, 82.58333333333333, 82.25, 81.83333333333334, 81.25, 80.91666666666667, 80.41666666666667, 80.0, 79.58333333333334, 79.5, 79.08333333333333, 78.58333333333334, 78.08333333333334, 77.5, 76.83333333333333, 76.58333333333334, 76.0, 75.5, 74.91666666666667, 74.58333333333333, 73.91666666666666, 73.33333333333334, 72.58333333333333, 72.08333333333333, 71.50000000000001, 70.66666666666667, 69.75, 68.91666666666667, 68.16666666666666, 67.41666666666667, 66.58333333333333, 65.91666666666667, 65.25000000000001, 64.5, 63.83333333333334, 62.916666666666664, 62.0, 61.25000000000001, 60.5, 59.583333333333336, 58.916666666666664, 58.25, 57.49999999999999, 56.666666666666664, 55.833333333333336, 55.08333333333333, 54.16666666666667, 53.333333333333336, 52.416666666666664, 51.5, 50.66666666666666, 49.833333333333336, 49.0, 48.25000000000001, 47.25, 46.41666666666667, 45.58333333333333, 44.833333333333336, 43.91666666666667, 43.08333333333333, 42.25, 41.333333333333336, 40.33333333333333, 39.33333333333333, 38.416666666666664, 37.416666666666664, 36.5, 35.5, 34.5, 33.5, 32.58333333333333, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 67.23166666666667, "threshold_area100": 67.23416666666667, "threshold_area25": 91.03333333333335}, "test": {"score": [81.429064549534, 80.98032447359338, 80.41076976182258, 79.01277183293061, 77.99447704521918, 76.32033137728685, 74.88781498101484, 73.74870555747324, 72.73041076976182, 71.41870900932, 69.60648947186745, 67.46634449430445, 66.3617535381429, 65.60234725578185, 64.18709009319986, 63.375906109768735, 62.06420434932689, 61.18398343113566, 59.76872626855367, 58.95754228512254, 58.3534691059717, 57.57680358992061, 56.98998964445978, 56.523990334829136, 55.60925094925786, 55.16051087331723, 54.95340006903693, 54.67725232999655, 54.29754918881602, 53.74525371073524, 53.40006903693476, 53.123921297894384, 52.74421815671384, 52.55436658612358, 52.450811183983426, 52.140144977563, 51.794960303762515, 51.656886434242324, 51.518812564722126, 51.1391094235416, 50.93199861926131, 50.84570245081119, 50.53503624439075, 50.27614773904039, 49.94822229892993, 49.603037625129446, 49.48222298929928, 49.24059371763894, 48.56748360372799, 48.08422506040732, 47.82533655505696, 47.480151881256475, 47.15222644114602, 46.703486365205386, 46.358301691404904, 45.94408008284432, 45.63341387642389, 45.11563686572316, 44.96030376251295, 44.56334138764239, 44.304452882292026, 43.90749050742147, 43.458750431480844, 43.02726958923024, 42.85467725233, 42.2851225405592, 42.09527096996894, 41.6465308940283, 41.43942008974802, 40.904383845357266, 40.352088367276494, 39.93786675871592, 39.67897825336556, 39.074905074214705, 38.505350362443906, 38.10838798757336, 37.849499482222996, 37.33172247152227, 36.589575422851226, 36.05453917846047, 35.450465999309635, 34.81187435277874, 34.466689678978256, 34.121505005177774, 33.86261649982741, 33.17224715222644, 32.68898860890577, 32.11943389713497, 31.72247152226441, 30.980324473593377, 30.341732827062483, 29.806696582671734, 29.47877114256128, 29.133586468760786, 28.840179496030384, 28.374180186399723, 28.046254746289268, 27.64929237141871, 27.37314463237832, 26.855367621677594, 26.561960648947192], "calls": [0.0, 3.3482913358646877, 6.213324128408699, 11.391094235415949, 15.13634794615119, 19.761822575077666, 24.059371763893683, 27.304107697618225, 29.720400414221608, 32.706247842595786, 36.26164998274077, 40.11045909561615, 42.06075250258888, 43.527787366240936, 45.66793234380393, 46.910597169485676, 48.843631342768376, 50.293406972730416, 52.55436658612358, 53.71073524335519, 54.60821539523645, 55.83362098722817, 56.748360372799446, 57.71487745944081, 59.19917155678288, 60.079392474974114, 60.47635484984467, 60.925094925785295, 61.89161201242664, 62.97894373489817, 63.824646185709355, 64.60131170176044, 65.2916810493614, 65.87849499482223, 66.17190196755264, 66.7069382119434, 67.6389368312047, 68.12219537452538, 68.62271315153608, 69.43389713496721, 69.81360027614774, 70.02071108042803, 70.5730065585088, 71.19433897134967, 71.93648602002071, 72.48878149810149, 72.73041076976182, 73.04107697618227, 73.99033482913359, 74.68070417673457, 75.14670348636521, 75.59544356230583, 76.18225750776666, 76.8553676216776, 77.23507076285813, 77.87366240938903, 78.28788401794961, 78.96099413186055, 79.18536416983086, 79.70314118053159, 80.04832585433206, 80.53158439765275, 81.08387987573352, 81.61891612012427, 81.84328615809459, 82.48187780462547, 82.70624784259579, 83.22402485329651, 83.51743182602692, 84.08698653779773, 84.77735588539869, 85.27787366240939, 85.62305833620987, 86.27890921643079, 86.95201933034173, 87.40075940628236, 87.76320331377286, 88.29823955816362, 89.12668277528478, 89.74801518812565, 90.40386606834656, 91.1460131170176, 91.54297549188816, 91.97445633413876, 92.31964100793924, 93.06178805661028, 93.57956506731101, 94.20089748015188, 94.59785985502243, 95.37452537107353, 96.06489471867448, 96.66896789782534, 96.9968933379358, 97.34207801173628, 97.65274421815671, 98.13600276147739, 98.46392820158785, 98.8608905764584, 99.1370383154988, 99.68933379357956, 100.0], "rescued": [0, 34, 51, 85, 110, 126, 146, 157, 170, 182, 190, 201, 208, 214, 217, 217, 220, 222, 223, 224, 225, 225, 225, 225, 226, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 229, 229, 229, 229, 230, 230, 230, 230, 230, 230, 230, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 60, 110, 225, 309, 422, 525, 602, 674, 762, 875, 1010, 1081, 1131, 1216, 1263, 1342, 1395, 1478, 1526, 1562, 1607, 1641, 1668, 1722, 1749, 1761, 1777, 1799, 1831, 1851, 1867, 1889, 1900, 1906, 1924, 1944, 1952, 1960, 1983, 1995, 2000, 2018, 2033, 2052, 2072, 2079, 2093, 2132, 2160, 2175, 2195, 2214, 2240, 2260, 2284, 2302, 2332, 2341, 2364, 2379, 2402, 2428, 2453, 2463, 2496, 2507, 2533, 2545, 2576, 2608, 2632, 2648, 2683, 2716, 2739, 2755, 2785, 2828, 2859, 2894, 2931, 2951, 2972, 2987, 3028, 3056, 3089, 3112, 3155, 3192, 3223, 3242, 3262, 3279, 3306, 3325, 3348, 3364, 3394, 3411], "exact_score": [81.429064549534, 81.37728684846392, 81.23921297894374, 81.01484294097342, 80.89402830514325, 80.66965826717293, 80.47980669658268, 80.18639972385226, 80.11736278909216, 79.84121505005179, 79.39247497411115, 79.08180876769072, 78.9264756644805, 78.58129099068002, 78.25336555056955, 78.04625474628926, 77.68381083879876, 77.32136693130826, 76.99344149119779, 76.63099758370728, 76.21677597514672, 75.87159130134621, 75.50914739385571, 75.2157404211253, 74.93959268208492, 74.62892647566449, 74.11114946496376, 73.86952019330342, 73.47255781843286, 73.11011391094236, 72.67863306869175, 72.2816706938212, 71.71211598205039, 71.2806351397998, 70.74559889540905, 70.17604418363825, 69.74456334138765, 69.15774939592683, 68.60545391784605, 68.15671384190543, 67.5008629616845, 66.93130824991371, 66.39627200552296, 65.9302726958923, 65.25716258198136, 64.5667932343804, 63.9972385226096, 63.324128408698655, 62.68553676216776, 61.977908180876774, 61.391094235415956, 60.83879875733518, 60.200207110804286, 59.45806006213324, 58.76769071453227, 58.094580600621335, 57.455988954090444, 56.869175008629625, 56.33413876423887, 55.73006558508802, 55.195029340697275, 54.65999309630652, 54.21125302036589, 53.74525371073524, 53.38280980324473, 52.916810493614086, 52.519848118743525, 52.0020711080428, 51.70866413531239, 51.363479461511915, 50.862961684501215, 50.362443907490515, 49.896444597859855, 49.29237141870901, 48.56748360372799, 47.894373489817056, 47.273041076976185, 46.58267172937522, 45.857783914394204, 45.098377632033134, 44.33897134967207, 43.51052813255092, 42.7166033828098, 41.8536416983086, 40.99068001380739, 40.17949603037626, 39.36831204694512, 38.47083189506386, 37.62512944425268, 36.7103900586814, 35.81290990680014, 34.94994822229893, 34.10424577148775, 33.24128408698654, 32.30928546772523, 31.32550914739386, 30.410769761822575, 29.47877114256128, 28.49499482222989, 27.511218501898515, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 59.822703821898536, "threshold_area100": 48.48438039351053, "threshold_area25": 67.19019675526407}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.21525436639785767, "operator": "ge"}, {"budget": 0.02, "threshold": -0.6004631519317627, "operator": "ge"}, {"budget": 0.03, "threshold": -0.8570535778999329, "operator": "ge"}, {"budget": 0.04, "threshold": -1.1159348487854004, "operator": "ge"}, {"budget": 0.05, "threshold": -1.3006268739700317, "operator": "ge"}, {"budget": 0.06, "threshold": -1.561065673828125, "operator": "ge"}, {"budget": 0.07, "threshold": -1.873005986213684, "operator": "ge"}, {"budget": 0.08, "threshold": -2.144855260848999, "operator": "ge"}, {"budget": 0.09, "threshold": -2.4501521587371826, "operator": "ge"}, {"budget": 0.1, "threshold": -2.7097389698028564, "operator": "ge"}, {"budget": 0.11, "threshold": -3.035614490509033, "operator": "ge"}, {"budget": 0.12, "threshold": -3.361393451690674, "operator": "ge"}, {"budget": 0.13, "threshold": -3.5016636848449707, "operator": "ge"}, {"budget": 0.14, "threshold": -3.800288200378418, "operator": "ge"}, {"budget": 0.15, "threshold": -4.0986647605896, "operator": "ge"}, {"budget": 0.16, "threshold": -4.40424919128418, "operator": "ge"}, {"budget": 0.17, "threshold": -4.801357746124268, "operator": "ge"}, {"budget": 0.18, "threshold": -5.038323402404785, "operator": "ge"}, {"budget": 0.19, "threshold": -5.426069259643555, "operator": "ge"}, {"budget": 0.2, "threshold": -5.93463134765625, "operator": "ge"}, {"budget": 0.21, "threshold": -6.235710620880127, "operator": "ge"}, {"budget": 0.22, "threshold": -6.515361309051514, "operator": "ge"}, {"budget": 0.23, "threshold": -6.754908561706543, "operator": "ge"}, {"budget": 0.24, "threshold": -6.975771427154541, "operator": "ge"}, {"budget": 0.25, "threshold": -7.24811315536499, "operator": "ge"}, {"budget": 0.26, "threshold": -7.5063018798828125, "operator": "ge"}, {"budget": 0.27, "threshold": -7.718111038208008, "operator": "ge"}, {"budget": 0.28, "threshold": -7.839001178741455, "operator": "ge"}, {"budget": 0.29, "threshold": -7.952550888061523, "operator": "ge"}, {"budget": 0.3, "threshold": -8.062766075134277, "operator": "ge"}, {"budget": 0.31, "threshold": -8.142936706542969, "operator": "ge"}, {"budget": 0.32, "threshold": -8.222963333129883, "operator": "ge"}, {"budget": 0.33, "threshold": -8.2814359664917, "operator": "ge"}, {"budget": 0.34, "threshold": -8.422525405883789, "operator": "ge"}, {"budget": 0.35, "threshold": -8.502758026123047, "operator": "ge"}, {"budget": 0.36, "threshold": -8.586461067199707, "operator": "ge"}, {"budget": 0.37, "threshold": -8.723908424377441, "operator": "ge"}, {"budget": 0.38, "threshold": -8.854707717895508, "operator": "ge"}, {"budget": 0.39, "threshold": -9.017977714538574, "operator": "ge"}, {"budget": 0.4, "threshold": -9.095817565917969, "operator": "ge"}, {"budget": 0.41, "threshold": -9.170808792114258, "operator": "ge"}, {"budget": 0.42, "threshold": -9.278587341308594, "operator": "ge"}, {"budget": 0.43, "threshold": -9.381091117858887, "operator": "ge"}, {"budget": 0.44, "threshold": -9.434070587158203, "operator": "ge"}, {"budget": 0.45, "threshold": -9.50454330444336, "operator": "ge"}, {"budget": 0.46, "threshold": -9.563300132751465, "operator": "ge"}, {"budget": 0.47, "threshold": -9.624507904052734, "operator": "ge"}, {"budget": 0.48, "threshold": -9.719067573547363, "operator": "ge"}, {"budget": 0.49, "threshold": -9.776866912841797, "operator": "ge"}, {"budget": 0.5, "threshold": -9.841348648071289, "operator": "ge"}, {"budget": 0.51, "threshold": -9.91812515258789, "operator": "ge"}, {"budget": 0.52, "threshold": -10.01771068572998, "operator": "ge"}, {"budget": 0.53, "threshold": -10.101547241210938, "operator": "ge"}, {"budget": 0.54, "threshold": -10.186783790588379, "operator": "ge"}, {"budget": 0.55, "threshold": -10.260391235351562, "operator": "ge"}, {"budget": 0.56, "threshold": -10.361360549926758, "operator": "ge"}, {"budget": 0.57, "threshold": -10.453540802001953, "operator": "ge"}, {"budget": 0.58, "threshold": -10.493557929992676, "operator": "ge"}, {"budget": 0.59, "threshold": -10.56474494934082, "operator": "ge"}, {"budget": 0.6, "threshold": -10.663182258605957, "operator": "ge"}, {"budget": 0.61, "threshold": -10.71540355682373, "operator": "ge"}, {"budget": 0.62, "threshold": -10.783601760864258, "operator": "ge"}, {"budget": 0.63, "threshold": -10.83590030670166, "operator": "ge"}, {"budget": 0.64, "threshold": -10.880393028259277, "operator": "ge"}, {"budget": 0.65, "threshold": -10.935688018798828, "operator": "ge"}, {"budget": 0.66, "threshold": -11.005619049072266, "operator": "ge"}, {"budget": 0.67, "threshold": -11.043066024780273, "operator": "ge"}, {"budget": 0.68, "threshold": -11.095056533813477, "operator": "ge"}, {"budget": 0.69, "threshold": -11.124109268188477, "operator": "ge"}, {"budget": 0.7, "threshold": -11.219579696655273, "operator": "ge"}, {"budget": 0.71, "threshold": -11.280949592590332, "operator": "ge"}, {"budget": 0.72, "threshold": -11.342942237854004, "operator": "ge"}, {"budget": 0.73, "threshold": -11.409889221191406, "operator": "ge"}, {"budget": 0.74, "threshold": -11.453367233276367, "operator": "ge"}, {"budget": 0.75, "threshold": -11.532169342041016, "operator": "ge"}, {"budget": 0.76, "threshold": -11.581769943237305, "operator": "ge"}, {"budget": 0.77, "threshold": -11.607793807983398, "operator": "ge"}, {"budget": 0.78, "threshold": -11.667736053466797, "operator": "ge"}, {"budget": 0.79, "threshold": -11.730830192565918, "operator": "ge"}, {"budget": 0.8, "threshold": -11.789213180541992, "operator": "ge"}, {"budget": 0.81, "threshold": -11.837865829467773, "operator": "ge"}, {"budget": 0.82, "threshold": -11.899864196777344, "operator": "ge"}, {"budget": 0.83, "threshold": -11.966184616088867, "operator": "ge"}, {"budget": 0.84, "threshold": -11.997827529907227, "operator": "ge"}, {"budget": 0.85, "threshold": -12.034004211425781, "operator": "ge"}, {"budget": 0.86, "threshold": -12.08709716796875, "operator": "ge"}, {"budget": 0.87, "threshold": -12.135446548461914, "operator": "ge"}, {"budget": 0.88, "threshold": -12.191187858581543, "operator": "ge"}, {"budget": 0.89, "threshold": -12.298397064208984, "operator": "ge"}, {"budget": 0.9, "threshold": -12.354024887084961, "operator": "ge"}, {"budget": 0.91, "threshold": -12.429829597473145, "operator": "ge"}, {"budget": 0.92, "threshold": -12.483664512634277, "operator": "ge"}, {"budget": 0.93, "threshold": -12.549131393432617, "operator": "ge"}, {"budget": 0.94, "threshold": -12.59619140625, "operator": "ge"}, {"budget": 0.95, "threshold": -12.673208236694336, "operator": "ge"}, {"budget": 0.96, "threshold": -12.75936508178711, "operator": "ge"}, {"budget": 0.97, "threshold": -12.836074829101562, "operator": "ge"}, {"budget": 0.98, "threshold": -12.970306396484375, "operator": "ge"}, {"budget": 0.99, "threshold": -13.215304374694824, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.66666666666667, 97.91666666666666, 97.33333333333334, 96.91666666666666, 96.33333333333334, 95.75, 95.0, 94.25, 93.75, 93.08333333333333, 92.25, 91.58333333333333, 91.0, 90.33333333333333, 89.41666666666667, 88.66666666666667, 87.83333333333333, 87.16666666666667, 86.41666666666666, 85.83333333333334, 85.25, 84.83333333333334, 84.0, 83.41666666666667, 82.75, 82.41666666666667, 82.08333333333333, 81.58333333333333, 81.0, 80.91666666666667, 80.41666666666667, 80.08333333333333, 79.58333333333334, 79.0, 78.75, 78.5, 78.25, 78.0, 77.41666666666667, 77.08333333333334, 76.75, 76.25, 75.66666666666667, 75.08333333333333, 74.5, 74.16666666666667, 73.33333333333334, 72.91666666666667, 72.16666666666667, 71.33333333333334, 70.83333333333333, 70.0, 69.16666666666667, 68.33333333333333, 67.5, 66.75, 66.0, 65.41666666666667, 64.41666666666667, 63.583333333333336, 62.74999999999999, 61.916666666666664, 61.0, 60.25, 59.5, 58.666666666666664, 57.91666666666666, 57.166666666666664, 56.49999999999999, 55.75, 54.91666666666667, 53.91666666666667, 53.16666666666667, 52.25, 51.5, 50.583333333333336, 49.583333333333336, 48.66666666666667, 47.66666666666667, 46.666666666666664, 45.833333333333336, 44.99999999999999, 43.91666666666667, 42.91666666666667, 42.00000000000001, 41.083333333333336, 40.083333333333336, 39.16666666666667, 38.24999999999999, 37.33333333333333, 36.416666666666664, 35.5, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 3, 3, 3, 3, 3, 3, 3, 3, 4, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6], "harmed": [0, 2, 11, 18, 23, 30, 37, 46, 55, 62, 71, 81, 90, 97, 105, 116, 125, 135, 143, 152, 159, 166, 171, 181, 188, 196, 200, 204, 210, 217, 218, 224, 228, 234, 241, 244, 247, 250, 253, 260, 264, 268, 274, 281, 288, 295, 299, 309, 314, 323, 333, 339, 349, 359, 369, 379, 388, 397, 404, 416, 426, 436, 446, 457, 466, 475, 485, 494, 503, 511, 520, 530, 542, 551, 562, 571, 582, 594, 605, 617, 629, 639, 649, 662, 674, 685, 696, 708, 719, 730, 741, 752, 763, 774, 785, 797, 809, 821, 833, 844, 856], "exact_score": [98.58333333333333, 98.66666666666667, 97.91666666666666, 97.33333333333334, 96.91666666666666, 96.33333333333334, 95.75, 95.0, 94.25, 93.75, 93.08333333333333, 92.25, 91.58333333333333, 91.0, 90.33333333333333, 89.41666666666667, 88.66666666666667, 87.83333333333333, 87.16666666666667, 86.41666666666666, 85.83333333333334, 85.25, 84.83333333333334, 84.0, 83.41666666666667, 82.75, 82.41666666666667, 82.08333333333333, 81.58333333333333, 81.0, 80.91666666666667, 80.41666666666667, 80.08333333333333, 79.58333333333334, 79.0, 78.75, 78.5, 78.25, 78.0, 77.41666666666667, 77.08333333333334, 76.75, 76.25, 75.66666666666667, 75.08333333333333, 74.5, 74.16666666666667, 73.33333333333334, 72.91666666666667, 72.16666666666667, 71.33333333333334, 70.83333333333333, 70.0, 69.16666666666667, 68.33333333333333, 67.5, 66.75, 65.91666666666667, 65.41666666666667, 64.41666666666667, 63.583333333333336, 62.74999999999999, 61.916666666666664, 61.0, 60.25, 59.5, 58.666666666666664, 57.91666666666666, 57.166666666666664, 56.41666666666667, 55.75, 54.91666666666667, 53.91666666666667, 53.16666666666667, 52.25, 51.5, 50.583333333333336, 49.583333333333336, 48.66666666666667, 47.66666666666667, 46.666666666666664, 45.833333333333336, 44.91666666666667, 43.91666666666667, 42.91666666666667, 42.00000000000001, 41.083333333333336, 40.083333333333336, 39.16666666666667, 38.24999999999999, 37.33333333333333, 36.416666666666664, 35.5, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 67.51416666666668, "threshold_area100": 67.51666666666667, "threshold_area25": 91.06666666666668}, "test": {"score": [81.429064549534, 81.1356575768036, 79.77217811529168, 78.44321712115982, 77.18329306178806, 76.19951674145669, 74.5598895409044, 72.52329996548154, 71.0044874007594, 69.46841560234725, 68.0704176734553, 66.51708664135313, 65.15360717984122, 64.72212633759061, 63.47946151190887, 62.53020365895754, 61.563686572316186, 60.44183638246462, 59.83776320331378, 58.83672764929238, 57.57680358992061, 57.04176734552986, 56.33413876423887, 55.695547117707974, 55.229547808077314, 54.780807732136694, 54.24577148774594, 53.848809112875394, 53.572661373835004, 53.31377286848464, 53.072143596824304, 52.83051432516397, 52.6406627545737, 52.46807041767345, 52.019330341732825, 51.846738004832595, 51.69140490162236, 51.363479461511915, 50.94925785295133, 50.48325854332067, 50.24162927166034, 50.05177770107008, 49.689333793579564, 49.27511218501899, 49.119779081808765, 48.8608905764584, 48.619261304798066, 48.37763203313773, 47.963410424577155, 47.773558853986884, 47.51467034863652, 47.03141180531585, 46.59993096306524, 46.22022782188471, 45.80600621332413, 45.51259924059372, 44.96030376251295, 44.442526751812224, 44.26993441491198, 43.90749050742147, 43.32067656196065, 43.079047290300316, 42.682084915429755, 42.3541594753193, 42.00897480151882, 41.7846047635485, 41.23230928546773, 40.904383845357266, 40.559199171556784, 40.33482913358647, 39.62720055229548, 39.074905074214705, 38.59164653089403, 38.19468415602347, 37.76320331377287, 37.05557473248188, 36.675871591301345, 36.451501553331035, 35.91646530894029, 35.329651363479464, 34.65654124956853, 34.08698653779773, 33.65550569554712, 33.03417328270625, 32.77528477735589, 32.361063168795305, 31.96410079392475, 31.61891612012427, 31.17017604418364, 30.4280289955126, 29.910251984811875, 29.4269934414912, 29.133586468760786, 28.80566102865033, 28.512254055919918, 28.184328615809463, 27.9254401104591, 27.75284777355885, 27.37314463237832, 26.855367621677594, 26.561960648947192], "calls": [0.0, 3.1756989989644464, 9.061097687262686, 13.324128408698655, 17.5008629616845, 20.486710390058683, 24.318260269244046, 28.753883327580255, 32.41284086986538, 36.037279944770454, 38.729720400414216, 41.24956851915775, 43.64860200207111, 44.26993441491198, 46.16845012081463, 47.82533655505696, 49.309630652399036, 51.12185018985157, 51.98481187435278, 53.43458750431481, 55.43665861235761, 56.50673110113911, 57.611322057300654, 58.50880220918191, 59.250949257852945, 60.13117017604418, 61.16672419744563, 62.01242664825681, 62.616499827407665, 63.25509147393855, 63.910942354159474, 64.48049706593028, 65.01553331032102, 65.41249568519157, 66.25819813600276, 66.84501208146358, 67.41456679323437, 68.45012081463582, 69.45115636865722, 70.52122885743873, 71.0044874007594, 71.29789437348981, 71.98826372109079, 72.7821884708319, 73.17915084570245, 73.576113220573, 73.97307559544356, 74.38729720400414, 75.00862961684501, 75.40559199171557, 75.83707283396618, 76.45840524680703, 77.13151536071798, 77.61477390403866, 78.23610631687953, 78.61580945806006, 79.34069727304107, 79.996548153262, 80.25543665861235, 80.7904729030031, 81.46358301691406, 81.73973075595443, 82.18847083189506, 82.65447014152572, 83.08595098377633, 83.3620987228167, 84.03520883672765, 84.39765274421815, 84.86365205384881, 85.17431826026925, 86.0027614773904, 86.64135312392129, 87.14187090093199, 87.55609250949257, 88.02209181912323, 88.78149810148429, 89.17846047635484, 89.43734898170521, 89.98964445978599, 90.61097687262686, 91.3531239212979, 91.93993786675871, 92.38867794269935, 93.02726958923024, 93.37245426303072, 93.82119433897135, 94.25267518122196, 94.63237832240249, 95.15015533310321, 95.96133931653435, 96.47911632723508, 97.01415257162583, 97.30755954435622, 97.65274421815671, 97.94615119088712, 98.29133586468761, 98.55022437003798, 98.75733517431826, 99.1370383154988, 99.70659302726959, 100.0], "rescued": [0, 26, 56, 78, 97, 114, 128, 143, 164, 186, 197, 202, 211, 213, 215, 217, 218, 220, 221, 221, 221, 223, 224, 225, 225, 226, 226, 226, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 227, 228, 228, 228, 228, 228, 228, 228, 228, 229, 229, 229, 229, 229, 229, 229, 229, 229, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, 231, 231, 231, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232, 232], "harmed": [0, 43, 152, 251, 343, 417, 526, 659, 768, 879, 971, 1066, 1154, 1181, 1255, 1312, 1369, 1436, 1472, 1530, 1603, 1636, 1678, 1716, 1743, 1770, 1801, 1824, 1841, 1856, 1870, 1884, 1895, 1905, 1931, 1941, 1950, 1969, 1993, 2020, 2034, 2045, 2066, 2090, 2099, 2114, 2128, 2142, 2166, 2177, 2193, 2221, 2246, 2268, 2292, 2309, 2341, 2371, 2382, 2403, 2437, 2451, 2474, 2493, 2513, 2526, 2558, 2578, 2598, 2611, 2652, 2684, 2712, 2735, 2760, 2801, 2823, 2836, 2867, 2901, 2940, 2973, 2998, 3034, 3049, 3074, 3097, 3117, 3144, 3187, 3217, 3245, 3262, 3281, 3298, 3317, 3332, 3342, 3364, 3394, 3411], "exact_score": [81.429064549534, 81.39454608215395, 81.36002761477391, 81.11839834311357, 80.94580600621333, 80.7904729030031, 80.61788056610285, 80.39351052813255, 80.16914049016223, 79.80669658267173, 79.54780807732136, 79.11632723507076, 78.87469796341043, 78.58129099068002, 78.27062478425958, 77.90818087676908, 77.58025543665862, 77.30410769761824, 77.07973765964792, 76.76907145322748, 76.40662754573697, 75.9233690024163, 75.45736969278565, 75.0776665516051, 74.68070417673457, 74.26648256817397, 73.83500172592336, 73.38626164998274, 72.95478080773215, 72.45426303072145, 71.93648602002072, 71.52226441146013, 71.17707973765965, 70.71108042802899, 70.38315498791854, 69.93441491197791, 69.46841560234725, 69.05419399378667, 68.38108387987573, 67.93234380393511, 67.31101139109424, 66.68967897825337, 66.10286503279255, 65.51605108733173, 64.96375560925095, 64.27338626164997, 63.61753538142907, 62.927166033828094, 62.3576113220573, 61.75353814290645, 61.06316879530549, 60.54539178460476, 59.83776320331378, 59.18191232309285, 58.49154297549188, 57.852951328961, 57.26613738350017, 56.73110113910942, 56.05799102519848, 55.41939937866759, 54.81532619951675, 54.332067656196074, 53.848809112875394, 53.40006903693476, 53.03762512944425, 52.6406627545737, 52.140144977563, 51.81221953745254, 51.518812564722126, 51.08733172247153, 50.72488781498101, 50.25888850535036, 49.689333793579564, 49.22333448394892, 48.619261304798066, 47.980669658267175, 47.39385571280636, 46.65170866413531, 45.90956161546427, 45.25371073524336, 44.442526751812224, 43.734898170521234, 42.85467725233, 42.06075250258889, 41.2840869865378, 40.45564376941664, 39.644459785985504, 38.74697963410424, 37.79772178115292, 36.86572316189161, 35.91646530894029, 35.01898515705903, 34.035208836727655, 33.0686917500863, 32.18847083189507, 31.32550914739386, 30.393510528132552, 29.444252675181225, 28.460476354849852, 27.511218501898515, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 59.6967298845925, "threshold_area100": 47.852519848118746, "threshold_area25": 66.4963755609251}, "selected_budget": 1, "selected": {"val": {"score": 98.66666666666667, "calls": 1.0, "rescued": 3, "harmed": 2}, "test": {"score": 81.1356575768036, "calls": 3.1756989989644464, "rescued": 26, "harmed": 43}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.66666666666667, "calls": 1.0}, "test": {"score": 81.46358301691406, "calls": 1.6051087331722471}}, "old_normalized_interior": {"val": {"score": 98.66666666666667, "calls": 1.0}, "test": {"score": 81.46358301691406, "calls": 1.6051087331722471}}, "old_normalized_endpoints": {"val": {"score": 98.66666666666667, "calls": 1.0}, "test": {"score": 81.46358301691406, "calls": 1.6051087331722471}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.08333333333333, 97.91666666666666, 97.66666666666667, 97.5, 97.25, 97.08333333333333, 97.0, 96.83333333333334, 96.75, 96.5, 96.25, 96.16666666666667, 95.75, 95.16666666666667, 94.5, 93.91666666666667, 93.25, 92.75, 92.25, 91.58333333333333, 91.0, 90.25, 89.66666666666667, 88.75, 87.83333333333333, 87.0, 86.33333333333333, 85.66666666666667, 84.91666666666666, 83.91666666666666, 83.25, 82.16666666666667, 81.41666666666667, 80.58333333333334, 79.66666666666666, 79.0, 78.16666666666667, 77.16666666666667, 76.16666666666667, 75.41666666666667, 74.41666666666666, 73.41666666666666, 72.50000000000001, 71.50000000000001, 70.5, 69.75, 68.91666666666667, 68.0, 67.0, 66.0, 65.25000000000001, 64.33333333333333, 63.416666666666664, 62.583333333333336, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.75, 57.91666666666666, 56.833333333333336, 55.833333333333336, 54.833333333333336, 53.833333333333336, 52.83333333333333, 52.0, 51.0, 50.0, 49.0, 48.0, 47.0, 46.08333333333333, 45.333333333333336, 44.25, 43.25, 42.333333333333336, 41.333333333333336, 40.33333333333333, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.58333333333333, 29.58333333333334, 28.666666666666664, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.08333333333333, 97.91666666666666, 97.66666666666667, 97.5, 97.25, 97.08333333333333, 97.0, 96.83333333333334, 96.75, 96.5, 96.25, 96.16666666666667, 95.75, 95.16666666666667, 94.5, 93.91666666666667, 93.25, 92.75, 92.25, 91.58333333333333, 91.0, 90.25, 89.66666666666667, 88.75, 87.83333333333333, 87.0, 86.33333333333333, 85.66666666666667, 84.91666666666666, 83.91666666666666, 83.16666666666667, 82.16666666666667, 81.41666666666667, 80.58333333333334, 79.66666666666666, 79.0, 78.16666666666667, 77.16666666666667, 76.16666666666667, 75.41666666666667, 74.41666666666666, 73.41666666666666, 72.50000000000001, 71.50000000000001, 70.5, 69.75, 68.83333333333333, 68.0, 67.0, 66.0, 65.25000000000001, 64.33333333333333, 63.416666666666664, 62.583333333333336, 61.583333333333336, 60.583333333333336, 59.66666666666667, 58.75, 57.833333333333336, 56.833333333333336, 55.833333333333336, 54.833333333333336, 53.833333333333336, 52.83333333333333, 52.0, 51.0, 50.0, 49.0, 48.0, 47.0, 46.08333333333333, 45.25, 44.25, 43.25, 42.333333333333336, 41.333333333333336, 40.33333333333333, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.58333333333333, 29.58333333333334, 28.666666666666664, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.49166666666667, "exact_area100": 71.7}, "test": {"score": [81.429064549534, 81.46358301691406, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.46358301691406, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.34276838108389, 81.30824991370383, 81.23921297894374, 81.17017604418363, 81.08387987573352, 80.94580600621333, 80.80773213669313, 80.63513979979288, 80.34173282706249, 79.996548153262, 79.7894373489817, 79.47877114256127, 78.9264756644805, 78.33966171901969, 77.89092164307905, 77.25232999654816, 76.30307214359682, 75.64722126337591, 74.85329651363479, 74.12840869865379, 73.43803935105282, 72.85122540559199, 71.9192267863307, 70.86641353123922, 70.33137728684846, 69.31308249913704, 68.69175008629618, 67.67345529858476, 67.34552985847428, 66.4998274076631, 65.30894028305143, 64.35968243010011, 63.4276838108388, 62.927166033828094, 61.75353814290645, 60.890576458405256, 60.26924404556438, 59.49257852951328, 58.59509837763204, 58.025543665861235, 57.455988954090444, 56.57576803589921, 55.902657921988265, 55.26406627545737, 53.60717984121506, 52.86503279254402, 52.39903348291336, 52.08836727649293, 51.31170176044184, 50.44874007594062, 49.98274076630998, 49.20607525025889, 48.3603727994477, 47.67000345184674, 46.68622713151536, 46.09941318605454, 45.61615464273386, 44.89126682775285, 44.20089748015189, 43.63134276838109, 43.06178805661029, 42.57852951328962, 41.974456334138765, 41.69830859509838, 41.07697618225751, 40.283051432516395, 39.76527442181567, 39.02312737314463, 38.280980324473596, 37.31446323783224, 37.021056265101834, 36.382464618570935, 36.05453917846047, 35.450465999309635, 34.79461511908871, 34.31135657576804, 33.690024162927166, 33.31032102174664, 32.654470141525714, 32.10217466344494, 31.532619951674146, 30.600621332412846, 30.35899206075251, 29.754918881601654, 28.9264756644805, 28.391439420089746, 27.666551605108737, 27.442181567138423, 27.114256127027957, 26.561960648947192], "calls": [0.0, 1.6051087331722471, 2.312737314463238, 2.7959958577839146, 3.3137728684846395, 3.8833275802554366, 4.798066965826718, 6.869175008629617, 7.8529513289609945, 8.733172247152227, 9.475319295823265, 10.217466344494303, 11.460131170176044, 12.909906800138074, 13.962720055229546, 15.4987918536417, 16.82775284777356, 17.949603037625128, 18.77804625474629, 20.417673455298583, 21.539523645150155, 23.38626164998274, 24.456334138764237, 25.975146703486363, 27.52847773558854, 28.89195719710045, 30.72143596824301, 32.861580945806004, 34.41491197790818, 36.27890921643079, 37.866758715913015, 39.299275112185015, 40.68001380738695, 42.474974111149464, 44.33897134967207, 45.4263030721436, 47.10044874007594, 48.25681739730756, 49.82740766309976, 50.46599930963065, 51.794960303762515, 53.727994477045215, 55.125992405937176, 56.351397997928885, 57.179841215050054, 58.87124611667242, 60.303762512944424, 61.58094580600621, 62.633759061097685, 63.84190541939938, 64.77390403866067, 65.51605108733172, 66.65516051087332, 67.44908526061442, 68.3638246461857, 70.22782188470832, 71.03900586813945, 71.74663444943045, 72.16085605799103, 72.97204004142216, 73.9212978943735, 74.5598895409044, 75.45736969278563, 76.40662754573697, 77.25232999654816, 78.32240248532966, 78.9955125992406, 79.58232654470142, 80.4280289955126, 81.18743527787366, 81.80876769071453, 82.4301001035554, 82.98239558163617, 83.62098722816707, 83.9316534345875, 84.67380048325855, 85.51950293406972, 86.0717984121505, 86.86572316189162, 87.64238867794269, 88.67794269934414, 89.00586813945462, 89.6444597859855, 90.02416292716603, 90.66275457369693, 91.37038315498792, 91.85364169830859, 92.47497411114946, 92.88919571971005, 93.61408353469106, 94.2181567138419, 94.80497065930273, 95.8577839143942, 96.11667241974456, 96.78978253365551, 97.61822575077666, 98.15326199516741, 98.89540904383846, 99.11977908180877, 99.44770452191922, 100.0], "exact_score": [81.429064549534, 81.39454608215395, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.429064549534, 81.46358301691406, 81.41180531584398, 81.41180531584398, 81.36002761477391, 81.34276838108389, 81.27373144632378, 81.20469451156369, 81.17017604418363, 81.10113910942354, 81.03210217466345, 80.9285467725233, 80.82499137038316, 80.63513979979288, 80.37625129444253, 80.20365895754229, 79.87573351743183, 79.77217811529168, 79.65136347946151, 79.25440110459095, 78.90921643079048, 78.59855022437004, 78.14981014842941, 77.90818087676908, 77.56299620296859, 77.18329306178806, 76.73455298584743, 76.23403520883673, 75.7852951328961, 75.35381429064549, 75.00862961684501, 74.59440800828443, 74.11114946496376, 73.67966862271315, 73.12737314463237, 72.67863306869175, 72.17811529168105, 71.59130134622022, 71.09078356920952, 70.59026579219882, 69.96893337935795, 69.36486020020712, 68.91612012426648, 68.2084915429755, 67.60441836382465, 66.96582671729375, 66.39627200552296, 65.70590265792198, 65.11908871246118, 64.41146013117019, 63.772868484639275, 63.099758370728345, 62.40938902312738, 61.66724197445633, 61.04590956161546, 60.424577148774596, 59.95857783914394, 59.31998619261305, 58.526061442871935, 57.818432861580945, 57.11080428028996, 56.24784259578875, 55.5056955471177, 54.729030031066614, 53.81429064549535, 52.916810493614086, 52.19192267863306, 51.294442526751816, 50.396962374870554, 49.620296858819465, 48.757335174318264, 47.84259578874698, 46.996893337935795, 46.09941318605454, 45.288229202623405, 44.39074905074215, 43.441491197790825, 42.56127027959959, 41.681049361408355, 40.7835692095271, 39.83431135657577, 38.90231273731447, 37.97031411805316, 37.03831549879186, 36.089057645840526, 35.157059026579226, 34.173282706247846, 33.24128408698654, 32.29202623403521, 31.39454608215395, 30.46254746289265, 29.54780807732137, 28.546772523299968, 27.56299620296859, 26.561960648947192], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 80.71971004487402, "exact_area100": 63.12111942782035}}, "old_normalized_endpoint_budget": 1}]}, {"expert": "cub", "endpoint": "Qwen3.8", "task": "cub", "small": 81.429064549534, "large": 54.81532619951675, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.0, "validation_area100": 0.8633916666666667, "training_seconds": 4.272997326916084, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.001442005392163992, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0018331073224544525, "operator": "ge"}, {"budget": 0.03, "threshold": -0.0025276984088122845, "operator": "ge"}, {"budget": 0.04, "threshold": -0.004120568744838238, "operator": "ge"}, {"budget": 0.05, "threshold": -0.005380135029554367, "operator": "ge"}, {"budget": 0.06, "threshold": -0.006801027338951826, "operator": "ge"}, {"budget": 0.07, "threshold": -0.00811977032572031, "operator": "ge"}, {"budget": 0.08, "threshold": -0.010691252537071705, "operator": "ge"}, {"budget": 0.09, "threshold": -0.012549041770398617, "operator": "ge"}, {"budget": 0.1, "threshold": -0.015795215964317322, "operator": "ge"}, {"budget": 0.11, "threshold": -0.017901193350553513, "operator": "ge"}, {"budget": 0.12, "threshold": -0.021294912323355675, "operator": "ge"}, {"budget": 0.13, "threshold": -0.026422681286931038, "operator": "ge"}, {"budget": 0.14, "threshold": -0.030373845249414444, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03392352536320686, "operator": "ge"}, {"budget": 0.16, "threshold": -0.036727745085954666, "operator": "ge"}, {"budget": 0.17, "threshold": -0.04092683270573616, "operator": "ge"}, {"budget": 0.18, "threshold": -0.04594634473323822, "operator": "ge"}, {"budget": 0.19, "threshold": -0.05233848839998245, "operator": "ge"}, {"budget": 0.2, "threshold": -0.05677293613553047, "operator": "ge"}, {"budget": 0.21, "threshold": -0.062040381133556366, "operator": "ge"}, {"budget": 0.22, "threshold": -0.06846699118614197, "operator": "ge"}, {"budget": 0.23, "threshold": -0.07250664383172989, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0813540667295456, "operator": "ge"}, {"budget": 0.25, "threshold": -0.08756022900342941, "operator": "ge"}, {"budget": 0.26, "threshold": -0.0955556184053421, "operator": "ge"}, {"budget": 0.27, "threshold": -0.10690999776124954, "operator": "ge"}, {"budget": 0.28, "threshold": -0.11175300925970078, "operator": "ge"}, {"budget": 0.29, "threshold": -0.12129933387041092, "operator": "ge"}, {"budget": 0.3, "threshold": -0.12914888560771942, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1391749083995819, "operator": "ge"}, {"budget": 0.32, "threshold": -0.15003623068332672, "operator": "ge"}, {"budget": 0.33, "threshold": -0.15892459452152252, "operator": "ge"}, {"budget": 0.34, "threshold": -0.17262552678585052, "operator": "ge"}, {"budget": 0.35, "threshold": -0.18669922649860382, "operator": "ge"}, {"budget": 0.36, "threshold": -0.19722527265548706, "operator": "ge"}, {"budget": 0.37, "threshold": -0.20567519962787628, "operator": "ge"}, {"budget": 0.38, "threshold": -0.21813517808914185, "operator": "ge"}, {"budget": 0.39, "threshold": -0.22922444343566895, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2428533434867859, "operator": "ge"}, {"budget": 0.41, "threshold": -0.25008919835090637, "operator": "ge"}, {"budget": 0.42, "threshold": -0.2594490349292755, "operator": "ge"}, {"budget": 0.43, "threshold": -0.2762831151485443, "operator": "ge"}, {"budget": 0.44, "threshold": -0.287083238363266, "operator": "ge"}, {"budget": 0.45, "threshold": -0.30573877692222595, "operator": "ge"}, {"budget": 0.46, "threshold": -0.32592329382896423, "operator": "ge"}, {"budget": 0.47, "threshold": -0.3333820104598999, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3508731722831726, "operator": "ge"}, {"budget": 0.49, "threshold": -0.36625584959983826, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3771088421344757, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3867522180080414, "operator": "ge"}, {"budget": 0.52, "threshold": -0.4169638454914093, "operator": "ge"}, {"budget": 0.53, "threshold": -0.43657341599464417, "operator": "ge"}, {"budget": 0.54, "threshold": -0.45045533776283264, "operator": "ge"}, {"budget": 0.55, "threshold": -0.4649678170681, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4779316186904907, "operator": "ge"}, {"budget": 0.57, "threshold": -0.4899302124977112, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5047887563705444, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5233995914459229, "operator": "ge"}, {"budget": 0.6, "threshold": -0.531293511390686, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5434707403182983, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5631458759307861, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5792310237884521, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6037454605102539, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6207953691482544, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6340857744216919, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6530932188034058, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6717094779014587, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6887598633766174, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7049583792686462, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7324817180633545, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7473524212837219, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7701643109321594, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7891415953636169, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8089513778686523, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8231191635131836, "operator": "ge"}, {"budget": 0.77, "threshold": -0.830375075340271, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8397074937820435, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8493979573249817, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8598932027816772, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8691785335540771, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8768173456192017, "operator": "ge"}, {"budget": 0.83, "threshold": -0.887729823589325, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8969012498855591, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9092473387718201, "operator": "ge"}, {"budget": 0.86, "threshold": -0.918154776096344, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9290921092033386, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9353194236755371, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9421746730804443, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9493097066879272, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9571669697761536, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9618374109268188, "operator": "ge"}, {"budget": 0.93, "threshold": -0.969157338142395, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9749355316162109, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9790472388267517, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9827449917793274, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9867113828659058, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9905815720558167, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9936653971672058, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.33333333333334, 98.08333333333333, 98.08333333333333, 98.0, 97.83333333333334, 97.58333333333333, 97.58333333333333, 97.58333333333333, 97.33333333333334, 97.16666666666667, 97.08333333333333, 97.0, 96.75, 96.75, 96.58333333333333, 96.5, 96.25, 96.08333333333333, 96.0, 95.83333333333334, 95.75, 95.58333333333333, 95.33333333333334, 95.16666666666667, 94.83333333333334, 94.33333333333334, 94.25, 93.58333333333333, 93.41666666666667, 93.25, 92.91666666666667, 92.41666666666667, 92.0, 91.58333333333333, 90.83333333333333, 90.41666666666667, 90.0, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.66666666666667, 87.25, 86.75, 86.5, 86.0, 85.41666666666667, 84.58333333333333, 84.25, 83.66666666666667, 83.0, 82.33333333333334, 81.66666666666667, 81.0, 80.08333333333333, 79.33333333333333, 78.58333333333334, 77.75, 76.91666666666667, 76.08333333333334, 75.33333333333333, 74.58333333333333, 73.66666666666667, 72.91666666666667, 72.08333333333333, 71.16666666666667, 70.33333333333334, 69.41666666666666, 68.5, 67.75, 67.16666666666666, 66.33333333333333, 65.5, 64.58333333333333, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.00000000000001, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 3, 6, 6, 7, 9, 12, 12, 13, 16, 18, 19, 20, 23, 23, 25, 26, 29, 31, 32, 34, 35, 37, 40, 42, 46, 52, 53, 61, 64, 66, 70, 76, 81, 86, 95, 100, 105, 113, 119, 125, 133, 138, 144, 147, 153, 160, 170, 174, 181, 189, 197, 205, 213, 224, 233, 242, 252, 262, 272, 281, 290, 301, 310, 320, 331, 341, 352, 363, 372, 379, 389, 399, 410, 420, 431, 443, 455, 467, 478, 490, 501, 513, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.33333333333334, 98.08333333333333, 98.08333333333333, 98.0, 97.83333333333334, 97.58333333333333, 97.58333333333333, 97.58333333333333, 97.33333333333334, 97.16666666666667, 97.08333333333333, 97.0, 96.75, 96.75, 96.58333333333333, 96.5, 96.25, 96.08333333333333, 96.0, 95.83333333333334, 95.75, 95.58333333333333, 95.33333333333334, 95.16666666666667, 94.83333333333334, 94.33333333333334, 94.25, 93.58333333333333, 93.41666666666667, 93.25, 92.91666666666667, 92.41666666666667, 92.0, 91.58333333333333, 90.83333333333333, 90.41666666666667, 90.0, 89.33333333333333, 88.83333333333333, 88.25, 87.66666666666667, 87.25, 86.75, 86.5, 86.0, 85.41666666666667, 84.58333333333333, 84.25, 83.66666666666667, 83.0, 82.33333333333334, 81.58333333333333, 81.0, 80.08333333333333, 79.33333333333333, 78.58333333333334, 77.75, 76.91666666666667, 76.08333333333334, 75.33333333333333, 74.58333333333333, 73.66666666666667, 72.91666666666667, 72.08333333333333, 71.08333333333333, 70.33333333333334, 69.41666666666666, 68.5, 67.75, 67.16666666666666, 66.33333333333333, 65.5, 64.58333333333333, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.00000000000001, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.33916666666667, "threshold_area100": 86.34166666666665, "threshold_area25": 98.36666666666667}, "test": {"score": [81.429064549534, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.39454608215395, 81.27373144632378, 81.2909906800138, 81.30824991370383, 81.2909906800138, 81.2909906800138, 81.25647221263375, 81.22195374525371, 81.20469451156369, 81.20469451156369, 81.15291681049361, 81.08387987573352, 81.01484294097342, 80.9975837072834, 81.01484294097342, 80.96306523990336, 80.87676907145323, 80.75595443562305, 80.70417673455299, 80.58336209872282, 80.46254746289266, 80.27269589230238, 80.08284432171212, 79.97928891957199, 79.96202968588196, 79.77217811529168, 79.73765964791163, 79.40973420780118, 79.34069727304109, 79.25440110459095, 79.11632723507076, 78.96099413186055, 78.70210562651019, 78.52951328960994, 78.39143942008975, 78.18432861580946, 77.97721781152917, 77.80462547462893, 77.856403175699, 77.58025543665862, 77.18329306178806, 77.06247842595789, 76.88988608905764, 76.51018294787711, 76.28581290990681, 75.99240593717639, 75.71625819813602, 75.57818432861582, 75.35381429064549, 75.04314808422507, 74.80151881256472, 74.31826026924405, 73.85226095961339, 73.38626164998274, 72.97204004142216, 72.36796686227132, 71.85018985157059, 71.38419054193994, 70.60752502588885, 70.17604418363825, 69.29582326544703, 68.53641698308596, 67.79426993441491, 67.10390058681395, 66.67241974456334, 66.17190196755264, 65.80945806006213, 65.34345875043148, 64.9292371418709, 64.46323783224025, 63.80738695201933, 63.41042457714878, 62.478425957887474, 61.977908180876774, 61.37383500172593, 60.976872626855375, 60.45909561615465, 59.993096306523995, 59.04383845357266, 58.715913013462206, 58.249913703831545, 57.66309975837073, 57.21435968243011, 56.66206420434933, 56.316879530548846, 55.64376941663791, 55.17777010700725, 54.81532619951675], "calls": [0.0, 0.9492578529513289, 1.501553331032102, 2.347255781843286, 3.538142906454953, 4.22851225405592, 5.022437003797031, 5.540214014497756, 6.610286503279254, 7.145322747670003, 8.180876769071453, 8.802209181912323, 9.699689333793579, 10.8560579910252, 11.64998274076631, 12.305833620987228, 12.668277528477736, 13.324128408698655, 14.463237832240248, 15.79219882637211, 16.53434587504315, 17.604418363824646, 18.588194684156022, 19.140490162236798, 20.434932688988606, 21.26337590610977, 22.298929927511217, 23.66240938902313, 24.421815671384188, 25.578184328615812, 26.630997583707284, 27.94269934414912, 29.237141870900935, 30.376251294442525, 31.687953054884368, 33.0686917500863, 34.31135657576804, 35.29513289609941, 36.79668622713152, 38.004832585433206, 39.62720055229548, 40.490162236796685, 41.335864687607874, 43.165343458750435, 44.44252675181222, 46.496375560925095, 48.30859509837763, 48.94718674490853, 50.673110113910944, 52.08836727649292, 53.05488436313428, 53.96962374870555, 56.2133241284087, 57.66309975837073, 58.664135312392126, 59.63065239903348, 60.8560579910252, 61.90887124611667, 62.94442526751812, 64.10079392474974, 64.68760787021056, 65.82671729375215, 66.96582671729375, 68.08767690714532, 69.6410079392475, 70.88367276492924, 71.7811529168105, 73.00655850880221, 74.2147048671039, 75.2847773558854, 76.2167759751467, 77.77010700724888, 78.75388332758025, 80.13462202278218, 81.30824991370383, 82.6372109078357, 83.8626164998274, 84.38039351052812, 85.05350362443907, 85.83016914049017, 86.7103900586814, 87.41801863997239, 88.16016568864342, 89.23023817742492, 89.76527442181566, 90.93890231273731, 91.69830859509838, 92.54401104590956, 93.09630652399034, 93.70037970314118, 94.25267518122196, 95.28822920262341, 95.70245081118398, 96.18570935450465, 96.96237487055575, 97.51467034863653, 98.08422506040732, 98.44666896789782, 99.1370383154988, 99.62029685881947, 100.0], "rescued": [0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 4, 4, 5, 7, 8, 9, 13, 13, 15, 17, 18, 18, 21, 21, 23, 27, 29, 32, 36, 43, 48, 52, 54, 56, 64, 68, 76, 86, 90, 95, 104, 116, 120, 135, 146, 148, 160, 165, 169, 179, 198, 202, 210, 217, 223, 231, 237, 248, 251, 259, 268, 277, 284, 287, 290, 295, 299, 303, 308, 316, 322, 325, 327, 330, 334, 334, 335, 340, 344, 348, 350, 351, 352, 355, 359, 360, 362, 363, 363, 363, 363, 363, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 1, 1, 1, 1, 1, 2, 3, 4, 5, 5, 6, 6, 6, 14, 15, 15, 17, 21, 23, 27, 30, 31, 34, 41, 45, 48, 51, 56, 64, 75, 85, 97, 108, 121, 134, 148, 153, 172, 184, 207, 216, 230, 250, 263, 293, 314, 324, 348, 365, 379, 386, 421, 448, 463, 480, 508, 529, 552, 579, 590, 611, 638, 661, 696, 726, 756, 785, 824, 858, 890, 943, 974, 1028, 1074, 1120, 1164, 1189, 1219, 1245, 1276, 1304, 1333, 1372, 1396, 1453, 1486, 1522, 1547, 1578, 1605, 1660, 1679, 1706, 1742, 1768, 1800, 1820, 1859, 1886, 1907], "exact_score": [81.429064549534, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.27373144632378, 81.30824991370383, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.22195374525371, 81.18743527787366, 81.11839834311357, 81.03210217466345, 81.03210217466345, 81.01484294097342, 80.96306523990336, 80.89402830514325, 80.87676907145323, 80.66965826717293, 80.70417673455299, 80.61788056610285, 80.53158439765275, 80.35899206075251, 80.27269589230238, 80.11736278909216, 80.03106662064205, 79.94477045219193, 79.89299275112185, 79.70314118053159, 79.72040041422161, 79.53054884363134, 79.39247497411115, 79.28891957197101, 79.28891957197101, 79.15084570245081, 78.9955125992406, 78.87469796341043, 78.73662409389023, 78.56403175698999, 78.54677252329996, 78.37418018639973, 78.30514325163963, 78.13255091473938, 77.99447704521918, 77.82188470831896, 77.856403175699, 77.68381083879876, 77.58025543665862, 77.3558853986883, 77.16603382809804, 77.06247842595789, 76.78633068691751, 76.49292371418709, 76.30307214359682, 75.99240593717639, 75.71625819813602, 75.52640662754574, 75.37107352433551, 75.02588885053504, 74.80151881256472, 74.59440800828443, 74.2147048671039, 73.7832240248533, 73.31722471522265, 72.98929927511219, 72.50604073179151, 72.02278218847084, 71.43596824301002, 70.90093199861927, 70.48671039005868, 70.05522954780807, 69.38211943389715, 68.69175008629618, 68.12219537452538, 67.62167759751468, 67.01760441836383, 66.24093890231273, 65.68864342423197, 65.17086641353124, 64.54953400069037, 63.9972385226096, 63.22057300655851, 62.44390749050742, 61.71901967552641, 61.04590956161546, 60.200207110804286, 59.31998619261305, 58.439765274421816, 57.62858129099069, 56.748360372799446, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.30036696574331, "threshold_area100": 74.11011391094236, "threshold_area25": 81.31515360717985}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.001442005392163992, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0018331073224544525, "operator": "ge"}, {"budget": 0.03, "threshold": -0.0025276984088122845, "operator": "ge"}, {"budget": 0.04, "threshold": -0.004120568744838238, "operator": "ge"}, {"budget": 0.05, "threshold": -0.005380135029554367, "operator": "ge"}, {"budget": 0.06, "threshold": -0.006801027338951826, "operator": "ge"}, {"budget": 0.07, "threshold": -0.00811977032572031, "operator": "ge"}, {"budget": 0.08, "threshold": -0.010691252537071705, "operator": "ge"}, {"budget": 0.09, "threshold": -0.012549041770398617, "operator": "ge"}, {"budget": 0.1, "threshold": -0.015795215964317322, "operator": "ge"}, {"budget": 0.11, "threshold": -0.017901193350553513, "operator": "ge"}, {"budget": 0.12, "threshold": -0.021294912323355675, "operator": "ge"}, {"budget": 0.13, "threshold": -0.026422681286931038, "operator": "ge"}, {"budget": 0.14, "threshold": -0.030373845249414444, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03392352536320686, "operator": "ge"}, {"budget": 0.16, "threshold": -0.036727745085954666, "operator": "ge"}, {"budget": 0.17, "threshold": -0.04092683270573616, "operator": "ge"}, {"budget": 0.18, "threshold": -0.04594634473323822, "operator": "ge"}, {"budget": 0.19, "threshold": -0.05233848839998245, "operator": "ge"}, {"budget": 0.2, "threshold": -0.05677293613553047, "operator": "ge"}, {"budget": 0.21, "threshold": -0.062040381133556366, "operator": "ge"}, {"budget": 0.22, "threshold": -0.06846699118614197, "operator": "ge"}, {"budget": 0.23, "threshold": -0.07250664383172989, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0813540667295456, "operator": "ge"}, {"budget": 0.25, "threshold": -0.08756022900342941, "operator": "ge"}, {"budget": 0.26, "threshold": -0.0955556184053421, "operator": "ge"}, {"budget": 0.27, "threshold": -0.10690999776124954, "operator": "ge"}, {"budget": 0.28, "threshold": -0.11175300925970078, "operator": "ge"}, {"budget": 0.29, "threshold": -0.12129933387041092, "operator": "ge"}, {"budget": 0.3, "threshold": -0.12914888560771942, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1391749083995819, "operator": "ge"}, {"budget": 0.32, "threshold": -0.15003623068332672, "operator": "ge"}, {"budget": 0.33, "threshold": -0.15892459452152252, "operator": "ge"}, {"budget": 0.34, "threshold": -0.17262552678585052, "operator": "ge"}, {"budget": 0.35, "threshold": -0.18669922649860382, "operator": "ge"}, {"budget": 0.36, "threshold": -0.19722527265548706, "operator": "ge"}, {"budget": 0.37, "threshold": -0.20567519962787628, "operator": "ge"}, {"budget": 0.38, "threshold": -0.21813517808914185, "operator": "ge"}, {"budget": 0.39, "threshold": -0.22922444343566895, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2428533434867859, "operator": "ge"}, {"budget": 0.41, "threshold": -0.25008919835090637, "operator": "ge"}, {"budget": 0.42, "threshold": -0.2594490349292755, "operator": "ge"}, {"budget": 0.43, "threshold": -0.2762831151485443, "operator": "ge"}, {"budget": 0.44, "threshold": -0.287083238363266, "operator": "ge"}, {"budget": 0.45, "threshold": -0.30573877692222595, "operator": "ge"}, {"budget": 0.46, "threshold": -0.32592329382896423, "operator": "ge"}, {"budget": 0.47, "threshold": -0.3333820104598999, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3508731722831726, "operator": "ge"}, {"budget": 0.49, "threshold": -0.36625584959983826, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3771088421344757, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3867522180080414, "operator": "ge"}, {"budget": 0.52, "threshold": -0.4169638454914093, "operator": "ge"}, {"budget": 0.53, "threshold": -0.43657341599464417, "operator": "ge"}, {"budget": 0.54, "threshold": -0.45045533776283264, "operator": "ge"}, {"budget": 0.55, "threshold": -0.4649678170681, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4779316186904907, "operator": "ge"}, {"budget": 0.57, "threshold": -0.4899302124977112, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5047887563705444, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5233995914459229, "operator": "ge"}, {"budget": 0.6, "threshold": -0.531293511390686, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5434707403182983, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5631458759307861, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5792310237884521, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6037454605102539, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6207953691482544, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6340857744216919, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6530932188034058, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6717094779014587, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6887598633766174, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7049583792686462, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7324817180633545, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7473524212837219, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7701643109321594, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7891415953636169, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8089513778686523, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8231191635131836, "operator": "ge"}, {"budget": 0.77, "threshold": -0.830375075340271, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8397074937820435, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8493979573249817, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8598932027816772, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8691785335540771, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8768173456192017, "operator": "ge"}, {"budget": 0.83, "threshold": -0.887729823589325, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8969012498855591, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9092473387718201, "operator": "ge"}, {"budget": 0.86, "threshold": -0.918154776096344, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9290921092033386, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9353194236755371, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9421746730804443, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9493097066879272, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9571669697761536, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9618374109268188, "operator": "ge"}, {"budget": 0.93, "threshold": -0.969157338142395, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9749355316162109, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9790472388267517, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9827449917793274, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9867113828659058, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9905815720558167, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9936653971672058, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.33333333333334, 98.08333333333333, 98.08333333333333, 98.0, 97.83333333333334, 97.58333333333333, 97.58333333333333, 97.58333333333333, 97.33333333333334, 97.16666666666667, 97.08333333333333, 97.0, 96.75, 96.75, 96.58333333333333, 96.5, 96.25, 96.08333333333333, 96.0, 95.83333333333334, 95.75, 95.58333333333333, 95.33333333333334, 95.16666666666667, 94.83333333333334, 94.33333333333334, 94.25, 93.58333333333333, 93.41666666666667, 93.25, 92.91666666666667, 92.41666666666667, 92.0, 91.58333333333333, 90.83333333333333, 90.41666666666667, 90.0, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.66666666666667, 87.25, 86.75, 86.5, 86.0, 85.41666666666667, 84.58333333333333, 84.25, 83.66666666666667, 83.0, 82.33333333333334, 81.66666666666667, 81.0, 80.08333333333333, 79.33333333333333, 78.58333333333334, 77.75, 76.91666666666667, 76.08333333333334, 75.33333333333333, 74.58333333333333, 73.66666666666667, 72.91666666666667, 72.08333333333333, 71.16666666666667, 70.33333333333334, 69.41666666666666, 68.5, 67.75, 67.16666666666666, 66.33333333333333, 65.5, 64.58333333333333, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.00000000000001, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 3, 6, 6, 7, 9, 12, 12, 13, 16, 18, 19, 20, 23, 23, 25, 26, 29, 31, 32, 34, 35, 37, 40, 42, 46, 52, 53, 61, 64, 66, 70, 76, 81, 86, 95, 100, 105, 113, 119, 125, 133, 138, 144, 147, 153, 160, 170, 174, 181, 189, 197, 205, 213, 224, 233, 242, 252, 262, 272, 281, 290, 301, 310, 320, 331, 341, 352, 363, 372, 379, 389, 399, 410, 420, 431, 443, 455, 467, 478, 490, 501, 513, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.33333333333334, 98.08333333333333, 98.08333333333333, 98.0, 97.83333333333334, 97.58333333333333, 97.58333333333333, 97.58333333333333, 97.33333333333334, 97.16666666666667, 97.08333333333333, 97.0, 96.75, 96.75, 96.58333333333333, 96.5, 96.25, 96.08333333333333, 96.0, 95.83333333333334, 95.75, 95.58333333333333, 95.33333333333334, 95.16666666666667, 94.83333333333334, 94.33333333333334, 94.25, 93.58333333333333, 93.41666666666667, 93.25, 92.91666666666667, 92.41666666666667, 92.0, 91.58333333333333, 90.83333333333333, 90.41666666666667, 90.0, 89.33333333333333, 88.83333333333333, 88.25, 87.66666666666667, 87.25, 86.75, 86.5, 86.0, 85.41666666666667, 84.58333333333333, 84.25, 83.66666666666667, 83.0, 82.33333333333334, 81.58333333333333, 81.0, 80.08333333333333, 79.33333333333333, 78.58333333333334, 77.75, 76.91666666666667, 76.08333333333334, 75.33333333333333, 74.58333333333333, 73.66666666666667, 72.91666666666667, 72.08333333333333, 71.08333333333333, 70.33333333333334, 69.41666666666666, 68.5, 67.75, 67.16666666666666, 66.33333333333333, 65.5, 64.58333333333333, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.00000000000001, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.33916666666667, "threshold_area100": 86.34166666666665, "threshold_area25": 98.36666666666667}, "test": {"score": [81.429064549534, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.39454608215395, 81.27373144632378, 81.2909906800138, 81.30824991370383, 81.2909906800138, 81.2909906800138, 81.25647221263375, 81.22195374525371, 81.20469451156369, 81.20469451156369, 81.15291681049361, 81.08387987573352, 81.01484294097342, 80.9975837072834, 81.01484294097342, 80.96306523990336, 80.87676907145323, 80.75595443562305, 80.70417673455299, 80.58336209872282, 80.46254746289266, 80.27269589230238, 80.08284432171212, 79.97928891957199, 79.96202968588196, 79.77217811529168, 79.73765964791163, 79.40973420780118, 79.34069727304109, 79.25440110459095, 79.11632723507076, 78.96099413186055, 78.70210562651019, 78.52951328960994, 78.39143942008975, 78.18432861580946, 77.97721781152917, 77.80462547462893, 77.856403175699, 77.58025543665862, 77.18329306178806, 77.06247842595789, 76.88988608905764, 76.51018294787711, 76.28581290990681, 75.99240593717639, 75.71625819813602, 75.57818432861582, 75.35381429064549, 75.04314808422507, 74.80151881256472, 74.31826026924405, 73.85226095961339, 73.38626164998274, 72.97204004142216, 72.36796686227132, 71.85018985157059, 71.38419054193994, 70.60752502588885, 70.17604418363825, 69.29582326544703, 68.53641698308596, 67.79426993441491, 67.10390058681395, 66.67241974456334, 66.17190196755264, 65.80945806006213, 65.34345875043148, 64.9292371418709, 64.46323783224025, 63.80738695201933, 63.41042457714878, 62.478425957887474, 61.977908180876774, 61.37383500172593, 60.976872626855375, 60.45909561615465, 59.993096306523995, 59.04383845357266, 58.715913013462206, 58.249913703831545, 57.66309975837073, 57.21435968243011, 56.66206420434933, 56.316879530548846, 55.64376941663791, 55.17777010700725, 54.81532619951675], "calls": [0.0, 0.9492578529513289, 1.501553331032102, 2.347255781843286, 3.538142906454953, 4.22851225405592, 5.022437003797031, 5.540214014497756, 6.610286503279254, 7.145322747670003, 8.180876769071453, 8.802209181912323, 9.699689333793579, 10.8560579910252, 11.64998274076631, 12.305833620987228, 12.668277528477736, 13.324128408698655, 14.463237832240248, 15.79219882637211, 16.53434587504315, 17.604418363824646, 18.588194684156022, 19.140490162236798, 20.434932688988606, 21.26337590610977, 22.298929927511217, 23.66240938902313, 24.421815671384188, 25.578184328615812, 26.630997583707284, 27.94269934414912, 29.237141870900935, 30.376251294442525, 31.687953054884368, 33.0686917500863, 34.31135657576804, 35.29513289609941, 36.79668622713152, 38.004832585433206, 39.62720055229548, 40.490162236796685, 41.335864687607874, 43.165343458750435, 44.44252675181222, 46.496375560925095, 48.30859509837763, 48.94718674490853, 50.673110113910944, 52.08836727649292, 53.05488436313428, 53.96962374870555, 56.2133241284087, 57.66309975837073, 58.664135312392126, 59.63065239903348, 60.8560579910252, 61.90887124611667, 62.94442526751812, 64.10079392474974, 64.68760787021056, 65.82671729375215, 66.96582671729375, 68.08767690714532, 69.6410079392475, 70.88367276492924, 71.7811529168105, 73.00655850880221, 74.2147048671039, 75.2847773558854, 76.2167759751467, 77.77010700724888, 78.75388332758025, 80.13462202278218, 81.30824991370383, 82.6372109078357, 83.8626164998274, 84.38039351052812, 85.05350362443907, 85.83016914049017, 86.7103900586814, 87.41801863997239, 88.16016568864342, 89.23023817742492, 89.76527442181566, 90.93890231273731, 91.69830859509838, 92.54401104590956, 93.09630652399034, 93.70037970314118, 94.25267518122196, 95.28822920262341, 95.70245081118398, 96.18570935450465, 96.96237487055575, 97.51467034863653, 98.08422506040732, 98.44666896789782, 99.1370383154988, 99.62029685881947, 100.0], "rescued": [0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 4, 4, 5, 7, 8, 9, 13, 13, 15, 17, 18, 18, 21, 21, 23, 27, 29, 32, 36, 43, 48, 52, 54, 56, 64, 68, 76, 86, 90, 95, 104, 116, 120, 135, 146, 148, 160, 165, 169, 179, 198, 202, 210, 217, 223, 231, 237, 248, 251, 259, 268, 277, 284, 287, 290, 295, 299, 303, 308, 316, 322, 325, 327, 330, 334, 334, 335, 340, 344, 348, 350, 351, 352, 355, 359, 360, 362, 363, 363, 363, 363, 363, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 1, 1, 1, 1, 1, 2, 3, 4, 5, 5, 6, 6, 6, 14, 15, 15, 17, 21, 23, 27, 30, 31, 34, 41, 45, 48, 51, 56, 64, 75, 85, 97, 108, 121, 134, 148, 153, 172, 184, 207, 216, 230, 250, 263, 293, 314, 324, 348, 365, 379, 386, 421, 448, 463, 480, 508, 529, 552, 579, 590, 611, 638, 661, 696, 726, 756, 785, 824, 858, 890, 943, 974, 1028, 1074, 1120, 1164, 1189, 1219, 1245, 1276, 1304, 1333, 1372, 1396, 1453, 1486, 1522, 1547, 1578, 1605, 1660, 1679, 1706, 1742, 1768, 1800, 1820, 1859, 1886, 1907], "exact_score": [81.429064549534, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.27373144632378, 81.30824991370383, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.22195374525371, 81.18743527787366, 81.11839834311357, 81.03210217466345, 81.03210217466345, 81.01484294097342, 80.96306523990336, 80.89402830514325, 80.87676907145323, 80.66965826717293, 80.70417673455299, 80.61788056610285, 80.53158439765275, 80.35899206075251, 80.27269589230238, 80.11736278909216, 80.03106662064205, 79.94477045219193, 79.89299275112185, 79.70314118053159, 79.72040041422161, 79.53054884363134, 79.39247497411115, 79.28891957197101, 79.28891957197101, 79.15084570245081, 78.9955125992406, 78.87469796341043, 78.73662409389023, 78.56403175698999, 78.54677252329996, 78.37418018639973, 78.30514325163963, 78.13255091473938, 77.99447704521918, 77.82188470831896, 77.856403175699, 77.68381083879876, 77.58025543665862, 77.3558853986883, 77.16603382809804, 77.06247842595789, 76.78633068691751, 76.49292371418709, 76.30307214359682, 75.99240593717639, 75.71625819813602, 75.52640662754574, 75.37107352433551, 75.02588885053504, 74.80151881256472, 74.59440800828443, 74.2147048671039, 73.7832240248533, 73.31722471522265, 72.98929927511219, 72.50604073179151, 72.02278218847084, 71.43596824301002, 70.90093199861927, 70.48671039005868, 70.05522954780807, 69.38211943389715, 68.69175008629618, 68.12219537452538, 67.62167759751468, 67.01760441836383, 66.24093890231273, 65.68864342423197, 65.17086641353124, 64.54953400069037, 63.9972385226096, 63.22057300655851, 62.44390749050742, 61.71901967552641, 61.04590956161546, 60.200207110804286, 59.31998619261305, 58.439765274421816, 57.62858129099069, 56.748360372799446, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.30036696574331, "threshold_area100": 74.11011391094236, "threshold_area25": 81.31515360717985}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.029417037963867, "operator": "ge"}, {"budget": 0.02, "threshold": -2.639176368713379, "operator": "ge"}, {"budget": 0.03, "threshold": -3.0524580478668213, "operator": "ge"}, {"budget": 0.04, "threshold": -3.499772787094116, "operator": "ge"}, {"budget": 0.05, "threshold": -3.9118638038635254, "operator": "ge"}, {"budget": 0.06, "threshold": -4.470894813537598, "operator": "ge"}, {"budget": 0.07, "threshold": -4.807645320892334, "operator": "ge"}, {"budget": 0.08, "threshold": -5.032894134521484, "operator": "ge"}, {"budget": 0.09, "threshold": -5.234536170959473, "operator": "ge"}, {"budget": 0.1, "threshold": -5.544802665710449, "operator": "ge"}, {"budget": 0.11, "threshold": -5.687115669250488, "operator": "ge"}, {"budget": 0.12, "threshold": -5.925183296203613, "operator": "ge"}, {"budget": 0.13, "threshold": -6.2357563972473145, "operator": "ge"}, {"budget": 0.14, "threshold": -6.48885440826416, "operator": "ge"}, {"budget": 0.15, "threshold": -6.775613784790039, "operator": "ge"}, {"budget": 0.16, "threshold": -6.9540534019470215, "operator": "ge"}, {"budget": 0.17, "threshold": -7.264383316040039, "operator": "ge"}, {"budget": 0.18, "threshold": -7.462177276611328, "operator": "ge"}, {"budget": 0.19, "threshold": -7.707637786865234, "operator": "ge"}, {"budget": 0.2, "threshold": -7.810858726501465, "operator": "ge"}, {"budget": 0.21, "threshold": -7.885828018188477, "operator": "ge"}, {"budget": 0.22, "threshold": -7.945908069610596, "operator": "ge"}, {"budget": 0.23, "threshold": -8.090822219848633, "operator": "ge"}, {"budget": 0.24, "threshold": -8.206433296203613, "operator": "ge"}, {"budget": 0.25, "threshold": -8.419279098510742, "operator": "ge"}, {"budget": 0.26, "threshold": -8.557801246643066, "operator": "ge"}, {"budget": 0.27, "threshold": -8.634066581726074, "operator": "ge"}, {"budget": 0.28, "threshold": -8.749115943908691, "operator": "ge"}, {"budget": 0.29, "threshold": -8.853114128112793, "operator": "ge"}, {"budget": 0.3, "threshold": -8.925102233886719, "operator": "ge"}, {"budget": 0.31, "threshold": -8.996530532836914, "operator": "ge"}, {"budget": 0.32, "threshold": -9.045321464538574, "operator": "ge"}, {"budget": 0.33, "threshold": -9.11540699005127, "operator": "ge"}, {"budget": 0.34, "threshold": -9.178821563720703, "operator": "ge"}, {"budget": 0.35, "threshold": -9.214600563049316, "operator": "ge"}, {"budget": 0.36, "threshold": -9.270318984985352, "operator": "ge"}, {"budget": 0.37, "threshold": -9.317437171936035, "operator": "ge"}, {"budget": 0.38, "threshold": -9.39316177368164, "operator": "ge"}, {"budget": 0.39, "threshold": -9.482254981994629, "operator": "ge"}, {"budget": 0.4, "threshold": -9.554158210754395, "operator": "ge"}, {"budget": 0.41, "threshold": -9.578679084777832, "operator": "ge"}, {"budget": 0.42, "threshold": -9.631503105163574, "operator": "ge"}, {"budget": 0.43, "threshold": -9.664334297180176, "operator": "ge"}, {"budget": 0.44, "threshold": -9.72779369354248, "operator": "ge"}, {"budget": 0.45, "threshold": -9.794625282287598, "operator": "ge"}, {"budget": 0.46, "threshold": -9.854365348815918, "operator": "ge"}, {"budget": 0.47, "threshold": -9.875853538513184, "operator": "ge"}, {"budget": 0.48, "threshold": -9.938636779785156, "operator": "ge"}, {"budget": 0.49, "threshold": -9.973649978637695, "operator": "ge"}, {"budget": 0.5, "threshold": -10.00605297088623, "operator": "ge"}, {"budget": 0.51, "threshold": -10.043863296508789, "operator": "ge"}, {"budget": 0.52, "threshold": -10.079432487487793, "operator": "ge"}, {"budget": 0.53, "threshold": -10.115791320800781, "operator": "ge"}, {"budget": 0.54, "threshold": -10.135351181030273, "operator": "ge"}, {"budget": 0.55, "threshold": -10.185953140258789, "operator": "ge"}, {"budget": 0.56, "threshold": -10.204585075378418, "operator": "ge"}, {"budget": 0.57, "threshold": -10.228290557861328, "operator": "ge"}, {"budget": 0.58, "threshold": -10.282106399536133, "operator": "ge"}, {"budget": 0.59, "threshold": -10.314718246459961, "operator": "ge"}, {"budget": 0.6, "threshold": -10.338382720947266, "operator": "ge"}, {"budget": 0.61, "threshold": -10.388566017150879, "operator": "ge"}, {"budget": 0.62, "threshold": -10.418401718139648, "operator": "ge"}, {"budget": 0.63, "threshold": -10.472168922424316, "operator": "ge"}, {"budget": 0.64, "threshold": -10.517801284790039, "operator": "ge"}, {"budget": 0.65, "threshold": -10.552417755126953, "operator": "ge"}, {"budget": 0.66, "threshold": -10.58862018585205, "operator": "ge"}, {"budget": 0.67, "threshold": -10.646422386169434, "operator": "ge"}, {"budget": 0.68, "threshold": -10.701560974121094, "operator": "ge"}, {"budget": 0.69, "threshold": -10.769938468933105, "operator": "ge"}, {"budget": 0.7, "threshold": -10.807914733886719, "operator": "ge"}, {"budget": 0.71, "threshold": -10.834306716918945, "operator": "ge"}, {"budget": 0.72, "threshold": -10.862771987915039, "operator": "ge"}, {"budget": 0.73, "threshold": -10.892539978027344, "operator": "ge"}, {"budget": 0.74, "threshold": -10.945112228393555, "operator": "ge"}, {"budget": 0.75, "threshold": -11.003654479980469, "operator": "ge"}, {"budget": 0.76, "threshold": -11.026092529296875, "operator": "ge"}, {"budget": 0.77, "threshold": -11.080142974853516, "operator": "ge"}, {"budget": 0.78, "threshold": -11.147644996643066, "operator": "ge"}, {"budget": 0.79, "threshold": -11.174825668334961, "operator": "ge"}, {"budget": 0.8, "threshold": -11.207698822021484, "operator": "ge"}, {"budget": 0.81, "threshold": -11.250320434570312, "operator": "ge"}, {"budget": 0.82, "threshold": -11.303594589233398, "operator": "ge"}, {"budget": 0.83, "threshold": -11.370959281921387, "operator": "ge"}, {"budget": 0.84, "threshold": -11.42825984954834, "operator": "ge"}, {"budget": 0.85, "threshold": -11.468926429748535, "operator": "ge"}, {"budget": 0.86, "threshold": -11.52442741394043, "operator": "ge"}, {"budget": 0.87, "threshold": -11.586919784545898, "operator": "ge"}, {"budget": 0.88, "threshold": -11.675308227539062, "operator": "ge"}, {"budget": 0.89, "threshold": -11.763701438903809, "operator": "ge"}, {"budget": 0.9, "threshold": -11.810338973999023, "operator": "ge"}, {"budget": 0.91, "threshold": -11.880824089050293, "operator": "ge"}, {"budget": 0.92, "threshold": -11.95109748840332, "operator": "ge"}, {"budget": 0.93, "threshold": -12.042871475219727, "operator": "ge"}, {"budget": 0.94, "threshold": -12.135205268859863, "operator": "ge"}, {"budget": 0.95, "threshold": -12.239728927612305, "operator": "ge"}, {"budget": 0.96, "threshold": -12.337142944335938, "operator": "ge"}, {"budget": 0.97, "threshold": -12.442788124084473, "operator": "ge"}, {"budget": 0.98, "threshold": -12.721490859985352, "operator": "ge"}, {"budget": 0.99, "threshold": -12.972831726074219, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.25, 97.58333333333333, 97.25, 96.66666666666667, 96.33333333333334, 95.66666666666667, 94.91666666666667, 94.41666666666667, 93.83333333333333, 93.41666666666667, 92.83333333333333, 92.41666666666667, 91.75, 91.08333333333334, 90.58333333333334, 89.83333333333333, 89.25, 88.5, 88.0, 87.41666666666667, 87.08333333333333, 86.75, 86.25, 85.83333333333334, 85.66666666666667, 85.41666666666667, 85.16666666666667, 84.83333333333334, 84.41666666666667, 84.16666666666667, 84.0, 83.75, 83.58333333333333, 83.16666666666667, 83.0, 82.83333333333334, 82.66666666666667, 82.25, 82.08333333333333, 81.66666666666667, 81.41666666666667, 81.16666666666667, 80.83333333333333, 80.5, 80.25, 80.0, 79.58333333333334, 79.33333333333333, 79.0, 78.66666666666666, 78.41666666666667, 78.16666666666667, 77.75, 77.25, 77.08333333333334, 76.75, 76.33333333333333, 75.83333333333333, 75.5, 75.16666666666667, 74.58333333333333, 74.16666666666667, 74.0, 73.75, 73.5, 73.33333333333334, 72.83333333333333, 72.50000000000001, 72.25, 71.58333333333333, 71.0, 70.5, 70.0, 69.75, 69.41666666666666, 69.08333333333334, 68.41666666666666, 68.0, 67.58333333333333, 66.75, 66.25000000000001, 65.91666666666667, 65.41666666666667, 65.0, 64.58333333333333, 64.16666666666666, 63.583333333333336, 63.083333333333336, 62.916666666666664, 62.083333333333336, 61.41666666666667, 60.74999999999999, 60.083333333333336, 59.583333333333336, 58.916666666666664, 58.25, 57.49999999999999, 56.58333333333334, 55.833333333333336, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 4, 12, 16, 23, 28, 36, 45, 51, 58, 63, 70, 76, 84, 92, 98, 107, 114, 123, 129, 136, 140, 144, 150, 155, 157, 160, 163, 167, 172, 175, 177, 180, 182, 187, 189, 191, 193, 198, 200, 205, 208, 211, 215, 219, 222, 225, 230, 233, 237, 241, 244, 247, 252, 258, 260, 264, 269, 275, 279, 283, 290, 295, 297, 300, 303, 305, 311, 315, 318, 326, 333, 339, 345, 348, 352, 356, 364, 369, 374, 384, 390, 394, 400, 405, 410, 415, 422, 428, 430, 440, 448, 456, 464, 470, 478, 486, 495, 506, 515, 524], "exact_score": [98.58333333333333, 98.25, 97.58333333333333, 97.25, 96.66666666666667, 96.33333333333334, 95.66666666666667, 94.91666666666667, 94.41666666666667, 93.83333333333333, 93.41666666666667, 92.83333333333333, 92.41666666666667, 91.75, 91.08333333333334, 90.58333333333334, 89.83333333333333, 89.25, 88.5, 88.0, 87.41666666666667, 87.08333333333333, 86.75, 86.25, 85.83333333333334, 85.66666666666667, 85.41666666666667, 85.16666666666667, 84.83333333333334, 84.41666666666667, 84.16666666666667, 84.0, 83.75, 83.58333333333333, 83.16666666666667, 83.0, 82.83333333333334, 82.66666666666667, 82.25, 82.08333333333333, 81.66666666666667, 81.33333333333333, 81.16666666666667, 80.83333333333333, 80.5, 80.25, 80.0, 79.58333333333334, 79.33333333333333, 79.0, 78.66666666666666, 78.41666666666667, 78.16666666666667, 77.75, 77.25, 77.08333333333334, 76.75, 76.33333333333333, 75.83333333333333, 75.5, 75.16666666666667, 74.58333333333333, 74.16666666666667, 74.0, 73.75, 73.5, 73.33333333333334, 72.83333333333333, 72.50000000000001, 72.25, 71.58333333333333, 71.0, 70.5, 70.0, 69.75, 69.41666666666666, 69.08333333333334, 68.41666666666666, 68.0, 67.58333333333333, 66.75, 66.25000000000001, 65.91666666666667, 65.41666666666667, 65.0, 64.58333333333333, 64.16666666666666, 63.583333333333336, 63.083333333333336, 62.916666666666664, 62.083333333333336, 61.41666666666667, 60.74999999999999, 60.083333333333336, 59.583333333333336, 58.916666666666664, 58.25, 57.49999999999999, 56.58333333333334, 55.833333333333336, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 77.85249999999999, "threshold_area100": 77.85333333333332, "threshold_area25": 91.97500000000002}, "test": {"score": [81.429064549534, 81.04936140835348, 80.75595443562305, 80.39351052813255, 79.66862271315154, 78.97825336555057, 77.4249223334484, 76.87262685536763, 76.18225750776666, 75.66448049706594, 74.69796341042458, 74.35277873662409, 73.576113220573, 72.79944770452192, 72.16085605799103, 71.59130134622022, 71.19433897134968, 70.62478425957887, 70.15878494994823, 69.7790818087677, 69.5719710044874, 69.5029340697273, 69.41663790127718, 69.22678633068692, 69.03693476009666, 68.65723161891613, 68.51915774939593, 68.3638246461857, 68.1394546082154, 68.01863997238523, 67.82878840179495, 67.69071453227477, 67.55264066275457, 67.38004832585433, 67.25923369002416, 67.22471522264412, 67.17293752157406, 67.01760441836383, 66.75871591301346, 66.5688643424232, 66.37901277183293, 66.27545736969279, 66.10286503279255, 65.9993096306524, 65.84397652744218, 65.65412495685192, 65.51605108733173, 65.44701415257163, 65.15360717984122, 65.08457024508112, 64.96375560925095, 64.80842250604073, 64.68760787021056, 64.58405246807042, 64.54953400069037, 64.29064549534, 64.25612702795996, 64.20434932688988, 64.03175698998965, 63.80738695201933, 63.65205384880912, 63.44494304452882, 63.35864687607871, 63.13427683810839, 63.03072143596824, 62.90990680013808, 62.771832930617876, 62.59924059371764, 62.3576113220573, 62.01242664825681, 61.7880566102865, 61.58094580600621, 61.44287193648602, 61.32205730065585, 61.08042802899551, 60.80428028995512, 60.66620642043493, 60.40731791508457, 60.13117017604419, 59.993096306523995, 59.94131860545392, 59.803244735933724, 59.68243010010356, 59.33724542630308, 59.078356920952714, 58.8539868829824, 58.64687607870211, 58.33620987228167, 58.00828443217121, 57.69761822575078, 57.525025888850536, 57.26613738350017, 57.04176734552986, 56.78287884017949, 56.50673110113911, 56.19606489471868, 55.91991715567829, 55.712806351397994, 55.3503624439075, 55.12599240593718, 54.81532619951675], "calls": [0.0, 3.2274767000345186, 5.67828788401795, 8.612357611322057, 13.134276838108388, 17.70797376596479, 24.68070417673455, 29.288919571971007, 32.706247842595786, 35.43320676561961, 39.10942354159475, 40.40386606834657, 42.682084915429755, 45.512599240593715, 47.204004142216085, 49.20607525025889, 50.43148084225061, 52.19192267863308, 53.572661373835004, 54.76354849844667, 55.36762167759751, 55.74732481877804, 56.10976872626855, 56.817397307559546, 57.54228512254056, 59.026579219882635, 60.04487400759406, 60.700724887814985, 61.46013117017605, 62.133241284086985, 62.84086986537798, 63.30686917500863, 63.72109078356921, 64.41146013117017, 64.96375560925095, 65.18812564722126, 65.72316189161201, 66.2064204349327, 67.03486365205384, 68.05315843976527, 69.01967552640663, 69.31308249913704, 69.88263721090784, 70.22782188470832, 71.0735243355195, 71.88470831895064, 72.60959613393166, 72.90300310666207, 73.86952019330342, 74.4218156713842, 74.90507421470487, 75.45736969278563, 75.85433206765619, 76.40662754573697, 76.64825681739731, 77.51121850189851, 77.73558853986883, 78.11529168104936, 78.8574387297204, 79.51328960994132, 79.96202968588196, 80.54884363134278, 80.91128753883328, 81.61891612012427, 82.05039696237488, 82.39558163617535, 82.80980324473593, 83.3620987228167, 84.03520883672765, 84.88091128753882, 85.45046599930963, 85.8474283741802, 86.26164998274076, 86.5723161891612, 87.26268553676216, 88.10838798757335, 88.38453572661373, 88.97134967207457, 89.79979288919571, 90.26579219882636, 90.57645840524681, 90.83534691059717, 91.37038315498792, 92.25060407317915, 92.83741801863997, 93.18260269244045, 93.61408353469106, 94.16637901277183, 94.9257852951329, 95.56437694166378, 95.89230238177426, 96.3583016914049, 96.7207455988954, 97.15222644114601, 97.5664480497066, 98.15326199516741, 98.51570590265793, 98.87814981014843, 99.3614083534691, 99.60303762512945, 100.0], "rescued": [0, 34, 56, 83, 128, 160, 214, 258, 282, 297, 316, 322, 329, 335, 340, 348, 351, 355, 355, 356, 356, 356, 356, 356, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 361, 361, 362, 363, 363, 363, 363, 364, 364, 364, 364, 364, 364, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 56, 95, 143, 230, 302, 446, 522, 586, 631, 706, 732, 784, 835, 877, 918, 944, 981, 1008, 1031, 1043, 1047, 1052, 1063, 1075, 1097, 1105, 1114, 1127, 1134, 1145, 1153, 1161, 1171, 1178, 1180, 1183, 1192, 1207, 1220, 1231, 1237, 1247, 1253, 1262, 1273, 1281, 1285, 1302, 1306, 1313, 1322, 1329, 1335, 1337, 1352, 1354, 1357, 1367, 1380, 1389, 1401, 1407, 1420, 1426, 1433, 1441, 1451, 1465, 1485, 1498, 1510, 1518, 1525, 1539, 1555, 1563, 1578, 1595, 1603, 1606, 1614, 1621, 1642, 1658, 1671, 1683, 1701, 1721, 1739, 1749, 1764, 1777, 1792, 1809, 1827, 1843, 1855, 1876, 1889, 1907], "exact_score": [81.429064549534, 81.44632378322403, 81.32550914739386, 81.1356575768036, 80.94580600621333, 80.91128753883328, 80.7904729030031, 80.65239903348291, 80.4970659302727, 80.4280289955126, 80.32447359337246, 80.18639972385226, 79.94477045219193, 79.66862271315154, 79.61684501208146, 79.4269934414912, 79.32343803935106, 79.13358646876078, 78.9264756644805, 78.8574387297204, 78.52951328960994, 78.39143942008975, 78.14981014842941, 77.83914394200899, 77.58025543665862, 77.37314463237833, 77.16603382809804, 77.07973765964792, 76.99344149119779, 76.88988608905764, 76.71729375215742, 76.63099758370728, 76.40662754573697, 76.23403520883673, 75.94062823610632, 75.68173973075596, 75.56092509492579, 75.25025888850536, 74.90507421470487, 74.76700034518467, 74.43907490507422, 74.19744563341388, 73.69692785640318, 73.50707628581291, 73.2827062478426, 72.9202623403521, 72.67863306869175, 72.2816706938212, 71.93648602002072, 71.60856057991025, 71.29789437348983, 70.93545046599931, 70.65930272695893, 70.33137728684846, 70.003451846738, 69.67552640662755, 69.45115636865722, 69.15774939592683, 68.88160165688643, 68.65723161891613, 68.53641698308596, 68.29478771142561, 68.0704176734553, 67.81152916810494, 67.5008629616845, 67.24197445633415, 67.08664135312392, 66.75871591301346, 66.55160510873317, 66.37901277183293, 66.08560579910252, 65.8612357611322, 65.65412495685192, 65.41249568519159, 65.13634794615119, 64.94649637556093, 64.67034863652054, 64.41146013117019, 64.23886779426994, 63.9972385226096, 63.61753538142907, 63.35864687607871, 63.03072143596824, 62.72005522954781, 62.3576113220573, 61.977908180876774, 61.51190887124611, 61.14946496375562, 60.890576458405256, 60.40731791508457, 60.079392474974114, 59.75146703486365, 59.423541594753196, 58.97480151881257, 58.45702450811184, 57.97376596479117, 57.455988954090444, 56.8519157749396, 56.24784259578875, 55.62651018294788, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 70.81646184160374, "threshold_area100": 65.49896444597859, "threshold_area25": 73.97652744218158}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.48314106464385986, "operator": "ge"}, {"budget": 0.02, "threshold": -0.8321943283081055, "operator": "ge"}, {"budget": 0.03, "threshold": -1.1142222881317139, "operator": "ge"}, {"budget": 0.04, "threshold": -1.4030569791793823, "operator": "ge"}, {"budget": 0.05, "threshold": -1.7015595436096191, "operator": "ge"}, {"budget": 0.06, "threshold": -2.018784523010254, "operator": "ge"}, {"budget": 0.07, "threshold": -2.229781150817871, "operator": "ge"}, {"budget": 0.08, "threshold": -2.6163721084594727, "operator": "ge"}, {"budget": 0.09, "threshold": -2.821030855178833, "operator": "ge"}, {"budget": 0.1, "threshold": -3.0390257835388184, "operator": "ge"}, {"budget": 0.11, "threshold": -3.2738046646118164, "operator": "ge"}, {"budget": 0.12, "threshold": -3.6231014728546143, "operator": "ge"}, {"budget": 0.13, "threshold": -3.9848074913024902, "operator": "ge"}, {"budget": 0.14, "threshold": -4.205269813537598, "operator": "ge"}, {"budget": 0.15, "threshold": -4.457680702209473, "operator": "ge"}, {"budget": 0.16, "threshold": -4.675643444061279, "operator": "ge"}, {"budget": 0.17, "threshold": -4.984733581542969, "operator": "ge"}, {"budget": 0.18, "threshold": -5.336781024932861, "operator": "ge"}, {"budget": 0.19, "threshold": -5.5369873046875, "operator": "ge"}, {"budget": 0.2, "threshold": -5.729785919189453, "operator": "ge"}, {"budget": 0.21, "threshold": -6.0655436515808105, "operator": "ge"}, {"budget": 0.22, "threshold": -6.30213737487793, "operator": "ge"}, {"budget": 0.23, "threshold": -6.562376022338867, "operator": "ge"}, {"budget": 0.24, "threshold": -6.677778244018555, "operator": "ge"}, {"budget": 0.25, "threshold": -6.8200883865356445, "operator": "ge"}, {"budget": 0.26, "threshold": -6.922270774841309, "operator": "ge"}, {"budget": 0.27, "threshold": -7.02507209777832, "operator": "ge"}, {"budget": 0.28, "threshold": -7.12619686126709, "operator": "ge"}, {"budget": 0.29, "threshold": -7.227838516235352, "operator": "ge"}, {"budget": 0.3, "threshold": -7.371431827545166, "operator": "ge"}, {"budget": 0.31, "threshold": -7.455485820770264, "operator": "ge"}, {"budget": 0.32, "threshold": -7.509410381317139, "operator": "ge"}, {"budget": 0.33, "threshold": -7.603240013122559, "operator": "ge"}, {"budget": 0.34, "threshold": -7.651220321655273, "operator": "ge"}, {"budget": 0.35, "threshold": -7.7209978103637695, "operator": "ge"}, {"budget": 0.36, "threshold": -7.796667575836182, "operator": "ge"}, {"budget": 0.37, "threshold": -7.882308483123779, "operator": "ge"}, {"budget": 0.38, "threshold": -7.918887615203857, "operator": "ge"}, {"budget": 0.39, "threshold": -7.957339286804199, "operator": "ge"}, {"budget": 0.4, "threshold": -8.019712448120117, "operator": "ge"}, {"budget": 0.41, "threshold": -8.095685005187988, "operator": "ge"}, {"budget": 0.42, "threshold": -8.173571586608887, "operator": "ge"}, {"budget": 0.43, "threshold": -8.227034568786621, "operator": "ge"}, {"budget": 0.44, "threshold": -8.297792434692383, "operator": "ge"}, {"budget": 0.45, "threshold": -8.322391510009766, "operator": "ge"}, {"budget": 0.46, "threshold": -8.377412796020508, "operator": "ge"}, {"budget": 0.47, "threshold": -8.437172889709473, "operator": "ge"}, {"budget": 0.48, "threshold": -8.502256393432617, "operator": "ge"}, {"budget": 0.49, "threshold": -8.530962944030762, "operator": "ge"}, {"budget": 0.5, "threshold": -8.590914726257324, "operator": "ge"}, {"budget": 0.51, "threshold": -8.625741958618164, "operator": "ge"}, {"budget": 0.52, "threshold": -8.689706802368164, "operator": "ge"}, {"budget": 0.53, "threshold": -8.74701976776123, "operator": "ge"}, {"budget": 0.54, "threshold": -8.79630184173584, "operator": "ge"}, {"budget": 0.55, "threshold": -8.831256866455078, "operator": "ge"}, {"budget": 0.56, "threshold": -8.858357429504395, "operator": "ge"}, {"budget": 0.57, "threshold": -8.881216049194336, "operator": "ge"}, {"budget": 0.58, "threshold": -8.927103996276855, "operator": "ge"}, {"budget": 0.59, "threshold": -8.95598030090332, "operator": "ge"}, {"budget": 0.6, "threshold": -8.990776062011719, "operator": "ge"}, {"budget": 0.61, "threshold": -9.025434494018555, "operator": "ge"}, {"budget": 0.62, "threshold": -9.066694259643555, "operator": "ge"}, {"budget": 0.63, "threshold": -9.115863800048828, "operator": "ge"}, {"budget": 0.64, "threshold": -9.156923294067383, "operator": "ge"}, {"budget": 0.65, "threshold": -9.189682960510254, "operator": "ge"}, {"budget": 0.66, "threshold": -9.245912551879883, "operator": "ge"}, {"budget": 0.67, "threshold": -9.2724609375, "operator": "ge"}, {"budget": 0.68, "threshold": -9.298357009887695, "operator": "ge"}, {"budget": 0.69, "threshold": -9.340913772583008, "operator": "ge"}, {"budget": 0.7, "threshold": -9.364568710327148, "operator": "ge"}, {"budget": 0.71, "threshold": -9.418102264404297, "operator": "ge"}, {"budget": 0.72, "threshold": -9.451517105102539, "operator": "ge"}, {"budget": 0.73, "threshold": -9.496134757995605, "operator": "ge"}, {"budget": 0.74, "threshold": -9.525677680969238, "operator": "ge"}, {"budget": 0.75, "threshold": -9.578970909118652, "operator": "ge"}, {"budget": 0.76, "threshold": -9.603596687316895, "operator": "ge"}, {"budget": 0.77, "threshold": -9.642570495605469, "operator": "ge"}, {"budget": 0.78, "threshold": -9.669364929199219, "operator": "ge"}, {"budget": 0.79, "threshold": -9.707634925842285, "operator": "ge"}, {"budget": 0.8, "threshold": -9.781588554382324, "operator": "ge"}, {"budget": 0.81, "threshold": -9.809176445007324, "operator": "ge"}, {"budget": 0.82, "threshold": -9.844257354736328, "operator": "ge"}, {"budget": 0.83, "threshold": -9.896987915039062, "operator": "ge"}, {"budget": 0.84, "threshold": -9.945965766906738, "operator": "ge"}, {"budget": 0.85, "threshold": -9.993200302124023, "operator": "ge"}, {"budget": 0.86, "threshold": -10.069826126098633, "operator": "ge"}, {"budget": 0.87, "threshold": -10.117427825927734, "operator": "ge"}, {"budget": 0.88, "threshold": -10.170068740844727, "operator": "ge"}, {"budget": 0.89, "threshold": -10.259443283081055, "operator": "ge"}, {"budget": 0.9, "threshold": -10.30198860168457, "operator": "ge"}, {"budget": 0.91, "threshold": -10.376884460449219, "operator": "ge"}, {"budget": 0.92, "threshold": -10.439160346984863, "operator": "ge"}, {"budget": 0.93, "threshold": -10.526966094970703, "operator": "ge"}, {"budget": 0.94, "threshold": -10.575226783752441, "operator": "ge"}, {"budget": 0.95, "threshold": -10.639808654785156, "operator": "ge"}, {"budget": 0.96, "threshold": -10.737247467041016, "operator": "ge"}, {"budget": 0.97, "threshold": -10.833024978637695, "operator": "ge"}, {"budget": 0.98, "threshold": -10.91268539428711, "operator": "ge"}, {"budget": 0.99, "threshold": -11.073308944702148, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.33333333333334, 97.75, 97.16666666666667, 96.58333333333333, 96.08333333333333, 95.91666666666667, 95.08333333333333, 94.5, 93.91666666666667, 93.25, 92.66666666666666, 92.0, 91.5, 91.08333333333334, 90.25, 89.66666666666667, 89.08333333333334, 88.41666666666667, 87.83333333333333, 87.25, 86.75, 86.33333333333333, 85.83333333333334, 85.33333333333333, 84.75, 84.5, 84.08333333333333, 83.91666666666666, 83.75, 83.41666666666667, 83.25, 83.08333333333333, 83.0, 82.5, 82.33333333333334, 81.83333333333334, 81.5, 81.08333333333333, 80.5, 80.25, 79.91666666666667, 79.5, 79.25, 79.08333333333333, 78.83333333333333, 78.41666666666667, 78.08333333333334, 77.66666666666666, 77.41666666666667, 76.75, 76.58333333333334, 76.16666666666667, 75.91666666666667, 75.5, 75.16666666666667, 74.75, 74.41666666666666, 74.08333333333333, 74.0, 73.66666666666667, 73.41666666666666, 73.0, 72.58333333333333, 72.50000000000001, 72.25, 71.83333333333333, 71.33333333333334, 70.83333333333333, 70.75, 70.16666666666667, 69.75, 69.16666666666667, 68.83333333333333, 68.16666666666666, 68.08333333333334, 67.58333333333333, 67.08333333333334, 66.66666666666667, 66.5, 66.0, 65.66666666666667, 65.25000000000001, 64.66666666666667, 64.08333333333334, 63.66666666666667, 63.16666666666666, 62.66666666666667, 62.083333333333336, 61.66666666666667, 61.083333333333336, 60.583333333333336, 60.00000000000001, 59.33333333333333, 58.833333333333336, 58.08333333333333, 57.49999999999999, 56.91666666666666, 56.333333333333336, 55.75, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 3, 10, 17, 24, 30, 33, 43, 50, 57, 65, 72, 81, 87, 92, 102, 109, 116, 124, 131, 138, 144, 149, 155, 161, 168, 171, 176, 178, 180, 184, 186, 188, 189, 195, 197, 203, 207, 212, 219, 222, 226, 231, 234, 236, 239, 244, 248, 253, 256, 264, 266, 271, 274, 279, 283, 288, 292, 296, 297, 301, 304, 309, 314, 315, 318, 323, 329, 335, 336, 343, 348, 355, 359, 367, 368, 374, 380, 385, 387, 393, 397, 402, 409, 416, 421, 427, 433, 440, 445, 452, 458, 465, 473, 479, 488, 495, 502, 509, 516, 524], "exact_score": [98.58333333333333, 98.33333333333334, 97.75, 97.16666666666667, 96.58333333333333, 96.08333333333333, 95.91666666666667, 95.08333333333333, 94.5, 93.91666666666667, 93.25, 92.66666666666666, 92.0, 91.5, 91.08333333333334, 90.25, 89.66666666666667, 89.08333333333334, 88.41666666666667, 87.83333333333333, 87.25, 86.75, 86.33333333333333, 85.83333333333334, 85.33333333333333, 84.75, 84.5, 84.08333333333333, 83.91666666666666, 83.75, 83.41666666666667, 83.25, 83.08333333333333, 83.0, 82.5, 82.33333333333334, 81.83333333333334, 81.5, 81.08333333333333, 80.5, 80.25, 79.83333333333333, 79.5, 79.25, 79.08333333333333, 78.83333333333333, 78.41666666666667, 78.08333333333334, 77.66666666666666, 77.41666666666667, 76.75, 76.58333333333334, 76.16666666666667, 75.91666666666667, 75.5, 75.16666666666667, 74.75, 74.33333333333334, 74.08333333333333, 74.0, 73.66666666666667, 73.41666666666666, 73.0, 72.58333333333333, 72.50000000000001, 72.25, 71.83333333333333, 71.33333333333334, 70.83333333333333, 70.75, 70.16666666666667, 69.75, 69.16666666666667, 68.83333333333333, 68.16666666666666, 68.08333333333334, 67.58333333333333, 67.08333333333334, 66.66666666666667, 66.5, 66.0, 65.66666666666667, 65.25000000000001, 64.66666666666667, 64.08333333333334, 63.66666666666667, 63.16666666666666, 62.66666666666667, 62.083333333333336, 61.66666666666667, 61.083333333333336, 60.583333333333336, 60.00000000000001, 59.33333333333333, 58.833333333333336, 58.08333333333333, 57.49999999999999, 56.91666666666666, 56.333333333333336, 55.75, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 76.92, "threshold_area100": 76.92166666666665, "threshold_area25": 91.7}, "test": {"score": [81.429064549534, 80.89402830514325, 80.01380738695202, 79.59958577839144, 79.16810493614084, 78.44321712115982, 77.51121850189851, 76.94166379012772, 75.73351743182603, 75.33655505695546, 74.5598895409044, 74.16292716603382, 73.19641007939248, 72.48878149810149, 72.29892992751122, 71.62581981360027, 71.17707973765965, 70.65930272695893, 69.95167414566794, 69.58923023817742, 69.33034173282707, 69.03693476009666, 68.6399723852261, 68.24301001035555, 68.05315843976527, 67.79426993441491, 67.67345529858476, 67.46634449430445, 67.27649292371419, 67.10390058681395, 66.82775284777355, 66.7069382119434, 66.58612357611322, 66.44804970659303, 66.34449430445288, 66.25819813600276, 65.9993096306524, 65.82671729375215, 65.70590265792198, 65.61960648947188, 65.4987918536417, 65.36071798412151, 65.10182947877115, 64.998274076631, 64.7911632723507, 64.67034863652054, 64.4977562996203, 64.34242319641008, 64.13531239212979, 64.08353469105973, 63.7901277183293, 63.70383154987918, 63.51397997928891, 63.34138764238868, 63.099758370728345, 62.90990680013808, 62.73731446323784, 62.616499827407665, 62.46116672419745, 62.34035208836728, 62.254055919917164, 62.115982050396966, 61.90887124611668, 61.66724197445633, 61.59820503969624, 61.477390403866075, 61.25302036589575, 61.14946496375562, 61.011391094235414, 60.82153952364515, 60.71798412150501, 60.39005868139456, 60.303762512944424, 60.04487400759406, 59.90680013807387, 59.716948567483605, 59.54435623058336, 59.45806006213324, 59.354504659993104, 59.16465308940283, 58.88850535036245, 58.69865377977218, 58.577839143942015, 58.370728339661724, 58.16361753538143, 57.90472903003106, 57.57680358992061, 57.3524335519503, 57.14532274767, 56.88643424231964, 56.83465654124957, 56.57576803589921, 56.368657231618926, 56.07525025888851, 55.902657921988265, 55.76458405246807, 55.62651018294788, 55.5056955471177, 55.31584397652745, 55.03969623748706, 54.81532619951675], "calls": [0.0, 4.590956161546427, 9.147393855712806, 12.806351397997929, 16.53434587504315, 20.79737659647912, 25.12944425267518, 27.75284777355885, 32.94787711425612, 35.12254055919917, 37.6423886779427, 39.782533655505695, 42.99275112185019, 45.581636175353815, 47.03141180531584, 48.51570590265793, 49.82740766309976, 51.36347946151191, 53.21021746634449, 54.280289955125994, 55.195029340697275, 56.43769416637902, 57.54228512254056, 58.802209181912325, 59.440800828443216, 60.33828098032448, 60.89057645840524, 61.44287193648602, 61.96064894718675, 62.46116672419745, 63.20331377286848, 63.962720055229546, 64.34242319641008, 65.13634794615119, 65.48153261995168, 65.96479116327235, 66.58612357611322, 67.31101139109424, 67.67345529858474, 67.98412150500518, 68.65723161891611, 69.45115636865722, 70.41767345529858, 70.98722816706939, 71.66033828098033, 71.9192267863307, 72.5750776665516, 73.23092854677252, 74.0766309975837, 74.43907490507421, 75.31929582326543, 75.716258198136, 76.59647911632723, 77.39040386606834, 78.04625474628926, 78.51225405591991, 78.96099413186055, 79.25440110459095, 80.18639972385226, 80.61788056610285, 80.9975837072834, 81.32550914739386, 82.17121159820505, 82.89609941318605, 83.55195029340697, 83.96617190196756, 84.69105971694857, 84.94994822229893, 85.36416983085951, 85.98550224370038, 86.24439074905074, 87.02105626510182, 87.45253710735244, 88.03935105281326, 88.41905419399379, 89.12668277528478, 89.50638591646532, 89.90334829133586, 90.24853296513635, 90.714532274767, 91.45667932343804, 91.85364169830859, 92.16430790472903, 92.7856403175699, 93.40697273041076, 93.87297204004142, 94.66689678978253, 95.08111839834311, 95.58163617535381, 96.13393165343459, 96.3583016914049, 96.82430100103555, 97.27304107697618, 97.7217811529168, 98.04970659302728, 98.32585433206765, 98.60200207110805, 98.79185364169831, 99.10251984811875, 99.51674145667933, 100.0], "rescued": [0, 35, 74, 110, 145, 184, 212, 230, 270, 286, 297, 311, 325, 334, 344, 349, 353, 355, 355, 356, 356, 357, 357, 357, 357, 357, 357, 357, 357, 358, 358, 358, 358, 358, 358, 358, 358, 358, 358, 358, 358, 359, 359, 359, 359, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 363, 363, 363, 363, 363, 363, 363, 363, 365, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 66, 156, 216, 276, 357, 439, 490, 600, 639, 695, 732, 802, 852, 873, 917, 947, 979, 1020, 1042, 1057, 1075, 1098, 1121, 1132, 1147, 1154, 1166, 1177, 1188, 1204, 1211, 1218, 1226, 1232, 1237, 1252, 1262, 1269, 1274, 1281, 1290, 1305, 1311, 1323, 1330, 1341, 1350, 1362, 1365, 1382, 1387, 1398, 1408, 1422, 1433, 1443, 1450, 1459, 1466, 1471, 1479, 1491, 1505, 1509, 1516, 1529, 1535, 1543, 1554, 1560, 1579, 1584, 1599, 1607, 1618, 1628, 1633, 1639, 1650, 1666, 1678, 1685, 1697, 1711, 1726, 1745, 1758, 1770, 1785, 1788, 1803, 1817, 1834, 1844, 1852, 1860, 1867, 1878, 1894, 1907], "exact_score": [81.429064549534, 81.2909906800138, 81.17017604418363, 81.0666206420435, 80.98032447359338, 80.82499137038316, 80.68691750086296, 80.4280289955126, 80.22091819123231, 80.06558508802209, 79.94477045219193, 79.8584742837418, 79.72040041422161, 79.59958577839144, 79.47877114256127, 79.27166033828098, 79.20262340352089, 79.13358646876078, 79.09906800138074, 78.94373489817053, 78.73662409389023, 78.40869865377978, 78.21884708318952, 78.02899551259924, 77.856403175699, 77.54573696927856, 77.30410769761824, 77.13151536071798, 76.9244045564377, 76.75181221953746, 76.68277528477736, 76.44114601311702, 76.04418363824647, 75.73351743182603, 75.4228512254056, 75.26751812219537, 75.09492578529513, 74.76700034518467, 74.50811183983431, 74.35277873662409, 74.0766309975837, 73.7832240248533, 73.43803935105282, 73.19641007939248, 72.98929927511219, 72.69589230238178, 72.48878149810149, 72.31618916120124, 71.85018985157059, 71.47048671039006, 71.1425612702796, 70.84915429754919, 70.4176734552986, 70.03797031411806, 69.69278564031757, 69.39937866758716, 69.1404901622368, 68.79530548843631, 68.5018985157059, 68.15671384190543, 67.863306869175, 67.67345529858476, 67.24197445633415, 66.93130824991371, 66.68967897825337, 66.46530894028305, 66.22367966862271, 65.89575422851226, 65.61960648947188, 65.4297549188816, 65.18812564722126, 64.998274076631, 64.67034863652054, 64.37694166379013, 64.13531239212979, 63.89368312046946, 63.61753538142907, 63.46220227821885, 63.099758370728345, 62.702795995857784, 62.4956851915775, 62.254055919917164, 61.96064894718675, 61.64998274076632, 61.477390403866075, 61.13220573006559, 60.82153952364515, 60.40731791508457, 60.079392474974114, 59.75146703486365, 59.440800828443216, 59.09561615464274, 58.62961684501209, 58.2844321712116, 57.852951328961, 57.369692785640325, 56.98998964445978, 56.454953400069044, 55.95443562305834, 55.384880911287546, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 70.38221964111517, "threshold_area100": 64.64842940973422, "threshold_area25": 73.7141870900932}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.44632378322403, "calls": 0.9492578529513289}}, "old_normalized_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.44632378322403, "calls": 0.9492578529513289}}, "old_normalized_endpoints": {"val": {"score": 98.58333333333333, "calls": 0.0}, "test": {"score": 81.429064549534, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.33333333333334, 98.08333333333333, 98.08333333333333, 98.0, 97.83333333333334, 97.58333333333333, 97.58333333333333, 97.58333333333333, 97.33333333333334, 97.16666666666667, 97.08333333333333, 97.0, 96.75, 96.75, 96.58333333333333, 96.5, 96.25, 96.08333333333333, 96.0, 95.83333333333334, 95.75, 95.58333333333333, 95.33333333333334, 95.16666666666667, 94.83333333333334, 94.33333333333334, 94.25, 93.58333333333333, 93.41666666666667, 93.25, 92.91666666666667, 92.41666666666667, 92.0, 91.58333333333333, 90.83333333333333, 90.41666666666667, 90.0, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.66666666666667, 87.25, 86.75, 86.5, 86.0, 85.41666666666667, 84.58333333333333, 84.25, 83.66666666666667, 83.0, 82.33333333333334, 81.66666666666667, 81.0, 80.08333333333333, 79.33333333333333, 78.58333333333334, 77.75, 76.91666666666667, 76.08333333333334, 75.33333333333333, 74.58333333333333, 73.66666666666667, 72.91666666666667, 72.08333333333333, 71.16666666666667, 70.33333333333334, 69.41666666666666, 68.5, 67.75, 67.16666666666666, 66.33333333333333, 65.5, 64.58333333333333, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.00000000000001, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.33333333333334, 98.08333333333333, 98.08333333333333, 98.0, 97.83333333333334, 97.58333333333333, 97.58333333333333, 97.58333333333333, 97.33333333333334, 97.16666666666667, 97.08333333333333, 97.0, 96.75, 96.75, 96.58333333333333, 96.5, 96.25, 96.08333333333333, 96.0, 95.83333333333334, 95.75, 95.58333333333333, 95.33333333333334, 95.16666666666667, 94.83333333333334, 94.33333333333334, 94.25, 93.58333333333333, 93.41666666666667, 93.25, 92.91666666666667, 92.41666666666667, 92.0, 91.58333333333333, 90.83333333333333, 90.41666666666667, 90.0, 89.33333333333333, 88.83333333333333, 88.25, 87.66666666666667, 87.25, 86.75, 86.5, 86.0, 85.41666666666667, 84.58333333333333, 84.25, 83.66666666666667, 83.0, 82.33333333333334, 81.58333333333333, 81.0, 80.08333333333333, 79.33333333333333, 78.58333333333334, 77.75, 76.91666666666667, 76.08333333333334, 75.33333333333333, 74.58333333333333, 73.66666666666667, 72.91666666666667, 72.08333333333333, 71.08333333333333, 70.33333333333334, 69.41666666666666, 68.5, 67.75, 67.16666666666666, 66.33333333333333, 65.5, 64.58333333333333, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.00000000000001, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.36666666666667, "exact_area100": 86.33916666666667}, "test": {"score": [81.429064549534, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.39454608215395, 81.27373144632378, 81.2909906800138, 81.30824991370383, 81.2909906800138, 81.2909906800138, 81.25647221263375, 81.22195374525371, 81.20469451156369, 81.20469451156369, 81.15291681049361, 81.08387987573352, 81.01484294097342, 80.9975837072834, 81.01484294097342, 80.96306523990336, 80.87676907145323, 80.75595443562305, 80.70417673455299, 80.58336209872282, 80.46254746289266, 80.27269589230238, 80.08284432171212, 79.97928891957199, 79.96202968588196, 79.77217811529168, 79.73765964791163, 79.40973420780118, 79.34069727304109, 79.25440110459095, 79.11632723507076, 78.96099413186055, 78.70210562651019, 78.52951328960994, 78.39143942008975, 78.18432861580946, 77.97721781152917, 77.80462547462893, 77.856403175699, 77.58025543665862, 77.18329306178806, 77.06247842595789, 76.88988608905764, 76.51018294787711, 76.28581290990681, 75.99240593717639, 75.71625819813602, 75.57818432861582, 75.35381429064549, 75.04314808422507, 74.80151881256472, 74.31826026924405, 73.85226095961339, 73.38626164998274, 72.97204004142216, 72.36796686227132, 71.85018985157059, 71.38419054193994, 70.60752502588885, 70.17604418363825, 69.29582326544703, 68.53641698308596, 67.79426993441491, 67.10390058681395, 66.67241974456334, 66.17190196755264, 65.80945806006213, 65.34345875043148, 64.9292371418709, 64.46323783224025, 63.80738695201933, 63.41042457714878, 62.478425957887474, 61.977908180876774, 61.37383500172593, 60.976872626855375, 60.45909561615465, 59.993096306523995, 59.04383845357266, 58.715913013462206, 58.249913703831545, 57.66309975837073, 57.21435968243011, 56.66206420434933, 56.316879530548846, 55.64376941663791, 55.17777010700725, 54.81532619951675], "calls": [0.0, 0.9492578529513289, 1.501553331032102, 2.347255781843286, 3.538142906454953, 4.22851225405592, 5.022437003797031, 5.540214014497756, 6.610286503279254, 7.145322747670003, 8.180876769071453, 8.802209181912323, 9.699689333793579, 10.8560579910252, 11.64998274076631, 12.305833620987228, 12.668277528477736, 13.324128408698655, 14.463237832240248, 15.79219882637211, 16.53434587504315, 17.604418363824646, 18.588194684156022, 19.140490162236798, 20.434932688988606, 21.26337590610977, 22.298929927511217, 23.66240938902313, 24.421815671384188, 25.578184328615812, 26.630997583707284, 27.94269934414912, 29.237141870900935, 30.376251294442525, 31.687953054884368, 33.0686917500863, 34.31135657576804, 35.29513289609941, 36.79668622713152, 38.004832585433206, 39.62720055229548, 40.490162236796685, 41.335864687607874, 43.165343458750435, 44.44252675181222, 46.496375560925095, 48.30859509837763, 48.94718674490853, 50.673110113910944, 52.08836727649292, 53.05488436313428, 53.96962374870555, 56.2133241284087, 57.66309975837073, 58.664135312392126, 59.63065239903348, 60.8560579910252, 61.90887124611667, 62.94442526751812, 64.10079392474974, 64.68760787021056, 65.82671729375215, 66.96582671729375, 68.08767690714532, 69.6410079392475, 70.88367276492924, 71.7811529168105, 73.00655850880221, 74.2147048671039, 75.2847773558854, 76.2167759751467, 77.77010700724888, 78.75388332758025, 80.13462202278218, 81.30824991370383, 82.6372109078357, 83.8626164998274, 84.38039351052812, 85.05350362443907, 85.83016914049017, 86.7103900586814, 87.41801863997239, 88.16016568864342, 89.23023817742492, 89.76527442181566, 90.93890231273731, 91.69830859509838, 92.54401104590956, 93.09630652399034, 93.70037970314118, 94.25267518122196, 95.28822920262341, 95.70245081118398, 96.18570935450465, 96.96237487055575, 97.51467034863653, 98.08422506040732, 98.44666896789782, 99.1370383154988, 99.62029685881947, 100.0], "exact_score": [81.429064549534, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.27373144632378, 81.30824991370383, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.22195374525371, 81.18743527787366, 81.11839834311357, 81.03210217466345, 81.03210217466345, 81.01484294097342, 80.96306523990336, 80.89402830514325, 80.87676907145323, 80.66965826717293, 80.70417673455299, 80.61788056610285, 80.53158439765275, 80.35899206075251, 80.27269589230238, 80.11736278909216, 80.03106662064205, 79.94477045219193, 79.89299275112185, 79.70314118053159, 79.72040041422161, 79.53054884363134, 79.39247497411115, 79.28891957197101, 79.28891957197101, 79.15084570245081, 78.9955125992406, 78.87469796341043, 78.73662409389023, 78.56403175698999, 78.54677252329996, 78.37418018639973, 78.30514325163963, 78.13255091473938, 77.99447704521918, 77.82188470831896, 77.856403175699, 77.68381083879876, 77.58025543665862, 77.3558853986883, 77.16603382809804, 77.06247842595789, 76.78633068691751, 76.49292371418709, 76.30307214359682, 75.99240593717639, 75.71625819813602, 75.52640662754574, 75.37107352433551, 75.02588885053504, 74.80151881256472, 74.59440800828443, 74.2147048671039, 73.7832240248533, 73.31722471522265, 72.98929927511219, 72.50604073179151, 72.02278218847084, 71.43596824301002, 70.90093199861927, 70.48671039005868, 70.05522954780807, 69.38211943389715, 68.69175008629618, 68.12219537452538, 67.62167759751468, 67.01760441836383, 66.24093890231273, 65.68864342423197, 65.17086641353124, 64.54953400069037, 63.9972385226096, 63.22057300655851, 62.44390749050742, 61.71901967552641, 61.04590956161546, 60.200207110804286, 59.31998619261305, 58.439765274421816, 57.62858129099069, 56.748360372799446, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 81.31515360717985, "exact_area100": 75.30036696574331}}, "old_normalized_endpoint_budget": 0}, {"seed": 2027, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.0, "validation_area100": 0.8637583333333334, "training_seconds": 2.916290438035503, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.008533324114978313, "operator": "ge"}, {"budget": 0.02, "threshold": -0.011071290820837021, "operator": "ge"}, {"budget": 0.03, "threshold": -0.01401523221284151, "operator": "ge"}, {"budget": 0.04, "threshold": -0.016187084838747978, "operator": "ge"}, {"budget": 0.05, "threshold": -0.019244296476244926, "operator": "ge"}, {"budget": 0.06, "threshold": -0.020807063207030296, "operator": "ge"}, {"budget": 0.07, "threshold": -0.024763597175478935, "operator": "ge"}, {"budget": 0.08, "threshold": -0.028510579839348793, "operator": "ge"}, {"budget": 0.09, "threshold": -0.032193977385759354, "operator": "ge"}, {"budget": 0.1, "threshold": -0.03540804237127304, "operator": "ge"}, {"budget": 0.11, "threshold": -0.037702783942222595, "operator": "ge"}, {"budget": 0.12, "threshold": -0.04150714725255966, "operator": "ge"}, {"budget": 0.13, "threshold": -0.04477820172905922, "operator": "ge"}, {"budget": 0.14, "threshold": -0.05058901384472847, "operator": "ge"}, {"budget": 0.15, "threshold": -0.052698828279972076, "operator": "ge"}, {"budget": 0.16, "threshold": -0.05720901861786842, "operator": "ge"}, {"budget": 0.17, "threshold": -0.06380543112754822, "operator": "ge"}, {"budget": 0.18, "threshold": -0.06863326579332352, "operator": "ge"}, {"budget": 0.19, "threshold": -0.07638317346572876, "operator": "ge"}, {"budget": 0.2, "threshold": -0.0806146115064621, "operator": "ge"}, {"budget": 0.21, "threshold": -0.0875542089343071, "operator": "ge"}, {"budget": 0.22, "threshold": -0.09239976108074188, "operator": "ge"}, {"budget": 0.23, "threshold": -0.09768571704626083, "operator": "ge"}, {"budget": 0.24, "threshold": -0.10350550711154938, "operator": "ge"}, {"budget": 0.25, "threshold": -0.11123152822256088, "operator": "ge"}, {"budget": 0.26, "threshold": -0.11538933962583542, "operator": "ge"}, {"budget": 0.27, "threshold": -0.12358158081769943, "operator": "ge"}, {"budget": 0.28, "threshold": -0.13256029784679413, "operator": "ge"}, {"budget": 0.29, "threshold": -0.14273002743721008, "operator": "ge"}, {"budget": 0.3, "threshold": -0.15074177086353302, "operator": "ge"}, {"budget": 0.31, "threshold": -0.15936650335788727, "operator": "ge"}, {"budget": 0.32, "threshold": -0.17102108895778656, "operator": "ge"}, {"budget": 0.33, "threshold": -0.176812544465065, "operator": "ge"}, {"budget": 0.34, "threshold": -0.1928248107433319, "operator": "ge"}, {"budget": 0.35, "threshold": -0.20769454538822174, "operator": "ge"}, {"budget": 0.36, "threshold": -0.22253963351249695, "operator": "ge"}, {"budget": 0.37, "threshold": -0.23989716172218323, "operator": "ge"}, {"budget": 0.38, "threshold": -0.25715187191963196, "operator": "ge"}, {"budget": 0.39, "threshold": -0.2724452614784241, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2804640531539917, "operator": "ge"}, {"budget": 0.41, "threshold": -0.3021600842475891, "operator": "ge"}, {"budget": 0.42, "threshold": -0.313558429479599, "operator": "ge"}, {"budget": 0.43, "threshold": -0.33909738063812256, "operator": "ge"}, {"budget": 0.44, "threshold": -0.3538327217102051, "operator": "ge"}, {"budget": 0.45, "threshold": -0.369531512260437, "operator": "ge"}, {"budget": 0.46, "threshold": -0.38212940096855164, "operator": "ge"}, {"budget": 0.47, "threshold": -0.39654847979545593, "operator": "ge"}, {"budget": 0.48, "threshold": -0.41025272011756897, "operator": "ge"}, {"budget": 0.49, "threshold": -0.42480167746543884, "operator": "ge"}, {"budget": 0.5, "threshold": -0.4341939687728882, "operator": "ge"}, {"budget": 0.51, "threshold": -0.44844329357147217, "operator": "ge"}, {"budget": 0.52, "threshold": -0.45461800694465637, "operator": "ge"}, {"budget": 0.53, "threshold": -0.46840736269950867, "operator": "ge"}, {"budget": 0.54, "threshold": -0.47916990518569946, "operator": "ge"}, {"budget": 0.55, "threshold": -0.49844756722450256, "operator": "ge"}, {"budget": 0.56, "threshold": -0.5077692866325378, "operator": "ge"}, {"budget": 0.57, "threshold": -0.5191248655319214, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5315659046173096, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5395240187644958, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5551988482475281, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5704793930053711, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5789745450019836, "operator": "ge"}, {"budget": 0.63, "threshold": -0.593044102191925, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6083297729492188, "operator": "ge"}, {"budget": 0.65, "threshold": -0.619415819644928, "operator": "ge"}, {"budget": 0.66, "threshold": -0.63303142786026, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6495474576950073, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6648348569869995, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6806942224502563, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6954341530799866, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7119276523590088, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7257133722305298, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7372979521751404, "operator": "ge"}, {"budget": 0.74, "threshold": -0.747248113155365, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7537643313407898, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7650197744369507, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7713863253593445, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7822249531745911, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8038679361343384, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8160618543624878, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8207222819328308, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8269317746162415, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8346217274665833, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8478053212165833, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8601889610290527, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8696433305740356, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8810157179832458, "operator": "ge"}, {"budget": 0.88, "threshold": -0.886795163154602, "operator": "ge"}, {"budget": 0.89, "threshold": -0.893090009689331, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9120025634765625, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9211760759353638, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9263505935668945, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9383871555328369, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9444429278373718, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9502613544464111, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9563841819763184, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9649584293365479, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9696553945541382, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9748005270957947, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.5, 98.5, 98.41666666666666, 98.33333333333334, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333334, 97.83333333333334, 97.83333333333334, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.5, 97.16666666666667, 97.0, 96.58333333333333, 96.33333333333334, 96.0, 95.58333333333333, 95.33333333333334, 95.0, 94.75, 94.66666666666667, 94.5, 94.41666666666667, 94.0, 93.91666666666667, 93.33333333333333, 92.91666666666667, 92.33333333333333, 91.91666666666667, 91.75, 91.25, 90.91666666666667, 90.66666666666667, 89.91666666666667, 89.5, 89.0, 88.5, 87.91666666666667, 87.5, 87.08333333333333, 86.5, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.75, 83.25, 82.66666666666667, 82.0, 81.33333333333333, 80.5, 80.0, 79.41666666666667, 78.66666666666666, 78.08333333333334, 77.33333333333333, 76.5, 75.66666666666667, 74.75, 74.0, 73.16666666666667, 72.25, 71.50000000000001, 70.5, 70.0, 69.16666666666667, 68.16666666666666, 67.33333333333333, 66.33333333333333, 65.5, 64.75, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 57.00000000000001, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 3, 5, 6, 7, 8, 8, 9, 9, 9, 10, 11, 11, 11, 12, 13, 13, 17, 19, 24, 27, 31, 36, 39, 43, 46, 47, 50, 51, 56, 57, 64, 69, 76, 81, 83, 89, 93, 96, 105, 110, 117, 123, 130, 135, 140, 147, 154, 161, 170, 180, 186, 193, 201, 209, 219, 225, 232, 241, 248, 257, 267, 277, 288, 297, 307, 318, 327, 339, 345, 355, 367, 377, 389, 399, 408, 420, 431, 443, 455, 467, 479, 491, 501, 512, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.5, 98.5, 98.41666666666666, 98.33333333333334, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333334, 97.83333333333334, 97.83333333333334, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.5, 97.16666666666667, 97.0, 96.58333333333333, 96.33333333333334, 96.0, 95.58333333333333, 95.33333333333334, 95.0, 94.75, 94.66666666666667, 94.5, 94.41666666666667, 94.0, 93.91666666666667, 93.33333333333333, 92.91666666666667, 92.33333333333333, 91.91666666666667, 91.75, 91.25, 90.91666666666667, 90.66666666666667, 89.91666666666667, 89.5, 89.0, 88.41666666666667, 87.91666666666667, 87.5, 87.08333333333333, 86.5, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.75, 83.25, 82.66666666666667, 82.0, 81.33333333333333, 80.5, 80.0, 79.41666666666667, 78.66666666666666, 78.08333333333334, 77.33333333333333, 76.5, 75.66666666666667, 74.75, 74.0, 73.16666666666667, 72.25, 71.41666666666666, 70.5, 70.0, 69.16666666666667, 68.16666666666666, 67.33333333333333, 66.33333333333333, 65.5, 64.75, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 57.00000000000001, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.37583333333335, "threshold_area100": 86.3775, "threshold_area25": 98.31666666666666}, "test": {"score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.34276838108389, 81.32550914739386, 81.30824991370383, 81.27373144632378, 81.27373144632378, 81.23921297894374, 81.20469451156369, 81.17017604418363, 81.17017604418363, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.04936140835348, 81.01484294097342, 80.98032447359338, 80.91128753883328, 80.84225060407319, 80.75595443562305, 80.73869520193303, 80.68691750086296, 80.70417673455299, 80.68691750086296, 80.65239903348291, 80.39351052813255, 80.2899551259924, 80.13462202278218, 80.06558508802209, 80.06558508802209, 80.03106662064205, 80.01380738695202, 79.87573351743183, 79.87573351743183, 79.8584742837418, 79.73765964791163, 79.65136347946151, 79.37521574042113, 79.16810493614084, 79.08180876769072, 78.77114256127028, 78.68484639282016, 78.47773558853987, 78.40869865377978, 78.13255091473938, 77.97721781152917, 77.37314463237833, 77.04521919226787, 76.80358992060754, 76.49292371418709, 76.40662754573697, 76.06144287193649, 75.94062823610632, 75.76803589920608, 75.47462892647566, 74.93959268208492, 74.31826026924405, 73.97307559544356, 73.31722471522265, 72.7821884708319, 71.98826372109079, 71.1425612702796, 70.26234035208837, 69.6410079392475, 68.98515705902658, 68.570935450466, 68.26026924404557, 67.65619606489473, 67.25923369002416, 66.39627200552296, 65.25716258198136, 64.51501553331033, 64.25612702795996, 63.962720055229546, 63.53123921297894, 62.823610631687956, 62.18501898515706, 61.615464273386266, 61.09768726268554, 60.71798412150501, 60.33828098032448, 59.302726958923024, 58.802209181912325, 58.54332067656196, 57.99102519848118, 57.54228512254056, 57.12806351397999, 56.73110113910942, 55.988954090438384, 55.5056955471177, 55.16051087331723, 54.81532619951675], "calls": [0.0, 0.6731101139109423, 1.4325163962720056, 2.4853296513634793, 3.296513634794615, 4.124956851915775, 4.470141525716258, 5.26406627545737, 5.937176389368312, 6.627545736969279, 7.145322747670003, 7.611322057300655, 8.318950638591646, 9.009319986192613, 10.027614773904038, 10.476354849844666, 11.253020365895754, 12.51294442526752, 13.237832240248535, 14.359682430100104, 14.946496375560924, 15.982050396962375, 16.724197445633415, 17.53538142906455, 18.20849154297549, 19.27856403175699, 19.65826717293752, 20.590265792198824, 21.625819813600273, 22.59233690024163, 23.110113910942353, 23.852260959613396, 25.07766655160511, 25.62996202968588, 27.114256127027957, 28.684846392820155, 30.203658957542284, 31.4981014842941, 32.9306178805661, 34.03520883672765, 34.69105971694857, 36.41698308595098, 37.573351743182606, 39.74801518812565, 41.24956851915775, 42.7856403175699, 44.011045909561616, 45.564376941663795, 46.996893337935795, 48.394891266827756, 49.413186054539175, 50.94925785295133, 51.43251639627201, 52.81325509147394, 54.36658612357611, 56.55850880220918, 57.594062823610635, 58.68139454608215, 59.924059371763896, 60.71798412150501, 62.39212978943735, 63.80738695201933, 64.58405246807042, 65.80945806006213, 67.32827062478425, 68.72626855367622, 69.96893337935795, 71.33241284086986, 72.85122540559199, 74.64618570935451, 76.38936831204694, 78.02899551259924, 79.27166033828098, 80.41076976182258, 81.27373144632378, 81.75698998964445, 82.68898860890576, 83.39661719019675, 84.51846738004832, 86.41698308595099, 87.5733517431826, 87.98757335174318, 88.36727649292372, 88.9886089057646, 90.09319986192614, 91.18053158439766, 91.87090093199862, 92.68208491542975, 93.19986192613048, 93.66586123576113, 95.01208146358302, 95.77148774594409, 96.04763548498447, 96.68622713151535, 97.13496720745599, 97.5664480497066, 97.9979288919572, 98.75733517431826, 99.25785295132896, 99.6375560925095, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 2, 3, 3, 5, 5, 6, 6, 10, 10, 14, 18, 18, 20, 26, 32, 37, 45, 51, 55, 61, 68, 83, 91, 100, 104, 115, 126, 135, 141, 153, 156, 162, 174, 189, 194, 199, 207, 212, 222, 241, 247, 259, 267, 272, 280, 288, 295, 305, 312, 317, 320, 323, 328, 328, 330, 332, 335, 342, 346, 347, 347, 347, 349, 354, 355, 357, 360, 361, 363, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 1, 1, 1, 1, 1, 2, 3, 3, 4, 4, 5, 6, 7, 9, 9, 11, 13, 15, 17, 18, 19, 21, 25, 27, 31, 35, 40, 45, 50, 53, 56, 61, 63, 80, 92, 107, 116, 124, 132, 137, 151, 158, 174, 189, 203, 223, 246, 262, 289, 300, 324, 331, 353, 374, 424, 448, 467, 493, 503, 533, 559, 575, 604, 643, 684, 712, 758, 796, 852, 908, 964, 1003, 1044, 1073, 1091, 1128, 1153, 1206, 1279, 1326, 1342, 1359, 1384, 1427, 1469, 1503, 1535, 1560, 1583, 1645, 1676, 1691, 1723, 1749, 1773, 1796, 1839, 1867, 1887, 1907], "exact_score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.18743527787366, 81.17017604418363, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.01484294097342, 80.9975837072834, 80.87676907145323, 80.82499137038316, 80.77321366931308, 80.70417673455299, 80.68691750086296, 80.68691750086296, 80.56610286503279, 80.41076976182258, 80.30721435968243, 80.23817742492233, 80.11736278909216, 80.10010355540214, 80.03106662064205, 80.08284432171212, 80.01380738695202, 79.996548153262, 79.996548153262, 79.89299275112185, 79.91025198481188, 79.8584742837418, 79.84121505005179, 79.77217811529168, 79.70314118053159, 79.58232654470142, 79.39247497411115, 79.27166033828098, 79.18536416983086, 79.08180876769072, 78.80566102865033, 78.77114256127028, 78.65032792544011, 78.47773558853987, 78.30514325163963, 78.11529168104936, 78.06351397997929, 77.89092164307905, 77.58025543665862, 77.25232999654816, 76.95892302381775, 76.71729375215742, 76.51018294787711, 76.33759061097687, 76.13047980669658, 76.04418363824647, 75.88885053503624, 75.71625819813602, 75.35381429064549, 75.0604073179151, 74.5771487745944, 74.18018639972385, 73.97307559544356, 73.43803935105282, 73.12737314463237, 72.71315153607179, 72.21263375906109, 71.7811529168105, 71.2806351397998, 70.84915429754919, 70.2795995857784, 69.74456334138765, 69.26130479806697, 68.65723161891613, 68.1394546082154, 67.53538142906454, 66.8622713151536, 66.10286503279255, 65.5678287884018, 64.94649637556093, 64.25612702795996, 63.53123921297894, 62.90990680013808, 62.2885743872972, 61.52916810493614, 60.87331722471523, 60.09665170866414, 59.31998619261305, 58.59509837763204, 57.680358992060754, 56.73110113910942, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.25180638106697, "threshold_area100": 74.23696927856402, "threshold_area25": 81.28063513979981}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.008533324114978313, "operator": "ge"}, {"budget": 0.02, "threshold": -0.011071290820837021, "operator": "ge"}, {"budget": 0.03, "threshold": -0.01401523221284151, "operator": "ge"}, {"budget": 0.04, "threshold": -0.016187084838747978, "operator": "ge"}, {"budget": 0.05, "threshold": -0.019244296476244926, "operator": "ge"}, {"budget": 0.06, "threshold": -0.020807063207030296, "operator": "ge"}, {"budget": 0.07, "threshold": -0.024763597175478935, "operator": "ge"}, {"budget": 0.08, "threshold": -0.028510579839348793, "operator": "ge"}, {"budget": 0.09, "threshold": -0.032193977385759354, "operator": "ge"}, {"budget": 0.1, "threshold": -0.03540804237127304, "operator": "ge"}, {"budget": 0.11, "threshold": -0.037702783942222595, "operator": "ge"}, {"budget": 0.12, "threshold": -0.04150714725255966, "operator": "ge"}, {"budget": 0.13, "threshold": -0.04477820172905922, "operator": "ge"}, {"budget": 0.14, "threshold": -0.05058901384472847, "operator": "ge"}, {"budget": 0.15, "threshold": -0.052698828279972076, "operator": "ge"}, {"budget": 0.16, "threshold": -0.05720901861786842, "operator": "ge"}, {"budget": 0.17, "threshold": -0.06380543112754822, "operator": "ge"}, {"budget": 0.18, "threshold": -0.06863326579332352, "operator": "ge"}, {"budget": 0.19, "threshold": -0.07638317346572876, "operator": "ge"}, {"budget": 0.2, "threshold": -0.0806146115064621, "operator": "ge"}, {"budget": 0.21, "threshold": -0.0875542089343071, "operator": "ge"}, {"budget": 0.22, "threshold": -0.09239976108074188, "operator": "ge"}, {"budget": 0.23, "threshold": -0.09768571704626083, "operator": "ge"}, {"budget": 0.24, "threshold": -0.10350550711154938, "operator": "ge"}, {"budget": 0.25, "threshold": -0.11123152822256088, "operator": "ge"}, {"budget": 0.26, "threshold": -0.11538933962583542, "operator": "ge"}, {"budget": 0.27, "threshold": -0.12358158081769943, "operator": "ge"}, {"budget": 0.28, "threshold": -0.13256029784679413, "operator": "ge"}, {"budget": 0.29, "threshold": -0.14273002743721008, "operator": "ge"}, {"budget": 0.3, "threshold": -0.15074177086353302, "operator": "ge"}, {"budget": 0.31, "threshold": -0.15936650335788727, "operator": "ge"}, {"budget": 0.32, "threshold": -0.17102108895778656, "operator": "ge"}, {"budget": 0.33, "threshold": -0.176812544465065, "operator": "ge"}, {"budget": 0.34, "threshold": -0.1928248107433319, "operator": "ge"}, {"budget": 0.35, "threshold": -0.20769454538822174, "operator": "ge"}, {"budget": 0.36, "threshold": -0.22253963351249695, "operator": "ge"}, {"budget": 0.37, "threshold": -0.23989716172218323, "operator": "ge"}, {"budget": 0.38, "threshold": -0.25715187191963196, "operator": "ge"}, {"budget": 0.39, "threshold": -0.2724452614784241, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2804640531539917, "operator": "ge"}, {"budget": 0.41, "threshold": -0.3021600842475891, "operator": "ge"}, {"budget": 0.42, "threshold": -0.313558429479599, "operator": "ge"}, {"budget": 0.43, "threshold": -0.33909738063812256, "operator": "ge"}, {"budget": 0.44, "threshold": -0.3538327217102051, "operator": "ge"}, {"budget": 0.45, "threshold": -0.369531512260437, "operator": "ge"}, {"budget": 0.46, "threshold": -0.38212940096855164, "operator": "ge"}, {"budget": 0.47, "threshold": -0.39654847979545593, "operator": "ge"}, {"budget": 0.48, "threshold": -0.41025272011756897, "operator": "ge"}, {"budget": 0.49, "threshold": -0.42480167746543884, "operator": "ge"}, {"budget": 0.5, "threshold": -0.4341939687728882, "operator": "ge"}, {"budget": 0.51, "threshold": -0.44844329357147217, "operator": "ge"}, {"budget": 0.52, "threshold": -0.45461800694465637, "operator": "ge"}, {"budget": 0.53, "threshold": -0.46840736269950867, "operator": "ge"}, {"budget": 0.54, "threshold": -0.47916990518569946, "operator": "ge"}, {"budget": 0.55, "threshold": -0.49844756722450256, "operator": "ge"}, {"budget": 0.56, "threshold": -0.5077692866325378, "operator": "ge"}, {"budget": 0.57, "threshold": -0.5191248655319214, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5315659046173096, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5395240187644958, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5551988482475281, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5704793930053711, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5789745450019836, "operator": "ge"}, {"budget": 0.63, "threshold": -0.593044102191925, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6083297729492188, "operator": "ge"}, {"budget": 0.65, "threshold": -0.619415819644928, "operator": "ge"}, {"budget": 0.66, "threshold": -0.63303142786026, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6495474576950073, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6648348569869995, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6806942224502563, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6954341530799866, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7119276523590088, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7257133722305298, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7372979521751404, "operator": "ge"}, {"budget": 0.74, "threshold": -0.747248113155365, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7537643313407898, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7650197744369507, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7713863253593445, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7822249531745911, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8038679361343384, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8160618543624878, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8207222819328308, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8269317746162415, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8346217274665833, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8478053212165833, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8601889610290527, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8696433305740356, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8810157179832458, "operator": "ge"}, {"budget": 0.88, "threshold": -0.886795163154602, "operator": "ge"}, {"budget": 0.89, "threshold": -0.893090009689331, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9120025634765625, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9211760759353638, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9263505935668945, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9383871555328369, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9444429278373718, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9502613544464111, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9563841819763184, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9649584293365479, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9696553945541382, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9748005270957947, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.5, 98.5, 98.41666666666666, 98.33333333333334, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333334, 97.83333333333334, 97.83333333333334, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.5, 97.16666666666667, 97.0, 96.58333333333333, 96.33333333333334, 96.0, 95.58333333333333, 95.33333333333334, 95.0, 94.75, 94.66666666666667, 94.5, 94.41666666666667, 94.0, 93.91666666666667, 93.33333333333333, 92.91666666666667, 92.33333333333333, 91.91666666666667, 91.75, 91.25, 90.91666666666667, 90.66666666666667, 89.91666666666667, 89.5, 89.0, 88.5, 87.91666666666667, 87.5, 87.08333333333333, 86.5, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.75, 83.25, 82.66666666666667, 82.0, 81.33333333333333, 80.5, 80.0, 79.41666666666667, 78.66666666666666, 78.08333333333334, 77.33333333333333, 76.5, 75.66666666666667, 74.75, 74.0, 73.16666666666667, 72.25, 71.50000000000001, 70.5, 70.0, 69.16666666666667, 68.16666666666666, 67.33333333333333, 66.33333333333333, 65.5, 64.75, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 57.00000000000001, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 3, 5, 6, 7, 8, 8, 9, 9, 9, 10, 11, 11, 11, 12, 13, 13, 17, 19, 24, 27, 31, 36, 39, 43, 46, 47, 50, 51, 56, 57, 64, 69, 76, 81, 83, 89, 93, 96, 105, 110, 117, 123, 130, 135, 140, 147, 154, 161, 170, 180, 186, 193, 201, 209, 219, 225, 232, 241, 248, 257, 267, 277, 288, 297, 307, 318, 327, 339, 345, 355, 367, 377, 389, 399, 408, 420, 431, 443, 455, 467, 479, 491, 501, 512, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.5, 98.5, 98.41666666666666, 98.33333333333334, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333334, 97.83333333333334, 97.83333333333334, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.5, 97.16666666666667, 97.0, 96.58333333333333, 96.33333333333334, 96.0, 95.58333333333333, 95.33333333333334, 95.0, 94.75, 94.66666666666667, 94.5, 94.41666666666667, 94.0, 93.91666666666667, 93.33333333333333, 92.91666666666667, 92.33333333333333, 91.91666666666667, 91.75, 91.25, 90.91666666666667, 90.66666666666667, 89.91666666666667, 89.5, 89.0, 88.41666666666667, 87.91666666666667, 87.5, 87.08333333333333, 86.5, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.75, 83.25, 82.66666666666667, 82.0, 81.33333333333333, 80.5, 80.0, 79.41666666666667, 78.66666666666666, 78.08333333333334, 77.33333333333333, 76.5, 75.66666666666667, 74.75, 74.0, 73.16666666666667, 72.25, 71.41666666666666, 70.5, 70.0, 69.16666666666667, 68.16666666666666, 67.33333333333333, 66.33333333333333, 65.5, 64.75, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 57.00000000000001, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.37583333333335, "threshold_area100": 86.3775, "threshold_area25": 98.31666666666666}, "test": {"score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.34276838108389, 81.32550914739386, 81.30824991370383, 81.27373144632378, 81.27373144632378, 81.23921297894374, 81.20469451156369, 81.17017604418363, 81.17017604418363, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.04936140835348, 81.01484294097342, 80.98032447359338, 80.91128753883328, 80.84225060407319, 80.75595443562305, 80.73869520193303, 80.68691750086296, 80.70417673455299, 80.68691750086296, 80.65239903348291, 80.39351052813255, 80.2899551259924, 80.13462202278218, 80.06558508802209, 80.06558508802209, 80.03106662064205, 80.01380738695202, 79.87573351743183, 79.87573351743183, 79.8584742837418, 79.73765964791163, 79.65136347946151, 79.37521574042113, 79.16810493614084, 79.08180876769072, 78.77114256127028, 78.68484639282016, 78.47773558853987, 78.40869865377978, 78.13255091473938, 77.97721781152917, 77.37314463237833, 77.04521919226787, 76.80358992060754, 76.49292371418709, 76.40662754573697, 76.06144287193649, 75.94062823610632, 75.76803589920608, 75.47462892647566, 74.93959268208492, 74.31826026924405, 73.97307559544356, 73.31722471522265, 72.7821884708319, 71.98826372109079, 71.1425612702796, 70.26234035208837, 69.6410079392475, 68.98515705902658, 68.570935450466, 68.26026924404557, 67.65619606489473, 67.25923369002416, 66.39627200552296, 65.25716258198136, 64.51501553331033, 64.25612702795996, 63.962720055229546, 63.53123921297894, 62.823610631687956, 62.18501898515706, 61.615464273386266, 61.09768726268554, 60.71798412150501, 60.33828098032448, 59.302726958923024, 58.802209181912325, 58.54332067656196, 57.99102519848118, 57.54228512254056, 57.12806351397999, 56.73110113910942, 55.988954090438384, 55.5056955471177, 55.16051087331723, 54.81532619951675], "calls": [0.0, 0.6731101139109423, 1.4325163962720056, 2.4853296513634793, 3.296513634794615, 4.124956851915775, 4.470141525716258, 5.26406627545737, 5.937176389368312, 6.627545736969279, 7.145322747670003, 7.611322057300655, 8.318950638591646, 9.009319986192613, 10.027614773904038, 10.476354849844666, 11.253020365895754, 12.51294442526752, 13.237832240248535, 14.359682430100104, 14.946496375560924, 15.982050396962375, 16.724197445633415, 17.53538142906455, 18.20849154297549, 19.27856403175699, 19.65826717293752, 20.590265792198824, 21.625819813600273, 22.59233690024163, 23.110113910942353, 23.852260959613396, 25.07766655160511, 25.62996202968588, 27.114256127027957, 28.684846392820155, 30.203658957542284, 31.4981014842941, 32.9306178805661, 34.03520883672765, 34.69105971694857, 36.41698308595098, 37.573351743182606, 39.74801518812565, 41.24956851915775, 42.7856403175699, 44.011045909561616, 45.564376941663795, 46.996893337935795, 48.394891266827756, 49.413186054539175, 50.94925785295133, 51.43251639627201, 52.81325509147394, 54.36658612357611, 56.55850880220918, 57.594062823610635, 58.68139454608215, 59.924059371763896, 60.71798412150501, 62.39212978943735, 63.80738695201933, 64.58405246807042, 65.80945806006213, 67.32827062478425, 68.72626855367622, 69.96893337935795, 71.33241284086986, 72.85122540559199, 74.64618570935451, 76.38936831204694, 78.02899551259924, 79.27166033828098, 80.41076976182258, 81.27373144632378, 81.75698998964445, 82.68898860890576, 83.39661719019675, 84.51846738004832, 86.41698308595099, 87.5733517431826, 87.98757335174318, 88.36727649292372, 88.9886089057646, 90.09319986192614, 91.18053158439766, 91.87090093199862, 92.68208491542975, 93.19986192613048, 93.66586123576113, 95.01208146358302, 95.77148774594409, 96.04763548498447, 96.68622713151535, 97.13496720745599, 97.5664480497066, 97.9979288919572, 98.75733517431826, 99.25785295132896, 99.6375560925095, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 2, 3, 3, 5, 5, 6, 6, 10, 10, 14, 18, 18, 20, 26, 32, 37, 45, 51, 55, 61, 68, 83, 91, 100, 104, 115, 126, 135, 141, 153, 156, 162, 174, 189, 194, 199, 207, 212, 222, 241, 247, 259, 267, 272, 280, 288, 295, 305, 312, 317, 320, 323, 328, 328, 330, 332, 335, 342, 346, 347, 347, 347, 349, 354, 355, 357, 360, 361, 363, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 1, 1, 1, 1, 1, 2, 3, 3, 4, 4, 5, 6, 7, 9, 9, 11, 13, 15, 17, 18, 19, 21, 25, 27, 31, 35, 40, 45, 50, 53, 56, 61, 63, 80, 92, 107, 116, 124, 132, 137, 151, 158, 174, 189, 203, 223, 246, 262, 289, 300, 324, 331, 353, 374, 424, 448, 467, 493, 503, 533, 559, 575, 604, 643, 684, 712, 758, 796, 852, 908, 964, 1003, 1044, 1073, 1091, 1128, 1153, 1206, 1279, 1326, 1342, 1359, 1384, 1427, 1469, 1503, 1535, 1560, 1583, 1645, 1676, 1691, 1723, 1749, 1773, 1796, 1839, 1867, 1887, 1907], "exact_score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.18743527787366, 81.17017604418363, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.01484294097342, 80.9975837072834, 80.87676907145323, 80.82499137038316, 80.77321366931308, 80.70417673455299, 80.68691750086296, 80.68691750086296, 80.56610286503279, 80.41076976182258, 80.30721435968243, 80.23817742492233, 80.11736278909216, 80.10010355540214, 80.03106662064205, 80.08284432171212, 80.01380738695202, 79.996548153262, 79.996548153262, 79.89299275112185, 79.91025198481188, 79.8584742837418, 79.84121505005179, 79.77217811529168, 79.70314118053159, 79.58232654470142, 79.39247497411115, 79.27166033828098, 79.18536416983086, 79.08180876769072, 78.80566102865033, 78.77114256127028, 78.65032792544011, 78.47773558853987, 78.30514325163963, 78.11529168104936, 78.06351397997929, 77.89092164307905, 77.58025543665862, 77.25232999654816, 76.95892302381775, 76.71729375215742, 76.51018294787711, 76.33759061097687, 76.13047980669658, 76.04418363824647, 75.88885053503624, 75.71625819813602, 75.35381429064549, 75.0604073179151, 74.5771487745944, 74.18018639972385, 73.97307559544356, 73.43803935105282, 73.12737314463237, 72.71315153607179, 72.21263375906109, 71.7811529168105, 71.2806351397998, 70.84915429754919, 70.2795995857784, 69.74456334138765, 69.26130479806697, 68.65723161891613, 68.1394546082154, 67.53538142906454, 66.8622713151536, 66.10286503279255, 65.5678287884018, 64.94649637556093, 64.25612702795996, 63.53123921297894, 62.90990680013808, 62.2885743872972, 61.52916810493614, 60.87331722471523, 60.09665170866414, 59.31998619261305, 58.59509837763204, 57.680358992060754, 56.73110113910942, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.25180638106697, "threshold_area100": 74.23696927856402, "threshold_area25": 81.28063513979981}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.2460412979125977, "operator": "ge"}, {"budget": 0.02, "threshold": -2.5209462642669678, "operator": "ge"}, {"budget": 0.03, "threshold": -3.040034770965576, "operator": "ge"}, {"budget": 0.04, "threshold": -3.2093536853790283, "operator": "ge"}, {"budget": 0.05, "threshold": -3.440647602081299, "operator": "ge"}, {"budget": 0.06, "threshold": -3.723665237426758, "operator": "ge"}, {"budget": 0.07, "threshold": -3.976388692855835, "operator": "ge"}, {"budget": 0.08, "threshold": -4.302084922790527, "operator": "ge"}, {"budget": 0.09, "threshold": -4.523153305053711, "operator": "ge"}, {"budget": 0.1, "threshold": -4.604337692260742, "operator": "ge"}, {"budget": 0.11, "threshold": -4.743232250213623, "operator": "ge"}, {"budget": 0.12, "threshold": -4.940221786499023, "operator": "ge"}, {"budget": 0.13, "threshold": -5.075122356414795, "operator": "ge"}, {"budget": 0.14, "threshold": -5.232641220092773, "operator": "ge"}, {"budget": 0.15, "threshold": -5.428228378295898, "operator": "ge"}, {"budget": 0.16, "threshold": -5.728024005889893, "operator": "ge"}, {"budget": 0.17, "threshold": -6.013313293457031, "operator": "ge"}, {"budget": 0.18, "threshold": -6.336449146270752, "operator": "ge"}, {"budget": 0.19, "threshold": -6.493381023406982, "operator": "ge"}, {"budget": 0.2, "threshold": -6.779406547546387, "operator": "ge"}, {"budget": 0.21, "threshold": -6.891701698303223, "operator": "ge"}, {"budget": 0.22, "threshold": -7.099642276763916, "operator": "ge"}, {"budget": 0.23, "threshold": -7.299744606018066, "operator": "ge"}, {"budget": 0.24, "threshold": -7.435730934143066, "operator": "ge"}, {"budget": 0.25, "threshold": -7.709187984466553, "operator": "ge"}, {"budget": 0.26, "threshold": -7.972471237182617, "operator": "ge"}, {"budget": 0.27, "threshold": -8.087268829345703, "operator": "ge"}, {"budget": 0.28, "threshold": -8.215205192565918, "operator": "ge"}, {"budget": 0.29, "threshold": -8.25761890411377, "operator": "ge"}, {"budget": 0.3, "threshold": -8.295166969299316, "operator": "ge"}, {"budget": 0.31, "threshold": -8.39441967010498, "operator": "ge"}, {"budget": 0.32, "threshold": -8.481338500976562, "operator": "ge"}, {"budget": 0.33, "threshold": -8.574865341186523, "operator": "ge"}, {"budget": 0.34, "threshold": -8.657732963562012, "operator": "ge"}, {"budget": 0.35, "threshold": -8.688224792480469, "operator": "ge"}, {"budget": 0.36, "threshold": -8.735235214233398, "operator": "ge"}, {"budget": 0.37, "threshold": -8.785056114196777, "operator": "ge"}, {"budget": 0.38, "threshold": -8.829675674438477, "operator": "ge"}, {"budget": 0.39, "threshold": -8.889436721801758, "operator": "ge"}, {"budget": 0.4, "threshold": -8.916549682617188, "operator": "ge"}, {"budget": 0.41, "threshold": -8.949319839477539, "operator": "ge"}, {"budget": 0.42, "threshold": -8.983076095581055, "operator": "ge"}, {"budget": 0.43, "threshold": -9.010019302368164, "operator": "ge"}, {"budget": 0.44, "threshold": -9.044090270996094, "operator": "ge"}, {"budget": 0.45, "threshold": -9.083270072937012, "operator": "ge"}, {"budget": 0.46, "threshold": -9.115509033203125, "operator": "ge"}, {"budget": 0.47, "threshold": -9.152891159057617, "operator": "ge"}, {"budget": 0.48, "threshold": -9.173632621765137, "operator": "ge"}, {"budget": 0.49, "threshold": -9.21182632446289, "operator": "ge"}, {"budget": 0.5, "threshold": -9.245906829833984, "operator": "ge"}, {"budget": 0.51, "threshold": -9.267708778381348, "operator": "ge"}, {"budget": 0.52, "threshold": -9.319317817687988, "operator": "ge"}, {"budget": 0.53, "threshold": -9.358169555664062, "operator": "ge"}, {"budget": 0.54, "threshold": -9.387127876281738, "operator": "ge"}, {"budget": 0.55, "threshold": -9.41965103149414, "operator": "ge"}, {"budget": 0.56, "threshold": -9.453126907348633, "operator": "ge"}, {"budget": 0.57, "threshold": -9.476287841796875, "operator": "ge"}, {"budget": 0.58, "threshold": -9.505528450012207, "operator": "ge"}, {"budget": 0.59, "threshold": -9.535224914550781, "operator": "ge"}, {"budget": 0.6, "threshold": -9.559906959533691, "operator": "ge"}, {"budget": 0.61, "threshold": -9.588593482971191, "operator": "ge"}, {"budget": 0.62, "threshold": -9.618890762329102, "operator": "ge"}, {"budget": 0.63, "threshold": -9.644631385803223, "operator": "ge"}, {"budget": 0.64, "threshold": -9.666936874389648, "operator": "ge"}, {"budget": 0.65, "threshold": -9.695419311523438, "operator": "ge"}, {"budget": 0.66, "threshold": -9.715241432189941, "operator": "ge"}, {"budget": 0.67, "threshold": -9.739761352539062, "operator": "ge"}, {"budget": 0.68, "threshold": -9.788324356079102, "operator": "ge"}, {"budget": 0.69, "threshold": -9.810823440551758, "operator": "ge"}, {"budget": 0.7, "threshold": -9.83875560760498, "operator": "ge"}, {"budget": 0.71, "threshold": -9.85421371459961, "operator": "ge"}, {"budget": 0.72, "threshold": -9.88781452178955, "operator": "ge"}, {"budget": 0.73, "threshold": -9.919258117675781, "operator": "ge"}, {"budget": 0.74, "threshold": -9.958549499511719, "operator": "ge"}, {"budget": 0.75, "threshold": -9.973541259765625, "operator": "ge"}, {"budget": 0.76, "threshold": -10.016708374023438, "operator": "ge"}, {"budget": 0.77, "threshold": -10.044300079345703, "operator": "ge"}, {"budget": 0.78, "threshold": -10.070785522460938, "operator": "ge"}, {"budget": 0.79, "threshold": -10.109575271606445, "operator": "ge"}, {"budget": 0.8, "threshold": -10.158269882202148, "operator": "ge"}, {"budget": 0.81, "threshold": -10.196369171142578, "operator": "ge"}, {"budget": 0.82, "threshold": -10.231271743774414, "operator": "ge"}, {"budget": 0.83, "threshold": -10.278003692626953, "operator": "ge"}, {"budget": 0.84, "threshold": -10.309980392456055, "operator": "ge"}, {"budget": 0.85, "threshold": -10.329660415649414, "operator": "ge"}, {"budget": 0.86, "threshold": -10.382329940795898, "operator": "ge"}, {"budget": 0.87, "threshold": -10.43031120300293, "operator": "ge"}, {"budget": 0.88, "threshold": -10.47170352935791, "operator": "ge"}, {"budget": 0.89, "threshold": -10.549520492553711, "operator": "ge"}, {"budget": 0.9, "threshold": -10.625275611877441, "operator": "ge"}, {"budget": 0.91, "threshold": -10.69144344329834, "operator": "ge"}, {"budget": 0.92, "threshold": -10.76373291015625, "operator": "ge"}, {"budget": 0.93, "threshold": -10.836273193359375, "operator": "ge"}, {"budget": 0.94, "threshold": -10.90355110168457, "operator": "ge"}, {"budget": 0.95, "threshold": -10.977415084838867, "operator": "ge"}, {"budget": 0.96, "threshold": -11.04134750366211, "operator": "ge"}, {"budget": 0.97, "threshold": -11.183906555175781, "operator": "ge"}, {"budget": 0.98, "threshold": -11.313591957092285, "operator": "ge"}, {"budget": 0.99, "threshold": -11.513017654418945, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 97.91666666666666, 97.5, 97.25, 96.58333333333333, 96.08333333333333, 95.33333333333334, 94.75, 94.58333333333333, 94.33333333333334, 93.66666666666667, 93.16666666666666, 92.58333333333333, 91.83333333333333, 91.16666666666666, 90.33333333333333, 89.83333333333333, 89.16666666666667, 88.5, 88.0, 87.41666666666667, 86.91666666666666, 86.5, 86.08333333333333, 85.5, 85.08333333333333, 84.75, 84.08333333333333, 83.66666666666667, 83.41666666666667, 83.08333333333333, 82.66666666666667, 82.25, 81.91666666666667, 81.66666666666667, 81.33333333333333, 81.0, 80.33333333333333, 80.16666666666666, 79.66666666666666, 79.58333333333334, 79.5, 79.0, 78.58333333333334, 77.91666666666667, 77.58333333333334, 77.25, 77.08333333333334, 76.83333333333333, 76.58333333333334, 76.16666666666667, 75.5, 75.0, 74.91666666666667, 74.75, 74.33333333333334, 74.0, 73.58333333333333, 73.0, 72.66666666666667, 72.25, 71.91666666666667, 71.58333333333333, 71.25, 71.0, 70.66666666666667, 70.41666666666666, 70.08333333333334, 69.66666666666667, 69.33333333333334, 69.0, 68.58333333333333, 68.25, 67.83333333333333, 67.33333333333333, 67.16666666666666, 66.75, 66.5, 65.91666666666667, 65.75, 65.5, 65.0, 64.41666666666667, 63.83333333333334, 63.416666666666664, 62.74999999999999, 62.33333333333333, 61.916666666666664, 61.583333333333336, 60.916666666666664, 60.33333333333333, 59.75, 59.25, 58.833333333333336, 58.41666666666667, 58.00000000000001, 57.333333333333336, 56.75, 56.333333333333336, 55.666666666666664, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 8, 13, 16, 24, 30, 39, 46, 50, 53, 61, 67, 74, 83, 91, 101, 107, 115, 123, 129, 136, 142, 147, 152, 159, 164, 168, 176, 181, 184, 188, 193, 198, 202, 205, 209, 213, 221, 223, 229, 230, 231, 237, 242, 250, 254, 258, 260, 263, 266, 271, 279, 285, 286, 288, 293, 297, 302, 309, 313, 318, 322, 326, 330, 333, 337, 340, 344, 349, 353, 357, 362, 366, 371, 377, 379, 384, 387, 394, 396, 399, 405, 412, 419, 424, 432, 437, 442, 446, 454, 461, 468, 474, 479, 484, 489, 497, 504, 509, 517, 524], "exact_score": [98.58333333333333, 97.91666666666666, 97.5, 97.25, 96.58333333333333, 96.08333333333333, 95.33333333333334, 94.75, 94.58333333333333, 94.33333333333334, 93.66666666666667, 93.16666666666666, 92.58333333333333, 91.83333333333333, 91.16666666666666, 90.33333333333333, 89.83333333333333, 89.16666666666667, 88.5, 88.0, 87.41666666666667, 86.91666666666666, 86.5, 86.08333333333333, 85.5, 85.08333333333333, 84.75, 84.08333333333333, 83.66666666666667, 83.41666666666667, 83.08333333333333, 82.66666666666667, 82.25, 81.91666666666667, 81.66666666666667, 81.33333333333333, 81.0, 80.33333333333333, 80.16666666666666, 79.66666666666666, 79.58333333333334, 79.41666666666667, 79.0, 78.58333333333334, 77.91666666666667, 77.58333333333334, 77.25, 77.08333333333334, 76.83333333333333, 76.58333333333334, 76.16666666666667, 75.5, 75.0, 74.91666666666667, 74.75, 74.33333333333334, 74.0, 73.58333333333333, 73.0, 72.66666666666667, 72.25, 71.91666666666667, 71.58333333333333, 71.25, 71.0, 70.66666666666667, 70.41666666666666, 70.08333333333334, 69.66666666666667, 69.33333333333334, 69.0, 68.58333333333333, 68.25, 67.83333333333333, 67.33333333333333, 67.16666666666666, 66.75, 66.5, 65.91666666666667, 65.75, 65.5, 65.0, 64.41666666666667, 63.83333333333334, 63.416666666666664, 62.74999999999999, 62.33333333333333, 61.916666666666664, 61.583333333333336, 60.916666666666664, 60.33333333333333, 59.75, 59.25, 58.833333333333336, 58.41666666666667, 58.00000000000001, 57.333333333333336, 56.75, 56.333333333333336, 55.666666666666664, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 76.34249999999999, "threshold_area100": 76.34333333333333, "threshold_area25": 91.86666666666666}, "test": {"score": [81.429064549534, 81.44632378322403, 81.30824991370383, 80.44528822920263, 80.41076976182258, 79.96202968588196, 78.9955125992406, 77.99447704521918, 76.52744218156714, 75.47462892647566, 75.0776665516051, 74.43907490507422, 73.47255781843286, 72.97204004142216, 72.33344839489126, 71.69485674836038, 70.98722816706939, 70.2795995857784, 69.69278564031757, 69.52019330341733, 68.89886089057646, 68.7780462547463, 68.46738004832585, 68.2084915429755, 68.10493614083535, 67.67345529858476, 67.29375215740421, 67.15567828788403, 66.93130824991371, 66.89678978253366, 66.8622713151536, 66.67241974456334, 66.39627200552296, 66.2064204349327, 65.9302726958923, 65.7921988263721, 65.61960648947188, 65.55056955471177, 65.46427338626165, 65.2916810493614, 65.25716258198136, 65.11908871246118, 65.05005177770107, 64.98101484294098, 64.9292371418709, 64.68760787021056, 64.48049706593028, 64.30790472903003, 64.20434932688988, 64.03175698998965, 63.91094235415948, 63.85916465308941, 63.70383154987918, 63.375906109768735, 63.272350707628576, 63.16879530548844, 63.01346220227821, 62.823610631687956, 62.59924059371764, 62.426648256817394, 62.23679668622714, 62.09872281670694, 61.80531584397653, 61.66724197445633, 61.58094580600621, 61.37383500172593, 61.30479806696583, 61.14946496375562, 60.90783569209527, 60.73524335519503, 60.56265101829479, 60.510873317224714, 60.2865032792544, 60.13117017604419, 59.803244735933724, 59.73420780117363, 59.50983776320331, 59.250949257852945, 59.04383845357266, 58.73317224715223, 58.526061442871935, 58.370728339661724, 58.301691404901625, 58.06006213324129, 57.92198826372109, 57.801173627890925, 57.57680358992061, 57.26613738350017, 57.04176734552986, 56.80013807386952, 56.523990334829136, 56.265101829478766, 56.092509492578536, 55.91991715567829, 55.8508802209182, 55.712806351397994, 55.64376941663791, 55.367621677597526, 55.12599240593718, 54.95340006903693, 54.81532619951675], "calls": [0.0, 3.003106662064204, 5.091473938557129, 11.11494649637556, 13.09975837072834, 16.931308249913705, 22.31618916120124, 27.459440800828443, 33.12046945115637, 37.141870900932, 38.229202623403516, 40.24853296513635, 42.99275112185019, 44.373489817052125, 46.03037625129444, 47.80807732136693, 49.913703831549874, 51.77770107007249, 53.58992060752502, 54.36658612357611, 55.78184328615809, 56.28236106316879, 57.26613738350017, 58.215395236451506, 58.56057991025199, 59.69968933379358, 60.71798412150501, 61.2875388332758, 61.977908180876774, 62.18501898515706, 62.3576113220573, 63.03072143596824, 63.72109078356921, 64.51501553331032, 65.37797721781153, 65.77493959268207, 66.2064204349327, 66.7069382119434, 67.138419054194, 67.82878840179495, 68.12219537452538, 68.46738004832585, 69.0024162927166, 69.34760096651709, 69.62374870555747, 70.33137728684846, 70.95270969968934, 71.62581981360027, 72.07455988954091, 72.81670693821194, 73.40352088367277, 73.73144632378322, 74.38729720400414, 75.47462892647566, 75.75077666551606, 76.35484984466689, 76.95892302381773, 77.52847773558854, 78.20158784949949, 78.90921643079048, 79.44425267518123, 79.996548153262, 80.61788056610285, 81.15291681049361, 81.46358301691406, 82.03313772868485, 82.53365550569555, 82.9306178805661, 83.96617190196756, 84.31135657576803, 84.86365205384881, 85.15705902657922, 85.69209527096997, 86.14083534691059, 87.10735243355195, 87.29720400414222, 87.86675871591302, 88.53986882982396, 88.91957197100449, 89.73075595443562, 90.38660683465655, 90.86986537797722, 91.30134622022781, 92.07801173627891, 92.42319641007938, 92.66482568173973, 93.35519502934069, 94.04556437694166, 94.61511908871246, 95.34000690369348, 96.23748705557473, 96.70348636520538, 97.20400414221608, 97.65274421815671, 97.98066965826717, 98.3603727994477, 98.61926130479807, 99.05074214704868, 99.41318605453918, 99.77562996202968, 100.0], "rescued": [0, 43, 67, 114, 140, 175, 217, 258, 292, 314, 316, 322, 331, 335, 342, 345, 346, 352, 355, 355, 355, 356, 356, 356, 356, 357, 358, 358, 358, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 361, 361, 361, 361, 361, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 363, 363, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 365, 365], "harmed": [0, 42, 74, 171, 199, 260, 358, 457, 576, 659, 684, 727, 792, 825, 869, 909, 951, 998, 1035, 1045, 1081, 1089, 1107, 1122, 1128, 1154, 1177, 1185, 1198, 1201, 1203, 1214, 1230, 1241, 1257, 1265, 1275, 1279, 1284, 1295, 1297, 1305, 1309, 1313, 1316, 1330, 1342, 1352, 1358, 1368, 1375, 1378, 1387, 1406, 1412, 1419, 1428, 1439, 1452, 1462, 1473, 1481, 1498, 1507, 1512, 1524, 1528, 1537, 1551, 1561, 1571, 1574, 1587, 1596, 1615, 1619, 1632, 1647, 1659, 1677, 1689, 1698, 1702, 1716, 1724, 1731, 1745, 1763, 1777, 1791, 1807, 1822, 1832, 1842, 1846, 1854, 1858, 1874, 1888, 1899, 1907], "exact_score": [81.429064549534, 81.44632378322403, 81.51536071798412, 81.44632378322403, 81.48084225060408, 81.36002761477391, 81.15291681049361, 81.17017604418363, 80.9975837072834, 80.75595443562305, 80.63513979979288, 80.47980669658268, 80.47980669658268, 80.46254746289266, 80.34173282706249, 80.30721435968243, 80.10010355540214, 79.97928891957199, 79.72040041422161, 79.70314118053159, 79.39247497411115, 79.21988263721092, 78.9955125992406, 78.8574387297204, 78.71936486020022, 78.42595788746979, 78.23610631687953, 78.18432861580946, 77.77010700724888, 77.40766309975837, 77.26958923023818, 77.01070072488781, 76.76907145322748, 76.51018294787711, 76.33759061097687, 76.11322057300656, 75.88885053503624, 75.54366586123577, 75.19848118743528, 74.93959268208492, 74.54263030721437, 74.18018639972385, 73.81774249223335, 73.47255781843286, 73.16189161201243, 72.7821884708319, 72.33344839489126, 72.04004142216085, 71.59130134622022, 71.29789437348983, 70.98722816706939, 70.62478425957887, 70.26234035208837, 69.91715567828788, 69.55471177079737, 69.26130479806697, 68.84708318950639, 68.53641698308596, 68.24301001035555, 67.94960303762512, 67.5698998964446, 67.25923369002416, 66.93130824991371, 66.67241974456334, 66.30997583707283, 66.03382809803244, 65.68864342423197, 65.48153261995168, 65.27442181567137, 65.05005177770107, 64.77390403866067, 64.48049706593028, 64.23886779426994, 63.97997928891957, 63.7901277183293, 63.61753538142907, 63.23783224024854, 62.9962029685882, 62.66827752847773, 62.40938902312738, 62.09872281670694, 61.701760441836385, 61.391094235415956, 61.13220573006559, 60.890576458405256, 60.54539178460476, 60.18294787711426, 59.87228167069383, 59.47531929582327, 59.00931998619261, 58.68139454608215, 58.33620987228167, 58.077321366931315, 57.69761822575078, 57.2833966171902, 56.92095270969969, 56.62754573696929, 56.14428719364861, 55.83362098722817, 55.402140144977565, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 70.47403405782397, "threshold_area100": 64.66102865032792, "threshold_area25": 74.03693476009666}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.9819905757904053, "operator": "ge"}, {"budget": 0.02, "threshold": -1.2407728433609009, "operator": "ge"}, {"budget": 0.03, "threshold": -1.379433274269104, "operator": "ge"}, {"budget": 0.04, "threshold": -1.5961730480194092, "operator": "ge"}, {"budget": 0.05, "threshold": -1.7973628044128418, "operator": "ge"}, {"budget": 0.06, "threshold": -1.941401720046997, "operator": "ge"}, {"budget": 0.07, "threshold": -2.131197929382324, "operator": "ge"}, {"budget": 0.08, "threshold": -2.2584264278411865, "operator": "ge"}, {"budget": 0.09, "threshold": -2.3656258583068848, "operator": "ge"}, {"budget": 0.1, "threshold": -2.4755067825317383, "operator": "ge"}, {"budget": 0.11, "threshold": -2.589787006378174, "operator": "ge"}, {"budget": 0.12, "threshold": -2.8566203117370605, "operator": "ge"}, {"budget": 0.13, "threshold": -2.987250328063965, "operator": "ge"}, {"budget": 0.14, "threshold": -3.2260355949401855, "operator": "ge"}, {"budget": 0.15, "threshold": -3.3902194499969482, "operator": "ge"}, {"budget": 0.16, "threshold": -3.5900468826293945, "operator": "ge"}, {"budget": 0.17, "threshold": -3.7451701164245605, "operator": "ge"}, {"budget": 0.18, "threshold": -4.069467544555664, "operator": "ge"}, {"budget": 0.19, "threshold": -4.2650017738342285, "operator": "ge"}, {"budget": 0.2, "threshold": -4.465061664581299, "operator": "ge"}, {"budget": 0.21, "threshold": -4.644355297088623, "operator": "ge"}, {"budget": 0.22, "threshold": -4.891790866851807, "operator": "ge"}, {"budget": 0.23, "threshold": -5.0395097732543945, "operator": "ge"}, {"budget": 0.24, "threshold": -5.3136444091796875, "operator": "ge"}, {"budget": 0.25, "threshold": -5.502932548522949, "operator": "ge"}, {"budget": 0.26, "threshold": -5.772789478302002, "operator": "ge"}, {"budget": 0.27, "threshold": -5.889659881591797, "operator": "ge"}, {"budget": 0.28, "threshold": -6.030814170837402, "operator": "ge"}, {"budget": 0.29, "threshold": -6.116622447967529, "operator": "ge"}, {"budget": 0.3, "threshold": -6.230241298675537, "operator": "ge"}, {"budget": 0.31, "threshold": -6.339517593383789, "operator": "ge"}, {"budget": 0.32, "threshold": -6.446399688720703, "operator": "ge"}, {"budget": 0.33, "threshold": -6.52310848236084, "operator": "ge"}, {"budget": 0.34, "threshold": -6.591789722442627, "operator": "ge"}, {"budget": 0.35, "threshold": -6.659395217895508, "operator": "ge"}, {"budget": 0.36, "threshold": -6.717305660247803, "operator": "ge"}, {"budget": 0.37, "threshold": -6.7548112869262695, "operator": "ge"}, {"budget": 0.38, "threshold": -6.8186798095703125, "operator": "ge"}, {"budget": 0.39, "threshold": -6.894723415374756, "operator": "ge"}, {"budget": 0.4, "threshold": -6.937100410461426, "operator": "ge"}, {"budget": 0.41, "threshold": -6.970241546630859, "operator": "ge"}, {"budget": 0.42, "threshold": -7.008471488952637, "operator": "ge"}, {"budget": 0.43, "threshold": -7.062079429626465, "operator": "ge"}, {"budget": 0.44, "threshold": -7.100081443786621, "operator": "ge"}, {"budget": 0.45, "threshold": -7.138404846191406, "operator": "ge"}, {"budget": 0.46, "threshold": -7.183412551879883, "operator": "ge"}, {"budget": 0.47, "threshold": -7.2143940925598145, "operator": "ge"}, {"budget": 0.48, "threshold": -7.255462169647217, "operator": "ge"}, {"budget": 0.49, "threshold": -7.283073425292969, "operator": "ge"}, {"budget": 0.5, "threshold": -7.301853179931641, "operator": "ge"}, {"budget": 0.51, "threshold": -7.33645486831665, "operator": "ge"}, {"budget": 0.52, "threshold": -7.36798095703125, "operator": "ge"}, {"budget": 0.53, "threshold": -7.398924350738525, "operator": "ge"}, {"budget": 0.54, "threshold": -7.441039085388184, "operator": "ge"}, {"budget": 0.55, "threshold": -7.474511623382568, "operator": "ge"}, {"budget": 0.56, "threshold": -7.522521495819092, "operator": "ge"}, {"budget": 0.57, "threshold": -7.552011489868164, "operator": "ge"}, {"budget": 0.58, "threshold": -7.5773491859436035, "operator": "ge"}, {"budget": 0.59, "threshold": -7.607394218444824, "operator": "ge"}, {"budget": 0.6, "threshold": -7.640082359313965, "operator": "ge"}, {"budget": 0.61, "threshold": -7.6635942459106445, "operator": "ge"}, {"budget": 0.62, "threshold": -7.681548595428467, "operator": "ge"}, {"budget": 0.63, "threshold": -7.70057487487793, "operator": "ge"}, {"budget": 0.64, "threshold": -7.718975067138672, "operator": "ge"}, {"budget": 0.65, "threshold": -7.745815277099609, "operator": "ge"}, {"budget": 0.66, "threshold": -7.765655517578125, "operator": "ge"}, {"budget": 0.67, "threshold": -7.785719394683838, "operator": "ge"}, {"budget": 0.68, "threshold": -7.819531440734863, "operator": "ge"}, {"budget": 0.69, "threshold": -7.869359016418457, "operator": "ge"}, {"budget": 0.7, "threshold": -7.89585018157959, "operator": "ge"}, {"budget": 0.71, "threshold": -7.926639080047607, "operator": "ge"}, {"budget": 0.72, "threshold": -7.962143898010254, "operator": "ge"}, {"budget": 0.73, "threshold": -7.981078147888184, "operator": "ge"}, {"budget": 0.74, "threshold": -8.023921966552734, "operator": "ge"}, {"budget": 0.75, "threshold": -8.055846214294434, "operator": "ge"}, {"budget": 0.76, "threshold": -8.09382438659668, "operator": "ge"}, {"budget": 0.77, "threshold": -8.115316390991211, "operator": "ge"}, {"budget": 0.78, "threshold": -8.159411430358887, "operator": "ge"}, {"budget": 0.79, "threshold": -8.201526641845703, "operator": "ge"}, {"budget": 0.8, "threshold": -8.244976043701172, "operator": "ge"}, {"budget": 0.81, "threshold": -8.296379089355469, "operator": "ge"}, {"budget": 0.82, "threshold": -8.324970245361328, "operator": "ge"}, {"budget": 0.83, "threshold": -8.36547565460205, "operator": "ge"}, {"budget": 0.84, "threshold": -8.414788246154785, "operator": "ge"}, {"budget": 0.85, "threshold": -8.457171440124512, "operator": "ge"}, {"budget": 0.86, "threshold": -8.4912748336792, "operator": "ge"}, {"budget": 0.87, "threshold": -8.532470703125, "operator": "ge"}, {"budget": 0.88, "threshold": -8.56576156616211, "operator": "ge"}, {"budget": 0.89, "threshold": -8.611520767211914, "operator": "ge"}, {"budget": 0.9, "threshold": -8.662858009338379, "operator": "ge"}, {"budget": 0.91, "threshold": -8.72958755493164, "operator": "ge"}, {"budget": 0.92, "threshold": -8.774177551269531, "operator": "ge"}, {"budget": 0.93, "threshold": -8.800392150878906, "operator": "ge"}, {"budget": 0.94, "threshold": -8.845844268798828, "operator": "ge"}, {"budget": 0.95, "threshold": -8.926572799682617, "operator": "ge"}, {"budget": 0.96, "threshold": -9.057340621948242, "operator": "ge"}, {"budget": 0.97, "threshold": -9.14522933959961, "operator": "ge"}, {"budget": 0.98, "threshold": -9.260102272033691, "operator": "ge"}, {"budget": 0.99, "threshold": -9.396123886108398, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.0, 97.66666666666667, 97.16666666666667, 96.75, 96.33333333333334, 95.91666666666667, 95.5, 94.66666666666667, 94.41666666666667, 93.66666666666667, 93.0, 92.25, 91.75, 91.33333333333333, 90.66666666666667, 90.0, 89.33333333333333, 88.75, 88.16666666666667, 87.83333333333333, 87.33333333333333, 86.91666666666666, 86.33333333333333, 85.91666666666666, 85.41666666666667, 84.91666666666666, 84.58333333333333, 84.16666666666667, 83.83333333333334, 83.41666666666667, 83.25, 83.0, 82.75, 82.66666666666667, 82.5, 82.25, 82.16666666666667, 82.0, 81.66666666666667, 81.25, 80.91666666666667, 80.33333333333333, 80.33333333333333, 80.08333333333333, 79.91666666666667, 79.75, 79.5, 79.08333333333333, 78.58333333333334, 78.33333333333333, 78.16666666666667, 78.08333333333334, 77.66666666666666, 77.5, 77.08333333333334, 76.83333333333333, 76.58333333333334, 76.16666666666667, 75.83333333333333, 75.66666666666667, 75.5, 75.25, 75.0, 74.58333333333333, 74.08333333333333, 73.75, 73.08333333333333, 72.50000000000001, 72.16666666666667, 71.41666666666666, 71.25, 70.66666666666667, 70.5, 70.25, 69.66666666666667, 69.25, 68.91666666666667, 68.25, 67.58333333333333, 67.08333333333334, 66.75, 66.16666666666666, 65.75, 65.33333333333333, 64.91666666666667, 64.33333333333333, 63.583333333333336, 63.083333333333336, 62.5, 61.583333333333336, 61.16666666666667, 60.41666666666667, 59.66666666666667, 59.25, 58.41666666666667, 58.166666666666664, 57.41666666666667, 56.666666666666664, 55.833333333333336, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 7, 11, 17, 22, 27, 33, 38, 48, 52, 61, 69, 78, 84, 89, 97, 105, 113, 120, 127, 131, 137, 142, 149, 154, 160, 166, 170, 175, 179, 184, 186, 189, 192, 193, 195, 198, 199, 201, 205, 210, 214, 221, 221, 224, 226, 228, 231, 236, 242, 245, 247, 248, 253, 255, 260, 263, 266, 271, 275, 277, 279, 282, 285, 290, 296, 300, 308, 315, 319, 328, 330, 337, 339, 342, 349, 354, 358, 366, 374, 380, 384, 391, 396, 401, 406, 413, 422, 428, 435, 446, 451, 460, 469, 474, 484, 487, 496, 505, 515, 524], "exact_score": [98.58333333333333, 98.0, 97.66666666666667, 97.16666666666667, 96.75, 96.33333333333334, 95.91666666666667, 95.5, 94.66666666666667, 94.41666666666667, 93.66666666666667, 93.0, 92.25, 91.75, 91.33333333333333, 90.66666666666667, 90.0, 89.33333333333333, 88.75, 88.16666666666667, 87.83333333333333, 87.33333333333333, 86.91666666666666, 86.33333333333333, 85.91666666666666, 85.41666666666667, 84.91666666666666, 84.58333333333333, 84.16666666666667, 83.83333333333334, 83.41666666666667, 83.25, 83.0, 82.75, 82.66666666666667, 82.5, 82.25, 82.16666666666667, 82.0, 81.66666666666667, 81.25, 80.83333333333333, 80.33333333333333, 80.33333333333333, 80.08333333333333, 79.91666666666667, 79.75, 79.5, 79.08333333333333, 78.58333333333334, 78.33333333333333, 78.16666666666667, 78.08333333333334, 77.66666666666666, 77.5, 77.08333333333334, 76.83333333333333, 76.5, 76.16666666666667, 75.83333333333333, 75.66666666666667, 75.5, 75.25, 75.0, 74.58333333333333, 74.08333333333333, 73.75, 73.08333333333333, 72.50000000000001, 72.08333333333333, 71.41666666666666, 71.25, 70.66666666666667, 70.5, 70.25, 69.66666666666667, 69.25, 68.91666666666667, 68.25, 67.58333333333333, 67.08333333333334, 66.75, 66.16666666666666, 65.75, 65.33333333333333, 64.91666666666667, 64.33333333333333, 63.583333333333336, 63.083333333333336, 62.5, 61.583333333333336, 61.16666666666667, 60.41666666666667, 59.66666666666667, 59.25, 58.41666666666667, 58.166666666666664, 57.41666666666667, 56.666666666666664, 55.833333333333336, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 77.84249999999999, "threshold_area100": 77.845, "threshold_area25": 92.10000000000001}, "test": {"score": [81.429064549534, 81.429064549534, 81.1356575768036, 80.72143596824301, 80.04832585433206, 79.20262340352089, 78.20158784949949, 77.09699689333793, 76.47566448049706, 75.7852951328961, 75.04314808422507, 74.19744563341388, 73.0583362098723, 72.73041076976182, 71.90196755264067, 71.52226441146013, 70.9181912323093, 70.52122885743873, 69.76182257507767, 69.53745253710736, 69.22678633068692, 68.98515705902658, 68.570935450466, 68.3638246461857, 67.94960303762512, 67.65619606489473, 67.32827062478425, 67.138419054194, 66.93130824991371, 66.6379012771833, 66.51708664135313, 66.39627200552296, 66.17190196755264, 66.08560579910252, 65.91301346220229, 65.82671729375215, 65.72316189161201, 65.67138419054194, 65.55056955471177, 65.41249568519159, 65.32619951674145, 65.22264411460131, 65.13634794615119, 64.998274076631, 64.89471867449086, 64.77390403866067, 64.63583016914049, 64.53227476700035, 64.41146013117019, 64.34242319641008, 64.30790472903003, 64.15257162581982, 64.04901622367967, 63.91094235415948, 63.7901277183293, 63.548498446668965, 63.375906109768735, 63.25509147393856, 63.13427683810839, 62.96168450120815, 62.80635139799793, 62.56472212633759, 62.44390749050742, 62.30583362098723, 62.16775975146703, 62.06420434932689, 61.96064894718675, 61.7880566102865, 61.563686572316186, 61.21850189851571, 61.09768726268554, 60.95961339316535, 60.61442871936487, 60.493614083534695, 60.200207110804286, 59.95857783914394, 59.76872626855367, 59.66517086641353, 59.302726958923024, 59.078356920952714, 58.802209181912325, 58.59509837763204, 58.50880220918191, 58.33620987228167, 58.094580600621335, 57.83569209527097, 57.680358992060754, 57.49050742147048, 57.3524335519503, 57.179841215050054, 57.04176734552986, 56.817397307559546, 56.6448049706593, 56.593027269589236, 56.3513979979289, 56.14428719364861, 55.79910251984812, 55.54021401449776, 55.17777010700725, 55.02243700379703, 54.81532619951675], "calls": [0.0, 3.9868829823955814, 8.698653779772178, 12.167759751467035, 17.897825336555055, 23.748705557473247, 27.683810838798756, 32.34380393510528, 34.84639282015878, 36.98653779772178, 39.12668277528478, 41.042457714877465, 44.26993441491198, 45.59889540904384, 47.911632723507076, 49.05074214704867, 50.742147048671036, 51.91577493959269, 53.84880911287539, 55.005177770107004, 55.79910251984812, 56.57576803589921, 57.85295132896099, 58.33620987228167, 59.423541594753196, 60.303762512944424, 61.30479806696583, 61.839834311356576, 62.4956851915775, 63.099758370728345, 63.61753538142907, 64.37694166379013, 65.34345875043148, 65.96479116327235, 66.68967897825337, 67.39730755954436, 67.82878840179495, 68.17397307559544, 68.7780462547463, 69.58923023817742, 70.19330341732827, 70.69382119433897, 71.29789437348981, 72.02278218847084, 72.62685536762167, 73.36900241629272, 74.19744563341388, 74.54263030721437, 75.2157404211253, 75.68173973075596, 75.97514670348636, 76.56196064894719, 77.13151536071798, 77.52847773558854, 78.28788401794961, 79.08180876769072, 79.9275112185019, 80.44528822920263, 80.89402830514325, 81.48084225060407, 81.96410079392476, 82.44735933724543, 82.7925440110459, 83.27580255436658, 83.62098722816707, 83.9316534345875, 84.29409734207802, 84.79461511908872, 85.31239212978944, 86.12357611322057, 86.45150155333103, 87.03831549879185, 87.86675871591302, 88.22920262340352, 88.91957197100449, 89.42008974801519, 90.00690369347602, 90.3520883672765, 91.18053158439766, 91.85364169830859, 92.31964100793924, 92.92371418709008, 93.21712115982051, 93.57956506731101, 94.13186054539179, 94.58060062133241, 94.96030376251295, 95.28822920262341, 95.59889540904383, 95.9958577839144, 96.25474628926476, 96.68622713151535, 97.01415257162583, 97.15222644114601, 97.53192958232655, 97.9288919571971, 98.567483603728, 99.03348291335865, 99.48222298929927, 99.75837072833966, 100.0], "rescued": [0, 44, 94, 130, 175, 219, 245, 278, 292, 299, 308, 313, 324, 335, 339, 343, 349, 350, 352, 356, 356, 356, 357, 357, 357, 357, 358, 358, 358, 358, 358, 359, 359, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 362, 363, 365, 365, 365, 365], "harmed": [0, 44, 111, 171, 255, 348, 432, 529, 579, 626, 678, 732, 809, 839, 891, 917, 958, 982, 1028, 1045, 1063, 1077, 1102, 1114, 1138, 1155, 1175, 1186, 1198, 1215, 1222, 1230, 1243, 1248, 1259, 1264, 1270, 1273, 1280, 1288, 1293, 1299, 1304, 1312, 1318, 1325, 1333, 1339, 1346, 1350, 1352, 1361, 1367, 1375, 1382, 1396, 1406, 1413, 1420, 1430, 1439, 1453, 1460, 1468, 1476, 1482, 1488, 1499, 1512, 1532, 1539, 1547, 1567, 1574, 1591, 1605, 1616, 1622, 1643, 1657, 1673, 1685, 1690, 1700, 1714, 1729, 1738, 1749, 1757, 1767, 1775, 1788, 1798, 1801, 1815, 1827, 1848, 1865, 1886, 1895, 1907], "exact_score": [81.429064549534, 81.429064549534, 81.51536071798412, 81.41180531584398, 81.429064549534, 81.44632378322403, 81.30824991370383, 81.27373144632378, 81.22195374525371, 81.04936140835348, 80.9975837072834, 80.75595443562305, 80.68691750086296, 80.70417673455299, 80.60062133241284, 80.47980669658268, 80.25543665861235, 80.03106662064205, 80.03106662064205, 80.01380738695202, 79.8584742837418, 79.56506731101139, 79.40973420780118, 79.32343803935106, 79.09906800138074, 78.71936486020022, 78.59855022437004, 78.40869865377978, 78.14981014842941, 77.90818087676908, 77.75284777355886, 77.59751467034863, 77.20055229547809, 76.97618225750776, 76.75181221953746, 76.423886779427, 76.07870210562652, 75.7852951328961, 75.4228512254056, 75.12944425267519, 74.73248187780463, 74.23196410079392, 73.85226095961339, 73.47255781843286, 73.09285467725233, 72.86848463928202, 72.50604073179151, 72.22989299275112, 71.90196755264067, 71.55678287884018, 71.0735243355195, 70.83189506385916, 70.5039696237487, 70.17604418363825, 69.7790818087677, 69.52019330341733, 69.15774939592683, 68.89886089057646, 68.51915774939593, 68.12219537452538, 67.7079737659648, 67.43182602692441, 67.08664135312392, 66.7069382119434, 66.46530894028305, 66.24093890231273, 66.08560579910252, 65.8612357611322, 65.70590265792198, 65.53331032102174, 65.36071798412151, 65.17086641353124, 64.998274076631, 64.84294097342078, 64.65308940283052, 64.44597859855023, 64.29064549534, 64.0662754573697, 63.824646185709355, 63.56575768035899, 63.375906109768735, 63.117017604418365, 62.7890921643079, 62.40938902312738, 62.02968588194684, 61.73627890921644, 61.25302036589575, 60.95961339316535, 60.57991025198481, 60.18294787711426, 59.7859855022437, 59.38902312737315, 59.00931998619261, 58.577839143942015, 58.12909906800139, 57.6458405246807, 57.179841215050054, 56.66206420434933, 56.092509492578536, 55.52295478080773, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 70.92215454097767, "threshold_area100": 64.83258543320677, "threshold_area25": 73.90749050742149}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.41180531584398, "calls": 0.6731101139109423}}, "old_normalized_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.41180531584398, "calls": 0.6731101139109423}}, "old_normalized_endpoints": {"val": {"score": 98.58333333333333, "calls": 0.0}, "test": {"score": 81.429064549534, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.5, 98.5, 98.41666666666666, 98.33333333333334, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333334, 97.83333333333334, 97.83333333333334, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.5, 97.16666666666667, 97.0, 96.58333333333333, 96.33333333333334, 96.0, 95.58333333333333, 95.33333333333334, 95.0, 94.75, 94.66666666666667, 94.5, 94.41666666666667, 94.0, 93.91666666666667, 93.33333333333333, 92.91666666666667, 92.33333333333333, 91.91666666666667, 91.75, 91.25, 90.91666666666667, 90.66666666666667, 89.91666666666667, 89.5, 89.0, 88.5, 87.91666666666667, 87.5, 87.08333333333333, 86.5, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.75, 83.25, 82.66666666666667, 82.0, 81.33333333333333, 80.5, 80.0, 79.41666666666667, 78.66666666666666, 78.08333333333334, 77.33333333333333, 76.5, 75.66666666666667, 74.75, 74.0, 73.16666666666667, 72.25, 71.50000000000001, 70.5, 70.0, 69.16666666666667, 68.16666666666666, 67.33333333333333, 66.33333333333333, 65.5, 64.75, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 57.00000000000001, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.5, 98.5, 98.41666666666666, 98.33333333333334, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333334, 97.83333333333334, 97.83333333333334, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.5, 97.16666666666667, 97.0, 96.58333333333333, 96.33333333333334, 96.0, 95.58333333333333, 95.33333333333334, 95.0, 94.75, 94.66666666666667, 94.5, 94.41666666666667, 94.0, 93.91666666666667, 93.33333333333333, 92.91666666666667, 92.33333333333333, 91.91666666666667, 91.75, 91.25, 90.91666666666667, 90.66666666666667, 89.91666666666667, 89.5, 89.0, 88.41666666666667, 87.91666666666667, 87.5, 87.08333333333333, 86.5, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.75, 83.25, 82.66666666666667, 82.0, 81.33333333333333, 80.5, 80.0, 79.41666666666667, 78.66666666666666, 78.08333333333334, 77.33333333333333, 76.5, 75.66666666666667, 74.75, 74.0, 73.16666666666667, 72.25, 71.41666666666666, 70.5, 70.0, 69.16666666666667, 68.16666666666666, 67.33333333333333, 66.33333333333333, 65.5, 64.75, 63.74999999999999, 62.83333333333334, 61.83333333333334, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 57.00000000000001, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.31666666666666, "exact_area100": 86.37583333333335}, "test": {"score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.34276838108389, 81.32550914739386, 81.30824991370383, 81.27373144632378, 81.27373144632378, 81.23921297894374, 81.20469451156369, 81.17017604418363, 81.17017604418363, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.04936140835348, 81.01484294097342, 80.98032447359338, 80.91128753883328, 80.84225060407319, 80.75595443562305, 80.73869520193303, 80.68691750086296, 80.70417673455299, 80.68691750086296, 80.65239903348291, 80.39351052813255, 80.2899551259924, 80.13462202278218, 80.06558508802209, 80.06558508802209, 80.03106662064205, 80.01380738695202, 79.87573351743183, 79.87573351743183, 79.8584742837418, 79.73765964791163, 79.65136347946151, 79.37521574042113, 79.16810493614084, 79.08180876769072, 78.77114256127028, 78.68484639282016, 78.47773558853987, 78.40869865377978, 78.13255091473938, 77.97721781152917, 77.37314463237833, 77.04521919226787, 76.80358992060754, 76.49292371418709, 76.40662754573697, 76.06144287193649, 75.94062823610632, 75.76803589920608, 75.47462892647566, 74.93959268208492, 74.31826026924405, 73.97307559544356, 73.31722471522265, 72.7821884708319, 71.98826372109079, 71.1425612702796, 70.26234035208837, 69.6410079392475, 68.98515705902658, 68.570935450466, 68.26026924404557, 67.65619606489473, 67.25923369002416, 66.39627200552296, 65.25716258198136, 64.51501553331033, 64.25612702795996, 63.962720055229546, 63.53123921297894, 62.823610631687956, 62.18501898515706, 61.615464273386266, 61.09768726268554, 60.71798412150501, 60.33828098032448, 59.302726958923024, 58.802209181912325, 58.54332067656196, 57.99102519848118, 57.54228512254056, 57.12806351397999, 56.73110113910942, 55.988954090438384, 55.5056955471177, 55.16051087331723, 54.81532619951675], "calls": [0.0, 0.6731101139109423, 1.4325163962720056, 2.4853296513634793, 3.296513634794615, 4.124956851915775, 4.470141525716258, 5.26406627545737, 5.937176389368312, 6.627545736969279, 7.145322747670003, 7.611322057300655, 8.318950638591646, 9.009319986192613, 10.027614773904038, 10.476354849844666, 11.253020365895754, 12.51294442526752, 13.237832240248535, 14.359682430100104, 14.946496375560924, 15.982050396962375, 16.724197445633415, 17.53538142906455, 18.20849154297549, 19.27856403175699, 19.65826717293752, 20.590265792198824, 21.625819813600273, 22.59233690024163, 23.110113910942353, 23.852260959613396, 25.07766655160511, 25.62996202968588, 27.114256127027957, 28.684846392820155, 30.203658957542284, 31.4981014842941, 32.9306178805661, 34.03520883672765, 34.69105971694857, 36.41698308595098, 37.573351743182606, 39.74801518812565, 41.24956851915775, 42.7856403175699, 44.011045909561616, 45.564376941663795, 46.996893337935795, 48.394891266827756, 49.413186054539175, 50.94925785295133, 51.43251639627201, 52.81325509147394, 54.36658612357611, 56.55850880220918, 57.594062823610635, 58.68139454608215, 59.924059371763896, 60.71798412150501, 62.39212978943735, 63.80738695201933, 64.58405246807042, 65.80945806006213, 67.32827062478425, 68.72626855367622, 69.96893337935795, 71.33241284086986, 72.85122540559199, 74.64618570935451, 76.38936831204694, 78.02899551259924, 79.27166033828098, 80.41076976182258, 81.27373144632378, 81.75698998964445, 82.68898860890576, 83.39661719019675, 84.51846738004832, 86.41698308595099, 87.5733517431826, 87.98757335174318, 88.36727649292372, 88.9886089057646, 90.09319986192614, 91.18053158439766, 91.87090093199862, 92.68208491542975, 93.19986192613048, 93.66586123576113, 95.01208146358302, 95.77148774594409, 96.04763548498447, 96.68622713151535, 97.13496720745599, 97.5664480497066, 97.9979288919572, 98.75733517431826, 99.25785295132896, 99.6375560925095, 100.0], "exact_score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.18743527787366, 81.17017604418363, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.01484294097342, 80.9975837072834, 80.87676907145323, 80.82499137038316, 80.77321366931308, 80.70417673455299, 80.68691750086296, 80.68691750086296, 80.56610286503279, 80.41076976182258, 80.30721435968243, 80.23817742492233, 80.11736278909216, 80.10010355540214, 80.03106662064205, 80.08284432171212, 80.01380738695202, 79.996548153262, 79.996548153262, 79.89299275112185, 79.91025198481188, 79.8584742837418, 79.84121505005179, 79.77217811529168, 79.70314118053159, 79.58232654470142, 79.39247497411115, 79.27166033828098, 79.18536416983086, 79.08180876769072, 78.80566102865033, 78.77114256127028, 78.65032792544011, 78.47773558853987, 78.30514325163963, 78.11529168104936, 78.06351397997929, 77.89092164307905, 77.58025543665862, 77.25232999654816, 76.95892302381775, 76.71729375215742, 76.51018294787711, 76.33759061097687, 76.13047980669658, 76.04418363824647, 75.88885053503624, 75.71625819813602, 75.35381429064549, 75.0604073179151, 74.5771487745944, 74.18018639972385, 73.97307559544356, 73.43803935105282, 73.12737314463237, 72.71315153607179, 72.21263375906109, 71.7811529168105, 71.2806351397998, 70.84915429754919, 70.2795995857784, 69.74456334138765, 69.26130479806697, 68.65723161891613, 68.1394546082154, 67.53538142906454, 66.8622713151536, 66.10286503279255, 65.5678287884018, 64.94649637556093, 64.25612702795996, 63.53123921297894, 62.90990680013808, 62.2885743872972, 61.52916810493614, 60.87331722471523, 60.09665170866414, 59.31998619261305, 58.59509837763204, 57.680358992060754, 56.73110113910942, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 81.28063513979981, "exact_area100": 75.25180638106697}}, "old_normalized_endpoint_budget": 0}, {"seed": 2028, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.0, "validation_area100": 0.8633916666666666, "training_seconds": 3.3468945929780602, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0035147510934621096, "operator": "ge"}, {"budget": 0.02, "threshold": -0.004862976260483265, "operator": "ge"}, {"budget": 0.03, "threshold": -0.005958778318017721, "operator": "ge"}, {"budget": 0.04, "threshold": -0.006726748775690794, "operator": "ge"}, {"budget": 0.05, "threshold": -0.00854812003672123, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01104010734707117, "operator": "ge"}, {"budget": 0.07, "threshold": -0.012473727576434612, "operator": "ge"}, {"budget": 0.08, "threshold": -0.014348694123327732, "operator": "ge"}, {"budget": 0.09, "threshold": -0.017029881477355957, "operator": "ge"}, {"budget": 0.1, "threshold": -0.01918027549982071, "operator": "ge"}, {"budget": 0.11, "threshold": -0.022077379748225212, "operator": "ge"}, {"budget": 0.12, "threshold": -0.025938954204320908, "operator": "ge"}, {"budget": 0.13, "threshold": -0.029204875230789185, "operator": "ge"}, {"budget": 0.14, "threshold": -0.03296282887458801, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03573034331202507, "operator": "ge"}, {"budget": 0.16, "threshold": -0.03975854441523552, "operator": "ge"}, {"budget": 0.17, "threshold": -0.042940158396959305, "operator": "ge"}, {"budget": 0.18, "threshold": -0.04439644515514374, "operator": "ge"}, {"budget": 0.19, "threshold": -0.048601601272821426, "operator": "ge"}, {"budget": 0.2, "threshold": -0.0533132404088974, "operator": "ge"}, {"budget": 0.21, "threshold": -0.058916036039590836, "operator": "ge"}, {"budget": 0.22, "threshold": -0.06130586192011833, "operator": "ge"}, {"budget": 0.23, "threshold": -0.06567063182592392, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0717887207865715, "operator": "ge"}, {"budget": 0.25, "threshold": -0.07682093232870102, "operator": "ge"}, {"budget": 0.26, "threshold": -0.08091072738170624, "operator": "ge"}, {"budget": 0.27, "threshold": -0.08559807389974594, "operator": "ge"}, {"budget": 0.28, "threshold": -0.09159402549266815, "operator": "ge"}, {"budget": 0.29, "threshold": -0.0986008271574974, "operator": "ge"}, {"budget": 0.3, "threshold": -0.10600607097148895, "operator": "ge"}, {"budget": 0.31, "threshold": -0.11637964844703674, "operator": "ge"}, {"budget": 0.32, "threshold": -0.12771214544773102, "operator": "ge"}, {"budget": 0.33, "threshold": -0.1343788355588913, "operator": "ge"}, {"budget": 0.34, "threshold": -0.14219801127910614, "operator": "ge"}, {"budget": 0.35, "threshold": -0.14962050318717957, "operator": "ge"}, {"budget": 0.36, "threshold": -0.15891912579536438, "operator": "ge"}, {"budget": 0.37, "threshold": -0.17304299771785736, "operator": "ge"}, {"budget": 0.38, "threshold": -0.18773770332336426, "operator": "ge"}, {"budget": 0.39, "threshold": -0.2011920064687729, "operator": "ge"}, {"budget": 0.4, "threshold": -0.21209877729415894, "operator": "ge"}, {"budget": 0.41, "threshold": -0.22225184738636017, "operator": "ge"}, {"budget": 0.42, "threshold": -0.23625671863555908, "operator": "ge"}, {"budget": 0.43, "threshold": -0.25916579365730286, "operator": "ge"}, {"budget": 0.44, "threshold": -0.2721913754940033, "operator": "ge"}, {"budget": 0.45, "threshold": -0.2826441526412964, "operator": "ge"}, {"budget": 0.46, "threshold": -0.29839950799942017, "operator": "ge"}, {"budget": 0.47, "threshold": -0.3106931149959564, "operator": "ge"}, {"budget": 0.48, "threshold": -0.321388840675354, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3407030701637268, "operator": "ge"}, {"budget": 0.5, "threshold": -0.35410991311073303, "operator": "ge"}, {"budget": 0.51, "threshold": -0.36935514211654663, "operator": "ge"}, {"budget": 0.52, "threshold": -0.38521191477775574, "operator": "ge"}, {"budget": 0.53, "threshold": -0.39922407269477844, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4229128658771515, "operator": "ge"}, {"budget": 0.55, "threshold": -0.4380481243133545, "operator": "ge"}, {"budget": 0.56, "threshold": -0.45623597502708435, "operator": "ge"}, {"budget": 0.57, "threshold": -0.467838317155838, "operator": "ge"}, {"budget": 0.58, "threshold": -0.4787340462207794, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5019978880882263, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5222599506378174, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5366401672363281, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5562952756881714, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5803082585334778, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5985275506973267, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6253252029418945, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6418166160583496, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6671537160873413, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6937616467475891, "operator": "ge"}, {"budget": 0.69, "threshold": -0.708234429359436, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7267551422119141, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7472893595695496, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7714237570762634, "operator": "ge"}, {"budget": 0.73, "threshold": -0.781021237373352, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7996335029602051, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8166395425796509, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8280400037765503, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8378686904907227, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8433469533920288, "operator": "ge"}, {"budget": 0.79, "threshold": -0.856132984161377, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8656450510025024, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8807071447372437, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8895121812820435, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8958224058151245, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9083128571510315, "operator": "ge"}, {"budget": 0.85, "threshold": -0.915919840335846, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9220972061157227, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9270078539848328, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9365485906600952, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9463539123535156, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9537533521652222, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9611157178878784, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9703249335289001, "operator": "ge"}, {"budget": 0.93, "threshold": -0.975242555141449, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9808505773544312, "operator": "ge"}, {"budget": 0.95, "threshold": -0.983691394329071, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9874285459518433, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9902699589729309, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9924488067626953, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9942666888237, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.08333333333333, 98.0, 98.0, 97.83333333333334, 97.58333333333333, 97.41666666666666, 97.25, 97.16666666666667, 97.16666666666667, 97.0, 96.91666666666666, 96.75, 96.58333333333333, 96.41666666666666, 96.33333333333334, 95.83333333333334, 95.75, 95.58333333333333, 95.16666666666667, 94.83333333333334, 94.41666666666667, 94.16666666666667, 93.91666666666667, 93.5, 92.91666666666667, 92.66666666666666, 92.25, 92.0, 91.75, 91.16666666666666, 90.58333333333334, 89.91666666666667, 89.33333333333333, 88.91666666666667, 88.58333333333334, 88.16666666666667, 87.75, 87.16666666666667, 86.66666666666667, 86.25, 85.41666666666667, 85.0, 84.58333333333333, 84.25, 83.58333333333333, 82.91666666666666, 82.5, 81.66666666666667, 80.75, 80.08333333333333, 79.5, 78.58333333333334, 77.75, 76.91666666666667, 76.33333333333333, 75.58333333333334, 74.91666666666667, 74.0, 73.25, 72.41666666666666, 71.66666666666667, 70.66666666666667, 69.66666666666667, 68.66666666666667, 67.91666666666667, 67.25, 66.41666666666667, 65.58333333333333, 64.75, 63.74999999999999, 62.74999999999999, 61.83333333333334, 60.916666666666664, 59.916666666666664, 58.916666666666664, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 4, 4, 6, 7, 7, 9, 12, 14, 16, 17, 17, 19, 20, 22, 24, 26, 27, 33, 35, 37, 42, 46, 51, 54, 57, 62, 69, 72, 77, 80, 83, 90, 97, 105, 112, 117, 121, 126, 131, 138, 144, 149, 159, 164, 169, 174, 182, 190, 195, 205, 216, 224, 231, 242, 252, 262, 269, 278, 286, 297, 306, 316, 325, 337, 349, 361, 370, 378, 388, 398, 408, 420, 432, 443, 454, 466, 478, 489, 501, 512, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.08333333333333, 98.0, 98.0, 97.83333333333334, 97.58333333333333, 97.41666666666666, 97.25, 97.16666666666667, 97.16666666666667, 97.0, 96.91666666666666, 96.75, 96.58333333333333, 96.41666666666666, 96.33333333333334, 95.83333333333334, 95.75, 95.58333333333333, 95.16666666666667, 94.83333333333334, 94.41666666666667, 94.16666666666667, 93.91666666666667, 93.5, 92.91666666666667, 92.66666666666666, 92.25, 92.0, 91.75, 91.16666666666666, 90.58333333333334, 89.91666666666667, 89.33333333333333, 88.91666666666667, 88.58333333333334, 88.08333333333334, 87.75, 87.16666666666667, 86.66666666666667, 86.25, 85.41666666666667, 85.0, 84.58333333333333, 84.25, 83.58333333333333, 82.91666666666666, 82.5, 81.58333333333333, 80.75, 80.08333333333333, 79.5, 78.58333333333334, 77.75, 76.91666666666667, 76.33333333333333, 75.58333333333334, 74.91666666666667, 74.0, 73.25, 72.41666666666666, 71.58333333333333, 70.66666666666667, 69.66666666666667, 68.66666666666667, 67.91666666666667, 67.25, 66.41666666666667, 65.58333333333333, 64.75, 63.74999999999999, 62.74999999999999, 61.83333333333334, 60.916666666666664, 59.916666666666664, 58.916666666666664, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.33916666666666, "threshold_area100": 86.34166666666667, "threshold_area25": 98.39166666666667}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.32550914739386, 81.32550914739386, 81.2909906800138, 81.2909906800138, 81.27373144632378, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.20469451156369, 81.11839834311357, 81.10113910942354, 81.0666206420435, 80.9975837072834, 80.91128753883328, 80.91128753883328, 80.84225060407319, 80.84225060407319, 80.73869520193303, 80.70417673455299, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.34173282706249, 80.37625129444253, 80.34173282706249, 80.30721435968243, 80.16914049016223, 80.16914049016223, 80.03106662064205, 79.8584742837418, 79.80669658267173, 79.58232654470142, 79.3579565067311, 79.23714187090093, 79.03003106662064, 78.94373489817053, 78.75388332758025, 78.63306869175008, 78.56403175698999, 78.33966171901969, 78.28788401794961, 78.11529168104936, 77.64929237141871, 77.45944080082845, 77.11425612702796, 76.97618225750776, 76.87262685536763, 76.61373835001726, 76.35484984466689, 76.18225750776666, 75.95788746979635, 75.47462892647566, 75.09492578529513, 74.76700034518467, 74.47359337245426, 74.04211253020367, 73.2827062478426, 72.86848463928202, 72.29892992751122, 71.71211598205039, 70.86641353123922, 70.53848809112876, 69.72730410769762, 68.96789782533655, 68.51915774939593, 67.79426993441491, 67.5008629616845, 66.89678978253366, 66.34449430445288, 65.5678287884018, 64.98101484294098, 64.67034863652054, 63.85916465308941, 63.23783224024854, 62.771832930617876, 62.323092854677256, 61.66724197445633, 61.114946496375566, 60.528132550914734, 59.75146703486365, 58.715913013462206, 58.23265447014153, 57.61132205730066, 57.248878149810146, 56.593027269589236, 56.14428719364861, 55.76458405246807, 55.45391784604764, 54.81532619951675], "calls": [0.0, 0.8111839834311357, 1.7777010700724887, 2.5198481187435275, 2.8822920262340355, 3.624439074905074, 4.780807732136693, 5.315843976527442, 5.8336209872281675, 6.748360372799447, 7.369692785640318, 8.353469105971694, 9.31998619261305, 10.217466344494303, 11.097687262685536, 11.61546427338626, 12.478425957887469, 13.09975837072834, 13.324128408698655, 14.187090093199862, 14.963755609250951, 15.930272695892302, 16.309975837072834, 17.311011391094237, 18.260269244045567, 19.140490162236798, 19.65826717293752, 20.210562651018293, 21.125302036589574, 21.884708318950636, 22.782188470831898, 23.938557128063515, 24.991370383154987, 25.81981360027615, 26.786330686917502, 27.338626164998274, 28.253365550569555, 29.806696582671727, 31.239212978943737, 32.430100103555404, 33.275802554366585, 34.15602347255782, 35.381429064549536, 37.12461166724197, 38.073869520193306, 38.97134967207456, 40.17949603037625, 41.00793924749741, 41.95719710044874, 43.389713496720745, 44.68415602347256, 45.875043148084224, 47.204004142216085, 48.343113565757676, 50.62133241284087, 52.071108042802905, 53.572661373835004, 54.556437694166384, 55.31584397652745, 57.05902657921988, 58.664135312392126, 59.69968933379358, 61.13220573006558, 62.927166033828094, 64.39420089748015, 66.25819813600276, 67.4318260269244, 69.08871246116672, 71.0735243355195, 72.16085605799103, 73.576113220573, 75.02588885053504, 76.75181221953746, 77.28684846392821, 78.63306869175008, 80.04832585433206, 81.15291681049361, 82.15395236451502, 82.68898860890576, 83.82809803244736, 84.89817052122885, 86.12357611322057, 86.95201933034173, 87.5043148084225, 88.83327580255437, 89.59268208491544, 90.30031066620641, 90.86986537797722, 91.88816016568865, 92.71660338280981, 93.51052813255092, 94.44252675181221, 95.65067311011391, 96.28926475664481, 97.04867103900587, 97.48015188125648, 98.17052122885744, 98.6365205384881, 99.03348291335865, 99.34414911977908, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 4, 6, 6, 10, 10, 13, 13, 14, 15, 17, 21, 23, 26, 30, 36, 38, 39, 43, 53, 58, 61, 71, 75, 79, 89, 100, 108, 120, 124, 130, 140, 148, 154, 161, 176, 188, 194, 206, 218, 228, 243, 252, 266, 283, 287, 290, 301, 309, 309, 313, 318, 322, 323, 325, 331, 337, 341, 342, 344, 351, 351, 354, 355, 358, 361, 363, 363, 363, 364, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 0, 0, 0, 0, 1, 2, 2, 3, 3, 4, 6, 7, 7, 9, 9, 10, 10, 11, 13, 14, 19, 20, 22, 28, 34, 36, 40, 44, 50, 55, 64, 70, 74, 80, 82, 86, 91, 103, 109, 119, 130, 137, 160, 178, 188, 210, 219, 234, 251, 266, 287, 302, 316, 349, 370, 398, 412, 425, 455, 482, 498, 523, 563, 595, 629, 655, 694, 755, 783, 819, 864, 921, 940, 991, 1040, 1070, 1113, 1132, 1173, 1211, 1260, 1295, 1315, 1369, 1405, 1435, 1462, 1503, 1538, 1574, 1619, 1679, 1708, 1745, 1766, 1804, 1830, 1852, 1870, 1907], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.32550914739386, 81.32550914739386, 81.2909906800138, 81.2909906800138, 81.27373144632378, 81.23921297894374, 81.20469451156369, 81.11839834311357, 81.0666206420435, 81.04936140835348, 80.9285467725233, 80.85950983776321, 80.84225060407319, 80.75595443562305, 80.68691750086296, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.39351052813255, 80.37625129444253, 80.30721435968243, 80.32447359337246, 80.18639972385226, 80.18639972385226, 80.13462202278218, 79.89299275112185, 79.82395581636176, 79.75491888160165, 79.59958577839144, 79.37521574042113, 79.23714187090093, 79.08180876769072, 78.94373489817053, 78.77114256127028, 78.68484639282016, 78.56403175698999, 78.49499482222988, 78.33966171901969, 78.30514325163963, 78.16706938211944, 77.94269934414912, 77.73558853986883, 77.56299620296859, 77.47670003451846, 77.28684846392821, 77.07973765964792, 76.82084915429755, 76.68277528477736, 76.61373835001726, 76.51018294787711, 76.21677597514672, 76.18225750776666, 75.99240593717639, 75.61270279599586, 75.44011045909562, 75.14670348636521, 74.90507421470487, 74.80151881256472, 74.5598895409044, 74.38729720400414, 74.12840869865379, 73.64515015533311, 73.36900241629272, 72.95478080773215, 72.48878149810149, 72.21263375906109, 71.72937521574042, 71.2806351397998, 70.72833966171902, 70.1415257162582, 69.52019330341733, 69.0024162927166, 68.570935450466, 67.93234380393511, 67.32827062478425, 66.87953054884363, 66.25819813600276, 65.70590265792198, 64.94649637556093, 64.32516396272005, 63.738350017259236, 62.97894373489817, 62.23679668622714, 61.615464273386266, 60.925094925785295, 60.11391094235417, 59.23369002416293, 58.47428374180187, 57.594062823610635, 56.748360372799446, 55.79910251984812, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.01296527694784, "threshold_area100": 74.52450811183984, "threshold_area25": 81.28753883327582}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0035147510934621096, "operator": "ge"}, {"budget": 0.02, "threshold": -0.004862976260483265, "operator": "ge"}, {"budget": 0.03, "threshold": -0.005958778318017721, "operator": "ge"}, {"budget": 0.04, "threshold": -0.006726748775690794, "operator": "ge"}, {"budget": 0.05, "threshold": -0.00854812003672123, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01104010734707117, "operator": "ge"}, {"budget": 0.07, "threshold": -0.012473727576434612, "operator": "ge"}, {"budget": 0.08, "threshold": -0.014348694123327732, "operator": "ge"}, {"budget": 0.09, "threshold": -0.017029881477355957, "operator": "ge"}, {"budget": 0.1, "threshold": -0.01918027549982071, "operator": "ge"}, {"budget": 0.11, "threshold": -0.022077379748225212, "operator": "ge"}, {"budget": 0.12, "threshold": -0.025938954204320908, "operator": "ge"}, {"budget": 0.13, "threshold": -0.029204875230789185, "operator": "ge"}, {"budget": 0.14, "threshold": -0.03296282887458801, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03573034331202507, "operator": "ge"}, {"budget": 0.16, "threshold": -0.03975854441523552, "operator": "ge"}, {"budget": 0.17, "threshold": -0.042940158396959305, "operator": "ge"}, {"budget": 0.18, "threshold": -0.04439644515514374, "operator": "ge"}, {"budget": 0.19, "threshold": -0.048601601272821426, "operator": "ge"}, {"budget": 0.2, "threshold": -0.0533132404088974, "operator": "ge"}, {"budget": 0.21, "threshold": -0.058916036039590836, "operator": "ge"}, {"budget": 0.22, "threshold": -0.06130586192011833, "operator": "ge"}, {"budget": 0.23, "threshold": -0.06567063182592392, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0717887207865715, "operator": "ge"}, {"budget": 0.25, "threshold": -0.07682093232870102, "operator": "ge"}, {"budget": 0.26, "threshold": -0.08091072738170624, "operator": "ge"}, {"budget": 0.27, "threshold": -0.08559807389974594, "operator": "ge"}, {"budget": 0.28, "threshold": -0.09159402549266815, "operator": "ge"}, {"budget": 0.29, "threshold": -0.0986008271574974, "operator": "ge"}, {"budget": 0.3, "threshold": -0.10600607097148895, "operator": "ge"}, {"budget": 0.31, "threshold": -0.11637964844703674, "operator": "ge"}, {"budget": 0.32, "threshold": -0.12771214544773102, "operator": "ge"}, {"budget": 0.33, "threshold": -0.1343788355588913, "operator": "ge"}, {"budget": 0.34, "threshold": -0.14219801127910614, "operator": "ge"}, {"budget": 0.35, "threshold": -0.14962050318717957, "operator": "ge"}, {"budget": 0.36, "threshold": -0.15891912579536438, "operator": "ge"}, {"budget": 0.37, "threshold": -0.17304299771785736, "operator": "ge"}, {"budget": 0.38, "threshold": -0.18773770332336426, "operator": "ge"}, {"budget": 0.39, "threshold": -0.2011920064687729, "operator": "ge"}, {"budget": 0.4, "threshold": -0.21209877729415894, "operator": "ge"}, {"budget": 0.41, "threshold": -0.22225184738636017, "operator": "ge"}, {"budget": 0.42, "threshold": -0.23625671863555908, "operator": "ge"}, {"budget": 0.43, "threshold": -0.25916579365730286, "operator": "ge"}, {"budget": 0.44, "threshold": -0.2721913754940033, "operator": "ge"}, {"budget": 0.45, "threshold": -0.2826441526412964, "operator": "ge"}, {"budget": 0.46, "threshold": -0.29839950799942017, "operator": "ge"}, {"budget": 0.47, "threshold": -0.3106931149959564, "operator": "ge"}, {"budget": 0.48, "threshold": -0.321388840675354, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3407030701637268, "operator": "ge"}, {"budget": 0.5, "threshold": -0.35410991311073303, "operator": "ge"}, {"budget": 0.51, "threshold": -0.36935514211654663, "operator": "ge"}, {"budget": 0.52, "threshold": -0.38521191477775574, "operator": "ge"}, {"budget": 0.53, "threshold": -0.39922407269477844, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4229128658771515, "operator": "ge"}, {"budget": 0.55, "threshold": -0.4380481243133545, "operator": "ge"}, {"budget": 0.56, "threshold": -0.45623597502708435, "operator": "ge"}, {"budget": 0.57, "threshold": -0.467838317155838, "operator": "ge"}, {"budget": 0.58, "threshold": -0.4787340462207794, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5019978880882263, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5222599506378174, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5366401672363281, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5562952756881714, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5803082585334778, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5985275506973267, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6253252029418945, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6418166160583496, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6671537160873413, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6937616467475891, "operator": "ge"}, {"budget": 0.69, "threshold": -0.708234429359436, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7267551422119141, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7472893595695496, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7714237570762634, "operator": "ge"}, {"budget": 0.73, "threshold": -0.781021237373352, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7996335029602051, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8166395425796509, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8280400037765503, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8378686904907227, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8433469533920288, "operator": "ge"}, {"budget": 0.79, "threshold": -0.856132984161377, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8656450510025024, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8807071447372437, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8895121812820435, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8958224058151245, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9083128571510315, "operator": "ge"}, {"budget": 0.85, "threshold": -0.915919840335846, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9220972061157227, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9270078539848328, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9365485906600952, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9463539123535156, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9537533521652222, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9611157178878784, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9703249335289001, "operator": "ge"}, {"budget": 0.93, "threshold": -0.975242555141449, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9808505773544312, "operator": "ge"}, {"budget": 0.95, "threshold": -0.983691394329071, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9874285459518433, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9902699589729309, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9924488067626953, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9942666888237, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.08333333333333, 98.0, 98.0, 97.83333333333334, 97.58333333333333, 97.41666666666666, 97.25, 97.16666666666667, 97.16666666666667, 97.0, 96.91666666666666, 96.75, 96.58333333333333, 96.41666666666666, 96.33333333333334, 95.83333333333334, 95.75, 95.58333333333333, 95.16666666666667, 94.83333333333334, 94.41666666666667, 94.16666666666667, 93.91666666666667, 93.5, 92.91666666666667, 92.66666666666666, 92.25, 92.0, 91.75, 91.16666666666666, 90.58333333333334, 89.91666666666667, 89.33333333333333, 88.91666666666667, 88.58333333333334, 88.16666666666667, 87.75, 87.16666666666667, 86.66666666666667, 86.25, 85.41666666666667, 85.0, 84.58333333333333, 84.25, 83.58333333333333, 82.91666666666666, 82.5, 81.66666666666667, 80.75, 80.08333333333333, 79.5, 78.58333333333334, 77.75, 76.91666666666667, 76.33333333333333, 75.58333333333334, 74.91666666666667, 74.0, 73.25, 72.41666666666666, 71.66666666666667, 70.66666666666667, 69.66666666666667, 68.66666666666667, 67.91666666666667, 67.25, 66.41666666666667, 65.58333333333333, 64.75, 63.74999999999999, 62.74999999999999, 61.83333333333334, 60.916666666666664, 59.916666666666664, 58.916666666666664, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 4, 4, 6, 7, 7, 9, 12, 14, 16, 17, 17, 19, 20, 22, 24, 26, 27, 33, 35, 37, 42, 46, 51, 54, 57, 62, 69, 72, 77, 80, 83, 90, 97, 105, 112, 117, 121, 126, 131, 138, 144, 149, 159, 164, 169, 174, 182, 190, 195, 205, 216, 224, 231, 242, 252, 262, 269, 278, 286, 297, 306, 316, 325, 337, 349, 361, 370, 378, 388, 398, 408, 420, 432, 443, 454, 466, 478, 489, 501, 512, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.08333333333333, 98.0, 98.0, 97.83333333333334, 97.58333333333333, 97.41666666666666, 97.25, 97.16666666666667, 97.16666666666667, 97.0, 96.91666666666666, 96.75, 96.58333333333333, 96.41666666666666, 96.33333333333334, 95.83333333333334, 95.75, 95.58333333333333, 95.16666666666667, 94.83333333333334, 94.41666666666667, 94.16666666666667, 93.91666666666667, 93.5, 92.91666666666667, 92.66666666666666, 92.25, 92.0, 91.75, 91.16666666666666, 90.58333333333334, 89.91666666666667, 89.33333333333333, 88.91666666666667, 88.58333333333334, 88.08333333333334, 87.75, 87.16666666666667, 86.66666666666667, 86.25, 85.41666666666667, 85.0, 84.58333333333333, 84.25, 83.58333333333333, 82.91666666666666, 82.5, 81.58333333333333, 80.75, 80.08333333333333, 79.5, 78.58333333333334, 77.75, 76.91666666666667, 76.33333333333333, 75.58333333333334, 74.91666666666667, 74.0, 73.25, 72.41666666666666, 71.58333333333333, 70.66666666666667, 69.66666666666667, 68.66666666666667, 67.91666666666667, 67.25, 66.41666666666667, 65.58333333333333, 64.75, 63.74999999999999, 62.74999999999999, 61.83333333333334, 60.916666666666664, 59.916666666666664, 58.916666666666664, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.33916666666666, "threshold_area100": 86.34166666666667, "threshold_area25": 98.39166666666667}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.32550914739386, 81.32550914739386, 81.2909906800138, 81.2909906800138, 81.27373144632378, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.20469451156369, 81.11839834311357, 81.10113910942354, 81.0666206420435, 80.9975837072834, 80.91128753883328, 80.91128753883328, 80.84225060407319, 80.84225060407319, 80.73869520193303, 80.70417673455299, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.34173282706249, 80.37625129444253, 80.34173282706249, 80.30721435968243, 80.16914049016223, 80.16914049016223, 80.03106662064205, 79.8584742837418, 79.80669658267173, 79.58232654470142, 79.3579565067311, 79.23714187090093, 79.03003106662064, 78.94373489817053, 78.75388332758025, 78.63306869175008, 78.56403175698999, 78.33966171901969, 78.28788401794961, 78.11529168104936, 77.64929237141871, 77.45944080082845, 77.11425612702796, 76.97618225750776, 76.87262685536763, 76.61373835001726, 76.35484984466689, 76.18225750776666, 75.95788746979635, 75.47462892647566, 75.09492578529513, 74.76700034518467, 74.47359337245426, 74.04211253020367, 73.2827062478426, 72.86848463928202, 72.29892992751122, 71.71211598205039, 70.86641353123922, 70.53848809112876, 69.72730410769762, 68.96789782533655, 68.51915774939593, 67.79426993441491, 67.5008629616845, 66.89678978253366, 66.34449430445288, 65.5678287884018, 64.98101484294098, 64.67034863652054, 63.85916465308941, 63.23783224024854, 62.771832930617876, 62.323092854677256, 61.66724197445633, 61.114946496375566, 60.528132550914734, 59.75146703486365, 58.715913013462206, 58.23265447014153, 57.61132205730066, 57.248878149810146, 56.593027269589236, 56.14428719364861, 55.76458405246807, 55.45391784604764, 54.81532619951675], "calls": [0.0, 0.8111839834311357, 1.7777010700724887, 2.5198481187435275, 2.8822920262340355, 3.624439074905074, 4.780807732136693, 5.315843976527442, 5.8336209872281675, 6.748360372799447, 7.369692785640318, 8.353469105971694, 9.31998619261305, 10.217466344494303, 11.097687262685536, 11.61546427338626, 12.478425957887469, 13.09975837072834, 13.324128408698655, 14.187090093199862, 14.963755609250951, 15.930272695892302, 16.309975837072834, 17.311011391094237, 18.260269244045567, 19.140490162236798, 19.65826717293752, 20.210562651018293, 21.125302036589574, 21.884708318950636, 22.782188470831898, 23.938557128063515, 24.991370383154987, 25.81981360027615, 26.786330686917502, 27.338626164998274, 28.253365550569555, 29.806696582671727, 31.239212978943737, 32.430100103555404, 33.275802554366585, 34.15602347255782, 35.381429064549536, 37.12461166724197, 38.073869520193306, 38.97134967207456, 40.17949603037625, 41.00793924749741, 41.95719710044874, 43.389713496720745, 44.68415602347256, 45.875043148084224, 47.204004142216085, 48.343113565757676, 50.62133241284087, 52.071108042802905, 53.572661373835004, 54.556437694166384, 55.31584397652745, 57.05902657921988, 58.664135312392126, 59.69968933379358, 61.13220573006558, 62.927166033828094, 64.39420089748015, 66.25819813600276, 67.4318260269244, 69.08871246116672, 71.0735243355195, 72.16085605799103, 73.576113220573, 75.02588885053504, 76.75181221953746, 77.28684846392821, 78.63306869175008, 80.04832585433206, 81.15291681049361, 82.15395236451502, 82.68898860890576, 83.82809803244736, 84.89817052122885, 86.12357611322057, 86.95201933034173, 87.5043148084225, 88.83327580255437, 89.59268208491544, 90.30031066620641, 90.86986537797722, 91.88816016568865, 92.71660338280981, 93.51052813255092, 94.44252675181221, 95.65067311011391, 96.28926475664481, 97.04867103900587, 97.48015188125648, 98.17052122885744, 98.6365205384881, 99.03348291335865, 99.34414911977908, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 4, 6, 6, 10, 10, 13, 13, 14, 15, 17, 21, 23, 26, 30, 36, 38, 39, 43, 53, 58, 61, 71, 75, 79, 89, 100, 108, 120, 124, 130, 140, 148, 154, 161, 176, 188, 194, 206, 218, 228, 243, 252, 266, 283, 287, 290, 301, 309, 309, 313, 318, 322, 323, 325, 331, 337, 341, 342, 344, 351, 351, 354, 355, 358, 361, 363, 363, 363, 364, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 0, 0, 0, 0, 1, 2, 2, 3, 3, 4, 6, 7, 7, 9, 9, 10, 10, 11, 13, 14, 19, 20, 22, 28, 34, 36, 40, 44, 50, 55, 64, 70, 74, 80, 82, 86, 91, 103, 109, 119, 130, 137, 160, 178, 188, 210, 219, 234, 251, 266, 287, 302, 316, 349, 370, 398, 412, 425, 455, 482, 498, 523, 563, 595, 629, 655, 694, 755, 783, 819, 864, 921, 940, 991, 1040, 1070, 1113, 1132, 1173, 1211, 1260, 1295, 1315, 1369, 1405, 1435, 1462, 1503, 1538, 1574, 1619, 1679, 1708, 1745, 1766, 1804, 1830, 1852, 1870, 1907], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.32550914739386, 81.32550914739386, 81.2909906800138, 81.2909906800138, 81.27373144632378, 81.23921297894374, 81.20469451156369, 81.11839834311357, 81.0666206420435, 81.04936140835348, 80.9285467725233, 80.85950983776321, 80.84225060407319, 80.75595443562305, 80.68691750086296, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.39351052813255, 80.37625129444253, 80.30721435968243, 80.32447359337246, 80.18639972385226, 80.18639972385226, 80.13462202278218, 79.89299275112185, 79.82395581636176, 79.75491888160165, 79.59958577839144, 79.37521574042113, 79.23714187090093, 79.08180876769072, 78.94373489817053, 78.77114256127028, 78.68484639282016, 78.56403175698999, 78.49499482222988, 78.33966171901969, 78.30514325163963, 78.16706938211944, 77.94269934414912, 77.73558853986883, 77.56299620296859, 77.47670003451846, 77.28684846392821, 77.07973765964792, 76.82084915429755, 76.68277528477736, 76.61373835001726, 76.51018294787711, 76.21677597514672, 76.18225750776666, 75.99240593717639, 75.61270279599586, 75.44011045909562, 75.14670348636521, 74.90507421470487, 74.80151881256472, 74.5598895409044, 74.38729720400414, 74.12840869865379, 73.64515015533311, 73.36900241629272, 72.95478080773215, 72.48878149810149, 72.21263375906109, 71.72937521574042, 71.2806351397998, 70.72833966171902, 70.1415257162582, 69.52019330341733, 69.0024162927166, 68.570935450466, 67.93234380393511, 67.32827062478425, 66.87953054884363, 66.25819813600276, 65.70590265792198, 64.94649637556093, 64.32516396272005, 63.738350017259236, 62.97894373489817, 62.23679668622714, 61.615464273386266, 60.925094925785295, 60.11391094235417, 59.23369002416293, 58.47428374180187, 57.594062823610635, 56.748360372799446, 55.79910251984812, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.01296527694784, "threshold_area100": 74.52450811183984, "threshold_area25": 81.28753883327582}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.7140793800354004, "operator": "ge"}, {"budget": 0.02, "threshold": -3.438946008682251, "operator": "ge"}, {"budget": 0.03, "threshold": -3.894822359085083, "operator": "ge"}, {"budget": 0.04, "threshold": -4.373204231262207, "operator": "ge"}, {"budget": 0.05, "threshold": -4.6801533699035645, "operator": "ge"}, {"budget": 0.06, "threshold": -5.222865581512451, "operator": "ge"}, {"budget": 0.07, "threshold": -5.500415802001953, "operator": "ge"}, {"budget": 0.08, "threshold": -5.85277795791626, "operator": "ge"}, {"budget": 0.09, "threshold": -6.145134925842285, "operator": "ge"}, {"budget": 0.1, "threshold": -6.425432205200195, "operator": "ge"}, {"budget": 0.11, "threshold": -6.721692085266113, "operator": "ge"}, {"budget": 0.12, "threshold": -7.008862495422363, "operator": "ge"}, {"budget": 0.13, "threshold": -7.165759086608887, "operator": "ge"}, {"budget": 0.14, "threshold": -7.4123430252075195, "operator": "ge"}, {"budget": 0.15, "threshold": -7.613123416900635, "operator": "ge"}, {"budget": 0.16, "threshold": -7.839532375335693, "operator": "ge"}, {"budget": 0.17, "threshold": -8.012378692626953, "operator": "ge"}, {"budget": 0.18, "threshold": -8.209273338317871, "operator": "ge"}, {"budget": 0.19, "threshold": -8.415201187133789, "operator": "ge"}, {"budget": 0.2, "threshold": -8.636344909667969, "operator": "ge"}, {"budget": 0.21, "threshold": -8.792269706726074, "operator": "ge"}, {"budget": 0.22, "threshold": -8.946111679077148, "operator": "ge"}, {"budget": 0.23, "threshold": -9.028448104858398, "operator": "ge"}, {"budget": 0.24, "threshold": -9.144393920898438, "operator": "ge"}, {"budget": 0.25, "threshold": -9.293400764465332, "operator": "ge"}, {"budget": 0.26, "threshold": -9.411436080932617, "operator": "ge"}, {"budget": 0.27, "threshold": -9.51431655883789, "operator": "ge"}, {"budget": 0.28, "threshold": -9.590877532958984, "operator": "ge"}, {"budget": 0.29, "threshold": -9.702300071716309, "operator": "ge"}, {"budget": 0.3, "threshold": -9.825119972229004, "operator": "ge"}, {"budget": 0.31, "threshold": -9.889363288879395, "operator": "ge"}, {"budget": 0.32, "threshold": -9.964670181274414, "operator": "ge"}, {"budget": 0.33, "threshold": -10.029831886291504, "operator": "ge"}, {"budget": 0.34, "threshold": -10.094985961914062, "operator": "ge"}, {"budget": 0.35, "threshold": -10.131464958190918, "operator": "ge"}, {"budget": 0.36, "threshold": -10.171127319335938, "operator": "ge"}, {"budget": 0.37, "threshold": -10.231128692626953, "operator": "ge"}, {"budget": 0.38, "threshold": -10.299059867858887, "operator": "ge"}, {"budget": 0.39, "threshold": -10.336823463439941, "operator": "ge"}, {"budget": 0.4, "threshold": -10.368650436401367, "operator": "ge"}, {"budget": 0.41, "threshold": -10.419641494750977, "operator": "ge"}, {"budget": 0.42, "threshold": -10.493046760559082, "operator": "ge"}, {"budget": 0.43, "threshold": -10.546884536743164, "operator": "ge"}, {"budget": 0.44, "threshold": -10.586587905883789, "operator": "ge"}, {"budget": 0.45, "threshold": -10.623796463012695, "operator": "ge"}, {"budget": 0.46, "threshold": -10.67824649810791, "operator": "ge"}, {"budget": 0.47, "threshold": -10.713786125183105, "operator": "ge"}, {"budget": 0.48, "threshold": -10.736589431762695, "operator": "ge"}, {"budget": 0.49, "threshold": -10.811405181884766, "operator": "ge"}, {"budget": 0.5, "threshold": -10.860966682434082, "operator": "ge"}, {"budget": 0.51, "threshold": -10.924915313720703, "operator": "ge"}, {"budget": 0.52, "threshold": -10.955754280090332, "operator": "ge"}, {"budget": 0.53, "threshold": -10.976844787597656, "operator": "ge"}, {"budget": 0.54, "threshold": -10.999963760375977, "operator": "ge"}, {"budget": 0.55, "threshold": -11.044801712036133, "operator": "ge"}, {"budget": 0.56, "threshold": -11.097976684570312, "operator": "ge"}, {"budget": 0.57, "threshold": -11.12870979309082, "operator": "ge"}, {"budget": 0.58, "threshold": -11.1575927734375, "operator": "ge"}, {"budget": 0.59, "threshold": -11.180644989013672, "operator": "ge"}, {"budget": 0.6, "threshold": -11.21660327911377, "operator": "ge"}, {"budget": 0.61, "threshold": -11.243936538696289, "operator": "ge"}, {"budget": 0.62, "threshold": -11.296487808227539, "operator": "ge"}, {"budget": 0.63, "threshold": -11.332469940185547, "operator": "ge"}, {"budget": 0.64, "threshold": -11.374946594238281, "operator": "ge"}, {"budget": 0.65, "threshold": -11.41341495513916, "operator": "ge"}, {"budget": 0.66, "threshold": -11.458240509033203, "operator": "ge"}, {"budget": 0.67, "threshold": -11.505064964294434, "operator": "ge"}, {"budget": 0.68, "threshold": -11.545747756958008, "operator": "ge"}, {"budget": 0.69, "threshold": -11.576164245605469, "operator": "ge"}, {"budget": 0.7, "threshold": -11.613048553466797, "operator": "ge"}, {"budget": 0.71, "threshold": -11.667732238769531, "operator": "ge"}, {"budget": 0.72, "threshold": -11.708873748779297, "operator": "ge"}, {"budget": 0.73, "threshold": -11.753090858459473, "operator": "ge"}, {"budget": 0.74, "threshold": -11.807941436767578, "operator": "ge"}, {"budget": 0.75, "threshold": -11.839826583862305, "operator": "ge"}, {"budget": 0.76, "threshold": -11.889188766479492, "operator": "ge"}, {"budget": 0.77, "threshold": -11.924442291259766, "operator": "ge"}, {"budget": 0.78, "threshold": -11.967779159545898, "operator": "ge"}, {"budget": 0.79, "threshold": -12.021002769470215, "operator": "ge"}, {"budget": 0.8, "threshold": -12.077291488647461, "operator": "ge"}, {"budget": 0.81, "threshold": -12.158405303955078, "operator": "ge"}, {"budget": 0.82, "threshold": -12.237895011901855, "operator": "ge"}, {"budget": 0.83, "threshold": -12.312013626098633, "operator": "ge"}, {"budget": 0.84, "threshold": -12.357112884521484, "operator": "ge"}, {"budget": 0.85, "threshold": -12.408843994140625, "operator": "ge"}, {"budget": 0.86, "threshold": -12.446671485900879, "operator": "ge"}, {"budget": 0.87, "threshold": -12.523115158081055, "operator": "ge"}, {"budget": 0.88, "threshold": -12.58499813079834, "operator": "ge"}, {"budget": 0.89, "threshold": -12.65081787109375, "operator": "ge"}, {"budget": 0.9, "threshold": -12.721450805664062, "operator": "ge"}, {"budget": 0.91, "threshold": -12.787553787231445, "operator": "ge"}, {"budget": 0.92, "threshold": -12.863877296447754, "operator": "ge"}, {"budget": 0.93, "threshold": -12.95071792602539, "operator": "ge"}, {"budget": 0.94, "threshold": -13.062498092651367, "operator": "ge"}, {"budget": 0.95, "threshold": -13.17424201965332, "operator": "ge"}, {"budget": 0.96, "threshold": -13.241031646728516, "operator": "ge"}, {"budget": 0.97, "threshold": -13.302789688110352, "operator": "ge"}, {"budget": 0.98, "threshold": -13.495340347290039, "operator": "ge"}, {"budget": 0.99, "threshold": -13.679011344909668, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.0, 97.41666666666666, 97.0, 96.33333333333334, 95.83333333333334, 95.33333333333334, 94.66666666666667, 94.08333333333333, 93.66666666666667, 92.83333333333333, 92.41666666666667, 91.66666666666666, 91.0, 90.41666666666667, 89.58333333333334, 88.75, 88.33333333333333, 87.66666666666667, 86.91666666666666, 86.25, 85.91666666666666, 85.41666666666667, 85.08333333333333, 84.75, 84.25, 83.75, 83.25, 83.0, 82.58333333333333, 82.16666666666667, 81.83333333333334, 81.33333333333333, 81.08333333333333, 80.75, 80.41666666666667, 80.0, 79.75, 79.25, 78.83333333333333, 78.58333333333334, 78.08333333333334, 77.58333333333334, 77.25, 76.83333333333333, 76.66666666666666, 76.25, 75.83333333333333, 75.41666666666667, 75.0, 74.58333333333333, 74.0, 73.41666666666666, 73.0, 72.41666666666666, 72.0, 71.75, 71.33333333333334, 71.0, 70.66666666666667, 70.5, 70.0, 69.83333333333334, 69.41666666666666, 68.91666666666667, 68.5, 68.16666666666666, 67.83333333333333, 67.33333333333333, 67.08333333333334, 66.66666666666667, 66.58333333333333, 66.33333333333333, 65.91666666666667, 65.58333333333333, 65.08333333333334, 64.5, 64.16666666666666, 63.74999999999999, 63.416666666666664, 63.0, 62.583333333333336, 62.33333333333333, 61.916666666666664, 61.41666666666667, 61.083333333333336, 60.916666666666664, 60.41666666666667, 59.916666666666664, 59.583333333333336, 59.166666666666664, 58.75, 58.41666666666667, 58.08333333333333, 57.75, 57.333333333333336, 56.833333333333336, 56.58333333333334, 56.00000000000001, 55.49999999999999, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 7, 14, 19, 27, 33, 39, 47, 54, 60, 70, 76, 85, 93, 100, 110, 120, 125, 133, 142, 150, 154, 160, 164, 168, 174, 180, 186, 189, 194, 199, 203, 209, 212, 216, 220, 225, 228, 234, 239, 242, 248, 254, 258, 263, 265, 270, 275, 280, 285, 290, 297, 304, 309, 316, 321, 324, 329, 333, 337, 339, 345, 347, 352, 358, 363, 367, 371, 377, 380, 385, 386, 389, 394, 398, 404, 411, 415, 420, 424, 429, 434, 437, 442, 448, 452, 454, 460, 466, 470, 475, 480, 484, 488, 492, 497, 503, 506, 513, 519, 524], "exact_score": [98.58333333333333, 98.0, 97.41666666666666, 97.0, 96.33333333333334, 95.83333333333334, 95.33333333333334, 94.66666666666667, 94.08333333333333, 93.66666666666667, 92.83333333333333, 92.41666666666667, 91.66666666666666, 91.0, 90.41666666666667, 89.58333333333334, 88.75, 88.33333333333333, 87.66666666666667, 86.91666666666666, 86.25, 85.91666666666666, 85.41666666666667, 85.08333333333333, 84.75, 84.25, 83.75, 83.25, 83.0, 82.58333333333333, 82.16666666666667, 81.83333333333334, 81.33333333333333, 81.08333333333333, 80.75, 80.41666666666667, 80.0, 79.75, 79.25, 78.83333333333333, 78.58333333333334, 78.08333333333334, 77.58333333333334, 77.25, 76.83333333333333, 76.66666666666666, 76.25, 75.83333333333333, 75.41666666666667, 75.0, 74.58333333333333, 74.0, 73.41666666666666, 73.0, 72.41666666666666, 72.0, 71.75, 71.33333333333334, 71.0, 70.66666666666667, 70.5, 70.0, 69.83333333333334, 69.41666666666666, 68.91666666666667, 68.5, 68.16666666666666, 67.83333333333333, 67.33333333333333, 67.0, 66.66666666666667, 66.58333333333333, 66.33333333333333, 65.91666666666667, 65.58333333333333, 65.08333333333334, 64.5, 64.16666666666666, 63.74999999999999, 63.416666666666664, 63.0, 62.583333333333336, 62.33333333333333, 61.916666666666664, 61.41666666666667, 61.083333333333336, 60.916666666666664, 60.41666666666667, 59.916666666666664, 59.583333333333336, 59.166666666666664, 58.75, 58.41666666666667, 58.08333333333333, 57.75, 57.333333333333336, 56.833333333333336, 56.58333333333334, 56.00000000000001, 55.49999999999999, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 75.07166666666667, "threshold_area100": 75.07249999999999, "threshold_area25": 91.18333333333334}, "test": {"score": [81.429064549534, 81.20469451156369, 80.80773213669313, 80.23817742492233, 79.40973420780118, 78.89195719710045, 77.32136693130826, 76.49292371418709, 75.49188816016569, 74.85329651363479, 74.2147048671039, 73.43803935105282, 72.48878149810149, 72.17811529168105, 71.62581981360027, 71.02174663444943, 70.46945115636866, 70.17604418363825, 69.81360027614774, 69.58923023817742, 69.0714532274767, 68.7780462547463, 68.55367621677597, 68.38108387987573, 68.12219537452538, 67.79426993441491, 67.53538142906454, 67.22471522264412, 67.05212288574387, 66.82775284777355, 66.46530894028305, 66.34449430445288, 66.08560579910252, 65.87849499482223, 65.53331032102174, 65.39523645150156, 65.17086641353124, 64.87745944080083, 64.60131170176044, 64.46323783224025, 64.34242319641008, 64.20434932688988, 63.772868484639275, 63.58301691404902, 63.41042457714878, 63.22057300655851, 62.97894373489817, 62.72005522954781, 62.616499827407665, 62.37487055574733, 62.08146358301691, 61.66724197445633, 61.563686572316186, 61.477390403866075, 61.44287193648602, 61.09768726268554, 60.925094925785295, 60.75250258888505, 60.64894718674492, 60.528132550914734, 60.39005868139456, 60.37279944770453, 60.16568864342423, 59.993096306523995, 59.82050396962375, 59.61339316534347, 59.371763893683124, 59.16465308940283, 59.078356920952714, 58.99206075250258, 58.8539868829824, 58.73317224715223, 58.62961684501209, 58.56057991025199, 58.38798757335175, 58.31895063859165, 58.11183983431136, 57.88746979634105, 57.76665516051087, 57.50776665516051, 57.3524335519503, 57.04176734552986, 56.817397307559546, 56.748360372799446, 56.679323438039354, 56.541249568519156, 56.472212633759064, 56.368657231618926, 56.21332412840869, 56.05799102519848, 55.95443562305834, 55.8508802209182, 55.74732481877804, 55.591991715567836, 55.47117707973766, 55.3503624439075, 55.24680704176734, 55.17777010700725, 55.03969623748706, 55.005177770107004, 54.81532619951675], "calls": [0.0, 2.847773558853987, 5.902657921988263, 8.905764584052468, 12.996202968588195, 15.740421125302037, 21.81567138419054, 25.198481187435277, 29.789437348981707, 33.15498791853641, 36.69313082499137, 39.73075595443562, 42.25060407317915, 43.527787366240936, 45.167414566793234, 46.75526406627546, 48.3603727994477, 49.63755609250949, 50.75940628236106, 51.950293406972726, 53.503624439074905, 54.40110459095616, 55.02243700379703, 55.57473248187781, 56.42043493268899, 57.594062823610635, 58.47428374180187, 59.54435623058336, 60.11391094235415, 60.873317224715215, 61.977908180876774, 62.478425957887474, 63.22057300655851, 63.85916465308941, 64.60131170176044, 65.03279254401104, 65.63686572316189, 66.3617535381429, 67.46634449430445, 67.88056610286503, 68.29478771142561, 68.8643424231964, 69.88263721090784, 70.5039696237487, 70.91819123230928, 71.43596824301001, 72.22989299275112, 72.74767000345184, 72.98929927511219, 73.86952019330342, 74.73248187780463, 75.81981360027615, 76.35484984466689, 76.56196064894719, 76.83810838798757, 77.56299620296859, 78.27062478425958, 78.70210562651019, 79.06454953400069, 79.46151190887124, 79.9275112185019, 80.2209181912323, 80.77321366931308, 81.32550914739386, 81.89506385916465, 82.6372109078357, 83.2930617880566, 83.87987573351742, 84.39765274421815, 84.81187435277874, 85.19157749395927, 85.88194684156024, 86.33068691750087, 86.77942699344149, 87.38350017259233, 87.72868484639282, 88.4880911287539, 88.97134967207457, 89.52364515015533, 90.2140144977563, 90.74905074214705, 91.62927166033829, 92.61304798066966, 93.30341732827063, 93.63134276838109, 94.06282361063168, 94.30445288229203, 94.8567483603728, 95.27096996893339, 95.75422851225406, 96.16845012081463, 96.59993096306523, 97.03141180531586, 97.61822575077666, 98.08422506040732, 98.53296513634795, 98.74007594062823, 98.87814981014843, 99.43044528822921, 99.6375560925095, 100.0], "rescued": [0, 40, 69, 93, 133, 161, 205, 229, 260, 283, 316, 330, 333, 336, 340, 341, 344, 349, 351, 353, 354, 355, 355, 355, 356, 356, 356, 356, 356, 356, 357, 357, 357, 357, 357, 357, 357, 357, 358, 358, 358, 358, 358, 358, 358, 358, 359, 359, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 362, 362, 362, 362, 362, 363, 363, 364, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 53, 105, 162, 250, 308, 443, 515, 604, 664, 734, 793, 851, 872, 908, 944, 979, 1001, 1024, 1039, 1070, 1088, 1101, 1111, 1127, 1146, 1161, 1179, 1189, 1202, 1224, 1231, 1246, 1258, 1278, 1286, 1299, 1316, 1333, 1341, 1348, 1356, 1381, 1392, 1402, 1413, 1428, 1443, 1449, 1464, 1481, 1505, 1511, 1516, 1518, 1538, 1548, 1558, 1564, 1571, 1579, 1580, 1592, 1602, 1612, 1624, 1639, 1651, 1657, 1662, 1670, 1677, 1683, 1688, 1698, 1703, 1716, 1729, 1736, 1751, 1760, 1778, 1791, 1795, 1799, 1807, 1811, 1817, 1826, 1835, 1841, 1847, 1853, 1862, 1869, 1876, 1882, 1886, 1894, 1896, 1907], "exact_score": [81.429064549534, 81.4981014842941, 81.2909906800138, 81.27373144632378, 81.08387987573352, 80.87676907145323, 80.80773213669313, 80.61788056610285, 80.44528822920263, 80.2899551259924, 80.15188125647221, 79.84121505005179, 79.61684501208146, 79.40973420780118, 79.28891957197101, 79.09906800138074, 78.8574387297204, 78.56403175698999, 78.30514325163963, 78.02899551259924, 77.92544011045909, 77.44218156713842, 77.26958923023818, 76.95892302381775, 76.70003451846739, 76.54470141525717, 76.32033137728685, 76.04418363824647, 75.88885053503624, 75.68173973075596, 75.49188816016569, 75.23299965481533, 75.11218501898516, 74.88781498101484, 74.66344494304454, 74.5598895409044, 74.38729720400414, 74.0766309975837, 73.7832240248533, 73.61063168795306, 73.2827062478426, 72.97204004142216, 72.52329996548154, 72.33344839489126, 72.05730065585088, 71.6430790472903, 71.33241284086986, 70.98722816706939, 70.6420434932689, 70.33137728684846, 70.05522954780807, 69.7790818087677, 69.5719710044874, 69.2095270969969, 68.84708318950639, 68.58819468415602, 68.29478771142561, 67.94960303762512, 67.65619606489473, 67.3627890921643, 67.08664135312392, 66.7932343803935, 66.46530894028305, 66.10286503279255, 65.80945806006213, 65.39523645150156, 64.998274076631, 64.72212633759061, 64.44597859855023, 64.15257162581982, 63.738350017259236, 63.393165343458755, 63.06523990334829, 62.616499827407665, 62.3576113220573, 62.01242664825681, 61.63272350707629, 61.339316534345876, 60.976872626855375, 60.64894718674492, 60.39005868139456, 60.06213324128409, 59.803244735933724, 59.440800828443216, 59.14739385571281, 58.92302381774249, 58.73317224715223, 58.49154297549188, 58.2844321712116, 57.88746979634105, 57.57680358992061, 57.30065585088022, 56.97273041076977, 56.76561960648947, 56.55850880220918, 56.33413876423887, 56.02347255781843, 55.74732481877804, 55.5056955471177, 55.12599240593718, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 69.38078641576176, "threshold_area100": 63.678633068691745, "threshold_area25": 73.56230583362098}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.7755550146102905, "operator": "ge"}, {"budget": 0.02, "threshold": -1.3035717010498047, "operator": "ge"}, {"budget": 0.03, "threshold": -1.6259263753890991, "operator": "ge"}, {"budget": 0.04, "threshold": -1.931239366531372, "operator": "ge"}, {"budget": 0.05, "threshold": -2.185121536254883, "operator": "ge"}, {"budget": 0.06, "threshold": -2.4510223865509033, "operator": "ge"}, {"budget": 0.07, "threshold": -2.744386911392212, "operator": "ge"}, {"budget": 0.08, "threshold": -2.9762938022613525, "operator": "ge"}, {"budget": 0.09, "threshold": -3.1322457790374756, "operator": "ge"}, {"budget": 0.1, "threshold": -3.329434871673584, "operator": "ge"}, {"budget": 0.11, "threshold": -3.649711847305298, "operator": "ge"}, {"budget": 0.12, "threshold": -3.9184389114379883, "operator": "ge"}, {"budget": 0.13, "threshold": -4.269773006439209, "operator": "ge"}, {"budget": 0.14, "threshold": -4.464216232299805, "operator": "ge"}, {"budget": 0.15, "threshold": -4.766515731811523, "operator": "ge"}, {"budget": 0.16, "threshold": -5.021233081817627, "operator": "ge"}, {"budget": 0.17, "threshold": -5.1463212966918945, "operator": "ge"}, {"budget": 0.18, "threshold": -5.36373233795166, "operator": "ge"}, {"budget": 0.19, "threshold": -5.576329708099365, "operator": "ge"}, {"budget": 0.2, "threshold": -5.788539886474609, "operator": "ge"}, {"budget": 0.21, "threshold": -6.023674011230469, "operator": "ge"}, {"budget": 0.22, "threshold": -6.1206769943237305, "operator": "ge"}, {"budget": 0.23, "threshold": -6.299748420715332, "operator": "ge"}, {"budget": 0.24, "threshold": -6.505497932434082, "operator": "ge"}, {"budget": 0.25, "threshold": -6.646816253662109, "operator": "ge"}, {"budget": 0.26, "threshold": -6.814128398895264, "operator": "ge"}, {"budget": 0.27, "threshold": -6.978409290313721, "operator": "ge"}, {"budget": 0.28, "threshold": -7.073182106018066, "operator": "ge"}, {"budget": 0.29, "threshold": -7.126215934753418, "operator": "ge"}, {"budget": 0.3, "threshold": -7.297386169433594, "operator": "ge"}, {"budget": 0.31, "threshold": -7.376584529876709, "operator": "ge"}, {"budget": 0.32, "threshold": -7.449786186218262, "operator": "ge"}, {"budget": 0.33, "threshold": -7.491927146911621, "operator": "ge"}, {"budget": 0.34, "threshold": -7.529845714569092, "operator": "ge"}, {"budget": 0.35, "threshold": -7.616968154907227, "operator": "ge"}, {"budget": 0.36, "threshold": -7.679852485656738, "operator": "ge"}, {"budget": 0.37, "threshold": -7.739450931549072, "operator": "ge"}, {"budget": 0.38, "threshold": -7.825156211853027, "operator": "ge"}, {"budget": 0.39, "threshold": -7.870200157165527, "operator": "ge"}, {"budget": 0.4, "threshold": -7.917389869689941, "operator": "ge"}, {"budget": 0.41, "threshold": -7.965387344360352, "operator": "ge"}, {"budget": 0.42, "threshold": -7.999626159667969, "operator": "ge"}, {"budget": 0.43, "threshold": -8.04222583770752, "operator": "ge"}, {"budget": 0.44, "threshold": -8.124732971191406, "operator": "ge"}, {"budget": 0.45, "threshold": -8.168571472167969, "operator": "ge"}, {"budget": 0.46, "threshold": -8.20992660522461, "operator": "ge"}, {"budget": 0.47, "threshold": -8.26936149597168, "operator": "ge"}, {"budget": 0.48, "threshold": -8.306124687194824, "operator": "ge"}, {"budget": 0.49, "threshold": -8.362170219421387, "operator": "ge"}, {"budget": 0.5, "threshold": -8.408601760864258, "operator": "ge"}, {"budget": 0.51, "threshold": -8.445173263549805, "operator": "ge"}, {"budget": 0.52, "threshold": -8.497930526733398, "operator": "ge"}, {"budget": 0.53, "threshold": -8.528223037719727, "operator": "ge"}, {"budget": 0.54, "threshold": -8.575630187988281, "operator": "ge"}, {"budget": 0.55, "threshold": -8.619881629943848, "operator": "ge"}, {"budget": 0.56, "threshold": -8.66649055480957, "operator": "ge"}, {"budget": 0.57, "threshold": -8.704977035522461, "operator": "ge"}, {"budget": 0.58, "threshold": -8.74914836883545, "operator": "ge"}, {"budget": 0.59, "threshold": -8.786575317382812, "operator": "ge"}, {"budget": 0.6, "threshold": -8.826277732849121, "operator": "ge"}, {"budget": 0.61, "threshold": -8.866241455078125, "operator": "ge"}, {"budget": 0.62, "threshold": -8.929118156433105, "operator": "ge"}, {"budget": 0.63, "threshold": -8.950650215148926, "operator": "ge"}, {"budget": 0.64, "threshold": -8.969189643859863, "operator": "ge"}, {"budget": 0.65, "threshold": -8.992374420166016, "operator": "ge"}, {"budget": 0.66, "threshold": -9.032524108886719, "operator": "ge"}, {"budget": 0.67, "threshold": -9.066031455993652, "operator": "ge"}, {"budget": 0.68, "threshold": -9.106071472167969, "operator": "ge"}, {"budget": 0.69, "threshold": -9.129886627197266, "operator": "ge"}, {"budget": 0.7, "threshold": -9.172294616699219, "operator": "ge"}, {"budget": 0.71, "threshold": -9.231005668640137, "operator": "ge"}, {"budget": 0.72, "threshold": -9.263386726379395, "operator": "ge"}, {"budget": 0.73, "threshold": -9.296911239624023, "operator": "ge"}, {"budget": 0.74, "threshold": -9.35315990447998, "operator": "ge"}, {"budget": 0.75, "threshold": -9.395198822021484, "operator": "ge"}, {"budget": 0.76, "threshold": -9.43735408782959, "operator": "ge"}, {"budget": 0.77, "threshold": -9.502967834472656, "operator": "ge"}, {"budget": 0.78, "threshold": -9.558652877807617, "operator": "ge"}, {"budget": 0.79, "threshold": -9.595308303833008, "operator": "ge"}, {"budget": 0.8, "threshold": -9.638570785522461, "operator": "ge"}, {"budget": 0.81, "threshold": -9.688539505004883, "operator": "ge"}, {"budget": 0.82, "threshold": -9.746527671813965, "operator": "ge"}, {"budget": 0.83, "threshold": -9.792844772338867, "operator": "ge"}, {"budget": 0.84, "threshold": -9.841348648071289, "operator": "ge"}, {"budget": 0.85, "threshold": -9.910432815551758, "operator": "ge"}, {"budget": 0.86, "threshold": -9.952404022216797, "operator": "ge"}, {"budget": 0.87, "threshold": -10.016504287719727, "operator": "ge"}, {"budget": 0.88, "threshold": -10.064027786254883, "operator": "ge"}, {"budget": 0.89, "threshold": -10.114227294921875, "operator": "ge"}, {"budget": 0.9, "threshold": -10.17898178100586, "operator": "ge"}, {"budget": 0.91, "threshold": -10.209320068359375, "operator": "ge"}, {"budget": 0.92, "threshold": -10.27849006652832, "operator": "ge"}, {"budget": 0.93, "threshold": -10.419960021972656, "operator": "ge"}, {"budget": 0.94, "threshold": -10.527139663696289, "operator": "ge"}, {"budget": 0.95, "threshold": -10.625962257385254, "operator": "ge"}, {"budget": 0.96, "threshold": -10.765897750854492, "operator": "ge"}, {"budget": 0.97, "threshold": -10.879802703857422, "operator": "ge"}, {"budget": 0.98, "threshold": -10.991156578063965, "operator": "ge"}, {"budget": 0.99, "threshold": -11.248336791992188, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.0, 97.66666666666667, 97.16666666666667, 96.66666666666667, 96.0, 95.5, 95.0, 94.16666666666667, 93.83333333333333, 93.08333333333333, 92.58333333333333, 92.08333333333334, 91.41666666666667, 90.66666666666667, 90.16666666666666, 89.5, 89.0, 88.33333333333333, 87.83333333333333, 87.58333333333333, 87.0, 86.41666666666666, 85.75, 85.25, 84.83333333333334, 84.33333333333334, 84.16666666666667, 84.08333333333333, 84.0, 83.83333333333334, 83.41666666666667, 83.08333333333333, 82.75, 82.41666666666667, 82.16666666666667, 81.83333333333334, 81.5, 81.25, 81.08333333333333, 80.83333333333333, 80.66666666666666, 80.5, 80.25, 79.91666666666667, 79.66666666666666, 79.41666666666667, 79.16666666666667, 79.0, 78.41666666666667, 78.0, 77.58333333333334, 77.08333333333334, 76.83333333333333, 76.5, 76.25, 76.0, 75.75, 75.5, 75.33333333333333, 75.08333333333333, 74.5, 74.41666666666666, 74.16666666666667, 73.66666666666667, 73.41666666666666, 73.08333333333333, 72.58333333333333, 72.16666666666667, 71.58333333333333, 71.16666666666667, 70.83333333333333, 70.33333333333334, 69.58333333333333, 69.08333333333334, 68.5, 67.83333333333333, 67.33333333333333, 67.0, 66.5, 66.08333333333334, 65.41666666666667, 65.08333333333334, 64.83333333333333, 64.25, 63.916666666666664, 63.416666666666664, 63.0, 62.5, 62.083333333333336, 61.41666666666667, 61.0, 60.41666666666667, 59.833333333333336, 59.25, 58.58333333333333, 57.91666666666666, 57.41666666666667, 56.666666666666664, 55.75, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 7, 11, 17, 23, 31, 37, 43, 53, 59, 68, 74, 80, 88, 97, 103, 111, 117, 125, 131, 134, 141, 148, 156, 162, 167, 173, 175, 176, 177, 179, 184, 188, 192, 196, 199, 203, 207, 210, 212, 215, 217, 219, 222, 226, 229, 232, 235, 237, 244, 249, 254, 260, 263, 267, 270, 273, 276, 279, 281, 284, 291, 292, 295, 301, 304, 308, 314, 319, 326, 331, 335, 341, 350, 356, 363, 371, 377, 381, 387, 392, 400, 404, 407, 414, 418, 424, 429, 435, 440, 448, 453, 460, 467, 474, 482, 490, 496, 505, 516, 524], "exact_score": [98.58333333333333, 98.0, 97.66666666666667, 97.16666666666667, 96.66666666666667, 96.0, 95.5, 95.0, 94.16666666666667, 93.83333333333333, 93.08333333333333, 92.58333333333333, 92.08333333333334, 91.41666666666667, 90.66666666666667, 90.16666666666666, 89.5, 89.0, 88.33333333333333, 87.83333333333333, 87.58333333333333, 87.0, 86.41666666666666, 85.75, 85.25, 84.83333333333334, 84.33333333333334, 84.16666666666667, 84.08333333333333, 84.0, 83.83333333333334, 83.41666666666667, 83.08333333333333, 82.75, 82.41666666666667, 82.16666666666667, 81.83333333333334, 81.5, 81.25, 81.08333333333333, 80.83333333333333, 80.66666666666666, 80.5, 80.25, 79.91666666666667, 79.66666666666666, 79.41666666666667, 79.16666666666667, 79.0, 78.41666666666667, 78.0, 77.58333333333334, 77.08333333333334, 76.83333333333333, 76.5, 76.25, 76.0, 75.66666666666667, 75.5, 75.33333333333333, 75.08333333333333, 74.5, 74.41666666666666, 74.16666666666667, 73.66666666666667, 73.41666666666666, 73.08333333333333, 72.58333333333333, 72.16666666666667, 71.58333333333333, 71.16666666666667, 70.83333333333333, 70.33333333333334, 69.58333333333333, 69.08333333333334, 68.5, 67.83333333333333, 67.33333333333333, 67.0, 66.5, 66.08333333333334, 65.41666666666667, 65.0, 64.83333333333333, 64.25, 63.916666666666664, 63.416666666666664, 63.0, 62.5, 62.083333333333336, 61.41666666666667, 61.0, 60.41666666666667, 59.833333333333336, 59.25, 58.58333333333333, 57.91666666666666, 57.41666666666667, 56.666666666666664, 55.75, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 77.36416666666666, "threshold_area100": 77.36583333333333, "threshold_area25": 91.70833333333334}, "test": {"score": [81.429064549534, 81.51536071798412, 80.66965826717293, 80.18639972385226, 79.28891957197101, 78.20158784949949, 77.75284777355886, 76.95892302381775, 76.28581290990681, 75.66448049706594, 74.97411114946496, 73.80048325854332, 72.98929927511219, 72.12633759061097, 71.71211598205039, 70.96996893337936, 70.45219192267864, 70.31411805315844, 69.91715567828788, 69.53745253710736, 69.17500862961685, 68.8643424231964, 68.72626855367622, 68.34656541249569, 68.0013807386952, 67.84604763548498, 67.5698998964446, 67.27649292371419, 67.0693821194339, 66.96582671729375, 66.68967897825337, 66.5688643424232, 66.43079047290301, 66.32723507076285, 66.2927166033828, 66.12012426648258, 66.01656886434243, 65.75768035899206, 65.53331032102174, 65.44701415257163, 65.2916810493614, 65.17086641353124, 65.08457024508112, 64.87745944080083, 64.68760787021056, 64.58405246807042, 64.46323783224025, 64.22160856057991, 64.15257162581982, 63.89368312046946, 63.63479461511909, 63.4967207455989, 63.375906109768735, 63.16879530548844, 62.9962029685882, 62.771832930617876, 62.702795995857784, 62.51294442526751, 62.3576113220573, 62.202278218847084, 62.08146358301691, 61.90887124611668, 61.68450120814636, 61.58094580600621, 61.4946496375561, 61.37383500172593, 61.16672419744563, 61.114946496375566, 60.87331722471523, 60.82153952364515, 60.54539178460476, 60.217466344494305, 60.04487400759406, 59.95857783914394, 59.716948567483605, 59.54435623058336, 59.285467725233, 58.92302381774249, 58.802209181912325, 58.61235761132206, 58.42250604073179, 58.2844321712116, 58.077321366931315, 57.93924749741112, 57.73213669313083, 57.42147048671039, 57.26613738350017, 57.07628581290991, 56.97273041076977, 56.83465654124957, 56.748360372799446, 56.679323438039354, 56.48947186744908, 56.092509492578536, 55.988954090438384, 55.78184328615809, 55.54021401449776, 55.367621677597526, 55.24680704176734, 55.03969623748706, 54.81532619951675], "calls": [0.0, 2.623403520883673, 7.939247497411114, 11.926130479806698, 16.517086641353124, 20.383154987918537, 24.7152226441146, 29.66862271315154, 32.79254401104591, 34.74283741801864, 37.33172247152227, 40.904383845357266, 43.19986192613048, 45.97859855022437, 47.290300310666204, 49.309630652399036, 50.75940628236106, 51.43251639627201, 52.76147739040387, 53.917846047635486, 55.125992405937176, 56.24784259578875, 56.69658267172938, 57.69761822575078, 58.7849499482223, 59.38902312737314, 60.47635484984467, 61.46013117017605, 62.2885743872972, 62.66827752847773, 63.876423886779435, 64.54953400069037, 65.11908871246118, 65.4987918536417, 65.8612357611322, 66.67241974456334, 67.3627890921643, 68.17397307559544, 68.98515705902658, 69.53745253710734, 70.15878494994823, 70.67656196064894, 71.03900586813945, 71.6430790472903, 72.71315153607179, 73.23092854677252, 73.80048325854332, 74.59440800828443, 75.16396272005524, 76.13047980669658, 76.78633068691751, 77.32136693130826, 77.856403175699, 78.46047635484985, 79.11632723507076, 79.73765964791163, 80.11736278909216, 80.721435968243, 81.3600276147739, 81.89506385916465, 82.32654470141526, 82.87884017949602, 83.89713496720745, 84.19054193993787, 84.44943044528823, 84.77735588539869, 85.31239212978944, 85.8474283741802, 86.52053848809112, 86.86572316189162, 87.43527787366241, 88.24646185709355, 88.69520193303417, 89.10942354159475, 89.69623748705557, 90.1449775629962, 90.714532274767, 91.59475319295824, 92.11253020365896, 92.54401104590956, 92.95823265447014, 93.42423196410078, 93.8557128063514, 94.25267518122196, 94.70141525716258, 95.3572661373835, 95.61615464273386, 95.94408008284432, 96.28926475664481, 96.5826717293752, 96.70348636520538, 96.82430100103555, 97.30755954435622, 97.9979288919572, 98.23955816361753, 98.58474283741802, 98.98170521228857, 99.25785295132896, 99.41318605453918, 99.72385226095962, 100.0], "rescued": [0, 38, 83, 126, 160, 179, 224, 273, 296, 305, 317, 327, 335, 344, 348, 350, 351, 352, 354, 356, 356, 356, 356, 356, 356, 357, 358, 358, 358, 358, 358, 358, 358, 358, 358, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 362, 363, 364, 364, 364, 364, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 33, 127, 198, 284, 366, 437, 532, 594, 639, 691, 769, 824, 883, 911, 956, 987, 996, 1021, 1045, 1066, 1084, 1092, 1114, 1134, 1144, 1161, 1178, 1190, 1196, 1212, 1219, 1227, 1233, 1235, 1246, 1252, 1267, 1280, 1285, 1294, 1301, 1306, 1318, 1329, 1335, 1342, 1356, 1360, 1375, 1390, 1399, 1406, 1418, 1428, 1441, 1445, 1456, 1465, 1474, 1481, 1491, 1505, 1511, 1516, 1523, 1535, 1538, 1552, 1555, 1571, 1590, 1600, 1605, 1619, 1629, 1644, 1665, 1673, 1685, 1697, 1705, 1717, 1725, 1738, 1756, 1765, 1776, 1782, 1790, 1795, 1799, 1810, 1833, 1839, 1851, 1865, 1875, 1882, 1894, 1907], "exact_score": [81.429064549534, 81.41180531584398, 81.46358301691406, 81.51536071798412, 81.25647221263375, 81.11839834311357, 80.94580600621333, 80.96306523990336, 80.63513979979288, 80.56610286503279, 80.4280289955126, 80.25543665861235, 80.18639972385226, 79.96202968588196, 79.63410424577148, 79.4960303762513, 79.40973420780118, 79.21988263721092, 78.8574387297204, 78.65032792544011, 78.30514325163963, 78.13255091473938, 77.92544011045909, 77.92544011045909, 77.856403175699, 77.70107007248879, 77.44218156713842, 77.44218156713842, 77.14877459440801, 77.13151536071798, 76.90714532274767, 76.75181221953746, 76.51018294787711, 76.23403520883673, 75.87159130134621, 75.61270279599586, 75.25025888850536, 75.04314808422507, 74.7842595788747, 74.5598895409044, 74.16292716603382, 73.7832240248533, 73.36900241629272, 73.04107697618227, 72.6441146013117, 72.40248532965137, 72.10907835692095, 71.81567138419054, 71.40144977562997, 71.0735243355195, 70.72833966171902, 70.40041422160857, 70.15878494994823, 69.81360027614774, 69.52019330341733, 69.19226786330687, 68.93337935795651, 68.62271315153608, 68.2775284777356, 67.89782533655506, 67.65619606489473, 67.3627890921643, 67.17293752157406, 66.87953054884363, 66.65516051087333, 66.48256817397308, 66.25819813600276, 66.05108733172247, 65.84397652744218, 65.53331032102174, 65.30894028305143, 65.11908871246118, 64.80842250604073, 64.63583016914049, 64.39420089748016, 64.18709009319986, 63.91094235415948, 63.56575768035899, 63.30686917500863, 63.047980669658266, 62.72005522954781, 62.46116672419745, 62.18501898515706, 61.891612012426656, 61.64998274076632, 61.30479806696583, 61.04590956161546, 60.75250258888505, 60.2865032792544, 59.95857783914394, 59.61339316534347, 59.19917155678288, 58.8539868829824, 58.40524680704176, 58.04280289955126, 57.61132205730066, 57.04176734552986, 56.593027269589236, 56.092509492578536, 55.54021401449776, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 70.47196527325413, "threshold_area100": 64.52865032792545, "threshold_area25": 73.59164653089404}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.429064549534, "calls": 0.8111839834311357}}, "old_normalized_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.429064549534, "calls": 0.8111839834311357}}, "old_normalized_endpoints": {"val": {"score": 98.58333333333333, "calls": 0.0}, "test": {"score": 81.429064549534, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.08333333333333, 98.0, 98.0, 97.83333333333334, 97.58333333333333, 97.41666666666666, 97.25, 97.16666666666667, 97.16666666666667, 97.0, 96.91666666666666, 96.75, 96.58333333333333, 96.41666666666666, 96.33333333333334, 95.83333333333334, 95.75, 95.58333333333333, 95.16666666666667, 94.83333333333334, 94.41666666666667, 94.16666666666667, 93.91666666666667, 93.5, 92.91666666666667, 92.66666666666666, 92.25, 92.0, 91.75, 91.16666666666666, 90.58333333333334, 89.91666666666667, 89.33333333333333, 88.91666666666667, 88.58333333333334, 88.16666666666667, 87.75, 87.16666666666667, 86.66666666666667, 86.25, 85.41666666666667, 85.0, 84.58333333333333, 84.25, 83.58333333333333, 82.91666666666666, 82.5, 81.66666666666667, 80.75, 80.08333333333333, 79.5, 78.58333333333334, 77.75, 76.91666666666667, 76.33333333333333, 75.58333333333334, 74.91666666666667, 74.0, 73.25, 72.41666666666666, 71.66666666666667, 70.66666666666667, 69.66666666666667, 68.66666666666667, 67.91666666666667, 67.25, 66.41666666666667, 65.58333333333333, 64.75, 63.74999999999999, 62.74999999999999, 61.83333333333334, 60.916666666666664, 59.916666666666664, 58.916666666666664, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.08333333333333, 98.0, 98.0, 97.83333333333334, 97.58333333333333, 97.41666666666666, 97.25, 97.16666666666667, 97.16666666666667, 97.0, 96.91666666666666, 96.75, 96.58333333333333, 96.41666666666666, 96.33333333333334, 95.83333333333334, 95.75, 95.58333333333333, 95.16666666666667, 94.83333333333334, 94.41666666666667, 94.16666666666667, 93.91666666666667, 93.5, 92.91666666666667, 92.66666666666666, 92.25, 92.0, 91.75, 91.16666666666666, 90.58333333333334, 89.91666666666667, 89.33333333333333, 88.91666666666667, 88.58333333333334, 88.08333333333334, 87.75, 87.16666666666667, 86.66666666666667, 86.25, 85.41666666666667, 85.0, 84.58333333333333, 84.25, 83.58333333333333, 82.91666666666666, 82.5, 81.58333333333333, 80.75, 80.08333333333333, 79.5, 78.58333333333334, 77.75, 76.91666666666667, 76.33333333333333, 75.58333333333334, 74.91666666666667, 74.0, 73.25, 72.41666666666666, 71.58333333333333, 70.66666666666667, 69.66666666666667, 68.66666666666667, 67.91666666666667, 67.25, 66.41666666666667, 65.58333333333333, 64.75, 63.74999999999999, 62.74999999999999, 61.83333333333334, 60.916666666666664, 59.916666666666664, 58.916666666666664, 58.00000000000001, 57.00000000000001, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.39166666666667, "exact_area100": 86.33916666666666}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.32550914739386, 81.32550914739386, 81.2909906800138, 81.2909906800138, 81.27373144632378, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.20469451156369, 81.11839834311357, 81.10113910942354, 81.0666206420435, 80.9975837072834, 80.91128753883328, 80.91128753883328, 80.84225060407319, 80.84225060407319, 80.73869520193303, 80.70417673455299, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.34173282706249, 80.37625129444253, 80.34173282706249, 80.30721435968243, 80.16914049016223, 80.16914049016223, 80.03106662064205, 79.8584742837418, 79.80669658267173, 79.58232654470142, 79.3579565067311, 79.23714187090093, 79.03003106662064, 78.94373489817053, 78.75388332758025, 78.63306869175008, 78.56403175698999, 78.33966171901969, 78.28788401794961, 78.11529168104936, 77.64929237141871, 77.45944080082845, 77.11425612702796, 76.97618225750776, 76.87262685536763, 76.61373835001726, 76.35484984466689, 76.18225750776666, 75.95788746979635, 75.47462892647566, 75.09492578529513, 74.76700034518467, 74.47359337245426, 74.04211253020367, 73.2827062478426, 72.86848463928202, 72.29892992751122, 71.71211598205039, 70.86641353123922, 70.53848809112876, 69.72730410769762, 68.96789782533655, 68.51915774939593, 67.79426993441491, 67.5008629616845, 66.89678978253366, 66.34449430445288, 65.5678287884018, 64.98101484294098, 64.67034863652054, 63.85916465308941, 63.23783224024854, 62.771832930617876, 62.323092854677256, 61.66724197445633, 61.114946496375566, 60.528132550914734, 59.75146703486365, 58.715913013462206, 58.23265447014153, 57.61132205730066, 57.248878149810146, 56.593027269589236, 56.14428719364861, 55.76458405246807, 55.45391784604764, 54.81532619951675], "calls": [0.0, 0.8111839834311357, 1.7777010700724887, 2.5198481187435275, 2.8822920262340355, 3.624439074905074, 4.780807732136693, 5.315843976527442, 5.8336209872281675, 6.748360372799447, 7.369692785640318, 8.353469105971694, 9.31998619261305, 10.217466344494303, 11.097687262685536, 11.61546427338626, 12.478425957887469, 13.09975837072834, 13.324128408698655, 14.187090093199862, 14.963755609250951, 15.930272695892302, 16.309975837072834, 17.311011391094237, 18.260269244045567, 19.140490162236798, 19.65826717293752, 20.210562651018293, 21.125302036589574, 21.884708318950636, 22.782188470831898, 23.938557128063515, 24.991370383154987, 25.81981360027615, 26.786330686917502, 27.338626164998274, 28.253365550569555, 29.806696582671727, 31.239212978943737, 32.430100103555404, 33.275802554366585, 34.15602347255782, 35.381429064549536, 37.12461166724197, 38.073869520193306, 38.97134967207456, 40.17949603037625, 41.00793924749741, 41.95719710044874, 43.389713496720745, 44.68415602347256, 45.875043148084224, 47.204004142216085, 48.343113565757676, 50.62133241284087, 52.071108042802905, 53.572661373835004, 54.556437694166384, 55.31584397652745, 57.05902657921988, 58.664135312392126, 59.69968933379358, 61.13220573006558, 62.927166033828094, 64.39420089748015, 66.25819813600276, 67.4318260269244, 69.08871246116672, 71.0735243355195, 72.16085605799103, 73.576113220573, 75.02588885053504, 76.75181221953746, 77.28684846392821, 78.63306869175008, 80.04832585433206, 81.15291681049361, 82.15395236451502, 82.68898860890576, 83.82809803244736, 84.89817052122885, 86.12357611322057, 86.95201933034173, 87.5043148084225, 88.83327580255437, 89.59268208491544, 90.30031066620641, 90.86986537797722, 91.88816016568865, 92.71660338280981, 93.51052813255092, 94.44252675181221, 95.65067311011391, 96.28926475664481, 97.04867103900587, 97.48015188125648, 98.17052122885744, 98.6365205384881, 99.03348291335865, 99.34414911977908, 100.0], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.32550914739386, 81.32550914739386, 81.2909906800138, 81.2909906800138, 81.27373144632378, 81.23921297894374, 81.20469451156369, 81.11839834311357, 81.0666206420435, 81.04936140835348, 80.9285467725233, 80.85950983776321, 80.84225060407319, 80.75595443562305, 80.68691750086296, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.39351052813255, 80.37625129444253, 80.30721435968243, 80.32447359337246, 80.18639972385226, 80.18639972385226, 80.13462202278218, 79.89299275112185, 79.82395581636176, 79.75491888160165, 79.59958577839144, 79.37521574042113, 79.23714187090093, 79.08180876769072, 78.94373489817053, 78.77114256127028, 78.68484639282016, 78.56403175698999, 78.49499482222988, 78.33966171901969, 78.30514325163963, 78.16706938211944, 77.94269934414912, 77.73558853986883, 77.56299620296859, 77.47670003451846, 77.28684846392821, 77.07973765964792, 76.82084915429755, 76.68277528477736, 76.61373835001726, 76.51018294787711, 76.21677597514672, 76.18225750776666, 75.99240593717639, 75.61270279599586, 75.44011045909562, 75.14670348636521, 74.90507421470487, 74.80151881256472, 74.5598895409044, 74.38729720400414, 74.12840869865379, 73.64515015533311, 73.36900241629272, 72.95478080773215, 72.48878149810149, 72.21263375906109, 71.72937521574042, 71.2806351397998, 70.72833966171902, 70.1415257162582, 69.52019330341733, 69.0024162927166, 68.570935450466, 67.93234380393511, 67.32827062478425, 66.87953054884363, 66.25819813600276, 65.70590265792198, 64.94649637556093, 64.32516396272005, 63.738350017259236, 62.97894373489817, 62.23679668622714, 61.615464273386266, 60.925094925785295, 60.11391094235417, 59.23369002416293, 58.47428374180187, 57.594062823610635, 56.748360372799446, 55.79910251984812, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 81.28753883327582, "exact_area100": 75.01296527694784}}, "old_normalized_endpoint_budget": 0}, {"seed": 2029, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.0, "validation_area100": 0.8634, "training_seconds": 3.226258024922572, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.014309233985841274, "operator": "ge"}, {"budget": 0.02, "threshold": -0.01661933958530426, "operator": "ge"}, {"budget": 0.03, "threshold": -0.018369795754551888, "operator": "ge"}, {"budget": 0.04, "threshold": -0.02152475155889988, "operator": "ge"}, {"budget": 0.05, "threshold": -0.025668704882264137, "operator": "ge"}, {"budget": 0.06, "threshold": -0.030534595251083374, "operator": "ge"}, {"budget": 0.07, "threshold": -0.03448747843503952, "operator": "ge"}, {"budget": 0.08, "threshold": -0.03853556513786316, "operator": "ge"}, {"budget": 0.09, "threshold": -0.04217861592769623, "operator": "ge"}, {"budget": 0.1, "threshold": -0.05311701074242592, "operator": "ge"}, {"budget": 0.11, "threshold": -0.06002523750066757, "operator": "ge"}, {"budget": 0.12, "threshold": -0.0638667643070221, "operator": "ge"}, {"budget": 0.13, "threshold": -0.07069898396730423, "operator": "ge"}, {"budget": 0.14, "threshold": -0.07431172579526901, "operator": "ge"}, {"budget": 0.15, "threshold": -0.080048106610775, "operator": "ge"}, {"budget": 0.16, "threshold": -0.08481432497501373, "operator": "ge"}, {"budget": 0.17, "threshold": -0.08996747434139252, "operator": "ge"}, {"budget": 0.18, "threshold": -0.09561602771282196, "operator": "ge"}, {"budget": 0.19, "threshold": -0.10050053894519806, "operator": "ge"}, {"budget": 0.2, "threshold": -0.10704144835472107, "operator": "ge"}, {"budget": 0.21, "threshold": -0.11964111775159836, "operator": "ge"}, {"budget": 0.22, "threshold": -0.12952950596809387, "operator": "ge"}, {"budget": 0.23, "threshold": -0.1367405354976654, "operator": "ge"}, {"budget": 0.24, "threshold": -0.14193564653396606, "operator": "ge"}, {"budget": 0.25, "threshold": -0.15267165005207062, "operator": "ge"}, {"budget": 0.26, "threshold": -0.15824103355407715, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1739869862794876, "operator": "ge"}, {"budget": 0.28, "threshold": -0.1841423660516739, "operator": "ge"}, {"budget": 0.29, "threshold": -0.19084779918193817, "operator": "ge"}, {"budget": 0.3, "threshold": -0.20524470508098602, "operator": "ge"}, {"budget": 0.31, "threshold": -0.2153952568769455, "operator": "ge"}, {"budget": 0.32, "threshold": -0.22961081564426422, "operator": "ge"}, {"budget": 0.33, "threshold": -0.23990996181964874, "operator": "ge"}, {"budget": 0.34, "threshold": -0.24921628832817078, "operator": "ge"}, {"budget": 0.35, "threshold": -0.2589516043663025, "operator": "ge"}, {"budget": 0.36, "threshold": -0.2708286941051483, "operator": "ge"}, {"budget": 0.37, "threshold": -0.2847394347190857, "operator": "ge"}, {"budget": 0.38, "threshold": -0.2987583875656128, "operator": "ge"}, {"budget": 0.39, "threshold": -0.304863840341568, "operator": "ge"}, {"budget": 0.4, "threshold": -0.3144229054450989, "operator": "ge"}, {"budget": 0.41, "threshold": -0.32378190755844116, "operator": "ge"}, {"budget": 0.42, "threshold": -0.33890700340270996, "operator": "ge"}, {"budget": 0.43, "threshold": -0.3486315608024597, "operator": "ge"}, {"budget": 0.44, "threshold": -0.36685168743133545, "operator": "ge"}, {"budget": 0.45, "threshold": -0.3790714740753174, "operator": "ge"}, {"budget": 0.46, "threshold": -0.38988661766052246, "operator": "ge"}, {"budget": 0.47, "threshold": -0.4066961407661438, "operator": "ge"}, {"budget": 0.48, "threshold": -0.4182493984699249, "operator": "ge"}, {"budget": 0.49, "threshold": -0.43080592155456543, "operator": "ge"}, {"budget": 0.5, "threshold": -0.440856009721756, "operator": "ge"}, {"budget": 0.51, "threshold": -0.46195363998413086, "operator": "ge"}, {"budget": 0.52, "threshold": -0.47530415654182434, "operator": "ge"}, {"budget": 0.53, "threshold": -0.4808334708213806, "operator": "ge"}, {"budget": 0.54, "threshold": -0.49808603525161743, "operator": "ge"}, {"budget": 0.55, "threshold": -0.5063815712928772, "operator": "ge"}, {"budget": 0.56, "threshold": -0.5202000141143799, "operator": "ge"}, {"budget": 0.57, "threshold": -0.5315780639648438, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5436853170394897, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5590388774871826, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5697216391563416, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5876728296279907, "operator": "ge"}, {"budget": 0.62, "threshold": -0.602412760257721, "operator": "ge"}, {"budget": 0.63, "threshold": -0.6152598857879639, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6243117451667786, "operator": "ge"}, {"budget": 0.65, "threshold": -0.645806610584259, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6579211950302124, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6692862510681152, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6904730200767517, "operator": "ge"}, {"budget": 0.69, "threshold": -0.7046011686325073, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7196836471557617, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7318490147590637, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7432858347892761, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7594826817512512, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7712365388870239, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7798190116882324, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7883363962173462, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7980760931968689, "operator": "ge"}, {"budget": 0.78, "threshold": -0.809689462184906, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8219385743141174, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8273773193359375, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8371906280517578, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8443652391433716, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8563387393951416, "operator": "ge"}, {"budget": 0.84, "threshold": -0.860862135887146, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8704327344894409, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8801860213279724, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8880859613418579, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8997895121574402, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9075936675071716, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9205921292304993, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9334502220153809, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9405697584152222, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9475224018096924, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9585538506507874, "operator": "ge"}, {"budget": 0.95, "threshold": -0.963855504989624, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9710896015167236, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9774948954582214, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9823516607284546, "operator": "ge"}, {"budget": 0.99, "threshold": -0.987107515335083, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.25, 98.16666666666667, 98.08333333333333, 98.08333333333333, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.5, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.16666666666667, 97.16666666666667, 97.0, 96.75, 96.33333333333334, 96.41666666666666, 96.16666666666667, 96.0, 95.83333333333334, 95.58333333333333, 95.41666666666667, 95.08333333333333, 94.83333333333334, 94.66666666666667, 94.25, 93.83333333333333, 93.66666666666667, 93.08333333333333, 92.58333333333333, 92.25, 91.83333333333333, 91.5, 91.16666666666666, 90.66666666666667, 90.16666666666666, 89.41666666666667, 88.75, 88.25, 87.75, 87.41666666666667, 86.83333333333334, 86.25, 85.58333333333333, 85.0, 84.25, 83.91666666666666, 83.08333333333333, 82.58333333333333, 81.83333333333334, 81.08333333333333, 80.41666666666667, 79.75, 79.16666666666667, 78.41666666666667, 77.66666666666666, 76.91666666666667, 76.16666666666667, 75.41666666666667, 74.58333333333333, 73.91666666666666, 73.0, 72.16666666666667, 71.50000000000001, 70.66666666666667, 69.83333333333334, 68.91666666666667, 67.91666666666667, 67.08333333333334, 66.16666666666666, 65.33333333333333, 64.5, 63.66666666666667, 62.66666666666667, 61.74999999999999, 60.916666666666664, 59.916666666666664, 58.916666666666664, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 4, 4, 4, 5, 6, 6, 8, 10, 11, 11, 13, 14, 15, 15, 17, 17, 19, 22, 27, 27, 30, 32, 34, 37, 39, 43, 46, 48, 53, 58, 60, 67, 73, 77, 82, 86, 90, 96, 102, 111, 119, 125, 131, 136, 143, 150, 158, 165, 174, 178, 188, 194, 203, 212, 220, 228, 235, 244, 253, 262, 271, 280, 290, 298, 309, 319, 327, 337, 347, 358, 370, 380, 391, 401, 411, 421, 433, 444, 454, 466, 478, 490, 502, 513, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.25, 98.16666666666667, 98.08333333333333, 98.08333333333333, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.5, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.16666666666667, 97.16666666666667, 97.0, 96.75, 96.33333333333334, 96.41666666666666, 96.16666666666667, 96.0, 95.83333333333334, 95.58333333333333, 95.41666666666667, 95.08333333333333, 94.83333333333334, 94.66666666666667, 94.25, 93.83333333333333, 93.66666666666667, 93.08333333333333, 92.58333333333333, 92.25, 91.83333333333333, 91.5, 91.16666666666666, 90.66666666666667, 90.16666666666666, 89.41666666666667, 88.75, 88.25, 87.75, 87.41666666666667, 86.83333333333334, 86.25, 85.58333333333333, 85.0, 84.25, 83.91666666666666, 83.08333333333333, 82.58333333333333, 81.83333333333334, 81.08333333333333, 80.41666666666667, 79.75, 79.16666666666667, 78.41666666666667, 77.66666666666666, 76.91666666666667, 76.16666666666667, 75.41666666666667, 74.58333333333333, 73.91666666666666, 73.0, 72.16666666666667, 71.41666666666666, 70.66666666666667, 69.83333333333334, 68.91666666666667, 67.91666666666667, 67.08333333333334, 66.16666666666666, 65.33333333333333, 64.5, 63.66666666666667, 62.66666666666667, 61.74999999999999, 60.916666666666664, 59.916666666666664, 58.916666666666664, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.33999999999999, "threshold_area100": 86.34083333333334, "threshold_area25": 98.34166666666667}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.32550914739386, 81.30824991370383, 81.2909906800138, 81.27373144632378, 81.27373144632378, 81.27373144632378, 81.25647221263375, 81.25647221263375, 81.23921297894374, 81.22195374525371, 81.11839834311357, 81.1356575768036, 81.11839834311357, 81.0666206420435, 81.01484294097342, 80.9975837072834, 80.9285467725233, 80.91128753883328, 80.84225060407319, 80.77321366931308, 80.77321366931308, 80.72143596824301, 80.70417673455299, 80.61788056610285, 80.53158439765275, 80.46254746289266, 80.41076976182258, 80.25543665861235, 80.2899551259924, 80.16914049016223, 80.10010355540214, 80.06558508802209, 79.97928891957199, 79.77217811529168, 79.66862271315154, 79.47877114256127, 79.32343803935106, 78.9955125992406, 78.75388332758025, 78.56403175698999, 78.20158784949949, 77.90818087676908, 77.63203313772868, 77.47670003451846, 77.30410769761824, 76.88988608905764, 76.68277528477736, 76.35484984466689, 75.7852951328961, 75.47462892647566, 74.95685191577495, 74.52537107352434, 74.2147048671039, 73.86952019330342, 73.1446323783224, 72.52329996548154, 72.09181912323092, 71.31515360717984, 70.62478425957887, 69.91715567828788, 69.29582326544703, 68.74352778736625, 67.98412150500518, 67.44908526061442, 67.01760441836383, 66.58612357611322, 65.9993096306524, 65.41249568519159, 64.68760787021056, 64.39420089748016, 63.9282015878495, 63.61753538142907, 62.90990680013808, 62.702795995857784, 62.23679668622714, 61.73627890921644, 61.339316534345876, 60.82153952364515, 60.40731791508457, 59.924059371763896, 59.09561615464274, 58.81946841560235, 58.40524680704176, 57.852951328961, 57.57680358992061, 56.8519157749396, 56.21332412840869, 55.712806351397994, 55.1432516396272, 54.81532619951675], "calls": [0.0, 1.0355540214014498, 1.725923369002416, 2.2091819123230927, 2.77873662409389, 3.486365205384881, 4.193993786675872, 4.694511563686572, 5.333103210217467, 5.712806351397997, 6.731101139109423, 7.680358992060753, 8.180876769071453, 8.940283051432516, 9.37176389368312, 10.11391094235416, 10.82153952364515, 11.322057300655851, 12.167759751467035, 12.685536762167759, 13.272350707628581, 14.791163272350708, 15.688643424231962, 16.344494304452883, 16.91404901622368, 17.949603037625128, 18.39834311356576, 19.882637210907834, 20.849154297549187, 21.52226441146013, 22.88574387297204, 23.731446323783224, 24.905074214704868, 25.854332067656195, 26.717293752157406, 27.666551605108737, 28.97825336555057, 30.600621332412842, 32.25750776665516, 33.03417328270625, 34.08698653779772, 34.898170521228856, 36.22713151536072, 37.33172247152227, 39.506385916465305, 40.714532274767, 41.905419399378665, 43.389713496720745, 44.64963755609251, 46.16845012081463, 47.53192958232654, 49.86192613047981, 51.605108733172244, 52.31273731446324, 54.38384535726614, 55.79910251984812, 57.90472903003106, 59.33724542630308, 60.700724887814985, 62.909906800138074, 64.0662754573697, 65.96479116327235, 67.55264066275457, 69.0024162927166, 69.86537797721782, 72.05730065585088, 73.52433551950294, 74.52537107352434, 76.30307214359682, 77.63203313772868, 79.13358646876078, 80.37625129444253, 81.32550914739386, 82.72350707628581, 83.62098722816707, 84.39765274421815, 85.08802209181913, 85.88194684156024, 86.83120469451157, 87.90127718329306, 88.28098032447359, 88.90231273731446, 89.4891266827753, 90.40386606834656, 90.74905074214705, 91.43942008974801, 92.07801173627891, 92.61304798066966, 93.33793579565068, 93.7866758715913, 94.4252675181222, 95.32274767000345, 95.66793234380393, 96.15119088712461, 96.77252329996548, 97.08318950638592, 97.91163272350708, 98.567483603728, 99.06800138073869, 99.67207455988955, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 4, 5, 5, 6, 6, 7, 9, 9, 10, 13, 15, 18, 20, 23, 27, 32, 39, 48, 51, 54, 63, 72, 81, 89, 96, 105, 109, 118, 127, 146, 160, 161, 176, 190, 206, 216, 224, 235, 244, 253, 263, 273, 277, 297, 304, 308, 317, 324, 332, 334, 339, 344, 346, 348, 349, 349, 350, 352, 352, 353, 357, 358, 360, 361, 361, 363, 364, 364, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 0, 0, 0, 1, 1, 2, 3, 3, 3, 4, 6, 7, 8, 9, 9, 9, 10, 10, 11, 13, 20, 21, 23, 26, 30, 31, 36, 39, 43, 48, 51, 56, 60, 67, 75, 83, 91, 107, 114, 124, 131, 142, 156, 177, 191, 209, 227, 250, 273, 293, 333, 364, 381, 405, 429, 469, 491, 518, 562, 589, 628, 663, 691, 715, 777, 820, 849, 903, 950, 999, 1037, 1074, 1123, 1156, 1183, 1209, 1243, 1278, 1322, 1339, 1367, 1389, 1431, 1445, 1473, 1502, 1527, 1558, 1582, 1611, 1659, 1675, 1699, 1731, 1747, 1789, 1826, 1855, 1888, 1907], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.25647221263375, 81.23921297894374, 81.18743527787366, 81.1356575768036, 81.15291681049361, 81.0666206420435, 80.9975837072834, 80.9975837072834, 80.91128753883328, 80.87676907145323, 80.7904729030031, 80.77321366931308, 80.7904729030031, 80.72143596824301, 80.66965826717293, 80.53158439765275, 80.4970659302727, 80.46254746289266, 80.4280289955126, 80.35899206075251, 80.22091819123231, 80.27269589230238, 80.16914049016223, 80.10010355540214, 80.08284432171212, 80.03106662064205, 79.91025198481188, 79.87573351743183, 79.80669658267173, 79.58232654470142, 79.4960303762513, 79.30617880566103, 79.18536416983086, 78.94373489817053, 78.77114256127028, 78.70210562651019, 78.44321712115982, 78.20158784949949, 78.20158784949949, 77.97721781152917, 77.77010700724888, 77.63203313772868, 77.47670003451846, 77.37314463237833, 77.28684846392821, 77.14877459440801, 76.94166379012772, 76.73455298584743, 76.45840524680705, 76.23403520883673, 76.07870210562652, 75.76803589920608, 75.49188816016569, 75.18122195374526, 74.9223334483949, 74.7842595788747, 74.38729720400414, 74.2147048671039, 73.80048325854332, 73.4207801173628, 73.17915084570245, 72.79944770452192, 72.19537452537108, 71.88470831895064, 71.47048671039006, 70.95270969968934, 70.4176734552986, 69.96893337935795, 69.46841560234725, 68.96789782533655, 68.39834311356576, 67.82878840179495, 67.24197445633415, 66.6379012771833, 65.91301346220229, 65.27442181567137, 64.63583016914049, 63.9282015878495, 63.25509147393856, 62.53020365895754, 61.822575077666556, 61.06316879530549, 60.25198481187435, 59.38902312737315, 58.54332067656196, 57.66309975837073, 56.76561960648947, 55.78184328615809, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.24287739366864, "threshold_area100": 74.04142216085607, "threshold_area25": 81.29789437348983}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.014309233985841274, "operator": "ge"}, {"budget": 0.02, "threshold": -0.01661933958530426, "operator": "ge"}, {"budget": 0.03, "threshold": -0.018369795754551888, "operator": "ge"}, {"budget": 0.04, "threshold": -0.02152475155889988, "operator": "ge"}, {"budget": 0.05, "threshold": -0.025668704882264137, "operator": "ge"}, {"budget": 0.06, "threshold": -0.030534595251083374, "operator": "ge"}, {"budget": 0.07, "threshold": -0.03448747843503952, "operator": "ge"}, {"budget": 0.08, "threshold": -0.03853556513786316, "operator": "ge"}, {"budget": 0.09, "threshold": -0.04217861592769623, "operator": "ge"}, {"budget": 0.1, "threshold": -0.05311701074242592, "operator": "ge"}, {"budget": 0.11, "threshold": -0.06002523750066757, "operator": "ge"}, {"budget": 0.12, "threshold": -0.0638667643070221, "operator": "ge"}, {"budget": 0.13, "threshold": -0.07069898396730423, "operator": "ge"}, {"budget": 0.14, "threshold": -0.07431172579526901, "operator": "ge"}, {"budget": 0.15, "threshold": -0.080048106610775, "operator": "ge"}, {"budget": 0.16, "threshold": -0.08481432497501373, "operator": "ge"}, {"budget": 0.17, "threshold": -0.08996747434139252, "operator": "ge"}, {"budget": 0.18, "threshold": -0.09561602771282196, "operator": "ge"}, {"budget": 0.19, "threshold": -0.10050053894519806, "operator": "ge"}, {"budget": 0.2, "threshold": -0.10704144835472107, "operator": "ge"}, {"budget": 0.21, "threshold": -0.11964111775159836, "operator": "ge"}, {"budget": 0.22, "threshold": -0.12952950596809387, "operator": "ge"}, {"budget": 0.23, "threshold": -0.1367405354976654, "operator": "ge"}, {"budget": 0.24, "threshold": -0.14193564653396606, "operator": "ge"}, {"budget": 0.25, "threshold": -0.15267165005207062, "operator": "ge"}, {"budget": 0.26, "threshold": -0.15824103355407715, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1739869862794876, "operator": "ge"}, {"budget": 0.28, "threshold": -0.1841423660516739, "operator": "ge"}, {"budget": 0.29, "threshold": -0.19084779918193817, "operator": "ge"}, {"budget": 0.3, "threshold": -0.20524470508098602, "operator": "ge"}, {"budget": 0.31, "threshold": -0.2153952568769455, "operator": "ge"}, {"budget": 0.32, "threshold": -0.22961081564426422, "operator": "ge"}, {"budget": 0.33, "threshold": -0.23990996181964874, "operator": "ge"}, {"budget": 0.34, "threshold": -0.24921628832817078, "operator": "ge"}, {"budget": 0.35, "threshold": -0.2589516043663025, "operator": "ge"}, {"budget": 0.36, "threshold": -0.2708286941051483, "operator": "ge"}, {"budget": 0.37, "threshold": -0.2847394347190857, "operator": "ge"}, {"budget": 0.38, "threshold": -0.2987583875656128, "operator": "ge"}, {"budget": 0.39, "threshold": -0.304863840341568, "operator": "ge"}, {"budget": 0.4, "threshold": -0.3144229054450989, "operator": "ge"}, {"budget": 0.41, "threshold": -0.32378190755844116, "operator": "ge"}, {"budget": 0.42, "threshold": -0.33890700340270996, "operator": "ge"}, {"budget": 0.43, "threshold": -0.3486315608024597, "operator": "ge"}, {"budget": 0.44, "threshold": -0.36685168743133545, "operator": "ge"}, {"budget": 0.45, "threshold": -0.3790714740753174, "operator": "ge"}, {"budget": 0.46, "threshold": -0.38988661766052246, "operator": "ge"}, {"budget": 0.47, "threshold": -0.4066961407661438, "operator": "ge"}, {"budget": 0.48, "threshold": -0.4182493984699249, "operator": "ge"}, {"budget": 0.49, "threshold": -0.43080592155456543, "operator": "ge"}, {"budget": 0.5, "threshold": -0.440856009721756, "operator": "ge"}, {"budget": 0.51, "threshold": -0.46195363998413086, "operator": "ge"}, {"budget": 0.52, "threshold": -0.47530415654182434, "operator": "ge"}, {"budget": 0.53, "threshold": -0.4808334708213806, "operator": "ge"}, {"budget": 0.54, "threshold": -0.49808603525161743, "operator": "ge"}, {"budget": 0.55, "threshold": -0.5063815712928772, "operator": "ge"}, {"budget": 0.56, "threshold": -0.5202000141143799, "operator": "ge"}, {"budget": 0.57, "threshold": -0.5315780639648438, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5436853170394897, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5590388774871826, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5697216391563416, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5876728296279907, "operator": "ge"}, {"budget": 0.62, "threshold": -0.602412760257721, "operator": "ge"}, {"budget": 0.63, "threshold": -0.6152598857879639, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6243117451667786, "operator": "ge"}, {"budget": 0.65, "threshold": -0.645806610584259, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6579211950302124, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6692862510681152, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6904730200767517, "operator": "ge"}, {"budget": 0.69, "threshold": -0.7046011686325073, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7196836471557617, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7318490147590637, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7432858347892761, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7594826817512512, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7712365388870239, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7798190116882324, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7883363962173462, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7980760931968689, "operator": "ge"}, {"budget": 0.78, "threshold": -0.809689462184906, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8219385743141174, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8273773193359375, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8371906280517578, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8443652391433716, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8563387393951416, "operator": "ge"}, {"budget": 0.84, "threshold": -0.860862135887146, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8704327344894409, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8801860213279724, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8880859613418579, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8997895121574402, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9075936675071716, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9205921292304993, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9334502220153809, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9405697584152222, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9475224018096924, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9585538506507874, "operator": "ge"}, {"budget": 0.95, "threshold": -0.963855504989624, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9710896015167236, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9774948954582214, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9823516607284546, "operator": "ge"}, {"budget": 0.99, "threshold": -0.987107515335083, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.25, 98.16666666666667, 98.08333333333333, 98.08333333333333, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.5, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.16666666666667, 97.16666666666667, 97.0, 96.75, 96.33333333333334, 96.41666666666666, 96.16666666666667, 96.0, 95.83333333333334, 95.58333333333333, 95.41666666666667, 95.08333333333333, 94.83333333333334, 94.66666666666667, 94.25, 93.83333333333333, 93.66666666666667, 93.08333333333333, 92.58333333333333, 92.25, 91.83333333333333, 91.5, 91.16666666666666, 90.66666666666667, 90.16666666666666, 89.41666666666667, 88.75, 88.25, 87.75, 87.41666666666667, 86.83333333333334, 86.25, 85.58333333333333, 85.0, 84.25, 83.91666666666666, 83.08333333333333, 82.58333333333333, 81.83333333333334, 81.08333333333333, 80.41666666666667, 79.75, 79.16666666666667, 78.41666666666667, 77.66666666666666, 76.91666666666667, 76.16666666666667, 75.41666666666667, 74.58333333333333, 73.91666666666666, 73.0, 72.16666666666667, 71.50000000000001, 70.66666666666667, 69.83333333333334, 68.91666666666667, 67.91666666666667, 67.08333333333334, 66.16666666666666, 65.33333333333333, 64.5, 63.66666666666667, 62.66666666666667, 61.74999999999999, 60.916666666666664, 59.916666666666664, 58.916666666666664, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 4, 4, 4, 5, 6, 6, 8, 10, 11, 11, 13, 14, 15, 15, 17, 17, 19, 22, 27, 27, 30, 32, 34, 37, 39, 43, 46, 48, 53, 58, 60, 67, 73, 77, 82, 86, 90, 96, 102, 111, 119, 125, 131, 136, 143, 150, 158, 165, 174, 178, 188, 194, 203, 212, 220, 228, 235, 244, 253, 262, 271, 280, 290, 298, 309, 319, 327, 337, 347, 358, 370, 380, 391, 401, 411, 421, 433, 444, 454, 466, 478, 490, 502, 513, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.25, 98.16666666666667, 98.08333333333333, 98.08333333333333, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.5, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.16666666666667, 97.16666666666667, 97.0, 96.75, 96.33333333333334, 96.41666666666666, 96.16666666666667, 96.0, 95.83333333333334, 95.58333333333333, 95.41666666666667, 95.08333333333333, 94.83333333333334, 94.66666666666667, 94.25, 93.83333333333333, 93.66666666666667, 93.08333333333333, 92.58333333333333, 92.25, 91.83333333333333, 91.5, 91.16666666666666, 90.66666666666667, 90.16666666666666, 89.41666666666667, 88.75, 88.25, 87.75, 87.41666666666667, 86.83333333333334, 86.25, 85.58333333333333, 85.0, 84.25, 83.91666666666666, 83.08333333333333, 82.58333333333333, 81.83333333333334, 81.08333333333333, 80.41666666666667, 79.75, 79.16666666666667, 78.41666666666667, 77.66666666666666, 76.91666666666667, 76.16666666666667, 75.41666666666667, 74.58333333333333, 73.91666666666666, 73.0, 72.16666666666667, 71.41666666666666, 70.66666666666667, 69.83333333333334, 68.91666666666667, 67.91666666666667, 67.08333333333334, 66.16666666666666, 65.33333333333333, 64.5, 63.66666666666667, 62.66666666666667, 61.74999999999999, 60.916666666666664, 59.916666666666664, 58.916666666666664, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.33999999999999, "threshold_area100": 86.34083333333334, "threshold_area25": 98.34166666666667}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.32550914739386, 81.30824991370383, 81.2909906800138, 81.27373144632378, 81.27373144632378, 81.27373144632378, 81.25647221263375, 81.25647221263375, 81.23921297894374, 81.22195374525371, 81.11839834311357, 81.1356575768036, 81.11839834311357, 81.0666206420435, 81.01484294097342, 80.9975837072834, 80.9285467725233, 80.91128753883328, 80.84225060407319, 80.77321366931308, 80.77321366931308, 80.72143596824301, 80.70417673455299, 80.61788056610285, 80.53158439765275, 80.46254746289266, 80.41076976182258, 80.25543665861235, 80.2899551259924, 80.16914049016223, 80.10010355540214, 80.06558508802209, 79.97928891957199, 79.77217811529168, 79.66862271315154, 79.47877114256127, 79.32343803935106, 78.9955125992406, 78.75388332758025, 78.56403175698999, 78.20158784949949, 77.90818087676908, 77.63203313772868, 77.47670003451846, 77.30410769761824, 76.88988608905764, 76.68277528477736, 76.35484984466689, 75.7852951328961, 75.47462892647566, 74.95685191577495, 74.52537107352434, 74.2147048671039, 73.86952019330342, 73.1446323783224, 72.52329996548154, 72.09181912323092, 71.31515360717984, 70.62478425957887, 69.91715567828788, 69.29582326544703, 68.74352778736625, 67.98412150500518, 67.44908526061442, 67.01760441836383, 66.58612357611322, 65.9993096306524, 65.41249568519159, 64.68760787021056, 64.39420089748016, 63.9282015878495, 63.61753538142907, 62.90990680013808, 62.702795995857784, 62.23679668622714, 61.73627890921644, 61.339316534345876, 60.82153952364515, 60.40731791508457, 59.924059371763896, 59.09561615464274, 58.81946841560235, 58.40524680704176, 57.852951328961, 57.57680358992061, 56.8519157749396, 56.21332412840869, 55.712806351397994, 55.1432516396272, 54.81532619951675], "calls": [0.0, 1.0355540214014498, 1.725923369002416, 2.2091819123230927, 2.77873662409389, 3.486365205384881, 4.193993786675872, 4.694511563686572, 5.333103210217467, 5.712806351397997, 6.731101139109423, 7.680358992060753, 8.180876769071453, 8.940283051432516, 9.37176389368312, 10.11391094235416, 10.82153952364515, 11.322057300655851, 12.167759751467035, 12.685536762167759, 13.272350707628581, 14.791163272350708, 15.688643424231962, 16.344494304452883, 16.91404901622368, 17.949603037625128, 18.39834311356576, 19.882637210907834, 20.849154297549187, 21.52226441146013, 22.88574387297204, 23.731446323783224, 24.905074214704868, 25.854332067656195, 26.717293752157406, 27.666551605108737, 28.97825336555057, 30.600621332412842, 32.25750776665516, 33.03417328270625, 34.08698653779772, 34.898170521228856, 36.22713151536072, 37.33172247152227, 39.506385916465305, 40.714532274767, 41.905419399378665, 43.389713496720745, 44.64963755609251, 46.16845012081463, 47.53192958232654, 49.86192613047981, 51.605108733172244, 52.31273731446324, 54.38384535726614, 55.79910251984812, 57.90472903003106, 59.33724542630308, 60.700724887814985, 62.909906800138074, 64.0662754573697, 65.96479116327235, 67.55264066275457, 69.0024162927166, 69.86537797721782, 72.05730065585088, 73.52433551950294, 74.52537107352434, 76.30307214359682, 77.63203313772868, 79.13358646876078, 80.37625129444253, 81.32550914739386, 82.72350707628581, 83.62098722816707, 84.39765274421815, 85.08802209181913, 85.88194684156024, 86.83120469451157, 87.90127718329306, 88.28098032447359, 88.90231273731446, 89.4891266827753, 90.40386606834656, 90.74905074214705, 91.43942008974801, 92.07801173627891, 92.61304798066966, 93.33793579565068, 93.7866758715913, 94.4252675181222, 95.32274767000345, 95.66793234380393, 96.15119088712461, 96.77252329996548, 97.08318950638592, 97.91163272350708, 98.567483603728, 99.06800138073869, 99.67207455988955, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 4, 5, 5, 6, 6, 7, 9, 9, 10, 13, 15, 18, 20, 23, 27, 32, 39, 48, 51, 54, 63, 72, 81, 89, 96, 105, 109, 118, 127, 146, 160, 161, 176, 190, 206, 216, 224, 235, 244, 253, 263, 273, 277, 297, 304, 308, 317, 324, 332, 334, 339, 344, 346, 348, 349, 349, 350, 352, 352, 353, 357, 358, 360, 361, 361, 363, 364, 364, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 0, 0, 0, 1, 1, 2, 3, 3, 3, 4, 6, 7, 8, 9, 9, 9, 10, 10, 11, 13, 20, 21, 23, 26, 30, 31, 36, 39, 43, 48, 51, 56, 60, 67, 75, 83, 91, 107, 114, 124, 131, 142, 156, 177, 191, 209, 227, 250, 273, 293, 333, 364, 381, 405, 429, 469, 491, 518, 562, 589, 628, 663, 691, 715, 777, 820, 849, 903, 950, 999, 1037, 1074, 1123, 1156, 1183, 1209, 1243, 1278, 1322, 1339, 1367, 1389, 1431, 1445, 1473, 1502, 1527, 1558, 1582, 1611, 1659, 1675, 1699, 1731, 1747, 1789, 1826, 1855, 1888, 1907], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.25647221263375, 81.23921297894374, 81.18743527787366, 81.1356575768036, 81.15291681049361, 81.0666206420435, 80.9975837072834, 80.9975837072834, 80.91128753883328, 80.87676907145323, 80.7904729030031, 80.77321366931308, 80.7904729030031, 80.72143596824301, 80.66965826717293, 80.53158439765275, 80.4970659302727, 80.46254746289266, 80.4280289955126, 80.35899206075251, 80.22091819123231, 80.27269589230238, 80.16914049016223, 80.10010355540214, 80.08284432171212, 80.03106662064205, 79.91025198481188, 79.87573351743183, 79.80669658267173, 79.58232654470142, 79.4960303762513, 79.30617880566103, 79.18536416983086, 78.94373489817053, 78.77114256127028, 78.70210562651019, 78.44321712115982, 78.20158784949949, 78.20158784949949, 77.97721781152917, 77.77010700724888, 77.63203313772868, 77.47670003451846, 77.37314463237833, 77.28684846392821, 77.14877459440801, 76.94166379012772, 76.73455298584743, 76.45840524680705, 76.23403520883673, 76.07870210562652, 75.76803589920608, 75.49188816016569, 75.18122195374526, 74.9223334483949, 74.7842595788747, 74.38729720400414, 74.2147048671039, 73.80048325854332, 73.4207801173628, 73.17915084570245, 72.79944770452192, 72.19537452537108, 71.88470831895064, 71.47048671039006, 70.95270969968934, 70.4176734552986, 69.96893337935795, 69.46841560234725, 68.96789782533655, 68.39834311356576, 67.82878840179495, 67.24197445633415, 66.6379012771833, 65.91301346220229, 65.27442181567137, 64.63583016914049, 63.9282015878495, 63.25509147393856, 62.53020365895754, 61.822575077666556, 61.06316879530549, 60.25198481187435, 59.38902312737315, 58.54332067656196, 57.66309975837073, 56.76561960648947, 55.78184328615809, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.24287739366864, "threshold_area100": 74.04142216085607, "threshold_area25": 81.29789437348983}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.7343597412109375, "operator": "ge"}, {"budget": 0.02, "threshold": -3.1413543224334717, "operator": "ge"}, {"budget": 0.03, "threshold": -3.5990641117095947, "operator": "ge"}, {"budget": 0.04, "threshold": -4.090095043182373, "operator": "ge"}, {"budget": 0.05, "threshold": -4.501915454864502, "operator": "ge"}, {"budget": 0.06, "threshold": -4.73119592666626, "operator": "ge"}, {"budget": 0.07, "threshold": -4.939070701599121, "operator": "ge"}, {"budget": 0.08, "threshold": -5.315733432769775, "operator": "ge"}, {"budget": 0.09, "threshold": -5.477572441101074, "operator": "ge"}, {"budget": 0.1, "threshold": -5.683679580688477, "operator": "ge"}, {"budget": 0.11, "threshold": -5.822273254394531, "operator": "ge"}, {"budget": 0.12, "threshold": -6.061275005340576, "operator": "ge"}, {"budget": 0.13, "threshold": -6.2423601150512695, "operator": "ge"}, {"budget": 0.14, "threshold": -6.559438705444336, "operator": "ge"}, {"budget": 0.15, "threshold": -6.89692497253418, "operator": "ge"}, {"budget": 0.16, "threshold": -7.1386213302612305, "operator": "ge"}, {"budget": 0.17, "threshold": -7.375921726226807, "operator": "ge"}, {"budget": 0.18, "threshold": -7.644311904907227, "operator": "ge"}, {"budget": 0.19, "threshold": -7.9135589599609375, "operator": "ge"}, {"budget": 0.2, "threshold": -8.085809707641602, "operator": "ge"}, {"budget": 0.21, "threshold": -8.273558616638184, "operator": "ge"}, {"budget": 0.22, "threshold": -8.44476318359375, "operator": "ge"}, {"budget": 0.23, "threshold": -8.662948608398438, "operator": "ge"}, {"budget": 0.24, "threshold": -8.783233642578125, "operator": "ge"}, {"budget": 0.25, "threshold": -8.914158821105957, "operator": "ge"}, {"budget": 0.26, "threshold": -9.032205581665039, "operator": "ge"}, {"budget": 0.27, "threshold": -9.155718803405762, "operator": "ge"}, {"budget": 0.28, "threshold": -9.268479347229004, "operator": "ge"}, {"budget": 0.29, "threshold": -9.316840171813965, "operator": "ge"}, {"budget": 0.3, "threshold": -9.384977340698242, "operator": "ge"}, {"budget": 0.31, "threshold": -9.447883605957031, "operator": "ge"}, {"budget": 0.32, "threshold": -9.5086669921875, "operator": "ge"}, {"budget": 0.33, "threshold": -9.562894821166992, "operator": "ge"}, {"budget": 0.34, "threshold": -9.62121295928955, "operator": "ge"}, {"budget": 0.35, "threshold": -9.688863754272461, "operator": "ge"}, {"budget": 0.36, "threshold": -9.73497200012207, "operator": "ge"}, {"budget": 0.37, "threshold": -9.794012069702148, "operator": "ge"}, {"budget": 0.38, "threshold": -9.843786239624023, "operator": "ge"}, {"budget": 0.39, "threshold": -9.909433364868164, "operator": "ge"}, {"budget": 0.4, "threshold": -9.950151443481445, "operator": "ge"}, {"budget": 0.41, "threshold": -9.98684310913086, "operator": "ge"}, {"budget": 0.42, "threshold": -10.042197227478027, "operator": "ge"}, {"budget": 0.43, "threshold": -10.064464569091797, "operator": "ge"}, {"budget": 0.44, "threshold": -10.089090347290039, "operator": "ge"}, {"budget": 0.45, "threshold": -10.13760757446289, "operator": "ge"}, {"budget": 0.46, "threshold": -10.172731399536133, "operator": "ge"}, {"budget": 0.47, "threshold": -10.202452659606934, "operator": "ge"}, {"budget": 0.48, "threshold": -10.230228424072266, "operator": "ge"}, {"budget": 0.49, "threshold": -10.277941703796387, "operator": "ge"}, {"budget": 0.5, "threshold": -10.315431594848633, "operator": "ge"}, {"budget": 0.51, "threshold": -10.366375923156738, "operator": "ge"}, {"budget": 0.52, "threshold": -10.40521240234375, "operator": "ge"}, {"budget": 0.53, "threshold": -10.458863258361816, "operator": "ge"}, {"budget": 0.54, "threshold": -10.496039390563965, "operator": "ge"}, {"budget": 0.55, "threshold": -10.534782409667969, "operator": "ge"}, {"budget": 0.56, "threshold": -10.565561294555664, "operator": "ge"}, {"budget": 0.57, "threshold": -10.585551261901855, "operator": "ge"}, {"budget": 0.58, "threshold": -10.61404800415039, "operator": "ge"}, {"budget": 0.59, "threshold": -10.643558502197266, "operator": "ge"}, {"budget": 0.6, "threshold": -10.67953872680664, "operator": "ge"}, {"budget": 0.61, "threshold": -10.715798377990723, "operator": "ge"}, {"budget": 0.62, "threshold": -10.738265037536621, "operator": "ge"}, {"budget": 0.63, "threshold": -10.769554138183594, "operator": "ge"}, {"budget": 0.64, "threshold": -10.807804107666016, "operator": "ge"}, {"budget": 0.65, "threshold": -10.849275588989258, "operator": "ge"}, {"budget": 0.66, "threshold": -10.886139869689941, "operator": "ge"}, {"budget": 0.67, "threshold": -10.916687965393066, "operator": "ge"}, {"budget": 0.68, "threshold": -10.955460548400879, "operator": "ge"}, {"budget": 0.69, "threshold": -11.00943660736084, "operator": "ge"}, {"budget": 0.7, "threshold": -11.056135177612305, "operator": "ge"}, {"budget": 0.71, "threshold": -11.0805082321167, "operator": "ge"}, {"budget": 0.72, "threshold": -11.102584838867188, "operator": "ge"}, {"budget": 0.73, "threshold": -11.125028610229492, "operator": "ge"}, {"budget": 0.74, "threshold": -11.16970157623291, "operator": "ge"}, {"budget": 0.75, "threshold": -11.204654693603516, "operator": "ge"}, {"budget": 0.76, "threshold": -11.252994537353516, "operator": "ge"}, {"budget": 0.77, "threshold": -11.294830322265625, "operator": "ge"}, {"budget": 0.78, "threshold": -11.34230899810791, "operator": "ge"}, {"budget": 0.79, "threshold": -11.375510215759277, "operator": "ge"}, {"budget": 0.8, "threshold": -11.460941314697266, "operator": "ge"}, {"budget": 0.81, "threshold": -11.503398895263672, "operator": "ge"}, {"budget": 0.82, "threshold": -11.534494400024414, "operator": "ge"}, {"budget": 0.83, "threshold": -11.632389068603516, "operator": "ge"}, {"budget": 0.84, "threshold": -11.684772491455078, "operator": "ge"}, {"budget": 0.85, "threshold": -11.752496719360352, "operator": "ge"}, {"budget": 0.86, "threshold": -11.810073852539062, "operator": "ge"}, {"budget": 0.87, "threshold": -11.892127990722656, "operator": "ge"}, {"budget": 0.88, "threshold": -11.985479354858398, "operator": "ge"}, {"budget": 0.89, "threshold": -12.0828857421875, "operator": "ge"}, {"budget": 0.9, "threshold": -12.144044876098633, "operator": "ge"}, {"budget": 0.91, "threshold": -12.232492446899414, "operator": "ge"}, {"budget": 0.92, "threshold": -12.336004257202148, "operator": "ge"}, {"budget": 0.93, "threshold": -12.43529224395752, "operator": "ge"}, {"budget": 0.94, "threshold": -12.538634300231934, "operator": "ge"}, {"budget": 0.95, "threshold": -12.647729873657227, "operator": "ge"}, {"budget": 0.96, "threshold": -12.779125213623047, "operator": "ge"}, {"budget": 0.97, "threshold": -12.890535354614258, "operator": "ge"}, {"budget": 0.98, "threshold": -13.05450439453125, "operator": "ge"}, {"budget": 0.99, "threshold": -13.39100456237793, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.0, 97.33333333333334, 96.66666666666667, 96.33333333333334, 95.66666666666667, 95.25, 94.66666666666667, 94.16666666666667, 93.83333333333333, 93.25, 92.66666666666666, 92.0, 91.58333333333333, 90.83333333333333, 90.16666666666666, 89.33333333333333, 88.75, 88.16666666666667, 87.41666666666667, 86.91666666666666, 86.25, 85.66666666666667, 85.5, 85.25, 84.91666666666666, 84.66666666666667, 84.25, 83.91666666666666, 83.66666666666667, 83.16666666666667, 82.75, 82.5, 82.16666666666667, 81.83333333333334, 81.41666666666667, 81.25, 80.91666666666667, 80.75, 80.08333333333333, 79.58333333333334, 79.25, 78.83333333333333, 78.5, 78.0, 77.91666666666667, 77.5, 77.0, 76.75, 76.41666666666667, 76.08333333333334, 75.83333333333333, 75.5, 75.08333333333333, 74.75, 74.33333333333334, 74.08333333333333, 73.66666666666667, 73.33333333333334, 72.91666666666667, 72.41666666666666, 71.91666666666667, 71.58333333333333, 71.25, 70.91666666666667, 70.66666666666667, 70.33333333333334, 70.08333333333334, 69.66666666666667, 69.58333333333333, 68.91666666666667, 68.41666666666666, 68.08333333333334, 67.75, 67.33333333333333, 66.83333333333333, 66.5, 66.0, 65.66666666666667, 65.08333333333334, 64.83333333333333, 64.58333333333333, 64.25, 63.66666666666667, 63.5, 63.33333333333333, 62.83333333333334, 62.66666666666667, 62.33333333333333, 61.74999999999999, 61.33333333333333, 60.74999999999999, 60.25, 59.583333333333336, 58.833333333333336, 58.08333333333333, 57.49999999999999, 57.166666666666664, 56.41666666666667, 55.666666666666664, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 7, 15, 23, 27, 35, 40, 47, 53, 58, 66, 73, 81, 86, 95, 103, 113, 120, 127, 136, 142, 150, 157, 159, 162, 166, 169, 174, 178, 181, 187, 192, 195, 199, 203, 208, 210, 214, 216, 224, 230, 234, 239, 243, 249, 250, 255, 261, 264, 268, 272, 275, 279, 284, 288, 293, 296, 301, 305, 310, 316, 322, 326, 330, 334, 337, 341, 344, 349, 350, 358, 364, 368, 372, 377, 383, 387, 393, 397, 404, 407, 410, 414, 421, 423, 425, 431, 433, 437, 444, 449, 456, 462, 470, 479, 488, 495, 499, 508, 517, 524], "exact_score": [98.58333333333333, 98.0, 97.33333333333334, 96.66666666666667, 96.33333333333334, 95.66666666666667, 95.25, 94.66666666666667, 94.16666666666667, 93.83333333333333, 93.25, 92.66666666666666, 92.0, 91.58333333333333, 90.83333333333333, 90.16666666666666, 89.33333333333333, 88.75, 88.16666666666667, 87.41666666666667, 86.91666666666666, 86.25, 85.66666666666667, 85.5, 85.25, 84.91666666666666, 84.66666666666667, 84.25, 83.91666666666666, 83.66666666666667, 83.16666666666667, 82.75, 82.5, 82.16666666666667, 81.83333333333334, 81.41666666666667, 81.25, 80.91666666666667, 80.75, 80.08333333333333, 79.58333333333334, 79.25, 78.83333333333333, 78.5, 78.0, 77.91666666666667, 77.5, 77.0, 76.75, 76.41666666666667, 76.08333333333334, 75.83333333333333, 75.5, 75.08333333333333, 74.75, 74.33333333333334, 74.08333333333333, 73.66666666666667, 73.33333333333334, 72.91666666666667, 72.41666666666666, 71.91666666666667, 71.58333333333333, 71.25, 70.91666666666667, 70.66666666666667, 70.33333333333334, 70.08333333333334, 69.66666666666667, 69.58333333333333, 68.91666666666667, 68.41666666666666, 68.08333333333334, 67.75, 67.33333333333333, 66.83333333333333, 66.5, 66.0, 65.66666666666667, 65.08333333333334, 64.83333333333333, 64.58333333333333, 64.25, 63.66666666666667, 63.5, 63.33333333333333, 62.83333333333334, 62.66666666666667, 62.33333333333333, 61.74999999999999, 61.33333333333333, 60.74999999999999, 60.25, 59.583333333333336, 58.833333333333336, 58.08333333333333, 57.49999999999999, 57.166666666666664, 56.41666666666667, 55.666666666666664, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 76.34416666666665, "threshold_area100": 76.34416666666665, "threshold_area25": 91.55}, "test": {"score": [81.429064549534, 81.27373144632378, 80.72143596824301, 80.08284432171212, 79.27166033828098, 78.23610631687953, 77.37314463237833, 76.64825681739731, 75.2157404211253, 74.74974111149464, 74.2147048671039, 73.7832240248533, 73.1446323783224, 72.79944770452192, 71.88470831895064, 71.33241284086986, 70.6420434932689, 70.29685881946843, 69.81360027614774, 69.55471177079737, 69.31308249913704, 69.03693476009666, 68.7780462547463, 68.39834311356576, 68.24301001035555, 68.0013807386952, 67.7079737659648, 67.43182602692441, 67.12115982050398, 66.94856748360372, 66.74145667932343, 66.5688643424232, 66.41353123921299, 66.30997583707283, 66.1373835001726, 65.89575422851226, 65.67138419054194, 65.5678287884018, 65.4987918536417, 65.23990334829134, 65.08457024508112, 64.96375560925095, 64.73938557128064, 64.5667932343804, 64.51501553331033, 64.30790472903003, 64.16983085950984, 64.13531239212979, 64.11805315843976, 63.9282015878495, 63.80738695201933, 63.53123921297894, 63.35864687607871, 63.06523990334829, 62.87538833275803, 62.63375906109769, 62.4956851915775, 62.46116672419745, 62.3576113220573, 62.23679668622714, 62.09872281670694, 61.977908180876774, 61.87435277873663, 61.66724197445633, 61.4946496375561, 61.37383500172593, 61.201242664825685, 61.09768726268554, 60.95961339316535, 60.68346565412496, 60.493614083534695, 60.37279944770453, 60.303762512944424, 60.23472557818433, 60.079392474974114, 59.94131860545392, 59.75146703486365, 59.630652399033494, 59.423541594753196, 59.285467725233, 58.97480151881257, 58.81946841560235, 58.715913013462206, 58.31895063859165, 58.06006213324129, 57.95650673110114, 57.818432861580945, 57.49050742147048, 57.23161891612012, 56.93821194338972, 56.69658267172938, 56.48947186744908, 56.10976872626856, 55.88539868829825, 55.79910251984812, 55.64376941663791, 55.402140144977565, 55.24680704176734, 55.1432516396272, 54.95340006903693, 54.81532619951675], "calls": [0.0, 3.8833275802554366, 6.368657231618916, 11.218501898515706, 17.552640662754573, 24.024853296513633, 27.562996202968588, 30.84225060407318, 36.19261304798067, 37.97031411805316, 40.075940628236104, 41.23230928546773, 42.95823265447014, 44.321712115982045, 46.565412495685194, 48.15326199516741, 49.87918536416983, 51.08733172247152, 52.36451501553331, 53.572661373835004, 54.36658612357611, 55.38488091128754, 56.2133241284087, 57.455988954090444, 58.077321366931315, 58.8539868829824, 59.7859855022437, 60.71798412150501, 61.54642733862617, 61.96064894718675, 62.53020365895754, 63.15153607179841, 63.66931308249914, 64.25612702795996, 64.77390403866067, 65.5678287884018, 66.24093890231273, 66.91404901622367, 67.39730755954436, 68.03589920607524, 68.62271315153608, 69.0024162927166, 69.60648947186745, 70.003451846738, 70.26234035208837, 71.0044874007594, 71.62581981360027, 72.04004142216085, 72.45426303072144, 73.24818778046254, 73.67966862271315, 74.43907490507421, 74.97411114946496, 75.94062823610632, 76.49292371418709, 77.18329306178806, 77.77010700724888, 78.04625474628926, 78.33966171901967, 78.80566102865033, 79.46151190887124, 80.13462202278218, 80.44528822920263, 80.91128753883328, 81.4981014842941, 82.17121159820505, 82.68898860890576, 83.15498791853642, 83.87987573351742, 84.67380048325855, 85.4332067656196, 85.72661373835003, 85.98550224370038, 86.38246461857094, 86.98653779772178, 87.41801863997239, 88.0048325854332, 88.62616499827408, 89.26475664480496, 89.69623748705557, 90.69727304107698, 91.09423541594754, 91.37038315498792, 92.44045564376941, 93.04452882292026, 93.63134276838109, 94.09734207801174, 94.87400759406283, 95.47808077321366, 96.09941318605453, 96.47911632723508, 96.91059716948568, 97.51467034863653, 98.01518812564723, 98.30859509837762, 98.60200207110805, 98.98170521228857, 99.24059371763893, 99.48222298929927, 99.81014842940974, 100.0], "rescued": [0, 54, 76, 123, 187, 238, 256, 272, 303, 311, 323, 327, 334, 341, 346, 346, 348, 351, 353, 355, 356, 356, 356, 356, 356, 356, 356, 356, 356, 356, 356, 356, 356, 356, 356, 356, 357, 357, 357, 357, 358, 358, 358, 358, 358, 358, 358, 359, 359, 359, 359, 359, 359, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 362, 362, 362, 362, 362, 362, 363, 363, 363, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 63, 117, 201, 312, 423, 491, 549, 663, 698, 741, 770, 814, 841, 899, 931, 973, 996, 1026, 1043, 1058, 1074, 1089, 1111, 1120, 1134, 1151, 1167, 1185, 1195, 1207, 1217, 1226, 1232, 1242, 1256, 1270, 1276, 1280, 1295, 1305, 1312, 1325, 1335, 1338, 1350, 1358, 1361, 1362, 1373, 1380, 1396, 1406, 1424, 1435, 1449, 1457, 1459, 1465, 1472, 1481, 1488, 1494, 1506, 1516, 1523, 1533, 1539, 1547, 1563, 1574, 1581, 1585, 1589, 1599, 1607, 1618, 1625, 1637, 1645, 1664, 1673, 1679, 1703, 1718, 1724, 1732, 1751, 1766, 1783, 1797, 1809, 1831, 1845, 1850, 1859, 1873, 1882, 1888, 1899, 1907], "exact_score": [81.429064549534, 81.429064549534, 81.46358301691406, 81.429064549534, 81.30824991370383, 81.11839834311357, 80.9285467725233, 80.66965826717293, 80.41076976182258, 80.25543665861235, 80.20365895754229, 80.10010355540214, 79.8584742837418, 79.87573351743183, 79.72040041422161, 79.51328960994132, 79.40973420780118, 79.37521574042113, 79.15084570245081, 79.06454953400069, 78.84017949603039, 78.87469796341043, 78.61580945806006, 78.44321712115982, 78.25336555056955, 78.02899551259924, 77.82188470831896, 77.51121850189851, 77.23507076285813, 76.97618225750776, 76.8553676216776, 76.66551605108734, 76.47566448049706, 76.23403520883673, 75.9233690024163, 75.59544356230583, 75.33655505695546, 74.97411114946496, 74.74974111149464, 74.47359337245426, 74.24922333448394, 73.9212978943735, 73.50707628581291, 73.12737314463237, 72.85122540559199, 72.60959613393166, 72.143596824301, 71.71211598205039, 71.36693130824992, 70.93545046599931, 70.60752502588885, 70.33137728684846, 69.98619261304798, 69.6410079392475, 69.43389713496721, 69.19226786330687, 68.82982395581637, 68.58819468415602, 68.26026924404557, 67.98412150500518, 67.62167759751468, 67.32827062478425, 66.91404901622369, 66.62064204349328, 66.34449430445288, 66.05108733172247, 65.75768035899206, 65.55056955471177, 65.23990334829134, 64.98101484294098, 64.58405246807042, 64.30790472903003, 64.13531239212979, 63.97997928891957, 63.70383154987918, 63.35864687607871, 63.047980669658266, 62.702795995857784, 62.478425957887474, 62.18501898515706, 62.01242664825681, 61.64998274076632, 61.40835346910597, 61.14946496375562, 60.94235415947532, 60.57991025198481, 60.303762512944424, 60.079392474974114, 59.76872626855367, 59.527096996893334, 59.19917155678288, 58.8539868829824, 58.50880220918191, 58.11183983431136, 57.83569209527097, 57.42147048671039, 56.955471177079744, 56.43769416637902, 55.88539868829825, 55.384880911287546, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 70.29266465291067, "threshold_area100": 64.55177770107008, "threshold_area25": 73.56230583362098}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.9065675735473633, "operator": "ge"}, {"budget": 0.02, "threshold": -1.166534423828125, "operator": "ge"}, {"budget": 0.03, "threshold": -1.3244881629943848, "operator": "ge"}, {"budget": 0.04, "threshold": -1.6370010375976562, "operator": "ge"}, {"budget": 0.05, "threshold": -1.9501656293869019, "operator": "ge"}, {"budget": 0.06, "threshold": -2.2148208618164062, "operator": "ge"}, {"budget": 0.07, "threshold": -2.3488986492156982, "operator": "ge"}, {"budget": 0.08, "threshold": -2.5860421657562256, "operator": "ge"}, {"budget": 0.09, "threshold": -2.797147035598755, "operator": "ge"}, {"budget": 0.1, "threshold": -3.0308480262756348, "operator": "ge"}, {"budget": 0.11, "threshold": -3.163985013961792, "operator": "ge"}, {"budget": 0.12, "threshold": -3.363757371902466, "operator": "ge"}, {"budget": 0.13, "threshold": -3.6403884887695312, "operator": "ge"}, {"budget": 0.14, "threshold": -3.918574571609497, "operator": "ge"}, {"budget": 0.15, "threshold": -4.242254257202148, "operator": "ge"}, {"budget": 0.16, "threshold": -4.354115962982178, "operator": "ge"}, {"budget": 0.17, "threshold": -4.773855209350586, "operator": "ge"}, {"budget": 0.18, "threshold": -4.953030586242676, "operator": "ge"}, {"budget": 0.19, "threshold": -5.334569931030273, "operator": "ge"}, {"budget": 0.2, "threshold": -5.738143444061279, "operator": "ge"}, {"budget": 0.21, "threshold": -5.899679183959961, "operator": "ge"}, {"budget": 0.22, "threshold": -6.201599597930908, "operator": "ge"}, {"budget": 0.23, "threshold": -6.446493148803711, "operator": "ge"}, {"budget": 0.24, "threshold": -6.696098327636719, "operator": "ge"}, {"budget": 0.25, "threshold": -6.930849075317383, "operator": "ge"}, {"budget": 0.26, "threshold": -7.063108921051025, "operator": "ge"}, {"budget": 0.27, "threshold": -7.232882022857666, "operator": "ge"}, {"budget": 0.28, "threshold": -7.410654067993164, "operator": "ge"}, {"budget": 0.29, "threshold": -7.584227085113525, "operator": "ge"}, {"budget": 0.3, "threshold": -7.7661333084106445, "operator": "ge"}, {"budget": 0.31, "threshold": -7.897588729858398, "operator": "ge"}, {"budget": 0.32, "threshold": -7.95840311050415, "operator": "ge"}, {"budget": 0.33, "threshold": -8.030013084411621, "operator": "ge"}, {"budget": 0.34, "threshold": -8.1123046875, "operator": "ge"}, {"budget": 0.35, "threshold": -8.166670799255371, "operator": "ge"}, {"budget": 0.36, "threshold": -8.226178169250488, "operator": "ge"}, {"budget": 0.37, "threshold": -8.274992942810059, "operator": "ge"}, {"budget": 0.38, "threshold": -8.328063011169434, "operator": "ge"}, {"budget": 0.39, "threshold": -8.37779426574707, "operator": "ge"}, {"budget": 0.4, "threshold": -8.426128387451172, "operator": "ge"}, {"budget": 0.41, "threshold": -8.482527732849121, "operator": "ge"}, {"budget": 0.42, "threshold": -8.551958084106445, "operator": "ge"}, {"budget": 0.43, "threshold": -8.633306503295898, "operator": "ge"}, {"budget": 0.44, "threshold": -8.690956115722656, "operator": "ge"}, {"budget": 0.45, "threshold": -8.749910354614258, "operator": "ge"}, {"budget": 0.46, "threshold": -8.783485412597656, "operator": "ge"}, {"budget": 0.47, "threshold": -8.801003456115723, "operator": "ge"}, {"budget": 0.48, "threshold": -8.835037231445312, "operator": "ge"}, {"budget": 0.49, "threshold": -8.873540878295898, "operator": "ge"}, {"budget": 0.5, "threshold": -8.931074142456055, "operator": "ge"}, {"budget": 0.51, "threshold": -8.989882469177246, "operator": "ge"}, {"budget": 0.52, "threshold": -9.027587890625, "operator": "ge"}, {"budget": 0.53, "threshold": -9.054798126220703, "operator": "ge"}, {"budget": 0.54, "threshold": -9.082120895385742, "operator": "ge"}, {"budget": 0.55, "threshold": -9.110709190368652, "operator": "ge"}, {"budget": 0.56, "threshold": -9.151256561279297, "operator": "ge"}, {"budget": 0.57, "threshold": -9.182638168334961, "operator": "ge"}, {"budget": 0.58, "threshold": -9.211566925048828, "operator": "ge"}, {"budget": 0.59, "threshold": -9.244813919067383, "operator": "ge"}, {"budget": 0.6, "threshold": -9.290145874023438, "operator": "ge"}, {"budget": 0.61, "threshold": -9.321355819702148, "operator": "ge"}, {"budget": 0.62, "threshold": -9.353793144226074, "operator": "ge"}, {"budget": 0.63, "threshold": -9.381553649902344, "operator": "ge"}, {"budget": 0.64, "threshold": -9.431309700012207, "operator": "ge"}, {"budget": 0.65, "threshold": -9.471324920654297, "operator": "ge"}, {"budget": 0.66, "threshold": -9.494584083557129, "operator": "ge"}, {"budget": 0.67, "threshold": -9.537944793701172, "operator": "ge"}, {"budget": 0.68, "threshold": -9.566207885742188, "operator": "ge"}, {"budget": 0.69, "threshold": -9.579904556274414, "operator": "ge"}, {"budget": 0.7, "threshold": -9.615096092224121, "operator": "ge"}, {"budget": 0.71, "threshold": -9.664339065551758, "operator": "ge"}, {"budget": 0.72, "threshold": -9.701135635375977, "operator": "ge"}, {"budget": 0.73, "threshold": -9.735332489013672, "operator": "ge"}, {"budget": 0.74, "threshold": -9.758995056152344, "operator": "gt"}, {"budget": 0.75, "threshold": -9.798196792602539, "operator": "ge"}, {"budget": 0.76, "threshold": -9.826529502868652, "operator": "ge"}, {"budget": 0.77, "threshold": -9.861682891845703, "operator": "ge"}, {"budget": 0.78, "threshold": -9.89771556854248, "operator": "ge"}, {"budget": 0.79, "threshold": -9.955131530761719, "operator": "ge"}, {"budget": 0.8, "threshold": -10.008405685424805, "operator": "ge"}, {"budget": 0.81, "threshold": -10.055174827575684, "operator": "ge"}, {"budget": 0.82, "threshold": -10.112680435180664, "operator": "ge"}, {"budget": 0.83, "threshold": -10.142480850219727, "operator": "ge"}, {"budget": 0.84, "threshold": -10.18010139465332, "operator": "ge"}, {"budget": 0.85, "threshold": -10.242002487182617, "operator": "ge"}, {"budget": 0.86, "threshold": -10.302589416503906, "operator": "ge"}, {"budget": 0.87, "threshold": -10.390048027038574, "operator": "ge"}, {"budget": 0.88, "threshold": -10.448141098022461, "operator": "ge"}, {"budget": 0.89, "threshold": -10.510570526123047, "operator": "ge"}, {"budget": 0.9, "threshold": -10.622114181518555, "operator": "ge"}, {"budget": 0.91, "threshold": -10.690901756286621, "operator": "ge"}, {"budget": 0.92, "threshold": -10.750247955322266, "operator": "ge"}, {"budget": 0.93, "threshold": -10.829465866088867, "operator": "ge"}, {"budget": 0.94, "threshold": -10.942495346069336, "operator": "ge"}, {"budget": 0.95, "threshold": -11.055521011352539, "operator": "ge"}, {"budget": 0.96, "threshold": -11.12191390991211, "operator": "ge"}, {"budget": 0.97, "threshold": -11.279150009155273, "operator": "ge"}, {"budget": 0.98, "threshold": -11.370201110839844, "operator": "ge"}, {"budget": 0.99, "threshold": -11.602167129516602, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.0, 97.33333333333334, 96.83333333333334, 96.25, 95.58333333333333, 95.0, 94.5, 93.83333333333333, 93.5, 92.83333333333333, 92.0, 91.5, 91.16666666666666, 90.5, 89.83333333333333, 89.08333333333334, 88.58333333333334, 88.0, 87.5, 87.08333333333333, 86.41666666666666, 85.91666666666666, 85.41666666666667, 85.08333333333333, 84.66666666666667, 84.33333333333334, 83.91666666666666, 83.33333333333334, 82.91666666666666, 82.5, 82.25, 81.66666666666667, 81.41666666666667, 81.16666666666667, 81.0, 80.58333333333334, 80.08333333333333, 80.0, 79.75, 79.41666666666667, 79.16666666666667, 78.91666666666667, 78.5, 78.16666666666667, 77.58333333333334, 77.16666666666667, 77.0, 76.83333333333333, 76.25, 75.91666666666667, 75.75, 75.25, 75.08333333333333, 74.75, 74.41666666666666, 74.0, 73.66666666666667, 73.33333333333334, 72.91666666666667, 72.66666666666667, 72.50000000000001, 72.08333333333333, 71.83333333333333, 71.41666666666666, 71.08333333333333, 70.75, 70.33333333333334, 70.25, 69.91666666666667, 69.66666666666667, 69.16666666666667, 68.58333333333333, 67.91666666666667, 67.58333333333333, 67.08333333333334, 66.91666666666667, 66.5, 66.08333333333334, 65.58333333333333, 64.91666666666667, 64.58333333333333, 64.08333333333334, 63.5, 63.0, 62.583333333333336, 62.0, 61.5, 61.16666666666667, 60.66666666666667, 60.00000000000001, 59.66666666666667, 59.33333333333333, 58.916666666666664, 58.666666666666664, 58.00000000000001, 57.333333333333336, 56.833333333333336, 56.00000000000001, 55.666666666666664, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 73.91666666666666, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 7, 15, 21, 28, 36, 43, 49, 57, 62, 70, 80, 86, 91, 99, 107, 116, 122, 129, 135, 140, 148, 154, 160, 164, 169, 173, 178, 185, 190, 195, 198, 205, 208, 211, 213, 218, 224, 225, 228, 232, 235, 238, 243, 247, 254, 259, 261, 263, 270, 274, 276, 282, 284, 288, 292, 297, 301, 305, 310, 313, 315, 320, 323, 328, 332, 336, 341, 342, 346, 349, 355, 362, 370, 374, 380, 382, 387, 392, 398, 406, 410, 416, 423, 429, 434, 441, 447, 451, 457, 465, 469, 473, 478, 481, 489, 497, 503, 513, 517, 524], "exact_score": [98.58333333333333, 98.0, 97.33333333333334, 96.83333333333334, 96.25, 95.58333333333333, 95.0, 94.5, 93.83333333333333, 93.5, 92.83333333333333, 92.0, 91.5, 91.16666666666666, 90.5, 89.83333333333333, 89.08333333333334, 88.58333333333334, 88.0, 87.5, 87.08333333333333, 86.41666666666666, 85.91666666666666, 85.41666666666667, 85.08333333333333, 84.66666666666667, 84.33333333333334, 83.91666666666666, 83.33333333333334, 82.91666666666666, 82.5, 82.25, 81.66666666666667, 81.41666666666667, 81.16666666666667, 81.0, 80.58333333333334, 80.08333333333333, 80.0, 79.75, 79.41666666666667, 79.16666666666667, 78.91666666666667, 78.5, 78.16666666666667, 77.58333333333334, 77.16666666666667, 77.0, 76.83333333333333, 76.25, 75.91666666666667, 75.75, 75.25, 75.08333333333333, 74.75, 74.41666666666666, 74.0, 73.66666666666667, 73.33333333333334, 72.91666666666667, 72.66666666666667, 72.50000000000001, 72.08333333333333, 71.83333333333333, 71.41666666666666, 71.08333333333333, 70.75, 70.33333333333334, 70.25, 69.83333333333334, 69.66666666666667, 69.16666666666667, 68.58333333333333, 67.91666666666667, 67.58333333333333, 67.08333333333334, 66.91666666666667, 66.5, 66.08333333333334, 65.58333333333333, 64.91666666666667, 64.58333333333333, 64.08333333333334, 63.5, 63.0, 62.583333333333336, 62.0, 61.5, 61.16666666666667, 60.66666666666667, 60.00000000000001, 59.66666666666667, 59.33333333333333, 58.916666666666664, 58.666666666666664, 58.00000000000001, 57.333333333333336, 56.833333333333336, 56.00000000000001, 55.666666666666664, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 76.185, "threshold_area100": 76.18583333333333, "threshold_area25": 91.39166666666667}, "test": {"score": [81.429064549534, 80.2899551259924, 79.40973420780118, 78.82292026234036, 77.56299620296859, 76.54470141525717, 75.73351743182603, 75.61270279599586, 74.90507421470487, 74.23196410079392, 73.61063168795306, 73.24818778046254, 72.73041076976182, 72.35070762858129, 71.67759751467035, 71.0735243355195, 70.84915429754919, 70.05522954780807, 69.83085950983777, 69.43389713496721, 68.81256472212634, 68.51915774939593, 68.19123230928547, 68.0013807386952, 67.72523299965482, 67.19019675526407, 67.05212288574387, 66.96582671729375, 66.6379012771833, 66.37901277183293, 65.98205039696238, 65.7921988263721, 65.67138419054194, 65.5678287884018, 65.44701415257163, 65.32619951674145, 65.23990334829134, 65.10182947877115, 65.01553331032103, 64.9292371418709, 64.73938557128064, 64.5667932343804, 64.32516396272005, 63.962720055229546, 63.75560925094926, 63.46220227821885, 63.35864687607871, 63.2896099413186, 63.20331377286848, 63.01346220227821, 62.72005522954781, 62.44390749050742, 62.271315153607176, 62.08146358301691, 61.94338971349672, 61.839834311356576, 61.71901967552641, 61.63272350707629, 61.52916810493614, 61.391094235415956, 61.21850189851571, 61.04590956161546, 60.80428028995512, 60.64894718674492, 60.47635484984467, 60.2865032792544, 60.14842940973422, 60.01035554021401, 59.90680013807387, 59.82050396962375, 59.66517086641353, 59.47531929582327, 59.26820849154297, 59.13013462202279, 59.00931998619261, 58.83672764929238, 58.76769071453227, 58.526061442871935, 58.40524680704176, 58.18087676907145, 57.99102519848118, 57.7839143942009, 57.57680358992061, 57.455988954090444, 57.2833966171902, 57.07628581290991, 56.88643424231964, 56.66206420434933, 56.55850880220918, 56.454953400069044, 56.24784259578875, 56.10976872626856, 55.93717638936832, 55.74732481877804, 55.54021401449776, 55.29858474283742, 55.229547808077314, 55.02243700379703, 54.97065930272696, 54.86710390058682, 54.81532619951675], "calls": [0.0, 5.868139454608215, 10.131170176044185, 13.237832240248535, 18.898860890576458, 24.559889540904383, 29.030031066620644, 31.22195374525371, 34.5012081463583, 37.090093199861926, 39.868829823955814, 40.86986537797721, 42.57852951328961, 44.287193648602, 46.289264756644805, 48.20503969623749, 48.87814981014843, 51.05281325509148, 51.81221953745254, 53.48636520538488, 55.195029340697275, 55.97169485674836, 57.179841215050054, 58.111839834311354, 58.940283051432516, 60.11391094235415, 60.49361408353469, 61.02865032792544, 61.89161201242664, 62.87538833275803, 64.08353469105973, 65.0673110113911, 65.5678287884018, 66.08560579910252, 66.68967897825337, 67.2074559889541, 67.79426993441491, 68.2084915429755, 68.60545391784605, 68.98515705902658, 69.55471177079737, 69.98619261304798, 70.93545046599931, 71.81567138419054, 72.40248532965137, 73.4207801173628, 73.88677942699344, 74.09389023127373, 74.5771487745944, 75.26751812219537, 76.30307214359682, 77.14877459440801, 77.77010700724888, 78.28788401794961, 78.70210562651019, 79.13358646876078, 79.7894373489817, 80.16914049016223, 80.53158439765275, 81.0666206420435, 81.75698998964445, 82.46461857093546, 83.1377286848464, 83.569209527097, 84.34587504314808, 85.08802209181913, 85.5022437003797, 85.88194684156024, 86.17535381429065, 86.41698308595099, 86.90024162927166, 87.72868484639282, 88.16016568864342, 88.72972040041422, 89.05764584052469, 89.73075595443562, 90.02416292716603, 90.54193993786676, 91.04245771487746, 91.73282706247844, 92.30238177424923, 92.88919571971005, 93.35519502934069, 93.56230583362098, 93.9247497411115, 94.54608215395236, 94.97756299620296, 95.58163617535381, 95.9958577839144, 96.34104245771488, 96.84156023472558, 97.23852260959613, 97.53192958232655, 97.9979288919572, 98.39489126682776, 98.89540904383846, 99.03348291335865, 99.51674145667933, 99.62029685881947, 99.82740766309975, 100.0], "rescued": [0, 59, 90, 119, 169, 218, 253, 274, 292, 304, 320, 323, 328, 335, 342, 348, 350, 353, 354, 356, 356, 356, 356, 356, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 357, 359, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 362, 362, 362, 362, 363, 363, 363, 363, 363, 363, 363, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 364, 365, 365, 365], "harmed": [0, 125, 207, 270, 393, 501, 583, 611, 670, 721, 773, 797, 832, 861, 907, 948, 963, 1012, 1026, 1051, 1087, 1104, 1123, 1134, 1151, 1182, 1190, 1195, 1214, 1229, 1252, 1263, 1270, 1276, 1283, 1290, 1297, 1305, 1311, 1316, 1327, 1337, 1351, 1372, 1384, 1401, 1407, 1411, 1416, 1427, 1444, 1460, 1470, 1481, 1490, 1496, 1503, 1508, 1514, 1522, 1532, 1542, 1556, 1565, 1575, 1586, 1594, 1602, 1608, 1613, 1622, 1633, 1645, 1653, 1660, 1670, 1675, 1689, 1696, 1709, 1721, 1733, 1745, 1752, 1762, 1774, 1785, 1799, 1805, 1811, 1823, 1831, 1841, 1852, 1864, 1878, 1882, 1894, 1898, 1904, 1907], "exact_score": [81.429064549534, 81.23921297894374, 81.10113910942354, 80.85950983776321, 80.68691750086296, 80.56610286503279, 80.25543665861235, 80.10010355540214, 79.8584742837418, 79.68588194684156, 79.4269934414912, 79.21988263721092, 79.01277183293061, 78.90921643079048, 78.66758715913014, 78.3569209527097, 78.25336555056955, 77.92544011045909, 77.77010700724888, 77.52847773558854, 77.25232999654816, 77.11425612702796, 76.97618225750776, 76.83810838798757, 76.66551605108734, 76.49292371418709, 76.30307214359682, 76.21677597514672, 75.99240593717639, 75.75077666551606, 75.75077666551606, 75.62996202968588, 75.56092509492579, 75.26751812219537, 75.09492578529513, 74.83603727994476, 74.50811183983431, 74.24922333448394, 74.0593717638937, 73.80048325854332, 73.50707628581291, 73.21366931308249, 72.86848463928202, 72.67863306869175, 72.4197445633414, 72.12633759061097, 71.81567138419054, 71.45322747670004, 71.17707973765965, 70.79737659647913, 70.36589575422852, 70.0724887814981, 69.81360027614774, 69.55471177079737, 69.33034173282707, 68.95063859164652, 68.51915774939593, 68.2775284777356, 68.0013807386952, 67.7079737659648, 67.22471522264412, 66.98308595098378, 66.60338280980325, 66.32723507076285, 66.03382809803244, 65.82671729375215, 65.5678287884018, 65.37797721781153, 65.17086641353124, 64.9292371418709, 64.5667932343804, 64.30790472903003, 63.91094235415948, 63.548498446668965, 63.30686917500863, 63.117017604418365, 62.7890921643079, 62.4956851915775, 62.16775975146703, 61.87435277873663, 61.66724197445633, 61.40835346910597, 61.114946496375566, 60.8560579910252, 60.54539178460476, 60.32102174663445, 59.97583707283397, 59.630652399033494, 59.33724542630308, 59.026579219882635, 58.7849499482223, 58.42250604073179, 58.06006213324129, 57.71487745944081, 57.248878149810146, 56.869175008629625, 56.55850880220918, 56.21332412840869, 55.74732481877804, 55.229547808077314, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 69.7119930167127, "threshold_area100": 63.83224024853297, "threshold_area25": 72.87021056265102}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.429064549534, "calls": 1.0355540214014498}}, "old_normalized_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.429064549534, "calls": 1.0355540214014498}}, "old_normalized_endpoints": {"val": {"score": 98.58333333333333, "calls": 0.0}, "test": {"score": 81.429064549534, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.25, 98.16666666666667, 98.08333333333333, 98.08333333333333, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.5, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.16666666666667, 97.16666666666667, 97.0, 96.75, 96.33333333333334, 96.41666666666666, 96.16666666666667, 96.0, 95.83333333333334, 95.58333333333333, 95.41666666666667, 95.08333333333333, 94.83333333333334, 94.66666666666667, 94.25, 93.83333333333333, 93.66666666666667, 93.08333333333333, 92.58333333333333, 92.25, 91.83333333333333, 91.5, 91.16666666666666, 90.66666666666667, 90.16666666666666, 89.41666666666667, 88.75, 88.25, 87.75, 87.41666666666667, 86.83333333333334, 86.25, 85.58333333333333, 85.0, 84.25, 83.91666666666666, 83.08333333333333, 82.58333333333333, 81.83333333333334, 81.08333333333333, 80.41666666666667, 79.75, 79.16666666666667, 78.41666666666667, 77.66666666666666, 76.91666666666667, 76.16666666666667, 75.41666666666667, 74.58333333333333, 73.91666666666666, 73.0, 72.16666666666667, 71.50000000000001, 70.66666666666667, 69.83333333333334, 68.91666666666667, 67.91666666666667, 67.08333333333334, 66.16666666666666, 65.33333333333333, 64.5, 63.66666666666667, 62.66666666666667, 61.74999999999999, 60.916666666666664, 59.916666666666664, 58.916666666666664, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.33333333333334, 98.33333333333334, 98.33333333333334, 98.25, 98.25, 98.25, 98.16666666666667, 98.08333333333333, 98.08333333333333, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.5, 97.41666666666666, 97.33333333333334, 97.33333333333334, 97.16666666666667, 97.16666666666667, 97.0, 96.75, 96.33333333333334, 96.41666666666666, 96.16666666666667, 96.0, 95.83333333333334, 95.58333333333333, 95.41666666666667, 95.08333333333333, 94.83333333333334, 94.66666666666667, 94.25, 93.83333333333333, 93.66666666666667, 93.08333333333333, 92.58333333333333, 92.25, 91.83333333333333, 91.5, 91.16666666666666, 90.66666666666667, 90.16666666666666, 89.41666666666667, 88.75, 88.25, 87.75, 87.41666666666667, 86.83333333333334, 86.25, 85.58333333333333, 85.0, 84.25, 83.91666666666666, 83.08333333333333, 82.58333333333333, 81.83333333333334, 81.08333333333333, 80.41666666666667, 79.75, 79.16666666666667, 78.41666666666667, 77.66666666666666, 76.91666666666667, 76.16666666666667, 75.41666666666667, 74.58333333333333, 73.91666666666666, 73.0, 72.16666666666667, 71.41666666666666, 70.66666666666667, 69.83333333333334, 68.91666666666667, 67.91666666666667, 67.08333333333334, 66.16666666666666, 65.33333333333333, 64.5, 63.66666666666667, 62.66666666666667, 61.74999999999999, 60.916666666666664, 59.916666666666664, 58.916666666666664, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.34166666666667, "exact_area100": 86.33999999999999}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.32550914739386, 81.30824991370383, 81.2909906800138, 81.27373144632378, 81.27373144632378, 81.27373144632378, 81.25647221263375, 81.25647221263375, 81.23921297894374, 81.22195374525371, 81.11839834311357, 81.1356575768036, 81.11839834311357, 81.0666206420435, 81.01484294097342, 80.9975837072834, 80.9285467725233, 80.91128753883328, 80.84225060407319, 80.77321366931308, 80.77321366931308, 80.72143596824301, 80.70417673455299, 80.61788056610285, 80.53158439765275, 80.46254746289266, 80.41076976182258, 80.25543665861235, 80.2899551259924, 80.16914049016223, 80.10010355540214, 80.06558508802209, 79.97928891957199, 79.77217811529168, 79.66862271315154, 79.47877114256127, 79.32343803935106, 78.9955125992406, 78.75388332758025, 78.56403175698999, 78.20158784949949, 77.90818087676908, 77.63203313772868, 77.47670003451846, 77.30410769761824, 76.88988608905764, 76.68277528477736, 76.35484984466689, 75.7852951328961, 75.47462892647566, 74.95685191577495, 74.52537107352434, 74.2147048671039, 73.86952019330342, 73.1446323783224, 72.52329996548154, 72.09181912323092, 71.31515360717984, 70.62478425957887, 69.91715567828788, 69.29582326544703, 68.74352778736625, 67.98412150500518, 67.44908526061442, 67.01760441836383, 66.58612357611322, 65.9993096306524, 65.41249568519159, 64.68760787021056, 64.39420089748016, 63.9282015878495, 63.61753538142907, 62.90990680013808, 62.702795995857784, 62.23679668622714, 61.73627890921644, 61.339316534345876, 60.82153952364515, 60.40731791508457, 59.924059371763896, 59.09561615464274, 58.81946841560235, 58.40524680704176, 57.852951328961, 57.57680358992061, 56.8519157749396, 56.21332412840869, 55.712806351397994, 55.1432516396272, 54.81532619951675], "calls": [0.0, 1.0355540214014498, 1.725923369002416, 2.2091819123230927, 2.77873662409389, 3.486365205384881, 4.193993786675872, 4.694511563686572, 5.333103210217467, 5.712806351397997, 6.731101139109423, 7.680358992060753, 8.180876769071453, 8.940283051432516, 9.37176389368312, 10.11391094235416, 10.82153952364515, 11.322057300655851, 12.167759751467035, 12.685536762167759, 13.272350707628581, 14.791163272350708, 15.688643424231962, 16.344494304452883, 16.91404901622368, 17.949603037625128, 18.39834311356576, 19.882637210907834, 20.849154297549187, 21.52226441146013, 22.88574387297204, 23.731446323783224, 24.905074214704868, 25.854332067656195, 26.717293752157406, 27.666551605108737, 28.97825336555057, 30.600621332412842, 32.25750776665516, 33.03417328270625, 34.08698653779772, 34.898170521228856, 36.22713151536072, 37.33172247152227, 39.506385916465305, 40.714532274767, 41.905419399378665, 43.389713496720745, 44.64963755609251, 46.16845012081463, 47.53192958232654, 49.86192613047981, 51.605108733172244, 52.31273731446324, 54.38384535726614, 55.79910251984812, 57.90472903003106, 59.33724542630308, 60.700724887814985, 62.909906800138074, 64.0662754573697, 65.96479116327235, 67.55264066275457, 69.0024162927166, 69.86537797721782, 72.05730065585088, 73.52433551950294, 74.52537107352434, 76.30307214359682, 77.63203313772868, 79.13358646876078, 80.37625129444253, 81.32550914739386, 82.72350707628581, 83.62098722816707, 84.39765274421815, 85.08802209181913, 85.88194684156024, 86.83120469451157, 87.90127718329306, 88.28098032447359, 88.90231273731446, 89.4891266827753, 90.40386606834656, 90.74905074214705, 91.43942008974801, 92.07801173627891, 92.61304798066966, 93.33793579565068, 93.7866758715913, 94.4252675181222, 95.32274767000345, 95.66793234380393, 96.15119088712461, 96.77252329996548, 97.08318950638592, 97.91163272350708, 98.567483603728, 99.06800138073869, 99.67207455988955, 100.0], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.25647221263375, 81.23921297894374, 81.18743527787366, 81.1356575768036, 81.15291681049361, 81.0666206420435, 80.9975837072834, 80.9975837072834, 80.91128753883328, 80.87676907145323, 80.7904729030031, 80.77321366931308, 80.7904729030031, 80.72143596824301, 80.66965826717293, 80.53158439765275, 80.4970659302727, 80.46254746289266, 80.4280289955126, 80.35899206075251, 80.22091819123231, 80.27269589230238, 80.16914049016223, 80.10010355540214, 80.08284432171212, 80.03106662064205, 79.91025198481188, 79.87573351743183, 79.80669658267173, 79.58232654470142, 79.4960303762513, 79.30617880566103, 79.18536416983086, 78.94373489817053, 78.77114256127028, 78.70210562651019, 78.44321712115982, 78.20158784949949, 78.20158784949949, 77.97721781152917, 77.77010700724888, 77.63203313772868, 77.47670003451846, 77.37314463237833, 77.28684846392821, 77.14877459440801, 76.94166379012772, 76.73455298584743, 76.45840524680705, 76.23403520883673, 76.07870210562652, 75.76803589920608, 75.49188816016569, 75.18122195374526, 74.9223334483949, 74.7842595788747, 74.38729720400414, 74.2147048671039, 73.80048325854332, 73.4207801173628, 73.17915084570245, 72.79944770452192, 72.19537452537108, 71.88470831895064, 71.47048671039006, 70.95270969968934, 70.4176734552986, 69.96893337935795, 69.46841560234725, 68.96789782533655, 68.39834311356576, 67.82878840179495, 67.24197445633415, 66.6379012771833, 65.91301346220229, 65.27442181567137, 64.63583016914049, 63.9282015878495, 63.25509147393856, 62.53020365895754, 61.822575077666556, 61.06316879530549, 60.25198481187435, 59.38902312737315, 58.54332067656196, 57.66309975837073, 56.76561960648947, 55.78184328615809, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 81.29789437348983, "exact_area100": 75.24287739366864}}, "old_normalized_endpoint_budget": 0}, {"seed": 2030, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.25, "validation_area100": 0.8641083333333334, "training_seconds": 3.346375847933814, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.018356603278121827, "operator": "ge"}, {"budget": 0.02, "threshold": -0.020972923773089333, "operator": "ge"}, {"budget": 0.03, "threshold": -0.024467329587366624, "operator": "ge"}, {"budget": 0.04, "threshold": -0.02863309118463944, "operator": "ge"}, {"budget": 0.05, "threshold": -0.030915678599495722, "operator": "ge"}, {"budget": 0.06, "threshold": -0.03458089555259861, "operator": "ge"}, {"budget": 0.07, "threshold": -0.03872119933605097, "operator": "ge"}, {"budget": 0.08, "threshold": -0.045012554123386776, "operator": "ge"}, {"budget": 0.09, "threshold": -0.048960511355128584, "operator": "ge"}, {"budget": 0.1, "threshold": -0.05166707369668397, "operator": "ge"}, {"budget": 0.11, "threshold": -0.0557196065343051, "operator": "ge"}, {"budget": 0.12, "threshold": -0.061312754893958765, "operator": "ge"}, {"budget": 0.13, "threshold": -0.06706684853229355, "operator": "ge"}, {"budget": 0.14, "threshold": -0.07130670741540417, "operator": "ge"}, {"budget": 0.15, "threshold": -0.07524586626853941, "operator": "ge"}, {"budget": 0.16, "threshold": -0.08019397551791718, "operator": "ge"}, {"budget": 0.17, "threshold": -0.08531177828292798, "operator": "ge"}, {"budget": 0.18, "threshold": -0.0890302966950709, "operator": "ge"}, {"budget": 0.19, "threshold": -0.09124244758830781, "operator": "ge"}, {"budget": 0.2, "threshold": -0.09640671146149143, "operator": "ge"}, {"budget": 0.21, "threshold": -0.10278940025837717, "operator": "ge"}, {"budget": 0.22, "threshold": -0.10885001530346007, "operator": "ge"}, {"budget": 0.23, "threshold": -0.11336628985989229, "operator": "ge"}, {"budget": 0.24, "threshold": -0.12148784292588378, "operator": "ge"}, {"budget": 0.25, "threshold": -0.13003937833710952, "operator": "ge"}, {"budget": 0.26, "threshold": -0.1410540152346142, "operator": "ge"}, {"budget": 0.27, "threshold": -0.14693667932469595, "operator": "ge"}, {"budget": 0.28, "threshold": -0.1511956618693147, "operator": "ge"}, {"budget": 0.29, "threshold": -0.16375916612902158, "operator": "ge"}, {"budget": 0.3, "threshold": -0.1702207606860085, "operator": "ge"}, {"budget": 0.31, "threshold": -0.17999284412982233, "operator": "ge"}, {"budget": 0.32, "threshold": -0.18950016221908647, "operator": "ge"}, {"budget": 0.33, "threshold": -0.2021271845900531, "operator": "ge"}, {"budget": 0.34, "threshold": -0.21001434024906238, "operator": "ge"}, {"budget": 0.35, "threshold": -0.2152777558081793, "operator": "ge"}, {"budget": 0.36, "threshold": -0.22576641925442872, "operator": "ge"}, {"budget": 0.37, "threshold": -0.23462385541410352, "operator": "ge"}, {"budget": 0.38, "threshold": -0.2451521061651035, "operator": "ge"}, {"budget": 0.39, "threshold": -0.25629899270633355, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2646652110796781, "operator": "ge"}, {"budget": 0.41, "threshold": -0.27056722104071657, "operator": "ge"}, {"budget": 0.42, "threshold": -0.284118274178787, "operator": "ge"}, {"budget": 0.43, "threshold": -0.30222800033192615, "operator": "ge"}, {"budget": 0.44, "threshold": -0.31299003257356695, "operator": "ge"}, {"budget": 0.45, "threshold": -0.32710405437309, "operator": "ge"}, {"budget": 0.46, "threshold": -0.33772536110968904, "operator": "ge"}, {"budget": 0.47, "threshold": -0.35299775064887556, "operator": "ge"}, {"budget": 0.48, "threshold": -0.36294576571758136, "operator": "ge"}, {"budget": 0.49, "threshold": -0.379741876587805, "operator": "ge"}, {"budget": 0.5, "threshold": -0.39443084803851064, "operator": "ge"}, {"budget": 0.51, "threshold": -0.41008148166029346, "operator": "ge"}, {"budget": 0.52, "threshold": -0.4243482681103974, "operator": "ge"}, {"budget": 0.53, "threshold": -0.4321175032904326, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4408994318809983, "operator": "ge"}, {"budget": 0.55, "threshold": -0.45270923048999484, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4629635662793249, "operator": "ge"}, {"budget": 0.57, "threshold": -0.47859057511358677, "operator": "ge"}, {"budget": 0.58, "threshold": -0.4899666057803349, "operator": "ge"}, {"budget": 0.59, "threshold": -0.4984368150473166, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5110750435782715, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5233034749523672, "operator": "ge"}, {"budget": 0.62, "threshold": -0.536083369963244, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5559942523817883, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5726685772764756, "operator": "ge"}, {"budget": 0.65, "threshold": -0.5841379033468923, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6130772491197338, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6251195739182788, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6357516658327236, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6432477835559988, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6595291584193005, "operator": "ge"}, {"budget": 0.71, "threshold": -0.6715804550594611, "operator": "ge"}, {"budget": 0.72, "threshold": -0.6814480423388593, "operator": "ge"}, {"budget": 0.73, "threshold": -0.6969262522808747, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7274823123029214, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7392516647674954, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7545112172049598, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7641724278335188, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7740992091409525, "operator": "ge"}, {"budget": 0.79, "threshold": -0.7891819377071372, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8038616931420739, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8095714036026712, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8176214722872392, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8238576939618382, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8302922186835947, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8505368823344703, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8653888659320021, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8765224963413518, "operator": "ge"}, {"budget": 0.88, "threshold": -0.885663358808972, "operator": "ge"}, {"budget": 0.89, "threshold": -0.8955248984044567, "operator": "ge"}, {"budget": 0.9, "threshold": -0.911753701583953, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9216910014216718, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9301202090309535, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9367981538030665, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9455734945630098, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9550286986196898, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9628198529172645, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9673619280785041, "operator": "ge"}, {"budget": 0.98, "threshold": -0.972994618492366, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9766875105265583, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.41666666666666, 97.08333333333333, 97.0, 96.58333333333333, 96.25, 96.16666666666667, 95.91666666666667, 95.58333333333333, 95.33333333333334, 95.08333333333333, 95.0, 94.83333333333334, 94.5, 94.25, 93.91666666666667, 93.41666666666667, 93.16666666666666, 93.08333333333333, 92.5, 92.25, 92.16666666666666, 91.75, 91.41666666666667, 90.75, 90.41666666666667, 89.83333333333333, 89.5, 89.16666666666667, 88.5, 87.91666666666667, 87.25, 86.66666666666667, 86.08333333333333, 85.58333333333333, 85.0, 84.25, 83.66666666666667, 83.0, 82.16666666666667, 81.41666666666667, 80.75, 80.0, 79.16666666666667, 78.41666666666667, 77.83333333333333, 77.08333333333334, 76.16666666666667, 75.5, 74.83333333333333, 74.0, 73.16666666666667, 72.41666666666666, 71.50000000000001, 70.5, 69.83333333333334, 68.91666666666667, 68.0, 67.33333333333333, 66.41666666666667, 65.5, 64.58333333333333, 63.66666666666667, 62.66666666666667, 61.66666666666667, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 56.91666666666666, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 3, 4, 6, 7, 7, 8, 10, 11, 11, 11, 11, 12, 13, 14, 18, 19, 24, 28, 29, 32, 36, 39, 42, 43, 45, 49, 52, 57, 63, 66, 67, 74, 77, 78, 83, 87, 95, 99, 106, 110, 114, 122, 129, 137, 144, 151, 158, 165, 174, 181, 189, 199, 208, 216, 225, 235, 244, 251, 260, 271, 279, 287, 297, 307, 316, 327, 339, 347, 358, 369, 377, 388, 399, 410, 421, 433, 445, 455, 467, 479, 491, 502, 512, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.41666666666666, 97.08333333333333, 97.0, 96.58333333333333, 96.25, 96.16666666666667, 95.91666666666667, 95.58333333333333, 95.33333333333334, 95.08333333333333, 95.0, 94.83333333333334, 94.5, 94.25, 93.91666666666667, 93.41666666666667, 93.16666666666666, 93.08333333333333, 92.5, 92.25, 92.16666666666666, 91.75, 91.41666666666667, 90.75, 90.41666666666667, 89.83333333333333, 89.5, 89.16666666666667, 88.5, 87.91666666666667, 87.25, 86.66666666666667, 86.08333333333333, 85.58333333333333, 85.0, 84.25, 83.66666666666667, 83.0, 82.16666666666667, 81.33333333333333, 80.75, 80.0, 79.16666666666667, 78.41666666666667, 77.83333333333333, 77.08333333333334, 76.16666666666667, 75.5, 74.83333333333333, 74.0, 73.16666666666667, 72.41666666666666, 71.41666666666666, 70.5, 69.83333333333334, 68.91666666666667, 68.0, 67.33333333333333, 66.41666666666667, 65.5, 64.58333333333333, 63.66666666666667, 62.66666666666667, 61.66666666666667, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 56.91666666666666, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.41083333333334, "threshold_area100": 86.41250000000001, "threshold_area25": 98.32499999999999}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.34276838108389, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.23921297894374, 81.22195374525371, 81.22195374525371, 81.18743527787366, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.10113910942354, 81.04936140835348, 80.9975837072834, 80.98032447359338, 80.96306523990336, 80.89402830514325, 80.84225060407319, 80.73869520193303, 80.73869520193303, 80.66965826717293, 80.56610286503279, 80.4970659302727, 80.39351052813255, 80.39351052813255, 80.30721435968243, 80.16914049016223, 80.06558508802209, 80.10010355540214, 80.10010355540214, 79.94477045219193, 79.87573351743183, 79.70314118053159, 79.66862271315154, 79.4960303762513, 79.3579565067311, 79.25440110459095, 79.08180876769072, 78.71936486020022, 78.47773558853987, 78.30514325163963, 78.21884708318952, 78.09803244735933, 77.80462547462893, 77.54573696927856, 77.4249223334484, 77.3558853986883, 76.97618225750776, 76.73455298584743, 76.40662754573697, 76.09596133931653, 75.75077666551606, 75.40559199171557, 74.61166724197446, 74.2147048671039, 74.0766309975837, 73.71418709009319, 72.90300310666207, 72.40248532965137, 72.02278218847084, 71.36693130824992, 69.86537797721782, 69.26130479806697, 68.5018985157059, 68.05315843976527, 67.60441836382465, 66.75871591301346, 66.30997583707283, 65.96479116327235, 65.70590265792198, 65.32619951674145, 64.96375560925095, 63.91094235415948, 62.97894373489817, 62.478425957887474, 61.94338971349672, 61.37383500172593, 60.424577148774596, 59.630652399033494, 58.97480151881257, 58.526061442871935, 57.870210562651025, 57.26613738350017, 56.55850880220918, 56.14428719364861, 55.73006558508802, 55.33310321021747, 54.81532619951675], "calls": [0.0, 0.6040731791508458, 1.2944425267518123, 2.0538488091128753, 2.968588194684156, 3.40006903693476, 4.159475319295823, 4.832585433206766, 5.954435623058337, 6.679323438039352, 7.197100448740076, 7.870210562651018, 8.715913013462202, 9.699689333793579, 10.390058681394546, 11.02865032792544, 11.719019675526408, 12.374870555747325, 12.909906800138074, 13.117017604418363, 13.807386952019328, 14.445978598550225, 15.08457024508112, 15.533310321021746, 16.775975146703487, 17.846047635484986, 19.330341732827065, 19.882637210907834, 20.348636520538488, 21.505005177770105, 22.09181912323093, 23.058336209872284, 24.00759406282361, 25.371073524335518, 26.268553676216776, 27.010700724887815, 27.787366240938905, 28.753883327580255, 29.89299275112185, 31.04936140835347, 31.756989989644456, 32.25750776665516, 33.53469105971695, 35.24335519502934, 36.39972385226096, 37.573351743182606, 38.798757335174315, 40.33482913358647, 41.24956851915775, 42.85467725233, 44.373489817052125, 46.20296858819468, 47.96341042457715, 48.80911287538833, 49.72385226095961, 51.05281325509148, 52.329996548153254, 54.15947531929582, 55.33310321021747, 56.10976872626855, 57.680358992060754, 59.233690024162925, 60.75250258888505, 63.117017604418365, 64.87745944080083, 66.12012426648258, 68.72626855367622, 69.98619261304798, 70.81463583016914, 71.60856057991025, 73.35174318260269, 74.30100103555402, 75.04314808422507, 76.2167759751467, 78.80566102865033, 79.89299275112185, 81.27373144632378, 82.03313772868485, 82.91335864687608, 84.17328270624785, 85.31239212978944, 85.88194684156024, 86.38246461857094, 86.93476009665171, 87.46979634104247, 88.74697963410425, 89.97238522609597, 90.69727304107698, 91.49119779081809, 92.33690024162927, 93.64860200207112, 94.61511908871246, 95.44356230583362, 96.03037625129444, 96.77252329996548, 97.4283741801864, 98.15326199516741, 98.58474283741802, 99.03348291335865, 99.46496375560925, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 2, 5, 7, 8, 9, 10, 10, 11, 13, 17, 18, 18, 18, 20, 22, 25, 29, 33, 41, 44, 48, 55, 62, 70, 73, 85, 95, 102, 113, 116, 124, 136, 144, 157, 170, 176, 188, 202, 211, 229, 243, 253, 266, 272, 279, 283, 295, 300, 304, 308, 318, 323, 326, 328, 330, 334, 345, 346, 348, 348, 349, 351, 353, 354, 357, 359, 361, 363, 365, 365, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 0, 0, 0, 0, 0, 1, 2, 2, 5, 5, 5, 7, 7, 7, 7, 7, 12, 13, 13, 15, 17, 18, 20, 21, 27, 32, 34, 36, 41, 44, 51, 53, 61, 68, 72, 78, 80, 87, 98, 108, 110, 118, 130, 138, 155, 164, 182, 193, 211, 231, 259, 284, 297, 310, 329, 354, 382, 402, 412, 446, 474, 502, 538, 572, 602, 661, 690, 705, 730, 789, 823, 849, 891, 988, 1028, 1075, 1103, 1131, 1184, 1221, 1242, 1259, 1281, 1303, 1366, 1422, 1452, 1486, 1521, 1578, 1626, 1666, 1692, 1730, 1765, 1806, 1830, 1854, 1877, 1907], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.37728684846392, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.22195374525371, 81.18743527787366, 81.1356575768036, 81.10113910942354, 81.10113910942354, 81.04936140835348, 81.01484294097342, 80.96306523990336, 80.94580600621333, 80.84225060407319, 80.75595443562305, 80.73869520193303, 80.68691750086296, 80.58336209872282, 80.4970659302727, 80.39351052813255, 80.39351052813255, 80.30721435968243, 80.16914049016223, 80.06558508802209, 80.08284432171212, 80.06558508802209, 80.01380738695202, 79.87573351743183, 79.72040041422161, 79.72040041422161, 79.70314118053159, 79.59958577839144, 79.3579565067311, 79.37521574042113, 79.25440110459095, 79.13358646876078, 79.01277183293061, 78.78840179496031, 78.70210562651019, 78.46047635484985, 78.28788401794961, 78.16706938211944, 78.08077321366932, 77.856403175699, 77.71832930617882, 77.56299620296859, 77.45944080082845, 77.39040386606834, 77.06247842595789, 76.87262685536763, 76.78633068691751, 76.51018294787711, 76.40662754573697, 76.26855367621678, 76.04418363824647, 76.00966517086641, 75.75077666551606, 75.44011045909562, 75.14670348636521, 74.88781498101484, 74.59440800828443, 74.2147048671039, 73.95581636175355, 73.558853986883, 73.16189161201243, 72.5750776665516, 72.04004142216085, 71.52226441146013, 70.9181912323093, 70.26234035208837, 69.76182257507767, 69.19226786330687, 68.6399723852261, 68.0704176734553, 67.53538142906454, 66.89678978253366, 66.48256817397308, 65.89575422851226, 65.2916810493614, 64.54953400069037, 63.7901277183293, 62.96168450120815, 62.271315153607176, 61.615464273386266, 60.94235415947532, 60.14842940973422, 59.33724542630308, 58.54332067656196, 57.66309975837073, 56.71384190541941, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.24926992309544, "threshold_area100": 74.78339661719019, "threshold_area25": 81.30134622022783}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.00484488345682621, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0057849702425301075, "operator": "ge"}, {"budget": 0.03, "threshold": -0.007105870638042688, "operator": "ge"}, {"budget": 0.04, "threshold": -0.008768989704549313, "operator": "ge"}, {"budget": 0.05, "threshold": -0.009706906042993069, "operator": "ge"}, {"budget": 0.06, "threshold": -0.011281123384833336, "operator": "ge"}, {"budget": 0.07, "threshold": -0.013108297251164913, "operator": "ge"}, {"budget": 0.08, "threshold": -0.016025936231017113, "operator": "ge"}, {"budget": 0.09, "threshold": -0.017921993508934975, "operator": "ge"}, {"budget": 0.1, "threshold": -0.019253280013799667, "operator": "ge"}, {"budget": 0.11, "threshold": -0.02129305712878704, "operator": "ge"}, {"budget": 0.12, "threshold": -0.02427445724606514, "operator": "ge"}, {"budget": 0.13, "threshold": -0.027457796037197113, "operator": "ge"}, {"budget": 0.14, "threshold": -0.029601730406284332, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03189365565776825, "operator": "ge"}, {"budget": 0.16, "threshold": -0.034652695059776306, "operator": "ge"}, {"budget": 0.17, "threshold": -0.037587687373161316, "operator": "ge"}, {"budget": 0.18, "threshold": -0.040461376309394836, "operator": "ge"}, {"budget": 0.19, "threshold": -0.04135923087596893, "operator": "ge"}, {"budget": 0.2, "threshold": -0.04423381760716438, "operator": "ge"}, {"budget": 0.21, "threshold": -0.04853860288858414, "operator": "ge"}, {"budget": 0.22, "threshold": -0.0519893653690815, "operator": "ge"}, {"budget": 0.23, "threshold": -0.05492440611124039, "operator": "ge"}, {"budget": 0.24, "threshold": -0.06024377420544624, "operator": "ge"}, {"budget": 0.25, "threshold": -0.06611049920320511, "operator": "ge"}, {"budget": 0.26, "threshold": -0.07411949336528778, "operator": "ge"}, {"budget": 0.27, "threshold": -0.07761380821466446, "operator": "ge"}, {"budget": 0.28, "threshold": -0.08151666074991226, "operator": "ge"}, {"budget": 0.29, "threshold": -0.08963171392679214, "operator": "ge"}, {"budget": 0.3, "threshold": -0.09437853842973709, "operator": "ge"}, {"budget": 0.31, "threshold": -0.10201043635606766, "operator": "ge"}, {"budget": 0.32, "threshold": -0.11165223270654678, "operator": "ge"}, {"budget": 0.33, "threshold": -0.12220855802297592, "operator": "ge"}, {"budget": 0.34, "threshold": -0.12643109261989594, "operator": "ge"}, {"budget": 0.35, "threshold": -0.12994574010372162, "operator": "ge"}, {"budget": 0.36, "threshold": -0.13971573114395142, "operator": "ge"}, {"budget": 0.37, "threshold": -0.14865601062774658, "operator": "ge"}, {"budget": 0.38, "threshold": -0.1572764366865158, "operator": "ge"}, {"budget": 0.39, "threshold": -0.16465327143669128, "operator": "ge"}, {"budget": 0.4, "threshold": -0.17279736697673798, "operator": "ge"}, {"budget": 0.41, "threshold": -0.18164198100566864, "operator": "ge"}, {"budget": 0.42, "threshold": -0.194500133395195, "operator": "ge"}, {"budget": 0.43, "threshold": -0.20701003074645996, "operator": "ge"}, {"budget": 0.44, "threshold": -0.217840313911438, "operator": "ge"}, {"budget": 0.45, "threshold": -0.2271958589553833, "operator": "ge"}, {"budget": 0.46, "threshold": -0.23724772036075592, "operator": "ge"}, {"budget": 0.47, "threshold": -0.2519002854824066, "operator": "ge"}, {"budget": 0.48, "threshold": -0.2674724757671356, "operator": "ge"}, {"budget": 0.49, "threshold": -0.2860160171985626, "operator": "ge"}, {"budget": 0.5, "threshold": -0.2983209788799286, "operator": "ge"}, {"budget": 0.51, "threshold": -0.31001466512680054, "operator": "ge"}, {"budget": 0.52, "threshold": -0.3251665234565735, "operator": "ge"}, {"budget": 0.53, "threshold": -0.3417123258113861, "operator": "ge"}, {"budget": 0.54, "threshold": -0.35106685757637024, "operator": "ge"}, {"budget": 0.55, "threshold": -0.3607790172100067, "operator": "ge"}, {"budget": 0.56, "threshold": -0.374600887298584, "operator": "ge"}, {"budget": 0.57, "threshold": -0.38562119007110596, "operator": "ge"}, {"budget": 0.58, "threshold": -0.39844197034835815, "operator": "ge"}, {"budget": 0.59, "threshold": -0.40476149320602417, "operator": "ge"}, {"budget": 0.6, "threshold": -0.42692241072654724, "operator": "ge"}, {"budget": 0.61, "threshold": -0.44713103771209717, "operator": "ge"}, {"budget": 0.62, "threshold": -0.46950608491897583, "operator": "ge"}, {"budget": 0.63, "threshold": -0.4834611415863037, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5009469389915466, "operator": "ge"}, {"budget": 0.65, "threshold": -0.5169509649276733, "operator": "ge"}, {"budget": 0.66, "threshold": -0.538577675819397, "operator": "ge"}, {"budget": 0.67, "threshold": -0.5515667796134949, "operator": "ge"}, {"budget": 0.68, "threshold": -0.5663758516311646, "operator": "ge"}, {"budget": 0.69, "threshold": -0.5761628150939941, "operator": "ge"}, {"budget": 0.7, "threshold": -0.589005708694458, "operator": "ge"}, {"budget": 0.71, "threshold": -0.599772572517395, "operator": "ge"}, {"budget": 0.72, "threshold": -0.6243270635604858, "operator": "ge"}, {"budget": 0.73, "threshold": -0.6451343894004822, "operator": "ge"}, {"budget": 0.74, "threshold": -0.6642482280731201, "operator": "ge"}, {"budget": 0.75, "threshold": -0.6808695793151855, "operator": "ge"}, {"budget": 0.76, "threshold": -0.6937553882598877, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7044099569320679, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7238258123397827, "operator": "ge"}, {"budget": 0.79, "threshold": -0.7452823519706726, "operator": "ge"}, {"budget": 0.8, "threshold": -0.753324568271637, "operator": "ge"}, {"budget": 0.81, "threshold": -0.7656300663948059, "operator": "ge"}, {"budget": 0.82, "threshold": -0.7712882161140442, "operator": "ge"}, {"budget": 0.83, "threshold": -0.779669463634491, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8020404577255249, "operator": "ge"}, {"budget": 0.85, "threshold": -0.816946804523468, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8321462273597717, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8452866077423096, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8582567572593689, "operator": "ge"}, {"budget": 0.89, "threshold": -0.8719387054443359, "operator": "ge"}, {"budget": 0.9, "threshold": -0.8889212012290955, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9001999497413635, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9122510552406311, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9206701517105103, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9343255758285522, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9431020021438599, "operator": "ge"}, {"budget": 0.96, "threshold": -0.952808141708374, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9578447937965393, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9643288254737854, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9700267910957336, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.41666666666666, 97.08333333333333, 96.83333333333334, 96.58333333333333, 96.33333333333334, 96.16666666666667, 95.83333333333334, 95.5, 95.41666666666667, 95.08333333333333, 95.08333333333333, 94.75, 94.5, 94.16666666666667, 93.58333333333333, 93.33333333333333, 93.16666666666666, 92.91666666666667, 92.5, 92.25, 92.08333333333334, 91.66666666666666, 91.25, 90.83333333333333, 90.33333333333333, 90.08333333333334, 89.5, 88.83333333333333, 88.25, 87.66666666666667, 87.16666666666667, 86.66666666666667, 85.91666666666666, 85.33333333333333, 84.91666666666666, 84.5, 83.66666666666667, 83.0, 82.25, 81.58333333333333, 80.75, 79.91666666666667, 79.33333333333333, 78.58333333333334, 77.75, 77.0, 76.25, 75.5, 74.75, 73.91666666666666, 73.25, 72.25, 71.50000000000001, 70.75, 69.83333333333334, 68.91666666666667, 68.33333333333333, 67.41666666666667, 66.41666666666667, 65.5, 64.58333333333333, 63.66666666666667, 62.66666666666667, 61.66666666666667, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 57.00000000000001, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 3, 4, 6, 7, 7, 8, 10, 11, 11, 11, 11, 12, 13, 14, 18, 21, 24, 27, 29, 33, 37, 38, 42, 42, 46, 49, 53, 60, 63, 66, 69, 74, 77, 79, 84, 89, 94, 100, 103, 110, 118, 125, 132, 138, 144, 153, 161, 166, 171, 181, 189, 198, 206, 216, 226, 233, 242, 252, 261, 270, 279, 288, 298, 306, 318, 327, 336, 347, 358, 365, 376, 388, 399, 410, 421, 433, 445, 455, 467, 479, 491, 501, 512, 524], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.41666666666666, 97.08333333333333, 96.83333333333334, 96.58333333333333, 96.33333333333334, 96.16666666666667, 95.83333333333334, 95.5, 95.41666666666667, 95.08333333333333, 95.08333333333333, 94.66666666666667, 94.5, 94.16666666666667, 93.58333333333333, 93.33333333333333, 93.16666666666666, 92.91666666666667, 92.5, 92.25, 92.08333333333334, 91.66666666666666, 91.25, 90.83333333333333, 90.33333333333333, 90.08333333333334, 89.5, 88.83333333333333, 88.25, 87.66666666666667, 87.16666666666667, 86.66666666666667, 85.91666666666666, 85.33333333333333, 84.91666666666666, 84.5, 83.66666666666667, 83.0, 82.25, 81.5, 80.75, 79.91666666666667, 79.33333333333333, 78.58333333333334, 77.75, 77.0, 76.25, 75.5, 74.75, 73.91666666666666, 73.25, 72.25, 71.41666666666666, 70.75, 69.83333333333334, 68.91666666666667, 68.33333333333333, 67.41666666666667, 66.41666666666667, 65.5, 64.58333333333333, 63.66666666666667, 62.66666666666667, 61.66666666666667, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 57.00000000000001, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.39833333333333, "threshold_area100": 86.40083333333332, "threshold_area25": 98.32499999999999}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.34276838108389, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.23921297894374, 81.22195374525371, 81.22195374525371, 81.20469451156369, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.10113910942354, 81.01484294097342, 80.9285467725233, 80.94580600621333, 80.91128753883328, 80.89402830514325, 80.84225060407319, 80.77321366931308, 80.65239903348291, 80.58336209872282, 80.56610286503279, 80.47980669658268, 80.39351052813255, 80.34173282706249, 80.2899551259924, 80.18639972385226, 80.06558508802209, 80.03106662064205, 79.97928891957199, 79.87573351743183, 79.89299275112185, 79.84121505005179, 79.75491888160165, 79.59958577839144, 79.40973420780118, 79.18536416983086, 79.08180876769072, 78.8574387297204, 78.58129099068002, 78.23610631687953, 78.20158784949949, 78.06351397997929, 77.75284777355886, 77.56299620296859, 77.44218156713842, 77.32136693130826, 77.01070072488781, 76.71729375215742, 76.30307214359682, 76.11322057300656, 75.7852951328961, 75.62996202968588, 75.14670348636521, 74.88781498101484, 74.50811183983431, 74.18018639972385, 73.80048325854332, 73.40352088367277, 72.59233690024163, 71.97100448740076, 71.2806351397998, 70.4176734552986, 69.96893337935795, 69.53745253710736, 68.3638246461857, 67.3627890921643, 66.91404901622369, 66.43079047290301, 66.2927166033828, 65.9993096306524, 64.87745944080083, 64.44597859855023, 63.70383154987918, 63.08249913703832, 62.56472212633759, 61.90887124611668, 60.94235415947532, 60.18294787711426, 59.285467725233, 58.7849499482223, 57.93924749741112, 57.33517431826027, 56.541249568519156, 56.16154642733863, 55.79910251984812, 55.29858474283742, 54.81532619951675], "calls": [0.0, 0.6040731791508458, 1.2944425267518123, 2.0538488091128753, 2.968588194684156, 3.40006903693476, 4.159475319295823, 4.832585433206766, 5.954435623058337, 6.662064204349328, 7.197100448740076, 7.870210562651018, 8.698653779772178, 9.751467034863651, 10.32102174663445, 10.959613393165343, 11.684501208146358, 12.288574387297203, 12.909906800138074, 12.996202968588195, 13.60027614773904, 14.376941663790127, 15.015533310321022, 15.429754918881603, 16.430790472903002, 17.552640662754573, 19.088712461166722, 19.537452537107352, 20.055229547808075, 21.1425612702796, 21.729375215740422, 22.695892302381772, 23.869520193303416, 25.112185018985155, 25.52640662754574, 26.130479806696584, 27.20055229547808, 28.132550914739383, 28.97825336555057, 29.66862271315154, 30.44528822920262, 31.256472212633756, 32.75802554366586, 34.121505005177774, 35.001725923369, 35.761132205730064, 36.7103900586814, 38.280980324473596, 39.66171901967553, 41.30134622022782, 42.198826372109075, 43.320676561960646, 44.83948912668277, 46.44459785985502, 47.221263375906105, 48.22229892992751, 49.51674145667933, 50.69036934760097, 52.03658957542285, 52.55436658612358, 54.88436313427684, 56.71384190541941, 58.90576458405247, 60.217466344494305, 62.08146358301691, 63.738350017259236, 65.68864342423197, 66.77597514670349, 68.12219537452538, 69.0024162927166, 70.0724887814981, 70.90093199861927, 72.83396617190196, 74.35277873662409, 75.88885053503624, 77.30410769761824, 78.16706938211944, 79.08180876769072, 80.96306523990336, 82.9306178805661, 83.67276492923715, 84.63928201587849, 85.001725923369, 85.60579910251985, 87.12461166724196, 88.03935105281326, 88.95409043838454, 89.7134967207456, 90.42112530203659, 91.37038315498792, 92.57852951328961, 93.68312046945115, 94.87400759406283, 95.51259924059372, 96.63444943044529, 97.3593372454263, 98.18778046254747, 98.567483603728, 98.96444597859855, 99.4994822229893, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 5, 6, 9, 9, 11, 11, 13, 13, 13, 16, 18, 20, 20, 23, 23, 28, 34, 39, 41, 44, 53, 59, 68, 72, 75, 81, 87, 95, 100, 106, 113, 127, 130, 145, 157, 176, 186, 205, 224, 237, 247, 255, 260, 267, 270, 278, 289, 300, 305, 308, 312, 317, 328, 330, 334, 335, 338, 342, 350, 350, 351, 353, 355, 357, 360, 361, 363, 365, 365, 365, 365, 365, 365, 365], "harmed": [0, 0, 0, 0, 0, 0, 1, 2, 2, 5, 5, 5, 7, 7, 7, 7, 7, 12, 13, 13, 14, 17, 18, 20, 20, 25, 32, 33, 36, 40, 43, 49, 56, 62, 63, 68, 76, 81, 86, 92, 102, 104, 112, 124, 128, 133, 141, 159, 176, 198, 208, 224, 246, 272, 282, 295, 319, 337, 358, 368, 401, 430, 473, 494, 532, 560, 601, 626, 656, 680, 709, 735, 790, 837, 888, 943, 972, 1001, 1074, 1143, 1171, 1203, 1212, 1232, 1301, 1334, 1377, 1414, 1446, 1486, 1544, 1591, 1644, 1675, 1726, 1761, 1807, 1829, 1850, 1879, 1907], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.37728684846392, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.22195374525371, 81.18743527787366, 81.1356575768036, 81.10113910942354, 81.04936140835348, 81.01484294097342, 80.94580600621333, 80.91128753883328, 80.91128753883328, 80.85950983776321, 80.75595443562305, 80.65239903348291, 80.58336209872282, 80.53158439765275, 80.41076976182258, 80.35899206075251, 80.2899551259924, 80.15188125647221, 80.04832585433206, 80.01380738695202, 80.01380738695202, 79.91025198481188, 79.89299275112185, 79.84121505005179, 79.7894373489817, 79.65136347946151, 79.51328960994132, 79.30617880566103, 79.23714187090093, 79.09906800138074, 78.89195719710045, 78.80566102865033, 78.54677252329996, 78.33966171901969, 78.23610631687953, 78.14981014842941, 77.856403175699, 77.71832930617882, 77.47670003451846, 77.44218156713842, 77.28684846392821, 77.18329306178806, 77.01070072488781, 76.88988608905764, 76.70003451846739, 76.45840524680705, 76.28581290990681, 76.13047980669658, 75.87159130134621, 75.76803589920608, 75.68173973075596, 75.56092509492579, 75.19848118743528, 75.09492578529513, 74.81877804625475, 74.54263030721437, 74.18018639972385, 73.85226095961339, 73.36900241629272, 72.93752157404212, 72.54055919917157, 72.10907835692095, 71.67759751467035, 71.21159820503969, 70.65930272695893, 70.02071108042803, 69.58923023817742, 68.93337935795651, 68.32930617880567, 67.94960303762512, 67.34552985847428, 66.77597514670349, 66.30997583707283, 65.75768035899206, 64.94649637556093, 64.46323783224025, 63.66931308249914, 62.87538833275803, 62.16775975146703, 61.391094235415956, 60.63168795305489, 59.94131860545392, 59.18191232309285, 58.40524680704176, 57.61132205730066, 56.71384190541941, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.03972542983952, "threshold_area100": 74.97428374180186, "threshold_area25": 81.30134622022783}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.5535104274749756, "operator": "ge"}, {"budget": 0.02, "threshold": -2.844651222229004, "operator": "ge"}, {"budget": 0.03, "threshold": -3.1594314575195312, "operator": "ge"}, {"budget": 0.04, "threshold": -3.4550952911376953, "operator": "ge"}, {"budget": 0.05, "threshold": -3.7349095344543457, "operator": "ge"}, {"budget": 0.06, "threshold": -3.938109874725342, "operator": "ge"}, {"budget": 0.07, "threshold": -4.196445465087891, "operator": "ge"}, {"budget": 0.08, "threshold": -4.402026176452637, "operator": "ge"}, {"budget": 0.09, "threshold": -4.563004970550537, "operator": "ge"}, {"budget": 0.1, "threshold": -4.672544479370117, "operator": "ge"}, {"budget": 0.11, "threshold": -4.8606061935424805, "operator": "ge"}, {"budget": 0.12, "threshold": -5.028955459594727, "operator": "ge"}, {"budget": 0.13, "threshold": -5.409761905670166, "operator": "ge"}, {"budget": 0.14, "threshold": -5.557253837585449, "operator": "ge"}, {"budget": 0.15, "threshold": -5.809474945068359, "operator": "ge"}, {"budget": 0.16, "threshold": -5.912836074829102, "operator": "ge"}, {"budget": 0.17, "threshold": -6.177219390869141, "operator": "ge"}, {"budget": 0.18, "threshold": -6.31450080871582, "operator": "ge"}, {"budget": 0.19, "threshold": -6.516612529754639, "operator": "ge"}, {"budget": 0.2, "threshold": -6.716794490814209, "operator": "ge"}, {"budget": 0.21, "threshold": -6.876352787017822, "operator": "ge"}, {"budget": 0.22, "threshold": -7.047280311584473, "operator": "ge"}, {"budget": 0.23, "threshold": -7.2339653968811035, "operator": "ge"}, {"budget": 0.24, "threshold": -7.347946643829346, "operator": "ge"}, {"budget": 0.25, "threshold": -7.457171440124512, "operator": "ge"}, {"budget": 0.26, "threshold": -7.558712482452393, "operator": "ge"}, {"budget": 0.27, "threshold": -7.6998372077941895, "operator": "ge"}, {"budget": 0.28, "threshold": -7.811957836151123, "operator": "ge"}, {"budget": 0.29, "threshold": -7.955146312713623, "operator": "ge"}, {"budget": 0.3, "threshold": -8.083540916442871, "operator": "ge"}, {"budget": 0.31, "threshold": -8.161201477050781, "operator": "ge"}, {"budget": 0.32, "threshold": -8.218658447265625, "operator": "ge"}, {"budget": 0.33, "threshold": -8.291160583496094, "operator": "ge"}, {"budget": 0.34, "threshold": -8.36865520477295, "operator": "ge"}, {"budget": 0.35, "threshold": -8.41447925567627, "operator": "ge"}, {"budget": 0.36, "threshold": -8.472776412963867, "operator": "ge"}, {"budget": 0.37, "threshold": -8.51130485534668, "operator": "ge"}, {"budget": 0.38, "threshold": -8.56041431427002, "operator": "ge"}, {"budget": 0.39, "threshold": -8.609442710876465, "operator": "ge"}, {"budget": 0.4, "threshold": -8.660689353942871, "operator": "ge"}, {"budget": 0.41, "threshold": -8.702646255493164, "operator": "ge"}, {"budget": 0.42, "threshold": -8.771341323852539, "operator": "ge"}, {"budget": 0.43, "threshold": -8.828548431396484, "operator": "ge"}, {"budget": 0.44, "threshold": -8.868492126464844, "operator": "ge"}, {"budget": 0.45, "threshold": -8.913352012634277, "operator": "ge"}, {"budget": 0.46, "threshold": -8.962907791137695, "operator": "ge"}, {"budget": 0.47, "threshold": -8.998760223388672, "operator": "ge"}, {"budget": 0.48, "threshold": -9.040860176086426, "operator": "ge"}, {"budget": 0.49, "threshold": -9.070505142211914, "operator": "ge"}, {"budget": 0.5, "threshold": -9.108476638793945, "operator": "ge"}, {"budget": 0.51, "threshold": -9.144754409790039, "operator": "ge"}, {"budget": 0.52, "threshold": -9.17724609375, "operator": "ge"}, {"budget": 0.53, "threshold": -9.213408470153809, "operator": "ge"}, {"budget": 0.54, "threshold": -9.252106666564941, "operator": "ge"}, {"budget": 0.55, "threshold": -9.27672004699707, "operator": "ge"}, {"budget": 0.56, "threshold": -9.318241119384766, "operator": "ge"}, {"budget": 0.57, "threshold": -9.355086326599121, "operator": "ge"}, {"budget": 0.58, "threshold": -9.397909164428711, "operator": "ge"}, {"budget": 0.59, "threshold": -9.441299438476562, "operator": "ge"}, {"budget": 0.6, "threshold": -9.49752426147461, "operator": "ge"}, {"budget": 0.61, "threshold": -9.527054786682129, "operator": "ge"}, {"budget": 0.62, "threshold": -9.552421569824219, "operator": "ge"}, {"budget": 0.63, "threshold": -9.590063095092773, "operator": "ge"}, {"budget": 0.64, "threshold": -9.644657135009766, "operator": "ge"}, {"budget": 0.65, "threshold": -9.702713012695312, "operator": "ge"}, {"budget": 0.66, "threshold": -9.74020004272461, "operator": "ge"}, {"budget": 0.67, "threshold": -9.76872444152832, "operator": "ge"}, {"budget": 0.68, "threshold": -9.814970016479492, "operator": "ge"}, {"budget": 0.69, "threshold": -9.849991798400879, "operator": "ge"}, {"budget": 0.7, "threshold": -9.894075393676758, "operator": "ge"}, {"budget": 0.71, "threshold": -9.964404106140137, "operator": "ge"}, {"budget": 0.72, "threshold": -9.998430252075195, "operator": "ge"}, {"budget": 0.73, "threshold": -10.057966232299805, "operator": "ge"}, {"budget": 0.74, "threshold": -10.082956314086914, "operator": "ge"}, {"budget": 0.75, "threshold": -10.114130020141602, "operator": "ge"}, {"budget": 0.76, "threshold": -10.176555633544922, "operator": "ge"}, {"budget": 0.77, "threshold": -10.211313247680664, "operator": "ge"}, {"budget": 0.78, "threshold": -10.250251770019531, "operator": "ge"}, {"budget": 0.79, "threshold": -10.28640079498291, "operator": "ge"}, {"budget": 0.8, "threshold": -10.367321014404297, "operator": "ge"}, {"budget": 0.81, "threshold": -10.44143295288086, "operator": "ge"}, {"budget": 0.82, "threshold": -10.482091903686523, "operator": "ge"}, {"budget": 0.83, "threshold": -10.544130325317383, "operator": "ge"}, {"budget": 0.84, "threshold": -10.595067977905273, "operator": "ge"}, {"budget": 0.85, "threshold": -10.642817497253418, "operator": "ge"}, {"budget": 0.86, "threshold": -10.6981201171875, "operator": "ge"}, {"budget": 0.87, "threshold": -10.776141166687012, "operator": "ge"}, {"budget": 0.88, "threshold": -10.836716651916504, "operator": "ge"}, {"budget": 0.89, "threshold": -10.919562339782715, "operator": "ge"}, {"budget": 0.9, "threshold": -10.999213218688965, "operator": "ge"}, {"budget": 0.91, "threshold": -11.06033706665039, "operator": "ge"}, {"budget": 0.92, "threshold": -11.137327194213867, "operator": "ge"}, {"budget": 0.93, "threshold": -11.233599662780762, "operator": "ge"}, {"budget": 0.94, "threshold": -11.341131210327148, "operator": "ge"}, {"budget": 0.95, "threshold": -11.428512573242188, "operator": "ge"}, {"budget": 0.96, "threshold": -11.592392921447754, "operator": "ge"}, {"budget": 0.97, "threshold": -11.74605941772461, "operator": "ge"}, {"budget": 0.98, "threshold": -11.923178672790527, "operator": "ge"}, {"budget": 0.99, "threshold": -12.175588607788086, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.16666666666667, 97.58333333333333, 97.08333333333333, 96.5, 96.08333333333333, 95.83333333333334, 95.33333333333334, 94.75, 94.25, 93.66666666666667, 93.16666666666666, 92.83333333333333, 91.91666666666667, 91.33333333333333, 90.83333333333333, 90.25, 89.83333333333333, 89.0, 88.66666666666667, 88.08333333333334, 87.58333333333333, 87.08333333333333, 86.91666666666666, 86.5, 86.0, 85.58333333333333, 85.25, 84.83333333333334, 84.66666666666667, 84.41666666666667, 84.25, 84.0, 83.5, 83.16666666666667, 82.58333333333333, 82.5, 82.33333333333334, 82.25, 82.08333333333333, 81.83333333333334, 81.75, 81.33333333333333, 81.33333333333333, 81.16666666666667, 81.0, 80.66666666666666, 80.5, 80.25, 80.0, 79.66666666666666, 79.25, 78.83333333333333, 78.58333333333334, 78.41666666666667, 77.91666666666667, 77.66666666666666, 77.58333333333334, 77.16666666666667, 76.66666666666666, 76.5, 76.16666666666667, 75.91666666666667, 75.66666666666667, 75.16666666666667, 74.83333333333333, 74.25, 74.0, 73.66666666666667, 73.33333333333334, 72.91666666666667, 72.41666666666666, 71.91666666666667, 71.50000000000001, 71.16666666666667, 70.58333333333333, 70.08333333333334, 69.58333333333333, 69.08333333333334, 68.66666666666667, 68.16666666666666, 67.58333333333333, 66.83333333333333, 66.33333333333333, 65.83333333333333, 65.25000000000001, 64.83333333333333, 64.08333333333334, 63.5, 62.83333333333334, 62.416666666666664, 61.83333333333334, 60.916666666666664, 60.25, 59.833333333333336, 59.166666666666664, 58.33333333333333, 57.666666666666664, 56.833333333333336, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 5, 12, 18, 25, 31, 34, 40, 47, 54, 61, 67, 71, 82, 89, 95, 102, 107, 117, 121, 128, 134, 140, 142, 147, 153, 158, 162, 167, 169, 172, 174, 177, 183, 187, 194, 195, 197, 198, 200, 203, 204, 209, 209, 211, 213, 217, 219, 222, 225, 229, 234, 239, 242, 244, 250, 253, 254, 259, 265, 267, 271, 274, 277, 283, 287, 294, 297, 301, 305, 310, 316, 322, 327, 331, 338, 344, 350, 356, 361, 367, 374, 383, 389, 395, 402, 407, 416, 423, 431, 436, 443, 454, 462, 467, 475, 485, 493, 503, 513, 524], "exact_score": [98.58333333333333, 98.16666666666667, 97.58333333333333, 97.08333333333333, 96.5, 96.08333333333333, 95.83333333333334, 95.33333333333334, 94.75, 94.25, 93.66666666666667, 93.16666666666666, 92.83333333333333, 91.91666666666667, 91.33333333333333, 90.83333333333333, 90.25, 89.83333333333333, 89.0, 88.66666666666667, 88.08333333333334, 87.58333333333333, 87.08333333333333, 86.91666666666666, 86.5, 86.0, 85.58333333333333, 85.25, 84.83333333333334, 84.66666666666667, 84.41666666666667, 84.25, 84.0, 83.5, 83.16666666666667, 82.58333333333333, 82.5, 82.33333333333334, 82.25, 82.08333333333333, 81.83333333333334, 81.75, 81.33333333333333, 81.33333333333333, 81.16666666666667, 81.0, 80.66666666666666, 80.5, 80.25, 80.0, 79.66666666666666, 79.25, 78.83333333333333, 78.58333333333334, 78.41666666666667, 77.91666666666667, 77.66666666666666, 77.58333333333334, 77.16666666666667, 76.66666666666666, 76.5, 76.16666666666667, 75.91666666666667, 75.66666666666667, 75.16666666666667, 74.83333333333333, 74.25, 74.0, 73.66666666666667, 73.25, 72.91666666666667, 72.41666666666666, 71.91666666666667, 71.50000000000001, 71.16666666666667, 70.58333333333333, 70.08333333333334, 69.58333333333333, 69.08333333333334, 68.66666666666667, 68.16666666666666, 67.58333333333333, 66.75, 66.33333333333333, 65.83333333333333, 65.25000000000001, 64.83333333333333, 64.08333333333334, 63.5, 62.83333333333334, 62.416666666666664, 61.83333333333334, 60.916666666666664, 60.25, 59.833333333333336, 59.166666666666664, 58.33333333333333, 57.666666666666664, 56.833333333333336, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 78.44833333333332, "threshold_area100": 78.45, "threshold_area25": 92.19166666666668}, "test": {"score": [81.429064549534, 81.23921297894374, 81.1356575768036, 80.89402830514325, 80.35899206075251, 79.37521574042113, 78.94373489817053, 77.97721781152917, 77.56299620296859, 76.76907145322748, 76.423886779427, 75.71625819813602, 75.00862961684501, 73.31722471522265, 72.73041076976182, 71.95374525371074, 71.69485674836038, 70.74559889540905, 70.33137728684846, 69.91715567828788, 69.52019330341733, 69.24404556437695, 68.88160165688643, 68.41560234725578, 68.1394546082154, 68.0013807386952, 67.79426993441491, 67.5698998964446, 67.41456679323439, 67.19019675526407, 67.01760441836383, 66.91404901622369, 66.77597514670349, 66.67241974456334, 66.58612357611322, 66.48256817397308, 66.32723507076285, 66.27545736969279, 66.2064204349327, 66.08560579910252, 65.9302726958923, 65.80945806006213, 65.67138419054194, 65.58508802209182, 65.55056955471177, 65.34345875043148, 65.20538488091128, 65.15360717984122, 64.96375560925095, 64.91197790818089, 64.84294097342078, 64.7911632723507, 64.67034863652054, 64.4977562996203, 64.34242319641008, 64.22160856057991, 64.13531239212979, 64.01449775629962, 63.75560925094926, 63.600276147739045, 63.393165343458755, 63.2896099413186, 63.22057300655851, 63.06523990334829, 62.87538833275803, 62.73731446323784, 62.59924059371764, 62.51294442526751, 62.37487055574733, 62.271315153607176, 62.09872281670694, 61.822575077666556, 61.66724197445633, 61.391094235415956, 61.2875388332758, 61.14946496375562, 60.94235415947532, 60.76976182257508, 60.68346565412496, 60.528132550914734, 60.11391094235417, 59.90680013807387, 59.75146703486365, 59.49257852951328, 59.26820849154297, 59.078356920952714, 58.83672764929238, 58.38798757335175, 58.19813600276148, 57.66309975837073, 57.38695201933035, 57.19710044874008, 56.93821194338972, 56.80013807386952, 56.50673110113911, 56.316879530548846, 55.95443562305834, 55.74732481877804, 55.47117707973766, 55.229547808077314, 54.81532619951675], "calls": [0.0, 2.847773558853987, 5.315843976527442, 9.112875388332759, 14.118053158439764, 19.330341732827065, 23.438039351052815, 28.356920952709704, 31.774249223334483, 34.41491197790818, 35.968243010010355, 38.93683120469451, 41.09423541594753, 45.63341387642389, 47.15222644114601, 49.395926820849155, 50.25888850535036, 52.39903348291336, 53.279254401104595, 54.504659993096304, 55.695547117707974, 56.50673110113911, 57.50776665516051, 58.56057991025199, 59.302726958923024, 59.88954090438384, 60.57991025198481, 61.4946496375561, 62.133241284086985, 63.4967207455989, 64.58405246807042, 65.2916810493614, 65.8612357611322, 66.430790472903, 67.17293752157404, 67.74249223334485, 68.2084915429755, 68.67449085260614, 69.22678633068692, 69.7790818087677, 70.41767345529858, 71.02174663444943, 71.86744908526062, 72.52329996548154, 72.93752157404211, 73.43803935105281, 74.04211253020367, 74.52537107352434, 75.0604073179151, 75.45736969278563, 75.85433206765619, 76.47566448049706, 76.9244045564377, 77.52847773558854, 77.89092164307905, 78.2188470831895, 78.65032792544011, 79.2198826372109, 79.8584742837418, 80.41076976182258, 81.17017604418363, 81.60165688643424, 81.91232309285468, 82.39558163617535, 83.12046945115637, 83.72454263030721, 84.22506040731793, 84.5012081463583, 85.03624439074905, 85.5712806351398, 86.17535381429065, 86.96927856403175, 87.41801863997239, 88.0738695201933, 88.31549879185364, 88.60890576458405, 89.16120124266482, 89.55816361753539, 89.88608905764585, 90.24853296513635, 91.11149464963756, 91.66379012771833, 92.09527096996894, 92.7856403175699, 93.19986192613048, 93.64860200207112, 93.99378667587159, 94.64963755609251, 94.96030376251295, 95.84052468070418, 96.28926475664481, 96.66896789782534, 97.04867103900587, 97.34207801173628, 97.73904038660683, 98.01518812564723, 98.567483603728, 98.91266827752848, 99.29237141870901, 99.58577839143942, 100.0], "rescued": [0, 36, 62, 102, 144, 183, 224, 257, 282, 289, 297, 318, 325, 338, 342, 348, 349, 352, 354, 355, 355, 356, 356, 357, 357, 357, 357, 357, 357, 357, 358, 358, 358, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 363, 363, 363, 363, 363, 363, 363, 364, 364, 364, 364, 364, 364, 365, 365, 365, 365, 365], "harmed": [0, 47, 79, 133, 206, 302, 368, 457, 506, 559, 587, 649, 697, 808, 846, 897, 913, 971, 997, 1022, 1045, 1062, 1083, 1111, 1127, 1135, 1147, 1160, 1169, 1182, 1193, 1199, 1207, 1214, 1220, 1226, 1235, 1238, 1242, 1249, 1258, 1265, 1273, 1278, 1280, 1292, 1300, 1303, 1314, 1317, 1321, 1325, 1332, 1342, 1351, 1358, 1363, 1370, 1385, 1394, 1406, 1412, 1416, 1425, 1436, 1444, 1452, 1457, 1465, 1471, 1481, 1497, 1506, 1522, 1528, 1536, 1548, 1558, 1563, 1572, 1596, 1608, 1617, 1634, 1647, 1658, 1672, 1698, 1709, 1740, 1757, 1768, 1783, 1791, 1808, 1819, 1841, 1853, 1869, 1883, 1907], "exact_score": [81.429064549534, 81.39454608215395, 81.41180531584398, 81.23921297894374, 81.22195374525371, 81.11839834311357, 80.96306523990336, 81.04936140835348, 80.96306523990336, 80.87676907145323, 80.87676907145323, 80.72143596824301, 80.53158439765275, 80.39351052813255, 80.35899206075251, 80.22091819123231, 79.96202968588196, 79.7894373489817, 79.54780807732136, 79.32343803935106, 79.3579565067311, 79.25440110459095, 79.11632723507076, 78.97825336555057, 78.82292026234036, 78.58129099068002, 78.3569209527097, 78.09803244735933, 78.01173627890921, 77.90818087676908, 77.83914394200899, 77.64929237141871, 77.47670003451846, 77.14877459440801, 76.88988608905764, 76.61373835001726, 76.40662754573697, 76.11322057300656, 76.04418363824647, 75.69899896444599, 75.35381429064549, 75.02588885053504, 74.7152226441146, 74.38729720400414, 73.93855712806352, 73.48981705212289, 73.1446323783224, 72.83396617190196, 72.54055919917157, 72.10907835692095, 71.76389368312047, 71.40144977562997, 70.93545046599931, 70.46945115636866, 70.12426648256817, 69.81360027614774, 69.46841560234725, 69.0714532274767, 68.58819468415602, 68.24301001035555, 67.98412150500518, 67.6389368312047, 67.44908526061442, 67.22471522264412, 67.08664135312392, 66.93130824991371, 66.7932343803935, 66.60338280980325, 66.37901277183293, 66.25819813600276, 66.01656886434243, 65.80945806006213, 65.65412495685192, 65.53331032102174, 65.20538488091128, 64.998274076631, 64.84294097342078, 64.63583016914049, 64.32516396272005, 64.0662754573697, 63.72109078356921, 63.44494304452882, 63.20331377286848, 62.892647566448055, 62.65101829478772, 62.392129789437355, 62.13324128408699, 61.80531584397653, 61.44287193648602, 60.994131860545394, 60.64894718674492, 60.200207110804286, 59.7859855022437, 59.371763893683124, 58.83672764929238, 58.18087676907145, 57.55954435623059, 56.955471177079744, 56.33413876423887, 55.73006558508802, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 71.21383082424072, "threshold_area100": 65.496548153262, "threshold_area25": 74.39765274421816}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.0112159252166748, "operator": "ge"}, {"budget": 0.02, "threshold": -1.235054612159729, "operator": "ge"}, {"budget": 0.03, "threshold": -1.465846300125122, "operator": "ge"}, {"budget": 0.04, "threshold": -1.641272783279419, "operator": "ge"}, {"budget": 0.05, "threshold": -1.8286017179489136, "operator": "ge"}, {"budget": 0.06, "threshold": -1.9725782871246338, "operator": "ge"}, {"budget": 0.07, "threshold": -2.0942301750183105, "operator": "ge"}, {"budget": 0.08, "threshold": -2.202746868133545, "operator": "ge"}, {"budget": 0.09, "threshold": -2.3623273372650146, "operator": "ge"}, {"budget": 0.1, "threshold": -2.5457193851470947, "operator": "ge"}, {"budget": 0.11, "threshold": -2.6371796131134033, "operator": "ge"}, {"budget": 0.12, "threshold": -2.845344066619873, "operator": "ge"}, {"budget": 0.13, "threshold": -3.0055484771728516, "operator": "ge"}, {"budget": 0.14, "threshold": -3.2264938354492188, "operator": "ge"}, {"budget": 0.15, "threshold": -3.3481011390686035, "operator": "ge"}, {"budget": 0.16, "threshold": -3.5316905975341797, "operator": "ge"}, {"budget": 0.17, "threshold": -3.761338949203491, "operator": "ge"}, {"budget": 0.18, "threshold": -3.902815818786621, "operator": "ge"}, {"budget": 0.19, "threshold": -4.053418159484863, "operator": "ge"}, {"budget": 0.2, "threshold": -4.146918296813965, "operator": "ge"}, {"budget": 0.21, "threshold": -4.297467231750488, "operator": "ge"}, {"budget": 0.22, "threshold": -4.522157669067383, "operator": "ge"}, {"budget": 0.23, "threshold": -4.706748962402344, "operator": "ge"}, {"budget": 0.24, "threshold": -4.898292541503906, "operator": "ge"}, {"budget": 0.25, "threshold": -5.0925726890563965, "operator": "ge"}, {"budget": 0.26, "threshold": -5.245798110961914, "operator": "ge"}, {"budget": 0.27, "threshold": -5.405848026275635, "operator": "ge"}, {"budget": 0.28, "threshold": -5.518589973449707, "operator": "ge"}, {"budget": 0.29, "threshold": -5.745026111602783, "operator": "ge"}, {"budget": 0.3, "threshold": -5.837724685668945, "operator": "ge"}, {"budget": 0.31, "threshold": -5.937538146972656, "operator": "ge"}, {"budget": 0.32, "threshold": -6.040367603302002, "operator": "ge"}, {"budget": 0.33, "threshold": -6.125903129577637, "operator": "ge"}, {"budget": 0.34, "threshold": -6.174678325653076, "operator": "ge"}, {"budget": 0.35, "threshold": -6.236018180847168, "operator": "ge"}, {"budget": 0.36, "threshold": -6.331932544708252, "operator": "ge"}, {"budget": 0.37, "threshold": -6.36304235458374, "operator": "ge"}, {"budget": 0.38, "threshold": -6.435866355895996, "operator": "ge"}, {"budget": 0.39, "threshold": -6.482751846313477, "operator": "ge"}, {"budget": 0.4, "threshold": -6.536414623260498, "operator": "ge"}, {"budget": 0.41, "threshold": -6.569835662841797, "operator": "ge"}, {"budget": 0.42, "threshold": -6.613265514373779, "operator": "ge"}, {"budget": 0.43, "threshold": -6.650545597076416, "operator": "ge"}, {"budget": 0.44, "threshold": -6.701668739318848, "operator": "ge"}, {"budget": 0.45, "threshold": -6.772107124328613, "operator": "ge"}, {"budget": 0.46, "threshold": -6.8072590827941895, "operator": "ge"}, {"budget": 0.47, "threshold": -6.864438056945801, "operator": "ge"}, {"budget": 0.48, "threshold": -6.91146993637085, "operator": "ge"}, {"budget": 0.49, "threshold": -6.94276237487793, "operator": "ge"}, {"budget": 0.5, "threshold": -6.982568264007568, "operator": "ge"}, {"budget": 0.51, "threshold": -7.0367112159729, "operator": "ge"}, {"budget": 0.52, "threshold": -7.076850891113281, "operator": "ge"}, {"budget": 0.53, "threshold": -7.097101211547852, "operator": "ge"}, {"budget": 0.54, "threshold": -7.129485130310059, "operator": "ge"}, {"budget": 0.55, "threshold": -7.154953956604004, "operator": "ge"}, {"budget": 0.56, "threshold": -7.196622371673584, "operator": "ge"}, {"budget": 0.57, "threshold": -7.235567569732666, "operator": "ge"}, {"budget": 0.58, "threshold": -7.267224311828613, "operator": "ge"}, {"budget": 0.59, "threshold": -7.305798530578613, "operator": "ge"}, {"budget": 0.6, "threshold": -7.347687721252441, "operator": "ge"}, {"budget": 0.61, "threshold": -7.370392799377441, "operator": "ge"}, {"budget": 0.62, "threshold": -7.4097723960876465, "operator": "ge"}, {"budget": 0.63, "threshold": -7.431776523590088, "operator": "ge"}, {"budget": 0.64, "threshold": -7.450066566467285, "operator": "ge"}, {"budget": 0.65, "threshold": -7.480756759643555, "operator": "ge"}, {"budget": 0.66, "threshold": -7.5238189697265625, "operator": "ge"}, {"budget": 0.67, "threshold": -7.538961410522461, "operator": "ge"}, {"budget": 0.68, "threshold": -7.566673755645752, "operator": "ge"}, {"budget": 0.69, "threshold": -7.5988993644714355, "operator": "ge"}, {"budget": 0.7, "threshold": -7.63292121887207, "operator": "ge"}, {"budget": 0.71, "threshold": -7.668366432189941, "operator": "ge"}, {"budget": 0.72, "threshold": -7.700061798095703, "operator": "ge"}, {"budget": 0.73, "threshold": -7.749792098999023, "operator": "ge"}, {"budget": 0.74, "threshold": -7.7703680992126465, "operator": "ge"}, {"budget": 0.75, "threshold": -7.797358512878418, "operator": "ge"}, {"budget": 0.76, "threshold": -7.81109094619751, "operator": "ge"}, {"budget": 0.77, "threshold": -7.844533920288086, "operator": "ge"}, {"budget": 0.78, "threshold": -7.864693641662598, "operator": "ge"}, {"budget": 0.79, "threshold": -7.889897346496582, "operator": "ge"}, {"budget": 0.8, "threshold": -7.934131145477295, "operator": "ge"}, {"budget": 0.81, "threshold": -7.95534086227417, "operator": "ge"}, {"budget": 0.82, "threshold": -7.995270729064941, "operator": "ge"}, {"budget": 0.83, "threshold": -8.05276107788086, "operator": "ge"}, {"budget": 0.84, "threshold": -8.088371276855469, "operator": "ge"}, {"budget": 0.85, "threshold": -8.150108337402344, "operator": "ge"}, {"budget": 0.86, "threshold": -8.207347869873047, "operator": "ge"}, {"budget": 0.87, "threshold": -8.243946075439453, "operator": "ge"}, {"budget": 0.88, "threshold": -8.292511940002441, "operator": "ge"}, {"budget": 0.89, "threshold": -8.325543403625488, "operator": "ge"}, {"budget": 0.9, "threshold": -8.378573417663574, "operator": "ge"}, {"budget": 0.91, "threshold": -8.439566612243652, "operator": "ge"}, {"budget": 0.92, "threshold": -8.49691390991211, "operator": "ge"}, {"budget": 0.93, "threshold": -8.596126556396484, "operator": "ge"}, {"budget": 0.94, "threshold": -8.669615745544434, "operator": "ge"}, {"budget": 0.95, "threshold": -8.742857933044434, "operator": "ge"}, {"budget": 0.96, "threshold": -8.852912902832031, "operator": "ge"}, {"budget": 0.97, "threshold": -8.972586631774902, "operator": "ge"}, {"budget": 0.98, "threshold": -9.088253021240234, "operator": "ge"}, {"budget": 0.99, "threshold": -9.391217231750488, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.58333333333333, 98.41666666666666, 97.75, 97.16666666666667, 96.75, 96.16666666666667, 95.58333333333333, 95.0, 94.58333333333333, 94.16666666666667, 93.83333333333333, 93.08333333333333, 92.41666666666667, 91.91666666666667, 91.25, 90.66666666666667, 90.08333333333334, 89.25, 88.83333333333333, 88.41666666666667, 87.83333333333333, 87.25, 86.83333333333334, 86.25, 85.66666666666667, 85.25, 84.91666666666666, 84.58333333333333, 84.0, 83.75, 83.5, 83.08333333333333, 83.0, 82.83333333333334, 82.58333333333333, 82.41666666666667, 82.25, 81.83333333333334, 81.66666666666667, 81.08333333333333, 80.83333333333333, 80.58333333333334, 80.33333333333333, 80.0, 79.75, 79.41666666666667, 79.16666666666667, 78.66666666666666, 78.5, 78.16666666666667, 77.91666666666667, 77.75, 77.41666666666667, 77.0, 76.75, 76.5, 76.33333333333333, 76.25, 75.91666666666667, 75.58333333333334, 75.16666666666667, 75.0, 74.75, 74.58333333333333, 74.41666666666666, 74.0, 73.58333333333333, 73.16666666666667, 72.91666666666667, 72.75, 72.08333333333333, 71.75, 71.50000000000001, 70.83333333333333, 70.5, 69.83333333333334, 69.33333333333334, 68.75, 68.41666666666666, 67.91666666666667, 67.58333333333333, 67.16666666666666, 66.75, 66.41666666666667, 65.91666666666667, 65.25000000000001, 64.58333333333333, 63.916666666666664, 63.33333333333333, 62.916666666666664, 62.16666666666666, 61.83333333333334, 61.25000000000001, 60.5, 59.66666666666667, 59.00000000000001, 58.33333333333333, 57.666666666666664, 57.00000000000001, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 2, 10, 17, 22, 29, 36, 43, 49, 54, 59, 68, 76, 82, 90, 97, 104, 114, 119, 124, 131, 138, 143, 150, 157, 162, 166, 170, 177, 180, 183, 188, 189, 191, 194, 196, 198, 203, 205, 212, 215, 218, 221, 225, 228, 232, 235, 241, 243, 247, 250, 252, 256, 261, 264, 267, 269, 270, 274, 278, 283, 285, 288, 290, 292, 297, 302, 307, 310, 312, 320, 324, 327, 335, 339, 347, 353, 360, 364, 370, 374, 379, 384, 388, 394, 402, 410, 418, 425, 430, 439, 443, 450, 459, 469, 477, 485, 493, 501, 513, 524], "exact_score": [98.58333333333333, 98.41666666666666, 97.75, 97.16666666666667, 96.75, 96.16666666666667, 95.58333333333333, 95.0, 94.58333333333333, 94.16666666666667, 93.83333333333333, 93.08333333333333, 92.41666666666667, 91.91666666666667, 91.25, 90.66666666666667, 90.08333333333334, 89.25, 88.83333333333333, 88.41666666666667, 87.83333333333333, 87.25, 86.83333333333334, 86.25, 85.66666666666667, 85.25, 84.91666666666666, 84.58333333333333, 84.0, 83.75, 83.5, 83.08333333333333, 83.0, 82.83333333333334, 82.58333333333333, 82.41666666666667, 82.25, 81.83333333333334, 81.66666666666667, 81.08333333333333, 80.83333333333333, 80.58333333333334, 80.33333333333333, 80.0, 79.75, 79.41666666666667, 79.16666666666667, 78.66666666666666, 78.5, 78.16666666666667, 77.91666666666667, 77.75, 77.41666666666667, 77.0, 76.75, 76.5, 76.33333333333333, 76.16666666666667, 75.91666666666667, 75.58333333333334, 75.16666666666667, 75.0, 74.75, 74.58333333333333, 74.41666666666666, 74.0, 73.58333333333333, 73.16666666666667, 72.91666666666667, 72.75, 72.08333333333333, 71.75, 71.50000000000001, 70.83333333333333, 70.5, 69.83333333333334, 69.33333333333334, 68.75, 68.41666666666666, 67.91666666666667, 67.58333333333333, 67.16666666666666, 66.75, 66.41666666666667, 65.91666666666667, 65.25000000000001, 64.58333333333333, 63.916666666666664, 63.33333333333333, 62.916666666666664, 62.16666666666666, 61.83333333333334, 61.25000000000001, 60.5, 59.66666666666667, 59.00000000000001, 58.33333333333333, 57.666666666666664, 57.00000000000001, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 77.84, "threshold_area100": 77.84083333333335, "threshold_area25": 92.08333333333334}, "test": {"score": [81.429064549534, 81.30824991370383, 80.9975837072834, 80.4970659302727, 79.70314118053159, 78.49499482222988, 77.87366240938903, 77.21781152916812, 76.87262685536763, 76.00966517086641, 75.12944425267519, 74.7152226441146, 73.71418709009319, 73.02381774249224, 71.90196755264067, 71.43596824301002, 70.88367276492924, 70.24508111839835, 69.93441491197791, 69.7100448740076, 69.5029340697273, 69.08871246116672, 68.570935450466, 68.22575077666552, 67.74249223334485, 67.38004832585433, 67.12115982050398, 67.0003451846738, 66.89678978253366, 66.51708664135313, 66.3617535381429, 66.2064204349327, 66.10286503279255, 65.84397652744218, 65.72316189161201, 65.65412495685192, 65.4297549188816, 65.37797721781153, 65.23990334829134, 65.11908871246118, 64.94649637556093, 64.87745944080083, 64.7911632723507, 64.65308940283052, 64.60131170176044, 64.34242319641008, 64.27338626164997, 64.18709009319986, 63.9972385226096, 63.89368312046946, 63.75560925094926, 63.56575768035899, 63.4276838108388, 63.375906109768735, 63.23783224024854, 63.117017604418365, 63.01346220227821, 62.80635139799793, 62.56472212633759, 62.426648256817394, 62.254055919917164, 62.16775975146703, 61.926130479806694, 61.839834311356576, 61.80531584397653, 61.63272350707629, 61.391094235415956, 61.37383500172593, 61.18398343113566, 61.08042802899551, 60.890576458405256, 60.700724887814985, 60.54539178460476, 60.424577148774596, 60.2865032792544, 60.14842940973422, 60.079392474974114, 60.01035554021401, 59.855022437003804, 59.76872626855367, 59.56161546427339, 59.47531929582327, 59.250949257852945, 58.97480151881257, 58.83672764929238, 58.526061442871935, 58.215395236451506, 58.06006213324129, 57.88746979634105, 57.73213669313083, 57.61132205730066, 57.38695201933035, 57.05902657921988, 56.76561960648947, 56.541249568519156, 56.29962029685882, 55.93717638936832, 55.868139454608226, 55.52295478080773, 55.07421470486711, 54.81532619951675], "calls": [0.0, 3.917846047635485, 7.870210562651018, 12.77183293061788, 17.82878840179496, 23.50707628581291, 27.407663099758373, 30.376251294442525, 32.80980324473594, 36.15809458060062, 39.19571971004488, 40.54193993786676, 43.458750431480844, 45.581636175353815, 48.46392820158785, 49.67207455988954, 51.08733172247152, 53.003106662064205, 53.71073524335519, 54.48740075940628, 55.16051087331723, 56.28236106316879, 57.50776665516051, 58.50880220918191, 59.66517086641353, 60.71798412150501, 61.649982740766305, 62.37487055574732, 62.87538833275803, 64.08353469105973, 64.70486710390058, 65.36071798412151, 65.9993096306524, 66.7932343803935, 67.22471522264412, 67.79426993441491, 68.65723161891611, 68.96789782533655, 69.62374870555747, 70.10700724887815, 70.86641353123922, 71.31515360717984, 71.74663444943045, 72.12633759061097, 72.86848463928202, 73.69692785640318, 74.19744563341388, 75.00862961684501, 75.73351743182603, 76.23403520883673, 76.88988608905764, 77.7183293061788, 78.37418018639973, 78.63306869175008, 79.20262340352089, 79.53054884363134, 80.23817742492233, 80.91128753883328, 81.5671384190542, 82.10217466344494, 82.6372109078357, 83.01691404901622, 83.7935795650673, 84.10424577148775, 84.41491197790818, 84.89817052122885, 85.62305833620987, 85.70935450466, 86.15809458060062, 86.65861235761132, 87.24542630307215, 87.74594408008285, 88.3500172592337, 88.88505350362443, 89.24749741111495, 89.73075595443562, 89.93786675871591, 90.47290300310667, 90.76630997583707, 91.02519848118743, 91.61201242664826, 91.80186399723851, 92.25060407317915, 92.95823265447014, 93.35519502934069, 94.04556437694166, 94.7186744908526, 95.01208146358302, 95.4263030721436, 95.75422851225406, 96.04763548498447, 96.46185709354505, 96.91059716948568, 97.39385571280636, 97.80807732136692, 98.18778046254747, 98.65377977217811, 98.8608905764584, 99.24059371763893, 99.74111149464963, 100.0], "rescued": [0, 42, 85, 135, 173, 215, 245, 265, 282, 302, 314, 320, 333, 343, 347, 349, 351, 355, 356, 356, 356, 356, 356, 357, 357, 357, 357, 358, 358, 359, 359, 359, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 360, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 361, 362, 362, 362, 362, 362, 362, 362, 362, 364, 364, 364, 364, 364, 364, 364, 365, 365, 365, 365], "harmed": [0, 49, 110, 189, 273, 385, 451, 509, 546, 616, 679, 709, 780, 830, 899, 928, 962, 1003, 1022, 1035, 1047, 1071, 1101, 1122, 1150, 1171, 1186, 1194, 1200, 1223, 1232, 1241, 1248, 1263, 1270, 1274, 1287, 1290, 1298, 1305, 1315, 1319, 1324, 1332, 1335, 1350, 1354, 1359, 1370, 1376, 1384, 1395, 1403, 1406, 1415, 1422, 1428, 1440, 1454, 1462, 1472, 1477, 1491, 1496, 1498, 1508, 1522, 1523, 1534, 1540, 1551, 1562, 1571, 1578, 1586, 1594, 1598, 1602, 1611, 1616, 1628, 1633, 1647, 1663, 1671, 1689, 1707, 1716, 1726, 1735, 1744, 1757, 1776, 1793, 1806, 1820, 1841, 1846, 1866, 1892, 1907], "exact_score": [81.429064549534, 81.32550914739386, 81.25647221263375, 81.30824991370383, 81.27373144632378, 81.27373144632378, 81.10113910942354, 81.10113910942354, 81.04936140835348, 80.98032447359338, 80.77321366931308, 80.66965826717293, 80.56610286503279, 80.4970659302727, 80.32447359337246, 80.11736278909216, 79.94477045219193, 79.84121505005179, 79.75491888160165, 79.51328960994132, 79.37521574042113, 79.03003106662064, 78.8574387297204, 78.63306869175008, 78.49499482222988, 78.27062478425958, 78.01173627890921, 77.90818087676908, 77.70107007248879, 77.61477390403866, 77.4249223334484, 77.07973765964792, 76.97618225750776, 76.8553676216776, 76.47566448049706, 76.21677597514672, 76.00966517086641, 75.71625819813602, 75.37107352433551, 75.19848118743528, 74.9223334483949, 74.5598895409044, 74.09389023127373, 73.80048325854332, 73.62789092164309, 73.26544701415257, 72.85122540559199, 72.50604073179151, 72.07455988954091, 71.69485674836038, 71.2806351397998, 70.88367276492924, 70.62478425957887, 70.24508111839835, 69.84811874352779, 69.58923023817742, 69.17500862961685, 68.8643424231964, 68.38108387987573, 68.0013807386952, 67.6389368312047, 67.31101139109424, 67.01760441836383, 66.7932343803935, 66.55160510873317, 66.30997583707283, 66.10286503279255, 65.77493959268207, 65.5678287884018, 65.36071798412151, 65.15360717984122, 64.9292371418709, 64.70486710390058, 64.54953400069037, 64.29064549534, 64.18709009319986, 63.94546082153953, 63.72109078356921, 63.51397997928891, 63.324128408698655, 63.03072143596824, 62.7890921643079, 62.4956851915775, 62.18501898515706, 61.8570935450466, 61.615464273386266, 61.2875388332758, 61.011391094235414, 60.59716948567484, 60.37279944770453, 60.079392474974114, 59.76872626855367, 59.371763893683124, 58.95754228512254, 58.56057991025199, 58.06006213324129, 57.61132205730066, 57.024508111839836, 56.43769416637902, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 70.83708266404403, "threshold_area100": 64.7870210562651, "threshold_area25": 73.79357956506732}, "selected_budget": 0, "selected": {"val": {"score": 98.58333333333333, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 81.429064549534, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.429064549534, "calls": 0.6040731791508458}}, "old_normalized_interior": {"val": {"score": 98.58333333333333, "calls": 1.0}, "test": {"score": 81.429064549534, "calls": 0.6040731791508458}}, "old_normalized_endpoints": {"val": {"score": 98.58333333333333, "calls": 0.0}, "test": {"score": 81.429064549534, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.41666666666666, 97.08333333333333, 97.0, 96.58333333333333, 96.25, 96.16666666666667, 95.91666666666667, 95.58333333333333, 95.33333333333334, 95.08333333333333, 95.0, 94.83333333333334, 94.5, 94.25, 93.91666666666667, 93.41666666666667, 93.16666666666666, 93.08333333333333, 92.5, 92.25, 92.16666666666666, 91.75, 91.41666666666667, 90.75, 90.41666666666667, 89.83333333333333, 89.5, 89.16666666666667, 88.5, 87.91666666666667, 87.25, 86.66666666666667, 86.08333333333333, 85.58333333333333, 85.0, 84.25, 83.66666666666667, 83.0, 82.16666666666667, 81.41666666666667, 80.75, 80.0, 79.16666666666667, 78.41666666666667, 77.83333333333333, 77.08333333333334, 76.16666666666667, 75.5, 74.83333333333333, 74.0, 73.16666666666667, 72.41666666666666, 71.50000000000001, 70.5, 69.83333333333334, 68.91666666666667, 68.0, 67.33333333333333, 66.41666666666667, 65.5, 64.58333333333333, 63.66666666666667, 62.66666666666667, 61.66666666666667, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 56.91666666666666, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.5, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.41666666666666, 98.33333333333334, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.75, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.66666666666667, 97.58333333333333, 97.5, 97.41666666666666, 97.08333333333333, 97.0, 96.58333333333333, 96.25, 96.16666666666667, 95.91666666666667, 95.58333333333333, 95.33333333333334, 95.08333333333333, 95.0, 94.83333333333334, 94.5, 94.25, 93.91666666666667, 93.41666666666667, 93.16666666666666, 93.08333333333333, 92.5, 92.25, 92.16666666666666, 91.75, 91.41666666666667, 90.75, 90.41666666666667, 89.83333333333333, 89.5, 89.16666666666667, 88.5, 87.91666666666667, 87.25, 86.66666666666667, 86.08333333333333, 85.58333333333333, 85.0, 84.25, 83.66666666666667, 83.0, 82.16666666666667, 81.33333333333333, 80.75, 80.0, 79.16666666666667, 78.41666666666667, 77.83333333333333, 77.08333333333334, 76.16666666666667, 75.5, 74.83333333333333, 74.0, 73.16666666666667, 72.41666666666666, 71.41666666666666, 70.5, 69.83333333333334, 68.91666666666667, 68.0, 67.33333333333333, 66.41666666666667, 65.5, 64.58333333333333, 63.66666666666667, 62.66666666666667, 61.66666666666667, 60.83333333333334, 59.833333333333336, 58.833333333333336, 57.833333333333336, 56.91666666666666, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.32499999999999, "exact_area100": 86.41083333333334}, "test": {"score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.34276838108389, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.23921297894374, 81.22195374525371, 81.22195374525371, 81.18743527787366, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.10113910942354, 81.04936140835348, 80.9975837072834, 80.98032447359338, 80.96306523990336, 80.89402830514325, 80.84225060407319, 80.73869520193303, 80.73869520193303, 80.66965826717293, 80.56610286503279, 80.4970659302727, 80.39351052813255, 80.39351052813255, 80.30721435968243, 80.16914049016223, 80.06558508802209, 80.10010355540214, 80.10010355540214, 79.94477045219193, 79.87573351743183, 79.70314118053159, 79.66862271315154, 79.4960303762513, 79.3579565067311, 79.25440110459095, 79.08180876769072, 78.71936486020022, 78.47773558853987, 78.30514325163963, 78.21884708318952, 78.09803244735933, 77.80462547462893, 77.54573696927856, 77.4249223334484, 77.3558853986883, 76.97618225750776, 76.73455298584743, 76.40662754573697, 76.09596133931653, 75.75077666551606, 75.40559199171557, 74.61166724197446, 74.2147048671039, 74.0766309975837, 73.71418709009319, 72.90300310666207, 72.40248532965137, 72.02278218847084, 71.36693130824992, 69.86537797721782, 69.26130479806697, 68.5018985157059, 68.05315843976527, 67.60441836382465, 66.75871591301346, 66.30997583707283, 65.96479116327235, 65.70590265792198, 65.32619951674145, 64.96375560925095, 63.91094235415948, 62.97894373489817, 62.478425957887474, 61.94338971349672, 61.37383500172593, 60.424577148774596, 59.630652399033494, 58.97480151881257, 58.526061442871935, 57.870210562651025, 57.26613738350017, 56.55850880220918, 56.14428719364861, 55.73006558508802, 55.33310321021747, 54.81532619951675], "calls": [0.0, 0.6040731791508458, 1.2944425267518123, 2.0538488091128753, 2.968588194684156, 3.40006903693476, 4.159475319295823, 4.832585433206766, 5.954435623058337, 6.679323438039352, 7.197100448740076, 7.870210562651018, 8.715913013462202, 9.699689333793579, 10.390058681394546, 11.02865032792544, 11.719019675526408, 12.374870555747325, 12.909906800138074, 13.117017604418363, 13.807386952019328, 14.445978598550225, 15.08457024508112, 15.533310321021746, 16.775975146703487, 17.846047635484986, 19.330341732827065, 19.882637210907834, 20.348636520538488, 21.505005177770105, 22.09181912323093, 23.058336209872284, 24.00759406282361, 25.371073524335518, 26.268553676216776, 27.010700724887815, 27.787366240938905, 28.753883327580255, 29.89299275112185, 31.04936140835347, 31.756989989644456, 32.25750776665516, 33.53469105971695, 35.24335519502934, 36.39972385226096, 37.573351743182606, 38.798757335174315, 40.33482913358647, 41.24956851915775, 42.85467725233, 44.373489817052125, 46.20296858819468, 47.96341042457715, 48.80911287538833, 49.72385226095961, 51.05281325509148, 52.329996548153254, 54.15947531929582, 55.33310321021747, 56.10976872626855, 57.680358992060754, 59.233690024162925, 60.75250258888505, 63.117017604418365, 64.87745944080083, 66.12012426648258, 68.72626855367622, 69.98619261304798, 70.81463583016914, 71.60856057991025, 73.35174318260269, 74.30100103555402, 75.04314808422507, 76.2167759751467, 78.80566102865033, 79.89299275112185, 81.27373144632378, 82.03313772868485, 82.91335864687608, 84.17328270624785, 85.31239212978944, 85.88194684156024, 86.38246461857094, 86.93476009665171, 87.46979634104247, 88.74697963410425, 89.97238522609597, 90.69727304107698, 91.49119779081809, 92.33690024162927, 93.64860200207112, 94.61511908871246, 95.44356230583362, 96.03037625129444, 96.77252329996548, 97.4283741801864, 98.15326199516741, 98.58474283741802, 99.03348291335865, 99.46496375560925, 100.0], "exact_score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.37728684846392, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.22195374525371, 81.18743527787366, 81.1356575768036, 81.10113910942354, 81.10113910942354, 81.04936140835348, 81.01484294097342, 80.96306523990336, 80.94580600621333, 80.84225060407319, 80.75595443562305, 80.73869520193303, 80.68691750086296, 80.58336209872282, 80.4970659302727, 80.39351052813255, 80.39351052813255, 80.30721435968243, 80.16914049016223, 80.06558508802209, 80.08284432171212, 80.06558508802209, 80.01380738695202, 79.87573351743183, 79.72040041422161, 79.72040041422161, 79.70314118053159, 79.59958577839144, 79.3579565067311, 79.37521574042113, 79.25440110459095, 79.13358646876078, 79.01277183293061, 78.78840179496031, 78.70210562651019, 78.46047635484985, 78.28788401794961, 78.16706938211944, 78.08077321366932, 77.856403175699, 77.71832930617882, 77.56299620296859, 77.45944080082845, 77.39040386606834, 77.06247842595789, 76.87262685536763, 76.78633068691751, 76.51018294787711, 76.40662754573697, 76.26855367621678, 76.04418363824647, 76.00966517086641, 75.75077666551606, 75.44011045909562, 75.14670348636521, 74.88781498101484, 74.59440800828443, 74.2147048671039, 73.95581636175355, 73.558853986883, 73.16189161201243, 72.5750776665516, 72.04004142216085, 71.52226441146013, 70.9181912323093, 70.26234035208837, 69.76182257507767, 69.19226786330687, 68.6399723852261, 68.0704176734553, 67.53538142906454, 66.89678978253366, 66.48256817397308, 65.89575422851226, 65.2916810493614, 64.54953400069037, 63.7901277183293, 62.96168450120815, 62.271315153607176, 61.615464273386266, 60.94235415947532, 60.14842940973422, 59.33724542630308, 58.54332067656196, 57.66309975837073, 56.71384190541941, 55.76458405246807, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 81.30134622022783, "exact_area100": 75.24926992309544}}, "old_normalized_endpoint_budget": 0}]}, {"expert": "glsim", "endpoint": "MiniCPM", "task": "cub", "small": 90.16223679668623, "large": 26.561960648947185, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.5, "validation_area100": 0.7194333333333333, "training_seconds": 3.9625079770339653, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.06870766614965555, "operator": "ge"}, {"budget": 0.02, "threshold": -0.07351362351858565, "operator": "ge"}, {"budget": 0.03, "threshold": -0.08216463718308659, "operator": "ge"}, {"budget": 0.04, "threshold": -0.08758971817968383, "operator": "ge"}, {"budget": 0.05, "threshold": -0.09569598546837305, "operator": "ge"}, {"budget": 0.06, "threshold": -0.10738841110218127, "operator": "ge"}, {"budget": 0.07, "threshold": -0.12797605151741165, "operator": "ge"}, {"budget": 0.08, "threshold": -0.15369326696887695, "operator": "ge"}, {"budget": 0.09, "threshold": -0.1771777701059389, "operator": "ge"}, {"budget": 0.1, "threshold": -0.20316298047576423, "operator": "ge"}, {"budget": 0.11, "threshold": -0.2267012416477621, "operator": "ge"}, {"budget": 0.12, "threshold": -0.24699896635530819, "operator": "ge"}, {"budget": 0.13, "threshold": -0.283686077211277, "operator": "ge"}, {"budget": 0.14, "threshold": -0.3303955884792244, "operator": "ge"}, {"budget": 0.15, "threshold": -0.35852417496746714, "operator": "ge"}, {"budget": 0.16, "threshold": -0.3861873939542344, "operator": "ge"}, {"budget": 0.17, "threshold": -0.41416699867977513, "operator": "ge"}, {"budget": 0.18, "threshold": -0.4457176509724127, "operator": "ge"}, {"budget": 0.19, "threshold": -0.4623460031450138, "operator": "ge"}, {"budget": 0.2, "threshold": -0.49700372534039794, "operator": "ge"}, {"budget": 0.21, "threshold": -0.5410581795727641, "operator": "ge"}, {"budget": 0.22, "threshold": -0.5738690364719841, "operator": "ge"}, {"budget": 0.23, "threshold": -0.6205602783300971, "operator": "ge"}, {"budget": 0.24, "threshold": -0.6605105012480326, "operator": "ge"}, {"budget": 0.25, "threshold": -0.6971934086918793, "operator": "ge"}, {"budget": 0.26, "threshold": -0.7247006830286521, "operator": "ge"}, {"budget": 0.27, "threshold": -0.749439393047229, "operator": "ge"}, {"budget": 0.28, "threshold": -0.7731126668699583, "operator": "ge"}, {"budget": 0.29, "threshold": -0.7974051873085927, "operator": "ge"}, {"budget": 0.3, "threshold": -0.8121398198470732, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8280494920734244, "operator": "ge"}, {"budget": 0.32, "threshold": -0.8467920208049415, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8616804954077336, "operator": "ge"}, {"budget": 0.34, "threshold": -0.878892941850816, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8951676260723422, "operator": "ge"}, {"budget": 0.36, "threshold": -0.910019835718688, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9211547930452413, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9252423698829855, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9344043172942639, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9410504153965769, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9446108820784888, "operator": "ge"}, {"budget": 0.42, "threshold": -0.951018153581606, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9597520677672784, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9639807736670053, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9666940901794594, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9699713411760402, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9723424957433755, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9760778418958036, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9776663208267949, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9799680808383586, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9809269916288323, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9823492885033178, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9838237814946061, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9848194268481602, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9858386144961166, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9863051907906227, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9873473666087641, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9883891976852512, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9889605214807916, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9896274651458979, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9902543409492157, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9906759109560114, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9916005905667739, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9920288988042884, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9924108578466632, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9928789146168099, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9931801904831123, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9934969219680996, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9938774724854708, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9943431528521613, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9947551240895408, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9950612034462073, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9954601819428691, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9957089945058138, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9959404204270411, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9961334433563446, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9963100587577887, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9965673104624763, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9967505612561146, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9970894152967382, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9974303606931891, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9977487151713617, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9979859117689177, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9981225242843793, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9983156737089575, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9985077950241805, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9986455218736306, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9987174984886596, "operator": "ge"}, {"budget": 0.89, "threshold": -0.998833345858935, "operator": "ge"}, {"budget": 0.9, "threshold": -0.998921087227017, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9990100182158348, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9990903307042871, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9991614269806225, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9992365939292999, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9992894463093274, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9993545236154682, "operator": "ge"}, {"budget": 0.97, "threshold": -0.999449092502219, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9995247781112185, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9996351934871907, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.58333333333333, 98.33333333333333, 98.25, 98.16666666666667, 97.91666666666666, 97.83333333333333, 97.66666666666667, 97.33333333333333, 97.33333333333333, 97.24999999999999, 97.0, 96.66666666666667, 96.58333333333333, 96.0, 95.58333333333333, 94.91666666666666, 94.75, 94.16666666666667, 93.41666666666666, 92.75, 91.83333333333333, 91.33333333333333, 90.75, 90.16666666666666, 89.5, 88.58333333333333, 87.83333333333333, 87.0, 86.08333333333333, 85.25, 84.41666666666666, 83.58333333333333, 82.58333333333333, 81.75, 80.83333333333333, 79.91666666666666, 79.16666666666666, 78.41666666666667, 77.58333333333333, 76.66666666666666, 75.75, 74.75, 73.75, 72.83333333333333, 71.91666666666666, 70.91666666666666, 69.91666666666667, 69.08333333333333, 68.08333333333333, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.25, 63.24999999999999, 62.33333333333333, 61.58333333333332, 60.58333333333332, 59.583333333333336, 58.58333333333333, 57.833333333333336, 56.749999999999986, 55.833333333333336, 54.833333333333336, 54.08333333333333, 53.08333333333333, 52.08333333333333, 51.08333333333333, 50.08333333333332, 49.166666666666664, 48.16666666666667, 47.24999999999999, 46.25, 45.33333333333333, 44.25, 43.25, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 0, 0, 0, 0, 1, 1, 1, 2, 3, 5, 8, 9, 10, 13, 14, 16, 20, 20, 21, 24, 28, 29, 36, 41, 49, 51, 58, 67, 75, 86, 92, 100, 108, 116, 127, 136, 146, 157, 167, 177, 187, 199, 209, 220, 231, 240, 249, 259, 270, 281, 293, 305, 316, 327, 339, 351, 361, 373, 384, 396, 408, 419, 431, 442, 451, 463, 475, 487, 496, 509, 520, 532, 541, 553, 565, 577, 589, 600, 612, 623, 635, 646, 659, 671, 682, 693, 705, 717, 729, 741, 753, 764, 775, 787, 799, 811, 821, 833, 845, 857], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.58333333333333, 98.33333333333333, 98.25, 98.16666666666667, 97.91666666666666, 97.83333333333333, 97.66666666666667, 97.33333333333333, 97.33333333333333, 97.24999999999999, 97.0, 96.66666666666667, 96.58333333333333, 96.0, 95.58333333333333, 94.91666666666666, 94.75, 94.16666666666667, 93.41666666666666, 92.75, 91.83333333333333, 91.33333333333333, 90.75, 90.16666666666666, 89.5, 88.58333333333333, 87.83333333333333, 87.0, 86.08333333333333, 85.25, 84.41666666666666, 83.5, 82.58333333333333, 81.75, 80.83333333333333, 79.91666666666666, 79.16666666666666, 78.41666666666667, 77.58333333333333, 76.66666666666666, 75.75, 74.75, 73.75, 72.83333333333333, 71.91666666666666, 70.91666666666666, 69.91666666666667, 69.0, 68.08333333333333, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.25, 63.24999999999999, 62.33333333333333, 61.58333333333332, 60.58333333333332, 59.583333333333336, 58.58333333333333, 57.74999999999999, 56.749999999999986, 55.833333333333336, 54.833333333333336, 54.08333333333333, 53.08333333333333, 52.08333333333333, 51.08333333333333, 50.08333333333332, 49.166666666666664, 48.16666666666667, 47.24999999999999, 46.25, 45.25, 44.25, 43.25, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.94333333333333, "threshold_area100": 71.94666666666664, "threshold_area25": 97.83333333333333}, "test": {"score": [90.16223679668623, 90.17949603037624, 90.17949603037624, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.02416292716603, 90.006903693476, 89.83431135657577, 89.7134967207456, 89.62720055229548, 89.60994131860545, 89.40283051432516, 89.29927511218501, 89.19571971004487, 89.0058681394546, 88.69520193303417, 88.41905419399379, 88.26372109078356, 88.0048325854332, 87.79772178115292, 87.52157404211253, 87.2109078356921, 86.91750086296169, 86.45150155333103, 86.0027614773904, 85.45046599930963, 84.86365205384881, 84.20780117362789, 83.74180186399724, 83.20676561960649, 82.4991370383155, 81.7224715222644, 81.15291681049361, 80.32447359337245, 79.56506731101139, 78.87469796341043, 78.3569209527097, 77.45944080082843, 76.52744218156712, 75.99240593717639, 75.11218501898516, 73.43803935105281, 72.5750776665516, 71.83293061788056, 70.78011736278908, 70.08974801518812, 68.65723161891611, 68.01863997238522, 66.8622713151536, 66.39627200552295, 65.51605108733172, 64.61857093545046, 63.89368312046945, 63.16879530548844, 62.68553676216775, 61.77079737659648, 60.83879875733518, 60.2865032792544, 59.33724542630308, 58.59509837763204, 58.00828443217121, 56.523990334829136, 55.79910251984812, 55.29858474283742, 54.349326889886086, 53.727994477045215, 52.951328960994125, 52.19192267863306, 50.88022091819122, 49.63755609250949, 48.981705212288574, 47.928891957197095, 47.134967207455986, 46.56541249568519, 45.944080082844316, 45.49534000690369, 44.77045219192268, 44.33897134967207, 43.13082499137038, 41.81912323092855, 41.04245771487746, 40.075940628236104, 39.6444597859855, 38.83327580255437, 37.78046254746289, 37.12461166724197, 36.64135312392129, 36.037279944770454, 35.174318260269246, 34.38039351052813, 33.690024162927166, 33.06869175008629, 32.06765619606489, 31.463583016914043, 30.497065930272694, 29.426993441491188, 28.52951328960994, 27.52847773558854, 26.56196064894718], "calls": [0.0, 1.794960303762513, 2.416292716603383, 3.7452537107352435, 4.366586123576114, 4.884363134276838, 6.023472557818433, 7.162581981360027, 8.62961684501208, 9.90680013807387, 11.511908871246117, 12.426648256817398, 13.134276838108388, 14.463237832240248, 15.895754228512255, 16.724197445633415, 17.53538142906455, 18.65723161891612, 19.796341042457716, 20.296858819468415, 21.34967207455989, 22.437003797031412, 23.38626164998274, 24.490852606144287, 25.146703486365208, 26.354849844666894, 27.355885398688294, 28.305143251639624, 29.461511908871245, 30.72143596824301, 31.4981014842941, 32.550914739385576, 33.86261649982741, 35.19157749395927, 36.19261304798067, 37.52157404211253, 38.90231273731447, 39.83431135657577, 40.490162236796685, 41.560234725578184, 42.85467725233, 43.61408353469106, 45.11563686572316, 47.497411114946495, 48.774594408008284, 49.72385226095961, 50.98377632033137, 51.81221953745254, 53.572661373835004, 54.47014152571625, 55.91991715567829, 56.48947186744908, 57.525025888850536, 58.64687607870211, 59.50983776320331, 60.40731791508457, 61.02865032792544, 62.046945115636866, 63.08249913703832, 63.65205384880911, 64.67034863652053, 65.55056955471177, 66.25819813600276, 67.9323438039351, 68.81256472212634, 69.46841560234725, 70.55574732481877, 71.21159820503969, 72.05730065585088, 72.88574387297204, 74.283741801864, 75.5781843286158, 76.35484984466689, 77.5802554366586, 78.42595788746979, 79.08180876769072, 79.7204004142216, 80.16914049016223, 80.9975837072834, 81.429064549534, 82.70624784259579, 84.10424577148775, 84.94994822229893, 85.9337245426303, 86.36520538488092, 87.24542630307215, 88.36727649292372, 89.02312737314463, 89.50638591646532, 90.17949603037624, 91.04245771487746, 91.88816016568865, 92.61304798066966, 93.23438039351053, 94.26993441491199, 94.9257852951329, 95.89230238177426, 96.9968933379358, 97.9288919571971, 98.9989644459786, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 6, 6, 6, 8, 8, 9, 12, 14, 16, 22, 24, 27, 31, 33, 34, 36, 38, 41, 43, 44, 47, 51, 56, 57, 57, 58, 59, 59, 67, 73, 76, 76, 77, 77, 78, 79, 82, 82, 83, 85, 86, 88, 90, 91, 91, 91, 91, 93, 94, 95, 96, 98, 99, 99, 99, 99, 99, 99, 101, 102, 102, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106], "harmed": [0, 0, 0, 1, 1, 2, 3, 9, 11, 21, 28, 33, 34, 47, 56, 62, 73, 93, 109, 119, 137, 151, 169, 193, 212, 242, 272, 306, 341, 381, 410, 444, 487, 533, 569, 621, 670, 711, 741, 794, 849, 880, 939, 1042, 1095, 1138, 1200, 1240, 1324, 1362, 1432, 1459, 1511, 1565, 1608, 1652, 1682, 1736, 1790, 1822, 1877, 1922, 1957, 2044, 2087, 2118, 2174, 2210, 2255, 2299, 2375, 2447, 2487, 2549, 2595, 2629, 2665, 2691, 2733, 2758, 2828, 2905, 2950, 3006, 3031, 3078, 3140, 3178, 3206, 3241, 3291, 3337, 3378, 3414, 3472, 3507, 3563, 3625, 3677, 3735, 3791], "exact_score": [90.16223679668623, 90.17949603037624, 90.17949603037624, 90.17949603037624, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.04142216085606, 90.006903693476, 89.98964445978598, 89.83431135657577, 89.76527442181566, 89.69623748705557, 89.60994131860545, 89.43734898170521, 89.3683120469451, 89.28201587849499, 89.12668277528478, 88.88505350362443, 88.574387297204, 88.33275802554367, 88.05661028650327, 87.91853641698309, 87.69416637901277, 87.36624093890231, 86.98653779772178, 86.62409389023126, 86.10631687953054, 85.67483603727995, 85.10528132550914, 84.5012081463583, 84.05246807041767, 83.5001725923369, 82.9306178805661, 82.44735933724542, 81.82602692440454, 81.30824991370382, 80.60062133241284, 80.13462202278218, 79.4960303762513, 78.73662409389023, 77.97721781152916, 77.13151536071798, 76.45840524680703, 75.75077666551606, 75.23299965481533, 74.43907490507421, 73.88677942699344, 73.0755954435623, 72.41974456334138, 71.60856057991025, 70.78011736278908, 69.93441491197791, 69.10597169485675, 68.38108387987573, 67.62167759751468, 66.81049361408353, 65.98205039696236, 65.17086641353124, 64.35968243010011, 63.51397997928891, 62.720055229547796, 61.822575077666556, 60.92509492578528, 59.97583707283396, 59.026579219882635, 58.23265447014152, 57.35243355195029, 56.50673110113911, 55.67828788401794, 54.79806696582672, 53.935105281325505, 53.02036589575423, 52.12288574387297, 51.15636865723161, 50.20711080428028, 49.27511218501898, 48.46392820158785, 47.54918881601656, 46.65170866413531, 45.667932343803926, 44.77045219192268, 43.80393510528132, 42.85467725232999, 41.905419399378665, 41.00793924749741, 40.02416292716603, 39.074905074214705, 38.1429064549534, 37.15913013462202, 36.22713151536071, 35.22609596133931, 34.276838108387985, 33.31032102174664, 32.34380393510528, 31.39454608215395, 30.393510528132552, 29.426993441491188, 28.46047635484984, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 67.23878712805516, "threshold_area100": 65.27226441146011, "threshold_area25": 89.0731791508457}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0050849150866270065, "operator": "ge"}, {"budget": 0.02, "threshold": -0.005799790378659964, "operator": "ge"}, {"budget": 0.03, "threshold": -0.007121337112039328, "operator": "ge"}, {"budget": 0.04, "threshold": -0.00809694454073906, "operator": "ge"}, {"budget": 0.05, "threshold": -0.009877420030534267, "operator": "ge"}, {"budget": 0.06, "threshold": -0.012776871211826801, "operator": "ge"}, {"budget": 0.07, "threshold": -0.02018723636865616, "operator": "ge"}, {"budget": 0.08, "threshold": -0.0291549451649189, "operator": "ge"}, {"budget": 0.09, "threshold": -0.03802625089883804, "operator": "ge"}, {"budget": 0.1, "threshold": -0.043336331844329834, "operator": "ge"}, {"budget": 0.11, "threshold": -0.05644913390278816, "operator": "ge"}, {"budget": 0.12, "threshold": -0.06818443536758423, "operator": "ge"}, {"budget": 0.13, "threshold": -0.09281858056783676, "operator": "ge"}, {"budget": 0.14, "threshold": -0.11969415098428726, "operator": "ge"}, {"budget": 0.15, "threshold": -0.13160249590873718, "operator": "ge"}, {"budget": 0.16, "threshold": -0.15243656933307648, "operator": "ge"}, {"budget": 0.17, "threshold": -0.17434929311275482, "operator": "ge"}, {"budget": 0.18, "threshold": -0.1998034566640854, "operator": "ge"}, {"budget": 0.19, "threshold": -0.21714238822460175, "operator": "ge"}, {"budget": 0.2, "threshold": -0.2531101405620575, "operator": "ge"}, {"budget": 0.21, "threshold": -0.3025662899017334, "operator": "ge"}, {"budget": 0.22, "threshold": -0.33916130661964417, "operator": "ge"}, {"budget": 0.23, "threshold": -0.41045108437538147, "operator": "ge"}, {"budget": 0.24, "threshold": -0.4652417302131653, "operator": "ge"}, {"budget": 0.25, "threshold": -0.49744805693626404, "operator": "ge"}, {"budget": 0.26, "threshold": -0.5476201176643372, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5830552577972412, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6215641498565674, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6538177132606506, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6772860288619995, "operator": "ge"}, {"budget": 0.31, "threshold": -0.706304132938385, "operator": "ge"}, {"budget": 0.32, "threshold": -0.736708402633667, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7645987868309021, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7992215156555176, "operator": "ge"}, {"budget": 0.35, "threshold": -0.828151285648346, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8498613238334656, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8561824560165405, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8741055727005005, "operator": "ge"}, {"budget": 0.39, "threshold": -0.886041522026062, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8948284983634949, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9037648439407349, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9171463251113892, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9285311698913574, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9342807531356812, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9395458698272705, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9434241652488708, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9526638984680176, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9560385346412659, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9605957865715027, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9625140428543091, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9663093686103821, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9688200950622559, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9703572392463684, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9721713662147522, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9729329943656921, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9748668074607849, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9763826131820679, "operator": "ge"}, {"budget": 0.58, "threshold": -0.977691650390625, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9788256883621216, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9804815649986267, "operator": "ge"}, {"budget": 0.61, "threshold": -0.981449544429779, "operator": "ge"}, {"budget": 0.62, "threshold": -0.983191192150116, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9838781356811523, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9847707152366638, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9858248233795166, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9864662289619446, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9871672987937927, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9878728985786438, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9887257218360901, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9896349310874939, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9901040196418762, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9909285306930542, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9912970066070557, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9918220639228821, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9922200441360474, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9925103187561035, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9931081533432007, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9934639930725098, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9940674901008606, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9947435855865479, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9950295090675354, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9957016110420227, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9960840940475464, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9964408278465271, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9968594312667847, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9970905184745789, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9973782896995544, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9975634217262268, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9977111220359802, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9978935718536377, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9980669021606445, "operator": "ge"}, {"budget": 0.92, "threshold": -0.998248279094696, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9983603358268738, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9985197186470032, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9986274838447571, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9987910389900208, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9989461302757263, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9990816116333008, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9993143081665039, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 98.0, 97.83333333333333, 97.5, 97.33333333333333, 97.33333333333333, 97.33333333333333, 97.0, 96.83333333333333, 96.58333333333333, 96.08333333333333, 95.5, 95.16666666666667, 94.75, 94.08333333333333, 93.5, 92.75, 92.16666666666666, 91.49999999999999, 90.66666666666666, 90.16666666666666, 89.16666666666666, 88.41666666666667, 87.75, 86.83333333333333, 86.0, 85.33333333333333, 84.41666666666666, 83.5, 82.58333333333333, 81.66666666666667, 80.75, 79.83333333333333, 79.08333333333333, 78.25, 77.41666666666667, 76.41666666666667, 75.41666666666667, 74.41666666666666, 73.58333333333333, 72.66666666666666, 71.75, 70.75, 69.83333333333333, 69.08333333333333, 68.08333333333333, 67.16666666666666, 66.16666666666666, 65.25, 64.25, 63.33333333333333, 62.416666666666664, 61.5, 60.58333333333332, 59.583333333333336, 58.666666666666664, 57.74999999999999, 56.749999999999986, 55.75, 54.833333333333336, 53.91666666666666, 53.0, 52.0, 51.0, 50.08333333333332, 49.08333333333333, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.33333333333333, 44.25, 43.25, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 0, 0, 0, 0, 1, 1, 1, 2, 3, 5, 7, 9, 10, 11, 13, 17, 19, 19, 20, 24, 26, 29, 35, 42, 46, 51, 59, 66, 75, 82, 90, 100, 107, 119, 128, 136, 147, 157, 166, 177, 188, 199, 210, 221, 232, 241, 251, 261, 273, 285, 297, 307, 318, 329, 341, 352, 361, 373, 384, 396, 407, 419, 430, 441, 452, 463, 475, 486, 497, 509, 521, 532, 543, 554, 566, 578, 589, 601, 613, 624, 636, 646, 659, 671, 681, 693, 705, 717, 729, 741, 753, 765, 776, 788, 800, 810, 821, 833, 845, 857], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 98.0, 97.83333333333333, 97.5, 97.33333333333333, 97.33333333333333, 97.33333333333333, 97.0, 96.83333333333333, 96.58333333333333, 96.08333333333333, 95.5, 95.16666666666667, 94.75, 94.08333333333333, 93.5, 92.75, 92.16666666666666, 91.49999999999999, 90.66666666666666, 90.16666666666666, 89.16666666666666, 88.41666666666667, 87.75, 86.83333333333333, 86.0, 85.33333333333333, 84.41666666666666, 83.5, 82.58333333333333, 81.66666666666667, 80.75, 79.83333333333333, 79.08333333333333, 78.25, 77.41666666666667, 76.41666666666667, 75.41666666666667, 74.41666666666666, 73.58333333333333, 72.66666666666666, 71.75, 70.75, 69.83333333333333, 69.0, 68.08333333333333, 67.16666666666666, 66.16666666666666, 65.25, 64.25, 63.33333333333333, 62.416666666666664, 61.5, 60.58333333333332, 59.583333333333336, 58.666666666666664, 57.666666666666664, 56.749999999999986, 55.75, 54.833333333333336, 53.91666666666666, 53.0, 52.0, 51.0, 50.08333333333332, 49.08333333333333, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.25, 44.25, 43.25, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.90916666666666, "threshold_area100": 71.91166666666666, "threshold_area25": 97.84166666666667}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.04142216085606, 89.98964445978598, 89.97238522609597, 89.95512599240594, 89.88608905764583, 89.79979288919571, 89.59268208491544, 89.47186744908525, 89.42008974801519, 89.29927511218501, 88.93683120469451, 88.72972040041421, 88.59164653089402, 88.35001725923368, 88.09112875388332, 87.84949948222298, 87.36624093890231, 86.91750086296169, 86.60683465654124, 86.24439074905074, 85.88194684156024, 85.34691059716948, 84.84639282015878, 84.34587504314808, 83.72454263030721, 83.01691404901622, 82.51639627200552, 81.60165688643424, 80.89402830514325, 80.16914049016223, 79.7894373489817, 79.01277183293061, 78.2878840179496, 77.61477390403866, 76.83810838798757, 75.45736969278563, 74.62892647566449, 74.05937176389368, 73.21366931308249, 72.73041076976182, 71.22885743872972, 70.36589575422852, 69.34760096651709, 68.91612012426648, 67.63893683120469, 66.81049361408353, 66.13738350017259, 65.18812564722126, 64.80842250604073, 63.87642388677942, 63.34138764238868, 62.58198136002762, 61.839834311356576, 60.89057645840524, 60.16568864342423, 58.64687607870211, 58.12909906800138, 57.421470486710376, 56.48947186744908, 55.74732481877804, 54.936140835346905, 54.19399378667586, 52.89955125992405, 51.62236796686227, 51.08733172247152, 49.77562996202968, 49.206075250258884, 48.60200207110804, 48.03244735933724, 47.54918881601656, 46.513634794615115, 46.15119088712461, 45.02934069727304, 43.838453572661365, 43.35519502934069, 42.147048671039, 41.128753883327576, 40.38660683465653, 39.12668277528477, 38.59164653089403, 37.745944080082836, 36.96927856403175, 36.43424231964101, 35.53676216775975, 34.76009665170866, 34.03520883672765, 33.551950293406975, 32.48187780462547, 31.70521228857438, 30.73869520193303, 29.789437348981707, 28.978253365550565, 27.56299620296858, 26.56196064894718], "calls": [0.0, 1.4152571625819814, 1.9502934069727302, 3.0203658957542285, 3.5554021401449774, 4.245771487745944, 5.074214704867104, 6.38591646530894, 7.766655160510873, 9.458060062133242, 10.234725578184328, 11.218501898515706, 12.029685881946842, 13.289609941318606, 14.566793234380393, 15.17086641353124, 16.102865032792543, 17.32827062478426, 18.39834311356576, 18.916120124266484, 19.968933379357956, 21.34967207455989, 22.160856057991023, 23.558853986882983, 24.37003797031412, 25.146703486365208, 26.078702105626512, 26.889886089057647, 28.184328615809456, 29.150845702450813, 29.94477045219192, 31.032102174663446, 32.39558163617535, 33.275802554366585, 34.60476354849845, 35.81290990680014, 37.021056265101834, 37.55609250949258, 38.81601656886434, 40.00690369347601, 41.042457714877465, 42.06075250258888, 43.872972040041425, 45.18467380048326, 45.94408008284432, 46.910597169485676, 47.51467034863652, 49.63755609250949, 50.70762858129099, 52.295478080773215, 52.88229202623403, 54.43562305833621, 55.43665861235761, 56.24784259578875, 57.26613738350017, 57.66309975837073, 58.715913013462206, 59.38902312737314, 60.23472557818433, 61.114946496375566, 62.23679668622714, 63.03072143596824, 64.80842250604073, 65.39523645150155, 66.22367966862271, 67.32827062478425, 68.17397307559544, 69.17500862961684, 70.02071108042803, 71.48774594408009, 72.85122540559199, 73.50707628581291, 74.97411114946496, 75.5781843286158, 76.26855367621678, 76.95892302381773, 77.52847773558854, 78.75388332758025, 79.2198826372109, 80.56610286503279, 81.80876769071453, 82.34380393510529, 83.62098722816707, 84.67380048325855, 85.5022437003797, 86.86572316189162, 87.41801863997239, 88.28098032447359, 89.14394200897479, 89.67897825336554, 90.59371763893684, 91.43942008974801, 92.21608560579911, 92.73386261649983, 93.8557128063514, 94.66689678978253, 95.63341387642389, 96.59993096306523, 97.44563341387642, 98.94718674490852, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 2, 2, 2, 2, 2, 3, 3, 4, 4, 6, 9, 10, 10, 12, 16, 20, 25, 25, 25, 28, 33, 34, 36, 41, 45, 46, 49, 51, 52, 53, 54, 58, 58, 58, 59, 65, 66, 74, 77, 79, 80, 80, 80, 80, 80, 81, 82, 84, 85, 86, 87, 87, 87, 89, 90, 91, 91, 94, 94, 95, 96, 96, 97, 97, 98, 98, 99, 101, 101, 101, 102, 103, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106], "harmed": [0, 0, 0, 0, 0, 2, 3, 7, 10, 12, 14, 18, 23, 35, 42, 45, 52, 74, 86, 95, 109, 126, 143, 172, 198, 218, 243, 268, 304, 333, 362, 401, 447, 477, 532, 578, 624, 647, 695, 739, 779, 825, 906, 958, 991, 1040, 1069, 1162, 1213, 1280, 1308, 1384, 1433, 1472, 1527, 1549, 1603, 1635, 1680, 1725, 1781, 1824, 1913, 1943, 1984, 2040, 2084, 2132, 2175, 2253, 2327, 2359, 2436, 2469, 2505, 2538, 2567, 2627, 2649, 2716, 2785, 2813, 2884, 2944, 2988, 3062, 3093, 3142, 3187, 3218, 3270, 3315, 3357, 3385, 3448, 3493, 3549, 3604, 3651, 3733, 3791], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.04142216085606, 90.04142216085606, 89.98964445978598, 89.98964445978598, 89.95512599240594, 89.90334829133586, 89.79979288919571, 89.62720055229548, 89.5063859164653, 89.45460821539524, 89.29927511218501, 89.02312737314463, 88.83327580255437, 88.55712806351399, 88.33275802554367, 88.16016568864342, 87.95305488436314, 87.55609250949257, 87.14187090093199, 86.7103900586814, 86.27890921643079, 85.81290990680014, 85.39868829823956, 84.91542975491888, 84.29409734207802, 83.75906109768727, 83.22402485329651, 82.70624784259579, 82.05039696237488, 81.41180531584398, 80.7904729030031, 80.18639972385225, 79.54780807732136, 78.89195719710045, 78.30514325163962, 77.64929237141871, 76.88988608905764, 76.0787021056265, 75.45736969278563, 74.76700034518467, 74.0075940628236, 73.1446323783224, 72.35070762858129, 71.67759751467034, 70.91819123230928, 70.19330341732827, 69.55471177079737, 68.84708318950639, 68.01863997238522, 67.13841905419399, 66.34449430445288, 65.46427338626164, 64.53227476700035, 63.61753538142907, 62.771832930617876, 61.926130479806694, 61.02865032792544, 60.20020711080427, 59.33724542630308, 58.49154297549188, 57.6458405246807, 56.817397307559546, 55.937176389368304, 55.09147393855712, 54.21125302036589, 53.33103210217466, 52.48532965136348, 51.5015533310321, 50.690369347600964, 49.77562996202968, 48.8608905764584, 47.997928891957194, 47.16948567483603, 46.306523990334824, 45.512599240593715, 44.63237832240248, 43.66586123576113, 42.75112185018985, 41.81912323092855, 40.83534691059717, 39.920607525025886, 39.005868139454606, 38.022091819123226, 37.09009319986192, 36.123576113220565, 35.19157749395927, 34.25957887469796, 33.32758025543665, 32.34380393510528, 31.377286848463925, 30.393510528132552, 29.409734207801176, 28.46047635484984, 27.511218501898515, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 66.95901119663742, "threshold_area100": 66.02373144632378, "threshold_area25": 89.24749741111495}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -3.2388687133789062, "operator": "ge"}, {"budget": 0.02, "threshold": -3.7979962825775146, "operator": "ge"}, {"budget": 0.03, "threshold": -4.8645524978637695, "operator": "ge"}, {"budget": 0.04, "threshold": -5.620965003967285, "operator": "ge"}, {"budget": 0.05, "threshold": -6.045894145965576, "operator": "ge"}, {"budget": 0.06, "threshold": -6.414947509765625, "operator": "ge"}, {"budget": 0.07, "threshold": -6.666481018066406, "operator": "ge"}, {"budget": 0.08, "threshold": -7.140348434448242, "operator": "ge"}, {"budget": 0.09, "threshold": -7.369501113891602, "operator": "ge"}, {"budget": 0.1, "threshold": -7.659746170043945, "operator": "ge"}, {"budget": 0.11, "threshold": -7.798518657684326, "operator": "ge"}, {"budget": 0.12, "threshold": -8.079998016357422, "operator": "ge"}, {"budget": 0.13, "threshold": -8.357129096984863, "operator": "ge"}, {"budget": 0.14, "threshold": -8.480718612670898, "operator": "ge"}, {"budget": 0.15, "threshold": -8.720115661621094, "operator": "ge"}, {"budget": 0.16, "threshold": -8.904866218566895, "operator": "ge"}, {"budget": 0.17, "threshold": -9.00664234161377, "operator": "ge"}, {"budget": 0.18, "threshold": -9.121261596679688, "operator": "ge"}, {"budget": 0.19, "threshold": -9.19284439086914, "operator": "ge"}, {"budget": 0.2, "threshold": -9.229316711425781, "operator": "ge"}, {"budget": 0.21, "threshold": -9.44803237915039, "operator": "ge"}, {"budget": 0.22, "threshold": -9.579092025756836, "operator": "ge"}, {"budget": 0.23, "threshold": -9.658777236938477, "operator": "ge"}, {"budget": 0.24, "threshold": -9.725111961364746, "operator": "ge"}, {"budget": 0.25, "threshold": -9.883519172668457, "operator": "ge"}, {"budget": 0.26, "threshold": -9.961000442504883, "operator": "ge"}, {"budget": 0.27, "threshold": -10.02938175201416, "operator": "ge"}, {"budget": 0.28, "threshold": -10.12794303894043, "operator": "ge"}, {"budget": 0.29, "threshold": -10.234554290771484, "operator": "ge"}, {"budget": 0.3, "threshold": -10.36191177368164, "operator": "ge"}, {"budget": 0.31, "threshold": -10.417999267578125, "operator": "ge"}, {"budget": 0.32, "threshold": -10.472286224365234, "operator": "ge"}, {"budget": 0.33, "threshold": -10.520647048950195, "operator": "ge"}, {"budget": 0.34, "threshold": -10.605891227722168, "operator": "ge"}, {"budget": 0.35, "threshold": -10.670586585998535, "operator": "ge"}, {"budget": 0.36, "threshold": -10.76855754852295, "operator": "ge"}, {"budget": 0.37, "threshold": -10.875452041625977, "operator": "ge"}, {"budget": 0.38, "threshold": -10.978904724121094, "operator": "ge"}, {"budget": 0.39, "threshold": -11.091995239257812, "operator": "ge"}, {"budget": 0.4, "threshold": -11.200432777404785, "operator": "ge"}, {"budget": 0.41, "threshold": -11.263056755065918, "operator": "ge"}, {"budget": 0.42, "threshold": -11.385233879089355, "operator": "ge"}, {"budget": 0.43, "threshold": -11.447386741638184, "operator": "ge"}, {"budget": 0.44, "threshold": -11.562555313110352, "operator": "ge"}, {"budget": 0.45, "threshold": -11.643065452575684, "operator": "ge"}, {"budget": 0.46, "threshold": -11.721789360046387, "operator": "ge"}, {"budget": 0.47, "threshold": -11.767793655395508, "operator": "ge"}, {"budget": 0.48, "threshold": -11.893890380859375, "operator": "ge"}, {"budget": 0.49, "threshold": -12.028749465942383, "operator": "ge"}, {"budget": 0.5, "threshold": -12.18617057800293, "operator": "ge"}, {"budget": 0.51, "threshold": -12.251415252685547, "operator": "ge"}, {"budget": 0.52, "threshold": -12.324994087219238, "operator": "ge"}, {"budget": 0.53, "threshold": -12.385865211486816, "operator": "ge"}, {"budget": 0.54, "threshold": -12.474225997924805, "operator": "ge"}, {"budget": 0.55, "threshold": -12.57071304321289, "operator": "ge"}, {"budget": 0.56, "threshold": -12.641304016113281, "operator": "ge"}, {"budget": 0.57, "threshold": -12.710404396057129, "operator": "ge"}, {"budget": 0.58, "threshold": -12.763933181762695, "operator": "ge"}, {"budget": 0.59, "threshold": -12.822336196899414, "operator": "ge"}, {"budget": 0.6, "threshold": -12.876343727111816, "operator": "ge"}, {"budget": 0.61, "threshold": -12.968345642089844, "operator": "ge"}, {"budget": 0.62, "threshold": -13.005258560180664, "operator": "ge"}, {"budget": 0.63, "threshold": -13.069035530090332, "operator": "ge"}, {"budget": 0.64, "threshold": -13.142006874084473, "operator": "ge"}, {"budget": 0.65, "threshold": -13.186302185058594, "operator": "ge"}, {"budget": 0.66, "threshold": -13.257468223571777, "operator": "ge"}, {"budget": 0.67, "threshold": -13.321859359741211, "operator": "ge"}, {"budget": 0.68, "threshold": -13.361542701721191, "operator": "ge"}, {"budget": 0.69, "threshold": -13.39726734161377, "operator": "ge"}, {"budget": 0.7, "threshold": -13.443231582641602, "operator": "ge"}, {"budget": 0.71, "threshold": -13.509673118591309, "operator": "ge"}, {"budget": 0.72, "threshold": -13.569561004638672, "operator": "ge"}, {"budget": 0.73, "threshold": -13.626289367675781, "operator": "ge"}, {"budget": 0.74, "threshold": -13.659266471862793, "operator": "ge"}, {"budget": 0.75, "threshold": -13.738889694213867, "operator": "ge"}, {"budget": 0.76, "threshold": -13.776302337646484, "operator": "ge"}, {"budget": 0.77, "threshold": -13.810892105102539, "operator": "ge"}, {"budget": 0.78, "threshold": -13.856094360351562, "operator": "ge"}, {"budget": 0.79, "threshold": -13.92386245727539, "operator": "ge"}, {"budget": 0.8, "threshold": -13.981120109558105, "operator": "ge"}, {"budget": 0.81, "threshold": -14.022302627563477, "operator": "ge"}, {"budget": 0.82, "threshold": -14.085610389709473, "operator": "ge"}, {"budget": 0.83, "threshold": -14.140149116516113, "operator": "ge"}, {"budget": 0.84, "threshold": -14.193641662597656, "operator": "ge"}, {"budget": 0.85, "threshold": -14.262664794921875, "operator": "ge"}, {"budget": 0.86, "threshold": -14.316864013671875, "operator": "ge"}, {"budget": 0.87, "threshold": -14.36271858215332, "operator": "ge"}, {"budget": 0.88, "threshold": -14.431118965148926, "operator": "ge"}, {"budget": 0.89, "threshold": -14.46969985961914, "operator": "ge"}, {"budget": 0.9, "threshold": -14.537124633789062, "operator": "ge"}, {"budget": 0.91, "threshold": -14.583137512207031, "operator": "ge"}, {"budget": 0.92, "threshold": -14.682177543640137, "operator": "ge"}, {"budget": 0.93, "threshold": -14.748077392578125, "operator": "ge"}, {"budget": 0.94, "threshold": -14.862448692321777, "operator": "ge"}, {"budget": 0.95, "threshold": -14.988475799560547, "operator": "ge"}, {"budget": 0.96, "threshold": -15.072047233581543, "operator": "ge"}, {"budget": 0.97, "threshold": -15.16843032836914, "operator": "ge"}, {"budget": 0.98, "threshold": -15.461877822875977, "operator": "ge"}, {"budget": 0.99, "threshold": -15.624141693115234, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.66666666666667, 98.16666666666667, 97.33333333333333, 96.91666666666666, 96.33333333333333, 95.91666666666666, 95.41666666666666, 94.91666666666666, 94.16666666666667, 93.5, 92.83333333333333, 92.41666666666666, 92.08333333333333, 91.66666666666666, 91.49999999999999, 91.08333333333333, 90.66666666666666, 90.25, 89.83333333333333, 89.58333333333333, 89.25, 89.0, 88.58333333333333, 88.33333333333333, 88.08333333333334, 87.41666666666667, 87.25, 86.83333333333333, 86.5, 85.91666666666666, 85.58333333333333, 85.16666666666666, 84.74999999999999, 84.25, 83.5, 82.83333333333334, 82.33333333333333, 81.75, 81.25, 80.58333333333333, 80.08333333333333, 79.5, 78.83333333333333, 77.91666666666667, 77.16666666666666, 76.41666666666667, 75.66666666666666, 74.91666666666667, 74.33333333333333, 73.66666666666666, 73.0, 72.08333333333333, 71.33333333333334, 70.5, 69.66666666666667, 68.75, 67.91666666666667, 66.91666666666667, 65.99999999999999, 64.99999999999999, 64.25, 63.33333333333333, 62.5, 61.66666666666667, 60.83333333333333, 60.25, 59.25, 58.5, 57.58333333333333, 56.49999999999999, 55.58333333333333, 54.58333333333333, 53.583333333333336, 52.666666666666664, 51.74999999999999, 50.916666666666664, 49.99999999999999, 49.0, 48.083333333333336, 47.24999999999999, 46.25, 45.41666666666666, 44.333333333333336, 43.41666666666666, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 4, 11, 21, 27, 34, 39, 45, 51, 60, 68, 76, 81, 85, 90, 92, 97, 102, 107, 112, 115, 119, 122, 127, 130, 133, 141, 143, 148, 152, 159, 163, 168, 173, 179, 188, 196, 202, 209, 215, 223, 229, 236, 244, 255, 264, 273, 282, 291, 298, 306, 314, 325, 334, 344, 354, 365, 375, 387, 398, 410, 419, 430, 440, 450, 460, 467, 479, 488, 499, 512, 523, 535, 547, 558, 569, 579, 590, 602, 613, 623, 635, 645, 658, 669, 681, 693, 705, 717, 728, 740, 752, 764, 776, 787, 799, 810, 822, 833, 845, 857], "exact_score": [98.91666666666666, 98.66666666666667, 98.16666666666667, 97.33333333333333, 96.91666666666666, 96.33333333333333, 95.91666666666666, 95.41666666666666, 94.91666666666666, 94.16666666666667, 93.5, 92.83333333333333, 92.41666666666666, 92.08333333333333, 91.66666666666666, 91.49999999999999, 91.08333333333333, 90.66666666666666, 90.25, 89.83333333333333, 89.58333333333333, 89.25, 89.0, 88.58333333333333, 88.33333333333333, 88.08333333333334, 87.41666666666667, 87.25, 86.83333333333333, 86.5, 85.91666666666666, 85.58333333333333, 85.16666666666666, 84.74999999999999, 84.25, 83.5, 82.83333333333334, 82.33333333333333, 81.75, 81.25, 80.58333333333333, 80.0, 79.5, 78.83333333333333, 77.91666666666667, 77.16666666666666, 76.41666666666667, 75.66666666666666, 74.91666666666667, 74.33333333333333, 73.66666666666666, 73.0, 72.08333333333333, 71.33333333333334, 70.5, 69.66666666666667, 68.75, 67.83333333333333, 66.91666666666667, 65.99999999999999, 64.99999999999999, 64.25, 63.33333333333333, 62.5, 61.66666666666667, 60.83333333333333, 60.25, 59.25, 58.5, 57.49999999999999, 56.49999999999999, 55.58333333333333, 54.58333333333333, 53.583333333333336, 52.666666666666664, 51.74999999999999, 50.916666666666664, 49.99999999999999, 49.0, 48.083333333333336, 47.24999999999999, 46.25, 45.33333333333333, 44.333333333333336, 43.41666666666666, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 69.35749999999999, "threshold_area100": 69.36083333333333, "threshold_area25": 92.88333333333331}, "test": {"score": [90.16223679668623, 89.04038660683466, 88.64342423196409, 87.36624093890231, 85.70935450465998, 84.22506040731791, 82.96513634794614, 82.08491542975491, 80.47980669658267, 79.5823265447014, 78.33966171901967, 77.7183293061788, 76.82084915429755, 75.64722126337591, 75.2847773558854, 74.37003797031412, 73.85226095961339, 73.38626164998274, 73.0065585088022, 72.81670693821194, 72.62685536762167, 71.85018985157059, 71.36693130824992, 71.05626510182947, 70.71108042802899, 69.88263721090784, 69.5719710044874, 69.15774939592681, 68.67449085260614, 68.31204694511564, 67.75975146703486, 67.38004832585432, 67.05212288574387, 66.8622713151536, 66.60338280980325, 66.17190196755264, 65.70590265792198, 65.05005177770107, 64.29064549534, 63.772868484639275, 63.2205730065585, 62.94442526751811, 62.426648256817394, 62.046945115636866, 61.59820503969624, 61.16672419744563, 60.61442871936485, 60.32102174663445, 59.527096996893334, 58.750431480842245, 57.66309975837072, 57.248878149810146, 56.541249568519156, 56.21332412840869, 55.5056955471177, 54.74628926475664, 54.26303072143597, 53.76251294442527, 53.227476700034515, 52.74421815671384, 52.07110804280289, 51.12185018985157, 50.828443217121155, 50.20711080428028, 49.44770452191922, 48.94718674490852, 48.066965826717286, 47.46289264756645, 47.11770797376596, 46.65170866413531, 46.18570935450465, 45.46082153952364, 44.615119088712454, 43.890231273731445, 43.42423196410079, 42.50949257852951, 42.23334483948912, 41.715567828788394, 41.059716948567484, 40.214014497756295, 39.45460821539524, 39.05764584052468, 38.298239558163615, 37.78046254746289, 37.31446323783224, 36.710390058681384, 36.19261304798067, 35.58853986882981, 34.777355885398684, 34.4321712115982, 33.84535726613738, 33.46565412495684, 32.80980324473594, 32.20573006558508, 31.20469451156368, 30.3589920607525, 29.73765964791163, 29.18536416983085, 27.787366240938905, 27.37314463237832, 26.56196064894718], "calls": [0.0, 4.4011045909561615, 6.282361063168795, 10.131170176044185, 14.066275457369692, 17.03486365205385, 19.65826717293752, 21.453227476700036, 24.577148774594406, 25.940628236106317, 28.063513979979287, 29.150845702450813, 30.704176734552984, 32.706247842595786, 33.67276492923714, 35.69209527096997, 37.03831549879185, 37.953054884363134, 38.88505350362444, 39.59268208491543, 40.04142216085606, 41.81912323092855, 42.95823265447014, 43.734898170521234, 44.304452882292026, 46.064894718674495, 46.68622713151536, 47.4283741801864, 48.37763203313773, 49.29237141870901, 50.604073179150845, 51.294442526751816, 51.84673800483258, 52.24370037970314, 52.88229202623403, 53.693476009665176, 54.625474628926476, 55.695547117707974, 56.851915774939584, 57.7839143942009, 58.715913013462206, 59.11287538833275, 59.83776320331378, 60.33828098032448, 60.90783569209527, 61.477390403866075, 62.115982050396966, 62.51294442526751, 63.56575768035899, 64.61857093545046, 65.94753192958233, 66.51708664135313, 67.32827062478425, 67.77701070072489, 68.62271315153608, 69.6410079392475, 70.17604418363824, 70.81463583016914, 71.40144977562997, 71.97100448740076, 72.73041076976182, 73.7832240248533, 74.14566793234381, 74.88781498101484, 75.75077666551606, 76.26855367621678, 77.23507076285813, 77.89092164307905, 78.25336555056955, 78.73662409389023, 79.27166033828098, 80.03106662064205, 80.98032447359337, 81.80876769071453, 82.32654470141526, 83.27580255436658, 83.58646876078703, 84.15602347255782, 84.84639282015878, 85.72661373835003, 86.52053848809112, 86.96927856403175, 87.74594408008285, 88.29823955816362, 88.78149810148429, 89.45460821539524, 90.00690369347602, 90.66275457369693, 91.49119779081809, 91.87090093199862, 92.45771487745944, 92.87193648602002, 93.56230583362098, 94.16637901277183, 95.2191922678633, 96.06489471867448, 96.68622713151535, 97.25578184328616, 98.68829823955816, 99.1370383154988, 100.0], "rescued": [0, 22, 35, 53, 65, 70, 77, 80, 88, 88, 88, 90, 92, 94, 94, 95, 97, 97, 97, 97, 97, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106], "harmed": [0, 87, 123, 215, 323, 414, 494, 548, 649, 701, 773, 811, 865, 935, 956, 1010, 1042, 1069, 1091, 1102, 1113, 1161, 1189, 1208, 1228, 1276, 1294, 1318, 1346, 1367, 1399, 1421, 1440, 1451, 1467, 1492, 1519, 1557, 1601, 1631, 1663, 1679, 1709, 1731, 1757, 1782, 1814, 1831, 1877, 1922, 1985, 2010, 2051, 2070, 2111, 2155, 2183, 2212, 2243, 2271, 2310, 2365, 2382, 2418, 2462, 2491, 2543, 2578, 2598, 2625, 2652, 2694, 2743, 2785, 2812, 2865, 2881, 2911, 2949, 2998, 3042, 3065, 3109, 3139, 3166, 3203, 3233, 3268, 3315, 3335, 3369, 3391, 3429, 3464, 3522, 3571, 3607, 3639, 3720, 3744, 3791], "exact_score": [90.16223679668623, 89.93786675871591, 89.66171901967553, 89.47186744908525, 89.14394200897479, 88.90231273731446, 88.69520193303417, 88.38453572661373, 87.98757335174318, 87.79772178115292, 87.40075940628236, 87.09009319986191, 86.79668622713152, 86.26164998274076, 85.74387297204004, 85.29513289609942, 84.76009665170866, 84.24231964100794, 83.84535726613737, 83.37935795650672, 82.861580945806, 82.32654470141524, 81.8605453917846, 81.39454608215395, 80.75595443562305, 80.27269589230238, 79.53054884363134, 79.06454953400069, 78.37418018639973, 77.80462547462892, 77.2868484639282, 76.61373835001726, 76.02692440455642, 75.52640662754573, 75.12944425267519, 74.7152226441146, 74.28374180186398, 73.85226095961339, 73.36900241629272, 72.98929927511219, 72.62685536762167, 72.28167069382118, 71.74663444943043, 71.33241284086986, 70.86641353123922, 70.34863652053849, 69.91715567828788, 69.38211943389715, 68.91612012426648, 68.4328615809458, 67.94960303762512, 67.55264066275457, 67.00034518467379, 66.55160510873317, 65.99930963065239, 65.49879185364169, 64.86020020711081, 64.23886779426992, 63.65205384880911, 63.01346220227821, 62.34035208836728, 61.52916810493614, 60.735243355195024, 59.97583707283396, 59.16465308940282, 58.474283741801855, 57.628581290990674, 56.86917500862961, 56.040731791508456, 55.229547808077314, 54.435623058336205, 53.60717984121505, 52.726958923023815, 51.84673800483258, 50.94925785295133, 50.15533310321021, 49.22333448394891, 48.25681739730756, 47.37659647911632, 46.41007939247498, 45.49534000690369, 44.615119088712454, 43.7176389368312, 42.78564031756989, 41.87090093199862, 40.921643079047286, 39.95512599240594, 39.04038660683465, 38.073869520193306, 37.14187090093199, 36.209872281670684, 35.277873662409384, 34.31135657576804, 33.34483948912668, 32.378322402485324, 31.429064549534004, 30.42802899551259, 29.444252675181215, 28.443217121159815, 27.511218501898515, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 64.54221208208318, "threshold_area100": 56.98904038660684, "threshold_area25": 77.91681049361408}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.2334613800048828, "operator": "ge"}, {"budget": 0.02, "threshold": -2.0643463134765625, "operator": "ge"}, {"budget": 0.03, "threshold": -2.6798975467681885, "operator": "ge"}, {"budget": 0.04, "threshold": -3.1501142978668213, "operator": "ge"}, {"budget": 0.05, "threshold": -3.669417142868042, "operator": "ge"}, {"budget": 0.06, "threshold": -4.181271553039551, "operator": "ge"}, {"budget": 0.07, "threshold": -4.611806869506836, "operator": "ge"}, {"budget": 0.08, "threshold": -4.8225531578063965, "operator": "ge"}, {"budget": 0.09, "threshold": -5.078166961669922, "operator": "ge"}, {"budget": 0.1, "threshold": -5.438965797424316, "operator": "ge"}, {"budget": 0.11, "threshold": -5.684579849243164, "operator": "ge"}, {"budget": 0.12, "threshold": -5.839029788970947, "operator": "ge"}, {"budget": 0.13, "threshold": -5.990607261657715, "operator": "ge"}, {"budget": 0.14, "threshold": -6.139204025268555, "operator": "ge"}, {"budget": 0.15, "threshold": -6.311241626739502, "operator": "ge"}, {"budget": 0.16, "threshold": -6.475531101226807, "operator": "ge"}, {"budget": 0.17, "threshold": -6.631926536560059, "operator": "ge"}, {"budget": 0.18, "threshold": -6.7513837814331055, "operator": "ge"}, {"budget": 0.19, "threshold": -6.827484130859375, "operator": "ge"}, {"budget": 0.2, "threshold": -6.93147611618042, "operator": "ge"}, {"budget": 0.21, "threshold": -7.033535003662109, "operator": "ge"}, {"budget": 0.22, "threshold": -7.2325053215026855, "operator": "ge"}, {"budget": 0.23, "threshold": -7.407278060913086, "operator": "ge"}, {"budget": 0.24, "threshold": -7.580160617828369, "operator": "ge"}, {"budget": 0.25, "threshold": -7.720475196838379, "operator": "ge"}, {"budget": 0.26, "threshold": -8.001514434814453, "operator": "ge"}, {"budget": 0.27, "threshold": -8.171192169189453, "operator": "ge"}, {"budget": 0.28, "threshold": -8.262948036193848, "operator": "ge"}, {"budget": 0.29, "threshold": -8.371583938598633, "operator": "ge"}, {"budget": 0.3, "threshold": -8.511863708496094, "operator": "ge"}, {"budget": 0.31, "threshold": -8.648561477661133, "operator": "ge"}, {"budget": 0.32, "threshold": -8.718362808227539, "operator": "ge"}, {"budget": 0.33, "threshold": -8.852940559387207, "operator": "ge"}, {"budget": 0.34, "threshold": -8.909941673278809, "operator": "ge"}, {"budget": 0.35, "threshold": -9.001968383789062, "operator": "ge"}, {"budget": 0.36, "threshold": -9.14199447631836, "operator": "ge"}, {"budget": 0.37, "threshold": -9.246193885803223, "operator": "ge"}, {"budget": 0.38, "threshold": -9.35572624206543, "operator": "ge"}, {"budget": 0.39, "threshold": -9.448408126831055, "operator": "ge"}, {"budget": 0.4, "threshold": -9.564777374267578, "operator": "ge"}, {"budget": 0.41, "threshold": -9.660431861877441, "operator": "ge"}, {"budget": 0.42, "threshold": -9.835567474365234, "operator": "ge"}, {"budget": 0.43, "threshold": -9.951797485351562, "operator": "ge"}, {"budget": 0.44, "threshold": -10.086734771728516, "operator": "ge"}, {"budget": 0.45, "threshold": -10.190652847290039, "operator": "ge"}, {"budget": 0.46, "threshold": -10.255531311035156, "operator": "ge"}, {"budget": 0.47, "threshold": -10.376455307006836, "operator": "ge"}, {"budget": 0.48, "threshold": -10.46585464477539, "operator": "ge"}, {"budget": 0.49, "threshold": -10.643064498901367, "operator": "ge"}, {"budget": 0.5, "threshold": -10.77701473236084, "operator": "ge"}, {"budget": 0.51, "threshold": -10.89891242980957, "operator": "ge"}, {"budget": 0.52, "threshold": -10.965646743774414, "operator": "ge"}, {"budget": 0.53, "threshold": -11.088808059692383, "operator": "ge"}, {"budget": 0.54, "threshold": -11.135111808776855, "operator": "ge"}, {"budget": 0.55, "threshold": -11.21792984008789, "operator": "ge"}, {"budget": 0.56, "threshold": -11.259868621826172, "operator": "ge"}, {"budget": 0.57, "threshold": -11.291940689086914, "operator": "ge"}, {"budget": 0.58, "threshold": -11.373044967651367, "operator": "ge"}, {"budget": 0.59, "threshold": -11.43155288696289, "operator": "ge"}, {"budget": 0.6, "threshold": -11.463493347167969, "operator": "ge"}, {"budget": 0.61, "threshold": -11.519563674926758, "operator": "ge"}, {"budget": 0.62, "threshold": -11.563699722290039, "operator": "ge"}, {"budget": 0.63, "threshold": -11.608370780944824, "operator": "ge"}, {"budget": 0.64, "threshold": -11.675646781921387, "operator": "ge"}, {"budget": 0.65, "threshold": -11.710464477539062, "operator": "ge"}, {"budget": 0.66, "threshold": -11.78122615814209, "operator": "ge"}, {"budget": 0.67, "threshold": -11.847488403320312, "operator": "ge"}, {"budget": 0.68, "threshold": -11.919282913208008, "operator": "ge"}, {"budget": 0.69, "threshold": -11.967964172363281, "operator": "ge"}, {"budget": 0.7, "threshold": -12.024632453918457, "operator": "ge"}, {"budget": 0.71, "threshold": -12.068136215209961, "operator": "ge"}, {"budget": 0.72, "threshold": -12.11375617980957, "operator": "ge"}, {"budget": 0.73, "threshold": -12.160348892211914, "operator": "ge"}, {"budget": 0.74, "threshold": -12.193585395812988, "operator": "ge"}, {"budget": 0.75, "threshold": -12.238128662109375, "operator": "ge"}, {"budget": 0.76, "threshold": -12.278036117553711, "operator": "ge"}, {"budget": 0.77, "threshold": -12.303524017333984, "operator": "ge"}, {"budget": 0.78, "threshold": -12.34054946899414, "operator": "ge"}, {"budget": 0.79, "threshold": -12.383853912353516, "operator": "ge"}, {"budget": 0.8, "threshold": -12.446000099182129, "operator": "ge"}, {"budget": 0.81, "threshold": -12.505677223205566, "operator": "ge"}, {"budget": 0.82, "threshold": -12.533935546875, "operator": "ge"}, {"budget": 0.83, "threshold": -12.591981887817383, "operator": "ge"}, {"budget": 0.84, "threshold": -12.67910385131836, "operator": "ge"}, {"budget": 0.85, "threshold": -12.744095802307129, "operator": "ge"}, {"budget": 0.86, "threshold": -12.8090181350708, "operator": "ge"}, {"budget": 0.87, "threshold": -12.88571548461914, "operator": "ge"}, {"budget": 0.88, "threshold": -12.938558578491211, "operator": "ge"}, {"budget": 0.89, "threshold": -12.991842269897461, "operator": "ge"}, {"budget": 0.9, "threshold": -13.065435409545898, "operator": "ge"}, {"budget": 0.91, "threshold": -13.111279487609863, "operator": "ge"}, {"budget": 0.92, "threshold": -13.159854888916016, "operator": "ge"}, {"budget": 0.93, "threshold": -13.204933166503906, "operator": "ge"}, {"budget": 0.94, "threshold": -13.350470542907715, "operator": "ge"}, {"budget": 0.95, "threshold": -13.423637390136719, "operator": "ge"}, {"budget": 0.96, "threshold": -13.495964050292969, "operator": "ge"}, {"budget": 0.97, "threshold": -13.611005783081055, "operator": "ge"}, {"budget": 0.98, "threshold": -13.717793464660645, "operator": "ge"}, {"budget": 0.99, "threshold": -13.955465316772461, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.75, 98.25, 97.83333333333333, 97.33333333333333, 96.83333333333333, 96.5, 95.83333333333333, 95.41666666666666, 94.91666666666666, 94.75, 94.41666666666666, 93.75, 93.16666666666666, 92.91666666666667, 92.75, 92.58333333333333, 92.16666666666666, 91.91666666666667, 91.33333333333333, 91.16666666666666, 90.83333333333333, 90.58333333333333, 89.99999999999999, 89.5, 88.91666666666667, 88.66666666666666, 88.25, 87.66666666666666, 87.16666666666666, 86.75, 86.25, 85.58333333333333, 85.33333333333333, 84.83333333333333, 84.33333333333333, 83.74999999999999, 83.0, 82.33333333333333, 81.83333333333334, 81.08333333333333, 80.58333333333333, 80.0, 79.33333333333333, 78.58333333333333, 77.75, 77.08333333333333, 76.41666666666667, 75.58333333333334, 74.83333333333333, 74.0, 73.25, 72.58333333333333, 71.83333333333333, 70.91666666666666, 70.16666666666667, 69.24999999999999, 68.41666666666666, 67.66666666666666, 66.66666666666666, 65.83333333333333, 64.83333333333333, 63.916666666666664, 63.16666666666666, 62.33333333333333, 61.33333333333333, 60.416666666666664, 59.5, 58.5, 57.666666666666664, 56.58333333333333, 55.58333333333333, 54.666666666666664, 53.833333333333336, 53.0, 52.166666666666664, 51.16666666666667, 50.16666666666667, 49.25, 48.24999999999999, 47.24999999999999, 46.33333333333333, 45.41666666666666, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 3, 9, 15, 22, 28, 32, 40, 45, 51, 53, 57, 65, 72, 75, 77, 79, 84, 87, 94, 96, 100, 103, 110, 116, 123, 126, 131, 138, 144, 149, 155, 163, 166, 172, 178, 185, 194, 202, 208, 217, 223, 230, 238, 247, 257, 265, 273, 283, 292, 302, 311, 319, 328, 339, 348, 359, 369, 378, 390, 400, 412, 423, 432, 442, 454, 465, 476, 488, 498, 511, 523, 534, 544, 554, 564, 576, 588, 599, 611, 623, 634, 645, 658, 670, 682, 693, 705, 717, 729, 741, 753, 765, 775, 786, 798, 809, 821, 833, 845, 857], "exact_score": [98.91666666666666, 98.75, 98.25, 97.83333333333333, 97.33333333333333, 96.83333333333333, 96.5, 95.83333333333333, 95.41666666666666, 94.91666666666666, 94.75, 94.41666666666666, 93.75, 93.16666666666666, 92.91666666666667, 92.75, 92.58333333333333, 92.16666666666666, 91.91666666666667, 91.33333333333333, 91.16666666666666, 90.83333333333333, 90.58333333333333, 89.99999999999999, 89.5, 88.91666666666667, 88.66666666666666, 88.25, 87.66666666666666, 87.16666666666666, 86.75, 86.25, 85.58333333333333, 85.33333333333333, 84.83333333333333, 84.33333333333333, 83.74999999999999, 83.0, 82.33333333333333, 81.83333333333334, 81.08333333333333, 80.5, 80.0, 79.33333333333333, 78.58333333333333, 77.75, 77.08333333333333, 76.41666666666667, 75.58333333333334, 74.83333333333333, 74.0, 73.25, 72.58333333333333, 71.83333333333333, 70.91666666666666, 70.16666666666667, 69.24999999999999, 68.41666666666666, 67.66666666666666, 66.66666666666666, 65.83333333333333, 64.83333333333333, 63.916666666666664, 63.16666666666666, 62.33333333333333, 61.33333333333333, 60.416666666666664, 59.5, 58.5, 57.58333333333333, 56.58333333333333, 55.58333333333333, 54.666666666666664, 53.833333333333336, 53.0, 52.166666666666664, 51.16666666666667, 50.16666666666667, 49.25, 48.24999999999999, 47.24999999999999, 46.33333333333333, 45.33333333333333, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.583333333333336, 33.666666666666664, 32.666666666666664, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 69.8925, "threshold_area100": 69.895, "threshold_area25": 93.88333333333334}, "test": {"score": [90.16223679668623, 89.54090438384536, 88.78149810148429, 87.83224024853297, 86.93476009665171, 85.64031756989989, 83.46565412495684, 82.15395236451502, 81.39454608215395, 80.39351052813255, 78.87469796341043, 78.11529168104936, 77.56299620296858, 76.95892302381773, 76.40662754573697, 75.68173973075595, 75.2157404211253, 74.61166724197446, 74.24922333448394, 73.90403866068347, 73.64515015533311, 73.36900241629272, 72.6441146013117, 72.05730065585088, 71.76389368312046, 71.29789437348981, 70.27959958577839, 69.7100448740076, 69.29582326544701, 68.76078702105626, 68.00138073869519, 67.3627890921643, 67.19019675526407, 66.5688643424232, 66.41353123921297, 66.05108733172247, 65.34345875043148, 64.80842250604073, 64.29064549534, 63.89368312046945, 63.35864687607871, 62.87538833275802, 62.150500517777004, 61.563686572316186, 61.011391094235414, 60.39005868139454, 60.01035554021401, 59.63065239903348, 59.078356920952714, 57.973765964791156, 57.23161891612012, 56.57576803589921, 55.95443562305833, 54.97065930272696, 54.67725232999654, 53.79703141180532, 53.48636520538488, 53.19295823265446, 52.76147739040387, 52.399033482913346, 52.05384880911288, 51.39799792889195, 50.9319986192613, 50.5005177770107, 49.79288919571971, 49.534000690369346, 48.56748360372799, 47.80807732136692, 46.910597169485676, 46.237487055574725, 45.46082153952364, 45.02934069727304, 44.511563686572316, 43.94200897480152, 43.51052813255091, 42.87193648602002, 42.35415947531929, 41.95719710044873, 41.37038315498792, 40.990680013807385, 40.24853296513635, 39.59268208491543, 39.24749741111494, 38.45357266137383, 37.297204004142216, 36.5032792544011, 35.674836037279945, 34.777355885398684, 34.15602347255782, 33.63824646185709, 32.91335864687608, 32.447359337245416, 31.964100793924743, 31.39454608215395, 30.445288229202617, 29.944770452191914, 29.46151190887124, 28.857438729720396, 28.253365550569555, 27.286848463928195, 26.56196064894718], "calls": [0.0, 4.1076976182257505, 7.317915084570245, 10.131170176044185, 12.909906800138074, 15.9993096306524, 20.0724887814981, 23.006558508802208, 24.30100103555402, 25.87159130134622, 28.71936486020021, 30.324473593372453, 31.532619951674146, 32.706247842595786, 33.94891266827753, 35.398688298239556, 36.69313082499137, 38.367276492923715, 39.24749741111494, 40.11045909561615, 40.7835692095271, 41.45667932343804, 42.85467725233, 43.76941663790127, 44.56334138764239, 45.25371073524336, 46.996893337935795, 47.9979288919572, 48.67103900586814, 49.534000690369346, 51.07007248878149, 52.381774249223334, 52.76147739040387, 53.76251294442527, 54.09043838453572, 54.6427338626165, 55.712806351398, 56.50673110113911, 57.19710044874008, 57.749395926820846, 58.42250604073179, 59.09561615464274, 60.217466344494305, 60.95961339316535, 61.701760441836385, 62.56472212633759, 62.97894373489817, 63.634794615119084, 64.39420089748015, 65.77493959268207, 66.65516051087332, 67.46634449430445, 68.24301001035555, 69.39937866758716, 69.7790818087677, 70.88367276492924, 71.22885743872972, 71.55678287884018, 72.09181912323092, 72.52329996548154, 72.93752157404211, 73.67966862271315, 74.19744563341388, 74.68070417673457, 75.45736969278563, 75.7852951328961, 76.83810838798757, 77.73558853986883, 78.70210562651019, 79.39247497411115, 80.30721435968243, 80.77321366931308, 81.32550914739386, 81.89506385916465, 82.37832240248532, 83.0686917500863, 83.62098722816707, 84.01794960303762, 84.62202278218847, 85.01898515705902, 85.8474283741802, 86.55505695547117, 86.90024162927166, 87.7114256127028, 88.88505350362443, 89.69623748705557, 90.54193993786676, 91.45667932343804, 92.09527096996894, 92.61304798066966, 93.35519502934069, 93.83845357266138, 94.33897134967208, 94.9257852951329, 95.90956161546428, 96.46185709354505, 96.96237487055575, 97.58370728339662, 98.23955816361753, 99.25785295132896, 100.0], "rescued": [0, 11, 32, 39, 50, 63, 73, 79, 80, 81, 87, 90, 93, 93, 94, 95, 95, 97, 97, 97, 97, 97, 97, 97, 97, 97, 98, 98, 98, 98, 99, 99, 99, 100, 100, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106], "harmed": [0, 47, 112, 174, 237, 325, 461, 543, 588, 647, 741, 788, 823, 858, 891, 934, 961, 998, 1019, 1039, 1054, 1070, 1112, 1146, 1163, 1190, 1250, 1283, 1307, 1338, 1383, 1420, 1430, 1467, 1476, 1498, 1540, 1571, 1601, 1624, 1655, 1683, 1725, 1759, 1791, 1827, 1849, 1871, 1903, 1967, 2010, 2048, 2084, 2141, 2158, 2209, 2227, 2244, 2269, 2290, 2310, 2348, 2375, 2400, 2442, 2457, 2513, 2558, 2610, 2649, 2694, 2719, 2749, 2782, 2807, 2845, 2875, 2898, 2932, 2954, 2997, 3035, 3055, 3101, 3168, 3214, 3262, 3314, 3350, 3380, 3422, 3449, 3477, 3510, 3565, 3595, 3623, 3658, 3693, 3749, 3791], "exact_score": [90.16223679668623, 90.12771832930618, 89.95512599240594, 89.69623748705557, 89.55816361753539, 89.45460821539524, 89.2129789437349, 88.88505350362443, 88.62616499827408, 88.26372109078356, 87.84949948222298, 87.5733517431826, 87.26268553676216, 86.88298239558164, 86.34794615119088, 86.0717984121505, 85.64031756989989, 85.13979979288919, 84.76009665170866, 84.1387642388678, 83.53469105971695, 83.08595098377631, 82.60269244045564, 82.17121159820505, 81.60165688643424, 80.91128753883328, 80.3589920607525, 79.9275112185019, 79.23714187090093, 78.7884017949603, 78.30514325163962, 77.7873662409389, 77.39040386606834, 76.7863306869175, 76.37210907835691, 75.92336900241628, 75.52640662754573, 75.12944425267519, 74.7152226441146, 74.37003797031412, 73.97307559544356, 73.55885398688298, 73.05833620987228, 72.60959613393166, 71.98826372109079, 71.47048671039006, 70.81463583016914, 70.27959958577839, 69.7100448740076, 69.08871246116672, 68.50189851570589, 68.03589920607524, 67.55264066275457, 67.0693821194339, 66.46530894028305, 65.7921988263721, 65.13634794615119, 64.44597859855021, 63.73835001725923, 62.96168450120814, 62.2885743872972, 61.54642733862617, 60.787021056265104, 60.01035554021401, 59.33724542630308, 58.64687607870211, 57.749395926820846, 56.92095270969968, 56.17880566102864, 55.350362443907485, 54.48740075940628, 53.69347600966516, 52.86503279254401, 52.0020711080428, 51.12185018985157, 50.25888850535036, 49.34414911977908, 48.412150500517775, 47.54918881601656, 46.61719019675526, 45.70245081118398, 44.804970659302725, 43.838453572661365, 42.94097342078012, 41.991715567828784, 41.02519848118743, 40.11045909561615, 39.16120124266482, 38.177424922333444, 37.19364860200207, 36.22713151536071, 35.22609596133931, 34.25957887469796, 33.25854332067656, 32.30928546772523, 31.32550914739385, 30.376251294442525, 29.426993441491188, 28.477735588539865, 27.545736969278558, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 65.02614830501456, "threshold_area100": 57.102088367276494, "threshold_area25": 78.91439420089749}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 1.4152571625819814}}, "old_normalized_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.17949603037624, "calls": 1.794960303762513}}, "old_normalized_endpoints": {"val": {"score": 98.91666666666666, "calls": 0.0}, "test": {"score": 90.16223679668623, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.58333333333333, 98.33333333333333, 98.25, 98.16666666666667, 97.91666666666666, 97.83333333333333, 97.66666666666667, 97.33333333333333, 97.33333333333333, 97.24999999999999, 97.0, 96.66666666666667, 96.58333333333333, 96.0, 95.58333333333333, 94.91666666666666, 94.75, 94.16666666666667, 93.41666666666666, 92.75, 91.83333333333333, 91.33333333333333, 90.75, 90.16666666666666, 89.5, 88.58333333333333, 87.83333333333333, 87.0, 86.08333333333333, 85.25, 84.41666666666666, 83.58333333333333, 82.58333333333333, 81.75, 80.83333333333333, 79.91666666666666, 79.16666666666666, 78.41666666666667, 77.58333333333333, 76.66666666666666, 75.75, 74.75, 73.75, 72.83333333333333, 71.91666666666666, 70.91666666666666, 69.91666666666667, 69.08333333333333, 68.08333333333333, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.25, 63.24999999999999, 62.33333333333333, 61.58333333333332, 60.58333333333332, 59.583333333333336, 58.58333333333333, 57.833333333333336, 56.749999999999986, 55.833333333333336, 54.833333333333336, 54.08333333333333, 53.08333333333333, 52.08333333333333, 51.08333333333333, 50.08333333333332, 49.166666666666664, 48.16666666666667, 47.24999999999999, 46.25, 45.33333333333333, 44.25, 43.25, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.58333333333333, 98.33333333333333, 98.25, 98.16666666666667, 97.91666666666666, 97.83333333333333, 97.66666666666667, 97.33333333333333, 97.33333333333333, 97.24999999999999, 97.0, 96.66666666666667, 96.58333333333333, 96.0, 95.58333333333333, 94.91666666666666, 94.75, 94.16666666666667, 93.41666666666666, 92.75, 91.83333333333333, 91.33333333333333, 90.75, 90.16666666666666, 89.5, 88.58333333333333, 87.83333333333333, 87.0, 86.08333333333333, 85.25, 84.41666666666666, 83.5, 82.58333333333333, 81.75, 80.83333333333333, 79.91666666666666, 79.16666666666666, 78.41666666666667, 77.58333333333333, 76.66666666666666, 75.75, 74.75, 73.75, 72.83333333333333, 71.91666666666666, 70.91666666666666, 69.91666666666667, 69.0, 68.08333333333333, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.25, 63.24999999999999, 62.33333333333333, 61.58333333333332, 60.58333333333332, 59.583333333333336, 58.58333333333333, 57.74999999999999, 56.749999999999986, 55.833333333333336, 54.833333333333336, 54.08333333333333, 53.08333333333333, 52.08333333333333, 51.08333333333333, 50.08333333333332, 49.166666666666664, 48.16666666666667, 47.24999999999999, 46.25, 45.25, 44.25, 43.25, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.83333333333333, "exact_area100": 71.94333333333333}, "test": {"score": [90.16223679668623, 90.17949603037624, 90.17949603037624, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.02416292716603, 90.006903693476, 89.83431135657577, 89.7134967207456, 89.62720055229548, 89.60994131860545, 89.40283051432516, 89.29927511218501, 89.19571971004487, 89.0058681394546, 88.69520193303417, 88.41905419399379, 88.26372109078356, 88.0048325854332, 87.79772178115292, 87.52157404211253, 87.2109078356921, 86.91750086296169, 86.45150155333103, 86.0027614773904, 85.45046599930963, 84.86365205384881, 84.20780117362789, 83.74180186399724, 83.20676561960649, 82.4991370383155, 81.7224715222644, 81.15291681049361, 80.32447359337245, 79.56506731101139, 78.87469796341043, 78.3569209527097, 77.45944080082843, 76.52744218156712, 75.99240593717639, 75.11218501898516, 73.43803935105281, 72.5750776665516, 71.83293061788056, 70.78011736278908, 70.08974801518812, 68.65723161891611, 68.01863997238522, 66.8622713151536, 66.39627200552295, 65.51605108733172, 64.61857093545046, 63.89368312046945, 63.16879530548844, 62.68553676216775, 61.77079737659648, 60.83879875733518, 60.2865032792544, 59.33724542630308, 58.59509837763204, 58.00828443217121, 56.523990334829136, 55.79910251984812, 55.29858474283742, 54.349326889886086, 53.727994477045215, 52.951328960994125, 52.19192267863306, 50.88022091819122, 49.63755609250949, 48.981705212288574, 47.928891957197095, 47.134967207455986, 46.56541249568519, 45.944080082844316, 45.49534000690369, 44.77045219192268, 44.33897134967207, 43.13082499137038, 41.81912323092855, 41.04245771487746, 40.075940628236104, 39.6444597859855, 38.83327580255437, 37.78046254746289, 37.12461166724197, 36.64135312392129, 36.037279944770454, 35.174318260269246, 34.38039351052813, 33.690024162927166, 33.06869175008629, 32.06765619606489, 31.463583016914043, 30.497065930272694, 29.426993441491188, 28.52951328960994, 27.52847773558854, 26.56196064894718], "calls": [0.0, 1.794960303762513, 2.416292716603383, 3.7452537107352435, 4.366586123576114, 4.884363134276838, 6.023472557818433, 7.162581981360027, 8.62961684501208, 9.90680013807387, 11.511908871246117, 12.426648256817398, 13.134276838108388, 14.463237832240248, 15.895754228512255, 16.724197445633415, 17.53538142906455, 18.65723161891612, 19.796341042457716, 20.296858819468415, 21.34967207455989, 22.437003797031412, 23.38626164998274, 24.490852606144287, 25.146703486365208, 26.354849844666894, 27.355885398688294, 28.305143251639624, 29.461511908871245, 30.72143596824301, 31.4981014842941, 32.550914739385576, 33.86261649982741, 35.19157749395927, 36.19261304798067, 37.52157404211253, 38.90231273731447, 39.83431135657577, 40.490162236796685, 41.560234725578184, 42.85467725233, 43.61408353469106, 45.11563686572316, 47.497411114946495, 48.774594408008284, 49.72385226095961, 50.98377632033137, 51.81221953745254, 53.572661373835004, 54.47014152571625, 55.91991715567829, 56.48947186744908, 57.525025888850536, 58.64687607870211, 59.50983776320331, 60.40731791508457, 61.02865032792544, 62.046945115636866, 63.08249913703832, 63.65205384880911, 64.67034863652053, 65.55056955471177, 66.25819813600276, 67.9323438039351, 68.81256472212634, 69.46841560234725, 70.55574732481877, 71.21159820503969, 72.05730065585088, 72.88574387297204, 74.283741801864, 75.5781843286158, 76.35484984466689, 77.5802554366586, 78.42595788746979, 79.08180876769072, 79.7204004142216, 80.16914049016223, 80.9975837072834, 81.429064549534, 82.70624784259579, 84.10424577148775, 84.94994822229893, 85.9337245426303, 86.36520538488092, 87.24542630307215, 88.36727649292372, 89.02312737314463, 89.50638591646532, 90.17949603037624, 91.04245771487746, 91.88816016568865, 92.61304798066966, 93.23438039351053, 94.26993441491199, 94.9257852951329, 95.89230238177426, 96.9968933379358, 97.9288919571971, 98.9989644459786, 100.0], "exact_score": [90.16223679668623, 90.17949603037624, 90.17949603037624, 90.17949603037624, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.04142216085606, 90.006903693476, 89.98964445978598, 89.83431135657577, 89.76527442181566, 89.69623748705557, 89.60994131860545, 89.43734898170521, 89.3683120469451, 89.28201587849499, 89.12668277528478, 88.88505350362443, 88.574387297204, 88.33275802554367, 88.05661028650327, 87.91853641698309, 87.69416637901277, 87.36624093890231, 86.98653779772178, 86.62409389023126, 86.10631687953054, 85.67483603727995, 85.10528132550914, 84.5012081463583, 84.05246807041767, 83.5001725923369, 82.9306178805661, 82.44735933724542, 81.82602692440454, 81.30824991370382, 80.60062133241284, 80.13462202278218, 79.4960303762513, 78.73662409389023, 77.97721781152916, 77.13151536071798, 76.45840524680703, 75.75077666551606, 75.23299965481533, 74.43907490507421, 73.88677942699344, 73.0755954435623, 72.41974456334138, 71.60856057991025, 70.78011736278908, 69.93441491197791, 69.10597169485675, 68.38108387987573, 67.62167759751468, 66.81049361408353, 65.98205039696236, 65.17086641353124, 64.35968243010011, 63.51397997928891, 62.720055229547796, 61.822575077666556, 60.92509492578528, 59.97583707283396, 59.026579219882635, 58.23265447014152, 57.35243355195029, 56.50673110113911, 55.67828788401794, 54.79806696582672, 53.935105281325505, 53.02036589575423, 52.12288574387297, 51.15636865723161, 50.20711080428028, 49.27511218501898, 48.46392820158785, 47.54918881601656, 46.65170866413531, 45.667932343803926, 44.77045219192268, 43.80393510528132, 42.85467725232999, 41.905419399378665, 41.00793924749741, 40.02416292716603, 39.074905074214705, 38.1429064549534, 37.15913013462202, 36.22713151536071, 35.22609596133931, 34.276838108387985, 33.31032102174664, 32.34380393510528, 31.39454608215395, 30.393510528132552, 29.426993441491188, 28.46047635484984, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 89.0731791508457, "exact_area100": 67.23878712805516}}, "old_normalized_endpoint_budget": 0}, {"seed": 2027, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.25, "validation_area100": 0.7195, "training_seconds": 3.0902553459163755, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.03342148085513866, "operator": "ge"}, {"budget": 0.02, "threshold": -0.03928115152862613, "operator": "ge"}, {"budget": 0.03, "threshold": -0.04334621216203399, "operator": "ge"}, {"budget": 0.04, "threshold": -0.04958435258786851, "operator": "ge"}, {"budget": 0.05, "threshold": -0.05422049990340706, "operator": "ge"}, {"budget": 0.06, "threshold": -0.06187153502517707, "operator": "ge"}, {"budget": 0.07, "threshold": -0.06812600595586103, "operator": "ge"}, {"budget": 0.08, "threshold": -0.08053927194539161, "operator": "ge"}, {"budget": 0.09, "threshold": -0.08826592112852516, "operator": "ge"}, {"budget": 0.1, "threshold": -0.09910084232652568, "operator": "ge"}, {"budget": 0.11, "threshold": -0.11811765700396405, "operator": "ge"}, {"budget": 0.12, "threshold": -0.1403755315627231, "operator": "ge"}, {"budget": 0.13, "threshold": -0.17219128145957555, "operator": "ge"}, {"budget": 0.14, "threshold": -0.1946826385613445, "operator": "ge"}, {"budget": 0.15, "threshold": -0.22583432088544939, "operator": "ge"}, {"budget": 0.16, "threshold": -0.26887269480468456, "operator": "ge"}, {"budget": 0.17, "threshold": -0.2853066585730447, "operator": "ge"}, {"budget": 0.18, "threshold": -0.3125254318104028, "operator": "ge"}, {"budget": 0.19, "threshold": -0.3375878747956235, "operator": "ge"}, {"budget": 0.2, "threshold": -0.36963710932300453, "operator": "ge"}, {"budget": 0.21, "threshold": -0.3944223347083226, "operator": "gt"}, {"budget": 0.22, "threshold": -0.421684129504293, "operator": "ge"}, {"budget": 0.23, "threshold": -0.456492489847627, "operator": "ge"}, {"budget": 0.24, "threshold": -0.4804349853653966, "operator": "ge"}, {"budget": 0.25, "threshold": -0.505822083332807, "operator": "ge"}, {"budget": 0.26, "threshold": -0.5478930421963053, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5913301809159472, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6284728286831445, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6495367124391799, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6768501592135059, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6951024265240331, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7203546645560578, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7455232130491035, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7711582092731125, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7914105727369658, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8080886500529838, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8237988872299817, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8384430482175251, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8470491806747449, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8539227976195758, "operator": "ge"}, {"budget": 0.41, "threshold": -0.8599950271145977, "operator": "ge"}, {"budget": 0.42, "threshold": -0.868171784358982, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8792989421107676, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8901589711771231, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8963024467567103, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9004133892483903, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9050156008628949, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9101979193998762, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9161008809804679, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9223230104344229, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9245661432494466, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9267386881111649, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9316793414573568, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9356288618782684, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9401525188289, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9447274871902691, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9483376003986349, "operator": "ge"}, {"budget": 0.58, "threshold": -0.950685465477805, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9539078155119264, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9572098878308251, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9597780246423037, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9625514955392293, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9638943943986273, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9649132972004971, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9658672313634572, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9670871213923231, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9679973490888615, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9694034234494752, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9705271900761737, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9720419857335232, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9729096950003638, "operator": "ge"}, {"budget": 0.72, "threshold": -0.973993854049906, "operator": "ge"}, {"budget": 0.73, "threshold": -0.975681164573791, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9771251744806824, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9787916968812246, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9800601072896706, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9814860228535272, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9826186810923652, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9835324285935407, "operator": "ge"}, {"budget": 0.8, "threshold": -0.984566513458882, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9856655535098148, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9868942515949385, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9878626452792204, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9884820575942188, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9891450018835776, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9895911978318792, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9899593155280197, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9904131796174761, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9909187800617801, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9913253884480003, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9916329335741257, "operator": "ge"}, {"budget": 0.92, "threshold": -0.992047685820248, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9924247489374275, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9928452354867179, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9932709400937435, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9937093144800158, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9940068143297361, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9943618017930365, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9951910981137031, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.5, 98.41666666666666, 98.33333333333333, 98.16666666666667, 98.0, 97.74999999999999, 97.58333333333333, 97.33333333333333, 97.24999999999999, 97.08333333333333, 96.91666666666666, 96.58333333333333, 96.5, 96.16666666666667, 95.91666666666666, 95.58333333333333, 95.08333333333333, 94.5, 94.08333333333333, 93.5, 92.91666666666667, 92.41666666666666, 91.66666666666666, 90.91666666666667, 90.25, 89.5, 88.5, 87.66666666666666, 87.08333333333333, 86.16666666666666, 85.5, 84.66666666666667, 83.83333333333333, 82.91666666666666, 82.0, 81.16666666666667, 80.33333333333333, 79.41666666666667, 78.41666666666667, 77.58333333333333, 76.83333333333333, 75.83333333333333, 74.83333333333333, 73.91666666666666, 72.91666666666666, 71.91666666666666, 71.0, 70.08333333333333, 69.24999999999999, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.41666666666666, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.416666666666664, 60.5, 59.583333333333336, 58.666666666666664, 57.91666666666666, 56.833333333333336, 55.91666666666666, 54.91666666666666, 53.91666666666666, 52.91666666666666, 52.0, 51.08333333333333, 50.08333333333332, 49.08333333333333, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.33333333333333, 44.25, 43.333333333333336, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.5, 32.58333333333333, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.666666666666664, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 20.916666666666668, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 1, 1, 1, 1, 1, 1, 2, 3, 5, 6, 7, 9, 11, 14, 16, 19, 20, 23, 25, 30, 31, 35, 38, 42, 48, 55, 60, 67, 75, 81, 90, 99, 107, 116, 128, 138, 145, 156, 164, 174, 184, 195, 206, 216, 226, 237, 249, 259, 268, 280, 292, 303, 315, 327, 338, 349, 359, 372, 384, 396, 405, 417, 429, 441, 453, 464, 475, 486, 495, 508, 519, 531, 543, 555, 566, 577, 589, 601, 613, 624, 636, 646, 659, 670, 681, 693, 705, 717, 729, 741, 753, 764, 776, 788, 799, 811, 823, 834, 846, 857], "exact_score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.5, 98.41666666666666, 98.33333333333333, 98.16666666666667, 98.0, 97.74999999999999, 97.58333333333333, 97.33333333333333, 97.24999999999999, 97.08333333333333, 96.91666666666666, 96.58333333333333, 96.5, 96.16666666666667, 95.91666666666666, 95.58333333333333, 95.08333333333333, 94.5, 94.08333333333333, 93.5, 92.91666666666667, 92.41666666666666, 91.66666666666666, 90.91666666666667, 90.25, 89.5, 88.5, 87.66666666666666, 87.08333333333333, 86.16666666666666, 85.5, 84.66666666666667, 83.74999999999999, 82.91666666666666, 82.0, 81.16666666666667, 80.33333333333333, 79.41666666666667, 78.41666666666667, 77.58333333333333, 76.83333333333333, 75.83333333333333, 74.83333333333333, 73.91666666666666, 72.91666666666666, 71.91666666666666, 71.0, 70.08333333333333, 69.16666666666667, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.41666666666666, 64.41666666666667, 63.416666666666664, 62.416666666666664, 61.416666666666664, 60.5, 59.583333333333336, 58.666666666666664, 57.833333333333336, 56.833333333333336, 55.91666666666666, 54.91666666666666, 53.91666666666666, 52.91666666666666, 52.0, 51.08333333333333, 50.08333333333332, 49.08333333333333, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.25, 44.25, 43.333333333333336, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.5, 32.58333333333333, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.666666666666664, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.95, "threshold_area100": 71.95333333333335, "threshold_area25": 97.68333333333334}, "test": {"score": [90.16223679668623, 90.17949603037624, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.07594062823611, 90.04142216085606, 90.04142216085606, 89.93786675871591, 89.92060752502589, 89.83431135657577, 89.79979288919571, 89.74801518812563, 89.57542285122541, 89.40283051432516, 89.26475664480496, 88.97134967207455, 88.79875733517432, 88.48809112875388, 88.2119433897135, 87.84949948222298, 87.62512944425266, 87.40075940628236, 87.03831549879185, 86.64135312392129, 86.19261304798067, 85.39868829823956, 84.44943044528823, 83.48291335864687, 82.861580945806, 82.18847083189506, 81.8605453917846, 80.91128753883328, 80.08284432171212, 78.96099413186055, 78.01173627890921, 77.11425612702796, 76.25129444252676, 75.26751812219537, 74.64618570935451, 74.0766309975837, 73.21366931308249, 72.22989299275112, 70.93545046599931, 69.67552640662754, 68.77804625474629, 68.10493614083535, 67.4318260269244, 66.55160510873317, 65.74042112530203, 64.4287193648602, 63.99723852260959, 63.56575768035899, 62.47842595788746, 61.822575077666556, 60.648947186744905, 59.44080082844321, 58.62961684501208, 57.92198826372108, 57.09354504659993, 55.88539868829824, 55.07421470486711, 53.52088367276493, 52.98584742837418, 52.519848118743525, 51.950293406972726, 51.34622022782188, 50.638591646530884, 49.68933379357956, 48.96444597859855, 48.136002761477386, 47.75629962029686, 47.25578184328615, 46.34104245771488, 45.61615464273386, 44.90852606144286, 44.183638246461854, 43.113565757680355, 42.45771487745944, 41.784604763548494, 40.90438384535726, 40.17949603037625, 38.91957197100449, 37.69416637901277, 37.03831549879185, 36.261649982740764, 35.79565067311011, 35.31239212978944, 34.63928201587849, 33.67276492923714, 33.12046945115637, 32.585433206765615, 31.929582326544704, 31.187435277873654, 30.393510528132552, 29.703141180531578, 29.047290300310657, 28.736624093890228, 28.063513979979284, 27.217811529168102, 26.56196064894718], "calls": [0.0, 0.9319986192613048, 1.9330341732827063, 2.623403520883673, 4.142216085605799, 5.212288574387297, 6.662064204349328, 7.611322057300655, 9.16465308940283, 9.751467034863651, 10.545391784604764, 11.839834311356576, 12.789092164307906, 14.22160856057991, 15.274421815671385, 16.62064204349327, 18.27752847773559, 18.881601656886435, 19.81360027614774, 20.883672764929237, 21.936486020020713, 22.7131515360718, 23.645150155333102, 24.991370383154987, 25.92336900241629, 26.958923023817743, 28.61580945806006, 30.289955125992407, 31.96410079392475, 33.15498791853641, 34.483948912668275, 35.29513289609941, 36.76216775975147, 38.12564722126338, 39.93786675871591, 41.2840869865378, 42.56127027959958, 43.803935105281326, 45.11563686572316, 46.03037625129444, 46.68622713151536, 47.82533655505696, 49.05074214704867, 50.62133241284087, 52.36451501553331, 53.58992060752502, 54.38384535726614, 55.10873317224715, 56.23058336209872, 57.35243355195029, 58.8539868829824, 59.38902312737314, 59.94131860545392, 61.25302036589575, 62.150500517777004, 63.44494304452882, 64.80842250604073, 65.82671729375215, 66.60338280980325, 67.55264066275457, 68.8643424231964, 69.81360027614774, 71.43596824301001, 72.00552295478082, 72.48878149810149, 73.0755954435623, 73.67966862271315, 74.4218156713842, 75.47462892647566, 76.30307214359682, 77.14877459440801, 77.5802554366586, 78.11529168104936, 79.08180876769072, 79.91025198481188, 80.66965826717293, 81.44632378322403, 82.60269244045564, 83.2930617880566, 84.01794960303762, 84.9326889886089, 85.70935450466, 87.0037970314118, 88.31549879185364, 89.02312737314463, 89.88608905764585, 90.42112530203659, 90.92164307904729, 91.62927166033829, 92.64756644804972, 93.23438039351053, 93.7866758715913, 94.47704521919226, 95.25371073524336, 96.06489471867448, 96.78978253365551, 97.46289264756645, 97.82533655505695, 98.4984466689679, 99.34414911977908, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 4, 6, 9, 11, 11, 12, 14, 16, 17, 19, 23, 28, 31, 33, 37, 40, 42, 45, 48, 50, 54, 60, 61, 65, 67, 69, 72, 72, 72, 73, 73, 79, 82, 83, 84, 86, 88, 88, 88, 88, 89, 92, 92, 92, 95, 95, 96, 98, 99, 99, 99, 99, 99, 99, 99, 100, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106], "harmed": [0, 0, 1, 1, 3, 6, 8, 9, 15, 16, 21, 23, 26, 38, 50, 61, 80, 90, 109, 127, 150, 164, 179, 204, 232, 261, 309, 368, 427, 465, 507, 529, 586, 638, 709, 765, 821, 873, 932, 971, 1004, 1054, 1112, 1187, 1266, 1321, 1361, 1401, 1454, 1503, 1579, 1604, 1629, 1693, 1734, 1802, 1872, 1922, 1963, 2012, 2084, 2132, 2222, 2253, 2280, 2313, 2348, 2389, 2445, 2489, 2537, 2559, 2588, 2641, 2683, 2724, 2766, 2828, 2866, 2905, 2956, 2999, 3072, 3143, 3181, 3227, 3255, 3283, 3322, 3378, 3410, 3441, 3479, 3522, 3568, 3609, 3647, 3665, 3704, 3753, 3791], "exact_score": [90.16223679668623, 90.17949603037624, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.05868139454608, 90.05868139454608, 90.04142216085606, 89.95512599240594, 89.8688298239558, 89.83431135657577, 89.78253365550569, 89.73075595443561, 89.6444597859855, 89.40283051432516, 89.26475664480496, 89.17846047635484, 89.04038660683466, 88.78149810148429, 88.38453572661373, 88.14290645495339, 87.83224024853297, 87.5733517431826, 87.24542630307214, 87.03831549879185, 86.62409389023126, 86.19261304798067, 85.76113220573006, 85.15705902657922, 84.55298584742837, 84.06972730410769, 83.46565412495684, 82.94787711425612, 82.48187780462547, 82.01587849499482, 81.39454608215395, 80.77321366931308, 80.16914049016223, 79.65136347946151, 78.94373489817052, 78.2188470831895, 77.51121850189851, 76.87262685536763, 76.0787021056265, 75.38833275802554, 74.66344494304452, 73.83500172592336, 73.0755954435623, 72.28167069382118, 71.47048671039006, 70.69382119433897, 69.96893337935795, 69.26130479806696, 68.48463928201586, 67.53538142906454, 66.68967897825337, 66.01656886434242, 65.20538488091128, 64.34242319641008, 63.51397997928891, 62.65101829478771, 61.90887124611667, 61.04590956161546, 60.16568864342423, 59.33724542630308, 58.45702450811183, 57.611322057300654, 56.69658267172937, 55.76458405246807, 54.936140835346905, 53.952364515015525, 53.003106662064205, 52.03658957542285, 51.05281325509148, 50.15533310321021, 49.24059371763893, 48.2913358646876, 47.37659647911632, 46.41007939247498, 45.54711770797376, 44.58060062133241, 43.66586123576113, 42.73386261649982, 41.81912323092855, 40.88712461166723, 39.920607525025886, 38.93683120469451, 37.98757335174317, 37.072833966171906, 36.15809458060062, 35.24335519502934, 34.29409734207801, 33.34483948912668, 32.39558163617535, 31.446323783224017, 30.46254746289264, 29.49603037625129, 28.564031756989984, 27.56299620296858, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 67.26970123354965, "threshold_area100": 63.586382464618566, "threshold_area25": 89.04038660683466}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0108592938631773, "operator": "ge"}, {"budget": 0.02, "threshold": -0.01345076598227024, "operator": "ge"}, {"budget": 0.03, "threshold": -0.01643543317914009, "operator": "ge"}, {"budget": 0.04, "threshold": -0.018877513706684113, "operator": "ge"}, {"budget": 0.05, "threshold": -0.021554332226514816, "operator": "ge"}, {"budget": 0.06, "threshold": -0.02546592243015766, "operator": "ge"}, {"budget": 0.07, "threshold": -0.029938675463199615, "operator": "ge"}, {"budget": 0.08, "threshold": -0.03515828028321266, "operator": "ge"}, {"budget": 0.09, "threshold": -0.04198404774069786, "operator": "ge"}, {"budget": 0.1, "threshold": -0.04629216715693474, "operator": "ge"}, {"budget": 0.11, "threshold": -0.05931616947054863, "operator": "ge"}, {"budget": 0.12, "threshold": -0.07723776996135712, "operator": "ge"}, {"budget": 0.13, "threshold": -0.10347901284694672, "operator": "ge"}, {"budget": 0.14, "threshold": -0.11588551104068756, "operator": "ge"}, {"budget": 0.15, "threshold": -0.13898064196109772, "operator": "ge"}, {"budget": 0.16, "threshold": -0.17358236014842987, "operator": "ge"}, {"budget": 0.17, "threshold": -0.18946786224842072, "operator": "ge"}, {"budget": 0.18, "threshold": -0.2182038277387619, "operator": "ge"}, {"budget": 0.19, "threshold": -0.2386011928319931, "operator": "ge"}, {"budget": 0.2, "threshold": -0.26739633083343506, "operator": "ge"}, {"budget": 0.21, "threshold": -0.29137399792671204, "operator": "ge"}, {"budget": 0.22, "threshold": -0.3197775185108185, "operator": "ge"}, {"budget": 0.23, "threshold": -0.3518471419811249, "operator": "ge"}, {"budget": 0.24, "threshold": -0.3778113126754761, "operator": "ge"}, {"budget": 0.25, "threshold": -0.413480669260025, "operator": "ge"}, {"budget": 0.26, "threshold": -0.4717594385147095, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5047510862350464, "operator": "ge"}, {"budget": 0.28, "threshold": -0.5578178763389587, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5799506306648254, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6119399070739746, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6387065649032593, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6661503911018372, "operator": "ge"}, {"budget": 0.33, "threshold": -0.693043053150177, "operator": "ge"}, {"budget": 0.34, "threshold": -0.723857581615448, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7465613484382629, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7692105174064636, "operator": "ge"}, {"budget": 0.37, "threshold": -0.7868311405181885, "operator": "ge"}, {"budget": 0.38, "threshold": -0.7997632026672363, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8115677833557129, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8194550275802612, "operator": "ge"}, {"budget": 0.41, "threshold": -0.8284794092178345, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8416597247123718, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8538106083869934, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8608629107475281, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8666256070137024, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8734807968139648, "operator": "ge"}, {"budget": 0.47, "threshold": -0.8795610070228577, "operator": "ge"}, {"budget": 0.48, "threshold": -0.8864057660102844, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8953688144683838, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9007324576377869, "operator": "ge"}, {"budget": 0.51, "threshold": -0.90273118019104, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9073853492736816, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9135693907737732, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9194111824035645, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9266641139984131, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9305832386016846, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9338294267654419, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9380355477333069, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9413579106330872, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9453051090240479, "operator": "ge"}, {"budget": 0.61, "threshold": -0.94832843542099, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9515128135681152, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9532365798950195, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9547104835510254, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9554929733276367, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9572466015815735, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9589229226112366, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9598114490509033, "operator": "ge"}, {"budget": 0.69, "threshold": -0.961624264717102, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9631640315055847, "operator": "ge"}, {"budget": 0.71, "threshold": -0.964513897895813, "operator": "ge"}, {"budget": 0.72, "threshold": -0.965935230255127, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9680970907211304, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9700964689254761, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9722571969032288, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9739535450935364, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9757660627365112, "operator": "ge"}, {"budget": 0.78, "threshold": -0.977238118648529, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9785242676734924, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9798031449317932, "operator": "ge"}, {"budget": 0.81, "threshold": -0.981091320514679, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9827484488487244, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9840288758277893, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9850157499313354, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9857688546180725, "operator": "ge"}, {"budget": 0.86, "threshold": -0.986380934715271, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9869428873062134, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9874731302261353, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9881009459495544, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9884836673736572, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9890466928482056, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9895020723342896, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9899362325668335, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9905596971511841, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9910634160041809, "operator": "ge"}, {"budget": 0.96, "threshold": -0.991654634475708, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9920552968978882, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9925562739372253, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9935983419418335, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.66666666666667, 98.5, 98.41666666666666, 98.33333333333333, 98.16666666666667, 98.0, 97.91666666666666, 97.58333333333333, 97.33333333333333, 97.24999999999999, 97.0, 96.75, 96.33333333333333, 96.33333333333333, 96.0, 95.91666666666666, 95.66666666666667, 95.25, 94.66666666666667, 94.16666666666667, 93.41666666666666, 92.75, 92.16666666666666, 91.58333333333333, 90.83333333333333, 90.33333333333333, 89.5, 88.66666666666666, 87.83333333333333, 87.0, 86.0, 85.41666666666666, 84.58333333333333, 83.83333333333333, 83.0, 82.08333333333333, 81.16666666666667, 80.33333333333333, 79.33333333333333, 78.41666666666667, 77.66666666666666, 76.75, 75.75, 74.83333333333333, 73.83333333333333, 72.83333333333333, 72.0, 71.08333333333333, 70.08333333333333, 69.16666666666667, 68.08333333333333, 67.16666666666666, 66.25, 65.33333333333333, 64.33333333333333, 63.33333333333333, 62.33333333333333, 61.416666666666664, 60.416666666666664, 59.583333333333336, 58.666666666666664, 57.833333333333336, 56.749999999999986, 55.833333333333336, 54.91666666666666, 53.91666666666666, 53.0, 52.0, 51.08333333333333, 50.08333333333332, 49.08333333333333, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.33333333333333, 44.25, 43.333333333333336, 42.333333333333336, 41.333333333333336, 40.33333333333333, 39.33333333333333, 38.33333333333333, 37.33333333333333, 36.416666666666664, 35.5, 34.5, 33.5, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.666666666666664, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 1, 1, 1, 1, 1, 2, 2, 3, 5, 6, 7, 9, 11, 12, 16, 19, 20, 23, 26, 31, 31, 36, 38, 41, 46, 53, 59, 68, 76, 83, 90, 99, 106, 116, 126, 136, 146, 158, 165, 175, 184, 194, 205, 216, 226, 238, 249, 258, 269, 281, 292, 304, 316, 326, 337, 349, 360, 373, 384, 395, 406, 418, 430, 442, 453, 465, 475, 486, 496, 509, 520, 531, 543, 554, 566, 577, 589, 601, 613, 624, 636, 646, 659, 670, 682, 694, 706, 718, 730, 742, 753, 764, 776, 788, 800, 811, 823, 834, 846, 857], "exact_score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.66666666666667, 98.5, 98.41666666666666, 98.33333333333333, 98.16666666666667, 98.0, 97.91666666666666, 97.58333333333333, 97.33333333333333, 97.24999999999999, 97.0, 96.75, 96.33333333333333, 96.33333333333333, 96.0, 95.91666666666666, 95.66666666666667, 95.25, 94.66666666666667, 94.16666666666667, 93.41666666666666, 92.75, 92.16666666666666, 91.58333333333333, 90.83333333333333, 90.33333333333333, 89.5, 88.66666666666666, 87.83333333333333, 87.0, 86.0, 85.41666666666666, 84.58333333333333, 83.74999999999999, 83.0, 82.08333333333333, 81.16666666666667, 80.33333333333333, 79.33333333333333, 78.41666666666667, 77.66666666666666, 76.75, 75.75, 74.83333333333333, 73.83333333333333, 72.83333333333333, 72.0, 71.08333333333333, 70.08333333333333, 69.08333333333333, 68.08333333333333, 67.16666666666666, 66.25, 65.33333333333333, 64.33333333333333, 63.33333333333333, 62.33333333333333, 61.416666666666664, 60.416666666666664, 59.583333333333336, 58.666666666666664, 57.74999999999999, 56.749999999999986, 55.833333333333336, 54.91666666666666, 53.91666666666666, 53.0, 52.0, 51.08333333333333, 50.08333333333332, 49.08333333333333, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.25, 44.25, 43.333333333333336, 42.333333333333336, 41.333333333333336, 40.33333333333333, 39.33333333333333, 38.33333333333333, 37.33333333333333, 36.416666666666664, 35.5, 34.5, 33.5, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.666666666666664, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.92833333333334, "threshold_area100": 71.93166666666667, "threshold_area25": 97.64999999999999}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.09319986192614, 90.05868139454608, 90.05868139454608, 90.04142216085606, 89.95512599240594, 89.90334829133586, 89.8688298239558, 89.8688298239558, 89.73075595443561, 89.59268208491544, 89.5063859164653, 89.3683120469451, 89.14394200897479, 88.98860890576458, 88.62616499827408, 88.47083189506385, 88.19468415602347, 87.93579565067311, 87.65964791163272, 87.27994477045219, 86.86572316189161, 86.39972385226096, 85.5022437003797, 85.03624439074905, 84.06972730410769, 83.5001725923369, 82.91335864687608, 82.17121159820505, 81.44632378322402, 80.63513979979288, 79.61684501208146, 78.73662409389023, 77.68381083879875, 76.76907145322747, 76.18225750776665, 75.26751812219537, 74.59440800828443, 73.95581636175353, 72.71315153607179, 71.79841215050051, 70.95270969968934, 70.19330341732827, 69.46841560234725, 68.65723161891611, 67.5698998964446, 66.34449430445288, 65.25716258198135, 64.91197790818087, 64.27338626164997, 63.272350707628576, 62.150500517777004, 60.75250258888505, 60.20020711080427, 59.527096996893334, 58.73317224715222, 57.87021056265102, 56.817397307559546, 55.88539868829824, 54.60821539523645, 54.02140144977563, 53.279254401104595, 53.089402830514324, 52.07110804280289, 51.08733172247152, 50.55229547808078, 49.56851915774939, 48.8608905764584, 48.343113565757676, 47.842595788746976, 46.87607870210562, 46.099413186054534, 45.34000690369348, 44.58060062133241, 43.476009665170864, 42.80289955125992, 42.216085605799094, 41.33586468760787, 40.68001380738695, 39.16120124266482, 37.84949948222298, 36.91750086296168, 36.330686917500856, 35.77839143942009, 35.32965136347946, 34.5702450811184, 33.8626164998274, 33.34483948912668, 32.70624784259578, 32.084915429754915, 31.39454608215395, 30.497065930272694, 29.910251984811875, 29.08180876769071, 28.753883327580255, 28.080773213669307, 27.235070762858125, 26.56196064894718], "calls": [0.0, 0.6731101139109423, 1.484294097342078, 2.7269589230238176, 3.935105281325509, 5.022437003797031, 6.161546427338626, 7.335174318260269, 8.318950638591646, 9.250949257852952, 9.682430100103554, 10.959613393165343, 12.443907490507423, 13.807386952019328, 14.515015533310322, 15.774939592682086, 17.397307559544355, 17.984121505005177, 19.261304798066966, 20.10700724887815, 20.987228167069382, 21.798412150500518, 22.73041076976182, 23.955816361753538, 24.83603727994477, 25.92336900241629, 28.02899551259924, 28.909216430790472, 30.773213669313083, 31.687953054884368, 33.08595098377632, 34.24231964100794, 35.65757680358992, 37.072833966171906, 38.643424231964104, 39.93786675871591, 41.45667932343804, 42.682084915429755, 43.458750431480844, 44.787711425612706, 45.70245081118398, 46.582671729375214, 48.04970659302727, 49.44770452191923, 50.43148084225061, 51.31170176044184, 52.19192267863308, 53.12392129789437, 54.45288229202624, 56.178805661028655, 57.611322057300654, 58.025543665861235, 58.81946841560235, 59.95857783914394, 61.32205730065585, 63.099758370728345, 63.772868484639275, 64.63583016914049, 65.53331032102174, 66.430790472903, 67.60441836382465, 68.6399723852261, 70.0724887814981, 70.74559889540905, 71.50500517777012, 71.74663444943045, 72.79944770452192, 73.86952019330342, 74.43907490507421, 75.52640662754574, 76.25129444252676, 76.83810838798757, 77.40766309975837, 78.42595788746979, 79.30617880566103, 80.11736278909216, 80.96306523990336, 82.136693130825, 82.87884017949602, 83.55195029340697, 84.44943044528823, 85.15705902657922, 86.76216775975146, 88.14290645495339, 89.12668277528478, 89.78253365550569, 90.42112530203659, 90.88712461166725, 91.69830859509838, 92.42319641007938, 92.97549188816016, 93.64860200207112, 94.30445288229203, 95.02934069727304, 95.96133931653435, 96.5826717293752, 97.41111494649638, 97.80807732136692, 98.48118743527787, 99.32688988608905, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 3, 7, 8, 10, 11, 11, 12, 14, 16, 17, 19, 29, 31, 34, 36, 40, 41, 44, 47, 50, 53, 59, 60, 61, 65, 67, 67, 69, 72, 72, 72, 73, 73, 75, 80, 84, 84, 86, 87, 88, 90, 91, 93, 94, 94, 94, 95, 97, 97, 97, 97, 97, 98, 98, 99, 99, 99, 100, 100, 101, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106], "harmed": [0, 0, 1, 2, 4, 6, 7, 8, 13, 16, 18, 19, 27, 35, 40, 49, 66, 76, 99, 109, 125, 141, 159, 183, 208, 237, 299, 328, 387, 422, 460, 504, 549, 599, 661, 715, 782, 836, 871, 928, 969, 1006, 1080, 1136, 1185, 1229, 1272, 1319, 1384, 1460, 1527, 1547, 1586, 1645, 1711, 1794, 1827, 1868, 1915, 1965, 2026, 2081, 2157, 2191, 2234, 2245, 2304, 2362, 2393, 2451, 2492, 2522, 2552, 2608, 2654, 2699, 2743, 2807, 2846, 2880, 2931, 2969, 3058, 3134, 3188, 3223, 3256, 3282, 3326, 3367, 3397, 3434, 3470, 3510, 3562, 3597, 3645, 3664, 3703, 3752, 3791], "exact_score": [90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.09319986192614, 90.05868139454608, 90.05868139454608, 90.04142216085606, 89.97238522609597, 89.95512599240594, 89.88608905764583, 89.85157059026578, 89.78253365550569, 89.7134967207456, 89.55816361753539, 89.43734898170521, 89.31653434587504, 89.2129789437349, 88.98860890576458, 88.7124611667242, 88.48809112875388, 88.19468415602347, 87.90127718329306, 87.60787021056264, 87.24542630307214, 86.77942699344149, 86.33068691750086, 86.02002071108042, 85.48498446668967, 85.01898515705902, 84.44943044528823, 83.94891266827752, 83.34483948912667, 82.98239558163617, 82.37832240248532, 81.80876769071452, 81.25647221263375, 80.68691750086295, 80.04832585433206, 79.47877114256127, 78.68484639282015, 77.97721781152916, 77.26958923023818, 76.47566448049706, 75.7852951328961, 75.09492578529513, 74.37003797031412, 73.576113220573, 72.73041076976182, 72.07455988954091, 71.34967207455989, 70.45219192267864, 69.64100793924749, 68.72626855367622, 67.91508457024507, 67.2074559889541, 66.46530894028305, 65.74042112530203, 64.94649637556091, 64.16983085950983, 63.23783224024852, 62.426648256817394, 61.61546427338626, 60.82153952364515, 59.99309630652399, 59.19917155678287, 58.26717293752157, 57.33517431826027, 56.50673110113911, 55.54021401449776, 54.67725232999654, 53.779772178115294, 52.84777355885398, 51.88125647221262, 50.966517086641346, 50.06903693476009, 49.10251984811874, 48.205039696237485, 47.290300310666204, 46.34104245771488, 45.46082153952364, 44.56334138764239, 43.614083534691055, 42.682084915429755, 41.80186399723852, 40.83534691059717, 39.920607525025886, 38.93683120469451, 38.0048325854332, 37.05557473248188, 36.15809458060062, 35.24335519502934, 34.29409734207801, 33.32758025543665, 32.39558163617535, 31.429064549534004, 30.46254746289264, 29.49603037625129, 28.564031756989984, 27.56299620296858, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 67.14935724993263, "threshold_area100": 64.01458405246807, "threshold_area25": 89.15429754918881}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.9744943380355835, "operator": "ge"}, {"budget": 0.02, "threshold": -2.493140935897827, "operator": "ge"}, {"budget": 0.03, "threshold": -3.0330190658569336, "operator": "ge"}, {"budget": 0.04, "threshold": -3.744378089904785, "operator": "ge"}, {"budget": 0.05, "threshold": -4.010400772094727, "operator": "ge"}, {"budget": 0.06, "threshold": -4.189113616943359, "operator": "ge"}, {"budget": 0.07, "threshold": -4.569300174713135, "operator": "ge"}, {"budget": 0.08, "threshold": -5.004278182983398, "operator": "ge"}, {"budget": 0.09, "threshold": -5.3115553855896, "operator": "ge"}, {"budget": 0.1, "threshold": -5.535503387451172, "operator": "ge"}, {"budget": 0.11, "threshold": -5.8027238845825195, "operator": "ge"}, {"budget": 0.12, "threshold": -6.00237512588501, "operator": "ge"}, {"budget": 0.13, "threshold": -6.166377067565918, "operator": "ge"}, {"budget": 0.14, "threshold": -6.348254203796387, "operator": "ge"}, {"budget": 0.15, "threshold": -6.464213848114014, "operator": "ge"}, {"budget": 0.16, "threshold": -6.632201194763184, "operator": "ge"}, {"budget": 0.17, "threshold": -6.926812648773193, "operator": "ge"}, {"budget": 0.18, "threshold": -7.016518592834473, "operator": "ge"}, {"budget": 0.19, "threshold": -7.229836463928223, "operator": "ge"}, {"budget": 0.2, "threshold": -7.421133995056152, "operator": "ge"}, {"budget": 0.21, "threshold": -7.564243316650391, "operator": "ge"}, {"budget": 0.22, "threshold": -7.639152526855469, "operator": "ge"}, {"budget": 0.23, "threshold": -7.742486953735352, "operator": "ge"}, {"budget": 0.24, "threshold": -7.836495876312256, "operator": "ge"}, {"budget": 0.25, "threshold": -7.927787780761719, "operator": "ge"}, {"budget": 0.26, "threshold": -8.050948143005371, "operator": "ge"}, {"budget": 0.27, "threshold": -8.14671516418457, "operator": "ge"}, {"budget": 0.28, "threshold": -8.242374420166016, "operator": "ge"}, {"budget": 0.29, "threshold": -8.436086654663086, "operator": "ge"}, {"budget": 0.3, "threshold": -8.49325180053711, "operator": "ge"}, {"budget": 0.31, "threshold": -8.605948448181152, "operator": "ge"}, {"budget": 0.32, "threshold": -8.688855171203613, "operator": "ge"}, {"budget": 0.33, "threshold": -8.844528198242188, "operator": "ge"}, {"budget": 0.34, "threshold": -8.957935333251953, "operator": "ge"}, {"budget": 0.35, "threshold": -9.065841674804688, "operator": "ge"}, {"budget": 0.36, "threshold": -9.139837265014648, "operator": "ge"}, {"budget": 0.37, "threshold": -9.236225128173828, "operator": "ge"}, {"budget": 0.38, "threshold": -9.325881958007812, "operator": "ge"}, {"budget": 0.39, "threshold": -9.40478515625, "operator": "ge"}, {"budget": 0.4, "threshold": -9.50322437286377, "operator": "ge"}, {"budget": 0.41, "threshold": -9.60912799835205, "operator": "ge"}, {"budget": 0.42, "threshold": -9.685680389404297, "operator": "ge"}, {"budget": 0.43, "threshold": -9.762237548828125, "operator": "ge"}, {"budget": 0.44, "threshold": -9.84579086303711, "operator": "ge"}, {"budget": 0.45, "threshold": -9.927042007446289, "operator": "ge"}, {"budget": 0.46, "threshold": -10.01598072052002, "operator": "ge"}, {"budget": 0.47, "threshold": -10.127509117126465, "operator": "ge"}, {"budget": 0.48, "threshold": -10.216546058654785, "operator": "ge"}, {"budget": 0.49, "threshold": -10.30439567565918, "operator": "ge"}, {"budget": 0.5, "threshold": -10.356515884399414, "operator": "ge"}, {"budget": 0.51, "threshold": -10.433530807495117, "operator": "ge"}, {"budget": 0.52, "threshold": -10.507295608520508, "operator": "ge"}, {"budget": 0.53, "threshold": -10.591638565063477, "operator": "ge"}, {"budget": 0.54, "threshold": -10.67320442199707, "operator": "ge"}, {"budget": 0.55, "threshold": -10.749878883361816, "operator": "ge"}, {"budget": 0.56, "threshold": -10.813492774963379, "operator": "ge"}, {"budget": 0.57, "threshold": -10.878471374511719, "operator": "ge"}, {"budget": 0.58, "threshold": -10.9417724609375, "operator": "ge"}, {"budget": 0.59, "threshold": -11.01083755493164, "operator": "ge"}, {"budget": 0.6, "threshold": -11.05410385131836, "operator": "ge"}, {"budget": 0.61, "threshold": -11.10641860961914, "operator": "ge"}, {"budget": 0.62, "threshold": -11.15766716003418, "operator": "ge"}, {"budget": 0.63, "threshold": -11.233060836791992, "operator": "ge"}, {"budget": 0.64, "threshold": -11.28077507019043, "operator": "ge"}, {"budget": 0.65, "threshold": -11.320640563964844, "operator": "ge"}, {"budget": 0.66, "threshold": -11.390268325805664, "operator": "ge"}, {"budget": 0.67, "threshold": -11.427821159362793, "operator": "ge"}, {"budget": 0.68, "threshold": -11.47745418548584, "operator": "ge"}, {"budget": 0.69, "threshold": -11.524770736694336, "operator": "ge"}, {"budget": 0.7, "threshold": -11.603070259094238, "operator": "ge"}, {"budget": 0.71, "threshold": -11.665225982666016, "operator": "ge"}, {"budget": 0.72, "threshold": -11.735443115234375, "operator": "ge"}, {"budget": 0.73, "threshold": -11.796609878540039, "operator": "ge"}, {"budget": 0.74, "threshold": -11.884687423706055, "operator": "ge"}, {"budget": 0.75, "threshold": -11.967097282409668, "operator": "ge"}, {"budget": 0.76, "threshold": -12.052562713623047, "operator": "ge"}, {"budget": 0.77, "threshold": -12.14381217956543, "operator": "ge"}, {"budget": 0.78, "threshold": -12.205371856689453, "operator": "ge"}, {"budget": 0.79, "threshold": -12.253009796142578, "operator": "ge"}, {"budget": 0.8, "threshold": -12.30168628692627, "operator": "ge"}, {"budget": 0.81, "threshold": -12.366617202758789, "operator": "ge"}, {"budget": 0.82, "threshold": -12.434754371643066, "operator": "ge"}, {"budget": 0.83, "threshold": -12.511672973632812, "operator": "ge"}, {"budget": 0.84, "threshold": -12.566146850585938, "operator": "ge"}, {"budget": 0.85, "threshold": -12.640589714050293, "operator": "ge"}, {"budget": 0.86, "threshold": -12.686967849731445, "operator": "ge"}, {"budget": 0.87, "threshold": -12.73474407196045, "operator": "ge"}, {"budget": 0.88, "threshold": -12.800268173217773, "operator": "ge"}, {"budget": 0.89, "threshold": -12.912971496582031, "operator": "ge"}, {"budget": 0.9, "threshold": -13.03258228302002, "operator": "ge"}, {"budget": 0.91, "threshold": -13.087692260742188, "operator": "ge"}, {"budget": 0.92, "threshold": -13.17664623260498, "operator": "ge"}, {"budget": 0.93, "threshold": -13.252569198608398, "operator": "ge"}, {"budget": 0.94, "threshold": -13.306600570678711, "operator": "ge"}, {"budget": 0.95, "threshold": -13.3990478515625, "operator": "ge"}, {"budget": 0.96, "threshold": -13.557208061218262, "operator": "ge"}, {"budget": 0.97, "threshold": -13.631867408752441, "operator": "ge"}, {"budget": 0.98, "threshold": -13.876087188720703, "operator": "ge"}, {"budget": 0.99, "threshold": -14.028858184814453, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.33333333333333, 97.91666666666666, 97.33333333333333, 96.5, 96.0, 95.41666666666666, 94.58333333333333, 93.83333333333333, 93.33333333333333, 92.58333333333333, 92.08333333333333, 91.58333333333333, 91.33333333333333, 90.66666666666666, 90.25, 90.08333333333333, 89.66666666666666, 89.16666666666666, 88.91666666666667, 88.58333333333333, 88.33333333333333, 87.91666666666667, 87.58333333333333, 87.08333333333333, 86.75, 86.33333333333333, 85.91666666666666, 85.25, 84.91666666666666, 84.41666666666666, 83.83333333333333, 83.08333333333333, 82.5, 81.91666666666666, 81.5, 81.08333333333333, 80.58333333333333, 79.91666666666666, 79.25, 78.66666666666666, 78.0, 77.25, 76.66666666666666, 76.08333333333333, 75.41666666666667, 74.83333333333333, 74.08333333333333, 73.58333333333333, 72.83333333333333, 72.24999999999999, 71.58333333333333, 71.0, 70.24999999999999, 69.41666666666666, 68.58333333333333, 67.75, 66.83333333333333, 65.91666666666667, 65.08333333333334, 64.25, 63.416666666666664, 62.58333333333332, 61.66666666666667, 60.83333333333333, 60.0, 59.416666666666664, 58.416666666666664, 57.58333333333333, 56.49999999999999, 55.58333333333333, 54.666666666666664, 53.666666666666664, 52.666666666666664, 51.66666666666666, 50.66666666666666, 49.75, 48.83333333333333, 47.83333333333333, 47.083333333333336, 46.08333333333333, 45.25, 44.166666666666664, 43.166666666666664, 42.166666666666664, 41.25, 40.33333333333333, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.666666666666664, 29.666666666666664, 28.666666666666664, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 2, 9, 15, 22, 32, 38, 45, 55, 64, 70, 79, 85, 91, 94, 102, 107, 109, 114, 120, 123, 127, 130, 135, 139, 145, 149, 154, 159, 167, 171, 177, 184, 193, 200, 207, 212, 217, 223, 231, 239, 246, 254, 263, 270, 277, 285, 292, 301, 307, 316, 323, 331, 338, 347, 357, 367, 377, 388, 399, 409, 419, 429, 439, 450, 460, 470, 477, 489, 499, 512, 523, 534, 546, 558, 570, 582, 593, 604, 616, 625, 637, 647, 660, 672, 684, 695, 706, 717, 728, 740, 752, 764, 776, 788, 800, 810, 822, 834, 846, 857], "exact_score": [98.91666666666666, 98.83333333333333, 98.33333333333333, 97.91666666666666, 97.33333333333333, 96.5, 96.0, 95.41666666666666, 94.58333333333333, 93.83333333333333, 93.33333333333333, 92.58333333333333, 92.08333333333333, 91.58333333333333, 91.33333333333333, 90.66666666666666, 90.25, 90.08333333333333, 89.66666666666666, 89.16666666666666, 88.91666666666667, 88.58333333333333, 88.33333333333333, 87.91666666666667, 87.58333333333333, 87.08333333333333, 86.75, 86.33333333333333, 85.91666666666666, 85.25, 84.91666666666666, 84.41666666666666, 83.83333333333333, 83.08333333333333, 82.5, 81.91666666666666, 81.5, 81.08333333333333, 80.58333333333333, 79.91666666666666, 79.25, 78.58333333333333, 78.0, 77.25, 76.66666666666666, 76.08333333333333, 75.41666666666667, 74.83333333333333, 74.08333333333333, 73.58333333333333, 72.83333333333333, 72.24999999999999, 71.58333333333333, 71.0, 70.24999999999999, 69.41666666666666, 68.58333333333333, 67.66666666666666, 66.83333333333333, 65.91666666666667, 65.08333333333334, 64.25, 63.416666666666664, 62.58333333333332, 61.66666666666667, 60.83333333333333, 60.0, 59.416666666666664, 58.416666666666664, 57.49999999999999, 56.49999999999999, 55.58333333333333, 54.666666666666664, 53.666666666666664, 52.666666666666664, 51.66666666666666, 50.66666666666666, 49.75, 48.83333333333333, 47.83333333333333, 47.083333333333336, 46.08333333333333, 45.166666666666664, 44.166666666666664, 43.166666666666664, 42.166666666666664, 41.25, 40.33333333333333, 39.416666666666664, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.666666666666664, 29.666666666666664, 28.666666666666664, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 68.92333333333333, "threshold_area100": 68.92666666666668, "threshold_area25": 92.48333333333333}, "test": {"score": [90.16223679668623, 89.43734898170521, 88.55712806351399, 87.53883327580255, 85.58853986882981, 84.46668967897824, 83.82809803244736, 82.41284086986538, 80.10010355540214, 78.54677252329995, 77.66655160510874, 76.38936831204694, 75.61270279599586, 74.92233344839488, 74.09389023127373, 73.67966862271315, 72.93752157404211, 71.83293061788056, 71.5050051777701, 70.71108042802899, 70.19330341732827, 69.69278564031757, 69.38211943389715, 68.9333793579565, 68.62271315153608, 68.38108387987573, 67.96686227131515, 67.48360372799446, 67.19019675526407, 66.44804970659303, 66.2927166033828, 65.84397652744218, 65.53331032102174, 64.89471867449085, 64.41146013117017, 63.99723852260959, 63.634794615119084, 63.134276838108384, 62.68553676216775, 62.23679668622712, 61.822575077666556, 61.32205730065585, 60.94235415947531, 60.528132550914734, 60.01035554021401, 59.47531929582326, 58.87124611667242, 58.12909906800138, 57.628581290990674, 56.86917500862961, 56.47221263375906, 55.95443562305833, 55.57473248187781, 54.97065930272696, 54.31480842250605, 53.84880911287539, 53.31377286848464, 52.726958923023815, 52.27821884708319, 51.48429409734207, 51.19088712461166, 50.72488781498101, 50.327925440110455, 49.86192613047981, 49.55125992405937, 49.29237141870901, 48.412150500517775, 47.96341042457715, 47.514670348636514, 46.910597169485676, 46.03037625129444, 45.52985847428374, 44.83948912668277, 44.114601311701755, 43.13082499137038, 42.405937176389365, 41.52571625819813, 40.59371763893683, 39.85157059026578, 39.28201587849499, 38.798757335174315, 38.19468415602347, 37.5043148084225, 36.64135312392129, 36.037279944770454, 35.277873662409384, 34.898170521228856, 34.32861580945805, 33.79357956506731, 32.70624784259578, 31.61891612012426, 31.342768381083875, 30.842250604073172, 30.289955125992407, 30.065585088022083, 29.68588194684155, 28.99551259924059, 28.58129099068001, 27.666551605108726, 27.18329306178805, 26.56196064894718], "calls": [0.0, 3.9868829823955814, 7.317915084570245, 10.683465654124957, 15.75768035899206, 17.638936831204692, 19.036934760096653, 21.625819813600273, 25.302036589575422, 27.73558853986883, 29.237141870900935, 31.273731446323783, 32.430100103555404, 33.72454263030721, 35.260614428719364, 36.05453917846047, 37.573351743182606, 39.83431135657577, 40.628236106316876, 42.11253020365896, 43.51052813255091, 44.54608215395236, 45.25371073524336, 46.20296858819468, 46.82430100103556, 47.46289264756645, 48.46392820158785, 49.24059371763894, 49.896444597859855, 51.225405591991716, 51.5015533310321, 52.26095961339317, 52.70969968933379, 53.86606834656541, 54.71177079737659, 55.55747324818778, 56.127027959958575, 56.851915774939584, 57.749395926820846, 58.56057991025199, 59.31998619261305, 60.217466344494305, 60.873317224715215, 61.46013117017605, 62.23679668622714, 62.89264756644805, 63.772868484639275, 64.73938557128064, 65.41249568519157, 66.46530894028305, 66.96582671729375, 67.7079737659648, 68.17397307559544, 68.98515705902658, 69.81360027614774, 70.48671039005868, 71.12530203658957, 71.83293061788056, 72.33344839489126, 73.26544701415257, 73.67966862271315, 74.16292716603382, 74.61166724197446, 75.14670348636521, 75.50914739385571, 75.80255436658612, 76.83810838798757, 77.30410769761824, 77.82188470831896, 78.42595788746979, 79.3579565067311, 79.9275112185019, 80.70417673455299, 81.4981014842941, 82.55091473938558, 83.2930617880566, 84.25957887469797, 85.36416983085951, 86.12357611322057, 86.72764929237142, 87.22816706938212, 87.849499482223, 88.574387297204, 89.52364515015533, 90.12771832930618, 90.93890231273731, 91.33586468760787, 91.92267863306868, 92.47497411114946, 93.61408353469106, 94.73593372454263, 95.02934069727304, 95.56437694166378, 96.13393165343459, 96.37556092509493, 96.77252329996548, 97.4974111149465, 97.91163272350708, 98.87814981014843, 99.3614083534691, 100.0], "rescued": [0, 20, 36, 45, 58, 62, 68, 77, 83, 87, 89, 91, 91, 92, 94, 94, 96, 98, 99, 100, 100, 100, 100, 100, 100, 100, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106], "harmed": [0, 62, 129, 197, 323, 392, 435, 526, 666, 760, 813, 889, 934, 975, 1025, 1049, 1094, 1160, 1180, 1227, 1257, 1286, 1304, 1330, 1348, 1362, 1388, 1416, 1433, 1476, 1485, 1511, 1529, 1566, 1594, 1618, 1639, 1668, 1694, 1720, 1744, 1773, 1796, 1820, 1850, 1881, 1916, 1959, 1988, 2032, 2055, 2085, 2107, 2142, 2180, 2207, 2238, 2273, 2299, 2345, 2362, 2389, 2412, 2439, 2457, 2472, 2523, 2549, 2575, 2610, 2661, 2690, 2730, 2772, 2829, 2871, 2922, 2976, 3019, 3052, 3080, 3115, 3155, 3206, 3241, 3285, 3307, 3340, 3371, 3434, 3497, 3513, 3542, 3574, 3587, 3609, 3649, 3673, 3727, 3755, 3791], "exact_score": [90.16223679668623, 90.09319986192614, 89.95512599240594, 89.78253365550569, 89.43734898170521, 89.23023817742491, 88.93683120469451, 88.64342423196409, 88.36727649292371, 88.12564722126336, 87.83224024853297, 87.41801863997239, 86.95201933034173, 86.58957542285121, 86.38246461857094, 85.96824301001035, 85.5022437003797, 84.86365205384881, 84.29409734207802, 83.84535726613737, 83.41387642388678, 82.87884017949602, 82.15395236451502, 81.53261995167415, 80.91128753883328, 80.30721435968242, 79.61684501208146, 79.01277183293061, 78.39143942008975, 77.77010700724888, 77.14877459440801, 76.54470141525715, 75.97514670348636, 75.33655505695546, 74.76700034518467, 74.23196410079392, 73.71418709009319, 73.24818778046254, 72.67863306869174, 72.24715222644113, 71.78115291681048, 71.31515360717984, 70.74559889540903, 70.40041422160856, 69.96893337935795, 69.5029340697273, 69.01967552640662, 68.570935450466, 68.2084915429755, 67.62167759751468, 67.15567828788402, 66.62064204349328, 65.99930963065239, 65.37797721781152, 64.84294097342078, 64.27338626164997, 63.75560925094925, 63.099758370728345, 62.56472212633759, 61.926130479806694, 61.47739040386606, 60.8560579910252, 60.16568864342423, 59.38902312737314, 58.681394546082146, 57.95650673110113, 57.21435968243009, 56.45495340006903, 55.73006558508802, 54.97065930272696, 54.21125302036589, 53.41732827062478, 52.60614428719364, 51.725923369002416, 50.89748015188125, 50.01725923369003, 49.137038315498785, 48.25681739730756, 47.34207801173628, 46.34104245771488, 45.47808077321366, 44.58060062133241, 43.6486020020711, 42.69934414911978, 41.76734552985847, 40.93890231273731, 39.98964445978598, 39.023127373144625, 38.05661028650328, 37.12461166724197, 36.175353814290645, 35.22609596133931, 34.25957887469796, 33.29306178805661, 32.32654470141525, 31.377286848463925, 30.42802899551259, 29.46151190887124, 28.49499482222989, 27.545736969278558, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 64.33434912790528, "threshold_area100": 55.95193303417329, "threshold_area25": 77.05557473248189}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.7768033146858215, "operator": "ge"}, {"budget": 0.02, "threshold": -1.348742961883545, "operator": "ge"}, {"budget": 0.03, "threshold": -1.5928977727890015, "operator": "ge"}, {"budget": 0.04, "threshold": -1.9993144273757935, "operator": "ge"}, {"budget": 0.05, "threshold": -2.338298797607422, "operator": "ge"}, {"budget": 0.06, "threshold": -2.606649160385132, "operator": "ge"}, {"budget": 0.07, "threshold": -2.9514517784118652, "operator": "ge"}, {"budget": 0.08, "threshold": -3.241736888885498, "operator": "ge"}, {"budget": 0.09, "threshold": -3.591003894805908, "operator": "ge"}, {"budget": 0.1, "threshold": -3.775120973587036, "operator": "ge"}, {"budget": 0.11, "threshold": -4.041332721710205, "operator": "ge"}, {"budget": 0.12, "threshold": -4.199012756347656, "operator": "ge"}, {"budget": 0.13, "threshold": -4.390369415283203, "operator": "ge"}, {"budget": 0.14, "threshold": -4.563669681549072, "operator": "ge"}, {"budget": 0.15, "threshold": -4.794926643371582, "operator": "ge"}, {"budget": 0.16, "threshold": -4.959489345550537, "operator": "ge"}, {"budget": 0.17, "threshold": -5.14778470993042, "operator": "ge"}, {"budget": 0.18, "threshold": -5.305157661437988, "operator": "ge"}, {"budget": 0.19, "threshold": -5.420577049255371, "operator": "ge"}, {"budget": 0.2, "threshold": -5.540416717529297, "operator": "ge"}, {"budget": 0.21, "threshold": -5.685969352722168, "operator": "ge"}, {"budget": 0.22, "threshold": -5.867242813110352, "operator": "ge"}, {"budget": 0.23, "threshold": -5.953217506408691, "operator": "ge"}, {"budget": 0.24, "threshold": -6.05061674118042, "operator": "ge"}, {"budget": 0.25, "threshold": -6.165902137756348, "operator": "ge"}, {"budget": 0.26, "threshold": -6.321401596069336, "operator": "ge"}, {"budget": 0.27, "threshold": -6.427433490753174, "operator": "ge"}, {"budget": 0.28, "threshold": -6.551488876342773, "operator": "ge"}, {"budget": 0.29, "threshold": -6.651771545410156, "operator": "ge"}, {"budget": 0.3, "threshold": -6.7473273277282715, "operator": "ge"}, {"budget": 0.31, "threshold": -6.926967620849609, "operator": "ge"}, {"budget": 0.32, "threshold": -7.075835704803467, "operator": "ge"}, {"budget": 0.33, "threshold": -7.166018009185791, "operator": "ge"}, {"budget": 0.34, "threshold": -7.256450653076172, "operator": "ge"}, {"budget": 0.35, "threshold": -7.385680198669434, "operator": "ge"}, {"budget": 0.36, "threshold": -7.486136436462402, "operator": "ge"}, {"budget": 0.37, "threshold": -7.581784725189209, "operator": "ge"}, {"budget": 0.38, "threshold": -7.635124206542969, "operator": "ge"}, {"budget": 0.39, "threshold": -7.721857070922852, "operator": "ge"}, {"budget": 0.4, "threshold": -7.823723793029785, "operator": "ge"}, {"budget": 0.41, "threshold": -7.8722405433654785, "operator": "ge"}, {"budget": 0.42, "threshold": -8.01618480682373, "operator": "ge"}, {"budget": 0.43, "threshold": -8.09221076965332, "operator": "ge"}, {"budget": 0.44, "threshold": -8.186891555786133, "operator": "ge"}, {"budget": 0.45, "threshold": -8.294881820678711, "operator": "ge"}, {"budget": 0.46, "threshold": -8.381950378417969, "operator": "ge"}, {"budget": 0.47, "threshold": -8.484762191772461, "operator": "ge"}, {"budget": 0.48, "threshold": -8.548910140991211, "operator": "ge"}, {"budget": 0.49, "threshold": -8.637065887451172, "operator": "ge"}, {"budget": 0.5, "threshold": -8.700263977050781, "operator": "ge"}, {"budget": 0.51, "threshold": -8.79838752746582, "operator": "ge"}, {"budget": 0.52, "threshold": -8.855894088745117, "operator": "ge"}, {"budget": 0.53, "threshold": -8.884683609008789, "operator": "ge"}, {"budget": 0.54, "threshold": -8.978423118591309, "operator": "ge"}, {"budget": 0.55, "threshold": -9.031068801879883, "operator": "ge"}, {"budget": 0.56, "threshold": -9.077596664428711, "operator": "ge"}, {"budget": 0.57, "threshold": -9.170930862426758, "operator": "ge"}, {"budget": 0.58, "threshold": -9.249368667602539, "operator": "ge"}, {"budget": 0.59, "threshold": -9.283486366271973, "operator": "ge"}, {"budget": 0.6, "threshold": -9.348828315734863, "operator": "ge"}, {"budget": 0.61, "threshold": -9.426949501037598, "operator": "ge"}, {"budget": 0.62, "threshold": -9.481666564941406, "operator": "ge"}, {"budget": 0.63, "threshold": -9.537588119506836, "operator": "ge"}, {"budget": 0.64, "threshold": -9.567899703979492, "operator": "ge"}, {"budget": 0.65, "threshold": -9.606036186218262, "operator": "ge"}, {"budget": 0.66, "threshold": -9.690923690795898, "operator": "ge"}, {"budget": 0.67, "threshold": -9.734394073486328, "operator": "ge"}, {"budget": 0.68, "threshold": -9.792373657226562, "operator": "ge"}, {"budget": 0.69, "threshold": -9.838409423828125, "operator": "ge"}, {"budget": 0.7, "threshold": -9.883743286132812, "operator": "ge"}, {"budget": 0.71, "threshold": -9.924327850341797, "operator": "ge"}, {"budget": 0.72, "threshold": -9.967263221740723, "operator": "ge"}, {"budget": 0.73, "threshold": -10.012194633483887, "operator": "ge"}, {"budget": 0.74, "threshold": -10.03750228881836, "operator": "ge"}, {"budget": 0.75, "threshold": -10.10474967956543, "operator": "ge"}, {"budget": 0.76, "threshold": -10.151445388793945, "operator": "ge"}, {"budget": 0.77, "threshold": -10.18699836730957, "operator": "ge"}, {"budget": 0.78, "threshold": -10.239514350891113, "operator": "ge"}, {"budget": 0.79, "threshold": -10.299726486206055, "operator": "ge"}, {"budget": 0.8, "threshold": -10.345675468444824, "operator": "ge"}, {"budget": 0.81, "threshold": -10.431896209716797, "operator": "ge"}, {"budget": 0.82, "threshold": -10.473712921142578, "operator": "ge"}, {"budget": 0.83, "threshold": -10.562744140625, "operator": "ge"}, {"budget": 0.84, "threshold": -10.615283966064453, "operator": "ge"}, {"budget": 0.85, "threshold": -10.681150436401367, "operator": "ge"}, {"budget": 0.86, "threshold": -10.748983383178711, "operator": "ge"}, {"budget": 0.87, "threshold": -10.819725036621094, "operator": "ge"}, {"budget": 0.88, "threshold": -10.88956069946289, "operator": "ge"}, {"budget": 0.89, "threshold": -10.934372901916504, "operator": "ge"}, {"budget": 0.9, "threshold": -10.992668151855469, "operator": "ge"}, {"budget": 0.91, "threshold": -11.03346061706543, "operator": "ge"}, {"budget": 0.92, "threshold": -11.124303817749023, "operator": "ge"}, {"budget": 0.93, "threshold": -11.22165584564209, "operator": "ge"}, {"budget": 0.94, "threshold": -11.30842113494873, "operator": "ge"}, {"budget": 0.95, "threshold": -11.380250930786133, "operator": "ge"}, {"budget": 0.96, "threshold": -11.552766799926758, "operator": "ge"}, {"budget": 0.97, "threshold": -11.670299530029297, "operator": "ge"}, {"budget": 0.98, "threshold": -11.826881408691406, "operator": "ge"}, {"budget": 0.99, "threshold": -11.990100860595703, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.58333333333333, 98.08333333333333, 97.41666666666666, 96.66666666666667, 96.0, 95.33333333333333, 94.58333333333333, 93.91666666666666, 93.33333333333333, 92.75, 92.41666666666666, 91.75, 91.08333333333333, 90.91666666666667, 90.5, 90.08333333333333, 89.5, 89.0, 88.5, 87.91666666666667, 87.5, 87.25, 86.83333333333333, 86.16666666666666, 85.83333333333333, 85.5, 85.16666666666666, 84.74999999999999, 84.33333333333333, 83.91666666666666, 83.5, 82.91666666666666, 82.25, 81.91666666666666, 81.16666666666667, 80.66666666666666, 80.16666666666666, 79.33333333333333, 78.83333333333333, 78.41666666666667, 77.83333333333333, 77.41666666666667, 77.0, 76.41666666666667, 75.91666666666667, 75.33333333333333, 74.5, 73.75, 73.0, 72.24999999999999, 71.5, 70.75, 70.0, 69.16666666666667, 68.5, 67.66666666666666, 66.91666666666667, 65.99999999999999, 65.33333333333333, 64.5, 63.58333333333333, 62.66666666666667, 61.74999999999999, 60.916666666666664, 60.0, 59.08333333333333, 58.166666666666664, 57.33333333333333, 56.49999999999999, 55.49999999999999, 54.49999999999999, 53.49999999999999, 52.49999999999999, 51.49999999999999, 50.583333333333336, 49.58333333333333, 48.66666666666667, 47.66666666666667, 46.83333333333333, 45.83333333333333, 45.08333333333333, 44.166666666666664, 43.166666666666664, 42.166666666666664, 41.25, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.5, 33.5, 32.49999999999999, 31.499999999999993, 30.666666666666664, 29.666666666666664, 28.666666666666664, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 1, 7, 13, 21, 30, 38, 46, 55, 63, 70, 77, 81, 89, 97, 99, 104, 109, 116, 122, 128, 135, 140, 143, 148, 156, 160, 164, 168, 173, 178, 183, 188, 195, 203, 207, 216, 222, 228, 238, 244, 249, 256, 261, 266, 273, 279, 286, 296, 305, 314, 323, 332, 341, 350, 360, 368, 378, 387, 398, 406, 416, 427, 438, 449, 459, 470, 481, 492, 502, 512, 524, 536, 548, 560, 572, 583, 595, 606, 618, 628, 640, 649, 660, 672, 684, 695, 705, 717, 729, 741, 753, 765, 776, 788, 800, 812, 822, 834, 846, 857], "exact_score": [98.91666666666666, 98.91666666666666, 98.58333333333333, 98.08333333333333, 97.41666666666666, 96.66666666666667, 96.0, 95.33333333333333, 94.58333333333333, 93.91666666666666, 93.33333333333333, 92.75, 92.41666666666666, 91.75, 91.08333333333333, 90.91666666666667, 90.5, 90.08333333333333, 89.5, 89.0, 88.5, 87.91666666666667, 87.5, 87.25, 86.83333333333333, 86.16666666666666, 85.83333333333333, 85.5, 85.16666666666666, 84.74999999999999, 84.33333333333333, 83.91666666666666, 83.5, 82.91666666666666, 82.25, 81.91666666666666, 81.16666666666667, 80.66666666666666, 80.16666666666666, 79.33333333333333, 78.83333333333333, 78.41666666666667, 77.83333333333333, 77.41666666666667, 77.0, 76.41666666666667, 75.91666666666667, 75.33333333333333, 74.5, 73.75, 73.0, 72.24999999999999, 71.5, 70.75, 70.0, 69.16666666666667, 68.5, 67.58333333333333, 66.91666666666667, 65.99999999999999, 65.33333333333333, 64.5, 63.58333333333333, 62.66666666666667, 61.74999999999999, 60.916666666666664, 60.0, 59.08333333333333, 58.166666666666664, 57.25, 56.49999999999999, 55.49999999999999, 54.49999999999999, 53.49999999999999, 52.49999999999999, 51.49999999999999, 50.583333333333336, 49.58333333333333, 48.66666666666667, 47.66666666666667, 46.83333333333333, 45.83333333333333, 44.99999999999999, 44.166666666666664, 43.166666666666664, 42.166666666666664, 41.25, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.5, 33.5, 32.49999999999999, 31.499999999999993, 30.666666666666664, 29.666666666666664, 28.666666666666664, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 68.815, "threshold_area100": 68.81750000000001, "threshold_area25": 92.39166666666667}, "test": {"score": [90.16223679668623, 89.6444597859855, 88.574387297204, 87.76320331377286, 86.19261304798067, 84.31135657576803, 82.58543320676561, 80.51432516396272, 79.03003106662064, 77.3558853986883, 76.2858129099068, 74.90507421470487, 74.33551950293406, 73.54159475319295, 72.86848463928202, 72.02278218847083, 71.34967207455989, 70.67656196064894, 70.14152571625819, 69.74456334138763, 69.41663790127718, 68.9333793579565, 68.32930617880567, 68.03589920607524, 67.65619606489471, 67.22471522264411, 66.7069382119434, 66.27545736969279, 65.84397652744218, 65.44701415257163, 65.17086641353124, 64.5667932343804, 63.89368312046945, 63.548498446668965, 63.16879530548844, 62.720055229547796, 62.30583362098723, 61.839834311356576, 61.684501208146344, 61.373835001725915, 60.99413186054539, 60.70072488781497, 59.872281670693816, 59.527096996893334, 58.81946841560235, 58.2844321712116, 57.68035899206075, 57.26613738350017, 56.83465654124957, 56.316879530548846, 55.74732481877804, 55.02243700379703, 54.60821539523645, 54.41836382464618, 53.676216775975135, 53.26199516741457, 52.76147739040387, 51.9330341732827, 51.27718329306179, 51.05281325509148, 50.53503624439075, 50.15533310321021, 49.46496375560925, 48.981705212288574, 48.636520538488085, 48.360372799447696, 47.39385571280635, 46.84156023472558, 46.2892647566448, 45.65067311011391, 44.96030376251294, 44.37348981705212, 43.838453572661365, 43.26889886089057, 43.01001035554021, 42.26786330686917, 41.698308595098375, 41.26682775284777, 40.83534691059717, 40.214014497756295, 39.696237487055576, 38.5053503624439, 37.86675871591301, 36.72764929237141, 36.0027614773904, 35.277873662409384, 34.63928201587849, 34.03520883672765, 33.224024853296505, 32.79254401104591, 32.153952364515014, 31.774249223334483, 31.066620642043485, 30.51432516396272, 29.875733517431826, 29.409734207801176, 28.49499482222989, 28.02899551259924, 27.442181567138412, 27.06247842595788, 26.56196064894718], "calls": [0.0, 3.9868829823955814, 8.422506040731792, 10.769761822575077, 14.773904038660685, 18.467380048325854, 21.1425612702796, 24.33551950293407, 26.66551605108733, 29.513289609941317, 31.152916810493615, 33.051432516396275, 33.9316534345875, 35.381429064549536, 36.36520538488091, 37.98757335174318, 39.40283051432517, 40.68001380738695, 41.83638246461857, 42.7166033828098, 43.44149119779082, 44.25267518122195, 45.77148774594408, 46.496375560925095, 47.25578184328616, 48.187780462547465, 49.27511218501899, 50.20711080428028, 51.27718329306179, 51.84673800483258, 52.31273731446324, 53.46910597169485, 54.504659993096304, 55.195029340697275, 55.97169485674836, 56.80013807386952, 57.628581290990674, 58.40524680704176, 58.76769071453227, 59.37176389368312, 60.027614773904034, 60.47635484984467, 61.75353814290645, 62.23679668622714, 63.22057300655851, 64.03175698998965, 64.9292371418709, 65.63686572316189, 66.2927166033828, 67.0693821194339, 67.77701070072489, 68.65723161891611, 69.24404556437695, 69.5029340697273, 70.45219192267864, 70.93545046599931, 71.57404211253021, 72.62685536762167, 73.2827062478426, 73.59337245426303, 74.23196410079392, 74.7152226441146, 75.47462892647566, 76.04418363824647, 76.40662754573697, 76.83810838798757, 77.90818087676908, 78.49499482222988, 79.08180876769072, 79.77217811529168, 80.4970659302727, 81.10113910942354, 81.67069382119433, 82.2920262340352, 82.60269244045564, 83.44839489126683, 84.0697273041077, 84.55298584742837, 85.001725923369, 85.65757680358992, 86.19261304798067, 87.43527787366241, 88.10838798757335, 89.28201587849499, 90.04142216085606, 90.81808767690714, 91.54297549188816, 92.16430790472903, 92.97549188816016, 93.42423196410078, 94.16637901277183, 94.58060062133241, 95.34000690369348, 95.90956161546428, 96.54815326199517, 97.04867103900587, 97.98066965826717, 98.46392820158785, 99.05074214704868, 99.44770452191922, 100.0], "rescued": [0, 10, 33, 41, 56, 66, 72, 80, 82, 92, 93, 94, 95, 97, 97, 97, 98, 98, 98, 98, 98, 98, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106], "harmed": [0, 40, 125, 180, 286, 405, 511, 639, 727, 834, 897, 978, 1012, 1060, 1099, 1148, 1188, 1227, 1258, 1281, 1300, 1328, 1365, 1382, 1405, 1430, 1460, 1485, 1510, 1533, 1549, 1584, 1624, 1644, 1666, 1692, 1716, 1743, 1752, 1770, 1792, 1809, 1857, 1877, 1919, 1950, 1985, 2009, 2034, 2064, 2097, 2139, 2163, 2174, 2217, 2241, 2270, 2318, 2356, 2370, 2400, 2422, 2462, 2490, 2510, 2526, 2582, 2614, 2646, 2683, 2723, 2757, 2788, 2821, 2836, 2880, 2913, 2938, 2963, 2999, 3029, 3098, 3135, 3201, 3243, 3285, 3322, 3357, 3404, 3429, 3466, 3488, 3529, 3561, 3598, 3625, 3678, 3705, 3739, 3761, 3791], "exact_score": [90.16223679668623, 90.11045909561615, 90.05868139454608, 89.90334829133586, 89.6444597859855, 89.43734898170521, 89.26475664480496, 89.02312737314463, 88.76423886779426, 88.31549879185364, 88.03935105281326, 87.64238867794269, 87.24542630307214, 86.91750086296169, 86.52053848809112, 86.12357611322057, 85.53676216775975, 85.03624439074905, 84.44943044528823, 83.96617190196754, 83.31032102174663, 82.67172937521575, 82.11943389713497, 81.39454608215395, 80.70417673455297, 80.08284432171212, 79.44425267518122, 78.8574387297204, 78.27062478425958, 77.64929237141871, 77.09699689333793, 76.37210907835691, 75.716258198136, 74.9395926820849, 74.30100103555401, 73.74870555747324, 73.16189161201243, 72.55781843286158, 72.02278218847083, 71.60856057991025, 71.09078356920952, 70.5039696237487, 70.05522954780807, 69.62374870555747, 69.12323092854677, 68.63997238522609, 68.27752847773559, 67.79426993441491, 67.31101139109424, 66.8622713151536, 66.30997583707283, 66.01656886434242, 65.37797721781152, 64.80842250604073, 64.18709009319986, 63.703831549879176, 63.16879530548844, 62.59924059371764, 62.06420434932689, 61.54642733862617, 61.011391094235414, 60.3555402140145, 59.66517086641353, 58.94028305143251, 58.301691404901625, 57.6458405246807, 57.05902657921988, 56.38591646530894, 55.55747324818778, 54.76354849844667, 54.03866068346566, 53.21021746634449, 52.450811183983426, 51.57059026579219, 50.70762858129099, 49.913703831549874, 49.03348291335865, 48.22229892992751, 47.32481877804625, 46.37556092509492, 45.443562305833616, 44.45978598550224, 43.527787366240936, 42.6475664480497, 41.784604763548494, 40.85260614428719, 39.88608905764583, 38.91957197100449, 37.98757335174317, 37.0037970314118, 36.05453917846047, 35.122540559199166, 34.207801173627885, 33.20676561960648, 32.274767000345186, 31.411805315843978, 30.42802899551259, 29.46151190887124, 28.477735588539865, 27.49395926820849, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 64.12117942108348, "threshold_area100": 55.27036589575422, "threshold_area25": 76.1460131170176}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 0.8802209181912324}}, "old_normalized_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 1.070072488781498}}, "old_normalized_endpoints": {"val": {"score": 98.91666666666666, "calls": 0.0}, "test": {"score": 90.16223679668623, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 97.91666666666666, 97.5, 97.41666666666666, 97.24999999999999, 97.16666666666667, 96.91666666666666, 96.58333333333333, 96.41666666666666, 95.75, 95.16666666666667, 94.75, 94.41666666666666, 93.91666666666666, 93.41666666666666, 92.75, 92.08333333333333, 91.41666666666667, 90.75, 89.83333333333333, 89.16666666666666, 88.33333333333333, 87.58333333333333, 86.75, 85.91666666666666, 84.91666666666666, 84.08333333333333, 83.16666666666667, 82.5, 81.66666666666667, 80.75, 79.91666666666666, 79.0, 78.08333333333333, 77.33333333333333, 76.33333333333333, 75.41666666666667, 74.41666666666666, 73.5, 72.58333333333333, 71.66666666666666, 70.75, 69.75, 68.91666666666667, 68.08333333333333, 67.25, 66.25, 65.25, 64.25, 63.33333333333333, 62.416666666666664, 61.416666666666664, 60.416666666666664, 59.5, 58.58333333333333, 57.74999999999999, 56.666666666666664, 55.666666666666664, 54.833333333333336, 53.833333333333336, 52.91666666666666, 51.916666666666664, 51.08333333333333, 50.16666666666667, 49.166666666666664, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.333333333333336, 42.41666666666667, 41.41666666666667, 40.5, 39.5, 38.5, 37.583333333333336, 36.583333333333336, 35.583333333333336, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 97.91666666666666, 97.5, 97.41666666666666, 97.24999999999999, 97.16666666666667, 96.91666666666666, 96.58333333333333, 96.41666666666666, 95.75, 95.16666666666667, 94.75, 94.41666666666666, 93.91666666666666, 93.41666666666666, 92.75, 92.08333333333333, 91.41666666666667, 90.75, 89.83333333333333, 89.16666666666666, 88.33333333333333, 87.58333333333333, 86.75, 85.91666666666666, 84.91666666666666, 84.08333333333333, 83.08333333333333, 82.5, 81.66666666666667, 80.75, 79.91666666666666, 79.0, 78.08333333333333, 77.33333333333333, 76.33333333333333, 75.41666666666667, 74.41666666666666, 73.5, 72.58333333333333, 71.66666666666666, 70.75, 69.75, 68.83333333333333, 68.08333333333333, 67.25, 66.25, 65.25, 64.25, 63.33333333333333, 62.416666666666664, 61.416666666666664, 60.416666666666664, 59.5, 58.58333333333333, 57.666666666666664, 56.666666666666664, 55.666666666666664, 54.833333333333336, 53.833333333333336, 52.91666666666666, 51.916666666666664, 51.08333333333333, 50.16666666666667, 49.166666666666664, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.333333333333336, 42.41666666666667, 41.41666666666667, 40.5, 39.5, 38.5, 37.583333333333336, 36.583333333333336, 35.583333333333336, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.85, "exact_area100": 71.87166666666667}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.07594062823611, 90.04142216085606, 90.04142216085606, 89.8688298239558, 89.85157059026578, 89.78253365550569, 89.69623748705557, 89.55816361753539, 89.19571971004487, 89.09216430790472, 88.95409043838454, 88.69520193303417, 88.48809112875388, 88.24646185709354, 88.03935105281326, 87.83224024853297, 87.53883327580255, 86.90024162927166, 86.27890921643079, 85.83016914049016, 85.26061442871936, 84.76009665170866, 84.05246807041767, 83.39661719019675, 82.56817397307559, 81.96410079392474, 81.17017604418363, 80.721435968243, 79.77217811529168, 78.75388332758025, 78.09803244735933, 77.51121850189851, 77.06247842595788, 75.80255436658612, 74.95685191577493, 73.69692785640318, 73.28270624784258, 72.54055919917155, 71.60856057991025, 70.62478425957887, 69.72730410769762, 68.72626855367622, 68.29478771142561, 67.19019675526407, 66.46530894028305, 65.87849499482222, 65.08457024508112, 64.48049706593027, 63.58301691404902, 62.65101829478771, 62.133241284086985, 61.2185018985157, 59.803244735933724, 57.90472903003106, 57.128063513979974, 56.21332412840869, 55.47117707973766, 54.936140835346905, 53.91784604763548, 53.03762512944425, 52.46807041767345, 51.829478771142554, 51.15636865723161, 50.75940628236106, 49.913703831549874, 48.826372109078356, 47.87711425612702, 47.497411114946495, 47.03141180531584, 46.513634794615115, 45.49534000690369, 44.70141525716258, 44.20089748015188, 43.406972730410764, 42.37141870900932, 41.59475319295823, 40.83534691059717, 39.86882982395581, 39.05764584052468, 38.12564722126337, 37.745944080082836, 37.27994477045219, 36.69313082499137, 35.8646876078702, 34.8636520538488, 34.22506040731791, 33.551950293406975, 32.68898860890575, 31.739730755954433, 31.17017604418364, 30.3589920607525, 29.27166033828098, 28.253365550569555, 27.26958923023818, 26.56196064894718], "calls": [0.0, 1.070072488781498, 2.226441146013117, 3.2274767000345186, 3.8833275802554366, 4.6427338626165, 5.816361753538143, 7.30065585088022, 8.422506040731792, 10.182947877114257, 10.925094925785295, 11.926130479806698, 12.737314463237832, 13.738350017259235, 15.119088712461167, 16.137383500172593, 16.896789782533656, 18.17397307559544, 19.347600966517085, 21.039005868139455, 22.109078356920953, 23.006558508802208, 23.990334829133587, 25.612702795995858, 27.096996893337938, 28.11529168104936, 29.306178805661027, 30.4280289955126, 32.34380393510528, 33.63824646185709, 35.001725923369, 35.950983776320335, 37.176389368312044, 37.81498101484294, 39.23023817742492, 40.73179150845703, 41.698308595098375, 42.61304798066966, 43.2861580945806, 44.874007594062824, 45.97859855022437, 47.53192958232654, 48.118743527787366, 49.119779081808765, 50.5005177770107, 51.794960303762515, 52.83051432516396, 54.02140144977563, 54.53917846047636, 55.95443562305834, 56.80013807386952, 57.594062823610635, 58.47428374180187, 59.11287538833275, 60.148429409734206, 61.25302036589575, 61.8570935450466, 62.87538833275803, 64.5667932343804, 66.6379012771833, 67.5008629616845, 68.48463928201588, 69.278564031757, 69.83085950983777, 70.93545046599931, 71.97100448740076, 72.59233690024163, 73.29996548153262, 74.02485329651364, 74.49085260614429, 75.38833275802554, 76.64825681739731, 77.61477390403866, 77.99447704521918, 78.52951328960994, 79.06454953400069, 80.2209181912323, 81.08387987573352, 81.61891612012427, 82.4991370383155, 83.53469105971695, 84.38039351052812, 85.19157749395927, 86.24439074905074, 87.05557473248187, 88.02209181912323, 88.41905419399379, 88.91957197100449, 89.52364515015533, 90.36934760096652, 91.37038315498792, 92.07801173627891, 92.75112185018986, 93.64860200207112, 94.68415602347255, 95.25371073524336, 96.08215395236451, 97.18674490852605, 98.27407663099758, 99.29237141870901, 100.0], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.05868139454608, 90.04142216085606, 90.04142216085606, 90.04142216085606, 89.90334829133586, 89.85157059026578, 89.78253365550569, 89.67897825336554, 89.52364515015533, 89.2129789437349, 89.12668277528478, 88.95409043838454, 88.76423886779426, 88.59164653089402, 88.40179496030376, 88.22920262340351, 88.0738695201933, 87.84949948222298, 87.53883327580255, 87.10735243355194, 86.69313082499137, 86.33068691750086, 85.88194684156024, 85.39868829823956, 84.94994822229893, 84.58750431480841, 84.15602347255782, 83.81083879875733, 83.20676561960649, 82.56817397307559, 81.96410079392474, 81.2909906800138, 80.58336209872282, 79.96202968588194, 79.23714187090093, 78.59855022437003, 77.89092164307905, 77.2868484639282, 76.47566448049706, 75.68173973075595, 74.9395926820849, 74.14566793234381, 73.36900241629272, 72.66137383500173, 71.97100448740076, 71.19433897134967, 70.45219192267864, 69.5719710044874, 68.76078702105626, 67.88056610286503, 67.17293752157404, 66.34449430445288, 65.51605108733172, 64.60131170176044, 63.7210907835692, 62.89264756644805, 62.01242664825681, 61.11494649637555, 60.2865032792544, 59.42354159475318, 58.49154297549188, 57.57680358992061, 56.67932343803934, 55.712806351397994, 54.780807732136694, 53.86606834656541, 53.02036589575423, 52.12288574387297, 51.19088712461166, 50.293406972730416, 49.39592682084915, 48.481187435277874, 47.497411114946495, 46.582671729375214, 45.68519157749395, 44.75319295823265, 43.838453572661365, 42.906454953400065, 41.95719710044873, 41.00793924749741, 40.12771832930617, 39.12668277528477, 38.160165688643424, 37.21090783569208, 36.22713151536071, 35.24335519502934, 34.31135657576804, 33.32758025543665, 32.34380393510528, 31.429064549534004, 30.445288229202617, 29.46151190887124, 28.52951328960994, 27.545736969278558, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 89.02140144977564, "exact_area100": 67.43292520835892}}, "old_normalized_endpoint_budget": 0}, {"seed": 2028, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.5, "validation_area100": 0.71905, "training_seconds": 3.2087002920452505, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.03182246644062272, "operator": "ge"}, {"budget": 0.02, "threshold": -0.03572296533709913, "operator": "ge"}, {"budget": 0.03, "threshold": -0.0394868949178349, "operator": "ge"}, {"budget": 0.04, "threshold": -0.04543171513426322, "operator": "ge"}, {"budget": 0.05, "threshold": -0.056371148806048856, "operator": "ge"}, {"budget": 0.06, "threshold": -0.05953878795068429, "operator": "ge"}, {"budget": 0.07, "threshold": -0.07503198583878844, "operator": "ge"}, {"budget": 0.08, "threshold": -0.0885823148784935, "operator": "ge"}, {"budget": 0.09, "threshold": -0.12638227204995328, "operator": "ge"}, {"budget": 0.1, "threshold": -0.14209242380008472, "operator": "ge"}, {"budget": 0.11, "threshold": -0.16918680505531983, "operator": "ge"}, {"budget": 0.12, "threshold": -0.1908086303012441, "operator": "ge"}, {"budget": 0.13, "threshold": -0.20882460780685685, "operator": "ge"}, {"budget": 0.14, "threshold": -0.25276341950958275, "operator": "ge"}, {"budget": 0.15, "threshold": -0.29413670783146995, "operator": "ge"}, {"budget": 0.16, "threshold": -0.3217119355398308, "operator": "ge"}, {"budget": 0.17, "threshold": -0.3659424420604959, "operator": "ge"}, {"budget": 0.18, "threshold": -0.42725293422461585, "operator": "ge"}, {"budget": 0.19, "threshold": -0.461083918571105, "operator": "ge"}, {"budget": 0.2, "threshold": -0.4986364743969771, "operator": "ge"}, {"budget": 0.21, "threshold": -0.5517842103296003, "operator": "ge"}, {"budget": 0.22, "threshold": -0.601927944331365, "operator": "ge"}, {"budget": 0.23, "threshold": -0.6368866368204726, "operator": "ge"}, {"budget": 0.24, "threshold": -0.6695142219513008, "operator": "ge"}, {"budget": 0.25, "threshold": -0.7253987428250056, "operator": "ge"}, {"budget": 0.26, "threshold": -0.7582896592948699, "operator": "ge"}, {"budget": 0.27, "threshold": -0.7850117105873972, "operator": "ge"}, {"budget": 0.28, "threshold": -0.8113057759923431, "operator": "ge"}, {"budget": 0.29, "threshold": -0.8261100961185737, "operator": "ge"}, {"budget": 0.3, "threshold": -0.8398064871529964, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8486025038545352, "operator": "ge"}, {"budget": 0.32, "threshold": -0.8586231424192103, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8717361494822609, "operator": "ge"}, {"budget": 0.34, "threshold": -0.8842994520183196, "operator": "ge"}, {"budget": 0.35, "threshold": -0.90303202967213, "operator": "ge"}, {"budget": 0.36, "threshold": -0.913916031957304, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9194088499967834, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9266571915096514, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9320540896151139, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9366356705488896, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9400985182369765, "operator": "ge"}, {"budget": 0.42, "threshold": -0.946500760855869, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9531547705934806, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9568049579959585, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9599363170255454, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9633302011202627, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9661704544553377, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9685483276447493, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9704164432870938, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9731513232088469, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9758166587346829, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9771937778970303, "operator": "ge"}, {"budget": 0.53, "threshold": -0.978474827297856, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9803334533617631, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9818400359984621, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9828047833096069, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9839379175788785, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9850859282903301, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9858717713725691, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9865278665535596, "operator": "ge"}, {"budget": 0.61, "threshold": -0.987168719190116, "operator": "ge"}, {"budget": 0.62, "threshold": -0.987769749646986, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9884161847706355, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9887525417232221, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9895135627642915, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9903420740228686, "operator": "ge"}, {"budget": 0.67, "threshold": -0.990972129104757, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9920115372885451, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9924284718035236, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9929329087962222, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9936570536501547, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9941195517379349, "operator": "ge"}, {"budget": 0.73, "threshold": -0.994808229921212, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9952008169545729, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9956262664657454, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9960344157037958, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9963452768628466, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9967919702544743, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9970599573096833, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9974075635865928, "operator": "ge"}, {"budget": 0.81, "threshold": -0.997643739893524, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9978631709335196, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9980240492527318, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9981380299083445, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9982350979615194, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9983273708847783, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9984775183158305, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9985567227927017, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9986628744358278, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9987390249687593, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9987985665497596, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9988731218316369, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9989863933225762, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9990668920884187, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9991308262714083, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9992348950224018, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9993149023508027, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9994028046233363, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9994969488087047, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.66666666666667, 98.25, 98.25, 98.16666666666667, 98.0, 97.74999999999999, 97.41666666666666, 97.24999999999999, 97.24999999999999, 97.0, 96.91666666666666, 96.75, 96.5, 96.0, 95.58333333333333, 94.91666666666666, 94.5, 93.91666666666666, 93.5, 92.75, 92.0, 91.41666666666667, 90.58333333333333, 89.83333333333333, 89.08333333333334, 88.33333333333333, 87.41666666666667, 86.58333333333333, 85.75, 84.91666666666666, 84.25, 83.41666666666666, 82.66666666666667, 81.75, 80.83333333333333, 79.83333333333333, 78.91666666666666, 78.16666666666666, 77.33333333333333, 76.5, 75.58333333333334, 74.58333333333333, 73.75, 72.83333333333333, 71.91666666666666, 71.16666666666667, 70.16666666666667, 69.24999999999999, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.25, 63.24999999999999, 62.33333333333333, 61.5, 60.5, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.833333333333336, 55.833333333333336, 55.00000000000001, 54.0, 53.0, 52.0, 51.0, 49.99999999999999, 49.0, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.5, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 4, 9, 9, 10, 12, 15, 19, 21, 21, 24, 25, 27, 30, 36, 41, 49, 54, 62, 67, 76, 85, 93, 103, 112, 121, 130, 141, 151, 161, 171, 179, 189, 198, 209, 220, 232, 243, 252, 262, 272, 283, 295, 305, 316, 327, 336, 348, 359, 372, 384, 396, 408, 419, 431, 442, 452, 464, 473, 484, 495, 508, 520, 530, 542, 554, 566, 578, 590, 602, 612, 624, 636, 647, 658, 670, 682, 693, 705, 716, 728, 740, 752, 764, 776, 788, 800, 810, 821, 833, 845, 857], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.66666666666667, 98.25, 98.25, 98.16666666666667, 98.0, 97.74999999999999, 97.41666666666666, 97.24999999999999, 97.24999999999999, 97.0, 96.91666666666666, 96.75, 96.5, 96.0, 95.58333333333333, 94.91666666666666, 94.5, 93.91666666666666, 93.5, 92.75, 92.0, 91.41666666666667, 90.58333333333333, 89.83333333333333, 89.08333333333334, 88.33333333333333, 87.41666666666667, 86.58333333333333, 85.75, 84.91666666666666, 84.25, 83.33333333333333, 82.66666666666667, 81.75, 80.83333333333333, 79.83333333333333, 78.91666666666666, 78.16666666666666, 77.33333333333333, 76.5, 75.58333333333334, 74.58333333333333, 73.75, 72.83333333333333, 71.91666666666666, 71.16666666666667, 70.16666666666667, 69.16666666666667, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.25, 63.24999999999999, 62.33333333333333, 61.5, 60.5, 59.74999999999999, 58.833333333333336, 57.833333333333336, 56.833333333333336, 55.833333333333336, 55.00000000000001, 54.0, 53.0, 52.0, 51.0, 49.99999999999999, 49.0, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.5, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.905, "threshold_area100": 71.90833333333333, "threshold_area25": 97.83333333333333}, "test": {"score": [90.16223679668623, 90.1449775629962, 90.12771832930618, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.07594062823611, 90.07594062823611, 89.90334829133586, 89.76527442181566, 89.7134967207456, 89.52364515015533, 89.47186744908525, 89.24749741111494, 89.05764584052469, 88.97134967207455, 88.53986882982396, 88.17742492233344, 88.0048325854332, 87.76320331377286, 87.52157404211253, 87.0728339661719, 86.67587159130134, 86.31342768381084, 85.41594753192958, 84.63928201587849, 84.05246807041767, 83.18950638591647, 82.70624784259579, 82.22298929927511, 81.8605453917846, 81.48084225060407, 80.89402830514325, 79.97928891957197, 78.42595788746979, 77.2868484639282, 76.54470141525715, 75.68173973075595, 74.92233344839488, 74.18018639972385, 73.5070762858129, 72.33344839489126, 70.98722816706939, 70.2105626510183, 69.38211943389715, 68.50189851570589, 67.75975146703486, 67.17293752157404, 66.53434587504314, 65.46427338626164, 64.48049706593027, 63.85916465308941, 63.099758370728345, 62.09872281670694, 61.13220573006558, 60.510873317224714, 59.69968933379357, 58.66413531239212, 57.87021056265102, 57.38695201933034, 56.61028650327925, 55.79910251984812, 55.1432516396272, 54.81532619951675, 54.004142216085604, 52.96858819468415, 52.226441146013116, 51.08733172247152, 50.77666551605109, 50.05177770107008, 49.03348291335865, 48.44666896789782, 47.37659647911632, 46.738004832585425, 46.237487055574725, 45.581636175353815, 45.06385916465309, 44.114601311701755, 43.66586123576113, 42.78564031756989, 42.07801173627891, 41.197790818087675, 40.38660683465653, 39.81705212288574, 39.43734898170521, 38.885053503624434, 37.84949948222298, 37.33172247152227, 36.5723161891612, 35.79565067311011, 35.10528132550914, 34.24231964100793, 32.96513634794614, 31.912323092854677, 31.101139109423535, 30.04832585433207, 28.96099413186054, 28.253365550569555, 27.33862616499827, 26.56196064894718], "calls": [0.0, 1.6741456679323439, 2.7096996893337937, 3.5726613738350017, 4.6427338626165, 5.574732481877804, 5.816361753538143, 7.404211253020366, 8.301691404901621, 10.700724887814982, 11.598205039696238, 12.961684501208145, 13.68657231618916, 14.169830859509839, 15.412495685191576, 16.896789782533656, 17.587159130134623, 18.86434242319641, 20.365895754228514, 21.17707973765965, 22.074559889540904, 23.403520883672766, 24.663444943044528, 25.62996202968588, 26.458405246807043, 28.339661719019677, 29.84121505005178, 31.083879875733516, 32.61995167414567, 33.53469105971695, 34.4321712115982, 35.22609596133932, 35.86468760787021, 36.969278564031754, 38.47083189506386, 40.88712461166724, 42.49223334483949, 43.49326889886089, 44.68415602347256, 45.857783914394204, 46.80704176734553, 47.75629962029686, 49.22333448394891, 50.84570245081118, 51.794960303762515, 52.70969968933379, 53.693476009665176, 54.71177079737659, 55.52295478080773, 56.24784259578875, 57.55954435623059, 58.76769071453227, 59.527096996893334, 60.39005868139454, 61.649982740766305, 62.771832930617876, 63.47946151190887, 64.34242319641008, 65.46427338626165, 66.41353123921297, 66.98308595098378, 67.89782533655506, 68.74352778736625, 69.46841560234725, 69.88263721090784, 70.79737659647911, 72.04004142216085, 72.83396617190196, 74.0766309975837, 74.45633413876423, 75.23299965481533, 76.30307214359682, 76.9244045564377, 78.06351397997929, 78.70210562651019, 79.23714187090093, 79.89299275112185, 80.46254746289266, 81.51536071798412, 82.01587849499482, 82.89609941318605, 83.75906109768727, 84.65654124956852, 85.51950293406972, 86.10631687953055, 86.5032792544011, 87.0728339661719, 88.22920262340352, 88.76423886779426, 89.55816361753539, 90.38660683465655, 91.11149464963756, 91.99171556782879, 93.33793579565068, 94.39074905074214, 95.25371073524336, 96.427338626165, 97.54918881601657, 98.27407663099758, 99.22333448394892, 100.0], "rescued": [0, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3, 4, 5, 5, 7, 9, 10, 11, 14, 14, 15, 20, 22, 24, 29, 32, 34, 37, 41, 45, 47, 49, 50, 54, 55, 58, 62, 64, 65, 70, 70, 74, 75, 77, 78, 78, 78, 81, 82, 83, 84, 86, 86, 86, 88, 90, 90, 91, 92, 94, 94, 94, 94, 94, 95, 95, 98, 98, 98, 98, 99, 99, 99, 99, 99, 99, 99, 100, 101, 101, 101, 101, 101, 102, 102, 102, 102, 103, 103, 104, 104, 104, 104, 104, 104, 104, 106, 106, 106, 106, 106], "harmed": [0, 2, 3, 3, 3, 4, 4, 8, 8, 18, 26, 30, 42, 45, 60, 73, 79, 105, 129, 139, 154, 173, 201, 226, 252, 307, 354, 391, 445, 477, 507, 530, 553, 591, 645, 738, 808, 853, 904, 953, 996, 1039, 1108, 1188, 1234, 1282, 1333, 1379, 1414, 1452, 1515, 1574, 1610, 1654, 1714, 1772, 1808, 1856, 1917, 1965, 1993, 2038, 2085, 2123, 2143, 2190, 2253, 2296, 2362, 2380, 2423, 2482, 2516, 2578, 2615, 2644, 2682, 2713, 2769, 2795, 2846, 2887, 2938, 2986, 3019, 3041, 3073, 3134, 3164, 3209, 3254, 3294, 3344, 3418, 3479, 3526, 3589, 3652, 3693, 3746, 3791], "exact_score": [90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.1449775629962, 90.12771832930618, 90.1449775629962, 90.11045909561615, 90.07594062823611, 90.02416292716603, 89.97238522609597, 89.8688298239558, 89.76527442181566, 89.69623748705557, 89.48912668277528, 89.29927511218501, 89.23023817742491, 89.05764584052469, 88.85053503624438, 88.47083189506385, 88.24646185709354, 88.03935105281326, 87.81498101484294, 87.5733517431826, 87.33172247152227, 86.93476009665171, 86.48602002071108, 86.03727994477045, 85.58853986882981, 85.08802209181911, 84.55298584742837, 84.06972730410769, 83.55195029340697, 82.94787711425612, 82.48187780462547, 81.98136002761477, 81.41180531584398, 80.89402830514325, 80.2899551259924, 79.70314118053157, 79.04729030031066, 78.32240248532965, 77.64929237141871, 76.90714532274767, 76.14773904038661, 75.47462892647566, 74.81877804625474, 74.04211253020365, 73.28270624784258, 72.54055919917155, 71.67759751467034, 70.93545046599931, 70.03797031411804, 69.12323092854677, 68.29478771142561, 67.58715913013462, 66.77597514670349, 65.96479116327234, 65.10182947877115, 64.29064549534, 63.46220227821884, 62.633759061097685, 61.839834311356576, 60.90783569209527, 60.01035554021401, 59.13013462202278, 58.249913703831545, 57.38695201933034, 56.541249568519156, 55.59199171556782, 54.71177079737659, 53.86606834656541, 53.02036589575423, 52.08836727649292, 51.17362789092164, 50.293406972730416, 49.34414911977908, 48.39489126682775, 47.44563341387642, 46.47911632723506, 45.47808077321366, 44.58060062133241, 43.683120469451154, 42.73386261649982, 41.85364169830859, 40.86986537797721, 39.93786675871591, 38.95409043838454, 38.073869520193306, 37.14187090093199, 36.175353814290645, 35.22609596133931, 34.24231964100793, 33.31032102174664, 32.30928546772523, 31.360027614773898, 30.410769761822575, 29.478771142561268, 28.52951328960994, 27.545736969278558, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 67.26107310611039, "threshold_area100": 64.41033828098031, "threshold_area25": 88.90403866068347}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0011190498480573297, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0013990203151479363, "operator": "ge"}, {"budget": 0.03, "threshold": -0.001830799737945199, "operator": "ge"}, {"budget": 0.04, "threshold": -0.0025035140570253134, "operator": "ge"}, {"budget": 0.05, "threshold": -0.003437571693211794, "operator": "ge"}, {"budget": 0.06, "threshold": -0.005187693051993847, "operator": "ge"}, {"budget": 0.07, "threshold": -0.0070608328096568584, "operator": "ge"}, {"budget": 0.08, "threshold": -0.011630993336439133, "operator": "ge"}, {"budget": 0.09, "threshold": -0.01857956312596798, "operator": "ge"}, {"budget": 0.1, "threshold": -0.022862177342176437, "operator": "ge"}, {"budget": 0.11, "threshold": -0.03005300462245941, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03824026510119438, "operator": "ge"}, {"budget": 0.13, "threshold": -0.04742180556058884, "operator": "ge"}, {"budget": 0.14, "threshold": -0.06446719914674759, "operator": "ge"}, {"budget": 0.15, "threshold": -0.08739972114562988, "operator": "ge"}, {"budget": 0.16, "threshold": -0.10366135835647583, "operator": "ge"}, {"budget": 0.17, "threshold": -0.13644328713417053, "operator": "ge"}, {"budget": 0.18, "threshold": -0.18430429697036743, "operator": "ge"}, {"budget": 0.19, "threshold": -0.23062685132026672, "operator": "ge"}, {"budget": 0.2, "threshold": -0.2647975981235504, "operator": "ge"}, {"budget": 0.21, "threshold": -0.33325955271720886, "operator": "ge"}, {"budget": 0.22, "threshold": -0.39448583126068115, "operator": "ge"}, {"budget": 0.23, "threshold": -0.4377390146255493, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5181083679199219, "operator": "ge"}, {"budget": 0.25, "threshold": -0.5647038817405701, "operator": "ge"}, {"budget": 0.26, "threshold": -0.6130234003067017, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6513184309005737, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6802272200584412, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6975713968276978, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7246795296669006, "operator": "ge"}, {"budget": 0.31, "threshold": -0.7409618496894836, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7667350769042969, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8024205565452576, "operator": "ge"}, {"budget": 0.34, "threshold": -0.8267925381660461, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8368687033653259, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8473595380783081, "operator": "ge"}, {"budget": 0.37, "threshold": -0.85830157995224, "operator": "ge"}, {"budget": 0.38, "threshold": -0.868726909160614, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8776310682296753, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8882609605789185, "operator": "ge"}, {"budget": 0.41, "threshold": -0.895064115524292, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9059866666793823, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9136863350868225, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9197357296943665, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9294564127922058, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9344344735145569, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9394068717956543, "operator": "ge"}, {"budget": 0.48, "threshold": -0.942288875579834, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9473953247070312, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9510630369186401, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9550990462303162, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9578934907913208, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9611067175865173, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9643106460571289, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9659181237220764, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9683947563171387, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9700448513031006, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9719187021255493, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9729506969451904, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9740110039710999, "operator": "ge"}, {"budget": 0.61, "threshold": -0.97538161277771, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9760122299194336, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9773836135864258, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9786421060562134, "operator": "ge"}, {"budget": 0.65, "threshold": -0.980779767036438, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9821118116378784, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9837565422058105, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9847415089607239, "operator": "ge"}, {"budget": 0.69, "threshold": -0.985465407371521, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9866735935211182, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9882080554962158, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9894421696662903, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9904011487960815, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9911808967590332, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9918985962867737, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9925606846809387, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9932406544685364, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9939240217208862, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9944728016853333, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9949681758880615, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9955384731292725, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9958388209342957, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9961917400360107, "operator": "ge"}, {"budget": 0.84, "threshold": -0.996372401714325, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9965961575508118, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9968523979187012, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9970159530639648, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9971617460250854, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9974223971366882, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9975407123565674, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9976755380630493, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9978775978088379, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9980362057685852, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9981564879417419, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9983261823654175, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9984853863716125, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9986380338668823, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9988088607788086, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9989950060844421, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.41666666666666, 98.25, 98.16666666666667, 98.16666666666667, 98.0, 97.83333333333333, 97.41666666666666, 97.33333333333333, 97.33333333333333, 97.16666666666667, 97.16666666666667, 96.91666666666666, 96.66666666666667, 96.08333333333333, 95.58333333333333, 95.08333333333333, 94.66666666666667, 94.16666666666667, 93.5, 92.66666666666666, 92.0, 91.25, 90.25, 89.66666666666666, 89.25, 88.33333333333333, 87.5, 86.58333333333333, 85.83333333333333, 85.08333333333333, 84.25, 83.58333333333333, 82.66666666666667, 81.66666666666667, 80.75, 79.83333333333333, 79.08333333333333, 78.25, 77.33333333333333, 76.41666666666667, 75.5, 74.58333333333333, 73.66666666666666, 72.75, 71.83333333333333, 70.83333333333333, 69.83333333333333, 68.91666666666667, 67.91666666666667, 66.91666666666667, 65.99999999999999, 64.99999999999999, 64.08333333333334, 63.24999999999999, 62.33333333333333, 61.416666666666664, 60.58333333333332, 59.833333333333336, 58.833333333333336, 57.91666666666666, 56.833333333333336, 55.91666666666666, 55.00000000000001, 54.0, 53.0, 52.0, 51.0, 49.99999999999999, 49.166666666666664, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.33333333333333, 44.25, 43.25, 42.333333333333336, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.5, 32.49999999999999, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 0, 0, 1, 1, 1, 1, 1, 1, 2, 6, 8, 9, 9, 12, 14, 19, 20, 20, 22, 22, 25, 28, 35, 41, 47, 52, 58, 66, 76, 84, 93, 105, 112, 119, 130, 140, 151, 160, 169, 179, 187, 198, 210, 221, 232, 241, 251, 262, 273, 284, 295, 306, 317, 328, 340, 352, 363, 375, 387, 398, 410, 421, 431, 442, 453, 463, 472, 484, 495, 508, 519, 530, 542, 554, 566, 578, 590, 600, 612, 624, 636, 646, 659, 671, 682, 694, 705, 717, 729, 741, 753, 764, 776, 788, 800, 809, 821, 833, 845, 857], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.41666666666666, 98.25, 98.16666666666667, 98.16666666666667, 98.0, 97.83333333333333, 97.41666666666666, 97.33333333333333, 97.33333333333333, 97.16666666666667, 97.16666666666667, 96.91666666666666, 96.66666666666667, 96.08333333333333, 95.58333333333333, 95.08333333333333, 94.66666666666667, 94.16666666666667, 93.5, 92.66666666666666, 92.0, 91.25, 90.25, 89.66666666666666, 89.25, 88.33333333333333, 87.5, 86.58333333333333, 85.83333333333333, 85.08333333333333, 84.25, 83.5, 82.66666666666667, 81.66666666666667, 80.75, 79.83333333333333, 79.08333333333333, 78.25, 77.33333333333333, 76.41666666666667, 75.5, 74.58333333333333, 73.66666666666666, 72.75, 71.83333333333333, 70.83333333333333, 69.83333333333333, 68.83333333333333, 67.91666666666667, 66.91666666666667, 65.99999999999999, 64.99999999999999, 64.08333333333334, 63.24999999999999, 62.33333333333333, 61.416666666666664, 60.58333333333332, 59.833333333333336, 58.833333333333336, 57.833333333333336, 56.833333333333336, 55.91666666666666, 55.00000000000001, 54.0, 53.0, 52.0, 51.0, 49.99999999999999, 49.166666666666664, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.25, 43.25, 42.333333333333336, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.5, 32.49999999999999, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.88749999999999, "threshold_area100": 71.89083333333332, "threshold_area25": 97.85000000000001}, "test": {"score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.11045909561615, 90.11045909561615, 90.05868139454608, 89.98964445978598, 89.90334829133586, 89.81705212288574, 89.73075595443561, 89.66171901967553, 89.48912668277528, 89.33379357956507, 89.28201587849499, 89.07490507421471, 88.79875733517432, 88.66068346565412, 88.43631342768381, 88.05661028650327, 87.60787021056264, 87.26268553676216, 86.48602002071108, 85.98550224370037, 85.46772523299964, 84.82913358646876, 84.32861580945806, 84.05246807041767, 83.6382464618571, 83.37935795650672, 82.75802554366585, 81.3600276147739, 80.32447359337245, 79.87573351743183, 79.2198826372109, 78.59855022437003, 77.7873662409389, 76.99344149119779, 75.83707283396618, 75.16396272005524, 73.95581636175353, 73.0755954435623, 72.35070762858129, 70.95270969968934, 70.36589575422852, 69.52019330341732, 68.96789782533655, 68.08767690714532, 67.25923369002416, 66.18916120124267, 65.11908871246118, 64.30790472903003, 63.15153607179841, 62.426648256817394, 61.39109423541594, 60.735243355195024, 59.66517086641353, 59.13013462202278, 58.577839143942015, 57.76665516051087, 57.26613738350017, 56.47221263375906, 55.69554711770797, 54.29754918881602, 53.52088367276493, 52.76147739040387, 52.31273731446324, 51.76044183638246, 50.91473938557127, 49.87918536416983, 48.826372109078356, 48.11874352778736, 47.58370728339661, 46.927856403175696, 46.18570935450465, 45.42630307214359, 44.56334138764239, 43.92474974111149, 43.2171211598205, 42.37141870900932, 41.59475319295823, 40.83534691059717, 40.421125302036586, 39.86882982395581, 38.86779426993441, 38.40179496030376, 37.93579565067311, 36.900241629271655, 36.27890921643079, 35.41594753192958, 34.08698653779771, 33.137728684846394, 32.41284086986538, 31.308249913703822, 30.27269589230238, 29.288919571971007, 28.443217121159815, 27.545736969278558, 26.56196064894718], "calls": [0.0, 1.052813255091474, 1.9157749395926822, 2.9168104936140837, 3.8660683465654127, 4.366586123576114, 5.661028650327926, 6.541249568519157, 8.267172937521575, 9.734207801173628, 10.787021056265102, 11.753538142906455, 12.461166724197446, 12.996202968588195, 14.169830859509839, 15.550569554711771, 16.189161201242666, 17.172937521574042, 18.484639282015877, 19.416637901277184, 20.279599585778392, 21.953745253710736, 23.33448394891267, 24.111149464963756, 25.854332067656195, 27.062478425957888, 28.14981014842941, 29.288919571971007, 30.238177424922334, 30.824991370383152, 31.567138419054192, 32.20573006558509, 33.258543320676566, 35.36416983085951, 37.021056265101834, 37.69416637901277, 38.76423886779427, 39.60994131860545, 40.766309975837075, 41.905419399378665, 43.57956506731101, 44.390749050742144, 46.082153952364514, 47.204004142216085, 48.25681739730756, 50.25888850535036, 51.03555402140145, 52.05384880911288, 52.67518122195375, 53.79703141180532, 54.83258543320677, 56.092509492578536, 57.26613738350017, 58.249913703831545, 59.59613393165344, 60.44183638246462, 61.615464273386266, 62.37487055574732, 63.634794615119084, 64.25612702795996, 65.03279254401104, 65.98205039696238, 66.5688643424232, 67.53538142906454, 68.46738004832585, 70.10700724887815, 71.0044874007594, 71.93648602002071, 72.54055919917157, 73.1446323783224, 74.05937176389368, 75.18122195374526, 76.33759061097687, 77.09699689333793, 77.68381083879876, 78.37418018639973, 79.15084570245081, 79.96202968588196, 80.91128753883328, 81.6361753538143, 82.46461857093546, 83.31032102174663, 84.10424577148775, 84.88091128753882, 85.36416983085951, 85.91646530894027, 87.03831549879185, 87.5733517431826, 88.09112875388332, 89.24749741111495, 89.88608905764585, 90.78356920952709, 92.16430790472903, 93.11356575768036, 93.83845357266138, 95.01208146358302, 96.15119088712461, 97.18674490852605, 98.08422506040732, 99.01622367966863, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 4, 7, 8, 12, 15, 16, 17, 20, 23, 24, 30, 31, 33, 35, 36, 38, 43, 44, 47, 48, 50, 50, 55, 55, 57, 59, 61, 66, 68, 68, 68, 72, 74, 75, 76, 79, 80, 80, 82, 83, 84, 84, 85, 86, 87, 88, 89, 90, 91, 92, 94, 94, 95, 96, 97, 98, 99, 99, 99, 99, 99, 99, 100, 100, 100, 100, 101, 101, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 106, 106, 106], "harmed": [0, 1, 1, 1, 1, 1, 3, 4, 8, 12, 17, 22, 27, 31, 42, 51, 54, 66, 83, 94, 108, 134, 163, 184, 230, 262, 295, 333, 368, 385, 411, 428, 465, 548, 613, 640, 681, 718, 767, 813, 885, 924, 996, 1049, 1093, 1179, 1215, 1264, 1296, 1351, 1401, 1464, 1527, 1577, 1645, 1687, 1749, 1788, 1851, 1882, 1915, 1963, 1993, 2040, 2086, 2168, 2214, 2259, 2287, 2319, 2369, 2430, 2492, 2534, 2566, 2604, 2647, 2691, 2741, 2778, 2820, 2869, 2914, 2958, 2983, 3015, 3075, 3102, 3130, 3190, 3226, 3276, 3353, 3408, 3450, 3514, 3575, 3632, 3682, 3734, 3791], "exact_score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.07594062823611, 90.05868139454608, 89.97238522609597, 89.8688298239558, 89.78253365550569, 89.66171901967553, 89.48912668277528, 89.38557128063513, 89.28201587849499, 89.07490507421471, 88.90231273731446, 88.72972040041421, 88.574387297204, 88.36727649292371, 88.05661028650327, 87.67690714532274, 87.31446323783224, 86.83120469451156, 86.41698308595099, 86.0027614773904, 85.5712806351398, 85.001725923369, 84.48394891266827, 83.96617190196754, 83.48291335864687, 82.861580945806, 82.18847083189506, 81.60165688643424, 80.9285467725233, 80.3589920607525, 79.70314118053157, 79.04729030031066, 78.30514325163962, 77.61477390403866, 76.90714532274767, 76.19951674145668, 75.50914739385571, 74.76700034518467, 74.02485329651363, 73.26544701415256, 72.55781843286158, 71.76389368312046, 71.10804280289955, 70.40041422160856, 69.58923023817742, 68.65723161891611, 67.96686227131515, 67.10390058681395, 66.2927166033828, 65.39523645150155, 64.51501553331032, 63.634794615119084, 62.7890921643079, 61.943389713496714, 61.08042802899551, 60.20020711080427, 59.33724542630308, 58.59509837763204, 57.749395926820846, 56.95547117707973, 56.127027959958575, 55.26406627545737, 54.41836382464618, 53.53814290645496, 52.70969968933379, 51.91577493959267, 50.98377632033137, 50.05177770107008, 49.17155678287884, 48.22229892992751, 47.27304107697618, 46.34104245771488, 45.391784604763544, 44.49430445288229, 43.59682430100103, 42.682084915429755, 41.715567828788394, 40.74905074214705, 39.799792889195714, 38.90231273731446, 38.0048325854332, 37.14187090093199, 36.175353814290645, 35.22609596133931, 34.25957887469796, 33.25854332067656, 32.25750776665516, 31.32550914739385, 30.42802899551259, 29.46151190887124, 28.52951328960994, 27.56299620296858, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 67.02684320215559, "threshold_area100": 65.18596824301002, "threshold_area25": 89.17846047635484}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.430908203125, "operator": "ge"}, {"budget": 0.02, "threshold": -3.455672025680542, "operator": "ge"}, {"budget": 0.03, "threshold": -4.303503036499023, "operator": "ge"}, {"budget": 0.04, "threshold": -4.775357246398926, "operator": "ge"}, {"budget": 0.05, "threshold": -5.333983421325684, "operator": "ge"}, {"budget": 0.06, "threshold": -6.060446739196777, "operator": "ge"}, {"budget": 0.07, "threshold": -6.509272575378418, "operator": "ge"}, {"budget": 0.08, "threshold": -7.142480850219727, "operator": "ge"}, {"budget": 0.09, "threshold": -7.486845970153809, "operator": "ge"}, {"budget": 0.1, "threshold": -8.040736198425293, "operator": "ge"}, {"budget": 0.11, "threshold": -8.507944107055664, "operator": "ge"}, {"budget": 0.12, "threshold": -8.756447792053223, "operator": "ge"}, {"budget": 0.13, "threshold": -9.145498275756836, "operator": "ge"}, {"budget": 0.14, "threshold": -9.32519817352295, "operator": "ge"}, {"budget": 0.15, "threshold": -9.517339706420898, "operator": "ge"}, {"budget": 0.16, "threshold": -9.610227584838867, "operator": "ge"}, {"budget": 0.17, "threshold": -9.818344116210938, "operator": "ge"}, {"budget": 0.18, "threshold": -10.04969596862793, "operator": "ge"}, {"budget": 0.19, "threshold": -10.20911979675293, "operator": "ge"}, {"budget": 0.2, "threshold": -10.303570747375488, "operator": "ge"}, {"budget": 0.21, "threshold": -10.594806671142578, "operator": "ge"}, {"budget": 0.22, "threshold": -10.71743392944336, "operator": "ge"}, {"budget": 0.23, "threshold": -10.87039566040039, "operator": "ge"}, {"budget": 0.24, "threshold": -11.04682731628418, "operator": "ge"}, {"budget": 0.25, "threshold": -11.221428871154785, "operator": "ge"}, {"budget": 0.26, "threshold": -11.339568138122559, "operator": "ge"}, {"budget": 0.27, "threshold": -11.43185043334961, "operator": "ge"}, {"budget": 0.28, "threshold": -11.530278205871582, "operator": "ge"}, {"budget": 0.29, "threshold": -11.656736373901367, "operator": "ge"}, {"budget": 0.3, "threshold": -11.749855995178223, "operator": "ge"}, {"budget": 0.31, "threshold": -11.816385269165039, "operator": "ge"}, {"budget": 0.32, "threshold": -11.914689064025879, "operator": "ge"}, {"budget": 0.33, "threshold": -12.007546424865723, "operator": "ge"}, {"budget": 0.34, "threshold": -12.068724632263184, "operator": "ge"}, {"budget": 0.35, "threshold": -12.184305191040039, "operator": "ge"}, {"budget": 0.36, "threshold": -12.310015678405762, "operator": "ge"}, {"budget": 0.37, "threshold": -12.383578300476074, "operator": "ge"}, {"budget": 0.38, "threshold": -12.480847358703613, "operator": "ge"}, {"budget": 0.39, "threshold": -12.578542709350586, "operator": "ge"}, {"budget": 0.4, "threshold": -12.680543899536133, "operator": "ge"}, {"budget": 0.41, "threshold": -12.745559692382812, "operator": "ge"}, {"budget": 0.42, "threshold": -12.837606430053711, "operator": "ge"}, {"budget": 0.43, "threshold": -12.941688537597656, "operator": "ge"}, {"budget": 0.44, "threshold": -13.042503356933594, "operator": "ge"}, {"budget": 0.45, "threshold": -13.151430130004883, "operator": "ge"}, {"budget": 0.46, "threshold": -13.350946426391602, "operator": "ge"}, {"budget": 0.47, "threshold": -13.418188095092773, "operator": "ge"}, {"budget": 0.48, "threshold": -13.523433685302734, "operator": "ge"}, {"budget": 0.49, "threshold": -13.599966049194336, "operator": "ge"}, {"budget": 0.5, "threshold": -13.677248001098633, "operator": "ge"}, {"budget": 0.51, "threshold": -13.732585906982422, "operator": "ge"}, {"budget": 0.52, "threshold": -13.778373718261719, "operator": "ge"}, {"budget": 0.53, "threshold": -13.8336181640625, "operator": "ge"}, {"budget": 0.54, "threshold": -13.89688491821289, "operator": "ge"}, {"budget": 0.55, "threshold": -13.94831657409668, "operator": "ge"}, {"budget": 0.56, "threshold": -14.02170181274414, "operator": "ge"}, {"budget": 0.57, "threshold": -14.113860130310059, "operator": "ge"}, {"budget": 0.58, "threshold": -14.195228576660156, "operator": "ge"}, {"budget": 0.59, "threshold": -14.260604858398438, "operator": "ge"}, {"budget": 0.6, "threshold": -14.314212799072266, "operator": "ge"}, {"budget": 0.61, "threshold": -14.353357315063477, "operator": "ge"}, {"budget": 0.62, "threshold": -14.441793441772461, "operator": "ge"}, {"budget": 0.63, "threshold": -14.482926368713379, "operator": "ge"}, {"budget": 0.64, "threshold": -14.55350112915039, "operator": "ge"}, {"budget": 0.65, "threshold": -14.620367050170898, "operator": "ge"}, {"budget": 0.66, "threshold": -14.671562194824219, "operator": "ge"}, {"budget": 0.67, "threshold": -14.762895584106445, "operator": "ge"}, {"budget": 0.68, "threshold": -14.821693420410156, "operator": "ge"}, {"budget": 0.69, "threshold": -14.884428024291992, "operator": "ge"}, {"budget": 0.7, "threshold": -14.932836532592773, "operator": "ge"}, {"budget": 0.71, "threshold": -14.977167129516602, "operator": "ge"}, {"budget": 0.72, "threshold": -15.017581939697266, "operator": "ge"}, {"budget": 0.73, "threshold": -15.067387580871582, "operator": "ge"}, {"budget": 0.74, "threshold": -15.112323760986328, "operator": "ge"}, {"budget": 0.75, "threshold": -15.160810470581055, "operator": "ge"}, {"budget": 0.76, "threshold": -15.209952354431152, "operator": "ge"}, {"budget": 0.77, "threshold": -15.259841918945312, "operator": "ge"}, {"budget": 0.78, "threshold": -15.318099021911621, "operator": "ge"}, {"budget": 0.79, "threshold": -15.386096954345703, "operator": "ge"}, {"budget": 0.8, "threshold": -15.435497283935547, "operator": "ge"}, {"budget": 0.81, "threshold": -15.511030197143555, "operator": "ge"}, {"budget": 0.82, "threshold": -15.589107513427734, "operator": "ge"}, {"budget": 0.83, "threshold": -15.65667724609375, "operator": "ge"}, {"budget": 0.84, "threshold": -15.712522506713867, "operator": "ge"}, {"budget": 0.85, "threshold": -15.794178009033203, "operator": "ge"}, {"budget": 0.86, "threshold": -15.859088897705078, "operator": "ge"}, {"budget": 0.87, "threshold": -15.894325256347656, "operator": "ge"}, {"budget": 0.88, "threshold": -15.955673217773438, "operator": "ge"}, {"budget": 0.89, "threshold": -16.00912857055664, "operator": "ge"}, {"budget": 0.9, "threshold": -16.06017303466797, "operator": "ge"}, {"budget": 0.91, "threshold": -16.14040756225586, "operator": "ge"}, {"budget": 0.92, "threshold": -16.24427032470703, "operator": "ge"}, {"budget": 0.93, "threshold": -16.301313400268555, "operator": "ge"}, {"budget": 0.94, "threshold": -16.368545532226562, "operator": "ge"}, {"budget": 0.95, "threshold": -16.471763610839844, "operator": "ge"}, {"budget": 0.96, "threshold": -16.609691619873047, "operator": "ge"}, {"budget": 0.97, "threshold": -16.856046676635742, "operator": "ge"}, {"budget": 0.98, "threshold": -17.087902069091797, "operator": "ge"}, {"budget": 0.99, "threshold": -17.281288146972656, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.66666666666667, 98.25, 97.5, 96.83333333333333, 96.16666666666667, 95.66666666666667, 95.08333333333333, 94.25, 93.58333333333333, 92.91666666666667, 92.16666666666666, 91.83333333333333, 91.33333333333333, 91.16666666666666, 90.41666666666667, 89.91666666666667, 89.75, 89.33333333333333, 89.0, 88.75, 88.5, 88.0, 87.83333333333333, 87.33333333333333, 87.0, 86.5, 86.08333333333333, 85.5, 85.08333333333333, 84.41666666666666, 83.66666666666667, 83.16666666666667, 82.66666666666667, 82.16666666666667, 81.66666666666667, 81.16666666666667, 80.58333333333333, 80.08333333333333, 79.5, 79.08333333333333, 78.58333333333333, 77.83333333333333, 77.25, 76.5, 76.0, 75.25, 74.5, 73.75, 72.91666666666666, 72.33333333333334, 71.75, 71.08333333333333, 70.33333333333334, 69.5, 68.91666666666667, 68.08333333333333, 67.41666666666666, 66.5, 65.75, 64.99999999999999, 64.16666666666666, 63.416666666666664, 62.5, 61.74999999999999, 60.916666666666664, 60.0, 59.25, 58.5, 57.58333333333333, 56.58333333333333, 55.666666666666664, 54.666666666666664, 53.666666666666664, 52.75, 51.74999999999999, 51.0, 49.99999999999999, 49.08333333333333, 48.083333333333336, 47.083333333333336, 46.166666666666664, 45.25, 44.166666666666664, 43.166666666666664, 42.25, 41.25, 40.33333333333333, 39.33333333333333, 38.33333333333333, 37.33333333333333, 36.33333333333333, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.666666666666664, 29.666666666666664, 28.666666666666664, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 4, 10, 20, 28, 36, 42, 49, 59, 67, 75, 84, 88, 94, 96, 105, 111, 113, 118, 122, 125, 128, 134, 136, 142, 146, 152, 157, 164, 169, 177, 186, 192, 198, 204, 210, 216, 223, 229, 236, 241, 247, 256, 263, 272, 278, 287, 296, 305, 315, 322, 329, 337, 346, 356, 363, 373, 381, 392, 401, 410, 420, 429, 440, 449, 459, 470, 479, 488, 499, 511, 522, 534, 546, 557, 569, 578, 590, 601, 613, 625, 636, 647, 660, 672, 683, 695, 706, 718, 730, 742, 754, 764, 776, 788, 800, 810, 822, 834, 846, 857], "exact_score": [98.91666666666666, 98.66666666666667, 98.25, 97.5, 96.83333333333333, 96.16666666666667, 95.66666666666667, 95.08333333333333, 94.25, 93.58333333333333, 92.91666666666667, 92.16666666666666, 91.83333333333333, 91.33333333333333, 91.16666666666666, 90.41666666666667, 89.91666666666667, 89.75, 89.33333333333333, 89.0, 88.75, 88.5, 88.0, 87.83333333333333, 87.33333333333333, 87.0, 86.5, 86.08333333333333, 85.5, 85.08333333333333, 84.41666666666666, 83.66666666666667, 83.16666666666667, 82.66666666666667, 82.16666666666667, 81.66666666666667, 81.16666666666667, 80.58333333333333, 80.08333333333333, 79.5, 79.08333333333333, 78.58333333333333, 77.83333333333333, 77.25, 76.5, 76.0, 75.25, 74.5, 73.75, 72.91666666666666, 72.33333333333334, 71.75, 71.08333333333333, 70.33333333333334, 69.5, 68.91666666666667, 68.08333333333333, 67.33333333333333, 66.5, 65.75, 64.99999999999999, 64.16666666666666, 63.416666666666664, 62.5, 61.74999999999999, 60.916666666666664, 60.0, 59.25, 58.5, 57.49999999999999, 56.58333333333333, 55.666666666666664, 54.666666666666664, 53.666666666666664, 52.75, 51.74999999999999, 51.0, 49.99999999999999, 49.08333333333333, 48.083333333333336, 47.083333333333336, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.166666666666664, 42.25, 41.25, 40.33333333333333, 39.33333333333333, 38.33333333333333, 37.33333333333333, 36.33333333333333, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.666666666666664, 29.666666666666664, 28.666666666666664, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 68.73916666666668, "threshold_area100": 68.74166666666667, "threshold_area25": 92.24166666666666}, "test": {"score": [90.16223679668623, 89.29927511218501, 87.95305488436314, 86.45150155333103, 85.19157749395927, 83.3620987228167, 80.96306523990334, 79.65136347946151, 77.80462547462892, 76.95892302381773, 75.62996202968588, 74.59440800828443, 73.97307559544356, 73.0755954435623, 72.71315153607179, 72.41974456334138, 72.17811529168104, 71.59130134622022, 70.98722816706939, 70.5730065585088, 70.26234035208836, 69.64100793924749, 69.24404556437693, 68.76078702105626, 68.2257507766655, 67.5698998964446, 67.19019675526407, 66.94856748360372, 66.68967897825337, 66.15464273386262, 65.84397652744218, 65.60234725578185, 65.32619951674145, 65.03279254401104, 64.77390403866067, 64.2216085605799, 63.80738695201933, 63.548498446668965, 63.2205730065585, 62.771832930617876, 62.2195374525371, 61.97790818087676, 61.63272350707628, 61.04590956161546, 60.54539178460476, 59.97583707283396, 59.04383845357266, 58.49154297549188, 57.73213669313082, 57.04176734552986, 56.55850880220918, 56.24784259578875, 55.95443562305833, 55.52295478080773, 54.90162236796685, 54.59095616154642, 53.98688298239558, 53.15843976527441, 52.519848118743525, 51.70866413531239, 51.242664825681736, 50.98377632033137, 50.08629616845012, 49.77562996202968, 49.17155678287884, 48.481187435277874, 47.980669658267175, 47.11770797376596, 46.3583016914049, 45.77148774594408, 45.3572661373835, 45.098377632033134, 44.64963755609251, 44.08008284432171, 43.614083534691055, 43.1480842250604, 42.69934414911978, 42.181567138419055, 41.422160856057985, 40.81808767690715, 40.33482913358647, 39.38557128063513, 38.522609596133925, 37.7114256127028, 37.12461166724197, 36.24439074905074, 35.5712806351398, 35.3641698308595, 34.708318950638585, 34.173282706247846, 33.690024162927166, 32.79254401104591, 32.18847083189505, 31.687953054884353, 31.23921297894373, 30.46254746289264, 29.634104245771486, 28.477735588539865, 27.75284777355885, 27.304107697618218, 26.56196064894718], "calls": [0.0, 3.727994477045219, 7.83569209527097, 12.236796686227132, 14.981014842940974, 18.32930617880566, 22.26441146013117, 24.525371073524337, 27.252329996548152, 28.736624093890235, 30.911287538833278, 32.61995167414567, 33.690024162927166, 35.34691059716948, 36.12357611322057, 36.90024162927166, 37.31446323783224, 38.76423886779427, 40.196755264066276, 41.16327235070763, 41.81912323092855, 43.64860200207111, 44.4943044528823, 45.47808077321367, 46.73800483258543, 48.01518812564722, 48.79185364169831, 49.37866758715913, 50.06903693476009, 50.93199861926131, 51.518812564722126, 51.96755264066275, 52.67518122195375, 53.29651363479462, 53.71073524335519, 54.79806696582672, 55.695547117707974, 56.178805661028655, 56.73110113910942, 57.71487745944081, 58.577839143942015, 59.00931998619261, 59.63065239903348, 60.40731791508457, 61.06316879530549, 61.926130479806694, 63.324128408698655, 63.962720055229546, 64.98101484294098, 65.8612357611322, 66.4998274076631, 66.98308595098378, 67.3627890921643, 67.84604763548498, 68.51915774939593, 69.0024162927166, 69.7790818087677, 70.83189506385916, 71.59130134622022, 72.54055919917157, 73.0755954435623, 73.35174318260269, 74.38729720400414, 74.73248187780463, 75.4401104590956, 76.37210907835691, 76.97618225750776, 77.90818087676908, 78.7193648602002, 79.4269934414912, 79.9275112185019, 80.23817742492233, 80.721435968243, 81.30824991370383, 81.82602692440456, 82.34380393510529, 82.82706247842596, 83.44839489126683, 84.32861580945806, 84.94994822229893, 85.48498446668968, 86.5032792544011, 87.43527787366241, 88.3500172592337, 89.00586813945462, 89.93786675871591, 90.62823610631689, 90.8526061442872, 91.54297549188816, 92.09527096996894, 92.57852951328961, 93.52778736624094, 94.16637901277183, 94.68415602347255, 95.16741456679323, 95.97859855022438, 96.80704176734552, 97.9979288919572, 98.75733517431826, 99.22333448394892, 100.0], "rescued": [0, 16, 35, 62, 70, 76, 81, 90, 92, 94, 95, 95, 95, 95, 97, 97, 97, 98, 99, 99, 99, 100, 100, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106], "harmed": [0, 66, 163, 277, 358, 470, 614, 699, 808, 859, 937, 997, 1033, 1085, 1108, 1125, 1139, 1174, 1210, 1234, 1252, 1289, 1312, 1341, 1372, 1411, 1433, 1447, 1462, 1493, 1511, 1525, 1541, 1558, 1573, 1605, 1629, 1644, 1663, 1689, 1721, 1735, 1756, 1790, 1819, 1852, 1906, 1938, 1982, 2022, 2050, 2068, 2085, 2110, 2146, 2164, 2199, 2247, 2284, 2331, 2358, 2373, 2425, 2443, 2478, 2518, 2548, 2598, 2642, 2676, 2700, 2715, 2741, 2774, 2801, 2828, 2854, 2884, 2929, 2964, 2992, 3047, 3097, 3144, 3178, 3229, 3268, 3280, 3318, 3349, 3377, 3430, 3465, 3494, 3520, 3565, 3613, 3680, 3722, 3748, 3791], "exact_score": [90.16223679668623, 89.98964445978598, 89.76527442181566, 89.59268208491544, 89.24749741111494, 88.90231273731446, 88.55712806351399, 88.28098032447359, 87.90127718329306, 87.62512944425266, 87.34898170521228, 86.91750086296169, 86.52053848809112, 86.05453917846047, 85.64031756989989, 85.19157749395927, 84.62202278218847, 84.15602347255782, 83.55195029340697, 82.94787711425612, 82.30928546772522, 81.77424922333448, 81.18743527787366, 80.66965826717293, 80.01380738695202, 79.3579565067311, 78.63306869175008, 77.94269934414912, 77.3558853986883, 76.76907145322747, 76.06144287193648, 75.5781843286158, 74.90507421470487, 74.33551950293406, 73.81774249223334, 73.29996548153261, 72.76492923714186, 72.35070762858129, 71.97100448740076, 71.5050051777701, 71.12530203658957, 70.69382119433897, 70.24508111839833, 69.89989644459786, 69.46841560234725, 68.98515705902656, 68.46738004832585, 68.08767690714532, 67.5698998964446, 67.15567828788402, 66.7069382119434, 66.12012426648258, 65.58508802209182, 65.17086641353124, 64.61857093545046, 64.10079392474974, 63.66931308249914, 63.15153607179841, 62.58198136002762, 61.99516741456679, 61.339316534345876, 60.61442871936485, 59.92405937176388, 59.250949257852945, 58.45702450811183, 57.71487745944079, 56.92095270969968, 56.24784259578875, 55.40214014497755, 54.59095616154642, 53.79703141180532, 53.003106662064205, 52.19192267863306, 51.328960994131855, 50.4314808422506, 49.534000690369346, 48.72281670693821, 47.96341042457715, 47.04867103900586, 46.11667241974456, 45.2882292026234, 44.390749050742144, 43.44149119779082, 42.544011045909556, 41.73282706247842, 40.783569209527094, 39.86882982395581, 38.93683120469451, 38.022091819123226, 37.12461166724197, 36.19261304798067, 35.22609596133931, 34.276838108387985, 33.31032102174664, 32.3610631687953, 31.39454608215395, 30.445288229202617, 29.444252675181215, 28.477735588539865, 27.511218501898515, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 63.90028565610527, "threshold_area100": 55.978339661719026, "threshold_area25": 76.10804280289956}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.8644430637359619, "operator": "ge"}, {"budget": 0.02, "threshold": -1.665302038192749, "operator": "ge"}, {"budget": 0.03, "threshold": -2.3325510025024414, "operator": "ge"}, {"budget": 0.04, "threshold": -2.5706610679626465, "operator": "ge"}, {"budget": 0.05, "threshold": -3.2139029502868652, "operator": "ge"}, {"budget": 0.06, "threshold": -3.761147975921631, "operator": "ge"}, {"budget": 0.07, "threshold": -4.082846164703369, "operator": "ge"}, {"budget": 0.08, "threshold": -4.687243938446045, "operator": "ge"}, {"budget": 0.09, "threshold": -5.179444313049316, "operator": "ge"}, {"budget": 0.1, "threshold": -5.4902873039245605, "operator": "ge"}, {"budget": 0.11, "threshold": -5.756847381591797, "operator": "ge"}, {"budget": 0.12, "threshold": -5.935919284820557, "operator": "ge"}, {"budget": 0.13, "threshold": -6.0660400390625, "operator": "ge"}, {"budget": 0.14, "threshold": -6.307485103607178, "operator": "ge"}, {"budget": 0.15, "threshold": -6.581851005554199, "operator": "ge"}, {"budget": 0.16, "threshold": -6.691441535949707, "operator": "ge"}, {"budget": 0.17, "threshold": -6.9854326248168945, "operator": "ge"}, {"budget": 0.18, "threshold": -7.121975898742676, "operator": "ge"}, {"budget": 0.19, "threshold": -7.394646644592285, "operator": "ge"}, {"budget": 0.2, "threshold": -7.667464256286621, "operator": "ge"}, {"budget": 0.21, "threshold": -7.802397727966309, "operator": "ge"}, {"budget": 0.22, "threshold": -7.995797157287598, "operator": "ge"}, {"budget": 0.23, "threshold": -8.191186904907227, "operator": "ge"}, {"budget": 0.24, "threshold": -8.426929473876953, "operator": "ge"}, {"budget": 0.25, "threshold": -8.658239364624023, "operator": "ge"}, {"budget": 0.26, "threshold": -8.83372688293457, "operator": "ge"}, {"budget": 0.27, "threshold": -8.950876235961914, "operator": "ge"}, {"budget": 0.28, "threshold": -9.01034927368164, "operator": "ge"}, {"budget": 0.29, "threshold": -9.105178833007812, "operator": "ge"}, {"budget": 0.3, "threshold": -9.260519981384277, "operator": "ge"}, {"budget": 0.31, "threshold": -9.45763874053955, "operator": "ge"}, {"budget": 0.32, "threshold": -9.554993629455566, "operator": "ge"}, {"budget": 0.33, "threshold": -9.67451286315918, "operator": "ge"}, {"budget": 0.34, "threshold": -9.80908203125, "operator": "ge"}, {"budget": 0.35, "threshold": -9.992188453674316, "operator": "ge"}, {"budget": 0.36, "threshold": -10.101238250732422, "operator": "ge"}, {"budget": 0.37, "threshold": -10.244037628173828, "operator": "ge"}, {"budget": 0.38, "threshold": -10.375652313232422, "operator": "ge"}, {"budget": 0.39, "threshold": -10.527076721191406, "operator": "ge"}, {"budget": 0.4, "threshold": -10.72991943359375, "operator": "ge"}, {"budget": 0.41, "threshold": -10.822267532348633, "operator": "ge"}, {"budget": 0.42, "threshold": -10.95367431640625, "operator": "ge"}, {"budget": 0.43, "threshold": -11.033767700195312, "operator": "ge"}, {"budget": 0.44, "threshold": -11.142394065856934, "operator": "ge"}, {"budget": 0.45, "threshold": -11.206069946289062, "operator": "ge"}, {"budget": 0.46, "threshold": -11.287501335144043, "operator": "ge"}, {"budget": 0.47, "threshold": -11.366386413574219, "operator": "ge"}, {"budget": 0.48, "threshold": -11.449825286865234, "operator": "ge"}, {"budget": 0.49, "threshold": -11.536567687988281, "operator": "ge"}, {"budget": 0.5, "threshold": -11.602880477905273, "operator": "ge"}, {"budget": 0.51, "threshold": -11.677043914794922, "operator": "ge"}, {"budget": 0.52, "threshold": -11.714349746704102, "operator": "ge"}, {"budget": 0.53, "threshold": -11.806986808776855, "operator": "ge"}, {"budget": 0.54, "threshold": -11.897613525390625, "operator": "ge"}, {"budget": 0.55, "threshold": -11.963288307189941, "operator": "ge"}, {"budget": 0.56, "threshold": -12.036121368408203, "operator": "ge"}, {"budget": 0.57, "threshold": -12.071749687194824, "operator": "ge"}, {"budget": 0.58, "threshold": -12.146543502807617, "operator": "ge"}, {"budget": 0.59, "threshold": -12.226700782775879, "operator": "ge"}, {"budget": 0.6, "threshold": -12.290882110595703, "operator": "ge"}, {"budget": 0.61, "threshold": -12.334800720214844, "operator": "ge"}, {"budget": 0.62, "threshold": -12.406729698181152, "operator": "ge"}, {"budget": 0.63, "threshold": -12.449054718017578, "operator": "ge"}, {"budget": 0.64, "threshold": -12.5208740234375, "operator": "ge"}, {"budget": 0.65, "threshold": -12.602163314819336, "operator": "ge"}, {"budget": 0.66, "threshold": -12.660693168640137, "operator": "ge"}, {"budget": 0.67, "threshold": -12.724346160888672, "operator": "ge"}, {"budget": 0.68, "threshold": -12.791887283325195, "operator": "ge"}, {"budget": 0.69, "threshold": -12.835044860839844, "operator": "ge"}, {"budget": 0.7, "threshold": -12.8699951171875, "operator": "ge"}, {"budget": 0.71, "threshold": -12.942532539367676, "operator": "ge"}, {"budget": 0.72, "threshold": -13.028615951538086, "operator": "ge"}, {"budget": 0.73, "threshold": -13.102958679199219, "operator": "ge"}, {"budget": 0.74, "threshold": -13.142379760742188, "operator": "ge"}, {"budget": 0.75, "threshold": -13.233627319335938, "operator": "ge"}, {"budget": 0.76, "threshold": -13.291032791137695, "operator": "ge"}, {"budget": 0.77, "threshold": -13.370016098022461, "operator": "ge"}, {"budget": 0.78, "threshold": -13.418105125427246, "operator": "ge"}, {"budget": 0.79, "threshold": -13.454270362854004, "operator": "ge"}, {"budget": 0.8, "threshold": -13.481744766235352, "operator": "ge"}, {"budget": 0.81, "threshold": -13.539335250854492, "operator": "ge"}, {"budget": 0.82, "threshold": -13.577001571655273, "operator": "ge"}, {"budget": 0.83, "threshold": -13.639177322387695, "operator": "ge"}, {"budget": 0.84, "threshold": -13.683879852294922, "operator": "ge"}, {"budget": 0.85, "threshold": -13.733757019042969, "operator": "ge"}, {"budget": 0.86, "threshold": -13.770645141601562, "operator": "ge"}, {"budget": 0.87, "threshold": -13.83245849609375, "operator": "ge"}, {"budget": 0.88, "threshold": -13.863391876220703, "operator": "ge"}, {"budget": 0.89, "threshold": -13.954004287719727, "operator": "ge"}, {"budget": 0.9, "threshold": -14.0649995803833, "operator": "ge"}, {"budget": 0.91, "threshold": -14.171740531921387, "operator": "ge"}, {"budget": 0.92, "threshold": -14.25703239440918, "operator": "ge"}, {"budget": 0.93, "threshold": -14.354146957397461, "operator": "ge"}, {"budget": 0.94, "threshold": -14.467082977294922, "operator": "ge"}, {"budget": 0.95, "threshold": -14.607792854309082, "operator": "ge"}, {"budget": 0.96, "threshold": -14.784038543701172, "operator": "ge"}, {"budget": 0.97, "threshold": -14.916997909545898, "operator": "ge"}, {"budget": 0.98, "threshold": -15.032594680786133, "operator": "ge"}, {"budget": 0.99, "threshold": -15.361166000366211, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.75, 98.33333333333333, 97.74999999999999, 97.08333333333333, 96.75, 95.91666666666666, 95.33333333333333, 94.91666666666666, 94.66666666666667, 94.25, 93.91666666666666, 93.5, 92.83333333333333, 92.58333333333333, 92.33333333333333, 91.91666666666667, 91.66666666666666, 91.25, 90.91666666666667, 90.33333333333333, 89.83333333333333, 89.41666666666667, 89.0, 88.5, 87.91666666666667, 87.41666666666667, 86.66666666666667, 86.25, 85.66666666666667, 85.08333333333333, 84.58333333333333, 84.08333333333333, 83.58333333333333, 83.08333333333333, 82.75, 82.5, 81.91666666666666, 81.25, 80.5, 79.83333333333333, 79.25, 78.58333333333333, 78.08333333333333, 77.33333333333333, 76.66666666666666, 75.75, 75.08333333333333, 74.5, 73.91666666666666, 73.16666666666667, 72.58333333333333, 71.83333333333333, 70.91666666666666, 70.24999999999999, 69.33333333333334, 68.41666666666666, 67.66666666666666, 66.75, 66.16666666666666, 65.5, 64.66666666666666, 63.66666666666667, 62.74999999999999, 61.83333333333333, 60.83333333333333, 59.916666666666664, 59.0, 58.08333333333333, 57.41666666666667, 56.49999999999999, 55.49999999999999, 54.49999999999999, 53.583333333333336, 52.583333333333336, 51.74999999999999, 50.83333333333333, 49.916666666666664, 48.916666666666664, 47.916666666666664, 46.916666666666664, 45.916666666666664, 44.99999999999999, 43.91666666666666, 42.99999999999999, 42.08333333333333, 41.08333333333333, 40.166666666666664, 39.16666666666666, 38.24999999999999, 37.33333333333333, 36.416666666666664, 35.416666666666664, 34.416666666666664, 33.416666666666664, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 3, 8, 16, 24, 29, 39, 46, 51, 54, 59, 63, 68, 76, 79, 82, 87, 90, 95, 99, 106, 112, 117, 122, 128, 135, 141, 150, 155, 162, 169, 175, 181, 187, 193, 197, 200, 207, 215, 224, 232, 239, 247, 253, 262, 270, 281, 289, 296, 303, 312, 319, 328, 339, 347, 358, 369, 378, 389, 396, 404, 414, 426, 437, 448, 460, 471, 482, 493, 501, 512, 524, 536, 547, 559, 569, 580, 591, 603, 615, 627, 639, 650, 663, 674, 685, 697, 708, 720, 731, 742, 753, 765, 777, 789, 800, 811, 823, 834, 845, 857], "exact_score": [98.91666666666666, 98.75, 98.33333333333333, 97.74999999999999, 97.08333333333333, 96.75, 95.91666666666666, 95.33333333333333, 94.91666666666666, 94.66666666666667, 94.25, 93.91666666666666, 93.5, 92.83333333333333, 92.58333333333333, 92.33333333333333, 91.91666666666667, 91.66666666666666, 91.25, 90.91666666666667, 90.33333333333333, 89.83333333333333, 89.41666666666667, 89.0, 88.5, 87.91666666666667, 87.41666666666667, 86.66666666666667, 86.25, 85.66666666666667, 85.08333333333333, 84.58333333333333, 84.08333333333333, 83.58333333333333, 83.08333333333333, 82.75, 82.5, 81.91666666666666, 81.25, 80.5, 79.83333333333333, 79.16666666666666, 78.58333333333333, 78.08333333333333, 77.33333333333333, 76.66666666666666, 75.75, 75.08333333333333, 74.5, 73.91666666666666, 73.16666666666667, 72.58333333333333, 71.83333333333333, 70.91666666666666, 70.24999999999999, 69.33333333333334, 68.41666666666666, 67.58333333333333, 66.75, 66.16666666666666, 65.5, 64.66666666666666, 63.66666666666667, 62.74999999999999, 61.83333333333333, 60.83333333333333, 59.916666666666664, 59.0, 58.08333333333333, 57.33333333333333, 56.49999999999999, 55.49999999999999, 54.49999999999999, 53.583333333333336, 52.583333333333336, 51.74999999999999, 50.83333333333333, 49.916666666666664, 48.916666666666664, 47.916666666666664, 46.916666666666664, 45.916666666666664, 44.91666666666666, 43.91666666666666, 42.99999999999999, 42.08333333333333, 41.08333333333333, 40.166666666666664, 39.16666666666666, 38.24999999999999, 37.33333333333333, 36.416666666666664, 35.416666666666664, 34.416666666666664, 33.416666666666664, 32.49999999999999, 31.58333333333333, 30.58333333333333, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 69.25333333333334, "threshold_area100": 69.25666666666666, "threshold_area25": 93.41666666666667}, "test": {"score": [90.16223679668623, 89.38557128063513, 88.28098032447359, 86.86572316189161, 86.0027614773904, 83.8626164998274, 81.89506385916465, 80.8595098377632, 79.13358646876078, 77.92544011045909, 77.01070072488781, 76.35484984466689, 75.69899896444598, 75.42285122540558, 74.90507421470487, 74.43907490507421, 74.14566793234381, 73.33448394891266, 73.05833620987228, 72.33344839489126, 71.66033828098031, 71.29789437348981, 70.84915429754919, 70.17604418363824, 69.65826717293751, 68.9333793579565, 68.4328615809458, 68.0704176734553, 67.94960303762512, 67.63893683120469, 67.05212288574387, 66.12012426648258, 65.8612357611322, 65.37797721781152, 64.82568173973075, 64.2216085605799, 63.979979288919566, 63.634794615119084, 63.324128408698655, 62.59924059371764, 61.73627890921642, 61.39109423541594, 60.683465654124944, 60.234725578184324, 59.44080082844321, 59.04383845357266, 58.802209181912325, 58.301691404901625, 57.76665516051087, 57.197100448740066, 56.71384190541939, 56.127027959958575, 55.88539868829824, 55.229547808077314, 54.64273386261649, 54.159475319295815, 53.53814290645496, 53.24473593372454, 52.62340352088367, 51.88125647221262, 51.328960994131855, 50.98377632033137, 50.39696237487055, 49.930963065239894, 48.929927511218494, 48.11874352778736, 47.61822575077666, 47.014152571625814, 46.39282015878495, 45.96133931653434, 45.61615464273386, 44.615119088712454, 43.59682430100103, 42.95823265447014, 42.613047980669656, 41.62927166033828, 40.74905074214705, 40.006903693476005, 39.67897825336555, 39.24749741111494, 39.005868139454606, 38.384535726613734, 37.86675871591301, 37.22816706938211, 36.72764929237141, 36.08905764584052, 35.709354504659984, 35.13979979288919, 34.82913358646875, 34.01794960303762, 33.06869175008629, 32.13669313082499, 31.722471522264406, 31.11839834311356, 30.46254746289264, 29.80669658267172, 29.133586468760786, 28.598550224370033, 28.02899551259924, 27.286848463928195, 26.56196064894718], "calls": [0.0, 4.280289955125992, 7.956506731101139, 11.891612012426648, 13.617535381429063, 18.0704176734553, 21.59130134622023, 23.593372454263033, 26.596479116327238, 29.06454953400069, 30.9285467725233, 32.430100103555404, 33.44839489126683, 34.03520883672765, 35.01898515705902, 36.468760787021054, 36.98653779772178, 38.729720400414216, 39.55816361753538, 41.266827752847774, 42.682084915429755, 43.33793579565067, 44.166379012771834, 45.23645150155333, 46.44459785985502, 47.67000345184674, 48.63652053848809, 49.257852951328964, 49.4994822229893, 50.06903693476009, 51.08733172247152, 52.55436658612358, 53.02036589575423, 53.71073524335519, 54.694511563686575, 55.695547117707974, 56.19606489471868, 56.851915774939584, 57.317915084570245, 58.40524680704176, 59.69968933379358, 60.200207110804286, 61.149464963755605, 61.7880566102865, 62.73731446323784, 63.22057300655851, 63.600276147739045, 64.29064549534, 64.98101484294098, 65.67138419054194, 66.39627200552295, 67.19019675526407, 67.5008629616845, 68.24301001035555, 69.01967552640663, 69.76182257507767, 70.52122885743873, 70.88367276492924, 71.57404211253021, 72.40248532965137, 73.11011391094236, 73.48981705212289, 74.16292716603382, 74.69796341042458, 75.76803589920608, 76.70003451846738, 77.23507076285813, 77.95995857783915, 78.70210562651019, 79.15084570245081, 79.53054884363134, 80.61788056610285, 81.7224715222644, 82.51639627200552, 82.96513634794616, 84.03520883672765, 84.9326889886089, 85.76113220573006, 86.10631687953055, 86.55505695547117, 86.79668622713152, 87.45253710735244, 87.98757335174318, 88.69520193303417, 89.26475664480496, 89.97238522609597, 90.40386606834656, 90.99068001380739, 91.3531239212979, 92.18156713841906, 93.18260269244045, 94.13186054539179, 94.59785985502243, 95.23645150155333, 95.89230238177426, 96.5826717293752, 97.34207801173628, 97.87711425612703, 98.46392820158785, 99.27511218501898, 100.0], "rescued": [0, 12, 27, 42, 47, 63, 75, 80, 85, 89, 92, 92, 92, 93, 93, 96, 96, 97, 97, 97, 97, 97, 97, 97, 97, 98, 99, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106], "harmed": [0, 57, 136, 233, 288, 428, 554, 619, 724, 798, 854, 892, 930, 947, 977, 1007, 1024, 1072, 1088, 1130, 1169, 1190, 1216, 1255, 1285, 1328, 1358, 1380, 1387, 1406, 1440, 1494, 1509, 1537, 1569, 1604, 1618, 1639, 1657, 1699, 1749, 1769, 1810, 1836, 1882, 1905, 1919, 1948, 1979, 2012, 2040, 2074, 2088, 2126, 2160, 2188, 2224, 2241, 2277, 2320, 2352, 2372, 2407, 2435, 2493, 2540, 2569, 2605, 2641, 2666, 2686, 2744, 2803, 2840, 2860, 2917, 2968, 3011, 3030, 3055, 3069, 3105, 3135, 3172, 3201, 3238, 3260, 3293, 3311, 3358, 3413, 3467, 3491, 3526, 3564, 3602, 3641, 3672, 3705, 3749, 3791], "exact_score": [90.16223679668623, 90.16223679668623, 90.05868139454608, 89.81705212288574, 89.42008974801519, 89.14394200897479, 88.93683120469451, 88.60890576458405, 88.26372109078356, 87.86675871591301, 87.5733517431826, 87.19364860200207, 86.84846392820158, 86.27890921643079, 85.81290990680014, 85.46772523299964, 84.91542975491888, 84.36313427683811, 83.89713496720745, 83.3620987228167, 82.94787711425612, 82.3610631687953, 81.65343458750431, 81.11839834311357, 80.63513979979288, 80.11736278909216, 79.59958577839143, 78.89195719710045, 78.42595788746979, 77.97721781152916, 77.47670003451846, 76.95892302381773, 76.54470141525715, 75.97514670348636, 75.42285122540558, 74.92233344839488, 74.61166724197446, 74.14566793234381, 73.71418709009319, 73.23092854677252, 72.86848463928202, 72.45426303072144, 71.98826372109079, 71.48774594408009, 70.91819123230928, 70.33137728684846, 69.86537797721782, 69.29582326544701, 68.79530548843631, 68.2084915429755, 67.67345529858474, 67.05212288574387, 66.49982740766309, 65.89575422851225, 65.22264411460131, 64.65308940283052, 64.0662754573697, 63.56575768035899, 62.80635139799793, 62.133241284086985, 61.47739040386606, 60.80428028995512, 60.04487400759406, 59.2164307904729, 58.526061442871935, 57.749395926820846, 56.95547117707973, 56.265101829478766, 55.436658612357604, 54.659993096306515, 54.004142216085604, 53.14118053158439, 52.27821884708319, 51.432516396272, 50.517777010700726, 49.67207455988954, 48.75733517431826, 47.859855022436996, 46.99689333793579, 46.11667241974456, 45.20193303417328, 44.287193648602, 43.35519502934069, 42.57852951328961, 41.68104936140835, 40.69727304107697, 39.799792889195714, 38.81601656886434, 37.86675871591301, 36.96927856403175, 36.0717984121505, 35.13979979288919, 34.207801173627885, 33.24128408698653, 32.274767000345186, 31.360027614773898, 30.3589920607525, 29.426993441491188, 28.49499482222989, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 64.39615201899672, "threshold_area100": 56.13332758025543, "threshold_area25": 77.30410769761822}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.1449775629962, "calls": 1.052813255091474}}, "old_normalized_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.1449775629962, "calls": 1.6741456679323439}}, "old_normalized_endpoints": {"val": {"score": 98.91666666666666, "calls": 0.0}, "test": {"score": 90.16223679668623, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.66666666666667, 98.25, 98.25, 98.16666666666667, 98.0, 97.74999999999999, 97.41666666666666, 97.24999999999999, 97.24999999999999, 97.0, 96.91666666666666, 96.75, 96.5, 96.0, 95.58333333333333, 94.91666666666666, 94.5, 93.91666666666666, 93.5, 92.75, 92.0, 91.41666666666667, 90.58333333333333, 89.83333333333333, 89.08333333333334, 88.33333333333333, 87.41666666666667, 86.58333333333333, 85.75, 84.91666666666666, 84.25, 83.41666666666666, 82.66666666666667, 81.75, 80.83333333333333, 79.83333333333333, 78.91666666666666, 78.16666666666666, 77.33333333333333, 76.5, 75.58333333333334, 74.58333333333333, 73.75, 72.83333333333333, 71.91666666666666, 71.16666666666667, 70.16666666666667, 69.24999999999999, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.25, 63.24999999999999, 62.33333333333333, 61.5, 60.5, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.833333333333336, 55.833333333333336, 55.00000000000001, 54.0, 53.0, 52.0, 51.0, 49.99999999999999, 49.0, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.5, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.66666666666667, 98.25, 98.25, 98.16666666666667, 98.0, 97.74999999999999, 97.41666666666666, 97.24999999999999, 97.24999999999999, 97.0, 96.91666666666666, 96.75, 96.5, 96.0, 95.58333333333333, 94.91666666666666, 94.5, 93.91666666666666, 93.5, 92.75, 92.0, 91.41666666666667, 90.58333333333333, 89.83333333333333, 89.08333333333334, 88.33333333333333, 87.41666666666667, 86.58333333333333, 85.75, 84.91666666666666, 84.25, 83.33333333333333, 82.66666666666667, 81.75, 80.83333333333333, 79.83333333333333, 78.91666666666666, 78.16666666666666, 77.33333333333333, 76.5, 75.58333333333334, 74.58333333333333, 73.75, 72.83333333333333, 71.91666666666666, 71.16666666666667, 70.16666666666667, 69.16666666666667, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.25, 63.24999999999999, 62.33333333333333, 61.5, 60.5, 59.74999999999999, 58.833333333333336, 57.833333333333336, 56.833333333333336, 55.833333333333336, 55.00000000000001, 54.0, 53.0, 52.0, 51.0, 49.99999999999999, 49.0, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.5, 38.5, 37.5, 36.5, 35.5, 34.5, 33.5, 32.49999999999999, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.83333333333333, "exact_area100": 71.905}, "test": {"score": [90.16223679668623, 90.1449775629962, 90.12771832930618, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.07594062823611, 90.07594062823611, 89.90334829133586, 89.76527442181566, 89.7134967207456, 89.52364515015533, 89.47186744908525, 89.24749741111494, 89.05764584052469, 88.97134967207455, 88.53986882982396, 88.17742492233344, 88.0048325854332, 87.76320331377286, 87.52157404211253, 87.0728339661719, 86.67587159130134, 86.31342768381084, 85.41594753192958, 84.63928201587849, 84.05246807041767, 83.18950638591647, 82.70624784259579, 82.22298929927511, 81.8605453917846, 81.48084225060407, 80.89402830514325, 79.97928891957197, 78.42595788746979, 77.2868484639282, 76.54470141525715, 75.68173973075595, 74.92233344839488, 74.18018639972385, 73.5070762858129, 72.33344839489126, 70.98722816706939, 70.2105626510183, 69.38211943389715, 68.50189851570589, 67.75975146703486, 67.17293752157404, 66.53434587504314, 65.46427338626164, 64.48049706593027, 63.85916465308941, 63.099758370728345, 62.09872281670694, 61.13220573006558, 60.510873317224714, 59.69968933379357, 58.66413531239212, 57.87021056265102, 57.38695201933034, 56.61028650327925, 55.79910251984812, 55.1432516396272, 54.81532619951675, 54.004142216085604, 52.96858819468415, 52.226441146013116, 51.08733172247152, 50.77666551605109, 50.05177770107008, 49.03348291335865, 48.44666896789782, 47.37659647911632, 46.738004832585425, 46.237487055574725, 45.581636175353815, 45.06385916465309, 44.114601311701755, 43.66586123576113, 42.78564031756989, 42.07801173627891, 41.197790818087675, 40.38660683465653, 39.81705212288574, 39.43734898170521, 38.885053503624434, 37.84949948222298, 37.33172247152227, 36.5723161891612, 35.79565067311011, 35.10528132550914, 34.24231964100793, 32.96513634794614, 31.912323092854677, 31.101139109423535, 30.04832585433207, 28.96099413186054, 28.253365550569555, 27.33862616499827, 26.56196064894718], "calls": [0.0, 1.6741456679323439, 2.7096996893337937, 3.5726613738350017, 4.6427338626165, 5.574732481877804, 5.816361753538143, 7.404211253020366, 8.301691404901621, 10.700724887814982, 11.598205039696238, 12.961684501208145, 13.68657231618916, 14.169830859509839, 15.412495685191576, 16.896789782533656, 17.587159130134623, 18.86434242319641, 20.365895754228514, 21.17707973765965, 22.074559889540904, 23.403520883672766, 24.663444943044528, 25.62996202968588, 26.458405246807043, 28.339661719019677, 29.84121505005178, 31.083879875733516, 32.61995167414567, 33.53469105971695, 34.4321712115982, 35.22609596133932, 35.86468760787021, 36.969278564031754, 38.47083189506386, 40.88712461166724, 42.49223334483949, 43.49326889886089, 44.68415602347256, 45.857783914394204, 46.80704176734553, 47.75629962029686, 49.22333448394891, 50.84570245081118, 51.794960303762515, 52.70969968933379, 53.693476009665176, 54.71177079737659, 55.52295478080773, 56.24784259578875, 57.55954435623059, 58.76769071453227, 59.527096996893334, 60.39005868139454, 61.649982740766305, 62.771832930617876, 63.47946151190887, 64.34242319641008, 65.46427338626165, 66.41353123921297, 66.98308595098378, 67.89782533655506, 68.74352778736625, 69.46841560234725, 69.88263721090784, 70.79737659647911, 72.04004142216085, 72.83396617190196, 74.0766309975837, 74.45633413876423, 75.23299965481533, 76.30307214359682, 76.9244045564377, 78.06351397997929, 78.70210562651019, 79.23714187090093, 79.89299275112185, 80.46254746289266, 81.51536071798412, 82.01587849499482, 82.89609941318605, 83.75906109768727, 84.65654124956852, 85.51950293406972, 86.10631687953055, 86.5032792544011, 87.0728339661719, 88.22920262340352, 88.76423886779426, 89.55816361753539, 90.38660683465655, 91.11149464963756, 91.99171556782879, 93.33793579565068, 94.39074905074214, 95.25371073524336, 96.427338626165, 97.54918881601657, 98.27407663099758, 99.22333448394892, 100.0], "exact_score": [90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.1449775629962, 90.12771832930618, 90.1449775629962, 90.11045909561615, 90.07594062823611, 90.02416292716603, 89.97238522609597, 89.8688298239558, 89.76527442181566, 89.69623748705557, 89.48912668277528, 89.29927511218501, 89.23023817742491, 89.05764584052469, 88.85053503624438, 88.47083189506385, 88.24646185709354, 88.03935105281326, 87.81498101484294, 87.5733517431826, 87.33172247152227, 86.93476009665171, 86.48602002071108, 86.03727994477045, 85.58853986882981, 85.08802209181911, 84.55298584742837, 84.06972730410769, 83.55195029340697, 82.94787711425612, 82.48187780462547, 81.98136002761477, 81.41180531584398, 80.89402830514325, 80.2899551259924, 79.70314118053157, 79.04729030031066, 78.32240248532965, 77.64929237141871, 76.90714532274767, 76.14773904038661, 75.47462892647566, 74.81877804625474, 74.04211253020365, 73.28270624784258, 72.54055919917155, 71.67759751467034, 70.93545046599931, 70.03797031411804, 69.12323092854677, 68.29478771142561, 67.58715913013462, 66.77597514670349, 65.96479116327234, 65.10182947877115, 64.29064549534, 63.46220227821884, 62.633759061097685, 61.839834311356576, 60.90783569209527, 60.01035554021401, 59.13013462202278, 58.249913703831545, 57.38695201933034, 56.541249568519156, 55.59199171556782, 54.71177079737659, 53.86606834656541, 53.02036589575423, 52.08836727649292, 51.17362789092164, 50.293406972730416, 49.34414911977908, 48.39489126682775, 47.44563341387642, 46.47911632723506, 45.47808077321366, 44.58060062133241, 43.683120469451154, 42.73386261649982, 41.85364169830859, 40.86986537797721, 39.93786675871591, 38.95409043838454, 38.073869520193306, 37.14187090093199, 36.175353814290645, 35.22609596133931, 34.24231964100793, 33.31032102174664, 32.30928546772523, 31.360027614773898, 30.410769761822575, 29.478771142561268, 28.52951328960994, 27.545736969278558, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 88.90403866068347, "exact_area100": 67.26107310611039}}, "old_normalized_endpoint_budget": 0}, {"seed": 2029, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.5, "validation_area100": 0.7197083333333333, "training_seconds": 3.370307442964986, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.053622053345030285, "operator": "ge"}, {"budget": 0.02, "threshold": -0.05748013485338335, "operator": "ge"}, {"budget": 0.03, "threshold": -0.06112885357224522, "operator": "ge"}, {"budget": 0.04, "threshold": -0.0676623006436499, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0771730866098073, "operator": "ge"}, {"budget": 0.06, "threshold": -0.08444767735085924, "operator": "ge"}, {"budget": 0.07, "threshold": -0.0958326811282785, "operator": "ge"}, {"budget": 0.08, "threshold": -0.12932717096413945, "operator": "ge"}, {"budget": 0.09, "threshold": -0.1458250113481958, "operator": "ge"}, {"budget": 0.1, "threshold": -0.16675803464500177, "operator": "ge"}, {"budget": 0.11, "threshold": -0.19875742665107105, "operator": "ge"}, {"budget": 0.12, "threshold": -0.22796668471056333, "operator": "ge"}, {"budget": 0.13, "threshold": -0.24987234305674547, "operator": "ge"}, {"budget": 0.14, "threshold": -0.28086433111032194, "operator": "ge"}, {"budget": 0.15, "threshold": -0.30252440376902623, "operator": "ge"}, {"budget": 0.16, "threshold": -0.32814546478116974, "operator": "ge"}, {"budget": 0.17, "threshold": -0.3767107721681109, "operator": "ge"}, {"budget": 0.18, "threshold": -0.3960035431852903, "operator": "ge"}, {"budget": 0.19, "threshold": -0.42520507398723767, "operator": "ge"}, {"budget": 0.2, "threshold": -0.4536096024872595, "operator": "ge"}, {"budget": 0.21, "threshold": -0.5012691924326916, "operator": "ge"}, {"budget": 0.22, "threshold": -0.5287875660850664, "operator": "ge"}, {"budget": 0.23, "threshold": -0.5547255196883348, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5865771578922796, "operator": "ge"}, {"budget": 0.25, "threshold": -0.6328960638882495, "operator": "ge"}, {"budget": 0.26, "threshold": -0.6668905607948029, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6877832912349795, "operator": "ge"}, {"budget": 0.28, "threshold": -0.718698694583004, "operator": "ge"}, {"budget": 0.29, "threshold": -0.7696233721522948, "operator": "ge"}, {"budget": 0.3, "threshold": -0.8003189484234415, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8201234986353617, "operator": "ge"}, {"budget": 0.32, "threshold": -0.8318330821158397, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8519727852374311, "operator": "ge"}, {"budget": 0.34, "threshold": -0.8705223830225619, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8795518036862575, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8897017508842354, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9006271624201881, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9081885897708468, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9155130989887134, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9249169575546141, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9295051709083858, "operator": "ge"}, {"budget": 0.42, "threshold": -0.936814379022344, "operator": "ge"}, {"budget": 0.43, "threshold": -0.941456256672784, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9461788940654925, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9499279216033738, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9540591872068387, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9590024303839764, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9623187715357159, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9643147889950477, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9684800277091931, "operator": "ge"}, {"budget": 0.51, "threshold": -0.970596763033393, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9730405420236072, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9744678827761974, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9759865634699934, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9777144434595442, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9792340494900158, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9803705245943273, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9809585002856056, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9815070161477378, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9826926156036492, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9839721533949572, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9851902236928974, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9857572837643991, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9863860231721925, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9870713949549168, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9880586233935729, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9885357134555717, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9890507241145747, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9895596292704147, "operator": "ge"}, {"budget": 0.7, "threshold": -0.990326566635025, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9907526510134156, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9917079612117069, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9921048646485578, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9926922399379079, "operator": "ge"}, {"budget": 0.75, "threshold": -0.993385301967943, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9938171232108384, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9944272605804703, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9948749776886917, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9954020078723097, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9956280750179551, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9959672055235907, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9963415311770715, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9966332739685997, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9969232562561147, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9971082397935854, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9972501847564977, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9974276402713962, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9976458626187249, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9977259332729014, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9979297263383534, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9980723486441664, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9981863753044314, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9983067900977344, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9983822379854598, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9985449231199159, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9986822444578229, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9988577168700455, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9990289359911744, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9991848375743773, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.33333333333333, 98.08333333333333, 98.0, 97.74999999999999, 97.58333333333333, 97.41666666666666, 97.33333333333333, 97.08333333333333, 97.0, 96.66666666666667, 96.08333333333333, 95.83333333333333, 95.41666666666666, 95.16666666666667, 94.5, 93.91666666666666, 93.5, 92.66666666666666, 92.33333333333333, 91.58333333333333, 90.83333333333333, 90.25, 89.41666666666667, 88.75, 87.91666666666667, 87.16666666666666, 86.58333333333333, 85.66666666666667, 84.83333333333333, 83.91666666666666, 83.0, 82.08333333333333, 81.08333333333333, 80.16666666666666, 79.25, 78.33333333333333, 77.33333333333333, 76.33333333333333, 75.33333333333333, 74.41666666666666, 73.5, 72.66666666666666, 71.75, 70.75, 69.83333333333333, 69.0, 67.91666666666667, 67.08333333333333, 66.08333333333334, 65.08333333333334, 64.25, 63.33333333333333, 62.33333333333333, 61.5, 60.66666666666667, 59.74999999999999, 58.74999999999999, 57.99999999999999, 56.91666666666666, 56.00000000000001, 55.00000000000001, 54.08333333333333, 53.08333333333333, 52.166666666666664, 51.16666666666667, 50.16666666666667, 49.166666666666664, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.33333333333333, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.5, 36.5, 35.66666666666667, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 0, 0, 1, 1, 1, 1, 1, 2, 4, 5, 5, 8, 11, 12, 15, 17, 19, 20, 23, 24, 28, 35, 38, 43, 46, 54, 61, 66, 76, 82, 91, 100, 107, 117, 125, 135, 144, 151, 162, 172, 183, 194, 205, 217, 228, 239, 250, 262, 274, 286, 297, 308, 318, 329, 341, 352, 362, 375, 385, 397, 409, 419, 430, 442, 452, 462, 473, 485, 494, 507, 518, 530, 541, 553, 564, 576, 588, 600, 612, 624, 636, 646, 658, 670, 682, 694, 705, 717, 729, 740, 752, 762, 774, 786, 798, 810, 822, 833, 845, 857], "exact_score": [98.91666666666666, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.33333333333333, 98.08333333333333, 98.0, 97.74999999999999, 97.58333333333333, 97.41666666666666, 97.33333333333333, 97.08333333333333, 97.0, 96.66666666666667, 96.08333333333333, 95.83333333333333, 95.41666666666666, 95.16666666666667, 94.5, 93.91666666666666, 93.5, 92.66666666666666, 92.33333333333333, 91.58333333333333, 90.83333333333333, 90.25, 89.41666666666667, 88.75, 87.91666666666667, 87.16666666666666, 86.58333333333333, 85.66666666666667, 84.83333333333333, 83.83333333333333, 83.0, 82.08333333333333, 81.08333333333333, 80.16666666666666, 79.25, 78.33333333333333, 77.33333333333333, 76.33333333333333, 75.33333333333333, 74.41666666666666, 73.5, 72.66666666666666, 71.75, 70.75, 69.83333333333333, 68.91666666666667, 67.91666666666667, 67.08333333333333, 66.08333333333334, 65.08333333333334, 64.25, 63.33333333333333, 62.33333333333333, 61.5, 60.66666666666667, 59.74999999999999, 58.74999999999999, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.00000000000001, 54.08333333333333, 53.08333333333333, 52.166666666666664, 51.16666666666667, 50.16666666666667, 49.166666666666664, 48.16666666666667, 47.16666666666667, 46.166666666666664, 45.25, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.5, 36.5, 35.66666666666667, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.97083333333333, "threshold_area100": 71.97416666666668, "threshold_area25": 97.85833333333333}, "test": {"score": [90.16223679668623, 90.09319986192614, 90.09319986192614, 90.09319986192614, 90.09319986192614, 90.05868139454608, 90.02416292716603, 89.98964445978598, 89.92060752502589, 89.81705212288574, 89.76527442181566, 89.52364515015533, 89.2129789437349, 89.12668277528478, 88.97134967207455, 88.97134967207455, 88.76423886779426, 88.45357266137384, 88.31549879185364, 88.09112875388332, 87.90127718329306, 87.31446323783224, 87.0728339661719, 86.84846392820158, 86.36520538488091, 85.79565067311012, 85.22609596133931, 85.05350362443907, 84.62202278218847, 83.82809803244736, 83.05143251639628, 82.48187780462547, 82.10217466344494, 81.2219537452537, 80.27269589230238, 79.68588194684155, 79.13358646876078, 78.16706938211944, 77.45944080082843, 76.32033137728685, 75.30203658957542, 74.61166724197446, 73.52433551950293, 72.74767000345183, 72.0055229547808, 71.12530203658957, 70.27959958577839, 68.9333793579565, 68.05315843976527, 67.55264066275457, 65.9302726958923, 65.34345875043148, 64.44597859855021, 63.85916465308941, 63.25509147393855, 62.18501898515705, 61.2185018985157, 60.09665170866414, 59.7859855022437, 59.37176389368312, 58.249913703831545, 56.98998964445978, 55.850880220918185, 55.436658612357604, 54.780807732136694, 54.055919917155684, 53.10666206420434, 52.53710735243355, 51.74318260269244, 51.294442526751816, 50.36244390749051, 49.758370728339656, 48.498446668967894, 48.136002761477386, 47.53192958232654, 46.37556092509492, 45.63341387642389, 44.70141525716258, 43.94200897480152, 42.7166033828098, 42.25060407317915, 41.50845702450811, 40.69727304107697, 39.81705212288574, 38.83327580255437, 38.1429064549534, 37.65964791163272, 36.882982395581635, 36.209872281670684, 35.88194684156023, 34.777355885398684, 33.94891266827752, 33.29306178805661, 32.56817397307559, 32.20573006558508, 31.066620642043485, 30.031066620642044, 28.874697963410423, 27.977217811529165, 27.26958923023818, 26.56196064894718], "calls": [0.0, 1.9848118743527785, 2.8132550914739385, 3.676216775975147, 4.521919226786331, 5.488436313427684, 6.057991025198481, 6.834656541249569, 8.923023817742493, 10.010355540214015, 11.287538833275802, 12.789092164307906, 14.100793924749741, 14.998274076630997, 15.826717293752157, 16.46530894028305, 17.466344494304455, 18.967897825336554, 19.537452537107352, 20.365895754228514, 21.28063513979979, 23.041076976182257, 23.904038660683465, 24.577148774594406, 25.785295132896096, 27.166033828098037, 28.356920952709704, 28.960994131860545, 29.806696582671727, 31.325509147393856, 32.706247842595786, 33.79357956506731, 34.5012081463583, 35.86468760787021, 37.53883327580255, 38.47083189506386, 39.42008974801519, 40.7835692095271, 41.7846047635485, 43.13082499137038, 44.64963755609251, 45.52985847428374, 46.996893337935795, 48.118743527787366, 49.257852951328964, 50.327925440110455, 51.48429409734207, 53.12392129789437, 54.17673455298585, 54.88436313427684, 56.76561960648947, 57.455988954090444, 58.62961684501208, 59.26820849154297, 59.90680013807387, 61.114946496375566, 62.2885743872972, 63.51397997928891, 63.876423886779435, 64.35968243010011, 65.68864342423197, 67.10390058681395, 68.45012081463582, 68.96789782533655, 69.69278564031757, 70.52122885743873, 71.57404211253021, 72.19537452537107, 73.09285467725233, 73.62789092164309, 74.62892647566449, 75.37107352433551, 76.76907145322748, 77.14877459440801, 77.78736624093891, 79.03003106662064, 79.80669658267173, 80.7904729030031, 81.70521228857439, 82.96513634794616, 83.48291335864687, 84.25957887469797, 85.15705902657922, 86.08905764584053, 87.14187090093199, 87.86675871591302, 88.45357266137384, 89.24749741111495, 89.95512599240594, 90.3520883672765, 91.47393855712806, 92.30238177424923, 92.95823265447014, 93.71763893683121, 94.11460131170176, 95.30548843631342, 96.44459785985502, 97.63548498446669, 98.55022437003798, 99.27511218501898, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 2, 3, 3, 3, 3, 3, 4, 5, 7, 8, 9, 9, 11, 14, 18, 19, 20, 20, 21, 23, 27, 30, 35, 38, 39, 42, 45, 52, 52, 53, 55, 57, 58, 60, 61, 66, 66, 69, 70, 73, 73, 75, 77, 78, 79, 81, 81, 81, 82, 84, 85, 85, 86, 88, 89, 92, 94, 95, 96, 96, 96, 96, 97, 98, 99, 100, 100, 100, 101, 101, 101, 101, 101, 102, 102, 103, 103, 103, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106], "harmed": [0, 5, 5, 5, 5, 7, 9, 12, 17, 23, 26, 40, 58, 64, 74, 76, 89, 108, 116, 131, 145, 183, 198, 212, 240, 274, 309, 323, 351, 402, 450, 484, 509, 563, 625, 659, 692, 750, 793, 860, 921, 962, 1030, 1075, 1121, 1173, 1225, 1303, 1356, 1387, 1482, 1517, 1571, 1605, 1640, 1703, 1761, 1827, 1845, 1870, 1937, 2011, 2080, 2106, 2145, 2188, 2243, 2276, 2322, 2349, 2404, 2440, 2514, 2535, 2570, 2638, 2681, 2735, 2779, 2850, 2878, 2921, 2969, 3020, 3077, 3118, 3147, 3192, 3231, 3250, 3314, 3362, 3400, 3442, 3463, 3530, 3590, 3657, 3709, 3750, 3791], "exact_score": [90.16223679668623, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.09319986192614, 90.09319986192614, 90.04142216085606, 89.98964445978598, 89.93786675871591, 89.92060752502589, 89.83431135657577, 89.76527442181566, 89.60994131860545, 89.45460821539524, 89.23023817742491, 89.12668277528478, 88.97134967207455, 88.88505350362443, 88.62616499827408, 88.45357266137384, 88.17742492233344, 87.93579565067311, 87.67690714532274, 87.34898170521228, 87.02105626510182, 86.65861235761132, 86.29616845012082, 85.83016914049016, 85.32965136347946, 85.03624439074905, 84.51846738004832, 84.01794960303762, 83.46565412495684, 82.861580945806, 82.44735933724542, 81.8605453917846, 81.15291681049361, 80.51432516396272, 80.01380738695202, 79.39247497411115, 78.7193648602002, 78.01173627890921, 77.26958923023818, 76.38936831204694, 75.76803589920607, 75.04314808422505, 74.30100103555401, 73.52433551950293, 72.81670693821194, 72.16085605799101, 71.43596824301001, 70.6420434932689, 69.89989644459786, 69.03693476009664, 68.24301001035553, 67.44908526061442, 66.5688643424232, 65.70590265792198, 64.94649637556091, 64.11805315843976, 63.18605453917845, 62.2885743872972, 61.49464963755609, 60.56265101829479, 59.66517086641353, 58.85398688298239, 57.973765964791156, 57.11080428028995, 56.265101829478766, 55.41939937866758, 54.52191922678633, 53.62443907490507, 52.74421815671384, 51.829478771142554, 50.98377632033137, 50.08629616845012, 49.206075250258884, 48.2913358646876, 47.32481877804625, 46.41007939247498, 45.443562305833616, 44.54608215395236, 43.6486020020711, 42.682084915429755, 41.784604763548494, 40.85260614428719, 39.920607525025886, 38.98860890576458, 38.05661028650328, 37.12461166724197, 36.175353814290645, 35.260614428719364, 34.25957887469796, 33.275802554366585, 32.32654470141525, 31.342768381083875, 30.410769761822575, 29.478771142561268, 28.512254055919918, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 67.0924321626326, "threshold_area100": 64.68890231273731, "threshold_area25": 88.9351052813255}, "selected_budget": 1, "selected": {"val": {"score": 99.0, "calls": 1.0, "rescued": 1, "harmed": 0}, "test": {"score": 90.09319986192614, "calls": 1.9848118743527785, "rescued": 1, "harmed": 5}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.003116270527243614, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0033900446724146605, "operator": "ge"}, {"budget": 0.03, "threshold": -0.004032316617667675, "operator": "ge"}, {"budget": 0.04, "threshold": -0.0049895369447767735, "operator": "ge"}, {"budget": 0.05, "threshold": -0.006266729906201363, "operator": "ge"}, {"budget": 0.06, "threshold": -0.007432812824845314, "operator": "ge"}, {"budget": 0.07, "threshold": -0.013322552666068077, "operator": "ge"}, {"budget": 0.08, "threshold": -0.019540660083293915, "operator": "ge"}, {"budget": 0.09, "threshold": -0.024505017325282097, "operator": "ge"}, {"budget": 0.1, "threshold": -0.03155878186225891, "operator": "ge"}, {"budget": 0.11, "threshold": -0.04344206675887108, "operator": "ge"}, {"budget": 0.12, "threshold": -0.05630725622177124, "operator": "ge"}, {"budget": 0.13, "threshold": -0.0677725151181221, "operator": "ge"}, {"budget": 0.14, "threshold": -0.08212851732969284, "operator": "ge"}, {"budget": 0.15, "threshold": -0.09465379267930984, "operator": "ge"}, {"budget": 0.16, "threshold": -0.12039705365896225, "operator": "ge"}, {"budget": 0.17, "threshold": -0.15265679359436035, "operator": "gt"}, {"budget": 0.18, "threshold": -0.16952282190322876, "operator": "ge"}, {"budget": 0.19, "threshold": -0.19205865263938904, "operator": "ge"}, {"budget": 0.2, "threshold": -0.22592243552207947, "operator": "ge"}, {"budget": 0.21, "threshold": -0.2661927342414856, "operator": "ge"}, {"budget": 0.22, "threshold": -0.2865561544895172, "operator": "ge"}, {"budget": 0.23, "threshold": -0.31902703642845154, "operator": "ge"}, {"budget": 0.24, "threshold": -0.3668045997619629, "operator": "ge"}, {"budget": 0.25, "threshold": -0.4181821644306183, "operator": "ge"}, {"budget": 0.26, "threshold": -0.4509396255016327, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5025128722190857, "operator": "ge"}, {"budget": 0.28, "threshold": -0.5811486840248108, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6406046152114868, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6810369491577148, "operator": "ge"}, {"budget": 0.31, "threshold": -0.7006232142448425, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7252780795097351, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7578660249710083, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7749558687210083, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7916521430015564, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8077175617218018, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8237268924713135, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8319712281227112, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8461760878562927, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8605977296829224, "operator": "ge"}, {"budget": 0.41, "threshold": -0.8668866753578186, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8858737945556641, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8943472504615784, "operator": "ge"}, {"budget": 0.44, "threshold": -0.904370903968811, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9113060832023621, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9211324453353882, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9260966777801514, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9293039441108704, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9374251365661621, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9421109557151794, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9471938610076904, "operator": "ge"}, {"budget": 0.52, "threshold": -0.951366126537323, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9535362124443054, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9567242860794067, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9589358568191528, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9613366723060608, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9624992609024048, "operator": "ge"}, {"budget": 0.58, "threshold": -0.964935302734375, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9668887257575989, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9695155024528503, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9707784056663513, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9719814658164978, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9732396006584167, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9742440581321716, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9759281277656555, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9771397113800049, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9783518314361572, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9792950749397278, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9803991913795471, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9817987084388733, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9830076098442078, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9841833114624023, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9852063655853271, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9864652752876282, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9874914288520813, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9887422919273376, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9895450472831726, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9904671311378479, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9909729361534119, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9916595220565796, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9924538731575012, "operator": "ge"}, {"budget": 0.82, "threshold": -0.993036687374115, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9937284588813782, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9940214157104492, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9944295883178711, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9948061108589172, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9951192140579224, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9954493045806885, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9957038164138794, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9960198998451233, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9963110685348511, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9965065717697144, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9966930747032166, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9969112277030945, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9971948862075806, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9974461197853088, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9977213144302368, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9981563687324524, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9984790682792664, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.5, 98.5, 98.5, 98.08333333333333, 98.08333333333333, 97.91666666666666, 97.74999999999999, 97.66666666666667, 97.5, 97.5, 97.33333333333333, 97.08333333333333, 96.58333333333333, 96.25, 95.91666666666666, 95.66666666666667, 95.25, 94.75, 94.16666666666667, 93.5, 92.83333333333333, 92.0, 91.25, 90.58333333333333, 89.83333333333333, 89.25, 88.5, 87.75, 87.33333333333333, 86.58333333333333, 85.66666666666667, 84.66666666666667, 83.91666666666666, 82.83333333333334, 81.83333333333334, 80.91666666666666, 80.0, 79.25, 78.25, 77.25, 76.25, 75.33333333333333, 74.33333333333333, 73.33333333333333, 72.58333333333333, 71.58333333333333, 70.66666666666667, 69.75, 68.91666666666667, 67.91666666666667, 66.91666666666667, 65.99999999999999, 65.16666666666666, 64.16666666666666, 63.16666666666666, 62.33333333333333, 61.416666666666664, 60.58333333333332, 59.66666666666667, 58.833333333333336, 57.91666666666666, 56.833333333333336, 55.91666666666666, 55.00000000000001, 54.0, 53.0, 52.08333333333333, 51.08333333333333, 50.16666666666667, 49.166666666666664, 48.16666666666667, 47.16666666666667, 46.25, 45.41666666666666, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.583333333333336, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 16.916666666666664, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 0, 1, 1, 1, 1, 1, 1, 3, 5, 5, 5, 10, 11, 13, 15, 16, 18, 18, 20, 23, 29, 33, 37, 40, 45, 51, 58, 66, 74, 84, 93, 101, 110, 117, 126, 136, 141, 151, 162, 174, 183, 196, 208, 219, 230, 239, 251, 263, 275, 286, 298, 310, 319, 331, 342, 353, 363, 375, 387, 398, 408, 420, 432, 442, 453, 463, 474, 484, 495, 508, 519, 530, 542, 554, 565, 577, 588, 600, 612, 624, 635, 645, 658, 670, 682, 693, 705, 717, 728, 740, 751, 763, 775, 787, 799, 811, 822, 833, 845, 857], "exact_score": [98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.5, 98.5, 98.5, 98.08333333333333, 98.08333333333333, 97.91666666666666, 97.74999999999999, 97.66666666666667, 97.5, 97.5, 97.33333333333333, 97.08333333333333, 96.58333333333333, 96.25, 95.91666666666666, 95.66666666666667, 95.25, 94.75, 94.16666666666667, 93.5, 92.83333333333333, 92.0, 91.25, 90.58333333333333, 89.83333333333333, 89.25, 88.5, 87.75, 87.33333333333333, 86.58333333333333, 85.66666666666667, 84.66666666666667, 83.83333333333333, 82.83333333333334, 81.83333333333334, 80.91666666666666, 80.0, 79.25, 78.25, 77.25, 76.25, 75.33333333333333, 74.33333333333333, 73.33333333333333, 72.58333333333333, 71.58333333333333, 70.66666666666667, 69.75, 68.83333333333333, 67.91666666666667, 66.91666666666667, 65.99999999999999, 65.16666666666666, 64.16666666666666, 63.16666666666666, 62.33333333333333, 61.416666666666664, 60.58333333333332, 59.66666666666667, 58.833333333333336, 57.833333333333336, 56.833333333333336, 55.91666666666666, 55.00000000000001, 54.0, 53.0, 52.08333333333333, 51.08333333333333, 50.16666666666667, 49.166666666666664, 48.16666666666667, 47.16666666666667, 46.25, 45.33333333333333, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.5, 37.5, 36.583333333333336, 35.583333333333336, 34.583333333333336, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.92583333333332, "threshold_area100": 71.92916666666666, "threshold_area25": 97.85000000000001}, "test": {"score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.09319986192614, 90.05868139454608, 90.02416292716603, 89.97238522609597, 89.92060752502589, 89.88608905764583, 89.73075595443561, 89.59268208491544, 89.54090438384536, 89.42008974801519, 89.35105281325508, 89.07490507421471, 88.88505350362443, 88.74697963410424, 88.60890576458405, 88.22920262340351, 87.78046254746289, 87.59061097687263, 87.31446323783224, 86.96927856403175, 86.45150155333103, 86.17535381429065, 85.69209527096997, 84.96720745598894, 84.15602347255782, 83.44839489126683, 82.9996548153262, 82.39558163617535, 81.51536071798412, 80.91128753883328, 80.4970659302727, 79.91025198481188, 79.20262340352087, 78.63306869175008, 77.7873662409389, 77.01070072488781, 76.49292371418709, 75.12944425267519, 74.69796341042458, 73.66240938902313, 72.95478080773213, 71.85018985157059, 71.19433897134967, 70.78011736278908, 69.01967552640662, 68.17397307559544, 67.10390058681395, 66.03382809803244, 65.41249568519157, 64.58405246807042, 63.75560925094925, 62.547462892647566, 62.2885743872972, 61.02865032792544, 59.958577839143935, 58.50880220918191, 57.73213669313082, 57.17984121505004, 56.541249568519156, 56.02347255781843, 55.07421470486711, 54.33206765619606, 53.64169830859509, 52.9340697273041, 52.07110804280289, 51.242664825681736, 50.34518467380048, 49.758370728339656, 49.08526061442872, 48.30859509837763, 47.428374180186395, 46.427338626164996, 45.7887469796341, 44.70141525716258, 44.028305143251636, 43.28615809458059, 42.181567138419055, 41.38764238867794, 40.144977562996196, 39.7134967207456, 38.81601656886434, 38.073869520193306, 37.41801863997238, 36.64135312392129, 36.0027614773904, 35.19157749395927, 34.173282706247846, 33.690024162927166, 33.06869175008629, 32.34380393510528, 31.29099068001381, 30.255436658612357, 29.27166033828098, 28.046254746289257, 27.235070762858125, 26.56196064894718], "calls": [0.0, 1.4497756299620297, 2.0538488091128753, 3.089402830514325, 3.8660683465654127, 4.590956161546427, 5.108733172247152, 6.765619606489472, 8.04280289955126, 9.061097687262686, 9.975837072833967, 11.529168104936142, 12.823610631687954, 13.531239212978944, 14.273386261649984, 14.808422506040731, 16.12012426648257, 17.311011391094237, 17.949603037625128, 18.70900931998619, 20.0724887814981, 21.52226441146013, 22.109078356920953, 22.989299275112185, 24.2147048671039, 25.578184328615812, 26.216775975146707, 27.235070762858125, 28.840179496030377, 30.203658957542284, 31.549879185364173, 32.22298929927511, 33.29306178805661, 34.79461511908871, 35.62305833620987, 36.26164998274077, 37.072833966171906, 38.24646185709354, 38.91957197100449, 40.12771832930618, 41.31860545391785, 41.95719710044874, 43.734898170521234, 44.56334138764239, 45.94408008284432, 46.96237487055575, 48.44666896789783, 49.257852951328964, 49.82740766309976, 52.05384880911288, 53.141180531584396, 54.47014152571625, 55.76458405246807, 56.50673110113911, 57.57680358992061, 58.56057991025199, 59.924059371763896, 60.2865032792544, 61.68450120814636, 62.909906800138074, 64.54953400069037, 65.44701415257163, 66.12012426648258, 66.84501208146358, 67.48360372799448, 68.6399723852261, 69.5029340697273, 70.31411805315844, 71.10804280289955, 72.07455988954091, 73.09285467725233, 74.11114946496376, 74.83603727994476, 75.56092509492579, 76.47566448049706, 77.51121850189851, 78.66758715913014, 79.4269934414912, 80.58336209872282, 81.30824991370383, 82.15395236451502, 83.37935795650672, 84.25957887469797, 85.62305833620987, 86.0717984121505, 87.0728339661719, 87.81498101484294, 88.62616499827408, 89.4891266827753, 90.19675526406627, 91.04245771487746, 92.07801173627891, 92.56127027959958, 93.19986192613048, 93.94200897480151, 95.04659993096305, 96.15119088712461, 97.16948567483604, 98.48118743527787, 99.30963065239902, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 2, 2, 2, 3, 3, 4, 4, 5, 6, 8, 8, 12, 13, 13, 14, 22, 22, 25, 26, 32, 35, 35, 35, 36, 39, 39, 42, 45, 45, 49, 52, 56, 56, 57, 59, 59, 63, 65, 67, 69, 70, 73, 74, 74, 75, 76, 76, 77, 78, 79, 80, 82, 84, 85, 85, 85, 87, 89, 89, 92, 92, 94, 96, 98, 98, 98, 98, 100, 102, 103, 103, 103, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106], "harmed": [0, 2, 2, 2, 2, 4, 6, 8, 11, 14, 17, 27, 35, 38, 45, 49, 66, 77, 86, 94, 117, 144, 157, 173, 197, 228, 244, 273, 323, 370, 414, 441, 482, 536, 571, 595, 630, 674, 707, 759, 807, 837, 920, 948, 1012, 1053, 1118, 1158, 1182, 1288, 1339, 1403, 1467, 1504, 1555, 1604, 1674, 1690, 1764, 1826, 1911, 1957, 1990, 2028, 2060, 2117, 2161, 2201, 2242, 2294, 2344, 2396, 2433, 2472, 2519, 2572, 2632, 2669, 2732, 2771, 2816, 2882, 2929, 3001, 3026, 3079, 3122, 3161, 3206, 3243, 3290, 3349, 3377, 3413, 3455, 3517, 3577, 3634, 3705, 3752, 3791], "exact_score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.07594062823611, 90.04142216085606, 90.02416292716603, 89.98964445978598, 89.92060752502589, 89.88608905764583, 89.81705212288574, 89.67897825336554, 89.59268208491544, 89.43734898170521, 89.29927511218501, 89.07490507421471, 88.97134967207455, 88.74697963410424, 88.53986882982396, 88.22920262340351, 87.93579565067311, 87.64238867794269, 87.31446323783224, 86.98653779772178, 86.7103900586814, 86.26164998274076, 85.79565067311012, 85.46772523299964, 84.86365205384881, 84.24231964100794, 83.70728339661719, 83.1377286848464, 82.56817397307559, 81.96410079392474, 81.39454608215395, 80.68691750086295, 79.97928891957197, 79.3579565067311, 78.59855022437003, 77.90818087676908, 77.23507076285813, 76.45840524680703, 75.61270279599586, 75.008629616845, 74.33551950293406, 73.61063168795306, 72.92026234035208, 72.17811529168104, 71.36693130824992, 70.65930272695891, 69.83085950983777, 69.0714532274767, 68.31204694511564, 67.48360372799446, 66.67241974456334, 65.84397652744218, 65.05005177770107, 64.25612702795995, 63.39316534345875, 62.495685191577486, 61.66724197445633, 60.717984121505, 59.88954090438384, 59.026579219882635, 58.111839834311354, 57.26613738350017, 56.45495340006903, 55.626510182947875, 54.74628926475664, 53.90058681394545, 53.05488436313428, 52.15740421125301, 51.31170176044184, 50.46599930963065, 49.603037625129446, 48.72281670693821, 47.89437348981705, 47.014152571625814, 46.15119088712461, 45.25371073524335, 44.304452882292026, 43.44149119779082, 42.492233344839484, 41.66379012771833, 40.714532274766995, 39.799792889195714, 38.86779426993441, 37.90127718329306, 37.107352433551945, 36.175353814290645, 35.22609596133931, 34.25957887469796, 33.25854332067656, 32.29202623403521, 31.342768381083875, 30.410769761822575, 29.426993441491188, 28.477735588539865, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 66.7772828449413, "threshold_area100": 65.47471522264411, "threshold_area25": 89.17328270624783}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.356537103652954, "operator": "ge"}, {"budget": 0.02, "threshold": -3.5432257652282715, "operator": "ge"}, {"budget": 0.03, "threshold": -4.322609901428223, "operator": "ge"}, {"budget": 0.04, "threshold": -4.99834680557251, "operator": "ge"}, {"budget": 0.05, "threshold": -5.665918827056885, "operator": "ge"}, {"budget": 0.06, "threshold": -5.977294921875, "operator": "ge"}, {"budget": 0.07, "threshold": -6.353410720825195, "operator": "ge"}, {"budget": 0.08, "threshold": -6.7125935554504395, "operator": "ge"}, {"budget": 0.09, "threshold": -7.052607536315918, "operator": "ge"}, {"budget": 0.1, "threshold": -7.269964218139648, "operator": "ge"}, {"budget": 0.11, "threshold": -7.4939351081848145, "operator": "ge"}, {"budget": 0.12, "threshold": -7.64349889755249, "operator": "ge"}, {"budget": 0.13, "threshold": -7.876706600189209, "operator": "ge"}, {"budget": 0.14, "threshold": -8.072257041931152, "operator": "ge"}, {"budget": 0.15, "threshold": -8.24519157409668, "operator": "ge"}, {"budget": 0.16, "threshold": -8.4192476272583, "operator": "ge"}, {"budget": 0.17, "threshold": -8.519012451171875, "operator": "ge"}, {"budget": 0.18, "threshold": -8.662227630615234, "operator": "ge"}, {"budget": 0.19, "threshold": -8.786550521850586, "operator": "ge"}, {"budget": 0.2, "threshold": -8.884908676147461, "operator": "ge"}, {"budget": 0.21, "threshold": -9.043693542480469, "operator": "ge"}, {"budget": 0.22, "threshold": -9.093488693237305, "operator": "ge"}, {"budget": 0.23, "threshold": -9.197589874267578, "operator": "ge"}, {"budget": 0.24, "threshold": -9.26119327545166, "operator": "ge"}, {"budget": 0.25, "threshold": -9.378862380981445, "operator": "ge"}, {"budget": 0.26, "threshold": -9.458412170410156, "operator": "ge"}, {"budget": 0.27, "threshold": -9.541732788085938, "operator": "ge"}, {"budget": 0.28, "threshold": -9.616962432861328, "operator": "ge"}, {"budget": 0.29, "threshold": -9.702178001403809, "operator": "ge"}, {"budget": 0.3, "threshold": -9.839187622070312, "operator": "ge"}, {"budget": 0.31, "threshold": -9.885873794555664, "operator": "ge"}, {"budget": 0.32, "threshold": -9.961847305297852, "operator": "ge"}, {"budget": 0.33, "threshold": -10.04655647277832, "operator": "ge"}, {"budget": 0.34, "threshold": -10.101309776306152, "operator": "ge"}, {"budget": 0.35, "threshold": -10.167162895202637, "operator": "ge"}, {"budget": 0.36, "threshold": -10.239412307739258, "operator": "ge"}, {"budget": 0.37, "threshold": -10.315418243408203, "operator": "ge"}, {"budget": 0.38, "threshold": -10.383615493774414, "operator": "ge"}, {"budget": 0.39, "threshold": -10.44958209991455, "operator": "ge"}, {"budget": 0.4, "threshold": -10.517107963562012, "operator": "ge"}, {"budget": 0.41, "threshold": -10.566692352294922, "operator": "ge"}, {"budget": 0.42, "threshold": -10.651063919067383, "operator": "ge"}, {"budget": 0.43, "threshold": -10.714725494384766, "operator": "ge"}, {"budget": 0.44, "threshold": -10.758544921875, "operator": "ge"}, {"budget": 0.45, "threshold": -10.837638854980469, "operator": "ge"}, {"budget": 0.46, "threshold": -10.926116943359375, "operator": "ge"}, {"budget": 0.47, "threshold": -10.998592376708984, "operator": "ge"}, {"budget": 0.48, "threshold": -11.08373737335205, "operator": "ge"}, {"budget": 0.49, "threshold": -11.140268325805664, "operator": "ge"}, {"budget": 0.5, "threshold": -11.200836181640625, "operator": "ge"}, {"budget": 0.51, "threshold": -11.285821914672852, "operator": "ge"}, {"budget": 0.52, "threshold": -11.365482330322266, "operator": "ge"}, {"budget": 0.53, "threshold": -11.423667907714844, "operator": "ge"}, {"budget": 0.54, "threshold": -11.483856201171875, "operator": "ge"}, {"budget": 0.55, "threshold": -11.531824111938477, "operator": "ge"}, {"budget": 0.56, "threshold": -11.579999923706055, "operator": "ge"}, {"budget": 0.57, "threshold": -11.645971298217773, "operator": "ge"}, {"budget": 0.58, "threshold": -11.692161560058594, "operator": "ge"}, {"budget": 0.59, "threshold": -11.727598190307617, "operator": "ge"}, {"budget": 0.6, "threshold": -11.836832046508789, "operator": "ge"}, {"budget": 0.61, "threshold": -11.911701202392578, "operator": "ge"}, {"budget": 0.62, "threshold": -11.937761306762695, "operator": "ge"}, {"budget": 0.63, "threshold": -12.01346206665039, "operator": "ge"}, {"budget": 0.64, "threshold": -12.085155487060547, "operator": "ge"}, {"budget": 0.65, "threshold": -12.116498947143555, "operator": "ge"}, {"budget": 0.66, "threshold": -12.180065155029297, "operator": "ge"}, {"budget": 0.67, "threshold": -12.247365951538086, "operator": "ge"}, {"budget": 0.68, "threshold": -12.328225135803223, "operator": "ge"}, {"budget": 0.69, "threshold": -12.407208442687988, "operator": "ge"}, {"budget": 0.7, "threshold": -12.465666770935059, "operator": "ge"}, {"budget": 0.71, "threshold": -12.508895874023438, "operator": "ge"}, {"budget": 0.72, "threshold": -12.538689613342285, "operator": "ge"}, {"budget": 0.73, "threshold": -12.596345901489258, "operator": "ge"}, {"budget": 0.74, "threshold": -12.61965560913086, "operator": "ge"}, {"budget": 0.75, "threshold": -12.654532432556152, "operator": "ge"}, {"budget": 0.76, "threshold": -12.695935249328613, "operator": "ge"}, {"budget": 0.77, "threshold": -12.747030258178711, "operator": "ge"}, {"budget": 0.78, "threshold": -12.802444458007812, "operator": "ge"}, {"budget": 0.79, "threshold": -12.829538345336914, "operator": "ge"}, {"budget": 0.8, "threshold": -12.872607231140137, "operator": "ge"}, {"budget": 0.81, "threshold": -12.895025253295898, "operator": "ge"}, {"budget": 0.82, "threshold": -12.933992385864258, "operator": "ge"}, {"budget": 0.83, "threshold": -12.986672401428223, "operator": "ge"}, {"budget": 0.84, "threshold": -13.035940170288086, "operator": "ge"}, {"budget": 0.85, "threshold": -13.097797393798828, "operator": "ge"}, {"budget": 0.86, "threshold": -13.139577865600586, "operator": "ge"}, {"budget": 0.87, "threshold": -13.209268569946289, "operator": "ge"}, {"budget": 0.88, "threshold": -13.256525039672852, "operator": "ge"}, {"budget": 0.89, "threshold": -13.305723190307617, "operator": "ge"}, {"budget": 0.9, "threshold": -13.384674072265625, "operator": "ge"}, {"budget": 0.91, "threshold": -13.4473876953125, "operator": "ge"}, {"budget": 0.92, "threshold": -13.502090454101562, "operator": "ge"}, {"budget": 0.93, "threshold": -13.558379173278809, "operator": "ge"}, {"budget": 0.94, "threshold": -13.641712188720703, "operator": "ge"}, {"budget": 0.95, "threshold": -13.773164749145508, "operator": "ge"}, {"budget": 0.96, "threshold": -13.894286155700684, "operator": "ge"}, {"budget": 0.97, "threshold": -14.060197830200195, "operator": "ge"}, {"budget": 0.98, "threshold": -14.246818542480469, "operator": "ge"}, {"budget": 0.99, "threshold": -14.497209548950195, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.0, 97.66666666666667, 97.08333333333333, 96.41666666666666, 95.75, 95.08333333333333, 94.25, 93.58333333333333, 93.0, 92.41666666666666, 92.08333333333333, 91.49999999999999, 91.08333333333333, 90.5, 90.33333333333333, 90.08333333333333, 89.66666666666666, 89.16666666666666, 88.75, 88.41666666666667, 88.0, 87.75, 87.25, 86.83333333333333, 86.25, 85.91666666666666, 85.5, 85.0, 84.5, 84.08333333333333, 83.91666666666666, 83.5, 82.83333333333334, 82.33333333333333, 81.75, 81.25, 80.83333333333333, 80.33333333333333, 79.66666666666666, 79.16666666666666, 78.75, 78.0, 77.25, 76.91666666666667, 76.33333333333333, 75.58333333333334, 74.91666666666667, 74.33333333333333, 73.5, 72.5, 71.83333333333333, 71.08333333333333, 70.33333333333334, 69.66666666666667, 68.91666666666667, 68.16666666666666, 67.16666666666666, 66.41666666666666, 65.5, 64.66666666666666, 63.916666666666664, 62.999999999999986, 62.083333333333336, 61.083333333333336, 60.33333333333333, 59.416666666666664, 58.58333333333333, 57.74999999999999, 56.749999999999986, 55.75, 54.833333333333336, 53.833333333333336, 52.91666666666666, 52.0, 51.0, 50.16666666666667, 49.166666666666664, 48.24999999999999, 47.24999999999999, 46.25, 45.33333333333333, 44.41666666666666, 43.41666666666666, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.5, 35.5, 34.5, 33.5, 32.58333333333333, 31.58333333333333, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 2, 12, 18, 25, 33, 41, 49, 59, 67, 74, 81, 85, 92, 97, 104, 106, 109, 114, 120, 125, 129, 134, 137, 143, 148, 155, 159, 164, 170, 176, 181, 183, 188, 196, 202, 209, 215, 220, 226, 234, 240, 245, 254, 263, 267, 274, 283, 291, 298, 308, 320, 328, 337, 346, 354, 363, 372, 384, 393, 404, 414, 423, 434, 445, 457, 466, 477, 487, 497, 509, 521, 532, 544, 555, 566, 578, 588, 600, 611, 623, 635, 646, 657, 669, 681, 693, 705, 717, 729, 741, 752, 764, 776, 788, 799, 811, 821, 833, 845, 857], "exact_score": [98.91666666666666, 98.83333333333333, 98.0, 97.66666666666667, 97.08333333333333, 96.41666666666666, 95.75, 95.08333333333333, 94.25, 93.58333333333333, 93.0, 92.41666666666666, 92.08333333333333, 91.49999999999999, 91.08333333333333, 90.5, 90.33333333333333, 90.08333333333333, 89.66666666666666, 89.16666666666666, 88.75, 88.41666666666667, 88.0, 87.75, 87.25, 86.83333333333333, 86.25, 85.91666666666666, 85.5, 85.0, 84.5, 84.08333333333333, 83.91666666666666, 83.5, 82.83333333333334, 82.33333333333333, 81.75, 81.25, 80.83333333333333, 80.33333333333333, 79.66666666666666, 79.16666666666666, 78.75, 78.0, 77.25, 76.91666666666667, 76.33333333333333, 75.58333333333334, 74.91666666666667, 74.33333333333333, 73.5, 72.5, 71.83333333333333, 71.08333333333333, 70.33333333333334, 69.66666666666667, 68.91666666666667, 68.08333333333333, 67.16666666666666, 66.41666666666666, 65.5, 64.66666666666666, 63.916666666666664, 62.999999999999986, 62.083333333333336, 61.083333333333336, 60.33333333333333, 59.416666666666664, 58.58333333333333, 57.666666666666664, 56.749999999999986, 55.75, 54.833333333333336, 53.833333333333336, 52.91666666666666, 52.0, 51.0, 50.16666666666667, 49.166666666666664, 48.24999999999999, 47.24999999999999, 46.25, 45.25, 44.41666666666666, 43.41666666666666, 42.41666666666667, 41.41666666666667, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.5, 35.5, 34.5, 33.5, 32.58333333333333, 31.58333333333333, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 69.05666666666666, "threshold_area100": 69.05916666666666, "threshold_area25": 92.30833333333334}, "test": {"score": [90.16223679668623, 89.42008974801519, 88.38453572661373, 86.93476009665171, 85.27787366240939, 83.1377286848464, 81.84328615809457, 80.08284432171212, 78.58129099068, 77.32136693130825, 76.59647911632723, 76.00966517086641, 75.47462892647566, 74.5081118398343, 73.76596479116327, 73.16189161201243, 72.45426303072144, 72.07455988954091, 71.62581981360027, 71.15982050396961, 70.93545046599931, 70.45219192267864, 70.24508111839833, 69.7100448740076, 69.43389713496721, 68.91612012426648, 68.48463928201586, 68.0704176734553, 67.69071453227477, 67.2074559889541, 66.22367966862271, 65.96479116327234, 65.46427338626164, 64.94649637556091, 64.77390403866067, 64.44597859855021, 64.10079392474974, 63.703831549879176, 63.2896099413186, 62.927166033828094, 62.58198136002762, 62.2885743872972, 61.73627890921642, 61.2185018985157, 60.873317224715215, 60.33828098032448, 59.647911632723506, 59.181912323092845, 58.56057991025199, 58.06006213324129, 57.59406282361063, 56.83465654124957, 56.161546427338614, 55.67828788401794, 55.212288574387294, 54.67725232999654, 54.124956851915776, 53.62443907490507, 53.227476700034515, 52.86503279254401, 51.898515705902646, 51.05281325509148, 50.77666551605109, 49.86192613047981, 49.15429754918881, 48.67103900586814, 47.89437348981705, 47.290300310666204, 46.220227821884706, 45.2191922678633, 44.58060062133241, 44.06282361063168, 43.73489817052123, 43.02726958923023, 42.78564031756989, 42.405937176389365, 41.784604763548494, 41.197790818087675, 40.61097687262685, 40.35208836727649, 39.74801518812564, 39.42008974801519, 38.86779426993441, 38.05661028650328, 37.573351743182606, 36.882982395581635, 36.36520538488091, 35.58853986882981, 34.94994822229892, 34.5702450811184, 33.96617190196755, 33.4311356575768, 32.84432171211598, 32.34380393510528, 31.722471522264406, 30.773213669313083, 29.96202968588194, 29.133586468760786, 28.30514325163962, 27.45944080082844, 26.56196064894718], "calls": [0.0, 3.1239212978943733, 7.1280635139799795, 10.873317224715223, 14.963755609250951, 18.916120124266484, 21.039005868139455, 23.990334829133587, 26.596479116327238, 28.477735588539872, 29.84121505005178, 31.083879875733516, 32.10217466344494, 33.79357956506731, 35.08802209181912, 36.53779772178115, 38.05661028650328, 38.81601656886434, 39.799792889195714, 40.7835692095271, 41.50845702450811, 42.83741801863997, 43.2861580945806, 44.47704521919227, 45.02934069727304, 46.220227821884706, 47.065930272695894, 47.894373489817056, 48.774594408008284, 49.70659302726959, 51.43251639627201, 52.0020711080428, 52.83051432516396, 53.95236451501554, 54.31480842250605, 55.005177770107004, 55.695547117707974, 56.50673110113911, 57.23161891612012, 57.93924749741112, 58.56057991025199, 59.04383845357266, 59.95857783914394, 60.82153952364515, 61.32205730065585, 62.08146358301691, 63.06523990334829, 63.70383154987918, 64.58405246807042, 65.25716258198136, 65.84397652744218, 66.82775284777355, 67.6389368312047, 68.32930617880567, 68.9333793579565, 69.5719710044874, 70.2105626510183, 70.81463583016914, 71.29789437348981, 71.72937521574042, 72.93752157404211, 73.93855712806351, 74.2147048671039, 75.25025888850536, 75.97514670348636, 76.47566448049706, 77.39040386606834, 78.06351397997929, 79.2198826372109, 80.30721435968243, 81.0666206420435, 81.65343458750431, 81.98136002761477, 82.74076630997584, 82.9996548153262, 83.37935795650672, 84.08698653779773, 84.69105971694857, 85.29513289609942, 85.5712806351398, 86.17535381429065, 86.5032792544011, 87.05557473248187, 87.93579565067311, 88.45357266137384, 89.16120124266482, 89.7134967207456, 90.52468070417673, 91.18053158439766, 91.62927166033829, 92.30238177424923, 92.87193648602002, 93.49326889886089, 94.01104590956162, 94.64963755609251, 95.61615464273386, 96.4963755609251, 97.3593372454263, 98.18778046254747, 99.08526061442872, 100.0], "rescued": [0, 12, 36, 50, 62, 69, 76, 82, 87, 88, 91, 92, 94, 96, 96, 97, 97, 97, 97, 97, 97, 98, 98, 99, 99, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106], "harmed": [0, 55, 139, 237, 345, 476, 558, 666, 758, 832, 877, 912, 945, 1003, 1046, 1082, 1123, 1145, 1171, 1198, 1211, 1240, 1252, 1284, 1300, 1331, 1357, 1381, 1403, 1431, 1488, 1503, 1532, 1562, 1572, 1591, 1611, 1635, 1659, 1680, 1700, 1717, 1749, 1779, 1799, 1830, 1870, 1897, 1933, 1962, 1989, 2033, 2072, 2100, 2127, 2158, 2190, 2219, 2242, 2263, 2319, 2368, 2384, 2437, 2478, 2506, 2551, 2586, 2649, 2707, 2745, 2775, 2794, 2835, 2849, 2871, 2907, 2941, 2975, 2990, 3025, 3044, 3076, 3123, 3151, 3191, 3221, 3266, 3303, 3326, 3361, 3392, 3426, 3455, 3491, 3546, 3593, 3642, 3690, 3739, 3791], "exact_score": [90.16223679668623, 89.88608905764583, 89.59268208491544, 89.45460821539524, 89.14394200897479, 88.91957197100449, 88.66068346565412, 88.43631342768381, 88.05661028650327, 87.69416637901277, 87.36624093890231, 86.84846392820158, 86.55505695547117, 86.20987228167068, 85.65757680358992, 85.29513289609942, 84.81187435277874, 84.25957887469797, 83.72454263030721, 83.08595098377631, 82.44735933724542, 81.89506385916465, 81.32550914739384, 80.75595443562305, 80.08284432171212, 79.53054884363134, 78.94373489817052, 78.2878840179496, 77.68381083879875, 77.01070072488781, 76.52744218156712, 76.06144287193648, 75.54366586123575, 74.90507421470487, 74.37003797031412, 73.81774249223334, 73.38626164998274, 72.95478080773213, 72.47152226441146, 72.02278218847083, 71.47048671039006, 71.09078356920952, 70.71108042802899, 70.40041422160856, 69.88263721090784, 69.46841560234725, 68.96789782533655, 68.51915774939592, 68.01863997238522, 67.55264066275457, 66.96582671729375, 66.46530894028305, 65.96479116327234, 65.41249568519157, 64.9292371418709, 64.44597859855021, 63.96272005522954, 63.46220227821884, 62.909906800138074, 62.323092854677256, 61.70176044183637, 61.13220573006558, 60.40731791508457, 59.7169485674836, 58.974801518812555, 58.23265447014152, 57.4387297204004, 56.73110113910942, 55.91991715567828, 55.160510873317214, 54.31480842250605, 53.48636520538488, 52.675181221953736, 51.84673800483258, 51.0010355540214, 50.06903693476009, 49.137038315498785, 48.25681739730756, 47.359337245426296, 46.44459785985502, 45.49534000690369, 44.64963755609251, 43.73489817052123, 42.78564031756989, 41.888160165688646, 40.90438384535726, 39.93786675871591, 38.93683120469451, 38.0048325854332, 37.05557473248188, 36.08905764584052, 35.13979979288919, 34.276838108387985, 33.31032102174664, 32.3610631687953, 31.377286848463925, 30.42802899551259, 29.46151190887124, 28.49499482222989, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 64.16313002309532, "threshold_area100": 56.3219710044874, "threshold_area25": 76.67414566793234}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.1409475803375244, "operator": "ge"}, {"budget": 0.02, "threshold": -1.9278680086135864, "operator": "ge"}, {"budget": 0.03, "threshold": -2.403024911880493, "operator": "ge"}, {"budget": 0.04, "threshold": -3.088517189025879, "operator": "ge"}, {"budget": 0.05, "threshold": -3.7740254402160645, "operator": "ge"}, {"budget": 0.06, "threshold": -4.206352233886719, "operator": "ge"}, {"budget": 0.07, "threshold": -4.640300750732422, "operator": "ge"}, {"budget": 0.08, "threshold": -5.165765285491943, "operator": "ge"}, {"budget": 0.09, "threshold": -5.461252212524414, "operator": "ge"}, {"budget": 0.1, "threshold": -5.575578689575195, "operator": "ge"}, {"budget": 0.11, "threshold": -5.734464645385742, "operator": "ge"}, {"budget": 0.12, "threshold": -6.028050422668457, "operator": "ge"}, {"budget": 0.13, "threshold": -6.166539192199707, "operator": "ge"}, {"budget": 0.14, "threshold": -6.3467698097229, "operator": "ge"}, {"budget": 0.15, "threshold": -6.658377647399902, "operator": "ge"}, {"budget": 0.16, "threshold": -6.777430057525635, "operator": "ge"}, {"budget": 0.17, "threshold": -6.857805252075195, "operator": "ge"}, {"budget": 0.18, "threshold": -7.024345397949219, "operator": "ge"}, {"budget": 0.19, "threshold": -7.0804033279418945, "operator": "ge"}, {"budget": 0.2, "threshold": -7.22163200378418, "operator": "ge"}, {"budget": 0.21, "threshold": -7.35360050201416, "operator": "ge"}, {"budget": 0.22, "threshold": -7.500158309936523, "operator": "ge"}, {"budget": 0.23, "threshold": -7.634245872497559, "operator": "ge"}, {"budget": 0.24, "threshold": -7.765318870544434, "operator": "ge"}, {"budget": 0.25, "threshold": -7.876718521118164, "operator": "ge"}, {"budget": 0.26, "threshold": -8.024473190307617, "operator": "ge"}, {"budget": 0.27, "threshold": -8.168396949768066, "operator": "ge"}, {"budget": 0.28, "threshold": -8.302908897399902, "operator": "ge"}, {"budget": 0.29, "threshold": -8.451554298400879, "operator": "ge"}, {"budget": 0.3, "threshold": -8.505966186523438, "operator": "ge"}, {"budget": 0.31, "threshold": -8.628833770751953, "operator": "ge"}, {"budget": 0.32, "threshold": -8.732389450073242, "operator": "ge"}, {"budget": 0.33, "threshold": -8.810497283935547, "operator": "ge"}, {"budget": 0.34, "threshold": -8.88241958618164, "operator": "ge"}, {"budget": 0.35, "threshold": -8.993463516235352, "operator": "ge"}, {"budget": 0.36, "threshold": -9.103425979614258, "operator": "ge"}, {"budget": 0.37, "threshold": -9.179638862609863, "operator": "ge"}, {"budget": 0.38, "threshold": -9.26620101928711, "operator": "ge"}, {"budget": 0.39, "threshold": -9.3756742477417, "operator": "ge"}, {"budget": 0.4, "threshold": -9.450754165649414, "operator": "ge"}, {"budget": 0.41, "threshold": -9.517480850219727, "operator": "ge"}, {"budget": 0.42, "threshold": -9.61479663848877, "operator": "ge"}, {"budget": 0.43, "threshold": -9.67477035522461, "operator": "ge"}, {"budget": 0.44, "threshold": -9.738214492797852, "operator": "ge"}, {"budget": 0.45, "threshold": -9.822309494018555, "operator": "ge"}, {"budget": 0.46, "threshold": -9.898529052734375, "operator": "ge"}, {"budget": 0.47, "threshold": -9.972625732421875, "operator": "ge"}, {"budget": 0.48, "threshold": -10.035598754882812, "operator": "ge"}, {"budget": 0.49, "threshold": -10.08387565612793, "operator": "ge"}, {"budget": 0.5, "threshold": -10.176271438598633, "operator": "ge"}, {"budget": 0.51, "threshold": -10.258859634399414, "operator": "ge"}, {"budget": 0.52, "threshold": -10.315396308898926, "operator": "ge"}, {"budget": 0.53, "threshold": -10.372230529785156, "operator": "ge"}, {"budget": 0.54, "threshold": -10.43293571472168, "operator": "ge"}, {"budget": 0.55, "threshold": -10.511472702026367, "operator": "ge"}, {"budget": 0.56, "threshold": -10.560112953186035, "operator": "ge"}, {"budget": 0.57, "threshold": -10.611003875732422, "operator": "ge"}, {"budget": 0.58, "threshold": -10.677961349487305, "operator": "ge"}, {"budget": 0.59, "threshold": -10.741056442260742, "operator": "ge"}, {"budget": 0.6, "threshold": -10.814672470092773, "operator": "ge"}, {"budget": 0.61, "threshold": -10.885852813720703, "operator": "ge"}, {"budget": 0.62, "threshold": -10.92662239074707, "operator": "ge"}, {"budget": 0.63, "threshold": -10.972253799438477, "operator": "ge"}, {"budget": 0.64, "threshold": -11.032533645629883, "operator": "ge"}, {"budget": 0.65, "threshold": -11.094223022460938, "operator": "ge"}, {"budget": 0.66, "threshold": -11.142768859863281, "operator": "ge"}, {"budget": 0.67, "threshold": -11.194076538085938, "operator": "ge"}, {"budget": 0.68, "threshold": -11.27794361114502, "operator": "ge"}, {"budget": 0.69, "threshold": -11.347893714904785, "operator": "ge"}, {"budget": 0.7, "threshold": -11.387163162231445, "operator": "ge"}, {"budget": 0.71, "threshold": -11.434527397155762, "operator": "ge"}, {"budget": 0.72, "threshold": -11.475875854492188, "operator": "ge"}, {"budget": 0.73, "threshold": -11.499131202697754, "operator": "ge"}, {"budget": 0.74, "threshold": -11.551277160644531, "operator": "ge"}, {"budget": 0.75, "threshold": -11.584932327270508, "operator": "ge"}, {"budget": 0.76, "threshold": -11.628055572509766, "operator": "ge"}, {"budget": 0.77, "threshold": -11.664972305297852, "operator": "ge"}, {"budget": 0.78, "threshold": -11.722628593444824, "operator": "ge"}, {"budget": 0.79, "threshold": -11.753801345825195, "operator": "ge"}, {"budget": 0.8, "threshold": -11.786031723022461, "operator": "ge"}, {"budget": 0.81, "threshold": -11.81893253326416, "operator": "ge"}, {"budget": 0.82, "threshold": -11.855462074279785, "operator": "ge"}, {"budget": 0.83, "threshold": -11.889535903930664, "operator": "ge"}, {"budget": 0.84, "threshold": -11.943818092346191, "operator": "ge"}, {"budget": 0.85, "threshold": -11.98500919342041, "operator": "ge"}, {"budget": 0.86, "threshold": -12.058572769165039, "operator": "ge"}, {"budget": 0.87, "threshold": -12.111164093017578, "operator": "ge"}, {"budget": 0.88, "threshold": -12.182007789611816, "operator": "ge"}, {"budget": 0.89, "threshold": -12.249434471130371, "operator": "ge"}, {"budget": 0.9, "threshold": -12.306083679199219, "operator": "ge"}, {"budget": 0.91, "threshold": -12.377756118774414, "operator": "ge"}, {"budget": 0.92, "threshold": -12.423175811767578, "operator": "ge"}, {"budget": 0.93, "threshold": -12.507503509521484, "operator": "ge"}, {"budget": 0.94, "threshold": -12.583846092224121, "operator": "ge"}, {"budget": 0.95, "threshold": -12.686685562133789, "operator": "ge"}, {"budget": 0.96, "threshold": -12.786934852600098, "operator": "ge"}, {"budget": 0.97, "threshold": -12.918523788452148, "operator": "ge"}, {"budget": 0.98, "threshold": -13.008172988891602, "operator": "ge"}, {"budget": 0.99, "threshold": -13.287321090698242, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.75, 98.25, 97.83333333333333, 97.41666666666666, 96.58333333333333, 95.83333333333333, 95.25, 94.75, 94.08333333333333, 93.66666666666667, 93.16666666666666, 92.66666666666666, 92.33333333333333, 92.0, 91.66666666666666, 91.08333333333333, 90.83333333333333, 90.5, 89.99999999999999, 89.66666666666666, 89.41666666666667, 89.0, 88.83333333333333, 88.08333333333334, 87.66666666666666, 87.25, 86.75, 86.16666666666666, 85.58333333333333, 85.0, 84.66666666666667, 84.33333333333333, 83.66666666666667, 83.16666666666667, 82.66666666666667, 82.16666666666667, 81.41666666666667, 80.83333333333333, 80.33333333333333, 79.66666666666666, 79.08333333333333, 78.49999999999999, 78.0, 77.41666666666667, 77.0, 76.33333333333333, 75.66666666666666, 74.83333333333333, 73.83333333333333, 73.41666666666666, 72.66666666666666, 71.75, 71.0, 70.16666666666667, 69.5, 68.75, 68.25, 67.25, 66.41666666666666, 65.66666666666666, 64.66666666666666, 63.74999999999999, 62.74999999999999, 62.0, 61.16666666666666, 60.16666666666666, 59.416666666666664, 58.5, 57.58333333333333, 56.666666666666664, 55.833333333333336, 54.91666666666666, 53.91666666666666, 52.91666666666666, 51.916666666666664, 50.916666666666664, 49.916666666666664, 48.916666666666664, 48.083333333333336, 47.083333333333336, 46.166666666666664, 45.25, 44.166666666666664, 43.166666666666664, 42.166666666666664, 41.166666666666664, 40.166666666666664, 39.16666666666666, 38.24999999999999, 37.33333333333333, 36.33333333333333, 35.416666666666664, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 3, 10, 16, 21, 31, 40, 47, 53, 61, 66, 72, 78, 82, 86, 90, 97, 100, 104, 110, 114, 117, 122, 124, 133, 138, 143, 149, 156, 163, 170, 174, 178, 186, 192, 198, 204, 213, 220, 226, 234, 241, 248, 254, 261, 266, 274, 282, 292, 304, 309, 318, 329, 338, 348, 356, 365, 371, 383, 393, 402, 414, 425, 437, 446, 456, 468, 477, 488, 499, 510, 520, 531, 543, 555, 567, 579, 591, 603, 613, 625, 636, 647, 660, 672, 684, 696, 708, 720, 731, 742, 754, 765, 775, 787, 798, 810, 822, 833, 845, 857], "exact_score": [98.91666666666666, 98.75, 98.25, 97.83333333333333, 97.41666666666666, 96.58333333333333, 95.83333333333333, 95.25, 94.75, 94.08333333333333, 93.66666666666667, 93.16666666666666, 92.66666666666666, 92.33333333333333, 92.0, 91.66666666666666, 91.08333333333333, 90.83333333333333, 90.5, 89.99999999999999, 89.66666666666666, 89.41666666666667, 89.0, 88.83333333333333, 88.08333333333334, 87.66666666666666, 87.25, 86.75, 86.16666666666666, 85.58333333333333, 85.0, 84.66666666666667, 84.33333333333333, 83.66666666666667, 83.16666666666667, 82.66666666666667, 82.16666666666667, 81.41666666666667, 80.83333333333333, 80.33333333333333, 79.66666666666666, 79.0, 78.49999999999999, 78.0, 77.41666666666667, 77.0, 76.33333333333333, 75.66666666666666, 74.83333333333333, 73.83333333333333, 73.41666666666666, 72.66666666666666, 71.75, 71.0, 70.16666666666667, 69.5, 68.75, 68.16666666666666, 67.25, 66.41666666666666, 65.66666666666666, 64.66666666666666, 63.74999999999999, 62.74999999999999, 62.0, 61.16666666666666, 60.16666666666666, 59.416666666666664, 58.5, 57.49999999999999, 56.666666666666664, 55.833333333333336, 54.91666666666666, 53.91666666666666, 52.91666666666666, 51.916666666666664, 50.916666666666664, 49.916666666666664, 48.916666666666664, 48.083333333333336, 47.083333333333336, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.166666666666664, 42.166666666666664, 41.166666666666664, 40.166666666666664, 39.16666666666666, 38.24999999999999, 37.33333333333333, 36.33333333333333, 35.416666666666664, 34.583333333333336, 33.58333333333333, 32.666666666666664, 31.666666666666664, 30.666666666666664, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 69.22999999999999, "threshold_area100": 69.23333333333333, "threshold_area25": 92.97500000000001}, "test": {"score": [90.16223679668623, 88.98860890576458, 88.09112875388332, 87.36624093890231, 85.76113220573006, 83.25854332067657, 81.34276838108387, 79.73765964791163, 78.23610631687953, 77.20055229547808, 76.7863306869175, 76.38936831204694, 75.26751812219537, 74.83603727994476, 74.38729720400414, 73.54159475319295, 73.28270624784258, 72.97204004142216, 72.33344839489126, 72.21263375906109, 71.66033828098031, 71.19433897134967, 70.69382119433897, 70.22782188470832, 69.81360027614774, 69.43389713496721, 68.88160165688643, 68.31204694511564, 67.62167759751468, 67.03486365205384, 66.75871591301346, 66.18916120124267, 65.75768035899205, 65.41249568519157, 64.99827407663099, 64.49775629962029, 63.94546082153951, 63.4276838108388, 62.909906800138074, 62.323092854677256, 61.8570935450466, 61.460131170176034, 60.92509492578528, 60.56265101829479, 60.04487400759406, 59.38902312737314, 58.974801518812555, 58.56057991025199, 58.00828443217121, 57.749395926820846, 56.851915774939584, 56.1442871936486, 55.69554711770797, 55.19502934069727, 54.79806696582672, 54.124956851915776, 53.71073524335519, 53.31377286848464, 52.53710735243355, 51.81221953745254, 51.17362789092164, 50.48325854332067, 50.08629616845012, 49.72385226095961, 48.929927511218494, 48.066965826717286, 47.46289264756645, 46.807041767345524, 45.90956161546427, 44.99482222989299, 44.45978598550224, 43.838453572661365, 43.28615809458059, 43.02726958923023, 42.336900241629266, 42.00897480151881, 41.31860545391785, 40.74905074214705, 39.74801518812564, 39.31653434587504, 39.023127373144625, 38.62616499827407, 38.298239558163615, 37.91853641698309, 37.33172247152227, 36.84846392820158, 35.985502243700374, 35.41594753192958, 34.82913358646875, 34.173282706247846, 33.62098722816707, 32.82706247842595, 32.430100103555404, 31.774249223334483, 31.135657576803588, 30.376251294442525, 29.720400414221604, 29.150845702450813, 28.61580945806006, 27.424922333448386, 26.56196064894718], "calls": [0.0, 5.091473938557129, 8.267172937521575, 10.355540214014498, 14.445978598550225, 19.399378667587158, 22.402485329651363, 24.92233344839489, 27.77010700724888, 29.68588194684156, 30.376251294442525, 31.152916810493615, 33.24128408698654, 34.10424577148775, 35.19157749395927, 37.141870900932, 37.88401794960304, 38.52260959613393, 39.851570590265794, 40.30031066620642, 41.43942008974801, 42.45771487745944, 43.57956506731101, 44.58060062133241, 45.34000690369348, 46.11667241974457, 47.16948567483604, 48.619261304798066, 49.810148429409736, 50.84570245081118, 51.449775629962026, 52.502588885053505, 53.279254401104595, 53.90058681394546, 54.504659993096304, 55.41939937866759, 56.351397997928885, 57.16258198136003, 57.88746979634104, 58.90576458405247, 59.50983776320331, 60.079392474974114, 60.976872626855375, 61.563686572316186, 62.323092854677256, 63.25509147393855, 63.824646185709355, 64.30790472903003, 65.10182947877115, 65.41249568519157, 66.53434587504314, 67.48360372799448, 68.05315843976527, 68.65723161891611, 69.278564031757, 70.15878494994823, 70.5730065585088, 71.03900586813945, 71.97100448740076, 72.79944770452192, 73.576113220573, 74.38729720400414, 74.80151881256472, 75.2157404211253, 76.09596133931653, 77.06247842595789, 77.77010700724888, 78.46047635484985, 79.46151190887124, 80.44528822920263, 81.01484294097342, 81.67069382119433, 82.27476700034518, 82.55091473938558, 83.34483948912668, 83.74180186399724, 84.44943044528823, 85.0707628581291, 86.10631687953055, 86.5723161891612, 86.93476009665171, 87.36624093890232, 87.69416637901277, 88.09112875388332, 88.67794269934414, 89.2129789437349, 90.12771832930618, 90.69727304107698, 91.3531239212979, 92.04349326889886, 92.59578874697964, 93.45875043148084, 93.8557128063514, 94.58060062133241, 95.25371073524336, 96.03037625129444, 96.68622713151535, 97.29030031066621, 97.859855022437, 99.08526061442872, 100.0], "rescued": [0, 16, 29, 40, 56, 72, 77, 82, 90, 93, 93, 93, 95, 96, 96, 97, 97, 97, 97, 97, 97, 97, 98, 98, 98, 99, 99, 99, 99, 99, 99, 99, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106], "harmed": [0, 84, 149, 202, 311, 472, 588, 686, 781, 844, 868, 891, 958, 984, 1010, 1060, 1075, 1093, 1130, 1137, 1169, 1196, 1226, 1253, 1277, 1300, 1332, 1365, 1405, 1439, 1455, 1488, 1514, 1535, 1559, 1588, 1620, 1650, 1680, 1714, 1741, 1764, 1795, 1817, 1847, 1885, 1909, 1933, 1965, 1980, 2032, 2073, 2099, 2128, 2151, 2190, 2214, 2237, 2282, 2324, 2361, 2401, 2424, 2445, 2491, 2542, 2577, 2615, 2667, 2720, 2752, 2788, 2820, 2835, 2875, 2894, 2934, 2967, 3025, 3050, 3067, 3090, 3109, 3131, 3165, 3194, 3244, 3277, 3311, 3349, 3381, 3427, 3450, 3488, 3525, 3569, 3607, 3640, 3671, 3740, 3791], "exact_score": [90.16223679668623, 90.07594062823611, 89.90334829133586, 89.69623748705557, 89.43734898170521, 89.04038660683466, 88.62616499827408, 88.40179496030376, 88.10838798757334, 87.86675871591301, 87.45253710735244, 87.22816706938211, 86.79668622713152, 86.41698308595099, 85.9337245426303, 85.45046599930963, 85.001725923369, 84.48394891266827, 84.08698653779771, 83.44839489126683, 82.80980324473592, 82.15395236451502, 81.65343458750431, 80.9975837072834, 80.39351052813255, 79.68588194684155, 79.2198826372109, 78.65032792544011, 78.11529168104936, 77.59751467034863, 77.04521919226785, 76.47566448049706, 75.90610976872627, 75.35381429064549, 74.87055574732481, 74.38729720400414, 74.02485329651363, 73.59337245426303, 73.24818778046254, 72.76492923714186, 72.29892992751121, 71.91922678633068, 71.48774594408009, 70.93545046599931, 70.5039696237487, 70.003451846738, 69.48567483603728, 68.96789782533655, 68.58819468415602, 68.05315843976527, 67.50086296168449, 66.94856748360372, 66.46530894028305, 65.96479116327234, 65.34345875043148, 64.7221263375906, 64.23886779426992, 63.58301691404902, 62.85812909906801, 62.23679668622712, 61.51190887124611, 60.92509492578528, 60.303762512944424, 59.578874697963414, 58.83672764929238, 58.077321366931315, 57.2833966171902, 56.47221263375906, 55.73006558508802, 54.98791853641698, 54.26303072143597, 53.365550569554706, 52.53710735243355, 51.63962720055228, 50.828443217121155, 49.930963065239894, 49.03348291335865, 48.11874352778736, 47.25578184328615, 46.34104245771488, 45.40904383845357, 44.47704521919226, 43.51052813255091, 42.613047980669656, 41.76734552985847, 40.83534691059717, 39.86882982395581, 38.97134967207455, 38.022091819123226, 37.03831549879185, 36.123576113220565, 35.15705902657922, 34.22506040731791, 33.224024853296505, 32.30928546772523, 31.360027614773898, 30.410769761822575, 29.426993441491188, 28.477735588539865, 27.511218501898515, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 64.34109613589766, "threshold_area100": 56.12418018639973, "threshold_area25": 77.00897480151882}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.1449775629962, "calls": 1.4670348636520538}}, "old_normalized_interior": {"val": {"score": 99.0, "calls": 1.0}, "test": {"score": 90.11045909561615, "calls": 1.5878494994822232}}, "old_normalized_endpoints": {"val": {"score": 99.0, "calls": 1.0}, "test": {"score": 90.11045909561615, "calls": 1.5878494994822232}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 99.0, 99.0, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.5, 98.25, 98.0, 98.0, 97.58333333333333, 97.5, 97.33333333333333, 97.08333333333333, 97.0, 96.83333333333333, 96.66666666666667, 96.41666666666666, 96.0, 95.75, 95.08333333333333, 94.58333333333333, 94.08333333333333, 93.58333333333333, 92.83333333333333, 92.0, 91.25, 90.5, 89.75, 89.08333333333334, 88.25, 87.41666666666667, 86.66666666666667, 86.0, 85.16666666666666, 84.58333333333333, 83.74999999999999, 82.83333333333334, 81.91666666666666, 81.08333333333333, 80.16666666666666, 79.16666666666666, 78.25, 77.33333333333333, 76.33333333333333, 75.41666666666667, 74.5, 73.58333333333333, 72.66666666666666, 71.75, 70.75, 69.83333333333333, 68.91666666666667, 68.0, 67.08333333333333, 66.16666666666666, 65.25, 64.33333333333333, 63.5, 62.5, 61.66666666666667, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.91666666666666, 56.08333333333333, 55.08333333333333, 54.08333333333333, 53.166666666666664, 52.166666666666664, 51.16666666666667, 50.16666666666667, 49.166666666666664, 48.16666666666667, 47.24999999999999, 46.25, 45.33333333333333, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.583333333333336, 35.66666666666667, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 99.0, 99.0, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.5, 98.25, 98.0, 98.0, 97.58333333333333, 97.5, 97.33333333333333, 97.08333333333333, 97.0, 96.83333333333333, 96.66666666666667, 96.41666666666666, 96.0, 95.75, 95.08333333333333, 94.58333333333333, 94.08333333333333, 93.58333333333333, 92.83333333333333, 92.0, 91.25, 90.5, 89.75, 89.08333333333334, 88.25, 87.41666666666667, 86.66666666666667, 86.0, 85.16666666666666, 84.58333333333333, 83.66666666666667, 82.83333333333334, 81.91666666666666, 81.08333333333333, 80.16666666666666, 79.16666666666666, 78.25, 77.33333333333333, 76.33333333333333, 75.41666666666667, 74.5, 73.58333333333333, 72.66666666666666, 71.75, 70.75, 69.83333333333333, 68.83333333333333, 68.0, 67.08333333333333, 66.16666666666666, 65.25, 64.33333333333333, 63.5, 62.5, 61.66666666666667, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.833333333333336, 56.91666666666666, 56.08333333333333, 55.08333333333333, 54.08333333333333, 53.166666666666664, 52.166666666666664, 51.16666666666667, 50.16666666666667, 49.166666666666664, 48.16666666666667, 47.24999999999999, 46.25, 45.25, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.583333333333336, 35.66666666666667, 34.66666666666667, 33.666666666666664, 32.666666666666664, 31.666666666666664, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.8, "exact_area100": 71.94}, "test": {"score": [90.16223679668623, 90.11045909561615, 90.09319986192614, 90.07594062823611, 90.05868139454608, 90.04142216085606, 90.006903693476, 89.97238522609597, 89.83431135657577, 89.69623748705557, 89.57542285122541, 89.42008974801519, 89.07490507421471, 88.95409043838454, 88.85053503624438, 88.64342423196409, 88.47083189506385, 88.22920262340351, 87.97031411805315, 87.59061097687263, 87.33172247152227, 87.12461166724196, 86.88298239558164, 86.43424231964102, 85.83016914049016, 85.41594753192958, 85.22609596133931, 84.98446668967897, 84.62202278218847, 84.10424577148774, 83.18950638591647, 82.32654470141524, 81.65343458750431, 81.17017604418363, 80.06558508802209, 79.34069727304107, 78.30514325163962, 78.02899551259924, 76.54470141525715, 76.25129444252676, 74.80151881256472, 74.05937176389368, 73.12737314463237, 72.48878149810149, 71.66033828098031, 71.05626510182947, 70.14152571625819, 69.29582326544701, 68.62271315153608, 67.75975146703486, 67.39730755954434, 65.80945806006213, 64.96375560925094, 64.37694166379013, 63.18605453917845, 62.34035208836728, 61.8570935450466, 60.33828098032448, 59.63065239903348, 58.802209181912325, 57.90472903003106, 57.45598895409043, 56.23058336209872, 55.91991715567828, 54.98791853641698, 54.48740075940628, 53.40006903693475, 52.4335519502934, 51.77770107007249, 51.27718329306179, 50.25888850535036, 49.51674145667932, 48.55022437003797, 48.10148429409734, 47.514670348636514, 46.720745598895405, 46.16845012081463, 45.098377632033134, 43.838453572661365, 42.52675181221953, 42.04349326889886, 41.16327235070762, 40.69727304107697, 40.02416292716603, 39.38557128063513, 38.81601656886434, 38.246461857093536, 36.900241629271655, 36.36520538488091, 35.761132205730064, 35.174318260269246, 34.397652744218156, 33.63824646185709, 32.84432171211598, 32.24024853296513, 31.17017604418364, 30.255436658612357, 29.237141870900928, 27.977217811529165, 27.235070762858125, 26.56196064894718], "calls": [0.0, 1.5878494994822232, 2.7096996893337937, 3.8488091128753883, 4.590956161546427, 5.453917846047635, 5.9889540904383844, 6.989989644459786, 9.026579219882636, 9.993096306523992, 11.494649637556092, 12.202278218847082, 14.135312392129789, 15.326199516741458, 15.861235761132205, 16.91404901622368, 17.966862271315154, 19.002416292716603, 20.19330341732827, 21.021746634449432, 21.936486020020713, 22.79944770452192, 23.50707628581291, 24.991370383154987, 26.52744218156714, 27.511218501898515, 27.97721781152917, 28.512254055919918, 29.375215740421122, 30.44528822920262, 31.89506385916465, 33.275802554366585, 34.173282706247846, 34.79461511908871, 36.39972385226096, 37.52157404211253, 39.17846047635485, 39.59268208491543, 41.698308595098375, 42.267863306869174, 44.304452882292026, 45.40904383845357, 46.634449430445294, 47.37659647911633, 48.4294097342078, 49.326889886089056, 50.53503624439075, 51.605108733172244, 52.43355195029341, 53.58992060752502, 54.21125302036589, 56.040731791508456, 57.05902657921988, 57.85295132896099, 59.302726958923024, 60.372799447704516, 60.976872626855375, 62.65101829478771, 63.4967207455989, 64.44597859855023, 65.46427338626165, 65.94753192958233, 67.29375215740421, 67.72523299965482, 68.74352778736625, 69.31308249913704, 70.59026579219882, 71.60856057991025, 72.35070762858129, 73.00655850880221, 74.18018639972385, 75.0776665516051, 76.33759061097687, 76.8553676216776, 77.56299620296859, 78.47773558853987, 79.04729030031066, 80.18639972385226, 81.58439765274423, 83.05143251639628, 83.58646876078703, 84.5012081463583, 85.03624439074905, 85.74387297204004, 86.41698308595099, 87.0037970314118, 87.64238867794269, 89.05764584052469, 89.73075595443562, 90.33482913358647, 90.97342078011737, 91.7846047635485, 92.56127027959958, 93.38971349672074, 94.08008284432171, 95.20193303417328, 96.11667241974456, 97.15222644114601, 98.48118743527787, 99.25785295132896, 100.0], "exact_score": [90.16223679668623, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.07594062823611, 90.04142216085606, 90.006903693476, 89.97238522609597, 89.88608905764583, 89.83431135657577, 89.69623748705557, 89.60994131860545, 89.47186744908525, 89.33379357956507, 89.10942354159475, 88.95409043838454, 88.83327580255437, 88.64342423196409, 88.47083189506385, 88.22920262340351, 88.02209181912323, 87.59061097687263, 87.31446323783224, 87.0728339661719, 86.77942699344149, 86.43424231964102, 86.10631687953054, 85.60579910251984, 85.20883672764928, 84.79461511908872, 84.38039351052812, 83.75906109768727, 83.1377286848464, 82.51639627200552, 81.7915084570245, 81.08387987573352, 80.2899551259924, 79.65136347946151, 78.9955125992406, 78.44321712115982, 77.68381083879875, 77.0797376596479, 76.37210907835691, 75.73351743182603, 74.99137038315499, 74.40455643769417, 73.62789092164309, 72.85122540559199, 72.02278218847083, 71.33241284086986, 70.46945115636866, 69.77908180876769, 68.95063859164652, 68.26026924404556, 67.46634449430445, 66.74145667932343, 65.8612357611322, 65.01553331032102, 64.27338626164997, 63.479461511908866, 62.66827752847773, 61.8570935450466, 60.92509492578528, 60.027614773904034, 59.19917155678287, 58.301691404901625, 57.404211253020364, 56.50673110113911, 55.661028650327914, 54.76354849844667, 53.88332758025543, 53.003106662064205, 52.12288574387297, 51.294442526751816, 50.39696237487055, 49.603037625129446, 48.8608905764584, 48.04970659302727, 47.134967207455986, 46.220227821884706, 45.2882292026234, 44.35623058336209, 43.458750431480844, 42.57852951328961, 41.66379012771833, 40.74905074214705, 39.799792889195714, 38.83327580255437, 37.91853641698309, 36.96927856403175, 36.10631687953054, 35.15705902657922, 34.190541939937866, 33.24128408698653, 32.30928546772523, 31.360027614773898, 30.376251294442525, 29.39247497411115, 28.42595788746979, 27.49395926820849, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 88.67621677597515, "exact_area100": 66.71238794753812}}, "old_normalized_endpoint_budget": 1}, {"seed": 2030, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.5, "validation_area100": 0.7196499999999999, "training_seconds": 3.2634864570572972, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.04478147329272479, "operator": "ge"}, {"budget": 0.02, "threshold": -0.05068092365014242, "operator": "ge"}, {"budget": 0.03, "threshold": -0.0523961036749554, "operator": "ge"}, {"budget": 0.04, "threshold": -0.057204843885131265, "operator": "ge"}, {"budget": 0.05, "threshold": -0.06252126756027837, "operator": "ge"}, {"budget": 0.06, "threshold": -0.07184087299511535, "operator": "ge"}, {"budget": 0.07, "threshold": -0.08210801599331748, "operator": "ge"}, {"budget": 0.08, "threshold": -0.09572997680330346, "operator": "ge"}, {"budget": 0.09, "threshold": -0.11463900391081444, "operator": "ge"}, {"budget": 0.1, "threshold": -0.14246485866057115, "operator": "ge"}, {"budget": 0.11, "threshold": -0.16196518774073476, "operator": "ge"}, {"budget": 0.12, "threshold": -0.18145279003084838, "operator": "ge"}, {"budget": 0.13, "threshold": -0.20042579952435305, "operator": "ge"}, {"budget": 0.14, "threshold": -0.22649845613031958, "operator": "ge"}, {"budget": 0.15, "threshold": -0.2459417956479153, "operator": "ge"}, {"budget": 0.16, "threshold": -0.285500538117901, "operator": "ge"}, {"budget": 0.17, "threshold": -0.3189616000416092, "operator": "ge"}, {"budget": 0.18, "threshold": -0.34936313841668915, "operator": "ge"}, {"budget": 0.19, "threshold": -0.3744764624998728, "operator": "ge"}, {"budget": 0.2, "threshold": -0.38880805309187555, "operator": "ge"}, {"budget": 0.21, "threshold": -0.4135924672860602, "operator": "ge"}, {"budget": 0.22, "threshold": -0.4486865437874066, "operator": "ge"}, {"budget": 0.23, "threshold": -0.5188688105906334, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5579590787385368, "operator": "ge"}, {"budget": 0.25, "threshold": -0.6111669234434558, "operator": "ge"}, {"budget": 0.26, "threshold": -0.661178236936351, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6783221709368793, "operator": "ge"}, {"budget": 0.28, "threshold": -0.7230168531000379, "operator": "ge"}, {"budget": 0.29, "threshold": -0.7488949989752253, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7702518984543609, "operator": "ge"}, {"budget": 0.31, "threshold": -0.7889466466348897, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7988020546403457, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8164654722703896, "operator": "ge"}, {"budget": 0.34, "threshold": -0.8317575808813374, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8472978757218032, "operator": "ge"}, {"budget": 0.36, "threshold": -0.8685389791530961, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8783340428309432, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8852076486581257, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8949958213410694, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9084040058384746, "operator": "ge"}, {"budget": 0.41, "threshold": -0.916397908209261, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9209897550591573, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9274943806197361, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9350771748977121, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9394711088642922, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9440814242777796, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9487389670697233, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9519281457283377, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9553078440539838, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9574712811160043, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9610267547815174, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9627619983825705, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9648908011510701, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9672817576218984, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9695418104825612, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9713255508987482, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9736608827589368, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9757598936947983, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9774594926478841, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9788360159028611, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9798702364222123, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9807042808039473, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9822868260396702, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9837729232325899, "operator": "ge"}, {"budget": 0.65, "threshold": -0.984540361967816, "operator": "ge"}, {"budget": 0.66, "threshold": -0.98550229378345, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9865949512147583, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9873399576325093, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9880783830396663, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9886446706035829, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9898061215298822, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9903107009171624, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9908424024662268, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9914249655508219, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9922684654906364, "operator": "ge"}, {"budget": 0.76, "threshold": -0.992915177481909, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9934536176549618, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9939232626632102, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9944858143214013, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9950764704439605, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9958367810190386, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9961773733405661, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9964327558540864, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9967426912528226, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9969341577227941, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9973360485474861, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9975052354768643, "operator": "ge"}, {"budget": 0.88, "threshold": -0.997709208792698, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9978840084857944, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9980343796815919, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9981662215214895, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9983252165666182, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9984164669568889, "operator": "ge"}, {"budget": 0.94, "threshold": -0.998498192593122, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9986421753143855, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9987481303700046, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9988866928922676, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9990440187880741, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9991837803510262, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 99.0, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.75, 98.5, 98.41666666666666, 98.25, 98.16666666666667, 97.91666666666666, 97.58333333333333, 97.33333333333333, 97.33333333333333, 97.0, 96.75, 96.58333333333333, 96.33333333333333, 96.0, 95.58333333333333, 95.16666666666667, 94.75, 94.0, 93.25, 92.75, 92.16666666666666, 91.58333333333333, 90.75, 89.99999999999999, 89.33333333333333, 88.66666666666666, 87.91666666666667, 86.91666666666666, 86.16666666666666, 85.5, 84.58333333333333, 83.74999999999999, 82.83333333333334, 81.83333333333334, 81.0, 80.16666666666666, 79.25, 78.25, 77.25, 76.33333333333333, 75.41666666666667, 74.5, 73.58333333333333, 72.91666666666666, 71.91666666666666, 70.91666666666666, 70.0, 69.16666666666667, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.25, 64.25, 63.33333333333333, 62.33333333333333, 61.33333333333333, 60.416666666666664, 59.416666666666664, 58.416666666666664, 57.666666666666664, 56.58333333333333, 55.58333333333333, 54.75, 53.91666666666666, 52.91666666666666, 51.916666666666664, 50.916666666666664, 49.99999999999999, 49.25, 48.24999999999999, 47.24999999999999, 46.33333333333333, 45.49999999999999, 44.41666666666666, 43.41666666666666, 42.41666666666667, 41.41666666666667, 40.58333333333333, 39.58333333333333, 38.58333333333333, 37.66666666666666, 36.66666666666666, 35.74999999999999, 34.74999999999999, 33.75, 32.75, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 3, 6, 7, 9, 10, 13, 17, 20, 20, 24, 27, 29, 32, 36, 41, 46, 51, 60, 69, 75, 82, 91, 101, 110, 118, 126, 135, 147, 156, 164, 175, 185, 196, 208, 218, 228, 239, 251, 263, 274, 285, 296, 307, 315, 327, 339, 350, 360, 372, 384, 396, 407, 419, 430, 442, 454, 465, 477, 489, 498, 511, 523, 533, 543, 555, 567, 579, 590, 599, 611, 623, 634, 644, 657, 669, 681, 693, 703, 715, 727, 738, 750, 761, 773, 785, 797, 809, 821, 833, 845, 857], "exact_score": [98.91666666666666, 99.0, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.75, 98.5, 98.41666666666666, 98.25, 98.16666666666667, 97.91666666666666, 97.58333333333333, 97.33333333333333, 97.33333333333333, 97.0, 96.75, 96.58333333333333, 96.33333333333333, 96.0, 95.58333333333333, 95.16666666666667, 94.75, 94.0, 93.25, 92.75, 92.16666666666666, 91.58333333333333, 90.75, 89.99999999999999, 89.33333333333333, 88.66666666666666, 87.91666666666667, 86.91666666666666, 86.16666666666666, 85.5, 84.58333333333333, 83.74999999999999, 82.83333333333334, 81.83333333333334, 81.0, 80.16666666666666, 79.25, 78.25, 77.25, 76.33333333333333, 75.41666666666667, 74.5, 73.58333333333333, 72.91666666666666, 71.91666666666666, 70.91666666666666, 70.0, 69.08333333333333, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.25, 64.25, 63.33333333333333, 62.33333333333333, 61.33333333333333, 60.416666666666664, 59.416666666666664, 58.416666666666664, 57.58333333333333, 56.58333333333333, 55.58333333333333, 54.75, 53.91666666666666, 52.91666666666666, 51.916666666666664, 50.916666666666664, 49.99999999999999, 49.25, 48.24999999999999, 47.24999999999999, 46.33333333333333, 45.41666666666666, 44.41666666666666, 43.41666666666666, 42.41666666666667, 41.41666666666667, 40.58333333333333, 39.58333333333333, 38.58333333333333, 37.66666666666666, 36.66666666666666, 35.74999999999999, 34.74999999999999, 33.75, 32.75, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.96499999999999, "threshold_area100": 71.9675, "threshold_area25": 97.875}, "test": {"score": [90.16223679668623, 90.07594062823611, 90.04142216085606, 90.006903693476, 90.006903693476, 89.95512599240594, 89.90334829133586, 89.92060752502589, 89.8688298239558, 89.78253365550569, 89.59268208491544, 89.45460821539524, 89.3683120469451, 89.28201587849499, 89.14394200897479, 88.98860890576458, 88.64342423196409, 88.47083189506385, 88.2119433897135, 88.0048325854332, 87.74594408008284, 87.48705557473248, 87.12461166724196, 86.48602002071108, 86.20987228167068, 85.77839143942009, 85.13979979288919, 85.01898515705902, 84.1387642388678, 83.69002416292717, 83.06869175008629, 82.4991370383155, 82.17121159820505, 81.41180531584398, 80.721435968243, 79.8584742837418, 78.7193648602002, 78.02899551259924, 77.49395926820849, 76.40662754573697, 75.02588885053503, 74.33551950293406, 73.66240938902313, 72.55781843286158, 71.5050051777701, 70.93545046599931, 70.003451846738, 68.88160165688643, 68.05315843976527, 67.29375215740421, 66.75871591301346, 65.5678287884018, 65.08457024508112, 64.23886779426992, 63.30686917500863, 62.53020365895754, 61.943389713496714, 60.735243355195024, 59.75146703486365, 58.7849499482223, 57.973765964791156, 57.421470486710376, 56.95547117707973, 55.988954090438384, 54.79806696582672, 54.055919917155684, 53.175698998964435, 52.295478080773215, 51.58784949948222, 50.98377632033137, 50.48325854332067, 49.51674145667932, 48.94718674490852, 48.46392820158785, 47.87711425612702, 46.97963410424577, 46.27200552295478, 45.61615464273386, 45.081118398343115, 44.14911977908181, 43.28615809458059, 42.112530203658956, 41.45667932343803, 40.66275457369692, 40.05868139454608, 39.35105281325508, 37.69416637901277, 37.176389368312044, 36.46876078702105, 35.5712806351398, 34.72557818432861, 34.08698653779771, 32.98239558163617, 32.48187780462547, 32.06765619606489, 31.32550914739385, 30.51432516396272, 29.426993441491188, 28.52951328960994, 27.718329306178802, 26.56196064894718], "calls": [0.0, 2.2609596133931653, 3.296513634794615, 3.6934760096651713, 4.452882292026234, 5.453917846047635, 6.731101139109423, 7.62858129099068, 8.66413531239213, 9.751467034863651, 11.632723507076285, 12.875388332758025, 13.669313082499137, 14.325163962720055, 15.602347255781842, 16.46530894028305, 17.70797376596479, 18.77804625474629, 19.882637210907834, 20.849154297549187, 21.505005177770105, 22.419744563341386, 23.62789092164308, 25.52640662754574, 26.337590610976875, 27.459440800828443, 28.564031756989987, 28.99551259924059, 30.54884363134277, 31.429064549534004, 32.63721090783569, 33.63824646185709, 34.190541939937866, 35.41594753192958, 36.5723161891612, 37.953054884363134, 39.83431135657577, 40.85260614428719, 41.62927166033828, 42.94097342078012, 44.9257852951329, 45.995857783914396, 46.97963410424577, 48.2913358646876, 49.70659302726959, 50.5005177770107, 51.70866413531239, 53.141180531584396, 54.17673455298585, 55.2122885743873, 55.86813945460821, 57.128063513979974, 57.749395926820846, 58.715913013462206, 59.803244735933724, 60.75250258888505, 61.54642733862617, 62.85812909906801, 63.89368312046945, 65.08457024508112, 66.03382809803244, 66.67241974456334, 67.17293752157404, 68.29478771142561, 69.7100448740076, 70.5730065585088, 71.52226441146013, 72.50604073179151, 73.24818778046254, 73.93855712806351, 74.47359337245426, 75.62996202968588, 76.28581290990681, 76.76907145322748, 77.40766309975837, 78.37418018639973, 79.15084570245081, 79.82395581636176, 80.4280289955126, 81.41180531584398, 82.37832240248532, 83.67276492923715, 84.39765274421815, 85.19157749395927, 85.8474283741802, 86.5723161891612, 88.40179496030376, 88.93683120469451, 89.69623748705557, 90.66275457369693, 91.59475319295824, 92.23334483948913, 93.38971349672074, 93.89023127373144, 94.32171211598205, 95.09837763203313, 95.9268208491543, 97.0659302726959, 97.9979288919572, 98.80911287538832, 100.0], "rescued": [0, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 6, 6, 9, 9, 9, 11, 12, 13, 13, 13, 18, 21, 22, 28, 31, 32, 36, 39, 43, 44, 44, 46, 47, 49, 51, 53, 54, 56, 59, 62, 65, 67, 69, 71, 74, 74, 74, 78, 79, 79, 81, 81, 81, 82, 84, 84, 85, 87, 88, 89, 90, 91, 93, 95, 95, 95, 95, 96, 96, 99, 100, 100, 100, 100, 100, 100, 100, 100, 101, 102, 103, 103, 103, 103, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106], "harmed": [0, 9, 11, 13, 13, 16, 19, 19, 22, 27, 38, 46, 52, 57, 68, 77, 97, 109, 125, 138, 153, 168, 194, 234, 251, 282, 322, 330, 385, 414, 454, 488, 507, 553, 594, 646, 714, 756, 788, 853, 936, 979, 1021, 1087, 1150, 1185, 1242, 1307, 1355, 1403, 1435, 1504, 1534, 1583, 1637, 1683, 1719, 1789, 1847, 1905, 1953, 1986, 2014, 2071, 2142, 2187, 2238, 2289, 2330, 2366, 2395, 2454, 2488, 2516, 2550, 2602, 2643, 2681, 2712, 2766, 2817, 2886, 2925, 2971, 3006, 3047, 3144, 3174, 3215, 3267, 3317, 3354, 3418, 3447, 3471, 3514, 3561, 3625, 3677, 3724, 3791], "exact_score": [90.16223679668623, 90.07594062823611, 90.07594062823611, 90.04142216085606, 90.006903693476, 89.98964445978598, 89.95512599240594, 89.92060752502589, 89.90334829133586, 89.85157059026578, 89.74801518812563, 89.67897825336554, 89.55816361753539, 89.45460821539524, 89.31653434587504, 89.24749741111494, 89.04038660683466, 88.81601656886434, 88.60890576458405, 88.41905419399379, 88.19468415602347, 87.95305488436314, 87.59061097687263, 87.27994477045219, 86.98653779772178, 86.72764929237141, 86.33068691750086, 85.98550224370037, 85.45046599930963, 85.01898515705902, 84.53572661373835, 83.98343113565757, 83.46565412495684, 82.84432171211597, 82.27476700034518, 81.7224715222644, 81.08387987573352, 80.4280289955126, 79.8584742837418, 79.27166033828098, 78.58129099068, 77.92544011045909, 77.20055229547808, 76.37210907835691, 75.69899896444598, 74.95685191577493, 74.33551950293406, 73.64515015533311, 72.81670693821194, 72.02278218847083, 71.31515360717984, 70.5039696237487, 69.79634104245771, 68.96789782533655, 68.17397307559544, 67.4318260269244, 66.65516051087332, 65.70590265792198, 64.86020020711081, 63.99723852260959, 63.15153607179841, 62.3576113220573, 61.58094580600621, 60.59716948567484, 59.682430100103545, 58.87124611667242, 58.00828443217121, 57.14532274767, 56.28236106316879, 55.36762167759751, 54.52191922678633, 53.676216775975135, 52.76147739040387, 51.84673800483258, 50.9319986192613, 50.03451846738005, 49.18881601656886, 48.274076630997584, 47.34207801173628, 46.41007939247498, 45.443562305833616, 44.54608215395236, 43.59682430100103, 42.7166033828098, 41.85364169830859, 40.86986537797721, 39.920607525025886, 38.97134967207455, 38.073869520193306, 37.12461166724197, 36.175353814290645, 35.32965136347946, 34.32861580945805, 33.37935795650673, 32.378322402485324, 31.429064549534004, 30.445288229202617, 29.478771142561268, 28.52951328960994, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 67.07656403390172, "threshold_area100": 64.83560579910252, "threshold_area25": 88.8505350362444}, "selected_budget": 1, "selected": {"val": {"score": 99.0, "calls": 1.0, "rescued": 1, "harmed": 0}, "test": {"score": 90.07594062823611, "calls": 2.2609596133931653, "rescued": 4, "harmed": 9}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.002133682370185852, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0026514013297855854, "operator": "ge"}, {"budget": 0.03, "threshold": -0.0028856557328253984, "operator": "ge"}, {"budget": 0.04, "threshold": -0.0035428081173449755, "operator": "ge"}, {"budget": 0.05, "threshold": -0.00441834656521678, "operator": "ge"}, {"budget": 0.06, "threshold": -0.005597907118499279, "operator": "ge"}, {"budget": 0.07, "threshold": -0.007639680989086628, "operator": "ge"}, {"budget": 0.08, "threshold": -0.011173984035849571, "operator": "ge"}, {"budget": 0.09, "threshold": -0.015333052724599838, "operator": "ge"}, {"budget": 0.1, "threshold": -0.021439993754029274, "operator": "ge"}, {"budget": 0.11, "threshold": -0.028188617900013924, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03327963501214981, "operator": "ge"}, {"budget": 0.13, "threshold": -0.04435265064239502, "operator": "ge"}, {"budget": 0.14, "threshold": -0.05148825794458389, "operator": "ge"}, {"budget": 0.15, "threshold": -0.06398367136716843, "operator": "ge"}, {"budget": 0.16, "threshold": -0.08434903621673584, "operator": "ge"}, {"budget": 0.17, "threshold": -0.10621272027492523, "operator": "ge"}, {"budget": 0.18, "threshold": -0.1307108849287033, "operator": "ge"}, {"budget": 0.19, "threshold": -0.14854519069194794, "operator": "ge"}, {"budget": 0.2, "threshold": -0.1596769243478775, "operator": "ge"}, {"budget": 0.21, "threshold": -0.18002256751060486, "operator": "ge"}, {"budget": 0.22, "threshold": -0.22165648639202118, "operator": "ge"}, {"budget": 0.23, "threshold": -0.28732776641845703, "operator": "ge"}, {"budget": 0.24, "threshold": -0.3273216485977173, "operator": "ge"}, {"budget": 0.25, "threshold": -0.3771752119064331, "operator": "ge"}, {"budget": 0.26, "threshold": -0.4471833109855652, "operator": "ge"}, {"budget": 0.27, "threshold": -0.4950650930404663, "operator": "ge"}, {"budget": 0.28, "threshold": -0.559723436832428, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5922043919563293, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6270208358764648, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6459426879882812, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6813140511512756, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7036126255989075, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7405718564987183, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7582324147224426, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7715026140213013, "operator": "ge"}, {"budget": 0.37, "threshold": -0.7825652956962585, "operator": "ge"}, {"budget": 0.38, "threshold": -0.7993513941764832, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8233674764633179, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8361178040504456, "operator": "ge"}, {"budget": 0.41, "threshold": -0.8479130268096924, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8584188222885132, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8740447759628296, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8844775557518005, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8913120031356812, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9001078605651855, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9080603718757629, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9134210348129272, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9188818335533142, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9238739609718323, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9277206063270569, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9329373836517334, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9367259740829468, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9411013126373291, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9455429911613464, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9495580792427063, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9525057077407837, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9556589126586914, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9581258893013, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9597184062004089, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9613532423973083, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9639307856559753, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9670090675354004, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9689922332763672, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9706032872200012, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9727604985237122, "operator": "ge"}, {"budget": 0.67, "threshold": -0.974728524684906, "operator": "ge"}, {"budget": 0.68, "threshold": -0.976111888885498, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9772968888282776, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9790754914283752, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9804345965385437, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9816128015518188, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9826175570487976, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9837523102760315, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9855620861053467, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9866762161254883, "operator": "ge"}, {"budget": 0.77, "threshold": -0.987586498260498, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9882983565330505, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9897027611732483, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9911486506462097, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9924576282501221, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9928831458091736, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9935380816459656, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9939517974853516, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9943990707397461, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9949199557304382, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9953208565711975, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9955912828445435, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9959743022918701, "operator": "ge"}, {"budget": 0.9, "threshold": -0.996232807636261, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9963866472244263, "operator": "ge"}, {"budget": 0.92, "threshold": -0.996749222278595, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9969011545181274, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9971569180488586, "operator": "ge"}, {"budget": 0.95, "threshold": -0.997321367263794, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9975613951683044, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9978359341621399, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9981625080108643, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9984052181243896, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.16666666666667, 97.91666666666666, 97.66666666666667, 97.5, 97.41666666666666, 97.08333333333333, 96.91666666666666, 96.75, 96.5, 96.08333333333333, 95.58333333333333, 95.25, 94.75, 94.08333333333333, 93.58333333333333, 93.0, 92.16666666666666, 91.41666666666667, 90.58333333333333, 89.99999999999999, 89.25, 88.5, 87.58333333333333, 86.75, 86.0, 85.41666666666666, 84.5, 83.83333333333333, 82.75, 82.0, 81.16666666666667, 80.16666666666666, 79.16666666666666, 78.16666666666666, 77.25, 76.33333333333333, 75.33333333333333, 74.41666666666666, 73.58333333333333, 72.58333333333333, 71.75, 70.75, 69.83333333333333, 68.91666666666667, 68.0, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.16666666666666, 63.16666666666666, 62.16666666666666, 61.33333333333333, 60.33333333333333, 59.33333333333333, 58.5, 57.58333333333333, 56.49999999999999, 55.58333333333333, 54.833333333333336, 53.833333333333336, 52.83333333333333, 51.916666666666664, 50.916666666666664, 50.08333333333332, 49.166666666666664, 48.24999999999999, 47.24999999999999, 46.33333333333333, 45.41666666666666, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.49999999999999, 40.5, 39.58333333333333, 38.66666666666666, 37.66666666666666, 36.66666666666666, 35.74999999999999, 34.74999999999999, 33.75, 32.75, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 4, 5, 7, 8, 10, 13, 16, 18, 19, 23, 25, 27, 30, 35, 41, 45, 51, 59, 65, 72, 82, 91, 101, 108, 117, 127, 138, 148, 157, 165, 176, 184, 197, 206, 216, 228, 240, 252, 263, 274, 286, 297, 307, 319, 329, 341, 352, 363, 374, 384, 396, 408, 420, 432, 444, 454, 466, 478, 488, 499, 512, 523, 532, 544, 556, 567, 579, 589, 600, 611, 623, 634, 645, 658, 670, 682, 692, 704, 715, 726, 738, 750, 761, 773, 785, 797, 809, 821, 833, 845, 857], "exact_score": [98.91666666666666, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.16666666666667, 97.91666666666666, 97.66666666666667, 97.5, 97.41666666666666, 97.08333333333333, 96.91666666666666, 96.75, 96.5, 96.08333333333333, 95.58333333333333, 95.25, 94.75, 94.08333333333333, 93.58333333333333, 93.0, 92.16666666666666, 91.41666666666667, 90.58333333333333, 89.99999999999999, 89.25, 88.5, 87.58333333333333, 86.75, 86.0, 85.41666666666666, 84.5, 83.74999999999999, 82.75, 82.0, 81.16666666666667, 80.16666666666666, 79.16666666666666, 78.16666666666666, 77.25, 76.33333333333333, 75.33333333333333, 74.41666666666666, 73.58333333333333, 72.58333333333333, 71.75, 70.75, 69.83333333333333, 68.91666666666667, 68.0, 67.16666666666666, 66.16666666666666, 65.16666666666666, 64.16666666666666, 63.16666666666666, 62.16666666666666, 61.33333333333333, 60.33333333333333, 59.33333333333333, 58.5, 57.49999999999999, 56.49999999999999, 55.58333333333333, 54.833333333333336, 53.833333333333336, 52.83333333333333, 51.916666666666664, 50.916666666666664, 50.08333333333332, 49.166666666666664, 48.24999999999999, 47.24999999999999, 46.33333333333333, 45.33333333333333, 44.333333333333336, 43.333333333333336, 42.333333333333336, 41.49999999999999, 40.5, 39.58333333333333, 38.66666666666666, 37.66666666666666, 36.66666666666666, 35.74999999999999, 34.74999999999999, 33.75, 32.75, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 71.945, "threshold_area100": 71.94749999999999, "threshold_area25": 97.89999999999999}, "test": {"score": [90.16223679668623, 90.12771832930618, 90.11045909561615, 90.11045909561615, 90.09319986192614, 90.05868139454608, 90.02416292716603, 90.006903693476, 89.98964445978598, 89.92060752502589, 89.83431135657577, 89.7134967207456, 89.66171901967553, 89.54090438384536, 89.45460821539524, 89.26475664480496, 89.05764584052469, 88.85053503624438, 88.60890576458405, 88.31549879185364, 88.17742492233344, 88.02209181912323, 87.5733517431826, 86.86572316189161, 86.69313082499137, 86.39972385226096, 85.77839143942009, 85.31239212978944, 84.53572661373835, 84.10424577148774, 83.44839489126683, 83.15498791853642, 82.44735933724542, 81.9986192613048, 81.03210217466345, 80.61788056610285, 80.13462202278218, 79.61684501208146, 78.63306869175008, 77.49395926820849, 76.90714532274767, 76.2167759751467, 75.4401104590956, 74.19744563341388, 73.42078011736278, 72.74767000345183, 71.74663444943043, 70.69382119433897, 69.95167414566794, 69.15774939592681, 68.27752847773559, 67.69071453227477, 66.48256817397306, 65.77493959268207, 64.77390403866067, 63.87642388677942, 62.66827752847773, 61.96064894718673, 60.92509492578528, 60.027614773904034, 59.50983776320331, 59.04383845357266, 58.21539523645149, 57.024508111839836, 56.351397997928885, 55.57473248187781, 54.41836382464618, 53.727994477045215, 52.84777355885398, 52.295478080773215, 51.432516396272, 50.70762858129099, 49.96548153261995, 49.3614083534691, 48.68829823955816, 47.75629962029686, 46.94511563686572, 46.46185709354504, 45.99585778391439, 44.68415602347255, 43.614083534691055, 42.388677942699346, 41.80186399723852, 41.00793924749741, 40.36934760096651, 39.38557128063513, 38.31549879185364, 37.55609250949258, 36.91750086296168, 35.709354504659984, 34.91542975491888, 34.483948912668275, 33.24128408698653, 32.79254401104591, 32.13669313082499, 31.687953054884353, 30.721435968243004, 29.720400414221604, 28.564031756989984, 27.787366240938905, 26.56196064894718], "calls": [0.0, 1.8294787711425613, 2.6061442871936484, 2.968588194684156, 3.8315498791853644, 4.849844666896789, 5.816361753538143, 6.679323438039352, 7.8529513289609945, 9.268208491542977, 10.648947186744909, 11.8570935450466, 12.443907490507423, 13.548498446668969, 14.204349326889886, 15.239903348291337, 16.275457369692788, 17.414566793234382, 18.570935450466, 19.589230238177425, 20.124266482568174, 20.900931998619264, 22.59233690024163, 24.439074905074214, 24.991370383154987, 25.733517431826026, 26.907145322747674, 27.890921643079047, 29.237141870900935, 30.151881256472212, 31.17017604418364, 31.705212288574387, 32.844321712115985, 33.620987228167074, 35.41594753192958, 36.106316879530546, 36.882982395581635, 37.6423886779427, 38.93683120469451, 40.59371763893683, 41.45667932343804, 42.474974111149464, 43.51052813255091, 45.15015533310321, 46.25474628926476, 47.04867103900587, 48.32585433206766, 49.810148429409736, 50.77666551605109, 51.77770107007249, 52.89955125992406, 53.58992060752502, 55.09147393855712, 55.95443562305834, 57.21435968243011, 58.439765274421816, 59.88954090438384, 60.71798412150501, 61.94338971349672, 62.9962029685882, 63.600276147739045, 64.16983085950983, 65.0673110113911, 66.48256817397308, 67.3627890921643, 68.22575077666552, 69.55471177079737, 70.36589575422852, 71.31515360717984, 71.95374525371074, 72.97204004142216, 73.83500172592336, 74.64618570935451, 75.31929582326543, 76.19951674145668, 77.28684846392821, 78.25336555056955, 78.80566102865033, 79.32343803935106, 80.721435968243, 81.9295823265447, 83.2930617880566, 83.89713496720745, 84.79461511908872, 85.5022437003797, 86.58957542285123, 87.67690714532274, 88.47083189506387, 89.12668277528478, 90.43838453572661, 91.24956851915775, 91.71556782878841, 93.02726958923024, 93.51052813255092, 94.23541594753193, 94.7186744908526, 95.70245081118398, 96.77252329996548, 97.96341042457715, 98.74007594062823, 100.0], "rescued": [0, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 6, 7, 9, 10, 13, 15, 16, 16, 17, 20, 23, 26, 27, 29, 31, 34, 39, 41, 43, 44, 46, 48, 48, 49, 51, 53, 55, 55, 56, 58, 60, 61, 64, 64, 66, 66, 68, 70, 74, 75, 77, 77, 78, 79, 79, 81, 83, 84, 85, 85, 86, 88, 91, 92, 92, 93, 96, 96, 99, 99, 99, 99, 100, 102, 102, 102, 103, 104, 104, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106], "harmed": [0, 4, 5, 5, 6, 8, 10, 11, 13, 17, 22, 29, 33, 40, 45, 56, 69, 81, 96, 114, 124, 134, 163, 206, 217, 234, 271, 301, 349, 377, 416, 435, 478, 507, 568, 594, 624, 655, 714, 782, 816, 857, 904, 978, 1025, 1064, 1123, 1186, 1231, 1278, 1332, 1366, 1438, 1479, 1539, 1593, 1667, 1709, 1771, 1823, 1854, 1882, 1930, 2001, 2042, 2088, 2156, 2196, 2248, 2282, 2335, 2378, 2421, 2457, 2499, 2553, 2603, 2631, 2658, 2734, 2797, 2870, 2904, 2950, 2988, 3046, 3108, 3152, 3189, 3259, 3305, 3331, 3403, 3429, 3467, 3493, 3549, 3607, 3675, 3720, 3791], "exact_score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.07594062823611, 90.05868139454608, 90.02416292716603, 90.006903693476, 89.98964445978598, 89.92060752502589, 89.92060752502589, 89.81705212288574, 89.69623748705557, 89.59268208491544, 89.48912668277528, 89.35105281325508, 89.10942354159475, 88.93683120469451, 88.74697963410424, 88.574387297204, 88.2119433897135, 87.98757335174318, 87.65964791163272, 87.43527787366241, 86.98653779772178, 86.69313082499137, 86.29616845012082, 85.74387297204004, 85.26061442871936, 84.74283741801864, 84.15602347255782, 83.58646876078701, 82.9996548153262, 82.3610631687953, 81.8605453917846, 81.2909906800138, 80.70417673455297, 80.04832585433206, 79.37521574042113, 78.59855022437003, 77.94269934414912, 77.2178115291681, 76.51018294787711, 75.85433206765619, 75.06040731791508, 74.26648256817397, 73.64515015533311, 72.79944770452191, 71.98826372109079, 71.24611667241973, 70.53848809112876, 69.84811874352779, 68.98515705902656, 68.19123230928547, 67.38004832585432, 66.5688643424232, 65.74042112530203, 64.94649637556091, 64.2216085605799, 63.4276838108388, 62.58198136002762, 61.7190196755264, 60.89057645840524, 60.027614773904034, 59.19917155678287, 58.2844321712116, 57.421470486710376, 56.6448049706593, 55.78184328615809, 54.91888160165688, 54.055919917155684, 53.14118053158439, 52.295478080773215, 51.41525716258197, 50.55229547808078, 49.67207455988954, 48.8608905764584, 47.997928891957194, 47.15222644114601, 46.2892647566448, 45.3572661373835, 44.45978598550224, 43.545046599930956, 42.66482568173973, 41.715567828788394, 40.81808767690715, 39.93786675871591, 38.98860890576458, 38.03935105281325, 37.05557473248188, 36.10631687953054, 35.15705902657922, 34.207801173627885, 33.275802554366585, 32.3610631687953, 31.429064549534004, 30.42802899551259, 29.49603037625129, 28.52951328960994, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 66.79903859455385, "threshold_area100": 65.6020883672765, "threshold_area25": 89.12323092854677}, "selected_budget": 1, "selected": {"val": {"score": 99.0, "calls": 1.0, "rescued": 1, "harmed": 0}, "test": {"score": 90.12771832930618, "calls": 1.8294787711425613, "rescued": 2, "harmed": 4}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.7466963529586792, "operator": "ge"}, {"budget": 0.02, "threshold": -2.9823641777038574, "operator": "ge"}, {"budget": 0.03, "threshold": -3.9431333541870117, "operator": "ge"}, {"budget": 0.04, "threshold": -4.670774459838867, "operator": "ge"}, {"budget": 0.05, "threshold": -5.033541202545166, "operator": "ge"}, {"budget": 0.06, "threshold": -5.540318965911865, "operator": "ge"}, {"budget": 0.07, "threshold": -5.853982925415039, "operator": "ge"}, {"budget": 0.08, "threshold": -6.192191123962402, "operator": "ge"}, {"budget": 0.09, "threshold": -6.463028907775879, "operator": "ge"}, {"budget": 0.1, "threshold": -6.632527828216553, "operator": "ge"}, {"budget": 0.11, "threshold": -6.83101224899292, "operator": "ge"}, {"budget": 0.12, "threshold": -6.985649585723877, "operator": "ge"}, {"budget": 0.13, "threshold": -7.210473537445068, "operator": "ge"}, {"budget": 0.14, "threshold": -7.306093692779541, "operator": "ge"}, {"budget": 0.15, "threshold": -7.419886112213135, "operator": "ge"}, {"budget": 0.16, "threshold": -7.558352470397949, "operator": "ge"}, {"budget": 0.17, "threshold": -7.697333812713623, "operator": "ge"}, {"budget": 0.18, "threshold": -7.867805004119873, "operator": "ge"}, {"budget": 0.19, "threshold": -7.974212646484375, "operator": "ge"}, {"budget": 0.2, "threshold": -8.121742248535156, "operator": "ge"}, {"budget": 0.21, "threshold": -8.291380882263184, "operator": "ge"}, {"budget": 0.22, "threshold": -8.391226768493652, "operator": "ge"}, {"budget": 0.23, "threshold": -8.495975494384766, "operator": "ge"}, {"budget": 0.24, "threshold": -8.686285972595215, "operator": "ge"}, {"budget": 0.25, "threshold": -8.856130599975586, "operator": "ge"}, {"budget": 0.26, "threshold": -9.024881362915039, "operator": "ge"}, {"budget": 0.27, "threshold": -9.138200759887695, "operator": "ge"}, {"budget": 0.28, "threshold": -9.243741989135742, "operator": "ge"}, {"budget": 0.29, "threshold": -9.357499122619629, "operator": "ge"}, {"budget": 0.3, "threshold": -9.494813919067383, "operator": "ge"}, {"budget": 0.31, "threshold": -9.606766700744629, "operator": "ge"}, {"budget": 0.32, "threshold": -9.709871292114258, "operator": "ge"}, {"budget": 0.33, "threshold": -9.797466278076172, "operator": "ge"}, {"budget": 0.34, "threshold": -9.922933578491211, "operator": "ge"}, {"budget": 0.35, "threshold": -9.998470306396484, "operator": "ge"}, {"budget": 0.36, "threshold": -10.075271606445312, "operator": "ge"}, {"budget": 0.37, "threshold": -10.224421501159668, "operator": "ge"}, {"budget": 0.38, "threshold": -10.328468322753906, "operator": "ge"}, {"budget": 0.39, "threshold": -10.390913009643555, "operator": "ge"}, {"budget": 0.4, "threshold": -10.449725151062012, "operator": "ge"}, {"budget": 0.41, "threshold": -10.544357299804688, "operator": "ge"}, {"budget": 0.42, "threshold": -10.647470474243164, "operator": "ge"}, {"budget": 0.43, "threshold": -10.747682571411133, "operator": "ge"}, {"budget": 0.44, "threshold": -10.79786491394043, "operator": "ge"}, {"budget": 0.45, "threshold": -10.918907165527344, "operator": "ge"}, {"budget": 0.46, "threshold": -11.026434898376465, "operator": "ge"}, {"budget": 0.47, "threshold": -11.094552993774414, "operator": "ge"}, {"budget": 0.48, "threshold": -11.165736198425293, "operator": "ge"}, {"budget": 0.49, "threshold": -11.242456436157227, "operator": "ge"}, {"budget": 0.5, "threshold": -11.312527656555176, "operator": "ge"}, {"budget": 0.51, "threshold": -11.398773193359375, "operator": "ge"}, {"budget": 0.52, "threshold": -11.465500831604004, "operator": "ge"}, {"budget": 0.53, "threshold": -11.585896492004395, "operator": "ge"}, {"budget": 0.54, "threshold": -11.68155288696289, "operator": "ge"}, {"budget": 0.55, "threshold": -11.77769947052002, "operator": "ge"}, {"budget": 0.56, "threshold": -11.819149017333984, "operator": "ge"}, {"budget": 0.57, "threshold": -11.873289108276367, "operator": "ge"}, {"budget": 0.58, "threshold": -11.97386360168457, "operator": "ge"}, {"budget": 0.59, "threshold": -12.053121566772461, "operator": "ge"}, {"budget": 0.6, "threshold": -12.12523078918457, "operator": "ge"}, {"budget": 0.61, "threshold": -12.221805572509766, "operator": "ge"}, {"budget": 0.62, "threshold": -12.282096862792969, "operator": "ge"}, {"budget": 0.63, "threshold": -12.325925827026367, "operator": "ge"}, {"budget": 0.64, "threshold": -12.376802444458008, "operator": "ge"}, {"budget": 0.65, "threshold": -12.436524391174316, "operator": "ge"}, {"budget": 0.66, "threshold": -12.496448516845703, "operator": "ge"}, {"budget": 0.67, "threshold": -12.532455444335938, "operator": "ge"}, {"budget": 0.68, "threshold": -12.577371597290039, "operator": "ge"}, {"budget": 0.69, "threshold": -12.613489151000977, "operator": "ge"}, {"budget": 0.7, "threshold": -12.681403160095215, "operator": "ge"}, {"budget": 0.71, "threshold": -12.732278823852539, "operator": "ge"}, {"budget": 0.72, "threshold": -12.817058563232422, "operator": "ge"}, {"budget": 0.73, "threshold": -12.874828338623047, "operator": "ge"}, {"budget": 0.74, "threshold": -12.970401763916016, "operator": "ge"}, {"budget": 0.75, "threshold": -13.005510330200195, "operator": "ge"}, {"budget": 0.76, "threshold": -13.06768798828125, "operator": "ge"}, {"budget": 0.77, "threshold": -13.104894638061523, "operator": "ge"}, {"budget": 0.78, "threshold": -13.157855987548828, "operator": "ge"}, {"budget": 0.79, "threshold": -13.189412117004395, "operator": "ge"}, {"budget": 0.8, "threshold": -13.262258529663086, "operator": "ge"}, {"budget": 0.81, "threshold": -13.2996826171875, "operator": "ge"}, {"budget": 0.82, "threshold": -13.364753723144531, "operator": "ge"}, {"budget": 0.83, "threshold": -13.431428909301758, "operator": "ge"}, {"budget": 0.84, "threshold": -13.471513748168945, "operator": "ge"}, {"budget": 0.85, "threshold": -13.506872177124023, "operator": "ge"}, {"budget": 0.86, "threshold": -13.568277359008789, "operator": "ge"}, {"budget": 0.87, "threshold": -13.610849380493164, "operator": "ge"}, {"budget": 0.88, "threshold": -13.654694557189941, "operator": "ge"}, {"budget": 0.89, "threshold": -13.714288711547852, "operator": "ge"}, {"budget": 0.9, "threshold": -13.764939308166504, "operator": "ge"}, {"budget": 0.91, "threshold": -13.829647064208984, "operator": "ge"}, {"budget": 0.92, "threshold": -13.912370681762695, "operator": "ge"}, {"budget": 0.93, "threshold": -13.956294059753418, "operator": "ge"}, {"budget": 0.94, "threshold": -14.03543472290039, "operator": "ge"}, {"budget": 0.95, "threshold": -14.223987579345703, "operator": "ge"}, {"budget": 0.96, "threshold": -14.304283142089844, "operator": "ge"}, {"budget": 0.97, "threshold": -14.447372436523438, "operator": "ge"}, {"budget": 0.98, "threshold": -14.61169147491455, "operator": "ge"}, {"budget": 0.99, "threshold": -14.763550758361816, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.25, 97.91666666666666, 97.24999999999999, 96.66666666666667, 96.0, 95.58333333333333, 95.08333333333333, 94.83333333333333, 94.41666666666666, 94.08333333333333, 93.75, 93.33333333333333, 93.08333333333333, 92.66666666666666, 92.5, 92.25, 91.91666666666667, 91.58333333333333, 91.33333333333333, 91.16666666666666, 90.83333333333333, 90.08333333333333, 89.58333333333333, 89.16666666666666, 88.58333333333333, 88.16666666666666, 87.66666666666666, 87.33333333333333, 86.75, 86.08333333333333, 85.83333333333333, 85.5, 84.83333333333333, 84.16666666666667, 83.66666666666667, 82.91666666666666, 82.25, 81.66666666666667, 81.25, 80.5, 79.5, 78.49999999999999, 77.83333333333333, 77.25, 76.5, 75.66666666666666, 75.08333333333333, 74.33333333333333, 73.58333333333333, 72.83333333333333, 72.16666666666667, 71.24999999999999, 70.5, 69.83333333333333, 69.0, 68.16666666666666, 67.16666666666666, 66.5, 65.5, 64.58333333333333, 63.916666666666664, 63.083333333333336, 62.16666666666666, 61.33333333333333, 60.33333333333333, 59.416666666666664, 58.5, 57.666666666666664, 56.58333333333333, 55.58333333333333, 54.666666666666664, 53.666666666666664, 52.83333333333333, 51.916666666666664, 50.916666666666664, 49.916666666666664, 49.0, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.25, 44.166666666666664, 43.166666666666664, 42.166666666666664, 41.166666666666664, 40.33333333333333, 39.33333333333333, 38.33333333333333, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.416666666666664, 33.5, 32.49999999999999, 31.499999999999993, 30.499999999999993, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 2, 9, 15, 23, 30, 38, 43, 49, 52, 57, 61, 65, 70, 73, 78, 80, 83, 87, 91, 94, 96, 100, 109, 115, 120, 127, 132, 138, 142, 149, 157, 160, 164, 172, 180, 186, 195, 203, 210, 215, 224, 236, 248, 256, 263, 272, 282, 289, 298, 307, 316, 324, 335, 344, 352, 362, 372, 384, 392, 404, 415, 423, 433, 444, 454, 466, 477, 488, 498, 511, 523, 534, 546, 556, 567, 579, 591, 602, 613, 624, 636, 647, 660, 672, 684, 696, 706, 718, 730, 741, 753, 765, 777, 788, 800, 812, 824, 833, 845, 857], "exact_score": [98.91666666666666, 98.83333333333333, 98.25, 97.91666666666666, 97.24999999999999, 96.66666666666667, 96.0, 95.58333333333333, 95.08333333333333, 94.83333333333333, 94.41666666666666, 94.08333333333333, 93.75, 93.33333333333333, 93.08333333333333, 92.66666666666666, 92.5, 92.25, 91.91666666666667, 91.58333333333333, 91.33333333333333, 91.16666666666666, 90.83333333333333, 90.08333333333333, 89.58333333333333, 89.16666666666666, 88.58333333333333, 88.16666666666666, 87.66666666666666, 87.33333333333333, 86.75, 86.08333333333333, 85.83333333333333, 85.5, 84.83333333333333, 84.16666666666667, 83.66666666666667, 82.91666666666666, 82.25, 81.66666666666667, 81.25, 80.41666666666667, 79.5, 78.49999999999999, 77.83333333333333, 77.25, 76.5, 75.66666666666666, 75.08333333333333, 74.33333333333333, 73.58333333333333, 72.83333333333333, 72.16666666666667, 71.24999999999999, 70.5, 69.83333333333333, 69.0, 68.08333333333333, 67.16666666666666, 66.5, 65.5, 64.58333333333333, 63.916666666666664, 63.083333333333336, 62.16666666666666, 61.33333333333333, 60.33333333333333, 59.416666666666664, 58.5, 57.58333333333333, 56.58333333333333, 55.58333333333333, 54.666666666666664, 53.666666666666664, 52.83333333333333, 51.916666666666664, 50.916666666666664, 49.916666666666664, 49.0, 48.083333333333336, 47.16666666666667, 46.166666666666664, 45.166666666666664, 44.166666666666664, 43.166666666666664, 42.166666666666664, 41.166666666666664, 40.33333333333333, 39.33333333333333, 38.33333333333333, 37.416666666666664, 36.416666666666664, 35.416666666666664, 34.416666666666664, 33.5, 32.49999999999999, 31.499999999999993, 30.499999999999993, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 69.75083333333332, "threshold_area100": 69.75416666666665, "threshold_area25": 93.82499999999999}, "test": {"score": [90.16223679668623, 89.55816361753539, 88.78149810148429, 87.72868484639281, 86.26164998274076, 85.10528132550914, 83.44839489126683, 82.32654470141524, 80.82499137038315, 79.87573351743183, 79.20262340352087, 78.51225405591991, 77.90818087676908, 77.0797376596479, 76.76907145322747, 76.32033137728685, 75.99240593717639, 75.45736969278563, 74.88781498101484, 74.38729720400414, 73.78322402485328, 73.1446323783224, 72.97204004142216, 72.66137383500173, 72.02278218847083, 71.41870900931998, 70.72833966171902, 70.36589575422852, 69.69278564031757, 69.00241629271659, 68.39834311356576, 67.91508457024507, 67.50086296168449, 67.17293752157404, 66.67241974456334, 66.30997583707283, 65.94753192958233, 65.17086641353124, 64.68760787021056, 64.41146013117017, 64.0662754573697, 63.53123921297894, 62.73731446323782, 62.37487055574732, 62.133241284086985, 61.49464963755609, 60.80428028995512, 60.26924404556438, 59.63065239903348, 59.06109768726269, 58.387987573351744, 57.83569209527097, 57.2833966171902, 56.45495340006903, 55.91991715567828, 54.936140835346905, 54.60821539523645, 54.19399378667586, 53.15843976527441, 52.41629271660337, 51.86399723852261, 51.03555402140145, 50.44874007594062, 50.17259233690024, 49.58577839143942, 48.981705212288574, 48.343113565757676, 48.03244735933724, 47.68726268553676, 47.221263375906105, 46.513634794615115, 45.99585778391439, 44.90852606144286, 44.14911977908181, 43.2171211598205, 42.7166033828098, 42.12978943734898, 41.47393855712806, 40.90438384535726, 40.38660683465653, 39.506385916465305, 39.023127373144625, 38.12564722126337, 37.5043148084225, 37.09009319986192, 36.624093890231265, 35.985502243700374, 35.51950293406973, 35.174318260269246, 34.60476354849844, 34.052468070417675, 33.34483948912668, 32.61995167414567, 32.41284086986538, 31.67069382119434, 30.479806696582667, 29.73765964791163, 28.598550224370033, 27.68381083879875, 27.200552295478076, 26.56196064894718], "calls": [0.0, 3.0721435968243007, 6.38591646530894, 10.303762512944425, 13.910942354159475, 16.137383500172593, 19.399378667587158, 21.884708318950636, 24.54263030721436, 26.30307214359682, 27.804625474628924, 29.375215740421122, 30.84225060407318, 32.61995167414567, 33.39661719019675, 34.397652744218156, 35.51950293406973, 36.62409389023127, 38.09112875388333, 39.33379357956507, 40.50742147048671, 41.7846047635485, 42.25060407317915, 43.06178805661028, 44.54608215395236, 45.66793234380393, 46.94511563686572, 47.58370728339661, 48.74007594062824, 49.86192613047981, 51.08733172247152, 51.950293406972726, 52.62340352088367, 53.17569899896445, 53.98688298239558, 54.504659993096304, 54.98791853641698, 56.16154642733863, 56.80013807386952, 57.14532274767, 57.6458405246807, 58.439765274421816, 59.45806006213324, 60.027614773904034, 60.372799447704516, 61.2875388332758, 62.271315153607176, 62.96168450120815, 63.72109078356921, 64.46323783224025, 65.30894028305143, 66.0683465654125, 66.7069382119434, 67.69071453227477, 68.4328615809458, 69.5719710044874, 70.02071108042803, 70.53848809112876, 71.76389368312047, 72.66137383500173, 73.29996548153262, 74.19744563341388, 74.80151881256472, 75.09492578529513, 75.75077666551606, 76.423886779427, 77.09699689333793, 77.45944080082843, 77.89092164307905, 78.37418018639973, 79.15084570245081, 79.70314118053159, 80.8595098377632, 81.65343458750431, 82.65447014152572, 83.24128408698654, 83.87987573351742, 84.63928201587849, 85.24335519502934, 85.76113220573006, 86.69313082499137, 87.19364860200207, 88.09112875388332, 88.74697963410425, 89.16120124266482, 89.6444597859855, 90.31756989989644, 90.81808767690714, 91.18053158439766, 91.76734552985847, 92.31964100793924, 93.04452882292026, 93.76941663790129, 93.97652744218156, 94.73593372454263, 95.97859855022438, 96.73800483258543, 97.91163272350708, 98.82637210907835, 99.34414911977908, 100.0], "rescued": [0, 11, 33, 51, 57, 62, 67, 74, 78, 81, 83, 86, 88, 92, 92, 92, 93, 94, 94, 96, 96, 96, 96, 98, 98, 99, 99, 100, 100, 100, 100, 100, 101, 101, 101, 101, 101, 101, 101, 101, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 104, 104, 104, 104, 104, 104, 104, 104, 104, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106], "harmed": [0, 46, 113, 192, 283, 355, 456, 528, 619, 677, 718, 761, 798, 850, 868, 894, 914, 946, 979, 1010, 1045, 1082, 1092, 1112, 1149, 1185, 1225, 1247, 1286, 1326, 1361, 1389, 1414, 1433, 1462, 1483, 1504, 1549, 1577, 1593, 1613, 1645, 1691, 1712, 1726, 1763, 1803, 1834, 1871, 1904, 1943, 1975, 2007, 2055, 2086, 2143, 2163, 2187, 2247, 2290, 2322, 2370, 2404, 2420, 2454, 2489, 2526, 2544, 2565, 2592, 2633, 2663, 2726, 2770, 2824, 2853, 2887, 2926, 2959, 2989, 3041, 3069, 3121, 3157, 3181, 3208, 3245, 3272, 3292, 3325, 3357, 3398, 3440, 3452, 3495, 3564, 3607, 3673, 3726, 3754, 3791], "exact_score": [90.16223679668623, 90.04142216085606, 89.83431135657577, 89.59268208491544, 89.38557128063513, 89.23023817742491, 88.97134967207455, 88.64342423196409, 88.40179496030376, 88.09112875388332, 87.79772178115292, 87.55609250949257, 87.12461166724196, 86.69313082499137, 86.19261304798067, 85.72661373835001, 85.13979979288919, 84.60476354849844, 84.03520883672765, 83.6382464618571, 83.27580255436658, 82.7925440110459, 82.27476700034518, 81.70521228857439, 81.10113910942354, 80.51432516396272, 80.08284432171212, 79.51328960994132, 79.09906800138073, 78.66758715913014, 78.27062478425958, 77.80462547462892, 77.32136693130825, 76.97618225750776, 76.51018294787711, 76.09596133931653, 75.80255436658612, 75.31929582326543, 74.90507421470487, 74.59440800828443, 74.04211253020365, 73.48981705212289, 73.04107697618225, 72.66137383500173, 72.26441146013116, 71.79841215050051, 71.28063513979978, 70.69382119433897, 70.15878494994821, 69.55471177079737, 68.91612012426648, 68.45012081463582, 67.89782533655504, 67.27649292371419, 66.67241974456334, 65.94753192958233, 65.22264411460131, 64.53227476700035, 63.80738695201933, 63.099758370728345, 62.40938902312737, 61.7190196755264, 61.011391094235414, 60.234725578184324, 59.40628236106317, 58.62961684501208, 57.90472903003106, 57.07628581290991, 56.265101829478766, 55.47117707973766, 54.64273386261649, 53.79703141180532, 52.98584742837418, 52.14014497756298, 51.242664825681736, 50.27614773904039, 49.3614083534691, 48.44666896789782, 47.58370728339661, 46.66896789782533, 45.736969278564025, 44.77045219192268, 43.838453572661365, 42.94097342078012, 42.04349326889886, 41.1460131170176, 40.16223679668622, 39.230238177424916, 38.229202623403516, 37.26268553676216, 36.29616845012082, 35.346910597169476, 34.38039351052813, 33.39661719019675, 32.39558163617535, 31.429064549534004, 30.46254746289264, 29.478771142561268, 28.512254055919918, 27.511218501898515, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 65.03820802327381, "threshold_area100": 57.42794269934416, "threshold_area25": 79.04038660683467}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.6152602434158325, "operator": "ge"}, {"budget": 0.02, "threshold": -1.6284151077270508, "operator": "ge"}, {"budget": 0.03, "threshold": -2.06864595413208, "operator": "ge"}, {"budget": 0.04, "threshold": -2.662316083908081, "operator": "ge"}, {"budget": 0.05, "threshold": -3.1937084197998047, "operator": "ge"}, {"budget": 0.06, "threshold": -3.9299674034118652, "operator": "ge"}, {"budget": 0.07, "threshold": -4.345799446105957, "operator": "ge"}, {"budget": 0.08, "threshold": -4.836969375610352, "operator": "ge"}, {"budget": 0.09, "threshold": -5.229514122009277, "operator": "ge"}, {"budget": 0.1, "threshold": -5.448423385620117, "operator": "ge"}, {"budget": 0.11, "threshold": -5.738457679748535, "operator": "ge"}, {"budget": 0.12, "threshold": -5.989415645599365, "operator": "ge"}, {"budget": 0.13, "threshold": -6.131404876708984, "operator": "ge"}, {"budget": 0.14, "threshold": -6.30983829498291, "operator": "ge"}, {"budget": 0.15, "threshold": -6.559885025024414, "operator": "ge"}, {"budget": 0.16, "threshold": -6.659660339355469, "operator": "ge"}, {"budget": 0.17, "threshold": -6.809648513793945, "operator": "ge"}, {"budget": 0.18, "threshold": -6.942375659942627, "operator": "ge"}, {"budget": 0.19, "threshold": -7.07661247253418, "operator": "ge"}, {"budget": 0.2, "threshold": -7.211508750915527, "operator": "ge"}, {"budget": 0.21, "threshold": -7.362095832824707, "operator": "ge"}, {"budget": 0.22, "threshold": -7.514638900756836, "operator": "ge"}, {"budget": 0.23, "threshold": -7.651154041290283, "operator": "ge"}, {"budget": 0.24, "threshold": -7.762887954711914, "operator": "ge"}, {"budget": 0.25, "threshold": -7.928164958953857, "operator": "ge"}, {"budget": 0.26, "threshold": -8.208758354187012, "operator": "ge"}, {"budget": 0.27, "threshold": -8.317952156066895, "operator": "ge"}, {"budget": 0.28, "threshold": -8.429122924804688, "operator": "ge"}, {"budget": 0.29, "threshold": -8.594034194946289, "operator": "ge"}, {"budget": 0.3, "threshold": -8.70992374420166, "operator": "ge"}, {"budget": 0.31, "threshold": -8.890748023986816, "operator": "ge"}, {"budget": 0.32, "threshold": -8.955741882324219, "operator": "ge"}, {"budget": 0.33, "threshold": -9.02018928527832, "operator": "ge"}, {"budget": 0.34, "threshold": -9.07969856262207, "operator": "ge"}, {"budget": 0.35, "threshold": -9.19747543334961, "operator": "ge"}, {"budget": 0.36, "threshold": -9.30573844909668, "operator": "ge"}, {"budget": 0.37, "threshold": -9.47726821899414, "operator": "ge"}, {"budget": 0.38, "threshold": -9.628190994262695, "operator": "ge"}, {"budget": 0.39, "threshold": -9.733383178710938, "operator": "ge"}, {"budget": 0.4, "threshold": -9.82680892944336, "operator": "ge"}, {"budget": 0.41, "threshold": -9.898916244506836, "operator": "ge"}, {"budget": 0.42, "threshold": -10.069674491882324, "operator": "ge"}, {"budget": 0.43, "threshold": -10.122800827026367, "operator": "ge"}, {"budget": 0.44, "threshold": -10.198413848876953, "operator": "ge"}, {"budget": 0.45, "threshold": -10.297887802124023, "operator": "ge"}, {"budget": 0.46, "threshold": -10.385493278503418, "operator": "ge"}, {"budget": 0.47, "threshold": -10.427077293395996, "operator": "ge"}, {"budget": 0.48, "threshold": -10.54481029510498, "operator": "ge"}, {"budget": 0.49, "threshold": -10.630060195922852, "operator": "ge"}, {"budget": 0.5, "threshold": -10.762897491455078, "operator": "ge"}, {"budget": 0.51, "threshold": -10.807727813720703, "operator": "ge"}, {"budget": 0.52, "threshold": -10.900025367736816, "operator": "ge"}, {"budget": 0.53, "threshold": -10.989446640014648, "operator": "ge"}, {"budget": 0.54, "threshold": -11.053470611572266, "operator": "ge"}, {"budget": 0.55, "threshold": -11.12564468383789, "operator": "ge"}, {"budget": 0.56, "threshold": -11.181075096130371, "operator": "ge"}, {"budget": 0.57, "threshold": -11.217009544372559, "operator": "ge"}, {"budget": 0.58, "threshold": -11.294578552246094, "operator": "ge"}, {"budget": 0.59, "threshold": -11.363370895385742, "operator": "ge"}, {"budget": 0.6, "threshold": -11.443817138671875, "operator": "ge"}, {"budget": 0.61, "threshold": -11.508207321166992, "operator": "ge"}, {"budget": 0.62, "threshold": -11.564644813537598, "operator": "ge"}, {"budget": 0.63, "threshold": -11.639480590820312, "operator": "ge"}, {"budget": 0.64, "threshold": -11.696155548095703, "operator": "ge"}, {"budget": 0.65, "threshold": -11.768087387084961, "operator": "ge"}, {"budget": 0.66, "threshold": -11.80080795288086, "operator": "ge"}, {"budget": 0.67, "threshold": -11.861387252807617, "operator": "ge"}, {"budget": 0.68, "threshold": -11.924256324768066, "operator": "ge"}, {"budget": 0.69, "threshold": -12.003341674804688, "operator": "ge"}, {"budget": 0.7, "threshold": -12.068948745727539, "operator": "ge"}, {"budget": 0.71, "threshold": -12.123649597167969, "operator": "ge"}, {"budget": 0.72, "threshold": -12.198553085327148, "operator": "ge"}, {"budget": 0.73, "threshold": -12.2374849319458, "operator": "ge"}, {"budget": 0.74, "threshold": -12.285113334655762, "operator": "ge"}, {"budget": 0.75, "threshold": -12.330913543701172, "operator": "ge"}, {"budget": 0.76, "threshold": -12.408227920532227, "operator": "ge"}, {"budget": 0.77, "threshold": -12.447820663452148, "operator": "ge"}, {"budget": 0.78, "threshold": -12.481817245483398, "operator": "ge"}, {"budget": 0.79, "threshold": -12.537659645080566, "operator": "ge"}, {"budget": 0.8, "threshold": -12.565202713012695, "operator": "ge"}, {"budget": 0.81, "threshold": -12.630536079406738, "operator": "ge"}, {"budget": 0.82, "threshold": -12.663246154785156, "operator": "ge"}, {"budget": 0.83, "threshold": -12.72298526763916, "operator": "ge"}, {"budget": 0.84, "threshold": -12.790492057800293, "operator": "ge"}, {"budget": 0.85, "threshold": -12.863876342773438, "operator": "ge"}, {"budget": 0.86, "threshold": -12.910736083984375, "operator": "ge"}, {"budget": 0.87, "threshold": -12.957991600036621, "operator": "ge"}, {"budget": 0.88, "threshold": -12.990530967712402, "operator": "ge"}, {"budget": 0.89, "threshold": -13.021265983581543, "operator": "ge"}, {"budget": 0.9, "threshold": -13.052850723266602, "operator": "ge"}, {"budget": 0.91, "threshold": -13.16306209564209, "operator": "ge"}, {"budget": 0.92, "threshold": -13.204496383666992, "operator": "ge"}, {"budget": 0.93, "threshold": -13.270824432373047, "operator": "ge"}, {"budget": 0.94, "threshold": -13.34498405456543, "operator": "ge"}, {"budget": 0.95, "threshold": -13.41617202758789, "operator": "ge"}, {"budget": 0.96, "threshold": -13.536554336547852, "operator": "ge"}, {"budget": 0.97, "threshold": -13.626371383666992, "operator": "ge"}, {"budget": 0.98, "threshold": -13.718172073364258, "operator": "ge"}, {"budget": 0.99, "threshold": -13.88601303100586, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.25, 97.74999999999999, 97.16666666666667, 96.41666666666666, 95.75, 95.08333333333333, 94.75, 94.41666666666666, 93.83333333333333, 93.41666666666666, 93.0, 92.66666666666666, 92.08333333333333, 91.91666666666667, 91.75, 91.49999999999999, 91.33333333333333, 90.99999999999999, 90.75, 90.5, 90.08333333333333, 89.75, 89.33333333333333, 88.83333333333333, 88.25, 87.83333333333333, 87.08333333333333, 86.41666666666666, 86.0, 85.41666666666666, 84.91666666666666, 84.41666666666666, 83.91666666666666, 83.66666666666667, 82.83333333333334, 82.41666666666666, 82.0, 81.33333333333333, 80.66666666666666, 80.16666666666666, 79.16666666666666, 78.49999999999999, 77.83333333333333, 77.0, 76.33333333333333, 75.58333333333334, 75.0, 74.33333333333333, 73.66666666666666, 72.75, 71.91666666666666, 71.16666666666667, 70.5, 69.58333333333333, 68.75, 68.16666666666666, 67.16666666666666, 66.33333333333333, 65.66666666666666, 64.75, 63.916666666666664, 62.999999999999986, 62.16666666666666, 61.16666666666666, 60.25, 59.25, 58.33333333333333, 57.41666666666667, 56.49999999999999, 55.49999999999999, 54.58333333333333, 53.583333333333336, 52.75, 51.74999999999999, 50.83333333333333, 49.916666666666664, 48.916666666666664, 48.0, 47.0, 46.0, 45.166666666666664, 44.08333333333333, 43.166666666666664, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 1, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3], "harmed": [0, 2, 9, 16, 24, 33, 41, 49, 53, 57, 64, 69, 74, 78, 85, 87, 89, 92, 94, 98, 101, 104, 109, 113, 118, 124, 131, 136, 145, 153, 158, 165, 171, 177, 183, 186, 196, 201, 206, 214, 222, 228, 240, 248, 256, 266, 274, 283, 290, 298, 306, 317, 327, 336, 344, 355, 365, 372, 384, 394, 402, 413, 423, 434, 444, 456, 467, 479, 490, 501, 512, 524, 535, 547, 557, 569, 580, 591, 603, 614, 626, 638, 648, 661, 672, 683, 694, 705, 717, 729, 741, 753, 764, 776, 787, 799, 811, 822, 834, 845, 857], "exact_score": [98.91666666666666, 98.83333333333333, 98.25, 97.74999999999999, 97.16666666666667, 96.41666666666666, 95.75, 95.08333333333333, 94.75, 94.41666666666666, 93.83333333333333, 93.41666666666666, 93.0, 92.66666666666666, 92.08333333333333, 91.91666666666667, 91.75, 91.49999999999999, 91.33333333333333, 90.99999999999999, 90.75, 90.5, 90.08333333333333, 89.75, 89.33333333333333, 88.83333333333333, 88.25, 87.83333333333333, 87.08333333333333, 86.41666666666666, 86.0, 85.41666666666666, 84.91666666666666, 84.41666666666666, 83.91666666666666, 83.66666666666667, 82.83333333333334, 82.41666666666666, 82.0, 81.33333333333333, 80.66666666666666, 80.08333333333333, 79.16666666666666, 78.49999999999999, 77.83333333333333, 77.0, 76.33333333333333, 75.58333333333334, 75.0, 74.33333333333333, 73.66666666666666, 72.75, 71.91666666666666, 71.16666666666667, 70.5, 69.58333333333333, 68.75, 68.08333333333333, 67.16666666666666, 66.33333333333333, 65.66666666666666, 64.75, 63.916666666666664, 62.999999999999986, 62.16666666666666, 61.16666666666666, 60.25, 59.25, 58.33333333333333, 57.33333333333333, 56.49999999999999, 55.49999999999999, 54.58333333333333, 53.583333333333336, 52.75, 51.74999999999999, 50.83333333333333, 49.916666666666664, 48.916666666666664, 48.0, 47.0, 46.0, 45.08333333333333, 44.08333333333333, 43.166666666666664, 42.25, 41.333333333333336, 40.416666666666664, 39.416666666666664, 38.416666666666664, 37.416666666666664, 36.416666666666664, 35.5, 34.5, 33.58333333333333, 32.58333333333333, 31.58333333333333, 30.666666666666664, 29.666666666666664, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 69.50416666666666, "threshold_area100": 69.5075, "threshold_area25": 93.35833333333333}, "test": {"score": [90.16223679668623, 89.54090438384536, 88.36727649292371, 87.55609250949257, 86.0717984121505, 84.15602347255782, 81.87780462547462, 80.39351052813255, 78.80566102865032, 77.59751467034863, 77.01070072488781, 76.26855367621677, 75.75077666551606, 75.19848118743528, 74.76700034518467, 74.0075940628236, 73.80048325854331, 73.40352088367277, 73.12737314463237, 72.67863306869174, 72.29892992751121, 71.81567138419054, 71.33241284086986, 70.98722816706939, 70.62478425957887, 70.02071108042801, 69.26130479806696, 68.9333793579565, 68.48463928201586, 67.75975146703486, 67.3627890921643, 66.55160510873317, 66.30997583707283, 66.03382809803244, 65.91301346220227, 65.48153261995166, 64.9292371418709, 64.11805315843976, 63.37590610976872, 62.85812909906801, 62.37487055574732, 61.99516741456679, 61.09768726268554, 60.82153952364515, 60.42457714877458, 60.04487400759406, 59.47531929582326, 59.26820849154297, 58.681394546082146, 58.12909906800138, 57.09354504659993, 56.69658267172937, 55.97169485674836, 55.33310321021746, 54.86710390058681, 54.29754918881602, 53.79703141180532, 53.503624439074905, 52.83051432516396, 52.26095961339317, 51.55333103210218, 51.17362789092164, 50.81118398343113, 50.27614773904039, 49.74111149464964, 48.94718674490852, 48.55022437003797, 47.94615119088712, 47.221263375906105, 46.2892647566448, 45.667932343803926, 44.99482222989299, 43.87297204004142, 43.320676561960646, 42.63030721435968, 42.04349326889886, 41.04245771487746, 40.50742147048671, 40.075940628236104, 39.42008974801519, 39.05764584052468, 38.36727649292371, 37.953054884363134, 37.245426303072136, 36.64135312392129, 35.84742837418018, 35.10528132550914, 34.63928201587849, 34.32861580945805, 34.03520883672765, 33.67276492923714, 32.63721090783569, 32.084915429754915, 31.342768381083875, 30.721435968243004, 30.186399723852254, 29.288919571971007, 28.7193648602002, 28.063513979979284, 27.37314463237832, 26.56196064894718], "calls": [0.0, 3.244735933724543, 7.369692785640318, 9.90680013807387, 13.548498446668969, 17.15567828788402, 21.574042112530204, 24.197445633413878, 26.907145322747674, 28.926475664480495, 30.082844321712116, 31.61891612012427, 32.861580945806004, 34.0697273041077, 34.984466689678975, 36.91750086296168, 37.59061097687263, 38.574387297204005, 39.54090438384536, 40.559199171556784, 41.560234725578184, 42.75112185018985, 43.9247497411115, 44.61511908871246, 45.495340006903696, 46.565412495685194, 48.06696582671729, 48.74007594062824, 49.44770452191923, 50.75940628236106, 51.605108733172244, 53.072143596824304, 53.48636520538488, 53.917846047635486, 54.193993786675875, 54.95340006903694, 55.6437694166379, 56.86917500862961, 57.92198826372109, 58.68139454608215, 59.38902312737314, 59.872281670693816, 61.09768726268554, 61.39109423541594, 61.977908180876774, 62.633759061097685, 63.53123921297894, 63.80738695201933, 64.67034863652053, 65.34345875043148, 66.68967897825337, 67.15567828788402, 67.94960303762512, 68.79530548843631, 69.36486020020712, 70.0724887814981, 70.83189506385916, 71.19433897134967, 71.95374525371074, 72.60959613393166, 73.47255781843286, 73.9212978943735, 74.33551950293406, 74.9395926820849, 75.61270279599586, 76.5792198826372, 76.99344149119779, 77.61477390403866, 78.46047635484985, 79.40973420780118, 80.04832585433206, 80.77321366931308, 81.98136002761477, 82.58543320676563, 83.27580255436658, 83.89713496720745, 85.001725923369, 85.5712806351398, 86.03727994477045, 86.7103900586814, 87.0728339661719, 87.79772178115292, 88.2119433897135, 88.95409043838454, 89.59268208491544, 90.42112530203659, 91.19779081808768, 91.69830859509838, 92.00897480151882, 92.30238177424923, 92.68208491542975, 93.7866758715913, 94.33897134967208, 95.09837763203313, 95.73696927856403, 96.27200552295479, 97.16948567483604, 97.75629962029686, 98.44666896789782, 99.17155678287884, 100.0], "rescued": [0, 8, 28, 37, 48, 61, 78, 83, 86, 88, 91, 94, 94, 94, 95, 95, 95, 96, 96, 96, 96, 96, 96, 96, 97, 97, 98, 100, 100, 101, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 102, 103, 103, 103, 103, 104, 104, 104, 104, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 105, 106, 106, 106, 106, 106, 106, 106, 106, 106, 106], "harmed": [0, 44, 132, 188, 285, 409, 558, 649, 744, 816, 853, 899, 929, 961, 987, 1031, 1043, 1067, 1083, 1109, 1131, 1159, 1187, 1207, 1229, 1264, 1309, 1330, 1356, 1399, 1423, 1470, 1484, 1500, 1507, 1532, 1564, 1611, 1654, 1684, 1712, 1734, 1786, 1802, 1825, 1847, 1880, 1892, 1926, 1958, 2018, 2041, 2083, 2120, 2147, 2180, 2209, 2226, 2265, 2298, 2340, 2362, 2383, 2414, 2446, 2492, 2515, 2550, 2593, 2647, 2683, 2722, 2787, 2819, 2859, 2893, 2951, 2982, 3007, 3045, 3066, 3106, 3130, 3171, 3206, 3252, 3295, 3322, 3340, 3357, 3378, 3439, 3471, 3514, 3550, 3581, 3633, 3666, 3704, 3744, 3791], "exact_score": [90.16223679668623, 90.07594062823611, 89.8688298239558, 89.59268208491544, 89.31653434587504, 89.07490507421471, 88.79875733517432, 88.52260959613393, 88.19468415602347, 87.93579565067311, 87.5043148084225, 87.05557473248187, 86.74490852606144, 86.33068691750086, 85.79565067311012, 85.20883672764928, 84.79461511908872, 84.24231964100794, 83.72454263030721, 83.31032102174663, 82.67172937521575, 82.17121159820505, 81.65343458750431, 80.96306523990334, 80.46254746289264, 79.91025198481188, 79.3579565067311, 78.73662409389023, 78.09803244735933, 77.56299620296858, 77.06247842595788, 76.56196064894718, 76.14773904038661, 75.716258198136, 75.26751812219537, 74.76700034518467, 74.35277873662409, 73.99033482913359, 73.61063168795306, 73.33448394891266, 72.95478080773213, 72.52329996548153, 72.12633759061097, 71.71211598205039, 71.31515360717984, 70.79737659647911, 70.34863652053849, 69.76182257507766, 69.27856403175699, 68.76078702105626, 68.13945460821539, 67.65619606489471, 67.2074559889541, 66.60338280980325, 66.01656886434242, 65.44701415257163, 64.7221263375906, 64.03175698998965, 63.324128408698655, 62.633759061097685, 61.90887124611667, 61.183983431135644, 60.42457714877458, 59.82050396962375, 59.13013462202278, 58.42250604073178, 57.628581290990674, 56.86917500862961, 55.95443562305833, 55.160510873317214, 54.38384535726614, 53.658957542285115, 52.795995857783915, 51.91577493959267, 51.104590956161545, 50.22437003797031, 49.46496375560925, 48.55022437003797, 47.60096651708664, 46.68622713151536, 45.719710044874006, 44.82222989299275, 43.85571280635139, 42.906454953400065, 41.95719710044873, 41.059716948567484, 40.12771832930617, 39.1439420089748, 38.177424922333444, 37.21090783569208, 36.261649982740764, 35.31239212978944, 34.34587504314808, 33.39661719019675, 32.430100103555404, 31.446323783224017, 30.46254746289264, 29.46151190887124, 28.477735588539865, 27.545736969278558, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 64.58704617360345, "threshold_area100": 56.4555574732482, "threshold_area25": 77.51294442526752}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 99.0, "calls": 1.0}, "test": {"score": 90.12771832930618, "calls": 1.8294787711425613}}, "old_normalized_interior": {"val": {"score": 99.0, "calls": 1.0}, "test": {"score": 90.07594062823611, "calls": 2.2609596133931653}}, "old_normalized_endpoints": {"val": {"score": 99.0, "calls": 1.0}, "test": {"score": 90.07594062823611, "calls": 2.2609596133931653}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 99.0, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.75, 98.5, 98.41666666666666, 98.25, 98.16666666666667, 97.91666666666666, 97.58333333333333, 97.33333333333333, 97.33333333333333, 97.0, 96.75, 96.58333333333333, 96.33333333333333, 96.0, 95.58333333333333, 95.16666666666667, 94.75, 94.0, 93.25, 92.75, 92.16666666666666, 91.58333333333333, 90.75, 89.99999999999999, 89.33333333333333, 88.66666666666666, 87.91666666666667, 86.91666666666666, 86.16666666666666, 85.5, 84.58333333333333, 83.74999999999999, 82.83333333333334, 81.83333333333334, 81.0, 80.16666666666666, 79.25, 78.25, 77.25, 76.33333333333333, 75.41666666666667, 74.5, 73.58333333333333, 72.91666666666666, 71.91666666666666, 70.91666666666666, 70.0, 69.16666666666667, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.25, 64.25, 63.33333333333333, 62.33333333333333, 61.33333333333333, 60.416666666666664, 59.416666666666664, 58.416666666666664, 57.666666666666664, 56.58333333333333, 55.58333333333333, 54.75, 53.91666666666666, 52.91666666666666, 51.916666666666664, 50.916666666666664, 49.99999999999999, 49.25, 48.24999999999999, 47.24999999999999, 46.33333333333333, 45.49999999999999, 44.41666666666666, 43.41666666666666, 42.41666666666667, 41.41666666666667, 40.58333333333333, 39.58333333333333, 38.58333333333333, 37.66666666666666, 36.66666666666666, 35.74999999999999, 34.74999999999999, 33.75, 32.75, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 99.0, 99.0, 99.0, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.75, 98.5, 98.41666666666666, 98.25, 98.16666666666667, 97.91666666666666, 97.58333333333333, 97.33333333333333, 97.33333333333333, 97.0, 96.75, 96.58333333333333, 96.33333333333333, 96.0, 95.58333333333333, 95.16666666666667, 94.75, 94.0, 93.25, 92.75, 92.16666666666666, 91.58333333333333, 90.75, 89.99999999999999, 89.33333333333333, 88.66666666666666, 87.91666666666667, 86.91666666666666, 86.16666666666666, 85.5, 84.58333333333333, 83.74999999999999, 82.83333333333334, 81.83333333333334, 81.0, 80.16666666666666, 79.25, 78.25, 77.25, 76.33333333333333, 75.41666666666667, 74.5, 73.58333333333333, 72.91666666666666, 71.91666666666666, 70.91666666666666, 70.0, 69.08333333333333, 68.16666666666666, 67.16666666666666, 66.16666666666666, 65.25, 64.25, 63.33333333333333, 62.33333333333333, 61.33333333333333, 60.416666666666664, 59.416666666666664, 58.416666666666664, 57.58333333333333, 56.58333333333333, 55.58333333333333, 54.75, 53.91666666666666, 52.91666666666666, 51.916666666666664, 50.916666666666664, 49.99999999999999, 49.25, 48.24999999999999, 47.24999999999999, 46.33333333333333, 45.41666666666666, 44.41666666666666, 43.41666666666666, 42.41666666666667, 41.41666666666667, 40.58333333333333, 39.58333333333333, 38.58333333333333, 37.66666666666666, 36.66666666666666, 35.74999999999999, 34.74999999999999, 33.75, 32.75, 31.75, 30.75, 29.75, 28.749999999999996, 27.749999999999996], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 97.875, "exact_area100": 71.96499999999999}, "test": {"score": [90.16223679668623, 90.07594062823611, 90.04142216085606, 90.006903693476, 90.006903693476, 89.95512599240594, 89.90334829133586, 89.92060752502589, 89.8688298239558, 89.78253365550569, 89.59268208491544, 89.45460821539524, 89.3683120469451, 89.28201587849499, 89.14394200897479, 88.98860890576458, 88.64342423196409, 88.47083189506385, 88.2119433897135, 88.0048325854332, 87.74594408008284, 87.48705557473248, 87.12461166724196, 86.48602002071108, 86.20987228167068, 85.77839143942009, 85.13979979288919, 85.01898515705902, 84.1387642388678, 83.69002416292717, 83.06869175008629, 82.4991370383155, 82.17121159820505, 81.41180531584398, 80.721435968243, 79.8584742837418, 78.7193648602002, 78.02899551259924, 77.49395926820849, 76.40662754573697, 75.02588885053503, 74.33551950293406, 73.66240938902313, 72.55781843286158, 71.5050051777701, 70.93545046599931, 70.003451846738, 68.88160165688643, 68.05315843976527, 67.29375215740421, 66.75871591301346, 65.5678287884018, 65.08457024508112, 64.23886779426992, 63.30686917500863, 62.53020365895754, 61.943389713496714, 60.735243355195024, 59.75146703486365, 58.7849499482223, 57.973765964791156, 57.421470486710376, 56.95547117707973, 55.988954090438384, 54.79806696582672, 54.055919917155684, 53.175698998964435, 52.295478080773215, 51.58784949948222, 50.98377632033137, 50.48325854332067, 49.51674145667932, 48.94718674490852, 48.46392820158785, 47.87711425612702, 46.97963410424577, 46.27200552295478, 45.61615464273386, 45.081118398343115, 44.14911977908181, 43.28615809458059, 42.112530203658956, 41.45667932343803, 40.66275457369692, 40.05868139454608, 39.35105281325508, 37.69416637901277, 37.176389368312044, 36.46876078702105, 35.5712806351398, 34.72557818432861, 34.08698653779771, 32.98239558163617, 32.48187780462547, 32.06765619606489, 31.32550914739385, 30.51432516396272, 29.426993441491188, 28.52951328960994, 27.718329306178802, 26.56196064894718], "calls": [0.0, 2.2609596133931653, 3.296513634794615, 3.6934760096651713, 4.452882292026234, 5.453917846047635, 6.731101139109423, 7.62858129099068, 8.66413531239213, 9.751467034863651, 11.632723507076285, 12.875388332758025, 13.669313082499137, 14.325163962720055, 15.602347255781842, 16.46530894028305, 17.70797376596479, 18.77804625474629, 19.882637210907834, 20.849154297549187, 21.505005177770105, 22.419744563341386, 23.62789092164308, 25.52640662754574, 26.337590610976875, 27.459440800828443, 28.564031756989987, 28.99551259924059, 30.54884363134277, 31.429064549534004, 32.63721090783569, 33.63824646185709, 34.190541939937866, 35.41594753192958, 36.5723161891612, 37.953054884363134, 39.83431135657577, 40.85260614428719, 41.62927166033828, 42.94097342078012, 44.9257852951329, 45.995857783914396, 46.97963410424577, 48.2913358646876, 49.70659302726959, 50.5005177770107, 51.70866413531239, 53.141180531584396, 54.17673455298585, 55.2122885743873, 55.86813945460821, 57.128063513979974, 57.749395926820846, 58.715913013462206, 59.803244735933724, 60.75250258888505, 61.54642733862617, 62.85812909906801, 63.89368312046945, 65.08457024508112, 66.03382809803244, 66.67241974456334, 67.17293752157404, 68.29478771142561, 69.7100448740076, 70.5730065585088, 71.52226441146013, 72.50604073179151, 73.24818778046254, 73.93855712806351, 74.47359337245426, 75.62996202968588, 76.28581290990681, 76.76907145322748, 77.40766309975837, 78.37418018639973, 79.15084570245081, 79.82395581636176, 80.4280289955126, 81.41180531584398, 82.37832240248532, 83.67276492923715, 84.39765274421815, 85.19157749395927, 85.8474283741802, 86.5723161891612, 88.40179496030376, 88.93683120469451, 89.69623748705557, 90.66275457369693, 91.59475319295824, 92.23334483948913, 93.38971349672074, 93.89023127373144, 94.32171211598205, 95.09837763203313, 95.9268208491543, 97.0659302726959, 97.9979288919572, 98.80911287538832, 100.0], "exact_score": [90.16223679668623, 90.07594062823611, 90.07594062823611, 90.04142216085606, 90.006903693476, 89.98964445978598, 89.95512599240594, 89.92060752502589, 89.90334829133586, 89.85157059026578, 89.74801518812563, 89.67897825336554, 89.55816361753539, 89.45460821539524, 89.31653434587504, 89.24749741111494, 89.04038660683466, 88.81601656886434, 88.60890576458405, 88.41905419399379, 88.19468415602347, 87.95305488436314, 87.59061097687263, 87.27994477045219, 86.98653779772178, 86.72764929237141, 86.33068691750086, 85.98550224370037, 85.45046599930963, 85.01898515705902, 84.53572661373835, 83.98343113565757, 83.46565412495684, 82.84432171211597, 82.27476700034518, 81.7224715222644, 81.08387987573352, 80.4280289955126, 79.8584742837418, 79.27166033828098, 78.58129099068, 77.92544011045909, 77.20055229547808, 76.37210907835691, 75.69899896444598, 74.95685191577493, 74.33551950293406, 73.64515015533311, 72.81670693821194, 72.02278218847083, 71.31515360717984, 70.5039696237487, 69.79634104245771, 68.96789782533655, 68.17397307559544, 67.4318260269244, 66.65516051087332, 65.70590265792198, 64.86020020711081, 63.99723852260959, 63.15153607179841, 62.3576113220573, 61.58094580600621, 60.59716948567484, 59.682430100103545, 58.87124611667242, 58.00828443217121, 57.14532274767, 56.28236106316879, 55.36762167759751, 54.52191922678633, 53.676216775975135, 52.76147739040387, 51.84673800483258, 50.9319986192613, 50.03451846738005, 49.18881601656886, 48.274076630997584, 47.34207801173628, 46.41007939247498, 45.443562305833616, 44.54608215395236, 43.59682430100103, 42.7166033828098, 41.85364169830859, 40.86986537797721, 39.920607525025886, 38.97134967207455, 38.073869520193306, 37.12461166724197, 36.175353814290645, 35.32965136347946, 34.32861580945805, 33.37935795650673, 32.378322402485324, 31.429064549534004, 30.445288229202617, 29.478771142561268, 28.52951328960994, 27.52847773558854, 26.56196064894718], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 88.8505350362444, "exact_area100": 67.07656403390172}}, "old_normalized_endpoint_budget": 1}]}, {"expert": "glsim", "endpoint": "Qwen3.8", "task": "cub", "small": 90.16223679668623, "large": 54.81532619951675, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.0, "validation_area100": 0.8650249999999999, "training_seconds": 3.986694528022781, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.002319731516763568, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0030661060009151697, "operator": "ge"}, {"budget": 0.03, "threshold": -0.003583932761102915, "operator": "ge"}, {"budget": 0.04, "threshold": -0.004538949113339186, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0055152541026473045, "operator": "ge"}, {"budget": 0.06, "threshold": -0.006949907634407282, "operator": "ge"}, {"budget": 0.07, "threshold": -0.007858322001993656, "operator": "ge"}, {"budget": 0.08, "threshold": -0.009692218154668808, "operator": "ge"}, {"budget": 0.09, "threshold": -0.011516870930790901, "operator": "ge"}, {"budget": 0.1, "threshold": -0.013871230185031891, "operator": "ge"}, {"budget": 0.11, "threshold": -0.015995511785149574, "operator": "ge"}, {"budget": 0.12, "threshold": -0.02032342739403248, "operator": "ge"}, {"budget": 0.13, "threshold": -0.024061406031250954, "operator": "ge"}, {"budget": 0.14, "threshold": -0.028295140713453293, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03263888508081436, "operator": "ge"}, {"budget": 0.16, "threshold": -0.03858954459428787, "operator": "ge"}, {"budget": 0.17, "threshold": -0.04196999967098236, "operator": "ge"}, {"budget": 0.18, "threshold": -0.0454777367413044, "operator": "ge"}, {"budget": 0.19, "threshold": -0.050907235592603683, "operator": "ge"}, {"budget": 0.2, "threshold": -0.057471565902233124, "operator": "ge"}, {"budget": 0.21, "threshold": -0.06376554071903229, "operator": "ge"}, {"budget": 0.22, "threshold": -0.0696253627538681, "operator": "ge"}, {"budget": 0.23, "threshold": -0.0735263004899025, "operator": "ge"}, {"budget": 0.24, "threshold": -0.08060379326343536, "operator": "ge"}, {"budget": 0.25, "threshold": -0.08624498546123505, "operator": "ge"}, {"budget": 0.26, "threshold": -0.09741979092359543, "operator": "ge"}, {"budget": 0.27, "threshold": -0.10490947216749191, "operator": "ge"}, {"budget": 0.28, "threshold": -0.11138848960399628, "operator": "ge"}, {"budget": 0.29, "threshold": -0.1218261644244194, "operator": "ge"}, {"budget": 0.3, "threshold": -0.1322326958179474, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1393558233976364, "operator": "ge"}, {"budget": 0.32, "threshold": -0.14585353434085846, "operator": "ge"}, {"budget": 0.33, "threshold": -0.1572570502758026, "operator": "ge"}, {"budget": 0.34, "threshold": -0.16415879130363464, "operator": "ge"}, {"budget": 0.35, "threshold": -0.17227540910243988, "operator": "ge"}, {"budget": 0.36, "threshold": -0.18772833049297333, "operator": "ge"}, {"budget": 0.37, "threshold": -0.19866502285003662, "operator": "ge"}, {"budget": 0.38, "threshold": -0.21087077260017395, "operator": "ge"}, {"budget": 0.39, "threshold": -0.22245928645133972, "operator": "ge"}, {"budget": 0.4, "threshold": -0.23491227626800537, "operator": "ge"}, {"budget": 0.41, "threshold": -0.24656431376934052, "operator": "ge"}, {"budget": 0.42, "threshold": -0.2697896361351013, "operator": "ge"}, {"budget": 0.43, "threshold": -0.29196593165397644, "operator": "ge"}, {"budget": 0.44, "threshold": -0.30500420928001404, "operator": "ge"}, {"budget": 0.45, "threshold": -0.3130740225315094, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3248336911201477, "operator": "ge"}, {"budget": 0.47, "threshold": -0.33853960037231445, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3494745194911957, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3742058277130127, "operator": "ge"}, {"budget": 0.5, "threshold": -0.38880011439323425, "operator": "ge"}, {"budget": 0.51, "threshold": -0.41233548521995544, "operator": "ge"}, {"budget": 0.52, "threshold": -0.432985782623291, "operator": "ge"}, {"budget": 0.53, "threshold": -0.4523588716983795, "operator": "ge"}, {"budget": 0.54, "threshold": -0.48142823576927185, "operator": "ge"}, {"budget": 0.55, "threshold": -0.5001511573791504, "operator": "ge"}, {"budget": 0.56, "threshold": -0.5124437808990479, "operator": "ge"}, {"budget": 0.57, "threshold": -0.5299684405326843, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5399276614189148, "operator": "ge"}, {"budget": 0.59, "threshold": -0.566520094871521, "operator": "ge"}, {"budget": 0.6, "threshold": -0.578437864780426, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5925268530845642, "operator": "ge"}, {"budget": 0.62, "threshold": -0.6135192513465881, "operator": "ge"}, {"budget": 0.63, "threshold": -0.62940913438797, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6467357277870178, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6679899096488953, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6791021823883057, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6878600120544434, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6997919678688049, "operator": "ge"}, {"budget": 0.69, "threshold": -0.7079272866249084, "operator": "ge"}, {"budget": 0.7, "threshold": -0.73135906457901, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7492585778236389, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7601076364517212, "operator": "ge"}, {"budget": 0.73, "threshold": -0.77741539478302, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7923457026481628, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8104785084724426, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8259429335594177, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8364963531494141, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8440443873405457, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8525980114936829, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8632519245147705, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8728476762771606, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8835811018943787, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8969945311546326, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9033701419830322, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9107188582420349, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9239863157272339, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9310542345046997, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9378506541252136, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9459103941917419, "operator": "ge"}, {"budget": 0.9, "threshold": -0.952576220035553, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9579152464866638, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9636959433555603, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9706143736839294, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9746701121330261, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9787147045135498, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9810968041419983, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9831351041793823, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9849997758865356, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9867321848869324, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.25, 98.25, 97.91666666666666, 97.91666666666666, 97.91666666666666, 97.91666666666666, 97.58333333333333, 97.58333333333333, 97.5, 97.33333333333333, 97.16666666666667, 97.08333333333333, 96.91666666666666, 96.66666666666667, 96.5, 96.33333333333333, 96.0, 95.66666666666667, 95.5, 95.33333333333333, 95.16666666666667, 94.83333333333333, 94.66666666666667, 94.41666666666666, 94.16666666666667, 93.75, 93.5, 93.41666666666666, 93.16666666666666, 92.83333333333333, 92.41666666666666, 92.25, 91.83333333333333, 91.41666666666667, 90.83333333333333, 90.16666666666666, 89.66666666666666, 88.91666666666667, 88.33333333333333, 87.75, 87.5, 86.75, 86.0, 85.16666666666666, 84.5, 83.74999999999999, 83.33333333333333, 82.66666666666667, 82.0, 81.33333333333333, 80.91666666666666, 80.16666666666666, 79.41666666666667, 78.75, 78.16666666666666, 77.16666666666666, 76.41666666666667, 75.5, 74.58333333333333, 73.83333333333333, 73.16666666666667, 72.24999999999999, 71.41666666666666, 70.58333333333333, 69.83333333333333, 68.91666666666667, 68.0, 67.08333333333333, 66.25, 65.25, 64.33333333333333, 63.5, 62.58333333333332, 61.66666666666667, 60.74999999999999, 59.916666666666664, 59.0, 57.99999999999999, 57.08333333333333, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 4, 4, 4, 5, 5, 6, 8, 8, 12, 12, 12, 12, 16, 16, 17, 19, 21, 22, 24, 27, 29, 31, 35, 39, 41, 43, 45, 49, 51, 54, 57, 62, 65, 66, 69, 73, 78, 80, 85, 90, 97, 105, 111, 120, 127, 134, 137, 146, 155, 165, 173, 182, 187, 195, 203, 211, 217, 226, 235, 243, 250, 262, 271, 282, 293, 302, 310, 321, 331, 341, 350, 361, 372, 383, 393, 405, 416, 427, 438, 449, 460, 470, 481, 493, 504, 516, 528], "exact_score": [98.91666666666666, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.25, 98.25, 97.91666666666666, 97.91666666666666, 97.91666666666666, 97.91666666666666, 97.58333333333333, 97.58333333333333, 97.5, 97.33333333333333, 97.16666666666667, 97.08333333333333, 96.91666666666666, 96.66666666666667, 96.5, 96.33333333333333, 96.0, 95.66666666666667, 95.5, 95.33333333333333, 95.16666666666667, 94.83333333333333, 94.66666666666667, 94.41666666666666, 94.16666666666667, 93.75, 93.5, 93.41666666666666, 93.16666666666666, 92.83333333333333, 92.41666666666666, 92.25, 91.83333333333333, 91.41666666666667, 90.83333333333333, 90.16666666666666, 89.66666666666666, 88.83333333333333, 88.33333333333333, 87.75, 87.5, 86.75, 86.0, 85.16666666666666, 84.5, 83.74999999999999, 83.33333333333333, 82.66666666666667, 82.0, 81.33333333333333, 80.91666666666666, 80.16666666666666, 79.41666666666667, 78.75, 78.16666666666666, 77.16666666666666, 76.41666666666667, 75.5, 74.58333333333333, 73.83333333333333, 73.16666666666667, 72.24999999999999, 71.33333333333334, 70.58333333333333, 69.83333333333333, 68.91666666666667, 68.0, 67.08333333333333, 66.25, 65.25, 64.33333333333333, 63.5, 62.58333333333332, 61.66666666666667, 60.74999999999999, 59.916666666666664, 59.0, 57.99999999999999, 57.08333333333333, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.5025, "threshold_area100": 86.50416666666666, "threshold_area25": 98.55000000000001}, "test": {"score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.07594062823611, 90.05868139454608, 90.02416292716603, 90.02416292716603, 90.02416292716603, 89.93786675871591, 89.85157059026578, 89.78253365550569, 89.73075595443561, 89.67897825336554, 89.6444597859855, 89.57542285122541, 89.55816361753539, 89.43734898170521, 89.29927511218501, 89.26475664480496, 89.19571971004487, 89.12668277528478, 89.05764584052469, 88.93683120469451, 88.81601656886434, 88.72972040041421, 88.574387297204, 88.36727649292371, 88.24646185709354, 88.19468415602347, 87.98757335174318, 87.91853641698309, 87.86675871591301, 87.53883327580255, 87.41801863997239, 87.27994477045219, 87.05557473248187, 86.76216775975146, 86.60683465654124, 86.26164998274076, 86.03727994477045, 85.9337245426303, 85.81290990680014, 85.72661373835001, 85.53676216775975, 85.29513289609942, 84.89817052122885, 84.79461511908872, 84.51846738004832, 84.22506040731791, 83.94891266827752, 83.37935795650672, 83.01691404901622, 82.72350707628581, 82.34380393510527, 82.05039696237488, 81.54987918536416, 81.37728684846392, 81.1356575768036, 80.44528822920262, 80.06558508802209, 79.68588194684155, 79.30617880566102, 78.87469796341043, 78.65032792544011, 78.30514325163962, 78.04625474628925, 77.32136693130825, 76.35484984466689, 75.87159130134621, 75.06040731791508, 74.49085260614429, 73.43803935105281, 72.92026234035208, 72.52329996548153, 72.0055229547808, 71.52226441146013, 70.96996893337936, 70.34863652053849, 69.62374870555747, 68.77804625474629, 68.24301001035553, 67.7079737659648, 66.44804970659303, 65.5678287884018, 64.86020020711081, 64.32516396272005, 63.61753538142907, 63.047980669658266, 62.09872281670694, 60.59716948567484, 59.75146703486365, 58.750431480842245, 58.19813600276147, 57.50776665516051, 57.024508111839836, 56.24784259578875, 54.81532619951675], "calls": [0.0, 1.0182947877114257, 1.8985157059026578, 2.657921988263721, 3.8833275802554366, 4.815326199516742, 6.127027959958578, 6.88643424231964, 7.74939592682085, 8.45702450811184, 9.40628236106317, 10.217466344494303, 11.322057300655851, 12.461166724197446, 13.703831549879187, 14.618570935450467, 15.878494994822228, 16.41353123921298, 17.241974456334137, 18.27752847773559, 19.537452537107352, 20.745598895409042, 21.6430790472903, 22.160856057991023, 22.989299275112185, 23.76596479116327, 25.232999654815323, 26.061442871936485, 26.66551605108733, 27.701070072488783, 29.11632723507076, 29.927511218501902, 30.54884363134277, 31.567138419054192, 32.084915429754915, 32.9306178805661, 34.41491197790818, 35.22609596133932, 36.00276147739041, 36.952019330341734, 37.866758715913015, 38.60890576458405, 40.02416292716603, 41.30134622022782, 42.14704867103901, 42.7166033828098, 43.389713496720745, 44.13186054539178, 44.874007594062824, 46.582671729375214, 47.30755954435623, 48.51570590265793, 49.482222989299274, 50.72488781498101, 52.502588885053505, 53.71073524335519, 54.349326889886086, 55.36762167759751, 56.07525025888851, 57.611322057300654, 58.37072833966172, 59.49257852951328, 60.89057645840524, 61.90887124611667, 63.047980669658266, 64.23886779426994, 65.11908871246118, 65.70590265792198, 66.51708664135313, 67.19019675526407, 69.15774939592683, 70.67656196064894, 71.60856057991025, 73.00655850880221, 73.90403866068347, 75.40559199171557, 76.26855367621678, 76.97618225750776, 77.73558853986883, 78.39143942008975, 79.20262340352089, 80.18639972385226, 81.27373144632378, 82.39558163617535, 83.15498791853642, 83.91439420089748, 85.69209527096997, 86.77942699344149, 87.81498101484294, 88.52260959613393, 89.43734898170521, 90.19675526406627, 91.28408698653779, 93.14808422506042, 94.13186054539179, 95.44356230583362, 96.15119088712461, 96.87607870210563, 97.44563341387642, 98.3603727994477, 100.0], "rescued": [0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 6, 6, 6, 7, 7, 8, 8, 8, 9, 10, 12, 14, 15, 15, 17, 17, 18, 18, 19, 21, 22, 23, 25, 31, 34, 37, 37, 39, 39, 44, 48, 52, 52, 53, 55, 63, 63, 67, 70, 73, 82, 85, 88, 93, 95, 96, 99, 103, 105, 106, 106, 109, 112, 113, 115, 117, 124, 124, 128, 130, 132, 133, 135, 139, 141, 143, 143, 143, 143, 144, 144], "harmed": [0, 4, 4, 4, 4, 4, 5, 7, 8, 10, 10, 10, 15, 21, 25, 28, 31, 33, 37, 39, 46, 54, 56, 60, 64, 68, 75, 82, 87, 97, 110, 117, 120, 133, 137, 141, 160, 167, 176, 190, 209, 220, 241, 254, 262, 269, 275, 286, 301, 326, 333, 350, 369, 391, 427, 451, 468, 492, 509, 543, 557, 575, 615, 638, 662, 692, 717, 734, 757, 775, 826, 885, 916, 968, 1003, 1065, 1098, 1125, 1157, 1186, 1218, 1257, 1302, 1352, 1385, 1418, 1498, 1549, 1594, 1627, 1670, 1704, 1761, 1852, 1903, 1963, 1995, 2035, 2063, 2109, 2192], "exact_score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.07594062823611, 90.02416292716603, 90.02416292716603, 90.02416292716603, 89.95512599240594, 89.90334829133586, 89.81705212288574, 89.74801518812563, 89.73075595443561, 89.67897825336554, 89.57542285122541, 89.55816361753539, 89.5063859164653, 89.40283051432516, 89.28201587849499, 89.2129789437349, 89.12668277528478, 89.04038660683466, 88.95409043838454, 88.81601656886434, 88.67794269934414, 88.55712806351399, 88.36727649292371, 88.22920262340351, 88.10838798757334, 87.93579565067311, 87.84949948222298, 87.64238867794269, 87.41801863997239, 87.27994477045219, 87.05557473248187, 86.72764929237141, 86.53779772178115, 86.26164998274076, 86.05453917846047, 85.96824301001035, 85.83016914049016, 85.5712806351398, 85.26061442871936, 85.001725923369, 84.84639282015878, 84.63928201587849, 84.34587504314808, 84.12150500517777, 83.87987573351742, 83.55195029340697, 83.24128408698654, 82.87884017949602, 82.53365550569555, 82.08491542975491, 81.77424922333448, 81.429064549534, 81.30824991370382, 80.89402830514325, 80.41076976182258, 80.03106662064205, 79.7204004142216, 79.4269934414912, 78.89195719710045, 78.51225405591991, 78.09803244735933, 77.70107007248878, 77.39040386606834, 76.76907145322747, 76.2167759751467, 75.68173973075595, 75.0776665516051, 74.45633413876423, 73.76596479116327, 73.0755954435623, 72.5060407317915, 71.85018985157059, 71.12530203658957, 70.48671039005868, 69.77908180876769, 69.0714532274767, 68.32930617880567, 67.65619606489471, 66.91404901622367, 66.17190196755264, 65.41249568519157, 64.70486710390058, 63.910942354159474, 63.134276838108384, 62.39212978943735, 61.563686572316186, 60.683465654124944, 59.85502243700379, 59.11287538833275, 58.33620987228167, 57.421470486710376, 56.541249568519156, 55.67828788401794, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.2413960307218, "threshold_area100": 80.52053848809112, "threshold_area25": 89.7859855022437}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.002319731516763568, "operator": "ge"}, {"budget": 0.02, "threshold": -0.0030661060009151697, "operator": "ge"}, {"budget": 0.03, "threshold": -0.003583932761102915, "operator": "ge"}, {"budget": 0.04, "threshold": -0.004538949113339186, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0055152541026473045, "operator": "ge"}, {"budget": 0.06, "threshold": -0.006949907634407282, "operator": "ge"}, {"budget": 0.07, "threshold": -0.007858322001993656, "operator": "ge"}, {"budget": 0.08, "threshold": -0.009692218154668808, "operator": "ge"}, {"budget": 0.09, "threshold": -0.011516870930790901, "operator": "ge"}, {"budget": 0.1, "threshold": -0.013871230185031891, "operator": "ge"}, {"budget": 0.11, "threshold": -0.015995511785149574, "operator": "ge"}, {"budget": 0.12, "threshold": -0.02032342739403248, "operator": "ge"}, {"budget": 0.13, "threshold": -0.024061406031250954, "operator": "ge"}, {"budget": 0.14, "threshold": -0.028295140713453293, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03263888508081436, "operator": "ge"}, {"budget": 0.16, "threshold": -0.03858954459428787, "operator": "ge"}, {"budget": 0.17, "threshold": -0.04196999967098236, "operator": "ge"}, {"budget": 0.18, "threshold": -0.0454777367413044, "operator": "ge"}, {"budget": 0.19, "threshold": -0.050907235592603683, "operator": "ge"}, {"budget": 0.2, "threshold": -0.057471565902233124, "operator": "ge"}, {"budget": 0.21, "threshold": -0.06376554071903229, "operator": "ge"}, {"budget": 0.22, "threshold": -0.0696253627538681, "operator": "ge"}, {"budget": 0.23, "threshold": -0.0735263004899025, "operator": "ge"}, {"budget": 0.24, "threshold": -0.08060379326343536, "operator": "ge"}, {"budget": 0.25, "threshold": -0.08624498546123505, "operator": "ge"}, {"budget": 0.26, "threshold": -0.09741979092359543, "operator": "ge"}, {"budget": 0.27, "threshold": -0.10490947216749191, "operator": "ge"}, {"budget": 0.28, "threshold": -0.11138848960399628, "operator": "ge"}, {"budget": 0.29, "threshold": -0.1218261644244194, "operator": "ge"}, {"budget": 0.3, "threshold": -0.1322326958179474, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1393558233976364, "operator": "ge"}, {"budget": 0.32, "threshold": -0.14585353434085846, "operator": "ge"}, {"budget": 0.33, "threshold": -0.1572570502758026, "operator": "ge"}, {"budget": 0.34, "threshold": -0.16415879130363464, "operator": "ge"}, {"budget": 0.35, "threshold": -0.17227540910243988, "operator": "ge"}, {"budget": 0.36, "threshold": -0.18772833049297333, "operator": "ge"}, {"budget": 0.37, "threshold": -0.19866502285003662, "operator": "ge"}, {"budget": 0.38, "threshold": -0.21087077260017395, "operator": "ge"}, {"budget": 0.39, "threshold": -0.22245928645133972, "operator": "ge"}, {"budget": 0.4, "threshold": -0.23491227626800537, "operator": "ge"}, {"budget": 0.41, "threshold": -0.24656431376934052, "operator": "ge"}, {"budget": 0.42, "threshold": -0.2697896361351013, "operator": "ge"}, {"budget": 0.43, "threshold": -0.29196593165397644, "operator": "ge"}, {"budget": 0.44, "threshold": -0.30500420928001404, "operator": "ge"}, {"budget": 0.45, "threshold": -0.3130740225315094, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3248336911201477, "operator": "ge"}, {"budget": 0.47, "threshold": -0.33853960037231445, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3494745194911957, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3742058277130127, "operator": "ge"}, {"budget": 0.5, "threshold": -0.38880011439323425, "operator": "ge"}, {"budget": 0.51, "threshold": -0.41233548521995544, "operator": "ge"}, {"budget": 0.52, "threshold": -0.432985782623291, "operator": "ge"}, {"budget": 0.53, "threshold": -0.4523588716983795, "operator": "ge"}, {"budget": 0.54, "threshold": -0.48142823576927185, "operator": "ge"}, {"budget": 0.55, "threshold": -0.5001511573791504, "operator": "ge"}, {"budget": 0.56, "threshold": -0.5124437808990479, "operator": "ge"}, {"budget": 0.57, "threshold": -0.5299684405326843, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5399276614189148, "operator": "ge"}, {"budget": 0.59, "threshold": -0.566520094871521, "operator": "ge"}, {"budget": 0.6, "threshold": -0.578437864780426, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5925268530845642, "operator": "ge"}, {"budget": 0.62, "threshold": -0.6135192513465881, "operator": "ge"}, {"budget": 0.63, "threshold": -0.62940913438797, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6467357277870178, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6679899096488953, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6791021823883057, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6878600120544434, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6997919678688049, "operator": "ge"}, {"budget": 0.69, "threshold": -0.7079272866249084, "operator": "ge"}, {"budget": 0.7, "threshold": -0.73135906457901, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7492585778236389, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7601076364517212, "operator": "ge"}, {"budget": 0.73, "threshold": -0.77741539478302, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7923457026481628, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8104785084724426, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8259429335594177, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8364963531494141, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8440443873405457, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8525980114936829, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8632519245147705, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8728476762771606, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8835811018943787, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8969945311546326, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9033701419830322, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9107188582420349, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9239863157272339, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9310542345046997, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9378506541252136, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9459103941917419, "operator": "ge"}, {"budget": 0.9, "threshold": -0.952576220035553, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9579152464866638, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9636959433555603, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9706143736839294, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9746701121330261, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9787147045135498, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9810968041419983, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9831351041793823, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9849997758865356, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9867321848869324, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.25, 98.25, 97.91666666666666, 97.91666666666666, 97.91666666666666, 97.91666666666666, 97.58333333333333, 97.58333333333333, 97.5, 97.33333333333333, 97.16666666666667, 97.08333333333333, 96.91666666666666, 96.66666666666667, 96.5, 96.33333333333333, 96.0, 95.66666666666667, 95.5, 95.33333333333333, 95.16666666666667, 94.83333333333333, 94.66666666666667, 94.41666666666666, 94.16666666666667, 93.75, 93.5, 93.41666666666666, 93.16666666666666, 92.83333333333333, 92.41666666666666, 92.25, 91.83333333333333, 91.41666666666667, 90.83333333333333, 90.16666666666666, 89.66666666666666, 88.91666666666667, 88.33333333333333, 87.75, 87.5, 86.75, 86.0, 85.16666666666666, 84.5, 83.74999999999999, 83.33333333333333, 82.66666666666667, 82.0, 81.33333333333333, 80.91666666666666, 80.16666666666666, 79.41666666666667, 78.75, 78.16666666666666, 77.16666666666666, 76.41666666666667, 75.5, 74.58333333333333, 73.83333333333333, 73.16666666666667, 72.24999999999999, 71.41666666666666, 70.58333333333333, 69.83333333333333, 68.91666666666667, 68.0, 67.08333333333333, 66.25, 65.25, 64.33333333333333, 63.5, 62.58333333333332, 61.66666666666667, 60.74999999999999, 59.916666666666664, 59.0, 57.99999999999999, 57.08333333333333, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 4, 4, 4, 5, 5, 6, 8, 8, 12, 12, 12, 12, 16, 16, 17, 19, 21, 22, 24, 27, 29, 31, 35, 39, 41, 43, 45, 49, 51, 54, 57, 62, 65, 66, 69, 73, 78, 80, 85, 90, 97, 105, 111, 120, 127, 134, 137, 146, 155, 165, 173, 182, 187, 195, 203, 211, 217, 226, 235, 243, 250, 262, 271, 282, 293, 302, 310, 321, 331, 341, 350, 361, 372, 383, 393, 405, 416, 427, 438, 449, 460, 470, 481, 493, 504, 516, 528], "exact_score": [98.91666666666666, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.58333333333333, 98.58333333333333, 98.58333333333333, 98.5, 98.5, 98.41666666666666, 98.25, 98.25, 97.91666666666666, 97.91666666666666, 97.91666666666666, 97.91666666666666, 97.58333333333333, 97.58333333333333, 97.5, 97.33333333333333, 97.16666666666667, 97.08333333333333, 96.91666666666666, 96.66666666666667, 96.5, 96.33333333333333, 96.0, 95.66666666666667, 95.5, 95.33333333333333, 95.16666666666667, 94.83333333333333, 94.66666666666667, 94.41666666666666, 94.16666666666667, 93.75, 93.5, 93.41666666666666, 93.16666666666666, 92.83333333333333, 92.41666666666666, 92.25, 91.83333333333333, 91.41666666666667, 90.83333333333333, 90.16666666666666, 89.66666666666666, 88.83333333333333, 88.33333333333333, 87.75, 87.5, 86.75, 86.0, 85.16666666666666, 84.5, 83.74999999999999, 83.33333333333333, 82.66666666666667, 82.0, 81.33333333333333, 80.91666666666666, 80.16666666666666, 79.41666666666667, 78.75, 78.16666666666666, 77.16666666666666, 76.41666666666667, 75.5, 74.58333333333333, 73.83333333333333, 73.16666666666667, 72.24999999999999, 71.33333333333334, 70.58333333333333, 69.83333333333333, 68.91666666666667, 68.0, 67.08333333333333, 66.25, 65.25, 64.33333333333333, 63.5, 62.58333333333332, 61.66666666666667, 60.74999999999999, 59.916666666666664, 59.0, 57.99999999999999, 57.08333333333333, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.5025, "threshold_area100": 86.50416666666666, "threshold_area25": 98.55000000000001}, "test": {"score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.07594062823611, 90.05868139454608, 90.02416292716603, 90.02416292716603, 90.02416292716603, 89.93786675871591, 89.85157059026578, 89.78253365550569, 89.73075595443561, 89.67897825336554, 89.6444597859855, 89.57542285122541, 89.55816361753539, 89.43734898170521, 89.29927511218501, 89.26475664480496, 89.19571971004487, 89.12668277528478, 89.05764584052469, 88.93683120469451, 88.81601656886434, 88.72972040041421, 88.574387297204, 88.36727649292371, 88.24646185709354, 88.19468415602347, 87.98757335174318, 87.91853641698309, 87.86675871591301, 87.53883327580255, 87.41801863997239, 87.27994477045219, 87.05557473248187, 86.76216775975146, 86.60683465654124, 86.26164998274076, 86.03727994477045, 85.9337245426303, 85.81290990680014, 85.72661373835001, 85.53676216775975, 85.29513289609942, 84.89817052122885, 84.79461511908872, 84.51846738004832, 84.22506040731791, 83.94891266827752, 83.37935795650672, 83.01691404901622, 82.72350707628581, 82.34380393510527, 82.05039696237488, 81.54987918536416, 81.37728684846392, 81.1356575768036, 80.44528822920262, 80.06558508802209, 79.68588194684155, 79.30617880566102, 78.87469796341043, 78.65032792544011, 78.30514325163962, 78.04625474628925, 77.32136693130825, 76.35484984466689, 75.87159130134621, 75.06040731791508, 74.49085260614429, 73.43803935105281, 72.92026234035208, 72.52329996548153, 72.0055229547808, 71.52226441146013, 70.96996893337936, 70.34863652053849, 69.62374870555747, 68.77804625474629, 68.24301001035553, 67.7079737659648, 66.44804970659303, 65.5678287884018, 64.86020020711081, 64.32516396272005, 63.61753538142907, 63.047980669658266, 62.09872281670694, 60.59716948567484, 59.75146703486365, 58.750431480842245, 58.19813600276147, 57.50776665516051, 57.024508111839836, 56.24784259578875, 54.81532619951675], "calls": [0.0, 1.0182947877114257, 1.8985157059026578, 2.657921988263721, 3.8833275802554366, 4.815326199516742, 6.127027959958578, 6.88643424231964, 7.74939592682085, 8.45702450811184, 9.40628236106317, 10.217466344494303, 11.322057300655851, 12.461166724197446, 13.703831549879187, 14.618570935450467, 15.878494994822228, 16.41353123921298, 17.241974456334137, 18.27752847773559, 19.537452537107352, 20.745598895409042, 21.6430790472903, 22.160856057991023, 22.989299275112185, 23.76596479116327, 25.232999654815323, 26.061442871936485, 26.66551605108733, 27.701070072488783, 29.11632723507076, 29.927511218501902, 30.54884363134277, 31.567138419054192, 32.084915429754915, 32.9306178805661, 34.41491197790818, 35.22609596133932, 36.00276147739041, 36.952019330341734, 37.866758715913015, 38.60890576458405, 40.02416292716603, 41.30134622022782, 42.14704867103901, 42.7166033828098, 43.389713496720745, 44.13186054539178, 44.874007594062824, 46.582671729375214, 47.30755954435623, 48.51570590265793, 49.482222989299274, 50.72488781498101, 52.502588885053505, 53.71073524335519, 54.349326889886086, 55.36762167759751, 56.07525025888851, 57.611322057300654, 58.37072833966172, 59.49257852951328, 60.89057645840524, 61.90887124611667, 63.047980669658266, 64.23886779426994, 65.11908871246118, 65.70590265792198, 66.51708664135313, 67.19019675526407, 69.15774939592683, 70.67656196064894, 71.60856057991025, 73.00655850880221, 73.90403866068347, 75.40559199171557, 76.26855367621678, 76.97618225750776, 77.73558853986883, 78.39143942008975, 79.20262340352089, 80.18639972385226, 81.27373144632378, 82.39558163617535, 83.15498791853642, 83.91439420089748, 85.69209527096997, 86.77942699344149, 87.81498101484294, 88.52260959613393, 89.43734898170521, 90.19675526406627, 91.28408698653779, 93.14808422506042, 94.13186054539179, 95.44356230583362, 96.15119088712461, 96.87607870210563, 97.44563341387642, 98.3603727994477, 100.0], "rescued": [0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 6, 6, 6, 7, 7, 8, 8, 8, 9, 10, 12, 14, 15, 15, 17, 17, 18, 18, 19, 21, 22, 23, 25, 31, 34, 37, 37, 39, 39, 44, 48, 52, 52, 53, 55, 63, 63, 67, 70, 73, 82, 85, 88, 93, 95, 96, 99, 103, 105, 106, 106, 109, 112, 113, 115, 117, 124, 124, 128, 130, 132, 133, 135, 139, 141, 143, 143, 143, 143, 144, 144], "harmed": [0, 4, 4, 4, 4, 4, 5, 7, 8, 10, 10, 10, 15, 21, 25, 28, 31, 33, 37, 39, 46, 54, 56, 60, 64, 68, 75, 82, 87, 97, 110, 117, 120, 133, 137, 141, 160, 167, 176, 190, 209, 220, 241, 254, 262, 269, 275, 286, 301, 326, 333, 350, 369, 391, 427, 451, 468, 492, 509, 543, 557, 575, 615, 638, 662, 692, 717, 734, 757, 775, 826, 885, 916, 968, 1003, 1065, 1098, 1125, 1157, 1186, 1218, 1257, 1302, 1352, 1385, 1418, 1498, 1549, 1594, 1627, 1670, 1704, 1761, 1852, 1903, 1963, 1995, 2035, 2063, 2109, 2192], "exact_score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.07594062823611, 90.02416292716603, 90.02416292716603, 90.02416292716603, 89.95512599240594, 89.90334829133586, 89.81705212288574, 89.74801518812563, 89.73075595443561, 89.67897825336554, 89.57542285122541, 89.55816361753539, 89.5063859164653, 89.40283051432516, 89.28201587849499, 89.2129789437349, 89.12668277528478, 89.04038660683466, 88.95409043838454, 88.81601656886434, 88.67794269934414, 88.55712806351399, 88.36727649292371, 88.22920262340351, 88.10838798757334, 87.93579565067311, 87.84949948222298, 87.64238867794269, 87.41801863997239, 87.27994477045219, 87.05557473248187, 86.72764929237141, 86.53779772178115, 86.26164998274076, 86.05453917846047, 85.96824301001035, 85.83016914049016, 85.5712806351398, 85.26061442871936, 85.001725923369, 84.84639282015878, 84.63928201587849, 84.34587504314808, 84.12150500517777, 83.87987573351742, 83.55195029340697, 83.24128408698654, 82.87884017949602, 82.53365550569555, 82.08491542975491, 81.77424922333448, 81.429064549534, 81.30824991370382, 80.89402830514325, 80.41076976182258, 80.03106662064205, 79.7204004142216, 79.4269934414912, 78.89195719710045, 78.51225405591991, 78.09803244735933, 77.70107007248878, 77.39040386606834, 76.76907145322747, 76.2167759751467, 75.68173973075595, 75.0776665516051, 74.45633413876423, 73.76596479116327, 73.0755954435623, 72.5060407317915, 71.85018985157059, 71.12530203658957, 70.48671039005868, 69.77908180876769, 69.0714532274767, 68.32930617880567, 67.65619606489471, 66.91404901622367, 66.17190196755264, 65.41249568519157, 64.70486710390058, 63.910942354159474, 63.134276838108384, 62.39212978943735, 61.563686572316186, 60.683465654124944, 59.85502243700379, 59.11287538833275, 58.33620987228167, 57.421470486710376, 56.541249568519156, 55.67828788401794, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.2413960307218, "threshold_area100": 80.52053848809112, "threshold_area25": 89.7859855022437}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.2356679439544678, "operator": "ge"}, {"budget": 0.02, "threshold": -3.431217670440674, "operator": "ge"}, {"budget": 0.03, "threshold": -3.8651185035705566, "operator": "ge"}, {"budget": 0.04, "threshold": -4.309472560882568, "operator": "ge"}, {"budget": 0.05, "threshold": -4.89716911315918, "operator": "ge"}, {"budget": 0.06, "threshold": -5.2780914306640625, "operator": "ge"}, {"budget": 0.07, "threshold": -5.642271995544434, "operator": "ge"}, {"budget": 0.08, "threshold": -5.942852020263672, "operator": "ge"}, {"budget": 0.09, "threshold": -6.216003894805908, "operator": "ge"}, {"budget": 0.1, "threshold": -6.424223899841309, "operator": "ge"}, {"budget": 0.11, "threshold": -6.543820858001709, "operator": "ge"}, {"budget": 0.12, "threshold": -6.639060974121094, "operator": "ge"}, {"budget": 0.13, "threshold": -6.760509967803955, "operator": "ge"}, {"budget": 0.14, "threshold": -6.927711486816406, "operator": "ge"}, {"budget": 0.15, "threshold": -7.036006450653076, "operator": "ge"}, {"budget": 0.16, "threshold": -7.164877891540527, "operator": "ge"}, {"budget": 0.17, "threshold": -7.254049777984619, "operator": "ge"}, {"budget": 0.18, "threshold": -7.330608367919922, "operator": "ge"}, {"budget": 0.19, "threshold": -7.3786821365356445, "operator": "ge"}, {"budget": 0.2, "threshold": -7.480747222900391, "operator": "ge"}, {"budget": 0.21, "threshold": -7.616907119750977, "operator": "ge"}, {"budget": 0.22, "threshold": -7.671538352966309, "operator": "ge"}, {"budget": 0.23, "threshold": -7.752789497375488, "operator": "ge"}, {"budget": 0.24, "threshold": -7.878142833709717, "operator": "ge"}, {"budget": 0.25, "threshold": -7.958076000213623, "operator": "ge"}, {"budget": 0.26, "threshold": -8.072710990905762, "operator": "ge"}, {"budget": 0.27, "threshold": -8.127370834350586, "operator": "ge"}, {"budget": 0.28, "threshold": -8.177840232849121, "operator": "ge"}, {"budget": 0.29, "threshold": -8.294240951538086, "operator": "ge"}, {"budget": 0.3, "threshold": -8.373394012451172, "operator": "ge"}, {"budget": 0.31, "threshold": -8.485615730285645, "operator": "ge"}, {"budget": 0.32, "threshold": -8.572763442993164, "operator": "ge"}, {"budget": 0.33, "threshold": -8.620635032653809, "operator": "ge"}, {"budget": 0.34, "threshold": -8.679057121276855, "operator": "ge"}, {"budget": 0.35, "threshold": -8.724209785461426, "operator": "ge"}, {"budget": 0.36, "threshold": -8.830562591552734, "operator": "ge"}, {"budget": 0.37, "threshold": -8.903064727783203, "operator": "ge"}, {"budget": 0.38, "threshold": -8.958612442016602, "operator": "ge"}, {"budget": 0.39, "threshold": -9.01429557800293, "operator": "ge"}, {"budget": 0.4, "threshold": -9.106941223144531, "operator": "ge"}, {"budget": 0.41, "threshold": -9.174939155578613, "operator": "ge"}, {"budget": 0.42, "threshold": -9.228843688964844, "operator": "ge"}, {"budget": 0.43, "threshold": -9.269659042358398, "operator": "ge"}, {"budget": 0.44, "threshold": -9.317996978759766, "operator": "ge"}, {"budget": 0.45, "threshold": -9.384103775024414, "operator": "ge"}, {"budget": 0.46, "threshold": -9.455546379089355, "operator": "ge"}, {"budget": 0.47, "threshold": -9.524253845214844, "operator": "ge"}, {"budget": 0.48, "threshold": -9.562366485595703, "operator": "ge"}, {"budget": 0.49, "threshold": -9.59985065460205, "operator": "ge"}, {"budget": 0.5, "threshold": -9.659618377685547, "operator": "ge"}, {"budget": 0.51, "threshold": -9.7109375, "operator": "ge"}, {"budget": 0.52, "threshold": -9.77214241027832, "operator": "ge"}, {"budget": 0.53, "threshold": -9.806901931762695, "operator": "ge"}, {"budget": 0.54, "threshold": -9.837231636047363, "operator": "ge"}, {"budget": 0.55, "threshold": -9.908557891845703, "operator": "ge"}, {"budget": 0.56, "threshold": -9.953896522521973, "operator": "ge"}, {"budget": 0.57, "threshold": -10.00082778930664, "operator": "ge"}, {"budget": 0.58, "threshold": -10.047393798828125, "operator": "ge"}, {"budget": 0.59, "threshold": -10.079193115234375, "operator": "ge"}, {"budget": 0.6, "threshold": -10.118508338928223, "operator": "ge"}, {"budget": 0.61, "threshold": -10.170221328735352, "operator": "ge"}, {"budget": 0.62, "threshold": -10.21993637084961, "operator": "ge"}, {"budget": 0.63, "threshold": -10.28605842590332, "operator": "ge"}, {"budget": 0.64, "threshold": -10.329093933105469, "operator": "ge"}, {"budget": 0.65, "threshold": -10.364373207092285, "operator": "ge"}, {"budget": 0.66, "threshold": -10.413491249084473, "operator": "ge"}, {"budget": 0.67, "threshold": -10.455049514770508, "operator": "ge"}, {"budget": 0.68, "threshold": -10.498977661132812, "operator": "ge"}, {"budget": 0.69, "threshold": -10.542814254760742, "operator": "ge"}, {"budget": 0.7, "threshold": -10.577463150024414, "operator": "ge"}, {"budget": 0.71, "threshold": -10.621834754943848, "operator": "ge"}, {"budget": 0.72, "threshold": -10.66136646270752, "operator": "ge"}, {"budget": 0.73, "threshold": -10.704217910766602, "operator": "ge"}, {"budget": 0.74, "threshold": -10.774110794067383, "operator": "ge"}, {"budget": 0.75, "threshold": -10.819963455200195, "operator": "ge"}, {"budget": 0.76, "threshold": -10.873523712158203, "operator": "ge"}, {"budget": 0.77, "threshold": -10.927906036376953, "operator": "ge"}, {"budget": 0.78, "threshold": -11.009574890136719, "operator": "ge"}, {"budget": 0.79, "threshold": -11.078128814697266, "operator": "ge"}, {"budget": 0.8, "threshold": -11.140304565429688, "operator": "ge"}, {"budget": 0.81, "threshold": -11.203908920288086, "operator": "ge"}, {"budget": 0.82, "threshold": -11.248991966247559, "operator": "ge"}, {"budget": 0.83, "threshold": -11.353987693786621, "operator": "ge"}, {"budget": 0.84, "threshold": -11.393806457519531, "operator": "ge"}, {"budget": 0.85, "threshold": -11.433164596557617, "operator": "ge"}, {"budget": 0.86, "threshold": -11.553201675415039, "operator": "ge"}, {"budget": 0.87, "threshold": -11.620599746704102, "operator": "ge"}, {"budget": 0.88, "threshold": -11.734665870666504, "operator": "ge"}, {"budget": 0.89, "threshold": -11.810776710510254, "operator": "ge"}, {"budget": 0.9, "threshold": -11.8574800491333, "operator": "ge"}, {"budget": 0.91, "threshold": -11.987293243408203, "operator": "ge"}, {"budget": 0.92, "threshold": -12.084464073181152, "operator": "ge"}, {"budget": 0.93, "threshold": -12.171207427978516, "operator": "ge"}, {"budget": 0.94, "threshold": -12.278242111206055, "operator": "ge"}, {"budget": 0.95, "threshold": -12.347491264343262, "operator": "ge"}, {"budget": 0.96, "threshold": -12.48214340209961, "operator": "ge"}, {"budget": 0.97, "threshold": -12.636581420898438, "operator": "ge"}, {"budget": 0.98, "threshold": -12.887552261352539, "operator": "ge"}, {"budget": 0.99, "threshold": -13.118070602416992, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.41666666666666, 98.0, 97.66666666666667, 97.41666666666666, 97.16666666666667, 96.83333333333333, 96.5, 95.91666666666666, 95.75, 95.58333333333333, 95.25, 95.08333333333333, 94.83333333333333, 94.5, 94.08333333333333, 93.83333333333333, 93.75, 93.58333333333333, 93.08333333333333, 93.0, 92.75, 92.58333333333333, 92.41666666666666, 92.25, 91.91666666666667, 91.75, 91.49999999999999, 91.16666666666666, 90.91666666666667, 90.66666666666666, 90.5, 90.08333333333333, 89.5, 89.16666666666666, 89.0, 88.41666666666667, 88.08333333333334, 87.83333333333333, 87.5, 87.33333333333333, 87.25, 86.66666666666667, 86.41666666666666, 86.08333333333333, 85.91666666666666, 85.75, 85.25, 84.66666666666667, 84.25, 84.0, 83.74999999999999, 83.41666666666666, 83.16666666666667, 82.66666666666667, 82.25, 81.83333333333334, 81.5, 80.91666666666666, 80.66666666666666, 80.33333333333333, 79.91666666666666, 79.41666666666667, 79.16666666666666, 78.66666666666666, 78.25, 77.66666666666666, 77.33333333333333, 76.91666666666667, 76.58333333333334, 76.08333333333333, 75.58333333333334, 75.08333333333333, 74.66666666666666, 74.08333333333333, 73.41666666666666, 73.0, 72.24999999999999, 71.66666666666666, 70.91666666666666, 70.41666666666666, 69.75, 69.33333333333334, 68.41666666666666, 67.91666666666667, 67.25, 66.41666666666666, 65.75, 65.16666666666666, 64.33333333333333, 63.5, 62.58333333333332, 61.66666666666667, 60.916666666666664, 60.25, 59.5, 58.74999999999999, 57.91666666666666, 56.99999999999999, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 6, 11, 15, 18, 22, 26, 30, 37, 39, 41, 45, 47, 50, 54, 59, 62, 63, 65, 71, 72, 75, 77, 79, 81, 85, 87, 91, 95, 98, 101, 103, 108, 115, 119, 121, 128, 132, 135, 139, 141, 142, 149, 152, 156, 158, 160, 166, 173, 178, 181, 184, 188, 191, 197, 202, 207, 211, 218, 221, 225, 230, 236, 239, 245, 250, 257, 261, 266, 270, 276, 282, 288, 293, 300, 308, 313, 322, 329, 338, 344, 352, 357, 368, 374, 382, 392, 400, 407, 417, 427, 438, 449, 458, 466, 475, 484, 494, 505, 517, 528], "exact_score": [98.91666666666666, 98.41666666666666, 98.0, 97.66666666666667, 97.41666666666666, 97.16666666666667, 96.83333333333333, 96.5, 95.91666666666666, 95.75, 95.58333333333333, 95.25, 95.08333333333333, 94.83333333333333, 94.5, 94.08333333333333, 93.83333333333333, 93.75, 93.58333333333333, 93.08333333333333, 93.0, 92.75, 92.58333333333333, 92.41666666666666, 92.25, 91.91666666666667, 91.75, 91.49999999999999, 91.16666666666666, 90.91666666666667, 90.66666666666666, 90.5, 90.08333333333333, 89.5, 89.16666666666666, 89.0, 88.41666666666667, 88.08333333333334, 87.83333333333333, 87.5, 87.33333333333333, 87.25, 86.66666666666667, 86.41666666666666, 86.08333333333333, 85.91666666666666, 85.75, 85.25, 84.66666666666667, 84.25, 84.0, 83.74999999999999, 83.41666666666666, 83.16666666666667, 82.66666666666667, 82.25, 81.83333333333334, 81.41666666666667, 80.91666666666666, 80.66666666666666, 80.33333333333333, 79.91666666666666, 79.41666666666667, 79.16666666666666, 78.66666666666666, 78.25, 77.66666666666666, 77.33333333333333, 76.91666666666667, 76.5, 76.08333333333333, 75.58333333333334, 75.08333333333333, 74.66666666666666, 74.08333333333333, 73.41666666666666, 73.0, 72.24999999999999, 71.66666666666666, 70.91666666666666, 70.41666666666666, 69.75, 69.24999999999999, 68.41666666666666, 67.91666666666667, 67.25, 66.41666666666666, 65.75, 65.16666666666666, 64.33333333333333, 63.5, 62.58333333333332, 61.66666666666667, 60.916666666666664, 60.25, 59.5, 58.74999999999999, 57.91666666666666, 56.99999999999999, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 81.82583333333334, "threshold_area100": 81.82833333333335, "threshold_area25": 95.04999999999998}, "test": {"score": [90.16223679668623, 89.73075595443561, 89.24749741111494, 88.98860890576458, 88.67794269934414, 88.05661028650327, 87.43527787366241, 86.79668622713152, 86.02002071108042, 85.4332067656196, 84.74283741801864, 84.39765274421815, 84.1387642388678, 83.65550569554712, 83.17224715222645, 82.77528477735588, 82.4301001035554, 82.136693130825, 81.7915084570245, 81.61891612012427, 81.25647221263375, 80.70417673455297, 80.44528822920262, 80.04832585433206, 79.70314118053157, 79.4960303762513, 79.04729030031066, 78.84017949603037, 78.70210562651017, 78.2878840179496, 77.99447704521918, 77.47670003451846, 77.14877459440801, 77.01070072488781, 76.7863306869175, 76.54470141525715, 76.13047980669658, 75.80255436658612, 75.56092509492578, 75.26751812219537, 74.76700034518467, 74.47359337245426, 74.18018639972385, 74.02485329651363, 73.73144632378322, 73.43803935105281, 73.02381774249223, 72.67863306869174, 72.43700379703141, 72.21263375906109, 71.90196755264066, 71.71211598205039, 71.48774594408009, 71.15982050396961, 70.98722816706939, 70.53848809112876, 70.38315498791854, 70.02071108042801, 69.79634104245771, 69.62374870555747, 69.39937866758716, 68.98515705902656, 68.67449085260614, 68.31204694511564, 68.15671384190541, 67.96686227131515, 67.7079737659648, 67.38004832585432, 67.10390058681395, 66.65516051087332, 66.39627200552295, 66.08560579910252, 65.80945806006213, 65.53331032102174, 65.30894028305143, 65.01553331032102, 64.87745944080083, 64.5667932343804, 64.10079392474974, 63.65205384880911, 63.272350707628576, 63.01346220227821, 62.80635139799793, 62.46116672419744, 62.23679668622712, 62.046945115636866, 61.47739040386606, 61.08042802899551, 60.510873317224714, 60.234725578184324, 59.97583707283396, 59.54435623058336, 59.13013462202278, 58.92302381774248, 58.474283741801855, 58.077321366931315, 57.473248187780456, 57.11080428028995, 56.316879530548846, 55.712806351397994, 54.81532619951675], "calls": [0.0, 2.865032792544011, 6.316879530548844, 8.198136002761478, 10.303762512944425, 13.703831549879187, 16.4998274076631, 19.002416292716603, 21.34967207455989, 23.67966862271315, 25.69899896444598, 26.734552985847426, 27.632033137728683, 28.94373489817052, 30.56610286503279, 31.61891612012427, 33.275802554366585, 34.58750431480842, 35.65757680358992, 36.00276147739041, 37.2799447704522, 39.506385916465305, 40.24853296513635, 41.42216085605799, 43.04452882292026, 44.011045909561616, 45.44356230583362, 46.18570935450466, 46.82430100103556, 48.08422506040731, 49.03348291335865, 50.742147048671036, 51.69140490162237, 52.39903348291336, 53.279254401104595, 53.96962374870555, 55.41939937866759, 56.36865723161891, 57.16258198136003, 58.077321366931315, 59.45806006213324, 60.66620642043493, 61.68450120814636, 62.30583362098723, 63.134276838108384, 63.962720055229546, 65.0673110113911, 65.98205039696238, 66.51708664135313, 67.19019675526407, 67.94960303762512, 68.76078702105627, 69.65826717293753, 70.26234035208837, 70.90093199861927, 72.05730065585088, 72.55781843286158, 73.55885398688298, 74.2147048671039, 74.61166724197446, 75.19848118743528, 75.94062823610632, 76.64825681739731, 77.59751467034863, 78.18432861580946, 78.66758715913014, 79.4269934414912, 80.06558508802209, 80.73869520193303, 81.429064549534, 81.94684156023473, 82.61995167414567, 83.1377286848464, 83.8626164998274, 84.46668967897826, 85.10528132550915, 85.65757680358992, 86.24439074905074, 87.29720400414222, 88.12564722126338, 88.86779426993442, 89.38557128063513, 89.74801518812565, 90.52468070417673, 90.8526061442872, 91.16327235070763, 92.06075250258888, 92.59578874697964, 93.32067656196065, 93.7866758715913, 94.13186054539179, 94.77045219192269, 95.28822920262341, 95.51259924059372, 96.01311701760442, 96.41007939247498, 97.04867103900587, 97.48015188125648, 98.34311356575768, 99.01622367966863, 100.0], "rescued": [0, 6, 29, 37, 52, 67, 76, 86, 91, 100, 103, 105, 105, 107, 108, 109, 114, 116, 116, 116, 117, 122, 122, 123, 124, 126, 127, 127, 127, 129, 129, 132, 132, 132, 132, 132, 132, 132, 134, 134, 134, 134, 134, 134, 134, 134, 134, 134, 134, 134, 136, 137, 137, 138, 138, 138, 138, 138, 138, 139, 139, 139, 139, 140, 140, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 143, 143, 143, 143, 143, 143, 143, 143, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 31, 82, 105, 138, 189, 234, 281, 331, 374, 417, 439, 454, 484, 513, 537, 562, 581, 601, 611, 633, 670, 685, 709, 730, 744, 771, 783, 791, 817, 834, 867, 886, 894, 907, 921, 945, 964, 980, 997, 1026, 1043, 1060, 1069, 1086, 1103, 1127, 1147, 1161, 1174, 1194, 1206, 1219, 1239, 1249, 1275, 1284, 1305, 1318, 1329, 1342, 1366, 1384, 1406, 1415, 1427, 1442, 1461, 1477, 1503, 1518, 1536, 1552, 1568, 1581, 1598, 1606, 1624, 1653, 1679, 1701, 1716, 1728, 1748, 1761, 1772, 1806, 1829, 1862, 1878, 1893, 1918, 1942, 1954, 1980, 2003, 2038, 2059, 2105, 2140, 2192], "exact_score": [90.16223679668623, 90.05868139454608, 89.88608905764583, 89.73075595443561, 89.57542285122541, 89.45460821539524, 89.28201587849499, 89.19571971004487, 89.0058681394546, 88.88505350362443, 88.72972040041421, 88.574387297204, 88.52260959613393, 88.24646185709354, 88.02209181912323, 87.84949948222298, 87.5733517431826, 87.36624093890231, 87.05557473248187, 86.79668622713152, 86.46876078702105, 86.12357611322057, 85.91646530894027, 85.5712806351398, 85.39868829823956, 84.94994822229893, 84.62202278218847, 84.32861580945806, 83.98343113565757, 83.62098722816707, 83.31032102174663, 83.03417328270625, 82.68898860890576, 82.55091473938558, 82.2920262340352, 82.01587849499482, 81.6361753538143, 81.37728684846392, 81.08387987573352, 80.87676907145323, 80.54884363134276, 80.18639972385225, 79.91025198481188, 79.7204004142216, 79.4960303762513, 79.20262340352087, 78.90921643079048, 78.65032792544011, 78.32240248532965, 77.99447704521918, 77.7873662409389, 77.37314463237833, 77.11425612702796, 76.83810838798757, 76.54470141525715, 76.2167759751467, 75.9406282361063, 75.5781843286158, 75.30203658957542, 74.90507421470487, 74.55988954090438, 74.37003797031412, 74.09389023127373, 73.80048325854331, 73.42078011736278, 73.05833620987228, 72.67863306869174, 72.26441146013116, 71.88470831895064, 71.69485674836037, 71.34967207455989, 70.96996893337936, 70.53848809112876, 70.26234035208836, 69.86537797721782, 69.5029340697273, 68.95063859164652, 68.55367621677597, 68.19123230928547, 67.84604763548498, 67.4318260269244, 66.94856748360372, 66.37901277183292, 65.89575422851225, 65.46427338626164, 65.0673110113911, 64.70486710390058, 64.25612702795995, 63.73835001725923, 63.16879530548844, 62.720055229547796, 62.11598205039696, 61.51190887124611, 60.75250258888505, 60.079392474974114, 59.35450465999309, 58.49154297549188, 57.525025888850536, 56.627545736969275, 55.73006558508802, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 76.60232950206544, "threshold_area100": 72.30134622022783, "threshold_area25": 84.33206765619606}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.6976723670959473, "operator": "ge"}, {"budget": 0.02, "threshold": -1.4398016929626465, "operator": "ge"}, {"budget": 0.03, "threshold": -1.8675593137741089, "operator": "ge"}, {"budget": 0.04, "threshold": -2.6264865398406982, "operator": "ge"}, {"budget": 0.05, "threshold": -2.9865341186523438, "operator": "ge"}, {"budget": 0.06, "threshold": -3.3050546646118164, "operator": "ge"}, {"budget": 0.07, "threshold": -3.474799871444702, "operator": "ge"}, {"budget": 0.08, "threshold": -3.902754783630371, "operator": "ge"}, {"budget": 0.09, "threshold": -4.208681583404541, "operator": "ge"}, {"budget": 0.1, "threshold": -4.495203018188477, "operator": "ge"}, {"budget": 0.11, "threshold": -4.6576128005981445, "operator": "ge"}, {"budget": 0.12, "threshold": -4.807751178741455, "operator": "ge"}, {"budget": 0.13, "threshold": -4.985372543334961, "operator": "ge"}, {"budget": 0.14, "threshold": -5.153468132019043, "operator": "ge"}, {"budget": 0.15, "threshold": -5.2971272468566895, "operator": "ge"}, {"budget": 0.16, "threshold": -5.388047218322754, "operator": "ge"}, {"budget": 0.17, "threshold": -5.498413562774658, "operator": "ge"}, {"budget": 0.18, "threshold": -5.588211536407471, "operator": "ge"}, {"budget": 0.19, "threshold": -5.668261528015137, "operator": "ge"}, {"budget": 0.2, "threshold": -5.769033432006836, "operator": "ge"}, {"budget": 0.21, "threshold": -5.886878490447998, "operator": "ge"}, {"budget": 0.22, "threshold": -5.990330219268799, "operator": "ge"}, {"budget": 0.23, "threshold": -6.065248489379883, "operator": "ge"}, {"budget": 0.24, "threshold": -6.169803142547607, "operator": "ge"}, {"budget": 0.25, "threshold": -6.289610862731934, "operator": "ge"}, {"budget": 0.26, "threshold": -6.35134744644165, "operator": "ge"}, {"budget": 0.27, "threshold": -6.430642604827881, "operator": "ge"}, {"budget": 0.28, "threshold": -6.54632568359375, "operator": "ge"}, {"budget": 0.29, "threshold": -6.6231160163879395, "operator": "ge"}, {"budget": 0.3, "threshold": -6.710622787475586, "operator": "ge"}, {"budget": 0.31, "threshold": -6.791231632232666, "operator": "ge"}, {"budget": 0.32, "threshold": -6.934707164764404, "operator": "ge"}, {"budget": 0.33, "threshold": -7.022155284881592, "operator": "ge"}, {"budget": 0.34, "threshold": -7.0862627029418945, "operator": "ge"}, {"budget": 0.35, "threshold": -7.207590579986572, "operator": "ge"}, {"budget": 0.36, "threshold": -7.287831783294678, "operator": "ge"}, {"budget": 0.37, "threshold": -7.326312065124512, "operator": "ge"}, {"budget": 0.38, "threshold": -7.4098801612854, "operator": "ge"}, {"budget": 0.39, "threshold": -7.507169723510742, "operator": "ge"}, {"budget": 0.4, "threshold": -7.555656433105469, "operator": "ge"}, {"budget": 0.41, "threshold": -7.600688934326172, "operator": "ge"}, {"budget": 0.42, "threshold": -7.668326377868652, "operator": "ge"}, {"budget": 0.43, "threshold": -7.720714569091797, "operator": "ge"}, {"budget": 0.44, "threshold": -7.799441337585449, "operator": "ge"}, {"budget": 0.45, "threshold": -7.867159843444824, "operator": "ge"}, {"budget": 0.46, "threshold": -7.910549640655518, "operator": "ge"}, {"budget": 0.47, "threshold": -7.976573467254639, "operator": "ge"}, {"budget": 0.48, "threshold": -8.024951934814453, "operator": "ge"}, {"budget": 0.49, "threshold": -8.072383880615234, "operator": "ge"}, {"budget": 0.5, "threshold": -8.12543773651123, "operator": "ge"}, {"budget": 0.51, "threshold": -8.200143814086914, "operator": "ge"}, {"budget": 0.52, "threshold": -8.240706443786621, "operator": "ge"}, {"budget": 0.53, "threshold": -8.300896644592285, "operator": "ge"}, {"budget": 0.54, "threshold": -8.346303939819336, "operator": "ge"}, {"budget": 0.55, "threshold": -8.369534492492676, "operator": "ge"}, {"budget": 0.56, "threshold": -8.440593719482422, "operator": "ge"}, {"budget": 0.57, "threshold": -8.48016357421875, "operator": "ge"}, {"budget": 0.58, "threshold": -8.552322387695312, "operator": "ge"}, {"budget": 0.59, "threshold": -8.590997695922852, "operator": "ge"}, {"budget": 0.6, "threshold": -8.620197296142578, "operator": "ge"}, {"budget": 0.61, "threshold": -8.662248611450195, "operator": "ge"}, {"budget": 0.62, "threshold": -8.699065208435059, "operator": "ge"}, {"budget": 0.63, "threshold": -8.726043701171875, "operator": "ge"}, {"budget": 0.64, "threshold": -8.792896270751953, "operator": "ge"}, {"budget": 0.65, "threshold": -8.832438468933105, "operator": "ge"}, {"budget": 0.66, "threshold": -8.872638702392578, "operator": "ge"}, {"budget": 0.67, "threshold": -8.920948028564453, "operator": "ge"}, {"budget": 0.68, "threshold": -8.97069263458252, "operator": "ge"}, {"budget": 0.69, "threshold": -9.032288551330566, "operator": "ge"}, {"budget": 0.7, "threshold": -9.065893173217773, "operator": "gt"}, {"budget": 0.71, "threshold": -9.105063438415527, "operator": "ge"}, {"budget": 0.72, "threshold": -9.140876770019531, "operator": "ge"}, {"budget": 0.73, "threshold": -9.233823776245117, "operator": "ge"}, {"budget": 0.74, "threshold": -9.274125099182129, "operator": "ge"}, {"budget": 0.75, "threshold": -9.35504150390625, "operator": "ge"}, {"budget": 0.76, "threshold": -9.395730972290039, "operator": "ge"}, {"budget": 0.77, "threshold": -9.432644844055176, "operator": "ge"}, {"budget": 0.78, "threshold": -9.486382484436035, "operator": "ge"}, {"budget": 0.79, "threshold": -9.581178665161133, "operator": "ge"}, {"budget": 0.8, "threshold": -9.690715789794922, "operator": "ge"}, {"budget": 0.81, "threshold": -9.771196365356445, "operator": "ge"}, {"budget": 0.82, "threshold": -9.81947135925293, "operator": "ge"}, {"budget": 0.83, "threshold": -9.882538795471191, "operator": "ge"}, {"budget": 0.84, "threshold": -9.94917106628418, "operator": "ge"}, {"budget": 0.85, "threshold": -10.018437385559082, "operator": "ge"}, {"budget": 0.86, "threshold": -10.083978652954102, "operator": "ge"}, {"budget": 0.87, "threshold": -10.159429550170898, "operator": "ge"}, {"budget": 0.88, "threshold": -10.213481903076172, "operator": "ge"}, {"budget": 0.89, "threshold": -10.288779258728027, "operator": "ge"}, {"budget": 0.9, "threshold": -10.372308731079102, "operator": "ge"}, {"budget": 0.91, "threshold": -10.484280586242676, "operator": "ge"}, {"budget": 0.92, "threshold": -10.561793327331543, "operator": "ge"}, {"budget": 0.93, "threshold": -10.62614631652832, "operator": "ge"}, {"budget": 0.94, "threshold": -10.707978248596191, "operator": "ge"}, {"budget": 0.95, "threshold": -10.812309265136719, "operator": "ge"}, {"budget": 0.96, "threshold": -10.947454452514648, "operator": "ge"}, {"budget": 0.97, "threshold": -11.025057792663574, "operator": "ge"}, {"budget": 0.98, "threshold": -11.161039352416992, "operator": "ge"}, {"budget": 0.99, "threshold": -11.453421592712402, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.58333333333333, 98.16666666666667, 97.91666666666666, 97.5, 97.16666666666667, 96.75, 96.33333333333333, 95.83333333333333, 95.33333333333333, 95.16666666666667, 94.66666666666667, 94.41666666666666, 94.25, 94.16666666666667, 93.83333333333333, 93.66666666666667, 93.5, 93.16666666666666, 93.0, 92.58333333333333, 92.5, 92.25, 91.91666666666667, 91.75, 91.58333333333333, 91.33333333333333, 90.91666666666667, 90.58333333333333, 90.33333333333333, 90.16666666666666, 89.83333333333333, 89.33333333333333, 89.16666666666666, 88.91666666666667, 88.5, 88.08333333333334, 87.91666666666667, 87.5, 87.33333333333333, 87.08333333333333, 86.66666666666667, 86.25, 85.83333333333333, 85.66666666666667, 85.41666666666666, 85.16666666666666, 84.91666666666666, 84.58333333333333, 84.33333333333333, 84.0, 83.5, 83.16666666666667, 82.75, 82.5, 82.08333333333333, 81.5, 81.16666666666667, 80.58333333333333, 80.08333333333333, 79.66666666666666, 79.58333333333333, 79.25, 79.0, 78.66666666666666, 78.25, 78.0, 77.49999999999999, 77.0, 76.33333333333333, 76.16666666666666, 75.66666666666666, 75.41666666666667, 74.83333333333333, 74.41666666666666, 74.0, 73.5, 72.66666666666666, 72.16666666666667, 71.33333333333334, 70.83333333333333, 70.24999999999999, 69.66666666666667, 68.91666666666667, 68.16666666666666, 67.41666666666666, 66.58333333333333, 65.99999999999999, 64.99999999999999, 63.99999999999999, 63.33333333333333, 62.58333333333332, 61.83333333333333, 61.083333333333336, 60.25, 59.33333333333333, 58.5, 57.833333333333336, 56.833333333333336, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 69.91666666666667, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 4, 9, 12, 17, 22, 27, 32, 38, 44, 46, 52, 55, 57, 58, 62, 64, 66, 70, 73, 78, 79, 82, 86, 88, 90, 93, 98, 102, 105, 107, 111, 117, 119, 122, 127, 132, 134, 139, 141, 144, 149, 154, 159, 161, 164, 167, 170, 174, 177, 181, 187, 191, 196, 199, 204, 211, 215, 222, 228, 233, 234, 238, 241, 245, 250, 253, 259, 265, 273, 275, 281, 284, 291, 296, 301, 307, 317, 323, 333, 339, 346, 353, 362, 371, 380, 390, 397, 409, 421, 429, 438, 447, 456, 466, 477, 487, 495, 507, 517, 528], "exact_score": [98.91666666666666, 98.58333333333333, 98.16666666666667, 97.91666666666666, 97.5, 97.16666666666667, 96.75, 96.33333333333333, 95.83333333333333, 95.33333333333333, 95.16666666666667, 94.66666666666667, 94.41666666666666, 94.25, 94.16666666666667, 93.83333333333333, 93.66666666666667, 93.5, 93.16666666666666, 93.0, 92.58333333333333, 92.5, 92.25, 91.91666666666667, 91.75, 91.58333333333333, 91.33333333333333, 90.91666666666667, 90.58333333333333, 90.33333333333333, 90.16666666666666, 89.83333333333333, 89.33333333333333, 89.16666666666666, 88.91666666666667, 88.5, 88.08333333333334, 87.91666666666667, 87.5, 87.33333333333333, 87.08333333333333, 86.66666666666667, 86.25, 85.83333333333333, 85.66666666666667, 85.41666666666666, 85.16666666666666, 84.91666666666666, 84.58333333333333, 84.33333333333333, 84.0, 83.5, 83.16666666666667, 82.75, 82.5, 82.08333333333333, 81.5, 81.16666666666667, 80.58333333333333, 80.08333333333333, 79.66666666666666, 79.58333333333333, 79.25, 79.0, 78.66666666666666, 78.25, 78.0, 77.49999999999999, 77.0, 76.33333333333333, 76.16666666666666, 75.66666666666666, 75.41666666666667, 74.83333333333333, 74.41666666666666, 74.0, 73.5, 72.66666666666666, 72.16666666666667, 71.33333333333334, 70.83333333333333, 70.24999999999999, 69.58333333333333, 68.91666666666667, 68.16666666666666, 67.41666666666666, 66.58333333333333, 65.99999999999999, 64.99999999999999, 63.99999999999999, 63.33333333333333, 62.58333333333332, 61.83333333333333, 61.083333333333336, 60.25, 59.33333333333333, 58.5, 57.833333333333336, 56.833333333333336, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 81.67916666666666, "threshold_area100": 81.68, "threshold_area25": 94.80000000000001}, "test": {"score": [90.16223679668623, 89.88608905764583, 89.31653434587504, 89.19571971004487, 88.24646185709354, 87.62512944425266, 87.0037970314118, 86.45150155333103, 85.38142906454954, 84.51846738004832, 83.48291335864687, 82.84432171211597, 82.22298929927511, 81.73973075595443, 81.1356575768036, 80.56610286503279, 80.39351052813255, 79.96202968588194, 79.73765964791163, 79.4269934414912, 79.23714187090093, 78.77114256127028, 78.46047635484985, 78.30514325163962, 77.77010700724888, 77.44218156713842, 77.30410769761822, 77.06247842595788, 76.76907145322747, 76.45840524680703, 76.30307214359682, 76.06144287193648, 75.5781843286158, 75.26751812219537, 74.97411114946496, 74.66344494304452, 74.31826026924404, 74.12840869865379, 73.80048325854331, 73.47255781843286, 73.35174318260269, 73.09285467725233, 72.83396617190196, 72.66137383500173, 72.29892992751121, 72.04004142216085, 71.86744908526062, 71.66033828098031, 71.53952364515015, 71.36693130824992, 71.22885743872972, 70.71108042802899, 70.5730065585088, 70.27959958577839, 70.17604418363824, 70.08974801518812, 69.69278564031757, 69.45115636865722, 69.2095270969969, 69.05419399378667, 68.81256472212634, 68.63997238522609, 68.45012081463582, 68.3638246461857, 68.01863997238522, 67.88056610286503, 67.55264066275457, 67.34552985847428, 67.00034518467379, 66.68967897825337, 66.51708664135312, 66.37901277183292, 66.15464273386262, 65.5678287884018, 65.32619951674145, 64.84294097342078, 64.61857093545046, 64.25612702795995, 63.979979288919566, 63.53123921297894, 62.89264756644805, 62.426648256817394, 62.20227821884707, 61.90887124611667, 61.649982740766305, 61.2875388332758, 61.06316879530549, 60.57991025198481, 60.2865032792544, 59.7859855022437, 59.50983776320331, 59.04383845357266, 58.81946841560235, 58.56057991025199, 58.23265447014152, 57.88746979634104, 57.33517431826027, 57.07628581290991, 56.523990334829136, 55.626510182947875, 54.81532619951675], "calls": [0.0, 2.8995512599240594, 6.109768726268554, 8.04280289955126, 12.858129099068002, 15.567828788401794, 17.725232999654818, 19.27856403175699, 22.575077666551604, 25.146703486365208, 27.73558853986883, 29.237141870900935, 30.84225060407318, 32.37832240248533, 33.845357266137384, 35.381429064549536, 36.175353814290645, 37.60787021056265, 38.55712806351398, 39.368312046945114, 40.45564376941664, 41.95719710044874, 43.13082499137038, 43.872972040041425, 45.512599240593715, 46.75526406627546, 47.480151881256475, 48.37763203313773, 49.482222989299274, 50.396962374870554, 51.27718329306179, 52.140144977563, 53.727994477045215, 54.6427338626165, 55.31584397652745, 56.48947186744908, 57.33517431826027, 57.801173627890925, 59.078356920952714, 60.26924404556438, 60.89057645840524, 61.58094580600621, 62.40938902312737, 63.134276838108384, 64.46323783224025, 65.60234725578185, 66.25819813600276, 67.39730755954436, 67.91508457024507, 68.65723161891611, 69.22678633068692, 70.36589575422852, 70.79737659647911, 71.52226441146013, 72.2816706938212, 72.52329996548154, 73.76596479116327, 74.40455643769417, 75.23299965481533, 75.81981360027615, 76.37210907835691, 76.73455298584743, 77.3558853986883, 77.61477390403866, 78.66758715913014, 79.18536416983086, 79.9275112185019, 80.56610286503279, 81.3600276147739, 82.0676561960649, 82.4301001035554, 82.84432171211598, 83.4311356575768, 84.48394891266828, 85.01898515705902, 86.05453917846047, 86.53779772178115, 87.09009319986193, 87.79772178115292, 88.69520193303417, 89.8515705902658, 90.52468070417673, 90.99068001380739, 91.38764238867795, 91.85364169830859, 92.44045564376941, 92.83741801863997, 93.37245426303072, 93.75215740421126, 94.3562305833621, 94.75319295823266, 95.28822920262341, 95.54711770797375, 95.87504314808423, 96.23748705557473, 96.63444943044529, 97.34207801173628, 97.60096651708665, 98.15326199516741, 99.11977908180877, 100.0], "rescued": [0, 10, 27, 42, 69, 82, 86, 87, 100, 106, 115, 118, 120, 122, 123, 126, 126, 127, 129, 129, 130, 130, 130, 130, 130, 130, 133, 133, 133, 133, 133, 133, 134, 134, 134, 134, 134, 134, 135, 135, 135, 135, 136, 137, 137, 137, 137, 137, 137, 138, 138, 138, 138, 138, 138, 138, 138, 138, 139, 139, 139, 139, 139, 139, 140, 140, 141, 142, 142, 142, 142, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 26, 76, 98, 180, 229, 269, 302, 377, 433, 502, 542, 580, 610, 646, 682, 692, 718, 733, 751, 763, 790, 808, 817, 848, 867, 878, 892, 909, 927, 936, 950, 979, 997, 1014, 1032, 1052, 1063, 1083, 1102, 1109, 1124, 1140, 1151, 1172, 1187, 1197, 1209, 1216, 1227, 1235, 1265, 1273, 1290, 1296, 1301, 1324, 1338, 1353, 1362, 1376, 1386, 1397, 1402, 1423, 1431, 1451, 1464, 1484, 1502, 1512, 1521, 1534, 1568, 1582, 1610, 1623, 1644, 1660, 1686, 1723, 1750, 1763, 1780, 1795, 1816, 1829, 1857, 1874, 1903, 1920, 1947, 1960, 1975, 1994, 2014, 2046, 2061, 2093, 2145, 2192], "exact_score": [90.16223679668623, 90.05868139454608, 90.006903693476, 89.85157059026578, 89.74801518812563, 89.5063859164653, 89.31653434587504, 89.23023817742491, 89.23023817742491, 88.90231273731446, 88.69520193303417, 88.55712806351399, 88.35001725923368, 88.26372109078356, 87.95305488436314, 87.76320331377286, 87.52157404211253, 87.17638936831204, 86.91750086296169, 86.52053848809112, 86.24439074905074, 85.95098377632033, 85.62305833620987, 85.39868829823956, 84.91542975491888, 84.57024508111839, 84.29409734207802, 83.87987573351742, 83.34483948912667, 82.87884017949602, 82.48187780462547, 82.18847083189506, 81.8605453917846, 81.4981014842941, 81.10113910942354, 80.721435968243, 80.44528822920262, 80.18639972385225, 79.8584742837418, 79.53054884363134, 79.34069727304107, 79.06454953400069, 78.75388332758025, 78.47773558853987, 78.27062478425958, 77.92544011045909, 77.64929237141871, 77.39040386606834, 77.18329306178805, 76.9416637901277, 76.59647911632723, 76.35484984466689, 76.09596133931653, 75.76803589920607, 75.45736969278563, 75.12944425267519, 74.81877804625474, 74.47359337245426, 74.05937176389368, 73.81774249223334, 73.52433551950293, 73.29996548153261, 72.98929927511219, 72.71315153607179, 72.41974456334138, 72.16085605799101, 71.90196755264066, 71.69485674836037, 71.53952364515015, 71.24611667241973, 70.91819123230928, 70.5039696237487, 70.22782188470832, 69.95167414566794, 69.58923023817742, 69.29582326544701, 68.98515705902656, 68.53641698308594, 68.2257507766655, 67.9323438039351, 67.53538142906454, 67.10390058681395, 66.7069382119434, 66.32723507076285, 65.87849499482222, 65.34345875043148, 64.86020020711081, 64.30790472903003, 63.89368312046945, 63.410424577148774, 62.7890921643079, 62.20227821884707, 61.54642733862617, 60.94235415947531, 60.09665170866414, 59.302726958923024, 58.45702450811183, 57.611322057300654, 56.67932343803934, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 76.14718051323491, "threshold_area100": 71.4357956506731, "threshold_area25": 82.94269934414912}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.83333333333333, "calls": 1.0}, "test": {"score": 90.07594062823611, "calls": 1.2426648256817396}}, "old_normalized_interior": {"val": {"score": 98.83333333333333, "calls": 1.0}, "test": {"score": 90.07594062823611, "calls": 1.2426648256817396}}, "old_normalized_endpoints": {"val": {"score": 98.91666666666666, "calls": 0.0}, "test": {"score": 90.16223679668623, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.58333333333333, 98.5, 98.41666666666666, 98.41666666666666, 98.33333333333333, 98.08333333333333, 98.0, 97.91666666666666, 97.91666666666666, 97.74999999999999, 97.58333333333333, 97.5, 97.33333333333333, 97.16666666666667, 97.08333333333333, 96.75, 96.5, 96.33333333333333, 96.25, 96.08333333333333, 96.0, 95.83333333333333, 95.66666666666667, 95.58333333333333, 95.25, 95.0, 94.75, 94.33333333333334, 94.16666666666667, 93.58333333333333, 93.33333333333333, 92.91666666666667, 92.33333333333333, 91.91666666666667, 91.75, 91.41666666666667, 91.08333333333333, 90.83333333333333, 90.33333333333333, 89.66666666666666, 89.16666666666666, 88.25, 87.5, 86.58333333333333, 86.0, 85.33333333333333, 84.83333333333333, 84.08333333333333, 83.58333333333333, 83.08333333333333, 82.25, 81.66666666666667, 81.25, 80.33333333333333, 79.75, 79.08333333333333, 78.49999999999999, 77.83333333333333, 76.91666666666667, 76.0, 75.33333333333333, 74.66666666666666, 74.08333333333333, 73.25, 72.5, 71.66666666666666, 70.66666666666667, 69.75, 68.91666666666667, 68.08333333333333, 67.25, 66.25, 65.33333333333333, 64.58333333333333, 63.58333333333333, 62.58333333333332, 61.66666666666667, 60.83333333333333, 59.916666666666664, 58.916666666666664, 57.91666666666666, 56.91666666666666, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.58333333333333, 98.5, 98.41666666666666, 98.41666666666666, 98.33333333333333, 98.08333333333333, 98.0, 97.91666666666666, 97.91666666666666, 97.74999999999999, 97.58333333333333, 97.5, 97.33333333333333, 97.16666666666667, 97.08333333333333, 96.75, 96.5, 96.33333333333333, 96.25, 96.08333333333333, 96.0, 95.83333333333333, 95.66666666666667, 95.58333333333333, 95.16666666666667, 95.0, 94.75, 94.33333333333334, 94.16666666666667, 93.58333333333333, 93.33333333333333, 92.91666666666667, 92.33333333333333, 91.91666666666667, 91.75, 91.41666666666667, 91.08333333333333, 90.83333333333333, 90.33333333333333, 89.66666666666666, 89.08333333333334, 88.25, 87.5, 86.58333333333333, 86.0, 85.33333333333333, 84.83333333333333, 84.08333333333333, 83.58333333333333, 83.08333333333333, 82.25, 81.66666666666667, 81.25, 80.33333333333333, 79.75, 79.08333333333333, 78.49999999999999, 77.83333333333333, 76.91666666666667, 76.0, 75.33333333333333, 74.66666666666666, 74.08333333333333, 73.25, 72.5, 71.58333333333333, 70.66666666666667, 69.75, 68.91666666666667, 68.08333333333333, 67.25, 66.25, 65.33333333333333, 64.58333333333333, 63.58333333333333, 62.58333333333332, 61.66666666666667, 60.83333333333333, 59.916666666666664, 58.916666666666664, 57.91666666666666, 56.91666666666666, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.61666666666667, "exact_area100": 86.45833333333333}, "test": {"score": [90.16223679668623, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.05868139454608, 90.02416292716603, 89.93786675871591, 89.92060752502589, 89.88608905764583, 89.85157059026578, 89.81705212288574, 89.76527442181566, 89.7134967207456, 89.66171901967553, 89.60994131860545, 89.54090438384536, 89.47186744908525, 89.3683120469451, 89.29927511218501, 89.2129789437349, 89.16120124266482, 89.07490507421471, 89.02312737314463, 88.95409043838454, 88.85053503624438, 88.72972040041421, 88.59164653089402, 88.574387297204, 88.38453572661373, 88.19468415602347, 88.03935105281326, 87.79772178115292, 87.65964791163272, 87.55609250949257, 87.40075940628236, 87.31446323783224, 87.22816706938211, 87.17638936831204, 87.09009319986191, 86.95201933034173, 86.72764929237141, 86.62409389023126, 86.46876078702105, 86.10631687953054, 85.83016914049016, 85.46772523299964, 85.05350362443907, 84.79461511908872, 84.67380048325855, 84.38039351052812, 84.05246807041767, 83.60372799447704, 83.24128408698654, 82.75802554366585, 82.18847083189506, 81.96410079392474, 81.67069382119433, 81.2219537452537, 80.73869520193303, 80.53158439765275, 80.2209181912323, 79.75491888160165, 79.3579565067311, 78.54677252329995, 78.18432861580945, 77.94269934414912, 77.18329306178805, 76.7172937521574, 76.35484984466689, 75.80255436658612, 75.42285122540558, 75.04314808422505, 74.37003797031412, 74.02485329651363, 73.61063168795306, 72.95478080773213, 72.143596824301, 71.29789437348981, 70.48671039005868, 69.52019330341732, 68.89886089057646, 68.2084915429755, 67.05212288574387, 66.48256817397306, 65.70590265792198, 65.23990334829134, 64.68760787021056, 63.703831549879176, 63.35864687607871, 62.09872281670694, 60.97687262685536, 60.39005868139454, 59.35450465999309, 59.181912323092845, 58.301691404901625, 57.50776665516051, 56.627545736969275, 55.95443562305833, 54.81532619951675], "calls": [0.0, 1.2426648256817396, 2.2609596133931653, 3.2792544011045908, 4.176734552985847, 5.471177079737659, 6.127027959958578, 7.059026579219882, 8.491542975491887, 9.216430790472902, 10.148429409734208, 11.132205730065586, 12.15050051777701, 13.358646876078703, 14.22160856057991, 15.395236451501553, 15.982050396962375, 17.0693821194339, 17.759751467034864, 18.536416983085953, 19.175008629616844, 19.968933379357956, 21.10804280289955, 22.59233690024163, 23.23092854677252, 24.266482568173974, 25.474628926475663, 26.613738350017258, 27.37314463237832, 27.994477045219192, 29.219882637210908, 30.600621332412842, 31.411805315843978, 32.96513634794615, 33.776320331377285, 34.5702450811184, 35.43320676561961, 36.12357611322057, 36.41698308595098, 37.141870900932, 37.93579565067311, 38.781498101484296, 39.62720055229548, 40.09319986192613, 40.81808767690715, 42.3541594753193, 43.527787366240936, 44.61511908871246, 46.358301691404904, 47.63548498446669, 48.20503969623749, 49.309630652399036, 50.327925440110455, 51.950293406972726, 53.10666206420434, 54.47014152571625, 56.316879530548846, 57.38695201933034, 58.04280289955126, 59.302726958923024, 60.424577148774596, 60.976872626855375, 62.271315153607176, 63.34138764238868, 64.22160856057991, 65.98205039696238, 66.77597514670349, 67.39730755954436, 68.67449085260614, 69.69278564031757, 70.33137728684846, 71.22885743872972, 71.93648602002071, 72.69589230238176, 73.69692785640318, 74.24922333448394, 74.88781498101484, 75.90610976872627, 77.52847773558854, 78.73662409389023, 79.91025198481188, 81.34276838108389, 82.136693130825, 83.08595098377633, 84.60476354849845, 85.36416983085951, 86.26164998274076, 86.88298239558164, 87.62512944425268, 88.95409043838454, 89.42008974801519, 91.11149464963756, 92.59578874697964, 93.32067656196065, 94.66689678978253, 95.01208146358302, 96.04763548498447, 96.9968933379358, 97.96341042457715, 98.75733517431826, 100.0], "exact_score": [90.16223679668623, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.04142216085606, 89.97238522609597, 89.93786675871591, 89.90334829133586, 89.85157059026578, 89.81705212288574, 89.78253365550569, 89.73075595443561, 89.69623748705557, 89.60994131860545, 89.55816361753539, 89.42008974801519, 89.31653434587504, 89.2129789437349, 89.16120124266482, 89.16120124266482, 89.05764584052469, 88.95409043838454, 88.88505350362443, 88.78149810148429, 88.69520193303417, 88.574387297204, 88.41905419399379, 88.28098032447359, 88.09112875388332, 87.97031411805315, 87.79772178115292, 87.64238867794269, 87.46979634104245, 87.31446323783224, 87.19364860200207, 87.09009319986191, 86.90024162927166, 86.65861235761132, 86.39972385226096, 86.19261304798067, 85.98550224370037, 85.70935450465998, 85.39868829823956, 85.12254055919917, 84.96720745598894, 84.74283741801864, 84.46668967897824, 84.1387642388678, 83.84535726613737, 83.58646876078701, 83.2930617880566, 82.9306178805661, 82.51639627200552, 82.30928546772522, 81.96410079392474, 81.70521228857439, 81.37728684846392, 80.96306523990334, 80.53158439765275, 80.30721435968242, 79.91025198481188, 79.40973420780118, 79.06454953400069, 78.54677252329995, 78.09803244735933, 77.56299620296858, 76.99344149119779, 76.54470141525715, 75.90610976872627, 75.40559199171557, 74.87055574732481, 74.19744563341388, 73.5070762858129, 72.88574387297204, 72.45426303072144, 71.81567138419054, 71.10804280289955, 70.46945115636866, 69.79634104245771, 69.00241629271659, 68.27752847773559, 67.53538142906454, 66.77597514670349, 65.96479116327234, 65.15360717984122, 64.4287193648602, 63.68657231618915, 62.96168450120814, 62.20227821884707, 61.44287193648601, 60.648947186744905, 59.82050396962375, 59.19917155678287, 58.33620987228167, 57.50776665516051, 56.61028650327925, 55.712806351397994, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 89.69623748705557, "exact_area100": 80.22881353104856}}, "old_normalized_endpoint_budget": 0}, {"seed": 2027, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.0, "validation_area100": 0.8654, "training_seconds": 3.1853928880300373, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.005822875536978245, "operator": "ge"}, {"budget": 0.02, "threshold": -0.008194983005523682, "operator": "ge"}, {"budget": 0.03, "threshold": -0.009852077811956406, "operator": "ge"}, {"budget": 0.04, "threshold": -0.011123085394501686, "operator": "ge"}, {"budget": 0.05, "threshold": -0.013215620070695877, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01507690828293562, "operator": "ge"}, {"budget": 0.07, "threshold": -0.016235405579209328, "operator": "ge"}, {"budget": 0.08, "threshold": -0.018723582848906517, "operator": "ge"}, {"budget": 0.09, "threshold": -0.022372635081410408, "operator": "ge"}, {"budget": 0.1, "threshold": -0.024574039503932, "operator": "ge"}, {"budget": 0.11, "threshold": -0.027492020279169083, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03147419914603233, "operator": "ge"}, {"budget": 0.13, "threshold": -0.03513000160455704, "operator": "ge"}, {"budget": 0.14, "threshold": -0.04340781271457672, "operator": "ge"}, {"budget": 0.15, "threshold": -0.046285904943943024, "operator": "ge"}, {"budget": 0.16, "threshold": -0.050531066954135895, "operator": "ge"}, {"budget": 0.17, "threshold": -0.05640571564435959, "operator": "ge"}, {"budget": 0.18, "threshold": -0.061027225106954575, "operator": "ge"}, {"budget": 0.19, "threshold": -0.0659809559583664, "operator": "ge"}, {"budget": 0.2, "threshold": -0.07032903283834457, "operator": "ge"}, {"budget": 0.21, "threshold": -0.07364045828580856, "operator": "ge"}, {"budget": 0.22, "threshold": -0.08061794191598892, "operator": "ge"}, {"budget": 0.23, "threshold": -0.09105519950389862, "operator": "ge"}, {"budget": 0.24, "threshold": -0.09580812603235245, "operator": "ge"}, {"budget": 0.25, "threshold": -0.10297761112451553, "operator": "ge"}, {"budget": 0.26, "threshold": -0.11240417510271072, "operator": "ge"}, {"budget": 0.27, "threshold": -0.12055569887161255, "operator": "ge"}, {"budget": 0.28, "threshold": -0.12937520444393158, "operator": "ge"}, {"budget": 0.29, "threshold": -0.1350562572479248, "operator": "ge"}, {"budget": 0.3, "threshold": -0.14734941720962524, "operator": "ge"}, {"budget": 0.31, "threshold": -0.15704752504825592, "operator": "ge"}, {"budget": 0.32, "threshold": -0.16756822168827057, "operator": "ge"}, {"budget": 0.33, "threshold": -0.18269264698028564, "operator": "ge"}, {"budget": 0.34, "threshold": -0.195773646235466, "operator": "ge"}, {"budget": 0.35, "threshold": -0.20113445818424225, "operator": "ge"}, {"budget": 0.36, "threshold": -0.21036337316036224, "operator": "ge"}, {"budget": 0.37, "threshold": -0.22073280811309814, "operator": "ge"}, {"budget": 0.38, "threshold": -0.227500781416893, "operator": "ge"}, {"budget": 0.39, "threshold": -0.23879553377628326, "operator": "ge"}, {"budget": 0.4, "threshold": -0.24838148057460785, "operator": "ge"}, {"budget": 0.41, "threshold": -0.2669059932231903, "operator": "ge"}, {"budget": 0.42, "threshold": -0.28264063596725464, "operator": "ge"}, {"budget": 0.43, "threshold": -0.292635440826416, "operator": "ge"}, {"budget": 0.44, "threshold": -0.3045767843723297, "operator": "ge"}, {"budget": 0.45, "threshold": -0.3175235092639923, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3345573842525482, "operator": "ge"}, {"budget": 0.47, "threshold": -0.34530431032180786, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3573724925518036, "operator": "ge"}, {"budget": 0.49, "threshold": -0.37014347314834595, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3872606158256531, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3973255455493927, "operator": "ge"}, {"budget": 0.52, "threshold": -0.41168510913848877, "operator": "ge"}, {"budget": 0.53, "threshold": -0.42062950134277344, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4306088387966156, "operator": "ge"}, {"budget": 0.55, "threshold": -0.44050878286361694, "operator": "ge"}, {"budget": 0.56, "threshold": -0.46700894832611084, "operator": "ge"}, {"budget": 0.57, "threshold": -0.481801301240921, "operator": "ge"}, {"budget": 0.58, "threshold": -0.4916684925556183, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5019019246101379, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5064781308174133, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5184428691864014, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5352264642715454, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5514137744903564, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5647250413894653, "operator": "ge"}, {"budget": 0.65, "threshold": -0.5757407546043396, "operator": "ge"}, {"budget": 0.66, "threshold": -0.5851686596870422, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6003575325012207, "operator": "ge"}, {"budget": 0.68, "threshold": -0.608793318271637, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6232461333274841, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6415075063705444, "operator": "ge"}, {"budget": 0.71, "threshold": -0.6512153148651123, "operator": "ge"}, {"budget": 0.72, "threshold": -0.6690859198570251, "operator": "ge"}, {"budget": 0.73, "threshold": -0.6856300830841064, "operator": "ge"}, {"budget": 0.74, "threshold": -0.6965747475624084, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7133986353874207, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7245059013366699, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7382105588912964, "operator": "ge"}, {"budget": 0.78, "threshold": -0.758280873298645, "operator": "ge"}, {"budget": 0.79, "threshold": -0.7702246904373169, "operator": "ge"}, {"budget": 0.8, "threshold": -0.7808175683021545, "operator": "ge"}, {"budget": 0.81, "threshold": -0.7941498756408691, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8038976788520813, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8226273059844971, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8297353386878967, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8392680287361145, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8461469411849976, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8564870953559875, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8707003593444824, "operator": "ge"}, {"budget": 0.89, "threshold": -0.8805455565452576, "operator": "ge"}, {"budget": 0.9, "threshold": -0.8995753526687622, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9097110033035278, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9156385660171509, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9300254583358765, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9409782886505127, "operator": "ge"}, {"budget": 0.95, "threshold": -0.94880610704422, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9540613889694214, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9617116451263428, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9678224921226501, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9738499522209167, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.08333333333333, 98.08333333333333, 98.0, 97.74999999999999, 97.33333333333333, 97.08333333333333, 96.91666666666666, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.5, 95.33333333333333, 95.16666666666667, 94.91666666666666, 94.66666666666667, 94.25, 94.08333333333333, 93.66666666666667, 93.25, 92.91666666666667, 92.41666666666666, 91.91666666666667, 91.66666666666666, 91.33333333333333, 90.75, 90.16666666666666, 89.75, 89.5, 88.91666666666667, 88.33333333333333, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.74999999999999, 84.16666666666667, 83.5, 82.75, 82.0, 81.41666666666667, 80.66666666666666, 80.08333333333333, 79.33333333333333, 78.66666666666666, 78.0, 77.08333333333333, 76.25, 75.5, 74.83333333333333, 73.83333333333333, 73.0, 72.16666666666667, 71.5, 70.41666666666666, 69.66666666666667, 68.91666666666667, 68.0, 67.08333333333333, 66.08333333333334, 65.16666666666666, 64.33333333333333, 63.74999999999999, 62.83333333333333, 61.83333333333333, 60.83333333333333, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.08333333333333, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 3, 4, 6, 7, 8, 9, 9, 9, 10, 10, 11, 14, 19, 22, 24, 24, 25, 28, 34, 36, 38, 41, 43, 45, 48, 51, 56, 58, 63, 68, 72, 78, 84, 87, 91, 98, 105, 110, 113, 120, 127, 135, 141, 148, 156, 163, 170, 177, 185, 194, 203, 210, 220, 227, 236, 244, 252, 263, 273, 282, 290, 302, 312, 322, 330, 343, 352, 361, 372, 383, 395, 406, 416, 424, 435, 447, 459, 471, 482, 494, 504, 516, 528], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.08333333333333, 98.08333333333333, 98.0, 97.74999999999999, 97.33333333333333, 97.08333333333333, 96.91666666666666, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.5, 95.33333333333333, 95.16666666666667, 94.91666666666666, 94.66666666666667, 94.25, 94.08333333333333, 93.66666666666667, 93.25, 92.91666666666667, 92.41666666666666, 91.91666666666667, 91.66666666666666, 91.33333333333333, 90.75, 90.16666666666666, 89.75, 89.5, 88.91666666666667, 88.33333333333333, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.74999999999999, 84.16666666666667, 83.5, 82.75, 82.0, 81.33333333333333, 80.66666666666666, 80.08333333333333, 79.33333333333333, 78.66666666666666, 78.0, 77.08333333333333, 76.25, 75.5, 74.83333333333333, 73.83333333333333, 73.0, 72.16666666666667, 71.41666666666666, 70.41666666666666, 69.66666666666667, 68.91666666666667, 68.0, 67.08333333333333, 66.08333333333334, 65.16666666666666, 64.33333333333333, 63.74999999999999, 62.83333333333333, 61.83333333333333, 60.83333333333333, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.08333333333333, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.53999999999999, "threshold_area100": 86.54166666666667, "threshold_area25": 98.65833333333333}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.09319986192614, 90.09319986192614, 90.05868139454608, 90.006903693476, 89.98964445978598, 89.95512599240594, 89.90334829133586, 89.8688298239558, 89.76527442181566, 89.7134967207456, 89.67897825336554, 89.60994131860545, 89.5063859164653, 89.42008974801519, 89.40283051432516, 89.29927511218501, 89.2129789437349, 89.04038660683466, 88.95409043838454, 88.88505350362443, 88.79875733517432, 88.74697963410424, 88.60890576458405, 88.53986882982396, 88.36727649292371, 88.19468415602347, 88.02209181912323, 87.7114256127028, 87.36624093890231, 87.29720400414222, 87.14187090093199, 86.96927856403175, 86.88298239558164, 86.79668622713152, 86.62409389023126, 86.24439074905074, 85.96824301001035, 85.81290990680014, 85.62305833620987, 85.27787366240939, 85.07076285812909, 84.74283741801864, 84.44943044528823, 84.27683810838799, 84.01794960303762, 83.7935795650673, 83.44839489126683, 83.25854332067657, 82.94787711425612, 82.75802554366585, 82.17121159820505, 81.96410079392474, 81.68795305488436, 81.51536071798412, 81.39454608215395, 81.04936140835346, 80.58336209872282, 79.94477045219192, 79.53054884363134, 79.13358646876078, 78.8574387297204, 78.44321712115982, 78.23610631687953, 77.5802554366586, 76.76907145322747, 76.51018294787711, 75.90610976872627, 75.12944425267519, 74.80151881256472, 74.0766309975837, 73.47255781843286, 72.74767000345183, 71.85018985157059, 71.36693130824992, 70.95270969968934, 70.27959958577839, 69.74456334138763, 68.72626855367622, 68.29478771142561, 67.65619606489471, 67.2074559889541, 66.53434587504314, 65.68864342423197, 65.0673110113911, 63.703831549879176, 62.909906800138074, 62.443907490507414, 60.959613393165334, 59.7169485674836, 58.81946841560235, 58.2844321712116, 57.6458405246807, 56.83465654124957, 56.07525025888851, 54.81532619951675], "calls": [0.0, 0.7594062823610631, 1.8467380048325857, 2.6924404556437693, 3.40006903693476, 4.452882292026234, 5.091473938557129, 5.453917846047635, 6.575768035899205, 7.973765964791163, 8.767690714532275, 9.509837763203313, 10.752502588885053, 11.56368657231619, 13.531239212978944, 14.135312392129789, 14.963755609250951, 16.25819813600276, 17.12115982050397, 17.966862271315154, 18.70900931998619, 19.27856403175699, 20.538488091128755, 22.143596824301, 22.85122540559199, 23.558853986882983, 24.352778736624096, 25.181221953745254, 26.19951674145668, 26.544701415257162, 27.925440110459093, 29.237141870900935, 30.462547462892648, 32.18847083189507, 33.759061097687265, 34.552985847428374, 35.329651363479464, 36.19261304798067, 36.76216775975147, 37.573351743182606, 38.48809112875388, 39.972385226095966, 41.16327235070763, 41.73282706247843, 42.76838108387987, 43.821194338971345, 44.8567483603728, 45.875043148084224, 46.789782533655504, 47.87711425612703, 49.05074214704867, 49.896444597859855, 51.08733172247152, 51.55333103210218, 52.39903348291336, 53.24473593372454, 55.38488091128754, 56.28236106316879, 57.21435968243011, 58.06006213324129, 58.26717293752157, 59.285467725233, 60.510873317224714, 62.08146358301691, 63.2896099413186, 64.23886779426994, 64.94649637556093, 66.1373835001726, 66.5688643424232, 67.7079737659648, 69.15774939592683, 69.98619261304798, 71.31515360717984, 72.48878149810149, 73.19641007939246, 74.38729720400414, 75.33655505695546, 76.49292371418709, 77.92544011045909, 78.66758715913014, 79.25440110459095, 80.27269589230238, 81.08387987573352, 82.5681739730756, 83.2067656196065, 84.0697273041077, 84.77735588539869, 85.70935450466, 86.93476009665171, 87.76320331377286, 89.59268208491544, 90.61097687262686, 91.1460131170176, 92.80289955125993, 94.33897134967208, 95.51259924059372, 96.11667241974456, 96.9278564031757, 97.79081808767691, 98.6365205384881, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 5, 7, 7, 8, 9, 10, 11, 12, 12, 13, 14, 16, 16, 17, 22, 24, 25, 29, 29, 30, 31, 35, 39, 43, 47, 48, 50, 55, 62, 64, 68, 68, 71, 72, 75, 77, 82, 89, 90, 93, 97, 100, 102, 105, 108, 109, 110, 113, 118, 120, 122, 125, 126, 128, 130, 135, 138, 139, 139, 141, 144, 144, 144, 144, 144, 144], "harmed": [0, 0, 0, 0, 0, 2, 4, 4, 6, 9, 10, 12, 15, 17, 23, 26, 28, 33, 39, 44, 45, 51, 56, 66, 71, 75, 80, 83, 92, 96, 106, 117, 127, 145, 166, 171, 182, 192, 198, 204, 215, 238, 255, 264, 276, 297, 311, 330, 348, 363, 380, 394, 418, 429, 448, 460, 498, 514, 534, 548, 556, 578, 610, 654, 680, 707, 723, 750, 763, 804, 853, 873, 915, 961, 983, 1029, 1067, 1111, 1166, 1197, 1222, 1262, 1296, 1360, 1387, 1426, 1455, 1495, 1546, 1584, 1668, 1717, 1745, 1831, 1905, 1960, 1991, 2028, 2075, 2119, 2192], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.09319986192614, 90.07594062823611, 90.05868139454608, 90.006903693476, 89.97238522609597, 89.95512599240594, 89.88608905764583, 89.85157059026578, 89.78253365550569, 89.74801518812563, 89.67897825336554, 89.62720055229548, 89.54090438384536, 89.42008974801519, 89.35105281325508, 89.24749741111494, 89.16120124266482, 89.05764584052469, 88.91957197100449, 88.85053503624438, 88.76423886779426, 88.66068346565412, 88.47083189506385, 88.33275802554367, 88.26372109078356, 88.09112875388332, 87.93579565067311, 87.76320331377286, 87.5733517431826, 87.34898170521228, 87.2109078356921, 87.02105626510182, 86.84846392820158, 86.72764929237141, 86.46876078702105, 86.24439074905074, 86.02002071108042, 85.77839143942009, 85.5712806351398, 85.26061442871936, 85.01898515705902, 84.72557818432861, 84.39765274421815, 84.24231964100794, 84.01794960303762, 83.72454263030721, 83.46565412495684, 83.12046945115637, 82.7925440110459, 82.56817397307559, 82.27476700034518, 82.01587849499482, 81.70521228857439, 81.51536071798412, 81.20469451156367, 80.70417673455297, 80.39351052813255, 79.9275112185019, 79.61684501208146, 79.2198826372109, 78.84017949603037, 78.47773558853987, 78.02899551259924, 77.39040386606834, 76.83810838798757, 76.51018294787711, 76.09596133931653, 75.45736969278563, 74.95685191577493, 74.33551950293406, 73.67966862271315, 73.02381774249223, 72.43700379703141, 71.86744908526062, 71.10804280289955, 70.43493268898861, 69.77908180876769, 69.19226786330687, 68.39834311356576, 67.7079737659648, 67.05212288574387, 66.3617535381429, 65.63686572316189, 64.9292371418709, 64.11805315843976, 63.410424577148774, 62.59924059371764, 61.73627890921642, 60.787021056265104, 59.99309630652399, 59.19917155678287, 58.387987573351744, 57.59406282361063, 56.6448049706593, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.1489128708367, "threshold_area100": 80.41750086296166, "threshold_area25": 89.75146703486365}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.005822875536978245, "operator": "ge"}, {"budget": 0.02, "threshold": -0.008194983005523682, "operator": "ge"}, {"budget": 0.03, "threshold": -0.009852077811956406, "operator": "ge"}, {"budget": 0.04, "threshold": -0.011123085394501686, "operator": "ge"}, {"budget": 0.05, "threshold": -0.013215620070695877, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01507690828293562, "operator": "ge"}, {"budget": 0.07, "threshold": -0.016235405579209328, "operator": "ge"}, {"budget": 0.08, "threshold": -0.018723582848906517, "operator": "ge"}, {"budget": 0.09, "threshold": -0.022372635081410408, "operator": "ge"}, {"budget": 0.1, "threshold": -0.024574039503932, "operator": "ge"}, {"budget": 0.11, "threshold": -0.027492020279169083, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03147419914603233, "operator": "ge"}, {"budget": 0.13, "threshold": -0.03513000160455704, "operator": "ge"}, {"budget": 0.14, "threshold": -0.04340781271457672, "operator": "ge"}, {"budget": 0.15, "threshold": -0.046285904943943024, "operator": "ge"}, {"budget": 0.16, "threshold": -0.050531066954135895, "operator": "ge"}, {"budget": 0.17, "threshold": -0.05640571564435959, "operator": "ge"}, {"budget": 0.18, "threshold": -0.061027225106954575, "operator": "ge"}, {"budget": 0.19, "threshold": -0.0659809559583664, "operator": "ge"}, {"budget": 0.2, "threshold": -0.07032903283834457, "operator": "ge"}, {"budget": 0.21, "threshold": -0.07364045828580856, "operator": "ge"}, {"budget": 0.22, "threshold": -0.08061794191598892, "operator": "ge"}, {"budget": 0.23, "threshold": -0.09105519950389862, "operator": "ge"}, {"budget": 0.24, "threshold": -0.09580812603235245, "operator": "ge"}, {"budget": 0.25, "threshold": -0.10297761112451553, "operator": "ge"}, {"budget": 0.26, "threshold": -0.11240417510271072, "operator": "ge"}, {"budget": 0.27, "threshold": -0.12055569887161255, "operator": "ge"}, {"budget": 0.28, "threshold": -0.12937520444393158, "operator": "ge"}, {"budget": 0.29, "threshold": -0.1350562572479248, "operator": "ge"}, {"budget": 0.3, "threshold": -0.14734941720962524, "operator": "ge"}, {"budget": 0.31, "threshold": -0.15704752504825592, "operator": "ge"}, {"budget": 0.32, "threshold": -0.16756822168827057, "operator": "ge"}, {"budget": 0.33, "threshold": -0.18269264698028564, "operator": "ge"}, {"budget": 0.34, "threshold": -0.195773646235466, "operator": "ge"}, {"budget": 0.35, "threshold": -0.20113445818424225, "operator": "ge"}, {"budget": 0.36, "threshold": -0.21036337316036224, "operator": "ge"}, {"budget": 0.37, "threshold": -0.22073280811309814, "operator": "ge"}, {"budget": 0.38, "threshold": -0.227500781416893, "operator": "ge"}, {"budget": 0.39, "threshold": -0.23879553377628326, "operator": "ge"}, {"budget": 0.4, "threshold": -0.24838148057460785, "operator": "ge"}, {"budget": 0.41, "threshold": -0.2669059932231903, "operator": "ge"}, {"budget": 0.42, "threshold": -0.28264063596725464, "operator": "ge"}, {"budget": 0.43, "threshold": -0.292635440826416, "operator": "ge"}, {"budget": 0.44, "threshold": -0.3045767843723297, "operator": "ge"}, {"budget": 0.45, "threshold": -0.3175235092639923, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3345573842525482, "operator": "ge"}, {"budget": 0.47, "threshold": -0.34530431032180786, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3573724925518036, "operator": "ge"}, {"budget": 0.49, "threshold": -0.37014347314834595, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3872606158256531, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3973255455493927, "operator": "ge"}, {"budget": 0.52, "threshold": -0.41168510913848877, "operator": "ge"}, {"budget": 0.53, "threshold": -0.42062950134277344, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4306088387966156, "operator": "ge"}, {"budget": 0.55, "threshold": -0.44050878286361694, "operator": "ge"}, {"budget": 0.56, "threshold": -0.46700894832611084, "operator": "ge"}, {"budget": 0.57, "threshold": -0.481801301240921, "operator": "ge"}, {"budget": 0.58, "threshold": -0.4916684925556183, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5019019246101379, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5064781308174133, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5184428691864014, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5352264642715454, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5514137744903564, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5647250413894653, "operator": "ge"}, {"budget": 0.65, "threshold": -0.5757407546043396, "operator": "ge"}, {"budget": 0.66, "threshold": -0.5851686596870422, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6003575325012207, "operator": "ge"}, {"budget": 0.68, "threshold": -0.608793318271637, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6232461333274841, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6415075063705444, "operator": "ge"}, {"budget": 0.71, "threshold": -0.6512153148651123, "operator": "ge"}, {"budget": 0.72, "threshold": -0.6690859198570251, "operator": "ge"}, {"budget": 0.73, "threshold": -0.6856300830841064, "operator": "ge"}, {"budget": 0.74, "threshold": -0.6965747475624084, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7133986353874207, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7245059013366699, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7382105588912964, "operator": "ge"}, {"budget": 0.78, "threshold": -0.758280873298645, "operator": "ge"}, {"budget": 0.79, "threshold": -0.7702246904373169, "operator": "ge"}, {"budget": 0.8, "threshold": -0.7808175683021545, "operator": "ge"}, {"budget": 0.81, "threshold": -0.7941498756408691, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8038976788520813, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8226273059844971, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8297353386878967, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8392680287361145, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8461469411849976, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8564870953559875, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8707003593444824, "operator": "ge"}, {"budget": 0.89, "threshold": -0.8805455565452576, "operator": "ge"}, {"budget": 0.9, "threshold": -0.8995753526687622, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9097110033035278, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9156385660171509, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9300254583358765, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9409782886505127, "operator": "ge"}, {"budget": 0.95, "threshold": -0.94880610704422, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9540613889694214, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9617116451263428, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9678224921226501, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9738499522209167, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.08333333333333, 98.08333333333333, 98.0, 97.74999999999999, 97.33333333333333, 97.08333333333333, 96.91666666666666, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.5, 95.33333333333333, 95.16666666666667, 94.91666666666666, 94.66666666666667, 94.25, 94.08333333333333, 93.66666666666667, 93.25, 92.91666666666667, 92.41666666666666, 91.91666666666667, 91.66666666666666, 91.33333333333333, 90.75, 90.16666666666666, 89.75, 89.5, 88.91666666666667, 88.33333333333333, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.74999999999999, 84.16666666666667, 83.5, 82.75, 82.0, 81.41666666666667, 80.66666666666666, 80.08333333333333, 79.33333333333333, 78.66666666666666, 78.0, 77.08333333333333, 76.25, 75.5, 74.83333333333333, 73.83333333333333, 73.0, 72.16666666666667, 71.5, 70.41666666666666, 69.66666666666667, 68.91666666666667, 68.0, 67.08333333333333, 66.08333333333334, 65.16666666666666, 64.33333333333333, 63.74999999999999, 62.83333333333333, 61.83333333333333, 60.83333333333333, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.08333333333333, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 3, 3, 3, 4, 6, 7, 8, 9, 9, 9, 10, 10, 11, 14, 19, 22, 24, 24, 25, 28, 34, 36, 38, 41, 43, 45, 48, 51, 56, 58, 63, 68, 72, 78, 84, 87, 91, 98, 105, 110, 113, 120, 127, 135, 141, 148, 156, 163, 170, 177, 185, 194, 203, 210, 220, 227, 236, 244, 252, 263, 273, 282, 290, 302, 312, 322, 330, 343, 352, 361, 372, 383, 395, 406, 416, 424, 435, 447, 459, 471, 482, 494, 504, 516, 528], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.08333333333333, 98.08333333333333, 98.0, 97.74999999999999, 97.33333333333333, 97.08333333333333, 96.91666666666666, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.5, 95.33333333333333, 95.16666666666667, 94.91666666666666, 94.66666666666667, 94.25, 94.08333333333333, 93.66666666666667, 93.25, 92.91666666666667, 92.41666666666666, 91.91666666666667, 91.66666666666666, 91.33333333333333, 90.75, 90.16666666666666, 89.75, 89.5, 88.91666666666667, 88.33333333333333, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.74999999999999, 84.16666666666667, 83.5, 82.75, 82.0, 81.33333333333333, 80.66666666666666, 80.08333333333333, 79.33333333333333, 78.66666666666666, 78.0, 77.08333333333333, 76.25, 75.5, 74.83333333333333, 73.83333333333333, 73.0, 72.16666666666667, 71.41666666666666, 70.41666666666666, 69.66666666666667, 68.91666666666667, 68.0, 67.08333333333333, 66.08333333333334, 65.16666666666666, 64.33333333333333, 63.74999999999999, 62.83333333333333, 61.83333333333333, 60.83333333333333, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.08333333333333, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.53999999999999, "threshold_area100": 86.54166666666667, "threshold_area25": 98.65833333333333}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.09319986192614, 90.09319986192614, 90.05868139454608, 90.006903693476, 89.98964445978598, 89.95512599240594, 89.90334829133586, 89.8688298239558, 89.76527442181566, 89.7134967207456, 89.67897825336554, 89.60994131860545, 89.5063859164653, 89.42008974801519, 89.40283051432516, 89.29927511218501, 89.2129789437349, 89.04038660683466, 88.95409043838454, 88.88505350362443, 88.79875733517432, 88.74697963410424, 88.60890576458405, 88.53986882982396, 88.36727649292371, 88.19468415602347, 88.02209181912323, 87.7114256127028, 87.36624093890231, 87.29720400414222, 87.14187090093199, 86.96927856403175, 86.88298239558164, 86.79668622713152, 86.62409389023126, 86.24439074905074, 85.96824301001035, 85.81290990680014, 85.62305833620987, 85.27787366240939, 85.07076285812909, 84.74283741801864, 84.44943044528823, 84.27683810838799, 84.01794960303762, 83.7935795650673, 83.44839489126683, 83.25854332067657, 82.94787711425612, 82.75802554366585, 82.17121159820505, 81.96410079392474, 81.68795305488436, 81.51536071798412, 81.39454608215395, 81.04936140835346, 80.58336209872282, 79.94477045219192, 79.53054884363134, 79.13358646876078, 78.8574387297204, 78.44321712115982, 78.23610631687953, 77.5802554366586, 76.76907145322747, 76.51018294787711, 75.90610976872627, 75.12944425267519, 74.80151881256472, 74.0766309975837, 73.47255781843286, 72.74767000345183, 71.85018985157059, 71.36693130824992, 70.95270969968934, 70.27959958577839, 69.74456334138763, 68.72626855367622, 68.29478771142561, 67.65619606489471, 67.2074559889541, 66.53434587504314, 65.68864342423197, 65.0673110113911, 63.703831549879176, 62.909906800138074, 62.443907490507414, 60.959613393165334, 59.7169485674836, 58.81946841560235, 58.2844321712116, 57.6458405246807, 56.83465654124957, 56.07525025888851, 54.81532619951675], "calls": [0.0, 0.7594062823610631, 1.8467380048325857, 2.6924404556437693, 3.40006903693476, 4.452882292026234, 5.091473938557129, 5.453917846047635, 6.575768035899205, 7.973765964791163, 8.767690714532275, 9.509837763203313, 10.752502588885053, 11.56368657231619, 13.531239212978944, 14.135312392129789, 14.963755609250951, 16.25819813600276, 17.12115982050397, 17.966862271315154, 18.70900931998619, 19.27856403175699, 20.538488091128755, 22.143596824301, 22.85122540559199, 23.558853986882983, 24.352778736624096, 25.181221953745254, 26.19951674145668, 26.544701415257162, 27.925440110459093, 29.237141870900935, 30.462547462892648, 32.18847083189507, 33.759061097687265, 34.552985847428374, 35.329651363479464, 36.19261304798067, 36.76216775975147, 37.573351743182606, 38.48809112875388, 39.972385226095966, 41.16327235070763, 41.73282706247843, 42.76838108387987, 43.821194338971345, 44.8567483603728, 45.875043148084224, 46.789782533655504, 47.87711425612703, 49.05074214704867, 49.896444597859855, 51.08733172247152, 51.55333103210218, 52.39903348291336, 53.24473593372454, 55.38488091128754, 56.28236106316879, 57.21435968243011, 58.06006213324129, 58.26717293752157, 59.285467725233, 60.510873317224714, 62.08146358301691, 63.2896099413186, 64.23886779426994, 64.94649637556093, 66.1373835001726, 66.5688643424232, 67.7079737659648, 69.15774939592683, 69.98619261304798, 71.31515360717984, 72.48878149810149, 73.19641007939246, 74.38729720400414, 75.33655505695546, 76.49292371418709, 77.92544011045909, 78.66758715913014, 79.25440110459095, 80.27269589230238, 81.08387987573352, 82.5681739730756, 83.2067656196065, 84.0697273041077, 84.77735588539869, 85.70935450466, 86.93476009665171, 87.76320331377286, 89.59268208491544, 90.61097687262686, 91.1460131170176, 92.80289955125993, 94.33897134967208, 95.51259924059372, 96.11667241974456, 96.9278564031757, 97.79081808767691, 98.6365205384881, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 5, 7, 7, 8, 9, 10, 11, 12, 12, 13, 14, 16, 16, 17, 22, 24, 25, 29, 29, 30, 31, 35, 39, 43, 47, 48, 50, 55, 62, 64, 68, 68, 71, 72, 75, 77, 82, 89, 90, 93, 97, 100, 102, 105, 108, 109, 110, 113, 118, 120, 122, 125, 126, 128, 130, 135, 138, 139, 139, 141, 144, 144, 144, 144, 144, 144], "harmed": [0, 0, 0, 0, 0, 2, 4, 4, 6, 9, 10, 12, 15, 17, 23, 26, 28, 33, 39, 44, 45, 51, 56, 66, 71, 75, 80, 83, 92, 96, 106, 117, 127, 145, 166, 171, 182, 192, 198, 204, 215, 238, 255, 264, 276, 297, 311, 330, 348, 363, 380, 394, 418, 429, 448, 460, 498, 514, 534, 548, 556, 578, 610, 654, 680, 707, 723, 750, 763, 804, 853, 873, 915, 961, 983, 1029, 1067, 1111, 1166, 1197, 1222, 1262, 1296, 1360, 1387, 1426, 1455, 1495, 1546, 1584, 1668, 1717, 1745, 1831, 1905, 1960, 1991, 2028, 2075, 2119, 2192], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.09319986192614, 90.07594062823611, 90.05868139454608, 90.006903693476, 89.97238522609597, 89.95512599240594, 89.88608905764583, 89.85157059026578, 89.78253365550569, 89.74801518812563, 89.67897825336554, 89.62720055229548, 89.54090438384536, 89.42008974801519, 89.35105281325508, 89.24749741111494, 89.16120124266482, 89.05764584052469, 88.91957197100449, 88.85053503624438, 88.76423886779426, 88.66068346565412, 88.47083189506385, 88.33275802554367, 88.26372109078356, 88.09112875388332, 87.93579565067311, 87.76320331377286, 87.5733517431826, 87.34898170521228, 87.2109078356921, 87.02105626510182, 86.84846392820158, 86.72764929237141, 86.46876078702105, 86.24439074905074, 86.02002071108042, 85.77839143942009, 85.5712806351398, 85.26061442871936, 85.01898515705902, 84.72557818432861, 84.39765274421815, 84.24231964100794, 84.01794960303762, 83.72454263030721, 83.46565412495684, 83.12046945115637, 82.7925440110459, 82.56817397307559, 82.27476700034518, 82.01587849499482, 81.70521228857439, 81.51536071798412, 81.20469451156367, 80.70417673455297, 80.39351052813255, 79.9275112185019, 79.61684501208146, 79.2198826372109, 78.84017949603037, 78.47773558853987, 78.02899551259924, 77.39040386606834, 76.83810838798757, 76.51018294787711, 76.09596133931653, 75.45736969278563, 74.95685191577493, 74.33551950293406, 73.67966862271315, 73.02381774249223, 72.43700379703141, 71.86744908526062, 71.10804280289955, 70.43493268898861, 69.77908180876769, 69.19226786330687, 68.39834311356576, 67.7079737659648, 67.05212288574387, 66.3617535381429, 65.63686572316189, 64.9292371418709, 64.11805315843976, 63.410424577148774, 62.59924059371764, 61.73627890921642, 60.787021056265104, 59.99309630652399, 59.19917155678287, 58.387987573351744, 57.59406282361063, 56.6448049706593, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.1489128708367, "threshold_area100": 80.41750086296166, "threshold_area25": 89.75146703486365}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.5695973634719849, "operator": "ge"}, {"budget": 0.02, "threshold": -2.4585700035095215, "operator": "ge"}, {"budget": 0.03, "threshold": -2.9119367599487305, "operator": "ge"}, {"budget": 0.04, "threshold": -3.4555273056030273, "operator": "ge"}, {"budget": 0.05, "threshold": -3.667440891265869, "operator": "ge"}, {"budget": 0.06, "threshold": -3.9496264457702637, "operator": "ge"}, {"budget": 0.07, "threshold": -4.185037612915039, "operator": "ge"}, {"budget": 0.08, "threshold": -4.387879371643066, "operator": "ge"}, {"budget": 0.09, "threshold": -4.705778121948242, "operator": "ge"}, {"budget": 0.1, "threshold": -5.036342620849609, "operator": "ge"}, {"budget": 0.11, "threshold": -5.231752395629883, "operator": "ge"}, {"budget": 0.12, "threshold": -5.39016580581665, "operator": "ge"}, {"budget": 0.13, "threshold": -5.511255264282227, "operator": "ge"}, {"budget": 0.14, "threshold": -5.634820938110352, "operator": "ge"}, {"budget": 0.15, "threshold": -5.778458118438721, "operator": "ge"}, {"budget": 0.16, "threshold": -5.902294158935547, "operator": "ge"}, {"budget": 0.17, "threshold": -6.042496204376221, "operator": "ge"}, {"budget": 0.18, "threshold": -6.115330219268799, "operator": "ge"}, {"budget": 0.19, "threshold": -6.199382781982422, "operator": "ge"}, {"budget": 0.2, "threshold": -6.310417175292969, "operator": "ge"}, {"budget": 0.21, "threshold": -6.378220558166504, "operator": "ge"}, {"budget": 0.22, "threshold": -6.472862720489502, "operator": "ge"}, {"budget": 0.23, "threshold": -6.527879238128662, "operator": "ge"}, {"budget": 0.24, "threshold": -6.614448547363281, "operator": "ge"}, {"budget": 0.25, "threshold": -6.751137733459473, "operator": "ge"}, {"budget": 0.26, "threshold": -6.8949127197265625, "operator": "ge"}, {"budget": 0.27, "threshold": -6.961767196655273, "operator": "ge"}, {"budget": 0.28, "threshold": -7.048026084899902, "operator": "ge"}, {"budget": 0.29, "threshold": -7.141147613525391, "operator": "ge"}, {"budget": 0.3, "threshold": -7.2313032150268555, "operator": "ge"}, {"budget": 0.31, "threshold": -7.337775230407715, "operator": "ge"}, {"budget": 0.32, "threshold": -7.401485443115234, "operator": "ge"}, {"budget": 0.33, "threshold": -7.469958305358887, "operator": "ge"}, {"budget": 0.34, "threshold": -7.5543928146362305, "operator": "ge"}, {"budget": 0.35, "threshold": -7.628043174743652, "operator": "ge"}, {"budget": 0.36, "threshold": -7.676340103149414, "operator": "ge"}, {"budget": 0.37, "threshold": -7.780412673950195, "operator": "ge"}, {"budget": 0.38, "threshold": -7.870548725128174, "operator": "ge"}, {"budget": 0.39, "threshold": -7.954520225524902, "operator": "ge"}, {"budget": 0.4, "threshold": -8.010281562805176, "operator": "ge"}, {"budget": 0.41, "threshold": -8.057971000671387, "operator": "ge"}, {"budget": 0.42, "threshold": -8.107420921325684, "operator": "ge"}, {"budget": 0.43, "threshold": -8.15423583984375, "operator": "ge"}, {"budget": 0.44, "threshold": -8.241119384765625, "operator": "ge"}, {"budget": 0.45, "threshold": -8.359932899475098, "operator": "ge"}, {"budget": 0.46, "threshold": -8.403696060180664, "operator": "ge"}, {"budget": 0.47, "threshold": -8.462090492248535, "operator": "ge"}, {"budget": 0.48, "threshold": -8.540986061096191, "operator": "ge"}, {"budget": 0.49, "threshold": -8.613133430480957, "operator": "ge"}, {"budget": 0.5, "threshold": -8.671871185302734, "operator": "ge"}, {"budget": 0.51, "threshold": -8.739179611206055, "operator": "ge"}, {"budget": 0.52, "threshold": -8.794334411621094, "operator": "ge"}, {"budget": 0.53, "threshold": -8.862997055053711, "operator": "ge"}, {"budget": 0.54, "threshold": -8.925596237182617, "operator": "ge"}, {"budget": 0.55, "threshold": -9.016674995422363, "operator": "ge"}, {"budget": 0.56, "threshold": -9.047162055969238, "operator": "ge"}, {"budget": 0.57, "threshold": -9.09421443939209, "operator": "ge"}, {"budget": 0.58, "threshold": -9.176586151123047, "operator": "ge"}, {"budget": 0.59, "threshold": -9.256757736206055, "operator": "ge"}, {"budget": 0.6, "threshold": -9.345659255981445, "operator": "ge"}, {"budget": 0.61, "threshold": -9.418987274169922, "operator": "ge"}, {"budget": 0.62, "threshold": -9.480707168579102, "operator": "ge"}, {"budget": 0.63, "threshold": -9.55443000793457, "operator": "ge"}, {"budget": 0.64, "threshold": -9.62117862701416, "operator": "ge"}, {"budget": 0.65, "threshold": -9.689565658569336, "operator": "ge"}, {"budget": 0.66, "threshold": -9.753011703491211, "operator": "ge"}, {"budget": 0.67, "threshold": -9.837554931640625, "operator": "ge"}, {"budget": 0.68, "threshold": -9.932019233703613, "operator": "ge"}, {"budget": 0.69, "threshold": -9.957855224609375, "operator": "ge"}, {"budget": 0.7, "threshold": -10.026153564453125, "operator": "ge"}, {"budget": 0.71, "threshold": -10.090438842773438, "operator": "ge"}, {"budget": 0.72, "threshold": -10.173689842224121, "operator": "ge"}, {"budget": 0.73, "threshold": -10.245698928833008, "operator": "ge"}, {"budget": 0.74, "threshold": -10.369461059570312, "operator": "ge"}, {"budget": 0.75, "threshold": -10.412298202514648, "operator": "ge"}, {"budget": 0.76, "threshold": -10.535475730895996, "operator": "ge"}, {"budget": 0.77, "threshold": -10.655950546264648, "operator": "ge"}, {"budget": 0.78, "threshold": -10.737972259521484, "operator": "ge"}, {"budget": 0.79, "threshold": -10.826505661010742, "operator": "ge"}, {"budget": 0.8, "threshold": -10.903038024902344, "operator": "ge"}, {"budget": 0.81, "threshold": -10.999421119689941, "operator": "ge"}, {"budget": 0.82, "threshold": -11.060949325561523, "operator": "ge"}, {"budget": 0.83, "threshold": -11.19660472869873, "operator": "ge"}, {"budget": 0.84, "threshold": -11.325077056884766, "operator": "ge"}, {"budget": 0.85, "threshold": -11.383319854736328, "operator": "ge"}, {"budget": 0.86, "threshold": -11.51839828491211, "operator": "ge"}, {"budget": 0.87, "threshold": -11.581999778747559, "operator": "ge"}, {"budget": 0.88, "threshold": -11.691503524780273, "operator": "ge"}, {"budget": 0.89, "threshold": -11.802730560302734, "operator": "ge"}, {"budget": 0.9, "threshold": -11.921025276184082, "operator": "ge"}, {"budget": 0.91, "threshold": -12.033376693725586, "operator": "ge"}, {"budget": 0.92, "threshold": -12.181856155395508, "operator": "ge"}, {"budget": 0.93, "threshold": -12.293683052062988, "operator": "ge"}, {"budget": 0.94, "threshold": -12.400674819946289, "operator": "ge"}, {"budget": 0.95, "threshold": -12.485926628112793, "operator": "ge"}, {"budget": 0.96, "threshold": -12.623748779296875, "operator": "ge"}, {"budget": 0.97, "threshold": -12.829598426818848, "operator": "ge"}, {"budget": 0.98, "threshold": -13.03901481628418, "operator": "ge"}, {"budget": 0.99, "threshold": -13.245745658874512, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.75, 98.5, 98.0, 97.5, 97.16666666666667, 96.83333333333333, 96.75, 96.25, 95.91666666666666, 95.5, 95.16666666666667, 95.0, 94.66666666666667, 94.25, 93.91666666666666, 93.75, 93.66666666666667, 93.5, 93.25, 93.25, 93.0, 92.75, 92.66666666666666, 92.41666666666666, 92.33333333333333, 92.16666666666666, 92.08333333333333, 92.08333333333333, 92.0, 91.58333333333333, 91.16666666666666, 90.75, 90.41666666666667, 90.08333333333333, 89.99999999999999, 89.58333333333333, 89.41666666666667, 89.25, 88.75, 88.75, 88.5, 88.33333333333333, 87.91666666666667, 87.33333333333333, 87.08333333333333, 86.66666666666667, 86.16666666666666, 85.91666666666666, 85.08333333333333, 84.66666666666667, 84.16666666666667, 84.08333333333333, 83.5, 83.08333333333333, 83.0, 82.75, 82.41666666666666, 82.16666666666667, 81.91666666666666, 81.58333333333333, 81.25, 80.83333333333333, 80.5, 80.0, 79.66666666666666, 79.5, 79.16666666666666, 78.33333333333333, 77.66666666666666, 77.0, 76.5, 75.91666666666667, 75.66666666666666, 75.16666666666667, 74.5, 74.16666666666667, 73.41666666666666, 72.58333333333333, 71.91666666666666, 71.5, 70.66666666666667, 69.83333333333333, 69.24999999999999, 68.33333333333333, 67.83333333333333, 66.91666666666667, 65.99999999999999, 65.5, 64.66666666666666, 64.08333333333334, 63.24999999999999, 62.33333333333333, 61.5, 60.5, 59.5, 58.58333333333333, 57.666666666666664, 56.91666666666666, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 2, 5, 11, 17, 21, 26, 27, 33, 37, 42, 46, 48, 52, 57, 61, 63, 64, 66, 69, 70, 73, 76, 77, 80, 81, 83, 84, 84, 85, 90, 95, 100, 104, 108, 109, 114, 116, 118, 124, 124, 127, 129, 134, 141, 144, 149, 155, 158, 168, 173, 179, 180, 187, 192, 193, 196, 200, 203, 206, 210, 214, 219, 223, 229, 233, 235, 239, 249, 257, 265, 271, 278, 281, 287, 295, 299, 308, 318, 326, 331, 341, 351, 358, 369, 375, 386, 397, 403, 413, 420, 430, 441, 451, 463, 475, 486, 497, 506, 516, 528], "exact_score": [98.91666666666666, 98.75, 98.5, 98.0, 97.5, 97.16666666666667, 96.83333333333333, 96.75, 96.25, 95.91666666666666, 95.5, 95.16666666666667, 95.0, 94.66666666666667, 94.25, 93.91666666666666, 93.75, 93.66666666666667, 93.5, 93.25, 93.25, 93.0, 92.75, 92.66666666666666, 92.41666666666666, 92.33333333333333, 92.16666666666666, 92.08333333333333, 92.08333333333333, 92.0, 91.58333333333333, 91.16666666666666, 90.75, 90.41666666666667, 90.08333333333333, 89.99999999999999, 89.58333333333333, 89.41666666666667, 89.25, 88.75, 88.75, 88.5, 88.33333333333333, 87.91666666666667, 87.33333333333333, 87.08333333333333, 86.66666666666667, 86.16666666666666, 85.91666666666666, 85.08333333333333, 84.66666666666667, 84.16666666666667, 84.08333333333333, 83.5, 83.08333333333333, 83.0, 82.75, 82.41666666666666, 82.16666666666667, 81.91666666666666, 81.58333333333333, 81.25, 80.83333333333333, 80.5, 80.0, 79.66666666666666, 79.5, 79.16666666666666, 78.33333333333333, 77.66666666666666, 77.0, 76.5, 75.91666666666667, 75.66666666666666, 75.16666666666667, 74.5, 74.16666666666667, 73.41666666666666, 72.58333333333333, 71.91666666666666, 71.5, 70.66666666666667, 69.83333333333333, 69.24999999999999, 68.33333333333333, 67.83333333333333, 66.91666666666667, 65.99999999999999, 65.5, 64.66666666666666, 64.08333333333334, 63.24999999999999, 62.33333333333333, 61.5, 60.5, 59.5, 58.58333333333333, 57.666666666666664, 56.91666666666666, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 82.50833333333333, "threshold_area100": 82.50833333333334, "threshold_area25": 95.09166666666667}, "test": {"score": [90.16223679668623, 89.90334829133586, 89.17846047635484, 88.69520193303417, 87.79772178115292, 87.65964791163272, 87.14187090093199, 86.72764929237141, 86.26164998274076, 85.29513289609942, 83.94891266827752, 83.34483948912667, 83.06869175008629, 82.6372109078357, 82.10217466344494, 81.6361753538143, 81.20469451156367, 80.63513979979288, 80.39351052813255, 80.13462202278218, 79.7204004142216, 79.46151190887124, 79.15084570245081, 78.8574387297204, 78.65032792544011, 78.30514325163962, 77.80462547462892, 77.4249223334484, 77.2178115291681, 77.02795995857782, 76.70003451846738, 76.35484984466689, 76.2167759751467, 75.99240593717639, 75.76803589920607, 75.50914739385571, 75.35381429064549, 75.04314808422505, 74.87055574732481, 74.5771487745944, 74.42181567138418, 74.24922333448394, 74.05937176389368, 73.80048325854331, 73.54159475319295, 73.17915084570245, 73.05833620987228, 72.85122540559199, 72.6441146013117, 72.40248532965136, 72.21263375906109, 71.97100448740076, 71.76389368312046, 71.52226441146013, 71.21159820503969, 70.98722816706939, 70.96996893337936, 70.74559889540903, 70.43493268898861, 69.98619261304798, 69.64100793924749, 69.34760096651709, 69.03693476009664, 68.81256472212634, 68.48463928201586, 68.19123230928547, 67.863306869175, 67.7079737659648, 67.27649292371419, 67.24197445633413, 67.00034518467379, 66.77597514670349, 66.37901277183292, 66.03382809803244, 65.53331032102174, 65.30894028305143, 64.91197790818087, 64.41146013117017, 64.16983085950983, 63.84190541939938, 63.548498446668965, 63.15153607179841, 62.89264756644805, 62.2195374525371, 61.8570935450466, 61.52916810493614, 61.02865032792544, 60.82153952364515, 60.39005868139454, 60.01035554021401, 59.69968933379357, 59.285467725233, 58.90576458405247, 58.43976527442181, 57.9392474974111, 57.68035899206075, 57.17984121505004, 56.48947186744908, 55.988954090438384, 55.52295478080773, 54.81532619951675], "calls": [0.0, 2.4680704176734554, 7.783914394200897, 10.804280289955127, 14.963755609250951, 16.189161201242666, 18.6399723852261, 20.0724887814981, 21.884708318950636, 24.611667241974455, 27.908180876769073, 29.513289609941317, 31.032102174663446, 32.153952364515014, 33.551950293406975, 35.29513289609941, 36.62409389023127, 38.280980324473596, 39.26475664480497, 40.33482913358647, 41.68104936140835, 42.664825681739735, 43.890231273731445, 44.666896789782534, 45.59889540904384, 47.290300310666204, 48.895409043838455, 49.75837072833966, 50.72488781498101, 51.77770107007249, 52.726958923023815, 54.073179150845704, 54.79806696582672, 55.66102865032793, 56.36865723161891, 57.11080428028995, 57.628581290990674, 58.8539868829824, 60.027614773904034, 60.976872626855375, 61.54642733862617, 61.90887124611667, 62.478425957887474, 63.099758370728345, 64.1525716258198, 65.44701415257163, 65.87849499482223, 66.4998274076631, 67.22471522264412, 67.88056610286503, 68.4328615809458, 69.15774939592683, 69.76182257507767, 70.67656196064894, 71.41870900932, 72.38522609596134, 72.62685536762167, 73.35174318260269, 74.47359337245426, 75.4401104590956, 76.44114601311702, 77.18329306178806, 77.92544011045909, 78.66758715913014, 79.37521574042113, 79.94477045219193, 80.51432516396272, 81.1356575768036, 82.03313772868485, 82.11943389713497, 82.72350707628581, 83.2067656196065, 83.98343113565757, 84.67380048325855, 85.67483603727995, 85.96824301001035, 86.86572316189162, 87.62512944425268, 88.0738695201933, 88.5053503624439, 88.9886089057646, 89.62720055229548, 89.98964445978599, 90.93890231273731, 91.56023472557818, 91.97445633413876, 92.57852951328961, 92.88919571971005, 93.47600966517086, 94.04556437694166, 94.49430445288229, 94.99482222989299, 95.4263030721436, 95.96133931653435, 96.54815326199517, 96.8588194684156, 97.4283741801864, 98.17052122885744, 98.7228167069382, 99.24059371763893, 100.0], "rescued": [0, 3, 24, 37, 59, 73, 83, 87, 94, 100, 108, 110, 117, 119, 121, 123, 125, 126, 126, 127, 128, 130, 130, 130, 130, 131, 131, 131, 132, 133, 133, 134, 134, 134, 134, 135, 135, 135, 136, 136, 136, 136, 136, 136, 136, 137, 137, 138, 138, 138, 138, 138, 138, 138, 138, 139, 139, 139, 139, 139, 140, 140, 140, 140, 140, 140, 140, 141, 141, 141, 142, 142, 142, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 18, 81, 122, 196, 218, 258, 286, 320, 382, 468, 505, 528, 555, 588, 617, 644, 678, 692, 708, 733, 750, 768, 785, 797, 818, 847, 869, 882, 894, 913, 934, 942, 955, 968, 984, 993, 1011, 1022, 1039, 1048, 1058, 1069, 1084, 1099, 1121, 1128, 1141, 1153, 1167, 1178, 1192, 1204, 1218, 1236, 1250, 1251, 1264, 1282, 1308, 1329, 1346, 1364, 1377, 1396, 1413, 1432, 1442, 1467, 1469, 1484, 1497, 1520, 1541, 1570, 1583, 1606, 1635, 1649, 1668, 1685, 1708, 1723, 1762, 1783, 1802, 1831, 1843, 1868, 1890, 1909, 1933, 1955, 1982, 2011, 2026, 2055, 2095, 2124, 2151, 2192], "exact_score": [90.16223679668623, 90.1449775629962, 90.02416292716603, 89.78253365550569, 89.67897825336554, 89.6444597859855, 89.5063859164653, 89.40283051432516, 89.17846047635484, 88.95409043838454, 88.81601656886434, 88.59164653089402, 88.47083189506385, 88.28098032447359, 87.91853641698309, 87.78046254746289, 87.65964791163272, 87.45253710735244, 87.24542630307214, 87.03831549879185, 86.7103900586814, 86.45150155333103, 86.20987228167068, 85.91646530894027, 85.55402140144977, 85.17431826026925, 84.77735588539869, 84.4321712115982, 83.91439420089748, 83.56920952709699, 83.25854332067657, 83.06869175008629, 82.70624784259579, 82.30928546772522, 81.91232309285468, 81.70521228857439, 81.44632378322402, 81.06662064204349, 80.68691750086295, 80.46254746289264, 80.18639972385225, 79.94477045219192, 79.65136347946151, 79.4269934414912, 79.11632723507076, 78.82292026234035, 78.52951328960994, 78.33966171901967, 78.09803244735933, 77.73558853986883, 77.39040386606834, 77.16603382809804, 76.88988608905764, 76.5792198826372, 76.37210907835691, 76.18225750776665, 75.85433206765619, 75.5781843286158, 75.2847773558854, 75.02588885053503, 74.88781498101484, 74.5771487745944, 74.24922333448394, 73.83500172592336, 73.576113220573, 73.28270624784258, 72.98929927511219, 72.69589230238176, 72.36796686227132, 72.0055229547808, 71.67759751467034, 71.47048671039006, 71.05626510182947, 70.90093199861926, 70.5730065585088, 70.19330341732827, 69.77908180876769, 69.43389713496721, 69.01967552640662, 68.67449085260614, 68.17397307559544, 67.74249223334483, 67.27649292371419, 66.84501208146358, 66.37901277183292, 65.82671729375215, 65.2916810493614, 64.82568173973075, 64.20434932688988, 63.548498446668965, 62.89264756644805, 62.16775975146703, 61.51190887124611, 60.75250258888505, 60.06213324128409, 59.285467725233, 58.42250604073178, 57.55954435623059, 56.66206420434931, 55.73006558508802, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 76.7369494396796, "threshold_area100": 71.98826372109079, "threshold_area25": 83.4397652744218}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.6106605529785156, "operator": "ge"}, {"budget": 0.02, "threshold": -1.001806616783142, "operator": "ge"}, {"budget": 0.03, "threshold": -1.3354465961456299, "operator": "ge"}, {"budget": 0.04, "threshold": -1.6759002208709717, "operator": "ge"}, {"budget": 0.05, "threshold": -2.0018181800842285, "operator": "ge"}, {"budget": 0.06, "threshold": -2.2191405296325684, "operator": "ge"}, {"budget": 0.07, "threshold": -2.391840934753418, "operator": "ge"}, {"budget": 0.08, "threshold": -2.760033369064331, "operator": "ge"}, {"budget": 0.09, "threshold": -2.9785969257354736, "operator": "ge"}, {"budget": 0.1, "threshold": -3.202439308166504, "operator": "ge"}, {"budget": 0.11, "threshold": -3.3737645149230957, "operator": "ge"}, {"budget": 0.12, "threshold": -3.5423648357391357, "operator": "ge"}, {"budget": 0.13, "threshold": -3.6976699829101562, "operator": "ge"}, {"budget": 0.14, "threshold": -3.9074854850769043, "operator": "ge"}, {"budget": 0.15, "threshold": -4.145237445831299, "operator": "ge"}, {"budget": 0.16, "threshold": -4.205291748046875, "operator": "ge"}, {"budget": 0.17, "threshold": -4.3692731857299805, "operator": "ge"}, {"budget": 0.18, "threshold": -4.473455905914307, "operator": "ge"}, {"budget": 0.19, "threshold": -4.68409538269043, "operator": "ge"}, {"budget": 0.2, "threshold": -4.773612976074219, "operator": "ge"}, {"budget": 0.21, "threshold": -4.9191365242004395, "operator": "ge"}, {"budget": 0.22, "threshold": -4.980933666229248, "operator": "ge"}, {"budget": 0.23, "threshold": -5.067440032958984, "operator": "ge"}, {"budget": 0.24, "threshold": -5.14710807800293, "operator": "ge"}, {"budget": 0.25, "threshold": -5.240271091461182, "operator": "ge"}, {"budget": 0.26, "threshold": -5.305929660797119, "operator": "ge"}, {"budget": 0.27, "threshold": -5.370907783508301, "operator": "ge"}, {"budget": 0.28, "threshold": -5.505476951599121, "operator": "ge"}, {"budget": 0.29, "threshold": -5.576348781585693, "operator": "ge"}, {"budget": 0.3, "threshold": -5.65109395980835, "operator": "ge"}, {"budget": 0.31, "threshold": -5.709198951721191, "operator": "ge"}, {"budget": 0.32, "threshold": -5.789547920227051, "operator": "ge"}, {"budget": 0.33, "threshold": -5.830751419067383, "operator": "ge"}, {"budget": 0.34, "threshold": -5.895720481872559, "operator": "ge"}, {"budget": 0.35, "threshold": -5.951825141906738, "operator": "ge"}, {"budget": 0.36, "threshold": -6.001065254211426, "operator": "ge"}, {"budget": 0.37, "threshold": -6.0746965408325195, "operator": "ge"}, {"budget": 0.38, "threshold": -6.167165756225586, "operator": "ge"}, {"budget": 0.39, "threshold": -6.212851524353027, "operator": "ge"}, {"budget": 0.4, "threshold": -6.283226013183594, "operator": "ge"}, {"budget": 0.41, "threshold": -6.336012840270996, "operator": "ge"}, {"budget": 0.42, "threshold": -6.395092964172363, "operator": "ge"}, {"budget": 0.43, "threshold": -6.495241165161133, "operator": "ge"}, {"budget": 0.44, "threshold": -6.537628173828125, "operator": "ge"}, {"budget": 0.45, "threshold": -6.606067657470703, "operator": "ge"}, {"budget": 0.46, "threshold": -6.636971473693848, "operator": "ge"}, {"budget": 0.47, "threshold": -6.694741249084473, "operator": "ge"}, {"budget": 0.48, "threshold": -6.7533063888549805, "operator": "ge"}, {"budget": 0.49, "threshold": -6.785447120666504, "operator": "ge"}, {"budget": 0.5, "threshold": -6.819397926330566, "operator": "ge"}, {"budget": 0.51, "threshold": -6.854033470153809, "operator": "ge"}, {"budget": 0.52, "threshold": -6.936741828918457, "operator": "ge"}, {"budget": 0.53, "threshold": -6.986557483673096, "operator": "ge"}, {"budget": 0.54, "threshold": -7.022680759429932, "operator": "ge"}, {"budget": 0.55, "threshold": -7.074264049530029, "operator": "ge"}, {"budget": 0.56, "threshold": -7.165837287902832, "operator": "ge"}, {"budget": 0.57, "threshold": -7.201237678527832, "operator": "ge"}, {"budget": 0.58, "threshold": -7.245043754577637, "operator": "ge"}, {"budget": 0.59, "threshold": -7.28933048248291, "operator": "ge"}, {"budget": 0.6, "threshold": -7.322768688201904, "operator": "ge"}, {"budget": 0.61, "threshold": -7.3574442863464355, "operator": "ge"}, {"budget": 0.62, "threshold": -7.422162055969238, "operator": "ge"}, {"budget": 0.63, "threshold": -7.454685211181641, "operator": "ge"}, {"budget": 0.64, "threshold": -7.48661994934082, "operator": "ge"}, {"budget": 0.65, "threshold": -7.529742240905762, "operator": "ge"}, {"budget": 0.66, "threshold": -7.565120220184326, "operator": "ge"}, {"budget": 0.67, "threshold": -7.605208396911621, "operator": "ge"}, {"budget": 0.68, "threshold": -7.662006378173828, "operator": "ge"}, {"budget": 0.69, "threshold": -7.743031978607178, "operator": "ge"}, {"budget": 0.7, "threshold": -7.787303924560547, "operator": "ge"}, {"budget": 0.71, "threshold": -7.830961227416992, "operator": "ge"}, {"budget": 0.72, "threshold": -7.885019779205322, "operator": "ge"}, {"budget": 0.73, "threshold": -7.929812431335449, "operator": "ge"}, {"budget": 0.74, "threshold": -7.9926605224609375, "operator": "ge"}, {"budget": 0.75, "threshold": -8.047054290771484, "operator": "ge"}, {"budget": 0.76, "threshold": -8.175680160522461, "operator": "ge"}, {"budget": 0.77, "threshold": -8.234615325927734, "operator": "ge"}, {"budget": 0.78, "threshold": -8.264442443847656, "operator": "ge"}, {"budget": 0.79, "threshold": -8.330612182617188, "operator": "ge"}, {"budget": 0.8, "threshold": -8.382145881652832, "operator": "ge"}, {"budget": 0.81, "threshold": -8.43161678314209, "operator": "ge"}, {"budget": 0.82, "threshold": -8.489925384521484, "operator": "ge"}, {"budget": 0.83, "threshold": -8.549840927124023, "operator": "ge"}, {"budget": 0.84, "threshold": -8.605883598327637, "operator": "ge"}, {"budget": 0.85, "threshold": -8.689186096191406, "operator": "ge"}, {"budget": 0.86, "threshold": -8.761935234069824, "operator": "ge"}, {"budget": 0.87, "threshold": -8.87347412109375, "operator": "ge"}, {"budget": 0.88, "threshold": -8.958694458007812, "operator": "ge"}, {"budget": 0.89, "threshold": -9.075889587402344, "operator": "ge"}, {"budget": 0.9, "threshold": -9.139366149902344, "operator": "ge"}, {"budget": 0.91, "threshold": -9.22891616821289, "operator": "ge"}, {"budget": 0.92, "threshold": -9.328126907348633, "operator": "ge"}, {"budget": 0.93, "threshold": -9.447833061218262, "operator": "ge"}, {"budget": 0.94, "threshold": -9.53222370147705, "operator": "ge"}, {"budget": 0.95, "threshold": -9.636629104614258, "operator": "ge"}, {"budget": 0.96, "threshold": -9.752985000610352, "operator": "ge"}, {"budget": 0.97, "threshold": -9.846579551696777, "operator": "ge"}, {"budget": 0.98, "threshold": -9.926396369934082, "operator": "ge"}, {"budget": 0.99, "threshold": -10.19007682800293, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.75, 98.58333333333333, 98.08333333333333, 97.5, 97.24999999999999, 96.66666666666667, 96.16666666666667, 95.66666666666667, 95.41666666666666, 95.0, 94.41666666666666, 93.66666666666667, 93.16666666666666, 93.0, 92.66666666666666, 92.41666666666666, 92.08333333333333, 91.91666666666667, 91.66666666666666, 91.41666666666667, 91.33333333333333, 91.25, 91.16666666666666, 90.83333333333333, 90.66666666666666, 90.33333333333333, 90.25, 90.08333333333333, 89.99999999999999, 89.91666666666667, 89.5, 88.83333333333333, 88.75, 88.16666666666666, 87.83333333333333, 87.33333333333333, 87.0, 86.75, 86.58333333333333, 86.41666666666666, 86.0, 85.58333333333333, 85.25, 85.08333333333333, 84.66666666666667, 84.58333333333333, 84.25, 83.83333333333333, 83.66666666666667, 83.41666666666666, 82.91666666666666, 82.66666666666667, 82.25, 82.0, 81.66666666666667, 81.41666666666667, 81.16666666666667, 80.83333333333333, 80.33333333333333, 80.08333333333333, 79.75, 79.5, 79.0, 78.58333333333333, 78.16666666666666, 77.66666666666666, 77.33333333333333, 76.91666666666667, 76.66666666666666, 76.16666666666666, 75.66666666666666, 75.25, 74.75, 74.08333333333333, 73.66666666666666, 73.16666666666667, 72.58333333333333, 72.24999999999999, 71.66666666666666, 71.16666666666667, 70.24999999999999, 69.5, 68.66666666666667, 67.83333333333333, 67.25, 66.5, 65.75, 64.91666666666667, 64.16666666666666, 63.5, 62.83333333333333, 62.083333333333336, 61.33333333333333, 60.416666666666664, 59.5, 58.58333333333333, 57.58333333333333, 56.833333333333336, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 2, 4, 10, 17, 21, 28, 34, 40, 43, 48, 55, 64, 70, 73, 77, 80, 84, 86, 89, 92, 93, 94, 95, 99, 101, 105, 106, 108, 109, 110, 115, 123, 124, 131, 135, 141, 145, 148, 150, 152, 157, 162, 166, 168, 173, 174, 178, 183, 185, 188, 194, 197, 202, 205, 209, 212, 215, 219, 225, 228, 232, 235, 241, 246, 251, 257, 261, 266, 269, 275, 281, 286, 292, 300, 305, 311, 318, 322, 329, 335, 346, 355, 365, 375, 382, 391, 400, 410, 419, 427, 435, 444, 453, 464, 475, 486, 498, 507, 517, 528], "exact_score": [98.91666666666666, 98.75, 98.58333333333333, 98.08333333333333, 97.5, 97.24999999999999, 96.66666666666667, 96.16666666666667, 95.66666666666667, 95.41666666666666, 95.0, 94.41666666666666, 93.66666666666667, 93.16666666666666, 93.0, 92.66666666666666, 92.41666666666666, 92.08333333333333, 91.91666666666667, 91.66666666666666, 91.41666666666667, 91.33333333333333, 91.25, 91.16666666666666, 90.83333333333333, 90.66666666666666, 90.33333333333333, 90.25, 90.08333333333333, 89.99999999999999, 89.91666666666667, 89.5, 88.83333333333333, 88.75, 88.16666666666666, 87.83333333333333, 87.33333333333333, 87.0, 86.75, 86.58333333333333, 86.41666666666666, 86.0, 85.58333333333333, 85.25, 85.08333333333333, 84.66666666666667, 84.58333333333333, 84.25, 83.83333333333333, 83.66666666666667, 83.41666666666666, 82.91666666666666, 82.66666666666667, 82.25, 82.0, 81.66666666666667, 81.41666666666667, 81.08333333333333, 80.83333333333333, 80.33333333333333, 80.08333333333333, 79.75, 79.5, 79.0, 78.58333333333333, 78.16666666666666, 77.66666666666666, 77.33333333333333, 76.91666666666667, 76.66666666666666, 76.16666666666666, 75.66666666666666, 75.25, 74.75, 74.08333333333333, 73.66666666666666, 73.16666666666667, 72.58333333333333, 72.24999999999999, 71.66666666666666, 71.16666666666667, 70.24999999999999, 69.41666666666666, 68.66666666666667, 67.83333333333333, 67.25, 66.5, 65.75, 64.91666666666667, 64.16666666666666, 63.5, 62.83333333333333, 62.083333333333336, 61.33333333333333, 60.416666666666664, 59.5, 58.58333333333333, 57.58333333333333, 56.833333333333336, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 81.34499999999998, "threshold_area100": 81.34666666666665, "threshold_area25": 94.22500000000001}, "test": {"score": [90.16223679668623, 89.83431135657577, 89.38557128063513, 88.90231273731446, 88.17742492233344, 87.27994477045219, 86.69313082499137, 86.08905764584051, 84.58750431480841, 83.60372799447704, 82.51639627200552, 82.03313772868485, 81.34276838108387, 80.68691750086295, 79.84121505005177, 78.75388332758025, 78.47773558853987, 77.97721781152916, 77.7183293061788, 77.09699689333793, 76.8553676216776, 76.54470141525715, 76.32033137728685, 76.11322057300656, 75.85433206765619, 75.66448049706594, 75.59544356230583, 75.42285122540558, 75.008629616845, 74.85329651363479, 74.68070417673455, 74.5081118398343, 74.19744563341388, 74.16292716603382, 73.90403866068347, 73.66240938902313, 73.48981705212289, 73.29996548153261, 72.98929927511219, 72.92026234035208, 72.78218847083188, 72.54055919917155, 72.28167069382118, 71.86744908526062, 71.62581981360027, 71.43596824301001, 71.36693130824992, 71.1425612702796, 70.79737659647911, 70.69382119433897, 70.6420434932689, 70.46945115636866, 70.17604418363824, 69.93441491197791, 69.77908180876769, 69.48567483603728, 69.00241629271659, 68.82982395581637, 68.60545391784605, 68.46738004832585, 68.31204694511564, 68.24301001035553, 67.91508457024507, 67.75975146703486, 67.63893683120469, 67.4318260269244, 67.31101139109424, 67.03486365205384, 66.7069382119434, 66.34449430445288, 66.17190196755264, 65.9302726958923, 65.68864342423197, 65.48153261995166, 65.18812564722126, 64.94649637556091, 64.1525716258198, 63.87642388677942, 63.53123921297894, 63.203313772868476, 62.996202968588186, 62.7890921643079, 62.46116672419744, 62.16775975146703, 61.96064894718673, 61.54642733862617, 61.2185018985157, 60.528132550914734, 60.234725578184324, 59.7859855022437, 59.49257852951328, 58.94028305143251, 58.43976527442181, 58.025543665861235, 57.73213669313082, 57.26613738350017, 56.90369347600965, 56.40317569899895, 56.1442871936486, 55.436658612357604, 54.81532619951675], "calls": [0.0, 2.7959958577839146, 6.50673110113911, 10.11391094235416, 13.652053848809112, 17.397307559544355, 19.589230238177425, 21.384190541939937, 25.353814290645495, 27.580255436658614, 29.703141180531585, 31.11839834311357, 32.775284777355886, 34.052468070417675, 35.84742837418018, 37.91853641698309, 38.53986882982395, 39.93786675871591, 40.904383845357266, 43.13082499137038, 44.14911977908181, 45.66793234380393, 46.46185709354504, 47.51467034863652, 48.394891266827756, 49.34414911977908, 50.01725923369003, 50.828443217121155, 52.74421815671384, 53.46910597169485, 54.47014152571625, 55.2122885743873, 55.988954090438384, 56.28236106316879, 57.05902657921988, 57.818432861580945, 58.439765274421816, 59.216430790472906, 60.510873317224714, 61.06316879530549, 61.839834311356576, 62.59924059371764, 63.4276838108388, 64.75664480497066, 65.36071798412151, 66.34449430445288, 66.65516051087332, 67.60441836382465, 68.6399723852261, 69.2095270969969, 69.58923023817742, 70.10700724887815, 71.21159820503969, 72.02278218847084, 72.67863306869174, 73.31722471522264, 74.81877804625474, 75.33655505695546, 75.9233690024163, 76.38936831204694, 76.87262685536763, 77.28684846392821, 78.16706938211944, 78.63306869175008, 79.09906800138073, 79.73765964791163, 80.08284432171212, 80.77321366931308, 81.4981014842941, 82.34380393510529, 82.77528477735588, 83.24128408698654, 83.94891266827753, 84.39765274421815, 85.13979979288919, 85.55402140144977, 87.05557473248187, 87.67690714532274, 88.12564722126338, 88.66068346565412, 89.02312737314463, 89.38557128063513, 89.92060752502589, 90.36934760096652, 90.83534691059717, 91.43942008974801, 92.11253020365896, 93.07904729030031, 93.47600966517086, 94.11460131170176, 94.52882292026234, 95.23645150155333, 95.8577839143942, 96.37556092509493, 96.75526406627546, 97.25578184328616, 97.68726268553675, 98.2222989299275, 98.51570590265793, 99.34414911977908, 100.0], "rescued": [0, 6, 21, 46, 68, 86, 95, 101, 115, 117, 120, 124, 128, 130, 130, 130, 131, 133, 134, 134, 134, 134, 134, 134, 134, 134, 136, 136, 136, 136, 136, 136, 137, 137, 137, 137, 138, 138, 138, 138, 139, 139, 139, 140, 140, 140, 140, 140, 141, 141, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 142, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 144, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 25, 66, 119, 183, 253, 296, 337, 438, 497, 563, 595, 639, 679, 728, 791, 808, 839, 855, 891, 905, 923, 936, 948, 963, 974, 980, 990, 1014, 1023, 1033, 1043, 1062, 1064, 1079, 1093, 1104, 1115, 1133, 1137, 1146, 1160, 1175, 1200, 1214, 1225, 1229, 1242, 1263, 1269, 1273, 1283, 1300, 1314, 1323, 1340, 1368, 1378, 1391, 1399, 1408, 1412, 1431, 1440, 1447, 1459, 1466, 1482, 1501, 1523, 1533, 1547, 1561, 1573, 1590, 1604, 1650, 1666, 1686, 1705, 1717, 1729, 1748, 1765, 1777, 1801, 1820, 1860, 1877, 1903, 1920, 1952, 1981, 2006, 2023, 2050, 2071, 2100, 2115, 2156, 2192], "exact_score": [90.16223679668623, 90.12771832930618, 90.02416292716603, 89.81705212288574, 89.69623748705557, 89.55816361753539, 89.42008974801519, 89.35105281325508, 89.38557128063513, 89.09216430790472, 88.91957197100449, 88.62616499827408, 88.45357266137384, 88.26372109078356, 88.09112875388332, 87.91853641698309, 87.5733517431826, 87.34898170521228, 87.15913013462202, 86.91750086296169, 86.62409389023126, 86.22713151536071, 85.84742837418018, 85.51950293406972, 85.10528132550914, 84.74283741801864, 84.24231964100794, 83.81083879875733, 83.44839489126683, 82.91335864687608, 82.46461857093544, 82.08491542975491, 81.70521228857439, 81.18743527787366, 80.721435968243, 80.23817742492233, 79.77217811529168, 79.20262340352087, 78.7193648602002, 78.30514325163962, 77.95995857783915, 77.70107007248878, 77.37314463237833, 77.09699689333793, 76.88988608905764, 76.66551605108734, 76.49292371418709, 76.2167759751467, 75.95788746979633, 75.73351743182603, 75.59544356230583, 75.40559199171557, 75.18122195374525, 74.99137038315499, 74.73248187780463, 74.55988954090438, 74.19744563341388, 73.93855712806351, 73.64515015533311, 73.36900241629272, 73.0755954435623, 72.92026234035208, 72.78218847083188, 72.40248532965136, 72.10907835692095, 71.74663444943043, 71.5050051777701, 71.31515360717984, 71.02174663444943, 70.72833966171902, 70.46945115636866, 70.27959958577839, 69.93441491197791, 69.62374870555747, 69.22678633068692, 68.95063859164652, 68.60545391784605, 68.27752847773559, 67.96686227131515, 67.69071453227477, 67.34552985847428, 66.96582671729375, 66.46530894028305, 66.0683465654125, 65.65412495685192, 65.22264411460131, 64.68760787021056, 64.16983085950983, 63.634794615119084, 63.03072143596824, 62.40938902312737, 61.87435277873662, 61.25302036589575, 60.57991025198481, 59.85502243700379, 59.13013462202278, 58.31895063859165, 57.50776665516051, 56.627545736969275, 55.73006558508802, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.81698075056278, "threshold_area100": 70.74801518812565, "threshold_area25": 81.66379012771833}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 0.7594062823610631}}, "old_normalized_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 0.7594062823610631}}, "old_normalized_endpoints": {"val": {"score": 98.91666666666666, "calls": 0.0}, "test": {"score": 90.16223679668623, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.08333333333333, 98.08333333333333, 98.0, 97.74999999999999, 97.33333333333333, 97.08333333333333, 96.91666666666666, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.5, 95.33333333333333, 95.16666666666667, 94.91666666666666, 94.66666666666667, 94.25, 94.08333333333333, 93.66666666666667, 93.25, 92.91666666666667, 92.41666666666666, 91.91666666666667, 91.66666666666666, 91.33333333333333, 90.75, 90.16666666666666, 89.75, 89.5, 88.91666666666667, 88.33333333333333, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.74999999999999, 84.16666666666667, 83.5, 82.75, 82.0, 81.41666666666667, 80.66666666666666, 80.08333333333333, 79.33333333333333, 78.66666666666666, 78.0, 77.08333333333333, 76.25, 75.5, 74.83333333333333, 73.83333333333333, 73.0, 72.16666666666667, 71.5, 70.41666666666666, 69.66666666666667, 68.91666666666667, 68.0, 67.08333333333333, 66.08333333333334, 65.16666666666666, 64.33333333333333, 63.74999999999999, 62.83333333333333, 61.83333333333333, 60.83333333333333, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.08333333333333, 56.08333333333333, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.08333333333333, 98.08333333333333, 98.0, 97.74999999999999, 97.33333333333333, 97.08333333333333, 96.91666666666666, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.5, 95.33333333333333, 95.16666666666667, 94.91666666666666, 94.66666666666667, 94.25, 94.08333333333333, 93.66666666666667, 93.25, 92.91666666666667, 92.41666666666666, 91.91666666666667, 91.66666666666666, 91.33333333333333, 90.75, 90.16666666666666, 89.75, 89.5, 88.91666666666667, 88.33333333333333, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.74999999999999, 84.16666666666667, 83.5, 82.75, 82.0, 81.33333333333333, 80.66666666666666, 80.08333333333333, 79.33333333333333, 78.66666666666666, 78.0, 77.08333333333333, 76.25, 75.5, 74.83333333333333, 73.83333333333333, 73.0, 72.16666666666667, 71.41666666666666, 70.41666666666666, 69.66666666666667, 68.91666666666667, 68.0, 67.08333333333333, 66.08333333333334, 65.16666666666666, 64.33333333333333, 63.74999999999999, 62.83333333333333, 61.83333333333333, 60.83333333333333, 59.833333333333336, 58.916666666666664, 57.91666666666666, 57.08333333333333, 56.08333333333333, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.65833333333333, "exact_area100": 86.53999999999999}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.09319986192614, 90.09319986192614, 90.05868139454608, 90.006903693476, 89.98964445978598, 89.95512599240594, 89.90334829133586, 89.8688298239558, 89.76527442181566, 89.7134967207456, 89.67897825336554, 89.60994131860545, 89.5063859164653, 89.42008974801519, 89.40283051432516, 89.29927511218501, 89.2129789437349, 89.04038660683466, 88.95409043838454, 88.88505350362443, 88.79875733517432, 88.74697963410424, 88.60890576458405, 88.53986882982396, 88.36727649292371, 88.19468415602347, 88.02209181912323, 87.7114256127028, 87.36624093890231, 87.29720400414222, 87.14187090093199, 86.96927856403175, 86.88298239558164, 86.79668622713152, 86.62409389023126, 86.24439074905074, 85.96824301001035, 85.81290990680014, 85.62305833620987, 85.27787366240939, 85.07076285812909, 84.74283741801864, 84.44943044528823, 84.27683810838799, 84.01794960303762, 83.7935795650673, 83.44839489126683, 83.25854332067657, 82.94787711425612, 82.75802554366585, 82.17121159820505, 81.96410079392474, 81.68795305488436, 81.51536071798412, 81.39454608215395, 81.04936140835346, 80.58336209872282, 79.94477045219192, 79.53054884363134, 79.13358646876078, 78.8574387297204, 78.44321712115982, 78.23610631687953, 77.5802554366586, 76.76907145322747, 76.51018294787711, 75.90610976872627, 75.12944425267519, 74.80151881256472, 74.0766309975837, 73.47255781843286, 72.74767000345183, 71.85018985157059, 71.36693130824992, 70.95270969968934, 70.27959958577839, 69.74456334138763, 68.72626855367622, 68.29478771142561, 67.65619606489471, 67.2074559889541, 66.53434587504314, 65.68864342423197, 65.0673110113911, 63.703831549879176, 62.909906800138074, 62.443907490507414, 60.959613393165334, 59.7169485674836, 58.81946841560235, 58.2844321712116, 57.6458405246807, 56.83465654124957, 56.07525025888851, 54.81532619951675], "calls": [0.0, 0.7594062823610631, 1.8467380048325857, 2.6924404556437693, 3.40006903693476, 4.452882292026234, 5.091473938557129, 5.453917846047635, 6.575768035899205, 7.973765964791163, 8.767690714532275, 9.509837763203313, 10.752502588885053, 11.56368657231619, 13.531239212978944, 14.135312392129789, 14.963755609250951, 16.25819813600276, 17.12115982050397, 17.966862271315154, 18.70900931998619, 19.27856403175699, 20.538488091128755, 22.143596824301, 22.85122540559199, 23.558853986882983, 24.352778736624096, 25.181221953745254, 26.19951674145668, 26.544701415257162, 27.925440110459093, 29.237141870900935, 30.462547462892648, 32.18847083189507, 33.759061097687265, 34.552985847428374, 35.329651363479464, 36.19261304798067, 36.76216775975147, 37.573351743182606, 38.48809112875388, 39.972385226095966, 41.16327235070763, 41.73282706247843, 42.76838108387987, 43.821194338971345, 44.8567483603728, 45.875043148084224, 46.789782533655504, 47.87711425612703, 49.05074214704867, 49.896444597859855, 51.08733172247152, 51.55333103210218, 52.39903348291336, 53.24473593372454, 55.38488091128754, 56.28236106316879, 57.21435968243011, 58.06006213324129, 58.26717293752157, 59.285467725233, 60.510873317224714, 62.08146358301691, 63.2896099413186, 64.23886779426994, 64.94649637556093, 66.1373835001726, 66.5688643424232, 67.7079737659648, 69.15774939592683, 69.98619261304798, 71.31515360717984, 72.48878149810149, 73.19641007939246, 74.38729720400414, 75.33655505695546, 76.49292371418709, 77.92544011045909, 78.66758715913014, 79.25440110459095, 80.27269589230238, 81.08387987573352, 82.5681739730756, 83.2067656196065, 84.0697273041077, 84.77735588539869, 85.70935450466, 86.93476009665171, 87.76320331377286, 89.59268208491544, 90.61097687262686, 91.1460131170176, 92.80289955125993, 94.33897134967208, 95.51259924059372, 96.11667241974456, 96.9278564031757, 97.79081808767691, 98.6365205384881, 100.0], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.09319986192614, 90.07594062823611, 90.05868139454608, 90.006903693476, 89.97238522609597, 89.95512599240594, 89.88608905764583, 89.85157059026578, 89.78253365550569, 89.74801518812563, 89.67897825336554, 89.62720055229548, 89.54090438384536, 89.42008974801519, 89.35105281325508, 89.24749741111494, 89.16120124266482, 89.05764584052469, 88.91957197100449, 88.85053503624438, 88.76423886779426, 88.66068346565412, 88.47083189506385, 88.33275802554367, 88.26372109078356, 88.09112875388332, 87.93579565067311, 87.76320331377286, 87.5733517431826, 87.34898170521228, 87.2109078356921, 87.02105626510182, 86.84846392820158, 86.72764929237141, 86.46876078702105, 86.24439074905074, 86.02002071108042, 85.77839143942009, 85.5712806351398, 85.26061442871936, 85.01898515705902, 84.72557818432861, 84.39765274421815, 84.24231964100794, 84.01794960303762, 83.72454263030721, 83.46565412495684, 83.12046945115637, 82.7925440110459, 82.56817397307559, 82.27476700034518, 82.01587849499482, 81.70521228857439, 81.51536071798412, 81.20469451156367, 80.70417673455297, 80.39351052813255, 79.9275112185019, 79.61684501208146, 79.2198826372109, 78.84017949603037, 78.47773558853987, 78.02899551259924, 77.39040386606834, 76.83810838798757, 76.51018294787711, 76.09596133931653, 75.45736969278563, 74.95685191577493, 74.33551950293406, 73.67966862271315, 73.02381774249223, 72.43700379703141, 71.86744908526062, 71.10804280289955, 70.43493268898861, 69.77908180876769, 69.19226786330687, 68.39834311356576, 67.7079737659648, 67.05212288574387, 66.3617535381429, 65.63686572316189, 64.9292371418709, 64.11805315843976, 63.410424577148774, 62.59924059371764, 61.73627890921642, 60.787021056265104, 59.99309630652399, 59.19917155678287, 58.387987573351744, 57.59406282361063, 56.6448049706593, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 89.75146703486365, "exact_area100": 80.1489128708367}}, "old_normalized_endpoint_budget": 0}, {"seed": 2028, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.0, "validation_area100": 0.8644916666666667, "training_seconds": 3.3679264889797196, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0023835061583667994, "operator": "ge"}, {"budget": 0.02, "threshold": -0.003112838137894869, "operator": "ge"}, {"budget": 0.03, "threshold": -0.0035786873195320368, "operator": "ge"}, {"budget": 0.04, "threshold": -0.004272919148206711, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0049032107926905155, "operator": "ge"}, {"budget": 0.06, "threshold": -0.006486354395747185, "operator": "ge"}, {"budget": 0.07, "threshold": -0.007351440843194723, "operator": "ge"}, {"budget": 0.08, "threshold": -0.008196823298931122, "operator": "ge"}, {"budget": 0.09, "threshold": -0.009530395269393921, "operator": "ge"}, {"budget": 0.1, "threshold": -0.010934695601463318, "operator": "ge"}, {"budget": 0.11, "threshold": -0.01228103507310152, "operator": "ge"}, {"budget": 0.12, "threshold": -0.015412414446473122, "operator": "ge"}, {"budget": 0.13, "threshold": -0.01884905807673931, "operator": "ge"}, {"budget": 0.14, "threshold": -0.020308757200837135, "operator": "ge"}, {"budget": 0.15, "threshold": -0.02319219522178173, "operator": "ge"}, {"budget": 0.16, "threshold": -0.025907443836331367, "operator": "ge"}, {"budget": 0.17, "threshold": -0.029415523633360863, "operator": "ge"}, {"budget": 0.18, "threshold": -0.03235640004277229, "operator": "ge"}, {"budget": 0.19, "threshold": -0.03525002673268318, "operator": "ge"}, {"budget": 0.2, "threshold": -0.03906075656414032, "operator": "ge"}, {"budget": 0.21, "threshold": -0.043676480650901794, "operator": "ge"}, {"budget": 0.22, "threshold": -0.04658643901348114, "operator": "ge"}, {"budget": 0.23, "threshold": -0.05024955794215202, "operator": "ge"}, {"budget": 0.24, "threshold": -0.05567043647170067, "operator": "ge"}, {"budget": 0.25, "threshold": -0.059784166514873505, "operator": "ge"}, {"budget": 0.26, "threshold": -0.06533505022525787, "operator": "ge"}, {"budget": 0.27, "threshold": -0.07190699130296707, "operator": "ge"}, {"budget": 0.28, "threshold": -0.07691418379545212, "operator": "ge"}, {"budget": 0.29, "threshold": -0.07986171543598175, "operator": "ge"}, {"budget": 0.3, "threshold": -0.08415331691503525, "operator": "ge"}, {"budget": 0.31, "threshold": -0.08973617106676102, "operator": "ge"}, {"budget": 0.32, "threshold": -0.0981699749827385, "operator": "ge"}, {"budget": 0.33, "threshold": -0.10380175709724426, "operator": "ge"}, {"budget": 0.34, "threshold": -0.11240337044000626, "operator": "ge"}, {"budget": 0.35, "threshold": -0.11869934946298599, "operator": "ge"}, {"budget": 0.36, "threshold": -0.1260053962469101, "operator": "ge"}, {"budget": 0.37, "threshold": -0.1374983936548233, "operator": "ge"}, {"budget": 0.38, "threshold": -0.1451984941959381, "operator": "ge"}, {"budget": 0.39, "threshold": -0.15530531108379364, "operator": "ge"}, {"budget": 0.4, "threshold": -0.1711006909608841, "operator": "ge"}, {"budget": 0.41, "threshold": -0.18386012315750122, "operator": "ge"}, {"budget": 0.42, "threshold": -0.20172694325447083, "operator": "ge"}, {"budget": 0.43, "threshold": -0.208962082862854, "operator": "ge"}, {"budget": 0.44, "threshold": -0.22240401804447174, "operator": "ge"}, {"budget": 0.45, "threshold": -0.23039843142032623, "operator": "ge"}, {"budget": 0.46, "threshold": -0.24597914516925812, "operator": "ge"}, {"budget": 0.47, "threshold": -0.2611447274684906, "operator": "ge"}, {"budget": 0.48, "threshold": -0.27193135023117065, "operator": "ge"}, {"budget": 0.49, "threshold": -0.2951240837574005, "operator": "ge"}, {"budget": 0.5, "threshold": -0.30604633688926697, "operator": "ge"}, {"budget": 0.51, "threshold": -0.32011500000953674, "operator": "ge"}, {"budget": 0.52, "threshold": -0.3307593762874603, "operator": "ge"}, {"budget": 0.53, "threshold": -0.3469543755054474, "operator": "ge"}, {"budget": 0.54, "threshold": -0.35997065901756287, "operator": "ge"}, {"budget": 0.55, "threshold": -0.38088566064834595, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4083470106124878, "operator": "ge"}, {"budget": 0.57, "threshold": -0.42813727259635925, "operator": "ge"}, {"budget": 0.58, "threshold": -0.44394779205322266, "operator": "ge"}, {"budget": 0.59, "threshold": -0.4610121548175812, "operator": "ge"}, {"budget": 0.6, "threshold": -0.47382327914237976, "operator": "ge"}, {"budget": 0.61, "threshold": -0.4922395646572113, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5115125179290771, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5236993432044983, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5474899411201477, "operator": "ge"}, {"budget": 0.65, "threshold": -0.5754616856575012, "operator": "ge"}, {"budget": 0.66, "threshold": -0.5859953761100769, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6108049750328064, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6345576643943787, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6603342294692993, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6795232892036438, "operator": "ge"}, {"budget": 0.71, "threshold": -0.6900299787521362, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7088171243667603, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7268452644348145, "operator": "ge"}, {"budget": 0.74, "threshold": -0.742982029914856, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7533509731292725, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7641764283180237, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7779803276062012, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7947103977203369, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8066563606262207, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8181406855583191, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8260201811790466, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8358357548713684, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8492600321769714, "operator": "ge"}, {"budget": 0.84, "threshold": -0.861258864402771, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8764960169792175, "operator": "ge"}, {"budget": 0.86, "threshold": -0.893776535987854, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9061459302902222, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9124121069908142, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9221014976501465, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9314813613891602, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9409233331680298, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9509974718093872, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9557244777679443, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9590325355529785, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9656230807304382, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9703435897827148, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9760898351669312, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9803281426429749, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9824733734130859, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.5, 98.41666666666666, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.74999999999999, 97.5, 97.41666666666666, 97.24999999999999, 97.16666666666667, 97.0, 97.0, 96.83333333333333, 96.5, 96.33333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.58333333333333, 95.25, 95.25, 94.75, 94.58333333333333, 94.25, 93.83333333333333, 93.58333333333333, 93.16666666666666, 92.91666666666667, 92.5, 92.33333333333333, 91.66666666666666, 90.99999999999999, 90.41666666666667, 90.08333333333333, 89.66666666666666, 89.16666666666666, 88.83333333333333, 88.25, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.25, 84.5, 84.08333333333333, 83.33333333333333, 82.58333333333333, 82.08333333333333, 81.58333333333333, 81.0, 80.33333333333333, 79.5, 78.58333333333333, 77.83333333333333, 76.91666666666667, 76.0, 75.5, 74.66666666666666, 73.83333333333333, 73.16666666666667, 72.24999999999999, 71.5, 70.66666666666667, 69.75, 69.0, 68.0, 67.08333333333333, 66.25, 65.33333333333333, 64.33333333333333, 63.416666666666664, 62.66666666666667, 61.74999999999999, 60.83333333333333, 59.916666666666664, 58.916666666666664, 57.99999999999999, 56.99999999999999, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 1, 1, 1, 1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3, 4, 5, 6, 8, 10, 11, 11, 12, 12, 13, 14, 14, 17, 18, 20, 21, 23, 23, 25, 29, 31, 34, 36, 38, 40, 44, 44, 50, 52, 56, 61, 64, 69, 72, 77, 79, 87, 95, 102, 106, 111, 117, 121, 128, 135, 141, 148, 156, 164, 173, 179, 188, 197, 203, 209, 216, 224, 234, 245, 254, 265, 276, 282, 292, 302, 310, 321, 330, 340, 351, 360, 372, 383, 393, 404, 416, 427, 437, 448, 459, 470, 482, 493, 505, 517, 528], "exact_score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.5, 98.41666666666666, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.74999999999999, 97.5, 97.41666666666666, 97.24999999999999, 97.16666666666667, 97.0, 97.0, 96.83333333333333, 96.5, 96.33333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.58333333333333, 95.25, 95.16666666666667, 94.75, 94.58333333333333, 94.25, 93.83333333333333, 93.58333333333333, 93.16666666666666, 92.91666666666667, 92.5, 92.33333333333333, 91.66666666666666, 90.99999999999999, 90.41666666666667, 90.08333333333333, 89.66666666666666, 89.16666666666666, 88.83333333333333, 88.25, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.25, 84.5, 84.08333333333333, 83.33333333333333, 82.58333333333333, 82.08333333333333, 81.58333333333333, 81.0, 80.33333333333333, 79.5, 78.58333333333333, 77.83333333333333, 76.91666666666667, 76.0, 75.5, 74.66666666666666, 73.83333333333333, 73.16666666666667, 72.24999999999999, 71.41666666666666, 70.66666666666667, 69.75, 69.0, 68.0, 67.08333333333333, 66.25, 65.33333333333333, 64.33333333333333, 63.416666666666664, 62.66666666666667, 61.74999999999999, 60.83333333333333, 59.916666666666664, 58.916666666666664, 57.99999999999999, 56.99999999999999, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.44916666666667, "threshold_area100": 86.45083333333334, "threshold_area25": 98.48333333333333}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.07594062823611, 90.05868139454608, 89.98964445978598, 89.90334829133586, 89.79979288919571, 89.78253365550569, 89.7134967207456, 89.59268208491544, 89.54090438384536, 89.47186744908525, 89.43734898170521, 89.23023817742491, 89.16120124266482, 89.04038660683466, 88.91957197100449, 88.85053503624438, 88.64342423196409, 88.43631342768381, 88.35001725923368, 88.28098032447359, 88.16016568864342, 88.0048325854332, 87.91853641698309, 87.79772178115292, 87.65964791163272, 87.52157404211253, 87.36624093890231, 87.19364860200207, 87.05557473248187, 86.81394546082154, 86.46876078702105, 86.26164998274076, 85.95098377632033, 85.84742837418018, 85.62305833620987, 85.41594753192958, 85.32965136347946, 85.19157749395927, 85.07076285812909, 84.38039351052812, 84.27683810838799, 83.94891266827752, 83.87987573351742, 83.46565412495684, 83.25854332067657, 82.72350707628581, 82.06765619606489, 81.61891612012427, 81.27373144632378, 81.10113910942354, 80.82499137038315, 80.2899551259924, 79.9275112185019, 79.56506731101139, 78.90921643079048, 78.33966171901967, 78.14981014842941, 77.5802554366586, 76.95892302381773, 76.35484984466689, 75.68173973075595, 75.35381429064549, 74.55988954090438, 73.99033482913359, 73.1446323783224, 72.78218847083188, 72.16085605799101, 71.57404211253021, 70.67656196064894, 70.22782188470832, 69.86537797721782, 69.46841560234725, 68.95063859164652, 68.48463928201586, 67.77701070072489, 67.01760441836382, 65.96479116327234, 65.11908871246118, 64.67034863652053, 63.9282015878495, 63.03072143596824, 62.30583362098723, 61.09768726268554, 60.49361408353469, 59.92405937176388, 58.802209181912325, 58.16361753538142, 57.2833966171902, 56.437694166379, 56.02347255781843, 54.81532619951675], "calls": [0.0, 0.8802209181912324, 2.105626510182948, 3.003106662064204, 3.9005868139454605, 4.625474628926476, 5.868139454608215, 6.368657231618916, 6.765619606489472, 7.62858129099068, 8.284432171211598, 8.888505350362443, 10.510873317224716, 11.8570935450466, 12.547462892647566, 13.77286848463928, 14.618570935450467, 15.982050396962375, 17.0693821194339, 18.191232309285468, 19.27856403175699, 20.55574732481878, 21.194338971349673, 22.040041422160854, 23.282706247842597, 24.024853296513633, 25.07766655160511, 26.182257507766654, 27.027959958577842, 27.49395926820849, 28.270624784259578, 29.150845702450813, 30.54884363134277, 31.34276838108388, 33.051432516396275, 33.82809803244736, 34.81187435277874, 36.0717984121505, 37.090093199861926, 38.05661028650328, 39.368312046945114, 40.52468070417674, 42.37141870900932, 42.9237141870901, 44.097342078011735, 44.90852606144287, 45.90956161546427, 47.03141180531584, 47.790818087676904, 49.98274076630997, 50.72488781498101, 51.656886434242324, 52.19192267863308, 53.34829133586469, 54.36658612357611, 55.93717638936832, 57.594062823610635, 58.99206075250258, 59.90680013807387, 60.73524335519503, 61.58094580600621, 62.72005522954781, 63.7901277183293, 64.46323783224025, 65.75768035899206, 67.17293752157404, 67.60441836382465, 68.79530548843631, 69.98619261304798, 71.29789437348981, 72.36796686227132, 72.90300310666207, 74.05937176389368, 74.95685191577493, 76.14773904038661, 76.83810838798757, 77.7183293061788, 78.49499482222988, 79.87573351743183, 80.61788056610285, 81.23921297894373, 81.73973075595443, 82.48187780462547, 83.27580255436658, 84.17328270624785, 85.26061442871936, 86.67587159130134, 87.86675871591302, 88.40179496030376, 89.33379357956507, 90.30031066620641, 91.23230928546772, 92.75112185018986, 93.47600966517086, 94.23541594753193, 95.59889540904383, 96.32378322402485, 97.34207801173628, 98.20503969623748, 98.65377977217811, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 3, 4, 6, 7, 8, 8, 9, 10, 12, 13, 13, 14, 18, 19, 21, 21, 24, 29, 32, 36, 40, 43, 46, 47, 52, 57, 60, 61, 63, 66, 69, 70, 75, 76, 77, 81, 82, 85, 90, 97, 98, 99, 100, 101, 103, 105, 107, 107, 111, 113, 115, 116, 118, 120, 123, 126, 130, 132, 133, 134, 134, 136, 140, 140, 143, 144, 144, 144, 144, 144, 144], "harmed": [0, 0, 0, 0, 1, 2, 2, 3, 4, 4, 5, 6, 10, 15, 21, 22, 26, 33, 36, 40, 42, 54, 58, 65, 72, 76, 88, 102, 107, 111, 119, 129, 136, 144, 153, 161, 171, 182, 192, 207, 227, 240, 262, 269, 284, 296, 304, 317, 327, 371, 381, 403, 410, 435, 452, 488, 529, 556, 578, 591, 610, 642, 668, 690, 729, 766, 778, 814, 855, 897, 937, 957, 1004, 1038, 1089, 1112, 1150, 1184, 1240, 1268, 1291, 1315, 1347, 1376, 1420, 1467, 1532, 1583, 1610, 1654, 1706, 1750, 1824, 1859, 1895, 1961, 1998, 2049, 2098, 2122, 2192], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.12771832930618, 90.09319986192614, 90.07594062823611, 90.04142216085606, 90.006903693476, 89.97238522609597, 89.88608905764583, 89.78253365550569, 89.78253365550569, 89.67897825336554, 89.59268208491544, 89.54090438384536, 89.47186744908525, 89.43734898170521, 89.38557128063513, 89.19571971004487, 89.05764584052469, 88.91957197100449, 88.86779426993441, 88.64342423196409, 88.48809112875388, 88.35001725923368, 88.22920262340351, 88.03935105281326, 87.97031411805315, 87.83224024853297, 87.7114256127028, 87.65964791163272, 87.46979634104245, 87.36624093890231, 87.19364860200207, 87.09009319986191, 86.83120469451156, 86.55505695547117, 86.34794615119088, 86.20987228167068, 86.0027614773904, 85.83016914049016, 85.62305833620987, 85.38142906454954, 85.32965136347946, 85.20883672764928, 85.03624439074905, 84.69105971694857, 84.38039351052812, 84.22506040731791, 83.89713496720745, 83.6382464618571, 83.3620987228167, 83.10321021746634, 82.70624784259579, 82.39558163617535, 81.96410079392474, 81.61891612012427, 81.27373144632378, 81.01484294097342, 80.61788056610285, 80.18639972385225, 79.77217811529168, 79.27166033828098, 78.84017949603037, 78.37418018639973, 78.04625474628925, 77.56299620296858, 76.95892302381773, 76.54470141525715, 75.95788746979633, 75.31929582326543, 74.62892647566449, 73.99033482913359, 73.23092854677252, 72.69589230238176, 71.95374525371074, 71.22885743872972, 70.59026579219882, 70.02071108042801, 69.29582326544701, 68.58819468415602, 67.91508457024507, 67.2074559889541, 66.49982740766309, 65.74042112530203, 65.03279254401104, 64.2216085605799, 63.324128408698655, 62.47842595788746, 61.73627890921642, 60.90783569209527, 60.13117017604418, 59.285467725233, 58.45702450811183, 57.59406282361063, 56.6448049706593, 55.69554711770797, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.37103688495438, "threshold_area100": 80.10459095616154, "threshold_area25": 89.7859855022437}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.0023835061583667994, "operator": "ge"}, {"budget": 0.02, "threshold": -0.003112838137894869, "operator": "ge"}, {"budget": 0.03, "threshold": -0.0035786873195320368, "operator": "ge"}, {"budget": 0.04, "threshold": -0.004272919148206711, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0049032107926905155, "operator": "ge"}, {"budget": 0.06, "threshold": -0.006486354395747185, "operator": "ge"}, {"budget": 0.07, "threshold": -0.007351440843194723, "operator": "ge"}, {"budget": 0.08, "threshold": -0.008196823298931122, "operator": "ge"}, {"budget": 0.09, "threshold": -0.009530395269393921, "operator": "ge"}, {"budget": 0.1, "threshold": -0.010934695601463318, "operator": "ge"}, {"budget": 0.11, "threshold": -0.01228103507310152, "operator": "ge"}, {"budget": 0.12, "threshold": -0.015412414446473122, "operator": "ge"}, {"budget": 0.13, "threshold": -0.01884905807673931, "operator": "ge"}, {"budget": 0.14, "threshold": -0.020308757200837135, "operator": "ge"}, {"budget": 0.15, "threshold": -0.02319219522178173, "operator": "ge"}, {"budget": 0.16, "threshold": -0.025907443836331367, "operator": "ge"}, {"budget": 0.17, "threshold": -0.029415523633360863, "operator": "ge"}, {"budget": 0.18, "threshold": -0.03235640004277229, "operator": "ge"}, {"budget": 0.19, "threshold": -0.03525002673268318, "operator": "ge"}, {"budget": 0.2, "threshold": -0.03906075656414032, "operator": "ge"}, {"budget": 0.21, "threshold": -0.043676480650901794, "operator": "ge"}, {"budget": 0.22, "threshold": -0.04658643901348114, "operator": "ge"}, {"budget": 0.23, "threshold": -0.05024955794215202, "operator": "ge"}, {"budget": 0.24, "threshold": -0.05567043647170067, "operator": "ge"}, {"budget": 0.25, "threshold": -0.059784166514873505, "operator": "ge"}, {"budget": 0.26, "threshold": -0.06533505022525787, "operator": "ge"}, {"budget": 0.27, "threshold": -0.07190699130296707, "operator": "ge"}, {"budget": 0.28, "threshold": -0.07691418379545212, "operator": "ge"}, {"budget": 0.29, "threshold": -0.07986171543598175, "operator": "ge"}, {"budget": 0.3, "threshold": -0.08415331691503525, "operator": "ge"}, {"budget": 0.31, "threshold": -0.08973617106676102, "operator": "ge"}, {"budget": 0.32, "threshold": -0.0981699749827385, "operator": "ge"}, {"budget": 0.33, "threshold": -0.10380175709724426, "operator": "ge"}, {"budget": 0.34, "threshold": -0.11240337044000626, "operator": "ge"}, {"budget": 0.35, "threshold": -0.11869934946298599, "operator": "ge"}, {"budget": 0.36, "threshold": -0.1260053962469101, "operator": "ge"}, {"budget": 0.37, "threshold": -0.1374983936548233, "operator": "ge"}, {"budget": 0.38, "threshold": -0.1451984941959381, "operator": "ge"}, {"budget": 0.39, "threshold": -0.15530531108379364, "operator": "ge"}, {"budget": 0.4, "threshold": -0.1711006909608841, "operator": "ge"}, {"budget": 0.41, "threshold": -0.18386012315750122, "operator": "ge"}, {"budget": 0.42, "threshold": -0.20172694325447083, "operator": "ge"}, {"budget": 0.43, "threshold": -0.208962082862854, "operator": "ge"}, {"budget": 0.44, "threshold": -0.22240401804447174, "operator": "ge"}, {"budget": 0.45, "threshold": -0.23039843142032623, "operator": "ge"}, {"budget": 0.46, "threshold": -0.24597914516925812, "operator": "ge"}, {"budget": 0.47, "threshold": -0.2611447274684906, "operator": "ge"}, {"budget": 0.48, "threshold": -0.27193135023117065, "operator": "ge"}, {"budget": 0.49, "threshold": -0.2951240837574005, "operator": "ge"}, {"budget": 0.5, "threshold": -0.30604633688926697, "operator": "ge"}, {"budget": 0.51, "threshold": -0.32011500000953674, "operator": "ge"}, {"budget": 0.52, "threshold": -0.3307593762874603, "operator": "ge"}, {"budget": 0.53, "threshold": -0.3469543755054474, "operator": "ge"}, {"budget": 0.54, "threshold": -0.35997065901756287, "operator": "ge"}, {"budget": 0.55, "threshold": -0.38088566064834595, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4083470106124878, "operator": "ge"}, {"budget": 0.57, "threshold": -0.42813727259635925, "operator": "ge"}, {"budget": 0.58, "threshold": -0.44394779205322266, "operator": "ge"}, {"budget": 0.59, "threshold": -0.4610121548175812, "operator": "ge"}, {"budget": 0.6, "threshold": -0.47382327914237976, "operator": "ge"}, {"budget": 0.61, "threshold": -0.4922395646572113, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5115125179290771, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5236993432044983, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5474899411201477, "operator": "ge"}, {"budget": 0.65, "threshold": -0.5754616856575012, "operator": "ge"}, {"budget": 0.66, "threshold": -0.5859953761100769, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6108049750328064, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6345576643943787, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6603342294692993, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6795232892036438, "operator": "ge"}, {"budget": 0.71, "threshold": -0.6900299787521362, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7088171243667603, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7268452644348145, "operator": "ge"}, {"budget": 0.74, "threshold": -0.742982029914856, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7533509731292725, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7641764283180237, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7779803276062012, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7947103977203369, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8066563606262207, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8181406855583191, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8260201811790466, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8358357548713684, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8492600321769714, "operator": "ge"}, {"budget": 0.84, "threshold": -0.861258864402771, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8764960169792175, "operator": "ge"}, {"budget": 0.86, "threshold": -0.893776535987854, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9061459302902222, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9124121069908142, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9221014976501465, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9314813613891602, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9409233331680298, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9509974718093872, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9557244777679443, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9590325355529785, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9656230807304382, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9703435897827148, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9760898351669312, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9803281426429749, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9824733734130859, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.5, 98.41666666666666, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.74999999999999, 97.5, 97.41666666666666, 97.24999999999999, 97.16666666666667, 97.0, 97.0, 96.83333333333333, 96.5, 96.33333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.58333333333333, 95.25, 95.25, 94.75, 94.58333333333333, 94.25, 93.83333333333333, 93.58333333333333, 93.16666666666666, 92.91666666666667, 92.5, 92.33333333333333, 91.66666666666666, 90.99999999999999, 90.41666666666667, 90.08333333333333, 89.66666666666666, 89.16666666666666, 88.83333333333333, 88.25, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.25, 84.5, 84.08333333333333, 83.33333333333333, 82.58333333333333, 82.08333333333333, 81.58333333333333, 81.0, 80.33333333333333, 79.5, 78.58333333333333, 77.83333333333333, 76.91666666666667, 76.0, 75.5, 74.66666666666666, 73.83333333333333, 73.16666666666667, 72.24999999999999, 71.5, 70.66666666666667, 69.75, 69.0, 68.0, 67.08333333333333, 66.25, 65.33333333333333, 64.33333333333333, 63.416666666666664, 62.66666666666667, 61.74999999999999, 60.83333333333333, 59.916666666666664, 58.916666666666664, 57.99999999999999, 56.99999999999999, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 1, 1, 1, 1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3, 4, 5, 6, 8, 10, 11, 11, 12, 12, 13, 14, 14, 17, 18, 20, 21, 23, 23, 25, 29, 31, 34, 36, 38, 40, 44, 44, 50, 52, 56, 61, 64, 69, 72, 77, 79, 87, 95, 102, 106, 111, 117, 121, 128, 135, 141, 148, 156, 164, 173, 179, 188, 197, 203, 209, 216, 224, 234, 245, 254, 265, 276, 282, 292, 302, 310, 321, 330, 340, 351, 360, 372, 383, 393, 404, 416, 427, 437, 448, 459, 470, 482, 493, 505, 517, 528], "exact_score": [98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.5, 98.41666666666666, 98.25, 98.08333333333333, 98.0, 98.0, 97.91666666666666, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.74999999999999, 97.5, 97.41666666666666, 97.24999999999999, 97.16666666666667, 97.0, 97.0, 96.83333333333333, 96.5, 96.33333333333333, 96.08333333333333, 95.91666666666666, 95.75, 95.58333333333333, 95.25, 95.16666666666667, 94.75, 94.58333333333333, 94.25, 93.83333333333333, 93.58333333333333, 93.16666666666666, 92.91666666666667, 92.5, 92.33333333333333, 91.66666666666666, 90.99999999999999, 90.41666666666667, 90.08333333333333, 89.66666666666666, 89.16666666666666, 88.83333333333333, 88.25, 87.66666666666666, 87.16666666666666, 86.58333333333333, 85.91666666666666, 85.25, 84.5, 84.08333333333333, 83.33333333333333, 82.58333333333333, 82.08333333333333, 81.58333333333333, 81.0, 80.33333333333333, 79.5, 78.58333333333333, 77.83333333333333, 76.91666666666667, 76.0, 75.5, 74.66666666666666, 73.83333333333333, 73.16666666666667, 72.24999999999999, 71.41666666666666, 70.66666666666667, 69.75, 69.0, 68.0, 67.08333333333333, 66.25, 65.33333333333333, 64.33333333333333, 63.416666666666664, 62.66666666666667, 61.74999999999999, 60.83333333333333, 59.916666666666664, 58.916666666666664, 57.99999999999999, 56.99999999999999, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.44916666666667, "threshold_area100": 86.45083333333334, "threshold_area25": 98.48333333333333}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.07594062823611, 90.05868139454608, 89.98964445978598, 89.90334829133586, 89.79979288919571, 89.78253365550569, 89.7134967207456, 89.59268208491544, 89.54090438384536, 89.47186744908525, 89.43734898170521, 89.23023817742491, 89.16120124266482, 89.04038660683466, 88.91957197100449, 88.85053503624438, 88.64342423196409, 88.43631342768381, 88.35001725923368, 88.28098032447359, 88.16016568864342, 88.0048325854332, 87.91853641698309, 87.79772178115292, 87.65964791163272, 87.52157404211253, 87.36624093890231, 87.19364860200207, 87.05557473248187, 86.81394546082154, 86.46876078702105, 86.26164998274076, 85.95098377632033, 85.84742837418018, 85.62305833620987, 85.41594753192958, 85.32965136347946, 85.19157749395927, 85.07076285812909, 84.38039351052812, 84.27683810838799, 83.94891266827752, 83.87987573351742, 83.46565412495684, 83.25854332067657, 82.72350707628581, 82.06765619606489, 81.61891612012427, 81.27373144632378, 81.10113910942354, 80.82499137038315, 80.2899551259924, 79.9275112185019, 79.56506731101139, 78.90921643079048, 78.33966171901967, 78.14981014842941, 77.5802554366586, 76.95892302381773, 76.35484984466689, 75.68173973075595, 75.35381429064549, 74.55988954090438, 73.99033482913359, 73.1446323783224, 72.78218847083188, 72.16085605799101, 71.57404211253021, 70.67656196064894, 70.22782188470832, 69.86537797721782, 69.46841560234725, 68.95063859164652, 68.48463928201586, 67.77701070072489, 67.01760441836382, 65.96479116327234, 65.11908871246118, 64.67034863652053, 63.9282015878495, 63.03072143596824, 62.30583362098723, 61.09768726268554, 60.49361408353469, 59.92405937176388, 58.802209181912325, 58.16361753538142, 57.2833966171902, 56.437694166379, 56.02347255781843, 54.81532619951675], "calls": [0.0, 0.8802209181912324, 2.105626510182948, 3.003106662064204, 3.9005868139454605, 4.625474628926476, 5.868139454608215, 6.368657231618916, 6.765619606489472, 7.62858129099068, 8.284432171211598, 8.888505350362443, 10.510873317224716, 11.8570935450466, 12.547462892647566, 13.77286848463928, 14.618570935450467, 15.982050396962375, 17.0693821194339, 18.191232309285468, 19.27856403175699, 20.55574732481878, 21.194338971349673, 22.040041422160854, 23.282706247842597, 24.024853296513633, 25.07766655160511, 26.182257507766654, 27.027959958577842, 27.49395926820849, 28.270624784259578, 29.150845702450813, 30.54884363134277, 31.34276838108388, 33.051432516396275, 33.82809803244736, 34.81187435277874, 36.0717984121505, 37.090093199861926, 38.05661028650328, 39.368312046945114, 40.52468070417674, 42.37141870900932, 42.9237141870901, 44.097342078011735, 44.90852606144287, 45.90956161546427, 47.03141180531584, 47.790818087676904, 49.98274076630997, 50.72488781498101, 51.656886434242324, 52.19192267863308, 53.34829133586469, 54.36658612357611, 55.93717638936832, 57.594062823610635, 58.99206075250258, 59.90680013807387, 60.73524335519503, 61.58094580600621, 62.72005522954781, 63.7901277183293, 64.46323783224025, 65.75768035899206, 67.17293752157404, 67.60441836382465, 68.79530548843631, 69.98619261304798, 71.29789437348981, 72.36796686227132, 72.90300310666207, 74.05937176389368, 74.95685191577493, 76.14773904038661, 76.83810838798757, 77.7183293061788, 78.49499482222988, 79.87573351743183, 80.61788056610285, 81.23921297894373, 81.73973075595443, 82.48187780462547, 83.27580255436658, 84.17328270624785, 85.26061442871936, 86.67587159130134, 87.86675871591302, 88.40179496030376, 89.33379357956507, 90.30031066620641, 91.23230928546772, 92.75112185018986, 93.47600966517086, 94.23541594753193, 95.59889540904383, 96.32378322402485, 97.34207801173628, 98.20503969623748, 98.65377977217811, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 3, 4, 6, 7, 8, 8, 9, 10, 12, 13, 13, 14, 18, 19, 21, 21, 24, 29, 32, 36, 40, 43, 46, 47, 52, 57, 60, 61, 63, 66, 69, 70, 75, 76, 77, 81, 82, 85, 90, 97, 98, 99, 100, 101, 103, 105, 107, 107, 111, 113, 115, 116, 118, 120, 123, 126, 130, 132, 133, 134, 134, 136, 140, 140, 143, 144, 144, 144, 144, 144, 144], "harmed": [0, 0, 0, 0, 1, 2, 2, 3, 4, 4, 5, 6, 10, 15, 21, 22, 26, 33, 36, 40, 42, 54, 58, 65, 72, 76, 88, 102, 107, 111, 119, 129, 136, 144, 153, 161, 171, 182, 192, 207, 227, 240, 262, 269, 284, 296, 304, 317, 327, 371, 381, 403, 410, 435, 452, 488, 529, 556, 578, 591, 610, 642, 668, 690, 729, 766, 778, 814, 855, 897, 937, 957, 1004, 1038, 1089, 1112, 1150, 1184, 1240, 1268, 1291, 1315, 1347, 1376, 1420, 1467, 1532, 1583, 1610, 1654, 1706, 1750, 1824, 1859, 1895, 1961, 1998, 2049, 2098, 2122, 2192], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.12771832930618, 90.09319986192614, 90.07594062823611, 90.04142216085606, 90.006903693476, 89.97238522609597, 89.88608905764583, 89.78253365550569, 89.78253365550569, 89.67897825336554, 89.59268208491544, 89.54090438384536, 89.47186744908525, 89.43734898170521, 89.38557128063513, 89.19571971004487, 89.05764584052469, 88.91957197100449, 88.86779426993441, 88.64342423196409, 88.48809112875388, 88.35001725923368, 88.22920262340351, 88.03935105281326, 87.97031411805315, 87.83224024853297, 87.7114256127028, 87.65964791163272, 87.46979634104245, 87.36624093890231, 87.19364860200207, 87.09009319986191, 86.83120469451156, 86.55505695547117, 86.34794615119088, 86.20987228167068, 86.0027614773904, 85.83016914049016, 85.62305833620987, 85.38142906454954, 85.32965136347946, 85.20883672764928, 85.03624439074905, 84.69105971694857, 84.38039351052812, 84.22506040731791, 83.89713496720745, 83.6382464618571, 83.3620987228167, 83.10321021746634, 82.70624784259579, 82.39558163617535, 81.96410079392474, 81.61891612012427, 81.27373144632378, 81.01484294097342, 80.61788056610285, 80.18639972385225, 79.77217811529168, 79.27166033828098, 78.84017949603037, 78.37418018639973, 78.04625474628925, 77.56299620296858, 76.95892302381773, 76.54470141525715, 75.95788746979633, 75.31929582326543, 74.62892647566449, 73.99033482913359, 73.23092854677252, 72.69589230238176, 71.95374525371074, 71.22885743872972, 70.59026579219882, 70.02071108042801, 69.29582326544701, 68.58819468415602, 67.91508457024507, 67.2074559889541, 66.49982740766309, 65.74042112530203, 65.03279254401104, 64.2216085605799, 63.324128408698655, 62.47842595788746, 61.73627890921642, 60.90783569209527, 60.13117017604418, 59.285467725233, 58.45702450811183, 57.59406282361063, 56.6448049706593, 55.69554711770797, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.37103688495438, "threshold_area100": 80.10459095616154, "threshold_area25": 89.7859855022437}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.701244354248047, "operator": "ge"}, {"budget": 0.02, "threshold": -3.516974687576294, "operator": "ge"}, {"budget": 0.03, "threshold": -4.142803192138672, "operator": "ge"}, {"budget": 0.04, "threshold": -4.590770721435547, "operator": "ge"}, {"budget": 0.05, "threshold": -4.979846000671387, "operator": "ge"}, {"budget": 0.06, "threshold": -5.257402420043945, "operator": "ge"}, {"budget": 0.07, "threshold": -5.493250846862793, "operator": "ge"}, {"budget": 0.08, "threshold": -5.6310014724731445, "operator": "ge"}, {"budget": 0.09, "threshold": -5.755727291107178, "operator": "ge"}, {"budget": 0.1, "threshold": -5.883551597595215, "operator": "ge"}, {"budget": 0.11, "threshold": -6.052886486053467, "operator": "ge"}, {"budget": 0.12, "threshold": -6.151517391204834, "operator": "ge"}, {"budget": 0.13, "threshold": -6.223684310913086, "operator": "ge"}, {"budget": 0.14, "threshold": -6.361034870147705, "operator": "ge"}, {"budget": 0.15, "threshold": -6.454439163208008, "operator": "ge"}, {"budget": 0.16, "threshold": -6.544651508331299, "operator": "ge"}, {"budget": 0.17, "threshold": -6.618029594421387, "operator": "ge"}, {"budget": 0.18, "threshold": -6.6814985275268555, "operator": "ge"}, {"budget": 0.19, "threshold": -6.7736406326293945, "operator": "ge"}, {"budget": 0.2, "threshold": -6.860288619995117, "operator": "ge"}, {"budget": 0.21, "threshold": -6.9270429611206055, "operator": "ge"}, {"budget": 0.22, "threshold": -7.0092692375183105, "operator": "ge"}, {"budget": 0.23, "threshold": -7.060999870300293, "operator": "ge"}, {"budget": 0.24, "threshold": -7.128376007080078, "operator": "ge"}, {"budget": 0.25, "threshold": -7.200171947479248, "operator": "ge"}, {"budget": 0.26, "threshold": -7.304090976715088, "operator": "ge"}, {"budget": 0.27, "threshold": -7.369202613830566, "operator": "ge"}, {"budget": 0.28, "threshold": -7.429177761077881, "operator": "ge"}, {"budget": 0.29, "threshold": -7.469448089599609, "operator": "ge"}, {"budget": 0.3, "threshold": -7.518032073974609, "operator": "ge"}, {"budget": 0.31, "threshold": -7.616887092590332, "operator": "ge"}, {"budget": 0.32, "threshold": -7.686186790466309, "operator": "ge"}, {"budget": 0.33, "threshold": -7.730569839477539, "operator": "ge"}, {"budget": 0.34, "threshold": -7.771920204162598, "operator": "ge"}, {"budget": 0.35, "threshold": -7.844364166259766, "operator": "ge"}, {"budget": 0.36, "threshold": -7.901529788970947, "operator": "ge"}, {"budget": 0.37, "threshold": -7.942480564117432, "operator": "ge"}, {"budget": 0.38, "threshold": -8.020444869995117, "operator": "ge"}, {"budget": 0.39, "threshold": -8.084893226623535, "operator": "ge"}, {"budget": 0.4, "threshold": -8.128029823303223, "operator": "ge"}, {"budget": 0.41, "threshold": -8.167457580566406, "operator": "ge"}, {"budget": 0.42, "threshold": -8.252185821533203, "operator": "ge"}, {"budget": 0.43, "threshold": -8.304393768310547, "operator": "ge"}, {"budget": 0.44, "threshold": -8.381402969360352, "operator": "ge"}, {"budget": 0.45, "threshold": -8.458871841430664, "operator": "ge"}, {"budget": 0.46, "threshold": -8.505516052246094, "operator": "ge"}, {"budget": 0.47, "threshold": -8.558755874633789, "operator": "ge"}, {"budget": 0.48, "threshold": -8.594503402709961, "operator": "ge"}, {"budget": 0.49, "threshold": -8.651769638061523, "operator": "ge"}, {"budget": 0.5, "threshold": -8.70377254486084, "operator": "ge"}, {"budget": 0.51, "threshold": -8.751157760620117, "operator": "ge"}, {"budget": 0.52, "threshold": -8.792488098144531, "operator": "ge"}, {"budget": 0.53, "threshold": -8.843733787536621, "operator": "ge"}, {"budget": 0.54, "threshold": -8.900239944458008, "operator": "ge"}, {"budget": 0.55, "threshold": -8.942278861999512, "operator": "ge"}, {"budget": 0.56, "threshold": -9.03270435333252, "operator": "ge"}, {"budget": 0.57, "threshold": -9.070072174072266, "operator": "ge"}, {"budget": 0.58, "threshold": -9.145186424255371, "operator": "ge"}, {"budget": 0.59, "threshold": -9.185422897338867, "operator": "ge"}, {"budget": 0.6, "threshold": -9.235937118530273, "operator": "ge"}, {"budget": 0.61, "threshold": -9.301259994506836, "operator": "ge"}, {"budget": 0.62, "threshold": -9.372109413146973, "operator": "ge"}, {"budget": 0.63, "threshold": -9.432830810546875, "operator": "ge"}, {"budget": 0.64, "threshold": -9.5193510055542, "operator": "ge"}, {"budget": 0.65, "threshold": -9.61212158203125, "operator": "ge"}, {"budget": 0.66, "threshold": -9.666360855102539, "operator": "ge"}, {"budget": 0.67, "threshold": -9.754644393920898, "operator": "ge"}, {"budget": 0.68, "threshold": -9.828554153442383, "operator": "ge"}, {"budget": 0.69, "threshold": -9.876275062561035, "operator": "ge"}, {"budget": 0.7, "threshold": -9.956195831298828, "operator": "ge"}, {"budget": 0.71, "threshold": -10.013998985290527, "operator": "ge"}, {"budget": 0.72, "threshold": -10.106887817382812, "operator": "ge"}, {"budget": 0.73, "threshold": -10.179841995239258, "operator": "ge"}, {"budget": 0.74, "threshold": -10.289916038513184, "operator": "ge"}, {"budget": 0.75, "threshold": -10.359243392944336, "operator": "ge"}, {"budget": 0.76, "threshold": -10.464448928833008, "operator": "ge"}, {"budget": 0.77, "threshold": -10.538214683532715, "operator": "ge"}, {"budget": 0.78, "threshold": -10.611804962158203, "operator": "ge"}, {"budget": 0.79, "threshold": -10.712141036987305, "operator": "ge"}, {"budget": 0.8, "threshold": -10.822002410888672, "operator": "ge"}, {"budget": 0.81, "threshold": -10.932355880737305, "operator": "ge"}, {"budget": 0.82, "threshold": -11.024893760681152, "operator": "ge"}, {"budget": 0.83, "threshold": -11.088069915771484, "operator": "ge"}, {"budget": 0.84, "threshold": -11.150269508361816, "operator": "ge"}, {"budget": 0.85, "threshold": -11.268232345581055, "operator": "ge"}, {"budget": 0.86, "threshold": -11.411209106445312, "operator": "ge"}, {"budget": 0.87, "threshold": -11.509292602539062, "operator": "ge"}, {"budget": 0.88, "threshold": -11.587913513183594, "operator": "ge"}, {"budget": 0.89, "threshold": -11.693353652954102, "operator": "ge"}, {"budget": 0.9, "threshold": -11.814263343811035, "operator": "ge"}, {"budget": 0.91, "threshold": -11.932039260864258, "operator": "ge"}, {"budget": 0.92, "threshold": -12.033369064331055, "operator": "ge"}, {"budget": 0.93, "threshold": -12.126937866210938, "operator": "ge"}, {"budget": 0.94, "threshold": -12.216347694396973, "operator": "ge"}, {"budget": 0.95, "threshold": -12.383750915527344, "operator": "ge"}, {"budget": 0.96, "threshold": -12.518563270568848, "operator": "ge"}, {"budget": 0.97, "threshold": -12.734485626220703, "operator": "ge"}, {"budget": 0.98, "threshold": -12.916337013244629, "operator": "ge"}, {"budget": 0.99, "threshold": -13.417989730834961, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.33333333333333, 98.0, 97.83333333333333, 97.24999999999999, 97.0, 96.66666666666667, 96.5, 96.08333333333333, 95.91666666666666, 95.58333333333333, 95.25, 95.16666666666667, 94.75, 94.33333333333334, 94.08333333333333, 93.91666666666666, 93.66666666666667, 93.5, 93.16666666666666, 93.0, 92.58333333333333, 92.41666666666666, 92.08333333333333, 91.83333333333333, 91.75, 91.49999999999999, 91.08333333333333, 90.83333333333333, 90.58333333333333, 90.25, 89.99999999999999, 89.91666666666667, 89.66666666666666, 89.41666666666667, 89.16666666666666, 88.91666666666667, 88.75, 88.41666666666667, 88.16666666666666, 87.75, 87.5, 87.08333333333333, 86.91666666666666, 86.66666666666667, 86.25, 85.91666666666666, 85.83333333333333, 85.58333333333333, 85.16666666666666, 84.83333333333333, 84.25, 84.08333333333333, 83.83333333333333, 83.66666666666667, 83.41666666666666, 83.0, 82.91666666666666, 82.33333333333333, 81.75, 81.33333333333333, 80.91666666666666, 80.58333333333333, 80.16666666666666, 79.75, 79.41666666666667, 78.91666666666666, 78.41666666666667, 78.0, 77.33333333333333, 76.75, 76.25, 75.75, 75.16666666666667, 74.41666666666666, 73.75, 73.16666666666667, 72.33333333333334, 71.75, 71.0, 70.33333333333334, 69.5, 68.91666666666667, 68.33333333333333, 67.83333333333333, 67.08333333333333, 66.33333333333333, 65.5, 64.5, 63.66666666666667, 63.16666666666666, 62.416666666666664, 61.58333333333332, 60.83333333333333, 60.083333333333336, 59.16666666666666, 58.416666666666664, 57.58333333333333, 56.749999999999986, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 7, 11, 13, 20, 23, 27, 29, 34, 37, 41, 45, 46, 51, 56, 59, 61, 64, 66, 70, 72, 77, 79, 83, 86, 87, 90, 95, 98, 101, 105, 108, 109, 112, 115, 118, 121, 123, 127, 130, 135, 138, 143, 145, 148, 153, 157, 158, 161, 166, 170, 177, 179, 182, 184, 187, 192, 194, 201, 208, 213, 218, 222, 227, 232, 236, 242, 248, 253, 261, 268, 274, 280, 287, 296, 304, 311, 321, 328, 337, 345, 355, 362, 369, 375, 384, 393, 403, 415, 425, 431, 440, 450, 459, 468, 479, 488, 498, 508, 517, 528], "exact_score": [98.91666666666666, 98.33333333333333, 98.0, 97.83333333333333, 97.24999999999999, 97.0, 96.66666666666667, 96.5, 96.08333333333333, 95.91666666666666, 95.58333333333333, 95.25, 95.16666666666667, 94.75, 94.33333333333334, 94.08333333333333, 93.91666666666666, 93.66666666666667, 93.5, 93.16666666666666, 93.0, 92.58333333333333, 92.41666666666666, 92.08333333333333, 91.83333333333333, 91.75, 91.49999999999999, 91.08333333333333, 90.83333333333333, 90.58333333333333, 90.25, 89.99999999999999, 89.91666666666667, 89.66666666666666, 89.41666666666667, 89.16666666666666, 88.91666666666667, 88.75, 88.41666666666667, 88.16666666666666, 87.75, 87.41666666666667, 87.08333333333333, 86.91666666666666, 86.66666666666667, 86.25, 85.91666666666666, 85.83333333333333, 85.58333333333333, 85.16666666666666, 84.83333333333333, 84.25, 84.08333333333333, 83.83333333333333, 83.66666666666667, 83.41666666666666, 83.0, 82.83333333333334, 82.33333333333333, 81.75, 81.33333333333333, 80.91666666666666, 80.58333333333333, 80.16666666666666, 79.75, 79.41666666666667, 78.91666666666666, 78.41666666666667, 78.0, 77.33333333333333, 76.75, 76.25, 75.75, 75.16666666666667, 74.41666666666666, 73.75, 73.16666666666667, 72.33333333333334, 71.75, 71.0, 70.33333333333334, 69.5, 68.83333333333333, 68.33333333333333, 67.83333333333333, 67.08333333333333, 66.33333333333333, 65.5, 64.5, 63.66666666666667, 63.16666666666666, 62.416666666666664, 61.58333333333332, 60.83333333333333, 60.083333333333336, 59.16666666666666, 58.416666666666664, 57.58333333333333, 56.749999999999986, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 82.06, "threshold_area100": 82.0625, "threshold_area25": 95.0}, "test": {"score": [90.16223679668623, 89.52364515015533, 88.85053503624438, 88.0048325854332, 87.36624093890231, 86.65861235761132, 86.14083534691059, 85.70935450465998, 85.45046599930963, 85.15705902657922, 84.86365205384881, 84.36313427683811, 84.12150500517777, 83.8626164998274, 83.53469105971695, 83.25854332067657, 83.03417328270625, 82.7925440110459, 82.48187780462547, 81.8605453917846, 81.60165688643424, 81.2909906800138, 81.03210217466345, 80.84225060407317, 80.54884363134276, 80.3589920607525, 80.11736278909216, 79.68588194684155, 79.25440110459095, 79.06454953400069, 78.8574387297204, 78.39143942008975, 78.09803244735933, 77.87366240938903, 77.73558853986883, 77.40766309975837, 77.11425612702796, 76.95892302381773, 76.54470141525715, 76.2858129099068, 75.97514670348636, 75.75077666551606, 75.12944425267519, 74.97411114946496, 74.61166724197446, 74.2147048671039, 74.0075940628236, 73.78322402485328, 73.59337245426303, 73.38626164998274, 73.24818778046254, 73.0065585088022, 72.78218847083188, 72.55781843286158, 72.22989299275112, 71.88470831895064, 71.53952364515015, 71.45322747670004, 71.1425612702796, 70.95270969968934, 70.67656196064894, 70.38315498791854, 69.93441491197791, 69.69278564031757, 69.46841560234725, 69.12323092854677, 68.89886089057646, 68.34656541249569, 67.88056610286503, 67.58715913013462, 67.08664135312392, 66.74145667932343, 66.39627200552295, 66.05108733172247, 65.5678287884018, 65.32619951674145, 64.84294097342078, 64.60131170176044, 64.34242319641008, 63.979979288919566, 63.410424577148774, 63.01346220227821, 62.633759061097685, 62.2885743872972, 62.02968588194684, 61.61546427338626, 61.20124266482567, 60.873317224715215, 60.648947186744905, 60.26924404556438, 59.75146703486365, 59.181912323092845, 58.974801518812555, 58.54332067656196, 58.21539523645149, 57.628581290990674, 57.21435968243009, 56.69658267172937, 56.10976872626855, 55.28132550914739, 54.81532619951675], "calls": [0.0, 4.142216085605799, 7.645840524680704, 10.683465654124957, 13.893683120469452, 16.91404901622368, 18.881601656886435, 20.711080428028996, 22.005522954780808, 23.041076976182257, 24.318260269244046, 26.04418363824646, 27.286848463928205, 27.994477045219192, 29.47877114256127, 30.738695201933037, 31.61891612012427, 32.430100103555404, 33.37935795650673, 35.053503624439074, 36.2098722816707, 37.19364860200207, 38.45357266137383, 39.24749741111494, 40.421125302036586, 41.68104936140835, 43.06178805661028, 44.47704521919227, 45.75422851225406, 46.53089402830514, 47.497411114946495, 49.10251984811874, 50.06903693476009, 50.72488781498101, 51.27718329306179, 52.27821884708319, 53.34829133586469, 53.84880911287539, 55.005177770107004, 55.95443562305834, 56.679323438039354, 57.525025888850536, 58.940283051432516, 59.647911632723506, 60.75250258888505, 61.822575077666556, 62.46116672419745, 63.34138764238868, 63.9972385226096, 64.80842250604073, 65.37797721781153, 66.01656886434243, 66.8622713151536, 67.60441836382465, 68.46738004832585, 69.38211943389715, 70.62478425957887, 71.03900586813945, 72.12633759061097, 72.59233690024163, 73.38626164998274, 74.18018639972385, 75.12944425267519, 75.69899896444598, 76.5792198826372, 77.56299620296859, 77.99447704521918, 79.01277183293061, 79.996548153262, 80.56610286503279, 81.46358301691406, 82.03313772868485, 82.65447014152572, 83.27580255436658, 84.1387642388678, 84.67380048325855, 85.65757680358992, 86.17535381429065, 86.72764929237142, 87.26268553676216, 88.2119433897135, 89.02312737314463, 89.76527442181566, 90.23127373144632, 90.57645840524681, 91.19779081808768, 91.90541939937866, 92.33690024162927, 92.68208491542975, 93.19986192613048, 93.8557128063514, 94.64963755609251, 94.9257852951329, 95.47808077321366, 95.90956161546428, 96.5826717293752, 97.10044874007593, 97.75629962029686, 98.42940973420781, 99.48222298929927, 100.0], "rescued": [0, 20, 33, 43, 58, 69, 80, 88, 92, 95, 96, 96, 99, 100, 104, 107, 107, 107, 107, 110, 112, 112, 113, 114, 115, 116, 118, 121, 121, 121, 122, 124, 125, 125, 125, 125, 128, 128, 128, 128, 128, 128, 128, 130, 130, 130, 131, 131, 131, 131, 131, 131, 131, 132, 133, 133, 134, 134, 135, 135, 135, 135, 136, 136, 138, 138, 138, 138, 139, 139, 139, 139, 139, 139, 139, 139, 139, 140, 140, 140, 142, 143, 143, 143, 143, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 57, 109, 168, 220, 272, 313, 346, 365, 385, 403, 432, 449, 465, 488, 507, 520, 534, 552, 591, 608, 626, 642, 654, 672, 684, 700, 728, 753, 764, 777, 806, 824, 837, 845, 864, 884, 893, 917, 932, 950, 963, 999, 1010, 1031, 1054, 1067, 1080, 1091, 1103, 1111, 1125, 1138, 1152, 1172, 1192, 1213, 1218, 1237, 1248, 1264, 1281, 1308, 1322, 1337, 1357, 1370, 1402, 1430, 1447, 1476, 1496, 1516, 1536, 1564, 1578, 1606, 1621, 1636, 1657, 1692, 1716, 1738, 1758, 1773, 1798, 1822, 1841, 1854, 1876, 1906, 1939, 1951, 1976, 1995, 2029, 2053, 2083, 2117, 2165, 2192], "exact_score": [90.16223679668623, 90.1449775629962, 89.98964445978598, 89.78253365550569, 89.55816361753539, 89.47186744908525, 89.16120124266482, 88.90231273731446, 88.78149810148429, 88.52260959613393, 88.17742492233344, 87.91853641698309, 87.74594408008284, 87.5733517431826, 87.34898170521228, 87.19364860200207, 86.95201933034173, 86.65861235761132, 86.43424231964102, 86.08905764584051, 85.84742837418018, 85.60579910251984, 85.45046599930963, 85.19157749395927, 84.88091128753882, 84.62202278218847, 84.38039351052812, 84.17328270624785, 83.8626164998274, 83.65550569554712, 83.46565412495684, 83.15498791853642, 82.91335864687608, 82.60269244045564, 82.24024853296513, 81.87780462547462, 81.6361753538143, 81.37728684846392, 81.11839834311357, 80.89402830514325, 80.66965826717293, 80.47980669658267, 80.27269589230238, 80.11736278909216, 79.82395581636175, 79.53054884363134, 79.20262340352087, 78.96099413186055, 78.7193648602002, 78.44321712115982, 78.13255091473938, 77.80462547462892, 77.52847773558852, 77.25232999654816, 76.9244045564377, 76.54470141525715, 76.26855367621677, 75.92336900241628, 75.56092509492578, 75.12944425267519, 74.88781498101484, 74.43907490507421, 74.12840869865379, 73.88677942699344, 73.59337245426303, 73.35174318260269, 73.0065585088022, 72.78218847083188, 72.45426303072144, 71.98826372109079, 71.69485674836037, 71.45322747670004, 71.1425612702796, 70.83189506385916, 70.43493268898861, 70.02071108042801, 69.58923023817742, 69.31308249913704, 68.89886089057646, 68.3638246461857, 67.88056610286503, 67.31101139109424, 66.77597514670349, 66.17190196755264, 65.63686572316189, 65.17086641353124, 64.67034863652053, 64.13531239212979, 63.56575768035899, 63.03072143596824, 62.46116672419744, 61.77079737659648, 61.149464963755605, 60.47635484984466, 59.66517086641353, 58.92302381774248, 58.1463583016914, 57.30065585088022, 56.50673110113911, 55.661028650327914, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 76.6352572841175, "threshold_area100": 72.77321366931308, "threshold_area25": 84.32861580945806}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.8911396861076355, "operator": "ge"}, {"budget": 0.02, "threshold": -1.511549949645996, "operator": "ge"}, {"budget": 0.03, "threshold": -2.0179505348205566, "operator": "ge"}, {"budget": 0.04, "threshold": -2.31427264213562, "operator": "ge"}, {"budget": 0.05, "threshold": -2.83888578414917, "operator": "ge"}, {"budget": 0.06, "threshold": -3.21124529838562, "operator": "ge"}, {"budget": 0.07, "threshold": -3.5776445865631104, "operator": "ge"}, {"budget": 0.08, "threshold": -3.896111011505127, "operator": "ge"}, {"budget": 0.09, "threshold": -4.150038719177246, "operator": "ge"}, {"budget": 0.1, "threshold": -4.41634464263916, "operator": "ge"}, {"budget": 0.11, "threshold": -4.699495792388916, "operator": "ge"}, {"budget": 0.12, "threshold": -4.83713960647583, "operator": "ge"}, {"budget": 0.13, "threshold": -5.016965866088867, "operator": "ge"}, {"budget": 0.14, "threshold": -5.122925758361816, "operator": "ge"}, {"budget": 0.15, "threshold": -5.296722888946533, "operator": "ge"}, {"budget": 0.16, "threshold": -5.448220729827881, "operator": "ge"}, {"budget": 0.17, "threshold": -5.520863056182861, "operator": "ge"}, {"budget": 0.18, "threshold": -5.5769853591918945, "operator": "ge"}, {"budget": 0.19, "threshold": -5.648608207702637, "operator": "ge"}, {"budget": 0.2, "threshold": -5.732787132263184, "operator": "ge"}, {"budget": 0.21, "threshold": -5.796462535858154, "operator": "ge"}, {"budget": 0.22, "threshold": -5.881669998168945, "operator": "ge"}, {"budget": 0.23, "threshold": -5.979785919189453, "operator": "ge"}, {"budget": 0.24, "threshold": -6.044834136962891, "operator": "ge"}, {"budget": 0.25, "threshold": -6.099849700927734, "operator": "ge"}, {"budget": 0.26, "threshold": -6.183533668518066, "operator": "ge"}, {"budget": 0.27, "threshold": -6.283578395843506, "operator": "ge"}, {"budget": 0.28, "threshold": -6.310473918914795, "operator": "ge"}, {"budget": 0.29, "threshold": -6.372833251953125, "operator": "ge"}, {"budget": 0.3, "threshold": -6.4374589920043945, "operator": "ge"}, {"budget": 0.31, "threshold": -6.519920825958252, "operator": "ge"}, {"budget": 0.32, "threshold": -6.599448204040527, "operator": "ge"}, {"budget": 0.33, "threshold": -6.665811538696289, "operator": "ge"}, {"budget": 0.34, "threshold": -6.737767696380615, "operator": "ge"}, {"budget": 0.35, "threshold": -6.794696807861328, "operator": "ge"}, {"budget": 0.36, "threshold": -6.826735019683838, "operator": "ge"}, {"budget": 0.37, "threshold": -6.863951683044434, "operator": "ge"}, {"budget": 0.38, "threshold": -6.905377388000488, "operator": "ge"}, {"budget": 0.39, "threshold": -6.992341041564941, "operator": "ge"}, {"budget": 0.4, "threshold": -7.035754203796387, "operator": "ge"}, {"budget": 0.41, "threshold": -7.096192359924316, "operator": "ge"}, {"budget": 0.42, "threshold": -7.133357048034668, "operator": "ge"}, {"budget": 0.43, "threshold": -7.175678730010986, "operator": "ge"}, {"budget": 0.44, "threshold": -7.260070323944092, "operator": "ge"}, {"budget": 0.45, "threshold": -7.320313453674316, "operator": "ge"}, {"budget": 0.46, "threshold": -7.361920356750488, "operator": "ge"}, {"budget": 0.47, "threshold": -7.42275333404541, "operator": "ge"}, {"budget": 0.48, "threshold": -7.455967903137207, "operator": "ge"}, {"budget": 0.49, "threshold": -7.487677097320557, "operator": "ge"}, {"budget": 0.5, "threshold": -7.539253234863281, "operator": "ge"}, {"budget": 0.51, "threshold": -7.571504592895508, "operator": "ge"}, {"budget": 0.52, "threshold": -7.599551200866699, "operator": "ge"}, {"budget": 0.53, "threshold": -7.642168045043945, "operator": "ge"}, {"budget": 0.54, "threshold": -7.684973239898682, "operator": "ge"}, {"budget": 0.55, "threshold": -7.753224849700928, "operator": "ge"}, {"budget": 0.56, "threshold": -7.817668914794922, "operator": "ge"}, {"budget": 0.57, "threshold": -7.870401382446289, "operator": "ge"}, {"budget": 0.58, "threshold": -7.964824676513672, "operator": "ge"}, {"budget": 0.59, "threshold": -8.011554718017578, "operator": "ge"}, {"budget": 0.6, "threshold": -8.075826644897461, "operator": "ge"}, {"budget": 0.61, "threshold": -8.146276473999023, "operator": "ge"}, {"budget": 0.62, "threshold": -8.203448295593262, "operator": "ge"}, {"budget": 0.63, "threshold": -8.24166488647461, "operator": "ge"}, {"budget": 0.64, "threshold": -8.323080062866211, "operator": "ge"}, {"budget": 0.65, "threshold": -8.380809783935547, "operator": "ge"}, {"budget": 0.66, "threshold": -8.433393478393555, "operator": "ge"}, {"budget": 0.67, "threshold": -8.489143371582031, "operator": "ge"}, {"budget": 0.68, "threshold": -8.568331718444824, "operator": "ge"}, {"budget": 0.69, "threshold": -8.603025436401367, "operator": "ge"}, {"budget": 0.7, "threshold": -8.67703628540039, "operator": "ge"}, {"budget": 0.71, "threshold": -8.73808765411377, "operator": "ge"}, {"budget": 0.72, "threshold": -8.80222225189209, "operator": "ge"}, {"budget": 0.73, "threshold": -8.869219779968262, "operator": "ge"}, {"budget": 0.74, "threshold": -8.956151962280273, "operator": "ge"}, {"budget": 0.75, "threshold": -9.058378219604492, "operator": "ge"}, {"budget": 0.76, "threshold": -9.11646556854248, "operator": "ge"}, {"budget": 0.77, "threshold": -9.248649597167969, "operator": "ge"}, {"budget": 0.78, "threshold": -9.303876876831055, "operator": "ge"}, {"budget": 0.79, "threshold": -9.379652976989746, "operator": "ge"}, {"budget": 0.8, "threshold": -9.459859848022461, "operator": "ge"}, {"budget": 0.81, "threshold": -9.518678665161133, "operator": "ge"}, {"budget": 0.82, "threshold": -9.612665176391602, "operator": "ge"}, {"budget": 0.83, "threshold": -9.724653244018555, "operator": "ge"}, {"budget": 0.84, "threshold": -9.802726745605469, "operator": "ge"}, {"budget": 0.85, "threshold": -9.918378829956055, "operator": "ge"}, {"budget": 0.86, "threshold": -9.999568939208984, "operator": "ge"}, {"budget": 0.87, "threshold": -10.090066909790039, "operator": "ge"}, {"budget": 0.88, "threshold": -10.158756256103516, "operator": "ge"}, {"budget": 0.89, "threshold": -10.264230728149414, "operator": "ge"}, {"budget": 0.9, "threshold": -10.321527481079102, "operator": "ge"}, {"budget": 0.91, "threshold": -10.376509666442871, "operator": "ge"}, {"budget": 0.92, "threshold": -10.501899719238281, "operator": "ge"}, {"budget": 0.93, "threshold": -10.552915573120117, "operator": "ge"}, {"budget": 0.94, "threshold": -10.681011199951172, "operator": "ge"}, {"budget": 0.95, "threshold": -10.818552017211914, "operator": "ge"}, {"budget": 0.96, "threshold": -10.953832626342773, "operator": "ge"}, {"budget": 0.97, "threshold": -11.148759841918945, "operator": "ge"}, {"budget": 0.98, "threshold": -11.292030334472656, "operator": "ge"}, {"budget": 0.99, "threshold": -11.637075424194336, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.5, 98.0, 97.58333333333333, 97.16666666666667, 96.75, 96.25, 95.75, 95.5, 95.08333333333333, 94.75, 94.25, 93.91666666666666, 93.75, 93.41666666666666, 93.25, 92.75, 92.66666666666666, 92.16666666666666, 92.0, 91.83333333333333, 91.58333333333333, 91.16666666666666, 90.99999999999999, 90.91666666666667, 90.75, 90.5, 90.33333333333333, 90.16666666666666, 90.08333333333333, 89.91666666666667, 89.66666666666666, 89.16666666666666, 88.83333333333333, 88.58333333333333, 88.08333333333334, 87.83333333333333, 87.5, 87.33333333333333, 87.25, 86.83333333333333, 86.41666666666666, 86.33333333333333, 86.16666666666666, 85.83333333333333, 85.66666666666667, 85.25, 85.08333333333333, 84.74999999999999, 84.5, 84.08333333333333, 84.0, 83.74999999999999, 83.33333333333333, 83.0, 82.75, 82.41666666666666, 82.25, 82.0, 81.66666666666667, 81.25, 80.83333333333333, 80.25, 80.0, 79.41666666666667, 78.83333333333333, 78.33333333333333, 77.75, 77.41666666666667, 77.0, 76.5, 75.91666666666667, 75.25, 74.75, 74.16666666666667, 73.41666666666666, 72.75, 72.24999999999999, 71.83333333333333, 71.16666666666667, 70.41666666666666, 69.75, 69.33333333333334, 68.5, 67.58333333333333, 67.16666666666666, 66.25, 65.58333333333333, 64.91666666666667, 64.08333333333334, 63.5, 62.58333333333332, 61.83333333333333, 61.083333333333336, 60.16666666666666, 59.25, 58.58333333333333, 57.833333333333336, 56.833333333333336, 55.833333333333336, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 5, 11, 16, 21, 26, 32, 39, 42, 47, 51, 57, 61, 63, 67, 69, 75, 76, 82, 84, 86, 90, 95, 97, 98, 100, 103, 105, 107, 108, 110, 113, 119, 123, 126, 132, 135, 139, 141, 142, 147, 152, 153, 155, 159, 161, 166, 168, 172, 175, 180, 181, 184, 189, 193, 196, 200, 202, 205, 209, 214, 219, 226, 229, 236, 243, 249, 256, 260, 265, 271, 278, 286, 292, 299, 308, 316, 322, 327, 335, 344, 352, 357, 367, 378, 383, 394, 402, 410, 420, 427, 438, 447, 456, 467, 478, 486, 495, 507, 519, 528], "exact_score": [98.91666666666666, 98.5, 98.0, 97.58333333333333, 97.16666666666667, 96.75, 96.25, 95.75, 95.5, 95.08333333333333, 94.75, 94.25, 93.91666666666666, 93.75, 93.41666666666666, 93.25, 92.75, 92.66666666666666, 92.16666666666666, 92.0, 91.83333333333333, 91.58333333333333, 91.16666666666666, 90.99999999999999, 90.91666666666667, 90.75, 90.5, 90.33333333333333, 90.16666666666666, 90.08333333333333, 89.91666666666667, 89.66666666666666, 89.16666666666666, 88.83333333333333, 88.58333333333333, 88.08333333333334, 87.83333333333333, 87.5, 87.33333333333333, 87.25, 86.83333333333333, 86.41666666666666, 86.33333333333333, 86.16666666666666, 85.83333333333333, 85.66666666666667, 85.25, 85.08333333333333, 84.74999999999999, 84.5, 84.08333333333333, 84.0, 83.74999999999999, 83.33333333333333, 83.0, 82.75, 82.41666666666666, 82.25, 82.0, 81.66666666666667, 81.25, 80.83333333333333, 80.25, 80.0, 79.41666666666667, 78.83333333333333, 78.33333333333333, 77.75, 77.41666666666667, 77.0, 76.5, 75.91666666666667, 75.25, 74.75, 74.16666666666667, 73.41666666666666, 72.75, 72.24999999999999, 71.83333333333333, 71.16666666666667, 70.41666666666666, 69.75, 69.24999999999999, 68.5, 67.58333333333333, 67.16666666666666, 66.25, 65.58333333333333, 64.91666666666667, 64.08333333333334, 63.5, 62.58333333333332, 61.83333333333333, 61.083333333333336, 60.16666666666666, 59.25, 58.58333333333333, 57.833333333333336, 56.833333333333336, 55.833333333333336, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 81.59166666666668, "threshold_area100": 81.59250000000002, "threshold_area25": 94.28333333333333}, "test": {"score": [90.16223679668623, 89.7134967207456, 88.86779426993441, 88.05661028650327, 87.29720400414222, 86.19261304798067, 85.38142906454954, 84.69105971694857, 83.9316534345875, 83.17224715222645, 82.30928546772522, 81.48084225060407, 81.08387987573352, 80.46254746289264, 80.13462202278218, 79.59958577839143, 79.18536416983085, 79.09906800138073, 78.90921643079048, 78.65032792544011, 78.39143942008975, 78.20158784949948, 77.92544011045909, 77.63203313772868, 77.44218156713842, 77.2178115291681, 76.9416637901277, 76.75181221953746, 76.66551605108734, 76.42388677942698, 76.25129444252676, 75.88885053503624, 75.66448049706594, 75.42285122540558, 75.19848118743528, 74.92233344839488, 74.83603727994476, 74.64618570935451, 74.52537107352433, 74.0075940628236, 73.85226095961339, 73.66240938902313, 73.5070762858129, 73.36900241629272, 72.97204004142216, 72.62685536762167, 72.45426303072144, 72.26441146013116, 72.05730065585088, 71.93648602002071, 71.78115291681048, 71.66033828098031, 71.5050051777701, 71.34967207455989, 71.21159820503969, 70.90093199861926, 70.6420434932689, 70.31411805315844, 69.91715567828788, 69.64100793924749, 69.22678633068692, 68.95063859164652, 68.69175008629617, 68.48463928201586, 68.00138073869519, 67.75975146703486, 67.60441836382465, 67.41456679323437, 67.17293752157404, 67.01760441836382, 66.68967897825337, 66.44804970659303, 66.10286503279255, 65.74042112530203, 65.41249568519157, 64.94649637556091, 64.60131170176044, 64.01449775629962, 63.7210907835692, 63.44494304452881, 63.15153607179841, 62.909906800138074, 62.58198136002762, 62.150500517777004, 61.90887124611667, 61.3565757680359, 60.99413186054539, 60.49361408353469, 60.2174663444943, 59.958577839143935, 59.734207801173625, 59.44080082844321, 58.957542285122535, 58.802209181912325, 58.19813600276147, 57.628581290990674, 57.197100448740066, 56.71384190541939, 56.437694166379, 55.5056955471177, 54.81532619951675], "calls": [0.0, 4.263030721435968, 7.9910251984811875, 11.56368657231619, 13.945460821539523, 17.91508457024508, 20.60752502588885, 22.92026234035209, 25.215740421125304, 27.304107697618225, 29.39247497411115, 31.567138419054192, 32.46461857093545, 34.22506040731791, 35.10528132550915, 36.77942699344149, 37.90127718329306, 38.66068346565413, 39.074905074214705, 39.90334829133587, 41.02519848118744, 41.68104936140835, 42.85467725233, 44.14911977908181, 44.83948912668277, 45.564376941663795, 46.70348636520538, 47.894373489817056, 48.274076630997584, 49.22333448394891, 50.22437003797031, 51.77770107007249, 53.072143596824304, 54.10769761822575, 55.350362443907485, 56.10976872626855, 56.523990334829136, 57.093545046599935, 57.55954435623059, 58.99206075250258, 59.716948567483605, 60.59716948567484, 61.114946496375566, 61.68450120814636, 63.03072143596824, 64.13531239212979, 64.84294097342078, 65.4987918536417, 66.05108733172247, 66.44804970659303, 67.0693821194339, 67.6389368312047, 68.19123230928547, 68.84708318950639, 69.5029340697273, 70.52122885743873, 71.52226441146013, 72.36796686227132, 73.66240938902313, 74.43907490507421, 75.47462892647566, 76.19951674145668, 76.99344149119779, 77.54573696927856, 78.61580945806006, 79.40973420780118, 79.97928891957197, 80.53158439765275, 81.25647221263375, 81.61891612012427, 82.32654470141526, 82.77528477735588, 83.46565412495686, 84.10424577148775, 84.86365205384881, 85.72661373835003, 86.31342768381084, 87.45253710735244, 87.88401794960303, 88.38453572661373, 88.88505350362443, 89.40283051432516, 89.95512599240594, 90.66275457369693, 91.1460131170176, 91.88816016568865, 92.40593717638936, 93.02726958923024, 93.33793579565068, 93.76941663790129, 94.14911977908181, 94.56334138764238, 95.15015533310321, 95.30548843631342, 95.97859855022438, 96.66896789782534, 97.11770797376596, 97.84259578874698, 98.2222989299275, 99.22333448394892, 100.0], "rescued": [0, 25, 47, 63, 71, 88, 97, 105, 111, 119, 121, 124, 125, 126, 127, 128, 128, 129, 129, 130, 132, 132, 132, 132, 132, 132, 132, 132, 132, 132, 133, 134, 135, 135, 135, 135, 135, 135, 135, 135, 135, 135, 135, 136, 137, 137, 137, 137, 137, 137, 137, 137, 137, 137, 137, 137, 138, 138, 138, 138, 139, 139, 139, 139, 139, 140, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 142, 142, 142, 142, 142, 142, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 51, 122, 185, 237, 318, 374, 422, 472, 524, 576, 627, 651, 688, 708, 740, 764, 770, 781, 797, 814, 825, 841, 858, 869, 882, 898, 909, 914, 928, 939, 961, 975, 989, 1002, 1018, 1023, 1034, 1041, 1071, 1080, 1091, 1100, 1109, 1133, 1153, 1163, 1174, 1186, 1193, 1202, 1209, 1218, 1227, 1235, 1253, 1269, 1288, 1311, 1327, 1352, 1368, 1383, 1395, 1423, 1438, 1448, 1459, 1473, 1482, 1501, 1515, 1535, 1556, 1575, 1602, 1622, 1657, 1674, 1690, 1707, 1721, 1740, 1767, 1781, 1813, 1834, 1863, 1879, 1894, 1907, 1924, 1952, 1961, 1996, 2029, 2054, 2082, 2098, 2152, 2192], "exact_score": [90.16223679668623, 90.07594062823611, 90.02416292716603, 89.90334829133586, 89.74801518812563, 89.62720055229548, 89.3683120469451, 89.2129789437349, 88.86779426993441, 88.72972040041421, 88.38453572661373, 88.16016568864342, 87.86675871591301, 87.62512944425266, 87.29720400414222, 86.91750086296169, 86.72764929237141, 86.48602002071108, 86.15809458060062, 85.86468760787021, 85.55402140144977, 85.24335519502934, 84.98446668967897, 84.65654124956852, 84.27683810838799, 84.0006903693476, 83.72454263030721, 83.3620987228167, 82.89609941318605, 82.4991370383155, 82.05039696237488, 81.68795305488436, 81.2909906800138, 80.94580600621332, 80.60062133241284, 80.16914049016223, 79.82395581636175, 79.51328960994132, 79.18536416983085, 78.92647566448049, 78.61580945806006, 78.39143942008975, 78.14981014842941, 77.90818087676908, 77.68381083879875, 77.37314463237833, 77.0797376596479, 76.87262685536763, 76.7172937521574, 76.51018294787711, 76.2858129099068, 76.02692440455642, 75.83707283396618, 75.68173973075595, 75.45736969278563, 75.2847773558854, 74.95685191577493, 74.7152226441146, 74.42181567138418, 74.0075940628236, 73.76596479116327, 73.54159475319295, 73.24818778046254, 72.98929927511219, 72.66137383500173, 72.40248532965136, 72.05730065585088, 71.83293061788056, 71.59130134622022, 71.29789437348981, 71.03900586813945, 70.81463583016914, 70.43493268898861, 70.12426648256817, 69.81360027614774, 69.43389713496721, 69.01967552640662, 68.69175008629617, 68.2257507766655, 67.91508457024507, 67.60441836382465, 67.25923369002416, 66.89678978253365, 66.30997583707283, 65.80945806006213, 65.36071798412151, 64.82568173973075, 64.23886779426992, 63.66931308249914, 63.099758370728345, 62.59924059371764, 61.99516741456679, 61.27027959958578, 60.510873317224714, 59.82050396962375, 59.078356920952714, 58.180876769071446, 57.317915084570245, 56.59302726958922, 55.712806351397994, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.91678582905507, "threshold_area100": 71.39074905074213, "threshold_area25": 82.03658957542285}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 0.535036244390749}}, "old_normalized_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 0.535036244390749}}, "old_normalized_endpoints": {"val": {"score": 98.91666666666666, "calls": 0.0}, "test": {"score": 90.16223679668623, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.5, 98.33333333333333, 98.08333333333333, 97.91666666666666, 97.74999999999999, 97.66666666666667, 97.5, 97.41666666666666, 97.24999999999999, 97.16666666666667, 97.0, 96.83333333333333, 96.58333333333333, 96.41666666666666, 96.25, 96.25, 96.25, 96.08333333333333, 95.83333333333333, 95.75, 95.5, 95.25, 94.75, 94.33333333333334, 94.0, 93.33333333333333, 93.25, 92.91666666666667, 92.66666666666666, 92.25, 91.83333333333333, 91.33333333333333, 90.75, 90.25, 89.91666666666667, 89.25, 88.83333333333333, 88.25, 87.91666666666667, 87.33333333333333, 86.83333333333333, 86.16666666666666, 85.33333333333333, 85.08333333333333, 84.41666666666666, 84.08333333333333, 83.66666666666667, 83.0, 82.33333333333333, 81.58333333333333, 80.75, 80.08333333333333, 79.08333333333333, 78.33333333333333, 77.58333333333333, 76.83333333333333, 76.0, 75.25, 74.41666666666666, 73.66666666666666, 72.75, 72.0, 71.24999999999999, 70.33333333333334, 69.58333333333333, 68.75, 68.0, 67.16666666666666, 66.25, 65.33333333333333, 64.33333333333333, 63.5, 62.5, 61.74999999999999, 60.916666666666664, 59.916666666666664, 59.0, 57.99999999999999, 56.99999999999999, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.75, 98.75, 98.75, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.5, 98.33333333333333, 98.08333333333333, 97.91666666666666, 97.74999999999999, 97.66666666666667, 97.5, 97.41666666666666, 97.24999999999999, 97.16666666666667, 97.0, 96.83333333333333, 96.58333333333333, 96.41666666666666, 96.25, 96.25, 96.25, 96.08333333333333, 95.83333333333333, 95.75, 95.5, 95.25, 94.75, 94.33333333333334, 94.0, 93.33333333333333, 93.25, 92.91666666666667, 92.66666666666666, 92.25, 91.83333333333333, 91.33333333333333, 90.75, 90.25, 89.91666666666667, 89.25, 88.83333333333333, 88.16666666666666, 87.91666666666667, 87.33333333333333, 86.83333333333333, 86.16666666666666, 85.33333333333333, 85.08333333333333, 84.41666666666666, 84.08333333333333, 83.66666666666667, 83.0, 82.33333333333333, 81.5, 80.75, 80.08333333333333, 79.08333333333333, 78.33333333333333, 77.58333333333333, 76.83333333333333, 76.0, 75.25, 74.41666666666666, 73.66666666666666, 72.75, 72.0, 71.16666666666667, 70.33333333333334, 69.58333333333333, 68.75, 68.0, 67.16666666666666, 66.25, 65.33333333333333, 64.33333333333333, 63.5, 62.5, 61.74999999999999, 60.916666666666664, 59.916666666666664, 59.0, 57.99999999999999, 56.99999999999999, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.625, "exact_area100": 86.37166666666667}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.04142216085606, 90.006903693476, 89.95512599240594, 89.92060752502589, 89.81705212288574, 89.78253365550569, 89.74801518812563, 89.73075595443561, 89.67897825336554, 89.5063859164653, 89.47186744908525, 89.35105281325508, 89.28201587849499, 89.19571971004487, 89.14394200897479, 88.97134967207455, 88.90231273731446, 88.79875733517432, 88.64342423196409, 88.53986882982396, 88.45357266137384, 88.26372109078356, 88.24646185709354, 88.03935105281326, 87.90127718329306, 87.78046254746289, 87.59061097687263, 87.48705557473248, 87.27994477045219, 86.98653779772178, 86.7103900586814, 86.45150155333103, 86.19261304798067, 86.02002071108042, 85.79565067311012, 85.67483603727995, 85.12254055919917, 84.94994822229893, 84.76009665170866, 84.58750431480841, 84.25957887469797, 84.19054193993787, 83.91439420089748, 83.44839489126683, 83.20676561960649, 82.80980324473592, 82.4991370383155, 82.05039696237488, 81.91232309285468, 81.65343458750431, 81.32550914739384, 80.8595098377632, 80.58336209872282, 80.20365895754227, 79.77217811529168, 79.4269934414912, 78.82292026234035, 78.20158784949948, 77.66655160510874, 77.20055229547808, 76.70003451846738, 75.73351743182603, 75.06040731791508, 74.30100103555401, 73.74870555747324, 73.26544701415256, 72.71315153607179, 72.21263375906109, 71.47048671039006, 71.21159820503969, 70.81463583016914, 70.0724887814981, 69.58923023817742, 68.81256472212634, 68.27752847773559, 67.4318260269244, 66.63790127718329, 65.80945806006213, 65.13634794615119, 64.29064549534, 63.49672074559889, 62.978943734898166, 62.2885743872972, 61.77079737659648, 60.92509492578528, 60.26924404556438, 59.26820849154297, 58.387987573351744, 57.45598895409043, 56.627545736969275, 56.42043493268898, 55.5056955471177, 54.81532619951675], "calls": [0.0, 0.535036244390749, 2.312737314463238, 3.0721435968243007, 3.727994477045219, 4.608215395236451, 5.505695547117708, 6.057991025198481, 7.179841215050052, 8.025543665861235, 8.7849499482223, 9.820503969623749, 11.218501898515706, 11.891612012426648, 12.77183293061788, 13.841905419399378, 14.60131170176044, 16.62064204349327, 17.67345529858474, 18.847083189506385, 19.675526406627547, 20.711080428028996, 21.246116672419745, 22.22989299275112, 22.88574387297204, 23.696927856403175, 24.628926475664482, 25.716258198136003, 26.52744218156714, 27.959958577839146, 28.512254055919918, 30.151881256472212, 30.9975837072834, 31.739730755954437, 32.75802554366586, 33.53469105971695, 34.86365205384881, 36.1408353469106, 37.297204004142216, 38.90231273731447, 40.04142216085606, 40.95616154642734, 41.93993786675871, 43.04452882292026, 45.65067311011391, 47.083189506385914, 48.2913358646876, 49.08526061442872, 50.03451846738005, 50.77666551605109, 51.881256472212634, 53.17569899896445, 53.93510528132551, 55.16051087331723, 56.05799102519848, 57.23161891612012, 57.801173627890925, 58.83672764929238, 60.217466344494305, 61.44287193648602, 62.37487055574732, 63.34138764238868, 64.63583016914049, 65.5678287884018, 67.08664135312392, 68.22575077666552, 69.22678633068692, 69.98619261304798, 70.84915429754919, 72.33344839489126, 73.45529858474283, 74.5598895409044, 75.45736969278563, 76.14773904038661, 76.9244045564377, 78.11529168104936, 79.13358646876078, 79.61684501208146, 80.11736278909216, 81.11839834311357, 81.7915084570245, 82.861580945806, 83.77632033137729, 85.0707628581291, 86.0717984121505, 87.09009319986193, 88.10838798757335, 89.23023817742492, 90.16223679668623, 90.80082844321711, 91.71556782878841, 92.42319641007938, 93.30341732827063, 94.06282361063168, 95.20193303417328, 96.2202278218847, 97.2212633759061, 98.0669658267173, 98.27407663099758, 99.25785295132896, 100.0], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.04142216085606, 90.006903693476, 89.95512599240594, 89.92060752502589, 89.83431135657577, 89.78253365550569, 89.74801518812563, 89.73075595443561, 89.67897825336554, 89.57542285122541, 89.47186744908525, 89.45460821539524, 89.35105281325508, 89.28201587849499, 89.17846047635484, 89.0058681394546, 88.90231273731446, 88.7124611667242, 88.62616499827408, 88.53986882982396, 88.35001725923368, 88.26372109078356, 88.2119433897135, 88.0738695201933, 87.90127718329306, 87.74594408008284, 87.5733517431826, 87.45253710735244, 87.27994477045219, 87.02105626510182, 86.79668622713152, 86.58957542285121, 86.46876078702105, 86.19261304798067, 86.0027614773904, 85.79565067311012, 85.69209527096997, 85.45046599930963, 85.24335519502934, 85.08802209181911, 84.9326889886089, 84.81187435277874, 84.62202278218847, 84.25957887469797, 84.12150500517777, 83.8626164998274, 83.53469105971695, 83.17224715222645, 82.80980324473592, 82.51639627200552, 82.15395236451502, 81.87780462547462, 81.56713841905419, 81.3600276147739, 81.04936140835346, 80.68691750086295, 80.39351052813255, 79.996548153262, 79.59958577839143, 79.27166033828098, 78.87469796341043, 78.3569209527097, 77.80462547462892, 77.20055229547808, 76.61373835001726, 75.99240593717639, 75.30203658957542, 74.69796341042458, 73.95581636175353, 73.35174318260269, 72.71315153607179, 72.29892992751121, 71.59130134622022, 70.88367276492924, 70.14152571625819, 69.43389713496721, 68.69175008629617, 68.2084915429755, 67.50086296168449, 66.72419744563341, 65.87849499482222, 65.23990334829134, 64.49775629962029, 63.66931308249914, 62.87538833275802, 62.08146358301691, 61.235761132205724, 60.33828098032448, 59.458060062133235, 58.577839143942015, 57.66309975837072, 56.69658267172937, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 89.7117707973766, "exact_area100": 80.39842854587887}}, "old_normalized_endpoint_budget": 0}, {"seed": 2029, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.0, "validation_area100": 0.865525, "training_seconds": 2.942180300015025, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.009776376187801361, "operator": "ge"}, {"budget": 0.02, "threshold": -0.011690952815115452, "operator": "ge"}, {"budget": 0.03, "threshold": -0.013366861268877983, "operator": "ge"}, {"budget": 0.04, "threshold": -0.01477673277258873, "operator": "ge"}, {"budget": 0.05, "threshold": -0.016528872773051262, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01886169984936714, "operator": "ge"}, {"budget": 0.07, "threshold": -0.020666426047682762, "operator": "ge"}, {"budget": 0.08, "threshold": -0.02305743470788002, "operator": "ge"}, {"budget": 0.09, "threshold": -0.02464599348604679, "operator": "ge"}, {"budget": 0.1, "threshold": -0.028449691832065582, "operator": "ge"}, {"budget": 0.11, "threshold": -0.03411589562892914, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03948720544576645, "operator": "ge"}, {"budget": 0.13, "threshold": -0.046315375715494156, "operator": "ge"}, {"budget": 0.14, "threshold": -0.04898626357316971, "operator": "ge"}, {"budget": 0.15, "threshold": -0.05473042652010918, "operator": "ge"}, {"budget": 0.16, "threshold": -0.05918615683913231, "operator": "ge"}, {"budget": 0.17, "threshold": -0.062189988791942596, "operator": "ge"}, {"budget": 0.18, "threshold": -0.06628260016441345, "operator": "ge"}, {"budget": 0.19, "threshold": -0.07246007025241852, "operator": "ge"}, {"budget": 0.2, "threshold": -0.07691557705402374, "operator": "ge"}, {"budget": 0.21, "threshold": -0.08545950055122375, "operator": "ge"}, {"budget": 0.22, "threshold": -0.09201904386281967, "operator": "ge"}, {"budget": 0.23, "threshold": -0.09787484258413315, "operator": "ge"}, {"budget": 0.24, "threshold": -0.10770110785961151, "operator": "ge"}, {"budget": 0.25, "threshold": -0.11223643273115158, "operator": "ge"}, {"budget": 0.26, "threshold": -0.11782287806272507, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1222425326704979, "operator": "ge"}, {"budget": 0.28, "threshold": -0.13205227255821228, "operator": "ge"}, {"budget": 0.29, "threshold": -0.1413000524044037, "operator": "ge"}, {"budget": 0.3, "threshold": -0.14664269983768463, "operator": "ge"}, {"budget": 0.31, "threshold": -0.15342111885547638, "operator": "gt"}, {"budget": 0.32, "threshold": -0.16459067165851593, "operator": "ge"}, {"budget": 0.33, "threshold": -0.16995729506015778, "operator": "ge"}, {"budget": 0.34, "threshold": -0.17908665537834167, "operator": "ge"}, {"budget": 0.35, "threshold": -0.19213815033435822, "operator": "ge"}, {"budget": 0.36, "threshold": -0.19907325506210327, "operator": "ge"}, {"budget": 0.37, "threshold": -0.20983611047267914, "operator": "ge"}, {"budget": 0.38, "threshold": -0.22507920861244202, "operator": "ge"}, {"budget": 0.39, "threshold": -0.2365240603685379, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2495710849761963, "operator": "ge"}, {"budget": 0.41, "threshold": -0.26382505893707275, "operator": "ge"}, {"budget": 0.42, "threshold": -0.27656635642051697, "operator": "ge"}, {"budget": 0.43, "threshold": -0.28587889671325684, "operator": "ge"}, {"budget": 0.44, "threshold": -0.3007773458957672, "operator": "ge"}, {"budget": 0.45, "threshold": -0.31401389837265015, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3234120309352875, "operator": "ge"}, {"budget": 0.47, "threshold": -0.32799971103668213, "operator": "ge"}, {"budget": 0.48, "threshold": -0.34790337085723877, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3658129572868347, "operator": "ge"}, {"budget": 0.5, "threshold": -0.38166481256484985, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3961901068687439, "operator": "ge"}, {"budget": 0.52, "threshold": -0.409551739692688, "operator": "ge"}, {"budget": 0.53, "threshold": -0.4242469072341919, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4388805627822876, "operator": "ge"}, {"budget": 0.55, "threshold": -0.4606001079082489, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4806416928768158, "operator": "ge"}, {"budget": 0.57, "threshold": -0.4929235875606537, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5063270330429077, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5204728245735168, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5375867486000061, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5602644085884094, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5815415382385254, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5976999998092651, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6164213418960571, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6248590350151062, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6474728584289551, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6658848524093628, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6837738752365112, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6996227502822876, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7151026129722595, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7269418835639954, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7508158683776855, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7652071118354797, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7813895344734192, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7957955598831177, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8110883831977844, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8243623971939087, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8332112431526184, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8452378511428833, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8587900400161743, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8675702214241028, "operator": "ge"}, {"budget": 0.82, "threshold": -0.87538081407547, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8884798288345337, "operator": "ge"}, {"budget": 0.84, "threshold": -0.893918514251709, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8980094790458679, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9058427810668945, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9123623967170715, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9194898009300232, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9299577474594116, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9473148584365845, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9552555084228516, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9617963433265686, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9649685025215149, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9687935709953308, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9719402194023132, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9750257730484009, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9799032807350159, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9848155379295349, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9880006909370422, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.66666666666667, 97.58333333333333, 97.58333333333333, 97.5, 97.41666666666666, 97.16666666666667, 96.83333333333333, 96.75, 96.58333333333333, 96.25, 96.0, 95.75, 95.66666666666667, 95.58333333333333, 95.5, 95.25, 95.08333333333333, 94.83333333333333, 94.58333333333333, 94.16666666666667, 93.91666666666666, 93.41666666666666, 93.16666666666666, 92.75, 92.25, 91.83333333333333, 91.49999999999999, 91.25, 90.75, 90.08333333333333, 89.5, 88.83333333333333, 88.08333333333334, 87.58333333333333, 87.25, 86.5, 85.75, 85.16666666666666, 84.58333333333333, 84.0, 83.33333333333333, 82.66666666666667, 82.0, 81.41666666666667, 80.91666666666666, 80.33333333333333, 79.5, 78.75, 78.08333333333333, 77.16666666666666, 76.25, 75.33333333333333, 74.75, 74.08333333333333, 73.25, 72.24999999999999, 71.5, 70.83333333333333, 70.0, 69.16666666666667, 68.33333333333333, 67.33333333333333, 66.5, 65.58333333333333, 64.58333333333333, 63.66666666666667, 62.74999999999999, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.99999999999999, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 30.916666666666664, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 3, 3, 3, 5, 7, 9, 10, 11, 12, 13, 14, 15, 16, 16, 17, 18, 21, 25, 26, 28, 32, 35, 38, 39, 40, 41, 44, 46, 49, 52, 57, 60, 66, 69, 74, 80, 85, 89, 92, 98, 106, 113, 121, 130, 136, 140, 149, 159, 166, 173, 180, 188, 196, 204, 211, 217, 224, 234, 243, 251, 262, 273, 284, 291, 299, 309, 321, 330, 338, 349, 359, 369, 381, 391, 402, 414, 425, 436, 448, 460, 472, 483, 494, 505, 517, 528], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.66666666666667, 97.58333333333333, 97.58333333333333, 97.5, 97.41666666666666, 97.16666666666667, 96.83333333333333, 96.75, 96.58333333333333, 96.25, 96.0, 95.75, 95.66666666666667, 95.58333333333333, 95.5, 95.25, 95.08333333333333, 94.83333333333333, 94.58333333333333, 94.16666666666667, 93.91666666666666, 93.41666666666666, 93.16666666666666, 92.75, 92.25, 91.83333333333333, 91.49999999999999, 91.25, 90.75, 90.08333333333333, 89.5, 88.75, 88.08333333333334, 87.58333333333333, 87.25, 86.5, 85.75, 85.16666666666666, 84.58333333333333, 84.0, 83.33333333333333, 82.66666666666667, 82.0, 81.41666666666667, 80.91666666666666, 80.33333333333333, 79.5, 78.75, 78.08333333333333, 77.16666666666666, 76.25, 75.33333333333333, 74.75, 74.08333333333333, 73.25, 72.24999999999999, 71.41666666666666, 70.83333333333333, 70.0, 69.16666666666667, 68.33333333333333, 67.33333333333333, 66.5, 65.58333333333333, 64.58333333333333, 63.66666666666667, 62.74999999999999, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.99999999999999, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.5525, "threshold_area100": 86.55416666666667, "threshold_area25": 98.56666666666666}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.09319986192614, 90.09319986192614, 90.09319986192614, 90.02416292716603, 89.97238522609597, 89.93786675871591, 89.90334829133586, 89.8688298239558, 89.7134967207456, 89.66171901967553, 89.66171901967553, 89.66171901967553, 89.54090438384536, 89.47186744908525, 89.26475664480496, 89.17846047635484, 89.12668277528478, 88.95409043838454, 88.91957197100449, 88.83327580255437, 88.74697963410424, 88.53986882982396, 88.43631342768381, 88.40179496030376, 88.33275802554367, 88.09112875388332, 88.02209181912323, 87.91853641698309, 87.72868484639281, 87.65964791163272, 87.5043148084225, 87.22816706938211, 87.12461166724196, 86.86572316189161, 86.74490852606144, 86.55505695547117, 86.33068691750086, 86.15809458060062, 85.84742837418018, 85.65757680358992, 85.53676216775975, 85.19157749395927, 84.74283741801864, 84.58750431480841, 84.39765274421815, 84.12150500517777, 83.87987573351742, 83.67276492923715, 83.32758025543666, 82.96513634794614, 82.80980324473592, 82.67172937521575, 82.41284086986538, 81.9986192613048, 81.51536071798412, 81.04936140835346, 80.58336209872282, 80.04832585433206, 79.77217811529168, 79.06454953400069, 78.54677252329995, 78.11529168104936, 77.75284777355886, 77.20055229547808, 76.7863306869175, 75.81981360027615, 75.42285122540558, 74.64618570935451, 74.04211253020365, 73.12737314463237, 72.45426303072144, 71.97100448740076, 71.40144977562996, 70.40041422160856, 69.72730410769762, 69.22678633068692, 68.01863997238522, 67.58715913013462, 67.38004832585432, 66.98308595098376, 66.430790472903, 65.63686572316189, 64.65308940283052, 62.73731446323782, 61.649982740766305, 60.683465654124944, 60.234725578184324, 59.42354159475318, 58.99206075250258, 58.40524680704176, 57.404211253020364, 56.627545736969275, 55.78184328615809, 54.81532619951675], "calls": [0.0, 0.7248878149810148, 1.6568864342423197, 2.744218156713842, 3.6071798412150504, 4.504659993096307, 5.453917846047635, 6.109768726268554, 6.920952709699689, 7.455988954090438, 8.646876078702105, 9.90680013807387, 10.959613393165343, 12.340352088367275, 12.806351397997929, 13.979979288919573, 14.791163272350708, 15.412495685191576, 16.189161201242666, 17.397307559544355, 18.139454608215395, 19.81360027614774, 20.952709699689333, 21.660338280980323, 22.833966171901967, 23.26544701415257, 23.955816361753538, 24.577148774594406, 26.09596133931654, 26.97618225750777, 27.82188470831895, 28.633068691750086, 30.117362789092166, 30.807732136693133, 32.067656196064895, 33.39661719019675, 34.173282706247846, 35.0707628581291, 36.382464618570935, 37.40075940628236, 38.40179496030377, 39.52364515015533, 40.66275457369693, 41.42216085605799, 42.267863306869174, 43.26889886089057, 44.04556437694166, 44.40800828443217, 45.96133931653434, 47.566448049706594, 48.619261304798066, 49.464963755609254, 50.20711080428028, 51.156368657231624, 52.24370037970314, 53.95236451501554, 55.33310321021747, 56.19606489471868, 56.972730410769756, 58.025543665861235, 59.14739385571281, 60.57991025198481, 61.96064894718675, 62.96168450120815, 64.27338626164997, 64.82568173973075, 66.2064204349327, 67.27649292371419, 68.22575077666552, 68.89886089057646, 69.93441491197791, 70.69382119433897, 72.36796686227132, 73.21366931308249, 74.45633413876423, 75.61270279599586, 76.90714532274767, 77.78736624093891, 78.66758715913014, 79.56506731101139, 80.91128753883328, 81.91232309285468, 82.61995167414567, 84.24231964100794, 84.81187435277874, 85.0707628581291, 85.70935450466, 86.46876078702105, 87.46979634104247, 88.7124611667242, 91.26682775284777, 92.56127027959958, 93.68312046945115, 94.16637901277183, 95.02934069727304, 95.56437694166378, 96.23748705557473, 97.25578184328616, 98.10148429409735, 98.96444597859855, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 5, 5, 5, 6, 6, 8, 8, 9, 10, 11, 11, 14, 14, 14, 15, 16, 17, 18, 20, 20, 25, 28, 30, 33, 38, 44, 51, 55, 59, 66, 66, 69, 74, 76, 80, 81, 85, 88, 91, 94, 97, 97, 100, 103, 107, 111, 113, 115, 118, 120, 121, 125, 127, 129, 130, 130, 131, 132, 134, 136, 142, 143, 143, 143, 143, 144, 144, 144, 144, 144, 144], "harmed": [0, 0, 0, 1, 2, 2, 2, 4, 4, 4, 8, 11, 13, 15, 17, 26, 29, 29, 30, 37, 41, 53, 58, 61, 71, 73, 78, 83, 96, 103, 107, 111, 125, 130, 136, 149, 153, 163, 180, 187, 202, 212, 223, 236, 247, 266, 278, 286, 308, 334, 348, 362, 380, 397, 414, 440, 468, 481, 493, 515, 539, 570, 602, 631, 666, 683, 728, 761, 789, 813, 848, 872, 931, 957, 1006, 1045, 1100, 1141, 1172, 1207, 1266, 1309, 1340, 1412, 1438, 1450, 1474, 1507, 1555, 1614, 1731, 1795, 1851, 1877, 1924, 1950, 1984, 2042, 2087, 2136, 2192], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.07594062823611, 89.98964445978598, 89.97238522609597, 89.93786675871591, 89.92060752502589, 89.83431135657577, 89.7134967207456, 89.66171901967553, 89.66171901967553, 89.59268208491544, 89.5063859164653, 89.3683120469451, 89.23023817742491, 89.17846047635484, 89.09216430790472, 88.93683120469451, 88.79875733517432, 88.7124611667242, 88.53986882982396, 88.43631342768381, 88.40179496030376, 88.24646185709354, 88.16016568864342, 88.02209181912323, 87.97031411805315, 87.79772178115292, 87.69416637901277, 87.53883327580255, 87.33172247152227, 87.19364860200207, 86.95201933034173, 86.76216775975146, 86.67587159130134, 86.41698308595099, 86.20987228167068, 85.91646530894027, 85.65757680358992, 85.51950293406972, 85.17431826026925, 84.86365205384881, 84.70831895063858, 84.51846738004832, 84.17328270624785, 83.9316534345875, 83.74180186399724, 83.48291335864687, 83.31032102174663, 83.12046945115637, 82.7925440110459, 82.68898860890576, 82.41284086986538, 82.08491542975491, 81.68795305488436, 81.44632378322402, 81.01484294097342, 80.54884363134276, 80.15188125647221, 79.70314118053157, 79.18536416983085, 78.70210562651017, 78.16706938211944, 77.70107007248878, 77.16603382809804, 76.63099758370728, 75.99240593717639, 75.45736969278563, 74.97411114946496, 74.37003797031412, 73.81774249223334, 73.04107697618225, 72.31618916120124, 71.69485674836037, 71.0735243355195, 70.33137728684846, 69.65826717293751, 68.95063859164652, 68.2084915429755, 67.46634449430445, 66.77597514670349, 65.98205039696236, 65.22264411460131, 64.44597859855021, 63.80738695201933, 62.96168450120814, 62.150500517777004, 61.2875388332758, 60.39005868139454, 59.44080082844321, 58.62961684501208, 57.66309975837072, 56.73110113910942, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.51449644562257, "threshold_area100": 80.52830514325163, "threshold_area25": 89.7756299620297}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.009776376187801361, "operator": "ge"}, {"budget": 0.02, "threshold": -0.011690952815115452, "operator": "ge"}, {"budget": 0.03, "threshold": -0.013366861268877983, "operator": "ge"}, {"budget": 0.04, "threshold": -0.01477673277258873, "operator": "ge"}, {"budget": 0.05, "threshold": -0.016528872773051262, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01886169984936714, "operator": "ge"}, {"budget": 0.07, "threshold": -0.020666426047682762, "operator": "ge"}, {"budget": 0.08, "threshold": -0.02305743470788002, "operator": "ge"}, {"budget": 0.09, "threshold": -0.02464599348604679, "operator": "ge"}, {"budget": 0.1, "threshold": -0.028449691832065582, "operator": "ge"}, {"budget": 0.11, "threshold": -0.03411589562892914, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03948720544576645, "operator": "ge"}, {"budget": 0.13, "threshold": -0.046315375715494156, "operator": "ge"}, {"budget": 0.14, "threshold": -0.04898626357316971, "operator": "ge"}, {"budget": 0.15, "threshold": -0.05473042652010918, "operator": "ge"}, {"budget": 0.16, "threshold": -0.05918615683913231, "operator": "ge"}, {"budget": 0.17, "threshold": -0.062189988791942596, "operator": "ge"}, {"budget": 0.18, "threshold": -0.06628260016441345, "operator": "ge"}, {"budget": 0.19, "threshold": -0.07246007025241852, "operator": "ge"}, {"budget": 0.2, "threshold": -0.07691557705402374, "operator": "ge"}, {"budget": 0.21, "threshold": -0.08545950055122375, "operator": "ge"}, {"budget": 0.22, "threshold": -0.09201904386281967, "operator": "ge"}, {"budget": 0.23, "threshold": -0.09787484258413315, "operator": "ge"}, {"budget": 0.24, "threshold": -0.10770110785961151, "operator": "ge"}, {"budget": 0.25, "threshold": -0.11223643273115158, "operator": "ge"}, {"budget": 0.26, "threshold": -0.11782287806272507, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1222425326704979, "operator": "ge"}, {"budget": 0.28, "threshold": -0.13205227255821228, "operator": "ge"}, {"budget": 0.29, "threshold": -0.1413000524044037, "operator": "ge"}, {"budget": 0.3, "threshold": -0.14664269983768463, "operator": "ge"}, {"budget": 0.31, "threshold": -0.15342111885547638, "operator": "gt"}, {"budget": 0.32, "threshold": -0.16459067165851593, "operator": "ge"}, {"budget": 0.33, "threshold": -0.16995729506015778, "operator": "ge"}, {"budget": 0.34, "threshold": -0.17908665537834167, "operator": "ge"}, {"budget": 0.35, "threshold": -0.19213815033435822, "operator": "ge"}, {"budget": 0.36, "threshold": -0.19907325506210327, "operator": "ge"}, {"budget": 0.37, "threshold": -0.20983611047267914, "operator": "ge"}, {"budget": 0.38, "threshold": -0.22507920861244202, "operator": "ge"}, {"budget": 0.39, "threshold": -0.2365240603685379, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2495710849761963, "operator": "ge"}, {"budget": 0.41, "threshold": -0.26382505893707275, "operator": "ge"}, {"budget": 0.42, "threshold": -0.27656635642051697, "operator": "ge"}, {"budget": 0.43, "threshold": -0.28587889671325684, "operator": "ge"}, {"budget": 0.44, "threshold": -0.3007773458957672, "operator": "ge"}, {"budget": 0.45, "threshold": -0.31401389837265015, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3234120309352875, "operator": "ge"}, {"budget": 0.47, "threshold": -0.32799971103668213, "operator": "ge"}, {"budget": 0.48, "threshold": -0.34790337085723877, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3658129572868347, "operator": "ge"}, {"budget": 0.5, "threshold": -0.38166481256484985, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3961901068687439, "operator": "ge"}, {"budget": 0.52, "threshold": -0.409551739692688, "operator": "ge"}, {"budget": 0.53, "threshold": -0.4242469072341919, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4388805627822876, "operator": "ge"}, {"budget": 0.55, "threshold": -0.4606001079082489, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4806416928768158, "operator": "ge"}, {"budget": 0.57, "threshold": -0.4929235875606537, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5063270330429077, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5204728245735168, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5375867486000061, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5602644085884094, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5815415382385254, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5976999998092651, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6164213418960571, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6248590350151062, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6474728584289551, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6658848524093628, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6837738752365112, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6996227502822876, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7151026129722595, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7269418835639954, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7508158683776855, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7652071118354797, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7813895344734192, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7957955598831177, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8110883831977844, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8243623971939087, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8332112431526184, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8452378511428833, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8587900400161743, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8675702214241028, "operator": "ge"}, {"budget": 0.82, "threshold": -0.87538081407547, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8884798288345337, "operator": "ge"}, {"budget": 0.84, "threshold": -0.893918514251709, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8980094790458679, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9058427810668945, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9123623967170715, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9194898009300232, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9299577474594116, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9473148584365845, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9552555084228516, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9617963433265686, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9649685025215149, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9687935709953308, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9719402194023132, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9750257730484009, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9799032807350159, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9848155379295349, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9880006909370422, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.66666666666667, 97.58333333333333, 97.58333333333333, 97.5, 97.41666666666666, 97.16666666666667, 96.83333333333333, 96.75, 96.58333333333333, 96.25, 96.0, 95.75, 95.66666666666667, 95.58333333333333, 95.5, 95.25, 95.08333333333333, 94.83333333333333, 94.58333333333333, 94.16666666666667, 93.91666666666666, 93.41666666666666, 93.16666666666666, 92.75, 92.25, 91.83333333333333, 91.49999999999999, 91.25, 90.75, 90.08333333333333, 89.5, 88.83333333333333, 88.08333333333334, 87.58333333333333, 87.25, 86.5, 85.75, 85.16666666666666, 84.58333333333333, 84.0, 83.33333333333333, 82.66666666666667, 82.0, 81.41666666666667, 80.91666666666666, 80.33333333333333, 79.5, 78.75, 78.08333333333333, 77.16666666666666, 76.25, 75.33333333333333, 74.75, 74.08333333333333, 73.25, 72.24999999999999, 71.5, 70.83333333333333, 70.0, 69.16666666666667, 68.33333333333333, 67.33333333333333, 66.5, 65.58333333333333, 64.58333333333333, 63.66666666666667, 62.74999999999999, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.99999999999999, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 30.916666666666664, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 3, 3, 3, 5, 7, 9, 10, 11, 12, 13, 14, 15, 16, 16, 17, 18, 21, 25, 26, 28, 32, 35, 38, 39, 40, 41, 44, 46, 49, 52, 57, 60, 66, 69, 74, 80, 85, 89, 92, 98, 106, 113, 121, 130, 136, 140, 149, 159, 166, 173, 180, 188, 196, 204, 211, 217, 224, 234, 243, 251, 262, 273, 284, 291, 299, 309, 321, 330, 338, 349, 359, 369, 381, 391, 402, 414, 425, 436, 448, 460, 472, 483, 494, 505, 517, 528], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.66666666666667, 97.58333333333333, 97.58333333333333, 97.5, 97.41666666666666, 97.16666666666667, 96.83333333333333, 96.75, 96.58333333333333, 96.25, 96.0, 95.75, 95.66666666666667, 95.58333333333333, 95.5, 95.25, 95.08333333333333, 94.83333333333333, 94.58333333333333, 94.16666666666667, 93.91666666666666, 93.41666666666666, 93.16666666666666, 92.75, 92.25, 91.83333333333333, 91.49999999999999, 91.25, 90.75, 90.08333333333333, 89.5, 88.75, 88.08333333333334, 87.58333333333333, 87.25, 86.5, 85.75, 85.16666666666666, 84.58333333333333, 84.0, 83.33333333333333, 82.66666666666667, 82.0, 81.41666666666667, 80.91666666666666, 80.33333333333333, 79.5, 78.75, 78.08333333333333, 77.16666666666666, 76.25, 75.33333333333333, 74.75, 74.08333333333333, 73.25, 72.24999999999999, 71.41666666666666, 70.83333333333333, 70.0, 69.16666666666667, 68.33333333333333, 67.33333333333333, 66.5, 65.58333333333333, 64.58333333333333, 63.66666666666667, 62.74999999999999, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.99999999999999, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.5525, "threshold_area100": 86.55416666666667, "threshold_area25": 98.56666666666666}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.09319986192614, 90.09319986192614, 90.09319986192614, 90.02416292716603, 89.97238522609597, 89.93786675871591, 89.90334829133586, 89.8688298239558, 89.7134967207456, 89.66171901967553, 89.66171901967553, 89.66171901967553, 89.54090438384536, 89.47186744908525, 89.26475664480496, 89.17846047635484, 89.12668277528478, 88.95409043838454, 88.91957197100449, 88.83327580255437, 88.74697963410424, 88.53986882982396, 88.43631342768381, 88.40179496030376, 88.33275802554367, 88.09112875388332, 88.02209181912323, 87.91853641698309, 87.72868484639281, 87.65964791163272, 87.5043148084225, 87.22816706938211, 87.12461166724196, 86.86572316189161, 86.74490852606144, 86.55505695547117, 86.33068691750086, 86.15809458060062, 85.84742837418018, 85.65757680358992, 85.53676216775975, 85.19157749395927, 84.74283741801864, 84.58750431480841, 84.39765274421815, 84.12150500517777, 83.87987573351742, 83.67276492923715, 83.32758025543666, 82.96513634794614, 82.80980324473592, 82.67172937521575, 82.41284086986538, 81.9986192613048, 81.51536071798412, 81.04936140835346, 80.58336209872282, 80.04832585433206, 79.77217811529168, 79.06454953400069, 78.54677252329995, 78.11529168104936, 77.75284777355886, 77.20055229547808, 76.7863306869175, 75.81981360027615, 75.42285122540558, 74.64618570935451, 74.04211253020365, 73.12737314463237, 72.45426303072144, 71.97100448740076, 71.40144977562996, 70.40041422160856, 69.72730410769762, 69.22678633068692, 68.01863997238522, 67.58715913013462, 67.38004832585432, 66.98308595098376, 66.430790472903, 65.63686572316189, 64.65308940283052, 62.73731446323782, 61.649982740766305, 60.683465654124944, 60.234725578184324, 59.42354159475318, 58.99206075250258, 58.40524680704176, 57.404211253020364, 56.627545736969275, 55.78184328615809, 54.81532619951675], "calls": [0.0, 0.7248878149810148, 1.6568864342423197, 2.744218156713842, 3.6071798412150504, 4.504659993096307, 5.453917846047635, 6.109768726268554, 6.920952709699689, 7.455988954090438, 8.646876078702105, 9.90680013807387, 10.959613393165343, 12.340352088367275, 12.806351397997929, 13.979979288919573, 14.791163272350708, 15.412495685191576, 16.189161201242666, 17.397307559544355, 18.139454608215395, 19.81360027614774, 20.952709699689333, 21.660338280980323, 22.833966171901967, 23.26544701415257, 23.955816361753538, 24.577148774594406, 26.09596133931654, 26.97618225750777, 27.82188470831895, 28.633068691750086, 30.117362789092166, 30.807732136693133, 32.067656196064895, 33.39661719019675, 34.173282706247846, 35.0707628581291, 36.382464618570935, 37.40075940628236, 38.40179496030377, 39.52364515015533, 40.66275457369693, 41.42216085605799, 42.267863306869174, 43.26889886089057, 44.04556437694166, 44.40800828443217, 45.96133931653434, 47.566448049706594, 48.619261304798066, 49.464963755609254, 50.20711080428028, 51.156368657231624, 52.24370037970314, 53.95236451501554, 55.33310321021747, 56.19606489471868, 56.972730410769756, 58.025543665861235, 59.14739385571281, 60.57991025198481, 61.96064894718675, 62.96168450120815, 64.27338626164997, 64.82568173973075, 66.2064204349327, 67.27649292371419, 68.22575077666552, 68.89886089057646, 69.93441491197791, 70.69382119433897, 72.36796686227132, 73.21366931308249, 74.45633413876423, 75.61270279599586, 76.90714532274767, 77.78736624093891, 78.66758715913014, 79.56506731101139, 80.91128753883328, 81.91232309285468, 82.61995167414567, 84.24231964100794, 84.81187435277874, 85.0707628581291, 85.70935450466, 86.46876078702105, 87.46979634104247, 88.7124611667242, 91.26682775284777, 92.56127027959958, 93.68312046945115, 94.16637901277183, 95.02934069727304, 95.56437694166378, 96.23748705557473, 97.25578184328616, 98.10148429409735, 98.96444597859855, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 5, 5, 5, 6, 6, 8, 8, 9, 10, 11, 11, 14, 14, 14, 15, 16, 17, 18, 20, 20, 25, 28, 30, 33, 38, 44, 51, 55, 59, 66, 66, 69, 74, 76, 80, 81, 85, 88, 91, 94, 97, 97, 100, 103, 107, 111, 113, 115, 118, 120, 121, 125, 127, 129, 130, 130, 131, 132, 134, 136, 142, 143, 143, 143, 143, 144, 144, 144, 144, 144, 144], "harmed": [0, 0, 0, 1, 2, 2, 2, 4, 4, 4, 8, 11, 13, 15, 17, 26, 29, 29, 30, 37, 41, 53, 58, 61, 71, 73, 78, 83, 96, 103, 107, 111, 125, 130, 136, 149, 153, 163, 180, 187, 202, 212, 223, 236, 247, 266, 278, 286, 308, 334, 348, 362, 380, 397, 414, 440, 468, 481, 493, 515, 539, 570, 602, 631, 666, 683, 728, 761, 789, 813, 848, 872, 931, 957, 1006, 1045, 1100, 1141, 1172, 1207, 1266, 1309, 1340, 1412, 1438, 1450, 1474, 1507, 1555, 1614, 1731, 1795, 1851, 1877, 1924, 1950, 1984, 2042, 2087, 2136, 2192], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.07594062823611, 89.98964445978598, 89.97238522609597, 89.93786675871591, 89.92060752502589, 89.83431135657577, 89.7134967207456, 89.66171901967553, 89.66171901967553, 89.59268208491544, 89.5063859164653, 89.3683120469451, 89.23023817742491, 89.17846047635484, 89.09216430790472, 88.93683120469451, 88.79875733517432, 88.7124611667242, 88.53986882982396, 88.43631342768381, 88.40179496030376, 88.24646185709354, 88.16016568864342, 88.02209181912323, 87.97031411805315, 87.79772178115292, 87.69416637901277, 87.53883327580255, 87.33172247152227, 87.19364860200207, 86.95201933034173, 86.76216775975146, 86.67587159130134, 86.41698308595099, 86.20987228167068, 85.91646530894027, 85.65757680358992, 85.51950293406972, 85.17431826026925, 84.86365205384881, 84.70831895063858, 84.51846738004832, 84.17328270624785, 83.9316534345875, 83.74180186399724, 83.48291335864687, 83.31032102174663, 83.12046945115637, 82.7925440110459, 82.68898860890576, 82.41284086986538, 82.08491542975491, 81.68795305488436, 81.44632378322402, 81.01484294097342, 80.54884363134276, 80.15188125647221, 79.70314118053157, 79.18536416983085, 78.70210562651017, 78.16706938211944, 77.70107007248878, 77.16603382809804, 76.63099758370728, 75.99240593717639, 75.45736969278563, 74.97411114946496, 74.37003797031412, 73.81774249223334, 73.04107697618225, 72.31618916120124, 71.69485674836037, 71.0735243355195, 70.33137728684846, 69.65826717293751, 68.95063859164652, 68.2084915429755, 67.46634449430445, 66.77597514670349, 65.98205039696236, 65.22264411460131, 64.44597859855021, 63.80738695201933, 62.96168450120814, 62.150500517777004, 61.2875388332758, 60.39005868139454, 59.44080082844321, 58.62961684501208, 57.66309975837072, 56.73110113910942, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.51449644562257, "threshold_area100": 80.52830514325163, "threshold_area25": 89.7756299620297}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.668215751647949, "operator": "ge"}, {"budget": 0.02, "threshold": -3.37603759765625, "operator": "ge"}, {"budget": 0.03, "threshold": -3.740628957748413, "operator": "ge"}, {"budget": 0.04, "threshold": -4.346371650695801, "operator": "ge"}, {"budget": 0.05, "threshold": -4.775775909423828, "operator": "ge"}, {"budget": 0.06, "threshold": -5.081321716308594, "operator": "ge"}, {"budget": 0.07, "threshold": -5.503873825073242, "operator": "ge"}, {"budget": 0.08, "threshold": -5.78034782409668, "operator": "ge"}, {"budget": 0.09, "threshold": -6.2783308029174805, "operator": "ge"}, {"budget": 0.1, "threshold": -6.542464256286621, "operator": "ge"}, {"budget": 0.11, "threshold": -6.797158241271973, "operator": "ge"}, {"budget": 0.12, "threshold": -7.089688777923584, "operator": "ge"}, {"budget": 0.13, "threshold": -7.270529270172119, "operator": "ge"}, {"budget": 0.14, "threshold": -7.617358207702637, "operator": "ge"}, {"budget": 0.15, "threshold": -7.824540138244629, "operator": "ge"}, {"budget": 0.16, "threshold": -8.04131031036377, "operator": "ge"}, {"budget": 0.17, "threshold": -8.213923454284668, "operator": "ge"}, {"budget": 0.18, "threshold": -8.393780708312988, "operator": "ge"}, {"budget": 0.19, "threshold": -8.53858757019043, "operator": "ge"}, {"budget": 0.2, "threshold": -8.64145278930664, "operator": "ge"}, {"budget": 0.21, "threshold": -8.703463554382324, "operator": "ge"}, {"budget": 0.22, "threshold": -8.806120872497559, "operator": "ge"}, {"budget": 0.23, "threshold": -8.931336402893066, "operator": "ge"}, {"budget": 0.24, "threshold": -8.996281623840332, "operator": "ge"}, {"budget": 0.25, "threshold": -9.140061378479004, "operator": "ge"}, {"budget": 0.26, "threshold": -9.18258285522461, "operator": "ge"}, {"budget": 0.27, "threshold": -9.250818252563477, "operator": "ge"}, {"budget": 0.28, "threshold": -9.36166000366211, "operator": "ge"}, {"budget": 0.29, "threshold": -9.395235061645508, "operator": "ge"}, {"budget": 0.3, "threshold": -9.479500770568848, "operator": "ge"}, {"budget": 0.31, "threshold": -9.566207885742188, "operator": "ge"}, {"budget": 0.32, "threshold": -9.612504959106445, "operator": "ge"}, {"budget": 0.33, "threshold": -9.6901273727417, "operator": "ge"}, {"budget": 0.34, "threshold": -9.720084190368652, "operator": "ge"}, {"budget": 0.35, "threshold": -9.79479694366455, "operator": "ge"}, {"budget": 0.36, "threshold": -9.838916778564453, "operator": "ge"}, {"budget": 0.37, "threshold": -9.898168563842773, "operator": "ge"}, {"budget": 0.38, "threshold": -9.941084861755371, "operator": "ge"}, {"budget": 0.39, "threshold": -9.987930297851562, "operator": "ge"}, {"budget": 0.4, "threshold": -10.035469055175781, "operator": "ge"}, {"budget": 0.41, "threshold": -10.051446914672852, "operator": "ge"}, {"budget": 0.42, "threshold": -10.093502044677734, "operator": "ge"}, {"budget": 0.43, "threshold": -10.146738052368164, "operator": "ge"}, {"budget": 0.44, "threshold": -10.190084457397461, "operator": "ge"}, {"budget": 0.45, "threshold": -10.247654914855957, "operator": "ge"}, {"budget": 0.46, "threshold": -10.29404354095459, "operator": "ge"}, {"budget": 0.47, "threshold": -10.323452949523926, "operator": "ge"}, {"budget": 0.48, "threshold": -10.35268497467041, "operator": "ge"}, {"budget": 0.49, "threshold": -10.391091346740723, "operator": "ge"}, {"budget": 0.5, "threshold": -10.430304527282715, "operator": "ge"}, {"budget": 0.51, "threshold": -10.475117683410645, "operator": "ge"}, {"budget": 0.52, "threshold": -10.512063980102539, "operator": "ge"}, {"budget": 0.53, "threshold": -10.53702163696289, "operator": "ge"}, {"budget": 0.54, "threshold": -10.59205436706543, "operator": "ge"}, {"budget": 0.55, "threshold": -10.60465145111084, "operator": "ge"}, {"budget": 0.56, "threshold": -10.666940689086914, "operator": "ge"}, {"budget": 0.57, "threshold": -10.709228515625, "operator": "ge"}, {"budget": 0.58, "threshold": -10.760709762573242, "operator": "ge"}, {"budget": 0.59, "threshold": -10.80488109588623, "operator": "ge"}, {"budget": 0.6, "threshold": -10.832392692565918, "operator": "ge"}, {"budget": 0.61, "threshold": -10.864496231079102, "operator": "ge"}, {"budget": 0.62, "threshold": -10.921165466308594, "operator": "ge"}, {"budget": 0.63, "threshold": -10.99016284942627, "operator": "ge"}, {"budget": 0.64, "threshold": -11.031970024108887, "operator": "ge"}, {"budget": 0.65, "threshold": -11.066926956176758, "operator": "ge"}, {"budget": 0.66, "threshold": -11.107050895690918, "operator": "ge"}, {"budget": 0.67, "threshold": -11.149925231933594, "operator": "ge"}, {"budget": 0.68, "threshold": -11.191434860229492, "operator": "ge"}, {"budget": 0.69, "threshold": -11.239511489868164, "operator": "ge"}, {"budget": 0.7, "threshold": -11.286600112915039, "operator": "ge"}, {"budget": 0.71, "threshold": -11.326547622680664, "operator": "ge"}, {"budget": 0.72, "threshold": -11.374025344848633, "operator": "ge"}, {"budget": 0.73, "threshold": -11.399137496948242, "operator": "ge"}, {"budget": 0.74, "threshold": -11.442228317260742, "operator": "ge"}, {"budget": 0.75, "threshold": -11.505289077758789, "operator": "ge"}, {"budget": 0.76, "threshold": -11.538166046142578, "operator": "ge"}, {"budget": 0.77, "threshold": -11.56137466430664, "operator": "ge"}, {"budget": 0.78, "threshold": -11.609557151794434, "operator": "ge"}, {"budget": 0.79, "threshold": -11.649456024169922, "operator": "ge"}, {"budget": 0.8, "threshold": -11.67734146118164, "operator": "ge"}, {"budget": 0.81, "threshold": -11.703268051147461, "operator": "ge"}, {"budget": 0.82, "threshold": -11.752939224243164, "operator": "ge"}, {"budget": 0.83, "threshold": -11.825626373291016, "operator": "ge"}, {"budget": 0.84, "threshold": -11.869805335998535, "operator": "ge"}, {"budget": 0.85, "threshold": -11.935235023498535, "operator": "ge"}, {"budget": 0.86, "threshold": -11.984770774841309, "operator": "ge"}, {"budget": 0.87, "threshold": -12.020709991455078, "operator": "ge"}, {"budget": 0.88, "threshold": -12.047248840332031, "operator": "ge"}, {"budget": 0.89, "threshold": -12.097284317016602, "operator": "ge"}, {"budget": 0.9, "threshold": -12.166501998901367, "operator": "ge"}, {"budget": 0.91, "threshold": -12.228219985961914, "operator": "ge"}, {"budget": 0.92, "threshold": -12.291597366333008, "operator": "ge"}, {"budget": 0.93, "threshold": -12.390893936157227, "operator": "ge"}, {"budget": 0.94, "threshold": -12.460453033447266, "operator": "ge"}, {"budget": 0.95, "threshold": -12.54010009765625, "operator": "ge"}, {"budget": 0.96, "threshold": -12.630685806274414, "operator": "ge"}, {"budget": 0.97, "threshold": -12.819265365600586, "operator": "ge"}, {"budget": 0.98, "threshold": -12.969779968261719, "operator": "ge"}, {"budget": 0.99, "threshold": -13.12373161315918, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.5, 98.25, 97.66666666666667, 97.33333333333333, 96.83333333333333, 96.33333333333333, 95.83333333333333, 95.16666666666667, 94.75, 94.16666666666667, 93.58333333333333, 93.08333333333333, 92.75, 92.41666666666666, 92.0, 91.58333333333333, 91.33333333333333, 90.99999999999999, 90.83333333333333, 90.33333333333333, 90.16666666666666, 89.99999999999999, 89.83333333333333, 89.75, 89.5, 89.33333333333333, 89.25, 89.0, 88.83333333333333, 88.66666666666666, 88.5, 88.08333333333334, 87.66666666666666, 87.41666666666667, 87.25, 86.83333333333333, 86.83333333333333, 86.58333333333333, 86.33333333333333, 85.83333333333333, 85.66666666666667, 85.33333333333333, 85.0, 84.74999999999999, 84.58333333333333, 84.41666666666666, 84.16666666666667, 83.66666666666667, 83.41666666666666, 83.08333333333333, 82.58333333333333, 82.25, 81.91666666666666, 81.58333333333333, 81.5, 81.25, 80.83333333333333, 80.41666666666667, 80.16666666666666, 79.91666666666666, 79.41666666666667, 79.16666666666666, 78.58333333333333, 78.08333333333333, 77.66666666666666, 77.33333333333333, 76.91666666666667, 76.5, 76.16666666666666, 75.5, 75.16666666666667, 75.0, 74.58333333333333, 74.0, 73.25, 72.41666666666666, 71.75, 71.0, 70.5, 69.83333333333333, 69.24999999999999, 68.83333333333333, 68.0, 67.33333333333333, 66.58333333333333, 65.91666666666667, 65.41666666666666, 64.75, 63.916666666666664, 63.083333333333336, 62.5, 61.83333333333333, 60.83333333333333, 60.083333333333336, 59.33333333333333, 58.5, 57.74999999999999, 56.91666666666666, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 5, 8, 15, 19, 26, 32, 39, 47, 52, 59, 66, 72, 76, 80, 85, 90, 93, 97, 99, 105, 107, 109, 111, 112, 115, 117, 118, 121, 123, 125, 127, 132, 137, 140, 142, 147, 147, 150, 153, 159, 161, 165, 169, 172, 174, 176, 179, 185, 188, 192, 198, 202, 206, 210, 211, 214, 219, 224, 227, 230, 236, 239, 246, 252, 257, 261, 266, 271, 275, 283, 287, 289, 294, 301, 310, 320, 328, 337, 343, 351, 358, 363, 373, 381, 390, 398, 404, 412, 422, 432, 439, 447, 459, 468, 477, 487, 496, 506, 517, 528], "exact_score": [98.91666666666666, 98.5, 98.25, 97.66666666666667, 97.33333333333333, 96.83333333333333, 96.33333333333333, 95.83333333333333, 95.16666666666667, 94.75, 94.16666666666667, 93.58333333333333, 93.08333333333333, 92.75, 92.41666666666666, 92.0, 91.58333333333333, 91.33333333333333, 90.99999999999999, 90.83333333333333, 90.33333333333333, 90.16666666666666, 89.99999999999999, 89.83333333333333, 89.75, 89.5, 89.33333333333333, 89.25, 89.0, 88.83333333333333, 88.66666666666666, 88.5, 88.08333333333334, 87.66666666666666, 87.41666666666667, 87.25, 86.83333333333333, 86.83333333333333, 86.58333333333333, 86.33333333333333, 85.83333333333333, 85.58333333333333, 85.33333333333333, 85.0, 84.74999999999999, 84.58333333333333, 84.41666666666666, 84.16666666666667, 83.66666666666667, 83.41666666666666, 83.08333333333333, 82.58333333333333, 82.25, 81.91666666666666, 81.58333333333333, 81.5, 81.25, 80.83333333333333, 80.41666666666667, 80.16666666666666, 79.91666666666666, 79.41666666666667, 79.16666666666666, 78.58333333333333, 78.08333333333333, 77.66666666666666, 77.33333333333333, 76.91666666666667, 76.5, 76.08333333333333, 75.5, 75.16666666666667, 75.0, 74.58333333333333, 74.0, 73.25, 72.41666666666666, 71.75, 71.0, 70.5, 69.83333333333333, 69.24999999999999, 68.83333333333333, 68.0, 67.33333333333333, 66.58333333333333, 65.91666666666667, 65.41666666666666, 64.75, 63.916666666666664, 63.083333333333336, 62.5, 61.83333333333333, 60.83333333333333, 60.083333333333336, 59.33333333333333, 58.5, 57.74999999999999, 56.91666666666666, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 80.82416666666667, "threshold_area100": 80.82583333333334, "threshold_area25": 93.50833333333335}, "test": {"score": [90.16223679668623, 89.7134967207456, 89.07490507421471, 88.83327580255437, 87.95305488436314, 86.67587159130134, 85.69209527096997, 84.25957887469797, 83.44839489126683, 81.54987918536416, 80.75595443562305, 80.10010355540214, 79.13358646876078, 78.7193648602002, 77.77010700724888, 77.51121850189851, 77.18329306178805, 76.64825681739731, 76.32033137728685, 76.04418363824645, 75.81981360027615, 75.69899896444598, 75.52640662754573, 75.30203658957542, 75.12944425267519, 74.87055574732481, 74.80151881256472, 74.55988954090438, 74.18018639972385, 74.04211253020365, 73.83500172592336, 73.62789092164309, 73.5070762858129, 73.31722471522264, 73.16189161201243, 72.95478080773213, 72.92026234035208, 72.67863306869174, 72.48878149810149, 72.35070762858129, 72.143596824301, 72.02278218847083, 71.95374525371074, 71.74663444943043, 71.5050051777701, 71.31515360717984, 71.10804280289955, 71.03900586813945, 70.91819123230928, 70.76285812909906, 70.6420434932689, 70.38315498791854, 70.19330341732827, 70.10700724887815, 69.81360027614774, 69.76182257507766, 69.38211943389715, 69.17500862961684, 68.96789782533655, 68.65723161891611, 68.4328615809458, 68.31204694511564, 67.84604763548498, 67.46634449430445, 67.32827062478425, 67.19019675526407, 67.13841905419399, 66.82775284777355, 66.5688643424232, 66.2927166033828, 65.9302726958923, 65.75768035899205, 65.48153261995166, 65.30894028305143, 65.03279254401104, 64.65308940283052, 64.39420089748015, 64.20434932688988, 63.824646185709355, 63.479461511908866, 63.272350707628576, 63.15153607179841, 62.75457369692785, 62.495685191577486, 62.20227821884707, 61.684501208146344, 61.47739040386606, 61.2185018985157, 60.97687262685536, 60.61442871936485, 60.2174663444943, 60.027614773904034, 59.682430100103545, 59.13013462202278, 58.66413531239212, 58.31895063859165, 57.76665516051087, 56.80013807386952, 56.265101829478766, 55.69554711770797, 54.81532619951675], "calls": [0.0, 4.124956851915775, 8.12909906800138, 10.493614083534691, 14.118053158439764, 17.79426993441491, 20.279599585778392, 23.62789092164308, 25.578184328615812, 29.54780807732137, 31.204694511563684, 32.775284777355886, 34.53572661373835, 35.277873662409384, 37.297204004142216, 38.40179496030377, 39.62720055229548, 40.93890231273731, 42.267863306869174, 43.476009665170864, 44.3562305833621, 44.9257852951329, 45.94408008284432, 47.34207801173628, 48.01518812564722, 49.4994822229893, 49.94822229892993, 50.880220918191235, 52.329996548153254, 52.76147739040387, 53.83154987918537, 54.901622367966866, 55.55747324818778, 56.66206420434933, 57.00724887814981, 57.97376596479117, 58.664135312392126, 59.54435623058336, 60.3555402140145, 61.114946496375566, 61.80531584397653, 62.150500517777004, 62.84086986537798, 63.824646185709355, 64.41146013117017, 65.48153261995168, 66.55160510873317, 66.87953054884363, 67.29375215740421, 67.84604763548498, 68.60545391784605, 69.46841560234725, 70.12426648256817, 70.5730065585088, 71.50500517777012, 71.66033828098033, 72.7821884708319, 73.50707628581291, 74.24922333448394, 75.04314808422507, 75.62996202968588, 76.11322057300656, 77.06247842595789, 78.2188470831895, 78.82292026234036, 79.32343803935106, 79.84121505005177, 80.53158439765275, 81.20469451156369, 81.9295823265447, 82.7925440110459, 83.24128408698654, 83.91439420089748, 84.24231964100794, 84.84639282015878, 85.65757680358992, 86.02002071108042, 86.36520538488092, 87.12461166724196, 87.67690714532274, 88.0738695201933, 88.33275802554367, 88.93683120469451, 89.50638591646532, 90.02416292716603, 90.97342078011737, 91.37038315498792, 91.69830859509838, 92.02623403520883, 92.52675181221953, 93.19986192613048, 93.63134276838109, 94.13186054539179, 94.83948912668278, 95.47808077321366, 95.9268208491543, 96.63444943044529, 97.79081808767691, 98.3603727994477, 99.03348291335865, 100.0], "rescued": [0, 19, 44, 58, 84, 96, 103, 115, 121, 124, 126, 128, 128, 129, 129, 130, 132, 132, 133, 134, 134, 134, 134, 135, 135, 135, 135, 135, 135, 135, 136, 136, 136, 137, 137, 137, 137, 137, 137, 137, 137, 137, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 140, 140, 140, 140, 141, 141, 141, 141, 141, 141, 141, 142, 143, 143, 143, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 45, 107, 135, 212, 298, 362, 457, 510, 623, 671, 711, 767, 792, 847, 863, 884, 915, 935, 952, 965, 972, 982, 996, 1006, 1021, 1025, 1039, 1061, 1069, 1082, 1094, 1101, 1113, 1122, 1134, 1136, 1150, 1161, 1169, 1181, 1188, 1194, 1206, 1220, 1231, 1243, 1247, 1254, 1263, 1270, 1285, 1296, 1301, 1318, 1321, 1343, 1355, 1367, 1385, 1398, 1405, 1432, 1455, 1463, 1471, 1474, 1493, 1508, 1524, 1545, 1555, 1571, 1581, 1598, 1621, 1636, 1647, 1670, 1690, 1702, 1709, 1732, 1747, 1764, 1794, 1806, 1821, 1835, 1856, 1879, 1890, 1910, 1942, 1969, 1989, 2021, 2077, 2108, 2141, 2192], "exact_score": [90.16223679668623, 90.1449775629962, 89.97238522609597, 89.90334829133586, 89.7134967207456, 89.57542285122541, 89.43734898170521, 89.31653434587504, 89.14394200897479, 89.02312737314463, 88.91957197100449, 88.79875733517432, 88.574387297204, 88.24646185709354, 88.0048325854332, 87.76320331377286, 87.38350017259233, 87.0037970314118, 86.62409389023126, 86.22713151536071, 85.88194684156024, 85.24335519502934, 84.81187435277874, 84.4321712115982, 84.1387642388678, 83.69002416292717, 83.2930617880566, 82.82706247842594, 82.3610631687953, 81.80876769071452, 81.25647221263375, 80.8595098377632, 80.39351052813255, 79.9275112185019, 79.3579565067311, 78.87469796341043, 78.42595788746979, 77.87366240938903, 77.5802554366586, 77.39040386606834, 76.97618225750776, 76.64825681739731, 76.35484984466689, 76.18225750776665, 75.9406282361063, 75.64722126337591, 75.52640662754573, 75.38833275802554, 75.12944425267519, 74.95685191577493, 74.7842595788747, 74.52537107352433, 74.26648256817397, 73.95581636175353, 73.81774249223334, 73.59337245426303, 73.38626164998274, 73.16189161201243, 72.95478080773213, 72.83396617190196, 72.59233690024163, 72.36796686227132, 72.07455988954091, 71.93648602002071, 71.67759751467034, 71.40144977562996, 71.21159820503969, 71.02174663444943, 70.76285812909906, 70.55574732481877, 70.2105626510183, 69.95167414566794, 69.65826717293751, 69.31308249913704, 69.00241629271659, 68.67449085260614, 68.32930617880567, 67.91508457024507, 67.53538142906454, 67.27649292371419, 67.0693821194339, 66.63790127718329, 66.2927166033828, 65.84397652744218, 65.46427338626164, 64.9292371418709, 64.4287193648602, 63.9282015878495, 63.30686917500863, 62.70279599585778, 62.23679668622712, 61.684501208146344, 61.011391094235414, 60.3555402140145, 59.7859855022437, 59.04383845357266, 58.26717293752157, 57.4387297204004, 56.59302726958922, 55.73006558508802, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.28655868514784, "threshold_area100": 70.46133931653435, "threshold_area25": 80.65585088022092}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -1.127821922302246, "operator": "ge"}, {"budget": 0.02, "threshold": -1.8536972999572754, "operator": "ge"}, {"budget": 0.03, "threshold": -2.211453676223755, "operator": "ge"}, {"budget": 0.04, "threshold": -2.6198861598968506, "operator": "ge"}, {"budget": 0.05, "threshold": -2.8415169715881348, "operator": "ge"}, {"budget": 0.06, "threshold": -3.2566654682159424, "operator": "ge"}, {"budget": 0.07, "threshold": -3.6557393074035645, "operator": "ge"}, {"budget": 0.08, "threshold": -4.026885986328125, "operator": "ge"}, {"budget": 0.09, "threshold": -4.29890775680542, "operator": "ge"}, {"budget": 0.1, "threshold": -4.662632942199707, "operator": "ge"}, {"budget": 0.11, "threshold": -4.981775760650635, "operator": "ge"}, {"budget": 0.12, "threshold": -5.62762451171875, "operator": "ge"}, {"budget": 0.13, "threshold": -5.817890644073486, "operator": "ge"}, {"budget": 0.14, "threshold": -6.02812385559082, "operator": "ge"}, {"budget": 0.15, "threshold": -6.392937660217285, "operator": "ge"}, {"budget": 0.16, "threshold": -6.675651550292969, "operator": "ge"}, {"budget": 0.17, "threshold": -6.850994110107422, "operator": "ge"}, {"budget": 0.18, "threshold": -7.080150604248047, "operator": "ge"}, {"budget": 0.19, "threshold": -7.281964302062988, "operator": "ge"}, {"budget": 0.2, "threshold": -7.465348243713379, "operator": "ge"}, {"budget": 0.21, "threshold": -7.584186553955078, "operator": "ge"}, {"budget": 0.22, "threshold": -7.728518009185791, "operator": "ge"}, {"budget": 0.23, "threshold": -7.853816509246826, "operator": "ge"}, {"budget": 0.24, "threshold": -7.957414150238037, "operator": "ge"}, {"budget": 0.25, "threshold": -8.038979530334473, "operator": "ge"}, {"budget": 0.26, "threshold": -8.128877639770508, "operator": "ge"}, {"budget": 0.27, "threshold": -8.221048355102539, "operator": "ge"}, {"budget": 0.28, "threshold": -8.291799545288086, "operator": "ge"}, {"budget": 0.29, "threshold": -8.391067504882812, "operator": "ge"}, {"budget": 0.3, "threshold": -8.510891914367676, "operator": "ge"}, {"budget": 0.31, "threshold": -8.56991195678711, "operator": "ge"}, {"budget": 0.32, "threshold": -8.619531631469727, "operator": "ge"}, {"budget": 0.33, "threshold": -8.66602897644043, "operator": "ge"}, {"budget": 0.34, "threshold": -8.719122886657715, "operator": "ge"}, {"budget": 0.35, "threshold": -8.778685569763184, "operator": "ge"}, {"budget": 0.36, "threshold": -8.843595504760742, "operator": "ge"}, {"budget": 0.37, "threshold": -8.881170272827148, "operator": "ge"}, {"budget": 0.38, "threshold": -8.918998718261719, "operator": "ge"}, {"budget": 0.39, "threshold": -8.947015762329102, "operator": "ge"}, {"budget": 0.4, "threshold": -8.977157592773438, "operator": "ge"}, {"budget": 0.41, "threshold": -9.008174896240234, "operator": "ge"}, {"budget": 0.42, "threshold": -9.071802139282227, "operator": "ge"}, {"budget": 0.43, "threshold": -9.111503601074219, "operator": "ge"}, {"budget": 0.44, "threshold": -9.144074440002441, "operator": "ge"}, {"budget": 0.45, "threshold": -9.183228492736816, "operator": "ge"}, {"budget": 0.46, "threshold": -9.225181579589844, "operator": "ge"}, {"budget": 0.47, "threshold": -9.265003204345703, "operator": "ge"}, {"budget": 0.48, "threshold": -9.292963981628418, "operator": "ge"}, {"budget": 0.49, "threshold": -9.341815948486328, "operator": "ge"}, {"budget": 0.5, "threshold": -9.386039733886719, "operator": "ge"}, {"budget": 0.51, "threshold": -9.421751022338867, "operator": "ge"}, {"budget": 0.52, "threshold": -9.457894325256348, "operator": "ge"}, {"budget": 0.53, "threshold": -9.490824699401855, "operator": "ge"}, {"budget": 0.54, "threshold": -9.522542953491211, "operator": "ge"}, {"budget": 0.55, "threshold": -9.546056747436523, "operator": "ge"}, {"budget": 0.56, "threshold": -9.575678825378418, "operator": "ge"}, {"budget": 0.57, "threshold": -9.599285125732422, "operator": "ge"}, {"budget": 0.58, "threshold": -9.63172435760498, "operator": "ge"}, {"budget": 0.59, "threshold": -9.654810905456543, "operator": "ge"}, {"budget": 0.6, "threshold": -9.680869102478027, "operator": "ge"}, {"budget": 0.61, "threshold": -9.709678649902344, "operator": "ge"}, {"budget": 0.62, "threshold": -9.735474586486816, "operator": "ge"}, {"budget": 0.63, "threshold": -9.769532203674316, "operator": "ge"}, {"budget": 0.64, "threshold": -9.789368629455566, "operator": "ge"}, {"budget": 0.65, "threshold": -9.847376823425293, "operator": "ge"}, {"budget": 0.66, "threshold": -9.869786262512207, "operator": "ge"}, {"budget": 0.67, "threshold": -9.901805877685547, "operator": "ge"}, {"budget": 0.68, "threshold": -9.935892105102539, "operator": "ge"}, {"budget": 0.69, "threshold": -9.96085262298584, "operator": "ge"}, {"budget": 0.7, "threshold": -9.997885704040527, "operator": "ge"}, {"budget": 0.71, "threshold": -10.028443336486816, "operator": "ge"}, {"budget": 0.72, "threshold": -10.048498153686523, "operator": "ge"}, {"budget": 0.73, "threshold": -10.078227996826172, "operator": "ge"}, {"budget": 0.74, "threshold": -10.097635269165039, "operator": "ge"}, {"budget": 0.75, "threshold": -10.141378402709961, "operator": "ge"}, {"budget": 0.76, "threshold": -10.184944152832031, "operator": "ge"}, {"budget": 0.77, "threshold": -10.207744598388672, "operator": "ge"}, {"budget": 0.78, "threshold": -10.245819091796875, "operator": "ge"}, {"budget": 0.79, "threshold": -10.268019676208496, "operator": "ge"}, {"budget": 0.8, "threshold": -10.319700241088867, "operator": "ge"}, {"budget": 0.81, "threshold": -10.355951309204102, "operator": "ge"}, {"budget": 0.82, "threshold": -10.397300720214844, "operator": "ge"}, {"budget": 0.83, "threshold": -10.42721939086914, "operator": "ge"}, {"budget": 0.84, "threshold": -10.450653076171875, "operator": "ge"}, {"budget": 0.85, "threshold": -10.488313674926758, "operator": "ge"}, {"budget": 0.86, "threshold": -10.527963638305664, "operator": "ge"}, {"budget": 0.87, "threshold": -10.558823585510254, "operator": "gt"}, {"budget": 0.88, "threshold": -10.607306480407715, "operator": "ge"}, {"budget": 0.89, "threshold": -10.6425199508667, "operator": "ge"}, {"budget": 0.9, "threshold": -10.670905113220215, "operator": "ge"}, {"budget": 0.91, "threshold": -10.719167709350586, "operator": "ge"}, {"budget": 0.92, "threshold": -10.776196479797363, "operator": "ge"}, {"budget": 0.93, "threshold": -10.824195861816406, "operator": "ge"}, {"budget": 0.94, "threshold": -10.900863647460938, "operator": "ge"}, {"budget": 0.95, "threshold": -11.012779235839844, "operator": "ge"}, {"budget": 0.96, "threshold": -11.059414863586426, "operator": "ge"}, {"budget": 0.97, "threshold": -11.139057159423828, "operator": "ge"}, {"budget": 0.98, "threshold": -11.200634002685547, "operator": "ge"}, {"budget": 0.99, "threshold": -11.324813842773438, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.5, 98.08333333333333, 97.58333333333333, 97.16666666666667, 96.75, 96.25, 95.58333333333333, 94.91666666666666, 94.25, 93.58333333333333, 93.08333333333333, 92.41666666666666, 92.08333333333333, 91.58333333333333, 91.33333333333333, 90.91666666666667, 90.5, 90.25, 89.91666666666667, 89.66666666666666, 89.33333333333333, 88.83333333333333, 88.5, 88.41666666666667, 88.16666666666666, 88.08333333333334, 87.58333333333333, 87.41666666666667, 87.08333333333333, 86.83333333333333, 86.58333333333333, 86.33333333333333, 86.08333333333333, 85.75, 85.58333333333333, 85.08333333333333, 84.83333333333333, 84.5, 84.16666666666667, 83.83333333333333, 83.58333333333333, 83.16666666666667, 82.83333333333334, 82.16666666666667, 82.0, 81.75, 81.25, 80.91666666666666, 80.5, 80.16666666666666, 79.83333333333333, 79.58333333333333, 79.25, 78.83333333333333, 78.41666666666667, 78.0, 77.91666666666667, 77.41666666666667, 77.08333333333333, 76.66666666666666, 76.58333333333334, 76.08333333333333, 75.83333333333333, 75.33333333333333, 75.0, 74.75, 74.16666666666667, 73.83333333333333, 73.5, 73.08333333333333, 72.58333333333333, 72.16666666666667, 71.5, 71.08333333333333, 70.33333333333334, 69.66666666666667, 69.08333333333333, 68.5, 68.08333333333333, 67.58333333333333, 67.08333333333333, 66.66666666666666, 66.08333333333334, 65.58333333333333, 65.08333333333334, 64.5, 63.99999999999999, 63.416666666666664, 62.74999999999999, 62.0, 61.416666666666664, 60.66666666666667, 60.0, 59.25, 58.58333333333333, 57.74999999999999, 56.99999999999999, 56.58333333333333, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 86.91666666666666, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 5, 10, 16, 22, 27, 34, 42, 50, 58, 66, 72, 80, 84, 90, 93, 98, 103, 106, 110, 113, 117, 123, 127, 128, 131, 132, 138, 140, 144, 147, 150, 153, 156, 160, 162, 168, 171, 175, 179, 183, 186, 191, 195, 203, 205, 208, 214, 218, 223, 227, 231, 234, 238, 243, 248, 253, 254, 260, 264, 269, 270, 276, 279, 285, 289, 292, 299, 303, 307, 312, 318, 323, 331, 336, 345, 353, 360, 367, 372, 378, 384, 389, 396, 402, 408, 415, 421, 428, 436, 445, 452, 461, 469, 478, 486, 496, 505, 510, 517, 528], "exact_score": [98.91666666666666, 98.5, 98.08333333333333, 97.58333333333333, 97.16666666666667, 96.75, 96.25, 95.58333333333333, 94.91666666666666, 94.25, 93.58333333333333, 93.08333333333333, 92.41666666666666, 92.08333333333333, 91.58333333333333, 91.33333333333333, 90.91666666666667, 90.5, 90.25, 89.91666666666667, 89.66666666666666, 89.33333333333333, 88.83333333333333, 88.5, 88.41666666666667, 88.16666666666666, 88.08333333333334, 87.58333333333333, 87.41666666666667, 87.08333333333333, 86.83333333333333, 86.58333333333333, 86.33333333333333, 86.08333333333333, 85.75, 85.58333333333333, 85.08333333333333, 84.83333333333333, 84.5, 84.16666666666667, 83.83333333333333, 83.58333333333333, 83.16666666666667, 82.83333333333334, 82.16666666666667, 82.0, 81.75, 81.25, 80.91666666666666, 80.5, 80.16666666666666, 79.83333333333333, 79.58333333333333, 79.25, 78.83333333333333, 78.41666666666667, 78.0, 77.91666666666667, 77.41666666666667, 77.08333333333333, 76.66666666666666, 76.58333333333334, 76.08333333333333, 75.83333333333333, 75.33333333333333, 75.0, 74.75, 74.16666666666667, 73.83333333333333, 73.5, 73.08333333333333, 72.58333333333333, 72.16666666666667, 71.5, 71.08333333333333, 70.33333333333334, 69.66666666666667, 69.08333333333333, 68.5, 68.08333333333333, 67.58333333333333, 67.08333333333333, 66.58333333333333, 66.08333333333334, 65.58333333333333, 65.08333333333334, 64.5, 63.99999999999999, 63.416666666666664, 62.74999999999999, 62.0, 61.416666666666664, 60.66666666666667, 60.0, 59.25, 58.58333333333333, 57.74999999999999, 56.99999999999999, 56.58333333333333, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 79.08833333333331, "threshold_area100": 79.08916666666666, "threshold_area25": 92.97500000000001}, "test": {"score": [90.16223679668623, 89.81705212288574, 88.81601656886434, 88.38453572661373, 87.29720400414222, 86.38246461857094, 84.77735588539869, 83.06869175008629, 81.94684156023472, 81.01484294097342, 80.20365895754227, 79.32343803935105, 78.06351397997928, 77.66655160510874, 77.2178115291681, 76.47566448049706, 75.97514670348636, 75.62996202968588, 75.33655505695546, 75.008629616845, 74.68070417673455, 74.47359337245426, 74.30100103555401, 74.0766309975837, 73.88677942699344, 73.67966862271315, 73.40352088367277, 73.26544701415256, 72.98929927511219, 72.73041076976182, 72.33344839489126, 72.09181912323092, 71.90196755264066, 71.72937521574042, 71.60856057991025, 71.34967207455989, 71.10804280289955, 71.02174663444943, 70.86641353123922, 70.71108042802899, 70.59026579219882, 70.38315498791854, 70.24508111839833, 70.08974801518812, 69.89989644459786, 69.7100448740076, 69.46841560234725, 69.19226786330687, 69.05419399378667, 68.91612012426648, 68.63997238522609, 68.55367621677597, 68.32930617880567, 68.10493614083535, 67.9323438039351, 67.82878840179495, 67.58715913013462, 67.41456679323437, 67.22471522264411, 67.0693821194339, 66.84501208146358, 66.72419744563341, 66.48256817397306, 66.27545736969279, 66.12012426648258, 65.68864342423197, 65.5678287884018, 65.2916810493614, 65.13634794615119, 64.89471867449085, 64.63583016914049, 64.4287193648602, 64.1525716258198, 63.99723852260959, 63.84190541939938, 63.53123921297894, 63.2205730065585, 63.03072143596824, 62.7890921643079, 62.633759061097685, 62.2195374525371, 61.90887124611667, 61.460131170176034, 61.30479806696583, 61.16672419744563, 60.873317224715215, 60.528132550914734, 60.2865032792544, 60.09665170866414, 59.88954090438384, 59.61339316534345, 59.31998619261305, 58.87124611667242, 58.577839143942015, 58.04280289955126, 57.317915084570245, 57.128063513979974, 56.523990334829136, 56.127027959958575, 55.52295478080773, 54.81532619951675], "calls": [0.0, 3.624439074905074, 9.19917155678288, 12.115982050396962, 15.395236451501553, 17.5008629616845, 20.900931998619264, 24.473593372454264, 26.70003451846738, 28.44321712115982, 30.531584397652743, 32.29202623403521, 35.260614428719364, 36.1408353469106, 37.15913013462202, 38.93683120469451, 40.31756989989644, 41.07697618225751, 42.388677942699346, 43.665861235761135, 44.75319295823266, 45.59889540904384, 46.84156023472558, 47.70452191922679, 48.619261304798066, 49.4304452882292, 50.41422160856058, 51.19088712461167, 52.05384880911288, 53.12392129789437, 54.556437694166384, 55.31584397652745, 56.07525025888851, 56.627545736969275, 57.14532274767, 58.18087676907145, 59.302726958923024, 59.924059371763896, 60.648947186744905, 61.149464963755605, 61.51190887124611, 62.06420434932689, 63.2896099413186, 63.9282015878495, 64.65308940283052, 65.23990334829134, 66.17190196755264, 66.94856748360372, 67.29375215740421, 68.22575077666552, 69.29582326544701, 69.98619261304798, 70.93545046599931, 71.66033828098033, 72.21263375906109, 72.6441146013117, 73.2827062478426, 73.76596479116327, 74.23196410079392, 74.80151881256472, 75.54366586123577, 76.18225750776666, 76.8553676216776, 77.47670003451846, 77.89092164307905, 79.08180876769072, 79.47877114256127, 80.13462202278218, 80.82499137038316, 81.44632378322403, 82.18847083189506, 82.72350707628581, 83.2067656196065, 83.67276492923715, 83.94891266827753, 84.77735588539869, 85.48498446668968, 85.88194684156024, 86.60683465654125, 86.90024162927166, 87.93579565067311, 88.43631342768381, 89.28201587849499, 89.81705212288574, 90.00690369347602, 90.62823610631689, 91.19779081808768, 91.57749395926821, 92.06075250258888, 92.47497411114946, 92.99275112185019, 93.49326889886089, 94.23541594753193, 94.7186744908526, 95.54711770797375, 96.63444943044529, 96.89333793579566, 97.65274421815671, 98.17052122885744, 98.96444597859855, 100.0], "rescued": [0, 22, 62, 84, 94, 100, 113, 118, 123, 126, 127, 129, 129, 129, 131, 133, 133, 133, 135, 135, 135, 136, 137, 137, 138, 138, 138, 138, 138, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 139, 140, 141, 141, 141, 141, 141, 141, 141, 141, 141, 142, 142, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 42, 140, 187, 260, 319, 425, 529, 599, 656, 704, 757, 830, 853, 881, 926, 955, 975, 994, 1013, 1032, 1045, 1056, 1069, 1081, 1093, 1109, 1117, 1133, 1149, 1172, 1186, 1197, 1207, 1214, 1229, 1243, 1248, 1257, 1266, 1273, 1285, 1293, 1302, 1313, 1324, 1338, 1354, 1362, 1371, 1388, 1393, 1406, 1419, 1429, 1435, 1449, 1459, 1470, 1480, 1493, 1501, 1515, 1527, 1536, 1561, 1568, 1584, 1593, 1607, 1622, 1634, 1650, 1659, 1668, 1686, 1704, 1715, 1729, 1738, 1762, 1780, 1806, 1815, 1823, 1841, 1861, 1875, 1886, 1898, 1914, 1931, 1957, 1974, 2005, 2047, 2058, 2093, 2116, 2151, 2192], "exact_score": [90.16223679668623, 90.07594062823611, 89.95512599240594, 89.81705212288574, 89.79979288919571, 89.66171901967553, 89.6444597859855, 89.33379357956507, 89.09216430790472, 88.83327580255437, 88.72972040041421, 88.59164653089402, 88.38453572661373, 88.0738695201933, 87.74594408008284, 87.34898170521228, 87.02105626510182, 86.60683465654124, 86.15809458060062, 85.74387297204004, 85.24335519502934, 84.77735588539869, 84.27683810838799, 83.75906109768727, 83.27580255436658, 82.87884017949602, 82.34380393510527, 81.75698998964445, 81.34276838108387, 80.80773213669312, 80.39351052813255, 80.01380738695202, 79.47877114256127, 79.01277183293061, 78.58129099068, 78.14981014842941, 77.73558853986883, 77.2868484639282, 76.9244045564377, 76.45840524680703, 76.09596133931653, 75.68173973075595, 75.40559199171557, 75.18122195374525, 74.90507421470487, 74.54263030721435, 74.42181567138418, 74.26648256817397, 73.99033482913359, 73.80048325854331, 73.55885398688298, 73.29996548153261, 73.0065585088022, 72.76492923714186, 72.52329996548153, 72.21263375906109, 71.91922678633068, 71.60856057991025, 71.38419054193994, 71.17707973765964, 71.0044874007594, 70.76285812909906, 70.41767345529858, 70.27959958577839, 70.08974801518812, 69.79634104245771, 69.5029340697273, 69.17500862961684, 68.96789782533655, 68.72626855367622, 68.55367621677597, 68.32930617880567, 67.98412150500516, 67.69071453227477, 67.38004832585432, 67.01760441836382, 66.72419744563341, 66.44804970659303, 66.05108733172247, 65.72316189161201, 65.36071798412151, 65.10182947877115, 64.67034863652053, 64.27338626164997, 63.84190541939938, 63.44494304452881, 62.96168450120814, 62.616499827407665, 62.16775975146703, 61.61546427338626, 61.183983431135644, 60.648947186744905, 60.148429409734206, 59.61339316534345, 59.00931998619261, 58.387987573351744, 57.68035899206075, 57.07628581290991, 56.265101829478766, 55.488436313427684, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 74.31245009746075, "threshold_area100": 69.25405591991716, "threshold_area25": 79.9326889886089}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 0.7248878149810148}}, "old_normalized_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.16223679668623, "calls": 0.7248878149810148}}, "old_normalized_endpoints": {"val": {"score": 98.91666666666666, "calls": 0.0}, "test": {"score": 90.16223679668623, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.66666666666667, 97.58333333333333, 97.58333333333333, 97.5, 97.41666666666666, 97.16666666666667, 96.83333333333333, 96.75, 96.58333333333333, 96.25, 96.0, 95.75, 95.66666666666667, 95.58333333333333, 95.5, 95.25, 95.08333333333333, 94.83333333333333, 94.58333333333333, 94.16666666666667, 93.91666666666666, 93.41666666666666, 93.16666666666666, 92.75, 92.25, 91.83333333333333, 91.49999999999999, 91.25, 90.75, 90.08333333333333, 89.5, 88.83333333333333, 88.08333333333334, 87.58333333333333, 87.25, 86.5, 85.75, 85.16666666666666, 84.58333333333333, 84.0, 83.33333333333333, 82.66666666666667, 82.0, 81.41666666666667, 80.91666666666666, 80.33333333333333, 79.5, 78.75, 78.08333333333333, 77.16666666666666, 76.25, 75.33333333333333, 74.75, 74.08333333333333, 73.25, 72.24999999999999, 71.5, 70.83333333333333, 70.0, 69.16666666666667, 68.33333333333333, 67.33333333333333, 66.5, 65.58333333333333, 64.58333333333333, 63.66666666666667, 62.74999999999999, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.99999999999999, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 30.916666666666664, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.75, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.5, 98.33333333333333, 98.16666666666667, 98.08333333333333, 98.0, 97.91666666666666, 97.83333333333333, 97.74999999999999, 97.66666666666667, 97.58333333333333, 97.58333333333333, 97.5, 97.41666666666666, 97.16666666666667, 96.83333333333333, 96.75, 96.58333333333333, 96.25, 96.0, 95.75, 95.66666666666667, 95.58333333333333, 95.5, 95.25, 95.08333333333333, 94.83333333333333, 94.58333333333333, 94.16666666666667, 93.91666666666666, 93.41666666666666, 93.16666666666666, 92.75, 92.25, 91.83333333333333, 91.49999999999999, 91.25, 90.75, 90.08333333333333, 89.5, 88.75, 88.08333333333334, 87.58333333333333, 87.25, 86.5, 85.75, 85.16666666666666, 84.58333333333333, 84.0, 83.33333333333333, 82.66666666666667, 82.0, 81.41666666666667, 80.91666666666666, 80.33333333333333, 79.5, 78.75, 78.08333333333333, 77.16666666666666, 76.25, 75.33333333333333, 74.75, 74.08333333333333, 73.25, 72.24999999999999, 71.41666666666666, 70.83333333333333, 70.0, 69.16666666666667, 68.33333333333333, 67.33333333333333, 66.5, 65.58333333333333, 64.58333333333333, 63.66666666666667, 62.74999999999999, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.99999999999999, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.56666666666666, "exact_area100": 86.5525}, "test": {"score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.09319986192614, 90.09319986192614, 90.09319986192614, 90.02416292716603, 89.97238522609597, 89.93786675871591, 89.90334829133586, 89.8688298239558, 89.7134967207456, 89.66171901967553, 89.66171901967553, 89.66171901967553, 89.54090438384536, 89.47186744908525, 89.26475664480496, 89.17846047635484, 89.12668277528478, 88.95409043838454, 88.91957197100449, 88.83327580255437, 88.74697963410424, 88.53986882982396, 88.43631342768381, 88.40179496030376, 88.33275802554367, 88.09112875388332, 88.02209181912323, 87.91853641698309, 87.72868484639281, 87.65964791163272, 87.5043148084225, 87.22816706938211, 87.12461166724196, 86.86572316189161, 86.74490852606144, 86.55505695547117, 86.33068691750086, 86.15809458060062, 85.84742837418018, 85.65757680358992, 85.53676216775975, 85.19157749395927, 84.74283741801864, 84.58750431480841, 84.39765274421815, 84.12150500517777, 83.87987573351742, 83.67276492923715, 83.32758025543666, 82.96513634794614, 82.80980324473592, 82.67172937521575, 82.41284086986538, 81.9986192613048, 81.51536071798412, 81.04936140835346, 80.58336209872282, 80.04832585433206, 79.77217811529168, 79.06454953400069, 78.54677252329995, 78.11529168104936, 77.75284777355886, 77.20055229547808, 76.7863306869175, 75.81981360027615, 75.42285122540558, 74.64618570935451, 74.04211253020365, 73.12737314463237, 72.45426303072144, 71.97100448740076, 71.40144977562996, 70.40041422160856, 69.72730410769762, 69.22678633068692, 68.01863997238522, 67.58715913013462, 67.38004832585432, 66.98308595098376, 66.430790472903, 65.63686572316189, 64.65308940283052, 62.73731446323782, 61.649982740766305, 60.683465654124944, 60.234725578184324, 59.42354159475318, 58.99206075250258, 58.40524680704176, 57.404211253020364, 56.627545736969275, 55.78184328615809, 54.81532619951675], "calls": [0.0, 0.7248878149810148, 1.6568864342423197, 2.744218156713842, 3.6071798412150504, 4.504659993096307, 5.453917846047635, 6.109768726268554, 6.920952709699689, 7.455988954090438, 8.646876078702105, 9.90680013807387, 10.959613393165343, 12.340352088367275, 12.806351397997929, 13.979979288919573, 14.791163272350708, 15.412495685191576, 16.189161201242666, 17.397307559544355, 18.139454608215395, 19.81360027614774, 20.952709699689333, 21.660338280980323, 22.833966171901967, 23.26544701415257, 23.955816361753538, 24.577148774594406, 26.09596133931654, 26.97618225750777, 27.82188470831895, 28.633068691750086, 30.117362789092166, 30.807732136693133, 32.067656196064895, 33.39661719019675, 34.173282706247846, 35.0707628581291, 36.382464618570935, 37.40075940628236, 38.40179496030377, 39.52364515015533, 40.66275457369693, 41.42216085605799, 42.267863306869174, 43.26889886089057, 44.04556437694166, 44.40800828443217, 45.96133931653434, 47.566448049706594, 48.619261304798066, 49.464963755609254, 50.20711080428028, 51.156368657231624, 52.24370037970314, 53.95236451501554, 55.33310321021747, 56.19606489471868, 56.972730410769756, 58.025543665861235, 59.14739385571281, 60.57991025198481, 61.96064894718675, 62.96168450120815, 64.27338626164997, 64.82568173973075, 66.2064204349327, 67.27649292371419, 68.22575077666552, 68.89886089057646, 69.93441491197791, 70.69382119433897, 72.36796686227132, 73.21366931308249, 74.45633413876423, 75.61270279599586, 76.90714532274767, 77.78736624093891, 78.66758715913014, 79.56506731101139, 80.91128753883328, 81.91232309285468, 82.61995167414567, 84.24231964100794, 84.81187435277874, 85.0707628581291, 85.70935450466, 86.46876078702105, 87.46979634104247, 88.7124611667242, 91.26682775284777, 92.56127027959958, 93.68312046945115, 94.16637901277183, 95.02934069727304, 95.56437694166378, 96.23748705557473, 97.25578184328616, 98.10148429409735, 98.96444597859855, 100.0], "exact_score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.07594062823611, 89.98964445978598, 89.97238522609597, 89.93786675871591, 89.92060752502589, 89.83431135657577, 89.7134967207456, 89.66171901967553, 89.66171901967553, 89.59268208491544, 89.5063859164653, 89.3683120469451, 89.23023817742491, 89.17846047635484, 89.09216430790472, 88.93683120469451, 88.79875733517432, 88.7124611667242, 88.53986882982396, 88.43631342768381, 88.40179496030376, 88.24646185709354, 88.16016568864342, 88.02209181912323, 87.97031411805315, 87.79772178115292, 87.69416637901277, 87.53883327580255, 87.33172247152227, 87.19364860200207, 86.95201933034173, 86.76216775975146, 86.67587159130134, 86.41698308595099, 86.20987228167068, 85.91646530894027, 85.65757680358992, 85.51950293406972, 85.17431826026925, 84.86365205384881, 84.70831895063858, 84.51846738004832, 84.17328270624785, 83.9316534345875, 83.74180186399724, 83.48291335864687, 83.31032102174663, 83.12046945115637, 82.7925440110459, 82.68898860890576, 82.41284086986538, 82.08491542975491, 81.68795305488436, 81.44632378322402, 81.01484294097342, 80.54884363134276, 80.15188125647221, 79.70314118053157, 79.18536416983085, 78.70210562651017, 78.16706938211944, 77.70107007248878, 77.16603382809804, 76.63099758370728, 75.99240593717639, 75.45736969278563, 74.97411114946496, 74.37003797031412, 73.81774249223334, 73.04107697618225, 72.31618916120124, 71.69485674836037, 71.0735243355195, 70.33137728684846, 69.65826717293751, 68.95063859164652, 68.2084915429755, 67.46634449430445, 66.77597514670349, 65.98205039696236, 65.22264411460131, 64.44597859855021, 63.80738695201933, 62.96168450120814, 62.150500517777004, 61.2875388332758, 60.39005868139454, 59.44080082844321, 58.62961684501208, 57.66309975837072, 56.73110113910942, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 89.7756299620297, "exact_area100": 80.51449644562257}}, "old_normalized_endpoint_budget": 0}, {"seed": 2030, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.0, "validation_area100": 0.8648999999999999, "training_seconds": 3.159104166086763, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.00619181152433157, "operator": "ge"}, {"budget": 0.02, "threshold": -0.007290798705071211, "operator": "ge"}, {"budget": 0.03, "threshold": -0.008338075131177902, "operator": "ge"}, {"budget": 0.04, "threshold": -0.009763783775269985, "operator": "ge"}, {"budget": 0.05, "threshold": -0.011446403339505196, "operator": "ge"}, {"budget": 0.06, "threshold": -0.013248167932033539, "operator": "ge"}, {"budget": 0.07, "threshold": -0.015015668235719204, "operator": "ge"}, {"budget": 0.08, "threshold": -0.017210856080055237, "operator": "ge"}, {"budget": 0.09, "threshold": -0.020268818363547325, "operator": "ge"}, {"budget": 0.1, "threshold": -0.022426800802350044, "operator": "ge"}, {"budget": 0.11, "threshold": -0.024381155148148537, "operator": "ge"}, {"budget": 0.12, "threshold": -0.02654416672885418, "operator": "ge"}, {"budget": 0.13, "threshold": -0.029038676992058754, "operator": "ge"}, {"budget": 0.14, "threshold": -0.03148255869746208, "operator": "ge"}, {"budget": 0.15, "threshold": -0.034451086074113846, "operator": "ge"}, {"budget": 0.16, "threshold": -0.0386638417840004, "operator": "ge"}, {"budget": 0.17, "threshold": -0.042844247072935104, "operator": "ge"}, {"budget": 0.18, "threshold": -0.04741759970784187, "operator": "ge"}, {"budget": 0.19, "threshold": -0.04946799576282501, "operator": "ge"}, {"budget": 0.2, "threshold": -0.054478466510772705, "operator": "ge"}, {"budget": 0.21, "threshold": -0.05948271602392197, "operator": "ge"}, {"budget": 0.22, "threshold": -0.06413965672254562, "operator": "ge"}, {"budget": 0.23, "threshold": -0.07122183591127396, "operator": "ge"}, {"budget": 0.24, "threshold": -0.07457482069730759, "operator": "ge"}, {"budget": 0.25, "threshold": -0.07976052165031433, "operator": "ge"}, {"budget": 0.26, "threshold": -0.08718205988407135, "operator": "ge"}, {"budget": 0.27, "threshold": -0.09899342805147171, "operator": "ge"}, {"budget": 0.28, "threshold": -0.1085185557603836, "operator": "ge"}, {"budget": 0.29, "threshold": -0.11284453421831131, "operator": "ge"}, {"budget": 0.3, "threshold": -0.12150177359580994, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1244988664984703, "operator": "ge"}, {"budget": 0.32, "threshold": -0.13004779815673828, "operator": "ge"}, {"budget": 0.33, "threshold": -0.13899697363376617, "operator": "ge"}, {"budget": 0.34, "threshold": -0.14809559285640717, "operator": "ge"}, {"budget": 0.35, "threshold": -0.1630883812904358, "operator": "ge"}, {"budget": 0.36, "threshold": -0.17250289022922516, "operator": "ge"}, {"budget": 0.37, "threshold": -0.18710358440876007, "operator": "ge"}, {"budget": 0.38, "threshold": -0.20939038693904877, "operator": "ge"}, {"budget": 0.39, "threshold": -0.22190967202186584, "operator": "ge"}, {"budget": 0.4, "threshold": -0.23558510839939117, "operator": "ge"}, {"budget": 0.41, "threshold": -0.25303101539611816, "operator": "ge"}, {"budget": 0.42, "threshold": -0.2634958028793335, "operator": "ge"}, {"budget": 0.43, "threshold": -0.27523526549339294, "operator": "ge"}, {"budget": 0.44, "threshold": -0.28429752588272095, "operator": "ge"}, {"budget": 0.45, "threshold": -0.29655933380126953, "operator": "ge"}, {"budget": 0.46, "threshold": -0.31311100721359253, "operator": "ge"}, {"budget": 0.47, "threshold": -0.323826402425766, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3366321325302124, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3481859564781189, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3547983467578888, "operator": "ge"}, {"budget": 0.51, "threshold": -0.36487552523612976, "operator": "ge"}, {"budget": 0.52, "threshold": -0.38014042377471924, "operator": "ge"}, {"budget": 0.53, "threshold": -0.39914369583129883, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4169025123119354, "operator": "ge"}, {"budget": 0.55, "threshold": -0.44888395071029663, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4597781002521515, "operator": "ge"}, {"budget": 0.57, "threshold": -0.47626611590385437, "operator": "ge"}, {"budget": 0.58, "threshold": -0.49381759762763977, "operator": "ge"}, {"budget": 0.59, "threshold": -0.507506787776947, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5165048241615295, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5377128720283508, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5518487691879272, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5734208226203918, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5869743227958679, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6075656414031982, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6289211511611938, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6555241346359253, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6664336323738098, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6823931336402893, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6941124796867371, "operator": "ge"}, {"budget": 0.71, "threshold": -0.709690272808075, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7265027165412903, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7432548999786377, "operator": "ge"}, {"budget": 0.74, "threshold": -0.755972146987915, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7766129970550537, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7831312417984009, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7971118688583374, "operator": "ge"}, {"budget": 0.78, "threshold": -0.807767391204834, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8190637826919556, "operator": "ge"}, {"budget": 0.8, "threshold": -0.828661322593689, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8401876091957092, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8477864861488342, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8598166704177856, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8702723383903503, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8827555179595947, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8942713141441345, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9033207297325134, "operator": "ge"}, {"budget": 0.88, "threshold": -0.909721314907074, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9190554022789001, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9250009059906006, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9325539469718933, "operator": "ge"}, {"budget": 0.92, "threshold": -0.94269198179245, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9469273090362549, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9553768634796143, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9604160785675049, "operator": "ge"}, {"budget": 0.96, "threshold": -0.965991735458374, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9712673425674438, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9742922186851501, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9795958399772644, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.0, 97.58333333333333, 97.5, 97.16666666666667, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.5, 96.33333333333333, 96.25, 96.25, 96.08333333333333, 95.91666666666666, 95.58333333333333, 95.25, 95.16666666666667, 94.83333333333333, 94.41666666666666, 94.0, 93.66666666666667, 93.33333333333333, 93.08333333333333, 92.75, 92.41666666666666, 91.83333333333333, 91.41666666666667, 91.16666666666666, 90.83333333333333, 90.25, 89.75, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.75, 87.33333333333333, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.83333333333333, 83.08333333333333, 82.41666666666666, 81.75, 81.41666666666667, 80.5, 80.08333333333333, 79.5, 78.75, 78.0, 77.41666666666667, 76.5, 75.75, 75.08333333333333, 74.25, 73.33333333333333, 72.33333333333334, 71.66666666666666, 70.75, 69.91666666666667, 69.0, 68.0, 67.16666666666666, 66.33333333333333, 65.33333333333333, 64.41666666666667, 63.5, 62.58333333333332, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 3, 4, 4, 6, 7, 7, 8, 9, 9, 9, 11, 16, 17, 21, 24, 25, 28, 29, 31, 32, 32, 34, 36, 40, 44, 45, 49, 54, 59, 63, 67, 70, 74, 78, 85, 90, 93, 97, 104, 110, 115, 121, 127, 134, 139, 148, 156, 163, 172, 181, 190, 198, 206, 210, 221, 226, 233, 242, 251, 258, 269, 278, 287, 297, 308, 320, 328, 339, 349, 360, 372, 382, 392, 404, 415, 426, 437, 448, 460, 472, 483, 494, 506, 517, 528], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.0, 97.58333333333333, 97.5, 97.16666666666667, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.5, 96.33333333333333, 96.25, 96.25, 96.08333333333333, 95.91666666666666, 95.58333333333333, 95.25, 95.16666666666667, 94.83333333333333, 94.41666666666666, 94.0, 93.66666666666667, 93.33333333333333, 93.08333333333333, 92.75, 92.41666666666666, 91.83333333333333, 91.41666666666667, 91.16666666666666, 90.83333333333333, 90.25, 89.75, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.75, 87.33333333333333, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.83333333333333, 83.08333333333333, 82.41666666666666, 81.75, 81.33333333333333, 80.5, 80.08333333333333, 79.5, 78.75, 78.0, 77.41666666666667, 76.5, 75.75, 75.08333333333333, 74.25, 73.33333333333333, 72.33333333333334, 71.58333333333333, 70.75, 69.91666666666667, 69.0, 68.0, 67.16666666666666, 66.33333333333333, 65.33333333333333, 64.41666666666667, 63.5, 62.58333333333332, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.49, "threshold_area100": 86.49166666666666, "threshold_area25": 98.64166666666667}, "test": {"score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.12771832930618, 90.11045909561615, 90.11045909561615, 90.07594062823611, 90.07594062823611, 90.02416292716603, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.90334829133586, 89.85157059026578, 89.78253365550569, 89.7134967207456, 89.66171901967553, 89.55816361753539, 89.52364515015533, 89.42008974801519, 89.3683120469451, 89.24749741111494, 89.12668277528478, 89.09216430790472, 89.0058681394546, 88.88505350362443, 88.78149810148429, 88.64342423196409, 88.53986882982396, 88.38453572661373, 88.35001725923368, 88.24646185709354, 88.14290645495339, 88.02209181912323, 87.91853641698309, 87.7114256127028, 87.43527787366241, 87.05557473248187, 86.81394546082154, 86.58957542285121, 86.19261304798067, 86.15809458060062, 86.03727994477045, 85.84742837418018, 85.60579910251984, 85.31239212978944, 85.03624439074905, 84.88091128753882, 84.72557818432861, 84.62202278218847, 84.53572661373835, 84.24231964100794, 83.77632033137729, 83.53469105971695, 83.10321021746634, 82.91335864687608, 82.53365550569555, 82.25750776665515, 82.01587849499482, 81.82602692440454, 81.56713841905419, 81.01484294097342, 80.46254746289264, 80.06558508802209, 79.47877114256127, 78.9955125992406, 78.06351397997928, 77.87366240938903, 77.4249223334484, 76.9416637901277, 76.38936831204694, 75.76803589920607, 75.18122195374525, 74.52537107352433, 73.52433551950293, 73.04107697618225, 72.52329996548153, 71.90196755264066, 71.34967207455989, 70.60752502588885, 69.77908180876769, 69.34760096651709, 68.63997238522609, 68.12219537452538, 67.25923369002416, 66.49982740766309, 65.63686572316189, 65.03279254401104, 64.10079392474974, 63.479461511908866, 62.495685191577486, 61.13220573006558, 60.787021056265104, 59.61339316534345, 58.7159130134622, 57.628581290990674, 56.972730410769756, 56.351397997928885, 55.74732481877804, 54.81532619951675], "calls": [0.0, 0.845702450811184, 1.8985157059026578, 2.5888850535036245, 3.40006903693476, 4.297549188816017, 5.281325509147393, 5.9889540904383844, 6.920952709699689, 7.83569209527097, 8.543320676561962, 9.233690024162927, 10.441836382464619, 11.580945806006214, 12.461166724197446, 13.341387642388677, 14.60131170176044, 15.79219882637211, 16.94856748360373, 17.552640662754573, 18.6399723852261, 19.571971004487402, 20.434932688988606, 21.78115291681049, 22.22989299275112, 23.16189161201243, 24.059371763893683, 25.75077666551605, 26.803589920607525, 27.355885398688294, 28.581290990680014, 28.99551259924059, 29.49603037625129, 30.583362098722816, 31.325509147393856, 32.568173973075595, 33.44839489126683, 34.673800483258546, 36.520538488091134, 37.65964791163272, 38.67794269934415, 40.04142216085606, 40.766309975837075, 41.57749395926821, 42.388677942699346, 43.19986192613048, 44.40800828443217, 45.18467380048326, 45.995857783914396, 46.858819468415604, 47.290300310666204, 47.80807732136693, 48.981705212288574, 50.172592336900244, 51.242664825681736, 53.33103210217466, 53.917846047635486, 55.125992405937176, 56.092509492578536, 56.851915774939584, 57.525025888850536, 58.64687607870211, 59.68243010010356, 60.90783569209527, 61.649982740766305, 63.25509147393855, 64.51501553331032, 66.44804970659303, 67.08664135312392, 68.2084915429755, 69.08871246116672, 70.12426648256817, 71.24611667241975, 72.54055919917157, 73.48981705212289, 75.33655505695546, 75.97514670348636, 77.06247842595789, 77.94269934414912, 78.97825336555057, 79.94477045219193, 81.10113910942354, 81.80876769071453, 82.72350707628581, 83.6382464618571, 84.86365205384881, 86.10631687953055, 87.29720400414222, 88.0738695201933, 89.3510528132551, 90.28305143251639, 91.45667932343804, 93.01001035554022, 93.40697273041076, 94.80497065930273, 95.78874697963411, 96.94511563686572, 97.68726268553675, 98.34311356575768, 98.9989644459786, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 4, 4, 4, 4, 4, 4, 4, 6, 7, 8, 8, 8, 9, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 24, 28, 37, 37, 41, 43, 44, 49, 51, 53, 56, 57, 63, 69, 72, 75, 77, 79, 80, 82, 88, 90, 98, 98, 101, 101, 108, 110, 112, 115, 116, 121, 124, 126, 130, 131, 135, 138, 138, 140, 141, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 1, 1, 1, 2, 3, 3, 5, 5, 8, 8, 8, 10, 15, 18, 22, 26, 30, 36, 38, 44, 47, 54, 61, 63, 68, 75, 84, 92, 98, 107, 109, 115, 121, 128, 136, 149, 166, 188, 202, 216, 239, 242, 251, 263, 278, 296, 313, 323, 333, 340, 346, 365, 394, 412, 446, 457, 483, 501, 516, 532, 549, 583, 618, 642, 682, 716, 773, 787, 815, 845, 878, 916, 956, 996, 1062, 1090, 1123, 1159, 1198, 1243, 1293, 1321, 1363, 1398, 1451, 1497, 1551, 1587, 1645, 1684, 1741, 1822, 1843, 1914, 1966, 2029, 2067, 2103, 2138, 2192], "exact_score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.11045909561615, 90.11045909561615, 90.07594062823611, 90.07594062823611, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.95512599240594, 89.8688298239558, 89.81705212288574, 89.74801518812563, 89.67897825336554, 89.6444597859855, 89.55816361753539, 89.45460821539524, 89.40283051432516, 89.33379357956507, 89.2129789437349, 89.12668277528478, 89.05764584052469, 88.88505350362443, 88.83327580255437, 88.76423886779426, 88.60890576458405, 88.45357266137384, 88.35001725923368, 88.19468415602347, 88.0738695201933, 87.97031411805315, 87.84949948222298, 87.60787021056264, 87.38350017259233, 87.2109078356921, 86.91750086296169, 86.69313082499137, 86.5032792544011, 86.19261304798067, 86.0717984121505, 85.95098377632033, 85.67483603727995, 85.38142906454954, 85.13979979288919, 84.88091128753882, 84.69105971694857, 84.46668967897824, 84.24231964100794, 83.82809803244736, 83.58646876078701, 83.32758025543666, 83.20676561960649, 82.87884017949602, 82.60269244045564, 82.30928546772522, 81.94684156023472, 81.68795305488436, 81.34276838108387, 80.8595098377632, 80.47980669658267, 79.996548153262, 79.59958577839143, 79.16810493614084, 78.75388332758025, 78.2878840179496, 77.89092164307905, 77.47670003451846, 76.99344149119779, 76.45840524680703, 75.90610976872627, 75.38833275802554, 74.85329651363479, 74.19744563341388, 73.71418709009319, 73.02381774249223, 72.54055919917155, 71.85018985157059, 71.33241284086986, 70.55574732481877, 69.84811874352779, 69.2095270969969, 68.45012081463582, 67.863306869175, 67.17293752157404, 66.53434587504314, 65.8612357611322, 65.08457024508112, 64.35968243010011, 63.73835001725923, 62.87538833275802, 61.97790818087676, 61.149464963755605, 60.2865032792544, 59.458060062133235, 58.50880220918191, 57.59406282361063, 56.66206420434931, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.24336204629573, "threshold_area100": 80.45460821539523, "threshold_area25": 89.7842595788747}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.00619181152433157, "operator": "ge"}, {"budget": 0.02, "threshold": -0.007290798705071211, "operator": "ge"}, {"budget": 0.03, "threshold": -0.008338075131177902, "operator": "ge"}, {"budget": 0.04, "threshold": -0.009763783775269985, "operator": "ge"}, {"budget": 0.05, "threshold": -0.011446403339505196, "operator": "ge"}, {"budget": 0.06, "threshold": -0.013248167932033539, "operator": "ge"}, {"budget": 0.07, "threshold": -0.015015668235719204, "operator": "ge"}, {"budget": 0.08, "threshold": -0.017210856080055237, "operator": "ge"}, {"budget": 0.09, "threshold": -0.020268818363547325, "operator": "ge"}, {"budget": 0.1, "threshold": -0.022426800802350044, "operator": "ge"}, {"budget": 0.11, "threshold": -0.024381155148148537, "operator": "ge"}, {"budget": 0.12, "threshold": -0.02654416672885418, "operator": "ge"}, {"budget": 0.13, "threshold": -0.029038676992058754, "operator": "ge"}, {"budget": 0.14, "threshold": -0.03148255869746208, "operator": "ge"}, {"budget": 0.15, "threshold": -0.034451086074113846, "operator": "ge"}, {"budget": 0.16, "threshold": -0.0386638417840004, "operator": "ge"}, {"budget": 0.17, "threshold": -0.042844247072935104, "operator": "ge"}, {"budget": 0.18, "threshold": -0.04741759970784187, "operator": "ge"}, {"budget": 0.19, "threshold": -0.04946799576282501, "operator": "ge"}, {"budget": 0.2, "threshold": -0.054478466510772705, "operator": "ge"}, {"budget": 0.21, "threshold": -0.05948271602392197, "operator": "ge"}, {"budget": 0.22, "threshold": -0.06413965672254562, "operator": "ge"}, {"budget": 0.23, "threshold": -0.07122183591127396, "operator": "ge"}, {"budget": 0.24, "threshold": -0.07457482069730759, "operator": "ge"}, {"budget": 0.25, "threshold": -0.07976052165031433, "operator": "ge"}, {"budget": 0.26, "threshold": -0.08718205988407135, "operator": "ge"}, {"budget": 0.27, "threshold": -0.09899342805147171, "operator": "ge"}, {"budget": 0.28, "threshold": -0.1085185557603836, "operator": "ge"}, {"budget": 0.29, "threshold": -0.11284453421831131, "operator": "ge"}, {"budget": 0.3, "threshold": -0.12150177359580994, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1244988664984703, "operator": "ge"}, {"budget": 0.32, "threshold": -0.13004779815673828, "operator": "ge"}, {"budget": 0.33, "threshold": -0.13899697363376617, "operator": "ge"}, {"budget": 0.34, "threshold": -0.14809559285640717, "operator": "ge"}, {"budget": 0.35, "threshold": -0.1630883812904358, "operator": "ge"}, {"budget": 0.36, "threshold": -0.17250289022922516, "operator": "ge"}, {"budget": 0.37, "threshold": -0.18710358440876007, "operator": "ge"}, {"budget": 0.38, "threshold": -0.20939038693904877, "operator": "ge"}, {"budget": 0.39, "threshold": -0.22190967202186584, "operator": "ge"}, {"budget": 0.4, "threshold": -0.23558510839939117, "operator": "ge"}, {"budget": 0.41, "threshold": -0.25303101539611816, "operator": "ge"}, {"budget": 0.42, "threshold": -0.2634958028793335, "operator": "ge"}, {"budget": 0.43, "threshold": -0.27523526549339294, "operator": "ge"}, {"budget": 0.44, "threshold": -0.28429752588272095, "operator": "ge"}, {"budget": 0.45, "threshold": -0.29655933380126953, "operator": "ge"}, {"budget": 0.46, "threshold": -0.31311100721359253, "operator": "ge"}, {"budget": 0.47, "threshold": -0.323826402425766, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3366321325302124, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3481859564781189, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3547983467578888, "operator": "ge"}, {"budget": 0.51, "threshold": -0.36487552523612976, "operator": "ge"}, {"budget": 0.52, "threshold": -0.38014042377471924, "operator": "ge"}, {"budget": 0.53, "threshold": -0.39914369583129883, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4169025123119354, "operator": "ge"}, {"budget": 0.55, "threshold": -0.44888395071029663, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4597781002521515, "operator": "ge"}, {"budget": 0.57, "threshold": -0.47626611590385437, "operator": "ge"}, {"budget": 0.58, "threshold": -0.49381759762763977, "operator": "ge"}, {"budget": 0.59, "threshold": -0.507506787776947, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5165048241615295, "operator": "ge"}, {"budget": 0.61, "threshold": -0.5377128720283508, "operator": "ge"}, {"budget": 0.62, "threshold": -0.5518487691879272, "operator": "ge"}, {"budget": 0.63, "threshold": -0.5734208226203918, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5869743227958679, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6075656414031982, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6289211511611938, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6555241346359253, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6664336323738098, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6823931336402893, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6941124796867371, "operator": "ge"}, {"budget": 0.71, "threshold": -0.709690272808075, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7265027165412903, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7432548999786377, "operator": "ge"}, {"budget": 0.74, "threshold": -0.755972146987915, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7766129970550537, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7831312417984009, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7971118688583374, "operator": "ge"}, {"budget": 0.78, "threshold": -0.807767391204834, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8190637826919556, "operator": "ge"}, {"budget": 0.8, "threshold": -0.828661322593689, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8401876091957092, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8477864861488342, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8598166704177856, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8702723383903503, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8827555179595947, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8942713141441345, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9033207297325134, "operator": "ge"}, {"budget": 0.88, "threshold": -0.909721314907074, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9190554022789001, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9250009059906006, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9325539469718933, "operator": "ge"}, {"budget": 0.92, "threshold": -0.94269198179245, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9469273090362549, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9553768634796143, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9604160785675049, "operator": "ge"}, {"budget": 0.96, "threshold": -0.965991735458374, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9712673425674438, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9742922186851501, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9795958399772644, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.0, 97.58333333333333, 97.5, 97.16666666666667, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.5, 96.33333333333333, 96.25, 96.25, 96.08333333333333, 95.91666666666666, 95.58333333333333, 95.25, 95.16666666666667, 94.83333333333333, 94.41666666666666, 94.0, 93.66666666666667, 93.33333333333333, 93.08333333333333, 92.75, 92.41666666666666, 91.83333333333333, 91.41666666666667, 91.16666666666666, 90.83333333333333, 90.25, 89.75, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.75, 87.33333333333333, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.83333333333333, 83.08333333333333, 82.41666666666666, 81.75, 81.41666666666667, 80.5, 80.08333333333333, 79.5, 78.75, 78.0, 77.41666666666667, 76.5, 75.75, 75.08333333333333, 74.25, 73.33333333333333, 72.33333333333334, 71.66666666666666, 70.75, 69.91666666666667, 69.0, 68.0, 67.16666666666666, 66.33333333333333, 65.33333333333333, 64.41666666666667, 63.5, 62.58333333333332, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 3, 4, 4, 6, 7, 7, 8, 9, 9, 9, 11, 16, 17, 21, 24, 25, 28, 29, 31, 32, 32, 34, 36, 40, 44, 45, 49, 54, 59, 63, 67, 70, 74, 78, 85, 90, 93, 97, 104, 110, 115, 121, 127, 134, 139, 148, 156, 163, 172, 181, 190, 198, 206, 210, 221, 226, 233, 242, 251, 258, 269, 278, 287, 297, 308, 320, 328, 339, 349, 360, 372, 382, 392, 404, 415, 426, 437, 448, 460, 472, 483, 494, 506, 517, 528], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.0, 97.58333333333333, 97.5, 97.16666666666667, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.5, 96.33333333333333, 96.25, 96.25, 96.08333333333333, 95.91666666666666, 95.58333333333333, 95.25, 95.16666666666667, 94.83333333333333, 94.41666666666666, 94.0, 93.66666666666667, 93.33333333333333, 93.08333333333333, 92.75, 92.41666666666666, 91.83333333333333, 91.41666666666667, 91.16666666666666, 90.83333333333333, 90.25, 89.75, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.75, 87.33333333333333, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.83333333333333, 83.08333333333333, 82.41666666666666, 81.75, 81.33333333333333, 80.5, 80.08333333333333, 79.5, 78.75, 78.0, 77.41666666666667, 76.5, 75.75, 75.08333333333333, 74.25, 73.33333333333333, 72.33333333333334, 71.58333333333333, 70.75, 69.91666666666667, 69.0, 68.0, 67.16666666666666, 66.33333333333333, 65.33333333333333, 64.41666666666667, 63.5, 62.58333333333332, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 86.49, "threshold_area100": 86.49166666666666, "threshold_area25": 98.64166666666667}, "test": {"score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.12771832930618, 90.11045909561615, 90.11045909561615, 90.07594062823611, 90.07594062823611, 90.02416292716603, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.90334829133586, 89.85157059026578, 89.78253365550569, 89.7134967207456, 89.66171901967553, 89.55816361753539, 89.52364515015533, 89.42008974801519, 89.3683120469451, 89.24749741111494, 89.12668277528478, 89.09216430790472, 89.0058681394546, 88.88505350362443, 88.78149810148429, 88.64342423196409, 88.53986882982396, 88.38453572661373, 88.35001725923368, 88.24646185709354, 88.14290645495339, 88.02209181912323, 87.91853641698309, 87.7114256127028, 87.43527787366241, 87.05557473248187, 86.81394546082154, 86.58957542285121, 86.19261304798067, 86.15809458060062, 86.03727994477045, 85.84742837418018, 85.60579910251984, 85.31239212978944, 85.03624439074905, 84.88091128753882, 84.72557818432861, 84.62202278218847, 84.53572661373835, 84.24231964100794, 83.77632033137729, 83.53469105971695, 83.10321021746634, 82.91335864687608, 82.53365550569555, 82.25750776665515, 82.01587849499482, 81.82602692440454, 81.56713841905419, 81.01484294097342, 80.46254746289264, 80.06558508802209, 79.47877114256127, 78.9955125992406, 78.06351397997928, 77.87366240938903, 77.4249223334484, 76.9416637901277, 76.38936831204694, 75.76803589920607, 75.18122195374525, 74.52537107352433, 73.52433551950293, 73.04107697618225, 72.52329996548153, 71.90196755264066, 71.34967207455989, 70.60752502588885, 69.77908180876769, 69.34760096651709, 68.63997238522609, 68.12219537452538, 67.25923369002416, 66.49982740766309, 65.63686572316189, 65.03279254401104, 64.10079392474974, 63.479461511908866, 62.495685191577486, 61.13220573006558, 60.787021056265104, 59.61339316534345, 58.7159130134622, 57.628581290990674, 56.972730410769756, 56.351397997928885, 55.74732481877804, 54.81532619951675], "calls": [0.0, 0.845702450811184, 1.8985157059026578, 2.5888850535036245, 3.40006903693476, 4.297549188816017, 5.281325509147393, 5.9889540904383844, 6.920952709699689, 7.83569209527097, 8.543320676561962, 9.233690024162927, 10.441836382464619, 11.580945806006214, 12.461166724197446, 13.341387642388677, 14.60131170176044, 15.79219882637211, 16.94856748360373, 17.552640662754573, 18.6399723852261, 19.571971004487402, 20.434932688988606, 21.78115291681049, 22.22989299275112, 23.16189161201243, 24.059371763893683, 25.75077666551605, 26.803589920607525, 27.355885398688294, 28.581290990680014, 28.99551259924059, 29.49603037625129, 30.583362098722816, 31.325509147393856, 32.568173973075595, 33.44839489126683, 34.673800483258546, 36.520538488091134, 37.65964791163272, 38.67794269934415, 40.04142216085606, 40.766309975837075, 41.57749395926821, 42.388677942699346, 43.19986192613048, 44.40800828443217, 45.18467380048326, 45.995857783914396, 46.858819468415604, 47.290300310666204, 47.80807732136693, 48.981705212288574, 50.172592336900244, 51.242664825681736, 53.33103210217466, 53.917846047635486, 55.125992405937176, 56.092509492578536, 56.851915774939584, 57.525025888850536, 58.64687607870211, 59.68243010010356, 60.90783569209527, 61.649982740766305, 63.25509147393855, 64.51501553331032, 66.44804970659303, 67.08664135312392, 68.2084915429755, 69.08871246116672, 70.12426648256817, 71.24611667241975, 72.54055919917157, 73.48981705212289, 75.33655505695546, 75.97514670348636, 77.06247842595789, 77.94269934414912, 78.97825336555057, 79.94477045219193, 81.10113910942354, 81.80876769071453, 82.72350707628581, 83.6382464618571, 84.86365205384881, 86.10631687953055, 87.29720400414222, 88.0738695201933, 89.3510528132551, 90.28305143251639, 91.45667932343804, 93.01001035554022, 93.40697273041076, 94.80497065930273, 95.78874697963411, 96.94511563686572, 97.68726268553675, 98.34311356575768, 98.9989644459786, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 4, 4, 4, 4, 4, 4, 4, 4, 6, 7, 8, 8, 8, 9, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 24, 28, 37, 37, 41, 43, 44, 49, 51, 53, 56, 57, 63, 69, 72, 75, 77, 79, 80, 82, 88, 90, 98, 98, 101, 101, 108, 110, 112, 115, 116, 121, 124, 126, 130, 131, 135, 138, 138, 140, 141, 144, 144, 144, 144, 144, 144, 144], "harmed": [0, 1, 1, 1, 2, 3, 3, 5, 5, 8, 8, 8, 10, 15, 18, 22, 26, 30, 36, 38, 44, 47, 54, 61, 63, 68, 75, 84, 92, 98, 107, 109, 115, 121, 128, 136, 149, 166, 188, 202, 216, 239, 242, 251, 263, 278, 296, 313, 323, 333, 340, 346, 365, 394, 412, 446, 457, 483, 501, 516, 532, 549, 583, 618, 642, 682, 716, 773, 787, 815, 845, 878, 916, 956, 996, 1062, 1090, 1123, 1159, 1198, 1243, 1293, 1321, 1363, 1398, 1451, 1497, 1551, 1587, 1645, 1684, 1741, 1822, 1843, 1914, 1966, 2029, 2067, 2103, 2138, 2192], "exact_score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.11045909561615, 90.11045909561615, 90.07594062823611, 90.07594062823611, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.95512599240594, 89.8688298239558, 89.81705212288574, 89.74801518812563, 89.67897825336554, 89.6444597859855, 89.55816361753539, 89.45460821539524, 89.40283051432516, 89.33379357956507, 89.2129789437349, 89.12668277528478, 89.05764584052469, 88.88505350362443, 88.83327580255437, 88.76423886779426, 88.60890576458405, 88.45357266137384, 88.35001725923368, 88.19468415602347, 88.0738695201933, 87.97031411805315, 87.84949948222298, 87.60787021056264, 87.38350017259233, 87.2109078356921, 86.91750086296169, 86.69313082499137, 86.5032792544011, 86.19261304798067, 86.0717984121505, 85.95098377632033, 85.67483603727995, 85.38142906454954, 85.13979979288919, 84.88091128753882, 84.69105971694857, 84.46668967897824, 84.24231964100794, 83.82809803244736, 83.58646876078701, 83.32758025543666, 83.20676561960649, 82.87884017949602, 82.60269244045564, 82.30928546772522, 81.94684156023472, 81.68795305488436, 81.34276838108387, 80.8595098377632, 80.47980669658267, 79.996548153262, 79.59958577839143, 79.16810493614084, 78.75388332758025, 78.2878840179496, 77.89092164307905, 77.47670003451846, 76.99344149119779, 76.45840524680703, 75.90610976872627, 75.38833275802554, 74.85329651363479, 74.19744563341388, 73.71418709009319, 73.02381774249223, 72.54055919917155, 71.85018985157059, 71.33241284086986, 70.55574732481877, 69.84811874352779, 69.2095270969969, 68.45012081463582, 67.863306869175, 67.17293752157404, 66.53434587504314, 65.8612357611322, 65.08457024508112, 64.35968243010011, 63.73835001725923, 62.87538833275802, 61.97790818087676, 61.149464963755605, 60.2865032792544, 59.458060062133235, 58.50880220918191, 57.59406282361063, 56.66206420434931, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 80.24336204629573, "threshold_area100": 80.45460821539523, "threshold_area25": 89.7842595788747}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -2.2285304069519043, "operator": "ge"}, {"budget": 0.02, "threshold": -3.0020294189453125, "operator": "ge"}, {"budget": 0.03, "threshold": -3.3863697052001953, "operator": "ge"}, {"budget": 0.04, "threshold": -3.7981278896331787, "operator": "ge"}, {"budget": 0.05, "threshold": -4.2803425788879395, "operator": "ge"}, {"budget": 0.06, "threshold": -4.552828311920166, "operator": "ge"}, {"budget": 0.07, "threshold": -4.845595359802246, "operator": "ge"}, {"budget": 0.08, "threshold": -5.023867130279541, "operator": "ge"}, {"budget": 0.09, "threshold": -5.220949172973633, "operator": "ge"}, {"budget": 0.1, "threshold": -5.33035945892334, "operator": "ge"}, {"budget": 0.11, "threshold": -5.562468528747559, "operator": "ge"}, {"budget": 0.12, "threshold": -5.714514255523682, "operator": "ge"}, {"budget": 0.13, "threshold": -5.952425479888916, "operator": "ge"}, {"budget": 0.14, "threshold": -6.152565002441406, "operator": "ge"}, {"budget": 0.15, "threshold": -6.313135623931885, "operator": "ge"}, {"budget": 0.16, "threshold": -6.42568302154541, "operator": "ge"}, {"budget": 0.17, "threshold": -6.604040622711182, "operator": "ge"}, {"budget": 0.18, "threshold": -6.7718186378479, "operator": "ge"}, {"budget": 0.19, "threshold": -6.848986625671387, "operator": "ge"}, {"budget": 0.2, "threshold": -6.893390655517578, "operator": "ge"}, {"budget": 0.21, "threshold": -7.002532482147217, "operator": "ge"}, {"budget": 0.22, "threshold": -7.0485992431640625, "operator": "ge"}, {"budget": 0.23, "threshold": -7.105982780456543, "operator": "ge"}, {"budget": 0.24, "threshold": -7.158512115478516, "operator": "ge"}, {"budget": 0.25, "threshold": -7.212160587310791, "operator": "ge"}, {"budget": 0.26, "threshold": -7.27382755279541, "operator": "ge"}, {"budget": 0.27, "threshold": -7.3614501953125, "operator": "ge"}, {"budget": 0.28, "threshold": -7.424917221069336, "operator": "ge"}, {"budget": 0.29, "threshold": -7.505541801452637, "operator": "ge"}, {"budget": 0.3, "threshold": -7.560840606689453, "operator": "ge"}, {"budget": 0.31, "threshold": -7.602362632751465, "operator": "ge"}, {"budget": 0.32, "threshold": -7.706550598144531, "operator": "ge"}, {"budget": 0.33, "threshold": -7.796195983886719, "operator": "ge"}, {"budget": 0.34, "threshold": -7.86275577545166, "operator": "ge"}, {"budget": 0.35, "threshold": -7.926080703735352, "operator": "ge"}, {"budget": 0.36, "threshold": -8.000301361083984, "operator": "ge"}, {"budget": 0.37, "threshold": -8.051445007324219, "operator": "ge"}, {"budget": 0.38, "threshold": -8.108687400817871, "operator": "ge"}, {"budget": 0.39, "threshold": -8.164106369018555, "operator": "ge"}, {"budget": 0.4, "threshold": -8.209956169128418, "operator": "ge"}, {"budget": 0.41, "threshold": -8.26807975769043, "operator": "ge"}, {"budget": 0.42, "threshold": -8.338956832885742, "operator": "ge"}, {"budget": 0.43, "threshold": -8.370797157287598, "operator": "ge"}, {"budget": 0.44, "threshold": -8.433926582336426, "operator": "ge"}, {"budget": 0.45, "threshold": -8.495862007141113, "operator": "ge"}, {"budget": 0.46, "threshold": -8.566483497619629, "operator": "ge"}, {"budget": 0.47, "threshold": -8.658262252807617, "operator": "ge"}, {"budget": 0.48, "threshold": -8.709386825561523, "operator": "ge"}, {"budget": 0.49, "threshold": -8.752836227416992, "operator": "ge"}, {"budget": 0.5, "threshold": -8.795503616333008, "operator": "ge"}, {"budget": 0.51, "threshold": -8.870376586914062, "operator": "ge"}, {"budget": 0.52, "threshold": -8.912891387939453, "operator": "ge"}, {"budget": 0.53, "threshold": -8.9476318359375, "operator": "ge"}, {"budget": 0.54, "threshold": -9.015677452087402, "operator": "ge"}, {"budget": 0.55, "threshold": -9.077632904052734, "operator": "ge"}, {"budget": 0.56, "threshold": -9.125749588012695, "operator": "ge"}, {"budget": 0.57, "threshold": -9.210676193237305, "operator": "ge"}, {"budget": 0.58, "threshold": -9.280244827270508, "operator": "ge"}, {"budget": 0.59, "threshold": -9.333209991455078, "operator": "ge"}, {"budget": 0.6, "threshold": -9.384072303771973, "operator": "ge"}, {"budget": 0.61, "threshold": -9.439435958862305, "operator": "ge"}, {"budget": 0.62, "threshold": -9.47788143157959, "operator": "ge"}, {"budget": 0.63, "threshold": -9.514579772949219, "operator": "ge"}, {"budget": 0.64, "threshold": -9.568933486938477, "operator": "ge"}, {"budget": 0.65, "threshold": -9.634160995483398, "operator": "ge"}, {"budget": 0.66, "threshold": -9.674182891845703, "operator": "ge"}, {"budget": 0.67, "threshold": -9.721668243408203, "operator": "ge"}, {"budget": 0.68, "threshold": -9.772406578063965, "operator": "ge"}, {"budget": 0.69, "threshold": -9.803915023803711, "operator": "ge"}, {"budget": 0.7, "threshold": -9.845270156860352, "operator": "ge"}, {"budget": 0.71, "threshold": -9.907662391662598, "operator": "ge"}, {"budget": 0.72, "threshold": -9.975290298461914, "operator": "ge"}, {"budget": 0.73, "threshold": -10.01884651184082, "operator": "ge"}, {"budget": 0.74, "threshold": -10.086421966552734, "operator": "ge"}, {"budget": 0.75, "threshold": -10.15369987487793, "operator": "ge"}, {"budget": 0.76, "threshold": -10.19869613647461, "operator": "ge"}, {"budget": 0.77, "threshold": -10.282617568969727, "operator": "ge"}, {"budget": 0.78, "threshold": -10.363945007324219, "operator": "ge"}, {"budget": 0.79, "threshold": -10.45659065246582, "operator": "ge"}, {"budget": 0.8, "threshold": -10.530123710632324, "operator": "ge"}, {"budget": 0.81, "threshold": -10.585103988647461, "operator": "ge"}, {"budget": 0.82, "threshold": -10.67477035522461, "operator": "ge"}, {"budget": 0.83, "threshold": -10.80380630493164, "operator": "ge"}, {"budget": 0.84, "threshold": -10.869378089904785, "operator": "ge"}, {"budget": 0.85, "threshold": -10.927131652832031, "operator": "ge"}, {"budget": 0.86, "threshold": -11.043790817260742, "operator": "ge"}, {"budget": 0.87, "threshold": -11.14809799194336, "operator": "ge"}, {"budget": 0.88, "threshold": -11.212828636169434, "operator": "ge"}, {"budget": 0.89, "threshold": -11.37900161743164, "operator": "ge"}, {"budget": 0.9, "threshold": -11.438210487365723, "operator": "ge"}, {"budget": 0.91, "threshold": -11.51498794555664, "operator": "ge"}, {"budget": 0.92, "threshold": -11.623661041259766, "operator": "ge"}, {"budget": 0.93, "threshold": -11.726646423339844, "operator": "ge"}, {"budget": 0.94, "threshold": -11.818840980529785, "operator": "ge"}, {"budget": 0.95, "threshold": -11.946514129638672, "operator": "ge"}, {"budget": 0.96, "threshold": -12.08578872680664, "operator": "ge"}, {"budget": 0.97, "threshold": -12.144615173339844, "operator": "ge"}, {"budget": 0.98, "threshold": -12.328662872314453, "operator": "ge"}, {"budget": 0.99, "threshold": -12.440884590148926, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.66666666666667, 98.16666666666667, 97.66666666666667, 97.33333333333333, 97.0, 96.66666666666667, 96.16666666666667, 95.83333333333333, 95.5, 95.16666666666667, 94.75, 94.5, 94.0, 93.5, 93.0, 92.75, 92.66666666666666, 92.33333333333333, 92.16666666666666, 92.25, 92.16666666666666, 92.08333333333333, 91.91666666666667, 91.66666666666666, 91.41666666666667, 91.25, 90.99999999999999, 90.91666666666667, 90.83333333333333, 90.5, 90.16666666666666, 90.16666666666666, 89.66666666666666, 89.5, 89.08333333333334, 88.5, 88.33333333333333, 87.91666666666667, 87.5, 87.0, 86.83333333333333, 86.75, 86.58333333333333, 86.41666666666666, 86.25, 86.16666666666666, 85.75, 85.33333333333333, 85.16666666666666, 84.66666666666667, 84.5, 84.33333333333333, 83.74999999999999, 83.41666666666666, 83.08333333333333, 82.5, 81.91666666666666, 81.41666666666667, 81.0, 80.41666666666667, 80.08333333333333, 79.58333333333333, 79.08333333333333, 78.58333333333333, 78.25, 77.91666666666667, 77.58333333333333, 77.16666666666666, 76.75, 76.16666666666666, 75.33333333333333, 75.0, 74.66666666666666, 74.33333333333333, 73.5, 72.83333333333333, 72.24999999999999, 71.83333333333333, 71.33333333333334, 70.66666666666667, 70.33333333333334, 69.91666666666667, 69.16666666666667, 68.66666666666667, 67.75, 66.91666666666667, 66.16666666666666, 65.5, 64.66666666666666, 63.83333333333333, 63.083333333333336, 62.16666666666666, 61.16666666666666, 60.416666666666664, 59.416666666666664, 58.416666666666664, 57.666666666666664, 56.833333333333336, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 3, 9, 15, 19, 23, 28, 34, 38, 42, 46, 51, 54, 60, 66, 72, 75, 76, 80, 82, 82, 83, 84, 86, 89, 92, 94, 97, 98, 99, 103, 107, 107, 113, 115, 120, 127, 129, 134, 139, 145, 147, 148, 150, 152, 154, 155, 160, 165, 167, 173, 175, 177, 184, 188, 192, 199, 206, 212, 217, 224, 228, 234, 240, 246, 250, 254, 258, 263, 268, 275, 285, 289, 293, 297, 307, 315, 322, 327, 333, 341, 345, 350, 359, 365, 376, 386, 395, 403, 413, 423, 432, 443, 455, 464, 476, 488, 497, 507, 517, 528], "exact_score": [98.91666666666666, 98.66666666666667, 98.16666666666667, 97.66666666666667, 97.33333333333333, 97.0, 96.66666666666667, 96.16666666666667, 95.83333333333333, 95.5, 95.16666666666667, 94.75, 94.5, 94.0, 93.5, 93.0, 92.75, 92.66666666666666, 92.33333333333333, 92.16666666666666, 92.25, 92.16666666666666, 92.08333333333333, 91.91666666666667, 91.66666666666666, 91.41666666666667, 91.25, 90.99999999999999, 90.91666666666667, 90.83333333333333, 90.5, 90.16666666666666, 90.16666666666666, 89.66666666666666, 89.5, 89.08333333333334, 88.5, 88.33333333333333, 87.91666666666667, 87.5, 87.0, 86.83333333333333, 86.75, 86.58333333333333, 86.41666666666666, 86.25, 86.16666666666666, 85.75, 85.33333333333333, 85.16666666666666, 84.66666666666667, 84.5, 84.33333333333333, 83.74999999999999, 83.41666666666666, 83.08333333333333, 82.5, 81.91666666666666, 81.41666666666667, 81.0, 80.41666666666667, 80.08333333333333, 79.58333333333333, 79.08333333333333, 78.58333333333333, 78.25, 77.91666666666667, 77.58333333333333, 77.16666666666666, 76.66666666666666, 76.16666666666666, 75.33333333333333, 75.0, 74.66666666666666, 74.33333333333333, 73.5, 72.83333333333333, 72.24999999999999, 71.83333333333333, 71.33333333333334, 70.66666666666667, 70.33333333333334, 69.91666666666667, 69.16666666666667, 68.66666666666667, 67.75, 66.91666666666667, 66.16666666666666, 65.5, 64.66666666666666, 63.83333333333333, 63.083333333333336, 62.16666666666666, 61.16666666666666, 60.416666666666664, 59.416666666666664, 58.416666666666664, 57.666666666666664, 56.833333333333336, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 81.85833333333333, "threshold_area100": 81.85916666666667, "threshold_area25": 94.51666666666667}, "test": {"score": [90.16223679668623, 89.73075595443561, 89.24749741111494, 88.74697963410424, 88.03935105281326, 87.0728339661719, 86.17535381429065, 85.26061442871936, 84.67380048325855, 84.1387642388678, 83.72454263030721, 82.82706247842594, 82.18847083189506, 81.51536071798412, 80.68691750086295, 80.03106662064205, 79.56506731101139, 78.89195719710045, 78.37418018639973, 78.11529168104936, 77.97721781152916, 77.59751467034863, 77.44218156713842, 77.26958923023818, 77.18329306178805, 77.06247842595788, 76.82084915429755, 76.441146013117, 76.23403520883673, 75.95788746979633, 75.76803589920607, 75.61270279599586, 75.19848118743528, 74.90507421470487, 74.68070417673455, 74.42181567138418, 74.19744563341388, 73.99033482913359, 73.71418709009319, 73.45529858474283, 73.31722471522264, 73.12737314463237, 72.81670693821194, 72.71315153607179, 72.48878149810149, 72.19537452537107, 71.95374525371074, 71.60856057991025, 71.38419054193994, 71.19433897134967, 71.03900586813945, 70.86641353123922, 70.72833966171902, 70.65930272695891, 70.33137728684846, 70.12426648256817, 69.98619261304798, 69.62374870555747, 69.31308249913704, 69.03693476009664, 68.74352778736623, 68.45012081463582, 68.31204694511564, 68.08767690714532, 67.9323438039351, 67.60441836382465, 67.38004832585432, 67.15567828788402, 66.96582671729375, 66.75871591301346, 66.5688643424232, 66.17190196755264, 65.7921988263721, 65.55056955471177, 65.23990334829134, 64.96375560925094, 64.7221263375906, 64.37694166379013, 64.04901622367967, 63.7210907835692, 63.4276838108388, 63.06523990334829, 62.75457369692785, 62.23679668622712, 61.87435277873662, 61.61546427338626, 61.04590956161546, 60.59716948567484, 60.26924404556438, 59.56161546427339, 59.35450465999309, 59.06109768726269, 58.7159130134622, 58.40524680704176, 58.077321366931315, 57.50776665516051, 56.9382119433897, 56.61028650327925, 56.09250949257852, 55.816361753538146, 54.81532619951675], "calls": [0.0, 3.1411805315843977, 7.317915084570245, 9.924059371763894, 13.082499137038313, 17.638936831204692, 20.245081118398343, 22.782188470831898, 24.318260269244046, 25.87159130134622, 26.907145322747674, 29.06454953400069, 30.63513979979289, 32.68898860890577, 34.70831895063859, 36.36520538488091, 37.797721781152916, 39.73075595443562, 41.7846047635485, 42.820158784949946, 43.596824301001035, 44.97756299620297, 45.61615464273386, 46.51363479461512, 47.186744908526066, 48.08422506040731, 49.10251984811874, 50.396962374870554, 51.242664825681736, 52.08836727649292, 52.76147739040387, 53.365550569554706, 54.76354849844667, 55.850880220918185, 57.024508111839836, 57.88746979634104, 58.940283051432516, 59.578874697963414, 60.49361408353469, 61.30479806696583, 61.822575077666556, 62.56472212633759, 63.66931308249914, 64.18709009319986, 65.0673110113911, 65.91301346220227, 66.96582671729375, 68.2084915429755, 68.89886089057646, 69.31308249913704, 69.89989644459786, 71.03900586813945, 71.62581981360027, 71.90196755264067, 72.76492923714187, 73.54159475319297, 73.93855712806351, 74.88781498101484, 75.85433206765619, 76.61373835001726, 77.40766309975837, 78.13255091473938, 78.52951328960994, 78.9955125992406, 79.68588194684156, 80.53158439765275, 80.98032447359337, 81.5671384190542, 82.18847083189506, 82.72350707628581, 83.27580255436658, 83.98343113565757, 84.76009665170866, 85.31239212978944, 86.0027614773904, 86.74490852606145, 87.12461166724196, 87.76320331377286, 88.36727649292372, 88.9886089057646, 89.40283051432516, 89.92060752502589, 90.3520883672765, 91.26682775284777, 91.75008629616845, 92.09527096996894, 92.80289955125993, 93.45875043148084, 93.8557128063514, 94.7186744908526, 94.96030376251295, 95.28822920262341, 95.70245081118398, 96.09941318605453, 96.46185709354505, 97.08318950638592, 97.67000345184674, 97.9979288919572, 98.6365205384881, 98.91266827752848, 100.0], "rescued": [0, 9, 32, 45, 63, 84, 89, 96, 101, 107, 111, 115, 118, 123, 126, 127, 128, 128, 130, 131, 132, 132, 132, 132, 132, 133, 133, 133, 134, 134, 134, 134, 134, 135, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 138, 138, 138, 138, 139, 139, 140, 140, 140, 140, 140, 140, 140, 140, 140, 140, 140, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 144, 144, 144], "harmed": [0, 34, 85, 127, 186, 263, 320, 380, 419, 456, 484, 540, 580, 624, 675, 714, 742, 781, 813, 829, 838, 860, 869, 879, 884, 892, 906, 928, 941, 957, 968, 977, 1001, 1019, 1033, 1048, 1061, 1073, 1089, 1104, 1112, 1123, 1141, 1147, 1160, 1177, 1191, 1213, 1226, 1237, 1246, 1257, 1265, 1270, 1289, 1301, 1309, 1330, 1348, 1364, 1381, 1398, 1406, 1419, 1429, 1448, 1461, 1474, 1485, 1497, 1508, 1531, 1553, 1567, 1585, 1603, 1617, 1637, 1656, 1675, 1692, 1713, 1731, 1761, 1782, 1797, 1830, 1856, 1875, 1916, 1928, 1945, 1965, 1983, 2002, 2035, 2068, 2087, 2118, 2134, 2192], "exact_score": [90.16223679668623, 90.1449775629962, 89.93786675871591, 89.74801518812563, 89.59268208491544, 89.57542285122541, 89.47186744908525, 89.31653434587504, 89.16120124266482, 88.91957197100449, 88.69520193303417, 88.47083189506385, 88.29823955816362, 88.03935105281326, 87.74594408008284, 87.64238867794269, 87.45253710735244, 87.24542630307214, 86.93476009665171, 86.62409389023126, 86.26164998274076, 85.91646530894027, 85.67483603727995, 85.26061442871936, 84.79461511908872, 84.48394891266827, 84.08698653779771, 83.70728339661719, 83.22402485329651, 82.84432171211597, 82.4991370383155, 82.15395236451502, 81.75698998964445, 81.34276838108387, 80.9285467725233, 80.65239903348291, 80.15188125647221, 79.8584742837418, 79.4960303762513, 79.11632723507076, 78.80566102865032, 78.54677252329995, 78.33966171901967, 78.09803244735933, 77.856403175699, 77.59751467034863, 77.37314463237833, 77.20055229547808, 77.0797376596479, 76.8553676216776, 76.54470141525715, 76.30307214359682, 75.99240593717639, 75.66448049706594, 75.4401104590956, 75.16396272005524, 74.92233344839488, 74.69796341042458, 74.40455643769417, 74.19744563341388, 73.83500172592336, 73.62789092164309, 73.26544701415256, 73.04107697618225, 72.76492923714186, 72.48878149810149, 72.17811529168104, 71.93648602002071, 71.71211598205039, 71.33241284086986, 71.03900586813945, 70.90093199861926, 70.60752502588885, 70.26234035208836, 69.96893337935795, 69.62374870555747, 69.26130479806696, 68.91612012426648, 68.51915774939592, 68.08767690714532, 67.82878840179495, 67.3627890921643, 67.01760441836382, 66.63790127718329, 66.17190196755264, 65.70590265792198, 65.23990334829134, 64.84294097342078, 64.29064549534, 63.7210907835692, 63.01346220227821, 62.37487055574732, 61.66724197445633, 60.89057645840524, 60.148429409734206, 59.31998619261305, 58.49154297549188, 57.59406282361063, 56.61028650327925, 55.73006558508802, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 76.18546866653742, "threshold_area100": 71.23507076285811, "threshold_area25": 82.48360372799446}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": -0.6878710985183716, "operator": "ge"}, {"budget": 0.02, "threshold": -1.0660183429718018, "operator": "ge"}, {"budget": 0.03, "threshold": -1.5089340209960938, "operator": "ge"}, {"budget": 0.04, "threshold": -2.018887519836426, "operator": "ge"}, {"budget": 0.05, "threshold": -2.3127264976501465, "operator": "ge"}, {"budget": 0.06, "threshold": -2.6959030628204346, "operator": "ge"}, {"budget": 0.07, "threshold": -3.1483004093170166, "operator": "ge"}, {"budget": 0.08, "threshold": -3.4298512935638428, "operator": "ge"}, {"budget": 0.09, "threshold": -3.67498517036438, "operator": "ge"}, {"budget": 0.1, "threshold": -3.867330551147461, "operator": "ge"}, {"budget": 0.11, "threshold": -4.110016345977783, "operator": "ge"}, {"budget": 0.12, "threshold": -4.299687385559082, "operator": "ge"}, {"budget": 0.13, "threshold": -4.505457401275635, "operator": "ge"}, {"budget": 0.14, "threshold": -4.763194561004639, "operator": "ge"}, {"budget": 0.15, "threshold": -4.982895851135254, "operator": "ge"}, {"budget": 0.16, "threshold": -5.135569095611572, "operator": "ge"}, {"budget": 0.17, "threshold": -5.301919937133789, "operator": "ge"}, {"budget": 0.18, "threshold": -5.5260009765625, "operator": "ge"}, {"budget": 0.19, "threshold": -5.697917461395264, "operator": "ge"}, {"budget": 0.2, "threshold": -5.789883613586426, "operator": "ge"}, {"budget": 0.21, "threshold": -5.862975120544434, "operator": "ge"}, {"budget": 0.22, "threshold": -5.943493843078613, "operator": "ge"}, {"budget": 0.23, "threshold": -6.011777400970459, "operator": "ge"}, {"budget": 0.24, "threshold": -6.117644786834717, "operator": "ge"}, {"budget": 0.25, "threshold": -6.17962646484375, "operator": "ge"}, {"budget": 0.26, "threshold": -6.239638328552246, "operator": "ge"}, {"budget": 0.27, "threshold": -6.329215049743652, "operator": "ge"}, {"budget": 0.28, "threshold": -6.421435356140137, "operator": "ge"}, {"budget": 0.29, "threshold": -6.470463752746582, "operator": "ge"}, {"budget": 0.3, "threshold": -6.563476085662842, "operator": "ge"}, {"budget": 0.31, "threshold": -6.648426055908203, "operator": "ge"}, {"budget": 0.32, "threshold": -6.730993270874023, "operator": "ge"}, {"budget": 0.33, "threshold": -6.807952880859375, "operator": "ge"}, {"budget": 0.34, "threshold": -6.929087162017822, "operator": "ge"}, {"budget": 0.35, "threshold": -7.0235066413879395, "operator": "ge"}, {"budget": 0.36, "threshold": -7.102287769317627, "operator": "ge"}, {"budget": 0.37, "threshold": -7.174023628234863, "operator": "ge"}, {"budget": 0.38, "threshold": -7.238718509674072, "operator": "ge"}, {"budget": 0.39, "threshold": -7.286537170410156, "operator": "ge"}, {"budget": 0.4, "threshold": -7.317743301391602, "operator": "ge"}, {"budget": 0.41, "threshold": -7.36616325378418, "operator": "ge"}, {"budget": 0.42, "threshold": -7.455073356628418, "operator": "ge"}, {"budget": 0.43, "threshold": -7.519470691680908, "operator": "ge"}, {"budget": 0.44, "threshold": -7.575250148773193, "operator": "ge"}, {"budget": 0.45, "threshold": -7.634824752807617, "operator": "ge"}, {"budget": 0.46, "threshold": -7.70875883102417, "operator": "ge"}, {"budget": 0.47, "threshold": -7.8051276206970215, "operator": "ge"}, {"budget": 0.48, "threshold": -7.878897666931152, "operator": "ge"}, {"budget": 0.49, "threshold": -7.935809135437012, "operator": "ge"}, {"budget": 0.5, "threshold": -7.975090980529785, "operator": "ge"}, {"budget": 0.51, "threshold": -8.02308177947998, "operator": "ge"}, {"budget": 0.52, "threshold": -8.047821044921875, "operator": "ge"}, {"budget": 0.53, "threshold": -8.15645980834961, "operator": "ge"}, {"budget": 0.54, "threshold": -8.245887756347656, "operator": "ge"}, {"budget": 0.55, "threshold": -8.287070274353027, "operator": "ge"}, {"budget": 0.56, "threshold": -8.377580642700195, "operator": "ge"}, {"budget": 0.57, "threshold": -8.407940864562988, "operator": "ge"}, {"budget": 0.58, "threshold": -8.454996109008789, "operator": "ge"}, {"budget": 0.59, "threshold": -8.558963775634766, "operator": "ge"}, {"budget": 0.6, "threshold": -8.619169235229492, "operator": "ge"}, {"budget": 0.61, "threshold": -8.664667129516602, "operator": "ge"}, {"budget": 0.62, "threshold": -8.698249816894531, "operator": "ge"}, {"budget": 0.63, "threshold": -8.755942344665527, "operator": "ge"}, {"budget": 0.64, "threshold": -8.799296379089355, "operator": "ge"}, {"budget": 0.65, "threshold": -8.836282730102539, "operator": "ge"}, {"budget": 0.66, "threshold": -8.88884162902832, "operator": "ge"}, {"budget": 0.67, "threshold": -8.93543529510498, "operator": "ge"}, {"budget": 0.68, "threshold": -8.992071151733398, "operator": "ge"}, {"budget": 0.69, "threshold": -9.071404457092285, "operator": "ge"}, {"budget": 0.7, "threshold": -9.123641967773438, "operator": "ge"}, {"budget": 0.71, "threshold": -9.190816879272461, "operator": "ge"}, {"budget": 0.72, "threshold": -9.244917869567871, "operator": "ge"}, {"budget": 0.73, "threshold": -9.315132141113281, "operator": "ge"}, {"budget": 0.74, "threshold": -9.360107421875, "operator": "ge"}, {"budget": 0.75, "threshold": -9.43863296508789, "operator": "ge"}, {"budget": 0.76, "threshold": -9.487561225891113, "operator": "ge"}, {"budget": 0.77, "threshold": -9.555824279785156, "operator": "ge"}, {"budget": 0.78, "threshold": -9.593345642089844, "operator": "ge"}, {"budget": 0.79, "threshold": -9.651384353637695, "operator": "ge"}, {"budget": 0.8, "threshold": -9.755602836608887, "operator": "ge"}, {"budget": 0.81, "threshold": -9.818059921264648, "operator": "ge"}, {"budget": 0.82, "threshold": -9.89205265045166, "operator": "ge"}, {"budget": 0.83, "threshold": -9.963532447814941, "operator": "ge"}, {"budget": 0.84, "threshold": -10.051921844482422, "operator": "ge"}, {"budget": 0.85, "threshold": -10.146818161010742, "operator": "ge"}, {"budget": 0.86, "threshold": -10.235475540161133, "operator": "ge"}, {"budget": 0.87, "threshold": -10.322894096374512, "operator": "ge"}, {"budget": 0.88, "threshold": -10.39116096496582, "operator": "ge"}, {"budget": 0.89, "threshold": -10.46475887298584, "operator": "ge"}, {"budget": 0.9, "threshold": -10.579079627990723, "operator": "ge"}, {"budget": 0.91, "threshold": -10.682514190673828, "operator": "ge"}, {"budget": 0.92, "threshold": -10.766014099121094, "operator": "ge"}, {"budget": 0.93, "threshold": -10.876904487609863, "operator": "ge"}, {"budget": 0.94, "threshold": -11.005767822265625, "operator": "ge"}, {"budget": 0.95, "threshold": -11.062943458557129, "operator": "ge"}, {"budget": 0.96, "threshold": -11.160730361938477, "operator": "ge"}, {"budget": 0.97, "threshold": -11.28989315032959, "operator": "ge"}, {"budget": 0.98, "threshold": -11.412420272827148, "operator": "ge"}, {"budget": 0.99, "threshold": -11.559720993041992, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [98.91666666666666, 98.58333333333333, 98.16666666666667, 97.74999999999999, 97.5, 96.91666666666666, 96.41666666666666, 95.91666666666666, 95.33333333333333, 95.0, 94.66666666666667, 94.08333333333333, 93.58333333333333, 93.5, 93.08333333333333, 92.75, 92.5, 92.16666666666666, 91.91666666666667, 91.75, 91.49999999999999, 91.41666666666667, 91.33333333333333, 91.25, 91.08333333333333, 90.99999999999999, 90.91666666666667, 90.75, 90.41666666666667, 90.16666666666666, 89.99999999999999, 89.83333333333333, 89.75, 89.58333333333333, 89.41666666666667, 89.08333333333334, 88.75, 88.33333333333333, 87.91666666666667, 87.5, 87.16666666666666, 86.83333333333333, 86.58333333333333, 86.41666666666666, 86.33333333333333, 86.16666666666666, 85.83333333333333, 85.5, 85.16666666666666, 84.83333333333333, 84.58333333333333, 84.25, 84.0, 83.74999999999999, 83.16666666666667, 82.66666666666667, 82.16666666666667, 81.83333333333334, 81.0, 80.33333333333333, 80.16666666666666, 79.66666666666666, 79.41666666666667, 79.08333333333333, 78.41666666666667, 78.0, 77.41666666666667, 76.91666666666667, 76.58333333333334, 76.33333333333333, 76.0, 75.41666666666667, 75.0, 74.25, 73.83333333333333, 73.25, 72.75, 72.0, 71.5, 71.0, 70.58333333333333, 70.0, 69.33333333333334, 68.75, 68.25, 67.58333333333333, 66.75, 65.99999999999999, 65.08333333333334, 64.58333333333333, 63.66666666666667, 62.83333333333333, 62.083333333333336, 61.25000000000001, 60.25, 59.416666666666664, 58.5, 57.49999999999999, 56.666666666666664, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "rescued": [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2], "harmed": [0, 4, 9, 14, 17, 24, 31, 37, 44, 48, 52, 59, 65, 67, 72, 76, 79, 83, 86, 88, 91, 92, 93, 94, 96, 97, 98, 100, 104, 107, 109, 111, 112, 114, 116, 120, 124, 129, 134, 139, 143, 147, 150, 152, 153, 155, 159, 163, 167, 171, 174, 178, 181, 184, 191, 197, 203, 207, 217, 225, 227, 233, 236, 240, 248, 253, 260, 266, 270, 273, 277, 284, 289, 298, 303, 310, 316, 325, 331, 337, 342, 349, 357, 364, 370, 378, 388, 397, 408, 414, 425, 435, 444, 454, 466, 476, 487, 499, 509, 517, 528], "exact_score": [98.91666666666666, 98.58333333333333, 98.16666666666667, 97.74999999999999, 97.5, 96.91666666666666, 96.41666666666666, 95.91666666666666, 95.33333333333333, 95.0, 94.66666666666667, 94.08333333333333, 93.58333333333333, 93.5, 93.08333333333333, 92.75, 92.5, 92.16666666666666, 91.91666666666667, 91.75, 91.49999999999999, 91.41666666666667, 91.33333333333333, 91.25, 91.08333333333333, 90.99999999999999, 90.91666666666667, 90.75, 90.41666666666667, 90.16666666666666, 89.99999999999999, 89.83333333333333, 89.75, 89.58333333333333, 89.41666666666667, 89.08333333333334, 88.75, 88.33333333333333, 87.91666666666667, 87.5, 87.16666666666666, 86.83333333333333, 86.58333333333333, 86.41666666666666, 86.33333333333333, 86.16666666666666, 85.83333333333333, 85.5, 85.16666666666666, 84.83333333333333, 84.58333333333333, 84.25, 84.0, 83.74999999999999, 83.16666666666667, 82.66666666666667, 82.16666666666667, 81.75, 81.0, 80.33333333333333, 80.16666666666666, 79.66666666666666, 79.41666666666667, 79.08333333333333, 78.41666666666667, 78.0, 77.41666666666667, 76.91666666666667, 76.58333333333334, 76.33333333333333, 76.0, 75.41666666666667, 75.0, 74.25, 73.83333333333333, 73.25, 72.75, 72.0, 71.5, 71.0, 70.58333333333333, 70.0, 69.24999999999999, 68.75, 68.25, 67.58333333333333, 66.75, 65.99999999999999, 65.08333333333334, 64.58333333333333, 63.66666666666667, 62.83333333333333, 62.083333333333336, 61.25000000000001, 60.25, 59.416666666666664, 58.5, 57.49999999999999, 56.666666666666664, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_area100": 81.59083333333334, "threshold_area100": 81.5925, "threshold_area25": 94.15833333333333}, "test": {"score": [90.16223679668623, 89.69623748705557, 89.38557128063513, 88.67794269934414, 87.65964791163272, 86.72764929237141, 85.5022437003797, 83.98343113565757, 83.01691404901622, 82.30928546772522, 81.67069382119433, 80.84225060407317, 80.3589920607525, 79.75491888160165, 79.13358646876078, 78.51225405591991, 78.27062478425958, 77.99447704521918, 77.5802554366586, 77.14877459440801, 76.95892302381773, 76.80358992060752, 76.70003451846738, 76.49292371418709, 76.18225750776665, 75.99240593717639, 75.80255436658612, 75.716258198136, 75.42285122540558, 75.26751812219537, 75.18122195374525, 75.06040731791508, 74.83603727994476, 74.61166724197446, 74.37003797031412, 74.09389023127373, 73.69692785640318, 73.48981705212289, 73.26544701415256, 73.0755954435623, 72.95478080773213, 72.73041076976182, 72.40248532965136, 72.09181912323092, 71.97100448740076, 71.83293061788056, 71.53952364515015, 71.36693130824992, 71.10804280289955, 70.90093199861926, 70.74559889540903, 70.65930272695891, 70.59026579219882, 70.24508111839833, 70.10700724887815, 70.03797031411804, 69.60648947186745, 69.43389713496721, 69.26130479806696, 68.9333793579565, 68.7090093199862, 68.46738004832585, 68.32930617880567, 68.10493614083535, 67.863306869175, 67.75975146703486, 67.53538142906454, 67.29375215740421, 67.00034518467379, 66.430790472903, 66.22367966862271, 65.82671729375215, 65.51605108733172, 65.15360717984122, 64.94649637556091, 64.41146013117017, 64.18709009319986, 63.89368312046945, 63.703831549879176, 63.479461511908866, 62.94442526751811, 62.65101829478771, 62.323092854677256, 62.11598205039696, 61.70176044183637, 61.235761132205724, 60.70072488781497, 60.32102174663445, 60.04487400759406, 59.69968933379357, 59.233690024162925, 58.81946841560235, 58.612357611322054, 58.301691404901625, 57.697618225750766, 57.4387297204004, 56.90369347600965, 56.40317569899895, 56.10976872626855, 55.59199171556782, 54.81532619951675], "calls": [0.0, 3.935105281325509, 6.454953400069037, 9.596133931653435, 13.859164653089403, 16.845012081463583, 20.521228857438732, 24.404556437694165, 26.924404556437693, 28.460476354849845, 30.013807386952017, 31.808767690714536, 32.87884017949603, 34.1387642388678, 35.881946841560236, 37.41801863997238, 38.35001725923369, 39.62720055229548, 41.698308595098375, 43.320676561960646, 44.3562305833621, 45.20193303417328, 46.013117017604415, 46.9278564031757, 48.46392820158785, 49.188816016568865, 49.86192613047981, 50.673110113910944, 51.81221953745254, 52.39903348291336, 53.34829133586469, 54.17673455298585, 55.10873317224715, 55.850880220918185, 57.14532274767, 58.26717293752157, 59.31998619261305, 60.13117017604418, 60.925094925785295, 61.63272350707628, 61.9951674145668, 62.58198136002762, 63.7901277183293, 64.86020020711081, 65.4987918536417, 66.18916120124267, 67.05212288574387, 68.19123230928547, 69.17500862961684, 69.91715567828788, 70.41767345529858, 70.91819123230928, 71.21159820503969, 72.47152226441146, 73.0755954435623, 73.48981705212289, 74.62892647566449, 74.95685191577493, 75.35381429064549, 76.59647911632723, 77.26958923023818, 77.856403175699, 78.37418018639973, 78.96099413186055, 79.51328960994132, 79.91025198481188, 80.39351052813255, 80.98032447359337, 81.67069382119433, 82.84432171211598, 83.41387642388678, 84.29409734207802, 84.94994822229893, 85.83016914049017, 86.22713151536072, 87.02105626510182, 87.46979634104247, 88.16016568864342, 88.4880911287539, 88.90231273731446, 89.81705212288574, 90.2140144977563, 90.68001380738696, 91.04245771487746, 91.6465308940283, 92.37141870900932, 93.04452882292026, 93.59682430100104, 94.02830514325164, 94.44252675181221, 95.01208146358302, 95.51259924059372, 95.77148774594409, 96.18570935450465, 96.82430100103555, 97.13496720745599, 97.70452191922678, 98.23955816361753, 98.55022437003798, 99.1370383154988, 100.0], "rescued": [0, 15, 29, 45, 75, 87, 98, 109, 116, 118, 121, 124, 125, 125, 129, 130, 131, 132, 132, 132, 132, 132, 132, 132, 133, 133, 133, 133, 133, 134, 134, 134, 134, 135, 136, 137, 137, 137, 137, 137, 137, 137, 137, 138, 138, 138, 138, 139, 139, 139, 139, 139, 139, 139, 139, 139, 140, 140, 140, 140, 140, 140, 140, 140, 141, 141, 141, 141, 141, 141, 141, 141, 141, 141, 142, 142, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, 144, 144], "harmed": [0, 42, 74, 131, 220, 286, 368, 467, 530, 573, 613, 664, 693, 728, 768, 805, 820, 837, 861, 886, 897, 906, 912, 924, 943, 954, 965, 970, 987, 997, 1002, 1009, 1022, 1036, 1051, 1068, 1091, 1103, 1116, 1127, 1134, 1147, 1166, 1185, 1192, 1200, 1217, 1228, 1243, 1255, 1264, 1269, 1273, 1293, 1301, 1305, 1331, 1341, 1351, 1370, 1383, 1397, 1405, 1418, 1433, 1439, 1452, 1466, 1483, 1516, 1528, 1551, 1569, 1590, 1603, 1634, 1648, 1665, 1676, 1689, 1720, 1737, 1756, 1768, 1792, 1819, 1850, 1872, 1888, 1908, 1935, 1959, 1971, 1989, 2024, 2039, 2070, 2099, 2116, 2147, 2192], "exact_score": [90.16223679668623, 90.09319986192614, 89.98964445978598, 89.79979288919571, 89.67897825336554, 89.54090438384536, 89.48912668277528, 89.33379357956507, 89.10942354159475, 88.83327580255437, 88.72972040041421, 88.38453572661373, 88.24646185709354, 87.84949948222298, 87.62512944425266, 87.19364860200207, 86.93476009665171, 86.65861235761132, 86.34794615119088, 86.05453917846047, 85.58853986882981, 85.39868829823956, 85.07076285812909, 84.60476354849844, 84.1387642388678, 83.70728339661719, 83.2930617880566, 82.98239558163617, 82.46461857093544, 82.06765619606489, 81.68795305488436, 81.18743527787366, 80.77321366931308, 80.2899551259924, 79.82395581636175, 79.40973420780118, 79.11632723507076, 78.66758715913014, 78.33966171901967, 78.13255091473938, 77.856403175699, 77.66655160510874, 77.51121850189851, 77.25232999654816, 77.02795995857782, 76.82084915429755, 76.70003451846738, 76.49292371418709, 76.23403520883673, 76.06144287193648, 75.7852951328961, 75.62996202968588, 75.40559199171557, 75.19848118743528, 75.0776665516051, 74.87055574732481, 74.59440800828443, 74.40455643769417, 74.2147048671039, 73.85226095961339, 73.52433551950293, 73.24818778046254, 72.95478080773213, 72.6441146013117, 72.29892992751121, 72.04004142216085, 71.83293061788056, 71.57404211253021, 71.38419054193994, 71.1425612702796, 70.86641353123922, 70.65930272695891, 70.40041422160856, 70.12426648256817, 69.81360027614774, 69.39937866758716, 69.05419399378667, 68.82982395581637, 68.41560234725577, 68.08767690714532, 67.7252329996548, 67.27649292371419, 66.8622713151536, 66.34449430445288, 66.01656886434242, 65.51605108733172, 65.03279254401104, 64.4287193648602, 63.96272005522954, 63.4276838108388, 62.80635139799793, 62.133241284086985, 61.52916810493614, 60.75250258888505, 60.06213324128409, 59.250949257852945, 58.45702450811183, 57.57680358992061, 56.61028650327925, 55.73006558508802, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "exact_area100": 75.77305519654259, "threshold_area100": 70.75940628236104, "threshold_area25": 81.38936831204695}, "selected_budget": 0, "selected": {"val": {"score": 98.91666666666666, "calls": 0.0, "rescued": 0, "harmed": 0}, "test": {"score": 90.16223679668623, "calls": 0.0, "rescued": 0, "harmed": 0}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.1449775629962, "calls": 0.845702450811184}}, "old_normalized_interior": {"val": {"score": 98.91666666666666, "calls": 1.0}, "test": {"score": 90.1449775629962, "calls": 0.845702450811184}}, "old_normalized_endpoints": {"val": {"score": 98.91666666666666, "calls": 0.0}, "test": {"score": 90.16223679668623, "calls": 0.0}}}, "baseline_old_normalized": {"val": {"score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.0, 97.58333333333333, 97.5, 97.16666666666667, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.5, 96.33333333333333, 96.25, 96.25, 96.08333333333333, 95.91666666666666, 95.58333333333333, 95.25, 95.16666666666667, 94.83333333333333, 94.41666666666666, 94.0, 93.66666666666667, 93.33333333333333, 93.08333333333333, 92.75, 92.41666666666666, 91.83333333333333, 91.41666666666667, 91.16666666666666, 90.83333333333333, 90.25, 89.75, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.75, 87.33333333333333, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.83333333333333, 83.08333333333333, 82.41666666666666, 81.75, 81.41666666666667, 80.5, 80.08333333333333, 79.5, 78.75, 78.0, 77.41666666666667, 76.5, 75.75, 75.08333333333333, 74.25, 73.33333333333333, 72.33333333333334, 71.66666666666666, 70.75, 69.91666666666667, 69.0, 68.0, 67.16666666666666, 66.33333333333333, 65.33333333333333, 64.41666666666667, 63.5, 62.58333333333332, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 40.91666666666667, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.91666666666667, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 68.91666666666667, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 81.91666666666667, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "exact_score": [98.91666666666666, 98.91666666666666, 98.91666666666666, 98.91666666666666, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.83333333333333, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.66666666666667, 98.58333333333333, 98.58333333333333, 98.41666666666666, 98.33333333333333, 98.33333333333333, 98.25, 98.16666666666667, 98.16666666666667, 98.16666666666667, 98.0, 97.58333333333333, 97.5, 97.16666666666667, 96.91666666666666, 96.83333333333333, 96.58333333333333, 96.5, 96.33333333333333, 96.25, 96.25, 96.08333333333333, 95.91666666666666, 95.58333333333333, 95.25, 95.16666666666667, 94.83333333333333, 94.41666666666666, 94.0, 93.66666666666667, 93.33333333333333, 93.08333333333333, 92.75, 92.41666666666666, 91.83333333333333, 91.41666666666667, 91.16666666666666, 90.83333333333333, 90.25, 89.75, 89.33333333333333, 88.83333333333333, 88.33333333333333, 87.75, 87.33333333333333, 86.58333333333333, 85.91666666666666, 85.33333333333333, 84.58333333333333, 83.83333333333333, 83.08333333333333, 82.41666666666666, 81.75, 81.33333333333333, 80.5, 80.08333333333333, 79.5, 78.75, 78.0, 77.41666666666667, 76.5, 75.75, 75.08333333333333, 74.25, 73.33333333333333, 72.33333333333334, 71.58333333333333, 70.75, 69.91666666666667, 69.0, 68.0, 67.16666666666666, 66.33333333333333, 65.33333333333333, 64.41666666666667, 63.5, 62.58333333333332, 61.74999999999999, 60.74999999999999, 59.74999999999999, 58.833333333333336, 57.91666666666666, 56.91666666666666, 56.00000000000001, 55.08333333333333], "exact_calls": [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.000000000000001, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.000000000000002, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.000000000000004, 28.999999999999996, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.00000000000001, 56.00000000000001, 56.99999999999999, 57.99999999999999, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0], "threshold_area25": 98.64166666666667, "exact_area100": 86.49}, "test": {"score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.12771832930618, 90.11045909561615, 90.11045909561615, 90.07594062823611, 90.07594062823611, 90.02416292716603, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.90334829133586, 89.85157059026578, 89.78253365550569, 89.7134967207456, 89.66171901967553, 89.55816361753539, 89.52364515015533, 89.42008974801519, 89.3683120469451, 89.24749741111494, 89.12668277528478, 89.09216430790472, 89.0058681394546, 88.88505350362443, 88.78149810148429, 88.64342423196409, 88.53986882982396, 88.38453572661373, 88.35001725923368, 88.24646185709354, 88.14290645495339, 88.02209181912323, 87.91853641698309, 87.7114256127028, 87.43527787366241, 87.05557473248187, 86.81394546082154, 86.58957542285121, 86.19261304798067, 86.15809458060062, 86.03727994477045, 85.84742837418018, 85.60579910251984, 85.31239212978944, 85.03624439074905, 84.88091128753882, 84.72557818432861, 84.62202278218847, 84.53572661373835, 84.24231964100794, 83.77632033137729, 83.53469105971695, 83.10321021746634, 82.91335864687608, 82.53365550569555, 82.25750776665515, 82.01587849499482, 81.82602692440454, 81.56713841905419, 81.01484294097342, 80.46254746289264, 80.06558508802209, 79.47877114256127, 78.9955125992406, 78.06351397997928, 77.87366240938903, 77.4249223334484, 76.9416637901277, 76.38936831204694, 75.76803589920607, 75.18122195374525, 74.52537107352433, 73.52433551950293, 73.04107697618225, 72.52329996548153, 71.90196755264066, 71.34967207455989, 70.60752502588885, 69.77908180876769, 69.34760096651709, 68.63997238522609, 68.12219537452538, 67.25923369002416, 66.49982740766309, 65.63686572316189, 65.03279254401104, 64.10079392474974, 63.479461511908866, 62.495685191577486, 61.13220573006558, 60.787021056265104, 59.61339316534345, 58.7159130134622, 57.628581290990674, 56.972730410769756, 56.351397997928885, 55.74732481877804, 54.81532619951675], "calls": [0.0, 0.845702450811184, 1.8985157059026578, 2.5888850535036245, 3.40006903693476, 4.297549188816017, 5.281325509147393, 5.9889540904383844, 6.920952709699689, 7.83569209527097, 8.543320676561962, 9.233690024162927, 10.441836382464619, 11.580945806006214, 12.461166724197446, 13.341387642388677, 14.60131170176044, 15.79219882637211, 16.94856748360373, 17.552640662754573, 18.6399723852261, 19.571971004487402, 20.434932688988606, 21.78115291681049, 22.22989299275112, 23.16189161201243, 24.059371763893683, 25.75077666551605, 26.803589920607525, 27.355885398688294, 28.581290990680014, 28.99551259924059, 29.49603037625129, 30.583362098722816, 31.325509147393856, 32.568173973075595, 33.44839489126683, 34.673800483258546, 36.520538488091134, 37.65964791163272, 38.67794269934415, 40.04142216085606, 40.766309975837075, 41.57749395926821, 42.388677942699346, 43.19986192613048, 44.40800828443217, 45.18467380048326, 45.995857783914396, 46.858819468415604, 47.290300310666204, 47.80807732136693, 48.981705212288574, 50.172592336900244, 51.242664825681736, 53.33103210217466, 53.917846047635486, 55.125992405937176, 56.092509492578536, 56.851915774939584, 57.525025888850536, 58.64687607870211, 59.68243010010356, 60.90783569209527, 61.649982740766305, 63.25509147393855, 64.51501553331032, 66.44804970659303, 67.08664135312392, 68.2084915429755, 69.08871246116672, 70.12426648256817, 71.24611667241975, 72.54055919917157, 73.48981705212289, 75.33655505695546, 75.97514670348636, 77.06247842595789, 77.94269934414912, 78.97825336555057, 79.94477045219193, 81.10113910942354, 81.80876769071453, 82.72350707628581, 83.6382464618571, 84.86365205384881, 86.10631687953055, 87.29720400414222, 88.0738695201933, 89.3510528132551, 90.28305143251639, 91.45667932343804, 93.01001035554022, 93.40697273041076, 94.80497065930273, 95.78874697963411, 96.94511563686572, 97.68726268553675, 98.34311356575768, 98.9989644459786, 100.0], "exact_score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.11045909561615, 90.11045909561615, 90.07594062823611, 90.07594062823611, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.95512599240594, 89.8688298239558, 89.81705212288574, 89.74801518812563, 89.67897825336554, 89.6444597859855, 89.55816361753539, 89.45460821539524, 89.40283051432516, 89.33379357956507, 89.2129789437349, 89.12668277528478, 89.05764584052469, 88.88505350362443, 88.83327580255437, 88.76423886779426, 88.60890576458405, 88.45357266137384, 88.35001725923368, 88.19468415602347, 88.0738695201933, 87.97031411805315, 87.84949948222298, 87.60787021056264, 87.38350017259233, 87.2109078356921, 86.91750086296169, 86.69313082499137, 86.5032792544011, 86.19261304798067, 86.0717984121505, 85.95098377632033, 85.67483603727995, 85.38142906454954, 85.13979979288919, 84.88091128753882, 84.69105971694857, 84.46668967897824, 84.24231964100794, 83.82809803244736, 83.58646876078701, 83.32758025543666, 83.20676561960649, 82.87884017949602, 82.60269244045564, 82.30928546772522, 81.94684156023472, 81.68795305488436, 81.34276838108387, 80.8595098377632, 80.47980669658267, 79.996548153262, 79.59958577839143, 79.16810493614084, 78.75388332758025, 78.2878840179496, 77.89092164307905, 77.47670003451846, 76.99344149119779, 76.45840524680703, 75.90610976872627, 75.38833275802554, 74.85329651363479, 74.19744563341388, 73.71418709009319, 73.02381774249223, 72.54055919917155, 71.85018985157059, 71.33241284086986, 70.55574732481877, 69.84811874352779, 69.2095270969969, 68.45012081463582, 67.863306869175, 67.17293752157404, 66.53434587504314, 65.8612357611322, 65.08457024508112, 64.35968243010011, 63.73835001725923, 62.87538833275802, 61.97790818087676, 61.149464963755605, 60.2865032792544, 59.458060062133235, 58.50880220918191, 57.59406282361063, 56.66206420434931, 55.74732481877804, 54.81532619951675], "exact_calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "threshold_area25": 89.7842595788747, "exact_area100": 80.24336204629573}}, "old_normalized_endpoint_budget": 0}]}, {"expert": "groundingdino", "endpoint": "MiniCPM", "task": "grefcoco", "small": 26.688029946402747, "large": 50.00708958398321, "runs": [{"seed": 2026, "selection": {"best_epoch": 16, "epochs": 24, "alpha": 0.0, "validation_area100": 0.6423162746884526, "training_seconds": 179.46991389698815, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.947008490562439, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9047155380249023, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8676165342330933, "operator": "ge"}, {"budget": 0.04, "threshold": 0.829997181892395, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7986266613006592, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7661951780319214, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7346175312995911, "operator": "ge"}, {"budget": 0.08, "threshold": 0.7087351083755493, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6854745149612427, "operator": "ge"}, {"budget": 0.1, "threshold": 0.661552369594574, "operator": "ge"}, {"budget": 0.11, "threshold": 0.636324942111969, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6168317198753357, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5935205221176147, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5725961327552795, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5545302033424377, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5375873446464539, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5206890106201172, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5042575001716614, "operator": "ge"}, {"budget": 0.19, "threshold": 0.4901709258556366, "operator": "ge"}, {"budget": 0.2, "threshold": 0.47479119896888733, "operator": "ge"}, {"budget": 0.21, "threshold": 0.45752084255218506, "operator": "ge"}, {"budget": 0.22, "threshold": 0.44289904832839966, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4254084825515747, "operator": "ge"}, {"budget": 0.24, "threshold": 0.4092045724391937, "operator": "ge"}, {"budget": 0.25, "threshold": 0.3946588337421417, "operator": "ge"}, {"budget": 0.26, "threshold": 0.3831115663051605, "operator": "ge"}, {"budget": 0.27, "threshold": 0.37004461884498596, "operator": "ge"}, {"budget": 0.28, "threshold": 0.35690614581108093, "operator": "ge"}, {"budget": 0.29, "threshold": 0.344345360994339, "operator": "ge"}, {"budget": 0.3, "threshold": 0.3312433063983917, "operator": "ge"}, {"budget": 0.31, "threshold": 0.3194664716720581, "operator": "gt"}, {"budget": 0.32, "threshold": 0.30649492144584656, "operator": "ge"}, {"budget": 0.33, "threshold": 0.2935950756072998, "operator": "ge"}, {"budget": 0.34, "threshold": 0.28041309118270874, "operator": "ge"}, {"budget": 0.35, "threshold": 0.27006345987319946, "operator": "ge"}, {"budget": 0.36, "threshold": 0.2593194544315338, "operator": "ge"}, {"budget": 0.37, "threshold": 0.24810585379600525, "operator": "ge"}, {"budget": 0.38, "threshold": 0.2352842092514038, "operator": "ge"}, {"budget": 0.39, "threshold": 0.2235669046640396, "operator": "ge"}, {"budget": 0.4, "threshold": 0.21322877705097198, "operator": "ge"}, {"budget": 0.41, "threshold": 0.202176034450531, "operator": "gt"}, {"budget": 0.42, "threshold": 0.19243107736110687, "operator": "ge"}, {"budget": 0.43, "threshold": 0.18201598525047302, "operator": "ge"}, {"budget": 0.44, "threshold": 0.17333270609378815, "operator": "ge"}, {"budget": 0.45, "threshold": 0.16453661024570465, "operator": "ge"}, {"budget": 0.46, "threshold": 0.15539170801639557, "operator": "ge"}, {"budget": 0.47, "threshold": 0.1462712585926056, "operator": "ge"}, {"budget": 0.48, "threshold": 0.1383156180381775, "operator": "ge"}, {"budget": 0.49, "threshold": 0.13048739731311798, "operator": "ge"}, {"budget": 0.5, "threshold": 0.12159770727157593, "operator": "ge"}, {"budget": 0.51, "threshold": 0.11353585124015808, "operator": "ge"}, {"budget": 0.52, "threshold": 0.10398432612419128, "operator": "ge"}, {"budget": 0.53, "threshold": 0.09489816427230835, "operator": "ge"}, {"budget": 0.54, "threshold": 0.08641988784074783, "operator": "ge"}, {"budget": 0.55, "threshold": 0.07751017808914185, "operator": "ge"}, {"budget": 0.56, "threshold": 0.06917332112789154, "operator": "ge"}, {"budget": 0.57, "threshold": 0.061869435012340546, "operator": "ge"}, {"budget": 0.58, "threshold": 0.05425605922937393, "operator": "ge"}, {"budget": 0.59, "threshold": 0.046554844826459885, "operator": "ge"}, {"budget": 0.6, "threshold": 0.038441311568021774, "operator": "ge"}, {"budget": 0.61, "threshold": 0.0312487855553627, "operator": "ge"}, {"budget": 0.62, "threshold": 0.02472321316599846, "operator": "ge"}, {"budget": 0.63, "threshold": 0.01812935620546341, "operator": "ge"}, {"budget": 0.64, "threshold": 0.01187086757272482, "operator": "gt"}, {"budget": 0.65, "threshold": 0.006847094744443893, "operator": "ge"}, {"budget": 0.66, "threshold": 0.0017002517124637961, "operator": "ge"}, {"budget": 0.67, "threshold": -2.6931804313790053e-05, "operator": "ge"}, {"budget": 0.68, "threshold": -0.00012182118371129036, "operator": "ge"}, {"budget": 0.69, "threshold": -0.0002286457602167502, "operator": "ge"}, {"budget": 0.7, "threshold": -0.00039261888014152646, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0005824727704748511, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0008697039447724819, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0012221806682646275, "operator": "ge"}, {"budget": 0.74, "threshold": -0.001637571956962347, "operator": "ge"}, {"budget": 0.75, "threshold": -0.0021878425031900406, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0028185085393488407, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0034966126549988985, "operator": "ge"}, {"budget": 0.78, "threshold": -0.0044568623416125774, "operator": "ge"}, {"budget": 0.79, "threshold": -0.005563683807849884, "operator": "ge"}, {"budget": 0.8, "threshold": -0.00683121383190155, "operator": "ge"}, {"budget": 0.81, "threshold": -0.008126546628773212, "operator": "ge"}, {"budget": 0.82, "threshold": -0.009987862780690193, "operator": "ge"}, {"budget": 0.83, "threshold": -0.012347264215350151, "operator": "ge"}, {"budget": 0.84, "threshold": -0.015440721996128559, "operator": "ge"}, {"budget": 0.85, "threshold": -0.018446899950504303, "operator": "ge"}, {"budget": 0.86, "threshold": -0.02264130488038063, "operator": "ge"}, {"budget": 0.87, "threshold": -0.02818317897617817, "operator": "ge"}, {"budget": 0.88, "threshold": -0.03458961844444275, "operator": "ge"}, {"budget": 0.89, "threshold": -0.042335160076618195, "operator": "ge"}, {"budget": 0.9, "threshold": -0.052433937788009644, "operator": "ge"}, {"budget": 0.91, "threshold": -0.06523693352937698, "operator": "ge"}, {"budget": 0.92, "threshold": -0.08252975344657898, "operator": "ge"}, {"budget": 0.93, "threshold": -0.10367166250944138, "operator": "ge"}, {"budget": 0.94, "threshold": -0.12893329560756683, "operator": "ge"}, {"budget": 0.95, "threshold": -0.16521666944026947, "operator": "ge"}, {"budget": 0.96, "threshold": -0.2091291844844818, "operator": "ge"}, {"budget": 0.97, "threshold": -0.2669602930545807, "operator": "ge"}, {"budget": 0.98, "threshold": -0.3632992208003998, "operator": "ge"}, {"budget": 0.99, "threshold": -0.49766337871551514, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.637852273525894, 57.26333544170355, 57.82556750298685, 58.247241548949326, 58.661887694145754, 59.04842223627802, 59.43495677841029, 59.79337971747838, 60.172886358844615, 60.58753250404104, 60.90378803851289, 61.19895987068662, 61.45196429826411, 61.64171761894721, 61.8947220465247, 62.17583807716635, 62.43587040550986, 62.660763230023186, 62.94890716143088, 63.208939489774394, 63.46194391735188, 63.63061353573687, 63.82739475718602, 63.98200857403893, 64.18581769625412, 64.36151521540515, 64.53018483379014, 64.65668704757888, 64.76913345983554, 64.91671937592241, 65.05024949047719, 65.190807505798, 65.33136552111884, 65.38758872724716, 65.49300723873779, 65.54220254410006, 65.61248155176048, 65.59139784946237, 65.66870475788882, 65.74601166631527, 65.78115117014546, 65.78115117014546, 65.90062548316817, 65.91468128470026, 65.89359758240214, 65.88656968163609, 65.90062548316817, 65.94982078853047, 65.97090449082859, 66.01307189542483, 66.06226720078712, 65.99901609389275, 66.05523930002109, 66.03415559772296, 65.95684868929649, 65.94279288776443, 65.89359758240214, 65.84440227703985, 65.8373743762738, 65.71087216248506, 65.67573265865485, 65.67573265865485, 65.65464895635674, 65.57031414716423, 65.4789514372057, 65.46489563567361, 65.4438119333755, 65.4438119333755, 65.4438119333755, 65.43678403260945, 65.43678403260945, 65.42272823107737, 65.40867242954529, 65.40867242954529, 65.40164452877924, 65.40164452877924, 65.3946166280132, 65.36650502494903, 65.34542132265092, 65.3243376203528, 65.28217021575655, 65.2751423149905, 65.24703071192634, 65.18377960503197, 65.09944479583947, 65.04322158971115, 64.97997048281677, 64.9518588797526, 64.87455197132616, 64.76913345983554, 64.68479865064305, 64.6074917422166, 64.56532433762034, 64.48098952842786, 64.44585002459765, 64.33340361234099, 64.24906880314849, 64.17176189472204, 64.13662239089184, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.978986576709538, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.98671726755218, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.98903647480497, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 102, 198, 292, 366, 436, 507, 574, 642, 714, 781, 842, 899, 948, 992, 1040, 1092, 1145, 1191, 1243, 1296, 1348, 1394, 1436, 1472, 1518, 1560, 1600, 1639, 1676, 1714, 1750, 1785, 1823, 1852, 1881, 1910, 1936, 1957, 1979, 2006, 2028, 2052, 2082, 2102, 2119, 2139, 2158, 2178, 2198, 2221, 2249, 2262, 2282, 2297, 2305, 2315, 2325, 2333, 2348, 2356, 2367, 2383, 2393, 2400, 2403, 2409, 2413, 2413, 2413, 2413, 2413, 2413, 2413, 2414, 2414, 2416, 2416, 2416, 2417, 2419, 2419, 2421, 2422, 2424, 2426, 2428, 2429, 2434, 2440, 2445, 2450, 2458, 2466, 2472, 2489, 2501, 2520, 2538, 2562, 2589], "harmed": [0, 19, 26, 40, 54, 65, 81, 93, 110, 128, 136, 152, 167, 180, 197, 209, 221, 237, 251, 262, 278, 294, 316, 330, 344, 361, 378, 394, 415, 436, 453, 470, 485, 503, 524, 538, 560, 576, 600, 611, 627, 644, 668, 681, 699, 719, 740, 757, 770, 787, 804, 825, 847, 859, 877, 896, 908, 925, 940, 956, 982, 998, 1014, 1027, 1046, 1062, 1070, 1077, 1077, 1077, 1078, 1078, 1080, 1082, 1083, 1084, 1086, 1087, 1091, 1095, 1100, 1106, 1109, 1114, 1125, 1139, 1149, 1159, 1168, 1185, 1205, 1222, 1241, 1255, 1273, 1295, 1323, 1354, 1383, 1412, 1441], "exact_score": [56.05453650994448, 56.637852273525894, 57.26333544170355, 57.82556750298685, 58.247241548949326, 58.661887694145754, 59.04842223627802, 59.43495677841029, 59.79337971747838, 60.172886358844615, 60.58753250404104, 60.90378803851289, 61.19895987068662, 61.45196429826411, 61.64171761894721, 61.8947220465247, 62.17583807716635, 62.43587040550986, 62.660763230023186, 62.94890716143088, 63.208939489774394, 63.46194391735188, 63.63061353573687, 63.82739475718602, 63.98200857403893, 64.18581769625412, 64.36151521540515, 64.53018483379014, 64.65668704757888, 64.76913345983554, 64.91671937592241, 65.05024949047719, 65.190807505798, 65.33136552111884, 65.38758872724716, 65.49300723873779, 65.54220254410006, 65.61248155176048, 65.59139784946237, 65.66870475788882, 65.74601166631527, 65.78817907091151, 65.78115117014546, 65.90062548316817, 65.91468128470026, 65.89359758240214, 65.88656968163609, 65.90062548316817, 65.94982078853047, 65.97090449082859, 66.01307189542483, 66.06226720078712, 65.99901609389275, 66.05523930002109, 66.03415559772296, 65.95684868929649, 65.94279288776443, 65.89359758240214, 65.84440227703985, 65.8373743762738, 65.71087216248506, 65.67573265865485, 65.67573265865485, 65.65464895635674, 65.57031414716423, 65.4789514372057, 65.46489563567361, 65.4438119333755, 65.4438119333755, 65.4438119333755, 65.43678403260945, 65.43678403260945, 65.42272823107737, 65.40867242954529, 65.40867242954529, 65.40164452877924, 65.40164452877924, 65.3946166280132, 65.36650502494903, 65.34542132265092, 65.3243376203528, 65.28217021575655, 65.2751423149905, 65.24703071192634, 65.18377960503197, 65.09944479583947, 65.04322158971115, 64.97997048281677, 64.9518588797526, 64.87455197132616, 64.76913345983554, 64.68479865064305, 64.6074917422166, 64.56532433762034, 64.48098952842786, 64.44585002459765, 64.33340361234099, 64.24906880314849, 64.17176189472204, 64.13662239089184, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.23162746884526, "threshold_area100": 64.23128821421041, "threshold_area25": 60.89465176751704}, "test": {"score": [26.688029946402747, 28.47176927657885, 29.81879023338911, 30.96163117148286, 32.027904602557925, 32.87581884695006, 33.74074809290191, 34.64254317556646, 35.38269574341378, 36.122848311261095, 36.82329920880243, 37.59748177976917, 38.141961829679836, 38.83957689362788, 39.46346028415053, 39.97674616453507, 40.473017043359896, 41.02884042764371, 41.47973796897598, 41.89376967359556, 42.327652213368125, 42.84944559453251, 43.334373138984205, 43.867509854521735, 44.417661571618986, 44.82602160905198, 45.17199330743272, 45.54065167455974, 45.90931004168676, 46.224087570541364, 46.5700592689221, 46.87916513059014, 47.219465161784306, 47.50588435470607, 47.82633355074724, 48.087230241329436, 48.3367835975385, 48.57215778578114, 48.80753197402377, 49.08544366616567, 49.278280350509036, 49.47678870203897, 49.66111788560247, 49.82276040041971, 49.90216374103168, 50.06380625584891, 50.216941269886284, 50.384255451890084, 50.47500212687519, 50.59694297138644, 50.73306298386411, 50.818137991662645, 50.97978050647989, 51.09037801661799, 51.17545302441653, 51.28888636814792, 51.342767206420326, 51.39948387828602, 51.45336471655844, 51.46470805093158, 51.50724555483084, 51.5157530556107, 51.535603890763696, 51.51858888920399, 51.5157530556107, 51.544111391543545, 51.544111391543545, 51.52993222357713, 51.52993222357713, 51.52426055639056, 51.51858888920399, 51.5157530556107, 51.51291722201742, 51.51291722201742, 51.50440972123755, 51.50157388764427, 51.50157388764427, 51.49873805405099, 51.4959022204577, 51.49873805405099, 51.4959022204577, 51.4959022204577, 51.476051385304714, 51.45336471655844, 51.45052888296515, 51.45620055015172, 51.433513881405446, 51.419334713439014, 51.38814054391288, 51.351274707200176, 51.32291637126733, 51.31440887048748, 51.26619969940164, 51.19246802597623, 51.16694552363668, 51.09604968380456, 51.070527181464996, 50.98545217366646, 50.8294813260358, 50.62813714091258, 50.00708958398321], "calls": [0.0, 2.223293537135241, 4.07509287355018, 5.620622181890367, 7.273913166775374, 8.635113291552052, 10.172135099112383, 11.768709412131697, 13.104387034568813, 14.485437994498481, 15.86365312083487, 17.45455576666761, 18.690979213339762, 20.049343504523154, 21.416215296486403, 22.49099622834132, 23.79264384765902, 25.11414230212971, 26.2314607378839, 27.331764172078383, 28.437739273459435, 29.784760230269686, 31.018347843348554, 32.51850381419618, 34.024331452230385, 35.24090406374954, 36.24195332217906, 37.40180926183251, 38.61838187335167, 39.744207809885715, 41.011825426083995, 42.12063636105833, 43.31735813742449, 44.37512406771971, 45.622890848764996, 46.7005076142132, 47.73558687576213, 48.83305447636332, 50.14320959646088, 51.413663046252445, 52.508294813260356, 53.67382242010039, 54.7315883503956, 55.78084677991095, 56.65995519382923, 57.70921362334458, 58.849218727845056, 59.98071633156567, 60.93355641890934, 61.858038170320164, 62.90162493264895, 63.93103252701131, 65.19297847602303, 66.4435810906616, 67.64030286702776, 68.83986047698721, 70.01673141820038, 71.16807985707399, 72.39883163655956, 73.60689674729886, 75.00779854238154, 76.25272948983354, 77.37855542636758, 78.5072171964949, 79.74364064316705, 80.719167399257, 82.10589002637325, 84.08813770807929, 85.10336613447524, 85.65068201797918, 86.2235204038227, 86.614865439696, 87.12247965289397, 87.5251680231404, 87.90233389104726, 88.22561892068174, 88.52054561438335, 88.82114397527154, 89.14726483849927, 89.50741570484644, 89.7541332274622, 90.03771658679068, 90.32413577971245, 90.67577914527976, 91.03025834444036, 91.34503587329496, 91.66548506933613, 91.99160593256389, 92.28936845985878, 92.64384765901937, 93.06071519723223, 93.5314635737175, 94.07877945722146, 94.58355783682613, 95.08266454924426, 95.65266710149449, 96.29640132717012, 96.99401639111817, 97.79372146442446, 98.6047698721039, 100.0], "rescued": [0, 662, 1173, 1598, 2008, 2336, 2671, 3020, 3305, 3594, 3868, 4180, 4404, 4675, 4927, 5121, 5340, 5556, 5741, 5913, 6091, 6305, 6499, 6722, 6953, 7113, 7254, 7412, 7563, 7700, 7850, 7981, 8129, 8251, 8386, 8504, 8613, 8726, 8835, 8953, 9049, 9146, 9231, 9316, 9373, 9449, 9529, 9604, 9665, 9728, 9793, 9851, 9930, 9995, 10054, 10121, 10169, 10216, 10260, 10290, 10330, 10358, 10388, 10405, 10428, 10449, 10460, 10462, 10462, 10462, 10462, 10463, 10463, 10463, 10463, 10465, 10466, 10468, 10471, 10476, 10479, 10481, 10483, 10485, 10490, 10497, 10501, 10504, 10510, 10516, 10525, 10544, 10558, 10567, 10584, 10597, 10627, 10654, 10679, 10700, 10723], "harmed": [0, 33, 69, 91, 125, 154, 184, 215, 239, 267, 294, 333, 365, 390, 422, 435, 479, 499, 525, 551, 576, 606, 629, 664, 701, 717, 736, 764, 785, 811, 839, 861, 889, 910, 932, 958, 979, 1009, 1035, 1055, 1083, 1110, 1130, 1158, 1187, 1206, 1232, 1248, 1277, 1297, 1314, 1342, 1364, 1390, 1419, 1446, 1475, 1502, 1527, 1553, 1578, 1603, 1626, 1649, 1673, 1684, 1695, 1702, 1702, 1704, 1706, 1708, 1709, 1709, 1712, 1715, 1716, 1719, 1723, 1727, 1731, 1733, 1742, 1752, 1758, 1763, 1775, 1783, 1800, 1819, 1838, 1860, 1891, 1926, 1952, 1990, 2029, 2086, 2166, 2258, 2500], "exact_score": [26.688029946402747, 27.507585854862036, 28.29311176020191, 29.041771828829088, 29.781924396676406, 30.539091966083433, 31.19416952613221, 31.886112922893684, 32.48163797748349, 33.07432719848, 33.65283725151008, 34.19731730142075, 34.79851402319712, 35.32881490514137, 35.850608286305764, 36.42911833933585, 36.89986671582112, 37.410316762612375, 37.83569180160509, 38.28942517653064, 38.81405439128832, 39.27345943340045, 39.747043643479, 40.178090349658284, 40.55525621756516, 40.98913875733772, 41.38331962680431, 41.75481382752461, 42.15750219777104, 42.51198139693163, 42.93735643592434, 43.328701471797636, 43.65765816861867, 44.03765987011882, 44.40631823724584, 44.718259932507166, 45.1010974676006, 45.41871083004849, 45.71363752375011, 46.03692255338457, 46.28647590959363, 46.56722343532882, 46.84513512747072, 47.151405155545476, 47.383943510194825, 47.66469103593001, 47.93409522729206, 48.17514108272127, 48.39917193659076, 48.60335195530726, 48.782009471684205, 49.0060403255537, 49.20454867708363, 49.352012023934435, 49.547684541871085, 49.69231205512861, 49.845447069165985, 49.950372912117516, 50.09216459178176, 50.24529960581914, 50.38709128548338, 50.48067379406177, 50.62530130731929, 50.74157048464396, 50.835152993222366, 50.945750503360465, 51.056348013498564, 51.138587187703834, 51.20097552675609, 51.30022970252106, 51.342767206420326, 51.41082721265916, 51.430678047812165, 51.44485721577858, 51.510081388424126, 51.50724555483084, 51.5157530556107, 51.535603890763696, 51.52709638998384, 51.52426055639056, 51.53843972435698, 51.54694722513683, 51.54694722513683, 51.535603890763696, 51.52993222357713, 51.52993222357713, 51.52142472279727, 51.5157530556107, 51.50157388764427, 51.4959022204577, 51.4959022204577, 51.45620055015172, 51.416498879845726, 51.3285880384539, 51.28037886736807, 51.16694552363668, 51.08187051583813, 50.98545217366646, 50.7897796557298, 50.514703797181184, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.46561877306366, "threshold_area100": 47.18712247965289, "threshold_area25": 37.55210844227662}, "selected_budget": 51, "selected": {"val": {"score": 66.06226720078712, "calls": 50.99444795839483, "rescued": 2249, "harmed": 825}, "test": {"score": 50.818137991662645, "calls": 63.93103252701131, "rescued": 9851, "harmed": 1342}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.947008490562439, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9047155380249023, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8676165342330933, "operator": "ge"}, {"budget": 0.04, "threshold": 0.829997181892395, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7986266613006592, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7661951780319214, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7346175312995911, "operator": "ge"}, {"budget": 0.08, "threshold": 0.7087351083755493, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6854745149612427, "operator": "ge"}, {"budget": 0.1, "threshold": 0.661552369594574, "operator": "ge"}, {"budget": 0.11, "threshold": 0.636324942111969, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6168317198753357, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5935205221176147, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5725961327552795, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5545302033424377, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5375873446464539, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5206890106201172, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5042575001716614, "operator": "ge"}, {"budget": 0.19, "threshold": 0.4901709258556366, "operator": "ge"}, {"budget": 0.2, "threshold": 0.47479119896888733, "operator": "ge"}, {"budget": 0.21, "threshold": 0.45752084255218506, "operator": "ge"}, {"budget": 0.22, "threshold": 0.44289904832839966, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4254084825515747, "operator": "ge"}, {"budget": 0.24, "threshold": 0.4092045724391937, "operator": "ge"}, {"budget": 0.25, "threshold": 0.3946588337421417, "operator": "ge"}, {"budget": 0.26, "threshold": 0.3831115663051605, "operator": "ge"}, {"budget": 0.27, "threshold": 0.37004461884498596, "operator": "ge"}, {"budget": 0.28, "threshold": 0.35690614581108093, "operator": "ge"}, {"budget": 0.29, "threshold": 0.344345360994339, "operator": "ge"}, {"budget": 0.3, "threshold": 0.3312433063983917, "operator": "ge"}, {"budget": 0.31, "threshold": 0.3194664716720581, "operator": "gt"}, {"budget": 0.32, "threshold": 0.30649492144584656, "operator": "ge"}, {"budget": 0.33, "threshold": 0.2935950756072998, "operator": "ge"}, {"budget": 0.34, "threshold": 0.28041309118270874, "operator": "ge"}, {"budget": 0.35, "threshold": 0.27006345987319946, "operator": "ge"}, {"budget": 0.36, "threshold": 0.2593194544315338, "operator": "ge"}, {"budget": 0.37, "threshold": 0.24810585379600525, "operator": "ge"}, {"budget": 0.38, "threshold": 0.2352842092514038, "operator": "ge"}, {"budget": 0.39, "threshold": 0.2235669046640396, "operator": "ge"}, {"budget": 0.4, "threshold": 0.21322877705097198, "operator": "ge"}, {"budget": 0.41, "threshold": 0.202176034450531, "operator": "gt"}, {"budget": 0.42, "threshold": 0.19243107736110687, "operator": "ge"}, {"budget": 0.43, "threshold": 0.18201598525047302, "operator": "ge"}, {"budget": 0.44, "threshold": 0.17333270609378815, "operator": "ge"}, {"budget": 0.45, "threshold": 0.16453661024570465, "operator": "ge"}, {"budget": 0.46, "threshold": 0.15539170801639557, "operator": "ge"}, {"budget": 0.47, "threshold": 0.1462712585926056, "operator": "ge"}, {"budget": 0.48, "threshold": 0.1383156180381775, "operator": "ge"}, {"budget": 0.49, "threshold": 0.13048739731311798, "operator": "ge"}, {"budget": 0.5, "threshold": 0.12159770727157593, "operator": "ge"}, {"budget": 0.51, "threshold": 0.11353585124015808, "operator": "ge"}, {"budget": 0.52, "threshold": 0.10398432612419128, "operator": "ge"}, {"budget": 0.53, "threshold": 0.09489816427230835, "operator": "ge"}, {"budget": 0.54, "threshold": 0.08641988784074783, "operator": "ge"}, {"budget": 0.55, "threshold": 0.07751017808914185, "operator": "ge"}, {"budget": 0.56, "threshold": 0.06917332112789154, "operator": "ge"}, {"budget": 0.57, "threshold": 0.061869435012340546, "operator": "ge"}, {"budget": 0.58, "threshold": 0.05425605922937393, "operator": "ge"}, {"budget": 0.59, "threshold": 0.046554844826459885, "operator": "ge"}, {"budget": 0.6, "threshold": 0.038441311568021774, "operator": "ge"}, {"budget": 0.61, "threshold": 0.0312487855553627, "operator": "ge"}, {"budget": 0.62, "threshold": 0.02472321316599846, "operator": "ge"}, {"budget": 0.63, "threshold": 0.01812935620546341, "operator": "ge"}, {"budget": 0.64, "threshold": 0.01187086757272482, "operator": "gt"}, {"budget": 0.65, "threshold": 0.006847094744443893, "operator": "ge"}, {"budget": 0.66, "threshold": 0.0017002517124637961, "operator": "ge"}, {"budget": 0.67, "threshold": -2.6931804313790053e-05, "operator": "ge"}, {"budget": 0.68, "threshold": -0.00012182118371129036, "operator": "ge"}, {"budget": 0.69, "threshold": -0.0002286457602167502, "operator": "ge"}, {"budget": 0.7, "threshold": -0.00039261888014152646, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0005824727704748511, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0008697039447724819, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0012221806682646275, "operator": "ge"}, {"budget": 0.74, "threshold": -0.001637571956962347, "operator": "ge"}, {"budget": 0.75, "threshold": -0.0021878425031900406, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0028185085393488407, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0034966126549988985, "operator": "ge"}, {"budget": 0.78, "threshold": -0.0044568623416125774, "operator": "ge"}, {"budget": 0.79, "threshold": -0.005563683807849884, "operator": "ge"}, {"budget": 0.8, "threshold": -0.00683121383190155, "operator": "ge"}, {"budget": 0.81, "threshold": -0.008126546628773212, "operator": "ge"}, {"budget": 0.82, "threshold": -0.009987862780690193, "operator": "ge"}, {"budget": 0.83, "threshold": -0.012347264215350151, "operator": "ge"}, {"budget": 0.84, "threshold": -0.015440721996128559, "operator": "ge"}, {"budget": 0.85, "threshold": -0.018446899950504303, "operator": "ge"}, {"budget": 0.86, "threshold": -0.02264130488038063, "operator": "ge"}, {"budget": 0.87, "threshold": -0.02818317897617817, "operator": "ge"}, {"budget": 0.88, "threshold": -0.03458961844444275, "operator": "ge"}, {"budget": 0.89, "threshold": -0.042335160076618195, "operator": "ge"}, {"budget": 0.9, "threshold": -0.052433937788009644, "operator": "ge"}, {"budget": 0.91, "threshold": -0.06523693352937698, "operator": "ge"}, {"budget": 0.92, "threshold": -0.08252975344657898, "operator": "ge"}, {"budget": 0.93, "threshold": -0.10367166250944138, "operator": "ge"}, {"budget": 0.94, "threshold": -0.12893329560756683, "operator": "ge"}, {"budget": 0.95, "threshold": -0.16521666944026947, "operator": "ge"}, {"budget": 0.96, "threshold": -0.2091291844844818, "operator": "ge"}, {"budget": 0.97, "threshold": -0.2669602930545807, "operator": "ge"}, {"budget": 0.98, "threshold": -0.3632992208003998, "operator": "ge"}, {"budget": 0.99, "threshold": -0.49766337871551514, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.637852273525894, 57.26333544170355, 57.82556750298685, 58.247241548949326, 58.661887694145754, 59.04842223627802, 59.43495677841029, 59.79337971747838, 60.172886358844615, 60.58753250404104, 60.90378803851289, 61.19895987068662, 61.45196429826411, 61.64171761894721, 61.8947220465247, 62.17583807716635, 62.43587040550986, 62.660763230023186, 62.94890716143088, 63.208939489774394, 63.46194391735188, 63.63061353573687, 63.82739475718602, 63.98200857403893, 64.18581769625412, 64.36151521540515, 64.53018483379014, 64.65668704757888, 64.76913345983554, 64.91671937592241, 65.05024949047719, 65.190807505798, 65.33136552111884, 65.38758872724716, 65.49300723873779, 65.54220254410006, 65.61248155176048, 65.59139784946237, 65.66870475788882, 65.74601166631527, 65.78115117014546, 65.78115117014546, 65.90062548316817, 65.91468128470026, 65.89359758240214, 65.88656968163609, 65.90062548316817, 65.94982078853047, 65.97090449082859, 66.01307189542483, 66.06226720078712, 65.99901609389275, 66.05523930002109, 66.03415559772296, 65.95684868929649, 65.94279288776443, 65.89359758240214, 65.84440227703985, 65.8373743762738, 65.71087216248506, 65.67573265865485, 65.67573265865485, 65.65464895635674, 65.57031414716423, 65.4789514372057, 65.46489563567361, 65.4438119333755, 65.4438119333755, 65.4438119333755, 65.43678403260945, 65.43678403260945, 65.42272823107737, 65.40867242954529, 65.40867242954529, 65.40164452877924, 65.40164452877924, 65.3946166280132, 65.36650502494903, 65.34542132265092, 65.3243376203528, 65.28217021575655, 65.2751423149905, 65.24703071192634, 65.18377960503197, 65.09944479583947, 65.04322158971115, 64.97997048281677, 64.9518588797526, 64.87455197132616, 64.76913345983554, 64.68479865064305, 64.6074917422166, 64.56532433762034, 64.48098952842786, 64.44585002459765, 64.33340361234099, 64.24906880314849, 64.17176189472204, 64.13662239089184, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.978986576709538, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.98671726755218, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.98903647480497, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 102, 198, 292, 366, 436, 507, 574, 642, 714, 781, 842, 899, 948, 992, 1040, 1092, 1145, 1191, 1243, 1296, 1348, 1394, 1436, 1472, 1518, 1560, 1600, 1639, 1676, 1714, 1750, 1785, 1823, 1852, 1881, 1910, 1936, 1957, 1979, 2006, 2028, 2052, 2082, 2102, 2119, 2139, 2158, 2178, 2198, 2221, 2249, 2262, 2282, 2297, 2305, 2315, 2325, 2333, 2348, 2356, 2367, 2383, 2393, 2400, 2403, 2409, 2413, 2413, 2413, 2413, 2413, 2413, 2413, 2414, 2414, 2416, 2416, 2416, 2417, 2419, 2419, 2421, 2422, 2424, 2426, 2428, 2429, 2434, 2440, 2445, 2450, 2458, 2466, 2472, 2489, 2501, 2520, 2538, 2562, 2589], "harmed": [0, 19, 26, 40, 54, 65, 81, 93, 110, 128, 136, 152, 167, 180, 197, 209, 221, 237, 251, 262, 278, 294, 316, 330, 344, 361, 378, 394, 415, 436, 453, 470, 485, 503, 524, 538, 560, 576, 600, 611, 627, 644, 668, 681, 699, 719, 740, 757, 770, 787, 804, 825, 847, 859, 877, 896, 908, 925, 940, 956, 982, 998, 1014, 1027, 1046, 1062, 1070, 1077, 1077, 1077, 1078, 1078, 1080, 1082, 1083, 1084, 1086, 1087, 1091, 1095, 1100, 1106, 1109, 1114, 1125, 1139, 1149, 1159, 1168, 1185, 1205, 1222, 1241, 1255, 1273, 1295, 1323, 1354, 1383, 1412, 1441], "exact_score": [56.05453650994448, 56.637852273525894, 57.26333544170355, 57.82556750298685, 58.247241548949326, 58.661887694145754, 59.04842223627802, 59.43495677841029, 59.79337971747838, 60.172886358844615, 60.58753250404104, 60.90378803851289, 61.19895987068662, 61.45196429826411, 61.64171761894721, 61.8947220465247, 62.17583807716635, 62.43587040550986, 62.660763230023186, 62.94890716143088, 63.208939489774394, 63.46194391735188, 63.63061353573687, 63.82739475718602, 63.98200857403893, 64.18581769625412, 64.36151521540515, 64.53018483379014, 64.65668704757888, 64.76913345983554, 64.91671937592241, 65.05024949047719, 65.190807505798, 65.33136552111884, 65.38758872724716, 65.49300723873779, 65.54220254410006, 65.61248155176048, 65.59139784946237, 65.66870475788882, 65.74601166631527, 65.78817907091151, 65.78115117014546, 65.90062548316817, 65.91468128470026, 65.89359758240214, 65.88656968163609, 65.90062548316817, 65.94982078853047, 65.97090449082859, 66.01307189542483, 66.06226720078712, 65.99901609389275, 66.05523930002109, 66.03415559772296, 65.95684868929649, 65.94279288776443, 65.89359758240214, 65.84440227703985, 65.8373743762738, 65.71087216248506, 65.67573265865485, 65.67573265865485, 65.65464895635674, 65.57031414716423, 65.4789514372057, 65.46489563567361, 65.4438119333755, 65.4438119333755, 65.4438119333755, 65.43678403260945, 65.43678403260945, 65.42272823107737, 65.40867242954529, 65.40867242954529, 65.40164452877924, 65.40164452877924, 65.3946166280132, 65.36650502494903, 65.34542132265092, 65.3243376203528, 65.28217021575655, 65.2751423149905, 65.24703071192634, 65.18377960503197, 65.09944479583947, 65.04322158971115, 64.97997048281677, 64.9518588797526, 64.87455197132616, 64.76913345983554, 64.68479865064305, 64.6074917422166, 64.56532433762034, 64.48098952842786, 64.44585002459765, 64.33340361234099, 64.24906880314849, 64.17176189472204, 64.13662239089184, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.23162746884526, "threshold_area100": 64.23128821421041, "threshold_area25": 60.89465176751704}, "test": {"score": [26.688029946402747, 28.47176927657885, 29.81879023338911, 30.96163117148286, 32.027904602557925, 32.87581884695006, 33.74074809290191, 34.64254317556646, 35.38269574341378, 36.122848311261095, 36.82329920880243, 37.59748177976917, 38.141961829679836, 38.83957689362788, 39.46346028415053, 39.97674616453507, 40.473017043359896, 41.02884042764371, 41.47973796897598, 41.89376967359556, 42.327652213368125, 42.84944559453251, 43.334373138984205, 43.867509854521735, 44.417661571618986, 44.82602160905198, 45.17199330743272, 45.54065167455974, 45.90931004168676, 46.224087570541364, 46.5700592689221, 46.87916513059014, 47.219465161784306, 47.50588435470607, 47.82633355074724, 48.087230241329436, 48.3367835975385, 48.57215778578114, 48.80753197402377, 49.08544366616567, 49.278280350509036, 49.47678870203897, 49.66111788560247, 49.82276040041971, 49.90216374103168, 50.06380625584891, 50.216941269886284, 50.384255451890084, 50.47500212687519, 50.59694297138644, 50.73306298386411, 50.818137991662645, 50.97978050647989, 51.09037801661799, 51.17545302441653, 51.28888636814792, 51.342767206420326, 51.39948387828602, 51.45336471655844, 51.46470805093158, 51.50724555483084, 51.5157530556107, 51.535603890763696, 51.51858888920399, 51.5157530556107, 51.544111391543545, 51.544111391543545, 51.52993222357713, 51.52993222357713, 51.52426055639056, 51.51858888920399, 51.5157530556107, 51.51291722201742, 51.51291722201742, 51.50440972123755, 51.50157388764427, 51.50157388764427, 51.49873805405099, 51.4959022204577, 51.49873805405099, 51.4959022204577, 51.4959022204577, 51.476051385304714, 51.45336471655844, 51.45052888296515, 51.45620055015172, 51.433513881405446, 51.419334713439014, 51.38814054391288, 51.351274707200176, 51.32291637126733, 51.31440887048748, 51.26619969940164, 51.19246802597623, 51.16694552363668, 51.09604968380456, 51.070527181464996, 50.98545217366646, 50.8294813260358, 50.62813714091258, 50.00708958398321], "calls": [0.0, 2.223293537135241, 4.07509287355018, 5.620622181890367, 7.273913166775374, 8.635113291552052, 10.172135099112383, 11.768709412131697, 13.104387034568813, 14.485437994498481, 15.86365312083487, 17.45455576666761, 18.690979213339762, 20.049343504523154, 21.416215296486403, 22.49099622834132, 23.79264384765902, 25.11414230212971, 26.2314607378839, 27.331764172078383, 28.437739273459435, 29.784760230269686, 31.018347843348554, 32.51850381419618, 34.024331452230385, 35.24090406374954, 36.24195332217906, 37.40180926183251, 38.61838187335167, 39.744207809885715, 41.011825426083995, 42.12063636105833, 43.31735813742449, 44.37512406771971, 45.622890848764996, 46.7005076142132, 47.73558687576213, 48.83305447636332, 50.14320959646088, 51.413663046252445, 52.508294813260356, 53.67382242010039, 54.7315883503956, 55.78084677991095, 56.65995519382923, 57.70921362334458, 58.849218727845056, 59.98071633156567, 60.93355641890934, 61.858038170320164, 62.90162493264895, 63.93103252701131, 65.19297847602303, 66.4435810906616, 67.64030286702776, 68.83986047698721, 70.01673141820038, 71.16807985707399, 72.39883163655956, 73.60689674729886, 75.00779854238154, 76.25272948983354, 77.37855542636758, 78.5072171964949, 79.74364064316705, 80.719167399257, 82.10589002637325, 84.08813770807929, 85.10336613447524, 85.65068201797918, 86.2235204038227, 86.614865439696, 87.12247965289397, 87.5251680231404, 87.90233389104726, 88.22561892068174, 88.52054561438335, 88.82114397527154, 89.14726483849927, 89.50741570484644, 89.7541332274622, 90.03771658679068, 90.32413577971245, 90.67577914527976, 91.03025834444036, 91.34503587329496, 91.66548506933613, 91.99160593256389, 92.28936845985878, 92.64384765901937, 93.06071519723223, 93.5314635737175, 94.07877945722146, 94.58355783682613, 95.08266454924426, 95.65266710149449, 96.29640132717012, 96.99401639111817, 97.79372146442446, 98.6047698721039, 100.0], "rescued": [0, 662, 1173, 1598, 2008, 2336, 2671, 3020, 3305, 3594, 3868, 4180, 4404, 4675, 4927, 5121, 5340, 5556, 5741, 5913, 6091, 6305, 6499, 6722, 6953, 7113, 7254, 7412, 7563, 7700, 7850, 7981, 8129, 8251, 8386, 8504, 8613, 8726, 8835, 8953, 9049, 9146, 9231, 9316, 9373, 9449, 9529, 9604, 9665, 9728, 9793, 9851, 9930, 9995, 10054, 10121, 10169, 10216, 10260, 10290, 10330, 10358, 10388, 10405, 10428, 10449, 10460, 10462, 10462, 10462, 10462, 10463, 10463, 10463, 10463, 10465, 10466, 10468, 10471, 10476, 10479, 10481, 10483, 10485, 10490, 10497, 10501, 10504, 10510, 10516, 10525, 10544, 10558, 10567, 10584, 10597, 10627, 10654, 10679, 10700, 10723], "harmed": [0, 33, 69, 91, 125, 154, 184, 215, 239, 267, 294, 333, 365, 390, 422, 435, 479, 499, 525, 551, 576, 606, 629, 664, 701, 717, 736, 764, 785, 811, 839, 861, 889, 910, 932, 958, 979, 1009, 1035, 1055, 1083, 1110, 1130, 1158, 1187, 1206, 1232, 1248, 1277, 1297, 1314, 1342, 1364, 1390, 1419, 1446, 1475, 1502, 1527, 1553, 1578, 1603, 1626, 1649, 1673, 1684, 1695, 1702, 1702, 1704, 1706, 1708, 1709, 1709, 1712, 1715, 1716, 1719, 1723, 1727, 1731, 1733, 1742, 1752, 1758, 1763, 1775, 1783, 1800, 1819, 1838, 1860, 1891, 1926, 1952, 1990, 2029, 2086, 2166, 2258, 2500], "exact_score": [26.688029946402747, 27.507585854862036, 28.29311176020191, 29.041771828829088, 29.781924396676406, 30.539091966083433, 31.19416952613221, 31.886112922893684, 32.48163797748349, 33.07432719848, 33.65283725151008, 34.19731730142075, 34.79851402319712, 35.32881490514137, 35.850608286305764, 36.42911833933585, 36.89986671582112, 37.410316762612375, 37.83569180160509, 38.28942517653064, 38.81405439128832, 39.27345943340045, 39.747043643479, 40.178090349658284, 40.55525621756516, 40.98913875733772, 41.38331962680431, 41.75481382752461, 42.15750219777104, 42.51198139693163, 42.93735643592434, 43.328701471797636, 43.65765816861867, 44.03765987011882, 44.40631823724584, 44.718259932507166, 45.1010974676006, 45.41871083004849, 45.71363752375011, 46.03692255338457, 46.28647590959363, 46.56722343532882, 46.84513512747072, 47.151405155545476, 47.383943510194825, 47.66469103593001, 47.93409522729206, 48.17514108272127, 48.39917193659076, 48.60335195530726, 48.782009471684205, 49.0060403255537, 49.20454867708363, 49.352012023934435, 49.547684541871085, 49.69231205512861, 49.845447069165985, 49.950372912117516, 50.09216459178176, 50.24529960581914, 50.38709128548338, 50.48067379406177, 50.62530130731929, 50.74157048464396, 50.835152993222366, 50.945750503360465, 51.056348013498564, 51.138587187703834, 51.20097552675609, 51.30022970252106, 51.342767206420326, 51.41082721265916, 51.430678047812165, 51.44485721577858, 51.510081388424126, 51.50724555483084, 51.5157530556107, 51.535603890763696, 51.52709638998384, 51.52426055639056, 51.53843972435698, 51.54694722513683, 51.54694722513683, 51.535603890763696, 51.52993222357713, 51.52993222357713, 51.52142472279727, 51.5157530556107, 51.50157388764427, 51.4959022204577, 51.4959022204577, 51.45620055015172, 51.416498879845726, 51.3285880384539, 51.28037886736807, 51.16694552363668, 51.08187051583813, 50.98545217366646, 50.7897796557298, 50.514703797181184, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.46561877306366, "threshold_area100": 47.18712247965289, "threshold_area25": 37.55210844227662}, "selected_budget": 51, "selected": {"val": {"score": 66.06226720078712, "calls": 50.99444795839483, "rescued": 2249, "harmed": 825}, "test": {"score": 50.818137991662645, "calls": 63.93103252701131, "rescued": 9851, "harmed": 1342}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 17.285442352294922, "operator": "ge"}, {"budget": 0.02, "threshold": 16.112497329711914, "operator": "ge"}, {"budget": 0.03, "threshold": 15.428411483764648, "operator": "ge"}, {"budget": 0.04, "threshold": 14.829635620117188, "operator": "gt"}, {"budget": 0.05, "threshold": 14.433398246765137, "operator": "ge"}, {"budget": 0.06, "threshold": 14.089945793151855, "operator": "ge"}, {"budget": 0.07, "threshold": 13.753314971923828, "operator": "gt"}, {"budget": 0.08, "threshold": 13.40865421295166, "operator": "ge"}, {"budget": 0.09, "threshold": 13.132294654846191, "operator": "gt"}, {"budget": 0.1, "threshold": 12.901185989379883, "operator": "ge"}, {"budget": 0.11, "threshold": 12.632733345031738, "operator": "ge"}, {"budget": 0.12, "threshold": 12.3507080078125, "operator": "ge"}, {"budget": 0.13, "threshold": 12.100586891174316, "operator": "ge"}, {"budget": 0.14, "threshold": 11.875007629394531, "operator": "ge"}, {"budget": 0.15, "threshold": 11.61896800994873, "operator": "ge"}, {"budget": 0.16, "threshold": 11.379321098327637, "operator": "ge"}, {"budget": 0.17, "threshold": 11.127031326293945, "operator": "ge"}, {"budget": 0.18, "threshold": 10.881548881530762, "operator": "ge"}, {"budget": 0.19, "threshold": 10.627211570739746, "operator": "ge"}, {"budget": 0.2, "threshold": 10.395356178283691, "operator": "ge"}, {"budget": 0.21, "threshold": 10.108654975891113, "operator": "ge"}, {"budget": 0.22, "threshold": 9.854650497436523, "operator": "ge"}, {"budget": 0.23, "threshold": 9.599750518798828, "operator": "ge"}, {"budget": 0.24, "threshold": 9.374853134155273, "operator": "ge"}, {"budget": 0.25, "threshold": 9.125868797302246, "operator": "ge"}, {"budget": 0.26, "threshold": 8.824769973754883, "operator": "ge"}, {"budget": 0.27, "threshold": 8.601552963256836, "operator": "ge"}, {"budget": 0.28, "threshold": 8.316250801086426, "operator": "ge"}, {"budget": 0.29, "threshold": 8.025965690612793, "operator": "ge"}, {"budget": 0.3, "threshold": 7.752985954284668, "operator": "ge"}, {"budget": 0.31, "threshold": 7.472476482391357, "operator": "ge"}, {"budget": 0.32, "threshold": 7.205559730529785, "operator": "ge"}, {"budget": 0.33, "threshold": 6.942440032958984, "operator": "ge"}, {"budget": 0.34, "threshold": 6.660266399383545, "operator": "ge"}, {"budget": 0.35, "threshold": 6.412991046905518, "operator": "ge"}, {"budget": 0.36, "threshold": 6.159868240356445, "operator": "ge"}, {"budget": 0.37, "threshold": 5.909291744232178, "operator": "ge"}, {"budget": 0.38, "threshold": 5.672426700592041, "operator": "ge"}, {"budget": 0.39, "threshold": 5.446509838104248, "operator": "ge"}, {"budget": 0.4, "threshold": 5.180779933929443, "operator": "ge"}, {"budget": 0.41, "threshold": 4.934677600860596, "operator": "ge"}, {"budget": 0.42, "threshold": 4.669310092926025, "operator": "ge"}, {"budget": 0.43, "threshold": 4.460076808929443, "operator": "ge"}, {"budget": 0.44, "threshold": 4.273587226867676, "operator": "ge"}, {"budget": 0.45, "threshold": 4.037932395935059, "operator": "ge"}, {"budget": 0.46, "threshold": 3.838202476501465, "operator": "ge"}, {"budget": 0.47, "threshold": 3.640995502471924, "operator": "ge"}, {"budget": 0.48, "threshold": 3.462536096572876, "operator": "ge"}, {"budget": 0.49, "threshold": 3.289860725402832, "operator": "ge"}, {"budget": 0.5, "threshold": 3.103149652481079, "operator": "ge"}, {"budget": 0.51, "threshold": 2.928440570831299, "operator": "ge"}, {"budget": 0.52, "threshold": 2.755647659301758, "operator": "ge"}, {"budget": 0.53, "threshold": 2.5723695755004883, "operator": "ge"}, {"budget": 0.54, "threshold": 2.403315782546997, "operator": "ge"}, {"budget": 0.55, "threshold": 2.237300395965576, "operator": "ge"}, {"budget": 0.56, "threshold": 2.062060832977295, "operator": "ge"}, {"budget": 0.57, "threshold": 1.8737694025039673, "operator": "ge"}, {"budget": 0.58, "threshold": 1.6647379398345947, "operator": "ge"}, {"budget": 0.59, "threshold": 1.4767149686813354, "operator": "ge"}, {"budget": 0.6, "threshold": 1.2860910892486572, "operator": "ge"}, {"budget": 0.61, "threshold": 1.0914396047592163, "operator": "ge"}, {"budget": 0.62, "threshold": 0.8823511600494385, "operator": "ge"}, {"budget": 0.63, "threshold": 0.6956336498260498, "operator": "ge"}, {"budget": 0.64, "threshold": 0.5082706212997437, "operator": "ge"}, {"budget": 0.65, "threshold": 0.32036399841308594, "operator": "ge"}, {"budget": 0.66, "threshold": 0.14582228660583496, "operator": "ge"}, {"budget": 0.67, "threshold": -0.049517691135406494, "operator": "ge"}, {"budget": 0.68, "threshold": -0.2566497325897217, "operator": "ge"}, {"budget": 0.69, "threshold": -0.4387894868850708, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6872824430465698, "operator": "ge"}, {"budget": 0.71, "threshold": -0.8732433319091797, "operator": "ge"}, {"budget": 0.72, "threshold": -1.0583648681640625, "operator": "ge"}, {"budget": 0.73, "threshold": -1.220979928970337, "operator": "ge"}, {"budget": 0.74, "threshold": -1.4394391775131226, "operator": "ge"}, {"budget": 0.75, "threshold": -1.6317648887634277, "operator": "ge"}, {"budget": 0.76, "threshold": -1.8723622560501099, "operator": "ge"}, {"budget": 0.77, "threshold": -2.0756564140319824, "operator": "ge"}, {"budget": 0.78, "threshold": -2.265080213546753, "operator": "ge"}, {"budget": 0.79, "threshold": -2.4627108573913574, "operator": "ge"}, {"budget": 0.8, "threshold": -2.6753432750701904, "operator": "ge"}, {"budget": 0.81, "threshold": -2.884326457977295, "operator": "ge"}, {"budget": 0.82, "threshold": -3.0891966819763184, "operator": "ge"}, {"budget": 0.83, "threshold": -3.3063039779663086, "operator": "ge"}, {"budget": 0.84, "threshold": -3.532700538635254, "operator": "ge"}, {"budget": 0.85, "threshold": -3.773592472076416, "operator": "ge"}, {"budget": 0.86, "threshold": -4.0215373039245605, "operator": "ge"}, {"budget": 0.87, "threshold": -4.32932186126709, "operator": "ge"}, {"budget": 0.88, "threshold": -4.590445041656494, "operator": "ge"}, {"budget": 0.89, "threshold": -4.874321460723877, "operator": "ge"}, {"budget": 0.9, "threshold": -5.194859504699707, "operator": "ge"}, {"budget": 0.91, "threshold": -5.519923210144043, "operator": "ge"}, {"budget": 0.92, "threshold": -5.827476978302002, "operator": "ge"}, {"budget": 0.93, "threshold": -6.275116920471191, "operator": "ge"}, {"budget": 0.94, "threshold": -6.738767623901367, "operator": "ge"}, {"budget": 0.95, "threshold": -7.344954967498779, "operator": "ge"}, {"budget": 0.96, "threshold": -7.955756187438965, "operator": "ge"}, {"budget": 0.97, "threshold": -8.665624618530273, "operator": "ge"}, {"budget": 0.98, "threshold": -9.743823051452637, "operator": "ge"}, {"budget": 0.99, "threshold": -11.682809829711914, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.72921498348443, 57.27739124323564, 57.88179070911518, 58.40888326656827, 58.971115327851564, 59.49820788530465, 60.0182725419917, 60.52428139714667, 61.02326235153559, 61.50818750439243, 61.972028954951156, 62.40073090167966, 62.88565605453651, 63.26516269590273, 63.68683674186521, 64.10148288706164, 64.50910113149202, 64.95888678051865, 65.28217021575655, 65.54923044486611, 65.8022348724436, 66.1606578115117, 66.51205284981376, 66.80019678122144, 67.12348021645934, 67.37648464403682, 67.62246117084825, 67.85438189612762, 68.09333052217302, 68.1917211328976, 68.33227914821842, 68.50094876660341, 68.59231147656195, 68.62745098039215, 68.67664628575444, 68.69070208728652, 68.67664628575444, 68.62042307962611, 68.57825567502987, 68.52203246890154, 68.43769765970904, 68.41661395741092, 68.29713964438821, 68.21983273596176, 68.00196781221447, 67.93871670532012, 67.90357720148991, 67.79815868999928, 67.60137746855014, 67.48190315552745, 67.38351254480285, 67.24295452948203, 67.12348021645934, 67.01806170496873, 66.86344788811581, 66.76505727739124, 66.65963876590062, 66.49799704828168, 66.30824372759857, 66.11849040691546, 65.99901609389275, 65.87251388010401, 65.80926277320964, 65.67573265865485, 65.59139784946237, 65.5140909410359, 65.4297561318434, 65.38056082648113, 65.2751423149905, 65.29622601728863, 65.2048633073301, 65.17675170426594, 65.190807505798, 65.16972380349989, 65.05727739124323, 65.01510998664698, 64.97294258205073, 64.9518588797526, 64.90266357439033, 64.86752407056012, 64.7972450628997, 64.79021716213367, 64.74804975753742, 64.74102185677137, 64.71291025370721, 64.66371494834493, 64.62154754374868, 64.6074917422166, 64.57938013915243, 64.53721273455618, 64.4880174291939, 64.4528779253637, 64.43882212383161, 64.38259891770328, 64.32637571157494, 64.30529200927683, 64.2842083069787, 64.24906880314849, 64.17878979548807, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.9918476351113923, 4.996837444655281, 5.994799353433129, 6.992761262210978, 7.997751071754868, 8.988685079766674, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 96, 174, 261, 336, 416, 491, 565, 638, 710, 779, 847, 909, 978, 1033, 1095, 1155, 1214, 1279, 1327, 1366, 1406, 1460, 1513, 1560, 1612, 1652, 1694, 1736, 1780, 1808, 1845, 1882, 1916, 1943, 1972, 1994, 2011, 2031, 2049, 2069, 2086, 2106, 2118, 2129, 2142, 2162, 2181, 2197, 2209, 2220, 2232, 2244, 2260, 2273, 2287, 2300, 2313, 2320, 2330, 2338, 2351, 2364, 2374, 2384, 2391, 2403, 2417, 2427, 2438, 2453, 2462, 2474, 2487, 2497, 2504, 2511, 2520, 2529, 2537, 2543, 2546, 2553, 2560, 2568, 2572, 2576, 2577, 2581, 2584, 2586, 2588, 2588, 2588, 2589, 2589, 2589, 2589, 2589, 2589, 2589], "harmed": [0, 0, 0, 1, 1, 1, 1, 1, 2, 3, 3, 5, 6, 6, 7, 9, 10, 11, 12, 14, 15, 19, 22, 25, 31, 37, 41, 48, 57, 67, 81, 98, 111, 132, 154, 176, 196, 215, 243, 267, 295, 324, 347, 376, 398, 442, 471, 495, 526, 566, 594, 620, 652, 685, 713, 749, 776, 804, 834, 871, 906, 936, 967, 986, 1015, 1034, 1057, 1083, 1100, 1126, 1138, 1160, 1176, 1187, 1200, 1223, 1236, 1251, 1263, 1278, 1289, 1302, 1310, 1323, 1332, 1340, 1351, 1358, 1364, 1371, 1379, 1388, 1393, 1395, 1404, 1412, 1415, 1418, 1423, 1433, 1441], "exact_score": [56.05453650994448, 56.72921498348443, 57.27739124323564, 57.88179070911518, 58.40888326656827, 58.971115327851564, 59.49820788530465, 60.0182725419917, 60.52428139714667, 61.03029025230163, 61.50818750439243, 61.972028954951156, 62.40073090167966, 62.88565605453651, 63.26516269590273, 63.68683674186521, 64.10148288706164, 64.50910113149202, 64.95888678051865, 65.28217021575655, 65.54923044486611, 65.8022348724436, 66.1606578115117, 66.51205284981376, 66.80019678122144, 67.12348021645934, 67.37648464403682, 67.62246117084825, 67.85438189612762, 68.09333052217302, 68.1917211328976, 68.33227914821842, 68.50094876660341, 68.59231147656195, 68.62745098039215, 68.67664628575444, 68.69070208728652, 68.67664628575444, 68.62042307962611, 68.57825567502987, 68.52203246890154, 68.43769765970904, 68.41661395741092, 68.29713964438821, 68.21983273596176, 68.00196781221447, 67.93871670532012, 67.90357720148991, 67.79815868999928, 67.60137746855014, 67.48190315552745, 67.38351254480285, 67.24295452948203, 67.12348021645934, 67.01806170496873, 66.86344788811581, 66.76505727739124, 66.65963876590062, 66.49799704828168, 66.30824372759857, 66.11849040691546, 65.99901609389275, 65.87251388010401, 65.80926277320964, 65.67573265865485, 65.59139784946237, 65.5140909410359, 65.4297561318434, 65.38056082648113, 65.2751423149905, 65.29622601728863, 65.2048633073301, 65.17675170426594, 65.190807505798, 65.16972380349989, 65.05727739124323, 65.01510998664698, 64.97294258205073, 64.9518588797526, 64.90266357439033, 64.86752407056012, 64.7972450628997, 64.79021716213367, 64.74804975753742, 64.74102185677137, 64.71291025370721, 64.66371494834493, 64.62154754374868, 64.6074917422166, 64.57938013915243, 64.53721273455618, 64.4880174291939, 64.4528779253637, 64.43882212383161, 64.38259891770328, 64.32637571157494, 64.30529200927683, 64.2842083069787, 64.24906880314849, 64.17878979548807, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 65.28264659570516, "threshold_area100": 65.28231077377188, "threshold_area25": 62.26087567643545}, "test": {"score": [26.688029946402747, 27.198479993194002, 27.72878087513825, 28.1654992485041, 28.73550180075433, 29.223265178799313, 29.6344610498256, 30.020134418512324, 30.52491279811701, 30.947452003516435, 31.293423701897172, 31.724470408076456, 32.14133794628932, 32.56387715168874, 32.895669682103055, 33.30119388594277, 33.732240592122054, 34.157615631114766, 34.54328899980149, 34.97433570598078, 35.39403907778692, 35.93851912769759, 36.43762584011571, 36.95658338768681, 37.39046592745938, 37.91793097581034, 38.56450103507926, 39.08345858265037, 39.778237813005134, 40.58928622068457, 41.25854294869977, 41.88242633922242, 42.563026401610756, 43.27482063352522, 43.96959986387999, 44.60766242236905, 45.257068315231265, 45.87244420497405, 46.43110342285115, 46.93021013526927, 47.59946686328447, 48.05887190539659, 48.55797861781471, 48.88977114822902, 49.16201117318436, 49.46544536766583, 49.714998723874885, 49.950372912117516, 50.19141876754672, 50.381419618296796, 50.574256302640165, 50.7160479823044, 50.80112299010294, 50.95425800414032, 51.01664634319258, 51.05351217990528, 51.172617190823246, 51.22649802909566, 51.26619969940164, 51.354110540793464, 51.376797209539745, 51.42784221421888, 51.44485721577858, 51.45052888296515, 51.53843972435698, 51.52426055639056, 51.54694722513683, 51.52709638998384, 51.54694722513683, 51.54694722513683, 51.544111391543545, 51.51858888920399, 51.4959022204577, 51.484558886084564, 51.476051385304714, 51.45336471655844, 51.39664804469274, 51.354110540793464, 51.33709553923376, 51.28605053455464, 51.22649802909566, 51.1981396931628, 51.14709468848368, 51.0676913478717, 51.002467175226165, 50.95992967132689, 50.877690497121634, 50.80679465728951, 50.72739131667754, 50.67351047840513, 50.58843547060658, 50.52321129796103, 50.47500212687519, 50.406942120636366, 50.3445537815841, 50.30768794487139, 50.24529960581914, 50.20559793551315, 50.154552930834015, 50.08932875818848, 50.00708958398321], "calls": [0.0, 0.8252275756458611, 1.7241868247171257, 2.4841902277174377, 3.471060318180529, 4.313302895386099, 5.016589626520716, 5.799279698267306, 6.689731446558716, 7.469585684712021, 8.121827411167512, 9.000935825085785, 9.854521736664493, 10.767660153702181, 11.516320222329353, 12.471996143266313, 13.368119558744294, 14.374840484360377, 15.381561409976463, 16.43649150667839, 17.360973258089217, 18.620083373507644, 19.768595978787964, 20.993676091086975, 22.04293452060233, 23.273686300087913, 24.858917278734083, 26.208774069137625, 27.932960893854748, 29.954910245866774, 31.701783739330175, 33.6131355812041, 35.54433825823101, 37.56061594305646, 39.73002864191929, 41.834217168136576, 43.91288319201429, 45.92348920965318, 47.83767688512038, 49.811417066046566, 51.96097892975641, 53.92904744349602, 56.0332359697133, 57.77160196239684, 59.29728043558404, 61.08952726654, 62.60953407254062, 64.16073504806738, 65.45103933301193, 66.59104443751241, 67.88702038964354, 68.95045798712532, 69.97702974789439, 71.2786773672121, 72.2995774607946, 73.24391004735841, 74.27331764172078, 75.20914272750476, 76.2584011570201, 77.25377874826304, 78.15840966452089, 79.18214559169668, 80.21722485324561, 80.99707909139892, 81.87051583813061, 82.61917590675779, 83.25723846524686, 83.95201769560163, 84.71485693219522, 85.29903865241188, 86.09590789212488, 86.69426878030797, 87.27277883333807, 87.80875138246887, 88.54039644953635, 89.11890650256643, 89.84487990244733, 90.32413577971245, 90.817570824944, 91.3932450443808, 91.9689192638176, 92.44250347389615, 92.94161018631428, 93.40385106201968, 93.86609193772509, 94.36236281654993, 94.87281286334118, 95.38042707653915, 95.78311544678559, 96.20565465218502, 96.74162720131582, 97.13864390437568, 97.47043643479, 97.92416980971556, 98.34103734792843, 98.74088988458158, 99.03865241187647, 99.31656410401838, 99.54910245866773, 99.7929841476902, 100.0], "rescued": [0, 180, 369, 525, 728, 900, 1046, 1186, 1364, 1513, 1635, 1791, 1941, 2092, 2211, 2356, 2510, 2664, 2804, 2958, 3112, 3312, 3491, 3682, 3842, 4035, 4282, 4479, 4746, 5059, 5317, 5566, 5840, 6127, 6414, 6689, 6956, 7217, 7463, 7699, 7986, 8201, 8435, 8615, 8758, 8923, 9066, 9206, 9326, 9433, 9548, 9636, 9698, 9792, 9846, 9900, 9988, 10041, 10096, 10162, 10204, 10257, 10305, 10338, 10394, 10418, 10444, 10468, 10496, 10513, 10547, 10559, 10576, 10587, 10608, 10625, 10637, 10645, 10654, 10666, 10671, 10681, 10687, 10688, 10693, 10697, 10699, 10704, 10708, 10711, 10715, 10718, 10718, 10721, 10722, 10722, 10722, 10723, 10723, 10723, 10723], "harmed": [0, 0, 2, 4, 6, 6, 7, 11, 11, 11, 11, 15, 18, 20, 22, 24, 26, 30, 34, 36, 42, 50, 53, 61, 68, 75, 94, 108, 130, 157, 179, 208, 242, 278, 320, 370, 408, 452, 501, 561, 612, 665, 723, 786, 833, 891, 946, 1003, 1038, 1078, 1125, 1163, 1195, 1235, 1267, 1308, 1354, 1388, 1429, 1464, 1498, 1533, 1575, 1606, 1631, 1660, 1678, 1709, 1730, 1747, 1782, 1803, 1828, 1843, 1867, 1892, 1924, 1947, 1962, 1992, 2018, 2038, 2062, 2091, 2119, 2138, 2169, 2199, 2231, 2253, 2287, 2313, 2330, 2357, 2380, 2393, 2415, 2430, 2448, 2471, 2500], "exact_score": [26.688029946402747, 27.300570002552256, 27.88758755636219, 28.480276777358704, 29.038935995235803, 29.62311771545246, 30.13356776224371, 30.68088364574767, 31.233871196438194, 31.724470408076456, 32.20939795252814, 32.66313132745371, 33.09701386722627, 33.54507557496527, 34.00448061707739, 34.41000482091711, 34.80702152397698, 35.21254572781669, 35.63508493321612, 36.12568414485438, 36.55956668462695, 36.9594192212801, 37.37345092589967, 37.80733346567224, 38.210021835918674, 38.615546039758385, 39.0040552420384, 39.406743612284835, 39.79525281456484, 40.19510535121799, 40.611972889430845, 40.98630292374443, 41.366304625244595, 41.6810821540992, 42.00436718373366, 42.370189717267394, 42.72466891642799, 43.05929728043558, 43.41661231318946, 43.77959901312991, 44.031988202932254, 44.34392989819358, 44.658707427048185, 44.9536341207498, 45.2882624847574, 45.566174176899295, 45.89796670731362, 46.15035589711596, 46.484984261123564, 46.779910954825176, 46.98125513994839, 47.296032668802994, 47.60797436406432, 47.851856053086806, 48.07021523976973, 48.314096928792225, 48.549471117034855, 48.72529279981851, 48.92947281853501, 49.15917533959108, 49.283952017695604, 49.45977370047926, 49.61007288092335, 49.774551229333866, 49.94470124493095, 50.13186626208774, 50.29067294331169, 50.44097212375578, 50.5855996370133, 50.72455548308425, 50.79828715650966, 50.937243002580615, 50.97410883929331, 51.05351217990528, 51.13007968692398, 51.21515469472251, 51.24067719706208, 51.32291637126733, 51.373961375946465, 51.38814054391288, 51.45336471655844, 51.45052888296515, 51.54694722513683, 51.5526188923234, 51.52993222357713, 51.53276805717041, 51.54694722513683, 51.50440972123755, 51.459036383745, 51.45052888296515, 51.382468876726314, 51.311573036894195, 51.22082636190909, 51.135751354110546, 50.99963134163288, 50.849332161188784, 50.69336131355813, 50.540226299520754, 50.398434619856516, 50.25380710659899, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 44.232538607166646, "threshold_area100": 44.96363043416613, "threshold_area25": 32.30298046110654}, "selected_budget": 36, "selected": {"val": {"score": 68.69070208728652, "calls": 35.99690772366294, "rescued": 1994, "harmed": 196}, "test": {"score": 45.257068315231265, "calls": 43.91288319201429, "rescued": 6956, "harmed": 408}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.0899109840393066, "operator": "ge"}, {"budget": 0.02, "threshold": 2.5191636085510254, "operator": "ge"}, {"budget": 0.03, "threshold": 2.1803345680236816, "operator": "ge"}, {"budget": 0.04, "threshold": 1.9274165630340576, "operator": "ge"}, {"budget": 0.05, "threshold": 1.725083351135254, "operator": "gt"}, {"budget": 0.06, "threshold": 1.5655643939971924, "operator": "ge"}, {"budget": 0.07, "threshold": 1.4527920484542847, "operator": "gt"}, {"budget": 0.08, "threshold": 1.3491191864013672, "operator": "ge"}, {"budget": 0.09, "threshold": 1.2678898572921753, "operator": "ge"}, {"budget": 0.1, "threshold": 1.2001237869262695, "operator": "ge"}, {"budget": 0.11, "threshold": 1.131476879119873, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0650619268417358, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0057083368301392, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9610939025878906, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9041269421577454, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8567029237747192, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8208816647529602, "operator": "ge"}, {"budget": 0.18, "threshold": 0.7852263450622559, "operator": "ge"}, {"budget": 0.19, "threshold": 0.7500686645507812, "operator": "ge"}, {"budget": 0.2, "threshold": 0.7156976461410522, "operator": "ge"}, {"budget": 0.21, "threshold": 0.687253475189209, "operator": "gt"}, {"budget": 0.22, "threshold": 0.6570569276809692, "operator": "ge"}, {"budget": 0.23, "threshold": 0.6310487985610962, "operator": "ge"}, {"budget": 0.24, "threshold": 0.6053711175918579, "operator": "ge"}, {"budget": 0.25, "threshold": 0.5794486999511719, "operator": "ge"}, {"budget": 0.26, "threshold": 0.5528364181518555, "operator": "ge"}, {"budget": 0.27, "threshold": 0.529852032661438, "operator": "ge"}, {"budget": 0.28, "threshold": 0.5074293613433838, "operator": "ge"}, {"budget": 0.29, "threshold": 0.49006569385528564, "operator": "ge"}, {"budget": 0.3, "threshold": 0.46957409381866455, "operator": "ge"}, {"budget": 0.31, "threshold": 0.44908279180526733, "operator": "ge"}, {"budget": 0.32, "threshold": 0.42779970169067383, "operator": "ge"}, {"budget": 0.33, "threshold": 0.4117307662963867, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3935176134109497, "operator": "ge"}, {"budget": 0.35, "threshold": 0.37679970264434814, "operator": "ge"}, {"budget": 0.36, "threshold": 0.35904932022094727, "operator": "ge"}, {"budget": 0.37, "threshold": 0.341269850730896, "operator": "ge"}, {"budget": 0.38, "threshold": 0.3246959447860718, "operator": "gt"}, {"budget": 0.39, "threshold": 0.3104187250137329, "operator": "ge"}, {"budget": 0.4, "threshold": 0.29604291915893555, "operator": "gt"}, {"budget": 0.41, "threshold": 0.2815293073654175, "operator": "ge"}, {"budget": 0.42, "threshold": 0.2658398151397705, "operator": "ge"}, {"budget": 0.43, "threshold": 0.2511870861053467, "operator": "ge"}, {"budget": 0.44, "threshold": 0.23784005641937256, "operator": "ge"}, {"budget": 0.45, "threshold": 0.22579574584960938, "operator": "ge"}, {"budget": 0.46, "threshold": 0.21322864294052124, "operator": "ge"}, {"budget": 0.47, "threshold": 0.20213675498962402, "operator": "ge"}, {"budget": 0.48, "threshold": 0.19105219841003418, "operator": "gt"}, {"budget": 0.49, "threshold": 0.17970609664916992, "operator": "ge"}, {"budget": 0.5, "threshold": 0.1677914261817932, "operator": "ge"}, {"budget": 0.51, "threshold": 0.1572604477405548, "operator": "ge"}, {"budget": 0.52, "threshold": 0.14697718620300293, "operator": "ge"}, {"budget": 0.53, "threshold": 0.1372089385986328, "operator": "ge"}, {"budget": 0.54, "threshold": 0.12676310539245605, "operator": "ge"}, {"budget": 0.55, "threshold": 0.1172260046005249, "operator": "ge"}, {"budget": 0.56, "threshold": 0.10522466897964478, "operator": "ge"}, {"budget": 0.57, "threshold": 0.09546405076980591, "operator": "ge"}, {"budget": 0.58, "threshold": 0.08483052253723145, "operator": "ge"}, {"budget": 0.59, "threshold": 0.07575666904449463, "operator": "ge"}, {"budget": 0.6, "threshold": 0.06642031669616699, "operator": "ge"}, {"budget": 0.61, "threshold": 0.057012200355529785, "operator": "ge"}, {"budget": 0.62, "threshold": 0.0479583740234375, "operator": "ge"}, {"budget": 0.63, "threshold": 0.03758525848388672, "operator": "ge"}, {"budget": 0.64, "threshold": 0.027888059616088867, "operator": "ge"}, {"budget": 0.65, "threshold": 0.019515514373779297, "operator": "gt"}, {"budget": 0.66, "threshold": 0.009775161743164062, "operator": "ge"}, {"budget": 0.67, "threshold": -0.009547755122184753, "operator": "ge"}, {"budget": 0.68, "threshold": -0.049600839614868164, "operator": "ge"}, {"budget": 0.69, "threshold": -0.10502105951309204, "operator": "ge"}, {"budget": 0.7, "threshold": -0.17410266399383545, "operator": "ge"}, {"budget": 0.71, "threshold": -0.2589174509048462, "operator": "ge"}, {"budget": 0.72, "threshold": -0.36075085401535034, "operator": "ge"}, {"budget": 0.73, "threshold": -0.4586300849914551, "operator": "ge"}, {"budget": 0.74, "threshold": -0.5729941725730896, "operator": "ge"}, {"budget": 0.75, "threshold": -0.683661937713623, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8145142793655396, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9485230445861816, "operator": "ge"}, {"budget": 0.78, "threshold": -1.0928168296813965, "operator": "ge"}, {"budget": 0.79, "threshold": -1.2459509372711182, "operator": "ge"}, {"budget": 0.8, "threshold": -1.4497835636138916, "operator": "ge"}, {"budget": 0.81, "threshold": -1.6282647848129272, "operator": "ge"}, {"budget": 0.82, "threshold": -1.826770305633545, "operator": "ge"}, {"budget": 0.83, "threshold": -2.040374517440796, "operator": "ge"}, {"budget": 0.84, "threshold": -2.2734920978546143, "operator": "ge"}, {"budget": 0.85, "threshold": -2.4576964378356934, "operator": "ge"}, {"budget": 0.86, "threshold": -2.6702218055725098, "operator": "ge"}, {"budget": 0.87, "threshold": -2.9192421436309814, "operator": "ge"}, {"budget": 0.88, "threshold": -3.1865458488464355, "operator": "ge"}, {"budget": 0.89, "threshold": -3.443434238433838, "operator": "ge"}, {"budget": 0.9, "threshold": -3.7480368614196777, "operator": "ge"}, {"budget": 0.91, "threshold": -4.116477966308594, "operator": "ge"}, {"budget": 0.92, "threshold": -4.491091251373291, "operator": "ge"}, {"budget": 0.93, "threshold": -4.8521728515625, "operator": "ge"}, {"budget": 0.94, "threshold": -5.316144943237305, "operator": "ge"}, {"budget": 0.95, "threshold": -5.8583269119262695, "operator": "ge"}, {"budget": 0.96, "threshold": -6.639556884765625, "operator": "ge"}, {"budget": 0.97, "threshold": -7.432790756225586, "operator": "ge"}, {"budget": 0.98, "threshold": -8.464765548706055, "operator": "ge"}, {"budget": 0.99, "threshold": -10.424098014831543, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.56757326586548, 57.09466582331857, 57.53039567081313, 58.0715440297983, 58.359687961205985, 58.711082999508044, 59.06247803781011, 59.4209009768782, 59.70201700751985, 60.09557945041816, 60.45400238948626, 60.728090519361864, 61.016234450769545, 61.24112727528287, 61.44493639749806, 61.66280132124534, 61.87363834422658, 62.14069857333614, 62.29531239018905, 62.527233115468405, 62.773209642279845, 63.04729777215545, 63.27921849743481, 63.49708342118209, 63.69386464263124, 63.82036685641999, 63.93984116944268, 64.08742708552954, 64.29123620774475, 64.37557101693724, 64.53721273455618, 64.65668704757888, 64.87455197132616, 64.94483097898657, 65.12755639890364, 65.16972380349989, 65.28217021575655, 65.38758872724716, 65.41570033031134, 65.50003513950382, 65.53517464333403, 65.50003513950382, 65.57734204793027, 65.64059315482464, 65.66870475788882, 65.68978846018693, 65.78115117014546, 65.78115117014546, 65.77412326937943, 65.78115117014546, 65.82331857474172, 65.89359758240214, 65.87251388010401, 65.91468128470026, 65.93576498699838, 65.96387659006254, 65.97090449082859, 65.91468128470026, 65.86548597933796, 65.83034647550777, 65.8022348724436, 65.78817907091151, 65.66870475788882, 65.68978846018693, 65.64059315482464, 65.54923044486611, 65.42272823107737, 65.26811441422447, 65.1556680019678, 65.0361936889451, 65.00808208588094, 65.0361936889451, 64.97294258205073, 64.91671937592241, 64.87455197132616, 64.86049616979408, 64.82535666596388, 64.78318926136762, 64.75507765830346, 64.71993815447325, 64.75507765830346, 64.70588235294117, 64.67074284911097, 64.62857544451472, 64.60046384145055, 64.57235223838639, 64.57235223838639, 64.56532433762034, 64.55126853608827, 64.50910113149202, 64.50207323072597, 64.47396162766181, 64.44585002459765, 64.38259891770328, 64.34745941387308, 64.30529200927683, 64.2842083069787, 64.24906880314849, 64.17878979548807, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.9898095438892405, 5.994799353433129, 6.992761262210978, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.992339588165017, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.992831541218635, 38.99782135076253, 39.988755358774334, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.986506430529204, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.98699838358283, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 91, 178, 255, 342, 401, 463, 530, 592, 646, 715, 778, 833, 886, 934, 975, 1024, 1067, 1115, 1160, 1209, 1256, 1308, 1361, 1402, 1447, 1484, 1515, 1557, 1598, 1630, 1672, 1707, 1751, 1780, 1818, 1844, 1881, 1911, 1936, 1962, 1982, 2003, 2026, 2048, 2071, 2094, 2121, 2137, 2160, 2177, 2197, 2219, 2235, 2254, 2277, 2299, 2316, 2322, 2335, 2345, 2354, 2370, 2377, 2393, 2403, 2409, 2414, 2421, 2432, 2442, 2451, 2465, 2473, 2484, 2493, 2503, 2512, 2519, 2530, 2536, 2549, 2553, 2558, 2564, 2569, 2573, 2577, 2583, 2585, 2586, 2587, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589], "harmed": [0, 18, 30, 45, 55, 73, 85, 102, 113, 127, 140, 152, 168, 180, 196, 208, 226, 239, 249, 272, 288, 300, 313, 333, 343, 360, 379, 393, 414, 426, 446, 465, 483, 496, 515, 527, 547, 568, 583, 604, 618, 633, 659, 671, 684, 703, 723, 737, 753, 777, 793, 807, 819, 838, 851, 871, 889, 905, 919, 939, 954, 967, 985, 1009, 1022, 1039, 1058, 1081, 1110, 1137, 1164, 1177, 1187, 1204, 1223, 1238, 1250, 1264, 1277, 1292, 1303, 1311, 1322, 1332, 1344, 1353, 1361, 1365, 1372, 1376, 1383, 1385, 1391, 1395, 1404, 1409, 1415, 1418, 1423, 1433, 1441], "exact_score": [56.05453650994448, 56.56757326586548, 57.09466582331857, 57.53039567081313, 58.0715440297983, 58.359687961205985, 58.711082999508044, 59.06950593857614, 59.4209009768782, 59.70201700751985, 60.09557945041816, 60.45400238948626, 60.728090519361864, 61.016234450769545, 61.24112727528287, 61.44493639749806, 61.66280132124534, 61.87363834422658, 62.14069857333614, 62.29531239018905, 62.527233115468405, 62.78023754304589, 63.04729777215545, 63.27921849743481, 63.49708342118209, 63.69386464263124, 63.82036685641999, 63.93984116944268, 64.08742708552954, 64.29123620774475, 64.37557101693724, 64.53721273455618, 64.65668704757888, 64.87455197132616, 64.94483097898657, 65.12755639890364, 65.16972380349989, 65.28217021575655, 65.38758872724716, 65.41570033031134, 65.50003513950382, 65.53517464333403, 65.50003513950382, 65.57734204793027, 65.64059315482464, 65.66870475788882, 65.68978846018693, 65.78115117014546, 65.78115117014546, 65.77412326937943, 65.78115117014546, 65.82331857474172, 65.89359758240214, 65.87251388010401, 65.91468128470026, 65.93576498699838, 65.96387659006254, 65.97090449082859, 65.91468128470026, 65.86548597933796, 65.83034647550777, 65.8022348724436, 65.78817907091151, 65.66870475788882, 65.68978846018693, 65.64059315482464, 65.54923044486611, 65.42272823107737, 65.26811441422447, 65.1556680019678, 65.0361936889451, 65.00808208588094, 65.0361936889451, 64.97294258205073, 64.91671937592241, 64.87455197132616, 64.86049616979408, 64.82535666596388, 64.78318926136762, 64.75507765830346, 64.71993815447325, 64.75507765830346, 64.70588235294117, 64.67074284911097, 64.62857544451472, 64.60046384145055, 64.57235223838639, 64.57235223838639, 64.56532433762034, 64.55126853608827, 64.50910113149202, 64.50207323072597, 64.47396162766181, 64.44585002459765, 64.38259891770328, 64.34745941387308, 64.30529200927683, 64.2842083069787, 64.24906880314849, 64.17878979548807, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 63.926201737321364, "threshold_area100": 63.925785367910606, "threshold_area25": 60.460327500175694}, "test": {"score": [26.688029946402747, 28.398037603153448, 29.6344610498256, 30.697898647307376, 31.63655956668463, 32.501488812636474, 33.33805972265548, 34.08671979128265, 34.787170688823984, 35.453591583245895, 36.01792246830956, 36.718373365850894, 37.31389842044069, 37.98599098204918, 38.47091852650087, 39.1515185888892, 39.72152114113944, 40.16107534809857, 40.59779372146443, 41.18481127527437, 41.5874996455208, 42.01287468451351, 42.494966395371925, 42.863624762498944, 43.30885063664465, 43.771091512350054, 44.22482488727562, 44.64736409267505, 45.00467912542893, 45.322292487876815, 45.64274168391799, 45.96602671355245, 46.297819243966764, 46.56722343532882, 46.890508464963276, 47.21379349459774, 47.50872018829935, 47.80648271559425, 48.132603578822, 48.38215693503105, 48.56648611859457, 48.78484530527749, 49.02589116070669, 49.283952017695604, 49.4711170348524, 49.686640387942035, 49.86813373791226, 49.970223747270516, 50.0269404191362, 50.16306043161387, 50.36156878314381, 50.509032129994615, 50.613957972946146, 50.73022715027082, 50.818137991662645, 50.92873550180076, 51.076198848651565, 51.17545302441653, 51.29172220174121, 51.37112554235318, 51.4420213821853, 51.46470805093158, 51.49023055327113, 51.5157530556107, 51.50440972123755, 51.53843972435698, 51.53843972435698, 51.52709638998384, 51.52426055639056, 51.49873805405099, 51.49873805405099, 51.46754388452486, 51.45052888296515, 51.42500638062559, 51.41082721265916, 51.38814054391288, 51.36261804157331, 51.33709553923376, 51.29739386892778, 51.24918469784193, 51.20097552675609, 51.16127385645011, 51.076198848651565, 51.02231801037915, 50.95142217054703, 50.88619799790148, 50.82097382525593, 50.74157048464396, 50.645152142472284, 50.59410713779315, 50.537390465927466, 50.46649462609534, 50.42395712219607, 50.38709128548338, 50.3445537815841, 50.30201627768483, 50.23395627144599, 50.18574710036015, 50.146045430054166, 50.10067209256162, 50.00708958398321], "calls": [0.0, 2.2062785355755326, 3.9134503587329497, 5.41927799676715, 6.964807305107336, 8.50182911266767, 10.004820917108583, 11.434081048124096, 12.752743669001504, 14.026032952386355, 15.18305305844653, 16.595298187902333, 17.90828914159317, 19.456654283526643, 20.454867708362872, 21.901142840938093, 23.265178799308057, 24.322944729603265, 25.528174006749282, 27.002807475257352, 28.210872585996654, 29.393415194396393, 30.68655531293424, 31.78402291353543, 33.07716303207327, 34.44403482403652, 35.88747412301846, 37.115390068910756, 38.31211184527692, 39.32734027167285, 40.50421121288603, 41.81720216657687, 43.06780478121544, 44.09154070839123, 45.293934151943965, 46.36871508379888, 47.60513853047103, 48.747979468564786, 50.00708958398321, 51.11873635255083, 52.22754728752517, 53.350537390465924, 54.58128916995151, 55.79502594787738, 56.835776876612876, 57.81697529988941, 58.85205456143834, 59.76519297847602, 60.75773473612568, 61.926098176558995, 63.22490996228341, 64.31387006210475, 65.28372515100814, 66.26775940787795, 67.30567450302016, 68.50239627938632, 69.89195474009585, 71.11136318520829, 72.3562941326603, 73.51615007231376, 74.76958852054561, 76.17616198281485, 77.47497376853926, 78.99781640813318, 80.45259904148824, 81.6521566514477, 83.05589428012364, 84.00873436746733, 84.74605110172135, 85.47202450160225, 86.0930720585316, 86.81620962481922, 87.47412301846128, 87.9703938972861, 88.60562062218189, 89.1075631681933, 89.63786405013754, 90.20786660238778, 90.79204832260443, 91.33369253892182, 91.92638175991833, 92.51056348013499, 93.02101352692624, 93.48892606981822, 94.07310779003488, 94.54669200011342, 95.04012704534497, 95.50803958823695, 96.03266880299464, 96.45520800839407, 96.87207554660692, 97.32013725434592, 97.71715395740578, 98.06879732297308, 98.44312735728667, 98.7720840541077, 99.13790658764144, 99.39596744463034, 99.57746079460057, 99.79865581487678, 100.0], "rescued": [0, 642, 1106, 1509, 1871, 2216, 2546, 2836, 3114, 3368, 3590, 3869, 4107, 4380, 4571, 4839, 5068, 5242, 5423, 5663, 5828, 6010, 6207, 6361, 6546, 6742, 6931, 7104, 7250, 7383, 7527, 7667, 7811, 7922, 8061, 8198, 8327, 8452, 8588, 8703, 8800, 8899, 9005, 9117, 9203, 9294, 9374, 9436, 9489, 9558, 9648, 9717, 9779, 9839, 9892, 9958, 10035, 10095, 10155, 10212, 10260, 10303, 10335, 10373, 10401, 10428, 10453, 10468, 10488, 10506, 10528, 10550, 10565, 10578, 10595, 10606, 10623, 10636, 10650, 10658, 10667, 10676, 10681, 10687, 10693, 10699, 10703, 10708, 10708, 10711, 10712, 10715, 10719, 10720, 10721, 10722, 10722, 10723, 10723, 10723, 10723], "harmed": [0, 39, 67, 95, 126, 166, 201, 227, 258, 277, 300, 332, 360, 396, 416, 444, 472, 491, 518, 551, 574, 606, 633, 657, 685, 718, 747, 771, 791, 812, 843, 869, 896, 912, 937, 960, 985, 1005, 1026, 1053, 1085, 1107, 1128, 1149, 1169, 1184, 1200, 1226, 1259, 1280, 1300, 1317, 1342, 1361, 1383, 1410, 1435, 1460, 1479, 1508, 1531, 1566, 1589, 1618, 1650, 1665, 1690, 1709, 1730, 1757, 1779, 1812, 1833, 1855, 1877, 1896, 1922, 1944, 1972, 1997, 2023, 2046, 2081, 2106, 2137, 2166, 2193, 2226, 2260, 2281, 2302, 2330, 2349, 2363, 2379, 2395, 2419, 2437, 2451, 2467, 2500], "exact_score": [26.688029946402747, 27.499078354082187, 28.25057425630264, 28.98505515696339, 29.68834188809801, 30.38312111845277, 31.03819867850155, 31.662082069024187, 32.24626378924085, 32.787908005558236, 33.33805972265548, 33.862688937413154, 34.3561239826447, 34.90343986614866, 35.44224824887276, 35.93001162691773, 36.420610838555994, 36.922553384567394, 37.37912259308624, 37.80166179848566, 38.25255933981794, 38.72330771630321, 39.18271275841534, 39.62793863256104, 40.0448061707739, 40.40212120352778, 40.79063040580779, 41.18481127527437, 41.52227547287525, 41.89376967359556, 42.22839803760316, 42.591384737543606, 42.92884893514449, 43.27482063352522, 43.6151206647194, 43.94691319513371, 44.258854890395035, 44.61333408955562, 44.90542494966395, 45.22020247851856, 45.50662167144032, 45.77886169639566, 46.03408671979128, 46.283640076000346, 46.553044267362395, 46.81961262513116, 47.11170348523949, 47.3499135070754, 47.60513853047104, 47.86887105464651, 48.12976774522871, 48.35663443269149, 48.552306950628136, 48.72529279981851, 48.88977114822902, 49.10529450131867, 49.320817854408304, 49.49380370359867, 49.714998723874885, 49.87380540509883, 49.99291041601679, 50.05813458866234, 50.18291126676687, 50.333210447210966, 50.46649462609534, 50.574256302640165, 50.69052547996484, 50.79828715650966, 50.85783966196864, 51.0138105095993, 51.0875421830247, 51.16694552363668, 51.24918469784193, 51.33425970564047, 51.39948387828602, 51.45336471655844, 51.459036383745, 51.47321555171143, 51.52993222357713, 51.5157530556107, 51.50724555483084, 51.50724555483084, 51.53843972435698, 51.53843972435698, 51.52709638998384, 51.52142472279727, 51.49873805405099, 51.47037971811814, 51.42217054703231, 51.38814054391288, 51.351274707200176, 51.28321470096135, 51.19530385956952, 51.07903468224485, 50.95142217054703, 50.8266454924425, 50.645152142472284, 50.511867963587896, 50.39276295266995, 50.25947877378556, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.05612067876949, "threshold_area100": 46.66734254033974, "threshold_area25": 36.897598048946485}, "selected_budget": 57, "selected": {"val": {"score": 65.97090449082859, "calls": 56.99627521259399, "rescued": 2316, "harmed": 905}, "test": {"score": 51.17545302441653, "calls": 71.11136318520829, "rescued": 10095, "harmed": 1460}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 65.54923044486611, "calls": 44.9996486049617}, "test": {"score": 50.33037461361768, "calls": 58.006976150639474}}, "old_normalized_interior": {"val": {"score": 65.54923044486611, "calls": 44.9996486049617}, "test": {"score": 50.33037461361768, "calls": 58.006976150639474}}, "old_normalized_endpoints": {"val": {"score": 65.54923044486611, "calls": 44.9996486049617}, "test": {"score": 50.33037461361768, "calls": 58.006976150639474}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 56.79949399114484, 57.46714456391875, 57.94504181600956, 58.47213437346264, 58.928947923255315, 59.37170567151592, 59.83554712207463, 60.17991425961065, 60.580504603274996, 60.98109494693934, 61.35357368753953, 61.63468971818118, 61.92283364958886, 62.18286597793239, 62.44992620704195, 62.766181741513805, 63.012158268325244, 63.23002319207253, 63.35652540586126, 63.518167123480204, 63.70792044416332, 63.98200857403893, 64.12256658935975, 64.22095720008433, 64.37557101693724, 64.54424063532223, 64.74804975753742, 64.89563567362428, 65.06430529200927, 65.16972380349989, 65.24703071192634, 65.24703071192634, 65.33136552111884, 65.3946166280132, 65.45786773490758, 65.52111884180195, 65.50706304026987, 65.45083983414153, 65.46489563567361, 65.45786773490758, 65.45083983414153, 65.43678403260945, 65.45786773490758, 65.48597933797174, 65.54923044486611, 65.54923044486611, 65.53517464333403, 65.50706304026987, 65.50706304026987, 65.4789514372057, 65.43678403260945, 65.40164452877924, 65.33839342188487, 65.34542132265092, 65.37353292571508, 65.36650502494903, 65.26108651345842, 65.28919811652258, 65.24703071192634, 65.23297491039426, 65.2048633073301, 65.17675170426594, 65.10647269660552, 65.01510998664698, 64.93780307822054, 64.89563567362428, 64.81130086443179, 64.76913345983554, 64.74102185677137, 64.73399395600535, 64.6918265514091, 64.67777074987701, 64.67777074987701, 64.67777074987701, 64.67777074987701, 64.67074284911097, 64.67074284911097, 64.67074284911097, 64.66371494834493, 64.64965914681284, 64.65668704757888, 64.64965914681284, 64.62857544451472, 64.60046384145055, 64.60046384145055, 64.56532433762034, 64.53018483379014, 64.51612903225806, 64.45990582612973, 64.43882212383161, 64.42476632229952, 64.32637571157494, 64.3544873146391, 64.29826410851078, 64.18581769625412, 64.164733993956, 64.14365029165788, 64.06634338323143, 64.05931548246538, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.992550425187996, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.98826340572071, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.98489001335301, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.98763089465177, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 56.79949399114484, 57.46714456391875, 57.94504181600956, 58.47213437346264, 58.928947923255315, 59.37170567151592, 59.83554712207463, 60.17991425961065, 60.580504603274996, 60.98109494693934, 61.35357368753953, 61.63468971818118, 61.92283364958886, 62.189893878698435, 62.44992620704195, 62.766181741513805, 63.012158268325244, 63.23002319207253, 63.35652540586126, 63.518167123480204, 63.70792044416332, 63.98200857403893, 64.12256658935975, 64.22095720008433, 64.37557101693724, 64.54424063532223, 64.74804975753742, 64.89563567362428, 65.06430529200927, 65.16972380349989, 65.24703071192634, 65.24703071192634, 65.33136552111884, 65.3946166280132, 65.47192353643966, 65.52111884180195, 65.50706304026987, 65.45083983414153, 65.46489563567361, 65.45786773490758, 65.45083983414153, 65.43678403260945, 65.45786773490758, 65.48597933797174, 65.54923044486611, 65.54923044486611, 65.53517464333403, 65.50706304026987, 65.50706304026987, 65.4789514372057, 65.43678403260945, 65.40164452877924, 65.33839342188487, 65.34542132265092, 65.37353292571508, 65.36650502494903, 65.26108651345842, 65.28919811652258, 65.24703071192634, 65.23297491039426, 65.2048633073301, 65.17675170426594, 65.10647269660552, 65.01510998664698, 64.93780307822054, 64.89563567362428, 64.81130086443179, 64.76913345983554, 64.74102185677137, 64.73399395600535, 64.6918265514091, 64.67777074987701, 64.67777074987701, 64.67777074987701, 64.67777074987701, 64.67074284911097, 64.67074284911097, 64.67074284911097, 64.66371494834493, 64.64965914681284, 64.65668704757888, 64.64965914681284, 64.62857544451472, 64.60046384145055, 64.60046384145055, 64.56532433762034, 64.53018483379014, 64.51612903225806, 64.45990582612973, 64.43882212383161, 64.42476632229952, 64.32637571157494, 64.3544873146391, 64.29826410851078, 64.18581769625412, 64.164733993956, 64.14365029165788, 64.06634338323143, 64.05931548246538, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 61.21863799283154, "exact_area100": 64.03076256733891}, "test": {"score": [26.688029946402747, 27.998185066500298, 29.51819187250092, 30.890735331650742, 31.87760542211383, 32.889998014916486, 33.797464764767604, 34.77299152085755, 35.564189093384, 36.35822249950373, 37.09553923375777, 37.82151263363866, 38.52479936477328, 39.24793693106089, 39.86047698721039, 40.438987040240484, 40.98630292374443, 41.462722967416276, 42.01854635170008, 42.5176530641182, 42.98556560701018, 43.42228398037603, 43.873181521708304, 44.46019907551825, 44.93661911919009, 45.24572498085813, 45.56901001049258, 45.86677253778748, 46.24110257210107, 46.527521765022826, 46.90185179933641, 47.16274848991861, 47.46618268440008, 47.82633355074724, 48.09573774210929, 48.410515270963906, 48.716785299038655, 48.986189490400704, 49.2300711794232, 49.40022119502028, 49.5448487082778, 49.73484955902788, 49.8993279074384, 50.05529875506905, 50.19425460114001, 50.33037461361768, 50.46365879250205, 50.642316308878996, 50.78694382213652, 50.90888466664776, 51.042168845532146, 51.113064685364265, 51.135751354110546, 51.20664719394266, 51.2747072001815, 51.30022970252106, 51.34560304001361, 51.39948387828602, 51.44485721577858, 51.46187221733829, 51.51291722201742, 51.51858888920399, 51.52709638998384, 51.53843972435698, 51.5157530556107, 51.50724555483084, 51.51291722201742, 51.51858888920399, 51.50440972123755, 51.47037971811814, 51.45336471655844, 51.42784221421888, 51.41082721265916, 51.40515554547259, 51.40231971187931, 51.39664804469274, 51.39381221109945, 51.39097637750617, 51.38814054391288, 51.382468876726314, 51.373961375946465, 51.373961375946465, 51.356946374386744, 51.351274707200176, 51.348438873606895, 51.33709553923376, 51.33142387204719, 51.32575220486062, 51.31440887048748, 51.30590136970763, 51.29455803533449, 51.26619969940164, 51.23500552987551, 51.1981396931628, 51.15276635567025, 51.1272438533307, 51.073363015058284, 50.97127300570003, 50.849332161188784, 50.65365964325213, 50.00708958398321], "calls": [0.0, 1.5937384794260274, 3.635538666591044, 5.456143833479851, 6.879732297308793, 8.428097439242265, 9.919745909310041, 11.470946884836799, 12.829311176020191, 14.264242974222272, 15.727533108357203, 17.074554065167455, 18.60023253835465, 19.95008932875819, 21.14113943793778, 22.434279556475627, 23.63950883362164, 24.82205144202138, 26.126534894932362, 27.31758500411196, 28.59087428749681, 29.88401440603465, 31.157303689419503, 32.671638828233554, 34.117913960808785, 35.13030655361143, 36.23628165499249, 37.34225675637354, 38.649576042877804, 39.90868615829623, 41.11958710262882, 42.18018886651731, 43.45064231630888, 44.82034994186541, 45.96602671355245, 47.188270992258175, 48.3169327623855, 49.488132036412104, 50.60545047216629, 51.52142472279727, 52.55933981793949, 53.72203159118622, 54.82517085897399, 55.78368261350424, 56.84712021098603, 58.006976150639474, 59.04772707937498, 60.122508011229904, 61.25967728213708, 62.249383206193464, 63.42341831381334, 64.44148257380256, 65.53327850721719, 66.557014434393, 67.58642202875535, 68.38612710206165, 69.24822051442021, 70.33434478064827, 71.31554320392479, 72.30808496157447, 73.41122422936223, 74.58809517057539, 75.67989110399002, 76.87661288035618, 78.17826049967388, 79.27572810027507, 80.17752318293962, 81.06230326404446, 81.77126166236566, 82.68723591299663, 83.42171681365738, 84.32351189632192, 86.21217706944957, 87.93636389416669, 88.696367297167, 89.2351756798911, 89.71443155715623, 90.08592575787652, 90.4063749539177, 90.75801831948502, 91.01891501006722, 91.29115503502254, 91.60593256387716, 91.90936675835863, 92.17593511612738, 92.37727930125061, 92.63250432464622, 92.90474434960157, 93.12877520347106, 93.40952272920624, 93.7328077588407, 94.07594362362816, 94.4757961602813, 94.91251453364717, 95.3322179054533, 95.80580211553186, 96.3361029974761, 97.05640473017043, 97.70013895584607, 98.46581402603294, 100.0], "exact_score": [26.688029946402747, 27.521765022828465, 28.276096758642204, 29.027592660862663, 29.77908856308312, 30.55043530045657, 31.26506536596433, 31.931486260386244, 32.59790715480816, 33.24731304767036, 33.84567393585345, 34.4865723279358, 35.087769049712165, 35.686129937895245, 36.20225165187307, 36.77509003771659, 37.25434591498171, 37.78748263051924, 38.26390267419108, 38.76300938660919, 39.27345943340045, 39.769730312225285, 40.23480702152398, 40.68854039644953, 41.13093043700196, 41.527947140061826, 41.964665513427676, 42.39571221960695, 42.792728922666825, 43.11601395230128, 43.47616481864844, 43.85049485296203, 44.21631738649576, 44.57930408643621, 44.899753282477384, 45.20885914414542, 45.48960666988061, 45.76751836202252, 46.04826588775771, 46.3091625783399, 46.56155176814225, 46.85080679465729, 47.10603181805292, 47.36692850863511, 47.62782519921731, 47.866035221053224, 48.09857357570258, 48.36514193347134, 48.643053625613256, 48.86424864588946, 49.079771998979105, 49.30096701925532, 49.46828120125911, 49.59589371295693, 49.75753622777416, 49.93902957774438, 50.09500042537505, 50.219777103479565, 50.33037461361768, 50.452315458128915, 50.61112213935286, 50.74157048464396, 50.87201882993506, 50.97978050647989, 51.0676913478717, 51.11873635255083, 51.172617190823246, 51.22933386268894, 51.27754303377478, 51.33425970564047, 51.38814054391288, 51.44485721577858, 51.46470805093158, 51.50440972123755, 51.50440972123755, 51.52142472279727, 51.52142472279727, 51.53276805717041, 51.52426055639056, 51.510081388424126, 51.510081388424126, 51.51858888920399, 51.4959022204577, 51.47321555171143, 51.433513881405446, 51.42784221421888, 51.41082721265916, 51.40799137906588, 51.40231971187931, 51.40231971187931, 51.39381221109945, 51.376797209539745, 51.34560304001361, 51.32575220486062, 51.2747072001815, 51.18963219238295, 51.113064685364265, 50.98261634007317, 50.74724215183053, 50.517539630774465, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 37.87539347191108, "exact_area100": 45.66128936093179}}, "old_normalized_endpoint_budget": 45}, {"seed": 2027, "selection": {"best_epoch": 9, "epochs": 17, "alpha": 0.0, "validation_area100": 0.6422057738030157, "training_seconds": 121.15734731999692, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9041603803634644, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8591342568397522, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8250337839126587, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7967910170555115, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7724076509475708, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7499251365661621, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7285953164100647, "operator": "ge"}, {"budget": 0.08, "threshold": 0.708764374256134, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6905443668365479, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6743236184120178, "operator": "ge"}, {"budget": 0.11, "threshold": 0.659709095954895, "operator": "ge"}, {"budget": 0.12, "threshold": 0.645710825920105, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6310664415359497, "operator": "ge"}, {"budget": 0.14, "threshold": 0.6174618005752563, "operator": "ge"}, {"budget": 0.15, "threshold": 0.605050802230835, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5929194688796997, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5799402594566345, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5670498013496399, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5558100938796997, "operator": "ge"}, {"budget": 0.2, "threshold": 0.5436378121376038, "operator": "ge"}, {"budget": 0.21, "threshold": 0.5320740342140198, "operator": "gt"}, {"budget": 0.22, "threshold": 0.5204253792762756, "operator": "gt"}, {"budget": 0.23, "threshold": 0.5094419717788696, "operator": "ge"}, {"budget": 0.24, "threshold": 0.49679458141326904, "operator": "ge"}, {"budget": 0.25, "threshold": 0.48403018712997437, "operator": "ge"}, {"budget": 0.26, "threshold": 0.47405382990837097, "operator": "gt"}, {"budget": 0.27, "threshold": 0.46293964982032776, "operator": "ge"}, {"budget": 0.28, "threshold": 0.4514990448951721, "operator": "ge"}, {"budget": 0.29, "threshold": 0.44061043858528137, "operator": "gt"}, {"budget": 0.3, "threshold": 0.43097689747810364, "operator": "ge"}, {"budget": 0.31, "threshold": 0.41897478699684143, "operator": "ge"}, {"budget": 0.32, "threshold": 0.40838557481765747, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3971496522426605, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3861183822154999, "operator": "ge"}, {"budget": 0.35, "threshold": 0.3739790618419647, "operator": "ge"}, {"budget": 0.36, "threshold": 0.3625051975250244, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3523077070713043, "operator": "gt"}, {"budget": 0.38, "threshold": 0.34332388639450073, "operator": "ge"}, {"budget": 0.39, "threshold": 0.33103594183921814, "operator": "ge"}, {"budget": 0.4, "threshold": 0.3213382363319397, "operator": "ge"}, {"budget": 0.41, "threshold": 0.31109774112701416, "operator": "ge"}, {"budget": 0.42, "threshold": 0.29935571551322937, "operator": "ge"}, {"budget": 0.43, "threshold": 0.28986531496047974, "operator": "ge"}, {"budget": 0.44, "threshold": 0.27781161665916443, "operator": "ge"}, {"budget": 0.45, "threshold": 0.26680710911750793, "operator": "ge"}, {"budget": 0.46, "threshold": 0.2567208707332611, "operator": "ge"}, {"budget": 0.47, "threshold": 0.24511665105819702, "operator": "ge"}, {"budget": 0.48, "threshold": 0.2330968827009201, "operator": "ge"}, {"budget": 0.49, "threshold": 0.22280633449554443, "operator": "ge"}, {"budget": 0.5, "threshold": 0.21218053996562958, "operator": "ge"}, {"budget": 0.51, "threshold": 0.20134106278419495, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1911592334508896, "operator": "ge"}, {"budget": 0.53, "threshold": 0.17767000198364258, "operator": "ge"}, {"budget": 0.54, "threshold": 0.166120246052742, "operator": "ge"}, {"budget": 0.55, "threshold": 0.15467528998851776, "operator": "ge"}, {"budget": 0.56, "threshold": 0.143578439950943, "operator": "ge"}, {"budget": 0.57, "threshold": 0.1328301578760147, "operator": "ge"}, {"budget": 0.58, "threshold": 0.12371194362640381, "operator": "ge"}, {"budget": 0.59, "threshold": 0.11189108341932297, "operator": "ge"}, {"budget": 0.6, "threshold": 0.10127071291208267, "operator": "ge"}, {"budget": 0.61, "threshold": 0.09120266139507294, "operator": "ge"}, {"budget": 0.62, "threshold": 0.0801340714097023, "operator": "ge"}, {"budget": 0.63, "threshold": 0.06878426671028137, "operator": "ge"}, {"budget": 0.64, "threshold": 0.05845651775598526, "operator": "ge"}, {"budget": 0.65, "threshold": 0.04533154517412186, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03502146154642105, "operator": "ge"}, {"budget": 0.67, "threshold": 0.022149808704853058, "operator": "ge"}, {"budget": 0.68, "threshold": 0.00919191725552082, "operator": "ge"}, {"budget": 0.69, "threshold": 0.00029996829107403755, "operator": "ge"}, {"budget": 0.7, "threshold": -0.0009712062892504036, "operator": "ge"}, {"budget": 0.71, "threshold": -0.00173279014416039, "operator": "ge"}, {"budget": 0.72, "threshold": -0.002569451928138733, "operator": "ge"}, {"budget": 0.73, "threshold": -0.003619840368628502, "operator": "ge"}, {"budget": 0.74, "threshold": -0.004838157445192337, "operator": "ge"}, {"budget": 0.75, "threshold": -0.006142979487776756, "operator": "ge"}, {"budget": 0.76, "threshold": -0.007424978073686361, "operator": "ge"}, {"budget": 0.77, "threshold": -0.009142988361418247, "operator": "ge"}, {"budget": 0.78, "threshold": -0.010955432429909706, "operator": "ge"}, {"budget": 0.79, "threshold": -0.013057566247880459, "operator": "ge"}, {"budget": 0.8, "threshold": -0.015146724879741669, "operator": "ge"}, {"budget": 0.81, "threshold": -0.017793286591768265, "operator": "ge"}, {"budget": 0.82, "threshold": -0.02160523273050785, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0252833291888237, "operator": "ge"}, {"budget": 0.84, "threshold": -0.02932233177125454, "operator": "ge"}, {"budget": 0.85, "threshold": -0.034281738102436066, "operator": "ge"}, {"budget": 0.86, "threshold": -0.04037557542324066, "operator": "ge"}, {"budget": 0.87, "threshold": -0.04656866192817688, "operator": "ge"}, {"budget": 0.88, "threshold": -0.05362626910209656, "operator": "ge"}, {"budget": 0.89, "threshold": -0.06338340789079666, "operator": "ge"}, {"budget": 0.9, "threshold": -0.07292332500219345, "operator": "ge"}, {"budget": 0.91, "threshold": -0.087591752409935, "operator": "ge"}, {"budget": 0.92, "threshold": -0.10431113094091415, "operator": "ge"}, {"budget": 0.93, "threshold": -0.12394901365041733, "operator": "ge"}, {"budget": 0.94, "threshold": -0.14758186042308807, "operator": "ge"}, {"budget": 0.95, "threshold": -0.17696145176887512, "operator": "ge"}, {"budget": 0.96, "threshold": -0.21001535654067993, "operator": "ge"}, {"budget": 0.97, "threshold": -0.2700211703777313, "operator": "ge"}, {"budget": 0.98, "threshold": -0.3659658133983612, "operator": "ge"}, {"budget": 0.99, "threshold": -0.4743328392505646, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.86977299880526, 57.57959097617541, 58.1066835336285, 58.64080399184763, 59.16789654930072, 59.6598496029236, 60.06043994658795, 60.482113992550424, 60.882704336214765, 61.220043572984736, 61.53629910745659, 61.81038723733221, 62.11258697027198, 62.456954107807995, 62.766181741513805, 63.06135357368754, 63.28624639820085, 63.49708342118209, 63.74305994799353, 63.925785367910606, 64.18581769625412, 64.36151521540515, 64.6074917422166, 64.77616136060158, 64.93780307822054, 65.00105418511491, 65.16972380349989, 65.26108651345842, 65.38056082648113, 65.45083983414153, 65.54220254410006, 65.55625834563216, 65.64762105559069, 65.64762105559069, 65.8022348724436, 65.87954178087006, 65.87251388010401, 65.87954178087006, 65.94982078853047, 66.02712769695691, 65.97793239159463, 65.97793239159463, 65.96387659006254, 65.94982078853047, 65.90765338393422, 65.8022348724436, 65.80926277320964, 65.7530395670813, 65.74601166631527, 65.7179000632511, 65.66870475788882, 65.5984257502284, 65.59139784946237, 65.53517464333403, 65.50706304026987, 65.48597933797174, 65.45083983414153, 65.4297561318434, 65.40164452877924, 65.31730971958676, 65.22594700962821, 65.14864010120176, 65.11350059737156, 65.00808208588094, 64.85346826902804, 64.80427296366575, 64.77616136060158, 64.74102185677137, 64.69885445217514, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.68479865064305, 64.68479865064305, 64.66371494834493, 64.65668704757888, 64.67074284911097, 64.65668704757888, 64.62154754374868, 64.62857544451472, 64.59343594068451, 64.59343594068451, 64.58640803991848, 64.56532433762034, 64.56532433762034, 64.60046384145055, 64.54424063532223, 64.5582964368543, 64.52315693302411, 64.51612903225806, 64.41071052076744, 64.41071052076744, 64.38259891770328, 64.3193478108089, 64.27015250544662, 64.22798510085038, 64.24906880314849, 64.20690139855225, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.992339588165017, 21.990301496942863, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.989177032820297, 26.994166842364187, 27.999156651908073, 28.990090659919883, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.98784173167475, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 121, 230, 318, 403, 488, 573, 645, 715, 786, 845, 902, 958, 1017, 1081, 1138, 1192, 1249, 1293, 1345, 1387, 1434, 1483, 1532, 1571, 1609, 1639, 1681, 1715, 1746, 1776, 1807, 1830, 1863, 1885, 1919, 1945, 1967, 1992, 2017, 2044, 2059, 2077, 2096, 2113, 2129, 2141, 2161, 2175, 2191, 2205, 2217, 2232, 2249, 2265, 2284, 2296, 2312, 2324, 2338, 2345, 2350, 2359, 2374, 2382, 2387, 2392, 2400, 2408, 2413, 2415, 2415, 2415, 2417, 2418, 2419, 2419, 2422, 2424, 2426, 2427, 2429, 2431, 2436, 2438, 2442, 2446, 2455, 2457, 2465, 2472, 2480, 2483, 2489, 2498, 2508, 2519, 2531, 2553, 2573, 2589], "harmed": [0, 5, 13, 26, 35, 45, 60, 75, 85, 99, 110, 122, 139, 155, 170, 183, 195, 220, 234, 251, 267, 277, 301, 315, 330, 345, 366, 384, 405, 419, 439, 457, 478, 498, 520, 532, 547, 570, 594, 609, 625, 647, 665, 686, 705, 727, 754, 773, 795, 812, 830, 849, 874, 892, 916, 939, 954, 975, 990, 1008, 1027, 1045, 1065, 1085, 1108, 1135, 1147, 1159, 1172, 1183, 1186, 1186, 1186, 1188, 1190, 1191, 1194, 1198, 1198, 1202, 1208, 1209, 1216, 1221, 1224, 1231, 1235, 1239, 1249, 1255, 1267, 1276, 1294, 1300, 1313, 1332, 1350, 1368, 1387, 1413, 1441], "exact_score": [56.05453650994448, 56.86977299880526, 57.57959097617541, 58.1066835336285, 58.64080399184763, 59.16789654930072, 59.6598496029236, 60.06043994658795, 60.482113992550424, 60.882704336214765, 61.220043572984736, 61.53629910745659, 61.81038723733221, 62.11258697027198, 62.456954107807995, 62.766181741513805, 63.06135357368754, 63.28624639820085, 63.49708342118209, 63.74305994799353, 63.925785367910606, 64.19284559702017, 64.36151521540515, 64.6074917422166, 64.77616136060158, 64.93780307822054, 65.00808208588094, 65.16972380349989, 65.26108651345842, 65.38758872724716, 65.45083983414153, 65.54220254410006, 65.55625834563216, 65.64762105559069, 65.64762105559069, 65.8022348724436, 65.87954178087006, 65.87251388010401, 65.87954178087006, 65.94982078853047, 66.02712769695691, 65.97793239159463, 65.97793239159463, 65.96387659006254, 65.94982078853047, 65.90765338393422, 65.8022348724436, 65.80926277320964, 65.7530395670813, 65.74601166631527, 65.7179000632511, 65.66870475788882, 65.5984257502284, 65.59139784946237, 65.53517464333403, 65.50706304026987, 65.48597933797174, 65.45083983414153, 65.4297561318434, 65.40164452877924, 65.31730971958676, 65.22594700962821, 65.14864010120176, 65.11350059737156, 65.00808208588094, 64.85346826902804, 64.80427296366575, 64.77616136060158, 64.74102185677137, 64.69885445217514, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.68479865064305, 64.68479865064305, 64.66371494834493, 64.65668704757888, 64.67074284911097, 64.65668704757888, 64.62154754374868, 64.62857544451472, 64.59343594068451, 64.59343594068451, 64.58640803991848, 64.56532433762034, 64.56532433762034, 64.60046384145055, 64.54424063532223, 64.5582964368543, 64.52315693302411, 64.51612903225806, 64.41071052076744, 64.41071052076744, 64.38259891770328, 64.3193478108089, 64.27015250544662, 64.22798510085038, 64.24906880314849, 64.20690139855225, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.22057738030156, "threshold_area100": 64.22011385199241, "threshold_area25": 61.51521540515847}, "test": {"score": [26.688029946402747, 27.714601707171827, 29.13535433740748, 30.394464452825908, 31.486260386240538, 32.31999546266625, 33.26716388282336, 34.17463063267447, 35.16433655673085, 35.955534129257295, 36.89135921504126, 37.614496781328874, 38.2610668405978, 38.84241272722117, 39.45495278337067, 39.99376116609478, 40.543912883192014, 41.136602104188526, 41.71511215721861, 42.19153220089045, 42.67645974534214, 43.12735728667442, 43.62079233190597, 44.034824036525535, 44.48572157785781, 44.964977455122934, 45.33647165584323, 45.69945835578368, 46.065280889317414, 46.43393925644443, 46.72319428295948, 47.06349431415365, 47.327226838329125, 47.64200436718373, 47.91140855854579, 48.18932025068769, 48.45305277486317, 48.66290446076624, 48.841561977143186, 49.06559283101268, 49.249922014576185, 49.47962453563225, 49.70365538950175, 49.85679040353912, 50.018432918356346, 50.1205229277146, 50.27932960893855, 50.42962878938264, 50.62813714091258, 50.74157048464396, 50.85216799478207, 50.9967955080396, 51.06485551427842, 51.172617190823246, 51.19530385956952, 51.27187136658821, 51.351274707200176, 51.40515554547259, 51.42500638062559, 51.481723052491276, 51.49023055327113, 51.49306638686441, 51.50440972123755, 51.51291722201742, 51.51291722201742, 51.510081388424126, 51.52426055639056, 51.51291722201742, 51.48739471967785, 51.478887218897995, 51.47321555171143, 51.476051385304714, 51.46754388452486, 51.46754388452486, 51.46754388452486, 51.45620055015172, 51.45620055015172, 51.45336471655844, 51.45336471655844, 51.439185548592015, 51.45336471655844, 51.439185548592015, 51.41082721265916, 51.3853047103196, 51.39664804469274, 51.3853047103196, 51.37963304313303, 51.37112554235318, 51.3853047103196, 51.348438873606895, 51.33142387204719, 51.29455803533449, 51.28037886736807, 51.22933386268894, 51.20097552675609, 51.1272438533307, 51.08470634943142, 51.008138842412734, 50.818137991662645, 50.65082380965885, 50.00708958398321], "calls": [0.0, 1.2307517794855798, 3.0428494455945323, 4.628080424240705, 6.14241556305476, 7.4780931854918755, 9.05198082976491, 10.634375974817797, 12.299010294075943, 13.847375436009415, 15.466636417775003, 16.827836542551683, 18.118140827496244, 19.43680344837365, 20.690241896605507, 21.852933669852252, 23.120551286050535, 24.44488557411451, 25.840115702010607, 27.022658310410346, 28.210872585996654, 29.45296769985537, 30.71774948246037, 31.8832770893004, 33.24164138048379, 34.625528174006746, 35.620905765249695, 36.712701698664326, 37.9065876414372, 39.08345858265037, 40.06465700592689, 41.306752119785614, 42.307801378215125, 43.49885148739472, 44.664379094234754, 45.82423503388821, 46.96424013838868, 47.99081189915776, 48.855741145109604, 50.1205229277146, 51.14993052207697, 52.270084791424445, 53.51501573887645, 54.374273317641716, 55.599353429940734, 56.52667101494484, 57.50786943822137, 58.667725377874824, 59.80205881518873, 60.80594390721152, 61.83818733516717, 62.98953577404078, 64.00192836684343, 65.42835266426566, 66.6108952726654, 67.70269120608002, 68.85403964495363, 69.8664322377563, 70.81927232509996, 72.12091994441766, 73.22689504579871, 74.34704931514618, 75.51824858917279, 76.78303037177778, 77.82661713410657, 79.05453307999886, 79.91946232595072, 80.74752573518987, 81.60678331395513, 82.31290587868304, 83.38768681053796, 84.08813770807929, 84.7375436009415, 85.3132178203783, 85.97113121402036, 86.56949210220344, 87.08277798258798, 87.61307886453224, 88.14905141366305, 88.68785979638714, 89.20681734395825, 89.72861072512265, 90.38368828517143, 90.87428749680969, 91.25428919830985, 91.62011173184358, 92.03130760286986, 92.51056348013499, 92.87638601366872, 93.37549272608683, 93.76116609477356, 94.31698947905737, 94.82460369225534, 95.32371040467346, 95.8143096163117, 96.29356549357684, 96.7643138700621, 97.3598389246519, 98.09715565890593, 98.6785015455293, 100.0], "rescued": [0, 375, 898, 1363, 1768, 2083, 2440, 2790, 3164, 3471, 3824, 4102, 4352, 4592, 4829, 5040, 5276, 5509, 5739, 5931, 6129, 6313, 6509, 6687, 6878, 7080, 7227, 7377, 7527, 7685, 7807, 7953, 8073, 8203, 8325, 8444, 8559, 8658, 8736, 8850, 8938, 9044, 9153, 9228, 9321, 9385, 9464, 9540, 9629, 9688, 9760, 9840, 9890, 9971, 10017, 10071, 10128, 10170, 10203, 10253, 10288, 10319, 10355, 10386, 10411, 10435, 10455, 10469, 10488, 10501, 10501, 10502, 10504, 10505, 10505, 10505, 10507, 10508, 10510, 10510, 10517, 10521, 10524, 10529, 10537, 10543, 10550, 10556, 10567, 10572, 10581, 10591, 10604, 10618, 10637, 10645, 10662, 10677, 10691, 10704, 10723], "harmed": [0, 13, 35, 56, 76, 97, 120, 150, 175, 203, 226, 249, 271, 306, 327, 348, 390, 414, 440, 464, 491, 516, 538, 570, 602, 635, 651, 673, 694, 722, 742, 768, 795, 814, 841, 862, 884, 909, 924, 959, 982, 1007, 1037, 1058, 1094, 1122, 1145, 1168, 1187, 1206, 1239, 1268, 1294, 1337, 1375, 1402, 1431, 1454, 1480, 1510, 1542, 1572, 1604, 1632, 1657, 1682, 1697, 1715, 1743, 1759, 1761, 1761, 1766, 1767, 1767, 1771, 1773, 1775, 1777, 1782, 1784, 1793, 1806, 1820, 1824, 1834, 1843, 1852, 1858, 1876, 1891, 1914, 1932, 1964, 1993, 2027, 2059, 2101, 2182, 2254, 2500], "exact_score": [26.688029946402747, 27.527436690015033, 28.33848509769447, 29.106996001474634, 29.88117857244137, 30.661032810594673, 31.358647874542722, 32.04775543771092, 32.65462382667385, 33.23596971329722, 33.797464764767604, 34.40716898732383, 34.994186541133764, 35.5273232566713, 36.03777330346256, 36.61344752289936, 37.21748007826901, 37.719422624280405, 38.207186002325386, 38.63539687491138, 39.106145251396654, 39.608087797408054, 40.041970337180615, 40.48436037773303, 40.9380937526586, 41.346453790091594, 41.78033632986416, 42.1830247001106, 42.56869806879733, 42.971386439043755, 43.32302980461107, 43.7257181748575, 44.07452570683152, 44.40631823724584, 44.72109576610045, 45.1209483027536, 45.47542750191419, 45.77886169639566, 46.08796755806369, 46.40841675410487, 46.714686782179626, 46.99543430791481, 47.262002665683575, 47.52857102345235, 47.752601877321844, 48.024841902277174, 48.23469358818024, 48.46156027564303, 48.66574029435953, 48.86708447948275, 49.06559283101268, 49.20171284349035, 49.40022119502028, 49.63843121685619, 49.80574539886, 49.950372912117516, 50.07514959022205, 50.20559793551315, 50.33604628080425, 50.47216629328191, 50.647987976065565, 50.758585486203664, 50.87201882993506, 50.9967955080396, 51.06485551427842, 51.1442588548904, 51.19246802597623, 51.19530385956952, 51.28605053455464, 51.373961375946465, 51.40799137906588, 51.433513881405446, 51.478887218897995, 51.46754388452486, 51.48739471967785, 51.50440972123755, 51.52142472279727, 51.50157388764427, 51.50157388764427, 51.50724555483084, 51.5157530556107, 51.49873805405099, 51.48739471967785, 51.47321555171143, 51.476051385304714, 51.46754388452486, 51.46470805093158, 51.45052888296515, 51.459036383745, 51.4420213821853, 51.43634971499873, 51.38814054391288, 51.37963304313303, 51.376797209539745, 51.3285880384539, 51.26903553299492, 51.16978135722996, 51.056348013498564, 50.86067549556192, 50.50619629640133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.650087926804666, "threshold_area100": 47.18272693758331, "threshold_area25": 37.541615857981455}, "selected_budget": 40, "selected": {"val": {"score": 66.02712769695691, "calls": 39.995783259540374, "rescued": 2044, "harmed": 625}, "test": {"score": 49.249922014576185, "calls": 51.14993052207697, "rescued": 8938, "harmed": 982}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9041603803634644, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8591342568397522, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8250337839126587, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7967910170555115, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7724076509475708, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7499251365661621, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7285953164100647, "operator": "ge"}, {"budget": 0.08, "threshold": 0.708764374256134, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6905443668365479, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6743236184120178, "operator": "ge"}, {"budget": 0.11, "threshold": 0.659709095954895, "operator": "ge"}, {"budget": 0.12, "threshold": 0.645710825920105, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6310664415359497, "operator": "ge"}, {"budget": 0.14, "threshold": 0.6174618005752563, "operator": "ge"}, {"budget": 0.15, "threshold": 0.605050802230835, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5929194688796997, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5799402594566345, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5670498013496399, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5558100938796997, "operator": "ge"}, {"budget": 0.2, "threshold": 0.5436378121376038, "operator": "ge"}, {"budget": 0.21, "threshold": 0.5320740342140198, "operator": "gt"}, {"budget": 0.22, "threshold": 0.5204253792762756, "operator": "gt"}, {"budget": 0.23, "threshold": 0.5094419717788696, "operator": "ge"}, {"budget": 0.24, "threshold": 0.49679458141326904, "operator": "ge"}, {"budget": 0.25, "threshold": 0.48403018712997437, "operator": "ge"}, {"budget": 0.26, "threshold": 0.47405382990837097, "operator": "gt"}, {"budget": 0.27, "threshold": 0.46293964982032776, "operator": "ge"}, {"budget": 0.28, "threshold": 0.4514990448951721, "operator": "ge"}, {"budget": 0.29, "threshold": 0.44061043858528137, "operator": "gt"}, {"budget": 0.3, "threshold": 0.43097689747810364, "operator": "ge"}, {"budget": 0.31, "threshold": 0.41897478699684143, "operator": "ge"}, {"budget": 0.32, "threshold": 0.40838557481765747, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3971496522426605, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3861183822154999, "operator": "ge"}, {"budget": 0.35, "threshold": 0.3739790618419647, "operator": "ge"}, {"budget": 0.36, "threshold": 0.3625051975250244, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3523077070713043, "operator": "gt"}, {"budget": 0.38, "threshold": 0.34332388639450073, "operator": "ge"}, {"budget": 0.39, "threshold": 0.33103594183921814, "operator": "ge"}, {"budget": 0.4, "threshold": 0.3213382363319397, "operator": "ge"}, {"budget": 0.41, "threshold": 0.31109774112701416, "operator": "ge"}, {"budget": 0.42, "threshold": 0.29935571551322937, "operator": "ge"}, {"budget": 0.43, "threshold": 0.28986531496047974, "operator": "ge"}, {"budget": 0.44, "threshold": 0.27781161665916443, "operator": "ge"}, {"budget": 0.45, "threshold": 0.26680710911750793, "operator": "ge"}, {"budget": 0.46, "threshold": 0.2567208707332611, "operator": "ge"}, {"budget": 0.47, "threshold": 0.24511665105819702, "operator": "ge"}, {"budget": 0.48, "threshold": 0.2330968827009201, "operator": "ge"}, {"budget": 0.49, "threshold": 0.22280633449554443, "operator": "ge"}, {"budget": 0.5, "threshold": 0.21218053996562958, "operator": "ge"}, {"budget": 0.51, "threshold": 0.20134106278419495, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1911592334508896, "operator": "ge"}, {"budget": 0.53, "threshold": 0.17767000198364258, "operator": "ge"}, {"budget": 0.54, "threshold": 0.166120246052742, "operator": "ge"}, {"budget": 0.55, "threshold": 0.15467528998851776, "operator": "ge"}, {"budget": 0.56, "threshold": 0.143578439950943, "operator": "ge"}, {"budget": 0.57, "threshold": 0.1328301578760147, "operator": "ge"}, {"budget": 0.58, "threshold": 0.12371194362640381, "operator": "ge"}, {"budget": 0.59, "threshold": 0.11189108341932297, "operator": "ge"}, {"budget": 0.6, "threshold": 0.10127071291208267, "operator": "ge"}, {"budget": 0.61, "threshold": 0.09120266139507294, "operator": "ge"}, {"budget": 0.62, "threshold": 0.0801340714097023, "operator": "ge"}, {"budget": 0.63, "threshold": 0.06878426671028137, "operator": "ge"}, {"budget": 0.64, "threshold": 0.05845651775598526, "operator": "ge"}, {"budget": 0.65, "threshold": 0.04533154517412186, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03502146154642105, "operator": "ge"}, {"budget": 0.67, "threshold": 0.022149808704853058, "operator": "ge"}, {"budget": 0.68, "threshold": 0.00919191725552082, "operator": "ge"}, {"budget": 0.69, "threshold": 0.00029996829107403755, "operator": "ge"}, {"budget": 0.7, "threshold": -0.0009712062892504036, "operator": "ge"}, {"budget": 0.71, "threshold": -0.00173279014416039, "operator": "ge"}, {"budget": 0.72, "threshold": -0.002569451928138733, "operator": "ge"}, {"budget": 0.73, "threshold": -0.003619840368628502, "operator": "ge"}, {"budget": 0.74, "threshold": -0.004838157445192337, "operator": "ge"}, {"budget": 0.75, "threshold": -0.006142979487776756, "operator": "ge"}, {"budget": 0.76, "threshold": -0.007424978073686361, "operator": "ge"}, {"budget": 0.77, "threshold": -0.009142988361418247, "operator": "ge"}, {"budget": 0.78, "threshold": -0.010955432429909706, "operator": "ge"}, {"budget": 0.79, "threshold": -0.013057566247880459, "operator": "ge"}, {"budget": 0.8, "threshold": -0.015146724879741669, "operator": "ge"}, {"budget": 0.81, "threshold": -0.017793286591768265, "operator": "ge"}, {"budget": 0.82, "threshold": -0.02160523273050785, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0252833291888237, "operator": "ge"}, {"budget": 0.84, "threshold": -0.02932233177125454, "operator": "ge"}, {"budget": 0.85, "threshold": -0.034281738102436066, "operator": "ge"}, {"budget": 0.86, "threshold": -0.04037557542324066, "operator": "ge"}, {"budget": 0.87, "threshold": -0.04656866192817688, "operator": "ge"}, {"budget": 0.88, "threshold": -0.05362626910209656, "operator": "ge"}, {"budget": 0.89, "threshold": -0.06338340789079666, "operator": "ge"}, {"budget": 0.9, "threshold": -0.07292332500219345, "operator": "ge"}, {"budget": 0.91, "threshold": -0.087591752409935, "operator": "ge"}, {"budget": 0.92, "threshold": -0.10431113094091415, "operator": "ge"}, {"budget": 0.93, "threshold": -0.12394901365041733, "operator": "ge"}, {"budget": 0.94, "threshold": -0.14758186042308807, "operator": "ge"}, {"budget": 0.95, "threshold": -0.17696145176887512, "operator": "ge"}, {"budget": 0.96, "threshold": -0.21001535654067993, "operator": "ge"}, {"budget": 0.97, "threshold": -0.2700211703777313, "operator": "ge"}, {"budget": 0.98, "threshold": -0.3659658133983612, "operator": "ge"}, {"budget": 0.99, "threshold": -0.4743328392505646, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.86977299880526, 57.57959097617541, 58.1066835336285, 58.64080399184763, 59.16789654930072, 59.6598496029236, 60.06043994658795, 60.482113992550424, 60.882704336214765, 61.220043572984736, 61.53629910745659, 61.81038723733221, 62.11258697027198, 62.456954107807995, 62.766181741513805, 63.06135357368754, 63.28624639820085, 63.49708342118209, 63.74305994799353, 63.925785367910606, 64.18581769625412, 64.36151521540515, 64.6074917422166, 64.77616136060158, 64.93780307822054, 65.00105418511491, 65.16972380349989, 65.26108651345842, 65.38056082648113, 65.45083983414153, 65.54220254410006, 65.55625834563216, 65.64762105559069, 65.64762105559069, 65.8022348724436, 65.87954178087006, 65.87251388010401, 65.87954178087006, 65.94982078853047, 66.02712769695691, 65.97793239159463, 65.97793239159463, 65.96387659006254, 65.94982078853047, 65.90765338393422, 65.8022348724436, 65.80926277320964, 65.7530395670813, 65.74601166631527, 65.7179000632511, 65.66870475788882, 65.5984257502284, 65.59139784946237, 65.53517464333403, 65.50706304026987, 65.48597933797174, 65.45083983414153, 65.4297561318434, 65.40164452877924, 65.31730971958676, 65.22594700962821, 65.14864010120176, 65.11350059737156, 65.00808208588094, 64.85346826902804, 64.80427296366575, 64.77616136060158, 64.74102185677137, 64.69885445217514, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.68479865064305, 64.68479865064305, 64.66371494834493, 64.65668704757888, 64.67074284911097, 64.65668704757888, 64.62154754374868, 64.62857544451472, 64.59343594068451, 64.59343594068451, 64.58640803991848, 64.56532433762034, 64.56532433762034, 64.60046384145055, 64.54424063532223, 64.5582964368543, 64.52315693302411, 64.51612903225806, 64.41071052076744, 64.41071052076744, 64.38259891770328, 64.3193478108089, 64.27015250544662, 64.22798510085038, 64.24906880314849, 64.20690139855225, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.992339588165017, 21.990301496942863, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.989177032820297, 26.994166842364187, 27.999156651908073, 28.990090659919883, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.98784173167475, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 121, 230, 318, 403, 488, 573, 645, 715, 786, 845, 902, 958, 1017, 1081, 1138, 1192, 1249, 1293, 1345, 1387, 1434, 1483, 1532, 1571, 1609, 1639, 1681, 1715, 1746, 1776, 1807, 1830, 1863, 1885, 1919, 1945, 1967, 1992, 2017, 2044, 2059, 2077, 2096, 2113, 2129, 2141, 2161, 2175, 2191, 2205, 2217, 2232, 2249, 2265, 2284, 2296, 2312, 2324, 2338, 2345, 2350, 2359, 2374, 2382, 2387, 2392, 2400, 2408, 2413, 2415, 2415, 2415, 2417, 2418, 2419, 2419, 2422, 2424, 2426, 2427, 2429, 2431, 2436, 2438, 2442, 2446, 2455, 2457, 2465, 2472, 2480, 2483, 2489, 2498, 2508, 2519, 2531, 2553, 2573, 2589], "harmed": [0, 5, 13, 26, 35, 45, 60, 75, 85, 99, 110, 122, 139, 155, 170, 183, 195, 220, 234, 251, 267, 277, 301, 315, 330, 345, 366, 384, 405, 419, 439, 457, 478, 498, 520, 532, 547, 570, 594, 609, 625, 647, 665, 686, 705, 727, 754, 773, 795, 812, 830, 849, 874, 892, 916, 939, 954, 975, 990, 1008, 1027, 1045, 1065, 1085, 1108, 1135, 1147, 1159, 1172, 1183, 1186, 1186, 1186, 1188, 1190, 1191, 1194, 1198, 1198, 1202, 1208, 1209, 1216, 1221, 1224, 1231, 1235, 1239, 1249, 1255, 1267, 1276, 1294, 1300, 1313, 1332, 1350, 1368, 1387, 1413, 1441], "exact_score": [56.05453650994448, 56.86977299880526, 57.57959097617541, 58.1066835336285, 58.64080399184763, 59.16789654930072, 59.6598496029236, 60.06043994658795, 60.482113992550424, 60.882704336214765, 61.220043572984736, 61.53629910745659, 61.81038723733221, 62.11258697027198, 62.456954107807995, 62.766181741513805, 63.06135357368754, 63.28624639820085, 63.49708342118209, 63.74305994799353, 63.925785367910606, 64.19284559702017, 64.36151521540515, 64.6074917422166, 64.77616136060158, 64.93780307822054, 65.00808208588094, 65.16972380349989, 65.26108651345842, 65.38758872724716, 65.45083983414153, 65.54220254410006, 65.55625834563216, 65.64762105559069, 65.64762105559069, 65.8022348724436, 65.87954178087006, 65.87251388010401, 65.87954178087006, 65.94982078853047, 66.02712769695691, 65.97793239159463, 65.97793239159463, 65.96387659006254, 65.94982078853047, 65.90765338393422, 65.8022348724436, 65.80926277320964, 65.7530395670813, 65.74601166631527, 65.7179000632511, 65.66870475788882, 65.5984257502284, 65.59139784946237, 65.53517464333403, 65.50706304026987, 65.48597933797174, 65.45083983414153, 65.4297561318434, 65.40164452877924, 65.31730971958676, 65.22594700962821, 65.14864010120176, 65.11350059737156, 65.00808208588094, 64.85346826902804, 64.80427296366575, 64.77616136060158, 64.74102185677137, 64.69885445217514, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.68479865064305, 64.68479865064305, 64.66371494834493, 64.65668704757888, 64.67074284911097, 64.65668704757888, 64.62154754374868, 64.62857544451472, 64.59343594068451, 64.59343594068451, 64.58640803991848, 64.56532433762034, 64.56532433762034, 64.60046384145055, 64.54424063532223, 64.5582964368543, 64.52315693302411, 64.51612903225806, 64.41071052076744, 64.41071052076744, 64.38259891770328, 64.3193478108089, 64.27015250544662, 64.22798510085038, 64.24906880314849, 64.20690139855225, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.22057738030156, "threshold_area100": 64.22011385199241, "threshold_area25": 61.51521540515847}, "test": {"score": [26.688029946402747, 27.714601707171827, 29.13535433740748, 30.394464452825908, 31.486260386240538, 32.31999546266625, 33.26716388282336, 34.17463063267447, 35.16433655673085, 35.955534129257295, 36.89135921504126, 37.614496781328874, 38.2610668405978, 38.84241272722117, 39.45495278337067, 39.99376116609478, 40.543912883192014, 41.136602104188526, 41.71511215721861, 42.19153220089045, 42.67645974534214, 43.12735728667442, 43.62079233190597, 44.034824036525535, 44.48572157785781, 44.964977455122934, 45.33647165584323, 45.69945835578368, 46.065280889317414, 46.43393925644443, 46.72319428295948, 47.06349431415365, 47.327226838329125, 47.64200436718373, 47.91140855854579, 48.18932025068769, 48.45305277486317, 48.66290446076624, 48.841561977143186, 49.06559283101268, 49.249922014576185, 49.47962453563225, 49.70365538950175, 49.85679040353912, 50.018432918356346, 50.1205229277146, 50.27932960893855, 50.42962878938264, 50.62813714091258, 50.74157048464396, 50.85216799478207, 50.9967955080396, 51.06485551427842, 51.172617190823246, 51.19530385956952, 51.27187136658821, 51.351274707200176, 51.40515554547259, 51.42500638062559, 51.481723052491276, 51.49023055327113, 51.49306638686441, 51.50440972123755, 51.51291722201742, 51.51291722201742, 51.510081388424126, 51.52426055639056, 51.51291722201742, 51.48739471967785, 51.478887218897995, 51.47321555171143, 51.476051385304714, 51.46754388452486, 51.46754388452486, 51.46754388452486, 51.45620055015172, 51.45620055015172, 51.45336471655844, 51.45336471655844, 51.439185548592015, 51.45336471655844, 51.439185548592015, 51.41082721265916, 51.3853047103196, 51.39664804469274, 51.3853047103196, 51.37963304313303, 51.37112554235318, 51.3853047103196, 51.348438873606895, 51.33142387204719, 51.29455803533449, 51.28037886736807, 51.22933386268894, 51.20097552675609, 51.1272438533307, 51.08470634943142, 51.008138842412734, 50.818137991662645, 50.65082380965885, 50.00708958398321], "calls": [0.0, 1.2307517794855798, 3.0428494455945323, 4.628080424240705, 6.14241556305476, 7.4780931854918755, 9.05198082976491, 10.634375974817797, 12.299010294075943, 13.847375436009415, 15.466636417775003, 16.827836542551683, 18.118140827496244, 19.43680344837365, 20.690241896605507, 21.852933669852252, 23.120551286050535, 24.44488557411451, 25.840115702010607, 27.022658310410346, 28.210872585996654, 29.45296769985537, 30.71774948246037, 31.8832770893004, 33.24164138048379, 34.625528174006746, 35.620905765249695, 36.712701698664326, 37.9065876414372, 39.08345858265037, 40.06465700592689, 41.306752119785614, 42.307801378215125, 43.49885148739472, 44.664379094234754, 45.82423503388821, 46.96424013838868, 47.99081189915776, 48.855741145109604, 50.1205229277146, 51.14993052207697, 52.270084791424445, 53.51501573887645, 54.374273317641716, 55.599353429940734, 56.52667101494484, 57.50786943822137, 58.667725377874824, 59.80205881518873, 60.80594390721152, 61.83818733516717, 62.98953577404078, 64.00192836684343, 65.42835266426566, 66.6108952726654, 67.70269120608002, 68.85403964495363, 69.8664322377563, 70.81927232509996, 72.12091994441766, 73.22689504579871, 74.34704931514618, 75.51824858917279, 76.78303037177778, 77.82661713410657, 79.05453307999886, 79.91946232595072, 80.74752573518987, 81.60678331395513, 82.31290587868304, 83.38768681053796, 84.08813770807929, 84.7375436009415, 85.3132178203783, 85.97113121402036, 86.56949210220344, 87.08277798258798, 87.61307886453224, 88.14905141366305, 88.68785979638714, 89.20681734395825, 89.72861072512265, 90.38368828517143, 90.87428749680969, 91.25428919830985, 91.62011173184358, 92.03130760286986, 92.51056348013499, 92.87638601366872, 93.37549272608683, 93.76116609477356, 94.31698947905737, 94.82460369225534, 95.32371040467346, 95.8143096163117, 96.29356549357684, 96.7643138700621, 97.3598389246519, 98.09715565890593, 98.6785015455293, 100.0], "rescued": [0, 375, 898, 1363, 1768, 2083, 2440, 2790, 3164, 3471, 3824, 4102, 4352, 4592, 4829, 5040, 5276, 5509, 5739, 5931, 6129, 6313, 6509, 6687, 6878, 7080, 7227, 7377, 7527, 7685, 7807, 7953, 8073, 8203, 8325, 8444, 8559, 8658, 8736, 8850, 8938, 9044, 9153, 9228, 9321, 9385, 9464, 9540, 9629, 9688, 9760, 9840, 9890, 9971, 10017, 10071, 10128, 10170, 10203, 10253, 10288, 10319, 10355, 10386, 10411, 10435, 10455, 10469, 10488, 10501, 10501, 10502, 10504, 10505, 10505, 10505, 10507, 10508, 10510, 10510, 10517, 10521, 10524, 10529, 10537, 10543, 10550, 10556, 10567, 10572, 10581, 10591, 10604, 10618, 10637, 10645, 10662, 10677, 10691, 10704, 10723], "harmed": [0, 13, 35, 56, 76, 97, 120, 150, 175, 203, 226, 249, 271, 306, 327, 348, 390, 414, 440, 464, 491, 516, 538, 570, 602, 635, 651, 673, 694, 722, 742, 768, 795, 814, 841, 862, 884, 909, 924, 959, 982, 1007, 1037, 1058, 1094, 1122, 1145, 1168, 1187, 1206, 1239, 1268, 1294, 1337, 1375, 1402, 1431, 1454, 1480, 1510, 1542, 1572, 1604, 1632, 1657, 1682, 1697, 1715, 1743, 1759, 1761, 1761, 1766, 1767, 1767, 1771, 1773, 1775, 1777, 1782, 1784, 1793, 1806, 1820, 1824, 1834, 1843, 1852, 1858, 1876, 1891, 1914, 1932, 1964, 1993, 2027, 2059, 2101, 2182, 2254, 2500], "exact_score": [26.688029946402747, 27.527436690015033, 28.33848509769447, 29.106996001474634, 29.88117857244137, 30.661032810594673, 31.358647874542722, 32.04775543771092, 32.65462382667385, 33.23596971329722, 33.797464764767604, 34.40716898732383, 34.994186541133764, 35.5273232566713, 36.03777330346256, 36.61344752289936, 37.21748007826901, 37.719422624280405, 38.207186002325386, 38.63539687491138, 39.106145251396654, 39.608087797408054, 40.041970337180615, 40.48436037773303, 40.9380937526586, 41.346453790091594, 41.78033632986416, 42.1830247001106, 42.56869806879733, 42.971386439043755, 43.32302980461107, 43.7257181748575, 44.07452570683152, 44.40631823724584, 44.72109576610045, 45.1209483027536, 45.47542750191419, 45.77886169639566, 46.08796755806369, 46.40841675410487, 46.714686782179626, 46.99543430791481, 47.262002665683575, 47.52857102345235, 47.752601877321844, 48.024841902277174, 48.23469358818024, 48.46156027564303, 48.66574029435953, 48.86708447948275, 49.06559283101268, 49.20171284349035, 49.40022119502028, 49.63843121685619, 49.80574539886, 49.950372912117516, 50.07514959022205, 50.20559793551315, 50.33604628080425, 50.47216629328191, 50.647987976065565, 50.758585486203664, 50.87201882993506, 50.9967955080396, 51.06485551427842, 51.1442588548904, 51.19246802597623, 51.19530385956952, 51.28605053455464, 51.373961375946465, 51.40799137906588, 51.433513881405446, 51.478887218897995, 51.46754388452486, 51.48739471967785, 51.50440972123755, 51.52142472279727, 51.50157388764427, 51.50157388764427, 51.50724555483084, 51.5157530556107, 51.49873805405099, 51.48739471967785, 51.47321555171143, 51.476051385304714, 51.46754388452486, 51.46470805093158, 51.45052888296515, 51.459036383745, 51.4420213821853, 51.43634971499873, 51.38814054391288, 51.37963304313303, 51.376797209539745, 51.3285880384539, 51.26903553299492, 51.16978135722996, 51.056348013498564, 50.86067549556192, 50.50619629640133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.650087926804666, "threshold_area100": 47.18272693758331, "threshold_area25": 37.541615857981455}, "selected_budget": 40, "selected": {"val": {"score": 66.02712769695691, "calls": 39.995783259540374, "rescued": 2044, "harmed": 625}, "test": {"score": 49.249922014576185, "calls": 51.14993052207697, "rescued": 8938, "harmed": 982}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 14.060022354125977, "operator": "ge"}, {"budget": 0.02, "threshold": 13.173101425170898, "operator": "ge"}, {"budget": 0.03, "threshold": 12.594588279724121, "operator": "gt"}, {"budget": 0.04, "threshold": 12.180870056152344, "operator": "ge"}, {"budget": 0.05, "threshold": 11.805278778076172, "operator": "ge"}, {"budget": 0.06, "threshold": 11.526593208312988, "operator": "ge"}, {"budget": 0.07, "threshold": 11.224812507629395, "operator": "ge"}, {"budget": 0.08, "threshold": 10.978164672851562, "operator": "gt"}, {"budget": 0.09, "threshold": 10.745453834533691, "operator": "ge"}, {"budget": 0.1, "threshold": 10.521129608154297, "operator": "ge"}, {"budget": 0.11, "threshold": 10.289153099060059, "operator": "ge"}, {"budget": 0.12, "threshold": 10.068374633789062, "operator": "ge"}, {"budget": 0.13, "threshold": 9.86807632446289, "operator": "ge"}, {"budget": 0.14, "threshold": 9.676948547363281, "operator": "ge"}, {"budget": 0.15, "threshold": 9.457161903381348, "operator": "ge"}, {"budget": 0.16, "threshold": 9.276337623596191, "operator": "ge"}, {"budget": 0.17, "threshold": 9.02929401397705, "operator": "ge"}, {"budget": 0.18, "threshold": 8.855338096618652, "operator": "ge"}, {"budget": 0.19, "threshold": 8.684965133666992, "operator": "ge"}, {"budget": 0.2, "threshold": 8.499258041381836, "operator": "ge"}, {"budget": 0.21, "threshold": 8.298155784606934, "operator": "ge"}, {"budget": 0.22, "threshold": 8.086816787719727, "operator": "ge"}, {"budget": 0.23, "threshold": 7.908999443054199, "operator": "gt"}, {"budget": 0.24, "threshold": 7.707330703735352, "operator": "ge"}, {"budget": 0.25, "threshold": 7.505842685699463, "operator": "ge"}, {"budget": 0.26, "threshold": 7.301146984100342, "operator": "ge"}, {"budget": 0.27, "threshold": 7.089812278747559, "operator": "ge"}, {"budget": 0.28, "threshold": 6.884395599365234, "operator": "ge"}, {"budget": 0.29, "threshold": 6.69130802154541, "operator": "ge"}, {"budget": 0.3, "threshold": 6.4914164543151855, "operator": "ge"}, {"budget": 0.31, "threshold": 6.260966777801514, "operator": "ge"}, {"budget": 0.32, "threshold": 6.078299522399902, "operator": "ge"}, {"budget": 0.33, "threshold": 5.889019966125488, "operator": "ge"}, {"budget": 0.34, "threshold": 5.728420257568359, "operator": "ge"}, {"budget": 0.35, "threshold": 5.558870792388916, "operator": "ge"}, {"budget": 0.36, "threshold": 5.35020637512207, "operator": "ge"}, {"budget": 0.37, "threshold": 5.163990497589111, "operator": "ge"}, {"budget": 0.38, "threshold": 4.976912498474121, "operator": "ge"}, {"budget": 0.39, "threshold": 4.807806491851807, "operator": "ge"}, {"budget": 0.4, "threshold": 4.625563621520996, "operator": "ge"}, {"budget": 0.41, "threshold": 4.480868339538574, "operator": "ge"}, {"budget": 0.42, "threshold": 4.316263675689697, "operator": "ge"}, {"budget": 0.43, "threshold": 4.159482955932617, "operator": "ge"}, {"budget": 0.44, "threshold": 4.007145881652832, "operator": "ge"}, {"budget": 0.45, "threshold": 3.849452018737793, "operator": "ge"}, {"budget": 0.46, "threshold": 3.7036900520324707, "operator": "ge"}, {"budget": 0.47, "threshold": 3.551976203918457, "operator": "ge"}, {"budget": 0.48, "threshold": 3.4103424549102783, "operator": "ge"}, {"budget": 0.49, "threshold": 3.2581329345703125, "operator": "ge"}, {"budget": 0.5, "threshold": 3.089433193206787, "operator": "ge"}, {"budget": 0.51, "threshold": 2.9490158557891846, "operator": "ge"}, {"budget": 0.52, "threshold": 2.8100931644439697, "operator": "ge"}, {"budget": 0.53, "threshold": 2.643014430999756, "operator": "ge"}, {"budget": 0.54, "threshold": 2.493321657180786, "operator": "ge"}, {"budget": 0.55, "threshold": 2.3240582942962646, "operator": "ge"}, {"budget": 0.56, "threshold": 2.1804733276367188, "operator": "ge"}, {"budget": 0.57, "threshold": 2.024972438812256, "operator": "ge"}, {"budget": 0.58, "threshold": 1.8636417388916016, "operator": "ge"}, {"budget": 0.59, "threshold": 1.7002202272415161, "operator": "ge"}, {"budget": 0.6, "threshold": 1.5555362701416016, "operator": "ge"}, {"budget": 0.61, "threshold": 1.4103327989578247, "operator": "ge"}, {"budget": 0.62, "threshold": 1.2660402059555054, "operator": "ge"}, {"budget": 0.63, "threshold": 1.0913748741149902, "operator": "ge"}, {"budget": 0.64, "threshold": 0.9137632846832275, "operator": "ge"}, {"budget": 0.65, "threshold": 0.7493799924850464, "operator": "ge"}, {"budget": 0.66, "threshold": 0.5594251155853271, "operator": "ge"}, {"budget": 0.67, "threshold": 0.4002516269683838, "operator": "ge"}, {"budget": 0.68, "threshold": 0.20414292812347412, "operator": "ge"}, {"budget": 0.69, "threshold": 0.008102118968963623, "operator": "ge"}, {"budget": 0.7, "threshold": -0.17228281497955322, "operator": "ge"}, {"budget": 0.71, "threshold": -0.3449113368988037, "operator": "ge"}, {"budget": 0.72, "threshold": -0.5017752647399902, "operator": "ge"}, {"budget": 0.73, "threshold": -0.6810997724533081, "operator": "ge"}, {"budget": 0.74, "threshold": -0.8785815238952637, "operator": "ge"}, {"budget": 0.75, "threshold": -1.045776128768921, "operator": "ge"}, {"budget": 0.76, "threshold": -1.2290995121002197, "operator": "ge"}, {"budget": 0.77, "threshold": -1.3969018459320068, "operator": "ge"}, {"budget": 0.78, "threshold": -1.595544695854187, "operator": "ge"}, {"budget": 0.79, "threshold": -1.78432035446167, "operator": "ge"}, {"budget": 0.8, "threshold": -1.9684276580810547, "operator": "ge"}, {"budget": 0.81, "threshold": -2.223667621612549, "operator": "ge"}, {"budget": 0.82, "threshold": -2.409163236618042, "operator": "ge"}, {"budget": 0.83, "threshold": -2.616818428039551, "operator": "ge"}, {"budget": 0.84, "threshold": -2.8223090171813965, "operator": "ge"}, {"budget": 0.85, "threshold": -3.036878824234009, "operator": "ge"}, {"budget": 0.86, "threshold": -3.268273115158081, "operator": "ge"}, {"budget": 0.87, "threshold": -3.5314340591430664, "operator": "ge"}, {"budget": 0.88, "threshold": -3.7727584838867188, "operator": "ge"}, {"budget": 0.89, "threshold": -4.04252815246582, "operator": "ge"}, {"budget": 0.9, "threshold": -4.324408531188965, "operator": "ge"}, {"budget": 0.91, "threshold": -4.666163444519043, "operator": "ge"}, {"budget": 0.92, "threshold": -4.974923133850098, "operator": "ge"}, {"budget": 0.93, "threshold": -5.324373722076416, "operator": "ge"}, {"budget": 0.94, "threshold": -5.67095422744751, "operator": "ge"}, {"budget": 0.95, "threshold": -6.11473274230957, "operator": "ge"}, {"budget": 0.96, "threshold": -6.563602447509766, "operator": "ge"}, {"budget": 0.97, "threshold": -7.198408126831055, "operator": "ge"}, {"budget": 0.98, "threshold": -8.021705627441406, "operator": "ge"}, {"budget": 0.99, "threshold": -9.455540657043457, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.820577693442964, 57.41092135779042, 58.050460327500176, 58.67594349567784, 59.22411975542905, 59.73012861058401, 60.22910956497294, 60.678895213999574, 61.19895987068662, 61.6909129243095, 62.189893878698435, 62.57642842083069, 63.02621406985733, 63.49005552041604, 63.84145055871811, 64.27015250544662, 64.6074917422166, 65.02213788741302, 65.31730971958676, 65.61950945252653, 65.85845807857193, 66.18876941457586, 66.4698854452175, 66.6736945674327, 66.88453159041394, 67.10239651416121, 67.36242884250474, 67.46784735399535, 67.65057277391243, 67.74193548387096, 67.84735399536159, 67.93168880455407, 67.97385620915031, 67.94574460608615, 67.88952139995783, 67.84735399536159, 67.8684376976597, 67.85438189612762, 67.74193548387096, 67.68571227774264, 67.60840536931619, 67.56623796471993, 67.41865204863306, 67.22889872794995, 67.09536861339518, 66.9618384988404, 66.81425258275353, 66.59638765900625, 66.40663433832313, 66.27310422376836, 66.1255183076815, 65.9919881931267, 65.85845807857193, 65.7530395670813, 65.57734204793027, 65.43678403260945, 65.35244922341697, 65.3243376203528, 65.17675170426594, 65.07133319277531, 64.97997048281677, 64.93780307822054, 64.91671937592241, 64.83238456672991, 64.846440368262, 64.85346826902804, 64.79021716213367, 64.7269660552393, 64.69885445217514, 64.71291025370721, 64.71993815447325, 64.79021716213367, 64.76913345983554, 64.71993815447325, 64.6918265514091, 64.69885445217514, 64.67777074987701, 64.6426312460468, 64.68479865064305, 64.68479865064305, 64.60046384145055, 64.60046384145055, 64.58640803991848, 64.57938013915243, 64.54424063532223, 64.53721273455618, 64.49504532995994, 64.44585002459765, 64.41773842153349, 64.38962681846931, 64.3685431161712, 64.34745941387308, 64.33340361234099, 64.2842083069787, 64.27718040621266, 64.27015250544662, 64.25609670391454, 64.21392929931828, 64.17878979548807, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.990723170988826, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.98826340572071, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 109, 194, 285, 374, 454, 527, 599, 664, 738, 808, 879, 935, 1000, 1067, 1118, 1181, 1233, 1293, 1342, 1391, 1432, 1482, 1527, 1566, 1604, 1645, 1687, 1719, 1750, 1782, 1820, 1848, 1873, 1892, 1912, 1934, 1965, 1988, 2006, 2023, 2045, 2063, 2075, 2086, 2098, 2113, 2122, 2134, 2144, 2158, 2171, 2185, 2196, 2208, 2215, 2228, 2240, 2260, 2272, 2286, 2293, 2309, 2325, 2338, 2356, 2374, 2386, 2400, 2413, 2430, 2443, 2463, 2473, 2484, 2496, 2506, 2512, 2523, 2534, 2543, 2553, 2559, 2565, 2572, 2575, 2581, 2583, 2583, 2585, 2586, 2588, 2588, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589], "harmed": [0, 0, 1, 1, 1, 3, 4, 5, 6, 6, 6, 6, 7, 8, 9, 10, 12, 16, 17, 24, 30, 37, 40, 45, 55, 63, 73, 78, 95, 100, 119, 142, 158, 177, 200, 228, 256, 284, 309, 343, 368, 401, 425, 458, 496, 527, 561, 591, 634, 671, 704, 738, 771, 801, 828, 860, 893, 917, 941, 974, 1003, 1023, 1045, 1064, 1089, 1105, 1122, 1143, 1166, 1183, 1198, 1210, 1220, 1233, 1251, 1267, 1276, 1285, 1301, 1306, 1315, 1337, 1343, 1351, 1359, 1367, 1374, 1382, 1389, 1395, 1400, 1405, 1408, 1411, 1418, 1419, 1420, 1422, 1428, 1433, 1441], "exact_score": [56.05453650994448, 56.820577693442964, 57.41092135779042, 58.050460327500176, 58.67594349567784, 59.22411975542905, 59.73012861058401, 60.22910956497294, 60.685923114765615, 61.19895987068662, 61.6909129243095, 62.189893878698435, 62.57642842083069, 63.02621406985733, 63.49005552041604, 63.84145055871811, 64.27015250544662, 64.6074917422166, 65.02213788741302, 65.31730971958676, 65.61950945252653, 65.85845807857193, 66.18876941457586, 66.4698854452175, 66.6736945674327, 66.88453159041394, 67.10239651416121, 67.36242884250474, 67.46784735399535, 67.65057277391243, 67.74193548387096, 67.84735399536159, 67.93168880455407, 67.97385620915031, 67.94574460608615, 67.88952139995783, 67.84735399536159, 67.8684376976597, 67.85438189612762, 67.74193548387096, 67.68571227774264, 67.60840536931619, 67.56623796471993, 67.41865204863306, 67.22889872794995, 67.09536861339518, 66.9618384988404, 66.81425258275353, 66.59638765900625, 66.40663433832313, 66.27310422376836, 66.1255183076815, 65.9919881931267, 65.85845807857193, 65.7530395670813, 65.57734204793027, 65.43678403260945, 65.35244922341697, 65.3243376203528, 65.17675170426594, 65.07133319277531, 64.97997048281677, 64.93780307822054, 64.91671937592241, 64.83238456672991, 64.846440368262, 64.85346826902804, 64.79021716213367, 64.7269660552393, 64.69885445217514, 64.71291025370721, 64.71993815447325, 64.79021716213367, 64.76913345983554, 64.71993815447325, 64.6918265514091, 64.69885445217514, 64.67777074987701, 64.6426312460468, 64.68479865064305, 64.68479865064305, 64.60046384145055, 64.60046384145055, 64.58640803991848, 64.57938013915243, 64.54424063532223, 64.53721273455618, 64.49504532995994, 64.44585002459765, 64.41773842153349, 64.38962681846931, 64.3685431161712, 64.34745941387308, 64.33340361234099, 64.2842083069787, 64.27718040621266, 64.27015250544662, 64.25609670391454, 64.21392929931828, 64.17878979548807, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.86519279699095, "threshold_area100": 64.86485346826903, "threshold_area25": 62.36910534823248}, "test": {"score": [26.688029946402747, 27.351615007231377, 27.90743839151519, 28.511470946884838, 28.976547656183538, 29.580580211553187, 29.906701074780933, 30.43983779031847, 30.890735331650742, 31.282080367524035, 31.710291240110035, 32.14700961347589, 32.54969798372231, 33.00343135864788, 33.457164733573435, 33.99880894989082, 34.38448231857755, 34.95732070442107, 35.453591583245895, 35.94702662847744, 36.482999177608264, 37.118225902504044, 37.7024076227207, 38.28091767575079, 38.987040240478684, 39.64495363412075, 40.43047953946063, 41.25570711510649, 42.02138218529336, 42.721833082834706, 43.41944814678275, 44.10004820917109, 44.67855826220117, 45.30527748631711, 45.861100870600914, 46.29498341037348, 46.887672631369995, 47.383943510194825, 47.851856053086806, 48.251708589739955, 48.57499361937442, 48.813203641210336, 49.12798117006494, 49.38887786064714, 49.64126705044948, 49.82843206760628, 49.984402915236934, 50.106343759748185, 50.25664294019228, 50.42112128860279, 50.59410713779315, 50.67067464481184, 50.718883815897684, 50.86351132915521, 50.94858633695375, 51.02231801037915, 51.06485551427842, 51.14709468848368, 51.20381136034937, 51.24067719706208, 51.2747072001815, 51.317244704080764, 51.35978220798003, 51.35978220798003, 51.36261804157331, 51.376797209539745, 51.45052888296515, 51.47037971811814, 51.476051385304714, 51.478887218897995, 51.46187221733829, 51.42784221421888, 51.45336471655844, 51.4420213821853, 51.42500638062559, 51.40515554547259, 51.34560304001361, 51.303065536114346, 51.2548563650285, 51.22933386268894, 51.20948302753594, 51.113064685364265, 51.09321385021127, 51.02798967756572, 50.96843717210674, 50.91455633383433, 50.8464963275955, 50.76142131979695, 50.66500297762527, 50.59977880497972, 50.54589796670732, 50.46082295890877, 50.398434619856516, 50.35022544877067, 50.276493775345266, 50.239627938632566, 50.17723959958029, 50.14320959646088, 50.114851260528035, 50.05813458866234, 50.00708958398321], "calls": [0.0, 0.8535859115787087, 1.6674701528514306, 2.589116070668973, 3.3689703088222784, 4.367183733658509, 5.016589626520716, 5.949578878711397, 6.780478121543827, 7.574511527663557, 8.365709100190001, 9.307205853160538, 10.214672603011655, 11.176020191135185, 12.072143606613164, 13.161103706434506, 14.00618211723336, 15.322008904517483, 16.371267334032837, 17.545302441652723, 18.77321838754502, 20.17979184981425, 21.623231148796187, 23.007117942319145, 24.697274763916855, 26.359073249581716, 28.080424240705554, 29.986104415392905, 31.999546266625075, 33.8541814366333, 35.70598077304823, 38.04270765391487, 39.86047698721039, 41.78317216345745, 43.57258316082012, 45.27691915038425, 47.47469018517994, 49.33783285596801, 51.20664719394266, 52.89396818194708, 54.609647505884354, 55.85174261974307, 57.400107761676544, 58.798173723165924, 60.06295550577092, 61.48087230241329, 62.654907410033175, 63.82894251765306, 65.00297762527295, 66.04656438760173, 67.33970450613957, 68.28970875988996, 69.15463800584182, 70.25777727362959, 71.1652440234807, 72.10390494285795, 72.91778918413068, 73.7486884269631, 74.66466267759408, 75.41615857981454, 76.09959447579617, 76.72064203272552, 77.32183875450188, 78.11020049343504, 78.79930805660324, 79.46856478461844, 80.21155318605904, 80.83543657658169, 81.575589144429, 82.33559254742931, 83.0445509457505, 83.70530017298586, 84.36888523381448, 85.02679862745654, 85.68187618750531, 86.2972520772481, 86.98068797322973, 87.58472052859938, 88.18875308396903, 88.78711397215211, 89.26920568301053, 90.09443325865638, 90.6105549726342, 91.24861753112327, 91.78742591384737, 92.32623429657147, 92.98414769021353, 93.63922525026231, 94.24609363922525, 94.83878286022176, 95.43430791481155, 96.10923631001333, 96.63670135836429, 97.16416640671525, 97.78521396364461, 98.26163400731645, 98.70118821427558, 99.14074242123472, 99.4810424524289, 99.87238748830218, 100.0], "rescued": [0, 235, 431, 644, 808, 1021, 1137, 1325, 1484, 1623, 1775, 1932, 2077, 2242, 2405, 2604, 2743, 2952, 3133, 3316, 3509, 3744, 3961, 4172, 4439, 4692, 4981, 5298, 5598, 5873, 6146, 6452, 6691, 6955, 7191, 7393, 7659, 7882, 8100, 8280, 8444, 8571, 8722, 8863, 8985, 9117, 9215, 9313, 9406, 9492, 9599, 9664, 9726, 9810, 9877, 9941, 9989, 10047, 10100, 10141, 10190, 10227, 10265, 10306, 10336, 10368, 10411, 10445, 10472, 10500, 10517, 10534, 10562, 10575, 10601, 10619, 10633, 10643, 10650, 10659, 10668, 10676, 10684, 10690, 10694, 10698, 10704, 10706, 10708, 10712, 10715, 10720, 10720, 10722, 10723, 10723, 10723, 10723, 10723, 10723, 10723], "harmed": [0, 1, 1, 1, 1, 1, 2, 2, 2, 3, 4, 7, 10, 15, 18, 26, 29, 36, 42, 51, 55, 66, 77, 84, 102, 123, 135, 161, 191, 219, 246, 312, 347, 390, 430, 479, 536, 584, 637, 676, 726, 769, 809, 858, 891, 957, 1000, 1055, 1095, 1123, 1169, 1207, 1252, 1285, 1322, 1360, 1393, 1422, 1455, 1483, 1520, 1542, 1565, 1606, 1635, 1662, 1679, 1706, 1731, 1758, 1781, 1810, 1829, 1846, 1878, 1903, 1938, 1963, 1987, 2005, 2021, 2063, 2078, 2107, 2132, 2155, 2185, 2217, 2253, 2280, 2302, 2337, 2359, 2378, 2405, 2418, 2440, 2452, 2462, 2482, 2500], "exact_score": [26.688029946402747, 27.467884184556052, 28.11729007741826, 28.74968096872076, 29.37072852565012, 29.89819357400108, 30.465360292658026, 31.001332841788848, 31.52312622295324, 32.00521793381165, 32.464622975923774, 32.924028018035905, 33.41179139608088, 33.916569775685566, 34.37881065139098, 34.84672319428296, 35.286277401242096, 35.72583160820123, 36.145534980007376, 36.59076085415308, 37.03598672829879, 37.50106343759748, 37.86121430394465, 38.27524600856422, 38.680770212403935, 39.09480191702352, 39.52584862320279, 39.9512236621955, 40.39644953634121, 40.8246604089272, 41.26137878229306, 41.62436548223351, 42.02138218529336, 42.3872047188271, 42.77571392110711, 43.14153645464084, 43.524373989734286, 43.81646484984261, 44.08303320761138, 44.42049740521227, 44.71542409891388, 45.05288829651476, 45.387516660522365, 45.688115021410546, 45.949011711992746, 46.221251736948076, 46.49916342908998, 46.76856762045203, 47.0351359782208, 47.30170433598957, 47.54275019141877, 47.81215438278082, 48.02200606868389, 48.27439525848624, 48.47290361001617, 48.648725292799824, 48.85006947792304, 49.04574199585968, 49.213056177863486, 49.42857953095313, 49.62708788248305, 49.75753622777416, 49.89365624025183, 50.0099254175765, 50.1205229277146, 50.25380710659899, 50.42395712219607, 50.543062133114034, 50.639480475285715, 50.707540481524546, 50.80679465728951, 50.91455633383433, 51.002467175226165, 51.076198848651565, 51.178288858009815, 51.212318861129226, 51.26903553299492, 51.356946374386744, 51.373961375946465, 51.356946374386744, 51.41082721265916, 51.46470805093158, 51.48739471967785, 51.46754388452486, 51.433513881405446, 51.4420213821853, 51.42500638062559, 51.34560304001361, 51.28888636814792, 51.22649802909566, 51.11873635255083, 51.050676346311995, 50.95142217054703, 50.84366049400222, 50.718883815897684, 50.59127130419987, 50.46933045968862, 50.36156878314381, 50.25664294019228, 50.146045430054166, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 44.50826578342142, "threshold_area100": 45.407410033179254, "threshold_area25": 32.987834273884815}, "selected_budget": 33, "selected": {"val": {"score": 67.97385620915031, "calls": 32.99599409656336, "rescued": 1873, "harmed": 177}, "test": {"score": 45.30527748631711, "calls": 41.78317216345745, "rescued": 6955, "harmed": 390}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.433995246887207, "operator": "ge"}, {"budget": 0.02, "threshold": 2.045276641845703, "operator": "ge"}, {"budget": 0.03, "threshold": 1.8186099529266357, "operator": "ge"}, {"budget": 0.04, "threshold": 1.6509273052215576, "operator": "ge"}, {"budget": 0.05, "threshold": 1.536529302597046, "operator": "ge"}, {"budget": 0.06, "threshold": 1.4515502452850342, "operator": "ge"}, {"budget": 0.07, "threshold": 1.3702774047851562, "operator": "ge"}, {"budget": 0.08, "threshold": 1.3000301122665405, "operator": "ge"}, {"budget": 0.09, "threshold": 1.244943380355835, "operator": "ge"}, {"budget": 0.1, "threshold": 1.191996455192566, "operator": "ge"}, {"budget": 0.11, "threshold": 1.1382683515548706, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0928685665130615, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0570321083068848, "operator": "ge"}, {"budget": 0.14, "threshold": 1.0204657316207886, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9828253984451294, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9513581991195679, "operator": "gt"}, {"budget": 0.17, "threshold": 0.9258183240890503, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8976867198944092, "operator": "ge"}, {"budget": 0.19, "threshold": 0.867983341217041, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8406204581260681, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8138808608055115, "operator": "ge"}, {"budget": 0.22, "threshold": 0.7898242473602295, "operator": "gt"}, {"budget": 0.23, "threshold": 0.7686227560043335, "operator": "gt"}, {"budget": 0.24, "threshold": 0.7472631931304932, "operator": "ge"}, {"budget": 0.25, "threshold": 0.7244426012039185, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7043342590332031, "operator": "ge"}, {"budget": 0.27, "threshold": 0.6818571090698242, "operator": "ge"}, {"budget": 0.28, "threshold": 0.6620323061943054, "operator": "ge"}, {"budget": 0.29, "threshold": 0.6430568695068359, "operator": "ge"}, {"budget": 0.3, "threshold": 0.6248967051506042, "operator": "ge"}, {"budget": 0.31, "threshold": 0.6038792133331299, "operator": "ge"}, {"budget": 0.32, "threshold": 0.5868833661079407, "operator": "ge"}, {"budget": 0.33, "threshold": 0.5699662566184998, "operator": "ge"}, {"budget": 0.34, "threshold": 0.5540202856063843, "operator": "ge"}, {"budget": 0.35, "threshold": 0.5328758955001831, "operator": "ge"}, {"budget": 0.36, "threshold": 0.5138828158378601, "operator": "ge"}, {"budget": 0.37, "threshold": 0.4975280165672302, "operator": "ge"}, {"budget": 0.38, "threshold": 0.48064300417900085, "operator": "ge"}, {"budget": 0.39, "threshold": 0.46370649337768555, "operator": "ge"}, {"budget": 0.4, "threshold": 0.4475395679473877, "operator": "ge"}, {"budget": 0.41, "threshold": 0.4333837032318115, "operator": "ge"}, {"budget": 0.42, "threshold": 0.4149419069290161, "operator": "ge"}, {"budget": 0.43, "threshold": 0.3970569670200348, "operator": "ge"}, {"budget": 0.44, "threshold": 0.38302910327911377, "operator": "ge"}, {"budget": 0.45, "threshold": 0.3683590888977051, "operator": "ge"}, {"budget": 0.46, "threshold": 0.352276086807251, "operator": "ge"}, {"budget": 0.47, "threshold": 0.33803653717041016, "operator": "ge"}, {"budget": 0.48, "threshold": 0.3236151933670044, "operator": "ge"}, {"budget": 0.49, "threshold": 0.3081068992614746, "operator": "ge"}, {"budget": 0.5, "threshold": 0.29383814334869385, "operator": "ge"}, {"budget": 0.51, "threshold": 0.2783069610595703, "operator": "ge"}, {"budget": 0.52, "threshold": 0.2645385265350342, "operator": "ge"}, {"budget": 0.53, "threshold": 0.2522449493408203, "operator": "ge"}, {"budget": 0.54, "threshold": 0.23820090293884277, "operator": "ge"}, {"budget": 0.55, "threshold": 0.22369122505187988, "operator": "ge"}, {"budget": 0.56, "threshold": 0.2086353302001953, "operator": "ge"}, {"budget": 0.57, "threshold": 0.19434714317321777, "operator": "ge"}, {"budget": 0.58, "threshold": 0.18065810203552246, "operator": "ge"}, {"budget": 0.59, "threshold": 0.166642427444458, "operator": "ge"}, {"budget": 0.6, "threshold": 0.1523762345314026, "operator": "ge"}, {"budget": 0.61, "threshold": 0.13964182138442993, "operator": "ge"}, {"budget": 0.62, "threshold": 0.127213716506958, "operator": "ge"}, {"budget": 0.63, "threshold": 0.11355975270271301, "operator": "ge"}, {"budget": 0.64, "threshold": 0.09826803207397461, "operator": "ge"}, {"budget": 0.65, "threshold": 0.08237731456756592, "operator": "ge"}, {"budget": 0.66, "threshold": 0.06779134273529053, "operator": "ge"}, {"budget": 0.67, "threshold": 0.05033731460571289, "operator": "ge"}, {"budget": 0.68, "threshold": 0.029083728790283203, "operator": "ge"}, {"budget": 0.69, "threshold": 0.0020747631788253784, "operator": "ge"}, {"budget": 0.7, "threshold": -0.047288477420806885, "operator": "ge"}, {"budget": 0.71, "threshold": -0.10879850387573242, "operator": "ge"}, {"budget": 0.72, "threshold": -0.17248600721359253, "operator": "ge"}, {"budget": 0.73, "threshold": -0.2512517273426056, "operator": "ge"}, {"budget": 0.74, "threshold": -0.33450138568878174, "operator": "ge"}, {"budget": 0.75, "threshold": -0.4288424849510193, "operator": "ge"}, {"budget": 0.76, "threshold": -0.5377907752990723, "operator": "ge"}, {"budget": 0.77, "threshold": -0.6502153277397156, "operator": "ge"}, {"budget": 0.78, "threshold": -0.771719217300415, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9089993834495544, "operator": "ge"}, {"budget": 0.8, "threshold": -1.056881070137024, "operator": "ge"}, {"budget": 0.81, "threshold": -1.2545695304870605, "operator": "ge"}, {"budget": 0.82, "threshold": -1.4091275930404663, "operator": "ge"}, {"budget": 0.83, "threshold": -1.5831162929534912, "operator": "ge"}, {"budget": 0.84, "threshold": -1.7592072486877441, "operator": "ge"}, {"budget": 0.85, "threshold": -2.0013082027435303, "operator": "gt"}, {"budget": 0.86, "threshold": -2.195796012878418, "operator": "ge"}, {"budget": 0.87, "threshold": -2.421173095703125, "operator": "ge"}, {"budget": 0.88, "threshold": -2.633657693862915, "operator": "ge"}, {"budget": 0.89, "threshold": -2.8961806297302246, "operator": "ge"}, {"budget": 0.9, "threshold": -3.186068534851074, "operator": "ge"}, {"budget": 0.91, "threshold": -3.4970948696136475, "operator": "ge"}, {"budget": 0.92, "threshold": -3.802410125732422, "operator": "ge"}, {"budget": 0.93, "threshold": -4.183797359466553, "operator": "ge"}, {"budget": 0.94, "threshold": -4.590959548950195, "operator": "ge"}, {"budget": 0.95, "threshold": -5.029201507568359, "operator": "ge"}, {"budget": 0.96, "threshold": -5.503259658813477, "operator": "ge"}, {"budget": 0.97, "threshold": -6.134362697601318, "operator": "ge"}, {"budget": 0.98, "threshold": -7.074155807495117, "operator": "ge"}, {"budget": 0.99, "threshold": -8.643445014953613, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.81354979267692, 57.48120036545083, 57.99423712137184, 58.556469182655135, 59.02031063321386, 59.37873357228195, 59.88474242743692, 60.208025862674816, 60.53833719867875, 60.910815939278926, 61.15679246609037, 61.45196429826411, 61.74713613043783, 62.05636376414364, 62.323423993253215, 62.625623726192984, 62.878628153770464, 63.075409375219614, 63.31435800126501, 63.52519502424625, 63.82739475718602, 63.97498067327289, 64.13662239089184, 64.32637571157494, 64.55126853608827, 64.63560334528076, 64.77616136060158, 64.89563567362428, 65.00808208588094, 65.09241689507344, 65.21891910886217, 65.28217021575655, 65.43678403260945, 65.57031414716423, 65.55625834563216, 65.65464895635674, 65.64059315482464, 65.74601166631527, 65.86548597933796, 65.87251388010401, 65.8373743762738, 65.8373743762738, 65.86548597933796, 65.85845807857193, 65.90062548316817, 65.89359758240214, 65.87251388010401, 65.88656968163609, 65.81629067397567, 65.78115117014546, 65.74601166631527, 65.7179000632511, 65.77412326937943, 65.73898376554924, 65.6827605594209, 65.62653735329256, 65.5984257502284, 65.528146742568, 65.41570033031134, 65.38056082648113, 65.3946166280132, 65.37353292571508, 65.2751423149905, 65.17675170426594, 65.04322158971115, 65.05727739124323, 64.91671937592241, 64.78318926136762, 64.69885445217514, 64.61451964298264, 64.65668704757888, 64.61451964298264, 64.64965914681284, 64.62857544451472, 64.58640803991848, 64.57235223838639, 64.53018483379014, 64.49504532995994, 64.51612903225806, 64.51612903225806, 64.49504532995994, 64.4528779253637, 64.4528779253637, 64.46693372689577, 64.53721273455618, 64.50207323072597, 64.44585002459765, 64.42476632229952, 64.4036826200014, 64.38962681846931, 64.38259891770328, 64.36151521540515, 64.3193478108089, 64.2842083069787, 64.27015250544662, 64.26312460468057, 64.24204090238246, 64.19284559702017, 64.17878979548807, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.988474242743692, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.990301496942863, 22.98826340572071, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.98840396373603, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 114, 216, 304, 393, 470, 538, 623, 685, 745, 810, 863, 917, 972, 1032, 1092, 1150, 1199, 1249, 1302, 1344, 1398, 1437, 1476, 1516, 1568, 1598, 1632, 1667, 1702, 1737, 1772, 1794, 1832, 1862, 1880, 1911, 1935, 1961, 1992, 2011, 2030, 2052, 2074, 2093, 2110, 2130, 2144, 2161, 2175, 2193, 2212, 2225, 2245, 2261, 2274, 2284, 2296, 2309, 2322, 2337, 2352, 2364, 2371, 2377, 2383, 2400, 2403, 2405, 2413, 2422, 2436, 2447, 2465, 2474, 2485, 2493, 2501, 2511, 2524, 2531, 2543, 2552, 2557, 2565, 2578, 2582, 2582, 2585, 2587, 2588, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589], "harmed": [0, 6, 13, 28, 37, 48, 65, 78, 94, 107, 119, 137, 149, 162, 178, 200, 215, 228, 250, 269, 281, 292, 310, 326, 339, 359, 377, 391, 409, 428, 451, 468, 481, 497, 508, 528, 545, 571, 582, 596, 614, 638, 660, 678, 698, 709, 730, 747, 762, 786, 809, 833, 850, 862, 883, 904, 922, 938, 961, 990, 1010, 1023, 1038, 1059, 1079, 1104, 1119, 1142, 1163, 1183, 1204, 1212, 1229, 1242, 1254, 1271, 1281, 1295, 1310, 1320, 1327, 1342, 1357, 1362, 1368, 1371, 1380, 1388, 1394, 1399, 1402, 1404, 1407, 1413, 1418, 1420, 1421, 1424, 1431, 1433, 1441], "exact_score": [56.05453650994448, 56.81354979267692, 57.48120036545083, 57.99423712137184, 58.556469182655135, 59.02031063321386, 59.37873357228195, 59.88474242743692, 60.208025862674816, 60.53833719867875, 60.910815939278926, 61.15679246609037, 61.45196429826411, 61.74713613043783, 62.05636376414364, 62.323423993253215, 62.625623726192984, 62.878628153770464, 63.075409375219614, 63.31435800126501, 63.52519502424625, 63.82739475718602, 63.97498067327289, 64.14365029165788, 64.32637571157494, 64.55126853608827, 64.63560334528076, 64.77616136060158, 64.89563567362428, 65.00808208588094, 65.09241689507344, 65.21891910886217, 65.28217021575655, 65.43678403260945, 65.57031414716423, 65.55625834563216, 65.65464895635674, 65.64059315482464, 65.74601166631527, 65.86548597933796, 65.87251388010401, 65.8373743762738, 65.8373743762738, 65.86548597933796, 65.85845807857193, 65.90062548316817, 65.89359758240214, 65.87251388010401, 65.88656968163609, 65.81629067397567, 65.78115117014546, 65.74601166631527, 65.7179000632511, 65.77412326937943, 65.73898376554924, 65.6827605594209, 65.62653735329256, 65.5984257502284, 65.528146742568, 65.41570033031134, 65.38056082648113, 65.3946166280132, 65.37353292571508, 65.2751423149905, 65.17675170426594, 65.04322158971115, 65.05727739124323, 64.91671937592241, 64.78318926136762, 64.69885445217514, 64.61451964298264, 64.65668704757888, 64.61451964298264, 64.64965914681284, 64.62857544451472, 64.58640803991848, 64.57235223838639, 64.53018483379014, 64.49504532995994, 64.51612903225806, 64.51612903225806, 64.49504532995994, 64.4528779253637, 64.4528779253637, 64.46693372689577, 64.53721273455618, 64.50207323072597, 64.44585002459765, 64.42476632229952, 64.4036826200014, 64.38962681846931, 64.38259891770328, 64.36151521540515, 64.3193478108089, 64.2842083069787, 64.27015250544662, 64.26312460468057, 64.24204090238246, 64.19284559702017, 64.17878979548807, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.10054593240893, "threshold_area100": 64.10021786492374, "threshold_area25": 61.216529622601726}, "test": {"score": [26.688029946402747, 27.666392536085983, 28.880129314011853, 30.122224427870574, 31.24521453081133, 32.20656211893486, 32.97223718912174, 33.73791225930863, 34.63119984119332, 35.40254657856677, 36.15404248078723, 37.013300059552506, 37.813005132858805, 38.41987352182174, 38.984204406885404, 39.537191957575935, 40.073164506706746, 40.51271871366588, 41.037347928423564, 41.578992144740944, 42.11212886027848, 42.599892238323456, 43.02526727731617, 43.40810481240961, 43.80228568187619, 44.22198905368233, 44.61333408955562, 45.04438079573491, 45.41871083004849, 45.7334883589031, 46.07095255650398, 46.42543175566458, 46.728865950146044, 47.018120976661095, 47.33006267192241, 47.61081019765761, 47.885886056206225, 48.20066358506083, 48.40200777018405, 48.614695289680405, 48.82454697558347, 49.03156282789326, 49.26410118254261, 49.5079828715651, 49.669625386382336, 49.83410373479285, 50.05529875506905, 50.20276210191986, 50.338882114397535, 50.446643790942346, 50.60261463857301, 50.744406318237246, 50.92873550180076, 51.005303008819446, 51.1272438533307, 51.21515469472251, 51.28888636814792, 51.348438873606895, 51.40515554547259, 51.47037971811814, 51.544111391543545, 51.569633893883115, 51.58097722825625, 51.606499730595814, 51.569633893883115, 51.569633893883115, 51.56396222669654, 51.54694722513683, 51.51858888920399, 51.478887218897995, 51.43634971499873, 51.39948387828602, 51.36545387516661, 51.356946374386744, 51.3853047103196, 51.376797209539745, 51.3853047103196, 51.35978220798003, 51.28888636814792, 51.22366219550237, 51.15560218926354, 51.09888551739784, 51.02231801037915, 51.005303008819446, 50.93157133539405, 50.818137991662645, 50.772764654170096, 50.69336131355813, 50.63097297450586, 50.56574880186031, 50.49201712843491, 50.406942120636366, 50.3445537815841, 50.30768794487139, 50.2509712730057, 50.19709043473329, 50.16589626520715, 50.12903042849446, 50.1035079261549, 50.05813458866234, 50.00708958398321], "calls": [0.0, 1.1711992740266, 2.7280719167399257, 4.3955420695913565, 5.997788049797237, 7.407197345659757, 8.674814961858038, 10.047358421007855, 11.592887729348043, 12.962595354904574, 14.39469131951337, 16.07350480673794, 17.511272438533307, 18.78456172191816, 19.964268496724614, 21.31128945353487, 22.4201003885092, 23.466522984431272, 24.67175226157729, 25.959220712928566, 27.283555000992543, 28.53132178203783, 29.742222726370414, 30.726256983240223, 31.939993761166097, 33.13387970393897, 34.25403397328645, 35.54433825823101, 36.59359668774636, 37.60315344695574, 38.65524771006437, 39.86331282080368, 40.878541247199614, 41.91078467515526, 42.98272977341689, 44.14825738025693, 45.24856081445141, 46.50767092986984, 47.517227689079206, 48.59200862093412, 49.64126705044948, 50.59410713779315, 51.89575475711086, 53.06695403113745, 54.05665995519383, 55.103082551115904, 56.228908487649946, 57.21577857811304, 58.191305334202994, 59.266086266057904, 60.366389700252384, 61.497887303972995, 62.479085727249526, 63.443269148966344, 64.42163173864958, 65.47089016816493, 66.73850778436321, 67.90970705838981, 69.03269716133057, 70.11031392677877, 71.25882653205909, 72.30241329438789, 73.35167172390325, 74.55406516745597, 75.96063862972521, 77.39557042792728, 78.72557638317784, 80.05841817202166, 81.41961829679835, 82.33559254742931, 83.05589428012364, 83.76201684485154, 84.36888523381448, 85.07500779854237, 85.7045628562516, 86.34829708192723, 86.92397130136403, 87.4883021864277, 88.13770807928991, 88.76726313699912, 89.41383319626804, 90.2390607719139, 90.85160082806341, 91.4301108810935, 92.05399427161615, 92.83384850976944, 93.37265689249354, 93.92848027677736, 94.52400533136715, 95.07415704846439, 95.67818960383406, 96.31908799591639, 96.92028471769277, 97.50446643790941, 98.05178232141338, 98.48566486118595, 98.84865156112639, 99.20596659388026, 99.60298329694014, 99.90074582423503, 100.0], "rescued": [0, 358, 802, 1267, 1688, 2048, 2339, 2636, 2982, 3276, 3566, 3902, 4208, 4450, 4673, 4895, 5112, 5294, 5505, 5720, 5934, 6135, 6319, 6468, 6633, 6811, 6972, 7155, 7308, 7441, 7584, 7734, 7868, 7991, 8123, 8247, 8369, 8497, 8596, 8694, 8792, 8883, 8998, 9103, 9185, 9272, 9367, 9444, 9510, 9578, 9654, 9732, 9814, 9861, 9928, 9985, 10050, 10107, 10156, 10203, 10254, 10289, 10320, 10357, 10388, 10419, 10450, 10470, 10486, 10500, 10511, 10529, 10541, 10559, 10588, 10601, 10619, 10632, 10642, 10649, 10658, 10670, 10674, 10687, 10694, 10703, 10709, 10711, 10713, 10715, 10717, 10718, 10723, 10723, 10723, 10723, 10723, 10723, 10723, 10723, 10723], "harmed": [0, 13, 29, 56, 81, 102, 123, 150, 181, 203, 228, 261, 285, 313, 337, 364, 392, 419, 445, 469, 495, 524, 558, 572, 598, 628, 651, 682, 703, 725, 749, 774, 801, 822, 844, 869, 894, 911, 939, 962, 986, 1004, 1037, 1056, 1081, 1110, 1127, 1152, 1170, 1200, 1221, 1249, 1266, 1286, 1310, 1336, 1375, 1411, 1440, 1464, 1489, 1515, 1542, 1570, 1614, 1645, 1678, 1704, 1730, 1758, 1784, 1815, 1839, 1860, 1879, 1895, 1910, 1932, 1967, 1997, 2030, 2062, 2093, 2112, 2145, 2194, 2216, 2246, 2270, 2295, 2323, 2354, 2381, 2394, 2414, 2433, 2444, 2457, 2466, 2482, 2500], "exact_score": [26.688029946402747, 27.521765022828465, 28.32714176332133, 29.087145166321644, 29.835805234948815, 30.558942801236427, 31.24521453081133, 31.957008762725803, 32.57238465246859, 33.16790970705839, 33.7067180897825, 34.27672064203273, 34.889260698182234, 35.422397413719764, 35.94419079488416, 36.44613334089556, 36.976434222839806, 37.543600941496756, 38.054050988288004, 38.52479936477328, 38.987040240478684, 39.409579445878116, 39.86047698721039, 40.28868785979639, 40.75943623628165, 41.19331877605422, 41.59600714630066, 42.032725519666506, 42.40705555398009, 42.74168391798769, 43.133028953860986, 43.50735898817458, 43.822136517029186, 44.165272381816635, 44.533930748943654, 44.87423078013782, 45.20318747695885, 45.54632334174631, 45.89229504012705, 46.15319173070924, 46.45662592519071, 46.73453761733262, 47.04080764540737, 47.33857017270226, 47.58528769531804, 47.82916938434053, 48.06737940617645, 48.29141026004594, 48.50976944672887, 48.69693446388566, 48.85857697870289, 49.11096616850523, 49.30096701925532, 49.49663953719195, 49.66395371919576, 49.82843206760628, 50.00708958398321, 50.17156793239372, 50.29634461049825, 50.409777954229654, 50.5685846354536, 50.66500297762527, 50.832317159629085, 50.96843717210674, 51.073363015058284, 51.17545302441653, 51.246348864248645, 51.29739386892778, 51.356946374386744, 51.39948387828602, 51.46754388452486, 51.52709638998384, 51.549783058730114, 51.56679806028982, 51.6263505657488, 51.58664889544282, 51.5724697274764, 51.569633893883115, 51.569633893883115, 51.569633893883115, 51.54694722513683, 51.53276805717041, 51.49873805405099, 51.439185548592015, 51.39664804469274, 51.351274707200176, 51.39381221109945, 51.376797209539745, 51.303065536114346, 51.178288858009815, 51.107393018177696, 51.02231801037915, 50.937243002580615, 50.81246632447608, 50.68201797918499, 50.5685846354536, 50.46649462609534, 50.341717947990816, 50.25664294019228, 50.1573887644273, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.507979796322594, "threshold_area100": 46.79192071009274, "threshold_area25": 37.092986983523815}, "selected_budget": 45, "selected": {"val": {"score": 65.90062548316817, "calls": 44.9996486049617, "rescued": 2110, "harmed": 709}, "test": {"score": 49.83410373479285, "calls": 55.103082551115904, "rescued": 9272, "harmed": 1110}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 66.02712769695691, "calls": 39.995783259540374}, "test": {"score": 49.249922014576185, "calls": 51.14993052207697}}, "old_normalized_interior": {"val": {"score": 66.02712769695691, "calls": 39.995783259540374}, "test": {"score": 49.249922014576185, "calls": 51.14993052207697}}, "old_normalized_endpoints": {"val": {"score": 66.02712769695691, "calls": 39.995783259540374}, "test": {"score": 49.249922014576185, "calls": 51.14993052207697}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 56.86977299880526, 57.57959097617541, 58.1066835336285, 58.64080399184763, 59.16789654930072, 59.6598496029236, 60.06043994658795, 60.482113992550424, 60.882704336214765, 61.220043572984736, 61.53629910745659, 61.81038723733221, 62.11258697027198, 62.456954107807995, 62.766181741513805, 63.06135357368754, 63.28624639820085, 63.49708342118209, 63.74305994799353, 63.925785367910606, 64.18581769625412, 64.36151521540515, 64.6074917422166, 64.77616136060158, 64.93780307822054, 65.00105418511491, 65.16972380349989, 65.26108651345842, 65.38056082648113, 65.45083983414153, 65.54220254410006, 65.55625834563216, 65.64762105559069, 65.64762105559069, 65.8022348724436, 65.87954178087006, 65.87251388010401, 65.87954178087006, 65.94982078853047, 66.02712769695691, 65.97793239159463, 65.97793239159463, 65.96387659006254, 65.94982078853047, 65.90765338393422, 65.8022348724436, 65.80926277320964, 65.7530395670813, 65.74601166631527, 65.7179000632511, 65.66870475788882, 65.5984257502284, 65.59139784946237, 65.53517464333403, 65.50706304026987, 65.48597933797174, 65.45083983414153, 65.4297561318434, 65.40164452877924, 65.31730971958676, 65.22594700962821, 65.14864010120176, 65.11350059737156, 65.00808208588094, 64.85346826902804, 64.80427296366575, 64.77616136060158, 64.74102185677137, 64.69885445217514, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.68479865064305, 64.68479865064305, 64.66371494834493, 64.65668704757888, 64.67074284911097, 64.65668704757888, 64.62154754374868, 64.62857544451472, 64.59343594068451, 64.59343594068451, 64.58640803991848, 64.56532433762034, 64.56532433762034, 64.60046384145055, 64.54424063532223, 64.5582964368543, 64.52315693302411, 64.51612903225806, 64.41071052076744, 64.41071052076744, 64.38259891770328, 64.3193478108089, 64.27015250544662, 64.22798510085038, 64.24906880314849, 64.20690139855225, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.992339588165017, 21.990301496942863, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.989177032820297, 26.994166842364187, 27.999156651908073, 28.990090659919883, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.98784173167475, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 56.86977299880526, 57.57959097617541, 58.1066835336285, 58.64080399184763, 59.16789654930072, 59.6598496029236, 60.06043994658795, 60.482113992550424, 60.882704336214765, 61.220043572984736, 61.53629910745659, 61.81038723733221, 62.11258697027198, 62.456954107807995, 62.766181741513805, 63.06135357368754, 63.28624639820085, 63.49708342118209, 63.74305994799353, 63.925785367910606, 64.19284559702017, 64.36151521540515, 64.6074917422166, 64.77616136060158, 64.93780307822054, 65.00808208588094, 65.16972380349989, 65.26108651345842, 65.38758872724716, 65.45083983414153, 65.54220254410006, 65.55625834563216, 65.64762105559069, 65.64762105559069, 65.8022348724436, 65.87954178087006, 65.87251388010401, 65.87954178087006, 65.94982078853047, 66.02712769695691, 65.97793239159463, 65.97793239159463, 65.96387659006254, 65.94982078853047, 65.90765338393422, 65.8022348724436, 65.80926277320964, 65.7530395670813, 65.74601166631527, 65.7179000632511, 65.66870475788882, 65.5984257502284, 65.59139784946237, 65.53517464333403, 65.50706304026987, 65.48597933797174, 65.45083983414153, 65.4297561318434, 65.40164452877924, 65.31730971958676, 65.22594700962821, 65.14864010120176, 65.11350059737156, 65.00808208588094, 64.85346826902804, 64.80427296366575, 64.77616136060158, 64.74102185677137, 64.69885445217514, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.6918265514091, 64.68479865064305, 64.68479865064305, 64.66371494834493, 64.65668704757888, 64.67074284911097, 64.65668704757888, 64.62154754374868, 64.62857544451472, 64.59343594068451, 64.59343594068451, 64.58640803991848, 64.56532433762034, 64.56532433762034, 64.60046384145055, 64.54424063532223, 64.5582964368543, 64.52315693302411, 64.51612903225806, 64.41071052076744, 64.41071052076744, 64.38259891770328, 64.3193478108089, 64.27015250544662, 64.22798510085038, 64.24906880314849, 64.20690139855225, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 61.51521540515847, "exact_area100": 64.22057738030156}, "test": {"score": [26.688029946402747, 27.714601707171827, 29.13535433740748, 30.394464452825908, 31.486260386240538, 32.31999546266625, 33.26716388282336, 34.17463063267447, 35.16433655673085, 35.955534129257295, 36.89135921504126, 37.614496781328874, 38.2610668405978, 38.84241272722117, 39.45495278337067, 39.99376116609478, 40.543912883192014, 41.136602104188526, 41.71511215721861, 42.19153220089045, 42.67645974534214, 43.12735728667442, 43.62079233190597, 44.034824036525535, 44.48572157785781, 44.964977455122934, 45.33647165584323, 45.69945835578368, 46.065280889317414, 46.43393925644443, 46.72319428295948, 47.06349431415365, 47.327226838329125, 47.64200436718373, 47.91140855854579, 48.18932025068769, 48.45305277486317, 48.66290446076624, 48.841561977143186, 49.06559283101268, 49.249922014576185, 49.47962453563225, 49.70365538950175, 49.85679040353912, 50.018432918356346, 50.1205229277146, 50.27932960893855, 50.42962878938264, 50.62813714091258, 50.74157048464396, 50.85216799478207, 50.9967955080396, 51.06485551427842, 51.172617190823246, 51.19530385956952, 51.27187136658821, 51.351274707200176, 51.40515554547259, 51.42500638062559, 51.481723052491276, 51.49023055327113, 51.49306638686441, 51.50440972123755, 51.51291722201742, 51.51291722201742, 51.510081388424126, 51.52426055639056, 51.51291722201742, 51.48739471967785, 51.478887218897995, 51.47321555171143, 51.476051385304714, 51.46754388452486, 51.46754388452486, 51.46754388452486, 51.45620055015172, 51.45620055015172, 51.45336471655844, 51.45336471655844, 51.439185548592015, 51.45336471655844, 51.439185548592015, 51.41082721265916, 51.3853047103196, 51.39664804469274, 51.3853047103196, 51.37963304313303, 51.37112554235318, 51.3853047103196, 51.348438873606895, 51.33142387204719, 51.29455803533449, 51.28037886736807, 51.22933386268894, 51.20097552675609, 51.1272438533307, 51.08470634943142, 51.008138842412734, 50.818137991662645, 50.65082380965885, 50.00708958398321], "calls": [0.0, 1.2307517794855798, 3.0428494455945323, 4.628080424240705, 6.14241556305476, 7.4780931854918755, 9.05198082976491, 10.634375974817797, 12.299010294075943, 13.847375436009415, 15.466636417775003, 16.827836542551683, 18.118140827496244, 19.43680344837365, 20.690241896605507, 21.852933669852252, 23.120551286050535, 24.44488557411451, 25.840115702010607, 27.022658310410346, 28.210872585996654, 29.45296769985537, 30.71774948246037, 31.8832770893004, 33.24164138048379, 34.625528174006746, 35.620905765249695, 36.712701698664326, 37.9065876414372, 39.08345858265037, 40.06465700592689, 41.306752119785614, 42.307801378215125, 43.49885148739472, 44.664379094234754, 45.82423503388821, 46.96424013838868, 47.99081189915776, 48.855741145109604, 50.1205229277146, 51.14993052207697, 52.270084791424445, 53.51501573887645, 54.374273317641716, 55.599353429940734, 56.52667101494484, 57.50786943822137, 58.667725377874824, 59.80205881518873, 60.80594390721152, 61.83818733516717, 62.98953577404078, 64.00192836684343, 65.42835266426566, 66.6108952726654, 67.70269120608002, 68.85403964495363, 69.8664322377563, 70.81927232509996, 72.12091994441766, 73.22689504579871, 74.34704931514618, 75.51824858917279, 76.78303037177778, 77.82661713410657, 79.05453307999886, 79.91946232595072, 80.74752573518987, 81.60678331395513, 82.31290587868304, 83.38768681053796, 84.08813770807929, 84.7375436009415, 85.3132178203783, 85.97113121402036, 86.56949210220344, 87.08277798258798, 87.61307886453224, 88.14905141366305, 88.68785979638714, 89.20681734395825, 89.72861072512265, 90.38368828517143, 90.87428749680969, 91.25428919830985, 91.62011173184358, 92.03130760286986, 92.51056348013499, 92.87638601366872, 93.37549272608683, 93.76116609477356, 94.31698947905737, 94.82460369225534, 95.32371040467346, 95.8143096163117, 96.29356549357684, 96.7643138700621, 97.3598389246519, 98.09715565890593, 98.6785015455293, 100.0], "exact_score": [26.688029946402747, 27.527436690015033, 28.33848509769447, 29.106996001474634, 29.88117857244137, 30.661032810594673, 31.358647874542722, 32.04775543771092, 32.65462382667385, 33.23596971329722, 33.797464764767604, 34.40716898732383, 34.994186541133764, 35.5273232566713, 36.03777330346256, 36.61344752289936, 37.21748007826901, 37.719422624280405, 38.207186002325386, 38.63539687491138, 39.106145251396654, 39.608087797408054, 40.041970337180615, 40.48436037773303, 40.9380937526586, 41.346453790091594, 41.78033632986416, 42.1830247001106, 42.56869806879733, 42.971386439043755, 43.32302980461107, 43.7257181748575, 44.07452570683152, 44.40631823724584, 44.72109576610045, 45.1209483027536, 45.47542750191419, 45.77886169639566, 46.08796755806369, 46.40841675410487, 46.714686782179626, 46.99543430791481, 47.262002665683575, 47.52857102345235, 47.752601877321844, 48.024841902277174, 48.23469358818024, 48.46156027564303, 48.66574029435953, 48.86708447948275, 49.06559283101268, 49.20171284349035, 49.40022119502028, 49.63843121685619, 49.80574539886, 49.950372912117516, 50.07514959022205, 50.20559793551315, 50.33604628080425, 50.47216629328191, 50.647987976065565, 50.758585486203664, 50.87201882993506, 50.9967955080396, 51.06485551427842, 51.1442588548904, 51.19246802597623, 51.19530385956952, 51.28605053455464, 51.373961375946465, 51.40799137906588, 51.433513881405446, 51.478887218897995, 51.46754388452486, 51.48739471967785, 51.50440972123755, 51.52142472279727, 51.50157388764427, 51.50157388764427, 51.50724555483084, 51.5157530556107, 51.49873805405099, 51.48739471967785, 51.47321555171143, 51.476051385304714, 51.46754388452486, 51.46470805093158, 51.45052888296515, 51.459036383745, 51.4420213821853, 51.43634971499873, 51.38814054391288, 51.37963304313303, 51.376797209539745, 51.3285880384539, 51.26903553299492, 51.16978135722996, 51.056348013498564, 50.86067549556192, 50.50619629640133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 37.541615857981455, "exact_area100": 45.650087926804666}}, "old_normalized_endpoint_budget": 40}, {"seed": 2028, "selection": {"best_epoch": 5, "epochs": 13, "alpha": 0.0, "validation_area100": 0.6409907036465725, "training_seconds": 94.88460223493166, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8850100040435791, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8432767391204834, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8065958023071289, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7791589498519897, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7538490891456604, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7345713376998901, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7146455645561218, "operator": "ge"}, {"budget": 0.08, "threshold": 0.6965819001197815, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6758154630661011, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6569340825080872, "operator": "gt"}, {"budget": 0.11, "threshold": 0.6402528285980225, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6244896650314331, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6116959452629089, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5974500775337219, "operator": "ge"}, {"budget": 0.15, "threshold": 0.584125816822052, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5725865364074707, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5597429275512695, "operator": "ge"}, {"budget": 0.18, "threshold": 0.548369824886322, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5369041562080383, "operator": "ge"}, {"budget": 0.2, "threshold": 0.5247812867164612, "operator": "ge"}, {"budget": 0.21, "threshold": 0.5140100121498108, "operator": "ge"}, {"budget": 0.22, "threshold": 0.5032576322555542, "operator": "ge"}, {"budget": 0.23, "threshold": 0.49275830388069153, "operator": "ge"}, {"budget": 0.24, "threshold": 0.48229315876960754, "operator": "ge"}, {"budget": 0.25, "threshold": 0.4705337584018707, "operator": "ge"}, {"budget": 0.26, "threshold": 0.458519846200943, "operator": "ge"}, {"budget": 0.27, "threshold": 0.4470427334308624, "operator": "ge"}, {"budget": 0.28, "threshold": 0.43683889508247375, "operator": "ge"}, {"budget": 0.29, "threshold": 0.42536988854408264, "operator": "ge"}, {"budget": 0.3, "threshold": 0.4161379337310791, "operator": "ge"}, {"budget": 0.31, "threshold": 0.40433281660079956, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3954279124736786, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3848888874053955, "operator": "ge"}, {"budget": 0.34, "threshold": 0.37405815720558167, "operator": "ge"}, {"budget": 0.35, "threshold": 0.36531704664230347, "operator": "ge"}, {"budget": 0.36, "threshold": 0.3549477159976959, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3450532555580139, "operator": "ge"}, {"budget": 0.38, "threshold": 0.33457866311073303, "operator": "ge"}, {"budget": 0.39, "threshold": 0.32583534717559814, "operator": "ge"}, {"budget": 0.4, "threshold": 0.31656506657600403, "operator": "ge"}, {"budget": 0.41, "threshold": 0.3043389916419983, "operator": "ge"}, {"budget": 0.42, "threshold": 0.29500797390937805, "operator": "ge"}, {"budget": 0.43, "threshold": 0.28526008129119873, "operator": "gt"}, {"budget": 0.44, "threshold": 0.2752467393875122, "operator": "ge"}, {"budget": 0.45, "threshold": 0.2652455270290375, "operator": "ge"}, {"budget": 0.46, "threshold": 0.2558439373970032, "operator": "ge"}, {"budget": 0.47, "threshold": 0.2454410046339035, "operator": "ge"}, {"budget": 0.48, "threshold": 0.2364504635334015, "operator": "ge"}, {"budget": 0.49, "threshold": 0.22705885767936707, "operator": "ge"}, {"budget": 0.5, "threshold": 0.2168606072664261, "operator": "ge"}, {"budget": 0.51, "threshold": 0.20703881978988647, "operator": "ge"}, {"budget": 0.52, "threshold": 0.19574327766895294, "operator": "ge"}, {"budget": 0.53, "threshold": 0.1850920170545578, "operator": "ge"}, {"budget": 0.54, "threshold": 0.17506363987922668, "operator": "ge"}, {"budget": 0.55, "threshold": 0.1657063066959381, "operator": "ge"}, {"budget": 0.56, "threshold": 0.15545037388801575, "operator": "ge"}, {"budget": 0.57, "threshold": 0.14530381560325623, "operator": "ge"}, {"budget": 0.58, "threshold": 0.13338765501976013, "operator": "ge"}, {"budget": 0.59, "threshold": 0.1226821094751358, "operator": "ge"}, {"budget": 0.6, "threshold": 0.11289012432098389, "operator": "ge"}, {"budget": 0.61, "threshold": 0.10178772360086441, "operator": "ge"}, {"budget": 0.62, "threshold": 0.0903121829032898, "operator": "ge"}, {"budget": 0.63, "threshold": 0.07642698287963867, "operator": "ge"}, {"budget": 0.64, "threshold": 0.06259369850158691, "operator": "ge"}, {"budget": 0.65, "threshold": 0.04906044155359268, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03442089632153511, "operator": "ge"}, {"budget": 0.67, "threshold": 0.018224645406007767, "operator": "ge"}, {"budget": 0.68, "threshold": 0.007184046786278486, "operator": "ge"}, {"budget": 0.69, "threshold": 0.0011200467124581337, "operator": "ge"}, {"budget": 0.7, "threshold": -0.00023023516405373812, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0006463035242632031, "operator": "ge"}, {"budget": 0.72, "threshold": -0.001125634997151792, "operator": "ge"}, {"budget": 0.73, "threshold": -0.001709158648736775, "operator": "ge"}, {"budget": 0.74, "threshold": -0.0022698617540299892, "operator": "ge"}, {"budget": 0.75, "threshold": -0.0030123332981020212, "operator": "ge"}, {"budget": 0.76, "threshold": -0.003841181518509984, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0046654678881168365, "operator": "ge"}, {"budget": 0.78, "threshold": -0.0057424502447247505, "operator": "ge"}, {"budget": 0.79, "threshold": -0.006944762542843819, "operator": "ge"}, {"budget": 0.8, "threshold": -0.008050257340073586, "operator": "ge"}, {"budget": 0.81, "threshold": -0.009573927149176598, "operator": "ge"}, {"budget": 0.82, "threshold": -0.011482877656817436, "operator": "ge"}, {"budget": 0.83, "threshold": -0.013435086235404015, "operator": "ge"}, {"budget": 0.84, "threshold": -0.015885010361671448, "operator": "ge"}, {"budget": 0.85, "threshold": -0.018484564498066902, "operator": "ge"}, {"budget": 0.86, "threshold": -0.021802883595228195, "operator": "ge"}, {"budget": 0.87, "threshold": -0.025964869186282158, "operator": "ge"}, {"budget": 0.88, "threshold": -0.030833570286631584, "operator": "ge"}, {"budget": 0.89, "threshold": -0.03768719732761383, "operator": "ge"}, {"budget": 0.9, "threshold": -0.04408536106348038, "operator": "ge"}, {"budget": 0.91, "threshold": -0.052715420722961426, "operator": "ge"}, {"budget": 0.92, "threshold": -0.06320929527282715, "operator": "ge"}, {"budget": 0.93, "threshold": -0.07925781607627869, "operator": "ge"}, {"budget": 0.94, "threshold": -0.09677854180335999, "operator": "ge"}, {"budget": 0.95, "threshold": -0.11996149271726608, "operator": "ge"}, {"budget": 0.96, "threshold": -0.15489567816257477, "operator": "ge"}, {"budget": 0.97, "threshold": -0.1991487741470337, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2632765471935272, "operator": "ge"}, {"budget": 0.99, "threshold": -0.4002554416656494, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.68704757888818, 57.24927964017148, 57.76231639609249, 58.359687961205985, 58.760278304870326, 59.22411975542905, 59.72310070981797, 60.208025862674816, 60.678895213999574, 61.03731815306767, 61.30437838217724, 61.55738280975472, 61.85958254269449, 62.24611708482676, 62.611567924660896, 62.99107456602713, 63.27219059666878, 63.58141823037459, 63.78522735258978, 63.89767376484643, 64.05931548246538, 64.29123620774475, 64.43882212383161, 64.63560334528076, 64.71993815447325, 64.8815798720922, 64.90266357439033, 65.07133319277531, 65.21891910886217, 65.30325391805466, 65.41570033031134, 65.34542132265092, 65.35244922341697, 65.40164452877924, 65.4789514372057, 65.54220254410006, 65.62653735329256, 65.60545365099443, 65.63356525405861, 65.6827605594209, 65.7179000632511, 65.8022348724436, 65.8022348724436, 65.80926277320964, 65.78817907091151, 65.79520697167756, 65.77412326937943, 65.68978846018693, 65.64762105559069, 65.59139784946237, 65.57734204793027, 65.54923044486611, 65.54220254410006, 65.50706304026987, 65.47192353643966, 65.46489563567361, 65.3946166280132, 65.3946166280132, 65.37353292571508, 65.26811441422447, 65.22594700962821, 65.21891910886217, 65.1556680019678, 65.06430529200927, 65.00105418511491, 64.94483097898657, 64.85346826902804, 64.81130086443179, 64.74804975753742, 64.73399395600535, 64.73399395600535, 64.74102185677137, 64.73399395600535, 64.71993815447325, 64.70588235294117, 64.70588235294117, 64.69885445217514, 64.68479865064305, 64.67074284911097, 64.66371494834493, 64.63560334528076, 64.61451964298264, 64.59343594068451, 64.5582964368543, 64.53018483379014, 64.49504532995994, 64.47396162766181, 64.41071052076744, 64.38259891770328, 64.34745941387308, 64.30529200927683, 64.24204090238246, 64.22095720008433, 64.09445498629559, 64.01012017710309, 64.00309227633704, 63.88361796331436, 63.88361796331436, 63.925785367910606, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.986646988544521, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.98966898587392, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 102, 192, 280, 375, 444, 519, 596, 677, 757, 825, 876, 931, 992, 1059, 1123, 1186, 1243, 1298, 1342, 1379, 1418, 1469, 1509, 1553, 1585, 1626, 1652, 1691, 1730, 1760, 1790, 1809, 1832, 1857, 1880, 1903, 1934, 1955, 1979, 2008, 2031, 2053, 2078, 2099, 2114, 2133, 2147, 2156, 2170, 2184, 2203, 2219, 2236, 2255, 2270, 2285, 2299, 2317, 2333, 2341, 2354, 2365, 2373, 2380, 2387, 2396, 2405, 2410, 2412, 2414, 2415, 2416, 2416, 2416, 2416, 2417, 2417, 2417, 2418, 2418, 2418, 2419, 2419, 2419, 2421, 2423, 2427, 2428, 2431, 2437, 2444, 2448, 2457, 2461, 2471, 2485, 2494, 2516, 2545, 2589], "harmed": [0, 12, 22, 37, 47, 59, 68, 74, 86, 99, 116, 129, 148, 166, 178, 190, 199, 216, 227, 242, 263, 279, 297, 316, 332, 352, 370, 393, 408, 426, 444, 458, 487, 509, 527, 539, 553, 572, 596, 616, 638, 656, 666, 691, 711, 729, 747, 764, 785, 805, 827, 848, 868, 886, 910, 930, 946, 970, 988, 1007, 1030, 1049, 1061, 1078, 1098, 1114, 1131, 1153, 1164, 1175, 1179, 1180, 1180, 1181, 1183, 1185, 1186, 1187, 1189, 1192, 1193, 1197, 1201, 1204, 1209, 1215, 1222, 1229, 1239, 1246, 1257, 1270, 1283, 1295, 1317, 1339, 1354, 1380, 1402, 1425, 1441], "exact_score": [56.05453650994448, 56.68704757888818, 57.24927964017148, 57.76231639609249, 58.359687961205985, 58.760278304870326, 59.22411975542905, 59.72310070981797, 60.208025862674816, 60.678895213999574, 61.03731815306767, 61.30437838217724, 61.55738280975472, 61.85958254269449, 62.24611708482676, 62.611567924660896, 62.99107456602713, 63.27219059666878, 63.58141823037459, 63.78522735258978, 63.89767376484643, 64.05931548246538, 64.29123620774475, 64.43882212383161, 64.63560334528076, 64.71993815447325, 64.8815798720922, 64.90266357439033, 65.07133319277531, 65.21891910886217, 65.30325391805466, 65.41570033031134, 65.34542132265092, 65.35244922341697, 65.40164452877924, 65.4789514372057, 65.54220254410006, 65.62653735329256, 65.60545365099443, 65.63356525405861, 65.6827605594209, 65.7179000632511, 65.8022348724436, 65.8022348724436, 65.80926277320964, 65.78817907091151, 65.79520697167756, 65.77412326937943, 65.68978846018693, 65.64762105559069, 65.59139784946237, 65.57734204793027, 65.54923044486611, 65.54220254410006, 65.50706304026987, 65.47192353643966, 65.46489563567361, 65.3946166280132, 65.3946166280132, 65.37353292571508, 65.26811441422447, 65.22594700962821, 65.21891910886217, 65.1556680019678, 65.06430529200927, 65.00105418511491, 64.94483097898657, 64.85346826902804, 64.81130086443179, 64.74804975753742, 64.73399395600535, 64.73399395600535, 64.74102185677137, 64.73399395600535, 64.71993815447325, 64.70588235294117, 64.70588235294117, 64.69885445217514, 64.68479865064305, 64.67074284911097, 64.66371494834493, 64.63560334528076, 64.61451964298264, 64.59343594068451, 64.5582964368543, 64.53018483379014, 64.49504532995994, 64.47396162766181, 64.41071052076744, 64.38259891770328, 64.34745941387308, 64.30529200927683, 64.24204090238246, 64.22095720008433, 64.09445498629559, 64.01012017710309, 64.00309227633704, 63.88361796331436, 63.88361796331436, 63.925785367910606, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.09907036465725, "threshold_area100": 64.09881228477053, "threshold_area25": 61.33881509593084}, "test": {"score": [26.688029946402747, 28.891472648384998, 30.476703627031164, 31.897456257266825, 32.98925219068145, 34.04418228738338, 34.80702152397698, 35.67195076992882, 36.45464084167541, 37.42733176417208, 38.22703683747838, 39.015398576411535, 39.650625301307315, 40.22629952074412, 40.87570541360633, 41.462722967416276, 41.93063551030826, 42.48929472818536, 43.01108810934975, 43.45347814990217, 43.89870402404787, 44.21348155290248, 44.57930408643621, 44.93094745200352, 45.24856081445141, 45.63423418313813, 46.031250886198, 46.459461758783995, 46.785582622011745, 47.196778493038025, 47.42931684768738, 47.75827354450841, 47.987976065564474, 48.23469358818024, 48.50126194594901, 48.69126279669909, 48.92380115134844, 49.12798117006494, 49.352012023934435, 49.513654538751666, 49.61007288092335, 49.83126790119956, 49.93619374415109, 50.06097042225562, 50.23395627144599, 50.39559878626323, 50.509032129994615, 50.645152142472284, 50.752913819017095, 50.849332161188784, 50.98261634007317, 51.02231801037915, 51.08470634943142, 51.11873635255083, 51.16410969004339, 51.19246802597623, 51.25769219862178, 51.30022970252106, 51.38814054391288, 51.376797209539745, 51.3853047103196, 51.43634971499873, 51.42784221421888, 51.430678047812165, 51.39664804469274, 51.382468876726314, 51.35978220798003, 51.356946374386744, 51.348438873606895, 51.33142387204719, 51.32575220486062, 51.32291637126733, 51.32008053767405, 51.32008053767405, 51.31440887048748, 51.308737203300915, 51.30590136970763, 51.29172220174121, 51.29739386892778, 51.29739386892778, 51.29455803533449, 51.28321470096135, 51.26336386580835, 51.24918469784193, 51.23500552987551, 51.22649802909566, 51.212318861129226, 51.20664719394266, 51.18396052519638, 51.16978135722996, 51.14993052207697, 51.12440801973741, 51.09604968380456, 51.050676346311995, 51.042168845532146, 51.03933301193886, 51.008138842412734, 50.92306383461419, 50.835152993222366, 50.6593313104387, 50.00708958398321], "calls": [0.0, 2.770609420639197, 4.85778294529677, 6.9251056348013496, 8.510336613447523, 10.078552590533988, 11.295125202053145, 12.803788673680627, 14.264242974222272, 15.979922298159543, 17.59634744633185, 19.08516008280634, 20.517256047415138, 21.762186994867143, 23.222641295408785, 24.606528088931743, 25.706831523126223, 27.068031647902902, 28.23639508833622, 29.435952698295665, 30.797152823072345, 31.894620423673537, 32.97507302271503, 34.01582395145053, 35.11329155205172, 36.553895017440375, 37.91225930862377, 39.17987692482205, 40.302867027762815, 41.62436548223351, 42.52332473130477, 43.84482318577546, 44.857215778578116, 45.91498170887333, 47.05498681337379, 47.9454385616652, 49.162011173184354, 50.06947792303548, 51.20381136034937, 52.15381561409976, 53.069789864730744, 54.371437484048435, 55.38383007685109, 56.46711850948586, 57.46816776791538, 58.56279953492329, 59.592207129285654, 60.74071973456597, 61.60848481411111, 62.60953407254062, 63.63894166690298, 64.6399909253325, 65.82253353373224, 66.93701613589315, 68.02314040212121, 68.9901596574313, 70.04508975413323, 71.04897484615603, 72.21733828658934, 73.15316337237331, 74.21660096985508, 75.33959107279586, 76.3916853359045, 77.74154212630803, 78.84751722768908, 79.73229730879392, 80.69648073051073, 81.60678331395513, 82.40932422085471, 83.50112015426934, 84.58157275331084, 85.23097864617304, 85.93142954371437, 86.58083543657659, 87.08844964977455, 87.6924822051442, 88.24263392224144, 88.68502396279386, 89.2153248447381, 89.63786405013754, 89.9583132461787, 90.29861327737288, 90.66159997731333, 90.98772084054107, 91.31951337095539, 91.63429089981, 91.9490684286646, 92.22414428721322, 92.57862348637383, 92.9557893542807, 93.29041771828828, 93.57967274480333, 93.97385361426991, 94.43042282278876, 94.91535036724045, 95.42012874684514, 96.04401213736779, 96.63953719195759, 97.37401809261831, 98.4204406885404, 100.0], "rescued": [0, 823, 1403, 1940, 2341, 2732, 3014, 3346, 3650, 4016, 4334, 4629, 4879, 5100, 5351, 5583, 5772, 5998, 6204, 6394, 6582, 6720, 6866, 7006, 7154, 7332, 7500, 7673, 7817, 7988, 8084, 8230, 8336, 8459, 8577, 8664, 8771, 8861, 8956, 9039, 9105, 9218, 9287, 9356, 9443, 9525, 9587, 9662, 9717, 9777, 9850, 9890, 9943, 9987, 10043, 10078, 10128, 10170, 10224, 10245, 10274, 10310, 10340, 10370, 10388, 10404, 10425, 10449, 10463, 10470, 10470, 10471, 10471, 10471, 10471, 10471, 10473, 10475, 10478, 10480, 10482, 10483, 10483, 10484, 10486, 10489, 10494, 10496, 10501, 10506, 10514, 10519, 10532, 10539, 10561, 10581, 10606, 10620, 10647, 10688, 10723], "harmed": [0, 46, 67, 103, 119, 138, 151, 178, 206, 229, 265, 282, 308, 326, 348, 373, 397, 426, 448, 482, 513, 540, 557, 573, 609, 651, 679, 701, 730, 756, 770, 800, 825, 861, 885, 905, 930, 948, 964, 990, 1022, 1057, 1089, 1114, 1140, 1165, 1187, 1214, 1231, 1257, 1283, 1309, 1340, 1372, 1412, 1437, 1464, 1491, 1514, 1539, 1565, 1583, 1616, 1645, 1675, 1696, 1725, 1750, 1767, 1780, 1782, 1784, 1785, 1785, 1787, 1789, 1792, 1799, 1800, 1802, 1805, 1810, 1817, 1823, 1830, 1836, 1846, 1850, 1863, 1873, 1888, 1902, 1925, 1948, 1973, 1994, 2030, 2074, 2132, 2235, 2500], "exact_score": [26.688029946402747, 27.578481694694158, 28.355500099254176, 29.07013016476193, 29.844312735728668, 30.5816294699827, 31.276408700337466, 31.95417292913252, 32.660295493860424, 33.33522388906219, 33.98179394833111, 34.6170206732269, 35.20120239344355, 35.79956328162663, 36.32419249638431, 36.880015880668125, 37.441510932138506, 37.92076680940363, 38.436888523381455, 38.978532739698835, 39.41808694665797, 39.88599948954995, 40.334061197288946, 40.796302072994365, 41.235856279953495, 41.635708816606645, 42.02705385247994, 42.469443893032356, 42.88063976405865, 43.297507302271505, 43.634971499872385, 43.94691319513371, 44.2446757224286, 44.58497575362278, 44.928111618410234, 45.21736664492528, 45.47826333550747, 45.74199585968295, 46.0681167229107, 46.38573008535859, 46.70334344780648, 47.00961347588124, 47.30454016958285, 47.590959362504606, 47.78096021325469, 48.00782690071747, 48.26021609051981, 48.48424694438931, 48.69977029747894, 48.892606981822304, 49.10529450131867, 49.306638686441886, 49.49663953719195, 49.59589371295693, 49.77738706292715, 49.88798457306526, 50.018432918356346, 50.16306043161387, 50.30768794487139, 50.43530045656921, 50.537390465927466, 50.67067464481184, 50.76425715339023, 50.88903383149477, 51.002467175226165, 51.010974676006015, 51.07903468224485, 51.11873635255083, 51.16410969004339, 51.19246802597623, 51.246348864248645, 51.29739386892778, 51.373961375946465, 51.37963304313303, 51.3853047103196, 51.42500638062559, 51.439185548592015, 51.44485721577858, 51.43634971499873, 51.39381221109945, 51.36261804157331, 51.368289708759896, 51.351274707200176, 51.33709553923376, 51.33142387204719, 51.32291637126733, 51.32008053767405, 51.31440887048748, 51.311573036894195, 51.29455803533449, 51.29455803533449, 51.24918469784193, 51.20381136034937, 51.16978135722996, 51.09321385021127, 51.03649717834558, 51.01664634319258, 50.883362164308195, 50.752913819017095, 50.46933045968862, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.66392551285273, "threshold_area100": 47.55773757195928, "threshold_area25": 38.75875563621927}, "selected_budget": 44, "selected": {"val": {"score": 65.80926277320964, "calls": 43.994658795417806, "rescued": 2099, "harmed": 711}, "test": {"score": 50.23395627144599, "calls": 57.46816776791538, "rescued": 9443, "harmed": 1140}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8850100040435791, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8432767391204834, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8065958023071289, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7791589498519897, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7538490891456604, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7345713376998901, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7146455645561218, "operator": "ge"}, {"budget": 0.08, "threshold": 0.6965819001197815, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6758154630661011, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6569340825080872, "operator": "gt"}, {"budget": 0.11, "threshold": 0.6402528285980225, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6244896650314331, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6116959452629089, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5974500775337219, "operator": "ge"}, {"budget": 0.15, "threshold": 0.584125816822052, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5725865364074707, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5597429275512695, "operator": "ge"}, {"budget": 0.18, "threshold": 0.548369824886322, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5369041562080383, "operator": "ge"}, {"budget": 0.2, "threshold": 0.5247812867164612, "operator": "ge"}, {"budget": 0.21, "threshold": 0.5140100121498108, "operator": "ge"}, {"budget": 0.22, "threshold": 0.5032576322555542, "operator": "ge"}, {"budget": 0.23, "threshold": 0.49275830388069153, "operator": "ge"}, {"budget": 0.24, "threshold": 0.48229315876960754, "operator": "ge"}, {"budget": 0.25, "threshold": 0.4705337584018707, "operator": "ge"}, {"budget": 0.26, "threshold": 0.458519846200943, "operator": "ge"}, {"budget": 0.27, "threshold": 0.4470427334308624, "operator": "ge"}, {"budget": 0.28, "threshold": 0.43683889508247375, "operator": "ge"}, {"budget": 0.29, "threshold": 0.42536988854408264, "operator": "ge"}, {"budget": 0.3, "threshold": 0.4161379337310791, "operator": "ge"}, {"budget": 0.31, "threshold": 0.40433281660079956, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3954279124736786, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3848888874053955, "operator": "ge"}, {"budget": 0.34, "threshold": 0.37405815720558167, "operator": "ge"}, {"budget": 0.35, "threshold": 0.36531704664230347, "operator": "ge"}, {"budget": 0.36, "threshold": 0.3549477159976959, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3450532555580139, "operator": "ge"}, {"budget": 0.38, "threshold": 0.33457866311073303, "operator": "ge"}, {"budget": 0.39, "threshold": 0.32583534717559814, "operator": "ge"}, {"budget": 0.4, "threshold": 0.31656506657600403, "operator": "ge"}, {"budget": 0.41, "threshold": 0.3043389916419983, "operator": "ge"}, {"budget": 0.42, "threshold": 0.29500797390937805, "operator": "ge"}, {"budget": 0.43, "threshold": 0.28526008129119873, "operator": "gt"}, {"budget": 0.44, "threshold": 0.2752467393875122, "operator": "ge"}, {"budget": 0.45, "threshold": 0.2652455270290375, "operator": "ge"}, {"budget": 0.46, "threshold": 0.2558439373970032, "operator": "ge"}, {"budget": 0.47, "threshold": 0.2454410046339035, "operator": "ge"}, {"budget": 0.48, "threshold": 0.2364504635334015, "operator": "ge"}, {"budget": 0.49, "threshold": 0.22705885767936707, "operator": "ge"}, {"budget": 0.5, "threshold": 0.2168606072664261, "operator": "ge"}, {"budget": 0.51, "threshold": 0.20703881978988647, "operator": "ge"}, {"budget": 0.52, "threshold": 0.19574327766895294, "operator": "ge"}, {"budget": 0.53, "threshold": 0.1850920170545578, "operator": "ge"}, {"budget": 0.54, "threshold": 0.17506363987922668, "operator": "ge"}, {"budget": 0.55, "threshold": 0.1657063066959381, "operator": "ge"}, {"budget": 0.56, "threshold": 0.15545037388801575, "operator": "ge"}, {"budget": 0.57, "threshold": 0.14530381560325623, "operator": "ge"}, {"budget": 0.58, "threshold": 0.13338765501976013, "operator": "ge"}, {"budget": 0.59, "threshold": 0.1226821094751358, "operator": "ge"}, {"budget": 0.6, "threshold": 0.11289012432098389, "operator": "ge"}, {"budget": 0.61, "threshold": 0.10178772360086441, "operator": "ge"}, {"budget": 0.62, "threshold": 0.0903121829032898, "operator": "ge"}, {"budget": 0.63, "threshold": 0.07642698287963867, "operator": "ge"}, {"budget": 0.64, "threshold": 0.06259369850158691, "operator": "ge"}, {"budget": 0.65, "threshold": 0.04906044155359268, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03442089632153511, "operator": "ge"}, {"budget": 0.67, "threshold": 0.018224645406007767, "operator": "ge"}, {"budget": 0.68, "threshold": 0.007184046786278486, "operator": "ge"}, {"budget": 0.69, "threshold": 0.0011200467124581337, "operator": "ge"}, {"budget": 0.7, "threshold": -0.00023023516405373812, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0006463035242632031, "operator": "ge"}, {"budget": 0.72, "threshold": -0.001125634997151792, "operator": "ge"}, {"budget": 0.73, "threshold": -0.001709158648736775, "operator": "ge"}, {"budget": 0.74, "threshold": -0.0022698617540299892, "operator": "ge"}, {"budget": 0.75, "threshold": -0.0030123332981020212, "operator": "ge"}, {"budget": 0.76, "threshold": -0.003841181518509984, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0046654678881168365, "operator": "ge"}, {"budget": 0.78, "threshold": -0.0057424502447247505, "operator": "ge"}, {"budget": 0.79, "threshold": -0.006944762542843819, "operator": "ge"}, {"budget": 0.8, "threshold": -0.008050257340073586, "operator": "ge"}, {"budget": 0.81, "threshold": -0.009573927149176598, "operator": "ge"}, {"budget": 0.82, "threshold": -0.011482877656817436, "operator": "ge"}, {"budget": 0.83, "threshold": -0.013435086235404015, "operator": "ge"}, {"budget": 0.84, "threshold": -0.015885010361671448, "operator": "ge"}, {"budget": 0.85, "threshold": -0.018484564498066902, "operator": "ge"}, {"budget": 0.86, "threshold": -0.021802883595228195, "operator": "ge"}, {"budget": 0.87, "threshold": -0.025964869186282158, "operator": "ge"}, {"budget": 0.88, "threshold": -0.030833570286631584, "operator": "ge"}, {"budget": 0.89, "threshold": -0.03768719732761383, "operator": "ge"}, {"budget": 0.9, "threshold": -0.04408536106348038, "operator": "ge"}, {"budget": 0.91, "threshold": -0.052715420722961426, "operator": "ge"}, {"budget": 0.92, "threshold": -0.06320929527282715, "operator": "ge"}, {"budget": 0.93, "threshold": -0.07925781607627869, "operator": "ge"}, {"budget": 0.94, "threshold": -0.09677854180335999, "operator": "ge"}, {"budget": 0.95, "threshold": -0.11996149271726608, "operator": "ge"}, {"budget": 0.96, "threshold": -0.15489567816257477, "operator": "ge"}, {"budget": 0.97, "threshold": -0.1991487741470337, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2632765471935272, "operator": "ge"}, {"budget": 0.99, "threshold": -0.4002554416656494, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.68704757888818, 57.24927964017148, 57.76231639609249, 58.359687961205985, 58.760278304870326, 59.22411975542905, 59.72310070981797, 60.208025862674816, 60.678895213999574, 61.03731815306767, 61.30437838217724, 61.55738280975472, 61.85958254269449, 62.24611708482676, 62.611567924660896, 62.99107456602713, 63.27219059666878, 63.58141823037459, 63.78522735258978, 63.89767376484643, 64.05931548246538, 64.29123620774475, 64.43882212383161, 64.63560334528076, 64.71993815447325, 64.8815798720922, 64.90266357439033, 65.07133319277531, 65.21891910886217, 65.30325391805466, 65.41570033031134, 65.34542132265092, 65.35244922341697, 65.40164452877924, 65.4789514372057, 65.54220254410006, 65.62653735329256, 65.60545365099443, 65.63356525405861, 65.6827605594209, 65.7179000632511, 65.8022348724436, 65.8022348724436, 65.80926277320964, 65.78817907091151, 65.79520697167756, 65.77412326937943, 65.68978846018693, 65.64762105559069, 65.59139784946237, 65.57734204793027, 65.54923044486611, 65.54220254410006, 65.50706304026987, 65.47192353643966, 65.46489563567361, 65.3946166280132, 65.3946166280132, 65.37353292571508, 65.26811441422447, 65.22594700962821, 65.21891910886217, 65.1556680019678, 65.06430529200927, 65.00105418511491, 64.94483097898657, 64.85346826902804, 64.81130086443179, 64.74804975753742, 64.73399395600535, 64.73399395600535, 64.74102185677137, 64.73399395600535, 64.71993815447325, 64.70588235294117, 64.70588235294117, 64.69885445217514, 64.68479865064305, 64.67074284911097, 64.66371494834493, 64.63560334528076, 64.61451964298264, 64.59343594068451, 64.5582964368543, 64.53018483379014, 64.49504532995994, 64.47396162766181, 64.41071052076744, 64.38259891770328, 64.34745941387308, 64.30529200927683, 64.24204090238246, 64.22095720008433, 64.09445498629559, 64.01012017710309, 64.00309227633704, 63.88361796331436, 63.88361796331436, 63.925785367910606, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.986646988544521, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.98966898587392, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 102, 192, 280, 375, 444, 519, 596, 677, 757, 825, 876, 931, 992, 1059, 1123, 1186, 1243, 1298, 1342, 1379, 1418, 1469, 1509, 1553, 1585, 1626, 1652, 1691, 1730, 1760, 1790, 1809, 1832, 1857, 1880, 1903, 1934, 1955, 1979, 2008, 2031, 2053, 2078, 2099, 2114, 2133, 2147, 2156, 2170, 2184, 2203, 2219, 2236, 2255, 2270, 2285, 2299, 2317, 2333, 2341, 2354, 2365, 2373, 2380, 2387, 2396, 2405, 2410, 2412, 2414, 2415, 2416, 2416, 2416, 2416, 2417, 2417, 2417, 2418, 2418, 2418, 2419, 2419, 2419, 2421, 2423, 2427, 2428, 2431, 2437, 2444, 2448, 2457, 2461, 2471, 2485, 2494, 2516, 2545, 2589], "harmed": [0, 12, 22, 37, 47, 59, 68, 74, 86, 99, 116, 129, 148, 166, 178, 190, 199, 216, 227, 242, 263, 279, 297, 316, 332, 352, 370, 393, 408, 426, 444, 458, 487, 509, 527, 539, 553, 572, 596, 616, 638, 656, 666, 691, 711, 729, 747, 764, 785, 805, 827, 848, 868, 886, 910, 930, 946, 970, 988, 1007, 1030, 1049, 1061, 1078, 1098, 1114, 1131, 1153, 1164, 1175, 1179, 1180, 1180, 1181, 1183, 1185, 1186, 1187, 1189, 1192, 1193, 1197, 1201, 1204, 1209, 1215, 1222, 1229, 1239, 1246, 1257, 1270, 1283, 1295, 1317, 1339, 1354, 1380, 1402, 1425, 1441], "exact_score": [56.05453650994448, 56.68704757888818, 57.24927964017148, 57.76231639609249, 58.359687961205985, 58.760278304870326, 59.22411975542905, 59.72310070981797, 60.208025862674816, 60.678895213999574, 61.03731815306767, 61.30437838217724, 61.55738280975472, 61.85958254269449, 62.24611708482676, 62.611567924660896, 62.99107456602713, 63.27219059666878, 63.58141823037459, 63.78522735258978, 63.89767376484643, 64.05931548246538, 64.29123620774475, 64.43882212383161, 64.63560334528076, 64.71993815447325, 64.8815798720922, 64.90266357439033, 65.07133319277531, 65.21891910886217, 65.30325391805466, 65.41570033031134, 65.34542132265092, 65.35244922341697, 65.40164452877924, 65.4789514372057, 65.54220254410006, 65.62653735329256, 65.60545365099443, 65.63356525405861, 65.6827605594209, 65.7179000632511, 65.8022348724436, 65.8022348724436, 65.80926277320964, 65.78817907091151, 65.79520697167756, 65.77412326937943, 65.68978846018693, 65.64762105559069, 65.59139784946237, 65.57734204793027, 65.54923044486611, 65.54220254410006, 65.50706304026987, 65.47192353643966, 65.46489563567361, 65.3946166280132, 65.3946166280132, 65.37353292571508, 65.26811441422447, 65.22594700962821, 65.21891910886217, 65.1556680019678, 65.06430529200927, 65.00105418511491, 64.94483097898657, 64.85346826902804, 64.81130086443179, 64.74804975753742, 64.73399395600535, 64.73399395600535, 64.74102185677137, 64.73399395600535, 64.71993815447325, 64.70588235294117, 64.70588235294117, 64.69885445217514, 64.68479865064305, 64.67074284911097, 64.66371494834493, 64.63560334528076, 64.61451964298264, 64.59343594068451, 64.5582964368543, 64.53018483379014, 64.49504532995994, 64.47396162766181, 64.41071052076744, 64.38259891770328, 64.34745941387308, 64.30529200927683, 64.24204090238246, 64.22095720008433, 64.09445498629559, 64.01012017710309, 64.00309227633704, 63.88361796331436, 63.88361796331436, 63.925785367910606, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.09907036465725, "threshold_area100": 64.09881228477053, "threshold_area25": 61.33881509593084}, "test": {"score": [26.688029946402747, 28.891472648384998, 30.476703627031164, 31.897456257266825, 32.98925219068145, 34.04418228738338, 34.80702152397698, 35.67195076992882, 36.45464084167541, 37.42733176417208, 38.22703683747838, 39.015398576411535, 39.650625301307315, 40.22629952074412, 40.87570541360633, 41.462722967416276, 41.93063551030826, 42.48929472818536, 43.01108810934975, 43.45347814990217, 43.89870402404787, 44.21348155290248, 44.57930408643621, 44.93094745200352, 45.24856081445141, 45.63423418313813, 46.031250886198, 46.459461758783995, 46.785582622011745, 47.196778493038025, 47.42931684768738, 47.75827354450841, 47.987976065564474, 48.23469358818024, 48.50126194594901, 48.69126279669909, 48.92380115134844, 49.12798117006494, 49.352012023934435, 49.513654538751666, 49.61007288092335, 49.83126790119956, 49.93619374415109, 50.06097042225562, 50.23395627144599, 50.39559878626323, 50.509032129994615, 50.645152142472284, 50.752913819017095, 50.849332161188784, 50.98261634007317, 51.02231801037915, 51.08470634943142, 51.11873635255083, 51.16410969004339, 51.19246802597623, 51.25769219862178, 51.30022970252106, 51.38814054391288, 51.376797209539745, 51.3853047103196, 51.43634971499873, 51.42784221421888, 51.430678047812165, 51.39664804469274, 51.382468876726314, 51.35978220798003, 51.356946374386744, 51.348438873606895, 51.33142387204719, 51.32575220486062, 51.32291637126733, 51.32008053767405, 51.32008053767405, 51.31440887048748, 51.308737203300915, 51.30590136970763, 51.29172220174121, 51.29739386892778, 51.29739386892778, 51.29455803533449, 51.28321470096135, 51.26336386580835, 51.24918469784193, 51.23500552987551, 51.22649802909566, 51.212318861129226, 51.20664719394266, 51.18396052519638, 51.16978135722996, 51.14993052207697, 51.12440801973741, 51.09604968380456, 51.050676346311995, 51.042168845532146, 51.03933301193886, 51.008138842412734, 50.92306383461419, 50.835152993222366, 50.6593313104387, 50.00708958398321], "calls": [0.0, 2.770609420639197, 4.85778294529677, 6.9251056348013496, 8.510336613447523, 10.078552590533988, 11.295125202053145, 12.803788673680627, 14.264242974222272, 15.979922298159543, 17.59634744633185, 19.08516008280634, 20.517256047415138, 21.762186994867143, 23.222641295408785, 24.606528088931743, 25.706831523126223, 27.068031647902902, 28.23639508833622, 29.435952698295665, 30.797152823072345, 31.894620423673537, 32.97507302271503, 34.01582395145053, 35.11329155205172, 36.553895017440375, 37.91225930862377, 39.17987692482205, 40.302867027762815, 41.62436548223351, 42.52332473130477, 43.84482318577546, 44.857215778578116, 45.91498170887333, 47.05498681337379, 47.9454385616652, 49.162011173184354, 50.06947792303548, 51.20381136034937, 52.15381561409976, 53.069789864730744, 54.371437484048435, 55.38383007685109, 56.46711850948586, 57.46816776791538, 58.56279953492329, 59.592207129285654, 60.74071973456597, 61.60848481411111, 62.60953407254062, 63.63894166690298, 64.6399909253325, 65.82253353373224, 66.93701613589315, 68.02314040212121, 68.9901596574313, 70.04508975413323, 71.04897484615603, 72.21733828658934, 73.15316337237331, 74.21660096985508, 75.33959107279586, 76.3916853359045, 77.74154212630803, 78.84751722768908, 79.73229730879392, 80.69648073051073, 81.60678331395513, 82.40932422085471, 83.50112015426934, 84.58157275331084, 85.23097864617304, 85.93142954371437, 86.58083543657659, 87.08844964977455, 87.6924822051442, 88.24263392224144, 88.68502396279386, 89.2153248447381, 89.63786405013754, 89.9583132461787, 90.29861327737288, 90.66159997731333, 90.98772084054107, 91.31951337095539, 91.63429089981, 91.9490684286646, 92.22414428721322, 92.57862348637383, 92.9557893542807, 93.29041771828828, 93.57967274480333, 93.97385361426991, 94.43042282278876, 94.91535036724045, 95.42012874684514, 96.04401213736779, 96.63953719195759, 97.37401809261831, 98.4204406885404, 100.0], "rescued": [0, 823, 1403, 1940, 2341, 2732, 3014, 3346, 3650, 4016, 4334, 4629, 4879, 5100, 5351, 5583, 5772, 5998, 6204, 6394, 6582, 6720, 6866, 7006, 7154, 7332, 7500, 7673, 7817, 7988, 8084, 8230, 8336, 8459, 8577, 8664, 8771, 8861, 8956, 9039, 9105, 9218, 9287, 9356, 9443, 9525, 9587, 9662, 9717, 9777, 9850, 9890, 9943, 9987, 10043, 10078, 10128, 10170, 10224, 10245, 10274, 10310, 10340, 10370, 10388, 10404, 10425, 10449, 10463, 10470, 10470, 10471, 10471, 10471, 10471, 10471, 10473, 10475, 10478, 10480, 10482, 10483, 10483, 10484, 10486, 10489, 10494, 10496, 10501, 10506, 10514, 10519, 10532, 10539, 10561, 10581, 10606, 10620, 10647, 10688, 10723], "harmed": [0, 46, 67, 103, 119, 138, 151, 178, 206, 229, 265, 282, 308, 326, 348, 373, 397, 426, 448, 482, 513, 540, 557, 573, 609, 651, 679, 701, 730, 756, 770, 800, 825, 861, 885, 905, 930, 948, 964, 990, 1022, 1057, 1089, 1114, 1140, 1165, 1187, 1214, 1231, 1257, 1283, 1309, 1340, 1372, 1412, 1437, 1464, 1491, 1514, 1539, 1565, 1583, 1616, 1645, 1675, 1696, 1725, 1750, 1767, 1780, 1782, 1784, 1785, 1785, 1787, 1789, 1792, 1799, 1800, 1802, 1805, 1810, 1817, 1823, 1830, 1836, 1846, 1850, 1863, 1873, 1888, 1902, 1925, 1948, 1973, 1994, 2030, 2074, 2132, 2235, 2500], "exact_score": [26.688029946402747, 27.578481694694158, 28.355500099254176, 29.07013016476193, 29.844312735728668, 30.5816294699827, 31.276408700337466, 31.95417292913252, 32.660295493860424, 33.33522388906219, 33.98179394833111, 34.6170206732269, 35.20120239344355, 35.79956328162663, 36.32419249638431, 36.880015880668125, 37.441510932138506, 37.92076680940363, 38.436888523381455, 38.978532739698835, 39.41808694665797, 39.88599948954995, 40.334061197288946, 40.796302072994365, 41.235856279953495, 41.635708816606645, 42.02705385247994, 42.469443893032356, 42.88063976405865, 43.297507302271505, 43.634971499872385, 43.94691319513371, 44.2446757224286, 44.58497575362278, 44.928111618410234, 45.21736664492528, 45.47826333550747, 45.74199585968295, 46.0681167229107, 46.38573008535859, 46.70334344780648, 47.00961347588124, 47.30454016958285, 47.590959362504606, 47.78096021325469, 48.00782690071747, 48.26021609051981, 48.48424694438931, 48.69977029747894, 48.892606981822304, 49.10529450131867, 49.306638686441886, 49.49663953719195, 49.59589371295693, 49.77738706292715, 49.88798457306526, 50.018432918356346, 50.16306043161387, 50.30768794487139, 50.43530045656921, 50.537390465927466, 50.67067464481184, 50.76425715339023, 50.88903383149477, 51.002467175226165, 51.010974676006015, 51.07903468224485, 51.11873635255083, 51.16410969004339, 51.19246802597623, 51.246348864248645, 51.29739386892778, 51.373961375946465, 51.37963304313303, 51.3853047103196, 51.42500638062559, 51.439185548592015, 51.44485721577858, 51.43634971499873, 51.39381221109945, 51.36261804157331, 51.368289708759896, 51.351274707200176, 51.33709553923376, 51.33142387204719, 51.32291637126733, 51.32008053767405, 51.31440887048748, 51.311573036894195, 51.29455803533449, 51.29455803533449, 51.24918469784193, 51.20381136034937, 51.16978135722996, 51.09321385021127, 51.03649717834558, 51.01664634319258, 50.883362164308195, 50.752913819017095, 50.46933045968862, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.66392551285273, "threshold_area100": 47.55773757195928, "threshold_area25": 38.75875563621927}, "selected_budget": 44, "selected": {"val": {"score": 65.80926277320964, "calls": 43.994658795417806, "rescued": 2099, "harmed": 711}, "test": {"score": 50.23395627144599, "calls": 57.46816776791538, "rescued": 9443, "harmed": 1140}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 13.133504867553711, "operator": "ge"}, {"budget": 0.02, "threshold": 12.417236328125, "operator": "ge"}, {"budget": 0.03, "threshold": 11.940855026245117, "operator": "gt"}, {"budget": 0.04, "threshold": 11.547698020935059, "operator": "ge"}, {"budget": 0.05, "threshold": 11.289063453674316, "operator": "ge"}, {"budget": 0.06, "threshold": 11.015571594238281, "operator": "ge"}, {"budget": 0.07, "threshold": 10.79482650756836, "operator": "ge"}, {"budget": 0.08, "threshold": 10.57309627532959, "operator": "ge"}, {"budget": 0.09, "threshold": 10.325749397277832, "operator": "ge"}, {"budget": 0.1, "threshold": 10.118492126464844, "operator": "ge"}, {"budget": 0.11, "threshold": 9.929458618164062, "operator": "ge"}, {"budget": 0.12, "threshold": 9.754900932312012, "operator": "ge"}, {"budget": 0.13, "threshold": 9.598201751708984, "operator": "ge"}, {"budget": 0.14, "threshold": 9.383218765258789, "operator": "ge"}, {"budget": 0.15, "threshold": 9.189183235168457, "operator": "ge"}, {"budget": 0.16, "threshold": 8.98122787475586, "operator": "ge"}, {"budget": 0.17, "threshold": 8.781401634216309, "operator": "ge"}, {"budget": 0.18, "threshold": 8.580816268920898, "operator": "ge"}, {"budget": 0.19, "threshold": 8.390666961669922, "operator": "ge"}, {"budget": 0.2, "threshold": 8.215206146240234, "operator": "ge"}, {"budget": 0.21, "threshold": 8.027751922607422, "operator": "ge"}, {"budget": 0.22, "threshold": 7.832191467285156, "operator": "ge"}, {"budget": 0.23, "threshold": 7.588691711425781, "operator": "ge"}, {"budget": 0.24, "threshold": 7.385389804840088, "operator": "ge"}, {"budget": 0.25, "threshold": 7.194098472595215, "operator": "ge"}, {"budget": 0.26, "threshold": 6.975524425506592, "operator": "ge"}, {"budget": 0.27, "threshold": 6.7683258056640625, "operator": "ge"}, {"budget": 0.28, "threshold": 6.5265021324157715, "operator": "ge"}, {"budget": 0.29, "threshold": 6.302308082580566, "operator": "ge"}, {"budget": 0.3, "threshold": 6.111576557159424, "operator": "ge"}, {"budget": 0.31, "threshold": 5.898871421813965, "operator": "ge"}, {"budget": 0.32, "threshold": 5.688283443450928, "operator": "ge"}, {"budget": 0.33, "threshold": 5.504998207092285, "operator": "ge"}, {"budget": 0.34, "threshold": 5.318801403045654, "operator": "gt"}, {"budget": 0.35, "threshold": 5.123789310455322, "operator": "ge"}, {"budget": 0.36, "threshold": 4.935052871704102, "operator": "ge"}, {"budget": 0.37, "threshold": 4.721581935882568, "operator": "ge"}, {"budget": 0.38, "threshold": 4.553483963012695, "operator": "ge"}, {"budget": 0.39, "threshold": 4.383638381958008, "operator": "ge"}, {"budget": 0.4, "threshold": 4.221519470214844, "operator": "ge"}, {"budget": 0.41, "threshold": 4.078494071960449, "operator": "ge"}, {"budget": 0.42, "threshold": 3.935518741607666, "operator": "ge"}, {"budget": 0.43, "threshold": 3.809988021850586, "operator": "ge"}, {"budget": 0.44, "threshold": 3.650036573410034, "operator": "ge"}, {"budget": 0.45, "threshold": 3.5121748447418213, "operator": "ge"}, {"budget": 0.46, "threshold": 3.3526463508605957, "operator": "ge"}, {"budget": 0.47, "threshold": 3.222421169281006, "operator": "ge"}, {"budget": 0.48, "threshold": 3.110581159591675, "operator": "ge"}, {"budget": 0.49, "threshold": 2.971681833267212, "operator": "ge"}, {"budget": 0.5, "threshold": 2.8196146488189697, "operator": "ge"}, {"budget": 0.51, "threshold": 2.6680238246917725, "operator": "ge"}, {"budget": 0.52, "threshold": 2.523556709289551, "operator": "ge"}, {"budget": 0.53, "threshold": 2.3935415744781494, "operator": "ge"}, {"budget": 0.54, "threshold": 2.2660036087036133, "operator": "ge"}, {"budget": 0.55, "threshold": 2.0990750789642334, "operator": "ge"}, {"budget": 0.56, "threshold": 1.9802772998809814, "operator": "ge"}, {"budget": 0.57, "threshold": 1.841697335243225, "operator": "ge"}, {"budget": 0.58, "threshold": 1.6999924182891846, "operator": "ge"}, {"budget": 0.59, "threshold": 1.557741403579712, "operator": "ge"}, {"budget": 0.6, "threshold": 1.4214593172073364, "operator": "ge"}, {"budget": 0.61, "threshold": 1.2979087829589844, "operator": "ge"}, {"budget": 0.62, "threshold": 1.1700975894927979, "operator": "ge"}, {"budget": 0.63, "threshold": 1.021497130393982, "operator": "ge"}, {"budget": 0.64, "threshold": 0.8753951191902161, "operator": "ge"}, {"budget": 0.65, "threshold": 0.7235904932022095, "operator": "ge"}, {"budget": 0.66, "threshold": 0.5751511454582214, "operator": "ge"}, {"budget": 0.67, "threshold": 0.40846073627471924, "operator": "ge"}, {"budget": 0.68, "threshold": 0.24632489681243896, "operator": "ge"}, {"budget": 0.69, "threshold": 0.07500076293945312, "operator": "ge"}, {"budget": 0.7, "threshold": -0.0582314133644104, "operator": "ge"}, {"budget": 0.71, "threshold": -0.20683592557907104, "operator": "ge"}, {"budget": 0.72, "threshold": -0.36172986030578613, "operator": "ge"}, {"budget": 0.73, "threshold": -0.488823801279068, "operator": "ge"}, {"budget": 0.74, "threshold": -0.6231534481048584, "operator": "ge"}, {"budget": 0.75, "threshold": -0.7766708135604858, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9150981903076172, "operator": "ge"}, {"budget": 0.77, "threshold": -1.0456881523132324, "operator": "ge"}, {"budget": 0.78, "threshold": -1.1785974502563477, "operator": "ge"}, {"budget": 0.79, "threshold": -1.330913782119751, "operator": "ge"}, {"budget": 0.8, "threshold": -1.48206627368927, "operator": "ge"}, {"budget": 0.81, "threshold": -1.6450201272964478, "operator": "ge"}, {"budget": 0.82, "threshold": -1.7992312908172607, "operator": "ge"}, {"budget": 0.83, "threshold": -1.9367225170135498, "operator": "ge"}, {"budget": 0.84, "threshold": -2.0596399307250977, "operator": "ge"}, {"budget": 0.85, "threshold": -2.193333625793457, "operator": "ge"}, {"budget": 0.86, "threshold": -2.323819637298584, "operator": "ge"}, {"budget": 0.87, "threshold": -2.4760286808013916, "operator": "ge"}, {"budget": 0.88, "threshold": -2.6170897483825684, "operator": "gt"}, {"budget": 0.89, "threshold": -2.7874996662139893, "operator": "ge"}, {"budget": 0.9, "threshold": -2.9720215797424316, "operator": "ge"}, {"budget": 0.91, "threshold": -3.153474807739258, "operator": "ge"}, {"budget": 0.92, "threshold": -3.3405237197875977, "operator": "ge"}, {"budget": 0.93, "threshold": -3.6072444915771484, "operator": "ge"}, {"budget": 0.94, "threshold": -3.884667158126831, "operator": "ge"}, {"budget": 0.95, "threshold": -4.149443626403809, "operator": "ge"}, {"budget": 0.96, "threshold": -4.528571128845215, "operator": "ge"}, {"budget": 0.97, "threshold": -4.910614490509033, "operator": "ge"}, {"budget": 0.98, "threshold": -5.423409938812256, "operator": "ge"}, {"budget": 0.99, "threshold": -6.294036865234375, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.708131281186304, 57.36172605242813, 57.83962330451894, 58.422939068100355, 58.928947923255315, 59.49117998453861, 60.05341204582191, 60.531309297912706, 61.016234450769545, 61.430880595965974, 61.95797315341907, 62.44992620704195, 62.864572352238376, 63.28624639820085, 63.70792044416332, 64.11553868859372, 64.45990582612973, 64.86752407056012, 65.21189120809613, 65.54923044486611, 65.85143017780588, 66.21688101764003, 66.45582964368543, 66.77911307892333, 67.0321175065008, 67.28512193407829, 67.54515426242182, 67.69976807927472, 67.84032609459554, 67.98791201068241, 68.09333052217302, 68.26902804132405, 68.33227914821842, 68.33227914821842, 68.24794433902592, 68.19874903366365, 68.12847002600323, 68.05116311757678, 68.00899571298052, 67.88952139995783, 67.76301918616909, 67.72787968233888, 67.58029376625203, 67.44676365169724, 67.24998243024808, 67.067257010331, 66.8283083842856, 66.69477826973082, 66.49799704828168, 66.35743903296085, 66.25202052147023, 66.11849040691546, 65.97793239159463, 65.79520697167756, 65.72492796401714, 65.54923044486611, 65.4438119333755, 65.4438119333755, 65.30325391805466, 65.21891910886217, 65.13458429966968, 65.0853889943074, 64.97997048281677, 64.92374727668845, 64.83941246749595, 64.77616136060158, 64.78318926136762, 64.77616136060158, 64.77616136060158, 64.7269660552393, 64.69885445217514, 64.70588235294117, 64.66371494834493, 64.6426312460468, 64.63560334528076, 64.65668704757888, 64.66371494834493, 64.66371494834493, 64.6426312460468, 64.60046384145055, 64.61451964298264, 64.58640803991848, 64.57235223838639, 64.54424063532223, 64.53018483379014, 64.50910113149202, 64.45990582612973, 64.44585002459765, 64.41071052076744, 64.39665471923536, 64.3685431161712, 64.34745941387308, 64.3544873146391, 64.32637571157494, 64.2842083069787, 64.27718040621266, 64.22095720008433, 64.19987349778621, 64.18581769625412, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.98692810457516, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.98931759083561, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 93, 186, 254, 337, 409, 489, 570, 638, 707, 767, 842, 912, 972, 1033, 1094, 1153, 1202, 1260, 1311, 1360, 1404, 1460, 1497, 1548, 1590, 1638, 1683, 1718, 1752, 1784, 1814, 1853, 1881, 1903, 1921, 1940, 1959, 1970, 1992, 2008, 2023, 2043, 2054, 2070, 2076, 2088, 2098, 2112, 2123, 2136, 2153, 2169, 2179, 2185, 2202, 2212, 2223, 2243, 2257, 2273, 2289, 2304, 2313, 2327, 2339, 2357, 2375, 2387, 2406, 2419, 2429, 2446, 2455, 2462, 2474, 2485, 2493, 2508, 2516, 2523, 2531, 2536, 2542, 2550, 2557, 2563, 2567, 2573, 2575, 2578, 2579, 2581, 2584, 2586, 2587, 2588, 2589, 2589, 2589, 2589], "harmed": [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 4, 5, 6, 6, 6, 8, 9, 10, 14, 17, 22, 28, 40, 48, 61, 75, 86, 101, 115, 134, 156, 186, 212, 241, 263, 291, 324, 357, 382, 414, 449, 483, 521, 565, 598, 637, 670, 702, 737, 767, 799, 826, 861, 887, 907, 941, 969, 997, 1019, 1043, 1065, 1089, 1116, 1133, 1146, 1165, 1185, 1199, 1215, 1230, 1240, 1253, 1261, 1268, 1283, 1294, 1307, 1313, 1322, 1330, 1342, 1351, 1360, 1371, 1379, 1386, 1391, 1396, 1401, 1403, 1409, 1416, 1418, 1427, 1430, 1432, 1441], "exact_score": [56.05453650994448, 56.708131281186304, 57.36172605242813, 57.84665120528497, 58.422939068100355, 58.928947923255315, 59.49117998453861, 60.05341204582191, 60.531309297912706, 61.016234450769545, 61.430880595965974, 61.95797315341907, 62.44992620704195, 62.864572352238376, 63.28624639820085, 63.70792044416332, 64.11553868859372, 64.45990582612973, 64.86752407056012, 65.21189120809613, 65.54923044486611, 65.85143017780588, 66.21688101764003, 66.45582964368543, 66.77911307892333, 67.0321175065008, 67.28512193407829, 67.54515426242182, 67.69976807927472, 67.84032609459554, 67.98791201068241, 68.09333052217302, 68.26902804132405, 68.33227914821842, 68.33227914821842, 68.24794433902592, 68.19874903366365, 68.12847002600323, 68.05116311757678, 68.00899571298052, 67.88952139995783, 67.76301918616909, 67.72787968233888, 67.58029376625203, 67.44676365169724, 67.24998243024808, 67.067257010331, 66.8283083842856, 66.69477826973082, 66.49799704828168, 66.35743903296085, 66.25202052147023, 66.11849040691546, 65.97793239159463, 65.79520697167756, 65.72492796401714, 65.54923044486611, 65.4438119333755, 65.4438119333755, 65.30325391805466, 65.21891910886217, 65.13458429966968, 65.0853889943074, 64.97997048281677, 64.92374727668845, 64.83941246749595, 64.77616136060158, 64.78318926136762, 64.77616136060158, 64.77616136060158, 64.7269660552393, 64.69885445217514, 64.70588235294117, 64.66371494834493, 64.6426312460468, 64.63560334528076, 64.65668704757888, 64.66371494834493, 64.66371494834493, 64.6426312460468, 64.60046384145055, 64.61451964298264, 64.58640803991848, 64.57235223838639, 64.54424063532223, 64.53018483379014, 64.50910113149202, 64.45990582612973, 64.44585002459765, 64.41071052076744, 64.39665471923536, 64.3685431161712, 64.34745941387308, 64.3544873146391, 64.32637571157494, 64.2842083069787, 64.27718040621266, 64.22095720008433, 64.19987349778621, 64.18581769625412, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.89674599205301, "threshold_area100": 64.89640874270854, "threshold_area25": 62.23206128329467}, "test": {"score": [26.688029946402747, 27.28922666817911, 27.90176672432862, 28.47176927657885, 29.007741825709672, 29.387743527209825, 29.866999404474946, 30.343419448146786, 30.75177948557979, 31.23670703003148, 31.681932904177184, 32.07894960723705, 32.436264639990924, 32.850296344610506, 33.38343306014803, 33.8173155999206, 34.2625414740663, 34.721946516178434, 35.15015738876443, 35.654935768369114, 36.09165414173496, 36.54822335025381, 37.08703173297791, 37.69106428834756, 38.28375350934407, 38.83390522644131, 39.5059977880498, 40.155403680912, 40.909735416725745, 41.69242548847234, 42.33899554774126, 43.16705895698041, 43.83347985140232, 44.366616566939854, 44.92527578481695, 45.566174176899295, 46.19572923460852, 46.80259762357145, 47.26483849927686, 47.690213538269575, 48.050364404616744, 48.37648526784448, 48.67991946232595, 48.90962198338202, 49.215892011456766, 49.43992286532627, 49.649774551229335, 49.82276040041971, 49.93052207696452, 50.041119587102635, 50.239627938632566, 50.404106287043085, 50.52604713155432, 50.59977880497972, 50.73873465105068, 50.874854663528346, 50.945750503360465, 50.97410883929331, 51.047840512718714, 51.073363015058284, 51.178288858009815, 51.19530385956952, 51.22082636190909, 51.23216969628223, 51.26903553299492, 51.317244704080764, 51.33709553923376, 51.351274707200176, 51.32575220486062, 51.33709553923376, 51.32008053767405, 51.30022970252106, 51.311573036894195, 51.29739386892778, 51.303065536114346, 51.31440887048748, 51.26903553299492, 51.24067719706208, 51.2179905283158, 51.18963219238295, 51.15560218926354, 51.14709468848368, 51.115900518957545, 51.09321385021127, 51.04500467912543, 50.9967955080396, 50.9570938377336, 50.920228001020895, 50.87201882993506, 50.815302158069365, 50.744406318237246, 50.68201797918499, 50.62530130731929, 50.537390465927466, 50.43530045656921, 50.35873294955052, 50.285001276125115, 50.20559793551315, 50.14320959646088, 50.06947792303548, 50.00708958398321], "calls": [0.0, 0.8564217451719933, 1.8432918356350847, 2.753594419079488, 3.6610611689306074, 4.3530045656920855, 5.183903808524516, 5.992116382610668, 6.808836457476675, 7.738989876074072, 8.578396619686357, 9.426310864078497, 10.189150100672093, 10.96333267163883, 12.006919433967616, 12.93140118537844, 13.938122110994527, 15.035589711595723, 16.03947480361852, 17.114255735473442, 18.10396165952982, 19.207100927317587, 20.474718543515866, 21.986217848736636, 23.50622465473726, 24.887275614666933, 26.614298272977344, 28.310126761761623, 30.368941950486345, 32.49014547826334, 34.299407310779, 36.457476675268694, 38.59002353741882, 40.3794345347815, 42.24541303916286, 44.258854890395035, 46.28647590959363, 48.67991946232595, 50.4381362901625, 52.32963729688341, 54.13322746221252, 55.64472676743328, 56.969061055497264, 58.28488784278139, 59.89847715736041, 61.29937895244307, 62.652071576439894, 63.83745001843292, 64.824320108896, 65.9614893798032, 67.26880866630745, 68.50239627938632, 69.6026997135808, 70.6264356407566, 71.53390239060772, 72.67957916229476, 73.50197090434733, 74.44913932450444, 75.34242690638914, 76.26407282420668, 76.97019538893458, 77.63094461616993, 78.37393301761053, 79.16796642373025, 79.93364149391714, 80.64259989223832, 81.36573745852593, 82.23066670447778, 83.0076851090378, 83.90380852451578, 84.51918441425858, 85.19127697586705, 85.9115787085614, 86.5581487678303, 87.19053965913281, 87.83994555199502, 88.48084394407736, 89.09338400022688, 89.62368488217112, 90.27025494144003, 90.87428749680969, 91.54354422482488, 92.19011428409381, 92.74310183478434, 93.26205938235545, 93.7725094291467, 94.2205711368857, 94.81609619147548, 95.27550123358762, 95.797294614752, 96.36446133340895, 96.8210305419278, 97.2662564160735, 97.81357229957746, 98.28148484246945, 98.6614865439696, 99.02730907750333, 99.3676091086975, 99.62283413209313, 99.84402915236934, 100.0], "rescued": [0, 212, 429, 633, 822, 957, 1126, 1296, 1443, 1614, 1772, 1913, 2039, 2185, 2377, 2532, 2690, 2854, 3011, 3191, 3350, 3518, 3715, 3940, 4164, 4369, 4628, 4883, 5176, 5475, 5729, 6054, 6328, 6551, 6789, 7061, 7326, 7602, 7812, 8015, 8206, 8366, 8511, 8640, 8804, 8927, 9061, 9164, 9249, 9337, 9446, 9555, 9634, 9705, 9786, 9867, 9922, 9983, 10044, 10089, 10146, 10176, 10213, 10248, 10293, 10329, 10361, 10399, 10422, 10458, 10479, 10499, 10520, 10542, 10566, 10590, 10601, 10611, 10625, 10637, 10649, 10663, 10674, 10680, 10685, 10689, 10690, 10697, 10697, 10699, 10704, 10708, 10710, 10717, 10720, 10722, 10722, 10723, 10723, 10723, 10723], "harmed": [0, 0, 1, 4, 4, 5, 5, 7, 10, 10, 11, 12, 12, 12, 16, 18, 19, 21, 27, 29, 34, 41, 48, 60, 75, 86, 108, 134, 161, 184, 210, 243, 282, 317, 358, 404, 447, 509, 556, 609, 673, 718, 756, 804, 860, 904, 964, 1006, 1053, 1102, 1141, 1192, 1228, 1273, 1305, 1338, 1368, 1419, 1454, 1490, 1510, 1534, 1562, 1593, 1625, 1644, 1669, 1702, 1734, 1766, 1793, 1820, 1837, 1864, 1886, 1906, 1933, 1953, 1975, 1997, 2021, 2038, 2060, 2074, 2096, 2117, 2132, 2152, 2169, 2191, 2221, 2247, 2269, 2307, 2346, 2375, 2401, 2430, 2452, 2478, 2500], "exact_score": [26.688029946402747, 27.38280917675751, 28.006692567280155, 28.613560956243088, 29.189235175679894, 29.750730227150274, 30.34625528174007, 30.834018659785045, 31.37849870969572, 31.8832770893004, 32.33984629781925, 32.87014717976349, 33.38059722655475, 33.851345603040016, 34.27672064203273, 34.70493151461873, 35.13314238720472, 35.61239826446984, 36.051952471428976, 36.485835011201544, 36.88852338144798, 37.27703258372799, 37.69390012194085, 38.06539432266115, 38.49360519524714, 38.88495023112044, 39.25077276465417, 39.62793863256104, 40.02211950202762, 40.38510620196807, 40.77928707143465, 41.10824376825568, 41.49391713694241, 41.86541133766271, 42.24541303916286, 42.59422057113689, 43.005416442163174, 43.325865638204355, 43.640643167058954, 43.961092363100136, 44.2475115560219, 44.522587414570516, 44.828857442645266, 45.20602331055213, 45.47826333550747, 45.8072200323285, 46.121997561183115, 46.36871508379888, 46.663641777500494, 46.890508464963276, 47.15991265632533, 47.40946601253439, 47.61081019765761, 47.83200521793381, 48.03334940305703, 48.280066925672806, 48.4473811076766, 48.69409863029237, 48.85857697870289, 49.04574199585968, 49.241414513796336, 49.39454952783371, 49.56469954343079, 49.743357059807735, 49.83693956838613, 49.941865411337666, 50.046791254289204, 50.22261293707285, 50.35306128236395, 50.47216629328191, 50.56574880186031, 50.66783881121856, 50.80395882369623, 50.89470549868133, 50.942914669767184, 51.042168845532146, 51.06485551427842, 51.186796358789664, 51.20097552675609, 51.243513030655365, 51.27187136658821, 51.34560304001361, 51.34560304001361, 51.32291637126733, 51.33709553923376, 51.303065536114346, 51.31440887048748, 51.308737203300915, 51.29455803533449, 51.24067719706208, 51.19246802597623, 51.16127385645011, 51.135751354110546, 51.06201968068513, 50.98261634007317, 50.89754133227462, 50.78694382213652, 50.65082380965885, 50.49485296202819, 50.285001276125115, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 44.28752396735968, "threshold_area100": 45.168661202960614, "threshold_area25": 32.74792275189292}, "selected_budget": 33, "selected": {"val": {"score": 68.33227914821842, "calls": 32.99599409656336, "rescued": 1881, "harmed": 134}, "test": {"score": 44.366616566939854, "calls": 40.3794345347815, "rescued": 6551, "harmed": 317}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.2425622940063477, "operator": "ge"}, {"budget": 0.02, "threshold": 1.9226473569869995, "operator": "ge"}, {"budget": 0.03, "threshold": 1.704145073890686, "operator": "ge"}, {"budget": 0.04, "threshold": 1.5772168636322021, "operator": "ge"}, {"budget": 0.05, "threshold": 1.4701694250106812, "operator": "ge"}, {"budget": 0.06, "threshold": 1.3792901039123535, "operator": "ge"}, {"budget": 0.07, "threshold": 1.3075963258743286, "operator": "ge"}, {"budget": 0.08, "threshold": 1.2416962385177612, "operator": "ge"}, {"budget": 0.09, "threshold": 1.180537223815918, "operator": "ge"}, {"budget": 0.1, "threshold": 1.1317799091339111, "operator": "ge"}, {"budget": 0.11, "threshold": 1.0813391208648682, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0405566692352295, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0048435926437378, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9662616848945618, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9350687265396118, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9049381017684937, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8795914649963379, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8546469211578369, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8300495147705078, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8047487735748291, "operator": "ge"}, {"budget": 0.21, "threshold": 0.7878674268722534, "operator": "ge"}, {"budget": 0.22, "threshold": 0.7617114186286926, "operator": "ge"}, {"budget": 0.23, "threshold": 0.7365957498550415, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7185842990875244, "operator": "ge"}, {"budget": 0.25, "threshold": 0.697291374206543, "operator": "ge"}, {"budget": 0.26, "threshold": 0.6747111082077026, "operator": "ge"}, {"budget": 0.27, "threshold": 0.6565794944763184, "operator": "ge"}, {"budget": 0.28, "threshold": 0.636346161365509, "operator": "ge"}, {"budget": 0.29, "threshold": 0.6167683601379395, "operator": "ge"}, {"budget": 0.3, "threshold": 0.5977715253829956, "operator": "ge"}, {"budget": 0.31, "threshold": 0.5820832252502441, "operator": "ge"}, {"budget": 0.32, "threshold": 0.5654611587524414, "operator": "gt"}, {"budget": 0.33, "threshold": 0.5484366416931152, "operator": "ge"}, {"budget": 0.34, "threshold": 0.531172513961792, "operator": "ge"}, {"budget": 0.35, "threshold": 0.5164363980293274, "operator": "ge"}, {"budget": 0.36, "threshold": 0.5009828805923462, "operator": "ge"}, {"budget": 0.37, "threshold": 0.4843813180923462, "operator": "ge"}, {"budget": 0.38, "threshold": 0.468316912651062, "operator": "ge"}, {"budget": 0.39, "threshold": 0.45495593547821045, "operator": "ge"}, {"budget": 0.4, "threshold": 0.4403691291809082, "operator": "ge"}, {"budget": 0.41, "threshold": 0.4246457815170288, "operator": "ge"}, {"budget": 0.42, "threshold": 0.4112623929977417, "operator": "ge"}, {"budget": 0.43, "threshold": 0.39685940742492676, "operator": "ge"}, {"budget": 0.44, "threshold": 0.38440561294555664, "operator": "ge"}, {"budget": 0.45, "threshold": 0.36856722831726074, "operator": "ge"}, {"budget": 0.46, "threshold": 0.35587596893310547, "operator": "gt"}, {"budget": 0.47, "threshold": 0.3427114486694336, "operator": "ge"}, {"budget": 0.48, "threshold": 0.3300357758998871, "operator": "ge"}, {"budget": 0.49, "threshold": 0.3183295726776123, "operator": "ge"}, {"budget": 0.5, "threshold": 0.304440975189209, "operator": "ge"}, {"budget": 0.51, "threshold": 0.29109513759613037, "operator": "ge"}, {"budget": 0.52, "threshold": 0.27927863597869873, "operator": "ge"}, {"budget": 0.53, "threshold": 0.26688075065612793, "operator": "ge"}, {"budget": 0.54, "threshold": 0.2553315758705139, "operator": "ge"}, {"budget": 0.55, "threshold": 0.24129366874694824, "operator": "ge"}, {"budget": 0.56, "threshold": 0.2299342155456543, "operator": "ge"}, {"budget": 0.57, "threshold": 0.21552610397338867, "operator": "ge"}, {"budget": 0.58, "threshold": 0.2029869556427002, "operator": "ge"}, {"budget": 0.59, "threshold": 0.1878414750099182, "operator": "ge"}, {"budget": 0.6, "threshold": 0.17511868476867676, "operator": "ge"}, {"budget": 0.61, "threshold": 0.16307854652404785, "operator": "ge"}, {"budget": 0.62, "threshold": 0.1493011713027954, "operator": "ge"}, {"budget": 0.63, "threshold": 0.13471919298171997, "operator": "ge"}, {"budget": 0.64, "threshold": 0.12061834335327148, "operator": "ge"}, {"budget": 0.65, "threshold": 0.10522723197937012, "operator": "ge"}, {"budget": 0.66, "threshold": 0.09054791927337646, "operator": "ge"}, {"budget": 0.67, "threshold": 0.07173609733581543, "operator": "ge"}, {"budget": 0.68, "threshold": 0.05317544937133789, "operator": "ge"}, {"budget": 0.69, "threshold": 0.01947861909866333, "operator": "ge"}, {"budget": 0.7, "threshold": -0.0191805362701416, "operator": "ge"}, {"budget": 0.71, "threshold": -0.059640586376190186, "operator": "ge"}, {"budget": 0.72, "threshold": -0.11909380555152893, "operator": "ge"}, {"budget": 0.73, "threshold": -0.1761123538017273, "operator": "ge"}, {"budget": 0.74, "threshold": -0.23174616694450378, "operator": "ge"}, {"budget": 0.75, "threshold": -0.2980295419692993, "operator": "ge"}, {"budget": 0.76, "threshold": -0.3642961382865906, "operator": "ge"}, {"budget": 0.77, "threshold": -0.4330380856990814, "operator": "ge"}, {"budget": 0.78, "threshold": -0.514572024345398, "operator": "ge"}, {"budget": 0.79, "threshold": -0.6008214950561523, "operator": "ge"}, {"budget": 0.8, "threshold": -0.6914976835250854, "operator": "ge"}, {"budget": 0.81, "threshold": -0.7957892417907715, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8967266082763672, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9923539161682129, "operator": "ge"}, {"budget": 0.84, "threshold": -1.088226556777954, "operator": "ge"}, {"budget": 0.85, "threshold": -1.1745885610580444, "operator": "ge"}, {"budget": 0.86, "threshold": -1.3008265495300293, "operator": "ge"}, {"budget": 0.87, "threshold": -1.4126536846160889, "operator": "ge"}, {"budget": 0.88, "threshold": -1.5338093042373657, "operator": "ge"}, {"budget": 0.89, "threshold": -1.685186505317688, "operator": "ge"}, {"budget": 0.9, "threshold": -1.8082163333892822, "operator": "ge"}, {"budget": 0.91, "threshold": -1.9807631969451904, "operator": "ge"}, {"budget": 0.92, "threshold": -2.172987461090088, "operator": "ge"}, {"budget": 0.93, "threshold": -2.3942630290985107, "operator": "ge"}, {"budget": 0.94, "threshold": -2.6256258487701416, "operator": "ge"}, {"budget": 0.95, "threshold": -2.9702649116516113, "operator": "ge"}, {"budget": 0.96, "threshold": -3.355416774749756, "operator": "ge"}, {"budget": 0.97, "threshold": -3.798835515975952, "operator": "ge"}, {"budget": 0.98, "threshold": -4.3395586013793945, "operator": "ge"}, {"budget": 0.99, "threshold": -5.327574729919434, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.609740670461726, 57.07358212102045, 57.55850727387729, 58.14885093822475, 58.605664488017425, 58.971115327851564, 59.47712418300654, 59.91988193126713, 60.36263967952772, 60.742146320893944, 61.06542975613184, 61.3324899852414, 61.48710380209431, 61.76821983273596, 62.12664277180405, 62.428842504743834, 62.702930634619435, 62.941879260664834, 63.28624639820085, 63.50411132194813, 63.66575303956707, 63.7571157495256, 63.8906458640804, 64.00309227633704, 64.24906880314849, 64.42476632229952, 64.57235223838639, 64.67074284911097, 64.7972450628997, 64.80427296366575, 64.93780307822054, 64.98699838358282, 65.10647269660552, 65.16972380349989, 65.17675170426594, 65.16972380349989, 65.22594700962821, 65.28217021575655, 65.359477124183, 65.33839342188487, 65.359477124183, 65.37353292571508, 65.4438119333755, 65.53517464333403, 65.54220254410006, 65.5632862463982, 65.61950945252653, 65.63356525405861, 65.60545365099443, 65.59139784946237, 65.59139784946237, 65.54923044486611, 65.52111884180195, 65.4789514372057, 65.43678403260945, 65.41570033031134, 65.36650502494903, 65.36650502494903, 65.30325391805466, 65.29622601728863, 65.24000281116031, 65.23297491039426, 65.190807505798, 65.14864010120176, 65.0853889943074, 65.0361936889451, 64.97997048281677, 64.846440368262, 64.7972450628997, 64.71993815447325, 64.64965914681284, 64.6074917422166, 64.55126853608827, 64.4880174291939, 64.44585002459765, 64.41071052076744, 64.43882212383161, 64.4036826200014, 64.41071052076744, 64.44585002459765, 64.41773842153349, 64.4036826200014, 64.4036826200014, 64.42476632229952, 64.41773842153349, 64.44585002459765, 64.44585002459765, 64.43882212383161, 64.41773842153349, 64.42476632229952, 64.39665471923536, 64.3685431161712, 64.32637571157494, 64.33340361234099, 64.33340361234099, 64.29123620774475, 64.23501300161641, 64.19284559702017, 64.17878979548807, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99100428701947, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.990582612973505, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 92, 174, 259, 351, 427, 491, 569, 642, 717, 787, 846, 901, 942, 1001, 1065, 1118, 1174, 1229, 1285, 1337, 1376, 1413, 1449, 1486, 1537, 1574, 1613, 1645, 1680, 1699, 1732, 1765, 1796, 1825, 1851, 1869, 1888, 1912, 1940, 1958, 1978, 1998, 2023, 2057, 2076, 2093, 2113, 2132, 2152, 2166, 2185, 2197, 2207, 2221, 2235, 2255, 2271, 2286, 2300, 2314, 2327, 2341, 2353, 2364, 2372, 2384, 2396, 2401, 2409, 2417, 2423, 2436, 2442, 2453, 2460, 2466, 2476, 2488, 2498, 2508, 2514, 2520, 2532, 2538, 2541, 2552, 2558, 2563, 2570, 2576, 2579, 2580, 2581, 2587, 2588, 2589, 2589, 2589, 2589, 2589], "harmed": [0, 13, 29, 45, 53, 64, 76, 82, 92, 104, 120, 133, 150, 169, 188, 201, 211, 228, 249, 256, 277, 293, 317, 334, 355, 371, 383, 401, 419, 436, 454, 468, 494, 508, 528, 553, 572, 583, 599, 616, 637, 654, 672, 687, 708, 726, 740, 752, 769, 793, 809, 828, 846, 860, 880, 900, 923, 946, 961, 984, 999, 1020, 1035, 1053, 1070, 1087, 1106, 1126, 1150, 1165, 1184, 1200, 1219, 1233, 1253, 1266, 1277, 1283, 1300, 1309, 1314, 1324, 1332, 1344, 1347, 1351, 1358, 1364, 1370, 1380, 1385, 1392, 1397, 1404, 1409, 1410, 1417, 1425, 1431, 1433, 1441], "exact_score": [56.05453650994448, 56.609740670461726, 57.07358212102045, 57.55850727387729, 58.14885093822475, 58.605664488017425, 58.971115327851564, 59.47712418300654, 59.91988193126713, 60.36263967952772, 60.742146320893944, 61.06542975613184, 61.3324899852414, 61.48710380209431, 61.76821983273596, 62.12664277180405, 62.428842504743834, 62.702930634619435, 62.941879260664834, 63.28624639820085, 63.50411132194813, 63.66575303956707, 63.7571157495256, 63.8906458640804, 64.00309227633704, 64.24906880314849, 64.42476632229952, 64.57235223838639, 64.67074284911097, 64.7972450628997, 64.80427296366575, 64.93780307822054, 64.98699838358282, 65.10647269660552, 65.16972380349989, 65.17675170426594, 65.16972380349989, 65.22594700962821, 65.28217021575655, 65.359477124183, 65.33839342188487, 65.359477124183, 65.37353292571508, 65.4438119333755, 65.53517464333403, 65.54220254410006, 65.5632862463982, 65.61950945252653, 65.63356525405861, 65.60545365099443, 65.59139784946237, 65.59139784946237, 65.54923044486611, 65.52111884180195, 65.4789514372057, 65.43678403260945, 65.41570033031134, 65.36650502494903, 65.36650502494903, 65.30325391805466, 65.29622601728863, 65.24000281116031, 65.23297491039426, 65.190807505798, 65.14864010120176, 65.0853889943074, 65.0361936889451, 64.97997048281677, 64.846440368262, 64.7972450628997, 64.71993815447325, 64.64965914681284, 64.6074917422166, 64.55126853608827, 64.4880174291939, 64.44585002459765, 64.41071052076744, 64.43882212383161, 64.4036826200014, 64.41071052076744, 64.44585002459765, 64.41773842153349, 64.4036826200014, 64.4036826200014, 64.42476632229952, 64.41773842153349, 64.44585002459765, 64.44585002459765, 64.43882212383161, 64.41773842153349, 64.42476632229952, 64.39665471923536, 64.3685431161712, 64.32637571157494, 64.33340361234099, 64.33340361234099, 64.29123620774475, 64.23501300161641, 64.19284559702017, 64.17878979548807, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 63.924362648945355, "threshold_area100": 63.924098671726746, "threshold_area25": 61.026073511842014}, "test": {"score": [26.688029946402747, 28.86595014604543, 30.35192694892664, 31.81238124946828, 32.80775884071123, 33.67552392025636, 34.63687150837989, 35.43941241527948, 36.27598332529847, 37.06434506423163, 37.7024076227207, 38.436888523381455, 39.14868275529592, 39.67898363724017, 40.29435952698296, 40.904063749539176, 41.45137963304313, 41.899441340782126, 42.27377137509571, 42.69631058049514, 43.17840229135355, 43.56407566004027, 44.060346538865105, 44.43467657317869, 44.7125882653206, 45.11811246916031, 45.47826333550747, 45.74766752686953, 46.07946005728384, 46.36871508379888, 46.714686782179626, 47.012449309474526, 47.318719337549275, 47.588123528911325, 47.851856053086806, 48.044692737430175, 48.285738592859374, 48.52678444828857, 48.76783030371778, 48.95499532087457, 49.1364886708448, 49.352012023934435, 49.56753537702407, 49.72917789184131, 49.88514873947197, 50.00708958398321, 50.086492924595184, 50.20843376910643, 50.338882114397535, 50.48918129484162, 50.62530130731929, 50.6990329807447, 50.76425715339023, 50.86918299634178, 50.945750503360465, 51.0676913478717, 51.07903468224485, 51.15276635567025, 51.15843802285682, 51.20664719394266, 51.25769219862178, 51.3285880384539, 51.37112554235318, 51.39948387828602, 51.39664804469274, 51.419334713439014, 51.40515554547259, 51.37963304313303, 51.368289708759896, 51.351274707200176, 51.311573036894195, 51.28321470096135, 51.26336386580835, 51.243513030655365, 51.23216969628223, 51.25769219862178, 51.26052803221507, 51.22649802909566, 51.18396052519638, 51.16127385645011, 51.135751354110546, 51.11873635255083, 51.11022885177098, 51.09888551739784, 51.0676913478717, 51.042168845532146, 50.99963134163288, 50.96560133851346, 50.90888466664776, 50.85783966196864, 50.79828715650966, 50.718883815897684, 50.62530130731929, 50.54589796670732, 50.46082295890877, 50.35873294955052, 50.2509712730057, 50.18858293395344, 50.12335876130789, 50.07798542381533, 50.00708958398321], "calls": [0.0, 2.7961319229787596, 4.809573774210929, 6.91943396761478, 8.385559935342995, 9.789297564018943, 11.283781867680004, 12.62513115730369, 14.164988798457307, 15.64245810055866, 16.924254884723364, 18.430082522757566, 19.7997901483141, 21.024870260613106, 22.454130391628617, 23.812494682812012, 25.074440631823723, 26.14638573008536, 27.15877832288801, 28.304455094575047, 29.521027706094205, 30.385956952046055, 31.786858747128722, 33.04029719536057, 34.004480617077384, 35.21254572781669, 36.400760003403, 37.42449593057879, 38.717636049116635, 39.89734282392309, 41.13376627059524, 42.09227802512549, 43.12452145308113, 44.27586989195474, 45.38184499333579, 46.3658792502056, 47.44066018206052, 48.53245611547514, 49.77171539574058, 50.718883815897684, 51.7567989110399, 52.76919150384255, 53.781584096645204, 54.81382752460085, 55.766667611944534, 56.88115021410544, 57.83399030144911, 58.93996540283016, 59.881462155800705, 60.83997391033095, 62.00550151717097, 63.114312452145306, 64.16073504806738, 65.12775430337749, 66.10044522587415, 67.27164449990075, 68.23582792161757, 69.48075886906956, 70.58106230326405, 71.91390409210787, 73.06525253098148, 74.06063012222442, 75.22048606187789, 76.37750616793807, 77.55721294274453, 78.86453222924878, 80.14916484700679, 81.50752913819016, 82.62484757394436, 83.8470918526501, 84.60993108924369, 85.19978447664691, 85.9824745483935, 86.67441794515499, 87.3011371692709, 87.92785639338683, 88.4921872784505, 89.09905566741344, 89.74278989308907, 90.36383745001844, 90.98488500694779, 91.71369424042197, 92.22130845361994, 92.71757933244477, 93.17982020815019, 93.59952357995633, 94.18937696735955, 94.67997617899782, 95.1478887218898, 95.77744377959901, 96.23401298811785, 96.73311970053598, 97.26909224966678, 97.77954229645805, 98.18223066670447, 98.66999404474946, 99.08686158296231, 99.43283328134305, 99.65969996880582, 99.8497008195559, 100.0], "rescued": [0, 816, 1369, 1920, 2290, 2617, 2978, 3281, 3602, 3901, 4159, 4443, 4711, 4925, 5173, 5411, 5622, 5799, 5960, 6138, 6335, 6481, 6692, 6853, 6976, 7140, 7291, 7420, 7570, 7700, 7858, 7985, 8113, 8228, 8347, 8440, 8553, 8665, 8776, 8861, 8948, 9044, 9135, 9212, 9287, 9366, 9423, 9496, 9563, 9634, 9705, 9759, 9813, 9870, 9915, 9984, 10016, 10069, 10105, 10158, 10198, 10241, 10286, 10318, 10348, 10380, 10406, 10428, 10441, 10463, 10477, 10492, 10513, 10524, 10538, 10566, 10584, 10593, 10604, 10616, 10626, 10642, 10654, 10664, 10672, 10678, 10688, 10691, 10694, 10698, 10702, 10705, 10710, 10714, 10715, 10719, 10722, 10723, 10723, 10723, 10723], "harmed": [0, 48, 77, 113, 132, 153, 175, 195, 221, 242, 275, 300, 317, 344, 375, 398, 416, 435, 464, 493, 520, 530, 566, 595, 620, 641, 665, 699, 732, 760, 796, 818, 838, 858, 884, 909, 937, 964, 990, 1009, 1032, 1052, 1067, 1087, 1107, 1143, 1172, 1202, 1223, 1241, 1264, 1292, 1323, 1343, 1361, 1387, 1415, 1442, 1476, 1512, 1534, 1552, 1582, 1604, 1635, 1659, 1690, 1721, 1738, 1766, 1794, 1819, 1847, 1865, 1883, 1902, 1919, 1940, 1966, 1986, 2005, 2027, 2042, 2056, 2075, 2090, 2115, 2130, 2153, 2175, 2200, 2231, 2269, 2301, 2332, 2372, 2413, 2436, 2459, 2475, 2500], "exact_score": [26.688029946402747, 27.541615857981455, 28.29311176020191, 29.01908516008281, 29.745058559963706, 30.47953946062445, 31.157303689419507, 31.860590420554125, 32.53551881575589, 33.17641720783825, 33.80313643195417, 34.452542324816385, 35.03955987862633, 35.62374159884298, 36.188072483906645, 36.70135836429119, 37.25434591498171, 37.73643762584012, 38.25823100700451, 38.737486884269636, 39.23375776309446, 39.670476136460316, 40.10152284263959, 40.55242038397187, 41.00048209171086, 41.42302129711028, 41.82854550095001, 42.22272637041659, 42.59422057113689, 42.95720727107734, 43.42228398037603, 43.756912344383636, 44.12273487791737, 44.426169072398835, 44.70975243172731, 45.05005246292148, 45.3421433230298, 45.64274168391799, 45.87811587216063, 46.14468422992939, 46.39707341973173, 46.7005076142132, 46.98692680713496, 47.281853500836576, 47.52857102345235, 47.71857187420242, 47.97663273119134, 48.229021920993674, 48.410515270963906, 48.645889459206536, 48.80469614043048, 49.00320449196042, 49.190369509117204, 49.40022119502028, 49.584550378583785, 49.76037206136744, 49.89082040665854, 50.0099254175765, 50.106343759748185, 50.214105436292996, 50.35589711595724, 50.52604713155432, 50.62530130731929, 50.704704647931266, 50.75574965261038, 50.86067549556192, 50.92873550180076, 51.047840512718714, 51.06201968068513, 51.1442588548904, 51.17545302441653, 51.186796358789664, 51.20948302753594, 51.252020531435214, 51.3285880384539, 51.351274707200176, 51.39381221109945, 51.39948387828602, 51.40515554547259, 51.41082721265916, 51.40515554547259, 51.39948387828602, 51.376797209539745, 51.354110540793464, 51.351274707200176, 51.28605053455464, 51.26336386580835, 51.243513030655365, 51.26052803221507, 51.237841363468796, 51.18396052519638, 51.135751354110546, 51.113064685364265, 51.06485551427842, 51.01664634319258, 50.93157133539405, 50.832317159629085, 50.679182145591696, 50.50052462921476, 50.282165442531834, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.50945722356528, "threshold_area100": 47.20495987295466, "threshold_area25": 38.27269375833026}, "selected_budget": 48, "selected": {"val": {"score": 65.63356525405861, "calls": 47.99353433129524, "rescued": 2132, "harmed": 769}, "test": {"score": 50.338882114397535, "calls": 59.881462155800705, "rescued": 9563, "harmed": 1223}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 65.528146742568, "calls": 46.9955724225174}, "test": {"score": 50.09783625896833, "calls": 58.08070782406488}}, "old_normalized_interior": {"val": {"score": 65.53517464333403, "calls": 48.99852414083913}, "test": {"score": 49.81708873323314, "calls": 58.3926495193262}}, "old_normalized_endpoints": {"val": {"score": 65.53517464333403, "calls": 48.99852414083913}, "test": {"score": 49.81708873323314, "calls": 58.3926495193262}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 56.806521891910876, 57.48120036545083, 58.02937662520205, 58.577552884953256, 59.16086864853468, 59.62471009909339, 60.03232834352379, 60.411834984890014, 60.791341626256234, 61.149764565324325, 61.43790849673202, 61.775247733502006, 62.09853116873989, 62.37964719938154, 62.6467074284911, 62.84348864994026, 63.01918616909129, 63.30030219973294, 63.41977651275563, 63.69386464263124, 63.89767376484643, 64.08039918476351, 64.24204090238246, 64.23501300161641, 64.38962681846931, 64.41773842153349, 64.60046384145055, 64.6426312460468, 64.75507765830346, 64.86049616979408, 64.9518588797526, 64.98699838358282, 65.04322158971115, 65.0853889943074, 65.09944479583947, 65.1556680019678, 65.24000281116031, 65.190807505798, 65.3243376203528, 65.359477124183, 65.37353292571508, 65.37353292571508, 65.4438119333755, 65.43678403260945, 65.4438119333755, 65.50003513950382, 65.5140909410359, 65.50706304026987, 65.53517464333403, 65.5140909410359, 65.46489563567361, 65.45786773490758, 65.40867242954529, 65.38056082648113, 65.37353292571508, 65.30325391805466, 65.25405861269238, 65.1556680019678, 65.10647269660552, 65.11350059737156, 65.0361936889451, 64.91671937592241, 64.81130086443179, 64.81832876519783, 64.78318926136762, 64.76210555906951, 64.71291025370721, 64.58640803991848, 64.54424063532223, 64.46693372689577, 64.43882212383161, 64.3193478108089, 64.27015250544662, 64.19987349778621, 64.15770609318996, 64.05931548246538, 64.03120387940122, 64.00309227633704, 63.89767376484643, 63.90470166561247, 63.91875746714456, 63.95389697097477, 63.96092487174081, 64.03823178016725, 64.08039918476351, 64.164733993956, 64.17176189472204, 64.23501300161641, 64.22798510085038, 64.22798510085038, 64.25609670391454, 64.29826410851078, 64.3193478108089, 64.3193478108089, 64.3193478108089, 64.27015250544662, 64.24906880314849, 64.22798510085038, 64.17878979548807, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.98945814885094, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.98749033663645, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 56.806521891910876, 57.48120036545083, 58.02937662520205, 58.577552884953256, 59.16086864853468, 59.62471009909339, 60.03232834352379, 60.411834984890014, 60.791341626256234, 61.149764565324325, 61.43790849673202, 61.775247733502006, 62.09853116873989, 62.37964719938154, 62.6467074284911, 62.84348864994026, 63.01918616909129, 63.30030219973294, 63.41977651275563, 63.69386464263124, 63.89767376484643, 64.08039918476351, 64.24204090238246, 64.23501300161641, 64.38962681846931, 64.41773842153349, 64.60046384145055, 64.6426312460468, 64.75507765830346, 64.86049616979408, 64.9518588797526, 64.98699838358282, 65.04322158971115, 65.0853889943074, 65.09944479583947, 65.1556680019678, 65.24000281116031, 65.190807505798, 65.3243376203528, 65.359477124183, 65.37353292571508, 65.37353292571508, 65.4438119333755, 65.43678403260945, 65.4438119333755, 65.50003513950382, 65.5140909410359, 65.50706304026987, 65.53517464333403, 65.5140909410359, 65.46489563567361, 65.45786773490758, 65.40867242954529, 65.38056082648113, 65.37353292571508, 65.30325391805466, 65.25405861269238, 65.1556680019678, 65.10647269660552, 65.11350059737156, 65.0361936889451, 64.91671937592241, 64.81130086443179, 64.81832876519783, 64.78318926136762, 64.76210555906951, 64.71291025370721, 64.58640803991848, 64.54424063532223, 64.46693372689577, 64.43882212383161, 64.3193478108089, 64.27015250544662, 64.19987349778621, 64.15770609318996, 64.05931548246538, 64.03120387940122, 64.00309227633704, 63.89767376484643, 63.90470166561247, 63.91875746714456, 63.95389697097477, 63.96092487174081, 64.03823178016725, 64.08039918476351, 64.164733993956, 64.17176189472204, 64.23501300161641, 64.22798510085038, 64.22798510085038, 64.25609670391454, 64.29826410851078, 64.3193478108089, 64.3193478108089, 64.3193478108089, 64.27015250544662, 64.24906880314849, 64.22798510085038, 64.17878979548807, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 61.37465738983765, "exact_area100": 63.90150752447437}, "test": {"score": [26.688029946402747, 28.259081757082495, 29.410430195956106, 30.607151972322267, 31.551484558886088, 32.54402631653575, 33.46567223435329, 34.285228142812585, 35.118963219238296, 35.751354110540795, 36.619119190085925, 37.18912174233616, 37.87822930550435, 38.411366021041886, 38.87077106315402, 39.42092278025125, 40.033462836400766, 40.45033037461362, 40.96078042140488, 41.533618807248395, 42.04123302044636, 42.500638062558494, 42.95437143748405, 43.40810481240961, 43.799449848282904, 44.25601905680175, 44.6190057567422, 45.06990329807447, 45.41303916286192, 45.71363752375011, 46.02557921901143, 46.394237586138445, 46.6806567790602, 46.947225136828976, 47.22797266256416, 47.47469018517994, 47.71006437342257, 47.959617729631624, 48.16663358194141, 48.350962765504924, 48.455888608456455, 48.614695289680405, 48.827382809176754, 48.980517823214136, 49.125145336471654, 49.255593681762754, 49.37469869268072, 49.52499787312481, 49.68096872075547, 49.81708873323314, 49.93902957774438, 50.07798542381533, 50.18574710036015, 50.24813543941242, 50.321867112837815, 50.44097212375578, 50.53171879874089, 50.58843547060658, 50.66783881121856, 50.72455548308425, 50.772764654170096, 50.86351132915521, 50.91739216742761, 50.9769446728866, 50.98828800725974, 51.047840512718714, 51.073363015058284, 51.06201968068513, 51.073363015058284, 51.073363015058284, 51.08187051583813, 51.113064685364265, 51.09604968380456, 51.15276635567025, 51.12440801973741, 51.121572186144114, 51.08470634943142, 51.0676913478717, 51.073363015058284, 51.09037801661799, 51.11022885177098, 51.113064685364265, 51.11873635255083, 51.10172135099113, 51.09321385021127, 51.08470634943142, 51.07903468224485, 51.03649717834558, 51.005303008819446, 50.93157133539405, 50.89470549868133, 50.832317159629085, 50.73022715027082, 50.6593313104387, 50.54589796670732, 50.384255451890084, 50.29067294331169, 50.216941269886284, 50.151717097240734, 50.086492924595184, 50.00708958398321], "calls": [0.0, 1.8773218387545019, 3.3349403057028613, 4.928678785128889, 6.301222244278707, 7.812721549499475, 9.233474179735133, 10.54646513342597, 11.99841193318776, 13.144088704874799, 14.64424467572243, 15.789921447409466, 17.2333607463914, 18.28261917590676, 19.320534271048974, 20.556957717721126, 21.960695346397074, 22.930550435300457, 24.16413804837932, 25.474293168476876, 26.71071661514902, 28.001020900093582, 29.124011003034344, 30.428494455945327, 31.69327623855032, 33.08850636644642, 34.27104897484616, 35.60956243087656, 36.690015029918044, 37.849870969571505, 39.015398576411535, 40.21495618637098, 41.35496129087145, 42.54884723364433, 43.62079233190597, 44.71542409891388, 45.6966225221904, 46.657970110313926, 47.556929359385194, 48.56081445140799, 49.388877860647135, 50.23962793863256, 51.49023055327113, 52.445906474208094, 53.38173155999206, 54.4508408246604, 55.45472591668321, 56.35652099934776, 57.31219692028472, 58.3926495193262, 59.31996710433032, 60.255792190114285, 61.23415477979752, 62.27206987493974, 63.15968578963786, 64.40745257068315, 65.43686016504552, 66.36134191645634, 67.3000028358336, 68.02881206930778, 68.89374131525962, 69.82105890026374, 70.77673482120069, 71.91106825851459, 72.76181833650001, 73.81674843320194, 74.72137934945977, 75.56645776025863, 76.81138870771062, 77.84646796925956, 79.17363809091682, 80.43274820633525, 81.45648413351104, 82.49723506224655, 83.696792672206, 84.8112752743669, 85.82933953435612, 86.8559112951252, 87.59039219578595, 88.43547060658481, 89.10189150100672, 89.78816323058163, 90.42906162266398, 90.95936250460824, 91.54921589201146, 92.16459178175424, 92.75728100275076, 93.3641493917137, 93.95116694552364, 94.61758783994554, 95.15072455548308, 95.74341377647958, 96.40416300371494, 97.02804639423759, 97.76819896208491, 98.38641068542098, 98.88835323143238, 99.34492243995122, 99.65402830161926, 99.85253665314919, 100.0], "exact_score": [26.688029946402747, 27.530272523608318, 28.36400760003403, 29.118339335847775, 29.94640274508692, 30.655361143408104, 31.35014037376287, 31.985367098658653, 32.65462382667385, 33.30119388594277, 33.95627144599155, 34.54328899980149, 35.118963219238296, 35.66060743555568, 36.28449082607833, 36.80912004083601, 37.274196750134706, 37.744945126619974, 38.25823100700451, 38.73181521708307, 39.143011088109354, 39.633610299747616, 40.05898533874033, 40.4900320449196, 40.91257125031903, 41.31242378697218, 41.766157161897745, 42.13481552902476, 42.500638062558494, 42.900490599211636, 43.266313132745374, 43.59243399597312, 43.932734027167285, 44.233332388055466, 44.533930748943654, 44.85154411139155, 45.191844142585715, 45.523636673000034, 45.77319002920909, 46.01423588463829, 46.34035674786604, 46.6069251056348, 46.825284292317725, 47.05215097978051, 47.327226838329125, 47.53991435782549, 47.78946771403454, 48.047528571023456, 48.26021609051981, 48.419022771743755, 48.56932195218785, 48.73096446700508, 48.915293650568586, 49.06559283101268, 49.20171284349035, 49.30096701925532, 49.47395286844568, 49.64126705044948, 49.74619289340102, 49.89365624025183, 50.05529875506905, 50.160224598020584, 50.2311204378527, 50.30768794487139, 50.41544962141622, 50.50052462921476, 50.5685846354536, 50.65082380965885, 50.7160479823044, 50.79545132291637, 50.877690497121634, 50.93440716898733, 50.97410883929331, 51.01664634319258, 51.056348013498564, 51.073363015058284, 51.07903468224485, 51.05351217990528, 51.0676913478717, 51.09037801661799, 51.10172135099113, 51.10455718458441, 51.138587187703834, 51.14993052207697, 51.107393018177696, 51.113064685364265, 51.076198848651565, 51.05918384709185, 51.09321385021127, 51.09888551739784, 51.113064685364265, 51.0875421830247, 51.07903468224485, 51.06201968068513, 51.008138842412734, 50.89470549868133, 50.7897796557298, 50.6593313104387, 50.49768879562148, 50.267986274565416, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 37.21946516178431, "exact_area100": 45.363070975578204}}, "old_normalized_endpoint_budget": 49}, {"seed": 2029, "selection": {"best_epoch": 9, "epochs": 17, "alpha": 0.0, "validation_area100": 0.643211614626195, "training_seconds": 115.94775290496182, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8979723453521729, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8489285707473755, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8209637999534607, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7933096885681152, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7614999413490295, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7336856722831726, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7092164754867554, "operator": "ge"}, {"budget": 0.08, "threshold": 0.6867752075195312, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6688904762268066, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6506221890449524, "operator": "ge"}, {"budget": 0.11, "threshold": 0.6320599317550659, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6147139668464661, "operator": "gt"}, {"budget": 0.13, "threshold": 0.5964675545692444, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5799756050109863, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5660858154296875, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5516048073768616, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5367701649665833, "operator": "ge"}, {"budget": 0.18, "threshold": 0.521061360836029, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5081828832626343, "operator": "ge"}, {"budget": 0.2, "threshold": 0.4955650866031647, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4830303192138672, "operator": "ge"}, {"budget": 0.22, "threshold": 0.47118788957595825, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4563196301460266, "operator": "ge"}, {"budget": 0.24, "threshold": 0.4418787658214569, "operator": "ge"}, {"budget": 0.25, "threshold": 0.42668646574020386, "operator": "ge"}, {"budget": 0.26, "threshold": 0.4134938716888428, "operator": "ge"}, {"budget": 0.27, "threshold": 0.4006917476654053, "operator": "ge"}, {"budget": 0.28, "threshold": 0.38952919840812683, "operator": "ge"}, {"budget": 0.29, "threshold": 0.380513995885849, "operator": "ge"}, {"budget": 0.3, "threshold": 0.36805158853530884, "operator": "ge"}, {"budget": 0.31, "threshold": 0.3566833734512329, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3455820083618164, "operator": "ge"}, {"budget": 0.33, "threshold": 0.33480262756347656, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3235478103160858, "operator": "ge"}, {"budget": 0.35, "threshold": 0.31241950392723083, "operator": "ge"}, {"budget": 0.36, "threshold": 0.30317339301109314, "operator": "ge"}, {"budget": 0.37, "threshold": 0.29484260082244873, "operator": "ge"}, {"budget": 0.38, "threshold": 0.28449615836143494, "operator": "ge"}, {"budget": 0.39, "threshold": 0.27535223960876465, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2668619155883789, "operator": "ge"}, {"budget": 0.41, "threshold": 0.2559983730316162, "operator": "ge"}, {"budget": 0.42, "threshold": 0.24649371206760406, "operator": "ge"}, {"budget": 0.43, "threshold": 0.23668867349624634, "operator": "ge"}, {"budget": 0.44, "threshold": 0.2275482714176178, "operator": "ge"}, {"budget": 0.45, "threshold": 0.21694645285606384, "operator": "ge"}, {"budget": 0.46, "threshold": 0.20826171338558197, "operator": "ge"}, {"budget": 0.47, "threshold": 0.19819559156894684, "operator": "ge"}, {"budget": 0.48, "threshold": 0.1883612424135208, "operator": "ge"}, {"budget": 0.49, "threshold": 0.1798878014087677, "operator": "ge"}, {"budget": 0.5, "threshold": 0.17083147168159485, "operator": "ge"}, {"budget": 0.51, "threshold": 0.1600838601589203, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1502004861831665, "operator": "ge"}, {"budget": 0.53, "threshold": 0.14141960442066193, "operator": "ge"}, {"budget": 0.54, "threshold": 0.13268065452575684, "operator": "ge"}, {"budget": 0.55, "threshold": 0.12383835762739182, "operator": "ge"}, {"budget": 0.56, "threshold": 0.11528711766004562, "operator": "ge"}, {"budget": 0.57, "threshold": 0.10678926110267639, "operator": "ge"}, {"budget": 0.58, "threshold": 0.09796129912137985, "operator": "ge"}, {"budget": 0.59, "threshold": 0.0891689881682396, "operator": "ge"}, {"budget": 0.6, "threshold": 0.08053667843341827, "operator": "ge"}, {"budget": 0.61, "threshold": 0.07275599986314774, "operator": "ge"}, {"budget": 0.62, "threshold": 0.06390669196844101, "operator": "ge"}, {"budget": 0.63, "threshold": 0.055729374289512634, "operator": "ge"}, {"budget": 0.64, "threshold": 0.04723424091935158, "operator": "ge"}, {"budget": 0.65, "threshold": 0.039548151195049286, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03283008933067322, "operator": "ge"}, {"budget": 0.67, "threshold": 0.02518165111541748, "operator": "ge"}, {"budget": 0.68, "threshold": 0.01788465678691864, "operator": "ge"}, {"budget": 0.69, "threshold": 0.01023905724287033, "operator": "ge"}, {"budget": 0.7, "threshold": 0.003975010477006435, "operator": "ge"}, {"budget": 0.71, "threshold": 7.774162804707885e-05, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0005267522064968944, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0010306058684363961, "operator": "ge"}, {"budget": 0.74, "threshold": -0.0015592302661389112, "operator": "ge"}, {"budget": 0.75, "threshold": -0.00216431706212461, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0028698602691292763, "operator": "ge"}, {"budget": 0.77, "threshold": -0.003715729108080268, "operator": "ge"}, {"budget": 0.78, "threshold": -0.004754906054586172, "operator": "ge"}, {"budget": 0.79, "threshold": -0.005801151040941477, "operator": "ge"}, {"budget": 0.8, "threshold": -0.006982957944273949, "operator": "ge"}, {"budget": 0.81, "threshold": -0.008602754212915897, "operator": "ge"}, {"budget": 0.82, "threshold": -0.01027899980545044, "operator": "ge"}, {"budget": 0.83, "threshold": -0.012374636717140675, "operator": "ge"}, {"budget": 0.84, "threshold": -0.014616246335208416, "operator": "ge"}, {"budget": 0.85, "threshold": -0.017737602815032005, "operator": "ge"}, {"budget": 0.86, "threshold": -0.021741455420851707, "operator": "ge"}, {"budget": 0.87, "threshold": -0.026340527459979057, "operator": "ge"}, {"budget": 0.88, "threshold": -0.03142254427075386, "operator": "ge"}, {"budget": 0.89, "threshold": -0.03804653882980347, "operator": "ge"}, {"budget": 0.9, "threshold": -0.04566209763288498, "operator": "ge"}, {"budget": 0.91, "threshold": -0.0561404824256897, "operator": "ge"}, {"budget": 0.92, "threshold": -0.06917421519756317, "operator": "ge"}, {"budget": 0.93, "threshold": -0.0844009667634964, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10254351794719696, "operator": "ge"}, {"budget": 0.95, "threshold": -0.12605692446231842, "operator": "ge"}, {"budget": 0.96, "threshold": -0.15871846675872803, "operator": "ge"}, {"budget": 0.97, "threshold": -0.20698675513267517, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2801450192928314, "operator": "ge"}, {"budget": 0.99, "threshold": -0.40608489513397217, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.806521891910876, 57.38280975472626, 58.085599831330384, 58.71811090027408, 59.21006395389696, 59.61768219832736, 60.03935624428982, 60.42589078642209, 60.91784384004497, 61.24112727528287, 61.48710380209431, 61.80335933656617, 62.147726474102186, 62.43587040550986, 62.70995853538548, 63.05432567292149, 63.26516269590273, 63.48302761965, 63.67278094033312, 63.88361796331436, 64.09445498629559, 64.29123620774475, 64.47396162766181, 64.64965914681284, 64.7972450628997, 64.93077517745449, 65.06430529200927, 65.28919811652258, 65.4297561318434, 65.57734204793027, 65.60545365099443, 65.66870475788882, 65.79520697167756, 65.87251388010401, 65.9217091854663, 65.92873708623233, 65.91468128470026, 65.85845807857193, 65.87251388010401, 65.9217091854663, 65.97793239159463, 65.94279288776443, 65.92873708623233, 65.97793239159463, 66.0060439946588, 66.11849040691546, 66.10443460538336, 66.0903788038513, 66.04821139925504, 66.0060439946588, 66.04821139925504, 66.041183498489, 65.93576498699838, 65.86548597933796, 65.86548597933796, 65.89359758240214, 65.85143017780588, 65.77412326937943, 65.69681636095298, 65.54220254410006, 65.528146742568, 65.5140909410359, 65.43678403260945, 65.33839342188487, 65.29622601728863, 65.190807505798, 65.11350059737156, 65.02213788741302, 64.9518588797526, 64.90969147515636, 64.87455197132616, 64.8815798720922, 64.87455197132616, 64.87455197132616, 64.86049616979408, 64.846440368262, 64.82535666596388, 64.82535666596388, 64.83238456672991, 64.81832876519783, 64.7972450628997, 64.76210555906951, 64.73399395600535, 64.71993815447325, 64.69885445217514, 64.66371494834493, 64.67074284911097, 64.60046384145055, 64.56532433762034, 64.50910113149202, 64.43179422306557, 64.38962681846931, 64.38259891770328, 64.3685431161712, 64.27718040621266, 64.13662239089184, 64.164733993956, 64.1295944901258, 64.11553868859372, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.989598706866259, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 115, 209, 314, 415, 497, 573, 645, 710, 789, 851, 902, 962, 1022, 1074, 1130, 1190, 1235, 1284, 1328, 1375, 1425, 1467, 1511, 1549, 1589, 1626, 1660, 1710, 1746, 1779, 1803, 1832, 1864, 1898, 1914, 1938, 1961, 1981, 2007, 2030, 2058, 2078, 2096, 2119, 2140, 2163, 2181, 2199, 2216, 2228, 2248, 2264, 2275, 2289, 2305, 2321, 2333, 2342, 2351, 2360, 2372, 2388, 2397, 2404, 2412, 2418, 2425, 2427, 2432, 2433, 2437, 2438, 2438, 2438, 2438, 2439, 2439, 2439, 2440, 2440, 2442, 2444, 2445, 2447, 2450, 2450, 2453, 2454, 2459, 2463, 2470, 2473, 2483, 2493, 2501, 2504, 2528, 2548, 2569, 2589], "harmed": [0, 8, 20, 25, 36, 48, 66, 78, 88, 97, 113, 129, 144, 155, 166, 183, 194, 209, 227, 244, 261, 281, 295, 313, 326, 345, 363, 378, 396, 412, 424, 444, 464, 478, 501, 510, 533, 558, 586, 610, 626, 646, 671, 691, 707, 724, 731, 751, 771, 794, 812, 826, 843, 869, 893, 909, 921, 939, 959, 979, 1010, 1024, 1042, 1062, 1083, 1097, 1118, 1136, 1151, 1166, 1173, 1182, 1182, 1183, 1183, 1185, 1188, 1191, 1191, 1191, 1193, 1198, 1205, 1210, 1214, 1220, 1225, 1227, 1238, 1248, 1260, 1278, 1287, 1298, 1310, 1331, 1354, 1374, 1399, 1422, 1441], "exact_score": [56.05453650994448, 56.806521891910876, 57.38280975472626, 58.085599831330384, 58.71811090027408, 59.21006395389696, 59.61768219832736, 60.03935624428982, 60.42589078642209, 60.91784384004497, 61.24112727528287, 61.48710380209431, 61.81038723733221, 62.147726474102186, 62.43587040550986, 62.70995853538548, 63.05432567292149, 63.26516269590273, 63.48302761965, 63.67278094033312, 63.88361796331436, 64.09445498629559, 64.29123620774475, 64.47396162766181, 64.64965914681284, 64.7972450628997, 64.93077517745449, 65.06430529200927, 65.28919811652258, 65.4297561318434, 65.57734204793027, 65.60545365099443, 65.66870475788882, 65.79520697167756, 65.87251388010401, 65.9217091854663, 65.92873708623233, 65.91468128470026, 65.85845807857193, 65.87251388010401, 65.9217091854663, 65.97793239159463, 65.94279288776443, 65.92873708623233, 65.97793239159463, 66.0060439946588, 66.11849040691546, 66.10443460538336, 66.0903788038513, 66.04821139925504, 66.0060439946588, 66.04821139925504, 66.041183498489, 65.93576498699838, 65.86548597933796, 65.86548597933796, 65.89359758240214, 65.85143017780588, 65.77412326937943, 65.69681636095298, 65.54220254410006, 65.528146742568, 65.5140909410359, 65.43678403260945, 65.33839342188487, 65.29622601728863, 65.190807505798, 65.11350059737156, 65.02213788741302, 64.9518588797526, 64.90969147515636, 64.87455197132616, 64.8815798720922, 64.87455197132616, 64.87455197132616, 64.86049616979408, 64.846440368262, 64.82535666596388, 64.82535666596388, 64.83238456672991, 64.81832876519783, 64.7972450628997, 64.76210555906951, 64.73399395600535, 64.71993815447325, 64.69885445217514, 64.66371494834493, 64.67074284911097, 64.60046384145055, 64.56532433762034, 64.50910113149202, 64.43179422306557, 64.38962681846931, 64.38259891770328, 64.3685431161712, 64.27718040621266, 64.13662239089184, 64.164733993956, 64.1295944901258, 64.11553868859372, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.3211614626195, "threshold_area100": 64.32082366996978, "threshold_area25": 61.49413170286036}, "test": {"score": [26.688029946402747, 27.97266256416074, 29.257295181918728, 30.08252275756459, 31.032527011314983, 32.10447210957661, 33.20193971017781, 34.19448146782747, 35.13881405439129, 35.793891614440064, 36.55956668462695, 37.23449507982872, 37.79882596489238, 38.52196353117999, 39.09196608343023, 39.582565295068484, 40.2121203527777, 40.770779570654796, 41.32943878853189, 41.80869466579701, 42.327652213368125, 42.78138558829368, 43.12735728667442, 43.63780733346567, 44.16243654822335, 44.6728865950146, 45.129455803533446, 45.520800839406746, 45.88662337294048, 46.158863397895814, 46.555880100955676, 46.96424013838868, 47.256330998497006, 47.57110852735161, 47.814990216374106, 48.07872274054959, 48.322604429572074, 48.55797861781471, 48.76783030371778, 48.99469699118056, 49.20171284349035, 49.40589286220685, 49.58738621217707, 49.743357059807735, 49.92485040977795, 50.109179593341466, 50.2878371097184, 50.446643790942346, 50.59410713779315, 50.73022715027082, 50.85783966196864, 50.9769446728866, 51.12440801973741, 51.20664719394266, 51.32008053767405, 51.3853047103196, 51.41082721265916, 51.46470805093158, 51.49306638686441, 51.54694722513683, 51.55829055950997, 51.59515639622268, 51.600828063409246, 51.58097722825625, 51.57814139466297, 51.609335564189095, 51.59515639622268, 51.59799222981596, 51.59232056262939, 51.58381306184954, 51.56679806028982, 51.56112639310325, 51.55545472591668, 51.54694722513683, 51.549783058730114, 51.54694722513683, 51.544111391543545, 51.541275557950264, 51.53843972435698, 51.52709638998384, 51.52709638998384, 51.50157388764427, 51.49306638686441, 51.476051385304714, 51.47321555171143, 51.478887218897995, 51.45620055015172, 51.4420213821853, 51.413663046252445, 51.39664804469274, 51.382468876726314, 51.382468876726314, 51.33709553923376, 51.29455803533449, 51.2548563650285, 51.23500552987551, 51.18396052519638, 51.06485551427842, 50.942914669767184, 50.73873465105068, 50.00708958398321], "calls": [0.0, 1.605081813799166, 3.2413577971244645, 4.3643479000652246, 5.663159685789638, 7.322122337861215, 8.935711652440235, 10.5039276295267, 12.083486940986303, 13.249014547826334, 14.672603011655275, 16.03096730283867, 17.32127158778323, 18.753367552392024, 20.018149334997023, 21.02770609420639, 22.25562204009869, 23.596971329722372, 24.94682812012591, 26.11235572696594, 27.340271672858236, 28.50579927969827, 29.64864021779202, 30.998497008195557, 32.30581629469983, 33.822987267107166, 35.223889062189826, 36.582253353373225, 37.7392734594334, 38.615546039758385, 39.89450699032981, 41.241527947140064, 42.401383886793525, 43.58392649519327, 44.667214927828034, 45.82139920029493, 46.79976178997816, 47.70155687264271, 48.818875308396905, 49.916342908998104, 51.010974676006015, 52.253069789864725, 53.23143237954797, 54.28636247624989, 55.27890423389955, 56.495476845418715, 57.522048606187795, 58.63085954116213, 59.7878796472223, 60.828630575957796, 61.90341150781272, 63.09446161699231, 64.29401922695176, 65.27805348382158, 66.19686356804583, 67.35104784051272, 68.3577687661288, 69.34747469018518, 70.49031562827894, 71.62464906559282, 72.7476391685336, 73.7883900972691, 75.04750021268752, 76.36616283356491, 77.49766043728555, 78.68020304568529, 79.68975980489465, 80.71349573207044, 81.68051498738053, 82.53977256614581, 83.39619431131781, 84.54754275019141, 85.4238153305164, 86.09590789212488, 86.73680628420725, 87.27277883333807, 87.75487054419646, 88.25964892380115, 88.74741230184613, 89.17845900802541, 89.52726653999943, 89.89308907353316, 90.2390607719139, 90.61906247341406, 90.95085500382838, 91.32518503814197, 91.76757507869439, 92.16742761534753, 92.51907098091485, 92.90758018319485, 93.23937271360917, 93.71862859087429, 94.12982446190058, 94.64027450869183, 95.0372912117517, 95.47117375152426, 96.03550463658792, 96.76714970365539, 97.49312310353628, 98.38073901823441, 100.0], "rescued": [0, 478, 952, 1261, 1608, 2022, 2431, 2809, 3174, 3427, 3726, 3984, 4216, 4496, 4722, 4905, 5148, 5370, 5596, 5791, 5995, 6178, 6326, 6531, 6743, 6953, 7140, 7311, 7472, 7584, 7745, 7918, 8042, 8177, 8292, 8417, 8527, 8626, 8724, 8827, 8930, 9028, 9111, 9188, 9279, 9373, 9457, 9545, 9628, 9697, 9765, 9841, 9916, 9967, 10024, 10071, 10114, 10157, 10191, 10237, 10274, 10309, 10348, 10383, 10405, 10434, 10454, 10478, 10493, 10507, 10516, 10519, 10520, 10520, 10523, 10525, 10525, 10525, 10527, 10527, 10529, 10530, 10532, 10533, 10538, 10547, 10548, 10553, 10556, 10562, 10567, 10577, 10584, 10594, 10603, 10618, 10635, 10653, 10674, 10699, 10723], "harmed": [0, 25, 46, 64, 76, 112, 134, 162, 194, 216, 245, 265, 298, 323, 348, 358, 379, 404, 433, 459, 480, 503, 529, 554, 581, 611, 637, 670, 702, 718, 739, 768, 789, 813, 842, 874, 898, 914, 938, 961, 991, 1017, 1036, 1058, 1085, 1114, 1135, 1167, 1198, 1219, 1242, 1276, 1299, 1321, 1338, 1362, 1396, 1420, 1444, 1471, 1504, 1526, 1563, 1605, 1628, 1646, 1671, 1694, 1711, 1728, 1743, 1748, 1751, 1754, 1756, 1759, 1760, 1761, 1764, 1768, 1770, 1780, 1785, 1792, 1798, 1805, 1814, 1824, 1837, 1849, 1859, 1869, 1892, 1917, 1940, 1962, 1997, 2057, 2121, 2218, 2500], "exact_score": [26.688029946402747, 27.501914187675467, 28.264753424269067, 29.081473499135075, 29.835805234948815, 30.584465303575985, 31.25372203159119, 31.886112922893684, 32.58089215324845, 33.250148881263655, 33.87686810537958, 34.47806482715595, 35.10194821767859, 35.649264101182546, 36.22777415421263, 36.74389586819046, 37.214644244675725, 37.654198451634855, 38.13629016249327, 38.62688937413153, 39.08913024983694, 39.56271445991549, 40.073164506706746, 40.54107704959873, 40.95227292062502, 41.35212545727816, 41.75764966111789, 42.18018886651731, 42.63675807503616, 42.906162266398205, 43.263477299152086, 43.63780733346567, 44.04616737089868, 44.37512406771971, 44.718259932507166, 45.05005246292148, 45.359158324589515, 45.6796075206307, 45.97453421433231, 46.26946090803392, 46.598417604854944, 46.89334429855656, 47.17692765788504, 47.44066018206052, 47.67603437030316, 47.89439355698608, 48.12693191163543, 48.359470266284774, 48.60618778890055, 48.81036780761705, 49.008876159146986, 49.19320534271049, 49.37469869268072, 49.5448487082778, 49.68096872075547, 49.8596262371324, 50.041119587102635, 50.19141876754672, 50.36156878314381, 50.49201712843491, 50.61112213935286, 50.758585486203664, 50.85216799478207, 50.96276550492017, 51.09321385021127, 51.18963219238295, 51.29172220174121, 51.376797209539745, 51.413663046252445, 51.45052888296515, 51.49023055327113, 51.52142472279727, 51.5526188923234, 51.56396222669654, 51.60366389700253, 51.600828063409246, 51.58664889544282, 51.58948472903611, 51.58097722825625, 51.606499730595814, 51.58948472903611, 51.59232056262939, 51.60366389700253, 51.569633893883115, 51.569633893883115, 51.55545472591668, 51.54694722513683, 51.541275557950264, 51.544111391543545, 51.53276805717041, 51.50157388764427, 51.47037971811814, 51.45620055015172, 51.39664804469274, 51.354110540793464, 51.26336386580835, 51.186796358789664, 51.005303008819446, 50.8266454924425, 50.50619629640133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.653945651260045, "threshold_area100": 47.129895357740416, "threshold_area25": 37.25094291466977}, "selected_budget": 46, "selected": {"val": {"score": 66.11849040691546, "calls": 45.997610513739545, "rescued": 2163, "harmed": 731}, "test": {"score": 50.2878371097184, "calls": 57.522048606187795, "rescued": 9457, "harmed": 1135}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8979723453521729, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8489285707473755, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8209637999534607, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7933096885681152, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7614999413490295, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7336856722831726, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7092164754867554, "operator": "ge"}, {"budget": 0.08, "threshold": 0.6867752075195312, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6688904762268066, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6506221890449524, "operator": "ge"}, {"budget": 0.11, "threshold": 0.6320599317550659, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6147139668464661, "operator": "gt"}, {"budget": 0.13, "threshold": 0.5964675545692444, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5799756050109863, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5660858154296875, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5516048073768616, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5367701649665833, "operator": "ge"}, {"budget": 0.18, "threshold": 0.521061360836029, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5081828832626343, "operator": "ge"}, {"budget": 0.2, "threshold": 0.4955650866031647, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4830303192138672, "operator": "ge"}, {"budget": 0.22, "threshold": 0.47118788957595825, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4563196301460266, "operator": "ge"}, {"budget": 0.24, "threshold": 0.4418787658214569, "operator": "ge"}, {"budget": 0.25, "threshold": 0.42668646574020386, "operator": "ge"}, {"budget": 0.26, "threshold": 0.4134938716888428, "operator": "ge"}, {"budget": 0.27, "threshold": 0.4006917476654053, "operator": "ge"}, {"budget": 0.28, "threshold": 0.38952919840812683, "operator": "ge"}, {"budget": 0.29, "threshold": 0.380513995885849, "operator": "ge"}, {"budget": 0.3, "threshold": 0.36805158853530884, "operator": "ge"}, {"budget": 0.31, "threshold": 0.3566833734512329, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3455820083618164, "operator": "ge"}, {"budget": 0.33, "threshold": 0.33480262756347656, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3235478103160858, "operator": "ge"}, {"budget": 0.35, "threshold": 0.31241950392723083, "operator": "ge"}, {"budget": 0.36, "threshold": 0.30317339301109314, "operator": "ge"}, {"budget": 0.37, "threshold": 0.29484260082244873, "operator": "ge"}, {"budget": 0.38, "threshold": 0.28449615836143494, "operator": "ge"}, {"budget": 0.39, "threshold": 0.27535223960876465, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2668619155883789, "operator": "ge"}, {"budget": 0.41, "threshold": 0.2559983730316162, "operator": "ge"}, {"budget": 0.42, "threshold": 0.24649371206760406, "operator": "ge"}, {"budget": 0.43, "threshold": 0.23668867349624634, "operator": "ge"}, {"budget": 0.44, "threshold": 0.2275482714176178, "operator": "ge"}, {"budget": 0.45, "threshold": 0.21694645285606384, "operator": "ge"}, {"budget": 0.46, "threshold": 0.20826171338558197, "operator": "ge"}, {"budget": 0.47, "threshold": 0.19819559156894684, "operator": "ge"}, {"budget": 0.48, "threshold": 0.1883612424135208, "operator": "ge"}, {"budget": 0.49, "threshold": 0.1798878014087677, "operator": "ge"}, {"budget": 0.5, "threshold": 0.17083147168159485, "operator": "ge"}, {"budget": 0.51, "threshold": 0.1600838601589203, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1502004861831665, "operator": "ge"}, {"budget": 0.53, "threshold": 0.14141960442066193, "operator": "ge"}, {"budget": 0.54, "threshold": 0.13268065452575684, "operator": "ge"}, {"budget": 0.55, "threshold": 0.12383835762739182, "operator": "ge"}, {"budget": 0.56, "threshold": 0.11528711766004562, "operator": "ge"}, {"budget": 0.57, "threshold": 0.10678926110267639, "operator": "ge"}, {"budget": 0.58, "threshold": 0.09796129912137985, "operator": "ge"}, {"budget": 0.59, "threshold": 0.0891689881682396, "operator": "ge"}, {"budget": 0.6, "threshold": 0.08053667843341827, "operator": "ge"}, {"budget": 0.61, "threshold": 0.07275599986314774, "operator": "ge"}, {"budget": 0.62, "threshold": 0.06390669196844101, "operator": "ge"}, {"budget": 0.63, "threshold": 0.055729374289512634, "operator": "ge"}, {"budget": 0.64, "threshold": 0.04723424091935158, "operator": "ge"}, {"budget": 0.65, "threshold": 0.039548151195049286, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03283008933067322, "operator": "ge"}, {"budget": 0.67, "threshold": 0.02518165111541748, "operator": "ge"}, {"budget": 0.68, "threshold": 0.01788465678691864, "operator": "ge"}, {"budget": 0.69, "threshold": 0.01023905724287033, "operator": "ge"}, {"budget": 0.7, "threshold": 0.003975010477006435, "operator": "ge"}, {"budget": 0.71, "threshold": 7.774162804707885e-05, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0005267522064968944, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0010306058684363961, "operator": "ge"}, {"budget": 0.74, "threshold": -0.0015592302661389112, "operator": "ge"}, {"budget": 0.75, "threshold": -0.00216431706212461, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0028698602691292763, "operator": "ge"}, {"budget": 0.77, "threshold": -0.003715729108080268, "operator": "ge"}, {"budget": 0.78, "threshold": -0.004754906054586172, "operator": "ge"}, {"budget": 0.79, "threshold": -0.005801151040941477, "operator": "ge"}, {"budget": 0.8, "threshold": -0.006982957944273949, "operator": "ge"}, {"budget": 0.81, "threshold": -0.008602754212915897, "operator": "ge"}, {"budget": 0.82, "threshold": -0.01027899980545044, "operator": "ge"}, {"budget": 0.83, "threshold": -0.012374636717140675, "operator": "ge"}, {"budget": 0.84, "threshold": -0.014616246335208416, "operator": "ge"}, {"budget": 0.85, "threshold": -0.017737602815032005, "operator": "ge"}, {"budget": 0.86, "threshold": -0.021741455420851707, "operator": "ge"}, {"budget": 0.87, "threshold": -0.026340527459979057, "operator": "ge"}, {"budget": 0.88, "threshold": -0.03142254427075386, "operator": "ge"}, {"budget": 0.89, "threshold": -0.03804653882980347, "operator": "ge"}, {"budget": 0.9, "threshold": -0.04566209763288498, "operator": "ge"}, {"budget": 0.91, "threshold": -0.0561404824256897, "operator": "ge"}, {"budget": 0.92, "threshold": -0.06917421519756317, "operator": "ge"}, {"budget": 0.93, "threshold": -0.0844009667634964, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10254351794719696, "operator": "ge"}, {"budget": 0.95, "threshold": -0.12605692446231842, "operator": "ge"}, {"budget": 0.96, "threshold": -0.15871846675872803, "operator": "ge"}, {"budget": 0.97, "threshold": -0.20698675513267517, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2801450192928314, "operator": "ge"}, {"budget": 0.99, "threshold": -0.40608489513397217, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.806521891910876, 57.38280975472626, 58.085599831330384, 58.71811090027408, 59.21006395389696, 59.61768219832736, 60.03935624428982, 60.42589078642209, 60.91784384004497, 61.24112727528287, 61.48710380209431, 61.80335933656617, 62.147726474102186, 62.43587040550986, 62.70995853538548, 63.05432567292149, 63.26516269590273, 63.48302761965, 63.67278094033312, 63.88361796331436, 64.09445498629559, 64.29123620774475, 64.47396162766181, 64.64965914681284, 64.7972450628997, 64.93077517745449, 65.06430529200927, 65.28919811652258, 65.4297561318434, 65.57734204793027, 65.60545365099443, 65.66870475788882, 65.79520697167756, 65.87251388010401, 65.9217091854663, 65.92873708623233, 65.91468128470026, 65.85845807857193, 65.87251388010401, 65.9217091854663, 65.97793239159463, 65.94279288776443, 65.92873708623233, 65.97793239159463, 66.0060439946588, 66.11849040691546, 66.10443460538336, 66.0903788038513, 66.04821139925504, 66.0060439946588, 66.04821139925504, 66.041183498489, 65.93576498699838, 65.86548597933796, 65.86548597933796, 65.89359758240214, 65.85143017780588, 65.77412326937943, 65.69681636095298, 65.54220254410006, 65.528146742568, 65.5140909410359, 65.43678403260945, 65.33839342188487, 65.29622601728863, 65.190807505798, 65.11350059737156, 65.02213788741302, 64.9518588797526, 64.90969147515636, 64.87455197132616, 64.8815798720922, 64.87455197132616, 64.87455197132616, 64.86049616979408, 64.846440368262, 64.82535666596388, 64.82535666596388, 64.83238456672991, 64.81832876519783, 64.7972450628997, 64.76210555906951, 64.73399395600535, 64.71993815447325, 64.69885445217514, 64.66371494834493, 64.67074284911097, 64.60046384145055, 64.56532433762034, 64.50910113149202, 64.43179422306557, 64.38962681846931, 64.38259891770328, 64.3685431161712, 64.27718040621266, 64.13662239089184, 64.164733993956, 64.1295944901258, 64.11553868859372, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.989598706866259, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 115, 209, 314, 415, 497, 573, 645, 710, 789, 851, 902, 962, 1022, 1074, 1130, 1190, 1235, 1284, 1328, 1375, 1425, 1467, 1511, 1549, 1589, 1626, 1660, 1710, 1746, 1779, 1803, 1832, 1864, 1898, 1914, 1938, 1961, 1981, 2007, 2030, 2058, 2078, 2096, 2119, 2140, 2163, 2181, 2199, 2216, 2228, 2248, 2264, 2275, 2289, 2305, 2321, 2333, 2342, 2351, 2360, 2372, 2388, 2397, 2404, 2412, 2418, 2425, 2427, 2432, 2433, 2437, 2438, 2438, 2438, 2438, 2439, 2439, 2439, 2440, 2440, 2442, 2444, 2445, 2447, 2450, 2450, 2453, 2454, 2459, 2463, 2470, 2473, 2483, 2493, 2501, 2504, 2528, 2548, 2569, 2589], "harmed": [0, 8, 20, 25, 36, 48, 66, 78, 88, 97, 113, 129, 144, 155, 166, 183, 194, 209, 227, 244, 261, 281, 295, 313, 326, 345, 363, 378, 396, 412, 424, 444, 464, 478, 501, 510, 533, 558, 586, 610, 626, 646, 671, 691, 707, 724, 731, 751, 771, 794, 812, 826, 843, 869, 893, 909, 921, 939, 959, 979, 1010, 1024, 1042, 1062, 1083, 1097, 1118, 1136, 1151, 1166, 1173, 1182, 1182, 1183, 1183, 1185, 1188, 1191, 1191, 1191, 1193, 1198, 1205, 1210, 1214, 1220, 1225, 1227, 1238, 1248, 1260, 1278, 1287, 1298, 1310, 1331, 1354, 1374, 1399, 1422, 1441], "exact_score": [56.05453650994448, 56.806521891910876, 57.38280975472626, 58.085599831330384, 58.71811090027408, 59.21006395389696, 59.61768219832736, 60.03935624428982, 60.42589078642209, 60.91784384004497, 61.24112727528287, 61.48710380209431, 61.81038723733221, 62.147726474102186, 62.43587040550986, 62.70995853538548, 63.05432567292149, 63.26516269590273, 63.48302761965, 63.67278094033312, 63.88361796331436, 64.09445498629559, 64.29123620774475, 64.47396162766181, 64.64965914681284, 64.7972450628997, 64.93077517745449, 65.06430529200927, 65.28919811652258, 65.4297561318434, 65.57734204793027, 65.60545365099443, 65.66870475788882, 65.79520697167756, 65.87251388010401, 65.9217091854663, 65.92873708623233, 65.91468128470026, 65.85845807857193, 65.87251388010401, 65.9217091854663, 65.97793239159463, 65.94279288776443, 65.92873708623233, 65.97793239159463, 66.0060439946588, 66.11849040691546, 66.10443460538336, 66.0903788038513, 66.04821139925504, 66.0060439946588, 66.04821139925504, 66.041183498489, 65.93576498699838, 65.86548597933796, 65.86548597933796, 65.89359758240214, 65.85143017780588, 65.77412326937943, 65.69681636095298, 65.54220254410006, 65.528146742568, 65.5140909410359, 65.43678403260945, 65.33839342188487, 65.29622601728863, 65.190807505798, 65.11350059737156, 65.02213788741302, 64.9518588797526, 64.90969147515636, 64.87455197132616, 64.8815798720922, 64.87455197132616, 64.87455197132616, 64.86049616979408, 64.846440368262, 64.82535666596388, 64.82535666596388, 64.83238456672991, 64.81832876519783, 64.7972450628997, 64.76210555906951, 64.73399395600535, 64.71993815447325, 64.69885445217514, 64.66371494834493, 64.67074284911097, 64.60046384145055, 64.56532433762034, 64.50910113149202, 64.43179422306557, 64.38962681846931, 64.38259891770328, 64.3685431161712, 64.27718040621266, 64.13662239089184, 64.164733993956, 64.1295944901258, 64.11553868859372, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.3211614626195, "threshold_area100": 64.32082366996978, "threshold_area25": 61.49413170286036}, "test": {"score": [26.688029946402747, 27.97266256416074, 29.257295181918728, 30.08252275756459, 31.032527011314983, 32.10447210957661, 33.20193971017781, 34.19448146782747, 35.13881405439129, 35.793891614440064, 36.55956668462695, 37.23449507982872, 37.79882596489238, 38.52196353117999, 39.09196608343023, 39.582565295068484, 40.2121203527777, 40.770779570654796, 41.32943878853189, 41.80869466579701, 42.327652213368125, 42.78138558829368, 43.12735728667442, 43.63780733346567, 44.16243654822335, 44.6728865950146, 45.129455803533446, 45.520800839406746, 45.88662337294048, 46.158863397895814, 46.555880100955676, 46.96424013838868, 47.256330998497006, 47.57110852735161, 47.814990216374106, 48.07872274054959, 48.322604429572074, 48.55797861781471, 48.76783030371778, 48.99469699118056, 49.20171284349035, 49.40589286220685, 49.58738621217707, 49.743357059807735, 49.92485040977795, 50.109179593341466, 50.2878371097184, 50.446643790942346, 50.59410713779315, 50.73022715027082, 50.85783966196864, 50.9769446728866, 51.12440801973741, 51.20664719394266, 51.32008053767405, 51.3853047103196, 51.41082721265916, 51.46470805093158, 51.49306638686441, 51.54694722513683, 51.55829055950997, 51.59515639622268, 51.600828063409246, 51.58097722825625, 51.57814139466297, 51.609335564189095, 51.59515639622268, 51.59799222981596, 51.59232056262939, 51.58381306184954, 51.56679806028982, 51.56112639310325, 51.55545472591668, 51.54694722513683, 51.549783058730114, 51.54694722513683, 51.544111391543545, 51.541275557950264, 51.53843972435698, 51.52709638998384, 51.52709638998384, 51.50157388764427, 51.49306638686441, 51.476051385304714, 51.47321555171143, 51.478887218897995, 51.45620055015172, 51.4420213821853, 51.413663046252445, 51.39664804469274, 51.382468876726314, 51.382468876726314, 51.33709553923376, 51.29455803533449, 51.2548563650285, 51.23500552987551, 51.18396052519638, 51.06485551427842, 50.942914669767184, 50.73873465105068, 50.00708958398321], "calls": [0.0, 1.605081813799166, 3.2413577971244645, 4.3643479000652246, 5.663159685789638, 7.322122337861215, 8.935711652440235, 10.5039276295267, 12.083486940986303, 13.249014547826334, 14.672603011655275, 16.03096730283867, 17.32127158778323, 18.753367552392024, 20.018149334997023, 21.02770609420639, 22.25562204009869, 23.596971329722372, 24.94682812012591, 26.11235572696594, 27.340271672858236, 28.50579927969827, 29.64864021779202, 30.998497008195557, 32.30581629469983, 33.822987267107166, 35.223889062189826, 36.582253353373225, 37.7392734594334, 38.615546039758385, 39.89450699032981, 41.241527947140064, 42.401383886793525, 43.58392649519327, 44.667214927828034, 45.82139920029493, 46.79976178997816, 47.70155687264271, 48.818875308396905, 49.916342908998104, 51.010974676006015, 52.253069789864725, 53.23143237954797, 54.28636247624989, 55.27890423389955, 56.495476845418715, 57.522048606187795, 58.63085954116213, 59.7878796472223, 60.828630575957796, 61.90341150781272, 63.09446161699231, 64.29401922695176, 65.27805348382158, 66.19686356804583, 67.35104784051272, 68.3577687661288, 69.34747469018518, 70.49031562827894, 71.62464906559282, 72.7476391685336, 73.7883900972691, 75.04750021268752, 76.36616283356491, 77.49766043728555, 78.68020304568529, 79.68975980489465, 80.71349573207044, 81.68051498738053, 82.53977256614581, 83.39619431131781, 84.54754275019141, 85.4238153305164, 86.09590789212488, 86.73680628420725, 87.27277883333807, 87.75487054419646, 88.25964892380115, 88.74741230184613, 89.17845900802541, 89.52726653999943, 89.89308907353316, 90.2390607719139, 90.61906247341406, 90.95085500382838, 91.32518503814197, 91.76757507869439, 92.16742761534753, 92.51907098091485, 92.90758018319485, 93.23937271360917, 93.71862859087429, 94.12982446190058, 94.64027450869183, 95.0372912117517, 95.47117375152426, 96.03550463658792, 96.76714970365539, 97.49312310353628, 98.38073901823441, 100.0], "rescued": [0, 478, 952, 1261, 1608, 2022, 2431, 2809, 3174, 3427, 3726, 3984, 4216, 4496, 4722, 4905, 5148, 5370, 5596, 5791, 5995, 6178, 6326, 6531, 6743, 6953, 7140, 7311, 7472, 7584, 7745, 7918, 8042, 8177, 8292, 8417, 8527, 8626, 8724, 8827, 8930, 9028, 9111, 9188, 9279, 9373, 9457, 9545, 9628, 9697, 9765, 9841, 9916, 9967, 10024, 10071, 10114, 10157, 10191, 10237, 10274, 10309, 10348, 10383, 10405, 10434, 10454, 10478, 10493, 10507, 10516, 10519, 10520, 10520, 10523, 10525, 10525, 10525, 10527, 10527, 10529, 10530, 10532, 10533, 10538, 10547, 10548, 10553, 10556, 10562, 10567, 10577, 10584, 10594, 10603, 10618, 10635, 10653, 10674, 10699, 10723], "harmed": [0, 25, 46, 64, 76, 112, 134, 162, 194, 216, 245, 265, 298, 323, 348, 358, 379, 404, 433, 459, 480, 503, 529, 554, 581, 611, 637, 670, 702, 718, 739, 768, 789, 813, 842, 874, 898, 914, 938, 961, 991, 1017, 1036, 1058, 1085, 1114, 1135, 1167, 1198, 1219, 1242, 1276, 1299, 1321, 1338, 1362, 1396, 1420, 1444, 1471, 1504, 1526, 1563, 1605, 1628, 1646, 1671, 1694, 1711, 1728, 1743, 1748, 1751, 1754, 1756, 1759, 1760, 1761, 1764, 1768, 1770, 1780, 1785, 1792, 1798, 1805, 1814, 1824, 1837, 1849, 1859, 1869, 1892, 1917, 1940, 1962, 1997, 2057, 2121, 2218, 2500], "exact_score": [26.688029946402747, 27.501914187675467, 28.264753424269067, 29.081473499135075, 29.835805234948815, 30.584465303575985, 31.25372203159119, 31.886112922893684, 32.58089215324845, 33.250148881263655, 33.87686810537958, 34.47806482715595, 35.10194821767859, 35.649264101182546, 36.22777415421263, 36.74389586819046, 37.214644244675725, 37.654198451634855, 38.13629016249327, 38.62688937413153, 39.08913024983694, 39.56271445991549, 40.073164506706746, 40.54107704959873, 40.95227292062502, 41.35212545727816, 41.75764966111789, 42.18018886651731, 42.63675807503616, 42.906162266398205, 43.263477299152086, 43.63780733346567, 44.04616737089868, 44.37512406771971, 44.718259932507166, 45.05005246292148, 45.359158324589515, 45.6796075206307, 45.97453421433231, 46.26946090803392, 46.598417604854944, 46.89334429855656, 47.17692765788504, 47.44066018206052, 47.67603437030316, 47.89439355698608, 48.12693191163543, 48.359470266284774, 48.60618778890055, 48.81036780761705, 49.008876159146986, 49.19320534271049, 49.37469869268072, 49.5448487082778, 49.68096872075547, 49.8596262371324, 50.041119587102635, 50.19141876754672, 50.36156878314381, 50.49201712843491, 50.61112213935286, 50.758585486203664, 50.85216799478207, 50.96276550492017, 51.09321385021127, 51.18963219238295, 51.29172220174121, 51.376797209539745, 51.413663046252445, 51.45052888296515, 51.49023055327113, 51.52142472279727, 51.5526188923234, 51.56396222669654, 51.60366389700253, 51.600828063409246, 51.58664889544282, 51.58948472903611, 51.58097722825625, 51.606499730595814, 51.58948472903611, 51.59232056262939, 51.60366389700253, 51.569633893883115, 51.569633893883115, 51.55545472591668, 51.54694722513683, 51.541275557950264, 51.544111391543545, 51.53276805717041, 51.50157388764427, 51.47037971811814, 51.45620055015172, 51.39664804469274, 51.354110540793464, 51.26336386580835, 51.186796358789664, 51.005303008819446, 50.8266454924425, 50.50619629640133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.653945651260045, "threshold_area100": 47.129895357740416, "threshold_area25": 37.25094291466977}, "selected_budget": 46, "selected": {"val": {"score": 66.11849040691546, "calls": 45.997610513739545, "rescued": 2163, "harmed": 731}, "test": {"score": 50.2878371097184, "calls": 57.522048606187795, "rescued": 9457, "harmed": 1135}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 14.332597732543945, "operator": "ge"}, {"budget": 0.02, "threshold": 13.396369934082031, "operator": "ge"}, {"budget": 0.03, "threshold": 12.81235408782959, "operator": "ge"}, {"budget": 0.04, "threshold": 12.373201370239258, "operator": "ge"}, {"budget": 0.05, "threshold": 11.990941047668457, "operator": "ge"}, {"budget": 0.06, "threshold": 11.589092254638672, "operator": "ge"}, {"budget": 0.07, "threshold": 11.28818130493164, "operator": "ge"}, {"budget": 0.08, "threshold": 10.978263854980469, "operator": "ge"}, {"budget": 0.09, "threshold": 10.684392929077148, "operator": "ge"}, {"budget": 0.1, "threshold": 10.430364608764648, "operator": "ge"}, {"budget": 0.11, "threshold": 10.18772029876709, "operator": "ge"}, {"budget": 0.12, "threshold": 9.966772079467773, "operator": "ge"}, {"budget": 0.13, "threshold": 9.734382629394531, "operator": "ge"}, {"budget": 0.14, "threshold": 9.519445419311523, "operator": "ge"}, {"budget": 0.15, "threshold": 9.316116333007812, "operator": "ge"}, {"budget": 0.16, "threshold": 9.082743644714355, "operator": "gt"}, {"budget": 0.17, "threshold": 8.900917053222656, "operator": "ge"}, {"budget": 0.18, "threshold": 8.701020240783691, "operator": "ge"}, {"budget": 0.19, "threshold": 8.501321792602539, "operator": "ge"}, {"budget": 0.2, "threshold": 8.269357681274414, "operator": "ge"}, {"budget": 0.21, "threshold": 8.08032512664795, "operator": "ge"}, {"budget": 0.22, "threshold": 7.885705471038818, "operator": "ge"}, {"budget": 0.23, "threshold": 7.701417922973633, "operator": "ge"}, {"budget": 0.24, "threshold": 7.493069171905518, "operator": "ge"}, {"budget": 0.25, "threshold": 7.2591729164123535, "operator": "ge"}, {"budget": 0.26, "threshold": 7.070548057556152, "operator": "ge"}, {"budget": 0.27, "threshold": 6.896729946136475, "operator": "ge"}, {"budget": 0.28, "threshold": 6.695370674133301, "operator": "ge"}, {"budget": 0.29, "threshold": 6.51784610748291, "operator": "ge"}, {"budget": 0.3, "threshold": 6.334848880767822, "operator": "ge"}, {"budget": 0.31, "threshold": 6.150264263153076, "operator": "ge"}, {"budget": 0.32, "threshold": 5.96376371383667, "operator": "ge"}, {"budget": 0.33, "threshold": 5.7795233726501465, "operator": "ge"}, {"budget": 0.34, "threshold": 5.595800876617432, "operator": "ge"}, {"budget": 0.35, "threshold": 5.382781982421875, "operator": "ge"}, {"budget": 0.36, "threshold": 5.213393211364746, "operator": "ge"}, {"budget": 0.37, "threshold": 5.026216506958008, "operator": "ge"}, {"budget": 0.38, "threshold": 4.875718116760254, "operator": "ge"}, {"budget": 0.39, "threshold": 4.688462257385254, "operator": "ge"}, {"budget": 0.4, "threshold": 4.519634246826172, "operator": "ge"}, {"budget": 0.41, "threshold": 4.364621162414551, "operator": "ge"}, {"budget": 0.42, "threshold": 4.19014835357666, "operator": "ge"}, {"budget": 0.43, "threshold": 4.046273231506348, "operator": "ge"}, {"budget": 0.44, "threshold": 3.8969950675964355, "operator": "ge"}, {"budget": 0.45, "threshold": 3.733830213546753, "operator": "ge"}, {"budget": 0.46, "threshold": 3.5680789947509766, "operator": "ge"}, {"budget": 0.47, "threshold": 3.431617498397827, "operator": "ge"}, {"budget": 0.48, "threshold": 3.291250228881836, "operator": "ge"}, {"budget": 0.49, "threshold": 3.158207416534424, "operator": "ge"}, {"budget": 0.5, "threshold": 3.019481897354126, "operator": "ge"}, {"budget": 0.51, "threshold": 2.8753035068511963, "operator": "ge"}, {"budget": 0.52, "threshold": 2.7191336154937744, "operator": "ge"}, {"budget": 0.53, "threshold": 2.5740137100219727, "operator": "ge"}, {"budget": 0.54, "threshold": 2.4367763996124268, "operator": "ge"}, {"budget": 0.55, "threshold": 2.3045876026153564, "operator": "ge"}, {"budget": 0.56, "threshold": 2.1541175842285156, "operator": "ge"}, {"budget": 0.57, "threshold": 2.035184860229492, "operator": "ge"}, {"budget": 0.58, "threshold": 1.9075841903686523, "operator": "ge"}, {"budget": 0.59, "threshold": 1.7590314149856567, "operator": "ge"}, {"budget": 0.6, "threshold": 1.611219882965088, "operator": "ge"}, {"budget": 0.61, "threshold": 1.4673312902450562, "operator": "ge"}, {"budget": 0.62, "threshold": 1.3151731491088867, "operator": "ge"}, {"budget": 0.63, "threshold": 1.1787729263305664, "operator": "ge"}, {"budget": 0.64, "threshold": 1.0458649396896362, "operator": "ge"}, {"budget": 0.65, "threshold": 0.8841891288757324, "operator": "ge"}, {"budget": 0.66, "threshold": 0.7268307209014893, "operator": "ge"}, {"budget": 0.67, "threshold": 0.5874079465866089, "operator": "ge"}, {"budget": 0.68, "threshold": 0.4499107599258423, "operator": "ge"}, {"budget": 0.69, "threshold": 0.3103524446487427, "operator": "ge"}, {"budget": 0.7, "threshold": 0.1647491455078125, "operator": "ge"}, {"budget": 0.71, "threshold": 0.006550431251525879, "operator": "ge"}, {"budget": 0.72, "threshold": -0.18176567554473877, "operator": "ge"}, {"budget": 0.73, "threshold": -0.321006178855896, "operator": "ge"}, {"budget": 0.74, "threshold": -0.4767780303955078, "operator": "ge"}, {"budget": 0.75, "threshold": -0.62592613697052, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7864581346511841, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9412049055099487, "operator": "ge"}, {"budget": 0.78, "threshold": -1.0962042808532715, "operator": "ge"}, {"budget": 0.79, "threshold": -1.258332371711731, "operator": "ge"}, {"budget": 0.8, "threshold": -1.4372979402542114, "operator": "ge"}, {"budget": 0.81, "threshold": -1.6124138832092285, "operator": "ge"}, {"budget": 0.82, "threshold": -1.7865549325942993, "operator": "ge"}, {"budget": 0.83, "threshold": -1.9990200996398926, "operator": "ge"}, {"budget": 0.84, "threshold": -2.1958353519439697, "operator": "ge"}, {"budget": 0.85, "threshold": -2.3873746395111084, "operator": "ge"}, {"budget": 0.86, "threshold": -2.577807664871216, "operator": "ge"}, {"budget": 0.87, "threshold": -2.757272720336914, "operator": "ge"}, {"budget": 0.88, "threshold": -2.943835973739624, "operator": "ge"}, {"budget": 0.89, "threshold": -3.159935712814331, "operator": "ge"}, {"budget": 0.9, "threshold": -3.390382766723633, "operator": "ge"}, {"budget": 0.91, "threshold": -3.64497971534729, "operator": "ge"}, {"budget": 0.92, "threshold": -3.8693697452545166, "operator": "ge"}, {"budget": 0.93, "threshold": -4.176920413970947, "operator": "ge"}, {"budget": 0.94, "threshold": -4.4930267333984375, "operator": "ge"}, {"budget": 0.95, "threshold": -4.8270182609558105, "operator": "ge"}, {"budget": 0.96, "threshold": -5.329054355621338, "operator": "ge"}, {"budget": 0.97, "threshold": -5.883049964904785, "operator": "ge"}, {"budget": 0.98, "threshold": -6.639498710632324, "operator": "ge"}, {"budget": 0.99, "threshold": -7.848849296569824, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.86274509803921, 57.57256307540936, 58.120739335160586, 58.682971396443875, 59.29439876308946, 59.87771452667088, 60.418862885656054, 60.924871740811014, 61.43790849673202, 61.91580574882283, 62.33747979478529, 62.794293344577966, 63.21596739054044, 63.56033452807645, 63.91172956637852, 64.22798510085038, 64.66371494834493, 64.97294258205073, 65.34542132265092, 65.69681636095298, 66.0060439946588, 66.27310422376836, 66.54016445287792, 66.75802937662519, 66.9618384988404, 67.17970342258766, 67.37648464403682, 67.59434956778409, 67.74896338463701, 67.92466090378804, 67.98791201068241, 68.12847002600323, 68.1073863237051, 68.21983273596176, 68.21280483519571, 68.18469323213155, 68.15658162906739, 68.20577693442968, 68.06521891910886, 67.96682830838428, 67.88249349919178, 67.84032609459554, 67.75599128540306, 67.59434956778409, 67.52407056012369, 67.42567994939911, 67.28512193407829, 67.12348021645934, 67.03914540726684, 66.96886639960643, 66.88453159041394, 66.75802937662519, 66.6385550636025, 66.50502494904771, 66.42069013985522, 66.28716002530044, 66.16768571227773, 65.9919881931267, 65.86548597933796, 65.76006746784735, 65.67573265865485, 65.5984257502284, 65.4789514372057, 65.359477124183, 65.25405861269238, 65.16269590273384, 65.05727739124323, 64.9659146812847, 64.94483097898657, 64.90266357439033, 64.87455197132616, 64.846440368262, 64.83238456672991, 64.80427296366575, 64.82535666596388, 64.81832876519783, 64.7972450628997, 64.73399395600535, 64.71291025370721, 64.66371494834493, 64.64965914681284, 64.62154754374868, 64.62154754374868, 64.6074917422166, 64.58640803991848, 64.59343594068451, 64.6074917422166, 64.56532433762034, 64.53018483379014, 64.4880174291939, 64.4528779253637, 64.41071052076744, 64.37557101693724, 64.34745941387308, 64.33340361234099, 64.29826410851078, 64.26312460468057, 64.20690139855225, 64.17176189472204, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.988474242743692, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 115, 217, 295, 376, 463, 547, 624, 697, 772, 842, 905, 972, 1033, 1084, 1136, 1183, 1246, 1294, 1349, 1401, 1447, 1488, 1534, 1575, 1611, 1653, 1693, 1734, 1767, 1806, 1837, 1869, 1890, 1925, 1948, 1969, 1983, 2009, 2021, 2035, 2056, 2073, 2091, 2098, 2114, 2127, 2136, 2144, 2157, 2174, 2189, 2204, 2217, 2229, 2245, 2258, 2275, 2281, 2292, 2302, 2312, 2323, 2332, 2346, 2358, 2375, 2384, 2397, 2411, 2424, 2437, 2450, 2462, 2479, 2491, 2501, 2508, 2517, 2526, 2533, 2542, 2548, 2561, 2567, 2568, 2574, 2580, 2583, 2585, 2585, 2585, 2587, 2587, 2588, 2588, 2588, 2589, 2589, 2589, 2589], "harmed": [0, 0, 1, 1, 2, 2, 3, 3, 4, 6, 8, 11, 13, 14, 16, 18, 20, 21, 25, 27, 29, 31, 34, 42, 52, 59, 70, 82, 92, 103, 117, 139, 151, 175, 194, 218, 243, 261, 280, 312, 340, 373, 396, 426, 456, 482, 509, 538, 569, 594, 621, 648, 681, 711, 742, 770, 802, 836, 867, 896, 921, 943, 965, 991, 1022, 1049, 1079, 1103, 1129, 1146, 1165, 1182, 1199, 1213, 1234, 1243, 1254, 1264, 1282, 1294, 1308, 1319, 1329, 1342, 1350, 1354, 1359, 1363, 1372, 1379, 1385, 1390, 1398, 1403, 1408, 1410, 1415, 1421, 1429, 1434, 1441], "exact_score": [56.05453650994448, 56.86274509803921, 57.57256307540936, 58.120739335160586, 58.682971396443875, 59.29439876308946, 59.87771452667088, 60.418862885656054, 60.924871740811014, 61.43790849673202, 61.91580574882283, 62.33747979478529, 62.794293344577966, 63.21596739054044, 63.56033452807645, 63.91172956637852, 64.22798510085038, 64.66371494834493, 64.97294258205073, 65.34542132265092, 65.69681636095298, 66.0060439946588, 66.27310422376836, 66.54016445287792, 66.75802937662519, 66.9618384988404, 67.17970342258766, 67.37648464403682, 67.59434956778409, 67.74896338463701, 67.92466090378804, 67.98791201068241, 68.12847002600323, 68.1073863237051, 68.21983273596176, 68.21280483519571, 68.18469323213155, 68.15658162906739, 68.20577693442968, 68.06521891910886, 67.96682830838428, 67.88249349919178, 67.84032609459554, 67.75599128540306, 67.59434956778409, 67.52407056012369, 67.42567994939911, 67.28512193407829, 67.12348021645934, 67.03914540726684, 66.96886639960643, 66.88453159041394, 66.75802937662519, 66.6385550636025, 66.50502494904771, 66.42069013985522, 66.28716002530044, 66.16768571227773, 65.9919881931267, 65.86548597933796, 65.76006746784735, 65.67573265865485, 65.5984257502284, 65.4789514372057, 65.359477124183, 65.25405861269238, 65.16269590273384, 65.05727739124323, 64.9659146812847, 64.94483097898657, 64.90266357439033, 64.87455197132616, 64.846440368262, 64.83238456672991, 64.80427296366575, 64.82535666596388, 64.81832876519783, 64.7972450628997, 64.73399395600535, 64.71291025370721, 64.66371494834493, 64.64965914681284, 64.62154754374868, 64.62154754374868, 64.6074917422166, 64.58640803991848, 64.59343594068451, 64.6074917422166, 64.56532433762034, 64.53018483379014, 64.4880174291939, 64.4528779253637, 64.41071052076744, 64.37557101693724, 64.34745941387308, 64.33340361234099, 64.29826410851078, 64.26312460468057, 64.20690139855225, 64.17176189472204, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 65.10308049599682, "threshold_area100": 65.10281818820718, "threshold_area25": 62.46538758872725}, "test": {"score": [26.688029946402747, 27.4593766837762, 28.111618410231685, 28.596545954683382, 29.07863766554179, 29.526699373280778, 30.122224427870574, 30.638346141848398, 31.126109519893376, 31.69327623855032, 32.24626378924085, 32.73402716728583, 33.16790970705839, 33.66701641947651, 34.157615631114766, 34.591498170887334, 35.10194821767859, 35.55568159260415, 36.07463914017526, 36.57658168618666, 37.30822675325412, 37.82151263363866, 38.39151518588889, 38.91614440064657, 39.61375946459462, 40.33122536369566, 40.92391458469217, 41.573320477554375, 42.24541303916286, 42.792728922666825, 43.371238975696905, 43.94691319513371, 44.54527408331679, 45.24005331367156, 45.71080169015682, 46.320505912713045, 46.84229929387744, 47.23364432975073, 47.57961602813147, 48.09290190851601, 48.47573944360945, 48.77350197090435, 48.986189490400704, 49.26126534894932, 49.46544536766583, 49.68096872075547, 49.93902957774438, 50.146045430054166, 50.29067294331169, 50.42395712219607, 50.52037546436775, 50.65649547684542, 50.78127215494995, 50.85216799478207, 50.98828800725974, 51.0875421830247, 51.16694552363668, 51.17545302441653, 51.26619969940164, 51.30590136970763, 51.40231971187931, 51.42500638062559, 51.447693049371864, 51.47037971811814, 51.50724555483084, 51.50157388764427, 51.52709638998384, 51.52709638998384, 51.549783058730114, 51.5526188923234, 51.569633893883115, 51.56112639310325, 51.55829055950997, 51.52709638998384, 51.52142472279727, 51.49023055327113, 51.46470805093158, 51.447693049371864, 51.43634971499873, 51.39381221109945, 51.35978220798003, 51.33425970564047, 51.2747072001815, 51.20664719394266, 51.12440801973741, 51.09321385021127, 51.047840512718714, 50.98545217366646, 50.90604883305447, 50.818137991662645, 50.73022715027082, 50.6593313104387, 50.579927969826734, 50.509032129994615, 50.42112128860279, 50.36156878314381, 50.27932960893855, 50.236792105039285, 50.17156793239372, 50.112015426934754, 50.00708958398321], "calls": [0.0, 1.0095567592093697, 1.9425460114000508, 2.6912060800272237, 3.471060318180529, 4.208377052434563, 5.215097978050648, 6.057340555256217, 7.013016476193178, 8.016901568215976, 9.029294161018631, 9.945268411649606, 10.82154099197459, 11.814082749624252, 12.783937838527635, 13.762300428210871, 14.896633865524771, 15.872160621614723, 17.063210730794317, 18.21172333607464, 19.847999319399936, 21.03904942857953, 22.522190397867455, 23.86921135467771, 25.45444233332388, 27.187136658820858, 28.749680968720753, 30.31222527862065, 32.05626293849077, 33.57626974449139, 35.36284490826078, 37.078524232198056, 38.80838272410175, 40.711227065195814, 42.53750389927119, 44.6927374301676, 46.37722258457874, 48.192156084280974, 49.62425204888977, 51.50157388764427, 53.22292487876812, 54.70606584805604, 56.302640161075345, 57.71488529053115, 59.05056291296826, 60.553554717409185, 61.90908317499929, 63.13132745370502, 64.21461588633979, 65.18730680883645, 66.31880441255707, 67.44463034909111, 68.62150129030428, 69.83807390182344, 70.79942148994697, 71.7607690780705, 72.76749000368659, 73.49346340356749, 74.3243626463999, 75.23750106343759, 76.09959447579617, 76.984374556901, 77.93721464424468, 78.63482970819273, 79.37498227604004, 80.2200606868389, 81.0566315968579, 81.7826049967388, 82.46036922553385, 83.11544678558262, 83.81022601593739, 84.57590108612426, 85.43232283129626, 86.02501205229277, 86.58650710376315, 87.20471882709923, 87.77755721294275, 88.39860476987211, 88.91756231744321, 89.58114737827184, 90.23622493832062, 90.86010832884325, 91.45846921702635, 92.19295011768709, 92.85653517851571, 93.4945977370048, 93.98236111504977, 94.51549783058731, 95.08266454924426, 95.64983126790119, 96.19714715140516, 96.77282137084195, 97.30028641919291, 97.75685562771177, 98.24461900575673, 98.64730737600318, 99.05850324702946, 99.3676091086975, 99.60014746334686, 99.80716331565664, 100.0], "rescued": [0, 272, 502, 675, 848, 1008, 1221, 1404, 1579, 1781, 1979, 2153, 2311, 2490, 2664, 2822, 3010, 3177, 3364, 3551, 3822, 4015, 4230, 4433, 4693, 4960, 5191, 5441, 5704, 5926, 6160, 6400, 6641, 6924, 7131, 7391, 7609, 7793, 7948, 8169, 8348, 8498, 8628, 8767, 8890, 9015, 9144, 9256, 9342, 9428, 9501, 9592, 9689, 9766, 9838, 9906, 9974, 10011, 10060, 10110, 10170, 10208, 10249, 10281, 10316, 10355, 10388, 10412, 10444, 10469, 10495, 10519, 10549, 10561, 10577, 10589, 10599, 10614, 10627, 10637, 10648, 10659, 10666, 10675, 10683, 10690, 10693, 10699, 10703, 10706, 10708, 10712, 10716, 10717, 10718, 10720, 10723, 10723, 10723, 10723, 10723], "harmed": [0, 0, 0, 2, 5, 7, 10, 11, 14, 16, 19, 21, 26, 29, 30, 35, 43, 50, 54, 64, 77, 89, 103, 121, 135, 149, 171, 192, 218, 247, 277, 314, 344, 382, 423, 468, 502, 548, 581, 621, 665, 710, 765, 807, 858, 907, 945, 984, 1019, 1058, 1097, 1140, 1193, 1245, 1269, 1302, 1342, 1376, 1393, 1429, 1455, 1485, 1518, 1542, 1564, 1605, 1629, 1653, 1677, 1701, 1721, 1748, 1779, 1802, 1820, 1843, 1862, 1883, 1900, 1925, 1948, 1968, 1996, 2029, 2066, 2084, 2103, 2131, 2163, 2197, 2230, 2259, 2291, 2317, 2349, 2372, 2404, 2419, 2442, 2463, 2500], "exact_score": [26.688029946402747, 27.45370501658963, 28.162663414910817, 28.778039304653603, 29.41326602954939, 30.01729858491904, 30.6014803051357, 31.11476618552024, 31.681932904177184, 32.223577120494575, 32.762385503218674, 33.26716388282336, 33.77194226242805, 34.23418313813345, 34.687916513059015, 35.15015738876443, 35.615234098063134, 36.06045997220883, 36.508521679947826, 36.94240421972039, 37.37345092589967, 37.80166179848566, 38.17882766639254, 38.59285937101211, 38.978532739698835, 39.40390777869155, 39.829282817684266, 40.25182202308369, 40.62898789099056, 41.008989592490714, 41.437200465076714, 41.85690383688285, 42.22556220400987, 42.55168306723762, 42.92884893514449, 43.2436264639991, 43.589598162379836, 43.932734027167285, 44.27303405836146, 44.59064742080935, 44.9706491223095, 45.33363582224995, 45.60304001361201, 45.83557836826135, 46.13334089555625, 46.41692425488473, 46.73170178373933, 46.992598474321525, 47.18259932507161, 47.39812267816125, 47.67887020389644, 47.97663273119134, 48.220514420213824, 48.42753027252361, 48.620366956866974, 48.80753197402377, 48.94648782009472, 49.125145336471654, 49.315146187221735, 49.45693786688597, 49.59022204577036, 49.749028726994304, 49.96171624649066, 50.12619459490117, 50.26515044097213, 50.398434619856516, 50.509032129994615, 50.613957972946146, 50.68485381277827, 50.79828715650966, 50.86067549556192, 50.99963134163288, 51.11873635255083, 51.16127385645011, 51.22082636190909, 51.28888636814792, 51.39664804469274, 51.42784221421888, 51.459036383745, 51.49873805405099, 51.51858888920399, 51.51858888920399, 51.52426055639056, 51.55545472591668, 51.56679806028982, 51.55545472591668, 51.535603890763696, 51.49306638686441, 51.46187221733829, 51.43634971499873, 51.37963304313303, 51.31440887048748, 51.22366219550237, 51.11873635255083, 51.047840512718714, 50.920228001020895, 50.76142131979695, 50.62530130731929, 50.46649462609534, 50.2878371097184, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 44.681195660624695, "threshold_area100": 45.57710631540142, "threshold_area25": 33.436463148342455}, "selected_budget": 34, "selected": {"val": {"score": 68.21983273596176, "calls": 33.993956005341204, "rescued": 1925, "harmed": 194}, "test": {"score": 45.71080169015682, "calls": 42.53750389927119, "rescued": 7131, "harmed": 423}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.383932113647461, "operator": "ge"}, {"budget": 0.02, "threshold": 1.9670859575271606, "operator": "ge"}, {"budget": 0.03, "threshold": 1.7816376686096191, "operator": "ge"}, {"budget": 0.04, "threshold": 1.632758378982544, "operator": "ge"}, {"budget": 0.05, "threshold": 1.5085314512252808, "operator": "ge"}, {"budget": 0.06, "threshold": 1.3956785202026367, "operator": "ge"}, {"budget": 0.07, "threshold": 1.309852957725525, "operator": "ge"}, {"budget": 0.08, "threshold": 1.23082435131073, "operator": "ge"}, {"budget": 0.09, "threshold": 1.166888952255249, "operator": "ge"}, {"budget": 0.1, "threshold": 1.1218442916870117, "operator": "gt"}, {"budget": 0.11, "threshold": 1.0741174221038818, "operator": "ge"}, {"budget": 0.12, "threshold": 1.028255581855774, "operator": "ge"}, {"budget": 0.13, "threshold": 0.987982988357544, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9419682621955872, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9069255590438843, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8679394721984863, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8367727398872375, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8091050982475281, "operator": "ge"}, {"budget": 0.19, "threshold": 0.7794936299324036, "operator": "ge"}, {"budget": 0.2, "threshold": 0.7525476217269897, "operator": "ge"}, {"budget": 0.21, "threshold": 0.7230163812637329, "operator": "ge"}, {"budget": 0.22, "threshold": 0.6995643377304077, "operator": "ge"}, {"budget": 0.23, "threshold": 0.679212749004364, "operator": "ge"}, {"budget": 0.24, "threshold": 0.6560543775558472, "operator": "ge"}, {"budget": 0.25, "threshold": 0.6310791969299316, "operator": "ge"}, {"budget": 0.26, "threshold": 0.6081565618515015, "operator": "ge"}, {"budget": 0.27, "threshold": 0.5862104296684265, "operator": "ge"}, {"budget": 0.28, "threshold": 0.5643634796142578, "operator": "ge"}, {"budget": 0.29, "threshold": 0.5436792969703674, "operator": "ge"}, {"budget": 0.3, "threshold": 0.5261755585670471, "operator": "ge"}, {"budget": 0.31, "threshold": 0.5098735094070435, "operator": "ge"}, {"budget": 0.32, "threshold": 0.49245893955230713, "operator": "ge"}, {"budget": 0.33, "threshold": 0.47727733850479126, "operator": "ge"}, {"budget": 0.34, "threshold": 0.4591684341430664, "operator": "ge"}, {"budget": 0.35, "threshold": 0.4434381127357483, "operator": "ge"}, {"budget": 0.36, "threshold": 0.4296996593475342, "operator": "ge"}, {"budget": 0.37, "threshold": 0.41233110427856445, "operator": "ge"}, {"budget": 0.38, "threshold": 0.3976646661758423, "operator": "ge"}, {"budget": 0.39, "threshold": 0.38454973697662354, "operator": "ge"}, {"budget": 0.4, "threshold": 0.37196385860443115, "operator": "gt"}, {"budget": 0.41, "threshold": 0.358717679977417, "operator": "ge"}, {"budget": 0.42, "threshold": 0.3444143533706665, "operator": "ge"}, {"budget": 0.43, "threshold": 0.33139514923095703, "operator": "ge"}, {"budget": 0.44, "threshold": 0.319729208946228, "operator": "ge"}, {"budget": 0.45, "threshold": 0.30798614025115967, "operator": "ge"}, {"budget": 0.46, "threshold": 0.2947844862937927, "operator": "ge"}, {"budget": 0.47, "threshold": 0.2827189564704895, "operator": "ge"}, {"budget": 0.48, "threshold": 0.2714524269104004, "operator": "ge"}, {"budget": 0.49, "threshold": 0.26019152998924255, "operator": "ge"}, {"budget": 0.5, "threshold": 0.24792873859405518, "operator": "ge"}, {"budget": 0.51, "threshold": 0.2362678050994873, "operator": "ge"}, {"budget": 0.52, "threshold": 0.22378265857696533, "operator": "ge"}, {"budget": 0.53, "threshold": 0.21204376220703125, "operator": "ge"}, {"budget": 0.54, "threshold": 0.20112574100494385, "operator": "gt"}, {"budget": 0.55, "threshold": 0.19211113452911377, "operator": "ge"}, {"budget": 0.56, "threshold": 0.17866742610931396, "operator": "ge"}, {"budget": 0.57, "threshold": 0.16593873500823975, "operator": "ge"}, {"budget": 0.58, "threshold": 0.15577411651611328, "operator": "ge"}, {"budget": 0.59, "threshold": 0.14341449737548828, "operator": "ge"}, {"budget": 0.6, "threshold": 0.13221323490142822, "operator": "ge"}, {"budget": 0.61, "threshold": 0.12238970398902893, "operator": "ge"}, {"budget": 0.62, "threshold": 0.11017227172851562, "operator": "ge"}, {"budget": 0.63, "threshold": 0.10030031204223633, "operator": "ge"}, {"budget": 0.64, "threshold": 0.08920454978942871, "operator": "gt"}, {"budget": 0.65, "threshold": 0.07905459403991699, "operator": "ge"}, {"budget": 0.66, "threshold": 0.06854581832885742, "operator": "ge"}, {"budget": 0.67, "threshold": 0.054985761642456055, "operator": "ge"}, {"budget": 0.68, "threshold": 0.044367313385009766, "operator": "ge"}, {"budget": 0.69, "threshold": 0.03278326988220215, "operator": "ge"}, {"budget": 0.7, "threshold": 0.01977945864200592, "operator": "ge"}, {"budget": 0.71, "threshold": 0.0011469125747680664, "operator": "ge"}, {"budget": 0.72, "threshold": -0.04646998643875122, "operator": "ge"}, {"budget": 0.73, "threshold": -0.09814423322677612, "operator": "ge"}, {"budget": 0.74, "threshold": -0.16138744354248047, "operator": "ge"}, {"budget": 0.75, "threshold": -0.22409707307815552, "operator": "ge"}, {"budget": 0.76, "threshold": -0.30176806449890137, "operator": "ge"}, {"budget": 0.77, "threshold": -0.38732296228408813, "operator": "ge"}, {"budget": 0.78, "threshold": -0.4780416488647461, "operator": "ge"}, {"budget": 0.79, "threshold": -0.5939136743545532, "operator": "ge"}, {"budget": 0.8, "threshold": -0.7223753929138184, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8300843238830566, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9772176742553711, "operator": "ge"}, {"budget": 0.83, "threshold": -1.1803860664367676, "operator": "ge"}, {"budget": 0.84, "threshold": -1.3229314088821411, "operator": "ge"}, {"budget": 0.85, "threshold": -1.4998924732208252, "operator": "ge"}, {"budget": 0.86, "threshold": -1.6778696775436401, "operator": "ge"}, {"budget": 0.87, "threshold": -1.8610286712646484, "operator": "ge"}, {"budget": 0.88, "threshold": -2.044112205505371, "operator": "ge"}, {"budget": 0.89, "threshold": -2.2532811164855957, "operator": "ge"}, {"budget": 0.9, "threshold": -2.4867188930511475, "operator": "ge"}, {"budget": 0.91, "threshold": -2.737570285797119, "operator": "ge"}, {"budget": 0.92, "threshold": -3.001455068588257, "operator": "ge"}, {"budget": 0.93, "threshold": -3.289929151535034, "operator": "ge"}, {"budget": 0.94, "threshold": -3.6288375854492188, "operator": "ge"}, {"budget": 0.95, "threshold": -3.975815773010254, "operator": "ge"}, {"budget": 0.96, "threshold": -4.393307209014893, "operator": "ge"}, {"budget": 0.97, "threshold": -4.995892524719238, "operator": "ge"}, {"budget": 0.98, "threshold": -5.7143473625183105, "operator": "ge"}, {"budget": 0.99, "threshold": -6.9789934158325195, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.820577693442964, 57.27739124323564, 57.980181319839765, 58.556469182655135, 59.13275704547052, 59.48415208377257, 59.82851922130858, 60.21505376344085, 60.55942090097688, 60.91784384004497, 61.29735048141119, 61.45196429826411, 61.70496872584159, 62.07041956567573, 62.2531449855928, 62.463982008574035, 62.695902733853394, 62.927823459132746, 63.15974418441211, 63.37058120739335, 63.54627872654438, 63.70792044416332, 63.91172956637852, 64.1295944901258, 64.2842083069787, 64.43179422306557, 64.50207323072597, 64.58640803991848, 64.65668704757888, 64.81130086443179, 64.97294258205073, 65.09944479583947, 65.26108651345842, 65.33136552111884, 65.40164452877924, 65.45786773490758, 65.5140909410359, 65.54220254410006, 65.60545365099443, 65.58436994869632, 65.61248155176048, 65.57031414716423, 65.5632862463982, 65.61950945252653, 65.67573265865485, 65.7179000632511, 65.72492796401714, 65.71087216248506, 65.73195586478319, 65.71087216248506, 65.8373743762738, 65.84440227703985, 65.8022348724436, 65.8022348724436, 65.77412326937943, 65.78817907091151, 65.68978846018693, 65.62653735329256, 65.57031414716423, 65.62653735329256, 65.63356525405861, 65.57031414716423, 65.45786773490758, 65.34542132265092, 65.359477124183, 65.26811441422447, 65.21891910886217, 65.18377960503197, 65.04322158971115, 64.94483097898657, 64.87455197132616, 64.75507765830346, 64.67777074987701, 64.67074284911097, 64.6426312460468, 64.62857544451472, 64.63560334528076, 64.62154754374868, 64.61451964298264, 64.60046384145055, 64.57938013915243, 64.57938013915243, 64.57235223838639, 64.5582964368543, 64.57938013915243, 64.55126853608827, 64.55126853608827, 64.55126853608827, 64.48098952842786, 64.41071052076744, 64.37557101693724, 64.3685431161712, 64.34043151310703, 64.3193478108089, 64.29123620774475, 64.27015250544662, 64.23501300161641, 64.19284559702017, 64.164733993956, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.986646988544521, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.988755358774334, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.988333684728374, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.98903647480497, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 115, 196, 301, 392, 485, 550, 613, 681, 741, 805, 871, 911, 960, 1024, 1064, 1110, 1157, 1210, 1256, 1298, 1342, 1381, 1428, 1476, 1515, 1549, 1583, 1612, 1643, 1679, 1717, 1754, 1790, 1817, 1839, 1863, 1892, 1919, 1943, 1959, 1987, 2004, 2028, 2050, 2074, 2097, 2119, 2134, 2153, 2170, 2194, 2210, 2228, 2247, 2259, 2276, 2296, 2308, 2317, 2335, 2350, 2359, 2367, 2373, 2389, 2400, 2410, 2420, 2423, 2431, 2437, 2448, 2458, 2470, 2481, 2492, 2507, 2517, 2526, 2533, 2541, 2555, 2561, 2567, 2575, 2577, 2581, 2585, 2586, 2586, 2587, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589], "harmed": [0, 6, 22, 27, 36, 47, 62, 76, 89, 100, 113, 125, 143, 156, 168, 182, 198, 212, 232, 245, 257, 276, 292, 310, 327, 344, 357, 381, 398, 419, 433, 448, 467, 480, 497, 509, 525, 546, 569, 584, 603, 627, 650, 675, 689, 705, 722, 743, 760, 776, 796, 802, 817, 841, 860, 876, 891, 925, 946, 963, 973, 987, 1005, 1029, 1051, 1065, 1089, 1106, 1121, 1144, 1166, 1182, 1210, 1231, 1244, 1259, 1272, 1286, 1298, 1308, 1317, 1328, 1342, 1349, 1357, 1362, 1368, 1372, 1376, 1387, 1397, 1403, 1406, 1410, 1413, 1417, 1420, 1425, 1431, 1435, 1441], "exact_score": [56.05453650994448, 56.820577693442964, 57.27739124323564, 57.980181319839765, 58.556469182655135, 59.13275704547052, 59.48415208377257, 59.82851922130858, 60.21505376344085, 60.55942090097688, 60.924871740811014, 61.29735048141119, 61.45196429826411, 61.70496872584159, 62.07041956567573, 62.2531449855928, 62.463982008574035, 62.695902733853394, 62.927823459132746, 63.15974418441211, 63.37058120739335, 63.54627872654438, 63.70792044416332, 63.91172956637852, 64.1295944901258, 64.2842083069787, 64.43179422306557, 64.50207323072597, 64.58640803991848, 64.65668704757888, 64.81130086443179, 64.97294258205073, 65.09944479583947, 65.26108651345842, 65.33136552111884, 65.40164452877924, 65.45786773490758, 65.5140909410359, 65.54220254410006, 65.60545365099443, 65.59139784946237, 65.61248155176048, 65.57031414716423, 65.5632862463982, 65.61950945252653, 65.67573265865485, 65.7179000632511, 65.72492796401714, 65.71087216248506, 65.73195586478319, 65.71087216248506, 65.8373743762738, 65.84440227703985, 65.8022348724436, 65.8022348724436, 65.77412326937943, 65.78817907091151, 65.68978846018693, 65.62653735329256, 65.57031414716423, 65.62653735329256, 65.63356525405861, 65.57031414716423, 65.45786773490758, 65.34542132265092, 65.359477124183, 65.26811441422447, 65.21891910886217, 65.18377960503197, 65.04322158971115, 64.94483097898657, 64.87455197132616, 64.75507765830346, 64.67777074987701, 64.67074284911097, 64.6426312460468, 64.62857544451472, 64.63560334528076, 64.62154754374868, 64.61451964298264, 64.60046384145055, 64.57938013915243, 64.57938013915243, 64.57235223838639, 64.5582964368543, 64.57938013915243, 64.55126853608827, 64.55126853608827, 64.55126853608827, 64.48098952842786, 64.41071052076744, 64.37557101693724, 64.3685431161712, 64.34043151310703, 64.3193478108089, 64.29123620774475, 64.27015250544662, 64.23501300161641, 64.19284559702017, 64.164733993956, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.08355060234996, "threshold_area100": 64.08314006606227, "threshold_area25": 61.16873989739264}, "test": {"score": [26.688029946402747, 27.830870884496502, 29.061622663982078, 29.971925247426483, 30.90207866602388, 31.761336244789156, 32.74537050165897, 33.61880724839067, 34.54896066698806, 35.34866574029436, 35.88180245583189, 36.54255168306724, 37.15509173921674, 37.68822845475428, 38.37733601792247, 38.87077106315402, 39.46346028415053, 39.92570115985594, 40.473017043359896, 41.011825426084, 41.42869296429686, 41.9221280095284, 42.370189717267394, 42.76437058673397, 43.11317811870799, 43.56407566004027, 43.966764030286704, 44.360944899753285, 44.84020077701841, 45.21736664492528, 45.5293083401866, 45.83274253466807, 46.20423673538837, 46.4963275954967, 46.825284292317725, 47.18259932507161, 47.420809346907525, 47.73558687576213, 47.94827439525849, 48.118424410855575, 48.33111193035193, 48.549471117034855, 48.81036780761705, 48.949323653688005, 49.16484700677765, 49.36619119190086, 49.5448487082778, 49.69514788872189, 49.82559623401299, 50.0099254175765, 50.20276210191986, 50.30768794487139, 50.49201712843491, 50.642316308878996, 50.7897796557298, 50.8663471627485, 51.0138105095993, 51.15560218926354, 51.21515469472251, 51.33142387204719, 51.39948387828602, 51.39664804469274, 51.45052888296515, 51.49306638686441, 51.544111391543545, 51.57814139466297, 51.58097722825625, 51.569633893883115, 51.60366389700253, 51.615007231375664, 51.609335564189095, 51.56112639310325, 51.52993222357713, 51.49873805405099, 51.481723052491276, 51.45336471655844, 51.42784221421888, 51.382468876726314, 51.368289708759896, 51.33142387204719, 51.29739386892778, 51.24067719706208, 51.16694552363668, 51.07903468224485, 51.03649717834558, 50.92306383461419, 50.8663471627485, 50.809630490882796, 50.69619714715141, 50.62530130731929, 50.55156963389389, 50.45798712531548, 50.404106287043085, 50.341717947990816, 50.31903127924453, 50.29067294331169, 50.24529960581914, 50.18858293395344, 50.151717097240734, 50.10067209256162, 50.00708958398321], "calls": [0.0, 1.4661259677282137, 3.0428494455945323, 4.279272892266682, 5.58092051158438, 6.9676431387006215, 8.447948274395259, 9.806312565578652, 11.377364376258402, 12.775430337747782, 13.819017100076566, 15.078127215494993, 16.43081983949182, 17.59634744633185, 19.087995916399624, 20.171284349034398, 21.50412613787823, 22.587414570513005, 23.65652383518135, 24.941156452939342, 26.064146555880104, 27.37997334316422, 28.624904290616225, 29.64864021779202, 30.8793919972776, 32.20089045174829, 33.360746391401754, 34.489408161529084, 35.87329495505204, 37.2004650767093, 38.30644017809035, 39.443609448997535, 40.572271219124865, 41.53361880724839, 42.84377392734594, 44.10855570995094, 45.08408246604089, 46.33752091427275, 47.392451010974675, 48.29708192723251, 49.31231035362845, 50.31903127924453, 51.54694722513683, 52.63023565777161, 53.74755409352579, 54.839350026940416, 55.962340129881184, 56.969061055497264, 57.97861781470664, 59.04489124578169, 60.09982134248363, 61.143408104812416, 62.27206987493974, 63.32983580523495, 64.43864674020928, 65.2355159799223, 66.45492442503473, 67.72821370841959, 68.67821796216998, 69.77852139636445, 70.96957150554405, 71.93091909366758, 73.23823838017185, 74.34137764795962, 75.56929359385191, 76.75467203584493, 78.05631965516262, 79.70677480645436, 80.98857159061906, 82.35827921617559, 83.59753849644103, 84.57590108612426, 85.46918866800897, 86.08456455775175, 86.66874627796841, 87.14516632164025, 87.72651220826361, 88.33905226441313, 88.89771148229022, 89.53010237359271, 90.1284632617758, 90.70980914839917, 91.38473754360095, 92.25250262314607, 92.77429600431046, 93.43220939795252, 94.0334061197289, 94.57788616963957, 95.15639622266966, 95.80580211553186, 96.42117800527465, 97.01953889345774, 97.46760060119672, 97.9553639792417, 98.43178402291353, 98.78909905566742, 99.10954825170859, 99.4413407821229, 99.65119246802597, 99.82417831721635, 100.0], "rescued": [0, 429, 882, 1222, 1568, 1898, 2270, 2598, 2956, 3270, 3485, 3743, 3977, 4191, 4466, 4661, 4903, 5083, 5291, 5506, 5675, 5880, 6065, 6226, 6382, 6565, 6732, 6893, 7083, 7251, 7387, 7531, 7678, 7796, 7943, 8094, 8195, 8329, 8429, 8509, 8610, 8710, 8830, 8914, 9015, 9104, 9189, 9261, 9330, 9415, 9503, 9569, 9656, 9736, 9808, 9848, 9926, 10003, 10043, 10108, 10153, 10184, 10230, 10276, 10317, 10355, 10389, 10431, 10458, 10488, 10507, 10519, 10545, 10561, 10573, 10585, 10601, 10610, 10623, 10635, 10651, 10658, 10669, 10682, 10694, 10694, 10701, 10705, 10708, 10713, 10717, 10717, 10719, 10720, 10721, 10722, 10723, 10723, 10723, 10723, 10723], "harmed": [0, 26, 45, 64, 82, 109, 134, 154, 184, 216, 243, 268, 286, 312, 344, 365, 398, 415, 430, 455, 477, 508, 535, 557, 590, 614, 639, 661, 682, 717, 743, 780, 796, 811, 842, 867, 884, 907, 932, 952, 978, 1001, 1029, 1064, 1089, 1107, 1129, 1148, 1171, 1191, 1211, 1240, 1262, 1289, 1309, 1322, 1348, 1375, 1394, 1418, 1439, 1471, 1498, 1529, 1552, 1578, 1611, 1657, 1672, 1698, 1719, 1748, 1785, 1812, 1830, 1852, 1877, 1902, 1920, 1945, 1973, 2000, 2037, 2081, 2108, 2148, 2175, 2199, 2242, 2272, 2302, 2335, 2356, 2379, 2388, 2399, 2416, 2436, 2449, 2467, 2500], "exact_score": [26.688029946402747, 27.48489918611576, 28.233559254742936, 29.016249326489525, 29.776252729489837, 30.50222612937073, 31.16013952301279, 31.781187079942153, 32.436264639990924, 33.119700535972555, 33.763434761648185, 34.324929813118565, 34.90627569974194, 35.48478575277203, 35.983892465190145, 36.50285001276126, 36.982105890026375, 37.43867509854522, 37.87539347191107, 38.32629101324335, 38.81405439128832, 39.26778776621388, 39.68465530442674, 40.17241868247171, 40.61480872302413, 41.02600459405043, 41.40884212914386, 41.768992995491026, 42.131979695431475, 42.543175566457755, 42.88063976405865, 43.14720812182741, 43.49885148739472, 43.80795734906276, 44.21348155290248, 44.533930748943654, 44.87423078013782, 45.17482914102601, 45.43856166520149, 45.727816691716534, 46.03692255338457, 46.329013413492895, 46.626775940787795, 46.87916513059014, 47.151405155545476, 47.40946601253439, 47.65618353515016, 47.88872188979951, 48.07872274054959, 48.2630519241131, 48.48708277798259, 48.73380030059836, 48.86141281229618, 49.03439866148654, 49.22439951223663, 49.377534526274005, 49.56469954343079, 49.69798372231517, 49.83410373479285, 50.00425375038993, 50.18574710036015, 50.29634461049825, 50.45798712531548, 50.574256302640165, 50.704704647931266, 50.8464963275955, 50.97410883929331, 51.07903468224485, 51.181124691603095, 51.252020531435214, 51.36261804157331, 51.39664804469274, 51.39948387828602, 51.4420213821853, 51.4959022204577, 51.535603890763696, 51.55545472591668, 51.57530556106968, 51.57814139466297, 51.58664889544282, 51.58664889544282, 51.60366389700253, 51.62067889856223, 51.606499730595814, 51.57814139466297, 51.52993222357713, 51.50724555483084, 51.46470805093158, 51.42217054703231, 51.373961375946465, 51.30590136970763, 51.212318861129226, 51.10455718458441, 50.9967955080396, 50.86918299634178, 50.73589881745739, 50.59977880497972, 50.45798712531548, 50.341717947990816, 50.267986274565416, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.35013871229555, "threshold_area100": 46.55300172985849, "threshold_area25": 36.61373110625868}, "selected_budget": 52, "selected": {"val": {"score": 65.84440227703985, "calls": 51.999437767938716, "rescued": 2210, "harmed": 817}, "test": {"score": 50.49201712843491, "calls": 62.27206987493974, "rescued": 9656, "harmed": 1262}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 65.50003513950382, "calls": 41.998734977862114}, "test": {"score": 49.33783285596801, "calls": 53.10665570144344}}, "old_normalized_interior": {"val": {"score": 65.50003513950382, "calls": 41.998734977862114}, "test": {"score": 49.33783285596801, "calls": 53.10665570144344}}, "old_normalized_endpoints": {"val": {"score": 65.50003513950382, "calls": 41.998734977862114}, "test": {"score": 49.33783285596801, "calls": 53.10665570144344}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 56.76435448731464, 57.34767025089605, 58.04343242673413, 58.668915594911795, 59.05545013704406, 59.57551479373111, 60.130718954248366, 60.59456040480708, 60.995150748471424, 61.34654578677349, 61.7401082296718, 62.02825216107948, 62.288284489423006, 62.569400520064654, 62.815377046876094, 63.15974418441211, 63.33544170356314, 63.49005552041604, 63.76414365029165, 64.00309227633704, 64.18581769625412, 64.36151521540515, 64.49504532995994, 64.69885445217514, 64.76913345983554, 64.89563567362428, 65.01510998664698, 65.05024949047719, 65.05727739124323, 65.18377960503197, 65.19783540656405, 65.21189120809613, 65.29622601728863, 65.24000281116031, 65.2751423149905, 65.24000281116031, 65.26108651345842, 65.37353292571508, 65.38056082648113, 65.4297561318434, 65.47192353643966, 65.50003513950382, 65.48597933797174, 65.40867242954529, 65.42272823107737, 65.40867242954529, 65.3946166280132, 65.45786773490758, 65.4297561318434, 65.37353292571508, 65.359477124183, 65.28919811652258, 65.19783540656405, 65.17675170426594, 65.2048633073301, 65.16269590273384, 65.16269590273384, 65.0853889943074, 65.00808208588094, 64.9518588797526, 64.90266357439033, 64.87455197132616, 64.76913345983554, 64.76210555906951, 64.69885445217514, 64.57235223838639, 64.47396162766181, 64.48098952842786, 64.39665471923536, 64.34745941387308, 64.3193478108089, 64.3193478108089, 64.30529200927683, 64.30529200927683, 64.2842083069787, 64.29826410851078, 64.29123620774475, 64.29123620774475, 64.29123620774475, 64.29123620774475, 64.2842083069787, 64.27718040621266, 64.24204090238246, 64.21392929931828, 64.20690139855225, 64.21392929931828, 64.19284559702017, 64.17176189472204, 64.17176189472204, 64.1295944901258, 64.07337128399746, 64.03120387940122, 63.98903647480497, 64.01012017710309, 64.02417597863518, 63.98200857403893, 64.00309227633704, 64.00309227633704, 64.07337128399746, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 56.76435448731464, 57.34767025089605, 58.04343242673413, 58.668915594911795, 59.05545013704406, 59.57551479373111, 60.130718954248366, 60.59456040480708, 60.995150748471424, 61.34654578677349, 61.7401082296718, 62.02825216107948, 62.288284489423006, 62.569400520064654, 62.815377046876094, 63.15974418441211, 63.33544170356314, 63.49005552041604, 63.76414365029165, 64.00309227633704, 64.18581769625412, 64.36151521540515, 64.49504532995994, 64.69885445217514, 64.76913345983554, 64.89563567362428, 65.01510998664698, 65.05024949047719, 65.05727739124323, 65.18377960503197, 65.19783540656405, 65.21189120809613, 65.29622601728863, 65.24000281116031, 65.2751423149905, 65.24000281116031, 65.26108651345842, 65.37353292571508, 65.38056082648113, 65.4297561318434, 65.47192353643966, 65.50003513950382, 65.48597933797174, 65.40867242954529, 65.42272823107737, 65.40867242954529, 65.3946166280132, 65.45786773490758, 65.4297561318434, 65.37353292571508, 65.359477124183, 65.28919811652258, 65.19783540656405, 65.17675170426594, 65.2048633073301, 65.16269590273384, 65.16269590273384, 65.0853889943074, 65.00808208588094, 64.9518588797526, 64.90266357439033, 64.87455197132616, 64.76913345983554, 64.76210555906951, 64.69885445217514, 64.57235223838639, 64.47396162766181, 64.48098952842786, 64.39665471923536, 64.34745941387308, 64.3193478108089, 64.3193478108089, 64.30529200927683, 64.30529200927683, 64.2842083069787, 64.29826410851078, 64.29123620774475, 64.29123620774475, 64.29123620774475, 64.29123620774475, 64.2842083069787, 64.27718040621266, 64.24204090238246, 64.21392929931828, 64.20690139855225, 64.21392929931828, 64.19284559702017, 64.17176189472204, 64.17176189472204, 64.1295944901258, 64.07337128399746, 64.03120387940122, 63.98903647480497, 64.01012017710309, 64.02417597863518, 63.98200857403893, 64.00309227633704, 64.00309227633704, 64.07337128399746, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 61.52646004638416, "exact_area100": 63.95971651140459}, "test": {"score": [26.688029946402747, 27.995349232907014, 29.053115163202225, 29.912372741967502, 30.924765334770154, 31.761336244789156, 32.86163967898364, 33.8343306014803, 34.75030485211128, 35.43941241527948, 36.07463914017526, 36.845985877548706, 37.51807843915719, 38.16181266483283, 38.79136772254204, 39.36420610838557, 39.877491988770096, 40.473017043359896, 40.901227915945896, 41.44854379944985, 41.8852621728157, 42.4807872274055, 42.95437143748405, 43.40810481240961, 43.873181521708304, 44.230496554462185, 44.5821399200295, 45.0273657941752, 45.48109916910076, 45.89229504012705, 46.24393840569436, 46.55020843376911, 46.78841845560503, 47.11170348523949, 47.50872018829935, 47.752601877321844, 47.96812523041149, 48.27439525848624, 48.48708277798259, 48.72529279981851, 48.96350282165442, 49.15350367240451, 49.33783285596801, 49.513654538751666, 49.63559538326291, 49.81992456682642, 49.984402915236934, 50.112015426934754, 50.22261293707285, 50.32753878002438, 50.42112128860279, 50.50052462921476, 50.62530130731929, 50.704704647931266, 50.80395882369623, 50.874854663528346, 50.883362164308195, 50.96560133851346, 50.99395967444631, 51.02231801037915, 51.09888551739784, 51.10455718458441, 51.14993052207697, 51.141423021297115, 51.115900518957545, 51.1272438533307, 51.135751354110546, 51.16410969004339, 51.16694552363668, 51.16410969004339, 51.16410969004339, 51.15276635567025, 51.15276635567025, 51.15560218926354, 51.15560218926354, 51.16410969004339, 51.15560218926354, 51.15276635567025, 51.14993052207697, 51.14993052207697, 51.1442588548904, 51.1442588548904, 51.138587187703834, 51.132915520517265, 51.135751354110546, 51.121572186144114, 51.121572186144114, 51.09888551739784, 51.09888551739784, 51.06485551427842, 51.073363015058284, 51.04500467912543, 51.042168845532146, 51.050676346311995, 51.047840512718714, 51.01664634319258, 50.9967955080396, 50.937243002580615, 50.840824660408934, 50.639480475285715, 50.00708958398321], "calls": [0.0, 1.562544309899895, 2.9095652667101497, 4.049570371210617, 5.470323001446275, 6.647193942659444, 8.272126591611604, 9.792133397612227, 11.31781187079942, 12.477667810452884, 13.69424042197204, 15.024246377222585, 16.291863993420865, 17.52261577290645, 18.93486090236225, 20.174120182627682, 21.44457363241925, 22.72920625017724, 23.872047188270994, 25.156679806028983, 26.29101324334288, 27.72878087513825, 28.982219323370106, 30.385956952046055, 31.61954456512492, 32.86163967898364, 33.91940560927885, 35.10761988486515, 36.4943425119814, 37.97748348126932, 39.21674276153475, 40.31704619572923, 41.306752119785614, 42.56019056801747, 43.91571902560757, 45.08975413322746, 46.17020673226895, 47.30454016958285, 48.384992768624336, 49.62425204888977, 50.82948132603578, 51.955307262569825, 53.10665570144344, 54.249496639537185, 55.22218756203387, 56.34517766497462, 57.391600260896695, 58.76981538723308, 59.71698380739018, 60.5875847205286, 61.71341065706264, 62.881774097495956, 64.14939171369424, 65.14760513853048, 66.25925190709809, 67.35955534129258, 68.39179876924823, 69.68210305419278, 70.90434733289851, 72.16912911550351, 73.30913422000397, 74.2903326432805, 75.26869523296374, 76.62989535774041, 77.74154212630803, 78.71990471599126, 79.72662564160736, 80.53483821569351, 81.27215494994753, 82.06618835606727, 82.86305759578028, 83.60888183081417, 84.52485608144514, 85.34724782349772, 86.25755040694212, 87.09695715055442, 87.59322802937923, 87.97890139806596, 88.42696310580496, 88.89203981510366, 89.38263902674191, 89.81368573292119, 90.12562742818251, 90.42055412188412, 90.69279414683946, 90.96219833820152, 91.25145336471657, 91.54638005841818, 91.88384425601906, 92.26101012392593, 92.58429515356039, 92.96146102146726, 93.32728355500099, 93.77534526273999, 94.30281031109095, 94.96923120551286, 95.76610044522587, 96.51759634744633, 97.30879391997279, 98.30984317840229, 100.0], "exact_score": [26.688029946402747, 27.521765022828465, 28.335649264101185, 29.106996001474634, 29.869835238068234, 30.578793636389417, 31.299095369083744, 32.01939710177807, 32.70283299775969, 33.31253722031591, 33.97895811473783, 34.585826503700765, 35.13314238720472, 35.72583160820123, 36.23911748858577, 36.828970875989, 37.3677792587131, 37.89808014065735, 38.38584351870232, 38.81972605847489, 39.2649519326206, 39.698834472393166, 40.12420951138588, 40.58645038709129, 40.98346709015115, 41.39182712758416, 41.75481382752461, 42.194368034483745, 42.57720556957718, 42.962878938263906, 43.28616396789837, 43.666165669398524, 43.9866148654397, 44.29004905992117, 44.599154921589204, 44.97348495590279, 45.31945665428353, 45.65408501829113, 45.89229504012705, 46.18155006664209, 46.47364092675042, 46.70901511499306, 46.97274763916853, 47.23080849615745, 47.52289935626578, 47.72707937498228, 47.93976689447863, 48.217678586620536, 48.43036610611689, 48.58917278734084, 48.80469614043048, 49.0060403255537, 49.15917533959108, 49.315146187221735, 49.482460369225535, 49.59589371295693, 49.77171539574058, 49.916342908998104, 50.046791254289204, 50.1403737628676, 50.25947877378556, 50.36724045033038, 50.446643790942346, 50.50052462921476, 50.59977880497972, 50.68768964637156, 50.78127215494995, 50.84366049400222, 50.883362164308195, 50.93157133539405, 50.9570938377336, 50.9967955080396, 51.01948217678587, 51.09321385021127, 51.10455718458441, 51.132915520517265, 51.1442588548904, 51.135751354110546, 51.121572186144114, 51.1272438533307, 51.141423021297115, 51.16978135722996, 51.15843802285682, 51.16978135722996, 51.16410969004339, 51.15843802285682, 51.15560218926354, 51.16127385645011, 51.15276635567025, 51.14993052207697, 51.141423021297115, 51.11873635255083, 51.08187051583813, 51.03933301193886, 51.0676913478717, 51.01664634319258, 50.95425800414032, 50.89470549868133, 50.707540481524546, 50.446643790942346, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 36.90894138331963, "exact_area100": 45.46714682439526}}, "old_normalized_endpoint_budget": 42}, {"seed": 2030, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.0, "validation_area100": 0.6404278023319658, "training_seconds": 90.17630370601546, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9101061224937439, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8768064975738525, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8453895449638367, "operator": "ge"}, {"budget": 0.04, "threshold": 0.8102530837059021, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7792243361473083, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7572708129882812, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7316238880157471, "operator": "ge"}, {"budget": 0.08, "threshold": 0.7067124843597412, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6833227276802063, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6604529619216919, "operator": "ge"}, {"budget": 0.11, "threshold": 0.6408420205116272, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6203196048736572, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6046263575553894, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5863299369812012, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5694336891174316, "operator": "ge"}, {"budget": 0.16, "threshold": 0.551995038986206, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5355014204978943, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5227124691009521, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5071903467178345, "operator": "ge"}, {"budget": 0.2, "threshold": 0.49114006757736206, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4774337112903595, "operator": "ge"}, {"budget": 0.22, "threshold": 0.4632410705089569, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4498099386692047, "operator": "ge"}, {"budget": 0.24, "threshold": 0.43469327688217163, "operator": "ge"}, {"budget": 0.25, "threshold": 0.4217132329940796, "operator": "ge"}, {"budget": 0.26, "threshold": 0.40962889790534973, "operator": "ge"}, {"budget": 0.27, "threshold": 0.3970445692539215, "operator": "ge"}, {"budget": 0.28, "threshold": 0.3821355104446411, "operator": "ge"}, {"budget": 0.29, "threshold": 0.3706032335758209, "operator": "ge"}, {"budget": 0.3, "threshold": 0.35607823729515076, "operator": "ge"}, {"budget": 0.31, "threshold": 0.34359925985336304, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3308469355106354, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3206399977207184, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3114170432090759, "operator": "ge"}, {"budget": 0.35, "threshold": 0.2999051809310913, "operator": "ge"}, {"budget": 0.36, "threshold": 0.2886940836906433, "operator": "ge"}, {"budget": 0.37, "threshold": 0.2773391604423523, "operator": "ge"}, {"budget": 0.38, "threshold": 0.26802846789360046, "operator": "ge"}, {"budget": 0.39, "threshold": 0.2566549479961395, "operator": "ge"}, {"budget": 0.4, "threshold": 0.24727515876293182, "operator": "ge"}, {"budget": 0.41, "threshold": 0.23630569875240326, "operator": "ge"}, {"budget": 0.42, "threshold": 0.2245061993598938, "operator": "ge"}, {"budget": 0.43, "threshold": 0.2141895443201065, "operator": "gt"}, {"budget": 0.44, "threshold": 0.2038193792104721, "operator": "ge"}, {"budget": 0.45, "threshold": 0.19331799447536469, "operator": "ge"}, {"budget": 0.46, "threshold": 0.1839086264371872, "operator": "ge"}, {"budget": 0.47, "threshold": 0.17440250515937805, "operator": "ge"}, {"budget": 0.48, "threshold": 0.16629476845264435, "operator": "ge"}, {"budget": 0.49, "threshold": 0.15670105814933777, "operator": "ge"}, {"budget": 0.5, "threshold": 0.14667880535125732, "operator": "ge"}, {"budget": 0.51, "threshold": 0.13787683844566345, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1294369399547577, "operator": "ge"}, {"budget": 0.53, "threshold": 0.12073683738708496, "operator": "ge"}, {"budget": 0.54, "threshold": 0.1117808148264885, "operator": "ge"}, {"budget": 0.55, "threshold": 0.1029859110713005, "operator": "gt"}, {"budget": 0.56, "threshold": 0.09374870359897614, "operator": "ge"}, {"budget": 0.57, "threshold": 0.08503615856170654, "operator": "ge"}, {"budget": 0.58, "threshold": 0.07604064047336578, "operator": "ge"}, {"budget": 0.59, "threshold": 0.06758139282464981, "operator": "ge"}, {"budget": 0.6, "threshold": 0.059407442808151245, "operator": "ge"}, {"budget": 0.61, "threshold": 0.05251051485538483, "operator": "ge"}, {"budget": 0.62, "threshold": 0.04490393027663231, "operator": "ge"}, {"budget": 0.63, "threshold": 0.03636697679758072, "operator": "ge"}, {"budget": 0.64, "threshold": 0.02884818986058235, "operator": "ge"}, {"budget": 0.65, "threshold": 0.02184281498193741, "operator": "ge"}, {"budget": 0.66, "threshold": 0.013659508898854256, "operator": "ge"}, {"budget": 0.67, "threshold": 0.004797672852873802, "operator": "ge"}, {"budget": 0.68, "threshold": 0.00038292896351777017, "operator": "ge"}, {"budget": 0.69, "threshold": -7.373245898634195e-05, "operator": "ge"}, {"budget": 0.7, "threshold": -0.00024335907073691487, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0005025751888751984, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0007893525762483478, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0011641879100352526, "operator": "ge"}, {"budget": 0.74, "threshold": -0.0015953246038407087, "operator": "ge"}, {"budget": 0.75, "threshold": -0.002122176345437765, "operator": "ge"}, {"budget": 0.76, "threshold": -0.002742515178397298, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0034732825588434935, "operator": "ge"}, {"budget": 0.78, "threshold": -0.004307878203690052, "operator": "ge"}, {"budget": 0.79, "threshold": -0.005403398536145687, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0064170327968895435, "operator": "ge"}, {"budget": 0.81, "threshold": -0.007836108095943928, "operator": "ge"}, {"budget": 0.82, "threshold": -0.009506202302873135, "operator": "ge"}, {"budget": 0.83, "threshold": -0.011201205663383007, "operator": "ge"}, {"budget": 0.84, "threshold": -0.013650274835526943, "operator": "ge"}, {"budget": 0.85, "threshold": -0.016405871137976646, "operator": "ge"}, {"budget": 0.86, "threshold": -0.01934446394443512, "operator": "ge"}, {"budget": 0.87, "threshold": -0.02308371476829052, "operator": "ge"}, {"budget": 0.88, "threshold": -0.028899511322379112, "operator": "ge"}, {"budget": 0.89, "threshold": -0.0344819575548172, "operator": "ge"}, {"budget": 0.9, "threshold": -0.04252246767282486, "operator": "ge"}, {"budget": 0.91, "threshold": -0.05138218402862549, "operator": "ge"}, {"budget": 0.92, "threshold": -0.06305389851331711, "operator": "ge"}, {"budget": 0.93, "threshold": -0.0794142484664917, "operator": "ge"}, {"budget": 0.94, "threshold": -0.09667174518108368, "operator": "ge"}, {"budget": 0.95, "threshold": -0.12060923874378204, "operator": "ge"}, {"budget": 0.96, "threshold": -0.14959388971328735, "operator": "ge"}, {"budget": 0.97, "threshold": -0.19123508036136627, "operator": "ge"}, {"budget": 0.98, "threshold": -0.25734150409698486, "operator": "ge"}, {"budget": 0.99, "threshold": -0.37200266122817993, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.68001967812214, 57.29847494553376, 57.90990231217935, 58.451050671164516, 58.900836320191154, 59.413873076112154, 59.80743551901047, 60.14477475578044, 60.531309297912706, 60.80539742778831, 61.26923887834703, 61.67685712277742, 61.91580574882283, 62.06339166490969, 62.33747979478529, 62.611567924660896, 62.82240494764213, 63.075409375219614, 63.34246960432919, 63.39869281045751, 63.49708342118209, 63.58141823037459, 63.70792044416332, 63.85550636025019, 64.09445498629559, 64.3193478108089, 64.38962681846931, 64.47396162766181, 64.58640803991848, 64.71993815447325, 64.75507765830346, 64.67777074987701, 64.67074284911097, 64.81832876519783, 64.92374727668845, 65.06430529200927, 65.09241689507344, 65.16269590273384, 65.21891910886217, 65.2751423149905, 65.35244922341697, 65.36650502494903, 65.40867242954529, 65.42272823107737, 65.4297561318434, 65.49300723873779, 65.49300723873779, 65.52111884180195, 65.45786773490758, 65.45083983414153, 65.47192353643966, 65.4438119333755, 65.41570033031134, 65.359477124183, 65.4297561318434, 65.46489563567361, 65.41570033031134, 65.38758872724716, 65.40164452877924, 65.42272823107737, 65.35244922341697, 65.34542132265092, 65.33839342188487, 65.34542132265092, 65.28919811652258, 65.24703071192634, 65.1556680019678, 65.11350059737156, 65.1205284981376, 65.1205284981376, 65.11350059737156, 65.09944479583947, 65.09241689507344, 65.0853889943074, 65.05024949047719, 65.04322158971115, 65.04322158971115, 65.05024949047719, 65.04322158971115, 65.0361936889451, 65.00105418511491, 64.9659146812847, 64.95888678051865, 64.91671937592241, 64.88860777285824, 64.86049616979408, 64.83941246749595, 64.76210555906951, 64.7269660552393, 64.66371494834493, 64.52315693302411, 64.4880174291939, 64.39665471923536, 64.32637571157494, 64.2842083069787, 64.22095720008433, 64.15067819242391, 64.12256658935975, 64.03120387940122, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.98966898587392, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.98629559350622, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 100, 197, 294, 384, 462, 545, 611, 672, 738, 797, 871, 939, 989, 1034, 1090, 1143, 1191, 1246, 1293, 1327, 1360, 1399, 1430, 1472, 1522, 1567, 1599, 1634, 1670, 1707, 1734, 1752, 1777, 1810, 1839, 1880, 1904, 1928, 1957, 1982, 2011, 2037, 2058, 2080, 2099, 2119, 2145, 2163, 2181, 2201, 2220, 2239, 2256, 2268, 2290, 2310, 2323, 2335, 2349, 2367, 2376, 2390, 2403, 2416, 2420, 2429, 2434, 2435, 2436, 2436, 2436, 2436, 2436, 2436, 2436, 2436, 2437, 2439, 2439, 2439, 2439, 2440, 2441, 2442, 2442, 2444, 2448, 2452, 2455, 2459, 2460, 2467, 2476, 2483, 2491, 2498, 2515, 2536, 2556, 2589], "harmed": [0, 11, 20, 30, 43, 57, 67, 77, 90, 101, 121, 129, 139, 155, 179, 196, 210, 228, 247, 256, 282, 301, 328, 341, 362, 378, 391, 413, 436, 456, 474, 496, 525, 551, 563, 577, 598, 618, 632, 653, 670, 688, 712, 727, 747, 765, 776, 802, 816, 843, 864, 880, 903, 924, 944, 956, 971, 991, 1007, 1019, 1034, 1053, 1068, 1082, 1094, 1106, 1121, 1139, 1146, 1146, 1146, 1147, 1149, 1150, 1151, 1156, 1157, 1158, 1159, 1160, 1161, 1166, 1172, 1174, 1181, 1185, 1191, 1198, 1213, 1221, 1234, 1255, 1267, 1289, 1306, 1320, 1336, 1363, 1388, 1421, 1441], "exact_score": [56.05453650994448, 56.68001967812214, 57.29847494553376, 57.90990231217935, 58.451050671164516, 58.900836320191154, 59.413873076112154, 59.80743551901047, 60.14477475578044, 60.531309297912706, 60.80539742778831, 61.26923887834703, 61.67685712277742, 61.91580574882283, 62.06339166490969, 62.33747979478529, 62.611567924660896, 62.82240494764213, 63.075409375219614, 63.34246960432919, 63.39869281045751, 63.49708342118209, 63.58141823037459, 63.70792044416332, 63.85550636025019, 64.09445498629559, 64.3193478108089, 64.38962681846931, 64.47396162766181, 64.58640803991848, 64.71993815447325, 64.75507765830346, 64.67777074987701, 64.67074284911097, 64.81832876519783, 64.92374727668845, 65.06430529200927, 65.09241689507344, 65.16269590273384, 65.21891910886217, 65.2751423149905, 65.35244922341697, 65.36650502494903, 65.40867242954529, 65.42272823107737, 65.4297561318434, 65.49300723873779, 65.49300723873779, 65.52111884180195, 65.45786773490758, 65.45083983414153, 65.47192353643966, 65.4438119333755, 65.41570033031134, 65.359477124183, 65.4297561318434, 65.46489563567361, 65.41570033031134, 65.38758872724716, 65.40164452877924, 65.42272823107737, 65.35244922341697, 65.34542132265092, 65.33839342188487, 65.34542132265092, 65.28919811652258, 65.24703071192634, 65.1556680019678, 65.11350059737156, 65.1205284981376, 65.1205284981376, 65.11350059737156, 65.09944479583947, 65.09241689507344, 65.0853889943074, 65.05024949047719, 65.04322158971115, 65.04322158971115, 65.05024949047719, 65.04322158971115, 65.0361936889451, 65.00105418511491, 64.9659146812847, 64.95888678051865, 64.91671937592241, 64.88860777285824, 64.86049616979408, 64.83941246749595, 64.76210555906951, 64.7269660552393, 64.66371494834493, 64.52315693302411, 64.4880174291939, 64.39665471923536, 64.32637571157494, 64.2842083069787, 64.22095720008433, 64.15067819242391, 64.12256658935975, 64.03120387940122, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.04278023319658, "threshold_area100": 64.04251879963456, "threshold_area25": 61.10338042026846}, "test": {"score": [26.688029946402747, 28.65042679295579, 29.736551059183846, 30.768794487139495, 32.00521793381165, 33.16223803987182, 33.97045061395797, 34.91194736692851, 35.850608286305764, 36.67016419476505, 37.37912259308624, 37.97748348126932, 38.85375606159431, 39.42092278025125, 40.08734367467317, 40.7140628987891, 41.241527947140064, 41.84556050250971, 42.276607208689, 42.74451975158098, 43.28616396789837, 43.70303150611122, 44.21348155290248, 44.62751325752205, 44.99900745824235, 45.27975498397754, 45.60304001361201, 45.94050421121288, 46.26662507444064, 46.57856676970196, 46.87065762981029, 47.196778493038025, 47.51155602189264, 47.76394521169498, 47.959617729631624, 48.26872359129967, 48.50126194594901, 48.75081530215807, 48.92380115134844, 49.18753367552392, 49.4144003629867, 49.578878711397216, 49.749028726994304, 49.941865411337666, 50.13186626208774, 50.2708221081587, 50.37007628392366, 50.48918129484162, 50.55156963389389, 50.710376315117834, 50.832317159629085, 50.89754133227462, 50.97127300570003, 51.03366134475229, 51.050676346311995, 51.10455718458441, 51.17545302441653, 51.21515469472251, 51.26903553299492, 51.311573036894195, 51.39948387828602, 51.439185548592015, 51.430678047812165, 51.416498879845726, 51.40515554547259, 51.373961375946465, 51.37112554235318, 51.382468876726314, 51.35978220798003, 51.35978220798003, 51.35978220798003, 51.354110540793464, 51.348438873606895, 51.348438873606895, 51.348438873606895, 51.342767206420326, 51.33993137282704, 51.3285880384539, 51.317244704080764, 51.30590136970763, 51.29455803533449, 51.28321470096135, 51.26903553299492, 51.26619969940164, 51.26052803221507, 51.23500552987551, 51.23500552987551, 51.237841363468796, 51.243513030655365, 51.20664719394266, 51.18963219238295, 51.16127385645011, 51.16410969004339, 51.14993052207697, 51.14709468848368, 51.107393018177696, 51.042168845532146, 50.97978050647989, 50.89754133227462, 50.66783881121856, 50.00708958398321], "calls": [0.0, 2.413294387885319, 3.896435357173241, 5.331367155375323, 7.0527181464991635, 8.751382468876727, 9.993477582735446, 11.479454385616652, 13.095879533788956, 14.607378839009726, 16.03947480361852, 17.36948075886907, 18.98874174063466, 20.19113518418739, 21.6771119870686, 23.120551286050535, 24.529960581913056, 25.911011541842726, 26.957434137764796, 28.22788758755636, 29.61177438107932, 30.81416782463205, 32.07611377364376, 33.156566372685255, 34.46388565918952, 35.53299492385787, 36.57941751977994, 37.753452627399824, 38.947338570172704, 39.99376116609477, 41.29257295181919, 42.472279726625636, 43.632135666279105, 44.72393159969373, 45.64557751751127, 46.81961262513116, 47.97096106400476, 49.16484700677764, 50.07514959022205, 51.288886368147914, 52.21053228596546, 53.31083572015994, 54.5274083316791, 55.68159260414599, 56.76204520318747, 57.92473697643422, 58.914442900490606, 59.994895499532085, 60.86266057907722, 61.93744151093213, 63.09446161699231, 64.19476505118679, 65.11073930181777, 66.04089272041517, 67.1497036553895, 68.2074695856847, 69.45523636673, 70.54136063295806, 71.84867991946233, 72.93764001928366, 74.14854096361626, 75.11839605251964, 76.22720698749397, 77.45795876697954, 78.50438136290163, 79.38632561041318, 80.22573235402547, 80.94886992031307, 81.68618665456711, 84.87933528060574, 86.21217706944957, 87.26427133255821, 87.79740804809573, 88.2312905878683, 88.57442645265576, 88.89487564869694, 89.24368318067096, 89.552789042339, 89.78816323058163, 90.07458242350339, 90.2759266086266, 90.55100246717522, 90.83174999291042, 91.04443751240677, 91.31100587017555, 91.58891756231743, 91.86399342086607, 92.10787510988855, 92.42265263874316, 92.72041516603807, 93.15146187221734, 93.50310523778464, 93.84907693616539, 94.30281031109095, 94.7593795196098, 95.28968040155404, 95.80863794912514, 96.46938717636048, 97.26342058248022, 98.26730567450302, 100.0], "rescued": [0, 727, 1133, 1516, 1978, 2408, 2711, 3059, 3414, 3727, 4006, 4259, 4589, 4812, 5075, 5326, 5551, 5788, 5965, 6167, 6383, 6562, 6768, 6936, 7097, 7219, 7365, 7514, 7655, 7788, 7916, 8057, 8194, 8310, 8400, 8526, 8637, 8753, 8838, 8963, 9063, 9154, 9250, 9348, 9438, 9510, 9575, 9641, 9688, 9774, 9843, 9901, 9948, 9992, 10030, 10072, 10136, 10175, 10225, 10262, 10313, 10345, 10370, 10390, 10407, 10417, 10429, 10443, 10453, 10455, 10455, 10456, 10457, 10458, 10459, 10460, 10462, 10462, 10462, 10462, 10463, 10465, 10467, 10471, 10474, 10476, 10482, 10488, 10498, 10498, 10511, 10521, 10535, 10554, 10572, 10592, 10605, 10627, 10655, 10685, 10723], "harmed": [0, 35, 58, 77, 103, 125, 143, 159, 183, 207, 236, 278, 299, 322, 350, 380, 419, 443, 468, 505, 530, 562, 588, 610, 640, 663, 695, 725, 751, 774, 799, 825, 851, 878, 899, 916, 945, 973, 997, 1029, 1049, 1082, 1118, 1148, 1171, 1194, 1224, 1248, 1273, 1303, 1329, 1364, 1385, 1407, 1439, 1462, 1501, 1526, 1557, 1579, 1599, 1617, 1645, 1670, 1691, 1712, 1725, 1735, 1753, 1755, 1755, 1758, 1761, 1762, 1763, 1766, 1769, 1773, 1777, 1781, 1786, 1792, 1799, 1804, 1809, 1820, 1826, 1831, 1839, 1852, 1871, 1891, 1904, 1928, 1947, 1981, 2017, 2061, 2118, 2229, 2500], "exact_score": [26.688029946402747, 27.541615857981455, 28.329977596914613, 29.061622663982078, 29.81879023338911, 30.539091966083433, 31.25939369877776, 31.957008762725803, 32.657459660267136, 33.31253722031591, 33.97328644755126, 34.61985650682018, 35.23523239656297, 35.8052349488132, 36.33269999716417, 36.880015880668125, 37.35643592433996, 37.81584096645209, 38.32912684683663, 38.86226356237416, 39.31316110370644, 39.778237813005134, 40.25182202308369, 40.65167455973684, 41.02033292686386, 41.44570796585656, 41.87959050562913, 42.28511470946885, 42.65944474378244, 43.064968947622155, 43.38258231007005, 43.79377818109633, 44.17377988259649, 44.58497575362278, 44.84303661061169, 45.126619969940165, 45.430054164421634, 45.72214502452996, 46.03408671979128, 46.27513257522049, 46.57856676970196, 46.825284292317725, 47.06065848056036, 47.3328985055157, 47.61081019765761, 47.820661883560675, 48.03618523665032, 48.30558942801236, 48.51260528032215, 48.711113631852086, 48.90395031619544, 49.13365283725151, 49.349176190341154, 49.53917704109123, 49.669625386382336, 49.80858123245328, 49.99291041601679, 50.16306043161387, 50.27932960893855, 50.37574795111023, 50.48918129484162, 50.582763803420015, 50.72739131667754, 50.823809658849214, 50.883362164308195, 50.95142217054703, 51.03082551115901, 51.056348013498564, 51.08187051583813, 51.15560218926354, 51.181124691603095, 51.22933386268894, 51.2747072001815, 51.32008053767405, 51.39097637750617, 51.430678047812165, 51.439185548592015, 51.4420213821853, 51.39664804469274, 51.382468876726314, 51.36545387516661, 51.382468876726314, 51.356946374386744, 51.356946374386744, 51.36261804157331, 51.35978220798003, 51.35978220798003, 51.354110540793464, 51.348438873606895, 51.34560304001361, 51.31440887048748, 51.26903553299492, 51.23500552987551, 51.1981396931628, 51.16978135722996, 51.121572186144114, 51.03366134475229, 50.945750503360465, 50.72739131667754, 50.50619629640133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.60176632922751, "threshold_area100": 47.30486629044608, "threshold_area25": 38.10509599296714}, "selected_budget": 48, "selected": {"val": {"score": 65.52111884180195, "calls": 47.99353433129524, "rescued": 2163, "harmed": 816}, "test": {"score": 50.55156963389389, "calls": 60.86266057907722, "rescued": 9688, "harmed": 1273}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9101061224937439, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8768064975738525, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8453895449638367, "operator": "ge"}, {"budget": 0.04, "threshold": 0.8102530837059021, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7792243361473083, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7572708129882812, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7316238880157471, "operator": "ge"}, {"budget": 0.08, "threshold": 0.7067124843597412, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6833227276802063, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6604529619216919, "operator": "ge"}, {"budget": 0.11, "threshold": 0.6408420205116272, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6203196048736572, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6046263575553894, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5863299369812012, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5694336891174316, "operator": "ge"}, {"budget": 0.16, "threshold": 0.551995038986206, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5355014204978943, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5227124691009521, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5071903467178345, "operator": "ge"}, {"budget": 0.2, "threshold": 0.49114006757736206, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4774337112903595, "operator": "ge"}, {"budget": 0.22, "threshold": 0.4632410705089569, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4498099386692047, "operator": "ge"}, {"budget": 0.24, "threshold": 0.43469327688217163, "operator": "ge"}, {"budget": 0.25, "threshold": 0.4217132329940796, "operator": "ge"}, {"budget": 0.26, "threshold": 0.40962889790534973, "operator": "ge"}, {"budget": 0.27, "threshold": 0.3970445692539215, "operator": "ge"}, {"budget": 0.28, "threshold": 0.3821355104446411, "operator": "ge"}, {"budget": 0.29, "threshold": 0.3706032335758209, "operator": "ge"}, {"budget": 0.3, "threshold": 0.35607823729515076, "operator": "ge"}, {"budget": 0.31, "threshold": 0.34359925985336304, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3308469355106354, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3206399977207184, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3114170432090759, "operator": "ge"}, {"budget": 0.35, "threshold": 0.2999051809310913, "operator": "ge"}, {"budget": 0.36, "threshold": 0.2886940836906433, "operator": "ge"}, {"budget": 0.37, "threshold": 0.2773391604423523, "operator": "ge"}, {"budget": 0.38, "threshold": 0.26802846789360046, "operator": "ge"}, {"budget": 0.39, "threshold": 0.2566549479961395, "operator": "ge"}, {"budget": 0.4, "threshold": 0.24727515876293182, "operator": "ge"}, {"budget": 0.41, "threshold": 0.23630569875240326, "operator": "ge"}, {"budget": 0.42, "threshold": 0.2245061993598938, "operator": "ge"}, {"budget": 0.43, "threshold": 0.2141895443201065, "operator": "gt"}, {"budget": 0.44, "threshold": 0.2038193792104721, "operator": "ge"}, {"budget": 0.45, "threshold": 0.19331799447536469, "operator": "ge"}, {"budget": 0.46, "threshold": 0.1839086264371872, "operator": "ge"}, {"budget": 0.47, "threshold": 0.17440250515937805, "operator": "ge"}, {"budget": 0.48, "threshold": 0.16629476845264435, "operator": "ge"}, {"budget": 0.49, "threshold": 0.15670105814933777, "operator": "ge"}, {"budget": 0.5, "threshold": 0.14667880535125732, "operator": "ge"}, {"budget": 0.51, "threshold": 0.13787683844566345, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1294369399547577, "operator": "ge"}, {"budget": 0.53, "threshold": 0.12073683738708496, "operator": "ge"}, {"budget": 0.54, "threshold": 0.1117808148264885, "operator": "ge"}, {"budget": 0.55, "threshold": 0.1029859110713005, "operator": "gt"}, {"budget": 0.56, "threshold": 0.09374870359897614, "operator": "ge"}, {"budget": 0.57, "threshold": 0.08503615856170654, "operator": "ge"}, {"budget": 0.58, "threshold": 0.07604064047336578, "operator": "ge"}, {"budget": 0.59, "threshold": 0.06758139282464981, "operator": "ge"}, {"budget": 0.6, "threshold": 0.059407442808151245, "operator": "ge"}, {"budget": 0.61, "threshold": 0.05251051485538483, "operator": "ge"}, {"budget": 0.62, "threshold": 0.04490393027663231, "operator": "ge"}, {"budget": 0.63, "threshold": 0.03636697679758072, "operator": "ge"}, {"budget": 0.64, "threshold": 0.02884818986058235, "operator": "ge"}, {"budget": 0.65, "threshold": 0.02184281498193741, "operator": "ge"}, {"budget": 0.66, "threshold": 0.013659508898854256, "operator": "ge"}, {"budget": 0.67, "threshold": 0.004797672852873802, "operator": "ge"}, {"budget": 0.68, "threshold": 0.00038292896351777017, "operator": "ge"}, {"budget": 0.69, "threshold": -7.373245898634195e-05, "operator": "ge"}, {"budget": 0.7, "threshold": -0.00024335907073691487, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0005025751888751984, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0007893525762483478, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0011641879100352526, "operator": "ge"}, {"budget": 0.74, "threshold": -0.0015953246038407087, "operator": "ge"}, {"budget": 0.75, "threshold": -0.002122176345437765, "operator": "ge"}, {"budget": 0.76, "threshold": -0.002742515178397298, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0034732825588434935, "operator": "ge"}, {"budget": 0.78, "threshold": -0.004307878203690052, "operator": "ge"}, {"budget": 0.79, "threshold": -0.005403398536145687, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0064170327968895435, "operator": "ge"}, {"budget": 0.81, "threshold": -0.007836108095943928, "operator": "ge"}, {"budget": 0.82, "threshold": -0.009506202302873135, "operator": "ge"}, {"budget": 0.83, "threshold": -0.011201205663383007, "operator": "ge"}, {"budget": 0.84, "threshold": -0.013650274835526943, "operator": "ge"}, {"budget": 0.85, "threshold": -0.016405871137976646, "operator": "ge"}, {"budget": 0.86, "threshold": -0.01934446394443512, "operator": "ge"}, {"budget": 0.87, "threshold": -0.02308371476829052, "operator": "ge"}, {"budget": 0.88, "threshold": -0.028899511322379112, "operator": "ge"}, {"budget": 0.89, "threshold": -0.0344819575548172, "operator": "ge"}, {"budget": 0.9, "threshold": -0.04252246767282486, "operator": "ge"}, {"budget": 0.91, "threshold": -0.05138218402862549, "operator": "ge"}, {"budget": 0.92, "threshold": -0.06305389851331711, "operator": "ge"}, {"budget": 0.93, "threshold": -0.0794142484664917, "operator": "ge"}, {"budget": 0.94, "threshold": -0.09667174518108368, "operator": "ge"}, {"budget": 0.95, "threshold": -0.12060923874378204, "operator": "ge"}, {"budget": 0.96, "threshold": -0.14959388971328735, "operator": "ge"}, {"budget": 0.97, "threshold": -0.19123508036136627, "operator": "ge"}, {"budget": 0.98, "threshold": -0.25734150409698486, "operator": "ge"}, {"budget": 0.99, "threshold": -0.37200266122817993, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.68001967812214, 57.29847494553376, 57.90990231217935, 58.451050671164516, 58.900836320191154, 59.413873076112154, 59.80743551901047, 60.14477475578044, 60.531309297912706, 60.80539742778831, 61.26923887834703, 61.67685712277742, 61.91580574882283, 62.06339166490969, 62.33747979478529, 62.611567924660896, 62.82240494764213, 63.075409375219614, 63.34246960432919, 63.39869281045751, 63.49708342118209, 63.58141823037459, 63.70792044416332, 63.85550636025019, 64.09445498629559, 64.3193478108089, 64.38962681846931, 64.47396162766181, 64.58640803991848, 64.71993815447325, 64.75507765830346, 64.67777074987701, 64.67074284911097, 64.81832876519783, 64.92374727668845, 65.06430529200927, 65.09241689507344, 65.16269590273384, 65.21891910886217, 65.2751423149905, 65.35244922341697, 65.36650502494903, 65.40867242954529, 65.42272823107737, 65.4297561318434, 65.49300723873779, 65.49300723873779, 65.52111884180195, 65.45786773490758, 65.45083983414153, 65.47192353643966, 65.4438119333755, 65.41570033031134, 65.359477124183, 65.4297561318434, 65.46489563567361, 65.41570033031134, 65.38758872724716, 65.40164452877924, 65.42272823107737, 65.35244922341697, 65.34542132265092, 65.33839342188487, 65.34542132265092, 65.28919811652258, 65.24703071192634, 65.1556680019678, 65.11350059737156, 65.1205284981376, 65.1205284981376, 65.11350059737156, 65.09944479583947, 65.09241689507344, 65.0853889943074, 65.05024949047719, 65.04322158971115, 65.04322158971115, 65.05024949047719, 65.04322158971115, 65.0361936889451, 65.00105418511491, 64.9659146812847, 64.95888678051865, 64.91671937592241, 64.88860777285824, 64.86049616979408, 64.83941246749595, 64.76210555906951, 64.7269660552393, 64.66371494834493, 64.52315693302411, 64.4880174291939, 64.39665471923536, 64.32637571157494, 64.2842083069787, 64.22095720008433, 64.15067819242391, 64.12256658935975, 64.03120387940122, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.98966898587392, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.98629559350622, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 100, 197, 294, 384, 462, 545, 611, 672, 738, 797, 871, 939, 989, 1034, 1090, 1143, 1191, 1246, 1293, 1327, 1360, 1399, 1430, 1472, 1522, 1567, 1599, 1634, 1670, 1707, 1734, 1752, 1777, 1810, 1839, 1880, 1904, 1928, 1957, 1982, 2011, 2037, 2058, 2080, 2099, 2119, 2145, 2163, 2181, 2201, 2220, 2239, 2256, 2268, 2290, 2310, 2323, 2335, 2349, 2367, 2376, 2390, 2403, 2416, 2420, 2429, 2434, 2435, 2436, 2436, 2436, 2436, 2436, 2436, 2436, 2436, 2437, 2439, 2439, 2439, 2439, 2440, 2441, 2442, 2442, 2444, 2448, 2452, 2455, 2459, 2460, 2467, 2476, 2483, 2491, 2498, 2515, 2536, 2556, 2589], "harmed": [0, 11, 20, 30, 43, 57, 67, 77, 90, 101, 121, 129, 139, 155, 179, 196, 210, 228, 247, 256, 282, 301, 328, 341, 362, 378, 391, 413, 436, 456, 474, 496, 525, 551, 563, 577, 598, 618, 632, 653, 670, 688, 712, 727, 747, 765, 776, 802, 816, 843, 864, 880, 903, 924, 944, 956, 971, 991, 1007, 1019, 1034, 1053, 1068, 1082, 1094, 1106, 1121, 1139, 1146, 1146, 1146, 1147, 1149, 1150, 1151, 1156, 1157, 1158, 1159, 1160, 1161, 1166, 1172, 1174, 1181, 1185, 1191, 1198, 1213, 1221, 1234, 1255, 1267, 1289, 1306, 1320, 1336, 1363, 1388, 1421, 1441], "exact_score": [56.05453650994448, 56.68001967812214, 57.29847494553376, 57.90990231217935, 58.451050671164516, 58.900836320191154, 59.413873076112154, 59.80743551901047, 60.14477475578044, 60.531309297912706, 60.80539742778831, 61.26923887834703, 61.67685712277742, 61.91580574882283, 62.06339166490969, 62.33747979478529, 62.611567924660896, 62.82240494764213, 63.075409375219614, 63.34246960432919, 63.39869281045751, 63.49708342118209, 63.58141823037459, 63.70792044416332, 63.85550636025019, 64.09445498629559, 64.3193478108089, 64.38962681846931, 64.47396162766181, 64.58640803991848, 64.71993815447325, 64.75507765830346, 64.67777074987701, 64.67074284911097, 64.81832876519783, 64.92374727668845, 65.06430529200927, 65.09241689507344, 65.16269590273384, 65.21891910886217, 65.2751423149905, 65.35244922341697, 65.36650502494903, 65.40867242954529, 65.42272823107737, 65.4297561318434, 65.49300723873779, 65.49300723873779, 65.52111884180195, 65.45786773490758, 65.45083983414153, 65.47192353643966, 65.4438119333755, 65.41570033031134, 65.359477124183, 65.4297561318434, 65.46489563567361, 65.41570033031134, 65.38758872724716, 65.40164452877924, 65.42272823107737, 65.35244922341697, 65.34542132265092, 65.33839342188487, 65.34542132265092, 65.28919811652258, 65.24703071192634, 65.1556680019678, 65.11350059737156, 65.1205284981376, 65.1205284981376, 65.11350059737156, 65.09944479583947, 65.09241689507344, 65.0853889943074, 65.05024949047719, 65.04322158971115, 65.04322158971115, 65.05024949047719, 65.04322158971115, 65.0361936889451, 65.00105418511491, 64.9659146812847, 64.95888678051865, 64.91671937592241, 64.88860777285824, 64.86049616979408, 64.83941246749595, 64.76210555906951, 64.7269660552393, 64.66371494834493, 64.52315693302411, 64.4880174291939, 64.39665471923536, 64.32637571157494, 64.2842083069787, 64.22095720008433, 64.15067819242391, 64.12256658935975, 64.03120387940122, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.04278023319658, "threshold_area100": 64.04251879963456, "threshold_area25": 61.10338042026846}, "test": {"score": [26.688029946402747, 28.65042679295579, 29.736551059183846, 30.768794487139495, 32.00521793381165, 33.16223803987182, 33.97045061395797, 34.91194736692851, 35.850608286305764, 36.67016419476505, 37.37912259308624, 37.97748348126932, 38.85375606159431, 39.42092278025125, 40.08734367467317, 40.7140628987891, 41.241527947140064, 41.84556050250971, 42.276607208689, 42.74451975158098, 43.28616396789837, 43.70303150611122, 44.21348155290248, 44.62751325752205, 44.99900745824235, 45.27975498397754, 45.60304001361201, 45.94050421121288, 46.26662507444064, 46.57856676970196, 46.87065762981029, 47.196778493038025, 47.51155602189264, 47.76394521169498, 47.959617729631624, 48.26872359129967, 48.50126194594901, 48.75081530215807, 48.92380115134844, 49.18753367552392, 49.4144003629867, 49.578878711397216, 49.749028726994304, 49.941865411337666, 50.13186626208774, 50.2708221081587, 50.37007628392366, 50.48918129484162, 50.55156963389389, 50.710376315117834, 50.832317159629085, 50.89754133227462, 50.97127300570003, 51.03366134475229, 51.050676346311995, 51.10455718458441, 51.17545302441653, 51.21515469472251, 51.26903553299492, 51.311573036894195, 51.39948387828602, 51.439185548592015, 51.430678047812165, 51.416498879845726, 51.40515554547259, 51.373961375946465, 51.37112554235318, 51.382468876726314, 51.35978220798003, 51.35978220798003, 51.35978220798003, 51.354110540793464, 51.348438873606895, 51.348438873606895, 51.348438873606895, 51.342767206420326, 51.33993137282704, 51.3285880384539, 51.317244704080764, 51.30590136970763, 51.29455803533449, 51.28321470096135, 51.26903553299492, 51.26619969940164, 51.26052803221507, 51.23500552987551, 51.23500552987551, 51.237841363468796, 51.243513030655365, 51.20664719394266, 51.18963219238295, 51.16127385645011, 51.16410969004339, 51.14993052207697, 51.14709468848368, 51.107393018177696, 51.042168845532146, 50.97978050647989, 50.89754133227462, 50.66783881121856, 50.00708958398321], "calls": [0.0, 2.413294387885319, 3.896435357173241, 5.331367155375323, 7.0527181464991635, 8.751382468876727, 9.993477582735446, 11.479454385616652, 13.095879533788956, 14.607378839009726, 16.03947480361852, 17.36948075886907, 18.98874174063466, 20.19113518418739, 21.6771119870686, 23.120551286050535, 24.529960581913056, 25.911011541842726, 26.957434137764796, 28.22788758755636, 29.61177438107932, 30.81416782463205, 32.07611377364376, 33.156566372685255, 34.46388565918952, 35.53299492385787, 36.57941751977994, 37.753452627399824, 38.947338570172704, 39.99376116609477, 41.29257295181919, 42.472279726625636, 43.632135666279105, 44.72393159969373, 45.64557751751127, 46.81961262513116, 47.97096106400476, 49.16484700677764, 50.07514959022205, 51.288886368147914, 52.21053228596546, 53.31083572015994, 54.5274083316791, 55.68159260414599, 56.76204520318747, 57.92473697643422, 58.914442900490606, 59.994895499532085, 60.86266057907722, 61.93744151093213, 63.09446161699231, 64.19476505118679, 65.11073930181777, 66.04089272041517, 67.1497036553895, 68.2074695856847, 69.45523636673, 70.54136063295806, 71.84867991946233, 72.93764001928366, 74.14854096361626, 75.11839605251964, 76.22720698749397, 77.45795876697954, 78.50438136290163, 79.38632561041318, 80.22573235402547, 80.94886992031307, 81.68618665456711, 84.87933528060574, 86.21217706944957, 87.26427133255821, 87.79740804809573, 88.2312905878683, 88.57442645265576, 88.89487564869694, 89.24368318067096, 89.552789042339, 89.78816323058163, 90.07458242350339, 90.2759266086266, 90.55100246717522, 90.83174999291042, 91.04443751240677, 91.31100587017555, 91.58891756231743, 91.86399342086607, 92.10787510988855, 92.42265263874316, 92.72041516603807, 93.15146187221734, 93.50310523778464, 93.84907693616539, 94.30281031109095, 94.7593795196098, 95.28968040155404, 95.80863794912514, 96.46938717636048, 97.26342058248022, 98.26730567450302, 100.0], "rescued": [0, 727, 1133, 1516, 1978, 2408, 2711, 3059, 3414, 3727, 4006, 4259, 4589, 4812, 5075, 5326, 5551, 5788, 5965, 6167, 6383, 6562, 6768, 6936, 7097, 7219, 7365, 7514, 7655, 7788, 7916, 8057, 8194, 8310, 8400, 8526, 8637, 8753, 8838, 8963, 9063, 9154, 9250, 9348, 9438, 9510, 9575, 9641, 9688, 9774, 9843, 9901, 9948, 9992, 10030, 10072, 10136, 10175, 10225, 10262, 10313, 10345, 10370, 10390, 10407, 10417, 10429, 10443, 10453, 10455, 10455, 10456, 10457, 10458, 10459, 10460, 10462, 10462, 10462, 10462, 10463, 10465, 10467, 10471, 10474, 10476, 10482, 10488, 10498, 10498, 10511, 10521, 10535, 10554, 10572, 10592, 10605, 10627, 10655, 10685, 10723], "harmed": [0, 35, 58, 77, 103, 125, 143, 159, 183, 207, 236, 278, 299, 322, 350, 380, 419, 443, 468, 505, 530, 562, 588, 610, 640, 663, 695, 725, 751, 774, 799, 825, 851, 878, 899, 916, 945, 973, 997, 1029, 1049, 1082, 1118, 1148, 1171, 1194, 1224, 1248, 1273, 1303, 1329, 1364, 1385, 1407, 1439, 1462, 1501, 1526, 1557, 1579, 1599, 1617, 1645, 1670, 1691, 1712, 1725, 1735, 1753, 1755, 1755, 1758, 1761, 1762, 1763, 1766, 1769, 1773, 1777, 1781, 1786, 1792, 1799, 1804, 1809, 1820, 1826, 1831, 1839, 1852, 1871, 1891, 1904, 1928, 1947, 1981, 2017, 2061, 2118, 2229, 2500], "exact_score": [26.688029946402747, 27.541615857981455, 28.329977596914613, 29.061622663982078, 29.81879023338911, 30.539091966083433, 31.25939369877776, 31.957008762725803, 32.657459660267136, 33.31253722031591, 33.97328644755126, 34.61985650682018, 35.23523239656297, 35.8052349488132, 36.33269999716417, 36.880015880668125, 37.35643592433996, 37.81584096645209, 38.32912684683663, 38.86226356237416, 39.31316110370644, 39.778237813005134, 40.25182202308369, 40.65167455973684, 41.02033292686386, 41.44570796585656, 41.87959050562913, 42.28511470946885, 42.65944474378244, 43.064968947622155, 43.38258231007005, 43.79377818109633, 44.17377988259649, 44.58497575362278, 44.84303661061169, 45.126619969940165, 45.430054164421634, 45.72214502452996, 46.03408671979128, 46.27513257522049, 46.57856676970196, 46.825284292317725, 47.06065848056036, 47.3328985055157, 47.61081019765761, 47.820661883560675, 48.03618523665032, 48.30558942801236, 48.51260528032215, 48.711113631852086, 48.90395031619544, 49.13365283725151, 49.349176190341154, 49.53917704109123, 49.669625386382336, 49.80858123245328, 49.99291041601679, 50.16306043161387, 50.27932960893855, 50.37574795111023, 50.48918129484162, 50.582763803420015, 50.72739131667754, 50.823809658849214, 50.883362164308195, 50.95142217054703, 51.03082551115901, 51.056348013498564, 51.08187051583813, 51.15560218926354, 51.181124691603095, 51.22933386268894, 51.2747072001815, 51.32008053767405, 51.39097637750617, 51.430678047812165, 51.439185548592015, 51.4420213821853, 51.39664804469274, 51.382468876726314, 51.36545387516661, 51.382468876726314, 51.356946374386744, 51.356946374386744, 51.36261804157331, 51.35978220798003, 51.35978220798003, 51.354110540793464, 51.348438873606895, 51.34560304001361, 51.31440887048748, 51.26903553299492, 51.23500552987551, 51.1981396931628, 51.16978135722996, 51.121572186144114, 51.03366134475229, 50.945750503360465, 50.72739131667754, 50.50619629640133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.60176632922751, "threshold_area100": 47.30486629044608, "threshold_area25": 38.10509599296714}, "selected_budget": 48, "selected": {"val": {"score": 65.52111884180195, "calls": 47.99353433129524, "rescued": 2163, "harmed": 816}, "test": {"score": 50.55156963389389, "calls": 60.86266057907722, "rescued": 9688, "harmed": 1273}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 13.76766586303711, "operator": "ge"}, {"budget": 0.02, "threshold": 12.995628356933594, "operator": "gt"}, {"budget": 0.03, "threshold": 12.446234703063965, "operator": "gt"}, {"budget": 0.04, "threshold": 12.043827056884766, "operator": "ge"}, {"budget": 0.05, "threshold": 11.700261116027832, "operator": "ge"}, {"budget": 0.06, "threshold": 11.417972564697266, "operator": "ge"}, {"budget": 0.07, "threshold": 11.188470840454102, "operator": "ge"}, {"budget": 0.08, "threshold": 10.956367492675781, "operator": "ge"}, {"budget": 0.09, "threshold": 10.712663650512695, "operator": "gt"}, {"budget": 0.1, "threshold": 10.506269454956055, "operator": "ge"}, {"budget": 0.11, "threshold": 10.297618865966797, "operator": "ge"}, {"budget": 0.12, "threshold": 10.088582992553711, "operator": "ge"}, {"budget": 0.13, "threshold": 9.86279010772705, "operator": "ge"}, {"budget": 0.14, "threshold": 9.6840181350708, "operator": "gt"}, {"budget": 0.15, "threshold": 9.460795402526855, "operator": "gt"}, {"budget": 0.16, "threshold": 9.291158676147461, "operator": "ge"}, {"budget": 0.17, "threshold": 9.08704948425293, "operator": "ge"}, {"budget": 0.18, "threshold": 8.894731521606445, "operator": "ge"}, {"budget": 0.19, "threshold": 8.702866554260254, "operator": "ge"}, {"budget": 0.2, "threshold": 8.496193885803223, "operator": "ge"}, {"budget": 0.21, "threshold": 8.252660751342773, "operator": "ge"}, {"budget": 0.22, "threshold": 8.0512113571167, "operator": "ge"}, {"budget": 0.23, "threshold": 7.817712783813477, "operator": "ge"}, {"budget": 0.24, "threshold": 7.6040143966674805, "operator": "ge"}, {"budget": 0.25, "threshold": 7.408594608306885, "operator": "ge"}, {"budget": 0.26, "threshold": 7.210777282714844, "operator": "gt"}, {"budget": 0.27, "threshold": 7.008300304412842, "operator": "ge"}, {"budget": 0.28, "threshold": 6.8006720542907715, "operator": "ge"}, {"budget": 0.29, "threshold": 6.586397171020508, "operator": "ge"}, {"budget": 0.3, "threshold": 6.382441520690918, "operator": "ge"}, {"budget": 0.31, "threshold": 6.175017356872559, "operator": "ge"}, {"budget": 0.32, "threshold": 5.965510845184326, "operator": "ge"}, {"budget": 0.33, "threshold": 5.757253170013428, "operator": "ge"}, {"budget": 0.34, "threshold": 5.5638628005981445, "operator": "ge"}, {"budget": 0.35, "threshold": 5.365009784698486, "operator": "ge"}, {"budget": 0.36, "threshold": 5.167756080627441, "operator": "ge"}, {"budget": 0.37, "threshold": 4.9936089515686035, "operator": "ge"}, {"budget": 0.38, "threshold": 4.78926944732666, "operator": "ge"}, {"budget": 0.39, "threshold": 4.573127746582031, "operator": "ge"}, {"budget": 0.4, "threshold": 4.433065891265869, "operator": "ge"}, {"budget": 0.41, "threshold": 4.260238170623779, "operator": "ge"}, {"budget": 0.42, "threshold": 4.090496063232422, "operator": "ge"}, {"budget": 0.43, "threshold": 3.9097065925598145, "operator": "ge"}, {"budget": 0.44, "threshold": 3.7315850257873535, "operator": "ge"}, {"budget": 0.45, "threshold": 3.561415910720825, "operator": "ge"}, {"budget": 0.46, "threshold": 3.3940720558166504, "operator": "ge"}, {"budget": 0.47, "threshold": 3.231860876083374, "operator": "ge"}, {"budget": 0.48, "threshold": 3.0800609588623047, "operator": "ge"}, {"budget": 0.49, "threshold": 2.9551353454589844, "operator": "ge"}, {"budget": 0.5, "threshold": 2.7834243774414062, "operator": "ge"}, {"budget": 0.51, "threshold": 2.641951322555542, "operator": "ge"}, {"budget": 0.52, "threshold": 2.483100414276123, "operator": "ge"}, {"budget": 0.53, "threshold": 2.3372154235839844, "operator": "ge"}, {"budget": 0.54, "threshold": 2.1936285495758057, "operator": "ge"}, {"budget": 0.55, "threshold": 2.021146059036255, "operator": "ge"}, {"budget": 0.56, "threshold": 1.8979138135910034, "operator": "ge"}, {"budget": 0.57, "threshold": 1.751358985900879, "operator": "ge"}, {"budget": 0.58, "threshold": 1.606612205505371, "operator": "ge"}, {"budget": 0.59, "threshold": 1.4577370882034302, "operator": "ge"}, {"budget": 0.6, "threshold": 1.3076798915863037, "operator": "ge"}, {"budget": 0.61, "threshold": 1.1572853326797485, "operator": "ge"}, {"budget": 0.62, "threshold": 1.0226638317108154, "operator": "ge"}, {"budget": 0.63, "threshold": 0.8604909777641296, "operator": "ge"}, {"budget": 0.64, "threshold": 0.7239235639572144, "operator": "ge"}, {"budget": 0.65, "threshold": 0.5764501094818115, "operator": "ge"}, {"budget": 0.66, "threshold": 0.416790246963501, "operator": "ge"}, {"budget": 0.67, "threshold": 0.2561919093132019, "operator": "ge"}, {"budget": 0.68, "threshold": 0.0792422890663147, "operator": "ge"}, {"budget": 0.69, "threshold": -0.08653521537780762, "operator": "ge"}, {"budget": 0.7, "threshold": -0.249167799949646, "operator": "ge"}, {"budget": 0.71, "threshold": -0.4182720184326172, "operator": "ge"}, {"budget": 0.72, "threshold": -0.5991735458374023, "operator": "ge"}, {"budget": 0.73, "threshold": -0.748469889163971, "operator": "ge"}, {"budget": 0.74, "threshold": -0.8869025707244873, "operator": "ge"}, {"budget": 0.75, "threshold": -1.0565943717956543, "operator": "ge"}, {"budget": 0.76, "threshold": -1.2027149200439453, "operator": "ge"}, {"budget": 0.77, "threshold": -1.3525954484939575, "operator": "ge"}, {"budget": 0.78, "threshold": -1.5118188858032227, "operator": "ge"}, {"budget": 0.79, "threshold": -1.6603361368179321, "operator": "ge"}, {"budget": 0.8, "threshold": -1.810987949371338, "operator": "ge"}, {"budget": 0.81, "threshold": -1.9708075523376465, "operator": "ge"}, {"budget": 0.82, "threshold": -2.1348671913146973, "operator": "ge"}, {"budget": 0.83, "threshold": -2.277512550354004, "operator": "ge"}, {"budget": 0.84, "threshold": -2.431856155395508, "operator": "ge"}, {"budget": 0.85, "threshold": -2.5894603729248047, "operator": "ge"}, {"budget": 0.86, "threshold": -2.751997232437134, "operator": "ge"}, {"budget": 0.87, "threshold": -2.93576717376709, "operator": "ge"}, {"budget": 0.88, "threshold": -3.105588436126709, "operator": "ge"}, {"budget": 0.89, "threshold": -3.2929370403289795, "operator": "ge"}, {"budget": 0.9, "threshold": -3.5004723072052, "operator": "ge"}, {"budget": 0.91, "threshold": -3.6995561122894287, "operator": "ge"}, {"budget": 0.92, "threshold": -3.930145025253296, "operator": "ge"}, {"budget": 0.93, "threshold": -4.172539710998535, "operator": "ge"}, {"budget": 0.94, "threshold": -4.448844909667969, "operator": "ge"}, {"budget": 0.95, "threshold": -4.793267250061035, "operator": "ge"}, {"budget": 0.96, "threshold": -5.18986701965332, "operator": "ge"}, {"budget": 0.97, "threshold": -5.664662837982178, "operator": "ge"}, {"budget": 0.98, "threshold": -6.418604850769043, "operator": "ge"}, {"budget": 0.99, "threshold": -7.712868690490723, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.806521891910876, 57.46714456391875, 58.14182303745871, 58.71811090027408, 59.23817555696114, 59.77229601518026, 60.29236067186731, 60.82648113008644, 61.33951788600744, 61.76119193196992, 62.23908918406072, 62.653735329257145, 63.04729777215545, 63.46194391735188, 63.88361796331436, 64.29123620774475, 64.58640803991848, 64.89563567362428, 65.21189120809613, 65.528146742568, 65.73898376554924, 66.041183498489, 66.18174151380983, 66.39257853679106, 66.58233185747416, 66.77911307892333, 66.87047578888186, 67.01806170496873, 67.20078712488579, 67.38351254480285, 67.36242884250474, 67.36242884250474, 67.43270785016516, 67.43973575093119, 67.51001475859161, 67.52407056012369, 67.52407056012369, 67.43973575093119, 67.3554009417387, 67.24295452948203, 67.24295452948203, 67.13753601799142, 67.04617330803289, 66.98292220113852, 66.80722468198749, 66.70883407126291, 66.59638765900625, 66.46285754445147, 66.37149483449292, 66.2449926207042, 66.0903788038513, 66.0060439946588, 65.86548597933796, 65.76006746784735, 65.73195586478319, 65.61248155176048, 65.49300723873779, 65.5140909410359, 65.47192353643966, 65.3946166280132, 65.33839342188487, 65.22594700962821, 65.13458429966968, 65.11350059737156, 65.13458429966968, 65.1556680019678, 65.09944479583947, 65.16269590273384, 65.09944479583947, 65.0853889943074, 65.07133319277531, 65.04322158971115, 65.00808208588094, 65.05024949047719, 65.05024949047719, 65.0361936889451, 65.00808208588094, 64.98699838358282, 64.93780307822054, 64.89563567362428, 64.86049616979408, 64.80427296366575, 64.74804975753742, 64.73399395600535, 64.66371494834493, 64.63560334528076, 64.63560334528076, 64.6074917422166, 64.52315693302411, 64.4880174291939, 64.47396162766181, 64.4528779253637, 64.38962681846931, 64.36151521540515, 64.33340361234099, 64.32637571157494, 64.25609670391454, 64.23501300161641, 64.18581769625412, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.988895916789655, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.988685079766674, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.992550425187996, 14.990512333965844, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.989177032820297, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 107, 201, 297, 380, 454, 530, 606, 682, 757, 818, 886, 945, 1004, 1066, 1126, 1184, 1228, 1274, 1324, 1374, 1412, 1465, 1495, 1531, 1568, 1608, 1639, 1674, 1715, 1750, 1768, 1799, 1826, 1848, 1878, 1898, 1920, 1938, 1957, 1971, 1991, 2011, 2029, 2046, 2062, 2080, 2094, 2103, 2114, 2130, 2139, 2156, 2172, 2184, 2203, 2221, 2232, 2258, 2274, 2286, 2304, 2318, 2329, 2348, 2368, 2390, 2406, 2431, 2441, 2459, 2469, 2478, 2488, 2504, 2515, 2525, 2536, 2544, 2548, 2554, 2557, 2559, 2564, 2567, 2571, 2575, 2580, 2584, 2585, 2586, 2588, 2588, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589], "harmed": [0, 0, 0, 0, 1, 1, 1, 3, 3, 5, 6, 6, 6, 9, 12, 12, 12, 14, 16, 21, 26, 34, 44, 54, 60, 70, 82, 100, 114, 129, 138, 159, 190, 207, 228, 248, 266, 288, 318, 349, 379, 399, 434, 465, 491, 532, 564, 594, 622, 646, 680, 711, 740, 776, 803, 826, 861, 889, 912, 934, 957, 983, 1013, 1037, 1059, 1076, 1095, 1119, 1135, 1154, 1174, 1186, 1199, 1214, 1224, 1235, 1247, 1262, 1273, 1284, 1296, 1304, 1314, 1327, 1332, 1346, 1354, 1359, 1367, 1380, 1386, 1390, 1393, 1403, 1407, 1411, 1412, 1422, 1425, 1432, 1441], "exact_score": [56.05453650994448, 56.806521891910876, 57.47417246468479, 58.14885093822475, 58.71811090027408, 59.23817555696114, 59.77229601518026, 60.29236067186731, 60.82648113008644, 61.34654578677349, 61.76119193196992, 62.23908918406072, 62.653735329257145, 63.04729777215545, 63.46897181811792, 63.88361796331436, 64.29123620774475, 64.58640803991848, 64.89563567362428, 65.21189120809613, 65.528146742568, 65.73898376554924, 66.041183498489, 66.18174151380983, 66.39257853679106, 66.58233185747416, 66.77911307892333, 66.87047578888186, 67.01806170496873, 67.20078712488579, 67.38351254480285, 67.36242884250474, 67.36242884250474, 67.43270785016516, 67.43973575093119, 67.51001475859161, 67.52407056012369, 67.52407056012369, 67.43973575093119, 67.3554009417387, 67.24295452948203, 67.24295452948203, 67.13753601799142, 67.04617330803289, 66.98292220113852, 66.80722468198749, 66.70883407126291, 66.59638765900625, 66.46285754445147, 66.37149483449292, 66.2449926207042, 66.0903788038513, 66.0060439946588, 65.86548597933796, 65.76006746784735, 65.73195586478319, 65.61248155176048, 65.49300723873779, 65.5140909410359, 65.47192353643966, 65.3946166280132, 65.33839342188487, 65.22594700962821, 65.13458429966968, 65.11350059737156, 65.13458429966968, 65.1556680019678, 65.09944479583947, 65.16269590273384, 65.09944479583947, 65.0853889943074, 65.07133319277531, 65.04322158971115, 65.00808208588094, 65.05024949047719, 65.05024949047719, 65.0361936889451, 65.00808208588094, 64.98699838358282, 64.93780307822054, 64.89563567362428, 64.86049616979408, 64.80427296366575, 64.74804975753742, 64.73399395600535, 64.66371494834493, 64.63560334528076, 64.63560334528076, 64.6074917422166, 64.52315693302411, 64.4880174291939, 64.47396162766181, 64.4528779253637, 64.38962681846931, 64.36151521540515, 64.33340361234099, 64.32637571157494, 64.25609670391454, 64.23501300161641, 64.18581769625412, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 64.86581241196816, "threshold_area100": 64.86527514231499, "threshold_area25": 62.34591327570455}, "test": {"score": [26.688029946402747, 27.260868332246265, 27.80818421575022, 28.389530102373595, 28.902815982758135, 29.46998270141508, 29.88685023962794, 30.320732779400505, 30.73192865042679, 31.242378697218047, 31.639395400277913, 32.10447210957661, 32.552533817315606, 33.14522303831211, 33.53940390777869, 33.97328644755126, 34.33343731389842, 34.83254402631653, 35.37135240904064, 35.90732495817146, 36.51135751354111, 37.17494257436974, 37.86688597113121, 38.57300853585912, 39.19122025919519, 39.815103649717834, 40.31988202932252, 41.08272126591611, 41.7916796642373, 42.48929472818536, 43.14153645464084, 43.873181521708304, 44.5254232481638, 45.129455803533446, 45.70229418937697, 46.29498341037348, 46.825284292317725, 47.17409182429175, 47.63066103281059, 48.09290190851601, 48.36797776706463, 48.67424779513938, 48.93514448572158, 49.25842951535604, 49.547684541871085, 49.86813373791226, 50.043955420695916, 50.19141876754672, 50.341717947990816, 50.41544962141622, 50.577092136233446, 50.67351047840513, 50.69619714715141, 50.80112299010294, 50.85783966196864, 51.01664634319258, 51.09604968380456, 51.15276635567025, 51.20664719394266, 51.252020531435214, 51.246348864248645, 51.26619969940164, 51.29172220174121, 51.33993137282704, 51.35978220798003, 51.35978220798003, 51.36261804157331, 51.373961375946465, 51.36261804157331, 51.3285880384539, 51.3853047103196, 51.373961375946465, 51.382468876726314, 51.35978220798003, 51.37112554235318, 51.356946374386744, 51.33425970564047, 51.26336386580835, 51.20664719394266, 51.1981396931628, 51.19246802597623, 51.11022885177098, 51.05918384709185, 51.02515384397244, 50.97410883929331, 50.945750503360465, 50.88619799790148, 50.82097382525593, 50.772764654170096, 50.68201797918499, 50.59127130419987, 50.5487338003006, 50.509032129994615, 50.446643790942346, 50.36724045033038, 50.285001276125115, 50.236792105039285, 50.20276210191986, 50.154552930834015, 50.114851260528035, 50.00708958398321], "calls": [0.0, 0.7401525678473188, 1.5313501403737628, 2.4189660550718886, 3.2782236338371664, 4.160167881348722, 4.97121628902816, 5.680174687349346, 6.482715594248929, 7.398689844879902, 8.178544083033207, 9.029294161018631, 9.919745909310041, 11.05407934662394, 11.893486090236225, 12.860505345546322, 13.680061254005615, 14.814394691319514, 15.931713127073705, 17.108584068286874, 18.356350849332163, 19.859342653773076, 21.263080282449025, 23.001446275132576, 24.50443807957349, 26.078325723846525, 27.660720868899414, 29.450131866262087, 31.409692879221847, 33.34656722343533, 35.24090406374954, 37.28554008450784, 39.2819669341803, 41.15078127215495, 43.042282278875874, 44.908260783257234, 46.822448458724445, 48.41335110455719, 50.30768794487139, 52.36933896718941, 53.571732410742136, 55.13427672064203, 56.6457760258628, 58.35011201542694, 60.054448004991066, 61.492215636786426, 63.02923744434676, 64.23730255508606, 65.3489493236537, 66.25074440631823, 67.46731701783739, 68.46269460908034, 69.51762470578227, 70.43359895641323, 71.38927487735019, 72.51510081388423, 73.24107421376513, 74.09466012534385, 74.93406686895612, 75.75362277741542, 76.58168618665457, 77.42109293026685, 78.12721549499474, 78.92408473470776, 79.63587896662224, 80.41289737118225, 81.15021410543629, 82.00663585060829, 82.92828176842583, 83.76768851203812, 84.53903524941157, 85.37277032583728, 86.24904290616226, 86.9693446388566, 87.64427303405836, 88.34755976519297, 89.00263732524175, 89.71159572356294, 90.32413577971245, 90.83174999291042, 91.39608087797409, 91.97742676459745, 92.49354847857528, 92.95295352068742, 93.45773190029209, 93.94833111193034, 94.46728865950146, 94.9068428664606, 95.36341207497944, 95.89654879051697, 96.4098346709015, 96.82953804270765, 97.24356974732723, 97.64909395116695, 98.1425289963985, 98.66999404474946, 99.02163741031677, 99.3676091086975, 99.60581913053342, 99.79865581487678, 100.0], "rescued": [0, 202, 395, 601, 782, 982, 1129, 1282, 1428, 1608, 1749, 1914, 2073, 2283, 2423, 2579, 2711, 2894, 3091, 3287, 3505, 3746, 3997, 4265, 4498, 4735, 4943, 5227, 5516, 5795, 6054, 6347, 6616, 6869, 7112, 7366, 7601, 7766, 7969, 8205, 8337, 8499, 8646, 8809, 8964, 9113, 9241, 9343, 9434, 9494, 9587, 9661, 9722, 9792, 9852, 9942, 9993, 10042, 10091, 10139, 10173, 10215, 10252, 10294, 10330, 10357, 10379, 10413, 10436, 10464, 10498, 10522, 10555, 10575, 10596, 10615, 10628, 10642, 10648, 10661, 10675, 10680, 10685, 10691, 10699, 10705, 10709, 10711, 10715, 10716, 10718, 10720, 10723, 10723, 10723, 10723, 10723, 10723, 10723, 10723, 10723], "harmed": [0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 3, 4, 5, 6, 7, 10, 15, 22, 29, 36, 41, 48, 55, 74, 89, 106, 136, 151, 190, 223, 252, 287, 326, 366, 407, 452, 500, 542, 584, 657, 692, 746, 801, 850, 903, 939, 1005, 1055, 1093, 1127, 1163, 1203, 1256, 1289, 1329, 1363, 1386, 1415, 1445, 1477, 1513, 1548, 1576, 1601, 1630, 1657, 1678, 1708, 1735, 1775, 1789, 1817, 1847, 1875, 1892, 1916, 1937, 1976, 2002, 2018, 2034, 2068, 2091, 2109, 2135, 2151, 2176, 2201, 2222, 2255, 2289, 2306, 2323, 2345, 2373, 2402, 2419, 2431, 2448, 2462, 2500], "exact_score": [26.688029946402747, 27.44519751580977, 28.12296174460483, 28.724158466381194, 29.365056858463547, 29.90386524118765, 30.48237529421774, 31.01551200975527, 31.545812891699516, 32.09029294161019, 32.58372798684173, 33.119700535972555, 33.58194141167796, 34.03851062019681, 34.47522899356266, 34.90911153333522, 35.39971074497349, 35.859115787085614, 36.35255083231716, 36.820463375209144, 37.25151008138843, 37.753452627399824, 38.144797663273124, 38.57017270226583, 38.95868190454584, 39.39256444431841, 39.78958114737827, 40.10152284263959, 40.46167370898676, 40.901227915945896, 41.25003544791992, 41.61585798145365, 42.01854635170008, 42.38436888523381, 42.71048974846156, 43.04795394606244, 43.41661231318946, 43.77392734594334, 44.10571987635766, 44.44318407395854, 44.80049910671242, 45.095425800414034, 45.37333749255594, 45.6796075206307, 46.04259422057114, 46.31767007911976, 46.57573093610867, 46.86215012903043, 47.10319598445963, 47.30454016958285, 47.53991435782549, 47.814990216374106, 48.00782690071747, 48.26872359129967, 48.458724442049736, 48.64021779201997, 48.79902447324391, 49.014547826333555, 49.19320534271049, 49.36335535830757, 49.53634120749795, 49.7688795621473, 49.95888041289737, 50.035447919916066, 50.19425460114001, 50.29634461049825, 50.42112128860279, 50.52037546436775, 50.622465473725995, 50.67067464481184, 50.758585486203664, 50.823809658849214, 50.95142217054703, 51.0676913478717, 51.15560218926354, 51.2179905283158, 51.246348864248645, 51.27187136658821, 51.28321470096135, 51.33709553923376, 51.36545387516661, 51.37112554235318, 51.373961375946465, 51.35978220798003, 51.35978220798003, 51.39097637750617, 51.376797209539745, 51.356946374386744, 51.36545387516661, 51.33142387204719, 51.237841363468796, 51.19530385956952, 51.11022885177098, 51.02798967756572, 50.94858633695375, 50.80679465728951, 50.67351047840513, 50.540226299520754, 50.4012704534498, 50.242463772225854, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 44.5553706738452, "threshold_area100": 45.37389048010663, "threshold_area25": 32.969117772169135}, "selected_budget": 36, "selected": {"val": {"score": 67.52407056012369, "calls": 35.99690772366294, "rescued": 1898, "harmed": 266}, "test": {"score": 46.825284292317725, "calls": 46.822448458724445, "rescued": 7601, "harmed": 500}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.464465618133545, "operator": "ge"}, {"budget": 0.02, "threshold": 2.137476921081543, "operator": "ge"}, {"budget": 0.03, "threshold": 1.9170101881027222, "operator": "ge"}, {"budget": 0.04, "threshold": 1.7361106872558594, "operator": "ge"}, {"budget": 0.05, "threshold": 1.5584654808044434, "operator": "ge"}, {"budget": 0.06, "threshold": 1.4554636478424072, "operator": "ge"}, {"budget": 0.07, "threshold": 1.3667434453964233, "operator": "ge"}, {"budget": 0.08, "threshold": 1.2847647666931152, "operator": "ge"}, {"budget": 0.09, "threshold": 1.1982109546661377, "operator": "ge"}, {"budget": 0.1, "threshold": 1.138193964958191, "operator": "ge"}, {"budget": 0.11, "threshold": 1.0734827518463135, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0270612239837646, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9801292419433594, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9387234449386597, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9017994403839111, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8602845668792725, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8263950347900391, "operator": "ge"}, {"budget": 0.18, "threshold": 0.7958202362060547, "operator": "ge"}, {"budget": 0.19, "threshold": 0.7620407938957214, "operator": "ge"}, {"budget": 0.2, "threshold": 0.7359527349472046, "operator": "ge"}, {"budget": 0.21, "threshold": 0.7061933279037476, "operator": "ge"}, {"budget": 0.22, "threshold": 0.682999849319458, "operator": "ge"}, {"budget": 0.23, "threshold": 0.6573234796524048, "operator": "ge"}, {"budget": 0.24, "threshold": 0.6335289478302002, "operator": "ge"}, {"budget": 0.25, "threshold": 0.6096422672271729, "operator": "gt"}, {"budget": 0.26, "threshold": 0.5884978771209717, "operator": "ge"}, {"budget": 0.27, "threshold": 0.5644290447235107, "operator": "ge"}, {"budget": 0.28, "threshold": 0.544765293598175, "operator": "ge"}, {"budget": 0.29, "threshold": 0.5252872705459595, "operator": "ge"}, {"budget": 0.3, "threshold": 0.5069500207901001, "operator": "ge"}, {"budget": 0.31, "threshold": 0.48923325538635254, "operator": "ge"}, {"budget": 0.32, "threshold": 0.46916478872299194, "operator": "ge"}, {"budget": 0.33, "threshold": 0.4502747058868408, "operator": "ge"}, {"budget": 0.34, "threshold": 0.4331258535385132, "operator": "ge"}, {"budget": 0.35, "threshold": 0.41507089138031006, "operator": "ge"}, {"budget": 0.36, "threshold": 0.3987739086151123, "operator": "ge"}, {"budget": 0.37, "threshold": 0.38419926166534424, "operator": "ge"}, {"budget": 0.38, "threshold": 0.3686037063598633, "operator": "ge"}, {"budget": 0.39, "threshold": 0.35427558422088623, "operator": "ge"}, {"budget": 0.4, "threshold": 0.33858776092529297, "operator": "ge"}, {"budget": 0.41, "threshold": 0.32313644886016846, "operator": "ge"}, {"budget": 0.42, "threshold": 0.31172847747802734, "operator": "ge"}, {"budget": 0.43, "threshold": 0.29739898443222046, "operator": "ge"}, {"budget": 0.44, "threshold": 0.2865335941314697, "operator": "ge"}, {"budget": 0.45, "threshold": 0.27423393726348877, "operator": "ge"}, {"budget": 0.46, "threshold": 0.25940653681755066, "operator": "ge"}, {"budget": 0.47, "threshold": 0.24613821506500244, "operator": "ge"}, {"budget": 0.48, "threshold": 0.23358142375946045, "operator": "ge"}, {"budget": 0.49, "threshold": 0.221574604511261, "operator": "ge"}, {"budget": 0.5, "threshold": 0.20859789848327637, "operator": "ge"}, {"budget": 0.51, "threshold": 0.19704341888427734, "operator": "ge"}, {"budget": 0.52, "threshold": 0.18669262528419495, "operator": "ge"}, {"budget": 0.53, "threshold": 0.1757773756980896, "operator": "ge"}, {"budget": 0.54, "threshold": 0.1628662347793579, "operator": "ge"}, {"budget": 0.55, "threshold": 0.1534121036529541, "operator": "ge"}, {"budget": 0.56, "threshold": 0.14304161071777344, "operator": "gt"}, {"budget": 0.57, "threshold": 0.13332831859588623, "operator": "ge"}, {"budget": 0.58, "threshold": 0.12340337038040161, "operator": "ge"}, {"budget": 0.59, "threshold": 0.11206459999084473, "operator": "ge"}, {"budget": 0.6, "threshold": 0.10070908069610596, "operator": "ge"}, {"budget": 0.61, "threshold": 0.09013199806213379, "operator": "ge"}, {"budget": 0.62, "threshold": 0.07911568880081177, "operator": "gt"}, {"budget": 0.63, "threshold": 0.06895947456359863, "operator": "ge"}, {"budget": 0.64, "threshold": 0.05767083168029785, "operator": "ge"}, {"budget": 0.65, "threshold": 0.04813838005065918, "operator": "ge"}, {"budget": 0.66, "threshold": 0.037329673767089844, "operator": "ge"}, {"budget": 0.67, "threshold": 0.027161836624145508, "operator": "ge"}, {"budget": 0.68, "threshold": 0.01435708999633789, "operator": "gt"}, {"budget": 0.69, "threshold": -0.023906409740447998, "operator": "ge"}, {"budget": 0.7, "threshold": -0.07099857926368713, "operator": "ge"}, {"budget": 0.71, "threshold": -0.12703967094421387, "operator": "ge"}, {"budget": 0.72, "threshold": -0.19544804096221924, "operator": "ge"}, {"budget": 0.73, "threshold": -0.28133314847946167, "operator": "ge"}, {"budget": 0.74, "threshold": -0.3645617961883545, "operator": "ge"}, {"budget": 0.75, "threshold": -0.455533504486084, "operator": "ge"}, {"budget": 0.76, "threshold": -0.548882246017456, "operator": "ge"}, {"budget": 0.77, "threshold": -0.6574874520301819, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7462047338485718, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8609235286712646, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9741929769515991, "operator": "ge"}, {"budget": 0.81, "threshold": -1.096213698387146, "operator": "ge"}, {"budget": 0.82, "threshold": -1.217301607131958, "operator": "ge"}, {"budget": 0.83, "threshold": -1.3645069599151611, "operator": "ge"}, {"budget": 0.84, "threshold": -1.4866176843643188, "operator": "ge"}, {"budget": 0.85, "threshold": -1.6236629486083984, "operator": "ge"}, {"budget": 0.86, "threshold": -1.7826200723648071, "operator": "ge"}, {"budget": 0.87, "threshold": -1.9466053247451782, "operator": "ge"}, {"budget": 0.88, "threshold": -2.1470255851745605, "operator": "ge"}, {"budget": 0.89, "threshold": -2.3245842456817627, "operator": "ge"}, {"budget": 0.9, "threshold": -2.5218594074249268, "operator": "ge"}, {"budget": 0.91, "threshold": -2.7426044940948486, "operator": "ge"}, {"budget": 0.92, "threshold": -2.96844482421875, "operator": "ge"}, {"budget": 0.93, "threshold": -3.1965019702911377, "operator": "ge"}, {"budget": 0.94, "threshold": -3.4590280055999756, "operator": "ge"}, {"budget": 0.95, "threshold": -3.7793872356414795, "operator": "ge"}, {"budget": 0.96, "threshold": -4.169293403625488, "operator": "ge"}, {"budget": 0.97, "threshold": -4.632381439208984, "operator": "ge"}, {"budget": 0.98, "threshold": -5.527586936950684, "operator": "ge"}, {"budget": 0.99, "threshold": -6.836864471435547, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.602712769695685, 57.17197273174502, 57.804483800688736, 58.331576358141824, 58.753250404104286, 59.23114765619509, 59.66687750368964, 59.96204933586338, 60.243165366505025, 60.636727809403325, 60.88973223698081, 61.21301567221871, 61.5714386112868, 61.79633143580012, 61.95797315341907, 62.19692177946447, 62.456954107807995, 62.695902733853394, 62.8505165507063, 63.11757677981586, 63.27219059666878, 63.28624639820085, 63.34246960432919, 63.46897181811792, 63.59547403190666, 63.76414365029165, 63.91172956637852, 64.12256658935975, 64.32637571157494, 64.44585002459765, 64.49504532995994, 64.5582964368543, 64.64965914681284, 64.63560334528076, 64.67777074987701, 64.65668704757888, 64.67777074987701, 64.79021716213367, 64.8815798720922, 65.00105418511491, 64.98699838358282, 65.00105418511491, 65.07133319277531, 65.10647269660552, 65.2048633073301, 65.26108651345842, 65.26811441422447, 65.33839342188487, 65.37353292571508, 65.35244922341697, 65.38056082648113, 65.359477124183, 65.33136552111884, 65.31730971958676, 65.28919811652258, 65.31028181882071, 65.31028181882071, 65.2751423149905, 65.29622601728863, 65.34542132265092, 65.33839342188487, 65.34542132265092, 65.3243376203528, 65.30325391805466, 65.31028181882071, 65.28919811652258, 65.2751423149905, 65.16269590273384, 65.0853889943074, 65.04322158971115, 65.00808208588094, 64.97997048281677, 64.94483097898657, 64.86752407056012, 64.88860777285824, 64.93780307822054, 64.90266357439033, 64.88860777285824, 64.85346826902804, 64.76210555906951, 64.77616136060158, 64.74102185677137, 64.71993815447325, 64.74804975753742, 64.70588235294117, 64.65668704757888, 64.57235223838639, 64.5582964368543, 64.49504532995994, 64.4528779253637, 64.41071052076744, 64.38962681846931, 64.3544873146391, 64.34043151310703, 64.3193478108089, 64.29123620774475, 64.25609670391454, 64.20690139855225, 64.17878979548807, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.991215124042448, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99128540305011, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.98608475648324, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.98791201068241, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 90, 178, 277, 365, 446, 523, 594, 649, 706, 776, 825, 884, 948, 994, 1038, 1086, 1139, 1186, 1231, 1283, 1321, 1349, 1375, 1414, 1451, 1488, 1526, 1569, 1614, 1646, 1674, 1705, 1741, 1768, 1795, 1815, 1840, 1869, 1897, 1932, 1951, 1973, 1999, 2020, 2048, 2075, 2095, 2117, 2138, 2154, 2181, 2195, 2213, 2234, 2247, 2267, 2282, 2296, 2313, 2334, 2349, 2364, 2375, 2386, 2401, 2416, 2428, 2432, 2442, 2453, 2467, 2474, 2485, 2490, 2505, 2521, 2531, 2542, 2548, 2554, 2563, 2566, 2571, 2580, 2583, 2584, 2585, 2588, 2588, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589, 2589], "harmed": [0, 12, 19, 28, 41, 62, 71, 80, 93, 110, 124, 137, 150, 163, 177, 198, 212, 228, 241, 264, 278, 294, 320, 338, 359, 378, 391, 408, 421, 437, 452, 473, 495, 518, 547, 568, 591, 613, 626, 641, 659, 680, 700, 716, 732, 746, 765, 784, 796, 812, 831, 854, 871, 893, 916, 933, 950, 965, 984, 998, 1012, 1028, 1042, 1056, 1070, 1084, 1102, 1116, 1136, 1157, 1174, 1193, 1204, 1220, 1236, 1248, 1257, 1272, 1285, 1296, 1315, 1322, 1330, 1338, 1343, 1352, 1360, 1373, 1378, 1387, 1394, 1400, 1403, 1408, 1410, 1413, 1417, 1422, 1429, 1433, 1441], "exact_score": [56.05453650994448, 56.602712769695685, 57.17197273174502, 57.804483800688736, 58.331576358141824, 58.753250404104286, 59.23114765619509, 59.66687750368964, 59.96204933586338, 60.243165366505025, 60.636727809403325, 60.88973223698081, 61.21301567221871, 61.5714386112868, 61.79633143580012, 61.95797315341907, 62.19692177946447, 62.456954107807995, 62.695902733853394, 62.8505165507063, 63.11757677981586, 63.27219059666878, 63.28624639820085, 63.34246960432919, 63.46897181811792, 63.60250193267271, 63.76414365029165, 63.91172956637852, 64.12256658935975, 64.32637571157494, 64.44585002459765, 64.49504532995994, 64.5582964368543, 64.64965914681284, 64.63560334528076, 64.67777074987701, 64.65668704757888, 64.67777074987701, 64.79021716213367, 64.8815798720922, 65.00105418511491, 64.98699838358282, 65.00105418511491, 65.07133319277531, 65.10647269660552, 65.2048633073301, 65.26108651345842, 65.26811441422447, 65.33839342188487, 65.37353292571508, 65.35244922341697, 65.38056082648113, 65.359477124183, 65.33136552111884, 65.31730971958676, 65.28919811652258, 65.31028181882071, 65.31028181882071, 65.2751423149905, 65.29622601728863, 65.34542132265092, 65.33839342188487, 65.34542132265092, 65.3243376203528, 65.30325391805466, 65.31028181882071, 65.28919811652258, 65.2751423149905, 65.16269590273384, 65.0853889943074, 65.04322158971115, 65.00808208588094, 64.97997048281677, 64.94483097898657, 64.86752407056012, 64.88860777285824, 64.93780307822054, 64.90266357439033, 64.88860777285824, 64.85346826902804, 64.76210555906951, 64.77616136060158, 64.74102185677137, 64.71993815447325, 64.74804975753742, 64.70588235294117, 64.65668704757888, 64.57235223838639, 64.5582964368543, 64.49504532995994, 64.4528779253637, 64.41071052076744, 64.38962681846931, 64.3544873146391, 64.34043151310703, 64.3193478108089, 64.29123620774475, 64.25609670391454, 64.20690139855225, 64.17878979548807, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 63.856972049724014, "threshold_area100": 63.856630824372765, "threshold_area25": 60.85810668353362}, "test": {"score": [26.688029946402747, 28.605053455463235, 29.657147718571874, 30.66670447778125, 31.73864957604288, 32.958058021155324, 33.81447976632732, 34.63687150837989, 35.43657658168618, 36.32986416357088, 36.99344922439952, 37.68822845475428, 38.18449933357911, 38.907636899866716, 39.49465445367666, 40.041970337180615, 40.739585401128664, 41.19048294246094, 41.638544650199925, 42.15183053058447, 42.602728071916744, 43.070640614808724, 43.47332898505516, 43.88736068967473, 44.329750730227154, 44.726767433287016, 45.0273657941752, 45.33647165584323, 45.620055015171715, 45.89513087372033, 46.19289340101523, 46.439610923631, 46.760060119672175, 47.00961347588124, 47.22513682897088, 47.520063522672494, 47.76961687888155, 47.979468564784625, 48.2630519241131, 48.50126194594901, 48.722456966225224, 48.943651986501436, 49.10813033491195, 49.3406686895613, 49.53066954031138, 49.66111788560247, 49.83126790119956, 50.021268751949634, 50.19992626832658, 50.282165442531834, 50.398434619856516, 50.55156963389389, 50.61112213935286, 50.73022715027082, 50.8663471627485, 50.93157133539405, 51.02798967756572, 51.06485551427842, 51.09888551739784, 51.12440801973741, 51.172617190823246, 51.23216969628223, 51.2747072001815, 51.33709553923376, 51.368289708759896, 51.40799137906588, 51.373961375946465, 51.36545387516661, 51.351274707200176, 51.33425970564047, 51.34560304001361, 51.33425970564047, 51.33709553923376, 51.33142387204719, 51.28321470096135, 51.2548563650285, 51.246348864248645, 51.21515469472251, 51.18963219238295, 51.135751354110546, 51.09888551739784, 51.0676913478717, 51.02515384397244, 50.96843717210674, 50.91739216742761, 50.8464963275955, 50.778436321356665, 50.7160479823044, 50.639480475285715, 50.574256302640165, 50.49485296202819, 50.42962878938264, 50.384255451890084, 50.341717947990816, 50.29918044409154, 50.27365794175198, 50.22828460425942, 50.18858293395344, 50.146045430054166, 50.10067209256162, 50.00708958398321], "calls": [0.0, 2.3821002183591866, 3.8567336868672544, 5.223605478830502, 6.701074780931855, 8.59257578765278, 9.848850069477923, 11.187363525508323, 12.557071151064855, 14.213197969543149, 15.526188923233985, 16.949777387062927, 18.075603323596972, 19.521878456172193, 20.809346907523466, 22.07696452372175, 23.628165499248503, 24.864588945920655, 26.030116552760685, 27.382809176757505, 28.474605110172135, 29.705356889657715, 30.712077815273798, 31.880441255707115, 33.04596886254715, 34.23134730454017, 35.34866574029436, 36.53688001588067, 37.554944275869886, 38.62405354053824, 39.68749113802002, 40.68570456285625, 41.805858832203725, 42.97422227263704, 44.00930153418597, 45.12378413634688, 46.24677423928764, 47.324391004735844, 48.492754445169155, 49.56753537702407, 50.778436321356665, 51.96381476334969, 52.85143067804782, 53.95456994583557, 54.83651419334713, 55.77801094631767, 56.903836882851714, 58.04951365453876, 59.1384737543601, 60.252956356521004, 61.47236480163344, 62.53580239911521, 63.53117999035817, 64.57760258628024, 65.85939937044493, 66.81791112497518, 67.82179621699798, 68.8029946402745, 69.77284972917789, 71.09434818364858, 72.32509996313416, 73.67212091994442, 75.07585854862036, 76.28959532654623, 77.78975129739388, 79.065876414372, 80.43274820633525, 81.66349998582083, 82.80066925672801, 83.87545018858293, 84.72620026656836, 85.48336783597539, 86.32844624677423, 87.25292799818507, 87.98457306525253, 88.64248645889458, 89.22950401270452, 89.941298244619, 90.48294246093639, 91.13234835379859, 91.66548506933613, 92.17593511612738, 92.71474349885149, 93.29608938547486, 93.7328077588407, 94.25460114000511, 94.74236451805008, 95.2216203953152, 95.7405779428863, 96.2084904857783, 96.653716359924, 97.10177806766299, 97.49879477072285, 97.90715480815587, 98.36655985026799, 98.72671071661514, 99.09536908374217, 99.41298244619006, 99.68238663755211, 99.81850665002978, 100.0], "rescued": [0, 714, 1108, 1484, 1880, 2344, 2658, 2966, 3271, 3614, 3875, 4150, 4354, 4638, 4871, 5088, 5367, 5558, 5747, 5961, 6147, 6336, 6497, 6665, 6850, 7017, 7159, 7293, 7415, 7541, 7667, 7780, 7919, 8031, 8134, 8261, 8379, 8478, 8605, 8711, 8817, 8922, 9006, 9113, 9199, 9273, 9358, 9442, 9532, 9589, 9657, 9727, 9773, 9839, 9915, 9962, 10017, 10054, 10089, 10135, 10181, 10229, 10279, 10316, 10358, 10395, 10409, 10426, 10437, 10465, 10490, 10510, 10538, 10571, 10588, 10602, 10622, 10637, 10648, 10657, 10669, 10682, 10693, 10701, 10705, 10708, 10713, 10716, 10719, 10719, 10720, 10721, 10722, 10723, 10723, 10723, 10723, 10723, 10723, 10723, 10723], "harmed": [0, 38, 61, 81, 99, 133, 145, 163, 186, 214, 241, 271, 300, 329, 355, 379, 412, 444, 475, 508, 535, 559, 578, 600, 629, 656, 692, 717, 739, 768, 789, 815, 841, 865, 892, 915, 945, 970, 997, 1019, 1047, 1074, 1100, 1125, 1144, 1172, 1197, 1214, 1241, 1269, 1296, 1312, 1337, 1361, 1389, 1413, 1434, 1458, 1481, 1518, 1547, 1574, 1609, 1624, 1655, 1678, 1704, 1724, 1740, 1774, 1795, 1819, 1846, 1881, 1915, 1939, 1962, 1988, 2008, 2036, 2061, 2085, 2111, 2139, 2161, 2189, 2218, 2243, 2273, 2296, 2325, 2349, 2366, 2382, 2397, 2406, 2422, 2436, 2451, 2467, 2500], "exact_score": [26.688029946402747, 27.535944190794886, 28.304455094575054, 29.027592660862663, 29.76774522870998, 30.507897796557298, 31.23670703003148, 31.934322093979528, 32.55536965090889, 33.24164138048379, 33.893883106939285, 34.53761733261492, 35.124634886424865, 35.66911493633554, 36.21926665343278, 36.749567535377025, 37.237330913422, 37.7024076227207, 38.15897683123954, 38.66091937725095, 39.140175254516066, 39.585401128661765, 39.99943283328135, 40.473017043359896, 40.841675410486914, 41.241527947140064, 41.61302214786037, 41.99018801576724, 42.401383886793525, 42.787057255480256, 43.18974562572669, 43.57825482800669, 43.94124152794714, 44.31557156226072, 44.653035759861616, 44.964977455122934, 45.197515809772284, 45.44706916598134, 45.73916002608967, 46.00289255026515, 46.283640076000346, 46.50199926268327, 46.79692595638488, 47.01528514306781, 47.219465161784306, 47.47469018517994, 47.704392706236, 47.93409522729206, 48.149618580381706, 48.39350026940419, 48.60335195530726, 48.753651135751355, 48.94648782009472, 49.15633950599779, 49.349176190341154, 49.5448487082778, 49.68947622153532, 49.8426112355727, 50.0099254175765, 50.16306043161387, 50.26515044097213, 50.36440461673709, 50.49485296202819, 50.5685846354536, 50.642316308878996, 50.76709298698352, 50.877690497121634, 50.95992967132689, 51.03649717834558, 51.076198848651565, 51.10455718458441, 51.12440801973741, 51.181124691603095, 51.18963219238295, 51.23216969628223, 51.26903553299492, 51.317244704080764, 51.3853047103196, 51.376797209539745, 51.40515554547259, 51.37963304313303, 51.37963304313303, 51.36261804157331, 51.35978220798003, 51.34560304001361, 51.33993137282704, 51.342767206420326, 51.3285880384539, 51.28321470096135, 51.252020531435214, 51.21515469472251, 51.15560218926354, 51.05918384709185, 50.9967955080396, 50.880526330714915, 50.73022715027082, 50.61112213935286, 50.446643790942346, 50.338882114397535, 50.239627938632566, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 45.41766344619803, "threshold_area100": 46.88309276011684, "threshold_area25": 37.66072086889942}, "selected_budget": 51, "selected": {"val": {"score": 65.38056082648113, "calls": 50.99444795839483, "rescued": 2181, "harmed": 854}, "test": {"score": 50.55156963389389, "calls": 62.53580239911521, "rescued": 9727, "harmed": 1312}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 65.52111884180195, "calls": 47.99353433129524}, "test": {"score": 50.55156963389389, "calls": 60.86266057907722}}, "old_normalized_interior": {"val": {"score": 65.52111884180195, "calls": 47.99353433129524}, "test": {"score": 50.55156963389389, "calls": 60.86266057907722}}, "old_normalized_endpoints": {"val": {"score": 65.52111884180195, "calls": 47.99353433129524}, "test": {"score": 50.55156963389389, "calls": 60.86266057907722}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 56.68001967812214, 57.29847494553376, 57.90990231217935, 58.451050671164516, 58.900836320191154, 59.413873076112154, 59.80743551901047, 60.14477475578044, 60.531309297912706, 60.80539742778831, 61.26923887834703, 61.67685712277742, 61.91580574882283, 62.06339166490969, 62.33747979478529, 62.611567924660896, 62.82240494764213, 63.075409375219614, 63.34246960432919, 63.39869281045751, 63.49708342118209, 63.58141823037459, 63.70792044416332, 63.85550636025019, 64.09445498629559, 64.3193478108089, 64.38962681846931, 64.47396162766181, 64.58640803991848, 64.71993815447325, 64.75507765830346, 64.67777074987701, 64.67074284911097, 64.81832876519783, 64.92374727668845, 65.06430529200927, 65.09241689507344, 65.16269590273384, 65.21891910886217, 65.2751423149905, 65.35244922341697, 65.36650502494903, 65.40867242954529, 65.42272823107737, 65.4297561318434, 65.49300723873779, 65.49300723873779, 65.52111884180195, 65.45786773490758, 65.45083983414153, 65.47192353643966, 65.4438119333755, 65.41570033031134, 65.359477124183, 65.4297561318434, 65.46489563567361, 65.41570033031134, 65.38758872724716, 65.40164452877924, 65.42272823107737, 65.35244922341697, 65.34542132265092, 65.33839342188487, 65.34542132265092, 65.28919811652258, 65.24703071192634, 65.1556680019678, 65.11350059737156, 65.1205284981376, 65.1205284981376, 65.11350059737156, 65.09944479583947, 65.09241689507344, 65.0853889943074, 65.05024949047719, 65.04322158971115, 65.04322158971115, 65.05024949047719, 65.04322158971115, 65.0361936889451, 65.00105418511491, 64.9659146812847, 64.95888678051865, 64.91671937592241, 64.88860777285824, 64.86049616979408, 64.83941246749595, 64.76210555906951, 64.7269660552393, 64.66371494834493, 64.52315693302411, 64.4880174291939, 64.39665471923536, 64.32637571157494, 64.2842083069787, 64.22095720008433, 64.15067819242391, 64.12256658935975, 64.03120387940122, 64.12256658935975], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.98966898587392, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.98629559350622, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 56.68001967812214, 57.29847494553376, 57.90990231217935, 58.451050671164516, 58.900836320191154, 59.413873076112154, 59.80743551901047, 60.14477475578044, 60.531309297912706, 60.80539742778831, 61.26923887834703, 61.67685712277742, 61.91580574882283, 62.06339166490969, 62.33747979478529, 62.611567924660896, 62.82240494764213, 63.075409375219614, 63.34246960432919, 63.39869281045751, 63.49708342118209, 63.58141823037459, 63.70792044416332, 63.85550636025019, 64.09445498629559, 64.3193478108089, 64.38962681846931, 64.47396162766181, 64.58640803991848, 64.71993815447325, 64.75507765830346, 64.67777074987701, 64.67074284911097, 64.81832876519783, 64.92374727668845, 65.06430529200927, 65.09241689507344, 65.16269590273384, 65.21891910886217, 65.2751423149905, 65.35244922341697, 65.36650502494903, 65.40867242954529, 65.42272823107737, 65.4297561318434, 65.49300723873779, 65.49300723873779, 65.52111884180195, 65.45786773490758, 65.45083983414153, 65.47192353643966, 65.4438119333755, 65.41570033031134, 65.359477124183, 65.4297561318434, 65.46489563567361, 65.41570033031134, 65.38758872724716, 65.40164452877924, 65.42272823107737, 65.35244922341697, 65.34542132265092, 65.33839342188487, 65.34542132265092, 65.28919811652258, 65.24703071192634, 65.1556680019678, 65.11350059737156, 65.1205284981376, 65.1205284981376, 65.11350059737156, 65.09944479583947, 65.09241689507344, 65.0853889943074, 65.05024949047719, 65.04322158971115, 65.04322158971115, 65.05024949047719, 65.04322158971115, 65.0361936889451, 65.00105418511491, 64.9659146812847, 64.95888678051865, 64.91671937592241, 64.88860777285824, 64.86049616979408, 64.83941246749595, 64.76210555906951, 64.7269660552393, 64.66371494834493, 64.52315693302411, 64.4880174291939, 64.39665471923536, 64.32637571157494, 64.2842083069787, 64.22095720008433, 64.15067819242391, 64.12256658935975, 64.03120387940122, 64.12256658935975], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 61.10338042026846, "exact_area100": 64.04278023319658}, "test": {"score": [26.688029946402747, 28.65042679295579, 29.736551059183846, 30.768794487139495, 32.00521793381165, 33.16223803987182, 33.97045061395797, 34.91194736692851, 35.850608286305764, 36.67016419476505, 37.37912259308624, 37.97748348126932, 38.85375606159431, 39.42092278025125, 40.08734367467317, 40.7140628987891, 41.241527947140064, 41.84556050250971, 42.276607208689, 42.74451975158098, 43.28616396789837, 43.70303150611122, 44.21348155290248, 44.62751325752205, 44.99900745824235, 45.27975498397754, 45.60304001361201, 45.94050421121288, 46.26662507444064, 46.57856676970196, 46.87065762981029, 47.196778493038025, 47.51155602189264, 47.76394521169498, 47.959617729631624, 48.26872359129967, 48.50126194594901, 48.75081530215807, 48.92380115134844, 49.18753367552392, 49.4144003629867, 49.578878711397216, 49.749028726994304, 49.941865411337666, 50.13186626208774, 50.2708221081587, 50.37007628392366, 50.48918129484162, 50.55156963389389, 50.710376315117834, 50.832317159629085, 50.89754133227462, 50.97127300570003, 51.03366134475229, 51.050676346311995, 51.10455718458441, 51.17545302441653, 51.21515469472251, 51.26903553299492, 51.311573036894195, 51.39948387828602, 51.439185548592015, 51.430678047812165, 51.416498879845726, 51.40515554547259, 51.373961375946465, 51.37112554235318, 51.382468876726314, 51.35978220798003, 51.35978220798003, 51.35978220798003, 51.354110540793464, 51.348438873606895, 51.348438873606895, 51.348438873606895, 51.342767206420326, 51.33993137282704, 51.3285880384539, 51.317244704080764, 51.30590136970763, 51.29455803533449, 51.28321470096135, 51.26903553299492, 51.26619969940164, 51.26052803221507, 51.23500552987551, 51.23500552987551, 51.237841363468796, 51.243513030655365, 51.20664719394266, 51.18963219238295, 51.16127385645011, 51.16410969004339, 51.14993052207697, 51.14709468848368, 51.107393018177696, 51.042168845532146, 50.97978050647989, 50.89754133227462, 50.66783881121856, 50.00708958398321], "calls": [0.0, 2.413294387885319, 3.896435357173241, 5.331367155375323, 7.0527181464991635, 8.751382468876727, 9.993477582735446, 11.479454385616652, 13.095879533788956, 14.607378839009726, 16.03947480361852, 17.36948075886907, 18.98874174063466, 20.19113518418739, 21.6771119870686, 23.120551286050535, 24.529960581913056, 25.911011541842726, 26.957434137764796, 28.22788758755636, 29.61177438107932, 30.81416782463205, 32.07611377364376, 33.156566372685255, 34.46388565918952, 35.53299492385787, 36.57941751977994, 37.753452627399824, 38.947338570172704, 39.99376116609477, 41.29257295181919, 42.472279726625636, 43.632135666279105, 44.72393159969373, 45.64557751751127, 46.81961262513116, 47.97096106400476, 49.16484700677764, 50.07514959022205, 51.288886368147914, 52.21053228596546, 53.31083572015994, 54.5274083316791, 55.68159260414599, 56.76204520318747, 57.92473697643422, 58.914442900490606, 59.994895499532085, 60.86266057907722, 61.93744151093213, 63.09446161699231, 64.19476505118679, 65.11073930181777, 66.04089272041517, 67.1497036553895, 68.2074695856847, 69.45523636673, 70.54136063295806, 71.84867991946233, 72.93764001928366, 74.14854096361626, 75.11839605251964, 76.22720698749397, 77.45795876697954, 78.50438136290163, 79.38632561041318, 80.22573235402547, 80.94886992031307, 81.68618665456711, 84.87933528060574, 86.21217706944957, 87.26427133255821, 87.79740804809573, 88.2312905878683, 88.57442645265576, 88.89487564869694, 89.24368318067096, 89.552789042339, 89.78816323058163, 90.07458242350339, 90.2759266086266, 90.55100246717522, 90.83174999291042, 91.04443751240677, 91.31100587017555, 91.58891756231743, 91.86399342086607, 92.10787510988855, 92.42265263874316, 92.72041516603807, 93.15146187221734, 93.50310523778464, 93.84907693616539, 94.30281031109095, 94.7593795196098, 95.28968040155404, 95.80863794912514, 96.46938717636048, 97.26342058248022, 98.26730567450302, 100.0], "exact_score": [26.688029946402747, 27.541615857981455, 28.329977596914613, 29.061622663982078, 29.81879023338911, 30.539091966083433, 31.25939369877776, 31.957008762725803, 32.657459660267136, 33.31253722031591, 33.97328644755126, 34.61985650682018, 35.23523239656297, 35.8052349488132, 36.33269999716417, 36.880015880668125, 37.35643592433996, 37.81584096645209, 38.32912684683663, 38.86226356237416, 39.31316110370644, 39.778237813005134, 40.25182202308369, 40.65167455973684, 41.02033292686386, 41.44570796585656, 41.87959050562913, 42.28511470946885, 42.65944474378244, 43.064968947622155, 43.38258231007005, 43.79377818109633, 44.17377988259649, 44.58497575362278, 44.84303661061169, 45.126619969940165, 45.430054164421634, 45.72214502452996, 46.03408671979128, 46.27513257522049, 46.57856676970196, 46.825284292317725, 47.06065848056036, 47.3328985055157, 47.61081019765761, 47.820661883560675, 48.03618523665032, 48.30558942801236, 48.51260528032215, 48.711113631852086, 48.90395031619544, 49.13365283725151, 49.349176190341154, 49.53917704109123, 49.669625386382336, 49.80858123245328, 49.99291041601679, 50.16306043161387, 50.27932960893855, 50.37574795111023, 50.48918129484162, 50.582763803420015, 50.72739131667754, 50.823809658849214, 50.883362164308195, 50.95142217054703, 51.03082551115901, 51.056348013498564, 51.08187051583813, 51.15560218926354, 51.181124691603095, 51.22933386268894, 51.2747072001815, 51.32008053767405, 51.39097637750617, 51.430678047812165, 51.439185548592015, 51.4420213821853, 51.39664804469274, 51.382468876726314, 51.36545387516661, 51.382468876726314, 51.356946374386744, 51.356946374386744, 51.36261804157331, 51.35978220798003, 51.35978220798003, 51.354110540793464, 51.348438873606895, 51.34560304001361, 51.31440887048748, 51.26903553299492, 51.23500552987551, 51.1981396931628, 51.16978135722996, 51.121572186144114, 51.03366134475229, 50.945750503360465, 50.72739131667754, 50.50619629640133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 38.10509599296714, "exact_area100": 45.60176632922751}}, "old_normalized_endpoint_budget": 48}]}, {"expert": "groundingdino", "endpoint": "Qwen3.8", "task": "grefcoco", "small": 26.688029946402747, "large": 74.47182599325072, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.0, "validation_area100": 0.7511367489958397, "training_seconds": 95.38503635604866, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9922640919685364, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9862691164016724, "operator": "ge"}, {"budget": 0.03, "threshold": 0.9814407229423523, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9753608107566833, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9701423645019531, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9642951488494873, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9582424759864807, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9519225358963013, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9452207684516907, "operator": "ge"}, {"budget": 0.1, "threshold": 0.938125491142273, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9311085343360901, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9239449501037598, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9166353344917297, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9091670513153076, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9008194804191589, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8922799825668335, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8834959864616394, "operator": "ge"}, {"budget": 0.18, "threshold": 0.876700222492218, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8671596050262451, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8579201102256775, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8500425219535828, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8411612510681152, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8317926526069641, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8213933706283569, "operator": "ge"}, {"budget": 0.25, "threshold": 0.8119421601295471, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8033541440963745, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7937958836555481, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7831993699073792, "operator": "ge"}, {"budget": 0.29, "threshold": 0.7742490172386169, "operator": "ge"}, {"budget": 0.3, "threshold": 0.7644743323326111, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7535082697868347, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7427704334259033, "operator": "ge"}, {"budget": 0.33, "threshold": 0.7340201735496521, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7244006991386414, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7127642035484314, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7010344862937927, "operator": "ge"}, {"budget": 0.37, "threshold": 0.6904802322387695, "operator": "ge"}, {"budget": 0.38, "threshold": 0.6793418526649475, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6682717204093933, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6579185128211975, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6466652750968933, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6340805888175964, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6193563342094421, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6085940599441528, "operator": "ge"}, {"budget": 0.45, "threshold": 0.5964756011962891, "operator": "ge"}, {"budget": 0.46, "threshold": 0.5820612907409668, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5684112906455994, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5549193620681763, "operator": "ge"}, {"budget": 0.49, "threshold": 0.5395828485488892, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5255477428436279, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5084819197654724, "operator": "ge"}, {"budget": 0.52, "threshold": 0.49154043197631836, "operator": "ge"}, {"budget": 0.53, "threshold": 0.47681379318237305, "operator": "ge"}, {"budget": 0.54, "threshold": 0.45811256766319275, "operator": "ge"}, {"budget": 0.55, "threshold": 0.44045373797416687, "operator": "ge"}, {"budget": 0.56, "threshold": 0.4224051237106323, "operator": "ge"}, {"budget": 0.57, "threshold": 0.40377265214920044, "operator": "ge"}, {"budget": 0.58, "threshold": 0.3830324411392212, "operator": "ge"}, {"budget": 0.59, "threshold": 0.3621925115585327, "operator": "ge"}, {"budget": 0.6, "threshold": 0.3452909290790558, "operator": "ge"}, {"budget": 0.61, "threshold": 0.32318127155303955, "operator": "ge"}, {"budget": 0.62, "threshold": 0.3027627468109131, "operator": "ge"}, {"budget": 0.63, "threshold": 0.280211478471756, "operator": "ge"}, {"budget": 0.64, "threshold": 0.25297194719314575, "operator": "ge"}, {"budget": 0.65, "threshold": 0.22940964996814728, "operator": "ge"}, {"budget": 0.66, "threshold": 0.20193518698215485, "operator": "ge"}, {"budget": 0.67, "threshold": 0.17596210539340973, "operator": "ge"}, {"budget": 0.68, "threshold": 0.14982692897319794, "operator": "ge"}, {"budget": 0.69, "threshold": 0.12638117372989655, "operator": "ge"}, {"budget": 0.7, "threshold": 0.10382703691720963, "operator": "ge"}, {"budget": 0.71, "threshold": 0.08035468310117722, "operator": "ge"}, {"budget": 0.72, "threshold": 0.06156225502490997, "operator": "ge"}, {"budget": 0.73, "threshold": 0.04408211261034012, "operator": "ge"}, {"budget": 0.74, "threshold": 0.03048100508749485, "operator": "ge"}, {"budget": 0.75, "threshold": 0.019863378256559372, "operator": "ge"}, {"budget": 0.76, "threshold": 0.011572040617465973, "operator": "ge"}, {"budget": 0.77, "threshold": 0.006126069463789463, "operator": "ge"}, {"budget": 0.78, "threshold": 0.002978223841637373, "operator": "ge"}, {"budget": 0.79, "threshold": 0.001286243088543415, "operator": "ge"}, {"budget": 0.8, "threshold": 0.0002584515605121851, "operator": "ge"}, {"budget": 0.81, "threshold": -0.0002134649985237047, "operator": "ge"}, {"budget": 0.82, "threshold": -0.0005554808303713799, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0009417837136425078, "operator": "ge"}, {"budget": 0.84, "threshold": -0.0013675017980858684, "operator": "ge"}, {"budget": 0.85, "threshold": -0.0019756844267249107, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0027503566816449165, "operator": "ge"}, {"budget": 0.87, "threshold": -0.0036324213724583387, "operator": "ge"}, {"budget": 0.88, "threshold": -0.004544638562947512, "operator": "ge"}, {"budget": 0.89, "threshold": -0.005598927848041058, "operator": "ge"}, {"budget": 0.9, "threshold": -0.0067649525590240955, "operator": "ge"}, {"budget": 0.91, "threshold": -0.008380209095776081, "operator": "ge"}, {"budget": 0.92, "threshold": -0.010450153611600399, "operator": "ge"}, {"budget": 0.93, "threshold": -0.0132239805534482, "operator": "ge"}, {"budget": 0.94, "threshold": -0.016601525247097015, "operator": "ge"}, {"budget": 0.95, "threshold": -0.021811814978718758, "operator": "ge"}, {"budget": 0.96, "threshold": -0.03221253305673599, "operator": "ge"}, {"budget": 0.97, "threshold": -0.05021269991993904, "operator": "ge"}, {"budget": 0.98, "threshold": -0.08189468085765839, "operator": "ge"}, {"budget": 0.99, "threshold": -0.14708127081394196, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.96816360952983, 57.811511701454776, 58.577552884953256, 59.39278937381404, 60.16585845807857, 60.924871740811014, 61.70496872584159, 62.281256588656966, 62.94890716143088, 63.61655773420479, 64.23501300161641, 64.7972450628997, 65.359477124183, 65.94982078853047, 66.38555063602502, 66.87047578888186, 67.39756834633495, 67.88952139995783, 68.35336285051655, 68.79612059877715, 69.22482254550565, 69.62541288917001, 69.9486963244079, 70.39848197343453, 70.7358212102045, 71.06613254620844, 71.45969498910675, 71.90948063813339, 72.26087567643545, 72.73877292852625, 73.04097266146601, 73.36425609670391, 73.69456743270784, 74.03190666947782, 74.39735750931197, 74.69955724225173, 74.91039426523297, 75.14934289127837, 75.42343102115397, 75.69049125026355, 76.01377468550143, 76.26677911307893, 76.59006254831682, 76.85009487666034, 77.06093189964157, 77.27879682338884, 77.55288495326445, 77.79183357930985, 77.99564270152504, 78.21350762527233, 78.42434464825357, 78.6843769765971, 78.8741302972802, 79.00063251106893, 79.0287441141331, 79.16930212945392, 79.30986014477475, 79.45744606086161, 79.65422728231077, 79.80181319839762, 79.90020380912222, 80.03373392367699, 80.12509663363552, 80.25862674819031, 80.37810106121302, 80.50460327500176, 80.56785438189613, 80.57488228266216, 80.64516129032258, 80.69435659568487, 80.75760770257924, 80.94033312249631, 80.98250052709255, 81.01764003092275, 81.08089113781713, 81.10900274088128, 81.17225384777566, 81.20739335160587, 81.19333755007378, 81.20036545083983, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.2144212523719, 81.2144212523719, 81.22144915313795, 81.20739335160587, 81.17225384777566, 81.17225384777566, 81.14414224471149, 81.12305854241337, 81.10197484011525, 81.03872373322089, 80.97547262632652, 80.8700541148359, 80.74355190104716, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 130, 256, 376, 499, 617, 731, 847, 942, 1047, 1152, 1250, 1340, 1431, 1528, 1603, 1688, 1769, 1854, 1936, 2016, 2091, 2161, 2230, 2304, 2368, 2433, 2504, 2582, 2655, 2730, 2792, 2856, 2918, 2978, 3046, 3103, 3158, 3216, 3273, 3327, 3386, 3439, 3496, 3549, 3595, 3639, 3689, 3743, 3788, 3834, 3879, 3930, 3973, 4011, 4035, 4071, 4110, 4146, 4183, 4221, 4250, 4280, 4309, 4338, 4372, 4399, 4423, 4439, 4462, 4480, 4505, 4539, 4551, 4563, 4574, 4586, 4599, 4605, 4607, 4609, 4611, 4612, 4612, 4612, 4612, 4613, 4616, 4617, 4618, 4618, 4619, 4621, 4621, 4623, 4623, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 6, 17, 24, 32, 38, 43, 56, 66, 76, 86, 96, 107, 120, 133, 149, 155, 170, 186, 203, 217, 230, 253, 263, 279, 297, 312, 326, 349, 356, 375, 393, 408, 420, 436, 450, 475, 499, 517, 533, 546, 563, 574, 590, 606, 619, 630, 650, 666, 681, 696, 710, 726, 746, 766, 782, 801, 816, 825, 842, 857, 868, 884, 894, 911, 920, 935, 950, 963, 974, 990, 998, 1004, 1011, 1013, 1021, 1025, 1026, 1030, 1031, 1034, 1034, 1034, 1034, 1034, 1035, 1036, 1037, 1037, 1039, 1045, 1047, 1051, 1056, 1059, 1071, 1080, 1095, 1113, 1133], "exact_score": [56.05453650994448, 56.96816360952983, 57.811511701454776, 58.577552884953256, 59.39278937381404, 60.16585845807857, 60.924871740811014, 61.70496872584159, 62.281256588656966, 62.94890716143088, 63.61655773420479, 64.23501300161641, 64.7972450628997, 65.359477124183, 65.94982078853047, 66.38555063602502, 66.87047578888186, 67.39756834633495, 67.88952139995783, 68.35336285051655, 68.79612059877715, 69.22482254550565, 69.62541288917001, 69.9486963244079, 70.39848197343453, 70.7358212102045, 71.06613254620844, 71.45969498910675, 71.90948063813339, 72.26087567643545, 72.73877292852625, 73.04097266146601, 73.36425609670391, 73.69456743270784, 74.03190666947782, 74.39735750931197, 74.69955724225173, 74.91039426523297, 75.14934289127837, 75.42343102115397, 75.69049125026355, 76.01377468550143, 76.26677911307893, 76.59006254831682, 76.85009487666034, 77.06093189964157, 77.27879682338884, 77.55288495326445, 77.79183357930985, 77.99564270152504, 78.21350762527233, 78.42434464825357, 78.6843769765971, 78.8741302972802, 79.00063251106893, 79.0287441141331, 79.16930212945392, 79.30986014477475, 79.45744606086161, 79.65422728231077, 79.80181319839762, 79.90020380912222, 80.03373392367699, 80.12509663363552, 80.25862674819031, 80.37810106121302, 80.50460327500176, 80.56785438189613, 80.57488228266216, 80.64516129032258, 80.69435659568487, 80.75760770257924, 80.94033312249631, 80.98250052709255, 81.01764003092275, 81.08089113781713, 81.10900274088128, 81.17225384777566, 81.20739335160587, 81.19333755007378, 81.20036545083983, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.2144212523719, 81.2144212523719, 81.22144915313795, 81.20739335160587, 81.17225384777566, 81.17225384777566, 81.14414224471149, 81.12305854241337, 81.10197484011525, 81.03872373322089, 80.97547262632652, 80.8700541148359, 80.74355190104716, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.11367489958397, "threshold_area100": 75.11283294679879, "threshold_area25": 64.47185325743202}, "test": {"score": [26.688029946402747, 27.918781725888326, 29.577744377959903, 30.873720330091032, 32.53268298216261, 33.68119558744293, 35.02821654425319, 36.39508833621643, 37.546436775090044, 38.845248560814454, 40.135552845759015, 41.29824461900576, 42.46377222584579, 43.51870232254772, 44.5821399200295, 45.61438334798515, 46.692000113433345, 47.721407707795706, 48.387828602217624, 49.380370359867285, 50.214105436292996, 50.835152993222366, 51.63202223293537, 52.383518135155825, 53.152029038936, 53.792927431018356, 54.36860165045515, 55.03785837847035, 55.78651844709752, 56.450103507926165, 57.01159855939654, 57.64398945069904, 58.2281711709157, 58.74145705130023, 59.26325043246463, 59.819073816748435, 60.482658877577066, 61.10937810169299, 61.63400731645068, 62.152964864021776, 62.62654907410033, 63.142670788078156, 63.687150837988824, 64.21461588633979, 64.73073760031762, 65.22133681195588, 65.81969770013896, 66.36417775004963, 66.87746363043416, 67.30283866942688, 67.75657204435244, 68.23015625443098, 68.72926296684912, 69.16030967302838, 69.65090888466665, 70.05926892209966, 70.43076312281995, 70.87882483055894, 71.30987153673823, 71.71823157417123, 72.06136743895868, 72.45554830842526, 72.77599750446643, 73.03405836145535, 73.3771942262428, 73.61540424807873, 73.84227093554149, 74.08331679097071, 74.33003431358648, 74.50018432918357, 74.65899101040752, 74.79794685647848, 74.90854436661657, 74.9964552080084, 75.06735104784052, 75.07585854862036, 75.08720188299351, 75.10988855173979, 75.1070527181465, 75.10988855173979, 75.1070527181465, 75.10421688455322, 75.09287355018009, 75.09287355018009, 75.09854521736665, 75.10138105095993, 75.1070527181465, 75.1070527181465, 75.10421688455322, 75.10988855173979, 75.10421688455322, 75.1070527181465, 75.1070527181465, 75.11839605251964, 75.11556021892636, 75.10421688455322, 75.07585854862036, 75.03048521112781, 74.98511187363526, 74.78660352210534, 74.47182599325072], "calls": [0.0, 1.3044834529109832, 3.0541927799676714, 4.489124578169753, 6.366446416924256, 7.685109037801661, 9.287355018007544, 10.949153503672404, 12.412443637807334, 14.034540453166208, 15.633950599778803, 17.18798740889885, 18.64277004225392, 20.09471684201571, 21.543827808184215, 22.984431273572866, 24.46190057567422, 25.922354876215863, 26.88086663074611, 28.335649264101182, 29.63162521623231, 30.655361143408104, 31.815217083061565, 32.88432634772992, 34.14910813033491, 35.189859059070415, 36.1029974761081, 37.24016674701529, 38.49927686243371, 39.585401128661765, 40.66301789410998, 41.734962992371614, 42.82675892578624, 43.7257181748575, 44.61900575674219, 45.66826418625755, 46.89334429855656, 47.93409522729206, 49.03439866148654, 50.10350792615489, 50.97410883929331, 51.89008308992429, 53.06695403113745, 54.22113830360434, 55.14845588860846, 56.16368431500439, 57.278166917165294, 58.41817202166577, 59.526982956640104, 60.6045997220883, 61.56878314380512, 62.711624081898876, 63.868644187959056, 64.89521594872812, 66.05223605478831, 67.11850948586337, 68.210305419278, 69.29642968550606, 70.43359895641323, 71.57076822732043, 72.47256330998498, 73.47361256841448, 74.45197515809772, 75.3849644102884, 76.57885035306128, 77.47213793494598, 78.42497802228965, 79.37498227604004, 80.37319570087628, 81.24663244760796, 82.09454669200011, 83.08992428324305, 83.96336102997476, 84.94739528684457, 85.74993619374415, 86.5581487678303, 87.44576468252842, 88.4156197714318, 89.29756401894336, 90.13413492896237, 91.00190000850749, 91.65130590136971, 92.08802427473556, 92.46519014264243, 92.7686243371239, 93.07205853160536, 93.44071689873238, 93.77534526273999, 94.10146612596773, 94.39639281966934, 94.7565436860165, 95.13654538751666, 95.4825170858974, 95.83983211865129, 96.22834132093129, 96.65655219351729, 97.23789808014067, 97.85894563707002, 98.37223151745455, 99.12939908686158, 100.0], "rescued": [0, 442, 1036, 1508, 2110, 2532, 3030, 3539, 3967, 4455, 4933, 5381, 5812, 6222, 6630, 7024, 7439, 7835, 8093, 8475, 8803, 9055, 9360, 9643, 9951, 10204, 10422, 10687, 10977, 11232, 11461, 11711, 11944, 12140, 12344, 12567, 12824, 13067, 13287, 13491, 13676, 13870, 14086, 14294, 14491, 14680, 14907, 15125, 15328, 15506, 15682, 15879, 16080, 16256, 16442, 16609, 16763, 16940, 17106, 17268, 17402, 17559, 17689, 17791, 17930, 18035, 18127, 18231, 18331, 18402, 18476, 18546, 18597, 18643, 18677, 18696, 18712, 18723, 18727, 18729, 18729, 18730, 18730, 18730, 18732, 18734, 18737, 18740, 18740, 18744, 18748, 18753, 18757, 18763, 18765, 18767, 18771, 18777, 18795, 18807, 18820], "harmed": [0, 8, 17, 32, 49, 66, 89, 116, 138, 168, 191, 229, 249, 287, 320, 350, 385, 418, 441, 473, 507, 540, 564, 582, 619, 646, 661, 690, 716, 737, 768, 795, 822, 837, 857, 884, 907, 929, 964, 985, 1003, 1015, 1039, 1061, 1076, 1092, 1108, 1134, 1156, 1184, 1200, 1230, 1255, 1279, 1292, 1315, 1338, 1357, 1371, 1389, 1402, 1420, 1437, 1448, 1466, 1487, 1499, 1518, 1531, 1542, 1560, 1581, 1593, 1608, 1617, 1633, 1645, 1648, 1653, 1654, 1655, 1657, 1661, 1661, 1661, 1662, 1663, 1666, 1667, 1669, 1675, 1679, 1683, 1685, 1688, 1694, 1708, 1730, 1764, 1846, 1970], "exact_score": [26.688029946402747, 27.635198366559855, 28.57102345234382, 29.523863539687493, 30.422822788758758, 31.327453705016595, 32.22074128690129, 33.1111930351927, 33.95059977880498, 34.784334855230696, 35.63224909962284, 36.443297507302276, 37.2004650767093, 38.02569265235516, 38.81972605847489, 39.61092363100134, 40.413464537900914, 41.15928877293481, 41.93347134390154, 42.71616141564814, 43.44780648271559, 44.16243654822335, 44.87423078013782, 45.628562515951565, 46.38005841817202, 47.07483764852679, 47.76961687888155, 48.47290361001617, 49.15633950599779, 49.82559623401299, 50.446643790942346, 51.07903468224485, 51.75112724385333, 52.45724980858124, 53.0414315287979, 53.68800158806681, 54.311884978589454, 54.904574199585966, 55.50577092136233, 56.07293764001928, 56.665626861015795, 57.21577857811304, 57.81413946629612, 58.32175367949408, 58.92578623486374, 59.45892295040127, 59.986387998752235, 60.550718883815904, 61.14057227121913, 61.616992314890965, 62.10475569293594, 62.63505657488019, 63.19087995916399, 63.65028500127612, 64.13237671213453, 64.64282675892579, 65.15611263931032, 65.69775685562772, 66.18835606726597, 66.63641777500496, 67.07030031477753, 67.51552618892323, 67.93806539432265, 68.36627626690866, 68.78881547230809, 69.20568301052094, 69.62255054873381, 70.00822391742052, 70.35703144939455, 70.77957065479397, 71.13688568754786, 71.48569321952188, 71.88554575617503, 72.23718912174235, 72.62569832402235, 72.92346085131724, 73.18719337549273, 73.50197090434733, 73.74868842696311, 73.99257011598559, 74.26764597453422, 74.44630349091115, 74.65048350962766, 74.7922751892919, 74.90854436661657, 75.00496270878824, 75.07302271502708, 75.08153021580695, 75.1070527181465, 75.11272438533308, 75.10988855173979, 75.1070527181465, 75.09570938377337, 75.10138105095993, 75.10421688455322, 75.10421688455322, 75.11272438533308, 75.08720188299351, 75.02197771034795, 74.82063352522475, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.754182194104786, "threshold_area100": 62.659785043813635, "threshold_area25": 41.977143181238134}, "selected_budget": 89, "selected": {"val": {"score": 81.22144915313795, "calls": 88.9943074003795, "rescued": 4618, "harmed": 1037}, "test": {"score": 75.10988855173979, "calls": 94.39639281966934, "rescued": 18744, "harmed": 1669}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9922640919685364, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9862691164016724, "operator": "ge"}, {"budget": 0.03, "threshold": 0.9814407229423523, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9753608107566833, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9701423645019531, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9642951488494873, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9582424759864807, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9519225358963013, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9452207684516907, "operator": "ge"}, {"budget": 0.1, "threshold": 0.938125491142273, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9311085343360901, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9239449501037598, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9166353344917297, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9091670513153076, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9008194804191589, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8922799825668335, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8834959864616394, "operator": "ge"}, {"budget": 0.18, "threshold": 0.876700222492218, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8671596050262451, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8579201102256775, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8500425219535828, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8411612510681152, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8317926526069641, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8213933706283569, "operator": "ge"}, {"budget": 0.25, "threshold": 0.8119421601295471, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8033541440963745, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7937958836555481, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7831993699073792, "operator": "ge"}, {"budget": 0.29, "threshold": 0.7742490172386169, "operator": "ge"}, {"budget": 0.3, "threshold": 0.7644743323326111, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7535082697868347, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7427704334259033, "operator": "ge"}, {"budget": 0.33, "threshold": 0.7340201735496521, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7244006991386414, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7127642035484314, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7010344862937927, "operator": "ge"}, {"budget": 0.37, "threshold": 0.6904802322387695, "operator": "ge"}, {"budget": 0.38, "threshold": 0.6793418526649475, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6682717204093933, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6579185128211975, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6466652750968933, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6340805888175964, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6193563342094421, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6085940599441528, "operator": "ge"}, {"budget": 0.45, "threshold": 0.5964756011962891, "operator": "ge"}, {"budget": 0.46, "threshold": 0.5820612907409668, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5684112906455994, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5549193620681763, "operator": "ge"}, {"budget": 0.49, "threshold": 0.5395828485488892, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5255477428436279, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5084819197654724, "operator": "ge"}, {"budget": 0.52, "threshold": 0.49154043197631836, "operator": "ge"}, {"budget": 0.53, "threshold": 0.47681379318237305, "operator": "ge"}, {"budget": 0.54, "threshold": 0.45811256766319275, "operator": "ge"}, {"budget": 0.55, "threshold": 0.44045373797416687, "operator": "ge"}, {"budget": 0.56, "threshold": 0.4224051237106323, "operator": "ge"}, {"budget": 0.57, "threshold": 0.40377265214920044, "operator": "ge"}, {"budget": 0.58, "threshold": 0.3830324411392212, "operator": "ge"}, {"budget": 0.59, "threshold": 0.3621925115585327, "operator": "ge"}, {"budget": 0.6, "threshold": 0.3452909290790558, "operator": "ge"}, {"budget": 0.61, "threshold": 0.32318127155303955, "operator": "ge"}, {"budget": 0.62, "threshold": 0.3027627468109131, "operator": "ge"}, {"budget": 0.63, "threshold": 0.280211478471756, "operator": "ge"}, {"budget": 0.64, "threshold": 0.25297194719314575, "operator": "ge"}, {"budget": 0.65, "threshold": 0.22940964996814728, "operator": "ge"}, {"budget": 0.66, "threshold": 0.20193518698215485, "operator": "ge"}, {"budget": 0.67, "threshold": 0.17596210539340973, "operator": "ge"}, {"budget": 0.68, "threshold": 0.14982692897319794, "operator": "ge"}, {"budget": 0.69, "threshold": 0.12638117372989655, "operator": "ge"}, {"budget": 0.7, "threshold": 0.10382703691720963, "operator": "ge"}, {"budget": 0.71, "threshold": 0.08035468310117722, "operator": "ge"}, {"budget": 0.72, "threshold": 0.06156225502490997, "operator": "ge"}, {"budget": 0.73, "threshold": 0.04408211261034012, "operator": "ge"}, {"budget": 0.74, "threshold": 0.03048100508749485, "operator": "ge"}, {"budget": 0.75, "threshold": 0.019863378256559372, "operator": "ge"}, {"budget": 0.76, "threshold": 0.011572040617465973, "operator": "ge"}, {"budget": 0.77, "threshold": 0.006126069463789463, "operator": "ge"}, {"budget": 0.78, "threshold": 0.002978223841637373, "operator": "ge"}, {"budget": 0.79, "threshold": 0.001286243088543415, "operator": "ge"}, {"budget": 0.8, "threshold": 0.0002584515605121851, "operator": "ge"}, {"budget": 0.81, "threshold": -0.0002134649985237047, "operator": "ge"}, {"budget": 0.82, "threshold": -0.0005554808303713799, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0009417837136425078, "operator": "ge"}, {"budget": 0.84, "threshold": -0.0013675017980858684, "operator": "ge"}, {"budget": 0.85, "threshold": -0.0019756844267249107, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0027503566816449165, "operator": "ge"}, {"budget": 0.87, "threshold": -0.0036324213724583387, "operator": "ge"}, {"budget": 0.88, "threshold": -0.004544638562947512, "operator": "ge"}, {"budget": 0.89, "threshold": -0.005598927848041058, "operator": "ge"}, {"budget": 0.9, "threshold": -0.0067649525590240955, "operator": "ge"}, {"budget": 0.91, "threshold": -0.008380209095776081, "operator": "ge"}, {"budget": 0.92, "threshold": -0.010450153611600399, "operator": "ge"}, {"budget": 0.93, "threshold": -0.0132239805534482, "operator": "ge"}, {"budget": 0.94, "threshold": -0.016601525247097015, "operator": "ge"}, {"budget": 0.95, "threshold": -0.021811814978718758, "operator": "ge"}, {"budget": 0.96, "threshold": -0.03221253305673599, "operator": "ge"}, {"budget": 0.97, "threshold": -0.05021269991993904, "operator": "ge"}, {"budget": 0.98, "threshold": -0.08189468085765839, "operator": "ge"}, {"budget": 0.99, "threshold": -0.14708127081394196, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.96816360952983, 57.811511701454776, 58.577552884953256, 59.39278937381404, 60.16585845807857, 60.924871740811014, 61.70496872584159, 62.281256588656966, 62.94890716143088, 63.61655773420479, 64.23501300161641, 64.7972450628997, 65.359477124183, 65.94982078853047, 66.38555063602502, 66.87047578888186, 67.39756834633495, 67.88952139995783, 68.35336285051655, 68.79612059877715, 69.22482254550565, 69.62541288917001, 69.9486963244079, 70.39848197343453, 70.7358212102045, 71.06613254620844, 71.45969498910675, 71.90948063813339, 72.26087567643545, 72.73877292852625, 73.04097266146601, 73.36425609670391, 73.69456743270784, 74.03190666947782, 74.39735750931197, 74.69955724225173, 74.91039426523297, 75.14934289127837, 75.42343102115397, 75.69049125026355, 76.01377468550143, 76.26677911307893, 76.59006254831682, 76.85009487666034, 77.06093189964157, 77.27879682338884, 77.55288495326445, 77.79183357930985, 77.99564270152504, 78.21350762527233, 78.42434464825357, 78.6843769765971, 78.8741302972802, 79.00063251106893, 79.0287441141331, 79.16930212945392, 79.30986014477475, 79.45744606086161, 79.65422728231077, 79.80181319839762, 79.90020380912222, 80.03373392367699, 80.12509663363552, 80.25862674819031, 80.37810106121302, 80.50460327500176, 80.56785438189613, 80.57488228266216, 80.64516129032258, 80.69435659568487, 80.75760770257924, 80.94033312249631, 80.98250052709255, 81.01764003092275, 81.08089113781713, 81.10900274088128, 81.17225384777566, 81.20739335160587, 81.19333755007378, 81.20036545083983, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.2144212523719, 81.2144212523719, 81.22144915313795, 81.20739335160587, 81.17225384777566, 81.17225384777566, 81.14414224471149, 81.12305854241337, 81.10197484011525, 81.03872373322089, 80.97547262632652, 80.8700541148359, 80.74355190104716, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 130, 256, 376, 499, 617, 731, 847, 942, 1047, 1152, 1250, 1340, 1431, 1528, 1603, 1688, 1769, 1854, 1936, 2016, 2091, 2161, 2230, 2304, 2368, 2433, 2504, 2582, 2655, 2730, 2792, 2856, 2918, 2978, 3046, 3103, 3158, 3216, 3273, 3327, 3386, 3439, 3496, 3549, 3595, 3639, 3689, 3743, 3788, 3834, 3879, 3930, 3973, 4011, 4035, 4071, 4110, 4146, 4183, 4221, 4250, 4280, 4309, 4338, 4372, 4399, 4423, 4439, 4462, 4480, 4505, 4539, 4551, 4563, 4574, 4586, 4599, 4605, 4607, 4609, 4611, 4612, 4612, 4612, 4612, 4613, 4616, 4617, 4618, 4618, 4619, 4621, 4621, 4623, 4623, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 6, 17, 24, 32, 38, 43, 56, 66, 76, 86, 96, 107, 120, 133, 149, 155, 170, 186, 203, 217, 230, 253, 263, 279, 297, 312, 326, 349, 356, 375, 393, 408, 420, 436, 450, 475, 499, 517, 533, 546, 563, 574, 590, 606, 619, 630, 650, 666, 681, 696, 710, 726, 746, 766, 782, 801, 816, 825, 842, 857, 868, 884, 894, 911, 920, 935, 950, 963, 974, 990, 998, 1004, 1011, 1013, 1021, 1025, 1026, 1030, 1031, 1034, 1034, 1034, 1034, 1034, 1035, 1036, 1037, 1037, 1039, 1045, 1047, 1051, 1056, 1059, 1071, 1080, 1095, 1113, 1133], "exact_score": [56.05453650994448, 56.96816360952983, 57.811511701454776, 58.577552884953256, 59.39278937381404, 60.16585845807857, 60.924871740811014, 61.70496872584159, 62.281256588656966, 62.94890716143088, 63.61655773420479, 64.23501300161641, 64.7972450628997, 65.359477124183, 65.94982078853047, 66.38555063602502, 66.87047578888186, 67.39756834633495, 67.88952139995783, 68.35336285051655, 68.79612059877715, 69.22482254550565, 69.62541288917001, 69.9486963244079, 70.39848197343453, 70.7358212102045, 71.06613254620844, 71.45969498910675, 71.90948063813339, 72.26087567643545, 72.73877292852625, 73.04097266146601, 73.36425609670391, 73.69456743270784, 74.03190666947782, 74.39735750931197, 74.69955724225173, 74.91039426523297, 75.14934289127837, 75.42343102115397, 75.69049125026355, 76.01377468550143, 76.26677911307893, 76.59006254831682, 76.85009487666034, 77.06093189964157, 77.27879682338884, 77.55288495326445, 77.79183357930985, 77.99564270152504, 78.21350762527233, 78.42434464825357, 78.6843769765971, 78.8741302972802, 79.00063251106893, 79.0287441141331, 79.16930212945392, 79.30986014477475, 79.45744606086161, 79.65422728231077, 79.80181319839762, 79.90020380912222, 80.03373392367699, 80.12509663363552, 80.25862674819031, 80.37810106121302, 80.50460327500176, 80.56785438189613, 80.57488228266216, 80.64516129032258, 80.69435659568487, 80.75760770257924, 80.94033312249631, 80.98250052709255, 81.01764003092275, 81.08089113781713, 81.10900274088128, 81.17225384777566, 81.20739335160587, 81.19333755007378, 81.20036545083983, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.2144212523719, 81.2144212523719, 81.22144915313795, 81.20739335160587, 81.17225384777566, 81.17225384777566, 81.14414224471149, 81.12305854241337, 81.10197484011525, 81.03872373322089, 80.97547262632652, 80.8700541148359, 80.74355190104716, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.11367489958397, "threshold_area100": 75.11283294679879, "threshold_area25": 64.47185325743202}, "test": {"score": [26.688029946402747, 27.918781725888326, 29.577744377959903, 30.873720330091032, 32.53268298216261, 33.68119558744293, 35.02821654425319, 36.39508833621643, 37.546436775090044, 38.845248560814454, 40.135552845759015, 41.29824461900576, 42.46377222584579, 43.51870232254772, 44.5821399200295, 45.61438334798515, 46.692000113433345, 47.721407707795706, 48.387828602217624, 49.380370359867285, 50.214105436292996, 50.835152993222366, 51.63202223293537, 52.383518135155825, 53.152029038936, 53.792927431018356, 54.36860165045515, 55.03785837847035, 55.78651844709752, 56.450103507926165, 57.01159855939654, 57.64398945069904, 58.2281711709157, 58.74145705130023, 59.26325043246463, 59.819073816748435, 60.482658877577066, 61.10937810169299, 61.63400731645068, 62.152964864021776, 62.62654907410033, 63.142670788078156, 63.687150837988824, 64.21461588633979, 64.73073760031762, 65.22133681195588, 65.81969770013896, 66.36417775004963, 66.87746363043416, 67.30283866942688, 67.75657204435244, 68.23015625443098, 68.72926296684912, 69.16030967302838, 69.65090888466665, 70.05926892209966, 70.43076312281995, 70.87882483055894, 71.30987153673823, 71.71823157417123, 72.06136743895868, 72.45554830842526, 72.77599750446643, 73.03405836145535, 73.3771942262428, 73.61540424807873, 73.84227093554149, 74.08331679097071, 74.33003431358648, 74.50018432918357, 74.65899101040752, 74.79794685647848, 74.90854436661657, 74.9964552080084, 75.06735104784052, 75.07585854862036, 75.08720188299351, 75.10988855173979, 75.1070527181465, 75.10988855173979, 75.1070527181465, 75.10421688455322, 75.09287355018009, 75.09287355018009, 75.09854521736665, 75.10138105095993, 75.1070527181465, 75.1070527181465, 75.10421688455322, 75.10988855173979, 75.10421688455322, 75.1070527181465, 75.1070527181465, 75.11839605251964, 75.11556021892636, 75.10421688455322, 75.07585854862036, 75.03048521112781, 74.98511187363526, 74.78660352210534, 74.47182599325072], "calls": [0.0, 1.3044834529109832, 3.0541927799676714, 4.489124578169753, 6.366446416924256, 7.685109037801661, 9.287355018007544, 10.949153503672404, 12.412443637807334, 14.034540453166208, 15.633950599778803, 17.18798740889885, 18.64277004225392, 20.09471684201571, 21.543827808184215, 22.984431273572866, 24.46190057567422, 25.922354876215863, 26.88086663074611, 28.335649264101182, 29.63162521623231, 30.655361143408104, 31.815217083061565, 32.88432634772992, 34.14910813033491, 35.189859059070415, 36.1029974761081, 37.24016674701529, 38.49927686243371, 39.585401128661765, 40.66301789410998, 41.734962992371614, 42.82675892578624, 43.7257181748575, 44.61900575674219, 45.66826418625755, 46.89334429855656, 47.93409522729206, 49.03439866148654, 50.10350792615489, 50.97410883929331, 51.89008308992429, 53.06695403113745, 54.22113830360434, 55.14845588860846, 56.16368431500439, 57.278166917165294, 58.41817202166577, 59.526982956640104, 60.6045997220883, 61.56878314380512, 62.711624081898876, 63.868644187959056, 64.89521594872812, 66.05223605478831, 67.11850948586337, 68.210305419278, 69.29642968550606, 70.43359895641323, 71.57076822732043, 72.47256330998498, 73.47361256841448, 74.45197515809772, 75.3849644102884, 76.57885035306128, 77.47213793494598, 78.42497802228965, 79.37498227604004, 80.37319570087628, 81.24663244760796, 82.09454669200011, 83.08992428324305, 83.96336102997476, 84.94739528684457, 85.74993619374415, 86.5581487678303, 87.44576468252842, 88.4156197714318, 89.29756401894336, 90.13413492896237, 91.00190000850749, 91.65130590136971, 92.08802427473556, 92.46519014264243, 92.7686243371239, 93.07205853160536, 93.44071689873238, 93.77534526273999, 94.10146612596773, 94.39639281966934, 94.7565436860165, 95.13654538751666, 95.4825170858974, 95.83983211865129, 96.22834132093129, 96.65655219351729, 97.23789808014067, 97.85894563707002, 98.37223151745455, 99.12939908686158, 100.0], "rescued": [0, 442, 1036, 1508, 2110, 2532, 3030, 3539, 3967, 4455, 4933, 5381, 5812, 6222, 6630, 7024, 7439, 7835, 8093, 8475, 8803, 9055, 9360, 9643, 9951, 10204, 10422, 10687, 10977, 11232, 11461, 11711, 11944, 12140, 12344, 12567, 12824, 13067, 13287, 13491, 13676, 13870, 14086, 14294, 14491, 14680, 14907, 15125, 15328, 15506, 15682, 15879, 16080, 16256, 16442, 16609, 16763, 16940, 17106, 17268, 17402, 17559, 17689, 17791, 17930, 18035, 18127, 18231, 18331, 18402, 18476, 18546, 18597, 18643, 18677, 18696, 18712, 18723, 18727, 18729, 18729, 18730, 18730, 18730, 18732, 18734, 18737, 18740, 18740, 18744, 18748, 18753, 18757, 18763, 18765, 18767, 18771, 18777, 18795, 18807, 18820], "harmed": [0, 8, 17, 32, 49, 66, 89, 116, 138, 168, 191, 229, 249, 287, 320, 350, 385, 418, 441, 473, 507, 540, 564, 582, 619, 646, 661, 690, 716, 737, 768, 795, 822, 837, 857, 884, 907, 929, 964, 985, 1003, 1015, 1039, 1061, 1076, 1092, 1108, 1134, 1156, 1184, 1200, 1230, 1255, 1279, 1292, 1315, 1338, 1357, 1371, 1389, 1402, 1420, 1437, 1448, 1466, 1487, 1499, 1518, 1531, 1542, 1560, 1581, 1593, 1608, 1617, 1633, 1645, 1648, 1653, 1654, 1655, 1657, 1661, 1661, 1661, 1662, 1663, 1666, 1667, 1669, 1675, 1679, 1683, 1685, 1688, 1694, 1708, 1730, 1764, 1846, 1970], "exact_score": [26.688029946402747, 27.635198366559855, 28.57102345234382, 29.523863539687493, 30.422822788758758, 31.327453705016595, 32.22074128690129, 33.1111930351927, 33.95059977880498, 34.784334855230696, 35.63224909962284, 36.443297507302276, 37.2004650767093, 38.02569265235516, 38.81972605847489, 39.61092363100134, 40.413464537900914, 41.15928877293481, 41.93347134390154, 42.71616141564814, 43.44780648271559, 44.16243654822335, 44.87423078013782, 45.628562515951565, 46.38005841817202, 47.07483764852679, 47.76961687888155, 48.47290361001617, 49.15633950599779, 49.82559623401299, 50.446643790942346, 51.07903468224485, 51.75112724385333, 52.45724980858124, 53.0414315287979, 53.68800158806681, 54.311884978589454, 54.904574199585966, 55.50577092136233, 56.07293764001928, 56.665626861015795, 57.21577857811304, 57.81413946629612, 58.32175367949408, 58.92578623486374, 59.45892295040127, 59.986387998752235, 60.550718883815904, 61.14057227121913, 61.616992314890965, 62.10475569293594, 62.63505657488019, 63.19087995916399, 63.65028500127612, 64.13237671213453, 64.64282675892579, 65.15611263931032, 65.69775685562772, 66.18835606726597, 66.63641777500496, 67.07030031477753, 67.51552618892323, 67.93806539432265, 68.36627626690866, 68.78881547230809, 69.20568301052094, 69.62255054873381, 70.00822391742052, 70.35703144939455, 70.77957065479397, 71.13688568754786, 71.48569321952188, 71.88554575617503, 72.23718912174235, 72.62569832402235, 72.92346085131724, 73.18719337549273, 73.50197090434733, 73.74868842696311, 73.99257011598559, 74.26764597453422, 74.44630349091115, 74.65048350962766, 74.7922751892919, 74.90854436661657, 75.00496270878824, 75.07302271502708, 75.08153021580695, 75.1070527181465, 75.11272438533308, 75.10988855173979, 75.1070527181465, 75.09570938377337, 75.10138105095993, 75.10421688455322, 75.10421688455322, 75.11272438533308, 75.08720188299351, 75.02197771034795, 74.82063352522475, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.754182194104786, "threshold_area100": 62.659785043813635, "threshold_area25": 41.977143181238134}, "selected_budget": 89, "selected": {"val": {"score": 81.22144915313795, "calls": 88.9943074003795, "rescued": 4618, "harmed": 1037}, "test": {"score": 75.10988855173979, "calls": 94.39639281966934, "rescued": 18744, "harmed": 1669}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 15.732444763183594, "operator": "ge"}, {"budget": 0.02, "threshold": 14.831884384155273, "operator": "ge"}, {"budget": 0.03, "threshold": 14.028438568115234, "operator": "ge"}, {"budget": 0.04, "threshold": 13.615962028503418, "operator": "ge"}, {"budget": 0.05, "threshold": 13.196578979492188, "operator": "ge"}, {"budget": 0.06, "threshold": 12.799554824829102, "operator": "gt"}, {"budget": 0.07, "threshold": 12.486167907714844, "operator": "ge"}, {"budget": 0.08, "threshold": 12.2177734375, "operator": "ge"}, {"budget": 0.09, "threshold": 11.918807983398438, "operator": "ge"}, {"budget": 0.1, "threshold": 11.657211303710938, "operator": "ge"}, {"budget": 0.11, "threshold": 11.433520317077637, "operator": "ge"}, {"budget": 0.12, "threshold": 11.170358657836914, "operator": "ge"}, {"budget": 0.13, "threshold": 10.908735275268555, "operator": "ge"}, {"budget": 0.14, "threshold": 10.692405700683594, "operator": "ge"}, {"budget": 0.15, "threshold": 10.466745376586914, "operator": "ge"}, {"budget": 0.16, "threshold": 10.221479415893555, "operator": "ge"}, {"budget": 0.17, "threshold": 10.024606704711914, "operator": "ge"}, {"budget": 0.18, "threshold": 9.838430404663086, "operator": "ge"}, {"budget": 0.19, "threshold": 9.622410774230957, "operator": "ge"}, {"budget": 0.2, "threshold": 9.413965225219727, "operator": "ge"}, {"budget": 0.21, "threshold": 9.204023361206055, "operator": "ge"}, {"budget": 0.22, "threshold": 8.970948219299316, "operator": "ge"}, {"budget": 0.23, "threshold": 8.743553161621094, "operator": "ge"}, {"budget": 0.24, "threshold": 8.523416519165039, "operator": "gt"}, {"budget": 0.25, "threshold": 8.33303451538086, "operator": "ge"}, {"budget": 0.26, "threshold": 8.117512702941895, "operator": "ge"}, {"budget": 0.27, "threshold": 7.896768569946289, "operator": "ge"}, {"budget": 0.28, "threshold": 7.695047855377197, "operator": "ge"}, {"budget": 0.29, "threshold": 7.464151382446289, "operator": "ge"}, {"budget": 0.3, "threshold": 7.2712297439575195, "operator": "ge"}, {"budget": 0.31, "threshold": 7.0580854415893555, "operator": "ge"}, {"budget": 0.32, "threshold": 6.883101463317871, "operator": "ge"}, {"budget": 0.33, "threshold": 6.6936187744140625, "operator": "ge"}, {"budget": 0.34, "threshold": 6.486319541931152, "operator": "ge"}, {"budget": 0.35, "threshold": 6.246307373046875, "operator": "ge"}, {"budget": 0.36, "threshold": 6.082680702209473, "operator": "ge"}, {"budget": 0.37, "threshold": 5.907139778137207, "operator": "ge"}, {"budget": 0.38, "threshold": 5.744866371154785, "operator": "ge"}, {"budget": 0.39, "threshold": 5.575127124786377, "operator": "ge"}, {"budget": 0.4, "threshold": 5.412110328674316, "operator": "ge"}, {"budget": 0.41, "threshold": 5.263937950134277, "operator": "ge"}, {"budget": 0.42, "threshold": 5.100046634674072, "operator": "ge"}, {"budget": 0.43, "threshold": 4.964834213256836, "operator": "ge"}, {"budget": 0.44, "threshold": 4.835040092468262, "operator": "ge"}, {"budget": 0.45, "threshold": 4.6661176681518555, "operator": "ge"}, {"budget": 0.46, "threshold": 4.524608612060547, "operator": "ge"}, {"budget": 0.47, "threshold": 4.384195327758789, "operator": "ge"}, {"budget": 0.48, "threshold": 4.2627434730529785, "operator": "ge"}, {"budget": 0.49, "threshold": 4.14858341217041, "operator": "ge"}, {"budget": 0.5, "threshold": 4.023747444152832, "operator": "ge"}, {"budget": 0.51, "threshold": 3.8952462673187256, "operator": "ge"}, {"budget": 0.52, "threshold": 3.7629499435424805, "operator": "ge"}, {"budget": 0.53, "threshold": 3.6389076709747314, "operator": "ge"}, {"budget": 0.54, "threshold": 3.5132484436035156, "operator": "ge"}, {"budget": 0.55, "threshold": 3.3766911029815674, "operator": "ge"}, {"budget": 0.56, "threshold": 3.2625515460968018, "operator": "ge"}, {"budget": 0.57, "threshold": 3.1178526878356934, "operator": "ge"}, {"budget": 0.58, "threshold": 2.994540214538574, "operator": "ge"}, {"budget": 0.59, "threshold": 2.8706212043762207, "operator": "ge"}, {"budget": 0.6, "threshold": 2.7443137168884277, "operator": "ge"}, {"budget": 0.61, "threshold": 2.594724178314209, "operator": "ge"}, {"budget": 0.62, "threshold": 2.476334810256958, "operator": "ge"}, {"budget": 0.63, "threshold": 2.3384475708007812, "operator": "ge"}, {"budget": 0.64, "threshold": 2.202145576477051, "operator": "ge"}, {"budget": 0.65, "threshold": 2.088697910308838, "operator": "ge"}, {"budget": 0.66, "threshold": 1.9449549913406372, "operator": "ge"}, {"budget": 0.67, "threshold": 1.8287278413772583, "operator": "ge"}, {"budget": 0.68, "threshold": 1.6946932077407837, "operator": "ge"}, {"budget": 0.69, "threshold": 1.5804625749588013, "operator": "ge"}, {"budget": 0.7, "threshold": 1.450761079788208, "operator": "ge"}, {"budget": 0.71, "threshold": 1.3126111030578613, "operator": "ge"}, {"budget": 0.72, "threshold": 1.1629548072814941, "operator": "ge"}, {"budget": 0.73, "threshold": 1.031177043914795, "operator": "ge"}, {"budget": 0.74, "threshold": 0.8895130157470703, "operator": "ge"}, {"budget": 0.75, "threshold": 0.751827597618103, "operator": "ge"}, {"budget": 0.76, "threshold": 0.629831075668335, "operator": "ge"}, {"budget": 0.77, "threshold": 0.4977760314941406, "operator": "ge"}, {"budget": 0.78, "threshold": 0.3498721122741699, "operator": "ge"}, {"budget": 0.79, "threshold": 0.20122140645980835, "operator": "ge"}, {"budget": 0.8, "threshold": 0.06276547908782959, "operator": "ge"}, {"budget": 0.81, "threshold": -0.07440567016601562, "operator": "ge"}, {"budget": 0.82, "threshold": -0.2462177276611328, "operator": "ge"}, {"budget": 0.83, "threshold": -0.3856300115585327, "operator": "ge"}, {"budget": 0.84, "threshold": -0.5312073230743408, "operator": "ge"}, {"budget": 0.85, "threshold": -0.6776944398880005, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8252835273742676, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9845547676086426, "operator": "ge"}, {"budget": 0.88, "threshold": -1.177164077758789, "operator": "ge"}, {"budget": 0.89, "threshold": -1.3523192405700684, "operator": "ge"}, {"budget": 0.9, "threshold": -1.5577495098114014, "operator": "ge"}, {"budget": 0.91, "threshold": -1.7850289344787598, "operator": "ge"}, {"budget": 0.92, "threshold": -2.049156904220581, "operator": "ge"}, {"budget": 0.93, "threshold": -2.298114061355591, "operator": "ge"}, {"budget": 0.94, "threshold": -2.5741517543792725, "operator": "ge"}, {"budget": 0.95, "threshold": -2.938091278076172, "operator": "ge"}, {"budget": 0.96, "threshold": -3.2955894470214844, "operator": "ge"}, {"budget": 0.97, "threshold": -3.726051092147827, "operator": "ge"}, {"budget": 0.98, "threshold": -4.217073917388916, "operator": "ge"}, {"budget": 0.99, "threshold": -4.828125, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.96113570876379, 57.91693021294538, 58.795417808700535, 59.64579380139152, 60.531309297912706, 61.430880595965974, 62.281256588656966, 63.145688382880024, 64.00309227633704, 64.86752407056012, 65.67573265865485, 66.4698854452175, 67.29917773561037, 68.10035842293907, 68.90153911026776, 69.6605523930002, 70.41253777496661, 71.12938365310282, 71.81109002740881, 72.50685220324688, 73.22369808138309, 73.84215334879471, 74.46763651697236, 75.05798018131983, 75.71860285332771, 76.37219762456954, 76.90631808278867, 77.41232693794363, 77.84805678543817, 78.24161922833649, 78.66329327429897, 79.00063251106893, 79.3168880455408, 79.5909761754164, 79.93534331295243, 80.20943144282802, 80.42026846580926, 80.5819101834282, 80.75760770257924, 80.89816571790006, 81.00358422939068, 81.12305854241337, 81.13008644317942, 81.08089113781713, 81.07386323705109, 81.03872373322089, 81.04575163398692, 81.01061213015672, 81.03169583245484, 81.00358422939068, 81.03872373322089, 81.04575163398692, 80.9543889240284, 80.89113781713402, 80.91222151943214, 80.91222151943214, 80.9051936186661, 80.88410991636798, 80.89816571790006, 80.83491461100569, 80.82788671023965, 80.87708201560193, 80.84194251177172, 80.84194251177172, 80.80680300794153, 80.87708201560193, 80.89816571790006, 81.0246679316888, 81.08791903858317, 81.19333755007378, 81.20739335160587, 81.31983976386252, 81.29172816079837, 81.27064445850024, 81.29172816079837, 81.27767235926629, 81.31281186309648, 81.27767235926629, 81.22847705390399, 81.22144915313795, 81.20036545083983, 81.13008644317942, 81.10197484011525, 81.04575163398692, 81.03872373322089, 81.01764003092275, 80.99655632862464, 80.94736102326235, 80.91222151943214, 80.8700541148359, 80.82788671023965, 80.80680300794153, 80.7857193056434, 80.75057980181319, 80.69435659568487, 80.64516129032258, 80.61704968725842, 80.61704968725842, 80.61002178649237, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.987771452667088, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.98622531449856, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 129, 265, 390, 511, 637, 765, 886, 1010, 1132, 1255, 1370, 1483, 1602, 1716, 1830, 1940, 2048, 2151, 2250, 2353, 2456, 2547, 2638, 2727, 2824, 2920, 3004, 3085, 3164, 3234, 3309, 3377, 3438, 3500, 3566, 3617, 3664, 3707, 3756, 3800, 3841, 3877, 3904, 3932, 3958, 3986, 4015, 4038, 4059, 4084, 4109, 4138, 4153, 4167, 4192, 4215, 4232, 4251, 4274, 4289, 4308, 4328, 4345, 4364, 4377, 4400, 4418, 4445, 4470, 4495, 4507, 4529, 4541, 4550, 4564, 4578, 4592, 4597, 4602, 4608, 4615, 4616, 4620, 4620, 4622, 4623, 4624, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 2, 2, 4, 5, 6, 8, 12, 13, 16, 18, 23, 26, 29, 37, 46, 63, 77, 92, 112, 128, 151, 168, 180, 197, 217, 241, 265, 291, 310, 336, 371, 398, 431, 459, 487, 505, 534, 554, 582, 610, 633, 655, 678, 696, 718, 739, 763, 783, 796, 818, 837, 855, 868, 883, 892, 908, 918, 928, 934, 950, 962, 973, 989, 998, 1008, 1020, 1027, 1037, 1048, 1056, 1064, 1067, 1071, 1075, 1083, 1089, 1095, 1101, 1104, 1107, 1112, 1120, 1127, 1131, 1131, 1132, 1133], "exact_score": [56.05453650994448, 56.96113570876379, 57.91693021294538, 58.795417808700535, 59.64579380139152, 60.531309297912706, 61.43790849673202, 62.281256588656966, 63.145688382880024, 64.00309227633704, 64.86752407056012, 65.67573265865485, 66.4698854452175, 67.29917773561037, 68.10035842293907, 68.90153911026776, 69.6605523930002, 70.41253777496661, 71.12938365310282, 71.81109002740881, 72.50685220324688, 73.22369808138309, 73.84215334879471, 74.46763651697236, 75.05798018131983, 75.71860285332771, 76.37219762456954, 76.90631808278867, 77.41232693794363, 77.84805678543817, 78.24161922833649, 78.66329327429897, 79.00063251106893, 79.3168880455408, 79.5909761754164, 79.93534331295243, 80.20943144282802, 80.42026846580926, 80.5819101834282, 80.75760770257924, 80.89816571790006, 81.00358422939068, 81.12305854241337, 81.13008644317942, 81.08089113781713, 81.07386323705109, 81.03872373322089, 81.04575163398692, 81.01061213015672, 81.03169583245484, 81.00358422939068, 81.03872373322089, 81.04575163398692, 80.9543889240284, 80.89113781713402, 80.91222151943214, 80.91222151943214, 80.9051936186661, 80.88410991636798, 80.89816571790006, 80.83491461100569, 80.82788671023965, 80.87708201560193, 80.84194251177172, 80.84194251177172, 80.80680300794153, 80.87708201560193, 80.89816571790006, 81.0246679316888, 81.08791903858317, 81.19333755007378, 81.20739335160587, 81.31983976386252, 81.29172816079837, 81.27064445850024, 81.29172816079837, 81.27767235926629, 81.31281186309648, 81.27767235926629, 81.22847705390399, 81.22144915313795, 81.20036545083983, 81.13008644317942, 81.10197484011525, 81.04575163398692, 81.03872373322089, 81.01764003092275, 80.99655632862464, 80.94736102326235, 80.91222151943214, 80.8700541148359, 80.82788671023965, 80.80680300794153, 80.7857193056434, 80.75057980181319, 80.69435659568487, 80.64516129032258, 80.61704968725842, 80.61704968725842, 80.61002178649237, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 77.06841469510194, "threshold_area100": 77.06750298685782, "threshold_area25": 66.53875887272473}, "test": {"score": [26.688029946402747, 27.357286674417946, 27.947140061821173, 28.85744264526558, 29.481326035788225, 30.28103110909452, 31.058049513654538, 31.730142075263025, 32.41357797124465, 33.17358137424496, 33.80597226554746, 34.427019822476815, 35.19836655985027, 35.98672829878343, 36.60777585571279, 37.29121175169441, 38.15897683123954, 38.79703938972862, 39.378385276351985, 40.141224512945584, 40.83316790970706, 41.59317131270737, 42.40705555398009, 43.34004480617077, 44.31557156226072, 45.24288914726484, 46.27513257522049, 47.44066018206052, 48.50126194594901, 50.018432918356346, 51.19246802597623, 52.58202648668576, 53.807106598984774, 55.19950089328758, 56.65995519382923, 58.355783682613506, 59.498624620707254, 60.655644726767434, 61.7587839945552, 62.82789325922356, 63.82610668405978, 64.58894592065339, 65.44820349941865, 66.12880356180699, 66.75552278592292, 67.55522785922923, 68.1564245810056, 68.78030797152823, 69.3247880214389, 69.74449139324504, 70.22658310410345, 70.64912230950289, 71.08584068286874, 71.46017071718231, 71.79479908118992, 72.18897995065649, 72.46121997561184, 72.77599750446643, 73.03973002864193, 73.27794005047784, 73.45659756685478, 73.68913592150412, 73.84510676913479, 73.99540594957887, 74.086152624564, 74.22794430422823, 74.31869097921334, 74.39809431982532, 74.46615432606414, 74.56824433542239, 74.66466267759408, 74.76958852054561, 74.84332019397102, 74.92272353458299, 75.00212687519496, 75.02764937753452, 75.06167938065394, 75.0900377165868, 75.10988855173979, 75.11839605251964, 75.11272438533308, 75.11272438533308, 75.11272438533308, 75.10421688455322, 75.07585854862036, 75.06451521424722, 75.03615687831437, 75.01630604316138, 74.98511187363526, 74.95958937129569, 74.92555936817628, 74.88869353146357, 74.83764852678445, 74.76958852054561, 74.71287184867992, 74.65331934322094, 74.60227433854182, 74.56540850182911, 74.54272183308284, 74.50302016277686, 74.47182599325072], "calls": [0.0, 0.714630065507756, 1.3470209568102545, 2.365085216799478, 3.0541927799676714, 3.9588236962255055, 4.85778294529677, 5.6291296826702215, 6.428834755976519, 7.316450670674644, 8.11331991038766, 8.859144145421546, 9.789297564018943, 10.73079431698948, 11.536171057482347, 12.409607804214048, 13.484388736068967, 14.309616311714828, 15.066783881121856, 16.09051980829765, 17.060374897201033, 18.14082749624252, 19.33187760542211, 20.58248022006069, 21.90397867453138, 23.22547712900207, 24.646229759237727, 26.3250432464623, 27.941468394634605, 30.130731928650427, 31.9768595978788, 34.12925729518192, 36.02642996908941, 38.26390267419108, 40.603465388650996, 43.39392564444318, 45.33930748943652, 47.41230184612767, 49.36902702549414, 51.286050534554626, 53.14635737174943, 54.734424183988885, 56.50965601338513, 57.87369197175509, 59.152652922326524, 60.83146640955108, 62.249383206193464, 63.72685250829482, 64.92641011825427, 65.95581771261662, 67.05044947962455, 68.11388707710631, 69.17448884099481, 70.19538893457732, 71.19643819300684, 72.19181578424978, 73.02838669426878, 73.96704761364603, 74.83764852678445, 75.59198026259818, 76.29526699373281, 77.19422624280408, 77.95990131299095, 78.7681138870771, 79.49125145336473, 80.16901568215977, 81.03110909451834, 81.7088733233134, 82.48022006068683, 83.07007344809007, 83.88962935654935, 84.6439610923631, 85.43799449848282, 86.21217706944957, 86.98919547400959, 87.74069137623005, 88.37591810112583, 89.0111448260216, 89.7030882227831, 90.41771828829084, 91.08413918271275, 91.71369424042197, 92.36593596687747, 92.87071434648215, 93.44922439951223, 93.93131611037064, 94.45027365794175, 94.99191787425913, 95.53072625698324, 95.89938462411027, 96.29923716076341, 96.7643138700621, 97.31162975356607, 97.7313331253722, 98.11984232765222, 98.63029237444346, 98.99895074157048, 99.38462411025722, 99.69372997192525, 99.87522332189546, 100.0], "rescued": [0, 236, 444, 766, 987, 1269, 1544, 1781, 2022, 2290, 2514, 2733, 3005, 3284, 3503, 3745, 4053, 4279, 4485, 4757, 5005, 5277, 5568, 5903, 6257, 6597, 6970, 7396, 7785, 8343, 8787, 9310, 9771, 10299, 10865, 11521, 11965, 12415, 12845, 13262, 13662, 13976, 14338, 14617, 14866, 15195, 15451, 15720, 15952, 16133, 16332, 16517, 16702, 16866, 17016, 17180, 17304, 17442, 17561, 17670, 17755, 17865, 17949, 18023, 18078, 18146, 18209, 18259, 18306, 18351, 18407, 18461, 18509, 18556, 18600, 18625, 18649, 18672, 18699, 18716, 18727, 18740, 18756, 18764, 18772, 18781, 18790, 18795, 18800, 18807, 18811, 18812, 18814, 18814, 18816, 18818, 18818, 18820, 18820, 18820, 18820], "harmed": [0, 0, 0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 5, 5, 6, 8, 9, 10, 13, 17, 21, 25, 31, 41, 54, 63, 78, 93, 116, 146, 179, 208, 245, 296, 354, 395, 437, 478, 518, 566, 611, 670, 709, 737, 784, 828, 877, 917, 950, 979, 1015, 1046, 1078, 1110, 1135, 1163, 1190, 1216, 1241, 1263, 1291, 1320, 1341, 1364, 1382, 1413, 1435, 1458, 1467, 1489, 1506, 1528, 1547, 1563, 1579, 1591, 1604, 1624, 1638, 1651, 1664, 1680, 1691, 1709, 1722, 1741, 1753, 1769, 1785, 1801, 1815, 1835, 1859, 1881, 1904, 1922, 1937, 1945, 1959, 1970], "exact_score": [26.688029946402747, 27.635198366559855, 28.519978447664695, 29.427445197515812, 30.320732779400505, 31.177154524572497, 32.04208377052435, 32.892833848509774, 33.70955392337578, 34.53761733261492, 35.37135240904064, 36.20508748546635, 36.97076255565324, 37.76479596177297, 38.547486033519554, 39.32450443807958, 40.0646570059269, 40.78779457221451, 41.51093213850211, 42.17735303292403, 42.89481893202507, 43.632135666279105, 44.38930323568613, 45.10393330119389, 45.824235033888215, 46.53035759861611, 47.23364432975073, 47.86319938745994, 48.54379944984829, 49.2300711794232, 49.92768624337124, 50.579927969826734, 51.20664719394266, 51.836202251651876, 52.49127981170065, 53.13501403737628, 53.7844199302385, 54.388452485608155, 55.035022544877066, 55.61069676431387, 56.23458015483651, 56.92085188441143, 57.561750276493775, 58.11473782718431, 58.73011371692709, 59.291608768397474, 59.889969656580554, 60.440121373677805, 61.021467260301165, 61.59997731333126, 62.12177069449565, 62.680429912372745, 63.22490996228341, 63.743867509854525, 64.24013838867934, 64.69954343079148, 65.21282931117602, 65.66656268610159, 66.19686356804583, 66.69313444687066, 67.16104698976264, 67.6147803646882, 68.05433457164733, 68.4513512747072, 68.8795621472932, 69.36732552533817, 69.7558347276182, 70.19822476817062, 70.60374897201032, 71.0347956781896, 71.3892748773502, 71.70121657261153, 72.12091994441766, 72.45838414201855, 72.78734083883958, 73.09361086691433, 73.3970450613958, 73.64376258401157, 73.85077843632135, 74.02376428551173, 74.18540680032896, 74.31585514562006, 74.40376598701188, 74.56540850182911, 74.69018517993365, 74.80645435725832, 74.87451436349714, 74.99929104160168, 75.03899271190765, 75.0900377165868, 75.10988855173979, 75.11272438533308, 75.12123188611292, 75.09570938377337, 75.05317187987409, 75.01630604316138, 74.94824603692255, 74.883021864277, 74.7355585174262, 74.60227433854182, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.59395779713299, "threshold_area100": 60.926892209965125, "threshold_area25": 35.63536851657545}, "selected_budget": 72, "selected": {"val": {"score": 81.31983976386252, "calls": 71.99381544732589, "rescued": 4529, "harmed": 934}, "test": {"score": 74.84332019397102, "calls": 85.43799449848282, "rescued": 18509, "harmed": 1528}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.942816734313965, "operator": "ge"}, {"budget": 0.02, "threshold": 4.3423261642456055, "operator": "ge"}, {"budget": 0.03, "threshold": 4.048277378082275, "operator": "ge"}, {"budget": 0.04, "threshold": 3.8097262382507324, "operator": "ge"}, {"budget": 0.05, "threshold": 3.5960497856140137, "operator": "ge"}, {"budget": 0.06, "threshold": 3.438167095184326, "operator": "gt"}, {"budget": 0.07, "threshold": 3.283477306365967, "operator": "ge"}, {"budget": 0.08, "threshold": 3.1515450477600098, "operator": "ge"}, {"budget": 0.09, "threshold": 3.024998903274536, "operator": "ge"}, {"budget": 0.1, "threshold": 2.914994478225708, "operator": "ge"}, {"budget": 0.11, "threshold": 2.8126730918884277, "operator": "ge"}, {"budget": 0.12, "threshold": 2.7172603607177734, "operator": "ge"}, {"budget": 0.13, "threshold": 2.636152744293213, "operator": "ge"}, {"budget": 0.14, "threshold": 2.5435895919799805, "operator": "ge"}, {"budget": 0.15, "threshold": 2.4623286724090576, "operator": "ge"}, {"budget": 0.16, "threshold": 2.391839027404785, "operator": "ge"}, {"budget": 0.17, "threshold": 2.3215556144714355, "operator": "ge"}, {"budget": 0.18, "threshold": 2.2510805130004883, "operator": "ge"}, {"budget": 0.19, "threshold": 2.187886953353882, "operator": "ge"}, {"budget": 0.2, "threshold": 2.1268680095672607, "operator": "gt"}, {"budget": 0.21, "threshold": 2.0786778926849365, "operator": "ge"}, {"budget": 0.22, "threshold": 2.020007610321045, "operator": "ge"}, {"budget": 0.23, "threshold": 1.9687409400939941, "operator": "ge"}, {"budget": 0.24, "threshold": 1.9236993789672852, "operator": "ge"}, {"budget": 0.25, "threshold": 1.8739149570465088, "operator": "ge"}, {"budget": 0.26, "threshold": 1.8363475799560547, "operator": "ge"}, {"budget": 0.27, "threshold": 1.7832307815551758, "operator": "ge"}, {"budget": 0.28, "threshold": 1.7324976921081543, "operator": "ge"}, {"budget": 0.29, "threshold": 1.6929960250854492, "operator": "ge"}, {"budget": 0.3, "threshold": 1.6525607109069824, "operator": "ge"}, {"budget": 0.31, "threshold": 1.6110472679138184, "operator": "ge"}, {"budget": 0.32, "threshold": 1.5679194927215576, "operator": "ge"}, {"budget": 0.33, "threshold": 1.5267099142074585, "operator": "ge"}, {"budget": 0.34, "threshold": 1.491599678993225, "operator": "ge"}, {"budget": 0.35, "threshold": 1.4544423818588257, "operator": "ge"}, {"budget": 0.36, "threshold": 1.417431354522705, "operator": "ge"}, {"budget": 0.37, "threshold": 1.3817501068115234, "operator": "ge"}, {"budget": 0.38, "threshold": 1.350987195968628, "operator": "ge"}, {"budget": 0.39, "threshold": 1.319274663925171, "operator": "ge"}, {"budget": 0.4, "threshold": 1.2876538038253784, "operator": "ge"}, {"budget": 0.41, "threshold": 1.2540779113769531, "operator": "ge"}, {"budget": 0.42, "threshold": 1.2285730838775635, "operator": "ge"}, {"budget": 0.43, "threshold": 1.1991527080535889, "operator": "ge"}, {"budget": 0.44, "threshold": 1.1685189008712769, "operator": "ge"}, {"budget": 0.45, "threshold": 1.140020489692688, "operator": "ge"}, {"budget": 0.46, "threshold": 1.1090362071990967, "operator": "ge"}, {"budget": 0.47, "threshold": 1.0789345502853394, "operator": "ge"}, {"budget": 0.48, "threshold": 1.0546276569366455, "operator": "ge"}, {"budget": 0.49, "threshold": 1.0293493270874023, "operator": "ge"}, {"budget": 0.5, "threshold": 0.9980242252349854, "operator": "ge"}, {"budget": 0.51, "threshold": 0.97041916847229, "operator": "ge"}, {"budget": 0.52, "threshold": 0.9439661502838135, "operator": "ge"}, {"budget": 0.53, "threshold": 0.9142996072769165, "operator": "ge"}, {"budget": 0.54, "threshold": 0.8873070478439331, "operator": "ge"}, {"budget": 0.55, "threshold": 0.8613482117652893, "operator": "ge"}, {"budget": 0.56, "threshold": 0.8344347476959229, "operator": "ge"}, {"budget": 0.57, "threshold": 0.8070595264434814, "operator": "ge"}, {"budget": 0.58, "threshold": 0.7794179916381836, "operator": "ge"}, {"budget": 0.59, "threshold": 0.7518935799598694, "operator": "ge"}, {"budget": 0.6, "threshold": 0.7214488387107849, "operator": "ge"}, {"budget": 0.61, "threshold": 0.6975243091583252, "operator": "ge"}, {"budget": 0.62, "threshold": 0.6721417903900146, "operator": "ge"}, {"budget": 0.63, "threshold": 0.6449856758117676, "operator": "ge"}, {"budget": 0.64, "threshold": 0.6206972599029541, "operator": "ge"}, {"budget": 0.65, "threshold": 0.5916056632995605, "operator": "ge"}, {"budget": 0.66, "threshold": 0.5663502216339111, "operator": "ge"}, {"budget": 0.67, "threshold": 0.534541666507721, "operator": "ge"}, {"budget": 0.68, "threshold": 0.5100564956665039, "operator": "ge"}, {"budget": 0.69, "threshold": 0.47909581661224365, "operator": "ge"}, {"budget": 0.7, "threshold": 0.4521108865737915, "operator": "ge"}, {"budget": 0.71, "threshold": 0.4217681884765625, "operator": "ge"}, {"budget": 0.72, "threshold": 0.39165574312210083, "operator": "ge"}, {"budget": 0.73, "threshold": 0.3599274456501007, "operator": "ge"}, {"budget": 0.74, "threshold": 0.3233039379119873, "operator": "ge"}, {"budget": 0.75, "threshold": 0.28370773792266846, "operator": "ge"}, {"budget": 0.76, "threshold": 0.2439783811569214, "operator": "ge"}, {"budget": 0.77, "threshold": 0.2013310194015503, "operator": "ge"}, {"budget": 0.78, "threshold": 0.1590433120727539, "operator": "ge"}, {"budget": 0.79, "threshold": 0.099799245595932, "operator": "ge"}, {"budget": 0.8, "threshold": 0.0352940559387207, "operator": "ge"}, {"budget": 0.81, "threshold": -0.04594874382019043, "operator": "ge"}, {"budget": 0.82, "threshold": -0.14974844455718994, "operator": "ge"}, {"budget": 0.83, "threshold": -0.24878132343292236, "operator": "ge"}, {"budget": 0.84, "threshold": -0.34363341331481934, "operator": "ge"}, {"budget": 0.85, "threshold": -0.4680085778236389, "operator": "ge"}, {"budget": 0.86, "threshold": -0.5657333731651306, "operator": "ge"}, {"budget": 0.87, "threshold": -0.7084776163101196, "operator": "ge"}, {"budget": 0.88, "threshold": -0.870661735534668, "operator": "ge"}, {"budget": 0.89, "threshold": -1.0281898975372314, "operator": "ge"}, {"budget": 0.9, "threshold": -1.2391875982284546, "operator": "ge"}, {"budget": 0.91, "threshold": -1.4339261054992676, "operator": "ge"}, {"budget": 0.92, "threshold": -1.6553505659103394, "operator": "ge"}, {"budget": 0.93, "threshold": -1.9114189147949219, "operator": "ge"}, {"budget": 0.94, "threshold": -2.2386250495910645, "operator": "ge"}, {"budget": 0.95, "threshold": -2.559725761413574, "operator": "ge"}, {"budget": 0.96, "threshold": -2.9186606407165527, "operator": "ge"}, {"budget": 0.97, "threshold": -3.3148746490478516, "operator": "ge"}, {"budget": 0.98, "threshold": -3.824056625366211, "operator": "ge"}, {"budget": 0.99, "threshold": -4.492067337036133, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.890856701103374, 57.6287862815377, 58.380771663504106, 59.104645442406344, 59.82851922130858, 60.468058191018336, 61.20598777145266, 61.775247733502006, 62.351535596317376, 62.962962962962955, 63.53222292501229, 64.05931548246538, 64.53721273455618, 64.97294258205073, 65.52111884180195, 66.04821139925504, 66.44177384215334, 66.86344788811581, 67.22187082718392, 67.74896338463701, 68.12847002600323, 68.4658092627732, 68.8593717056715, 69.2318504462717, 69.56918968304167, 69.8292220113852, 70.11736594279289, 70.56715159181952, 70.80610021786492, 71.10127205003865, 71.48077869140488, 71.79703422587673, 72.23979197413732, 72.63335441703562, 73.08314006606227, 73.39236769976807, 73.70862323423992, 73.89837655492303, 74.23571579169301, 74.58008292922904, 74.86119895987068, 75.02986857825566, 75.1704265935765, 75.43748682268607, 75.73968655562582, 76.00674678473538, 76.28786281537704, 76.59006254831682, 76.84306697589429, 77.09607140347178, 77.24365731955865, 77.43341064024175, 77.74966617471361, 77.98158689999298, 78.22053552603838, 78.45245625131773, 78.6351816712348, 78.74060018272542, 78.91629770187645, 79.12713472485768, 79.3168880455408, 79.4363623585635, 79.68233888537493, 79.86506430529201, 80.01967812214491, 80.13212453440157, 80.24457094665823, 80.34998945814884, 80.49054747346966, 80.65218919108862, 80.7857193056434, 80.86302621406986, 80.96141682479443, 81.059807435519, 81.10900274088128, 81.23550495467003, 81.20036545083983, 81.20739335160587, 81.2144212523719, 81.20036545083983, 81.19333755007378, 81.17225384777566, 81.13711434394546, 81.07386323705109, 81.03169583245484, 81.01764003092275, 80.98250052709255, 80.96141682479443, 80.94033312249631, 80.89113781713402, 80.85599831330381, 80.80680300794153, 80.77869140487735, 80.76463560334528, 80.7013844964509, 80.64516129032258, 80.62407758802446, 80.61704968725842, 80.61002178649237, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.980743551901047, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.987349778621123, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 123, 239, 353, 467, 578, 676, 789, 881, 974, 1069, 1165, 1253, 1337, 1414, 1502, 1588, 1652, 1728, 1797, 1887, 1960, 2026, 2093, 2162, 2227, 2286, 2346, 2418, 2469, 2529, 2595, 2657, 2727, 2794, 2869, 2927, 2986, 3037, 3095, 3157, 3216, 3262, 3306, 3359, 3418, 3475, 3526, 3580, 3629, 3678, 3715, 3757, 3809, 3852, 3898, 3942, 3984, 4016, 4057, 4096, 4134, 4167, 4208, 4248, 4277, 4304, 4329, 4357, 4392, 4424, 4455, 4479, 4503, 4525, 4545, 4572, 4582, 4593, 4601, 4608, 4614, 4617, 4620, 4621, 4622, 4624, 4624, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 4, 15, 22, 33, 41, 48, 56, 67, 78, 86, 101, 114, 130, 145, 155, 166, 174, 190, 208, 223, 242, 260, 271, 287, 304, 326, 345, 353, 370, 388, 400, 417, 424, 435, 446, 460, 474, 498, 508, 521, 540, 562, 586, 601, 617, 636, 647, 658, 671, 684, 700, 715, 722, 732, 744, 755, 771, 788, 804, 813, 824, 840, 846, 860, 867, 878, 887, 900, 915, 924, 936, 949, 959, 967, 980, 989, 1004, 1014, 1021, 1030, 1037, 1043, 1051, 1061, 1068, 1072, 1077, 1082, 1085, 1092, 1097, 1104, 1108, 1110, 1119, 1127, 1130, 1131, 1132, 1133], "exact_score": [56.05453650994448, 56.890856701103374, 57.6287862815377, 58.380771663504106, 59.104645442406344, 59.82851922130858, 60.482113992550424, 61.20598777145266, 61.775247733502006, 62.351535596317376, 62.962962962962955, 63.53222292501229, 64.05931548246538, 64.53721273455618, 64.97294258205073, 65.52111884180195, 66.04821139925504, 66.44177384215334, 66.86344788811581, 67.22187082718392, 67.74896338463701, 68.12847002600323, 68.4658092627732, 68.8593717056715, 69.2318504462717, 69.56918968304167, 69.8292220113852, 70.11736594279289, 70.56715159181952, 70.80610021786492, 71.10127205003865, 71.48077869140488, 71.79703422587673, 72.23979197413732, 72.63335441703562, 73.08314006606227, 73.39236769976807, 73.70862323423992, 73.89837655492303, 74.23571579169301, 74.58008292922904, 74.86119895987068, 75.02986857825566, 75.1704265935765, 75.43748682268607, 75.73968655562582, 76.00674678473538, 76.28786281537704, 76.59006254831682, 76.84306697589429, 77.09607140347178, 77.24365731955865, 77.43341064024175, 77.74966617471361, 77.98158689999298, 78.22053552603838, 78.45245625131773, 78.6351816712348, 78.74060018272542, 78.91629770187645, 79.12713472485768, 79.3168880455408, 79.4363623585635, 79.68233888537493, 79.86506430529201, 80.01967812214491, 80.13212453440157, 80.24457094665823, 80.34998945814884, 80.49054747346966, 80.65218919108862, 80.7857193056434, 80.86302621406986, 80.96141682479443, 81.059807435519, 81.10900274088128, 81.23550495467003, 81.20036545083983, 81.20739335160587, 81.2144212523719, 81.20036545083983, 81.19333755007378, 81.17225384777566, 81.13711434394546, 81.07386323705109, 81.03169583245484, 81.01764003092275, 80.98250052709255, 80.96141682479443, 80.94033312249631, 80.89113781713402, 80.85599831330381, 80.80680300794153, 80.77869140487735, 80.76463560334528, 80.7013844964509, 80.64516129032258, 80.62407758802446, 80.61704968725842, 80.61002178649237, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 74.43091130259263, "threshold_area100": 74.42993182936256, "threshold_area25": 63.77468550144072}, "test": {"score": [26.688029946402747, 28.00385673368687, 29.835805234948815, 31.089243683180673, 32.47596631029691, 33.87970393897286, 35.06508238096588, 36.39508833621643, 37.56912344383632, 38.62121770694496, 39.815103649717834, 40.87286958001305, 42.01854635170008, 43.005416442163174, 44.060346538865105, 45.001843291835634, 45.87811587216063, 46.77140345404532, 47.616481864844175, 48.489918611575874, 49.23574284660977, 49.86813373791226, 50.622465473725995, 51.24067719706208, 51.861724753991446, 52.553668150752905, 53.04710319598447, 53.72203159118622, 54.38278081842158, 54.913081700365815, 55.49442758698919, 56.061594305646146, 56.67980602898221, 57.269659416385444, 57.80846779910955, 58.31041034512094, 58.89175623174432, 59.36534044182288, 59.83608881830814, 60.28131469245385, 60.70952556503985, 61.23131894620424, 61.62549981567083, 62.16430819839491, 62.589683237387625, 63.09162578339903, 63.53968749113802, 63.96506253013074, 64.29401922695178, 64.68252842923177, 65.12491846978419, 65.54745767518362, 65.96432521339648, 66.41805858832204, 66.8349261265349, 67.28865950146046, 67.70269120608003, 68.11672291069961, 68.5080679465729, 68.90792048322605, 69.33613135581204, 69.72180472449877, 70.0507614213198, 70.38255395173411, 70.67464481184244, 71.08016901568216, 71.36942404219721, 71.7607690780705, 72.01882993505941, 72.3194282959476, 72.64554915917533, 72.99435669114936, 73.31197005359726, 73.6210759152653, 73.87630093866092, 74.18257096673567, 74.39242265263874, 74.56257266823583, 74.7724243541389, 74.95958937129569, 75.11556021892636, 75.10988855173979, 75.10988855173979, 75.10138105095993, 75.07869438221365, 75.0503360462808, 75.03332104472109, 74.98794770722854, 74.94541020332927, 74.92555936817628, 74.88869353146357, 74.85749936193744, 74.81496185803817, 74.77809602132547, 74.7157076822732, 74.64764767603438, 74.61078183932167, 74.56824433542239, 74.54272183308284, 74.50302016277686, 74.47182599325072], "calls": [0.0, 1.4094092958625188, 3.360462808042424, 4.781215438278082, 6.37495391770411, 8.056603238521962, 9.514221705470323, 11.16751269035533, 12.633638658083543, 14.08842129143862, 15.631114766185519, 17.049031562827892, 18.532172532115815, 19.828148484246945, 21.333976122281147, 22.73487791736381, 23.976973031222528, 25.338173155999204, 26.585939937044493, 27.8677367212092, 29.0389359952358, 30.01162691773247, 31.222527862065053, 32.263278790800555, 33.18208887502481, 34.288063976405866, 35.04806737940618, 36.17672914953351, 37.33658508918697, 38.22420100388509, 39.17987692482205, 40.25465785667697, 41.32376712134532, 42.46377222584579, 43.4648214842753, 44.42049740521226, 45.43288999801492, 46.357371749425745, 47.24782349771716, 48.158126081161555, 49.09962283413209, 50.1006720925616, 50.85783966196864, 51.779485579786176, 52.68411649604401, 53.716359923999654, 54.68621501290304, 55.599353429940734, 56.3848793352806, 57.23846524685931, 58.34444034824037, 59.26325043246462, 60.182060516688885, 61.254005614950515, 62.19833820151434, 63.28162663414911, 64.32804923007119, 65.40850182911267, 66.48328276096758, 67.46448118424411, 68.618665456711, 69.59419221280095, 70.56688313529762, 71.53106655701443, 72.50659331310439, 73.618240081672, 74.59660267135526, 75.84436945240053, 76.72631369991208, 77.7840796302073, 78.85318889487564, 80.02155233530897, 81.08498993279073, 82.19380086776508, 83.45574681677678, 84.80560360718033, 85.85486203669569, 87.02038964353572, 88.02994640274508, 89.50174403765988, 90.817570824944, 91.82712758415337, 92.6381759918328, 93.23937271360917, 93.76683776196013, 94.38788531888949, 94.78490202194935, 95.2981879023339, 95.85117545302441, 96.2623713240507, 96.66505969429714, 97.09043473328985, 97.45058559963702, 97.91566230893571, 98.37790318464113, 98.78059155488756, 99.1265632532683, 99.4073107790035, 99.69656580551853, 99.87522332189546, 100.0], "rescued": [0, 476, 1131, 1589, 2095, 2618, 3061, 3562, 3997, 4401, 4855, 5257, 5687, 6056, 6475, 6841, 7177, 7527, 7857, 8195, 8490, 8736, 9030, 9280, 9516, 9787, 9981, 10249, 10516, 10723, 10951, 11173, 11417, 11658, 11873, 12070, 12300, 12487, 12674, 12854, 13036, 13238, 13391, 13598, 13771, 13964, 14142, 14306, 14445, 14595, 14768, 14929, 15088, 15266, 15430, 15611, 15776, 15945, 16102, 16253, 16422, 16569, 16703, 16834, 16950, 17108, 17224, 17380, 17484, 17598, 17729, 17865, 17991, 18109, 18212, 18332, 18413, 18490, 18569, 18651, 18720, 18742, 18763, 18773, 18776, 18785, 18791, 18795, 18799, 18805, 18808, 18812, 18813, 18814, 18817, 18818, 18819, 18819, 18820, 18820, 18820], "harmed": [0, 12, 21, 37, 54, 82, 107, 139, 160, 193, 226, 255, 281, 302, 349, 383, 410, 445, 477, 507, 539, 562, 590, 622, 639, 666, 686, 716, 750, 770, 793, 815, 841, 874, 899, 919, 944, 964, 985, 1008, 1039, 1057, 1071, 1088, 1111, 1127, 1147, 1161, 1184, 1197, 1214, 1226, 1238, 1256, 1273, 1294, 1313, 1336, 1355, 1365, 1383, 1394, 1412, 1426, 1439, 1454, 1468, 1486, 1499, 1507, 1523, 1536, 1550, 1559, 1572, 1584, 1591, 1608, 1613, 1629, 1643, 1667, 1688, 1701, 1712, 1731, 1743, 1763, 1782, 1795, 1811, 1826, 1842, 1856, 1881, 1906, 1920, 1935, 1945, 1959, 1970], "exact_score": [26.688029946402747, 27.618183365000142, 28.559680117970682, 29.489833536568078, 30.408643620792336, 31.276408700337466, 32.14984544706917, 33.02328219380087, 33.822987267107166, 34.63119984119332, 35.479114085585465, 36.25046082295891, 37.08703173297791, 37.8328559680118, 38.55315770070612, 39.372713609165416, 40.1185378441993, 40.83600374330034, 41.60167881348723, 42.37869721804724, 43.12735728667442, 43.83347985140232, 44.531094915350366, 45.197515809772284, 45.9008025409069, 46.53602926580268, 47.256330998497006, 47.89722939057937, 48.580665286560986, 49.20171284349035, 49.86529790431897, 50.514703797181184, 51.056348013498564, 51.75396307744662, 52.35515979922298, 53.01590902645833, 53.60859824745484, 54.15874996455208, 54.808155857414285, 55.39233757763095, 55.91129512520205, 56.49264101182543, 57.028613560956245, 57.570257777273625, 58.089215324844744, 58.622352040382275, 59.186682925445936, 59.67444630349091, 60.184896350282166, 60.655644726767434, 61.174602274338554, 61.70490315628279, 62.27206987493974, 62.73431075064515, 63.22207412869013, 63.67580750361569, 64.12386921135467, 64.56058758472054, 64.96327595496696, 65.4198451634858, 65.89059353997108, 66.31596857896379, 66.76686612029606, 67.18373365850891, 67.58358619516207, 67.99478206618836, 68.3379179309758, 68.70657629810283, 69.0837421660097, 69.46374386750986, 69.85225306978987, 70.20389643535718, 70.53852479936478, 70.86464566259252, 71.2021098601934, 71.49136488670844, 71.80897824915634, 72.10674077645123, 72.38181663499986, 72.68525082948133, 72.9886850239628, 73.28361171766441, 73.56435924339961, 73.79689759804894, 73.99824178317216, 74.24212347219465, 74.4207809885716, 74.55973683464254, 74.76108101976577, 74.89720103224343, 75.03615687831437, 75.09854521736665, 75.1240677197062, 75.10988855173979, 75.06451521424722, 75.02481354394124, 74.93690270254942, 74.86884269631058, 74.76675268695233, 74.62496100728809, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.07627437625374, "threshold_area100": 61.42217054703229, "threshold_area25": 41.51064855514279}, "selected_budget": 76, "selected": {"val": {"score": 81.23550495467003, "calls": 75.99971888396936, "rescued": 4572, "harmed": 989}, "test": {"score": 74.39242265263874, "calls": 85.85486203669569, "rescued": 18413, "harmed": 1591}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 81.22144915313795, "calls": 88.9943074003795}, "test": {"score": 75.10988855173979, "calls": 94.39639281966934}}, "old_normalized_interior": {"val": {"score": 81.22144915313795, "calls": 88.9943074003795}, "test": {"score": 75.10988855173979, "calls": 94.39639281966934}}, "old_normalized_endpoints": {"val": {"score": 81.22144915313795, "calls": 88.9943074003795}, "test": {"score": 75.10988855173979, "calls": 94.39639281966934}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 56.96816360952983, 57.811511701454776, 58.577552884953256, 59.39278937381404, 60.16585845807857, 60.924871740811014, 61.70496872584159, 62.281256588656966, 62.94890716143088, 63.61655773420479, 64.23501300161641, 64.7972450628997, 65.359477124183, 65.94982078853047, 66.38555063602502, 66.87047578888186, 67.39756834633495, 67.88952139995783, 68.35336285051655, 68.79612059877715, 69.22482254550565, 69.62541288917001, 69.9486963244079, 70.39848197343453, 70.7358212102045, 71.06613254620844, 71.45969498910675, 71.90948063813339, 72.26087567643545, 72.73877292852625, 73.04097266146601, 73.36425609670391, 73.69456743270784, 74.03190666947782, 74.39735750931197, 74.69955724225173, 74.91039426523297, 75.14934289127837, 75.42343102115397, 75.69049125026355, 76.01377468550143, 76.26677911307893, 76.59006254831682, 76.85009487666034, 77.06093189964157, 77.27879682338884, 77.55288495326445, 77.79183357930985, 77.99564270152504, 78.21350762527233, 78.42434464825357, 78.6843769765971, 78.8741302972802, 79.00063251106893, 79.0287441141331, 79.16930212945392, 79.30986014477475, 79.45744606086161, 79.65422728231077, 79.80181319839762, 79.90020380912222, 80.03373392367699, 80.12509663363552, 80.25862674819031, 80.37810106121302, 80.50460327500176, 80.56785438189613, 80.57488228266216, 80.64516129032258, 80.69435659568487, 80.75760770257924, 80.94033312249631, 80.98250052709255, 81.01764003092275, 81.08089113781713, 81.10900274088128, 81.17225384777566, 81.20739335160587, 81.19333755007378, 81.20036545083983, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.2144212523719, 81.2144212523719, 81.22144915313795, 81.20739335160587, 81.17225384777566, 81.17225384777566, 81.14414224471149, 81.12305854241337, 81.10197484011525, 81.03872373322089, 80.97547262632652, 80.8700541148359, 80.74355190104716, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 56.96816360952983, 57.811511701454776, 58.577552884953256, 59.39278937381404, 60.16585845807857, 60.924871740811014, 61.70496872584159, 62.281256588656966, 62.94890716143088, 63.61655773420479, 64.23501300161641, 64.7972450628997, 65.359477124183, 65.94982078853047, 66.38555063602502, 66.87047578888186, 67.39756834633495, 67.88952139995783, 68.35336285051655, 68.79612059877715, 69.22482254550565, 69.62541288917001, 69.9486963244079, 70.39848197343453, 70.7358212102045, 71.06613254620844, 71.45969498910675, 71.90948063813339, 72.26087567643545, 72.73877292852625, 73.04097266146601, 73.36425609670391, 73.69456743270784, 74.03190666947782, 74.39735750931197, 74.69955724225173, 74.91039426523297, 75.14934289127837, 75.42343102115397, 75.69049125026355, 76.01377468550143, 76.26677911307893, 76.59006254831682, 76.85009487666034, 77.06093189964157, 77.27879682338884, 77.55288495326445, 77.79183357930985, 77.99564270152504, 78.21350762527233, 78.42434464825357, 78.6843769765971, 78.8741302972802, 79.00063251106893, 79.0287441141331, 79.16930212945392, 79.30986014477475, 79.45744606086161, 79.65422728231077, 79.80181319839762, 79.90020380912222, 80.03373392367699, 80.12509663363552, 80.25862674819031, 80.37810106121302, 80.50460327500176, 80.56785438189613, 80.57488228266216, 80.64516129032258, 80.69435659568487, 80.75760770257924, 80.94033312249631, 80.98250052709255, 81.01764003092275, 81.08089113781713, 81.10900274088128, 81.17225384777566, 81.20739335160587, 81.19333755007378, 81.20036545083983, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.2144212523719, 81.2144212523719, 81.22144915313795, 81.20739335160587, 81.17225384777566, 81.17225384777566, 81.14414224471149, 81.12305854241337, 81.10197484011525, 81.03872373322089, 80.97547262632652, 80.8700541148359, 80.74355190104716, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 64.47185325743202, "exact_area100": 75.11367489958397}, "test": {"score": [26.688029946402747, 27.918781725888326, 29.577744377959903, 30.873720330091032, 32.53268298216261, 33.68119558744293, 35.02821654425319, 36.39508833621643, 37.546436775090044, 38.845248560814454, 40.135552845759015, 41.29824461900576, 42.46377222584579, 43.51870232254772, 44.5821399200295, 45.61438334798515, 46.692000113433345, 47.721407707795706, 48.387828602217624, 49.380370359867285, 50.214105436292996, 50.835152993222366, 51.63202223293537, 52.383518135155825, 53.152029038936, 53.792927431018356, 54.36860165045515, 55.03785837847035, 55.78651844709752, 56.450103507926165, 57.01159855939654, 57.64398945069904, 58.2281711709157, 58.74145705130023, 59.26325043246463, 59.819073816748435, 60.482658877577066, 61.10937810169299, 61.63400731645068, 62.152964864021776, 62.62654907410033, 63.142670788078156, 63.687150837988824, 64.21461588633979, 64.73073760031762, 65.22133681195588, 65.81969770013896, 66.36417775004963, 66.87746363043416, 67.30283866942688, 67.75657204435244, 68.23015625443098, 68.72926296684912, 69.16030967302838, 69.65090888466665, 70.05926892209966, 70.43076312281995, 70.87882483055894, 71.30987153673823, 71.71823157417123, 72.06136743895868, 72.45554830842526, 72.77599750446643, 73.03405836145535, 73.3771942262428, 73.61540424807873, 73.84227093554149, 74.08331679097071, 74.33003431358648, 74.50018432918357, 74.65899101040752, 74.79794685647848, 74.90854436661657, 74.9964552080084, 75.06735104784052, 75.07585854862036, 75.08720188299351, 75.10988855173979, 75.1070527181465, 75.10988855173979, 75.1070527181465, 75.10421688455322, 75.09287355018009, 75.09287355018009, 75.09854521736665, 75.10138105095993, 75.1070527181465, 75.1070527181465, 75.10421688455322, 75.10988855173979, 75.10421688455322, 75.1070527181465, 75.1070527181465, 75.11839605251964, 75.11556021892636, 75.10421688455322, 75.07585854862036, 75.03048521112781, 74.98511187363526, 74.78660352210534, 74.47182599325072], "calls": [0.0, 1.3044834529109832, 3.0541927799676714, 4.489124578169753, 6.366446416924256, 7.685109037801661, 9.287355018007544, 10.949153503672404, 12.412443637807334, 14.034540453166208, 15.633950599778803, 17.18798740889885, 18.64277004225392, 20.09471684201571, 21.543827808184215, 22.984431273572866, 24.46190057567422, 25.922354876215863, 26.88086663074611, 28.335649264101182, 29.63162521623231, 30.655361143408104, 31.815217083061565, 32.88432634772992, 34.14910813033491, 35.189859059070415, 36.1029974761081, 37.24016674701529, 38.49927686243371, 39.585401128661765, 40.66301789410998, 41.734962992371614, 42.82675892578624, 43.7257181748575, 44.61900575674219, 45.66826418625755, 46.89334429855656, 47.93409522729206, 49.03439866148654, 50.10350792615489, 50.97410883929331, 51.89008308992429, 53.06695403113745, 54.22113830360434, 55.14845588860846, 56.16368431500439, 57.278166917165294, 58.41817202166577, 59.526982956640104, 60.6045997220883, 61.56878314380512, 62.711624081898876, 63.868644187959056, 64.89521594872812, 66.05223605478831, 67.11850948586337, 68.210305419278, 69.29642968550606, 70.43359895641323, 71.57076822732043, 72.47256330998498, 73.47361256841448, 74.45197515809772, 75.3849644102884, 76.57885035306128, 77.47213793494598, 78.42497802228965, 79.37498227604004, 80.37319570087628, 81.24663244760796, 82.09454669200011, 83.08992428324305, 83.96336102997476, 84.94739528684457, 85.74993619374415, 86.5581487678303, 87.44576468252842, 88.4156197714318, 89.29756401894336, 90.13413492896237, 91.00190000850749, 91.65130590136971, 92.08802427473556, 92.46519014264243, 92.7686243371239, 93.07205853160536, 93.44071689873238, 93.77534526273999, 94.10146612596773, 94.39639281966934, 94.7565436860165, 95.13654538751666, 95.4825170858974, 95.83983211865129, 96.22834132093129, 96.65655219351729, 97.23789808014067, 97.85894563707002, 98.37223151745455, 99.12939908686158, 100.0], "exact_score": [26.688029946402747, 27.635198366559855, 28.57102345234382, 29.523863539687493, 30.422822788758758, 31.327453705016595, 32.22074128690129, 33.1111930351927, 33.95059977880498, 34.784334855230696, 35.63224909962284, 36.443297507302276, 37.2004650767093, 38.02569265235516, 38.81972605847489, 39.61092363100134, 40.413464537900914, 41.15928877293481, 41.93347134390154, 42.71616141564814, 43.44780648271559, 44.16243654822335, 44.87423078013782, 45.628562515951565, 46.38005841817202, 47.07483764852679, 47.76961687888155, 48.47290361001617, 49.15633950599779, 49.82559623401299, 50.446643790942346, 51.07903468224485, 51.75112724385333, 52.45724980858124, 53.0414315287979, 53.68800158806681, 54.311884978589454, 54.904574199585966, 55.50577092136233, 56.07293764001928, 56.665626861015795, 57.21577857811304, 57.81413946629612, 58.32175367949408, 58.92578623486374, 59.45892295040127, 59.986387998752235, 60.550718883815904, 61.14057227121913, 61.616992314890965, 62.10475569293594, 62.63505657488019, 63.19087995916399, 63.65028500127612, 64.13237671213453, 64.64282675892579, 65.15611263931032, 65.69775685562772, 66.18835606726597, 66.63641777500496, 67.07030031477753, 67.51552618892323, 67.93806539432265, 68.36627626690866, 68.78881547230809, 69.20568301052094, 69.62255054873381, 70.00822391742052, 70.35703144939455, 70.77957065479397, 71.13688568754786, 71.48569321952188, 71.88554575617503, 72.23718912174235, 72.62569832402235, 72.92346085131724, 73.18719337549273, 73.50197090434733, 73.74868842696311, 73.99257011598559, 74.26764597453422, 74.44630349091115, 74.65048350962766, 74.7922751892919, 74.90854436661657, 75.00496270878824, 75.07302271502708, 75.08153021580695, 75.1070527181465, 75.11272438533308, 75.10988855173979, 75.1070527181465, 75.09570938377337, 75.10138105095993, 75.10421688455322, 75.10421688455322, 75.11272438533308, 75.08720188299351, 75.02197771034795, 74.82063352522475, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 41.977143181238134, "exact_area100": 58.754182194104786}}, "old_normalized_endpoint_budget": 89}, {"seed": 2027, "selection": {"best_epoch": 8, "epochs": 16, "alpha": 0.0, "validation_area100": 0.7550596057507206, "training_seconds": 112.02416833094321, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9959196448326111, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9930047392845154, "operator": "ge"}, {"budget": 0.03, "threshold": 0.9894710183143616, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9859538674354553, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9826302528381348, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9785681962966919, "operator": "gt"}, {"budget": 0.07, "threshold": 0.9741941094398499, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9694491028785706, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9633454084396362, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9581807851791382, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9517661333084106, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9442238211631775, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9366728663444519, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9303407669067383, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9238371849060059, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9159175157546997, "operator": "ge"}, {"budget": 0.17, "threshold": 0.9081736207008362, "operator": "ge"}, {"budget": 0.18, "threshold": 0.901128351688385, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8936429619789124, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8855038285255432, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8767584562301636, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8674139380455017, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8591026663780212, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8492180109024048, "operator": "ge"}, {"budget": 0.25, "threshold": 0.841093122959137, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8319987654685974, "operator": "ge"}, {"budget": 0.27, "threshold": 0.822016716003418, "operator": "ge"}, {"budget": 0.28, "threshold": 0.8119505643844604, "operator": "ge"}, {"budget": 0.29, "threshold": 0.8031176328659058, "operator": "ge"}, {"budget": 0.3, "threshold": 0.7930914759635925, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7845637798309326, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7741467952728271, "operator": "ge"}, {"budget": 0.33, "threshold": 0.765068769454956, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7546434998512268, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7438755035400391, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7317760586738586, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7199864983558655, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7079719305038452, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6961519122123718, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6844857335090637, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6720460653305054, "operator": "ge"}, {"budget": 0.42, "threshold": 0.660209059715271, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6464763283729553, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6314742565155029, "operator": "ge"}, {"budget": 0.45, "threshold": 0.6155399680137634, "operator": "ge"}, {"budget": 0.46, "threshold": 0.6020910739898682, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5865488052368164, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5709533095359802, "operator": "ge"}, {"budget": 0.49, "threshold": 0.555860698223114, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5402244925498962, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5201042890548706, "operator": "ge"}, {"budget": 0.52, "threshold": 0.502646267414093, "operator": "ge"}, {"budget": 0.53, "threshold": 0.48565033078193665, "operator": "ge"}, {"budget": 0.54, "threshold": 0.4660703241825104, "operator": "ge"}, {"budget": 0.55, "threshold": 0.4483605921268463, "operator": "ge"}, {"budget": 0.56, "threshold": 0.4247564673423767, "operator": "ge"}, {"budget": 0.57, "threshold": 0.4050874710083008, "operator": "ge"}, {"budget": 0.58, "threshold": 0.3865959048271179, "operator": "ge"}, {"budget": 0.59, "threshold": 0.36624211072921753, "operator": "ge"}, {"budget": 0.6, "threshold": 0.3421928286552429, "operator": "ge"}, {"budget": 0.61, "threshold": 0.31574493646621704, "operator": "ge"}, {"budget": 0.62, "threshold": 0.29090845584869385, "operator": "ge"}, {"budget": 0.63, "threshold": 0.26976534724235535, "operator": "ge"}, {"budget": 0.64, "threshold": 0.2419680953025818, "operator": "ge"}, {"budget": 0.65, "threshold": 0.21937642991542816, "operator": "ge"}, {"budget": 0.66, "threshold": 0.19378620386123657, "operator": "ge"}, {"budget": 0.67, "threshold": 0.166166290640831, "operator": "ge"}, {"budget": 0.68, "threshold": 0.13672100007534027, "operator": "ge"}, {"budget": 0.69, "threshold": 0.11188539117574692, "operator": "ge"}, {"budget": 0.7, "threshold": 0.08852220326662064, "operator": "ge"}, {"budget": 0.71, "threshold": 0.06637850403785706, "operator": "ge"}, {"budget": 0.72, "threshold": 0.047062505036592484, "operator": "ge"}, {"budget": 0.73, "threshold": 0.032255928963422775, "operator": "ge"}, {"budget": 0.74, "threshold": 0.021534375846385956, "operator": "ge"}, {"budget": 0.75, "threshold": 0.013070274144411087, "operator": "ge"}, {"budget": 0.76, "threshold": 0.007067253813147545, "operator": "ge"}, {"budget": 0.77, "threshold": 0.0031854999251663685, "operator": "ge"}, {"budget": 0.78, "threshold": 0.0013215304352343082, "operator": "ge"}, {"budget": 0.79, "threshold": 0.0002180344017688185, "operator": "ge"}, {"budget": 0.8, "threshold": -0.00016492334543727338, "operator": "ge"}, {"budget": 0.81, "threshold": -0.0004969676956534386, "operator": "ge"}, {"budget": 0.82, "threshold": -0.000926342501770705, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0014550506602972746, "operator": "ge"}, {"budget": 0.84, "threshold": -0.0019633581396192312, "operator": "ge"}, {"budget": 0.85, "threshold": -0.002694987691938877, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0036991145461797714, "operator": "ge"}, {"budget": 0.87, "threshold": -0.004686216823756695, "operator": "ge"}, {"budget": 0.88, "threshold": -0.006122380495071411, "operator": "ge"}, {"budget": 0.89, "threshold": -0.007851533591747284, "operator": "ge"}, {"budget": 0.9, "threshold": -0.009940072894096375, "operator": "ge"}, {"budget": 0.91, "threshold": -0.012908228673040867, "operator": "ge"}, {"budget": 0.92, "threshold": -0.01660967990756035, "operator": "ge"}, {"budget": 0.93, "threshold": -0.02162068709731102, "operator": "ge"}, {"budget": 0.94, "threshold": -0.028304941952228546, "operator": "ge"}, {"budget": 0.95, "threshold": -0.03931177407503128, "operator": "ge"}, {"budget": 0.96, "threshold": -0.054138630628585815, "operator": "ge"}, {"budget": 0.97, "threshold": -0.08158548176288605, "operator": "ge"}, {"budget": 0.98, "threshold": -0.1277489960193634, "operator": "ge"}, {"budget": 0.99, "threshold": -0.227629616856575, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.0314147164242, 57.94504181600956, 58.823529411764696, 59.65282170215757, 60.503197694848545, 61.3324899852414, 62.11258697027198, 62.864572352238376, 63.64466933726896, 64.31231991004287, 64.89563567362428, 65.4789514372057, 66.02712769695691, 66.60341555977229, 67.07428491109705, 67.53812636165577, 67.99493991144845, 68.52203246890154, 68.943706514864, 69.38646426312461, 69.8151662098531, 70.24386815658163, 70.69365380560826, 71.11532785157073, 71.57214140136341, 71.95867594349568, 72.27493147796753, 72.70363342469604, 73.09719586759435, 73.4837304097266, 73.72970693653805, 74.0459624710099, 74.3341064024176, 74.71361304378381, 74.92445006676505, 75.13528708974629, 75.38829151732376, 75.62724014336916, 75.94349567784103, 76.16838850235435, 76.30894651767517, 76.60411834984889, 76.88523438049054, 77.06093189964157, 77.33502002951718, 77.63019186169092, 77.91130789233256, 78.09403331224962, 78.22756342680441, 78.49462365591397, 78.71248857966124, 78.86007449574811, 79.07091151872935, 79.2676927401785, 79.49961346545787, 79.66828308384285, 79.82289690069577, 80.00562232061283, 80.1532082366997, 80.34296155738281, 80.45540796963945, 80.60299388572632, 80.64516129032258, 80.79977510717549, 80.89816571790006, 80.91924942019818, 80.96141682479443, 81.01061213015672, 81.03872373322089, 81.11603064164734, 81.15117014547755, 81.22144915313795, 81.24253285543608, 81.27767235926629, 81.31281186309648, 81.37606296999085, 81.3479513669267, 81.33389556539461, 81.31281186309648, 81.2987560615644, 81.2987560615644, 81.2987560615644, 81.31281186309648, 81.30578396233045, 81.30578396233045, 81.2987560615644, 81.2987560615644, 81.28470026003231, 81.29172816079837, 81.29172816079837, 81.2987560615644, 81.25658865696816, 81.23550495467003, 81.20739335160587, 81.19333755007378, 81.17225384777566, 81.08089113781713, 80.96844472556047, 80.84194251177172, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.987771452667088, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 140, 272, 401, 525, 652, 777, 895, 1009, 1124, 1228, 1326, 1421, 1512, 1606, 1684, 1762, 1846, 1931, 2010, 2090, 2166, 2242, 2320, 2396, 2476, 2549, 2610, 2685, 2754, 2823, 2882, 2942, 3001, 3069, 3124, 3174, 3226, 3279, 3336, 3390, 3437, 3495, 3548, 3585, 3641, 3696, 3753, 3800, 3838, 3889, 3937, 3975, 4015, 4056, 4098, 4133, 4167, 4208, 4238, 4276, 4304, 4338, 4357, 4396, 4426, 4446, 4462, 4483, 4498, 4517, 4533, 4550, 4562, 4574, 4584, 4596, 4596, 4596, 4596, 4596, 4596, 4596, 4599, 4599, 4600, 4600, 4602, 4602, 4605, 4607, 4608, 4609, 4610, 4610, 4611, 4617, 4620, 4623, 4626, 4626], "harmed": [0, 1, 3, 7, 13, 19, 26, 33, 40, 44, 53, 68, 80, 93, 105, 116, 128, 147, 157, 176, 193, 208, 223, 237, 253, 268, 286, 302, 316, 329, 343, 367, 382, 400, 414, 439, 459, 475, 494, 506, 528, 555, 571, 584, 596, 613, 626, 643, 664, 683, 696, 713, 730, 740, 753, 762, 773, 785, 800, 809, 820, 832, 845, 858, 875, 891, 908, 918, 932, 943, 951, 962, 969, 978, 985, 990, 993, 997, 999, 1002, 1004, 1004, 1004, 1005, 1006, 1007, 1008, 1010, 1012, 1014, 1016, 1016, 1023, 1027, 1031, 1034, 1043, 1059, 1078, 1099, 1133], "exact_score": [56.05453650994448, 57.0314147164242, 57.94504181600956, 58.823529411764696, 59.65282170215757, 60.503197694848545, 61.33951788600744, 62.11258697027198, 62.864572352238376, 63.64466933726896, 64.31231991004287, 64.89563567362428, 65.4789514372057, 66.02712769695691, 66.60341555977229, 67.07428491109705, 67.53812636165577, 67.99493991144845, 68.52203246890154, 68.943706514864, 69.38646426312461, 69.8151662098531, 70.24386815658163, 70.69365380560826, 71.11532785157073, 71.57214140136341, 71.95867594349568, 72.27493147796753, 72.70363342469604, 73.09719586759435, 73.4837304097266, 73.72970693653805, 74.0459624710099, 74.3341064024176, 74.71361304378381, 74.92445006676505, 75.13528708974629, 75.38829151732376, 75.62724014336916, 75.94349567784103, 76.16838850235435, 76.30894651767517, 76.60411834984889, 76.88523438049054, 77.06093189964157, 77.33502002951718, 77.63019186169092, 77.91130789233256, 78.09403331224962, 78.22756342680441, 78.49462365591397, 78.71248857966124, 78.86007449574811, 79.07091151872935, 79.2676927401785, 79.49961346545787, 79.66828308384285, 79.82289690069577, 80.00562232061283, 80.1532082366997, 80.34296155738281, 80.45540796963945, 80.60299388572632, 80.64516129032258, 80.79977510717549, 80.89816571790006, 80.91924942019818, 80.96141682479443, 81.01061213015672, 81.03872373322089, 81.11603064164734, 81.15117014547755, 81.22144915313795, 81.24253285543608, 81.27767235926629, 81.31281186309648, 81.37606296999085, 81.3479513669267, 81.33389556539461, 81.31281186309648, 81.2987560615644, 81.2987560615644, 81.2987560615644, 81.31281186309648, 81.30578396233045, 81.30578396233045, 81.2987560615644, 81.2987560615644, 81.28470026003231, 81.29172816079837, 81.29172816079837, 81.2987560615644, 81.25658865696816, 81.23550495467003, 81.20739335160587, 81.19333755007378, 81.17225384777566, 81.08089113781713, 80.96844472556047, 80.84194251177172, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.50596057507207, "threshold_area100": 75.50506008855155, "threshold_area25": 65.01792114695341}, "test": {"score": [26.688029946402747, 28.298783427388486, 29.781924396676406, 31.551484558886088, 32.997759691461304, 34.228511470946884, 35.589711595723564, 36.9395683861271, 38.21852933669852, 39.70167030598645, 40.838839576893626, 42.08377052434563, 43.45064231630889, 44.54527408331679, 45.60871168079858, 46.54453676658254, 47.64200436718373, 48.65439695998639, 49.52499787312481, 50.276493775345266, 51.07903468224485, 51.887247256331, 52.64725065933131, 53.302328219380094, 54.070839123160255, 54.67203584493663, 55.36114340810481, 56.01338513456031, 56.68547769616878, 57.164733573433914, 57.84533363582225, 58.389813685732925, 59.02220457703542, 59.60355046365879, 60.16788134872246, 60.81445140799138, 61.45534980007373, 62.053710688256814, 62.57266823582792, 63.049088279499756, 63.616254998156705, 64.06998837308227, 64.54073674956754, 64.91506678388113, 65.43118849785895, 65.91044437512407, 66.32164024615037, 66.76403028670278, 67.21776366162834, 67.64597453421433, 68.05149873805405, 68.64418795905056, 69.11493633553583, 69.54031137452854, 70.06777642287952, 70.44494229078637, 70.95822817117093, 71.4261407140629, 71.76644074525707, 72.17196494909679, 72.61151915605592, 73.05674503020163, 73.3601792246831, 73.56435924339961, 73.87630093866092, 74.1031676261237, 74.29884014406034, 74.46331849247086, 74.67033434478066, 74.8121260244449, 74.92839520176956, 74.99078354082182, 75.05317187987409, 75.10421688455322, 75.1240677197062, 75.13257522048606, 75.14108272126592, 75.15242605563905, 75.15526188923234, 75.16093355641891, 75.16093355641891, 75.15809772282563, 75.15526188923234, 75.1439185548592, 75.1439185548592, 75.14108272126592, 75.13257522048606, 75.12973938689278, 75.11556021892636, 75.11839605251964, 75.11556021892636, 75.10988855173979, 75.11272438533308, 75.10988855173979, 75.1070527181465, 75.08153021580695, 75.03048521112781, 75.02197771034795, 74.9964552080084, 74.86033519553072, 74.47182599325072], "calls": [0.0, 1.6589626520715766, 3.272551966650597, 5.1697246405580914, 6.820179791849815, 8.238096588492187, 9.826163400731645, 11.400051045004679, 12.90587868303888, 14.700961347588123, 16.181266483282762, 17.851572469727476, 19.61546096475059, 21.163826106684063, 22.542041233020445, 23.908913024983693, 25.42041233020446, 26.81564245810056, 27.98684173212716, 29.138190171000765, 30.31222527862065, 31.59118622919207, 32.87014717976349, 33.95059977880498, 35.17567989110399, 36.16254998156708, 37.29688341888098, 38.411366021041886, 39.46346028415053, 40.43898704024048, 41.544962141621525, 42.48929472818535, 43.51303065536115, 44.55094575050336, 45.603040013612, 46.63811927516093, 47.83484105152709, 48.90395031619544, 50.018432918356346, 51.03649717834558, 52.09993477582735, 53.13784987096957, 54.04815245441398, 55.05487338003006, 56.1835351501574, 57.363241924963845, 58.429515356038905, 59.48444545274083, 60.57340555256218, 61.577290644584984, 62.56983240223464, 63.95088336216431, 65.06536596432522, 66.19119190085925, 67.34254033973286, 68.39747043643479, 69.71046139012562, 70.86180982899923, 71.7976349147832, 72.82704250914557, 74.00674928395202, 75.26018773218388, 76.35765533278507, 77.13750957093838, 78.20661883560672, 79.09423475030485, 79.95065649547685, 80.82692907580183, 81.83365000141791, 82.63902674191078, 83.41037347928423, 84.18172021665768, 85.00694779230355, 85.71307035703146, 86.33695374755409, 87.02889714431558, 87.72651220826361, 88.46382894251765, 89.24651901426425, 90.1114482602161, 90.66159997731333, 91.10966168505232, 91.60026089669057, 92.09369594192212, 92.41130930437002, 92.90190851600828, 93.39817939483311, 93.72713609165415, 94.18370530017299, 94.59490117119927, 94.97490287269943, 95.4655020843377, 95.88520545614384, 96.30207299435669, 96.66505969429714, 97.08476306610329, 97.47894393556986, 97.92984147690214, 98.43178402291353, 98.98760740719734, 100.0], "rescued": [0, 573, 1109, 1747, 2280, 2730, 3234, 3726, 4193, 4744, 5172, 5649, 6167, 6592, 6991, 7349, 7766, 8145, 8467, 8762, 9069, 9385, 9689, 9948, 10251, 10487, 10762, 11017, 11279, 11483, 11740, 11952, 12195, 12414, 12634, 12884, 13138, 13370, 13583, 13774, 13998, 14183, 14366, 14532, 14737, 14935, 15113, 15295, 15477, 15648, 15810, 16050, 16231, 16406, 16615, 16768, 16971, 17153, 17284, 17441, 17613, 17786, 17910, 17998, 18122, 18213, 18296, 18366, 18457, 18516, 18568, 18604, 18639, 18674, 18690, 18701, 18710, 18716, 18721, 18724, 18725, 18725, 18725, 18725, 18725, 18726, 18726, 18727, 18727, 18729, 18732, 18736, 18743, 18749, 18756, 18763, 18765, 18777, 18795, 18810, 18820], "harmed": [0, 5, 18, 32, 55, 71, 95, 111, 127, 155, 182, 220, 256, 295, 319, 347, 377, 399, 414, 444, 468, 499, 535, 563, 595, 619, 651, 676, 701, 736, 753, 773, 793, 807, 828, 850, 878, 899, 929, 952, 976, 1001, 1018, 1052, 1075, 1104, 1137, 1163, 1185, 1205, 1224, 1255, 1270, 1295, 1318, 1338, 1360, 1377, 1388, 1402, 1419, 1435, 1452, 1468, 1482, 1493, 1507, 1519, 1537, 1546, 1557, 1571, 1584, 1601, 1610, 1618, 1624, 1626, 1630, 1631, 1632, 1633, 1634, 1638, 1638, 1640, 1643, 1645, 1650, 1651, 1655, 1661, 1667, 1674, 1682, 1698, 1718, 1733, 1760, 1823, 1970], "exact_score": [26.688029946402747, 27.66922836967927, 28.616396789836372, 29.53520687406063, 30.459688625471458, 31.384170376882288, 32.28312962595355, 33.15373053909197, 34.01298811785725, 34.8779173638091, 35.74284660976094, 36.619119190085925, 37.447182599325075, 38.29226101012393, 39.108981084989935, 39.939880327822365, 40.69137623004282, 41.44287213226328, 42.20571136885688, 42.98556560701018, 43.75124067719707, 44.431840739585404, 45.197515809772284, 45.92348920965318, 46.621104273601226, 47.33857017270226, 48.07305107336302, 48.8018603068372, 49.53350537390466, 50.20559793551315, 50.849332161188784, 51.50157388764427, 52.12262144457364, 52.73516150072314, 53.33635822249951, 53.963077446615436, 54.58979667073136, 55.17397839094802, 55.75816011116468, 56.37920766809403, 56.969061055497264, 57.51070527181466, 58.120409494370875, 58.684740379434544, 59.30295210277061, 59.850267986274574, 60.43161387289794, 60.99310892436832, 61.560275643025264, 62.09057652496952, 62.56132490145478, 63.03490911153333, 63.568045827070875, 63.999092533250156, 64.51521424722797, 64.9008876159147, 65.35462099084026, 65.77716019623969, 66.15999773133312, 66.55134276720642, 66.99656864135213, 67.38224201003885, 67.79627371465843, 68.22164875365114, 68.66687462779683, 69.08090633241642, 69.49210220344271, 69.92314890962199, 70.27479227518928, 70.65479397668946, 71.0744973484956, 71.48569321952188, 71.82882908430935, 72.22868162096248, 72.60868332246264, 72.95749085443666, 73.2637608825114, 73.52182173950033, 73.83376343476165, 74.086152624564, 74.30451181124693, 74.50018432918357, 74.70720018149335, 74.87451436349714, 74.97376853926212, 75.05317187987409, 75.1240677197062, 75.13257522048606, 75.14959022204577, 75.15809772282563, 75.15809772282563, 75.16093355641891, 75.14675438845248, 75.14108272126592, 75.11839605251964, 75.11556021892636, 75.10988855173979, 75.08720188299351, 75.03048521112781, 74.85466352834416, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.90618240827161, "threshold_area100": 63.15239769730313, "threshold_area25": 42.674191078467516}, "selected_budget": 76, "selected": {"val": {"score": 81.37606296999085, "calls": 75.99971888396936, "rescued": 4596, "harmed": 993}, "test": {"score": 75.14108272126592, "calls": 87.72651220826361, "rescued": 18710, "harmed": 1624}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9959196448326111, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9930047392845154, "operator": "ge"}, {"budget": 0.03, "threshold": 0.9894710183143616, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9859538674354553, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9826302528381348, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9785681962966919, "operator": "gt"}, {"budget": 0.07, "threshold": 0.9741941094398499, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9694491028785706, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9633454084396362, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9581807851791382, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9517661333084106, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9442238211631775, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9366728663444519, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9303407669067383, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9238371849060059, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9159175157546997, "operator": "ge"}, {"budget": 0.17, "threshold": 0.9081736207008362, "operator": "ge"}, {"budget": 0.18, "threshold": 0.901128351688385, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8936429619789124, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8855038285255432, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8767584562301636, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8674139380455017, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8591026663780212, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8492180109024048, "operator": "ge"}, {"budget": 0.25, "threshold": 0.841093122959137, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8319987654685974, "operator": "ge"}, {"budget": 0.27, "threshold": 0.822016716003418, "operator": "ge"}, {"budget": 0.28, "threshold": 0.8119505643844604, "operator": "ge"}, {"budget": 0.29, "threshold": 0.8031176328659058, "operator": "ge"}, {"budget": 0.3, "threshold": 0.7930914759635925, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7845637798309326, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7741467952728271, "operator": "ge"}, {"budget": 0.33, "threshold": 0.765068769454956, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7546434998512268, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7438755035400391, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7317760586738586, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7199864983558655, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7079719305038452, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6961519122123718, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6844857335090637, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6720460653305054, "operator": "ge"}, {"budget": 0.42, "threshold": 0.660209059715271, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6464763283729553, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6314742565155029, "operator": "ge"}, {"budget": 0.45, "threshold": 0.6155399680137634, "operator": "ge"}, {"budget": 0.46, "threshold": 0.6020910739898682, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5865488052368164, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5709533095359802, "operator": "ge"}, {"budget": 0.49, "threshold": 0.555860698223114, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5402244925498962, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5201042890548706, "operator": "ge"}, {"budget": 0.52, "threshold": 0.502646267414093, "operator": "ge"}, {"budget": 0.53, "threshold": 0.48565033078193665, "operator": "ge"}, {"budget": 0.54, "threshold": 0.4660703241825104, "operator": "ge"}, {"budget": 0.55, "threshold": 0.4483605921268463, "operator": "ge"}, {"budget": 0.56, "threshold": 0.4247564673423767, "operator": "ge"}, {"budget": 0.57, "threshold": 0.4050874710083008, "operator": "ge"}, {"budget": 0.58, "threshold": 0.3865959048271179, "operator": "ge"}, {"budget": 0.59, "threshold": 0.36624211072921753, "operator": "ge"}, {"budget": 0.6, "threshold": 0.3421928286552429, "operator": "ge"}, {"budget": 0.61, "threshold": 0.31574493646621704, "operator": "ge"}, {"budget": 0.62, "threshold": 0.29090845584869385, "operator": "ge"}, {"budget": 0.63, "threshold": 0.26976534724235535, "operator": "ge"}, {"budget": 0.64, "threshold": 0.2419680953025818, "operator": "ge"}, {"budget": 0.65, "threshold": 0.21937642991542816, "operator": "ge"}, {"budget": 0.66, "threshold": 0.19378620386123657, "operator": "ge"}, {"budget": 0.67, "threshold": 0.166166290640831, "operator": "ge"}, {"budget": 0.68, "threshold": 0.13672100007534027, "operator": "ge"}, {"budget": 0.69, "threshold": 0.11188539117574692, "operator": "ge"}, {"budget": 0.7, "threshold": 0.08852220326662064, "operator": "ge"}, {"budget": 0.71, "threshold": 0.06637850403785706, "operator": "ge"}, {"budget": 0.72, "threshold": 0.047062505036592484, "operator": "ge"}, {"budget": 0.73, "threshold": 0.032255928963422775, "operator": "ge"}, {"budget": 0.74, "threshold": 0.021534375846385956, "operator": "ge"}, {"budget": 0.75, "threshold": 0.013070274144411087, "operator": "ge"}, {"budget": 0.76, "threshold": 0.007067253813147545, "operator": "ge"}, {"budget": 0.77, "threshold": 0.0031854999251663685, "operator": "ge"}, {"budget": 0.78, "threshold": 0.0013215304352343082, "operator": "ge"}, {"budget": 0.79, "threshold": 0.0002180344017688185, "operator": "ge"}, {"budget": 0.8, "threshold": -0.00016492334543727338, "operator": "ge"}, {"budget": 0.81, "threshold": -0.0004969676956534386, "operator": "ge"}, {"budget": 0.82, "threshold": -0.000926342501770705, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0014550506602972746, "operator": "ge"}, {"budget": 0.84, "threshold": -0.0019633581396192312, "operator": "ge"}, {"budget": 0.85, "threshold": -0.002694987691938877, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0036991145461797714, "operator": "ge"}, {"budget": 0.87, "threshold": -0.004686216823756695, "operator": "ge"}, {"budget": 0.88, "threshold": -0.006122380495071411, "operator": "ge"}, {"budget": 0.89, "threshold": -0.007851533591747284, "operator": "ge"}, {"budget": 0.9, "threshold": -0.009940072894096375, "operator": "ge"}, {"budget": 0.91, "threshold": -0.012908228673040867, "operator": "ge"}, {"budget": 0.92, "threshold": -0.01660967990756035, "operator": "ge"}, {"budget": 0.93, "threshold": -0.02162068709731102, "operator": "ge"}, {"budget": 0.94, "threshold": -0.028304941952228546, "operator": "ge"}, {"budget": 0.95, "threshold": -0.03931177407503128, "operator": "ge"}, {"budget": 0.96, "threshold": -0.054138630628585815, "operator": "ge"}, {"budget": 0.97, "threshold": -0.08158548176288605, "operator": "ge"}, {"budget": 0.98, "threshold": -0.1277489960193634, "operator": "ge"}, {"budget": 0.99, "threshold": -0.227629616856575, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.0314147164242, 57.94504181600956, 58.823529411764696, 59.65282170215757, 60.503197694848545, 61.3324899852414, 62.11258697027198, 62.864572352238376, 63.64466933726896, 64.31231991004287, 64.89563567362428, 65.4789514372057, 66.02712769695691, 66.60341555977229, 67.07428491109705, 67.53812636165577, 67.99493991144845, 68.52203246890154, 68.943706514864, 69.38646426312461, 69.8151662098531, 70.24386815658163, 70.69365380560826, 71.11532785157073, 71.57214140136341, 71.95867594349568, 72.27493147796753, 72.70363342469604, 73.09719586759435, 73.4837304097266, 73.72970693653805, 74.0459624710099, 74.3341064024176, 74.71361304378381, 74.92445006676505, 75.13528708974629, 75.38829151732376, 75.62724014336916, 75.94349567784103, 76.16838850235435, 76.30894651767517, 76.60411834984889, 76.88523438049054, 77.06093189964157, 77.33502002951718, 77.63019186169092, 77.91130789233256, 78.09403331224962, 78.22756342680441, 78.49462365591397, 78.71248857966124, 78.86007449574811, 79.07091151872935, 79.2676927401785, 79.49961346545787, 79.66828308384285, 79.82289690069577, 80.00562232061283, 80.1532082366997, 80.34296155738281, 80.45540796963945, 80.60299388572632, 80.64516129032258, 80.79977510717549, 80.89816571790006, 80.91924942019818, 80.96141682479443, 81.01061213015672, 81.03872373322089, 81.11603064164734, 81.15117014547755, 81.22144915313795, 81.24253285543608, 81.27767235926629, 81.31281186309648, 81.37606296999085, 81.3479513669267, 81.33389556539461, 81.31281186309648, 81.2987560615644, 81.2987560615644, 81.2987560615644, 81.31281186309648, 81.30578396233045, 81.30578396233045, 81.2987560615644, 81.2987560615644, 81.28470026003231, 81.29172816079837, 81.29172816079837, 81.2987560615644, 81.25658865696816, 81.23550495467003, 81.20739335160587, 81.19333755007378, 81.17225384777566, 81.08089113781713, 80.96844472556047, 80.84194251177172, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.987771452667088, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 140, 272, 401, 525, 652, 777, 895, 1009, 1124, 1228, 1326, 1421, 1512, 1606, 1684, 1762, 1846, 1931, 2010, 2090, 2166, 2242, 2320, 2396, 2476, 2549, 2610, 2685, 2754, 2823, 2882, 2942, 3001, 3069, 3124, 3174, 3226, 3279, 3336, 3390, 3437, 3495, 3548, 3585, 3641, 3696, 3753, 3800, 3838, 3889, 3937, 3975, 4015, 4056, 4098, 4133, 4167, 4208, 4238, 4276, 4304, 4338, 4357, 4396, 4426, 4446, 4462, 4483, 4498, 4517, 4533, 4550, 4562, 4574, 4584, 4596, 4596, 4596, 4596, 4596, 4596, 4596, 4599, 4599, 4600, 4600, 4602, 4602, 4605, 4607, 4608, 4609, 4610, 4610, 4611, 4617, 4620, 4623, 4626, 4626], "harmed": [0, 1, 3, 7, 13, 19, 26, 33, 40, 44, 53, 68, 80, 93, 105, 116, 128, 147, 157, 176, 193, 208, 223, 237, 253, 268, 286, 302, 316, 329, 343, 367, 382, 400, 414, 439, 459, 475, 494, 506, 528, 555, 571, 584, 596, 613, 626, 643, 664, 683, 696, 713, 730, 740, 753, 762, 773, 785, 800, 809, 820, 832, 845, 858, 875, 891, 908, 918, 932, 943, 951, 962, 969, 978, 985, 990, 993, 997, 999, 1002, 1004, 1004, 1004, 1005, 1006, 1007, 1008, 1010, 1012, 1014, 1016, 1016, 1023, 1027, 1031, 1034, 1043, 1059, 1078, 1099, 1133], "exact_score": [56.05453650994448, 57.0314147164242, 57.94504181600956, 58.823529411764696, 59.65282170215757, 60.503197694848545, 61.33951788600744, 62.11258697027198, 62.864572352238376, 63.64466933726896, 64.31231991004287, 64.89563567362428, 65.4789514372057, 66.02712769695691, 66.60341555977229, 67.07428491109705, 67.53812636165577, 67.99493991144845, 68.52203246890154, 68.943706514864, 69.38646426312461, 69.8151662098531, 70.24386815658163, 70.69365380560826, 71.11532785157073, 71.57214140136341, 71.95867594349568, 72.27493147796753, 72.70363342469604, 73.09719586759435, 73.4837304097266, 73.72970693653805, 74.0459624710099, 74.3341064024176, 74.71361304378381, 74.92445006676505, 75.13528708974629, 75.38829151732376, 75.62724014336916, 75.94349567784103, 76.16838850235435, 76.30894651767517, 76.60411834984889, 76.88523438049054, 77.06093189964157, 77.33502002951718, 77.63019186169092, 77.91130789233256, 78.09403331224962, 78.22756342680441, 78.49462365591397, 78.71248857966124, 78.86007449574811, 79.07091151872935, 79.2676927401785, 79.49961346545787, 79.66828308384285, 79.82289690069577, 80.00562232061283, 80.1532082366997, 80.34296155738281, 80.45540796963945, 80.60299388572632, 80.64516129032258, 80.79977510717549, 80.89816571790006, 80.91924942019818, 80.96141682479443, 81.01061213015672, 81.03872373322089, 81.11603064164734, 81.15117014547755, 81.22144915313795, 81.24253285543608, 81.27767235926629, 81.31281186309648, 81.37606296999085, 81.3479513669267, 81.33389556539461, 81.31281186309648, 81.2987560615644, 81.2987560615644, 81.2987560615644, 81.31281186309648, 81.30578396233045, 81.30578396233045, 81.2987560615644, 81.2987560615644, 81.28470026003231, 81.29172816079837, 81.29172816079837, 81.2987560615644, 81.25658865696816, 81.23550495467003, 81.20739335160587, 81.19333755007378, 81.17225384777566, 81.08089113781713, 80.96844472556047, 80.84194251177172, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.50596057507207, "threshold_area100": 75.50506008855155, "threshold_area25": 65.01792114695341}, "test": {"score": [26.688029946402747, 28.298783427388486, 29.781924396676406, 31.551484558886088, 32.997759691461304, 34.228511470946884, 35.589711595723564, 36.9395683861271, 38.21852933669852, 39.70167030598645, 40.838839576893626, 42.08377052434563, 43.45064231630889, 44.54527408331679, 45.60871168079858, 46.54453676658254, 47.64200436718373, 48.65439695998639, 49.52499787312481, 50.276493775345266, 51.07903468224485, 51.887247256331, 52.64725065933131, 53.302328219380094, 54.070839123160255, 54.67203584493663, 55.36114340810481, 56.01338513456031, 56.68547769616878, 57.164733573433914, 57.84533363582225, 58.389813685732925, 59.02220457703542, 59.60355046365879, 60.16788134872246, 60.81445140799138, 61.45534980007373, 62.053710688256814, 62.57266823582792, 63.049088279499756, 63.616254998156705, 64.06998837308227, 64.54073674956754, 64.91506678388113, 65.43118849785895, 65.91044437512407, 66.32164024615037, 66.76403028670278, 67.21776366162834, 67.64597453421433, 68.05149873805405, 68.64418795905056, 69.11493633553583, 69.54031137452854, 70.06777642287952, 70.44494229078637, 70.95822817117093, 71.4261407140629, 71.76644074525707, 72.17196494909679, 72.61151915605592, 73.05674503020163, 73.3601792246831, 73.56435924339961, 73.87630093866092, 74.1031676261237, 74.29884014406034, 74.46331849247086, 74.67033434478066, 74.8121260244449, 74.92839520176956, 74.99078354082182, 75.05317187987409, 75.10421688455322, 75.1240677197062, 75.13257522048606, 75.14108272126592, 75.15242605563905, 75.15526188923234, 75.16093355641891, 75.16093355641891, 75.15809772282563, 75.15526188923234, 75.1439185548592, 75.1439185548592, 75.14108272126592, 75.13257522048606, 75.12973938689278, 75.11556021892636, 75.11839605251964, 75.11556021892636, 75.10988855173979, 75.11272438533308, 75.10988855173979, 75.1070527181465, 75.08153021580695, 75.03048521112781, 75.02197771034795, 74.9964552080084, 74.86033519553072, 74.47182599325072], "calls": [0.0, 1.6589626520715766, 3.272551966650597, 5.1697246405580914, 6.820179791849815, 8.238096588492187, 9.826163400731645, 11.400051045004679, 12.90587868303888, 14.700961347588123, 16.181266483282762, 17.851572469727476, 19.61546096475059, 21.163826106684063, 22.542041233020445, 23.908913024983693, 25.42041233020446, 26.81564245810056, 27.98684173212716, 29.138190171000765, 30.31222527862065, 31.59118622919207, 32.87014717976349, 33.95059977880498, 35.17567989110399, 36.16254998156708, 37.29688341888098, 38.411366021041886, 39.46346028415053, 40.43898704024048, 41.544962141621525, 42.48929472818535, 43.51303065536115, 44.55094575050336, 45.603040013612, 46.63811927516093, 47.83484105152709, 48.90395031619544, 50.018432918356346, 51.03649717834558, 52.09993477582735, 53.13784987096957, 54.04815245441398, 55.05487338003006, 56.1835351501574, 57.363241924963845, 58.429515356038905, 59.48444545274083, 60.57340555256218, 61.577290644584984, 62.56983240223464, 63.95088336216431, 65.06536596432522, 66.19119190085925, 67.34254033973286, 68.39747043643479, 69.71046139012562, 70.86180982899923, 71.7976349147832, 72.82704250914557, 74.00674928395202, 75.26018773218388, 76.35765533278507, 77.13750957093838, 78.20661883560672, 79.09423475030485, 79.95065649547685, 80.82692907580183, 81.83365000141791, 82.63902674191078, 83.41037347928423, 84.18172021665768, 85.00694779230355, 85.71307035703146, 86.33695374755409, 87.02889714431558, 87.72651220826361, 88.46382894251765, 89.24651901426425, 90.1114482602161, 90.66159997731333, 91.10966168505232, 91.60026089669057, 92.09369594192212, 92.41130930437002, 92.90190851600828, 93.39817939483311, 93.72713609165415, 94.18370530017299, 94.59490117119927, 94.97490287269943, 95.4655020843377, 95.88520545614384, 96.30207299435669, 96.66505969429714, 97.08476306610329, 97.47894393556986, 97.92984147690214, 98.43178402291353, 98.98760740719734, 100.0], "rescued": [0, 573, 1109, 1747, 2280, 2730, 3234, 3726, 4193, 4744, 5172, 5649, 6167, 6592, 6991, 7349, 7766, 8145, 8467, 8762, 9069, 9385, 9689, 9948, 10251, 10487, 10762, 11017, 11279, 11483, 11740, 11952, 12195, 12414, 12634, 12884, 13138, 13370, 13583, 13774, 13998, 14183, 14366, 14532, 14737, 14935, 15113, 15295, 15477, 15648, 15810, 16050, 16231, 16406, 16615, 16768, 16971, 17153, 17284, 17441, 17613, 17786, 17910, 17998, 18122, 18213, 18296, 18366, 18457, 18516, 18568, 18604, 18639, 18674, 18690, 18701, 18710, 18716, 18721, 18724, 18725, 18725, 18725, 18725, 18725, 18726, 18726, 18727, 18727, 18729, 18732, 18736, 18743, 18749, 18756, 18763, 18765, 18777, 18795, 18810, 18820], "harmed": [0, 5, 18, 32, 55, 71, 95, 111, 127, 155, 182, 220, 256, 295, 319, 347, 377, 399, 414, 444, 468, 499, 535, 563, 595, 619, 651, 676, 701, 736, 753, 773, 793, 807, 828, 850, 878, 899, 929, 952, 976, 1001, 1018, 1052, 1075, 1104, 1137, 1163, 1185, 1205, 1224, 1255, 1270, 1295, 1318, 1338, 1360, 1377, 1388, 1402, 1419, 1435, 1452, 1468, 1482, 1493, 1507, 1519, 1537, 1546, 1557, 1571, 1584, 1601, 1610, 1618, 1624, 1626, 1630, 1631, 1632, 1633, 1634, 1638, 1638, 1640, 1643, 1645, 1650, 1651, 1655, 1661, 1667, 1674, 1682, 1698, 1718, 1733, 1760, 1823, 1970], "exact_score": [26.688029946402747, 27.66922836967927, 28.616396789836372, 29.53520687406063, 30.459688625471458, 31.384170376882288, 32.28312962595355, 33.15373053909197, 34.01298811785725, 34.8779173638091, 35.74284660976094, 36.619119190085925, 37.447182599325075, 38.29226101012393, 39.108981084989935, 39.939880327822365, 40.69137623004282, 41.44287213226328, 42.20571136885688, 42.98556560701018, 43.75124067719707, 44.431840739585404, 45.197515809772284, 45.92348920965318, 46.621104273601226, 47.33857017270226, 48.07305107336302, 48.8018603068372, 49.53350537390466, 50.20559793551315, 50.849332161188784, 51.50157388764427, 52.12262144457364, 52.73516150072314, 53.33635822249951, 53.963077446615436, 54.58979667073136, 55.17397839094802, 55.75816011116468, 56.37920766809403, 56.969061055497264, 57.51070527181466, 58.120409494370875, 58.684740379434544, 59.30295210277061, 59.850267986274574, 60.43161387289794, 60.99310892436832, 61.560275643025264, 62.09057652496952, 62.56132490145478, 63.03490911153333, 63.568045827070875, 63.999092533250156, 64.51521424722797, 64.9008876159147, 65.35462099084026, 65.77716019623969, 66.15999773133312, 66.55134276720642, 66.99656864135213, 67.38224201003885, 67.79627371465843, 68.22164875365114, 68.66687462779683, 69.08090633241642, 69.49210220344271, 69.92314890962199, 70.27479227518928, 70.65479397668946, 71.0744973484956, 71.48569321952188, 71.82882908430935, 72.22868162096248, 72.60868332246264, 72.95749085443666, 73.2637608825114, 73.52182173950033, 73.83376343476165, 74.086152624564, 74.30451181124693, 74.50018432918357, 74.70720018149335, 74.87451436349714, 74.97376853926212, 75.05317187987409, 75.1240677197062, 75.13257522048606, 75.14959022204577, 75.15809772282563, 75.15809772282563, 75.16093355641891, 75.14675438845248, 75.14108272126592, 75.11839605251964, 75.11556021892636, 75.10988855173979, 75.08720188299351, 75.03048521112781, 74.85466352834416, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.90618240827161, "threshold_area100": 63.15239769730313, "threshold_area25": 42.674191078467516}, "selected_budget": 76, "selected": {"val": {"score": 81.37606296999085, "calls": 75.99971888396936, "rescued": 4596, "harmed": 993}, "test": {"score": 75.14108272126592, "calls": 87.72651220826361, "rescued": 18710, "harmed": 1624}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 16.278011322021484, "operator": "ge"}, {"budget": 0.02, "threshold": 15.374265670776367, "operator": "gt"}, {"budget": 0.03, "threshold": 14.695771217346191, "operator": "ge"}, {"budget": 0.04, "threshold": 14.216829299926758, "operator": "ge"}, {"budget": 0.05, "threshold": 13.8637113571167, "operator": "ge"}, {"budget": 0.06, "threshold": 13.479995727539062, "operator": "ge"}, {"budget": 0.07, "threshold": 13.17596435546875, "operator": "gt"}, {"budget": 0.08, "threshold": 12.911428451538086, "operator": "ge"}, {"budget": 0.09, "threshold": 12.661660194396973, "operator": "ge"}, {"budget": 0.1, "threshold": 12.389680862426758, "operator": "gt"}, {"budget": 0.11, "threshold": 12.131473541259766, "operator": "ge"}, {"budget": 0.12, "threshold": 11.890420913696289, "operator": "ge"}, {"budget": 0.13, "threshold": 11.676507949829102, "operator": "ge"}, {"budget": 0.14, "threshold": 11.412691116333008, "operator": "ge"}, {"budget": 0.15, "threshold": 11.194421768188477, "operator": "ge"}, {"budget": 0.16, "threshold": 10.980568885803223, "operator": "ge"}, {"budget": 0.17, "threshold": 10.753252029418945, "operator": "ge"}, {"budget": 0.18, "threshold": 10.543036460876465, "operator": "ge"}, {"budget": 0.19, "threshold": 10.292377471923828, "operator": "ge"}, {"budget": 0.2, "threshold": 10.055194854736328, "operator": "ge"}, {"budget": 0.21, "threshold": 9.824130058288574, "operator": "ge"}, {"budget": 0.22, "threshold": 9.613037109375, "operator": "ge"}, {"budget": 0.23, "threshold": 9.403998374938965, "operator": "ge"}, {"budget": 0.24, "threshold": 9.124241828918457, "operator": "ge"}, {"budget": 0.25, "threshold": 8.879643440246582, "operator": "ge"}, {"budget": 0.26, "threshold": 8.638233184814453, "operator": "ge"}, {"budget": 0.27, "threshold": 8.405908584594727, "operator": "ge"}, {"budget": 0.28, "threshold": 8.142363548278809, "operator": "ge"}, {"budget": 0.29, "threshold": 7.911582946777344, "operator": "ge"}, {"budget": 0.3, "threshold": 7.674145698547363, "operator": "ge"}, {"budget": 0.31, "threshold": 7.461151123046875, "operator": "ge"}, {"budget": 0.32, "threshold": 7.221146583557129, "operator": "ge"}, {"budget": 0.33, "threshold": 6.979557037353516, "operator": "ge"}, {"budget": 0.34, "threshold": 6.750462055206299, "operator": "ge"}, {"budget": 0.35, "threshold": 6.507512092590332, "operator": "gt"}, {"budget": 0.36, "threshold": 6.299169063568115, "operator": "ge"}, {"budget": 0.37, "threshold": 6.094077110290527, "operator": "ge"}, {"budget": 0.38, "threshold": 5.873193264007568, "operator": "ge"}, {"budget": 0.39, "threshold": 5.701839923858643, "operator": "ge"}, {"budget": 0.4, "threshold": 5.508599758148193, "operator": "ge"}, {"budget": 0.41, "threshold": 5.346133232116699, "operator": "ge"}, {"budget": 0.42, "threshold": 5.178073883056641, "operator": "ge"}, {"budget": 0.43, "threshold": 5.035011291503906, "operator": "ge"}, {"budget": 0.44, "threshold": 4.876079559326172, "operator": "ge"}, {"budget": 0.45, "threshold": 4.716022968292236, "operator": "ge"}, {"budget": 0.46, "threshold": 4.558294296264648, "operator": "ge"}, {"budget": 0.47, "threshold": 4.4052581787109375, "operator": "ge"}, {"budget": 0.48, "threshold": 4.270764350891113, "operator": "ge"}, {"budget": 0.49, "threshold": 4.125513076782227, "operator": "ge"}, {"budget": 0.5, "threshold": 3.9895026683807373, "operator": "ge"}, {"budget": 0.51, "threshold": 3.8487889766693115, "operator": "ge"}, {"budget": 0.52, "threshold": 3.753502607345581, "operator": "ge"}, {"budget": 0.53, "threshold": 3.624183416366577, "operator": "ge"}, {"budget": 0.54, "threshold": 3.483443260192871, "operator": "ge"}, {"budget": 0.55, "threshold": 3.3489480018615723, "operator": "ge"}, {"budget": 0.56, "threshold": 3.2149102687835693, "operator": "ge"}, {"budget": 0.57, "threshold": 3.084137439727783, "operator": "ge"}, {"budget": 0.58, "threshold": 2.971334218978882, "operator": "ge"}, {"budget": 0.59, "threshold": 2.8411214351654053, "operator": "ge"}, {"budget": 0.6, "threshold": 2.7223715782165527, "operator": "ge"}, {"budget": 0.61, "threshold": 2.606628179550171, "operator": "ge"}, {"budget": 0.62, "threshold": 2.499879837036133, "operator": "ge"}, {"budget": 0.63, "threshold": 2.365750551223755, "operator": "ge"}, {"budget": 0.64, "threshold": 2.234246253967285, "operator": "ge"}, {"budget": 0.65, "threshold": 2.0883891582489014, "operator": "ge"}, {"budget": 0.66, "threshold": 1.9607768058776855, "operator": "ge"}, {"budget": 0.67, "threshold": 1.8319553136825562, "operator": "ge"}, {"budget": 0.68, "threshold": 1.6873178482055664, "operator": "ge"}, {"budget": 0.69, "threshold": 1.5622968673706055, "operator": "ge"}, {"budget": 0.7, "threshold": 1.416879415512085, "operator": "ge"}, {"budget": 0.71, "threshold": 1.270616054534912, "operator": "ge"}, {"budget": 0.72, "threshold": 1.1343650817871094, "operator": "ge"}, {"budget": 0.73, "threshold": 0.9775638580322266, "operator": "ge"}, {"budget": 0.74, "threshold": 0.8182463049888611, "operator": "ge"}, {"budget": 0.75, "threshold": 0.6729269027709961, "operator": "ge"}, {"budget": 0.76, "threshold": 0.5339034795761108, "operator": "ge"}, {"budget": 0.77, "threshold": 0.3918660283088684, "operator": "ge"}, {"budget": 0.78, "threshold": 0.24066638946533203, "operator": "ge"}, {"budget": 0.79, "threshold": 0.0731738805770874, "operator": "ge"}, {"budget": 0.8, "threshold": -0.10319852828979492, "operator": "ge"}, {"budget": 0.81, "threshold": -0.2828487157821655, "operator": "ge"}, {"budget": 0.82, "threshold": -0.4999234676361084, "operator": "ge"}, {"budget": 0.83, "threshold": -0.70870041847229, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9017311334609985, "operator": "ge"}, {"budget": 0.85, "threshold": -1.1009821891784668, "operator": "ge"}, {"budget": 0.86, "threshold": -1.293165683746338, "operator": "ge"}, {"budget": 0.87, "threshold": -1.5038022994995117, "operator": "ge"}, {"budget": 0.88, "threshold": -1.7441608905792236, "operator": "ge"}, {"budget": 0.89, "threshold": -2.0024917125701904, "operator": "ge"}, {"budget": 0.9, "threshold": -2.285191774368286, "operator": "ge"}, {"budget": 0.91, "threshold": -2.551013946533203, "operator": "ge"}, {"budget": 0.92, "threshold": -2.8249893188476562, "operator": "ge"}, {"budget": 0.93, "threshold": -3.1194841861724854, "operator": "ge"}, {"budget": 0.94, "threshold": -3.4649057388305664, "operator": "ge"}, {"budget": 0.95, "threshold": -3.8191611766815186, "operator": "ge"}, {"budget": 0.96, "threshold": -4.328587532043457, "operator": "ge"}, {"budget": 0.97, "threshold": -4.964831352233887, "operator": "ge"}, {"budget": 0.98, "threshold": -5.7588701248168945, "operator": "ge"}, {"budget": 0.99, "threshold": -7.046417236328125, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.982219411061905, 57.92395811371143, 58.81650151099866, 59.70904490828589, 60.59456040480708, 61.45899219903015, 62.29531239018905, 63.145688382880024, 63.99606437557102, 64.82535666596388, 65.67573265865485, 66.54016445287792, 67.34837304097266, 68.10035842293907, 68.83828800337339, 69.63244078993604, 70.36334246960432, 71.12235575233677, 71.83217372970694, 72.6122707147375, 73.32208869210766, 73.89837655492303, 74.60116663152716, 75.25476140276899, 75.91538407477685, 76.47761613606015, 76.9976807927472, 77.60208025862674, 77.99564270152504, 78.46651205284981, 78.9092698011104, 79.36608335090308, 79.73856209150327, 80.04076182444304, 80.23051514512615, 80.36404525968092, 80.63813338955653, 80.79977510717549, 80.85599831330381, 80.91924942019818, 81.06683533628505, 81.13008644317942, 81.09494693934921, 81.15117014547755, 81.18630964930776, 81.15117014547755, 81.15117014547755, 81.06683533628505, 81.08089113781713, 81.01061213015672, 81.01764003092275, 81.05277953475296, 81.00358422939068, 80.99655632862464, 80.96844472556047, 80.98250052709255, 80.98250052709255, 81.01061213015672, 80.99655632862464, 81.05277953475296, 81.05277953475296, 81.10197484011525, 81.11603064164734, 81.10900274088128, 81.15117014547755, 81.16522594700962, 81.23550495467003, 81.27064445850024, 81.27767235926629, 81.33389556539461, 81.34092346616065, 81.3479513669267, 81.40417457305502, 81.39011877152294, 81.37606296999085, 81.31983976386252, 81.31983976386252, 81.35497926769273, 81.33389556539461, 81.31983976386252, 81.2144212523719, 81.14414224471149, 81.15117014547755, 81.06683533628505, 81.01061213015672, 80.98250052709255, 80.97547262632652, 80.94033312249631, 80.8700541148359, 80.8208588094736, 80.73652400028111, 80.715440297983, 80.69435659568487, 80.66624499262069, 80.65218919108862, 80.63813338955653, 80.6311054887905, 80.61002178649237, 80.60299388572632, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.988895916789655, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.992761262210978, 7.997751071754868, 8.995712980532716, 9.986646988544521, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.99191791411905, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 132, 266, 393, 520, 646, 769, 888, 1009, 1131, 1249, 1370, 1493, 1610, 1718, 1825, 1939, 2044, 2153, 2256, 2369, 2473, 2559, 2661, 2759, 2857, 2943, 3026, 3118, 3187, 3267, 3344, 3420, 3484, 3545, 3595, 3642, 3701, 3743, 3780, 3819, 3859, 3894, 3920, 3952, 3984, 4009, 4038, 4059, 4080, 4101, 4126, 4146, 4168, 4192, 4210, 4228, 4253, 4274, 4286, 4307, 4327, 4350, 4370, 4388, 4402, 4421, 4439, 4456, 4470, 4490, 4505, 4518, 4534, 4544, 4558, 4565, 4576, 4587, 4595, 4602, 4606, 4609, 4617, 4620, 4621, 4621, 4622, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 3, 4, 6, 7, 8, 9, 11, 13, 16, 20, 22, 27, 31, 37, 46, 52, 65, 78, 92, 103, 114, 132, 155, 183, 203, 222, 251, 281, 300, 326, 357, 381, 408, 438, 467, 500, 519, 550, 574, 589, 618, 643, 665, 681, 706, 723, 737, 750, 770, 786, 804, 823, 831, 848, 856, 868, 881, 893, 907, 919, 927, 939, 955, 970, 981, 987, 998, 1007, 1026, 1039, 1046, 1061, 1070, 1074, 1076, 1084, 1095, 1102, 1114, 1117, 1120, 1124, 1126, 1128, 1129, 1132, 1133, 1133], "exact_score": [56.05453650994448, 56.982219411061905, 57.93098601447747, 58.81650151099866, 59.70904490828589, 60.59456040480708, 61.45899219903015, 62.30234029095508, 63.145688382880024, 63.99606437557102, 64.83238456672991, 65.67573265865485, 66.54016445287792, 67.34837304097266, 68.10035842293907, 68.83828800337339, 69.63244078993604, 70.36334246960432, 71.12235575233677, 71.83217372970694, 72.6122707147375, 73.32208869210766, 73.89837655492303, 74.60116663152716, 75.25476140276899, 75.91538407477685, 76.47761613606015, 76.9976807927472, 77.60208025862674, 77.99564270152504, 78.46651205284981, 78.9092698011104, 79.36608335090308, 79.73856209150327, 80.04076182444304, 80.23051514512615, 80.36404525968092, 80.63813338955653, 80.79977510717549, 80.85599831330381, 80.91924942019818, 81.06683533628505, 81.13008644317942, 81.09494693934921, 81.15117014547755, 81.18630964930776, 81.15117014547755, 81.15117014547755, 81.06683533628505, 81.08089113781713, 81.01061213015672, 81.01764003092275, 81.05277953475296, 81.00358422939068, 80.99655632862464, 80.96844472556047, 80.98250052709255, 80.98250052709255, 81.01061213015672, 80.99655632862464, 81.05277953475296, 81.05277953475296, 81.10197484011525, 81.11603064164734, 81.10900274088128, 81.15117014547755, 81.16522594700962, 81.23550495467003, 81.27064445850024, 81.27767235926629, 81.33389556539461, 81.34092346616065, 81.3479513669267, 81.40417457305502, 81.39011877152294, 81.37606296999085, 81.31983976386252, 81.31983976386252, 81.35497926769273, 81.33389556539461, 81.31983976386252, 81.2144212523719, 81.14414224471149, 81.15117014547755, 81.06683533628505, 81.01061213015672, 80.98250052709255, 80.97547262632652, 80.94033312249631, 80.8700541148359, 80.8208588094736, 80.73652400028111, 80.715440297983, 80.69435659568487, 80.66624499262069, 80.65218919108862, 80.63813338955653, 80.6311054887905, 80.61002178649237, 80.60299388572632, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 77.15241067936468, "threshold_area100": 77.1513458429967, "threshold_area25": 66.5710872162485}, "test": {"score": [26.688029946402747, 27.558630859541168, 28.312962595354907, 29.209086010832884, 30.005955250545902, 30.76028698635964, 31.557156226072657, 32.198054618155005, 32.796415506338086, 33.36641805858832, 34.032838953010234, 34.790006522417265, 35.430904914499614, 36.015086634716276, 36.848821711141994, 37.540765107903475, 38.207186002325386, 39.02674191078468, 39.670476136460316, 40.48152454413975, 41.38899129399087, 42.1461588633979, 42.999744774976605, 43.83631568499561, 45.05005246292148, 46.20707256898166, 47.36409267504183, 48.4672319428296, 49.785894563707004, 51.10172135099113, 52.44874230780139, 53.62561324901455, 54.9159175339591, 56.458611008706015, 57.81130363270284, 59.186682925445936, 60.29832969401356, 61.458185633667014, 62.64356407566004, 63.50282165442531, 64.43581090661601, 65.20148597680289, 66.03805688682188, 66.69880611405723, 67.4474661826844, 68.10537957632647, 68.69806879732297, 69.26239968238663, 69.67075971981964, 70.10464225959221, 70.4789722939058, 70.89300399852536, 71.13121402036128, 71.42046904687633, 71.75509741088393, 72.03300910302583, 72.3562941326603, 72.6171908232425, 72.8554008450784, 73.04256586223521, 73.26092504891812, 73.42540339732865, 73.58421007855259, 73.8252559339818, 73.9897342823923, 74.10883929331027, 74.23928763860137, 74.36122848311261, 74.48884099481043, 74.62212517369481, 74.67033434478066, 74.79511102288518, 74.87167852990386, 74.90570853302329, 74.97376853926212, 75.03048521112781, 75.06735104784052, 75.07585854862036, 75.10988855173979, 75.13541105407936, 75.1439185548592, 75.14959022204577, 75.1240677197062, 75.1269035532995, 75.10988855173979, 75.08153021580695, 75.05884354706066, 75.03048521112781, 74.99078354082182, 74.96809687207555, 74.91421603380314, 74.863171029124, 74.81496185803817, 74.7724243541389, 74.7355585174262, 74.6816776791538, 74.63914017525452, 74.6079460057284, 74.57958766979553, 74.51719933074328, 74.47182599325072], "calls": [0.0, 0.9301534185973968, 1.752545160649973, 2.702549414400363, 3.5958369962850583, 4.435243739897342, 5.314352153815614, 6.054504721662933, 6.76346311998412, 7.463914017525451, 8.232424921305617, 9.151235005529875, 9.945268411649606, 10.6712418115305, 11.71766440745257, 12.588265320590988, 13.413492896236848, 14.47125882653206, 15.304993902957776, 16.45634234183138, 17.749482460369226, 18.804412557071153, 20.001134333437314, 21.203527776990047, 22.893684598587754, 24.632050591271305, 26.28250574256303, 27.94430422822789, 29.932223577120492, 31.840739585401128, 33.87970393897286, 35.79105578084678, 37.83002013441851, 40.23764285511726, 42.497802228965206, 44.82602160905198, 46.822448458724445, 48.878427813855886, 51.14142302129711, 52.73232566712986, 54.74576751836202, 56.254430989989515, 57.902050307687944, 59.283101267617624, 60.723704733006265, 62.19833820151434, 63.55386665910444, 65.0682017979185, 66.20537106882568, 67.39642117800527, 68.38612710206165, 69.50344553781585, 70.20673226895046, 71.09718401724187, 72.0953974420781, 73.09644670050761, 74.13436179564982, 74.9766043728554, 75.719592774296, 76.63840285852027, 77.34452542324817, 78.05348382156934, 78.61497887303973, 79.39483311119304, 80.1803590165329, 80.96021325468622, 81.68902248816039, 82.35260754898903, 83.0984317840229, 83.72798684173213, 84.42560190568017, 85.14306780478121, 85.76695119530386, 86.4475512576922, 87.18486799194623, 87.86546805433457, 88.4156197714318, 88.97427898930891, 89.48756486969344, 90.08308992428324, 90.76368998667158, 91.37623004282108, 92.0993676091087, 92.77146017071718, 93.37265689249354, 93.94833111193034, 94.42758698919548, 94.96072370473301, 95.47684541871084, 95.97595213112895, 96.48073051073362, 96.95998638799875, 97.35416725746533, 97.74267645974534, 98.16237983155148, 98.4743215268128, 98.86283072909282, 99.21447409466012, 99.4980574539886, 99.8156708164365, 100.0], "rescued": [0, 307, 573, 890, 1171, 1437, 1718, 1945, 2156, 2358, 2594, 2861, 3087, 3294, 3591, 3836, 4072, 4366, 4594, 4886, 5214, 5485, 5793, 6094, 6532, 6962, 7381, 7786, 8280, 8764, 9263, 9714, 10198, 10784, 11303, 11831, 12273, 12730, 13207, 13553, 13955, 14258, 14585, 14863, 15156, 15435, 15683, 15937, 16115, 16303, 16463, 16634, 16742, 16876, 17024, 17158, 17299, 17412, 17517, 17621, 17718, 17796, 17866, 17969, 18043, 18109, 18177, 18240, 18309, 18366, 18409, 18467, 18508, 18540, 18578, 18611, 18637, 18664, 18691, 18712, 18728, 18744, 18762, 18779, 18792, 18798, 18802, 18809, 18810, 18812, 18813, 18814, 18815, 18815, 18816, 18817, 18817, 18820, 18820, 18820, 18820], "harmed": [0, 0, 0, 1, 1, 1, 1, 2, 2, 3, 4, 4, 4, 5, 8, 9, 10, 15, 16, 22, 30, 34, 41, 47, 57, 79, 90, 106, 135, 155, 179, 215, 244, 286, 328, 371, 421, 469, 528, 571, 644, 677, 709, 754, 783, 830, 869, 924, 958, 993, 1021, 1046, 1070, 1102, 1132, 1168, 1195, 1216, 1237, 1275, 1295, 1315, 1329, 1347, 1363, 1387, 1409, 1429, 1453, 1463, 1489, 1503, 1517, 1537, 1551, 1564, 1577, 1601, 1616, 1628, 1641, 1655, 1682, 1698, 1717, 1733, 1745, 1762, 1777, 1787, 1807, 1826, 1844, 1859, 1873, 1893, 1908, 1922, 1932, 1954, 1970], "exact_score": [26.688029946402747, 27.626690865779995, 28.548336783597538, 29.478490202194934, 30.374613617672917, 31.270737033150898, 32.14700961347589, 32.97507302271503, 33.83149476788702, 34.66522984431274, 35.479114085585465, 36.27598332529847, 37.08986756657119, 37.88673680628421, 38.672262711624086, 39.443609448997535, 40.14689618013215, 40.861526245639915, 41.570484643961095, 42.27093554150243, 42.996908941383325, 43.68601650455152, 44.40064657005927, 45.1209483027536, 45.78453336358223, 46.47931259393699, 47.18259932507161, 47.83200521793381, 48.50126194594901, 49.181862008337355, 49.82843206760628, 50.517539630774465, 51.1981396931628, 51.86456058758473, 52.52247398122678, 53.14919320534271, 53.75889742789893, 54.396959986388005, 55.02084337691065, 55.60786093072059, 56.2742818251425, 56.963389388310695, 57.49369027025495, 58.10906615999774, 58.70459121458753, 59.277429600431056, 59.83041715112157, 60.391912202591946, 60.947735586875766, 61.50355897115958, 62.01684485154411, 62.55565323426821, 63.1171482857386, 63.61909083174999, 64.12670504494797, 64.56909508550038, 65.0682017979185, 65.57865184470975, 66.09477355868758, 66.56268610157956, 67.0759719819641, 67.55806369282251, 68.00045373337494, 68.4683662762669, 68.8795621472932, 69.22836967926722, 69.57434137764797, 69.9401639111817, 70.32300144627513, 70.72285398292829, 71.07733318208888, 71.39494654453678, 71.72106740776452, 72.01315826787284, 72.30241329438789, 72.6171908232425, 72.92062501772396, 73.16167087315317, 73.40555256217564, 73.69764342228397, 73.96421178005275, 74.12301846127669, 74.2903326432805, 74.47182599325072, 74.63630434166124, 74.77809602132547, 74.88585769787028, 74.95958937129569, 75.03615687831437, 75.07585854862036, 75.1269035532995, 75.15242605563905, 75.1269035532995, 75.11272438533308, 75.07869438221365, 75.02481354394124, 74.96242520488899, 74.85749936193744, 74.75824518617247, 74.62496100728809, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.532276265256854, "threshold_area100": 61.207838244051835, "threshold_area25": 36.03408671979128}, "selected_budget": 73, "selected": {"val": {"score": 81.40417457305502, "calls": 72.99880525686977, "rescued": 4534, "harmed": 927}, "test": {"score": 74.90570853302329, "calls": 86.4475512576922, "rescued": 18540, "harmed": 1537}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 5.513422012329102, "operator": "ge"}, {"budget": 0.02, "threshold": 4.971607208251953, "operator": "ge"}, {"budget": 0.03, "threshold": 4.59298038482666, "operator": "ge"}, {"budget": 0.04, "threshold": 4.310050010681152, "operator": "ge"}, {"budget": 0.05, "threshold": 4.107213020324707, "operator": "ge"}, {"budget": 0.06, "threshold": 3.9080052375793457, "operator": "ge"}, {"budget": 0.07, "threshold": 3.7130820751190186, "operator": "ge"}, {"budget": 0.08, "threshold": 3.555689811706543, "operator": "ge"}, {"budget": 0.09, "threshold": 3.4275989532470703, "operator": "ge"}, {"budget": 0.1, "threshold": 3.276766300201416, "operator": "ge"}, {"budget": 0.11, "threshold": 3.171642780303955, "operator": "ge"}, {"budget": 0.12, "threshold": 3.0351879596710205, "operator": "ge"}, {"budget": 0.13, "threshold": 2.9159369468688965, "operator": "ge"}, {"budget": 0.14, "threshold": 2.807969093322754, "operator": "ge"}, {"budget": 0.15, "threshold": 2.7214736938476562, "operator": "ge"}, {"budget": 0.16, "threshold": 2.6419475078582764, "operator": "ge"}, {"budget": 0.17, "threshold": 2.556790351867676, "operator": "ge"}, {"budget": 0.18, "threshold": 2.4780149459838867, "operator": "ge"}, {"budget": 0.19, "threshold": 2.408257484436035, "operator": "ge"}, {"budget": 0.2, "threshold": 2.346925973892212, "operator": "ge"}, {"budget": 0.21, "threshold": 2.2788853645324707, "operator": "ge"}, {"budget": 0.22, "threshold": 2.217371940612793, "operator": "ge"}, {"budget": 0.23, "threshold": 2.157748222351074, "operator": "ge"}, {"budget": 0.24, "threshold": 2.0954806804656982, "operator": "ge"}, {"budget": 0.25, "threshold": 2.041421890258789, "operator": "ge"}, {"budget": 0.26, "threshold": 1.9938161373138428, "operator": "ge"}, {"budget": 0.27, "threshold": 1.9404834508895874, "operator": "ge"}, {"budget": 0.28, "threshold": 1.8910812139511108, "operator": "ge"}, {"budget": 0.29, "threshold": 1.8501873016357422, "operator": "ge"}, {"budget": 0.3, "threshold": 1.808741569519043, "operator": "ge"}, {"budget": 0.31, "threshold": 1.7616702318191528, "operator": "ge"}, {"budget": 0.32, "threshold": 1.7175052165985107, "operator": "ge"}, {"budget": 0.33, "threshold": 1.6743698120117188, "operator": "ge"}, {"budget": 0.34, "threshold": 1.6340675354003906, "operator": "ge"}, {"budget": 0.35, "threshold": 1.5994853973388672, "operator": "ge"}, {"budget": 0.36, "threshold": 1.5626376867294312, "operator": "ge"}, {"budget": 0.37, "threshold": 1.5229638814926147, "operator": "ge"}, {"budget": 0.38, "threshold": 1.4824328422546387, "operator": "ge"}, {"budget": 0.39, "threshold": 1.449504017829895, "operator": "ge"}, {"budget": 0.4, "threshold": 1.4127181768417358, "operator": "ge"}, {"budget": 0.41, "threshold": 1.3768707513809204, "operator": "ge"}, {"budget": 0.42, "threshold": 1.340232014656067, "operator": "gt"}, {"budget": 0.43, "threshold": 1.3051859140396118, "operator": "ge"}, {"budget": 0.44, "threshold": 1.2695146799087524, "operator": "ge"}, {"budget": 0.45, "threshold": 1.2363641262054443, "operator": "ge"}, {"budget": 0.46, "threshold": 1.203322172164917, "operator": "ge"}, {"budget": 0.47, "threshold": 1.1753050088882446, "operator": "ge"}, {"budget": 0.48, "threshold": 1.146432876586914, "operator": "ge"}, {"budget": 0.49, "threshold": 1.1132113933563232, "operator": "ge"}, {"budget": 0.5, "threshold": 1.0852174758911133, "operator": "ge"}, {"budget": 0.51, "threshold": 1.0559437274932861, "operator": "ge"}, {"budget": 0.52, "threshold": 1.0252716541290283, "operator": "ge"}, {"budget": 0.53, "threshold": 0.9916750192642212, "operator": "ge"}, {"budget": 0.54, "threshold": 0.9637157917022705, "operator": "ge"}, {"budget": 0.55, "threshold": 0.9363530278205872, "operator": "ge"}, {"budget": 0.56, "threshold": 0.9063480496406555, "operator": "ge"}, {"budget": 0.57, "threshold": 0.8776009678840637, "operator": "ge"}, {"budget": 0.58, "threshold": 0.8450373411178589, "operator": "ge"}, {"budget": 0.59, "threshold": 0.8144797086715698, "operator": "ge"}, {"budget": 0.6, "threshold": 0.7863087058067322, "operator": "ge"}, {"budget": 0.61, "threshold": 0.7551042437553406, "operator": "ge"}, {"budget": 0.62, "threshold": 0.7243313789367676, "operator": "ge"}, {"budget": 0.63, "threshold": 0.693578839302063, "operator": "ge"}, {"budget": 0.64, "threshold": 0.6627678871154785, "operator": "ge"}, {"budget": 0.65, "threshold": 0.6251521110534668, "operator": "ge"}, {"budget": 0.66, "threshold": 0.5992445349693298, "operator": "ge"}, {"budget": 0.67, "threshold": 0.5618923902511597, "operator": "ge"}, {"budget": 0.68, "threshold": 0.5294699668884277, "operator": "ge"}, {"budget": 0.69, "threshold": 0.4961005449295044, "operator": "ge"}, {"budget": 0.7, "threshold": 0.463900089263916, "operator": "ge"}, {"budget": 0.71, "threshold": 0.4291664659976959, "operator": "ge"}, {"budget": 0.72, "threshold": 0.38610410690307617, "operator": "ge"}, {"budget": 0.73, "threshold": 0.3433784246444702, "operator": "ge"}, {"budget": 0.74, "threshold": 0.29851841926574707, "operator": "ge"}, {"budget": 0.75, "threshold": 0.25917690992355347, "operator": "ge"}, {"budget": 0.76, "threshold": 0.20911288261413574, "operator": "ge"}, {"budget": 0.77, "threshold": 0.15597248077392578, "operator": "ge"}, {"budget": 0.78, "threshold": 0.0964721292257309, "operator": "ge"}, {"budget": 0.79, "threshold": 0.036075249314308167, "operator": "ge"}, {"budget": 0.8, "threshold": -0.060429930686950684, "operator": "ge"}, {"budget": 0.81, "threshold": -0.17710715532302856, "operator": "ge"}, {"budget": 0.82, "threshold": -0.30438125133514404, "operator": "ge"}, {"budget": 0.83, "threshold": -0.45530450344085693, "operator": "ge"}, {"budget": 0.84, "threshold": -0.6032681465148926, "operator": "ge"}, {"budget": 0.85, "threshold": -0.7488055229187012, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9009974002838135, "operator": "ge"}, {"budget": 0.87, "threshold": -1.0889995098114014, "operator": "ge"}, {"budget": 0.88, "threshold": -1.2786434888839722, "operator": "ge"}, {"budget": 0.89, "threshold": -1.4963078498840332, "operator": "ge"}, {"budget": 0.9, "threshold": -1.7262187004089355, "operator": "ge"}, {"budget": 0.91, "threshold": -1.9492652416229248, "operator": "ge"}, {"budget": 0.92, "threshold": -2.202242851257324, "operator": "ge"}, {"budget": 0.93, "threshold": -2.498277187347412, "operator": "ge"}, {"budget": 0.94, "threshold": -2.8053910732269287, "operator": "ge"}, {"budget": 0.95, "threshold": -3.1487367153167725, "operator": "ge"}, {"budget": 0.96, "threshold": -3.578270435333252, "operator": "ge"}, {"budget": 0.97, "threshold": -4.130502223968506, "operator": "ge"}, {"budget": 0.98, "threshold": -4.884133815765381, "operator": "ge"}, {"budget": 0.99, "threshold": -6.15078592300415, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.01033101412608, 57.91693021294538, 58.73919460257221, 59.48415208377257, 60.348583877995644, 61.16382036685641, 61.87363834422658, 62.611567924660896, 63.15271628364607, 63.7571157495256, 64.3193478108089, 64.90266357439033, 65.4438119333755, 65.91468128470026, 66.28013212453439, 66.72991777356103, 67.17970342258766, 67.53109846088972, 67.93871670532012, 68.4658092627732, 68.89451120950172, 69.30212945393211, 69.61135708763791, 70.02600323283434, 70.40550987420058, 70.82718392016304, 71.24183006535947, 71.60025300442757, 71.96570384426171, 72.2538477756694, 72.5771312109073, 72.8160798369527, 73.1745027760208, 73.54698151662097, 73.8070138449645, 74.10218567713824, 74.34113430318364, 74.69955724225173, 74.9314779675311, 75.16339869281046, 75.40234731885586, 75.65535174643334, 75.80996556328624, 75.99971888396935, 76.21055590695059, 76.52681144142244, 76.6743973575093, 76.87117857895846, 76.98362499121512, 77.28582472415488, 77.48963384637008, 77.66533136552111, 77.86211258697027, 78.10808911378172, 78.29784243446483, 78.52273525897814, 78.74060018272542, 78.96549300723873, 79.1482184271558, 79.33797174783892, 79.51366926698995, 79.66828308384285, 79.82289690069577, 79.99156651908075, 80.11104083210346, 80.26565464895636, 80.40621266427718, 80.546770679598, 80.60299388572632, 80.7857193056434, 80.88410991636798, 80.94736102326235, 81.13008644317942, 81.18630964930776, 81.22847705390399, 81.29172816079837, 81.31983976386252, 81.36903506922482, 81.34092346616065, 81.30578396233045, 81.24253285543608, 81.1792817485417, 81.15819804624358, 81.14414224471149, 81.04575163398692, 80.98952842785859, 80.94033312249631, 80.9051936186661, 80.88410991636798, 80.85599831330381, 80.8208588094736, 80.75760770257924, 80.715440297983, 80.68030079415279, 80.66624499262069, 80.65218919108862, 80.63813338955653, 80.61704968725842, 80.61002178649237, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.991707077096066, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 137, 270, 393, 511, 637, 758, 868, 980, 1071, 1169, 1260, 1354, 1445, 1529, 1596, 1676, 1753, 1818, 1891, 1976, 2053, 2123, 2185, 2258, 2326, 2397, 2467, 2536, 2603, 2660, 2720, 2769, 2835, 2896, 2954, 3019, 3069, 3128, 3176, 3228, 3279, 3327, 3368, 3412, 3458, 3511, 3550, 3595, 3633, 3682, 3722, 3759, 3798, 3841, 3882, 3926, 3971, 4020, 4061, 4099, 4139, 4169, 4207, 4242, 4272, 4308, 4335, 4367, 4391, 4422, 4449, 4470, 4504, 4524, 4543, 4562, 4571, 4587, 4596, 4601, 4607, 4610, 4614, 4618, 4620, 4621, 4621, 4621, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 1, 5, 11, 23, 26, 31, 40, 47, 61, 73, 84, 95, 109, 126, 141, 157, 170, 185, 200, 210, 226, 238, 256, 270, 284, 295, 306, 324, 339, 355, 369, 384, 399, 407, 428, 451, 467, 475, 490, 509, 526, 538, 557, 574, 590, 598, 616, 633, 655, 661, 672, 684, 695, 703, 717, 729, 743, 760, 775, 786, 801, 809, 825, 836, 849, 863, 870, 882, 898, 903, 916, 928, 936, 948, 961, 971, 976, 985, 998, 1008, 1023, 1035, 1042, 1048, 1064, 1073, 1080, 1085, 1092, 1097, 1102, 1111, 1117, 1122, 1124, 1126, 1128, 1131, 1132, 1133], "exact_score": [56.05453650994448, 57.01033101412608, 57.91693021294538, 58.73919460257221, 59.48415208377257, 60.348583877995644, 61.16382036685641, 61.87363834422658, 62.611567924660896, 63.15271628364607, 63.7571157495256, 64.3193478108089, 64.90266357439033, 65.4438119333755, 65.91468128470026, 66.28013212453439, 66.72991777356103, 67.17970342258766, 67.53109846088972, 67.93871670532012, 68.4658092627732, 68.89451120950172, 69.30212945393211, 69.61135708763791, 70.02600323283434, 70.40550987420058, 70.82718392016304, 71.24183006535947, 71.60025300442757, 71.96570384426171, 72.2538477756694, 72.5771312109073, 72.8160798369527, 73.1745027760208, 73.54698151662097, 73.8070138449645, 74.10218567713824, 74.34113430318364, 74.69955724225173, 74.9314779675311, 75.16339869281046, 75.40234731885586, 75.65535174643334, 75.80996556328624, 75.99971888396935, 76.21055590695059, 76.52681144142244, 76.6743973575093, 76.87117857895846, 76.98362499121512, 77.28582472415488, 77.48963384637008, 77.66533136552111, 77.86211258697027, 78.10808911378172, 78.29784243446483, 78.52273525897814, 78.74060018272542, 78.96549300723873, 79.1482184271558, 79.33797174783892, 79.51366926698995, 79.66828308384285, 79.82289690069577, 79.99156651908075, 80.11104083210346, 80.26565464895636, 80.40621266427718, 80.546770679598, 80.60299388572632, 80.7857193056434, 80.88410991636798, 80.94736102326235, 81.13008644317942, 81.18630964930776, 81.22847705390399, 81.29172816079837, 81.31983976386252, 81.36903506922482, 81.34092346616065, 81.30578396233045, 81.24253285543608, 81.1792817485417, 81.15819804624358, 81.14414224471149, 81.04575163398692, 80.98952842785859, 80.94033312249631, 80.9051936186661, 80.88410991636798, 80.85599831330381, 80.8208588094736, 80.75760770257924, 80.715440297983, 80.68030079415279, 80.66624499262069, 80.65218919108862, 80.63813338955653, 80.61704968725842, 80.61002178649237, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 74.82242449675398, "threshold_area100": 74.82159673905404, "threshold_area25": 64.41633284138028}, "test": {"score": [26.688029946402747, 28.514306780478123, 30.15341859739671, 31.89462042367354, 33.42313473045402, 34.62552817400675, 35.91016079176474, 37.347928423560106, 38.643904375691235, 39.6931628052066, 40.95227292062502, 41.95899384624111, 43.201088960099824, 44.332586563820435, 45.34781499021637, 46.27229674162721, 47.123046819612625, 48.00499106712418, 48.92947281853501, 49.6752970535689, 50.37007628392366, 51.11022885177098, 51.70858973995407, 52.42321980546182, 53.117999035816574, 53.6908374216601, 54.201287468451355, 54.84502169412699, 55.39233757763095, 55.897115957235634, 56.37637183450075, 56.9435385531577, 57.41145109604968, 58.03249865297905, 58.511754530244175, 58.93712956923688, 59.45892295040127, 59.95235799563281, 60.56489805178232, 60.99594475796161, 61.46952896804015, 61.9629640132717, 62.40818988741741, 62.9101324334288, 63.34968664038794, 63.79491251453364, 64.21461588633979, 64.60028925502652, 64.94626095340725, 65.3461134900604, 65.68357768766128, 66.03238521963532, 66.43507358988174, 66.84059779372147, 67.18089782491563, 67.48716785299038, 67.87000538808383, 68.18761875053173, 68.5619487848453, 68.8795621472932, 69.1773246745881, 69.58001304483453, 69.97419391430111, 70.34852394861471, 70.77673482120069, 71.2588265320591, 71.55091739216743, 71.964949096787, 72.30808496157447, 72.64838499276863, 72.94331168647024, 73.31480588719054, 73.71749425743698, 74.02092845191845, 74.28466097609392, 74.55973683464254, 74.78660352210534, 74.95958937129569, 75.08153021580695, 75.13824688767264, 75.14108272126592, 75.15526188923234, 75.1240677197062, 75.12123188611292, 75.12123188611292, 75.11556021892636, 75.08720188299351, 75.05600771346738, 75.01063437597482, 74.9766043728554, 74.95675353770241, 74.90570853302329, 74.8660068627173, 74.7922751892919, 74.74690185179934, 74.69869268071349, 74.64764767603438, 74.61078183932167, 74.58242350338881, 74.52287099792984, 74.47182599325072], "calls": [0.0, 1.9085160082806343, 3.700762839236594, 5.589428012364235, 7.373167342540339, 8.768397470436435, 10.296911777216913, 12.018262768340755, 13.597822079800356, 14.916484700677765, 16.55843235118963, 17.862915804100616, 19.59277429600431, 21.163826106684063, 22.63562374159884, 23.83518135155829, 25.0602614638573, 26.344894081615287, 27.67773587045912, 28.85460681167229, 29.963417746646627, 31.13461702067323, 32.18671128378187, 33.23313387970394, 34.48090066074923, 35.61523409806313, 36.5312083486941, 37.60031761336245, 38.578680203045685, 39.45495278337067, 40.40212120352778, 41.48540963616255, 42.46377222584579, 43.54706065848056, 44.4516915747384, 45.27691915038425, 46.21558006976151, 47.191106825851456, 48.32827609675864, 49.24141451379633, 50.18574710036015, 51.172617190823246, 52.133964778946776, 53.17187987408899, 54.249496639537185, 55.171142557354734, 56.175027649377526, 57.05130022970252, 57.893542806908094, 58.90877123330402, 59.77653631284916, 60.655644726767434, 61.619828148484245, 62.615205739727195, 63.616254998156705, 64.51521424722797, 65.49924850409778, 66.50880526330715, 67.5382128576695, 68.4683662762669, 69.43538553157701, 70.44777812437965, 71.50554405467487, 72.51226498029095, 73.64092675041829, 74.97376853926212, 75.89541445707965, 77.11765873578538, 78.18393216686044, 79.20766809403624, 80.2370756883986, 81.28633411791397, 82.68156424581005, 83.85559935342994, 84.98993279074384, 85.99948954995321, 87.13382298726711, 88.18875308396903, 89.19830984317841, 90.02353741882426, 90.80622749057086, 91.47832005217934, 92.13056177863483, 92.87071434648215, 93.44638856591895, 93.97952528145649, 94.4247511556022, 95.03445537815841, 95.54490542494966, 95.9844596319088, 96.44670050761421, 96.92028471769277, 97.46192893401016, 97.90431897456257, 98.26446984090974, 98.61044153929048, 98.94506990329808, 99.23716076340641, 99.52357995632816, 99.7958199812835, 100.0], "rescued": [0, 654, 1249, 1877, 2444, 2882, 3357, 3887, 4363, 4759, 5235, 5610, 6093, 6529, 6918, 7262, 7588, 7922, 8276, 8567, 8841, 9129, 9368, 9636, 9909, 10155, 10360, 10617, 10834, 11028, 11228, 11454, 11651, 11896, 12083, 12251, 12453, 12648, 12888, 13055, 13237, 13433, 13612, 13812, 14003, 14171, 14343, 14498, 14642, 14810, 14943, 15085, 15245, 15402, 15548, 15676, 15825, 15955, 16102, 16231, 16357, 16515, 16666, 16811, 16977, 17170, 17284, 17445, 17583, 17712, 17834, 17978, 18136, 18256, 18359, 18465, 18556, 18630, 18682, 18712, 18728, 18744, 18758, 18776, 18788, 18796, 18803, 18807, 18808, 18811, 18813, 18814, 18815, 18815, 18815, 18815, 18817, 18819, 18820, 18820, 18820], "harmed": [0, 10, 27, 41, 69, 83, 105, 128, 147, 173, 205, 225, 270, 307, 338, 356, 382, 405, 433, 461, 490, 517, 545, 561, 589, 633, 658, 688, 712, 728, 759, 785, 817, 843, 861, 879, 897, 918, 942, 957, 972, 994, 1016, 1039, 1075, 1086, 1110, 1129, 1151, 1178, 1192, 1211, 1229, 1243, 1269, 1289, 1303, 1321, 1336, 1353, 1374, 1390, 1402, 1415, 1430, 1453, 1464, 1479, 1496, 1505, 1523, 1536, 1552, 1565, 1575, 1584, 1595, 1608, 1617, 1627, 1642, 1653, 1678, 1697, 1709, 1719, 1736, 1751, 1768, 1783, 1792, 1811, 1826, 1852, 1868, 1885, 1905, 1920, 1931, 1952, 1970], "exact_score": [26.688029946402747, 27.65788503530613, 28.596545954683382, 29.523863539687493, 30.43983779031847, 31.352976207356154, 32.260442957207275, 33.11686470237927, 33.96761478036469, 34.81552902475683, 35.654935768369114, 36.485835011201544, 37.32807758840712, 38.141961829679836, 38.930323568613, 39.75555114425886, 40.53256954881888, 41.281229617446044, 42.05257635481951, 42.78138558829368, 43.48750815302158, 44.22482488727562, 44.91960411763038, 45.63423418313813, 46.394237586138445, 47.077673482120076, 47.792303547627824, 48.45305277486317, 49.13081700365823, 49.7688795621473, 50.398434619856516, 51.02515384397244, 51.6263505657488, 52.253069789864725, 52.83725151008139, 53.35337322405922, 53.909196608343024, 54.476363326999966, 55.06054504721662, 55.630547599466865, 56.17502764937754, 56.72234353288149, 57.21010691092647, 57.72055695771772, 58.27354450840825, 58.758472052859936, 59.36250460822959, 59.83325298471486, 60.38907636899866, 60.902362249383216, 61.38161812664833, 61.8892323398463, 62.34296571477186, 62.8562515951564, 63.23341746306327, 63.71834500751496, 64.15506338088082, 64.57760258628025, 64.97178345574682, 65.37163599239997, 65.76298102827327, 66.15716189773984, 66.57119360235941, 67.0022403085387, 67.30851033661345, 67.6714970365539, 67.99478206618836, 68.3776196012818, 68.72926296684912, 69.03269716133057, 69.40419136205087, 69.7898647307376, 70.13867226271162, 70.56688313529762, 70.91852650086493, 71.26733403283896, 71.59629072965998, 71.91390409210787, 72.24002495533563, 72.58883248730965, 72.8554008450784, 73.209880044239, 73.5246575730936, 73.8054050988288, 74.05495845503786, 74.2874968096872, 74.55973683464254, 74.76958852054561, 74.91138020020985, 75.05884354706066, 75.12973938689278, 75.13541105407936, 75.14108272126592, 75.12973938689278, 75.11556021892636, 75.05600771346738, 74.9766043728554, 74.89152936505685, 74.78093185491875, 74.6448118424411, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.186926410827546, "threshold_area100": 61.95320874571081, "threshold_area25": 42.481921560842814}, "selected_budget": 78, "selected": {"val": {"score": 81.36903506922482, "calls": 77.99564270152506, "rescued": 4587, "harmed": 985}, "test": {"score": 75.08153021580695, "calls": 89.19830984317841, "rescued": 18682, "harmed": 1617}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 81.37606296999085, "calls": 75.99971888396936}, "test": {"score": 75.14108272126592, "calls": 87.72651220826361}}, "old_normalized_interior": {"val": {"score": 81.37606296999085, "calls": 75.99971888396936}, "test": {"score": 75.14108272126592, "calls": 87.72651220826361}}, "old_normalized_endpoints": {"val": {"score": 81.37606296999085, "calls": 75.99971888396936}, "test": {"score": 75.14108272126592, "calls": 87.72651220826361}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 57.0314147164242, 57.94504181600956, 58.823529411764696, 59.65282170215757, 60.503197694848545, 61.3324899852414, 62.11258697027198, 62.864572352238376, 63.64466933726896, 64.31231991004287, 64.89563567362428, 65.4789514372057, 66.02712769695691, 66.60341555977229, 67.07428491109705, 67.53812636165577, 67.99493991144845, 68.52203246890154, 68.943706514864, 69.38646426312461, 69.8151662098531, 70.24386815658163, 70.69365380560826, 71.11532785157073, 71.57214140136341, 71.95867594349568, 72.27493147796753, 72.70363342469604, 73.09719586759435, 73.4837304097266, 73.72970693653805, 74.0459624710099, 74.3341064024176, 74.71361304378381, 74.92445006676505, 75.13528708974629, 75.38829151732376, 75.62724014336916, 75.94349567784103, 76.16838850235435, 76.30894651767517, 76.60411834984889, 76.88523438049054, 77.06093189964157, 77.33502002951718, 77.63019186169092, 77.91130789233256, 78.09403331224962, 78.22756342680441, 78.49462365591397, 78.71248857966124, 78.86007449574811, 79.07091151872935, 79.2676927401785, 79.49961346545787, 79.66828308384285, 79.82289690069577, 80.00562232061283, 80.1532082366997, 80.34296155738281, 80.45540796963945, 80.60299388572632, 80.64516129032258, 80.79977510717549, 80.89816571790006, 80.91924942019818, 80.96141682479443, 81.01061213015672, 81.03872373322089, 81.11603064164734, 81.15117014547755, 81.22144915313795, 81.24253285543608, 81.27767235926629, 81.31281186309648, 81.37606296999085, 81.3479513669267, 81.33389556539461, 81.31281186309648, 81.2987560615644, 81.2987560615644, 81.2987560615644, 81.31281186309648, 81.30578396233045, 81.30578396233045, 81.2987560615644, 81.2987560615644, 81.28470026003231, 81.29172816079837, 81.29172816079837, 81.2987560615644, 81.25658865696816, 81.23550495467003, 81.20739335160587, 81.19333755007378, 81.17225384777566, 81.08089113781713, 80.96844472556047, 80.84194251177172, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.987771452667088, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 57.0314147164242, 57.94504181600956, 58.823529411764696, 59.65282170215757, 60.503197694848545, 61.33951788600744, 62.11258697027198, 62.864572352238376, 63.64466933726896, 64.31231991004287, 64.89563567362428, 65.4789514372057, 66.02712769695691, 66.60341555977229, 67.07428491109705, 67.53812636165577, 67.99493991144845, 68.52203246890154, 68.943706514864, 69.38646426312461, 69.8151662098531, 70.24386815658163, 70.69365380560826, 71.11532785157073, 71.57214140136341, 71.95867594349568, 72.27493147796753, 72.70363342469604, 73.09719586759435, 73.4837304097266, 73.72970693653805, 74.0459624710099, 74.3341064024176, 74.71361304378381, 74.92445006676505, 75.13528708974629, 75.38829151732376, 75.62724014336916, 75.94349567784103, 76.16838850235435, 76.30894651767517, 76.60411834984889, 76.88523438049054, 77.06093189964157, 77.33502002951718, 77.63019186169092, 77.91130789233256, 78.09403331224962, 78.22756342680441, 78.49462365591397, 78.71248857966124, 78.86007449574811, 79.07091151872935, 79.2676927401785, 79.49961346545787, 79.66828308384285, 79.82289690069577, 80.00562232061283, 80.1532082366997, 80.34296155738281, 80.45540796963945, 80.60299388572632, 80.64516129032258, 80.79977510717549, 80.89816571790006, 80.91924942019818, 80.96141682479443, 81.01061213015672, 81.03872373322089, 81.11603064164734, 81.15117014547755, 81.22144915313795, 81.24253285543608, 81.27767235926629, 81.31281186309648, 81.37606296999085, 81.3479513669267, 81.33389556539461, 81.31281186309648, 81.2987560615644, 81.2987560615644, 81.2987560615644, 81.31281186309648, 81.30578396233045, 81.30578396233045, 81.2987560615644, 81.2987560615644, 81.28470026003231, 81.29172816079837, 81.29172816079837, 81.2987560615644, 81.25658865696816, 81.23550495467003, 81.20739335160587, 81.19333755007378, 81.17225384777566, 81.08089113781713, 80.96844472556047, 80.84194251177172, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 65.01792114695341, "exact_area100": 75.50596057507207}, "test": {"score": [26.688029946402747, 28.298783427388486, 29.781924396676406, 31.551484558886088, 32.997759691461304, 34.228511470946884, 35.589711595723564, 36.9395683861271, 38.21852933669852, 39.70167030598645, 40.838839576893626, 42.08377052434563, 43.45064231630889, 44.54527408331679, 45.60871168079858, 46.54453676658254, 47.64200436718373, 48.65439695998639, 49.52499787312481, 50.276493775345266, 51.07903468224485, 51.887247256331, 52.64725065933131, 53.302328219380094, 54.070839123160255, 54.67203584493663, 55.36114340810481, 56.01338513456031, 56.68547769616878, 57.164733573433914, 57.84533363582225, 58.389813685732925, 59.02220457703542, 59.60355046365879, 60.16788134872246, 60.81445140799138, 61.45534980007373, 62.053710688256814, 62.57266823582792, 63.049088279499756, 63.616254998156705, 64.06998837308227, 64.54073674956754, 64.91506678388113, 65.43118849785895, 65.91044437512407, 66.32164024615037, 66.76403028670278, 67.21776366162834, 67.64597453421433, 68.05149873805405, 68.64418795905056, 69.11493633553583, 69.54031137452854, 70.06777642287952, 70.44494229078637, 70.95822817117093, 71.4261407140629, 71.76644074525707, 72.17196494909679, 72.61151915605592, 73.05674503020163, 73.3601792246831, 73.56435924339961, 73.87630093866092, 74.1031676261237, 74.29884014406034, 74.46331849247086, 74.67033434478066, 74.8121260244449, 74.92839520176956, 74.99078354082182, 75.05317187987409, 75.10421688455322, 75.1240677197062, 75.13257522048606, 75.14108272126592, 75.15242605563905, 75.15526188923234, 75.16093355641891, 75.16093355641891, 75.15809772282563, 75.15526188923234, 75.1439185548592, 75.1439185548592, 75.14108272126592, 75.13257522048606, 75.12973938689278, 75.11556021892636, 75.11839605251964, 75.11556021892636, 75.10988855173979, 75.11272438533308, 75.10988855173979, 75.1070527181465, 75.08153021580695, 75.03048521112781, 75.02197771034795, 74.9964552080084, 74.86033519553072, 74.47182599325072], "calls": [0.0, 1.6589626520715766, 3.272551966650597, 5.1697246405580914, 6.820179791849815, 8.238096588492187, 9.826163400731645, 11.400051045004679, 12.90587868303888, 14.700961347588123, 16.181266483282762, 17.851572469727476, 19.61546096475059, 21.163826106684063, 22.542041233020445, 23.908913024983693, 25.42041233020446, 26.81564245810056, 27.98684173212716, 29.138190171000765, 30.31222527862065, 31.59118622919207, 32.87014717976349, 33.95059977880498, 35.17567989110399, 36.16254998156708, 37.29688341888098, 38.411366021041886, 39.46346028415053, 40.43898704024048, 41.544962141621525, 42.48929472818535, 43.51303065536115, 44.55094575050336, 45.603040013612, 46.63811927516093, 47.83484105152709, 48.90395031619544, 50.018432918356346, 51.03649717834558, 52.09993477582735, 53.13784987096957, 54.04815245441398, 55.05487338003006, 56.1835351501574, 57.363241924963845, 58.429515356038905, 59.48444545274083, 60.57340555256218, 61.577290644584984, 62.56983240223464, 63.95088336216431, 65.06536596432522, 66.19119190085925, 67.34254033973286, 68.39747043643479, 69.71046139012562, 70.86180982899923, 71.7976349147832, 72.82704250914557, 74.00674928395202, 75.26018773218388, 76.35765533278507, 77.13750957093838, 78.20661883560672, 79.09423475030485, 79.95065649547685, 80.82692907580183, 81.83365000141791, 82.63902674191078, 83.41037347928423, 84.18172021665768, 85.00694779230355, 85.71307035703146, 86.33695374755409, 87.02889714431558, 87.72651220826361, 88.46382894251765, 89.24651901426425, 90.1114482602161, 90.66159997731333, 91.10966168505232, 91.60026089669057, 92.09369594192212, 92.41130930437002, 92.90190851600828, 93.39817939483311, 93.72713609165415, 94.18370530017299, 94.59490117119927, 94.97490287269943, 95.4655020843377, 95.88520545614384, 96.30207299435669, 96.66505969429714, 97.08476306610329, 97.47894393556986, 97.92984147690214, 98.43178402291353, 98.98760740719734, 100.0], "exact_score": [26.688029946402747, 27.66922836967927, 28.616396789836372, 29.53520687406063, 30.459688625471458, 31.384170376882288, 32.28312962595355, 33.15373053909197, 34.01298811785725, 34.8779173638091, 35.74284660976094, 36.619119190085925, 37.447182599325075, 38.29226101012393, 39.108981084989935, 39.939880327822365, 40.69137623004282, 41.44287213226328, 42.20571136885688, 42.98556560701018, 43.75124067719707, 44.431840739585404, 45.197515809772284, 45.92348920965318, 46.621104273601226, 47.33857017270226, 48.07305107336302, 48.8018603068372, 49.53350537390466, 50.20559793551315, 50.849332161188784, 51.50157388764427, 52.12262144457364, 52.73516150072314, 53.33635822249951, 53.963077446615436, 54.58979667073136, 55.17397839094802, 55.75816011116468, 56.37920766809403, 56.969061055497264, 57.51070527181466, 58.120409494370875, 58.684740379434544, 59.30295210277061, 59.850267986274574, 60.43161387289794, 60.99310892436832, 61.560275643025264, 62.09057652496952, 62.56132490145478, 63.03490911153333, 63.568045827070875, 63.999092533250156, 64.51521424722797, 64.9008876159147, 65.35462099084026, 65.77716019623969, 66.15999773133312, 66.55134276720642, 66.99656864135213, 67.38224201003885, 67.79627371465843, 68.22164875365114, 68.66687462779683, 69.08090633241642, 69.49210220344271, 69.92314890962199, 70.27479227518928, 70.65479397668946, 71.0744973484956, 71.48569321952188, 71.82882908430935, 72.22868162096248, 72.60868332246264, 72.95749085443666, 73.2637608825114, 73.52182173950033, 73.83376343476165, 74.086152624564, 74.30451181124693, 74.50018432918357, 74.70720018149335, 74.87451436349714, 74.97376853926212, 75.05317187987409, 75.1240677197062, 75.13257522048606, 75.14959022204577, 75.15809772282563, 75.15809772282563, 75.16093355641891, 75.14675438845248, 75.14108272126592, 75.11839605251964, 75.11556021892636, 75.10988855173979, 75.08720188299351, 75.03048521112781, 74.85466352834416, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 42.674191078467516, "exact_area100": 58.90618240827161}}, "old_normalized_endpoint_budget": 76}, {"seed": 2028, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 0.0, "validation_area100": 0.7525505923284571, "training_seconds": 66.36357700801454, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9854216575622559, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9784228801727295, "operator": "ge"}, {"budget": 0.03, "threshold": 0.972293496131897, "operator": "gt"}, {"budget": 0.04, "threshold": 0.9666717052459717, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9596355557441711, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9525913000106812, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9453409910202026, "operator": "ge"}, {"budget": 0.08, "threshold": 0.937260627746582, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9303016066551208, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9241647124290466, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9172589778900146, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9104824662208557, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9028980135917664, "operator": "ge"}, {"budget": 0.14, "threshold": 0.8964369893074036, "operator": "gt"}, {"budget": 0.15, "threshold": 0.8887524604797363, "operator": "ge"}, {"budget": 0.16, "threshold": 0.881582498550415, "operator": "ge"}, {"budget": 0.17, "threshold": 0.873101532459259, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8647458553314209, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8572472333908081, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8505748510360718, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8427769541740417, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8349409699440002, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8279629349708557, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8202810287475586, "operator": "ge"}, {"budget": 0.25, "threshold": 0.8128625750541687, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8054322004318237, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7981043457984924, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7898609042167664, "operator": "ge"}, {"budget": 0.29, "threshold": 0.7827404737472534, "operator": "ge"}, {"budget": 0.3, "threshold": 0.773883581161499, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7651587128639221, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7564888000488281, "operator": "ge"}, {"budget": 0.33, "threshold": 0.7480437755584717, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7401528358459473, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7310640215873718, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7223069071769714, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7141271233558655, "operator": "ge"}, {"budget": 0.38, "threshold": 0.704535186290741, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6924020051956177, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6817856431007385, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6707493662834167, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6596584320068359, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6496873497962952, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6404384970664978, "operator": "ge"}, {"budget": 0.45, "threshold": 0.6287065744400024, "operator": "ge"}, {"budget": 0.46, "threshold": 0.6173385381698608, "operator": "ge"}, {"budget": 0.47, "threshold": 0.6064399480819702, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5961914658546448, "operator": "ge"}, {"budget": 0.49, "threshold": 0.5834250450134277, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5702475309371948, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5543240904808044, "operator": "ge"}, {"budget": 0.52, "threshold": 0.5409159064292908, "operator": "ge"}, {"budget": 0.53, "threshold": 0.5273998975753784, "operator": "ge"}, {"budget": 0.54, "threshold": 0.5112614035606384, "operator": "ge"}, {"budget": 0.55, "threshold": 0.4960525631904602, "operator": "ge"}, {"budget": 0.56, "threshold": 0.47963404655456543, "operator": "ge"}, {"budget": 0.57, "threshold": 0.4602435529232025, "operator": "ge"}, {"budget": 0.58, "threshold": 0.4419208765029907, "operator": "ge"}, {"budget": 0.59, "threshold": 0.42240381240844727, "operator": "ge"}, {"budget": 0.6, "threshold": 0.402505099773407, "operator": "ge"}, {"budget": 0.61, "threshold": 0.38025006651878357, "operator": "ge"}, {"budget": 0.62, "threshold": 0.35781174898147583, "operator": "ge"}, {"budget": 0.63, "threshold": 0.33575165271759033, "operator": "ge"}, {"budget": 0.64, "threshold": 0.3126713037490845, "operator": "ge"}, {"budget": 0.65, "threshold": 0.28448978066444397, "operator": "ge"}, {"budget": 0.66, "threshold": 0.2569108307361603, "operator": "ge"}, {"budget": 0.67, "threshold": 0.2264796793460846, "operator": "ge"}, {"budget": 0.68, "threshold": 0.1984940469264984, "operator": "ge"}, {"budget": 0.69, "threshold": 0.16497844457626343, "operator": "ge"}, {"budget": 0.7, "threshold": 0.13659578561782837, "operator": "ge"}, {"budget": 0.71, "threshold": 0.1082397997379303, "operator": "ge"}, {"budget": 0.72, "threshold": 0.08217708766460419, "operator": "ge"}, {"budget": 0.73, "threshold": 0.062448568642139435, "operator": "ge"}, {"budget": 0.74, "threshold": 0.043413881212472916, "operator": "ge"}, {"budget": 0.75, "threshold": 0.030792146921157837, "operator": "ge"}, {"budget": 0.76, "threshold": 0.019726920872926712, "operator": "ge"}, {"budget": 0.77, "threshold": 0.010284006595611572, "operator": "ge"}, {"budget": 0.78, "threshold": 0.004654444754123688, "operator": "ge"}, {"budget": 0.79, "threshold": 0.0003034239634871483, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0015933213289827108, "operator": "ge"}, {"budget": 0.81, "threshold": -0.002620724029839039, "operator": "ge"}, {"budget": 0.82, "threshold": -0.0035120416432619095, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0044587417505681515, "operator": "ge"}, {"budget": 0.84, "threshold": -0.005336405243724585, "operator": "ge"}, {"budget": 0.85, "threshold": -0.006285250186920166, "operator": "ge"}, {"budget": 0.86, "threshold": -0.007330134976655245, "operator": "ge"}, {"budget": 0.87, "threshold": -0.008415879681706429, "operator": "ge"}, {"budget": 0.88, "threshold": -0.009683653712272644, "operator": "ge"}, {"budget": 0.89, "threshold": -0.011170215904712677, "operator": "ge"}, {"budget": 0.9, "threshold": -0.013171156868338585, "operator": "ge"}, {"budget": 0.91, "threshold": -0.01579274609684944, "operator": "ge"}, {"budget": 0.92, "threshold": -0.018646853044629097, "operator": "ge"}, {"budget": 0.93, "threshold": -0.023024145513772964, "operator": "ge"}, {"budget": 0.94, "threshold": -0.02863401174545288, "operator": "ge"}, {"budget": 0.95, "threshold": -0.03727847337722778, "operator": "ge"}, {"budget": 0.96, "threshold": -0.048757173120975494, "operator": "ge"}, {"budget": 0.97, "threshold": -0.07016337662935257, "operator": "ge"}, {"budget": 0.98, "threshold": -0.1170363798737526, "operator": "ge"}, {"budget": 0.99, "threshold": -0.21500259637832642, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.024386815658154, 57.89584651064727, 58.760278304870326, 59.65282170215757, 60.482113992550424, 61.22707147375078, 61.8947220465247, 62.597512123128816, 63.208939489774394, 63.82036685641999, 64.3685431161712, 64.8815798720922, 65.3243376203528, 65.85845807857193, 66.30121582683252, 66.77208517815728, 67.10942441492726, 67.58732166701805, 68.08630262140699, 68.52203246890154, 68.98587391946025, 69.33024105699627, 69.71677559912854, 70.08222643896268, 70.45470517956286, 70.8693513247593, 71.30508117225385, 71.70567151591818, 72.14140136341274, 72.54199170707709, 72.92852624920936, 73.28694918827745, 73.55400941738702, 73.82809754726263, 74.15840888326657, 74.58008292922904, 74.89633846370089, 75.33909621196149, 75.60615644107105, 75.89430037247872, 76.21055590695059, 76.45653243376202, 76.77981586899992, 77.02579239581137, 77.34204793028321, 77.72858247241548, 78.03078220535525, 78.20647972450628, 78.41731674748752, 78.76871178578958, 78.9092698011104, 79.14119052638976, 79.37311125166913, 79.63314358001264, 79.79478529763159, 79.82289690069577, 80.01265022137886, 80.21645934359405, 80.42026846580926, 80.53271487806592, 80.65921709185466, 80.81383090870756, 80.91924942019818, 80.92627732096422, 81.05277953475296, 81.059807435519, 81.06683533628505, 81.09494693934921, 81.1792817485417, 81.20739335160587, 81.23550495467003, 81.24956075620211, 81.2636165577342, 81.27767235926629, 81.31281186309648, 81.31983976386252, 81.28470026003231, 81.24253285543608, 81.19333755007378, 81.17225384777566, 81.17225384777566, 81.15819804624358, 81.15117014547755, 81.15117014547755, 81.14414224471149, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.06683533628505, 81.06683533628505, 81.03169583245484, 80.98250052709255, 80.94736102326235, 80.94033312249631, 80.88410991636798, 80.84897041253777, 80.80680300794153, 80.77166350411132, 80.7013844964509, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.992550425187996, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 138, 266, 392, 522, 643, 758, 864, 973, 1070, 1168, 1258, 1341, 1419, 1506, 1584, 1668, 1735, 1817, 1899, 1974, 2055, 2122, 2194, 2259, 2329, 2405, 2480, 2554, 2625, 2697, 2766, 2830, 2888, 2950, 3015, 3085, 3149, 3224, 3281, 3339, 3399, 3446, 3506, 3565, 3625, 3688, 3744, 3789, 3831, 3893, 3941, 3990, 4034, 4084, 4122, 4154, 4197, 4242, 4286, 4317, 4353, 4387, 4417, 4433, 4464, 4483, 4498, 4518, 4544, 4563, 4579, 4592, 4602, 4609, 4620, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 4, 7, 10, 13, 22, 33, 42, 52, 63, 75, 85, 100, 111, 126, 143, 162, 176, 187, 200, 215, 233, 250, 263, 280, 297, 310, 327, 336, 351, 365, 378, 398, 421, 439, 449, 468, 480, 499, 516, 531, 543, 557, 581, 596, 604, 617, 637, 649, 661, 689, 705, 716, 729, 744, 772, 788, 804, 819, 834, 852, 864, 879, 894, 907, 925, 939, 955, 969, 984, 996, 1007, 1015, 1020, 1026, 1030, 1036, 1042, 1049, 1052, 1052, 1054, 1055, 1055, 1056, 1059, 1061, 1063, 1067, 1067, 1072, 1079, 1084, 1085, 1093, 1098, 1104, 1109, 1119, 1133], "exact_score": [56.05453650994448, 57.024386815658154, 57.89584651064727, 58.76730620563637, 59.65282170215757, 60.482113992550424, 61.22707147375078, 61.8947220465247, 62.597512123128816, 63.208939489774394, 63.82036685641999, 64.3685431161712, 64.8815798720922, 65.3243376203528, 65.85845807857193, 66.30121582683252, 66.77208517815728, 67.10942441492726, 67.58732166701805, 68.08630262140699, 68.52203246890154, 68.98587391946025, 69.33024105699627, 69.71677559912854, 70.08222643896268, 70.45470517956286, 70.8693513247593, 71.30508117225385, 71.70567151591818, 72.14140136341274, 72.54199170707709, 72.92852624920936, 73.28694918827745, 73.55400941738702, 73.82809754726263, 74.15840888326657, 74.58008292922904, 74.89633846370089, 75.33909621196149, 75.60615644107105, 75.89430037247872, 76.21055590695059, 76.45653243376202, 76.77981586899992, 77.02579239581137, 77.34204793028321, 77.72858247241548, 78.03078220535525, 78.20647972450628, 78.41731674748752, 78.76871178578958, 78.9092698011104, 79.14119052638976, 79.37311125166913, 79.63314358001264, 79.79478529763159, 79.82289690069577, 80.01265022137886, 80.21645934359405, 80.42026846580926, 80.53271487806592, 80.65921709185466, 80.81383090870756, 80.91924942019818, 80.92627732096422, 81.05277953475296, 81.059807435519, 81.06683533628505, 81.09494693934921, 81.1792817485417, 81.20739335160587, 81.23550495467003, 81.24956075620211, 81.2636165577342, 81.27767235926629, 81.31281186309648, 81.31983976386252, 81.28470026003231, 81.24253285543608, 81.19333755007378, 81.17225384777566, 81.17225384777566, 81.15819804624358, 81.15117014547755, 81.15117014547755, 81.14414224471149, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.06683533628505, 81.06683533628505, 81.03169583245484, 80.98250052709255, 80.94736102326235, 80.94033312249631, 80.88410991636798, 80.84897041253777, 80.80680300794153, 80.77166350411132, 80.7013844964509, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.25505923284571, "threshold_area100": 75.25416403120387, "threshold_area25": 64.47606999789164}, "test": {"score": [26.688029946402747, 27.81669171653008, 29.265802682698585, 30.510733630150582, 31.667753736210763, 33.27567138360321, 34.61134900604033, 35.97538496441029, 37.57763094461617, 38.68360604599722, 39.79808864815813, 40.88988458157276, 41.97600884780081, 43.17840229135355, 44.10855570995094, 45.21453081133199, 46.17587839945552, 47.15991265632533, 48.24603692255339, 49.184697841930635, 49.87380540509883, 50.6593313104387, 51.476051385304714, 52.204860618778895, 52.90814734991351, 53.571732410742136, 54.16725746533193, 54.819499191787436, 55.51995008932876, 56.08144514079913, 56.71667186569492, 57.36607775855713, 57.89921447409466, 58.4493661911919, 58.96832373876302, 59.60922213084536, 60.142358846382905, 60.63295805802116, 61.19445310949154, 61.92893401015229, 62.459234892096525, 63.00371494200721, 63.59073249581715, 64.10969004338826, 64.48118424410856, 65.0682017979185, 65.63820435016873, 66.09477355868758, 66.57686526954598, 67.08164364915066, 67.64597453421433, 68.26135042395713, 68.81433797464764, 69.34180302299862, 69.83523806823015, 70.36553895017441, 70.76255565323427, 71.33822987267106, 71.83166491790263, 72.24002495533563, 72.71644499900746, 73.14749170518674, 73.47077673482121, 73.73734509258996, 74.01525678473187, 74.28466097609392, 74.46048265887758, 74.5512293338627, 74.65899101040752, 74.76391685335905, 74.85749936193744, 74.9198877009897, 74.94824603692255, 74.96526103848227, 74.98227604004197, 74.97376853926212, 74.9936193744151, 74.97944020644869, 74.98227604004197, 74.98227604004197, 74.97944020644869, 74.9766043728554, 74.97944020644869, 74.97944020644869, 74.9766043728554, 74.97376853926212, 74.96526103848227, 74.96809687207555, 74.95391770410913, 74.942574369736, 74.93690270254942, 74.92839520176956, 74.92272353458299, 74.90570853302329, 74.89720103224343, 74.88585769787028, 74.87451436349714, 74.84048436037773, 74.79794685647848, 74.69018517993365, 74.47182599325072], "calls": [0.0, 1.1768709412131695, 2.68269857924737, 4.052406204803902, 5.31151632022233, 7.120778152737997, 8.737203300910302, 10.336613447522899, 12.307517794855798, 13.699912089158609, 15.174545557666677, 16.58679068712248, 18.027394152511132, 19.6324759663103, 21.005019425460112, 22.5136828970876, 23.86921135467771, 25.360859824745486, 26.92907580183195, 28.290275926608626, 29.382071860023252, 30.58730113716927, 31.843575418994412, 32.94955052037547, 34.10657062643564, 35.2295607293764, 36.1965799846865, 37.220315911862286, 38.26673850778437, 39.17987692482205, 40.268837024643396, 41.42302129711028, 42.49213056177864, 43.45631398349545, 44.38646740209285, 45.464084167541046, 46.527521765022826, 47.488869353146356, 48.53812778266171, 49.90216374103167, 50.976944672886596, 52.1112781102005, 53.29382071860024, 54.334571647335736, 55.21935172844058, 56.438760173553014, 57.47951110228852, 58.49757536227774, 59.53549045741996, 60.70101806425999, 61.82967983438732, 63.0405807787199, 64.23446672149278, 65.31208348694099, 66.53716359924, 67.73955704279273, 68.83986047698721, 70.2180756033236, 71.46867821796216, 72.63704165839549, 73.99257011598559, 75.25735189859058, 76.4455661741769, 77.43243626463999, 78.44199302384936, 79.50826645492442, 80.41006153758897, 81.36573745852593, 82.10872585996654, 82.99917760825795, 83.86977852139637, 84.7006777642288, 85.46918866800897, 86.02784788588606, 86.71979128264753, 87.18486799194623, 87.75487054419646, 88.42129143861838, 89.00547315883503, 89.52726653999943, 90.10577659302952, 90.80339165697757, 91.42727504750022, 92.31489096219833, 93.19399937611661, 93.91713694240423, 94.7196778493038, 95.32938207186002, 95.80580211553186, 96.26804299123728, 96.66222386070386, 97.04222556220401, 97.41088392933102, 97.80222896520432, 98.05178232141338, 98.31267901199557, 98.47999319399938, 98.70969571505545, 99.01312990953691, 99.39029577744378, 100.0], "rescued": [0, 402, 917, 1363, 1775, 2359, 2860, 3358, 3952, 4368, 4786, 5193, 5610, 6068, 6433, 6854, 7225, 7609, 8031, 8391, 8663, 8964, 9283, 9562, 9843, 10105, 10337, 10592, 10855, 11078, 11326, 11588, 11797, 12012, 12218, 12467, 12690, 12886, 13113, 13394, 13607, 13821, 14056, 14260, 14416, 14647, 14859, 15044, 15229, 15432, 15649, 15885, 16106, 16309, 16506, 16717, 16888, 17110, 17302, 17467, 17659, 17832, 17967, 18076, 18188, 18304, 18388, 18445, 18499, 18557, 18607, 18646, 18675, 18696, 18719, 18727, 18745, 18752, 18762, 18766, 18769, 18769, 18771, 18772, 18772, 18774, 18775, 18777, 18779, 18781, 18782, 18783, 18786, 18789, 18790, 18792, 18794, 18797, 18808, 18813, 18820], "harmed": [0, 4, 8, 15, 19, 36, 66, 83, 112, 138, 163, 185, 219, 253, 290, 321, 353, 390, 429, 458, 487, 511, 542, 564, 597, 625, 647, 672, 688, 713, 737, 770, 791, 812, 835, 858, 893, 916, 945, 967, 993, 1015, 1043, 1064, 1089, 1113, 1124, 1148, 1163, 1188, 1206, 1225, 1251, 1268, 1291, 1315, 1346, 1365, 1383, 1404, 1428, 1449, 1470, 1485, 1499, 1520, 1542, 1567, 1583, 1604, 1621, 1638, 1657, 1672, 1689, 1700, 1711, 1723, 1732, 1736, 1740, 1741, 1742, 1743, 1744, 1747, 1751, 1752, 1759, 1765, 1768, 1772, 1777, 1786, 1790, 1796, 1802, 1817, 1843, 1886, 1970], "exact_score": [26.688029946402747, 27.646541700932993, 28.610725122649804, 29.560729376400197, 30.468196126251314, 31.384170376882288, 32.257607123613994, 33.17641720783825, 33.97612228114454, 34.826872359129965, 35.686129937895245, 36.508521679947826, 37.345092589966825, 38.12778266171341, 38.91047273346, 39.68181947083346, 40.407792870714346, 41.19331877605422, 41.95048634546124, 42.71616141564814, 43.41661231318946, 44.10288404276437, 44.82602160905198, 45.591696679238865, 46.283640076000346, 46.93304596886255, 47.5937951960979, 48.29708192723251, 48.99186115758727, 49.66395371919576, 50.2708221081587, 50.93157133539405, 51.59799222981596, 52.23605478830502, 52.84008734367467, 53.407254062331624, 54.04248078722741, 54.689050846496336, 55.3271134049854, 55.96801179706775, 56.57488018603068, 57.153390239060776, 57.63548194991919, 58.17712616623656, 58.727277883333805, 59.33981793948331, 59.884297989393986, 60.37206136743896, 60.89669058219664, 61.46669313444687, 61.96863568045827, 62.47057822646968, 62.9498341037348, 63.4687916513059, 63.94804752857103, 64.38193006834358, 64.85835011201543, 65.3489493236537, 65.9047727079375, 66.33014774693021, 66.7782094546692, 67.24045033037463, 67.73388537560616, 68.24717125599071, 68.69806879732297, 69.20001134333438, 69.6026997135808, 70.06210475569294, 70.45344979156623, 70.82777982587982, 71.2418115304994, 71.66151490230554, 72.0018149334997, 72.37330913422, 72.72211666619404, 73.0822675325412, 73.38853756061594, 73.60122508011231, 73.89615177381393, 74.13436179564982, 74.38391515185889, 74.52287099792984, 74.64764767603438, 74.76391685335905, 74.87451436349714, 74.93123103536284, 74.96526103848227, 74.97093270566883, 74.99078354082182, 74.98227604004197, 74.98227604004197, 74.9766043728554, 74.9766043728554, 74.9766043728554, 74.97376853926212, 74.96526103848227, 74.94824603692255, 74.93123103536284, 74.90570853302329, 74.79511102288518, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.68362757430317, "threshold_area100": 62.56393386836062, "threshold_area25": 41.65839548535292}, "selected_budget": 76, "selected": {"val": {"score": 81.31983976386252, "calls": 75.99971888396936, "rescued": 4625, "harmed": 1030}, "test": {"score": 74.9936193744151, "calls": 87.75487054419646, "rescued": 18745, "harmed": 1711}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9854216575622559, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9784228801727295, "operator": "ge"}, {"budget": 0.03, "threshold": 0.972293496131897, "operator": "gt"}, {"budget": 0.04, "threshold": 0.9666717052459717, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9596355557441711, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9525913000106812, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9453409910202026, "operator": "ge"}, {"budget": 0.08, "threshold": 0.937260627746582, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9303016066551208, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9241647124290466, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9172589778900146, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9104824662208557, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9028980135917664, "operator": "ge"}, {"budget": 0.14, "threshold": 0.8964369893074036, "operator": "gt"}, {"budget": 0.15, "threshold": 0.8887524604797363, "operator": "ge"}, {"budget": 0.16, "threshold": 0.881582498550415, "operator": "ge"}, {"budget": 0.17, "threshold": 0.873101532459259, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8647458553314209, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8572472333908081, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8505748510360718, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8427769541740417, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8349409699440002, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8279629349708557, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8202810287475586, "operator": "ge"}, {"budget": 0.25, "threshold": 0.8128625750541687, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8054322004318237, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7981043457984924, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7898609042167664, "operator": "ge"}, {"budget": 0.29, "threshold": 0.7827404737472534, "operator": "ge"}, {"budget": 0.3, "threshold": 0.773883581161499, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7651587128639221, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7564888000488281, "operator": "ge"}, {"budget": 0.33, "threshold": 0.7480437755584717, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7401528358459473, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7310640215873718, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7223069071769714, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7141271233558655, "operator": "ge"}, {"budget": 0.38, "threshold": 0.704535186290741, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6924020051956177, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6817856431007385, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6707493662834167, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6596584320068359, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6496873497962952, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6404384970664978, "operator": "ge"}, {"budget": 0.45, "threshold": 0.6287065744400024, "operator": "ge"}, {"budget": 0.46, "threshold": 0.6173385381698608, "operator": "ge"}, {"budget": 0.47, "threshold": 0.6064399480819702, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5961914658546448, "operator": "ge"}, {"budget": 0.49, "threshold": 0.5834250450134277, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5702475309371948, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5543240904808044, "operator": "ge"}, {"budget": 0.52, "threshold": 0.5409159064292908, "operator": "ge"}, {"budget": 0.53, "threshold": 0.5273998975753784, "operator": "ge"}, {"budget": 0.54, "threshold": 0.5112614035606384, "operator": "ge"}, {"budget": 0.55, "threshold": 0.4960525631904602, "operator": "ge"}, {"budget": 0.56, "threshold": 0.47963404655456543, "operator": "ge"}, {"budget": 0.57, "threshold": 0.4602435529232025, "operator": "ge"}, {"budget": 0.58, "threshold": 0.4419208765029907, "operator": "ge"}, {"budget": 0.59, "threshold": 0.42240381240844727, "operator": "ge"}, {"budget": 0.6, "threshold": 0.402505099773407, "operator": "ge"}, {"budget": 0.61, "threshold": 0.38025006651878357, "operator": "ge"}, {"budget": 0.62, "threshold": 0.35781174898147583, "operator": "ge"}, {"budget": 0.63, "threshold": 0.33575165271759033, "operator": "ge"}, {"budget": 0.64, "threshold": 0.3126713037490845, "operator": "ge"}, {"budget": 0.65, "threshold": 0.28448978066444397, "operator": "ge"}, {"budget": 0.66, "threshold": 0.2569108307361603, "operator": "ge"}, {"budget": 0.67, "threshold": 0.2264796793460846, "operator": "ge"}, {"budget": 0.68, "threshold": 0.1984940469264984, "operator": "ge"}, {"budget": 0.69, "threshold": 0.16497844457626343, "operator": "ge"}, {"budget": 0.7, "threshold": 0.13659578561782837, "operator": "ge"}, {"budget": 0.71, "threshold": 0.1082397997379303, "operator": "ge"}, {"budget": 0.72, "threshold": 0.08217708766460419, "operator": "ge"}, {"budget": 0.73, "threshold": 0.062448568642139435, "operator": "ge"}, {"budget": 0.74, "threshold": 0.043413881212472916, "operator": "ge"}, {"budget": 0.75, "threshold": 0.030792146921157837, "operator": "ge"}, {"budget": 0.76, "threshold": 0.019726920872926712, "operator": "ge"}, {"budget": 0.77, "threshold": 0.010284006595611572, "operator": "ge"}, {"budget": 0.78, "threshold": 0.004654444754123688, "operator": "ge"}, {"budget": 0.79, "threshold": 0.0003034239634871483, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0015933213289827108, "operator": "ge"}, {"budget": 0.81, "threshold": -0.002620724029839039, "operator": "ge"}, {"budget": 0.82, "threshold": -0.0035120416432619095, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0044587417505681515, "operator": "ge"}, {"budget": 0.84, "threshold": -0.005336405243724585, "operator": "ge"}, {"budget": 0.85, "threshold": -0.006285250186920166, "operator": "ge"}, {"budget": 0.86, "threshold": -0.007330134976655245, "operator": "ge"}, {"budget": 0.87, "threshold": -0.008415879681706429, "operator": "ge"}, {"budget": 0.88, "threshold": -0.009683653712272644, "operator": "ge"}, {"budget": 0.89, "threshold": -0.011170215904712677, "operator": "ge"}, {"budget": 0.9, "threshold": -0.013171156868338585, "operator": "ge"}, {"budget": 0.91, "threshold": -0.01579274609684944, "operator": "ge"}, {"budget": 0.92, "threshold": -0.018646853044629097, "operator": "ge"}, {"budget": 0.93, "threshold": -0.023024145513772964, "operator": "ge"}, {"budget": 0.94, "threshold": -0.02863401174545288, "operator": "ge"}, {"budget": 0.95, "threshold": -0.03727847337722778, "operator": "ge"}, {"budget": 0.96, "threshold": -0.048757173120975494, "operator": "ge"}, {"budget": 0.97, "threshold": -0.07016337662935257, "operator": "ge"}, {"budget": 0.98, "threshold": -0.1170363798737526, "operator": "ge"}, {"budget": 0.99, "threshold": -0.21500259637832642, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.024386815658154, 57.89584651064727, 58.760278304870326, 59.65282170215757, 60.482113992550424, 61.22707147375078, 61.8947220465247, 62.597512123128816, 63.208939489774394, 63.82036685641999, 64.3685431161712, 64.8815798720922, 65.3243376203528, 65.85845807857193, 66.30121582683252, 66.77208517815728, 67.10942441492726, 67.58732166701805, 68.08630262140699, 68.52203246890154, 68.98587391946025, 69.33024105699627, 69.71677559912854, 70.08222643896268, 70.45470517956286, 70.8693513247593, 71.30508117225385, 71.70567151591818, 72.14140136341274, 72.54199170707709, 72.92852624920936, 73.28694918827745, 73.55400941738702, 73.82809754726263, 74.15840888326657, 74.58008292922904, 74.89633846370089, 75.33909621196149, 75.60615644107105, 75.89430037247872, 76.21055590695059, 76.45653243376202, 76.77981586899992, 77.02579239581137, 77.34204793028321, 77.72858247241548, 78.03078220535525, 78.20647972450628, 78.41731674748752, 78.76871178578958, 78.9092698011104, 79.14119052638976, 79.37311125166913, 79.63314358001264, 79.79478529763159, 79.82289690069577, 80.01265022137886, 80.21645934359405, 80.42026846580926, 80.53271487806592, 80.65921709185466, 80.81383090870756, 80.91924942019818, 80.92627732096422, 81.05277953475296, 81.059807435519, 81.06683533628505, 81.09494693934921, 81.1792817485417, 81.20739335160587, 81.23550495467003, 81.24956075620211, 81.2636165577342, 81.27767235926629, 81.31281186309648, 81.31983976386252, 81.28470026003231, 81.24253285543608, 81.19333755007378, 81.17225384777566, 81.17225384777566, 81.15819804624358, 81.15117014547755, 81.15117014547755, 81.14414224471149, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.06683533628505, 81.06683533628505, 81.03169583245484, 80.98250052709255, 80.94736102326235, 80.94033312249631, 80.88410991636798, 80.84897041253777, 80.80680300794153, 80.77166350411132, 80.7013844964509, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.992550425187996, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 138, 266, 392, 522, 643, 758, 864, 973, 1070, 1168, 1258, 1341, 1419, 1506, 1584, 1668, 1735, 1817, 1899, 1974, 2055, 2122, 2194, 2259, 2329, 2405, 2480, 2554, 2625, 2697, 2766, 2830, 2888, 2950, 3015, 3085, 3149, 3224, 3281, 3339, 3399, 3446, 3506, 3565, 3625, 3688, 3744, 3789, 3831, 3893, 3941, 3990, 4034, 4084, 4122, 4154, 4197, 4242, 4286, 4317, 4353, 4387, 4417, 4433, 4464, 4483, 4498, 4518, 4544, 4563, 4579, 4592, 4602, 4609, 4620, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 4, 7, 10, 13, 22, 33, 42, 52, 63, 75, 85, 100, 111, 126, 143, 162, 176, 187, 200, 215, 233, 250, 263, 280, 297, 310, 327, 336, 351, 365, 378, 398, 421, 439, 449, 468, 480, 499, 516, 531, 543, 557, 581, 596, 604, 617, 637, 649, 661, 689, 705, 716, 729, 744, 772, 788, 804, 819, 834, 852, 864, 879, 894, 907, 925, 939, 955, 969, 984, 996, 1007, 1015, 1020, 1026, 1030, 1036, 1042, 1049, 1052, 1052, 1054, 1055, 1055, 1056, 1059, 1061, 1063, 1067, 1067, 1072, 1079, 1084, 1085, 1093, 1098, 1104, 1109, 1119, 1133], "exact_score": [56.05453650994448, 57.024386815658154, 57.89584651064727, 58.76730620563637, 59.65282170215757, 60.482113992550424, 61.22707147375078, 61.8947220465247, 62.597512123128816, 63.208939489774394, 63.82036685641999, 64.3685431161712, 64.8815798720922, 65.3243376203528, 65.85845807857193, 66.30121582683252, 66.77208517815728, 67.10942441492726, 67.58732166701805, 68.08630262140699, 68.52203246890154, 68.98587391946025, 69.33024105699627, 69.71677559912854, 70.08222643896268, 70.45470517956286, 70.8693513247593, 71.30508117225385, 71.70567151591818, 72.14140136341274, 72.54199170707709, 72.92852624920936, 73.28694918827745, 73.55400941738702, 73.82809754726263, 74.15840888326657, 74.58008292922904, 74.89633846370089, 75.33909621196149, 75.60615644107105, 75.89430037247872, 76.21055590695059, 76.45653243376202, 76.77981586899992, 77.02579239581137, 77.34204793028321, 77.72858247241548, 78.03078220535525, 78.20647972450628, 78.41731674748752, 78.76871178578958, 78.9092698011104, 79.14119052638976, 79.37311125166913, 79.63314358001264, 79.79478529763159, 79.82289690069577, 80.01265022137886, 80.21645934359405, 80.42026846580926, 80.53271487806592, 80.65921709185466, 80.81383090870756, 80.91924942019818, 80.92627732096422, 81.05277953475296, 81.059807435519, 81.06683533628505, 81.09494693934921, 81.1792817485417, 81.20739335160587, 81.23550495467003, 81.24956075620211, 81.2636165577342, 81.27767235926629, 81.31281186309648, 81.31983976386252, 81.28470026003231, 81.24253285543608, 81.19333755007378, 81.17225384777566, 81.17225384777566, 81.15819804624358, 81.15117014547755, 81.15117014547755, 81.14414224471149, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.06683533628505, 81.06683533628505, 81.03169583245484, 80.98250052709255, 80.94736102326235, 80.94033312249631, 80.88410991636798, 80.84897041253777, 80.80680300794153, 80.77166350411132, 80.7013844964509, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.25505923284571, "threshold_area100": 75.25416403120387, "threshold_area25": 64.47606999789164}, "test": {"score": [26.688029946402747, 27.81669171653008, 29.265802682698585, 30.510733630150582, 31.667753736210763, 33.27567138360321, 34.61134900604033, 35.97538496441029, 37.57763094461617, 38.68360604599722, 39.79808864815813, 40.88988458157276, 41.97600884780081, 43.17840229135355, 44.10855570995094, 45.21453081133199, 46.17587839945552, 47.15991265632533, 48.24603692255339, 49.184697841930635, 49.87380540509883, 50.6593313104387, 51.476051385304714, 52.204860618778895, 52.90814734991351, 53.571732410742136, 54.16725746533193, 54.819499191787436, 55.51995008932876, 56.08144514079913, 56.71667186569492, 57.36607775855713, 57.89921447409466, 58.4493661911919, 58.96832373876302, 59.60922213084536, 60.142358846382905, 60.63295805802116, 61.19445310949154, 61.92893401015229, 62.459234892096525, 63.00371494200721, 63.59073249581715, 64.10969004338826, 64.48118424410856, 65.0682017979185, 65.63820435016873, 66.09477355868758, 66.57686526954598, 67.08164364915066, 67.64597453421433, 68.26135042395713, 68.81433797464764, 69.34180302299862, 69.83523806823015, 70.36553895017441, 70.76255565323427, 71.33822987267106, 71.83166491790263, 72.24002495533563, 72.71644499900746, 73.14749170518674, 73.47077673482121, 73.73734509258996, 74.01525678473187, 74.28466097609392, 74.46048265887758, 74.5512293338627, 74.65899101040752, 74.76391685335905, 74.85749936193744, 74.9198877009897, 74.94824603692255, 74.96526103848227, 74.98227604004197, 74.97376853926212, 74.9936193744151, 74.97944020644869, 74.98227604004197, 74.98227604004197, 74.97944020644869, 74.9766043728554, 74.97944020644869, 74.97944020644869, 74.9766043728554, 74.97376853926212, 74.96526103848227, 74.96809687207555, 74.95391770410913, 74.942574369736, 74.93690270254942, 74.92839520176956, 74.92272353458299, 74.90570853302329, 74.89720103224343, 74.88585769787028, 74.87451436349714, 74.84048436037773, 74.79794685647848, 74.69018517993365, 74.47182599325072], "calls": [0.0, 1.1768709412131695, 2.68269857924737, 4.052406204803902, 5.31151632022233, 7.120778152737997, 8.737203300910302, 10.336613447522899, 12.307517794855798, 13.699912089158609, 15.174545557666677, 16.58679068712248, 18.027394152511132, 19.6324759663103, 21.005019425460112, 22.5136828970876, 23.86921135467771, 25.360859824745486, 26.92907580183195, 28.290275926608626, 29.382071860023252, 30.58730113716927, 31.843575418994412, 32.94955052037547, 34.10657062643564, 35.2295607293764, 36.1965799846865, 37.220315911862286, 38.26673850778437, 39.17987692482205, 40.268837024643396, 41.42302129711028, 42.49213056177864, 43.45631398349545, 44.38646740209285, 45.464084167541046, 46.527521765022826, 47.488869353146356, 48.53812778266171, 49.90216374103167, 50.976944672886596, 52.1112781102005, 53.29382071860024, 54.334571647335736, 55.21935172844058, 56.438760173553014, 57.47951110228852, 58.49757536227774, 59.53549045741996, 60.70101806425999, 61.82967983438732, 63.0405807787199, 64.23446672149278, 65.31208348694099, 66.53716359924, 67.73955704279273, 68.83986047698721, 70.2180756033236, 71.46867821796216, 72.63704165839549, 73.99257011598559, 75.25735189859058, 76.4455661741769, 77.43243626463999, 78.44199302384936, 79.50826645492442, 80.41006153758897, 81.36573745852593, 82.10872585996654, 82.99917760825795, 83.86977852139637, 84.7006777642288, 85.46918866800897, 86.02784788588606, 86.71979128264753, 87.18486799194623, 87.75487054419646, 88.42129143861838, 89.00547315883503, 89.52726653999943, 90.10577659302952, 90.80339165697757, 91.42727504750022, 92.31489096219833, 93.19399937611661, 93.91713694240423, 94.7196778493038, 95.32938207186002, 95.80580211553186, 96.26804299123728, 96.66222386070386, 97.04222556220401, 97.41088392933102, 97.80222896520432, 98.05178232141338, 98.31267901199557, 98.47999319399938, 98.70969571505545, 99.01312990953691, 99.39029577744378, 100.0], "rescued": [0, 402, 917, 1363, 1775, 2359, 2860, 3358, 3952, 4368, 4786, 5193, 5610, 6068, 6433, 6854, 7225, 7609, 8031, 8391, 8663, 8964, 9283, 9562, 9843, 10105, 10337, 10592, 10855, 11078, 11326, 11588, 11797, 12012, 12218, 12467, 12690, 12886, 13113, 13394, 13607, 13821, 14056, 14260, 14416, 14647, 14859, 15044, 15229, 15432, 15649, 15885, 16106, 16309, 16506, 16717, 16888, 17110, 17302, 17467, 17659, 17832, 17967, 18076, 18188, 18304, 18388, 18445, 18499, 18557, 18607, 18646, 18675, 18696, 18719, 18727, 18745, 18752, 18762, 18766, 18769, 18769, 18771, 18772, 18772, 18774, 18775, 18777, 18779, 18781, 18782, 18783, 18786, 18789, 18790, 18792, 18794, 18797, 18808, 18813, 18820], "harmed": [0, 4, 8, 15, 19, 36, 66, 83, 112, 138, 163, 185, 219, 253, 290, 321, 353, 390, 429, 458, 487, 511, 542, 564, 597, 625, 647, 672, 688, 713, 737, 770, 791, 812, 835, 858, 893, 916, 945, 967, 993, 1015, 1043, 1064, 1089, 1113, 1124, 1148, 1163, 1188, 1206, 1225, 1251, 1268, 1291, 1315, 1346, 1365, 1383, 1404, 1428, 1449, 1470, 1485, 1499, 1520, 1542, 1567, 1583, 1604, 1621, 1638, 1657, 1672, 1689, 1700, 1711, 1723, 1732, 1736, 1740, 1741, 1742, 1743, 1744, 1747, 1751, 1752, 1759, 1765, 1768, 1772, 1777, 1786, 1790, 1796, 1802, 1817, 1843, 1886, 1970], "exact_score": [26.688029946402747, 27.646541700932993, 28.610725122649804, 29.560729376400197, 30.468196126251314, 31.384170376882288, 32.257607123613994, 33.17641720783825, 33.97612228114454, 34.826872359129965, 35.686129937895245, 36.508521679947826, 37.345092589966825, 38.12778266171341, 38.91047273346, 39.68181947083346, 40.407792870714346, 41.19331877605422, 41.95048634546124, 42.71616141564814, 43.41661231318946, 44.10288404276437, 44.82602160905198, 45.591696679238865, 46.283640076000346, 46.93304596886255, 47.5937951960979, 48.29708192723251, 48.99186115758727, 49.66395371919576, 50.2708221081587, 50.93157133539405, 51.59799222981596, 52.23605478830502, 52.84008734367467, 53.407254062331624, 54.04248078722741, 54.689050846496336, 55.3271134049854, 55.96801179706775, 56.57488018603068, 57.153390239060776, 57.63548194991919, 58.17712616623656, 58.727277883333805, 59.33981793948331, 59.884297989393986, 60.37206136743896, 60.89669058219664, 61.46669313444687, 61.96863568045827, 62.47057822646968, 62.9498341037348, 63.4687916513059, 63.94804752857103, 64.38193006834358, 64.85835011201543, 65.3489493236537, 65.9047727079375, 66.33014774693021, 66.7782094546692, 67.24045033037463, 67.73388537560616, 68.24717125599071, 68.69806879732297, 69.20001134333438, 69.6026997135808, 70.06210475569294, 70.45344979156623, 70.82777982587982, 71.2418115304994, 71.66151490230554, 72.0018149334997, 72.37330913422, 72.72211666619404, 73.0822675325412, 73.38853756061594, 73.60122508011231, 73.89615177381393, 74.13436179564982, 74.38391515185889, 74.52287099792984, 74.64764767603438, 74.76391685335905, 74.87451436349714, 74.93123103536284, 74.96526103848227, 74.97093270566883, 74.99078354082182, 74.98227604004197, 74.98227604004197, 74.9766043728554, 74.9766043728554, 74.9766043728554, 74.97376853926212, 74.96526103848227, 74.94824603692255, 74.93123103536284, 74.90570853302329, 74.79511102288518, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.68362757430317, "threshold_area100": 62.56393386836062, "threshold_area25": 41.65839548535292}, "selected_budget": 76, "selected": {"val": {"score": 81.31983976386252, "calls": 75.99971888396936, "rescued": 4625, "harmed": 1030}, "test": {"score": 74.9936193744151, "calls": 87.75487054419646, "rescued": 18745, "harmed": 1711}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 14.437692642211914, "operator": "ge"}, {"budget": 0.02, "threshold": 13.6863374710083, "operator": "ge"}, {"budget": 0.03, "threshold": 13.186406135559082, "operator": "gt"}, {"budget": 0.04, "threshold": 12.718012809753418, "operator": "ge"}, {"budget": 0.05, "threshold": 12.37393569946289, "operator": "ge"}, {"budget": 0.06, "threshold": 12.063810348510742, "operator": "ge"}, {"budget": 0.07, "threshold": 11.77579402923584, "operator": "ge"}, {"budget": 0.08, "threshold": 11.517487525939941, "operator": "ge"}, {"budget": 0.09, "threshold": 11.2471923828125, "operator": "ge"}, {"budget": 0.1, "threshold": 10.977311134338379, "operator": "ge"}, {"budget": 0.11, "threshold": 10.746260643005371, "operator": "ge"}, {"budget": 0.12, "threshold": 10.51978874206543, "operator": "ge"}, {"budget": 0.13, "threshold": 10.273673057556152, "operator": "gt"}, {"budget": 0.14, "threshold": 10.054679870605469, "operator": "ge"}, {"budget": 0.15, "threshold": 9.870990753173828, "operator": "ge"}, {"budget": 0.16, "threshold": 9.686843872070312, "operator": "ge"}, {"budget": 0.17, "threshold": 9.501411437988281, "operator": "ge"}, {"budget": 0.18, "threshold": 9.346303939819336, "operator": "ge"}, {"budget": 0.19, "threshold": 9.172711372375488, "operator": "ge"}, {"budget": 0.2, "threshold": 8.963956832885742, "operator": "ge"}, {"budget": 0.21, "threshold": 8.742783546447754, "operator": "ge"}, {"budget": 0.22, "threshold": 8.553441047668457, "operator": "ge"}, {"budget": 0.23, "threshold": 8.33365249633789, "operator": "ge"}, {"budget": 0.24, "threshold": 8.144926071166992, "operator": "ge"}, {"budget": 0.25, "threshold": 7.9869561195373535, "operator": "ge"}, {"budget": 0.26, "threshold": 7.821695327758789, "operator": "ge"}, {"budget": 0.27, "threshold": 7.6344099044799805, "operator": "ge"}, {"budget": 0.28, "threshold": 7.469508171081543, "operator": "ge"}, {"budget": 0.29, "threshold": 7.2728776931762695, "operator": "ge"}, {"budget": 0.3, "threshold": 7.075226783752441, "operator": "ge"}, {"budget": 0.31, "threshold": 6.90737247467041, "operator": "ge"}, {"budget": 0.32, "threshold": 6.7365007400512695, "operator": "ge"}, {"budget": 0.33, "threshold": 6.562010765075684, "operator": "ge"}, {"budget": 0.34, "threshold": 6.3767805099487305, "operator": "ge"}, {"budget": 0.35, "threshold": 6.1676201820373535, "operator": "ge"}, {"budget": 0.36, "threshold": 5.966756820678711, "operator": "ge"}, {"budget": 0.37, "threshold": 5.779446125030518, "operator": "ge"}, {"budget": 0.38, "threshold": 5.62085485458374, "operator": "ge"}, {"budget": 0.39, "threshold": 5.455382347106934, "operator": "ge"}, {"budget": 0.4, "threshold": 5.292477607727051, "operator": "ge"}, {"budget": 0.41, "threshold": 5.139331817626953, "operator": "ge"}, {"budget": 0.42, "threshold": 4.986861228942871, "operator": "ge"}, {"budget": 0.43, "threshold": 4.81995153427124, "operator": "ge"}, {"budget": 0.44, "threshold": 4.676268100738525, "operator": "ge"}, {"budget": 0.45, "threshold": 4.539680480957031, "operator": "ge"}, {"budget": 0.46, "threshold": 4.404746055603027, "operator": "ge"}, {"budget": 0.47, "threshold": 4.251511573791504, "operator": "ge"}, {"budget": 0.48, "threshold": 4.117469787597656, "operator": "ge"}, {"budget": 0.49, "threshold": 3.9758129119873047, "operator": "ge"}, {"budget": 0.5, "threshold": 3.857360601425171, "operator": "ge"}, {"budget": 0.51, "threshold": 3.7328691482543945, "operator": "ge"}, {"budget": 0.52, "threshold": 3.613007068634033, "operator": "ge"}, {"budget": 0.53, "threshold": 3.491179943084717, "operator": "ge"}, {"budget": 0.54, "threshold": 3.3661139011383057, "operator": "ge"}, {"budget": 0.55, "threshold": 3.26045298576355, "operator": "ge"}, {"budget": 0.56, "threshold": 3.129809856414795, "operator": "ge"}, {"budget": 0.57, "threshold": 3.021760940551758, "operator": "ge"}, {"budget": 0.58, "threshold": 2.9005911350250244, "operator": "ge"}, {"budget": 0.59, "threshold": 2.770753860473633, "operator": "ge"}, {"budget": 0.6, "threshold": 2.651949405670166, "operator": "ge"}, {"budget": 0.61, "threshold": 2.5140233039855957, "operator": "ge"}, {"budget": 0.62, "threshold": 2.3885481357574463, "operator": "ge"}, {"budget": 0.63, "threshold": 2.263747453689575, "operator": "ge"}, {"budget": 0.64, "threshold": 2.123654842376709, "operator": "ge"}, {"budget": 0.65, "threshold": 1.9717750549316406, "operator": "ge"}, {"budget": 0.66, "threshold": 1.8429615497589111, "operator": "ge"}, {"budget": 0.67, "threshold": 1.6931943893432617, "operator": "ge"}, {"budget": 0.68, "threshold": 1.558964490890503, "operator": "ge"}, {"budget": 0.69, "threshold": 1.4061992168426514, "operator": "ge"}, {"budget": 0.7, "threshold": 1.2680275440216064, "operator": "ge"}, {"budget": 0.71, "threshold": 1.1225662231445312, "operator": "ge"}, {"budget": 0.72, "threshold": 0.9925870299339294, "operator": "ge"}, {"budget": 0.73, "threshold": 0.8277363777160645, "operator": "ge"}, {"budget": 0.74, "threshold": 0.6726577877998352, "operator": "ge"}, {"budget": 0.75, "threshold": 0.5469142198562622, "operator": "ge"}, {"budget": 0.76, "threshold": 0.40720808506011963, "operator": "ge"}, {"budget": 0.77, "threshold": 0.2826426029205322, "operator": "ge"}, {"budget": 0.78, "threshold": 0.1486060470342636, "operator": "ge"}, {"budget": 0.79, "threshold": 0.014503836631774902, "operator": "ge"}, {"budget": 0.8, "threshold": -0.10049104690551758, "operator": "ge"}, {"budget": 0.81, "threshold": -0.2145754098892212, "operator": "ge"}, {"budget": 0.82, "threshold": -0.3233379125595093, "operator": "ge"}, {"budget": 0.83, "threshold": -0.4530538320541382, "operator": "ge"}, {"budget": 0.84, "threshold": -0.5681672096252441, "operator": "ge"}, {"budget": 0.85, "threshold": -0.6792418956756592, "operator": "ge"}, {"budget": 0.86, "threshold": -0.7899943590164185, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8964380025863647, "operator": "ge"}, {"budget": 0.88, "threshold": -1.039798617362976, "operator": "ge"}, {"budget": 0.89, "threshold": -1.1474087238311768, "operator": "ge"}, {"budget": 0.9, "threshold": -1.2759920358657837, "operator": "ge"}, {"budget": 0.91, "threshold": -1.3838919401168823, "operator": "ge"}, {"budget": 0.92, "threshold": -1.5095463991165161, "operator": "ge"}, {"budget": 0.93, "threshold": -1.6557155847549438, "operator": "ge"}, {"budget": 0.94, "threshold": -1.8267155885696411, "operator": "ge"}, {"budget": 0.95, "threshold": -2.050581455230713, "operator": "ge"}, {"budget": 0.96, "threshold": -2.3332958221435547, "operator": "ge"}, {"budget": 0.97, "threshold": -2.6984939575195312, "operator": "ge"}, {"budget": 0.98, "threshold": -3.2003138065338135, "operator": "ge"}, {"budget": 0.99, "threshold": -3.7556982040405273, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.98924731182795, 57.94504181600956, 58.865696816360945, 59.79337971747838, 60.72106261859582, 61.64874551971325, 62.53426101623445, 63.39166490969147, 64.26312460468057, 65.1556680019678, 65.96387659006254, 66.72991777356103, 67.52407056012369, 68.31822334668634, 69.11237613324899, 69.90652891981165, 70.67959800407617, 71.48780659217091, 72.23276407337129, 72.87230304308103, 73.53995361585494, 74.2146320893949, 74.87525476140277, 75.51479373111252, 76.15433270082225, 76.66034155597723, 77.26474102185676, 77.67235926628715, 78.1221449153138, 78.62112586970271, 79.04279991566519, 79.40122285473329, 79.61205987771453, 79.73856209150327, 79.97751071754865, 80.1391524351676, 80.34296155738281, 80.51865907653384, 80.58893808419425, 80.75057980181319, 80.76463560334528, 80.83491461100569, 80.89113781713402, 80.97547262632652, 81.0246679316888, 81.01764003092275, 81.15819804624358, 81.25658865696816, 81.27767235926629, 81.32686766462858, 81.35497926769273, 81.4674256799494, 81.51662098531169, 81.69231850446272, 81.741513809825, 81.79773701595333, 81.83287651978354, 81.88207182514581, 81.93126713050812, 81.95937873357228, 82.05074144353081, 82.05074144353081, 82.00857403893457, 81.96640663433831, 81.98749033663644, 81.95235083280623, 81.91018342820999, 81.82584861901749, 81.72043010752688, 81.71340220676085, 81.72745800829291, 81.62906739756835, 81.59392789373814, 81.56581629067396, 81.49553728301355, 81.41120247382106, 81.33389556539461, 81.30578396233045, 81.19333755007378, 81.13008644317942, 81.06683533628505, 81.03872373322089, 80.98250052709255, 80.93330522173027, 80.91222151943214, 80.87708201560193, 80.84194251177172, 80.8208588094736, 80.7857193056434, 80.75057980181319, 80.73652400028111, 80.7013844964509, 80.68732869491882, 80.68030079415279, 80.67327289338674, 80.65218919108862, 80.62407758802446, 80.61002178649237, 80.60299388572632, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.987560615644107, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 133, 269, 400, 532, 664, 796, 922, 1044, 1168, 1295, 1410, 1519, 1632, 1745, 1860, 1973, 2084, 2199, 2307, 2400, 2499, 2596, 2692, 2785, 2880, 2961, 3057, 3123, 3196, 3280, 3353, 3418, 3468, 3510, 3568, 3612, 3658, 3705, 3743, 3787, 3820, 3857, 3893, 3930, 3960, 3990, 4035, 4071, 4102, 4134, 4170, 4212, 4241, 4281, 4307, 4340, 4363, 4391, 4416, 4442, 4474, 4497, 4519, 4535, 4553, 4563, 4575, 4584, 4589, 4602, 4611, 4617, 4620, 4623, 4623, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 3, 3, 5, 7, 11, 12, 14, 16, 20, 29, 39, 47, 56, 69, 82, 96, 116, 140, 164, 185, 202, 224, 252, 273, 304, 331, 359, 384, 407, 438, 463, 485, 513, 538, 570, 596, 618, 633, 652, 677, 695, 716, 734, 756, 775, 798, 826, 848, 863, 878, 896, 917, 937, 951, 958, 978, 986, 993, 1003, 1018, 1029, 1033, 1049, 1058, 1067, 1071, 1079, 1086, 1089, 1094, 1099, 1102, 1107, 1112, 1114, 1119, 1121, 1122, 1123, 1126, 1130, 1132, 1133, 1133], "exact_score": [56.05453650994448, 56.98924731182795, 57.94504181600956, 58.87272471712699, 59.79337971747838, 60.72106261859582, 61.64874551971325, 62.53426101623445, 63.39166490969147, 64.26312460468057, 65.1556680019678, 65.96387659006254, 66.72991777356103, 67.53109846088972, 68.31822334668634, 69.11237613324899, 69.90652891981165, 70.67959800407617, 71.48780659217091, 72.23276407337129, 72.87230304308103, 73.53995361585494, 74.2146320893949, 74.87525476140277, 75.51479373111252, 76.15433270082225, 76.66034155597723, 77.26474102185676, 77.67235926628715, 78.1221449153138, 78.62112586970271, 79.04279991566519, 79.40122285473329, 79.61205987771453, 79.73856209150327, 79.97751071754865, 80.1391524351676, 80.34296155738281, 80.51865907653384, 80.58893808419425, 80.75057980181319, 80.76463560334528, 80.83491461100569, 80.89113781713402, 80.97547262632652, 81.0246679316888, 81.01764003092275, 81.15819804624358, 81.25658865696816, 81.27767235926629, 81.32686766462858, 81.35497926769273, 81.4674256799494, 81.51662098531169, 81.69231850446272, 81.741513809825, 81.79773701595333, 81.83287651978354, 81.88207182514581, 81.93126713050812, 81.95937873357228, 82.05074144353081, 82.05074144353081, 82.00857403893457, 81.96640663433831, 81.98749033663644, 81.95235083280623, 81.91018342820999, 81.82584861901749, 81.72043010752688, 81.71340220676085, 81.72745800829291, 81.62906739756835, 81.59392789373814, 81.56581629067396, 81.49553728301355, 81.41120247382106, 81.33389556539461, 81.30578396233045, 81.19333755007378, 81.13008644317942, 81.06683533628505, 81.03872373322089, 80.98250052709255, 80.93330522173027, 80.91222151943214, 80.87708201560193, 80.84194251177172, 80.8208588094736, 80.7857193056434, 80.75057980181319, 80.73652400028111, 80.7013844964509, 80.68732869491882, 80.68030079415279, 80.67327289338674, 80.65218919108862, 80.62407758802446, 80.61002178649237, 80.60299388572632, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 77.33568903588358, "threshold_area100": 77.33470377398271, "threshold_area25": 66.7931688804554}, "test": {"score": [26.688029946402747, 27.34594334004481, 27.96699089697417, 28.62206845702294, 29.5040127045345, 30.23282193800868, 30.978646173042566, 31.715962907296603, 32.39089130249837, 33.09984970081956, 33.888211439752716, 34.57164733573434, 35.24090406374954, 36.054788305022264, 36.77225420412331, 37.42449593057879, 38.06823015625444, 38.802711056915186, 39.36137027479227, 40.11003034341945, 40.94376541984517, 41.876754672035844, 42.63108640784959, 43.70870317329779, 44.67855826220117, 45.54348750815302, 46.533193432209394, 47.70722853982928, 48.77066613731106, 50.09500042537505, 51.52426055639056, 52.75217650228286, 53.97158494739529, 55.298755069052554, 56.58622352040382, 58.10339449281117, 59.501460454300535, 60.81161557439809, 61.741768992995496, 62.89595326546238, 63.86013668717921, 64.69103593001164, 65.47372600175822, 66.2932819102175, 67.04194197884468, 67.62045203187476, 68.16776791537873, 68.86821881292006, 69.42971386439044, 69.9940447494541, 70.42792728922666, 70.89016816493208, 71.27584153361882, 71.68987323823839, 72.03017326943255, 72.43286163967899, 72.82704250914557, 73.08793919972776, 73.36868672546295, 73.62674758245187, 73.8621217706945, 74.0492867878513, 74.27331764172078, 74.42361682216487, 74.56824433542239, 74.68734934634035, 74.78660352210534, 74.87167852990386, 74.96242520488899, 74.98227604004197, 75.01914187675467, 75.03899271190765, 75.00496270878824, 75.02764937753452, 75.0503360462808, 75.04750021268752, 75.03615687831437, 75.04182854550095, 75.01630604316138, 74.98227604004197, 74.98227604004197, 74.95958937129569, 74.93123103536284, 74.92272353458299, 74.89436519865014, 74.88018603068372, 74.85182769475087, 74.81779769163145, 74.78943935569862, 74.76675268695233, 74.74690185179934, 74.72421518305306, 74.70436434790007, 74.67884184556051, 74.64764767603438, 74.62212517369481, 74.57108016901569, 74.52854266511642, 74.50018432918357, 74.48033349403057, 74.47182599325072], "calls": [0.0, 0.691943396761478, 1.3441851232169697, 2.030456852791878, 2.9407594362362817, 3.7376286759492956, 4.554348750815302, 5.337038822561892, 6.108385559935343, 6.927941468394634, 7.849587386212177, 8.683322462637893, 9.539744207809886, 10.495420128746845, 11.38870771063154, 12.202591951904262, 13.067521197856111, 13.997674616453507, 14.703797181181407, 15.687831438051214, 16.79097070583898, 18.04440915407084, 19.107846751552618, 20.622181890366672, 21.997561183109777, 23.23682046337521, 24.674588095170574, 26.418625755040694, 27.890423389955476, 29.844312735728668, 31.908799591639962, 33.822987267107166, 35.657771601962395, 37.7024076227207, 39.88883532314324, 42.438249723506225, 44.763633269999715, 46.89901596574313, 48.66574029435953, 50.66500297762527, 52.49411564529394, 54.10203329268638, 55.695771772112415, 57.42279443042282, 58.97399540594957, 60.43728554008451, 61.73042565862236, 63.34685080679465, 64.65133425970564, 65.87357853841137, 66.95686697104614, 68.09403624195333, 69.13195133709554, 70.22091143691688, 71.27016986643224, 72.23718912174235, 73.3942092278025, 74.29884014406035, 75.22615772906445, 76.15631114766185, 76.984374556901, 77.89184130675211, 78.75677055270397, 79.5111022885177, 80.2937923602643, 81.21543827808185, 81.84782916938434, 82.59648923801151, 83.28559680117971, 84.01157020106061, 84.68366276266909, 85.28202365085217, 85.81232453279641, 86.44471542409892, 87.02889714431558, 87.46561551768143, 87.9703938972861, 88.43263477299152, 88.96293565493578, 89.52726653999943, 90.05756742194369, 90.57936080310807, 91.19757252644415, 91.92354592632505, 92.5616084848141, 93.31877605422115, 94.13266029549386, 94.81042452428892, 95.66968210305419, 96.1517738139126, 96.74162720131582, 97.21521141139438, 97.69163145506622, 98.12834982843206, 98.58491903695091, 99.0159657431302, 99.51223662195503, 99.80999914924992, 99.9262683265746, 99.97447749766044, 100.0], "rescued": [0, 232, 451, 682, 993, 1250, 1513, 1773, 2011, 2261, 2541, 2782, 3018, 3306, 3559, 3791, 4020, 4279, 4477, 4744, 5040, 5372, 5645, 6038, 6388, 6704, 7066, 7495, 7880, 8371, 8899, 9367, 9827, 10325, 10823, 11413, 11943, 12452, 12829, 13282, 13667, 13998, 14319, 14664, 14969, 15232, 15461, 15762, 15996, 16232, 16418, 16617, 16784, 16967, 17129, 17289, 17461, 17590, 17718, 17836, 17939, 18039, 18137, 18225, 18304, 18381, 18432, 18493, 18542, 18576, 18609, 18632, 18645, 18674, 18699, 18716, 18729, 18743, 18756, 18766, 18777, 18781, 18786, 18795, 18798, 18801, 18803, 18805, 18808, 18810, 18811, 18813, 18814, 18815, 18815, 18816, 18819, 18820, 18820, 18820, 18820], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 3, 3, 5, 7, 7, 8, 11, 13, 16, 23, 36, 44, 55, 68, 83, 93, 117, 141, 176, 206, 236, 280, 335, 372, 419, 468, 514, 559, 597, 642, 698, 739, 798, 834, 888, 924, 961, 994, 1030, 1061, 1098, 1140, 1158, 1191, 1228, 1257, 1284, 1304, 1338, 1357, 1392, 1420, 1455, 1471, 1502, 1519, 1546, 1566, 1582, 1607, 1628, 1645, 1663, 1680, 1692, 1714, 1736, 1747, 1759, 1774, 1786, 1799, 1807, 1819, 1833, 1846, 1856, 1864, 1874, 1882, 1892, 1903, 1913, 1934, 1950, 1960, 1967, 1970], "exact_score": [26.688029946402747, 27.640870033746424, 28.59087428749681, 29.552221875620337, 30.468196126251314, 31.395513711255425, 32.30014462751326, 33.159402206278536, 34.01015228426396, 34.81269319116355, 35.6209057652497, 36.45464084167541, 37.26568924935485, 38.014349317982024, 38.802711056915186, 39.579729461475196, 40.35391203244194, 41.08272126591611, 41.839888835323144, 42.54884723364433, 43.27482063352522, 43.99228653262627, 44.67855826220117, 45.35065082380966, 46.051101721350996, 46.760060119672175, 47.4434960156538, 48.118424410855575, 48.847233644329755, 49.53350537390466, 50.19709043473329, 50.911720500241046, 51.59232056262939, 52.20202478518561, 52.87978901398066, 53.56322490996228, 54.21546663641777, 54.84502169412699, 55.48592008620934, 56.03890763689987, 56.64861185945609, 57.2413010804526, 57.83966196863568, 58.44085869041204, 58.99951790828914, 59.64608796755806, 60.27847885886056, 60.87967558063694, 61.41281229617446, 61.94027734452543, 62.5584890678615, 63.0604316138729, 63.59356832941043, 64.11536171057483, 64.63431925814594, 65.1220826361909, 65.60133851345603, 66.10044522587415, 66.56268610157956, 67.05328531321781, 67.41910784675156, 67.83597538496441, 68.24433542239741, 68.70657629810283, 69.15180217224854, 69.59702804639424, 70.04792558772651, 70.45628562515951, 70.84479482743953, 71.21912486175312, 71.62464906559282, 71.92241159288773, 72.32793579672744, 72.70793749822761, 73.00002835833594, 73.30913422000398, 73.58421007855259, 73.86779343788106, 74.07480929019086, 74.30451181124693, 74.52287099792984, 74.67317017837394, 74.8121260244449, 74.93973853614271, 74.98227604004197, 75.02481354394124, 74.9936193744151, 75.04750021268752, 75.04182854550095, 75.0134702095681, 74.97944020644869, 74.942574369736, 74.91705186739642, 74.89152936505685, 74.85749936193744, 74.80929019085161, 74.77526018773219, 74.7355585174262, 74.69018517993365, 74.62496100728809, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.83372675139682, "threshold_area100": 61.02237472705102, "threshold_area25": 35.721010691092644}, "selected_budget": 61, "selected": {"val": {"score": 82.05074144353081, "calls": 60.995150748471424, "rescued": 4474, "harmed": 775}, "test": {"score": 74.0492867878513, "calls": 77.89184130675211, "rescued": 18039, "harmed": 1338}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.230041980743408, "operator": "ge"}, {"budget": 0.02, "threshold": 3.844472646713257, "operator": "gt"}, {"budget": 0.03, "threshold": 3.597620964050293, "operator": "ge"}, {"budget": 0.04, "threshold": 3.4190192222595215, "operator": "ge"}, {"budget": 0.05, "threshold": 3.222951650619507, "operator": "ge"}, {"budget": 0.06, "threshold": 3.0817313194274902, "operator": "ge"}, {"budget": 0.07, "threshold": 2.935257911682129, "operator": "ge"}, {"budget": 0.08, "threshold": 2.807858943939209, "operator": "ge"}, {"budget": 0.09, "threshold": 2.707568407058716, "operator": "ge"}, {"budget": 0.1, "threshold": 2.622330904006958, "operator": "ge"}, {"budget": 0.11, "threshold": 2.5429580211639404, "operator": "ge"}, {"budget": 0.12, "threshold": 2.472980499267578, "operator": "ge"}, {"budget": 0.13, "threshold": 2.4061975479125977, "operator": "ge"}, {"budget": 0.14, "threshold": 2.3344035148620605, "operator": "ge"}, {"budget": 0.15, "threshold": 2.275629758834839, "operator": "ge"}, {"budget": 0.16, "threshold": 2.2175822257995605, "operator": "ge"}, {"budget": 0.17, "threshold": 2.153965473175049, "operator": "ge"}, {"budget": 0.18, "threshold": 2.0950229167938232, "operator": "ge"}, {"budget": 0.19, "threshold": 2.0386157035827637, "operator": "ge"}, {"budget": 0.2, "threshold": 1.9920676946640015, "operator": "ge"}, {"budget": 0.21, "threshold": 1.9451904296875, "operator": "ge"}, {"budget": 0.22, "threshold": 1.9042418003082275, "operator": "ge"}, {"budget": 0.23, "threshold": 1.8576440811157227, "operator": "ge"}, {"budget": 0.24, "threshold": 1.8160371780395508, "operator": "ge"}, {"budget": 0.25, "threshold": 1.777901291847229, "operator": "ge"}, {"budget": 0.26, "threshold": 1.745612382888794, "operator": "ge"}, {"budget": 0.27, "threshold": 1.7085671424865723, "operator": "ge"}, {"budget": 0.28, "threshold": 1.6777533292770386, "operator": "ge"}, {"budget": 0.29, "threshold": 1.6464855670928955, "operator": "ge"}, {"budget": 0.3, "threshold": 1.614311695098877, "operator": "ge"}, {"budget": 0.31, "threshold": 1.5822107791900635, "operator": "ge"}, {"budget": 0.32, "threshold": 1.5492782592773438, "operator": "ge"}, {"budget": 0.33, "threshold": 1.51857590675354, "operator": "ge"}, {"budget": 0.34, "threshold": 1.489561676979065, "operator": "ge"}, {"budget": 0.35, "threshold": 1.4611142873764038, "operator": "ge"}, {"budget": 0.36, "threshold": 1.4322868585586548, "operator": "ge"}, {"budget": 0.37, "threshold": 1.4074842929840088, "operator": "ge"}, {"budget": 0.38, "threshold": 1.3804194927215576, "operator": "ge"}, {"budget": 0.39, "threshold": 1.3520455360412598, "operator": "ge"}, {"budget": 0.4, "threshold": 1.3238614797592163, "operator": "ge"}, {"budget": 0.41, "threshold": 1.2954521179199219, "operator": "ge"}, {"budget": 0.42, "threshold": 1.268854022026062, "operator": "ge"}, {"budget": 0.43, "threshold": 1.2452988624572754, "operator": "ge"}, {"budget": 0.44, "threshold": 1.2200770378112793, "operator": "ge"}, {"budget": 0.45, "threshold": 1.1899731159210205, "operator": "ge"}, {"budget": 0.46, "threshold": 1.165830373764038, "operator": "ge"}, {"budget": 0.47, "threshold": 1.1348813772201538, "operator": "ge"}, {"budget": 0.48, "threshold": 1.1055660247802734, "operator": "ge"}, {"budget": 0.49, "threshold": 1.0777099132537842, "operator": "ge"}, {"budget": 0.5, "threshold": 1.0512492656707764, "operator": "ge"}, {"budget": 0.51, "threshold": 1.0285214185714722, "operator": "ge"}, {"budget": 0.52, "threshold": 1.0000646114349365, "operator": "ge"}, {"budget": 0.53, "threshold": 0.9756843447685242, "operator": "ge"}, {"budget": 0.54, "threshold": 0.9527468681335449, "operator": "ge"}, {"budget": 0.55, "threshold": 0.9251505732536316, "operator": "ge"}, {"budget": 0.56, "threshold": 0.8974554538726807, "operator": "ge"}, {"budget": 0.57, "threshold": 0.8642416000366211, "operator": "ge"}, {"budget": 0.58, "threshold": 0.8396342396736145, "operator": "ge"}, {"budget": 0.59, "threshold": 0.8084213733673096, "operator": "ge"}, {"budget": 0.6, "threshold": 0.7796074748039246, "operator": "ge"}, {"budget": 0.61, "threshold": 0.7484841346740723, "operator": "ge"}, {"budget": 0.62, "threshold": 0.7202924489974976, "operator": "ge"}, {"budget": 0.63, "threshold": 0.6891469359397888, "operator": "ge"}, {"budget": 0.64, "threshold": 0.6615337133407593, "operator": "ge"}, {"budget": 0.65, "threshold": 0.6328758001327515, "operator": "ge"}, {"budget": 0.66, "threshold": 0.604158341884613, "operator": "ge"}, {"budget": 0.67, "threshold": 0.5741617679595947, "operator": "ge"}, {"budget": 0.68, "threshold": 0.5420289039611816, "operator": "ge"}, {"budget": 0.69, "threshold": 0.5032674074172974, "operator": "ge"}, {"budget": 0.7, "threshold": 0.47339463233947754, "operator": "ge"}, {"budget": 0.71, "threshold": 0.4337151050567627, "operator": "ge"}, {"budget": 0.72, "threshold": 0.4004795551300049, "operator": "ge"}, {"budget": 0.73, "threshold": 0.36091911792755127, "operator": "ge"}, {"budget": 0.74, "threshold": 0.31952980160713196, "operator": "ge"}, {"budget": 0.75, "threshold": 0.2696484923362732, "operator": "ge"}, {"budget": 0.76, "threshold": 0.21190905570983887, "operator": "ge"}, {"budget": 0.77, "threshold": 0.1484241485595703, "operator": "ge"}, {"budget": 0.78, "threshold": 0.08455324172973633, "operator": "ge"}, {"budget": 0.79, "threshold": 0.006819605827331543, "operator": "ge"}, {"budget": 0.8, "threshold": -0.05940699577331543, "operator": "ge"}, {"budget": 0.81, "threshold": -0.13670432567596436, "operator": "ge"}, {"budget": 0.82, "threshold": -0.20640933513641357, "operator": "ge"}, {"budget": 0.83, "threshold": -0.29338395595550537, "operator": "ge"}, {"budget": 0.84, "threshold": -0.37569373846054077, "operator": "ge"}, {"budget": 0.85, "threshold": -0.46031683683395386, "operator": "ge"}, {"budget": 0.86, "threshold": -0.5415228605270386, "operator": "ge"}, {"budget": 0.87, "threshold": -0.6359655261039734, "operator": "ge"}, {"budget": 0.88, "threshold": -0.7326368093490601, "operator": "ge"}, {"budget": 0.89, "threshold": -0.8257349729537964, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9233217239379883, "operator": "ge"}, {"budget": 0.91, "threshold": -1.0323225259780884, "operator": "ge"}, {"budget": 0.92, "threshold": -1.1320240497589111, "operator": "ge"}, {"budget": 0.93, "threshold": -1.2669434547424316, "operator": "ge"}, {"budget": 0.94, "threshold": -1.403332233428955, "operator": "ge"}, {"budget": 0.95, "threshold": -1.6273550987243652, "operator": "ge"}, {"budget": 0.96, "threshold": -1.9042006731033325, "operator": "ge"}, {"budget": 0.97, "threshold": -2.3250198364257812, "operator": "ge"}, {"budget": 0.98, "threshold": -2.802602767944336, "operator": "ge"}, {"budget": 0.99, "threshold": -3.3511195182800293, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.01033101412608, 57.86070700681706, 58.73919460257221, 59.57551479373111, 60.348583877995644, 61.058401855365794, 61.79633143580012, 62.43587040550986, 62.99107456602713, 63.60250193267271, 64.164733993956, 64.63560334528076, 65.16269590273384, 65.4438119333755, 65.90062548316817, 66.28716002530044, 66.8283083842856, 67.22187082718392, 67.67868437697659, 68.05116311757678, 68.50797666736945, 68.96479021716213, 69.43565956848688, 69.78705460678894, 70.15953334738913, 70.53903998875535, 70.88340712629137, 71.27696956918967, 71.71269941668423, 72.19059666877503, 72.62632651626959, 73.03394476069998, 73.39236769976807, 73.71565113500597, 74.10921357790427, 74.54494342539884, 74.93850586829713, 75.19151029587462, 75.54290533417668, 75.8380771663504, 76.15433270082225, 76.52681144142244, 76.81495537283013, 77.13121090730198, 77.37718743411342, 77.64424766322298, 77.9534752969288, 78.16431231991004, 78.44542835055168, 78.69140487736313, 79.01468831260102, 79.26066483941247, 79.51366926698995, 79.80884109916367, 80.04076182444304, 80.24457094665823, 80.37810106121302, 80.61002178649237, 80.72949609951507, 80.91222151943214, 81.0246679316888, 81.17225384777566, 81.2636165577342, 81.30578396233045, 81.32686766462858, 81.43228617611919, 81.51662098531169, 81.59392789373814, 81.65717900063251, 81.74854171059106, 81.73448590905896, 81.66420690139856, 81.62906739756835, 81.53067678684376, 81.47445358071543, 81.39714667228898, 81.3479513669267, 81.29172816079837, 81.19333755007378, 81.12305854241337, 81.06683533628505, 81.03872373322089, 80.98250052709255, 80.94033312249631, 80.89816571790006, 80.86302621406986, 80.84897041253777, 80.81383090870756, 80.79274720640944, 80.77166350411132, 80.73652400028111, 80.72246819874903, 80.7013844964509, 80.68732869491882, 80.68030079415279, 80.66624499262069, 80.63813338955653, 80.61002178649237, 80.60299388572632, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.988895916789655, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 137, 262, 390, 513, 630, 738, 852, 954, 1044, 1141, 1231, 1312, 1398, 1463, 1542, 1611, 1702, 1774, 1848, 1920, 1999, 2079, 2158, 2223, 2289, 2355, 2419, 2488, 2567, 2645, 2720, 2789, 2853, 2919, 2986, 3056, 3123, 3181, 3244, 3303, 3366, 3430, 3487, 3548, 3598, 3652, 3710, 3757, 3809, 3863, 3921, 3970, 4024, 4077, 4125, 4165, 4203, 4253, 4285, 4324, 4361, 4395, 4421, 4445, 4471, 4496, 4520, 4545, 4566, 4588, 4601, 4612, 4619, 4621, 4624, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 1, 5, 8, 12, 19, 26, 35, 46, 57, 67, 77, 91, 102, 127, 141, 155, 169, 185, 194, 213, 227, 242, 254, 269, 282, 294, 309, 322, 339, 349, 362, 373, 386, 406, 417, 425, 436, 458, 471, 488, 506, 517, 533, 549, 564, 580, 594, 611, 623, 642, 654, 668, 686, 697, 712, 723, 742, 759, 774, 787, 808, 821, 834, 852, 875, 885, 897, 911, 923, 932, 947, 968, 980, 996, 1007, 1020, 1027, 1035, 1049, 1059, 1067, 1071, 1079, 1085, 1091, 1096, 1098, 1103, 1106, 1109, 1114, 1116, 1119, 1121, 1122, 1124, 1128, 1132, 1133, 1133], "exact_score": [56.05453650994448, 57.01033101412608, 57.8677349075831, 58.73919460257221, 59.57551479373111, 60.348583877995644, 61.058401855365794, 61.79633143580012, 62.43587040550986, 62.99107456602713, 63.60250193267271, 64.164733993956, 64.63560334528076, 65.16269590273384, 65.4438119333755, 65.90062548316817, 66.28716002530044, 66.8283083842856, 67.22187082718392, 67.67868437697659, 68.05116311757678, 68.50797666736945, 68.96479021716213, 69.43565956848688, 69.78705460678894, 70.15953334738913, 70.53903998875535, 70.88340712629137, 71.27696956918967, 71.71269941668423, 72.19059666877503, 72.62632651626959, 73.03394476069998, 73.39236769976807, 73.71565113500597, 74.10921357790427, 74.54494342539884, 74.93850586829713, 75.19151029587462, 75.54290533417668, 75.8380771663504, 76.15433270082225, 76.52681144142244, 76.81495537283013, 77.13121090730198, 77.37718743411342, 77.64424766322298, 77.9534752969288, 78.16431231991004, 78.44542835055168, 78.69140487736313, 79.01468831260102, 79.26066483941247, 79.51366926698995, 79.80884109916367, 80.04076182444304, 80.24457094665823, 80.37810106121302, 80.61002178649237, 80.72949609951507, 80.91222151943214, 81.0246679316888, 81.17225384777566, 81.2636165577342, 81.30578396233045, 81.32686766462858, 81.43228617611919, 81.51662098531169, 81.59392789373814, 81.65717900063251, 81.74854171059106, 81.73448590905896, 81.66420690139856, 81.62906739756835, 81.53067678684376, 81.47445358071543, 81.39714667228898, 81.3479513669267, 81.29172816079837, 81.19333755007378, 81.12305854241337, 81.06683533628505, 81.03872373322089, 80.98250052709255, 80.94033312249631, 80.89816571790006, 80.86302621406986, 80.84897041253777, 80.81383090870756, 80.79274720640944, 80.77166350411132, 80.73652400028111, 80.72246819874903, 80.7013844964509, 80.68732869491882, 80.68030079415279, 80.66624499262069, 80.63813338955653, 80.61002178649237, 80.60299388572632, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.20510404096095, "threshold_area100": 75.20419565675732, "threshold_area25": 64.20198186801602}, "test": {"score": [26.688029946402747, 27.88758755636219, 29.399086861582962, 30.649689476221536, 31.820888750248137, 33.46000056716672, 34.693588180245584, 36.09165414173496, 37.5719592774296, 38.68360604599722, 39.80659614893798, 40.85585457845334, 41.83705300172986, 42.75019141876755, 43.859002353741886, 44.71542409891388, 45.620055015171715, 46.66931344468706, 47.59946686328447, 48.577829452967705, 49.45693786688597, 50.242463772225854, 50.90321299946119, 51.66038056886823, 52.377846467969256, 53.03008819442475, 53.62277741542126, 54.226809970790924, 54.72024501602246, 55.261889232339854, 55.905623458015484, 56.489805178232146, 57.142046904687625, 57.743243626464, 58.14593199671043, 58.59399370444942, 59.0619062473414, 59.47593795196098, 59.93817882766639, 60.5308680486629, 61.038482261860885, 61.52624563990585, 62.050874854663526, 62.49326489521595, 62.95834160451466, 63.406403312253644, 63.84595751921278, 64.37625840115702, 64.89805178232142, 65.40283016192609, 65.92178770949721, 66.2762669086578, 66.77253778748263, 67.20925616084848, 67.6913478717069, 68.13940957944588, 68.66120296061027, 69.16314550662167, 69.55165470890168, 70.01956725179367, 70.49031562827894, 70.98942234069705, 71.38360321016364, 71.8174857499362, 72.13226327879082, 72.52360831466409, 72.90928168335083, 73.26092504891812, 73.60122508011231, 73.97839094801917, 74.1995859682954, 74.43496015653801, 74.6051101721351, 74.78943935569862, 74.9198877009897, 74.97093270566883, 75.00496270878824, 75.03048521112781, 75.01063437597482, 74.98227604004197, 74.98227604004197, 74.96242520488899, 74.93973853614271, 74.9198877009897, 74.87735019709044, 74.86033519553072, 74.84048436037773, 74.82063352522475, 74.80078269007176, 74.77526018773219, 74.75257351898591, 74.7327226838329, 74.72137934945977, 74.68734934634035, 74.67317017837394, 74.63914017525452, 74.59660267135524, 74.53988599948956, 74.50018432918357, 74.48033349403057, 74.47182599325072], "calls": [0.0, 1.250602614638573, 2.8443410940646006, 4.216884553214417, 5.512860505345547, 7.435555681592604, 8.930039985253666, 10.59467430451181, 12.440801973740182, 13.884241272722116, 15.36171057482347, 16.731418200380002, 18.08694665797011, 19.363071774948246, 20.919944417661572, 22.21024870260613, 23.486373819584266, 25.023395627144602, 26.486685761279528, 27.930125060261464, 29.24028018035902, 30.476703627031164, 31.52029038935995, 32.765221336811955, 33.998808949890815, 35.10761988486515, 36.08314664095511, 37.15509173921674, 38.02285681876187, 38.95868190454584, 40.05614950514704, 41.05719876357655, 42.12914386183819, 43.161387289793836, 43.97810736465984, 44.871394946544534, 45.90647420809347, 46.70334344780648, 47.582451861724756, 48.61185945608712, 49.59022204577035, 50.687689646371545, 51.79082891415931, 52.66142982729774, 53.50083657091002, 54.64367750900377, 55.536965090888465, 56.67697019538893, 57.85100530300882, 58.88041289737118, 59.96370133000596, 60.754898902532396, 61.91191900859258, 62.85908742874968, 63.79491251453365, 64.8810367807617, 66.07208688994129, 67.32268950457987, 68.27836542551682, 69.43254969798373, 70.62076397357004, 71.88270992258174, 72.92346085131724, 74.04361512066473, 75.06735104784052, 76.15914698125515, 77.25094291466976, 78.33990301449111, 79.49692312055129, 80.83827241017497, 81.74290332643281, 82.87723676374671, 83.70530017298586, 84.71202109860194, 85.54859200862094, 86.35964041630038, 87.15650965601338, 88.11218557695034, 88.82114397527154, 89.52159487281286, 90.16249326489522, 90.99055667413435, 91.77891841306753, 92.6920568301052, 93.65340441822873, 94.54102033292686, 95.21311289453534, 95.91639962566997, 96.49774551229335, 96.9344638856592, 97.35700309105862, 97.80790063239088, 98.17372316592461, 98.57357570257777, 98.85999489549954, 99.20880242747356, 99.54626662507444, 99.7958199812835, 99.91492499220146, 99.96880583047387, 100.0], "rescued": [0, 427, 968, 1413, 1835, 2439, 2899, 3412, 3960, 4382, 4804, 5196, 5570, 5928, 6356, 6691, 7038, 7444, 7810, 8187, 8525, 8830, 9085, 9388, 9673, 9926, 10155, 10395, 10591, 10803, 11055, 11286, 11536, 11773, 11932, 12108, 12294, 12464, 12648, 12880, 13081, 13288, 13497, 13669, 13849, 14041, 14212, 14422, 14637, 14835, 15037, 15176, 15366, 15538, 15718, 15900, 16102, 16304, 16460, 16650, 16841, 17040, 17193, 17367, 17501, 17651, 17804, 17945, 18085, 18239, 18331, 18429, 18501, 18585, 18644, 18684, 18710, 18741, 18754, 18766, 18777, 18783, 18787, 18793, 18797, 18800, 18803, 18805, 18806, 18808, 18809, 18812, 18814, 18815, 18815, 18815, 18818, 18820, 18820, 18820, 18820], "harmed": [0, 4, 12, 16, 25, 51, 76, 96, 122, 152, 178, 200, 228, 264, 301, 334, 362, 398, 436, 468, 496, 524, 546, 582, 614, 637, 657, 684, 706, 727, 752, 777, 797, 822, 839, 857, 878, 902, 923, 946, 968, 1003, 1027, 1043, 1059, 1093, 1109, 1132, 1163, 1183, 1202, 1216, 1231, 1249, 1259, 1283, 1301, 1326, 1345, 1370, 1395, 1418, 1432, 1453, 1476, 1488, 1505, 1522, 1542, 1563, 1577, 1592, 1604, 1623, 1636, 1658, 1672, 1694, 1714, 1736, 1747, 1760, 1772, 1785, 1804, 1813, 1823, 1832, 1840, 1851, 1860, 1870, 1876, 1889, 1894, 1906, 1924, 1946, 1960, 1967, 1970], "exact_score": [26.688029946402747, 27.64370586733971, 28.605053455463235, 29.5408785412472, 30.44834529109832, 31.333125372203163, 32.223577120494575, 33.094178033632986, 33.93642061083856, 34.74179735133143, 35.60956243087656, 36.39508833621643, 37.23449507982872, 38.02852848594845, 38.76868105379577, 39.51734112242294, 40.27450869182997, 41.01749709327057, 41.7916796642373, 42.44959305787936, 43.201088960099824, 43.92706235998072, 44.57646825284293, 45.285426651164116, 45.97737004792559, 46.65229844312736, 47.29886850239629, 47.922751892918924, 48.617531123273686, 49.292459518475454, 49.93619374415109, 50.58843547060658, 51.19246802597623, 51.776649746192895, 52.377846467969256, 52.94501318662621, 53.577404077928705, 54.12755579502595, 54.703230014462754, 55.27890423389956, 55.88293678926921, 56.458611008706015, 57.0739868984488, 57.661004452258744, 58.16294699827014, 58.66488954428155, 59.124294586393674, 59.6205654652185, 60.19340385106202, 60.729376400192834, 61.20012477667811, 61.66520148597681, 62.14729319683521, 62.66625074440633, 63.171029124011, 63.59640416300372, 64.05864503870913, 64.49819924566826, 64.95193262059382, 65.45103933301193, 65.93596687746364, 66.38970025238919, 66.80940362419534, 67.28865950146046, 67.77925871309871, 68.190454584125, 68.62717295749086, 69.02702549414401, 69.40702719564416, 69.84374556901001, 70.21240393613704, 70.64345064231632, 71.04613901256273, 71.4261407140629, 71.80047074837648, 72.10674077645123, 72.47539914357826, 72.82420667555229, 73.1361483708136, 73.46794090122792, 73.74868842696311, 74.02376428551173, 74.2506309729745, 74.454810991691, 74.68451351274707, 74.85182769475087, 74.95958937129569, 75.00212687519496, 75.04182854550095, 74.99929104160168, 74.9766043728554, 74.95958937129569, 74.93690270254942, 74.90003686583671, 74.87451436349714, 74.85466352834416, 74.82063352522475, 74.77526018773219, 74.72988685023962, 74.66466267759408, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.22481659119777, "threshold_area100": 61.79199160593256, "threshold_area25": 41.45960355046366}, "selected_budget": 70, "selected": {"val": {"score": 81.74854171059106, "calls": 69.9978916297702, "rescued": 4588, "harmed": 932}, "test": {"score": 74.1995859682954, "calls": 81.74290332643281, "rescued": 18331, "harmed": 1577}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 81.08791903858317, "calls": 73.99676716564763}, "test": {"score": 74.90003686583671, "calls": 86.46456625925191}}, "old_normalized_interior": {"val": {"score": 81.08791903858317, "calls": 73.99676716564763}, "test": {"score": 74.90003686583671, "calls": 86.46456625925191}}, "old_normalized_endpoints": {"val": {"score": 81.08791903858317, "calls": 73.99676716564763}, "test": {"score": 74.90003686583671, "calls": 86.46456625925191}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 56.97519151029587, 57.90990231217935, 58.760278304870326, 59.59659849602923, 60.446974488720215, 61.19895987068662, 61.95797315341907, 62.597512123128816, 63.2440789936046, 63.83442265795206, 64.53721273455618, 65.10647269660552, 65.67573265865485, 66.25904842223628, 66.76505727739124, 67.24295452948203, 67.73490758310491, 68.07927472064094, 68.50797666736945, 68.95073441563004, 69.33024105699627, 69.73083140066062, 70.07519853819663, 70.39848197343453, 70.77798861480076, 71.0239651416122, 71.28399746995572, 71.60728090519362, 71.92353643966547, 72.23276407337129, 72.53496380631105, 72.85824724154895, 73.0550284629981, 73.27992128751141, 73.54698151662097, 73.87026495185887, 74.18652048633074, 74.4887202192705, 74.75578044838007, 75.06500808208587, 75.33206831119544, 75.66237964719937, 75.94349567784103, 76.21758380771662, 76.4846440368262, 76.77981586899992, 76.89929018202263, 77.14526670883407, 77.3561037318153, 77.58099655632861, 77.76372197624569, 77.89022419003443, 78.0378101061213, 78.31892613676294, 78.45948415208377, 78.53679106051023, 78.71248857966124, 78.9092698011104, 79.0779394194954, 79.23255323634831, 79.352027549371, 79.5066413662239, 79.63314358001264, 79.80884109916367, 79.94237121371846, 80.09698503057136, 80.24457094665823, 80.4624358704055, 80.63813338955653, 80.81383090870756, 80.9543889240284, 81.05277953475296, 81.04575163398692, 81.08791903858317, 81.03872373322089, 81.07386323705109, 81.06683533628505, 81.059807435519, 81.05277953475296, 81.04575163398692, 81.03169583245484, 81.03169583245484, 81.03169583245484, 81.03169583245484, 81.03169583245484, 81.0246679316888, 81.01764003092275, 81.01764003092275, 81.00358422939068, 80.98952842785859, 80.98250052709255, 80.96844472556047, 80.96141682479443, 80.94033312249631, 80.89816571790006, 80.89113781713402, 80.87708201560193, 80.85599831330381, 80.75760770257924, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.98784173167475, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99037177595053, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 56.97519151029587, 57.90990231217935, 58.760278304870326, 59.59659849602923, 60.446974488720215, 61.19895987068662, 61.95797315341907, 62.597512123128816, 63.2440789936046, 63.83442265795206, 64.53721273455618, 65.10647269660552, 65.67573265865485, 66.25904842223628, 66.76505727739124, 67.24295452948203, 67.73490758310491, 68.07927472064094, 68.50797666736945, 68.95073441563004, 69.33024105699627, 69.73083140066062, 70.07519853819663, 70.39848197343453, 70.77798861480076, 71.0239651416122, 71.28399746995572, 71.60728090519362, 71.92353643966547, 72.23276407337129, 72.53496380631105, 72.85824724154895, 73.0550284629981, 73.27992128751141, 73.54698151662097, 73.87026495185887, 74.19354838709677, 74.4887202192705, 74.75578044838007, 75.06500808208587, 75.33206831119544, 75.66237964719937, 75.94349567784103, 76.21758380771662, 76.4846440368262, 76.77981586899992, 76.89929018202263, 77.14526670883407, 77.3561037318153, 77.58099655632861, 77.76372197624569, 77.89022419003443, 78.0378101061213, 78.31892613676294, 78.45948415208377, 78.53679106051023, 78.71248857966124, 78.9092698011104, 79.0779394194954, 79.23255323634831, 79.352027549371, 79.5066413662239, 79.63314358001264, 79.80884109916367, 79.94237121371846, 80.09698503057136, 80.24457094665823, 80.4624358704055, 80.63813338955653, 80.81383090870756, 80.9543889240284, 81.05277953475296, 81.04575163398692, 81.08791903858317, 81.03872373322089, 81.07386323705109, 81.06683533628505, 81.059807435519, 81.05277953475296, 81.04575163398692, 81.03169583245484, 81.03169583245484, 81.03169583245484, 81.03169583245484, 81.03169583245484, 81.0246679316888, 81.01764003092275, 81.01764003092275, 81.00358422939068, 80.98952842785859, 80.98250052709255, 80.96844472556047, 80.96141682479443, 80.94033312249631, 80.89816571790006, 80.89113781713402, 80.87708201560193, 80.85599831330381, 80.75760770257924, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 64.68269028041324, "exact_area100": 74.89488314641879}, "test": {"score": [26.688029946402747, 27.839378385276355, 29.223265178799313, 30.672376144967817, 32.30014462751326, 33.74074809290191, 35.15866488954428, 36.35822249950373, 37.85270680316479, 38.99271190766526, 40.413464537900914, 41.695261322065626, 42.97989393982361, 44.20213821852934, 45.132291637126734, 46.09363922525026, 47.08050931571336, 48.05603607180331, 49.247086180982905, 50.19992626832658, 50.98828800725974, 51.83903808524516, 52.52814564841335, 53.239939880327825, 53.95456994583559, 54.62382667385077, 55.36965090888467, 56.030400136120015, 56.70532853132178, 57.30936108669143, 58.03817032016562, 58.71309871536739, 59.31145960355047, 59.99205966593881, 60.5138530471032, 61.03281059467432, 61.651022318010384, 62.158636531208344, 62.69177324674588, 63.19087995916399, 63.5453591583246, 64.09267504182854, 64.61730425658622, 65.11357513541105, 65.54462184159033, 66.0720868899413, 66.60238777188555, 67.02492697728498, 67.45880951705755, 67.85299038652413, 68.43150043955421, 68.93060715197232, 69.35031052377848, 69.76150639480477, 70.27479227518928, 70.61792813997675, 71.07733318208888, 71.44315571562261, 71.84867991946233, 72.22868162096248, 72.48390664435811, 72.7986841732127, 73.07659586535462, 73.3942092278025, 73.64376258401157, 73.87062927147436, 74.04077928707143, 74.23645180500809, 74.4009301534186, 74.52003516433656, 74.63630434166124, 74.74690185179934, 74.82630519241131, 74.863171029124, 74.90003686583671, 74.89152936505685, 74.91138020020985, 74.92272353458299, 74.90854436661657, 74.90287269943, 74.90287269943, 74.90570853302329, 74.90570853302329, 74.90287269943, 74.90570853302329, 74.90570853302329, 74.90287269943, 74.89720103224343, 74.89152936505685, 74.88585769787028, 74.87735019709044, 74.863171029124, 74.86033519553072, 74.85182769475087, 74.85182769475087, 74.8660068627173, 74.85182769475087, 74.83481269319117, 74.81496185803817, 74.76108101976577, 74.47182599325072], "calls": [0.0, 1.1683634404333154, 2.6373252417548136, 4.267929557893543, 6.094206391968919, 7.7219748745143635, 9.395116694552364, 10.864078495873862, 12.713041998695518, 14.079913790658763, 15.920369792700564, 17.664407452570682, 19.340385106201968, 21.01069109264668, 22.23860703853898, 23.5544338258231, 24.955335620905768, 26.353401582395147, 28.111618410231685, 29.577744377959903, 30.774466154326063, 32.01656126818479, 33.16223803987182, 34.25403397328645, 35.40254657856677, 36.49150667838811, 37.688228454754274, 38.79703938972861, 39.89450699032981, 41.0288404276437, 42.31630887899498, 43.54422482488727, 44.61900575674219, 45.77886169639566, 46.83379179309758, 47.917080225732356, 49.091115333352235, 50.06947792303548, 51.26052803221507, 52.309786461730425, 53.13501403737628, 54.26367580750362, 55.34980007373167, 56.34517766497462, 57.30936108669143, 58.39832118651278, 59.57235629413265, 60.50250971273006, 61.40714062898789, 62.35714488273828, 63.60491166378357, 64.74208093469075, 65.76014519467998, 66.74985111873634, 67.99194623259507, 68.97598048946489, 70.06210475569293, 70.98942234069705, 72.17763661628337, 73.22122337861214, 74.13436179564982, 74.95958937129569, 75.91526529223265, 76.8539262116099, 77.75004962708788, 78.6943822136517, 79.49125145336473, 80.52916654850694, 81.34872245696623, 82.24201003885092, 83.24589513087372, 84.09097354167258, 84.90769361653858, 85.63366701641948, 86.46456625925191, 87.27277883333807, 88.06397640586451, 88.95159232056264, 89.85622323682047, 90.75518248589172, 91.40175254516065, 91.95474009585118, 92.54742931684768, 93.03802852848595, 93.42370189717268, 93.79519609789297, 94.0702719564416, 94.46445282590817, 94.78490202194935, 95.17908289141593, 95.45699458355784, 95.75759294444602, 96.11207214360661, 96.5431188497859, 96.95431472081218, 97.42789893089073, 97.73416895896548, 98.21342483623062, 98.58208320335763, 99.14924992201458, 100.0], "exact_score": [26.688029946402747, 27.66922836967927, 28.616396789836372, 29.555057709213628, 30.434166123131895, 31.324617871423303, 32.20939795252814, 33.11402886878598, 33.99597311629754, 34.826872359129965, 35.63792076680941, 36.46882000964183, 37.28554008450784, 38.09658849218728, 38.93315940220628, 39.72152114113944, 40.48152454413975, 41.233020446360214, 41.94765051186796, 42.727504750021275, 43.44497064912231, 44.19646655134277, 44.94229078637666, 45.688115021410546, 46.388565918951876, 47.10319598445963, 47.78946771403454, 48.47290361001617, 49.15633950599779, 49.8426112355727, 50.48067379406177, 51.14993052207697, 51.82202308368545, 52.41754813827525, 53.05844653035761, 53.69367325525338, 54.30054164421632, 54.913081700365815, 55.534129257295184, 56.135325979071546, 56.77338853756062, 57.292346085131726, 57.85100530300882, 58.4493661911919, 58.9513087372033, 59.54399795819981, 60.099821342483615, 60.60459972208832, 61.06967643138701, 61.58579814536483, 62.116099027309076, 62.57266823582792, 63.017894109973625, 63.491478320052174, 63.976405864503874, 64.46416924254885, 64.95193262059382, 65.38297932677311, 65.88208603919122, 66.35567024926978, 66.80940362419534, 67.27731616708732, 67.70836287326661, 68.17060374897201, 68.59314295437144, 69.04404049570371, 69.45523636673, 69.8664322377563, 70.27762810878258, 70.62076397357004, 71.0546465133426, 71.44599154921589, 71.79479908118992, 72.14927828035052, 72.45838414201855, 72.81286334117914, 73.09077503332104, 73.43107506451521, 73.69764342228397, 73.91316677537363, 74.15704846439611, 74.35272098233277, 74.50018432918357, 74.62779684088137, 74.7355585174262, 74.82914102600459, 74.87451436349714, 74.90570853302329, 74.90570853302329, 74.9198877009897, 74.90854436661657, 74.90287269943, 74.90570853302329, 74.90287269943, 74.90287269943, 74.88585769787028, 74.863171029124, 74.85749936193744, 74.83481269319117, 74.76958852054561, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 42.378413634687924, "exact_area100": 58.69102713533274}}, "old_normalized_endpoint_budget": 74}, {"seed": 2029, "selection": {"best_epoch": 6, "epochs": 14, "alpha": 0.0, "validation_area100": 0.7559451019845999, "training_seconds": 100.93606583098881, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9875683188438416, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9808871150016785, "operator": "ge"}, {"budget": 0.03, "threshold": 0.974826455116272, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9684870839118958, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9612699747085571, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9536386728286743, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9454932808876038, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9377712607383728, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9296523928642273, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9211155772209167, "operator": "gt"}, {"budget": 0.11, "threshold": 0.9126267433166504, "operator": "ge"}, {"budget": 0.12, "threshold": 0.903630256652832, "operator": "ge"}, {"budget": 0.13, "threshold": 0.8949076533317566, "operator": "ge"}, {"budget": 0.14, "threshold": 0.8854262232780457, "operator": "ge"}, {"budget": 0.15, "threshold": 0.8747515678405762, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8650788068771362, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8555740118026733, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8467943072319031, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8369271159172058, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8259050250053406, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8154526948928833, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8062324523925781, "operator": "ge"}, {"budget": 0.23, "threshold": 0.795799195766449, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7833213806152344, "operator": "ge"}, {"budget": 0.25, "threshold": 0.7708060145378113, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7592283487319946, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7499667406082153, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7400203347206116, "operator": "ge"}, {"budget": 0.29, "threshold": 0.7311113476753235, "operator": "ge"}, {"budget": 0.3, "threshold": 0.7201755046844482, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7089675068855286, "operator": "ge"}, {"budget": 0.32, "threshold": 0.6983689665794373, "operator": "ge"}, {"budget": 0.33, "threshold": 0.6877073049545288, "operator": "ge"}, {"budget": 0.34, "threshold": 0.6757748126983643, "operator": "ge"}, {"budget": 0.35, "threshold": 0.6643661856651306, "operator": "ge"}, {"budget": 0.36, "threshold": 0.6524447202682495, "operator": "ge"}, {"budget": 0.37, "threshold": 0.6419010162353516, "operator": "ge"}, {"budget": 0.38, "threshold": 0.6301938891410828, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6187803745269775, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6083812713623047, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5967915058135986, "operator": "ge"}, {"budget": 0.42, "threshold": 0.585848867893219, "operator": "ge"}, {"budget": 0.43, "threshold": 0.5720805525779724, "operator": "gt"}, {"budget": 0.44, "threshold": 0.5570419430732727, "operator": "ge"}, {"budget": 0.45, "threshold": 0.5446390509605408, "operator": "ge"}, {"budget": 0.46, "threshold": 0.5330154299736023, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5197655558586121, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5044548511505127, "operator": "ge"}, {"budget": 0.49, "threshold": 0.4901292622089386, "operator": "ge"}, {"budget": 0.5, "threshold": 0.47427722811698914, "operator": "gt"}, {"budget": 0.51, "threshold": 0.45847707986831665, "operator": "ge"}, {"budget": 0.52, "threshold": 0.4435591399669647, "operator": "ge"}, {"budget": 0.53, "threshold": 0.43161648511886597, "operator": "ge"}, {"budget": 0.54, "threshold": 0.4101560115814209, "operator": "ge"}, {"budget": 0.55, "threshold": 0.39175093173980713, "operator": "ge"}, {"budget": 0.56, "threshold": 0.37282711267471313, "operator": "ge"}, {"budget": 0.57, "threshold": 0.3518920838832855, "operator": "ge"}, {"budget": 0.58, "threshold": 0.32810187339782715, "operator": "gt"}, {"budget": 0.59, "threshold": 0.3026052415370941, "operator": "ge"}, {"budget": 0.6, "threshold": 0.2732238471508026, "operator": "ge"}, {"budget": 0.61, "threshold": 0.24377943575382233, "operator": "ge"}, {"budget": 0.62, "threshold": 0.2165258377790451, "operator": "ge"}, {"budget": 0.63, "threshold": 0.1831938922405243, "operator": "ge"}, {"budget": 0.64, "threshold": 0.15286245942115784, "operator": "ge"}, {"budget": 0.65, "threshold": 0.12120598554611206, "operator": "ge"}, {"budget": 0.66, "threshold": 0.09190155565738678, "operator": "ge"}, {"budget": 0.67, "threshold": 0.06722721457481384, "operator": "ge"}, {"budget": 0.68, "threshold": 0.04430210962891579, "operator": "ge"}, {"budget": 0.69, "threshold": 0.02323049306869507, "operator": "ge"}, {"budget": 0.7, "threshold": 0.006556786596775055, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0009213460143655539, "operator": "ge"}, {"budget": 0.72, "threshold": -0.002615367528051138, "operator": "ge"}, {"budget": 0.73, "threshold": -0.004329673945903778, "operator": "ge"}, {"budget": 0.74, "threshold": -0.005830397829413414, "operator": "ge"}, {"budget": 0.75, "threshold": -0.007402407471090555, "operator": "ge"}, {"budget": 0.76, "threshold": -0.009151192381978035, "operator": "ge"}, {"budget": 0.77, "threshold": -0.01103814598172903, "operator": "ge"}, {"budget": 0.78, "threshold": -0.013571416959166527, "operator": "ge"}, {"budget": 0.79, "threshold": -0.016011450439691544, "operator": "ge"}, {"budget": 0.8, "threshold": -0.01820778101682663, "operator": "ge"}, {"budget": 0.81, "threshold": -0.020859302952885628, "operator": "ge"}, {"budget": 0.82, "threshold": -0.02398671768605709, "operator": "ge"}, {"budget": 0.83, "threshold": -0.026836123317480087, "operator": "ge"}, {"budget": 0.84, "threshold": -0.030111171305179596, "operator": "ge"}, {"budget": 0.85, "threshold": -0.03387557342648506, "operator": "ge"}, {"budget": 0.86, "threshold": -0.038466233760118484, "operator": "ge"}, {"budget": 0.87, "threshold": -0.043386176228523254, "operator": "ge"}, {"budget": 0.88, "threshold": -0.048148758709430695, "operator": "ge"}, {"budget": 0.89, "threshold": -0.05432625114917755, "operator": "ge"}, {"budget": 0.9, "threshold": -0.0616065189242363, "operator": "ge"}, {"budget": 0.91, "threshold": -0.0698748230934143, "operator": "ge"}, {"budget": 0.92, "threshold": -0.07953815907239914, "operator": "ge"}, {"budget": 0.93, "threshold": -0.09021913260221481, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10347466170787811, "operator": "ge"}, {"budget": 0.95, "threshold": -0.12264236062765121, "operator": "ge"}, {"budget": 0.96, "threshold": -0.14538948237895966, "operator": "ge"}, {"budget": 0.97, "threshold": -0.18343040347099304, "operator": "ge"}, {"budget": 0.98, "threshold": -0.22193756699562073, "operator": "ge"}, {"budget": 0.99, "threshold": -0.3011038303375244, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.01033101412608, 57.874762808349146, 58.71811090027408, 59.62471009909339, 60.376695481059805, 61.13570876379225, 61.90174994729074, 62.611567924660896, 63.30030219973294, 63.93984116944268, 64.60046384145055, 65.17675170426594, 65.62653735329256, 66.1606578115117, 66.69477826973082, 67.27106613254621, 67.74896338463701, 68.14955372830134, 68.59231147656195, 69.04209712558857, 69.37943636235856, 69.89247311827957, 70.27900766041184, 70.6655422025441, 71.00288143931407, 71.31210907301988, 71.73378311898236, 72.21168037107316, 72.6122707147375, 72.99177735610373, 73.31506079134162, 73.61023262351534, 73.891348654157, 74.20760418862885, 74.49574812003654, 74.74172464684798, 75.0368964790217, 75.33909621196149, 75.66940754796542, 75.99971888396935, 76.33003021997328, 76.57600674678473, 76.86415067819242, 77.16635041113219, 77.44746644177384, 77.74966617471361, 77.98861480075901, 78.22053552603838, 78.5157073582121, 78.83196289268395, 79.1130789233256, 79.29580434324266, 79.5909761754164, 79.76667369456743, 79.92128751142033, 80.11806873286949, 80.37107316044697, 80.58893808419425, 80.82788671023965, 80.98250052709255, 80.97547262632652, 81.11603064164734, 81.18630964930776, 81.2144212523719, 81.2987560615644, 81.44634197765127, 81.57284419144003, 81.67123480216458, 81.80476491671938, 81.90315552744394, 81.87504392437978, 81.88207182514581, 81.86801602361375, 81.8609881228477, 81.86801602361375, 81.85396022208167, 81.83287651978354, 81.82584861901749, 81.81882071825146, 81.81179281748541, 81.81179281748541, 81.79070911518728, 81.79070911518728, 81.79070911518728, 81.76259751212312, 81.73448590905896, 81.70637430599479, 81.63609529833438, 81.58689999297209, 81.56581629067396, 81.47445358071543, 81.43931407688522, 81.41120247382106, 81.36903506922482, 81.27767235926629, 81.18630964930776, 81.059807435519, 80.92627732096422, 80.79274720640944, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.986646988544521, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.98966898587392, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.98945814885094, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.987209220605806, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 139, 267, 391, 524, 638, 755, 870, 979, 1084, 1186, 1291, 1384, 1468, 1557, 1644, 1736, 1814, 1885, 1958, 2039, 2105, 2192, 2264, 2339, 2405, 2469, 2543, 2624, 2693, 2763, 2823, 2883, 2939, 3002, 3061, 3119, 3179, 3243, 3303, 3365, 3427, 3477, 3532, 3588, 3642, 3698, 3747, 3795, 3848, 3905, 3960, 4003, 4060, 4102, 4141, 4180, 4227, 4271, 4318, 4352, 4369, 4398, 4422, 4443, 4470, 4502, 4529, 4556, 4582, 4601, 4605, 4609, 4612, 4612, 4613, 4614, 4614, 4614, 4615, 4616, 4619, 4619, 4619, 4621, 4621, 4621, 4621, 4621, 4621, 4621, 4621, 4621, 4623, 4624, 4625, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 3, 8, 12, 16, 23, 32, 38, 46, 53, 64, 75, 86, 106, 119, 130, 140, 150, 164, 174, 191, 209, 223, 240, 260, 278, 298, 312, 325, 337, 353, 367, 385, 401, 419, 437, 460, 478, 499, 512, 527, 542, 557, 571, 584, 598, 611, 626, 641, 652, 664, 679, 696, 711, 728, 745, 756, 767, 780, 793, 805, 823, 832, 846, 863, 878, 889, 898, 911, 918, 923, 931, 934, 939, 940, 940, 943, 946, 947, 949, 951, 954, 957, 957, 959, 963, 967, 971, 981, 988, 991, 1004, 1009, 1015, 1022, 1036, 1050, 1068, 1087, 1106, 1133], "exact_score": [56.05453650994448, 57.01033101412608, 57.874762808349146, 58.71811090027408, 59.62471009909339, 60.376695481059805, 61.13570876379225, 61.90174994729074, 62.611567924660896, 63.30030219973294, 63.94686907020872, 64.60046384145055, 65.17675170426594, 65.62653735329256, 66.1606578115117, 66.69477826973082, 67.27106613254621, 67.74896338463701, 68.14955372830134, 68.59231147656195, 69.04209712558857, 69.37943636235856, 69.89247311827957, 70.27900766041184, 70.6655422025441, 71.00288143931407, 71.31210907301988, 71.73378311898236, 72.21168037107316, 72.6122707147375, 72.99177735610373, 73.31506079134162, 73.61023262351534, 73.891348654157, 74.20760418862885, 74.49574812003654, 74.74172464684798, 75.0368964790217, 75.33909621196149, 75.66940754796542, 75.99971888396935, 76.33003021997328, 76.57600674678473, 76.87117857895846, 77.16635041113219, 77.44746644177384, 77.74966617471361, 77.98861480075901, 78.22053552603838, 78.5157073582121, 78.83899079345, 79.1130789233256, 79.29580434324266, 79.5909761754164, 79.76667369456743, 79.92128751142033, 80.11806873286949, 80.37107316044697, 80.59596598496029, 80.82788671023965, 80.98250052709255, 80.97547262632652, 81.11603064164734, 81.18630964930776, 81.2144212523719, 81.2987560615644, 81.44634197765127, 81.57284419144003, 81.67123480216458, 81.80476491671938, 81.90315552744394, 81.87504392437978, 81.88207182514581, 81.86801602361375, 81.8609881228477, 81.86801602361375, 81.85396022208167, 81.83287651978354, 81.82584861901749, 81.81882071825146, 81.81179281748541, 81.81179281748541, 81.79070911518728, 81.79070911518728, 81.79070911518728, 81.76259751212312, 81.73448590905896, 81.70637430599479, 81.63609529833438, 81.58689999297209, 81.56581629067396, 81.47445358071543, 81.43931407688522, 81.41120247382106, 81.36903506922482, 81.27767235926629, 81.18630964930776, 81.059807435519, 80.92627732096422, 80.79274720640944, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.59451019845999, "threshold_area100": 75.59340080118068, "threshold_area25": 64.71642420409023}, "test": {"score": [26.688029946402747, 28.23639508833622, 29.78476023026969, 31.25939369877776, 32.589399654028306, 33.94209227802513, 35.46777075121233, 36.780761704903156, 37.980319314862605, 39.310325270113154, 40.5098828800726, 41.75197799393132, 42.727504750021275, 43.81362901624933, 44.93094745200352, 46.15319173070924, 47.148569321952195, 48.06454357258316, 48.86708447948275, 49.683804554348754, 50.74157048464396, 51.609335564189095, 52.40336897030882, 53.174715707682275, 54.08785412471997, 54.92726086833225, 55.67875677055271, 56.26861015795593, 56.861299378952445, 57.436973598389244, 58.01548365141933, 58.622352040382275, 59.27459376683777, 59.85877548705443, 60.45713637523751, 61.038482261860885, 61.59997731333126, 62.118934860902364, 62.67475824518618, 63.23908913024984, 63.678643337208975, 64.29118339335848, 64.69387176360492, 65.32342682131413, 65.83387686810538, 66.27343107506451, 66.6846269460908, 67.20358449366192, 67.71119870685989, 68.30105209426311, 68.83418880980065, 69.35031052377848, 69.80120806511074, 70.1415080963049, 70.61509230638346, 71.06882568130902, 71.46017071718231, 71.94509826163402, 72.39032413577972, 72.7476391685336, 73.12764087003374, 73.48495590278763, 73.83092760116836, 74.1031676261237, 74.38391515185889, 74.57675183620225, 74.80645435725832, 74.93123103536284, 75.06735104784052, 75.15242605563905, 75.21197856109804, 75.24317273062418, 75.2375010634376, 75.24033689703089, 75.24600856421746, 75.25451606499732, 75.26585939937046, 75.2743669001503, 75.28003856733687, 75.29138190171001, 75.28571023452344, 75.28571023452344, 75.28287440093015, 75.28003856733687, 75.28571023452344, 75.2772027337436, 75.2573518985906, 75.24600856421746, 75.2375010634376, 75.24884439781074, 75.23466522984432, 75.22332189547117, 75.21765022828461, 75.20630689391147, 75.19496355953834, 75.12123188611292, 75.07869438221365, 75.02197771034795, 74.96242520488899, 74.83764852678445, 74.47182599325072], "calls": [0.0, 1.630604316138729, 3.2782236338371664, 4.849275444516916, 6.3721180841108245, 7.960184896350282, 9.743924226526387, 11.357513541105407, 12.817967841647052, 14.47125882653206, 15.979922298159543, 17.528287440093017, 18.980234239854806, 20.4123302044636, 21.957859512803786, 23.63383716643507, 25.00638062558489, 26.33071491364887, 27.518929189235173, 28.749680968720753, 30.229986104415392, 31.50043955420696, 32.736863000879104, 33.888211439752716, 35.30612823639509, 36.624790857272494, 37.86688597113121, 38.93315940220628, 39.942716161415646, 40.969287922184726, 42.02138218529336, 43.17840229135355, 44.21348155290248, 45.293934151943965, 46.39707341973173, 47.426481014094094, 48.52678444828857, 49.53917704109123, 50.65082380965885, 51.69157473839435, 52.644414825738025, 53.7645690950855, 54.63517000822392, 55.81487678303038, 56.969061055497264, 57.94458781158721, 58.79533788957264, 59.87011882142755, 61.012959759521316, 62.218189036667326, 63.38938831069393, 64.57476675268695, 65.64387601735531, 66.42940192269518, 67.77075121231886, 68.8965771488529, 69.94299974477498, 70.98658650710377, 72.19465161784306, 73.26943254969798, 74.41510932138502, 75.57780109463177, 76.57317868587471, 77.67631795366248, 78.66885971131214, 79.61035646428267, 80.55468905084649, 81.39976746164535, 82.29021920993677, 83.22320846212745, 84.14201854635171, 85.39262116099027, 86.09590789212488, 86.6885971131214, 87.20755466069251, 87.83710971840173, 88.40711227065195, 88.97144315571562, 89.66055071888381, 90.31279244533931, 90.73816748433202, 91.20324419363072, 91.71653007401525, 92.1730992825341, 92.52757848169469, 92.9160876839747, 93.38967189405327, 93.90012194084451, 94.35952698295664, 94.78206618835607, 95.21027706094206, 95.61012959759522, 95.9844596319088, 96.37013300059553, 96.74446303490912, 97.24356974732723, 97.61789978164082, 98.09431982531265, 98.46864985962624, 99.05566741343618, 100.0], "rescued": [0, 554, 1112, 1639, 2125, 2623, 3187, 3680, 4129, 4628, 5075, 5537, 5932, 6348, 6778, 7241, 7614, 7967, 8281, 8607, 9010, 9339, 9650, 9947, 10305, 10628, 10920, 11155, 11389, 11613, 11843, 12086, 12339, 12564, 12800, 13026, 13249, 13451, 13670, 13885, 14065, 14300, 14462, 14700, 14909, 15081, 15247, 15448, 15649, 15880, 16090, 16288, 16466, 16595, 16788, 16963, 17116, 17294, 17468, 17612, 17766, 17911, 18048, 18159, 18268, 18359, 18457, 18514, 18576, 18624, 18660, 18680, 18683, 18685, 18687, 18691, 18696, 18699, 18702, 18707, 18708, 18710, 18715, 18719, 18723, 18726, 18727, 18729, 18737, 18745, 18748, 18753, 18757, 18763, 18768, 18775, 18780, 18790, 18796, 18809, 18820], "harmed": [0, 8, 20, 27, 44, 65, 91, 121, 147, 177, 201, 225, 276, 309, 345, 377, 399, 429, 460, 498, 528, 551, 582, 607, 643, 670, 697, 724, 749, 770, 796, 825, 848, 867, 892, 913, 938, 957, 980, 996, 1021, 1040, 1060, 1076, 1105, 1122, 1143, 1161, 1183, 1206, 1228, 1244, 1263, 1272, 1298, 1313, 1328, 1335, 1352, 1370, 1390, 1409, 1424, 1439, 1449, 1472, 1489, 1502, 1516, 1534, 1549, 1558, 1563, 1564, 1564, 1565, 1566, 1566, 1567, 1568, 1571, 1573, 1579, 1584, 1586, 1592, 1600, 1606, 1617, 1621, 1629, 1638, 1644, 1654, 1663, 1696, 1716, 1746, 1773, 1830, 1970], "exact_score": [26.688029946402747, 27.64370586733971, 28.60221762186995, 29.532371040467346, 30.468196126251314, 31.406857045628566, 32.257607123613994, 33.11402886878598, 33.97045061395797, 34.86657402943595, 35.67478660352211, 36.4943425119814, 37.25434591498171, 38.116439327340274, 38.941666902986135, 39.72435697473272, 40.52973371522559, 41.33511045571846, 42.07809885715906, 42.736012250801124, 43.51019482176786, 44.2276607208689, 44.9536341207498, 45.71930919093668, 46.40841675410487, 47.14289765476562, 47.817826049967394, 48.53529194906842, 49.190369509117204, 49.88514873947197, 50.59410713779315, 51.26903553299492, 51.96097892975639, 52.576354819499194, 53.25411904829424, 53.88083827241017, 54.518900830899256, 55.17965005813459, 55.74965261038483, 56.30831182826193, 56.90100104925844, 57.451152766355676, 57.998468649859625, 58.52309786461731, 59.14698125513995, 59.688625471457335, 60.252956356521004, 60.7747497376854, 61.31639395400278, 61.860874003913445, 62.33445821399201, 62.87043076312282, 63.352522473981224, 63.90267419107847, 64.38760173553015, 64.87820094716842, 65.39715849473953, 65.83954853529195, 66.30746107818393, 66.7782094546692, 67.24895783115447, 67.71119870685989, 68.15926041459888, 68.65836712701699, 69.1035930011627, 69.52613220656212, 69.95150724555484, 70.36553895017441, 70.72285398292829, 71.11419901880159, 71.49136488670844, 71.94509826163402, 72.33077163032074, 72.65122082636191, 72.98017752318295, 73.31764172078383, 73.64092675041829, 73.92451010974676, 74.19107846751552, 74.454810991691, 74.65899101040752, 74.88869353146357, 75.02764937753452, 75.13824688767264, 75.19779939313162, 75.24884439781074, 75.23466522984432, 75.24317273062418, 75.26302356577716, 75.2743669001503, 75.28854606811673, 75.28854606811673, 75.28571023452344, 75.2743669001503, 75.24033689703089, 75.22899356265775, 75.21765022828461, 75.15526188923234, 75.03899271190765, 74.84899186115759, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.945711819962156, "threshold_area100": 63.2781669171653, "threshold_area25": 42.43087655616369}, "selected_budget": 70, "selected": {"val": {"score": 81.90315552744394, "calls": 69.9978916297702, "rescued": 4601, "harmed": 923}, "test": {"score": 75.21197856109804, "calls": 84.14201854635171, "rescued": 18660, "harmed": 1549}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9875683188438416, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9808871150016785, "operator": "ge"}, {"budget": 0.03, "threshold": 0.974826455116272, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9684870839118958, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9612699747085571, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9536386728286743, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9454932808876038, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9377712607383728, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9296523928642273, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9211155772209167, "operator": "gt"}, {"budget": 0.11, "threshold": 0.9126267433166504, "operator": "ge"}, {"budget": 0.12, "threshold": 0.903630256652832, "operator": "ge"}, {"budget": 0.13, "threshold": 0.8949076533317566, "operator": "ge"}, {"budget": 0.14, "threshold": 0.8854262232780457, "operator": "ge"}, {"budget": 0.15, "threshold": 0.8747515678405762, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8650788068771362, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8555740118026733, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8467943072319031, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8369271159172058, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8259050250053406, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8154526948928833, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8062324523925781, "operator": "ge"}, {"budget": 0.23, "threshold": 0.795799195766449, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7833213806152344, "operator": "ge"}, {"budget": 0.25, "threshold": 0.7708060145378113, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7592283487319946, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7499667406082153, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7400203347206116, "operator": "ge"}, {"budget": 0.29, "threshold": 0.7311113476753235, "operator": "ge"}, {"budget": 0.3, "threshold": 0.7201755046844482, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7089675068855286, "operator": "ge"}, {"budget": 0.32, "threshold": 0.6983689665794373, "operator": "ge"}, {"budget": 0.33, "threshold": 0.6877073049545288, "operator": "ge"}, {"budget": 0.34, "threshold": 0.6757748126983643, "operator": "ge"}, {"budget": 0.35, "threshold": 0.6643661856651306, "operator": "ge"}, {"budget": 0.36, "threshold": 0.6524447202682495, "operator": "ge"}, {"budget": 0.37, "threshold": 0.6419010162353516, "operator": "ge"}, {"budget": 0.38, "threshold": 0.6301938891410828, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6187803745269775, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6083812713623047, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5967915058135986, "operator": "ge"}, {"budget": 0.42, "threshold": 0.585848867893219, "operator": "ge"}, {"budget": 0.43, "threshold": 0.5720805525779724, "operator": "gt"}, {"budget": 0.44, "threshold": 0.5570419430732727, "operator": "ge"}, {"budget": 0.45, "threshold": 0.5446390509605408, "operator": "ge"}, {"budget": 0.46, "threshold": 0.5330154299736023, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5197655558586121, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5044548511505127, "operator": "ge"}, {"budget": 0.49, "threshold": 0.4901292622089386, "operator": "ge"}, {"budget": 0.5, "threshold": 0.47427722811698914, "operator": "gt"}, {"budget": 0.51, "threshold": 0.45847707986831665, "operator": "ge"}, {"budget": 0.52, "threshold": 0.4435591399669647, "operator": "ge"}, {"budget": 0.53, "threshold": 0.43161648511886597, "operator": "ge"}, {"budget": 0.54, "threshold": 0.4101560115814209, "operator": "ge"}, {"budget": 0.55, "threshold": 0.39175093173980713, "operator": "ge"}, {"budget": 0.56, "threshold": 0.37282711267471313, "operator": "ge"}, {"budget": 0.57, "threshold": 0.3518920838832855, "operator": "ge"}, {"budget": 0.58, "threshold": 0.32810187339782715, "operator": "gt"}, {"budget": 0.59, "threshold": 0.3026052415370941, "operator": "ge"}, {"budget": 0.6, "threshold": 0.2732238471508026, "operator": "ge"}, {"budget": 0.61, "threshold": 0.24377943575382233, "operator": "ge"}, {"budget": 0.62, "threshold": 0.2165258377790451, "operator": "ge"}, {"budget": 0.63, "threshold": 0.1831938922405243, "operator": "ge"}, {"budget": 0.64, "threshold": 0.15286245942115784, "operator": "ge"}, {"budget": 0.65, "threshold": 0.12120598554611206, "operator": "ge"}, {"budget": 0.66, "threshold": 0.09190155565738678, "operator": "ge"}, {"budget": 0.67, "threshold": 0.06722721457481384, "operator": "ge"}, {"budget": 0.68, "threshold": 0.04430210962891579, "operator": "ge"}, {"budget": 0.69, "threshold": 0.02323049306869507, "operator": "ge"}, {"budget": 0.7, "threshold": 0.006556786596775055, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0009213460143655539, "operator": "ge"}, {"budget": 0.72, "threshold": -0.002615367528051138, "operator": "ge"}, {"budget": 0.73, "threshold": -0.004329673945903778, "operator": "ge"}, {"budget": 0.74, "threshold": -0.005830397829413414, "operator": "ge"}, {"budget": 0.75, "threshold": -0.007402407471090555, "operator": "ge"}, {"budget": 0.76, "threshold": -0.009151192381978035, "operator": "ge"}, {"budget": 0.77, "threshold": -0.01103814598172903, "operator": "ge"}, {"budget": 0.78, "threshold": -0.013571416959166527, "operator": "ge"}, {"budget": 0.79, "threshold": -0.016011450439691544, "operator": "ge"}, {"budget": 0.8, "threshold": -0.01820778101682663, "operator": "ge"}, {"budget": 0.81, "threshold": -0.020859302952885628, "operator": "ge"}, {"budget": 0.82, "threshold": -0.02398671768605709, "operator": "ge"}, {"budget": 0.83, "threshold": -0.026836123317480087, "operator": "ge"}, {"budget": 0.84, "threshold": -0.030111171305179596, "operator": "ge"}, {"budget": 0.85, "threshold": -0.03387557342648506, "operator": "ge"}, {"budget": 0.86, "threshold": -0.038466233760118484, "operator": "ge"}, {"budget": 0.87, "threshold": -0.043386176228523254, "operator": "ge"}, {"budget": 0.88, "threshold": -0.048148758709430695, "operator": "ge"}, {"budget": 0.89, "threshold": -0.05432625114917755, "operator": "ge"}, {"budget": 0.9, "threshold": -0.0616065189242363, "operator": "ge"}, {"budget": 0.91, "threshold": -0.0698748230934143, "operator": "ge"}, {"budget": 0.92, "threshold": -0.07953815907239914, "operator": "ge"}, {"budget": 0.93, "threshold": -0.09021913260221481, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10347466170787811, "operator": "ge"}, {"budget": 0.95, "threshold": -0.12264236062765121, "operator": "ge"}, {"budget": 0.96, "threshold": -0.14538948237895966, "operator": "ge"}, {"budget": 0.97, "threshold": -0.18343040347099304, "operator": "ge"}, {"budget": 0.98, "threshold": -0.22193756699562073, "operator": "ge"}, {"budget": 0.99, "threshold": -0.3011038303375244, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.01033101412608, 57.874762808349146, 58.71811090027408, 59.62471009909339, 60.376695481059805, 61.13570876379225, 61.90174994729074, 62.611567924660896, 63.30030219973294, 63.93984116944268, 64.60046384145055, 65.17675170426594, 65.62653735329256, 66.1606578115117, 66.69477826973082, 67.27106613254621, 67.74896338463701, 68.14955372830134, 68.59231147656195, 69.04209712558857, 69.37943636235856, 69.89247311827957, 70.27900766041184, 70.6655422025441, 71.00288143931407, 71.31210907301988, 71.73378311898236, 72.21168037107316, 72.6122707147375, 72.99177735610373, 73.31506079134162, 73.61023262351534, 73.891348654157, 74.20760418862885, 74.49574812003654, 74.74172464684798, 75.0368964790217, 75.33909621196149, 75.66940754796542, 75.99971888396935, 76.33003021997328, 76.57600674678473, 76.86415067819242, 77.16635041113219, 77.44746644177384, 77.74966617471361, 77.98861480075901, 78.22053552603838, 78.5157073582121, 78.83196289268395, 79.1130789233256, 79.29580434324266, 79.5909761754164, 79.76667369456743, 79.92128751142033, 80.11806873286949, 80.37107316044697, 80.58893808419425, 80.82788671023965, 80.98250052709255, 80.97547262632652, 81.11603064164734, 81.18630964930776, 81.2144212523719, 81.2987560615644, 81.44634197765127, 81.57284419144003, 81.67123480216458, 81.80476491671938, 81.90315552744394, 81.87504392437978, 81.88207182514581, 81.86801602361375, 81.8609881228477, 81.86801602361375, 81.85396022208167, 81.83287651978354, 81.82584861901749, 81.81882071825146, 81.81179281748541, 81.81179281748541, 81.79070911518728, 81.79070911518728, 81.79070911518728, 81.76259751212312, 81.73448590905896, 81.70637430599479, 81.63609529833438, 81.58689999297209, 81.56581629067396, 81.47445358071543, 81.43931407688522, 81.41120247382106, 81.36903506922482, 81.27767235926629, 81.18630964930776, 81.059807435519, 80.92627732096422, 80.79274720640944, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.986646988544521, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.98966898587392, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.98945814885094, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.987209220605806, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 139, 267, 391, 524, 638, 755, 870, 979, 1084, 1186, 1291, 1384, 1468, 1557, 1644, 1736, 1814, 1885, 1958, 2039, 2105, 2192, 2264, 2339, 2405, 2469, 2543, 2624, 2693, 2763, 2823, 2883, 2939, 3002, 3061, 3119, 3179, 3243, 3303, 3365, 3427, 3477, 3532, 3588, 3642, 3698, 3747, 3795, 3848, 3905, 3960, 4003, 4060, 4102, 4141, 4180, 4227, 4271, 4318, 4352, 4369, 4398, 4422, 4443, 4470, 4502, 4529, 4556, 4582, 4601, 4605, 4609, 4612, 4612, 4613, 4614, 4614, 4614, 4615, 4616, 4619, 4619, 4619, 4621, 4621, 4621, 4621, 4621, 4621, 4621, 4621, 4621, 4623, 4624, 4625, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 3, 8, 12, 16, 23, 32, 38, 46, 53, 64, 75, 86, 106, 119, 130, 140, 150, 164, 174, 191, 209, 223, 240, 260, 278, 298, 312, 325, 337, 353, 367, 385, 401, 419, 437, 460, 478, 499, 512, 527, 542, 557, 571, 584, 598, 611, 626, 641, 652, 664, 679, 696, 711, 728, 745, 756, 767, 780, 793, 805, 823, 832, 846, 863, 878, 889, 898, 911, 918, 923, 931, 934, 939, 940, 940, 943, 946, 947, 949, 951, 954, 957, 957, 959, 963, 967, 971, 981, 988, 991, 1004, 1009, 1015, 1022, 1036, 1050, 1068, 1087, 1106, 1133], "exact_score": [56.05453650994448, 57.01033101412608, 57.874762808349146, 58.71811090027408, 59.62471009909339, 60.376695481059805, 61.13570876379225, 61.90174994729074, 62.611567924660896, 63.30030219973294, 63.94686907020872, 64.60046384145055, 65.17675170426594, 65.62653735329256, 66.1606578115117, 66.69477826973082, 67.27106613254621, 67.74896338463701, 68.14955372830134, 68.59231147656195, 69.04209712558857, 69.37943636235856, 69.89247311827957, 70.27900766041184, 70.6655422025441, 71.00288143931407, 71.31210907301988, 71.73378311898236, 72.21168037107316, 72.6122707147375, 72.99177735610373, 73.31506079134162, 73.61023262351534, 73.891348654157, 74.20760418862885, 74.49574812003654, 74.74172464684798, 75.0368964790217, 75.33909621196149, 75.66940754796542, 75.99971888396935, 76.33003021997328, 76.57600674678473, 76.87117857895846, 77.16635041113219, 77.44746644177384, 77.74966617471361, 77.98861480075901, 78.22053552603838, 78.5157073582121, 78.83899079345, 79.1130789233256, 79.29580434324266, 79.5909761754164, 79.76667369456743, 79.92128751142033, 80.11806873286949, 80.37107316044697, 80.59596598496029, 80.82788671023965, 80.98250052709255, 80.97547262632652, 81.11603064164734, 81.18630964930776, 81.2144212523719, 81.2987560615644, 81.44634197765127, 81.57284419144003, 81.67123480216458, 81.80476491671938, 81.90315552744394, 81.87504392437978, 81.88207182514581, 81.86801602361375, 81.8609881228477, 81.86801602361375, 81.85396022208167, 81.83287651978354, 81.82584861901749, 81.81882071825146, 81.81179281748541, 81.81179281748541, 81.79070911518728, 81.79070911518728, 81.79070911518728, 81.76259751212312, 81.73448590905896, 81.70637430599479, 81.63609529833438, 81.58689999297209, 81.56581629067396, 81.47445358071543, 81.43931407688522, 81.41120247382106, 81.36903506922482, 81.27767235926629, 81.18630964930776, 81.059807435519, 80.92627732096422, 80.79274720640944, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.59451019845999, "threshold_area100": 75.59340080118068, "threshold_area25": 64.71642420409023}, "test": {"score": [26.688029946402747, 28.23639508833622, 29.78476023026969, 31.25939369877776, 32.589399654028306, 33.94209227802513, 35.46777075121233, 36.780761704903156, 37.980319314862605, 39.310325270113154, 40.5098828800726, 41.75197799393132, 42.727504750021275, 43.81362901624933, 44.93094745200352, 46.15319173070924, 47.148569321952195, 48.06454357258316, 48.86708447948275, 49.683804554348754, 50.74157048464396, 51.609335564189095, 52.40336897030882, 53.174715707682275, 54.08785412471997, 54.92726086833225, 55.67875677055271, 56.26861015795593, 56.861299378952445, 57.436973598389244, 58.01548365141933, 58.622352040382275, 59.27459376683777, 59.85877548705443, 60.45713637523751, 61.038482261860885, 61.59997731333126, 62.118934860902364, 62.67475824518618, 63.23908913024984, 63.678643337208975, 64.29118339335848, 64.69387176360492, 65.32342682131413, 65.83387686810538, 66.27343107506451, 66.6846269460908, 67.20358449366192, 67.71119870685989, 68.30105209426311, 68.83418880980065, 69.35031052377848, 69.80120806511074, 70.1415080963049, 70.61509230638346, 71.06882568130902, 71.46017071718231, 71.94509826163402, 72.39032413577972, 72.7476391685336, 73.12764087003374, 73.48495590278763, 73.83092760116836, 74.1031676261237, 74.38391515185889, 74.57675183620225, 74.80645435725832, 74.93123103536284, 75.06735104784052, 75.15242605563905, 75.21197856109804, 75.24317273062418, 75.2375010634376, 75.24033689703089, 75.24600856421746, 75.25451606499732, 75.26585939937046, 75.2743669001503, 75.28003856733687, 75.29138190171001, 75.28571023452344, 75.28571023452344, 75.28287440093015, 75.28003856733687, 75.28571023452344, 75.2772027337436, 75.2573518985906, 75.24600856421746, 75.2375010634376, 75.24884439781074, 75.23466522984432, 75.22332189547117, 75.21765022828461, 75.20630689391147, 75.19496355953834, 75.12123188611292, 75.07869438221365, 75.02197771034795, 74.96242520488899, 74.83764852678445, 74.47182599325072], "calls": [0.0, 1.630604316138729, 3.2782236338371664, 4.849275444516916, 6.3721180841108245, 7.960184896350282, 9.743924226526387, 11.357513541105407, 12.817967841647052, 14.47125882653206, 15.979922298159543, 17.528287440093017, 18.980234239854806, 20.4123302044636, 21.957859512803786, 23.63383716643507, 25.00638062558489, 26.33071491364887, 27.518929189235173, 28.749680968720753, 30.229986104415392, 31.50043955420696, 32.736863000879104, 33.888211439752716, 35.30612823639509, 36.624790857272494, 37.86688597113121, 38.93315940220628, 39.942716161415646, 40.969287922184726, 42.02138218529336, 43.17840229135355, 44.21348155290248, 45.293934151943965, 46.39707341973173, 47.426481014094094, 48.52678444828857, 49.53917704109123, 50.65082380965885, 51.69157473839435, 52.644414825738025, 53.7645690950855, 54.63517000822392, 55.81487678303038, 56.969061055497264, 57.94458781158721, 58.79533788957264, 59.87011882142755, 61.012959759521316, 62.218189036667326, 63.38938831069393, 64.57476675268695, 65.64387601735531, 66.42940192269518, 67.77075121231886, 68.8965771488529, 69.94299974477498, 70.98658650710377, 72.19465161784306, 73.26943254969798, 74.41510932138502, 75.57780109463177, 76.57317868587471, 77.67631795366248, 78.66885971131214, 79.61035646428267, 80.55468905084649, 81.39976746164535, 82.29021920993677, 83.22320846212745, 84.14201854635171, 85.39262116099027, 86.09590789212488, 86.6885971131214, 87.20755466069251, 87.83710971840173, 88.40711227065195, 88.97144315571562, 89.66055071888381, 90.31279244533931, 90.73816748433202, 91.20324419363072, 91.71653007401525, 92.1730992825341, 92.52757848169469, 92.9160876839747, 93.38967189405327, 93.90012194084451, 94.35952698295664, 94.78206618835607, 95.21027706094206, 95.61012959759522, 95.9844596319088, 96.37013300059553, 96.74446303490912, 97.24356974732723, 97.61789978164082, 98.09431982531265, 98.46864985962624, 99.05566741343618, 100.0], "rescued": [0, 554, 1112, 1639, 2125, 2623, 3187, 3680, 4129, 4628, 5075, 5537, 5932, 6348, 6778, 7241, 7614, 7967, 8281, 8607, 9010, 9339, 9650, 9947, 10305, 10628, 10920, 11155, 11389, 11613, 11843, 12086, 12339, 12564, 12800, 13026, 13249, 13451, 13670, 13885, 14065, 14300, 14462, 14700, 14909, 15081, 15247, 15448, 15649, 15880, 16090, 16288, 16466, 16595, 16788, 16963, 17116, 17294, 17468, 17612, 17766, 17911, 18048, 18159, 18268, 18359, 18457, 18514, 18576, 18624, 18660, 18680, 18683, 18685, 18687, 18691, 18696, 18699, 18702, 18707, 18708, 18710, 18715, 18719, 18723, 18726, 18727, 18729, 18737, 18745, 18748, 18753, 18757, 18763, 18768, 18775, 18780, 18790, 18796, 18809, 18820], "harmed": [0, 8, 20, 27, 44, 65, 91, 121, 147, 177, 201, 225, 276, 309, 345, 377, 399, 429, 460, 498, 528, 551, 582, 607, 643, 670, 697, 724, 749, 770, 796, 825, 848, 867, 892, 913, 938, 957, 980, 996, 1021, 1040, 1060, 1076, 1105, 1122, 1143, 1161, 1183, 1206, 1228, 1244, 1263, 1272, 1298, 1313, 1328, 1335, 1352, 1370, 1390, 1409, 1424, 1439, 1449, 1472, 1489, 1502, 1516, 1534, 1549, 1558, 1563, 1564, 1564, 1565, 1566, 1566, 1567, 1568, 1571, 1573, 1579, 1584, 1586, 1592, 1600, 1606, 1617, 1621, 1629, 1638, 1644, 1654, 1663, 1696, 1716, 1746, 1773, 1830, 1970], "exact_score": [26.688029946402747, 27.64370586733971, 28.60221762186995, 29.532371040467346, 30.468196126251314, 31.406857045628566, 32.257607123613994, 33.11402886878598, 33.97045061395797, 34.86657402943595, 35.67478660352211, 36.4943425119814, 37.25434591498171, 38.116439327340274, 38.941666902986135, 39.72435697473272, 40.52973371522559, 41.33511045571846, 42.07809885715906, 42.736012250801124, 43.51019482176786, 44.2276607208689, 44.9536341207498, 45.71930919093668, 46.40841675410487, 47.14289765476562, 47.817826049967394, 48.53529194906842, 49.190369509117204, 49.88514873947197, 50.59410713779315, 51.26903553299492, 51.96097892975639, 52.576354819499194, 53.25411904829424, 53.88083827241017, 54.518900830899256, 55.17965005813459, 55.74965261038483, 56.30831182826193, 56.90100104925844, 57.451152766355676, 57.998468649859625, 58.52309786461731, 59.14698125513995, 59.688625471457335, 60.252956356521004, 60.7747497376854, 61.31639395400278, 61.860874003913445, 62.33445821399201, 62.87043076312282, 63.352522473981224, 63.90267419107847, 64.38760173553015, 64.87820094716842, 65.39715849473953, 65.83954853529195, 66.30746107818393, 66.7782094546692, 67.24895783115447, 67.71119870685989, 68.15926041459888, 68.65836712701699, 69.1035930011627, 69.52613220656212, 69.95150724555484, 70.36553895017441, 70.72285398292829, 71.11419901880159, 71.49136488670844, 71.94509826163402, 72.33077163032074, 72.65122082636191, 72.98017752318295, 73.31764172078383, 73.64092675041829, 73.92451010974676, 74.19107846751552, 74.454810991691, 74.65899101040752, 74.88869353146357, 75.02764937753452, 75.13824688767264, 75.19779939313162, 75.24884439781074, 75.23466522984432, 75.24317273062418, 75.26302356577716, 75.2743669001503, 75.28854606811673, 75.28854606811673, 75.28571023452344, 75.2743669001503, 75.24033689703089, 75.22899356265775, 75.21765022828461, 75.15526188923234, 75.03899271190765, 74.84899186115759, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.945711819962156, "threshold_area100": 63.2781669171653, "threshold_area25": 42.43087655616369}, "selected_budget": 70, "selected": {"val": {"score": 81.90315552744394, "calls": 69.9978916297702, "rescued": 4601, "harmed": 923}, "test": {"score": 75.21197856109804, "calls": 84.14201854635171, "rescued": 18660, "harmed": 1549}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 13.26347541809082, "operator": "ge"}, {"budget": 0.02, "threshold": 12.509763717651367, "operator": "ge"}, {"budget": 0.03, "threshold": 11.964030265808105, "operator": "ge"}, {"budget": 0.04, "threshold": 11.506260871887207, "operator": "ge"}, {"budget": 0.05, "threshold": 11.188762664794922, "operator": "ge"}, {"budget": 0.06, "threshold": 10.853771209716797, "operator": "ge"}, {"budget": 0.07, "threshold": 10.595624923706055, "operator": "ge"}, {"budget": 0.08, "threshold": 10.31808090209961, "operator": "ge"}, {"budget": 0.09, "threshold": 10.113731384277344, "operator": "ge"}, {"budget": 0.1, "threshold": 9.851536750793457, "operator": "ge"}, {"budget": 0.11, "threshold": 9.621603965759277, "operator": "ge"}, {"budget": 0.12, "threshold": 9.427099227905273, "operator": "gt"}, {"budget": 0.13, "threshold": 9.20619010925293, "operator": "ge"}, {"budget": 0.14, "threshold": 9.00728988647461, "operator": "ge"}, {"budget": 0.15, "threshold": 8.800006866455078, "operator": "ge"}, {"budget": 0.16, "threshold": 8.614294052124023, "operator": "gt"}, {"budget": 0.17, "threshold": 8.444397926330566, "operator": "ge"}, {"budget": 0.18, "threshold": 8.25673770904541, "operator": "ge"}, {"budget": 0.19, "threshold": 8.088077545166016, "operator": "ge"}, {"budget": 0.2, "threshold": 7.90799617767334, "operator": "gt"}, {"budget": 0.21, "threshold": 7.737740516662598, "operator": "ge"}, {"budget": 0.22, "threshold": 7.5681633949279785, "operator": "ge"}, {"budget": 0.23, "threshold": 7.378425598144531, "operator": "ge"}, {"budget": 0.24, "threshold": 7.208701133728027, "operator": "ge"}, {"budget": 0.25, "threshold": 7.0264201164245605, "operator": "ge"}, {"budget": 0.26, "threshold": 6.852926731109619, "operator": "ge"}, {"budget": 0.27, "threshold": 6.67259407043457, "operator": "gt"}, {"budget": 0.28, "threshold": 6.509899616241455, "operator": "ge"}, {"budget": 0.29, "threshold": 6.331399440765381, "operator": "ge"}, {"budget": 0.3, "threshold": 6.17408561706543, "operator": "ge"}, {"budget": 0.31, "threshold": 6.00994873046875, "operator": "ge"}, {"budget": 0.32, "threshold": 5.876248359680176, "operator": "ge"}, {"budget": 0.33, "threshold": 5.7160797119140625, "operator": "ge"}, {"budget": 0.34, "threshold": 5.580066204071045, "operator": "ge"}, {"budget": 0.35, "threshold": 5.417982578277588, "operator": "ge"}, {"budget": 0.36, "threshold": 5.2637939453125, "operator": "ge"}, {"budget": 0.37, "threshold": 5.121455669403076, "operator": "ge"}, {"budget": 0.38, "threshold": 4.980216026306152, "operator": "ge"}, {"budget": 0.39, "threshold": 4.820289134979248, "operator": "ge"}, {"budget": 0.4, "threshold": 4.680587291717529, "operator": "ge"}, {"budget": 0.41, "threshold": 4.554009437561035, "operator": "ge"}, {"budget": 0.42, "threshold": 4.413954734802246, "operator": "ge"}, {"budget": 0.43, "threshold": 4.2623610496521, "operator": "ge"}, {"budget": 0.44, "threshold": 4.148448944091797, "operator": "ge"}, {"budget": 0.45, "threshold": 4.006083965301514, "operator": "ge"}, {"budget": 0.46, "threshold": 3.8798179626464844, "operator": "ge"}, {"budget": 0.47, "threshold": 3.7611324787139893, "operator": "ge"}, {"budget": 0.48, "threshold": 3.6135213375091553, "operator": "ge"}, {"budget": 0.49, "threshold": 3.4920144081115723, "operator": "ge"}, {"budget": 0.5, "threshold": 3.3644418716430664, "operator": "ge"}, {"budget": 0.51, "threshold": 3.2269020080566406, "operator": "ge"}, {"budget": 0.52, "threshold": 3.086276054382324, "operator": "ge"}, {"budget": 0.53, "threshold": 2.948383092880249, "operator": "ge"}, {"budget": 0.54, "threshold": 2.8228707313537598, "operator": "ge"}, {"budget": 0.55, "threshold": 2.6809887886047363, "operator": "ge"}, {"budget": 0.56, "threshold": 2.536836862564087, "operator": "ge"}, {"budget": 0.57, "threshold": 2.383521318435669, "operator": "ge"}, {"budget": 0.58, "threshold": 2.2226688861846924, "operator": "ge"}, {"budget": 0.59, "threshold": 2.0606751441955566, "operator": "ge"}, {"budget": 0.6, "threshold": 1.9063340425491333, "operator": "ge"}, {"budget": 0.61, "threshold": 1.7456607818603516, "operator": "ge"}, {"budget": 0.62, "threshold": 1.581493854522705, "operator": "ge"}, {"budget": 0.63, "threshold": 1.4075219631195068, "operator": "ge"}, {"budget": 0.64, "threshold": 1.2695386409759521, "operator": "ge"}, {"budget": 0.65, "threshold": 1.1098905801773071, "operator": "ge"}, {"budget": 0.66, "threshold": 0.9442943334579468, "operator": "ge"}, {"budget": 0.67, "threshold": 0.7534387707710266, "operator": "ge"}, {"budget": 0.68, "threshold": 0.5660443305969238, "operator": "ge"}, {"budget": 0.69, "threshold": 0.37758392095565796, "operator": "ge"}, {"budget": 0.7, "threshold": 0.1442994922399521, "operator": "ge"}, {"budget": 0.71, "threshold": -0.06422603130340576, "operator": "ge"}, {"budget": 0.72, "threshold": -0.30565834045410156, "operator": "ge"}, {"budget": 0.73, "threshold": -0.5370098948478699, "operator": "ge"}, {"budget": 0.74, "threshold": -0.776084840297699, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9789827466011047, "operator": "ge"}, {"budget": 0.76, "threshold": -1.1979796886444092, "operator": "ge"}, {"budget": 0.77, "threshold": -1.4191449880599976, "operator": "ge"}, {"budget": 0.78, "threshold": -1.625069499015808, "operator": "ge"}, {"budget": 0.79, "threshold": -1.8297028541564941, "operator": "ge"}, {"budget": 0.8, "threshold": -2.0645973682403564, "operator": "ge"}, {"budget": 0.81, "threshold": -2.2777066230773926, "operator": "ge"}, {"budget": 0.82, "threshold": -2.493241548538208, "operator": "ge"}, {"budget": 0.83, "threshold": -2.7106876373291016, "operator": "ge"}, {"budget": 0.84, "threshold": -2.8963117599487305, "operator": "ge"}, {"budget": 0.85, "threshold": -3.117058277130127, "operator": "ge"}, {"budget": 0.86, "threshold": -3.2736501693725586, "operator": "ge"}, {"budget": 0.87, "threshold": -3.470362663269043, "operator": "ge"}, {"budget": 0.88, "threshold": -3.7088608741760254, "operator": "ge"}, {"budget": 0.89, "threshold": -3.9370040893554688, "operator": "ge"}, {"budget": 0.9, "threshold": -4.13022518157959, "operator": "ge"}, {"budget": 0.91, "threshold": -4.381486892700195, "operator": "ge"}, {"budget": 0.92, "threshold": -4.66915225982666, "operator": "ge"}, {"budget": 0.93, "threshold": -5.012143135070801, "operator": "ge"}, {"budget": 0.94, "threshold": -5.3234663009643555, "operator": "ge"}, {"budget": 0.95, "threshold": -5.6868414878845215, "operator": "ge"}, {"budget": 0.96, "threshold": -6.121379375457764, "operator": "ge"}, {"budget": 0.97, "threshold": -6.625758647918701, "operator": "ge"}, {"budget": 0.98, "threshold": -7.301473140716553, "operator": "ge"}, {"budget": 0.99, "threshold": -8.241576194763184, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.01033101412608, 57.95206971677559, 58.85164101482887, 59.73715651135005, 60.636727809403325, 61.50818750439243, 62.36559139784945, 63.27219059666878, 64.11553868859372, 64.97294258205073, 65.83034647550777, 66.63152716283646, 67.46081945322932, 68.24794433902592, 68.99992972099234, 69.78002670602291, 70.48984468339306, 71.18560685923114, 71.83217372970694, 72.58415911167334, 73.31506079134162, 73.92648815798721, 74.58008292922904, 75.25476140276899, 75.88024457094666, 76.42842083069786, 76.9976807927472, 77.51071754866821, 77.98158689999298, 78.3962330451894, 78.7195164804273, 79.20444163328413, 79.52069716775598, 79.90020380912222, 80.24457094665823, 80.46946377117155, 80.77166350411132, 81.01764003092275, 81.12305854241337, 81.23550495467003, 81.42525827535314, 81.47445358071543, 81.64312319910043, 81.74854171059106, 81.82584861901749, 81.90315552744394, 81.79070911518728, 81.7766533136552, 81.79773701595333, 81.79773701595333, 81.83990442054959, 81.91721132897602, 82.00154613816852, 81.97343453510436, 81.9453229320402, 81.99451823740247, 81.97343453510436, 81.93829503127415, 81.86801602361375, 81.86801602361375, 81.93829503127415, 82.00857403893457, 81.98046243587041, 82.00857403893457, 82.04371354276478, 82.06479724506289, 82.07885304659499, 82.05776934429684, 82.02965774123268, 81.9453229320402, 81.84693232131562, 81.76962541288917, 81.67123480216458, 81.58689999297209, 81.51662098531169, 81.41120247382106, 81.36200716845879, 81.27064445850024, 81.2144212523719, 81.15117014547755, 81.10197484011525, 81.059807435519, 80.98952842785859, 80.9543889240284, 80.91924942019818, 80.89113781713402, 80.86302621406986, 80.83491461100569, 80.80680300794153, 80.76463560334528, 80.75057980181319, 80.73652400028111, 80.72246819874903, 80.69435659568487, 80.68732869491882, 80.65218919108862, 80.64516129032258, 80.61704968725842, 80.60299388572632, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.989598706866259, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.988474242743692, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.987349778621123, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.987138941598143, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 136, 270, 398, 524, 652, 776, 899, 1028, 1148, 1270, 1392, 1506, 1624, 1738, 1846, 1958, 2062, 2162, 2258, 2367, 2473, 2566, 2668, 2766, 2861, 2948, 3035, 3123, 3200, 3272, 3331, 3411, 3476, 3543, 3607, 3659, 3718, 3769, 3810, 3850, 3895, 3930, 3975, 4009, 4038, 4069, 4093, 4123, 4149, 4172, 4204, 4239, 4269, 4294, 4317, 4345, 4365, 4386, 4400, 4420, 4445, 4470, 4489, 4506, 4528, 4549, 4566, 4583, 4595, 4604, 4609, 4616, 4622, 4624, 4624, 4624, 4625, 4625, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 3, 4, 5, 8, 9, 13, 15, 17, 23, 32, 34, 40, 49, 55, 70, 80, 93, 106, 117, 137, 150, 165, 185, 201, 217, 243, 267, 285, 313, 334, 353, 371, 391, 431, 463, 486, 509, 535, 559, 577, 606, 633, 654, 677, 703, 727, 747, 762, 777, 800, 813, 830, 848, 863, 883, 899, 920, 939, 957, 977, 991, 1001, 1016, 1024, 1037, 1045, 1055, 1062, 1068, 1078, 1083, 1088, 1092, 1096, 1100, 1104, 1110, 1112, 1114, 1116, 1120, 1121, 1126, 1127, 1131, 1133, 1133], "exact_score": [56.05453650994448, 57.01033101412608, 57.95206971677559, 58.85164101482887, 59.73715651135005, 60.636727809403325, 61.50818750439243, 62.36559139784945, 63.27219059666878, 64.11553868859372, 64.97294258205073, 65.83034647550777, 66.6385550636025, 67.46081945322932, 68.24794433902592, 68.99992972099234, 69.78705460678894, 70.48984468339306, 71.18560685923114, 71.83217372970694, 72.59118701243938, 73.31506079134162, 73.92648815798721, 74.58008292922904, 75.25476140276899, 75.88024457094666, 76.42842083069786, 76.9976807927472, 77.51071754866821, 77.98158689999298, 78.3962330451894, 78.7195164804273, 79.20444163328413, 79.52069716775598, 79.90020380912222, 80.24457094665823, 80.46946377117155, 80.77166350411132, 81.01764003092275, 81.12305854241337, 81.23550495467003, 81.42525827535314, 81.47445358071543, 81.64312319910043, 81.74854171059106, 81.82584861901749, 81.90315552744394, 81.79070911518728, 81.7766533136552, 81.79773701595333, 81.79773701595333, 81.83990442054959, 81.91721132897602, 82.00154613816852, 81.97343453510436, 81.9453229320402, 81.99451823740247, 81.97343453510436, 81.93829503127415, 81.86801602361375, 81.86801602361375, 81.93829503127415, 82.00857403893457, 81.98046243587041, 82.00857403893457, 82.04371354276478, 82.06479724506289, 82.07885304659499, 82.05776934429684, 82.02965774123268, 81.9453229320402, 81.84693232131562, 81.76962541288917, 81.67123480216458, 81.58689999297209, 81.51662098531169, 81.41120247382106, 81.36200716845879, 81.27064445850024, 81.2144212523719, 81.15117014547755, 81.10197484011525, 81.059807435519, 80.98952842785859, 80.9543889240284, 80.91924942019818, 80.89113781713402, 80.86302621406986, 80.83491461100569, 80.80680300794153, 80.76463560334528, 80.75057980181319, 80.73652400028111, 80.72246819874903, 80.69435659568487, 80.68732869491882, 80.65218919108862, 80.64516129032258, 80.61704968725842, 80.60299388572632, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 77.4113084874988, "threshold_area100": 77.41025370721766, "threshold_area25": 66.63222995291306}, "test": {"score": [26.688029946402747, 27.4593766837762, 28.162663414910817, 28.86595014604543, 29.625953549045743, 30.249836939568386, 31.024019510535123, 31.64506706746448, 32.54686215012903, 33.11686470237927, 33.94492811161841, 34.77582735445085, 35.55568159260415, 36.30150582763804, 36.99912089158609, 37.855542636758074, 38.70345688115022, 39.582565295068484, 40.54958455037859, 41.411677962737144, 42.37869721804724, 43.371238975696905, 44.457363241924966, 45.58035334486573, 46.81394095794459, 47.93693106088534, 49.0429061622664, 50.29067294331169, 51.413663046252445, 52.71531066557015, 53.80427076539149, 55.032186711283785, 56.10980347673199, 57.26115191560559, 58.253693673255256, 59.48728128633412, 60.615943056461454, 61.560275643025264, 62.52445906474209, 63.568045827070875, 64.45282590817571, 65.2156651447693, 66.0153702180756, 66.86895612965431, 67.46164535065083, 68.19896208490486, 68.74060630122226, 69.22269801208066, 69.91180557524885, 70.30598644471542, 70.81643649150668, 71.21912486175312, 71.6303207327794, 72.02733743583927, 72.35912996625358, 72.64271332558205, 72.9518191872501, 73.23823838017185, 73.4906275699742, 73.70898675665711, 73.8791367722542, 74.08898845815727, 74.2506309729745, 74.42928848935144, 74.56540850182911, 74.7157076822732, 74.85182769475087, 74.97376853926212, 75.0701868814338, 75.16093355641891, 75.22332189547117, 75.25168023140402, 75.26869523296374, 75.24600856421746, 75.24033689703089, 75.22615772906445, 75.19496355953834, 75.16376939001219, 75.12973938689278, 75.1269035532995, 75.08436604940023, 75.04182854550095, 75.00779854238154, 74.97093270566883, 74.942574369736, 74.90570853302329, 74.87735019709044, 74.8660068627173, 74.82063352522475, 74.78943935569862, 74.7724243541389, 74.73839435101948, 74.72137934945977, 74.66749851118738, 74.65048350962766, 74.61928934010153, 74.59093100416867, 74.57108016901569, 74.53421433230298, 74.51436349714999, 74.47182599325072], "calls": [0.0, 0.8252275756458611, 1.5823951450528884, 2.3253835464934918, 3.1761336244789153, 3.8822561892068173, 4.761364603125089, 5.478830502226129, 6.474208093469075, 7.112270651958143, 8.116155743980944, 9.094518333664181, 10.001985083515299, 10.986019340385106, 11.893486090236225, 12.982446190057567, 14.048719621132634, 15.160366389700252, 16.39111816918583, 17.53679494087287, 18.84695006097042, 20.120239344355273, 21.532484473811074, 23.126222953237104, 24.734140600629555, 26.359073249581716, 27.998185066500298, 29.8386410685421, 31.474917051867397, 33.46850806794657, 35.22672489578312, 37.10121090094433, 38.717636049116635, 40.640331225363695, 42.32198054618155, 44.42049740521226, 46.30065507756005, 48.02200606868389, 49.79440206448685, 51.776649746192895, 53.472478234977174, 54.88188753083969, 56.58055185321725, 58.253693673255256, 59.461758783994554, 61.03848226186087, 62.249383206193464, 63.35252247398123, 64.79312593936987, 65.83387686810538, 66.99089697416557, 68.14508124663244, 69.23687718004707, 70.23225477129003, 71.15390068910756, 72.09823327567139, 73.12480503644046, 74.12301846127669, 75.1269035532995, 75.96914613050507, 76.68094036241953, 77.55437710915125, 78.40229135354338, 79.20766809403624, 79.87976065564473, 80.65677906020474, 81.39693162805206, 82.16544253183223, 82.860221762187, 83.63724016674702, 84.48231857754587, 85.21396364461333, 86.0760570569719, 86.80486629044609, 87.52800385673368, 88.11785724413691, 88.8069648073051, 89.47054986813374, 90.12562742818251, 90.72682414995889, 91.37906587641437, 91.9490684286646, 92.57862348637383, 93.14579020503076, 93.61086691432948, 94.09295862518788, 94.46445282590817, 94.89833536568074, 95.45699458355784, 95.87953378895726, 96.23684882171113, 96.66222386070386, 97.08759889969657, 97.71998979099907, 98.15387233077163, 98.59342653773076, 98.99044324079063, 99.3307432719848, 99.67104330317898, 99.84402915236934, 100.0], "rescued": [0, 272, 520, 768, 1036, 1256, 1529, 1749, 2067, 2269, 2562, 2858, 3135, 3402, 3648, 3953, 4255, 4566, 4909, 5220, 5565, 5922, 6317, 6729, 7174, 7587, 8000, 8469, 8879, 9371, 9791, 10254, 10661, 11106, 11488, 11967, 12410, 12782, 13163, 13576, 13931, 14229, 14564, 14916, 15159, 15466, 15688, 15895, 16176, 16345, 16561, 16743, 16918, 17084, 17228, 17362, 17508, 17640, 17769, 17870, 17948, 18046, 18132, 18219, 18289, 18362, 18430, 18502, 18553, 18607, 18653, 18688, 18717, 18733, 18756, 18764, 18775, 18780, 18789, 18794, 18798, 18802, 18805, 18806, 18808, 18809, 18809, 18810, 18812, 18812, 18813, 18814, 18817, 18817, 18818, 18818, 18819, 18820, 18820, 18820, 18820], "harmed": [0, 0, 0, 0, 0, 0, 0, 1, 1, 2, 3, 6, 8, 12, 12, 15, 18, 19, 21, 28, 32, 39, 51, 67, 77, 94, 117, 146, 160, 193, 229, 259, 286, 325, 357, 401, 446, 485, 526, 571, 614, 643, 696, 747, 781, 828, 859, 896, 934, 964, 1000, 1040, 1070, 1096, 1123, 1157, 1194, 1225, 1265, 1289, 1307, 1331, 1360, 1384, 1406, 1426, 1446, 1475, 1492, 1514, 1538, 1563, 1586, 1610, 1635, 1648, 1670, 1686, 1707, 1713, 1732, 1751, 1766, 1780, 1792, 1806, 1816, 1821, 1839, 1850, 1857, 1870, 1879, 1898, 1905, 1916, 1927, 1935, 1948, 1955, 1970], "exact_score": [26.688029946402747, 27.62385503218671, 28.556844284377398, 29.467146867821796, 30.357598616113208, 31.228199529251626, 32.11865127754304, 33.0176105266143, 33.84567393585345, 34.696424013838865, 35.552845759010864, 36.312849162011176, 37.081360065791344, 37.8697218047245, 38.65808354365766, 39.46062445055725, 40.23480702152398, 40.98630292374443, 41.77182882908431, 42.49213056177864, 43.266313132745374, 44.04616737089868, 44.76930493718629, 45.50378583784704, 46.24110257210107, 47.00677764228796, 47.687377704676294, 48.34812693191164, 49.0429061622664, 49.73201372543459, 50.409777954229654, 51.09604968380456, 51.76247057822647, 52.443070640614806, 53.03575986161132, 53.64546408416755, 54.32889998014916, 54.95278337067181, 55.647562601026564, 56.262938490769365, 56.861299378952445, 57.44831693276239, 58.055185321725325, 58.61384453960241, 59.19802625981907, 59.84743215268129, 60.46280804242408, 61.018631426707884, 61.54042480787227, 62.09057652496952, 62.62087740691376, 63.13699912089159, 63.68431500439554, 64.2117800527465, 64.72506593313105, 65.26954598304171, 65.69775685562772, 66.19969940163911, 66.73283611717665, 67.24895783115447, 67.72254204123303, 68.17060374897201, 68.638516291864, 69.0837421660097, 69.49777387062927, 69.98270141508097, 70.37404645095427, 70.81076482432012, 71.16240818988742, 71.54524572498087, 71.94226242804072, 72.31659246235431, 72.60017582168278, 72.91211751694411, 73.19853670986586, 73.44525423248164, 73.72033009103026, 73.94152511130648, 74.18257096673567, 74.40660182060516, 74.58242350338881, 74.80361852366504, 74.95391770410913, 75.09854521736665, 75.17511272438533, 75.24600856421746, 75.26585939937046, 75.22899356265775, 75.23182939625104, 75.18645605875847, 75.14108272126592, 75.11272438533308, 75.03615687831437, 74.97093270566883, 74.91705186739642, 74.86033519553072, 74.78376768851204, 74.72421518305306, 74.6618268440008, 74.59093100416867, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.95098585282372, "threshold_area100": 61.55398009244819, "threshold_area25": 36.348297081927235}, "selected_budget": 67, "selected": {"val": {"score": 82.07885304659499, "calls": 66.9969780026706, "rescued": 4566, "harmed": 863}, "test": {"score": 74.97376853926212, "calls": 82.16544253183223, "rescued": 18502, "harmed": 1475}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.406416416168213, "operator": "ge"}, {"budget": 0.02, "threshold": 3.98079252243042, "operator": "ge"}, {"budget": 0.03, "threshold": 3.701655387878418, "operator": "gt"}, {"budget": 0.04, "threshold": 3.4911234378814697, "operator": "ge"}, {"budget": 0.05, "threshold": 3.29880690574646, "operator": "ge"}, {"budget": 0.06, "threshold": 3.136819839477539, "operator": "ge"}, {"budget": 0.07, "threshold": 2.993649482727051, "operator": "ge"}, {"budget": 0.08, "threshold": 2.8654091358184814, "operator": "ge"}, {"budget": 0.09, "threshold": 2.753793954849243, "operator": "ge"}, {"budget": 0.1, "threshold": 2.6521854400634766, "operator": "ge"}, {"budget": 0.11, "threshold": 2.5586066246032715, "operator": "ge"}, {"budget": 0.12, "threshold": 2.473827838897705, "operator": "ge"}, {"budget": 0.13, "threshold": 2.390028238296509, "operator": "gt"}, {"budget": 0.14, "threshold": 2.320079803466797, "operator": "ge"}, {"budget": 0.15, "threshold": 2.242867946624756, "operator": "ge"}, {"budget": 0.16, "threshold": 2.173914909362793, "operator": "ge"}, {"budget": 0.17, "threshold": 2.104205369949341, "operator": "ge"}, {"budget": 0.18, "threshold": 2.0430288314819336, "operator": "ge"}, {"budget": 0.19, "threshold": 1.9812525510787964, "operator": "ge"}, {"budget": 0.2, "threshold": 1.9268290996551514, "operator": "ge"}, {"budget": 0.21, "threshold": 1.8748130798339844, "operator": "ge"}, {"budget": 0.22, "threshold": 1.8218700885772705, "operator": "ge"}, {"budget": 0.23, "threshold": 1.772438645362854, "operator": "ge"}, {"budget": 0.24, "threshold": 1.716322660446167, "operator": "ge"}, {"budget": 0.25, "threshold": 1.6674937009811401, "operator": "ge"}, {"budget": 0.26, "threshold": 1.619567632675171, "operator": "ge"}, {"budget": 0.27, "threshold": 1.5763840675354004, "operator": "ge"}, {"budget": 0.28, "threshold": 1.5268383026123047, "operator": "ge"}, {"budget": 0.29, "threshold": 1.4822180271148682, "operator": "ge"}, {"budget": 0.3, "threshold": 1.4411826133728027, "operator": "ge"}, {"budget": 0.31, "threshold": 1.4059747457504272, "operator": "ge"}, {"budget": 0.32, "threshold": 1.3736166954040527, "operator": "ge"}, {"budget": 0.33, "threshold": 1.3408480882644653, "operator": "ge"}, {"budget": 0.34, "threshold": 1.3105307817459106, "operator": "ge"}, {"budget": 0.35, "threshold": 1.2742701768875122, "operator": "ge"}, {"budget": 0.36, "threshold": 1.2395293712615967, "operator": "ge"}, {"budget": 0.37, "threshold": 1.2067779302597046, "operator": "ge"}, {"budget": 0.38, "threshold": 1.1758334636688232, "operator": "ge"}, {"budget": 0.39, "threshold": 1.14585542678833, "operator": "ge"}, {"budget": 0.4, "threshold": 1.117655634880066, "operator": "ge"}, {"budget": 0.41, "threshold": 1.0879427194595337, "operator": "ge"}, {"budget": 0.42, "threshold": 1.0603392124176025, "operator": "ge"}, {"budget": 0.43, "threshold": 1.0358442068099976, "operator": "ge"}, {"budget": 0.44, "threshold": 1.0061681270599365, "operator": "ge"}, {"budget": 0.45, "threshold": 0.9762258529663086, "operator": "ge"}, {"budget": 0.46, "threshold": 0.9520658254623413, "operator": "ge"}, {"budget": 0.47, "threshold": 0.9256227612495422, "operator": "ge"}, {"budget": 0.48, "threshold": 0.8977717757225037, "operator": "ge"}, {"budget": 0.49, "threshold": 0.87018221616745, "operator": "ge"}, {"budget": 0.5, "threshold": 0.8411545157432556, "operator": "ge"}, {"budget": 0.51, "threshold": 0.8116355538368225, "operator": "ge"}, {"budget": 0.52, "threshold": 0.7849956750869751, "operator": "ge"}, {"budget": 0.53, "threshold": 0.7629612684249878, "operator": "ge"}, {"budget": 0.54, "threshold": 0.7328059673309326, "operator": "ge"}, {"budget": 0.55, "threshold": 0.7069882154464722, "operator": "ge"}, {"budget": 0.56, "threshold": 0.6842174530029297, "operator": "ge"}, {"budget": 0.57, "threshold": 0.6561480760574341, "operator": "ge"}, {"budget": 0.58, "threshold": 0.630508542060852, "operator": "ge"}, {"budget": 0.59, "threshold": 0.5991063117980957, "operator": "ge"}, {"budget": 0.6, "threshold": 0.5746657252311707, "operator": "ge"}, {"budget": 0.61, "threshold": 0.5481499433517456, "operator": "ge"}, {"budget": 0.62, "threshold": 0.5143506526947021, "operator": "ge"}, {"budget": 0.63, "threshold": 0.48178911209106445, "operator": "ge"}, {"budget": 0.64, "threshold": 0.44752058386802673, "operator": "ge"}, {"budget": 0.65, "threshold": 0.4132862389087677, "operator": "ge"}, {"budget": 0.66, "threshold": 0.36980050802230835, "operator": "ge"}, {"budget": 0.67, "threshold": 0.315234899520874, "operator": "ge"}, {"budget": 0.68, "threshold": 0.2624332904815674, "operator": "ge"}, {"budget": 0.69, "threshold": 0.18256622552871704, "operator": "ge"}, {"budget": 0.7, "threshold": 0.08004328608512878, "operator": "ge"}, {"budget": 0.71, "threshold": -0.041453540325164795, "operator": "ge"}, {"budget": 0.72, "threshold": -0.19082707166671753, "operator": "ge"}, {"budget": 0.73, "threshold": -0.36773329973220825, "operator": "ge"}, {"budget": 0.74, "threshold": -0.5471178293228149, "operator": "ge"}, {"budget": 0.75, "threshold": -0.6968772411346436, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8785563707351685, "operator": "ge"}, {"budget": 0.77, "threshold": -1.0528063774108887, "operator": "ge"}, {"budget": 0.78, "threshold": -1.2383923530578613, "operator": "ge"}, {"budget": 0.79, "threshold": -1.4287482500076294, "operator": "ge"}, {"budget": 0.8, "threshold": -1.6401907205581665, "operator": "ge"}, {"budget": 0.81, "threshold": -1.8221209049224854, "operator": "ge"}, {"budget": 0.82, "threshold": -1.993744134902954, "operator": "ge"}, {"budget": 0.83, "threshold": -2.183239459991455, "operator": "ge"}, {"budget": 0.84, "threshold": -2.380019187927246, "operator": "ge"}, {"budget": 0.85, "threshold": -2.5604183673858643, "operator": "ge"}, {"budget": 0.86, "threshold": -2.733421802520752, "operator": "ge"}, {"budget": 0.87, "threshold": -2.9435853958129883, "operator": "ge"}, {"budget": 0.88, "threshold": -3.151235818862915, "operator": "ge"}, {"budget": 0.89, "threshold": -3.3482110500335693, "operator": "ge"}, {"budget": 0.9, "threshold": -3.5610251426696777, "operator": "ge"}, {"budget": 0.91, "threshold": -3.7991249561309814, "operator": "ge"}, {"budget": 0.92, "threshold": -4.067526817321777, "operator": "ge"}, {"budget": 0.93, "threshold": -4.384528636932373, "operator": "ge"}, {"budget": 0.94, "threshold": -4.765129089355469, "operator": "ge"}, {"budget": 0.95, "threshold": -5.076054096221924, "operator": "ge"}, {"budget": 0.96, "threshold": -5.494600772857666, "operator": "ge"}, {"budget": 0.97, "threshold": -6.078342914581299, "operator": "ge"}, {"budget": 0.98, "threshold": -6.649518966674805, "operator": "ge"}, {"budget": 0.99, "threshold": -7.528114318847656, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.97519151029587, 57.83962330451894, 58.605664488017425, 59.44198467917633, 60.16585845807857, 60.910815939278926, 61.5714386112868, 62.225033382528636, 62.773209642279845, 63.36355330662731, 63.94686907020872, 64.45990582612973, 64.97294258205073, 65.54220254410006, 65.94982078853047, 66.45582964368543, 66.8283083842856, 67.24295452948203, 67.7138238808068, 68.10035842293907, 68.43769765970904, 68.84531590413943, 69.22482254550565, 69.64649659146812, 70.04005903436644, 70.47578888186099, 70.89746292782345, 71.18560685923114, 71.5370018975332, 71.78297842434465, 72.0992339588165, 72.52090800477897, 72.94258205074145, 73.24478178368122, 73.57509311968514, 73.89837655492303, 74.15840888326657, 74.4535807154403, 74.74875254761402, 74.95256166982922, 75.10717548668212, 75.33206831119544, 75.67643544873147, 75.97863518167122, 76.23866751001475, 76.4846440368262, 76.90631808278867, 77.26474102185676, 77.5599128540305, 77.89725209080048, 78.24161922833649, 78.57193056434042, 78.76871178578958, 79.01468831260102, 79.29580434324266, 79.54178087005411, 79.81586899992972, 80.04778972520907, 80.30079415278657, 80.63813338955653, 80.84194251177172, 81.04575163398692, 81.24956075620211, 81.43228617611919, 81.65015109986646, 81.79773701595333, 81.89612762667791, 81.9453229320402, 82.02262984046665, 81.9453229320402, 81.84693232131562, 81.79070911518728, 81.65015109986646, 81.58689999297209, 81.50959308454564, 81.44634197765127, 81.37606296999085, 81.31281186309648, 81.23550495467003, 81.15819804624358, 81.13008644317942, 81.07386323705109, 81.0246679316888, 80.98250052709255, 80.94736102326235, 80.91924942019818, 80.89113781713402, 80.86302621406986, 80.82788671023965, 80.79977510717549, 80.75760770257924, 80.74355190104716, 80.73652400028111, 80.715440297983, 80.68030079415279, 80.66624499262069, 80.64516129032258, 80.62407758802446, 80.61002178649237, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.987560615644107, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 136, 264, 380, 505, 613, 728, 832, 934, 1024, 1123, 1218, 1304, 1392, 1485, 1556, 1638, 1708, 1781, 1857, 1925, 1994, 2066, 2133, 2211, 2284, 2356, 2431, 2488, 2553, 2607, 2667, 2737, 2813, 2875, 2937, 2997, 3052, 3108, 3166, 3214, 3262, 3311, 3377, 3433, 3483, 3533, 3607, 3668, 3722, 3787, 3847, 3905, 3947, 3991, 4038, 4084, 4138, 4187, 4240, 4295, 4336, 4374, 4413, 4447, 4488, 4520, 4551, 4572, 4594, 4602, 4608, 4618, 4621, 4623, 4624, 4624, 4625, 4625, 4625, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 5, 10, 17, 23, 28, 37, 47, 56, 68, 83, 95, 108, 123, 135, 148, 158, 175, 189, 198, 211, 232, 246, 259, 277, 294, 304, 319, 335, 350, 369, 384, 394, 410, 429, 444, 458, 476, 490, 506, 525, 551, 568, 585, 598, 611, 626, 640, 650, 662, 679, 690, 701, 715, 724, 731, 742, 757, 773, 790, 797, 809, 818, 828, 836, 846, 857, 874, 888, 899, 918, 938, 956, 979, 990, 1002, 1011, 1022, 1031, 1042, 1053, 1058, 1066, 1073, 1079, 1084, 1088, 1092, 1096, 1101, 1105, 1111, 1113, 1114, 1117, 1122, 1124, 1127, 1130, 1132, 1133], "exact_score": [56.05453650994448, 56.97519151029587, 57.83962330451894, 58.61269238878346, 59.44198467917633, 60.16585845807857, 60.910815939278926, 61.5714386112868, 62.225033382528636, 62.773209642279845, 63.36355330662731, 63.94686907020872, 64.45990582612973, 64.97997048281677, 65.54220254410006, 65.94982078853047, 66.45582964368543, 66.8283083842856, 67.24295452948203, 67.7138238808068, 68.10035842293907, 68.43769765970904, 68.84531590413943, 69.22482254550565, 69.64649659146812, 70.04005903436644, 70.47578888186099, 70.89746292782345, 71.18560685923114, 71.5370018975332, 71.78297842434465, 72.0992339588165, 72.52090800477897, 72.94258205074145, 73.24478178368122, 73.57509311968514, 73.89837655492303, 74.15840888326657, 74.4535807154403, 74.74875254761402, 74.95256166982922, 75.10717548668212, 75.33206831119544, 75.67643544873147, 75.97863518167122, 76.23866751001475, 76.4846440368262, 76.90631808278867, 77.26474102185676, 77.5599128540305, 77.89725209080048, 78.24161922833649, 78.57193056434042, 78.76871178578958, 79.01468831260102, 79.29580434324266, 79.54178087005411, 79.81586899992972, 80.04778972520907, 80.30079415278657, 80.63813338955653, 80.84194251177172, 81.04575163398692, 81.24956075620211, 81.43228617611919, 81.65015109986646, 81.79773701595333, 81.89612762667791, 81.9453229320402, 82.02262984046665, 81.9453229320402, 81.84693232131562, 81.79070911518728, 81.65015109986646, 81.58689999297209, 81.50959308454564, 81.44634197765127, 81.37606296999085, 81.31281186309648, 81.23550495467003, 81.15819804624358, 81.13008644317942, 81.07386323705109, 81.0246679316888, 80.98250052709255, 80.94736102326235, 80.91924942019818, 80.89113781713402, 80.86302621406986, 80.82788671023965, 80.79977510717549, 80.75760770257924, 80.74355190104716, 80.73652400028111, 80.715440297983, 80.68030079415279, 80.66624499262069, 80.64516129032258, 80.62407758802446, 80.61002178649237, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 74.97241355088127, "threshold_area100": 74.97143158338604, "threshold_area25": 64.12537774966619}, "test": {"score": [26.688029946402747, 28.437739273459435, 30.142075263023564, 31.64506706746448, 32.86731134617021, 34.32209397952528, 35.74001077616766, 36.976434222839806, 38.21569350310524, 39.39823611150498, 40.478688710546464, 41.51660380568868, 42.58571307035703, 43.600941496752974, 44.45452740833168, 45.46975583472762, 46.419760088478014, 47.383943510194825, 48.16096191475484, 48.983353656807424, 49.714998723874885, 50.47216629328191, 51.28888636814792, 52.046053937554944, 52.92799818506651, 53.767404928678786, 54.433825823100705, 55.02367921050393, 55.88010095567593, 56.498312679011995, 57.08249439922866, 57.63548194991919, 58.16294699827014, 58.65921787709498, 59.18384709185265, 59.76235714488274, 60.3635538666591, 60.93922808609591, 61.38161812664833, 61.93176984374558, 62.31177154524572, 62.774012420951145, 63.29296996852225, 63.678643337208975, 64.18625755040695, 64.61730425658622, 65.05118679635879, 65.49924850409778, 65.87641437200466, 66.39820775316905, 66.89164279840058, 67.353883674106, 67.77075121231886, 68.06851373961376, 68.64135212545727, 69.12627966990897, 69.54314720812182, 70.0110597510138, 70.49598729546551, 71.0177806766299, 71.36375237501063, 71.74942574369736, 72.16062161472365, 72.5803249865298, 73.03973002864193, 73.40838839576892, 73.74301675977654, 74.14286929642968, 74.51719933074328, 74.85182769475087, 75.16093355641891, 75.24317273062418, 75.29421773530329, 75.28571023452344, 75.24317273062418, 75.24317273062418, 75.21765022828461, 75.18362022516519, 75.17511272438533, 75.1269035532995, 75.09287355018009, 75.06451521424722, 75.03332104472109, 74.9964552080084, 74.9766043728554, 74.93973853614271, 74.90003686583671, 74.863171029124, 74.82914102600459, 74.78660352210534, 74.76108101976577, 74.74406601820604, 74.72137934945977, 74.69302101352693, 74.65048350962766, 74.63630434166124, 74.6079460057284, 74.57108016901569, 74.53988599948956, 74.51436349714999, 74.47182599325072], "calls": [0.0, 1.8376201684485154, 3.6695686697104617, 5.325695488188753, 6.80600062388339, 8.558545784533363, 10.214672603011655, 11.82542608399739, 13.314238720471883, 14.865439695998639, 16.31171482857386, 17.653064118197545, 19.025607577347362, 20.42934520602331, 21.688455321441737, 23.055327113404985, 24.45055724130108, 25.794742364518047, 27.065195814309618, 28.37251510081388, 29.574908544366618, 30.66670447778124, 31.85491875336755, 33.026118027394155, 34.424183988883534, 35.74001077616765, 36.916881717380825, 38.02569265235516, 39.39540027791169, 40.46167370898676, 41.52227547287525, 42.58571307035703, 43.496015653801436, 44.51124408019738, 45.43005416442163, 46.50199926268326, 47.62782519921731, 48.59200862093412, 49.49947537078524, 50.517539630774465, 51.419334713439014, 52.48844397810737, 53.43844823185775, 54.28636247624989, 55.341292572951815, 56.31398349544848, 57.224286078892895, 58.27354450840825, 59.17250375747951, 60.28415052604713, 61.33340895556248, 62.459234892096525, 63.42908998099992, 64.18342171681365, 65.40566599551939, 66.42940192269518, 67.39074951081871, 68.51657544735275, 69.60553554717409, 70.82494399228653, 71.78629158041007, 72.75331083572016, 73.8819726058475, 74.96809687207555, 76.1109378101693, 77.19706207639736, 78.33990301449111, 79.76065564472677, 80.98573575702578, 82.61066840597793, 84.1788843830644, 85.2763519836656, 86.2235204038227, 87.00337464197601, 87.79740804809573, 88.44681394095795, 89.09621983382016, 89.73144655871593, 90.36667328361172, 91.01324334288063, 91.73638090916825, 92.25250262314607, 92.76011683634404, 93.33862688937413, 93.82071860023254, 94.24609363922525, 94.68848367977768, 95.16206788985623, 95.63565209993477, 96.0581913053342, 96.46371550917392, 96.90043388253977, 97.32864475512577, 97.76252729489833, 98.26446984090974, 98.53670986586506, 98.85715906190626, 99.28253410089896, 99.58596829538043, 99.82984998440291, 100.0], "rescued": [0, 626, 1241, 1784, 2237, 2784, 3302, 3771, 4229, 4686, 5097, 5492, 5894, 6291, 6630, 7011, 7377, 7736, 8051, 8378, 8673, 8962, 9276, 9572, 9914, 10242, 10507, 10756, 11078, 11320, 11549, 11769, 11975, 12176, 12377, 12606, 12845, 13067, 13242, 13456, 13616, 13812, 14009, 14162, 14361, 14534, 14706, 14882, 15037, 15239, 15433, 15623, 15785, 15904, 16115, 16298, 16456, 16640, 16826, 17026, 17163, 17302, 17458, 17620, 17790, 17932, 18066, 18223, 18368, 18509, 18631, 18688, 18719, 18741, 18756, 18764, 18771, 18779, 18788, 18792, 18798, 18800, 18803, 18805, 18808, 18809, 18809, 18809, 18810, 18811, 18812, 18814, 18816, 18817, 18818, 18818, 18818, 18820, 18820, 18820, 18820], "harmed": [0, 9, 23, 36, 58, 92, 110, 143, 164, 204, 234, 263, 288, 327, 365, 388, 419, 438, 479, 516, 553, 575, 601, 630, 661, 693, 723, 764, 784, 808, 831, 856, 876, 902, 918, 943, 970, 989, 1008, 1028, 1054, 1087, 1101, 1118, 1138, 1159, 1178, 1196, 1218, 1236, 1256, 1283, 1298, 1312, 1321, 1333, 1344, 1363, 1378, 1394, 1409, 1412, 1423, 1437, 1445, 1457, 1473, 1489, 1502, 1525, 1538, 1566, 1579, 1604, 1634, 1642, 1658, 1678, 1690, 1711, 1729, 1741, 1755, 1770, 1780, 1794, 1808, 1821, 1834, 1850, 1860, 1868, 1878, 1889, 1905, 1910, 1920, 1935, 1946, 1955, 1970], "exact_score": [26.688029946402747, 27.626690865779995, 28.58520262031024, 29.509684371721068, 30.451181124691608, 31.361483708136006, 32.232084621274424, 33.04596886254715, 33.85985310381987, 34.696424013838865, 35.55851742619743, 36.36673000028359, 37.12673340328389, 37.95479681252304, 38.74883021864277, 39.50316195445651, 40.27450869182997, 40.99764625811758, 41.774664662677594, 42.563026401610756, 43.28899980149165, 43.97243569747328, 44.70124493094745, 45.427218330828346, 46.1248333947764, 46.78841845560503, 47.50588435470607, 48.12126024444886, 48.73947196778493, 49.39738536142699, 50.024104585542915, 50.68485381277827, 51.37112554235318, 52.031874769588526, 52.641578992144744, 53.285313217820374, 53.92337577630945, 54.48487082777983, 55.00950004253751, 55.64189093384, 56.262938490769365, 56.798911039900176, 57.32921192184443, 57.88503530612824, 58.40399285369935, 58.94847290361002, 59.4901171199274, 60.014746334685086, 60.58474888693532, 61.132064770439264, 61.6481864844171, 62.144457363241926, 62.581175736607776, 63.04625244590647, 63.56520999347759, 64.01327170121658, 64.44999007458243, 64.92357428466097, 65.37447182599325, 65.7941751977994, 66.2762669086578, 66.73567195076993, 67.16388282335592, 67.59209369594193, 68.00045373337494, 68.49388877860648, 68.93060715197232, 69.36165385815161, 69.80404389870402, 70.24359810566317, 70.65195814309617, 71.09151235005531, 71.45166321640247, 71.83166491790263, 72.19748745143634, 72.58599665371636, 73.00286419192922, 73.33465672234354, 73.64659841760486, 73.9358534441199, 74.23078013782151, 74.52003516433656, 74.74973768539263, 74.95391770410913, 75.14675438845248, 75.25168023140402, 75.29138190171001, 75.28571023452344, 75.24884439781074, 75.22332189547117, 75.17227689079205, 75.1269035532995, 75.07869438221365, 75.01063437597482, 74.96242520488899, 74.87451436349714, 74.79794685647848, 74.73839435101948, 74.6816776791538, 74.59943850494854, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.45718365226591, "threshold_area100": 62.467940901227905, "threshold_area25": 42.04265093724301}, "selected_budget": 69, "selected": {"val": {"score": 82.02262984046665, "calls": 68.99992972099234, "rescued": 4594, "harmed": 899}, "test": {"score": 74.85182769475087, "calls": 82.61066840597793, "rescued": 18509, "harmed": 1525}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 81.46039777918335, "calls": 76.9976807927472}, "test": {"score": 75.25168023140402, "calls": 88.96009982134248}}, "old_normalized_interior": {"val": {"score": 81.46039777918335, "calls": 76.9976807927472}, "test": {"score": 75.25168023140402, "calls": 88.96009982134248}}, "old_normalized_endpoints": {"val": {"score": 81.46039777918335, "calls": 76.9976807927472}, "test": {"score": 75.25168023140402, "calls": 88.96009982134248}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 57.05249841872232, 57.9872092206058, 58.795417808700535, 59.70904490828589, 60.573476702508955, 61.43790849673202, 62.18286597793239, 62.93485135989879, 63.64466933726896, 64.2842083069787, 64.92374727668845, 65.5140909410359, 66.11146250614941, 66.61044346053832, 67.1164523156933, 67.64354487314638, 68.10035842293907, 68.52906036966758, 68.88045540796963, 69.32321315623022, 69.70271979759646, 70.11033804202684, 70.53201208798932, 70.77096071403471, 71.14343945463489, 71.54402979829925, 71.9305643404315, 72.19059666877503, 72.47874060018272, 72.90744254691124, 73.26586548597933, 73.60320472274931, 73.94054395951929, 74.24274369245906, 74.48169231850446, 74.79091995221026, 74.9666174713613, 75.2758451050671, 75.49371002881439, 75.79590976175416, 76.08405369316185, 76.4003092276337, 76.75170426593576, 76.95551338815095, 77.26474102185676, 77.48260594560405, 77.73561037318153, 77.90427999156651, 78.1221449153138, 78.37514934289128, 78.54381896127627, 78.81087919038583, 78.92332560264248, 79.05685571719727, 79.27472064094454, 79.41527865626537, 79.60503197694848, 79.88614800759012, 80.01967812214491, 80.13212453440157, 80.28673835125447, 80.44135216810739, 80.52568697729987, 80.69435659568487, 80.79274720640944, 80.80680300794153, 80.88410991636798, 81.01061213015672, 81.08791903858317, 81.19333755007378, 81.2636165577342, 81.29172816079837, 81.39714667228898, 81.39714667228898, 81.42525827535314, 81.45336987841732, 81.46039777918335, 81.45336987841732, 81.45336987841732, 81.44634197765127, 81.44634197765127, 81.43931407688522, 81.42525827535314, 81.42525827535314, 81.42525827535314, 81.41823037458711, 81.40417457305502, 81.40417457305502, 81.39714667228898, 81.39714667228898, 81.33389556539461, 81.2987560615644, 81.29172816079837, 81.29172816079837, 81.25658865696816, 81.20739335160587, 81.13711434394546, 81.04575163398692, 80.85599831330381, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.990301496942863, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.98854452175135, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.9895284278586, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 57.05249841872232, 57.9872092206058, 58.802445709466575, 59.70904490828589, 60.573476702508955, 61.43790849673202, 62.18286597793239, 62.93485135989879, 63.64466933726896, 64.2842083069787, 64.92374727668845, 65.5140909410359, 66.11146250614941, 66.61044346053832, 67.1164523156933, 67.64354487314638, 68.10035842293907, 68.52906036966758, 68.88045540796963, 69.32321315623022, 69.70271979759646, 70.11736594279289, 70.53201208798932, 70.77096071403471, 71.14343945463489, 71.54402979829925, 71.9305643404315, 72.19059666877503, 72.47874060018272, 72.90744254691124, 73.26586548597933, 73.60320472274931, 73.94054395951929, 74.24274369245906, 74.48169231850446, 74.79091995221026, 74.9666174713613, 75.2758451050671, 75.49371002881439, 75.79590976175416, 76.08405369316185, 76.4003092276337, 76.75170426593576, 76.95551338815095, 77.26474102185676, 77.48260594560405, 77.73561037318153, 77.90427999156651, 78.1221449153138, 78.37514934289128, 78.54381896127627, 78.81087919038583, 78.92332560264248, 79.05685571719727, 79.27472064094454, 79.41527865626537, 79.60503197694848, 79.88614800759012, 80.01967812214491, 80.13212453440157, 80.28673835125447, 80.44135216810739, 80.52568697729987, 80.69435659568487, 80.79274720640944, 80.80680300794153, 80.88410991636798, 81.01061213015672, 81.08791903858317, 81.19333755007378, 81.2636165577342, 81.29172816079837, 81.39714667228898, 81.39714667228898, 81.42525827535314, 81.45336987841732, 81.46039777918335, 81.45336987841732, 81.45336987841732, 81.44634197765127, 81.44634197765127, 81.43931407688522, 81.42525827535314, 81.42525827535314, 81.42525827535314, 81.41823037458711, 81.40417457305502, 81.40417457305502, 81.39714667228898, 81.39714667228898, 81.33389556539461, 81.2987560615644, 81.29172816079837, 81.29172816079837, 81.25658865696816, 81.20739335160587, 81.13711434394546, 81.04575163398692, 80.85599831330381, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 64.97926769274017, "exact_area100": 75.4235979640494}, "test": {"score": [26.688029946402747, 28.060573405552564, 29.642968550605453, 30.910586166803732, 32.14700961347589, 33.60462808042424, 35.23523239656297, 36.579417519779945, 38.139125996086555, 39.42092278025125, 40.870033746419764, 42.01854635170008, 42.98840144060347, 44.09154070839123, 45.09826163400732, 46.0681167229107, 47.06349431415365, 47.96245356322492, 48.88977114822902, 49.79156623089357, 50.837988826815646, 51.57530556106968, 52.38635396874911, 53.174715707682275, 53.99143578254828, 54.734424183988885, 55.3441284065451, 56.22607265405666, 56.92085188441143, 57.436973598389244, 57.961602813146925, 58.66205371068827, 59.18951875903922, 59.802058815188715, 60.51668888069648, 61.06400476420044, 61.685052321129795, 62.29192071009273, 62.82505742563027, 63.32132830445509, 63.79491251453364, 64.3790942347503, 65.02282846042596, 65.55312934237018, 66.0352210532286, 66.47761109378102, 66.9483594702663, 67.413436179565, 67.84731871933755, 68.3038879278564, 68.78314380512151, 69.19150384255452, 69.63389388310695, 70.11031392677877, 70.47046479312594, 70.83912316025295, 71.26449819924568, 71.67002240308538, 72.03584493661911, 72.39599580296628, 72.82420667555229, 73.19853670986586, 73.4168958965488, 73.68062842072428, 73.93018177693334, 74.18257096673567, 74.42361682216487, 74.56824433542239, 74.76675268695233, 74.863171029124, 75.01630604316138, 75.08720188299351, 75.12973938689278, 75.2006352267249, 75.24317273062418, 75.26302356577716, 75.24884439781074, 75.25168023140402, 75.25451606499732, 75.25168023140402, 75.24600856421746, 75.24884439781074, 75.24600856421746, 75.24884439781074, 75.24033689703089, 75.25168023140402, 75.25168023140402, 75.24884439781074, 75.24600856421746, 75.24884439781074, 75.23182939625104, 75.24033689703089, 75.23182939625104, 75.21765022828461, 75.19779939313162, 75.18362022516519, 75.17511272438533, 75.13257522048606, 75.08720188299351, 74.883021864277, 74.47182599325072], "calls": [0.0, 1.4122451294558036, 3.096730283866943, 4.474945410203329, 5.821966367013584, 7.469585684712021, 9.32705668831353, 10.89810849899328, 12.78110200493435, 14.326631313274538, 16.158579814536484, 17.647392451010973, 18.903666732836115, 20.372628534157617, 21.816067833139552, 23.17159629072966, 24.609363922525027, 25.916683209029294, 27.286390834585827, 28.602217621869947, 30.18177693332955, 31.378498709695712, 32.64044465870743, 33.87970393897286, 35.21538156140998, 36.30150582763804, 37.38763009386609, 38.80554689050847, 39.92286532626266, 40.78212290502793, 41.85123216969628, 43.05929728043558, 44.05751070527181, 45.163485806652865, 46.37438675098545, 47.420809346907525, 48.60618778890055, 49.7121628902816, 50.6961971471514, 51.69157473839435, 52.76635567024927, 53.82412160054449, 55.000992541757654, 56.070101806425996, 57.07115106485552, 57.95025947877378, 58.93429373564359, 60.02325383546494, 60.964750588435464, 61.9629640132717, 63.03490911153333, 63.97357003091059, 65.0313359612058, 66.19969940163911, 67.26880866630745, 68.28687292629668, 69.33613135581204, 70.4222556220401, 71.38076737657035, 72.63704165839549, 73.81674843320194, 74.97376853926212, 75.83586195162067, 76.74616453506509, 77.77840796302074, 78.68587471287185, 79.69543147208121, 80.56319655162635, 81.6124549811417, 82.46036922553385, 83.39335847772452, 84.12783937838527, 84.79993193999375, 85.68187618750531, 86.41919292175935, 87.21889799506565, 88.07531974023765, 88.96009982134248, 89.47054986813374, 89.88174573916002, 90.30995661174602, 90.73816748433202, 91.07846751552619, 91.43294671468678, 91.77324674588095, 92.14757678019454, 92.47936931060885, 92.76295266993732, 93.2705668831353, 93.6250460822959, 94.07310779003488, 94.58072200323285, 95.06564954768454, 95.51087542183025, 96.06953463970734, 96.52893968181947, 97.13864390437568, 97.54983977540198, 98.0659614893798, 98.85148739471967, 100.0], "exact_score": [26.688029946402747, 27.6635567024927, 28.62206845702294, 29.549386042027052, 30.48237529421774, 31.39834954484871, 32.294472960326694, 33.21044721095766, 34.07537645690951, 34.94881320364121, 35.816578283186345, 36.675835861951626, 37.48688426963106, 38.3347985140232, 39.1515185888892, 39.92853699344923, 40.75660040268837, 41.52227547287525, 42.29078637665542, 43.064968947622155, 43.79661401468962, 44.51407991379066, 45.22870997929842, 45.92348920965318, 46.660805943907214, 47.310211836769426, 48.02200606868389, 48.688426963105805, 49.41156452939342, 50.06664208944219, 50.704704647931266, 51.348438873606895, 51.95814309616311, 52.596205654652195, 53.25411904829424, 53.86382327085046, 54.54158749964553, 55.13994838782861, 55.7354734424184, 56.34801349856791, 56.971896889090544, 57.522048606187795, 58.0410061537589, 58.63085954116213, 59.16966792388622, 59.714147973796905, 60.292658026827, 60.86266057907722, 61.36460312508862, 61.90341150781272, 62.450727391316676, 62.98953577404078, 63.47162748489919, 63.92536085982474, 64.48402007770184, 65.01999262683266, 65.51342767206421, 65.99835521651589, 66.49746192893402, 66.96821030541929, 67.40776451237842, 67.85582622011741, 68.31523126222953, 68.77180047074837, 69.20851884411422, 69.61687888154724, 70.03658225335337, 70.38255395173411, 70.7370331508947, 71.11703485239487, 71.52823072342115, 71.85718742024218, 72.2059949522162, 72.51793664747753, 72.87525168023141, 73.20420837705244, 73.4736125684145, 73.73734509258996, 73.97555511442589, 74.26481014094092, 74.46899015965744, 74.65615517681424, 74.82630519241131, 74.92839520176956, 75.0701868814338, 75.15809772282563, 75.22332189547117, 75.2573518985906, 75.24884439781074, 75.25451606499732, 75.24884439781074, 75.25168023140402, 75.24600856421746, 75.24600856421746, 75.2375010634376, 75.23466522984432, 75.19779939313162, 75.17227689079205, 75.08436604940023, 74.83764852678445, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 42.41839888835324, "exact_area100": 58.99763673483781}}, "old_normalized_endpoint_budget": 77}, {"seed": 2030, "selection": {"best_epoch": 10, "epochs": 18, "alpha": 0.0, "validation_area100": 0.7533904978405565, "training_seconds": 125.012417144957, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9967598915100098, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9937601685523987, "operator": "ge"}, {"budget": 0.03, "threshold": 0.99077308177948, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9871912002563477, "operator": "ge"}, {"budget": 0.05, "threshold": 0.982589602470398, "operator": "gt"}, {"budget": 0.06, "threshold": 0.9785684943199158, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9740803241729736, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9685916900634766, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9635497331619263, "operator": "gt"}, {"budget": 0.1, "threshold": 0.9582695960998535, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9526731371879578, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9463918805122375, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9409440159797668, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9358049631118774, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9292457699775696, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9214676022529602, "operator": "ge"}, {"budget": 0.17, "threshold": 0.9158018231391907, "operator": "gt"}, {"budget": 0.18, "threshold": 0.9091598987579346, "operator": "ge"}, {"budget": 0.19, "threshold": 0.9025521278381348, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8960803151130676, "operator": "ge"}, {"budget": 0.21, "threshold": 0.888519287109375, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8816622495651245, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8748523592948914, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8679065704345703, "operator": "ge"}, {"budget": 0.25, "threshold": 0.8599172830581665, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8505688905715942, "operator": "ge"}, {"budget": 0.27, "threshold": 0.8412942290306091, "operator": "ge"}, {"budget": 0.28, "threshold": 0.8326643705368042, "operator": "ge"}, {"budget": 0.29, "threshold": 0.824346661567688, "operator": "ge"}, {"budget": 0.3, "threshold": 0.8163280487060547, "operator": "ge"}, {"budget": 0.31, "threshold": 0.8066961169242859, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7971821427345276, "operator": "ge"}, {"budget": 0.33, "threshold": 0.7877359986305237, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7792707681655884, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7690312266349792, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7577517032623291, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7468674182891846, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7362363338470459, "operator": "ge"}, {"budget": 0.39, "threshold": 0.7244880795478821, "operator": "ge"}, {"budget": 0.4, "threshold": 0.7105474472045898, "operator": "ge"}, {"budget": 0.41, "threshold": 0.69770747423172, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6856735944747925, "operator": "ge"}, {"budget": 0.43, "threshold": 0.673361599445343, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6597388982772827, "operator": "ge"}, {"budget": 0.45, "threshold": 0.6446368098258972, "operator": "ge"}, {"budget": 0.46, "threshold": 0.630344569683075, "operator": "ge"}, {"budget": 0.47, "threshold": 0.614311933517456, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5964605808258057, "operator": "ge"}, {"budget": 0.49, "threshold": 0.5804252624511719, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5635330677032471, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5458479523658752, "operator": "ge"}, {"budget": 0.52, "threshold": 0.5263619422912598, "operator": "ge"}, {"budget": 0.53, "threshold": 0.5075632929801941, "operator": "ge"}, {"budget": 0.54, "threshold": 0.48848843574523926, "operator": "ge"}, {"budget": 0.55, "threshold": 0.4670047461986542, "operator": "ge"}, {"budget": 0.56, "threshold": 0.44491004943847656, "operator": "ge"}, {"budget": 0.57, "threshold": 0.4220549762248993, "operator": "ge"}, {"budget": 0.58, "threshold": 0.4002220630645752, "operator": "ge"}, {"budget": 0.59, "threshold": 0.37942296266555786, "operator": "ge"}, {"budget": 0.6, "threshold": 0.355282723903656, "operator": "ge"}, {"budget": 0.61, "threshold": 0.33406952023506165, "operator": "ge"}, {"budget": 0.62, "threshold": 0.30724185705184937, "operator": "ge"}, {"budget": 0.63, "threshold": 0.2812000513076782, "operator": "ge"}, {"budget": 0.64, "threshold": 0.25384393334388733, "operator": "ge"}, {"budget": 0.65, "threshold": 0.22369873523712158, "operator": "ge"}, {"budget": 0.66, "threshold": 0.18935877084732056, "operator": "ge"}, {"budget": 0.67, "threshold": 0.15926989912986755, "operator": "ge"}, {"budget": 0.68, "threshold": 0.13482369482517242, "operator": "ge"}, {"budget": 0.69, "threshold": 0.11018542945384979, "operator": "ge"}, {"budget": 0.7, "threshold": 0.08518518507480621, "operator": "ge"}, {"budget": 0.71, "threshold": 0.06320127099752426, "operator": "ge"}, {"budget": 0.72, "threshold": 0.043827250599861145, "operator": "ge"}, {"budget": 0.73, "threshold": 0.029215632006525993, "operator": "ge"}, {"budget": 0.74, "threshold": 0.01613093912601471, "operator": "ge"}, {"budget": 0.75, "threshold": 0.00806256290525198, "operator": "ge"}, {"budget": 0.76, "threshold": 0.0033720792271196842, "operator": "ge"}, {"budget": 0.77, "threshold": 0.0009468437638133764, "operator": "ge"}, {"budget": 0.78, "threshold": 1.7360958736389875e-05, "operator": "ge"}, {"budget": 0.79, "threshold": -0.00011777919280575588, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0002999650896526873, "operator": "ge"}, {"budget": 0.81, "threshold": -0.00044979839003644884, "operator": "ge"}, {"budget": 0.82, "threshold": -0.0006581878405995667, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0008978879195638001, "operator": "ge"}, {"budget": 0.84, "threshold": -0.0012115653371438384, "operator": "ge"}, {"budget": 0.85, "threshold": -0.0015670637367293239, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0019866398070007563, "operator": "ge"}, {"budget": 0.87, "threshold": -0.002524170558899641, "operator": "ge"}, {"budget": 0.88, "threshold": -0.003286577295511961, "operator": "ge"}, {"budget": 0.89, "threshold": -0.004067749250680208, "operator": "ge"}, {"budget": 0.9, "threshold": -0.005162482149899006, "operator": "ge"}, {"budget": 0.91, "threshold": -0.006668877322226763, "operator": "ge"}, {"budget": 0.92, "threshold": -0.008884547278285027, "operator": "ge"}, {"budget": 0.93, "threshold": -0.01181841641664505, "operator": "ge"}, {"budget": 0.94, "threshold": -0.016028083860874176, "operator": "ge"}, {"budget": 0.95, "threshold": -0.021989408880472183, "operator": "ge"}, {"budget": 0.96, "threshold": -0.030595770105719566, "operator": "ge"}, {"budget": 0.97, "threshold": -0.049442365765571594, "operator": "ge"}, {"budget": 0.98, "threshold": -0.07469181716442108, "operator": "ge"}, {"budget": 0.99, "threshold": -0.14000016450881958, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.01033101412608, 57.93098601447747, 58.865696816360945, 59.73012861058401, 60.51725349638063, 61.29735048141119, 62.07744746644177, 62.80132124534401, 63.47599971888397, 64.15770609318996, 64.69885445217514, 65.31730971958676, 65.93576498699838, 66.57530395670813, 67.1164523156933, 67.60840536931619, 68.13549792676928, 68.69772998805257, 69.15454353784524, 69.58324548457375, 70.06114273666455, 70.5460678895214, 71.03802094314428, 71.44563918757467, 71.80406212664276, 72.10626185958253, 72.46468479865064, 72.79499613465458, 73.18855857755288, 73.45561880666244, 73.79998594419847, 74.12326937943637, 74.41844121161009, 74.62927823459133, 74.91039426523297, 75.11420338744816, 75.40234731885586, 75.71157495256166, 75.94349567784103, 76.25975121231288, 76.44950453299599, 76.56897884601868, 76.74467636516972, 76.96956918968304, 77.08904350270574, 77.29988052568697, 77.49666174713612, 77.65127556398903, 77.93239159463067, 78.11511701454775, 78.23459132757044, 78.43137254901961, 78.60707006817063, 78.6843769765971, 78.81087919038583, 78.91629770187645, 79.12713472485768, 79.28877644247663, 79.4223065570314, 79.56989247311827, 79.6612551830768, 79.79478529763159, 79.97048281678263, 80.07590132827325, 80.18131983976386, 80.28673835125447, 80.41324056504322, 80.48351957270363, 80.58893808419425, 80.68732869491882, 80.79977510717549, 80.9051936186661, 81.00358422939068, 81.10900274088128, 81.13008644317942, 81.1792817485417, 81.2144212523719, 81.19333755007378, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.19333755007378, 81.18630964930776, 81.1792817485417, 81.17225384777566, 81.17225384777566, 81.16522594700962, 81.16522594700962, 81.13711434394546, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.08791903858317, 81.08089113781713, 81.059807435519, 81.01061213015672, 80.94033312249631, 80.80680300794153, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.9898095438892405, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.988685079766674, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.98643615152154, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 137, 270, 405, 532, 649, 768, 886, 996, 1101, 1209, 1303, 1404, 1506, 1609, 1697, 1783, 1869, 1960, 2037, 2113, 2193, 2272, 2358, 2433, 2500, 2560, 2625, 2695, 2768, 2822, 2886, 2946, 3007, 3060, 3113, 3164, 3220, 3281, 3335, 3392, 3440, 3482, 3533, 3588, 3628, 3676, 3718, 3760, 3811, 3854, 3895, 3937, 3970, 4001, 4035, 4068, 4114, 4149, 4190, 4220, 4252, 4284, 4317, 4345, 4373, 4397, 4423, 4443, 4467, 4493, 4515, 4540, 4558, 4575, 4586, 4596, 4604, 4604, 4605, 4606, 4607, 4607, 4607, 4607, 4607, 4607, 4607, 4607, 4607, 4608, 4608, 4609, 4610, 4612, 4617, 4618, 4621, 4622, 4625, 4626], "harmed": [0, 1, 3, 5, 9, 14, 22, 29, 36, 45, 56, 73, 86, 100, 112, 123, 139, 150, 161, 173, 188, 200, 210, 226, 243, 259, 276, 290, 313, 330, 346, 361, 375, 394, 417, 430, 452, 467, 484, 505, 517, 538, 563, 589, 612, 635, 653, 667, 687, 698, 715, 739, 753, 761, 781, 797, 815, 831, 843, 865, 874, 893, 906, 914, 927, 940, 949, 957, 967, 976, 988, 994, 1004, 1008, 1010, 1018, 1021, 1024, 1027, 1028, 1028, 1029, 1029, 1030, 1031, 1032, 1033, 1033, 1034, 1034, 1039, 1041, 1044, 1047, 1050, 1056, 1060, 1070, 1081, 1103, 1133], "exact_score": [56.05453650994448, 57.01033101412608, 57.93098601447747, 58.865696816360945, 59.73012861058401, 60.52428139714667, 61.29735048141119, 62.07744746644177, 62.80132124534401, 63.48302761965, 64.15770609318996, 64.69885445217514, 65.31730971958676, 65.93576498699838, 66.57530395670813, 67.1164523156933, 67.60840536931619, 68.14252582753531, 68.69772998805257, 69.15454353784524, 69.58324548457375, 70.06114273666455, 70.5460678895214, 71.03802094314428, 71.44563918757467, 71.80406212664276, 72.10626185958253, 72.46468479865064, 72.79499613465458, 73.18855857755288, 73.45561880666244, 73.79998594419847, 74.12326937943637, 74.41844121161009, 74.62927823459133, 74.91039426523297, 75.11420338744816, 75.40234731885586, 75.71157495256166, 75.94349567784103, 76.25975121231288, 76.44950453299599, 76.56897884601868, 76.74467636516972, 76.96956918968304, 77.08904350270574, 77.29988052568697, 77.49666174713612, 77.65127556398903, 77.93239159463067, 78.11511701454775, 78.23459132757044, 78.43137254901961, 78.60707006817063, 78.6843769765971, 78.81087919038583, 78.91629770187645, 79.12713472485768, 79.28877644247663, 79.4223065570314, 79.56989247311827, 79.6612551830768, 79.79478529763159, 79.97048281678263, 80.07590132827325, 80.18131983976386, 80.28673835125447, 80.41324056504322, 80.48351957270363, 80.58893808419425, 80.68732869491882, 80.79977510717549, 80.9051936186661, 81.00358422939068, 81.10900274088128, 81.13008644317942, 81.1792817485417, 81.2144212523719, 81.19333755007378, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.19333755007378, 81.18630964930776, 81.1792817485417, 81.17225384777566, 81.17225384777566, 81.16522594700962, 81.16522594700962, 81.13711434394546, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.08791903858317, 81.08089113781713, 81.059807435519, 81.01061213015672, 80.94033312249631, 80.80680300794153, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.33904978405565, "threshold_area100": 75.33800688734274, "threshold_area25": 65.06079134162626}, "test": {"score": [26.688029946402747, 27.8677367212092, 29.197742676459747, 30.507897796557298, 31.840739585401128, 33.41746306326745, 34.690752346652296, 35.96971329722372, 37.31673425403398, 38.507784363213574, 39.764058645038716, 40.95794458781159, 42.236905538383006, 43.28332813430508, 44.14825738025693, 45.257068315231265, 46.48214842753028, 47.256330998497006, 48.24036525536682, 49.10529450131867, 49.8794770722854, 50.744406318237246, 51.50157388764427, 52.21903978674531, 52.97620735615234, 53.73337492555936, 54.58979667073136, 55.32994923857868, 55.97368346425432, 56.58055185321725, 57.159061906247345, 57.868020304568525, 58.49757536227774, 59.10160791764739, 59.52981879023339, 60.1054930096702, 60.76907807049883, 61.32490145478264, 61.87505317187987, 62.379831551484564, 63.017894109973625, 63.60491166378357, 64.16357088166068, 64.60028925502652, 65.04835096276551, 65.55880100955676, 65.96716104698976, 66.53149193205343, 67.02776281087826, 67.50134702095681, 67.99478206618836, 68.45702294189377, 68.90508464963277, 69.42404219720387, 69.91464140884214, 70.41091228766696, 70.94972067039107, 71.3524090406375, 71.72957490854436, 72.10106910926467, 72.46972747639168, 72.80435584039928, 73.13898420440688, 73.48779173638091, 73.73734509258996, 74.02660011910501, 74.28182514250064, 74.45197515809772, 74.61078183932167, 74.73839435101948, 74.87167852990386, 74.95675353770241, 75.04182854550095, 75.07869438221365, 75.09854521736665, 75.14675438845248, 75.15809772282563, 75.15242605563905, 75.15526188923234, 75.15242605563905, 75.15526188923234, 75.15526188923234, 75.15526188923234, 75.15526188923234, 75.15809772282563, 75.15526188923234, 75.15242605563905, 75.14959022204577, 75.14108272126592, 75.1240677197062, 75.13257522048606, 75.12123188611292, 75.1269035532995, 75.1269035532995, 75.11556021892636, 75.10421688455322, 75.09287355018009, 75.07302271502708, 75.01063437597482, 74.90570853302329, 74.47182599325072], "calls": [0.0, 1.250602614638573, 2.6798627456540847, 4.08076454073675, 5.558233842838102, 7.330629838641069, 8.768397470436435, 10.268553441284066, 11.958710262881773, 13.529762073561525, 15.066783881121856, 16.510223180103793, 18.132319995462666, 19.445310949153505, 20.593823554433826, 22.130845361994158, 23.826673850778437, 24.881603947480365, 26.299520744122734, 27.56430252672773, 28.761024303093897, 30.09670192553101, 31.282080367524035, 32.35118963219239, 33.47985140231971, 34.67373734509259, 36.00090746674985, 37.251510081388425, 38.43405268978816, 39.52017695601622, 40.46734537617333, 41.59317131270737, 42.70481808127499, 43.799449848282904, 44.64736409267504, 45.6767716870374, 46.95006097042226, 48.02767773587046, 49.0797719989791, 50.1573887644273, 51.29455803533449, 52.375010634375975, 53.506508238096586, 54.51039333011939, 55.62487593228029, 56.71667186569492, 57.765930295210275, 58.98250290672943, 60.187732183875454, 61.34758812352891, 62.48759322802938, 63.621926665343274, 64.78178260499674, 65.92745937668377, 67.12985282023651, 68.27836542551682, 69.48359470266284, 70.61509230638346, 71.59061906247342, 72.49241414513796, 73.5813742449593, 74.52570683152314, 75.54944275869892, 76.56467118509485, 77.62810878257665, 78.657516376939, 79.74364064316705, 80.53200238210022, 81.36857329211922, 82.12290502793296, 83.0275359441908, 83.8073901823441, 84.63545359158324, 85.43515866488954, 86.33695374755409, 86.99770297478943, 87.66412386921135, 88.31352976207356, 89.28338485097694, 89.8108498993279, 90.41771828829084, 90.82891415931714, 91.26846836627627, 91.72503757479511, 92.24399512236621, 92.62966849105294, 93.0295210277061, 93.46340356747866, 93.9029577744378, 94.27728780875138, 94.67997617899782, 95.12520205314352, 95.64132376712135, 96.02416130221478, 96.50341717947991, 96.9316280520659, 97.37685392621161, 97.84476646910359, 98.28432067606273, 98.84014406034653, 100.0], "rescued": [0, 420, 898, 1368, 1853, 2424, 2888, 3356, 3869, 4321, 4788, 5231, 5709, 6105, 6434, 6855, 7314, 7612, 7985, 8322, 8632, 8970, 9267, 9549, 9837, 10131, 10460, 10754, 11013, 11261, 11485, 11756, 12001, 12243, 12412, 12634, 12896, 13120, 13340, 13545, 13793, 14019, 14235, 14409, 14595, 14798, 14982, 15212, 15415, 15613, 15803, 15992, 16174, 16374, 16571, 16763, 16967, 17134, 17281, 17420, 17569, 17704, 17835, 17969, 18073, 18195, 18304, 18379, 18442, 18499, 18560, 18604, 18649, 18680, 18704, 18729, 18741, 18745, 18749, 18749, 18750, 18751, 18751, 18751, 18752, 18752, 18752, 18754, 18754, 18754, 18758, 18760, 18765, 18768, 18771, 18775, 18785, 18791, 18796, 18807, 18820], "harmed": [0, 4, 13, 21, 36, 51, 66, 83, 121, 153, 177, 199, 226, 253, 277, 307, 334, 359, 385, 417, 454, 487, 517, 546, 567, 594, 621, 654, 686, 720, 740, 761, 784, 813, 831, 850, 878, 906, 932, 959, 982, 1001, 1020, 1040, 1068, 1091, 1131, 1162, 1190, 1221, 1237, 1263, 1287, 1304, 1328, 1345, 1359, 1384, 1398, 1406, 1425, 1442, 1455, 1466, 1482, 1502, 1521, 1536, 1543, 1555, 1569, 1583, 1598, 1616, 1633, 1641, 1649, 1655, 1658, 1659, 1659, 1660, 1660, 1660, 1660, 1661, 1662, 1665, 1668, 1674, 1675, 1681, 1684, 1687, 1694, 1702, 1716, 1729, 1756, 1804, 1970], "exact_score": [26.688029946402747, 27.63803420015314, 28.582366786716957, 29.495505203754647, 30.425658622352042, 31.347304540169585, 32.21223378612143, 33.10268553441284, 34.01298811785725, 34.8949323653688, 35.74851827694751, 36.56240251822023, 37.353600090746674, 38.09375265859399, 38.87077106315402, 39.7101778067663, 40.51271871366588, 41.35779712446473, 42.14048719621133, 42.90899809999149, 43.70303150611122, 44.47437824348467, 45.17199330743272, 45.84692170263449, 46.61543260641466, 47.33573433910898, 48.053200238210025, 48.713949465445374, 49.380370359867285, 50.03261208632277, 50.67067464481184, 51.34560304001361, 51.99217309928253, 52.66426566089102, 53.302328219380094, 53.93188327708931, 54.58979667073136, 55.18248589172787, 55.73830927601169, 56.2941326602955, 56.88115021410544, 57.49652610384823, 58.07220032328504, 58.653546209908406, 59.1923545926325, 59.71131214020362, 60.28131469245385, 60.780421404871966, 61.319229787596065, 61.82967983438732, 62.320279046025576, 62.878938263902675, 63.380880809914075, 63.919689192638174, 64.36491506678388, 64.81864844170944, 65.25253098148201, 65.68924935484786, 66.09477355868758, 66.53999943283328, 66.95119530385958, 67.3567195076993, 67.78493038028527, 68.190454584125, 68.62433712389758, 69.02135382695744, 69.45523636673, 69.84090973541672, 70.271956441596, 70.74270481808126, 71.16240818988742, 71.49703655389501, 71.8742024218019, 72.25420412330205, 72.60584748886936, 72.94331168647024, 73.29779088563083, 73.59838924651902, 73.85361426991463, 74.12585429486998, 74.33003431358648, 74.54555766667612, 74.71854351586649, 74.87167852990386, 74.97944020644869, 75.05600771346738, 75.08436604940023, 75.14675438845248, 75.14675438845248, 75.15526188923234, 75.15242605563905, 75.15526188923234, 75.15526188923234, 75.15242605563905, 75.13824688767264, 75.12973938689278, 75.1269035532995, 75.09854521736665, 75.05884354706066, 74.863171029124, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.85652665082966, "threshold_area100": 62.73898987607407, "threshold_area25": 41.705753906360776}, "selected_budget": 77, "selected": {"val": {"score": 81.2144212523719, "calls": 76.9976807927472, "rescued": 4604, "harmed": 1024}, "test": {"score": 75.15242605563905, "calls": 88.31352976207356, "rescued": 18745, "harmed": 1655}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9967598915100098, "operator": "ge"}, {"budget": 0.02, "threshold": 0.9937601685523987, "operator": "ge"}, {"budget": 0.03, "threshold": 0.99077308177948, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9871912002563477, "operator": "ge"}, {"budget": 0.05, "threshold": 0.982589602470398, "operator": "gt"}, {"budget": 0.06, "threshold": 0.9785684943199158, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9740803241729736, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9685916900634766, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9635497331619263, "operator": "gt"}, {"budget": 0.1, "threshold": 0.9582695960998535, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9526731371879578, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9463918805122375, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9409440159797668, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9358049631118774, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9292457699775696, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9214676022529602, "operator": "ge"}, {"budget": 0.17, "threshold": 0.9158018231391907, "operator": "gt"}, {"budget": 0.18, "threshold": 0.9091598987579346, "operator": "ge"}, {"budget": 0.19, "threshold": 0.9025521278381348, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8960803151130676, "operator": "ge"}, {"budget": 0.21, "threshold": 0.888519287109375, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8816622495651245, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8748523592948914, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8679065704345703, "operator": "ge"}, {"budget": 0.25, "threshold": 0.8599172830581665, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8505688905715942, "operator": "ge"}, {"budget": 0.27, "threshold": 0.8412942290306091, "operator": "ge"}, {"budget": 0.28, "threshold": 0.8326643705368042, "operator": "ge"}, {"budget": 0.29, "threshold": 0.824346661567688, "operator": "ge"}, {"budget": 0.3, "threshold": 0.8163280487060547, "operator": "ge"}, {"budget": 0.31, "threshold": 0.8066961169242859, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7971821427345276, "operator": "ge"}, {"budget": 0.33, "threshold": 0.7877359986305237, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7792707681655884, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7690312266349792, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7577517032623291, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7468674182891846, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7362363338470459, "operator": "ge"}, {"budget": 0.39, "threshold": 0.7244880795478821, "operator": "ge"}, {"budget": 0.4, "threshold": 0.7105474472045898, "operator": "ge"}, {"budget": 0.41, "threshold": 0.69770747423172, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6856735944747925, "operator": "ge"}, {"budget": 0.43, "threshold": 0.673361599445343, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6597388982772827, "operator": "ge"}, {"budget": 0.45, "threshold": 0.6446368098258972, "operator": "ge"}, {"budget": 0.46, "threshold": 0.630344569683075, "operator": "ge"}, {"budget": 0.47, "threshold": 0.614311933517456, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5964605808258057, "operator": "ge"}, {"budget": 0.49, "threshold": 0.5804252624511719, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5635330677032471, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5458479523658752, "operator": "ge"}, {"budget": 0.52, "threshold": 0.5263619422912598, "operator": "ge"}, {"budget": 0.53, "threshold": 0.5075632929801941, "operator": "ge"}, {"budget": 0.54, "threshold": 0.48848843574523926, "operator": "ge"}, {"budget": 0.55, "threshold": 0.4670047461986542, "operator": "ge"}, {"budget": 0.56, "threshold": 0.44491004943847656, "operator": "ge"}, {"budget": 0.57, "threshold": 0.4220549762248993, "operator": "ge"}, {"budget": 0.58, "threshold": 0.4002220630645752, "operator": "ge"}, {"budget": 0.59, "threshold": 0.37942296266555786, "operator": "ge"}, {"budget": 0.6, "threshold": 0.355282723903656, "operator": "ge"}, {"budget": 0.61, "threshold": 0.33406952023506165, "operator": "ge"}, {"budget": 0.62, "threshold": 0.30724185705184937, "operator": "ge"}, {"budget": 0.63, "threshold": 0.2812000513076782, "operator": "ge"}, {"budget": 0.64, "threshold": 0.25384393334388733, "operator": "ge"}, {"budget": 0.65, "threshold": 0.22369873523712158, "operator": "ge"}, {"budget": 0.66, "threshold": 0.18935877084732056, "operator": "ge"}, {"budget": 0.67, "threshold": 0.15926989912986755, "operator": "ge"}, {"budget": 0.68, "threshold": 0.13482369482517242, "operator": "ge"}, {"budget": 0.69, "threshold": 0.11018542945384979, "operator": "ge"}, {"budget": 0.7, "threshold": 0.08518518507480621, "operator": "ge"}, {"budget": 0.71, "threshold": 0.06320127099752426, "operator": "ge"}, {"budget": 0.72, "threshold": 0.043827250599861145, "operator": "ge"}, {"budget": 0.73, "threshold": 0.029215632006525993, "operator": "ge"}, {"budget": 0.74, "threshold": 0.01613093912601471, "operator": "ge"}, {"budget": 0.75, "threshold": 0.00806256290525198, "operator": "ge"}, {"budget": 0.76, "threshold": 0.0033720792271196842, "operator": "ge"}, {"budget": 0.77, "threshold": 0.0009468437638133764, "operator": "ge"}, {"budget": 0.78, "threshold": 1.7360958736389875e-05, "operator": "ge"}, {"budget": 0.79, "threshold": -0.00011777919280575588, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0002999650896526873, "operator": "ge"}, {"budget": 0.81, "threshold": -0.00044979839003644884, "operator": "ge"}, {"budget": 0.82, "threshold": -0.0006581878405995667, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0008978879195638001, "operator": "ge"}, {"budget": 0.84, "threshold": -0.0012115653371438384, "operator": "ge"}, {"budget": 0.85, "threshold": -0.0015670637367293239, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0019866398070007563, "operator": "ge"}, {"budget": 0.87, "threshold": -0.002524170558899641, "operator": "ge"}, {"budget": 0.88, "threshold": -0.003286577295511961, "operator": "ge"}, {"budget": 0.89, "threshold": -0.004067749250680208, "operator": "ge"}, {"budget": 0.9, "threshold": -0.005162482149899006, "operator": "ge"}, {"budget": 0.91, "threshold": -0.006668877322226763, "operator": "ge"}, {"budget": 0.92, "threshold": -0.008884547278285027, "operator": "ge"}, {"budget": 0.93, "threshold": -0.01181841641664505, "operator": "ge"}, {"budget": 0.94, "threshold": -0.016028083860874176, "operator": "ge"}, {"budget": 0.95, "threshold": -0.021989408880472183, "operator": "ge"}, {"budget": 0.96, "threshold": -0.030595770105719566, "operator": "ge"}, {"budget": 0.97, "threshold": -0.049442365765571594, "operator": "ge"}, {"budget": 0.98, "threshold": -0.07469181716442108, "operator": "ge"}, {"budget": 0.99, "threshold": -0.14000016450881958, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 57.01033101412608, 57.93098601447747, 58.865696816360945, 59.73012861058401, 60.51725349638063, 61.29735048141119, 62.07744746644177, 62.80132124534401, 63.47599971888397, 64.15770609318996, 64.69885445217514, 65.31730971958676, 65.93576498699838, 66.57530395670813, 67.1164523156933, 67.60840536931619, 68.13549792676928, 68.69772998805257, 69.15454353784524, 69.58324548457375, 70.06114273666455, 70.5460678895214, 71.03802094314428, 71.44563918757467, 71.80406212664276, 72.10626185958253, 72.46468479865064, 72.79499613465458, 73.18855857755288, 73.45561880666244, 73.79998594419847, 74.12326937943637, 74.41844121161009, 74.62927823459133, 74.91039426523297, 75.11420338744816, 75.40234731885586, 75.71157495256166, 75.94349567784103, 76.25975121231288, 76.44950453299599, 76.56897884601868, 76.74467636516972, 76.96956918968304, 77.08904350270574, 77.29988052568697, 77.49666174713612, 77.65127556398903, 77.93239159463067, 78.11511701454775, 78.23459132757044, 78.43137254901961, 78.60707006817063, 78.6843769765971, 78.81087919038583, 78.91629770187645, 79.12713472485768, 79.28877644247663, 79.4223065570314, 79.56989247311827, 79.6612551830768, 79.79478529763159, 79.97048281678263, 80.07590132827325, 80.18131983976386, 80.28673835125447, 80.41324056504322, 80.48351957270363, 80.58893808419425, 80.68732869491882, 80.79977510717549, 80.9051936186661, 81.00358422939068, 81.10900274088128, 81.13008644317942, 81.1792817485417, 81.2144212523719, 81.19333755007378, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.19333755007378, 81.18630964930776, 81.1792817485417, 81.17225384777566, 81.17225384777566, 81.16522594700962, 81.16522594700962, 81.13711434394546, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.08791903858317, 81.08089113781713, 81.059807435519, 81.01061213015672, 80.94033312249631, 80.80680300794153, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.9898095438892405, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.988685079766674, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.98643615152154, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 137, 270, 405, 532, 649, 768, 886, 996, 1101, 1209, 1303, 1404, 1506, 1609, 1697, 1783, 1869, 1960, 2037, 2113, 2193, 2272, 2358, 2433, 2500, 2560, 2625, 2695, 2768, 2822, 2886, 2946, 3007, 3060, 3113, 3164, 3220, 3281, 3335, 3392, 3440, 3482, 3533, 3588, 3628, 3676, 3718, 3760, 3811, 3854, 3895, 3937, 3970, 4001, 4035, 4068, 4114, 4149, 4190, 4220, 4252, 4284, 4317, 4345, 4373, 4397, 4423, 4443, 4467, 4493, 4515, 4540, 4558, 4575, 4586, 4596, 4604, 4604, 4605, 4606, 4607, 4607, 4607, 4607, 4607, 4607, 4607, 4607, 4607, 4608, 4608, 4609, 4610, 4612, 4617, 4618, 4621, 4622, 4625, 4626], "harmed": [0, 1, 3, 5, 9, 14, 22, 29, 36, 45, 56, 73, 86, 100, 112, 123, 139, 150, 161, 173, 188, 200, 210, 226, 243, 259, 276, 290, 313, 330, 346, 361, 375, 394, 417, 430, 452, 467, 484, 505, 517, 538, 563, 589, 612, 635, 653, 667, 687, 698, 715, 739, 753, 761, 781, 797, 815, 831, 843, 865, 874, 893, 906, 914, 927, 940, 949, 957, 967, 976, 988, 994, 1004, 1008, 1010, 1018, 1021, 1024, 1027, 1028, 1028, 1029, 1029, 1030, 1031, 1032, 1033, 1033, 1034, 1034, 1039, 1041, 1044, 1047, 1050, 1056, 1060, 1070, 1081, 1103, 1133], "exact_score": [56.05453650994448, 57.01033101412608, 57.93098601447747, 58.865696816360945, 59.73012861058401, 60.52428139714667, 61.29735048141119, 62.07744746644177, 62.80132124534401, 63.48302761965, 64.15770609318996, 64.69885445217514, 65.31730971958676, 65.93576498699838, 66.57530395670813, 67.1164523156933, 67.60840536931619, 68.14252582753531, 68.69772998805257, 69.15454353784524, 69.58324548457375, 70.06114273666455, 70.5460678895214, 71.03802094314428, 71.44563918757467, 71.80406212664276, 72.10626185958253, 72.46468479865064, 72.79499613465458, 73.18855857755288, 73.45561880666244, 73.79998594419847, 74.12326937943637, 74.41844121161009, 74.62927823459133, 74.91039426523297, 75.11420338744816, 75.40234731885586, 75.71157495256166, 75.94349567784103, 76.25975121231288, 76.44950453299599, 76.56897884601868, 76.74467636516972, 76.96956918968304, 77.08904350270574, 77.29988052568697, 77.49666174713612, 77.65127556398903, 77.93239159463067, 78.11511701454775, 78.23459132757044, 78.43137254901961, 78.60707006817063, 78.6843769765971, 78.81087919038583, 78.91629770187645, 79.12713472485768, 79.28877644247663, 79.4223065570314, 79.56989247311827, 79.6612551830768, 79.79478529763159, 79.97048281678263, 80.07590132827325, 80.18131983976386, 80.28673835125447, 80.41324056504322, 80.48351957270363, 80.58893808419425, 80.68732869491882, 80.79977510717549, 80.9051936186661, 81.00358422939068, 81.10900274088128, 81.13008644317942, 81.1792817485417, 81.2144212523719, 81.19333755007378, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.19333755007378, 81.18630964930776, 81.1792817485417, 81.17225384777566, 81.17225384777566, 81.16522594700962, 81.16522594700962, 81.13711434394546, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.08791903858317, 81.08089113781713, 81.059807435519, 81.01061213015672, 80.94033312249631, 80.80680300794153, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 75.33904978405565, "threshold_area100": 75.33800688734274, "threshold_area25": 65.06079134162626}, "test": {"score": [26.688029946402747, 27.8677367212092, 29.197742676459747, 30.507897796557298, 31.840739585401128, 33.41746306326745, 34.690752346652296, 35.96971329722372, 37.31673425403398, 38.507784363213574, 39.764058645038716, 40.95794458781159, 42.236905538383006, 43.28332813430508, 44.14825738025693, 45.257068315231265, 46.48214842753028, 47.256330998497006, 48.24036525536682, 49.10529450131867, 49.8794770722854, 50.744406318237246, 51.50157388764427, 52.21903978674531, 52.97620735615234, 53.73337492555936, 54.58979667073136, 55.32994923857868, 55.97368346425432, 56.58055185321725, 57.159061906247345, 57.868020304568525, 58.49757536227774, 59.10160791764739, 59.52981879023339, 60.1054930096702, 60.76907807049883, 61.32490145478264, 61.87505317187987, 62.379831551484564, 63.017894109973625, 63.60491166378357, 64.16357088166068, 64.60028925502652, 65.04835096276551, 65.55880100955676, 65.96716104698976, 66.53149193205343, 67.02776281087826, 67.50134702095681, 67.99478206618836, 68.45702294189377, 68.90508464963277, 69.42404219720387, 69.91464140884214, 70.41091228766696, 70.94972067039107, 71.3524090406375, 71.72957490854436, 72.10106910926467, 72.46972747639168, 72.80435584039928, 73.13898420440688, 73.48779173638091, 73.73734509258996, 74.02660011910501, 74.28182514250064, 74.45197515809772, 74.61078183932167, 74.73839435101948, 74.87167852990386, 74.95675353770241, 75.04182854550095, 75.07869438221365, 75.09854521736665, 75.14675438845248, 75.15809772282563, 75.15242605563905, 75.15526188923234, 75.15242605563905, 75.15526188923234, 75.15526188923234, 75.15526188923234, 75.15526188923234, 75.15809772282563, 75.15526188923234, 75.15242605563905, 75.14959022204577, 75.14108272126592, 75.1240677197062, 75.13257522048606, 75.12123188611292, 75.1269035532995, 75.1269035532995, 75.11556021892636, 75.10421688455322, 75.09287355018009, 75.07302271502708, 75.01063437597482, 74.90570853302329, 74.47182599325072], "calls": [0.0, 1.250602614638573, 2.6798627456540847, 4.08076454073675, 5.558233842838102, 7.330629838641069, 8.768397470436435, 10.268553441284066, 11.958710262881773, 13.529762073561525, 15.066783881121856, 16.510223180103793, 18.132319995462666, 19.445310949153505, 20.593823554433826, 22.130845361994158, 23.826673850778437, 24.881603947480365, 26.299520744122734, 27.56430252672773, 28.761024303093897, 30.09670192553101, 31.282080367524035, 32.35118963219239, 33.47985140231971, 34.67373734509259, 36.00090746674985, 37.251510081388425, 38.43405268978816, 39.52017695601622, 40.46734537617333, 41.59317131270737, 42.70481808127499, 43.799449848282904, 44.64736409267504, 45.6767716870374, 46.95006097042226, 48.02767773587046, 49.0797719989791, 50.1573887644273, 51.29455803533449, 52.375010634375975, 53.506508238096586, 54.51039333011939, 55.62487593228029, 56.71667186569492, 57.765930295210275, 58.98250290672943, 60.187732183875454, 61.34758812352891, 62.48759322802938, 63.621926665343274, 64.78178260499674, 65.92745937668377, 67.12985282023651, 68.27836542551682, 69.48359470266284, 70.61509230638346, 71.59061906247342, 72.49241414513796, 73.5813742449593, 74.52570683152314, 75.54944275869892, 76.56467118509485, 77.62810878257665, 78.657516376939, 79.74364064316705, 80.53200238210022, 81.36857329211922, 82.12290502793296, 83.0275359441908, 83.8073901823441, 84.63545359158324, 85.43515866488954, 86.33695374755409, 86.99770297478943, 87.66412386921135, 88.31352976207356, 89.28338485097694, 89.8108498993279, 90.41771828829084, 90.82891415931714, 91.26846836627627, 91.72503757479511, 92.24399512236621, 92.62966849105294, 93.0295210277061, 93.46340356747866, 93.9029577744378, 94.27728780875138, 94.67997617899782, 95.12520205314352, 95.64132376712135, 96.02416130221478, 96.50341717947991, 96.9316280520659, 97.37685392621161, 97.84476646910359, 98.28432067606273, 98.84014406034653, 100.0], "rescued": [0, 420, 898, 1368, 1853, 2424, 2888, 3356, 3869, 4321, 4788, 5231, 5709, 6105, 6434, 6855, 7314, 7612, 7985, 8322, 8632, 8970, 9267, 9549, 9837, 10131, 10460, 10754, 11013, 11261, 11485, 11756, 12001, 12243, 12412, 12634, 12896, 13120, 13340, 13545, 13793, 14019, 14235, 14409, 14595, 14798, 14982, 15212, 15415, 15613, 15803, 15992, 16174, 16374, 16571, 16763, 16967, 17134, 17281, 17420, 17569, 17704, 17835, 17969, 18073, 18195, 18304, 18379, 18442, 18499, 18560, 18604, 18649, 18680, 18704, 18729, 18741, 18745, 18749, 18749, 18750, 18751, 18751, 18751, 18752, 18752, 18752, 18754, 18754, 18754, 18758, 18760, 18765, 18768, 18771, 18775, 18785, 18791, 18796, 18807, 18820], "harmed": [0, 4, 13, 21, 36, 51, 66, 83, 121, 153, 177, 199, 226, 253, 277, 307, 334, 359, 385, 417, 454, 487, 517, 546, 567, 594, 621, 654, 686, 720, 740, 761, 784, 813, 831, 850, 878, 906, 932, 959, 982, 1001, 1020, 1040, 1068, 1091, 1131, 1162, 1190, 1221, 1237, 1263, 1287, 1304, 1328, 1345, 1359, 1384, 1398, 1406, 1425, 1442, 1455, 1466, 1482, 1502, 1521, 1536, 1543, 1555, 1569, 1583, 1598, 1616, 1633, 1641, 1649, 1655, 1658, 1659, 1659, 1660, 1660, 1660, 1660, 1661, 1662, 1665, 1668, 1674, 1675, 1681, 1684, 1687, 1694, 1702, 1716, 1729, 1756, 1804, 1970], "exact_score": [26.688029946402747, 27.63803420015314, 28.582366786716957, 29.495505203754647, 30.425658622352042, 31.347304540169585, 32.21223378612143, 33.10268553441284, 34.01298811785725, 34.8949323653688, 35.74851827694751, 36.56240251822023, 37.353600090746674, 38.09375265859399, 38.87077106315402, 39.7101778067663, 40.51271871366588, 41.35779712446473, 42.14048719621133, 42.90899809999149, 43.70303150611122, 44.47437824348467, 45.17199330743272, 45.84692170263449, 46.61543260641466, 47.33573433910898, 48.053200238210025, 48.713949465445374, 49.380370359867285, 50.03261208632277, 50.67067464481184, 51.34560304001361, 51.99217309928253, 52.66426566089102, 53.302328219380094, 53.93188327708931, 54.58979667073136, 55.18248589172787, 55.73830927601169, 56.2941326602955, 56.88115021410544, 57.49652610384823, 58.07220032328504, 58.653546209908406, 59.1923545926325, 59.71131214020362, 60.28131469245385, 60.780421404871966, 61.319229787596065, 61.82967983438732, 62.320279046025576, 62.878938263902675, 63.380880809914075, 63.919689192638174, 64.36491506678388, 64.81864844170944, 65.25253098148201, 65.68924935484786, 66.09477355868758, 66.53999943283328, 66.95119530385958, 67.3567195076993, 67.78493038028527, 68.190454584125, 68.62433712389758, 69.02135382695744, 69.45523636673, 69.84090973541672, 70.271956441596, 70.74270481808126, 71.16240818988742, 71.49703655389501, 71.8742024218019, 72.25420412330205, 72.60584748886936, 72.94331168647024, 73.29779088563083, 73.59838924651902, 73.85361426991463, 74.12585429486998, 74.33003431358648, 74.54555766667612, 74.71854351586649, 74.87167852990386, 74.97944020644869, 75.05600771346738, 75.08436604940023, 75.14675438845248, 75.14675438845248, 75.15526188923234, 75.15242605563905, 75.15526188923234, 75.15526188923234, 75.15242605563905, 75.13824688767264, 75.12973938689278, 75.1269035532995, 75.09854521736665, 75.05884354706066, 74.863171029124, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.85652665082966, "threshold_area100": 62.73898987607407, "threshold_area25": 41.705753906360776}, "selected_budget": 77, "selected": {"val": {"score": 81.2144212523719, "calls": 76.9976807927472, "rescued": 4604, "harmed": 1024}, "test": {"score": 75.15242605563905, "calls": 88.31352976207356, "rescued": 18745, "harmed": 1655}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 17.54019546508789, "operator": "ge"}, {"budget": 0.02, "threshold": 16.529525756835938, "operator": "ge"}, {"budget": 0.03, "threshold": 15.816265106201172, "operator": "ge"}, {"budget": 0.04, "threshold": 15.283686637878418, "operator": "ge"}, {"budget": 0.05, "threshold": 14.916730880737305, "operator": "ge"}, {"budget": 0.06, "threshold": 14.5411958694458, "operator": "ge"}, {"budget": 0.07, "threshold": 14.236766815185547, "operator": "ge"}, {"budget": 0.08, "threshold": 13.932340621948242, "operator": "ge"}, {"budget": 0.09, "threshold": 13.640451431274414, "operator": "ge"}, {"budget": 0.1, "threshold": 13.336616516113281, "operator": "ge"}, {"budget": 0.11, "threshold": 13.074666976928711, "operator": "ge"}, {"budget": 0.12, "threshold": 12.788263320922852, "operator": "ge"}, {"budget": 0.13, "threshold": 12.510702133178711, "operator": "ge"}, {"budget": 0.14, "threshold": 12.239437103271484, "operator": "ge"}, {"budget": 0.15, "threshold": 11.973392486572266, "operator": "ge"}, {"budget": 0.16, "threshold": 11.728887557983398, "operator": "ge"}, {"budget": 0.17, "threshold": 11.532155990600586, "operator": "ge"}, {"budget": 0.18, "threshold": 11.31263542175293, "operator": "ge"}, {"budget": 0.19, "threshold": 11.11290168762207, "operator": "ge"}, {"budget": 0.2, "threshold": 10.877655029296875, "operator": "ge"}, {"budget": 0.21, "threshold": 10.616968154907227, "operator": "ge"}, {"budget": 0.22, "threshold": 10.381548881530762, "operator": "ge"}, {"budget": 0.23, "threshold": 10.189513206481934, "operator": "ge"}, {"budget": 0.24, "threshold": 9.97066593170166, "operator": "ge"}, {"budget": 0.25, "threshold": 9.768073081970215, "operator": "ge"}, {"budget": 0.26, "threshold": 9.539064407348633, "operator": "ge"}, {"budget": 0.27, "threshold": 9.288617134094238, "operator": "ge"}, {"budget": 0.28, "threshold": 9.048038482666016, "operator": "ge"}, {"budget": 0.29, "threshold": 8.809770584106445, "operator": "ge"}, {"budget": 0.3, "threshold": 8.588784217834473, "operator": "ge"}, {"budget": 0.31, "threshold": 8.343320846557617, "operator": "ge"}, {"budget": 0.32, "threshold": 8.094608306884766, "operator": "ge"}, {"budget": 0.33, "threshold": 7.852621078491211, "operator": "ge"}, {"budget": 0.34, "threshold": 7.548157691955566, "operator": "ge"}, {"budget": 0.35, "threshold": 7.325734615325928, "operator": "ge"}, {"budget": 0.36, "threshold": 7.071319103240967, "operator": "ge"}, {"budget": 0.37, "threshold": 6.819001197814941, "operator": "ge"}, {"budget": 0.38, "threshold": 6.575403213500977, "operator": "ge"}, {"budget": 0.39, "threshold": 6.371612548828125, "operator": "ge"}, {"budget": 0.4, "threshold": 6.173492431640625, "operator": "ge"}, {"budget": 0.41, "threshold": 5.960362911224365, "operator": "ge"}, {"budget": 0.42, "threshold": 5.775658130645752, "operator": "ge"}, {"budget": 0.43, "threshold": 5.594803810119629, "operator": "ge"}, {"budget": 0.44, "threshold": 5.398891448974609, "operator": "ge"}, {"budget": 0.45, "threshold": 5.246407985687256, "operator": "ge"}, {"budget": 0.46, "threshold": 5.074526786804199, "operator": "ge"}, {"budget": 0.47, "threshold": 4.883544921875, "operator": "ge"}, {"budget": 0.48, "threshold": 4.7076520919799805, "operator": "ge"}, {"budget": 0.49, "threshold": 4.537606239318848, "operator": "ge"}, {"budget": 0.5, "threshold": 4.392768383026123, "operator": "ge"}, {"budget": 0.51, "threshold": 4.245129108428955, "operator": "ge"}, {"budget": 0.52, "threshold": 4.0826592445373535, "operator": "ge"}, {"budget": 0.53, "threshold": 3.9393515586853027, "operator": "ge"}, {"budget": 0.54, "threshold": 3.8138866424560547, "operator": "ge"}, {"budget": 0.55, "threshold": 3.6649346351623535, "operator": "ge"}, {"budget": 0.56, "threshold": 3.5383286476135254, "operator": "ge"}, {"budget": 0.57, "threshold": 3.403449535369873, "operator": "ge"}, {"budget": 0.58, "threshold": 3.2657806873321533, "operator": "ge"}, {"budget": 0.59, "threshold": 3.125092029571533, "operator": "ge"}, {"budget": 0.6, "threshold": 2.9820804595947266, "operator": "ge"}, {"budget": 0.61, "threshold": 2.8371691703796387, "operator": "ge"}, {"budget": 0.62, "threshold": 2.6953563690185547, "operator": "ge"}, {"budget": 0.63, "threshold": 2.534966230392456, "operator": "ge"}, {"budget": 0.64, "threshold": 2.3792264461517334, "operator": "ge"}, {"budget": 0.65, "threshold": 2.2352294921875, "operator": "ge"}, {"budget": 0.66, "threshold": 2.1027331352233887, "operator": "ge"}, {"budget": 0.67, "threshold": 1.9387731552124023, "operator": "ge"}, {"budget": 0.68, "threshold": 1.778928518295288, "operator": "ge"}, {"budget": 0.69, "threshold": 1.6336385011672974, "operator": "ge"}, {"budget": 0.7, "threshold": 1.4564870595932007, "operator": "ge"}, {"budget": 0.71, "threshold": 1.3030998706817627, "operator": "ge"}, {"budget": 0.72, "threshold": 1.1261252164840698, "operator": "ge"}, {"budget": 0.73, "threshold": 0.9712412357330322, "operator": "ge"}, {"budget": 0.74, "threshold": 0.7987312078475952, "operator": "ge"}, {"budget": 0.75, "threshold": 0.5905213356018066, "operator": "ge"}, {"budget": 0.76, "threshold": 0.38805699348449707, "operator": "ge"}, {"budget": 0.77, "threshold": 0.21613693237304688, "operator": "ge"}, {"budget": 0.78, "threshold": 0.009458065032958984, "operator": "ge"}, {"budget": 0.79, "threshold": -0.2575984001159668, "operator": "ge"}, {"budget": 0.8, "threshold": -0.45520031452178955, "operator": "ge"}, {"budget": 0.81, "threshold": -0.6469626426696777, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8672342300415039, "operator": "ge"}, {"budget": 0.83, "threshold": -1.0831878185272217, "operator": "ge"}, {"budget": 0.84, "threshold": -1.2977733612060547, "operator": "ge"}, {"budget": 0.85, "threshold": -1.5240554809570312, "operator": "ge"}, {"budget": 0.86, "threshold": -1.790666103363037, "operator": "ge"}, {"budget": 0.87, "threshold": -2.045408010482788, "operator": "ge"}, {"budget": 0.88, "threshold": -2.3297104835510254, "operator": "ge"}, {"budget": 0.89, "threshold": -2.570915460586548, "operator": "ge"}, {"budget": 0.9, "threshold": -2.8366596698760986, "operator": "ge"}, {"budget": 0.91, "threshold": -3.0962915420532227, "operator": "ge"}, {"budget": 0.92, "threshold": -3.375018835067749, "operator": "ge"}, {"budget": 0.93, "threshold": -3.7021334171295166, "operator": "ge"}, {"budget": 0.94, "threshold": -4.092837810516357, "operator": "ge"}, {"budget": 0.95, "threshold": -4.54275369644165, "operator": "ge"}, {"budget": 0.96, "threshold": -5.037501811981201, "operator": "ge"}, {"budget": 0.97, "threshold": -5.791299819946289, "operator": "ge"}, {"budget": 0.98, "threshold": -6.606553554534912, "operator": "ge"}, {"budget": 0.99, "threshold": -7.528782367706299, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.97519151029587, 57.92395811371143, 58.858668915594905, 59.80743551901047, 60.699978916297695, 61.66982922201139, 62.548316817766526, 63.40572071122356, 64.18581769625412, 65.01510998664698, 65.83034647550777, 66.6385550636025, 67.4748752547614, 68.28308384285613, 69.07723662941878, 69.87138941598144, 70.59526319488369, 71.35427647761612, 72.03598285192211, 72.74580082929228, 73.4837304097266, 74.18652048633074, 74.80497575374235, 75.45857052498418, 76.14027689929017, 76.7587321667018, 77.32799212875115, 77.77074987701174, 78.23459132757044, 78.7195164804273, 79.16930212945392, 79.52069716775598, 79.90020380912222, 80.23051514512615, 80.41324056504322, 80.60299388572632, 80.7013844964509, 80.86302621406986, 80.99655632862464, 81.13008644317942, 81.14414224471149, 81.20739335160587, 81.24956075620211, 81.22144915313795, 81.35497926769273, 81.39011877152294, 81.3479513669267, 81.24956075620211, 81.22847705390399, 81.1792817485417, 81.20739335160587, 81.08089113781713, 81.00358422939068, 80.8700541148359, 80.85599831330381, 80.7857193056434, 80.76463560334528, 80.70841239721695, 80.75057980181319, 80.72246819874903, 80.69435659568487, 80.715440297983, 80.68732869491882, 80.65218919108862, 80.64516129032258, 80.68030079415279, 80.7857193056434, 80.84194251177172, 80.86302621406986, 80.89816571790006, 80.97547262632652, 81.04575163398692, 81.09494693934921, 81.09494693934921, 81.15117014547755, 81.2144212523719, 81.23550495467003, 81.19333755007378, 81.19333755007378, 81.14414224471149, 81.09494693934921, 81.07386323705109, 81.05277953475296, 81.0246679316888, 80.96844472556047, 80.93330522173027, 80.89113781713402, 80.86302621406986, 80.84897041253777, 80.80680300794153, 80.77166350411132, 80.72949609951507, 80.7013844964509, 80.65921709185466, 80.63813338955653, 80.62407758802446, 80.61704968725842, 80.61002178649237, 80.61002178649237, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 131, 266, 399, 534, 661, 799, 924, 1046, 1158, 1276, 1392, 1508, 1627, 1742, 1855, 1968, 2071, 2183, 2281, 2384, 2490, 2593, 2684, 2782, 2881, 2976, 3065, 3137, 3210, 3288, 3369, 3432, 3500, 3563, 3613, 3667, 3704, 3752, 3796, 3835, 3868, 3908, 3942, 3974, 4014, 4043, 4065, 4081, 4104, 4124, 4152, 4166, 4181, 4193, 4211, 4227, 4245, 4262, 4284, 4302, 4316, 4337, 4350, 4365, 4384, 4407, 4434, 4454, 4470, 4486, 4504, 4524, 4543, 4553, 4568, 4586, 4598, 4604, 4610, 4613, 4615, 4619, 4620, 4622, 4623, 4623, 4623, 4624, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 6, 7, 9, 10, 13, 16, 21, 23, 30, 38, 47, 54, 63, 80, 93, 107, 123, 147, 174, 197, 222, 247, 267, 298, 329, 357, 393, 414, 438, 466, 496, 522, 549, 573, 605, 631, 662, 682, 708, 729, 754, 770, 792, 810, 828, 845, 865, 885, 903, 915, 927, 940, 951, 958, 968, 980, 990, 997, 1006, 1015, 1027, 1033, 1043, 1052, 1059, 1063, 1069, 1078, 1083, 1089, 1094, 1097, 1104, 1109, 1115, 1119, 1125, 1128, 1130, 1131, 1132, 1132, 1133], "exact_score": [56.05453650994448, 56.97519151029587, 57.92395811371143, 58.858668915594905, 59.80743551901047, 60.699978916297695, 61.66982922201139, 62.548316817766526, 63.40572071122356, 64.18581769625412, 65.01510998664698, 65.83034647550777, 66.6385550636025, 67.4748752547614, 68.28308384285613, 69.07723662941878, 69.87138941598144, 70.59526319488369, 71.35427647761612, 72.03598285192211, 72.74580082929228, 73.4837304097266, 74.18652048633074, 74.80497575374235, 75.45857052498418, 76.14027689929017, 76.7587321667018, 77.32799212875115, 77.77074987701174, 78.23459132757044, 78.7195164804273, 79.16930212945392, 79.52069716775598, 79.90020380912222, 80.23051514512615, 80.41324056504322, 80.60299388572632, 80.7013844964509, 80.86302621406986, 80.99655632862464, 81.13008644317942, 81.14414224471149, 81.20739335160587, 81.24956075620211, 81.22144915313795, 81.35497926769273, 81.39011877152294, 81.3479513669267, 81.24956075620211, 81.22847705390399, 81.1792817485417, 81.20739335160587, 81.08089113781713, 81.00358422939068, 80.8700541148359, 80.85599831330381, 80.7857193056434, 80.76463560334528, 80.70841239721695, 80.75057980181319, 80.72246819874903, 80.69435659568487, 80.715440297983, 80.68732869491882, 80.65218919108862, 80.64516129032258, 80.68030079415279, 80.7857193056434, 80.84194251177172, 80.86302621406986, 80.89816571790006, 80.97547262632652, 81.04575163398692, 81.09494693934921, 81.09494693934921, 81.15117014547755, 81.2144212523719, 81.23550495467003, 81.19333755007378, 81.19333755007378, 81.14414224471149, 81.09494693934921, 81.07386323705109, 81.05277953475296, 81.0246679316888, 80.96844472556047, 80.93330522173027, 80.89113781713402, 80.86302621406986, 80.84897041253777, 80.80680300794153, 80.77166350411132, 80.72949609951507, 80.7013844964509, 80.65921709185466, 80.63813338955653, 80.62407758802446, 80.61704968725842, 80.61002178649237, 80.61002178649237, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 77.15471330592813, "threshold_area100": 77.15387588727246, "threshold_area25": 66.7271066132546}, "test": {"score": [26.688029946402747, 27.39415251113065, 28.103110909451836, 28.85744264526558, 29.608938547486037, 30.235657771601964, 30.92192950117687, 31.46924538468083, 32.195218784561725, 32.952386353968755, 33.69537475540936, 34.35045231545813, 35.10194821767859, 35.99523579956328, 36.79494087286958, 37.62300428210873, 38.45106769134787, 39.08345858265037, 39.86047698721039, 40.54958455037859, 41.26988628307291, 42.18586053370388, 42.999744774976605, 43.75124067719707, 44.60766242236905, 45.50945750503361, 46.428267589257864, 47.52573518985906, 48.50976944672887, 49.69231205512861, 50.80679465728951, 52.077248107081076, 53.56889657714885, 54.986813373791236, 56.59756685477697, 57.902050307687944, 59.3795196097893, 60.689674729886846, 61.94878484530528, 63.00371494200721, 63.905510024671756, 64.90372344950799, 65.67223435328815, 66.52582026486687, 67.32268950457987, 67.86433372089726, 68.598814621558, 69.24822051442021, 69.8267305674503, 70.30598644471542, 70.78240648838727, 71.13972152114114, 71.48285738592858, 71.85151575305562, 72.03584493661911, 72.34778663188044, 72.6171908232425, 72.89793834897769, 73.0992825341009, 73.28077588407112, 73.49346340356749, 73.62674758245187, 73.79973343164224, 73.9557042792729, 74.13436179564982, 74.29316847687377, 74.4009301534186, 74.50302016277686, 74.59943850494854, 74.68451351274707, 74.8121260244449, 74.89152936505685, 74.97944020644869, 75.01630604316138, 75.06735104784052, 75.12123188611292, 75.1439185548592, 75.15526188923234, 75.15809772282563, 75.16376939001219, 75.14108272126592, 75.1240677197062, 75.10988855173979, 75.09287355018009, 75.0701868814338, 75.04750021268752, 75.00496270878824, 74.96242520488899, 74.91705186739642, 74.883021864277, 74.83764852678445, 74.80929019085161, 74.77809602132547, 74.7355585174262, 74.67600601196722, 74.65331934322094, 74.61645350650824, 74.56540850182911, 74.52570683152314, 74.5115276635567, 74.47182599325072], "calls": [0.0, 0.7571675694070271, 1.5143351388140542, 2.3310552136800613, 3.1591186229192068, 3.8227036837478376, 4.591214587528004, 5.200918810084224, 6.0204747185435155, 6.8598814621558, 7.744661543260642, 8.55003828375351, 9.525565039843462, 10.597510138105095, 11.541842724668916, 12.602444488557413, 13.623344582139921, 14.391855485920086, 15.398576411536173, 16.314550662167143, 17.307092419816804, 18.492470861809828, 19.606953463970733, 20.653376059892807, 21.765022828460427, 23.032640444658707, 24.359810566315968, 25.933698210589, 27.37430167597765, 29.152369338967187, 30.842526160564898, 32.81059467430451, 35.031052377846464, 37.19195757592944, 39.85196948643054, 41.99585968295381, 44.45452740833168, 46.83662762669086, 49.20738451067692, 51.16127385645011, 53.06128236395088, 55.139948387828596, 56.733686867254626, 58.37847035135978, 60.11116467685677, 61.36460312508862, 62.89311743186909, 64.40745257068315, 65.8310410345121, 67.08447948274396, 68.14224541303916, 69.16881717380824, 70.33150894705499, 71.29852820236509, 72.10957661004453, 72.94614752006352, 73.7316734254034, 74.48884099481042, 75.30839690326971, 76.06556447267674, 76.86526954598304, 77.57139211071095, 78.3058730113717, 79.09423475030485, 79.82020815018575, 80.5717040524062, 81.18991577574228, 81.92439667640303, 82.59365340441823, 83.23171596290729, 83.95201769560163, 84.56455775175112, 85.31888948756487, 85.8888920398151, 86.53262626549075, 87.28695800130448, 88.05830473867793, 88.6197997901483, 89.3457731900292, 90.03771658679068, 90.5765249695148, 91.14652752176502, 91.70518673964212, 92.24966678955279, 92.8054901738366, 93.33295522218756, 93.91997277599751, 94.50699032980745, 95.00326120863227, 95.46833791793098, 95.8710262881774, 96.29640132717012, 96.6707313614837, 97.09327056688313, 97.55834727618183, 97.97805064798798, 98.44029152369339, 98.9592490712645, 99.38178827666393, 99.63134163287299, 100.0], "rescued": [0, 249, 499, 766, 1031, 1252, 1494, 1687, 1943, 2210, 2472, 2704, 2969, 3284, 3567, 3862, 4156, 4380, 4656, 4904, 5159, 5484, 5778, 6050, 6354, 6680, 7017, 7417, 7776, 8215, 8631, 9102, 9657, 10184, 10803, 11307, 11871, 12397, 12914, 13334, 13699, 14110, 14420, 14763, 15098, 15329, 15625, 15897, 16149, 16364, 16557, 16720, 16887, 17050, 17152, 17284, 17405, 17526, 17631, 17721, 17821, 17891, 17977, 18057, 18143, 18221, 18279, 18335, 18382, 18431, 18489, 18536, 18588, 18615, 18648, 18691, 18711, 18729, 18749, 18763, 18771, 18780, 18790, 18800, 18806, 18810, 18812, 18816, 18816, 18816, 18817, 18817, 18818, 18818, 18818, 18819, 18819, 18819, 18819, 18820, 18820], "harmed": [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 3, 6, 8, 9, 11, 16, 17, 19, 26, 33, 35, 43, 56, 69, 81, 103, 126, 149, 178, 205, 256, 300, 343, 407, 480, 528, 575, 634, 673, 715, 769, 809, 846, 889, 937, 983, 1008, 1045, 1091, 1124, 1161, 1183, 1209, 1231, 1265, 1291, 1316, 1339, 1364, 1389, 1412, 1434, 1454, 1474, 1487, 1506, 1519, 1538, 1559, 1573, 1588, 1612, 1624, 1638, 1657, 1669, 1685, 1700, 1715, 1731, 1745, 1757, 1774, 1793, 1809, 1821, 1838, 1848, 1860, 1875, 1896, 1905, 1918, 1936, 1950, 1956, 1970], "exact_score": [26.688029946402747, 27.621019198593427, 28.551172617190822, 29.45013186626209, 30.380285284859486, 31.29625953549046, 32.17253211581544, 33.07999886566657, 33.927913110058704, 34.68508067946573, 35.5074724215183, 36.35255083231716, 37.180614241556306, 37.93778181096334, 38.76017355301591, 39.55420695913564, 40.30003119416953, 41.045855429203414, 41.79735133142387, 42.58287723676375, 43.28616396789837, 44.00930153418597, 44.76079743640643, 45.48109916910076, 46.1985650682018, 46.918866800896126, 47.56543686016504, 48.248872756146675, 48.920965317755154, 49.584550378583785, 50.2311204378527, 50.90604883305447, 51.54694722513683, 52.199188951592326, 52.874117346794094, 53.55188157558914, 54.204123302044636, 54.864872529279985, 55.51995008932876, 56.11547514391856, 56.69398519694864, 57.28100275075858, 57.90488614128123, 58.511754530244175, 59.12713041998696, 59.64608796755806, 60.19340385106202, 60.76624223690554, 61.32206562118936, 61.79848566486119, 62.38833905226442, 62.91580410061538, 63.380880809914075, 63.871480021552344, 64.30536256132491, 64.81014094092959, 65.3092476533477, 65.8111901993591, 66.36701358364292, 66.78388112185577, 67.27731616708732, 67.76791537872558, 68.19612625131157, 68.64418795905056, 69.06956299804328, 69.54031137452854, 69.91464140884214, 70.28329977596914, 70.72852565011485, 71.07733318208888, 71.38360321016364, 71.73808240932422, 72.01882993505941, 72.36196579984686, 72.7277883333806, 73.02838669426879, 73.27510421688456, 73.51898590590704, 73.72033009103026, 73.94152511130648, 74.17122763236253, 74.36122848311261, 74.52287099792984, 74.65899101040752, 74.82063352522475, 74.93973853614271, 75.03615687831437, 75.10988855173979, 75.13257522048606, 75.16660522360547, 75.16376939001219, 75.1269035532995, 75.09287355018009, 75.05600771346738, 74.99929104160168, 74.9198877009897, 74.82914102600459, 74.74690185179934, 74.65048350962766, 74.56540850182911, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.762917161501726, "threshold_area100": 61.020361285199776, "threshold_area25": 35.784533363582234}, "selected_budget": 46, "selected": {"val": {"score": 81.39011877152294, "calls": 45.997610513739545, "rescued": 4043, "harmed": 438}, "test": {"score": 68.598814621558, "calls": 62.89311743186909, "rescued": 15625, "harmed": 846}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 5.756277084350586, "operator": "ge"}, {"budget": 0.02, "threshold": 5.113413333892822, "operator": "ge"}, {"budget": 0.03, "threshold": 4.7365264892578125, "operator": "ge"}, {"budget": 0.04, "threshold": 4.391803741455078, "operator": "ge"}, {"budget": 0.05, "threshold": 4.116751670837402, "operator": "ge"}, {"budget": 0.06, "threshold": 3.899620771408081, "operator": "ge"}, {"budget": 0.07, "threshold": 3.718111515045166, "operator": "ge"}, {"budget": 0.08, "threshold": 3.5510716438293457, "operator": "ge"}, {"budget": 0.09, "threshold": 3.3958804607391357, "operator": "ge"}, {"budget": 0.1, "threshold": 3.267597198486328, "operator": "ge"}, {"budget": 0.11, "threshold": 3.148311138153076, "operator": "ge"}, {"budget": 0.12, "threshold": 3.0296597480773926, "operator": "ge"}, {"budget": 0.13, "threshold": 2.9331765174865723, "operator": "ge"}, {"budget": 0.14, "threshold": 2.8465166091918945, "operator": "ge"}, {"budget": 0.15, "threshold": 2.7647271156311035, "operator": "ge"}, {"budget": 0.16, "threshold": 2.691399097442627, "operator": "ge"}, {"budget": 0.17, "threshold": 2.6020348072052, "operator": "ge"}, {"budget": 0.18, "threshold": 2.5228137969970703, "operator": "ge"}, {"budget": 0.19, "threshold": 2.4631316661834717, "operator": "ge"}, {"budget": 0.2, "threshold": 2.39955472946167, "operator": "gt"}, {"budget": 0.21, "threshold": 2.336721420288086, "operator": "ge"}, {"budget": 0.22, "threshold": 2.276280164718628, "operator": "ge"}, {"budget": 0.23, "threshold": 2.2198097705841064, "operator": "ge"}, {"budget": 0.24, "threshold": 2.163891315460205, "operator": "ge"}, {"budget": 0.25, "threshold": 2.1080729961395264, "operator": "ge"}, {"budget": 0.26, "threshold": 2.0633156299591064, "operator": "ge"}, {"budget": 0.27, "threshold": 2.0113306045532227, "operator": "ge"}, {"budget": 0.28, "threshold": 1.9543977975845337, "operator": "ge"}, {"budget": 0.29, "threshold": 1.90749990940094, "operator": "ge"}, {"budget": 0.3, "threshold": 1.8619070053100586, "operator": "gt"}, {"budget": 0.31, "threshold": 1.8178479671478271, "operator": "ge"}, {"budget": 0.32, "threshold": 1.775130271911621, "operator": "ge"}, {"budget": 0.33, "threshold": 1.7350088357925415, "operator": "ge"}, {"budget": 0.34, "threshold": 1.6977273225784302, "operator": "ge"}, {"budget": 0.35, "threshold": 1.654141902923584, "operator": "ge"}, {"budget": 0.36, "threshold": 1.6119238138198853, "operator": "ge"}, {"budget": 0.37, "threshold": 1.574217438697815, "operator": "ge"}, {"budget": 0.38, "threshold": 1.5392265319824219, "operator": "ge"}, {"budget": 0.39, "threshold": 1.501664161682129, "operator": "ge"}, {"budget": 0.4, "threshold": 1.4631248712539673, "operator": "ge"}, {"budget": 0.41, "threshold": 1.4267749786376953, "operator": "ge"}, {"budget": 0.42, "threshold": 1.3852872848510742, "operator": "ge"}, {"budget": 0.43, "threshold": 1.3527307510375977, "operator": "ge"}, {"budget": 0.44, "threshold": 1.3132174015045166, "operator": "ge"}, {"budget": 0.45, "threshold": 1.2775875329971313, "operator": "ge"}, {"budget": 0.46, "threshold": 1.2398653030395508, "operator": "gt"}, {"budget": 0.47, "threshold": 1.2045999765396118, "operator": "ge"}, {"budget": 0.48, "threshold": 1.1717249155044556, "operator": "ge"}, {"budget": 0.49, "threshold": 1.1410512924194336, "operator": "ge"}, {"budget": 0.5, "threshold": 1.1100819110870361, "operator": "ge"}, {"budget": 0.51, "threshold": 1.0749379396438599, "operator": "ge"}, {"budget": 0.52, "threshold": 1.0440301895141602, "operator": "ge"}, {"budget": 0.53, "threshold": 1.0075252056121826, "operator": "ge"}, {"budget": 0.54, "threshold": 0.9757992029190063, "operator": "ge"}, {"budget": 0.55, "threshold": 0.9423596858978271, "operator": "ge"}, {"budget": 0.56, "threshold": 0.9025282859802246, "operator": "ge"}, {"budget": 0.57, "threshold": 0.873839259147644, "operator": "ge"}, {"budget": 0.58, "threshold": 0.8438341617584229, "operator": "ge"}, {"budget": 0.59, "threshold": 0.8078212738037109, "operator": "ge"}, {"budget": 0.6, "threshold": 0.7743121385574341, "operator": "ge"}, {"budget": 0.61, "threshold": 0.7416501045227051, "operator": "gt"}, {"budget": 0.62, "threshold": 0.7117264270782471, "operator": "ge"}, {"budget": 0.63, "threshold": 0.6786792278289795, "operator": "ge"}, {"budget": 0.64, "threshold": 0.6449959874153137, "operator": "ge"}, {"budget": 0.65, "threshold": 0.6127792000770569, "operator": "ge"}, {"budget": 0.66, "threshold": 0.5760512351989746, "operator": "ge"}, {"budget": 0.67, "threshold": 0.5417666435241699, "operator": "ge"}, {"budget": 0.68, "threshold": 0.5092778205871582, "operator": "ge"}, {"budget": 0.69, "threshold": 0.473333477973938, "operator": "ge"}, {"budget": 0.7, "threshold": 0.43694305419921875, "operator": "ge"}, {"budget": 0.71, "threshold": 0.40074288845062256, "operator": "ge"}, {"budget": 0.72, "threshold": 0.3553115427494049, "operator": "ge"}, {"budget": 0.73, "threshold": 0.3068726062774658, "operator": "ge"}, {"budget": 0.74, "threshold": 0.2584495544433594, "operator": "ge"}, {"budget": 0.75, "threshold": 0.19825172424316406, "operator": "ge"}, {"budget": 0.76, "threshold": 0.14916884899139404, "operator": "ge"}, {"budget": 0.77, "threshold": 0.08741044998168945, "operator": "ge"}, {"budget": 0.78, "threshold": 0.0059449076652526855, "operator": "ge"}, {"budget": 0.79, "threshold": -0.1609477400779724, "operator": "ge"}, {"budget": 0.8, "threshold": -0.2828364372253418, "operator": "ge"}, {"budget": 0.81, "threshold": -0.42921292781829834, "operator": "ge"}, {"budget": 0.82, "threshold": -0.5666028261184692, "operator": "ge"}, {"budget": 0.83, "threshold": -0.737231969833374, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9069046378135681, "operator": "ge"}, {"budget": 0.85, "threshold": -1.1165146827697754, "operator": "ge"}, {"budget": 0.86, "threshold": -1.3032355308532715, "operator": "ge"}, {"budget": 0.87, "threshold": -1.510521411895752, "operator": "ge"}, {"budget": 0.88, "threshold": -1.7634050846099854, "operator": "ge"}, {"budget": 0.89, "threshold": -2.0227627754211426, "operator": "ge"}, {"budget": 0.9, "threshold": -2.28717303276062, "operator": "ge"}, {"budget": 0.91, "threshold": -2.5223917961120605, "operator": "ge"}, {"budget": 0.92, "threshold": -2.780292510986328, "operator": "ge"}, {"budget": 0.93, "threshold": -3.063382148742676, "operator": "ge"}, {"budget": 0.94, "threshold": -3.4396796226501465, "operator": "ge"}, {"budget": 0.95, "threshold": -3.9047300815582275, "operator": "ge"}, {"budget": 0.96, "threshold": -4.405457019805908, "operator": "ge"}, {"budget": 0.97, "threshold": -5.085731029510498, "operator": "ge"}, {"budget": 0.98, "threshold": -5.944705486297607, "operator": "ge"}, {"budget": 0.99, "threshold": -6.784185886383057, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [56.05453650994448, 56.99627521259399, 57.874762808349146, 58.74622250333825, 59.54740319066695, 60.306416473399395, 61.086513458429955, 61.76119193196992, 62.463982008574035, 63.068381474453574, 63.63061353573687, 64.23501300161641, 64.73399395600535, 65.359477124183, 65.87954178087006, 66.4347459413873, 66.91967109424415, 67.32026143790848, 67.74896338463701, 68.24091643825989, 68.63447888115819, 69.07020872865274, 69.52702227844543, 69.9838358282381, 70.48984468339306, 70.86232342399325, 71.22074636306135, 71.60025300442757, 71.86731323353715, 72.12031766111463, 72.45062899711856, 72.67552182163188, 72.99880525686977, 73.26586548597933, 73.5680652189191, 73.77890224190034, 74.10921357790427, 74.36221800548176, 74.72064094454986, 74.9666174713613, 75.1704265935765, 75.26178930353502, 75.48668212804834, 75.73265865485979, 76.01377468550143, 76.17541640312038, 76.4003092276337, 76.59709044908286, 76.7235926628716, 76.85009487666034, 77.0468760981095, 77.34204793028321, 77.51071754866821, 77.72858247241548, 77.89725209080048, 78.03078220535525, 78.22053552603838, 78.42434464825357, 78.57193056434042, 78.71248857966124, 78.84601869421604, 79.00063251106893, 79.1974137325181, 79.3168880455408, 79.40122285473329, 79.52069716775598, 79.77370159533346, 79.88614800759012, 80.1040129313374, 80.25159884742426, 80.41324056504322, 80.56785438189613, 80.74355190104716, 80.91222151943214, 81.01061213015672, 81.05277953475296, 81.11603064164734, 81.1792817485417, 81.19333755007378, 81.18630964930776, 81.17225384777566, 81.10197484011525, 81.08089113781713, 81.05277953475296, 81.03872373322089, 81.01061213015672, 80.96141682479443, 80.91222151943214, 80.8700541148359, 80.84897041253777, 80.8208588094736, 80.76463560334528, 80.76463560334528, 80.70841239721695, 80.65921709185466, 80.63813338955653, 80.62407758802446, 80.62407758802446, 80.61002178649237, 80.61002178649237, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.987349778621123, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.988052568697732, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.990582612973505, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.9881228477054, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 136, 264, 395, 513, 627, 744, 853, 961, 1057, 1152, 1254, 1342, 1441, 1528, 1618, 1703, 1774, 1848, 1929, 2003, 2076, 2154, 2231, 2309, 2383, 2454, 2524, 2580, 2632, 2689, 2741, 2808, 2861, 2915, 2966, 3027, 3076, 3142, 3191, 3239, 3275, 3328, 3381, 3435, 3479, 3525, 3566, 3605, 3648, 3695, 3742, 3783, 3827, 3865, 3899, 3939, 3981, 4014, 4049, 4085, 4120, 4158, 4188, 4218, 4248, 4292, 4322, 4360, 4391, 4420, 4448, 4480, 4512, 4535, 4553, 4572, 4592, 4604, 4611, 4614, 4617, 4620, 4621, 4622, 4623, 4623, 4623, 4623, 4624, 4625, 4625, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626, 4626], "harmed": [0, 2, 5, 12, 16, 22, 28, 41, 49, 59, 74, 90, 107, 117, 130, 141, 157, 171, 184, 195, 213, 224, 237, 249, 255, 276, 296, 312, 330, 346, 356, 376, 397, 412, 423, 444, 458, 471, 486, 500, 519, 542, 563, 581, 595, 616, 630, 643, 664, 689, 708, 713, 730, 743, 757, 772, 785, 798, 810, 825, 842, 855, 865, 878, 896, 909, 917, 931, 938, 948, 954, 960, 967, 975, 984, 996, 1006, 1017, 1027, 1035, 1040, 1053, 1059, 1064, 1067, 1072, 1079, 1086, 1092, 1096, 1101, 1109, 1110, 1118, 1125, 1128, 1130, 1130, 1132, 1132, 1133], "exact_score": [56.05453650994448, 56.99627521259399, 57.874762808349146, 58.74622250333825, 59.54740319066695, 60.306416473399395, 61.086513458429955, 61.76119193196992, 62.463982008574035, 63.068381474453574, 63.63061353573687, 64.23501300161641, 64.73399395600535, 65.359477124183, 65.87954178087006, 66.4347459413873, 66.91967109424415, 67.32026143790848, 67.74896338463701, 68.24091643825989, 68.64150678192424, 69.07020872865274, 69.52702227844543, 69.9838358282381, 70.48984468339306, 70.86232342399325, 71.22074636306135, 71.60025300442757, 71.86731323353715, 72.12031766111463, 72.45062899711856, 72.67552182163188, 72.99880525686977, 73.26586548597933, 73.5680652189191, 73.77890224190034, 74.10921357790427, 74.36221800548176, 74.72064094454986, 74.9666174713613, 75.1704265935765, 75.26178930353502, 75.48668212804834, 75.73265865485979, 76.01377468550143, 76.17541640312038, 76.4003092276337, 76.59709044908286, 76.7235926628716, 76.85009487666034, 77.0468760981095, 77.34204793028321, 77.51071754866821, 77.72858247241548, 77.89725209080048, 78.03078220535525, 78.22053552603838, 78.42434464825357, 78.57193056434042, 78.71248857966124, 78.84601869421604, 79.00063251106893, 79.1974137325181, 79.3168880455408, 79.40122285473329, 79.52069716775598, 79.77370159533346, 79.88614800759012, 80.1040129313374, 80.25159884742426, 80.41324056504322, 80.56785438189613, 80.74355190104716, 80.91222151943214, 81.01061213015672, 81.05277953475296, 81.11603064164734, 81.1792817485417, 81.19333755007378, 81.18630964930776, 81.17225384777566, 81.10197484011525, 81.08089113781713, 81.05277953475296, 81.03872373322089, 81.01061213015672, 80.96141682479443, 80.91222151943214, 80.8700541148359, 80.84897041253777, 80.8208588094736, 80.76463560334528, 80.76463560334528, 80.70841239721695, 80.65921709185466, 80.63813338955653, 80.62407758802446, 80.62407758802446, 80.61002178649237, 80.61002178649237, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 74.74490644013878, "threshold_area100": 74.74400871459694, "threshold_area25": 64.49293695973013}, "test": {"score": [26.688029946402747, 28.08042424070556, 29.37072852565012, 30.663868644187964, 32.05626293849077, 33.564926410118254, 34.94030570286136, 36.12568414485438, 37.282704250914556, 38.453903524941154, 39.68181947083346, 40.88421291438619, 42.097949692312056, 43.11884978589456, 44.08870487479795, 44.950798287156516, 45.74766752686953, 46.760060119672175, 47.786631880441256, 48.540963616255, 49.312310353628455, 50.1035079261549, 50.823809658849214, 51.52709638998384, 52.22471145393188, 52.933669852253075, 53.580239911521986, 54.25233247313048, 55.00666420894422, 55.57950259478774, 56.158012647817834, 56.7024926977285, 57.23846524685931, 57.74040779287072, 58.26787284122168, 58.857726228624905, 59.40220627853559, 59.923999659699966, 60.3465388650994, 60.73221223378613, 61.32490145478264, 61.82967983438732, 62.29475654368601, 62.756997419391425, 63.25610413180955, 63.66446416924255, 64.11252587698154, 64.6229759237728, 65.04267929557895, 65.4396959986388, 65.77432436264641, 66.20820690241898, 66.52865609846015, 66.99656864135213, 67.45313784987097, 67.83313955137113, 68.3379179309758, 68.618665456711, 69.00150299180444, 69.45523636673, 69.81822306667044, 70.16703059864447, 70.50449479624537, 70.88166066415222, 71.26449819924568, 71.64449990074583, 72.04151660380569, 72.37330913422, 72.69659416385447, 73.04823752942178, 73.3771942262428, 73.69764342228397, 74.03227178629159, 74.31301931202677, 74.61361767291496, 74.85466352834416, 74.99078354082182, 75.09570938377337, 75.14959022204577, 75.15242605563905, 75.1439185548592, 75.11839605251964, 75.12123188611292, 75.09854521736665, 75.0900377165868, 75.0503360462808, 75.03048521112781, 75.00496270878824, 74.95958937129569, 74.89152936505685, 74.84332019397102, 74.81779769163145, 74.7724243541389, 74.74406601820604, 74.70152851430679, 74.65048350962766, 74.61645350650824, 74.57958766979553, 74.53705016589626, 74.51719933074328, 74.47182599325072], "calls": [0.0, 1.4661259677282137, 2.869863596404163, 4.293452060233106, 5.864503870912855, 7.563168193290418, 9.14839917193659, 10.54646513342597, 12.018262768340755, 13.583642911833934, 15.134843887360688, 16.578283186342627, 18.10396165952982, 19.48501261945949, 20.82352607548989, 22.02591951904262, 23.18861129228937, 24.70011059751014, 26.123699061339078, 27.206987493973855, 28.409380937526585, 29.64864021779202, 30.870884496497748, 32.10730794316989, 33.19059637580467, 34.3929898193574, 35.36284490826078, 36.457476675268694, 37.716586790687124, 38.72047188270992, 39.83779031846411, 40.86436207923319, 41.837053001729856, 42.82959475937952, 43.68034483736494, 44.75228993562658, 45.7703541956158, 46.78274678841846, 47.67036270311658, 48.577829452967705, 49.68947622153532, 50.72171964949097, 51.77381391259961, 52.689788163230574, 53.75889742789893, 54.68905084649632, 55.65323426821314, 56.71667186569492, 57.72622862490429, 58.67906871224796, 59.53549045741996, 60.56489805178232, 61.36176729149534, 62.476249893656245, 63.40073164506707, 64.41879590505629, 65.58999517908289, 66.40955108754218, 67.33119700535973, 68.47970961064004, 69.60553554717409, 70.62359980716332, 71.55091739216742, 72.55763831778351, 73.61256841448544, 74.61078183932167, 75.73377194226242, 76.73198536709866, 77.64228795054306, 78.66318804412558, 79.80602898221932, 80.84394407736154, 82.11723336074638, 83.41320931287753, 84.65814026032953, 86.1696395655503, 87.17068882397982, 88.27666392536086, 89.32308652128293, 90.24756827269375, 90.79204832260443, 91.3733942092278, 91.88951592320562, 92.44533930748943, 92.99549102458667, 93.58250857839661, 94.05042112128861, 94.5892295040127, 95.15356038907636, 95.66968210305419, 96.16028131469245, 96.54595468337918, 96.94864305362562, 97.28894308481978, 97.77387062927147, 98.23894733857017, 98.63596404163005, 99.07551824858918, 99.48387828602218, 99.75611831097751, 100.0], "rescued": [0, 495, 961, 1430, 1937, 2485, 2991, 3425, 3864, 4311, 4770, 5213, 5666, 6057, 6430, 6764, 7069, 7455, 7846, 8135, 8439, 8763, 9052, 9338, 9603, 9883, 10125, 10389, 10688, 10915, 11153, 11375, 11586, 11789, 11990, 12225, 12442, 12650, 12825, 12983, 13209, 13409, 13601, 13788, 13987, 14151, 14330, 14529, 14700, 14857, 14990, 15171, 15302, 15496, 15675, 15830, 16026, 16138, 16289, 16472, 16625, 16765, 16898, 17041, 17193, 17334, 17490, 17620, 17746, 17884, 18011, 18134, 18263, 18379, 18499, 18602, 18660, 18712, 18746, 18763, 18775, 18779, 18789, 18794, 18802, 18807, 18809, 18812, 18816, 18816, 18817, 18817, 18817, 18818, 18818, 18818, 18819, 18819, 18819, 18820, 18820], "harmed": [0, 4, 15, 28, 44, 60, 81, 97, 128, 162, 188, 207, 232, 263, 294, 324, 348, 377, 406, 429, 461, 506, 541, 579, 598, 628, 642, 669, 702, 727, 761, 791, 813, 839, 854, 881, 906, 930, 956, 978, 995, 1017, 1045, 1069, 1092, 1112, 1133, 1152, 1175, 1192, 1207, 1235, 1253, 1282, 1300, 1321, 1339, 1352, 1368, 1391, 1416, 1433, 1447, 1457, 1474, 1481, 1497, 1510, 1522, 1536, 1547, 1557, 1568, 1585, 1599, 1617, 1627, 1642, 1657, 1673, 1688, 1701, 1710, 1723, 1734, 1753, 1762, 1774, 1794, 1818, 1836, 1845, 1861, 1872, 1887, 1905, 1918, 1931, 1946, 1954, 1970], "exact_score": [26.688029946402747, 27.64370586733971, 28.57102345234382, 29.481326035788225, 30.397300286419192, 31.33028953860988, 32.17820378300201, 33.07999886566657, 33.95627144599155, 34.818364858350115, 35.66344326914897, 36.49150667838811, 37.26852508294813, 37.98315514845589, 38.76868105379577, 39.57122196069535, 40.407792870714346, 41.22451294558036, 42.02421801888665, 42.7586989195474, 43.49317982020815, 44.22198905368233, 44.928111618410234, 45.63707001673142, 46.31767007911976, 46.93304596886255, 47.69872103904943, 48.39066443581091, 49.05141366304625, 49.69231205512861, 50.30485211127811, 50.87201882993506, 51.478887218897995, 52.08575560786093, 52.68411649604401, 53.32217905453308, 53.98576411536171, 54.572781669171654, 55.17965005813459, 55.721294274451985, 56.2544309899895, 56.7592093695942, 57.33488358903101, 57.84816946941553, 58.44085869041204, 58.985338740322725, 59.52981879023339, 60.020418001871654, 60.50250971273006, 60.9590789212489, 61.50639480475286, 61.92893401015229, 62.39968238663756, 62.887445764682525, 63.34968664038794, 63.803420015313506, 64.27700422539205, 64.73924510109748, 65.15894847290362, 65.55880100955676, 65.95014604543006, 66.36984941723621, 66.78388112185577, 67.23477866318804, 67.6913478717069, 68.0770212403936, 68.45985877548706, 68.8823979808865, 69.25389218160677, 69.63105804951365, 69.92598474321528, 70.29747894393556, 70.67180897824916, 71.02061651022318, 71.38643904375692, 71.7976349147832, 72.12375577801096, 72.47823497717154, 72.79301250602614, 73.13898420440688, 73.43674673170179, 73.72600175821682, 73.99540594957887, 74.24212347219465, 74.43779599013129, 74.68451351274707, 74.83197685959789, 74.97093270566883, 75.07302271502708, 75.16093355641891, 75.15526188923234, 75.13541105407936, 75.10988855173979, 75.0900377165868, 75.03048521112781, 74.9766043728554, 74.86033519553072, 74.76675268695233, 74.67600601196722, 74.58809517057539, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 58.30997418984508, "threshold_area100": 61.7304823752942, "threshold_area25": 41.46414088421292}, "selected_budget": 78, "selected": {"val": {"score": 81.19333755007378, "calls": 77.99564270152506, "rescued": 4604, "harmed": 1027}, "test": {"score": 75.14959022204577, "calls": 89.32308652128293, "rescued": 18746, "harmed": 1657}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 81.2144212523719, "calls": 76.9976807927472}, "test": {"score": 75.15242605563905, "calls": 88.31352976207356}}, "old_normalized_interior": {"val": {"score": 81.2144212523719, "calls": 76.9976807927472}, "test": {"score": 75.15242605563905, "calls": 88.31352976207356}}, "old_normalized_endpoints": {"val": {"score": 81.2144212523719, "calls": 76.9976807927472}, "test": {"score": 75.15242605563905, "calls": 88.31352976207356}}}, "baseline_old_normalized": {"val": {"score": [56.05453650994448, 57.01033101412608, 57.93098601447747, 58.865696816360945, 59.73012861058401, 60.51725349638063, 61.29735048141119, 62.07744746644177, 62.80132124534401, 63.47599971888397, 64.15770609318996, 64.69885445217514, 65.31730971958676, 65.93576498699838, 66.57530395670813, 67.1164523156933, 67.60840536931619, 68.13549792676928, 68.69772998805257, 69.15454353784524, 69.58324548457375, 70.06114273666455, 70.5460678895214, 71.03802094314428, 71.44563918757467, 71.80406212664276, 72.10626185958253, 72.46468479865064, 72.79499613465458, 73.18855857755288, 73.45561880666244, 73.79998594419847, 74.12326937943637, 74.41844121161009, 74.62927823459133, 74.91039426523297, 75.11420338744816, 75.40234731885586, 75.71157495256166, 75.94349567784103, 76.25975121231288, 76.44950453299599, 76.56897884601868, 76.74467636516972, 76.96956918968304, 77.08904350270574, 77.29988052568697, 77.49666174713612, 77.65127556398903, 77.93239159463067, 78.11511701454775, 78.23459132757044, 78.43137254901961, 78.60707006817063, 78.6843769765971, 78.81087919038583, 78.91629770187645, 79.12713472485768, 79.28877644247663, 79.4223065570314, 79.56989247311827, 79.6612551830768, 79.79478529763159, 79.97048281678263, 80.07590132827325, 80.18131983976386, 80.28673835125447, 80.41324056504322, 80.48351957270363, 80.58893808419425, 80.68732869491882, 80.79977510717549, 80.9051936186661, 81.00358422939068, 81.10900274088128, 81.13008644317942, 81.1792817485417, 81.2144212523719, 81.19333755007378, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.19333755007378, 81.18630964930776, 81.1792817485417, 81.17225384777566, 81.17225384777566, 81.16522594700962, 81.16522594700962, 81.13711434394546, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.08791903858317, 81.08089113781713, 81.059807435519, 81.01061213015672, 80.94033312249631, 80.80680300794153, 80.60299388572632], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.9898095438892405, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.988685079766674, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.98643615152154, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [56.05453650994448, 57.01033101412608, 57.93098601447747, 58.865696816360945, 59.73012861058401, 60.52428139714667, 61.29735048141119, 62.07744746644177, 62.80132124534401, 63.48302761965, 64.15770609318996, 64.69885445217514, 65.31730971958676, 65.93576498699838, 66.57530395670813, 67.1164523156933, 67.60840536931619, 68.14252582753531, 68.69772998805257, 69.15454353784524, 69.58324548457375, 70.06114273666455, 70.5460678895214, 71.03802094314428, 71.44563918757467, 71.80406212664276, 72.10626185958253, 72.46468479865064, 72.79499613465458, 73.18855857755288, 73.45561880666244, 73.79998594419847, 74.12326937943637, 74.41844121161009, 74.62927823459133, 74.91039426523297, 75.11420338744816, 75.40234731885586, 75.71157495256166, 75.94349567784103, 76.25975121231288, 76.44950453299599, 76.56897884601868, 76.74467636516972, 76.96956918968304, 77.08904350270574, 77.29988052568697, 77.49666174713612, 77.65127556398903, 77.93239159463067, 78.11511701454775, 78.23459132757044, 78.43137254901961, 78.60707006817063, 78.6843769765971, 78.81087919038583, 78.91629770187645, 79.12713472485768, 79.28877644247663, 79.4223065570314, 79.56989247311827, 79.6612551830768, 79.79478529763159, 79.97048281678263, 80.07590132827325, 80.18131983976386, 80.28673835125447, 80.41324056504322, 80.48351957270363, 80.58893808419425, 80.68732869491882, 80.79977510717549, 80.9051936186661, 81.00358422939068, 81.10900274088128, 81.13008644317942, 81.1792817485417, 81.2144212523719, 81.19333755007378, 81.19333755007378, 81.20036545083983, 81.20036545083983, 81.20036545083983, 81.19333755007378, 81.18630964930776, 81.1792817485417, 81.17225384777566, 81.17225384777566, 81.16522594700962, 81.16522594700962, 81.13711434394546, 81.12305854241337, 81.10900274088128, 81.09494693934921, 81.08791903858317, 81.08089113781713, 81.059807435519, 81.01061213015672, 80.94033312249631, 80.80680300794153, 80.60299388572632], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 65.06079134162626, "exact_area100": 75.33904978405565}, "test": {"score": [26.688029946402747, 27.8677367212092, 29.197742676459747, 30.507897796557298, 31.840739585401128, 33.41746306326745, 34.690752346652296, 35.96971329722372, 37.31673425403398, 38.507784363213574, 39.764058645038716, 40.95794458781159, 42.236905538383006, 43.28332813430508, 44.14825738025693, 45.257068315231265, 46.48214842753028, 47.256330998497006, 48.24036525536682, 49.10529450131867, 49.8794770722854, 50.744406318237246, 51.50157388764427, 52.21903978674531, 52.97620735615234, 53.73337492555936, 54.58979667073136, 55.32994923857868, 55.97368346425432, 56.58055185321725, 57.159061906247345, 57.868020304568525, 58.49757536227774, 59.10160791764739, 59.52981879023339, 60.1054930096702, 60.76907807049883, 61.32490145478264, 61.87505317187987, 62.379831551484564, 63.017894109973625, 63.60491166378357, 64.16357088166068, 64.60028925502652, 65.04835096276551, 65.55880100955676, 65.96716104698976, 66.53149193205343, 67.02776281087826, 67.50134702095681, 67.99478206618836, 68.45702294189377, 68.90508464963277, 69.42404219720387, 69.91464140884214, 70.41091228766696, 70.94972067039107, 71.3524090406375, 71.72957490854436, 72.10106910926467, 72.46972747639168, 72.80435584039928, 73.13898420440688, 73.48779173638091, 73.73734509258996, 74.02660011910501, 74.28182514250064, 74.45197515809772, 74.61078183932167, 74.73839435101948, 74.87167852990386, 74.95675353770241, 75.04182854550095, 75.07869438221365, 75.09854521736665, 75.14675438845248, 75.15809772282563, 75.15242605563905, 75.15526188923234, 75.15242605563905, 75.15526188923234, 75.15526188923234, 75.15526188923234, 75.15526188923234, 75.15809772282563, 75.15526188923234, 75.15242605563905, 75.14959022204577, 75.14108272126592, 75.1240677197062, 75.13257522048606, 75.12123188611292, 75.1269035532995, 75.1269035532995, 75.11556021892636, 75.10421688455322, 75.09287355018009, 75.07302271502708, 75.01063437597482, 74.90570853302329, 74.47182599325072], "calls": [0.0, 1.250602614638573, 2.6798627456540847, 4.08076454073675, 5.558233842838102, 7.330629838641069, 8.768397470436435, 10.268553441284066, 11.958710262881773, 13.529762073561525, 15.066783881121856, 16.510223180103793, 18.132319995462666, 19.445310949153505, 20.593823554433826, 22.130845361994158, 23.826673850778437, 24.881603947480365, 26.299520744122734, 27.56430252672773, 28.761024303093897, 30.09670192553101, 31.282080367524035, 32.35118963219239, 33.47985140231971, 34.67373734509259, 36.00090746674985, 37.251510081388425, 38.43405268978816, 39.52017695601622, 40.46734537617333, 41.59317131270737, 42.70481808127499, 43.799449848282904, 44.64736409267504, 45.6767716870374, 46.95006097042226, 48.02767773587046, 49.0797719989791, 50.1573887644273, 51.29455803533449, 52.375010634375975, 53.506508238096586, 54.51039333011939, 55.62487593228029, 56.71667186569492, 57.765930295210275, 58.98250290672943, 60.187732183875454, 61.34758812352891, 62.48759322802938, 63.621926665343274, 64.78178260499674, 65.92745937668377, 67.12985282023651, 68.27836542551682, 69.48359470266284, 70.61509230638346, 71.59061906247342, 72.49241414513796, 73.5813742449593, 74.52570683152314, 75.54944275869892, 76.56467118509485, 77.62810878257665, 78.657516376939, 79.74364064316705, 80.53200238210022, 81.36857329211922, 82.12290502793296, 83.0275359441908, 83.8073901823441, 84.63545359158324, 85.43515866488954, 86.33695374755409, 86.99770297478943, 87.66412386921135, 88.31352976207356, 89.28338485097694, 89.8108498993279, 90.41771828829084, 90.82891415931714, 91.26846836627627, 91.72503757479511, 92.24399512236621, 92.62966849105294, 93.0295210277061, 93.46340356747866, 93.9029577744378, 94.27728780875138, 94.67997617899782, 95.12520205314352, 95.64132376712135, 96.02416130221478, 96.50341717947991, 96.9316280520659, 97.37685392621161, 97.84476646910359, 98.28432067606273, 98.84014406034653, 100.0], "exact_score": [26.688029946402747, 27.63803420015314, 28.582366786716957, 29.495505203754647, 30.425658622352042, 31.347304540169585, 32.21223378612143, 33.10268553441284, 34.01298811785725, 34.8949323653688, 35.74851827694751, 36.56240251822023, 37.353600090746674, 38.09375265859399, 38.87077106315402, 39.7101778067663, 40.51271871366588, 41.35779712446473, 42.14048719621133, 42.90899809999149, 43.70303150611122, 44.47437824348467, 45.17199330743272, 45.84692170263449, 46.61543260641466, 47.33573433910898, 48.053200238210025, 48.713949465445374, 49.380370359867285, 50.03261208632277, 50.67067464481184, 51.34560304001361, 51.99217309928253, 52.66426566089102, 53.302328219380094, 53.93188327708931, 54.58979667073136, 55.18248589172787, 55.73830927601169, 56.2941326602955, 56.88115021410544, 57.49652610384823, 58.07220032328504, 58.653546209908406, 59.1923545926325, 59.71131214020362, 60.28131469245385, 60.780421404871966, 61.319229787596065, 61.82967983438732, 62.320279046025576, 62.878938263902675, 63.380880809914075, 63.919689192638174, 64.36491506678388, 64.81864844170944, 65.25253098148201, 65.68924935484786, 66.09477355868758, 66.53999943283328, 66.95119530385958, 67.3567195076993, 67.78493038028527, 68.190454584125, 68.62433712389758, 69.02135382695744, 69.45523636673, 69.84090973541672, 70.271956441596, 70.74270481808126, 71.16240818988742, 71.49703655389501, 71.8742024218019, 72.25420412330205, 72.60584748886936, 72.94331168647024, 73.29779088563083, 73.59838924651902, 73.85361426991463, 74.12585429486998, 74.33003431358648, 74.54555766667612, 74.71854351586649, 74.87167852990386, 74.97944020644869, 75.05600771346738, 75.08436604940023, 75.14675438845248, 75.14675438845248, 75.15526188923234, 75.15242605563905, 75.15526188923234, 75.15526188923234, 75.15242605563905, 75.13824688767264, 75.12973938689278, 75.1269035532995, 75.09854521736665, 75.05884354706066, 74.863171029124, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 41.705753906360776, "exact_area100": 58.85652665082966}}, "old_normalized_endpoint_budget": 77}]}, {"expert": "instancevg", "endpoint": "MiniCPM", "task": "grefcoco", "small": 66.95119530385956, "large": 50.00708958398321, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.75, "validation_area100": 0.7377444145858679, "training_seconds": 101.77041542797815, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.04389889682914449, "operator": "ge"}, {"budget": 0.02, "threshold": 0.00215462816141909, "operator": "ge"}, {"budget": 0.03, "threshold": -0.027139643543089946, "operator": "ge"}, {"budget": 0.04, "threshold": -0.14133671477003992, "operator": "ge"}, {"budget": 0.05, "threshold": -0.2969517126521992, "operator": "ge"}, {"budget": 0.06, "threshold": -0.3472029061501712, "operator": "ge"}, {"budget": 0.07, "threshold": -0.3696867229777693, "operator": "ge"}, {"budget": 0.08, "threshold": -0.3883393460288425, "operator": "ge"}, {"budget": 0.09, "threshold": -0.4066388358238757, "operator": "ge"}, {"budget": 0.1, "threshold": -0.4225419251342562, "operator": "ge"}, {"budget": 0.11, "threshold": -0.4360783616122191, "operator": "ge"}, {"budget": 0.12, "threshold": -0.448160324112066, "operator": "ge"}, {"budget": 0.13, "threshold": -0.4591842028117787, "operator": "ge"}, {"budget": 0.14, "threshold": -0.469362688002524, "operator": "ge"}, {"budget": 0.15, "threshold": -0.47843356546419724, "operator": "ge"}, {"budget": 0.16, "threshold": -0.4889844473635279, "operator": "ge"}, {"budget": 0.17, "threshold": -0.4971595376140168, "operator": "ge"}, {"budget": 0.18, "threshold": -0.5070383095066151, "operator": "ge"}, {"budget": 0.19, "threshold": -0.5143868308873606, "operator": "ge"}, {"budget": 0.2, "threshold": -0.5228069260704153, "operator": "ge"}, {"budget": 0.21, "threshold": -0.5308072596103411, "operator": "ge"}, {"budget": 0.22, "threshold": -0.5399883651403257, "operator": "ge"}, {"budget": 0.23, "threshold": -0.5482863731043298, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5568355646601356, "operator": "ge"}, {"budget": 0.25, "threshold": -0.5659599478004229, "operator": "gt"}, {"budget": 0.26, "threshold": -0.5733186818008809, "operator": "ge"}, {"budget": 0.27, "threshold": -0.581681327329951, "operator": "ge"}, {"budget": 0.28, "threshold": -0.5886290895112327, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5957469239525549, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6024590677310944, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6110669112033718, "operator": "ge"}, {"budget": 0.32, "threshold": -0.617985546511672, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6240500151619538, "operator": "ge"}, {"budget": 0.34, "threshold": -0.6307143403205276, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6365189625751633, "operator": "ge"}, {"budget": 0.36, "threshold": -0.6429600149068206, "operator": "ge"}, {"budget": 0.37, "threshold": -0.6489721629924183, "operator": "ge"}, {"budget": 0.38, "threshold": -0.6555931215911194, "operator": "ge"}, {"budget": 0.39, "threshold": -0.6614576024306664, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6679084571984769, "operator": "ge"}, {"budget": 0.41, "threshold": -0.6742311166578667, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6800682700837887, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6853722186851923, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6909710408948254, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6975349946423306, "operator": "ge"}, {"budget": 0.46, "threshold": -0.7028545099715219, "operator": "ge"}, {"budget": 0.47, "threshold": -0.7084915809300314, "operator": "ge"}, {"budget": 0.48, "threshold": -0.7145393000250574, "operator": "ge"}, {"budget": 0.49, "threshold": -0.7195926049571042, "operator": "ge"}, {"budget": 0.5, "threshold": -0.7240632825010443, "operator": "gt"}, {"budget": 0.51, "threshold": -0.7299581383096704, "operator": "ge"}, {"budget": 0.52, "threshold": -0.7357259718902982, "operator": "ge"}, {"budget": 0.53, "threshold": -0.7407224326541726, "operator": "ge"}, {"budget": 0.54, "threshold": -0.7462650937069523, "operator": "ge"}, {"budget": 0.55, "threshold": -0.7515114339206843, "operator": "ge"}, {"budget": 0.56, "threshold": -0.7572088527308036, "operator": "ge"}, {"budget": 0.57, "threshold": -0.7621762630842719, "operator": "ge"}, {"budget": 0.58, "threshold": -0.7670744513643496, "operator": "ge"}, {"budget": 0.59, "threshold": -0.7718820764782972, "operator": "ge"}, {"budget": 0.6, "threshold": -0.778326234941031, "operator": "ge"}, {"budget": 0.61, "threshold": -0.7840760165847149, "operator": "ge"}, {"budget": 0.62, "threshold": -0.7894414787624581, "operator": "ge"}, {"budget": 0.63, "threshold": -0.7943079371879556, "operator": "ge"}, {"budget": 0.64, "threshold": -0.7996286766322993, "operator": "ge"}, {"budget": 0.65, "threshold": -0.8049946100394058, "operator": "ge"}, {"budget": 0.66, "threshold": -0.8105411914988797, "operator": "ge"}, {"budget": 0.67, "threshold": -0.8164901448673985, "operator": "ge"}, {"budget": 0.68, "threshold": -0.8212848046054702, "operator": "ge"}, {"budget": 0.69, "threshold": -0.8277315400711193, "operator": "ge"}, {"budget": 0.7, "threshold": -0.8335676132404041, "operator": "gt"}, {"budget": 0.71, "threshold": -0.8388257501111389, "operator": "ge"}, {"budget": 0.72, "threshold": -0.8439995580460913, "operator": "ge"}, {"budget": 0.73, "threshold": -0.8483921119610348, "operator": "ge"}, {"budget": 0.74, "threshold": -0.8530875172849962, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8591512710427348, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8642772427331891, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8693218671234456, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8746782096749292, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8798922310334752, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8857101992750291, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8914937163392417, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8963215902425697, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9012728759911196, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9063357452472987, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9118481888211338, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9165894564492303, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9219913928157457, "operator": "gt"}, {"budget": 0.88, "threshold": -0.9265708027631055, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9311055900230665, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9357602190508768, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9408003047072228, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9461471133880726, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9522547810828463, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9580944555827444, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9631462112852083, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9690040250042603, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9749814279684219, "operator": "ge"}, {"budget": 0.98, "threshold": -0.980932702353965, "operator": "gt"}, {"budget": 0.99, "threshold": -0.9867439718305506, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.24070560123691, 75.47965422728231, 75.59210063953897, 75.58507273877294, 75.54993323494273, 75.56398903647481, 75.59210063953897, 75.60615644107105, 75.6131843418371, 75.63426804413523, 75.62724014336918, 75.69049125026355, 75.71157495256166, 75.73968655562584, 75.73265865485979, 75.78888186098813, 75.83104926558437, 75.85916086864853, 75.88727247171269, 75.9224119755429, 75.96457938013916, 76.0067467847354, 75.97863518167124, 76.03485838779956, 76.07702579239582, 76.07702579239582, 76.03485838779956, 76.05594209009769, 76.08405369316186, 76.11919319699206, 76.18244430388643, 76.21055590695059, 76.20352800618456, 76.18947220465247, 76.11216529622602, 76.1613606015883, 76.0910815939279, 76.03485838779956, 76.0418862885656, 75.99269098320332, 75.90835617401083, 75.88024457094666, 75.78888186098813, 75.73968655562584, 75.64832384566729, 75.64832384566729, 75.57804483800689, 75.52182163187857, 75.47262632651626, 75.4445147234521, 75.43748682268607, 75.35315201349357, 75.26178930353502, 75.17745449434254, 75.06500808208588, 75.00175697519151, 74.94553376906318, 74.8822826621688, 74.8120036545084, 74.72064094454987, 74.67847353995361, 74.59413873076113, 74.48169231850447, 74.31302270011948, 74.23571579169302, 74.07407407407408, 73.93351605875326, 73.7367348373041, 73.66645582964368, 73.65942792887765, 73.53995361585494, 73.3431723944058, 73.19558647831893, 72.97069365380561, 72.80202403542062, 72.59118701243938, 72.34521048562794, 72.07815025651838, 71.83920163047299, 71.55808559983133, 71.2980532714878, 70.96071403471782, 70.72176540867244, 70.32117506500808, 70.02600323283436, 69.72380349989459, 69.48485487384917, 69.16157143861129, 68.90856701103381, 68.71178578958465, 68.21983273596177, 67.82627029306346, 67.33431723944058, 66.84939208658373, 66.49799704828168, 66.1606578115117, 65.70384426171903, 65.359477124183, 64.83238456672991, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.991215124042448, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.98945814885094, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.99086372900415, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.99135568205777, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99002038091223, 98.99501019045611, 100.0], "rescued": [0, 41, 78, 97, 111, 132, 144, 152, 155, 158, 164, 170, 181, 189, 196, 200, 211, 222, 229, 239, 250, 258, 268, 274, 285, 293, 300, 305, 314, 324, 333, 347, 358, 366, 373, 379, 397, 401, 407, 419, 430, 435, 444, 452, 461, 470, 483, 494, 501, 510, 526, 544, 559, 571, 584, 592, 609, 621, 636, 654, 671, 693, 711, 730, 743, 764, 779, 798, 810, 838, 863, 882, 894, 908, 922, 938, 958, 976, 993, 1008, 1028, 1051, 1068, 1091, 1106, 1127, 1144, 1167, 1187, 1207, 1230, 1243, 1257, 1266, 1274, 1293, 1316, 1331, 1353, 1366, 1369], "harmed": [0, 11, 14, 17, 32, 58, 68, 72, 73, 75, 78, 85, 87, 92, 95, 100, 103, 108, 111, 117, 123, 125, 129, 139, 142, 144, 151, 162, 168, 174, 178, 183, 190, 199, 208, 225, 236, 250, 264, 275, 293, 310, 323, 344, 360, 382, 395, 416, 431, 447, 467, 486, 513, 538, 563, 587, 613, 633, 657, 685, 715, 743, 773, 808, 845, 877, 915, 954, 994, 1032, 1058, 1094, 1134, 1169, 1215, 1255, 1305, 1358, 1413, 1462, 1522, 1582, 1647, 1704, 1776, 1839, 1899, 1956, 2022, 2078, 2129, 2212, 2282, 2361, 2438, 2507, 2578, 2658, 2729, 2817, 2921], "exact_score": [75.02986857825567, 75.24070560123691, 75.47965422728231, 75.59210063953897, 75.58507273877294, 75.54993323494273, 75.56398903647481, 75.59210063953897, 75.60615644107105, 75.6131843418371, 75.63426804413523, 75.62724014336918, 75.69049125026355, 75.71157495256166, 75.73968655562584, 75.73265865485979, 75.78888186098813, 75.83104926558437, 75.85916086864853, 75.88727247171269, 75.9224119755429, 75.96457938013916, 76.0067467847354, 75.97863518167124, 76.03485838779956, 76.07702579239582, 76.07702579239582, 76.03485838779956, 76.05594209009769, 76.08405369316186, 76.11919319699206, 76.18244430388643, 76.21055590695059, 76.20352800618456, 76.18947220465247, 76.11216529622602, 76.1613606015883, 76.0910815939279, 76.03485838779956, 76.0418862885656, 75.99269098320332, 75.90835617401083, 75.88024457094666, 75.78888186098813, 75.73968655562584, 75.64832384566729, 75.64832384566729, 75.57804483800689, 75.52182163187857, 75.47262632651626, 75.4445147234521, 75.43748682268607, 75.35315201349357, 75.26178930353502, 75.17745449434254, 75.06500808208588, 75.00175697519151, 74.94553376906318, 74.8822826621688, 74.8120036545084, 74.72064094454987, 74.67847353995361, 74.59413873076113, 74.48169231850447, 74.31302270011948, 74.23571579169302, 74.07407407407408, 73.93351605875326, 73.7367348373041, 73.66645582964368, 73.65240002811161, 73.53995361585494, 73.3431723944058, 73.19558647831893, 72.97069365380561, 72.80202403542062, 72.59118701243938, 72.34521048562794, 72.07815025651838, 71.83920163047299, 71.55808559983133, 71.2980532714878, 70.96071403471782, 70.72176540867244, 70.32117506500808, 70.02600323283436, 69.72380349989459, 69.47782697308314, 69.16157143861129, 68.90856701103381, 68.71178578958465, 68.21983273596177, 67.82627029306346, 67.33431723944058, 66.84939208658373, 66.49799704828168, 66.1606578115117, 65.70384426171903, 65.35244922341697, 64.83238456672991, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 73.7744414585868, "threshold_area100": 73.77503689647902, "threshold_area25": 75.67854381896129}, "test": {"score": [66.95119530385956, 67.33119700535973, 67.7253778748263, 67.8756770552704, 67.99478206618835, 68.07418540680032, 68.08269290758018, 68.10821540991975, 68.1337379122593, 68.13940957944588, 68.170603748972, 68.17627541615859, 68.170603748972, 68.16776791537872, 68.170603748972, 68.15075291381902, 68.14224541303916, 68.14508124663244, 68.170603748972, 68.16209624819216, 68.17627541615859, 68.170603748972, 68.1535887474123, 68.12806624507274, 68.11105124351303, 68.09403624195332, 68.09403624195332, 68.09403624195332, 68.03731957008763, 67.98911039900179, 67.98060289822193, 67.94373706150922, 67.92388622635623, 67.8756770552704, 67.85299038652411, 67.83030371777784, 67.7991095482517, 67.75090037716586, 67.69701953889346, 67.62895953265462, 67.54672035844936, 67.49283952017694, 67.44463034909111, 67.40492867878513, 67.34537617332614, 67.25746533193431, 67.1497036553895, 67.06746448118425, 66.95686697104614, 66.89164279840058, 66.82074695856846, 66.6846269460908, 66.58820860391911, 66.45776025862801, 66.35283441567648, 66.20537106882568, 66.05507188838159, 65.9047727079375, 65.73462269234042, 65.63253268298216, 65.38581516036639, 65.20148597680287, 65.00864929245952, 64.81297677452287, 64.55491591753396, 64.38476590193687, 64.13237671213453, 63.90267419107847, 63.60491166378357, 63.36953747554094, 63.07177494824604, 62.776848254544426, 62.484757394436095, 62.22953237104046, 61.9629640132717, 61.56878314380512, 61.23415477979752, 60.84848141111079, 60.4940022119502, 60.105493009670184, 59.69996880583047, 59.35399710744973, 59.04489124578169, 58.653546209908406, 58.21682783654255, 57.84533363582225, 57.48801860306837, 56.99174772424354, 56.59189518759039, 56.23458015483651, 55.834727618183365, 55.38666591044438, 54.969798372231516, 54.479199160593254, 53.92621160990273, 53.43844823185775, 52.82590817570825, 52.13112894535349, 51.37112554235317, 50.70186881433797, 50.00708958398321], "calls": [0.0, 1.5001559708476306, 5.271814649916343, 7.909139891671156, 9.497206703910614, 10.997362674758245, 13.090207866602388, 14.505288829651475, 15.798428948189319, 17.30992825341009, 18.554859200862094, 19.541729291325186, 20.483226044295723, 21.305617786348297, 21.980546181550068, 22.587414570513005, 23.262342965714772, 23.809658849218728, 24.51010974676006, 25.040410628704308, 25.641607350480673, 26.171908232424922, 26.753254119048293, 27.368630008791083, 28.060573405552564, 28.724158466381194, 29.32535518815756, 29.943566911493637, 30.553271134049854, 31.162975356606076, 31.6790970705839, 32.492981311856624, 33.17074554065167, 33.76910642883475, 34.43552732325667, 35.10478405127187, 35.853444119899045, 36.69568669710461, 37.56061594305646, 38.30360434449707, 39.22525026231461, 40.0448061707739, 40.838839576893626, 41.51943963928197, 42.27660720868899, 43.25213396477895, 44.040495703712104, 44.79482743952585, 45.72781669171653, 46.561551768142245, 47.324391004735844, 48.36797776706463, 49.30947452003516, 50.16873209880044, 51.186796358789664, 52.06306893911466, 53.047103195984455, 53.903524941156455, 54.697558347276185, 55.46039758386978, 56.671298528202364, 57.67234778663188, 58.588322037262856, 59.50713212148712, 60.482658877577066, 61.514902305532715, 62.61804157332048, 63.79774834812694, 64.77894677140345, 66.01253438448231, 67.23477866318804, 68.28687292629668, 69.31344468706577, 70.26912060800272, 71.19643819300684, 72.4668916427984, 73.50197090434733, 74.62212517369481, 75.75362277741542, 77.0410912287667, 78.23214133794629, 79.4373706150923, 80.41006153758897, 81.47917080225733, 82.61066840597793, 83.676941837053, 84.68366276266909, 85.9059070413748, 86.88710546465134, 87.79173638090917, 88.71621813232, 89.78532739698835, 90.84309332728355, 91.99727759975045, 93.13728270425091, 94.14683946346028, 95.30385956952047, 96.52610384822619, 97.77954229645805, 98.85999489549954, 100.0], "rescued": [0, 163, 322, 400, 494, 574, 609, 634, 654, 677, 697, 710, 723, 732, 747, 756, 764, 776, 798, 810, 829, 841, 850, 865, 882, 896, 913, 931, 942, 953, 969, 987, 1001, 1011, 1027, 1044, 1065, 1083, 1105, 1125, 1145, 1160, 1181, 1198, 1218, 1239, 1258, 1275, 1292, 1311, 1333, 1352, 1374, 1397, 1424, 1447, 1470, 1495, 1514, 1532, 1561, 1584, 1611, 1624, 1645, 1689, 1722, 1756, 1775, 1821, 1851, 1881, 1903, 1934, 1964, 1999, 2022, 2048, 2076, 2115, 2153, 2185, 2214, 2243, 2264, 2294, 2316, 2343, 2365, 2389, 2410, 2441, 2476, 2506, 2525, 2546, 2564, 2583, 2599, 2617, 2627], "harmed": [0, 29, 49, 74, 126, 178, 210, 226, 237, 258, 267, 278, 293, 303, 317, 333, 344, 355, 368, 383, 397, 411, 426, 450, 473, 493, 510, 528, 559, 587, 606, 637, 658, 685, 709, 734, 766, 801, 842, 886, 935, 969, 1007, 1038, 1079, 1131, 1188, 1234, 1290, 1332, 1379, 1446, 1502, 1571, 1635, 1710, 1786, 1864, 1943, 1997, 2113, 2201, 2296, 2378, 2490, 2594, 2716, 2831, 2955, 3084, 3219, 3353, 3478, 3599, 3723, 3897, 4038, 4200, 4353, 4529, 4710, 4864, 5002, 5169, 5344, 5505, 5653, 5855, 6018, 6168, 6330, 6519, 6701, 6904, 7118, 7311, 7545, 7809, 8093, 8347, 8602], "exact_score": [66.95119530385956, 67.22910699600148, 67.43045118112468, 67.5410486912628, 67.61761619828148, 67.70552703967331, 67.75373621075916, 67.81045288262484, 67.8926920568301, 67.97209539744208, 68.02314040212121, 68.07418540680032, 68.07418540680032, 68.0770212403936, 68.08269290758018, 68.11955874429289, 68.13940957944588, 68.14791708022572, 68.15642458100558, 68.17911124975187, 68.18478291693843, 68.16493208178544, 68.170603748972, 68.14224541303916, 68.15075291381902, 68.1535887474123, 68.15926041459886, 68.15075291381902, 68.11388707710631, 68.08552874117346, 68.08836457476674, 67.98911039900179, 67.99194623259507, 67.93806539432265, 67.86149788730397, 67.83030371777784, 67.7991095482517, 67.74239287638602, 67.65731786858747, 67.57224286078892, 67.49000368658366, 67.41627201315826, 67.36239117488586, 67.26313699912089, 67.15537532257608, 67.05044947962453, 66.92567280152001, 66.8320902929416, 66.73283611717665, 66.61089527266539, 66.47477526018774, 66.37835691801605, 66.21387856960553, 66.06925105634801, 65.8877577063778, 65.68641352125456, 65.51059183847092, 65.32909848850069, 65.15044097212376, 64.91223095028784, 64.69103593001162, 64.48685591129512, 64.27133255820549, 64.05013753792927, 63.817599183279924, 63.56237415988429, 63.36670164194766, 63.12565578651844, 62.847744094376544, 62.581175736607776, 62.33445821399201, 62.028188185917244, 61.69639565550293, 61.39863312820803, 61.078183932166866, 60.70101806425998, 60.423106372118085, 60.122508011229904, 59.78504381362902, 59.46743045118112, 59.149817088733236, 58.81802455831892, 58.460709525565036, 58.08921532484474, 57.737571959277425, 57.36607775855712, 56.93503105237785, 56.5521935172844, 56.140997646258114, 55.74398094319826, 55.31009840342569, 54.9159175339591, 54.479199160593254, 53.99710744973485, 53.50934407168987, 52.987550690525474, 52.414712304681956, 51.83903808524516, 51.223662195502364, 50.60545047216629, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.88991454829913, "threshold_area100": 64.22217338286588, "threshold_area25": 68.01888665173128}, "selected_budget": 32, "selected": {"val": {"score": 76.21055590695059, "calls": 31.99803218778551, "rescued": 358, "harmed": 190}, "test": {"score": 67.92388622635623, "calls": 33.17074554065167, "rescued": 1001, "harmed": 658}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.06684014201164246, "operator": "ge"}, {"budget": 0.02, "threshold": 0.003607924096286297, "operator": "ge"}, {"budget": 0.03, "threshold": -0.013560247607529163, "operator": "ge"}, {"budget": 0.04, "threshold": -0.018461542204022408, "operator": "ge"}, {"budget": 0.05, "threshold": -0.02295459248125553, "operator": "ge"}, {"budget": 0.06, "threshold": -0.02855498529970646, "operator": "ge"}, {"budget": 0.07, "threshold": -0.03360140696167946, "operator": "ge"}, {"budget": 0.08, "threshold": -0.03835809975862503, "operator": "ge"}, {"budget": 0.09, "threshold": -0.04352317377924919, "operator": "ge"}, {"budget": 0.1, "threshold": -0.04846625775098801, "operator": "ge"}, {"budget": 0.11, "threshold": -0.05318620428442955, "operator": "ge"}, {"budget": 0.12, "threshold": -0.058227814733982086, "operator": "ge"}, {"budget": 0.13, "threshold": -0.06333580613136292, "operator": "ge"}, {"budget": 0.14, "threshold": -0.06811457127332687, "operator": "ge"}, {"budget": 0.15, "threshold": -0.0733768418431282, "operator": "ge"}, {"budget": 0.16, "threshold": -0.07817420363426208, "operator": "ge"}, {"budget": 0.17, "threshold": -0.08408960700035095, "operator": "ge"}, {"budget": 0.18, "threshold": -0.09063904732465744, "operator": "ge"}, {"budget": 0.19, "threshold": -0.0981009304523468, "operator": "ge"}, {"budget": 0.2, "threshold": -0.10491407662630081, "operator": "ge"}, {"budget": 0.21, "threshold": -0.11162105202674866, "operator": "ge"}, {"budget": 0.22, "threshold": -0.11805468052625656, "operator": "ge"}, {"budget": 0.23, "threshold": -0.1246107891201973, "operator": "ge"}, {"budget": 0.24, "threshold": -0.1319780796766281, "operator": "ge"}, {"budget": 0.25, "threshold": -0.13983626663684845, "operator": "ge"}, {"budget": 0.26, "threshold": -0.1471484750509262, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1539750099182129, "operator": "ge"}, {"budget": 0.28, "threshold": -0.16117174923419952, "operator": "ge"}, {"budget": 0.29, "threshold": -0.16766901314258575, "operator": "ge"}, {"budget": 0.3, "threshold": -0.17500510811805725, "operator": "ge"}, {"budget": 0.31, "threshold": -0.18342731893062592, "operator": "gt"}, {"budget": 0.32, "threshold": -0.1906864196062088, "operator": "ge"}, {"budget": 0.33, "threshold": -0.19913350045681, "operator": "ge"}, {"budget": 0.34, "threshold": -0.2085910141468048, "operator": "ge"}, {"budget": 0.35, "threshold": -0.21744704246520996, "operator": "ge"}, {"budget": 0.36, "threshold": -0.22520104050636292, "operator": "ge"}, {"budget": 0.37, "threshold": -0.23367348313331604, "operator": "ge"}, {"budget": 0.38, "threshold": -0.24345801770687103, "operator": "ge"}, {"budget": 0.39, "threshold": -0.25276413559913635, "operator": "ge"}, {"budget": 0.4, "threshold": -0.26275667548179626, "operator": "ge"}, {"budget": 0.41, "threshold": -0.2716602385044098, "operator": "ge"}, {"budget": 0.42, "threshold": -0.28249409794807434, "operator": "ge"}, {"budget": 0.43, "threshold": -0.29068174958229065, "operator": "ge"}, {"budget": 0.44, "threshold": -0.30058524012565613, "operator": "ge"}, {"budget": 0.45, "threshold": -0.31016069650650024, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3189404308795929, "operator": "ge"}, {"budget": 0.47, "threshold": -0.32779955863952637, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3390337824821472, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3477238118648529, "operator": "ge"}, {"budget": 0.5, "threshold": -0.35742878913879395, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3688663840293884, "operator": "ge"}, {"budget": 0.52, "threshold": -0.3800906240940094, "operator": "ge"}, {"budget": 0.53, "threshold": -0.39211219549179077, "operator": "gt"}, {"budget": 0.54, "threshold": -0.4044570028781891, "operator": "ge"}, {"budget": 0.55, "threshold": -0.4159507751464844, "operator": "ge"}, {"budget": 0.56, "threshold": -0.42614808678627014, "operator": "ge"}, {"budget": 0.57, "threshold": -0.4382757842540741, "operator": "ge"}, {"budget": 0.58, "threshold": -0.4507174491882324, "operator": "gt"}, {"budget": 0.59, "threshold": -0.4625535309314728, "operator": "ge"}, {"budget": 0.6, "threshold": -0.47711268067359924, "operator": "ge"}, {"budget": 0.61, "threshold": -0.488974928855896, "operator": "ge"}, {"budget": 0.62, "threshold": -0.4998367428779602, "operator": "ge"}, {"budget": 0.63, "threshold": -0.512238621711731, "operator": "ge"}, {"budget": 0.64, "threshold": -0.5239039063453674, "operator": "ge"}, {"budget": 0.65, "threshold": -0.5351022481918335, "operator": "ge"}, {"budget": 0.66, "threshold": -0.5474764704704285, "operator": "ge"}, {"budget": 0.67, "threshold": -0.558942973613739, "operator": "ge"}, {"budget": 0.68, "threshold": -0.5721806883811951, "operator": "ge"}, {"budget": 0.69, "threshold": -0.5868760943412781, "operator": "ge"}, {"budget": 0.7, "threshold": -0.5979882478713989, "operator": "ge"}, {"budget": 0.71, "threshold": -0.6128913164138794, "operator": "ge"}, {"budget": 0.72, "threshold": -0.627692699432373, "operator": "ge"}, {"budget": 0.73, "threshold": -0.6414316296577454, "operator": "ge"}, {"budget": 0.74, "threshold": -0.6548546552658081, "operator": "ge"}, {"budget": 0.75, "threshold": -0.6688836812973022, "operator": "ge"}, {"budget": 0.76, "threshold": -0.6820736527442932, "operator": "ge"}, {"budget": 0.77, "threshold": -0.6951404809951782, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7093390822410583, "operator": "ge"}, {"budget": 0.79, "threshold": -0.7233247756958008, "operator": "ge"}, {"budget": 0.8, "threshold": -0.7363866567611694, "operator": "ge"}, {"budget": 0.81, "threshold": -0.7513159513473511, "operator": "ge"}, {"budget": 0.82, "threshold": -0.7632137537002563, "operator": "ge"}, {"budget": 0.83, "threshold": -0.7748051285743713, "operator": "ge"}, {"budget": 0.84, "threshold": -0.7870767116546631, "operator": "ge"}, {"budget": 0.85, "threshold": -0.7990886569023132, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8104186058044434, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8243083357810974, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8393727540969849, "operator": "ge"}, {"budget": 0.89, "threshold": -0.8520792722702026, "operator": "ge"}, {"budget": 0.9, "threshold": -0.865047037601471, "operator": "ge"}, {"budget": 0.91, "threshold": -0.8773683309555054, "operator": "ge"}, {"budget": 0.92, "threshold": -0.889702558517456, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9031264781951904, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9142016768455505, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9252929091453552, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9367820024490356, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9490301609039307, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9599527716636658, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9733754396438599, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.24070560123691, 75.47965422728231, 75.55696113570876, 75.57101693724084, 75.57804483800689, 75.58507273877294, 75.59210063953897, 75.56398903647481, 75.57101693724084, 75.54993323494273, 75.55696113570876, 75.54993323494273, 75.55696113570876, 75.56398903647481, 75.55696113570876, 75.58507273877294, 75.57804483800689, 75.54290533417668, 75.5288495326446, 75.56398903647481, 75.55696113570876, 75.51479373111252, 75.50776583034647, 75.50073792958042, 75.53587743341063, 75.51479373111252, 75.47965422728231, 75.47262632651626, 75.43748682268607, 75.35315201349357, 75.2899009065992, 75.22664979970483, 75.19151029587462, 75.10717548668214, 75.0509522805538, 74.94553376906318, 74.90336636446693, 74.8120036545084, 74.74172464684798, 74.67144563918757, 74.60116663152716, 74.53088762386675, 74.51683182233467, 74.39735750931197, 74.3341064024176, 74.19354838709677, 74.07407407407408, 74.0459624710099, 73.9756834633495, 73.91946025722117, 73.8070138449645, 73.76484644036826, 73.72970693653805, 73.65240002811161, 73.53995361585494, 73.47670250896059, 73.36425609670391, 73.30803289057559, 73.15341907372269, 73.03394476069998, 73.04800056223206, 72.92149834844332, 72.77391243235645, 72.67552182163189, 72.53496380631105, 72.40846159252231, 72.23276407337129, 72.17654086724295, 71.99381544732589, 71.81811792817486, 71.64242040902383, 71.5510576990653, 71.3121090730199, 71.13641155386887, 70.96071403471782, 70.81312811863097, 70.7498770117366, 70.50390048492515, 70.32820296577412, 70.18061704968726, 70.01897533206831, 69.75191510295875, 69.66758029376625, 69.4216037669548, 69.30212945393211, 69.16859933937734, 68.99992972099234, 68.7539531941809, 68.40255815587884, 68.12847002600323, 67.86843769765972, 67.51704265935766, 67.35540094173871, 66.9266989950102, 66.68072246819875, 66.39257853679106, 65.9217091854663, 65.36650502494905, 64.74102185677138, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.986014477475578, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99037177595053, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.987209220605806, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 41, 78, 89, 92, 93, 95, 97, 98, 99, 99, 102, 104, 107, 109, 110, 115, 119, 120, 122, 129, 132, 135, 138, 141, 147, 150, 154, 161, 166, 169, 171, 174, 183, 187, 191, 198, 203, 211, 217, 222, 230, 238, 254, 266, 276, 284, 294, 309, 320, 334, 348, 369, 391, 413, 430, 449, 464, 486, 501, 515, 540, 554, 570, 590, 608, 623, 640, 667, 685, 705, 729, 755, 773, 798, 827, 856, 889, 910, 932, 961, 997, 1015, 1045, 1063, 1093, 1123, 1150, 1172, 1196, 1216, 1240, 1258, 1284, 1300, 1321, 1342, 1351, 1360, 1365, 1369], "harmed": [0, 11, 14, 14, 15, 15, 16, 17, 22, 22, 25, 27, 30, 32, 33, 35, 36, 41, 47, 51, 53, 57, 66, 70, 74, 75, 81, 90, 98, 108, 123, 134, 146, 160, 176, 188, 210, 221, 242, 258, 273, 291, 309, 327, 356, 375, 403, 430, 449, 470, 492, 522, 549, 576, 609, 642, 670, 701, 731, 768, 799, 822, 854, 891, 925, 963, 996, 1038, 1073, 1117, 1162, 1211, 1250, 1302, 1352, 1406, 1456, 1498, 1554, 1601, 1651, 1710, 1766, 1808, 1861, 1908, 1957, 2008, 2065, 2139, 2198, 2259, 2327, 2376, 2453, 2509, 2571, 2647, 2735, 2829, 2921], "exact_score": [75.02986857825567, 75.24070560123691, 75.47965422728231, 75.55696113570876, 75.57101693724084, 75.57804483800689, 75.58507273877294, 75.59210063953897, 75.56398903647481, 75.57101693724084, 75.54993323494273, 75.55696113570876, 75.54993323494273, 75.55696113570876, 75.56398903647481, 75.55696113570876, 75.58507273877294, 75.57804483800689, 75.54290533417668, 75.5288495326446, 75.56398903647481, 75.55696113570876, 75.51479373111252, 75.50776583034647, 75.50073792958042, 75.53587743341063, 75.51479373111252, 75.47965422728231, 75.47262632651626, 75.43748682268607, 75.35315201349357, 75.2899009065992, 75.22664979970483, 75.19151029587462, 75.10717548668214, 75.0509522805538, 74.94553376906318, 74.90336636446693, 74.8120036545084, 74.74172464684798, 74.67144563918757, 74.60116663152716, 74.53088762386675, 74.51683182233467, 74.39735750931197, 74.3341064024176, 74.19354838709677, 74.07407407407408, 74.0459624710099, 73.9756834633495, 73.91946025722117, 73.8070138449645, 73.76484644036826, 73.72267903577202, 73.65240002811161, 73.53995361585494, 73.47670250896059, 73.36425609670391, 73.30803289057559, 73.15341907372269, 73.03394476069998, 73.04800056223206, 72.92149834844332, 72.77391243235645, 72.67552182163189, 72.53496380631105, 72.40846159252231, 72.23276407337129, 72.17654086724295, 71.99381544732589, 71.81811792817486, 71.64242040902383, 71.5510576990653, 71.3121090730199, 71.13641155386887, 70.96071403471782, 70.81312811863097, 70.7498770117366, 70.50390048492515, 70.32820296577412, 70.18061704968726, 70.01897533206831, 69.75191510295875, 69.66758029376625, 69.4216037669548, 69.30212945393211, 69.16859933937734, 68.99992972099234, 68.7539531941809, 68.40255815587884, 68.12847002600323, 67.86843769765972, 67.51704265935766, 67.35540094173871, 66.9266989950102, 66.68072246819875, 66.39257853679106, 65.9217091854663, 65.36650502494905, 64.74102185677138, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 72.86784966847586, "threshold_area100": 72.86829713964438, "threshold_area25": 75.50636025019328}, "test": {"score": [66.95119530385956, 67.33686867254629, 67.7253778748263, 67.82746788418456, 67.8388112185577, 67.84164705215098, 67.87000538808383, 67.87851288886368, 67.88702038964354, 67.88418455605026, 67.88985622323682, 67.88418455605026, 67.87851288886368, 67.8756770552704, 67.88702038964354, 67.90119955760996, 67.88418455605026, 67.90119955760996, 67.8955278904234, 67.86433372089725, 67.85866205371069, 67.85015455293083, 67.82463205059128, 67.81328871621814, 67.78776621387857, 67.77358704591214, 67.75090037716586, 67.71970620763973, 67.68284037092704, 67.65731786858747, 67.6317953662479, 67.58358619516207, 67.5013470209568, 67.41060034597169, 67.36522700847914, 67.32268950457987, 67.2773161670873, 67.20074866006863, 67.1128378186768, 66.99940447494541, 66.92567280152001, 66.86895612965431, 66.73850778436321, 66.65626861015795, 66.58253693673255, 66.47477526018774, 66.35283441567648, 66.26775940787795, 66.18552023367269, 66.06357938916145, 65.97283271417633, 65.78850353061281, 65.700592689221, 65.56163684315004, 65.39715849473953, 65.16462014009018, 65.02282846042594, 64.85551427842215, 64.63148342455266, 64.41028840427644, 64.1749142160338, 64.01610753480985, 63.84595751921277, 63.66730000283584, 63.40073164506707, 63.19087995916399, 62.952669937328075, 62.72863908345858, 62.46774239287638, 62.20117403510762, 62.00833735076425, 61.69923148909622, 61.41564812976774, 61.149079771998984, 60.84848141111079, 60.457136375237496, 60.21041885262173, 59.844596319088, 59.5468337917931, 59.129966253580236, 58.81802455831892, 58.500411195871024, 58.17712616623656, 57.89354280690808, 57.59861611320647, 57.26115191560559, 56.95488188753084, 56.55502935087768, 56.08428097439242, 55.67024926977284, 55.346964240138384, 54.873380030059835, 54.41681082154099, 53.883674106003454, 53.38173155999206, 52.871281513200806, 52.343816464849844, 51.79650058134588, 51.2548563650285, 50.62530130731928, 50.00708958398321], "calls": [0.0, 1.5001559708476306, 5.274650483509627, 8.473470776734821, 10.041686753821285, 11.320647704392705, 12.849162011173185, 13.972152114113944, 14.933499702237473, 15.926041459887132, 16.57544735274934, 17.247539914357827, 17.8657516376939, 18.46978419306355, 18.974562572668237, 19.47934095227292, 19.986955165470892, 20.49456937866886, 20.993676091086975, 21.609051980829765, 22.164875365113577, 22.737713750957095, 23.222641295408785, 23.78697218047245, 24.36548223350254, 25.077276465417008, 25.681309020786657, 26.214445736324194, 26.86952329637297, 27.487735019709042, 28.15699174772424, 28.89147264838499, 29.60326688029946, 30.385956952046055, 31.23670703003148, 32.008053767404924, 32.699997164166405, 33.428806397640585, 34.3362731474917, 35.08493321611888, 35.93284746051102, 36.79494087286958, 37.81016929926552, 38.56166520148598, 39.43793778181096, 40.33689703088223, 41.19331877605422, 41.94765051186796, 42.977058106230324, 43.77676317953663, 44.536766582536934, 45.534980007373164, 46.553044267362395, 47.491705186739644, 48.59484445452741, 49.550520375464366, 50.3984346198565, 51.43067804781215, 52.56501148512606, 53.72203159118622, 55.003828375350935, 56.03890763689987, 57.05697189688909, 58.08354365765817, 59.12145875280038, 60.08847800811048, 61.17460227433854, 62.17281569917478, 63.258939965402824, 64.5095425800414, 65.5502935087769, 66.82358279216174, 68.16493208178545, 69.37583302611803, 70.56971896889091, 71.95360576241386, 73.12480503644046, 74.34704931514618, 75.59198026259818, 76.8539262116099, 77.97975214814394, 79.26438476590194, 80.26543402433146, 81.27782661713411, 82.36962255054874, 83.38201514335138, 84.39440773615404, 85.54292034143437, 86.84740379434534, 87.96188639650624, 89.04517482914103, 90.15398576411536, 91.17488585769787, 92.34324929813118, 93.46340356747866, 94.53818449933358, 95.5590845929161, 96.71894053256955, 97.73416895896548, 98.9195474009585, 100.0], "rescued": [0, 162, 322, 369, 379, 382, 396, 400, 409, 416, 424, 428, 432, 442, 448, 458, 467, 478, 485, 492, 501, 513, 521, 531, 547, 559, 567, 574, 580, 592, 605, 614, 625, 633, 647, 663, 676, 686, 705, 711, 732, 750, 770, 778, 799, 816, 836, 866, 904, 923, 945, 970, 1011, 1040, 1076, 1087, 1120, 1154, 1188, 1229, 1264, 1311, 1347, 1388, 1424, 1460, 1499, 1532, 1560, 1611, 1643, 1687, 1737, 1783, 1823, 1861, 1908, 1948, 1995, 2040, 2079, 2124, 2151, 2182, 2228, 2259, 2298, 2335, 2374, 2404, 2445, 2467, 2488, 2511, 2532, 2556, 2575, 2597, 2609, 2622, 2627], "harmed": [0, 26, 49, 60, 66, 68, 72, 73, 79, 87, 93, 99, 105, 116, 118, 123, 138, 143, 152, 170, 181, 196, 213, 227, 252, 269, 285, 303, 322, 343, 365, 391, 431, 471, 501, 532, 561, 598, 648, 694, 741, 779, 845, 882, 929, 984, 1047, 1107, 1174, 1236, 1290, 1380, 1452, 1530, 1624, 1717, 1800, 1893, 2006, 2125, 2243, 2346, 2442, 2546, 2676, 2786, 2909, 3021, 3141, 3286, 3386, 3539, 3689, 3829, 3975, 4151, 4285, 4454, 4606, 4798, 4947, 5104, 5245, 5376, 5526, 5676, 5823, 6001, 6206, 6382, 6537, 6726, 6908, 7119, 7317, 7521, 7726, 7941, 8144, 8379, 8602], "exact_score": [66.95119530385956, 67.22910699600148, 67.42477951393812, 67.54388452485608, 67.6147803646882, 67.70552703967331, 67.75090037716586, 67.79627371465841, 67.81612454981142, 67.8388112185577, 67.8388112185577, 67.8388112185577, 67.8558262201174, 67.87000538808383, 67.87851288886368, 67.88702038964354, 67.88702038964354, 67.88702038964354, 67.8756770552704, 67.88702038964354, 67.88418455605026, 67.8955278904234, 67.86716955449053, 67.83597538496441, 67.79060204747185, 67.77642287950543, 67.73388537560616, 67.66582536936733, 67.6346311998412, 67.56089952641578, 67.46164535065083, 67.37940617644557, 67.31985367098659, 67.26880866630745, 67.14403198820294, 67.01358364291184, 66.92283696792673, 66.84343362731475, 66.72149278280351, 66.63358194141168, 66.53432776564671, 66.40104358676233, 66.25925190709809, 66.18835606726597, 66.02671355244874, 65.89342937356436, 65.73178685874713, 65.64104018376202, 65.45954683379179, 65.31208348694099, 65.11073930181777, 64.93208178544083, 64.77611093781017, 64.53222924878767, 64.37625840115702, 64.18058588322037, 64.03028670277628, 63.84312168561949, 63.678643337208975, 63.42341831381334, 63.21356662791027, 63.023565777160194, 62.756997419391425, 62.55565323426821, 62.33162238039872, 62.096248192156075, 61.86370983750673, 61.656693985196945, 61.45534980007373, 61.22848311261095, 60.993108924368315, 60.74639140175254, 60.44862887445765, 60.230269687774715, 59.926835493293254, 59.67161046989763, 59.42489294728185, 59.084592916087686, 58.81235289113235, 58.54578453336357, 58.27070867481496, 57.9672744803335, 57.67801945381845, 57.38592859371012, 57.068315231262225, 56.74503020162776, 56.45293934151944, 56.0133851345603, 55.66174176899299, 55.36397924169809, 54.90741003317925, 54.49054249496639, 54.048152454413966, 53.59441907948841, 53.157700706122554, 52.66142982729774, 52.14530811331991, 51.640529733715226, 51.09888551739784, 50.574256302640165, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.13558265164632, "threshold_area100": 63.594404900320455, "threshold_area25": 67.77075121231887}, "selected_budget": 7, "selected": {"val": {"score": 75.59210063953897, "calls": 6.999789162977018, "rescued": 97, "harmed": 17}, "test": {"score": 67.87851288886368, "calls": 13.972152114113944, "rescued": 400, "harmed": 73}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.907368779182434, "operator": "ge"}, {"budget": 0.02, "threshold": 0.29674291610717773, "operator": "ge"}, {"budget": 0.03, "threshold": -0.7350304126739502, "operator": "ge"}, {"budget": 0.04, "threshold": -1.3913686275482178, "operator": "ge"}, {"budget": 0.05, "threshold": -1.974894404411316, "operator": "ge"}, {"budget": 0.06, "threshold": -2.410123109817505, "operator": "ge"}, {"budget": 0.07, "threshold": -2.7351534366607666, "operator": "ge"}, {"budget": 0.08, "threshold": -3.040325403213501, "operator": "ge"}, {"budget": 0.09, "threshold": -3.326674222946167, "operator": "ge"}, {"budget": 0.1, "threshold": -3.5511679649353027, "operator": "ge"}, {"budget": 0.11, "threshold": -3.764580488204956, "operator": "ge"}, {"budget": 0.12, "threshold": -3.9540843963623047, "operator": "ge"}, {"budget": 0.13, "threshold": -4.152113437652588, "operator": "ge"}, {"budget": 0.14, "threshold": -4.2994794845581055, "operator": "ge"}, {"budget": 0.15, "threshold": -4.4818902015686035, "operator": "ge"}, {"budget": 0.16, "threshold": -4.649930000305176, "operator": "ge"}, {"budget": 0.17, "threshold": -4.764713764190674, "operator": "ge"}, {"budget": 0.18, "threshold": -4.9197998046875, "operator": "ge"}, {"budget": 0.19, "threshold": -5.028555870056152, "operator": "ge"}, {"budget": 0.2, "threshold": -5.145881175994873, "operator": "ge"}, {"budget": 0.21, "threshold": -5.254449844360352, "operator": "ge"}, {"budget": 0.22, "threshold": -5.367876052856445, "operator": "ge"}, {"budget": 0.23, "threshold": -5.477180480957031, "operator": "ge"}, {"budget": 0.24, "threshold": -5.581151008605957, "operator": "ge"}, {"budget": 0.25, "threshold": -5.677910804748535, "operator": "ge"}, {"budget": 0.26, "threshold": -5.77174711227417, "operator": "ge"}, {"budget": 0.27, "threshold": -5.856370449066162, "operator": "ge"}, {"budget": 0.28, "threshold": -5.943875789642334, "operator": "ge"}, {"budget": 0.29, "threshold": -6.013935089111328, "operator": "ge"}, {"budget": 0.3, "threshold": -6.073960781097412, "operator": "ge"}, {"budget": 0.31, "threshold": -6.1430792808532715, "operator": "ge"}, {"budget": 0.32, "threshold": -6.20357608795166, "operator": "ge"}, {"budget": 0.33, "threshold": -6.257369518280029, "operator": "ge"}, {"budget": 0.34, "threshold": -6.309410572052002, "operator": "ge"}, {"budget": 0.35, "threshold": -6.355228424072266, "operator": "ge"}, {"budget": 0.36, "threshold": -6.409513473510742, "operator": "ge"}, {"budget": 0.37, "threshold": -6.45858907699585, "operator": "ge"}, {"budget": 0.38, "threshold": -6.505068778991699, "operator": "ge"}, {"budget": 0.39, "threshold": -6.548607349395752, "operator": "ge"}, {"budget": 0.4, "threshold": -6.586440086364746, "operator": "ge"}, {"budget": 0.41, "threshold": -6.625413417816162, "operator": "ge"}, {"budget": 0.42, "threshold": -6.667481422424316, "operator": "ge"}, {"budget": 0.43, "threshold": -6.703432559967041, "operator": "ge"}, {"budget": 0.44, "threshold": -6.738443374633789, "operator": "ge"}, {"budget": 0.45, "threshold": -6.780999660491943, "operator": "ge"}, {"budget": 0.46, "threshold": -6.827785968780518, "operator": "ge"}, {"budget": 0.47, "threshold": -6.864542007446289, "operator": "ge"}, {"budget": 0.48, "threshold": -6.905089378356934, "operator": "ge"}, {"budget": 0.49, "threshold": -6.940399169921875, "operator": "ge"}, {"budget": 0.5, "threshold": -6.975949764251709, "operator": "ge"}, {"budget": 0.51, "threshold": -7.0092058181762695, "operator": "ge"}, {"budget": 0.52, "threshold": -7.042628288269043, "operator": "ge"}, {"budget": 0.53, "threshold": -7.073745250701904, "operator": "ge"}, {"budget": 0.54, "threshold": -7.102903366088867, "operator": "ge"}, {"budget": 0.55, "threshold": -7.136603832244873, "operator": "ge"}, {"budget": 0.56, "threshold": -7.168099403381348, "operator": "ge"}, {"budget": 0.57, "threshold": -7.201175689697266, "operator": "ge"}, {"budget": 0.58, "threshold": -7.233397006988525, "operator": "gt"}, {"budget": 0.59, "threshold": -7.2634124755859375, "operator": "ge"}, {"budget": 0.6, "threshold": -7.29368782043457, "operator": "ge"}, {"budget": 0.61, "threshold": -7.327654838562012, "operator": "ge"}, {"budget": 0.62, "threshold": -7.361531734466553, "operator": "ge"}, {"budget": 0.63, "threshold": -7.3981757164001465, "operator": "ge"}, {"budget": 0.64, "threshold": -7.432180404663086, "operator": "ge"}, {"budget": 0.65, "threshold": -7.464946269989014, "operator": "ge"}, {"budget": 0.66, "threshold": -7.5045599937438965, "operator": "ge"}, {"budget": 0.67, "threshold": -7.540406227111816, "operator": "ge"}, {"budget": 0.68, "threshold": -7.569867134094238, "operator": "ge"}, {"budget": 0.69, "threshold": -7.599384307861328, "operator": "ge"}, {"budget": 0.7, "threshold": -7.633002758026123, "operator": "ge"}, {"budget": 0.71, "threshold": -7.668171405792236, "operator": "ge"}, {"budget": 0.72, "threshold": -7.6982808113098145, "operator": "ge"}, {"budget": 0.73, "threshold": -7.734137535095215, "operator": "ge"}, {"budget": 0.74, "threshold": -7.773462772369385, "operator": "ge"}, {"budget": 0.75, "threshold": -7.806607723236084, "operator": "ge"}, {"budget": 0.76, "threshold": -7.848659992218018, "operator": "ge"}, {"budget": 0.77, "threshold": -7.882244110107422, "operator": "ge"}, {"budget": 0.78, "threshold": -7.917792797088623, "operator": "ge"}, {"budget": 0.79, "threshold": -7.9579877853393555, "operator": "ge"}, {"budget": 0.8, "threshold": -7.9925217628479, "operator": "ge"}, {"budget": 0.81, "threshold": -8.034534454345703, "operator": "ge"}, {"budget": 0.82, "threshold": -8.072068214416504, "operator": "ge"}, {"budget": 0.83, "threshold": -8.111568450927734, "operator": "ge"}, {"budget": 0.84, "threshold": -8.162764549255371, "operator": "ge"}, {"budget": 0.85, "threshold": -8.206071853637695, "operator": "ge"}, {"budget": 0.86, "threshold": -8.260101318359375, "operator": "ge"}, {"budget": 0.87, "threshold": -8.308558464050293, "operator": "ge"}, {"budget": 0.88, "threshold": -8.369813919067383, "operator": "ge"}, {"budget": 0.89, "threshold": -8.426634788513184, "operator": "ge"}, {"budget": 0.9, "threshold": -8.492280006408691, "operator": "ge"}, {"budget": 0.91, "threshold": -8.550548553466797, "operator": "ge"}, {"budget": 0.92, "threshold": -8.629343032836914, "operator": "ge"}, {"budget": 0.93, "threshold": -8.73451042175293, "operator": "ge"}, {"budget": 0.94, "threshold": -8.840222358703613, "operator": "ge"}, {"budget": 0.95, "threshold": -8.961996078491211, "operator": "ge"}, {"budget": 0.96, "threshold": -9.10254955291748, "operator": "ge"}, {"budget": 0.97, "threshold": -9.278970718383789, "operator": "ge"}, {"budget": 0.98, "threshold": -9.516218185424805, "operator": "ge"}, {"budget": 0.99, "threshold": -9.929460525512695, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.36017991425962, 75.47965422728231, 75.50776583034647, 75.57804483800689, 75.59210063953897, 75.76077025792397, 75.88024457094666, 76.01377468550145, 76.15433270082227, 76.21055590695059, 76.40733712839975, 76.66034155597723, 76.76576006746785, 76.94145758661888, 77.06795980040762, 77.1663504111322, 77.15229460960012, 77.15932251036615, 77.3209642279851, 77.30690842645302, 77.36313163258134, 77.43341064024176, 77.38421533487947, 77.38421533487947, 77.28582472415489, 77.1663504111322, 76.99065289198117, 76.94145758661888, 76.81495537283014, 76.64628575444516, 76.5197835406564, 76.44247663222995, 76.35814182303746, 76.28786281537705, 76.18947220465247, 76.0208025862675, 75.85916086864853, 75.6131843418371, 75.39531941808981, 75.27584510506712, 75.1352870897463, 75.0509522805538, 74.86119895987069, 74.67144563918757, 74.53088762386675, 74.36924590624781, 74.17949258556469, 74.03190666947783, 73.94757186028534, 73.77890224190034, 73.64537212734557, 73.50481411202475, 73.35020029517183, 73.08314006606227, 72.84419144001687, 72.6614660200998, 72.52793590554502, 72.34521048562794, 72.20465247030712, 72.07815025651838, 71.9446201419636, 71.74783892051444, 71.60025300442759, 71.41752758451051, 71.24885796612553, 71.10829995080469, 70.99585353854803, 70.86232342399326, 70.7358212102045, 70.52498418722328, 70.32117506500808, 70.13844964509101, 69.96275212593999, 69.71677559912854, 69.58324548457377, 69.39349216389064, 69.20373884320755, 69.02804132405652, 68.7890926980111, 68.73286949188278, 68.59231147656195, 68.38850235434676, 68.17063743059948, 68.00196781221449, 67.86140979689367, 67.72085178157285, 67.55218216318785, 67.34837304097266, 67.13050811722539, 66.94778269730831, 66.74397357509312, 66.52610865134585, 66.37852273525898, 66.1255183076815, 65.85845807857194, 65.528146742568, 65.24703071192634, 65.04322158971115, 64.65668704757888, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.987209220605806, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 48, 76, 93, 121, 147, 185, 227, 263, 301, 329, 378, 427, 470, 515, 555, 588, 614, 644, 688, 717, 750, 790, 814, 843, 862, 881, 896, 913, 933, 942, 955, 970, 989, 1005, 1018, 1032, 1039, 1042, 1046, 1060, 1068, 1079, 1085, 1091, 1096, 1104, 1108, 1114, 1124, 1133, 1137, 1142, 1146, 1148, 1152, 1156, 1165, 1172, 1176, 1178, 1188, 1189, 1192, 1197, 1200, 1206, 1214, 1217, 1224, 1224, 1227, 1230, 1236, 1237, 1241, 1244, 1251, 1258, 1260, 1267, 1272, 1274, 1277, 1281, 1286, 1292, 1296, 1296, 1300, 1304, 1310, 1315, 1320, 1327, 1332, 1335, 1343, 1352, 1360, 1369], "harmed": [0, 1, 12, 25, 43, 67, 81, 106, 123, 141, 161, 182, 195, 223, 243, 265, 284, 312, 341, 362, 393, 418, 448, 479, 508, 541, 577, 617, 641, 679, 712, 743, 769, 800, 826, 853, 891, 921, 959, 994, 1025, 1053, 1076, 1109, 1142, 1167, 1198, 1229, 1256, 1278, 1311, 1334, 1359, 1385, 1425, 1463, 1493, 1521, 1554, 1578, 1598, 1627, 1656, 1680, 1711, 1738, 1764, 1788, 1810, 1835, 1865, 1897, 1926, 1957, 1993, 2016, 2046, 2080, 2112, 2148, 2163, 2188, 2219, 2253, 2281, 2306, 2332, 2360, 2389, 2424, 2454, 2489, 2525, 2551, 2594, 2637, 2687, 2735, 2773, 2836, 2921], "exact_score": [75.02986857825567, 75.36017991425962, 75.47965422728231, 75.50776583034647, 75.57804483800689, 75.59210063953897, 75.76077025792397, 75.88024457094666, 76.01377468550145, 76.15433270082227, 76.21055590695059, 76.40733712839975, 76.66034155597723, 76.76576006746785, 76.94145758661888, 77.06795980040762, 77.1663504111322, 77.15229460960012, 77.15932251036615, 77.3209642279851, 77.30690842645302, 77.36313163258134, 77.43341064024176, 77.38421533487947, 77.38421533487947, 77.28582472415489, 77.1663504111322, 76.99065289198117, 76.94145758661888, 76.81495537283014, 76.64628575444516, 76.5197835406564, 76.44247663222995, 76.35814182303746, 76.28786281537705, 76.18947220465247, 76.0208025862675, 75.85916086864853, 75.6131843418371, 75.39531941808981, 75.27584510506712, 75.1352870897463, 75.0509522805538, 74.86119895987069, 74.67144563918757, 74.53088762386675, 74.36924590624781, 74.17949258556469, 74.03190666947783, 73.94757186028534, 73.77890224190034, 73.64537212734557, 73.50481411202475, 73.35020029517183, 73.08314006606227, 72.84419144001687, 72.6614660200998, 72.52793590554502, 72.34521048562794, 72.20465247030712, 72.07815025651838, 71.9446201419636, 71.74783892051444, 71.60025300442759, 71.41752758451051, 71.24885796612553, 71.10829995080469, 70.99585353854803, 70.86232342399326, 70.7358212102045, 70.52498418722328, 70.32117506500808, 70.13844964509101, 69.96275212593999, 69.71677559912854, 69.58324548457377, 69.39349216389064, 69.20373884320755, 69.02804132405652, 68.7890926980111, 68.73286949188278, 68.59231147656195, 68.38850235434676, 68.17063743059948, 68.00196781221449, 67.86140979689367, 67.72085178157285, 67.55218216318785, 67.34837304097266, 67.13050811722539, 66.94778269730831, 66.74397357509312, 66.52610865134585, 66.37852273525898, 66.1255183076815, 65.85845807857194, 65.528146742568, 65.24703071192634, 65.04322158971115, 64.65668704757888, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 72.7807536336714, "threshold_area100": 72.78115117014549, "threshold_area25": 76.4670742849111}, "test": {"score": [66.95119530385956, 67.41627201315826, 67.73955704279273, 67.90119955760996, 67.99761789978163, 68.06284207242719, 68.15075291381902, 68.20463375209142, 68.21597708646456, 68.21597708646456, 68.23866375521084, 68.24433542239741, 68.22448458724442, 68.16493208178544, 68.0770212403936, 67.93239372713609, 67.83597538496441, 67.75090037716586, 67.58358619516207, 67.41627201315826, 67.30567450302016, 67.11850948586337, 66.92567280152001, 66.7782094546692, 66.63358194141168, 66.36417775004962, 66.10328105946743, 65.87357853841137, 65.63820435016872, 65.39148682755295, 65.17596347446332, 64.97178345574682, 64.81864844170944, 64.58327425346681, 64.38193006834358, 64.22595922071292, 63.99909253325015, 63.79774834812692, 63.58789666222386, 63.36953747554094, 63.25610413180954, 63.10013328417888, 62.92714743498852, 62.816549924850406, 62.67475824518617, 62.5017723959958, 62.283413209312876, 62.06505402262995, 61.90624734140601, 61.727589825029064, 61.55743980943198, 61.424155630547595, 61.27102061651022, 61.14057227121912, 60.98176558999518, 60.84280974392422, 60.664152227547284, 60.479823043983785, 60.33519553072626, 60.13385134560304, 59.94952216203953, 59.77086464566259, 59.631908799591635, 59.40220627853557, 59.19802625981907, 59.025040410628705, 58.854890395031624, 58.67623287865468, 58.56563536851658, 58.40399285369935, 58.256529506848544, 58.09205115843802, 57.97011031392678, 57.79145279754984, 57.60712361398633, 57.48518276947509, 57.27533108357201, 57.1108527351615, 56.93786688597113, 56.76488103678076, 56.63159685789638, 56.464282675892576, 56.27144599154922, 56.126818478291696, 55.894280123642346, 55.74681677679154, 55.57099509400788, 55.412188412783934, 55.15129172220175, 54.92726086833224, 54.71457334883589, 54.46218415903354, 54.18710830048492, 53.883674106003454, 53.512179905283155, 53.14635737174942, 52.79187817258884, 52.318293962510275, 51.844709752431726, 51.18963219238295, 50.00708958398321], "calls": [0.0, 4.168675382128577, 6.031818052916655, 7.619884865156113, 8.663471627484899, 9.99631341632873, 11.130646853642629, 12.228114454243826, 13.481552902475682, 14.715140515554548, 15.826787284122169, 16.95828488784278, 17.993364149391713, 19.224115928877296, 20.182627683407535, 21.404871962113265, 22.578907069733148, 23.520403822703685, 24.78518560530868, 25.820264866857613, 26.838329126846837, 27.89325922354876, 29.0389359952358, 30.164761931769846, 31.36999120891586, 32.52417548138275, 33.6868672546295, 34.81269319116354, 35.96971329722372, 36.96792672205995, 37.784646796925955, 38.86793522956073, 39.81226781612455, 40.72540623316224, 41.56764881036781, 42.37869721804724, 43.39392564444318, 44.338258231007, 45.197515809772284, 46.096475058843545, 46.88483679777671, 47.60513853047103, 48.424694438930324, 49.119473669285085, 49.87096957150555, 50.73306298386411, 51.75396307744662, 52.627399824178326, 53.475314068570455, 54.33740748092902, 55.14845588860846, 55.92547429316848, 56.63726852508295, 57.298017752318295, 57.992796982673056, 58.718770382553956, 59.4901171199274, 60.20474718543516, 60.893854748603346, 61.68788815472308, 62.411025721010695, 63.13699912089159, 63.94804752857103, 64.76476760343704, 65.58999517908289, 66.32447607974365, 67.23761449678133, 68.02030456852792, 68.58747128718487, 69.37866885971131, 70.13867226271162, 70.94688483679778, 71.5934548960667, 72.35062246547372, 73.2467458809517, 73.95854011286617, 74.87451436349714, 75.60332359697132, 76.27541615857982, 77.10631540141225, 77.78975129739388, 78.63199387459943, 79.38632561041318, 80.12080651107392, 81.07931826560417, 81.87902333891047, 82.73828091767575, 83.54082182457533, 84.58157275331084, 85.5173978390948, 86.42486458894592, 87.22456966225222, 88.38726143549897, 89.65487905169725, 90.83742166009698, 92.14190511300797, 93.38967189405327, 94.7593795196098, 96.09222130845362, 97.84476646910359, 100.0], "rescued": [0, 182, 320, 416, 491, 573, 656, 728, 807, 869, 945, 1014, 1074, 1131, 1183, 1241, 1303, 1345, 1402, 1438, 1486, 1517, 1561, 1614, 1655, 1687, 1716, 1744, 1770, 1791, 1812, 1839, 1868, 1886, 1900, 1913, 1933, 1951, 1967, 1981, 1997, 2004, 2018, 2034, 2044, 2059, 2076, 2086, 2100, 2108, 2116, 2136, 2146, 2158, 2166, 2173, 2185, 2195, 2202, 2212, 2217, 2225, 2235, 2241, 2245, 2250, 2265, 2278, 2285, 2294, 2301, 2311, 2315, 2322, 2331, 2346, 2354, 2364, 2373, 2382, 2388, 2397, 2402, 2409, 2417, 2425, 2435, 2448, 2456, 2463, 2480, 2484, 2491, 2500, 2511, 2527, 2544, 2560, 2578, 2601, 2627], "harmed": [0, 18, 42, 81, 122, 181, 233, 286, 361, 423, 491, 558, 625, 703, 786, 895, 991, 1063, 1179, 1274, 1361, 1458, 1570, 1675, 1767, 1894, 2015, 2124, 2233, 2341, 2438, 2537, 2620, 2721, 2806, 2874, 2974, 3063, 3153, 3244, 3300, 3362, 3437, 3492, 3552, 3628, 3722, 3809, 3879, 3950, 4018, 4085, 4149, 4207, 4271, 4327, 4402, 4477, 4535, 4616, 4686, 4757, 4816, 4903, 4979, 5045, 5120, 5196, 5242, 5308, 5367, 5435, 5482, 5552, 5626, 5684, 5766, 5834, 5904, 5974, 6027, 6095, 6168, 6226, 6316, 6376, 6448, 6517, 6617, 6703, 6795, 6888, 6992, 7108, 7250, 7395, 7537, 7720, 7905, 8159, 8602], "exact_score": [66.95119530385956, 67.05328531321781, 67.17239032413578, 67.26313699912089, 67.39358534441199, 67.58358619516207, 67.73672120919944, 67.85299038652411, 67.93806539432265, 68.01463290134134, 68.06567790602047, 68.13940957944588, 68.21314125287128, 68.17627541615859, 68.210305419278, 68.20179791849814, 68.23866375521084, 68.23299208802428, 68.22448458724442, 68.18761875053173, 68.0997079091399, 68.00045373337493, 67.8926920568301, 67.81328871621814, 67.69134787170688, 67.56940702719564, 67.39358534441199, 67.28015200068059, 67.11000198508351, 66.93985196948643, 66.80656779060205, 66.7044777812438, 66.50596942971386, 66.25925190709809, 66.0153702180756, 65.84238436888523, 65.63536851657544, 65.38865099395967, 65.13059013697075, 64.96044012137368, 64.79312593936987, 64.54073674956753, 64.29401922695176, 64.07566004026883, 63.87148002155233, 63.64461333408955, 63.39222414428721, 63.22490996228341, 63.023565777160194, 62.83072909281683, 62.65774324362646, 62.43371238975697, 62.21251736948076, 62.01684485154411, 61.80982899923433, 61.59714147973797, 61.40146896180132, 61.20863227745795, 60.98176558999518, 60.786093072058534, 60.54788305022261, 60.30967302838669, 60.04310467061793, 59.796387148002154, 59.61772963162522, 59.325638771516886, 59.095936250460824, 58.888920398151036, 58.68757621302782, 58.48056036071802, 58.27638034200153, 58.08354365765817, 57.876527805348374, 57.66100445225874, 57.47667526869523, 57.2413010804526, 56.99174772424354, 56.793239372713614, 56.58055185321725, 56.35368516575448, 56.14950514703797, 55.90845929160877, 55.72413010804526, 55.51995008932875, 55.29308340186597, 55.06905254799648, 54.799648356634435, 54.53591583245895, 54.26367580750362, 54.05949578878712, 53.773076595865355, 53.45829906701074, 53.208745710801686, 52.92516235147321, 52.58202648668576, 52.22754728752517, 51.884411422737706, 51.50440972123755, 51.11873635255083, 50.574256302640165, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.808982985866166, "threshold_area100": 61.48003573150327, "threshold_area25": 67.63945211694978}, "selected_budget": 22, "selected": {"val": {"score": 77.43341064024176, "calls": 21.997329397708903, "rescued": 790, "harmed": 448}, "test": {"score": 66.92567280152001, "calls": 29.0389359952358, "rescued": 1561, "harmed": 1570}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.0763542652130127, "operator": "ge"}, {"budget": 0.02, "threshold": 0.003633737564086914, "operator": "ge"}, {"budget": 0.03, "threshold": -0.049897193908691406, "operator": "ge"}, {"budget": 0.04, "threshold": -0.26650968194007874, "operator": "ge"}, {"budget": 0.05, "threshold": -0.5990999937057495, "operator": "ge"}, {"budget": 0.06, "threshold": -0.8345023393630981, "operator": "ge"}, {"budget": 0.07, "threshold": -1.0436244010925293, "operator": "ge"}, {"budget": 0.08, "threshold": -1.2323840856552124, "operator": "ge"}, {"budget": 0.09, "threshold": -1.4131433963775635, "operator": "ge"}, {"budget": 0.1, "threshold": -1.6083934307098389, "operator": "ge"}, {"budget": 0.11, "threshold": -1.7804291248321533, "operator": "ge"}, {"budget": 0.12, "threshold": -1.941501259803772, "operator": "ge"}, {"budget": 0.13, "threshold": -2.1020569801330566, "operator": "ge"}, {"budget": 0.14, "threshold": -2.2606441974639893, "operator": "ge"}, {"budget": 0.15, "threshold": -2.413522720336914, "operator": "ge"}, {"budget": 0.16, "threshold": -2.564608335494995, "operator": "ge"}, {"budget": 0.17, "threshold": -2.696331024169922, "operator": "ge"}, {"budget": 0.18, "threshold": -2.8192176818847656, "operator": "ge"}, {"budget": 0.19, "threshold": -2.949359893798828, "operator": "ge"}, {"budget": 0.2, "threshold": -3.067861557006836, "operator": "ge"}, {"budget": 0.21, "threshold": -3.1964218616485596, "operator": "ge"}, {"budget": 0.22, "threshold": -3.3214797973632812, "operator": "ge"}, {"budget": 0.23, "threshold": -3.453948497772217, "operator": "ge"}, {"budget": 0.24, "threshold": -3.5595266819000244, "operator": "ge"}, {"budget": 0.25, "threshold": -3.699683427810669, "operator": "ge"}, {"budget": 0.26, "threshold": -3.839150905609131, "operator": "ge"}, {"budget": 0.27, "threshold": -3.9591264724731445, "operator": "ge"}, {"budget": 0.28, "threshold": -4.076188564300537, "operator": "ge"}, {"budget": 0.29, "threshold": -4.199010372161865, "operator": "ge"}, {"budget": 0.3, "threshold": -4.330788612365723, "operator": "ge"}, {"budget": 0.31, "threshold": -4.430103302001953, "operator": "ge"}, {"budget": 0.32, "threshold": -4.523832321166992, "operator": "ge"}, {"budget": 0.33, "threshold": -4.609530448913574, "operator": "ge"}, {"budget": 0.34, "threshold": -4.700711250305176, "operator": "ge"}, {"budget": 0.35, "threshold": -4.788448810577393, "operator": "ge"}, {"budget": 0.36, "threshold": -4.869828224182129, "operator": "ge"}, {"budget": 0.37, "threshold": -4.946594715118408, "operator": "ge"}, {"budget": 0.38, "threshold": -5.014568328857422, "operator": "ge"}, {"budget": 0.39, "threshold": -5.076678276062012, "operator": "ge"}, {"budget": 0.4, "threshold": -5.136124610900879, "operator": "ge"}, {"budget": 0.41, "threshold": -5.191631317138672, "operator": "ge"}, {"budget": 0.42, "threshold": -5.242198467254639, "operator": "ge"}, {"budget": 0.43, "threshold": -5.304067611694336, "operator": "ge"}, {"budget": 0.44, "threshold": -5.350436210632324, "operator": "ge"}, {"budget": 0.45, "threshold": -5.39475154876709, "operator": "ge"}, {"budget": 0.46, "threshold": -5.444183349609375, "operator": "ge"}, {"budget": 0.47, "threshold": -5.495214462280273, "operator": "ge"}, {"budget": 0.48, "threshold": -5.528594493865967, "operator": "ge"}, {"budget": 0.49, "threshold": -5.5701680183410645, "operator": "ge"}, {"budget": 0.5, "threshold": -5.610035419464111, "operator": "ge"}, {"budget": 0.51, "threshold": -5.648606777191162, "operator": "ge"}, {"budget": 0.52, "threshold": -5.687045574188232, "operator": "ge"}, {"budget": 0.53, "threshold": -5.721292972564697, "operator": "ge"}, {"budget": 0.54, "threshold": -5.764960765838623, "operator": "ge"}, {"budget": 0.55, "threshold": -5.81017541885376, "operator": "ge"}, {"budget": 0.56, "threshold": -5.846894264221191, "operator": "ge"}, {"budget": 0.57, "threshold": -5.88607120513916, "operator": "ge"}, {"budget": 0.58, "threshold": -5.918486595153809, "operator": "ge"}, {"budget": 0.59, "threshold": -5.957854270935059, "operator": "ge"}, {"budget": 0.6, "threshold": -5.989086151123047, "operator": "ge"}, {"budget": 0.61, "threshold": -6.020326137542725, "operator": "ge"}, {"budget": 0.62, "threshold": -6.054653644561768, "operator": "ge"}, {"budget": 0.63, "threshold": -6.085872173309326, "operator": "ge"}, {"budget": 0.64, "threshold": -6.122371196746826, "operator": "ge"}, {"budget": 0.65, "threshold": -6.153076171875, "operator": "ge"}, {"budget": 0.66, "threshold": -6.187953948974609, "operator": "ge"}, {"budget": 0.67, "threshold": -6.224445343017578, "operator": "ge"}, {"budget": 0.68, "threshold": -6.253366947174072, "operator": "ge"}, {"budget": 0.69, "threshold": -6.285972595214844, "operator": "ge"}, {"budget": 0.7, "threshold": -6.318670272827148, "operator": "ge"}, {"budget": 0.71, "threshold": -6.353808403015137, "operator": "ge"}, {"budget": 0.72, "threshold": -6.384437561035156, "operator": "ge"}, {"budget": 0.73, "threshold": -6.422285079956055, "operator": "ge"}, {"budget": 0.74, "threshold": -6.45474910736084, "operator": "ge"}, {"budget": 0.75, "threshold": -6.494692802429199, "operator": "ge"}, {"budget": 0.76, "threshold": -6.5261688232421875, "operator": "gt"}, {"budget": 0.77, "threshold": -6.558401107788086, "operator": "ge"}, {"budget": 0.78, "threshold": -6.591778755187988, "operator": "ge"}, {"budget": 0.79, "threshold": -6.6378655433654785, "operator": "ge"}, {"budget": 0.8, "threshold": -6.679819107055664, "operator": "ge"}, {"budget": 0.81, "threshold": -6.725316047668457, "operator": "ge"}, {"budget": 0.82, "threshold": -6.762538433074951, "operator": "ge"}, {"budget": 0.83, "threshold": -6.809793472290039, "operator": "ge"}, {"budget": 0.84, "threshold": -6.853604316711426, "operator": "ge"}, {"budget": 0.85, "threshold": -6.910438060760498, "operator": "ge"}, {"budget": 0.86, "threshold": -6.961002349853516, "operator": "ge"}, {"budget": 0.87, "threshold": -7.013422012329102, "operator": "gt"}, {"budget": 0.88, "threshold": -7.072228908538818, "operator": "ge"}, {"budget": 0.89, "threshold": -7.137127876281738, "operator": "ge"}, {"budget": 0.9, "threshold": -7.195544719696045, "operator": "ge"}, {"budget": 0.91, "threshold": -7.265119552612305, "operator": "ge"}, {"budget": 0.92, "threshold": -7.3482160568237305, "operator": "ge"}, {"budget": 0.93, "threshold": -7.432100296020508, "operator": "ge"}, {"budget": 0.94, "threshold": -7.5394134521484375, "operator": "ge"}, {"budget": 0.95, "threshold": -7.641193866729736, "operator": "ge"}, {"budget": 0.96, "threshold": -7.774601936340332, "operator": "ge"}, {"budget": 0.97, "threshold": -7.939242839813232, "operator": "ge"}, {"budget": 0.98, "threshold": -8.16510009765625, "operator": "ge"}, {"budget": 0.99, "threshold": -8.53662109375, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.24070560123691, 75.47965422728231, 75.59912854030502, 75.59912854030502, 75.54290533417668, 75.60615644107105, 75.74671445639187, 75.99269098320332, 76.06296999086372, 76.12622109775809, 76.20352800618456, 76.2456954107808, 76.29489071614309, 76.3651697238035, 76.47058823529413, 76.51275563989036, 76.55492304448661, 76.56195094525266, 76.5197835406564, 76.42842083069787, 76.40733712839975, 76.38625342610162, 76.30894651767517, 76.26677911307893, 76.19650010541851, 76.06999789162977, 75.92943987630895, 75.69049125026355, 75.58507273877294, 75.48668212804836, 75.30395670813128, 75.19151029587462, 74.95256166982922, 74.7768641506782, 74.62225033382529, 74.4887202192705, 74.404385410078, 74.19354838709677, 74.02487876871179, 73.95459976105137, 73.78593014266639, 73.64537212734557, 73.46967460819454, 73.23775388291519, 73.09016796682832, 72.95663785227353, 72.83716353925082, 72.6614660200998, 72.5068522032469, 72.39440579099023, 72.27493147796753, 72.183568768009, 72.0289549511561, 71.80406212664278, 71.62836460749175, 71.49483449293696, 71.32616487455196, 71.16452315693303, 71.00288143931408, 70.82718392016305, 70.69365380560826, 70.50390048492515, 70.3422587673062, 70.13844964509101, 69.9978916297702, 69.80813830908707, 69.6605523930002, 69.56216178227564, 69.41457586618877, 69.40754796542274, 69.28104575163398, 69.13345983554713, 68.943706514864, 68.80314849954318, 68.66961838498841, 68.58528357579591, 68.44472556047508, 68.24794433902593, 68.1214421252372, 67.89654930072388, 67.66462857544452, 67.53109846088974, 67.36945674327079, 67.2499824302481, 67.08834071262913, 66.95481059807436, 66.83533628505167, 66.6877503689648, 66.45582964368543, 66.2801321245344, 66.11146250614941, 65.90062548316817, 65.6827605594209, 65.41570033031134, 65.15566800196781, 64.93780307822054, 64.75507765830346, 64.50910113149202, 64.25609670391455, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99269098320332, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.99135568205777, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 41, 78, 97, 112, 132, 158, 192, 239, 273, 309, 344, 381, 417, 456, 495, 531, 566, 603, 633, 665, 698, 734, 763, 795, 830, 853, 877, 895, 920, 946, 962, 981, 994, 1010, 1026, 1040, 1056, 1069, 1081, 1096, 1103, 1113, 1120, 1126, 1136, 1140, 1145, 1150, 1157, 1162, 1169, 1174, 1179, 1183, 1185, 1190, 1192, 1196, 1199, 1202, 1208, 1213, 1217, 1218, 1222, 1226, 1230, 1231, 1236, 1249, 1252, 1253, 1254, 1257, 1260, 1266, 1272, 1273, 1275, 1278, 1280, 1282, 1283, 1286, 1291, 1298, 1301, 1303, 1307, 1311, 1315, 1320, 1324, 1328, 1332, 1335, 1344, 1350, 1358, 1369], "harmed": [0, 11, 14, 16, 31, 59, 76, 90, 102, 126, 153, 177, 208, 237, 266, 290, 320, 349, 385, 421, 466, 502, 541, 581, 619, 664, 705, 749, 801, 841, 881, 923, 958, 1005, 1046, 1084, 1117, 1145, 1188, 1224, 1249, 1280, 1310, 1342, 1381, 1412, 1435, 1457, 1487, 1516, 1537, 1561, 1579, 1606, 1642, 1669, 1693, 1719, 1746, 1772, 1800, 1825, 1857, 1884, 1914, 1938, 1969, 1994, 2009, 2035, 2049, 2070, 2092, 2120, 2143, 2165, 2183, 2209, 2238, 2258, 2293, 2328, 2349, 2373, 2393, 2421, 2447, 2467, 2490, 2527, 2556, 2584, 2619, 2654, 2696, 2737, 2771, 2806, 2847, 2891, 2921], "exact_score": [75.02986857825567, 75.24070560123691, 75.47965422728231, 75.59912854030502, 75.59912854030502, 75.54290533417668, 75.60615644107105, 75.74671445639187, 75.99269098320332, 76.06296999086372, 76.12622109775809, 76.20352800618456, 76.2456954107808, 76.29489071614309, 76.3651697238035, 76.47058823529413, 76.51275563989036, 76.55492304448661, 76.56195094525266, 76.5197835406564, 76.42842083069787, 76.40733712839975, 76.38625342610162, 76.30894651767517, 76.26677911307893, 76.19650010541851, 76.06999789162977, 75.92943987630895, 75.69049125026355, 75.58507273877294, 75.48668212804836, 75.30395670813128, 75.19151029587462, 74.95256166982922, 74.7768641506782, 74.62225033382529, 74.4887202192705, 74.404385410078, 74.19354838709677, 74.02487876871179, 73.95459976105137, 73.78593014266639, 73.64537212734557, 73.46967460819454, 73.23775388291519, 73.09016796682832, 72.95663785227353, 72.83716353925082, 72.6614660200998, 72.5068522032469, 72.39440579099023, 72.27493147796753, 72.183568768009, 72.0289549511561, 71.80406212664278, 71.62836460749175, 71.49483449293696, 71.32616487455196, 71.16452315693303, 71.00288143931408, 70.82718392016305, 70.69365380560826, 70.50390048492515, 70.3422587673062, 70.13844964509101, 69.9978916297702, 69.80813830908707, 69.6605523930002, 69.56216178227564, 69.41457586618877, 69.40754796542274, 69.28104575163398, 69.13345983554713, 68.943706514864, 68.80314849954318, 68.66961838498841, 68.58528357579591, 68.44472556047508, 68.24794433902593, 68.1214421252372, 67.89654930072388, 67.66462857544452, 67.53109846088974, 67.36945674327079, 67.2499824302481, 67.08834071262913, 66.95481059807436, 66.83533628505167, 66.6877503689648, 66.45582964368543, 66.2801321245344, 66.11146250614941, 65.90062548316817, 65.6827605594209, 65.41570033031134, 65.15566800196781, 64.93780307822054, 64.75507765830346, 64.50910113149202, 64.25609670391455, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 71.85749397841175, "threshold_area100": 71.85789584651067, "threshold_area25": 76.03626396795278}, "test": {"score": [66.95119530385956, 67.33403283895301, 67.7253778748263, 67.87851288886368, 67.99194623259507, 68.07134957320704, 68.16493208178544, 68.18761875053173, 68.10821540991975, 68.0968720755466, 68.05433457164733, 68.0259762357145, 67.9692595638488, 67.84164705215098, 67.69985537248674, 67.5382128576695, 67.353883674106, 67.15821115616936, 66.9625386382327, 66.73283611717665, 66.56835776876613, 66.36134191645634, 66.13163939540027, 65.93596687746363, 65.76298102827326, 65.50775600487762, 65.23835181351558, 64.94342511981397, 64.71372259875791, 64.42446757224286, 64.08983920823526, 63.80625584890678, 63.51416498879846, 63.27028329977596, 62.90446076624223, 62.6690865779996, 62.354309049144995, 62.12460652808893, 61.8892323398463, 61.682216487536515, 61.50072313756628, 61.30505061962964, 61.117885602472846, 60.90519808297648, 60.689674729886846, 60.519524714289766, 60.32101636275983, 60.162209681535884, 60.04594050421122, 59.83325298471486, 59.668774636304335, 59.495788787113966, 59.31429543714375, 59.12713041998695, 58.95698040438987, 58.76981538723307, 58.62235204038227, 58.474888693531454, 58.35294784902022, 58.194141167796275, 58.103394492811155, 57.998468649859625, 57.85667697019539, 57.68369112100502, 57.496526103848225, 57.38592859371012, 57.26115191560559, 57.10518106797493, 56.98324022346368, 56.85279187817258, 56.71383603210164, 56.62025352352323, 56.52099934775827, 56.35368516575448, 56.228908487649946, 56.04174347049315, 55.891444290049066, 55.77517511272438, 55.65039843461985, 55.468905084649634, 55.31860590420554, 55.165470890168166, 55.05203754643677, 54.85920086209342, 54.68054334571647, 54.47352749340668, 54.3203924793693, 54.16158579814536, 53.948898278649004, 53.747554093525785, 53.60009074667498, 53.46397073419732, 53.208745710801686, 52.99605819130533, 52.71247483197685, 52.49127981170065, 52.23889062189831, 51.884411422737706, 51.57814139466296, 51.13575135411054, 50.00708958398321], "calls": [0.0, 1.4774693021013525, 5.268978816323059, 7.946005728383858, 9.534072540623315, 10.912287666959703, 12.30184612766923, 13.535433740748093, 14.749170518673965, 15.96007146300655, 17.278734083883958, 18.413067521197856, 19.558744292884896, 20.80651107393018, 21.943680344837365, 23.04681961262513, 24.23503388821144, 25.20772481070811, 26.149221563678644, 27.22400249553356, 28.122961744604822, 29.064458497575362, 30.16192609817656, 31.23670703003148, 32.008053767404924, 33.29268638516292, 34.4667214927828, 35.589711595723564, 36.67300002835834, 37.951960978929755, 39.29331026855344, 40.32555369650909, 41.39749879477072, 42.41839888835323, 43.65765816861867, 44.74378243484672, 45.84975753622778, 46.89334429855656, 47.78379604684797, 48.73096446700508, 49.66678955278904, 50.49768879562148, 51.32575220486062, 52.270084791424445, 53.03292402801804, 53.79009159742506, 54.62099084025749, 55.55398009244817, 56.10413180954542, 56.91518021722485, 57.72622862490429, 58.540112866177004, 59.266086266057904, 59.949522162039536, 60.92504891812949, 61.81833650001418, 62.57550406942121, 63.42908998099992, 64.05864503870913, 64.77894677140345, 65.39148682755295, 66.05507188838159, 66.82074695856846, 67.53537702407623, 68.3010520942631, 68.89090548166634, 69.68210305419278, 70.53852479936478, 71.148229021921, 71.90256075773473, 72.69375833026118, 73.40271672858236, 74.10033179253041, 74.92555936817628, 75.61183109775118, 76.4455661741769, 77.06661373110626, 77.65079545132292, 78.32005217933812, 79.30125060261463, 80.02722400249553, 80.84394407736154, 81.60111164676856, 82.53977256614581, 83.34231347304541, 84.24978022289652, 85.14023197118793, 85.93142954371437, 86.81053795763265, 87.72084054107705, 88.47233644329751, 89.28338485097694, 90.2759266086266, 91.0954825170859, 92.07100927317585, 92.91325185038141, 93.87743527209823, 94.91818620083373, 96.00714630065508, 97.35700309105862, 100.0], "rescued": [0, 163, 322, 402, 493, 579, 673, 749, 800, 867, 941, 1003, 1070, 1123, 1172, 1229, 1286, 1324, 1365, 1412, 1447, 1484, 1527, 1568, 1603, 1659, 1702, 1738, 1777, 1809, 1840, 1862, 1887, 1908, 1937, 1968, 1994, 2021, 2034, 2050, 2068, 2081, 2095, 2106, 2114, 2123, 2135, 2154, 2158, 2164, 2173, 2179, 2186, 2188, 2200, 2214, 2220, 2232, 2239, 2243, 2249, 2258, 2266, 2270, 2272, 2277, 2285, 2294, 2299, 2309, 2316, 2327, 2338, 2344, 2349, 2356, 2361, 2367, 2372, 2378, 2388, 2397, 2402, 2409, 2415, 2425, 2435, 2443, 2453, 2461, 2472, 2487, 2495, 2508, 2515, 2525, 2538, 2549, 2564, 2581, 2627], "harmed": [0, 28, 49, 75, 126, 184, 245, 313, 392, 463, 552, 624, 711, 809, 908, 1022, 1144, 1251, 1361, 1489, 1582, 1692, 1816, 1926, 2022, 2168, 2306, 2446, 2566, 2700, 2849, 2971, 3099, 3206, 3364, 3478, 3615, 3723, 3819, 3908, 3990, 4072, 4152, 4238, 4322, 4391, 4473, 4548, 4593, 4674, 4741, 4808, 4879, 4947, 5019, 5099, 5157, 5221, 5271, 5331, 5369, 5415, 5473, 5538, 5606, 5650, 5702, 5766, 5814, 5870, 5926, 5970, 6016, 6081, 6130, 6203, 6261, 6308, 6357, 6427, 6490, 6553, 6598, 6673, 6742, 6825, 6889, 6953, 7038, 7117, 7180, 7243, 7341, 7429, 7536, 7624, 7726, 7862, 7985, 8158, 8602], "exact_score": [66.95119530385956, 67.2234353288149, 67.43045118112468, 67.5382128576695, 67.61761619828148, 67.70552703967331, 67.75373621075916, 67.81045288262484, 67.88985622323682, 67.97493123103536, 68.0032895669682, 68.07418540680032, 68.12523041147945, 68.1734395825653, 68.15075291381902, 68.08836457476674, 68.10537957632647, 68.03731957008763, 68.05717040524061, 68.04582707086749, 67.93522956072937, 67.81328871621814, 67.69134787170688, 67.5410486912628, 67.39358534441199, 67.20074866006863, 66.98522530697898, 66.76686612029606, 66.58253693673255, 66.36134191645634, 66.18268440007941, 65.98701188214275, 65.77148852905312, 65.57298017752318, 65.34894932365368, 65.0852167994782, 64.844170944049, 64.6768567620452, 64.415960071463, 64.16357088166066, 63.87998752233218, 63.607747497376856, 63.358194141167786, 63.11147661855202, 62.85057992796983, 62.56983240223464, 62.328786546805425, 62.096248192156075, 61.84953066954031, 61.62266398207753, 61.432663131327445, 61.19445310949153, 60.956243087655615, 60.70668973144656, 60.47131554320392, 60.261463857300846, 60.07429884014406, 59.819073816748435, 59.60071463006551, 59.37384794260272, 59.11295125202053, 58.94563707001673, 58.727277883333805, 58.579814536483, 58.355783682613506, 58.160111164676856, 58.00130448345291, 57.81413946629612, 57.564586110087056, 57.38025692652356, 57.18742024218019, 57.01443439298982, 56.83010520942631, 56.68547769616878, 56.52950684853813, 56.34234183138134, 56.16084848141111, 55.9027876244222, 55.72696594163854, 55.52562175651532, 55.31860590420554, 55.14845588860845, 54.969798372231516, 54.7684541871083, 54.51322916371267, 54.3402433145223, 54.15591413095879, 53.909196608343024, 53.69650908884667, 53.52068740606302, 53.26829821626067, 53.02441652723818, 52.73232566712986, 52.4629214757678, 52.19068145081247, 51.87023225477129, 51.575305561069676, 51.252020531435214, 50.92306383461418, 50.52037546436775, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.144545759575976, "threshold_area100": 60.31674843320194, "threshold_area25": 67.29234608513173}, "selected_budget": 18, "selected": {"val": {"score": 76.56195094525266, "calls": 17.99845386183147, "rescued": 603, "harmed": 385}, "test": {"score": 66.9625386382327, "calls": 26.149221563678644, "rescued": 1365, "harmed": 1361}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 75.59210063953897, "calls": 6.999789162977018}, "test": {"score": 67.87851288886368, "calls": 13.972152114113944}}, "old_normalized_interior": {"val": {"score": 76.21055590695059, "calls": 31.99803218778551}, "test": {"score": 67.92388622635623, "calls": 33.17074554065167}}, "old_normalized_endpoints": {"val": {"score": 76.21055590695059, "calls": 31.99803218778551}, "test": {"score": 67.92388622635623, "calls": 33.17074554065167}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.24070560123691, 75.47965422728231, 75.59210063953897, 75.58507273877294, 75.54993323494273, 75.56398903647481, 75.59210063953897, 75.60615644107105, 75.6131843418371, 75.63426804413523, 75.62724014336918, 75.69049125026355, 75.71157495256166, 75.73968655562584, 75.73265865485979, 75.78888186098813, 75.83104926558437, 75.85916086864853, 75.88727247171269, 75.9224119755429, 75.96457938013916, 76.0067467847354, 75.97863518167124, 76.03485838779956, 76.07702579239582, 76.07702579239582, 76.03485838779956, 76.05594209009769, 76.08405369316186, 76.11919319699206, 76.18244430388643, 76.21055590695059, 76.20352800618456, 76.18947220465247, 76.11216529622602, 76.1613606015883, 76.0910815939279, 76.03485838779956, 76.0418862885656, 75.99269098320332, 75.90835617401083, 75.88024457094666, 75.78888186098813, 75.73968655562584, 75.64832384566729, 75.64832384566729, 75.57804483800689, 75.52182163187857, 75.47262632651626, 75.4445147234521, 75.43748682268607, 75.35315201349357, 75.26178930353502, 75.17745449434254, 75.06500808208588, 75.00175697519151, 74.94553376906318, 74.8822826621688, 74.8120036545084, 74.72064094454987, 74.67847353995361, 74.59413873076113, 74.48169231850447, 74.31302270011948, 74.23571579169302, 74.07407407407408, 73.93351605875326, 73.7367348373041, 73.66645582964368, 73.65942792887765, 73.53995361585494, 73.3431723944058, 73.19558647831893, 72.97069365380561, 72.80202403542062, 72.59118701243938, 72.34521048562794, 72.07815025651838, 71.83920163047299, 71.55808559983133, 71.2980532714878, 70.96071403471782, 70.72176540867244, 70.32117506500808, 70.02600323283436, 69.72380349989459, 69.48485487384917, 69.16157143861129, 68.90856701103381, 68.71178578958465, 68.21983273596177, 67.82627029306346, 67.33431723944058, 66.84939208658373, 66.49799704828168, 66.1606578115117, 65.70384426171903, 65.359477124183, 64.83238456672991, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.991215124042448, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.98945814885094, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.99086372900415, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.99135568205777, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99002038091223, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.24070560123691, 75.47965422728231, 75.59210063953897, 75.58507273877294, 75.54993323494273, 75.56398903647481, 75.59210063953897, 75.60615644107105, 75.6131843418371, 75.63426804413523, 75.62724014336918, 75.69049125026355, 75.71157495256166, 75.73968655562584, 75.73265865485979, 75.78888186098813, 75.83104926558437, 75.85916086864853, 75.88727247171269, 75.9224119755429, 75.96457938013916, 76.0067467847354, 75.97863518167124, 76.03485838779956, 76.07702579239582, 76.07702579239582, 76.03485838779956, 76.05594209009769, 76.08405369316186, 76.11919319699206, 76.18244430388643, 76.21055590695059, 76.20352800618456, 76.18947220465247, 76.11216529622602, 76.1613606015883, 76.0910815939279, 76.03485838779956, 76.0418862885656, 75.99269098320332, 75.90835617401083, 75.88024457094666, 75.78888186098813, 75.73968655562584, 75.64832384566729, 75.64832384566729, 75.57804483800689, 75.52182163187857, 75.47262632651626, 75.4445147234521, 75.43748682268607, 75.35315201349357, 75.26178930353502, 75.17745449434254, 75.06500808208588, 75.00175697519151, 74.94553376906318, 74.8822826621688, 74.8120036545084, 74.72064094454987, 74.67847353995361, 74.59413873076113, 74.48169231850447, 74.31302270011948, 74.23571579169302, 74.07407407407408, 73.93351605875326, 73.7367348373041, 73.66645582964368, 73.65240002811161, 73.53995361585494, 73.3431723944058, 73.19558647831893, 72.97069365380561, 72.80202403542062, 72.59118701243938, 72.34521048562794, 72.07815025651838, 71.83920163047299, 71.55808559983133, 71.2980532714878, 70.96071403471782, 70.72176540867244, 70.32117506500808, 70.02600323283436, 69.72380349989459, 69.47782697308314, 69.16157143861129, 68.90856701103381, 68.71178578958465, 68.21983273596177, 67.82627029306346, 67.33431723944058, 66.84939208658373, 66.49799704828168, 66.1606578115117, 65.70384426171903, 65.35244922341697, 64.83238456672991, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 75.67854381896129, "exact_area100": 73.7744414585868}, "test": {"score": [66.95119530385956, 67.33119700535973, 67.7253778748263, 67.8756770552704, 67.99478206618835, 68.07418540680032, 68.08269290758018, 68.10821540991975, 68.1337379122593, 68.13940957944588, 68.170603748972, 68.17627541615859, 68.170603748972, 68.16776791537872, 68.170603748972, 68.15075291381902, 68.14224541303916, 68.14508124663244, 68.170603748972, 68.16209624819216, 68.17627541615859, 68.170603748972, 68.1535887474123, 68.12806624507274, 68.11105124351303, 68.09403624195332, 68.09403624195332, 68.09403624195332, 68.03731957008763, 67.98911039900179, 67.98060289822193, 67.94373706150922, 67.92388622635623, 67.8756770552704, 67.85299038652411, 67.83030371777784, 67.7991095482517, 67.75090037716586, 67.69701953889346, 67.62895953265462, 67.54672035844936, 67.49283952017694, 67.44463034909111, 67.40492867878513, 67.34537617332614, 67.25746533193431, 67.1497036553895, 67.06746448118425, 66.95686697104614, 66.89164279840058, 66.82074695856846, 66.6846269460908, 66.58820860391911, 66.45776025862801, 66.35283441567648, 66.20537106882568, 66.05507188838159, 65.9047727079375, 65.73462269234042, 65.63253268298216, 65.38581516036639, 65.20148597680287, 65.00864929245952, 64.81297677452287, 64.55491591753396, 64.38476590193687, 64.13237671213453, 63.90267419107847, 63.60491166378357, 63.36953747554094, 63.07177494824604, 62.776848254544426, 62.484757394436095, 62.22953237104046, 61.9629640132717, 61.56878314380512, 61.23415477979752, 60.84848141111079, 60.4940022119502, 60.105493009670184, 59.69996880583047, 59.35399710744973, 59.04489124578169, 58.653546209908406, 58.21682783654255, 57.84533363582225, 57.48801860306837, 56.99174772424354, 56.59189518759039, 56.23458015483651, 55.834727618183365, 55.38666591044438, 54.969798372231516, 54.479199160593254, 53.92621160990273, 53.43844823185775, 52.82590817570825, 52.13112894535349, 51.37112554235317, 50.70186881433797, 50.00708958398321], "calls": [0.0, 1.5001559708476306, 5.271814649916343, 7.909139891671156, 9.497206703910614, 10.997362674758245, 13.090207866602388, 14.505288829651475, 15.798428948189319, 17.30992825341009, 18.554859200862094, 19.541729291325186, 20.483226044295723, 21.305617786348297, 21.980546181550068, 22.587414570513005, 23.262342965714772, 23.809658849218728, 24.51010974676006, 25.040410628704308, 25.641607350480673, 26.171908232424922, 26.753254119048293, 27.368630008791083, 28.060573405552564, 28.724158466381194, 29.32535518815756, 29.943566911493637, 30.553271134049854, 31.162975356606076, 31.6790970705839, 32.492981311856624, 33.17074554065167, 33.76910642883475, 34.43552732325667, 35.10478405127187, 35.853444119899045, 36.69568669710461, 37.56061594305646, 38.30360434449707, 39.22525026231461, 40.0448061707739, 40.838839576893626, 41.51943963928197, 42.27660720868899, 43.25213396477895, 44.040495703712104, 44.79482743952585, 45.72781669171653, 46.561551768142245, 47.324391004735844, 48.36797776706463, 49.30947452003516, 50.16873209880044, 51.186796358789664, 52.06306893911466, 53.047103195984455, 53.903524941156455, 54.697558347276185, 55.46039758386978, 56.671298528202364, 57.67234778663188, 58.588322037262856, 59.50713212148712, 60.482658877577066, 61.514902305532715, 62.61804157332048, 63.79774834812694, 64.77894677140345, 66.01253438448231, 67.23477866318804, 68.28687292629668, 69.31344468706577, 70.26912060800272, 71.19643819300684, 72.4668916427984, 73.50197090434733, 74.62212517369481, 75.75362277741542, 77.0410912287667, 78.23214133794629, 79.4373706150923, 80.41006153758897, 81.47917080225733, 82.61066840597793, 83.676941837053, 84.68366276266909, 85.9059070413748, 86.88710546465134, 87.79173638090917, 88.71621813232, 89.78532739698835, 90.84309332728355, 91.99727759975045, 93.13728270425091, 94.14683946346028, 95.30385956952047, 96.52610384822619, 97.77954229645805, 98.85999489549954, 100.0], "exact_score": [66.95119530385956, 67.22910699600148, 67.43045118112468, 67.5410486912628, 67.61761619828148, 67.70552703967331, 67.75373621075916, 67.81045288262484, 67.8926920568301, 67.97209539744208, 68.02314040212121, 68.07418540680032, 68.07418540680032, 68.0770212403936, 68.08269290758018, 68.11955874429289, 68.13940957944588, 68.14791708022572, 68.15642458100558, 68.17911124975187, 68.18478291693843, 68.16493208178544, 68.170603748972, 68.14224541303916, 68.15075291381902, 68.1535887474123, 68.15926041459886, 68.15075291381902, 68.11388707710631, 68.08552874117346, 68.08836457476674, 67.98911039900179, 67.99194623259507, 67.93806539432265, 67.86149788730397, 67.83030371777784, 67.7991095482517, 67.74239287638602, 67.65731786858747, 67.57224286078892, 67.49000368658366, 67.41627201315826, 67.36239117488586, 67.26313699912089, 67.15537532257608, 67.05044947962453, 66.92567280152001, 66.8320902929416, 66.73283611717665, 66.61089527266539, 66.47477526018774, 66.37835691801605, 66.21387856960553, 66.06925105634801, 65.8877577063778, 65.68641352125456, 65.51059183847092, 65.32909848850069, 65.15044097212376, 64.91223095028784, 64.69103593001162, 64.48685591129512, 64.27133255820549, 64.05013753792927, 63.817599183279924, 63.56237415988429, 63.36670164194766, 63.12565578651844, 62.847744094376544, 62.581175736607776, 62.33445821399201, 62.028188185917244, 61.69639565550293, 61.39863312820803, 61.078183932166866, 60.70101806425998, 60.423106372118085, 60.122508011229904, 59.78504381362902, 59.46743045118112, 59.149817088733236, 58.81802455831892, 58.460709525565036, 58.08921532484474, 57.737571959277425, 57.36607775855712, 56.93503105237785, 56.5521935172844, 56.140997646258114, 55.74398094319826, 55.31009840342569, 54.9159175339591, 54.479199160593254, 53.99710744973485, 53.50934407168987, 52.987550690525474, 52.414712304681956, 51.83903808524516, 51.223662195502364, 50.60545047216629, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 68.01888665173128, "exact_area100": 63.88991454829913}}, "old_normalized_endpoint_budget": 32}, {"seed": 2027, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.75, "validation_area100": 0.736821758740341, "training_seconds": 85.64773952995893, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.1838025111663994, "operator": "ge"}, {"budget": 0.02, "threshold": 0.07985975639102347, "operator": "ge"}, {"budget": 0.03, "threshold": 0.03101618547161909, "operator": "ge"}, {"budget": 0.04, "threshold": 0.011002634583059334, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0008777671040181587, "operator": "ge"}, {"budget": 0.06, "threshold": -0.06858542448084638, "operator": "ge"}, {"budget": 0.07, "threshold": -0.14720921288071134, "operator": "ge"}, {"budget": 0.08, "threshold": -0.1970178377903904, "operator": "ge"}, {"budget": 0.09, "threshold": -0.26029791876181935, "operator": "ge"}, {"budget": 0.1, "threshold": -0.29301501658819845, "operator": "ge"}, {"budget": 0.11, "threshold": -0.31737544160879044, "operator": "ge"}, {"budget": 0.12, "threshold": -0.33576390817894614, "operator": "ge"}, {"budget": 0.13, "threshold": -0.3575196169243603, "operator": "ge"}, {"budget": 0.14, "threshold": -0.37340467776438296, "operator": "ge"}, {"budget": 0.15, "threshold": -0.38824439508819364, "operator": "ge"}, {"budget": 0.16, "threshold": -0.3990808618389394, "operator": "ge"}, {"budget": 0.17, "threshold": -0.41052719716613856, "operator": "ge"}, {"budget": 0.18, "threshold": -0.42250528570548507, "operator": "ge"}, {"budget": 0.19, "threshold": -0.4322239876944151, "operator": "ge"}, {"budget": 0.2, "threshold": -0.44389107240067505, "operator": "ge"}, {"budget": 0.21, "threshold": -0.4533967525003671, "operator": "ge"}, {"budget": 0.22, "threshold": -0.46273993698686017, "operator": "ge"}, {"budget": 0.23, "threshold": -0.4714396957813262, "operator": "ge"}, {"budget": 0.24, "threshold": -0.48041356391872075, "operator": "ge"}, {"budget": 0.25, "threshold": -0.48962475263886696, "operator": "ge"}, {"budget": 0.26, "threshold": -0.4987134855400229, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5067570086040428, "operator": "ge"}, {"budget": 0.28, "threshold": -0.5145840417649868, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5209860073993409, "operator": "ge"}, {"budget": 0.3, "threshold": -0.529074693424314, "operator": "ge"}, {"budget": 0.31, "threshold": -0.5369248770728956, "operator": "ge"}, {"budget": 0.32, "threshold": -0.543482409220218, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5508701177659296, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5585619098602258, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5657202189380612, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5736102957830453, "operator": "ge"}, {"budget": 0.37, "threshold": -0.5818264806696751, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5891696878467909, "operator": "ge"}, {"budget": 0.39, "threshold": -0.5960026275293587, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6023642905558723, "operator": "ge"}, {"budget": 0.41, "threshold": -0.6099884185457473, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6158449458483645, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6220295276674055, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6278321954395699, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6345972224526598, "operator": "ge"}, {"budget": 0.46, "threshold": -0.6413615985047146, "operator": "ge"}, {"budget": 0.47, "threshold": -0.6479205264440487, "operator": "ge"}, {"budget": 0.48, "threshold": -0.6541624724083701, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6602686822303517, "operator": "ge"}, {"budget": 0.5, "threshold": -0.6672925519890099, "operator": "ge"}, {"budget": 0.51, "threshold": -0.6736180253634975, "operator": "ge"}, {"budget": 0.52, "threshold": -0.6802408443306965, "operator": "ge"}, {"budget": 0.53, "threshold": -0.6871982236222831, "operator": "ge"}, {"budget": 0.54, "threshold": -0.6937428426287666, "operator": "ge"}, {"budget": 0.55, "threshold": -0.7001194193490633, "operator": "ge"}, {"budget": 0.56, "threshold": -0.7062524708893293, "operator": "ge"}, {"budget": 0.57, "threshold": -0.7124122630374025, "operator": "ge"}, {"budget": 0.58, "threshold": -0.7182706638520163, "operator": "ge"}, {"budget": 0.59, "threshold": -0.7254297213704448, "operator": "ge"}, {"budget": 0.6, "threshold": -0.7312385902000369, "operator": "ge"}, {"budget": 0.61, "threshold": -0.7372953754863506, "operator": "ge"}, {"budget": 0.62, "threshold": -0.7435432585631226, "operator": "ge"}, {"budget": 0.63, "threshold": -0.7492876047652924, "operator": "ge"}, {"budget": 0.64, "threshold": -0.755233920303894, "operator": "ge"}, {"budget": 0.65, "threshold": -0.7612731959445983, "operator": "ge"}, {"budget": 0.66, "threshold": -0.7674373369071877, "operator": "ge"}, {"budget": 0.67, "threshold": -0.7738551981161199, "operator": "ge"}, {"budget": 0.68, "threshold": -0.7801811833780682, "operator": "ge"}, {"budget": 0.69, "threshold": -0.7864478788991577, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7924685475786526, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7989419610297013, "operator": "ge"}, {"budget": 0.72, "threshold": -0.8045102410216282, "operator": "ge"}, {"budget": 0.73, "threshold": -0.8106347481336055, "operator": "ge"}, {"budget": 0.74, "threshold": -0.8165978579067018, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8224963285890461, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8282619307290832, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8331887627578294, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8394892698862985, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8460907096946254, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8528620380161824, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8591434321078519, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8647128128908742, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8706941130767775, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8763152440238643, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8821571181641451, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8883499888319946, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8937723894124214, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8995765770312665, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9058337430592378, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9120050750750591, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9188722575298894, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9251127344219303, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9324500327880662, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9393654790302219, "operator": "ge"}, {"budget": 0.95, "threshold": -0.946716361569655, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9543885460231233, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9614613300279455, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9690001756944823, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9784645365522499, "operator": "gt"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 74.97364537212735, 75.01581277672359, 75.12825918898025, 75.19151029587462, 75.27584510506712, 75.2055660974067, 75.07203598285193, 75.06500808208588, 75.04392437978777, 75.11420338744817, 75.18448239510859, 75.23367770047086, 75.27584510506712, 75.27584510506712, 75.31801250966336, 75.30395670813128, 75.31801250966336, 75.36720781502565, 75.34612411272754, 75.3742357157917, 75.3742357157917, 75.38126361655773, 75.45154262421815, 75.45154262421815, 75.47262632651626, 75.46559842575023, 75.48668212804836, 75.4937100288144, 75.45154262421815, 75.4585705249842, 75.39531941808981, 75.38126361655773, 75.39531941808981, 75.36720781502565, 75.31098460889733, 75.38126361655773, 75.33206831119544, 75.33909621196149, 75.35315201349357, 75.32504041042941, 75.33206831119544, 75.30395670813128, 75.29692880736523, 75.24773350200296, 75.14934289127838, 75.2055660974067, 75.14934289127838, 75.086091784384, 75.02986857825567, 75.04392437978777, 75.02986857825567, 75.02284067748963, 75.03689647902172, 75.02986857825567, 75.02284067748963, 75.11420338744817, 75.10014758591609, 74.98067327289338, 74.92445006676506, 74.74875254761403, 74.73469674608194, 74.76983624991215, 74.74875254761403, 74.66441773842153, 74.62225033382529, 74.51683182233467, 74.39735750931197, 74.20057628786282, 74.11624147867032, 73.98973926488158, 73.94757186028534, 73.68753953194181, 73.49075831049265, 73.32911659287372, 73.13233537142456, 72.99880525686977, 72.85824724154895, 72.6614660200998, 72.52793590554502, 72.27493147796753, 72.12734556188066, 71.83920163047299, 71.56511350059738, 71.3121090730199, 70.93260243165366, 70.6655422025441, 70.32117506500808, 69.96978002670602, 69.56918968304167, 69.05615292712068, 68.52906036966758, 68.1565816290674, 67.85438189612762, 67.34837304097266, 66.89155949117999, 66.46988544521751, 66.03415559772297, 65.50003513950384, 64.86752407056012, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.98798228969007, 100.0], "rescued": [0, 15, 42, 70, 92, 108, 124, 134, 155, 175, 195, 210, 220, 231, 237, 246, 251, 258, 269, 271, 280, 287, 294, 305, 311, 320, 325, 333, 341, 344, 351, 356, 364, 370, 377, 381, 394, 400, 414, 423, 437, 448, 463, 474, 483, 494, 511, 522, 533, 548, 564, 576, 592, 613, 633, 649, 673, 689, 704, 723, 734, 755, 775, 792, 812, 829, 849, 861, 873, 896, 911, 933, 940, 960, 979, 994, 1015, 1034, 1050, 1068, 1089, 1108, 1120, 1138, 1153, 1165, 1182, 1196, 1217, 1228, 1237, 1253, 1270, 1288, 1302, 1316, 1333, 1347, 1360, 1367, 1369], "harmed": [0, 23, 44, 56, 69, 73, 99, 128, 150, 173, 183, 188, 191, 196, 202, 205, 212, 217, 221, 226, 231, 238, 244, 245, 251, 257, 263, 268, 275, 284, 290, 304, 314, 318, 329, 341, 344, 357, 370, 377, 395, 405, 424, 436, 452, 477, 486, 505, 525, 548, 562, 576, 593, 612, 633, 650, 661, 679, 711, 738, 774, 797, 812, 832, 864, 887, 922, 951, 991, 1026, 1059, 1087, 1131, 1179, 1221, 1264, 1304, 1343, 1387, 1424, 1481, 1521, 1574, 1631, 1682, 1748, 1803, 1866, 1937, 2005, 2087, 2178, 2248, 2309, 2395, 2474, 2551, 2627, 2716, 2813, 2921], "exact_score": [75.02986857825567, 74.97364537212735, 75.01581277672359, 75.12825918898025, 75.19151029587462, 75.27584510506712, 75.2055660974067, 75.07203598285193, 75.06500808208588, 75.04392437978777, 75.11420338744817, 75.18448239510859, 75.23367770047086, 75.27584510506712, 75.27584510506712, 75.31801250966336, 75.30395670813128, 75.31801250966336, 75.36720781502565, 75.34612411272754, 75.3742357157917, 75.3742357157917, 75.38126361655773, 75.45154262421815, 75.45154262421815, 75.47262632651626, 75.46559842575023, 75.48668212804836, 75.4937100288144, 75.45154262421815, 75.4585705249842, 75.39531941808981, 75.38126361655773, 75.39531941808981, 75.36720781502565, 75.31098460889733, 75.38126361655773, 75.33206831119544, 75.33909621196149, 75.35315201349357, 75.32504041042941, 75.33206831119544, 75.30395670813128, 75.29692880736523, 75.24773350200296, 75.14934289127838, 75.2055660974067, 75.14934289127838, 75.086091784384, 75.02986857825567, 75.04392437978777, 75.02986857825567, 75.02284067748963, 75.03689647902172, 75.02986857825567, 75.02284067748963, 75.11420338744817, 75.10014758591609, 74.98067327289338, 74.92445006676506, 74.74875254761403, 74.73469674608194, 74.76983624991215, 74.74875254761403, 74.66441773842153, 74.62225033382529, 74.51683182233467, 74.39735750931197, 74.20057628786282, 74.11624147867032, 73.98973926488158, 73.94757186028534, 73.68753953194181, 73.49075831049265, 73.32911659287372, 73.13233537142456, 72.99880525686977, 72.85824724154895, 72.6614660200998, 72.52793590554502, 72.27493147796753, 72.12734556188066, 71.83920163047299, 71.56511350059738, 71.3121090730199, 70.93260243165366, 70.6655422025441, 70.32117506500808, 69.96978002670602, 69.56918968304167, 69.05615292712068, 68.52906036966758, 68.1565816290674, 67.85438189612762, 67.34837304097266, 66.89155949117999, 66.46988544521751, 66.03415559772297, 65.50003513950384, 64.86049616979408, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 73.6821758740341, "threshold_area100": 73.68262000140558, "threshold_area25": 75.26670883407127}, "test": {"score": [66.95119530385956, 67.21209199444176, 67.49283952017694, 67.74806454357258, 67.92672205994951, 68.00045373337493, 68.0203045685279, 67.97493123103536, 67.97493123103536, 67.9692595638488, 67.93522956072937, 67.90970705838981, 67.88418455605026, 67.8756770552704, 67.86433372089725, 67.8388112185577, 67.82746788418456, 67.80761704903156, 67.7452287099793, 67.78493038028527, 67.77075121231886, 67.77075121231886, 67.73955704279273, 67.71119870685988, 67.70552703967331, 67.68567620452032, 67.66582536936733, 67.65731786858747, 67.62612369906134, 67.60060119672178, 67.5779145279755, 67.58075036156879, 67.58925786234865, 67.55522785922922, 67.50985452173667, 67.46448118424411, 67.42477951393812, 67.36806284207243, 67.31134617020673, 67.26880866630745, 67.2262711624082, 67.15821115616936, 67.08731531633724, 67.04194197884468, 66.99940447494541, 66.95970280463942, 66.87462779684088, 66.79806028982219, 66.68746277968408, 66.61089527266539, 66.5002977625273, 66.41522275472876, 66.29611774381078, 66.18835606726597, 66.06925105634801, 65.98984771573603, 65.85089186966509, 65.72611519156055, 65.62686101579558, 65.46805433457165, 65.3092476533477, 65.1617843064969, 65.00014179167967, 64.85267844482885, 64.68252842923177, 64.50670674644812, 64.25431755664577, 64.03879420355614, 63.83461418483963, 63.55386665910444, 63.33267163882823, 63.06043161387289, 62.779684088137714, 62.515951563962226, 62.24654737260017, 62.022516518730676, 61.656693985196945, 61.30788645322293, 60.95340725406233, 60.64146555880101, 60.21041885262173, 59.86728298783427, 59.47310211836769, 58.97966707313614, 58.58265037007628, 58.194141167796275, 57.740407792870705, 57.35757025777727, 56.91801605081813, 56.44726767433287, 55.97084763066103, 55.49442758698919, 55.0463658792502, 54.44800499106712, 53.84680826929076, 53.342029889686074, 52.644414825738025, 51.98082976490939, 51.356946374386744, 50.65365964325213, 50.00708958398321], "calls": [0.0, 1.4264242974222272, 3.224342795564756, 5.212262144457363, 7.756004877633781, 10.679749312310353, 12.256472790176673, 13.532597907154809, 14.485437994498481, 16.01962396846553, 17.59067577914528, 19.0000850750078, 20.171284349034398, 21.750843660494002, 22.86816209624819, 23.957122196069534, 24.725633099849702, 25.468621501290308, 26.245639905850325, 26.909224966678956, 27.706094206391967, 28.34132093128775, 29.01341349289624, 29.65998355216516, 30.23849360519525, 30.862376995717895, 31.511782888580097, 32.07611377364376, 32.64895215948728, 33.13955137112554, 33.6500014179168, 34.222839803760316, 34.716274848991866, 35.34582990670108, 36.04060913705584, 36.63897002523892, 37.30255508606755, 38.082409324220855, 38.69494938037036, 39.28763860136687, 39.99092533250149, 40.77928707143465, 41.45988713382299, 42.0525763548195, 42.72466891642798, 43.50735898817457, 44.298556560701016, 45.1379633043133, 45.96602671355245, 46.73453761733262, 47.71290020701585, 48.53812778266171, 49.454102033292685, 50.40410628704307, 51.422170547032295, 52.270084791424445, 53.19173070924198, 54.039644953634124, 54.90173836599268, 55.996370133000596, 56.898165215665145, 57.79996029832969, 58.8208603919122, 59.7878796472223, 60.73788390097269, 61.64818648441709, 62.69177324674588, 63.68998667158211, 64.67969259563849, 65.78850353061281, 66.8150752913819, 68.01746873493464, 69.04404049570371, 70.05643308850637, 71.128378186768, 72.19748745143634, 73.33749255593682, 74.26764597453422, 75.4048152454414, 76.66676119445312, 78.01378215126337, 79.19065309247654, 80.15767234778663, 81.36857329211922, 82.46320505912713, 83.48410515270965, 84.66948359470267, 85.61948784845306, 86.66874627796841, 87.80591554887559, 88.8466664776111, 89.96965658055186, 90.94234750304852, 92.22414428721322, 93.3641493917137, 94.43609448997533, 95.73207044210645, 96.81819470833452, 97.84476646910359, 99.00745824235034, 100.0], "rescued": [0, 144, 297, 427, 523, 572, 640, 687, 737, 785, 809, 829, 841, 865, 884, 897, 904, 912, 919, 938, 953, 964, 973, 985, 996, 1009, 1024, 1041, 1052, 1062, 1073, 1088, 1105, 1116, 1125, 1138, 1150, 1167, 1177, 1191, 1207, 1220, 1232, 1248, 1262, 1285, 1299, 1316, 1338, 1365, 1383, 1405, 1425, 1450, 1476, 1501, 1523, 1550, 1575, 1604, 1623, 1646, 1677, 1702, 1740, 1766, 1794, 1820, 1842, 1868, 1903, 1931, 1952, 1969, 1997, 2032, 2056, 2079, 2116, 2162, 2189, 2222, 2247, 2270, 2301, 2321, 2349, 2368, 2392, 2420, 2442, 2464, 2481, 2513, 2536, 2557, 2576, 2593, 2608, 2621, 2627], "harmed": [0, 52, 106, 146, 179, 202, 263, 326, 376, 426, 462, 491, 512, 539, 562, 584, 595, 610, 639, 644, 664, 675, 695, 717, 730, 750, 772, 792, 814, 833, 852, 866, 880, 903, 928, 957, 983, 1020, 1050, 1079, 1110, 1147, 1184, 1216, 1245, 1282, 1326, 1370, 1431, 1485, 1542, 1594, 1656, 1719, 1787, 1840, 1911, 1982, 2042, 2127, 2202, 2277, 2365, 2442, 2540, 2628, 2745, 2847, 2941, 3066, 3179, 3303, 3423, 3533, 3656, 3770, 3923, 4069, 4231, 4387, 4566, 4720, 4884, 5081, 5252, 5409, 5597, 5751, 5930, 6124, 6314, 6504, 6679, 6922, 7157, 7356, 7621, 7872, 8107, 8368, 8602], "exact_score": [66.95119530385956, 67.13836032101636, 67.31134617020673, 67.46731701783739, 67.60627286390834, 67.7253778748263, 67.8388112185577, 67.88702038964354, 67.94090122791594, 67.96075206306894, 67.98060289822193, 68.00896123415477, 68.01746873493462, 67.98060289822193, 67.97209539744208, 67.97776706462865, 67.9692595638488, 67.9465728951025, 67.94373706150922, 67.90970705838981, 67.8926920568301, 67.87851288886368, 67.8728412216771, 67.86433372089725, 67.83597538496441, 67.81612454981142, 67.77642287950543, 67.77925871309871, 67.7650795451323, 67.73955704279273, 67.71403454045316, 67.6885120381136, 67.65448203499419, 67.60910869750163, 67.59492952953521, 67.57224286078892, 67.51552618892323, 67.44179451549783, 67.37089867566571, 67.30283866942688, 67.22910699600148, 67.14119615460964, 67.05328531321781, 66.9823894733857, 66.91716530074015, 66.79806028982219, 66.68746277968408, 66.58537277032583, 66.48328276096758, 66.33581941411678, 66.23089357116524, 66.108952726654, 66.00969855088903, 65.8849218727845, 65.7374585259337, 65.62969684938888, 65.46805433457165, 65.28939681819472, 65.13342597056405, 64.98596262371323, 64.81864844170944, 64.62581175736608, 64.46984090973541, 64.19192921759351, 63.984913365283724, 63.77506167938065, 63.5056574880186, 63.3014774693021, 63.06893911465274, 62.7995349232907, 62.53296656552193, 62.274905708533026, 62.05938235544338, 61.792813997674614, 61.40997646258117, 61.06400476420043, 60.80594390721152, 60.54221138303604, 60.2160905198083, 59.94385049485296, 59.54116212460653, 59.11578708561381, 58.74429288489351, 58.38981368573292, 57.9842894818932, 57.59578027961319, 57.212942744519744, 56.76488103678076, 56.37070016731418, 55.92547429316848, 55.48308425261605, 55.01800754331736, 54.53591583245895, 54.05665995519383, 53.56322490996228, 53.044267362391174, 52.48844397810737, 51.87873975555114, 51.24918469784193, 50.65649547684542, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.86218123288884, "threshold_area100": 64.12697444913933, "threshold_area25": 67.77273629583416}, "selected_budget": 28, "selected": {"val": {"score": 75.4937100288144, "calls": 27.999156651908073, "rescued": 341, "harmed": 275}, "test": {"score": 67.62612369906134, "calls": 32.64895215948728, "rescued": 1052, "harmed": 814}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.25672271847724915, "operator": "ge"}, {"budget": 0.02, "threshold": 0.11791373789310455, "operator": "ge"}, {"budget": 0.03, "threshold": 0.04957158863544464, "operator": "gt"}, {"budget": 0.04, "threshold": 0.017866630107164383, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0013224435970187187, "operator": "ge"}, {"budget": 0.06, "threshold": -0.00909316074103117, "operator": "ge"}, {"budget": 0.07, "threshold": -0.013078357093036175, "operator": "ge"}, {"budget": 0.08, "threshold": -0.01818186417222023, "operator": "ge"}, {"budget": 0.09, "threshold": -0.02189076691865921, "operator": "ge"}, {"budget": 0.1, "threshold": -0.025470755994319916, "operator": "ge"}, {"budget": 0.11, "threshold": -0.029175594449043274, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03313097730278969, "operator": "ge"}, {"budget": 0.13, "threshold": -0.036998189985752106, "operator": "gt"}, {"budget": 0.14, "threshold": -0.04146111011505127, "operator": "ge"}, {"budget": 0.15, "threshold": -0.04546751081943512, "operator": "ge"}, {"budget": 0.16, "threshold": -0.0500960536301136, "operator": "ge"}, {"budget": 0.17, "threshold": -0.05467940494418144, "operator": "ge"}, {"budget": 0.18, "threshold": -0.05963748320937157, "operator": "ge"}, {"budget": 0.19, "threshold": -0.06407909840345383, "operator": "ge"}, {"budget": 0.2, "threshold": -0.0691784992814064, "operator": "ge"}, {"budget": 0.21, "threshold": -0.07349736988544464, "operator": "ge"}, {"budget": 0.22, "threshold": -0.07889802008867264, "operator": "ge"}, {"budget": 0.23, "threshold": -0.08441638946533203, "operator": "ge"}, {"budget": 0.24, "threshold": -0.08934013545513153, "operator": "ge"}, {"budget": 0.25, "threshold": -0.09490790963172913, "operator": "ge"}, {"budget": 0.26, "threshold": -0.10101201385259628, "operator": "ge"}, {"budget": 0.27, "threshold": -0.10687714070081711, "operator": "ge"}, {"budget": 0.28, "threshold": -0.11333230137825012, "operator": "ge"}, {"budget": 0.29, "threshold": -0.12108742445707321, "operator": "ge"}, {"budget": 0.3, "threshold": -0.12670165300369263, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1335829645395279, "operator": "gt"}, {"budget": 0.32, "threshold": -0.14167168736457825, "operator": "ge"}, {"budget": 0.33, "threshold": -0.14757168292999268, "operator": "ge"}, {"budget": 0.34, "threshold": -0.15368057787418365, "operator": "ge"}, {"budget": 0.35, "threshold": -0.16099315881729126, "operator": "ge"}, {"budget": 0.36, "threshold": -0.1683836430311203, "operator": "ge"}, {"budget": 0.37, "threshold": -0.17669525742530823, "operator": "ge"}, {"budget": 0.38, "threshold": -0.1847192794084549, "operator": "ge"}, {"budget": 0.39, "threshold": -0.19368530809879303, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2027239203453064, "operator": "ge"}, {"budget": 0.41, "threshold": -0.21068745851516724, "operator": "ge"}, {"budget": 0.42, "threshold": -0.2192581593990326, "operator": "ge"}, {"budget": 0.43, "threshold": -0.22835376858711243, "operator": "ge"}, {"budget": 0.44, "threshold": -0.23734194040298462, "operator": "ge"}, {"budget": 0.45, "threshold": -0.24618029594421387, "operator": "ge"}, {"budget": 0.46, "threshold": -0.2561686038970947, "operator": "ge"}, {"budget": 0.47, "threshold": -0.2656792998313904, "operator": "ge"}, {"budget": 0.48, "threshold": -0.2750607132911682, "operator": "ge"}, {"budget": 0.49, "threshold": -0.28569525480270386, "operator": "ge"}, {"budget": 0.5, "threshold": -0.29527515172958374, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3045318126678467, "operator": "ge"}, {"budget": 0.52, "threshold": -0.31442880630493164, "operator": "ge"}, {"budget": 0.53, "threshold": -0.3253513276576996, "operator": "ge"}, {"budget": 0.54, "threshold": -0.3360317051410675, "operator": "ge"}, {"budget": 0.55, "threshold": -0.34609177708625793, "operator": "ge"}, {"budget": 0.56, "threshold": -0.35820284485816956, "operator": "ge"}, {"budget": 0.57, "threshold": -0.36913740634918213, "operator": "ge"}, {"budget": 0.58, "threshold": -0.3809331953525543, "operator": "ge"}, {"budget": 0.59, "threshold": -0.39169302582740784, "operator": "ge"}, {"budget": 0.6, "threshold": -0.40263476967811584, "operator": "ge"}, {"budget": 0.61, "threshold": -0.4140335023403168, "operator": "gt"}, {"budget": 0.62, "threshold": -0.42482757568359375, "operator": "ge"}, {"budget": 0.63, "threshold": -0.43596288561820984, "operator": "ge"}, {"budget": 0.64, "threshold": -0.4478895366191864, "operator": "ge"}, {"budget": 0.65, "threshold": -0.4593420624732971, "operator": "ge"}, {"budget": 0.66, "threshold": -0.4722622036933899, "operator": "ge"}, {"budget": 0.67, "threshold": -0.48158740997314453, "operator": "ge"}, {"budget": 0.68, "threshold": -0.4947904050350189, "operator": "ge"}, {"budget": 0.69, "threshold": -0.507918119430542, "operator": "ge"}, {"budget": 0.7, "threshold": -0.5213947296142578, "operator": "ge"}, {"budget": 0.71, "threshold": -0.5349299311637878, "operator": "ge"}, {"budget": 0.72, "threshold": -0.5495501160621643, "operator": "ge"}, {"budget": 0.73, "threshold": -0.5621535778045654, "operator": "ge"}, {"budget": 0.74, "threshold": -0.5734384655952454, "operator": "ge"}, {"budget": 0.75, "threshold": -0.586997926235199, "operator": "ge"}, {"budget": 0.76, "threshold": -0.6001017093658447, "operator": "ge"}, {"budget": 0.77, "threshold": -0.6120903491973877, "operator": "ge"}, {"budget": 0.78, "threshold": -0.6251040101051331, "operator": "ge"}, {"budget": 0.79, "threshold": -0.6372134685516357, "operator": "ge"}, {"budget": 0.8, "threshold": -0.6490820050239563, "operator": "ge"}, {"budget": 0.81, "threshold": -0.661868691444397, "operator": "ge"}, {"budget": 0.82, "threshold": -0.6746664643287659, "operator": "gt"}, {"budget": 0.83, "threshold": -0.6903525590896606, "operator": "ge"}, {"budget": 0.84, "threshold": -0.7025459408760071, "operator": "ge"}, {"budget": 0.85, "threshold": -0.7157581448554993, "operator": "ge"}, {"budget": 0.86, "threshold": -0.7285323143005371, "operator": "ge"}, {"budget": 0.87, "threshold": -0.7435368895530701, "operator": "ge"}, {"budget": 0.88, "threshold": -0.7583184242248535, "operator": "ge"}, {"budget": 0.89, "threshold": -0.775209903717041, "operator": "ge"}, {"budget": 0.9, "threshold": -0.7891313433647156, "operator": "ge"}, {"budget": 0.91, "threshold": -0.8053207993507385, "operator": "ge"}, {"budget": 0.92, "threshold": -0.8221033215522766, "operator": "ge"}, {"budget": 0.93, "threshold": -0.837272584438324, "operator": "ge"}, {"budget": 0.94, "threshold": -0.8539634943008423, "operator": "ge"}, {"budget": 0.95, "threshold": -0.8716719746589661, "operator": "ge"}, {"budget": 0.96, "threshold": -0.8881494402885437, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9082721471786499, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9281731843948364, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9495141506195068, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.00878487595756, 75.02284067748963, 75.14934289127838, 75.19151029587462, 75.27584510506712, 75.28287300583317, 75.28287300583317, 75.27584510506712, 75.254761402769, 75.254761402769, 75.26881720430107, 75.23367770047086, 75.24070560123691, 75.254761402769, 75.22664979970483, 75.22664979970483, 75.2196218989388, 75.2196218989388, 75.21259399817275, 75.17745449434254, 75.18448239510859, 75.15637079204441, 75.10717548668214, 75.06500808208588, 75.03689647902172, 74.98770117365943, 74.95256166982922, 74.94553376906318, 74.90336636446693, 74.86119895987069, 74.82605945604048, 74.70658514301778, 74.62927823459134, 74.573055028463, 74.4887202192705, 74.45358071544031, 74.38330170777989, 74.29193899782135, 74.19354838709677, 74.13732518096845, 74.07407407407408, 74.05299037177595, 74.01785086794574, 73.89134865415701, 73.89837655492305, 73.81404174573055, 73.70862323423994, 73.61023262351536, 73.5821210204512, 73.44859090589641, 73.50481411202475, 73.43453510436433, 73.42750720359828, 73.33614449363975, 73.30100498980954, 73.27289338674538, 73.24478178368122, 73.27992128751141, 73.20964227985101, 73.12530747065851, 73.12530747065851, 73.02691685993395, 72.90744254691124, 72.87933094384708, 72.81607983695271, 72.7809403331225, 72.73877292852625, 72.58415911167334, 72.47171269941668, 72.4225173940544, 72.3381825848619, 72.21168037107316, 72.05706655422026, 71.90245273736735, 71.76892262281257, 71.58619720289549, 71.36833227914822, 71.23480216459343, 71.08018834774053, 70.91854663012158, 70.71473750790639, 70.4265935764987, 70.20872865275143, 69.9486963244079, 69.69569189683041, 69.48485487384917, 69.21779464473961, 68.97884601869421, 68.77503689647902, 68.47986506430529, 68.24794433902593, 68.00196781221449, 67.65760067467848, 67.30620563637642, 66.78614097968936, 66.3996064375571, 66.0060439946588, 65.42272823107737, 64.7972450628997, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.987560615644107, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.986014477475578, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.9881228477054, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.98749033663645, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 18, 43, 72, 92, 108, 111, 111, 111, 111, 113, 115, 116, 118, 121, 123, 125, 126, 128, 131, 132, 136, 137, 140, 142, 146, 148, 149, 156, 161, 164, 171, 174, 176, 181, 183, 191, 196, 200, 204, 211, 220, 231, 244, 250, 267, 277, 285, 303, 317, 323, 345, 354, 370, 384, 403, 429, 453, 478, 496, 510, 536, 555, 578, 602, 624, 652, 677, 699, 712, 737, 763, 785, 805, 831, 856, 879, 903, 928, 952, 983, 1009, 1026, 1051, 1076, 1105, 1132, 1155, 1178, 1205, 1230, 1257, 1282, 1294, 1315, 1321, 1337, 1354, 1362, 1368, 1369], "harmed": [0, 21, 44, 55, 69, 73, 75, 75, 76, 79, 81, 81, 87, 88, 89, 95, 97, 99, 101, 105, 111, 114, 119, 129, 137, 145, 154, 160, 168, 179, 188, 200, 220, 233, 246, 260, 273, 288, 305, 323, 338, 356, 370, 388, 412, 428, 450, 473, 505, 523, 548, 562, 581, 598, 625, 649, 679, 707, 727, 755, 781, 807, 840, 880, 908, 939, 972, 1003, 1047, 1076, 1108, 1146, 1186, 1228, 1276, 1320, 1369, 1424, 1468, 1514, 1568, 1623, 1681, 1737, 1799, 1864, 1921, 1982, 2039, 2095, 2162, 2222, 2282, 2343, 2414, 2494, 2565, 2638, 2729, 2824, 2921], "exact_score": [75.02986857825567, 75.00878487595756, 75.02284067748963, 75.15637079204441, 75.19151029587462, 75.27584510506712, 75.28287300583317, 75.28287300583317, 75.27584510506712, 75.254761402769, 75.254761402769, 75.26881720430107, 75.23367770047086, 75.24070560123691, 75.254761402769, 75.22664979970483, 75.22664979970483, 75.2196218989388, 75.2196218989388, 75.21259399817275, 75.17745449434254, 75.18448239510859, 75.15637079204441, 75.10717548668214, 75.06500808208588, 75.03689647902172, 74.98770117365943, 74.95256166982922, 74.94553376906318, 74.90336636446693, 74.86119895987069, 74.82605945604048, 74.70658514301778, 74.62927823459134, 74.573055028463, 74.4887202192705, 74.45358071544031, 74.38330170777989, 74.29193899782135, 74.19354838709677, 74.13732518096845, 74.07407407407408, 74.05299037177595, 74.01785086794574, 73.89134865415701, 73.89837655492305, 73.81404174573055, 73.70862323423994, 73.61023262351536, 73.5821210204512, 73.44859090589641, 73.50481411202475, 73.43453510436433, 73.42750720359828, 73.33614449363975, 73.30100498980954, 73.27289338674538, 73.24478178368122, 73.27992128751141, 73.20964227985101, 73.12530747065851, 73.11827956989248, 73.02691685993395, 72.90744254691124, 72.87933094384708, 72.81607983695271, 72.7809403331225, 72.73877292852625, 72.58415911167334, 72.47171269941668, 72.4225173940544, 72.3381825848619, 72.21168037107316, 72.05706655422026, 71.90245273736735, 71.76892262281257, 71.58619720289549, 71.36833227914822, 71.23480216459343, 71.08018834774053, 70.91854663012158, 70.71473750790639, 70.41956567573267, 70.20872865275143, 69.9486963244079, 69.69569189683041, 69.48485487384917, 69.21779464473961, 68.97884601869421, 68.77503689647902, 68.47986506430529, 68.24794433902593, 68.00196781221449, 67.65760067467848, 67.30620563637642, 66.78614097968936, 66.3996064375571, 66.0060439946588, 65.42272823107737, 64.7972450628997, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 72.8070619571167, "threshold_area100": 72.80750579801814, "threshold_area25": 75.19361866610443}, "test": {"score": [66.95119530385956, 67.2234353288149, 67.5382128576695, 67.77642287950543, 67.93239372713609, 68.00045373337493, 68.00896123415477, 67.99761789978163, 67.97776706462865, 67.97209539744208, 67.97493123103536, 67.96358789666222, 67.96358789666222, 67.97776706462865, 67.97776706462865, 67.96642373025551, 67.94373706150922, 67.94090122791594, 67.9465728951025, 67.96358789666222, 67.95508039588238, 67.93522956072937, 67.92672205994951, 67.9295578935428, 67.90970705838981, 67.90403539120324, 67.88418455605026, 67.8558262201174, 67.83597538496441, 67.80478121543828, 67.762243711539, 67.71403454045316, 67.64030286702776, 67.59776536312849, 67.55522785922922, 67.53537702407623, 67.47298868502396, 67.38791367722541, 67.30851033661345, 67.23477866318804, 67.14686782179622, 67.01358364291184, 66.95403113745286, 66.85477696168789, 66.78104528826249, 66.74985111873634, 66.60522360547883, 66.53716359923999, 66.41522275472876, 66.3499985820832, 66.21671440319882, 66.108952726654, 65.96999688058305, 65.81686186654568, 65.69208518844114, 65.55312934237018, 65.35178515724697, 65.20999347758273, 65.04551512917222, 64.9008876159147, 64.69103593001162, 64.52088591441454, 64.3053625613249, 64.11536171057483, 63.93386836060459, 63.73536000907467, 63.51983665598503, 63.324164138048374, 63.04625244590647, 62.8364007600034, 62.61520573972719, 62.24654737260017, 61.917590675779145, 61.64818648441709, 61.31639395400278, 60.85131724470407, 60.5138530471032, 60.25579219011428, 59.95235799563281, 59.64041630037149, 59.322802937923605, 58.99101040750928, 58.633695374755405, 58.27070867481496, 57.941751977993924, 57.626974449139325, 57.334883589030994, 56.971896889090544, 56.48413351104558, 55.936817627541615, 55.55965175963474, 55.02651504409721, 54.53307999886567, 54.09352579190653, 53.580239911521986, 52.979043189745624, 52.40904063749539, 51.80217224853245, 51.18963219238295, 50.54306213311403, 50.00708958398321], "calls": [0.0, 1.4519467997617899, 3.323596971329722, 5.240620480390211, 7.801378215126337, 10.685420979496923, 13.016476193176985, 14.366332983580524, 16.04231063721181, 16.93276238550322, 17.772169129115504, 18.472620026656834, 19.187250092164593, 19.757252644414823, 20.43501687320988, 21.01352692623997, 21.685619487848452, 22.25562204009869, 22.876669597028044, 23.31622380398718, 23.92309219295012, 24.405183903808524, 24.941156452939342, 25.51399483878286, 25.979071548081556, 26.628477440943765, 27.266539999432833, 27.81385588293679, 28.522814281257975, 29.277146017071722, 29.807446899015964, 30.507897796557298, 31.367155375322575, 31.97118793069223, 32.60641465558801, 33.34373138984205, 33.990301449110966, 34.8580665286561, 35.67478660352211, 36.55105918384709, 37.4415109321385, 38.272410174970936, 39.080622749057085, 39.99943283328135, 40.872869580013045, 41.74630632674475, 42.67645974534214, 43.59243399597312, 44.533930748943654, 45.50378583784703, 46.394237586138445, 47.284689334429856, 48.24603692255339, 49.2102203442702, 50.22828460425942, 51.03933301193886, 52.11411394379378, 53.06695403113745, 54.10203329268638, 55.100246717522616, 56.1438334798514, 57.13921107109435, 58.10623032640445, 59.07608541530783, 60.06862717295749, 61.11504976887956, 62.283413209312876, 63.15968578963786, 64.41312423786972, 65.52760684003063, 66.8122394577886, 68.13090207866603, 69.35031052377848, 70.41658395485354, 71.58494739528685, 72.85256501148513, 74.11734679409012, 74.98511187363526, 76.13362447891558, 77.22825624592349, 78.1754246660806, 79.25587726512208, 80.33916569775685, 81.575589144429, 82.50007089583983, 83.49828432067606, 84.43410940646004, 85.52590533987465, 86.69994044749454, 87.98740889884581, 88.88069648073052, 90.1483140969288, 91.30817003658225, 92.35742846609762, 93.44922439951223, 94.7026628477441, 95.73490627569974, 96.9316280520659, 98.0290956526671, 99.07268241499588, 100.0], "rescued": [0, 147, 310, 431, 525, 572, 584, 584, 587, 589, 592, 594, 600, 607, 614, 619, 625, 633, 644, 652, 658, 661, 671, 682, 685, 697, 708, 711, 723, 737, 749, 756, 766, 775, 783, 799, 807, 819, 834, 853, 867, 877, 896, 914, 934, 964, 992, 1017, 1043, 1075, 1091, 1114, 1145, 1171, 1213, 1232, 1263, 1290, 1324, 1356, 1386, 1417, 1442, 1468, 1509, 1534, 1573, 1603, 1650, 1696, 1755, 1793, 1826, 1854, 1895, 1926, 1963, 1991, 2032, 2069, 2097, 2139, 2174, 2220, 2247, 2281, 2313, 2340, 2374, 2405, 2427, 2460, 2492, 2516, 2532, 2559, 2578, 2601, 2613, 2623, 2627], "harmed": [0, 51, 103, 140, 179, 202, 211, 215, 225, 229, 231, 237, 243, 245, 252, 261, 275, 284, 293, 295, 304, 314, 327, 337, 347, 361, 379, 392, 411, 436, 463, 487, 523, 547, 570, 593, 623, 665, 708, 753, 798, 855, 895, 948, 994, 1035, 1114, 1163, 1232, 1287, 1350, 1411, 1491, 1571, 1657, 1725, 1827, 1904, 1996, 2079, 2183, 2274, 2375, 2468, 2573, 2668, 2783, 2882, 3027, 3147, 3284, 3452, 3601, 3724, 3882, 4077, 4233, 4352, 4500, 4647, 4787, 4946, 5107, 5281, 5424, 5569, 5704, 5859, 6065, 6289, 6444, 6665, 6871, 7050, 7247, 7486, 7706, 7943, 8171, 8409, 8602], "exact_score": [66.95119530385956, 67.14403198820294, 67.31985367098659, 67.4843320193971, 67.61761619828148, 67.7452287099793, 67.84448288574426, 67.88985622323682, 67.93806539432265, 67.96075206306894, 67.98060289822193, 68.01463290134134, 68.01179706774806, 68.00612540056149, 67.99761789978163, 67.99478206618835, 67.97776706462865, 67.97209539744208, 67.96642373025551, 67.96358789666222, 67.97776706462865, 67.96642373025551, 67.93806539432265, 67.9465728951025, 67.95508039588238, 67.93806539432265, 67.9125428919831, 67.89836372401668, 67.85015455293083, 67.82463205059128, 67.7452287099793, 67.6686612029606, 67.59492952953521, 67.54388452485608, 67.47298868502396, 67.37089867566571, 67.28865950146046, 67.17239032413578, 67.06462864759096, 66.95686697104614, 66.85477696168789, 66.78104528826249, 66.7044777812438, 66.58253693673255, 66.5002977625273, 66.39253608598247, 66.27059524147123, 66.13163939540027, 66.01253438448231, 65.85939937044493, 65.7176076907807, 65.55596517596346, 65.38865099395967, 65.2156651447693, 65.07103763151179, 64.90939511669454, 64.70805093157134, 64.54073674956753, 64.32804923007119, 64.13237671213453, 63.93670419419788, 63.743867509854525, 63.56237415988429, 63.372373309134225, 63.17386495760429, 62.93849076936166, 62.74848991861157, 62.56416073504807, 62.274905708533026, 62.022516518730676, 61.70490315628279, 61.489379803193145, 61.17743810793183, 60.81161557439809, 60.55639055100246, 60.25295635652099, 59.986387998752235, 59.68295380427077, 59.388027110569155, 59.0817570824944, 58.7471287184868, 58.44085869041204, 58.123245327964156, 57.797124464736406, 57.47100360150866, 57.15622607265406, 56.798911039900176, 56.35368516575448, 55.931145960355046, 55.502935087769046, 55.09457505033605, 54.6578566769702, 54.240989138757335, 53.767404928678786, 53.32501488812637, 52.81456484133511, 52.28142812579758, 51.75963474463319, 51.20097552675609, 50.59977880497972, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.505631391883824, "threshold_area100": 63.84166123131892, "threshold_area25": 67.86490088761592}, "selected_budget": 6, "selected": {"val": {"score": 75.28287300583317, "calls": 5.994799353433129, "rescued": 111, "harmed": 75}, "test": {"score": 68.00896123415477, "calls": 13.016476193176985, "rescued": 584, "harmed": 211}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 5.248384952545166, "operator": "ge"}, {"budget": 0.02, "threshold": 3.3020334243774414, "operator": "ge"}, {"budget": 0.03, "threshold": 1.9653422832489014, "operator": "ge"}, {"budget": 0.04, "threshold": 0.8922786116600037, "operator": "ge"}, {"budget": 0.05, "threshold": -0.04384356737136841, "operator": "ge"}, {"budget": 0.06, "threshold": -0.8761905431747437, "operator": "ge"}, {"budget": 0.07, "threshold": -1.4418458938598633, "operator": "ge"}, {"budget": 0.08, "threshold": -1.8330726623535156, "operator": "ge"}, {"budget": 0.09, "threshold": -2.187870740890503, "operator": "ge"}, {"budget": 0.1, "threshold": -2.492055654525757, "operator": "ge"}, {"budget": 0.11, "threshold": -2.7826569080352783, "operator": "gt"}, {"budget": 0.12, "threshold": -3.011385917663574, "operator": "ge"}, {"budget": 0.13, "threshold": -3.1917712688446045, "operator": "ge"}, {"budget": 0.14, "threshold": -3.404916286468506, "operator": "ge"}, {"budget": 0.15, "threshold": -3.582495927810669, "operator": "ge"}, {"budget": 0.16, "threshold": -3.7181365489959717, "operator": "ge"}, {"budget": 0.17, "threshold": -3.881146192550659, "operator": "ge"}, {"budget": 0.18, "threshold": -4.0235700607299805, "operator": "ge"}, {"budget": 0.19, "threshold": -4.169328689575195, "operator": "ge"}, {"budget": 0.2, "threshold": -4.2896857261657715, "operator": "ge"}, {"budget": 0.21, "threshold": -4.415888786315918, "operator": "ge"}, {"budget": 0.22, "threshold": -4.522805690765381, "operator": "ge"}, {"budget": 0.23, "threshold": -4.645427703857422, "operator": "ge"}, {"budget": 0.24, "threshold": -4.764692306518555, "operator": "ge"}, {"budget": 0.25, "threshold": -4.86661958694458, "operator": "ge"}, {"budget": 0.26, "threshold": -4.967249870300293, "operator": "ge"}, {"budget": 0.27, "threshold": -5.080850124359131, "operator": "ge"}, {"budget": 0.28, "threshold": -5.1706647872924805, "operator": "ge"}, {"budget": 0.29, "threshold": -5.2490997314453125, "operator": "ge"}, {"budget": 0.3, "threshold": -5.3401384353637695, "operator": "ge"}, {"budget": 0.31, "threshold": -5.428611755371094, "operator": "ge"}, {"budget": 0.32, "threshold": -5.5189924240112305, "operator": "ge"}, {"budget": 0.33, "threshold": -5.604736328125, "operator": "ge"}, {"budget": 0.34, "threshold": -5.684410095214844, "operator": "ge"}, {"budget": 0.35, "threshold": -5.756799221038818, "operator": "ge"}, {"budget": 0.36, "threshold": -5.83175802230835, "operator": "ge"}, {"budget": 0.37, "threshold": -5.900543212890625, "operator": "ge"}, {"budget": 0.38, "threshold": -5.972257137298584, "operator": "ge"}, {"budget": 0.39, "threshold": -6.038463592529297, "operator": "ge"}, {"budget": 0.4, "threshold": -6.113181114196777, "operator": "ge"}, {"budget": 0.41, "threshold": -6.182395935058594, "operator": "ge"}, {"budget": 0.42, "threshold": -6.247591018676758, "operator": "ge"}, {"budget": 0.43, "threshold": -6.309103965759277, "operator": "ge"}, {"budget": 0.44, "threshold": -6.373902320861816, "operator": "ge"}, {"budget": 0.45, "threshold": -6.433617115020752, "operator": "ge"}, {"budget": 0.46, "threshold": -6.494715690612793, "operator": "ge"}, {"budget": 0.47, "threshold": -6.555920124053955, "operator": "ge"}, {"budget": 0.48, "threshold": -6.620940208435059, "operator": "ge"}, {"budget": 0.49, "threshold": -6.6832146644592285, "operator": "ge"}, {"budget": 0.5, "threshold": -6.741873264312744, "operator": "ge"}, {"budget": 0.51, "threshold": -6.79601526260376, "operator": "ge"}, {"budget": 0.52, "threshold": -6.848924160003662, "operator": "ge"}, {"budget": 0.53, "threshold": -6.903500556945801, "operator": "ge"}, {"budget": 0.54, "threshold": -6.9624481201171875, "operator": "ge"}, {"budget": 0.55, "threshold": -7.017719745635986, "operator": "ge"}, {"budget": 0.56, "threshold": -7.065712928771973, "operator": "ge"}, {"budget": 0.57, "threshold": -7.122844219207764, "operator": "ge"}, {"budget": 0.58, "threshold": -7.1757097244262695, "operator": "ge"}, {"budget": 0.59, "threshold": -7.224441051483154, "operator": "ge"}, {"budget": 0.6, "threshold": -7.280536651611328, "operator": "ge"}, {"budget": 0.61, "threshold": -7.336173057556152, "operator": "ge"}, {"budget": 0.62, "threshold": -7.384378433227539, "operator": "ge"}, {"budget": 0.63, "threshold": -7.436959743499756, "operator": "ge"}, {"budget": 0.64, "threshold": -7.492459297180176, "operator": "ge"}, {"budget": 0.65, "threshold": -7.53985071182251, "operator": "ge"}, {"budget": 0.66, "threshold": -7.5890679359436035, "operator": "ge"}, {"budget": 0.67, "threshold": -7.6327314376831055, "operator": "ge"}, {"budget": 0.68, "threshold": -7.6819257736206055, "operator": "ge"}, {"budget": 0.69, "threshold": -7.736146926879883, "operator": "ge"}, {"budget": 0.7, "threshold": -7.780304908752441, "operator": "ge"}, {"budget": 0.71, "threshold": -7.824363708496094, "operator": "ge"}, {"budget": 0.72, "threshold": -7.875487327575684, "operator": "ge"}, {"budget": 0.73, "threshold": -7.923114776611328, "operator": "ge"}, {"budget": 0.74, "threshold": -7.965633392333984, "operator": "ge"}, {"budget": 0.75, "threshold": -8.011942863464355, "operator": "ge"}, {"budget": 0.76, "threshold": -8.061466217041016, "operator": "ge"}, {"budget": 0.77, "threshold": -8.106587409973145, "operator": "ge"}, {"budget": 0.78, "threshold": -8.160847663879395, "operator": "ge"}, {"budget": 0.79, "threshold": -8.213508605957031, "operator": "ge"}, {"budget": 0.8, "threshold": -8.260320663452148, "operator": "ge"}, {"budget": 0.81, "threshold": -8.315142631530762, "operator": "ge"}, {"budget": 0.82, "threshold": -8.371238708496094, "operator": "ge"}, {"budget": 0.83, "threshold": -8.42589282989502, "operator": "ge"}, {"budget": 0.84, "threshold": -8.476493835449219, "operator": "gt"}, {"budget": 0.85, "threshold": -8.529458999633789, "operator": "ge"}, {"budget": 0.86, "threshold": -8.588991165161133, "operator": "ge"}, {"budget": 0.87, "threshold": -8.6572847366333, "operator": "ge"}, {"budget": 0.88, "threshold": -8.72627067565918, "operator": "ge"}, {"budget": 0.89, "threshold": -8.790143966674805, "operator": "gt"}, {"budget": 0.9, "threshold": -8.857277870178223, "operator": "ge"}, {"budget": 0.91, "threshold": -8.928359985351562, "operator": "ge"}, {"budget": 0.92, "threshold": -9.0015230178833, "operator": "ge"}, {"budget": 0.93, "threshold": -9.090072631835938, "operator": "ge"}, {"budget": 0.94, "threshold": -9.174156188964844, "operator": "ge"}, {"budget": 0.95, "threshold": -9.27954387664795, "operator": "ge"}, {"budget": 0.96, "threshold": -9.395980834960938, "operator": "ge"}, {"budget": 0.97, "threshold": -9.550141334533691, "operator": "ge"}, {"budget": 0.98, "threshold": -9.731410026550293, "operator": "ge"}, {"budget": 0.99, "threshold": -10.035175323486328, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.31098460889733, 75.47262632651626, 75.47262632651626, 75.40234731885586, 75.28287300583317, 75.24773350200296, 75.17745449434254, 75.1704265935765, 75.254761402769, 75.36017991425962, 75.47262632651626, 75.60615644107105, 75.74671445639187, 75.97863518167124, 76.25975121231289, 76.47058823529413, 76.59006254831682, 76.7587321667018, 76.85009487666035, 76.85712277742638, 76.91334598355472, 76.94848548738491, 76.94145758661888, 76.94145758661888, 76.90631808278867, 76.77278796823389, 76.73764846440369, 76.73764846440369, 76.57600674678474, 76.37922552533558, 76.26677911307893, 76.14730480005622, 76.01377468550145, 75.8521329678825, 75.67643544873147, 75.62021224260313, 75.35315201349357, 75.26178930353502, 75.09311968515004, 74.8963384637009, 74.75578044838007, 74.46763651697238, 74.25679949399114, 74.10921357790427, 73.87729285262492, 73.67348373040973, 73.46967460819454, 73.23775388291519, 72.99177735610374, 72.85121934078292, 72.66849392086584, 72.4225173940544, 72.19059666877504, 72.06409445498629, 71.84622953123903, 71.67755991285402, 71.46672288987278, 71.29102537072175, 71.08721624850656, 70.96774193548387, 70.77798861480076, 70.52498418722328, 70.32117506500808, 70.06114273666456, 69.8995010190456, 69.72380349989459, 69.47782697308314, 69.29510155316608, 69.10534823248295, 68.92262281256589, 68.74692529341486, 68.5501440719657, 68.32525124745239, 68.17766533136552, 67.95277250685221, 67.78410288846722, 67.67165647621056, 67.52407056012369, 67.36242884250474, 67.1656476210556, 66.98995010190455, 66.80019678122146, 66.64558296436854, 66.52610865134585, 66.4347459413873, 66.31527162836461, 66.17471361304379, 66.08335090308525, 65.97793239159463, 65.8514301778059, 65.67573265865487, 65.50706304026987, 65.35244922341697, 65.24000281116031, 65.0361936889451, 64.86752407056012, 64.73399395600535, 64.54424063532223, 64.42476632229953, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.991636798088411, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99044205495818, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.98727949961346, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 40, 72, 85, 98, 109, 130, 146, 169, 208, 238, 276, 312, 354, 403, 458, 504, 545, 589, 629, 660, 690, 723, 753, 782, 808, 825, 846, 871, 888, 903, 918, 931, 946, 961, 970, 986, 991, 1009, 1020, 1028, 1042, 1046, 1052, 1067, 1076, 1085, 1092, 1099, 1107, 1117, 1125, 1133, 1142, 1153, 1157, 1167, 1178, 1183, 1193, 1203, 1213, 1220, 1223, 1224, 1230, 1233, 1236, 1239, 1244, 1247, 1250, 1253, 1256, 1259, 1261, 1265, 1272, 1275, 1280, 1281, 1287, 1287, 1288, 1292, 1298, 1303, 1308, 1312, 1314, 1320, 1322, 1328, 1332, 1335, 1337, 1339, 1344, 1350, 1360, 1369], "harmed": [0, 0, 9, 22, 45, 73, 99, 125, 149, 176, 191, 213, 230, 252, 268, 283, 299, 323, 343, 370, 400, 422, 450, 481, 510, 541, 577, 603, 628, 668, 711, 742, 772, 806, 844, 878, 902, 945, 976, 1011, 1047, 1081, 1126, 1162, 1198, 1240, 1278, 1314, 1354, 1397, 1427, 1461, 1504, 1546, 1575, 1610, 1644, 1685, 1715, 1754, 1781, 1818, 1861, 1893, 1931, 1960, 1988, 2026, 2055, 2087, 2116, 2144, 2175, 2210, 2234, 2268, 2296, 2319, 2343, 2371, 2400, 2431, 2458, 2481, 2502, 2521, 2543, 2568, 2585, 2602, 2626, 2653, 2683, 2709, 2728, 2759, 2785, 2809, 2842, 2869, 2921], "exact_score": [75.02986857825567, 75.31098460889733, 75.47262632651626, 75.47262632651626, 75.40234731885586, 75.28287300583317, 75.24773350200296, 75.17745449434254, 75.1704265935765, 75.254761402769, 75.36017991425962, 75.47262632651626, 75.60615644107105, 75.74671445639187, 75.97863518167124, 76.25975121231289, 76.47058823529413, 76.59006254831682, 76.7587321667018, 76.85009487666035, 76.85712277742638, 76.91334598355472, 76.94848548738491, 76.94145758661888, 76.94145758661888, 76.90631808278867, 76.77278796823389, 76.73764846440369, 76.73764846440369, 76.57600674678474, 76.37922552533558, 76.26677911307893, 76.14730480005622, 76.01377468550145, 75.8521329678825, 75.67643544873147, 75.62021224260313, 75.35315201349357, 75.26178930353502, 75.09311968515004, 74.8963384637009, 74.75578044838007, 74.46763651697238, 74.25679949399114, 74.10921357790427, 73.87729285262492, 73.67348373040973, 73.46967460819454, 73.23775388291519, 72.99177735610374, 72.85121934078292, 72.66849392086584, 72.4225173940544, 72.19059666877504, 72.06409445498629, 71.84622953123903, 71.67755991285402, 71.46672288987278, 71.29102537072175, 71.08721624850656, 70.96774193548387, 70.77798861480076, 70.52498418722328, 70.32117506500808, 70.06114273666456, 69.8995010190456, 69.72380349989459, 69.47782697308314, 69.29510155316608, 69.10534823248295, 68.92262281256589, 68.74692529341486, 68.5501440719657, 68.32525124745239, 68.17766533136552, 67.95277250685221, 67.78410288846722, 67.67165647621056, 67.52407056012369, 67.36242884250474, 67.1656476210556, 66.98995010190455, 66.80019678122146, 66.64558296436854, 66.52610865134585, 66.4347459413873, 66.31527162836461, 66.17471361304379, 66.08335090308525, 65.97090449082859, 65.8514301778059, 65.67573265865487, 65.50706304026987, 65.35244922341697, 65.24000281116031, 65.0361936889451, 64.86752407056012, 64.73399395600535, 64.54424063532223, 64.42476632229953, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 71.91232212475276, "threshold_area100": 71.91278375149344, "threshold_area25": 75.94560404807085}, "test": {"score": [66.95119530385956, 67.49851118736352, 67.7650795451323, 67.90687122479653, 67.99478206618835, 67.99761789978163, 68.05149873805405, 68.0770212403936, 68.07985707398689, 68.10254374273318, 68.07418540680032, 68.03448373649434, 68.05717040524061, 68.01746873493462, 67.94940872869579, 67.89836372401668, 67.8756770552704, 67.77642287950543, 67.68284037092704, 67.62612369906134, 67.51269035532995, 67.38791367722541, 67.26313699912089, 67.14686782179622, 66.98522530697898, 66.82641862575504, 66.65059694297139, 66.4067152539489, 66.2365652383518, 66.03238521963532, 65.8877577063778, 65.65805518532173, 65.4396959986388, 65.24118764710887, 65.05118679635878, 64.85551427842215, 64.6967075971982, 64.48685591129512, 64.25715339023907, 64.03879420355614, 63.783569180160505, 63.491478320052174, 63.27879080055582, 63.12565578651844, 62.86759492952954, 62.60953407254062, 62.416697388197264, 62.2607265405666, 61.9431131781187, 61.656693985196945, 61.429827297734164, 61.27952811729007, 61.03564642826759, 60.80594390721152, 60.54221138303604, 60.33519553072626, 60.1593738479426, 59.90982049173355, 59.72549130817003, 59.53549045741996, 59.322802937923605, 59.05339874656155, 58.87190539659133, 58.64787454272184, 58.35861951620679, 58.174290332643274, 57.94458781158721, 57.760258628023706, 57.561750276493775, 57.30085358591157, 57.071151064855506, 56.9236877180047, 56.75353770240762, 56.61174602274338, 56.404730170433595, 56.23458015483651, 56.03607180330658, 55.831891784590084, 55.64472676743328, 55.4745767518362, 55.31860590420554, 55.13427672064203, 54.904574199585966, 54.70039418086946, 54.53591583245895, 54.377109151235004, 54.19561580126477, 53.965913280208724, 53.75889742789893, 53.57456824433542, 53.344865723279355, 53.149193205342705, 52.981879023338905, 52.68695232963729, 52.48844397810737, 52.18784561721919, 51.890083089924275, 51.63769390012194, 51.288886368147914, 50.82948132603578, 50.00708958398321], "calls": [0.0, 4.877633780449763, 6.573462269234041, 8.067946572895103, 9.429146697671781, 10.705271814649917, 12.205427785497546, 13.427672064203271, 14.380512151546949, 15.401412245129457, 16.43649150667839, 17.60201911351842, 18.574710036015087, 19.42546011400051, 20.57680855287412, 21.501290304284947, 22.26980120806511, 23.29070130164762, 24.192496384312168, 25.21339647789468, 26.049967387913675, 26.93191163542523, 27.853557553242776, 28.888636814791706, 29.898193574001077, 30.887899498057454, 31.761336244789156, 33.028953860987436, 33.88537560615943, 34.71343901539858, 35.654935768369114, 36.718373365850894, 37.76196012817968, 38.76584522020248, 39.79525281456484, 40.71689873238238, 41.703768822845475, 42.63959390862944, 43.603777330346254, 44.567960752063065, 45.63707001673142, 46.7373734509259, 47.6476760343703, 48.583501120154274, 49.61007288092335, 50.67351047840513, 51.63769390012194, 52.46859314295437, 53.72203159118622, 54.70039418086947, 55.695771772112415, 56.47562601026572, 57.27533108357201, 58.10623032640445, 59.152652922326524, 60.17355301590903, 60.93072058531605, 62.00266568357768, 62.816549924850406, 63.69849417236196, 64.77611093781017, 65.7941751977994, 66.70164194765051, 67.6147803646882, 68.51090378016617, 69.30210135269263, 70.20389643535717, 70.95539233757762, 71.8174857499362, 72.76465417009331, 73.63525508323173, 74.46048265887758, 75.23750106343759, 75.95213112894535, 76.61571618977399, 77.37004792558773, 78.1725888324873, 78.96662223860704, 79.78050647987976, 80.52349488132037, 81.21543827808185, 82.07753169044041, 82.96798343873182, 83.84142018546352, 84.54470691659813, 85.27068031647903, 86.08172872415847, 86.98068797322973, 87.88248305589428, 88.68785979638714, 89.51025153843972, 90.30995661174602, 91.10115418427247, 92.26101012392593, 93.06071519723223, 94.00504778379604, 94.98341037347929, 96.01848963502822, 97.16983807390183, 98.28432067606273, 100.0], "rescued": [0, 210, 341, 441, 511, 574, 656, 720, 774, 839, 894, 960, 1020, 1065, 1125, 1181, 1224, 1264, 1311, 1371, 1404, 1442, 1479, 1523, 1556, 1589, 1614, 1650, 1678, 1693, 1727, 1760, 1794, 1813, 1839, 1866, 1890, 1913, 1929, 1946, 1960, 1980, 1994, 2015, 2034, 2054, 2076, 2098, 2116, 2125, 2151, 2164, 2174, 2188, 2202, 2216, 2227, 2253, 2264, 2275, 2296, 2309, 2322, 2332, 2343, 2354, 2365, 2374, 2384, 2394, 2401, 2408, 2415, 2427, 2434, 2446, 2451, 2467, 2473, 2481, 2489, 2500, 2501, 2509, 2515, 2524, 2533, 2541, 2548, 2557, 2558, 2560, 2566, 2576, 2578, 2585, 2590, 2597, 2601, 2609, 2627], "harmed": [0, 17, 54, 104, 143, 205, 268, 323, 376, 433, 498, 578, 630, 689, 773, 847, 898, 973, 1053, 1133, 1206, 1288, 1369, 1454, 1544, 1633, 1720, 1842, 1930, 2017, 2102, 2216, 2327, 2416, 2509, 2605, 2685, 2782, 2879, 2973, 3077, 3200, 3289, 3364, 3474, 3585, 3675, 3752, 3882, 3992, 4098, 4164, 4260, 4355, 4462, 4549, 4622, 4736, 4812, 4890, 4986, 5094, 5171, 5260, 5373, 5449, 5541, 5615, 5695, 5797, 5885, 5944, 6011, 6073, 6153, 6225, 6300, 6388, 6460, 6528, 6591, 6667, 6749, 6829, 6893, 6958, 7031, 7120, 7200, 7274, 7356, 7427, 7492, 7606, 7678, 7791, 7901, 7997, 8124, 8294, 8602], "exact_score": [66.95119530385956, 67.01358364291184, 67.11567365227008, 67.21209199444176, 67.36806284207243, 67.51552618892323, 67.6885120381136, 67.82179621699798, 67.90970705838981, 67.96075206306894, 67.99478206618835, 68.01463290134134, 68.0429912372742, 68.0429912372742, 68.0997079091399, 68.08269290758018, 68.0997079091399, 68.03448373649434, 68.02881206930778, 68.04866290446077, 68.0203045685279, 67.94090122791594, 67.8756770552704, 67.7991095482517, 67.69985537248674, 67.64030286702776, 67.52119785610981, 67.37657034285228, 67.26030116552761, 67.12418115304993, 66.98806114057227, 66.79522445622891, 66.60522360547883, 66.41238692113546, 66.21671440319882, 66.00686271729575, 65.80835436576582, 65.61835351501574, 65.39148682755295, 65.19581430961631, 65.00581345886623, 64.80446927374301, 64.61446842299293, 64.39894506990329, 64.16640671525394, 63.950883362164305, 63.69565833876868, 63.420582480220055, 63.19938745994384, 63.023565777160194, 62.74565408501829, 62.55281740067493, 62.35714488273828, 62.118934860902364, 61.87788900547315, 61.57445481099168, 61.35609562430876, 61.10370643450642, 60.854153078297365, 60.590420554121884, 60.38056886821881, 60.14519467997618, 59.91265632532683, 59.691461305050616, 59.48444545274083, 59.25474293168477, 59.025040410628705, 58.78115872160621, 58.494739528684455, 58.233842838102255, 57.998468649859625, 57.74891529365057, 57.50786943822136, 57.227121912486176, 56.99458355783682, 56.80741854068003, 56.59756685477696, 56.31965516263505, 56.070101806425996, 55.82054845021693, 55.6050250971273, 55.36681507529138, 55.15412755579503, 54.89039503161954, 54.66352834415677, 54.41113915435443, 54.21546663641777, 53.95740577942886, 53.71919575759294, 53.47814990216374, 53.220089045174824, 52.987550690525474, 52.76635567024927, 52.50262314607379, 52.185009783625894, 51.881575589144425, 51.64336556730851, 51.34560304001361, 50.96276550492017, 50.56291296826703, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.69274155327648, "threshold_area100": 61.121331140288696, "threshold_area25": 67.67433287014718}, "selected_budget": 22, "selected": {"val": {"score": 76.94848548738491, "calls": 21.997329397708903, "rescued": 723, "harmed": 450}, "test": {"score": 67.26313699912089, "calls": 27.853557553242776, "rescued": 1479, "harmed": 1369}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.3393183946609497, "operator": "ge"}, {"budget": 0.02, "threshold": 0.14114713668823242, "operator": "ge"}, {"budget": 0.03, "threshold": 0.05335664749145508, "operator": "ge"}, {"budget": 0.04, "threshold": 0.01854848861694336, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0015606880187988281, "operator": "ge"}, {"budget": 0.06, "threshold": -0.12459909915924072, "operator": "ge"}, {"budget": 0.07, "threshold": -0.27598655223846436, "operator": "ge"}, {"budget": 0.08, "threshold": -0.3804362416267395, "operator": "ge"}, {"budget": 0.09, "threshold": -0.518202543258667, "operator": "ge"}, {"budget": 0.1, "threshold": -0.648638129234314, "operator": "ge"}, {"budget": 0.11, "threshold": -0.8102673888206482, "operator": "ge"}, {"budget": 0.12, "threshold": -0.9980943202972412, "operator": "ge"}, {"budget": 0.13, "threshold": -1.1596251726150513, "operator": "ge"}, {"budget": 0.14, "threshold": -1.3220181465148926, "operator": "ge"}, {"budget": 0.15, "threshold": -1.4799506664276123, "operator": "ge"}, {"budget": 0.16, "threshold": -1.6539175510406494, "operator": "ge"}, {"budget": 0.17, "threshold": -1.8267258405685425, "operator": "ge"}, {"budget": 0.18, "threshold": -1.9595658779144287, "operator": "ge"}, {"budget": 0.19, "threshold": -2.1140263080596924, "operator": "ge"}, {"budget": 0.2, "threshold": -2.241058111190796, "operator": "ge"}, {"budget": 0.21, "threshold": -2.381551504135132, "operator": "ge"}, {"budget": 0.22, "threshold": -2.519536018371582, "operator": "ge"}, {"budget": 0.23, "threshold": -2.661513090133667, "operator": "ge"}, {"budget": 0.24, "threshold": -2.793231725692749, "operator": "ge"}, {"budget": 0.25, "threshold": -2.9237654209136963, "operator": "ge"}, {"budget": 0.26, "threshold": -3.043084144592285, "operator": "ge"}, {"budget": 0.27, "threshold": -3.1683995723724365, "operator": "ge"}, {"budget": 0.28, "threshold": -3.2786448001861572, "operator": "ge"}, {"budget": 0.29, "threshold": -3.3867852687835693, "operator": "ge"}, {"budget": 0.3, "threshold": -3.4970555305480957, "operator": "ge"}, {"budget": 0.31, "threshold": -3.6016759872436523, "operator": "ge"}, {"budget": 0.32, "threshold": -3.703610420227051, "operator": "ge"}, {"budget": 0.33, "threshold": -3.813966989517212, "operator": "ge"}, {"budget": 0.34, "threshold": -3.91042423248291, "operator": "ge"}, {"budget": 0.35, "threshold": -4.010792255401611, "operator": "ge"}, {"budget": 0.36, "threshold": -4.103292942047119, "operator": "ge"}, {"budget": 0.37, "threshold": -4.1824493408203125, "operator": "ge"}, {"budget": 0.38, "threshold": -4.263962268829346, "operator": "ge"}, {"budget": 0.39, "threshold": -4.352852821350098, "operator": "ge"}, {"budget": 0.4, "threshold": -4.448176860809326, "operator": "ge"}, {"budget": 0.41, "threshold": -4.53317928314209, "operator": "ge"}, {"budget": 0.42, "threshold": -4.614540100097656, "operator": "ge"}, {"budget": 0.43, "threshold": -4.681990146636963, "operator": "ge"}, {"budget": 0.44, "threshold": -4.758218765258789, "operator": "ge"}, {"budget": 0.45, "threshold": -4.833991050720215, "operator": "ge"}, {"budget": 0.46, "threshold": -4.9070634841918945, "operator": "ge"}, {"budget": 0.47, "threshold": -4.98075532913208, "operator": "ge"}, {"budget": 0.48, "threshold": -5.040156841278076, "operator": "ge"}, {"budget": 0.49, "threshold": -5.10205078125, "operator": "ge"}, {"budget": 0.5, "threshold": -5.156235694885254, "operator": "ge"}, {"budget": 0.51, "threshold": -5.208989143371582, "operator": "ge"}, {"budget": 0.52, "threshold": -5.2693071365356445, "operator": "ge"}, {"budget": 0.53, "threshold": -5.317171573638916, "operator": "ge"}, {"budget": 0.54, "threshold": -5.372905731201172, "operator": "ge"}, {"budget": 0.55, "threshold": -5.426263809204102, "operator": "ge"}, {"budget": 0.56, "threshold": -5.480266571044922, "operator": "gt"}, {"budget": 0.57, "threshold": -5.528471946716309, "operator": "ge"}, {"budget": 0.58, "threshold": -5.578763008117676, "operator": "ge"}, {"budget": 0.59, "threshold": -5.627248764038086, "operator": "ge"}, {"budget": 0.6, "threshold": -5.6810126304626465, "operator": "ge"}, {"budget": 0.61, "threshold": -5.732137203216553, "operator": "ge"}, {"budget": 0.62, "threshold": -5.772554874420166, "operator": "ge"}, {"budget": 0.63, "threshold": -5.813444137573242, "operator": "ge"}, {"budget": 0.64, "threshold": -5.8603620529174805, "operator": "ge"}, {"budget": 0.65, "threshold": -5.9021897315979, "operator": "ge"}, {"budget": 0.66, "threshold": -5.951406955718994, "operator": "ge"}, {"budget": 0.67, "threshold": -5.9967241287231445, "operator": "ge"}, {"budget": 0.68, "threshold": -6.0443243980407715, "operator": "ge"}, {"budget": 0.69, "threshold": -6.099249362945557, "operator": "ge"}, {"budget": 0.7, "threshold": -6.1413774490356445, "operator": "ge"}, {"budget": 0.71, "threshold": -6.186395645141602, "operator": "ge"}, {"budget": 0.72, "threshold": -6.227931022644043, "operator": "ge"}, {"budget": 0.73, "threshold": -6.2691450119018555, "operator": "ge"}, {"budget": 0.74, "threshold": -6.314609527587891, "operator": "ge"}, {"budget": 0.75, "threshold": -6.358847141265869, "operator": "ge"}, {"budget": 0.76, "threshold": -6.4010186195373535, "operator": "ge"}, {"budget": 0.77, "threshold": -6.449102401733398, "operator": "ge"}, {"budget": 0.78, "threshold": -6.492105007171631, "operator": "ge"}, {"budget": 0.79, "threshold": -6.538804531097412, "operator": "ge"}, {"budget": 0.8, "threshold": -6.578289031982422, "operator": "ge"}, {"budget": 0.81, "threshold": -6.615086078643799, "operator": "ge"}, {"budget": 0.82, "threshold": -6.665858268737793, "operator": "ge"}, {"budget": 0.83, "threshold": -6.7156596183776855, "operator": "ge"}, {"budget": 0.84, "threshold": -6.765621185302734, "operator": "ge"}, {"budget": 0.85, "threshold": -6.81533670425415, "operator": "ge"}, {"budget": 0.86, "threshold": -6.864131927490234, "operator": "ge"}, {"budget": 0.87, "threshold": -6.91412353515625, "operator": "ge"}, {"budget": 0.88, "threshold": -6.96208381652832, "operator": "ge"}, {"budget": 0.89, "threshold": -7.016883850097656, "operator": "ge"}, {"budget": 0.9, "threshold": -7.07081413269043, "operator": "ge"}, {"budget": 0.91, "threshold": -7.136663436889648, "operator": "ge"}, {"budget": 0.92, "threshold": -7.196423053741455, "operator": "ge"}, {"budget": 0.93, "threshold": -7.261832237243652, "operator": "ge"}, {"budget": 0.94, "threshold": -7.341192245483398, "operator": "ge"}, {"budget": 0.95, "threshold": -7.429665565490723, "operator": "ge"}, {"budget": 0.96, "threshold": -7.551607131958008, "operator": "ge"}, {"budget": 0.97, "threshold": -7.688793182373047, "operator": "ge"}, {"budget": 0.98, "threshold": -7.883120536804199, "operator": "ge"}, {"budget": 0.99, "threshold": -8.136220932006836, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 74.95958957059527, 75.00175697519151, 75.12825918898025, 75.19151029587462, 75.27584510506712, 75.2055660974067, 75.086091784384, 75.0509522805538, 75.10717548668214, 75.24773350200296, 75.28287300583317, 75.45154262421815, 75.51479373111252, 75.50776583034647, 75.53587743341063, 75.65535174643334, 75.83104926558437, 75.99971888396936, 76.03485838779956, 76.0910815939279, 76.03485838779956, 76.0418862885656, 76.0418862885656, 75.95052357860708, 75.94349567784103, 75.90835617401083, 75.83807716635042, 75.75374235715792, 75.76779815869, 75.70454705179563, 75.55696113570876, 75.56398903647481, 75.52182163187857, 75.42343102115399, 75.27584510506712, 75.14231499051233, 74.91039426523297, 74.76280834914611, 74.66441773842153, 74.50277602080259, 74.36221800548176, 74.13029728020241, 73.95459976105137, 73.77890224190034, 73.64537212734557, 73.36425609670391, 73.16747487525477, 73.08314006606227, 72.94258205074145, 72.76688453159042, 72.53496380631105, 72.39440579099023, 72.15545716494483, 72.01489914962401, 71.81811792817486, 71.6213367067257, 71.40347178297843, 71.20669056152927, 70.96774193548387, 70.84123972169513, 70.70770960714034, 70.5952631948837, 70.40550987420058, 70.32117506500808, 70.13844964509101, 69.93464052287581, 69.80111040832104, 69.68163609529834, 69.56216178227564, 69.4216037669548, 69.18968304167545, 69.02804132405652, 68.83126010260735, 68.62745098039215, 68.50094876660341, 68.33227914821842, 68.11441422447115, 67.95980040761825, 67.80518659076535, 67.62948907161432, 67.46081945322932, 67.3202614379085, 67.20078712488579, 67.06022910956497, 66.9126431934781, 66.79316888045541, 66.61044346053833, 66.47691334598356, 66.2801321245344, 66.08335090308525, 65.93576498699838, 65.82331857474173, 65.6476210555907, 65.48597933797174, 65.2891981165226, 65.1205284981376, 64.90266357439033, 64.65668704757888, 64.45990582612974, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99128540305011, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 14, 41, 70, 92, 108, 124, 137, 156, 182, 217, 248, 291, 323, 351, 387, 428, 476, 527, 565, 599, 631, 660, 690, 713, 740, 767, 792, 819, 847, 876, 896, 931, 956, 978, 992, 1004, 1021, 1037, 1047, 1057, 1073, 1083, 1093, 1105, 1116, 1126, 1137, 1151, 1164, 1170, 1176, 1190, 1193, 1200, 1204, 1209, 1215, 1221, 1225, 1234, 1235, 1243, 1246, 1251, 1256, 1266, 1274, 1279, 1284, 1291, 1295, 1298, 1302, 1304, 1306, 1311, 1312, 1313, 1317, 1320, 1322, 1325, 1329, 1329, 1333, 1337, 1338, 1341, 1343, 1348, 1348, 1351, 1354, 1355, 1358, 1360, 1361, 1362, 1365, 1369], "harmed": [0, 24, 45, 56, 69, 73, 99, 129, 153, 171, 186, 212, 231, 254, 283, 315, 339, 362, 389, 422, 448, 488, 516, 546, 582, 610, 642, 677, 716, 742, 780, 821, 855, 886, 922, 957, 988, 1038, 1075, 1099, 1132, 1168, 1211, 1246, 1283, 1313, 1363, 1402, 1428, 1461, 1492, 1531, 1565, 1602, 1629, 1661, 1694, 1731, 1765, 1803, 1830, 1850, 1874, 1904, 1921, 1952, 1991, 2018, 2040, 2062, 2089, 2126, 2152, 2184, 2215, 2235, 2264, 2296, 2319, 2345, 2373, 2399, 2422, 2443, 2463, 2488, 2509, 2536, 2558, 2588, 2621, 2642, 2661, 2689, 2713, 2744, 2770, 2802, 2838, 2869, 2921], "exact_score": [75.02986857825567, 74.95958957059527, 75.00175697519151, 75.12825918898025, 75.19151029587462, 75.27584510506712, 75.2055660974067, 75.086091784384, 75.0509522805538, 75.10717548668214, 75.24773350200296, 75.28287300583317, 75.45154262421815, 75.51479373111252, 75.50776583034647, 75.53587743341063, 75.65535174643334, 75.83104926558437, 75.99971888396936, 76.03485838779956, 76.0910815939279, 76.03485838779956, 76.0418862885656, 76.0418862885656, 75.95052357860708, 75.94349567784103, 75.90835617401083, 75.83807716635042, 75.75374235715792, 75.76779815869, 75.70454705179563, 75.55696113570876, 75.56398903647481, 75.52182163187857, 75.42343102115399, 75.27584510506712, 75.14231499051233, 74.91039426523297, 74.76280834914611, 74.66441773842153, 74.50277602080259, 74.36221800548176, 74.13029728020241, 73.95459976105137, 73.77890224190034, 73.64537212734557, 73.36425609670391, 73.16747487525477, 73.08314006606227, 72.94258205074145, 72.76688453159042, 72.53496380631105, 72.39440579099023, 72.15545716494483, 72.01489914962401, 71.81811792817486, 71.61430880595965, 71.40347178297843, 71.20669056152927, 70.96774193548387, 70.84123972169513, 70.70770960714034, 70.5952631948837, 70.40550987420058, 70.32117506500808, 70.13844964509101, 69.93464052287581, 69.80111040832104, 69.68163609529834, 69.56216178227564, 69.4216037669548, 69.18968304167545, 69.02804132405652, 68.83126010260735, 68.62745098039215, 68.50094876660341, 68.33227914821842, 68.11441422447115, 67.95980040761825, 67.80518659076535, 67.62948907161432, 67.46081945322932, 67.3202614379085, 67.20078712488579, 67.06022910956497, 66.9126431934781, 66.79316888045541, 66.61044346053833, 66.47691334598356, 66.2801321245344, 66.08335090308525, 65.93576498699838, 65.82331857474173, 65.6476210555907, 65.48597933797174, 65.2891981165226, 65.1205284981376, 64.90266357439033, 64.65668704757888, 64.45990582612974, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 71.8026381728928, "threshold_area100": 71.80307822053553, "threshold_area25": 75.5274439524914}, "test": {"score": [66.95119530385956, 67.1865694921022, 67.4843320193971, 67.7452287099793, 67.92672205994951, 68.00045373337493, 68.01463290134134, 67.97776706462865, 67.97493123103536, 67.95791622947566, 67.92105039276295, 67.82179621699798, 67.70552703967331, 67.6516462014009, 67.52686952329637, 67.4077645123784, 67.28015200068059, 67.1128378186768, 66.95403113745286, 66.79806028982219, 66.69313444687066, 66.55985026798628, 66.37835691801605, 66.23089357116524, 66.02104188526216, 65.83954853529195, 65.73178685874713, 65.54745767518362, 65.40566599551939, 65.2156651447693, 65.04267929557894, 64.86685761279529, 64.66551342767207, 64.46700507614213, 64.2486458894592, 64.02177920199642, 63.79491251453364, 63.57088166066415, 63.41491081303349, 63.19938745994384, 62.958341604514644, 62.69744491393246, 62.48759322802938, 62.30326404446587, 62.06788985622323, 61.78430649689476, 61.55176814224541, 61.322065621189346, 61.126393103252695, 60.93072058531605, 60.68683889629356, 60.49116637835692, 60.26997135808071, 60.07713467373734, 59.855939653461135, 59.67161046989763, 59.416385446502, 59.195190426225786, 58.959816237983155, 58.71593454896067, 58.51459036383745, 58.259365340441825, 58.10623032640444, 57.870856138161805, 57.67234778663188, 57.48518276947509, 57.29801775231829, 57.068315231262225, 56.85279187817258, 56.52099934775827, 56.26293849076936, 56.07860930720585, 55.85174261974307, 55.698607605705696, 55.48592008620934, 55.31009840342569, 55.10875421830247, 54.89606669880611, 54.69472251368289, 54.547259166832085, 54.36292998326858, 54.22680997079091, 54.02830161926098, 53.8751666052236, 53.65680741854067, 53.43844823185775, 53.273969883447236, 53.109491535036724, 52.95635652099935, 52.74934066868956, 52.54799648356634, 52.38635396874911, 52.244562289084875, 52.119785610980344, 51.92978476023027, 51.72844057510705, 51.50724555483084, 51.217990528315795, 50.920228001020895, 50.58843547060658, 50.00708958398321], "calls": [0.0, 1.386722627116241, 3.1704619572923463, 5.198082976490939, 7.756004877633781, 10.679749312310353, 12.250801122990103, 13.521254572781668, 14.556333834330601, 15.554547259166831, 16.55843235118963, 17.831721634574482, 19.1673992570116, 20.281881859172504, 21.31128945353487, 22.33502538071066, 23.568612993789525, 24.66608059439072, 25.471457334883592, 26.478178260499675, 27.21549499475371, 28.12012591101154, 28.939681819470835, 29.92371607634064, 30.873720330091032, 31.72730624166974, 32.58372798684173, 33.44865723279358, 34.353288149051416, 35.20687406063012, 36.00374330034314, 36.9395683861271, 37.81016929926552, 38.86509939596745, 39.73853614269915, 40.702719564415965, 41.72078382440518, 42.608399739103305, 43.603777330346254, 44.6728865950146, 45.84975753622778, 46.933045968862544, 48.03618523665031, 48.98051782321413, 50.07514959022205, 51.20381136034937, 52.23605478830502, 53.342029889686074, 54.306213311402885, 55.236366730000285, 56.2742818251425, 57.167569407027194, 58.34727618183365, 59.16683209029294, 60.19340385106202, 61.23982644698409, 62.23236820463375, 63.16252162323115, 64.17491421603381, 65.1419334713439, 66.09760939228086, 67.08447948274396, 67.96075206306894, 68.8597113121402, 69.85225306978987, 70.6633014774693, 71.66435073589882, 72.63987749198877, 73.638090916825, 74.77809602132547, 75.67705527039674, 76.5192978476023, 77.42109293026685, 78.22930550435301, 79.105578084678, 79.96767149703655, 80.72767490003686, 81.63797748348128, 82.5029067294331, 83.34514930663869, 84.05410770495988, 84.72336443297507, 85.61948784845306, 86.53829793267731, 87.5081530215807, 88.35606726597284, 89.01398065961489, 89.68890905481666, 90.42055412188412, 91.1493633553583, 91.8951875903922, 92.70907183166491, 93.34997022374726, 93.95400277911692, 94.68848367977768, 95.38042707653915, 96.25669965686413, 97.10177806766299, 97.97237898080141, 98.78626322207413, 100.0], "rescued": [0, 136, 293, 426, 523, 572, 638, 688, 741, 793, 839, 893, 950, 1009, 1057, 1104, 1167, 1215, 1252, 1295, 1328, 1370, 1405, 1462, 1501, 1544, 1586, 1626, 1660, 1692, 1712, 1755, 1780, 1822, 1843, 1859, 1882, 1900, 1932, 1955, 1993, 2016, 2053, 2075, 2101, 2125, 2147, 2168, 2183, 2200, 2213, 2223, 2234, 2242, 2251, 2271, 2286, 2293, 2307, 2319, 2329, 2342, 2360, 2369, 2378, 2388, 2400, 2406, 2420, 2427, 2433, 2438, 2444, 2451, 2457, 2464, 2467, 2479, 2481, 2483, 2487, 2495, 2504, 2520, 2531, 2535, 2539, 2545, 2555, 2560, 2560, 2571, 2577, 2579, 2583, 2587, 2595, 2604, 2611, 2617, 2627], "harmed": [0, 53, 105, 146, 179, 202, 263, 326, 380, 438, 497, 586, 684, 762, 854, 943, 1051, 1158, 1251, 1349, 1419, 1508, 1607, 1716, 1829, 1936, 2016, 2121, 2205, 2304, 2385, 2490, 2586, 2698, 2796, 2892, 2995, 3092, 3179, 3278, 3401, 3516, 3627, 3714, 3823, 3947, 4051, 4153, 4237, 4323, 4422, 4501, 4590, 4666, 4753, 4838, 4943, 5028, 5125, 5223, 5304, 5407, 5479, 5571, 5650, 5726, 5804, 5891, 5981, 6105, 6202, 6272, 6358, 6419, 6500, 6569, 6643, 6730, 6803, 6857, 6926, 6982, 7061, 7131, 7219, 7300, 7362, 7426, 7490, 7568, 7639, 7707, 7763, 7809, 7880, 7955, 8041, 8152, 8264, 8387, 8602], "exact_score": [66.95119530385956, 67.12701698664321, 67.31418200380001, 67.4474661826844, 67.60060119672178, 67.71687037404645, 67.8388112185577, 67.88418455605026, 67.94090122791594, 67.95508039588238, 67.98060289822193, 68.00896123415477, 68.00896123415477, 67.99194623259507, 67.95791622947566, 67.96075206306894, 67.9295578935428, 67.88985622323682, 67.80761704903156, 67.71119870685988, 67.6516462014009, 67.5410486912628, 67.45597368346425, 67.33686867254629, 67.23194282959476, 67.05044947962453, 66.88313529762073, 66.71298528202365, 66.59671610469897, 66.3698494172362, 66.21671440319882, 65.98701188214275, 65.80551853217254, 65.63253268298216, 65.45103933301193, 65.27521765022829, 65.04267929557894, 64.86118594560871, 64.63715509173922, 64.435810906616, 64.19760088478009, 63.953719195757586, 63.72968834188809, 63.51132915520517, 63.32983580523495, 63.154014122451294, 62.92714743498852, 62.67759407877945, 62.484757394436095, 62.29475654368601, 62.07639735700309, 61.83818733516717, 61.59997731333126, 61.40714062898789, 61.183109775118396, 60.99027309077503, 60.74071973456597, 60.5308680486629, 60.34086719791283, 60.13668717919632, 59.90698465814026, 59.71131214020361, 59.48160961914755, 59.22354876215863, 58.99951790828914, 58.758472052859936, 58.537277032583724, 58.290559509967956, 58.08637949125145, 57.84816946941553, 57.64115361710574, 57.40294359526983, 57.204435243739894, 56.98607605705697, 56.76204520318747, 56.472790176672426, 56.192042650937246, 55.97368346425432, 55.74114510960497, 55.511442588548896, 55.29308340186597, 55.0662167144032, 54.81666335819414, 54.6011400051045, 54.374273317641716, 54.181436633298354, 53.943226611462435, 53.773076595865355, 53.50934407168987, 53.28247738422709, 53.06411819754416, 52.80038567336869, 52.53948898278649, 52.315458128916994, 52.11411394379377, 51.83903808524516, 51.56112639310325, 51.252020531435214, 50.90604883305447, 50.47783796046848, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.30250204456782, "threshold_area100": 60.42865042679296, "threshold_area25": 67.28355500099254}, "selected_budget": 20, "selected": {"val": {"score": 76.0910815939279, "calls": 19.994377679387167, "rescued": 599, "harmed": 448}, "test": {"score": 66.69313444687066, "calls": 27.21549499475371, "rescued": 1328, "harmed": 1419}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 75.62021224260313, "calls": 16.99346405228758}, "test": {"score": 67.94090122791594, "calls": 24.06771970620764}}, "old_normalized_interior": {"val": {"score": 75.56398903647481, "calls": 5.994799353433129}, "test": {"score": 67.98911039900179, "calls": 12.973938689277714}}, "old_normalized_endpoints": {"val": {"score": 75.56398903647481, "calls": 5.994799353433129}, "test": {"score": 67.98911039900179, "calls": 12.973938689277714}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.24773350200296, 75.26881720430107, 75.33909621196149, 75.42343102115399, 75.54993323494273, 75.56398903647481, 75.42343102115399, 75.36720781502565, 75.33206831119544, 75.31098460889733, 75.31801250966336, 75.35315201349357, 75.36017991425962, 75.38126361655773, 75.4093752196219, 75.41640312038794, 75.41640312038794, 75.4093752196219, 75.4445147234521, 75.47965422728231, 75.4585705249842, 75.46559842575023, 75.43748682268607, 75.46559842575023, 75.45154262421815, 75.43045892192002, 75.43045892192002, 75.4585705249842, 75.4585705249842, 75.43748682268607, 75.41640312038794, 75.4445147234521, 75.41640312038794, 75.38829151732378, 75.35315201349357, 75.28287300583317, 75.28287300583317, 75.22664979970483, 75.12825918898025, 75.12825918898025, 75.086091784384, 75.10717548668214, 74.95256166982922, 74.91039426523297, 74.81903155527444, 74.70658514301778, 74.62225033382529, 74.58008292922904, 74.50277602080259, 74.49574812003654, 74.49574812003654, 74.4184412116101, 74.39032960854593, 74.31302270011948, 74.25679949399114, 74.25679949399114, 74.2849110970553, 74.22165999016094, 74.11624147867032, 74.00379506641366, 73.91946025722117, 73.7367348373041, 73.68051163117578, 73.5821210204512, 73.30100498980954, 73.16747487525477, 73.00583315763582, 72.94258205074145, 72.83013563848478, 72.59118701243938, 72.49982430248085, 72.28195937873357, 72.14842926417879, 71.95164804272964, 71.79703422587673, 71.68458781362007, 71.5510576990653, 71.31913697378593, 71.04504884391034, 70.72879330943847, 70.4617330803289, 70.1525054466231, 69.90652891981165, 69.70271979759646, 69.44971537001898, 69.1826551409094, 68.97181811792818, 68.70475788881862, 68.38147445358072, 68.0371073160447, 67.74193548387098, 67.41865204863308, 67.08131281186309, 66.73694567432707, 66.34338323142877, 65.97793239159463, 65.57031414716424, 65.14864010120178, 64.68479865064305, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.988895916789655, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.98727949961346, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.98910675381264, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.98798228969007, 100.0], "exact_score": [75.02986857825567, 75.24773350200296, 75.27584510506712, 75.33909621196149, 75.42343102115399, 75.54993323494273, 75.56398903647481, 75.42343102115399, 75.36720781502565, 75.33206831119544, 75.31098460889733, 75.31801250966336, 75.35315201349357, 75.36017991425962, 75.38126361655773, 75.4093752196219, 75.41640312038794, 75.41640312038794, 75.4093752196219, 75.4445147234521, 75.47965422728231, 75.4585705249842, 75.46559842575023, 75.43748682268607, 75.46559842575023, 75.45154262421815, 75.43045892192002, 75.43045892192002, 75.4585705249842, 75.4585705249842, 75.43748682268607, 75.41640312038794, 75.4445147234521, 75.41640312038794, 75.38829151732378, 75.35315201349357, 75.28287300583317, 75.28287300583317, 75.22664979970483, 75.12825918898025, 75.12825918898025, 75.086091784384, 75.10717548668214, 74.95256166982922, 74.91039426523297, 74.81903155527444, 74.70658514301778, 74.62225033382529, 74.58008292922904, 74.50277602080259, 74.49574812003654, 74.49574812003654, 74.4184412116101, 74.39032960854593, 74.31302270011948, 74.25679949399114, 74.25679949399114, 74.2849110970553, 74.22165999016094, 74.11624147867032, 74.00379506641366, 73.91946025722117, 73.7367348373041, 73.68051163117578, 73.5821210204512, 73.30100498980954, 73.16747487525477, 73.00583315763582, 72.94258205074145, 72.83013563848478, 72.59118701243938, 72.49982430248085, 72.28195937873357, 72.14842926417879, 71.95164804272964, 71.79703422587673, 71.68458781362007, 71.5510576990653, 71.31913697378593, 71.04504884391034, 70.72879330943847, 70.4617330803289, 70.1525054466231, 69.90652891981165, 69.70271979759646, 69.44971537001898, 69.1826551409094, 68.97181811792818, 68.70475788881862, 68.37444655281467, 68.0371073160447, 67.74193548387098, 67.41865204863308, 67.08131281186309, 66.73694567432707, 66.33635533066273, 65.97793239159463, 65.57031414716424, 65.14864010120178, 64.68479865064305, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 75.39813057839623, "exact_area100": 73.1486777473186}, "test": {"score": [66.95119530385956, 67.34537617332614, 67.56657119360236, 67.71403454045316, 67.84448288574426, 67.94940872869579, 67.98911039900179, 67.95508039588238, 67.88134872245696, 67.8756770552704, 67.85015455293083, 67.8388112185577, 67.82179621699798, 67.8189603834047, 67.82463205059128, 67.82463205059128, 67.79060204747185, 67.77358704591214, 67.72821370841959, 67.7083628732666, 67.70269120608002, 67.71119870685988, 67.68284037092704, 67.64313870062104, 67.60060119672178, 67.5382128576695, 67.4843320193971, 67.47582451861724, 67.43895868190455, 67.38791367722541, 67.3000028358336, 67.25462949834103, 67.20358449366191, 67.17522615772906, 67.1128378186768, 67.07030031477753, 67.03059864447154, 67.00507614213198, 66.9086577999603, 66.80373195700876, 66.77537362107591, 66.73567195076993, 66.6477611093781, 66.5541786007997, 66.47761109378102, 66.39537191957575, 66.27910274225108, 66.1458185633667, 66.02954938604204, 65.96432521339648, 65.85372770325837, 65.75163769390012, 65.60133851345603, 65.50492017128434, 65.32909848850069, 65.11073930181777, 64.93491761903411, 64.76193176984374, 64.55775175112724, 64.33939256444432, 64.15222754728752, 63.93954002779117, 63.746703343447805, 63.51700082239174, 63.32699997164166, 63.10013328417888, 62.8364007600034, 62.646399909253326, 62.40251822023084, 62.20400986870091, 61.96863568045827, 61.7587839945552, 61.54893230865213, 61.27102061651022, 60.995944757961595, 60.71236139863313, 60.417434704931516, 60.06295550577092, 59.73967047613647, 59.42205711368857, 59.10160791764739, 58.798173723165924, 58.559963701330005, 58.28772367637467, 57.88503530612823, 57.530556106967644, 57.14488273828091, 56.74786603522105, 56.373536000907464, 55.98502679862746, 55.57099509400788, 55.171142557354734, 54.79397668944787, 54.26367580750362, 53.73904659274593, 53.174715707682275, 52.576354819499194, 51.93829226101012, 51.31440887048748, 50.59410713779315, 50.00708958398321], "calls": [0.0, 2.5267277316167087, 4.735842100785526, 6.740776451237841, 8.887502481354394, 11.876471088676517, 12.973938689277714, 14.085585457845335, 15.214247227972663, 16.705895698040436, 18.27127584153362, 19.660834302243142, 20.650540226299523, 21.801888665173127, 22.595922071292858, 23.69055383830077, 24.50443807957349, 25.392053994271617, 26.265490741003315, 27.082210815869324, 27.788333380597223, 28.528485948444544, 29.34520602331055, 30.18744860051612, 30.936108669143298, 31.761336244789156, 32.53268298216261, 33.057312196920286, 33.83149476788702, 34.57731900292091, 35.388367410600345, 36.057624138615544, 36.718373365850894, 37.47554093525792, 37.985990982049174, 38.69778521396364, 39.341519439639285, 39.942716161415646, 40.79630207299436, 41.672574653319344, 42.302129711028556, 43.093327283555, 43.87601735530159, 44.6360207583019, 45.4357258316082, 46.17020673226895, 47.03230014462751, 47.95678189603834, 48.855741145109604, 49.49380370359867, 50.27932960893855, 51.07052718146499, 51.81635141649888, 52.550832317159625, 53.475314068570455, 54.50472166293282, 55.31293423701897, 56.11831097751184, 57.088166066415226, 58.17145449904999, 59.11295125202053, 60.13952301278961, 61.083855599353434, 62.19266653432777, 63.06326744746619, 64.00476420043671, 65.04267929557894, 65.94447437824348, 66.79522445622891, 67.70269120608002, 68.55911295125202, 69.41269886283072, 70.18120976661089, 71.18509485863369, 72.10674077645123, 73.14182003800018, 74.04361512066473, 75.13824688767264, 76.21586365312083, 77.21124124436378, 78.16124549811417, 79.065876414372, 79.92796982673057, 80.8496157445481, 81.97544168108215, 82.92261010123926, 84.02007770184045, 85.12888863681479, 86.3000879108414, 87.38621217706944, 88.62547145733488, 89.59249071264497, 90.70130164761932, 91.8923517567989, 92.96429685506055, 94.1269886283073, 95.40311374528542, 96.56580551853217, 97.71715395740578, 99.02163741031677, 100.0], "exact_score": [66.95119530385956, 67.13836032101636, 67.29716700224031, 67.41343617956498, 67.49851118736352, 67.56940702719564, 67.64313870062104, 67.73955704279273, 67.782094546692, 67.84731871933755, 67.88702038964354, 67.92672205994951, 67.96075206306894, 67.9862745654085, 67.96358789666222, 67.9125428919831, 67.8756770552704, 67.87851288886368, 67.85299038652411, 67.85015455293083, 67.83597538496441, 67.82463205059128, 67.82179621699798, 67.83030371777784, 67.8189603834047, 67.762243711539, 67.75373621075916, 67.7083628732666, 67.71687037404645, 67.68284037092704, 67.67433287014718, 67.60060119672178, 67.53537702407623, 67.47582451861724, 67.42194368034482, 67.3510478405127, 67.26030116552761, 67.19791282647533, 67.1128378186768, 67.0391061452514, 67.00507614213198, 66.87462779684088, 66.79522445622891, 66.74417945154978, 66.62507444063182, 66.50596942971386, 66.4265660891019, 66.29044607662422, 66.14014689618013, 66.01820605166888, 65.8849218727845, 65.75163769390012, 65.59283101267617, 65.42551683067236, 65.19014264242973, 64.99730595808639, 64.79029010577659, 64.57476675268695, 64.38193006834358, 64.16640671525394, 63.96222669653745, 63.752375010634374, 63.556702492697724, 63.3383433060148, 63.10296911777217, 62.85341576156311, 62.62938490769362, 62.3713240507047, 62.12177069449565, 61.86370983750673, 61.58579814536482, 61.31072228681621, 61.02430309389445, 60.75773473612568, 60.43161387289794, 60.11116467685675, 59.802058815188715, 59.498624620707254, 59.16683209029294, 58.82653205909877, 58.54578453336357, 58.231007004508974, 57.876527805348374, 57.51921277259451, 57.15339023906076, 56.793239372713614, 56.46711850948586, 56.115475143918545, 55.80353344865723, 55.403680912004084, 55.04353004565692, 54.66636417775005, 54.21546663641777, 53.71919575759294, 53.24844738110768, 52.777699004622406, 52.26724895783115, 51.69441057198764, 51.16978135722996, 50.60828630575958, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 67.7143181238125, "exact_area100": 63.444718871683804}}, "old_normalized_endpoint_budget": 6}, {"seed": 2028, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 0.75, "validation_area100": 0.7366296855061084, "training_seconds": 71.64574939897284, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.09950050546538433, "operator": "ge"}, {"budget": 0.02, "threshold": 0.03087574677953869, "operator": "ge"}, {"budget": 0.03, "threshold": -0.011228500772213757, "operator": "ge"}, {"budget": 0.04, "threshold": -0.11769492376171528, "operator": "ge"}, {"budget": 0.05, "threshold": -0.2115661690006723, "operator": "ge"}, {"budget": 0.06, "threshold": -0.2514193704539225, "operator": "ge"}, {"budget": 0.07, "threshold": -0.2676926753243753, "operator": "ge"}, {"budget": 0.08, "threshold": -0.2883414626331634, "operator": "ge"}, {"budget": 0.09, "threshold": -0.30147071058809527, "operator": "ge"}, {"budget": 0.1, "threshold": -0.3141994759364118, "operator": "ge"}, {"budget": 0.11, "threshold": -0.3262325483535869, "operator": "ge"}, {"budget": 0.12, "threshold": -0.3382976898612685, "operator": "ge"}, {"budget": 0.13, "threshold": -0.34874708704300605, "operator": "ge"}, {"budget": 0.14, "threshold": -0.3588251612988481, "operator": "ge"}, {"budget": 0.15, "threshold": -0.3680212036327043, "operator": "ge"}, {"budget": 0.16, "threshold": -0.377096262376382, "operator": "ge"}, {"budget": 0.17, "threshold": -0.38519215180196265, "operator": "ge"}, {"budget": 0.18, "threshold": -0.392769508422161, "operator": "ge"}, {"budget": 0.19, "threshold": -0.40140163667110373, "operator": "ge"}, {"budget": 0.2, "threshold": -0.40926457366717517, "operator": "ge"}, {"budget": 0.21, "threshold": -0.41643420416462235, "operator": "ge"}, {"budget": 0.22, "threshold": -0.42519244179862387, "operator": "ge"}, {"budget": 0.23, "threshold": -0.43308499550336227, "operator": "ge"}, {"budget": 0.24, "threshold": -0.4414087173648011, "operator": "ge"}, {"budget": 0.25, "threshold": -0.44972918643664633, "operator": "ge"}, {"budget": 0.26, "threshold": -0.458576756795169, "operator": "ge"}, {"budget": 0.27, "threshold": -0.46536279294614463, "operator": "ge"}, {"budget": 0.28, "threshold": -0.47256444743916953, "operator": "ge"}, {"budget": 0.29, "threshold": -0.48032365715937386, "operator": "ge"}, {"budget": 0.3, "threshold": -0.4882664966260909, "operator": "ge"}, {"budget": 0.31, "threshold": -0.4962547654764027, "operator": "ge"}, {"budget": 0.32, "threshold": -0.50276721633463, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5096783035489058, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5185147075549561, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5258226175989577, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5343966107436507, "operator": "ge"}, {"budget": 0.37, "threshold": -0.5434482034380096, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5534691418907961, "operator": "ge"}, {"budget": 0.39, "threshold": -0.5611505526452428, "operator": "ge"}, {"budget": 0.4, "threshold": -0.5702754189623751, "operator": "ge"}, {"budget": 0.41, "threshold": -0.5793145746785744, "operator": "ge"}, {"budget": 0.42, "threshold": -0.5868461350859472, "operator": "ge"}, {"budget": 0.43, "threshold": -0.5948740080150086, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6053459169524069, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6156803703077974, "operator": "ge"}, {"budget": 0.46, "threshold": -0.6253137408782871, "operator": "ge"}, {"budget": 0.47, "threshold": -0.6328405503838228, "operator": "ge"}, {"budget": 0.48, "threshold": -0.6405057829447616, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6496223255972319, "operator": "ge"}, {"budget": 0.5, "threshold": -0.6571980679632631, "operator": "ge"}, {"budget": 0.51, "threshold": -0.6647775744720175, "operator": "ge"}, {"budget": 0.52, "threshold": -0.6723582963705649, "operator": "ge"}, {"budget": 0.53, "threshold": -0.6804921247130873, "operator": "ge"}, {"budget": 0.54, "threshold": -0.6889174445269806, "operator": "ge"}, {"budget": 0.55, "threshold": -0.6967371366436446, "operator": "ge"}, {"budget": 0.56, "threshold": -0.7043582573826809, "operator": "ge"}, {"budget": 0.57, "threshold": -0.7116377568308426, "operator": "ge"}, {"budget": 0.58, "threshold": -0.7189510794739915, "operator": "ge"}, {"budget": 0.59, "threshold": -0.7254834042769933, "operator": "ge"}, {"budget": 0.6, "threshold": -0.7323229932115835, "operator": "ge"}, {"budget": 0.61, "threshold": -0.7395537902549234, "operator": "ge"}, {"budget": 0.62, "threshold": -0.7458619374538089, "operator": "ge"}, {"budget": 0.63, "threshold": -0.752431821867258, "operator": "ge"}, {"budget": 0.64, "threshold": -0.7594243963545357, "operator": "ge"}, {"budget": 0.65, "threshold": -0.7655220446864961, "operator": "ge"}, {"budget": 0.66, "threshold": -0.7712470594217844, "operator": "ge"}, {"budget": 0.67, "threshold": -0.7763609597879539, "operator": "ge"}, {"budget": 0.68, "threshold": -0.7822733497680585, "operator": "ge"}, {"budget": 0.69, "threshold": -0.7876052322639031, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7921969174088306, "operator": "gt"}, {"budget": 0.71, "threshold": -0.797504072028512, "operator": "ge"}, {"budget": 0.72, "threshold": -0.8019365169882847, "operator": "ge"}, {"budget": 0.73, "threshold": -0.8067008835546355, "operator": "ge"}, {"budget": 0.74, "threshold": -0.8120656031395171, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8175924784183914, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8211732357171361, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8255077776895967, "operator": "gt"}, {"budget": 0.78, "threshold": -0.8300537921587301, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8344600645461908, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8402630803872795, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8450329017851816, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8501622122410145, "operator": "ge"}, {"budget": 0.83, "threshold": -0.856017809789283, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8612831700679914, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8665767602427753, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8719279107050942, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8780261353394161, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8836408279369372, "operator": "ge"}, {"budget": 0.89, "threshold": -0.8895834111472646, "operator": "ge"}, {"budget": 0.9, "threshold": -0.8950673704828713, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9009201839823565, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9068579881345127, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9136724839338809, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9186861211482547, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9260625946469194, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9329092380999863, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9410042983858636, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9494798012594147, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9582785053142033, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.32504041042941, 75.4937100288144, 75.65535174643334, 75.59912854030502, 75.53587743341063, 75.43045892192002, 75.43748682268607, 75.50776583034647, 75.54290533417668, 75.55696113570876, 75.54993323494273, 75.57101693724084, 75.53587743341063, 75.52182163187857, 75.5288495326446, 75.54993323494273, 75.54290533417668, 75.55696113570876, 75.56398903647481, 75.55696113570876, 75.54993323494273, 75.55696113570876, 75.55696113570876, 75.54993323494273, 75.52182163187857, 75.4585705249842, 75.41640312038794, 75.39531941808981, 75.39531941808981, 75.3742357157917, 75.34612411272754, 75.32504041042941, 75.31098460889733, 75.254761402769, 75.2055660974067, 75.19853819664067, 75.14231499051233, 75.10014758591609, 75.1212312882142, 75.07203598285193, 75.0509522805538, 75.02284067748963, 75.00878487595756, 74.98067327289338, 74.8963384637009, 74.87525476140277, 74.74172464684798, 74.7276688453159, 74.74875254761403, 74.67847353995361, 74.58008292922904, 74.53088762386675, 74.49574812003654, 74.48169231850447, 74.38330170777989, 74.34113430318364, 74.25679949399114, 74.23571579169302, 74.16543678403261, 74.13029728020241, 74.10921357790427, 74.03190666947783, 73.9756834633495, 73.91946025722117, 73.94054395951929, 73.92648815798721, 73.70862323423994, 73.68051163117578, 73.66645582964368, 73.66645582964368, 73.60320472274931, 73.52589781432286, 73.4134514020662, 73.37128399746996, 73.32208869210767, 73.25180968444725, 73.06908426453018, 72.99177735610374, 72.82310773771874, 72.63335441703563, 72.47171269941668, 72.29601518026566, 72.07815025651838, 71.7900063251107, 71.501862393703, 71.17857895846511, 70.8342118209291, 70.44064937803078, 70.01897533206831, 69.53405017921148, 69.09832033171692, 68.70475788881862, 68.19172113289761, 67.68571227774264, 67.26403823178016, 66.6736945674327, 66.09740670461733, 65.6124815517605, 64.91671937592241, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.99086372900415, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.99065289198117, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 46, 71, 95, 108, 122, 137, 150, 164, 174, 183, 190, 199, 202, 208, 215, 226, 230, 234, 240, 243, 246, 250, 254, 257, 260, 266, 272, 278, 287, 291, 296, 306, 311, 315, 317, 327, 334, 341, 355, 363, 371, 381, 395, 407, 415, 427, 434, 454, 470, 481, 491, 511, 532, 556, 569, 588, 602, 621, 641, 664, 689, 707, 730, 753, 783, 808, 823, 847, 873, 901, 925, 949, 971, 998, 1020, 1037, 1055, 1079, 1099, 1117, 1136, 1155, 1178, 1193, 1205, 1218, 1234, 1245, 1258, 1270, 1284, 1299, 1308, 1320, 1335, 1343, 1352, 1366, 1369, 1369], "harmed": [0, 4, 5, 6, 27, 50, 80, 92, 96, 101, 108, 116, 122, 130, 138, 144, 152, 157, 159, 164, 168, 172, 175, 179, 183, 190, 205, 217, 226, 235, 242, 251, 264, 271, 283, 292, 303, 318, 331, 342, 357, 368, 382, 398, 414, 434, 449, 475, 497, 510, 531, 555, 582, 608, 634, 661, 686, 712, 734, 764, 792, 820, 849, 880, 911, 938, 965, 1011, 1039, 1067, 1095, 1128, 1163, 1201, 1234, 1263, 1290, 1334, 1369, 1413, 1458, 1500, 1544, 1598, 1654, 1707, 1766, 1831, 1898, 1971, 2052, 2128, 2199, 2281, 2365, 2440, 2532, 2623, 2706, 2808, 2921], "exact_score": [75.02986857825567, 75.32504041042941, 75.4937100288144, 75.65535174643334, 75.59912854030502, 75.53587743341063, 75.43045892192002, 75.43748682268607, 75.50776583034647, 75.54290533417668, 75.55696113570876, 75.54993323494273, 75.57101693724084, 75.53587743341063, 75.52182163187857, 75.5288495326446, 75.54993323494273, 75.54290533417668, 75.55696113570876, 75.56398903647481, 75.55696113570876, 75.54993323494273, 75.55696113570876, 75.55696113570876, 75.54993323494273, 75.52182163187857, 75.4585705249842, 75.41640312038794, 75.39531941808981, 75.39531941808981, 75.3742357157917, 75.34612411272754, 75.32504041042941, 75.31098460889733, 75.254761402769, 75.2055660974067, 75.19853819664067, 75.14231499051233, 75.10014758591609, 75.1212312882142, 75.07203598285193, 75.0509522805538, 75.02284067748963, 75.00878487595756, 74.98067327289338, 74.8963384637009, 74.87525476140277, 74.74172464684798, 74.7276688453159, 74.74875254761403, 74.67847353995361, 74.58008292922904, 74.53088762386675, 74.49574812003654, 74.48169231850447, 74.38330170777989, 74.34113430318364, 74.25679949399114, 74.23571579169302, 74.16543678403261, 74.13029728020241, 74.10921357790427, 74.03190666947783, 73.9756834633495, 73.91946025722117, 73.94054395951929, 73.92648815798721, 73.70862323423994, 73.68051163117578, 73.66645582964368, 73.65942792887765, 73.60320472274931, 73.52589781432286, 73.4134514020662, 73.37128399746996, 73.32208869210767, 73.25180968444725, 73.06908426453018, 72.99177735610374, 72.82310773771874, 72.63335441703563, 72.47171269941668, 72.29601518026566, 72.07815025651838, 71.7900063251107, 71.501862393703, 71.17857895846511, 70.8342118209291, 70.44064937803078, 70.01897533206831, 69.53405017921148, 69.09832033171692, 68.70475788881862, 68.19172113289761, 67.68571227774264, 67.26403823178016, 66.6736945674327, 66.09740670461733, 65.6124815517605, 64.91671937592241, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 73.66296855061084, "threshold_area100": 73.6634338323143, "threshold_area25": 75.49089886850798}, "test": {"score": [66.95119530385956, 67.51269035532995, 67.7650795451323, 67.86716955449053, 67.93239372713609, 68.01746873493462, 68.06284207242719, 68.0429912372742, 68.05149873805405, 68.0600062388339, 68.06284207242719, 68.05433457164733, 68.03164790290106, 68.0032895669682, 67.97209539744208, 67.96358789666222, 67.92105039276295, 67.89836372401668, 67.88134872245696, 67.87000538808383, 67.84731871933755, 67.81612454981142, 67.80194538184499, 67.782094546692, 67.75940787794572, 67.7083628732666, 67.67716870374046, 67.65731786858747, 67.62045203187476, 67.60910869750163, 67.58075036156879, 67.58075036156879, 67.54388452485608, 67.50985452173667, 67.49000368658366, 67.43045118112468, 67.3510478405127, 67.29433116864702, 67.26597283271417, 67.2432861639679, 67.16388282335592, 67.12701698664321, 67.02776281087826, 66.97955363979241, 66.86612029606103, 66.82358279216174, 66.76119445310948, 66.6846269460908, 66.57119360235941, 66.41805858832204, 66.30178941099736, 66.21387856960553, 66.09477355868756, 65.98984771573603, 65.82536936732552, 65.72611519156055, 65.58432351189633, 65.45387516660523, 65.24402348070215, 65.06253013073191, 64.90372344950798, 64.71655843235119, 64.47834841051528, 64.28834755976519, 64.06998837308227, 63.89133085670533, 63.681479170802255, 63.46311998411933, 63.258939965402824, 63.05192411309304, 62.83356492641011, 62.50744406318237, 62.25789070697332, 62.00833735076425, 61.64818648441709, 61.28803561806993, 61.00728809233475, 60.7180330658197, 60.394748036185234, 60.01758216827836, 59.54966962538638, 59.24623543090492, 58.86056206221819, 58.395485352919486, 57.90772197487451, 57.38876442730341, 56.95204605393756, 56.51816351416499, 56.10413180954541, 55.52278592292203, 55.080395882369615, 54.64651334259706, 54.19277996767149, 53.70501658962652, 53.31083572015993, 52.777699004622406, 52.26724895783115, 51.70858973995406, 51.093213850211264, 50.64798797606556, 50.00708958398321], "calls": [0.0, 3.3491194736692846, 6.003459716983808, 8.969741655559652, 10.61168930607152, 11.89632192382951, 13.243342880639764, 14.44290049059921, 16.382610668405977, 17.695601622096817, 18.770382553951734, 19.692028471769277, 20.469046876329298, 21.05039276295267, 21.745171993307434, 22.35771204945694, 22.919207100927316, 23.65652383518135, 24.2236905538383, 24.966678955278905, 25.596234012988116, 26.137878229305507, 26.668179111249753, 27.11624081898874, 27.575645861100874, 28.02937923602643, 28.502963446104985, 28.908487649944703, 29.308340186597846, 29.78759606386297, 30.272523608314668, 30.910586166803732, 31.355812040949438, 31.846411252587696, 32.53835464934917, 33.062983864106855, 33.74074809290191, 34.390153985764115, 35.18135155829056, 35.87045912145875, 36.59926835493293, 37.35076425715339, 38.03136431954173, 38.717636049116635, 39.613759464594615, 40.57794288631143, 41.53361880724839, 42.37586138445396, 43.21810396165953, 44.2078098857159, 45.03870912854834, 45.96886254714573, 46.97274763916853, 48.02767773587046, 49.23290701301647, 50.3984346198565, 51.53843972435698, 52.71247483197686, 53.841136602104186, 55.032186711283785, 56.12398264469841, 57.24413691404588, 58.37279868417321, 59.48728128633412, 60.8484814111108, 62.01117318435754, 63.080282449025894, 64.047301704336, 65.26103848226185, 66.34149108130335, 67.373734509259, 68.65269545983041, 69.6792672205995, 70.76822732042083, 72.07838244051838, 73.33749255593682, 74.2903326432805, 75.33108357201598, 76.49377534526273, 77.71885545756176, 79.23035476278251, 80.38170320165612, 81.60111164676856, 82.84604259422056, 84.09380937526586, 85.28485948444545, 86.36531208348694, 87.51949635595383, 88.60562062218189, 89.6832373876301, 90.6871224796529, 91.614440064657, 92.58145931996711, 93.61370274792276, 94.35385531577008, 95.38609874372573, 96.30490882794997, 97.27759975044664, 98.32402234636871, 99.03865241187647, 100.0], "rescued": [0, 264, 390, 458, 542, 634, 687, 703, 728, 749, 770, 794, 807, 813, 822, 837, 843, 851, 859, 873, 882, 887, 895, 905, 910, 916, 925, 936, 942, 954, 959, 978, 986, 995, 1010, 1015, 1025, 1034, 1055, 1075, 1085, 1108, 1120, 1139, 1159, 1187, 1207, 1228, 1249, 1260, 1273, 1291, 1317, 1342, 1369, 1407, 1443, 1475, 1500, 1526, 1557, 1590, 1610, 1636, 1686, 1727, 1762, 1787, 1827, 1865, 1895, 1937, 1974, 2000, 2032, 2059, 2083, 2110, 2148, 2189, 2233, 2269, 2298, 2322, 2352, 2373, 2398, 2427, 2458, 2469, 2489, 2505, 2522, 2541, 2552, 2569, 2585, 2593, 2609, 2622, 2627], "harmed": [0, 66, 103, 135, 196, 258, 295, 318, 340, 358, 378, 405, 426, 442, 462, 480, 501, 517, 531, 549, 566, 582, 595, 612, 625, 649, 669, 687, 706, 722, 737, 756, 777, 798, 820, 846, 884, 913, 944, 972, 1010, 1046, 1093, 1129, 1189, 1232, 1274, 1322, 1383, 1448, 1502, 1551, 1619, 1681, 1766, 1839, 1925, 2003, 2102, 2192, 2279, 2378, 2482, 2575, 2702, 2806, 2915, 3017, 3129, 3240, 3347, 3504, 3629, 3743, 3902, 4056, 4179, 4308, 4460, 4634, 4843, 4986, 5151, 5339, 5541, 5745, 5924, 6106, 6283, 6499, 6675, 6844, 7021, 7212, 7362, 7567, 7763, 7968, 8201, 8371, 8602], "exact_score": [66.95119530385956, 67.19791282647533, 67.34254033973286, 67.45597368346425, 67.59492952953521, 67.68284037092704, 67.762243711539, 67.8189603834047, 67.83030371777784, 67.86149788730397, 67.92672205994951, 67.98911039900179, 68.0203045685279, 68.06284207242719, 68.04582707086749, 68.0429912372742, 68.04866290446077, 68.06284207242719, 68.06284207242719, 68.05717040524061, 68.03448373649434, 68.01463290134134, 67.97776706462865, 67.91821455916967, 67.8926920568301, 67.8728412216771, 67.8189603834047, 67.79627371465841, 67.71687037404645, 67.6516462014009, 67.59776536312849, 67.55522785922922, 67.51836202251653, 67.43612284831127, 67.34254033973286, 67.2773161670873, 67.24045033037461, 67.13836032101636, 67.02776281087826, 66.95403113745286, 66.83492612653488, 66.80373195700876, 66.70164194765051, 66.59671610469897, 66.45492442503473, 66.31596857896378, 66.21387856960553, 66.09193772509428, 65.99551938292261, 65.86223520403823, 65.74313019312027, 65.67790602047472, 65.54178600799705, 65.39715849473953, 65.19865014320959, 65.07103763151179, 64.91506678388112, 64.7676034370303, 64.56058758472054, 64.36775090037716, 64.21745171993307, 64.04446587074271, 63.89133085670533, 63.70133000595525, 63.47162748489919, 63.29296996852225, 63.122819952925155, 62.9101324334288, 62.6690865779996, 62.43654822335025, 62.19833820151434, 61.92893401015228, 61.682216487536515, 61.37594645946176, 61.0725122649803, 60.794600572838384, 60.53937554944275, 60.25862802370756, 59.9353429940731, 59.631908799591635, 59.34548960666988, 59.064742080934685, 58.70459121458753, 58.33309701386722, 57.936080310807355, 57.53339194056093, 57.09100190000851, 56.71950769928821, 56.32816266341491, 55.90845929160877, 55.39517341122423, 54.92442503473895, 54.47352749340668, 54.02262995207441, 53.515015738876436, 52.99605819130533, 52.43172730624167, 51.86172475399143, 51.311573036894195, 50.67634631199841, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.65964663779582, "threshold_area100": 63.872997192524736, "threshold_area25": 67.84419930238494}, "selected_budget": 3, "selected": {"val": {"score": 75.65535174643334, "calls": 2.993885726333544, "rescued": 95, "harmed": 6}, "test": {"score": 67.86716955449053, "calls": 8.969741655559652, "rescued": 458, "harmed": 135}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.1544547826051712, "operator": "ge"}, {"budget": 0.02, "threshold": 0.049897633492946625, "operator": "ge"}, {"budget": 0.03, "threshold": -0.003543814644217491, "operator": "ge"}, {"budget": 0.04, "threshold": -0.006108797620981932, "operator": "gt"}, {"budget": 0.05, "threshold": -0.008360672742128372, "operator": "ge"}, {"budget": 0.06, "threshold": -0.01049087755382061, "operator": "ge"}, {"budget": 0.07, "threshold": -0.01265284325927496, "operator": "ge"}, {"budget": 0.08, "threshold": -0.014772536233067513, "operator": "ge"}, {"budget": 0.09, "threshold": -0.016942724585533142, "operator": "ge"}, {"budget": 0.1, "threshold": -0.019172482192516327, "operator": "ge"}, {"budget": 0.11, "threshold": -0.02153279446065426, "operator": "ge"}, {"budget": 0.12, "threshold": -0.023471437394618988, "operator": "ge"}, {"budget": 0.13, "threshold": -0.02601846307516098, "operator": "ge"}, {"budget": 0.14, "threshold": -0.02845270000398159, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03105883114039898, "operator": "ge"}, {"budget": 0.16, "threshold": -0.03378230333328247, "operator": "ge"}, {"budget": 0.17, "threshold": -0.03673769533634186, "operator": "ge"}, {"budget": 0.18, "threshold": -0.04058026522397995, "operator": "ge"}, {"budget": 0.19, "threshold": -0.04423169046640396, "operator": "ge"}, {"budget": 0.2, "threshold": -0.04740248993039131, "operator": "ge"}, {"budget": 0.21, "threshold": -0.05141737684607506, "operator": "ge"}, {"budget": 0.22, "threshold": -0.05491285398602486, "operator": "ge"}, {"budget": 0.23, "threshold": -0.05950405076146126, "operator": "ge"}, {"budget": 0.24, "threshold": -0.06368502974510193, "operator": "ge"}, {"budget": 0.25, "threshold": -0.06790583580732346, "operator": "ge"}, {"budget": 0.26, "threshold": -0.07332099974155426, "operator": "ge"}, {"budget": 0.27, "threshold": -0.07789897918701172, "operator": "ge"}, {"budget": 0.28, "threshold": -0.08419009298086166, "operator": "ge"}, {"budget": 0.29, "threshold": -0.09098390489816666, "operator": "ge"}, {"budget": 0.3, "threshold": -0.09837404638528824, "operator": "ge"}, {"budget": 0.31, "threshold": -0.10657686740159988, "operator": "ge"}, {"budget": 0.32, "threshold": -0.11448478698730469, "operator": "ge"}, {"budget": 0.33, "threshold": -0.12209826707839966, "operator": "ge"}, {"budget": 0.34, "threshold": -0.13180333375930786, "operator": "ge"}, {"budget": 0.35, "threshold": -0.1426800787448883, "operator": "ge"}, {"budget": 0.36, "threshold": -0.15402351319789886, "operator": "ge"}, {"budget": 0.37, "threshold": -0.16350609064102173, "operator": "ge"}, {"budget": 0.38, "threshold": -0.17413140833377838, "operator": "ge"}, {"budget": 0.39, "threshold": -0.18648290634155273, "operator": "ge"}, {"budget": 0.4, "threshold": -0.1973411589860916, "operator": "ge"}, {"budget": 0.41, "threshold": -0.2077212780714035, "operator": "ge"}, {"budget": 0.42, "threshold": -0.21888631582260132, "operator": "ge"}, {"budget": 0.43, "threshold": -0.23326830565929413, "operator": "ge"}, {"budget": 0.44, "threshold": -0.24691236019134521, "operator": "ge"}, {"budget": 0.45, "threshold": -0.2577569782733917, "operator": "ge"}, {"budget": 0.46, "threshold": -0.26804840564727783, "operator": "ge"}, {"budget": 0.47, "threshold": -0.2805540859699249, "operator": "ge"}, {"budget": 0.48, "threshold": -0.29370710253715515, "operator": "ge"}, {"budget": 0.49, "threshold": -0.30668678879737854, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3173186480998993, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3289005160331726, "operator": "ge"}, {"budget": 0.52, "threshold": -0.341293603181839, "operator": "ge"}, {"budget": 0.53, "threshold": -0.35342422127723694, "operator": "ge"}, {"budget": 0.54, "threshold": -0.3631677031517029, "operator": "ge"}, {"budget": 0.55, "threshold": -0.37302538752555847, "operator": "ge"}, {"budget": 0.56, "threshold": -0.38629037141799927, "operator": "ge"}, {"budget": 0.57, "threshold": -0.39748141169548035, "operator": "ge"}, {"budget": 0.58, "threshold": -0.40845102071762085, "operator": "ge"}, {"budget": 0.59, "threshold": -0.42066720128059387, "operator": "ge"}, {"budget": 0.6, "threshold": -0.43041765689849854, "operator": "ge"}, {"budget": 0.61, "threshold": -0.44167637825012207, "operator": "ge"}, {"budget": 0.62, "threshold": -0.451485276222229, "operator": "ge"}, {"budget": 0.63, "threshold": -0.46064963936805725, "operator": "ge"}, {"budget": 0.64, "threshold": -0.4704032838344574, "operator": "ge"}, {"budget": 0.65, "threshold": -0.4793141186237335, "operator": "ge"}, {"budget": 0.66, "threshold": -0.48823514580726624, "operator": "ge"}, {"budget": 0.67, "threshold": -0.49745333194732666, "operator": "ge"}, {"budget": 0.68, "threshold": -0.5069347023963928, "operator": "ge"}, {"budget": 0.69, "threshold": -0.5177450180053711, "operator": "ge"}, {"budget": 0.7, "threshold": -0.5269182920455933, "operator": "ge"}, {"budget": 0.71, "threshold": -0.5359976887702942, "operator": "ge"}, {"budget": 0.72, "threshold": -0.5447730422019958, "operator": "ge"}, {"budget": 0.73, "threshold": -0.554571270942688, "operator": "ge"}, {"budget": 0.74, "threshold": -0.5648353695869446, "operator": "ge"}, {"budget": 0.75, "threshold": -0.5752351880073547, "operator": "ge"}, {"budget": 0.76, "threshold": -0.5857824087142944, "operator": "ge"}, {"budget": 0.77, "threshold": -0.5971100330352783, "operator": "ge"}, {"budget": 0.78, "threshold": -0.6073217988014221, "operator": "ge"}, {"budget": 0.79, "threshold": -0.6178173422813416, "operator": "ge"}, {"budget": 0.8, "threshold": -0.6286174654960632, "operator": "ge"}, {"budget": 0.81, "threshold": -0.6396207213401794, "operator": "ge"}, {"budget": 0.82, "threshold": -0.6501505970954895, "operator": "ge"}, {"budget": 0.83, "threshold": -0.6633895635604858, "operator": "ge"}, {"budget": 0.84, "threshold": -0.6751917004585266, "operator": "ge"}, {"budget": 0.85, "threshold": -0.6862177848815918, "operator": "ge"}, {"budget": 0.86, "threshold": -0.6979153156280518, "operator": "ge"}, {"budget": 0.87, "threshold": -0.7116177082061768, "operator": "ge"}, {"budget": 0.88, "threshold": -0.7259416580200195, "operator": "ge"}, {"budget": 0.89, "threshold": -0.7376711368560791, "operator": "ge"}, {"budget": 0.9, "threshold": -0.7527480721473694, "operator": "ge"}, {"budget": 0.91, "threshold": -0.7679901123046875, "operator": "ge"}, {"budget": 0.92, "threshold": -0.7834141254425049, "operator": "ge"}, {"budget": 0.93, "threshold": -0.7993448376655579, "operator": "ge"}, {"budget": 0.94, "threshold": -0.8135656118392944, "operator": "ge"}, {"budget": 0.95, "threshold": -0.8281911015510559, "operator": "ge"}, {"budget": 0.96, "threshold": -0.8468084931373596, "operator": "ge"}, {"budget": 0.97, "threshold": -0.8647508025169373, "operator": "ge"}, {"budget": 0.98, "threshold": -0.8856467604637146, "operator": "ge"}, {"budget": 0.99, "threshold": -0.908820629119873, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.33206831119544, 75.50776583034647, 75.64832384566729, 75.64832384566729, 75.63426804413523, 75.63426804413523, 75.6131843418371, 75.60615644107105, 75.57804483800689, 75.56398903647481, 75.5288495326446, 75.5288495326446, 75.5288495326446, 75.50776583034647, 75.50073792958042, 75.47965422728231, 75.45154262421815, 75.42343102115399, 75.36017991425962, 75.28287300583317, 75.24773350200296, 75.19853819664067, 75.16339869281046, 75.1212312882142, 75.09311968515004, 75.04392437978777, 75.00175697519151, 74.9666174713613, 74.90336636446693, 74.8471431583386, 74.79091995221027, 74.74875254761403, 74.69955724225173, 74.65036193688945, 74.65036193688945, 74.58711082999508, 74.52385972310071, 74.4887202192705, 74.46763651697238, 74.39735750931197, 74.32707850165156, 74.27085529552323, 74.22868789092698, 74.15138098250053, 74.10921357790427, 74.03190666947783, 73.94054395951929, 73.89134865415701, 73.87026495185887, 73.76484644036826, 73.68753953194181, 73.58914892121723, 73.56806521891912, 73.53995361585494, 73.54698151662099, 73.55400941738702, 73.54698151662099, 73.50481411202475, 73.45561880666244, 73.43453510436433, 73.42750720359828, 73.46264670742849, 73.46967460819454, 73.50481411202475, 73.53292571508891, 73.36425609670391, 73.33614449363975, 73.37128399746996, 73.25180968444725, 73.28694918827746, 73.29397708904351, 73.23775388291519, 73.10422376836038, 72.96366575303956, 72.81607983695271, 72.72471712699416, 72.57713121090731, 72.45062899711856, 72.34521048562794, 72.1343734626467, 71.8743411343032, 71.57916930212946, 71.26994166842364, 71.04504884391034, 70.820156019397, 70.4968725841591, 70.25089605734766, 69.92761262210978, 69.62541288917001, 69.28104575163398, 68.90856701103381, 68.46580926277322, 68.08630262140699, 67.61543327008222, 67.22187082718392, 66.7580293766252, 66.17471361304379, 65.60545365099445, 64.86049616979408, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.9918476351113923, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 46, 73, 94, 94, 94, 95, 95, 95, 95, 95, 95, 96, 97, 98, 98, 98, 98, 98, 98, 98, 99, 99, 101, 102, 103, 106, 106, 107, 111, 115, 118, 120, 125, 131, 140, 147, 155, 164, 171, 181, 194, 207, 224, 236, 245, 261, 280, 295, 316, 336, 350, 370, 389, 414, 439, 466, 492, 519, 539, 557, 583, 612, 635, 659, 686, 704, 728, 754, 777, 808, 838, 861, 886, 911, 934, 957, 982, 1010, 1038, 1059, 1085, 1104, 1124, 1147, 1174, 1194, 1218, 1239, 1260, 1275, 1291, 1306, 1317, 1327, 1343, 1352, 1361, 1368, 1369, 1369], "harmed": [0, 3, 5, 6, 6, 8, 9, 12, 13, 17, 19, 24, 25, 26, 30, 31, 34, 38, 42, 51, 62, 68, 75, 82, 89, 94, 104, 110, 116, 129, 141, 152, 160, 172, 185, 194, 210, 227, 241, 251, 271, 294, 315, 338, 361, 376, 403, 435, 457, 481, 516, 541, 575, 597, 626, 650, 676, 703, 736, 763, 784, 811, 835, 857, 876, 899, 941, 969, 990, 1030, 1056, 1085, 1116, 1160, 1205, 1249, 1285, 1331, 1377, 1420, 1471, 1534, 1595, 1659, 1714, 1773, 1839, 1898, 1965, 2029, 2093, 2162, 2240, 2305, 2382, 2454, 2529, 2621, 2709, 2816, 2921], "exact_score": [75.02986857825567, 75.33206831119544, 75.50776583034647, 75.64832384566729, 75.64832384566729, 75.63426804413523, 75.63426804413523, 75.6131843418371, 75.60615644107105, 75.57804483800689, 75.56398903647481, 75.5288495326446, 75.5288495326446, 75.5288495326446, 75.50776583034647, 75.50073792958042, 75.47965422728231, 75.45154262421815, 75.42343102115399, 75.36017991425962, 75.28287300583317, 75.24773350200296, 75.19853819664067, 75.16339869281046, 75.1212312882142, 75.09311968515004, 75.04392437978777, 75.00175697519151, 74.9666174713613, 74.90336636446693, 74.8471431583386, 74.79091995221027, 74.74875254761403, 74.69955724225173, 74.65036193688945, 74.65036193688945, 74.58711082999508, 74.52385972310071, 74.4887202192705, 74.46763651697238, 74.39735750931197, 74.32707850165156, 74.27085529552323, 74.22868789092698, 74.15138098250053, 74.10921357790427, 74.03190666947783, 73.94054395951929, 73.89134865415701, 73.87026495185887, 73.76484644036826, 73.68753953194181, 73.58914892121723, 73.56806521891912, 73.53995361585494, 73.54698151662099, 73.55400941738702, 73.54698151662099, 73.50481411202475, 73.45561880666244, 73.43453510436433, 73.42750720359828, 73.46264670742849, 73.46967460819454, 73.50481411202475, 73.53292571508891, 73.36425609670391, 73.33614449363975, 73.37128399746996, 73.25180968444725, 73.28694918827746, 73.29397708904351, 73.23775388291519, 73.10422376836038, 72.96366575303956, 72.81607983695271, 72.72471712699416, 72.57713121090731, 72.45062899711856, 72.34521048562794, 72.1343734626467, 71.8743411343032, 71.57916930212946, 71.26994166842364, 71.04504884391034, 70.820156019397, 70.4968725841591, 70.25089605734766, 69.92761262210978, 69.62541288917001, 69.28104575163398, 68.90856701103381, 68.46580926277322, 68.08630262140699, 67.61543327008222, 67.22187082718392, 66.7580293766252, 66.17471361304379, 65.60545365099445, 64.86049616979408, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 73.2402506176381, "threshold_area100": 73.24063532222924, "threshold_area25": 75.4086724295453}, "test": {"score": [66.95119530385956, 67.50701868814339, 67.7650795451323, 67.85015455293083, 67.84448288574426, 67.84448288574426, 67.84164705215098, 67.83597538496441, 67.83597538496441, 67.83313955137112, 67.82463205059128, 67.81328871621814, 67.81328871621814, 67.80761704903156, 67.79343788106513, 67.78776621387857, 67.77075121231886, 67.77358704591214, 67.762243711539, 67.75090037716586, 67.74806454357258, 67.72254204123301, 67.72254204123301, 67.69701953889346, 67.67716870374046, 67.6714970365539, 67.66015370218076, 67.64597453421433, 67.6346311998412, 67.59209369594193, 67.58075036156879, 67.56940702719564, 67.52403368970309, 67.43612284831127, 67.4077645123784, 67.36806284207243, 67.34537617332614, 67.28298783427388, 67.2234353288149, 67.1497036553895, 67.0759719819641, 66.94835947026628, 66.86895612965431, 66.79238862263563, 66.6846269460908, 66.59104443751241, 66.46626775940787, 66.29044607662422, 66.14298272977341, 66.00969855088903, 65.86507103763151, 65.70910019000084, 65.55880100955676, 65.36312849162012, 65.20432181039617, 65.05402262995207, 64.84984261123557, 64.64566259251907, 64.41312423786972, 64.14939171369424, 63.96506253013074, 63.76088251141423, 63.52267248957831, 63.29864163570882, 63.08595411621246, 62.881774097495956, 62.586847403794344, 62.33162238039872, 62.04520318747696, 61.78430649689476, 61.59714147973797, 61.34475228993562, 61.10937810169299, 60.763406403312246, 60.5138530471032, 60.221762186994866, 59.88146215580069, 59.58086379491251, 59.30862376995718, 58.99668207469586, 58.70459121458753, 58.256529506848544, 57.91622947565437, 57.496526103848225, 57.28100275075858, 56.94070271956442, 56.62025352352323, 56.192042650937246, 55.74114510960497, 55.36397924169809, 54.867708362873266, 54.38845248560814, 53.96024161302214, 53.45829906701074, 53.00740152567847, 52.579190653092475, 52.06590477270794, 51.59799222981596, 51.050676346311995, 50.514703797181184, 50.00708958398321], "calls": [0.0, 3.252701131497604, 6.011967217763662, 8.969741655559652, 10.89810849899328, 12.574086152624563, 13.549612908714517, 14.176332132830446, 14.649916342908998, 15.114993052207696, 15.568726427133257, 16.141564812976775, 16.626492357428464, 17.20216657686527, 17.695601622096817, 18.143663329835803, 18.509485863369537, 18.87530839690327, 19.272325099963133, 19.6324759663103, 19.94441766157162, 20.330091030258345, 20.71576439894507, 21.132631937157928, 21.572186144117065, 22.020247851856052, 22.564727901766723, 23.024132943878854, 23.65652383518135, 24.21801888665173, 24.907126449819923, 25.618920681734398, 26.26265490741003, 26.948926636984943, 27.69475087201883, 28.548336783597538, 29.5408785412472, 30.434166123131895, 31.347304540169585, 32.41641380483794, 33.425970564047304, 34.47806482715594, 35.564189093384, 36.922553384567394, 38.28942517653064, 39.35286277401242, 40.38227036837478, 41.666902986132776, 43.030938944502736, 44.27870572554802, 45.2684116496044, 46.34035674786603, 47.588123528911325, 48.86708447948274, 49.8596262371324, 50.79828715650966, 52.091427275047494, 53.31367155375323, 54.49621416215297, 55.82054845021695, 56.966225221903976, 58.27921617559482, 59.45041544962142, 60.547883050222616, 61.54893230865213, 62.66625074440631, 63.877151688738905, 65.20148597680289, 66.38970025238919, 67.6885120381136, 68.78030797152823, 69.85792473697643, 70.84763066103281, 72.10106910926467, 73.26092504891812, 74.49167682840371, 75.6628761024303, 76.79437370615092, 77.89467714034541, 79.06020474718544, 80.07543317358137, 81.24379661401468, 82.2278308708845, 83.36500014179168, 84.29515356038908, 85.24515781413947, 86.20366956866971, 87.32665967161047, 88.42129143861838, 89.32875818846951, 90.51130079686925, 91.63145506621672, 92.71757933244477, 93.66191191900859, 94.56654283526643, 95.44281541559141, 96.44670050761421, 97.31730142075263, 98.25029067294331, 99.17477242435415, 100.0], "rescued": [0, 259, 389, 444, 447, 451, 455, 456, 459, 460, 462, 464, 467, 469, 471, 473, 474, 478, 480, 483, 488, 490, 495, 496, 501, 510, 514, 521, 526, 532, 544, 558, 564, 571, 592, 612, 635, 650, 668, 695, 718, 742, 759, 808, 846, 877, 904, 930, 969, 1003, 1036, 1062, 1100, 1134, 1163, 1189, 1225, 1264, 1297, 1341, 1375, 1421, 1457, 1494, 1517, 1559, 1603, 1650, 1693, 1734, 1779, 1814, 1849, 1893, 1939, 1981, 2028, 2066, 2113, 2158, 2196, 2221, 2248, 2276, 2317, 2341, 2374, 2400, 2432, 2450, 2481, 2503, 2538, 2548, 2561, 2580, 2595, 2605, 2616, 2624, 2627], "harmed": [0, 63, 102, 127, 132, 136, 141, 144, 147, 149, 154, 160, 163, 167, 174, 178, 185, 188, 194, 201, 207, 218, 223, 233, 245, 256, 264, 276, 285, 306, 322, 340, 362, 400, 431, 465, 496, 533, 572, 625, 674, 743, 788, 864, 940, 1004, 1075, 1163, 1254, 1335, 1419, 1500, 1591, 1694, 1779, 1858, 1966, 2077, 2192, 2329, 2428, 2546, 2666, 2782, 2880, 2994, 3142, 3279, 3423, 3556, 3667, 3791, 3909, 4075, 4209, 4354, 4521, 4665, 4808, 4963, 5104, 5287, 5434, 5610, 5727, 5871, 6017, 6194, 6385, 6536, 6742, 6933, 7119, 7306, 7478, 7648, 7844, 8019, 8223, 8420, 8602], "exact_score": [66.95119530385956, 67.20074866006863, 67.34821200691943, 67.45313784987097, 67.59776536312849, 67.69701953889346, 67.762243711539, 67.81045288262484, 67.82746788418456, 67.85015455293083, 67.8558262201174, 67.84448288574426, 67.84731871933755, 67.84164705215098, 67.83313955137112, 67.83313955137112, 67.8189603834047, 67.80761704903156, 67.78776621387857, 67.77075121231886, 67.7452287099793, 67.71687037404645, 67.6714970365539, 67.64313870062104, 67.61761619828148, 67.58358619516207, 67.53537702407623, 67.43612284831127, 67.39642117800527, 67.353883674106, 67.32268950457987, 67.24612199756118, 67.18373365850891, 67.10716615149023, 67.00791197572526, 66.91716530074015, 66.8150752913819, 66.79238862263563, 66.71298528202365, 66.61940277344526, 66.52582026486685, 66.39253608598247, 66.24790857272494, 66.1458185633667, 66.0352210532286, 65.89626520715764, 65.72895102515383, 65.62969684938888, 65.51059183847092, 65.33193432209397, 65.18730680883645, 65.00864929245952, 64.86685761279529, 64.71088676516462, 64.50387091285484, 64.32804923007119, 64.14088421291439, 63.979241698097155, 63.83177835124635, 63.60491166378357, 63.41491081303349, 63.21356662791027, 62.98669994044749, 62.81087825766384, 62.56416073504807, 62.36848821711142, 62.113263193715795, 61.917590675779145, 61.727589825029064, 61.55176814224541, 61.313558120409496, 61.058333097013865, 60.788928905651815, 60.56773388537561, 60.357882199472535, 60.0856421745172, 59.824745483935004, 59.53832629101324, 59.29444460199076, 59.00518957547571, 58.727277883333805, 58.33309701386722, 58.01264781782605, 57.6127952811729, 57.343391089810844, 57.03995689532938, 56.69114936335535, 56.29696849388878, 55.92831012676176, 55.49442758698919, 55.08890338314948, 54.64651334259706, 54.23815330516405, 53.832629101324336, 53.273969883447236, 52.78620650540227, 52.29844312735729, 51.79650058134588, 51.178288858009815, 50.60828630575958, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.22006782246306, "threshold_area100": 63.44461616992315, "threshold_area25": 67.70325837279869}, "selected_budget": 3, "selected": {"val": {"score": 75.64832384566729, "calls": 2.993885726333544, "rescued": 94, "harmed": 6}, "test": {"score": 67.85015455293083, "calls": 8.969741655559652, "rescued": 444, "harmed": 127}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.5972986221313477, "operator": "ge"}, {"budget": 0.02, "threshold": 1.4712660312652588, "operator": "ge"}, {"budget": 0.03, "threshold": -0.1282534897327423, "operator": "ge"}, {"budget": 0.04, "threshold": -0.8709152340888977, "operator": "ge"}, {"budget": 0.05, "threshold": -1.439894199371338, "operator": "ge"}, {"budget": 0.06, "threshold": -1.8395394086837769, "operator": "ge"}, {"budget": 0.07, "threshold": -2.1475515365600586, "operator": "ge"}, {"budget": 0.08, "threshold": -2.454209327697754, "operator": "ge"}, {"budget": 0.09, "threshold": -2.731010913848877, "operator": "ge"}, {"budget": 0.1, "threshold": -2.9716310501098633, "operator": "ge"}, {"budget": 0.11, "threshold": -3.195523262023926, "operator": "ge"}, {"budget": 0.12, "threshold": -3.398465156555176, "operator": "ge"}, {"budget": 0.13, "threshold": -3.5626206398010254, "operator": "ge"}, {"budget": 0.14, "threshold": -3.722533702850342, "operator": "ge"}, {"budget": 0.15, "threshold": -3.8939318656921387, "operator": "ge"}, {"budget": 0.16, "threshold": -4.039223670959473, "operator": "ge"}, {"budget": 0.17, "threshold": -4.176375865936279, "operator": "ge"}, {"budget": 0.18, "threshold": -4.314467906951904, "operator": "ge"}, {"budget": 0.19, "threshold": -4.461513519287109, "operator": "ge"}, {"budget": 0.2, "threshold": -4.572041988372803, "operator": "ge"}, {"budget": 0.21, "threshold": -4.695913314819336, "operator": "ge"}, {"budget": 0.22, "threshold": -4.825379371643066, "operator": "ge"}, {"budget": 0.23, "threshold": -4.935969352722168, "operator": "ge"}, {"budget": 0.24, "threshold": -5.042302131652832, "operator": "ge"}, {"budget": 0.25, "threshold": -5.143505096435547, "operator": "ge"}, {"budget": 0.26, "threshold": -5.243686676025391, "operator": "ge"}, {"budget": 0.27, "threshold": -5.340150833129883, "operator": "ge"}, {"budget": 0.28, "threshold": -5.435124397277832, "operator": "ge"}, {"budget": 0.29, "threshold": -5.52886962890625, "operator": "ge"}, {"budget": 0.3, "threshold": -5.626001834869385, "operator": "ge"}, {"budget": 0.31, "threshold": -5.7072248458862305, "operator": "ge"}, {"budget": 0.32, "threshold": -5.790672779083252, "operator": "ge"}, {"budget": 0.33, "threshold": -5.871399402618408, "operator": "ge"}, {"budget": 0.34, "threshold": -5.949779987335205, "operator": "ge"}, {"budget": 0.35, "threshold": -6.026626110076904, "operator": "ge"}, {"budget": 0.36, "threshold": -6.103729248046875, "operator": "ge"}, {"budget": 0.37, "threshold": -6.184317588806152, "operator": "ge"}, {"budget": 0.38, "threshold": -6.268381595611572, "operator": "ge"}, {"budget": 0.39, "threshold": -6.330653190612793, "operator": "ge"}, {"budget": 0.4, "threshold": -6.3957085609436035, "operator": "gt"}, {"budget": 0.41, "threshold": -6.464390754699707, "operator": "ge"}, {"budget": 0.42, "threshold": -6.539183616638184, "operator": "ge"}, {"budget": 0.43, "threshold": -6.612374305725098, "operator": "ge"}, {"budget": 0.44, "threshold": -6.68431282043457, "operator": "ge"}, {"budget": 0.45, "threshold": -6.750932216644287, "operator": "ge"}, {"budget": 0.46, "threshold": -6.821629047393799, "operator": "ge"}, {"budget": 0.47, "threshold": -6.890283584594727, "operator": "ge"}, {"budget": 0.48, "threshold": -6.95211124420166, "operator": "ge"}, {"budget": 0.49, "threshold": -7.0222907066345215, "operator": "ge"}, {"budget": 0.5, "threshold": -7.090860366821289, "operator": "ge"}, {"budget": 0.51, "threshold": -7.16093635559082, "operator": "ge"}, {"budget": 0.52, "threshold": -7.226806163787842, "operator": "ge"}, {"budget": 0.53, "threshold": -7.299644470214844, "operator": "ge"}, {"budget": 0.54, "threshold": -7.369707107543945, "operator": "ge"}, {"budget": 0.55, "threshold": -7.426769256591797, "operator": "ge"}, {"budget": 0.56, "threshold": -7.4936628341674805, "operator": "ge"}, {"budget": 0.57, "threshold": -7.559700965881348, "operator": "ge"}, {"budget": 0.58, "threshold": -7.621016502380371, "operator": "ge"}, {"budget": 0.59, "threshold": -7.686408996582031, "operator": "ge"}, {"budget": 0.6, "threshold": -7.744324684143066, "operator": "ge"}, {"budget": 0.61, "threshold": -7.802737236022949, "operator": "ge"}, {"budget": 0.62, "threshold": -7.864293098449707, "operator": "ge"}, {"budget": 0.63, "threshold": -7.924903869628906, "operator": "ge"}, {"budget": 0.64, "threshold": -7.987915992736816, "operator": "ge"}, {"budget": 0.65, "threshold": -8.04612922668457, "operator": "ge"}, {"budget": 0.66, "threshold": -8.111651420593262, "operator": "ge"}, {"budget": 0.67, "threshold": -8.170337677001953, "operator": "ge"}, {"budget": 0.68, "threshold": -8.223731994628906, "operator": "ge"}, {"budget": 0.69, "threshold": -8.277022361755371, "operator": "ge"}, {"budget": 0.7, "threshold": -8.328353881835938, "operator": "ge"}, {"budget": 0.71, "threshold": -8.381696701049805, "operator": "gt"}, {"budget": 0.72, "threshold": -8.430950164794922, "operator": "ge"}, {"budget": 0.73, "threshold": -8.481253623962402, "operator": "ge"}, {"budget": 0.74, "threshold": -8.533804893493652, "operator": "ge"}, {"budget": 0.75, "threshold": -8.584084510803223, "operator": "ge"}, {"budget": 0.76, "threshold": -8.635817527770996, "operator": "ge"}, {"budget": 0.77, "threshold": -8.680022239685059, "operator": "ge"}, {"budget": 0.78, "threshold": -8.724714279174805, "operator": "ge"}, {"budget": 0.79, "threshold": -8.781978607177734, "operator": "ge"}, {"budget": 0.8, "threshold": -8.839081764221191, "operator": "ge"}, {"budget": 0.81, "threshold": -8.893980979919434, "operator": "ge"}, {"budget": 0.82, "threshold": -8.945696830749512, "operator": "ge"}, {"budget": 0.83, "threshold": -8.991144180297852, "operator": "ge"}, {"budget": 0.84, "threshold": -9.049049377441406, "operator": "ge"}, {"budget": 0.85, "threshold": -9.097501754760742, "operator": "ge"}, {"budget": 0.86, "threshold": -9.14026165008545, "operator": "ge"}, {"budget": 0.87, "threshold": -9.193133354187012, "operator": "ge"}, {"budget": 0.88, "threshold": -9.251825332641602, "operator": "gt"}, {"budget": 0.89, "threshold": -9.310909271240234, "operator": "ge"}, {"budget": 0.9, "threshold": -9.360374450683594, "operator": "ge"}, {"budget": 0.91, "threshold": -9.425851821899414, "operator": "gt"}, {"budget": 0.92, "threshold": -9.482333183288574, "operator": "ge"}, {"budget": 0.93, "threshold": -9.562174797058105, "operator": "ge"}, {"budget": 0.94, "threshold": -9.6319580078125, "operator": "ge"}, {"budget": 0.95, "threshold": -9.713939666748047, "operator": "ge"}, {"budget": 0.96, "threshold": -9.813028335571289, "operator": "ge"}, {"budget": 0.97, "threshold": -9.936725616455078, "operator": "ge"}, {"budget": 0.98, "threshold": -10.06577205657959, "operator": "ge"}, {"budget": 0.99, "threshold": -10.316147804260254, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.26881720430107, 75.54290533417668, 75.63426804413523, 75.57804483800689, 75.59210063953897, 75.62724014336918, 75.69049125026355, 75.73265865485979, 75.63426804413523, 75.73265865485979, 75.83807716635042, 75.90132827324479, 75.95755147937311, 75.99971888396936, 76.15433270082227, 76.2456954107808, 76.2456954107808, 76.2456954107808, 76.23163960924872, 76.15433270082227, 76.22461170848268, 76.1754164031204, 76.25975121231289, 76.31597441844121, 76.1754164031204, 76.14027689929019, 76.21055590695059, 76.21758380771664, 76.1754164031204, 76.11919319699206, 75.91538407477687, 75.78185396022208, 75.6131843418371, 75.40234731885586, 75.26881720430107, 75.0509522805538, 74.97364537212735, 74.7768641506782, 74.50980392156863, 74.41141331084405, 74.27788319628928, 74.13029728020241, 73.90540445568908, 73.67348373040973, 73.42750720359828, 73.19558647831893, 73.00583315763582, 72.83716353925082, 72.59821491320542, 72.26087567643546, 72.03598285192213, 71.75486682128049, 71.51591819523509, 71.2277742638274, 70.98179773701595, 70.67257010331014, 70.39848197343453, 70.15953334738913, 69.96275212593999, 69.7097476983625, 69.3513247592944, 69.09129243095087, 68.92965071333192, 68.71178578958465, 68.52906036966758, 68.36741865204863, 68.21280483519573, 68.05116311757679, 67.85438189612762, 67.68571227774264, 67.56623796471995, 67.43973575093119, 67.29214983484432, 67.09536861339518, 66.9126431934781, 66.70180617049687, 66.54016445287793, 66.3996064375571, 66.31527162836461, 66.1606578115117, 66.02712769695692, 65.89359758240214, 65.8022348724436, 65.69681636095298, 65.5632862463982, 65.45083983414153, 65.34542132265092, 65.27514231499052, 65.17675170426594, 65.0502494904772, 64.97997048281678, 64.85346826902804, 64.71291025370722, 64.65668704757888, 64.52315693302411, 64.44585002459765, 64.34745941387308, 64.29826410851078, 64.23501300161641, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.988755358774334, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.988825637782, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.98228969006958, 88.9943074003795, 89.9992972099234, 90.9902312179352, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 34, 74, 95, 109, 130, 161, 188, 220, 243, 274, 317, 350, 387, 420, 460, 499, 534, 567, 600, 627, 663, 691, 728, 767, 791, 817, 853, 882, 906, 934, 955, 971, 991, 1003, 1025, 1041, 1064, 1082, 1091, 1111, 1128, 1140, 1157, 1170, 1177, 1190, 1198, 1205, 1217, 1223, 1230, 1235, 1246, 1252, 1260, 1266, 1271, 1277, 1285, 1292, 1296, 1298, 1304, 1309, 1312, 1316, 1319, 1323, 1325, 1329, 1331, 1332, 1335, 1335, 1337, 1338, 1343, 1344, 1348, 1351, 1352, 1352, 1353, 1356, 1357, 1358, 1360, 1360, 1361, 1362, 1362, 1362, 1362, 1365, 1365, 1367, 1367, 1369, 1369, 1369], "harmed": [0, 0, 1, 9, 31, 50, 76, 94, 120, 157, 174, 202, 226, 255, 282, 300, 326, 361, 394, 429, 467, 493, 528, 553, 584, 628, 659, 685, 713, 743, 779, 829, 864, 908, 950, 991, 1038, 1072, 1118, 1165, 1199, 1235, 1268, 1317, 1363, 1405, 1451, 1486, 1517, 1563, 1617, 1656, 1701, 1746, 1793, 1836, 1886, 1930, 1970, 2006, 2049, 2104, 2143, 2172, 2208, 2237, 2264, 2289, 2316, 2346, 2374, 2393, 2412, 2436, 2464, 2492, 2523, 2551, 2572, 2588, 2613, 2633, 2652, 2666, 2684, 2704, 2721, 2738, 2748, 2763, 2782, 2792, 2810, 2830, 2841, 2860, 2873, 2887, 2896, 2905, 2921], "exact_score": [75.02986857825567, 75.26881720430107, 75.54290533417668, 75.63426804413523, 75.57804483800689, 75.59210063953897, 75.62724014336918, 75.69049125026355, 75.73265865485979, 75.63426804413523, 75.73265865485979, 75.83807716635042, 75.90132827324479, 75.95755147937311, 75.99971888396936, 76.15433270082227, 76.2456954107808, 76.2456954107808, 76.2456954107808, 76.23163960924872, 76.15433270082227, 76.22461170848268, 76.1754164031204, 76.25975121231289, 76.31597441844121, 76.1754164031204, 76.14027689929019, 76.21055590695059, 76.21758380771664, 76.1754164031204, 76.11919319699206, 75.91538407477687, 75.78185396022208, 75.6131843418371, 75.40234731885586, 75.26881720430107, 75.0509522805538, 74.97364537212735, 74.7768641506782, 74.50980392156863, 74.41141331084405, 74.27788319628928, 74.13029728020241, 73.90540445568908, 73.67348373040973, 73.42750720359828, 73.19558647831893, 73.00583315763582, 72.83716353925082, 72.59821491320542, 72.26087567643546, 72.03598285192213, 71.75486682128049, 71.51591819523509, 71.2277742638274, 70.98179773701595, 70.67257010331014, 70.39848197343453, 70.15953334738913, 69.96275212593999, 69.7097476983625, 69.3513247592944, 69.09129243095087, 68.92965071333192, 68.71178578958465, 68.52906036966758, 68.36741865204863, 68.21280483519573, 68.05116311757679, 67.85438189612762, 67.68571227774264, 67.5592100639539, 67.43973575093119, 67.29214983484432, 67.09536861339518, 66.9126431934781, 66.70180617049687, 66.54016445287793, 66.3996064375571, 66.31527162836461, 66.1606578115117, 66.02712769695692, 65.89359758240214, 65.8022348724436, 65.69681636095298, 65.5632862463982, 65.45083983414153, 65.34542132265092, 65.27514231499052, 65.17675170426594, 65.0502494904772, 64.97294258205073, 64.85346826902804, 64.71291025370722, 64.65668704757888, 64.52315693302411, 64.44585002459765, 64.34745941387308, 64.29826410851078, 64.23501300161641, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 71.31523851143818, "threshold_area100": 71.31576358141824, "threshold_area25": 75.84721343734627}, "test": {"score": [66.95119530385956, 67.43045118112468, 67.72254204123301, 67.8756770552704, 67.93806539432265, 68.10254374273318, 68.13657374585259, 68.13657374585259, 68.09403624195332, 68.04582707086749, 68.00612540056149, 67.96075206306894, 67.93806539432265, 67.83597538496441, 67.75657204435244, 67.64313870062104, 67.56089952641578, 67.44179451549783, 67.29716700224031, 67.1695544905425, 67.06746448118425, 66.96821030541928, 66.74985111873634, 66.61656693985196, 66.4067152539489, 66.2195502367921, 66.03805688682188, 65.83954853529195, 65.60984601423588, 65.38865099395967, 65.12491846978419, 64.86969344638857, 64.59745342143323, 64.27984005898534, 64.00192836684343, 63.77506167938065, 63.56237415988429, 63.24192496384312, 62.84207242718997, 62.584011570201056, 62.34296571477186, 62.01968068513739, 61.71057482346936, 61.39012562742818, 61.04698976264073, 60.80877974080481, 60.397583869778515, 60.03743300343136, 59.697132972237185, 59.31429543714375, 58.954144570796586, 58.605337038822555, 58.30757451152766, 58.00414031704619, 57.66667611944531, 57.40294359526983, 57.12219606953464, 56.801746873493464, 56.46711850948586, 56.20338598531038, 55.95383262910132, 55.704279272892265, 55.443382582310065, 55.15412755579503, 54.96129087145166, 54.72308084961575, 54.439497490287266, 54.19845163485806, 54.008450784107985, 53.775912429458636, 53.577404077928705, 53.37605989280549, 53.19740237642855, 53.0215806936449, 52.84008734367467, 52.703967331197, 52.57068315231261, 52.420383971868524, 52.23038312111845, 52.06874060630122, 51.9241130930437, 51.770978079006326, 51.61217139778238, 51.49873805405099, 51.3654538751666, 51.226498029095644, 51.16694552363668, 51.06485551427842, 50.97978050647988, 50.877690497121634, 50.7699288205768, 50.70186881433797, 50.63664464169242, 50.52321129796103, 50.4381362901625, 50.378583784703515, 50.313359612057965, 50.250971273005696, 50.191418767546715, 50.07231375662876, 50.00708958398321], "calls": [0.0, 4.330317896945807, 6.808836457476675, 8.83645747667527, 10.245866772537788, 11.65244023480702, 12.817967841647052, 13.878569605535548, 14.959022204577035, 16.1557439809432, 17.258883248730964, 18.36202251651873, 19.44814678274679, 20.46337520914273, 21.368006125400562, 22.527862065054023, 23.676374670334344, 24.606528088931743, 25.65011485126053, 26.92340413464538, 27.88758755636219, 29.007741825709665, 30.295210277060942, 31.432379547968125, 32.671638828233554, 33.692538921816066, 34.97717153957406, 36.049116637835695, 37.14091257125032, 38.47942602728072, 39.83779031846411, 41.08272126591611, 42.373025550860675, 43.60944899753282, 44.96214162152965, 46.2354309049145, 47.43498851487395, 48.74230780137822, 50.22828460425942, 51.3285880384539, 52.443070640614806, 53.71352409040637, 55.14562005501518, 56.39338683606046, 57.77443779599013, 58.93712956923688, 60.230269687774715, 61.616992314890965, 62.586847403794344, 63.86297252077247, 65.00014179167967, 66.19686356804583, 67.32268950457987, 68.47687377704676, 69.66225221903979, 70.58673397045061, 71.51121572186145, 72.52644414825738, 73.46510506763462, 74.32152681280662, 75.1807843915719, 76.00884780081105, 76.83974704364348, 77.70184045600205, 78.45050052462922, 79.27856393386836, 80.27961319229787, 81.08782576638403, 81.76842582877238, 82.5057425630264, 83.24873096446701, 84.0285852026203, 84.70918526500864, 85.33874032271787, 86.0562062218189, 86.71411961546096, 87.31531633723733, 87.85696055355471, 88.41845560502509, 89.12741400334629, 89.77398406261521, 90.40921078751099, 91.0756316819329, 91.58891756231743, 92.19862178487367, 92.7686243371239, 93.18832770893005, 93.73847942602728, 94.34534781499022, 94.9068428664606, 95.37475540935257, 95.80863794912514, 96.19714715140516, 96.75297053568896, 97.27192808326008, 97.7511839605252, 98.21342483623062, 98.71536738224201, 99.07551824858918, 99.52357995632816, 100.0], "rescued": [0, 185, 346, 458, 547, 660, 731, 789, 845, 907, 954, 1010, 1075, 1118, 1165, 1214, 1269, 1304, 1352, 1409, 1447, 1498, 1549, 1593, 1640, 1667, 1716, 1754, 1792, 1828, 1867, 1901, 1937, 1967, 2000, 2031, 2056, 2082, 2111, 2134, 2155, 2176, 2201, 2227, 2252, 2281, 2292, 2320, 2330, 2343, 2359, 2379, 2396, 2415, 2432, 2440, 2459, 2468, 2482, 2496, 2503, 2515, 2523, 2525, 2529, 2533, 2540, 2546, 2551, 2554, 2561, 2565, 2571, 2574, 2577, 2581, 2583, 2587, 2588, 2593, 2595, 2599, 2602, 2603, 2608, 2608, 2610, 2611, 2612, 2613, 2613, 2615, 2616, 2616, 2618, 2620, 2620, 2625, 2625, 2626, 2627], "harmed": [0, 16, 74, 132, 199, 254, 313, 371, 442, 521, 582, 654, 727, 806, 881, 970, 1054, 1131, 1230, 1332, 1406, 1492, 1620, 1711, 1832, 1925, 2038, 2146, 2265, 2379, 2511, 2635, 2767, 2909, 3040, 3151, 3251, 3390, 3560, 3674, 3780, 3915, 4049, 4188, 4334, 4447, 4603, 4758, 4888, 5036, 5179, 5322, 5444, 5570, 5706, 5807, 5925, 6047, 6179, 6286, 6381, 6481, 6581, 6685, 6757, 6845, 6952, 7043, 7115, 7200, 7277, 7352, 7421, 7486, 7553, 7605, 7654, 7711, 7779, 7841, 7894, 7952, 8011, 8052, 8104, 8153, 8176, 8213, 8244, 8281, 8319, 8345, 8369, 8409, 8441, 8464, 8487, 8514, 8535, 8578, 8602], "exact_score": [66.95119530385956, 67.00791197572526, 67.08164364915066, 67.22910699600148, 67.37657034285228, 67.50985452173667, 67.66015370218076, 67.72821370841959, 67.8388112185577, 67.88985622323682, 67.94090122791594, 68.06567790602047, 68.12806624507274, 68.1337379122593, 68.13090207866603, 68.08836457476674, 68.05433457164733, 68.00896123415477, 68.00045373337493, 67.92388622635623, 67.90119955760996, 67.782094546692, 67.6885120381136, 67.59776536312849, 67.52119785610981, 67.36806284207243, 67.26880866630745, 67.15821115616936, 67.07030031477753, 66.96821030541928, 66.80373195700876, 66.6846269460908, 66.50596942971386, 66.32731191333693, 66.17984856648611, 66.04089272041517, 65.85089186966509, 65.63253268298216, 65.47939766894478, 65.29506848538128, 65.11073930181777, 64.8810367807617, 64.6768567620452, 64.43864674020928, 64.22028755352636, 63.99909253325015, 63.811927516093355, 63.64461333408955, 63.43476164818648, 63.19087995916399, 62.92147576780194, 62.660579077219744, 62.43938405694354, 62.221024870260614, 61.96863568045827, 61.750276493775345, 61.492215636786426, 61.23699061339081, 61.00161642514818, 60.788928905651815, 60.488330544763635, 60.227433854181434, 59.91265632532683, 59.57235629413265, 59.27175793324447, 58.95698040438987, 58.667725377874824, 58.40966452088592, 58.13458866233729, 57.83399030144911, 57.54757110852735, 57.27533108357201, 56.98324022346368, 56.634432691489664, 56.32816266341491, 55.99920596659388, 55.70995094007883, 55.406516745597365, 55.09457505033605, 54.808155857414285, 54.476363326999966, 54.2296458043842, 53.92337577630945, 53.6341207497944, 53.38740322717863, 53.13784987096957, 52.85426651164109, 52.6500864929246, 52.37217480078269, 52.10560644301392, 51.87873975555114, 51.6263505657488, 51.39097637750617, 51.19246802597623, 51.02798967756572, 50.85500382837536, 50.66500297762527, 50.48067379406176, 50.35589711595724, 50.19709043473328, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.36970944303772, "threshold_area100": 59.267887020389644, "threshold_area25": 67.48092901908515}, "selected_budget": 24, "selected": {"val": {"score": 76.31597441844121, "calls": 23.993253215264602, "rescued": 767, "harmed": 584}, "test": {"score": 66.4067152539489, "calls": 32.671638828233554, "rescued": 1640, "harmed": 1832}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.17287802696228027, "operator": "ge"}, {"budget": 0.02, "threshold": 0.05346250534057617, "operator": "ge"}, {"budget": 0.03, "threshold": -0.019392728805541992, "operator": "ge"}, {"budget": 0.04, "threshold": -0.227622389793396, "operator": "ge"}, {"budget": 0.05, "threshold": -0.4029449224472046, "operator": "ge"}, {"budget": 0.06, "threshold": -0.4963408410549164, "operator": "ge"}, {"budget": 0.07, "threshold": -0.5943831205368042, "operator": "ge"}, {"budget": 0.08, "threshold": -0.7146822214126587, "operator": "ge"}, {"budget": 0.09, "threshold": -0.8297547101974487, "operator": "ge"}, {"budget": 0.1, "threshold": -0.9688904285430908, "operator": "ge"}, {"budget": 0.11, "threshold": -1.1175286769866943, "operator": "ge"}, {"budget": 0.12, "threshold": -1.2775331735610962, "operator": "ge"}, {"budget": 0.13, "threshold": -1.4381539821624756, "operator": "ge"}, {"budget": 0.14, "threshold": -1.5861600637435913, "operator": "ge"}, {"budget": 0.15, "threshold": -1.7266428470611572, "operator": "ge"}, {"budget": 0.16, "threshold": -1.8801575899124146, "operator": "ge"}, {"budget": 0.17, "threshold": -2.0385212898254395, "operator": "ge"}, {"budget": 0.18, "threshold": -2.1786370277404785, "operator": "ge"}, {"budget": 0.19, "threshold": -2.3101181983947754, "operator": "ge"}, {"budget": 0.2, "threshold": -2.4219539165496826, "operator": "ge"}, {"budget": 0.21, "threshold": -2.5513203144073486, "operator": "ge"}, {"budget": 0.22, "threshold": -2.6902475357055664, "operator": "ge"}, {"budget": 0.23, "threshold": -2.8004372119903564, "operator": "ge"}, {"budget": 0.24, "threshold": -2.929018974304199, "operator": "ge"}, {"budget": 0.25, "threshold": -3.0349860191345215, "operator": "ge"}, {"budget": 0.26, "threshold": -3.152747631072998, "operator": "ge"}, {"budget": 0.27, "threshold": -3.2732083797454834, "operator": "ge"}, {"budget": 0.28, "threshold": -3.394394636154175, "operator": "ge"}, {"budget": 0.29, "threshold": -3.4924817085266113, "operator": "ge"}, {"budget": 0.3, "threshold": -3.594287395477295, "operator": "ge"}, {"budget": 0.31, "threshold": -3.6877288818359375, "operator": "gt"}, {"budget": 0.32, "threshold": -3.772076368331909, "operator": "ge"}, {"budget": 0.33, "threshold": -3.875454902648926, "operator": "ge"}, {"budget": 0.34, "threshold": -3.980539321899414, "operator": "ge"}, {"budget": 0.35, "threshold": -4.063233852386475, "operator": "ge"}, {"budget": 0.36, "threshold": -4.155494213104248, "operator": "ge"}, {"budget": 0.37, "threshold": -4.2395734786987305, "operator": "ge"}, {"budget": 0.38, "threshold": -4.315152168273926, "operator": "ge"}, {"budget": 0.39, "threshold": -4.3830671310424805, "operator": "ge"}, {"budget": 0.4, "threshold": -4.461608409881592, "operator": "ge"}, {"budget": 0.41, "threshold": -4.539497375488281, "operator": "ge"}, {"budget": 0.42, "threshold": -4.614551067352295, "operator": "ge"}, {"budget": 0.43, "threshold": -4.690187454223633, "operator": "ge"}, {"budget": 0.44, "threshold": -4.758944511413574, "operator": "ge"}, {"budget": 0.45, "threshold": -4.828167915344238, "operator": "ge"}, {"budget": 0.46, "threshold": -4.903008937835693, "operator": "ge"}, {"budget": 0.47, "threshold": -4.9725260734558105, "operator": "ge"}, {"budget": 0.48, "threshold": -5.0326128005981445, "operator": "ge"}, {"budget": 0.49, "threshold": -5.097414970397949, "operator": "ge"}, {"budget": 0.5, "threshold": -5.168319225311279, "operator": "ge"}, {"budget": 0.51, "threshold": -5.228858947753906, "operator": "ge"}, {"budget": 0.52, "threshold": -5.286471366882324, "operator": "ge"}, {"budget": 0.53, "threshold": -5.3460822105407715, "operator": "ge"}, {"budget": 0.54, "threshold": -5.40635347366333, "operator": "ge"}, {"budget": 0.55, "threshold": -5.465575695037842, "operator": "ge"}, {"budget": 0.56, "threshold": -5.523006439208984, "operator": "ge"}, {"budget": 0.57, "threshold": -5.57278299331665, "operator": "ge"}, {"budget": 0.58, "threshold": -5.629328727722168, "operator": "ge"}, {"budget": 0.59, "threshold": -5.688631057739258, "operator": "ge"}, {"budget": 0.6, "threshold": -5.734009265899658, "operator": "ge"}, {"budget": 0.61, "threshold": -5.7863593101501465, "operator": "ge"}, {"budget": 0.62, "threshold": -5.838769435882568, "operator": "gt"}, {"budget": 0.63, "threshold": -5.886713981628418, "operator": "ge"}, {"budget": 0.64, "threshold": -5.9318976402282715, "operator": "ge"}, {"budget": 0.65, "threshold": -5.977260589599609, "operator": "ge"}, {"budget": 0.66, "threshold": -6.029107093811035, "operator": "ge"}, {"budget": 0.67, "threshold": -6.077320098876953, "operator": "ge"}, {"budget": 0.68, "threshold": -6.121368408203125, "operator": "gt"}, {"budget": 0.69, "threshold": -6.172276973724365, "operator": "ge"}, {"budget": 0.7, "threshold": -6.220683574676514, "operator": "ge"}, {"budget": 0.71, "threshold": -6.276643753051758, "operator": "ge"}, {"budget": 0.72, "threshold": -6.325779914855957, "operator": "ge"}, {"budget": 0.73, "threshold": -6.381286144256592, "operator": "ge"}, {"budget": 0.74, "threshold": -6.430922031402588, "operator": "ge"}, {"budget": 0.75, "threshold": -6.484508037567139, "operator": "ge"}, {"budget": 0.76, "threshold": -6.530290126800537, "operator": "ge"}, {"budget": 0.77, "threshold": -6.582942008972168, "operator": "ge"}, {"budget": 0.78, "threshold": -6.628327369689941, "operator": "ge"}, {"budget": 0.79, "threshold": -6.678914546966553, "operator": "ge"}, {"budget": 0.8, "threshold": -6.72775411605835, "operator": "ge"}, {"budget": 0.81, "threshold": -6.771326065063477, "operator": "ge"}, {"budget": 0.82, "threshold": -6.828560829162598, "operator": "ge"}, {"budget": 0.83, "threshold": -6.888408184051514, "operator": "ge"}, {"budget": 0.84, "threshold": -6.939929008483887, "operator": "ge"}, {"budget": 0.85, "threshold": -6.99190092086792, "operator": "gt"}, {"budget": 0.86, "threshold": -7.043490409851074, "operator": "ge"}, {"budget": 0.87, "threshold": -7.103491306304932, "operator": "ge"}, {"budget": 0.88, "threshold": -7.162592887878418, "operator": "ge"}, {"budget": 0.89, "threshold": -7.226926326751709, "operator": "ge"}, {"budget": 0.9, "threshold": -7.286648273468018, "operator": "ge"}, {"budget": 0.91, "threshold": -7.358317852020264, "operator": "ge"}, {"budget": 0.92, "threshold": -7.432895183563232, "operator": "ge"}, {"budget": 0.93, "threshold": -7.503235816955566, "operator": "ge"}, {"budget": 0.94, "threshold": -7.587697982788086, "operator": "ge"}, {"budget": 0.95, "threshold": -7.686333179473877, "operator": "ge"}, {"budget": 0.96, "threshold": -7.793121814727783, "operator": "ge"}, {"budget": 0.97, "threshold": -7.92948579788208, "operator": "ge"}, {"budget": 0.98, "threshold": -8.117609977722168, "operator": "ge"}, {"budget": 0.99, "threshold": -8.403219223022461, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.32504041042941, 75.4937100288144, 75.64832384566729, 75.60615644107105, 75.4937100288144, 75.45154262421815, 75.54290533417668, 75.6131843418371, 75.67643544873147, 75.77482605945605, 75.78888186098813, 75.77482605945605, 75.76779815869, 75.73265865485979, 75.55696113570876, 75.4585705249842, 75.32504041042941, 75.2055660974067, 75.09311968515004, 75.04392437978777, 74.91742216599901, 75.02284067748963, 74.97364537212735, 74.88931056293485, 74.75578044838007, 74.67144563918757, 74.72064094454987, 74.73469674608194, 74.67144563918757, 74.61522243305924, 74.53088762386675, 74.38330170777989, 74.22165999016094, 74.08812987560616, 73.92648815798721, 73.79998594419847, 73.64537212734557, 73.55400941738702, 73.44156300513038, 73.29397708904351, 73.11125166912643, 72.80202403542062, 72.58415911167334, 72.30304308103169, 72.1132897603486, 71.86731323353715, 71.67755991285402, 71.43158338604259, 71.19966266076322, 70.96774193548387, 70.72176540867244, 70.37739827113641, 70.2368402558156, 69.95572422517394, 69.73083140066062, 69.50593857614732, 69.29510155316608, 69.16157143861129, 68.9577623163961, 68.71881369035069, 68.44472556047508, 68.24794433902593, 68.02305151451262, 67.85438189612762, 67.66462857544452, 67.51704265935766, 67.34837304097266, 67.17267552182163, 66.9126431934781, 66.722889872795, 66.60341555977229, 66.44177384215335, 66.30824372759857, 66.21688101764003, 66.13957410921358, 66.06226720078713, 65.94982078853047, 65.86548597933796, 65.68978846018693, 65.6124815517605, 65.55625834563216, 65.46489563567363, 65.3735329257151, 65.26108651345844, 65.14864010120178, 65.10647269660552, 65.02916578817907, 65.00808208588094, 64.91671937592241, 64.83238456672991, 64.73399395600535, 64.59343594068451, 64.52315693302411, 64.45990582612974, 64.43179422306558, 64.35448731463912, 64.27718040621266, 64.24204090238246, 64.19284559702018, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.986014477475578, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.98608475648324, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.98791201068241, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.98840396373603, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 46, 71, 94, 108, 121, 146, 176, 205, 238, 267, 296, 323, 358, 391, 412, 435, 458, 483, 509, 541, 571, 617, 646, 677, 698, 725, 756, 789, 819, 851, 881, 904, 924, 947, 964, 994, 1020, 1042, 1066, 1088, 1105, 1120, 1138, 1152, 1171, 1184, 1199, 1211, 1225, 1230, 1242, 1244, 1258, 1262, 1268, 1275, 1286, 1294, 1304, 1307, 1308, 1310, 1313, 1314, 1320, 1324, 1330, 1333, 1334, 1336, 1337, 1339, 1344, 1348, 1353, 1356, 1358, 1359, 1359, 1360, 1362, 1363, 1364, 1364, 1364, 1365, 1365, 1365, 1366, 1367, 1367, 1368, 1369, 1369, 1369, 1369, 1369, 1369, 1369, 1369], "harmed": [0, 4, 5, 6, 26, 55, 86, 103, 122, 146, 161, 188, 217, 253, 291, 337, 374, 416, 458, 500, 539, 587, 618, 654, 697, 737, 776, 800, 831, 870, 910, 952, 996, 1039, 1081, 1121, 1169, 1217, 1252, 1292, 1335, 1378, 1437, 1486, 1540, 1586, 1634, 1676, 1723, 1770, 1808, 1855, 1906, 1940, 1984, 2022, 2061, 2102, 2129, 2168, 2205, 2245, 2275, 2310, 2335, 2368, 2393, 2423, 2451, 2489, 2518, 2536, 2561, 2585, 2602, 2618, 2632, 2650, 2663, 2688, 2700, 2710, 2724, 2738, 2754, 2770, 2777, 2788, 2791, 2805, 2818, 2832, 2853, 2864, 2873, 2877, 2888, 2899, 2904, 2911, 2921], "exact_score": [75.02986857825567, 75.32504041042941, 75.4937100288144, 75.64832384566729, 75.60615644107105, 75.4937100288144, 75.45154262421815, 75.54290533417668, 75.6131843418371, 75.67643544873147, 75.77482605945605, 75.78888186098813, 75.77482605945605, 75.76779815869, 75.73265865485979, 75.55696113570876, 75.4585705249842, 75.32504041042941, 75.2055660974067, 75.09311968515004, 75.04392437978777, 74.91742216599901, 75.02284067748963, 74.97364537212735, 74.88931056293485, 74.75578044838007, 74.67144563918757, 74.72064094454987, 74.73469674608194, 74.67144563918757, 74.61522243305924, 74.53088762386675, 74.38330170777989, 74.22165999016094, 74.08812987560616, 73.92648815798721, 73.79998594419847, 73.64537212734557, 73.55400941738702, 73.44156300513038, 73.29397708904351, 73.11125166912643, 72.80202403542062, 72.58415911167334, 72.30304308103169, 72.1132897603486, 71.86731323353715, 71.67755991285402, 71.43158338604259, 71.19966266076322, 70.96774193548387, 70.72176540867244, 70.37739827113641, 70.2368402558156, 69.95572422517394, 69.73083140066062, 69.50593857614732, 69.29510155316608, 69.16157143861129, 68.9577623163961, 68.71881369035069, 68.44472556047508, 68.24091643825989, 68.02305151451262, 67.85438189612762, 67.66462857544452, 67.51704265935766, 67.34837304097266, 67.17267552182163, 66.9126431934781, 66.722889872795, 66.60341555977229, 66.44177384215335, 66.30824372759857, 66.21688101764003, 66.13957410921358, 66.06226720078713, 65.94982078853047, 65.86548597933796, 65.68978846018693, 65.6124815517605, 65.55625834563216, 65.46489563567363, 65.3735329257151, 65.26108651345844, 65.14864010120178, 65.10647269660552, 65.02916578817907, 65.00808208588094, 64.91671937592241, 64.83238456672991, 64.73399395600535, 64.59343594068451, 64.52315693302411, 64.45990582612974, 64.43179422306558, 64.35448731463912, 64.27718040621266, 64.24204090238246, 64.19284559702018, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 70.51784490887462, "threshold_area100": 70.51830768149554, "threshold_area25": 75.35244922341697}, "test": {"score": [66.95119530385956, 67.51269035532995, 67.762243711539, 67.86433372089725, 67.92105039276295, 68.01179706774806, 68.0429912372742, 68.06284207242719, 68.01463290134134, 67.8955278904234, 67.762243711539, 67.70552703967331, 67.51836202251653, 67.37089867566571, 67.24612199756118, 67.08731531633724, 66.91432946714687, 66.69597028046394, 66.51447693049371, 66.33581941411678, 66.17701273289283, 65.98701188214275, 65.74313019312027, 65.53611434081049, 65.32059098772083, 65.11924680259762, 64.95193262059382, 64.69954343079148, 64.40461673708987, 64.18342171681365, 63.87998752233218, 63.66446416924255, 63.440433315373056, 63.23625329665655, 62.89878909905566, 62.63505657488019, 62.320279046025576, 62.01684485154411, 61.7389331594022, 61.489379803193145, 61.2001247766781, 60.854153078297365, 60.5308680486629, 60.21892635340158, 59.88146215580069, 59.51563962226696, 59.12713041998695, 58.772651220826354, 58.54578453336357, 58.27921617559482, 57.927572810027506, 57.57876527805348, 57.164733573433914, 56.87264271332558, 56.63159685789638, 56.27995349232907, 55.96234012988117, 55.65607010180642, 55.332785072171966, 55.080395882369615, 54.88188753083969, 54.6578566769702, 54.45934832544026, 54.23248163797748, 54.02830161926098, 53.80994243257805, 53.57456824433542, 53.38173155999206, 53.214417377988255, 52.973371522559056, 52.806057340555256, 52.58486232027904, 52.36650313359612, 52.20202478518561, 52.04038227036838, 51.901426424297426, 51.75963474463319, 51.580977228256245, 51.45620055015172, 51.32291637126733, 51.18396052519638, 51.090378016617976, 50.97978050647988, 50.874854663528346, 50.7699288205768, 50.713212148711115, 50.64231630887899, 50.53739046592746, 50.48067379406176, 50.41544962141622, 50.35873294955052, 50.30768794487139, 50.24246377222584, 50.191418767546715, 50.1573887644273, 50.12335876130789, 50.10350792615489, 50.07798542381533, 50.04962708788248, 50.018432918356346, 50.00708958398321], "calls": [0.0, 3.3632986416357094, 5.983608881830814, 8.978249156339507, 10.665570144343931, 11.88497858945637, 12.843490343986616, 14.054391288319202, 15.565890593539972, 16.944105719876358, 18.09261832515668, 19.26381759918328, 20.43501687320988, 21.46158863397896, 22.360547883050224, 23.211297961035644, 24.20383971868531, 25.21906814508125, 26.135042395712222, 27.05952414712305, 27.910274225108473, 28.797890139806597, 29.861327737288374, 30.712077815273798, 31.741485409636162, 32.5525338173156, 33.632986416357085, 34.761648186484415, 35.992399965969994, 36.857329211921844, 37.93211014377677, 38.99838357485183, 40.03062700280747, 41.238692113546776, 42.571533902390605, 43.70870317329779, 45.06990329807447, 46.28647590959363, 47.43215268128066, 48.36797776706463, 49.59872954655021, 50.84933216118878, 52.12829311176021, 53.37322405921221, 54.632334174630635, 55.9765192978476, 57.394436094489976, 58.6365312083487, 59.751013810509605, 60.84280974392423, 62.24654737260018, 63.40640331225364, 64.6967075971982, 65.85372770325837, 66.99940447494541, 68.24433542239741, 69.34747469018518, 70.40524062048038, 71.57927572810027, 72.75331083572016, 73.65227008479143, 74.63346850806795, 75.6458611008706, 76.55899951790829, 77.40407792870715, 78.31154467855826, 79.2899072682415, 80.22573235402547, 81.04812409607804, 82.06618835606727, 82.86872926296684, 83.8641068542098, 84.7942602728072, 85.78113036327028, 86.56382043501686, 87.40039134503587, 88.1745739160026, 88.97995065649548, 89.69741655559652, 90.39786745313785, 91.15787085613816, 91.7448884099481, 92.51056348013499, 93.23653688001589, 93.83206193460568, 94.34251198139694, 94.83027535944191, 95.388934577319, 95.92490712644982, 96.42684967246123, 96.85506054504721, 97.27759975044664, 97.66894478631994, 98.05461815500666, 98.4204406885404, 98.74372571817486, 98.98193574001077, 99.23716076340641, 99.50940078836173, 99.74761081019766, 100.0], "rescued": [0, 266, 389, 458, 541, 633, 688, 748, 821, 873, 911, 980, 1025, 1073, 1112, 1140, 1181, 1231, 1272, 1314, 1351, 1388, 1423, 1453, 1502, 1537, 1588, 1633, 1671, 1699, 1732, 1785, 1819, 1863, 1896, 1932, 1974, 2007, 2031, 2054, 2093, 2120, 2154, 2184, 2203, 2227, 2250, 2275, 2292, 2311, 2336, 2349, 2367, 2385, 2402, 2418, 2434, 2448, 2457, 2474, 2484, 2495, 2509, 2521, 2529, 2536, 2539, 2546, 2551, 2561, 2567, 2572, 2577, 2582, 2587, 2592, 2597, 2599, 2603, 2605, 2605, 2607, 2610, 2610, 2611, 2614, 2617, 2618, 2619, 2619, 2621, 2622, 2625, 2626, 2626, 2627, 2627, 2627, 2627, 2627, 2627], "harmed": [0, 68, 103, 136, 199, 259, 303, 356, 446, 540, 625, 714, 825, 925, 1008, 1092, 1194, 1321, 1426, 1531, 1624, 1728, 1849, 1952, 2077, 2183, 2293, 2427, 2569, 2675, 2815, 2944, 3057, 3173, 3325, 3454, 3607, 3747, 3869, 3980, 4121, 4270, 4418, 4558, 4696, 4849, 5009, 5159, 5256, 5369, 5518, 5654, 5818, 5939, 6041, 6181, 6309, 6431, 6554, 6660, 6740, 6830, 6914, 7006, 7086, 7170, 7256, 7331, 7395, 7490, 7555, 7638, 7720, 7783, 7845, 7899, 7954, 8019, 8067, 8116, 8165, 8200, 8242, 8279, 8317, 8340, 8368, 8406, 8427, 8450, 8472, 8491, 8517, 8536, 8548, 8561, 8568, 8577, 8587, 8598, 8602], "exact_score": [66.95119530385956, 67.19507699288205, 67.33970450613957, 67.44463034909111, 67.58642202875535, 67.67716870374046, 67.7650795451323, 67.82463205059128, 67.83313955137112, 67.86149788730397, 67.92105039276295, 67.97776706462865, 68.00045373337493, 68.03731957008763, 68.06851373961376, 68.03731957008763, 67.97493123103536, 67.8926920568301, 67.77925871309871, 67.73388537560616, 67.56089952641578, 67.43612284831127, 67.30283866942688, 67.12985282023651, 66.95686697104614, 66.75268695232963, 66.53149193205343, 66.34716274848992, 66.1628335649264, 65.91895187590391, 65.72611519156055, 65.45954683379179, 65.26954598304171, 65.04551512917222, 64.86969344638857, 64.65700592689221, 64.40178090349659, 64.14088421291439, 63.87148002155233, 63.66446416924255, 63.44894081615291, 63.253268298216256, 63.023565777160194, 62.77401242095113, 62.56132490145478, 62.33162238039872, 62.087740691376226, 61.8325156679806, 61.58296231177154, 61.34758812352891, 61.10370643450642, 60.79743640643167, 60.57907721974875, 60.31534469557326, 60.054448004991066, 59.80773048237529, 59.5071321214871, 59.2490712644982, 58.94847290361002, 58.71593454896067, 58.474888693531454, 58.22817117091569, 57.998468649859625, 57.692198621784875, 57.400107761676544, 57.099509400788364, 56.84712021098601, 56.63159685789638, 56.33667016419477, 56.075773473612564, 55.763831778351246, 55.49726342058248, 55.24487423078014, 55.02651504409721, 54.808155857414285, 54.572781669171654, 54.371437484048435, 54.14173496299237, 53.886509939596735, 53.642628250574255, 53.42426906389133, 53.22292487876812, 52.98471485693219, 52.77486317102912, 52.56501148512604, 52.34098063125656, 52.15097978050648, 51.96665059694296, 51.785157246972744, 51.57246972747639, 51.39664804469274, 51.20664719394266, 51.03933301193886, 50.89470549868133, 50.750077985423815, 50.61679380653943, 50.47216629328191, 50.35873294955052, 50.194254601139995, 50.09783625896832, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 60.364887262538446, "threshold_area100": 58.50127612511699, "threshold_area25": 67.01471797634916}, "selected_budget": 11, "selected": {"val": {"score": 75.78888186098813, "calls": 10.998664698854451, "rescued": 296, "harmed": 188}, "test": {"score": 67.70552703967331, "calls": 19.26381759918328, "rescued": 980, "harmed": 714}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 75.64832384566729, "calls": 5.994799353433129}, "test": {"score": 68.11672291069961, "calls": 13.07319286504268}}, "old_normalized_interior": {"val": {"score": 75.62021224260313, "calls": 4.996837444655281}, "test": {"score": 68.10254374273318, "calls": 11.79423191447126}}, "old_normalized_endpoints": {"val": {"score": 75.62021224260313, "calls": 4.996837444655281}, "test": {"score": 68.10254374273318, "calls": 11.79423191447126}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.24773350200296, 75.3742357157917, 75.5288495326446, 75.60615644107105, 75.62021224260313, 75.57804483800689, 75.57804483800689, 75.54290533417668, 75.54290533417668, 75.5288495326446, 75.50073792958042, 75.47262632651626, 75.43045892192002, 75.45154262421815, 75.45154262421815, 75.42343102115399, 75.41640312038794, 75.41640312038794, 75.41640312038794, 75.38126361655773, 75.35315201349357, 75.28287300583317, 75.27584510506712, 75.254761402769, 75.19853819664067, 75.15637079204441, 75.1352870897463, 75.14231499051233, 75.10014758591609, 75.07906388361796, 75.03689647902172, 75.02284067748963, 74.98770117365943, 74.8963384637009, 74.91742216599901, 74.8822826621688, 74.78389205144424, 74.7276688453159, 74.70658514301778, 74.6573898376555, 74.66441773842153, 74.65036193688945, 74.62927823459134, 74.5379155246328, 74.50980392156863, 74.404385410078, 74.41141331084405, 74.34113430318364, 74.20760418862886, 74.13029728020241, 74.16543678403261, 74.12326937943637, 74.05299037177595, 74.00379506641366, 73.90540445568908, 73.77890224190034, 73.74376273807015, 73.68051163117578, 73.65942792887765, 73.63834422657952, 73.55400941738702, 73.44156300513038, 73.35020029517183, 73.30100498980954, 73.13233537142456, 73.03394476069998, 72.9355541499754, 72.7809403331225, 72.6122707147375, 72.49982430248085, 72.32412678332982, 72.16248506571087, 72.14842926417879, 72.07815025651838, 71.82514582894089, 71.77595052357862, 71.43861128680862, 71.2277742638274, 70.93963033241971, 70.7498770117366, 70.45470517956286, 70.27900766041184, 70.13142174432497, 69.87841731674749, 69.55513388150959, 69.25293414856982, 68.93667861409797, 68.53608827043362, 68.38850235434676, 68.10035842293907, 67.76301918616909, 67.53812636165577, 67.1164523156933, 66.80019678122146, 66.47691334598356, 66.16768571227774, 65.83034647550777, 65.33136552111884, 64.83941246749596, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.98840396373603, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.24773350200296, 75.3742357157917, 75.5288495326446, 75.60615644107105, 75.62021224260313, 75.57804483800689, 75.57804483800689, 75.54290533417668, 75.54290533417668, 75.5288495326446, 75.50073792958042, 75.47262632651626, 75.43045892192002, 75.45154262421815, 75.45154262421815, 75.42343102115399, 75.41640312038794, 75.41640312038794, 75.41640312038794, 75.38126361655773, 75.35315201349357, 75.28287300583317, 75.27584510506712, 75.254761402769, 75.19853819664067, 75.15637079204441, 75.1352870897463, 75.14231499051233, 75.10014758591609, 75.07906388361796, 75.03689647902172, 75.02284067748963, 74.98770117365943, 74.8963384637009, 74.91742216599901, 74.8822826621688, 74.78389205144424, 74.7276688453159, 74.70658514301778, 74.6573898376555, 74.66441773842153, 74.65036193688945, 74.62927823459134, 74.5379155246328, 74.50980392156863, 74.404385410078, 74.41141331084405, 74.34113430318364, 74.20760418862886, 74.13029728020241, 74.16543678403261, 74.12326937943637, 74.05299037177595, 74.00379506641366, 73.90540445568908, 73.77890224190034, 73.74376273807015, 73.68051163117578, 73.65942792887765, 73.63834422657952, 73.55400941738702, 73.44156300513038, 73.35020029517183, 73.30100498980954, 73.13233537142456, 73.03394476069998, 72.9355541499754, 72.7809403331225, 72.6122707147375, 72.49982430248085, 72.32412678332982, 72.16248506571087, 72.14842926417879, 72.07815025651838, 71.82514582894089, 71.77595052357862, 71.43861128680862, 71.2277742638274, 70.93963033241971, 70.7498770117366, 70.45470517956286, 70.27900766041184, 70.13142174432497, 69.87841731674749, 69.55513388150959, 69.25293414856982, 68.93667861409797, 68.53608827043362, 68.38850235434676, 68.10035842293907, 67.76301918616909, 67.53812636165577, 67.1164523156933, 66.80019678122146, 66.47691334598356, 66.16768571227774, 65.83034647550777, 65.33136552111884, 64.83941246749596, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 75.41921428069436, "exact_area100": 73.01571069460044}, "test": {"score": [66.95119530385956, 67.47298868502396, 67.79060204747185, 68.0032895669682, 68.11955874429289, 68.10254374273318, 68.09120040836004, 68.0997079091399, 68.08552874117346, 68.08269290758018, 68.07134957320704, 68.0600062388339, 68.06851373961376, 68.05149873805405, 68.02314040212121, 68.03164790290106, 68.03164790290106, 68.03164790290106, 68.03448373649434, 68.02881206930778, 68.00896123415477, 68.01463290134134, 68.00896123415477, 68.00045373337493, 67.99761789978163, 67.96358789666222, 67.95224456228908, 67.93239372713609, 67.9295578935428, 67.92388622635623, 67.93239372713609, 67.93239372713609, 67.88418455605026, 67.8756770552704, 67.86149788730397, 67.85015455293083, 67.81045288262484, 67.79060204747185, 67.73104954201288, 67.6885120381136, 67.66015370218076, 67.60343703031506, 67.53537702407623, 67.4843320193971, 67.45030201627769, 67.37940617644557, 67.27164449990075, 67.19224115928877, 67.10433031789694, 67.02209114369168, 66.94835947026628, 66.8150752913819, 66.69597028046394, 66.59104443751241, 66.47477526018774, 66.35283441567648, 66.19119190085925, 66.00402688370247, 65.79701103139267, 65.68641352125456, 65.52477100643735, 65.34044182287383, 65.16745597368346, 64.9547684541871, 64.72223009953775, 64.51521424722797, 64.3223775628846, 64.03595836996286, 63.80342001531349, 63.607747497376856, 63.26744746618268, 62.99237160763407, 62.73714658423843, 62.399682386637544, 62.05938235544338, 61.71057482346936, 61.46669313444687, 61.20296061027138, 60.81161557439809, 60.50818137991663, 60.14235884638289, 59.77086464566259, 59.45892295040127, 59.149817088733236, 58.832203726285336, 58.335932847460505, 57.836826135042394, 57.23846524685931, 56.79040353912033, 56.42174517199331, 55.996370133000596, 55.528457590108616, 54.99248504097779, 54.547259166832085, 54.08785412471996, 53.58874741230184, 52.86560984601423, 52.116949777387056, 51.52993222357711, 50.752913819017095, 50.00708958398321], "calls": [0.0, 1.9028443410940645, 3.8935995235799563, 6.431670589569805, 9.908402574936904, 11.79423191447126, 13.495732070442108, 14.635737174942577, 15.716189773984063, 16.822164875365115, 17.61903411507813, 18.48679919462326, 19.14471258826532, 19.92173099282534, 20.46337520914273, 20.922780251254856, 21.45875280038567, 21.810396165952984, 22.30666704477781, 22.788758755636216, 23.319059637580466, 23.741598842979894, 24.209511385871878, 24.637722258457874, 25.046082295890876, 25.451606499730595, 25.916683209029294, 26.39593908629442, 26.86101579559311, 27.38848084394408, 27.805348382156936, 28.199529251623517, 28.66177012732893, 29.138190171000765, 29.708192723251, 30.35192694892664, 30.817003658225335, 31.352976207356154, 32.008053767404924, 32.62342965714772, 33.400448061707735, 34.228511470946884, 34.994186541133764, 35.8052349488132, 36.712701698664326, 37.60598928054902, 38.584351870232254, 39.60241613022148, 40.41913620508748, 41.25570711510649, 42.26809970790914, 43.37974647647676, 44.332586563820435, 45.24572498085812, 46.46513342597056, 47.48036185236651, 48.68559112951252, 49.77171539574058, 50.9570938377336, 51.977993931316114, 53.12083486940986, 54.14173496299237, 55.091739216742766, 56.365028500127615, 57.6496611178856, 58.6904120466211, 59.810566315968586, 61.04698976264073, 62.03669568669711, 62.94132660295494, 64.43297507302272, 65.53327850721719, 66.69880611405723, 67.92105039276295, 69.2907580183195, 70.43076312281995, 71.59629072965998, 72.58032498652979, 73.96421178005275, 75.07302271502708, 76.15914698125515, 77.4494512661997, 78.5809488699203, 79.6075206306894, 80.74185406800329, 81.9300683435896, 83.1778351246349, 84.71202109860194, 85.85486203669569, 86.82755295919236, 87.86546805433457, 88.94308481978278, 90.22771743754076, 91.32518503814197, 92.41981680514988, 93.45773190029209, 94.83878286022176, 96.28505799279698, 97.51297393868927, 98.80044239004056, 100.0], "exact_score": [66.95119530385956, 67.2234353288149, 67.49283952017694, 67.69134787170688, 67.80478121543828, 67.91537872557637, 67.97776706462865, 68.05433457164733, 68.10254374273318, 68.12239457788617, 68.11388707710631, 68.11105124351303, 68.10537957632647, 68.08836457476674, 68.0968720755466, 68.09120040836004, 68.08269290758018, 68.08552874117346, 68.06567790602047, 68.0600062388339, 68.04866290446077, 68.03731957008763, 68.04015540368091, 68.02314040212121, 68.00896123415477, 68.00045373337493, 67.9465728951025, 67.90970705838981, 67.94090122791594, 67.88985622323682, 67.85299038652411, 67.7991095482517, 67.73104954201288, 67.6885120381136, 67.6147803646882, 67.53537702407623, 67.46731701783739, 67.44179451549783, 67.34254033973286, 67.25746533193431, 67.13552448742307, 67.05044947962453, 66.97388197260584, 66.88313529762073, 66.72149278280351, 66.62223860703854, 66.52865609846013, 66.39253608598247, 66.27343107506451, 66.14865439695998, 65.96999688058305, 65.79417519779939, 65.67790602047472, 65.5502935087769, 65.35745682443354, 65.18730680883645, 65.01715679323937, 64.84700677764229, 64.65417009329893, 64.45566174176899, 64.27416839179877, 64.03879420355614, 63.81476334968664, 63.59356832941043, 63.34968664038794, 63.0972974505856, 62.90162493264895, 62.646399909253326, 62.39684655304426, 62.12460652808893, 61.84102316876046, 61.60564898051783, 61.34475228993562, 61.05549726342058, 60.80027224002495, 60.522360547883046, 60.20474718543516, 59.9183279925134, 59.63474463318492, 59.34265377307659, 59.03638374500184, 58.727277883333805, 58.304738677934374, 57.90488614128123, 57.516376939001226, 57.11652440234807, 56.75920936959419, 56.34801349856791, 55.931145960355046, 55.50577092136233, 55.105918384709184, 54.70039418086946, 54.27785497547004, 53.812778266171335, 53.273969883447236, 52.77486317102912, 52.26441312423786, 51.76814224541304, 51.26052803221507, 50.63097297450585, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 67.9343788106514, "exact_area100": 63.6608655162763}}, "old_normalized_endpoint_budget": 5}, {"seed": 2029, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.75, "validation_area100": 0.7337420105291478, "training_seconds": 79.95480367110576, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.16692296553807612, "operator": "ge"}, {"budget": 0.02, "threshold": 0.08607489058973843, "operator": "ge"}, {"budget": 0.03, "threshold": 0.03635904952328238, "operator": "ge"}, {"budget": 0.04, "threshold": 0.015187430591807431, "operator": "ge"}, {"budget": 0.05, "threshold": 0.0020421056906284726, "operator": "gt"}, {"budget": 0.06, "threshold": -0.042668141493016135, "operator": "ge"}, {"budget": 0.07, "threshold": -0.10838702922555074, "operator": "ge"}, {"budget": 0.08, "threshold": -0.16409712158684178, "operator": "ge"}, {"budget": 0.09, "threshold": -0.20435543719208224, "operator": "ge"}, {"budget": 0.1, "threshold": -0.23032695181857366, "operator": "ge"}, {"budget": 0.11, "threshold": -0.24640221302439008, "operator": "ge"}, {"budget": 0.12, "threshold": -0.26261734509243045, "operator": "ge"}, {"budget": 0.13, "threshold": -0.27717584039304427, "operator": "ge"}, {"budget": 0.14, "threshold": -0.2901198409542432, "operator": "gt"}, {"budget": 0.15, "threshold": -0.30205530684285076, "operator": "ge"}, {"budget": 0.16, "threshold": -0.3129459448848854, "operator": "ge"}, {"budget": 0.17, "threshold": -0.3216680374260657, "operator": "ge"}, {"budget": 0.18, "threshold": -0.33185952031918453, "operator": "ge"}, {"budget": 0.19, "threshold": -0.34152993708794555, "operator": "ge"}, {"budget": 0.2, "threshold": -0.35157745428826015, "operator": "ge"}, {"budget": 0.21, "threshold": -0.3604797867012146, "operator": "ge"}, {"budget": 0.22, "threshold": -0.36933188600828193, "operator": "ge"}, {"budget": 0.23, "threshold": -0.3784766761663996, "operator": "ge"}, {"budget": 0.24, "threshold": -0.3870394728314379, "operator": "ge"}, {"budget": 0.25, "threshold": -0.39478602887403363, "operator": "ge"}, {"budget": 0.26, "threshold": -0.40207511347251823, "operator": "ge"}, {"budget": 0.27, "threshold": -0.41034844421453104, "operator": "ge"}, {"budget": 0.28, "threshold": -0.4183763490949104, "operator": "ge"}, {"budget": 0.29, "threshold": -0.42700726094945013, "operator": "ge"}, {"budget": 0.3, "threshold": -0.4348407186142327, "operator": "ge"}, {"budget": 0.31, "threshold": -0.44216162899880856, "operator": "ge"}, {"budget": 0.32, "threshold": -0.4507132540928653, "operator": "ge"}, {"budget": 0.33, "threshold": -0.4575710352363224, "operator": "ge"}, {"budget": 0.34, "threshold": -0.46697914180249106, "operator": "ge"}, {"budget": 0.35, "threshold": -0.4761627249040435, "operator": "ge"}, {"budget": 0.36, "threshold": -0.4850233480725178, "operator": "ge"}, {"budget": 0.37, "threshold": -0.49253568894865435, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5007069793838866, "operator": "ge"}, {"budget": 0.39, "threshold": -0.509633354467415, "operator": "ge"}, {"budget": 0.4, "threshold": -0.5186361898619004, "operator": "ge"}, {"budget": 0.41, "threshold": -0.5270289762977454, "operator": "ge"}, {"budget": 0.42, "threshold": -0.535135733842065, "operator": "ge"}, {"budget": 0.43, "threshold": -0.5432311572257519, "operator": "ge"}, {"budget": 0.44, "threshold": -0.5515578144740926, "operator": "ge"}, {"budget": 0.45, "threshold": -0.5588359971149406, "operator": "ge"}, {"budget": 0.46, "threshold": -0.5662171948866788, "operator": "ge"}, {"budget": 0.47, "threshold": -0.5741042821418358, "operator": "ge"}, {"budget": 0.48, "threshold": -0.5812564189956474, "operator": "ge"}, {"budget": 0.49, "threshold": -0.5878379488237409, "operator": "ge"}, {"budget": 0.5, "threshold": -0.5952990420770383, "operator": "ge"}, {"budget": 0.51, "threshold": -0.6030123888927473, "operator": "ge"}, {"budget": 0.52, "threshold": -0.6117452578306238, "operator": "ge"}, {"budget": 0.53, "threshold": -0.6189145351491524, "operator": "ge"}, {"budget": 0.54, "threshold": -0.6270213458465284, "operator": "ge"}, {"budget": 0.55, "threshold": -0.6349632761681487, "operator": "ge"}, {"budget": 0.56, "threshold": -0.6434040636482603, "operator": "ge"}, {"budget": 0.57, "threshold": -0.6502113502880903, "operator": "ge"}, {"budget": 0.58, "threshold": -0.657713251279608, "operator": "ge"}, {"budget": 0.59, "threshold": -0.6654872027008538, "operator": "ge"}, {"budget": 0.6, "threshold": -0.6743568517010036, "operator": "ge"}, {"budget": 0.61, "threshold": -0.6823062845601301, "operator": "ge"}, {"budget": 0.62, "threshold": -0.6901843662142952, "operator": "ge"}, {"budget": 0.63, "threshold": -0.6969479929088369, "operator": "ge"}, {"budget": 0.64, "threshold": -0.7040830050815202, "operator": "ge"}, {"budget": 0.65, "threshold": -0.7108762691718895, "operator": "ge"}, {"budget": 0.66, "threshold": -0.7189716273629391, "operator": "ge"}, {"budget": 0.67, "threshold": -0.7253217120529616, "operator": "ge"}, {"budget": 0.68, "threshold": -0.7326506648776747, "operator": "ge"}, {"budget": 0.69, "threshold": -0.739517004101993, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7462735908478872, "operator": "ge"}, {"budget": 0.71, "threshold": -0.7527609277155212, "operator": "ge"}, {"budget": 0.72, "threshold": -0.7583540591490249, "operator": "ge"}, {"budget": 0.73, "threshold": -0.7651627453788583, "operator": "ge"}, {"budget": 0.74, "threshold": -0.7716395250786607, "operator": "ge"}, {"budget": 0.75, "threshold": -0.779294840657563, "operator": "ge"}, {"budget": 0.76, "threshold": -0.7864279760940639, "operator": "ge"}, {"budget": 0.77, "threshold": -0.7928649637192149, "operator": "gt"}, {"budget": 0.78, "threshold": -0.8002489844127998, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8081082913876372, "operator": "ge"}, {"budget": 0.8, "threshold": -0.815345193680177, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8244740387758175, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8324138806041156, "operator": "ge"}, {"budget": 0.83, "threshold": -0.840269465400574, "operator": "ge"}, {"budget": 0.84, "threshold": -0.848177154195927, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8555000463377284, "operator": "ge"}, {"budget": 0.86, "threshold": -0.862978302604714, "operator": "ge"}, {"budget": 0.87, "threshold": -0.8700560415216452, "operator": "ge"}, {"budget": 0.88, "threshold": -0.8792889200711369, "operator": "ge"}, {"budget": 0.89, "threshold": -0.886836953919397, "operator": "ge"}, {"budget": 0.9, "threshold": -0.8947858678292017, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9032137253714742, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9096927657019206, "operator": "ge"}, {"budget": 0.93, "threshold": -0.917394670763701, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9255454969262188, "operator": "ge"}, {"budget": 0.95, "threshold": -0.933469753349035, "operator": "gt"}, {"budget": 0.96, "threshold": -0.9419257497264331, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9504341870516765, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9599076106630247, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9688850113338117, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.07906388361796, 75.14934289127838, 75.14934289127838, 75.2899009065992, 75.40234731885586, 75.39531941808981, 75.26881720430107, 75.1704265935765, 75.16339869281046, 75.19853819664067, 75.18448239510859, 75.2196218989388, 75.23367770047086, 75.254761402769, 75.26178930353502, 75.24070560123691, 75.254761402769, 75.23367770047086, 75.23367770047086, 75.26178930353502, 75.2899009065992, 75.30395670813128, 75.32504041042941, 75.29692880736523, 75.32504041042941, 75.31098460889733, 75.28287300583317, 75.31098460889733, 75.34612411272754, 75.33206831119544, 75.35315201349357, 75.33909621196149, 75.31098460889733, 75.29692880736523, 75.26881720430107, 75.24070560123691, 75.23367770047086, 75.26178930353502, 75.19151029587462, 75.16339869281046, 75.19853819664067, 75.2055660974067, 75.1704265935765, 75.14231499051233, 75.15637079204441, 75.10014758591609, 75.07203598285193, 75.0509522805538, 74.94553376906318, 74.90336636446693, 74.86822686063674, 74.81903155527444, 74.7768641506782, 74.69252934148571, 74.6081945322932, 74.54494342539884, 74.51683182233467, 74.44655281467426, 74.44655281467426, 74.37627380701385, 74.26382739475719, 74.12326937943637, 74.07407407407408, 74.10921357790427, 73.99676716564763, 73.84215334879471, 73.66645582964368, 73.54698151662099, 73.44156300513038, 73.37831189823599, 73.16747487525477, 72.94258205074145, 72.69660552393, 72.44360109635252, 72.2187082718392, 72.0289549511561, 71.96570384426172, 71.78297842434465, 71.60728090519362, 71.31913697378593, 71.12235575233677, 70.91151872935554, 70.65148640101202, 70.3914540726685, 70.08925433972873, 69.87841731674749, 69.68866399606438, 69.41457586618877, 69.05615292712068, 68.7539531941809, 68.37444655281467, 68.02305151451262, 67.57326586548598, 67.20078712488579, 66.69477826973083, 66.1606578115117, 65.73898376554924, 65.32433762035281, 64.75507765830346, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.9898095438892405, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.992550425187996, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.99065289198117, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.98910675381264, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 24, 50, 72, 99, 119, 138, 150, 165, 181, 193, 201, 211, 216, 223, 230, 236, 242, 246, 251, 257, 267, 272, 279, 284, 291, 298, 303, 313, 322, 333, 344, 349, 355, 363, 372, 380, 386, 399, 406, 411, 427, 435, 447, 462, 474, 490, 500, 510, 518, 534, 544, 561, 574, 589, 601, 614, 628, 645, 670, 689, 703, 713, 731, 757, 775, 789, 804, 820, 837, 858, 869, 881, 895, 907, 923, 942, 967, 987, 1005, 1018, 1039, 1064, 1081, 1100, 1118, 1139, 1168, 1185, 1201, 1221, 1235, 1256, 1269, 1286, 1303, 1316, 1336, 1349, 1357, 1369], "harmed": [0, 17, 33, 55, 62, 66, 86, 116, 145, 162, 169, 179, 184, 187, 191, 197, 206, 210, 217, 222, 224, 230, 233, 237, 246, 249, 258, 267, 273, 277, 290, 298, 305, 315, 325, 338, 350, 357, 366, 383, 392, 403, 410, 427, 446, 456, 480, 494, 507, 530, 552, 567, 591, 610, 637, 661, 683, 701, 728, 753, 782, 812, 842, 867, 888, 922, 958, 998, 1031, 1063, 1093, 1134, 1178, 1227, 1275, 1323, 1369, 1403, 1449, 1492, 1546, 1595, 1650, 1704, 1760, 1821, 1872, 1928, 1984, 2051, 2114, 2182, 2253, 2330, 2400, 2489, 2578, 2658, 2730, 2819, 2921], "exact_score": [75.02986857825567, 75.07906388361796, 75.14934289127838, 75.14934289127838, 75.2899009065992, 75.40234731885586, 75.39531941808981, 75.26881720430107, 75.1704265935765, 75.16339869281046, 75.19853819664067, 75.18448239510859, 75.2196218989388, 75.23367770047086, 75.24773350200296, 75.26178930353502, 75.24070560123691, 75.254761402769, 75.23367770047086, 75.23367770047086, 75.26178930353502, 75.2899009065992, 75.30395670813128, 75.32504041042941, 75.29692880736523, 75.32504041042941, 75.31098460889733, 75.28287300583317, 75.31098460889733, 75.34612411272754, 75.33206831119544, 75.35315201349357, 75.33909621196149, 75.31098460889733, 75.29692880736523, 75.26881720430107, 75.24070560123691, 75.23367770047086, 75.26178930353502, 75.19151029587462, 75.16339869281046, 75.19853819664067, 75.2055660974067, 75.1704265935765, 75.14231499051233, 75.15637079204441, 75.10014758591609, 75.07203598285193, 75.0509522805538, 74.94553376906318, 74.90336636446693, 74.86822686063674, 74.81903155527444, 74.7768641506782, 74.69252934148571, 74.6081945322932, 74.54494342539884, 74.51683182233467, 74.44655281467426, 74.44655281467426, 74.37627380701385, 74.26382739475719, 74.12326937943637, 74.07407407407408, 74.10921357790427, 73.99676716564763, 73.84215334879471, 73.66645582964368, 73.54698151662099, 73.44156300513038, 73.37831189823599, 73.16747487525477, 72.94258205074145, 72.69660552393, 72.44360109635252, 72.2187082718392, 72.0289549511561, 71.96570384426172, 71.78297842434465, 71.60728090519362, 71.31913697378593, 71.12235575233677, 70.91151872935554, 70.65148640101202, 70.3914540726685, 70.08925433972873, 69.87841731674749, 69.68866399606438, 69.41457586618877, 69.05615292712068, 68.7539531941809, 68.37444655281467, 68.02305151451262, 67.57326586548598, 67.20078712488579, 66.69477826973083, 66.1606578115117, 65.73898376554924, 65.32433762035281, 64.75507765830346, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 73.37420105291478, "threshold_area100": 73.37465738983765, "threshold_area25": 75.26038372338184}, "test": {"score": [66.95119530385956, 67.29716700224031, 67.54955619204264, 67.82746788418456, 67.95791622947566, 68.07985707398689, 68.12523041147945, 68.08269290758018, 68.06851373961376, 68.03731957008763, 68.04015540368091, 68.0600062388339, 68.0600062388339, 68.08269290758018, 68.07985707398689, 68.07985707398689, 68.09120040836004, 68.08269290758018, 68.05149873805405, 68.01746873493462, 67.98343873181521, 67.96358789666222, 67.9295578935428, 67.90970705838981, 67.88702038964354, 67.8955278904234, 67.8756770552704, 67.81612454981142, 67.79343788106513, 67.77925871309871, 67.77075121231886, 67.75090037716586, 67.7083628732666, 67.70269120608002, 67.68000453733374, 67.68284037092704, 67.62328786546806, 67.58075036156879, 67.5041828545501, 67.46164535065083, 67.39358534441199, 67.37940617644557, 67.31418200380001, 67.24045033037461, 67.18089782491563, 67.13836032101636, 67.10433031789694, 66.99373280775885, 66.91432946714687, 66.86612029606103, 66.78955278904233, 66.69597028046394, 66.56268610157956, 66.5002977625273, 66.42940192269518, 66.33581941411678, 66.20537106882568, 66.09477355868756, 65.92178770949721, 65.78850353061281, 65.61835351501574, 65.44536766582537, 65.26103848226185, 65.1419334713439, 64.97178345574682, 64.77894677140345, 64.54357258316081, 64.40178090349659, 64.22028755352636, 63.950883362164305, 63.724016674701524, 63.50849332161188, 63.27028329977596, 63.012222442787056, 62.759833252984706, 62.50460822958909, 62.19833820151434, 61.93744151093213, 61.62266398207753, 61.25967728213708, 60.94206391968919, 60.488330544763635, 60.0487763378045, 59.58086379491251, 59.16966792388622, 58.74145705130023, 58.355783682613506, 57.933244477214075, 57.35757025777727, 56.88398604769872, 56.33950599778805, 55.865921787709496, 55.463233417463066, 54.96129087145166, 54.35442248248873, 53.73337492555936, 53.14068570456285, 52.34665229844313, 51.609335564189095, 51.008138842412734, 50.00708958398321], "calls": [0.0, 1.8829935059410714, 3.482403652553668, 5.348382156935031, 7.058389813685732, 10.09840342568698, 11.848112752743669, 12.993789524430706, 13.997674616453507, 15.211411394379379, 16.788134872245696, 17.965005813458866, 19.04262257890707, 20.154269347474692, 21.192184442616906, 22.096815358874743, 23.012789609505717, 23.676374670334344, 24.546975583472765, 25.31265065365964, 26.058474888693535, 26.64549244250347, 27.292062501772396, 27.913110058701758, 28.474605110172135, 29.087145166321644, 29.56923687718005, 30.190284434109405, 30.703570314493945, 31.28491620111732, 31.74715707682273, 32.21506961971471, 32.824773842270936, 33.25582054845022, 33.91940560927885, 34.5234381646485, 35.13881405439129, 35.74851827694751, 36.45464084167541, 37.121061736097325, 37.7761392961461, 38.47375436009415, 39.19122025919519, 39.9342086606358, 40.6204803902107, 41.301080452599045, 41.99018801576723, 42.79272892266682, 43.56407566004027, 44.27870572554802, 45.04154496214162, 45.83841420185463, 46.794090122791594, 47.57961602813147, 48.458724442049736, 49.27828035050903, 50.276493775345266, 51.172617190823246, 52.108442276607214, 53.09247653347702, 54.32889998014917, 55.36965090888467, 56.302640161075345, 57.23562941326603, 58.231007004508974, 59.16966792388622, 60.414598871338235, 61.34758812352891, 62.37699571789127, 63.40923914584692, 64.46133340895557, 65.59566684626947, 66.46059609222131, 67.5580636928225, 68.61299378952444, 69.79270056433089, 70.80509315713354, 71.81181408274962, 73.00002835833594, 74.17973513314239, 75.4048152454414, 76.80004537333748, 78.07049882312907, 79.4572214502453, 80.77871990471598, 81.94424751155603, 83.04171511215722, 84.17321271587784, 85.6308311828262, 86.83038879278564, 88.06397640586451, 89.26353401582395, 90.22488160394748, 91.47832005217934, 92.66653432776565, 93.87743527209823, 95.08550038283754, 96.37580466778209, 97.59804894648782, 98.58491903695091, 100.0], "rescued": [0, 180, 315, 446, 520, 582, 642, 687, 733, 774, 793, 815, 830, 851, 866, 879, 892, 900, 909, 923, 934, 942, 951, 963, 975, 992, 999, 1004, 1013, 1024, 1037, 1052, 1065, 1079, 1096, 1113, 1126, 1142, 1154, 1170, 1187, 1205, 1221, 1234, 1248, 1264, 1275, 1287, 1305, 1322, 1342, 1356, 1378, 1397, 1419, 1441, 1469, 1492, 1505, 1528, 1563, 1586, 1608, 1631, 1660, 1680, 1713, 1736, 1759, 1778, 1807, 1836, 1852, 1881, 1916, 1946, 1972, 1996, 2029, 2063, 2099, 2132, 2157, 2191, 2229, 2262, 2294, 2320, 2359, 2391, 2423, 2453, 2478, 2504, 2521, 2540, 2565, 2582, 2599, 2613, 2627], "harmed": [0, 58, 104, 137, 165, 184, 228, 288, 339, 391, 409, 424, 439, 452, 468, 481, 490, 501, 521, 547, 570, 585, 606, 625, 645, 659, 673, 699, 716, 732, 748, 770, 798, 814, 839, 855, 889, 920, 959, 990, 1031, 1054, 1093, 1132, 1167, 1198, 1221, 1272, 1318, 1352, 1399, 1446, 1515, 1556, 1603, 1658, 1732, 1794, 1868, 1938, 2033, 2117, 2204, 2269, 2358, 2446, 2562, 2635, 2722, 2836, 2945, 3050, 3150, 3270, 3394, 3514, 3648, 3764, 3908, 4070, 4218, 4411, 4591, 4790, 4973, 5157, 5325, 5500, 5742, 5941, 6165, 6362, 6529, 6732, 6963, 7201, 7435, 7732, 8009, 8235, 8602], "exact_score": [66.95119530385956, 67.14403198820294, 67.30567450302016, 67.4843320193971, 67.64313870062104, 67.77642287950543, 67.88985622323682, 67.95508039588238, 68.0429912372742, 68.0600062388339, 68.08269290758018, 68.08552874117346, 68.11672291069961, 68.07985707398689, 68.06851373961376, 68.04015540368091, 68.04582707086749, 68.0429912372742, 68.06284207242719, 68.0600062388339, 68.0770212403936, 68.07418540680032, 68.08269290758018, 68.09120040836004, 68.08269290758018, 68.01179706774806, 67.98911039900179, 67.95508039588238, 67.90970705838981, 67.8955278904234, 67.82746788418456, 67.79343788106513, 67.762243711539, 67.69701953889346, 67.68000453733374, 67.64313870062104, 67.55239202563594, 67.47582451861724, 67.38224201003885, 67.32836117176643, 67.23194282959476, 67.16671865694921, 67.10149448430366, 66.9823894733857, 66.89447863199388, 66.78104528826249, 66.65626861015795, 66.55985026798628, 66.46059609222131, 66.3698494172362, 66.25358023991153, 66.12880356180699, 65.93596687746363, 65.80268269857925, 65.65521935172843, 65.4935768369112, 65.32342682131413, 65.1787993080566, 65.0285001276125, 64.82999177608258, 64.60879675580637, 64.44148257380256, 64.29401922695176, 64.05013753792927, 63.84312168561949, 63.607747497376856, 63.375209142727506, 63.11147661855202, 62.878938263902675, 62.686101579559306, 62.44221989053682, 62.13878569605535, 61.88639650625301, 61.62266398207753, 61.31072228681621, 61.04131809545416, 60.80310807361824, 60.423106372118085, 60.08280634092391, 59.73683464254318, 59.416385446502, 59.08742874968097, 58.71309871536739, 58.364291183393355, 58.00130448345291, 57.629810282732606, 57.207271077333175, 56.81025437427331, 56.36219266653433, 55.97368346425432, 55.568159260414596, 55.156963389388316, 54.69472251368289, 54.20128746845134, 53.67382242010039, 53.177551541275555, 52.576354819499194, 51.96665059694296, 51.354110540793464, 50.72455548308425, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.87633418409118, "threshold_area100": 64.42754445169156, "threshold_area25": 67.92133397612228}, "selected_budget": 5, "selected": {"val": {"score": 75.40234731885586, "calls": 4.9898095438892405, "rescued": 119, "harmed": 66}, "test": {"score": 68.07985707398689, "calls": 10.09840342568698, "rescued": 582, "harmed": 184}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.23509226739406586, "operator": "ge"}, {"budget": 0.02, "threshold": 0.11479708552360535, "operator": "ge"}, {"budget": 0.03, "threshold": 0.05327939987182617, "operator": "ge"}, {"budget": 0.04, "threshold": 0.02226780354976654, "operator": "ge"}, {"budget": 0.05, "threshold": 0.003426128998398781, "operator": "ge"}, {"budget": 0.06, "threshold": -0.0026292293332517147, "operator": "ge"}, {"budget": 0.07, "threshold": -0.003961257170885801, "operator": "ge"}, {"budget": 0.08, "threshold": -0.005439733155071735, "operator": "ge"}, {"budget": 0.09, "threshold": -0.006962636020034552, "operator": "ge"}, {"budget": 0.1, "threshold": -0.008565680123865604, "operator": "ge"}, {"budget": 0.11, "threshold": -0.010184178128838539, "operator": "ge"}, {"budget": 0.12, "threshold": -0.011451071128249168, "operator": "ge"}, {"budget": 0.13, "threshold": -0.01333847176283598, "operator": "ge"}, {"budget": 0.14, "threshold": -0.015354275703430176, "operator": "ge"}, {"budget": 0.15, "threshold": -0.017394227907061577, "operator": "ge"}, {"budget": 0.16, "threshold": -0.01954074762761593, "operator": "ge"}, {"budget": 0.17, "threshold": -0.021936673671007156, "operator": "ge"}, {"budget": 0.18, "threshold": -0.024444345384836197, "operator": "ge"}, {"budget": 0.19, "threshold": -0.026809418573975563, "operator": "ge"}, {"budget": 0.2, "threshold": -0.029522622004151344, "operator": "ge"}, {"budget": 0.21, "threshold": -0.03233795240521431, "operator": "ge"}, {"budget": 0.22, "threshold": -0.03533323109149933, "operator": "ge"}, {"budget": 0.23, "threshold": -0.03845459222793579, "operator": "ge"}, {"budget": 0.24, "threshold": -0.04221322014927864, "operator": "ge"}, {"budget": 0.25, "threshold": -0.0462757870554924, "operator": "ge"}, {"budget": 0.26, "threshold": -0.050889864563941956, "operator": "ge"}, {"budget": 0.27, "threshold": -0.055297162383794785, "operator": "ge"}, {"budget": 0.28, "threshold": -0.05990667641162872, "operator": "ge"}, {"budget": 0.29, "threshold": -0.06434302777051926, "operator": "ge"}, {"budget": 0.3, "threshold": -0.07027000188827515, "operator": "ge"}, {"budget": 0.31, "threshold": -0.07556195557117462, "operator": "ge"}, {"budget": 0.32, "threshold": -0.08136012405157089, "operator": "ge"}, {"budget": 0.33, "threshold": -0.08783522993326187, "operator": "ge"}, {"budget": 0.34, "threshold": -0.09421156346797943, "operator": "ge"}, {"budget": 0.35, "threshold": -0.10062450170516968, "operator": "ge"}, {"budget": 0.36, "threshold": -0.10740188509225845, "operator": "ge"}, {"budget": 0.37, "threshold": -0.11453241109848022, "operator": "ge"}, {"budget": 0.38, "threshold": -0.12085147202014923, "operator": "ge"}, {"budget": 0.39, "threshold": -0.12807175517082214, "operator": "ge"}, {"budget": 0.4, "threshold": -0.13567079603672028, "operator": "ge"}, {"budget": 0.41, "threshold": -0.14514976739883423, "operator": "ge"}, {"budget": 0.42, "threshold": -0.1526041030883789, "operator": "ge"}, {"budget": 0.43, "threshold": -0.1611805558204651, "operator": "ge"}, {"budget": 0.44, "threshold": -0.17025446891784668, "operator": "ge"}, {"budget": 0.45, "threshold": -0.17924568057060242, "operator": "ge"}, {"budget": 0.46, "threshold": -0.18767361342906952, "operator": "ge"}, {"budget": 0.47, "threshold": -0.19730253517627716, "operator": "ge"}, {"budget": 0.48, "threshold": -0.20814108848571777, "operator": "ge"}, {"budget": 0.49, "threshold": -0.21905384957790375, "operator": "ge"}, {"budget": 0.5, "threshold": -0.23016369342803955, "operator": "ge"}, {"budget": 0.51, "threshold": -0.23905602097511292, "operator": "gt"}, {"budget": 0.52, "threshold": -0.24700933694839478, "operator": "ge"}, {"budget": 0.53, "threshold": -0.2562357485294342, "operator": "ge"}, {"budget": 0.54, "threshold": -0.26776623725891113, "operator": "ge"}, {"budget": 0.55, "threshold": -0.2769918143749237, "operator": "ge"}, {"budget": 0.56, "threshold": -0.28995123505592346, "operator": "ge"}, {"budget": 0.57, "threshold": -0.2997936010360718, "operator": "ge"}, {"budget": 0.58, "threshold": -0.31127986311912537, "operator": "ge"}, {"budget": 0.59, "threshold": -0.3244946599006653, "operator": "ge"}, {"budget": 0.6, "threshold": -0.3344406485557556, "operator": "ge"}, {"budget": 0.61, "threshold": -0.3464459776878357, "operator": "ge"}, {"budget": 0.62, "threshold": -0.3577840328216553, "operator": "ge"}, {"budget": 0.63, "threshold": -0.3691169023513794, "operator": "ge"}, {"budget": 0.64, "threshold": -0.38094496726989746, "operator": "ge"}, {"budget": 0.65, "threshold": -0.3941086530685425, "operator": "ge"}, {"budget": 0.66, "threshold": -0.4060373902320862, "operator": "ge"}, {"budget": 0.67, "threshold": -0.41692033410072327, "operator": "ge"}, {"budget": 0.68, "threshold": -0.4278205931186676, "operator": "ge"}, {"budget": 0.69, "threshold": -0.44030147790908813, "operator": "ge"}, {"budget": 0.7, "threshold": -0.4527391791343689, "operator": "ge"}, {"budget": 0.71, "threshold": -0.46733686327934265, "operator": "ge"}, {"budget": 0.72, "threshold": -0.48290151357650757, "operator": "ge"}, {"budget": 0.73, "threshold": -0.497890830039978, "operator": "ge"}, {"budget": 0.74, "threshold": -0.5133432149887085, "operator": "ge"}, {"budget": 0.75, "threshold": -0.5274530053138733, "operator": "ge"}, {"budget": 0.76, "threshold": -0.5439835786819458, "operator": "ge"}, {"budget": 0.77, "threshold": -0.5579758286476135, "operator": "ge"}, {"budget": 0.78, "threshold": -0.5715032815933228, "operator": "ge"}, {"budget": 0.79, "threshold": -0.5854043960571289, "operator": "gt"}, {"budget": 0.8, "threshold": -0.6005325317382812, "operator": "ge"}, {"budget": 0.81, "threshold": -0.6142565608024597, "operator": "ge"}, {"budget": 0.82, "threshold": -0.6277443766593933, "operator": "ge"}, {"budget": 0.83, "threshold": -0.642208456993103, "operator": "ge"}, {"budget": 0.84, "threshold": -0.6552051901817322, "operator": "ge"}, {"budget": 0.85, "threshold": -0.6708604097366333, "operator": "ge"}, {"budget": 0.86, "threshold": -0.6842661499977112, "operator": "ge"}, {"budget": 0.87, "threshold": -0.6988332271575928, "operator": "ge"}, {"budget": 0.88, "threshold": -0.7126888036727905, "operator": "ge"}, {"budget": 0.89, "threshold": -0.7275274991989136, "operator": "ge"}, {"budget": 0.9, "threshold": -0.745084822177887, "operator": "ge"}, {"budget": 0.91, "threshold": -0.7633023858070374, "operator": "ge"}, {"budget": 0.92, "threshold": -0.7785232663154602, "operator": "ge"}, {"budget": 0.93, "threshold": -0.7943630218505859, "operator": "ge"}, {"budget": 0.94, "threshold": -0.8116153478622437, "operator": "ge"}, {"budget": 0.95, "threshold": -0.8297523856163025, "operator": "ge"}, {"budget": 0.96, "threshold": -0.8460037708282471, "operator": "ge"}, {"budget": 0.97, "threshold": -0.8673251867294312, "operator": "ge"}, {"budget": 0.98, "threshold": -0.8946853876113892, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9210361242294312, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.1212312882142, 75.16339869281046, 75.17745449434254, 75.30395670813128, 75.40234731885586, 75.43045892192002, 75.43748682268607, 75.43045892192002, 75.43045892192002, 75.42343102115399, 75.41640312038794, 75.39531941808981, 75.38829151732378, 75.3742357157917, 75.36720781502565, 75.34612411272754, 75.33909621196149, 75.31801250966336, 75.31098460889733, 75.27584510506712, 75.24773350200296, 75.24070560123691, 75.2055660974067, 75.19853819664067, 75.17745449434254, 75.1352870897463, 75.09311968515004, 75.02986857825567, 74.98067327289338, 74.9666174713613, 74.92445006676506, 74.8822826621688, 74.81903155527444, 74.80497575374235, 74.76280834914611, 74.70658514301778, 74.65036193688945, 74.55899922693092, 74.49574812003654, 74.39735750931197, 74.31302270011948, 74.2146320893949, 74.13732518096845, 74.0108229671797, 73.94054395951929, 73.89134865415701, 73.72970693653805, 73.65240002811161, 73.53995361585494, 73.35722819593788, 73.31506079134162, 73.3431723944058, 73.27992128751141, 73.18855857755288, 73.18153067678685, 73.06908426453018, 72.94258205074145, 72.92852624920936, 72.87933094384708, 72.72471712699416, 72.61929861550355, 72.57010331014126, 72.48576850094877, 72.4576568978846, 72.35926628716003, 72.38034998945815, 72.19059666877504, 72.07815025651838, 72.07112235575234, 72.00084334809192, 71.8743411343032, 71.69161571438612, 71.55808559983133, 71.39644388221238, 71.2277742638274, 71.05207674467636, 70.91854663012158, 70.7498770117366, 70.65851430177807, 70.57417949258557, 70.40550987420058, 70.2368402558156, 70.0330311336004, 69.92761262210978, 69.73083140066062, 69.54810598074356, 69.40052006465669, 69.09832033171692, 68.90856701103381, 68.69070208728652, 68.26200014055802, 67.90357720148992, 67.51704265935766, 67.17267552182163, 66.77208517815728, 66.40663433832314, 65.91468128470026, 65.40164452877926, 64.82535666596388, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.98742005762878, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.98657670953686, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 27, 49, 75, 99, 119, 124, 125, 126, 126, 126, 126, 127, 127, 127, 129, 129, 130, 130, 131, 132, 134, 137, 140, 143, 144, 146, 150, 150, 154, 158, 163, 166, 171, 175, 182, 190, 193, 195, 200, 204, 210, 216, 223, 228, 238, 252, 258, 271, 283, 292, 308, 329, 343, 355, 375, 392, 406, 430, 450, 465, 483, 506, 529, 555, 580, 610, 630, 646, 674, 702, 727, 745, 772, 794, 817, 843, 870, 895, 924, 958, 990, 1018, 1046, 1073, 1102, 1132, 1165, 1185, 1212, 1232, 1243, 1262, 1275, 1294, 1314, 1330, 1343, 1354, 1363, 1369], "harmed": [0, 14, 30, 54, 60, 66, 67, 67, 69, 69, 70, 71, 75, 76, 78, 81, 84, 86, 89, 91, 97, 103, 107, 115, 119, 123, 131, 141, 150, 161, 167, 178, 187, 201, 207, 220, 236, 247, 262, 276, 294, 312, 332, 350, 373, 393, 414, 443, 467, 495, 530, 552, 569, 592, 617, 638, 671, 703, 729, 756, 793, 826, 856, 891, 921, 960, 987, 1034, 1066, 1095, 1133, 1176, 1220, 1266, 1311, 1358, 1409, 1455, 1504, 1546, 1592, 1648, 1700, 1757, 1799, 1856, 1912, 1966, 2029, 2083, 2134, 2206, 2276, 2344, 2412, 2489, 2557, 2640, 2724, 2815, 2921], "exact_score": [75.02986857825567, 75.1212312882142, 75.16339869281046, 75.17745449434254, 75.30395670813128, 75.40234731885586, 75.43045892192002, 75.43748682268607, 75.43045892192002, 75.43045892192002, 75.42343102115399, 75.41640312038794, 75.39531941808981, 75.38829151732378, 75.3742357157917, 75.36720781502565, 75.34612411272754, 75.33909621196149, 75.31801250966336, 75.31098460889733, 75.27584510506712, 75.24773350200296, 75.24070560123691, 75.2055660974067, 75.19853819664067, 75.17745449434254, 75.1352870897463, 75.09311968515004, 75.02986857825567, 74.98067327289338, 74.9666174713613, 74.92445006676506, 74.8822826621688, 74.81903155527444, 74.80497575374235, 74.76280834914611, 74.70658514301778, 74.65036193688945, 74.55899922693092, 74.49574812003654, 74.39735750931197, 74.31302270011948, 74.2146320893949, 74.13732518096845, 74.0108229671797, 73.94054395951929, 73.89134865415701, 73.72970693653805, 73.65240002811161, 73.53995361585494, 73.35722819593788, 73.30803289057559, 73.3431723944058, 73.27992128751141, 73.18855857755288, 73.18153067678685, 73.06908426453018, 72.94258205074145, 72.92852624920936, 72.87933094384708, 72.72471712699416, 72.61929861550355, 72.57010331014126, 72.48576850094877, 72.4576568978846, 72.35926628716003, 72.38034998945815, 72.19059666877504, 72.07815025651838, 72.07112235575234, 72.00084334809192, 71.8743411343032, 71.69161571438612, 71.55808559983133, 71.39644388221238, 71.2277742638274, 71.05207674467636, 70.91854663012158, 70.7498770117366, 70.65851430177807, 70.57417949258557, 70.40550987420058, 70.2368402558156, 70.0330311336004, 69.92761262210978, 69.73083140066062, 69.54810598074356, 69.40052006465669, 69.09832033171692, 68.90856701103381, 68.69070208728652, 68.26200014055802, 67.90357720148992, 67.51704265935766, 67.17267552182163, 66.77208517815728, 66.40663433832314, 65.91468128470026, 65.40164452877926, 64.82535666596388, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 72.7537192568052, "threshold_area100": 72.75416403120387, "threshold_area25": 75.31449855928034}, "test": {"score": [66.95119530385956, 67.29716700224031, 67.61194453109492, 67.82179621699798, 67.98060289822193, 68.07985707398689, 68.07418540680032, 68.07134957320704, 68.06851373961376, 68.06851373961376, 68.06851373961376, 68.0770212403936, 68.0770212403936, 68.07985707398689, 68.07985707398689, 68.10254374273318, 68.0997079091399, 68.10254374273318, 68.08836457476674, 68.07985707398689, 68.06284207242719, 68.06567790602047, 68.05717040524061, 68.05149873805405, 68.04015540368091, 68.00612540056149, 68.00045373337493, 68.00612540056149, 67.98060289822193, 67.9295578935428, 67.88702038964354, 67.8388112185577, 67.81045288262484, 67.74806454357258, 67.73672120919944, 67.70269120608002, 67.66582536936733, 67.62612369906134, 67.56089952641578, 67.49851118736352, 67.47298868502396, 67.36239117488586, 67.28298783427388, 67.22910699600148, 67.13552448742307, 67.05895698040439, 66.89731446558716, 66.82641862575504, 66.69029861327736, 66.60805943907211, 66.42089442191532, 66.3499985820832, 66.25358023991153, 66.13163939540027, 65.98134021495619, 65.89910104075093, 65.66656268610159, 65.54178600799705, 65.34044182287383, 65.16462014009018, 64.9746192893401, 64.86402177920199, 64.71088676516462, 64.62581175736608, 64.45566174176899, 64.26282505742563, 64.0813317074554, 63.8970025238919, 63.7098375067351, 63.46028415052605, 63.19938745994384, 62.983864106854206, 62.60102657176076, 62.286249042906164, 61.97147151405156, 61.67370898675666, 61.22564727901767, 60.90519808297648, 60.49683804554349, 60.1962396846553, 59.804894648782, 59.52414712304682, 59.22638459575191, 58.81518872472563, 58.46354535915832, 58.123245327964156, 57.811303632702824, 57.47100360150866, 57.08249439922866, 56.65428352664266, 56.10129597595213, 55.52278592292203, 54.9896492073845, 54.482034994186535, 53.9318832770893, 53.30799988656665, 52.79754983977541, 52.09426310864078, 51.37963304313302, 50.758585486203664, 50.00708958398321], "calls": [0.0, 1.8971726739074952, 3.7319570087627256, 5.47315883503956, 7.2427189972492405, 10.087060091313841, 12.046621104273601, 13.359612057964437, 14.468422992938773, 15.554547259166831, 16.399625669965687, 17.28724158466381, 17.81754246660806, 18.563366701641947, 19.201429260131015, 19.774267645974533, 20.341434364631482, 20.83486940986303, 21.393528627740125, 21.85860533703882, 22.32368204633752, 22.77741542126308, 23.248163797748347, 23.645180500808213, 24.183988883532315, 24.680259762357146, 25.321158154439498, 25.83160820123075, 26.384595751921275, 26.971613305731218, 27.595496696253864, 28.182514250063807, 28.829084309332732, 29.538042707653915, 30.198791934889265, 30.81983949181862, 31.625216232311487, 32.399398803278224, 33.04596886254715, 33.86552477100644, 34.60567733885376, 35.623741598842976, 36.363894166690294, 37.121061736097325, 38.03420015313501, 38.916144400646566, 39.78958114737827, 40.69704789722939, 41.78884383064402, 42.838102260159374, 43.867509854521735, 44.661543260641466, 45.40453166208207, 46.30632674474662, 47.48319768595979, 48.34812693191164, 49.60156538014349, 50.49201712843491, 51.575305561069676, 52.81740067492839, 53.670986586507105, 54.808155857414285, 56.0133851345603, 56.966225221903976, 58.03533448657233, 59.203697927005635, 60.221762186994866, 61.16042310637212, 62.09908402574938, 63.12565578651844, 64.29118339335848, 65.56163684315004, 67.01358364291184, 68.38612710206165, 69.75867056121146, 70.98942234069705, 72.38465246859315, 73.55018007543318, 74.80645435725832, 76.06556447267674, 77.35303292402801, 78.38527635198366, 79.5111022885177, 80.73051073363014, 81.72872415846638, 82.88007259733999, 83.80455434875081, 84.91336528372516, 85.92859371012109, 87.06009131384171, 88.44397810736466, 89.80234239854805, 90.97354167257465, 92.042650937243, 93.17698437455691, 94.38221365170291, 95.37759124294587, 96.61968635680458, 97.89297564018943, 98.85999489549954, 100.0], "rescued": [0, 178, 339, 443, 527, 582, 589, 591, 593, 597, 600, 604, 605, 611, 613, 623, 628, 632, 639, 644, 648, 650, 655, 662, 666, 668, 677, 687, 693, 699, 710, 719, 728, 737, 755, 762, 774, 788, 799, 814, 829, 851, 862, 875, 897, 911, 925, 946, 971, 1004, 1032, 1052, 1069, 1092, 1122, 1148, 1173, 1194, 1217, 1256, 1277, 1329, 1384, 1424, 1460, 1500, 1532, 1567, 1600, 1626, 1666, 1712, 1751, 1800, 1846, 1885, 1925, 1970, 2006, 2050, 2087, 2118, 2158, 2202, 2231, 2277, 2303, 2349, 2374, 2401, 2431, 2464, 2483, 2504, 2527, 2549, 2569, 2590, 2611, 2616, 2627], "harmed": [0, 56, 106, 136, 164, 184, 193, 196, 199, 203, 206, 207, 208, 213, 215, 217, 223, 226, 238, 246, 256, 257, 265, 274, 282, 296, 307, 315, 330, 354, 380, 406, 425, 456, 478, 497, 522, 550, 584, 621, 645, 706, 745, 777, 832, 873, 944, 990, 1063, 1125, 1219, 1264, 1315, 1381, 1464, 1519, 1626, 1691, 1785, 1886, 1974, 2065, 2174, 2244, 2340, 2448, 2544, 2644, 2743, 2857, 2989, 3111, 3285, 3445, 3602, 3746, 3944, 4102, 4282, 4432, 4607, 4737, 4882, 5071, 5224, 5390, 5526, 5692, 5854, 6032, 6257, 6494, 6701, 6901, 7118, 7360, 7560, 7829, 8102, 8326, 8602], "exact_score": [66.95119530385956, 67.13836032101636, 67.3170178373933, 67.5041828545501, 67.64313870062104, 67.78493038028527, 67.88418455605026, 67.96358789666222, 68.0429912372742, 68.0600062388339, 68.08269290758018, 68.07134957320704, 68.07418540680032, 68.0770212403936, 68.07134957320704, 68.06284207242719, 68.06851373961376, 68.0770212403936, 68.08269290758018, 68.07985707398689, 68.10254374273318, 68.0968720755466, 68.06567790602047, 68.05433457164733, 68.03731957008763, 67.99478206618835, 68.00612540056149, 67.9295578935428, 67.86149788730397, 67.782094546692, 67.73955704279273, 67.69418370530018, 67.64313870062104, 67.56089952641578, 67.49283952017694, 67.42194368034482, 67.31418200380001, 67.2432861639679, 67.13268865382979, 67.03343447806482, 66.8717919632476, 66.80089612341547, 66.66477611093781, 66.57970110313927, 66.41805858832204, 66.31029691177721, 66.17417689929954, 66.04372855400845, 65.93313104387035, 65.7743243626464, 65.61268184782917, 65.45103933301193, 65.28088931741485, 65.11924680259762, 64.92641011825427, 64.824320108896, 64.71088676516462, 64.62864759095936, 64.45566174176899, 64.30819839491818, 64.12386921135467, 63.914017525451605, 63.73536000907467, 63.5056574880186, 63.27595496696253, 63.07177494824604, 62.86475909593625, 62.60669823894733, 62.37415988429799, 62.155800697615064, 61.920426509372426, 61.67087315316338, 61.34758812352891, 61.052661429827296, 60.76624223690553, 60.44579304086436, 60.20758301902844, 59.94101466125967, 59.61489379803193, 59.35116127385645, 59.07608541530782, 58.732949550520374, 58.41533618807249, 58.09205115843802, 57.77727362958341, 57.434137764795956, 57.06264356407566, 56.674134361795645, 56.28562515951564, 55.880100955675914, 55.44054674871678, 54.9726342058248, 54.51039333011939, 54.01695828488784, 53.52635907324959, 52.99038652411876, 52.43172730624167, 51.89575475711086, 51.303065536114346, 50.66783881121856, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.60769502363895, "threshold_area100": 64.15921787709497, "threshold_area25": 67.9584833961943}, "selected_budget": 7, "selected": {"val": {"score": 75.43748682268607, "calls": 6.999789162977018, "rescued": 125, "harmed": 67}, "test": {"score": 68.07134957320704, "calls": 13.359612057964437, "rescued": 591, "harmed": 196}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.432985782623291, "operator": "ge"}, {"budget": 0.02, "threshold": 2.7345266342163086, "operator": "ge"}, {"budget": 0.03, "threshold": 1.5168554782867432, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6665180921554565, "operator": "ge"}, {"budget": 0.05, "threshold": 0.0752803236246109, "operator": "ge"}, {"budget": 0.06, "threshold": -0.3990105986595154, "operator": "ge"}, {"budget": 0.07, "threshold": -0.8301525115966797, "operator": "ge"}, {"budget": 0.08, "threshold": -1.1804776191711426, "operator": "ge"}, {"budget": 0.09, "threshold": -1.5244282484054565, "operator": "ge"}, {"budget": 0.1, "threshold": -1.8440021276474, "operator": "ge"}, {"budget": 0.11, "threshold": -2.080580234527588, "operator": "ge"}, {"budget": 0.12, "threshold": -2.322816848754883, "operator": "ge"}, {"budget": 0.13, "threshold": -2.5553648471832275, "operator": "ge"}, {"budget": 0.14, "threshold": -2.7509450912475586, "operator": "ge"}, {"budget": 0.15, "threshold": -2.951807737350464, "operator": "ge"}, {"budget": 0.16, "threshold": -3.145460367202759, "operator": "ge"}, {"budget": 0.17, "threshold": -3.3209798336029053, "operator": "ge"}, {"budget": 0.18, "threshold": -3.4699761867523193, "operator": "gt"}, {"budget": 0.19, "threshold": -3.6168763637542725, "operator": "ge"}, {"budget": 0.2, "threshold": -3.7514986991882324, "operator": "gt"}, {"budget": 0.21, "threshold": -3.8815155029296875, "operator": "ge"}, {"budget": 0.22, "threshold": -3.99088978767395, "operator": "ge"}, {"budget": 0.23, "threshold": -4.092554569244385, "operator": "ge"}, {"budget": 0.24, "threshold": -4.191201686859131, "operator": "ge"}, {"budget": 0.25, "threshold": -4.295077323913574, "operator": "ge"}, {"budget": 0.26, "threshold": -4.397650718688965, "operator": "ge"}, {"budget": 0.27, "threshold": -4.496854782104492, "operator": "ge"}, {"budget": 0.28, "threshold": -4.585165977478027, "operator": "ge"}, {"budget": 0.29, "threshold": -4.6687774658203125, "operator": "ge"}, {"budget": 0.3, "threshold": -4.748768329620361, "operator": "ge"}, {"budget": 0.31, "threshold": -4.832798957824707, "operator": "ge"}, {"budget": 0.32, "threshold": -4.902081489562988, "operator": "ge"}, {"budget": 0.33, "threshold": -4.982451915740967, "operator": "ge"}, {"budget": 0.34, "threshold": -5.042781829833984, "operator": "ge"}, {"budget": 0.35, "threshold": -5.104104042053223, "operator": "ge"}, {"budget": 0.36, "threshold": -5.179507255554199, "operator": "gt"}, {"budget": 0.37, "threshold": -5.2409586906433105, "operator": "ge"}, {"budget": 0.38, "threshold": -5.3011956214904785, "operator": "ge"}, {"budget": 0.39, "threshold": -5.362185955047607, "operator": "ge"}, {"budget": 0.4, "threshold": -5.419180870056152, "operator": "ge"}, {"budget": 0.41, "threshold": -5.464240550994873, "operator": "ge"}, {"budget": 0.42, "threshold": -5.5061445236206055, "operator": "ge"}, {"budget": 0.43, "threshold": -5.559621810913086, "operator": "ge"}, {"budget": 0.44, "threshold": -5.610294342041016, "operator": "ge"}, {"budget": 0.45, "threshold": -5.656978130340576, "operator": "ge"}, {"budget": 0.46, "threshold": -5.700567245483398, "operator": "ge"}, {"budget": 0.47, "threshold": -5.749128341674805, "operator": "ge"}, {"budget": 0.48, "threshold": -5.797772407531738, "operator": "ge"}, {"budget": 0.49, "threshold": -5.839014053344727, "operator": "ge"}, {"budget": 0.5, "threshold": -5.885193824768066, "operator": "gt"}, {"budget": 0.51, "threshold": -5.92584228515625, "operator": "ge"}, {"budget": 0.52, "threshold": -5.965978145599365, "operator": "ge"}, {"budget": 0.53, "threshold": -6.0060319900512695, "operator": "ge"}, {"budget": 0.54, "threshold": -6.052212715148926, "operator": "ge"}, {"budget": 0.55, "threshold": -6.093831539154053, "operator": "ge"}, {"budget": 0.56, "threshold": -6.1380414962768555, "operator": "ge"}, {"budget": 0.57, "threshold": -6.176716327667236, "operator": "ge"}, {"budget": 0.58, "threshold": -6.217168807983398, "operator": "ge"}, {"budget": 0.59, "threshold": -6.252453804016113, "operator": "ge"}, {"budget": 0.6, "threshold": -6.2884697914123535, "operator": "gt"}, {"budget": 0.61, "threshold": -6.329712390899658, "operator": "gt"}, {"budget": 0.62, "threshold": -6.358566761016846, "operator": "ge"}, {"budget": 0.63, "threshold": -6.398166656494141, "operator": "ge"}, {"budget": 0.64, "threshold": -6.440831661224365, "operator": "ge"}, {"budget": 0.65, "threshold": -6.482100009918213, "operator": "ge"}, {"budget": 0.66, "threshold": -6.523586273193359, "operator": "ge"}, {"budget": 0.67, "threshold": -6.557399749755859, "operator": "ge"}, {"budget": 0.68, "threshold": -6.596208572387695, "operator": "ge"}, {"budget": 0.69, "threshold": -6.639351844787598, "operator": "ge"}, {"budget": 0.7, "threshold": -6.680416107177734, "operator": "ge"}, {"budget": 0.71, "threshold": -6.7230424880981445, "operator": "ge"}, {"budget": 0.72, "threshold": -6.759870529174805, "operator": "ge"}, {"budget": 0.73, "threshold": -6.805361747741699, "operator": "ge"}, {"budget": 0.74, "threshold": -6.853259086608887, "operator": "ge"}, {"budget": 0.75, "threshold": -6.894647598266602, "operator": "ge"}, {"budget": 0.76, "threshold": -6.942071437835693, "operator": "ge"}, {"budget": 0.77, "threshold": -6.989616394042969, "operator": "ge"}, {"budget": 0.78, "threshold": -7.0342206954956055, "operator": "ge"}, {"budget": 0.79, "threshold": -7.082003593444824, "operator": "ge"}, {"budget": 0.8, "threshold": -7.129994869232178, "operator": "ge"}, {"budget": 0.81, "threshold": -7.175380229949951, "operator": "ge"}, {"budget": 0.82, "threshold": -7.223580837249756, "operator": "ge"}, {"budget": 0.83, "threshold": -7.280198097229004, "operator": "ge"}, {"budget": 0.84, "threshold": -7.338405609130859, "operator": "ge"}, {"budget": 0.85, "threshold": -7.3938446044921875, "operator": "ge"}, {"budget": 0.86, "threshold": -7.444509506225586, "operator": "ge"}, {"budget": 0.87, "threshold": -7.5037736892700195, "operator": "ge"}, {"budget": 0.88, "threshold": -7.565516471862793, "operator": "ge"}, {"budget": 0.89, "threshold": -7.627552032470703, "operator": "ge"}, {"budget": 0.9, "threshold": -7.7002177238464355, "operator": "ge"}, {"budget": 0.91, "threshold": -7.780788421630859, "operator": "ge"}, {"budget": 0.92, "threshold": -7.86931037902832, "operator": "ge"}, {"budget": 0.93, "threshold": -7.954150676727295, "operator": "ge"}, {"budget": 0.94, "threshold": -8.056906700134277, "operator": "ge"}, {"budget": 0.95, "threshold": -8.168041229248047, "operator": "ge"}, {"budget": 0.96, "threshold": -8.306427001953125, "operator": "ge"}, {"budget": 0.97, "threshold": -8.485522270202637, "operator": "ge"}, {"budget": 0.98, "threshold": -8.710949897766113, "operator": "ge"}, {"budget": 0.99, "threshold": -9.086508750915527, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.34612411272754, 75.47965422728231, 75.54993323494273, 75.54290533417668, 75.41640312038794, 75.34612411272754, 75.34612411272754, 75.26881720430107, 75.23367770047086, 75.28287300583317, 75.31801250966336, 75.36017991425962, 75.53587743341063, 75.60615644107105, 75.6975191510296, 75.76077025792397, 75.85916086864853, 75.90132827324479, 76.01377468550145, 76.0418862885656, 76.0418862885656, 75.99269098320332, 76.04891418933165, 76.0208025862675, 75.96457938013916, 75.90835617401083, 75.80996556328626, 75.77482605945605, 75.6131843418371, 75.5288495326446, 75.50073792958042, 75.30395670813128, 75.16339869281046, 74.99472907442546, 74.86119895987069, 74.71361304378382, 74.58711082999508, 74.44655281467426, 74.2849110970553, 74.16543678403261, 73.99676716564763, 73.89837655492305, 73.7367348373041, 73.55400941738702, 73.39236769976807, 73.16747487525477, 72.99880525686977, 72.81607983695271, 72.61929861550355, 72.49279640171481, 72.30304308103169, 72.17654086724295, 72.00787124885797, 71.81109002740882, 71.68458781362007, 71.48780659217093, 71.2980532714878, 71.11532785157074, 70.98179773701595, 70.84826762246117, 70.64445850024597, 70.48984468339307, 70.36334246960433, 70.19467285121934, 70.07519853819663, 69.99086372900415, 69.8643615152154, 69.66758029376625, 69.52702227844543, 69.36538056082648, 69.16157143861129, 69.09832033171692, 68.95073441563005, 68.80314849954318, 68.61339517886007, 68.53608827043362, 68.36741865204863, 68.19172113289761, 68.0722468198749, 67.90357720148992, 67.70679598004077, 67.60137746855015, 67.45379155246329, 67.31323353714245, 67.08131281186309, 66.95481059807436, 66.74397357509312, 66.54719235364396, 66.35743903296085, 66.209853116874, 66.05523930002109, 65.81629067397569, 65.6827605594209, 65.53517464333403, 65.29622601728863, 65.0361936889451, 64.8113008644318, 64.53721273455619, 64.26312460468057, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99142596106543, 18.996415770609318, 19.987349778621123, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.9898798228969, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.98945814885094, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.99016093892754, 60.9881228477054, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 45, 69, 90, 106, 120, 139, 160, 176, 205, 234, 265, 291, 331, 365, 407, 440, 479, 514, 558, 592, 620, 644, 679, 702, 728, 753, 776, 796, 809, 826, 852, 863, 876, 887, 901, 910, 924, 933, 942, 955, 957, 973, 981, 991, 1001, 1005, 1014, 1019, 1028, 1039, 1046, 1049, 1055, 1057, 1066, 1071, 1077, 1084, 1096, 1102, 1104, 1111, 1115, 1117, 1126, 1133, 1140, 1145, 1147, 1152, 1161, 1173, 1179, 1187, 1190, 1197, 1204, 1208, 1214, 1219, 1223, 1226, 1232, 1236, 1241, 1248, 1253, 1260, 1264, 1272, 1279, 1284, 1292, 1298, 1304, 1308, 1316, 1329, 1340, 1369], "harmed": [0, 0, 5, 16, 33, 65, 94, 115, 142, 176, 198, 224, 244, 259, 283, 312, 336, 361, 390, 418, 448, 476, 507, 534, 561, 595, 628, 665, 690, 726, 755, 785, 824, 857, 892, 925, 955, 987, 1016, 1048, 1078, 1104, 1134, 1165, 1201, 1234, 1270, 1303, 1334, 1371, 1400, 1434, 1455, 1485, 1515, 1542, 1575, 1608, 1641, 1672, 1697, 1728, 1757, 1779, 1805, 1831, 1850, 1875, 1908, 1930, 1958, 1996, 2017, 2044, 2073, 2103, 2121, 2152, 2181, 2204, 2233, 2265, 2283, 2310, 2334, 2372, 2397, 2432, 2467, 2498, 2527, 2556, 2595, 2622, 2649, 2689, 2730, 2770, 2822, 2872, 2921], "exact_score": [75.02986857825567, 75.34612411272754, 75.47965422728231, 75.54993323494273, 75.54290533417668, 75.41640312038794, 75.34612411272754, 75.34612411272754, 75.26881720430107, 75.23367770047086, 75.28287300583317, 75.31801250966336, 75.36017991425962, 75.53587743341063, 75.60615644107105, 75.6975191510296, 75.76077025792397, 75.85916086864853, 75.89430037247874, 76.01377468550145, 76.0418862885656, 76.0418862885656, 75.99269098320332, 76.04891418933165, 76.0208025862675, 75.96457938013916, 75.90835617401083, 75.80996556328626, 75.77482605945605, 75.6131843418371, 75.5288495326446, 75.50073792958042, 75.30395670813128, 75.16339869281046, 74.99472907442546, 74.86119895987069, 74.71361304378382, 74.58711082999508, 74.44655281467426, 74.2849110970553, 74.16543678403261, 73.99676716564763, 73.89837655492305, 73.7367348373041, 73.55400941738702, 73.39236769976807, 73.16747487525477, 72.99880525686977, 72.81607983695271, 72.61929861550355, 72.48576850094877, 72.30304308103169, 72.17654086724295, 72.00787124885797, 71.81109002740882, 71.68458781362007, 71.48780659217093, 71.2980532714878, 71.11532785157074, 70.98179773701595, 70.84826762246117, 70.64445850024597, 70.48984468339307, 70.36334246960433, 70.19467285121934, 70.07519853819663, 69.99086372900415, 69.8643615152154, 69.66758029376625, 69.52702227844543, 69.36538056082648, 69.16157143861129, 69.09832033171692, 68.95073441563005, 68.80314849954318, 68.61339517886007, 68.53608827043362, 68.36741865204863, 68.19172113289761, 68.0722468198749, 67.90357720148992, 67.70679598004077, 67.60137746855015, 67.45379155246329, 67.31323353714245, 67.08131281186309, 66.95481059807436, 66.74397357509312, 66.54719235364396, 66.35743903296085, 66.209853116874, 66.05523930002109, 65.81629067397569, 65.6827605594209, 65.53517464333403, 65.29622601728863, 65.0361936889451, 64.8113008644318, 64.53721273455619, 64.26312460468057, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 71.78428581441618, "threshold_area100": 71.7848056785438, "threshold_area25": 75.58718110900276}, "test": {"score": [66.95119530385956, 67.47866035221053, 67.75657204435244, 67.93806539432265, 68.04015540368091, 68.07418540680032, 68.12523041147945, 68.12239457788617, 68.15642458100558, 68.08269290758018, 68.0770212403936, 68.0770212403936, 68.07985707398689, 68.04015540368091, 67.9692595638488, 67.95508039588238, 67.85299038652411, 67.782094546692, 67.68284037092704, 67.60343703031506, 67.51552618892323, 67.36239117488586, 67.26880866630745, 67.16388282335592, 66.98522530697898, 66.83492612653488, 66.68746277968408, 66.50880526330715, 66.32731191333693, 66.15149023055326, 65.98134021495619, 65.82253353373224, 65.61835351501574, 65.32909848850069, 65.25820264866857, 65.09088846666478, 64.91223095028784, 64.6967075971982, 64.4726767433287, 64.29685506054504, 64.10969004338826, 64.01327170121657, 63.88282335592548, 63.726852508294805, 63.5255083231716, 63.380880809914075, 63.21073079431698, 63.023565777160194, 62.816549924850406, 62.6690865779996, 62.5017723959958, 62.323114879618856, 62.15012903042849, 61.99699401639111, 61.8325156679806, 61.67938065394323, 61.52057397271928, 61.34191645634234, 61.20863227745795, 61.052661429827296, 60.90803391656977, 60.72654056659955, 60.5875847205286, 60.43161387289794, 60.233105521368, 60.096985508890334, 59.93817882766639, 59.796387148002154, 59.58653546209908, 59.40220627853557, 59.257578765278055, 59.07892124890112, 58.903099566117454, 58.7471287184868, 58.48906786149789, 58.304738677934374, 58.08637949125145, 57.88503530612823, 57.72622862490429, 57.55891444290049, 57.3518985905907, 57.19309190936676, 57.00025522502339, 56.82159770864645, 56.61458185633666, 56.464282675892576, 56.2544309899895, 56.05025097127301, 55.85457845333636, 55.71278677367212, 55.48875591980262, 55.208008394067434, 54.918753367552384, 54.677707512123185, 54.32322831296259, 53.99427161614157, 53.59725491308169, 53.19173070924198, 52.76351983665598, 52.03754643677508, 50.00708958398321], "calls": [0.0, 4.823752942177353, 6.64435810906616, 8.249439922865326, 9.59078921248901, 10.512435130306553, 11.51348438873607, 12.593936987777557, 13.61200124776678, 14.689618013214984, 15.733204775543772, 16.42514817230525, 17.42336159714148, 18.387545018858294, 19.277996767149705, 20.26203102401951, 21.40770779570655, 22.40592122054278, 23.30488046961404, 24.280407225703996, 25.187873975555114, 26.19743073476448, 27.107733318208886, 27.94430422822789, 28.928338485097694, 29.864163570881665, 30.859541162124604, 31.925814593199668, 32.75104216884554, 33.62164308198395, 34.58299067010748, 35.61239826446984, 36.448969174488845, 37.50389927119077, 38.24688767263137, 39.09763775061679, 40.013612001247765, 40.93242208547203, 41.80869466579701, 42.74735558517426, 43.640643167058954, 44.295720727107735, 44.896917448884096, 45.69945835578368, 46.555880100955676, 47.3130476703627, 48.0758869069563, 48.86992031307603, 49.69514788872189, 50.3984346198565, 51.14709468848368, 51.907098091483995, 52.59053398746561, 53.271134049853956, 54.07651079034682, 54.81099169100757, 55.61636843150044, 56.39338683606046, 57.161897739840626, 57.81981113348269, 58.48906786149789, 59.260414598871336, 59.82190965034172, 60.590420554121884, 61.424155630547595, 62.15012903042849, 62.83072909281683, 63.46028415052605, 64.21461588633979, 65.0880526330715, 65.74029435952698, 66.52582026486685, 67.20925616084848, 68.05149873805405, 69.09508550038284, 69.90329807446899, 70.78807815557383, 71.6501715679324, 72.393159969373, 73.1559992059666, 73.95854011286617, 74.68451351274707, 75.50690525479965, 76.46825284292318, 77.44661543260641, 78.3058730113717, 79.16229475654369, 80.00737316734255, 80.9828999234325, 81.81096333267163, 82.89425176530642, 84.02007770184045, 85.21396364461333, 86.23202790460256, 87.51666052236054, 88.82965147605138, 90.2418966055072, 91.87250092164592, 93.61937441510932, 95.72072710773331, 100.0], "rescued": [0, 207, 335, 441, 526, 581, 642, 695, 765, 812, 867, 907, 969, 1020, 1061, 1121, 1168, 1224, 1255, 1297, 1335, 1376, 1421, 1452, 1489, 1521, 1557, 1592, 1616, 1642, 1669, 1699, 1708, 1733, 1763, 1790, 1812, 1832, 1856, 1875, 1895, 1918, 1925, 1944, 1960, 1978, 1991, 2003, 2020, 2033, 2041, 2053, 2063, 2075, 2087, 2102, 2114, 2121, 2130, 2141, 2150, 2159, 2165, 2171, 2179, 2189, 2200, 2206, 2217, 2236, 2242, 2248, 2255, 2266, 2279, 2286, 2295, 2304, 2319, 2331, 2340, 2347, 2356, 2367, 2382, 2397, 2404, 2412, 2427, 2443, 2457, 2468, 2478, 2489, 2498, 2509, 2523, 2542, 2570, 2589, 2627], "harmed": [0, 21, 51, 93, 142, 185, 228, 282, 340, 413, 470, 510, 571, 636, 702, 767, 850, 931, 997, 1067, 1136, 1231, 1309, 1377, 1477, 1562, 1650, 1748, 1836, 1924, 2011, 2097, 2178, 2305, 2360, 2446, 2531, 2627, 2730, 2811, 2897, 2954, 3007, 3081, 3168, 3237, 3310, 3388, 3478, 3543, 3610, 3685, 3756, 3822, 3892, 3961, 4029, 4099, 4155, 4221, 4281, 4354, 4409, 4470, 4548, 4606, 4673, 4729, 4814, 4898, 4955, 5024, 5093, 5159, 5263, 5335, 5421, 5501, 5572, 5643, 5725, 5788, 5865, 5939, 6027, 6095, 6176, 6256, 6340, 6406, 6499, 6609, 6721, 6817, 6951, 7078, 7232, 7394, 7573, 7848, 8602], "exact_score": [66.95119530385956, 67.0192553100984, 67.11000198508351, 67.2234353288149, 67.33970450613957, 67.51836202251653, 67.71403454045316, 67.81612454981142, 67.92105039276295, 68.0032895669682, 68.0600062388339, 68.08269290758018, 68.13940957944588, 68.1337379122593, 68.11672291069961, 68.0770212403936, 68.0968720755466, 68.05717040524061, 68.0770212403936, 68.0203045685279, 67.9692595638488, 67.90403539120324, 67.81612454981142, 67.69418370530018, 67.64881036780761, 67.52403368970309, 67.41060034597169, 67.29716700224031, 67.16104698976264, 66.97388197260584, 66.80373195700876, 66.67328361171766, 66.49179026174744, 66.29044607662422, 66.10044522587415, 65.91044437512407, 65.72611519156055, 65.45387516660523, 65.28372515100813, 65.10223180103792, 64.92073845106769, 64.66834926126535, 64.435810906616, 64.23730255508606, 64.05013753792927, 63.857300853585905, 63.64177750049627, 63.45744831693276, 63.22207412869013, 62.99237160763407, 62.759833252984706, 62.5386382327085, 62.30609987805915, 62.08206902418966, 61.852366503133595, 61.656693985196945, 61.426991464140876, 61.22564727901767, 61.03564642826759, 60.769078070498814, 60.56206221818904, 60.33235969713297, 60.13385134560304, 59.88713382298727, 59.64608796755806, 59.430564614468416, 59.18101125825936, 58.962652071576436, 58.764143720046505, 58.50891869665088, 58.28488784278139, 58.02966281938575, 57.822646967075976, 57.59578027961319, 57.337719422624275, 57.11368856875478, 56.901001049258426, 56.72801520006806, 56.52950684853813, 56.299804327482065, 56.05592263845958, 55.85174261974307, 55.68159260414599, 55.471740918242915, 55.210844227660715, 54.96129087145166, 54.72308084961575, 54.4508408246604, 54.19277996767149, 53.940390777869155, 53.676658253693674, 53.410089895924905, 53.16620820690242, 52.916654850693355, 52.6500864929246, 52.338144797663276, 51.946799761789975, 51.524260556390544, 51.050676346311995, 50.56574880186031, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.81190245576202, "threshold_area100": 62.12887445764681, "threshold_area25": 67.70297478943937}, "selected_budget": 23, "selected": {"val": {"score": 76.04891418933165, "calls": 22.995291306486752, "rescued": 679, "harmed": 534}, "test": {"score": 67.16388282335592, "calls": 27.94430422822789, "rescued": 1452, "harmed": 1377}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.3079502582550049, "operator": "ge"}, {"budget": 0.02, "threshold": 0.16113483905792236, "operator": "ge"}, {"budget": 0.03, "threshold": 0.06413638591766357, "operator": "ge"}, {"budget": 0.04, "threshold": 0.0271759033203125, "operator": "ge"}, {"budget": 0.05, "threshold": 0.003437042236328125, "operator": "gt"}, {"budget": 0.06, "threshold": -0.07955974340438843, "operator": "ge"}, {"budget": 0.07, "threshold": -0.21230757236480713, "operator": "ge"}, {"budget": 0.08, "threshold": -0.33101773262023926, "operator": "ge"}, {"budget": 0.09, "threshold": -0.448405921459198, "operator": "ge"}, {"budget": 0.1, "threshold": -0.5770863890647888, "operator": "ge"}, {"budget": 0.11, "threshold": -0.7155847549438477, "operator": "ge"}, {"budget": 0.12, "threshold": -0.8351725339889526, "operator": "ge"}, {"budget": 0.13, "threshold": -0.9600927233695984, "operator": "ge"}, {"budget": 0.14, "threshold": -1.0856602191925049, "operator": "ge"}, {"budget": 0.15, "threshold": -1.2276204824447632, "operator": "ge"}, {"budget": 0.16, "threshold": -1.3501057624816895, "operator": "ge"}, {"budget": 0.17, "threshold": -1.4739320278167725, "operator": "ge"}, {"budget": 0.18, "threshold": -1.595446228981018, "operator": "ge"}, {"budget": 0.19, "threshold": -1.742626428604126, "operator": "ge"}, {"budget": 0.2, "threshold": -1.8695335388183594, "operator": "ge"}, {"budget": 0.21, "threshold": -1.9881231784820557, "operator": "ge"}, {"budget": 0.22, "threshold": -2.1249258518218994, "operator": "ge"}, {"budget": 0.23, "threshold": -2.266244411468506, "operator": "ge"}, {"budget": 0.24, "threshold": -2.399022340774536, "operator": "ge"}, {"budget": 0.25, "threshold": -2.5407800674438477, "operator": "ge"}, {"budget": 0.26, "threshold": -2.654710531234741, "operator": "ge"}, {"budget": 0.27, "threshold": -2.772461414337158, "operator": "ge"}, {"budget": 0.28, "threshold": -2.8950717449188232, "operator": "ge"}, {"budget": 0.29, "threshold": -3.008777379989624, "operator": "ge"}, {"budget": 0.3, "threshold": -3.131202220916748, "operator": "ge"}, {"budget": 0.31, "threshold": -3.2421092987060547, "operator": "ge"}, {"budget": 0.32, "threshold": -3.3534293174743652, "operator": "ge"}, {"budget": 0.33, "threshold": -3.451734781265259, "operator": "ge"}, {"budget": 0.34, "threshold": -3.5465245246887207, "operator": "ge"}, {"budget": 0.35, "threshold": -3.623401641845703, "operator": "ge"}, {"budget": 0.36, "threshold": -3.70544171333313, "operator": "ge"}, {"budget": 0.37, "threshold": -3.7814714908599854, "operator": "ge"}, {"budget": 0.38, "threshold": -3.8601441383361816, "operator": "ge"}, {"budget": 0.39, "threshold": -3.9299700260162354, "operator": "gt"}, {"budget": 0.4, "threshold": -4.003727436065674, "operator": "ge"}, {"budget": 0.41, "threshold": -4.066573143005371, "operator": "ge"}, {"budget": 0.42, "threshold": -4.12800931930542, "operator": "ge"}, {"budget": 0.43, "threshold": -4.197843551635742, "operator": "ge"}, {"budget": 0.44, "threshold": -4.2646589279174805, "operator": "ge"}, {"budget": 0.45, "threshold": -4.330211639404297, "operator": "ge"}, {"budget": 0.46, "threshold": -4.386117458343506, "operator": "ge"}, {"budget": 0.47, "threshold": -4.435880661010742, "operator": "ge"}, {"budget": 0.48, "threshold": -4.498851776123047, "operator": "ge"}, {"budget": 0.49, "threshold": -4.553383827209473, "operator": "ge"}, {"budget": 0.5, "threshold": -4.600886344909668, "operator": "ge"}, {"budget": 0.51, "threshold": -4.6530046463012695, "operator": "ge"}, {"budget": 0.52, "threshold": -4.694458961486816, "operator": "ge"}, {"budget": 0.53, "threshold": -4.743110179901123, "operator": "gt"}, {"budget": 0.54, "threshold": -4.788471698760986, "operator": "ge"}, {"budget": 0.55, "threshold": -4.830667018890381, "operator": "ge"}, {"budget": 0.56, "threshold": -4.877418041229248, "operator": "ge"}, {"budget": 0.57, "threshold": -4.917552947998047, "operator": "ge"}, {"budget": 0.58, "threshold": -4.962395191192627, "operator": "ge"}, {"budget": 0.59, "threshold": -5.0037078857421875, "operator": "ge"}, {"budget": 0.6, "threshold": -5.041315078735352, "operator": "ge"}, {"budget": 0.61, "threshold": -5.0819244384765625, "operator": "ge"}, {"budget": 0.62, "threshold": -5.128849029541016, "operator": "ge"}, {"budget": 0.63, "threshold": -5.164909839630127, "operator": "ge"}, {"budget": 0.64, "threshold": -5.208532810211182, "operator": "ge"}, {"budget": 0.65, "threshold": -5.2479658126831055, "operator": "ge"}, {"budget": 0.66, "threshold": -5.28883695602417, "operator": "ge"}, {"budget": 0.67, "threshold": -5.325705051422119, "operator": "ge"}, {"budget": 0.68, "threshold": -5.3632307052612305, "operator": "ge"}, {"budget": 0.69, "threshold": -5.400832653045654, "operator": "ge"}, {"budget": 0.7, "threshold": -5.439769268035889, "operator": "ge"}, {"budget": 0.71, "threshold": -5.4834208488464355, "operator": "ge"}, {"budget": 0.72, "threshold": -5.523332118988037, "operator": "ge"}, {"budget": 0.73, "threshold": -5.561697006225586, "operator": "ge"}, {"budget": 0.74, "threshold": -5.593939304351807, "operator": "ge"}, {"budget": 0.75, "threshold": -5.630603790283203, "operator": "ge"}, {"budget": 0.76, "threshold": -5.670010566711426, "operator": "ge"}, {"budget": 0.77, "threshold": -5.707545757293701, "operator": "ge"}, {"budget": 0.78, "threshold": -5.745391368865967, "operator": "ge"}, {"budget": 0.79, "threshold": -5.794095993041992, "operator": "ge"}, {"budget": 0.8, "threshold": -5.842386245727539, "operator": "ge"}, {"budget": 0.81, "threshold": -5.8960771560668945, "operator": "gt"}, {"budget": 0.82, "threshold": -5.9452314376831055, "operator": "ge"}, {"budget": 0.83, "threshold": -5.984879970550537, "operator": "ge"}, {"budget": 0.84, "threshold": -6.029914855957031, "operator": "ge"}, {"budget": 0.85, "threshold": -6.078059673309326, "operator": "ge"}, {"budget": 0.86, "threshold": -6.1309919357299805, "operator": "ge"}, {"budget": 0.87, "threshold": -6.181056976318359, "operator": "ge"}, {"budget": 0.88, "threshold": -6.237303256988525, "operator": "ge"}, {"budget": 0.89, "threshold": -6.2894439697265625, "operator": "ge"}, {"budget": 0.9, "threshold": -6.351993560791016, "operator": "ge"}, {"budget": 0.91, "threshold": -6.430540561676025, "operator": "ge"}, {"budget": 0.92, "threshold": -6.498497009277344, "operator": "ge"}, {"budget": 0.93, "threshold": -6.582510471343994, "operator": "ge"}, {"budget": 0.94, "threshold": -6.6591997146606445, "operator": "ge"}, {"budget": 0.95, "threshold": -6.762933731079102, "operator": "ge"}, {"budget": 0.96, "threshold": -6.88032341003418, "operator": "ge"}, {"budget": 0.97, "threshold": -7.026987552642822, "operator": "ge"}, {"budget": 0.98, "threshold": -7.214895725250244, "operator": "ge"}, {"budget": 0.99, "threshold": -7.560860633850098, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.06500808208588, 75.14231499051233, 75.14934289127838, 75.27584510506712, 75.40234731885586, 75.40234731885586, 75.27584510506712, 75.18448239510859, 75.14934289127838, 75.24070560123691, 75.32504041042941, 75.38126361655773, 75.45154262421815, 75.50073792958042, 75.45154262421815, 75.46559842575023, 75.58507273877294, 75.66237964719939, 75.71860285332771, 75.66237964719939, 75.59912854030502, 75.66237964719939, 75.57101693724084, 75.54290533417668, 75.47965422728231, 75.4093752196219, 75.42343102115399, 75.3742357157917, 75.31098460889733, 75.23367770047086, 75.07203598285193, 74.91742216599901, 74.73469674608194, 74.5379155246328, 74.36924590624781, 74.22165999016094, 74.060018272542, 73.91946025722117, 73.72970693653805, 73.4837304097266, 73.42750720359828, 73.2588375852133, 73.1745027760208, 73.10422376836038, 72.92852624920936, 72.76688453159042, 72.6122707147375, 72.3733220886921, 72.23276407337129, 72.00084334809192, 71.76189472204652, 71.60025300442759, 71.37536017991425, 71.16452315693303, 71.01693724084616, 70.89746292782347, 70.7498770117366, 70.62337479794786, 70.51795628645723, 70.36334246960433, 70.11033804202684, 69.94166842364186, 69.80111040832104, 69.63244078993604, 69.49891067538127, 69.36538056082648, 69.1826551409094, 69.05615292712068, 68.91559491179984, 68.73286949188278, 68.58528357579591, 68.43066975894301, 68.33930704898447, 68.19172113289761, 68.01602361374658, 67.88952139995783, 67.7981586899993, 67.58732166701806, 67.48190315552745, 67.34837304097266, 67.17267552182163, 66.96886639960644, 66.80019678122146, 66.60341555977229, 66.40663433832314, 66.25202052147023, 66.10443460538337, 65.92873708623235, 65.73898376554924, 65.54220254410008, 65.38056082648113, 65.2891981165226, 65.1697238034999, 65.00105418511491, 64.90969147515638, 64.78318926136764, 64.62857544451472, 64.44585002459765, 64.27015250544662, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.9898095438892405, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99079344999649, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.98334387518449, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.9895284278586, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 22, 49, 72, 98, 119, 138, 151, 168, 191, 220, 253, 284, 319, 354, 381, 419, 461, 501, 541, 564, 596, 637, 665, 700, 731, 761, 791, 818, 839, 861, 872, 889, 904, 919, 935, 955, 970, 984, 996, 1007, 1023, 1028, 1042, 1058, 1067, 1081, 1089, 1092, 1097, 1103, 1107, 1115, 1120, 1126, 1134, 1138, 1142, 1150, 1158, 1164, 1165, 1168, 1171, 1175, 1178, 1182, 1189, 1197, 1201, 1203, 1208, 1212, 1220, 1224, 1225, 1230, 1234, 1239, 1245, 1249, 1255, 1257, 1262, 1264, 1265, 1269, 1273, 1275, 1282, 1289, 1296, 1300, 1301, 1305, 1313, 1324, 1330, 1338, 1348, 1369], "harmed": [0, 17, 33, 55, 63, 66, 85, 116, 146, 174, 190, 211, 234, 259, 287, 321, 357, 382, 411, 443, 474, 515, 547, 588, 627, 667, 707, 735, 769, 799, 832, 866, 905, 946, 989, 1029, 1070, 1108, 1142, 1181, 1227, 1251, 1280, 1306, 1332, 1366, 1403, 1433, 1470, 1495, 1534, 1572, 1603, 1640, 1676, 1705, 1726, 1751, 1777, 1800, 1828, 1865, 1892, 1915, 1943, 1965, 1988, 2021, 2047, 2071, 2099, 2125, 2151, 2172, 2197, 2223, 2246, 2263, 2298, 2319, 2342, 2373, 2404, 2433, 2463, 2492, 2518, 2543, 2570, 2604, 2639, 2669, 2686, 2704, 2732, 2753, 2782, 2810, 2844, 2879, 2921], "exact_score": [75.02986857825567, 75.06500808208588, 75.14231499051233, 75.14934289127838, 75.27584510506712, 75.40234731885586, 75.40234731885586, 75.27584510506712, 75.18448239510859, 75.14934289127838, 75.24070560123691, 75.32504041042941, 75.38126361655773, 75.45154262421815, 75.50073792958042, 75.45154262421815, 75.46559842575023, 75.58507273877294, 75.66237964719939, 75.71860285332771, 75.66237964719939, 75.59912854030502, 75.66237964719939, 75.57101693724084, 75.54290533417668, 75.47965422728231, 75.4093752196219, 75.42343102115399, 75.3742357157917, 75.31098460889733, 75.23367770047086, 75.07203598285193, 74.91742216599901, 74.73469674608194, 74.5379155246328, 74.36924590624781, 74.22165999016094, 74.060018272542, 73.91946025722117, 73.72970693653805, 73.4837304097266, 73.42750720359828, 73.2588375852133, 73.1745027760208, 73.10422376836038, 72.92852624920936, 72.76688453159042, 72.6122707147375, 72.3733220886921, 72.23276407337129, 72.00084334809192, 71.76189472204652, 71.60025300442759, 71.36130437838217, 71.16452315693303, 71.01693724084616, 70.89746292782347, 70.7498770117366, 70.62337479794786, 70.51795628645723, 70.36334246960433, 70.11033804202684, 69.94166842364186, 69.80111040832104, 69.63244078993604, 69.49891067538127, 69.36538056082648, 69.1826551409094, 69.05615292712068, 68.91559491179984, 68.73286949188278, 68.58528357579591, 68.43066975894301, 68.33930704898447, 68.19172113289761, 68.01602361374658, 67.88952139995783, 67.7981586899993, 67.58732166701806, 67.48190315552745, 67.34837304097266, 67.17267552182163, 66.96886639960644, 66.80019678122146, 66.60341555977229, 66.40663433832314, 66.25202052147023, 66.10443460538337, 65.92873708623235, 65.73898376554924, 65.54220254410008, 65.38056082648113, 65.2891981165226, 65.1697238034999, 65.00105418511491, 64.90969147515638, 64.78318926136764, 64.62857544451472, 64.44585002459765, 64.27015250544662, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 71.33921358565873, "threshold_area100": 71.33972872303043, "threshold_area25": 75.40234731885586}, "test": {"score": [66.95119530385956, 67.30283866942688, 67.52686952329637, 67.8189603834047, 67.94940872869579, 68.07985707398689, 68.12523041147945, 68.07418540680032, 68.04015540368091, 68.02314040212121, 68.05433457164733, 67.96075206306894, 67.88985622323682, 67.84448288574426, 67.67716870374046, 67.50985452173667, 67.37089867566571, 67.31985367098659, 67.19791282647533, 67.01641947650512, 66.84910529450133, 66.66761194453109, 66.43223775628846, 66.24507273913166, 66.06925105634801, 65.87641437200465, 65.76014519467998, 65.55312934237018, 65.36880015880668, 65.16462014009018, 64.92924595184755, 64.72790176672433, 64.52939341519439, 64.35640756600402, 64.1749142160338, 64.01894336840314, 63.86013668717919, 63.71550917392167, 63.53401582395145, 63.3014774693021, 63.12565578651844, 62.91296826702209, 62.691773246745875, 62.47057822646966, 62.2607265405666, 62.0735615234098, 61.923262342965714, 61.750276493775345, 61.560275643025264, 61.384453960241615, 61.22848311261095, 61.018631426707884, 60.91370558375634, 60.7350480673794, 60.61027138927487, 60.43161387289794, 60.25295635652099, 60.122508011229904, 59.97220883078581, 59.7680288120693, 59.61772963162522, 59.44474378243485, 59.27459376683776, 59.135637920766804, 58.90877123330402, 58.76981538723307, 58.61668037319569, 58.466381192751605, 58.31608201230751, 58.126081161557444, 57.91906530924765, 57.71488529053115, 57.561750276493775, 57.40577942886311, 57.204435243739894, 57.06264356407566, 56.9236877180047, 56.78189603834048, 56.61458185633666, 56.41607350480674, 56.228908487649946, 56.070101806425996, 55.8829367892692, 55.72980177523183, 55.593681762754166, 55.43487508153021, 55.18248589172787, 54.969798372231516, 54.77696168788815, 54.62382667385077, 54.41964665513428, 54.136063295805805, 53.9318832770893, 53.750389927119066, 53.56889657714885, 53.208745710801686, 52.86844567960751, 52.58769815387233, 52.244562289084875, 51.62067889856223, 50.00708958398321], "calls": [0.0, 1.829112667668661, 3.3009103025834445, 5.294501318662621, 6.942120636361058, 10.101239259280266, 11.845276919150384, 13.05617786348297, 14.145137963304313, 15.151858888920398, 16.17275898250291, 17.431869097921336, 18.50381419618297, 19.524714289765477, 20.7100927317585, 21.89263534015824, 22.845475427501913, 23.710404673453763, 24.694438930323567, 25.808921532484476, 26.80997079091399, 27.731616708731533, 28.718486799194626, 29.716700224030856, 30.692226980120807, 31.795366247908575, 32.71417633213283, 33.71238975696906, 34.64254317556646, 35.59821909650342, 36.556730851033656, 37.617332614922155, 38.60420270538525, 39.65913280208717, 40.6204803902107, 41.440036298669995, 42.28511470946884, 43.09899895074157, 43.997958199812835, 44.84303661061169, 45.78736919717551, 46.618268440007945, 47.426481014094094, 48.35663443269149, 49.26410118254261, 50.09783625896832, 50.88619799790148, 51.64336556730851, 52.5366531491932, 53.342029889686074, 54.01695828488784, 54.83084252616056, 55.45472591668321, 56.23174432124323, 56.923687718004714, 57.655332785072176, 58.364291183393355, 59.033547911408554, 59.84176048549471, 60.6443013923943, 61.27952811729007, 61.982814848424695, 62.79386325610413, 63.482970819272325, 64.33372089725775, 65.09372430025806, 65.89059353997108, 66.54283526642656, 67.25179366474775, 68.01179706774806, 68.79732297308794, 69.61120721436066, 70.35703144939454, 71.10852735161501, 71.7976349147832, 72.41868247171256, 73.1191333692539, 73.90465927459377, 74.61645350650824, 75.46436775090038, 76.30661032810595, 77.2651220826362, 78.13288716218132, 78.8616963956555, 79.59901312990954, 80.34483736494342, 81.27499078354083, 82.13141252871281, 83.06723761449678, 83.8811218557695, 84.76306610328106, 85.89172787340839, 86.67725377874827, 87.68113887077106, 88.4921872784505, 89.71726739074951, 90.91398916711567, 92.27235345829908, 93.73847942602728, 95.95042962878938, 100.0], "rescued": [0, 180, 302, 444, 516, 582, 640, 690, 738, 791, 853, 914, 965, 1017, 1059, 1102, 1142, 1190, 1227, 1274, 1315, 1352, 1390, 1443, 1483, 1533, 1576, 1615, 1648, 1686, 1713, 1746, 1773, 1809, 1843, 1872, 1900, 1922, 1943, 1960, 1980, 1999, 2011, 2028, 2048, 2063, 2076, 2088, 2105, 2119, 2125, 2137, 2148, 2162, 2173, 2183, 2193, 2204, 2216, 2224, 2231, 2241, 2249, 2260, 2266, 2273, 2283, 2292, 2298, 2301, 2309, 2322, 2329, 2334, 2336, 2338, 2347, 2362, 2368, 2379, 2384, 2395, 2399, 2409, 2420, 2424, 2430, 2436, 2448, 2457, 2464, 2474, 2484, 2502, 2509, 2522, 2533, 2549, 2565, 2589, 2627], "harmed": [0, 56, 99, 138, 164, 184, 226, 294, 354, 413, 464, 558, 634, 702, 803, 905, 994, 1060, 1140, 1251, 1351, 1452, 1573, 1692, 1794, 1912, 1996, 2108, 2206, 2316, 2426, 2530, 2627, 2724, 2822, 2906, 2990, 3063, 3148, 3247, 3329, 3423, 3513, 3608, 3702, 3783, 3849, 3922, 4006, 4082, 4143, 4229, 4277, 4354, 4409, 4482, 4555, 4612, 4677, 4757, 4817, 4888, 4956, 5016, 5102, 5158, 5222, 5284, 5343, 5413, 5494, 5579, 5640, 5700, 5773, 5825, 5883, 5948, 6013, 6094, 6165, 6232, 6302, 6366, 6425, 6485, 6580, 6661, 6741, 6804, 6883, 6993, 7075, 7157, 7228, 7368, 7499, 7614, 7751, 7995, 8602], "exact_score": [66.95119530385956, 67.13836032101636, 67.3000028358336, 67.48716785299038, 67.64030286702776, 67.77075121231886, 67.88702038964354, 67.95224456228908, 68.04582707086749, 68.0600062388339, 68.08269290758018, 68.08269290758018, 68.11105124351303, 68.07418540680032, 68.04582707086749, 68.03164790290106, 68.06567790602047, 68.00045373337493, 67.90970705838981, 67.8926920568301, 67.76791537872558, 67.60627286390834, 67.49283952017694, 67.353883674106, 67.29433116864702, 67.13836032101636, 66.98522530697898, 66.83776196012818, 66.62507444063182, 66.35283441567648, 66.21671440319882, 66.00402688370247, 65.84238436888523, 65.69775685562772, 65.49924850409778, 65.28656098460142, 65.06253013073191, 64.85551427842215, 64.63715509173922, 64.44999007458242, 64.27700422539205, 64.0983467090151, 63.91118169185832, 63.743867509854525, 63.53401582395145, 63.253268298216256, 63.06893911465274, 62.7995349232907, 62.56132490145478, 62.31460737883901, 62.09908402574936, 61.89773984062615, 61.67370898675666, 61.45534980007373, 61.23415477979752, 60.98460142358846, 60.794600572838384, 60.5875847205286, 60.3465388650994, 60.125343844823185, 59.94101466125967, 59.688625471457335, 59.433400448061704, 59.21504126137877, 58.97966707313614, 58.798173723165924, 58.588322037262856, 58.37279868417321, 58.128916995150725, 57.876527805348374, 57.65249695147888, 57.436973598389244, 57.159061906247345, 56.963389388310695, 56.76204520318747, 56.51816351416499, 56.28278932592236, 56.10413180954541, 55.90845929160877, 55.7156226072654, 55.50009925417576, 55.270396733119696, 55.01800754331736, 54.799648356634435, 54.612483339477635, 54.35442248248873, 54.10203329268638, 53.86665910444375, 53.670986586507105, 53.42426906389133, 53.12934237018971, 52.86277401242094, 52.66142982729774, 52.423219805461805, 52.17366644925276, 51.884411422737706, 51.615007231375664, 51.23784136346878, 50.877690497121634, 50.4551512917222, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.372660839483686, "threshold_area100": 61.20714346482149, "threshold_area25": 67.38139125996088}, "selected_budget": 19, "selected": {"val": {"score": 75.71860285332771, "calls": 18.996415770609318, "rescued": 541, "harmed": 443}, "test": {"score": 67.01641947650512, "calls": 25.808921532484476, "rescued": 1274, "harmed": 1251}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 75.71157495256166, "calls": 4.996837444655281}, "test": {"score": 68.01463290134134, "calls": 11.62124606528089}}, "old_normalized_interior": {"val": {"score": 75.6834633494975, "calls": 6.999789162977018}, "test": {"score": 68.1734395825653, "calls": 14.048719621132634}}, "old_normalized_endpoints": {"val": {"score": 75.6834633494975, "calls": 6.999789162977018}, "test": {"score": 68.1734395825653, "calls": 14.048719621132634}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.31801250966336, 75.4585705249842, 75.64832384566729, 75.66940754796543, 75.65535174643334, 75.66940754796543, 75.6834633494975, 75.67643544873147, 75.66940754796543, 75.66940754796543, 75.63426804413523, 75.59912854030502, 75.57804483800689, 75.56398903647481, 75.57101693724084, 75.54290533417668, 75.4937100288144, 75.47262632651626, 75.4445147234521, 75.42343102115399, 75.36720781502565, 75.33206831119544, 75.36017991425962, 75.31098460889733, 75.2899009065992, 75.26178930353502, 75.22664979970483, 75.21259399817275, 75.19853819664067, 75.1352870897463, 75.1212312882142, 75.05798018131983, 75.01581277672359, 74.9666174713613, 74.93850586829714, 74.95958957059527, 74.97364537212735, 75.00175697519151, 74.92445006676506, 74.85417105910464, 74.8471431583386, 74.79091995221027, 74.76983624991215, 74.69252934148571, 74.6573898376555, 74.6573898376555, 74.67144563918757, 74.60116663152716, 74.49574812003654, 74.4887202192705, 74.47466441773842, 74.39735750931197, 74.34816220394968, 74.26382739475719, 74.22868789092698, 74.10921357790427, 74.02487876871179, 74.0108229671797, 73.9756834633495, 73.87729285262492, 73.87026495185887, 73.82809754726264, 73.71565113500597, 73.70862323423994, 73.60320472274931, 73.44859090589641, 73.22369808138309, 73.11125166912643, 73.03394476069998, 72.89338674537915, 72.89338674537915, 72.78796823388853, 72.67552182163189, 72.49279640171481, 72.26790357720148, 72.05003865345422, 71.86731323353715, 71.67755991285402, 71.501862393703, 71.26994166842364, 71.08721624850656, 70.820156019397, 70.47578888186099, 70.21575655351747, 69.85030571368333, 69.51296647691335, 69.27401785086795, 68.89451120950172, 68.59231147656195, 68.24794433902593, 67.82627029306346, 67.46784735399535, 67.10942441492726, 66.77911307892333, 66.42771804062126, 65.99198819312672, 65.64059315482466, 65.26811441422447, 64.67777074987701, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.98945814885094, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.98840396373603, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.31801250966336, 75.4585705249842, 75.64832384566729, 75.66940754796543, 75.65535174643334, 75.66940754796543, 75.6834633494975, 75.67643544873147, 75.66940754796543, 75.66940754796543, 75.63426804413523, 75.59912854030502, 75.57804483800689, 75.56398903647481, 75.57101693724084, 75.54290533417668, 75.4937100288144, 75.47262632651626, 75.4445147234521, 75.42343102115399, 75.36720781502565, 75.33206831119544, 75.36017991425962, 75.31098460889733, 75.2899009065992, 75.26178930353502, 75.22664979970483, 75.21259399817275, 75.19853819664067, 75.1352870897463, 75.1212312882142, 75.05798018131983, 75.01581277672359, 74.9666174713613, 74.93850586829714, 74.95958957059527, 74.97364537212735, 75.00175697519151, 74.92445006676506, 74.85417105910464, 74.8471431583386, 74.79091995221027, 74.76983624991215, 74.69252934148571, 74.6573898376555, 74.6573898376555, 74.67144563918757, 74.60116663152716, 74.49574812003654, 74.4887202192705, 74.47466441773842, 74.39735750931197, 74.34816220394968, 74.26382739475719, 74.22868789092698, 74.10921357790427, 74.02487876871179, 74.0108229671797, 73.9756834633495, 73.87729285262492, 73.87026495185887, 73.82809754726264, 73.71565113500597, 73.70862323423994, 73.60320472274931, 73.44859090589641, 73.22369808138309, 73.11125166912643, 73.03394476069998, 72.89338674537915, 72.89338674537915, 72.78796823388853, 72.67552182163189, 72.49279640171481, 72.26790357720148, 72.05003865345422, 71.86731323353715, 71.67755991285402, 71.501862393703, 71.26994166842364, 71.08721624850656, 70.820156019397, 70.47578888186099, 70.21575655351747, 69.84327781291728, 69.51296647691335, 69.27401785086795, 68.89451120950172, 68.59231147656195, 68.24794433902593, 67.82627029306346, 67.46784735399535, 67.10942441492726, 66.77911307892333, 66.42771804062126, 65.99198819312672, 65.64059315482466, 65.26811441422447, 64.67777074987701, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 75.4958183990442, "exact_area100": 73.21944770538742}, "test": {"score": [66.95119530385956, 67.44179451549783, 67.65448203499419, 67.88985622323682, 68.08836457476674, 68.14224541303916, 68.1734395825653, 68.1734395825653, 68.14791708022572, 68.1337379122593, 68.13090207866603, 68.15642458100558, 68.16209624819216, 68.15926041459886, 68.14508124663244, 68.13090207866603, 68.13657374585259, 68.11105124351303, 68.11105124351303, 68.12806624507274, 68.10821540991975, 68.10821540991975, 68.10537957632647, 68.0968720755466, 68.08552874117346, 68.08269290758018, 68.09120040836004, 68.09403624195332, 68.09120040836004, 68.07418540680032, 68.05717040524061, 68.04015540368091, 68.02314040212121, 68.0203045685279, 67.97209539744208, 67.95224456228908, 67.90687122479653, 67.90687122479653, 67.85015455293083, 67.8189603834047, 67.75090037716586, 67.72821370841959, 67.6714970365539, 67.6147803646882, 67.57224286078892, 67.52970535688966, 67.41343617956498, 67.37940617644557, 67.23761449678133, 67.1865694921022, 67.12134531945665, 67.03343447806482, 66.93134446870658, 66.86044862887445, 66.7044777812438, 66.59388027110569, 66.4804469273743, 66.40955108754218, 66.24223690553838, 66.05507188838159, 65.9047727079375, 65.70910019000084, 65.5673085103366, 65.3461134900604, 65.12491846978419, 64.9008876159147, 64.65700592689221, 64.435810906616, 64.20327255196665, 63.94804752857102, 63.67013583642912, 63.412074979440206, 63.10296911777217, 62.91296826702209, 62.60386240535405, 62.40535405382413, 62.087740691376226, 61.78430649689476, 61.40997646258117, 61.04982559623401, 60.69251056348013, 60.1962396846553, 59.688625471457335, 59.27175793324447, 58.84071122706519, 58.51175453024416, 58.08637949125145, 57.59010861242662, 57.19592774296004, 56.6826418625755, 56.2544309899895, 55.865921787709496, 55.205172560474146, 54.6181550066642, 54.048152454413966, 53.5235232396563, 52.91098318350679, 52.315458128916994, 51.770978079006326, 50.91455633383433, 50.00708958398321], "calls": [0.0, 1.9170235090604881, 3.743300343135865, 6.58480560360718, 9.718401724186824, 11.05407934662394, 12.61945949011712, 14.048719621132634, 15.472308084961576, 16.459178175424665, 17.35530159090265, 18.089782491563394, 18.699486714119615, 19.29501176870941, 19.865014320959645, 20.298896860732214, 20.840541077049597, 21.399200294926693, 21.87562033859853, 22.34069704789723, 22.7745795876698, 23.18861129228937, 23.6848821711142, 24.201003885092025, 24.61787142330488, 25.020559793551318, 25.49981567081644, 25.891160706689732, 26.38175991832799, 26.897881632305815, 27.465048350962768, 27.938632561041317, 28.409380937526585, 29.00490599211638, 29.606102713892753, 30.30655361143408, 31.0098403425687, 31.6790970705839, 32.35402546578567, 33.19059637580467, 33.91373394209228, 34.79284235601055, 35.70598077304823, 36.587925020559794, 37.47270510166463, 38.2412160054448, 39.3556986076057, 40.212120352777696, 41.42585713070357, 42.23123387119644, 43.206760627286386, 44.236168221648754, 45.37617332614922, 46.439610923631, 47.38961517738139, 48.33111193035193, 49.417236196579985, 50.449479624535634, 51.544111391543545, 52.56784731871934, 53.58874741230185, 54.75143918554859, 55.67875677055271, 56.87547854691887, 58.0778719904716, 59.212205427785506, 60.45146470805093, 61.52908147349913, 62.63789240847347, 63.752375010634374, 64.89238011513484, 65.98984771573603, 67.13552448742308, 68.12239457788617, 69.33045968862547, 70.37688228454753, 71.50270822108159, 72.58316082012307, 73.7118225902504, 74.7894393556986, 76.11377364376258, 77.42676459745343, 78.8815472308085, 80.08394067436123, 81.20693077730199, 82.22215920369793, 83.38485097694466, 84.65814026032953, 85.71307035703146, 87.00337464197601, 88.05830473867793, 89.11039900178658, 90.39219578595127, 91.5605592263846, 92.84802767773587, 93.91430110881093, 95.1478887218898, 96.18296798343873, 97.2123755778011, 98.51969486430536, 100.0], "exact_score": [66.95119530385956, 67.20642032725519, 67.4474661826844, 67.56940702719564, 67.69701953889346, 67.79343788106513, 67.86433372089725, 67.9295578935428, 67.99478206618835, 68.02881206930778, 68.11955874429289, 68.1337379122593, 68.170603748972, 68.1734395825653, 68.1734395825653, 68.15926041459886, 68.1337379122593, 68.13940957944588, 68.15075291381902, 68.16776791537872, 68.14508124663244, 68.13090207866603, 68.11388707710631, 68.10254374273318, 68.0968720755466, 68.08269290758018, 68.09403624195332, 68.07134957320704, 68.03164790290106, 68.01746873493462, 67.97209539744208, 67.90970705838981, 67.85866205371069, 67.80478121543828, 67.75657204435244, 67.70552703967331, 67.6516462014009, 67.59492952953521, 67.5410486912628, 67.45597368346425, 67.37940617644557, 67.28015200068059, 67.18940532569549, 67.12985282023651, 67.06179281399767, 66.95403113745286, 66.87746363043416, 66.76970195388935, 66.62507444063182, 66.52298443127357, 66.43790942347503, 66.32164024615035, 66.18268440007941, 65.99551938292261, 65.81686186654568, 65.65805518532173, 65.49924850409778, 65.33760598928055, 65.15611263931032, 64.9377534526274, 64.76193176984374, 64.55775175112724, 64.3422283980376, 64.12954087854125, 63.8970025238919, 63.63043416612313, 63.40640331225364, 63.139834954484876, 62.9299832685818, 62.70028074752574, 62.47057822646966, 62.25505487338003, 61.93744151093213, 61.63684315004395, 61.33908062274905, 61.012959759521316, 60.74071973456597, 60.352210532285966, 60.02041800187165, 59.6574313019312, 59.291608768397474, 58.903099566117454, 58.58265037007628, 58.24802200606868, 57.84816946941553, 57.44831693276238, 57.08533023282194, 56.68547769616878, 56.28278932592236, 55.894280123642346, 55.41502424637722, 54.89039503161954, 54.40263165357456, 53.95740577942886, 53.46964240138389, 52.973371522559056, 52.43172730624167, 51.87590392195786, 51.2548563650285, 50.61679380653943, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 68.00584181720217, "exact_area100": 63.91254500379973}}, "old_normalized_endpoint_budget": 7}, {"seed": 2030, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.75, "validation_area100": 0.7358251837417713, "training_seconds": 88.34615037601907, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.058975938404609955, "operator": "ge"}, {"budget": 0.02, "threshold": 0.01565434815799124, "operator": "ge"}, {"budget": 0.03, "threshold": 0.0021225865820896197, "operator": "ge"}, {"budget": 0.04, "threshold": -0.02570354767256211, "operator": "ge"}, {"budget": 0.05, "threshold": -0.07387148293506139, "operator": "ge"}, {"budget": 0.06, "threshold": -0.12555714748771554, "operator": "ge"}, {"budget": 0.07, "threshold": -0.17444689361660703, "operator": "ge"}, {"budget": 0.08, "threshold": -0.22662740449590982, "operator": "ge"}, {"budget": 0.09, "threshold": -0.2555557851363563, "operator": "ge"}, {"budget": 0.1, "threshold": -0.27987885329586687, "operator": "ge"}, {"budget": 0.11, "threshold": -0.2977770956737207, "operator": "ge"}, {"budget": 0.12, "threshold": -0.31437641302065655, "operator": "ge"}, {"budget": 0.13, "threshold": -0.32673917287663146, "operator": "ge"}, {"budget": 0.14, "threshold": -0.33836667076161414, "operator": "ge"}, {"budget": 0.15, "threshold": -0.3502090724199067, "operator": "ge"}, {"budget": 0.16, "threshold": -0.3608216005083674, "operator": "ge"}, {"budget": 0.17, "threshold": -0.3722169843759485, "operator": "ge"}, {"budget": 0.18, "threshold": -0.3832361469036911, "operator": "ge"}, {"budget": 0.19, "threshold": -0.3932171382825106, "operator": "ge"}, {"budget": 0.2, "threshold": -0.40294729903502574, "operator": "ge"}, {"budget": 0.21, "threshold": -0.4125047444763331, "operator": "ge"}, {"budget": 0.22, "threshold": -0.42208740764569314, "operator": "ge"}, {"budget": 0.23, "threshold": -0.4318931612676494, "operator": "ge"}, {"budget": 0.24, "threshold": -0.4404366148346014, "operator": "ge"}, {"budget": 0.25, "threshold": -0.4493075225745864, "operator": "ge"}, {"budget": 0.26, "threshold": -0.45958363385751994, "operator": "ge"}, {"budget": 0.27, "threshold": -0.4678143716929766, "operator": "ge"}, {"budget": 0.28, "threshold": -0.4770223845711215, "operator": "ge"}, {"budget": 0.29, "threshold": -0.48534694641375814, "operator": "ge"}, {"budget": 0.3, "threshold": -0.4926639328827801, "operator": "ge"}, {"budget": 0.31, "threshold": -0.5016371401847798, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5108149748828072, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5187800520757099, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5281847276552537, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5361423714002009, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5440216017480323, "operator": "ge"}, {"budget": 0.37, "threshold": -0.5523588600274603, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5602410986602896, "operator": "ge"}, {"budget": 0.39, "threshold": -0.5678071705335551, "operator": "ge"}, {"budget": 0.4, "threshold": -0.5759122648793465, "operator": "ge"}, {"budget": 0.41, "threshold": -0.5852344693515058, "operator": "ge"}, {"budget": 0.42, "threshold": -0.5933934135019865, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6026261584220592, "operator": "ge"}, {"budget": 0.44, "threshold": -0.611194616910437, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6182098172167246, "operator": "ge"}, {"budget": 0.46, "threshold": -0.6255491904722018, "operator": "ge"}, {"budget": 0.47, "threshold": -0.6335920239081522, "operator": "ge"}, {"budget": 0.48, "threshold": -0.6420381311137882, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6502551316083365, "operator": "ge"}, {"budget": 0.5, "threshold": -0.6570776320327347, "operator": "ge"}, {"budget": 0.51, "threshold": -0.6652372474433956, "operator": "ge"}, {"budget": 0.52, "threshold": -0.6723833575197399, "operator": "ge"}, {"budget": 0.53, "threshold": -0.6799225255191327, "operator": "ge"}, {"budget": 0.54, "threshold": -0.6878753976941884, "operator": "ge"}, {"budget": 0.55, "threshold": -0.6940928033705289, "operator": "ge"}, {"budget": 0.56, "threshold": -0.7011359034907967, "operator": "ge"}, {"budget": 0.57, "threshold": -0.7080790644710028, "operator": "ge"}, {"budget": 0.58, "threshold": -0.7153420958060955, "operator": "ge"}, {"budget": 0.59, "threshold": -0.7232906111484001, "operator": "ge"}, {"budget": 0.6, "threshold": -0.7299494372166054, "operator": "ge"}, {"budget": 0.61, "threshold": -0.7364674704116034, "operator": "ge"}, {"budget": 0.62, "threshold": -0.7420901813267728, "operator": "ge"}, {"budget": 0.63, "threshold": -0.7493559705466986, "operator": "ge"}, {"budget": 0.64, "threshold": -0.7560047858996884, "operator": "ge"}, {"budget": 0.65, "threshold": -0.7622005901483049, "operator": "ge"}, {"budget": 0.66, "threshold": -0.7688027816644093, "operator": "ge"}, {"budget": 0.67, "threshold": -0.7758476675491973, "operator": "ge"}, {"budget": 0.68, "threshold": -0.7818825037720939, "operator": "ge"}, {"budget": 0.69, "threshold": -0.7889367262273073, "operator": "ge"}, {"budget": 0.7, "threshold": -0.7951930315000468, "operator": "ge"}, {"budget": 0.71, "threshold": -0.8013858039493634, "operator": "ge"}, {"budget": 0.72, "threshold": -0.8073345416791735, "operator": "ge"}, {"budget": 0.73, "threshold": -0.8150280594381075, "operator": "ge"}, {"budget": 0.74, "threshold": -0.8211266328551455, "operator": "ge"}, {"budget": 0.75, "threshold": -0.8264297691305632, "operator": "ge"}, {"budget": 0.76, "threshold": -0.8322405496102336, "operator": "ge"}, {"budget": 0.77, "threshold": -0.8389939216183547, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8449776331815579, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8514779678035344, "operator": "ge"}, {"budget": 0.8, "threshold": -0.8588019808606009, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8649583751641382, "operator": "ge"}, {"budget": 0.82, "threshold": -0.8704538483281619, "operator": "ge"}, {"budget": 0.83, "threshold": -0.8762657476024779, "operator": "ge"}, {"budget": 0.84, "threshold": -0.8824504148273218, "operator": "ge"}, {"budget": 0.85, "threshold": -0.8894404226930437, "operator": "ge"}, {"budget": 0.86, "threshold": -0.8970132737509008, "operator": "gt"}, {"budget": 0.87, "threshold": -0.9028249835534026, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9103177030908706, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9168181004929273, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9238832434032461, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9299063639439051, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9354979975543392, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9430296313788437, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9502176799891314, "operator": "ge"}, {"budget": 0.95, "threshold": -0.95705955612892, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9639062321104339, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9700329612885149, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9764495309961478, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9840186684416239, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.2899009065992, 75.4093752196219, 75.56398903647481, 75.57804483800689, 75.38829151732378, 75.16339869281046, 75.10717548668214, 75.14934289127838, 75.14934289127838, 75.1704265935765, 75.2196218989388, 75.24070560123691, 75.23367770047086, 75.22664979970483, 75.24070560123691, 75.254761402769, 75.26881720430107, 75.31098460889733, 75.26178930353502, 75.31801250966336, 75.31801250966336, 75.29692880736523, 75.31098460889733, 75.2899009065992, 75.26881720430107, 75.31098460889733, 75.31801250966336, 75.2899009065992, 75.33206831119544, 75.32504041042941, 75.29692880736523, 75.34612411272754, 75.38126361655773, 75.36720781502565, 75.3742357157917, 75.4445147234521, 75.45154262421815, 75.4937100288144, 75.4445147234521, 75.50073792958042, 75.53587743341063, 75.56398903647481, 75.57804483800689, 75.54290533417668, 75.50073792958042, 75.4937100288144, 75.41640312038794, 75.38829151732378, 75.31098460889733, 75.22664979970483, 75.18448239510859, 75.1704265935765, 75.086091784384, 75.0509522805538, 74.9314779675311, 74.86822686063674, 74.81903155527444, 74.75578044838007, 74.60116663152716, 74.55899922693092, 74.47466441773842, 74.4184412116101, 74.34816220394968, 74.22868789092698, 74.2146320893949, 74.10218567713824, 73.9756834633495, 73.94054395951929, 73.78593014266639, 73.65942792887765, 73.60320472274931, 73.51184201279078, 73.30100498980954, 73.07611216529622, 72.89338674537915, 72.71066132546208, 72.55604750860918, 72.29601518026566, 72.12031766111463, 71.97273174502776, 71.68458781362007, 71.39644388221238, 71.08721624850656, 70.69365380560826, 70.52498418722328, 70.307119263476, 69.92058472134374, 69.54810598074356, 69.18968304167545, 68.81017640030923, 68.33227914821842, 67.98088410991637, 67.5943495677841, 67.24295452948205, 66.82830838428562, 66.35743903296085, 65.72492796401714, 65.29622601728863, 64.71993815447325, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.98636587251389, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 42, 64, 88, 103, 114, 123, 141, 166, 174, 183, 191, 198, 203, 207, 213, 218, 223, 232, 236, 246, 250, 257, 266, 272, 279, 290, 298, 303, 317, 324, 330, 340, 352, 359, 370, 385, 397, 412, 422, 439, 452, 468, 484, 495, 507, 521, 530, 544, 555, 570, 584, 598, 605, 620, 636, 654, 669, 681, 693, 711, 725, 740, 757, 771, 794, 814, 829, 857, 868, 889, 908, 920, 931, 948, 965, 981, 1001, 1019, 1035, 1058, 1071, 1082, 1098, 1112, 1137, 1157, 1173, 1188, 1206, 1225, 1236, 1253, 1270, 1286, 1306, 1322, 1329, 1344, 1354, 1369], "harmed": [0, 5, 10, 12, 25, 63, 104, 130, 149, 157, 163, 164, 168, 174, 179, 183, 186, 189, 192, 203, 205, 209, 219, 226, 235, 245, 250, 257, 266, 274, 282, 292, 295, 302, 311, 321, 326, 337, 346, 363, 372, 380, 392, 406, 422, 440, 455, 475, 493, 515, 542, 562, 578, 597, 617, 650, 677, 699, 720, 754, 778, 804, 827, 854, 885, 910, 946, 979, 1012, 1045, 1084, 1111, 1136, 1177, 1226, 1269, 1311, 1353, 1408, 1449, 1493, 1547, 1599, 1659, 1729, 1778, 1829, 1900, 1968, 2037, 2110, 2189, 2256, 2328, 2394, 2473, 2556, 2653, 2729, 2821, 2921], "exact_score": [75.02986857825567, 75.2899009065992, 75.4093752196219, 75.56398903647481, 75.57804483800689, 75.38829151732378, 75.16339869281046, 75.10717548668214, 75.14934289127838, 75.14934289127838, 75.1704265935765, 75.2196218989388, 75.24070560123691, 75.23367770047086, 75.22664979970483, 75.24070560123691, 75.254761402769, 75.26881720430107, 75.31098460889733, 75.26178930353502, 75.31801250966336, 75.31801250966336, 75.29692880736523, 75.31098460889733, 75.2899009065992, 75.26881720430107, 75.31098460889733, 75.31801250966336, 75.2899009065992, 75.33206831119544, 75.32504041042941, 75.29692880736523, 75.34612411272754, 75.38126361655773, 75.36720781502565, 75.3742357157917, 75.4445147234521, 75.45154262421815, 75.4937100288144, 75.4445147234521, 75.50073792958042, 75.53587743341063, 75.56398903647481, 75.57804483800689, 75.54290533417668, 75.50073792958042, 75.4937100288144, 75.41640312038794, 75.38829151732378, 75.31098460889733, 75.22664979970483, 75.18448239510859, 75.1704265935765, 75.086091784384, 75.0509522805538, 74.9314779675311, 74.86822686063674, 74.81903155527444, 74.75578044838007, 74.60116663152716, 74.55899922693092, 74.47466441773842, 74.4184412116101, 74.34816220394968, 74.22868789092698, 74.2146320893949, 74.10218567713824, 73.9756834633495, 73.94054395951929, 73.78593014266639, 73.65942792887765, 73.60320472274931, 73.51184201279078, 73.30100498980954, 73.07611216529622, 72.89338674537915, 72.71066132546208, 72.55604750860918, 72.29601518026566, 72.12031766111463, 71.97273174502776, 71.68458781362007, 71.39644388221238, 71.08721624850656, 70.69365380560826, 70.52498418722328, 70.30009136270996, 69.92058472134374, 69.54810598074356, 69.18968304167545, 68.81017640030923, 68.33227914821842, 67.98088410991637, 67.5943495677841, 67.24295452948205, 66.82830838428562, 66.35743903296085, 65.72492796401714, 65.29622601728863, 64.71993815447325, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 73.58251837417713, "threshold_area100": 73.58296436854312, "threshold_area25": 75.25335582261579}, "test": {"score": [66.95119530385956, 67.45880951705755, 67.782094546692, 68.01179706774806, 68.07134957320704, 68.03164790290106, 67.99478206618835, 67.99194623259507, 67.94090122791594, 67.93239372713609, 67.93522956072937, 67.94090122791594, 67.91821455916967, 67.9125428919831, 67.90403539120324, 67.90119955760996, 67.86433372089725, 67.86716955449053, 67.86716955449053, 67.85015455293083, 67.85299038652411, 67.83030371777784, 67.85015455293083, 67.85299038652411, 67.86149788730397, 67.86149788730397, 67.86433372089725, 67.83597538496441, 67.78776621387857, 67.77925871309871, 67.74239287638602, 67.74806454357258, 67.69418370530018, 67.64597453421433, 67.60627286390834, 67.60060119672178, 67.54955619204264, 67.50701868814339, 67.44179451549783, 67.40492867878513, 67.33686867254629, 67.28015200068059, 67.2234353288149, 67.15253948898278, 67.07313614837082, 66.99656864135213, 66.89731446558716, 66.80940362419533, 66.71582111561693, 66.60522360547883, 66.54283526642656, 66.4435810906616, 66.35850608286306, 66.21104273601225, 66.10611689306072, 66.00686271729575, 65.86507103763151, 65.77148852905312, 65.62118934860902, 65.48506933613135, 65.32342682131413, 65.09656013385134, 64.94909678700054, 64.78178260499674, 64.56909508550038, 64.37625840115702, 64.11819754416811, 63.89416669029862, 63.70416583954853, 63.485806652865605, 63.258939965402824, 63.00371494200721, 62.73714658423843, 62.413861554603976, 62.17848736636134, 61.90624734140601, 61.60564898051783, 61.2738564501035, 60.93355641890934, 60.661316393954, 60.24444885574114, 59.864447154240985, 59.54966962538638, 59.22922042934521, 58.78683038879279, 58.25369367325525, 57.79145279754984, 57.400107761676544, 56.88398604769872, 56.475626010265714, 56.03890763689987, 55.599353429940734, 55.23920256359357, 54.6380058418172, 54.09636162549981, 53.55188157558914, 52.89963984913365, 52.27008479142443, 51.711425573547345, 50.95425800414032, 50.00708958398321], "calls": [0.0, 2.0077701840456004, 4.330317896945807, 8.016901568215976, 10.302583444403481, 11.351841873918838, 12.304681961262514, 13.260357882199472, 14.451407991379067, 15.78141394662961, 17.264554915917536, 18.603068371947934, 19.935910160791764, 20.917108584068288, 21.82741116751269, 22.666817911124976, 23.509060488330544, 24.40234807021524, 25.170858973995408, 25.976235714488272, 26.656835776876616, 27.3289283384851, 27.94430422822789, 28.704307631228197, 29.228936845985874, 29.742222726370414, 30.439837790318464, 31.075064515214244, 31.79820208150186, 32.430592972804355, 32.97507302271503, 33.53656807418541, 34.27388480843944, 34.90627569974194, 35.626577432436264, 36.2334458213992, 36.84031421036214, 37.62016844851544, 38.36882851714261, 38.944502736579416, 39.71868530754615, 40.54107704959873, 41.366304625244595, 42.22839803760316, 43.09899895074157, 43.830644017809036, 44.57079658565635, 45.39318832770893, 46.30632674474662, 47.193942659444744, 47.99081189915776, 48.94081615290815, 49.79440206448685, 50.67351047840513, 51.72276890792048, 52.50262314607379, 53.4299407310779, 54.39695998638799, 55.38383007685109, 56.424581005586596, 57.400107761676544, 58.39548535291949, 59.098772084054104, 60.1792246830956, 61.183109775118396, 62.12177069449565, 63.20222329353713, 64.31103422851146, 65.26954598304171, 66.38402858520263, 67.37089867566571, 68.4683662762669, 69.42687803079716, 70.63777897512973, 71.64166406715255, 72.48390664435811, 73.40271672858236, 74.45197515809772, 75.39630774466154, 76.36049116637835, 77.60825794742364, 78.59796387148002, 79.59617729631626, 80.56319655162635, 81.66633581941412, 82.95947593795196, 84.18739188384426, 85.23381447976632, 86.52695459830417, 87.5450188582934, 88.6197997901483, 89.6804015540368, 90.55100246717522, 91.8186200833735, 93.03519269489267, 94.11280946034086, 95.24714289765477, 96.35028216544254, 97.33431642231234, 98.5310381986785, 100.0], "rescued": [0, 222, 361, 471, 545, 598, 638, 687, 728, 751, 770, 794, 806, 818, 831, 842, 851, 864, 877, 885, 901, 915, 934, 951, 965, 983, 1001, 1012, 1027, 1047, 1062, 1079, 1095, 1109, 1127, 1148, 1161, 1186, 1200, 1214, 1233, 1249, 1269, 1292, 1319, 1340, 1354, 1370, 1395, 1412, 1432, 1459, 1483, 1506, 1538, 1556, 1573, 1605, 1632, 1665, 1685, 1708, 1723, 1752, 1774, 1794, 1818, 1842, 1860, 1891, 1918, 1944, 1965, 1990, 2018, 2040, 2061, 2087, 2110, 2142, 2173, 2190, 2207, 2231, 2252, 2276, 2307, 2324, 2361, 2389, 2417, 2441, 2464, 2491, 2518, 2543, 2561, 2579, 2598, 2608, 2627], "harmed": [0, 43, 68, 97, 150, 217, 270, 320, 379, 405, 423, 445, 465, 479, 495, 507, 529, 541, 554, 568, 583, 605, 617, 633, 644, 662, 679, 700, 732, 755, 783, 798, 833, 864, 896, 919, 950, 990, 1027, 1054, 1097, 1133, 1173, 1221, 1276, 1324, 1373, 1420, 1478, 1534, 1576, 1638, 1692, 1767, 1836, 1889, 1956, 2021, 2101, 2182, 2259, 2362, 2429, 2517, 2614, 2702, 2817, 2920, 3005, 3113, 3220, 3336, 3451, 3590, 3701, 3819, 3946, 4089, 4232, 4360, 4538, 4689, 4817, 4954, 5131, 5343, 5537, 5692, 5911, 6083, 6265, 6444, 6594, 6833, 7051, 7268, 7516, 7756, 7972, 8249, 8602], "exact_score": [66.95119530385956, 67.21776366162834, 67.45880951705755, 67.64030286702776, 67.75373621075916, 67.86433372089725, 67.9295578935428, 67.97493123103536, 68.01463290134134, 68.03164790290106, 68.07134957320704, 68.02881206930778, 67.99478206618835, 68.00045373337493, 67.97209539744208, 67.94940872869579, 67.93522956072937, 67.93239372713609, 67.93806539432265, 67.94940872869579, 67.91821455916967, 67.91537872557637, 67.90403539120324, 67.87851288886368, 67.86149788730397, 67.86716955449053, 67.84731871933755, 67.84164705215098, 67.84448288574426, 67.85299038652411, 67.86433372089725, 67.84731871933755, 67.78493038028527, 67.7452287099793, 67.71403454045316, 67.63746703343448, 67.61194453109492, 67.54388452485608, 67.47866035221053, 67.39925701159855, 67.32836117176643, 67.25462949834103, 67.17239032413578, 67.08164364915066, 66.97388197260584, 66.85761279528117, 66.73567195076993, 66.64208944219153, 66.53999943283327, 66.43507358988174, 66.32447607974365, 66.17134106570626, 66.07208688994129, 65.93313104387035, 65.81402603295238, 65.69775685562772, 65.55880100955676, 65.36880015880668, 65.18163514164989, 64.96327595496696, 64.80446927374301, 64.6201400901795, 64.415960071463, 64.16640671525394, 63.976405864503874, 63.746703343447805, 63.55386665910444, 63.352522473981224, 63.128491620111724, 62.86475909593625, 62.589683237387625, 62.325950713212144, 62.01968068513739, 61.747440660182065, 61.39012562742818, 61.10370643450642, 60.74071973456597, 60.46564387601735, 60.08280634092391, 59.751013810509605, 59.40220627853557, 59.04205541218841, 58.659217877094974, 58.24518617247539, 57.85951280378867, 57.496526103848225, 57.099509400788364, 56.708164364915056, 56.29129682670221, 55.88577262286248, 55.48875591980262, 55.01517170972407, 54.56143833479852, 54.10770495987295, 53.61710574823469, 53.03008819442475, 52.505458979667075, 51.884411422737706, 51.30022970252106, 50.622465473725995, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.892467586644386, "threshold_area100": 64.19225533845673, "threshold_area25": 67.82548280066926}, "selected_budget": 4, "selected": {"val": {"score": 75.57804483800689, "calls": 3.998875535877434, "rescued": 103, "harmed": 25}, "test": {"score": 68.07134957320704, "calls": 10.302583444403481, "rescued": 545, "harmed": 150}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.09087849408388138, "operator": "ge"}, {"budget": 0.02, "threshold": 0.025624053552746773, "operator": "ge"}, {"budget": 0.03, "threshold": 0.0035271011292934418, "operator": "ge"}, {"budget": 0.04, "threshold": -0.00392303429543972, "operator": "ge"}, {"budget": 0.05, "threshold": -0.006280256900936365, "operator": "ge"}, {"budget": 0.06, "threshold": -0.008528647944331169, "operator": "ge"}, {"budget": 0.07, "threshold": -0.010710839182138443, "operator": "ge"}, {"budget": 0.08, "threshold": -0.012798050418496132, "operator": "ge"}, {"budget": 0.09, "threshold": -0.014948787167668343, "operator": "ge"}, {"budget": 0.1, "threshold": -0.017046784982085228, "operator": "ge"}, {"budget": 0.11, "threshold": -0.019566567614674568, "operator": "ge"}, {"budget": 0.12, "threshold": -0.02255132980644703, "operator": "ge"}, {"budget": 0.13, "threshold": -0.025517083704471588, "operator": "ge"}, {"budget": 0.14, "threshold": -0.028461743146181107, "operator": "ge"}, {"budget": 0.15, "threshold": -0.03177615627646446, "operator": "ge"}, {"budget": 0.16, "threshold": -0.03508714586496353, "operator": "ge"}, {"budget": 0.17, "threshold": -0.038769062608480453, "operator": "ge"}, {"budget": 0.18, "threshold": -0.04240446537733078, "operator": "ge"}, {"budget": 0.19, "threshold": -0.04660781845450401, "operator": "ge"}, {"budget": 0.2, "threshold": -0.051215823739767075, "operator": "ge"}, {"budget": 0.21, "threshold": -0.05579766258597374, "operator": "ge"}, {"budget": 0.22, "threshold": -0.05984849855303764, "operator": "ge"}, {"budget": 0.23, "threshold": -0.06462667882442474, "operator": "ge"}, {"budget": 0.24, "threshold": -0.06996370851993561, "operator": "ge"}, {"budget": 0.25, "threshold": -0.07576054334640503, "operator": "ge"}, {"budget": 0.26, "threshold": -0.08100122213363647, "operator": "ge"}, {"budget": 0.27, "threshold": -0.08751485496759415, "operator": "ge"}, {"budget": 0.28, "threshold": -0.09441965818405151, "operator": "ge"}, {"budget": 0.29, "threshold": -0.10075697302818298, "operator": "ge"}, {"budget": 0.3, "threshold": -0.10771475732326508, "operator": "ge"}, {"budget": 0.31, "threshold": -0.11542642116546631, "operator": "ge"}, {"budget": 0.32, "threshold": -0.1225140243768692, "operator": "ge"}, {"budget": 0.33, "threshold": -0.13089440762996674, "operator": "ge"}, {"budget": 0.34, "threshold": -0.14016804099082947, "operator": "ge"}, {"budget": 0.35, "threshold": -0.14772357046604156, "operator": "ge"}, {"budget": 0.36, "threshold": -0.15598978102207184, "operator": "ge"}, {"budget": 0.37, "threshold": -0.16440105438232422, "operator": "ge"}, {"budget": 0.38, "threshold": -0.17336462438106537, "operator": "ge"}, {"budget": 0.39, "threshold": -0.1839776486158371, "operator": "ge"}, {"budget": 0.4, "threshold": -0.19268420338630676, "operator": "ge"}, {"budget": 0.41, "threshold": -0.2031482756137848, "operator": "ge"}, {"budget": 0.42, "threshold": -0.21325555443763733, "operator": "ge"}, {"budget": 0.43, "threshold": -0.22487220168113708, "operator": "ge"}, {"budget": 0.44, "threshold": -0.23556573688983917, "operator": "ge"}, {"budget": 0.45, "threshold": -0.24880677461624146, "operator": "ge"}, {"budget": 0.46, "threshold": -0.25941410660743713, "operator": "ge"}, {"budget": 0.47, "threshold": -0.27117857336997986, "operator": "ge"}, {"budget": 0.48, "threshold": -0.2819139361381531, "operator": "ge"}, {"budget": 0.49, "threshold": -0.2936640977859497, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3047676086425781, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3153848946094513, "operator": "ge"}, {"budget": 0.52, "threshold": -0.32588592171669006, "operator": "ge"}, {"budget": 0.53, "threshold": -0.33750295639038086, "operator": "ge"}, {"budget": 0.54, "threshold": -0.3486907482147217, "operator": "ge"}, {"budget": 0.55, "threshold": -0.3619370758533478, "operator": "gt"}, {"budget": 0.56, "threshold": -0.37500736117362976, "operator": "ge"}, {"budget": 0.57, "threshold": -0.3876194953918457, "operator": "ge"}, {"budget": 0.58, "threshold": -0.3992593288421631, "operator": "ge"}, {"budget": 0.59, "threshold": -0.4104945659637451, "operator": "ge"}, {"budget": 0.6, "threshold": -0.42273497581481934, "operator": "ge"}, {"budget": 0.61, "threshold": -0.43385225534439087, "operator": "ge"}, {"budget": 0.62, "threshold": -0.44697239995002747, "operator": "ge"}, {"budget": 0.63, "threshold": -0.45939379930496216, "operator": "ge"}, {"budget": 0.64, "threshold": -0.4721294045448303, "operator": "ge"}, {"budget": 0.65, "threshold": -0.48377177119255066, "operator": "ge"}, {"budget": 0.66, "threshold": -0.4978811740875244, "operator": "ge"}, {"budget": 0.67, "threshold": -0.5114349722862244, "operator": "ge"}, {"budget": 0.68, "threshold": -0.5222681164741516, "operator": "ge"}, {"budget": 0.69, "threshold": -0.5358705520629883, "operator": "ge"}, {"budget": 0.7, "threshold": -0.5494012236595154, "operator": "ge"}, {"budget": 0.71, "threshold": -0.5620163679122925, "operator": "ge"}, {"budget": 0.72, "threshold": -0.5756837725639343, "operator": "ge"}, {"budget": 0.73, "threshold": -0.5892966389656067, "operator": "ge"}, {"budget": 0.74, "threshold": -0.6027467846870422, "operator": "ge"}, {"budget": 0.75, "threshold": -0.615939736366272, "operator": "ge"}, {"budget": 0.76, "threshold": -0.6326971650123596, "operator": "ge"}, {"budget": 0.77, "threshold": -0.646866500377655, "operator": "ge"}, {"budget": 0.78, "threshold": -0.6601597666740417, "operator": "ge"}, {"budget": 0.79, "threshold": -0.6743647456169128, "operator": "ge"}, {"budget": 0.8, "threshold": -0.6882981657981873, "operator": "ge"}, {"budget": 0.81, "threshold": -0.702907919883728, "operator": "ge"}, {"budget": 0.82, "threshold": -0.715613842010498, "operator": "ge"}, {"budget": 0.83, "threshold": -0.730353593826294, "operator": "ge"}, {"budget": 0.84, "threshold": -0.7433176040649414, "operator": "ge"}, {"budget": 0.85, "threshold": -0.7571570873260498, "operator": "ge"}, {"budget": 0.86, "threshold": -0.7709317803382874, "operator": "ge"}, {"budget": 0.87, "threshold": -0.7828677892684937, "operator": "ge"}, {"budget": 0.88, "threshold": -0.7988614439964294, "operator": "ge"}, {"budget": 0.89, "threshold": -0.8125402927398682, "operator": "ge"}, {"budget": 0.9, "threshold": -0.8271538615226746, "operator": "ge"}, {"budget": 0.91, "threshold": -0.84130859375, "operator": "ge"}, {"budget": 0.92, "threshold": -0.8540235161781311, "operator": "ge"}, {"budget": 0.93, "threshold": -0.8678356409072876, "operator": "ge"}, {"budget": 0.94, "threshold": -0.8825212121009827, "operator": "ge"}, {"budget": 0.95, "threshold": -0.8961950540542603, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9106016755104065, "operator": "gt"}, {"budget": 0.97, "threshold": -0.9269237518310547, "operator": "gt"}, {"budget": 0.98, "threshold": -0.9444175958633423, "operator": "gt"}, {"budget": 0.99, "threshold": -0.9627425670623779, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.29692880736523, 75.4093752196219, 75.56398903647481, 75.6131843418371, 75.60615644107105, 75.6131843418371, 75.6131843418371, 75.59210063953897, 75.57804483800689, 75.57804483800689, 75.57804483800689, 75.55696113570876, 75.55696113570876, 75.54290533417668, 75.5288495326446, 75.51479373111252, 75.50776583034647, 75.47262632651626, 75.4445147234521, 75.39531941808981, 75.36017991425962, 75.36017991425962, 75.35315201349357, 75.31801250966336, 75.33206831119544, 75.254761402769, 75.2055660974067, 75.1704265935765, 75.1352870897463, 75.1212312882142, 75.10717548668214, 75.02986857825567, 74.91742216599901, 74.87525476140277, 74.78389205144424, 74.74172464684798, 74.67847353995361, 74.58711082999508, 74.56602712769696, 74.42546911237613, 74.404385410078, 74.3341064024176, 74.27085529552323, 74.2849110970553, 74.17949258556469, 74.15138098250053, 74.08812987560616, 74.01785086794574, 74.0108229671797, 73.95459976105137, 73.93351605875326, 73.84918124956076, 73.76484644036826, 73.71565113500597, 73.70159533347389, 73.5821210204512, 73.4837304097266, 73.49075831049265, 73.42047930283225, 73.46264670742849, 73.45561880666244, 73.44859090589641, 73.27992128751141, 73.16747487525477, 73.05502846299811, 72.95663785227353, 72.80202403542062, 72.71066132546208, 72.59118701243938, 72.40846159252231, 72.3733220886921, 72.10626185958255, 71.95164804272964, 71.76189472204652, 71.61430880595965, 71.501862393703, 71.41752758451051, 71.17857895846511, 71.05207674467636, 70.85529552322721, 70.67959800407618, 70.41253777496662, 70.2719797596458, 69.9838358282381, 69.79408250755499, 69.37240846159253, 69.17562724014337, 68.90856701103381, 68.71178578958465, 68.42364185817696, 68.13549792676928, 67.8332981938295, 67.44676365169724, 67.04617330803289, 66.6736945674327, 66.30121582683253, 65.82331857474173, 65.30325391805468, 64.71291025370722, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.98629559350622, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.98706866259047, 96.99205847213437, 97.99002038091223, 98.99501019045611, 100.0], "rescued": [0, 42, 64, 88, 97, 99, 100, 100, 100, 100, 102, 103, 103, 105, 106, 108, 109, 111, 113, 116, 119, 122, 125, 129, 133, 136, 136, 138, 139, 150, 159, 164, 168, 176, 183, 188, 193, 201, 208, 220, 225, 236, 246, 254, 266, 279, 291, 301, 315, 334, 348, 364, 377, 387, 409, 429, 447, 465, 490, 513, 538, 562, 586, 607, 623, 639, 657, 673, 698, 717, 736, 762, 774, 795, 820, 847, 875, 903, 919, 947, 972, 995, 1020, 1057, 1076, 1103, 1114, 1140, 1167, 1196, 1216, 1240, 1258, 1274, 1293, 1309, 1325, 1335, 1349, 1358, 1369], "harmed": [0, 4, 10, 12, 14, 17, 17, 17, 20, 22, 24, 25, 28, 30, 33, 37, 40, 43, 50, 57, 67, 75, 78, 83, 92, 93, 104, 113, 119, 135, 146, 153, 168, 192, 205, 223, 234, 251, 271, 286, 311, 325, 345, 362, 372, 400, 416, 435, 459, 479, 501, 520, 545, 567, 596, 618, 653, 685, 709, 742, 761, 786, 811, 856, 888, 920, 952, 990, 1028, 1064, 1109, 1140, 1190, 1233, 1285, 1333, 1377, 1417, 1467, 1513, 1566, 1614, 1677, 1734, 1794, 1848, 1919, 1973, 2038, 2095, 2156, 2221, 2282, 2353, 2429, 2498, 2567, 2645, 2733, 2826, 2921], "exact_score": [75.02986857825567, 75.29692880736523, 75.4093752196219, 75.56398903647481, 75.6131843418371, 75.60615644107105, 75.6131843418371, 75.6131843418371, 75.59210063953897, 75.57804483800689, 75.57804483800689, 75.57804483800689, 75.55696113570876, 75.55696113570876, 75.54290533417668, 75.5288495326446, 75.51479373111252, 75.50776583034647, 75.47262632651626, 75.4445147234521, 75.39531941808981, 75.36017991425962, 75.36017991425962, 75.35315201349357, 75.31801250966336, 75.33206831119544, 75.254761402769, 75.2055660974067, 75.1704265935765, 75.1352870897463, 75.1212312882142, 75.10717548668214, 75.02986857825567, 74.91742216599901, 74.87525476140277, 74.78389205144424, 74.74172464684798, 74.67847353995361, 74.58711082999508, 74.56602712769696, 74.42546911237613, 74.404385410078, 74.3341064024176, 74.27085529552323, 74.2849110970553, 74.17949258556469, 74.15138098250053, 74.08812987560616, 74.01785086794574, 74.0108229671797, 73.95459976105137, 73.93351605875326, 73.84918124956076, 73.76484644036826, 73.71565113500597, 73.70159533347389, 73.5821210204512, 73.4837304097266, 73.49075831049265, 73.42047930283225, 73.46264670742849, 73.45561880666244, 73.44859090589641, 73.27992128751141, 73.16747487525477, 73.05502846299811, 72.95663785227353, 72.80202403542062, 72.71066132546208, 72.59118701243938, 72.40846159252231, 72.3733220886921, 72.10626185958255, 71.95164804272964, 71.76189472204652, 71.61430880595965, 71.501862393703, 71.41752758451051, 71.17857895846511, 71.05207674467636, 70.85529552322721, 70.67959800407618, 70.41253777496662, 70.2719797596458, 69.9838358282381, 69.79408250755499, 69.37240846159253, 69.17562724014337, 68.90856701103381, 68.71178578958465, 68.42364185817696, 68.13549792676928, 67.8332981938295, 67.44676365169724, 67.04617330803289, 66.6736945674327, 66.29418792606648, 65.81629067397569, 65.29622601728863, 64.71291025370722, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 72.99202554783434, "threshold_area100": 72.99262070419566, "threshold_area25": 75.4578677349076}, "test": {"score": [66.95119530385956, 67.46448118424411, 67.77358704591214, 68.01746873493462, 68.04582707086749, 68.04582707086749, 68.04866290446077, 68.03731957008763, 68.02314040212121, 68.0259762357145, 68.01746873493462, 68.03164790290106, 68.03731957008763, 68.02881206930778, 68.02881206930778, 68.03448373649434, 68.0429912372742, 68.04582707086749, 68.03731957008763, 68.03731957008763, 68.02314040212121, 68.01179706774806, 68.00896123415477, 67.98343873181521, 67.93522956072937, 67.87851288886368, 67.84731871933755, 67.81045288262484, 67.79060204747185, 67.78493038028527, 67.75940787794572, 67.71119870685988, 67.67716870374046, 67.64597453421433, 67.59492952953521, 67.56657119360236, 67.52119785610981, 67.48149618580382, 67.43045118112468, 67.3170178373933, 67.25462949834103, 67.16671865694921, 67.1128378186768, 67.01074780931854, 66.92567280152001, 66.83776196012818, 66.78104528826249, 66.69029861327736, 66.56268610157956, 66.45492442503473, 66.30746107818393, 66.19119190085925, 66.10044522587415, 65.96716104698976, 65.85089186966509, 65.680741854068, 65.4935768369112, 65.292232651788, 65.17312764087004, 65.0285001276125, 64.86402177920199, 64.6768567620452, 64.48402007770184, 64.28834755976519, 64.07566004026883, 63.82894251765306, 63.49431415364546, 63.23908913024984, 63.09162578339903, 62.878938263902675, 62.6123699061339, 62.35998071633156, 62.048039021070245, 61.7587839945552, 61.432663131327445, 61.1632589399654, 60.780421404871966, 60.49683804554349, 60.1593738479426, 59.804894648782, 59.48444545274083, 59.05056291296826, 58.698919547400955, 58.25369367325525, 57.97294614752006, 57.61563111476619, 57.19025607577347, 56.844284377392725, 56.3848793352806, 55.98786263222074, 55.534129257295184, 55.12576921986218, 54.75427501914187, 54.31755664577602, 53.841136602104186, 53.36188072483906, 52.82590817570825, 52.21903978674531, 51.49873805405099, 50.74724215183053, 50.00708958398321], "calls": [0.0, 2.010606017638885, 4.324646229759238, 8.011229901029408, 10.21750843660494, 11.746022743385417, 13.144088704874799, 14.391855485920086, 15.24827723109208, 16.03947480361852, 16.75410486912628, 17.61903411507813, 18.345007514959022, 19.09083174999291, 19.709043473328986, 20.281881859172504, 20.83486940986303, 21.350991123840853, 21.784873663613418, 22.400249553356208, 23.038312111845276, 23.594135496129088, 24.053540538241215, 24.535632249099624, 25.20772481070811, 25.81175736607776, 26.285341576156313, 26.889374131525962, 27.555795025947877, 28.19102175084366, 28.80072597339988, 29.447296032668802, 30.0796869239713, 30.73760031761336, 31.51461872217338, 32.166860448628874, 32.88149051413663, 33.630150582763804, 34.293735643592434, 35.20970989422341, 35.955534129257295, 36.792105039276294, 37.674049286787856, 38.55315770070612, 39.55704279272892, 40.69704789722939, 41.630037149420076, 42.52616056489805, 43.42228398037603, 44.40631823724584, 45.44423333238806, 46.323341746306326, 47.31588350395599, 48.36230609987806, 49.23574284660976, 50.3984346198565, 51.549783058730114, 52.58769815387233, 53.46113490060404, 54.39979581998128, 55.363979241698104, 56.33667016419477, 57.496526103848225, 58.51175453024416, 59.626237132405066, 60.73788390097269, 62.07072568981652, 63.32699997164166, 64.18342171681365, 65.34044182287383, 66.50596942971386, 67.5580636928225, 68.77180047074837, 69.85508890338315, 71.0177806766299, 72.03584493661911, 73.4707767348212, 74.6816776791538, 75.83586195162067, 77.1346737373451, 78.28318634262541, 79.50259478773786, 80.5717040524062, 81.6691716530074, 82.64469840909734, 83.71097184017242, 84.86515611263931, 85.778294529677, 86.94665797011032, 87.99591639962567, 89.0508464963276, 90.09726909224966, 90.99622834132093, 91.99727759975045, 93.06922269801208, 94.07877945722146, 95.13654538751666, 96.27655049201714, 97.55267560899526, 98.74656155176814, 100.0], "rescued": [0, 222, 358, 471, 500, 504, 509, 510, 512, 518, 521, 531, 538, 542, 550, 555, 564, 573, 577, 587, 597, 606, 612, 619, 630, 636, 641, 656, 675, 688, 703, 710, 721, 732, 742, 758, 775, 794, 809, 827, 840, 853, 873, 892, 916, 954, 980, 1002, 1023, 1057, 1085, 1108, 1135, 1171, 1203, 1228, 1260, 1286, 1317, 1344, 1382, 1410, 1447, 1481, 1521, 1562, 1604, 1635, 1664, 1708, 1745, 1776, 1820, 1845, 1878, 1916, 1965, 2011, 2048, 2088, 2126, 2148, 2174, 2202, 2243, 2278, 2316, 2341, 2364, 2393, 2418, 2446, 2473, 2496, 2524, 2542, 2564, 2588, 2605, 2610, 2627], "harmed": [0, 41, 68, 95, 114, 118, 122, 127, 134, 139, 145, 150, 155, 162, 170, 173, 179, 187, 194, 204, 219, 232, 239, 255, 283, 309, 325, 353, 379, 394, 418, 442, 465, 487, 515, 541, 574, 607, 640, 698, 733, 777, 816, 871, 925, 994, 1040, 1094, 1160, 1232, 1312, 1376, 1435, 1518, 1591, 1676, 1774, 1871, 1944, 2022, 2118, 2212, 2317, 2420, 2535, 2663, 2823, 2944, 3025, 3144, 3275, 3395, 3549, 3676, 3824, 3957, 4141, 4287, 4443, 4608, 4759, 4934, 5084, 5269, 5409, 5570, 5758, 5905, 6090, 6259, 6444, 6616, 6774, 6951, 7147, 7334, 7545, 7783, 8054, 8324, 8602], "exact_score": [66.95119530385956, 67.22059949522162, 67.46448118424411, 67.64313870062104, 67.74239287638602, 67.86149788730397, 67.9295578935428, 67.97493123103536, 68.01746873493462, 68.03164790290106, 68.04015540368091, 68.04015540368091, 68.04582707086749, 68.04866290446077, 68.03731957008763, 68.02881206930778, 68.0203045685279, 68.01463290134134, 68.03164790290106, 68.02881206930778, 68.03731957008763, 68.03731957008763, 68.03731957008763, 68.02314040212121, 68.01179706774806, 67.94090122791594, 67.86149788730397, 67.81045288262484, 67.7650795451323, 67.75090037716586, 67.67716870374046, 67.63746703343448, 67.57507869438221, 67.51552618892323, 67.45597368346425, 67.33686867254629, 67.24612199756118, 67.1695544905425, 67.07880781555738, 66.98522530697898, 66.89164279840058, 66.82925445934832, 66.73850778436321, 66.64208944219153, 66.50596942971386, 66.39537191957575, 66.2195502367921, 66.1458185633667, 66.01820605166888, 65.89626520715764, 65.74313019312027, 65.5701443439299, 65.39432266114625, 65.22700847914244, 65.0880526330715, 64.92357428466097, 64.74208093469075, 64.59461758783995, 64.3790942347503, 64.17207838244052, 63.95655502935087, 63.77222584578737, 63.51700082239174, 63.34685080679465, 63.128491620111724, 62.9668491052945, 62.73714658423843, 62.518787397555506, 62.24654737260017, 61.99415818279783, 61.71908232424921, 61.43833479851401, 61.17460227433854, 60.91370558375634, 60.66698806114057, 60.40609137055837, 60.13385134560304, 59.844596319088, 59.592207129285654, 59.24339959731163, 58.8775770637779, 58.52026203102402, 58.1884695006097, 57.870856138161805, 57.516376939001226, 57.12503190312792, 56.75353770240762, 56.37637183450075, 55.98502679862746, 55.55965175963474, 55.171142557354734, 54.75711085273516, 54.31755664577602, 53.90352494115645, 53.401582395145056, 52.90531151632022, 52.36933896718941, 51.819187250092156, 51.243513030655365, 50.5855996370133, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 63.543122932685, "threshold_area100": 63.94213481552903, "threshold_area25": 67.90715480815585}, "selected_budget": 4, "selected": {"val": {"score": 75.6131843418371, "calls": 3.998875535877434, "rescued": 97, "harmed": 14}, "test": {"score": 68.04582707086749, "calls": 10.21750843660494, "rescued": 500, "harmed": 114}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.381363868713379, "operator": "ge"}, {"budget": 0.02, "threshold": 1.7016054391860962, "operator": "ge"}, {"budget": 0.03, "threshold": 0.20276392996311188, "operator": "ge"}, {"budget": 0.04, "threshold": -0.4260694682598114, "operator": "ge"}, {"budget": 0.05, "threshold": -0.8387303352355957, "operator": "ge"}, {"budget": 0.06, "threshold": -1.155752182006836, "operator": "ge"}, {"budget": 0.07, "threshold": -1.5446114540100098, "operator": "ge"}, {"budget": 0.08, "threshold": -1.928466558456421, "operator": "ge"}, {"budget": 0.09, "threshold": -2.2415478229522705, "operator": "ge"}, {"budget": 0.1, "threshold": -2.5337891578674316, "operator": "ge"}, {"budget": 0.11, "threshold": -2.817047119140625, "operator": "ge"}, {"budget": 0.12, "threshold": -3.0483081340789795, "operator": "ge"}, {"budget": 0.13, "threshold": -3.2518348693847656, "operator": "ge"}, {"budget": 0.14, "threshold": -3.4545834064483643, "operator": "ge"}, {"budget": 0.15, "threshold": -3.625873327255249, "operator": "ge"}, {"budget": 0.16, "threshold": -3.789567232131958, "operator": "ge"}, {"budget": 0.17, "threshold": -3.9365460872650146, "operator": "ge"}, {"budget": 0.18, "threshold": -4.085773468017578, "operator": "ge"}, {"budget": 0.19, "threshold": -4.2132134437561035, "operator": "ge"}, {"budget": 0.2, "threshold": -4.317574977874756, "operator": "ge"}, {"budget": 0.21, "threshold": -4.427010536193848, "operator": "ge"}, {"budget": 0.22, "threshold": -4.528843879699707, "operator": "ge"}, {"budget": 0.23, "threshold": -4.6277079582214355, "operator": "ge"}, {"budget": 0.24, "threshold": -4.709056377410889, "operator": "ge"}, {"budget": 0.25, "threshold": -4.81101655960083, "operator": "ge"}, {"budget": 0.26, "threshold": -4.894411087036133, "operator": "ge"}, {"budget": 0.27, "threshold": -4.979164123535156, "operator": "ge"}, {"budget": 0.28, "threshold": -5.0636444091796875, "operator": "ge"}, {"budget": 0.29, "threshold": -5.146122455596924, "operator": "ge"}, {"budget": 0.3, "threshold": -5.233458042144775, "operator": "ge"}, {"budget": 0.31, "threshold": -5.305995464324951, "operator": "ge"}, {"budget": 0.32, "threshold": -5.389784812927246, "operator": "ge"}, {"budget": 0.33, "threshold": -5.456990718841553, "operator": "ge"}, {"budget": 0.34, "threshold": -5.523298740386963, "operator": "ge"}, {"budget": 0.35, "threshold": -5.58878231048584, "operator": "ge"}, {"budget": 0.36, "threshold": -5.659017086029053, "operator": "ge"}, {"budget": 0.37, "threshold": -5.720398902893066, "operator": "ge"}, {"budget": 0.38, "threshold": -5.793390274047852, "operator": "ge"}, {"budget": 0.39, "threshold": -5.862624645233154, "operator": "ge"}, {"budget": 0.4, "threshold": -5.939906120300293, "operator": "ge"}, {"budget": 0.41, "threshold": -6.012700080871582, "operator": "ge"}, {"budget": 0.42, "threshold": -6.104462146759033, "operator": "ge"}, {"budget": 0.43, "threshold": -6.175923824310303, "operator": "ge"}, {"budget": 0.44, "threshold": -6.243216037750244, "operator": "ge"}, {"budget": 0.45, "threshold": -6.312685966491699, "operator": "ge"}, {"budget": 0.46, "threshold": -6.375117778778076, "operator": "ge"}, {"budget": 0.47, "threshold": -6.440752983093262, "operator": "ge"}, {"budget": 0.48, "threshold": -6.507706165313721, "operator": "ge"}, {"budget": 0.49, "threshold": -6.574911117553711, "operator": "ge"}, {"budget": 0.5, "threshold": -6.638130187988281, "operator": "ge"}, {"budget": 0.51, "threshold": -6.701938629150391, "operator": "gt"}, {"budget": 0.52, "threshold": -6.779728412628174, "operator": "ge"}, {"budget": 0.53, "threshold": -6.837524890899658, "operator": "ge"}, {"budget": 0.54, "threshold": -6.907869338989258, "operator": "ge"}, {"budget": 0.55, "threshold": -6.972716808319092, "operator": "ge"}, {"budget": 0.56, "threshold": -7.042940616607666, "operator": "ge"}, {"budget": 0.57, "threshold": -7.116978168487549, "operator": "ge"}, {"budget": 0.58, "threshold": -7.191741943359375, "operator": "ge"}, {"budget": 0.59, "threshold": -7.267473220825195, "operator": "ge"}, {"budget": 0.6, "threshold": -7.333744049072266, "operator": "ge"}, {"budget": 0.61, "threshold": -7.413714408874512, "operator": "ge"}, {"budget": 0.62, "threshold": -7.477631092071533, "operator": "ge"}, {"budget": 0.63, "threshold": -7.5748467445373535, "operator": "ge"}, {"budget": 0.64, "threshold": -7.645713806152344, "operator": "ge"}, {"budget": 0.65, "threshold": -7.705920219421387, "operator": "ge"}, {"budget": 0.66, "threshold": -7.771568298339844, "operator": "ge"}, {"budget": 0.67, "threshold": -7.843412399291992, "operator": "ge"}, {"budget": 0.68, "threshold": -7.915142059326172, "operator": "ge"}, {"budget": 0.69, "threshold": -7.986194610595703, "operator": "ge"}, {"budget": 0.7, "threshold": -8.051437377929688, "operator": "ge"}, {"budget": 0.71, "threshold": -8.130361557006836, "operator": "ge"}, {"budget": 0.72, "threshold": -8.209001541137695, "operator": "ge"}, {"budget": 0.73, "threshold": -8.267797470092773, "operator": "ge"}, {"budget": 0.74, "threshold": -8.341339111328125, "operator": "ge"}, {"budget": 0.75, "threshold": -8.39980411529541, "operator": "ge"}, {"budget": 0.76, "threshold": -8.47028636932373, "operator": "ge"}, {"budget": 0.77, "threshold": -8.53243637084961, "operator": "ge"}, {"budget": 0.78, "threshold": -8.598625183105469, "operator": "ge"}, {"budget": 0.79, "threshold": -8.657618522644043, "operator": "ge"}, {"budget": 0.8, "threshold": -8.728010177612305, "operator": "ge"}, {"budget": 0.81, "threshold": -8.792327880859375, "operator": "ge"}, {"budget": 0.82, "threshold": -8.857803344726562, "operator": "ge"}, {"budget": 0.83, "threshold": -8.925423622131348, "operator": "ge"}, {"budget": 0.84, "threshold": -8.992267608642578, "operator": "ge"}, {"budget": 0.85, "threshold": -9.070560455322266, "operator": "ge"}, {"budget": 0.86, "threshold": -9.148712158203125, "operator": "ge"}, {"budget": 0.87, "threshold": -9.241538047790527, "operator": "ge"}, {"budget": 0.88, "threshold": -9.326690673828125, "operator": "ge"}, {"budget": 0.89, "threshold": -9.422123908996582, "operator": "ge"}, {"budget": 0.9, "threshold": -9.504585266113281, "operator": "ge"}, {"budget": 0.91, "threshold": -9.60620403289795, "operator": "ge"}, {"budget": 0.92, "threshold": -9.709369659423828, "operator": "ge"}, {"budget": 0.93, "threshold": -9.810715675354004, "operator": "ge"}, {"budget": 0.94, "threshold": -9.936190605163574, "operator": "ge"}, {"budget": 0.95, "threshold": -10.064810752868652, "operator": "ge"}, {"budget": 0.96, "threshold": -10.210124969482422, "operator": "ge"}, {"budget": 0.97, "threshold": -10.415746688842773, "operator": "ge"}, {"budget": 0.98, "threshold": -10.642526626586914, "operator": "ge"}, {"budget": 0.99, "threshold": -11.007469177246094, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.29692880736523, 75.53587743341063, 75.59912854030502, 75.50776583034647, 75.30395670813128, 75.30395670813128, 75.23367770047086, 75.27584510506712, 75.32504041042941, 75.36017991425962, 75.51479373111252, 75.71157495256166, 75.89430037247874, 76.0418862885656, 76.0418862885656, 76.0067467847354, 76.11919319699206, 76.32300231920726, 76.42139292993183, 76.37922552533558, 76.44950453299599, 76.28786281537705, 76.30191861690912, 76.2456954107808, 76.10513739545998, 76.02783048703353, 75.89430037247874, 75.76779815869, 75.6834633494975, 75.62724014336918, 75.4445147234521, 75.33206831119544, 75.1352870897463, 74.91039426523297, 74.79091995221027, 74.67847353995361, 74.47466441773842, 74.3341064024176, 74.2146320893949, 74.01785086794574, 73.84918124956076, 73.60320472274931, 73.39236769976807, 73.1745027760208, 72.97772155457164, 72.83716353925082, 72.57713121090731, 72.3381825848619, 72.1343734626467, 71.88136903506923, 71.64242040902383, 71.36833227914822, 71.17857895846511, 70.94665823318574, 70.7358212102045, 70.48984468339307, 70.19467285121934, 69.97680792747207, 69.78705460678896, 69.62541288917001, 69.33726895776232, 69.09129243095087, 68.8734275072036, 68.65556258345632, 68.45175346124113, 68.14252582753532, 67.96682830838428, 67.69976807927472, 67.49595895705953, 67.29917773561037, 67.10239651416123, 66.8775036896479, 66.73694567432707, 66.5190807505798, 66.39257853679106, 66.1957973153419, 66.0903788038513, 65.91468128470026, 65.73195586478319, 65.63356525405861, 65.47192353643968, 65.38056082648113, 65.29622601728863, 65.18377960503197, 65.09944479583949, 65.02213788741304, 64.92374727668846, 64.8113008644318, 64.76913345983554, 64.67074284911098, 64.58640803991848, 64.53018483379014, 64.46693372689577, 64.4036826200014, 64.33340361234099, 64.29123620774475, 64.24204090238246, 64.15770609318997, 64.12256658935976, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.98742005762878, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 38, 74, 91, 101, 115, 138, 159, 185, 215, 245, 286, 326, 372, 412, 441, 472, 515, 562, 601, 630, 663, 684, 712, 740, 763, 789, 814, 830, 854, 872, 887, 910, 925, 934, 952, 970, 983, 1004, 1024, 1036, 1047, 1061, 1075, 1086, 1100, 1112, 1124, 1140, 1157, 1168, 1179, 1187, 1195, 1207, 1217, 1224, 1232, 1240, 1247, 1259, 1265, 1269, 1273, 1276, 1282, 1287, 1295, 1299, 1309, 1312, 1314, 1318, 1324, 1329, 1333, 1333, 1336, 1338, 1338, 1342, 1344, 1346, 1348, 1351, 1354, 1357, 1358, 1359, 1361, 1361, 1361, 1363, 1363, 1364, 1364, 1365, 1366, 1367, 1369, 1369], "harmed": [0, 0, 2, 10, 33, 76, 99, 130, 150, 173, 198, 217, 229, 249, 268, 297, 333, 360, 378, 403, 438, 461, 505, 531, 567, 610, 647, 691, 725, 761, 787, 828, 867, 910, 951, 986, 1020, 1062, 1103, 1140, 1180, 1215, 1264, 1308, 1350, 1392, 1424, 1473, 1523, 1569, 1616, 1661, 1708, 1743, 1788, 1828, 1870, 1920, 1959, 1993, 2028, 2075, 2114, 2149, 2183, 2218, 2267, 2300, 2342, 2381, 2412, 2442, 2478, 2504, 2540, 2562, 2590, 2608, 2635, 2661, 2679, 2704, 2719, 2733, 2752, 2767, 2781, 2796, 2813, 2821, 2835, 2847, 2857, 2866, 2876, 2886, 2893, 2901, 2914, 2921, 2921], "exact_score": [75.02986857825567, 75.29692880736523, 75.53587743341063, 75.59912854030502, 75.50776583034647, 75.30395670813128, 75.30395670813128, 75.23367770047086, 75.27584510506712, 75.32504041042941, 75.36017991425962, 75.51479373111252, 75.71157495256166, 75.89430037247874, 76.0418862885656, 76.0418862885656, 76.0067467847354, 76.11919319699206, 76.32300231920726, 76.42139292993183, 76.37922552533558, 76.44950453299599, 76.28786281537705, 76.30191861690912, 76.2456954107808, 76.10513739545998, 76.02783048703353, 75.89430037247874, 75.76779815869, 75.6834633494975, 75.62724014336918, 75.4445147234521, 75.33206831119544, 75.1352870897463, 74.91039426523297, 74.79091995221027, 74.67847353995361, 74.47466441773842, 74.3341064024176, 74.2146320893949, 74.01785086794574, 73.84918124956076, 73.60320472274931, 73.39236769976807, 73.1745027760208, 72.97772155457164, 72.83716353925082, 72.57713121090731, 72.3381825848619, 72.1343734626467, 71.88136903506923, 71.6353925082578, 71.36833227914822, 71.17857895846511, 70.94665823318574, 70.7358212102045, 70.48984468339307, 70.19467285121934, 69.97680792747207, 69.78705460678896, 69.62541288917001, 69.33726895776232, 69.09129243095087, 68.8734275072036, 68.65556258345632, 68.45175346124113, 68.14252582753532, 67.96682830838428, 67.69976807927472, 67.49595895705953, 67.29917773561037, 67.10239651416123, 66.8775036896479, 66.73694567432707, 66.5190807505798, 66.39257853679106, 66.1957973153419, 66.0903788038513, 65.91468128470026, 65.73195586478319, 65.63356525405861, 65.47192353643968, 65.38056082648113, 65.29622601728863, 65.18377960503197, 65.09944479583949, 65.02213788741304, 64.92374727668846, 64.8113008644318, 64.76913345983554, 64.67074284911098, 64.58640803991848, 64.53018483379014, 64.46693372689577, 64.4036826200014, 64.33340361234099, 64.29123620774475, 64.24204090238246, 64.15770609318997, 64.12256658935976, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 71.02132517186067, "threshold_area100": 71.02178649237474, "threshold_area25": 75.73055028463}, "test": {"score": [66.95119530385956, 67.4474661826844, 67.81045288262484, 68.03448373649434, 68.04015540368091, 68.06284207242719, 68.07985707398689, 68.0770212403936, 68.04015540368091, 68.02314040212121, 68.01179706774806, 67.95508039588238, 67.8728412216771, 67.80761704903156, 67.71403454045316, 67.66015370218076, 67.5779145279755, 67.41627201315826, 67.21209199444176, 67.04477781243796, 66.905821966367, 66.71014944843037, 66.52298443127357, 66.38402858520261, 66.21387856960553, 66.00969855088903, 65.78566769701953, 65.60417434704931, 65.43402433145222, 65.24118764710887, 65.00297762527295, 64.83566344326914, 64.52939341519439, 64.37625840115702, 64.12386921135467, 63.94521169497773, 63.69282250517539, 63.47162748489919, 63.23341746306327, 63.049088279499756, 62.782519921730994, 62.515951563962226, 62.13878569605535, 61.83535150157389, 61.55743980943198, 61.30505061962964, 61.09519893372657, 60.86266057907722, 60.59609222130845, 60.292658026826984, 60.04594050421122, 59.77086464566259, 59.4901171199274, 59.266086266057904, 59.01369707625557, 58.80384539035249, 58.469217026344886, 58.11473782718431, 57.81697529988941, 57.55891444290049, 57.29801775231829, 56.98891189065025, 56.78756770552704, 56.438760173553014, 56.16652014859768, 55.91696679238862, 55.68726427133256, 55.38383007685109, 55.103082551115904, 54.910245866772534, 54.66069251056348, 54.35725831608201, 54.1190482942461, 53.934719110682586, 53.70501658962652, 53.472478234977174, 53.27680571704052, 53.112327368630005, 52.95352068740606, 52.76635567024927, 52.576354819499194, 52.414712304681956, 52.24172645549159, 52.06590477270794, 51.901426424297426, 51.73411224229362, 51.55829055950997, 51.388140543912876, 51.226498029095644, 51.09604968380456, 50.985452173666445, 50.86067549556192, 50.75574965261038, 50.628137140912564, 50.47783796046848, 50.35022544877066, 50.2878371097184, 50.211269602699716, 50.11768709412132, 50.04395542069591, 50.00708958398321], "calls": [0.0, 4.29628789382639, 6.652865609846015, 8.555709950940079, 9.792133397612227, 10.742137651362619, 11.624081898874174, 12.766922836967925, 13.923942943028104, 14.899469699118056, 16.05648980517823, 17.205002410458555, 18.31948501261946, 19.513370955392336, 20.832033576269744, 21.932337010464227, 23.04114794543856, 24.12727221166662, 25.24459064742081, 26.44131242378697, 27.470720018149336, 28.664605960922213, 29.801775231829396, 30.85670532853132, 31.81238124946828, 33.094178033632986, 34.19448146782747, 35.31179990358166, 36.40643167058957, 37.54927260868332, 38.80838272410175, 39.85764115361711, 41.22451294558036, 42.11212886027848, 43.237954796812524, 44.236168221648754, 45.44706916598134, 46.47080509315714, 47.61364603125088, 48.62603862405354, 49.890820406658534, 51.11873635255083, 52.5933698210589, 53.73904659274594, 54.90173836599268, 56.03890763689987, 56.901001049258426, 57.95025947877378, 59.03638374500184, 60.09982134248363, 60.9760939228086, 61.886396506253014, 63.11714828573859, 63.99058503247029, 64.90939511669455, 65.74596602671356, 66.75552278592292, 67.82746788418456, 68.86538297932677, 69.81822306667044, 70.7370331508947, 71.82032158352948, 72.60017582168278, 73.84227093554149, 74.72421518305306, 75.57780109463177, 76.30944616169924, 77.16019623968465, 77.97975214814394, 78.75109888551741, 79.49125145336473, 80.33916569775685, 81.09633326716389, 81.60394748036185, 82.29021920993677, 82.95664010435868, 83.60604599722087, 84.13067521197856, 84.64963275954966, 85.20545614383349, 85.76127952811729, 86.25755040694212, 86.75382128576695, 87.32098800442391, 87.75487054419646, 88.30502226129371, 88.89203981510366, 89.5159232056263, 90.13413492896237, 90.71831664917903, 91.23160252956356, 91.9320534271049, 92.47936931060885, 93.05787936363895, 93.70444942290787, 94.35952698295664, 95.04579871253155, 95.89938462411027, 96.86073221223378, 98.30417151121573, 100.0], "rescued": [0, 189, 350, 468, 531, 587, 644, 707, 761, 818, 892, 954, 1020, 1084, 1147, 1200, 1266, 1303, 1346, 1390, 1433, 1474, 1508, 1543, 1571, 1612, 1640, 1675, 1708, 1733, 1762, 1793, 1826, 1850, 1871, 1892, 1916, 1938, 1959, 1987, 2009, 2038, 2064, 2083, 2101, 2119, 2137, 2164, 2186, 2200, 2217, 2231, 2258, 2273, 2294, 2314, 2332, 2349, 2366, 2380, 2387, 2402, 2416, 2434, 2449, 2465, 2476, 2489, 2503, 2514, 2524, 2531, 2538, 2546, 2550, 2555, 2563, 2567, 2571, 2572, 2573, 2578, 2581, 2585, 2585, 2586, 2588, 2592, 2593, 2597, 2605, 2607, 2612, 2614, 2615, 2619, 2623, 2625, 2627, 2627, 2627], "harmed": [0, 14, 47, 86, 147, 195, 246, 310, 377, 440, 518, 600, 695, 782, 878, 950, 1045, 1139, 1254, 1357, 1449, 1559, 1659, 1743, 1831, 1944, 2051, 2150, 2243, 2336, 2449, 2539, 2680, 2758, 2868, 2952, 3065, 3165, 3270, 3363, 3479, 3602, 3761, 3887, 4003, 4110, 4202, 4311, 4427, 4548, 4652, 4763, 4889, 4983, 5093, 5187, 5323, 5465, 5587, 5692, 5791, 5915, 6000, 6141, 6252, 6356, 6448, 6568, 6681, 6760, 6858, 6972, 7063, 7136, 7221, 7308, 7385, 7447, 7507, 7574, 7642, 7704, 7768, 7834, 7892, 7952, 8016, 8080, 8138, 8188, 8235, 8281, 8323, 8370, 8424, 8473, 8499, 8528, 8563, 8589, 8602], "exact_score": [66.95119530385956, 67.01074780931854, 67.09015114993052, 67.23194282959476, 67.39925701159855, 67.56657119360236, 67.73104954201288, 67.86716955449053, 67.99761789978163, 68.02881206930778, 68.07985707398689, 68.06851373961376, 68.06567790602047, 68.08269290758018, 68.0429912372742, 68.03164790290106, 68.00896123415477, 67.9692595638488, 67.8926920568301, 67.82179621699798, 67.7991095482517, 67.69985537248674, 67.64597453421433, 67.56940702719564, 67.43045118112468, 67.23761449678133, 67.09015114993052, 67.0022403085387, 66.83776196012818, 66.67895527890423, 66.49179026174744, 66.35850608286306, 66.17134106570626, 66.0352210532286, 65.82820520091882, 65.64671185094858, 65.48223350253807, 65.33193432209397, 65.1617843064969, 64.97745512293338, 64.79596177296317, 64.58327425346681, 64.39043756912343, 64.1749142160338, 63.98207753169044, 63.77506167938065, 63.55386665910444, 63.34401497320137, 63.20222329353713, 62.95550577092136, 62.77401242095113, 62.56416073504807, 62.29192071009273, 62.03953152029039, 61.77863482970819, 61.53191730709242, 61.313558120409496, 61.078183932166866, 60.85698891189065, 60.6045997220883, 60.326688029946396, 60.04310467061793, 59.7311629753566, 59.51280378867367, 59.26892209965119, 58.98250290672943, 58.727277883333805, 58.38130618495306, 58.06936448969174, 57.78294529676998, 57.527720273374364, 57.212942744519744, 56.94070271956442, 56.68547769616878, 56.37920766809403, 56.08995264157899, 55.809205115843795, 55.44621841590336, 55.097410883929335, 54.82800669256728, 54.4877066613731, 54.178600799705066, 53.82412160054449, 53.45829906701074, 53.16053653971584, 52.842923177267956, 52.49411564529394, 52.15948728128633, 51.83336641805858, 51.527096389983825, 51.26619969940164, 51.02798967756572, 50.85216799478206, 50.64231630887899, 50.443807957349065, 50.29350877690497, 50.20559793551313, 50.11485126052803, 50.06380625584891, 50.024104585542915, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.20253611582617, "threshold_area100": 59.70732779400504, "threshold_area25": 67.42421234721947}, "selected_budget": 21, "selected": {"val": {"score": 76.44950453299599, "calls": 20.999367488931057, "rescued": 663, "harmed": 461}, "test": {"score": 66.71014944843037, "calls": 28.664605960922213, "rescued": 1474, "harmed": 1559}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.10614180564880371, "operator": "ge"}, {"budget": 0.02, "threshold": 0.026966333389282227, "operator": "ge"}, {"budget": 0.03, "threshold": 0.0035791397094726562, "operator": "ge"}, {"budget": 0.04, "threshold": -0.04653942584991455, "operator": "ge"}, {"budget": 0.05, "threshold": -0.13634860515594482, "operator": "ge"}, {"budget": 0.06, "threshold": -0.23222970962524414, "operator": "ge"}, {"budget": 0.07, "threshold": -0.33145076036453247, "operator": "ge"}, {"budget": 0.08, "threshold": -0.4615098237991333, "operator": "ge"}, {"budget": 0.09, "threshold": -0.6107139587402344, "operator": "ge"}, {"budget": 0.1, "threshold": -0.7852044105529785, "operator": "ge"}, {"budget": 0.11, "threshold": -0.9284204244613647, "operator": "ge"}, {"budget": 0.12, "threshold": -1.0605698823928833, "operator": "ge"}, {"budget": 0.13, "threshold": -1.213750958442688, "operator": "ge"}, {"budget": 0.14, "threshold": -1.3669084310531616, "operator": "ge"}, {"budget": 0.15, "threshold": -1.5106532573699951, "operator": "ge"}, {"budget": 0.16, "threshold": -1.6362152099609375, "operator": "ge"}, {"budget": 0.17, "threshold": -1.7914605140686035, "operator": "ge"}, {"budget": 0.18, "threshold": -1.952275037765503, "operator": "ge"}, {"budget": 0.19, "threshold": -2.073723793029785, "operator": "ge"}, {"budget": 0.2, "threshold": -2.213550567626953, "operator": "ge"}, {"budget": 0.21, "threshold": -2.3248345851898193, "operator": "ge"}, {"budget": 0.22, "threshold": -2.430243968963623, "operator": "ge"}, {"budget": 0.23, "threshold": -2.56710147857666, "operator": "gt"}, {"budget": 0.24, "threshold": -2.6854686737060547, "operator": "ge"}, {"budget": 0.25, "threshold": -2.812276840209961, "operator": "ge"}, {"budget": 0.26, "threshold": -2.931393623352051, "operator": "ge"}, {"budget": 0.27, "threshold": -3.051278591156006, "operator": "ge"}, {"budget": 0.28, "threshold": -3.1585049629211426, "operator": "gt"}, {"budget": 0.29, "threshold": -3.2541279792785645, "operator": "ge"}, {"budget": 0.3, "threshold": -3.3637137413024902, "operator": "ge"}, {"budget": 0.31, "threshold": -3.4591987133026123, "operator": "ge"}, {"budget": 0.32, "threshold": -3.5602152347564697, "operator": "ge"}, {"budget": 0.33, "threshold": -3.650628089904785, "operator": "ge"}, {"budget": 0.34, "threshold": -3.7416257858276367, "operator": "ge"}, {"budget": 0.35, "threshold": -3.838078498840332, "operator": "ge"}, {"budget": 0.36, "threshold": -3.9285292625427246, "operator": "ge"}, {"budget": 0.37, "threshold": -4.013824939727783, "operator": "ge"}, {"budget": 0.38, "threshold": -4.108944892883301, "operator": "ge"}, {"budget": 0.39, "threshold": -4.1786417961120605, "operator": "ge"}, {"budget": 0.4, "threshold": -4.2613630294799805, "operator": "ge"}, {"budget": 0.41, "threshold": -4.342169284820557, "operator": "ge"}, {"budget": 0.42, "threshold": -4.4318037033081055, "operator": "ge"}, {"budget": 0.43, "threshold": -4.515063762664795, "operator": "ge"}, {"budget": 0.44, "threshold": -4.581536293029785, "operator": "ge"}, {"budget": 0.45, "threshold": -4.65653657913208, "operator": "ge"}, {"budget": 0.46, "threshold": -4.7424492835998535, "operator": "ge"}, {"budget": 0.47, "threshold": -4.819340229034424, "operator": "ge"}, {"budget": 0.48, "threshold": -4.901113510131836, "operator": "ge"}, {"budget": 0.49, "threshold": -4.976139068603516, "operator": "ge"}, {"budget": 0.5, "threshold": -5.044043064117432, "operator": "ge"}, {"budget": 0.51, "threshold": -5.115956783294678, "operator": "ge"}, {"budget": 0.52, "threshold": -5.185346603393555, "operator": "ge"}, {"budget": 0.53, "threshold": -5.250713348388672, "operator": "ge"}, {"budget": 0.54, "threshold": -5.324597358703613, "operator": "ge"}, {"budget": 0.55, "threshold": -5.405856609344482, "operator": "ge"}, {"budget": 0.56, "threshold": -5.480432510375977, "operator": "ge"}, {"budget": 0.57, "threshold": -5.557149887084961, "operator": "ge"}, {"budget": 0.58, "threshold": -5.646034240722656, "operator": "ge"}, {"budget": 0.59, "threshold": -5.716063499450684, "operator": "ge"}, {"budget": 0.6, "threshold": -5.78645133972168, "operator": "gt"}, {"budget": 0.61, "threshold": -5.8630266189575195, "operator": "ge"}, {"budget": 0.62, "threshold": -5.946810722351074, "operator": "ge"}, {"budget": 0.63, "threshold": -6.020058631896973, "operator": "ge"}, {"budget": 0.64, "threshold": -6.0910515785217285, "operator": "ge"}, {"budget": 0.65, "threshold": -6.169153690338135, "operator": "ge"}, {"budget": 0.66, "threshold": -6.230364799499512, "operator": "ge"}, {"budget": 0.67, "threshold": -6.305612087249756, "operator": "ge"}, {"budget": 0.68, "threshold": -6.370851516723633, "operator": "ge"}, {"budget": 0.69, "threshold": -6.431934356689453, "operator": "ge"}, {"budget": 0.7, "threshold": -6.500517845153809, "operator": "ge"}, {"budget": 0.71, "threshold": -6.582881450653076, "operator": "ge"}, {"budget": 0.72, "threshold": -6.651588439941406, "operator": "ge"}, {"budget": 0.73, "threshold": -6.7247114181518555, "operator": "ge"}, {"budget": 0.74, "threshold": -6.791232109069824, "operator": "ge"}, {"budget": 0.75, "threshold": -6.86007022857666, "operator": "ge"}, {"budget": 0.76, "threshold": -6.930532932281494, "operator": "ge"}, {"budget": 0.77, "threshold": -7.0109663009643555, "operator": "ge"}, {"budget": 0.78, "threshold": -7.075525283813477, "operator": "ge"}, {"budget": 0.79, "threshold": -7.143087387084961, "operator": "ge"}, {"budget": 0.8, "threshold": -7.209010601043701, "operator": "ge"}, {"budget": 0.81, "threshold": -7.278168678283691, "operator": "ge"}, {"budget": 0.82, "threshold": -7.3419189453125, "operator": "ge"}, {"budget": 0.83, "threshold": -7.412956714630127, "operator": "ge"}, {"budget": 0.84, "threshold": -7.488970756530762, "operator": "ge"}, {"budget": 0.85, "threshold": -7.5564727783203125, "operator": "ge"}, {"budget": 0.86, "threshold": -7.625124931335449, "operator": "ge"}, {"budget": 0.87, "threshold": -7.693497657775879, "operator": "ge"}, {"budget": 0.88, "threshold": -7.760097026824951, "operator": "ge"}, {"budget": 0.89, "threshold": -7.839372634887695, "operator": "ge"}, {"budget": 0.9, "threshold": -7.908748626708984, "operator": "ge"}, {"budget": 0.91, "threshold": -7.984408378601074, "operator": "ge"}, {"budget": 0.92, "threshold": -8.0657958984375, "operator": "ge"}, {"budget": 0.93, "threshold": -8.147634506225586, "operator": "ge"}, {"budget": 0.94, "threshold": -8.242995262145996, "operator": "ge"}, {"budget": 0.95, "threshold": -8.35625171661377, "operator": "ge"}, {"budget": 0.96, "threshold": -8.489726066589355, "operator": "ge"}, {"budget": 0.97, "threshold": -8.652472496032715, "operator": "ge"}, {"budget": 0.98, "threshold": -8.864336967468262, "operator": "ge"}, {"budget": 0.99, "threshold": -9.136297225952148, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.2899009065992, 75.4093752196219, 75.56398903647481, 75.59210063953897, 75.40234731885586, 75.17745449434254, 75.09311968515004, 75.1352870897463, 75.2196218989388, 75.26178930353502, 75.28287300583317, 75.33909621196149, 75.41640312038794, 75.54290533417668, 75.56398903647481, 75.63426804413523, 75.66940754796543, 75.66237964719939, 75.75374235715792, 75.84510506711646, 75.83104926558437, 75.74671445639187, 75.80293766252021, 75.80293766252021, 75.83807716635042, 75.78888186098813, 75.70454705179563, 75.57804483800689, 75.52182163187857, 75.43045892192002, 75.30395670813128, 75.2196218989388, 75.05798018131983, 74.91742216599901, 74.67144563918757, 74.5379155246328, 74.31302270011948, 74.08110197484011, 73.88432075339097, 73.76484644036826, 73.52589781432286, 73.27992128751141, 73.04800056223206, 72.80202403542062, 72.57010331014126, 72.4225173940544, 72.24681987490337, 71.95164804272964, 71.67755991285402, 71.47375079063883, 71.23480216459343, 70.99585353854803, 70.70770960714034, 70.50390048492515, 70.34928666807225, 70.00491953053623, 69.78002670602291, 69.51999437767938, 69.30212945393211, 69.03506922482254, 68.78206479724507, 68.50797666736945, 68.30416754515426, 68.05819101834281, 67.84032609459555, 67.57326586548598, 67.40459624710098, 67.26403823178016, 67.02508960573476, 66.80722468198749, 66.65963876590062, 66.4839412467496, 66.30824372759857, 66.1606578115117, 66.04821139925504, 65.86548597933796, 65.72492796401714, 65.58436994869632, 65.47192353643968, 65.41570033031134, 65.34542132265092, 65.26108651345844, 65.21189120809613, 65.15566800196781, 65.1205284981376, 65.0502494904772, 64.91671937592241, 64.8815798720922, 64.8113008644318, 64.76210555906951, 64.63560334528077, 64.55126853608827, 64.45990582612974, 64.44585002459765, 64.41773842153349, 64.4036826200014, 64.34745941387308, 64.2490688031485, 64.18581769625413, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.98826340572071, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.992128751142033, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.99016093892754, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 42, 64, 88, 104, 114, 124, 139, 168, 194, 222, 255, 286, 323, 363, 396, 435, 474, 508, 550, 593, 627, 659, 697, 726, 764, 794, 814, 840, 866, 888, 913, 936, 955, 974, 986, 1005, 1022, 1038, 1057, 1073, 1086, 1097, 1109, 1118, 1132, 1147, 1158, 1163, 1173, 1186, 1193, 1206, 1218, 1226, 1239, 1245, 1251, 1259, 1265, 1266, 1270, 1275, 1281, 1286, 1289, 1291, 1294, 1301, 1305, 1312, 1315, 1318, 1322, 1326, 1330, 1330, 1332, 1333, 1336, 1340, 1345, 1347, 1350, 1350, 1350, 1353, 1354, 1355, 1355, 1355, 1355, 1356, 1357, 1360, 1361, 1364, 1365, 1365, 1367, 1369], "harmed": [0, 5, 10, 12, 24, 61, 103, 130, 153, 167, 189, 219, 242, 268, 290, 320, 349, 383, 418, 447, 477, 513, 557, 587, 616, 649, 686, 718, 762, 796, 831, 874, 909, 951, 990, 1037, 1075, 1124, 1173, 1220, 1253, 1300, 1346, 1391, 1435, 1482, 1518, 1554, 1601, 1650, 1692, 1733, 1780, 1833, 1870, 1905, 1960, 1998, 2043, 2080, 2119, 2159, 2203, 2238, 2278, 2312, 2352, 2379, 2406, 2444, 2482, 2506, 2534, 2563, 2588, 2608, 2634, 2656, 2677, 2696, 2708, 2723, 2737, 2747, 2755, 2760, 2773, 2793, 2799, 2809, 2816, 2834, 2847, 2861, 2866, 2871, 2876, 2885, 2899, 2910, 2921], "exact_score": [75.02986857825567, 75.2899009065992, 75.4093752196219, 75.56398903647481, 75.59210063953897, 75.40234731885586, 75.17745449434254, 75.09311968515004, 75.1352870897463, 75.2196218989388, 75.26178930353502, 75.28287300583317, 75.33909621196149, 75.41640312038794, 75.54290533417668, 75.56398903647481, 75.63426804413523, 75.66940754796543, 75.66237964719939, 75.75374235715792, 75.84510506711646, 75.83104926558437, 75.74671445639187, 75.80293766252021, 75.80293766252021, 75.83807716635042, 75.78888186098813, 75.70454705179563, 75.57804483800689, 75.52182163187857, 75.43045892192002, 75.30395670813128, 75.2196218989388, 75.05798018131983, 74.91742216599901, 74.67144563918757, 74.5379155246328, 74.31302270011948, 74.08110197484011, 73.88432075339097, 73.76484644036826, 73.52589781432286, 73.27992128751141, 73.04800056223206, 72.80202403542062, 72.57010331014126, 72.4225173940544, 72.24681987490337, 71.95164804272964, 71.67755991285402, 71.47375079063883, 71.23480216459343, 70.99585353854803, 70.70770960714034, 70.50390048492515, 70.34928666807225, 70.00491953053623, 69.78002670602291, 69.51999437767938, 69.30212945393211, 69.03506922482254, 68.78206479724507, 68.50797666736945, 68.30416754515426, 68.05819101834281, 67.84032609459555, 67.57326586548598, 67.40459624710098, 67.26403823178016, 67.02508960573476, 66.80722468198749, 66.65963876590062, 66.4839412467496, 66.30824372759857, 66.1606578115117, 66.04821139925504, 65.86548597933796, 65.72492796401714, 65.58436994869632, 65.47192353643968, 65.41570033031134, 65.34542132265092, 65.26108651345844, 65.21189120809613, 65.15566800196781, 65.1205284981376, 65.0502494904772, 64.91671937592241, 64.8815798720922, 64.8113008644318, 64.76210555906951, 64.63560334528077, 64.55126853608827, 64.45990582612974, 64.44585002459765, 64.41773842153349, 64.4036826200014, 64.34745941387308, 64.2490688031485, 64.18581769625413, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 70.75125009235572, "threshold_area100": 70.7516339869281, "threshold_area25": 75.50144071965704}, "test": {"score": [66.95119530385956, 67.45313784987097, 67.78493038028527, 68.01463290134134, 68.07418540680032, 68.03164790290106, 67.98060289822193, 67.99194623259507, 67.94373706150922, 67.92105039276295, 67.8926920568301, 67.85015455293083, 67.79627371465841, 67.6516462014009, 67.53537702407623, 67.36522700847914, 67.2773161670873, 67.11850948586337, 66.91716530074015, 66.74134361795649, 66.557014434393, 66.38970025238919, 66.24507273913166, 66.06357938916145, 65.91328020871735, 65.70626435640756, 65.46805433457165, 65.20715764398945, 65.02282846042594, 64.86969344638857, 64.64282675892579, 64.40745257068315, 64.13237671213453, 63.905510024671756, 63.647449167682836, 63.39789581147378, 63.0972974505856, 62.80804242407056, 62.52729489833536, 62.2975923772793, 62.01684485154411, 61.730425658622345, 61.4043047953946, 61.14057227121912, 60.854153078297365, 60.5308680486629, 60.230269687774715, 59.960865496412666, 59.67161046989763, 59.35399710744973, 59.07608541530782, 58.78115872160621, 58.557127867736725, 58.338768681053786, 58.07220032328503, 57.825482800669256, 57.564586110087056, 57.25831608201231, 56.9236877180047, 56.711000198508344, 56.498312679011995, 56.25159515639622, 55.95383262910132, 55.695771772112415, 55.42920341434364, 55.19950089328758, 55.01517170972407, 54.774125854294866, 54.57561750276494, 54.38561665201485, 54.20695913563792, 54.005614950514705, 53.85531577007061, 53.69650908884667, 53.54337407480929, 53.35904489124578, 53.20307404361512, 53.00740152567847, 52.85993817882766, 52.73516150072314, 52.58202648668576, 52.4629214757678, 52.3523239656297, 52.244562289084875, 52.10560644301392, 51.97515809772282, 51.867396421178, 51.7369480758869, 51.62351473215552, 51.52993222357711, 51.439185548592, 51.33142387204719, 51.24067719706207, 51.14142302129711, 51.042168845532146, 50.9372430025806, 50.795451322916364, 50.63380880809913, 50.452315458128915, 50.30485211127811, 50.00708958398321], "calls": [0.0, 1.9482176785866205, 4.318974562572668, 8.022573235402547, 10.356464282675892, 11.397215211411394, 12.310353628449082, 13.302895386098742, 14.357825482800667, 15.441113915435443, 16.70305986444715, 17.862915804100616, 19.022771743754078, 20.23083685449338, 21.55233530896407, 22.69234041346454, 23.665031335961206, 24.89861894904007, 26.186087400391344, 27.170121657261152, 28.361171766440744, 29.279981850665003, 30.051328588038455, 31.077900348807532, 32.07611377364376, 33.2586563820435, 34.452542324816385, 35.56702492697729, 36.661656693985194, 37.63151178288858, 38.93315940220628, 40.08734367467317, 41.41734962992372, 42.4609363922525, 43.7257181748575, 44.91960411763038, 46.18722173382866, 47.43782434846723, 48.83021864277004, 49.780222896520435, 51.02798967756572, 52.21053228596546, 53.605762413861555, 54.82800669256728, 55.931145960355046, 57.11936023594135, 58.27354450840825, 59.54399795819981, 60.71236139863313, 61.860874003913445, 62.83640076000341, 63.95088336216431, 64.88387261435498, 65.63820435016872, 66.66477611093781, 67.69701953889346, 68.68388962935656, 69.70762555653234, 70.76539148682755, 71.536738224201, 72.29674162720131, 73.1559992059666, 74.08048095737743, 75.01630604316138, 75.86138445396024, 76.74049286787852, 77.37004792558773, 78.09602132546863, 78.82199472534953, 79.40617644556617, 80.0499106712418, 80.84394407736154, 81.40827496242521, 82.02081501857471, 82.63051924113093, 83.22320846212745, 83.7336585089187, 84.5673935853444, 85.10053030088194, 85.70739868984488, 86.24620707256898, 86.86725462949833, 87.39471967784931, 87.93919972775997, 88.58009811984233, 89.10189150100672, 89.6804015540368, 90.16816493208178, 90.73249581714545, 91.36772254204124, 91.88951592320562, 92.45668264186257, 93.09474520035165, 93.67325525338173, 94.33400448061707, 95.07415704846439, 95.93057879363639, 96.82953804270765, 97.83342313473045, 98.74088988458158, 100.0], "rescued": [0, 219, 362, 472, 548, 600, 635, 690, 739, 801, 865, 929, 988, 1038, 1103, 1153, 1213, 1274, 1322, 1351, 1405, 1446, 1478, 1522, 1566, 1613, 1658, 1683, 1717, 1752, 1792, 1827, 1858, 1883, 1915, 1942, 1963, 1989, 2021, 2042, 2064, 2089, 2115, 2141, 2161, 2180, 2198, 2225, 2244, 2264, 2280, 2303, 2312, 2326, 2341, 2356, 2371, 2385, 2396, 2409, 2423, 2436, 2444, 2459, 2461, 2468, 2474, 2482, 2494, 2501, 2510, 2519, 2524, 2529, 2536, 2539, 2540, 2549, 2552, 2557, 2566, 2572, 2575, 2578, 2582, 2586, 2592, 2594, 2597, 2600, 2600, 2602, 2602, 2604, 2606, 2611, 2615, 2616, 2618, 2621, 2627], "harmed": [0, 42, 68, 97, 152, 219, 272, 323, 389, 459, 533, 612, 690, 791, 897, 1007, 1098, 1215, 1334, 1425, 1544, 1644, 1727, 1835, 1932, 2052, 2181, 2298, 2397, 2486, 2606, 2724, 2852, 2957, 3080, 3195, 3322, 3450, 3581, 3683, 3804, 3930, 4071, 4190, 4311, 4444, 4568, 4690, 4811, 4943, 5057, 5184, 5272, 5363, 5472, 5574, 5681, 5803, 5932, 6020, 6109, 6209, 6322, 6428, 6524, 6612, 6683, 6776, 6858, 6932, 7004, 7084, 7142, 7203, 7264, 7332, 7388, 7466, 7521, 7570, 7633, 7681, 7723, 7764, 7817, 7867, 7911, 7959, 8002, 8038, 8070, 8110, 8142, 8179, 8216, 8258, 8312, 8370, 8436, 8491, 8602], "exact_score": [66.95119530385956, 67.21776366162834, 67.45880951705755, 67.6346311998412, 67.75657204435244, 67.8558262201174, 67.92672205994951, 67.97493123103536, 68.01179706774806, 68.03164790290106, 68.07134957320704, 68.02314040212121, 67.99761789978163, 67.98343873181521, 67.94090122791594, 67.92388622635623, 67.90687122479653, 67.87851288886368, 67.87000538808383, 67.79060204747185, 67.68000453733374, 67.59492952953521, 67.44463034909111, 67.33970450613957, 67.21209199444176, 67.10716615149023, 66.93134446870658, 66.76970195388935, 66.60805943907211, 66.41238692113546, 66.25641607350481, 66.07775855712786, 65.91611604231063, 65.74313019312027, 65.56447267674332, 65.37730765958653, 65.1220826361909, 64.97178345574682, 64.78745427218331, 64.63148342455266, 64.42163173864958, 64.19476505118679, 64.01327170121657, 63.82894251765306, 63.599239996597, 63.3950599778805, 63.148342455264725, 62.90729659983552, 62.68042991237274, 62.464906559283094, 62.2408757054136, 62.022516518730676, 61.80132149845446, 61.546096475058846, 61.32490145478263, 61.086691432946715, 60.834302243144364, 60.56206221818904, 60.26997135808071, 60.06295550577092, 59.858775487054416, 59.589371295692374, 59.31145960355047, 59.03638374500184, 58.772651220826354, 58.525933698210586, 58.231007004508974, 57.98145364829992, 57.77727362958341, 57.456824433542245, 57.164733573433914, 56.8698068797323, 56.58338768681054, 56.294132660295496, 55.98786263222074, 55.704279272892265, 55.398009244817516, 55.12009755267561, 54.808155857414285, 54.52173666449253, 54.21830247001106, 53.97158494739529, 53.69934492243995, 53.41292572951819, 53.117999035816574, 52.87978901398066, 52.64725065933131, 52.440234807021525, 52.2218756203386, 51.99500893287582, 51.779485579786176, 51.589484729036094, 51.416498879845726, 51.252020531435214, 51.10455718458441, 50.95425800414032, 50.77560048776337, 50.61112213935286, 50.43530045656921, 50.2708221081587, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 61.05450420295702, "threshold_area100": 59.40179508266454, "threshold_area25": 67.23506224654737}, "selected_budget": 20, "selected": {"val": {"score": 75.84510506711646, "calls": 19.994377679387167, "rescued": 593, "harmed": 477}, "test": {"score": 66.557014434393, "calls": 28.361171766440744, "rescued": 1405, "harmed": 1544}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 75.6975191510296, "calls": 5.994799353433129}, "test": {"score": 68.17627541615859, "calls": 12.883192014292602}}, "old_normalized_interior": {"val": {"score": 75.70454705179563, "calls": 4.996837444655281}, "test": {"score": 68.15926041459886, "calls": 12.103337776139297}}, "old_normalized_endpoints": {"val": {"score": 75.70454705179563, "calls": 4.996837444655281}, "test": {"score": 68.15926041459886, "calls": 12.103337776139297}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.26178930353502, 75.40234731885586, 75.50073792958042, 75.65535174643334, 75.70454705179563, 75.70454705179563, 75.6834633494975, 75.66940754796543, 75.67643544873147, 75.64832384566729, 75.59210063953897, 75.53587743341063, 75.51479373111252, 75.50073792958042, 75.48668212804836, 75.4937100288144, 75.47965422728231, 75.47262632651626, 75.4585705249842, 75.43045892192002, 75.45154262421815, 75.47965422728231, 75.48668212804836, 75.4445147234521, 75.38829151732378, 75.38126361655773, 75.36720781502565, 75.31098460889733, 75.26178930353502, 75.23367770047086, 75.23367770047086, 75.16339869281046, 75.1352870897463, 75.1352870897463, 75.10014758591609, 75.07203598285193, 75.03689647902172, 75.03689647902172, 74.98067327289338, 74.92445006676506, 74.91039426523297, 74.87525476140277, 74.76983624991215, 74.63630613535737, 74.5379155246328, 74.44655281467426, 74.39032960854593, 74.39735750931197, 74.35519010471572, 74.27085529552323, 74.20760418862886, 74.2146320893949, 74.19354838709677, 74.10921357790427, 74.10921357790427, 74.05299037177595, 73.99676716564763, 73.92648815798721, 73.74376273807015, 73.63131632581347, 73.53292571508891, 73.44156300513038, 73.32208869210767, 73.23072598214914, 73.11827956989248, 73.09719586759435, 72.92852624920936, 72.83716353925082, 72.67552182163189, 72.46468479865065, 72.3381825848619, 72.24681987490337, 72.06409445498629, 71.88839693583526, 71.77595052357862, 71.6353925082578, 71.35427647761614, 71.17155105769906, 70.99585353854803, 70.77096071403471, 70.55309579028744, 70.34928666807225, 70.05411483589852, 69.80813830908707, 69.64649659146814, 69.31618525546419, 68.99992972099234, 68.64853468269028, 68.21983273596177, 67.78410288846722, 67.46081945322932, 67.12348021645934, 66.82830838428562, 66.53313655211188, 66.13957410921358, 65.7670953686134, 65.47895143720571, 65.05727739124325, 64.71993815447325, 64.12256658935976], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.98657670953686, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.26178930353502, 75.40234731885586, 75.50073792958042, 75.65535174643334, 75.70454705179563, 75.70454705179563, 75.6834633494975, 75.66940754796543, 75.67643544873147, 75.64832384566729, 75.59210063953897, 75.53587743341063, 75.51479373111252, 75.50073792958042, 75.48668212804836, 75.4937100288144, 75.47965422728231, 75.47262632651626, 75.4585705249842, 75.43045892192002, 75.45154262421815, 75.47965422728231, 75.48668212804836, 75.4445147234521, 75.38829151732378, 75.38126361655773, 75.36720781502565, 75.31098460889733, 75.26178930353502, 75.23367770047086, 75.23367770047086, 75.16339869281046, 75.1352870897463, 75.1352870897463, 75.10014758591609, 75.07203598285193, 75.03689647902172, 75.03689647902172, 74.98067327289338, 74.92445006676506, 74.91039426523297, 74.87525476140277, 74.76983624991215, 74.63630613535737, 74.5379155246328, 74.44655281467426, 74.39032960854593, 74.39735750931197, 74.35519010471572, 74.27085529552323, 74.20760418862886, 74.2146320893949, 74.19354838709677, 74.10921357790427, 74.10921357790427, 74.05299037177595, 73.99676716564763, 73.92648815798721, 73.74376273807015, 73.63131632581347, 73.53292571508891, 73.44156300513038, 73.32208869210767, 73.23072598214914, 73.11827956989248, 73.09719586759435, 72.92852624920936, 72.83716353925082, 72.67552182163189, 72.46468479865065, 72.3381825848619, 72.24681987490337, 72.06409445498629, 71.88839693583526, 71.77595052357862, 71.6353925082578, 71.35427647761614, 71.17155105769906, 71.00288143931408, 70.77096071403471, 70.55309579028744, 70.34928666807225, 70.05411483589852, 69.80813830908707, 69.64649659146814, 69.31618525546419, 68.99992972099234, 68.64853468269028, 68.21983273596177, 67.78410288846722, 67.46081945322932, 67.12348021645934, 66.82830838428562, 66.53313655211188, 66.13957410921358, 65.7670953686134, 65.47895143720571, 65.05727739124325, 64.71993815447325, 64.12256658935976], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 75.4958183990442, "exact_area100": 73.06196375490943}, "test": {"score": [66.95119530385956, 67.48716785299038, 67.71970620763973, 67.9465728951025, 68.13090207866603, 68.15926041459886, 68.06284207242719, 68.0203045685279, 67.9862745654085, 67.96358789666222, 67.95224456228908, 67.9295578935428, 67.92105039276295, 67.93239372713609, 67.93806539432265, 67.90403539120324, 67.90687122479653, 67.88985622323682, 67.87851288886368, 67.86149788730397, 67.86433372089725, 67.83313955137112, 67.80194538184499, 67.80761704903156, 67.762243711539, 67.71687037404645, 67.71403454045316, 67.6686612029606, 67.6346311998412, 67.61194453109492, 67.59492952953521, 67.56940702719564, 67.50701868814339, 67.45030201627769, 67.40209284519185, 67.3510478405127, 67.3170178373933, 67.25179366474775, 67.19791282647533, 67.14119615460964, 67.13552448742307, 67.07030031477753, 66.99089697416557, 66.92567280152001, 66.84343362731475, 66.75268695232963, 66.67044777812437, 66.58253693673255, 66.47193942659445, 66.35850608286306, 66.2394010719451, 66.14865439695998, 66.03805688682188, 65.93029521027705, 65.79417519779939, 65.65521935172843, 65.48223350253807, 65.34327765646711, 65.13909763775061, 64.98596262371323, 64.78461843859003, 64.63431925814594, 64.40178090349659, 64.21461588633979, 63.99342086606358, 63.82327085046649, 63.655956668462686, 63.46595581771262, 63.27311913336925, 63.04625244590647, 62.759833252984706, 62.53580239911521, 62.24938320619346, 61.91191900859258, 61.67370898675666, 61.432663131327445, 61.16609477355869, 60.86266057907722, 60.60743555568159, 60.3465388650994, 60.02041800187165, 59.694297138643904, 59.35683294104302, 59.067577914527966, 58.659217877094974, 58.28488784278139, 57.88787113972151, 57.5390636077475, 57.198763576553326, 56.79040353912033, 56.48129767745229, 56.09846014235884, 55.64189093384, 55.00950004253751, 54.493378328559686, 54.02830161926098, 53.24844738110768, 52.52530981482006, 51.850381419618294, 51.03933301193886, 50.00708958398321], "calls": [0.0, 1.9680685137396137, 4.131809545415875, 6.349431415364546, 9.239145846921701, 12.103337776139297, 13.623344582139921, 14.828573859285937, 15.991265632532684, 17.162464906559283, 18.13515582905595, 18.9632192382951, 19.626804299123727, 20.31307602869864, 21.104273601225078, 21.77920199642685, 22.425772055695774, 23.055327113404985, 23.611150497688797, 24.16980971556589, 24.790857272495252, 25.224739812267817, 25.774891529365057, 26.32787908005558, 26.812806624507274, 27.303405836145533, 27.932960893854748, 28.483112610951988, 29.007741825709665, 29.60326688029946, 30.125060261463858, 30.74327198479993, 31.387006210475572, 32.00521793381164, 32.72551966650597, 33.52238890621898, 34.18313813345433, 34.94597737004793, 35.70598077304823, 36.519865014320956, 37.29971925247427, 38.02285681876187, 38.921816067833134, 39.80659614893798, 40.78495873862122, 41.79451549783059, 42.79272892266682, 43.7257181748575, 44.73811076766015, 45.6767716870374, 46.70901511499305, 47.54275019141877, 48.56932195218784, 49.59305787936364, 50.554405467487165, 51.459036383745, 52.505458979667075, 53.61710574823469, 54.6181550066642, 55.54547259166832, 56.43592433995973, 57.456824433542245, 58.460709525565036, 59.561012959759516, 60.649973059580866, 61.52057397271928, 62.62087740691376, 63.49998582083204, 64.49536341207498, 65.51626350565749, 66.85761279528117, 67.7991095482517, 68.89090548166634, 70.19538893457732, 71.05181067974932, 72.03300910302583, 73.09644670050761, 74.06063012222442, 75.06167938065394, 76.05989280549016, 76.99855372486742, 78.16975299889403, 79.21050392762953, 80.20588151887247, 81.30618495306695, 82.37245838414202, 83.39052264413124, 84.36888523381448, 85.350083657091, 86.32844624677423, 87.27277883333807, 88.288007259734, 89.35144485721578, 90.63324164138048, 91.83563508493322, 92.87638601366872, 94.38221365170291, 95.88804128973712, 97.12730057000255, 98.41476902135383, 100.0], "exact_score": [66.95119530385956, 67.28582366786718, 67.49000368658366, 67.63746703343448, 67.70269120608002, 67.86149788730397, 67.93522956072937, 68.0429912372742, 68.0997079091399, 68.12806624507274, 68.15926041459886, 68.17911124975187, 68.14508124663244, 68.12806624507274, 68.04582707086749, 68.0203045685279, 67.9862745654085, 67.96075206306894, 67.95791622947566, 67.9295578935428, 67.92672205994951, 67.94090122791594, 67.90119955760996, 67.90119955760996, 67.87000538808383, 67.85015455293083, 67.80194538184499, 67.75090037716586, 67.71119870685988, 67.6346311998412, 67.60627286390834, 67.5382128576695, 67.45313784987097, 67.38507784363213, 67.32836117176643, 67.24612199756118, 67.17239032413578, 67.14119615460964, 67.07313614837082, 66.97671780619913, 66.91149363355358, 66.81791112497518, 66.73000028358335, 66.66761194453109, 66.55134276720642, 66.43507358988174, 66.31880441255706, 66.19969940163911, 66.09760939228086, 65.98417604854947, 65.87641437200465, 65.69775685562772, 65.58715934548961, 65.39432266114625, 65.24969514788872, 65.0880526330715, 64.8810367807617, 64.69954343079148, 64.48402007770184, 64.2855117261719, 64.11252587698154, 63.93670419419788, 63.752375010634374, 63.58506082863058, 63.372373309134225, 63.171029124011, 62.96117743810793, 62.72013158267873, 62.51027989677566, 62.215353203074045, 61.97430734764484, 61.690723988316364, 61.44684229929388, 61.191617275898246, 60.87967558063693, 60.62728639083459, 60.352210532285966, 60.02041800187165, 59.756685477696166, 59.41071377931543, 59.138473754360085, 58.758472052859936, 58.426679522445625, 58.038170320165605, 57.65816861866545, 57.30368941950486, 56.92085188441142, 56.58055185321725, 56.21189348609023, 55.79219011428409, 55.29024756827269, 54.87621586365312, 54.41681082154099, 53.974420780988574, 53.4299407310779, 52.981879023338905, 52.46859314295437, 51.92694892663698, 51.30022970252106, 50.65365964325213, 50.00708958398321], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 67.8427813855883, "exact_area100": 63.6215324086382}}, "old_normalized_endpoint_budget": 5}]}, {"expert": "instancevg", "endpoint": "Qwen3.8", "task": "grefcoco", "small": 66.95119530385956, "large": 74.47182599325072, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 1.0, "validation_area100": 0.8152730952888659, "training_seconds": 102.29816406499594, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.45007216566359826, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3017627978167088, "operator": "ge"}, {"budget": 0.03, "threshold": 0.16981772707682596, "operator": "ge"}, {"budget": 0.04, "threshold": 0.0577853955501419, "operator": "ge"}, {"budget": 0.05, "threshold": -0.09263492549281954, "operator": "ge"}, {"budget": 0.06, "threshold": -0.25195611957382863, "operator": "ge"}, {"budget": 0.07, "threshold": -0.3661583230852021, "operator": "ge"}, {"budget": 0.08, "threshold": -0.45966486659997585, "operator": "ge"}, {"budget": 0.09, "threshold": -0.5311272620394902, "operator": "ge"}, {"budget": 0.1, "threshold": -0.601189456311128, "operator": "ge"}, {"budget": 0.11, "threshold": -0.6564992275594009, "operator": "ge"}, {"budget": 0.12, "threshold": -0.7069584844583502, "operator": "ge"}, {"budget": 0.13, "threshold": -0.7533632237407714, "operator": "ge"}, {"budget": 0.14, "threshold": -0.7892928007280389, "operator": "ge"}, {"budget": 0.15, "threshold": -0.8164880899941104, "operator": "ge"}, {"budget": 0.16, "threshold": -0.8397245366506846, "operator": "ge"}, {"budget": 0.17, "threshold": -0.8593700983530377, "operator": "ge"}, {"budget": 0.18, "threshold": -0.8754477933674516, "operator": "ge"}, {"budget": 0.19, "threshold": -0.8893420827555616, "operator": "ge"}, {"budget": 0.2, "threshold": -0.9015899290346095, "operator": "ge"}, {"budget": 0.21, "threshold": -0.9109076308790096, "operator": "ge"}, {"budget": 0.22, "threshold": -0.9197813062336394, "operator": "ge"}, {"budget": 0.23, "threshold": -0.9277945514770205, "operator": "ge"}, {"budget": 0.24, "threshold": -0.9339929439336867, "operator": "ge"}, {"budget": 0.25, "threshold": -0.9396264805217868, "operator": "ge"}, {"budget": 0.26, "threshold": -0.9451595731612781, "operator": "ge"}, {"budget": 0.27, "threshold": -0.9500315437877975, "operator": "ge"}, {"budget": 0.28, "threshold": -0.9545350574793711, "operator": "gt"}, {"budget": 0.29, "threshold": -0.9580362757805881, "operator": "ge"}, {"budget": 0.3, "threshold": -0.9611240669892509, "operator": "ge"}, {"budget": 0.31, "threshold": -0.9637702359971367, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9660456920214532, "operator": "ge"}, {"budget": 0.33, "threshold": -0.9680604550397871, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9704559170845773, "operator": "ge"}, {"budget": 0.35, "threshold": -0.9729278373904081, "operator": "ge"}, {"budget": 0.36, "threshold": -0.9747122382247724, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9763861912749262, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9779411208630033, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9796469382670336, "operator": "gt"}, {"budget": 0.4, "threshold": -0.9810440107454511, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9822015344708781, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9834576372347595, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9844380169951562, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9854324014097393, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9863835463765994, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9874203897542649, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9882556696330353, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9890743071566287, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9898907740125092, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9906814144100876, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9912987257476996, "operator": "ge"}, {"budget": 0.52, "threshold": -0.991924191498564, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9924546973267871, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9929621038282563, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9935044196195223, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9939407688239352, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9943500109287086, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9947942270504065, "operator": "ge"}, {"budget": 0.59, "threshold": -0.995085069966917, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9954709177166077, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9957599964814522, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9960714881106381, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9963909644400907, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9966045096438727, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9968322718245212, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9970209711682757, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9972233802495665, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9974059156340686, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9975709019424024, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9976905312919334, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9978278778758977, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9979523433288222, "operator": "ge"}, {"budget": 0.73, "threshold": -0.998057071250286, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9981660539629839, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9982645178762415, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9983547499275425, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9984443137749838, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9985230137752487, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9986014609121803, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9986883140899848, "operator": "ge"}, {"budget": 0.81, "threshold": -0.998773635843096, "operator": "ge"}, {"budget": 0.82, "threshold": -0.998841472568386, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9989162131926199, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9989753230295565, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9990340780934276, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9990897044540891, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9991389202878209, "operator": "ge"}, {"budget": 0.88, "threshold": -0.999191940263597, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9992434657998013, "operator": "ge"}, {"budget": 0.9, "threshold": -0.999296354365332, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9993402256013763, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9993859664247036, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9994286629976115, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9994827917468274, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9995291843667137, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9995741223716498, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9996283363897818, "operator": "ge"}, {"budget": 0.98, "threshold": -0.999683612905228, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9997570540301356, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.47965422728231, 75.91538407477687, 76.21758380771664, 76.54789514372057, 76.74467636516972, 76.99065289198117, 77.33502002951718, 77.62316396092487, 77.94644739616277, 78.23459132757046, 78.5508468620423, 78.81087919038583, 79.04982781643123, 79.32391594630684, 79.51366926698995, 79.73153419073724, 79.90723170988826, 80.13212453440157, 80.41324056504322, 80.68732869491882, 80.80680300794153, 81.05277953475297, 81.21442125237192, 81.46039777918335, 81.59392789373814, 81.80476491671938, 81.98749033663645, 82.1842715580856, 82.33888537493851, 82.41619228336496, 82.52863869562161, 82.71136411553869, 82.78867102396515, 82.90111743622181, 83.01356384847847, 83.14709396303324, 83.23142877222574, 83.27359617682198, 83.35090308524843, 83.33684728371635, 83.39307048984467, 83.35090308524843, 83.40712629137677, 83.4984890013353, 83.54065640593154, 83.55471220746364, 83.54768430669759, 83.69527022278446, 83.63904701665612, 83.60390751282591, 83.57579590976175, 83.5336285051655, 83.54065640593154, 83.48443319980322, 83.40712629137677, 83.31576358141824, 83.24548457375782, 83.2244008714597, 83.16817766533137, 83.13303816150116, 83.07681495537284, 82.95734064235013, 82.90814533698784, 82.88706163468972, 82.79569892473118, 82.71136411553869, 82.68325251247452, 82.61297350481411, 82.54972239791975, 82.47944339025933, 82.44430388642913, 82.38808068030079, 82.34591327570455, 82.24049476421393, 82.1842715580856, 82.12102045119123, 82.07182514582894, 82.05074144353081, 81.95937873357228, 81.93126713050812, 81.86098812284771, 81.81179281748541, 81.74854171059106, 81.7063743059948, 81.62906739756835, 81.54473258837585, 81.43228617611919, 81.3479513669267, 81.31983976386253, 81.228477053904, 81.22144915313795, 81.15819804624358, 81.1090027408813, 81.03872373322089, 80.98250052709255, 80.9051936186661, 80.84897041253778, 80.76463560334528, 80.67327289338675, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.992128751142033, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99079344999649, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 73, 138, 192, 246, 286, 331, 385, 437, 493, 546, 601, 651, 702, 757, 798, 842, 888, 932, 984, 1030, 1068, 1118, 1156, 1203, 1237, 1277, 1310, 1347, 1381, 1409, 1439, 1475, 1503, 1530, 1561, 1587, 1609, 1626, 1645, 1658, 1678, 1689, 1711, 1732, 1748, 1765, 1781, 1807, 1821, 1831, 1843, 1856, 1870, 1874, 1877, 1882, 1891, 1899, 1905, 1910, 1916, 1918, 1924, 1933, 1937, 1940, 1943, 1944, 1947, 1947, 1950, 1952, 1952, 1952, 1954, 1956, 1957, 1958, 1958, 1958, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1962, 1962, 1962, 1963, 1963, 1963, 1963, 1963, 1963, 1964], "harmed": [0, 9, 12, 23, 30, 42, 52, 57, 68, 78, 90, 100, 113, 130, 146, 160, 173, 194, 206, 218, 225, 246, 261, 276, 288, 303, 313, 320, 329, 341, 358, 372, 382, 399, 410, 425, 432, 442, 453, 461, 476, 488, 505, 519, 527, 537, 552, 569, 574, 596, 611, 627, 646, 659, 671, 685, 703, 722, 733, 747, 757, 771, 790, 803, 815, 832, 847, 854, 865, 877, 887, 895, 905, 911, 926, 936, 947, 955, 959, 972, 976, 988, 995, 1004, 1010, 1021, 1033, 1049, 1061, 1065, 1078, 1081, 1090, 1097, 1108, 1116, 1127, 1135, 1147, 1160, 1171], "exact_score": [75.02986857825567, 75.47965422728231, 75.91538407477687, 76.21758380771664, 76.54789514372057, 76.74467636516972, 76.99065289198117, 77.33502002951718, 77.62316396092487, 77.94644739616277, 78.23459132757046, 78.5508468620423, 78.81087919038583, 79.04982781643123, 79.32391594630684, 79.51366926698995, 79.73153419073724, 79.90723170988826, 80.13212453440157, 80.41324056504322, 80.68732869491882, 80.80680300794153, 81.05277953475297, 81.21442125237192, 81.46039777918335, 81.59392789373814, 81.80476491671938, 81.98749033663645, 82.19129945885165, 82.33888537493851, 82.41619228336496, 82.52863869562161, 82.71136411553869, 82.78867102396515, 82.90111743622181, 83.01356384847847, 83.14709396303324, 83.23142877222574, 83.27359617682198, 83.35090308524843, 83.33684728371635, 83.39307048984467, 83.35090308524843, 83.40712629137677, 83.4984890013353, 83.54065640593154, 83.55471220746364, 83.54768430669759, 83.69527022278446, 83.63904701665612, 83.60390751282591, 83.57579590976175, 83.5336285051655, 83.54065640593154, 83.48443319980322, 83.40712629137677, 83.31576358141824, 83.24548457375782, 83.2244008714597, 83.16817766533137, 83.13303816150116, 83.07681495537284, 82.95734064235013, 82.90814533698784, 82.88706163468972, 82.79569892473118, 82.71136411553869, 82.68325251247452, 82.61297350481411, 82.54972239791975, 82.47944339025933, 82.44430388642913, 82.38808068030079, 82.34591327570455, 82.24049476421393, 82.1842715580856, 82.12102045119123, 82.07182514582894, 82.05074144353081, 81.95937873357228, 81.93126713050812, 81.86098812284771, 81.81179281748541, 81.74854171059106, 81.7063743059948, 81.62906739756835, 81.54473258837585, 81.43228617611919, 81.3479513669267, 81.31983976386253, 81.228477053904, 81.22144915313795, 81.15819804624358, 81.1090027408813, 81.03872373322089, 80.98250052709255, 80.9051936186661, 80.84897041253778, 80.76463560334528, 80.67327289338675, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.52730952888659, "threshold_area100": 81.52705741794925, "threshold_area25": 78.69843277812917}, "test": {"score": [66.95119530385956, 68.23866375521084, 69.25672801520007, 70.16703059864446, 70.97240733913733, 71.59061906247342, 71.97345659756685, 72.36196579984686, 72.67957916229474, 73.01137169270908, 73.31480588719053, 73.60973258089216, 73.78271843008251, 73.99257011598559, 74.17973513314239, 74.44063182372457, 74.59660267135524, 74.73839435101948, 74.87735019709044, 74.98511187363526, 75.11839605251963, 75.22332189547117, 75.30556106967643, 75.39347191106826, 75.49839775401979, 75.5721294274452, 75.64018943368403, 75.67421943680344, 75.75362277741542, 75.78198111334827, 75.82451861724753, 75.90392195785951, 75.95780279613193, 75.99183279925134, 76.06840030627002, 76.1109378101693, 76.13929614610214, 76.18183365000142, 76.21019198593426, 76.17899781640813, 76.17899781640813, 76.19317698437456, 76.25272948983354, 76.25556532342682, 76.2782519921731, 76.28392365935966, 76.26407282420668, 76.2413861554604, 76.24705782264697, 76.26407282420668, 76.24705782264697, 76.21869948671412, 76.22720698749397, 76.23855032186711, 76.21586365312083, 76.19317698437456, 76.21019198593426, 76.24705782264697, 76.20735615234098, 76.16481864844171, 76.15347531406856, 76.10526614298273, 76.08257947423644, 76.10243030938945, 76.06556447267674, 76.04854947111703, 76.03720613674389, 75.9975044664379, 75.97481779769163, 75.94929529535207, 75.90959362504609, 75.89257862348637, 75.86705612114682, 75.83302611802739, 75.7933244477214, 75.76213027819527, 75.71675694070272, 75.68839860476987, 75.6657119360236, 75.61183109775118, 75.54944275869892, 75.50690525479965, 75.4586960837138, 75.41899441340783, 75.3849644102884, 75.34242690638912, 75.31406857045629, 75.26585939937044, 75.19496355953832, 75.12123188611292, 75.0701868814338, 75.04182854550095, 74.98794770722854, 74.93406686895612, 74.87451436349714, 74.83481269319117, 74.79227518929189, 74.73839435101948, 74.67033434478064, 74.58242350338881, 74.47182599325072], "calls": [0.0, 2.2261293707285255, 4.3643479000652246, 6.9449564699543425, 9.613475881235289, 11.5078127215495, 13.033491194736694, 14.516632164024616, 15.860817287241586, 17.085897399540595, 18.435754189944134, 19.666505969429714, 20.817854408303322, 22.110994526841164, 23.36726880866631, 24.518617247539915, 25.81742903326433, 26.9942999744775, 28.259081757082495, 29.5040127045345, 30.81416782463205, 31.99103876584522, 33.30970138672263, 34.69925984743215, 35.94135496129087, 37.35076425715339, 38.745994385049485, 40.13271701216573, 41.610186314267075, 42.897654765618356, 44.18512321696963, 45.38184499333579, 46.618268440007945, 47.792303547627824, 49.30380285284859, 50.90321299946119, 52.199188951592326, 53.52919490684287, 54.7883050222613, 56.294132660295496, 57.60145194679976, 58.81235289113235, 60.14235884638289, 61.1632589399654, 62.33162238039872, 63.502821654425325, 64.83849927686244, 66.00969855088903, 67.20925616084848, 68.39179876924823, 69.69344638856592, 70.79658565635368, 71.80897824915634, 72.81002750758586, 73.80824093242208, 74.88869353146357, 75.79332444772142, 76.67526869523297, 77.76706462864759, 78.45050052462922, 79.41751977993931, 80.15767234778663, 80.99707909139892, 81.83365000141791, 82.48305589428013, 83.18917845900803, 83.78470351359782, 84.46530357598616, 85.12038113603494, 85.78396619686357, 86.25187873975555, 86.84173212715878, 87.46845135127471, 87.99591639962567, 88.48367977767064, 88.93457731900291, 89.40532569548819, 89.93279074383915, 90.38935995235799, 90.95936250460824, 91.52369338967189, 92.08802427473556, 92.54175764966112, 93.07773019879193, 93.4945977370048, 93.91997277599751, 94.29713864390438, 94.61191617275898, 95.05997788049797, 95.45699458355784, 95.89654879051697, 96.20281881859172, 96.56864135212547, 96.9514788872189, 97.32013725434592, 97.68879562147293, 98.00357315032754, 98.46014235884638, 98.89969656580551, 99.42148994696991, 100.0], "rescued": [0, 469, 842, 1178, 1486, 1729, 1894, 2064, 2214, 2368, 2515, 2662, 2765, 2888, 2990, 3104, 3218, 3307, 3394, 3471, 3553, 3621, 3692, 3766, 3837, 3902, 3966, 4021, 4083, 4130, 4171, 4231, 4274, 4313, 4366, 4414, 4448, 4486, 4523, 4559, 4588, 4621, 4665, 4693, 4724, 4753, 4776, 4808, 4847, 4879, 4905, 4931, 4958, 4988, 5008, 5036, 5063, 5092, 5114, 5128, 5149, 5163, 5179, 5209, 5220, 5235, 5242, 5251, 5260, 5269, 5273, 5285, 5296, 5306, 5310, 5317, 5322, 5330, 5337, 5343, 5350, 5359, 5363, 5373, 5376, 5380, 5385, 5387, 5389, 5390, 5396, 5397, 5398, 5401, 5402, 5407, 5411, 5416, 5418, 5420, 5423], "harmed": [0, 15, 29, 44, 68, 93, 123, 156, 194, 231, 271, 314, 356, 405, 441, 463, 522, 561, 599, 638, 673, 704, 746, 789, 823, 862, 902, 945, 979, 1016, 1042, 1074, 1098, 1125, 1151, 1184, 1208, 1231, 1258, 1305, 1334, 1362, 1385, 1412, 1435, 1462, 1492, 1532, 1569, 1595, 1627, 1663, 1687, 1713, 1741, 1777, 1798, 1814, 1850, 1879, 1904, 1935, 1959, 1982, 2006, 2027, 2038, 2061, 2078, 2096, 2114, 2132, 2152, 2174, 2192, 2210, 2231, 2249, 2264, 2289, 2318, 2342, 2363, 2387, 2402, 2421, 2436, 2455, 2482, 2509, 2533, 2544, 2564, 2586, 2608, 2627, 2646, 2670, 2696, 2729, 2771], "exact_score": [66.95119530385956, 67.57224286078892, 68.12806624507274, 68.63568045827071, 69.09224966678956, 69.50911720500241, 69.86359640416299, 70.18971726739075, 70.50165896265207, 70.81643649150668, 71.10569151802171, 71.43748404843603, 71.7239032413578, 71.97062076397357, 72.22017412018262, 72.47256330998498, 72.73629583416044, 73.00002835833594, 73.22689504579871, 73.42540339732864, 73.6749567535377, 73.82242010038851, 73.98122678161245, 74.15137679720954, 74.33003431358648, 74.50585599637013, 74.61361767291496, 74.73839435101948, 74.84332019397102, 74.95391770410912, 75.04466437909423, 75.13257522048606, 75.22615772906445, 75.28287440093015, 75.36511357513541, 75.42466608059439, 75.50406942120637, 75.54377109151234, 75.60048776337804, 75.63168193290417, 75.66004026883702, 75.69974193914301, 75.75929444460199, 75.77914527975497, 75.83019028443411, 75.88690695629981, 75.93228029379236, 75.96063862972521, 75.9975044664379, 76.04571363752375, 76.07974364064316, 76.11660947735587, 76.127952811729, 76.15347531406856, 76.19034115078126, 76.21586365312083, 76.19034115078126, 76.17332614922157, 76.1846694835947, 76.19884865156112, 76.24422198905368, 76.25556532342682, 76.27258032498652, 76.28108782576638, 76.2584011570201, 76.26123699061338, 76.2413861554604, 76.25272948983354, 76.27258032498652, 76.24989365624025, 76.24705782264697, 76.23287865468053, 76.23287865468053, 76.24705782264697, 76.21019198593426, 76.19884865156112, 76.22437115390068, 76.24422198905368, 76.17899781640813, 76.15914698125515, 76.11377364376258, 76.08257947423644, 76.09392280860959, 76.05705697189688, 76.02586280237075, 75.98616113206477, 75.9407877945722, 75.88690695629981, 75.83586195162067, 75.7564586110087, 75.6827269375833, 75.61466693134446, 75.51257692198622, 75.41048691262796, 75.34526273998242, 75.20630689391147, 75.05884354706066, 74.92555936817628, 74.79227518929189, 74.6589910104075, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.72655933169075, "threshold_area100": 75.13980659614894, "threshold_area25": 73.14522303831211}, "selected_budget": 48, "selected": {"val": {"score": 83.69527022278446, "calls": 47.99353433129524, "rescued": 1807, "harmed": 574}, "test": {"score": 76.24705782264697, "calls": 67.20925616084848, "rescued": 4847, "harmed": 1569}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.573948860168457, "operator": "ge"}, {"budget": 0.02, "threshold": 0.38755080103874207, "operator": "ge"}, {"budget": 0.03, "threshold": 0.2466556429862976, "operator": "ge"}, {"budget": 0.04, "threshold": 0.07339704036712646, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0032633570954203606, "operator": "ge"}, {"budget": 0.06, "threshold": -0.005819469690322876, "operator": "gt"}, {"budget": 0.07, "threshold": -0.007481281645596027, "operator": "ge"}, {"budget": 0.08, "threshold": -0.00895753875374794, "operator": "ge"}, {"budget": 0.09, "threshold": -0.010291138663887978, "operator": "ge"}, {"budget": 0.1, "threshold": -0.01159602589905262, "operator": "ge"}, {"budget": 0.11, "threshold": -0.012949137017130852, "operator": "ge"}, {"budget": 0.12, "threshold": -0.01437219139188528, "operator": "ge"}, {"budget": 0.13, "threshold": -0.015932515263557434, "operator": "ge"}, {"budget": 0.14, "threshold": -0.0174487866461277, "operator": "ge"}, {"budget": 0.15, "threshold": -0.01917361468076706, "operator": "ge"}, {"budget": 0.16, "threshold": -0.020674405619502068, "operator": "ge"}, {"budget": 0.17, "threshold": -0.022186074405908585, "operator": "ge"}, {"budget": 0.18, "threshold": -0.02396838180720806, "operator": "ge"}, {"budget": 0.19, "threshold": -0.02590451017022133, "operator": "ge"}, {"budget": 0.2, "threshold": -0.02795705758035183, "operator": "ge"}, {"budget": 0.21, "threshold": -0.029793979600071907, "operator": "ge"}, {"budget": 0.22, "threshold": -0.031743329018354416, "operator": "ge"}, {"budget": 0.23, "threshold": -0.033428873866796494, "operator": "ge"}, {"budget": 0.24, "threshold": -0.03527842089533806, "operator": "ge"}, {"budget": 0.25, "threshold": -0.03786410391330719, "operator": "ge"}, {"budget": 0.26, "threshold": -0.04020475223660469, "operator": "ge"}, {"budget": 0.27, "threshold": -0.04209170863032341, "operator": "ge"}, {"budget": 0.28, "threshold": -0.0445290207862854, "operator": "ge"}, {"budget": 0.29, "threshold": -0.046741943806409836, "operator": "ge"}, {"budget": 0.3, "threshold": -0.04936780035495758, "operator": "ge"}, {"budget": 0.31, "threshold": -0.052003391087055206, "operator": "ge"}, {"budget": 0.32, "threshold": -0.0549691841006279, "operator": "ge"}, {"budget": 0.33, "threshold": -0.057396017014980316, "operator": "ge"}, {"budget": 0.34, "threshold": -0.06000719591975212, "operator": "ge"}, {"budget": 0.35, "threshold": -0.06298531591892242, "operator": "ge"}, {"budget": 0.36, "threshold": -0.06586035341024399, "operator": "ge"}, {"budget": 0.37, "threshold": -0.06881004571914673, "operator": "ge"}, {"budget": 0.38, "threshold": -0.0720471739768982, "operator": "ge"}, {"budget": 0.39, "threshold": -0.07513151317834854, "operator": "ge"}, {"budget": 0.4, "threshold": -0.07837627828121185, "operator": "ge"}, {"budget": 0.41, "threshold": -0.08238297700881958, "operator": "ge"}, {"budget": 0.42, "threshold": -0.08652637153863907, "operator": "ge"}, {"budget": 0.43, "threshold": -0.09030600637197495, "operator": "ge"}, {"budget": 0.44, "threshold": -0.09452221542596817, "operator": "ge"}, {"budget": 0.45, "threshold": -0.09909220784902573, "operator": "ge"}, {"budget": 0.46, "threshold": -0.10331308841705322, "operator": "ge"}, {"budget": 0.47, "threshold": -0.10807211697101593, "operator": "ge"}, {"budget": 0.48, "threshold": -0.11248557269573212, "operator": "ge"}, {"budget": 0.49, "threshold": -0.1173807680606842, "operator": "ge"}, {"budget": 0.5, "threshold": -0.12285421788692474, "operator": "ge"}, {"budget": 0.51, "threshold": -0.12865325808525085, "operator": "ge"}, {"budget": 0.52, "threshold": -0.13370662927627563, "operator": "ge"}, {"budget": 0.53, "threshold": -0.13970832526683807, "operator": "gt"}, {"budget": 0.54, "threshold": -0.14629897475242615, "operator": "ge"}, {"budget": 0.55, "threshold": -0.15216608345508575, "operator": "ge"}, {"budget": 0.56, "threshold": -0.15859480202198029, "operator": "ge"}, {"budget": 0.57, "threshold": -0.16551005840301514, "operator": "ge"}, {"budget": 0.58, "threshold": -0.1724981665611267, "operator": "ge"}, {"budget": 0.59, "threshold": -0.17939217388629913, "operator": "ge"}, {"budget": 0.6, "threshold": -0.18723498284816742, "operator": "ge"}, {"budget": 0.61, "threshold": -0.19511458277702332, "operator": "ge"}, {"budget": 0.62, "threshold": -0.2024100124835968, "operator": "ge"}, {"budget": 0.63, "threshold": -0.21082165837287903, "operator": "ge"}, {"budget": 0.64, "threshold": -0.21871230006217957, "operator": "ge"}, {"budget": 0.65, "threshold": -0.22792650759220123, "operator": "ge"}, {"budget": 0.66, "threshold": -0.23843523859977722, "operator": "ge"}, {"budget": 0.67, "threshold": -0.24677279591560364, "operator": "ge"}, {"budget": 0.68, "threshold": -0.25741615891456604, "operator": "ge"}, {"budget": 0.69, "threshold": -0.2660486698150635, "operator": "ge"}, {"budget": 0.7, "threshold": -0.2763559818267822, "operator": "ge"}, {"budget": 0.71, "threshold": -0.28569531440734863, "operator": "ge"}, {"budget": 0.72, "threshold": -0.29541924595832825, "operator": "ge"}, {"budget": 0.73, "threshold": -0.3067678213119507, "operator": "ge"}, {"budget": 0.74, "threshold": -0.31603243947029114, "operator": "ge"}, {"budget": 0.75, "threshold": -0.32698363065719604, "operator": "ge"}, {"budget": 0.76, "threshold": -0.33897680044174194, "operator": "ge"}, {"budget": 0.77, "threshold": -0.35307663679122925, "operator": "ge"}, {"budget": 0.78, "threshold": -0.3666793704032898, "operator": "ge"}, {"budget": 0.79, "threshold": -0.37736913561820984, "operator": "ge"}, {"budget": 0.8, "threshold": -0.39106741547584534, "operator": "ge"}, {"budget": 0.81, "threshold": -0.405178964138031, "operator": "ge"}, {"budget": 0.82, "threshold": -0.4179260730743408, "operator": "ge"}, {"budget": 0.83, "threshold": -0.43310827016830444, "operator": "ge"}, {"budget": 0.84, "threshold": -0.4468170404434204, "operator": "ge"}, {"budget": 0.85, "threshold": -0.46056699752807617, "operator": "ge"}, {"budget": 0.86, "threshold": -0.47593262791633606, "operator": "ge"}, {"budget": 0.87, "threshold": -0.49203240871429443, "operator": "ge"}, {"budget": 0.88, "threshold": -0.5075500011444092, "operator": "ge"}, {"budget": 0.89, "threshold": -0.5186344981193542, "operator": "ge"}, {"budget": 0.9, "threshold": -0.5305202603340149, "operator": "ge"}, {"budget": 0.91, "threshold": -0.5501494407653809, "operator": "ge"}, {"budget": 0.92, "threshold": -0.573620617389679, "operator": "ge"}, {"budget": 0.93, "threshold": -0.6014746427536011, "operator": "ge"}, {"budget": 0.94, "threshold": -0.6358799934387207, "operator": "gt"}, {"budget": 0.95, "threshold": -0.6702705025672913, "operator": "ge"}, {"budget": 0.96, "threshold": -0.7095680832862854, "operator": "ge"}, {"budget": 0.97, "threshold": -0.7546408772468567, "operator": "ge"}, {"budget": 0.98, "threshold": -0.8076281547546387, "operator": "ge"}, {"budget": 0.99, "threshold": -0.8638710975646973, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.5288495326446, 75.95755147937311, 76.29489071614309, 76.55492304448661, 76.61114625061495, 76.66034155597723, 76.66736945674327, 76.69548105980743, 76.71656476210555, 76.75170426593576, 76.77278796823389, 76.82198327359617, 76.89226228125659, 76.94848548738491, 76.95551338815096, 76.962541288917, 77.00470869351325, 77.03282029657741, 77.06795980040762, 77.10309930423782, 77.09607140347178, 77.12418300653596, 77.20148991496241, 77.20851781572844, 77.21554571649449, 77.25771312109073, 77.29988052568699, 77.3209642279851, 77.3209642279851, 77.36313163258134, 77.39827113641155, 77.43341064024176, 77.44043854100781, 77.47557804483802, 77.48963384637008, 77.51071754866821, 77.52477335020029, 77.55288495326445, 77.58802445709466, 77.59505235786071, 77.62316396092487, 77.65830346475508, 77.6793871670532, 77.75669407547966, 77.79183357930987, 77.86914048773632, 77.92536369386465, 77.98158689999298, 77.98861480075901, 77.96050319769485, 77.99564270152506, 78.05889380841943, 78.10808911378172, 78.12917281607984, 78.1221449153138, 78.10808911378172, 78.15025651837796, 78.15025651837796, 78.17836812144212, 78.21350762527233, 78.24864712910254, 78.29784243446483, 78.35406564059315, 78.41028884672149, 78.52273525897814, 78.47353995361586, 78.51570735821211, 78.5508468620423, 78.62112586970272, 78.6843769765971, 78.69140487736314, 78.7195164804273, 78.74060018272542, 78.78276758732167, 78.88115819804624, 78.90224190034438, 78.96549300723873, 79.03577201489915, 79.0427999156652, 79.16930212945394, 79.33094384707287, 79.49258556469184, 79.63314358001266, 79.75261789303535, 79.87209220605806, 79.98453861831472, 80.04076182444304, 80.12509663363554, 80.20240354206199, 80.1180687328695, 80.11104083210346, 80.25862674819032, 80.27268254972239, 80.3570173589149, 80.49757537423572, 80.64516129032259, 80.80680300794153, 80.77869140487736, 80.72246819874903, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.987771452667088, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99037177595053, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.9911448450348, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 76, 142, 198, 247, 263, 273, 278, 282, 289, 295, 298, 306, 316, 326, 330, 333, 340, 347, 352, 359, 364, 370, 381, 386, 390, 399, 408, 414, 419, 428, 436, 447, 453, 459, 467, 478, 487, 498, 508, 520, 533, 544, 556, 573, 590, 610, 623, 635, 643, 651, 663, 680, 697, 713, 725, 736, 751, 766, 780, 801, 817, 836, 857, 881, 909, 922, 942, 965, 992, 1017, 1037, 1059, 1086, 1116, 1146, 1166, 1192, 1221, 1240, 1276, 1318, 1360, 1393, 1428, 1464, 1503, 1533, 1566, 1596, 1617, 1649, 1695, 1724, 1759, 1803, 1846, 1884, 1912, 1938, 1964], "harmed": [0, 5, 10, 18, 30, 38, 41, 45, 45, 49, 50, 50, 51, 51, 53, 56, 58, 59, 62, 62, 64, 70, 72, 72, 76, 79, 82, 85, 88, 93, 96, 99, 105, 110, 111, 117, 125, 132, 139, 144, 155, 164, 170, 179, 185, 197, 206, 211, 215, 222, 234, 241, 249, 259, 272, 285, 298, 307, 322, 332, 348, 359, 371, 384, 400, 412, 432, 446, 464, 481, 497, 516, 534, 558, 582, 598, 615, 632, 651, 669, 687, 706, 725, 738, 756, 775, 798, 820, 841, 860, 893, 926, 951, 978, 1001, 1025, 1047, 1062, 1094, 1128, 1171], "exact_score": [75.02986857825567, 75.5288495326446, 75.95755147937311, 76.29489071614309, 76.55492304448661, 76.61114625061495, 76.66034155597723, 76.66736945674327, 76.69548105980743, 76.71656476210555, 76.75170426593576, 76.77278796823389, 76.82198327359617, 76.89226228125659, 76.94848548738491, 76.95551338815096, 76.962541288917, 77.00470869351325, 77.03282029657741, 77.06795980040762, 77.10309930423782, 77.09607140347178, 77.12418300653596, 77.20148991496241, 77.20851781572844, 77.21554571649449, 77.25771312109073, 77.29988052568699, 77.3209642279851, 77.3209642279851, 77.36313163258134, 77.39827113641155, 77.43341064024176, 77.44043854100781, 77.47557804483802, 77.48963384637008, 77.51071754866821, 77.52477335020029, 77.55288495326445, 77.58802445709466, 77.59505235786071, 77.62316396092487, 77.65830346475508, 77.6793871670532, 77.75669407547966, 77.79183357930987, 77.86914048773632, 77.92536369386465, 77.98158689999298, 77.98861480075901, 77.96050319769485, 77.99564270152506, 78.05889380841943, 78.10808911378172, 78.12917281607984, 78.1221449153138, 78.10808911378172, 78.15025651837796, 78.15025651837796, 78.17836812144212, 78.21350762527233, 78.24864712910254, 78.29784243446483, 78.35406564059315, 78.41028884672149, 78.52273525897814, 78.47353995361586, 78.51570735821211, 78.5508468620423, 78.62112586970272, 78.6843769765971, 78.69140487736314, 78.7195164804273, 78.74060018272542, 78.78276758732167, 78.88115819804624, 78.90224190034438, 78.96549300723873, 79.03577201489915, 79.0427999156652, 79.16930212945394, 79.33094384707287, 79.49258556469184, 79.63314358001266, 79.75261789303535, 79.87209220605806, 79.98453861831472, 80.04076182444304, 80.12509663363554, 80.20240354206199, 80.1180687328695, 80.11104083210346, 80.25862674819032, 80.27268254972239, 80.34998945814885, 80.49757537423572, 80.64516129032259, 80.80680300794153, 80.77869140487736, 80.72246819874903, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 78.15308145888197, "threshold_area100": 78.1529622601729, "threshold_area25": 76.70883407126293}, "test": {"score": [66.95119530385956, 68.35209709894224, 69.43538553157701, 70.2152397697303, 71.03763151178288, 71.36375237501063, 71.40345404531662, 71.41763321328304, 71.43748404843603, 71.45449904999575, 71.44882738280917, 71.48285738592858, 71.52539488982787, 71.52539488982787, 71.5537532257607, 71.58494739528685, 71.62181323199954, 71.61614156481298, 71.63882823355925, 71.66151490230554, 71.66435073589882, 71.69554490542495, 71.74942574369736, 71.77211241244363, 71.77211241244363, 71.80330658196978, 71.8345007514959, 71.85718742024217, 71.88838158976831, 71.90256075773473, 71.93375492726086, 71.94226242804072, 71.98763576553327, 72.01599410146613, 72.00748660068626, 72.03868077021241, 72.07271077333182, 72.12375577801095, 72.15778578113036, 72.18330828346993, 72.21733828658934, 72.24286078892891, 72.24002495533561, 72.23435328814904, 72.25420412330205, 72.27689079204832, 72.32226412954088, 72.36763746703343, 72.37047330062671, 72.43853330686555, 72.4668916427984, 72.48957831154468, 72.51793664747753, 72.60584748886934, 72.6710716615149, 72.73346000056716, 72.75047500212686, 72.77316167087315, 72.80435584039928, 72.84689334429856, 72.91495335053739, 72.99435669114936, 73.02271502708221, 73.0255508606755, 73.07376003176134, 73.10495420128747, 73.1730142075263, 73.19853670986586, 73.24958171454499, 73.30913422000397, 73.44525423248164, 73.50197090434733, 73.56152340980631, 73.57853841136603, 73.64376258401157, 73.69197175509741, 73.81107676601538, 73.86212177069449, 73.92451010974675, 73.96421178005275, 73.99257011598559, 74.066301789411, 74.09182429175057, 74.18257096673567, 74.23078013782151, 74.3271984799932, 74.38391515185889, 74.36690015029917, 74.39242265263874, 74.28466097609392, 74.13436179564982, 74.17406346595583, 74.26197430734764, 74.33570598077304, 74.36122848311261, 74.39809431982532, 74.48033349403057, 74.57108016901569, 74.5852593369821, 74.54555766667612, 74.47182599325072], "calls": [0.0, 2.413294387885319, 4.843603777330347, 7.04421064571931, 9.809148399171937, 12.16572611519156, 14.091257125031904, 15.45812891699515, 16.816493208178542, 18.061424155630547, 19.31769843745569, 20.52576354819499, 21.70263448940816, 22.919207100927316, 24.01383886793523, 25.173694807588692, 26.17757989961149, 27.119076652582024, 28.30729092816833, 29.350877690497125, 30.65819697700139, 31.670589569804044, 32.634772991520855, 33.434478064827154, 34.27955647562601, 35.43657658168618, 36.47165584323512, 37.29971925247427, 38.16181266483283, 38.918980234239854, 39.840626152057396, 40.682868729262964, 41.64705215097978, 42.307801378215125, 43.167058956980405, 43.95542069591357, 44.70691659813402, 45.43288999801492, 46.20423673538836, 46.95006097042226, 47.5739443609449, 48.47290361001617, 49.27828035050903, 49.99574624961007, 50.73306298386411, 51.575305561069676, 52.32396562969684, 53.14352153815614, 53.81277826617135, 54.55009500042538, 55.35547174091824, 56.16652014859768, 56.866971046139014, 57.635481949919175, 58.50891869665088, 59.322802937923605, 60.13385134560304, 61.083855599353434, 61.96012817967842, 62.80804242407056, 63.738195842667956, 64.61730425658622, 65.32909848850069, 66.17134106570627, 66.90015029918044, 67.7991095482517, 68.76329296996853, 69.51195303859569, 70.48747979468565, 71.38927487735019, 72.26554745767518, 73.05390919660834, 73.84510676913479, 74.61928934010153, 75.37362107591527, 76.21019198593426, 77.11198706859882, 78.16124549811417, 79.1027422510847, 79.79752148143947, 80.62558489067861, 81.49902163741032, 82.2448458724442, 83.16082012307517, 83.92649519326206, 84.72903610016165, 85.6308311828262, 86.54964126705045, 87.47128718486799, 88.40427643705867, 89.27487735019709, 90.21353826957434, 91.09264668349262, 92.12772594504155, 93.32444772140771, 94.37370615092307, 95.49102458667726, 96.70192553100983, 97.91282647534243, 98.9422340697048, 100.0], "rescued": [0, 507, 902, 1190, 1507, 1636, 1656, 1664, 1673, 1682, 1688, 1703, 1722, 1726, 1739, 1754, 1772, 1780, 1794, 1808, 1824, 1845, 1870, 1883, 1891, 1914, 1936, 1948, 1974, 1990, 2012, 2030, 2061, 2080, 2093, 2112, 2135, 2159, 2184, 2204, 2223, 2248, 2267, 2283, 2305, 2332, 2360, 2396, 2413, 2452, 2483, 2513, 2544, 2594, 2634, 2674, 2710, 2748, 2789, 2834, 2894, 2946, 2987, 3027, 3068, 3115, 3170, 3210, 3265, 3320, 3395, 3448, 3502, 3549, 3604, 3662, 3737, 3807, 3874, 3921, 3982, 4054, 4103, 4179, 4242, 4309, 4374, 4429, 4497, 4542, 4581, 4655, 4731, 4820, 4911, 4982, 5079, 5184, 5278, 5344, 5423], "harmed": [0, 13, 26, 39, 66, 80, 86, 89, 91, 94, 102, 105, 109, 113, 116, 120, 125, 135, 141, 147, 162, 172, 178, 183, 191, 203, 214, 218, 233, 244, 255, 270, 285, 294, 310, 318, 329, 335, 348, 359, 366, 382, 402, 420, 435, 454, 466, 486, 502, 517, 538, 560, 581, 600, 617, 635, 665, 695, 725, 755, 791, 815, 846, 885, 909, 945, 976, 1007, 1044, 1078, 1105, 1138, 1171, 1212, 1244, 1285, 1318, 1370, 1415, 1448, 1499, 1545, 1585, 1629, 1675, 1708, 1753, 1814, 1873, 1956, 2048, 2108, 2153, 2216, 2298, 2356, 2424, 2497, 2586, 2666, 2771], "exact_score": [66.95119530385956, 67.56373536000908, 68.11388707710631, 68.638516291864, 69.0837421660097, 69.4921022034427, 69.84374556901001, 70.20106060176388, 70.50165896265207, 70.81927232509996, 71.10001985083515, 71.33539403907778, 71.35808070782407, 71.38076737657035, 71.40345404531662, 71.4062898789099, 71.4233048804696, 71.43748404843603, 71.45449904999575, 71.45166321640247, 71.46867821796216, 71.49420072030172, 71.52539488982787, 71.52823072342115, 71.5537532257607, 71.57643989450699, 71.6133057312197, 71.6104698976264, 71.65017156793239, 71.65300740152568, 71.67002240308538, 71.6671865694921, 71.70972407339137, 71.7607690780705, 71.76927657885035, 71.79196324759663, 71.80614241556306, 71.84017241868247, 71.88270992258174, 71.89688909054817, 71.93091909366758, 71.964949096787, 71.99047159912656, 72.01032243427954, 72.04151660380569, 72.08688994129824, 72.15211411394378, 72.18614411706321, 72.22017412018262, 72.25420412330205, 72.23718912174233, 72.25420412330205, 72.31092079516773, 72.36480163344015, 72.39032413577972, 72.47539914357826, 72.49808581232453, 72.52360831466409, 72.6171908232425, 72.69942999744775, 72.75047500212686, 72.78166917165301, 72.80719167399256, 72.86674417945154, 72.94614752006352, 73.01987919348892, 73.03405836145535, 73.0822675325412, 73.10211836769417, 73.17868587471287, 73.24391004735841, 73.27226838329126, 73.3970450613958, 73.50197090434733, 73.57853841136603, 73.61540424807872, 73.657941751978, 73.79122593086238, 73.85361426991463, 73.91316677537361, 73.97555511442589, 74.01809261832514, 74.0634659558177, 74.16272013158267, 74.2336159714148, 74.3442134815529, 74.39242265263874, 74.36406431670589, 74.36973598389245, 74.16555596517595, 74.16839179876925, 74.27615347531406, 74.3271984799932, 74.38675098545218, 74.3810793182656, 74.42928848935144, 74.54839350026941, 74.56257266823583, 74.57958766979553, 74.54272183308284, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 72.39094119476962, "threshold_area100": 72.64733573433911, "threshold_area25": 71.08782576638403}, "selected_budget": 97, "selected": {"val": {"score": 80.80680300794153, "calls": 96.99908637290041, "rescued": 1884, "harmed": 1062}, "test": {"score": 74.57108016901569, "calls": 96.70192553100983, "rescued": 5184, "harmed": 2497}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.747201919555664, "operator": "ge"}, {"budget": 0.02, "threshold": 2.7758572101593018, "operator": "ge"}, {"budget": 0.03, "threshold": 1.656004548072815, "operator": "ge"}, {"budget": 0.04, "threshold": 0.468650221824646, "operator": "ge"}, {"budget": 0.05, "threshold": -0.36535266041755676, "operator": "ge"}, {"budget": 0.06, "threshold": -0.9593333005905151, "operator": "ge"}, {"budget": 0.07, "threshold": -1.309997320175171, "operator": "ge"}, {"budget": 0.08, "threshold": -1.6956663131713867, "operator": "ge"}, {"budget": 0.09, "threshold": -1.9763190746307373, "operator": "gt"}, {"budget": 0.1, "threshold": -2.201284646987915, "operator": "ge"}, {"budget": 0.11, "threshold": -2.4310760498046875, "operator": "ge"}, {"budget": 0.12, "threshold": -2.6705069541931152, "operator": "ge"}, {"budget": 0.13, "threshold": -2.8773701190948486, "operator": "ge"}, {"budget": 0.14, "threshold": -3.038264513015747, "operator": "ge"}, {"budget": 0.15, "threshold": -3.2197303771972656, "operator": "ge"}, {"budget": 0.16, "threshold": -3.3832201957702637, "operator": "ge"}, {"budget": 0.17, "threshold": -3.521120071411133, "operator": "ge"}, {"budget": 0.18, "threshold": -3.6394383907318115, "operator": "ge"}, {"budget": 0.19, "threshold": -3.7571184635162354, "operator": "ge"}, {"budget": 0.2, "threshold": -3.8660881519317627, "operator": "ge"}, {"budget": 0.21, "threshold": -3.973968505859375, "operator": "ge"}, {"budget": 0.22, "threshold": -4.064821720123291, "operator": "ge"}, {"budget": 0.23, "threshold": -4.1582512855529785, "operator": "ge"}, {"budget": 0.24, "threshold": -4.235827445983887, "operator": "ge"}, {"budget": 0.25, "threshold": -4.33338737487793, "operator": "ge"}, {"budget": 0.26, "threshold": -4.412886619567871, "operator": "ge"}, {"budget": 0.27, "threshold": -4.4845170974731445, "operator": "ge"}, {"budget": 0.28, "threshold": -4.559881210327148, "operator": "gt"}, {"budget": 0.29, "threshold": -4.636843681335449, "operator": "ge"}, {"budget": 0.3, "threshold": -4.710953712463379, "operator": "ge"}, {"budget": 0.31, "threshold": -4.7859015464782715, "operator": "ge"}, {"budget": 0.32, "threshold": -4.8608784675598145, "operator": "ge"}, {"budget": 0.33, "threshold": -4.926512718200684, "operator": "ge"}, {"budget": 0.34, "threshold": -4.996589183807373, "operator": "ge"}, {"budget": 0.35, "threshold": -5.069282531738281, "operator": "ge"}, {"budget": 0.36, "threshold": -5.125629425048828, "operator": "ge"}, {"budget": 0.37, "threshold": -5.183311462402344, "operator": "ge"}, {"budget": 0.38, "threshold": -5.249453544616699, "operator": "ge"}, {"budget": 0.39, "threshold": -5.310798168182373, "operator": "ge"}, {"budget": 0.4, "threshold": -5.3762383460998535, "operator": "ge"}, {"budget": 0.41, "threshold": -5.433351516723633, "operator": "ge"}, {"budget": 0.42, "threshold": -5.498702526092529, "operator": "ge"}, {"budget": 0.43, "threshold": -5.560158729553223, "operator": "ge"}, {"budget": 0.44, "threshold": -5.614567279815674, "operator": "ge"}, {"budget": 0.45, "threshold": -5.67452335357666, "operator": "ge"}, {"budget": 0.46, "threshold": -5.745790481567383, "operator": "ge"}, {"budget": 0.47, "threshold": -5.802664279937744, "operator": "gt"}, {"budget": 0.48, "threshold": -5.865262508392334, "operator": "ge"}, {"budget": 0.49, "threshold": -5.922348499298096, "operator": "ge"}, {"budget": 0.5, "threshold": -5.991913795471191, "operator": "ge"}, {"budget": 0.51, "threshold": -6.064230918884277, "operator": "ge"}, {"budget": 0.52, "threshold": -6.129998207092285, "operator": "ge"}, {"budget": 0.53, "threshold": -6.185393810272217, "operator": "ge"}, {"budget": 0.54, "threshold": -6.247965335845947, "operator": "ge"}, {"budget": 0.55, "threshold": -6.31732177734375, "operator": "ge"}, {"budget": 0.56, "threshold": -6.372010707855225, "operator": "ge"}, {"budget": 0.57, "threshold": -6.431121826171875, "operator": "ge"}, {"budget": 0.58, "threshold": -6.493961334228516, "operator": "ge"}, {"budget": 0.59, "threshold": -6.560245037078857, "operator": "ge"}, {"budget": 0.6, "threshold": -6.615140438079834, "operator": "ge"}, {"budget": 0.61, "threshold": -6.674010276794434, "operator": "ge"}, {"budget": 0.62, "threshold": -6.719710350036621, "operator": "ge"}, {"budget": 0.63, "threshold": -6.779465675354004, "operator": "ge"}, {"budget": 0.64, "threshold": -6.840047836303711, "operator": "ge"}, {"budget": 0.65, "threshold": -6.898130893707275, "operator": "ge"}, {"budget": 0.66, "threshold": -6.947852611541748, "operator": "ge"}, {"budget": 0.67, "threshold": -6.9987263679504395, "operator": "ge"}, {"budget": 0.68, "threshold": -7.045519828796387, "operator": "ge"}, {"budget": 0.69, "threshold": -7.0969438552856445, "operator": "ge"}, {"budget": 0.7, "threshold": -7.141877174377441, "operator": "ge"}, {"budget": 0.71, "threshold": -7.18295955657959, "operator": "ge"}, {"budget": 0.72, "threshold": -7.231268882751465, "operator": "ge"}, {"budget": 0.73, "threshold": -7.274943828582764, "operator": "ge"}, {"budget": 0.74, "threshold": -7.323902606964111, "operator": "ge"}, {"budget": 0.75, "threshold": -7.368987560272217, "operator": "ge"}, {"budget": 0.76, "threshold": -7.415596961975098, "operator": "ge"}, {"budget": 0.77, "threshold": -7.465680122375488, "operator": "ge"}, {"budget": 0.78, "threshold": -7.515416622161865, "operator": "ge"}, {"budget": 0.79, "threshold": -7.567330360412598, "operator": "gt"}, {"budget": 0.8, "threshold": -7.620960235595703, "operator": "ge"}, {"budget": 0.81, "threshold": -7.680673599243164, "operator": "ge"}, {"budget": 0.82, "threshold": -7.726430892944336, "operator": "ge"}, {"budget": 0.83, "threshold": -7.782312393188477, "operator": "ge"}, {"budget": 0.84, "threshold": -7.831414222717285, "operator": "ge"}, {"budget": 0.85, "threshold": -7.880924224853516, "operator": "ge"}, {"budget": 0.86, "threshold": -7.933337688446045, "operator": "ge"}, {"budget": 0.87, "threshold": -7.989975929260254, "operator": "ge"}, {"budget": 0.88, "threshold": -8.038725852966309, "operator": "ge"}, {"budget": 0.89, "threshold": -8.109935760498047, "operator": "ge"}, {"budget": 0.9, "threshold": -8.181662559509277, "operator": "ge"}, {"budget": 0.91, "threshold": -8.243900299072266, "operator": "ge"}, {"budget": 0.92, "threshold": -8.308279991149902, "operator": "ge"}, {"budget": 0.93, "threshold": -8.390151977539062, "operator": "ge"}, {"budget": 0.94, "threshold": -8.478909492492676, "operator": "ge"}, {"budget": 0.95, "threshold": -8.567283630371094, "operator": "ge"}, {"budget": 0.96, "threshold": -8.658409118652344, "operator": "ge"}, {"budget": 0.97, "threshold": -8.78647232055664, "operator": "ge"}, {"budget": 0.98, "threshold": -8.940401077270508, "operator": "ge"}, {"budget": 0.99, "threshold": -9.206958770751953, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.5288495326446, 76.02783048703353, 76.44247663222995, 76.57600674678474, 76.7235926628716, 76.92740178508679, 77.20851781572844, 77.44746644177386, 77.74966617471361, 78.0026706022911, 78.3118982359969, 78.50165155668002, 78.79682338885375, 79.1130789233256, 79.35905545013705, 79.62611567924661, 79.81586899992972, 80.00562232061283, 80.16726403823178, 80.31484995431865, 80.55379858036405, 80.76463560334528, 80.99655632862464, 81.1441422447115, 81.29172816079837, 81.52364888607772, 81.65015109986648, 81.7907091151873, 81.98749033663645, 82.07885304659499, 82.14210415348934, 82.2686063672781, 82.43024808489704, 82.51458289408954, 82.57783400098391, 82.63405720711224, 82.69730831400662, 82.75353152013494, 82.83786632932744, 82.87300583315765, 82.90111743622181, 82.92922903928597, 82.96436854311618, 82.97139644388221, 83.07681495537284, 83.12601026073511, 83.15412186379928, 83.14006606226721, 83.104926558437, 83.09789865767095, 83.06275915384074, 83.03464755077658, 83.06978705460679, 83.11898235996908, 83.12601026073511, 83.104926558437, 83.06275915384074, 83.01356384847847, 82.97139644388221, 82.91517323775388, 82.85895003162555, 82.73947571860285, 82.71136411553869, 82.68325251247452, 82.68325251247452, 82.64108510787828, 82.61297350481411, 82.55675029868578, 82.48647129102538, 82.44430388642913, 82.34591327570455, 82.32482957340642, 82.26157846651205, 82.1842715580856, 82.12804835195728, 82.0647972450629, 82.00154613816854, 82.00857403893457, 81.9453229320402, 81.84693232131562, 81.80476491671938, 81.74854171059106, 81.69231850446272, 81.66420690139856, 81.62203949680232, 81.49553728301356, 81.45336987841732, 81.39011877152295, 81.36903506922482, 81.34092346616066, 81.30578396233045, 81.24956075620211, 81.15819804624358, 81.1090027408813, 81.05277953475297, 80.96844472556047, 80.89113781713402, 80.79977510717549, 80.69435659568487, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.988685079766674, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.992128751142033, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.98854452175135, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.98657670953686, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 71, 145, 210, 246, 284, 324, 375, 419, 472, 527, 584, 634, 685, 739, 783, 833, 878, 915, 951, 987, 1034, 1074, 1117, 1157, 1191, 1231, 1256, 1285, 1323, 1347, 1366, 1397, 1427, 1451, 1469, 1493, 1514, 1535, 1555, 1571, 1596, 1616, 1634, 1653, 1674, 1692, 1710, 1722, 1737, 1753, 1763, 1774, 1788, 1800, 1812, 1819, 1828, 1839, 1846, 1852, 1858, 1862, 1864, 1872, 1881, 1886, 1889, 1895, 1897, 1902, 1902, 1905, 1909, 1912, 1912, 1915, 1920, 1925, 1927, 1929, 1934, 1938, 1940, 1943, 1945, 1946, 1947, 1949, 1952, 1953, 1957, 1959, 1959, 1960, 1961, 1961, 1961, 1961, 1961, 1964], "harmed": [0, 0, 3, 9, 26, 43, 54, 65, 75, 85, 104, 117, 140, 149, 158, 167, 179, 197, 207, 220, 235, 248, 258, 268, 287, 300, 307, 314, 323, 333, 344, 354, 367, 374, 386, 395, 411, 423, 436, 444, 455, 476, 492, 505, 523, 529, 540, 554, 568, 588, 605, 620, 635, 644, 649, 660, 670, 685, 703, 716, 730, 744, 765, 771, 783, 792, 803, 810, 824, 836, 847, 861, 867, 880, 894, 902, 914, 928, 932, 943, 959, 970, 982, 992, 999, 1007, 1026, 1033, 1044, 1050, 1055, 1064, 1074, 1087, 1095, 1104, 1116, 1127, 1140, 1155, 1171], "exact_score": [75.02986857825567, 75.5288495326446, 76.02783048703353, 76.44247663222995, 76.57600674678474, 76.7235926628716, 76.92740178508679, 77.20851781572844, 77.44746644177386, 77.74966617471361, 78.0026706022911, 78.3118982359969, 78.50165155668002, 78.79682338885375, 79.1130789233256, 79.35905545013705, 79.62611567924661, 79.81586899992972, 80.00562232061283, 80.16726403823178, 80.31484995431865, 80.55379858036405, 80.76463560334528, 80.99655632862464, 81.1441422447115, 81.29172816079837, 81.52364888607772, 81.65015109986648, 81.7907091151873, 81.98749033663645, 82.07885304659499, 82.14210415348934, 82.2686063672781, 82.43024808489704, 82.51458289408954, 82.57783400098391, 82.63405720711224, 82.69730831400662, 82.75353152013494, 82.83786632932744, 82.87300583315765, 82.90111743622181, 82.92922903928597, 82.96436854311618, 82.97139644388221, 83.07681495537284, 83.12601026073511, 83.15412186379928, 83.14006606226721, 83.104926558437, 83.09789865767095, 83.06275915384074, 83.03464755077658, 83.06978705460679, 83.11898235996908, 83.12601026073511, 83.104926558437, 83.06275915384074, 83.01356384847847, 82.97139644388221, 82.91517323775388, 82.85895003162555, 82.73947571860285, 82.71136411553869, 82.68325251247452, 82.68325251247452, 82.64108510787828, 82.61297350481411, 82.55675029868578, 82.48647129102538, 82.44430388642913, 82.34591327570455, 82.32482957340642, 82.26157846651205, 82.1842715580856, 82.12804835195728, 82.0647972450629, 82.00154613816854, 82.00857403893457, 81.9453229320402, 81.84693232131562, 81.80476491671938, 81.74854171059106, 81.69231850446272, 81.66420690139856, 81.62203949680232, 81.49553728301356, 81.45336987841732, 81.39011877152295, 81.36903506922482, 81.34092346616066, 81.30578396233045, 81.24956075620211, 81.15819804624358, 81.1090027408813, 81.05277953475297, 80.96844472556047, 80.89113781713402, 80.79977510717549, 80.69435659568487, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.33276968867833, "threshold_area100": 81.3325954037529, "threshold_area25": 78.51219340782909}, "test": {"score": [66.95119530385956, 68.64985962623713, 69.82956640104359, 70.46762895953266, 71.06598984771574, 71.556589059354, 71.95927742960043, 72.23151745455576, 72.62569832402235, 72.95749085443666, 73.20420837705244, 73.45943340044806, 73.76286759492953, 73.89048010662734, 74.0095851175453, 74.22227263704166, 74.40660182060516, 74.54839350026941, 74.66749851118736, 74.73839435101948, 74.88869353146357, 74.97944020644869, 75.03048521112781, 75.14108272126592, 75.20914272750476, 75.2743669001503, 75.35093440716899, 75.43033774778097, 75.49272608683322, 75.52959192354592, 75.58347276181834, 75.67138360321016, 75.68556277117658, 75.72810027507586, 75.76780194538185, 75.84436945240053, 75.88123528911323, 75.90959362504609, 75.95496696253863, 75.99183279925134, 76.05138530471032, 76.11660947735587, 76.13646031250886, 76.15347531406856, 76.147803646882, 76.17049031562829, 76.19601281796784, 76.2016844851544, 76.2413861554604, 76.24422198905368, 76.27541615857982, 76.24422198905368, 76.24705782264697, 76.22437115390068, 76.2045203187477, 76.19884865156112, 76.21302781952754, 76.21019198593426, 76.19884865156112, 76.17899781640813, 76.16481864844171, 76.10526614298273, 76.08825114142302, 76.0542211383036, 76.02869863596405, 76.00601196721776, 75.95780279613193, 75.92660862660578, 75.90959362504609, 75.91526529223265, 75.89257862348637, 75.82735445084083, 75.78481694694155, 75.7933244477214, 75.76213027819527, 75.74795111022885, 75.69123443836315, 75.64586110087059, 75.59481609619148, 75.54660692510564, 75.5522785922922, 75.5154127555795, 75.50123358761309, 75.4785469188668, 75.44451691574739, 75.3849644102884, 75.33959107279584, 75.27153106655702, 75.21481439469132, 75.17511272438533, 75.14108272126592, 75.08153021580694, 75.03615687831437, 74.95108187051584, 74.90287269942999, 74.87167852990386, 74.79511102288518, 74.74690185179934, 74.70436434790007, 74.60794600572838, 74.47182599325072], "calls": [0.0, 4.4692737430167595, 6.650029776252729, 8.033916569775686, 9.874372571817485, 11.49646938717636, 13.124237869721805, 14.201854635170008, 15.594248929472817, 16.853359044891246, 17.916796642373026, 18.91784590080254, 20.11173184357542, 21.263080282449025, 22.184726200266567, 23.17726795791623, 24.31443722882341, 25.352352323965633, 26.43280492300712, 27.58698919547401, 28.778039304653603, 29.83296940135553, 30.947452003516435, 32.07044210645719, 33.08850636644642, 34.46955732637609, 35.56135325979072, 36.70419419788446, 37.80449763207895, 39.006891075631685, 40.203612851997846, 41.411677962737144, 42.54884723364433, 43.805121515469466, 45.05288829651476, 46.34035674786603, 47.46334685080679, 48.56932195218784, 49.7688795621473, 50.920228001020895, 52.16799478206619, 53.217253211581536, 54.41964665513428, 55.5312934237019, 56.51249184697842, 57.67801945381845, 59.10444375124067, 60.19907551824859, 61.37594645946176, 62.43938405694354, 63.70416583954853, 65.03984346198564, 66.20537106882568, 67.22059949522162, 68.3577687661288, 69.53180387374869, 70.4591214587528, 71.34106570626436, 72.30241329438789, 73.37435839264953, 74.24212347219465, 75.23750106343759, 75.97765363128491, 76.88512038113603, 77.70751212318862, 78.5270680316479, 79.27856393386836, 80.03856733686867, 80.63692822505175, 81.35722995774609, 81.95842667952245, 82.53977256614581, 83.18067095822818, 83.77619601281798, 84.41142273771375, 84.98993279074384, 85.67904035391203, 86.36531208348694, 86.96650880526332, 87.65278053483821, 88.33905226441313, 89.02532399398804, 89.4762215353203, 90.09726909224966, 90.60488330544764, 91.08697501630604, 91.64563423418313, 92.22414428721322, 92.64101182542609, 93.23653688001589, 93.79519609789297, 94.31415364546409, 94.84445452740833, 95.42012874684514, 95.94192212800952, 96.37296883418881, 96.83804554348751, 97.47610810197658, 97.94118481127528, 98.80327822363384, 100.0], "rescued": [0, 605, 1030, 1263, 1507, 1718, 1892, 2015, 2192, 2350, 2465, 2581, 2724, 2812, 2889, 2996, 3105, 3177, 3252, 3319, 3406, 3471, 3528, 3598, 3652, 3722, 3779, 3838, 3887, 3937, 3986, 4039, 4083, 4133, 4177, 4235, 4275, 4311, 4345, 4380, 4424, 4460, 4490, 4529, 4554, 4587, 4632, 4662, 4710, 4734, 4771, 4801, 4831, 4847, 4869, 4901, 4926, 4946, 4967, 4986, 5003, 5019, 5032, 5051, 5067, 5083, 5098, 5110, 5120, 5136, 5146, 5153, 5163, 5179, 5192, 5204, 5210, 5224, 5231, 5247, 5268, 5278, 5286, 5297, 5305, 5309, 5317, 5322, 5324, 5329, 5334, 5339, 5350, 5354, 5361, 5368, 5371, 5383, 5390, 5401, 5423], "harmed": [0, 6, 15, 23, 56, 94, 126, 153, 191, 232, 260, 286, 322, 365, 400, 432, 476, 498, 531, 573, 607, 640, 679, 710, 740, 787, 817, 848, 875, 912, 942, 964, 1003, 1038, 1068, 1099, 1126, 1152, 1170, 1192, 1215, 1228, 1251, 1284, 1311, 1336, 1372, 1400, 1434, 1457, 1483, 1524, 1553, 1577, 1606, 1640, 1660, 1681, 1706, 1732, 1754, 1791, 1810, 1841, 1866, 1890, 1922, 1945, 1961, 1975, 1993, 2023, 2048, 2061, 2085, 2102, 2128, 2158, 2183, 2216, 2235, 2258, 2271, 2290, 2310, 2335, 2359, 2388, 2410, 2429, 2446, 2472, 2499, 2533, 2557, 2575, 2605, 2634, 2656, 2701, 2771], "exact_score": [66.95119530385956, 67.19791282647533, 67.51552618892323, 67.90119955760996, 68.4144854379945, 68.99299549102459, 69.50344553781585, 70.01389558460708, 70.45344979156623, 70.80792899072682, 71.0915123500553, 71.43748404843603, 71.68136573745852, 71.93942659444743, 72.17763661628337, 72.46972747639168, 72.74196750134702, 73.00286419192922, 73.23256671298527, 73.4877917363809, 73.72033009103025, 73.86212177069449, 73.99257011598559, 74.16839179876925, 74.3810793182656, 74.51719933074327, 74.59093100416867, 74.69302101352693, 74.79511102288518, 74.92839520176956, 74.9567535377024, 75.03332104472109, 75.13541105407936, 75.20630689391147, 75.24033689703089, 75.30272523608315, 75.37645690950855, 75.43600941496753, 75.48421858605336, 75.52675608995264, 75.58630859541162, 75.62033859853103, 75.6657119360236, 75.68556277117658, 75.73093610866914, 75.75929444460199, 75.83586195162067, 75.86705612114682, 75.87839945551995, 75.90675779145279, 75.95213112894535, 75.9975044664379, 76.04854947111703, 76.10810197657601, 76.13362447891558, 76.15347531406856, 76.15631114766185, 76.15063948047528, 76.16198281484843, 76.19601281796784, 76.20735615234098, 76.23855032186711, 76.2413861554604, 76.26123699061338, 76.28675949295295, 76.24705782264697, 76.24422198905368, 76.23855032186711, 76.21302781952754, 76.19601281796784, 76.21869948671412, 76.2215353203074, 76.21586365312083, 76.19601281796784, 76.17049031562829, 76.11944531094915, 76.09108697501631, 76.04571363752375, 76.00884780081105, 75.98332529847148, 75.92944446019906, 75.93228029379236, 75.89257862348637, 75.79048861412812, 75.77347361256841, 75.74795111022885, 75.67138360321016, 75.59198026259818, 75.55511442588548, 75.5154127555795, 75.47571108527352, 75.4048152454414, 75.29705356889657, 75.19496355953832, 75.1269035532995, 75.01914187675467, 74.90287269942999, 74.79227518929189, 74.70436434790007, 74.58809517057539, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.61278476792883, "threshold_area100": 75.07324958171453, "threshold_area25": 72.99690894138332}, "selected_budget": 47, "selected": {"val": {"score": 83.15412186379928, "calls": 46.98854452175135, "rescued": 1710, "harmed": 554}, "test": {"score": 76.2016844851544, "calls": 60.19907551824859, "rescued": 4662, "harmed": 1400}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9695816040039062, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6229157447814941, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3429579734802246, "operator": "ge"}, {"budget": 0.04, "threshold": 0.11569976806640625, "operator": "ge"}, {"budget": 0.05, "threshold": -0.18580257892608643, "operator": "ge"}, {"budget": 0.06, "threshold": -0.5150008797645569, "operator": "ge"}, {"budget": 0.07, "threshold": -0.7679587602615356, "operator": "ge"}, {"budget": 0.08, "threshold": -0.9937725067138672, "operator": "ge"}, {"budget": 0.09, "threshold": -1.1834280490875244, "operator": "ge"}, {"budget": 0.1, "threshold": -1.3900154829025269, "operator": "ge"}, {"budget": 0.11, "threshold": -1.573272705078125, "operator": "ge"}, {"budget": 0.12, "threshold": -1.7621541023254395, "operator": "ge"}, {"budget": 0.13, "threshold": -1.961374282836914, "operator": "ge"}, {"budget": 0.14, "threshold": -2.139106512069702, "operator": "ge"}, {"budget": 0.15, "threshold": -2.2923808097839355, "operator": "ge"}, {"budget": 0.16, "threshold": -2.440477132797241, "operator": "ge"}, {"budget": 0.17, "threshold": -2.581861972808838, "operator": "ge"}, {"budget": 0.18, "threshold": -2.7118782997131348, "operator": "ge"}, {"budget": 0.19, "threshold": -2.8375401496887207, "operator": "ge"}, {"budget": 0.2, "threshold": -2.961303234100342, "operator": "ge"}, {"budget": 0.21, "threshold": -3.0656604766845703, "operator": "ge"}, {"budget": 0.22, "threshold": -3.175210475921631, "operator": "ge"}, {"budget": 0.23, "threshold": -3.284616470336914, "operator": "ge"}, {"budget": 0.24, "threshold": -3.377582311630249, "operator": "ge"}, {"budget": 0.25, "threshold": -3.469700813293457, "operator": "ge"}, {"budget": 0.26, "threshold": -3.568671226501465, "operator": "ge"}, {"budget": 0.27, "threshold": -3.6642141342163086, "operator": "ge"}, {"budget": 0.28, "threshold": -3.760967254638672, "operator": "gt"}, {"budget": 0.29, "threshold": -3.8430309295654297, "operator": "ge"}, {"budget": 0.3, "threshold": -3.920898675918579, "operator": "ge"}, {"budget": 0.31, "threshold": -3.9928202629089355, "operator": "ge"}, {"budget": 0.32, "threshold": -4.058765411376953, "operator": "ge"}, {"budget": 0.33, "threshold": -4.120963096618652, "operator": "ge"}, {"budget": 0.34, "threshold": -4.2001447677612305, "operator": "ge"}, {"budget": 0.35, "threshold": -4.288780689239502, "operator": "ge"}, {"budget": 0.36, "threshold": -4.357901573181152, "operator": "ge"}, {"budget": 0.37, "threshold": -4.427194118499756, "operator": "ge"}, {"budget": 0.38, "threshold": -4.496133804321289, "operator": "ge"}, {"budget": 0.39, "threshold": -4.577457427978516, "operator": "gt"}, {"budget": 0.4, "threshold": -4.649257659912109, "operator": "ge"}, {"budget": 0.41, "threshold": -4.712872505187988, "operator": "ge"}, {"budget": 0.42, "threshold": -4.786676406860352, "operator": "ge"}, {"budget": 0.43, "threshold": -4.848266124725342, "operator": "ge"}, {"budget": 0.44, "threshold": -4.914794445037842, "operator": "ge"}, {"budget": 0.45, "threshold": -4.982792854309082, "operator": "ge"}, {"budget": 0.46, "threshold": -5.062514305114746, "operator": "ge"}, {"budget": 0.47, "threshold": -5.131683826446533, "operator": "ge"}, {"budget": 0.48, "threshold": -5.204346179962158, "operator": "ge"}, {"budget": 0.49, "threshold": -5.282500743865967, "operator": "ge"}, {"budget": 0.5, "threshold": -5.36422061920166, "operator": "ge"}, {"budget": 0.51, "threshold": -5.433103084564209, "operator": "ge"}, {"budget": 0.52, "threshold": -5.507991790771484, "operator": "ge"}, {"budget": 0.53, "threshold": -5.576225757598877, "operator": "ge"}, {"budget": 0.54, "threshold": -5.6461310386657715, "operator": "ge"}, {"budget": 0.55, "threshold": -5.726531505584717, "operator": "ge"}, {"budget": 0.56, "threshold": -5.796299457550049, "operator": "ge"}, {"budget": 0.57, "threshold": -5.866419792175293, "operator": "ge"}, {"budget": 0.58, "threshold": -5.9485626220703125, "operator": "ge"}, {"budget": 0.59, "threshold": -6.0061750411987305, "operator": "ge"}, {"budget": 0.6, "threshold": -6.088115692138672, "operator": "ge"}, {"budget": 0.61, "threshold": -6.154220104217529, "operator": "ge"}, {"budget": 0.62, "threshold": -6.230716705322266, "operator": "ge"}, {"budget": 0.63, "threshold": -6.3156561851501465, "operator": "ge"}, {"budget": 0.64, "threshold": -6.376838684082031, "operator": "ge"}, {"budget": 0.65, "threshold": -6.446814060211182, "operator": "ge"}, {"budget": 0.66, "threshold": -6.507835388183594, "operator": "ge"}, {"budget": 0.67, "threshold": -6.578446388244629, "operator": "ge"}, {"budget": 0.68, "threshold": -6.646512985229492, "operator": "ge"}, {"budget": 0.69, "threshold": -6.712711811065674, "operator": "ge"}, {"budget": 0.7, "threshold": -6.7627668380737305, "operator": "ge"}, {"budget": 0.71, "threshold": -6.824196815490723, "operator": "ge"}, {"budget": 0.72, "threshold": -6.883285999298096, "operator": "ge"}, {"budget": 0.73, "threshold": -6.935741901397705, "operator": "ge"}, {"budget": 0.74, "threshold": -6.993564128875732, "operator": "ge"}, {"budget": 0.75, "threshold": -7.048834323883057, "operator": "ge"}, {"budget": 0.76, "threshold": -7.102805137634277, "operator": "ge"}, {"budget": 0.77, "threshold": -7.158435821533203, "operator": "ge"}, {"budget": 0.78, "threshold": -7.2102370262146, "operator": "ge"}, {"budget": 0.79, "threshold": -7.264819622039795, "operator": "ge"}, {"budget": 0.8, "threshold": -7.329179763793945, "operator": "ge"}, {"budget": 0.81, "threshold": -7.396449565887451, "operator": "ge"}, {"budget": 0.82, "threshold": -7.453205585479736, "operator": "ge"}, {"budget": 0.83, "threshold": -7.520247936248779, "operator": "ge"}, {"budget": 0.84, "threshold": -7.576286792755127, "operator": "ge"}, {"budget": 0.85, "threshold": -7.635186195373535, "operator": "ge"}, {"budget": 0.86, "threshold": -7.694907188415527, "operator": "ge"}, {"budget": 0.87, "threshold": -7.750107288360596, "operator": "ge"}, {"budget": 0.88, "threshold": -7.813844203948975, "operator": "ge"}, {"budget": 0.89, "threshold": -7.879778861999512, "operator": "ge"}, {"budget": 0.9, "threshold": -7.951992988586426, "operator": "ge"}, {"budget": 0.91, "threshold": -8.016525268554688, "operator": "ge"}, {"budget": 0.92, "threshold": -8.08846378326416, "operator": "ge"}, {"budget": 0.93, "threshold": -8.16071605682373, "operator": "ge"}, {"budget": 0.94, "threshold": -8.260148048400879, "operator": "ge"}, {"budget": 0.95, "threshold": -8.353991508483887, "operator": "ge"}, {"budget": 0.96, "threshold": -8.454599380493164, "operator": "ge"}, {"budget": 0.97, "threshold": -8.591462135314941, "operator": "ge"}, {"budget": 0.98, "threshold": -8.751845359802246, "operator": "ge"}, {"budget": 0.99, "threshold": -9.016016960144043, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.47965422728231, 75.91538407477687, 76.21758380771664, 76.54789514372057, 76.74467636516972, 76.99065289198117, 77.33502002951718, 77.62316396092487, 77.94644739616277, 78.23459132757046, 78.5508468620423, 78.81087919038583, 79.04982781643123, 79.32391594630684, 79.51366926698995, 79.73153419073724, 79.90723170988826, 80.13212453440157, 80.41324056504322, 80.68732869491882, 80.80680300794153, 81.05277953475297, 81.21442125237192, 81.46039777918335, 81.59392789373814, 81.80476491671938, 81.98749033663645, 82.1842715580856, 82.33888537493851, 82.41619228336496, 82.52863869562161, 82.71136411553869, 82.78867102396515, 82.90111743622181, 83.01356384847847, 83.14709396303324, 83.23142877222574, 83.27359617682198, 83.35090308524843, 83.33684728371635, 83.39307048984467, 83.35090308524843, 83.40712629137677, 83.4984890013353, 83.54065640593154, 83.55471220746364, 83.54768430669759, 83.68824232201841, 83.63904701665612, 83.60390751282591, 83.57579590976175, 83.5336285051655, 83.54065640593154, 83.48443319980322, 83.40712629137677, 83.32279148218427, 83.24548457375782, 83.2244008714597, 83.16817766533137, 83.13303816150116, 83.07681495537284, 82.95734064235013, 82.90814533698784, 82.88706163468972, 82.79569892473118, 82.71136411553869, 82.68325251247452, 82.61297350481411, 82.54972239791975, 82.47944339025933, 82.44430388642913, 82.38808068030079, 82.34591327570455, 82.24049476421393, 82.1842715580856, 82.12102045119123, 82.07182514582894, 82.05074144353081, 81.95937873357228, 81.93126713050812, 81.86098812284771, 81.81179281748541, 81.74854171059106, 81.7063743059948, 81.62906739756835, 81.54473258837585, 81.43228617611919, 81.3479513669267, 81.31983976386253, 81.228477053904, 81.22144915313795, 81.15819804624358, 81.1090027408813, 81.03872373322089, 80.98250052709255, 80.9051936186661, 80.84897041253778, 80.76463560334528, 80.67327289338675, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.992128751142033, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99079344999649, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 73, 138, 192, 246, 286, 331, 385, 437, 493, 546, 601, 651, 702, 757, 798, 842, 888, 932, 984, 1030, 1068, 1118, 1156, 1203, 1237, 1277, 1310, 1347, 1381, 1409, 1439, 1475, 1503, 1530, 1561, 1587, 1609, 1626, 1645, 1658, 1678, 1689, 1711, 1732, 1748, 1765, 1781, 1807, 1821, 1831, 1843, 1856, 1870, 1874, 1877, 1883, 1891, 1899, 1905, 1910, 1916, 1918, 1924, 1933, 1937, 1940, 1943, 1944, 1947, 1947, 1950, 1952, 1952, 1952, 1954, 1956, 1957, 1958, 1958, 1958, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1962, 1962, 1962, 1963, 1963, 1963, 1963, 1963, 1963, 1964], "harmed": [0, 9, 12, 23, 30, 42, 52, 57, 68, 78, 90, 100, 113, 130, 146, 160, 173, 194, 206, 218, 225, 246, 261, 276, 288, 303, 313, 320, 329, 341, 358, 372, 382, 399, 410, 425, 432, 442, 453, 461, 476, 488, 505, 519, 527, 537, 552, 569, 575, 596, 611, 627, 646, 659, 671, 685, 703, 722, 733, 747, 757, 771, 790, 803, 815, 832, 847, 854, 865, 877, 887, 895, 905, 911, 926, 936, 947, 955, 959, 972, 976, 988, 995, 1004, 1010, 1021, 1033, 1049, 1061, 1065, 1078, 1081, 1090, 1097, 1108, 1116, 1127, 1135, 1147, 1160, 1171], "exact_score": [75.02986857825567, 75.47965422728231, 75.91538407477687, 76.21758380771664, 76.54789514372057, 76.74467636516972, 76.99065289198117, 77.33502002951718, 77.62316396092487, 77.94644739616277, 78.23459132757046, 78.5508468620423, 78.81087919038583, 79.04982781643123, 79.32391594630684, 79.51366926698995, 79.73153419073724, 79.90723170988826, 80.13212453440157, 80.41324056504322, 80.68732869491882, 80.80680300794153, 81.05277953475297, 81.21442125237192, 81.46039777918335, 81.59392789373814, 81.80476491671938, 81.98749033663645, 82.19129945885165, 82.33888537493851, 82.41619228336496, 82.52863869562161, 82.71136411553869, 82.78867102396515, 82.90111743622181, 83.01356384847847, 83.14709396303324, 83.23142877222574, 83.27359617682198, 83.35090308524843, 83.33684728371635, 83.39307048984467, 83.35090308524843, 83.40712629137677, 83.4984890013353, 83.54065640593154, 83.55471220746364, 83.54768430669759, 83.68824232201841, 83.63904701665612, 83.60390751282591, 83.57579590976175, 83.5336285051655, 83.54065640593154, 83.48443319980322, 83.40712629137677, 83.32279148218427, 83.24548457375782, 83.2244008714597, 83.16817766533137, 83.13303816150116, 83.07681495537284, 82.95734064235013, 82.90814533698784, 82.88706163468972, 82.79569892473118, 82.71136411553869, 82.68325251247452, 82.61297350481411, 82.54972239791975, 82.47944339025933, 82.44430388642913, 82.38808068030079, 82.34591327570455, 82.24049476421393, 82.1842715580856, 82.12102045119123, 82.07182514582894, 82.05074144353081, 81.95937873357228, 81.93126713050812, 81.86098812284771, 81.81179281748541, 81.74854171059106, 81.7063743059948, 81.62906739756835, 81.54473258837585, 81.43228617611919, 81.3479513669267, 81.31983976386253, 81.228477053904, 81.22144915313795, 81.15819804624358, 81.1090027408813, 81.03872373322089, 80.98250052709255, 80.9051936186661, 80.84897041253778, 80.76463560334528, 80.67327289338675, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.52730952888659, "threshold_area100": 81.52705741794925, "threshold_area25": 78.69843277812917}, "test": {"score": [66.95119530385956, 68.23866375521084, 69.25672801520007, 70.16703059864446, 70.97240733913733, 71.59061906247342, 71.97345659756685, 72.36196579984686, 72.67957916229474, 73.01137169270908, 73.31480588719053, 73.60973258089216, 73.78271843008251, 73.99257011598559, 74.17973513314239, 74.44063182372457, 74.59660267135524, 74.73839435101948, 74.87735019709044, 74.98511187363526, 75.11839605251963, 75.22332189547117, 75.30556106967643, 75.39347191106826, 75.49839775401979, 75.5721294274452, 75.64018943368403, 75.67421943680344, 75.75362277741542, 75.78198111334827, 75.82451861724753, 75.90392195785951, 75.95780279613193, 75.99183279925134, 76.06840030627002, 76.1109378101693, 76.13929614610214, 76.18183365000142, 76.21019198593426, 76.17899781640813, 76.17899781640813, 76.19317698437456, 76.25272948983354, 76.25556532342682, 76.2782519921731, 76.28392365935966, 76.26407282420668, 76.2413861554604, 76.24705782264697, 76.26690865779996, 76.24705782264697, 76.21869948671412, 76.22720698749397, 76.23855032186711, 76.21586365312083, 76.19317698437456, 76.21019198593426, 76.24705782264697, 76.20735615234098, 76.16481864844171, 76.15347531406856, 76.10526614298273, 76.08257947423644, 76.10243030938945, 76.06556447267674, 76.04854947111703, 76.03720613674389, 75.9975044664379, 75.97481779769163, 75.94929529535207, 75.90959362504609, 75.89257862348637, 75.86705612114682, 75.83302611802739, 75.7933244477214, 75.76213027819527, 75.719592774296, 75.68839860476987, 75.6657119360236, 75.61183109775118, 75.54944275869892, 75.50690525479965, 75.4586960837138, 75.41615857981454, 75.3849644102884, 75.34242690638912, 75.31406857045629, 75.26302356577716, 75.19496355953832, 75.12123188611292, 75.0701868814338, 75.04182854550095, 74.98794770722854, 74.93406686895612, 74.87451436349714, 74.83481269319117, 74.79227518929189, 74.73839435101948, 74.67033434478064, 74.58242350338881, 74.47182599325072], "calls": [0.0, 2.2261293707285255, 4.3643479000652246, 6.9449564699543425, 9.613475881235289, 11.5078127215495, 13.033491194736694, 14.516632164024616, 15.860817287241586, 17.085897399540595, 18.435754189944134, 19.666505969429714, 20.817854408303322, 22.110994526841164, 23.36726880866631, 24.518617247539915, 25.81742903326433, 26.9942999744775, 28.259081757082495, 29.5040127045345, 30.81416782463205, 31.99103876584522, 33.30970138672263, 34.69925984743215, 35.94135496129087, 37.35076425715339, 38.745994385049485, 40.13271701216573, 41.610186314267075, 42.897654765618356, 44.18512321696963, 45.38468082692908, 46.618268440007945, 47.78946771403454, 49.30380285284859, 50.90321299946119, 52.20202478518561, 53.52919490684287, 54.7883050222613, 56.294132660295496, 57.59861611320648, 58.80951705753906, 60.14235884638289, 61.1632589399654, 62.32878654680544, 63.502821654425325, 64.83849927686244, 66.00969855088903, 67.20925616084848, 68.3946346028415, 69.69344638856592, 70.79658565635368, 71.80897824915634, 72.81002750758586, 73.80824093242208, 74.88869353146357, 75.79332444772142, 76.67526869523297, 77.76706462864759, 78.45050052462922, 79.41751977993931, 80.15483651419335, 80.99424325780564, 81.83081416782463, 82.48305589428013, 83.19201429260131, 83.78470351359782, 84.46530357598616, 85.12605280322151, 85.78963786405014, 86.25187873975555, 86.84173212715878, 87.47412301846128, 87.99591639962567, 88.48367977767064, 88.93174148540963, 89.40816152908148, 89.92995491024587, 90.38085245157814, 90.95936250460824, 91.52369338967189, 92.08802427473556, 92.54175764966112, 93.07489436519865, 93.4945977370048, 93.91997277599751, 94.29713864390438, 94.61191617275898, 95.05997788049797, 95.45699458355784, 95.89654879051697, 96.20281881859172, 96.56864135212547, 96.9514788872189, 97.32013725434592, 97.68879562147293, 98.00357315032754, 98.46014235884638, 98.89969656580551, 99.42148994696991, 100.0], "rescued": [0, 469, 842, 1178, 1486, 1729, 1894, 2064, 2214, 2368, 2515, 2662, 2765, 2888, 2990, 3104, 3218, 3307, 3394, 3471, 3553, 3621, 3692, 3766, 3837, 3902, 3966, 4021, 4083, 4130, 4171, 4231, 4274, 4313, 4366, 4414, 4448, 4486, 4523, 4559, 4588, 4621, 4665, 4693, 4724, 4753, 4776, 4808, 4847, 4880, 4905, 4931, 4958, 4988, 5008, 5036, 5063, 5092, 5114, 5128, 5149, 5163, 5179, 5209, 5220, 5235, 5242, 5251, 5260, 5269, 5273, 5285, 5296, 5306, 5310, 5317, 5323, 5330, 5337, 5343, 5350, 5359, 5363, 5372, 5376, 5380, 5385, 5387, 5389, 5390, 5396, 5397, 5398, 5401, 5402, 5407, 5411, 5416, 5418, 5420, 5423], "harmed": [0, 15, 29, 44, 68, 93, 123, 156, 194, 231, 271, 314, 356, 405, 441, 463, 522, 561, 599, 638, 673, 704, 746, 789, 823, 862, 902, 945, 979, 1016, 1042, 1074, 1098, 1125, 1151, 1184, 1208, 1231, 1258, 1305, 1334, 1362, 1385, 1412, 1435, 1462, 1492, 1532, 1569, 1595, 1627, 1663, 1687, 1713, 1741, 1777, 1798, 1814, 1850, 1879, 1904, 1935, 1959, 1982, 2006, 2027, 2038, 2061, 2078, 2096, 2114, 2132, 2152, 2174, 2192, 2210, 2231, 2249, 2264, 2289, 2318, 2342, 2363, 2387, 2402, 2421, 2436, 2456, 2482, 2509, 2533, 2544, 2564, 2586, 2608, 2627, 2646, 2670, 2696, 2729, 2771], "exact_score": [66.95119530385956, 67.57224286078892, 68.12806624507274, 68.63568045827071, 69.09224966678956, 69.50911720500241, 69.86359640416299, 70.18971726739075, 70.50165896265207, 70.81643649150668, 71.10569151802171, 71.43748404843603, 71.7239032413578, 71.97062076397357, 72.22017412018262, 72.47256330998498, 72.73629583416044, 73.00002835833594, 73.22689504579871, 73.42540339732864, 73.6749567535377, 73.82242010038851, 73.98122678161245, 74.15137679720954, 74.33003431358648, 74.50585599637013, 74.61361767291496, 74.73839435101948, 74.84332019397102, 74.95391770410912, 75.04466437909423, 75.13257522048606, 75.22615772906445, 75.28287440093015, 75.36511357513541, 75.42466608059439, 75.50406942120637, 75.54377109151234, 75.60048776337804, 75.63168193290417, 75.66004026883702, 75.69974193914301, 75.75929444460199, 75.77914527975497, 75.83019028443411, 75.88690695629981, 75.93228029379236, 75.96063862972521, 75.9975044664379, 76.04571363752375, 76.07974364064316, 76.11660947735587, 76.127952811729, 76.15347531406856, 76.19034115078126, 76.21586365312083, 76.19034115078126, 76.17332614922157, 76.1846694835947, 76.19884865156112, 76.24422198905368, 76.25556532342682, 76.27258032498652, 76.28108782576638, 76.2584011570201, 76.26123699061338, 76.2413861554604, 76.25272948983354, 76.27258032498652, 76.24989365624025, 76.24705782264697, 76.23287865468053, 76.23287865468053, 76.24705782264697, 76.21019198593426, 76.19884865156112, 76.22437115390068, 76.2413861554604, 76.17899781640813, 76.15631114766185, 76.11377364376258, 76.08257947423644, 76.09392280860959, 76.05705697189688, 76.02586280237075, 75.98616113206477, 75.9407877945722, 75.88690695629981, 75.83586195162067, 75.7564586110087, 75.6827269375833, 75.61466693134446, 75.51257692198622, 75.41048691262796, 75.34526273998242, 75.20630689391147, 75.05884354706066, 74.92272353458299, 74.79227518929189, 74.6589910104075, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.72647424783679, "threshold_area100": 75.13980659614894, "threshold_area25": 73.14522303831211}, "selected_budget": 48, "selected": {"val": {"score": 83.68824232201841, "calls": 47.99353433129524, "rescued": 1807, "harmed": 575}, "test": {"score": 76.24705782264697, "calls": 67.20925616084848, "rescued": 4847, "harmed": 1569}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 80.80680300794153, "calls": 96.99908637290041}, "test": {"score": 74.57108016901569, "calls": 96.70192553100983}}, "old_normalized_interior": {"val": {"score": 83.69527022278446, "calls": 47.99353433129524}, "test": {"score": 76.24705782264697, "calls": 67.20925616084848}}, "old_normalized_endpoints": {"val": {"score": 83.69527022278446, "calls": 47.99353433129524}, "test": {"score": 76.24705782264697, "calls": 67.20925616084848}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.47965422728231, 75.91538407477687, 76.21758380771664, 76.54789514372057, 76.74467636516972, 76.99065289198117, 77.33502002951718, 77.62316396092487, 77.94644739616277, 78.23459132757046, 78.5508468620423, 78.81087919038583, 79.04982781643123, 79.32391594630684, 79.51366926698995, 79.73153419073724, 79.90723170988826, 80.13212453440157, 80.41324056504322, 80.68732869491882, 80.80680300794153, 81.05277953475297, 81.21442125237192, 81.46039777918335, 81.59392789373814, 81.80476491671938, 81.98749033663645, 82.1842715580856, 82.33888537493851, 82.41619228336496, 82.52863869562161, 82.71136411553869, 82.78867102396515, 82.90111743622181, 83.01356384847847, 83.14709396303324, 83.23142877222574, 83.27359617682198, 83.35090308524843, 83.33684728371635, 83.39307048984467, 83.35090308524843, 83.40712629137677, 83.4984890013353, 83.54065640593154, 83.55471220746364, 83.54768430669759, 83.69527022278446, 83.63904701665612, 83.60390751282591, 83.57579590976175, 83.5336285051655, 83.54065640593154, 83.48443319980322, 83.40712629137677, 83.31576358141824, 83.24548457375782, 83.2244008714597, 83.16817766533137, 83.13303816150116, 83.07681495537284, 82.95734064235013, 82.90814533698784, 82.88706163468972, 82.79569892473118, 82.71136411553869, 82.68325251247452, 82.61297350481411, 82.54972239791975, 82.47944339025933, 82.44430388642913, 82.38808068030079, 82.34591327570455, 82.24049476421393, 82.1842715580856, 82.12102045119123, 82.07182514582894, 82.05074144353081, 81.95937873357228, 81.93126713050812, 81.86098812284771, 81.81179281748541, 81.74854171059106, 81.7063743059948, 81.62906739756835, 81.54473258837585, 81.43228617611919, 81.3479513669267, 81.31983976386253, 81.228477053904, 81.22144915313795, 81.15819804624358, 81.1090027408813, 81.03872373322089, 80.98250052709255, 80.9051936186661, 80.84897041253778, 80.76463560334528, 80.67327289338675, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.992128751142033, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99079344999649, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.47965422728231, 75.91538407477687, 76.21758380771664, 76.54789514372057, 76.74467636516972, 76.99065289198117, 77.33502002951718, 77.62316396092487, 77.94644739616277, 78.23459132757046, 78.5508468620423, 78.81087919038583, 79.04982781643123, 79.32391594630684, 79.51366926698995, 79.73153419073724, 79.90723170988826, 80.13212453440157, 80.41324056504322, 80.68732869491882, 80.80680300794153, 81.05277953475297, 81.21442125237192, 81.46039777918335, 81.59392789373814, 81.80476491671938, 81.98749033663645, 82.19129945885165, 82.33888537493851, 82.41619228336496, 82.52863869562161, 82.71136411553869, 82.78867102396515, 82.90111743622181, 83.01356384847847, 83.14709396303324, 83.23142877222574, 83.27359617682198, 83.35090308524843, 83.33684728371635, 83.39307048984467, 83.35090308524843, 83.40712629137677, 83.4984890013353, 83.54065640593154, 83.55471220746364, 83.54768430669759, 83.69527022278446, 83.63904701665612, 83.60390751282591, 83.57579590976175, 83.5336285051655, 83.54065640593154, 83.48443319980322, 83.40712629137677, 83.31576358141824, 83.24548457375782, 83.2244008714597, 83.16817766533137, 83.13303816150116, 83.07681495537284, 82.95734064235013, 82.90814533698784, 82.88706163468972, 82.79569892473118, 82.71136411553869, 82.68325251247452, 82.61297350481411, 82.54972239791975, 82.47944339025933, 82.44430388642913, 82.38808068030079, 82.34591327570455, 82.24049476421393, 82.1842715580856, 82.12102045119123, 82.07182514582894, 82.05074144353081, 81.95937873357228, 81.93126713050812, 81.86098812284771, 81.81179281748541, 81.74854171059106, 81.7063743059948, 81.62906739756835, 81.54473258837585, 81.43228617611919, 81.3479513669267, 81.31983976386253, 81.228477053904, 81.22144915313795, 81.15819804624358, 81.1090027408813, 81.03872373322089, 80.98250052709255, 80.9051936186661, 80.84897041253778, 80.76463560334528, 80.67327289338675, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 78.69843277812917, "exact_area100": 81.52730952888659}, "test": {"score": [66.95119530385956, 68.23866375521084, 69.25672801520007, 70.16703059864446, 70.97240733913733, 71.59061906247342, 71.97345659756685, 72.36196579984686, 72.67957916229474, 73.01137169270908, 73.31480588719053, 73.60973258089216, 73.78271843008251, 73.99257011598559, 74.17973513314239, 74.44063182372457, 74.59660267135524, 74.73839435101948, 74.87735019709044, 74.98511187363526, 75.11839605251963, 75.22332189547117, 75.30556106967643, 75.39347191106826, 75.49839775401979, 75.5721294274452, 75.64018943368403, 75.67421943680344, 75.75362277741542, 75.78198111334827, 75.82451861724753, 75.90392195785951, 75.95780279613193, 75.99183279925134, 76.06840030627002, 76.1109378101693, 76.13929614610214, 76.18183365000142, 76.21019198593426, 76.17899781640813, 76.17899781640813, 76.19317698437456, 76.25272948983354, 76.25556532342682, 76.2782519921731, 76.28392365935966, 76.26407282420668, 76.2413861554604, 76.24705782264697, 76.26407282420668, 76.24705782264697, 76.21869948671412, 76.22720698749397, 76.23855032186711, 76.21586365312083, 76.19317698437456, 76.21019198593426, 76.24705782264697, 76.20735615234098, 76.16481864844171, 76.15347531406856, 76.10526614298273, 76.08257947423644, 76.10243030938945, 76.06556447267674, 76.04854947111703, 76.03720613674389, 75.9975044664379, 75.97481779769163, 75.94929529535207, 75.90959362504609, 75.89257862348637, 75.86705612114682, 75.83302611802739, 75.7933244477214, 75.76213027819527, 75.71675694070272, 75.68839860476987, 75.6657119360236, 75.61183109775118, 75.54944275869892, 75.50690525479965, 75.4586960837138, 75.41899441340783, 75.3849644102884, 75.34242690638912, 75.31406857045629, 75.26585939937044, 75.19496355953832, 75.12123188611292, 75.0701868814338, 75.04182854550095, 74.98794770722854, 74.93406686895612, 74.87451436349714, 74.83481269319117, 74.79227518929189, 74.73839435101948, 74.67033434478064, 74.58242350338881, 74.47182599325072], "calls": [0.0, 2.2261293707285255, 4.3643479000652246, 6.9449564699543425, 9.613475881235289, 11.5078127215495, 13.033491194736694, 14.516632164024616, 15.860817287241586, 17.085897399540595, 18.435754189944134, 19.666505969429714, 20.817854408303322, 22.110994526841164, 23.36726880866631, 24.518617247539915, 25.81742903326433, 26.9942999744775, 28.259081757082495, 29.5040127045345, 30.81416782463205, 31.99103876584522, 33.30970138672263, 34.69925984743215, 35.94135496129087, 37.35076425715339, 38.745994385049485, 40.13271701216573, 41.610186314267075, 42.897654765618356, 44.18512321696963, 45.38184499333579, 46.618268440007945, 47.792303547627824, 49.30380285284859, 50.90321299946119, 52.199188951592326, 53.52919490684287, 54.7883050222613, 56.294132660295496, 57.60145194679976, 58.81235289113235, 60.14235884638289, 61.1632589399654, 62.33162238039872, 63.502821654425325, 64.83849927686244, 66.00969855088903, 67.20925616084848, 68.39179876924823, 69.69344638856592, 70.79658565635368, 71.80897824915634, 72.81002750758586, 73.80824093242208, 74.88869353146357, 75.79332444772142, 76.67526869523297, 77.76706462864759, 78.45050052462922, 79.41751977993931, 80.15767234778663, 80.99707909139892, 81.83365000141791, 82.48305589428013, 83.18917845900803, 83.78470351359782, 84.46530357598616, 85.12038113603494, 85.78396619686357, 86.25187873975555, 86.84173212715878, 87.46845135127471, 87.99591639962567, 88.48367977767064, 88.93457731900291, 89.40532569548819, 89.93279074383915, 90.38935995235799, 90.95936250460824, 91.52369338967189, 92.08802427473556, 92.54175764966112, 93.07773019879193, 93.4945977370048, 93.91997277599751, 94.29713864390438, 94.61191617275898, 95.05997788049797, 95.45699458355784, 95.89654879051697, 96.20281881859172, 96.56864135212547, 96.9514788872189, 97.32013725434592, 97.68879562147293, 98.00357315032754, 98.46014235884638, 98.89969656580551, 99.42148994696991, 100.0], "exact_score": [66.95119530385956, 67.57224286078892, 68.12806624507274, 68.63568045827071, 69.09224966678956, 69.50911720500241, 69.86359640416299, 70.18971726739075, 70.50165896265207, 70.81643649150668, 71.10569151802171, 71.43748404843603, 71.7239032413578, 71.97062076397357, 72.22017412018262, 72.47256330998498, 72.73629583416044, 73.00002835833594, 73.22689504579871, 73.42540339732864, 73.6749567535377, 73.82242010038851, 73.98122678161245, 74.15137679720954, 74.33003431358648, 74.50585599637013, 74.61361767291496, 74.73839435101948, 74.84332019397102, 74.95391770410912, 75.04466437909423, 75.13257522048606, 75.22615772906445, 75.28287440093015, 75.36511357513541, 75.42466608059439, 75.50406942120637, 75.54377109151234, 75.60048776337804, 75.63168193290417, 75.66004026883702, 75.69974193914301, 75.75929444460199, 75.77914527975497, 75.83019028443411, 75.88690695629981, 75.93228029379236, 75.96063862972521, 75.9975044664379, 76.04571363752375, 76.07974364064316, 76.11660947735587, 76.127952811729, 76.15347531406856, 76.19034115078126, 76.21586365312083, 76.19034115078126, 76.17332614922157, 76.1846694835947, 76.19884865156112, 76.24422198905368, 76.25556532342682, 76.27258032498652, 76.28108782576638, 76.2584011570201, 76.26123699061338, 76.2413861554604, 76.25272948983354, 76.27258032498652, 76.24989365624025, 76.24705782264697, 76.23287865468053, 76.23287865468053, 76.24705782264697, 76.21019198593426, 76.19884865156112, 76.22437115390068, 76.24422198905368, 76.17899781640813, 76.15914698125515, 76.11377364376258, 76.08257947423644, 76.09392280860959, 76.05705697189688, 76.02586280237075, 75.98616113206477, 75.9407877945722, 75.88690695629981, 75.83586195162067, 75.7564586110087, 75.6827269375833, 75.61466693134446, 75.51257692198622, 75.41048691262796, 75.34526273998242, 75.20630689391147, 75.05884354706066, 74.92555936817628, 74.79227518929189, 74.6589910104075, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 73.14522303831211, "exact_area100": 74.72655933169075}}, "old_normalized_endpoint_budget": 48}, {"seed": 2027, "selection": {"best_epoch": 9, "epochs": 17, "alpha": 1.0, "validation_area100": 0.8106951654784657, "training_seconds": 124.69994783203583, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.37617828998910174, "operator": "ge"}, {"budget": 0.02, "threshold": 0.22773928455493783, "operator": "ge"}, {"budget": 0.03, "threshold": 0.15001217938457273, "operator": "ge"}, {"budget": 0.04, "threshold": 0.0925927921401443, "operator": "ge"}, {"budget": 0.05, "threshold": 0.04179709825308234, "operator": "ge"}, {"budget": 0.06, "threshold": 0.008819817773875239, "operator": "ge"}, {"budget": 0.07, "threshold": -0.05087447439840497, "operator": "ge"}, {"budget": 0.08, "threshold": -0.12992834899481953, "operator": "ge"}, {"budget": 0.09, "threshold": -0.1894545567846612, "operator": "ge"}, {"budget": 0.1, "threshold": -0.24448671389994667, "operator": "ge"}, {"budget": 0.11, "threshold": -0.3036519070768913, "operator": "ge"}, {"budget": 0.12, "threshold": -0.3649620575571973, "operator": "ge"}, {"budget": 0.13, "threshold": -0.41381312619501776, "operator": "ge"}, {"budget": 0.14, "threshold": -0.46268941106368994, "operator": "ge"}, {"budget": 0.15, "threshold": -0.5165260835216066, "operator": "ge"}, {"budget": 0.16, "threshold": -0.5621244820336804, "operator": "ge"}, {"budget": 0.17, "threshold": -0.6116064623566179, "operator": "ge"}, {"budget": 0.18, "threshold": -0.6526398563925192, "operator": "gt"}, {"budget": 0.19, "threshold": -0.6947154835461855, "operator": "gt"}, {"budget": 0.2, "threshold": -0.7302300325714979, "operator": "ge"}, {"budget": 0.21, "threshold": -0.7593356681568674, "operator": "ge"}, {"budget": 0.22, "threshold": -0.7872684679280252, "operator": "ge"}, {"budget": 0.23, "threshold": -0.8101018132785649, "operator": "ge"}, {"budget": 0.24, "threshold": -0.8302985896981078, "operator": "ge"}, {"budget": 0.25, "threshold": -0.849551667201818, "operator": "ge"}, {"budget": 0.26, "threshold": -0.8678473917139679, "operator": "ge"}, {"budget": 0.27, "threshold": -0.882150884082145, "operator": "ge"}, {"budget": 0.28, "threshold": -0.8951216394108034, "operator": "ge"}, {"budget": 0.29, "threshold": -0.9062152880000964, "operator": "ge"}, {"budget": 0.3, "threshold": -0.9159304952800015, "operator": "ge"}, {"budget": 0.31, "threshold": -0.9250210697612712, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9325834116926337, "operator": "ge"}, {"budget": 0.33, "threshold": -0.9386306743254933, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9447639047547287, "operator": "ge"}, {"budget": 0.35, "threshold": -0.948954787680013, "operator": "ge"}, {"budget": 0.36, "threshold": -0.9533479224713509, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9572778418119923, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9606159286740565, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9633752667746842, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9664239135879495, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9689467679289253, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9713314688165889, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9736597659788016, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9755012397330728, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9773796154594315, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9788176550519293, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9803178764832525, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9816381658305964, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9829309389564583, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9840894607822125, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9852617947894589, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9862805763856013, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9872187217846237, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9879920430992157, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9888077498638421, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9893871364409534, "operator": "ge"}, {"budget": 0.57, "threshold": -0.990030300870591, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9906115580757672, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9911561687702576, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9916197308097264, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9920956384958399, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9925643818609496, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9929686766380619, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9933253380761189, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9936756293177081, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9940530911678453, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9943694837507328, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9947189545842993, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9950854011509436, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9953663735651463, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9956178345316601, "operator": "ge"}, {"budget": 0.72, "threshold": -0.995866933078281, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9961127215406664, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9963436555155712, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9965950959161425, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9968176956334711, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9970263803509212, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9972268351962982, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9973943540522145, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9975499194938481, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9976950540766907, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9978371136165838, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9979697965005886, "operator": "ge"}, {"budget": 0.84, "threshold": -0.998100677977345, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9982559284378092, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9984038809220929, "operator": "gt"}, {"budget": 0.87, "threshold": -0.9985269268919423, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9986397607679298, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9987580838523349, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9988632449231722, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9989696674633073, "operator": "ge"}, {"budget": 0.92, "threshold": -0.999070451301977, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9991829446084922, "operator": "ge"}, {"budget": 0.94, "threshold": -0.999286621979455, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9993885376736794, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9994948861829562, "operator": "ge"}, {"budget": 0.97, "threshold": -0.999585099996506, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9996947440795529, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9997986768150976, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.39531941808981, 75.60615644107105, 75.6834633494975, 75.78185396022208, 75.95052357860708, 76.02783048703353, 76.22461170848268, 76.47761613606015, 76.83603907512826, 77.15932251036615, 77.51071754866821, 77.80588938084195, 78.11511701454775, 78.3962330451894, 78.82493499191791, 79.17633003021997, 79.38013915243516, 79.72450628997119, 79.95642701525054, 80.14618033593366, 80.32890575585073, 80.53974277883196, 80.63813338955654, 80.91924942019818, 81.1090027408813, 81.23550495467005, 81.50959308454566, 81.57284419144003, 81.72043010752688, 81.81882071825146, 82.00857403893457, 82.17021575655352, 82.28969006957621, 82.35996907723663, 82.423220184131, 82.43727598566308, 82.44430388642913, 82.47944339025933, 82.46538758872724, 82.47944339025933, 82.52161079485558, 82.57783400098391, 82.64108510787828, 82.71136411553869, 82.71839201630473, 82.71839201630473, 82.71839201630473, 82.76055942090098, 82.7465036193689, 82.71136411553869, 82.69730831400662, 82.66919671094244, 82.68325251247452, 82.65514090941036, 82.67622461170848, 82.60594560404807, 82.6270293063462, 82.591889802516, 82.59891770328203, 82.60594560404807, 82.61297350481411, 82.58486190174995, 82.58486190174995, 82.57783400098391, 82.46538758872724, 82.41619228336496, 82.38105277953476, 82.38105277953476, 82.31077377187435, 82.33185747417247, 82.31077377187435, 82.2686063672781, 82.19832735961768, 82.17021575655352, 82.12804835195728, 82.0647972450629, 82.05074144353081, 81.99451823740249, 81.9453229320402, 81.89612762667792, 81.85396022208167, 81.75556961135709, 81.7063743059948, 81.62906739756835, 81.60095579450419, 81.50256518377961, 81.47445358071545, 81.41120247382108, 81.36200716845879, 81.31983976386253, 81.29875606156442, 81.27064445850026, 81.18630964930776, 81.13711434394546, 81.03169583245484, 81.01764003092276, 80.91924942019818, 80.85599831330381, 80.74355190104716, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99142596106543, 18.989387869843277, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.98636587251389, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 62, 110, 138, 168, 204, 231, 273, 322, 377, 435, 493, 547, 598, 651, 718, 777, 821, 878, 924, 968, 1009, 1051, 1080, 1131, 1169, 1204, 1255, 1279, 1311, 1340, 1378, 1409, 1437, 1462, 1482, 1502, 1520, 1540, 1560, 1578, 1597, 1612, 1636, 1657, 1670, 1686, 1699, 1714, 1726, 1738, 1746, 1754, 1768, 1775, 1785, 1791, 1801, 1808, 1816, 1826, 1834, 1839, 1846, 1853, 1854, 1856, 1859, 1869, 1870, 1877, 1883, 1886, 1890, 1892, 1897, 1899, 1905, 1908, 1913, 1915, 1915, 1915, 1919, 1919, 1924, 1926, 1929, 1931, 1934, 1937, 1943, 1945, 1948, 1951, 1951, 1957, 1958, 1960, 1961, 1964], "harmed": [0, 10, 28, 45, 61, 73, 89, 103, 116, 120, 132, 140, 152, 159, 172, 178, 187, 202, 210, 223, 240, 255, 267, 282, 293, 304, 321, 333, 348, 359, 374, 385, 393, 404, 419, 430, 448, 465, 480, 502, 518, 531, 538, 553, 564, 576, 592, 605, 614, 628, 645, 655, 667, 679, 690, 697, 713, 720, 732, 739, 748, 755, 764, 771, 779, 796, 805, 813, 823, 834, 838, 847, 856, 870, 876, 887, 898, 906, 917, 929, 938, 944, 958, 969, 980, 989, 1005, 1012, 1023, 1033, 1042, 1051, 1057, 1072, 1082, 1097, 1105, 1120, 1131, 1148, 1171], "exact_score": [75.02986857825567, 75.39531941808981, 75.60615644107105, 75.6834633494975, 75.78185396022208, 75.95052357860708, 76.02783048703353, 76.22461170848268, 76.47761613606015, 76.83603907512826, 77.15932251036615, 77.51071754866821, 77.80588938084195, 78.11511701454775, 78.3962330451894, 78.82493499191791, 79.17633003021997, 79.38013915243516, 79.72450628997119, 79.95642701525054, 80.14618033593366, 80.32890575585073, 80.53974277883196, 80.63813338955654, 80.91924942019818, 81.1090027408813, 81.23550495467005, 81.50959308454566, 81.57284419144003, 81.72043010752688, 81.81882071825146, 82.00857403893457, 82.17021575655352, 82.28969006957621, 82.35996907723663, 82.423220184131, 82.43727598566308, 82.44430388642913, 82.47944339025933, 82.46538758872724, 82.47944339025933, 82.52161079485558, 82.57783400098391, 82.64108510787828, 82.71136411553869, 82.71839201630473, 82.71839201630473, 82.71839201630473, 82.76055942090098, 82.7465036193689, 82.71136411553869, 82.69730831400662, 82.66919671094244, 82.68325251247452, 82.65514090941036, 82.67622461170848, 82.60594560404807, 82.6270293063462, 82.591889802516, 82.59891770328203, 82.60594560404807, 82.61297350481411, 82.58486190174995, 82.58486190174995, 82.57783400098391, 82.46538758872724, 82.41619228336496, 82.38105277953476, 82.38105277953476, 82.31077377187435, 82.33185747417247, 82.31077377187435, 82.2686063672781, 82.19832735961768, 82.17021575655352, 82.12804835195728, 82.0647972450629, 82.05074144353081, 81.99451823740249, 81.9453229320402, 81.89612762667792, 81.85396022208167, 81.75556961135709, 81.7063743059948, 81.62906739756835, 81.60095579450419, 81.50256518377961, 81.47445358071545, 81.41120247382108, 81.36200716845879, 81.31983976386253, 81.29875606156442, 81.27064445850026, 81.18630964930776, 81.13711434394546, 81.03169583245484, 81.01764003092276, 80.91924942019818, 80.85599831330381, 80.74355190104716, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.06951654784656, "threshold_area100": 81.069330241057, "threshold_area25": 78.03007941527866}, "test": {"score": [66.95119530385956, 68.0997079091399, 68.91359215041261, 69.48926636984942, 70.03374641976009, 70.70867481496185, 71.25599069846581, 71.56226072654056, 71.93375492726086, 72.33927913110058, 72.6909224966679, 73.00286419192922, 73.33465672234352, 73.47928423560106, 73.71465842384369, 73.95286844567961, 74.1768992995491, 74.3243626463999, 74.45764682528429, 74.59660267135524, 74.76108101976577, 74.90003686583671, 75.00212687519496, 75.01630604316138, 75.10988855173977, 75.24600856421745, 75.4019794118481, 75.4785469188668, 75.55795025947877, 75.6827269375833, 75.72526444148258, 75.83586195162067, 75.88407112270652, 75.91810112582593, 75.97198196409835, 76.00884780081105, 76.00884780081105, 76.00034030003118, 76.01451946799762, 76.04854947111703, 76.10526614298273, 76.12511697813572, 76.12511697813572, 76.147803646882, 76.15347531406856, 76.11377364376258, 76.09392280860959, 76.10810197657601, 76.09959447579617, 76.13929614610214, 76.15347531406856, 76.16198281484843, 76.17899781640813, 76.127952811729, 76.10526614298273, 76.04571363752375, 76.04287780393047, 76.02302696877747, 76.00034030003118, 75.9436236281655, 75.94929529535207, 75.96914613050507, 75.97481779769163, 75.92944446019906, 75.93228029379236, 75.92093695941922, 75.89825029067295, 75.88407112270652, 75.89257862348637, 75.88690695629981, 75.88123528911323, 75.85287695318038, 75.83019028443411, 75.81601111646769, 75.7961602813147, 75.80183194850126, 75.7933244477214, 75.77630944616169, 75.76213027819527, 75.72810027507586, 75.71675694070272, 75.68839860476987, 75.6657119360236, 75.6089952641579, 75.58063692822505, 75.56362192666535, 75.50406942120637, 75.47287525168022, 75.41899441340783, 75.36227774154213, 75.33108357201598, 75.28571023452344, 75.20063522672488, 75.13824688767264, 75.06735104784052, 74.99078354082182, 74.91705186739642, 74.81496185803816, 74.7157076822732, 74.63063267447467, 74.47182599325072], "calls": [0.0, 1.945381844993336, 3.7971811814082748, 5.424949663953719, 7.123613986331281, 9.15407083912316, 11.122139352862774, 12.497518645605876, 13.855882936789268, 15.211411394379379, 16.68320902929416, 18.05008082125741, 19.402773445254233, 20.528599381788275, 21.69129115503502, 22.958908771233304, 24.161302214786037, 25.587726512208263, 26.855344128406543, 28.17117091569067, 29.5408785412472, 30.834018659785045, 32.217905453308, 33.43164223123387, 34.71343901539858, 36.14553498000737, 37.753452627399824, 39.15719025607577, 40.611972889430845, 41.876754672035844, 43.2237756288461, 44.51975158097723, 45.861100870600914, 46.99543430791481, 48.37932110143777, 49.357683691121004, 50.4551512917222, 51.61784306496895, 52.641578992144744, 53.537702407622724, 54.59546833791793, 55.4745767518362, 56.373536000907464, 57.524884439781076, 58.50324702946431, 59.561012959759516, 60.36071803306582, 61.30505061962964, 62.1671440319882, 63.04625244590647, 64.07566004026883, 65.0483509627655, 66.02671355244874, 67.05044947962455, 67.86716955449054, 68.75478546918868, 69.46090803391657, 70.30315061112213, 71.03763151178288, 71.76927657885035, 72.4498766412387, 73.18152170830615, 73.94719677849304, 74.59376683776196, 75.22048606187789, 75.78481694694156, 76.4824320108896, 77.01840456002041, 77.70751212318862, 78.41079885432323, 79.00348807531974, 79.68408813770807, 80.36468820009642, 81.05946743045118, 81.69185832175367, 82.46604089272041, 83.1749992910416, 83.85559935342994, 84.62127442361682, 85.25366531491933, 85.79530953123671, 86.33411791396081, 86.90128463261776, 87.47412301846128, 88.02427473555852, 88.70203896435356, 89.42517653064118, 89.980999914925, 90.63324164138048, 91.28264753424268, 91.92354592632505, 92.58996682074697, 93.27907438391514, 94.03624195332218, 94.88132036412104, 95.73774210929302, 96.57431301931203, 97.29461475200635, 98.11984232765222, 98.95641323767121, 100.0], "rescued": [0, 424, 739, 972, 1183, 1435, 1642, 1778, 1931, 2096, 2248, 2395, 2549, 2632, 2753, 2864, 2974, 3077, 3163, 3254, 3346, 3434, 3515, 3569, 3640, 3714, 3808, 3879, 3948, 4019, 4075, 4140, 4191, 4235, 4299, 4335, 4363, 4397, 4428, 4460, 4500, 4531, 4556, 4596, 4622, 4643, 4663, 4691, 4711, 4741, 4777, 4810, 4838, 4865, 4880, 4894, 4913, 4929, 4940, 4951, 4969, 4988, 5009, 5020, 5035, 5050, 5064, 5077, 5092, 5104, 5115, 5124, 5140, 5152, 5159, 5174, 5189, 5204, 5216, 5222, 5232, 5237, 5246, 5249, 5261, 5269, 5278, 5284, 5292, 5300, 5308, 5317, 5321, 5335, 5348, 5361, 5371, 5383, 5390, 5399, 5423], "harmed": [0, 19, 47, 77, 96, 110, 124, 152, 174, 196, 224, 261, 298, 330, 368, 395, 426, 477, 516, 558, 592, 631, 676, 725, 763, 789, 828, 872, 913, 940, 981, 1007, 1041, 1073, 1118, 1141, 1169, 1206, 1232, 1252, 1272, 1296, 1321, 1353, 1377, 1412, 1439, 1462, 1485, 1501, 1532, 1562, 1584, 1629, 1652, 1687, 1707, 1730, 1749, 1780, 1796, 1808, 1827, 1854, 1868, 1887, 1909, 1927, 1939, 1953, 1966, 1985, 2009, 2026, 2040, 2053, 2071, 2092, 2109, 2127, 2141, 2156, 2173, 2196, 2218, 2232, 2262, 2279, 2306, 2334, 2353, 2378, 2412, 2448, 2486, 2526, 2562, 2610, 2652, 2691, 2771], "exact_score": [66.95119530385956, 67.5382128576695, 68.12239457788617, 68.61015795593114, 68.96463715509174, 69.31911635425233, 69.69344638856592, 70.01389558460708, 70.31165811190199, 70.66613731106258, 70.93554150242464, 71.22479652893968, 71.48852905311516, 71.72957490854436, 71.99047159912656, 72.28823412642146, 72.52360831466409, 72.76465417009331, 72.98301335677623, 73.2127158778323, 73.38570172702265, 73.5643592433996, 73.77988259648923, 73.94436094489976, 74.15137679720954, 74.2874968096872, 74.36122848311261, 74.47182599325072, 74.57108016901569, 74.67600601196722, 74.82346935881802, 74.90287269942999, 74.98511187363526, 75.01063437597482, 75.05600771346738, 75.13257522048606, 75.23182939625103, 75.33675523920256, 75.43600941496753, 75.48421858605336, 75.54660692510564, 75.60332359697132, 75.6827269375833, 75.69974193914301, 75.78765278053483, 75.86705612114682, 75.88690695629981, 75.91810112582593, 75.94645946175879, 75.9975044664379, 76.02019113518419, 75.98616113206477, 76.02302696877747, 76.03437030315061, 76.07690780704988, 76.127952811729, 76.10810197657601, 76.11944531094915, 76.13929614610214, 76.11944531094915, 76.11377364376258, 76.08825114142302, 76.10526614298273, 76.14213197969542, 76.14496781328872, 76.15631114766185, 76.1846694835947, 76.13362447891558, 76.09959447579617, 76.0542211383036, 76.01735530159091, 76.00034030003118, 75.9436236281655, 75.96631029691177, 75.96347446331849, 75.92660862660578, 75.93228029379236, 75.88407112270652, 75.88407112270652, 75.88123528911323, 75.83302611802739, 75.8131752828744, 75.81033944928112, 75.7961602813147, 75.77914527975497, 75.7564586110087, 75.7224286078893, 75.64869693446389, 75.58063692822505, 75.53809942432578, 75.47003941808694, 75.37645690950855, 75.32257607123614, 75.24600856421745, 75.13541105407936, 75.05884354706066, 74.95958937129569, 74.85749936193744, 74.7355585174262, 74.62779684088137, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.55076575552991, "threshold_area100": 74.99296713268866, "threshold_area25": 72.64242974222273}, "selected_budget": 48, "selected": {"val": {"score": 82.76055942090098, "calls": 47.99353433129524, "rescued": 1714, "harmed": 614}, "test": {"score": 76.09959447579617, "calls": 62.1671440319882, "rescued": 4711, "harmed": 1485}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.48370417952537537, "operator": "ge"}, {"budget": 0.02, "threshold": 0.321429580450058, "operator": "ge"}, {"budget": 0.03, "threshold": 0.22199803590774536, "operator": "ge"}, {"budget": 0.04, "threshold": 0.13186949491500854, "operator": "ge"}, {"budget": 0.05, "threshold": 0.060304105281829834, "operator": "ge"}, {"budget": 0.06, "threshold": 0.01115468144416809, "operator": "ge"}, {"budget": 0.07, "threshold": -0.0007401704788208008, "operator": "ge"}, {"budget": 0.08, "threshold": -0.0014615381369367242, "operator": "ge"}, {"budget": 0.09, "threshold": -0.0021554972045123577, "operator": "ge"}, {"budget": 0.1, "threshold": -0.002860722364857793, "operator": "ge"}, {"budget": 0.11, "threshold": -0.003684587776660919, "operator": "ge"}, {"budget": 0.12, "threshold": -0.004516770597547293, "operator": "ge"}, {"budget": 0.13, "threshold": -0.0053261276334524155, "operator": "ge"}, {"budget": 0.14, "threshold": -0.006438038311898708, "operator": "ge"}, {"budget": 0.15, "threshold": -0.007454972714185715, "operator": "gt"}, {"budget": 0.16, "threshold": -0.008507009595632553, "operator": "ge"}, {"budget": 0.17, "threshold": -0.009503130801022053, "operator": "ge"}, {"budget": 0.18, "threshold": -0.010840261355042458, "operator": "ge"}, {"budget": 0.19, "threshold": -0.01225803792476654, "operator": "ge"}, {"budget": 0.2, "threshold": -0.01332111842930317, "operator": "ge"}, {"budget": 0.21, "threshold": -0.014902095310389996, "operator": "ge"}, {"budget": 0.22, "threshold": -0.01634080335497856, "operator": "ge"}, {"budget": 0.23, "threshold": -0.017915448173880577, "operator": "ge"}, {"budget": 0.24, "threshold": -0.019506406038999557, "operator": "ge"}, {"budget": 0.25, "threshold": -0.021201731637120247, "operator": "ge"}, {"budget": 0.26, "threshold": -0.023268381133675575, "operator": "ge"}, {"budget": 0.27, "threshold": -0.02557978965342045, "operator": "ge"}, {"budget": 0.28, "threshold": -0.02831880934536457, "operator": "ge"}, {"budget": 0.29, "threshold": -0.030966386198997498, "operator": "ge"}, {"budget": 0.3, "threshold": -0.03366497904062271, "operator": "ge"}, {"budget": 0.31, "threshold": -0.03651263937354088, "operator": "ge"}, {"budget": 0.32, "threshold": -0.03925338760018349, "operator": "ge"}, {"budget": 0.33, "threshold": -0.04223093017935753, "operator": "ge"}, {"budget": 0.34, "threshold": -0.045290231704711914, "operator": "ge"}, {"budget": 0.35, "threshold": -0.048187486827373505, "operator": "ge"}, {"budget": 0.36, "threshold": -0.05118826404213905, "operator": "ge"}, {"budget": 0.37, "threshold": -0.054374199360609055, "operator": "ge"}, {"budget": 0.38, "threshold": -0.05776463821530342, "operator": "ge"}, {"budget": 0.39, "threshold": -0.0614350400865078, "operator": "ge"}, {"budget": 0.4, "threshold": -0.06439229100942612, "operator": "ge"}, {"budget": 0.41, "threshold": -0.06775042414665222, "operator": "ge"}, {"budget": 0.42, "threshold": -0.07157717645168304, "operator": "ge"}, {"budget": 0.43, "threshold": -0.07486411929130554, "operator": "ge"}, {"budget": 0.44, "threshold": -0.0791749358177185, "operator": "ge"}, {"budget": 0.45, "threshold": -0.08305935561656952, "operator": "ge"}, {"budget": 0.46, "threshold": -0.08727102726697922, "operator": "ge"}, {"budget": 0.47, "threshold": -0.09163302928209305, "operator": "ge"}, {"budget": 0.48, "threshold": -0.09544381499290466, "operator": "ge"}, {"budget": 0.49, "threshold": -0.09990990906953812, "operator": "ge"}, {"budget": 0.5, "threshold": -0.10376328229904175, "operator": "ge"}, {"budget": 0.51, "threshold": -0.10883321613073349, "operator": "ge"}, {"budget": 0.52, "threshold": -0.11378997564315796, "operator": "ge"}, {"budget": 0.53, "threshold": -0.11955861747264862, "operator": "ge"}, {"budget": 0.54, "threshold": -0.1249021589756012, "operator": "ge"}, {"budget": 0.55, "threshold": -0.13052284717559814, "operator": "ge"}, {"budget": 0.56, "threshold": -0.13616737723350525, "operator": "ge"}, {"budget": 0.57, "threshold": -0.14193356037139893, "operator": "ge"}, {"budget": 0.58, "threshold": -0.14744330942630768, "operator": "ge"}, {"budget": 0.59, "threshold": -0.15338987112045288, "operator": "ge"}, {"budget": 0.6, "threshold": -0.15923093259334564, "operator": "ge"}, {"budget": 0.61, "threshold": -0.16531001031398773, "operator": "ge"}, {"budget": 0.62, "threshold": -0.17217551171779633, "operator": "ge"}, {"budget": 0.63, "threshold": -0.17927959561347961, "operator": "ge"}, {"budget": 0.64, "threshold": -0.18656867742538452, "operator": "ge"}, {"budget": 0.65, "threshold": -0.19307011365890503, "operator": "ge"}, {"budget": 0.66, "threshold": -0.20040607452392578, "operator": "ge"}, {"budget": 0.67, "threshold": -0.2079729586839676, "operator": "ge"}, {"budget": 0.68, "threshold": -0.21407417953014374, "operator": "ge"}, {"budget": 0.69, "threshold": -0.2226216346025467, "operator": "ge"}, {"budget": 0.7, "threshold": -0.22980806231498718, "operator": "ge"}, {"budget": 0.71, "threshold": -0.23701617121696472, "operator": "ge"}, {"budget": 0.72, "threshold": -0.24520540237426758, "operator": "ge"}, {"budget": 0.73, "threshold": -0.25405171513557434, "operator": "ge"}, {"budget": 0.74, "threshold": -0.2620045840740204, "operator": "ge"}, {"budget": 0.75, "threshold": -0.27179452776908875, "operator": "ge"}, {"budget": 0.76, "threshold": -0.2809945344924927, "operator": "ge"}, {"budget": 0.77, "threshold": -0.2900545597076416, "operator": "ge"}, {"budget": 0.78, "threshold": -0.3003483712673187, "operator": "ge"}, {"budget": 0.79, "threshold": -0.31155529618263245, "operator": "ge"}, {"budget": 0.8, "threshold": -0.32423752546310425, "operator": "ge"}, {"budget": 0.81, "threshold": -0.33521759510040283, "operator": "ge"}, {"budget": 0.82, "threshold": -0.34755051136016846, "operator": "ge"}, {"budget": 0.83, "threshold": -0.36092567443847656, "operator": "ge"}, {"budget": 0.84, "threshold": -0.3737483620643616, "operator": "ge"}, {"budget": 0.85, "threshold": -0.38795119524002075, "operator": "ge"}, {"budget": 0.86, "threshold": -0.40265893936157227, "operator": "ge"}, {"budget": 0.87, "threshold": -0.41643452644348145, "operator": "ge"}, {"budget": 0.88, "threshold": -0.4334578514099121, "operator": "ge"}, {"budget": 0.89, "threshold": -0.453862726688385, "operator": "ge"}, {"budget": 0.9, "threshold": -0.47606372833251953, "operator": "ge"}, {"budget": 0.91, "threshold": -0.498926043510437, "operator": "ge"}, {"budget": 0.92, "threshold": -0.5254952907562256, "operator": "ge"}, {"budget": 0.93, "threshold": -0.5567249059677124, "operator": "ge"}, {"budget": 0.94, "threshold": -0.5899993181228638, "operator": "ge"}, {"budget": 0.95, "threshold": -0.6330450773239136, "operator": "ge"}, {"budget": 0.96, "threshold": -0.6807356476783752, "operator": "ge"}, {"budget": 0.97, "threshold": -0.7366846203804016, "operator": "ge"}, {"budget": 0.98, "threshold": -0.8011246919631958, "operator": "ge"}, {"budget": 0.99, "threshold": -0.8786851167678833, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.39531941808981, 75.59210063953897, 75.72563075409376, 75.78888186098813, 75.95755147937311, 76.03485838779956, 76.0910815939279, 76.06999789162977, 76.0418862885656, 76.0418862885656, 76.0418862885656, 76.04891418933165, 76.05594209009769, 76.06999789162977, 76.0910815939279, 76.08405369316186, 76.09810949469393, 76.10513739545998, 76.12622109775809, 76.12622109775809, 76.18244430388643, 76.1754164031204, 76.21055590695059, 76.21055590695059, 76.21758380771664, 76.20352800618456, 76.18947220465247, 76.21758380771664, 76.2456954107808, 76.26677911307893, 76.32300231920726, 76.39328132686767, 76.43544873146392, 76.51275563989036, 76.56195094525266, 76.57600674678474, 76.59006254831682, 76.68845315904139, 76.76576006746785, 76.8079274720641, 76.90631808278867, 76.96956918968304, 77.01173659427928, 76.98362499121512, 77.02579239581138, 77.06795980040762, 77.13121090730199, 77.20851781572844, 77.31393632721905, 77.34204793028323, 77.4052990371776, 77.46152224330592, 77.58099655632863, 77.63019186169092, 77.67235926628716, 77.69344296858527, 77.81994518237403, 77.89725209080048, 77.9183357930986, 78.0026706022911, 78.01672640382318, 78.06592170918547, 78.17134022067609, 78.26270293063462, 78.28378663293275, 78.3118982359969, 78.31892613676294, 78.45245625131774, 78.61409796893668, 78.64923747276688, 78.7195164804273, 78.81790709115187, 78.82493499191791, 78.9444093049406, 78.9444093049406, 79.0287441141331, 79.09199522102747, 79.09902312179352, 79.12713472485768, 79.2676927401785, 79.4363623585635, 79.520697167756, 79.60503197694848, 79.68936678614098, 79.8439806029939, 79.8791201068241, 79.90723170988826, 80.01265022137888, 80.12509663363554, 80.19537564129595, 80.3078220535526, 80.38512896197906, 80.37810106121302, 80.44135216810739, 80.546770679598, 80.69435659568487, 80.75057980181319, 80.69435659568487, 80.72246819874903, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.990512333965844, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 63, 107, 141, 168, 203, 231, 245, 245, 245, 246, 246, 248, 251, 256, 260, 261, 265, 267, 272, 274, 283, 285, 291, 295, 298, 300, 307, 312, 320, 327, 339, 350, 361, 374, 386, 398, 405, 420, 439, 452, 471, 485, 496, 504, 516, 530, 547, 564, 586, 599, 615, 633, 658, 674, 691, 708, 736, 755, 769, 791, 806, 822, 849, 874, 895, 914, 931, 963, 991, 1013, 1040, 1068, 1092, 1121, 1144, 1179, 1202, 1227, 1250, 1284, 1321, 1352, 1387, 1418, 1459, 1496, 1525, 1557, 1595, 1625, 1659, 1693, 1722, 1754, 1794, 1836, 1872, 1898, 1937, 1964], "harmed": [0, 11, 27, 42, 60, 71, 88, 94, 97, 101, 102, 102, 103, 105, 108, 109, 111, 113, 114, 116, 118, 119, 122, 123, 127, 129, 133, 142, 143, 147, 151, 155, 156, 161, 163, 168, 178, 183, 184, 192, 199, 204, 209, 214, 226, 232, 240, 248, 254, 261, 270, 277, 287, 295, 304, 315, 329, 339, 347, 358, 368, 381, 390, 402, 414, 432, 447, 463, 476, 481, 498, 515, 529, 552, 564, 587, 610, 624, 648, 667, 681, 694, 713, 736, 755, 774, 806, 831, 848, 870, 890, 908, 931, 961, 984, 1009, 1030, 1058, 1092, 1127, 1171], "exact_score": [75.02986857825567, 75.39531941808981, 75.59210063953897, 75.72563075409376, 75.78888186098813, 75.95755147937311, 76.03485838779956, 76.0910815939279, 76.06999789162977, 76.0418862885656, 76.0418862885656, 76.0418862885656, 76.04891418933165, 76.05594209009769, 76.06999789162977, 76.0910815939279, 76.08405369316186, 76.09810949469393, 76.10513739545998, 76.12622109775809, 76.12622109775809, 76.18244430388643, 76.1754164031204, 76.21055590695059, 76.21055590695059, 76.21758380771664, 76.20352800618456, 76.18947220465247, 76.21758380771664, 76.2456954107808, 76.26677911307893, 76.32300231920726, 76.39328132686767, 76.43544873146392, 76.51275563989036, 76.56195094525266, 76.57600674678474, 76.59006254831682, 76.68845315904139, 76.76576006746785, 76.8079274720641, 76.90631808278867, 76.96956918968304, 77.01173659427928, 76.98362499121512, 77.02579239581138, 77.06795980040762, 77.13121090730199, 77.20851781572844, 77.31393632721905, 77.34204793028323, 77.4052990371776, 77.46152224330592, 77.58099655632863, 77.63019186169092, 77.67235926628716, 77.69344296858527, 77.81994518237403, 77.89725209080048, 77.9183357930986, 78.0026706022911, 78.01672640382318, 78.06592170918547, 78.17134022067609, 78.26270293063462, 78.28378663293275, 78.3118982359969, 78.31892613676294, 78.45245625131774, 78.61409796893668, 78.64923747276688, 78.7195164804273, 78.81790709115187, 78.82493499191791, 78.9444093049406, 78.9444093049406, 79.0287441141331, 79.09199522102747, 79.09902312179352, 79.12713472485768, 79.2676927401785, 79.4363623585635, 79.520697167756, 79.60503197694848, 79.68936678614098, 79.8439806029939, 79.8791201068241, 79.90723170988826, 80.01265022137888, 80.12509663363554, 80.19537564129595, 80.3078220535526, 80.38512896197906, 80.37810106121302, 80.44135216810739, 80.546770679598, 80.69435659568487, 80.75057980181319, 80.69435659568487, 80.72246819874903, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 77.6938724267142, "threshold_area100": 77.6936889451121, "threshold_area25": 75.96809333052218}, "test": {"score": [66.95119530385956, 68.13940957944588, 68.96180132149846, 69.5289680401554, 70.16419476505118, 70.75121231886112, 71.28718486799195, 71.35240904063748, 71.35808070782407, 71.36658820860391, 71.37509570938377, 71.38927487735019, 71.39778237813005, 71.4062898789099, 71.4062898789099, 71.41763321328304, 71.41479737968976, 71.41479737968976, 71.4261407140629, 71.4630065507756, 71.47434988514874, 71.4630065507756, 71.45733488358903, 71.46867821796216, 71.4800215523353, 71.51405155545473, 71.55091739216742, 71.56509656013384, 71.58778322888013, 71.60196239684655, 71.6104698976264, 71.67285823667868, 71.67852990386524, 71.72106740776452, 71.75793324447721, 71.79479908118991, 71.85151575305561, 71.88270992258174, 71.88838158976831, 71.93091909366758, 71.95076992882058, 71.99614326631313, 72.01882993505941, 72.03017326943255, 72.06703910614524, 72.09823327567139, 72.17196494909679, 72.2059949522162, 72.24286078892891, 72.2825624592349, 72.34211496469388, 72.39599580296628, 72.42718997249241, 72.50942914669767, 72.61435498964921, 72.68525082948132, 72.75047500212686, 72.83555000992543, 72.90644584975755, 72.91495335053739, 72.9319683520971, 72.99152085755608, 73.0822675325412, 73.1730142075263, 73.26092504891812, 73.34883589030996, 73.42256756373536, 73.51615007231376, 73.63241924963843, 73.71465842384369, 73.72316592462353, 73.79122593086238, 73.8791367722542, 73.93018177693332, 73.97839094801917, 74.05495845503786, 74.12301846127669, 74.16272013158267, 74.24212347219465, 74.32152681280662, 74.36406431670589, 74.44063182372457, 74.46615432606414, 74.52003516433655, 74.57675183620225, 74.60794600572838, 74.69302101352693, 74.76391685335905, 74.78660352210532, 74.87167852990386, 74.94257436973598, 74.9198877009897, 74.92839520176956, 74.88018603068372, 74.85182769475087, 74.83764852678445, 74.81779769163145, 74.8092901908516, 74.69302101352693, 74.61928934010153, 74.47182599325072], "calls": [0.0, 2.0701585230978647, 3.9247936931060887, 5.47315883503956, 7.327794005047784, 9.284519184414258, 11.195871026288177, 12.001247766781045, 12.6790119955761, 13.382298726710717, 14.079913790658763, 14.868275529591923, 15.662308935711652, 16.314550662167143, 17.290077418257095, 18.118140827496244, 18.86396506253013, 19.70053597254913, 20.800839406743613, 21.694126988628305, 22.414428721322633, 23.492045486770834, 24.246377222584577, 25.116978135722995, 25.86847403794345, 26.78161245498114, 27.8677367212092, 28.89714431557156, 30.164761931769846, 31.23670703003148, 32.27178629158041, 33.3522388906219, 34.25686980687973, 35.34299407310779, 36.32419249638431, 37.288375918101124, 38.079573490627574, 38.978532739698835, 39.90868615829623, 40.91257125031903, 41.72078382440518, 42.65093724300258, 43.697359838924655, 44.536766582536934, 45.57184584408587, 46.41408842129144, 47.34424183988884, 48.450216941269886, 49.283952017695604, 50.2311204378527, 51.02798967756572, 52.074412273487795, 52.96769985537249, 54.08785412471996, 55.00666420894422, 56.0133851345603, 56.932195218784564, 57.896378640501375, 58.78115872160622, 59.71131214020361, 60.655644726767434, 61.651022318010384, 62.68610157955932, 63.67580750361569, 64.65700592689221, 65.58432351189633, 66.58820860391911, 67.60910869750163, 68.40881377080792, 69.42404219720387, 70.30031477752885, 71.21345319456654, 72.22300995377591, 73.15316337237331, 73.98689844879902, 74.97093270566883, 75.87556362192667, 76.66392536085982, 77.47497376853926, 78.50154552930833, 79.54229645804384, 80.35334486572327, 81.30051328588038, 82.20230836854493, 83.07007344809007, 83.9747043643479, 84.83679777670646, 85.6308311828262, 86.60068627172957, 87.66979553639793, 88.73890480106627, 89.76547656183536, 90.89413833196268, 91.96324759663102, 93.0862376995718, 94.25176530641181, 95.39177041091229, 96.55729801775232, 97.6037206136744, 98.83163655956668, 100.0], "rescued": [0, 437, 754, 981, 1223, 1448, 1652, 1682, 1685, 1689, 1692, 1698, 1703, 1706, 1709, 1715, 1719, 1724, 1729, 1743, 1748, 1751, 1761, 1771, 1780, 1795, 1812, 1822, 1841, 1856, 1871, 1899, 1916, 1939, 1961, 1982, 2014, 2036, 2061, 2086, 2107, 2139, 2162, 2183, 2213, 2241, 2277, 2312, 2341, 2372, 2413, 2458, 2490, 2541, 2592, 2644, 2696, 2745, 2784, 2821, 2864, 2916, 2981, 3043, 3096, 3153, 3217, 3278, 3348, 3408, 3454, 3513, 3571, 3618, 3672, 3737, 3796, 3847, 3907, 3982, 4062, 4121, 4177, 4236, 4297, 4350, 4418, 4479, 4547, 4634, 4710, 4779, 4855, 4919, 4995, 5068, 5133, 5212, 5278, 5349, 5423], "harmed": [0, 18, 45, 72, 90, 108, 123, 130, 131, 132, 132, 133, 135, 135, 138, 140, 145, 150, 151, 152, 153, 160, 172, 178, 183, 186, 190, 195, 206, 216, 228, 234, 249, 257, 266, 274, 286, 297, 320, 330, 344, 360, 375, 392, 409, 426, 436, 459, 475, 492, 512, 538, 559, 581, 595, 622, 651, 670, 684, 718, 755, 786, 819, 849, 871, 897, 935, 963, 992, 1023, 1066, 1101, 1128, 1157, 1194, 1232, 1267, 1304, 1336, 1383, 1448, 1480, 1527, 1567, 1608, 1650, 1688, 1724, 1784, 1841, 1892, 1969, 2042, 2123, 2209, 2287, 2359, 2441, 2548, 2645, 2771], "exact_score": [66.95119530385956, 67.56657119360236, 68.0968720755466, 68.58747128718487, 68.9901596574313, 69.35598219096504, 69.71613305731219, 70.04508975413323, 70.37404645095425, 70.67464481184244, 70.93837733601792, 71.23330402971953, 71.35240904063748, 71.36658820860391, 71.37509570938377, 71.38927487735019, 71.40061821172333, 71.40912571250318, 71.41763321328304, 71.41479737968976, 71.42046904687632, 71.43464821484275, 71.4630065507756, 71.46584238436888, 71.45166321640247, 71.46867821796216, 71.48285738592858, 71.53106655701443, 71.54808155857414, 71.57076822732041, 71.58211156169357, 71.60763406403312, 71.60763406403312, 71.65300740152568, 71.67285823667868, 71.70688823979809, 71.74942574369736, 71.79196324759663, 71.83166491790261, 71.88554575617503, 71.89405325695488, 71.93659076085414, 71.97629243116013, 71.99614326631313, 72.03017326943255, 72.05853160536539, 72.07271077333182, 72.14644244675722, 72.20315911862292, 72.22868162096248, 72.28539829282818, 72.34495079828716, 72.38465246859313, 72.43569747327227, 72.49524997873125, 72.61435498964921, 72.68241499588804, 72.75614666931345, 72.8582366786717, 72.91778918413068, 72.91495335053739, 72.9518191872501, 73.0057000255225, 73.0992825341009, 73.20704421064572, 73.28928338485098, 73.38570172702265, 73.46794090122792, 73.57003091058615, 73.66361341916456, 73.71465842384369, 73.78271843008251, 73.85361426991463, 73.91033094178033, 73.98406261520574, 74.05495845503786, 74.12585429486998, 74.1768992995491, 74.24779513938122, 74.36973598389245, 74.4179451549783, 74.46048265887757, 74.51436349714999, 74.57958766979553, 74.60511017213508, 74.71003601508663, 74.76675268695233, 74.81779769163145, 74.883021864277, 74.92272353458299, 74.94257436973598, 74.92555936817628, 74.883021864277, 74.83481269319117, 74.82063352522474, 74.83481269319117, 74.80361852366502, 74.76108101976577, 74.65048350962765, 74.59660267135524, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 72.53176643283197, "threshold_area100": 72.6580268269858, "threshold_area25": 70.85018291126676}, "selected_budget": 97, "selected": {"val": {"score": 80.75057980181319, "calls": 96.99908637290041, "rescued": 1872, "harmed": 1058}, "test": {"score": 74.8092901908516, "calls": 96.55729801775232, "rescued": 5212, "harmed": 2441}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 5.658199310302734, "operator": "ge"}, {"budget": 0.02, "threshold": 3.3951127529144287, "operator": "ge"}, {"budget": 0.03, "threshold": 2.252560615539551, "operator": "gt"}, {"budget": 0.04, "threshold": 1.521453857421875, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7730463147163391, "operator": "gt"}, {"budget": 0.06, "threshold": 0.14381206035614014, "operator": "ge"}, {"budget": 0.07, "threshold": -0.34007441997528076, "operator": "gt"}, {"budget": 0.08, "threshold": -0.7090458869934082, "operator": "ge"}, {"budget": 0.09, "threshold": -1.0348577499389648, "operator": "ge"}, {"budget": 0.1, "threshold": -1.2642632722854614, "operator": "ge"}, {"budget": 0.11, "threshold": -1.4735485315322876, "operator": "ge"}, {"budget": 0.12, "threshold": -1.6671316623687744, "operator": "ge"}, {"budget": 0.13, "threshold": -1.8471906185150146, "operator": "ge"}, {"budget": 0.14, "threshold": -2.015871524810791, "operator": "ge"}, {"budget": 0.15, "threshold": -2.1809985637664795, "operator": "ge"}, {"budget": 0.16, "threshold": -2.328176259994507, "operator": "ge"}, {"budget": 0.17, "threshold": -2.4874658584594727, "operator": "ge"}, {"budget": 0.18, "threshold": -2.6481356620788574, "operator": "ge"}, {"budget": 0.19, "threshold": -2.807546377182007, "operator": "ge"}, {"budget": 0.2, "threshold": -2.9325482845306396, "operator": "ge"}, {"budget": 0.21, "threshold": -3.0585689544677734, "operator": "ge"}, {"budget": 0.22, "threshold": -3.19926118850708, "operator": "ge"}, {"budget": 0.23, "threshold": -3.3282666206359863, "operator": "ge"}, {"budget": 0.24, "threshold": -3.4512100219726562, "operator": "ge"}, {"budget": 0.25, "threshold": -3.5860047340393066, "operator": "ge"}, {"budget": 0.26, "threshold": -3.705479145050049, "operator": "ge"}, {"budget": 0.27, "threshold": -3.811220645904541, "operator": "ge"}, {"budget": 0.28, "threshold": -3.922506332397461, "operator": "ge"}, {"budget": 0.29, "threshold": -4.031318664550781, "operator": "ge"}, {"budget": 0.3, "threshold": -4.126918792724609, "operator": "ge"}, {"budget": 0.31, "threshold": -4.225854396820068, "operator": "ge"}, {"budget": 0.32, "threshold": -4.324596405029297, "operator": "ge"}, {"budget": 0.33, "threshold": -4.415475368499756, "operator": "ge"}, {"budget": 0.34, "threshold": -4.501857757568359, "operator": "ge"}, {"budget": 0.35, "threshold": -4.585749626159668, "operator": "ge"}, {"budget": 0.36, "threshold": -4.680737018585205, "operator": "ge"}, {"budget": 0.37, "threshold": -4.764990329742432, "operator": "ge"}, {"budget": 0.38, "threshold": -4.835890293121338, "operator": "ge"}, {"budget": 0.39, "threshold": -4.9141411781311035, "operator": "ge"}, {"budget": 0.4, "threshold": -5.006046772003174, "operator": "ge"}, {"budget": 0.41, "threshold": -5.078083038330078, "operator": "ge"}, {"budget": 0.42, "threshold": -5.145503997802734, "operator": "ge"}, {"budget": 0.43, "threshold": -5.216470718383789, "operator": "ge"}, {"budget": 0.44, "threshold": -5.283779144287109, "operator": "ge"}, {"budget": 0.45, "threshold": -5.344618320465088, "operator": "ge"}, {"budget": 0.46, "threshold": -5.410429000854492, "operator": "ge"}, {"budget": 0.47, "threshold": -5.4661030769348145, "operator": "ge"}, {"budget": 0.48, "threshold": -5.530414581298828, "operator": "ge"}, {"budget": 0.49, "threshold": -5.594730377197266, "operator": "ge"}, {"budget": 0.5, "threshold": -5.652868270874023, "operator": "ge"}, {"budget": 0.51, "threshold": -5.711558818817139, "operator": "ge"}, {"budget": 0.52, "threshold": -5.769279956817627, "operator": "ge"}, {"budget": 0.53, "threshold": -5.832253932952881, "operator": "ge"}, {"budget": 0.54, "threshold": -5.888298034667969, "operator": "ge"}, {"budget": 0.55, "threshold": -5.942006587982178, "operator": "ge"}, {"budget": 0.56, "threshold": -6.0071282386779785, "operator": "ge"}, {"budget": 0.57, "threshold": -6.0615339279174805, "operator": "gt"}, {"budget": 0.58, "threshold": -6.118642330169678, "operator": "ge"}, {"budget": 0.59, "threshold": -6.170944690704346, "operator": "ge"}, {"budget": 0.6, "threshold": -6.220767974853516, "operator": "ge"}, {"budget": 0.61, "threshold": -6.275781631469727, "operator": "ge"}, {"budget": 0.62, "threshold": -6.3308563232421875, "operator": "ge"}, {"budget": 0.63, "threshold": -6.3823418617248535, "operator": "ge"}, {"budget": 0.64, "threshold": -6.4325995445251465, "operator": "ge"}, {"budget": 0.65, "threshold": -6.482461452484131, "operator": "ge"}, {"budget": 0.66, "threshold": -6.533106803894043, "operator": "ge"}, {"budget": 0.67, "threshold": -6.586996555328369, "operator": "ge"}, {"budget": 0.68, "threshold": -6.638925075531006, "operator": "ge"}, {"budget": 0.69, "threshold": -6.695315361022949, "operator": "ge"}, {"budget": 0.7, "threshold": -6.743926048278809, "operator": "ge"}, {"budget": 0.71, "threshold": -6.798994541168213, "operator": "ge"}, {"budget": 0.72, "threshold": -6.853353023529053, "operator": "ge"}, {"budget": 0.73, "threshold": -6.904611110687256, "operator": "ge"}, {"budget": 0.74, "threshold": -6.96276330947876, "operator": "ge"}, {"budget": 0.75, "threshold": -7.025025844573975, "operator": "ge"}, {"budget": 0.76, "threshold": -7.076309680938721, "operator": "ge"}, {"budget": 0.77, "threshold": -7.138849258422852, "operator": "ge"}, {"budget": 0.78, "threshold": -7.19195032119751, "operator": "ge"}, {"budget": 0.79, "threshold": -7.248815059661865, "operator": "ge"}, {"budget": 0.8, "threshold": -7.308775901794434, "operator": "ge"}, {"budget": 0.81, "threshold": -7.371160984039307, "operator": "ge"}, {"budget": 0.82, "threshold": -7.423250198364258, "operator": "ge"}, {"budget": 0.83, "threshold": -7.487290382385254, "operator": "ge"}, {"budget": 0.84, "threshold": -7.555489540100098, "operator": "ge"}, {"budget": 0.85, "threshold": -7.62477970123291, "operator": "ge"}, {"budget": 0.86, "threshold": -7.698545932769775, "operator": "ge"}, {"budget": 0.87, "threshold": -7.771554946899414, "operator": "ge"}, {"budget": 0.88, "threshold": -7.855341911315918, "operator": "ge"}, {"budget": 0.89, "threshold": -7.9328460693359375, "operator": "ge"}, {"budget": 0.9, "threshold": -8.040260314941406, "operator": "ge"}, {"budget": 0.91, "threshold": -8.127034187316895, "operator": "ge"}, {"budget": 0.92, "threshold": -8.231775283813477, "operator": "ge"}, {"budget": 0.93, "threshold": -8.356995582580566, "operator": "ge"}, {"budget": 0.94, "threshold": -8.492548942565918, "operator": "ge"}, {"budget": 0.95, "threshold": -8.64663314819336, "operator": "ge"}, {"budget": 0.96, "threshold": -8.847848892211914, "operator": "ge"}, {"budget": 0.97, "threshold": -9.075157165527344, "operator": "ge"}, {"budget": 0.98, "threshold": -9.373311996459961, "operator": "ge"}, {"budget": 0.99, "threshold": -10.055503845214844, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.4585705249842, 75.74671445639187, 75.86618876941458, 75.95755147937311, 75.98566308243728, 76.07702579239582, 76.25272331154684, 76.52681144142245, 76.8430669758943, 77.13823880806802, 77.51774544943424, 77.89022419003444, 78.19242392297421, 78.41028884672149, 78.72654438119334, 79.02171621336707, 79.1974137325181, 79.46447396162766, 79.75261789303535, 80.04076182444304, 80.27971045048844, 80.52568697729988, 80.64516129032259, 80.80680300794153, 80.99655632862464, 81.11603064164734, 81.23550495467005, 81.27767235926629, 81.35497926769274, 81.44634197765129, 81.53067678684377, 81.56581629067398, 81.62203949680232, 81.66420690139856, 81.73448590905896, 81.87504392437978, 81.99451823740249, 81.99451823740249, 82.03668564199873, 82.0647972450629, 82.11399255042518, 82.12102045119123, 82.19129945885165, 82.24049476421393, 82.23346686344789, 82.1842715580856, 82.19832735961768, 82.24752266497997, 82.20535526038373, 82.20535526038373, 82.16318785578748, 82.22643896268184, 82.1842715580856, 82.16318785578748, 82.19832735961768, 82.24752266497997, 82.23346686344789, 82.21238316114976, 82.14913205425539, 82.11399255042518, 82.11399255042518, 82.09993674889311, 82.10696464965915, 82.05776934429686, 82.0647972450629, 82.01560193970062, 82.02262984046665, 82.0647972450629, 82.00857403893457, 81.98749033663645, 81.96640663433833, 81.93126713050812, 81.90315552744394, 81.83990442054959, 81.86801602361375, 81.81882071825146, 81.81179281748541, 81.76259751212312, 81.69231850446272, 81.64312319910043, 81.60095579450419, 81.54473258837585, 81.4814814814815, 81.44634197765129, 81.3830908707569, 81.34092346616066, 81.29172816079837, 81.24253285543608, 81.17225384777566, 81.10197484011525, 81.13008644317942, 81.06683533628505, 81.07386323705109, 81.03872373322089, 80.92627732096422, 80.89816571790006, 80.8208588094736, 80.83491461100569, 80.76463560334528, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.9898095438892405, 5.994799353433129, 6.992761262210978, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.98924731182796, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 65, 117, 150, 179, 202, 234, 280, 331, 384, 436, 496, 558, 613, 657, 711, 761, 806, 858, 904, 956, 999, 1045, 1074, 1111, 1148, 1180, 1208, 1231, 1257, 1278, 1303, 1325, 1349, 1374, 1397, 1429, 1457, 1471, 1490, 1506, 1524, 1542, 1562, 1574, 1586, 1599, 1613, 1628, 1633, 1646, 1651, 1665, 1672, 1682, 1695, 1707, 1713, 1717, 1724, 1729, 1737, 1743, 1751, 1754, 1762, 1767, 1775, 1789, 1792, 1799, 1802, 1806, 1809, 1813, 1821, 1825, 1829, 1833, 1835, 1841, 1846, 1852, 1854, 1858, 1864, 1871, 1875, 1880, 1882, 1887, 1896, 1902, 1910, 1915, 1918, 1926, 1934, 1943, 1952, 1964], "harmed": [0, 4, 15, 31, 47, 66, 85, 106, 118, 126, 136, 142, 151, 163, 176, 185, 193, 213, 227, 232, 243, 252, 263, 275, 289, 299, 314, 325, 342, 357, 365, 378, 395, 411, 430, 443, 455, 466, 480, 493, 505, 516, 533, 543, 548, 561, 581, 593, 601, 612, 625, 636, 641, 654, 667, 675, 680, 688, 695, 711, 721, 729, 737, 744, 754, 761, 773, 780, 788, 799, 809, 815, 824, 831, 844, 848, 859, 864, 875, 887, 900, 911, 925, 936, 945, 960, 973, 984, 996, 1008, 1023, 1028, 1043, 1050, 1060, 1079, 1091, 1110, 1117, 1136, 1171], "exact_score": [75.02986857825567, 75.4585705249842, 75.74671445639187, 75.86618876941458, 75.95755147937311, 75.99269098320332, 76.07702579239582, 76.25272331154684, 76.52681144142245, 76.8430669758943, 77.13823880806802, 77.51774544943424, 77.89022419003444, 78.19242392297421, 78.41028884672149, 78.72654438119334, 79.02171621336707, 79.1974137325181, 79.46447396162766, 79.75261789303535, 80.04076182444304, 80.27971045048844, 80.52568697729988, 80.64516129032259, 80.80680300794153, 80.99655632862464, 81.11603064164734, 81.23550495467005, 81.27767235926629, 81.35497926769274, 81.44634197765129, 81.53067678684377, 81.56581629067398, 81.62203949680232, 81.66420690139856, 81.73448590905896, 81.87504392437978, 81.99451823740249, 81.99451823740249, 82.03668564199873, 82.0647972450629, 82.11399255042518, 82.12102045119123, 82.19129945885165, 82.24049476421393, 82.23346686344789, 82.1842715580856, 82.19832735961768, 82.24752266497997, 82.20535526038373, 82.20535526038373, 82.16318785578748, 82.22643896268184, 82.1842715580856, 82.16318785578748, 82.19832735961768, 82.24752266497997, 82.22643896268184, 82.21238316114976, 82.14913205425539, 82.11399255042518, 82.11399255042518, 82.09993674889311, 82.10696464965915, 82.05776934429686, 82.0647972450629, 82.01560193970062, 82.02262984046665, 82.0647972450629, 82.00857403893457, 81.98749033663645, 81.96640663433833, 81.93126713050812, 81.90315552744394, 81.83990442054959, 81.86801602361375, 81.81882071825146, 81.81179281748541, 81.76259751212312, 81.69231850446272, 81.64312319910043, 81.60095579450419, 81.54473258837585, 81.4814814814815, 81.44634197765129, 81.3830908707569, 81.34092346616066, 81.29172816079837, 81.24253285543608, 81.17225384777566, 81.10197484011525, 81.13008644317942, 81.06683533628505, 81.07386323705109, 81.03872373322089, 80.92627732096422, 80.89816571790006, 80.8208588094736, 80.83491461100569, 80.76463560334528, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 80.80442258994015, "threshold_area100": 80.80423782416193, "threshold_area25": 77.98088410991637}, "test": {"score": [66.95119530385956, 68.66687462779683, 69.61404304795394, 70.20389643535717, 70.54419646655134, 70.93554150242464, 71.26733403283895, 71.62748489918611, 72.01599410146613, 72.39032413577972, 72.71928083260075, 72.97167002240307, 73.19853670986586, 73.40555256217564, 73.657941751978, 73.86495760428777, 74.046450954258, 74.23645180500809, 74.44630349091115, 74.59376683776196, 74.77526018773219, 74.90854436661657, 75.05600771346738, 75.15526188923234, 75.24317273062417, 75.33391940560928, 75.41615857981454, 75.48138275246008, 75.59481609619148, 75.63168193290417, 75.6628761024303, 75.70541360632957, 75.74511527663557, 75.79048861412812, 75.83019028443411, 75.8131752828744, 75.88123528911323, 75.93795196097892, 75.95496696253863, 75.98332529847148, 76.04004197033719, 76.09959447579617, 76.1109378101693, 76.14496781328872, 76.147803646882, 76.13362447891558, 76.15347531406856, 76.15063948047528, 76.12228114454244, 76.13929614610214, 76.15914698125515, 76.15063948047528, 76.147803646882, 76.16481864844171, 76.14496781328872, 76.14496781328872, 76.13929614610214, 76.11660947735587, 76.11660947735587, 76.0712361398633, 76.04004197033719, 76.03437030315061, 76.02302696877747, 76.02302696877747, 75.98899696565805, 75.97198196409835, 75.9804894648782, 75.94645946175879, 75.93228029379236, 75.92093695941922, 75.89825029067295, 75.86138445396024, 75.85571278677367, 75.82168278365425, 75.8131752828744, 75.80750361568784, 75.80183194850126, 75.77347361256841, 75.75362277741542, 75.72526444148258, 75.70824943992285, 75.65436860165046, 75.62884609931089, 75.6260102657176, 75.56362192666535, 75.52675608995264, 75.45586025012052, 75.45018858293395, 75.38212857669512, 75.33391940560928, 75.25735189859058, 75.23750106343759, 75.16944105719875, 75.06735104784052, 75.05317187987409, 75.01914187675467, 74.88585769787028, 74.8092901908516, 74.75540935257918, 74.5852593369821, 74.47182599325072], "calls": [0.0, 4.628080424240705, 6.596148937980319, 7.980035731503275, 9.006607492272353, 10.160791764739246, 11.244080197374018, 12.429458639367041, 13.731106258684742, 15.0214105436293, 16.25216232311488, 17.431869097921336, 18.57754586960837, 19.666505969429714, 20.90576524969515, 21.957859512803786, 22.944729603266882, 24.144287213226328, 25.494144003629867, 26.639820775316903, 27.67206420327255, 28.809233474179734, 30.249836939568386, 31.54297705810623, 32.70283299775969, 34.03567478660352, 35.31747157076823, 36.50568584635454, 37.883900972690924, 39.080622749057085, 40.070328673113465, 41.13376627059524, 42.19153220089045, 43.34571647335734, 44.31273572866744, 45.265575816011115, 46.30065507756005, 47.23364432975073, 48.024841902277174, 48.915293650568586, 50.09216459178175, 50.97978050647988, 51.89859059070414, 52.79471400618212, 53.64829991776082, 54.38845248560814, 55.281740067492834, 55.959504296287896, 56.827269375833026, 57.63831778351246, 58.46354535915832, 59.26325043246462, 60.0657913393642, 60.86833224626379, 61.60564898051783, 62.2975923772793, 63.22207412869013, 64.03312253636958, 64.86118594560871, 65.58715934548961, 66.25641607350481, 67.05612114681111, 67.73672120919944, 68.36344043331538, 69.04971216289029, 69.70762555653234, 70.35135978220798, 71.08016901568216, 71.7976349147832, 72.5037574795111, 73.13331253722032, 73.81107676601538, 74.46048265887758, 75.18929189235176, 75.9436236281655, 76.77168703740465, 77.33601792246832, 78.12721549499474, 78.7879647222301, 79.41184811275275, 80.12364234466722, 80.8893174148541, 81.43946913195134, 82.13141252871281, 82.90843093327284, 83.64574766752688, 84.35470606584805, 85.13456030400135, 85.99665371635993, 86.71979128264753, 87.75487054419646, 88.58009811984233, 89.43084819782776, 90.48577829452967, 91.55772339279132, 92.54175764966112, 93.85474860335195, 94.98057453988599, 96.19431131781188, 98.02625981907381, 100.0], "rescued": [0, 611, 965, 1196, 1343, 1500, 1643, 1794, 1949, 2107, 2252, 2362, 2464, 2569, 2690, 2791, 2873, 2978, 3083, 3165, 3252, 3335, 3420, 3497, 3556, 3625, 3688, 3747, 3816, 3870, 3907, 3945, 3990, 4041, 4077, 4103, 4151, 4191, 4223, 4255, 4310, 4348, 4379, 4412, 4438, 4460, 4491, 4512, 4529, 4547, 4571, 4592, 4613, 4640, 4655, 4678, 4701, 4716, 4738, 4753, 4767, 4784, 4799, 4813, 4829, 4847, 4866, 4880, 4897, 4910, 4921, 4932, 4946, 4955, 4971, 4987, 4994, 5012, 5026, 5035, 5046, 5056, 5067, 5084, 5097, 5107, 5114, 5126, 5134, 5147, 5165, 5184, 5196, 5209, 5230, 5253, 5275, 5292, 5317, 5358, 5423], "harmed": [0, 6, 26, 49, 76, 95, 121, 145, 163, 189, 218, 239, 261, 293, 325, 353, 371, 409, 440, 470, 493, 529, 562, 604, 632, 669, 703, 739, 768, 809, 835, 858, 889, 924, 946, 978, 1002, 1022, 1048, 1070, 1105, 1122, 1149, 1170, 1195, 1222, 1246, 1268, 1295, 1307, 1324, 1348, 1370, 1391, 1413, 1436, 1461, 1484, 1506, 1537, 1562, 1581, 1600, 1614, 1642, 1666, 1682, 1708, 1730, 1747, 1766, 1790, 1806, 1827, 1846, 1864, 1873, 1901, 1922, 1941, 1958, 1987, 2007, 2025, 2060, 2083, 2115, 2129, 2161, 2191, 2236, 2262, 2298, 2347, 2373, 2408, 2477, 2521, 2565, 2666, 2771], "exact_score": [66.95119530385956, 67.16104698976264, 67.45597368346425, 67.85866205371069, 68.3776196012818, 68.86254714573349, 69.35881802455832, 69.78135722995773, 70.21240393613702, 70.54419646655134, 70.87882483055894, 71.18509485863369, 71.51972322264129, 71.79196324759663, 72.08688994129824, 72.38181663499985, 72.66539999432833, 72.8582366786717, 73.07659586535462, 73.27794005047782, 73.48495590278762, 73.67779258713098, 73.87062927147434, 74.05495845503786, 74.20242180188866, 74.36690015029917, 74.5313784987097, 74.6589910104075, 74.80078269007174, 74.94257436973598, 75.03615687831437, 75.11556021892635, 75.1977993931316, 75.2545160649973, 75.33108357201598, 75.4048152454414, 75.46720358449366, 75.53809942432578, 75.60048776337804, 75.6260102657176, 75.66004026883702, 75.69974193914301, 75.75362277741542, 75.78198111334827, 75.81884695006097, 75.80183194850126, 75.85571278677367, 75.9237727930125, 75.95213112894535, 75.9975044664379, 76.03437030315061, 76.09959447579617, 76.10810197657601, 76.15914698125515, 76.14496781328872, 76.13646031250886, 76.147803646882, 76.13078864532228, 76.14213197969542, 76.15631114766185, 76.15631114766185, 76.16198281484843, 76.16481864844171, 76.14496781328872, 76.11944531094915, 76.11377364376258, 76.04854947111703, 76.03437030315061, 76.01451946799762, 75.99466863284462, 75.97765363128491, 75.95496696253863, 75.92660862660578, 75.90392195785951, 75.84720528599381, 75.82451861724753, 75.8131752828744, 75.81033944928112, 75.77914527975497, 75.74511527663557, 75.70824943992285, 75.65720443524374, 75.64018943368403, 75.55511442588548, 75.50406942120637, 75.45302441652724, 75.38212857669512, 75.30556106967643, 75.23750106343759, 75.20914272750476, 75.10988855173977, 75.06451521424722, 75.04750021268751, 74.95958937129569, 74.87167852990386, 74.80645435725832, 74.76391685335905, 74.64764767603437, 74.58809517057539, 74.53988599948956, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.50312543193073, "threshold_area100": 75.0176105266143, "threshold_area25": 72.68751949635596}, "selected_budget": 48, "selected": {"val": {"score": 82.24752266497997, "calls": 47.99353433129524, "rescued": 1628, "harmed": 601}, "test": {"score": 76.12228114454244, "calls": 56.827269375833026, "rescued": 4529, "harmed": 1295}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.791201114654541, "operator": "ge"}, {"budget": 0.02, "threshold": 0.463607519865036, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3023056983947754, "operator": "ge"}, {"budget": 0.04, "threshold": 0.18571758270263672, "operator": "ge"}, {"budget": 0.05, "threshold": 0.08364289999008179, "operator": "ge"}, {"budget": 0.06, "threshold": 0.01764005422592163, "operator": "ge"}, {"budget": 0.07, "threshold": -0.10183686017990112, "operator": "ge"}, {"budget": 0.08, "threshold": -0.2613338828086853, "operator": "ge"}, {"budget": 0.09, "threshold": -0.38354265689849854, "operator": "ge"}, {"budget": 0.1, "threshold": -0.4990811347961426, "operator": "ge"}, {"budget": 0.11, "threshold": -0.6270751953125, "operator": "ge"}, {"budget": 0.12, "threshold": -0.765197217464447, "operator": "ge"}, {"budget": 0.13, "threshold": -0.88040691614151, "operator": "ge"}, {"budget": 0.14, "threshold": -1.0014557838439941, "operator": "ge"}, {"budget": 0.15, "threshold": -1.1431803703308105, "operator": "ge"}, {"budget": 0.16, "threshold": -1.2718673944473267, "operator": "ge"}, {"budget": 0.17, "threshold": -1.4229676723480225, "operator": "ge"}, {"budget": 0.18, "threshold": -1.559767246246338, "operator": "gt"}, {"budget": 0.19, "threshold": -1.7140259742736816, "operator": "gt"}, {"budget": 0.2, "threshold": -1.8584401607513428, "operator": "ge"}, {"budget": 0.21, "threshold": -1.989288330078125, "operator": "ge"}, {"budget": 0.22, "threshold": -2.128413200378418, "operator": "ge"}, {"budget": 0.23, "threshold": -2.2546517848968506, "operator": "ge"}, {"budget": 0.24, "threshold": -2.3781938552856445, "operator": "ge"}, {"budget": 0.25, "threshold": -2.5090792179107666, "operator": "ge"}, {"budget": 0.26, "threshold": -2.648585796356201, "operator": "ge"}, {"budget": 0.27, "threshold": -2.770765781402588, "operator": "ge"}, {"budget": 0.28, "threshold": -2.894237518310547, "operator": "ge"}, {"budget": 0.29, "threshold": -3.011873245239258, "operator": "ge"}, {"budget": 0.3, "threshold": -3.1263155937194824, "operator": "ge"}, {"budget": 0.31, "threshold": -3.2454848289489746, "operator": "ge"}, {"budget": 0.32, "threshold": -3.355722188949585, "operator": "ge"}, {"budget": 0.33, "threshold": -3.4528369903564453, "operator": "ge"}, {"budget": 0.34, "threshold": -3.5612809658050537, "operator": "ge"}, {"budget": 0.35, "threshold": -3.6423466205596924, "operator": "ge"}, {"budget": 0.36, "threshold": -3.73458194732666, "operator": "ge"}, {"budget": 0.37, "threshold": -3.8246819972991943, "operator": "ge"}, {"budget": 0.38, "threshold": -3.907654047012329, "operator": "ge"}, {"budget": 0.39, "threshold": -3.981703281402588, "operator": "ge"}, {"budget": 0.4, "threshold": -4.070158958435059, "operator": "ge"}, {"budget": 0.41, "threshold": -4.149557113647461, "operator": "ge"}, {"budget": 0.42, "threshold": -4.230738639831543, "operator": "ge"}, {"budget": 0.43, "threshold": -4.316549777984619, "operator": "ge"}, {"budget": 0.44, "threshold": -4.389975547790527, "operator": "ge"}, {"budget": 0.45, "threshold": -4.470677375793457, "operator": "ge"}, {"budget": 0.46, "threshold": -4.5370893478393555, "operator": "ge"}, {"budget": 0.47, "threshold": -4.611368656158447, "operator": "ge"}, {"budget": 0.48, "threshold": -4.681408882141113, "operator": "ge"}, {"budget": 0.49, "threshold": -4.755184173583984, "operator": "ge"}, {"budget": 0.5, "threshold": -4.825933456420898, "operator": "ge"}, {"budget": 0.51, "threshold": -4.9030866622924805, "operator": "ge"}, {"budget": 0.52, "threshold": -4.975246429443359, "operator": "ge"}, {"budget": 0.53, "threshold": -5.04669189453125, "operator": "ge"}, {"budget": 0.54, "threshold": -5.109314441680908, "operator": "ge"}, {"budget": 0.55, "threshold": -5.180075168609619, "operator": "ge"}, {"budget": 0.56, "threshold": -5.233514785766602, "operator": "ge"}, {"budget": 0.57, "threshold": -5.296395301818848, "operator": "ge"}, {"budget": 0.58, "threshold": -5.356722354888916, "operator": "ge"}, {"budget": 0.59, "threshold": -5.416804313659668, "operator": "ge"}, {"budget": 0.6, "threshold": -5.470822334289551, "operator": "ge"}, {"budget": 0.61, "threshold": -5.530477046966553, "operator": "ge"}, {"budget": 0.62, "threshold": -5.590939521789551, "operator": "ge"}, {"budget": 0.63, "threshold": -5.647007942199707, "operator": "ge"}, {"budget": 0.64, "threshold": -5.699427604675293, "operator": "ge"}, {"budget": 0.65, "threshold": -5.75341796875, "operator": "ge"}, {"budget": 0.66, "threshold": -5.81531286239624, "operator": "ge"}, {"budget": 0.67, "threshold": -5.870070457458496, "operator": "ge"}, {"budget": 0.68, "threshold": -5.935003757476807, "operator": "ge"}, {"budget": 0.69, "threshold": -6.006438255310059, "operator": "ge"}, {"budget": 0.7, "threshold": -6.0653181076049805, "operator": "ge"}, {"budget": 0.71, "threshold": -6.121241092681885, "operator": "ge"}, {"budget": 0.72, "threshold": -6.179999351501465, "operator": "ge"}, {"budget": 0.73, "threshold": -6.24144172668457, "operator": "ge"}, {"budget": 0.74, "threshold": -6.303466796875, "operator": "ge"}, {"budget": 0.75, "threshold": -6.374295234680176, "operator": "gt"}, {"budget": 0.76, "threshold": -6.441818714141846, "operator": "ge"}, {"budget": 0.77, "threshold": -6.509643077850342, "operator": "ge"}, {"budget": 0.78, "threshold": -6.579524040222168, "operator": "ge"}, {"budget": 0.79, "threshold": -6.642149925231934, "operator": "ge"}, {"budget": 0.8, "threshold": -6.703721046447754, "operator": "ge"}, {"budget": 0.81, "threshold": -6.7652482986450195, "operator": "ge"}, {"budget": 0.82, "threshold": -6.82840633392334, "operator": "ge"}, {"budget": 0.83, "threshold": -6.891758918762207, "operator": "ge"}, {"budget": 0.84, "threshold": -6.958848476409912, "operator": "ge"}, {"budget": 0.85, "threshold": -7.043850898742676, "operator": "ge"}, {"budget": 0.86, "threshold": -7.132628440856934, "operator": "gt"}, {"budget": 0.87, "threshold": -7.212976455688477, "operator": "ge"}, {"budget": 0.88, "threshold": -7.292725086212158, "operator": "ge"}, {"budget": 0.89, "threshold": -7.383803367614746, "operator": "ge"}, {"budget": 0.9, "threshold": -7.472460746765137, "operator": "ge"}, {"budget": 0.91, "threshold": -7.570735931396484, "operator": "ge"}, {"budget": 0.92, "threshold": -7.673715591430664, "operator": "ge"}, {"budget": 0.93, "threshold": -7.802931785583496, "operator": "ge"}, {"budget": 0.94, "threshold": -7.938742160797119, "operator": "ge"}, {"budget": 0.95, "threshold": -8.09698486328125, "operator": "gt"}, {"budget": 0.96, "threshold": -8.283764839172363, "operator": "ge"}, {"budget": 0.97, "threshold": -8.480810165405273, "operator": "ge"}, {"budget": 0.98, "threshold": -8.787460327148438, "operator": "ge"}, {"budget": 0.99, "threshold": -9.204487800598145, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.39531941808981, 75.60615644107105, 75.6834633494975, 75.78185396022208, 75.95052357860708, 76.02783048703353, 76.22461170848268, 76.47761613606015, 76.83603907512826, 77.15932251036615, 77.51071754866821, 77.80588938084195, 78.11511701454775, 78.3962330451894, 78.82493499191791, 79.17633003021997, 79.38013915243516, 79.72450628997119, 79.95642701525054, 80.14618033593366, 80.32890575585073, 80.53974277883196, 80.63813338955654, 80.91924942019818, 81.1090027408813, 81.23550495467005, 81.50959308454566, 81.57284419144003, 81.72043010752688, 81.81882071825146, 82.00857403893457, 82.17021575655352, 82.28969006957621, 82.35996907723663, 82.423220184131, 82.43727598566308, 82.44430388642913, 82.47944339025933, 82.46538758872724, 82.47944339025933, 82.52161079485558, 82.57783400098391, 82.64108510787828, 82.71136411553869, 82.71839201630473, 82.71839201630473, 82.71839201630473, 82.76055942090098, 82.7465036193689, 82.71136411553869, 82.69730831400662, 82.66919671094244, 82.68325251247452, 82.65514090941036, 82.67622461170848, 82.60594560404807, 82.6270293063462, 82.591889802516, 82.59891770328203, 82.60594560404807, 82.61297350481411, 82.58486190174995, 82.58486190174995, 82.57080610021787, 82.46538758872724, 82.41619228336496, 82.37402487876871, 82.38105277953476, 82.31077377187435, 82.33888537493851, 82.31077377187435, 82.2686063672781, 82.19832735961768, 82.17021575655352, 82.12804835195728, 82.0647972450629, 82.05074144353081, 81.99451823740249, 81.93829503127415, 81.89612762667792, 81.85396022208167, 81.75556961135709, 81.7063743059948, 81.62906739756835, 81.60095579450419, 81.50256518377961, 81.47445358071545, 81.41120247382108, 81.36200716845879, 81.31983976386253, 81.29875606156442, 81.27064445850026, 81.18630964930776, 81.13711434394546, 81.03169583245484, 81.01764003092276, 80.91924942019818, 80.85599831330381, 80.74355190104716, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99142596106543, 18.989387869843277, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.98770117365943, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.98636587251389, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.98910675381264, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 62, 110, 138, 168, 204, 231, 273, 322, 377, 435, 493, 547, 598, 651, 718, 777, 821, 878, 924, 968, 1009, 1051, 1080, 1131, 1169, 1204, 1255, 1279, 1311, 1340, 1378, 1409, 1437, 1462, 1482, 1502, 1520, 1540, 1560, 1578, 1597, 1612, 1636, 1657, 1670, 1686, 1699, 1714, 1726, 1738, 1746, 1754, 1768, 1775, 1785, 1791, 1801, 1808, 1816, 1826, 1834, 1839, 1846, 1853, 1854, 1856, 1859, 1869, 1870, 1878, 1883, 1886, 1890, 1892, 1897, 1899, 1905, 1908, 1913, 1915, 1915, 1915, 1919, 1919, 1924, 1926, 1929, 1931, 1934, 1937, 1943, 1945, 1948, 1951, 1951, 1957, 1958, 1960, 1961, 1964], "harmed": [0, 10, 28, 45, 61, 73, 89, 103, 116, 120, 132, 140, 152, 159, 172, 178, 187, 202, 210, 223, 240, 255, 267, 282, 293, 304, 321, 333, 348, 359, 374, 385, 393, 404, 419, 430, 448, 465, 480, 502, 518, 531, 538, 553, 564, 576, 592, 605, 614, 628, 645, 655, 667, 679, 690, 697, 713, 720, 732, 739, 748, 755, 764, 771, 780, 796, 805, 814, 823, 834, 838, 847, 856, 870, 876, 887, 898, 906, 917, 930, 938, 944, 958, 969, 980, 989, 1005, 1012, 1023, 1033, 1042, 1051, 1057, 1072, 1082, 1097, 1105, 1120, 1131, 1148, 1171], "exact_score": [75.02986857825567, 75.39531941808981, 75.60615644107105, 75.6834633494975, 75.78185396022208, 75.95052357860708, 76.02783048703353, 76.22461170848268, 76.47761613606015, 76.83603907512826, 77.15932251036615, 77.51071754866821, 77.80588938084195, 78.11511701454775, 78.3962330451894, 78.82493499191791, 79.17633003021997, 79.38013915243516, 79.72450628997119, 79.95642701525054, 80.14618033593366, 80.32890575585073, 80.53974277883196, 80.63813338955654, 80.91924942019818, 81.1090027408813, 81.23550495467005, 81.50959308454566, 81.57284419144003, 81.72043010752688, 81.81882071825146, 82.00857403893457, 82.17021575655352, 82.28969006957621, 82.35996907723663, 82.423220184131, 82.43727598566308, 82.44430388642913, 82.47944339025933, 82.46538758872724, 82.47944339025933, 82.52161079485558, 82.57783400098391, 82.64108510787828, 82.71136411553869, 82.71839201630473, 82.71839201630473, 82.71839201630473, 82.76055942090098, 82.7465036193689, 82.71136411553869, 82.69730831400662, 82.66919671094244, 82.68325251247452, 82.65514090941036, 82.67622461170848, 82.60594560404807, 82.6270293063462, 82.591889802516, 82.59891770328203, 82.60594560404807, 82.61297350481411, 82.58486190174995, 82.58486190174995, 82.57080610021787, 82.46538758872724, 82.41619228336496, 82.37402487876871, 82.38105277953476, 82.31077377187435, 82.33888537493851, 82.31077377187435, 82.2686063672781, 82.19832735961768, 82.17021575655352, 82.12804835195728, 82.0647972450629, 82.05074144353081, 81.99451823740249, 81.93829503127415, 81.89612762667792, 81.85396022208167, 81.75556961135709, 81.7063743059948, 81.62906739756835, 81.60095579450419, 81.50256518377961, 81.47445358071545, 81.41120247382108, 81.36200716845879, 81.31983976386253, 81.29875606156442, 81.27064445850026, 81.18630964930776, 81.13711434394546, 81.03169583245484, 81.01764003092276, 80.91924942019818, 80.85599831330381, 80.74355190104716, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.06937602934435, "threshold_area100": 81.06918968304169, "threshold_area25": 78.03007941527866}, "test": {"score": [66.95119530385956, 68.0997079091399, 68.91359215041261, 69.48926636984942, 70.03374641976009, 70.70867481496185, 71.25599069846581, 71.56226072654056, 71.93375492726086, 72.33927913110058, 72.6909224966679, 73.00286419192922, 73.33465672234352, 73.47928423560106, 73.71465842384369, 73.95286844567961, 74.1768992995491, 74.3243626463999, 74.45764682528429, 74.59660267135524, 74.76108101976577, 74.90003686583671, 75.00212687519496, 75.01630604316138, 75.10988855173977, 75.24600856421745, 75.4019794118481, 75.4785469188668, 75.55795025947877, 75.6827269375833, 75.72526444148258, 75.83586195162067, 75.88407112270652, 75.91810112582593, 75.97198196409835, 76.00884780081105, 76.00884780081105, 76.00034030003118, 76.01451946799762, 76.04854947111703, 76.10526614298273, 76.12511697813572, 76.12511697813572, 76.147803646882, 76.15347531406856, 76.11377364376258, 76.09392280860959, 76.10810197657601, 76.09959447579617, 76.13929614610214, 76.15347531406856, 76.16198281484843, 76.17899781640813, 76.127952811729, 76.10526614298273, 76.04571363752375, 76.04287780393047, 76.02302696877747, 76.00034030003118, 75.9436236281655, 75.94929529535207, 75.96631029691177, 75.97481779769163, 75.92944446019906, 75.93228029379236, 75.92093695941922, 75.89825029067295, 75.88407112270652, 75.89257862348637, 75.88690695629981, 75.88123528911323, 75.85287695318038, 75.83019028443411, 75.81601111646769, 75.7961602813147, 75.80183194850126, 75.7933244477214, 75.77630944616169, 75.76213027819527, 75.72810027507586, 75.71675694070272, 75.68839860476987, 75.6657119360236, 75.6089952641579, 75.58063692822505, 75.56362192666535, 75.50406942120637, 75.47287525168022, 75.41899441340783, 75.36227774154213, 75.33108357201598, 75.28287440093015, 75.20063522672488, 75.13824688767264, 75.06735104784052, 74.98227604004197, 74.91705186739642, 74.81496185803816, 74.7157076822732, 74.63063267447467, 74.47182599325072], "calls": [0.0, 1.945381844993336, 3.7971811814082748, 5.424949663953719, 7.123613986331281, 9.15407083912316, 11.122139352862774, 12.497518645605876, 13.855882936789268, 15.211411394379379, 16.68320902929416, 18.05008082125741, 19.402773445254233, 20.528599381788275, 21.69129115503502, 22.958908771233304, 24.161302214786037, 25.587726512208263, 26.855344128406543, 28.17117091569067, 29.5408785412472, 30.834018659785045, 32.217905453308, 33.43164223123387, 34.71343901539858, 36.14553498000737, 37.753452627399824, 39.15719025607577, 40.611972889430845, 41.876754672035844, 43.2237756288461, 44.51975158097723, 45.861100870600914, 46.99543430791481, 48.37932110143777, 49.357683691121004, 50.4551512917222, 51.62067889856223, 52.641578992144744, 53.537702407622724, 54.59546833791793, 55.4745767518362, 56.37637183450075, 57.524884439781076, 58.50324702946431, 59.561012959759516, 60.36071803306582, 61.30505061962964, 62.1671440319882, 63.05475994668633, 64.07282420667555, 65.04267929557894, 66.02671355244874, 67.05044947962455, 67.86716955449054, 68.75478546918868, 69.46090803391657, 70.30315061112213, 71.0347956781896, 71.77211241244363, 72.4498766412387, 73.19570087627258, 73.94719677849304, 74.59093100416867, 75.22048606187789, 75.78481694694156, 76.4824320108896, 77.01840456002041, 77.71318379037518, 78.41363468791651, 79.00348807531974, 79.68408813770807, 80.3675240336897, 81.05946743045118, 81.69752998894026, 82.47171255990699, 83.1749992910416, 83.85276351983666, 84.62127442361682, 85.25366531491933, 85.79247369764343, 86.33695374755409, 86.89844879902448, 87.47412301846128, 88.01860306837195, 88.69920313076028, 89.42517653064118, 89.980999914925, 90.63324164138048, 91.28264753424268, 91.92354592632505, 92.59280265434025, 93.27623855032186, 94.03624195332218, 94.8869920313076, 95.76893627881915, 96.57431301931203, 97.29177891841306, 98.11984232765222, 98.95357740407793, 100.0], "rescued": [0, 424, 739, 972, 1183, 1435, 1642, 1778, 1931, 2096, 2248, 2395, 2549, 2632, 2753, 2864, 2974, 3077, 3163, 3254, 3346, 3434, 3515, 3569, 3640, 3714, 3808, 3879, 3948, 4019, 4075, 4140, 4191, 4235, 4299, 4335, 4363, 4397, 4428, 4460, 4500, 4531, 4556, 4596, 4622, 4643, 4663, 4691, 4711, 4741, 4777, 4810, 4838, 4865, 4880, 4894, 4913, 4929, 4940, 4951, 4969, 4988, 5009, 5020, 5035, 5050, 5064, 5077, 5092, 5104, 5115, 5124, 5140, 5152, 5159, 5174, 5189, 5204, 5216, 5222, 5232, 5237, 5246, 5249, 5261, 5269, 5278, 5284, 5292, 5300, 5308, 5317, 5321, 5335, 5348, 5361, 5371, 5383, 5390, 5399, 5423], "harmed": [0, 19, 47, 77, 96, 110, 124, 152, 174, 196, 224, 261, 298, 330, 368, 395, 426, 477, 516, 558, 592, 631, 676, 725, 763, 789, 828, 872, 913, 940, 981, 1007, 1041, 1073, 1118, 1141, 1169, 1206, 1232, 1252, 1272, 1296, 1321, 1353, 1377, 1412, 1439, 1462, 1485, 1501, 1532, 1562, 1584, 1629, 1652, 1687, 1707, 1730, 1749, 1780, 1796, 1809, 1827, 1854, 1868, 1887, 1909, 1927, 1939, 1953, 1966, 1985, 2009, 2026, 2040, 2053, 2071, 2092, 2109, 2127, 2141, 2156, 2173, 2196, 2218, 2232, 2262, 2279, 2306, 2334, 2353, 2379, 2412, 2448, 2486, 2529, 2562, 2610, 2652, 2691, 2771], "exact_score": [66.95119530385956, 67.5382128576695, 68.12239457788617, 68.61015795593114, 68.96463715509174, 69.31911635425233, 69.69344638856592, 70.01389558460708, 70.31165811190199, 70.66613731106258, 70.93554150242464, 71.22479652893968, 71.48852905311516, 71.72957490854436, 71.99047159912656, 72.28823412642146, 72.52360831466409, 72.76465417009331, 72.98301335677623, 73.2127158778323, 73.38570172702265, 73.5643592433996, 73.77988259648923, 73.94436094489976, 74.15137679720954, 74.2874968096872, 74.36122848311261, 74.47182599325072, 74.57108016901569, 74.67600601196722, 74.82346935881802, 74.90287269942999, 74.98511187363526, 75.01063437597482, 75.05600771346738, 75.13257522048606, 75.23182939625103, 75.33675523920256, 75.43600941496753, 75.48421858605336, 75.54660692510564, 75.60332359697132, 75.6827269375833, 75.69974193914301, 75.78765278053483, 75.86705612114682, 75.88690695629981, 75.91810112582593, 75.94645946175879, 75.9975044664379, 76.02019113518419, 75.98616113206477, 76.02302696877747, 76.03437030315061, 76.07690780704988, 76.127952811729, 76.10810197657601, 76.11944531094915, 76.13929614610214, 76.11944531094915, 76.11377364376258, 76.08825114142302, 76.10526614298273, 76.14213197969542, 76.14496781328872, 76.15631114766185, 76.1846694835947, 76.13362447891558, 76.09959447579617, 76.0542211383036, 76.01735530159091, 76.00034030003118, 75.9436236281655, 75.96631029691177, 75.96347446331849, 75.92660862660578, 75.93228029379236, 75.88407112270652, 75.88407112270652, 75.88123528911323, 75.83019028443411, 75.8131752828744, 75.81033944928112, 75.7961602813147, 75.77914527975497, 75.7564586110087, 75.7224286078893, 75.64869693446389, 75.58063692822505, 75.53809942432578, 75.47003941808694, 75.37645690950855, 75.32257607123614, 75.24600856421745, 75.13541105407936, 75.05884354706066, 74.95958937129569, 74.85749936193744, 74.7355585174262, 74.62779684088137, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.55073736743877, "threshold_area100": 74.992825341009, "threshold_area25": 72.64242974222273}, "selected_budget": 48, "selected": {"val": {"score": 82.76055942090098, "calls": 47.99353433129524, "rescued": 1714, "harmed": 614}, "test": {"score": 76.09959447579617, "calls": 62.1671440319882, "rescued": 4711, "harmed": 1485}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 80.84194251177173, "calls": 95.99409656335652}, "test": {"score": 75.00212687519496, "calls": 95.93057879363639}}, "old_normalized_interior": {"val": {"score": 82.50052709255745, "calls": 50.99444795839483}, "test": {"score": 76.0740719734566, "calls": 69.54031137452854}}, "old_normalized_endpoints": {"val": {"score": 82.50052709255745, "calls": 50.99444795839483}, "test": {"score": 76.0740719734566, "calls": 69.54031137452854}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.36017991425962, 75.6975191510296, 75.99971888396936, 76.31597441844121, 76.42842083069787, 76.55492304448661, 76.65331365521119, 76.77278796823389, 77.03282029657741, 77.36313163258134, 77.65127556398905, 77.86914048773632, 78.22756342680441, 78.52273525897814, 78.6843769765971, 78.86007449574812, 79.23255323634831, 79.40122285473329, 79.60503197694848, 79.90020380912222, 80.21645934359407, 80.36404525968094, 80.46946377117156, 80.65218919108862, 80.82788671023965, 80.96844472556047, 81.19333755007379, 81.31281186309648, 81.39714667228898, 81.57987209220606, 81.69934640522875, 81.74854171059106, 81.83287651978354, 81.95937873357228, 82.00154613816854, 82.12102045119123, 82.22643896268184, 82.27563426804414, 82.29671797034226, 82.28266216881018, 82.3037458711083, 82.31780167264037, 82.41619228336496, 82.4583596879612, 82.40916438259892, 82.43024808489704, 82.38105277953476, 82.44430388642913, 82.46538758872724, 82.47241548949329, 82.50052709255745, 82.4583596879612, 82.39510858106684, 82.36699697800267, 82.38105277953476, 82.37402487876871, 82.38105277953476, 82.34591327570455, 82.31780167264037, 82.2686063672781, 82.23346686344789, 82.2194110619158, 82.15615995502144, 82.17021575655352, 82.19129945885165, 82.21238316114976, 82.17724365731955, 82.09290884812707, 82.00154613816854, 82.01560193970062, 81.95235083280625, 81.91721132897604, 81.87504392437978, 81.81179281748541, 81.73448590905896, 81.67826270293064, 81.60798369527022, 81.5517604891419, 81.49553728301356, 81.42525827535316, 81.36903506922482, 81.33389556539463, 81.29875606156442, 81.27064445850026, 81.21442125237192, 81.17928174854171, 81.13008644317942, 81.08791903858318, 81.04575163398692, 81.00358422939068, 80.96844472556047, 80.9051936186661, 80.85599831330381, 80.83491461100569, 80.80680300794153, 80.72949609951507, 80.715440297983, 80.70841239721696, 80.6662449926207, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.98692810457516, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.98636587251389, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.36017991425962, 75.6975191510296, 75.99971888396936, 76.31597441844121, 76.42842083069787, 76.55492304448661, 76.65331365521119, 76.77278796823389, 77.03282029657741, 77.36313163258134, 77.65127556398905, 77.86914048773632, 78.22756342680441, 78.52273525897814, 78.6843769765971, 78.86007449574812, 79.23255323634831, 79.40122285473329, 79.60503197694848, 79.90020380912222, 80.21645934359407, 80.36404525968094, 80.46946377117156, 80.65218919108862, 80.82788671023965, 80.96844472556047, 81.19333755007379, 81.31281186309648, 81.39714667228898, 81.57987209220606, 81.69934640522875, 81.74854171059106, 81.83287651978354, 81.95235083280625, 82.00154613816854, 82.12102045119123, 82.22643896268184, 82.27563426804414, 82.29671797034226, 82.28266216881018, 82.3037458711083, 82.31780167264037, 82.41619228336496, 82.4583596879612, 82.40916438259892, 82.43024808489704, 82.38105277953476, 82.44430388642913, 82.46538758872724, 82.47241548949329, 82.50052709255745, 82.4583596879612, 82.39510858106684, 82.36699697800267, 82.38105277953476, 82.37402487876871, 82.38105277953476, 82.34591327570455, 82.31780167264037, 82.2686063672781, 82.23346686344789, 82.2194110619158, 82.15615995502144, 82.17021575655352, 82.19129945885165, 82.21238316114976, 82.17724365731955, 82.09290884812707, 82.00154613816854, 82.01560193970062, 81.95235083280625, 81.91721132897604, 81.87504392437978, 81.81179281748541, 81.73448590905896, 81.67826270293064, 81.60798369527022, 81.5517604891419, 81.49553728301356, 81.42525827535316, 81.36903506922482, 81.33389556539463, 81.29875606156442, 81.27064445850026, 81.21442125237192, 81.17928174854171, 81.13008644317942, 81.08791903858318, 81.04575163398692, 81.00358422939068, 80.96844472556047, 80.9051936186661, 80.85599831330381, 80.83491461100569, 80.80680300794153, 80.72949609951507, 80.715440297983, 80.70841239721696, 80.6662449926207, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 78.06100217864925, "exact_area100": 80.84921144251696}, "test": {"score": [66.95119530385956, 68.45702294189377, 69.42120636361058, 70.18120976661089, 70.82210815869324, 71.45733488358903, 71.98479993193999, 72.34211496469388, 72.64838499276863, 72.88092334741798, 73.16450670674645, 73.38853756061594, 73.67779258713098, 73.8252559339818, 73.97271928083259, 74.18257096673567, 74.34988514873947, 74.50018432918355, 74.70152851430679, 74.85182769475087, 74.97944020644869, 75.11272438533307, 75.18645605875847, 75.26302356577716, 75.42183024700111, 75.52675608995264, 75.62033859853103, 75.67989110399002, 75.73377194226242, 75.81884695006097, 75.87839945551995, 75.91526529223265, 76.00601196721776, 76.03153446955733, 76.06272863908346, 76.1109378101693, 76.1846694835947, 76.16198281484843, 76.2016844851544, 76.21019198593426, 76.21302781952754, 76.2215353203074, 76.17049031562829, 76.18750531718798, 76.19034115078126, 76.21586365312083, 76.21586365312083, 76.19601281796784, 76.16481864844171, 76.10526614298273, 76.12228114454244, 76.0740719734566, 76.06272863908346, 76.0712361398633, 76.0712361398633, 76.07690780704988, 76.0542211383036, 76.02869863596405, 76.04571363752375, 76.02869863596405, 76.04854947111703, 76.0542211383036, 76.04854947111703, 76.03153446955733, 75.98616113206477, 75.95213112894535, 75.94929529535207, 75.92660862660578, 75.90959362504609, 75.8698919547401, 75.80466778209455, 75.76780194538185, 75.75929444460199, 75.71675694070272, 75.69974193914301, 75.64869693446389, 75.60332359697132, 75.55795025947877, 75.54377109151234, 75.47571108527352, 75.4416810821541, 75.39347191106826, 75.36227774154213, 75.29421773530329, 75.26585939937044, 75.22332189547117, 75.1807843915719, 75.13824688767264, 75.09570938377335, 75.04182854550095, 74.9936193744151, 74.93406686895612, 74.88869353146357, 74.83481269319117, 74.7724243541389, 74.72705101664634, 74.70436434790007, 74.6589910104075, 74.59376683776196, 74.5313784987097, 74.47182599325072], "calls": [0.0, 3.3264328049230074, 5.833309701386723, 7.792870714346482, 9.695715055440548, 11.44258854890395, 13.229163712673339, 14.58469217026345, 15.764398945069905, 16.898732382383802, 18.0699316564104, 19.178742591384736, 20.56546521850098, 21.906814508124665, 23.216969628222216, 24.68309559595043, 26.149221563678644, 27.4593766837762, 28.916995150724556, 30.28103110909452, 31.775515412755578, 33.289850551569636, 34.4128406545104, 35.81090661599977, 37.3677792587131, 38.96435357173241, 40.32555369650909, 41.703768822845475, 43.05362561324901, 44.533930748943654, 46.04826588775771, 47.44633184924709, 48.620366956866974, 49.75753622777415, 51.16127385645011, 52.34098063125656, 53.778748263051924, 54.85352919490685, 56.00771346737373, 56.95204605393756, 57.91339364206108, 59.07892124890112, 60.2160905198083, 61.42982729773417, 62.456399058503244, 63.62476249893656, 64.64849842611235, 65.65521935172845, 66.6846269460908, 67.58358619516207, 68.59030712077815, 69.54031137452854, 70.33718061424156, 71.2049456937867, 71.96778493038029, 72.79584833961943, 73.63525508323173, 74.46048265887758, 75.23182939625103, 76.23004282108727, 76.97303122252787, 77.56004877633781, 78.33139551371126, 78.95811473782719, 79.71528230723422, 80.39588236962256, 81.00275075858548, 81.75424666080595, 82.44051839038086, 82.95664010435868, 83.63156849956044, 84.29515356038908, 84.84530527748632, 85.54859200862094, 86.17247539914358, 86.77934378810652, 87.35218217395003, 87.94487139494655, 88.45815727533109, 89.00547315883503, 89.56696821030542, 90.06607492272354, 90.65025664294019, 91.24861753112327, 91.7647392451011, 92.26668179111249, 92.81116184102316, 93.37549272608683, 93.86892777131838, 94.30281031109095, 94.78773785554264, 95.3322179054533, 95.83416045146471, 96.40132717012165, 96.94864305362562, 97.41939143011088, 97.94118481127528, 98.41193318776055, 98.90253239939881, 99.41014661259678, 100.0], "exact_score": [66.95119530385956, 67.49567535377024, 67.95791622947566, 68.31239542863624, 68.69806879732297, 69.06105549726342, 69.47508720188299, 69.89195474009585, 70.24076227206987, 70.62359980716332, 70.91852650086493, 71.30419986955165, 71.6303207327794, 71.91673992570115, 72.17196494909679, 72.47823497717154, 72.68525082948132, 72.91778918413068, 73.13898420440688, 73.34883589030996, 73.5643592433996, 73.71749425743697, 73.84227093554149, 73.96137594645947, 74.1031676261237, 74.21660096985508, 74.34137764795962, 74.44346765731787, 74.54839350026941, 74.72421518305306, 74.82630519241131, 74.91421603380314, 74.98794770722854, 75.09570938377335, 75.15526188923234, 75.20914272750476, 75.2743669001503, 75.3849644102884, 75.45586025012052, 75.52675608995264, 75.58630859541162, 75.62033859853103, 75.67989110399002, 75.73093610866914, 75.80466778209455, 75.85571278677367, 75.87556362192667, 75.90392195785951, 75.94645946175879, 76.02586280237075, 76.03153446955733, 76.03437030315061, 76.07690780704988, 76.15631114766185, 76.19601281796784, 76.16765448203499, 76.2016844851544, 76.20735615234098, 76.2016844851544, 76.2215353203074, 76.19034115078126, 76.16765448203499, 76.2045203187477, 76.21869948671412, 76.21302781952754, 76.20735615234098, 76.19884865156112, 76.147803646882, 76.10810197657601, 76.10526614298273, 76.05989280549016, 76.0740719734566, 76.07690780704988, 76.05705697189688, 76.04854947111703, 76.03153446955733, 76.03720613674389, 76.04571363752375, 76.04571363752375, 76.02586280237075, 75.97198196409835, 75.94929529535207, 75.92944446019906, 75.8698919547401, 75.76213027819527, 75.76213027819527, 75.70541360632957, 75.64018943368403, 75.55511442588548, 75.47571108527352, 75.39347191106826, 75.31690440404957, 75.24600856421745, 75.16660522360547, 75.08436604940022, 74.96809687207553, 74.87735019709044, 74.77526018773219, 74.70720018149335, 74.58242350338881, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 73.0045656920852, "exact_area100": 74.57810380899112}}, "old_normalized_endpoint_budget": 51}, {"seed": 2028, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 1.0, "validation_area100": 0.8151248593820977, "training_seconds": 76.37714479700662, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.43514082363625173, "operator": "ge"}, {"budget": 0.02, "threshold": 0.20717252319790158, "operator": "ge"}, {"budget": 0.03, "threshold": 0.08656428569920187, "operator": "ge"}, {"budget": 0.04, "threshold": -0.01768701481730409, "operator": "ge"}, {"budget": 0.05, "threshold": -0.13558625469623706, "operator": "ge"}, {"budget": 0.06, "threshold": -0.22990237788286919, "operator": "ge"}, {"budget": 0.07, "threshold": -0.31375828241168907, "operator": "ge"}, {"budget": 0.08, "threshold": -0.40777279452929993, "operator": "ge"}, {"budget": 0.09, "threshold": -0.4914163775725296, "operator": "ge"}, {"budget": 0.1, "threshold": -0.5763073068883737, "operator": "ge"}, {"budget": 0.11, "threshold": -0.6427998379570377, "operator": "ge"}, {"budget": 0.12, "threshold": -0.7083469008097267, "operator": "ge"}, {"budget": 0.13, "threshold": -0.765534103209231, "operator": "ge"}, {"budget": 0.14, "threshold": -0.8102278167372404, "operator": "ge"}, {"budget": 0.15, "threshold": -0.8441100116636652, "operator": "ge"}, {"budget": 0.16, "threshold": -0.870227402563797, "operator": "ge"}, {"budget": 0.17, "threshold": -0.891793431030085, "operator": "ge"}, {"budget": 0.18, "threshold": -0.9072517755194595, "operator": "ge"}, {"budget": 0.19, "threshold": -0.9202675744237843, "operator": "ge"}, {"budget": 0.2, "threshold": -0.9301268918682357, "operator": "gt"}, {"budget": 0.21, "threshold": -0.9379401305025009, "operator": "ge"}, {"budget": 0.22, "threshold": -0.944409681470687, "operator": "ge"}, {"budget": 0.23, "threshold": -0.9499926724301463, "operator": "ge"}, {"budget": 0.24, "threshold": -0.9562291359641691, "operator": "ge"}, {"budget": 0.25, "threshold": -0.9600639346246623, "operator": "ge"}, {"budget": 0.26, "threshold": -0.9637325676928291, "operator": "ge"}, {"budget": 0.27, "threshold": -0.9669052687338652, "operator": "ge"}, {"budget": 0.28, "threshold": -0.970105851433257, "operator": "ge"}, {"budget": 0.29, "threshold": -0.9726391845574929, "operator": "ge"}, {"budget": 0.3, "threshold": -0.9748304831053511, "operator": "ge"}, {"budget": 0.31, "threshold": -0.9769359327427697, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9787757164550625, "operator": "ge"}, {"budget": 0.33, "threshold": -0.9804542772487893, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9820519195718633, "operator": "ge"}, {"budget": 0.35, "threshold": -0.9835044819902952, "operator": "ge"}, {"budget": 0.36, "threshold": -0.9846964873479627, "operator": "ge"}, {"budget": 0.37, "threshold": -0.985929846626182, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9869836699716039, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9879222458881589, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9887963938968118, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9895967325006765, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9903308223190858, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9910323303127401, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9915804553686984, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9922779890661184, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9928372005590863, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9933647435889569, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9938542722324596, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9943472922954376, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9946723524657739, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9950290602779397, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9954168313222082, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9957602005969074, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9960216230073873, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9962930547863258, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9965070721755567, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9967293147219926, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9969478678046364, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9971768423562838, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9973484798337042, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9975224589345509, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9976610158052759, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9977978040681836, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9979412689840491, "operator": "ge"}, {"budget": 0.65, "threshold": -0.998073073288444, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9981787798168388, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9982824813256317, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9983585099373902, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9984398432670509, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9985187969614643, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9985926366971615, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9986697435411667, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9987210885520366, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9987801860848589, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9988320778190681, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9988813806483756, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9989367547714121, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9989802132399326, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9990243237180256, "operator": "ge"}, {"budget": 0.8, "threshold": -0.999059022430179, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9990950910737045, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9991290719366214, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9991680232803496, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9992040357907809, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9992454996163869, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9992772355381888, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9993118952976062, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9993420322243197, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9993735069851779, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9994038978202083, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9994298200538634, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9994567127874943, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9994866352217489, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9995224787570459, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9995588916593803, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9995935152639076, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9996329650899864, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9996723018795555, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9997298490230153, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.57101693724084, 75.95052357860708, 76.35111392227142, 76.52681144142245, 76.79387167053201, 77.05390399887554, 77.34204793028323, 77.5599128540305, 77.89022419003444, 78.09403331224964, 78.29081453369878, 78.45245625131774, 78.66329327429898, 78.9092698011104, 79.07793941949541, 79.21146953405018, 79.47852976315974, 79.7596457938014, 80.0829292290393, 80.48351957270363, 80.77166350411133, 81.09494693934923, 81.36903506922482, 81.66420690139856, 81.86098812284771, 82.07182514582894, 82.2194110619158, 82.53566659638766, 82.64811300864432, 82.77461522243306, 82.83786632932744, 82.90814533698784, 83.05573125307471, 83.18223346686345, 83.32279148218427, 83.4141541921428, 83.47740529903717, 83.63201911589009, 83.84285613887133, 83.9061072457657, 83.96233045189402, 84.00449785649027, 84.01855365802236, 84.01152575725631, 84.01152575725631, 83.93421884882986, 83.8569119404034, 83.87096774193549, 83.77960503197694, 83.6531028181882, 83.61093541359196, 83.4984890013353, 83.42820999367488, 83.35090308524843, 83.35793098601448, 83.2595403752899, 83.19628926839553, 83.1752055660974, 83.06978705460679, 82.97842434464826, 82.90111743622181, 82.83786632932744, 82.71136411553869, 82.6270293063462, 82.52863869562161, 82.4583596879612, 82.40916438259892, 82.35996907723663, 82.31077377187435, 82.25455056574602, 82.1842715580856, 82.13507625272331, 82.07885304659499, 82.03668564199873, 81.98046243587041, 81.91018342820999, 81.83990442054959, 81.78368121442125, 81.73448590905896, 81.67826270293064, 81.65015109986648, 81.55878838990793, 81.50959308454566, 81.43931407688524, 81.41120247382108, 81.39011877152295, 81.3479513669267, 81.28470026003232, 81.25658865696816, 81.20036545083984, 81.15819804624358, 81.08089113781713, 81.03872373322089, 80.96844472556047, 80.94736102326236, 80.91222151943215, 80.85599831330381, 80.7857193056434, 80.69435659568487, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.987349778621123, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 83, 142, 206, 244, 286, 336, 383, 426, 483, 529, 572, 611, 654, 710, 756, 798, 853, 906, 964, 1025, 1079, 1129, 1179, 1232, 1274, 1315, 1346, 1396, 1426, 1454, 1481, 1508, 1543, 1578, 1609, 1635, 1655, 1688, 1724, 1742, 1763, 1783, 1797, 1813, 1829, 1839, 1847, 1859, 1868, 1875, 1883, 1890, 1895, 1903, 1912, 1918, 1925, 1932, 1935, 1938, 1940, 1944, 1946, 1946, 1947, 1948, 1949, 1952, 1953, 1954, 1954, 1954, 1955, 1955, 1955, 1955, 1955, 1956, 1956, 1957, 1957, 1957, 1958, 1959, 1961, 1961, 1961, 1961, 1961, 1961, 1961, 1961, 1961, 1961, 1962, 1963, 1963, 1963, 1964, 1964], "harmed": [0, 6, 11, 18, 31, 35, 48, 54, 66, 76, 93, 108, 124, 137, 158, 180, 203, 220, 233, 245, 249, 262, 266, 277, 288, 302, 313, 323, 328, 342, 352, 370, 387, 401, 418, 429, 442, 453, 464, 470, 479, 492, 506, 518, 535, 551, 572, 591, 601, 623, 648, 662, 685, 700, 719, 727, 747, 763, 773, 791, 807, 820, 833, 853, 865, 880, 891, 899, 909, 917, 926, 936, 943, 952, 958, 966, 976, 986, 995, 1002, 1011, 1015, 1028, 1036, 1047, 1053, 1056, 1062, 1071, 1075, 1083, 1089, 1100, 1106, 1116, 1120, 1126, 1134, 1144, 1158, 1171], "exact_score": [75.02986857825567, 75.57101693724084, 75.95052357860708, 76.35111392227142, 76.52681144142245, 76.79387167053201, 77.05390399887554, 77.34204793028323, 77.5599128540305, 77.89022419003444, 78.09403331224964, 78.29081453369878, 78.45245625131774, 78.66329327429898, 78.9092698011104, 79.07793941949541, 79.21146953405018, 79.47852976315974, 79.7596457938014, 80.0829292290393, 80.48351957270363, 80.77166350411133, 81.09494693934923, 81.36903506922482, 81.66420690139856, 81.86098812284771, 82.07182514582894, 82.2194110619158, 82.53566659638766, 82.64811300864432, 82.77461522243306, 82.83786632932744, 82.90814533698784, 83.05573125307471, 83.18223346686345, 83.32279148218427, 83.4141541921428, 83.47740529903717, 83.63201911589009, 83.84285613887133, 83.9061072457657, 83.96233045189402, 84.00449785649027, 84.01855365802236, 84.01152575725631, 84.01152575725631, 83.93421884882986, 83.8569119404034, 83.87096774193549, 83.77960503197694, 83.6531028181882, 83.61093541359196, 83.4984890013353, 83.42820999367488, 83.35090308524843, 83.35793098601448, 83.2595403752899, 83.19628926839553, 83.1752055660974, 83.06978705460679, 82.97842434464826, 82.90111743622181, 82.83786632932744, 82.71136411553869, 82.6270293063462, 82.52863869562161, 82.4583596879612, 82.40916438259892, 82.35996907723663, 82.31077377187435, 82.25455056574602, 82.1842715580856, 82.13507625272331, 82.07885304659499, 82.03668564199873, 81.98046243587041, 81.91018342820999, 81.83990442054959, 81.78368121442125, 81.73448590905896, 81.67826270293064, 81.65015109986648, 81.55878838990793, 81.50959308454566, 81.43931407688524, 81.41120247382108, 81.39011877152295, 81.3479513669267, 81.28470026003232, 81.25658865696816, 81.20036545083984, 81.15819804624358, 81.08089113781713, 81.03872373322089, 80.96844472556047, 80.94736102326236, 80.91222151943215, 80.85599831330381, 80.7857193056434, 80.69435659568487, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.51248593820976, "threshold_area100": 81.51229882634058, "threshold_area25": 78.57895846510647}, "test": {"score": [66.95119530385956, 68.33508209738253, 69.48359470266284, 70.35986728298784, 71.09718401724187, 71.56226072654056, 71.97912826475343, 72.35912996625358, 72.68525082948132, 72.9518191872501, 73.26659671610469, 73.42256756373536, 73.60973258089216, 73.86212177069449, 74.07480929019084, 74.19958596829538, 74.30451181124691, 74.43212432294473, 74.55406516745597, 74.67884184556051, 74.7894393556986, 74.89720103224342, 75.02481354394124, 75.17794855797861, 75.28003856733687, 75.32257607123614, 75.3849644102884, 75.45018858293395, 75.56362192666535, 75.6628761024303, 75.73093610866914, 75.84153361880725, 75.89257862348637, 75.93228029379236, 76.01735530159091, 76.10810197657601, 76.15347531406856, 76.19034115078126, 76.22720698749397, 76.26974449139324, 76.2952669937328, 76.27258032498652, 76.28108782576638, 76.27258032498652, 76.28392365935966, 76.29810282732609, 76.30377449451267, 76.2952669937328, 76.28108782576638, 76.24422198905368, 76.24705782264697, 76.23571448827381, 76.26407282420668, 76.24989365624025, 76.23287865468053, 76.2016844851544, 76.16481864844171, 76.11660947735587, 76.08257947423644, 76.02302696877747, 75.96063862972521, 75.92944446019906, 75.87556362192667, 75.84436945240053, 75.83019028443411, 75.80750361568784, 75.77347361256841, 75.73660777585572, 75.70824943992285, 75.67138360321016, 75.64302526727731, 75.61750276493775, 75.5891444290049, 75.57496526103849, 75.52108442276607, 75.48138275246008, 75.46436775090038, 75.45302441652724, 75.43884524856081, 75.4048152454414, 75.37645690950855, 75.35093440716899, 75.33959107279584, 75.311232736863, 75.30272523608315, 75.29138190171001, 75.28287440093015, 75.22899356265775, 75.19496355953832, 75.15809772282563, 75.11272438533307, 75.09854521736663, 75.06735104784052, 75.03899271190765, 74.99078354082182, 74.92555936817628, 74.87735019709044, 74.79794685647846, 74.7355585174262, 74.63914017525451, 74.47182599325072], "calls": [0.0, 2.4416527238181662, 5.376740492867879, 7.897796557298017, 10.101239259280266, 11.601395230127896, 12.90587868303888, 14.286929642968552, 15.741712276323625, 16.972464055809205, 18.345007514959022, 19.547400958511755, 20.724271899724926, 21.847262002665683, 23.00428210872586, 24.096078042140487, 25.128321470096132, 26.308028244902587, 27.428182514250064, 28.701471797634916, 29.813118566202533, 30.899242832430595, 32.15268128066245, 33.326716388282335, 34.8382156935031, 36.094489975328244, 37.4783767688512, 38.93599523579957, 40.473017043359896, 41.98168051498738, 43.4648214842753, 45.07273913166775, 46.555880100955676, 48.163797748348124, 49.87096957150555, 51.55829055950997, 52.908147349913506, 54.49621416215297, 55.8857726228625, 57.34906275699741, 58.77265122082637, 60.13101551200975, 61.41848396336103, 62.92147576780195, 64.06715253948899, 65.66372685250829, 66.90298613277373, 68.10537957632647, 69.45807220032329, 70.70300314777529, 71.56226072654057, 72.49808581232453, 73.59271757933244, 74.6079460057284, 75.39063607747497, 76.25272948983354, 76.93900121940844, 77.61960128179678, 78.32855968011798, 79.12542891983098, 79.83438731815217, 80.64259989223832, 81.26364744916769, 81.85917250375748, 82.48589172787341, 83.11828261917591, 83.696792672206, 84.21858605337039, 84.63261775798996, 85.14023197118793, 85.73008535859115, 86.22919207100928, 86.82188129200578, 87.23307716303206, 87.6953180387375, 88.1008422425772, 88.52905311516321, 89.03666732836118, 89.43651986501432, 89.92144740946601, 90.25607577347361, 90.66159997731333, 91.00190000850749, 91.49249922014576, 91.90936675835863, 92.40847347077673, 92.84519184414258, 93.34713439015398, 93.78101692992655, 94.2205711368857, 94.73385701727022, 95.15072455548308, 95.57609959447579, 96.08654964126706, 96.66222386070386, 97.31162975356607, 97.88446813940958, 98.5310381986785, 99.00178657516378, 99.43850494852961, 100.0], "rescued": [0, 503, 923, 1248, 1528, 1721, 1895, 2056, 2215, 2350, 2509, 2625, 2747, 2871, 2988, 3080, 3162, 3248, 3329, 3413, 3492, 3566, 3645, 3723, 3806, 3855, 3915, 3975, 4047, 4113, 4172, 4245, 4302, 4349, 4420, 4483, 4531, 4582, 4627, 4672, 4716, 4748, 4784, 4812, 4842, 4883, 4914, 4941, 4975, 5005, 5026, 5046, 5080, 5115, 5132, 5148, 5163, 5176, 5193, 5209, 5215, 5227, 5242, 5258, 5272, 5289, 5297, 5305, 5310, 5315, 5324, 5334, 5347, 5356, 5359, 5362, 5365, 5373, 5377, 5378, 5380, 5383, 5385, 5387, 5389, 5395, 5399, 5402, 5403, 5407, 5409, 5412, 5414, 5415, 5417, 5420, 5421, 5421, 5422, 5422, 5423], "harmed": [0, 15, 30, 46, 66, 95, 122, 149, 193, 234, 282, 343, 399, 434, 476, 524, 569, 610, 648, 688, 728, 764, 798, 822, 869, 903, 941, 978, 1010, 1041, 1076, 1110, 1149, 1182, 1223, 1254, 1286, 1324, 1356, 1386, 1421, 1461, 1494, 1525, 1551, 1587, 1616, 1646, 1685, 1728, 1748, 1772, 1796, 1836, 1859, 1886, 1914, 1944, 1973, 2010, 2038, 2061, 2095, 2122, 2141, 2166, 2186, 2207, 2222, 2240, 2259, 2278, 2301, 2315, 2337, 2354, 2363, 2375, 2384, 2397, 2409, 2421, 2427, 2439, 2444, 2454, 2461, 2483, 2496, 2513, 2531, 2539, 2552, 2563, 2582, 2608, 2626, 2654, 2677, 2711, 2771], "exact_score": [66.95119530385956, 67.60627286390834, 68.1535887474123, 68.58463545359157, 69.01851799336414, 69.34747469018518, 69.70478972293905, 70.05926892209965, 70.38538978532739, 70.72001814933499, 71.05181067974931, 71.36658820860391, 71.69554490542495, 72.01315826787284, 72.2995774607946, 72.5434591498171, 72.75331083572016, 72.95749085443666, 73.21555171142558, 73.35734339108981, 73.5246575730936, 73.6749567535377, 73.8989876074072, 74.07197345659756, 74.17122763236253, 74.2704818081275, 74.40376598701188, 74.51436349714999, 74.60511017213508, 74.72137934945977, 74.81212602444488, 74.91705186739642, 75.01914187675467, 75.11272438533307, 75.22899356265775, 75.28854606811673, 75.31406857045629, 75.37078524232199, 75.41899441340783, 75.45586025012052, 75.51257692198622, 75.60615943056462, 75.6628761024303, 75.71392110710944, 75.73660777585572, 75.83302611802739, 75.86422028755354, 75.90392195785951, 75.92660862660578, 75.98899696565805, 76.02019113518419, 76.08825114142302, 76.11377364376258, 76.15063948047528, 76.16765448203499, 76.21019198593426, 76.23855032186711, 76.2584011570201, 76.2952669937328, 76.28959532654623, 76.26974449139324, 76.2782519921731, 76.27258032498652, 76.27541615857982, 76.2782519921731, 76.28675949295295, 76.30377449451267, 76.29810282732609, 76.30661032810595, 76.27541615857982, 76.26974449139324, 76.23855032186711, 76.24705782264697, 76.2413861554604, 76.24705782264697, 76.24705782264697, 76.2045203187477, 76.16481864844171, 76.09108697501631, 76.02869863596405, 75.9407877945722, 75.90108612426623, 75.84153361880725, 75.81601111646769, 75.76496611178855, 75.67989110399002, 75.61466693134446, 75.58347276181834, 75.49272608683322, 75.4586960837138, 75.38780024388168, 75.33959107279584, 75.29988940248987, 75.26869523296374, 75.16944105719875, 75.11556021892635, 75.04182854550095, 74.95958937129569, 74.87167852990386, 74.7355585174262, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.64040911488723, "threshold_area100": 75.049400221195, "threshold_area25": 72.99095369083743}, "selected_budget": 43, "selected": {"val": {"score": 84.01855365802236, "calls": 42.99669688663996, "rescued": 1797, "harmed": 518}, "test": {"score": 76.27258032498652, "calls": 62.92147576780195, "rescued": 4812, "harmed": 1525}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5338249802589417, "operator": "ge"}, {"budget": 0.02, "threshold": 0.27718859910964966, "operator": "ge"}, {"budget": 0.03, "threshold": 0.12145932018756866, "operator": "ge"}, {"budget": 0.04, "threshold": -0.0012431842042133212, "operator": "ge"}, {"budget": 0.05, "threshold": -0.005379714537411928, "operator": "ge"}, {"budget": 0.06, "threshold": -0.008768636733293533, "operator": "ge"}, {"budget": 0.07, "threshold": -0.012893827632069588, "operator": "ge"}, {"budget": 0.08, "threshold": -0.016521969810128212, "operator": "ge"}, {"budget": 0.09, "threshold": -0.020378829911351204, "operator": "ge"}, {"budget": 0.1, "threshold": -0.024623069912195206, "operator": "ge"}, {"budget": 0.11, "threshold": -0.028061797842383385, "operator": "ge"}, {"budget": 0.12, "threshold": -0.031991321593523026, "operator": "ge"}, {"budget": 0.13, "threshold": -0.035478729754686356, "operator": "ge"}, {"budget": 0.14, "threshold": -0.03950003534555435, "operator": "ge"}, {"budget": 0.15, "threshold": -0.04366030916571617, "operator": "ge"}, {"budget": 0.16, "threshold": -0.048041932284832, "operator": "ge"}, {"budget": 0.17, "threshold": -0.05180920660495758, "operator": "ge"}, {"budget": 0.18, "threshold": -0.05590024217963219, "operator": "ge"}, {"budget": 0.19, "threshold": -0.05974539741873741, "operator": "ge"}, {"budget": 0.2, "threshold": -0.0638163611292839, "operator": "ge"}, {"budget": 0.21, "threshold": -0.06845870614051819, "operator": "ge"}, {"budget": 0.22, "threshold": -0.07260140776634216, "operator": "ge"}, {"budget": 0.23, "threshold": -0.07683342695236206, "operator": "ge"}, {"budget": 0.24, "threshold": -0.08063622564077377, "operator": "ge"}, {"budget": 0.25, "threshold": -0.08506232500076294, "operator": "ge"}, {"budget": 0.26, "threshold": -0.08921544253826141, "operator": "ge"}, {"budget": 0.27, "threshold": -0.09385530650615692, "operator": "ge"}, {"budget": 0.28, "threshold": -0.09829162806272507, "operator": "ge"}, {"budget": 0.29, "threshold": -0.10227511823177338, "operator": "ge"}, {"budget": 0.3, "threshold": -0.1074746623635292, "operator": "ge"}, {"budget": 0.31, "threshold": -0.11233928799629211, "operator": "ge"}, {"budget": 0.32, "threshold": -0.11637624353170395, "operator": "gt"}, {"budget": 0.33, "threshold": -0.12126876413822174, "operator": "ge"}, {"budget": 0.34, "threshold": -0.12595608830451965, "operator": "ge"}, {"budget": 0.35, "threshold": -0.13055381178855896, "operator": "ge"}, {"budget": 0.36, "threshold": -0.1355379968881607, "operator": "ge"}, {"budget": 0.37, "threshold": -0.13995786011219025, "operator": "ge"}, {"budget": 0.38, "threshold": -0.14579394459724426, "operator": "ge"}, {"budget": 0.39, "threshold": -0.1511736959218979, "operator": "ge"}, {"budget": 0.4, "threshold": -0.1562771201133728, "operator": "ge"}, {"budget": 0.41, "threshold": -0.1615424007177353, "operator": "ge"}, {"budget": 0.42, "threshold": -0.16686685383319855, "operator": "ge"}, {"budget": 0.43, "threshold": -0.17225851118564606, "operator": "ge"}, {"budget": 0.44, "threshold": -0.17708756029605865, "operator": "ge"}, {"budget": 0.45, "threshold": -0.1829516440629959, "operator": "ge"}, {"budget": 0.46, "threshold": -0.18876895308494568, "operator": "ge"}, {"budget": 0.47, "threshold": -0.195005401968956, "operator": "ge"}, {"budget": 0.48, "threshold": -0.2010069638490677, "operator": "gt"}, {"budget": 0.49, "threshold": -0.20757600665092468, "operator": "ge"}, {"budget": 0.5, "threshold": -0.21369366347789764, "operator": "ge"}, {"budget": 0.51, "threshold": -0.2198004275560379, "operator": "ge"}, {"budget": 0.52, "threshold": -0.22662988305091858, "operator": "ge"}, {"budget": 0.53, "threshold": -0.23318900167942047, "operator": "ge"}, {"budget": 0.54, "threshold": -0.23906785249710083, "operator": "ge"}, {"budget": 0.55, "threshold": -0.24544557929039001, "operator": "ge"}, {"budget": 0.56, "threshold": -0.25294092297554016, "operator": "ge"}, {"budget": 0.57, "threshold": -0.2598547339439392, "operator": "ge"}, {"budget": 0.58, "threshold": -0.2675437927246094, "operator": "ge"}, {"budget": 0.59, "threshold": -0.275374174118042, "operator": "ge"}, {"budget": 0.6, "threshold": -0.2831237018108368, "operator": "ge"}, {"budget": 0.61, "threshold": -0.2906046211719513, "operator": "ge"}, {"budget": 0.62, "threshold": -0.3002321720123291, "operator": "ge"}, {"budget": 0.63, "threshold": -0.307657927274704, "operator": "gt"}, {"budget": 0.64, "threshold": -0.31593143939971924, "operator": "ge"}, {"budget": 0.65, "threshold": -0.32411670684814453, "operator": "ge"}, {"budget": 0.66, "threshold": -0.3325658440589905, "operator": "ge"}, {"budget": 0.67, "threshold": -0.34061115980148315, "operator": "ge"}, {"budget": 0.68, "threshold": -0.34942710399627686, "operator": "ge"}, {"budget": 0.69, "threshold": -0.3573535680770874, "operator": "ge"}, {"budget": 0.7, "threshold": -0.3659776747226715, "operator": "ge"}, {"budget": 0.71, "threshold": -0.375048965215683, "operator": "ge"}, {"budget": 0.72, "threshold": -0.3827228546142578, "operator": "ge"}, {"budget": 0.73, "threshold": -0.3913613557815552, "operator": "ge"}, {"budget": 0.74, "threshold": -0.40208443999290466, "operator": "ge"}, {"budget": 0.75, "threshold": -0.411685973405838, "operator": "ge"}, {"budget": 0.76, "threshold": -0.4223134219646454, "operator": "ge"}, {"budget": 0.77, "threshold": -0.4326930344104767, "operator": "ge"}, {"budget": 0.78, "threshold": -0.4409719407558441, "operator": "ge"}, {"budget": 0.79, "threshold": -0.45199209451675415, "operator": "ge"}, {"budget": 0.8, "threshold": -0.4627479314804077, "operator": "ge"}, {"budget": 0.81, "threshold": -0.474081814289093, "operator": "ge"}, {"budget": 0.82, "threshold": -0.48352551460266113, "operator": "ge"}, {"budget": 0.83, "threshold": -0.4945851266384125, "operator": "ge"}, {"budget": 0.84, "threshold": -0.505406379699707, "operator": "ge"}, {"budget": 0.85, "threshold": -0.5173789262771606, "operator": "ge"}, {"budget": 0.86, "threshold": -0.5311217904090881, "operator": "ge"}, {"budget": 0.87, "threshold": -0.5436577200889587, "operator": "ge"}, {"budget": 0.88, "threshold": -0.5575807094573975, "operator": "ge"}, {"budget": 0.89, "threshold": -0.5699482560157776, "operator": "ge"}, {"budget": 0.9, "threshold": -0.5847934484481812, "operator": "ge"}, {"budget": 0.91, "threshold": -0.5976582169532776, "operator": "ge"}, {"budget": 0.92, "threshold": -0.614446759223938, "operator": "ge"}, {"budget": 0.93, "threshold": -0.6342154145240784, "operator": "ge"}, {"budget": 0.94, "threshold": -0.6534122824668884, "operator": "ge"}, {"budget": 0.95, "threshold": -0.6750573515892029, "operator": "ge"}, {"budget": 0.96, "threshold": -0.70353102684021, "operator": "ge"}, {"budget": 0.97, "threshold": -0.7309830784797668, "operator": "gt"}, {"budget": 0.98, "threshold": -0.7694240808486938, "operator": "ge"}, {"budget": 0.99, "threshold": -0.8216003179550171, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.56398903647481, 75.97863518167124, 76.38625342610162, 76.5197835406564, 76.51275563989036, 76.53383934218849, 76.54086724295453, 76.61817415138098, 76.64628575444516, 76.73764846440369, 76.78684376976597, 76.87820647972451, 76.92037388432075, 76.94848548738491, 77.00470869351325, 77.0468760981095, 77.07498770117365, 77.11012720500386, 77.15932251036615, 77.20148991496241, 77.28582472415489, 77.34907583104926, 77.44746644177386, 77.48260594560405, 77.52477335020029, 77.58099655632863, 77.62316396092487, 77.644247663223, 77.70047086935132, 77.74966617471361, 77.77777777777779, 77.82697308314008, 77.89725209080048, 77.98158689999298, 78.0026706022911, 78.07997751071754, 78.11511701454775, 78.157284419144, 78.17836812144212, 78.26270293063462, 78.31892613676294, 78.3610935413592, 78.43137254901961, 78.45245625131774, 78.46651205284981, 78.5508468620423, 78.6000421674046, 78.59301426663856, 78.6000421674046, 78.59301426663856, 78.64923747276688, 78.66329327429898, 78.67734907583105, 78.670321175065, 78.7195164804273, 78.76168388502354, 78.76871178578959, 78.81087919038583, 78.8741302972802, 78.89521399957833, 78.95143720570665, 79.04982781643123, 79.12010682409164, 79.18335793098602, 79.2676927401785, 79.23255323634831, 79.2676927401785, 79.32391594630684, 79.39419495396724, 79.46447396162766, 79.60503197694848, 79.66125518307682, 79.62611567924661, 79.73153419073724, 79.7455899922693, 79.81586899992972, 79.88614800759012, 80.04076182444304, 79.97751071754867, 80.05481762597513, 80.08995712980533, 80.23051514512616, 80.24457094665823, 80.27971045048844, 80.27971045048844, 80.29376625202053, 80.34998945814885, 80.31484995431865, 80.33593365661676, 80.36404525968094, 80.3921568627451, 80.46243587040551, 80.52568697729988, 80.59596598496029, 80.59596598496029, 80.62407758802446, 80.73652400028112, 80.6803007941528, 80.70138449645091, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99100428701947, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.986506430529204, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99107456602713, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99205847213437, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 80, 144, 209, 238, 240, 244, 251, 264, 273, 287, 299, 314, 324, 331, 343, 352, 358, 365, 372, 380, 395, 409, 426, 434, 446, 457, 466, 478, 488, 497, 507, 517, 530, 545, 554, 569, 582, 592, 603, 618, 633, 648, 662, 672, 685, 703, 718, 725, 740, 748, 761, 775, 788, 801, 816, 830, 845, 860, 881, 895, 917, 945, 965, 988, 1009, 1025, 1042, 1062, 1086, 1113, 1141, 1169, 1191, 1220, 1241, 1266, 1299, 1330, 1350, 1375, 1400, 1432, 1456, 1484, 1512, 1536, 1566, 1590, 1616, 1648, 1674, 1706, 1738, 1775, 1801, 1831, 1875, 1905, 1940, 1964], "harmed": [0, 4, 9, 16, 26, 29, 30, 36, 38, 43, 44, 49, 51, 55, 58, 62, 65, 67, 69, 69, 71, 74, 79, 82, 85, 91, 94, 97, 106, 108, 110, 116, 119, 122, 125, 131, 135, 143, 147, 155, 158, 165, 174, 178, 185, 196, 202, 210, 218, 232, 241, 246, 258, 269, 283, 291, 299, 313, 322, 334, 345, 359, 373, 383, 397, 406, 427, 439, 451, 465, 482, 490, 510, 537, 551, 570, 585, 608, 617, 646, 660, 680, 692, 714, 737, 765, 787, 809, 838, 861, 889, 911, 933, 956, 983, 1009, 1035, 1063, 1101, 1133, 1171], "exact_score": [75.02986857825567, 75.56398903647481, 75.97863518167124, 76.38625342610162, 76.5197835406564, 76.51275563989036, 76.53383934218849, 76.54086724295453, 76.61817415138098, 76.64628575444516, 76.73764846440369, 76.78684376976597, 76.87820647972451, 76.92037388432075, 76.94848548738491, 77.00470869351325, 77.0468760981095, 77.07498770117365, 77.11012720500386, 77.15932251036615, 77.20148991496241, 77.28582472415489, 77.34907583104926, 77.44746644177386, 77.48260594560405, 77.52477335020029, 77.58099655632863, 77.62316396092487, 77.644247663223, 77.70047086935132, 77.74966617471361, 77.77777777777779, 77.82697308314008, 77.89725209080048, 77.98158689999298, 78.0026706022911, 78.07997751071754, 78.11511701454775, 78.157284419144, 78.17836812144212, 78.26270293063462, 78.31892613676294, 78.3610935413592, 78.43137254901961, 78.45245625131774, 78.46651205284981, 78.5508468620423, 78.6000421674046, 78.59301426663856, 78.6000421674046, 78.59301426663856, 78.64923747276688, 78.66329327429898, 78.67734907583105, 78.670321175065, 78.7195164804273, 78.76168388502354, 78.76871178578959, 78.81087919038583, 78.8741302972802, 78.89521399957833, 78.95143720570665, 79.04982781643123, 79.12010682409164, 79.18335793098602, 79.2676927401785, 79.23255323634831, 79.2676927401785, 79.32391594630684, 79.39419495396724, 79.46447396162766, 79.60503197694848, 79.66125518307682, 79.62611567924661, 79.73153419073724, 79.7455899922693, 79.81586899992972, 79.88614800759012, 80.04076182444304, 79.97751071754867, 80.05481762597513, 80.08995712980533, 80.23051514512616, 80.24457094665823, 80.27971045048844, 80.27971045048844, 80.29376625202053, 80.34998945814885, 80.31484995431865, 80.33593365661676, 80.36404525968094, 80.3921568627451, 80.46243587040551, 80.52568697729988, 80.59596598496029, 80.59596598496029, 80.62407758802446, 80.74355190104716, 80.6803007941528, 80.70138449645091, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 78.58634963953992, "threshold_area100": 78.58609178438401, "threshold_area25": 76.74678473539954}, "test": {"score": [66.95119530385956, 68.38896293565493, 69.61971471514052, 70.3655389501744, 70.96389983835748, 71.00927317585004, 71.04897484615603, 71.07733318208888, 71.10285568442843, 71.13972152114114, 71.18225902504041, 71.24748319768595, 71.28151320080538, 71.32688653829793, 71.38360321016363, 71.40061821172333, 71.48852905311516, 71.51688738904801, 71.56509656013384, 71.59629072965998, 71.6473357343391, 71.6870374046451, 71.7437540765108, 71.77494824603691, 71.77778407963021, 71.80330658196978, 71.81181408274962, 71.85718742024217, 71.89688909054817, 71.92241159288773, 71.97629243116013, 72.01032243427954, 72.05002410458555, 72.06703910614524, 72.12375577801095, 72.12375577801095, 72.12659161160423, 72.15778578113036, 72.19465161784306, 72.25420412330205, 72.25136828970876, 72.2825624592349, 72.32226412954088, 72.35912996625358, 72.40166747015286, 72.43569747327227, 72.44987664123869, 72.5037574795111, 72.58032498652979, 72.64838499276863, 72.65972832714176, 72.69942999744775, 72.72211666619403, 72.75047500212686, 72.77883333805973, 72.8185350083657, 72.88943084819782, 72.9688341888098, 73.03689419504863, 73.0794316989479, 73.1361483708136, 73.209880044239, 73.30062671922411, 73.34600005671668, 73.3970450613958, 73.44809006607493, 73.55018007543318, 73.60406091370558, 73.66361341916456, 73.68630008791084, 73.7515242605564, 73.79406176445566, 73.91033094178033, 73.94719677849304, 74.046450954258, 74.12301846127669, 74.11167512690355, 74.16272013158267, 74.15137679720954, 74.1598842979894, 74.26197430734764, 74.28466097609392, 74.34137764795962, 74.31301931202677, 74.36406431670589, 74.40660182060516, 74.42928848935144, 74.44063182372457, 74.474661826844, 74.54272183308284, 74.60794600572838, 74.66182684400079, 74.69585684712021, 74.65615517681422, 74.66182684400079, 74.67884184556051, 74.6589910104075, 74.66749851118736, 74.64764767603437, 74.55406516745597, 74.47182599325072], "calls": [0.0, 2.555086067549556, 5.691518021722485, 7.931826560417435, 10.033179253041432, 11.71766440745257, 13.359612057964437, 15.177381391259962, 16.870374046450955, 18.73351671723903, 20.48889771148229, 21.94084451124408, 23.61965799846865, 25.088619799790145, 26.61146243938406, 28.037886736806282, 29.59475937951961, 30.69506281371409, 31.852082919774265, 32.84462467742393, 33.80313643195417, 34.965828205200914, 35.87613078864532, 36.83464254317557, 37.62584011570201, 38.621217706944954, 39.47196778493038, 40.50137537929274, 41.38899129399087, 42.25108470634943, 43.328701471797636, 44.35810906616, 45.14930663868644, 46.031250886198, 46.924538468082694, 47.73275104216885, 48.56648611859456, 49.218727845050054, 50.1573887644273, 51.03082551115901, 51.75112724385333, 52.4629214757678, 53.24561154751439, 53.95173411224229, 54.68621501290304, 55.39233757763095, 56.12114681110513, 56.838612710206164, 57.632646116325894, 58.5230978646173, 59.16683209029294, 59.87862632220742, 60.70952556503985, 61.52057397271928, 62.118934860902364, 62.881774097495956, 63.811927516093355, 64.60596092221309, 65.41133766270595, 66.23940107194511, 67.04761364603125, 67.72254204123303, 68.68956129654312, 69.40986302923744, 70.27762810878258, 71.15106485551428, 71.99614326631313, 72.75614666931345, 73.57286674417945, 74.27331764172078, 75.06167938065394, 75.88974278989309, 76.63840285852027, 77.48631710291241, 78.44482885744264, 79.25587726512208, 80.16617984856649, 80.90066074922724, 81.575589144429, 82.35544338258231, 83.16932762385503, 83.9548535291949, 84.63261775798996, 85.42097949692312, 86.0760570569719, 86.85023962793863, 87.77755721294275, 88.62830729092816, 89.45637070016731, 90.16249326489522, 91.0217508436605, 91.7647392451011, 92.59280265434025, 93.5683294104302, 94.37087031732977, 95.24997873124805, 96.33326716388282, 97.23789808014067, 98.18223066670447, 99.16626492357429, 100.0], "rescued": [0, 521, 966, 1250, 1481, 1499, 1518, 1537, 1558, 1578, 1601, 1627, 1652, 1677, 1702, 1720, 1757, 1777, 1803, 1825, 1848, 1874, 1903, 1921, 1937, 1959, 1977, 2010, 2032, 2058, 2089, 2117, 2144, 2168, 2195, 2215, 2238, 2259, 2294, 2326, 2341, 2367, 2400, 2429, 2454, 2486, 2516, 2554, 2596, 2641, 2666, 2701, 2735, 2778, 2808, 2848, 2903, 2956, 3006, 3053, 3102, 3147, 3210, 3253, 3308, 3362, 3428, 3479, 3538, 3582, 3639, 3696, 3764, 3823, 3897, 3963, 4010, 4064, 4102, 4153, 4226, 4285, 4333, 4386, 4441, 4501, 4567, 4620, 4673, 4727, 4795, 4852, 4916, 4972, 5028, 5093, 5170, 5233, 5302, 5367, 5423], "harmed": [0, 14, 25, 46, 66, 68, 73, 82, 94, 101, 109, 112, 125, 134, 139, 151, 157, 167, 176, 187, 192, 204, 213, 220, 235, 248, 263, 280, 288, 305, 317, 333, 346, 364, 371, 391, 413, 423, 445, 456, 472, 487, 506, 522, 532, 552, 577, 596, 611, 632, 653, 674, 700, 733, 753, 779, 809, 834, 860, 892, 921, 940, 971, 998, 1035, 1071, 1101, 1133, 1171, 1207, 1241, 1283, 1310, 1356, 1395, 1434, 1485, 1521, 1563, 1611, 1648, 1699, 1727, 1790, 1827, 1872, 1930, 1979, 2020, 2050, 2095, 2133, 2185, 2255, 2309, 2368, 2452, 2512, 2588, 2686, 2771], "exact_score": [66.95119530385956, 67.58075036156879, 68.12239457788617, 68.60732212233785, 68.99299549102459, 69.35881802455832, 69.71896889090547, 70.05076142131979, 70.39106145251397, 70.7540481524544, 70.96673567195076, 70.99509400788362, 71.01210900944332, 71.04613901256273, 71.06882568130902, 71.07733318208888, 71.0744973484956, 71.10001985083515, 71.12270651958143, 71.15106485551428, 71.1652440234807, 71.20494569378668, 71.24748319768595, 71.27584153361882, 71.30703570314493, 71.32405070470465, 71.35808070782407, 71.37509570938377, 71.39778237813005, 71.44031988202933, 71.49136488670844, 71.52823072342115, 71.56793239372713, 71.60196239684655, 71.65584323511897, 71.6870374046451, 71.7437540765108, 71.77494824603691, 71.78345574681677, 71.80330658196978, 71.85718742024217, 71.88838158976831, 71.90823242492131, 71.97062076397357, 71.9989790999064, 72.05285993817883, 72.06420327255196, 72.10957661004453, 72.1322632787908, 72.14360661316394, 72.18330828346993, 72.25703995689533, 72.26838329126846, 72.30808496157445, 72.37047330062671, 72.4101749709327, 72.43853330686555, 72.52360831466409, 72.61151915605592, 72.64838499276863, 72.70510166463431, 72.73062416697388, 72.76749000368659, 72.82987834273884, 72.90361001616425, 73.02271502708221, 73.06808836457476, 73.12764087003374, 73.24107421376513, 73.3204775543771, 73.38853756061594, 73.42256756373536, 73.55018007543318, 73.6012250801123, 73.66928508635114, 73.7515242605564, 73.80256926523552, 73.93018177693332, 74.00674928395202, 74.09749595893713, 74.10883929331027, 74.16839179876925, 74.15137679720954, 74.24779513938122, 74.29316847687377, 74.3243626463999, 74.37257181748575, 74.4207809885716, 74.44346765731787, 74.45764682528429, 74.53988599948956, 74.6022743385418, 74.66749851118736, 74.69302101352693, 74.66182684400079, 74.65331934322094, 74.6816776791538, 74.69018517993365, 74.65048350962765, 74.57675183620225, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 72.35134865375457, "threshold_area100": 72.75912429458639, "threshold_area25": 70.92334741797352}, "selected_budget": 97, "selected": {"val": {"score": 80.73652400028112, "calls": 96.99205847213437, "rescued": 1875, "harmed": 1063}, "test": {"score": 74.66749851118736, "calls": 97.23789808014067, "rescued": 5233, "harmed": 2512}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.1807010173797607, "operator": "ge"}, {"budget": 0.02, "threshold": 1.7016446590423584, "operator": "ge"}, {"budget": 0.03, "threshold": 0.7272830009460449, "operator": "gt"}, {"budget": 0.04, "threshold": -0.149221271276474, "operator": "ge"}, {"budget": 0.05, "threshold": -0.781380295753479, "operator": "ge"}, {"budget": 0.06, "threshold": -1.200270175933838, "operator": "ge"}, {"budget": 0.07, "threshold": -1.6254407167434692, "operator": "ge"}, {"budget": 0.08, "threshold": -1.9138612747192383, "operator": "ge"}, {"budget": 0.09, "threshold": -2.150641441345215, "operator": "ge"}, {"budget": 0.1, "threshold": -2.396206855773926, "operator": "ge"}, {"budget": 0.11, "threshold": -2.640716075897217, "operator": "ge"}, {"budget": 0.12, "threshold": -2.869621992111206, "operator": "ge"}, {"budget": 0.13, "threshold": -3.0810179710388184, "operator": "ge"}, {"budget": 0.14, "threshold": -3.266911506652832, "operator": "ge"}, {"budget": 0.15, "threshold": -3.4335062503814697, "operator": "ge"}, {"budget": 0.16, "threshold": -3.5806851387023926, "operator": "ge"}, {"budget": 0.17, "threshold": -3.7310752868652344, "operator": "ge"}, {"budget": 0.18, "threshold": -3.853217601776123, "operator": "ge"}, {"budget": 0.19, "threshold": -3.9914772510528564, "operator": "ge"}, {"budget": 0.2, "threshold": -4.104372978210449, "operator": "ge"}, {"budget": 0.21, "threshold": -4.199089527130127, "operator": "ge"}, {"budget": 0.22, "threshold": -4.306891441345215, "operator": "ge"}, {"budget": 0.23, "threshold": -4.408714771270752, "operator": "ge"}, {"budget": 0.24, "threshold": -4.505574703216553, "operator": "ge"}, {"budget": 0.25, "threshold": -4.6044793128967285, "operator": "ge"}, {"budget": 0.26, "threshold": -4.699550628662109, "operator": "ge"}, {"budget": 0.27, "threshold": -4.7818498611450195, "operator": "ge"}, {"budget": 0.28, "threshold": -4.870875358581543, "operator": "ge"}, {"budget": 0.29, "threshold": -4.958505153656006, "operator": "ge"}, {"budget": 0.3, "threshold": -5.041875839233398, "operator": "ge"}, {"budget": 0.31, "threshold": -5.133337020874023, "operator": "ge"}, {"budget": 0.32, "threshold": -5.208829879760742, "operator": "ge"}, {"budget": 0.33, "threshold": -5.291959285736084, "operator": "ge"}, {"budget": 0.34, "threshold": -5.372551918029785, "operator": "ge"}, {"budget": 0.35, "threshold": -5.452914714813232, "operator": "ge"}, {"budget": 0.36, "threshold": -5.523379325866699, "operator": "ge"}, {"budget": 0.37, "threshold": -5.598590850830078, "operator": "ge"}, {"budget": 0.38, "threshold": -5.667969703674316, "operator": "ge"}, {"budget": 0.39, "threshold": -5.730137825012207, "operator": "ge"}, {"budget": 0.4, "threshold": -5.808537006378174, "operator": "ge"}, {"budget": 0.41, "threshold": -5.870964050292969, "operator": "ge"}, {"budget": 0.42, "threshold": -5.9404072761535645, "operator": "ge"}, {"budget": 0.43, "threshold": -6.008207321166992, "operator": "ge"}, {"budget": 0.44, "threshold": -6.0730671882629395, "operator": "gt"}, {"budget": 0.45, "threshold": -6.132779598236084, "operator": "ge"}, {"budget": 0.46, "threshold": -6.203226089477539, "operator": "ge"}, {"budget": 0.47, "threshold": -6.258633613586426, "operator": "ge"}, {"budget": 0.48, "threshold": -6.316098690032959, "operator": "ge"}, {"budget": 0.49, "threshold": -6.389966011047363, "operator": "ge"}, {"budget": 0.5, "threshold": -6.45597505569458, "operator": "ge"}, {"budget": 0.51, "threshold": -6.513321399688721, "operator": "ge"}, {"budget": 0.52, "threshold": -6.5777506828308105, "operator": "ge"}, {"budget": 0.53, "threshold": -6.632788181304932, "operator": "ge"}, {"budget": 0.54, "threshold": -6.689544677734375, "operator": "ge"}, {"budget": 0.55, "threshold": -6.7423095703125, "operator": "ge"}, {"budget": 0.56, "threshold": -6.80186128616333, "operator": "ge"}, {"budget": 0.57, "threshold": -6.867194175720215, "operator": "ge"}, {"budget": 0.58, "threshold": -6.927822113037109, "operator": "ge"}, {"budget": 0.59, "threshold": -6.9989423751831055, "operator": "ge"}, {"budget": 0.6, "threshold": -7.06195068359375, "operator": "ge"}, {"budget": 0.61, "threshold": -7.124990940093994, "operator": "ge"}, {"budget": 0.62, "threshold": -7.182358741760254, "operator": "ge"}, {"budget": 0.63, "threshold": -7.245880126953125, "operator": "ge"}, {"budget": 0.64, "threshold": -7.30183219909668, "operator": "ge"}, {"budget": 0.65, "threshold": -7.3513054847717285, "operator": "ge"}, {"budget": 0.66, "threshold": -7.403548717498779, "operator": "ge"}, {"budget": 0.67, "threshold": -7.465986251831055, "operator": "ge"}, {"budget": 0.68, "threshold": -7.512401580810547, "operator": "ge"}, {"budget": 0.69, "threshold": -7.568910598754883, "operator": "ge"}, {"budget": 0.7, "threshold": -7.621840000152588, "operator": "ge"}, {"budget": 0.71, "threshold": -7.67211389541626, "operator": "ge"}, {"budget": 0.72, "threshold": -7.723695278167725, "operator": "ge"}, {"budget": 0.73, "threshold": -7.769769191741943, "operator": "ge"}, {"budget": 0.74, "threshold": -7.816502094268799, "operator": "ge"}, {"budget": 0.75, "threshold": -7.858847618103027, "operator": "ge"}, {"budget": 0.76, "threshold": -7.899866104125977, "operator": "ge"}, {"budget": 0.77, "threshold": -7.941775798797607, "operator": "ge"}, {"budget": 0.78, "threshold": -7.984594821929932, "operator": "ge"}, {"budget": 0.79, "threshold": -8.021862983703613, "operator": "ge"}, {"budget": 0.8, "threshold": -8.064108848571777, "operator": "ge"}, {"budget": 0.81, "threshold": -8.110682487487793, "operator": "ge"}, {"budget": 0.82, "threshold": -8.151888847351074, "operator": "ge"}, {"budget": 0.83, "threshold": -8.196581840515137, "operator": "ge"}, {"budget": 0.84, "threshold": -8.241005897521973, "operator": "ge"}, {"budget": 0.85, "threshold": -8.28570556640625, "operator": "ge"}, {"budget": 0.86, "threshold": -8.328243255615234, "operator": "ge"}, {"budget": 0.87, "threshold": -8.370985984802246, "operator": "ge"}, {"budget": 0.88, "threshold": -8.413557052612305, "operator": "ge"}, {"budget": 0.89, "threshold": -8.45538330078125, "operator": "ge"}, {"budget": 0.9, "threshold": -8.497403144836426, "operator": "ge"}, {"budget": 0.91, "threshold": -8.544747352600098, "operator": "ge"}, {"budget": 0.92, "threshold": -8.606725692749023, "operator": "ge"}, {"budget": 0.93, "threshold": -8.655834197998047, "operator": "ge"}, {"budget": 0.94, "threshold": -8.717656135559082, "operator": "ge"}, {"budget": 0.95, "threshold": -8.7861328125, "operator": "ge"}, {"budget": 0.96, "threshold": -8.86816692352295, "operator": "ge"}, {"budget": 0.97, "threshold": -8.96171760559082, "operator": "ge"}, {"budget": 0.98, "threshold": -9.059114456176758, "operator": "ge"}, {"budget": 0.99, "threshold": -9.268800735473633, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.52182163187857, 75.98566308243728, 76.35111392227142, 76.54086724295453, 76.75170426593576, 76.93442968585283, 77.18743411343031, 77.41232693794365, 77.72858247241548, 77.99564270152506, 78.19945182374025, 78.45948415208377, 78.6351816712348, 78.7897954880877, 79.01468831260102, 79.25363693864644, 79.44339025932953, 79.68233888537493, 79.89317590835617, 80.16023613746574, 80.38512896197906, 80.546770679598, 80.67327289338675, 80.87708201560194, 81.05277953475297, 81.27767235926629, 81.46039777918335, 81.67826270293064, 81.81882071825146, 81.90315552744394, 82.05776934429686, 82.19832735961768, 82.35996907723663, 82.51458289408954, 82.51458289408954, 82.53566659638766, 82.60594560404807, 82.71839201630473, 82.7816431231991, 82.89408953545576, 82.92922903928597, 83.05573125307471, 83.07681495537284, 83.07681495537284, 83.13303816150116, 83.13303816150116, 83.16114976456532, 83.18926136762948, 83.21737297069366, 83.18926136762948, 83.11195445920305, 83.03464755077658, 83.04870335230868, 83.04167545154263, 83.03464755077658, 83.01356384847847, 82.91517323775388, 82.84489423009347, 82.78867102396515, 82.76758732166702, 82.68325251247452, 82.62000140558015, 82.57783400098391, 82.52161079485558, 82.44430388642913, 82.35996907723663, 82.34591327570455, 82.23346686344789, 82.1842715580856, 82.14210415348934, 82.10696464965915, 82.09290884812707, 82.02262984046665, 81.98749033663645, 81.88207182514583, 81.81179281748541, 81.75556961135709, 81.71340220676085, 81.62906739756835, 81.55878838990793, 81.50256518377961, 81.48850938224751, 81.43931407688524, 81.41823037458711, 81.36200716845879, 81.32686766462858, 81.26361655773421, 81.19333755007379, 81.15819804624358, 81.11603064164734, 81.05980743551902, 81.03169583245484, 81.00358422939068, 80.96844472556047, 80.94033312249631, 80.88410991636799, 80.83491461100569, 80.75057980181319, 80.67327289338675, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.98763089465177, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 73, 142, 203, 244, 287, 322, 368, 415, 469, 514, 557, 608, 654, 696, 743, 786, 828, 874, 917, 967, 1016, 1055, 1088, 1127, 1162, 1202, 1243, 1283, 1316, 1344, 1379, 1412, 1447, 1477, 1498, 1516, 1536, 1560, 1582, 1609, 1633, 1661, 1682, 1700, 1723, 1738, 1753, 1771, 1789, 1804, 1812, 1823, 1835, 1848, 1860, 1870, 1876, 1885, 1891, 1902, 1906, 1910, 1917, 1920, 1926, 1927, 1935, 1937, 1940, 1943, 1943, 1947, 1948, 1951, 1952, 1953, 1954, 1956, 1956, 1956, 1956, 1956, 1958, 1959, 1959, 1959, 1959, 1959, 1959, 1959, 1960, 1960, 1961, 1961, 1961, 1961, 1962, 1962, 1962, 1964], "harmed": [0, 3, 6, 15, 29, 42, 51, 61, 76, 85, 92, 106, 120, 141, 161, 176, 185, 200, 212, 225, 237, 254, 270, 285, 295, 305, 313, 328, 337, 350, 366, 379, 392, 404, 412, 433, 448, 458, 466, 479, 490, 509, 519, 537, 555, 570, 585, 596, 610, 624, 643, 662, 684, 694, 708, 721, 734, 754, 773, 787, 801, 817, 830, 843, 854, 871, 884, 894, 912, 922, 931, 936, 942, 953, 961, 977, 988, 997, 1005, 1017, 1027, 1035, 1037, 1046, 1050, 1058, 1063, 1072, 1082, 1087, 1093, 1102, 1106, 1111, 1116, 1120, 1128, 1136, 1148, 1159, 1171], "exact_score": [75.02986857825567, 75.52182163187857, 75.98566308243728, 76.35111392227142, 76.54086724295453, 76.75170426593576, 76.93442968585283, 77.18743411343031, 77.41232693794365, 77.72858247241548, 77.99564270152506, 78.19945182374025, 78.45948415208377, 78.6351816712348, 78.7897954880877, 79.01468831260102, 79.25363693864644, 79.44339025932953, 79.68233888537493, 79.89317590835617, 80.16023613746574, 80.38512896197906, 80.546770679598, 80.67327289338675, 80.87708201560194, 81.05277953475297, 81.27767235926629, 81.46039777918335, 81.67826270293064, 81.81882071825146, 81.90315552744394, 82.05776934429686, 82.19832735961768, 82.35996907723663, 82.51458289408954, 82.51458289408954, 82.53566659638766, 82.60594560404807, 82.71839201630473, 82.7816431231991, 82.89408953545576, 82.92922903928597, 83.05573125307471, 83.07681495537284, 83.07681495537284, 83.13303816150116, 83.13303816150116, 83.16114976456532, 83.18926136762948, 83.21737297069366, 83.18926136762948, 83.11195445920305, 83.03464755077658, 83.04870335230868, 83.04167545154263, 83.03464755077658, 83.01356384847847, 82.91517323775388, 82.84489423009347, 82.78867102396515, 82.76758732166702, 82.68325251247452, 82.62000140558015, 82.57783400098391, 82.52161079485558, 82.44430388642913, 82.35996907723663, 82.34591327570455, 82.23346686344789, 82.1842715580856, 82.14210415348934, 82.10696464965915, 82.09290884812707, 82.02262984046665, 81.98749033663645, 81.88207182514583, 81.81179281748541, 81.75556961135709, 81.71340220676085, 81.62906739756835, 81.55878838990793, 81.50256518377961, 81.48850938224751, 81.43931407688524, 81.41823037458711, 81.36200716845879, 81.32686766462858, 81.26361655773421, 81.19333755007379, 81.15819804624358, 81.11603064164734, 81.05980743551902, 81.03169583245484, 81.00358422939068, 80.96844472556047, 80.94033312249631, 80.88410991636799, 80.83491461100569, 80.75057980181319, 80.67327289338675, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.19482398393109, "threshold_area100": 81.19463771171552, "threshold_area25": 78.39271909480638}, "test": {"score": [66.95119530385956, 68.9334429855656, 69.94867141196156, 70.54419646655134, 71.13121402036128, 71.60196239684655, 71.95360576241386, 72.32793579672744, 72.66256416073504, 72.88092334741798, 73.10779003488075, 73.35450755749653, 73.60973258089216, 73.79689759804894, 74.02660011910501, 74.16555596517595, 74.36122848311261, 74.44913932450443, 74.56257266823583, 74.65048350962765, 74.74406601820604, 74.82063352522474, 74.88018603068372, 74.95108187051584, 75.00212687519496, 75.10705271814649, 75.14108272126592, 75.18645605875847, 75.22048606187789, 75.29705356889657, 75.37362107591527, 75.42750191418767, 75.50690525479965, 75.56645776025863, 75.64018943368403, 75.69690610554971, 75.70824943992285, 75.75929444460199, 75.83302611802739, 75.86422028755354, 75.93795196097892, 75.95213112894535, 75.99466863284462, 76.03437030315061, 76.04854947111703, 76.08541530782973, 76.09392280860959, 76.09108697501631, 76.09675864220287, 76.127952811729, 76.09959447579617, 76.13362447891558, 76.15347531406856, 76.16765448203499, 76.14213197969542, 76.1109378101693, 76.08541530782973, 76.07974364064316, 76.03720613674389, 75.9804894648782, 75.94929529535207, 75.92944446019906, 75.88123528911323, 75.88407112270652, 75.88690695629981, 75.85287695318038, 75.82451861724753, 75.7933244477214, 75.76496611178855, 75.70824943992285, 75.66004026883702, 75.64302526727731, 75.62033859853103, 75.6089952641579, 75.57780109463177, 75.56362192666535, 75.54660692510564, 75.52392025635936, 75.49272608683322, 75.45018858293395, 75.42466608059439, 75.38212857669512, 75.37362107591527, 75.35377024076227, 75.31406857045629, 75.2743669001503, 75.23182939625103, 75.18362022516519, 75.15809772282563, 75.13541105407936, 75.11556021892635, 75.07869438221365, 75.03048521112781, 74.99078354082182, 74.94541020332926, 74.89152936505685, 74.83481269319117, 74.77526018773219, 74.69869268071349, 74.59660267135524, 74.47182599325072], "calls": [0.0, 4.891812948416187, 6.916598134021496, 8.462127442361682, 10.197657601451947, 11.811246916030967, 13.084536199415819, 14.550662167144033, 15.639622266965375, 16.643507358988174, 17.721124124436376, 18.86396506253013, 20.055015171709726, 21.263080282449025, 22.451294558035332, 23.441000482091713, 24.470408076454074, 25.485636502850014, 26.41011825426084, 27.606840030627, 28.59654595468338, 29.572072710773334, 30.575957802796133, 31.616708731531634, 32.631937157927574, 33.77194226242804, 34.92045486770836, 35.98105663159686, 37.19195757592944, 38.408530187448605, 39.62510279896775, 40.946601253438445, 42.12063636105833, 43.43362731474917, 44.70408076454074, 45.9745342143323, 47.24498766412387, 48.57499361937441, 49.83693956838613, 51.02515384397244, 52.54799648356634, 53.79576326461163, 55.22218756203387, 56.58905935399711, 57.890706973314806, 59.00235374188243, 60.5308680486629, 61.77296316252162, 63.04341661231319, 64.52939341519439, 65.75163769390012, 66.89447863199388, 68.2273204208377, 69.28225051753964, 70.30315061112213, 71.34957320704422, 72.42151830530584, 73.57003091058617, 74.66466267759408, 75.87272778833338, 76.82273204208377, 77.79258713098716, 78.6546805433457, 79.61319229787595, 80.45827070867482, 81.09633326716389, 81.77693332955222, 82.60216090519809, 83.18917845900803, 83.90380852451578, 84.54187108300485, 85.07784363213567, 85.6279953492329, 86.15262456399059, 86.64605960922212, 87.11964381930069, 87.59039219578595, 88.07248390664436, 88.53472478234977, 88.95442815415592, 89.41099736267476, 89.91010407509287, 90.36667328361172, 90.77503332104472, 91.27130419986955, 91.76190341150782, 92.2468309559595, 92.68638516291864, 93.1032527011315, 93.4945977370048, 93.96251027989678, 94.38504948529621, 94.98341037347929, 95.4258004140317, 95.96177296316252, 96.45237217480079, 97.20386807702124, 97.97237898080141, 98.54238153305164, 99.4073107790035, 100.0], "rescued": [0, 706, 1081, 1306, 1542, 1740, 1892, 2064, 2211, 2325, 2438, 2558, 2687, 2805, 2921, 3003, 3103, 3178, 3244, 3310, 3367, 3424, 3476, 3538, 3590, 3661, 3718, 3767, 3815, 3871, 3929, 3986, 4044, 4099, 4156, 4217, 4252, 4299, 4356, 4399, 4464, 4510, 4560, 4605, 4641, 4679, 4719, 4753, 4787, 4837, 4863, 4908, 4944, 4972, 4998, 5021, 5040, 5074, 5098, 5125, 5150, 5172, 5187, 5212, 5233, 5246, 5261, 5279, 5289, 5299, 5308, 5319, 5334, 5341, 5345, 5355, 5364, 5368, 5371, 5372, 5376, 5380, 5384, 5388, 5392, 5395, 5398, 5399, 5400, 5403, 5405, 5406, 5409, 5413, 5415, 5417, 5419, 5420, 5420, 5422, 5423], "harmed": [0, 7, 24, 39, 68, 100, 128, 168, 197, 234, 267, 300, 339, 391, 426, 459, 490, 534, 560, 595, 619, 649, 680, 717, 751, 785, 830, 863, 899, 928, 959, 997, 1027, 1061, 1092, 1133, 1164, 1193, 1224, 1256, 1295, 1336, 1371, 1402, 1433, 1458, 1495, 1530, 1562, 1601, 1637, 1670, 1699, 1722, 1757, 1791, 1819, 1855, 1894, 1941, 1977, 2006, 2038, 2062, 2082, 2107, 2132, 2161, 2181, 2211, 2237, 2254, 2277, 2288, 2303, 2318, 2333, 2345, 2359, 2375, 2388, 2407, 2414, 2425, 2443, 2460, 2478, 2496, 2506, 2517, 2526, 2540, 2560, 2578, 2596, 2617, 2639, 2661, 2688, 2726, 2771], "exact_score": [66.95119530385956, 67.18940532569549, 67.46448118424411, 67.88134872245696, 68.44284377392734, 68.9901596574313, 69.50060970422255, 69.99120891586081, 70.37121061736097, 70.72852565011485, 71.04613901256273, 71.36658820860391, 71.66435073589882, 71.93091909366758, 72.22017412018262, 72.46405580920512, 72.76749000368659, 72.9688341888098, 73.17017837393301, 73.38853756061594, 73.5813742449593, 73.75436009414967, 73.96704761364603, 74.11734679409012, 74.25630264016108, 74.40376598701188, 74.52287099792983, 74.59376683776196, 74.68451351274707, 74.76958852054561, 74.84332019397102, 74.91705186739642, 74.95391770410912, 75.03048521112781, 75.11556021892635, 75.13257522048606, 75.18929189235175, 75.22048606187789, 75.26585939937044, 75.3282477384227, 75.3849644102884, 75.42466608059439, 75.49272608683322, 75.53809942432578, 75.59198026259818, 75.64018943368403, 75.69407027195643, 75.71108527351615, 75.76213027819527, 75.77347361256841, 75.83869778521397, 75.86138445396024, 75.88407112270652, 75.9407877945722, 75.95496696253863, 76.00317613362448, 76.02586280237075, 76.02869863596405, 76.0542211383036, 76.08541530782973, 76.10526614298273, 76.12228114454244, 76.08541530782973, 76.09959447579617, 76.11377364376258, 76.11377364376258, 76.10810197657601, 76.13078864532228, 76.14496781328872, 76.16765448203499, 76.14496781328872, 76.13078864532228, 76.09108697501631, 76.09108697501631, 76.0740719734566, 76.01451946799762, 75.9804894648782, 75.95213112894535, 75.90108612426623, 75.88974278989309, 75.87556362192667, 75.86138445396024, 75.80750361568784, 75.77914527975497, 75.69690610554971, 75.65153276805717, 75.62033859853103, 75.56645776025863, 75.5324277571392, 75.45018858293395, 75.39063607747497, 75.34242690638912, 75.25168023140402, 75.16944105719875, 75.11272438533307, 75.02764937753452, 74.94541020332926, 74.84332019397102, 74.7724243541389, 74.66182684400079, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.52428288326236, "threshold_area100": 74.96911777216911, "threshold_area25": 72.92969968522247}, "selected_budget": 49, "selected": {"val": {"score": 83.21737297069366, "calls": 48.99852414083913, "rescued": 1789, "harmed": 624}, "test": {"score": 76.127952811729, "calls": 64.52939341519439, "rescued": 4837, "harmed": 1601}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9324418306350708, "operator": "ge"}, {"budget": 0.02, "threshold": 0.42043066024780273, "operator": "ge"}, {"budget": 0.03, "threshold": 0.17356300354003906, "operator": "ge"}, {"budget": 0.04, "threshold": -0.03537774085998535, "operator": "ge"}, {"budget": 0.05, "threshold": -0.2728528380393982, "operator": "ge"}, {"budget": 0.06, "threshold": -0.4681727886199951, "operator": "ge"}, {"budget": 0.07, "threshold": -0.6494172811508179, "operator": "ge"}, {"budget": 0.08, "threshold": -0.8658738136291504, "operator": "ge"}, {"budget": 0.09, "threshold": -1.0758519172668457, "operator": "ge"}, {"budget": 0.1, "threshold": -1.3138319253921509, "operator": "ge"}, {"budget": 0.11, "threshold": -1.5258610248565674, "operator": "ge"}, {"budget": 0.12, "threshold": -1.7677162885665894, "operator": "ge"}, {"budget": 0.13, "threshold": -2.0188982486724854, "operator": "ge"}, {"budget": 0.14, "threshold": -2.2553839683532715, "operator": "ge"}, {"budget": 0.15, "threshold": -2.4706013202667236, "operator": "ge"}, {"budget": 0.16, "threshold": -2.668031692504883, "operator": "ge"}, {"budget": 0.17, "threshold": -2.861238718032837, "operator": "ge"}, {"budget": 0.18, "threshold": -3.0235307216644287, "operator": "ge"}, {"budget": 0.19, "threshold": -3.181550979614258, "operator": "ge"}, {"budget": 0.2, "threshold": -3.3186769485473633, "operator": "gt"}, {"budget": 0.21, "threshold": -3.441282272338867, "operator": "ge"}, {"budget": 0.22, "threshold": -3.5547053813934326, "operator": "ge"}, {"budget": 0.23, "threshold": -3.663412094116211, "operator": "ge"}, {"budget": 0.24, "threshold": -3.7998056411743164, "operator": "ge"}, {"budget": 0.25, "threshold": -3.893454074859619, "operator": "ge"}, {"budget": 0.26, "threshold": -3.991682767868042, "operator": "ge"}, {"budget": 0.27, "threshold": -4.0848517417907715, "operator": "ge"}, {"budget": 0.28, "threshold": -4.18819522857666, "operator": "ge"}, {"budget": 0.29, "threshold": -4.278017997741699, "operator": "ge"}, {"budget": 0.3, "threshold": -4.362612247467041, "operator": "ge"}, {"budget": 0.31, "threshold": -4.45103120803833, "operator": "ge"}, {"budget": 0.32, "threshold": -4.535090446472168, "operator": "ge"}, {"budget": 0.33, "threshold": -4.6183762550354, "operator": "ge"}, {"budget": 0.34, "threshold": -4.704410552978516, "operator": "ge"}, {"budget": 0.35, "threshold": -4.789534091949463, "operator": "ge"}, {"budget": 0.36, "threshold": -4.865166664123535, "operator": "ge"}, {"budget": 0.37, "threshold": -4.949787616729736, "operator": "ge"}, {"budget": 0.38, "threshold": -5.028185844421387, "operator": "ge"}, {"budget": 0.39, "threshold": -5.103508949279785, "operator": "ge"}, {"budget": 0.4, "threshold": -5.179078578948975, "operator": "ge"}, {"budget": 0.41, "threshold": -5.253572940826416, "operator": "ge"}, {"budget": 0.42, "threshold": -5.327116966247559, "operator": "ge"}, {"budget": 0.43, "threshold": -5.402782917022705, "operator": "ge"}, {"budget": 0.44, "threshold": -5.466145992279053, "operator": "ge"}, {"budget": 0.45, "threshold": -5.55296516418457, "operator": "ge"}, {"budget": 0.46, "threshold": -5.628415107727051, "operator": "ge"}, {"budget": 0.47, "threshold": -5.705243110656738, "operator": "ge"}, {"budget": 0.48, "threshold": -5.782072067260742, "operator": "ge"}, {"budget": 0.49, "threshold": -5.86594295501709, "operator": "ge"}, {"budget": 0.5, "threshold": -5.925342082977295, "operator": "ge"}, {"budget": 0.51, "threshold": -5.994813919067383, "operator": "ge"}, {"budget": 0.52, "threshold": -6.0762224197387695, "operator": "ge"}, {"budget": 0.53, "threshold": -6.1543145179748535, "operator": "ge"}, {"budget": 0.54, "threshold": -6.218047142028809, "operator": "ge"}, {"budget": 0.55, "threshold": -6.2888407707214355, "operator": "ge"}, {"budget": 0.56, "threshold": -6.348431587219238, "operator": "ge"}, {"budget": 0.57, "threshold": -6.414340019226074, "operator": "ge"}, {"budget": 0.58, "threshold": -6.483578681945801, "operator": "ge"}, {"budget": 0.59, "threshold": -6.561649322509766, "operator": "ge"}, {"budget": 0.6, "threshold": -6.624451637268066, "operator": "ge"}, {"budget": 0.61, "threshold": -6.69242000579834, "operator": "ge"}, {"budget": 0.62, "threshold": -6.750024795532227, "operator": "ge"}, {"budget": 0.63, "threshold": -6.810375690460205, "operator": "ge"}, {"budget": 0.64, "threshold": -6.8778076171875, "operator": "ge"}, {"budget": 0.65, "threshold": -6.944007873535156, "operator": "ge"}, {"budget": 0.66, "threshold": -7.000616073608398, "operator": "ge"}, {"budget": 0.67, "threshold": -7.059303283691406, "operator": "ge"}, {"budget": 0.68, "threshold": -7.104508876800537, "operator": "ge"}, {"budget": 0.69, "threshold": -7.1555023193359375, "operator": "ge"}, {"budget": 0.7, "threshold": -7.207324028015137, "operator": "ge"}, {"budget": 0.71, "threshold": -7.2585344314575195, "operator": "ge"}, {"budget": 0.72, "threshold": -7.315065860748291, "operator": "ge"}, {"budget": 0.73, "threshold": -7.354301929473877, "operator": "ge"}, {"budget": 0.74, "threshold": -7.401861667633057, "operator": "ge"}, {"budget": 0.75, "threshold": -7.445201873779297, "operator": "ge"}, {"budget": 0.76, "threshold": -7.488316059112549, "operator": "ge"}, {"budget": 0.77, "threshold": -7.5390825271606445, "operator": "ge"}, {"budget": 0.78, "threshold": -7.580822467803955, "operator": "ge"}, {"budget": 0.79, "threshold": -7.625108242034912, "operator": "ge"}, {"budget": 0.8, "threshold": -7.66151237487793, "operator": "ge"}, {"budget": 0.81, "threshold": -7.700440406799316, "operator": "ge"}, {"budget": 0.82, "threshold": -7.738754749298096, "operator": "ge"}, {"budget": 0.83, "threshold": -7.784523963928223, "operator": "ge"}, {"budget": 0.84, "threshold": -7.828794479370117, "operator": "ge"}, {"budget": 0.85, "threshold": -7.882353782653809, "operator": "ge"}, {"budget": 0.86, "threshold": -7.925512313842773, "operator": "ge"}, {"budget": 0.87, "threshold": -7.974482536315918, "operator": "ge"}, {"budget": 0.88, "threshold": -8.019180297851562, "operator": "ge"}, {"budget": 0.89, "threshold": -8.068323135375977, "operator": "ge"}, {"budget": 0.9, "threshold": -8.118167877197266, "operator": "ge"}, {"budget": 0.91, "threshold": -8.162459373474121, "operator": "ge"}, {"budget": 0.92, "threshold": -8.210793495178223, "operator": "ge"}, {"budget": 0.93, "threshold": -8.267545700073242, "operator": "ge"}, {"budget": 0.94, "threshold": -8.339912414550781, "operator": "ge"}, {"budget": 0.95, "threshold": -8.419537544250488, "operator": "ge"}, {"budget": 0.96, "threshold": -8.501111030578613, "operator": "ge"}, {"budget": 0.97, "threshold": -8.603200912475586, "operator": "ge"}, {"budget": 0.98, "threshold": -8.71660041809082, "operator": "ge"}, {"budget": 0.99, "threshold": -8.909952163696289, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.57101693724084, 75.95052357860708, 76.35111392227142, 76.52681144142245, 76.79387167053201, 77.05390399887554, 77.34204793028323, 77.5599128540305, 77.89022419003444, 78.09403331224964, 78.29081453369878, 78.45245625131774, 78.66329327429898, 78.9092698011104, 79.07793941949541, 79.21146953405018, 79.47852976315974, 79.7596457938014, 80.0829292290393, 80.48351957270363, 80.77166350411133, 81.09494693934923, 81.36903506922482, 81.66420690139856, 81.86098812284771, 82.07182514582894, 82.2194110619158, 82.53566659638766, 82.64811300864432, 82.77461522243306, 82.83786632932744, 82.90814533698784, 83.05573125307471, 83.18223346686345, 83.32279148218427, 83.42118209290885, 83.47740529903717, 83.63201911589009, 83.84285613887133, 83.9061072457657, 83.96233045189402, 84.00449785649027, 84.01855365802236, 84.01152575725631, 84.01152575725631, 83.93421884882986, 83.8569119404034, 83.87096774193549, 83.77960503197694, 83.6531028181882, 83.61093541359196, 83.4984890013353, 83.42820999367488, 83.35090308524843, 83.35793098601448, 83.2595403752899, 83.19628926839553, 83.1752055660974, 83.06978705460679, 82.97842434464826, 82.90111743622181, 82.83786632932744, 82.71136411553869, 82.6270293063462, 82.52863869562161, 82.4583596879612, 82.40916438259892, 82.35996907723663, 82.31077377187435, 82.25455056574602, 82.1842715580856, 82.13507625272331, 82.07885304659499, 82.03668564199873, 81.98046243587041, 81.91018342820999, 81.83990442054959, 81.78368121442125, 81.73448590905896, 81.67826270293064, 81.65015109986648, 81.55878838990793, 81.50256518377961, 81.43931407688524, 81.41120247382108, 81.39011877152295, 81.3479513669267, 81.28470026003232, 81.25658865696816, 81.20036545083984, 81.15819804624358, 81.08089113781713, 81.03872373322089, 80.96844472556047, 80.94736102326236, 80.91222151943215, 80.85599831330381, 80.7857193056434, 80.69435659568487, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.987349778621123, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 83, 142, 206, 244, 286, 336, 383, 426, 483, 529, 572, 611, 654, 710, 756, 798, 853, 906, 964, 1025, 1079, 1129, 1179, 1232, 1274, 1315, 1346, 1396, 1426, 1454, 1481, 1508, 1543, 1578, 1609, 1636, 1655, 1688, 1724, 1742, 1763, 1783, 1797, 1813, 1829, 1839, 1847, 1859, 1868, 1875, 1883, 1890, 1895, 1903, 1912, 1918, 1925, 1932, 1935, 1938, 1940, 1944, 1946, 1946, 1947, 1948, 1949, 1952, 1953, 1954, 1954, 1954, 1955, 1955, 1955, 1955, 1955, 1956, 1956, 1957, 1957, 1957, 1958, 1959, 1961, 1961, 1961, 1961, 1961, 1961, 1961, 1961, 1961, 1961, 1962, 1963, 1963, 1963, 1964, 1964], "harmed": [0, 6, 11, 18, 31, 35, 48, 54, 66, 76, 93, 108, 124, 137, 158, 180, 203, 220, 233, 245, 249, 262, 266, 277, 288, 302, 313, 323, 328, 342, 352, 370, 387, 401, 418, 429, 442, 453, 464, 470, 479, 492, 506, 518, 535, 551, 572, 591, 601, 623, 648, 662, 685, 700, 719, 727, 747, 763, 773, 791, 807, 820, 833, 853, 865, 880, 891, 899, 909, 917, 926, 936, 943, 952, 958, 966, 976, 986, 995, 1002, 1011, 1015, 1028, 1037, 1047, 1053, 1056, 1062, 1071, 1075, 1083, 1089, 1100, 1106, 1116, 1120, 1126, 1134, 1144, 1158, 1171], "exact_score": [75.02986857825567, 75.57101693724084, 75.95052357860708, 76.35111392227142, 76.52681144142245, 76.79387167053201, 77.05390399887554, 77.34204793028323, 77.5599128540305, 77.89022419003444, 78.09403331224964, 78.29081453369878, 78.45245625131774, 78.66329327429898, 78.9092698011104, 79.07793941949541, 79.21146953405018, 79.47852976315974, 79.7596457938014, 80.0829292290393, 80.48351957270363, 80.77166350411133, 81.09494693934923, 81.36903506922482, 81.66420690139856, 81.86098812284771, 82.07182514582894, 82.2194110619158, 82.53566659638766, 82.64811300864432, 82.77461522243306, 82.83786632932744, 82.90814533698784, 83.05573125307471, 83.18223346686345, 83.32279148218427, 83.42118209290885, 83.47740529903717, 83.63201911589009, 83.84285613887133, 83.9061072457657, 83.96233045189402, 84.00449785649027, 84.01855365802236, 84.01152575725631, 84.01152575725631, 83.93421884882986, 83.8569119404034, 83.87096774193549, 83.77960503197694, 83.6531028181882, 83.61093541359196, 83.4984890013353, 83.42820999367488, 83.35090308524843, 83.35793098601448, 83.2595403752899, 83.19628926839553, 83.1752055660974, 83.06978705460679, 82.97842434464826, 82.90111743622181, 82.83786632932744, 82.71136411553869, 82.6270293063462, 82.52863869562161, 82.4583596879612, 82.40916438259892, 82.35996907723663, 82.31077377187435, 82.25455056574602, 82.1842715580856, 82.13507625272331, 82.07885304659499, 82.03668564199873, 81.98046243587041, 81.91018342820999, 81.83990442054959, 81.78368121442125, 81.73448590905896, 81.67826270293064, 81.65015109986648, 81.55878838990793, 81.50256518377961, 81.43931407688524, 81.41120247382108, 81.39011877152295, 81.3479513669267, 81.28470026003232, 81.25658865696816, 81.20036545083984, 81.15819804624358, 81.08089113781713, 81.03872373322089, 80.96844472556047, 80.94736102326236, 80.91222151943215, 80.85599831330381, 80.7857193056434, 80.69435659568487, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.51248569125282, "threshold_area100": 81.51229882634058, "threshold_area25": 78.57895846510647}, "test": {"score": [66.95119530385956, 68.33508209738253, 69.48359470266284, 70.35986728298784, 71.09718401724187, 71.56226072654056, 71.97912826475343, 72.35912996625358, 72.68525082948132, 72.9518191872501, 73.26659671610469, 73.42256756373536, 73.60973258089216, 73.86212177069449, 74.07480929019084, 74.19958596829538, 74.30451181124691, 74.43212432294473, 74.55406516745597, 74.67884184556051, 74.7894393556986, 74.89720103224342, 75.02481354394124, 75.17794855797861, 75.28003856733687, 75.32257607123614, 75.3849644102884, 75.45018858293395, 75.56362192666535, 75.6628761024303, 75.73093610866914, 75.84153361880725, 75.89257862348637, 75.93228029379236, 76.01735530159091, 76.10810197657601, 76.15347531406856, 76.19034115078126, 76.22720698749397, 76.26974449139324, 76.2952669937328, 76.27258032498652, 76.28108782576638, 76.27258032498652, 76.28392365935966, 76.29810282732609, 76.30377449451267, 76.29810282732609, 76.28108782576638, 76.24422198905368, 76.24705782264697, 76.23571448827381, 76.26407282420668, 76.24989365624025, 76.23287865468053, 76.2016844851544, 76.16481864844171, 76.11660947735587, 76.08257947423644, 76.02302696877747, 75.96063862972521, 75.92944446019906, 75.87556362192667, 75.84436945240053, 75.83019028443411, 75.80750361568784, 75.77347361256841, 75.73660777585572, 75.70824943992285, 75.67138360321016, 75.64302526727731, 75.61750276493775, 75.5891444290049, 75.57496526103849, 75.52108442276607, 75.48138275246008, 75.46436775090038, 75.45302441652724, 75.43884524856081, 75.4048152454414, 75.37645690950855, 75.35093440716899, 75.33959107279584, 75.311232736863, 75.30272523608315, 75.29138190171001, 75.28287440093015, 75.22899356265775, 75.19496355953832, 75.15809772282563, 75.11272438533307, 75.09854521736663, 75.06735104784052, 75.03899271190765, 74.98794770722854, 74.92555936817628, 74.87735019709044, 74.79794685647846, 74.7355585174262, 74.63914017525451, 74.47182599325072], "calls": [0.0, 2.4416527238181662, 5.376740492867879, 7.897796557298017, 10.101239259280266, 11.601395230127896, 12.90587868303888, 14.286929642968552, 15.741712276323625, 16.972464055809205, 18.345007514959022, 19.547400958511755, 20.724271899724926, 21.847262002665683, 23.00428210872586, 24.096078042140487, 25.128321470096132, 26.308028244902587, 27.428182514250064, 28.701471797634916, 29.813118566202533, 30.899242832430595, 32.15268128066245, 33.326716388282335, 34.8382156935031, 36.094489975328244, 37.4783767688512, 38.93599523579957, 40.473017043359896, 41.98168051498738, 43.4648214842753, 45.07273913166775, 46.555880100955676, 48.163797748348124, 49.86529790431897, 51.55829055950997, 52.908147349913506, 54.49621416215297, 55.8857726228625, 57.34906275699741, 58.77265122082637, 60.12817967841647, 61.41848396336103, 62.92147576780195, 64.06715253948899, 65.66372685250829, 66.90298613277373, 68.10254374273318, 69.45807220032329, 70.70300314777529, 71.56226072654057, 72.49808581232453, 73.59271757933244, 74.6079460057284, 75.39063607747497, 76.25272948983354, 76.93900121940844, 77.61960128179678, 78.32855968011798, 79.12542891983098, 79.83438731815217, 80.64259989223832, 81.26364744916769, 81.85917250375748, 82.48589172787341, 83.11544678558262, 83.696792672206, 84.21858605337039, 84.63261775798996, 85.14023197118793, 85.73008535859115, 86.22635623741598, 86.82755295919236, 87.23307716303206, 87.69815387233078, 88.1008422425772, 88.52905311516321, 89.0338314947679, 89.43368403142104, 89.92144740946601, 90.25607577347361, 90.65592831012677, 91.00757167569407, 91.48966338655248, 91.90936675835863, 92.40847347077673, 92.84519184414258, 93.3442985565607, 93.78101692992655, 94.2205711368857, 94.73102118367694, 95.15072455548308, 95.57326376088251, 96.08371380767376, 96.66222386070386, 97.31162975356607, 97.87596063862972, 98.5310381986785, 98.99895074157048, 99.43850494852961, 100.0], "rescued": [0, 503, 923, 1248, 1528, 1721, 1895, 2056, 2215, 2350, 2509, 2625, 2747, 2871, 2988, 3080, 3162, 3248, 3329, 3413, 3492, 3566, 3645, 3723, 3806, 3855, 3915, 3975, 4047, 4113, 4172, 4245, 4302, 4349, 4420, 4483, 4531, 4582, 4627, 4672, 4716, 4748, 4784, 4812, 4842, 4883, 4914, 4941, 4975, 5005, 5026, 5046, 5080, 5115, 5132, 5148, 5163, 5176, 5193, 5209, 5215, 5227, 5242, 5258, 5272, 5289, 5297, 5305, 5310, 5315, 5324, 5334, 5347, 5356, 5359, 5362, 5365, 5373, 5377, 5378, 5380, 5383, 5385, 5387, 5389, 5395, 5399, 5402, 5403, 5407, 5409, 5412, 5414, 5415, 5417, 5420, 5421, 5421, 5422, 5422, 5423], "harmed": [0, 15, 30, 46, 66, 95, 122, 149, 193, 234, 282, 343, 399, 434, 476, 524, 569, 610, 648, 688, 728, 764, 798, 822, 869, 903, 941, 978, 1010, 1041, 1076, 1110, 1149, 1182, 1223, 1254, 1286, 1324, 1356, 1386, 1421, 1461, 1494, 1525, 1551, 1587, 1616, 1645, 1685, 1728, 1748, 1772, 1796, 1836, 1859, 1886, 1914, 1944, 1973, 2010, 2038, 2061, 2095, 2122, 2141, 2166, 2186, 2207, 2222, 2240, 2259, 2278, 2301, 2315, 2337, 2354, 2363, 2375, 2384, 2397, 2409, 2421, 2427, 2439, 2444, 2454, 2461, 2483, 2496, 2513, 2531, 2539, 2552, 2563, 2583, 2608, 2626, 2654, 2677, 2711, 2771], "exact_score": [66.95119530385956, 67.60627286390834, 68.1535887474123, 68.58463545359157, 69.01851799336414, 69.34747469018518, 69.70478972293905, 70.05926892209965, 70.38538978532739, 70.72001814933499, 71.05181067974931, 71.36658820860391, 71.69554490542495, 72.01315826787284, 72.2995774607946, 72.5434591498171, 72.75331083572016, 72.95749085443666, 73.21555171142558, 73.35734339108981, 73.5246575730936, 73.6749567535377, 73.8989876074072, 74.07197345659756, 74.17122763236253, 74.2704818081275, 74.40376598701188, 74.51436349714999, 74.60511017213508, 74.72137934945977, 74.81212602444488, 74.91705186739642, 75.01914187675467, 75.11272438533307, 75.22899356265775, 75.28854606811673, 75.31406857045629, 75.37078524232199, 75.41899441340783, 75.45586025012052, 75.51257692198622, 75.60615943056462, 75.6628761024303, 75.71392110710944, 75.73660777585572, 75.83302611802739, 75.86422028755354, 75.90392195785951, 75.92660862660578, 75.98899696565805, 76.02019113518419, 76.08825114142302, 76.11377364376258, 76.15063948047528, 76.16765448203499, 76.21019198593426, 76.23855032186711, 76.2584011570201, 76.2952669937328, 76.28959532654623, 76.26974449139324, 76.2782519921731, 76.27258032498652, 76.27541615857982, 76.2782519921731, 76.28675949295295, 76.30377449451267, 76.29810282732609, 76.30661032810595, 76.27541615857982, 76.26974449139324, 76.23855032186711, 76.24705782264697, 76.2413861554604, 76.24705782264697, 76.24705782264697, 76.2045203187477, 76.16481864844171, 76.09108697501631, 76.02869863596405, 75.9407877945722, 75.90108612426623, 75.84153361880725, 75.81601111646769, 75.76496611178855, 75.67989110399002, 75.61466693134446, 75.58347276181834, 75.49272608683322, 75.4586960837138, 75.38780024388168, 75.33959107279584, 75.29988940248987, 75.26869523296374, 75.16944105719875, 75.11556021892635, 75.04182854550095, 74.96242520488897, 74.87167852990386, 74.7355585174262, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.64043746276863, "threshold_area100": 75.049400221195, "threshold_area25": 72.99095369083743}, "selected_budget": 43, "selected": {"val": {"score": 84.01855365802236, "calls": 42.99669688663996, "rescued": 1797, "harmed": 518}, "test": {"score": 76.27258032498652, "calls": 62.92147576780195, "rescued": 4812, "harmed": 1525}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 80.73652400028112, "calls": 96.99205847213437}, "test": {"score": 74.66749851118736, "calls": 97.23789808014067}}, "old_normalized_interior": {"val": {"score": 84.01855365802236, "calls": 42.99669688663996}, "test": {"score": 76.27258032498652, "calls": 62.92147576780195}}, "old_normalized_endpoints": {"val": {"score": 84.01855365802236, "calls": 42.99669688663996}, "test": {"score": 76.27258032498652, "calls": 62.92147576780195}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.57101693724084, 75.95052357860708, 76.35111392227142, 76.52681144142245, 76.79387167053201, 77.05390399887554, 77.34204793028323, 77.5599128540305, 77.89022419003444, 78.09403331224964, 78.29081453369878, 78.45245625131774, 78.66329327429898, 78.9092698011104, 79.07793941949541, 79.21146953405018, 79.47852976315974, 79.7596457938014, 80.0829292290393, 80.48351957270363, 80.77166350411133, 81.09494693934923, 81.36903506922482, 81.66420690139856, 81.86098812284771, 82.07182514582894, 82.2194110619158, 82.53566659638766, 82.64811300864432, 82.77461522243306, 82.83786632932744, 82.90814533698784, 83.05573125307471, 83.18223346686345, 83.32279148218427, 83.4141541921428, 83.47740529903717, 83.63201911589009, 83.84285613887133, 83.9061072457657, 83.96233045189402, 84.00449785649027, 84.01855365802236, 84.01152575725631, 84.01152575725631, 83.93421884882986, 83.8569119404034, 83.87096774193549, 83.77960503197694, 83.6531028181882, 83.61093541359196, 83.4984890013353, 83.42820999367488, 83.35090308524843, 83.35793098601448, 83.2595403752899, 83.19628926839553, 83.1752055660974, 83.06978705460679, 82.97842434464826, 82.90111743622181, 82.83786632932744, 82.71136411553869, 82.6270293063462, 82.52863869562161, 82.4583596879612, 82.40916438259892, 82.35996907723663, 82.31077377187435, 82.25455056574602, 82.1842715580856, 82.13507625272331, 82.07885304659499, 82.03668564199873, 81.98046243587041, 81.91018342820999, 81.83990442054959, 81.78368121442125, 81.73448590905896, 81.67826270293064, 81.65015109986648, 81.55878838990793, 81.50959308454566, 81.43931407688524, 81.41120247382108, 81.39011877152295, 81.3479513669267, 81.28470026003232, 81.25658865696816, 81.20036545083984, 81.15819804624358, 81.08089113781713, 81.03872373322089, 80.96844472556047, 80.94736102326236, 80.91222151943215, 80.85599831330381, 80.7857193056434, 80.69435659568487, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.987349778621123, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.57101693724084, 75.95052357860708, 76.35111392227142, 76.52681144142245, 76.79387167053201, 77.05390399887554, 77.34204793028323, 77.5599128540305, 77.89022419003444, 78.09403331224964, 78.29081453369878, 78.45245625131774, 78.66329327429898, 78.9092698011104, 79.07793941949541, 79.21146953405018, 79.47852976315974, 79.7596457938014, 80.0829292290393, 80.48351957270363, 80.77166350411133, 81.09494693934923, 81.36903506922482, 81.66420690139856, 81.86098812284771, 82.07182514582894, 82.2194110619158, 82.53566659638766, 82.64811300864432, 82.77461522243306, 82.83786632932744, 82.90814533698784, 83.05573125307471, 83.18223346686345, 83.32279148218427, 83.4141541921428, 83.47740529903717, 83.63201911589009, 83.84285613887133, 83.9061072457657, 83.96233045189402, 84.00449785649027, 84.01855365802236, 84.01152575725631, 84.01152575725631, 83.93421884882986, 83.8569119404034, 83.87096774193549, 83.77960503197694, 83.6531028181882, 83.61093541359196, 83.4984890013353, 83.42820999367488, 83.35090308524843, 83.35793098601448, 83.2595403752899, 83.19628926839553, 83.1752055660974, 83.06978705460679, 82.97842434464826, 82.90111743622181, 82.83786632932744, 82.71136411553869, 82.6270293063462, 82.52863869562161, 82.4583596879612, 82.40916438259892, 82.35996907723663, 82.31077377187435, 82.25455056574602, 82.1842715580856, 82.13507625272331, 82.07885304659499, 82.03668564199873, 81.98046243587041, 81.91018342820999, 81.83990442054959, 81.78368121442125, 81.73448590905896, 81.67826270293064, 81.65015109986648, 81.55878838990793, 81.50959308454566, 81.43931407688524, 81.41120247382108, 81.39011877152295, 81.3479513669267, 81.28470026003232, 81.25658865696816, 81.20036545083984, 81.15819804624358, 81.08089113781713, 81.03872373322089, 80.96844472556047, 80.94736102326236, 80.91222151943215, 80.85599831330381, 80.7857193056434, 80.69435659568487, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 78.57895846510647, "exact_area100": 81.51248593820976}, "test": {"score": [66.95119530385956, 68.33508209738253, 69.48359470266284, 70.35986728298784, 71.09718401724187, 71.56226072654056, 71.97912826475343, 72.35912996625358, 72.68525082948132, 72.9518191872501, 73.26659671610469, 73.42256756373536, 73.60973258089216, 73.86212177069449, 74.07480929019084, 74.19958596829538, 74.30451181124691, 74.43212432294473, 74.55406516745597, 74.67884184556051, 74.7894393556986, 74.89720103224342, 75.02481354394124, 75.17794855797861, 75.28003856733687, 75.32257607123614, 75.3849644102884, 75.45018858293395, 75.56362192666535, 75.6628761024303, 75.73093610866914, 75.84153361880725, 75.89257862348637, 75.93228029379236, 76.01735530159091, 76.10810197657601, 76.15347531406856, 76.19034115078126, 76.22720698749397, 76.26974449139324, 76.2952669937328, 76.27258032498652, 76.28108782576638, 76.27258032498652, 76.28392365935966, 76.29810282732609, 76.30377449451267, 76.2952669937328, 76.28108782576638, 76.24422198905368, 76.24705782264697, 76.23571448827381, 76.26407282420668, 76.24989365624025, 76.23287865468053, 76.2016844851544, 76.16481864844171, 76.11660947735587, 76.08257947423644, 76.02302696877747, 75.96063862972521, 75.92944446019906, 75.87556362192667, 75.84436945240053, 75.83019028443411, 75.80750361568784, 75.77347361256841, 75.73660777585572, 75.70824943992285, 75.67138360321016, 75.64302526727731, 75.61750276493775, 75.5891444290049, 75.57496526103849, 75.52108442276607, 75.48138275246008, 75.46436775090038, 75.45302441652724, 75.43884524856081, 75.4048152454414, 75.37645690950855, 75.35093440716899, 75.33959107279584, 75.311232736863, 75.30272523608315, 75.29138190171001, 75.28287440093015, 75.22899356265775, 75.19496355953832, 75.15809772282563, 75.11272438533307, 75.09854521736663, 75.06735104784052, 75.03899271190765, 74.99078354082182, 74.92555936817628, 74.87735019709044, 74.79794685647846, 74.7355585174262, 74.63914017525451, 74.47182599325072], "calls": [0.0, 2.4416527238181662, 5.376740492867879, 7.897796557298017, 10.101239259280266, 11.601395230127896, 12.90587868303888, 14.286929642968552, 15.741712276323625, 16.972464055809205, 18.345007514959022, 19.547400958511755, 20.724271899724926, 21.847262002665683, 23.00428210872586, 24.096078042140487, 25.128321470096132, 26.308028244902587, 27.428182514250064, 28.701471797634916, 29.813118566202533, 30.899242832430595, 32.15268128066245, 33.326716388282335, 34.8382156935031, 36.094489975328244, 37.4783767688512, 38.93599523579957, 40.473017043359896, 41.98168051498738, 43.4648214842753, 45.07273913166775, 46.555880100955676, 48.163797748348124, 49.87096957150555, 51.55829055950997, 52.908147349913506, 54.49621416215297, 55.8857726228625, 57.34906275699741, 58.77265122082637, 60.13101551200975, 61.41848396336103, 62.92147576780195, 64.06715253948899, 65.66372685250829, 66.90298613277373, 68.10537957632647, 69.45807220032329, 70.70300314777529, 71.56226072654057, 72.49808581232453, 73.59271757933244, 74.6079460057284, 75.39063607747497, 76.25272948983354, 76.93900121940844, 77.61960128179678, 78.32855968011798, 79.12542891983098, 79.83438731815217, 80.64259989223832, 81.26364744916769, 81.85917250375748, 82.48589172787341, 83.11828261917591, 83.696792672206, 84.21858605337039, 84.63261775798996, 85.14023197118793, 85.73008535859115, 86.22919207100928, 86.82188129200578, 87.23307716303206, 87.6953180387375, 88.1008422425772, 88.52905311516321, 89.03666732836118, 89.43651986501432, 89.92144740946601, 90.25607577347361, 90.66159997731333, 91.00190000850749, 91.49249922014576, 91.90936675835863, 92.40847347077673, 92.84519184414258, 93.34713439015398, 93.78101692992655, 94.2205711368857, 94.73385701727022, 95.15072455548308, 95.57609959447579, 96.08654964126706, 96.66222386070386, 97.31162975356607, 97.88446813940958, 98.5310381986785, 99.00178657516378, 99.43850494852961, 100.0], "exact_score": [66.95119530385956, 67.60627286390834, 68.1535887474123, 68.58463545359157, 69.01851799336414, 69.34747469018518, 69.70478972293905, 70.05926892209965, 70.38538978532739, 70.72001814933499, 71.05181067974931, 71.36658820860391, 71.69554490542495, 72.01315826787284, 72.2995774607946, 72.5434591498171, 72.75331083572016, 72.95749085443666, 73.21555171142558, 73.35734339108981, 73.5246575730936, 73.6749567535377, 73.8989876074072, 74.07197345659756, 74.17122763236253, 74.2704818081275, 74.40376598701188, 74.51436349714999, 74.60511017213508, 74.72137934945977, 74.81212602444488, 74.91705186739642, 75.01914187675467, 75.11272438533307, 75.22899356265775, 75.28854606811673, 75.31406857045629, 75.37078524232199, 75.41899441340783, 75.45586025012052, 75.51257692198622, 75.60615943056462, 75.6628761024303, 75.71392110710944, 75.73660777585572, 75.83302611802739, 75.86422028755354, 75.90392195785951, 75.92660862660578, 75.98899696565805, 76.02019113518419, 76.08825114142302, 76.11377364376258, 76.15063948047528, 76.16765448203499, 76.21019198593426, 76.23855032186711, 76.2584011570201, 76.2952669937328, 76.28959532654623, 76.26974449139324, 76.2782519921731, 76.27258032498652, 76.27541615857982, 76.2782519921731, 76.28675949295295, 76.30377449451267, 76.29810282732609, 76.30661032810595, 76.27541615857982, 76.26974449139324, 76.23855032186711, 76.24705782264697, 76.2413861554604, 76.24705782264697, 76.24705782264697, 76.2045203187477, 76.16481864844171, 76.09108697501631, 76.02869863596405, 75.9407877945722, 75.90108612426623, 75.84153361880725, 75.81601111646769, 75.76496611178855, 75.67989110399002, 75.61466693134446, 75.58347276181834, 75.49272608683322, 75.4586960837138, 75.38780024388168, 75.33959107279584, 75.29988940248987, 75.26869523296374, 75.16944105719875, 75.11556021892635, 75.04182854550095, 74.95958937129569, 74.87167852990386, 74.7355585174262, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 72.99095369083743, "exact_area100": 74.64040911488723}}, "old_normalized_endpoint_budget": 43}, {"seed": 2029, "selection": {"best_epoch": 9, "epochs": 17, "alpha": 1.0, "validation_area100": 0.8110959321493891, "training_seconds": 116.77155173604842, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5325723145621717, "operator": "ge"}, {"budget": 0.02, "threshold": 0.4067695427443491, "operator": "ge"}, {"budget": 0.03, "threshold": 0.31187039210598305, "operator": "gt"}, {"budget": 0.04, "threshold": 0.2331521202884942, "operator": "ge"}, {"budget": 0.05, "threshold": 0.15709308026360513, "operator": "ge"}, {"budget": 0.06, "threshold": 0.08008151112520076, "operator": "ge"}, {"budget": 0.07, "threshold": 0.028390458738361116, "operator": "ge"}, {"budget": 0.08, "threshold": -0.006843246731169336, "operator": "ge"}, {"budget": 0.09, "threshold": -0.1098364561786197, "operator": "ge"}, {"budget": 0.1, "threshold": -0.1758496390141978, "operator": "ge"}, {"budget": 0.11, "threshold": -0.23171011992415072, "operator": "ge"}, {"budget": 0.12, "threshold": -0.27879973002406827, "operator": "ge"}, {"budget": 0.13, "threshold": -0.3328027863514404, "operator": "ge"}, {"budget": 0.14, "threshold": -0.3774502916816955, "operator": "ge"}, {"budget": 0.15, "threshold": -0.42269571749061857, "operator": "ge"}, {"budget": 0.16, "threshold": -0.47185128574847046, "operator": "ge"}, {"budget": 0.17, "threshold": -0.5172180070814879, "operator": "ge"}, {"budget": 0.18, "threshold": -0.5561248875864963, "operator": "ge"}, {"budget": 0.19, "threshold": -0.5962985061774282, "operator": "ge"}, {"budget": 0.2, "threshold": -0.6385400839356341, "operator": "ge"}, {"budget": 0.21, "threshold": -0.67022728780168, "operator": "ge"}, {"budget": 0.22, "threshold": -0.7051588603740776, "operator": "ge"}, {"budget": 0.23, "threshold": -0.7381060435224306, "operator": "ge"}, {"budget": 0.24, "threshold": -0.7650144763779847, "operator": "ge"}, {"budget": 0.25, "threshold": -0.791776753143636, "operator": "gt"}, {"budget": 0.26, "threshold": -0.8141108249835886, "operator": "ge"}, {"budget": 0.27, "threshold": -0.8319589804321335, "operator": "ge"}, {"budget": 0.28, "threshold": -0.8498608222866426, "operator": "ge"}, {"budget": 0.29, "threshold": -0.8628887312261464, "operator": "ge"}, {"budget": 0.3, "threshold": -0.8780153515440623, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8922865885799318, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9023421554965512, "operator": "ge"}, {"budget": 0.33, "threshold": -0.9103581611311545, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9182073599571471, "operator": "ge"}, {"budget": 0.35, "threshold": -0.9249834926269718, "operator": "ge"}, {"budget": 0.36, "threshold": -0.931179630867001, "operator": "gt"}, {"budget": 0.37, "threshold": -0.9374721983714199, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9426640296723524, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9474177794598272, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9513507293745237, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9548810568439754, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9583213465307607, "operator": "gt"}, {"budget": 0.43, "threshold": -0.9615091969946127, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9642834129211594, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9669265301487364, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9692689422242841, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9713415490172952, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9736670459029321, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9756335055960895, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9772827447891147, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9787984754119685, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9804333790079598, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9816076365963224, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9828119418586799, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9838575854978311, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9850802532915908, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9861366086388176, "operator": "ge"}, {"budget": 0.58, "threshold": -0.986909486800482, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9878463549161831, "operator": "ge"}, {"budget": 0.6, "threshold": -0.988515471922583, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9892765264360719, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9900681241107026, "operator": "ge"}, {"budget": 0.63, "threshold": -0.990666993386295, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9912750611053628, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9918197970521729, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9923604265516668, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9928092003821612, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9932157450920265, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9936328553327375, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9940165359292052, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9943510355978359, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9947043286646556, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9950764262704339, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9953543767770301, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9956860416932254, "operator": "ge"}, {"budget": 0.76, "threshold": -0.996008923433623, "operator": "ge"}, {"budget": 0.77, "threshold": -0.996297370001308, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9965235058196293, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9967682967556245, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9969873232494728, "operator": "gt"}, {"budget": 0.81, "threshold": -0.9972060356889526, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9973664199226747, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9975257264440612, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9977011213416939, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9978886012048336, "operator": "ge"}, {"budget": 0.86, "threshold": -0.998047231251578, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9982046663176033, "operator": "ge"}, {"budget": 0.88, "threshold": -0.998375397139681, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9984994908981886, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9986294819045215, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9987655018166931, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9988859142024981, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9990017082335876, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9991290375777043, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9992528770429754, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9993821583145016, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9995020143195598, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9996316412690897, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9997690209102057, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.21259399817275, 75.254761402769, 75.26178930353502, 75.27584510506712, 75.4445147234521, 75.54290533417668, 75.77482605945605, 76.04891418933165, 76.280834914611, 76.61817415138098, 76.94145758661888, 77.36313163258134, 77.81291728160798, 78.1221449153138, 78.42434464825357, 78.79682338885375, 79.19038583175205, 79.54880877082016, 79.85100850375993, 80.25159884742428, 80.546770679598, 80.76463560334528, 80.87708201560194, 81.17928174854171, 81.37606296999087, 81.53067678684377, 81.75556961135709, 81.91721132897604, 82.04371354276478, 82.19832735961768, 82.32482957340642, 82.40213648183288, 82.48647129102538, 82.70433621477264, 82.83786632932744, 82.88003373392368, 82.95031274158409, 83.04870335230868, 83.04870335230868, 83.18223346686345, 83.28765197835408, 83.3438751844824, 83.37901468831261, 83.42118209290885, 83.3438751844824, 83.32279148218427, 83.35090308524843, 83.3298193829503, 83.30170777988614, 83.21034506992761, 83.20331716916158, 83.18223346686345, 83.14006606226721, 83.13303816150116, 83.13303816150116, 83.13303816150116, 83.05573125307471, 82.9854522454143, 82.90814533698784, 82.91517323775388, 82.83083842856139, 82.7465036193689, 82.66919671094244, 82.65514090941036, 82.5426944971537, 82.50052709255745, 82.35294117647058, 82.28266216881018, 82.2194110619158, 82.17724365731955, 82.07885304659499, 82.05074144353081, 82.00857403893457, 81.9453229320402, 81.86801602361375, 81.81179281748541, 81.75556961135709, 81.63609529833438, 81.57987209220606, 81.53067678684377, 81.48850938224751, 81.43228617611919, 81.39011877152295, 81.3479513669267, 81.31983976386253, 81.29875606156442, 81.28470026003232, 81.22144915313795, 81.17225384777566, 81.11603064164734, 81.02466793168881, 80.94736102326236, 80.93330522173027, 80.88410991636799, 80.85599831330381, 80.80680300794153, 80.76463560334528, 80.74355190104716, 80.68732869491882, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.991215124042448, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.9898798228969, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.991707077096066, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.99156651908075, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 42, 69, 92, 116, 154, 188, 233, 280, 322, 377, 432, 497, 565, 621, 677, 738, 803, 867, 919, 985, 1037, 1082, 1119, 1175, 1217, 1253, 1295, 1336, 1370, 1399, 1427, 1453, 1475, 1515, 1540, 1559, 1579, 1604, 1617, 1645, 1670, 1690, 1710, 1726, 1736, 1751, 1760, 1774, 1786, 1793, 1805, 1817, 1823, 1833, 1843, 1848, 1852, 1858, 1866, 1872, 1876, 1880, 1885, 1892, 1893, 1900, 1901, 1902, 1906, 1909, 1911, 1914, 1918, 1923, 1926, 1928, 1928, 1931, 1934, 1937, 1938, 1939, 1941, 1945, 1948, 1950, 1951, 1951, 1953, 1954, 1955, 1955, 1956, 1956, 1958, 1958, 1960, 1963, 1964, 1964], "harmed": [0, 16, 37, 59, 81, 95, 115, 127, 135, 144, 151, 160, 165, 169, 181, 194, 202, 211, 224, 233, 242, 252, 266, 287, 300, 314, 328, 338, 356, 372, 379, 389, 404, 414, 423, 429, 442, 452, 463, 476, 485, 495, 507, 522, 532, 553, 571, 576, 593, 609, 629, 642, 657, 669, 680, 690, 695, 710, 726, 745, 750, 766, 782, 798, 807, 824, 837, 859, 870, 883, 892, 908, 915, 925, 939, 953, 963, 971, 991, 1002, 1012, 1019, 1028, 1036, 1046, 1053, 1058, 1061, 1070, 1079, 1088, 1102, 1113, 1116, 1123, 1129, 1136, 1144, 1150, 1159, 1171], "exact_score": [75.02986857825567, 75.21259399817275, 75.254761402769, 75.26881720430107, 75.27584510506712, 75.4445147234521, 75.54290533417668, 75.77482605945605, 76.04891418933165, 76.280834914611, 76.61817415138098, 76.94145758661888, 77.36313163258134, 77.81291728160798, 78.1221449153138, 78.42434464825357, 78.79682338885375, 79.19038583175205, 79.54880877082016, 79.85100850375993, 80.25159884742428, 80.546770679598, 80.76463560334528, 80.87708201560194, 81.17928174854171, 81.37606296999087, 81.53067678684377, 81.75556961135709, 81.91721132897604, 82.04371354276478, 82.19832735961768, 82.32482957340642, 82.40213648183288, 82.48647129102538, 82.70433621477264, 82.83786632932744, 82.88003373392368, 82.95031274158409, 83.04870335230868, 83.04870335230868, 83.18223346686345, 83.28765197835408, 83.3438751844824, 83.37901468831261, 83.42118209290885, 83.3438751844824, 83.32279148218427, 83.35090308524843, 83.3298193829503, 83.30170777988614, 83.21034506992761, 83.20331716916158, 83.18223346686345, 83.14006606226721, 83.13303816150116, 83.13303816150116, 83.13303816150116, 83.05573125307471, 82.9854522454143, 82.90814533698784, 82.91517323775388, 82.83083842856139, 82.7465036193689, 82.66919671094244, 82.65514090941036, 82.5426944971537, 82.50052709255745, 82.35294117647058, 82.28266216881018, 82.2194110619158, 82.17724365731955, 82.07885304659499, 82.05074144353081, 82.00857403893457, 81.9453229320402, 81.86801602361375, 81.81179281748541, 81.75556961135709, 81.63609529833438, 81.57987209220606, 81.53067678684377, 81.48850938224751, 81.43228617611919, 81.39011877152295, 81.3479513669267, 81.31983976386253, 81.29875606156442, 81.28470026003232, 81.22144915313795, 81.17225384777566, 81.11603064164734, 81.02466793168881, 80.94736102326236, 80.93330522173027, 80.88410991636799, 80.85599831330381, 80.80680300794153, 80.76463560334528, 80.74355190104716, 80.68732869491882, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.10959321493891, "threshold_area100": 81.10931899641577, "threshold_area25": 77.78831962892684}, "test": {"score": [66.95119530385956, 67.52403368970309, 67.94090122791594, 68.34642543175566, 68.79448713949465, 69.2879221847262, 69.94583557836826, 70.77106315401413, 71.45166321640247, 71.99614326631313, 72.41868247171256, 72.74763916853358, 73.00286419192922, 73.3034625528174, 73.5246575730936, 73.81391259960866, 74.00107761676544, 74.1570484643961, 74.4179451549783, 74.59376683776196, 74.78376768851204, 74.90570853302327, 75.05884354706066, 75.20630689391147, 75.35377024076227, 75.4955619204265, 75.6657119360236, 75.76496611178855, 75.86422028755354, 75.93228029379236, 76.02586280237075, 76.11660947735587, 76.18750531718798, 76.20735615234098, 76.25272948983354, 76.30093866091937, 76.32078949607236, 76.34347616481864, 76.39735700309106, 76.39452116949778, 76.4087003374642, 76.42571533902391, 76.42571533902391, 76.40586450387092, 76.43138700621047, 76.46258117573662, 76.44556617417689, 76.43138700621047, 76.3916853359045, 76.38601366871791, 76.37750616793807, 76.35765533278507, 76.32646116325894, 76.30377449451267, 76.2952669937328, 76.26974449139324, 76.2215353203074, 76.19601281796784, 76.17616198281485, 76.14496781328872, 76.14496781328872, 76.127952811729, 76.127952811729, 76.07974364064316, 76.03153446955733, 76.01735530159091, 75.9975044664379, 75.96914613050507, 75.91810112582593, 75.87839945551995, 75.81884695006097, 75.77914527975497, 75.74795111022885, 75.68839860476987, 75.6657119360236, 75.64586110087059, 75.57496526103849, 75.54660692510564, 75.49839775401979, 75.46720358449366, 75.41048691262796, 75.37645690950855, 75.32541190482942, 75.31406857045629, 75.28287440093015, 75.23182939625103, 75.20063522672488, 75.18362022516519, 75.15809772282563, 75.14675438845248, 75.12123188611292, 75.07869438221365, 75.0701868814338, 75.02764937753452, 74.96526103848225, 74.92272353458299, 74.85182769475087, 74.83481269319117, 74.76958852054561, 74.66182684400079, 74.47182599325072], "calls": [0.0, 1.182542608399739, 2.1807560332359697, 3.2413577971244645, 4.472109576610044, 5.949578878711397, 8.045259904148825, 10.506763463119984, 12.90587868303888, 14.831409692879221, 16.218132319995462, 17.604854947111704, 18.736352550832315, 20.066358506082864, 21.22337861214304, 22.44278705725548, 23.795479681252303, 25.02906729433117, 26.19743073476448, 27.510421688455324, 29.016249326489525, 30.21580693644897, 31.707455406516743, 32.997759691461304, 34.20582480220061, 35.61239826446984, 37.07285256501148, 38.3347985140232, 39.7839094801917, 41.02316876045713, 42.571533902390605, 44.25601905680175, 45.560502509712734, 46.67782094546692, 47.919916059325644, 49.156339505997785, 50.39559878626322, 51.67739557042793, 52.908147349913506, 54.13322746221252, 55.18248589172787, 56.21756515327681, 57.44831693276239, 58.69324788021439, 59.7680288120693, 60.893854748603346, 61.91475484218586, 62.92147576780195, 64.03595836996286, 65.05118679635878, 65.99551938292261, 67.04194197884468, 68.12523041147945, 68.83986047698721, 69.80120806511074, 70.67180897824915, 71.67285823667868, 72.49241414513796, 73.13331253722032, 74.02660011910501, 74.6419760088478, 75.37078524232199, 76.19884865156112, 76.92482205144202, 77.68482545444233, 78.33706718089782, 78.9751297393869, 79.6415506338088, 80.28812069307773, 80.88364574766753, 81.47633496866405, 81.96977001389558, 82.61066840597793, 83.28843263477299, 83.7705243456314, 84.36321356662792, 85.01829112667669, 85.64217451719934, 86.13844539602417, 86.69143294671468, 87.21039049428579, 87.79173638090917, 88.21994725349516, 88.64532229248788, 89.11607066897315, 89.74846156027564, 90.24756827269375, 90.7410033179253, 91.32802087173525, 91.78175424666081, 92.26101012392593, 92.82817684258288, 93.3442985565607, 93.8831069392848, 94.50982616340073, 95.21878456172192, 95.98729546550209, 96.65938802711057, 97.55551144258855, 98.52820236508522, 100.0], "rescued": [0, 228, 390, 556, 729, 924, 1186, 1496, 1753, 1981, 2156, 2310, 2427, 2568, 2680, 2810, 2924, 3015, 3135, 3233, 3343, 3417, 3514, 3609, 3689, 3773, 3865, 3943, 4014, 4077, 4137, 4208, 4262, 4293, 4334, 4375, 4418, 4466, 4513, 4544, 4571, 4608, 4636, 4665, 4701, 4736, 4770, 4795, 4820, 4847, 4868, 4890, 4913, 4923, 4950, 4970, 4992, 5015, 5031, 5048, 5064, 5082, 5106, 5118, 5130, 5149, 5161, 5174, 5190, 5203, 5212, 5219, 5225, 5236, 5241, 5253, 5260, 5274, 5277, 5286, 5290, 5298, 5305, 5314, 5324, 5331, 5335, 5339, 5349, 5355, 5362, 5368, 5379, 5382, 5389, 5395, 5397, 5406, 5414, 5416, 5423], "harmed": [0, 26, 41, 64, 79, 100, 130, 149, 166, 202, 228, 266, 293, 328, 362, 390, 438, 474, 502, 538, 581, 612, 655, 698, 726, 760, 792, 835, 871, 910, 937, 976, 1005, 1029, 1054, 1078, 1114, 1154, 1182, 1214, 1236, 1267, 1295, 1331, 1358, 1382, 1422, 1452, 1491, 1520, 1544, 1573, 1607, 1625, 1655, 1684, 1723, 1755, 1778, 1806, 1822, 1846, 1870, 1899, 1928, 1952, 1971, 1994, 2028, 2055, 2085, 2106, 2123, 2155, 2168, 2187, 2219, 2243, 2263, 2283, 2307, 2327, 2352, 2365, 2386, 2411, 2426, 2436, 2455, 2465, 2481, 2502, 2516, 2534, 2563, 2584, 2611, 2626, 2657, 2697, 2771], "exact_score": [66.95119530385956, 67.43895868190455, 67.88134872245696, 68.25000708958399, 68.618665456711, 68.98448799024473, 69.31060885347247, 69.62538638232708, 69.94299974477497, 70.27479227518928, 70.62359980716332, 70.89016816493208, 71.15957235629413, 71.46584238436888, 71.74942574369736, 72.05285993817883, 72.33077163032074, 72.62002665683578, 72.83838584351871, 73.0822675325412, 73.29211921844426, 73.4509258996682, 73.73450925899668, 73.8791367722542, 74.02376428551173, 74.15137679720954, 74.37257181748575, 74.51152766355669, 74.67884184556051, 74.78376768851204, 74.88018603068372, 74.95958937129569, 75.12973938689278, 75.20630689391147, 75.32257607123614, 75.42750191418767, 75.54377109151234, 75.65436860165046, 75.72810027507586, 75.81033944928112, 75.90675779145279, 75.93228029379236, 75.97765363128491, 76.05705697189688, 76.09108697501631, 76.16481864844171, 76.1846694835947, 76.21869948671412, 76.25556532342682, 76.30093866091937, 76.32646116325894, 76.33213283044552, 76.34347616481864, 76.40586450387092, 76.40019283668434, 76.38601366871791, 76.40019283668434, 76.43422283980375, 76.40019283668434, 76.41153617105748, 76.43422283980375, 76.45407367495676, 76.43989450699033, 76.43422283980375, 76.39452116949778, 76.38034200153135, 76.37750616793807, 76.35481949919179, 76.33213283044552, 76.2952669937328, 76.29810282732609, 76.24422198905368, 76.19317698437456, 76.19317698437456, 76.14496781328872, 76.14496781328872, 76.13646031250886, 76.0712361398633, 76.01168363440433, 75.9975044664379, 75.9237727930125, 75.86422028755354, 75.77914527975497, 75.71108527351615, 75.6628761024303, 75.5721294274452, 75.51824858917279, 75.43600941496753, 75.35944190794885, 75.28854606811673, 75.21481439469132, 75.16376939001219, 75.14108272126592, 75.0872018829935, 75.0134702095681, 74.92839520176956, 74.85466352834416, 74.82346935881802, 74.7355585174262, 74.59943850494852, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.60342618735096, "threshold_area100": 74.94872812863342, "threshold_area25": 72.3055327113405}, "selected_budget": 44, "selected": {"val": {"score": 83.42118209290885, "calls": 43.994658795417806, "rescued": 1726, "harmed": 532}, "test": {"score": 76.43138700621047, "calls": 59.7680288120693, "rescued": 4701, "harmed": 1358}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6689887642860413, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5461115837097168, "operator": "ge"}, {"budget": 0.03, "threshold": 0.43494293093681335, "operator": "ge"}, {"budget": 0.04, "threshold": 0.322523832321167, "operator": "ge"}, {"budget": 0.05, "threshold": 0.22379478812217712, "operator": "ge"}, {"budget": 0.06, "threshold": 0.1223272830247879, "operator": "ge"}, {"budget": 0.07, "threshold": 0.04737471044063568, "operator": "ge"}, {"budget": 0.08, "threshold": -0.00032596098026260734, "operator": "ge"}, {"budget": 0.09, "threshold": -0.0015114336274564266, "operator": "gt"}, {"budget": 0.1, "threshold": -0.0023568435572087765, "operator": "ge"}, {"budget": 0.11, "threshold": -0.0031604457180947065, "operator": "ge"}, {"budget": 0.12, "threshold": -0.004000193905085325, "operator": "ge"}, {"budget": 0.13, "threshold": -0.004960428923368454, "operator": "ge"}, {"budget": 0.14, "threshold": -0.005849175155162811, "operator": "ge"}, {"budget": 0.15, "threshold": -0.007008006330579519, "operator": "ge"}, {"budget": 0.16, "threshold": -0.008114097639918327, "operator": "ge"}, {"budget": 0.17, "threshold": -0.009143037721514702, "operator": "ge"}, {"budget": 0.18, "threshold": -0.010183100588619709, "operator": "gt"}, {"budget": 0.19, "threshold": -0.011444058269262314, "operator": "ge"}, {"budget": 0.2, "threshold": -0.012691826559603214, "operator": "ge"}, {"budget": 0.21, "threshold": -0.013975732959806919, "operator": "ge"}, {"budget": 0.22, "threshold": -0.015170459635555744, "operator": "ge"}, {"budget": 0.23, "threshold": -0.01687026023864746, "operator": "ge"}, {"budget": 0.24, "threshold": -0.018603745847940445, "operator": "ge"}, {"budget": 0.25, "threshold": -0.020507585257291794, "operator": "ge"}, {"budget": 0.26, "threshold": -0.022112276405096054, "operator": "ge"}, {"budget": 0.27, "threshold": -0.024001482874155045, "operator": "ge"}, {"budget": 0.28, "threshold": -0.026147902011871338, "operator": "gt"}, {"budget": 0.29, "threshold": -0.028318127617239952, "operator": "gt"}, {"budget": 0.3, "threshold": -0.03044208139181137, "operator": "ge"}, {"budget": 0.31, "threshold": -0.032616931945085526, "operator": "ge"}, {"budget": 0.32, "threshold": -0.03511759638786316, "operator": "ge"}, {"budget": 0.33, "threshold": -0.037255480885505676, "operator": "ge"}, {"budget": 0.34, "threshold": -0.04017435386776924, "operator": "ge"}, {"budget": 0.35, "threshold": -0.0430263951420784, "operator": "ge"}, {"budget": 0.36, "threshold": -0.04615183174610138, "operator": "ge"}, {"budget": 0.37, "threshold": -0.048930756747722626, "operator": "ge"}, {"budget": 0.38, "threshold": -0.051923610270023346, "operator": "ge"}, {"budget": 0.39, "threshold": -0.05536773055791855, "operator": "ge"}, {"budget": 0.4, "threshold": -0.05870985984802246, "operator": "ge"}, {"budget": 0.41, "threshold": -0.06270545721054077, "operator": "ge"}, {"budget": 0.42, "threshold": -0.0655231848359108, "operator": "ge"}, {"budget": 0.43, "threshold": -0.06912699341773987, "operator": "ge"}, {"budget": 0.44, "threshold": -0.07231558859348297, "operator": "ge"}, {"budget": 0.45, "threshold": -0.07589725404977798, "operator": "ge"}, {"budget": 0.46, "threshold": -0.0805300921201706, "operator": "ge"}, {"budget": 0.47, "threshold": -0.08471489697694778, "operator": "ge"}, {"budget": 0.48, "threshold": -0.08899372816085815, "operator": "ge"}, {"budget": 0.49, "threshold": -0.09267771244049072, "operator": "ge"}, {"budget": 0.5, "threshold": -0.09677847474813461, "operator": "ge"}, {"budget": 0.51, "threshold": -0.10091215372085571, "operator": "ge"}, {"budget": 0.52, "threshold": -0.10529367625713348, "operator": "ge"}, {"budget": 0.53, "threshold": -0.1100253239274025, "operator": "ge"}, {"budget": 0.54, "threshold": -0.11440827697515488, "operator": "ge"}, {"budget": 0.55, "threshold": -0.11890578269958496, "operator": "ge"}, {"budget": 0.56, "threshold": -0.12347389757633209, "operator": "ge"}, {"budget": 0.57, "threshold": -0.12885543704032898, "operator": "ge"}, {"budget": 0.58, "threshold": -0.133906289935112, "operator": "ge"}, {"budget": 0.59, "threshold": -0.13983702659606934, "operator": "ge"}, {"budget": 0.6, "threshold": -0.14513447880744934, "operator": "ge"}, {"budget": 0.61, "threshold": -0.15098188817501068, "operator": "ge"}, {"budget": 0.62, "threshold": -0.15752391517162323, "operator": "ge"}, {"budget": 0.63, "threshold": -0.16420426964759827, "operator": "ge"}, {"budget": 0.64, "threshold": -0.17003700137138367, "operator": "ge"}, {"budget": 0.65, "threshold": -0.17614299058914185, "operator": "gt"}, {"budget": 0.66, "threshold": -0.18457426130771637, "operator": "ge"}, {"budget": 0.67, "threshold": -0.1921650916337967, "operator": "ge"}, {"budget": 0.68, "threshold": -0.19909140467643738, "operator": "ge"}, {"budget": 0.69, "threshold": -0.20806914567947388, "operator": "ge"}, {"budget": 0.7, "threshold": -0.21659953892230988, "operator": "ge"}, {"budget": 0.71, "threshold": -0.22426943480968475, "operator": "ge"}, {"budget": 0.72, "threshold": -0.23323661088943481, "operator": "ge"}, {"budget": 0.73, "threshold": -0.24373473227024078, "operator": "ge"}, {"budget": 0.74, "threshold": -0.25242623686790466, "operator": "ge"}, {"budget": 0.75, "threshold": -0.26291623711586, "operator": "ge"}, {"budget": 0.76, "threshold": -0.2719281017780304, "operator": "ge"}, {"budget": 0.77, "threshold": -0.28163012862205505, "operator": "ge"}, {"budget": 0.78, "threshold": -0.2939147353172302, "operator": "ge"}, {"budget": 0.79, "threshold": -0.3058071732521057, "operator": "ge"}, {"budget": 0.8, "threshold": -0.3174147605895996, "operator": "ge"}, {"budget": 0.81, "threshold": -0.33085399866104126, "operator": "ge"}, {"budget": 0.82, "threshold": -0.34322744607925415, "operator": "ge"}, {"budget": 0.83, "threshold": -0.3592815101146698, "operator": "ge"}, {"budget": 0.84, "threshold": -0.37414729595184326, "operator": "ge"}, {"budget": 0.85, "threshold": -0.3890427350997925, "operator": "ge"}, {"budget": 0.86, "threshold": -0.40622732043266296, "operator": "ge"}, {"budget": 0.87, "threshold": -0.4252924621105194, "operator": "ge"}, {"budget": 0.88, "threshold": -0.44474488496780396, "operator": "ge"}, {"budget": 0.89, "threshold": -0.464829683303833, "operator": "ge"}, {"budget": 0.9, "threshold": -0.48571035265922546, "operator": "ge"}, {"budget": 0.91, "threshold": -0.5094377398490906, "operator": "ge"}, {"budget": 0.92, "threshold": -0.5364593267440796, "operator": "ge"}, {"budget": 0.93, "threshold": -0.5663648843765259, "operator": "ge"}, {"budget": 0.94, "threshold": -0.6030965447425842, "operator": "ge"}, {"budget": 0.95, "threshold": -0.6461341977119446, "operator": "ge"}, {"budget": 0.96, "threshold": -0.6913468241691589, "operator": "ge"}, {"budget": 0.97, "threshold": -0.7391377091407776, "operator": "ge"}, {"budget": 0.98, "threshold": -0.8005883693695068, "operator": "ge"}, {"budget": 0.99, "threshold": -0.8701862692832947, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.19151029587462, 75.23367770047086, 75.2899009065992, 75.32504041042941, 75.47262632651626, 75.62021224260313, 75.81699346405229, 76.0208025862675, 75.99269098320332, 76.0208025862675, 76.03485838779956, 76.05594209009769, 76.09810949469393, 76.11216529622602, 76.10513739545998, 76.09810949469393, 76.0910815939279, 76.11216529622602, 76.11216529622602, 76.13324899852414, 76.14027689929019, 76.15433270082227, 76.1754164031204, 76.18244430388643, 76.20352800618456, 76.21758380771664, 76.23866751001475, 76.27380701384496, 76.30191861690912, 76.32300231920726, 76.35111392227142, 76.37219762456954, 76.37219762456954, 76.42139292993183, 76.42139292993183, 76.4143650291658, 76.47761613606015, 76.54086724295453, 76.59006254831682, 76.65331365521119, 76.69548105980743, 76.70953686133953, 76.7587321667018, 76.77278796823389, 76.83603907512826, 76.88523438049056, 76.90631808278867, 76.93442968585283, 76.99065289198117, 77.06093189964159, 77.07498770117365, 77.06795980040762, 77.15932251036615, 77.19446201419636, 77.34907583104926, 77.4052990371776, 77.39124323564552, 77.41935483870968, 77.51071754866821, 77.5599128540305, 77.58802445709466, 77.67235926628716, 77.644247663223, 77.76372197624569, 77.84102888467214, 77.96050319769485, 78.03078220535527, 78.08700541148359, 78.16431231991004, 78.25567502986857, 78.34000983906108, 78.3610935413592, 78.4805678543819, 78.67734907583105, 78.79682338885375, 78.90224190034438, 79.0427999156652, 79.15524632792184, 79.20444163328413, 79.34499964860497, 79.39419495396724, 79.44339025932953, 79.53475296928806, 79.66125518307682, 79.73153419073724, 79.86506430529201, 79.92128751142033, 80.02670602291096, 80.11104083210346, 80.25159884742428, 80.37810106121302, 80.44838006887343, 80.51865907653384, 80.6311054887905, 80.6662449926207, 80.74355190104716, 80.79274720640944, 80.75760770257924, 80.75057980181319, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.981657179000631, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99142596106543, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.992128751142033, 28.990090659919883, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.98699838358283, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 39, 67, 94, 120, 157, 197, 237, 274, 275, 281, 283, 286, 293, 297, 299, 301, 303, 306, 309, 313, 317, 320, 325, 328, 333, 337, 344, 352, 357, 362, 370, 378, 386, 396, 402, 408, 420, 433, 444, 459, 472, 481, 493, 502, 518, 533, 544, 556, 570, 584, 596, 610, 634, 648, 674, 691, 706, 722, 744, 762, 781, 806, 820, 847, 868, 892, 911, 937, 960, 991, 1017, 1042, 1075, 1116, 1148, 1177, 1211, 1240, 1266, 1298, 1324, 1353, 1383, 1423, 1455, 1492, 1520, 1553, 1584, 1625, 1666, 1702, 1737, 1772, 1802, 1838, 1872, 1905, 1940, 1964], "harmed": [0, 16, 38, 57, 78, 94, 113, 125, 133, 138, 140, 140, 140, 141, 143, 146, 149, 152, 152, 155, 156, 159, 160, 162, 164, 166, 168, 172, 175, 176, 178, 182, 187, 195, 198, 204, 211, 214, 218, 222, 228, 235, 242, 247, 254, 261, 269, 277, 285, 291, 295, 305, 320, 331, 340, 344, 353, 370, 382, 391, 402, 417, 430, 448, 458, 468, 475, 484, 502, 514, 532, 546, 568, 584, 597, 612, 626, 640, 653, 672, 684, 703, 725, 742, 764, 786, 804, 824, 842, 861, 882, 905, 931, 956, 975, 1000, 1025, 1052, 1090, 1126, 1171], "exact_score": [75.02986857825567, 75.19151029587462, 75.23367770047086, 75.2899009065992, 75.32504041042941, 75.47262632651626, 75.62021224260313, 75.81699346405229, 76.0208025862675, 75.99269098320332, 76.0208025862675, 76.03485838779956, 76.05594209009769, 76.09810949469393, 76.11216529622602, 76.10513739545998, 76.09810949469393, 76.0910815939279, 76.11216529622602, 76.11216529622602, 76.13324899852414, 76.14027689929019, 76.15433270082227, 76.1754164031204, 76.18244430388643, 76.20352800618456, 76.21758380771664, 76.23866751001475, 76.27380701384496, 76.30191861690912, 76.32300231920726, 76.35111392227142, 76.37219762456954, 76.37219762456954, 76.42139292993183, 76.42139292993183, 76.4143650291658, 76.47761613606015, 76.54086724295453, 76.59006254831682, 76.65331365521119, 76.69548105980743, 76.70953686133953, 76.7587321667018, 76.77278796823389, 76.83603907512826, 76.88523438049056, 76.90631808278867, 76.93442968585283, 76.99065289198117, 77.06093189964159, 77.07498770117365, 77.06795980040762, 77.15932251036615, 77.19446201419636, 77.34907583104926, 77.4052990371776, 77.39124323564552, 77.41935483870968, 77.51071754866821, 77.5599128540305, 77.58802445709466, 77.67235926628716, 77.644247663223, 77.76372197624569, 77.84805678543819, 77.96050319769485, 78.03078220535527, 78.08700541148359, 78.16431231991004, 78.25567502986857, 78.34000983906108, 78.3610935413592, 78.4805678543819, 78.67734907583105, 78.79682338885375, 78.90224190034438, 79.0427999156652, 79.15524632792184, 79.20444163328413, 79.34499964860497, 79.39419495396724, 79.44339025932953, 79.53475296928806, 79.66125518307682, 79.73153419073724, 79.86506430529201, 79.92128751142033, 80.02670602291096, 80.11104083210346, 80.25159884742428, 80.37810106121302, 80.44838006887343, 80.51865907653384, 80.6311054887905, 80.6662449926207, 80.74355190104716, 80.79274720640944, 80.75760770257924, 80.75057980181319, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 77.542281115922, "threshold_area100": 77.54202684658092, "threshold_area25": 75.86970271979759}, "test": {"score": [66.95119530385956, 67.54388452485608, 67.95224456228908, 68.37478376768851, 68.91075631681933, 69.40135552845759, 70.07911975725264, 70.78524232198055, 71.37509570938377, 71.39211071094348, 71.4062898789099, 71.41763321328304, 71.44031988202933, 71.44315571562261, 71.45449904999575, 71.47151405155545, 71.49703655389501, 71.4998723874883, 71.51121572186145, 71.53390239060772, 71.536738224201, 71.55942489294728, 71.53390239060772, 71.53957405779428, 71.52255905623458, 71.54524572498086, 71.56793239372713, 71.5934548960667, 71.62181323199954, 71.64166406715253, 71.67002240308538, 71.69270907183166, 71.73524657573094, 71.76927657885035, 71.78345574681677, 71.80330658196978, 71.84584408586903, 71.88838158976831, 71.91957575929445, 71.97345659756685, 72.00748660068626, 72.07271077333182, 72.10390494285795, 72.15211411394378, 72.18897995065649, 72.21166661940276, 72.25136828970876, 72.2995774607946, 72.3194282959476, 72.38748830218643, 72.41584663811928, 72.43569747327227, 72.4668916427984, 72.48390664435811, 72.5434591498171, 72.55196665059694, 72.5945041544962, 72.64271332558205, 72.68525082948132, 72.75898250290673, 72.81002750758586, 72.91778918413068, 73.00002835833594, 73.10779003488075, 73.1559992059666, 73.21838754501886, 73.25525338173155, 73.28077588407112, 73.35167172390325, 73.39988089498908, 73.48212006919434, 73.55585174261974, 73.59555341292572, 73.69764342228397, 73.75719592774296, 73.83092760116836, 73.9160026089669, 74.00674928395202, 74.07197345659756, 74.12585429486998, 74.15421263080282, 74.21660096985508, 74.26197430734764, 74.31869097921334, 74.37257181748575, 74.45764682528429, 74.53421433230298, 74.5852593369821, 74.66749851118736, 74.63063267447467, 74.69302101352693, 74.76675268695233, 74.80645435725832, 74.81779769163145, 74.77526018773219, 74.72705101664634, 74.69018517993365, 74.66466267759408, 74.67317017837392, 74.6589910104075, 74.47182599325072], "calls": [0.0, 1.1995576099594476, 2.166576865269546, 3.23285029634461, 4.605393755494427, 5.9892805490173835, 8.11331991038766, 10.37915095142217, 12.94841618693815, 14.414542154666366, 15.526188923233985, 16.57544735274934, 17.55664577602586, 18.628590874287497, 19.55023679210504, 20.633525224739813, 21.70830615659473, 22.553384567393586, 23.324731304767035, 24.220854720245015, 25.063097297450586, 25.933698210589, 26.73056745030202, 27.833706718089786, 28.851770978079006, 29.91804440915407, 30.876556163684317, 31.81805291665485, 32.864475512576924, 33.85134560304001, 34.696424013838865, 35.555681592604145, 36.50285001276125, 37.30255508606755, 38.43688852338145, 39.350026940419134, 40.47018120976661, 41.32943878853189, 42.242577205569575, 43.297507302271505, 44.20497405212262, 45.200351643365565, 45.94901171199274, 46.88483679777671, 47.596631029691174, 48.33394776394521, 49.283952017695604, 50.160224598020584, 51.04500467912543, 51.844709752431726, 52.66993732807759, 53.46964240138389, 54.260839973910336, 55.139948387828596, 55.94816096191475, 56.70532853132178, 57.408615262456394, 58.35861951620679, 59.22922042934521, 60.17638884950232, 60.964750588435464, 61.82684400079403, 62.782519921730994, 63.68431500439554, 64.56058758472054, 65.44536766582537, 66.4804469273743, 67.353883674106, 68.12239457788617, 69.07239883163656, 69.87777557212944, 70.61509230638346, 71.55091739216742, 72.72778833338059, 73.57853841136603, 74.55690100104925, 75.36227774154213, 76.14213197969542, 77.09497206703911, 78.10736465984176, 78.95244307064061, 79.96483566344327, 80.89215324844739, 81.92156084280975, 82.84604259422056, 83.74783767688511, 84.68366276266909, 85.69889118906502, 86.69143294671468, 87.65278053483821, 88.62830729092816, 89.66905821966367, 90.72965998355217, 91.73921674276153, 92.94728185350084, 94.37937781810963, 95.56475626010266, 96.68774636304342, 97.79655729801775, 98.85999489549954, 100.0], "rescued": [0, 233, 395, 565, 773, 962, 1229, 1496, 1726, 1734, 1741, 1748, 1758, 1765, 1776, 1786, 1796, 1802, 1807, 1817, 1826, 1834, 1835, 1844, 1853, 1868, 1884, 1899, 1916, 1929, 1945, 1960, 1986, 2006, 2025, 2037, 2065, 2091, 2113, 2139, 2163, 2199, 2218, 2252, 2274, 2298, 2327, 2360, 2385, 2421, 2448, 2475, 2504, 2531, 2567, 2601, 2633, 2680, 2719, 2774, 2806, 2858, 2909, 2967, 3013, 3061, 3114, 3152, 3199, 3250, 3305, 3357, 3408, 3495, 3552, 3622, 3690, 3751, 3813, 3883, 3932, 4003, 4064, 4129, 4194, 4261, 4332, 4407, 4476, 4528, 4602, 4684, 4767, 4840, 4933, 5030, 5117, 5200, 5274, 5351, 5423], "harmed": [0, 24, 42, 63, 82, 98, 126, 144, 166, 168, 170, 173, 175, 181, 188, 192, 193, 198, 199, 201, 209, 209, 219, 226, 241, 248, 256, 262, 269, 275, 281, 288, 299, 307, 321, 326, 339, 350, 361, 368, 380, 393, 401, 418, 427, 443, 458, 474, 492, 504, 521, 541, 559, 580, 595, 626, 643, 673, 697, 726, 740, 754, 776, 796, 825, 851, 891, 920, 942, 976, 1002, 1028, 1065, 1116, 1152, 1196, 1234, 1263, 1302, 1353, 1392, 1441, 1486, 1531, 1577, 1614, 1658, 1715, 1755, 1820, 1872, 1928, 1997, 2066, 2174, 2288, 2388, 2480, 2551, 2633, 2771], "exact_score": [66.95119530385956, 67.4474661826844, 67.88985622323682, 68.28120125911012, 68.67538212857669, 69.05254799648355, 69.40702719564416, 69.72747639168534, 70.03941808694665, 70.37971811814081, 70.68882397980887, 70.93270566883135, 71.17375152426055, 71.37509570938377, 71.38360321016363, 71.40061821172333, 71.40912571250318, 71.42897654765619, 71.44599154921589, 71.44315571562261, 71.4630065507756, 71.4800215523353, 71.50554405467487, 71.50837988826815, 71.53390239060772, 71.536738224201, 71.55942489294728, 71.53390239060772, 71.536738224201, 71.53106655701443, 71.5537532257607, 71.57360406091371, 71.59629072965998, 71.62464906559282, 71.6473357343391, 71.67569407027196, 71.70688823979809, 71.76927657885035, 71.77494824603691, 71.79196324759663, 71.82599325071605, 71.86285908742875, 71.92241159288773, 71.95076992882058, 71.99614326631313, 72.05002410458555, 72.10674077645123, 72.15494994753708, 72.19748745143634, 72.24853245611548, 72.29390579360803, 72.31659246235431, 72.393159969373, 72.42435413889913, 72.45271247483197, 72.49241414513796, 72.54629498341038, 72.55763831778351, 72.63987749198877, 72.65122082636191, 72.73913166775372, 72.8185350083657, 72.92629668491053, 73.01987919348892, 73.1361483708136, 73.20420837705244, 73.23823838017185, 73.26092504891812, 73.3403283895301, 73.38853756061594, 73.49346340356749, 73.5813742449593, 73.60973258089216, 73.72033009103025, 73.79689759804894, 73.87062927147434, 73.97839094801917, 74.05779428863114, 74.12301846127669, 74.14854096361626, 74.21092930266852, 74.26197430734764, 74.3243626463999, 74.40660182060516, 74.47749766043728, 74.5512293338627, 74.6419760088478, 74.66466267759408, 74.67033434478064, 74.71854351586649, 74.75824518617247, 74.79227518929189, 74.81779769163145, 74.77526018773219, 74.7355585174262, 74.68734934634035, 74.66466267759408, 74.70152851430679, 74.69018517993365, 74.63630434166123, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 72.4293078608058, "threshold_area100": 72.53529194906841, "threshold_area25": 70.61282363950883}, "selected_budget": 97, "selected": {"val": {"score": 80.79274720640944, "calls": 96.99908637290041, "rescued": 1872, "harmed": 1052}, "test": {"score": 74.66466267759408, "calls": 96.68774636304342, "rescued": 5200, "harmed": 2480}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 6.749822616577148, "operator": "ge"}, {"budget": 0.02, "threshold": 4.975802898406982, "operator": "ge"}, {"budget": 0.03, "threshold": 3.9777207374572754, "operator": "ge"}, {"budget": 0.04, "threshold": 3.112983226776123, "operator": "ge"}, {"budget": 0.05, "threshold": 2.329322338104248, "operator": "ge"}, {"budget": 0.06, "threshold": 1.618781566619873, "operator": "ge"}, {"budget": 0.07, "threshold": 0.8046356439590454, "operator": "ge"}, {"budget": 0.08, "threshold": -0.1130174994468689, "operator": "ge"}, {"budget": 0.09, "threshold": -0.6344428062438965, "operator": "ge"}, {"budget": 0.1, "threshold": -0.9508568048477173, "operator": "ge"}, {"budget": 0.11, "threshold": -1.2510828971862793, "operator": "ge"}, {"budget": 0.12, "threshold": -1.448169231414795, "operator": "ge"}, {"budget": 0.13, "threshold": -1.6354414224624634, "operator": "ge"}, {"budget": 0.14, "threshold": -1.8172059059143066, "operator": "ge"}, {"budget": 0.15, "threshold": -1.963573694229126, "operator": "ge"}, {"budget": 0.16, "threshold": -2.119497299194336, "operator": "ge"}, {"budget": 0.17, "threshold": -2.2676682472229004, "operator": "ge"}, {"budget": 0.18, "threshold": -2.4538228511810303, "operator": "ge"}, {"budget": 0.19, "threshold": -2.5974009037017822, "operator": "ge"}, {"budget": 0.2, "threshold": -2.740063190460205, "operator": "ge"}, {"budget": 0.21, "threshold": -2.8747305870056152, "operator": "ge"}, {"budget": 0.22, "threshold": -3.0178470611572266, "operator": "ge"}, {"budget": 0.23, "threshold": -3.1289026737213135, "operator": "ge"}, {"budget": 0.24, "threshold": -3.2475626468658447, "operator": "ge"}, {"budget": 0.25, "threshold": -3.3850319385528564, "operator": "ge"}, {"budget": 0.26, "threshold": -3.512310743331909, "operator": "ge"}, {"budget": 0.27, "threshold": -3.6399147510528564, "operator": "ge"}, {"budget": 0.28, "threshold": -3.7515807151794434, "operator": "ge"}, {"budget": 0.29, "threshold": -3.8619353771209717, "operator": "ge"}, {"budget": 0.3, "threshold": -3.964756965637207, "operator": "ge"}, {"budget": 0.31, "threshold": -4.088045120239258, "operator": "ge"}, {"budget": 0.32, "threshold": -4.191977024078369, "operator": "ge"}, {"budget": 0.33, "threshold": -4.285384178161621, "operator": "ge"}, {"budget": 0.34, "threshold": -4.371772289276123, "operator": "ge"}, {"budget": 0.35, "threshold": -4.45338249206543, "operator": "ge"}, {"budget": 0.36, "threshold": -4.548285484313965, "operator": "ge"}, {"budget": 0.37, "threshold": -4.619338512420654, "operator": "ge"}, {"budget": 0.38, "threshold": -4.7038893699646, "operator": "ge"}, {"budget": 0.39, "threshold": -4.787435054779053, "operator": "ge"}, {"budget": 0.4, "threshold": -4.861482620239258, "operator": "ge"}, {"budget": 0.41, "threshold": -4.927631855010986, "operator": "ge"}, {"budget": 0.42, "threshold": -5.008474826812744, "operator": "ge"}, {"budget": 0.43, "threshold": -5.074597358703613, "operator": "ge"}, {"budget": 0.44, "threshold": -5.1422505378723145, "operator": "gt"}, {"budget": 0.45, "threshold": -5.210206508636475, "operator": "gt"}, {"budget": 0.46, "threshold": -5.276137351989746, "operator": "ge"}, {"budget": 0.47, "threshold": -5.356076240539551, "operator": "ge"}, {"budget": 0.48, "threshold": -5.419894218444824, "operator": "ge"}, {"budget": 0.49, "threshold": -5.487074375152588, "operator": "ge"}, {"budget": 0.5, "threshold": -5.547891616821289, "operator": "ge"}, {"budget": 0.51, "threshold": -5.614802360534668, "operator": "ge"}, {"budget": 0.52, "threshold": -5.6740803718566895, "operator": "ge"}, {"budget": 0.53, "threshold": -5.736490249633789, "operator": "ge"}, {"budget": 0.54, "threshold": -5.798757553100586, "operator": "ge"}, {"budget": 0.55, "threshold": -5.868160247802734, "operator": "ge"}, {"budget": 0.56, "threshold": -5.926918983459473, "operator": "gt"}, {"budget": 0.57, "threshold": -5.9886932373046875, "operator": "ge"}, {"budget": 0.58, "threshold": -6.047069549560547, "operator": "ge"}, {"budget": 0.59, "threshold": -6.114784240722656, "operator": "ge"}, {"budget": 0.6, "threshold": -6.169126510620117, "operator": "ge"}, {"budget": 0.61, "threshold": -6.223626613616943, "operator": "gt"}, {"budget": 0.62, "threshold": -6.285763740539551, "operator": "ge"}, {"budget": 0.63, "threshold": -6.348562717437744, "operator": "ge"}, {"budget": 0.64, "threshold": -6.408201217651367, "operator": "ge"}, {"budget": 0.65, "threshold": -6.4623284339904785, "operator": "ge"}, {"budget": 0.66, "threshold": -6.525418758392334, "operator": "ge"}, {"budget": 0.67, "threshold": -6.581043720245361, "operator": "ge"}, {"budget": 0.68, "threshold": -6.642117023468018, "operator": "ge"}, {"budget": 0.69, "threshold": -6.7073493003845215, "operator": "ge"}, {"budget": 0.7, "threshold": -6.76893424987793, "operator": "ge"}, {"budget": 0.71, "threshold": -6.829129219055176, "operator": "ge"}, {"budget": 0.72, "threshold": -6.889241695404053, "operator": "ge"}, {"budget": 0.73, "threshold": -6.9524149894714355, "operator": "ge"}, {"budget": 0.74, "threshold": -7.0080766677856445, "operator": "ge"}, {"budget": 0.75, "threshold": -7.068061828613281, "operator": "ge"}, {"budget": 0.76, "threshold": -7.12949275970459, "operator": "ge"}, {"budget": 0.77, "threshold": -7.195889472961426, "operator": "ge"}, {"budget": 0.78, "threshold": -7.25340461730957, "operator": "ge"}, {"budget": 0.79, "threshold": -7.316459655761719, "operator": "ge"}, {"budget": 0.8, "threshold": -7.381444931030273, "operator": "ge"}, {"budget": 0.81, "threshold": -7.44892692565918, "operator": "ge"}, {"budget": 0.82, "threshold": -7.515078544616699, "operator": "ge"}, {"budget": 0.83, "threshold": -7.586947917938232, "operator": "ge"}, {"budget": 0.84, "threshold": -7.660242080688477, "operator": "ge"}, {"budget": 0.85, "threshold": -7.732838153839111, "operator": "ge"}, {"budget": 0.86, "threshold": -7.817453384399414, "operator": "ge"}, {"budget": 0.87, "threshold": -7.879543304443359, "operator": "ge"}, {"budget": 0.88, "threshold": -7.961699962615967, "operator": "ge"}, {"budget": 0.89, "threshold": -8.040040969848633, "operator": "ge"}, {"budget": 0.9, "threshold": -8.142465591430664, "operator": "ge"}, {"budget": 0.91, "threshold": -8.243783950805664, "operator": "ge"}, {"budget": 0.92, "threshold": -8.337139129638672, "operator": "ge"}, {"budget": 0.93, "threshold": -8.470602035522461, "operator": "ge"}, {"budget": 0.94, "threshold": -8.619091033935547, "operator": "ge"}, {"budget": 0.95, "threshold": -8.764074325561523, "operator": "ge"}, {"budget": 0.96, "threshold": -8.948814392089844, "operator": "ge"}, {"budget": 0.97, "threshold": -9.160879135131836, "operator": "ge"}, {"budget": 0.98, "threshold": -9.471949577331543, "operator": "ge"}, {"budget": 0.99, "threshold": -9.996987342834473, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.27584510506712, 75.38126361655773, 75.48668212804836, 75.60615644107105, 75.64129594490126, 75.81699346405229, 75.90835617401083, 76.03485838779956, 76.23866751001475, 76.59006254831682, 76.97659709044909, 77.31393632721905, 77.76372197624569, 78.12917281607984, 78.53679106051023, 78.86007449574812, 79.23255323634831, 79.49258556469184, 79.77370159533348, 80.10401293133741, 80.3078220535526, 80.56082648113009, 80.86302621406986, 81.03872373322089, 81.228477053904, 81.45336987841732, 81.5868999929721, 81.73448590905896, 81.80476491671938, 81.92423922974208, 82.12804835195728, 82.2194110619158, 82.33185747417247, 82.35996907723663, 82.45133178719517, 82.41619228336496, 82.52161079485558, 82.58486190174995, 82.64811300864432, 82.73947571860285, 82.7816431231991, 82.75353152013494, 82.76758732166702, 82.7465036193689, 82.80272682549723, 82.73244781783681, 82.71839201630473, 82.76758732166702, 82.7816431231991, 82.77461522243306, 82.8519221308595, 82.80975472626326, 82.84489423009347, 82.80272682549723, 82.70433621477264, 82.69028041324057, 82.67622461170848, 82.591889802516, 82.53566659638766, 82.5075549933235, 82.45133178719517, 82.38808068030079, 82.33185747417247, 82.3037458711083, 82.28969006957621, 82.28266216881018, 82.28969006957621, 82.23346686344789, 82.19129945885165, 82.09993674889311, 82.07182514582894, 82.07182514582894, 81.97343453510436, 81.93829503127415, 81.89612762667792, 81.81882071825146, 81.76259751212312, 81.74151380982501, 81.71340220676085, 81.69934640522875, 81.66420690139856, 81.61501159603627, 81.5517604891419, 81.50959308454566, 81.45336987841732, 81.37606296999087, 81.30578396233045, 81.24956075620211, 81.24253285543608, 81.19333755007379, 81.20036545083984, 81.16522594700963, 81.1090027408813, 81.03872373322089, 80.96141682479443, 80.94033312249631, 80.84897041253778, 80.76463560334528, 80.69435659568487, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.98763089465177, 44.99262070419566, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99128540305011, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.98109494693935, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 49, 82, 111, 144, 175, 211, 245, 280, 323, 383, 446, 501, 573, 636, 702, 759, 819, 867, 920, 976, 1020, 1066, 1114, 1155, 1189, 1229, 1257, 1289, 1318, 1348, 1385, 1407, 1436, 1451, 1476, 1494, 1520, 1543, 1569, 1590, 1601, 1614, 1629, 1640, 1655, 1663, 1676, 1692, 1707, 1719, 1738, 1747, 1759, 1765, 1773, 1782, 1791, 1799, 1806, 1813, 1823, 1827, 1830, 1840, 1847, 1855, 1861, 1866, 1871, 1876, 1879, 1888, 1891, 1895, 1897, 1900, 1905, 1909, 1913, 1917, 1923, 1925, 1927, 1930, 1932, 1934, 1936, 1938, 1943, 1944, 1947, 1948, 1950, 1950, 1951, 1954, 1956, 1958, 1961, 1964], "harmed": [0, 14, 32, 46, 62, 88, 99, 120, 137, 151, 161, 169, 176, 184, 195, 203, 214, 221, 232, 245, 254, 269, 279, 284, 300, 307, 315, 324, 335, 354, 367, 375, 384, 397, 408, 420, 443, 454, 468, 485, 493, 498, 515, 528, 542, 549, 567, 582, 591, 604, 617, 625, 640, 647, 659, 681, 692, 703, 723, 738, 749, 767, 780, 791, 805, 814, 823, 828, 841, 852, 870, 877, 886, 903, 912, 920, 934, 947, 954, 962, 968, 979, 988, 999, 1008, 1018, 1031, 1043, 1053, 1059, 1067, 1069, 1075, 1085, 1095, 1107, 1113, 1128, 1142, 1155, 1171], "exact_score": [75.02986857825567, 75.27584510506712, 75.38126361655773, 75.48668212804836, 75.60615644107105, 75.64129594490126, 75.81699346405229, 75.90835617401083, 76.03485838779956, 76.23866751001475, 76.59006254831682, 76.97659709044909, 77.31393632721905, 77.76372197624569, 78.12917281607984, 78.53679106051023, 78.86007449574812, 79.23255323634831, 79.49258556469184, 79.77370159533348, 80.10401293133741, 80.3078220535526, 80.56082648113009, 80.86302621406986, 81.03872373322089, 81.228477053904, 81.45336987841732, 81.5868999929721, 81.73448590905896, 81.80476491671938, 81.92423922974208, 82.12804835195728, 82.2194110619158, 82.33185747417247, 82.35996907723663, 82.45133178719517, 82.41619228336496, 82.52161079485558, 82.58486190174995, 82.64811300864432, 82.73947571860285, 82.7816431231991, 82.75353152013494, 82.76758732166702, 82.7465036193689, 82.80272682549723, 82.73244781783681, 82.71839201630473, 82.76758732166702, 82.7816431231991, 82.77461522243306, 82.8519221308595, 82.80975472626326, 82.84489423009347, 82.80272682549723, 82.70433621477264, 82.69028041324057, 82.67622461170848, 82.591889802516, 82.53566659638766, 82.5075549933235, 82.44430388642913, 82.38808068030079, 82.33185747417247, 82.3037458711083, 82.28969006957621, 82.28266216881018, 82.28969006957621, 82.23346686344789, 82.19129945885165, 82.09993674889311, 82.07182514582894, 82.07182514582894, 81.97343453510436, 81.93829503127415, 81.89612762667792, 81.81882071825146, 81.76259751212312, 81.74151380982501, 81.71340220676085, 81.69934640522875, 81.66420690139856, 81.61501159603627, 81.5517604891419, 81.50959308454566, 81.45336987841732, 81.37606296999087, 81.30578396233045, 81.24956075620211, 81.24253285543608, 81.19333755007379, 81.20036545083984, 81.16522594700963, 81.1090027408813, 81.03872373322089, 80.96141682479443, 80.94033312249631, 80.84897041253778, 80.76463560334528, 80.69435659568487, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 80.9797133710013, "threshold_area100": 80.97958394827465, "threshold_area25": 77.80026706022912}, "test": {"score": [66.95119530385956, 68.02881206930778, 68.79165130590137, 69.31060885347247, 69.7529988940249, 70.20673226895046, 70.57255480248419, 71.00927317585004, 71.4261407140629, 71.78629158041005, 72.05285993817883, 72.44704080764541, 72.70793749822761, 73.02271502708221, 73.31480588719053, 73.58704591214588, 73.80256926523552, 73.9897342823923, 74.22794430422822, 74.43212432294473, 74.65615517681422, 74.83481269319117, 75.04750021268751, 75.15526188923234, 75.311232736863, 75.49839775401979, 75.61466693134446, 75.77347361256841, 75.8698919547401, 75.96063862972521, 76.03437030315061, 76.14213197969542, 76.15914698125515, 76.18750531718798, 76.19034115078126, 76.21869948671412, 76.23571448827381, 76.26123699061338, 76.23571448827381, 76.23571448827381, 76.25272948983354, 76.23855032186711, 76.27541615857982, 76.25556532342682, 76.24422198905368, 76.24989365624025, 76.25272948983354, 76.25556532342682, 76.25272948983354, 76.27541615857982, 76.24989365624025, 76.25556532342682, 76.26690865779996, 76.24422198905368, 76.2045203187477, 76.17332614922157, 76.15631114766185, 76.13646031250886, 76.10810197657601, 76.1109378101693, 76.08257947423644, 76.06556447267674, 76.03153446955733, 76.01168363440433, 76.00601196721776, 76.00034030003118, 75.98332529847148, 75.97198196409835, 75.93511612738564, 75.92093695941922, 75.87839945551995, 75.81601111646769, 75.78765278053483, 75.73093610866914, 75.69123443836315, 75.65153276805717, 75.62884609931089, 75.58630859541162, 75.5324277571392, 75.50123358761309, 75.47571108527352, 75.45018858293395, 75.43317358137425, 75.39630774466154, 75.3849644102884, 75.35093440716899, 75.311232736863, 75.2743669001503, 75.21481439469132, 75.17794855797861, 75.14108272126592, 75.11839605251963, 75.08436604940022, 75.02481354394124, 74.95391770410912, 74.93406686895612, 74.87167852990386, 74.85182769475087, 74.80645435725832, 74.6589910104075, 74.47182599325072], "calls": [0.0, 3.485239486146953, 5.115843802285682, 6.30972974505856, 7.361824008167201, 8.584068286872926, 9.687207554660693, 11.065422680997079, 12.920057851005303, 14.386183818733517, 15.633950599778803, 17.117091569066726, 18.268440007940335, 19.547400958511755, 20.817854408303322, 21.920993676091086, 23.180103791509516, 24.382497235062246, 25.845787369197176, 27.170121657261152, 28.554008450784107, 29.779088563083118, 31.111930351926947, 32.25193545642742, 33.54223974137198, 35.07358988174574, 36.67016419476505, 38.249723506224655, 39.5995802966282, 40.946601253438445, 42.327652213368125, 43.98377903184641, 45.41020332926864, 46.660805943907214, 47.72424354138899, 48.86141281229617, 50.06664208944219, 51.13007968692397, 52.28709979298415, 53.52068740606302, 54.55293083401867, 55.55965175963474, 56.798911039900176, 57.68369112100502, 58.64220287553527, 59.57519212772594, 60.522360547883046, 61.60281314692454, 62.49610072880923, 63.502821654425325, 64.24297422227264, 65.14760513853048, 65.9416385446502, 66.74985111873634, 67.61761619828148, 68.57612795281173, 69.37583302611803, 70.1415080963049, 70.8873323313388, 71.7409182429175, 72.46121997561184, 73.12480503644046, 73.79122593086238, 74.55406516745597, 75.24884439781073, 75.89825029067295, 76.67810452882625, 77.30765958653546, 78.10736465984176, 78.78512888863682, 79.49975895414457, 80.16901568215977, 80.7220032328503, 81.34872245696623, 81.81947083345149, 82.42066755522785, 82.94813260357881, 83.52664265660891, 84.00873436746733, 84.493661911919, 85.01829112667669, 85.53724867424779, 85.98814621558007, 86.56665626861016, 87.15934548960666, 87.67830303717777, 88.28233559254744, 88.73323313387971, 89.3089073533165, 89.76264072824208, 90.37518078439157, 91.0217508436605, 91.56339505997788, 92.30071179423192, 93.10041686753821, 93.76400192836685, 94.59206533760599, 95.53639792416982, 96.75864220287554, 98.11984232765222, 100.0], "rescued": [0, 388, 677, 883, 1056, 1238, 1387, 1563, 1746, 1907, 2032, 2199, 2322, 2460, 2589, 2701, 2812, 2910, 3026, 3128, 3233, 3320, 3427, 3495, 3575, 3675, 3747, 3842, 3905, 3968, 4032, 4107, 4149, 4197, 4226, 4261, 4299, 4331, 4361, 4395, 4425, 4451, 4491, 4514, 4543, 4567, 4589, 4614, 4639, 4673, 4689, 4715, 4739, 4760, 4786, 4815, 4833, 4858, 4879, 4905, 4925, 4945, 4961, 4989, 5009, 5023, 5050, 5066, 5078, 5093, 5113, 5124, 5137, 5149, 5154, 5163, 5173, 5183, 5192, 5198, 5211, 5219, 5228, 5236, 5249, 5257, 5265, 5272, 5284, 5289, 5296, 5305, 5316, 5333, 5338, 5351, 5357, 5371, 5392, 5405, 5423], "harmed": [0, 8, 28, 51, 68, 90, 110, 132, 168, 202, 233, 261, 292, 319, 345, 361, 396, 428, 460, 490, 516, 540, 572, 602, 627, 661, 692, 731, 760, 791, 829, 866, 902, 940, 968, 993, 1025, 1048, 1087, 1121, 1145, 1176, 1203, 1233, 1266, 1288, 1309, 1333, 1359, 1385, 1410, 1434, 1454, 1483, 1523, 1563, 1587, 1619, 1650, 1675, 1705, 1731, 1759, 1794, 1816, 1832, 1865, 1885, 1910, 1930, 1965, 1998, 2021, 2053, 2072, 2095, 2113, 2138, 2166, 2183, 2205, 2222, 2237, 2258, 2275, 2295, 2317, 2337, 2370, 2388, 2408, 2425, 2448, 2486, 2516, 2536, 2564, 2585, 2622, 2687, 2771], "exact_score": [66.95119530385956, 67.20074866006863, 67.46448118424411, 67.83313955137112, 68.2840370927034, 68.73493463403567, 69.1574738394351, 69.6026997135808, 69.99971641664067, 70.33150894705499, 70.67748064543572, 70.98658650710377, 71.2219606953464, 71.44599154921589, 71.71539574057793, 71.93375492726086, 72.13509911238408, 72.413010804526, 72.64554915917533, 72.8752516802314, 73.10211836769417, 73.36585089186967, 73.59271757933244, 73.78271843008251, 73.92451010974675, 74.10033179253041, 74.25346680656779, 74.40376598701188, 74.5852593369821, 74.72705101664634, 74.87167852990386, 75.03899271190765, 75.13257522048606, 75.22048606187789, 75.34242690638912, 75.49839775401979, 75.5522785922922, 75.63735360009073, 75.75362277741542, 75.84436945240053, 75.90392195785951, 75.96347446331849, 75.99466863284462, 76.05705697189688, 76.14213197969542, 76.15914698125515, 76.17049031562829, 76.2016844851544, 76.1846694835947, 76.23571448827381, 76.23571448827381, 76.2584011570201, 76.25272948983354, 76.23004282108725, 76.24422198905368, 76.26123699061338, 76.24705782264697, 76.2782519921731, 76.26407282420668, 76.24989365624025, 76.26123699061338, 76.26123699061338, 76.2584011570201, 76.26974449139324, 76.24422198905368, 76.25272948983354, 76.2584011570201, 76.21586365312083, 76.17332614922157, 76.17616198281485, 76.16765448203499, 76.10810197657601, 76.09675864220287, 76.0740719734566, 76.03437030315061, 75.9975044664379, 75.9975044664379, 75.97198196409835, 75.92660862660578, 75.91526529223265, 75.83869778521397, 75.76780194538185, 75.67138360321016, 75.62884609931089, 75.5352635907325, 75.47571108527352, 75.43033774778097, 75.39347191106826, 75.33959107279584, 75.24317273062417, 75.16376939001219, 75.11556021892635, 75.04466437909423, 74.9567535377024, 74.90003686583671, 74.86033519553072, 74.82914102600459, 74.79227518929189, 74.6816776791538, 74.60511017213508, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.51580171987587, "threshold_area100": 74.94991917874259, "threshold_area25": 72.34551796500583}, "selected_budget": 51, "selected": {"val": {"score": 82.8519221308595, "calls": 50.99444795839483, "rescued": 1738, "harmed": 625}, "test": {"score": 76.25556532342682, "calls": 65.14760513853048, "rescued": 4715, "harmed": 1434}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.1874582767486572, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8634681701660156, "operator": "ge"}, {"budget": 0.03, "threshold": 0.645232081413269, "operator": "gt"}, {"budget": 0.04, "threshold": 0.4750404357910156, "operator": "ge"}, {"budget": 0.05, "threshold": 0.31680968403816223, "operator": "ge"}, {"budget": 0.06, "threshold": 0.16050678491592407, "operator": "ge"}, {"budget": 0.07, "threshold": 0.05679607391357422, "operator": "ge"}, {"budget": 0.08, "threshold": -0.013686656951904297, "operator": "ge"}, {"budget": 0.09, "threshold": -0.22056269645690918, "operator": "ge"}, {"budget": 0.1, "threshold": -0.35539326071739197, "operator": "ge"}, {"budget": 0.11, "threshold": -0.4719916582107544, "operator": "ge"}, {"budget": 0.12, "threshold": -0.572760283946991, "operator": "ge"}, {"budget": 0.13, "threshold": -0.6919536590576172, "operator": "ge"}, {"budget": 0.14, "threshold": -0.794166088104248, "operator": "ge"}, {"budget": 0.15, "threshold": -0.9019392132759094, "operator": "ge"}, {"budget": 0.16, "threshold": -1.0248985290527344, "operator": "ge"}, {"budget": 0.17, "threshold": -1.1450684070587158, "operator": "ge"}, {"budget": 0.18, "threshold": -1.254410743713379, "operator": "ge"}, {"budget": 0.19, "threshold": -1.3747671842575073, "operator": "ge"}, {"budget": 0.2, "threshold": -1.511409878730774, "operator": "ge"}, {"budget": 0.21, "threshold": -1.6223112344741821, "operator": "ge"}, {"budget": 0.22, "threshold": -1.7549769878387451, "operator": "ge"}, {"budget": 0.23, "threshold": -1.8926117420196533, "operator": "ge"}, {"budget": 0.24, "threshold": -2.016390085220337, "operator": "ge"}, {"budget": 0.25, "threshold": -2.152353286743164, "operator": "gt"}, {"budget": 0.26, "threshold": -2.278200149536133, "operator": "ge"}, {"budget": 0.27, "threshold": -2.3889336585998535, "operator": "ge"}, {"budget": 0.28, "threshold": -2.511303424835205, "operator": "ge"}, {"budget": 0.29, "threshold": -2.6090917587280273, "operator": "ge"}, {"budget": 0.3, "threshold": -2.7340757846832275, "operator": "ge"}, {"budget": 0.31, "threshold": -2.8660755157470703, "operator": "ge"}, {"budget": 0.32, "threshold": -2.9693808555603027, "operator": "ge"}, {"budget": 0.33, "threshold": -3.0592238903045654, "operator": "ge"}, {"budget": 0.34, "threshold": -3.154991626739502, "operator": "ge"}, {"budget": 0.35, "threshold": -3.244964122772217, "operator": "ge"}, {"budget": 0.36, "threshold": -3.3343923091888428, "operator": "gt"}, {"budget": 0.37, "threshold": -3.4335360527038574, "operator": "ge"}, {"budget": 0.38, "threshold": -3.522890567779541, "operator": "ge"}, {"budget": 0.39, "threshold": -3.6118826866149902, "operator": "ge"}, {"budget": 0.4, "threshold": -3.691640615463257, "operator": "ge"}, {"budget": 0.41, "threshold": -3.7687840461730957, "operator": "ge"}, {"budget": 0.42, "threshold": -3.849903106689453, "operator": "gt"}, {"budget": 0.43, "threshold": -3.9310503005981445, "operator": "ge"}, {"budget": 0.44, "threshold": -4.007272720336914, "operator": "ge"}, {"budget": 0.45, "threshold": -4.085500240325928, "operator": "ge"}, {"budget": 0.46, "threshold": -4.160145282745361, "operator": "ge"}, {"budget": 0.47, "threshold": -4.231039047241211, "operator": "ge"}, {"budget": 0.48, "threshold": -4.316828727722168, "operator": "ge"}, {"budget": 0.49, "threshold": -4.395437240600586, "operator": "ge"}, {"budget": 0.5, "threshold": -4.466353893280029, "operator": "ge"}, {"budget": 0.51, "threshold": -4.536172866821289, "operator": "ge"}, {"budget": 0.52, "threshold": -4.617250919342041, "operator": "ge"}, {"budget": 0.53, "threshold": -4.6797308921813965, "operator": "ge"}, {"budget": 0.54, "threshold": -4.748057842254639, "operator": "ge"}, {"budget": 0.55, "threshold": -4.811383247375488, "operator": "ge"}, {"budget": 0.56, "threshold": -4.890744209289551, "operator": "ge"}, {"budget": 0.57, "threshold": -4.964700222015381, "operator": "ge"}, {"budget": 0.58, "threshold": -5.022488117218018, "operator": "ge"}, {"budget": 0.59, "threshold": -5.097198963165283, "operator": "ge"}, {"budget": 0.6, "threshold": -5.154156684875488, "operator": "ge"}, {"budget": 0.61, "threshold": -5.2230963706970215, "operator": "ge"}, {"budget": 0.62, "threshold": -5.300177097320557, "operator": "ge"}, {"budget": 0.63, "threshold": -5.362676620483398, "operator": "ge"}, {"budget": 0.64, "threshold": -5.430358409881592, "operator": "ge"}, {"budget": 0.65, "threshold": -5.495090007781982, "operator": "ge"}, {"budget": 0.66, "threshold": -5.563767433166504, "operator": "ge"}, {"budget": 0.67, "threshold": -5.624509811401367, "operator": "ge"}, {"budget": 0.68, "threshold": -5.683340549468994, "operator": "ge"}, {"budget": 0.69, "threshold": -5.746768474578857, "operator": "ge"}, {"budget": 0.7, "threshold": -5.808923721313477, "operator": "ge"}, {"budget": 0.71, "threshold": -5.866608619689941, "operator": "ge"}, {"budget": 0.72, "threshold": -5.9313883781433105, "operator": "ge"}, {"budget": 0.73, "threshold": -6.00441312789917, "operator": "ge"}, {"budget": 0.74, "threshold": -6.0627121925354, "operator": "ge"}, {"budget": 0.75, "threshold": -6.1369242668151855, "operator": "ge"}, {"budget": 0.76, "threshold": -6.214853763580322, "operator": "ge"}, {"budget": 0.77, "threshold": -6.290029525756836, "operator": "ge"}, {"budget": 0.78, "threshold": -6.353241443634033, "operator": "ge"}, {"budget": 0.79, "threshold": -6.426292419433594, "operator": "ge"}, {"budget": 0.8, "threshold": -6.49661111831665, "operator": "ge"}, {"budget": 0.81, "threshold": -6.572136878967285, "operator": "ge"}, {"budget": 0.82, "threshold": -6.631259918212891, "operator": "ge"}, {"budget": 0.83, "threshold": -6.693775653839111, "operator": "ge"}, {"budget": 0.84, "threshold": -6.767479419708252, "operator": "ge"}, {"budget": 0.85, "threshold": -6.85263729095459, "operator": "ge"}, {"budget": 0.86, "threshold": -6.931582450866699, "operator": "ge"}, {"budget": 0.87, "threshold": -7.015088081359863, "operator": "ge"}, {"budget": 0.88, "threshold": -7.115461826324463, "operator": "ge"}, {"budget": 0.89, "threshold": -7.195625305175781, "operator": "ge"}, {"budget": 0.9, "threshold": -7.286130905151367, "operator": "ge"}, {"budget": 0.91, "threshold": -7.390300750732422, "operator": "ge"}, {"budget": 0.92, "threshold": -7.492946624755859, "operator": "ge"}, {"budget": 0.93, "threshold": -7.602433681488037, "operator": "ge"}, {"budget": 0.94, "threshold": -7.7398505210876465, "operator": "ge"}, {"budget": 0.95, "threshold": -7.8948445320129395, "operator": "ge"}, {"budget": 0.96, "threshold": -8.082283020019531, "operator": "ge"}, {"budget": 0.97, "threshold": -8.299729347229004, "operator": "ge"}, {"budget": 0.98, "threshold": -8.600214958190918, "operator": "ge"}, {"budget": 0.99, "threshold": -9.068506240844727, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.21259399817275, 75.254761402769, 75.26178930353502, 75.27584510506712, 75.4445147234521, 75.54290533417668, 75.77482605945605, 76.04891418933165, 76.280834914611, 76.61817415138098, 76.94145758661888, 77.36313163258134, 77.81291728160798, 78.1221449153138, 78.42434464825357, 78.79682338885375, 79.19038583175205, 79.54880877082016, 79.85100850375993, 80.25159884742428, 80.546770679598, 80.76463560334528, 80.87708201560194, 81.17928174854171, 81.37606296999087, 81.53067678684377, 81.75556961135709, 81.91721132897604, 82.04371354276478, 82.19832735961768, 82.32482957340642, 82.40213648183288, 82.48647129102538, 82.70433621477264, 82.83786632932744, 82.88003373392368, 82.95031274158409, 83.04870335230868, 83.04870335230868, 83.18223346686345, 83.28765197835408, 83.3438751844824, 83.37901468831261, 83.42118209290885, 83.3438751844824, 83.32279148218427, 83.35090308524843, 83.3298193829503, 83.30170777988614, 83.21034506992761, 83.20331716916158, 83.18223346686345, 83.14006606226721, 83.13303816150116, 83.13303816150116, 83.13303816150116, 83.05573125307471, 82.9854522454143, 82.90814533698784, 82.91517323775388, 82.83083842856139, 82.7465036193689, 82.66919671094244, 82.65514090941036, 82.5426944971537, 82.50052709255745, 82.35294117647058, 82.28266216881018, 82.2194110619158, 82.17724365731955, 82.07885304659499, 82.05074144353081, 82.00857403893457, 81.9453229320402, 81.86801602361375, 81.81179281748541, 81.75556961135709, 81.63609529833438, 81.57987209220606, 81.53067678684377, 81.48850938224751, 81.43228617611919, 81.39011877152295, 81.3479513669267, 81.31983976386253, 81.29172816079837, 81.28470026003232, 81.22144915313795, 81.17225384777566, 81.11603064164734, 81.02466793168881, 80.94736102326236, 80.93330522173027, 80.88410991636799, 80.85599831330381, 80.80680300794153, 80.75760770257924, 80.74355190104716, 80.68732869491882, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.986857825567503, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.991215124042448, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.9898798228969, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.991707077096066, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 42, 69, 92, 116, 154, 188, 233, 280, 322, 377, 432, 497, 565, 621, 677, 738, 803, 867, 919, 985, 1037, 1082, 1119, 1175, 1217, 1253, 1295, 1336, 1370, 1399, 1427, 1453, 1475, 1515, 1540, 1559, 1579, 1604, 1617, 1645, 1670, 1690, 1710, 1726, 1736, 1751, 1760, 1774, 1786, 1793, 1805, 1817, 1823, 1833, 1843, 1848, 1852, 1858, 1866, 1872, 1876, 1880, 1885, 1892, 1893, 1900, 1901, 1902, 1906, 1909, 1911, 1914, 1918, 1923, 1926, 1928, 1928, 1931, 1934, 1937, 1938, 1939, 1941, 1945, 1948, 1950, 1951, 1951, 1953, 1954, 1955, 1955, 1956, 1956, 1958, 1958, 1960, 1963, 1964, 1964], "harmed": [0, 16, 37, 59, 81, 95, 115, 127, 135, 144, 151, 160, 165, 169, 181, 194, 202, 211, 224, 233, 242, 252, 266, 287, 300, 314, 328, 338, 356, 372, 379, 389, 404, 414, 423, 429, 442, 452, 463, 476, 485, 495, 507, 522, 532, 553, 571, 576, 593, 609, 629, 642, 657, 669, 680, 690, 695, 710, 726, 745, 750, 766, 782, 798, 807, 824, 837, 859, 870, 883, 892, 908, 915, 925, 939, 953, 963, 971, 991, 1002, 1012, 1019, 1028, 1036, 1046, 1053, 1059, 1061, 1070, 1079, 1088, 1102, 1113, 1116, 1123, 1129, 1136, 1145, 1150, 1159, 1171], "exact_score": [75.02986857825567, 75.21259399817275, 75.254761402769, 75.26881720430107, 75.27584510506712, 75.4445147234521, 75.54290533417668, 75.77482605945605, 76.04891418933165, 76.280834914611, 76.61817415138098, 76.94145758661888, 77.36313163258134, 77.81291728160798, 78.1221449153138, 78.42434464825357, 78.79682338885375, 79.19038583175205, 79.54880877082016, 79.85100850375993, 80.25159884742428, 80.546770679598, 80.76463560334528, 80.87708201560194, 81.17928174854171, 81.37606296999087, 81.53067678684377, 81.75556961135709, 81.91721132897604, 82.04371354276478, 82.19832735961768, 82.32482957340642, 82.40213648183288, 82.48647129102538, 82.70433621477264, 82.83786632932744, 82.88003373392368, 82.95031274158409, 83.04870335230868, 83.04870335230868, 83.18223346686345, 83.28765197835408, 83.3438751844824, 83.37901468831261, 83.42118209290885, 83.3438751844824, 83.32279148218427, 83.35090308524843, 83.3298193829503, 83.30170777988614, 83.21034506992761, 83.20331716916158, 83.18223346686345, 83.14006606226721, 83.13303816150116, 83.13303816150116, 83.13303816150116, 83.05573125307471, 82.9854522454143, 82.90814533698784, 82.91517323775388, 82.83083842856139, 82.7465036193689, 82.66919671094244, 82.65514090941036, 82.5426944971537, 82.50052709255745, 82.35294117647058, 82.28266216881018, 82.2194110619158, 82.17724365731955, 82.07885304659499, 82.05074144353081, 82.00857403893457, 81.9453229320402, 81.86801602361375, 81.81179281748541, 81.75556961135709, 81.63609529833438, 81.57987209220606, 81.53067678684377, 81.48850938224751, 81.43228617611919, 81.39011877152295, 81.3479513669267, 81.31983976386253, 81.29172816079837, 81.28470026003232, 81.22144915313795, 81.17225384777566, 81.11603064164734, 81.02466793168881, 80.94736102326236, 80.93330522173027, 80.88410991636799, 80.85599831330381, 80.80680300794153, 80.75760770257924, 80.74355190104716, 80.68732869491882, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.10945244947978, "threshold_area100": 81.10917843840045, "threshold_area25": 77.78831962892684}, "test": {"score": [66.95119530385956, 67.52403368970309, 67.94090122791594, 68.34642543175566, 68.79448713949465, 69.2879221847262, 69.94583557836826, 70.77106315401413, 71.45166321640247, 71.99614326631313, 72.41868247171256, 72.74763916853358, 73.00286419192922, 73.3034625528174, 73.5246575730936, 73.81391259960866, 74.00107761676544, 74.1570484643961, 74.4179451549783, 74.59376683776196, 74.78376768851204, 74.90570853302327, 75.05884354706066, 75.20630689391147, 75.35377024076227, 75.4955619204265, 75.6657119360236, 75.76496611178855, 75.86422028755354, 75.93228029379236, 76.02586280237075, 76.11660947735587, 76.18750531718798, 76.20735615234098, 76.25272948983354, 76.30093866091937, 76.32078949607236, 76.34347616481864, 76.39735700309106, 76.39452116949778, 76.4087003374642, 76.42571533902391, 76.42571533902391, 76.40586450387092, 76.43138700621047, 76.46258117573662, 76.44556617417689, 76.43138700621047, 76.3916853359045, 76.38601366871791, 76.37750616793807, 76.35765533278507, 76.32646116325894, 76.30377449451267, 76.2952669937328, 76.26974449139324, 76.2215353203074, 76.19601281796784, 76.17616198281485, 76.14496781328872, 76.14496781328872, 76.127952811729, 76.127952811729, 76.07974364064316, 76.03153446955733, 76.01735530159091, 75.9975044664379, 75.96914613050507, 75.91810112582593, 75.87839945551995, 75.81884695006097, 75.77914527975497, 75.74795111022885, 75.68839860476987, 75.6657119360236, 75.64586110087059, 75.57496526103849, 75.54660692510564, 75.49839775401979, 75.46720358449366, 75.41048691262796, 75.37645690950855, 75.32541190482942, 75.31406857045629, 75.28287440093015, 75.23182939625103, 75.20063522672488, 75.18362022516519, 75.15809772282563, 75.14675438845248, 75.12123188611292, 75.08153021580694, 75.0701868814338, 75.02764937753452, 74.96526103848225, 74.92272353458299, 74.85182769475087, 74.83481269319117, 74.76958852054561, 74.66182684400079, 74.47182599325072], "calls": [0.0, 1.182542608399739, 2.1807560332359697, 3.2413577971244645, 4.472109576610044, 5.949578878711397, 8.045259904148825, 10.506763463119984, 12.90587868303888, 14.831409692879221, 16.218132319995462, 17.604854947111704, 18.736352550832315, 20.066358506082864, 21.22337861214304, 22.44278705725548, 23.795479681252303, 25.02906729433117, 26.19743073476448, 27.510421688455324, 29.016249326489525, 30.21580693644897, 31.707455406516743, 32.997759691461304, 34.20582480220061, 35.61239826446984, 37.07285256501148, 38.3347985140232, 39.7839094801917, 41.02316876045713, 42.571533902390605, 44.25601905680175, 45.560502509712734, 46.67782094546692, 47.919916059325644, 49.156339505997785, 50.39559878626322, 51.67739557042793, 52.908147349913506, 54.13322746221252, 55.18248589172787, 56.21756515327681, 57.451152766355676, 58.69324788021439, 59.7680288120693, 60.893854748603346, 61.91475484218586, 62.92147576780195, 64.03595836996286, 65.05118679635878, 65.99268354932933, 67.04194197884468, 68.12523041147945, 68.83986047698721, 69.80120806511074, 70.67180897824915, 71.67285823667868, 72.49241414513796, 73.13331253722032, 74.02660011910501, 74.6419760088478, 75.37078524232199, 76.19884865156112, 76.92482205144202, 77.68482545444233, 78.33706718089782, 78.97796557298018, 79.6415506338088, 80.28812069307773, 80.88364574766753, 81.47633496866405, 81.96977001389558, 82.61066840597793, 83.28843263477299, 83.77336017922468, 84.36321356662792, 85.01829112667669, 85.63933868360604, 86.13844539602417, 86.6885971131214, 87.21039049428579, 87.79173638090917, 88.21994725349516, 88.64532229248788, 89.11607066897315, 89.74846156027564, 90.25323993988033, 90.73533165073874, 91.32802087173525, 91.77891841306753, 92.26101012392593, 92.82817684258288, 93.3442985565607, 93.88027110569152, 94.50699032980745, 95.23012789609507, 95.97878796472223, 96.65655219351729, 97.55551144258855, 98.52253069789865, 100.0], "rescued": [0, 228, 390, 556, 729, 924, 1186, 1496, 1753, 1981, 2156, 2310, 2427, 2568, 2680, 2810, 2924, 3015, 3135, 3233, 3343, 3417, 3514, 3609, 3689, 3773, 3865, 3943, 4014, 4077, 4137, 4208, 4262, 4293, 4334, 4375, 4418, 4466, 4513, 4544, 4571, 4608, 4636, 4665, 4701, 4736, 4770, 4795, 4820, 4847, 4868, 4890, 4913, 4923, 4950, 4970, 4992, 5015, 5031, 5048, 5064, 5082, 5106, 5118, 5130, 5149, 5161, 5174, 5190, 5203, 5212, 5219, 5225, 5236, 5241, 5253, 5260, 5274, 5277, 5286, 5290, 5298, 5305, 5314, 5324, 5331, 5335, 5339, 5349, 5355, 5362, 5369, 5379, 5382, 5389, 5395, 5397, 5406, 5414, 5416, 5423], "harmed": [0, 26, 41, 64, 79, 100, 130, 149, 166, 202, 228, 266, 293, 328, 362, 390, 438, 474, 502, 538, 581, 612, 655, 698, 726, 760, 792, 835, 871, 910, 937, 976, 1005, 1029, 1054, 1078, 1114, 1154, 1182, 1214, 1236, 1267, 1295, 1331, 1358, 1382, 1422, 1452, 1491, 1520, 1544, 1573, 1607, 1625, 1655, 1684, 1723, 1755, 1778, 1806, 1822, 1846, 1870, 1899, 1928, 1952, 1971, 1994, 2028, 2055, 2085, 2106, 2123, 2155, 2168, 2187, 2219, 2243, 2263, 2283, 2307, 2327, 2352, 2365, 2386, 2411, 2426, 2436, 2455, 2465, 2481, 2502, 2516, 2534, 2563, 2584, 2611, 2626, 2657, 2697, 2771], "exact_score": [66.95119530385956, 67.43895868190455, 67.88134872245696, 68.25000708958399, 68.618665456711, 68.98448799024473, 69.31060885347247, 69.62538638232708, 69.94299974477497, 70.27479227518928, 70.62359980716332, 70.89016816493208, 71.15957235629413, 71.46584238436888, 71.74942574369736, 72.05285993817883, 72.33077163032074, 72.62002665683578, 72.83838584351871, 73.0822675325412, 73.29211921844426, 73.4509258996682, 73.73450925899668, 73.8791367722542, 74.02376428551173, 74.15137679720954, 74.37257181748575, 74.51152766355669, 74.67884184556051, 74.78376768851204, 74.88018603068372, 74.95958937129569, 75.12973938689278, 75.20630689391147, 75.32257607123614, 75.42750191418767, 75.54377109151234, 75.65436860165046, 75.72810027507586, 75.81033944928112, 75.90675779145279, 75.93228029379236, 75.97765363128491, 76.05705697189688, 76.09108697501631, 76.16481864844171, 76.1846694835947, 76.21869948671412, 76.25556532342682, 76.30093866091937, 76.32646116325894, 76.33213283044552, 76.34347616481864, 76.4087003374642, 76.40019283668434, 76.38601366871791, 76.40019283668434, 76.43422283980375, 76.40019283668434, 76.41153617105748, 76.43422283980375, 76.45407367495676, 76.43989450699033, 76.43422283980375, 76.39452116949778, 76.38034200153135, 76.37750616793807, 76.35481949919179, 76.33213283044552, 76.2952669937328, 76.29810282732609, 76.24422198905368, 76.19317698437456, 76.19317698437456, 76.14496781328872, 76.14496781328872, 76.13646031250886, 76.0712361398633, 76.01168363440433, 75.9975044664379, 75.9237727930125, 75.86422028755354, 75.77914527975497, 75.71108527351615, 75.6628761024303, 75.5721294274452, 75.51824858917279, 75.43317358137425, 75.35660607435555, 75.28854606811673, 75.21481439469132, 75.16376939001219, 75.14108272126592, 75.0872018829935, 75.0134702095681, 74.92555936817628, 74.85466352834416, 74.82063352522474, 74.7355585174262, 74.6022743385418, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.60336953179791, "threshold_area100": 74.94875648696934, "threshold_area25": 72.3055327113405}, "selected_budget": 44, "selected": {"val": {"score": 83.42118209290885, "calls": 43.994658795417806, "rescued": 1726, "harmed": 532}, "test": {"score": 76.43138700621047, "calls": 59.7680288120693, "rescued": 4701, "harmed": 1358}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 80.85599831330381, "calls": 97.99704828167826}, "test": {"score": 74.77809602132547, "calls": 97.96387148002155}}, "old_normalized_interior": {"val": {"score": 81.4814814814815, "calls": 42.99669688663996}, "test": {"score": 75.60332359697132, "calls": 50.883362164308195}}, "old_normalized_endpoints": {"val": {"score": 81.4814814814815, "calls": 42.99669688663996}, "test": {"score": 75.60332359697132, "calls": 50.883362164308195}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.4093752196219, 75.6131843418371, 75.84510506711646, 76.0208025862675, 76.27380701384496, 76.50572773912432, 76.54086724295453, 76.69548105980743, 76.89929018202263, 77.13121090730199, 77.37718743411344, 77.60208025862674, 77.95347529692881, 78.31892613676294, 78.53679106051023, 78.76168388502354, 79.0287441141331, 79.18335793098602, 79.38716705320121, 79.5909761754164, 79.78072949609951, 79.93534331295243, 80.13915243516762, 80.3218778550847, 80.4272963665753, 80.5819101834282, 80.6311054887905, 80.72949609951507, 80.93330522173027, 81.05277953475297, 81.05277953475297, 81.05980743551902, 81.11603064164734, 81.19333755007379, 81.19333755007379, 81.17928174854171, 81.21442125237192, 81.31281186309648, 81.31983976386253, 81.3830908707569, 81.35497926769274, 81.43228617611919, 81.4814814814815, 81.44634197765129, 81.42525827535316, 81.34092346616066, 81.39011877152295, 81.40417457305503, 81.39714667228898, 81.40417457305503, 81.37606296999087, 81.43931407688524, 81.45336987841732, 81.40417457305503, 81.36903506922482, 81.37606296999087, 81.37606296999087, 81.33389556539463, 81.34092346616066, 81.28470026003232, 81.24956075620211, 81.21442125237192, 81.20739335160587, 81.26361655773421, 81.29172816079837, 81.24956075620211, 81.24956075620211, 81.30578396233045, 81.34092346616066, 81.30578396233045, 81.30578396233045, 81.27064445850026, 81.228477053904, 81.17928174854171, 81.11603064164734, 81.03872373322089, 81.01764003092276, 80.97547262632652, 80.94033312249631, 80.96844472556047, 80.92627732096422, 80.89816571790006, 80.84897041253778, 80.8208588094736, 80.79977510717549, 80.77869140487736, 80.79977510717549, 80.75760770257924, 80.715440297983, 80.715440297983, 80.68732869491882, 80.64516129032259, 80.6311054887905, 80.61002178649238, 80.59596598496029, 80.58893808419425, 80.5819101834282, 80.58893808419425, 80.5819101834282, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.990723170988826, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.98622531449856, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.98706866259047, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.4093752196219, 75.6131843418371, 75.84510506711646, 76.0208025862675, 76.27380701384496, 76.50572773912432, 76.54086724295453, 76.70250896057348, 76.89929018202263, 77.13121090730199, 77.37718743411344, 77.60208025862674, 77.95347529692881, 78.31892613676294, 78.53679106051023, 78.76168388502354, 79.0287441141331, 79.18335793098602, 79.38716705320121, 79.5909761754164, 79.78072949609951, 79.93534331295243, 80.13915243516762, 80.3218778550847, 80.4272963665753, 80.5819101834282, 80.6311054887905, 80.72949609951507, 80.93330522173027, 81.05277953475297, 81.05277953475297, 81.05980743551902, 81.11603064164734, 81.19333755007379, 81.19333755007379, 81.17928174854171, 81.21442125237192, 81.31281186309648, 81.31983976386253, 81.3830908707569, 81.35497926769274, 81.43228617611919, 81.4814814814815, 81.44634197765129, 81.42525827535316, 81.34092346616066, 81.39011877152295, 81.40417457305503, 81.39714667228898, 81.40417457305503, 81.37606296999087, 81.43931407688524, 81.45336987841732, 81.40417457305503, 81.36903506922482, 81.37606296999087, 81.37606296999087, 81.33389556539463, 81.34092346616066, 81.28470026003232, 81.24956075620211, 81.21442125237192, 81.20739335160587, 81.26361655773421, 81.29172816079837, 81.24956075620211, 81.24956075620211, 81.30578396233045, 81.34092346616066, 81.30578396233045, 81.30578396233045, 81.27064445850026, 81.228477053904, 81.17928174854171, 81.11603064164734, 81.03872373322089, 81.01764003092276, 80.97547262632652, 80.94033312249631, 80.96844472556047, 80.92627732096422, 80.89816571790006, 80.84897041253778, 80.8208588094736, 80.79977510717549, 80.77869140487736, 80.79977510717549, 80.75760770257924, 80.715440297983, 80.715440297983, 80.68732869491882, 80.64516129032259, 80.6311054887905, 80.61002178649238, 80.59596598496029, 80.58893808419425, 80.5819101834282, 80.58893808419425, 80.5819101834282, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 77.85227352589781, "exact_area100": 80.28196961262941}, "test": {"score": [66.95119530385956, 67.77642287950543, 68.46553044267361, 69.20568301052093, 69.94299974477497, 70.90151149930522, 71.64166406715253, 72.06987493973853, 72.27405495845504, 72.57748915293651, 72.90361001616425, 73.09361086691433, 73.40555256217564, 73.59555341292572, 73.80256926523552, 73.98406261520574, 74.17973513314239, 74.29316847687377, 74.48033349403057, 74.6419760088478, 74.80361852366502, 74.88585769787028, 74.96526103848225, 75.02197771034795, 75.03332104472109, 75.11839605251963, 75.14959022204577, 75.21481439469132, 75.29421773530329, 75.34526273998242, 75.34242690638912, 75.36227774154213, 75.39063607747497, 75.39347191106826, 75.42750191418767, 75.44451691574739, 75.48421858605336, 75.52675608995264, 75.5891444290049, 75.59481609619148, 75.60615943056462, 75.5891444290049, 75.61750276493775, 75.60332359697132, 75.61183109775118, 75.64586110087059, 75.60615943056462, 75.64302526727731, 75.63168193290417, 75.65153276805717, 75.65436860165046, 75.69407027195643, 75.67989110399002, 75.69690610554971, 75.69123443836315, 75.69974193914301, 75.69123443836315, 75.64869693446389, 75.65153276805717, 75.67138360321016, 75.64586110087059, 75.6657119360236, 75.6628761024303, 75.66004026883702, 75.63451776649745, 75.60332359697132, 75.6260102657176, 75.61750276493775, 75.5891444290049, 75.54660692510564, 75.50974108839293, 75.47287525168022, 75.4586960837138, 75.43033774778097, 75.36794940872869, 75.3282477384227, 75.32257607123614, 75.30272523608315, 75.26018773218387, 75.21765022828461, 75.15526188923234, 75.07869438221365, 75.04750021268751, 75.0134702095681, 74.9567535377024, 74.93123103536284, 74.88869353146357, 74.84332019397102, 74.79794685647846, 74.77809602132547, 74.74973768539262, 74.7327226838329, 74.71003601508663, 74.68451351274707, 74.65615517681422, 74.62496100728809, 74.61645350650824, 74.6022743385418, 74.56540850182911, 74.54272183308284, 74.47182599325072], "calls": [0.0, 1.8716501715679326, 3.4568811502141057, 5.155545472591668, 7.143464821484276, 9.987805915548876, 12.537220315911862, 14.133794628931174, 14.94484303661061, 15.985593965346116, 17.134106570626436, 18.123812494682813, 19.161727589825027, 20.072030173269432, 21.118452769191503, 22.162039531520293, 23.114879618863966, 24.062048039021068, 25.046082295890876, 26.03295238635397, 27.184300825227574, 28.225051753963076, 29.10416016788135, 30.005955250545895, 30.995661174602272, 31.979695431472084, 32.91552051725605, 33.85134560304001, 34.957320704421065, 35.94419079488416, 36.94240421972039, 37.949125145336474, 39.06077191390409, 39.973910330941784, 40.99481042452429, 42.05824802200607, 43.0564614468423, 44.08019737401809, 45.15214247227973, 46.27513257522048, 47.41230184612767, 48.35663443269149, 49.87380540509883, 50.883362164308195, 51.93262059382355, 53.07546153191731, 54.266511641096905, 55.295919235459266, 56.50114851260528, 57.700706122564725, 58.51459036383745, 59.79071548081558, 60.77191390409211, 61.70490315628279, 62.76266908657801, 63.63326999971641, 64.61730425658622, 65.54745767518362, 66.48328276096758, 67.45313784987097, 68.36060459972208, 69.33613135581204, 70.30031477752885, 71.24748319768595, 72.10957661004453, 72.98017752318295, 74.14570513002296, 75.03332104472109, 75.94645946175879, 77.06661373110626, 78.00527465048351, 78.83900972690923, 79.72095397442078, 80.5915548875592, 81.69752998894026, 82.56813090207866, 83.23455179650058, 84.16470521509798, 84.88217111419902, 85.65351785157247, 86.41919292175935, 87.25859966537163, 88.0441255707115, 88.76442730340584, 89.41099736267476, 90.08876159146982, 90.66443581090662, 91.25996086549641, 91.96608343022432, 92.53608598247455, 93.14295437143748, 93.64489691744889, 94.25460114000511, 94.86714119615462, 95.52505458979667, 96.16028131469245, 96.77565720443523, 97.38252559339819, 98.00073731673426, 98.7919348892607, 100.0], "exact_score": [66.95119530385956, 67.40209284519185, 67.84731871933755, 68.27836542551682, 68.71791963247597, 69.12911550350225, 69.54314720812182, 69.90046224087571, 70.26061310722287, 70.60374897201032, 70.91001900008507, 71.2021098601934, 71.49136488670844, 71.77211241244363, 72.02450160224598, 72.2797266256416, 72.58032498652979, 72.8752516802314, 73.07376003176134, 73.3403283895301, 73.58704591214588, 73.79406176445566, 73.96137594645947, 74.1598842979894, 74.2903326432805, 74.45764682528429, 74.63914017525451, 74.77526018773219, 74.87735019709044, 74.96526103848225, 75.01914187675467, 75.03332104472109, 75.12123188611292, 75.16660522360547, 75.23750106343759, 75.29705356889657, 75.33959107279584, 75.33959107279584, 75.36511357513541, 75.37929274310183, 75.39630774466154, 75.42750191418767, 75.43884524856081, 75.48138275246008, 75.53809942432578, 75.59765192978476, 75.61183109775118, 75.60615943056462, 75.5891444290049, 75.6089952641579, 75.63168193290417, 75.61466693134446, 75.61466693134446, 75.64302526727731, 75.62033859853103, 75.63451776649745, 75.64302526727731, 75.62317443212432, 75.63168193290417, 75.67421943680344, 75.67705527039674, 75.68839860476987, 75.69690610554971, 75.67989110399002, 75.69974193914301, 75.69407027195643, 75.64869693446389, 75.65436860165046, 75.6657119360236, 75.65153276805717, 75.66854776961688, 75.65720443524374, 75.63451776649745, 75.60615943056462, 75.62033859853103, 75.61750276493775, 75.58347276181834, 75.5522785922922, 75.5154127555795, 75.46153191730708, 75.43884524856081, 75.42183024700111, 75.3480985735757, 75.3282477384227, 75.29988940248987, 75.26018773218387, 75.20630689391147, 75.10138105095993, 75.05033604628079, 75.00496270878824, 74.93406686895612, 74.8660068627173, 74.79794685647846, 74.76108101976577, 74.72137934945977, 74.68451351274707, 74.63346850806795, 74.61645350650824, 74.56540850182911, 74.53705016589626, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 72.72551966650597, "exact_area100": 74.33637575475943}}, "old_normalized_endpoint_budget": 43}, {"seed": 2030, "selection": {"best_epoch": 8, "epochs": 16, "alpha": 1.0, "validation_area100": 0.8124211623906836, "training_seconds": 111.47141420689877, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.4084151119892558, "operator": "ge"}, {"budget": 0.02, "threshold": 0.25435385089607426, "operator": "ge"}, {"budget": 0.03, "threshold": 0.13568222099766614, "operator": "ge"}, {"budget": 0.04, "threshold": 0.04993721440761059, "operator": "ge"}, {"budget": 0.05, "threshold": -0.006949051902607715, "operator": "ge"}, {"budget": 0.06, "threshold": -0.0674436260565543, "operator": "ge"}, {"budget": 0.07, "threshold": -0.14004580063461272, "operator": "ge"}, {"budget": 0.08, "threshold": -0.22629711237789737, "operator": "ge"}, {"budget": 0.09, "threshold": -0.3023964456325138, "operator": "ge"}, {"budget": 0.1, "threshold": -0.3895406922911177, "operator": "ge"}, {"budget": 0.11, "threshold": -0.460657354239823, "operator": "ge"}, {"budget": 0.12, "threshold": -0.5248079168751083, "operator": "ge"}, {"budget": 0.13, "threshold": -0.5898487229250416, "operator": "ge"}, {"budget": 0.14, "threshold": -0.6492335457759217, "operator": "ge"}, {"budget": 0.15, "threshold": -0.700588365406644, "operator": "ge"}, {"budget": 0.16, "threshold": -0.7531586810434099, "operator": "ge"}, {"budget": 0.17, "threshold": -0.792862649537323, "operator": "ge"}, {"budget": 0.18, "threshold": -0.8232061984718168, "operator": "ge"}, {"budget": 0.19, "threshold": -0.8490227221675235, "operator": "ge"}, {"budget": 0.2, "threshold": -0.8695071442335849, "operator": "ge"}, {"budget": 0.21, "threshold": -0.8903844536904425, "operator": "ge"}, {"budget": 0.22, "threshold": -0.9053952981263073, "operator": "ge"}, {"budget": 0.23, "threshold": -0.9195710392763264, "operator": "ge"}, {"budget": 0.24, "threshold": -0.9298560563877756, "operator": "ge"}, {"budget": 0.25, "threshold": -0.9380819333539481, "operator": "ge"}, {"budget": 0.26, "threshold": -0.9459074609151428, "operator": "ge"}, {"budget": 0.27, "threshold": -0.9525644718476897, "operator": "ge"}, {"budget": 0.28, "threshold": -0.9583509561440396, "operator": "ge"}, {"budget": 0.29, "threshold": -0.9631149953051729, "operator": "ge"}, {"budget": 0.3, "threshold": -0.9673493713331437, "operator": "ge"}, {"budget": 0.31, "threshold": -0.9707576916894388, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9737188561925362, "operator": "ge"}, {"budget": 0.33, "threshold": -0.9764049535755023, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9786185774936709, "operator": "ge"}, {"budget": 0.35, "threshold": -0.980782856949954, "operator": "ge"}, {"budget": 0.36, "threshold": -0.9823131955379187, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9840286025944344, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9854951433113787, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9866779253840376, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9876365625943485, "operator": "ge"}, {"budget": 0.41, "threshold": -0.988464539550193, "operator": "ge"}, {"budget": 0.42, "threshold": -0.9892188931238586, "operator": "ge"}, {"budget": 0.43, "threshold": -0.9899224259699648, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9906140870272334, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9911641218451251, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9917080082166599, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9922252577024179, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9926510913645288, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9930212681937612, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9933945455490752, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9937318476003716, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9940555586441773, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9943486967601567, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9946601555956202, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9949364714698641, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9951948516005283, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9954233373200311, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9956192571082473, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9958518075009188, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9960412726552844, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9962485262105106, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9964307594955053, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9965999795014964, "operator": "gt"}, {"budget": 0.64, "threshold": -0.9967550089513169, "operator": "ge"}, {"budget": 0.65, "threshold": -0.996903086617129, "operator": "ge"}, {"budget": 0.66, "threshold": -0.997045173307432, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9971909476227019, "operator": "ge"}, {"budget": 0.68, "threshold": -0.997326317176743, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9974504230418503, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9975904407405742, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9977161195118253, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9978405652374387, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9979632314796176, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9980765024278241, "operator": "ge"}, {"budget": 0.75, "threshold": -0.998175241468199, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9982760450629043, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9983647724270213, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9984499649938404, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9985387272262919, "operator": "gt"}, {"budget": 0.8, "threshold": -0.9986187153128947, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9986931350429981, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9987639276762911, "operator": "ge"}, {"budget": 0.83, "threshold": -0.998838781966811, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9989098463023015, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9989857640560296, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9990632094955049, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9991295257680505, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9992007231822028, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9992537820052362, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9993097860492575, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9993689626145519, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9994326629387036, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9994933014104472, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9995641096491824, "operator": "ge"}, {"budget": 0.95, "threshold": -0.999627001632924, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9996937319763263, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9997567646635128, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9998198742546731, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9998895100203956, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.43045892192002, 75.60615644107105, 75.86618876941458, 75.96457938013916, 76.1613606015883, 76.26677911307893, 76.35111392227142, 76.67439735750932, 77.0468760981095, 77.3209642279851, 77.644247663223, 77.91130789233256, 78.32595403752899, 78.57895846510647, 78.88115819804624, 79.16930212945394, 79.49258556469184, 79.7104504884391, 80.05481762597513, 80.34296155738281, 80.67327289338675, 80.94033312249631, 81.17928174854171, 81.35497926769274, 81.47445358071545, 81.59392789373814, 81.79773701595333, 82.00857403893457, 82.14210415348934, 82.3037458711083, 82.43024808489704, 82.5075549933235, 82.52863869562161, 82.58486190174995, 82.63405720711224, 82.69028041324057, 82.80975472626326, 82.90111743622181, 82.9854522454143, 82.99950804694637, 82.9854522454143, 82.94328484081805, 82.94328484081805, 82.99950804694637, 83.00653594771242, 82.99248014618034, 82.99248014618034, 82.9854522454143, 82.97842434464826, 82.95031274158409, 82.9854522454143, 83.00653594771242, 82.97842434464826, 82.95734064235013, 82.97139644388221, 82.94328484081805, 82.92220113851992, 82.80975472626326, 82.76758732166702, 82.73244781783681, 82.69028041324057, 82.60594560404807, 82.57080610021787, 82.56377819945182, 82.49349919179141, 82.47241548949329, 82.45133178719517, 82.43024808489704, 82.39510858106684, 82.35294117647058, 82.25455056574602, 82.20535526038373, 82.19832735961768, 82.17724365731955, 82.15615995502144, 82.14210415348934, 82.11399255042518, 82.0647972450629, 81.98749033663645, 81.93829503127415, 81.87504392437978, 81.81179281748541, 81.76259751212312, 81.78368121442125, 81.69231850446272, 81.60798369527022, 81.54473258837585, 81.45336987841732, 81.39011877152295, 81.36200716845879, 81.3479513669267, 81.30578396233045, 81.24956075620211, 81.15819804624358, 81.05980743551902, 80.97547262632652, 80.91924942019818, 80.84897041253778, 80.77166350411133, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99107456602713, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.98657670953686, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 66, 107, 154, 188, 232, 261, 291, 346, 403, 454, 509, 558, 624, 676, 731, 785, 846, 892, 953, 1003, 1062, 1107, 1151, 1194, 1232, 1265, 1307, 1349, 1378, 1413, 1444, 1466, 1487, 1506, 1523, 1547, 1572, 1595, 1619, 1631, 1645, 1654, 1664, 1680, 1696, 1707, 1719, 1730, 1740, 1749, 1761, 1772, 1777, 1781, 1790, 1797, 1800, 1806, 1812, 1819, 1825, 1828, 1834, 1841, 1844, 1849, 1856, 1861, 1865, 1873, 1874, 1877, 1884, 1888, 1893, 1898, 1902, 1903, 1905, 1907, 1908, 1911, 1912, 1919, 1921, 1923, 1927, 1928, 1932, 1934, 1941, 1943, 1946, 1948, 1951, 1955, 1958, 1961, 1964, 1964], "harmed": [0, 9, 25, 35, 55, 71, 85, 103, 112, 116, 128, 137, 148, 155, 171, 183, 196, 211, 226, 238, 247, 259, 266, 276, 294, 315, 331, 344, 356, 366, 378, 391, 402, 420, 431, 441, 457, 465, 475, 487, 497, 513, 528, 538, 546, 561, 574, 586, 598, 609, 622, 629, 637, 646, 653, 660, 671, 677, 699, 711, 723, 735, 750, 761, 769, 782, 790, 800, 808, 817, 831, 846, 856, 864, 871, 879, 886, 894, 902, 915, 924, 934, 946, 954, 958, 973, 987, 1000, 1014, 1027, 1033, 1042, 1050, 1061, 1076, 1093, 1109, 1120, 1133, 1147, 1171], "exact_score": [75.02986857825567, 75.43045892192002, 75.60615644107105, 75.86618876941458, 75.96457938013916, 76.1613606015883, 76.26677911307893, 76.35111392227142, 76.67439735750932, 77.0468760981095, 77.3209642279851, 77.644247663223, 77.91130789233256, 78.32595403752899, 78.57895846510647, 78.88115819804624, 79.16930212945394, 79.49258556469184, 79.7104504884391, 80.05481762597513, 80.34296155738281, 80.67327289338675, 80.94033312249631, 81.17928174854171, 81.35497926769274, 81.47445358071545, 81.59392789373814, 81.79773701595333, 82.00857403893457, 82.14210415348934, 82.3037458711083, 82.43024808489704, 82.5075549933235, 82.52863869562161, 82.58486190174995, 82.63405720711224, 82.69028041324057, 82.80975472626326, 82.90111743622181, 82.9854522454143, 82.99950804694637, 82.9854522454143, 82.94328484081805, 82.94328484081805, 82.99950804694637, 83.00653594771242, 82.99248014618034, 82.99248014618034, 82.9854522454143, 82.97842434464826, 82.95031274158409, 82.9854522454143, 83.00653594771242, 82.97842434464826, 82.95734064235013, 82.97139644388221, 82.94328484081805, 82.92220113851992, 82.80975472626326, 82.76758732166702, 82.73244781783681, 82.69028041324057, 82.60594560404807, 82.57080610021787, 82.56377819945182, 82.49349919179141, 82.47241548949329, 82.45133178719517, 82.43024808489704, 82.39510858106684, 82.35294117647058, 82.25455056574602, 82.20535526038373, 82.19832735961768, 82.17724365731955, 82.15615995502144, 82.14210415348934, 82.11399255042518, 82.0647972450629, 81.98749033663645, 81.93829503127415, 81.87504392437978, 81.81179281748541, 81.76259751212312, 81.78368121442125, 81.69231850446272, 81.60798369527022, 81.54473258837585, 81.45336987841732, 81.39011877152295, 81.36200716845879, 81.3479513669267, 81.30578396233045, 81.24956075620211, 81.15819804624358, 81.05980743551902, 80.97547262632652, 80.91924942019818, 80.84897041253778, 80.77166350411133, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.24211623906837, "threshold_area100": 81.24193548387096, "threshold_area25": 78.19172113289761}, "test": {"score": [66.95119530385956, 68.14508124663244, 69.0100104925843, 69.84374556901001, 70.57255480248419, 71.13972152114114, 71.42897654765619, 71.71255990698465, 72.16062161472364, 72.47256330998498, 72.80435584039928, 73.10495420128747, 73.40838839576892, 73.67779258713098, 73.89615177381391, 74.1031676261237, 74.33003431358648, 74.50018432918355, 74.74406601820604, 74.84048436037773, 74.91705186739642, 75.02764937753452, 75.1439185548592, 75.311232736863, 75.4019794118481, 75.47571108527352, 75.54093525791906, 75.61183109775118, 75.68556277117658, 75.79048861412812, 75.84153361880725, 75.90675779145279, 75.94929529535207, 75.98899696565805, 76.04004197033719, 76.01451946799762, 76.04571363752375, 76.03437030315061, 76.04287780393047, 76.11377364376258, 76.1109378101693, 76.127952811729, 76.14496781328872, 76.13362447891558, 76.12511697813572, 76.16765448203499, 76.15914698125515, 76.15347531406856, 76.17332614922157, 76.19034115078126, 76.19317698437456, 76.19034115078126, 76.19034115078126, 76.19601281796784, 76.19601281796784, 76.2045203187477, 76.21586365312083, 76.2215353203074, 76.21869948671412, 76.1846694835947, 76.19884865156112, 76.19034115078126, 76.17899781640813, 76.15914698125515, 76.14496781328872, 76.11944531094915, 76.08825114142302, 76.08825114142302, 76.0740719734566, 76.04287780393047, 76.00601196721776, 75.98899696565805, 75.91810112582593, 75.88123528911323, 75.86422028755354, 75.83019028443411, 75.79899611490798, 75.79048861412812, 75.76496611178855, 75.74227944304228, 75.719592774296, 75.69974193914301, 75.69407027195643, 75.67138360321016, 75.63735360009073, 75.6089952641579, 75.55511442588548, 75.5324277571392, 75.47287525168022, 75.4416810821541, 75.41899441340783, 75.39347191106826, 75.32541190482942, 75.28854606811673, 75.1977993931316, 75.15242605563905, 75.08153021580694, 74.9964552080084, 74.89720103224342, 74.73839435101948, 74.47182599325072], "calls": [0.0, 2.143890196523268, 4.001361200124777, 6.366446416924256, 8.65212829311176, 10.501091795933416, 11.547514391855486, 12.673340328389529, 14.145137963304313, 15.313501403737629, 16.79097070583898, 18.15784249780223, 19.226951762470577, 20.55979355131441, 21.782037830020133, 22.95607293764002, 24.456228908487653, 25.706831523126223, 26.818478291693843, 27.947140061821173, 28.916995150724556, 30.218642770042255, 31.367155375322575, 32.657459660267136, 33.80313643195417, 34.8949323653688, 36.09165414173496, 37.23449507982872, 38.43405268978816, 39.58823696225505, 40.86152624563991, 42.09227802512549, 43.35989564132377, 44.71542409891388, 45.96886254714573, 47.21379349459774, 48.322604429572074, 49.584550378583785, 50.85216799478207, 52.00351643365567, 53.04993902957774, 54.0623316223804, 54.98114170660465, 55.91696679238862, 56.93786688597113, 57.941751977993924, 58.77832288801294, 59.844596319088, 60.69251056348014, 61.546096475058846, 62.43087655616368, 63.28162663414911, 64.07566004026883, 64.8810367807617, 65.74880186030684, 66.44925275784817, 67.23477866318804, 67.97493123103536, 68.59030712077815, 69.38150469330459, 70.0876272580325, 70.99225817429033, 71.70972407339138, 72.393159969373, 73.05390919660834, 73.7685392621161, 74.48033349403057, 75.16660522360547, 75.84436945240053, 76.51646201400902, 77.24243541388991, 77.9882596489238, 78.73691971755098, 79.52811729007742, 80.2200606868389, 80.87230241329438, 81.63514164988798, 82.31290587868304, 82.98499844029152, 83.65425516830672, 84.2526160564898, 84.89918611575872, 85.42948699770298, 86.06187788900547, 86.65740294359527, 87.30964467005076, 88.06114057227123, 88.68785979638714, 89.42801236423446, 89.95547741258542, 90.54816663358194, 91.206080027224, 91.89802342398548, 92.60414598871338, 93.48892606981822, 94.28579530953124, 95.03445537815841, 95.9107279584834, 96.92879221847261, 98.26163400731645, 100.0], "rescued": [0, 435, 752, 1071, 1348, 1567, 1689, 1817, 2002, 2151, 2311, 2454, 2590, 2728, 2841, 2952, 3084, 3180, 3297, 3371, 3435, 3525, 3602, 3688, 3755, 3817, 3876, 3941, 4002, 4070, 4123, 4178, 4229, 4276, 4325, 4356, 4395, 4428, 4471, 4520, 4552, 4588, 4613, 4641, 4662, 4697, 4716, 4736, 4764, 4788, 4814, 4836, 4861, 4883, 4903, 4921, 4937, 4954, 4967, 4984, 5003, 5021, 5033, 5043, 5057, 5072, 5086, 5104, 5113, 5124, 5131, 5143, 5147, 5160, 5172, 5179, 5191, 5202, 5215, 5221, 5229, 5240, 5251, 5267, 5277, 5288, 5300, 5310, 5315, 5320, 5326, 5337, 5343, 5351, 5360, 5371, 5378, 5387, 5400, 5411, 5423], "harmed": [0, 14, 26, 51, 71, 90, 110, 138, 165, 204, 247, 284, 313, 356, 392, 430, 482, 518, 549, 589, 626, 677, 713, 740, 775, 811, 847, 887, 922, 953, 988, 1020, 1056, 1089, 1120, 1160, 1188, 1225, 1265, 1289, 1322, 1352, 1371, 1403, 1427, 1447, 1469, 1491, 1512, 1530, 1555, 1578, 1603, 1623, 1643, 1658, 1670, 1685, 1699, 1728, 1742, 1763, 1779, 1796, 1815, 1839, 1864, 1882, 1896, 1918, 1938, 1956, 1985, 2011, 2029, 2048, 2071, 2085, 2107, 2121, 2137, 2155, 2168, 2192, 2214, 2235, 2266, 2284, 2310, 2326, 2340, 2360, 2390, 2411, 2452, 2479, 2511, 2550, 2598, 2665, 2771], "exact_score": [66.95119530385956, 67.60910869750163, 68.08552874117346, 68.58179961999829, 69.0100104925843, 69.40135552845759, 69.71613305731219, 70.05359725491309, 70.35986728298784, 70.68315231262228, 70.96673567195076, 71.29852820236509, 71.56226072654056, 71.78912741400335, 72.12091994441766, 72.37330913422, 72.63987749198877, 72.86390834585826, 73.08510336613448, 73.3403283895301, 73.5643592433996, 73.74585259336982, 73.92734594334004, 74.1031676261237, 74.23645180500809, 74.4179451549783, 74.56824433542239, 74.74973768539262, 74.83764852678445, 74.92555936817628, 74.99929104160168, 75.0872018829935, 75.21197856109804, 75.33391940560928, 75.41899441340783, 75.49839775401979, 75.5352635907325, 75.59765192978476, 75.6657119360236, 75.71108527351615, 75.79899611490798, 75.84720528599381, 75.90959362504609, 75.95780279613193, 75.94929529535207, 76.00034030003118, 76.04287780393047, 76.02586280237075, 76.03437030315061, 76.0542211383036, 76.02586280237075, 76.04854947111703, 76.11377364376258, 76.11377364376258, 76.11377364376258, 76.147803646882, 76.13362447891558, 76.12228114454244, 76.17049031562829, 76.15631114766185, 76.15631114766185, 76.18750531718798, 76.21019198593426, 76.2016844851544, 76.19034115078126, 76.2016844851544, 76.19884865156112, 76.21302781952754, 76.2215353203074, 76.20735615234098, 76.19317698437456, 76.19034115078126, 76.17616198281485, 76.147803646882, 76.11660947735587, 76.10526614298273, 76.07974364064316, 76.00884780081105, 75.98899696565805, 75.90959362504609, 75.86422028755354, 75.8131752828744, 75.78765278053483, 75.76496611178855, 75.73377194226242, 75.69690610554971, 75.67705527039674, 75.62884609931089, 75.55795025947877, 75.50406942120637, 75.4416810821541, 75.41048691262796, 75.31974023764285, 75.23750106343759, 75.17227689079205, 75.08436604940022, 74.99078354082182, 74.90003686583671, 74.79227518929189, 74.66182684400079, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.65109690952931, "threshold_area100": 75.11383036043445, "threshold_area25": 72.83555000992543}, "selected_budget": 45, "selected": {"val": {"score": 83.00653594771242, "calls": 44.9996486049617, "rescued": 1696, "harmed": 561}, "test": {"score": 76.16765448203499, "calls": 57.941751977993924, "rescued": 4697, "harmed": 1447}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5115378499031067, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3289926052093506, "operator": "ge"}, {"budget": 0.03, "threshold": 0.17179089784622192, "operator": "ge"}, {"budget": 0.04, "threshold": 0.07152929902076721, "operator": "ge"}, {"budget": 0.05, "threshold": -0.0007246686145663261, "operator": "ge"}, {"budget": 0.06, "threshold": -0.002751978114247322, "operator": "ge"}, {"budget": 0.07, "threshold": -0.004143918864428997, "operator": "ge"}, {"budget": 0.08, "threshold": -0.00524786626920104, "operator": "ge"}, {"budget": 0.09, "threshold": -0.0063668605871498585, "operator": "ge"}, {"budget": 0.1, "threshold": -0.007743085268884897, "operator": "ge"}, {"budget": 0.11, "threshold": -0.009024576283991337, "operator": "ge"}, {"budget": 0.12, "threshold": -0.010502965189516544, "operator": "ge"}, {"budget": 0.13, "threshold": -0.011759944260120392, "operator": "ge"}, {"budget": 0.14, "threshold": -0.012913555838167667, "operator": "ge"}, {"budget": 0.15, "threshold": -0.014432443305850029, "operator": "ge"}, {"budget": 0.16, "threshold": -0.01568721979856491, "operator": "ge"}, {"budget": 0.17, "threshold": -0.01726122945547104, "operator": "ge"}, {"budget": 0.18, "threshold": -0.01892448216676712, "operator": "ge"}, {"budget": 0.19, "threshold": -0.02055029198527336, "operator": "ge"}, {"budget": 0.2, "threshold": -0.022162139415740967, "operator": "ge"}, {"budget": 0.21, "threshold": -0.024047452956438065, "operator": "ge"}, {"budget": 0.22, "threshold": -0.025779837742447853, "operator": "ge"}, {"budget": 0.23, "threshold": -0.02755023166537285, "operator": "ge"}, {"budget": 0.24, "threshold": -0.029346881434321404, "operator": "ge"}, {"budget": 0.25, "threshold": -0.03124883584678173, "operator": "ge"}, {"budget": 0.26, "threshold": -0.03307047486305237, "operator": "ge"}, {"budget": 0.27, "threshold": -0.03514649346470833, "operator": "ge"}, {"budget": 0.28, "threshold": -0.03729185089468956, "operator": "ge"}, {"budget": 0.29, "threshold": -0.03960200771689415, "operator": "ge"}, {"budget": 0.3, "threshold": -0.0416237935423851, "operator": "ge"}, {"budget": 0.31, "threshold": -0.04400009661912918, "operator": "ge"}, {"budget": 0.32, "threshold": -0.0463406965136528, "operator": "ge"}, {"budget": 0.33, "threshold": -0.04872193932533264, "operator": "ge"}, {"budget": 0.34, "threshold": -0.051100220531225204, "operator": "ge"}, {"budget": 0.35, "threshold": -0.05372170731425285, "operator": "ge"}, {"budget": 0.36, "threshold": -0.05627727136015892, "operator": "ge"}, {"budget": 0.37, "threshold": -0.05889715254306793, "operator": "ge"}, {"budget": 0.38, "threshold": -0.06153840571641922, "operator": "ge"}, {"budget": 0.39, "threshold": -0.06465929746627808, "operator": "ge"}, {"budget": 0.4, "threshold": -0.06749889999628067, "operator": "ge"}, {"budget": 0.41, "threshold": -0.07055685669183731, "operator": "ge"}, {"budget": 0.42, "threshold": -0.0738954171538353, "operator": "ge"}, {"budget": 0.43, "threshold": -0.07699476182460785, "operator": "ge"}, {"budget": 0.44, "threshold": -0.0801405981183052, "operator": "ge"}, {"budget": 0.45, "threshold": -0.08355733007192612, "operator": "ge"}, {"budget": 0.46, "threshold": -0.0873873233795166, "operator": "ge"}, {"budget": 0.47, "threshold": -0.09097685664892197, "operator": "ge"}, {"budget": 0.48, "threshold": -0.09524355828762054, "operator": "ge"}, {"budget": 0.49, "threshold": -0.09951979666948318, "operator": "ge"}, {"budget": 0.5, "threshold": -0.10345254093408585, "operator": "ge"}, {"budget": 0.51, "threshold": -0.10760441422462463, "operator": "ge"}, {"budget": 0.52, "threshold": -0.11169501394033432, "operator": "ge"}, {"budget": 0.53, "threshold": -0.11706531047821045, "operator": "ge"}, {"budget": 0.54, "threshold": -0.12167420983314514, "operator": "ge"}, {"budget": 0.55, "threshold": -0.1270155906677246, "operator": "ge"}, {"budget": 0.56, "threshold": -0.13297536969184875, "operator": "ge"}, {"budget": 0.57, "threshold": -0.1390104591846466, "operator": "ge"}, {"budget": 0.58, "threshold": -0.14556165039539337, "operator": "gt"}, {"budget": 0.59, "threshold": -0.15065652132034302, "operator": "ge"}, {"budget": 0.6, "threshold": -0.15660007297992706, "operator": "ge"}, {"budget": 0.61, "threshold": -0.16284890472888947, "operator": "ge"}, {"budget": 0.62, "threshold": -0.17015263438224792, "operator": "ge"}, {"budget": 0.63, "threshold": -0.1759498119354248, "operator": "ge"}, {"budget": 0.64, "threshold": -0.1823025941848755, "operator": "ge"}, {"budget": 0.65, "threshold": -0.18876898288726807, "operator": "ge"}, {"budget": 0.66, "threshold": -0.19567199051380157, "operator": "ge"}, {"budget": 0.67, "threshold": -0.20206835865974426, "operator": "ge"}, {"budget": 0.68, "threshold": -0.2086959034204483, "operator": "ge"}, {"budget": 0.69, "threshold": -0.21672043204307556, "operator": "ge"}, {"budget": 0.7, "threshold": -0.2248132973909378, "operator": "ge"}, {"budget": 0.71, "threshold": -0.23241616785526276, "operator": "ge"}, {"budget": 0.72, "threshold": -0.23966526985168457, "operator": "ge"}, {"budget": 0.73, "threshold": -0.24765487015247345, "operator": "ge"}, {"budget": 0.74, "threshold": -0.25623026490211487, "operator": "ge"}, {"budget": 0.75, "threshold": -0.2645299732685089, "operator": "ge"}, {"budget": 0.76, "threshold": -0.2744270861148834, "operator": "ge"}, {"budget": 0.77, "threshold": -0.2848210632801056, "operator": "ge"}, {"budget": 0.78, "threshold": -0.29372939467430115, "operator": "ge"}, {"budget": 0.79, "threshold": -0.30450600385665894, "operator": "ge"}, {"budget": 0.8, "threshold": -0.31556227803230286, "operator": "ge"}, {"budget": 0.81, "threshold": -0.32766246795654297, "operator": "ge"}, {"budget": 0.82, "threshold": -0.3398944139480591, "operator": "ge"}, {"budget": 0.83, "threshold": -0.35101431608200073, "operator": "ge"}, {"budget": 0.84, "threshold": -0.3631327152252197, "operator": "ge"}, {"budget": 0.85, "threshold": -0.3779473602771759, "operator": "ge"}, {"budget": 0.86, "threshold": -0.39178839325904846, "operator": "ge"}, {"budget": 0.87, "threshold": -0.4049668610095978, "operator": "ge"}, {"budget": 0.88, "threshold": -0.4167397916316986, "operator": "ge"}, {"budget": 0.89, "threshold": -0.43419623374938965, "operator": "ge"}, {"budget": 0.9, "threshold": -0.4533675014972687, "operator": "ge"}, {"budget": 0.91, "threshold": -0.4725618064403534, "operator": "ge"}, {"budget": 0.92, "threshold": -0.49469831585884094, "operator": "ge"}, {"budget": 0.93, "threshold": -0.5252401232719421, "operator": "ge"}, {"budget": 0.94, "threshold": -0.5526185631752014, "operator": "ge"}, {"budget": 0.95, "threshold": -0.5878258943557739, "operator": "ge"}, {"budget": 0.96, "threshold": -0.6362938284873962, "operator": "ge"}, {"budget": 0.97, "threshold": -0.6877058744430542, "operator": "ge"}, {"budget": 0.98, "threshold": -0.7421974539756775, "operator": "ge"}, {"budget": 0.99, "threshold": -0.8186884522438049, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.42343102115399, 75.62724014336918, 75.8521329678825, 76.0208025862675, 76.14027689929019, 76.13324899852414, 76.14730480005622, 76.15433270082227, 76.1613606015883, 76.18947220465247, 76.21055590695059, 76.22461170848268, 76.2456954107808, 76.25272331154684, 76.2456954107808, 76.27380701384496, 76.3300302199733, 76.34408602150538, 76.40030922763371, 76.42842083069787, 76.42842083069787, 76.47761613606015, 76.54086724295453, 76.57600674678474, 76.59006254831682, 76.59006254831682, 76.62520205214702, 76.61817415138098, 76.66034155597723, 76.70953686133953, 76.73062056363764, 76.73764846440369, 76.79387167053201, 76.86415067819242, 76.90631808278867, 76.90631808278867, 76.96956918968304, 76.97659709044909, 76.98362499121512, 77.11012720500386, 77.19446201419636, 77.23662941879262, 77.29988052568699, 77.26474102185678, 77.22257361726052, 77.22960151802657, 77.22960151802657, 77.27176892262281, 77.34907583104926, 77.43341064024176, 77.45449434253987, 77.50368964790218, 77.58099655632863, 77.60208025862674, 77.644247663223, 77.68641506781925, 77.72155457164945, 77.7988614800759, 77.79183357930987, 77.76372197624569, 77.83400098390611, 77.86914048773632, 77.91130789233256, 77.96050319769485, 78.13620071684588, 78.24864712910254, 78.34000983906108, 78.38217724365732, 78.43137254901961, 78.4454283505517, 78.57193056434043, 78.6351816712348, 78.75465598425751, 78.78276758732167, 78.81087919038583, 78.8741302972802, 78.93035350340854, 78.96549300723873, 79.06388361796331, 79.17633003021997, 79.26066483941247, 79.39419495396724, 79.49961346545787, 79.60503197694848, 79.68233888537493, 79.80884109916369, 79.90723170988826, 79.98453861831472, 80.04778972520909, 80.20240354206199, 80.24457094665823, 80.25159884742428, 80.36404525968094, 80.4272963665753, 80.48351957270363, 80.6662449926207, 80.69435659568487, 80.6803007941528, 80.70138449645091, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.987209220605806, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 66, 107, 154, 194, 230, 231, 234, 237, 239, 243, 247, 250, 254, 258, 262, 266, 274, 279, 290, 297, 301, 310, 321, 331, 336, 342, 352, 355, 362, 374, 382, 387, 397, 411, 420, 430, 442, 448, 458, 476, 491, 504, 518, 528, 535, 546, 557, 572, 590, 605, 618, 635, 654, 666, 681, 698, 716, 738, 751, 765, 788, 807, 829, 850, 885, 916, 942, 961, 981, 1000, 1034, 1058, 1087, 1112, 1140, 1169, 1195, 1219, 1252, 1289, 1316, 1352, 1386, 1418, 1452, 1489, 1522, 1554, 1585, 1630, 1665, 1688, 1722, 1757, 1787, 1830, 1866, 1900, 1934, 1964], "harmed": [0, 10, 22, 37, 53, 72, 74, 75, 77, 78, 78, 79, 80, 81, 84, 89, 89, 89, 92, 95, 98, 102, 104, 106, 111, 114, 120, 125, 129, 130, 135, 140, 144, 146, 150, 153, 163, 166, 171, 180, 180, 183, 190, 195, 210, 223, 233, 244, 253, 260, 263, 273, 283, 291, 300, 309, 320, 333, 344, 358, 376, 389, 403, 419, 433, 443, 458, 471, 484, 497, 514, 530, 545, 557, 578, 602, 622, 640, 659, 678, 699, 714, 731, 750, 767, 790, 809, 828, 849, 871, 894, 923, 945, 963, 989, 1011, 1028, 1060, 1096, 1127, 1171], "exact_score": [75.02986857825567, 75.42343102115399, 75.62724014336918, 75.8521329678825, 76.0208025862675, 76.14027689929019, 76.13324899852414, 76.14730480005622, 76.15433270082227, 76.1613606015883, 76.18947220465247, 76.21055590695059, 76.22461170848268, 76.2456954107808, 76.25272331154684, 76.2456954107808, 76.27380701384496, 76.3300302199733, 76.34408602150538, 76.40030922763371, 76.42842083069787, 76.42842083069787, 76.47761613606015, 76.54086724295453, 76.57600674678474, 76.59006254831682, 76.59006254831682, 76.62520205214702, 76.61817415138098, 76.66034155597723, 76.70953686133953, 76.73062056363764, 76.73764846440369, 76.79387167053201, 76.86415067819242, 76.90631808278867, 76.90631808278867, 76.96956918968304, 76.97659709044909, 76.98362499121512, 77.11012720500386, 77.19446201419636, 77.23662941879262, 77.29988052568699, 77.26474102185678, 77.22257361726052, 77.22960151802657, 77.22960151802657, 77.27176892262281, 77.34907583104926, 77.43341064024176, 77.45449434253987, 77.50368964790218, 77.58099655632863, 77.60208025862674, 77.644247663223, 77.68641506781925, 77.72155457164945, 77.7988614800759, 77.79183357930987, 77.76372197624569, 77.83400098390611, 77.86914048773632, 77.91130789233256, 77.96050319769485, 78.13620071684588, 78.24864712910254, 78.34000983906108, 78.38217724365732, 78.43137254901961, 78.4454283505517, 78.57193056434043, 78.6351816712348, 78.75465598425751, 78.78276758732167, 78.81087919038583, 78.8741302972802, 78.93035350340854, 78.96549300723873, 79.06388361796331, 79.17633003021997, 79.26066483941247, 79.39419495396724, 79.49961346545787, 79.60503197694848, 79.68233888537493, 79.80884109916369, 79.90723170988826, 79.98453861831472, 80.04778972520909, 80.20240354206199, 80.24457094665823, 80.25159884742428, 80.36404525968094, 80.4272963665753, 80.48351957270363, 80.6662449926207, 80.69435659568487, 80.6803007941528, 80.70138449645091, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 77.76747152355512, "threshold_area100": 77.76727106613254, "threshold_area25": 76.16276618174152}, "test": {"score": [66.95119530385956, 68.22164875365114, 69.12344383631567, 69.95434307914812, 70.58106230326405, 71.08584068286874, 71.10285568442843, 71.11136318520829, 71.13121402036128, 71.15106485551428, 71.15957235629413, 71.19076652582027, 71.21061736097326, 71.21628902815984, 71.23330402971953, 71.25599069846581, 71.28718486799195, 71.2956923687718, 71.31554320392479, 71.3325582054845, 71.35808070782407, 71.39211071094348, 71.40345404531662, 71.45166321640247, 71.4630065507756, 71.48569321952188, 71.4998723874883, 71.51405155545473, 71.54808155857414, 71.58211156169357, 71.60196239684655, 71.62181323199954, 71.64449990074581, 71.69270907183166, 71.7409182429175, 71.74942574369736, 71.76644074525707, 71.78912741400335, 71.78345574681677, 71.84017241868247, 71.86285908742875, 71.86285908742875, 71.90256075773473, 71.95360576241386, 72.00465076709298, 72.03300910302583, 72.06703910614524, 72.06420327255196, 72.10106910926467, 72.14360661316394, 72.2059949522162, 72.24853245611548, 72.26838329126846, 72.31375662876103, 72.3364432975073, 72.37614496781329, 72.43569747327227, 72.5037574795111, 72.50659331310439, 72.56330998497008, 72.61435498964921, 72.70510166463431, 72.77883333805973, 72.82987834273884, 72.90361001616425, 72.9319683520971, 72.9688341888098, 73.02271502708221, 73.03973002864191, 73.07092419816806, 73.1361483708136, 73.19853670986586, 73.22122337861214, 73.30062671922411, 73.3771942262428, 73.45659756685478, 73.56719507699287, 73.60689674729886, 73.66077758557128, 73.70615092306383, 73.75436009414967, 73.83943510194821, 73.88480843944077, 73.93018177693332, 73.99824178317216, 74.07764512378414, 74.13436179564982, 74.16272013158267, 74.21092930266852, 74.31018347843347, 74.40660182060516, 74.44063182372457, 74.49167682840371, 74.56540850182911, 74.57675183620225, 74.59093100416867, 74.59660267135524, 74.58809517057539, 74.53988599948956, 74.55690100104925, 74.47182599325072], "calls": [0.0, 2.2516518730680883, 4.2253920539942715, 6.635850608286306, 8.584068286872926, 10.526614298272978, 12.199756118310978, 13.592150412613787, 14.607378839009726, 15.648129767745228, 16.989479057368914, 18.18336500014179, 19.48501261945949, 20.49456937866886, 21.30845361994158, 22.37189121742336, 23.25383546493492, 24.322944729603265, 25.397725661458185, 26.359073249581716, 27.297734168958964, 28.27893259223549, 29.217593511612737, 30.113716927090717, 30.992825341008988, 31.951337095539234, 32.80492300711794, 33.732240592122054, 34.64821484275303, 35.64359243399598, 36.4943425119814, 37.447182599325075, 38.26957434137765, 39.2451010974676, 39.973910330941784, 40.85301874486005, 41.681082154099194, 42.48929472818535, 43.24929813118566, 44.29004905992117, 45.08975413322746, 45.87811587216063, 46.80259762357145, 47.63066103281059, 48.424694438930324, 49.17051867396422, 49.93619374415109, 50.72171964949097, 51.649037234495076, 52.47426481014094, 53.3193432209398, 54.16158579814536, 54.95561920426509, 55.894280123642346, 56.77906020474719, 57.73190029209086, 58.727277883333805, 59.756685477696166, 60.788928905651815, 61.53758897427899, 62.35998071633156, 63.2844624677424, 64.3620792331906, 65.1787993080566, 66.07208688994129, 66.84343362731475, 67.64881036780763, 68.46269460908034, 69.26807134957322, 70.10747809318549, 71.06598984771574, 71.81464991634292, 72.5604741513768, 73.32331338797039, 74.31301931202677, 75.16660522360547, 76.18750531718798, 77.13183790375182, 77.96840881377081, 78.8418455605025, 79.74080480957377, 80.79857073986898, 81.82797833423135, 82.70141508096305, 83.6202251651873, 84.61560275643025, 85.43515866488954, 86.34262541474065, 87.10546465133426, 88.1405439128832, 89.10472733460001, 90.02070158523098, 91.07846751552619, 92.21280095284008, 93.31026855344129, 94.43325865638205, 95.75192127725946, 96.90326971613305, 97.90999064174915, 98.9422340697048, 100.0], "rescued": [0, 460, 789, 1110, 1346, 1550, 1556, 1560, 1570, 1580, 1586, 1598, 1607, 1618, 1629, 1642, 1657, 1668, 1681, 1693, 1709, 1727, 1737, 1762, 1774, 1790, 1802, 1818, 1836, 1857, 1876, 1897, 1915, 1942, 1964, 1987, 2005, 2024, 2035, 2067, 2090, 2108, 2138, 2165, 2195, 2216, 2247, 2270, 2300, 2336, 2372, 2408, 2436, 2478, 2511, 2550, 2601, 2657, 2694, 2739, 2781, 2838, 2907, 2952, 3012, 3056, 3100, 3148, 3190, 3244, 3307, 3358, 3394, 3447, 3509, 3570, 3651, 3717, 3771, 3829, 3890, 3967, 4048, 4111, 4180, 4268, 4327, 4386, 4449, 4536, 4615, 4681, 4768, 4852, 4933, 5016, 5128, 5213, 5275, 5358, 5423], "harmed": [0, 12, 23, 51, 66, 92, 92, 93, 96, 99, 102, 103, 105, 114, 119, 124, 128, 136, 142, 148, 155, 161, 167, 175, 183, 191, 198, 209, 215, 224, 236, 250, 260, 270, 275, 295, 307, 318, 331, 343, 358, 376, 392, 401, 413, 424, 443, 467, 484, 505, 519, 540, 561, 587, 612, 637, 667, 699, 735, 760, 784, 809, 852, 879, 913, 947, 978, 1007, 1043, 1086, 1126, 1155, 1183, 1208, 1243, 1276, 1318, 1370, 1405, 1447, 1491, 1538, 1603, 1650, 1695, 1755, 1794, 1843, 1889, 1941, 1986, 2040, 2109, 2167, 2244, 2322, 2432, 2520, 2599, 2676, 2771], "exact_score": [66.95119530385956, 67.61194453109492, 68.0997079091399, 68.57896378640501, 69.02418966055072, 69.43538553157701, 69.78702889714431, 70.09329892521907, 70.41658395485354, 70.69733148058872, 70.97524317273061, 71.08867651646202, 71.10285568442843, 71.10852735161501, 71.11987068598815, 71.13121402036128, 71.15390068910756, 71.15957235629413, 71.19076652582027, 71.20494569378668, 71.21345319456654, 71.2219606953464, 71.24181153049939, 71.27016986643223, 71.29285653517852, 71.30987153673823, 71.32688653829793, 71.3495732070442, 71.38643904375691, 71.40061821172333, 71.44882738280917, 71.4630065507756, 71.48569321952188, 71.4998723874883, 71.53106655701443, 71.56793239372713, 71.5934548960667, 71.6104698976264, 71.63599239996597, 71.67569407027196, 71.7437540765108, 71.75793324447721, 71.78061991322349, 71.77778407963021, 71.81181408274962, 71.86002325383546, 71.86569492102203, 71.92241159288773, 71.97345659756685, 72.01315826787284, 72.06136743895868, 72.06987493973853, 72.1322632787908, 72.1492782803505, 72.22300995377591, 72.26838329126846, 72.30808496157445, 72.34211496469388, 72.39032413577972, 72.46405580920512, 72.50942914669767, 72.5406233162238, 72.60584748886934, 72.67674332870146, 72.75047500212686, 72.801520006806, 72.88659501460454, 72.94047585287696, 72.99152085755608, 73.03689419504863, 73.07092419816806, 73.15032753878002, 73.20420837705244, 73.25241754813827, 73.34600005671668, 73.42823923092193, 73.56152340980631, 73.60406091370558, 73.657941751978, 73.70047925587726, 73.77704676289595, 73.86779343788106, 73.8791367722542, 73.96704761364603, 74.02092845191845, 74.10600345971699, 74.1967501347021, 74.19958596829538, 74.2903326432805, 74.38391515185889, 74.44063182372457, 74.47749766043728, 74.54272183308284, 74.56540850182911, 74.57391600260897, 74.58242350338881, 74.59376683776196, 74.5852593369821, 74.54555766667612, 74.56257266823583, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 72.2301736041386, "threshold_area100": 72.41108243768257, "threshold_area25": 70.81558574142869}, "selected_budget": 99, "selected": {"val": {"score": 80.70138449645091, "calls": 98.99501019045611, "rescued": 1934, "harmed": 1127}, "test": {"score": 74.55690100104925, "calls": 98.9422340697048, "rescued": 5358, "harmed": 2676}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.683022975921631, "operator": "ge"}, {"budget": 0.02, "threshold": 2.694261074066162, "operator": "ge"}, {"budget": 0.03, "threshold": 1.4840075969696045, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5859242677688599, "operator": "ge"}, {"budget": 0.05, "threshold": -0.040401995182037354, "operator": "ge"}, {"budget": 0.06, "threshold": -0.3908153176307678, "operator": "ge"}, {"budget": 0.07, "threshold": -0.8052469491958618, "operator": "ge"}, {"budget": 0.08, "threshold": -1.1779401302337646, "operator": "ge"}, {"budget": 0.09, "threshold": -1.4883003234863281, "operator": "ge"}, {"budget": 0.1, "threshold": -1.784356951713562, "operator": "ge"}, {"budget": 0.11, "threshold": -2.076655626296997, "operator": "ge"}, {"budget": 0.12, "threshold": -2.335136651992798, "operator": "ge"}, {"budget": 0.13, "threshold": -2.5507495403289795, "operator": "ge"}, {"budget": 0.14, "threshold": -2.750570774078369, "operator": "ge"}, {"budget": 0.15, "threshold": -2.9617691040039062, "operator": "ge"}, {"budget": 0.16, "threshold": -3.1511268615722656, "operator": "ge"}, {"budget": 0.17, "threshold": -3.3383262157440186, "operator": "ge"}, {"budget": 0.18, "threshold": -3.492380380630493, "operator": "ge"}, {"budget": 0.19, "threshold": -3.642937660217285, "operator": "ge"}, {"budget": 0.2, "threshold": -3.777494430541992, "operator": "ge"}, {"budget": 0.21, "threshold": -3.938577175140381, "operator": "ge"}, {"budget": 0.22, "threshold": -4.091330051422119, "operator": "ge"}, {"budget": 0.23, "threshold": -4.2251129150390625, "operator": "ge"}, {"budget": 0.24, "threshold": -4.365009784698486, "operator": "ge"}, {"budget": 0.25, "threshold": -4.490166664123535, "operator": "ge"}, {"budget": 0.26, "threshold": -4.597294807434082, "operator": "ge"}, {"budget": 0.27, "threshold": -4.712502479553223, "operator": "ge"}, {"budget": 0.28, "threshold": -4.824742794036865, "operator": "ge"}, {"budget": 0.29, "threshold": -4.930624961853027, "operator": "ge"}, {"budget": 0.3, "threshold": -5.0331830978393555, "operator": "ge"}, {"budget": 0.31, "threshold": -5.119122505187988, "operator": "ge"}, {"budget": 0.32, "threshold": -5.217314720153809, "operator": "ge"}, {"budget": 0.33, "threshold": -5.2906060218811035, "operator": "ge"}, {"budget": 0.34, "threshold": -5.363044738769531, "operator": "ge"}, {"budget": 0.35, "threshold": -5.430039405822754, "operator": "ge"}, {"budget": 0.36, "threshold": -5.497452735900879, "operator": "ge"}, {"budget": 0.37, "threshold": -5.565081596374512, "operator": "ge"}, {"budget": 0.38, "threshold": -5.645102500915527, "operator": "ge"}, {"budget": 0.39, "threshold": -5.7045793533325195, "operator": "ge"}, {"budget": 0.4, "threshold": -5.756680488586426, "operator": "ge"}, {"budget": 0.41, "threshold": -5.811800003051758, "operator": "ge"}, {"budget": 0.42, "threshold": -5.862669467926025, "operator": "ge"}, {"budget": 0.43, "threshold": -5.915455341339111, "operator": "ge"}, {"budget": 0.44, "threshold": -5.970100402832031, "operator": "ge"}, {"budget": 0.45, "threshold": -6.020954132080078, "operator": "ge"}, {"budget": 0.46, "threshold": -6.070663928985596, "operator": "ge"}, {"budget": 0.47, "threshold": -6.120986461639404, "operator": "ge"}, {"budget": 0.48, "threshold": -6.171880722045898, "operator": "ge"}, {"budget": 0.49, "threshold": -6.216395378112793, "operator": "ge"}, {"budget": 0.5, "threshold": -6.263408660888672, "operator": "ge"}, {"budget": 0.51, "threshold": -6.309070587158203, "operator": "ge"}, {"budget": 0.52, "threshold": -6.360585689544678, "operator": "ge"}, {"budget": 0.53, "threshold": -6.4092512130737305, "operator": "ge"}, {"budget": 0.54, "threshold": -6.4575514793396, "operator": "ge"}, {"budget": 0.55, "threshold": -6.500258445739746, "operator": "ge"}, {"budget": 0.56, "threshold": -6.548063278198242, "operator": "ge"}, {"budget": 0.57, "threshold": -6.596400260925293, "operator": "ge"}, {"budget": 0.58, "threshold": -6.639191150665283, "operator": "ge"}, {"budget": 0.59, "threshold": -6.684575080871582, "operator": "ge"}, {"budget": 0.6, "threshold": -6.725885391235352, "operator": "ge"}, {"budget": 0.61, "threshold": -6.763058662414551, "operator": "ge"}, {"budget": 0.62, "threshold": -6.808574676513672, "operator": "ge"}, {"budget": 0.63, "threshold": -6.862550735473633, "operator": "ge"}, {"budget": 0.64, "threshold": -6.902648448944092, "operator": "ge"}, {"budget": 0.65, "threshold": -6.956545352935791, "operator": "ge"}, {"budget": 0.66, "threshold": -7.0050859451293945, "operator": "ge"}, {"budget": 0.67, "threshold": -7.049993515014648, "operator": "ge"}, {"budget": 0.68, "threshold": -7.1016154289245605, "operator": "ge"}, {"budget": 0.69, "threshold": -7.155762195587158, "operator": "ge"}, {"budget": 0.7, "threshold": -7.197110652923584, "operator": "ge"}, {"budget": 0.71, "threshold": -7.2473249435424805, "operator": "ge"}, {"budget": 0.72, "threshold": -7.290528297424316, "operator": "ge"}, {"budget": 0.73, "threshold": -7.345314025878906, "operator": "ge"}, {"budget": 0.74, "threshold": -7.399074554443359, "operator": "ge"}, {"budget": 0.75, "threshold": -7.446943759918213, "operator": "ge"}, {"budget": 0.76, "threshold": -7.503885746002197, "operator": "ge"}, {"budget": 0.77, "threshold": -7.55491304397583, "operator": "ge"}, {"budget": 0.78, "threshold": -7.607553482055664, "operator": "ge"}, {"budget": 0.79, "threshold": -7.66355562210083, "operator": "ge"}, {"budget": 0.8, "threshold": -7.720902442932129, "operator": "ge"}, {"budget": 0.81, "threshold": -7.784616470336914, "operator": "ge"}, {"budget": 0.82, "threshold": -7.831765174865723, "operator": "ge"}, {"budget": 0.83, "threshold": -7.898553371429443, "operator": "ge"}, {"budget": 0.84, "threshold": -7.9670796394348145, "operator": "ge"}, {"budget": 0.85, "threshold": -8.041617393493652, "operator": "ge"}, {"budget": 0.86, "threshold": -8.101239204406738, "operator": "ge"}, {"budget": 0.87, "threshold": -8.168102264404297, "operator": "ge"}, {"budget": 0.88, "threshold": -8.246735572814941, "operator": "ge"}, {"budget": 0.89, "threshold": -8.318323135375977, "operator": "ge"}, {"budget": 0.9, "threshold": -8.414779663085938, "operator": "ge"}, {"budget": 0.91, "threshold": -8.517500877380371, "operator": "ge"}, {"budget": 0.92, "threshold": -8.621607780456543, "operator": "ge"}, {"budget": 0.93, "threshold": -8.758171081542969, "operator": "ge"}, {"budget": 0.94, "threshold": -8.898602485656738, "operator": "ge"}, {"budget": 0.95, "threshold": -9.059383392333984, "operator": "ge"}, {"budget": 0.96, "threshold": -9.237964630126953, "operator": "ge"}, {"budget": 0.97, "threshold": -9.493114471435547, "operator": "ge"}, {"budget": 0.98, "threshold": -9.795079231262207, "operator": "ge"}, {"budget": 0.99, "threshold": -10.397958755493164, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.4585705249842, 75.80996556328626, 75.99269098320332, 76.12622109775809, 76.14730480005622, 76.23163960924872, 76.45653243376204, 76.70953686133953, 77.03984819734345, 77.24365731955865, 77.59505235786071, 77.90427999156651, 78.17836812144212, 78.4805678543819, 78.86007449574812, 79.05685571719727, 79.30283224400871, 79.47852976315974, 79.7455899922693, 79.9493991144845, 80.14618033593366, 80.34296155738281, 80.53271487806593, 80.72246819874903, 80.85599831330381, 81.03169583245484, 81.05980743551902, 81.15117014547755, 81.25658865696816, 81.39011877152295, 81.50256518377961, 81.64312319910043, 81.71340220676085, 81.69934640522875, 81.72745800829293, 81.74151380982501, 81.83990442054959, 81.87504392437978, 81.90315552744394, 81.97343453510436, 81.92423922974208, 81.95937873357228, 81.98749033663645, 81.91721132897604, 81.95235083280625, 81.95235083280625, 81.99451823740249, 82.00154613816854, 82.05776934429686, 82.02262984046665, 82.04371354276478, 82.02262984046665, 81.99451823740249, 81.95235083280625, 81.89612762667792, 81.88909972591188, 81.86801602361375, 81.8258486190175, 81.84693232131562, 81.83287651978354, 81.83990442054959, 81.80476491671938, 81.79773701595333, 81.80476491671938, 81.74854171059106, 81.72043010752688, 81.74151380982501, 81.72745800829293, 81.69934640522875, 81.66420690139856, 81.65015109986648, 81.65717900063252, 81.65717900063252, 81.69231850446272, 81.65015109986648, 81.60095579450419, 81.60798369527022, 81.5868999929721, 81.55878838990793, 81.52364888607772, 81.50256518377961, 81.41120247382108, 81.35497926769274, 81.3479513669267, 81.35497926769274, 81.31281186309648, 81.24253285543608, 81.20036545083984, 81.18630964930776, 81.16522594700963, 81.08791903858318, 81.05980743551902, 80.98250052709255, 80.89816571790006, 80.85599831330381, 80.80680300794153, 80.76463560334528, 80.75760770257924, 80.72949609951507, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 66, 125, 167, 202, 231, 261, 305, 354, 411, 450, 511, 567, 618, 673, 732, 775, 824, 867, 916, 960, 1008, 1049, 1085, 1117, 1151, 1188, 1214, 1239, 1263, 1289, 1312, 1340, 1357, 1370, 1386, 1397, 1415, 1427, 1443, 1464, 1473, 1494, 1505, 1510, 1522, 1532, 1543, 1557, 1573, 1582, 1594, 1606, 1616, 1622, 1628, 1637, 1644, 1651, 1663, 1671, 1679, 1683, 1691, 1701, 1707, 1717, 1727, 1733, 1737, 1741, 1750, 1758, 1767, 1777, 1783, 1787, 1795, 1800, 1806, 1813, 1819, 1822, 1829, 1835, 1843, 1847, 1850, 1858, 1866, 1875, 1878, 1886, 1892, 1897, 1908, 1920, 1932, 1944, 1956, 1964], "harmed": [0, 5, 14, 30, 46, 72, 90, 102, 115, 125, 135, 146, 158, 170, 182, 187, 202, 216, 234, 245, 260, 280, 293, 302, 307, 322, 334, 356, 368, 377, 384, 391, 399, 406, 421, 433, 442, 446, 453, 465, 476, 492, 508, 515, 530, 537, 547, 552, 565, 573, 587, 596, 611, 625, 637, 651, 661, 671, 684, 693, 703, 710, 719, 728, 737, 751, 765, 772, 780, 788, 797, 808, 815, 824, 829, 841, 852, 859, 867, 877, 889, 898, 914, 929, 936, 943, 953, 966, 980, 990, 1002, 1016, 1028, 1045, 1062, 1079, 1098, 1116, 1129, 1145, 1171], "exact_score": [75.02986857825567, 75.4585705249842, 75.80996556328626, 75.99269098320332, 76.12622109775809, 76.14730480005622, 76.23163960924872, 76.45653243376204, 76.70953686133953, 77.03984819734345, 77.24365731955865, 77.59505235786071, 77.90427999156651, 78.17836812144212, 78.4805678543819, 78.86007449574812, 79.05685571719727, 79.30283224400871, 79.47852976315974, 79.7455899922693, 79.9493991144845, 80.14618033593366, 80.34296155738281, 80.53271487806593, 80.72246819874903, 80.85599831330381, 81.03169583245484, 81.05980743551902, 81.15117014547755, 81.25658865696816, 81.39011877152295, 81.50256518377961, 81.64312319910043, 81.71340220676085, 81.69934640522875, 81.72745800829293, 81.74151380982501, 81.83990442054959, 81.87504392437978, 81.90315552744394, 81.97343453510436, 81.92423922974208, 81.95937873357228, 81.98749033663645, 81.91721132897604, 81.95235083280625, 81.95235083280625, 81.99451823740249, 82.00154613816854, 82.05776934429686, 82.02262984046665, 82.04371354276478, 82.02262984046665, 81.99451823740249, 81.95235083280625, 81.89612762667792, 81.88909972591188, 81.86801602361375, 81.8258486190175, 81.84693232131562, 81.83287651978354, 81.83990442054959, 81.80476491671938, 81.79773701595333, 81.80476491671938, 81.74854171059106, 81.72043010752688, 81.74151380982501, 81.72745800829293, 81.69934640522875, 81.66420690139856, 81.65015109986648, 81.65717900063252, 81.65717900063252, 81.69231850446272, 81.65015109986648, 81.60095579450419, 81.60798369527022, 81.5868999929721, 81.55878838990793, 81.52364888607772, 81.50256518377961, 81.41120247382108, 81.35497926769274, 81.3479513669267, 81.35497926769274, 81.31281186309648, 81.24253285543608, 81.20036545083984, 81.18630964930776, 81.16522594700963, 81.08791903858318, 81.05980743551902, 80.98250052709255, 80.89816571790006, 80.85599831330381, 80.80680300794153, 80.76463560334528, 80.75760770257924, 80.72949609951507, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 80.69407382998183, "threshold_area100": 80.69389978213508, "threshold_area25": 78.02867383512546}, "test": {"score": [66.95119530385956, 68.9164279840059, 69.66792388622636, 70.33150894705499, 70.73419731730142, 71.13404985395458, 71.43181238124947, 71.7607690780705, 72.12659161160423, 72.4101749709327, 72.76465417009331, 73.0255508606755, 73.3601792246831, 73.61256841448544, 73.86495760428777, 74.03227178629159, 74.2336159714148, 74.39242265263874, 74.54555766667612, 74.69302101352693, 74.81496185803816, 74.89436519865014, 75.04466437909423, 75.09287355018007, 75.17511272438533, 75.24033689703089, 75.28287440093015, 75.33959107279584, 75.35377024076227, 75.43600941496753, 75.43317358137425, 75.47287525168022, 75.51257692198622, 75.55511442588548, 75.59198026259818, 75.6089952641579, 75.62317443212432, 75.64302526727731, 75.66854776961688, 75.67421943680344, 75.67421943680344, 75.6827269375833, 75.69407027195643, 75.72810027507586, 75.73093610866914, 75.73660777585572, 75.739443609449, 75.75078694382213, 75.7961602813147, 75.80183194850126, 75.7961602813147, 75.78765278053483, 75.78481694694155, 75.7961602813147, 75.78481694694155, 75.81601111646769, 75.79048861412812, 75.7933244477214, 75.79048861412812, 75.76496611178855, 75.78481694694155, 75.77063777897513, 75.7564586110087, 75.75929444460199, 75.76496611178855, 75.77630944616169, 75.79048861412812, 75.77347361256841, 75.75078694382213, 75.73093610866914, 75.69974193914301, 75.6827269375833, 75.64586110087059, 75.65436860165046, 75.65436860165046, 75.67138360321016, 75.68839860476987, 75.67705527039674, 75.65153276805717, 75.63168193290417, 75.55511442588548, 75.51824858917279, 75.53809942432578, 75.50974108839293, 75.48421858605336, 75.4785469188668, 75.4586960837138, 75.43600941496753, 75.4019794118481, 75.3480985735757, 75.29421773530329, 75.21197856109804, 75.18645605875847, 75.15242605563905, 75.12973938689278, 75.13541105407936, 75.03048521112781, 74.99078354082182, 74.88585769787028, 74.67884184556051, 74.47182599325072], "calls": [0.0, 4.942857953095312, 6.533760598928056, 7.917647392451011, 9.239145846921701, 10.481240960780422, 11.570201060601763, 12.826475342426907, 14.105436292998327, 15.262456399058502, 16.46201400901795, 17.723959958029663, 18.954711737515243, 20.060686838896295, 21.192184442616906, 22.230099537759124, 23.262342965714772, 24.328616396789837, 25.19921730992825, 26.08116155743981, 26.940419136205087, 27.972662564160732, 29.0389359952358, 30.116552760684, 31.287752034710604, 32.27745795876698, 33.16223803987182, 34.14343646314834, 35.26075489890253, 36.292998326858175, 37.35927175793324, 38.18449933357911, 39.23942943028103, 40.041970337180615, 40.81048124096078, 41.56764881036781, 42.279443042282274, 43.13586478745427, 44.1539290474435, 44.86855911295125, 45.591696679238865, 46.37722258457874, 47.10319598445963, 47.792303547627824, 48.63171029124011, 49.40022119502028, 50.148881263647446, 50.874854663528346, 51.60366389700253, 52.23605478830502, 52.89963984913365, 53.55755324277571, 54.374273317641716, 55.14562005501518, 55.905623458015484, 56.61741768992995, 57.30368941950486, 58.04667782094547, 58.76981538723308, 59.456087116807986, 60.13101551200975, 60.6443013923943, 61.37311062586848, 62.184159033547914, 62.89878909905566, 63.82043501687321, 64.6229759237728, 65.31491932053427, 66.15999773133312, 67.02776281087826, 67.78776621387857, 68.55911295125202, 69.3446388565919, 70.28897144315572, 71.25882653205909, 71.96211326319371, 72.98301335677623, 73.68062842072428, 74.42361682216487, 75.26018773218388, 76.07974364064317, 77.01840456002041, 77.69333295522219, 78.5072171964949, 79.3806539432266, 80.34200153135014, 81.03110909451834, 81.91872500921646, 82.85171426140714, 83.59753849644103, 84.58440858690412, 85.69038368828518, 86.7453137849871, 87.93069222698013, 89.11607066897315, 90.30712077815274, 91.63996256699657, 93.19683520970989, 94.83027535944191, 97.25491308170037, 100.0], "rescued": [0, 699, 978, 1232, 1391, 1565, 1690, 1836, 1997, 2125, 2280, 2414, 2559, 2680, 2800, 2891, 2995, 3089, 3165, 3250, 3324, 3391, 3474, 3537, 3602, 3657, 3700, 3756, 3796, 3852, 3888, 3929, 3982, 4020, 4050, 4075, 4102, 4136, 4175, 4199, 4227, 4256, 4278, 4308, 4335, 4356, 4375, 4392, 4424, 4444, 4460, 4474, 4496, 4523, 4537, 4565, 4581, 4602, 4624, 4639, 4662, 4667, 4685, 4709, 4730, 4755, 4777, 4797, 4816, 4834, 4843, 4862, 4868, 4897, 4923, 4941, 4971, 4981, 4996, 5015, 5030, 5046, 5065, 5075, 5093, 5116, 5129, 5146, 5158, 5171, 5190, 5199, 5219, 5242, 5269, 5295, 5313, 5338, 5359, 5381, 5423], "harmed": [0, 6, 20, 40, 57, 90, 110, 140, 172, 200, 230, 272, 299, 331, 362, 394, 427, 465, 487, 520, 551, 590, 620, 666, 702, 734, 762, 798, 833, 860, 897, 924, 963, 986, 1003, 1022, 1044, 1071, 1101, 1123, 1151, 1177, 1195, 1213, 1239, 1258, 1276, 1289, 1305, 1323, 1341, 1358, 1381, 1404, 1422, 1439, 1464, 1484, 1507, 1531, 1547, 1557, 1580, 1603, 1622, 1643, 1660, 1686, 1713, 1738, 1758, 1783, 1802, 1828, 1854, 1866, 1890, 1904, 1928, 1954, 1996, 2025, 2037, 2057, 2084, 2109, 2129, 2154, 2178, 2210, 2248, 2286, 2315, 2350, 2385, 2409, 2464, 2503, 2561, 2656, 2771], "exact_score": [66.95119530385956, 67.16104698976264, 67.46448118424411, 67.8756770552704, 68.3776196012818, 68.95045798712532, 69.39284802767773, 69.90613390806227, 70.3655389501744, 70.66613731106258, 70.9809148399172, 71.28718486799195, 71.55091739216742, 71.8174857499362, 72.0925616084848, 72.34495079828716, 72.67390749510818, 72.8582366786717, 73.11346170206731, 73.37152255905623, 73.59271757933244, 73.81107676601538, 74.02376428551173, 74.17406346595583, 74.34137764795962, 74.52570683152312, 74.6816776791538, 74.82063352522474, 74.89436519865014, 75.03899271190765, 75.0872018829935, 75.16944105719875, 75.22615772906445, 75.26018773218387, 75.33108357201598, 75.35093440716899, 75.4019794118481, 75.43884524856081, 75.46153191730708, 75.5324277571392, 75.55511442588548, 75.60332359697132, 75.61750276493775, 75.63168193290417, 75.66854776961688, 75.67421943680344, 75.67138360321016, 75.68556277117658, 75.73660777585572, 75.73377194226242, 75.74227944304228, 75.76213027819527, 75.79048861412812, 75.7961602813147, 75.78481694694155, 75.80466778209455, 75.7933244477214, 75.7961602813147, 75.7933244477214, 75.78765278053483, 75.77347361256841, 75.75362277741542, 75.7564586110087, 75.76780194538185, 75.78198111334827, 75.79048861412812, 75.74227944304228, 75.73377194226242, 75.69690610554971, 75.6628761024303, 75.64869693446389, 75.65436860165046, 75.67421943680344, 75.69123443836315, 75.6657119360236, 75.63451776649745, 75.56362192666535, 75.51824858917279, 75.52959192354592, 75.48705441964665, 75.47571108527352, 75.4586960837138, 75.43317358137425, 75.39630774466154, 75.31406857045629, 75.26585939937044, 75.22899356265775, 75.19496355953832, 75.15809772282563, 75.13257522048606, 75.14108272126592, 75.09003771658679, 75.02197771034795, 75.00212687519496, 74.94824603692255, 74.87167852990386, 74.7894393556986, 74.70720018149335, 74.64764767603437, 74.58809517057539, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.39958726803569, "threshold_area100": 74.90188015767237, "threshold_area25": 72.76834075376458}, "selected_budget": 49, "selected": {"val": {"score": 82.05776934429686, "calls": 48.99852414083913, "rescued": 1573, "harmed": 573}, "test": {"score": 75.80183194850126, "calls": 52.23605478830502, "rescued": 4444, "harmed": 1323}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8674151301383972, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5201246738433838, "operator": "ge"}, {"budget": 0.03, "threshold": 0.27304816246032715, "operator": "ge"}, {"budget": 0.04, "threshold": 0.09995746612548828, "operator": "ge"}, {"budget": 0.05, "threshold": -0.013898372650146484, "operator": "ge"}, {"budget": 0.06, "threshold": -0.1350923478603363, "operator": "ge"}, {"budget": 0.07, "threshold": -0.28194460272789, "operator": "ge"}, {"budget": 0.08, "threshold": -0.4605664610862732, "operator": "ge"}, {"budget": 0.09, "threshold": -0.624310314655304, "operator": "ge"}, {"budget": 0.1, "threshold": -0.822516918182373, "operator": "ge"}, {"budget": 0.11, "threshold": -0.9962908029556274, "operator": "ge"}, {"budget": 0.12, "threshold": -1.1659047603607178, "operator": "ge"}, {"budget": 0.13, "threshold": -1.3548681735992432, "operator": "ge"}, {"budget": 0.14, "threshold": -1.547945261001587, "operator": "ge"}, {"budget": 0.15, "threshold": -1.7369102239608765, "operator": "ge"}, {"budget": 0.16, "threshold": -1.9604287147521973, "operator": "ge"}, {"budget": 0.17, "threshold": -2.158186674118042, "operator": "ge"}, {"budget": 0.18, "threshold": -2.3333683013916016, "operator": "ge"}, {"budget": 0.19, "threshold": -2.5052833557128906, "operator": "ge"}, {"budget": 0.2, "threshold": -2.662112236022949, "operator": "ge"}, {"budget": 0.21, "threshold": -2.8475568294525146, "operator": "ge"}, {"budget": 0.22, "threshold": -3.002737522125244, "operator": "ge"}, {"budget": 0.23, "threshold": -3.172483444213867, "operator": "ge"}, {"budget": 0.24, "threshold": -3.314655303955078, "operator": "ge"}, {"budget": 0.25, "threshold": -3.4436428546905518, "operator": "ge"}, {"budget": 0.26, "threshold": -3.5827879905700684, "operator": "ge"}, {"budget": 0.27, "threshold": -3.7175285816192627, "operator": "ge"}, {"budget": 0.28, "threshold": -3.8505825996398926, "operator": "ge"}, {"budget": 0.29, "threshold": -3.974482536315918, "operator": "ge"}, {"budget": 0.3, "threshold": -4.098580360412598, "operator": "ge"}, {"budget": 0.31, "threshold": -4.210565090179443, "operator": "ge"}, {"budget": 0.32, "threshold": -4.318823337554932, "operator": "ge"}, {"budget": 0.33, "threshold": -4.42799711227417, "operator": "ge"}, {"budget": 0.34, "threshold": -4.527631759643555, "operator": "ge"}, {"budget": 0.35, "threshold": -4.635447025299072, "operator": "ge"}, {"budget": 0.36, "threshold": -4.719259262084961, "operator": "ge"}, {"budget": 0.37, "threshold": -4.822091102600098, "operator": "ge"}, {"budget": 0.38, "threshold": -4.91916036605835, "operator": "ge"}, {"budget": 0.39, "threshold": -5.004803657531738, "operator": "ge"}, {"budget": 0.4, "threshold": -5.079963207244873, "operator": "ge"}, {"budget": 0.41, "threshold": -5.149703502655029, "operator": "ge"}, {"budget": 0.42, "threshold": -5.217712879180908, "operator": "ge"}, {"budget": 0.43, "threshold": -5.285669326782227, "operator": "ge"}, {"budget": 0.44, "threshold": -5.356995105743408, "operator": "ge"}, {"budget": 0.45, "threshold": -5.417693614959717, "operator": "ge"}, {"budget": 0.46, "threshold": -5.481529712677002, "operator": "ge"}, {"budget": 0.47, "threshold": -5.546131610870361, "operator": "ge"}, {"budget": 0.48, "threshold": -5.602685928344727, "operator": "ge"}, {"budget": 0.49, "threshold": -5.654850006103516, "operator": "ge"}, {"budget": 0.5, "threshold": -5.709723949432373, "operator": "ge"}, {"budget": 0.51, "threshold": -5.762299060821533, "operator": "ge"}, {"budget": 0.52, "threshold": -5.81566047668457, "operator": "ge"}, {"budget": 0.53, "threshold": -5.866298198699951, "operator": "ge"}, {"budget": 0.54, "threshold": -5.9230451583862305, "operator": "ge"}, {"budget": 0.55, "threshold": -5.9763078689575195, "operator": "ge"}, {"budget": 0.56, "threshold": -6.02880859375, "operator": "ge"}, {"budget": 0.57, "threshold": -6.077649116516113, "operator": "ge"}, {"budget": 0.58, "threshold": -6.12150239944458, "operator": "ge"}, {"budget": 0.59, "threshold": -6.176313400268555, "operator": "ge"}, {"budget": 0.6, "threshold": -6.223060131072998, "operator": "ge"}, {"budget": 0.61, "threshold": -6.277067184448242, "operator": "ge"}, {"budget": 0.62, "threshold": -6.3267927169799805, "operator": "ge"}, {"budget": 0.63, "threshold": -6.375457763671875, "operator": "gt"}, {"budget": 0.64, "threshold": -6.422411918640137, "operator": "ge"}, {"budget": 0.65, "threshold": -6.468991279602051, "operator": "ge"}, {"budget": 0.66, "threshold": -6.5159912109375, "operator": "ge"}, {"budget": 0.67, "threshold": -6.566991329193115, "operator": "ge"}, {"budget": 0.68, "threshold": -6.616264343261719, "operator": "ge"}, {"budget": 0.69, "threshold": -6.663776874542236, "operator": "ge"}, {"budget": 0.7, "threshold": -6.7202653884887695, "operator": "ge"}, {"budget": 0.71, "threshold": -6.774552822113037, "operator": "ge"}, {"budget": 0.72, "threshold": -6.82999849319458, "operator": "ge"}, {"budget": 0.73, "threshold": -6.888731956481934, "operator": "ge"}, {"budget": 0.74, "threshold": -6.94586706161499, "operator": "ge"}, {"budget": 0.75, "threshold": -6.998580455780029, "operator": "ge"}, {"budget": 0.76, "threshold": -7.055685520172119, "operator": "ge"}, {"budget": 0.77, "threshold": -7.108395576477051, "operator": "ge"}, {"budget": 0.78, "threshold": -7.161933898925781, "operator": "ge"}, {"budget": 0.79, "threshold": -7.221984386444092, "operator": "ge"}, {"budget": 0.8, "threshold": -7.277372360229492, "operator": "ge"}, {"budget": 0.81, "threshold": -7.332868576049805, "operator": "ge"}, {"budget": 0.82, "threshold": -7.388655662536621, "operator": "ge"}, {"budget": 0.83, "threshold": -7.450983047485352, "operator": "ge"}, {"budget": 0.84, "threshold": -7.514261245727539, "operator": "ge"}, {"budget": 0.85, "threshold": -7.586353778839111, "operator": "ge"}, {"budget": 0.86, "threshold": -7.665974140167236, "operator": "ge"}, {"budget": 0.87, "threshold": -7.740190505981445, "operator": "ge"}, {"budget": 0.88, "threshold": -7.82462215423584, "operator": "ge"}, {"budget": 0.89, "threshold": -7.893333911895752, "operator": "ge"}, {"budget": 0.9, "threshold": -7.971500873565674, "operator": "ge"}, {"budget": 0.91, "threshold": -8.061151504516602, "operator": "ge"}, {"budget": 0.92, "threshold": -8.167433738708496, "operator": "ge"}, {"budget": 0.93, "threshold": -8.280590057373047, "operator": "ge"}, {"budget": 0.94, "threshold": -8.431100845336914, "operator": "ge"}, {"budget": 0.95, "threshold": -8.5877685546875, "operator": "ge"}, {"budget": 0.96, "threshold": -8.784993171691895, "operator": "ge"}, {"budget": 0.97, "threshold": -9.014664649963379, "operator": "ge"}, {"budget": 0.98, "threshold": -9.314933776855469, "operator": "ge"}, {"budget": 0.99, "threshold": -9.806072235107422, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.02986857825567, 75.43045892192002, 75.60615644107105, 75.86618876941458, 75.96457938013916, 76.1613606015883, 76.26677911307893, 76.35111392227142, 76.67439735750932, 77.0468760981095, 77.3209642279851, 77.644247663223, 77.91130789233256, 78.32595403752899, 78.57895846510647, 78.88115819804624, 79.16930212945394, 79.49258556469184, 79.7104504884391, 80.05481762597513, 80.34296155738281, 80.67327289338675, 80.94033312249631, 81.17928174854171, 81.35497926769274, 81.47445358071545, 81.59392789373814, 81.79773701595333, 82.00857403893457, 82.14210415348934, 82.3037458711083, 82.43024808489704, 82.5075549933235, 82.52863869562161, 82.58486190174995, 82.63405720711224, 82.69028041324057, 82.80975472626326, 82.90111743622181, 82.9854522454143, 82.99950804694637, 82.9854522454143, 82.94328484081805, 82.94328484081805, 82.99950804694637, 83.00653594771242, 82.99248014618034, 82.99248014618034, 82.9854522454143, 82.97842434464826, 82.95031274158409, 82.9854522454143, 83.00653594771242, 82.97842434464826, 82.95734064235013, 82.97139644388221, 82.94328484081805, 82.92220113851992, 82.80975472626326, 82.76758732166702, 82.73244781783681, 82.69028041324057, 82.60594560404807, 82.57080610021787, 82.56377819945182, 82.49349919179141, 82.47241548949329, 82.45133178719517, 82.43024808489704, 82.39510858106684, 82.35294117647058, 82.25455056574602, 82.20535526038373, 82.19832735961768, 82.17724365731955, 82.15615995502144, 82.14210415348934, 82.11399255042518, 82.0647972450629, 81.98749033663645, 81.93829503127415, 81.87504392437978, 81.81179281748541, 81.76259751212312, 81.78368121442125, 81.69231850446272, 81.60798369527022, 81.54473258837585, 81.45336987841732, 81.39011877152295, 81.36200716845879, 81.3479513669267, 81.31281186309648, 81.24956075620211, 81.15819804624358, 81.05980743551902, 80.97547262632652, 80.91924942019818, 80.84897041253778, 80.77166350411133, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99107456602713, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "rescued": [0, 66, 107, 154, 188, 232, 261, 291, 346, 403, 454, 509, 558, 624, 676, 731, 785, 846, 892, 953, 1003, 1062, 1107, 1151, 1194, 1232, 1265, 1307, 1349, 1378, 1413, 1444, 1466, 1487, 1506, 1523, 1547, 1572, 1595, 1619, 1631, 1645, 1654, 1664, 1680, 1696, 1707, 1719, 1730, 1740, 1749, 1761, 1772, 1777, 1781, 1790, 1797, 1800, 1806, 1812, 1819, 1825, 1828, 1834, 1841, 1844, 1849, 1856, 1861, 1865, 1873, 1874, 1877, 1884, 1888, 1893, 1898, 1902, 1903, 1905, 1907, 1908, 1911, 1912, 1919, 1921, 1923, 1927, 1928, 1932, 1934, 1941, 1943, 1946, 1948, 1951, 1955, 1958, 1961, 1964, 1964], "harmed": [0, 9, 25, 35, 55, 71, 85, 103, 112, 116, 128, 137, 148, 155, 171, 183, 196, 211, 226, 238, 247, 259, 266, 276, 294, 315, 331, 344, 356, 366, 378, 391, 402, 420, 431, 441, 457, 465, 475, 487, 497, 513, 528, 538, 546, 561, 574, 586, 598, 609, 622, 629, 637, 646, 653, 660, 671, 677, 699, 711, 723, 735, 750, 761, 769, 782, 790, 800, 808, 817, 831, 846, 856, 864, 871, 879, 886, 894, 902, 915, 924, 934, 946, 954, 958, 973, 987, 1000, 1014, 1027, 1033, 1042, 1049, 1061, 1076, 1093, 1109, 1120, 1133, 1147, 1171], "exact_score": [75.02986857825567, 75.43045892192002, 75.60615644107105, 75.86618876941458, 75.96457938013916, 76.1613606015883, 76.26677911307893, 76.35111392227142, 76.67439735750932, 77.0468760981095, 77.3209642279851, 77.644247663223, 77.91130789233256, 78.32595403752899, 78.57895846510647, 78.88115819804624, 79.16930212945394, 79.49258556469184, 79.7104504884391, 80.05481762597513, 80.34296155738281, 80.67327289338675, 80.94033312249631, 81.17928174854171, 81.35497926769274, 81.47445358071545, 81.59392789373814, 81.79773701595333, 82.00857403893457, 82.14210415348934, 82.3037458711083, 82.43024808489704, 82.5075549933235, 82.52863869562161, 82.58486190174995, 82.63405720711224, 82.69028041324057, 82.80975472626326, 82.90111743622181, 82.9854522454143, 82.99950804694637, 82.9854522454143, 82.94328484081805, 82.94328484081805, 82.99950804694637, 83.00653594771242, 82.99248014618034, 82.99248014618034, 82.9854522454143, 82.97842434464826, 82.95031274158409, 82.9854522454143, 83.00653594771242, 82.97842434464826, 82.95734064235013, 82.97139644388221, 82.94328484081805, 82.92220113851992, 82.80975472626326, 82.76758732166702, 82.73244781783681, 82.69028041324057, 82.60594560404807, 82.57080610021787, 82.56377819945182, 82.49349919179141, 82.47241548949329, 82.45133178719517, 82.43024808489704, 82.39510858106684, 82.35294117647058, 82.25455056574602, 82.20535526038373, 82.19832735961768, 82.17724365731955, 82.15615995502144, 82.14210415348934, 82.11399255042518, 82.0647972450629, 81.98749033663645, 81.93829503127415, 81.87504392437978, 81.81179281748541, 81.76259751212312, 81.78368121442125, 81.69231850446272, 81.60798369527022, 81.54473258837585, 81.45336987841732, 81.39011877152295, 81.36200716845879, 81.3479513669267, 81.31281186309648, 81.24956075620211, 81.15819804624358, 81.05980743551902, 80.97547262632652, 80.91924942019818, 80.84897041253778, 80.77166350411133, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_area100": 81.242186374841, "threshold_area100": 81.24200576287862, "threshold_area25": 78.19172113289761}, "test": {"score": [66.95119530385956, 68.14508124663244, 69.0100104925843, 69.84374556901001, 70.57255480248419, 71.13972152114114, 71.42897654765619, 71.71255990698465, 72.16062161472364, 72.47256330998498, 72.80435584039928, 73.10495420128747, 73.40838839576892, 73.67779258713098, 73.89615177381391, 74.1031676261237, 74.33003431358648, 74.50018432918355, 74.74406601820604, 74.84048436037773, 74.91705186739642, 75.02764937753452, 75.1439185548592, 75.311232736863, 75.4019794118481, 75.47571108527352, 75.54093525791906, 75.61183109775118, 75.68556277117658, 75.79048861412812, 75.84153361880725, 75.90675779145279, 75.94929529535207, 75.98899696565805, 76.04004197033719, 76.01451946799762, 76.04571363752375, 76.03437030315061, 76.04287780393047, 76.11377364376258, 76.1109378101693, 76.127952811729, 76.14496781328872, 76.13362447891558, 76.12511697813572, 76.16765448203499, 76.15914698125515, 76.15347531406856, 76.17332614922157, 76.19034115078126, 76.19317698437456, 76.19317698437456, 76.19034115078126, 76.19601281796784, 76.19601281796784, 76.2045203187477, 76.21586365312083, 76.2215353203074, 76.21869948671412, 76.1846694835947, 76.19884865156112, 76.19034115078126, 76.17899781640813, 76.15914698125515, 76.14496781328872, 76.11944531094915, 76.08825114142302, 76.08825114142302, 76.0740719734566, 76.04287780393047, 76.00601196721776, 75.98899696565805, 75.91810112582593, 75.88123528911323, 75.86422028755354, 75.83019028443411, 75.79899611490798, 75.79048861412812, 75.76496611178855, 75.74227944304228, 75.719592774296, 75.69974193914301, 75.69123443836315, 75.67138360321016, 75.63735360009073, 75.6089952641579, 75.55511442588548, 75.5324277571392, 75.47287525168022, 75.4416810821541, 75.41899441340783, 75.39347191106826, 75.32541190482942, 75.28854606811673, 75.1977993931316, 75.15242605563905, 75.08153021580694, 74.9964552080084, 74.89720103224342, 74.73839435101948, 74.47182599325072], "calls": [0.0, 2.143890196523268, 4.001361200124777, 6.366446416924256, 8.65212829311176, 10.501091795933416, 11.547514391855486, 12.673340328389529, 14.145137963304313, 15.313501403737629, 16.79097070583898, 18.15784249780223, 19.226951762470577, 20.55979355131441, 21.782037830020133, 22.95607293764002, 24.456228908487653, 25.706831523126223, 26.818478291693843, 27.947140061821173, 28.916995150724556, 30.218642770042255, 31.367155375322575, 32.657459660267136, 33.80313643195417, 34.8949323653688, 36.09165414173496, 37.23449507982872, 38.43405268978816, 39.58823696225505, 40.86152624563991, 42.09227802512549, 43.35705980773048, 44.71542409891388, 45.96886254714573, 47.21379349459774, 48.32544026316536, 49.584550378583785, 50.85216799478207, 52.00351643365567, 53.04993902957774, 54.0623316223804, 54.98114170660465, 55.91696679238862, 56.93786688597113, 57.941751977993924, 58.77548705441965, 59.844596319088, 60.69251056348014, 61.554603975838695, 62.43087655616368, 63.27311913336925, 64.07566004026883, 64.88387261435498, 65.74880186030684, 66.44925275784817, 67.23477866318804, 67.97493123103536, 68.59030712077815, 69.38434052689789, 70.08479142443922, 70.99509400788362, 71.70688823979809, 72.393159969373, 73.05390919660834, 73.76570342852281, 74.47749766043728, 75.17227689079205, 75.84436945240053, 76.51079034682245, 77.23676374670335, 77.99393131611036, 78.7312480503644, 79.5309531236707, 80.2200606868389, 80.87230241329438, 81.63797748348128, 82.31290587868304, 82.98499844029152, 83.66276266908658, 84.2554518900831, 84.89067861497888, 85.42948699770298, 86.05904205541219, 86.65456711000199, 87.30680883645748, 88.06397640586451, 88.69069562998042, 89.42801236423446, 89.94980574539886, 90.54816663358194, 91.206080027224, 91.8951875903922, 92.60414598871338, 93.48325440263166, 94.28579530953124, 95.03445537815841, 95.90222045770355, 96.92312055128605, 98.25879817372316, 100.0], "rescued": [0, 435, 752, 1071, 1348, 1567, 1689, 1817, 2002, 2151, 2311, 2454, 2590, 2728, 2841, 2952, 3084, 3180, 3297, 3371, 3435, 3525, 3602, 3688, 3755, 3817, 3876, 3941, 4002, 4070, 4123, 4178, 4229, 4276, 4325, 4356, 4395, 4428, 4471, 4520, 4552, 4588, 4613, 4641, 4662, 4697, 4716, 4736, 4764, 4788, 4814, 4836, 4861, 4883, 4903, 4921, 4937, 4954, 4967, 4984, 5003, 5021, 5033, 5043, 5057, 5072, 5086, 5104, 5113, 5124, 5131, 5143, 5147, 5160, 5172, 5179, 5191, 5202, 5215, 5221, 5229, 5240, 5251, 5267, 5277, 5288, 5300, 5310, 5315, 5320, 5326, 5337, 5343, 5351, 5360, 5371, 5378, 5387, 5400, 5411, 5423], "harmed": [0, 14, 26, 51, 71, 90, 110, 138, 165, 204, 247, 284, 313, 356, 392, 430, 482, 518, 549, 589, 626, 677, 713, 740, 775, 811, 847, 887, 922, 953, 988, 1020, 1056, 1089, 1120, 1160, 1188, 1225, 1265, 1289, 1322, 1352, 1371, 1403, 1427, 1447, 1469, 1491, 1512, 1530, 1555, 1577, 1603, 1623, 1643, 1658, 1670, 1685, 1699, 1728, 1742, 1763, 1779, 1796, 1815, 1839, 1864, 1882, 1896, 1918, 1938, 1956, 1985, 2011, 2029, 2048, 2071, 2085, 2107, 2121, 2137, 2155, 2169, 2192, 2214, 2235, 2266, 2284, 2310, 2326, 2340, 2360, 2390, 2411, 2452, 2479, 2511, 2550, 2598, 2665, 2771], "exact_score": [66.95119530385956, 67.60910869750163, 68.08552874117346, 68.58179961999829, 69.0100104925843, 69.40135552845759, 69.71613305731219, 70.05359725491309, 70.35986728298784, 70.68315231262228, 70.96673567195076, 71.29852820236509, 71.56226072654056, 71.78912741400335, 72.12091994441766, 72.37330913422, 72.63987749198877, 72.86390834585826, 73.08510336613448, 73.3403283895301, 73.5643592433996, 73.74585259336982, 73.92734594334004, 74.1031676261237, 74.23645180500809, 74.4179451549783, 74.56824433542239, 74.74973768539262, 74.83764852678445, 74.92555936817628, 74.99929104160168, 75.0872018829935, 75.21197856109804, 75.33391940560928, 75.41899441340783, 75.49839775401979, 75.5352635907325, 75.59765192978476, 75.6657119360236, 75.71108527351615, 75.79899611490798, 75.84720528599381, 75.90959362504609, 75.95780279613193, 75.95213112894535, 76.00034030003118, 76.04287780393047, 76.02586280237075, 76.03437030315061, 76.0542211383036, 76.02586280237075, 76.04854947111703, 76.11377364376258, 76.11377364376258, 76.11377364376258, 76.15063948047528, 76.13362447891558, 76.12228114454244, 76.17049031562829, 76.15631114766185, 76.15631114766185, 76.18750531718798, 76.21019198593426, 76.2016844851544, 76.19034115078126, 76.2016844851544, 76.19884865156112, 76.21302781952754, 76.2215353203074, 76.20735615234098, 76.19317698437456, 76.19034115078126, 76.17616198281485, 76.147803646882, 76.11660947735587, 76.10526614298273, 76.07974364064316, 76.00884780081105, 75.98899696565805, 75.90959362504609, 75.86422028755354, 75.8131752828744, 75.78765278053483, 75.76496611178855, 75.73377194226242, 75.69407027195643, 75.67705527039674, 75.62884609931089, 75.55795025947877, 75.50406942120637, 75.4416810821541, 75.41048691262796, 75.31974023764285, 75.23750106343759, 75.17227689079205, 75.08153021580694, 74.99078354082182, 74.89720103224342, 74.79227518929189, 74.66182684400079, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "exact_area100": 74.65106852143816, "threshold_area100": 75.11383036043445, "threshold_area25": 72.83555000992543}, "selected_budget": 45, "selected": {"val": {"score": 83.00653594771242, "calls": 44.9996486049617, "rescued": 1696, "harmed": 561}, "test": {"score": 76.16765448203499, "calls": 57.941751977993924, "rescued": 4697, "harmed": 1447}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 80.91222151943215, "calls": 95.99409656335652}, "test": {"score": 74.7327226838329, "calls": 96.41834217168137}}, "old_normalized_interior": {"val": {"score": 82.20535526038373, "calls": 62.99810246679317}, "test": {"score": 75.77630944616169, "calls": 80.11513484388736}}, "old_normalized_endpoints": {"val": {"score": 82.20535526038373, "calls": 62.99810246679317}, "test": {"score": 75.77630944616169, "calls": 80.11513484388736}}}, "baseline_old_normalized": {"val": {"score": [75.02986857825567, 75.35315201349357, 75.79590976175416, 76.16838850235435, 76.45653243376204, 76.56195094525266, 76.58303464755078, 76.71656476210555, 76.92740178508679, 77.27879682338884, 77.55288495326445, 77.87616838850235, 78.09403331224964, 78.38217724365732, 78.61409796893668, 78.81790709115187, 79.01468831260102, 79.14119052638978, 79.26066483941247, 79.33094384707287, 79.51366926698995, 79.61205987771453, 79.7455899922693, 79.90020380912222, 80.00562232061283, 80.19537564129595, 80.3218778550847, 80.46243587040551, 80.52568697729988, 80.5819101834282, 80.61002178649238, 80.72246819874903, 80.76463560334528, 80.8208588094736, 80.93330522173027, 81.01061213015672, 81.05980743551902, 81.13711434394546, 81.1441422447115, 81.23550495467005, 81.21442125237192, 81.29875606156442, 81.39714667228898, 81.43228617611919, 81.4814814814815, 81.41120247382108, 81.39011877152295, 81.41823037458711, 81.47445358071545, 81.4674256799494, 81.41120247382108, 81.49553728301356, 81.53067678684377, 81.65015109986648, 81.67123480216459, 81.76962541288917, 81.69934640522875, 81.72745800829293, 81.77665331365522, 81.85396022208167, 82.01560193970062, 82.0296577412327, 82.12102045119123, 82.20535526038373, 82.20535526038373, 82.16318785578748, 82.15615995502144, 82.11399255042518, 82.0647972450629, 82.03668564199873, 82.01560193970062, 82.00154613816854, 81.9453229320402, 81.88909972591188, 81.83287651978354, 81.81179281748541, 81.78368121442125, 81.74854171059106, 81.67826270293064, 81.65015109986648, 81.57284419144003, 81.53067678684377, 81.50959308454566, 81.47445358071545, 81.44634197765129, 81.41823037458711, 81.3830908707569, 81.3479513669267, 81.28470026003232, 81.26361655773421, 81.24956075620211, 81.19333755007379, 81.15819804624358, 81.08089113781713, 81.05980743551902, 81.01061213015672, 80.93330522173027, 80.8700541148359, 80.77166350411133, 80.67327289338675, 80.60299388572633], "calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.99262070419566, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99037177595053, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "exact_score": [75.02986857825567, 75.35315201349357, 75.79590976175416, 76.16838850235435, 76.45653243376204, 76.56195094525266, 76.58303464755078, 76.71656476210555, 76.92740178508679, 77.27879682338884, 77.55288495326445, 77.87616838850235, 78.09403331224964, 78.38217724365732, 78.61409796893668, 78.81790709115187, 79.01468831260102, 79.14119052638978, 79.26066483941247, 79.33094384707287, 79.51366926698995, 79.61205987771453, 79.7455899922693, 79.90020380912222, 80.00562232061283, 80.19537564129595, 80.3218778550847, 80.46243587040551, 80.52568697729988, 80.5819101834282, 80.61002178649238, 80.72246819874903, 80.76463560334528, 80.8208588094736, 80.93330522173027, 81.01061213015672, 81.05980743551902, 81.13711434394546, 81.1441422447115, 81.23550495467005, 81.21442125237192, 81.29875606156442, 81.39714667228898, 81.43228617611919, 81.4814814814815, 81.41120247382108, 81.39011877152295, 81.41823037458711, 81.47445358071545, 81.4674256799494, 81.41120247382108, 81.49553728301356, 81.53067678684377, 81.65015109986648, 81.67123480216459, 81.76962541288917, 81.69934640522875, 81.72745800829293, 81.77665331365522, 81.85396022208167, 82.01560193970062, 82.0296577412327, 82.12102045119123, 82.20535526038373, 82.20535526038373, 82.16318785578748, 82.15615995502144, 82.11399255042518, 82.0647972450629, 82.03668564199873, 82.01560193970062, 82.00154613816854, 81.9453229320402, 81.88909972591188, 81.83287651978354, 81.81179281748541, 81.78368121442125, 81.74854171059106, 81.67826270293064, 81.65015109986648, 81.57284419144003, 81.53067678684377, 81.50959308454566, 81.47445358071545, 81.44634197765129, 81.41823037458711, 81.3830908707569, 81.3479513669267, 81.28470026003232, 81.26361655773421, 81.24956075620211, 81.19333755007379, 81.15819804624358, 81.08089113781713, 81.05980743551902, 81.01061213015672, 80.93330522173027, 80.8700541148359, 80.77166350411133, 80.67327289338675, 80.60299388572633], "exact_calls": [0.0, 0.9979619087778481, 1.9959238175556961, 2.993885726333544, 3.998875535877434, 4.996837444655281, 5.994799353433129, 6.999789162977018, 7.997751071754868, 8.995712980532716, 9.993674889310562, 10.998664698854451, 11.996626607632301, 12.994588516410149, 13.999578325954037, 14.997540234731884, 15.995502143509736, 16.99346405228758, 17.99845386183147, 18.996415770609318, 19.994377679387167, 20.999367488931057, 21.997329397708903, 22.995291306486752, 23.993253215264602, 24.998243024808488, 25.996204933586338, 26.994166842364187, 27.999156651908073, 28.997118560685927, 29.99508046946377, 30.99304237824162, 31.99803218778551, 32.99599409656336, 33.993956005341204, 34.9989458148851, 35.99690772366294, 36.99486963244079, 37.99985944198468, 38.99782135076253, 39.995783259540374, 40.99374516831823, 41.998734977862114, 42.99669688663996, 43.994658795417806, 44.9996486049617, 45.997610513739545, 46.9955724225174, 47.99353433129524, 48.99852414083913, 49.996486049616976, 50.99444795839483, 51.999437767938716, 52.99739967671656, 53.995361585494415, 54.99332349427226, 55.99831330381615, 56.99627521259399, 57.99423712137185, 58.99922693091574, 59.997188839693585, 60.995150748471424, 61.993112657249284, 62.99810246679317, 63.99606437557102, 64.99402628434886, 65.99901609389276, 66.9969780026706, 67.99493991144845, 68.99992972099234, 69.9978916297702, 70.99585353854803, 71.99381544732589, 72.99880525686977, 73.99676716564763, 74.99472907442546, 75.99971888396936, 76.9976807927472, 77.99564270152506, 78.99360461030291, 79.9985944198468, 80.99655632862464, 81.99451823740249, 82.99950804694637, 83.99746995572423, 84.99543186450207, 85.99339377327992, 86.9983835828238, 87.99634549160166, 88.9943074003795, 89.9992972099234, 90.99725911870125, 91.99522102747909, 92.99318293625693, 93.99817274580083, 94.99613465457868, 95.99409656335652, 96.99908637290041, 97.99704828167826, 98.99501019045611, 100.0], "threshold_area25": 78.01180687328694, "exact_area100": 80.56337828725194}, "test": {"score": [66.95119530385956, 67.7991095482517, 68.86254714573349, 69.93449224399512, 70.96389983835748, 71.32972237189121, 71.63882823355925, 71.8912174233616, 72.21166661940276, 72.61151915605592, 72.92913251850382, 73.2864475512577, 73.53883674106004, 73.81674843320194, 74.01809261832514, 74.18540680032896, 74.3243626463999, 74.49734849559027, 74.57108016901569, 74.65615517681422, 74.72421518305306, 74.72421518305306, 74.84899186115759, 74.8660068627173, 74.84048436037773, 74.87735019709044, 74.91138020020985, 74.94541020332926, 75.03332104472109, 75.09854521736663, 75.14675438845248, 75.20347106031818, 75.1977993931316, 75.21197856109804, 75.26302356577716, 75.24600856421745, 75.27153106655702, 75.26302356577716, 75.24884439781073, 75.26869523296374, 75.31690440404957, 75.33675523920256, 75.39630774466154, 75.39347191106826, 75.43033774778097, 75.46153191730708, 75.46720358449366, 75.52392025635936, 75.54093525791906, 75.58630859541162, 75.5721294274452, 75.59198026259818, 75.64302526727731, 75.6628761024303, 75.70824943992285, 75.71108527351615, 75.72526444148258, 75.71108527351615, 75.71675694070272, 75.76780194538185, 75.78765278053483, 75.77630944616169, 75.76213027819527, 75.77630944616169, 75.78198111334827, 75.75078694382213, 75.73377194226242, 75.68556277117658, 75.6827269375833, 75.67421943680344, 75.6628761024303, 75.6260102657176, 75.61183109775118, 75.58063692822505, 75.56078609307205, 75.54093525791906, 75.51824858917279, 75.50123358761309, 75.4785469188668, 75.45586025012052, 75.41332274622124, 75.38780024388168, 75.34526273998242, 75.32541190482942, 75.2772027337436, 75.25168023140402, 75.22899356265775, 75.20630689391147, 75.18362022516519, 75.16660522360547, 75.13541105407936, 75.11556021892635, 75.08153021580694, 75.03899271190765, 74.98511187363526, 74.9198877009897, 74.87167852990386, 74.78660352210532, 74.72421518305306, 74.61928934010153, 74.47182599325072], "calls": [0.0, 1.5341859739670478, 3.658225335337322, 6.471372259875791, 9.806312565578652, 11.266766866120296, 12.545727816691715, 13.532597907154809, 14.678274678841847, 16.124549811417065, 17.244704080764542, 18.699486714119615, 19.774267645974533, 20.968153588747413, 21.997561183109777, 23.055327113404985, 24.135779712446475, 25.097127300570005, 25.859966537163597, 26.741910784675156, 27.524600856421742, 28.41221677111987, 29.574908544366618, 30.38312111845277, 31.35014037376287, 32.31432379547968, 33.18776054221138, 34.11224229362221, 35.189859059070415, 36.114340810481245, 37.115390068910756, 38.03420015313501, 38.92748773501971, 39.713013640359584, 40.68854039644953, 41.499588804128976, 42.401383886793525, 43.32302980461106, 44.11422737713751, 45.18900830899243, 46.30349091115333, 47.09752431727306, 48.02767773587046, 49.213056177863486, 50.347389615177384, 51.68873890480107, 52.803221506961975, 53.741882426339224, 55.100246717522616, 56.46144684229929, 57.81697529988941, 59.04772707937498, 60.63012222442787, 62.02251651873068, 63.27028329977596, 64.68820009641834, 66.51447693049371, 68.04015540368091, 69.65090888466665, 71.89972492414145, 73.58704591214588, 75.7961602813147, 78.22079800357315, 80.11513484388736, 81.92723250999632, 83.45858265037008, 84.58440858690412, 85.69038368828518, 86.64038794203556, 87.4145705130023, 88.16323058162948, 88.85233814479766, 89.44786319938746, 90.07458242350339, 90.7410033179253, 91.19757252644415, 91.65414173496299, 92.09653177551542, 92.66653432776565, 93.09474520035165, 93.49176190341151, 93.92280860959079, 94.39639281966934, 94.79908118991578, 95.30385956952047, 95.62430876556164, 95.90222045770355, 96.19998298499844, 96.43819300683435, 96.69908969741655, 96.99685222471145, 97.27192808326008, 97.54133227462212, 97.84193063551031, 98.08014065734622, 98.29849984402915, 98.55939653461135, 98.89118906502567, 99.13223492045486, 99.50940078836173, 100.0], "exact_score": [66.95119530385956, 67.49567535377024, 68.06567790602047, 68.55911295125202, 68.99583132461787, 69.39284802767773, 69.75583472761818, 70.09897059240564, 70.41941978844682, 70.72285398292829, 71.00076567507018, 71.27300570002552, 71.49703655389501, 71.78345574681677, 72.05853160536539, 72.25987579048862, 72.5604741513768, 72.8554008450784, 73.10495420128747, 73.36585089186967, 73.58421007855259, 73.82809176757507, 74.01809261832514, 74.16555596517595, 74.30167597765363, 74.46899015965744, 74.58242350338881, 74.67600601196722, 74.72705101664634, 74.7894393556986, 74.84332019397102, 74.82914102600459, 74.86884269631058, 74.90287269942999, 74.9397385361427, 75.01914187675467, 75.08436604940022, 75.1439185548592, 75.20063522672488, 75.19496355953832, 75.22899356265775, 75.24884439781073, 75.26018773218387, 75.2772027337436, 75.26018773218387, 75.26869523296374, 75.29138190171001, 75.3282477384227, 75.39347191106826, 75.39063607747497, 75.41615857981454, 75.4416810821541, 75.44735274934067, 75.48138275246008, 75.5324277571392, 75.54093525791906, 75.58063692822505, 75.58630859541162, 75.57496526103849, 75.59198026259818, 75.6089952641579, 75.65436860165046, 75.6628761024303, 75.68839860476987, 75.71392110710944, 75.70824943992285, 75.70824943992285, 75.71392110710944, 75.71675694070272, 75.70824943992285, 75.719592774296, 75.72810027507586, 75.76496611178855, 75.77347361256841, 75.76496611178855, 75.77347361256841, 75.77630944616169, 75.76213027819527, 75.75362277741542, 75.77063777897513, 75.78198111334827, 75.78198111334827, 75.77347361256841, 75.74795111022885, 75.74511527663557, 75.719592774296, 75.67989110399002, 75.67138360321016, 75.6628761024303, 75.62033859853103, 75.5891444290049, 75.56078609307205, 75.50974108839293, 75.46153191730708, 75.37645690950855, 75.31690440404957, 75.21765022828461, 75.13541105407936, 75.00496270878824, 74.76108101976577, 74.47182599325072], "exact_calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "threshold_area25": 72.8165499248504, "exact_area100": 74.41887082388268}}, "old_normalized_endpoint_budget": 63}]}, {"expert": "nlvr2", "endpoint": "MiniCPM", "task": "nlvr2", "small": 80.00287005811867, "large": 90.7368874219703, "runs": [{"seed": 2026, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 1.0, "validation_area100": 0.8769117990600045, "training_seconds": 34.49981618800666, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7393332253014886, "operator": "ge"}, {"budget": 0.02, "threshold": 0.664790123739233, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5905857158559317, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5268104726070402, "operator": "ge"}, {"budget": 0.05, "threshold": 0.48200011611721194, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4328714273365525, "operator": "ge"}, {"budget": 0.07, "threshold": 0.39703806778339684, "operator": "ge"}, {"budget": 0.08, "threshold": 0.36566112226513875, "operator": "ge"}, {"budget": 0.09, "threshold": 0.34107180661098835, "operator": "ge"}, {"budget": 0.1, "threshold": 0.30450126429208624, "operator": "ge"}, {"budget": 0.11, "threshold": 0.2784913741717816, "operator": "ge"}, {"budget": 0.12, "threshold": 0.2538776845492074, "operator": "ge"}, {"budget": 0.13, "threshold": 0.22680442553903588, "operator": "ge"}, {"budget": 0.14, "threshold": 0.20470502986988437, "operator": "ge"}, {"budget": 0.15, "threshold": 0.17790770881441573, "operator": "ge"}, {"budget": 0.16, "threshold": 0.1544125319220188, "operator": "ge"}, {"budget": 0.17, "threshold": 0.13410646162182238, "operator": "ge"}, {"budget": 0.18, "threshold": 0.11165566854177338, "operator": "ge"}, {"budget": 0.19, "threshold": 0.0912375496959033, "operator": "ge"}, {"budget": 0.2, "threshold": 0.07283258496870766, "operator": "ge"}, {"budget": 0.21, "threshold": 0.053448731581761416, "operator": "ge"}, {"budget": 0.22, "threshold": 0.03418169043246344, "operator": "ge"}, {"budget": 0.23, "threshold": 0.014678456893619947, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0001918163758804253, "operator": "ge"}, {"budget": 0.25, "threshold": -0.017943248852753994, "operator": "ge"}, {"budget": 0.26, "threshold": -0.038072956831963344, "operator": "ge"}, {"budget": 0.27, "threshold": -0.05826355755533358, "operator": "ge"}, {"budget": 0.28, "threshold": -0.07743252497987517, "operator": "ge"}, {"budget": 0.29, "threshold": -0.10521805424758125, "operator": "ge"}, {"budget": 0.3, "threshold": -0.13514235992621781, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1548074811538734, "operator": "ge"}, {"budget": 0.32, "threshold": -0.1836565069207265, "operator": "ge"}, {"budget": 0.33, "threshold": -0.20903667816242796, "operator": "ge"}, {"budget": 0.34, "threshold": -0.23741469532358386, "operator": "ge"}, {"budget": 0.35, "threshold": -0.25979882500064017, "operator": "ge"}, {"budget": 0.36, "threshold": -0.2820551188962453, "operator": "ge"}, {"budget": 0.37, "threshold": -0.3144424025575484, "operator": "ge"}, {"budget": 0.38, "threshold": -0.3460823052668325, "operator": "ge"}, {"budget": 0.39, "threshold": -0.3749929068676208, "operator": "ge"}, {"budget": 0.4, "threshold": -0.40528092398520743, "operator": "ge"}, {"budget": 0.41, "threshold": -0.43502577724855834, "operator": "ge"}, {"budget": 0.42, "threshold": -0.4670196180359008, "operator": "ge"}, {"budget": 0.43, "threshold": -0.4953619499153169, "operator": "ge"}, {"budget": 0.44, "threshold": -0.5244220566486463, "operator": "ge"}, {"budget": 0.45, "threshold": -0.5487518511284135, "operator": "ge"}, {"budget": 0.46, "threshold": -0.5760344076581171, "operator": "ge"}, {"budget": 0.47, "threshold": -0.5999531972114088, "operator": "ge"}, {"budget": 0.48, "threshold": -0.6193229512481766, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6414949598983569, "operator": "ge"}, {"budget": 0.5, "threshold": -0.6600309771177953, "operator": "ge"}, {"budget": 0.51, "threshold": -0.6789512684666976, "operator": "ge"}, {"budget": 0.52, "threshold": -0.6993216264808503, "operator": "ge"}, {"budget": 0.53, "threshold": -0.7170075909936111, "operator": "ge"}, {"budget": 0.54, "threshold": -0.7321923207552604, "operator": "ge"}, {"budget": 0.55, "threshold": -0.744946838945889, "operator": "ge"}, {"budget": 0.56, "threshold": -0.7611567810624863, "operator": "ge"}, {"budget": 0.57, "threshold": -0.7734029523155965, "operator": "ge"}, {"budget": 0.58, "threshold": -0.785585749611841, "operator": "ge"}, {"budget": 0.59, "threshold": -0.7974000677021961, "operator": "ge"}, {"budget": 0.6, "threshold": -0.8092955775108228, "operator": "ge"}, {"budget": 0.61, "threshold": -0.8242021952021628, "operator": "ge"}, {"budget": 0.62, "threshold": -0.8353493882042546, "operator": "ge"}, {"budget": 0.63, "threshold": -0.8441538045154002, "operator": "ge"}, {"budget": 0.64, "threshold": -0.8525328037979881, "operator": "ge"}, {"budget": 0.65, "threshold": -0.8616336398718646, "operator": "ge"}, {"budget": 0.66, "threshold": -0.8698869681028346, "operator": "ge"}, {"budget": 0.67, "threshold": -0.8794876573379117, "operator": "ge"}, {"budget": 0.68, "threshold": -0.8878537778311125, "operator": "ge"}, {"budget": 0.69, "threshold": -0.8954695345195154, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9018304018837107, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9083112009436606, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9140908646092232, "operator": "ge"}, {"budget": 0.73, "threshold": -0.919342477855753, "operator": "ge"}, {"budget": 0.74, "threshold": -0.924020686610261, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9287956961669069, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9333202016579287, "operator": "ge"}, {"budget": 0.77, "threshold": -0.936668265466352, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9402493167850022, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9440823352702147, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9469554504901883, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9505162777165886, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9539478986789897, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9568423360830359, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9592733865803043, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9619812276221236, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9648629783912844, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9675187842292352, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9700922452727474, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9719967641348217, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9736419994474717, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9755516646205806, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9779963184224421, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9799021266232693, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9819087709532769, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9836896605081553, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9857215482158455, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9877857367946115, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9896488054392807, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9922040785971863, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.2492122600974, 80.46405041535377, 80.89372672586651, 81.25179031796046, 81.52391864795187, 81.86765969636207, 82.19707820108852, 82.51217416213119, 82.74133486107132, 83.21397880263535, 83.5720423947293, 83.78688054998568, 84.07333142366083, 84.30249212260097, 84.61758808364365, 84.8037811515325, 85.00429676310513, 85.06158693784016, 85.20481237467774, 85.47694070466915, 85.62016614150673, 85.86364938413061, 85.96390718991692, 86.1787453451733, 86.29332569464336, 86.37926095674591, 86.57977656831852, 86.68003437410484, 86.78029217989113, 86.83758235462618, 86.95216270409625, 87.08106559725006, 87.26725866513893, 87.39616155829275, 87.5107419077628, 87.62532225723288, 87.854482956173, 87.95474076195931, 88.15525637353193, 88.29848181036952, 88.4417072472071, 88.62790031509596, 88.68519048983099, 88.65654540246346, 88.71383557719851, 88.77112575193354, 88.84273847035233, 88.85706101403609, 88.95731881982239, 88.97164136350615, 89.07189916929245, 89.15783443139502, 89.20080206244629, 89.18647951876252, 89.31538241191636, 89.31538241191636, 89.37267258665138, 89.45860784875393, 89.47293039243769, 89.51589802348897, 89.47293039243769, 89.573188198224, 89.58751074190776, 89.58751074190776, 89.60183328559151, 89.573188198224, 89.58751074190776, 89.60183328559151, 89.55886565454024, 89.60183328559151, 89.6734460040103, 89.73073617874535, 89.71641363506157, 89.73073617874535, 89.74505872242909, 89.75938126611285, 89.77370380979661, 89.80234889716414, 89.81667144084788, 89.78802635348038, 89.78802635348038, 89.81667144084788, 89.78802635348038, 89.80234889716414, 89.87396161558291, 89.90260670295044, 89.88828415926669, 89.9455743340017, 89.9455743340017, 89.90260670295044, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.97421942136923, 90.00286450873675, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 35, 59, 92, 125, 152, 179, 205, 232, 252, 288, 317, 341, 364, 384, 410, 432, 451, 467, 484, 507, 520, 544, 556, 576, 590, 598, 615, 628, 642, 654, 669, 681, 698, 713, 726, 737, 755, 769, 785, 801, 812, 830, 838, 843, 851, 862, 868, 874, 882, 886, 895, 903, 909, 913, 926, 930, 936, 944, 948, 955, 958, 966, 969, 974, 979, 981, 985, 990, 990, 994, 1000, 1006, 1010, 1012, 1014, 1017, 1022, 1026, 1029, 1030, 1035, 1039, 1041, 1044, 1050, 1056, 1058, 1062, 1066, 1068, 1072, 1075, 1076, 1080, 1084, 1085, 1086, 1086, 1087, 1087], "harmed": [0, 5, 14, 17, 25, 33, 36, 39, 44, 48, 51, 55, 64, 67, 71, 75, 84, 89, 101, 108, 112, 115, 122, 127, 132, 138, 140, 143, 149, 156, 164, 171, 174, 178, 184, 189, 192, 194, 201, 203, 209, 210, 215, 219, 226, 230, 237, 238, 243, 244, 247, 249, 251, 254, 259, 263, 267, 269, 271, 274, 278, 284, 285, 287, 292, 296, 300, 303, 307, 310, 311, 312, 314, 319, 320, 321, 323, 327, 329, 331, 334, 339, 341, 345, 347, 348, 352, 355, 355, 359, 364, 366, 368, 369, 371, 373, 376, 378, 380, 383, 383], "exact_score": [79.81953594958465, 80.2492122600974, 80.46405041535377, 80.89372672586651, 81.25179031796046, 81.52391864795187, 81.86765969636207, 82.19707820108852, 82.51217416213119, 82.74133486107132, 83.21397880263535, 83.5720423947293, 83.78688054998568, 84.07333142366083, 84.30249212260097, 84.61758808364365, 84.8037811515325, 85.00429676310513, 85.06158693784016, 85.20481237467774, 85.47694070466915, 85.62016614150673, 85.86364938413061, 85.96390718991692, 86.1787453451733, 86.29332569464336, 86.37926095674591, 86.57977656831852, 86.68003437410484, 86.78029217989113, 86.83758235462618, 86.95216270409625, 87.08106559725006, 87.26725866513893, 87.39616155829275, 87.5107419077628, 87.62532225723288, 87.854482956173, 87.95474076195931, 88.15525637353193, 88.29848181036952, 88.4417072472071, 88.62790031509596, 88.68519048983099, 88.65654540246346, 88.71383557719851, 88.77112575193354, 88.84273847035233, 88.85706101403609, 88.95731881982239, 88.97164136350615, 89.07189916929245, 89.15783443139502, 89.20080206244629, 89.18647951876252, 89.31538241191636, 89.31538241191636, 89.37267258665138, 89.45860784875393, 89.47293039243769, 89.51589802348897, 89.47293039243769, 89.573188198224, 89.58751074190776, 89.58751074190776, 89.60183328559151, 89.573188198224, 89.58751074190776, 89.60183328559151, 89.55886565454024, 89.60183328559151, 89.6734460040103, 89.73073617874535, 89.71641363506157, 89.73073617874535, 89.74505872242909, 89.75938126611285, 89.77370380979661, 89.80234889716414, 89.81667144084788, 89.78802635348038, 89.78802635348038, 89.81667144084788, 89.78802635348038, 89.80234889716414, 89.87396161558291, 89.90260670295044, 89.88828415926669, 89.9455743340017, 89.9455743340017, 89.90260670295044, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.97421942136923, 90.00286450873675, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.69117990600044, "threshold_area100": 87.69048983099398, "threshold_area25": 83.5777714122028}, "test": {"score": [80.00287005811867, 80.57688168185406, 81.1006672885126, 81.57422687809428, 82.01191074119252, 82.3850182966205, 82.71507498026835, 83.03795651861951, 83.28191145870704, 83.5115161082012, 83.90614909951927, 84.20033005668365, 84.39405897969434, 84.73129080863887, 84.97524574872641, 85.37705388534118, 85.61383368013202, 85.8290880390328, 86.11609385090048, 86.2811221927244, 86.49637655162518, 86.61117887637225, 86.78338236349286, 86.96276099591017, 87.0345124488771, 87.19236564540432, 87.35021884193155, 87.48654660256871, 87.62287436320587, 87.63722465379924, 87.83095357680993, 87.9601061921504, 88.0462079357107, 88.08208366219417, 88.16101026045777, 88.26146229461148, 88.49824208940231, 88.67044557652292, 88.84264906364353, 88.95027624309392, 89.02920284135753, 89.13683002080792, 89.21575661907154, 89.3377340891153, 89.50276243093923, 89.60321446509292, 89.67496591805984, 89.75389251632346, 89.75389251632346, 89.811293678697, 89.83281911458707, 89.84716940518047, 89.904570567554, 89.99067231111431, 90.00502260170768, 90.09112434526799, 90.19875152471838, 90.23462725120184, 90.32072899476215, 90.3566047212456, 90.37095501183899, 90.44270646480591, 90.48575733658608, 90.48575733658608, 90.52163306306953, 90.51445791777284, 90.5503336442563, 90.5503336442563, 90.53598335366291, 90.57903422544307, 90.58620937073975, 90.60773480662984, 90.62926024251992, 90.65078567840999, 90.60055966133315, 90.58620937073975, 90.58620937073975, 90.58620937073975, 90.59338451603645, 90.6364353878166, 90.67948625959676, 90.68666140489346, 90.70818684078354, 90.70818684078354, 90.7297122766736, 90.744062567267, 90.744062567267, 90.79428858434385, 90.80863887493722, 90.83733945612398, 90.82298916553061, 90.77993829375045, 90.84451460142068, 90.84451460142068, 90.83733945612398, 90.80863887493722, 90.79428858434385, 90.77993829375045, 90.744062567267, 90.75123771256368, 90.73688742197031], "calls": [0.0, 0.9758197603501471, 1.9803401018870632, 3.2933916911817462, 4.5418669728062, 5.460285570782808, 6.572433091770108, 7.4836765444500255, 8.280117672382865, 9.11960967209586, 10.511587859654158, 11.544808782377842, 12.592379995694913, 13.848030422616059, 14.93147736241659, 16.208653225227813, 17.327975891511805, 18.30379565186195, 19.50922006170625, 20.406113223792783, 21.303006385879314, 22.135323240295616, 23.060916983568916, 23.89323383798522, 24.59639807706106, 25.464590657960823, 26.332783238860586, 27.17227523857358, 27.918490349429575, 29.04498816101026, 30.142785391404175, 30.932051374040327, 31.82894453612686, 32.66843653583985, 33.64425629619, 34.41917198823276, 35.165387099088754, 36.3277606371529, 37.4470833034369, 38.35115161082012, 39.31262108057688, 40.38171772978403, 41.49386525077133, 42.491210447011554, 43.53160651503193, 44.42132453182177, 45.626748941666065, 46.58821841142283, 47.520987299992825, 48.62595967568343, 49.479801965989815, 50.56324890579034, 51.840424768601565, 52.97409772547894, 53.91404175934563, 54.703307741981774, 55.80110497237569, 56.67647269857214, 57.54466527947191, 58.549185621008824, 59.41737820190859, 60.97438473129081, 62.057831671091336, 62.81839707254072, 63.672239362847094, 64.60500825141709, 65.5306019946904, 66.77907727631485, 67.84817392552199, 68.98184688239937, 69.90026548037598, 70.92631125780296, 72.05998421468036, 73.16495659037095, 74.18382722250125, 75.23857358111502, 76.1785176149817, 77.05388534117816, 77.91490277678122, 78.91224797302145, 79.65846308387745, 80.69168400660114, 81.73208007462152, 82.6648489631915, 83.43976465523427, 84.4299347061778, 85.50620650068164, 86.61835402166894, 87.68745067087609, 88.53411781588576, 89.30185836263185, 90.31355384946545, 91.59072971227667, 92.45892229317644, 93.54236923297697, 94.48231326684365, 95.58011049723757, 96.62768171055464, 97.57480088971802, 98.75152471837555, 100.0], "rescued": [0, 88, 167, 247, 318, 377, 432, 483, 522, 561, 629, 679, 723, 783, 832, 897, 942, 985, 1030, 1067, 1101, 1129, 1164, 1198, 1219, 1252, 1282, 1309, 1335, 1357, 1394, 1418, 1444, 1466, 1486, 1507, 1543, 1574, 1601, 1624, 1647, 1668, 1687, 1710, 1739, 1758, 1776, 1792, 1801, 1816, 1827, 1839, 1857, 1875, 1882, 1897, 1918, 1928, 1943, 1951, 1957, 1978, 1989, 1994, 2002, 2005, 2014, 2023, 2027, 2036, 2040, 2048, 2055, 2065, 2069, 2076, 2084, 2089, 2095, 2103, 2113, 2118, 2125, 2129, 2135, 2141, 2148, 2158, 2164, 2169, 2173, 2175, 2189, 2192, 2197, 2200, 2207, 2209, 2212, 2217, 2225], "harmed": [0, 8, 14, 28, 38, 45, 54, 60, 65, 72, 85, 94, 111, 124, 139, 148, 160, 173, 178, 192, 196, 208, 219, 228, 239, 250, 258, 266, 273, 293, 303, 309, 323, 340, 349, 356, 359, 366, 369, 377, 389, 395, 403, 409, 415, 420, 428, 433, 442, 449, 457, 467, 477, 483, 488, 491, 497, 502, 505, 508, 512, 523, 528, 533, 536, 540, 544, 553, 559, 562, 565, 570, 574, 581, 592, 601, 609, 614, 619, 621, 625, 629, 633, 637, 640, 644, 651, 654, 658, 659, 665, 673, 678, 681, 687, 694, 703, 707, 715, 719, 729], "exact_score": [80.00287005811867, 80.59840711774415, 81.10784243380928, 81.4594245533472, 81.81818181818183, 82.2056396642032, 82.55722178374111, 82.84422759560881, 83.19580971514674, 83.47564038171772, 83.7841716294755, 84.02095142426634, 84.27925665494726, 84.45863528736457, 84.75999138982566, 84.99677118461649, 85.30530243237426, 85.55643251775848, 85.75733658606588, 86.0084666714501, 86.24524646624094, 86.41744995336155, 86.59682858577887, 86.79055750878956, 86.95558585061349, 87.06321303006386, 87.3143431154481, 87.46502116667862, 87.62287436320587, 87.63722465379924, 87.82377843151325, 87.96728133744709, 88.06773337160078, 88.14665996986439, 88.21841142283131, 88.44084092702877, 88.64174499533615, 88.77807275597331, 88.93592595250054, 88.971801678984, 89.0722537131377, 89.18705603788477, 89.28750807203846, 89.39513525148884, 89.54581330271938, 89.65344048216977, 89.68931620865322, 89.73954222573008, 89.8041185334003, 89.78259309751023, 89.84716940518047, 89.85434455047715, 89.91892085814737, 89.99067231111431, 90.01937289230108, 90.0982994905647, 90.19157637942169, 90.26332783238861, 90.34942957594892, 90.3566047212456, 90.39965559302576, 90.44270646480591, 90.48575733658608, 90.47858219128938, 90.50010762717945, 90.51445791777284, 90.54315849895961, 90.55750878955298, 90.5503336442563, 90.57903422544307, 90.58620937073975, 90.61490995192652, 90.62926024251992, 90.65078567840999, 90.61490995192652, 90.58620937073975, 90.57185908014638, 90.57903422544307, 90.60773480662984, 90.66513596900337, 90.67948625959676, 90.68666140489346, 90.71536198608023, 90.72253713137691, 90.73688742197031, 90.73688742197031, 90.75123771256368, 90.78711343904715, 90.80146372964053, 90.80146372964053, 90.78711343904715, 90.8301643108273, 90.85886489201407, 90.84451460142068, 90.85886489201407, 90.75841285786038, 90.78711343904715, 90.76558800315708, 90.75123771256368, 90.75123771256368, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.50314169891574, "threshold_area100": 88.53619860802182, "threshold_area25": 84.35244313697352}, "selected_budget": 95, "selected": {"val": {"score": 90.00286450873675, "calls": 94.98710971068462, "rescued": 1084, "harmed": 373}, "test": {"score": 90.80863887493722, "calls": 94.48231326684365, "rescued": 2200, "harmed": 694}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.573597252368927, "operator": "ge"}, {"budget": 0.02, "threshold": 0.467054545879364, "operator": "ge"}, {"budget": 0.03, "threshold": 0.38870108127593994, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3201385736465454, "operator": "ge"}, {"budget": 0.05, "threshold": 0.27068841457366943, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2337329387664795, "operator": "ge"}, {"budget": 0.07, "threshold": 0.2000429332256317, "operator": "ge"}, {"budget": 0.08, "threshold": 0.17774733901023865, "operator": "ge"}, {"budget": 0.09, "threshold": 0.15693923830986023, "operator": "ge"}, {"budget": 0.1, "threshold": 0.13545580208301544, "operator": "ge"}, {"budget": 0.11, "threshold": 0.11789820343255997, "operator": "ge"}, {"budget": 0.12, "threshold": 0.10202538967132568, "operator": "ge"}, {"budget": 0.13, "threshold": 0.08808434009552002, "operator": "ge"}, {"budget": 0.14, "threshold": 0.07842271029949188, "operator": "ge"}, {"budget": 0.15, "threshold": 0.06826462596654892, "operator": "ge"}, {"budget": 0.16, "threshold": 0.0565154105424881, "operator": "ge"}, {"budget": 0.17, "threshold": 0.045510873198509216, "operator": "ge"}, {"budget": 0.18, "threshold": 0.0361267626285553, "operator": "ge"}, {"budget": 0.19, "threshold": 0.0285777747631073, "operator": "ge"}, {"budget": 0.2, "threshold": 0.02070014923810959, "operator": "ge"}, {"budget": 0.21, "threshold": 0.014114804565906525, "operator": "ge"}, {"budget": 0.22, "threshold": 0.009071225300431252, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0037600696086883545, "operator": "ge"}, {"budget": 0.24, "threshold": -4.303455352783203e-05, "operator": "ge"}, {"budget": 0.25, "threshold": -0.003944970667362213, "operator": "ge"}, {"budget": 0.26, "threshold": -0.0061277300119400024, "operator": "ge"}, {"budget": 0.27, "threshold": -0.007703454699367285, "operator": "ge"}, {"budget": 0.28, "threshold": -0.008571228012442589, "operator": "ge"}, {"budget": 0.29, "threshold": -0.009746411815285683, "operator": "ge"}, {"budget": 0.3, "threshold": -0.01070293691009283, "operator": "ge"}, {"budget": 0.31, "threshold": -0.011521071195602417, "operator": "ge"}, {"budget": 0.32, "threshold": -0.01231210958212614, "operator": "ge"}, {"budget": 0.33, "threshold": -0.01305999793112278, "operator": "ge"}, {"budget": 0.34, "threshold": -0.013766501098871231, "operator": "ge"}, {"budget": 0.35, "threshold": -0.014655189588665962, "operator": "ge"}, {"budget": 0.36, "threshold": -0.015428543090820312, "operator": "ge"}, {"budget": 0.37, "threshold": -0.016353139653801918, "operator": "ge"}, {"budget": 0.38, "threshold": -0.01717471145093441, "operator": "ge"}, {"budget": 0.39, "threshold": -0.01784132421016693, "operator": "ge"}, {"budget": 0.4, "threshold": -0.01849311962723732, "operator": "ge"}, {"budget": 0.41, "threshold": -0.01931106112897396, "operator": "ge"}, {"budget": 0.42, "threshold": -0.020126480609178543, "operator": "ge"}, {"budget": 0.43, "threshold": -0.020881343632936478, "operator": "ge"}, {"budget": 0.44, "threshold": -0.02175307832658291, "operator": "ge"}, {"budget": 0.45, "threshold": -0.02276230975985527, "operator": "ge"}, {"budget": 0.46, "threshold": -0.02357630617916584, "operator": "ge"}, {"budget": 0.47, "threshold": -0.024464929476380348, "operator": "ge"}, {"budget": 0.48, "threshold": -0.025521136820316315, "operator": "ge"}, {"budget": 0.49, "threshold": -0.026406755670905113, "operator": "ge"}, {"budget": 0.5, "threshold": -0.027374831959605217, "operator": "ge"}, {"budget": 0.51, "threshold": -0.028442319482564926, "operator": "ge"}, {"budget": 0.52, "threshold": -0.029475389048457146, "operator": "ge"}, {"budget": 0.53, "threshold": -0.030632546171545982, "operator": "ge"}, {"budget": 0.54, "threshold": -0.031830403953790665, "operator": "ge"}, {"budget": 0.55, "threshold": -0.03317571058869362, "operator": "ge"}, {"budget": 0.56, "threshold": -0.034447893500328064, "operator": "ge"}, {"budget": 0.57, "threshold": -0.035507604479789734, "operator": "ge"}, {"budget": 0.58, "threshold": -0.03689383342862129, "operator": "ge"}, {"budget": 0.59, "threshold": -0.038047946989536285, "operator": "ge"}, {"budget": 0.6, "threshold": -0.03914044797420502, "operator": "ge"}, {"budget": 0.61, "threshold": -0.04032886400818825, "operator": "ge"}, {"budget": 0.62, "threshold": -0.04149208962917328, "operator": "ge"}, {"budget": 0.63, "threshold": -0.042960528284311295, "operator": "ge"}, {"budget": 0.64, "threshold": -0.0442170612514019, "operator": "ge"}, {"budget": 0.65, "threshold": -0.045526884496212006, "operator": "ge"}, {"budget": 0.66, "threshold": -0.04681326448917389, "operator": "ge"}, {"budget": 0.67, "threshold": -0.04848075285553932, "operator": "ge"}, {"budget": 0.68, "threshold": -0.04972432553768158, "operator": "ge"}, {"budget": 0.69, "threshold": -0.051324523985385895, "operator": "ge"}, {"budget": 0.7, "threshold": -0.053144536912441254, "operator": "ge"}, {"budget": 0.71, "threshold": -0.054645754396915436, "operator": "ge"}, {"budget": 0.72, "threshold": -0.05627942457795143, "operator": "ge"}, {"budget": 0.73, "threshold": -0.057676635682582855, "operator": "ge"}, {"budget": 0.74, "threshold": -0.059252239763736725, "operator": "ge"}, {"budget": 0.75, "threshold": -0.0607621893286705, "operator": "ge"}, {"budget": 0.76, "threshold": -0.06233196333050728, "operator": "ge"}, {"budget": 0.77, "threshold": -0.06430236995220184, "operator": "ge"}, {"budget": 0.78, "threshold": -0.06591057777404785, "operator": "ge"}, {"budget": 0.79, "threshold": -0.0676574558019638, "operator": "ge"}, {"budget": 0.8, "threshold": -0.06951721012592316, "operator": "ge"}, {"budget": 0.81, "threshold": -0.07135815173387527, "operator": "ge"}, {"budget": 0.82, "threshold": -0.0732610672712326, "operator": "ge"}, {"budget": 0.83, "threshold": -0.07527083158493042, "operator": "ge"}, {"budget": 0.84, "threshold": -0.07744777947664261, "operator": "ge"}, {"budget": 0.85, "threshold": -0.07942485064268112, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0826490968465805, "operator": "ge"}, {"budget": 0.87, "threshold": -0.08439412713050842, "operator": "ge"}, {"budget": 0.88, "threshold": -0.08643028885126114, "operator": "ge"}, {"budget": 0.89, "threshold": -0.08867266029119492, "operator": "ge"}, {"budget": 0.9, "threshold": -0.09101386368274689, "operator": "ge"}, {"budget": 0.91, "threshold": -0.09365352988243103, "operator": "ge"}, {"budget": 0.92, "threshold": -0.09667940437793732, "operator": "ge"}, {"budget": 0.93, "threshold": -0.10105226933956146, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10510002076625824, "operator": "ge"}, {"budget": 0.95, "threshold": -0.11044986546039581, "operator": "ge"}, {"budget": 0.96, "threshold": -0.11598265916109085, "operator": "ge"}, {"budget": 0.97, "threshold": -0.12346357107162476, "operator": "ge"}, {"budget": 0.98, "threshold": -0.13342399895191193, "operator": "ge"}, {"budget": 0.99, "threshold": -0.14839251339435577, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.13463191062732, 80.53566313377254, 80.90804926955026, 81.25179031796046, 81.60985391005443, 81.95359495846462, 82.24004583213978, 82.52649670581495, 82.68404468633629, 83.04210827843025, 83.45746204525923, 83.80120309366943, 84.0876539673446, 84.30249212260097, 84.5316528215411, 84.74649097679747, 84.91836150100256, 85.17616728731022, 85.43397307361786, 85.51990833572042, 85.67745631624176, 85.92093955886565, 86.02119736465195, 86.1787453451733, 86.27900315095961, 86.43655113148094, 86.46519621884846, 86.47951876253221, 86.50816384989973, 86.53680893726725, 86.55113148095101, 86.56545402463478, 86.59409911200228, 86.6227441993698, 86.68003437410484, 86.72300200515612, 86.79461472357491, 86.85190489830994, 86.80893726725866, 86.83758235462618, 86.80893726725866, 86.83758235462618, 86.86622744199369, 86.89487252936121, 86.88054998567745, 86.92351761672873, 87.00945287883127, 87.02377542251504, 87.02377542251504, 87.05242050988255, 87.10971068461758, 87.18132340303637, 87.26725866513893, 87.35319392724148, 87.41048410197651, 87.4820968203953, 87.5107419077628, 87.59667716986537, 87.62532225723288, 87.65396734460039, 87.71125751933543, 87.7542251503867, 87.84016041248925, 87.8258378688055, 87.8258378688055, 87.86880549985678, 87.91177313090805, 87.96906330564308, 88.0263534803781, 88.06932111142939, 88.21254654826697, 88.29848181036952, 88.32712689773703, 88.41306215983958, 88.49899742194212, 88.57061014036091, 88.65654540246346, 88.69951303351475, 88.77112575193354, 88.79977083930105, 88.85706101403609, 88.90002864508736, 88.98596390718991, 89.07189916929245, 89.07189916929245, 89.17215697507876, 89.17215697507876, 89.1291893440275, 89.18647951876252, 89.24376969349757, 89.25809223718132, 89.34402749928387, 89.35835004296763, 89.41564021770266, 89.53022056717272, 89.58751074190776, 89.71641363506157, 89.75938126611285, 89.80234889716414, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 66, 95, 126, 157, 186, 214, 239, 262, 288, 319, 345, 373, 394, 418, 439, 455, 475, 498, 512, 530, 551, 560, 576, 586, 598, 601, 605, 610, 612, 614, 616, 619, 623, 629, 632, 638, 642, 643, 646, 646, 649, 652, 655, 659, 663, 671, 674, 678, 683, 688, 695, 703, 709, 719, 725, 732, 741, 746, 751, 757, 764, 772, 776, 780, 787, 792, 797, 805, 811, 822, 831, 840, 848, 859, 867, 875, 882, 890, 893, 900, 909, 916, 926, 935, 948, 956, 960, 970, 978, 985, 996, 1003, 1014, 1024, 1033, 1047, 1058, 1071, 1087], "harmed": [0, 9, 16, 19, 26, 32, 37, 45, 50, 62, 63, 65, 67, 75, 81, 89, 95, 99, 101, 106, 114, 121, 125, 127, 132, 135, 136, 137, 140, 143, 143, 144, 145, 146, 148, 150, 150, 151, 151, 155, 156, 158, 159, 160, 161, 166, 167, 169, 171, 175, 178, 179, 181, 183, 183, 189, 190, 195, 198, 201, 204, 206, 210, 212, 217, 221, 225, 227, 228, 232, 235, 236, 239, 246, 248, 253, 256, 258, 262, 265, 266, 269, 275, 276, 280, 289, 295, 303, 310, 316, 320, 326, 331, 337, 344, 346, 351, 356, 364, 374, 383], "exact_score": [79.81953594958465, 80.13463191062732, 80.53566313377254, 80.90804926955026, 81.25179031796046, 81.60985391005443, 81.95359495846462, 82.24004583213978, 82.52649670581495, 82.68404468633629, 83.04210827843025, 83.45746204525923, 83.80120309366943, 84.0876539673446, 84.30249212260097, 84.5316528215411, 84.74649097679747, 84.91836150100256, 85.17616728731022, 85.43397307361786, 85.51990833572042, 85.67745631624176, 85.92093955886565, 86.02119736465195, 86.1787453451733, 86.27900315095961, 86.43655113148094, 86.46519621884846, 86.47951876253221, 86.50816384989973, 86.53680893726725, 86.55113148095101, 86.56545402463478, 86.59409911200228, 86.6227441993698, 86.68003437410484, 86.72300200515612, 86.79461472357491, 86.85190489830994, 86.80893726725866, 86.83758235462618, 86.80893726725866, 86.83758235462618, 86.86622744199369, 86.89487252936121, 86.88054998567745, 86.92351761672873, 87.00945287883127, 87.02377542251504, 87.02377542251504, 87.05242050988255, 87.10971068461758, 87.18132340303637, 87.26725866513893, 87.35319392724148, 87.41048410197651, 87.4820968203953, 87.5107419077628, 87.59667716986537, 87.62532225723288, 87.65396734460039, 87.71125751933543, 87.7542251503867, 87.84016041248925, 87.8258378688055, 87.8258378688055, 87.86880549985678, 87.91177313090805, 87.96906330564308, 88.0263534803781, 88.06932111142939, 88.21254654826697, 88.29848181036952, 88.32712689773703, 88.41306215983958, 88.49899742194212, 88.57061014036091, 88.65654540246346, 88.69951303351475, 88.77112575193354, 88.79977083930105, 88.85706101403609, 88.90002864508736, 88.98596390718991, 89.07189916929245, 89.07189916929245, 89.17215697507876, 89.17215697507876, 89.1291893440275, 89.18647951876252, 89.24376969349757, 89.25809223718132, 89.34402749928387, 89.35835004296763, 89.41564021770266, 89.53022056717272, 89.58751074190776, 89.71641363506157, 89.75938126611285, 89.80234889716414, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 86.81450258679662, "threshold_area100": 86.81380693211115, "threshold_area25": 83.55055857920365}, "test": {"score": [80.00287005811867, 80.61993255363421, 81.06479156202914, 81.58140202339098, 82.0262610317859, 82.45676974958744, 82.76530099734521, 83.0451316639162, 83.28908660400374, 83.6047929970582, 83.89179880892588, 84.25773121905719, 84.59496304800173, 84.8963191504628, 85.09722321877018, 85.31965272296765, 85.59230824424195, 85.83626318432948, 86.07304297912033, 86.25242161153763, 86.45332567984502, 86.62552916696563, 86.85513381645978, 86.91253497883332, 87.0345124488771, 87.14213962832747, 87.24259166248117, 87.3071679701514, 87.33586855133817, 87.3932697137117, 87.42197029489847, 87.46502116667862, 87.52242232905216, 87.52959747434886, 87.55112291023893, 87.5941737820191, 87.63722465379924, 87.68745067087609, 87.75920212384301, 87.78072755973308, 87.84530386740332, 87.8596541579967, 87.92423046566694, 87.91705532037024, 87.90988017507355, 87.945755901557, 87.96728133744709, 88.06055822630408, 88.11795938867762, 88.13230967927102, 88.16101026045777, 88.17536055105116, 88.21841142283131, 88.29733802109493, 88.34038889287508, 88.39061490995192, 88.46236636291886, 88.46954150821554, 88.54129296118248, 88.61304441414939, 88.65609528592955, 88.66327043122624, 88.66327043122624, 88.69914615770969, 88.74219702948984, 88.74937217478654, 88.75654732008323, 88.86417449953362, 88.92157566190716, 88.98615196957739, 89.04355313195093, 89.065078567841, 89.12247973021454, 89.23010690966493, 89.23728205496163, 89.27315778144508, 89.29468321733516, 89.3377340891153, 89.39513525148884, 89.40948554208224, 89.44536126856569, 89.47406184975246, 89.47406184975246, 89.56733873860946, 89.64626533687307, 89.68214106335654, 89.67496591805984, 89.76824280691685, 89.83999425988375, 89.98349716581761, 90.06959890937792, 90.16287579823492, 90.20592667001507, 90.33507928535553, 90.39248044772907, 90.50010762717945, 90.65078567840999, 90.69383655019014, 90.69383655019014, 90.72253713137691, 90.73688742197031], "calls": [0.0, 1.0475712133170696, 2.0162158283705245, 3.1068379134677477, 4.41988950276243, 5.43158498959604, 6.371529023462726, 7.354523929109565, 8.20119107411925, 9.133959962689245, 10.260457774269929, 11.422831312334075, 12.728707756332067, 13.733228097868983, 14.529669225801822, 15.4767884049652, 16.603286216545886, 17.772834899906723, 18.928033292674176, 19.88950276243094, 21.051876300495085, 22.042046351438614, 22.967640094711918, 23.792781803831527, 24.58922293176437, 25.407189495587286, 26.3184329482672, 27.308602999210734, 27.93284064002296, 28.85125923799957, 29.669225801822485, 30.50154265623879, 31.369735237138553, 32.22357752744493, 33.02001865537777, 34.23261821051876, 34.98600846667145, 36.012054244098444, 37.095501183898975, 37.970868910095426, 38.76013489273158, 39.865107268422186, 40.84810217406903, 41.73064504556217, 42.67058907942886, 43.76121116452608, 44.64375403601923, 45.66979981344622, 46.87522422329052, 47.822343402453896, 48.82686374399082, 49.8600846667145, 50.7856784099878, 51.768673315634636, 52.873645691325244, 53.92839204993901, 55.004663844442845, 55.765229245892236, 56.827150749802676, 57.59489129654875, 58.55636076630552, 59.35280189423836, 60.3357967998852, 61.48382004735596, 62.330487192365645, 63.205854918562096, 64.16732438831886, 65.329697926383, 66.25529166965632, 67.42484035301715, 68.58721389108129, 69.57738394202482, 70.77563320657244, 71.63665064217551, 72.52636865896534, 73.45913754753533, 74.49953361555572, 75.76953433307024, 76.78122981990386, 77.77857501614407, 78.876372246538, 79.8952428786683, 81.02174069024898, 82.16258879242304, 83.26038602281696, 84.1572791849035, 85.53490708186841, 86.25959675683433, 87.14931477362417, 87.98880677333716, 88.90005022601709, 89.94762143933414, 90.98084236205783, 92.3010690966492, 93.39886632704312, 94.69039248044773, 95.7307885484681, 96.9433881036091, 98.07706106048647, 99.13180741910024, 100.0], "rescued": [0, 94, 163, 243, 319, 388, 436, 485, 532, 586, 635, 699, 762, 811, 847, 888, 938, 986, 1030, 1062, 1101, 1133, 1174, 1194, 1219, 1241, 1260, 1270, 1276, 1285, 1290, 1298, 1308, 1314, 1319, 1327, 1335, 1344, 1357, 1363, 1375, 1381, 1393, 1398, 1403, 1410, 1415, 1431, 1442, 1448, 1457, 1463, 1475, 1490, 1501, 1512, 1528, 1537, 1555, 1571, 1581, 1589, 1597, 1612, 1623, 1631, 1642, 1660, 1673, 1691, 1708, 1719, 1738, 1756, 1765, 1775, 1787, 1802, 1814, 1823, 1839, 1854, 1864, 1886, 1904, 1920, 1940, 1957, 1975, 1997, 2018, 2041, 2053, 2080, 2099, 2124, 2153, 2175, 2192, 2210, 2225], "harmed": [0, 8, 15, 23, 37, 46, 51, 61, 74, 84, 93, 106, 122, 129, 137, 147, 159, 173, 184, 191, 202, 210, 219, 231, 239, 246, 251, 252, 254, 255, 256, 258, 260, 265, 267, 269, 271, 273, 276, 279, 282, 286, 289, 295, 301, 303, 305, 308, 311, 315, 320, 324, 330, 334, 339, 343, 349, 357, 365, 371, 375, 382, 390, 400, 405, 412, 422, 425, 430, 439, 448, 456, 467, 470, 478, 483, 492, 501, 505, 512, 523, 534, 544, 553, 560, 571, 592, 596, 604, 606, 615, 625, 631, 640, 651, 661, 669, 685, 702, 716, 729], "exact_score": [80.00287005811867, 80.57688168185406, 81.05044127143574, 81.52400086101744, 81.87558298055536, 82.26304082657674, 82.63614838200473, 82.93750448446582, 83.2245102963335, 83.56174212527804, 83.82722250125565, 84.21468034727704, 84.39405897969434, 84.6954150821554, 84.96089545813304, 85.22637583411064, 85.47033077419817, 85.67840998780225, 85.90083949199972, 86.06586783382365, 86.25959675683433, 86.43897538925164, 86.61835402166894, 86.86230896175648, 86.96993614120686, 87.06321303006386, 87.22824137188778, 87.28564253426131, 87.33586855133817, 87.3932697137117, 87.45067087608524, 87.48654660256871, 87.52959747434886, 87.55112291023893, 87.57264834612901, 87.63722465379924, 87.68745067087609, 87.76637726913971, 87.78790270502978, 87.8596541579967, 87.87400444859009, 87.91705532037024, 87.91705532037024, 87.93140561096362, 87.96728133744709, 88.02468249982063, 88.07490851689748, 88.13230967927102, 88.12513453397432, 88.15383511516109, 88.19688598694124, 88.23993685872139, 88.31886345698501, 88.33321374757838, 88.40496520054532, 88.46236636291886, 88.505417234699, 88.56281839707253, 88.6058692688527, 88.65609528592955, 88.67762072181962, 88.691971012413, 88.70632130300639, 88.74219702948984, 88.77807275597331, 88.84264906364353, 88.89287508072039, 88.94310109779724, 89.03637798665423, 89.04355313195093, 89.08660400373107, 89.15118031140132, 89.20858147377486, 89.2659826361484, 89.31620865322523, 89.28750807203846, 89.3520843797087, 89.39513525148884, 89.41666068737892, 89.438186123269, 89.47406184975246, 89.46688670445577, 89.54581330271938, 89.624739900983, 89.66061562746646, 89.67496591805984, 89.73236708043338, 89.82564396929038, 89.98349716581761, 90.06242376408122, 90.16287579823492, 90.20592667001507, 90.32790414005885, 90.3637798665423, 90.43553131950922, 90.53598335366291, 90.67231111430006, 90.70101169548683, 90.70101169548683, 90.69383655019014, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.68803448480782, "threshold_area100": 87.683289086604, "threshold_area25": 84.3388103609098}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1087, "harmed": 383}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2225, "harmed": 729}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.6143991947174072, "operator": "ge"}, {"budget": 0.02, "threshold": 2.1917550563812256, "operator": "ge"}, {"budget": 0.03, "threshold": 1.8477084636688232, "operator": "ge"}, {"budget": 0.04, "threshold": 1.5999751091003418, "operator": "ge"}, {"budget": 0.05, "threshold": 1.4162168502807617, "operator": "ge"}, {"budget": 0.06, "threshold": 1.2315855026245117, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1119039058685303, "operator": "ge"}, {"budget": 0.08, "threshold": 1.0183032751083374, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9312806129455566, "operator": "ge"}, {"budget": 0.1, "threshold": 0.8551982641220093, "operator": "ge"}, {"budget": 0.11, "threshold": 0.7817307710647583, "operator": "ge"}, {"budget": 0.12, "threshold": 0.7094591856002808, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6372448801994324, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5616516470909119, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5006622076034546, "operator": "ge"}, {"budget": 0.16, "threshold": 0.4315449595451355, "operator": "ge"}, {"budget": 0.17, "threshold": 0.3716297149658203, "operator": "ge"}, {"budget": 0.18, "threshold": 0.3104792833328247, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2427484542131424, "operator": "ge"}, {"budget": 0.2, "threshold": 0.19476136565208435, "operator": "ge"}, {"budget": 0.21, "threshold": 0.14159247279167175, "operator": "ge"}, {"budget": 0.22, "threshold": 0.08786715567111969, "operator": "ge"}, {"budget": 0.23, "threshold": 0.039625152945518494, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0004989951848983765, "operator": "ge"}, {"budget": 0.25, "threshold": -0.04795902967453003, "operator": "ge"}, {"budget": 0.26, "threshold": -0.10286489129066467, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1544487476348877, "operator": "ge"}, {"budget": 0.28, "threshold": -0.2106933444738388, "operator": "ge"}, {"budget": 0.29, "threshold": -0.26306796073913574, "operator": "ge"}, {"budget": 0.3, "threshold": -0.34731346368789673, "operator": "ge"}, {"budget": 0.31, "threshold": -0.4114231765270233, "operator": "ge"}, {"budget": 0.32, "threshold": -0.4622241258621216, "operator": "ge"}, {"budget": 0.33, "threshold": -0.517564058303833, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5796453356742859, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6280306577682495, "operator": "ge"}, {"budget": 0.36, "threshold": -0.6994533538818359, "operator": "ge"}, {"budget": 0.37, "threshold": -0.7631089091300964, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8389581441879272, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9259989261627197, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9846653342247009, "operator": "ge"}, {"budget": 0.41, "threshold": -1.079922080039978, "operator": "ge"}, {"budget": 0.42, "threshold": -1.1571439504623413, "operator": "ge"}, {"budget": 0.43, "threshold": -1.2317063808441162, "operator": "ge"}, {"budget": 0.44, "threshold": -1.31443190574646, "operator": "ge"}, {"budget": 0.45, "threshold": -1.3861316442489624, "operator": "ge"}, {"budget": 0.46, "threshold": -1.4535765647888184, "operator": "ge"}, {"budget": 0.47, "threshold": -1.521194577217102, "operator": "ge"}, {"budget": 0.48, "threshold": -1.594057321548462, "operator": "ge"}, {"budget": 0.49, "threshold": -1.6676247119903564, "operator": "ge"}, {"budget": 0.5, "threshold": -1.7423464059829712, "operator": "ge"}, {"budget": 0.51, "threshold": -1.811171054840088, "operator": "ge"}, {"budget": 0.52, "threshold": -1.8826005458831787, "operator": "ge"}, {"budget": 0.53, "threshold": -1.9469577074050903, "operator": "ge"}, {"budget": 0.54, "threshold": -2.0163629055023193, "operator": "ge"}, {"budget": 0.55, "threshold": -2.080169200897217, "operator": "ge"}, {"budget": 0.56, "threshold": -2.1512293815612793, "operator": "ge"}, {"budget": 0.57, "threshold": -2.222127676010132, "operator": "ge"}, {"budget": 0.58, "threshold": -2.281824827194214, "operator": "ge"}, {"budget": 0.59, "threshold": -2.34586763381958, "operator": "ge"}, {"budget": 0.6, "threshold": -2.421055555343628, "operator": "ge"}, {"budget": 0.61, "threshold": -2.500234365463257, "operator": "ge"}, {"budget": 0.62, "threshold": -2.5784752368927, "operator": "ge"}, {"budget": 0.63, "threshold": -2.6362991333007812, "operator": "ge"}, {"budget": 0.64, "threshold": -2.707097053527832, "operator": "ge"}, {"budget": 0.65, "threshold": -2.760679006576538, "operator": "ge"}, {"budget": 0.66, "threshold": -2.835916042327881, "operator": "ge"}, {"budget": 0.67, "threshold": -2.9165048599243164, "operator": "ge"}, {"budget": 0.68, "threshold": -2.9922900199890137, "operator": "ge"}, {"budget": 0.69, "threshold": -3.067936420440674, "operator": "ge"}, {"budget": 0.7, "threshold": -3.1345441341400146, "operator": "ge"}, {"budget": 0.71, "threshold": -3.209646224975586, "operator": "ge"}, {"budget": 0.72, "threshold": -3.27290415763855, "operator": "ge"}, {"budget": 0.73, "threshold": -3.344179630279541, "operator": "ge"}, {"budget": 0.74, "threshold": -3.412776470184326, "operator": "ge"}, {"budget": 0.75, "threshold": -3.4885294437408447, "operator": "ge"}, {"budget": 0.76, "threshold": -3.5543899536132812, "operator": "ge"}, {"budget": 0.77, "threshold": -3.6029608249664307, "operator": "ge"}, {"budget": 0.78, "threshold": -3.6592860221862793, "operator": "ge"}, {"budget": 0.79, "threshold": -3.726844072341919, "operator": "ge"}, {"budget": 0.8, "threshold": -3.7917721271514893, "operator": "ge"}, {"budget": 0.81, "threshold": -3.8568859100341797, "operator": "ge"}, {"budget": 0.82, "threshold": -3.940647840499878, "operator": "ge"}, {"budget": 0.83, "threshold": -4.00895357131958, "operator": "ge"}, {"budget": 0.84, "threshold": -4.06782865524292, "operator": "ge"}, {"budget": 0.85, "threshold": -4.133095741271973, "operator": "ge"}, {"budget": 0.86, "threshold": -4.219880104064941, "operator": "ge"}, {"budget": 0.87, "threshold": -4.292804718017578, "operator": "ge"}, {"budget": 0.88, "threshold": -4.372105598449707, "operator": "ge"}, {"budget": 0.89, "threshold": -4.44080924987793, "operator": "ge"}, {"budget": 0.9, "threshold": -4.5181450843811035, "operator": "ge"}, {"budget": 0.91, "threshold": -4.612076759338379, "operator": "ge"}, {"budget": 0.92, "threshold": -4.689380645751953, "operator": "ge"}, {"budget": 0.93, "threshold": -4.780884742736816, "operator": "ge"}, {"budget": 0.94, "threshold": -4.893935680389404, "operator": "ge"}, {"budget": 0.95, "threshold": -5.006608963012695, "operator": "ge"}, {"budget": 0.96, "threshold": -5.153130531311035, "operator": "ge"}, {"budget": 0.97, "threshold": -5.294569969177246, "operator": "ge"}, {"budget": 0.98, "threshold": -5.4713521003723145, "operator": "ge"}, {"budget": 0.99, "threshold": -5.77486515045166, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.1775995416786, 80.50701804640504, 80.89372672586651, 81.25179031796046, 81.56688627900314, 81.88198224004583, 82.24004583213978, 82.52649670581495, 82.78430249212259, 83.11372099684904, 83.45746204525923, 83.71526783156689, 84.04468633629332, 84.41707247207103, 84.68920080206243, 84.7035233457462, 84.91836150100256, 85.03294185047264, 85.24778000572901, 85.44829561730162, 85.62016614150673, 85.86364938413061, 85.96390718991692, 86.1787453451733, 86.33629332569464, 86.42222858779719, 86.59409911200228, 86.6943569177886, 86.83758235462618, 86.86622744199369, 86.86622744199369, 87.03809796619879, 87.2099684904039, 87.35319392724148, 87.53938699513033, 87.63964480091664, 87.81151532512173, 87.9977083930106, 88.06932111142939, 88.2411916356345, 88.41306215983958, 88.5419650529934, 88.58493268404469, 88.68519048983099, 88.69951303351475, 88.75680320824978, 88.87138355771984, 88.90002864508736, 88.97164136350615, 89.00028645087367, 89.10054425665997, 89.07189916929245, 89.10054425665997, 89.15783443139502, 89.24376969349757, 89.3297049556001, 89.34402749928387, 89.37267258665138, 89.4013176740189, 89.45860784875393, 89.48725293612145, 89.51589802348897, 89.51589802348897, 89.53022056717272, 89.5445431108565, 89.573188198224, 89.61615582927529, 89.64480091664278, 89.58751074190776, 89.573188198224, 89.61615582927529, 89.68776854769406, 89.70209109137782, 89.73073617874535, 89.74505872242909, 89.77370380979661, 89.78802635348038, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.78802635348038, 89.73073617874535, 89.80234889716414, 89.8453165282154, 89.87396161558291, 89.85963907189917, 89.88828415926669, 89.9169292466342, 89.93125179031796, 89.90260670295044, 89.87396161558291, 89.95989687768548, 89.9455743340017, 89.97421942136923, 89.95989687768548, 89.97421942136923, 89.97421942136923, 89.9455743340017, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 33, 62, 95, 125, 153, 181, 210, 234, 258, 286, 315, 338, 365, 393, 416, 429, 451, 467, 486, 505, 522, 545, 556, 576, 591, 600, 615, 630, 643, 653, 662, 678, 696, 709, 725, 734, 751, 769, 780, 795, 810, 821, 831, 841, 850, 856, 867, 874, 880, 886, 894, 896, 903, 912, 921, 930, 933, 937, 942, 951, 956, 960, 964, 969, 974, 979, 985, 988, 989, 991, 996, 1003, 1008, 1012, 1013, 1016, 1021, 1025, 1027, 1029, 1031, 1034, 1041, 1045, 1050, 1055, 1057, 1060, 1065, 1067, 1068, 1075, 1076, 1080, 1083, 1085, 1086, 1086, 1087, 1087], "harmed": [0, 8, 14, 20, 25, 31, 37, 41, 45, 51, 56, 61, 66, 70, 72, 76, 88, 95, 103, 107, 112, 117, 123, 127, 132, 136, 139, 142, 150, 153, 161, 170, 174, 180, 183, 186, 188, 193, 198, 204, 207, 210, 212, 219, 222, 230, 232, 235, 240, 241, 245, 246, 250, 255, 260, 263, 266, 268, 270, 273, 278, 281, 283, 287, 291, 295, 298, 301, 302, 307, 310, 312, 314, 318, 320, 320, 321, 325, 330, 332, 332, 335, 342, 344, 345, 348, 354, 354, 355, 359, 363, 366, 367, 369, 371, 375, 376, 377, 379, 383, 383], "exact_score": [79.81953594958465, 80.1775995416786, 80.50701804640504, 80.89372672586651, 81.25179031796046, 81.56688627900314, 81.88198224004583, 82.24004583213978, 82.52649670581495, 82.78430249212259, 83.11372099684904, 83.45746204525923, 83.71526783156689, 84.04468633629332, 84.41707247207103, 84.68920080206243, 84.7035233457462, 84.91836150100256, 85.03294185047264, 85.24778000572901, 85.44829561730162, 85.62016614150673, 85.86364938413061, 85.96390718991692, 86.1787453451733, 86.33629332569464, 86.42222858779719, 86.59409911200228, 86.6943569177886, 86.83758235462618, 86.86622744199369, 86.86622744199369, 87.03809796619879, 87.2099684904039, 87.35319392724148, 87.53938699513033, 87.63964480091664, 87.81151532512173, 87.9977083930106, 88.06932111142939, 88.2411916356345, 88.41306215983958, 88.5419650529934, 88.58493268404469, 88.68519048983099, 88.69951303351475, 88.75680320824978, 88.87138355771984, 88.90002864508736, 88.97164136350615, 89.00028645087367, 89.10054425665997, 89.07189916929245, 89.10054425665997, 89.15783443139502, 89.24376969349757, 89.3297049556001, 89.34402749928387, 89.37267258665138, 89.4013176740189, 89.45860784875393, 89.48725293612145, 89.51589802348897, 89.51589802348897, 89.53022056717272, 89.5445431108565, 89.573188198224, 89.61615582927529, 89.64480091664278, 89.58751074190776, 89.573188198224, 89.61615582927529, 89.68776854769406, 89.70209109137782, 89.73073617874535, 89.74505872242909, 89.77370380979661, 89.78802635348038, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.78802635348038, 89.73073617874535, 89.80234889716414, 89.8453165282154, 89.87396161558291, 89.85963907189917, 89.88828415926669, 89.9169292466342, 89.93125179031796, 89.90260670295044, 89.87396161558291, 89.95989687768548, 89.9455743340017, 89.97421942136923, 89.95989687768548, 89.97421942136923, 89.97421942136923, 89.9455743340017, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.67729430041533, "threshold_area100": 87.67659696362074, "threshold_area25": 83.57920366657119}, "test": {"score": [80.00287005811867, 80.64145798952428, 81.07196670732583, 81.57422687809428, 82.04778646767598, 82.35631771543373, 82.77965128793858, 83.0451316639162, 83.34648776637728, 83.58326756116811, 83.8774485183325, 84.07835258663988, 84.46581043266126, 84.7025902274521, 85.06852263758341, 85.34835330415441, 85.59948338953865, 85.85778862021957, 86.10891870560378, 86.26677190213103, 86.48202626103179, 86.63270431226233, 86.80490779938295, 86.94123556002009, 87.0345124488771, 87.18519050010764, 87.33586855133817, 87.52242232905216, 87.64439979909594, 87.70897610676617, 87.82377843151325, 88.01750735452394, 88.02468249982063, 88.15383511516109, 88.15383511516109, 88.22558656812801, 88.45519121762216, 88.64892014063285, 88.77807275597331, 88.91440051661046, 88.99332711487408, 89.1440051661046, 89.27315778144508, 89.37360981559877, 89.4310109779723, 89.55298844801608, 89.65344048216977, 89.70366649924661, 89.75389251632346, 89.78976824280693, 89.811293678697, 89.84716940518047, 89.8973954222573, 89.96914687522423, 89.96914687522423, 90.04807347348785, 90.16287579823492, 90.22745210590514, 90.277678122982, 90.3566047212456, 90.37813015713569, 90.43553131950922, 90.44988161010261, 90.47858219128938, 90.51445791777284, 90.5503336442563, 90.52163306306953, 90.52163306306953, 90.54315849895961, 90.5503336442563, 90.57185908014638, 90.59338451603645, 90.61490995192652, 90.65078567840999, 90.62926024251992, 90.57903422544307, 90.57185908014638, 90.59338451603645, 90.57903422544307, 90.62208509722322, 90.65078567840999, 90.65078567840999, 90.67948625959676, 90.7297122766736, 90.75123771256368, 90.744062567267, 90.72253713137691, 90.744062567267, 90.78711343904715, 90.77276314845376, 90.78711343904715, 90.81581402023392, 90.82298916553061, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.80146372964053, 90.78711343904715, 90.75123771256368, 90.75841285786038, 90.73688742197031], "calls": [0.0, 1.0690966492071465, 1.9875152471837556, 3.171414221137978, 4.4342397933558155, 5.388534117815886, 6.558082801176724, 7.469326253856641, 8.315993398866327, 9.21288656095286, 10.260457774269929, 11.358255004663844, 12.45605223505776, 13.647126354308675, 14.816675037669514, 15.943172849250198, 17.11989667790773, 18.181818181818183, 19.171988232761713, 20.406113223792783, 21.346057257659467, 22.149673530889, 23.03221640238215, 23.89323383798522, 24.58922293176437, 25.464590657960823, 26.361483820047354, 27.200975819760348, 28.105044127143575, 28.930185836263185, 30.34368945971156, 31.147305732941096, 31.886345698500396, 32.740187988806774, 33.586855133816464, 34.22544306522207, 35.28736456913252, 36.24165889359259, 37.24617923512951, 38.437253354380424, 39.23369448231327, 40.46064432804764, 41.544091267848174, 42.55578675468178, 43.52443136973524, 44.56482743775562, 45.605223505775996, 46.48776637726914, 47.42771041113583, 48.54703307741982, 49.68788117959389, 50.67805123053741, 51.72562244385449, 52.75166822128148, 53.78488914400516, 54.703307741981774, 55.686302647628615, 56.69082298916553, 57.50161440769176, 58.44873358685513, 59.63263256080935, 60.89545813302719, 62.057831671091336, 62.85427279902418, 63.82291741407764, 64.67675970438401, 65.74585635359117, 66.74320154983138, 67.76207218196168, 68.87421970294898, 69.8859151897826, 70.97653727487982, 71.97388247112005, 73.05015426562387, 74.13360120542441, 75.31750017937863, 76.22156848676185, 76.98213388821124, 77.80010045203414, 78.78309535768099, 79.6943388103609, 80.67733371600775, 81.80383152758843, 82.71507498026835, 83.55456697998135, 84.36535839850757, 85.52055679127503, 86.59682858577887, 87.62287436320587, 88.42649063643539, 89.50993757623593, 90.6364353878166, 91.4759273875296, 92.37282054961614, 93.57106981416374, 94.39621152328334, 95.58011049723757, 96.53440482169763, 97.53175001793787, 98.77305015426563, 100.0], "rescued": [0, 96, 163, 245, 321, 372, 442, 484, 530, 573, 625, 669, 732, 778, 838, 890, 942, 989, 1028, 1064, 1103, 1133, 1165, 1196, 1219, 1251, 1279, 1312, 1338, 1360, 1394, 1426, 1442, 1468, 1481, 1496, 1537, 1566, 1587, 1616, 1635, 1664, 1690, 1713, 1727, 1748, 1770, 1782, 1795, 1809, 1822, 1834, 1846, 1864, 1876, 1890, 1908, 1925, 1935, 1952, 1961, 1974, 1980, 1990, 1999, 2006, 2010, 2017, 2025, 2031, 2038, 2047, 2054, 2063, 2068, 2078, 2079, 2086, 2090, 2102, 2109, 2112, 2120, 2129, 2135, 2141, 2145, 2152, 2159, 2165, 2171, 2180, 2186, 2188, 2194, 2199, 2207, 2210, 2210, 2217, 2225], "harmed": [0, 7, 14, 26, 36, 44, 55, 60, 64, 74, 85, 101, 110, 123, 132, 145, 162, 173, 177, 191, 200, 209, 217, 229, 239, 250, 257, 264, 273, 286, 304, 309, 324, 332, 345, 350, 359, 361, 364, 374, 382, 390, 398, 407, 413, 417, 425, 430, 436, 445, 455, 462, 467, 475, 487, 490, 492, 500, 503, 509, 515, 520, 524, 530, 534, 536, 544, 551, 556, 561, 565, 571, 575, 579, 587, 604, 606, 610, 616, 622, 625, 628, 632, 634, 637, 644, 651, 655, 656, 664, 668, 673, 678, 683, 689, 692, 702, 707, 712, 718, 729], "exact_score": [80.00287005811867, 80.59123197244745, 81.07914185262251, 81.53117600631413, 81.85405754466528, 82.22716510009327, 82.5930975102246, 82.90162875798235, 83.24603573222358, 83.5043409629045, 83.7841716294755, 83.99225084307957, 84.30078209083734, 84.56626246681496, 84.83174284279256, 85.14027409055034, 85.3698787400445, 85.56360766305518, 85.80038745784603, 86.08021812441702, 86.23807132094426, 86.41027480806487, 86.5681280045921, 86.79055750878956, 86.95558585061349, 87.06321303006386, 87.27846738896463, 87.48654660256871, 87.64439979909594, 87.71615125206286, 87.78072755973308, 87.98880677333716, 88.06055822630408, 88.1394848245677, 88.20406113223794, 88.38343976465524, 88.6058692688527, 88.74219702948984, 88.87134964483032, 88.93592595250054, 89.09377914902777, 89.17988089258809, 89.3520843797087, 89.40231039678554, 89.52428786682931, 89.60321446509292, 89.68214106335654, 89.72519193513669, 89.75389251632346, 89.78259309751023, 89.83281911458707, 89.87586998636723, 89.92609600344407, 89.96914687522423, 89.98349716581761, 90.0982994905647, 90.18440123412499, 90.23462725120184, 90.29920355887207, 90.37095501183899, 90.41400588361914, 90.44270646480591, 90.44988161010261, 90.48575733658608, 90.51445791777284, 90.56468393484968, 90.52880820836621, 90.52880820836621, 90.53598335366291, 90.5503336442563, 90.57185908014638, 90.59338451603645, 90.62208509722322, 90.6364353878166, 90.64361053311329, 90.57903422544307, 90.57185908014638, 90.59338451603645, 90.59338451603645, 90.62926024251992, 90.64361053311329, 90.65078567840999, 90.69383655019014, 90.744062567267, 90.75841285786038, 90.73688742197031, 90.7297122766736, 90.77276314845376, 90.80863887493722, 90.77993829375045, 90.80146372964053, 90.82298916553061, 90.80863887493722, 90.82298916553061, 90.81581402023392, 90.79428858434385, 90.80863887493722, 90.76558800315708, 90.75841285786038, 90.75123771256368, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.50421485600629, "threshold_area100": 88.5303867403315, "threshold_area25": 84.33163521561312}, "selected_budget": 94, "selected": {"val": {"score": 89.97421942136923, "calls": 93.9988541965053, "rescued": 1080, "harmed": 371}, "test": {"score": 90.80146372964053, "calls": 93.57106981416374, "rescued": 2194, "harmed": 689}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.8980143070220947, "operator": "ge"}, {"budget": 0.02, "threshold": 1.6026976108551025, "operator": "ge"}, {"budget": 0.03, "threshold": 1.3571300506591797, "operator": "ge"}, {"budget": 0.04, "threshold": 1.1714403629302979, "operator": "ge"}, {"budget": 0.05, "threshold": 1.0511729717254639, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9268496036529541, "operator": "ge"}, {"budget": 0.07, "threshold": 0.840255618095398, "operator": "ge"}, {"budget": 0.08, "threshold": 0.7668105363845825, "operator": "ge"}, {"budget": 0.09, "threshold": 0.7106098532676697, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6289467811584473, "operator": "ge"}, {"budget": 0.11, "threshold": 0.5720918774604797, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5191066265106201, "operator": "ge"}, {"budget": 0.13, "threshold": 0.46163612604141235, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4152769446372986, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3596423268318176, "operator": "ge"}, {"budget": 0.16, "threshold": 0.31131523847579956, "operator": "ge"}, {"budget": 0.17, "threshold": 0.2698381841182709, "operator": "ge"}, {"budget": 0.18, "threshold": 0.22424638271331787, "operator": "ge"}, {"budget": 0.19, "threshold": 0.18298399448394775, "operator": "ge"}, {"budget": 0.2, "threshold": 0.14592355489730835, "operator": "ge"}, {"budget": 0.21, "threshold": 0.10699942708015442, "operator": "ge"}, {"budget": 0.22, "threshold": 0.06839002668857574, "operator": "ge"}, {"budget": 0.23, "threshold": 0.029359012842178345, "operator": "ge"}, {"budget": 0.24, "threshold": -0.000383719801902771, "operator": "ge"}, {"budget": 0.25, "threshold": -0.035890430212020874, "operator": "ge"}, {"budget": 0.26, "threshold": -0.0761827677488327, "operator": "ge"}, {"budget": 0.27, "threshold": -0.11665931344032288, "operator": "ge"}, {"budget": 0.28, "threshold": -0.15517565608024597, "operator": "ge"}, {"budget": 0.29, "threshold": -0.211217999458313, "operator": "ge"}, {"budget": 0.3, "threshold": -0.27194857597351074, "operator": "ge"}, {"budget": 0.31, "threshold": -0.31212449073791504, "operator": "ge"}, {"budget": 0.32, "threshold": -0.3715284764766693, "operator": "ge"}, {"budget": 0.33, "threshold": -0.4243276119232178, "operator": "ge"}, {"budget": 0.34, "threshold": -0.48406514525413513, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5317854881286621, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5798271894454956, "operator": "ge"}, {"budget": 0.37, "threshold": -0.6509353518486023, "operator": "ge"}, {"budget": 0.38, "threshold": -0.7219722270965576, "operator": "ge"}, {"budget": 0.39, "threshold": -0.7884409427642822, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8599035739898682, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9321580529212952, "operator": "ge"}, {"budget": 0.42, "threshold": -1.012503743171692, "operator": "ge"}, {"budget": 0.43, "threshold": -1.0862822532653809, "operator": "ge"}, {"budget": 0.44, "threshold": -1.1648399829864502, "operator": "ge"}, {"budget": 0.45, "threshold": -1.2331873178482056, "operator": "ge"}, {"budget": 0.46, "threshold": -1.3130152225494385, "operator": "ge"}, {"budget": 0.47, "threshold": -1.38614821434021, "operator": "ge"}, {"budget": 0.48, "threshold": -1.4478124380111694, "operator": "ge"}, {"budget": 0.49, "threshold": -1.5214203596115112, "operator": "ge"}, {"budget": 0.5, "threshold": -1.585737705230713, "operator": "ge"}, {"budget": 0.51, "threshold": -1.6543333530426025, "operator": "ge"}, {"budget": 0.52, "threshold": -1.7319440841674805, "operator": "ge"}, {"budget": 0.53, "threshold": -1.8029192686080933, "operator": "ge"}, {"budget": 0.54, "threshold": -1.866876482963562, "operator": "ge"}, {"budget": 0.55, "threshold": -1.9230092763900757, "operator": "ge"}, {"budget": 0.56, "threshold": -1.997918963432312, "operator": "ge"}, {"budget": 0.57, "threshold": -2.057482957839966, "operator": "ge"}, {"budget": 0.58, "threshold": -2.1195924282073975, "operator": "ge"}, {"budget": 0.59, "threshold": -2.182864189147949, "operator": "ge"}, {"budget": 0.6, "threshold": -2.249969005584717, "operator": "ge"}, {"budget": 0.61, "threshold": -2.339564085006714, "operator": "ge"}, {"budget": 0.62, "threshold": -2.411167860031128, "operator": "ge"}, {"budget": 0.63, "threshold": -2.4709067344665527, "operator": "ge"}, {"budget": 0.64, "threshold": -2.5307071208953857, "operator": "ge"}, {"budget": 0.65, "threshold": -2.599306106567383, "operator": "ge"}, {"budget": 0.66, "threshold": -2.6652307510375977, "operator": "ge"}, {"budget": 0.67, "threshold": -2.747002601623535, "operator": "ge"}, {"budget": 0.68, "threshold": -2.8233938217163086, "operator": "ge"}, {"budget": 0.69, "threshold": -2.8977460861206055, "operator": "ge"}, {"budget": 0.7, "threshold": -2.9638803005218506, "operator": "ge"}, {"budget": 0.71, "threshold": -3.035574436187744, "operator": "ge"}, {"budget": 0.72, "threshold": -3.1037096977233887, "operator": "ge"}, {"budget": 0.73, "threshold": -3.169534921646118, "operator": "ge"}, {"budget": 0.74, "threshold": -3.231717824935913, "operator": "ge"}, {"budget": 0.75, "threshold": -3.299100875854492, "operator": "ge"}, {"budget": 0.76, "threshold": -3.367093086242676, "operator": "ge"}, {"budget": 0.77, "threshold": -3.420340061187744, "operator": "ge"}, {"budget": 0.78, "threshold": -3.4804041385650635, "operator": "ge"}, {"budget": 0.79, "threshold": -3.54866886138916, "operator": "ge"}, {"budget": 0.8, "threshold": -3.602893352508545, "operator": "ge"}, {"budget": 0.81, "threshold": -3.674208641052246, "operator": "ge"}, {"budget": 0.82, "threshold": -3.7478480339050293, "operator": "ge"}, {"budget": 0.83, "threshold": -3.8142333030700684, "operator": "ge"}, {"budget": 0.84, "threshold": -3.873462677001953, "operator": "ge"}, {"budget": 0.85, "threshold": -3.943634510040283, "operator": "ge"}, {"budget": 0.86, "threshold": -4.023928642272949, "operator": "ge"}, {"budget": 0.87, "threshold": -4.103893756866455, "operator": "ge"}, {"budget": 0.88, "threshold": -4.187721252441406, "operator": "ge"}, {"budget": 0.89, "threshold": -4.254486083984375, "operator": "ge"}, {"budget": 0.9, "threshold": -4.3158745765686035, "operator": "ge"}, {"budget": 0.91, "threshold": -4.392045974731445, "operator": "ge"}, {"budget": 0.92, "threshold": -4.498641014099121, "operator": "ge"}, {"budget": 0.93, "threshold": -4.590198993682861, "operator": "ge"}, {"budget": 0.94, "threshold": -4.696394443511963, "operator": "ge"}, {"budget": 0.95, "threshold": -4.800943374633789, "operator": "ge"}, {"budget": 0.96, "threshold": -4.935001373291016, "operator": "ge"}, {"budget": 0.97, "threshold": -5.092187404632568, "operator": "ge"}, {"budget": 0.98, "threshold": -5.258648872375488, "operator": "ge"}, {"budget": 0.99, "threshold": -5.543411731719971, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.2492122600974, 80.46405041535377, 80.89372672586651, 81.25179031796046, 81.52391864795187, 81.86765969636207, 82.19707820108852, 82.51217416213119, 82.74133486107132, 83.21397880263535, 83.5720423947293, 83.78688054998568, 84.07333142366083, 84.30249212260097, 84.61758808364365, 84.8037811515325, 85.00429676310513, 85.06158693784016, 85.20481237467774, 85.47694070466915, 85.62016614150673, 85.86364938413061, 85.96390718991692, 86.1787453451733, 86.29332569464336, 86.37926095674591, 86.57977656831852, 86.68003437410484, 86.78029217989113, 86.83758235462618, 86.95216270409625, 87.08106559725006, 87.26725866513893, 87.39616155829275, 87.5107419077628, 87.62532225723288, 87.854482956173, 87.95474076195931, 88.15525637353193, 88.29848181036952, 88.4417072472071, 88.62790031509596, 88.68519048983099, 88.65654540246346, 88.71383557719851, 88.77112575193354, 88.84273847035233, 88.85706101403609, 88.95731881982239, 88.97164136350615, 89.07189916929245, 89.15783443139502, 89.20080206244629, 89.18647951876252, 89.31538241191636, 89.31538241191636, 89.37267258665138, 89.45860784875393, 89.47293039243769, 89.51589802348897, 89.47293039243769, 89.573188198224, 89.58751074190776, 89.58751074190776, 89.60183328559151, 89.573188198224, 89.58751074190776, 89.60183328559151, 89.55886565454024, 89.60183328559151, 89.6734460040103, 89.73073617874535, 89.71641363506157, 89.73073617874535, 89.74505872242909, 89.75938126611285, 89.77370380979661, 89.80234889716414, 89.81667144084788, 89.78802635348038, 89.78802635348038, 89.81667144084788, 89.78802635348038, 89.80234889716414, 89.87396161558291, 89.90260670295044, 89.88828415926669, 89.9455743340017, 89.9455743340017, 89.90260670295044, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.97421942136923, 90.00286450873675, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 35, 59, 92, 125, 152, 179, 205, 232, 252, 288, 317, 341, 364, 384, 410, 432, 451, 467, 484, 507, 520, 544, 556, 576, 590, 598, 615, 628, 642, 654, 669, 681, 698, 713, 726, 737, 755, 769, 785, 801, 812, 830, 838, 843, 851, 862, 868, 874, 882, 886, 895, 903, 909, 913, 926, 930, 936, 944, 948, 955, 958, 966, 969, 974, 979, 981, 985, 990, 990, 994, 1000, 1006, 1010, 1012, 1014, 1017, 1022, 1026, 1029, 1030, 1035, 1039, 1041, 1044, 1050, 1056, 1058, 1062, 1066, 1068, 1072, 1075, 1076, 1080, 1084, 1085, 1086, 1086, 1087, 1087], "harmed": [0, 5, 14, 17, 25, 33, 36, 39, 44, 48, 51, 55, 64, 67, 71, 75, 84, 89, 101, 108, 112, 115, 122, 127, 132, 138, 140, 143, 149, 156, 164, 171, 174, 178, 184, 189, 192, 194, 201, 203, 209, 210, 215, 219, 226, 230, 237, 238, 243, 244, 247, 249, 251, 254, 259, 263, 267, 269, 271, 274, 278, 284, 285, 287, 292, 296, 300, 303, 307, 310, 311, 312, 314, 319, 320, 321, 323, 327, 329, 331, 334, 339, 341, 345, 347, 348, 352, 355, 355, 359, 364, 366, 368, 369, 371, 373, 376, 378, 380, 383, 383], "exact_score": [79.81953594958465, 80.2492122600974, 80.46405041535377, 80.89372672586651, 81.25179031796046, 81.52391864795187, 81.86765969636207, 82.19707820108852, 82.51217416213119, 82.74133486107132, 83.21397880263535, 83.5720423947293, 83.78688054998568, 84.07333142366083, 84.30249212260097, 84.61758808364365, 84.8037811515325, 85.00429676310513, 85.06158693784016, 85.20481237467774, 85.47694070466915, 85.62016614150673, 85.86364938413061, 85.96390718991692, 86.1787453451733, 86.29332569464336, 86.37926095674591, 86.57977656831852, 86.68003437410484, 86.78029217989113, 86.83758235462618, 86.95216270409625, 87.08106559725006, 87.26725866513893, 87.39616155829275, 87.5107419077628, 87.62532225723288, 87.854482956173, 87.95474076195931, 88.15525637353193, 88.29848181036952, 88.4417072472071, 88.62790031509596, 88.68519048983099, 88.65654540246346, 88.71383557719851, 88.77112575193354, 88.84273847035233, 88.85706101403609, 88.95731881982239, 88.97164136350615, 89.07189916929245, 89.15783443139502, 89.20080206244629, 89.18647951876252, 89.31538241191636, 89.31538241191636, 89.37267258665138, 89.45860784875393, 89.47293039243769, 89.51589802348897, 89.47293039243769, 89.573188198224, 89.58751074190776, 89.58751074190776, 89.60183328559151, 89.573188198224, 89.58751074190776, 89.60183328559151, 89.55886565454024, 89.60183328559151, 89.6734460040103, 89.73073617874535, 89.71641363506157, 89.73073617874535, 89.74505872242909, 89.75938126611285, 89.77370380979661, 89.80234889716414, 89.81667144084788, 89.78802635348038, 89.78802635348038, 89.81667144084788, 89.78802635348038, 89.80234889716414, 89.87396161558291, 89.90260670295044, 89.88828415926669, 89.9455743340017, 89.9455743340017, 89.90260670295044, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.97421942136923, 90.00286450873675, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.69117990600044, "threshold_area100": 87.69048983099398, "threshold_area25": 83.5777714122028}, "test": {"score": [80.00287005811867, 80.57688168185406, 81.1006672885126, 81.57422687809428, 82.01191074119252, 82.3850182966205, 82.71507498026835, 83.03795651861951, 83.28191145870704, 83.5115161082012, 83.90614909951927, 84.20033005668365, 84.39405897969434, 84.73129080863887, 84.97524574872641, 85.37705388534118, 85.61383368013202, 85.8290880390328, 86.11609385090048, 86.2811221927244, 86.49637655162518, 86.61117887637225, 86.78338236349286, 86.96276099591017, 87.0345124488771, 87.19236564540432, 87.35021884193155, 87.48654660256871, 87.62287436320587, 87.63722465379924, 87.83095357680993, 87.9601061921504, 88.0462079357107, 88.08208366219417, 88.16101026045777, 88.26146229461148, 88.49824208940231, 88.67044557652292, 88.84264906364353, 88.95027624309392, 89.02920284135753, 89.13683002080792, 89.21575661907154, 89.3377340891153, 89.50276243093923, 89.60321446509292, 89.67496591805984, 89.75389251632346, 89.75389251632346, 89.811293678697, 89.83281911458707, 89.84716940518047, 89.904570567554, 89.99067231111431, 90.00502260170768, 90.09112434526799, 90.19875152471838, 90.23462725120184, 90.32072899476215, 90.3566047212456, 90.37095501183899, 90.44270646480591, 90.48575733658608, 90.48575733658608, 90.52163306306953, 90.51445791777284, 90.5503336442563, 90.5503336442563, 90.53598335366291, 90.57903422544307, 90.58620937073975, 90.60773480662984, 90.62926024251992, 90.65078567840999, 90.60055966133315, 90.58620937073975, 90.58620937073975, 90.58620937073975, 90.59338451603645, 90.6364353878166, 90.67948625959676, 90.68666140489346, 90.70818684078354, 90.70818684078354, 90.7297122766736, 90.744062567267, 90.744062567267, 90.79428858434385, 90.80863887493722, 90.83733945612398, 90.82298916553061, 90.77993829375045, 90.84451460142068, 90.84451460142068, 90.83733945612398, 90.80863887493722, 90.79428858434385, 90.77993829375045, 90.744062567267, 90.75123771256368, 90.73688742197031], "calls": [0.0, 0.9758197603501471, 1.9803401018870632, 3.2933916911817462, 4.5418669728062, 5.460285570782808, 6.572433091770108, 7.4836765444500255, 8.280117672382865, 9.11960967209586, 10.511587859654158, 11.544808782377842, 12.592379995694913, 13.848030422616059, 14.93147736241659, 16.208653225227813, 17.327975891511805, 18.30379565186195, 19.50922006170625, 20.406113223792783, 21.303006385879314, 22.135323240295616, 23.060916983568916, 23.89323383798522, 24.59639807706106, 25.464590657960823, 26.332783238860586, 27.17227523857358, 27.918490349429575, 29.04498816101026, 30.142785391404175, 30.932051374040327, 31.82894453612686, 32.66843653583985, 33.64425629619, 34.41917198823276, 35.165387099088754, 36.3277606371529, 37.4470833034369, 38.35115161082012, 39.31262108057688, 40.38171772978403, 41.49386525077133, 42.491210447011554, 43.53160651503193, 44.42132453182177, 45.626748941666065, 46.58821841142283, 47.520987299992825, 48.62595967568343, 49.479801965989815, 50.56324890579034, 51.840424768601565, 52.97409772547894, 53.91404175934563, 54.703307741981774, 55.80110497237569, 56.67647269857214, 57.54466527947191, 58.549185621008824, 59.41737820190859, 60.97438473129081, 62.057831671091336, 62.81839707254072, 63.672239362847094, 64.60500825141709, 65.5306019946904, 66.77907727631485, 67.84817392552199, 68.98184688239937, 69.90026548037598, 70.92631125780296, 72.05998421468036, 73.16495659037095, 74.18382722250125, 75.23857358111502, 76.1785176149817, 77.05388534117816, 77.91490277678122, 78.91224797302145, 79.65846308387745, 80.69168400660114, 81.73208007462152, 82.6648489631915, 83.43976465523427, 84.4299347061778, 85.50620650068164, 86.61835402166894, 87.68745067087609, 88.53411781588576, 89.30185836263185, 90.31355384946545, 91.59072971227667, 92.45892229317644, 93.54236923297697, 94.48231326684365, 95.58011049723757, 96.62768171055464, 97.57480088971802, 98.75152471837555, 100.0], "rescued": [0, 88, 167, 247, 318, 377, 432, 483, 522, 561, 629, 679, 723, 783, 832, 897, 942, 985, 1030, 1067, 1101, 1129, 1164, 1198, 1219, 1252, 1282, 1309, 1335, 1357, 1394, 1418, 1444, 1466, 1486, 1507, 1543, 1574, 1601, 1624, 1647, 1668, 1687, 1710, 1739, 1758, 1776, 1792, 1801, 1816, 1827, 1839, 1857, 1875, 1882, 1897, 1918, 1928, 1943, 1951, 1957, 1978, 1989, 1994, 2002, 2005, 2014, 2023, 2027, 2036, 2040, 2048, 2055, 2065, 2069, 2076, 2084, 2089, 2095, 2103, 2113, 2118, 2125, 2129, 2135, 2141, 2148, 2158, 2164, 2169, 2173, 2175, 2189, 2192, 2197, 2200, 2207, 2209, 2212, 2217, 2225], "harmed": [0, 8, 14, 28, 38, 45, 54, 60, 65, 72, 85, 94, 111, 124, 139, 148, 160, 173, 178, 192, 196, 208, 219, 228, 239, 250, 258, 266, 273, 293, 303, 309, 323, 340, 349, 356, 359, 366, 369, 377, 389, 395, 403, 409, 415, 420, 428, 433, 442, 449, 457, 467, 477, 483, 488, 491, 497, 502, 505, 508, 512, 523, 528, 533, 536, 540, 544, 553, 559, 562, 565, 570, 574, 581, 592, 601, 609, 614, 619, 621, 625, 629, 633, 637, 640, 644, 651, 654, 658, 659, 665, 673, 678, 681, 687, 694, 703, 707, 715, 719, 729], "exact_score": [80.00287005811867, 80.59840711774415, 81.10784243380928, 81.4594245533472, 81.81818181818183, 82.2056396642032, 82.55722178374111, 82.84422759560881, 83.19580971514674, 83.47564038171772, 83.7841716294755, 84.02095142426634, 84.27925665494726, 84.45863528736457, 84.75999138982566, 84.99677118461649, 85.30530243237426, 85.55643251775848, 85.75733658606588, 86.0084666714501, 86.24524646624094, 86.41744995336155, 86.59682858577887, 86.79055750878956, 86.95558585061349, 87.06321303006386, 87.3143431154481, 87.46502116667862, 87.62287436320587, 87.63722465379924, 87.82377843151325, 87.96728133744709, 88.06773337160078, 88.14665996986439, 88.21841142283131, 88.44084092702877, 88.64174499533615, 88.77807275597331, 88.93592595250054, 88.971801678984, 89.0722537131377, 89.18705603788477, 89.28750807203846, 89.39513525148884, 89.54581330271938, 89.65344048216977, 89.68931620865322, 89.73954222573008, 89.8041185334003, 89.78259309751023, 89.84716940518047, 89.85434455047715, 89.91892085814737, 89.99067231111431, 90.01937289230108, 90.0982994905647, 90.19157637942169, 90.26332783238861, 90.34942957594892, 90.3566047212456, 90.39965559302576, 90.44270646480591, 90.48575733658608, 90.47858219128938, 90.50010762717945, 90.51445791777284, 90.54315849895961, 90.55750878955298, 90.5503336442563, 90.57903422544307, 90.58620937073975, 90.61490995192652, 90.62926024251992, 90.65078567840999, 90.61490995192652, 90.58620937073975, 90.57185908014638, 90.57903422544307, 90.60773480662984, 90.66513596900337, 90.67948625959676, 90.68666140489346, 90.71536198608023, 90.72253713137691, 90.73688742197031, 90.73688742197031, 90.75123771256368, 90.78711343904715, 90.80146372964053, 90.80146372964053, 90.78711343904715, 90.8301643108273, 90.85886489201407, 90.84451460142068, 90.85886489201407, 90.75841285786038, 90.78711343904715, 90.76558800315708, 90.75123771256368, 90.75123771256368, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.50314169891574, "threshold_area100": 88.53619860802182, "threshold_area25": 84.35244313697352}, "selected_budget": 95, "selected": {"val": {"score": 90.00286450873675, "calls": 94.98710971068462, "rescued": 1084, "harmed": 373}, "test": {"score": 90.80863887493722, "calls": 94.48231326684365, "rescued": 2200, "harmed": 694}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.9455743340017, "calls": 98.99742194213692}, "test": {"score": 90.71536198608023, "calls": 99.01700509435317}}, "old_normalized_interior": {"val": {"score": 90.00286450873675, "calls": 94.98710971068462}, "test": {"score": 90.85168974671738, "calls": 94.50383870273373}}, "old_normalized_endpoints": {"val": {"score": 90.00286450873675, "calls": 94.98710971068462}, "test": {"score": 90.85168974671738, "calls": 94.50383870273373}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.14895445431108, 80.59295330850759, 80.93669435691778, 81.3090804926955, 81.6528215411057, 81.89630478372959, 82.21140074477226, 82.48352907476367, 82.74133486107132, 83.02778573474649, 83.28559152105413, 83.52907476367803, 83.8441707247207, 84.15926668576338, 84.4313950157548, 84.775136064165, 85.07590948152391, 85.26210254941276, 85.36236035519907, 85.43397307361786, 85.54855342308794, 85.73474649097679, 85.96390718991692, 86.1071326267545, 86.12145517043827, 86.26468060727585, 86.33629332569464, 86.49384130621598, 86.6227441993698, 86.78029217989113, 86.99513033514752, 87.09538814093382, 87.10971068461758, 87.33887138355772, 87.46777427671154, 87.69693497565167, 87.76854769407046, 87.7542251503867, 87.76854769407046, 87.91177313090805, 87.9977083930106, 88.06932111142939, 88.22686909195072, 88.28415926668576, 88.45602979089085, 88.5133199656259, 88.6135777714122, 88.69951303351475, 88.74248066456603, 88.77112575193354, 88.84273847035233, 88.94299627613864, 89.00028645087367, 89.17215697507876, 89.21512460613005, 89.31538241191636, 89.37267258665138, 89.38699513033515, 89.41564021770266, 89.47293039243769, 89.47293039243769, 89.48725293612145, 89.50157547980521, 89.64480091664278, 89.6734460040103, 89.65912346032655, 89.61615582927529, 89.58751074190776, 89.55886565454024, 89.61615582927529, 89.573188198224, 89.58751074190776, 89.58751074190776, 89.61615582927529, 89.63047837295903, 89.64480091664278, 89.70209109137782, 89.73073617874535, 89.75938126611285, 89.80234889716414, 89.75938126611285, 89.83099398453164, 89.83099398453164, 89.81667144084788, 89.87396161558291, 89.9455743340017, 89.98854196505299, 89.98854196505299, 89.9455743340017, 89.97421942136923, 89.93125179031796, 89.97421942136923, 89.95989687768548, 89.98854196505299, 90.00286450873675, 89.97421942136923, 89.9169292466342, 89.9169292466342, 89.93125179031796, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.14895445431108, 80.59295330850759, 80.93669435691778, 81.3090804926955, 81.6528215411057, 81.89630478372959, 82.21140074477226, 82.48352907476367, 82.74133486107132, 83.02778573474649, 83.28559152105413, 83.52907476367803, 83.8441707247207, 84.15926668576338, 84.4313950157548, 84.775136064165, 85.07590948152391, 85.26210254941276, 85.36236035519907, 85.43397307361786, 85.54855342308794, 85.73474649097679, 85.96390718991692, 86.1071326267545, 86.12145517043827, 86.26468060727585, 86.33629332569464, 86.49384130621598, 86.6227441993698, 86.78029217989113, 86.99513033514752, 87.09538814093382, 87.10971068461758, 87.33887138355772, 87.46777427671154, 87.69693497565167, 87.76854769407046, 87.7542251503867, 87.76854769407046, 87.91177313090805, 87.9977083930106, 88.06932111142939, 88.22686909195072, 88.28415926668576, 88.45602979089085, 88.5133199656259, 88.6135777714122, 88.69951303351475, 88.74248066456603, 88.77112575193354, 88.84273847035233, 88.94299627613864, 89.00028645087367, 89.17215697507876, 89.21512460613005, 89.31538241191636, 89.37267258665138, 89.38699513033515, 89.41564021770266, 89.47293039243769, 89.47293039243769, 89.48725293612145, 89.50157547980521, 89.64480091664278, 89.6734460040103, 89.65912346032655, 89.61615582927529, 89.58751074190776, 89.55886565454024, 89.61615582927529, 89.573188198224, 89.58751074190776, 89.58751074190776, 89.61615582927529, 89.63047837295903, 89.64480091664278, 89.70209109137782, 89.73073617874535, 89.75938126611285, 89.80234889716414, 89.75938126611285, 89.83099398453164, 89.83099398453164, 89.81667144084788, 89.87396161558291, 89.9455743340017, 89.98854196505299, 89.98854196505299, 89.9455743340017, 89.97421942136923, 89.93125179031796, 89.97421942136923, 89.95989687768548, 89.98854196505299, 90.00286450873675, 89.97421942136923, 89.9169292466342, 89.9169292466342, 89.93125179031796, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 83.50329418504727, "exact_area100": 87.61487574424096}, "test": {"score": [80.00287005811867, 80.49077993829376, 81.10784243380928, 81.60292745928105, 82.09801248475283, 82.35631771543373, 82.58592236492788, 82.8585778862022, 83.07383224510296, 83.43976465523427, 83.71242017650857, 83.99942598837626, 84.22903063787042, 84.60931333859511, 84.8173925521992, 85.0828729281768, 85.38422903063787, 85.68558513309895, 85.84343832962618, 86.00129152615341, 86.21654588505417, 86.3815742268781, 86.53942742340533, 86.71880605582263, 86.89818468823994, 87.1134390471407, 87.23541651718448, 87.33586855133817, 87.47219631197532, 87.60134892731578, 87.70180096146947, 87.8596541579967, 87.97445648274378, 88.11795938867762, 88.24711200401808, 88.38343976465524, 88.505417234699, 88.62739470474278, 88.691971012413, 88.78524790127, 88.8856999354237, 88.95027624309392, 89.065078567841, 89.17270574729139, 89.23728205496163, 89.38796010619215, 89.51711272153261, 89.54581330271938, 89.59603931979623, 89.64626533687307, 89.72519193513669, 89.811293678697, 89.84716940518047, 89.93327114874077, 89.97632202052091, 89.97632202052091, 90.07677405467462, 90.09112434526799, 90.20592667001507, 90.22745210590514, 90.29920355887207, 90.33507928535553, 90.3637798665423, 90.37095501183899, 90.39965559302576, 90.37813015713569, 90.41400588361914, 90.39248044772907, 90.42118102891584, 90.47140704599268, 90.51445791777284, 90.53598335366291, 90.57903422544307, 90.57185908014638, 90.59338451603645, 90.64361053311329, 90.67231111430006, 90.68666140489346, 90.7297122766736, 90.7297122766736, 90.72253713137691, 90.744062567267, 90.73688742197031, 90.744062567267, 90.76558800315708, 90.82298916553061, 90.80863887493722, 90.81581402023392, 90.82298916553061, 90.83733945612398, 90.85886489201407, 90.85168974671738, 90.85886489201407, 90.88039032790415, 90.8301643108273, 90.85168974671738, 90.85168974671738, 90.85168974671738, 90.82298916553061, 90.75841285786038, 90.73688742197031], "calls": [0.0, 0.8753677261964555, 2.087967281337447, 3.3507928535552844, 4.649494152256583, 5.582263040826577, 6.457630767023033, 7.411925091483103, 8.481021740690249, 9.514242663413935, 10.411135825500466, 11.501757910597691, 12.398651072684222, 13.553849465451675, 14.515318935208438, 15.34763578962474, 16.56741049006242, 17.600631412786107, 18.583626318432948, 19.595321805266558, 20.55679127502332, 21.561311616560236, 22.36492788978977, 23.33357250484322, 24.25199110281983, 25.091483102532823, 26.05295257228959, 27.01442204204635, 27.975891511803113, 28.82973380210949, 29.740977254789406, 30.66657099806271, 31.656741049006243, 32.59668508287293, 33.46487766377269, 34.361770825859224, 35.28736456913252, 36.29906005596613, 37.533185046997204, 38.65968285857789, 39.958384157279184, 40.99160508000287, 42.10375260099017, 43.072397216043626, 44.313697352371385, 45.5047714716223, 46.509291813159216, 47.355958958168905, 48.18110066728851, 48.99189208581474, 50.11121475209873, 51.13726052952572, 52.42878668293033, 53.3830810073904, 54.34455047714716, 55.234268493937, 56.31771543373753, 57.200258305230676, 58.24782951854775, 59.41737820190859, 60.75913037239003, 61.67037382506996, 62.91884910669441, 63.81574226878094, 64.62653368730716, 65.68128004592093, 66.72167611394131, 67.53964267776422, 68.60873932697137, 69.57738394202482, 70.38817536055106, 71.16309105259381, 72.06715935997704, 73.0286288297338, 73.90399655593026, 74.87264117098371, 75.79105976896032, 76.70947836693693, 77.69964841788047, 78.58219128937361, 79.24230465666929, 80.01004520341537, 80.97868981846882, 81.74643036521489, 82.77965128793858, 84.0640022960465, 85.0828729281768, 86.13761928679055, 87.0345124488771, 88.00315706393054, 89.0579034225443, 90.39965559302576, 91.3754753533759, 92.47327258376981, 93.55671952357035, 94.50383870273373, 95.45813302719381, 96.55593025758772, 97.5030494367511, 98.64389753892516, 100.0], "exact_score": [80.00287005811867, 80.54818110066729, 81.02891583554567, 81.4665996986439, 81.79665638229174, 82.17693908301644, 82.46394489488412, 82.73660041615844, 82.93032933916912, 83.2173351510368, 83.55456697998135, 83.84874793714573, 84.12140345842005, 84.4299347061778, 84.73846595393557, 84.96807060342972, 85.22637583411064, 85.48468106479157, 85.76451173136256, 85.92236492788979, 86.05869268852695, 86.2882973380211, 86.47485111573509, 86.61117887637225, 86.8336083805697, 87.09191361125063, 87.22824137188778, 87.32151826074478, 87.47219631197532, 87.665925234986, 87.76637726913971, 87.86682930329339, 88.02468249982063, 88.16818540575447, 88.31886345698501, 88.46954150821554, 88.57716868766593, 88.67762072181962, 88.72067159359977, 88.81394848245677, 88.8856999354237, 88.94310109779724, 89.03637798665423, 89.17988089258809, 89.23728205496163, 89.35925952500538, 89.48123699504916, 89.5386381574227, 89.58886417449953, 89.64626533687307, 89.73236708043338, 89.8041185334003, 89.811293678697, 89.8973954222573, 89.94762143933414, 89.98349716581761, 90.04807347348785, 90.0839491999713, 90.1700509435316, 90.23462725120184, 90.24180239649853, 90.29920355887207, 90.34225443065222, 90.3566047212456, 90.37813015713569, 90.39248044772907, 90.3566047212456, 90.38530530243239, 90.42118102891584, 90.43553131950922, 90.50728277247615, 90.52880820836621, 90.56468393484968, 90.57185908014638, 90.60773480662984, 90.65078567840999, 90.67231111430006, 90.68666140489346, 90.72253713137691, 90.7297122766736, 90.744062567267, 90.73688742197031, 90.73688742197031, 90.78711343904715, 90.82298916553061, 90.80863887493722, 90.81581402023392, 90.82298916553061, 90.8301643108273, 90.85886489201407, 90.85886489201407, 90.83733945612398, 90.88039032790415, 90.87321518260745, 90.86604003731075, 90.85168974671738, 90.85168974671738, 90.84451460142068, 90.78711343904715, 90.75123771256368, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 84.1852622515606, "exact_area100": 88.4311052891488}}, "old_normalized_endpoint_budget": 95}, {"seed": 2027, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 1.0, "validation_area100": 0.8767670863925495, "training_seconds": 30.570854755002074, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8774552498952365, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8165023852423694, "operator": "ge"}, {"budget": 0.03, "threshold": 0.753329025523615, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6894347902327229, "operator": "ge"}, {"budget": 0.05, "threshold": 0.6329239251104694, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5862724199793191, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5360197694423353, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4956878923828348, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4519372056372569, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4167258147995879, "operator": "ge"}, {"budget": 0.11, "threshold": 0.3721659256383154, "operator": "ge"}, {"budget": 0.12, "threshold": 0.33920914223181403, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3049954934941409, "operator": "ge"}, {"budget": 0.14, "threshold": 0.27925295761221625, "operator": "ge"}, {"budget": 0.15, "threshold": 0.2443293234784915, "operator": "ge"}, {"budget": 0.16, "threshold": 0.212585621883959, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1808737284052832, "operator": "ge"}, {"budget": 0.18, "threshold": 0.15835688067942416, "operator": "ge"}, {"budget": 0.19, "threshold": 0.13022656067665914, "operator": "ge"}, {"budget": 0.2, "threshold": 0.10557947225769948, "operator": "ge"}, {"budget": 0.21, "threshold": 0.074338197424024, "operator": "ge"}, {"budget": 0.22, "threshold": 0.0451563286330686, "operator": "ge"}, {"budget": 0.23, "threshold": 0.015528100744050657, "operator": "ge"}, {"budget": 0.24, "threshold": -0.008855887607981335, "operator": "ge"}, {"budget": 0.25, "threshold": -0.03236236929159645, "operator": "ge"}, {"budget": 0.26, "threshold": -0.05904385115763621, "operator": "ge"}, {"budget": 0.27, "threshold": -0.08881516760210897, "operator": "ge"}, {"budget": 0.28, "threshold": -0.11365764386319524, "operator": "ge"}, {"budget": 0.29, "threshold": -0.139490106914786, "operator": "ge"}, {"budget": 0.3, "threshold": -0.16939779235798227, "operator": "ge"}, {"budget": 0.31, "threshold": -0.1953525317419323, "operator": "ge"}, {"budget": 0.32, "threshold": -0.21694996349477777, "operator": "ge"}, {"budget": 0.33, "threshold": -0.24242722468990663, "operator": "ge"}, {"budget": 0.34, "threshold": -0.2674390846911102, "operator": "ge"}, {"budget": 0.35, "threshold": -0.30031399483772236, "operator": "ge"}, {"budget": 0.36, "threshold": -0.32817163511754965, "operator": "ge"}, {"budget": 0.37, "threshold": -0.35431838743186744, "operator": "ge"}, {"budget": 0.38, "threshold": -0.37663508681177066, "operator": "ge"}, {"budget": 0.39, "threshold": -0.40462649250211313, "operator": "ge"}, {"budget": 0.4, "threshold": -0.4285648872263138, "operator": "ge"}, {"budget": 0.41, "threshold": -0.4510650205126218, "operator": "ge"}, {"budget": 0.42, "threshold": -0.476494684356718, "operator": "ge"}, {"budget": 0.43, "threshold": -0.5005460626606587, "operator": "ge"}, {"budget": 0.44, "threshold": -0.5247490478700233, "operator": "ge"}, {"budget": 0.45, "threshold": -0.5440949689723058, "operator": "ge"}, {"budget": 0.46, "threshold": -0.5666389976794982, "operator": "ge"}, {"budget": 0.47, "threshold": -0.5892428856715348, "operator": "ge"}, {"budget": 0.48, "threshold": -0.6097308253393442, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6318449722235038, "operator": "ge"}, {"budget": 0.5, "threshold": -0.648055931461899, "operator": "ge"}, {"budget": 0.51, "threshold": -0.6722793357830739, "operator": "ge"}, {"budget": 0.52, "threshold": -0.6885420356366019, "operator": "ge"}, {"budget": 0.53, "threshold": -0.7101349376418649, "operator": "ge"}, {"budget": 0.54, "threshold": -0.7268951646388185, "operator": "ge"}, {"budget": 0.55, "threshold": -0.741304697508524, "operator": "ge"}, {"budget": 0.56, "threshold": -0.7606813602015081, "operator": "ge"}, {"budget": 0.57, "threshold": -0.7779947665918041, "operator": "ge"}, {"budget": 0.58, "threshold": -0.7899516501722503, "operator": "ge"}, {"budget": 0.59, "threshold": -0.8047693609408387, "operator": "ge"}, {"budget": 0.6, "threshold": -0.8162863037058965, "operator": "ge"}, {"budget": 0.61, "threshold": -0.827843143580019, "operator": "ge"}, {"budget": 0.62, "threshold": -0.8420421143754694, "operator": "ge"}, {"budget": 0.63, "threshold": -0.8534714717931309, "operator": "ge"}, {"budget": 0.64, "threshold": -0.8661956637304324, "operator": "ge"}, {"budget": 0.65, "threshold": -0.8759102791551122, "operator": "ge"}, {"budget": 0.66, "threshold": -0.884051098678216, "operator": "ge"}, {"budget": 0.67, "threshold": -0.8918305456051574, "operator": "ge"}, {"budget": 0.68, "threshold": -0.8992517304248568, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9051924733361885, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9143617427070696, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9196874651256121, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9246641900841006, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9304039075987436, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9345446348854783, "operator": "ge"}, {"budget": 0.75, "threshold": -0.939253462625852, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9422072804848722, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9460015130657026, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9491952827901219, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9524319303328299, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9557293458623359, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9586460955917643, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9612933094769364, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9637353836093608, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9667165045206355, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9689011521497999, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9712141961913802, "operator": "ge"}, {"budget": 0.87, "threshold": -0.973111119197609, "operator": "ge"}, {"budget": 0.88, "threshold": -0.975119057173111, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9771168845887548, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9786968835880164, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9808406178183726, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9827405564317755, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9842630006912911, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9859202056551734, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9874585205591465, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9891053440388567, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9908245891131359, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9924540088864335, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9947033935200758, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.2492122600974, 80.53566313377254, 80.86508163849899, 81.26611286164422, 81.60985391005443, 82.03953022056717, 82.31165855055858, 82.59810942423374, 82.92752792896017, 83.27126897737037, 83.51475221999426, 83.75823546261816, 84.0160412489258, 84.23087940418218, 84.44571755943855, 84.60326553995989, 84.90403895731882, 85.09023202520767, 85.26210254941276, 85.44829561730162, 85.63448868519048, 85.9066170151819, 86.06416499570322, 86.23603551990833, 86.40790604411343, 86.50816384989973, 86.59409911200228, 86.70867946147234, 86.89487252936121, 86.95216270409625, 87.03809796619879, 87.10971068461758, 87.22429103408766, 87.32454883987396, 87.46777427671154, 87.61099971354912, 87.81151532512173, 87.86880549985678, 88.05499856774563, 88.1695789172157, 88.31280435405327, 88.32712689773703, 88.5133199656259, 88.68519048983099, 88.85706101403609, 88.85706101403609, 88.84273847035233, 88.91435118877112, 88.94299627613864, 89.00028645087367, 89.0575766256087, 88.98596390718991, 89.0575766256087, 89.08622171297623, 89.15783443139502, 89.25809223718132, 89.37267258665138, 89.31538241191636, 89.37267258665138, 89.42996276138642, 89.48725293612145, 89.48725293612145, 89.50157547980521, 89.55886565454024, 89.53022056717272, 89.51589802348897, 89.51589802348897, 89.55886565454024, 89.58751074190776, 89.55886565454024, 89.64480091664278, 89.6734460040103, 89.70209109137782, 89.70209109137782, 89.71641363506157, 89.73073617874535, 89.73073617874535, 89.74505872242909, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.78802635348038, 89.85963907189917, 89.9455743340017, 89.88828415926669, 89.90260670295044, 89.9169292466342, 89.95989687768548, 89.95989687768548, 89.9169292466342, 89.90260670295044, 89.9455743340017, 89.95989687768548, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.9169292466342, 89.93125179031796, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 37, 65, 92, 122, 150, 182, 207, 233, 261, 288, 308, 330, 352, 378, 402, 418, 444, 463, 481, 498, 514, 537, 553, 570, 589, 601, 613, 627, 641, 652, 663, 674, 687, 701, 716, 729, 747, 758, 774, 789, 803, 811, 825, 839, 853, 860, 864, 874, 880, 887, 896, 899, 906, 914, 924, 933, 941, 945, 951, 959, 964, 967, 972, 977, 980, 985, 988, 993, 996, 997, 1003, 1007, 1013, 1015, 1020, 1023, 1026, 1031, 1036, 1038, 1043, 1046, 1052, 1059, 1061, 1065, 1068, 1072, 1073, 1073, 1075, 1079, 1080, 1082, 1082, 1083, 1084, 1087, 1087, 1087], "harmed": [0, 7, 15, 19, 21, 25, 27, 33, 39, 44, 47, 50, 55, 59, 70, 79, 84, 89, 95, 101, 105, 108, 112, 117, 122, 129, 134, 140, 146, 147, 154, 159, 165, 170, 177, 182, 185, 189, 196, 199, 206, 210, 217, 218, 220, 222, 229, 234, 239, 243, 246, 251, 259, 261, 267, 272, 274, 274, 282, 284, 288, 289, 292, 296, 297, 302, 308, 311, 313, 314, 317, 317, 319, 323, 325, 329, 331, 334, 338, 341, 343, 346, 350, 351, 352, 358, 361, 363, 364, 365, 368, 371, 372, 372, 373, 374, 377, 379, 381, 383, 383], "exact_score": [79.81953594958465, 80.2492122600974, 80.53566313377254, 80.86508163849899, 81.26611286164422, 81.60985391005443, 82.03953022056717, 82.31165855055858, 82.59810942423374, 82.92752792896017, 83.27126897737037, 83.51475221999426, 83.75823546261816, 84.0160412489258, 84.23087940418218, 84.44571755943855, 84.60326553995989, 84.90403895731882, 85.09023202520767, 85.26210254941276, 85.44829561730162, 85.63448868519048, 85.9066170151819, 86.06416499570322, 86.23603551990833, 86.40790604411343, 86.50816384989973, 86.59409911200228, 86.70867946147234, 86.89487252936121, 86.95216270409625, 87.03809796619879, 87.10971068461758, 87.22429103408766, 87.32454883987396, 87.46777427671154, 87.61099971354912, 87.81151532512173, 87.86880549985678, 88.05499856774563, 88.1695789172157, 88.31280435405327, 88.32712689773703, 88.5133199656259, 88.68519048983099, 88.85706101403609, 88.85706101403609, 88.84273847035233, 88.91435118877112, 88.94299627613864, 89.00028645087367, 89.0575766256087, 88.98596390718991, 89.0575766256087, 89.08622171297623, 89.15783443139502, 89.25809223718132, 89.37267258665138, 89.31538241191636, 89.37267258665138, 89.42996276138642, 89.48725293612145, 89.48725293612145, 89.50157547980521, 89.55886565454024, 89.53022056717272, 89.51589802348897, 89.51589802348897, 89.55886565454024, 89.58751074190776, 89.55886565454024, 89.64480091664278, 89.6734460040103, 89.70209109137782, 89.70209109137782, 89.71641363506157, 89.73073617874535, 89.73073617874535, 89.74505872242909, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.78802635348038, 89.85963907189917, 89.9455743340017, 89.88828415926669, 89.90260670295044, 89.9169292466342, 89.95989687768548, 89.95989687768548, 89.9169292466342, 89.90260670295044, 89.9455743340017, 89.95989687768548, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.9169292466342, 89.93125179031796, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.67670863925495, "threshold_area100": 87.67602406187339, "threshold_area25": 83.57777141220281}, "test": {"score": [80.00287005811867, 80.62710769893091, 81.11501757910598, 81.60292745928105, 82.00473559589582, 82.34914257013705, 82.6791992537849, 82.98055535624597, 83.26038602281697, 83.65501901413504, 83.89179880892588, 84.25773121905719, 84.45146014206787, 84.68823993685872, 84.8891440051661, 85.18332496233049, 85.51338164597834, 85.73581115017579, 85.8290880390328, 86.0084666714501, 86.17349501327402, 86.35287364569133, 86.49637655162518, 86.66858003874579, 86.8479586711631, 86.96993614120686, 87.13496448303079, 87.29999282485471, 87.52959747434886, 87.5941737820191, 87.72332639735954, 87.82377843151325, 87.80942814091985, 87.98880677333716, 88.16818540575447, 88.34038889287508, 88.44801607232546, 88.54129296118248, 88.72784673889646, 88.81394848245677, 88.93592595250054, 89.02202769606085, 89.1440051661046, 89.251632345555, 89.344909234412, 89.41666068737892, 89.58168902920285, 89.68214106335654, 89.71084164454331, 89.8184688239937, 89.86151969577384, 89.91892085814737, 89.904570567554, 89.96197172992754, 89.99067231111431, 90.03372318289445, 90.10547463586138, 90.18440123412499, 90.22745210590514, 90.277678122982, 90.28485326827868, 90.34225443065222, 90.42118102891584, 90.39965559302576, 90.41400588361914, 90.46423190069599, 90.47858219128938, 90.49293248188276, 90.47858219128938, 90.47858219128938, 90.51445791777284, 90.5503336442563, 90.54315849895961, 90.61490995192652, 90.59338451603645, 90.57903422544307, 90.58620937073975, 90.59338451603645, 90.62208509722322, 90.6364353878166, 90.70818684078354, 90.75123771256368, 90.744062567267, 90.75841285786038, 90.77993829375045, 90.75841285786038, 90.77993829375045, 90.81581402023392, 90.80146372964053, 90.81581402023392, 90.80863887493722, 90.80863887493722, 90.80146372964053, 90.80146372964053, 90.78711343904715, 90.77276314845376, 90.78711343904715, 90.77993829375045, 90.76558800315708, 90.7297122766736, 90.73688742197031], "calls": [0.0, 1.0977972303939154, 2.195594460787831, 3.171414221137978, 4.283561742125278, 5.266556647772117, 6.256726698715649, 7.512377125636795, 8.495372031283633, 9.499892372820549, 10.468536987874003, 11.774413431871995, 12.850685226375836, 13.97718303795652, 14.723398148812514, 15.871421396283274, 17.026619789050727, 18.038315275884337, 18.748654660256868, 19.745999856497093, 20.657243309177012, 21.654588505417234, 22.630408265767382, 23.61340317141422, 24.49594604290737, 25.292387170840207, 26.239506350003587, 27.122049221496734, 27.975891511803113, 28.923010690966493, 29.877305015426565, 30.81724904929325, 31.628040467819474, 32.582334792279546, 33.56532969792638, 34.70617780010045, 35.753749013417526, 36.72956877376767, 37.56188562818397, 38.609456841501036, 39.46329913180742, 40.37454258448734, 41.56561670373825, 42.433809284638016, 43.4670302073617, 44.19889502762431, 45.30386740331492, 46.236636291884906, 47.20528090693836, 48.360479299705815, 49.02059266700151, 50.40539570926311, 51.20901198249265, 52.407261247040246, 53.26827868264332, 54.100595537059625, 55.420822271651005, 56.74822415153907, 57.6522924589223, 58.77879027050298, 59.54653081724906, 60.508000287005814, 61.92867905575088, 63.033651431441484, 64.12427351653871, 65.17184472985578, 66.15483963550261, 67.15935997703953, 68.13517973738968, 68.93162086532251, 70.27337303580397, 71.23484250556074, 72.18196168472412, 73.23670804333787, 74.15512664131448, 75.20987299992825, 76.01348927315779, 77.04671019588146, 77.88620219559445, 78.81179593886776, 80.08179665638228, 81.01456554495229, 82.04061132237928, 82.90880390327904, 84.08552773193657, 84.82456769749588, 85.87931405610964, 86.64705460285572, 87.42914544019517, 88.34756403817178, 89.20858147377484, 90.40683073832245, 91.2965487551123, 92.24366793427566, 93.34146516466959, 94.23835832675613, 95.35050584774342, 96.55593025758772, 97.62502690679486, 98.85197675252924, 100.0], "rescued": [0, 93, 170, 246, 305, 359, 418, 471, 519, 580, 625, 690, 724, 768, 803, 855, 907, 947, 974, 1014, 1049, 1085, 1115, 1151, 1185, 1211, 1244, 1277, 1317, 1336, 1363, 1386, 1402, 1434, 1466, 1500, 1525, 1548, 1580, 1604, 1627, 1645, 1668, 1690, 1710, 1726, 1756, 1772, 1784, 1804, 1817, 1835, 1841, 1860, 1875, 1887, 1908, 1924, 1932, 1943, 1950, 1962, 1983, 1990, 2001, 2014, 2021, 2026, 2034, 2042, 2053, 2062, 2067, 2081, 2087, 2090, 2094, 2100, 2108, 2115, 2128, 2138, 2144, 2150, 2155, 2159, 2166, 2171, 2172, 2178, 2181, 2187, 2192, 2195, 2199, 2204, 2211, 2215, 2218, 2221, 2225], "harmed": [0, 6, 15, 23, 26, 32, 45, 56, 65, 71, 83, 97, 104, 115, 122, 133, 139, 148, 162, 177, 189, 200, 210, 222, 231, 240, 250, 260, 268, 278, 287, 296, 314, 321, 328, 338, 348, 358, 364, 376, 382, 388, 394, 401, 408, 414, 421, 423, 431, 436, 443, 453, 461, 472, 483, 489, 500, 505, 507, 511, 517, 521, 531, 541, 550, 556, 561, 564, 574, 582, 588, 592, 598, 602, 611, 616, 619, 624, 628, 633, 636, 640, 647, 651, 653, 660, 664, 664, 667, 671, 675, 681, 687, 690, 696, 703, 708, 713, 718, 726, 729], "exact_score": [80.00287005811867, 80.54100595537061, 81.02174069024898, 81.50247542512736, 81.90428356174213, 82.21998995479659, 82.58592236492788, 82.84422759560881, 83.1312334074765, 83.5043409629045, 83.80569706536558, 84.01377627896966, 84.32230752672743, 84.48733586855134, 84.6954150821554, 84.95372031283634, 85.20485039822057, 85.51338164597834, 85.73581115017579, 85.85778862021957, 86.04434239793356, 86.21654588505417, 86.43897538925164, 86.5609528592954, 86.74033149171271, 86.94123556002009, 87.1206141924374, 87.25694195307454, 87.52959747434886, 87.60852407261247, 87.73767668795294, 87.80225299562316, 87.88835473918347, 88.01033220922724, 88.21841142283131, 88.36191432876515, 88.51259237999571, 88.6058692688527, 88.77089761067663, 88.85699935423692, 89.01485255076416, 89.12965487551124, 89.19423118318146, 89.28750807203846, 89.37360981559877, 89.56016359331277, 89.66061562746646, 89.71084164454331, 89.76824280691685, 89.86151969577384, 89.91174571285069, 89.91174571285069, 89.92609600344407, 90.02654803759776, 90.01937289230108, 90.0839491999713, 90.15570065293824, 90.20592667001507, 90.24180239649853, 90.27050297768531, 90.34225443065222, 90.3637798665423, 90.41400588361914, 90.40683073832245, 90.42118102891584, 90.44270646480591, 90.47140704599268, 90.49293248188276, 90.48575733658608, 90.47858219128938, 90.50010762717945, 90.5503336442563, 90.5503336442563, 90.61490995192652, 90.58620937073975, 90.58620937073975, 90.58620937073975, 90.59338451603645, 90.62926024251992, 90.66513596900337, 90.70101169548683, 90.744062567267, 90.744062567267, 90.75841285786038, 90.77276314845376, 90.75841285786038, 90.78711343904715, 90.81581402023392, 90.82298916553061, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.80863887493722, 90.79428858434385, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.77993829375045, 90.75123771256368, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.45492505201132, "threshold_area100": 88.47564038171774, "threshold_area25": 84.21683289086606}, "selected_budget": 94, "selected": {"val": {"score": 89.97421942136923, "calls": 93.9988541965053, "rescued": 1082, "harmed": 373}, "test": {"score": 90.78711343904715, "calls": 93.34146516466959, "rescued": 2199, "harmed": 696}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7735223770141602, "operator": "ge"}, {"budget": 0.02, "threshold": 0.678681492805481, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5941566824913025, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4917241036891937, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4183084964752197, "operator": "ge"}, {"budget": 0.06, "threshold": 0.38043642044067383, "operator": "ge"}, {"budget": 0.07, "threshold": 0.33504873514175415, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2886853516101837, "operator": "ge"}, {"budget": 0.09, "threshold": 0.2554480731487274, "operator": "ge"}, {"budget": 0.1, "threshold": 0.22340448200702667, "operator": "ge"}, {"budget": 0.11, "threshold": 0.1924695521593094, "operator": "ge"}, {"budget": 0.12, "threshold": 0.1672762930393219, "operator": "ge"}, {"budget": 0.13, "threshold": 0.14148887991905212, "operator": "ge"}, {"budget": 0.14, "threshold": 0.11944408714771271, "operator": "ge"}, {"budget": 0.15, "threshold": 0.09781815856695175, "operator": "ge"}, {"budget": 0.16, "threshold": 0.08286494016647339, "operator": "ge"}, {"budget": 0.17, "threshold": 0.06493152678012848, "operator": "ge"}, {"budget": 0.18, "threshold": 0.04984486475586891, "operator": "ge"}, {"budget": 0.19, "threshold": 0.03738771378993988, "operator": "ge"}, {"budget": 0.2, "threshold": 0.02629224956035614, "operator": "ge"}, {"budget": 0.21, "threshold": 0.018246501684188843, "operator": "ge"}, {"budget": 0.22, "threshold": 0.009166702628135681, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0032933279871940613, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0013195041101425886, "operator": "ge"}, {"budget": 0.25, "threshold": -0.0034227913711220026, "operator": "ge"}, {"budget": 0.26, "threshold": -0.004397500306367874, "operator": "ge"}, {"budget": 0.27, "threshold": -0.00525419507175684, "operator": "ge"}, {"budget": 0.28, "threshold": -0.005925184115767479, "operator": "ge"}, {"budget": 0.29, "threshold": -0.0065130311995744705, "operator": "ge"}, {"budget": 0.3, "threshold": -0.007210030220448971, "operator": "ge"}, {"budget": 0.31, "threshold": -0.007857135497033596, "operator": "ge"}, {"budget": 0.32, "threshold": -0.008563704788684845, "operator": "ge"}, {"budget": 0.33, "threshold": -0.00916779413819313, "operator": "ge"}, {"budget": 0.34, "threshold": -0.009883200749754906, "operator": "ge"}, {"budget": 0.35, "threshold": -0.010585494339466095, "operator": "ge"}, {"budget": 0.36, "threshold": -0.011320805177092552, "operator": "ge"}, {"budget": 0.37, "threshold": -0.011978660710155964, "operator": "ge"}, {"budget": 0.38, "threshold": -0.01257580891251564, "operator": "ge"}, {"budget": 0.39, "threshold": -0.01317264698445797, "operator": "ge"}, {"budget": 0.4, "threshold": -0.013948772102594376, "operator": "ge"}, {"budget": 0.41, "threshold": -0.014652503654360771, "operator": "ge"}, {"budget": 0.42, "threshold": -0.015493977814912796, "operator": "ge"}, {"budget": 0.43, "threshold": -0.016249267384409904, "operator": "ge"}, {"budget": 0.44, "threshold": -0.017065396532416344, "operator": "ge"}, {"budget": 0.45, "threshold": -0.01774836890399456, "operator": "ge"}, {"budget": 0.46, "threshold": -0.018540572375059128, "operator": "ge"}, {"budget": 0.47, "threshold": -0.019290657714009285, "operator": "ge"}, {"budget": 0.48, "threshold": -0.019955238327383995, "operator": "ge"}, {"budget": 0.49, "threshold": -0.020828302949666977, "operator": "ge"}, {"budget": 0.5, "threshold": -0.02149306610226631, "operator": "ge"}, {"budget": 0.51, "threshold": -0.022162839770317078, "operator": "ge"}, {"budget": 0.52, "threshold": -0.022859353572130203, "operator": "ge"}, {"budget": 0.53, "threshold": -0.02361835166811943, "operator": "ge"}, {"budget": 0.54, "threshold": -0.024299776181578636, "operator": "ge"}, {"budget": 0.55, "threshold": -0.02523793652653694, "operator": "ge"}, {"budget": 0.56, "threshold": -0.02624206803739071, "operator": "ge"}, {"budget": 0.57, "threshold": -0.027363073080778122, "operator": "ge"}, {"budget": 0.58, "threshold": -0.02835284359753132, "operator": "ge"}, {"budget": 0.59, "threshold": -0.029368318617343903, "operator": "ge"}, {"budget": 0.6, "threshold": -0.03026185743510723, "operator": "ge"}, {"budget": 0.61, "threshold": -0.03140158951282501, "operator": "ge"}, {"budget": 0.62, "threshold": -0.03222745656967163, "operator": "ge"}, {"budget": 0.63, "threshold": -0.03299492225050926, "operator": "ge"}, {"budget": 0.64, "threshold": -0.03405889868736267, "operator": "ge"}, {"budget": 0.65, "threshold": -0.03499223664402962, "operator": "ge"}, {"budget": 0.66, "threshold": -0.036093682050704956, "operator": "ge"}, {"budget": 0.67, "threshold": -0.03718995675444603, "operator": "ge"}, {"budget": 0.68, "threshold": -0.03869063779711723, "operator": "ge"}, {"budget": 0.69, "threshold": -0.03993428125977516, "operator": "ge"}, {"budget": 0.7, "threshold": -0.041288577020168304, "operator": "ge"}, {"budget": 0.71, "threshold": -0.04246238246560097, "operator": "ge"}, {"budget": 0.72, "threshold": -0.044231414794921875, "operator": "ge"}, {"budget": 0.73, "threshold": -0.04543956741690636, "operator": "ge"}, {"budget": 0.74, "threshold": -0.046991754323244095, "operator": "ge"}, {"budget": 0.75, "threshold": -0.04851866513490677, "operator": "ge"}, {"budget": 0.76, "threshold": -0.050919950008392334, "operator": "ge"}, {"budget": 0.77, "threshold": -0.052588190883398056, "operator": "ge"}, {"budget": 0.78, "threshold": -0.05424189940094948, "operator": "ge"}, {"budget": 0.79, "threshold": -0.05614176765084267, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0577564463019371, "operator": "ge"}, {"budget": 0.81, "threshold": -0.06031603366136551, "operator": "ge"}, {"budget": 0.82, "threshold": -0.06219925731420517, "operator": "ge"}, {"budget": 0.83, "threshold": -0.06453216075897217, "operator": "ge"}, {"budget": 0.84, "threshold": -0.06697412580251694, "operator": "ge"}, {"budget": 0.85, "threshold": -0.06942468136548996, "operator": "ge"}, {"budget": 0.86, "threshold": -0.07298015058040619, "operator": "ge"}, {"budget": 0.87, "threshold": -0.07621940225362778, "operator": "ge"}, {"budget": 0.88, "threshold": -0.07937739789485931, "operator": "ge"}, {"budget": 0.89, "threshold": -0.08297882229089737, "operator": "ge"}, {"budget": 0.9, "threshold": -0.08794552087783813, "operator": "ge"}, {"budget": 0.91, "threshold": -0.09197894483804703, "operator": "ge"}, {"budget": 0.92, "threshold": -0.09722182899713516, "operator": "ge"}, {"budget": 0.93, "threshold": -0.10315143316984177, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10993850976228714, "operator": "ge"}, {"budget": 0.95, "threshold": -0.116593137383461, "operator": "ge"}, {"budget": 0.96, "threshold": -0.12581190466880798, "operator": "ge"}, {"budget": 0.97, "threshold": -0.1370251178741455, "operator": "ge"}, {"budget": 0.98, "threshold": -0.1519312560558319, "operator": "ge"}, {"budget": 0.99, "threshold": -0.18117938935756683, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.13463191062732, 80.5213405900888, 80.92237181323402, 81.43798338584932, 81.73875680320825, 82.01088513319965, 82.31165855055858, 82.7126897737038, 83.04210827843025, 83.19965625895158, 83.48610713262676, 83.7439129189344, 83.97307361787453, 84.1879117731309, 84.35978229733601, 84.60326553995989, 84.93268404468634, 85.21913491836149, 85.33371526783156, 85.5915210541392, 85.74906903466055, 85.9352621025494, 86.078487539387, 86.22171297622458, 86.33629332569464, 86.3506158693784, 86.39358350042967, 86.43655113148094, 86.46519621884846, 86.50816384989973, 86.52248639358349, 86.50816384989973, 86.55113148095101, 86.57977656831852, 86.60842165568604, 86.63706674305357, 86.63706674305357, 86.63706674305357, 86.6943569177886, 86.72300200515612, 86.72300200515612, 86.76596963620739, 86.76596963620739, 86.82325981094242, 86.79461472357491, 86.82325981094242, 86.89487252936121, 86.93784016041248, 86.99513033514752, 87.02377542251504, 86.99513033514752, 87.08106559725006, 87.15267831566887, 87.1383557719851, 87.18132340303637, 87.22429103408766, 87.3102262961902, 87.28158120882267, 87.36751647092524, 87.42480664566027, 87.46777427671154, 87.4820968203953, 87.4820968203953, 87.49641936407906, 87.52506445144657, 87.5107419077628, 87.55370953881409, 87.62532225723288, 87.68261243196791, 87.63964480091664, 87.7542251503867, 87.78287023775422, 87.79719278143799, 87.88312804354052, 87.88312804354052, 87.95474076195931, 87.89745058722428, 88.05499856774563, 88.04067602406187, 88.0263534803781, 88.14093382984818, 88.19822400458321, 88.269836723002, 88.35577198510455, 88.52764250930964, 88.59925522772843, 88.67086794614723, 88.8857061014036, 88.97164136350615, 89.00028645087367, 89.0575766256087, 89.11486680034373, 89.11486680034373, 89.27241478086508, 89.37267258665138, 89.47293039243769, 89.64480091664278, 89.74505872242909, 89.83099398453164, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 33, 63, 97, 135, 162, 188, 212, 244, 273, 297, 322, 343, 363, 384, 405, 426, 452, 477, 491, 510, 525, 541, 554, 568, 579, 580, 584, 588, 591, 595, 598, 599, 602, 605, 610, 613, 614, 617, 621, 626, 628, 631, 633, 637, 639, 642, 647, 653, 659, 664, 666, 674, 679, 680, 683, 688, 695, 698, 705, 710, 715, 718, 724, 728, 732, 736, 742, 749, 758, 762, 774, 778, 782, 790, 795, 802, 807, 820, 826, 831, 843, 850, 858, 867, 882, 893, 904, 921, 932, 943, 952, 963, 972, 991, 1003, 1019, 1042, 1057, 1073, 1087], "harmed": [0, 11, 14, 20, 22, 28, 35, 38, 42, 48, 61, 66, 69, 73, 79, 88, 92, 95, 100, 106, 107, 111, 114, 117, 121, 124, 124, 125, 126, 127, 128, 130, 132, 132, 133, 136, 137, 138, 141, 141, 144, 146, 146, 148, 148, 152, 153, 153, 156, 158, 161, 165, 167, 167, 169, 169, 171, 172, 177, 178, 179, 181, 183, 189, 192, 194, 199, 202, 204, 209, 216, 220, 222, 225, 227, 232, 234, 243, 245, 252, 258, 262, 265, 268, 271, 274, 280, 286, 288, 293, 302, 307, 314, 323, 331, 336, 345, 356, 364, 374, 383], "exact_score": [79.81953594958465, 80.13463191062732, 80.5213405900888, 80.92237181323402, 81.43798338584932, 81.73875680320825, 82.01088513319965, 82.31165855055858, 82.7126897737038, 83.04210827843025, 83.19965625895158, 83.48610713262676, 83.7439129189344, 83.97307361787453, 84.1879117731309, 84.35978229733601, 84.60326553995989, 84.93268404468634, 85.21913491836149, 85.33371526783156, 85.5915210541392, 85.74906903466055, 85.9352621025494, 86.078487539387, 86.22171297622458, 86.33629332569464, 86.3506158693784, 86.39358350042967, 86.43655113148094, 86.46519621884846, 86.50816384989973, 86.52248639358349, 86.50816384989973, 86.55113148095101, 86.57977656831852, 86.60842165568604, 86.63706674305357, 86.63706674305357, 86.63706674305357, 86.6943569177886, 86.72300200515612, 86.72300200515612, 86.76596963620739, 86.76596963620739, 86.82325981094242, 86.79461472357491, 86.82325981094242, 86.89487252936121, 86.93784016041248, 86.99513033514752, 87.02377542251504, 86.99513033514752, 87.08106559725006, 87.15267831566887, 87.1383557719851, 87.18132340303637, 87.22429103408766, 87.3102262961902, 87.28158120882267, 87.36751647092524, 87.42480664566027, 87.46777427671154, 87.4820968203953, 87.4820968203953, 87.49641936407906, 87.52506445144657, 87.5107419077628, 87.55370953881409, 87.62532225723288, 87.68261243196791, 87.63964480091664, 87.7542251503867, 87.78287023775422, 87.79719278143799, 87.88312804354052, 87.88312804354052, 87.95474076195931, 87.89745058722428, 88.05499856774563, 88.04067602406187, 88.0263534803781, 88.14093382984818, 88.19822400458321, 88.269836723002, 88.35577198510455, 88.52764250930964, 88.59925522772843, 88.67086794614723, 88.8857061014036, 88.97164136350615, 89.00028645087367, 89.0575766256087, 89.11486680034373, 89.11486680034373, 89.27241478086508, 89.37267258665138, 89.47293039243769, 89.64480091664278, 89.74505872242909, 89.83099398453164, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 86.62460375047971, "threshold_area100": 86.6238900028645, "threshold_area25": 83.59352621025494}, "test": {"score": [80.00287005811867, 80.70603429719452, 81.14371816029275, 81.60292745928105, 82.09083733945613, 82.47112004018081, 82.72942527086174, 83.0451316639162, 83.33931262108058, 83.57609241587143, 83.84157279184905, 84.17162947549687, 84.50886130444142, 84.73129080863887, 84.94654516753964, 85.31247757767095, 85.47750591949487, 85.67840998780225, 85.87213891081295, 86.0156418167468, 86.20937073975749, 86.38874937217479, 86.54660256870201, 86.67575518404249, 86.81208294467963, 86.90535983353664, 86.97711128650356, 87.01298701298703, 87.0273373035804, 87.08473846595393, 87.09191361125063, 87.09191361125063, 87.14931477362417, 87.16366506421755, 87.18519050010764, 87.2067159359977, 87.23541651718448, 87.25694195307454, 87.25694195307454, 87.28564253426131, 87.37891942311832, 87.42197029489847, 87.45784602138194, 87.48654660256871, 87.52242232905216, 87.55829805553563, 87.5941737820191, 87.63722465379924, 87.65875008968932, 87.63722465379924, 87.665925234986, 87.69462581617277, 87.70897610676617, 87.80225299562316, 87.78790270502978, 87.80942814091985, 87.8596541579967, 87.88835473918347, 87.945755901557, 87.98880677333716, 88.06773337160078, 88.12513453397432, 88.16101026045777, 88.17536055105116, 88.23993685872139, 88.28298773050155, 88.29733802109493, 88.35473918346847, 88.41931549113869, 88.49824208940231, 88.53411781588576, 88.59869412355602, 88.66327043122624, 88.70632130300639, 88.82829877305016, 88.91440051661046, 88.971801678984, 89.03637798665423, 89.12965487551124, 89.1440051661046, 89.17988089258809, 89.30903350792853, 89.30903350792853, 89.38796010619215, 89.40231039678554, 89.48123699504916, 89.56016359331277, 89.64626533687307, 89.73954222573008, 89.78259309751023, 89.8973954222573, 89.94044629403746, 90.04089832819116, 90.00502260170768, 90.14852550764154, 90.28485326827868, 90.3566047212456, 90.42835617421254, 90.5503336442563, 90.67948625959676, 90.73688742197031], "calls": [0.0, 1.2054244098442994, 2.1740690248977543, 3.0566118963909017, 4.276386596828586, 5.424409844299348, 6.127574083375189, 7.074693262538566, 8.287292817679557, 9.270287723326396, 10.253282628973237, 11.293678696993615, 12.434526799167683, 13.431871995407906, 14.400516610461361, 15.648991892085814, 16.409557293535194, 17.564755686302647, 18.68407835258664, 19.695773839420248, 20.78639592451747, 21.582837052450312, 22.745210590514457, 23.54882686374399, 24.517471478797447, 25.292387170840207, 26.096003444069744, 26.94267058907943, 27.79651287938581, 28.679055750878955, 29.590299203558875, 30.523068092128863, 31.656741049006243, 32.56080935638947, 33.67295687737677, 34.71335294539715, 35.71787328693406, 36.62911673961398, 37.38968214106336, 38.22199899547966, 39.391547678840496, 40.431943746860874, 41.46516466958456, 42.390758412857856, 43.459855062065, 44.53612685656885, 45.64109923225945, 46.70302073616991, 47.822343402453896, 48.95601635933128, 49.709406615483964, 50.57042405108704, 51.35969003372318, 52.313984358183255, 53.12477577670948, 54.32302504125709, 55.449522852837774, 56.7195235703523, 57.81732080074622, 58.88641744995336, 59.71155915907297, 60.86675755184042, 61.648848389179875, 62.4668149530028, 63.50721102102318, 64.3180024395494, 65.2507713281194, 66.33421826791992, 67.63291956662123, 68.60873932697137, 69.74241228384874, 70.67518117241875, 71.97388247112005, 72.81337447083304, 73.85377053885341, 74.81524000861017, 76.15699217909163, 77.1112865035517, 78.036880246825, 79.062926024252, 79.83066657099806, 80.97868981846882, 81.70337949343475, 82.84422759560881, 83.82722250125565, 84.66671450096864, 85.96541579966994, 86.96993614120686, 87.83095357680993, 88.82112362775347, 89.91892085814737, 90.77993829375045, 91.8490349429576, 92.9683576092416, 93.94417736959173, 94.79801965989812, 95.79536485613833, 96.77835976178517, 97.84028126569562, 99.10310683791347, 100.0], "rescued": [0, 107, 177, 243, 318, 380, 424, 477, 537, 578, 626, 681, 738, 779, 821, 881, 915, 953, 997, 1029, 1063, 1095, 1130, 1153, 1179, 1194, 1204, 1210, 1213, 1221, 1224, 1228, 1237, 1243, 1250, 1255, 1260, 1265, 1265, 1272, 1287, 1293, 1301, 1308, 1318, 1326, 1338, 1348, 1355, 1361, 1370, 1379, 1384, 1401, 1405, 1415, 1429, 1441, 1456, 1467, 1481, 1495, 1503, 1512, 1524, 1533, 1543, 1557, 1573, 1590, 1602, 1615, 1632, 1646, 1669, 1686, 1705, 1720, 1739, 1753, 1768, 1792, 1802, 1826, 1839, 1856, 1885, 1905, 1924, 1939, 1970, 1988, 2013, 2032, 2060, 2086, 2112, 2136, 2166, 2201, 2225], "harmed": [0, 9, 18, 20, 27, 36, 44, 53, 72, 80, 91, 100, 110, 120, 132, 141, 152, 162, 179, 191, 198, 205, 218, 223, 230, 232, 232, 233, 234, 234, 236, 240, 241, 245, 249, 251, 252, 254, 254, 257, 259, 259, 262, 265, 270, 273, 280, 284, 288, 297, 302, 307, 310, 314, 320, 327, 334, 342, 349, 354, 357, 363, 366, 373, 376, 379, 387, 393, 400, 406, 413, 417, 425, 433, 439, 444, 455, 461, 467, 479, 489, 495, 505, 518, 529, 535, 553, 561, 567, 576, 591, 603, 614, 638, 646, 653, 669, 683, 696, 713, 729], "exact_score": [80.00287005811867, 80.59123197244745, 81.08631699791921, 81.57422687809428, 81.98321016000575, 82.32044198895028, 82.69354954437827, 83.01643108272943, 83.28908660400374, 83.51869125349789, 83.7913467747722, 84.0568271507498, 84.3366578173208, 84.64518906507857, 84.9034942957595, 85.15462438114372, 85.38422903063787, 85.58513309894525, 85.75016144076918, 85.9080146372964, 86.10891870560378, 86.25959675683433, 86.47485111573509, 86.58965344048217, 86.7546817823061, 86.87665925234987, 86.97711128650356, 87.01298701298703, 87.0273373035804, 87.07756332065725, 87.1134390471407, 87.1134390471407, 87.15648991892087, 87.19954079070101, 87.17801535481094, 87.2138910812944, 87.24259166248117, 87.25694195307454, 87.29281767955801, 87.34304369663487, 87.38609456841502, 87.44349573078856, 87.47937145727201, 87.51524718375548, 87.55829805553563, 87.55112291023893, 87.60134892731578, 87.63004950850255, 87.65157494439264, 87.63722465379924, 87.6731003802827, 87.70180096146947, 87.7735524144364, 87.80225299562316, 87.80942814091985, 87.8596541579967, 87.86682930329339, 87.91705532037024, 87.96728133744709, 87.99598191863386, 88.08925880749085, 88.12513453397432, 88.17536055105116, 88.22558656812801, 88.24711200401808, 88.31168831168831, 88.30451316639162, 88.40496520054532, 88.46954150821554, 88.5197675252924, 88.54846810647916, 88.64174499533615, 88.66327043122624, 88.72067159359977, 88.81394848245677, 88.92157566190716, 88.95027624309392, 89.04355313195093, 89.10812943962115, 89.12965487551124, 89.17988089258809, 89.30185836263185, 89.33055894381862, 89.36643467030207, 89.4238358326756, 89.5386381574227, 89.56016359331277, 89.65344048216977, 89.74671737102676, 89.811293678697, 89.91892085814737, 89.96914687522423, 90.01219774700438, 89.997847456411, 90.14135036234484, 90.33507928535553, 90.3637798665423, 90.47858219128938, 90.56468393484968, 90.66513596900337, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.3686002287202, "threshold_area100": 87.37978044055393, "threshold_area25": 84.25773121905719}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1087, "harmed": 383}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2225, "harmed": 729}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.563981533050537, "operator": "ge"}, {"budget": 0.02, "threshold": 2.9895849227905273, "operator": "ge"}, {"budget": 0.03, "threshold": 2.6145596504211426, "operator": "ge"}, {"budget": 0.04, "threshold": 2.1730751991271973, "operator": "ge"}, {"budget": 0.05, "threshold": 1.9345393180847168, "operator": "ge"}, {"budget": 0.06, "threshold": 1.7683360576629639, "operator": "ge"}, {"budget": 0.07, "threshold": 1.5833985805511475, "operator": "ge"}, {"budget": 0.08, "threshold": 1.4571309089660645, "operator": "ge"}, {"budget": 0.09, "threshold": 1.3362324237823486, "operator": "ge"}, {"budget": 0.1, "threshold": 1.2333251237869263, "operator": "ge"}, {"budget": 0.11, "threshold": 1.113690733909607, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9946746826171875, "operator": "ge"}, {"budget": 0.13, "threshold": 0.8867597579956055, "operator": "ge"}, {"budget": 0.14, "threshold": 0.7687084674835205, "operator": "ge"}, {"budget": 0.15, "threshold": 0.6774669289588928, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5765423774719238, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4914746880531311, "operator": "ge"}, {"budget": 0.18, "threshold": 0.42640572786331177, "operator": "ge"}, {"budget": 0.19, "threshold": 0.34522759914398193, "operator": "ge"}, {"budget": 0.2, "threshold": 0.2817878723144531, "operator": "ge"}, {"budget": 0.21, "threshold": 0.1985819935798645, "operator": "ge"}, {"budget": 0.22, "threshold": 0.11426177620887756, "operator": "ge"}, {"budget": 0.23, "threshold": 0.041011229157447815, "operator": "ge"}, {"budget": 0.24, "threshold": -0.021097302436828613, "operator": "ge"}, {"budget": 0.25, "threshold": -0.08029145002365112, "operator": "ge"}, {"budget": 0.26, "threshold": -0.1502995491027832, "operator": "ge"}, {"budget": 0.27, "threshold": -0.2186097800731659, "operator": "ge"}, {"budget": 0.28, "threshold": -0.2834632694721222, "operator": "ge"}, {"budget": 0.29, "threshold": -0.35061922669410706, "operator": "ge"}, {"budget": 0.3, "threshold": -0.4135677218437195, "operator": "ge"}, {"budget": 0.31, "threshold": -0.4803994596004486, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5296753644943237, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5839248895645142, "operator": "ge"}, {"budget": 0.34, "threshold": -0.6525911688804626, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7179703712463379, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7912495136260986, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8421829342842102, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9058919548988342, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9702538847923279, "operator": "ge"}, {"budget": 0.4, "threshold": -1.0375566482543945, "operator": "ge"}, {"budget": 0.41, "threshold": -1.101025104522705, "operator": "ge"}, {"budget": 0.42, "threshold": -1.1630529165267944, "operator": "ge"}, {"budget": 0.43, "threshold": -1.224051833152771, "operator": "ge"}, {"budget": 0.44, "threshold": -1.2858222723007202, "operator": "ge"}, {"budget": 0.45, "threshold": -1.3385586738586426, "operator": "ge"}, {"budget": 0.46, "threshold": -1.4065749645233154, "operator": "ge"}, {"budget": 0.47, "threshold": -1.4721554517745972, "operator": "ge"}, {"budget": 0.48, "threshold": -1.5385451316833496, "operator": "ge"}, {"budget": 0.49, "threshold": -1.6015303134918213, "operator": "ge"}, {"budget": 0.5, "threshold": -1.6812994480133057, "operator": "ge"}, {"budget": 0.51, "threshold": -1.7473585605621338, "operator": "ge"}, {"budget": 0.52, "threshold": -1.8230148553848267, "operator": "ge"}, {"budget": 0.53, "threshold": -1.8974876403808594, "operator": "ge"}, {"budget": 0.54, "threshold": -1.9626227617263794, "operator": "ge"}, {"budget": 0.55, "threshold": -2.0310356616973877, "operator": "ge"}, {"budget": 0.56, "threshold": -2.1131815910339355, "operator": "ge"}, {"budget": 0.57, "threshold": -2.2032885551452637, "operator": "ge"}, {"budget": 0.58, "threshold": -2.275815963745117, "operator": "ge"}, {"budget": 0.59, "threshold": -2.3496031761169434, "operator": "ge"}, {"budget": 0.6, "threshold": -2.4171481132507324, "operator": "ge"}, {"budget": 0.61, "threshold": -2.5089781284332275, "operator": "ge"}, {"budget": 0.62, "threshold": -2.590648651123047, "operator": "ge"}, {"budget": 0.63, "threshold": -2.681382656097412, "operator": "ge"}, {"budget": 0.64, "threshold": -2.7490270137786865, "operator": "ge"}, {"budget": 0.65, "threshold": -2.8412859439849854, "operator": "ge"}, {"budget": 0.66, "threshold": -2.9167721271514893, "operator": "ge"}, {"budget": 0.67, "threshold": -2.9887166023254395, "operator": "ge"}, {"budget": 0.68, "threshold": -3.0700299739837646, "operator": "ge"}, {"budget": 0.69, "threshold": -3.150252342224121, "operator": "ge"}, {"budget": 0.7, "threshold": -3.2381908893585205, "operator": "ge"}, {"budget": 0.71, "threshold": -3.3172731399536133, "operator": "ge"}, {"budget": 0.72, "threshold": -3.3861610889434814, "operator": "ge"}, {"budget": 0.73, "threshold": -3.4603607654571533, "operator": "ge"}, {"budget": 0.74, "threshold": -3.5238378047943115, "operator": "ge"}, {"budget": 0.75, "threshold": -3.5990047454833984, "operator": "ge"}, {"budget": 0.76, "threshold": -3.6792190074920654, "operator": "ge"}, {"budget": 0.77, "threshold": -3.74053955078125, "operator": "ge"}, {"budget": 0.78, "threshold": -3.7999753952026367, "operator": "ge"}, {"budget": 0.79, "threshold": -3.873276472091675, "operator": "ge"}, {"budget": 0.8, "threshold": -3.9560208320617676, "operator": "ge"}, {"budget": 0.81, "threshold": -4.034006595611572, "operator": "ge"}, {"budget": 0.82, "threshold": -4.101747035980225, "operator": "ge"}, {"budget": 0.83, "threshold": -4.170332431793213, "operator": "ge"}, {"budget": 0.84, "threshold": -4.243505001068115, "operator": "ge"}, {"budget": 0.85, "threshold": -4.306577682495117, "operator": "ge"}, {"budget": 0.86, "threshold": -4.390953540802002, "operator": "ge"}, {"budget": 0.87, "threshold": -4.472058296203613, "operator": "ge"}, {"budget": 0.88, "threshold": -4.546594619750977, "operator": "ge"}, {"budget": 0.89, "threshold": -4.629818916320801, "operator": "ge"}, {"budget": 0.9, "threshold": -4.710977077484131, "operator": "ge"}, {"budget": 0.91, "threshold": -4.8119964599609375, "operator": "ge"}, {"budget": 0.92, "threshold": -4.923864364624023, "operator": "ge"}, {"budget": 0.93, "threshold": -5.036630153656006, "operator": "ge"}, {"budget": 0.94, "threshold": -5.144361972808838, "operator": "ge"}, {"budget": 0.95, "threshold": -5.258960723876953, "operator": "ge"}, {"budget": 0.96, "threshold": -5.41632080078125, "operator": "ge"}, {"budget": 0.97, "threshold": -5.564481735229492, "operator": "ge"}, {"budget": 0.98, "threshold": -5.7869157791137695, "operator": "ge"}, {"budget": 0.99, "threshold": -6.159723281860352, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.16327699799484, 80.56430822114007, 80.92237181323402, 81.29475794901174, 81.72443425952449, 81.9249498710971, 82.28301346319105, 82.68404468633629, 83.05643082211401, 83.28559152105413, 83.45746204525923, 83.71526783156689, 84.0160412489258, 84.1879117731309, 84.35978229733601, 84.58894299627613, 84.84674878258377, 85.07590948152391, 85.24778000572901, 85.46261816098539, 85.69177885992552, 85.9066170151819, 86.06416499570322, 86.20739043254082, 86.42222858779719, 86.52248639358349, 86.59409911200228, 86.6943569177886, 86.79461472357491, 86.88054998567745, 86.99513033514752, 87.10971068461758, 87.2386135777714, 87.38183901460899, 87.5107419077628, 87.56803208249785, 87.72558006301918, 87.89745058722428, 87.91177313090805, 88.14093382984818, 88.269836723002, 88.37009452878831, 88.52764250930964, 88.69951303351475, 88.87138355771984, 88.82841592666857, 88.84273847035233, 88.8857061014036, 88.92867373245488, 88.97164136350615, 89.10054425665997, 89.04325408192494, 89.04325408192494, 89.04325408192494, 89.1291893440275, 89.24376969349757, 89.30105986823258, 89.28673732454884, 89.37267258665138, 89.48725293612145, 89.48725293612145, 89.47293039243769, 89.48725293612145, 89.5445431108565, 89.5445431108565, 89.55886565454024, 89.48725293612145, 89.5445431108565, 89.58751074190776, 89.60183328559151, 89.58751074190776, 89.65912346032655, 89.68776854769406, 89.73073617874535, 89.73073617874535, 89.70209109137782, 89.75938126611285, 89.71641363506157, 89.75938126611285, 89.75938126611285, 89.74505872242909, 89.80234889716414, 89.80234889716414, 89.81667144084788, 89.85963907189917, 89.88828415926669, 89.9169292466342, 89.90260670295044, 89.9169292466342, 89.9169292466342, 89.93125179031796, 89.9169292466342, 89.95989687768548, 89.97421942136923, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.9169292466342, 89.9169292466342, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 33, 64, 96, 125, 158, 178, 208, 238, 267, 290, 311, 336, 361, 380, 399, 421, 441, 463, 482, 500, 517, 538, 553, 569, 588, 601, 612, 623, 634, 647, 659, 675, 687, 701, 716, 727, 742, 759, 768, 788, 799, 811, 823, 838, 851, 855, 862, 868, 876, 884, 895, 898, 903, 911, 920, 931, 937, 941, 950, 958, 962, 965, 970, 977, 980, 985, 986, 991, 997, 999, 1000, 1006, 1010, 1017, 1020, 1023, 1027, 1028, 1034, 1037, 1041, 1045, 1048, 1052, 1059, 1063, 1068, 1069, 1073, 1073, 1076, 1076, 1081, 1082, 1082, 1083, 1085, 1087, 1087, 1087], "harmed": [0, 9, 12, 19, 22, 25, 31, 36, 38, 41, 48, 57, 64, 68, 75, 82, 88, 90, 96, 103, 106, 107, 113, 117, 123, 127, 133, 139, 143, 147, 154, 158, 166, 169, 173, 179, 186, 190, 195, 203, 207, 209, 214, 215, 218, 219, 226, 232, 235, 240, 245, 247, 254, 259, 267, 270, 273, 275, 280, 283, 283, 287, 291, 295, 298, 301, 305, 311, 312, 315, 316, 318, 319, 321, 325, 328, 333, 333, 337, 340, 343, 348, 348, 351, 354, 358, 360, 363, 365, 368, 368, 370, 371, 373, 373, 376, 376, 378, 382, 382, 383], "exact_score": [79.81953594958465, 80.16327699799484, 80.56430822114007, 80.92237181323402, 81.29475794901174, 81.72443425952449, 81.9249498710971, 82.28301346319105, 82.68404468633629, 83.05643082211401, 83.28559152105413, 83.45746204525923, 83.71526783156689, 84.0160412489258, 84.1879117731309, 84.35978229733601, 84.58894299627613, 84.84674878258377, 85.07590948152391, 85.24778000572901, 85.46261816098539, 85.69177885992552, 85.9066170151819, 86.06416499570322, 86.20739043254082, 86.42222858779719, 86.52248639358349, 86.59409911200228, 86.6943569177886, 86.79461472357491, 86.88054998567745, 86.99513033514752, 87.10971068461758, 87.2386135777714, 87.38183901460899, 87.5107419077628, 87.56803208249785, 87.72558006301918, 87.89745058722428, 87.91177313090805, 88.14093382984818, 88.269836723002, 88.37009452878831, 88.52764250930964, 88.69951303351475, 88.87138355771984, 88.82841592666857, 88.84273847035233, 88.8857061014036, 88.92867373245488, 88.97164136350615, 89.10054425665997, 89.04325408192494, 89.04325408192494, 89.04325408192494, 89.1291893440275, 89.24376969349757, 89.30105986823258, 89.28673732454884, 89.37267258665138, 89.48725293612145, 89.48725293612145, 89.47293039243769, 89.48725293612145, 89.5445431108565, 89.5445431108565, 89.55886565454024, 89.48725293612145, 89.5445431108565, 89.58751074190776, 89.60183328559151, 89.58751074190776, 89.65912346032655, 89.68776854769406, 89.73073617874535, 89.73073617874535, 89.70209109137782, 89.75938126611285, 89.71641363506157, 89.75938126611285, 89.75938126611285, 89.74505872242909, 89.80234889716414, 89.80234889716414, 89.81667144084788, 89.85963907189917, 89.88828415926669, 89.9169292466342, 89.90260670295044, 89.9169292466342, 89.9169292466342, 89.93125179031796, 89.9169292466342, 89.95989687768548, 89.97421942136923, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.9169292466342, 89.9169292466342, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.667134976784, "threshold_area100": 87.66642795760528, "threshold_area25": 83.5906617015182}, "test": {"score": [80.00287005811867, 80.65580828011768, 81.1867690320729, 81.62445289517113, 82.09083733945613, 82.42089402310397, 82.6648489631915, 83.10253282628973, 83.41823921934419, 83.69806988591519, 83.91332424481595, 84.09270287723328, 84.37253354380427, 84.68823993685872, 84.95372031283634, 85.14744923584703, 85.47750591949487, 85.67840998780225, 85.87213891081295, 85.98694123556002, 86.14479443208725, 86.39592451747149, 86.49637655162518, 86.64705460285572, 86.85513381645978, 86.97711128650356, 87.13496448303079, 87.28564253426131, 87.43632058549186, 87.65157494439264, 87.78790270502978, 87.78072755973308, 87.89552988448015, 88.0462079357107, 88.18971084164454, 88.29016287579825, 88.49106694410563, 88.6058692688527, 88.71349644830309, 88.7924230465667, 88.90722537131377, 89.10812943962115, 89.18705603788477, 89.2588074908517, 89.30903350792853, 89.41666068737892, 89.57451388390615, 89.64626533687307, 89.66779077276315, 89.76824280691685, 89.88304513166392, 89.94044629403746, 89.94762143933414, 90.01937289230108, 89.997847456411, 90.04807347348785, 90.11264978115807, 90.1772260888283, 90.19875152471838, 90.22745210590514, 90.24897754179523, 90.28485326827868, 90.40683073832245, 90.42118102891584, 90.41400588361914, 90.44988161010261, 90.47858219128938, 90.48575733658608, 90.49293248188276, 90.51445791777284, 90.51445791777284, 90.54315849895961, 90.57903422544307, 90.5503336442563, 90.5503336442563, 90.55750878955298, 90.57903422544307, 90.61490995192652, 90.62926024251992, 90.64361053311329, 90.73688742197031, 90.75123771256368, 90.7297122766736, 90.744062567267, 90.75841285786038, 90.75841285786038, 90.75123771256368, 90.77276314845376, 90.79428858434385, 90.80863887493722, 90.80863887493722, 90.80863887493722, 90.80146372964053, 90.744062567267, 90.77276314845376, 90.77276314845376, 90.77993829375045, 90.78711343904715, 90.75841285786038, 90.73688742197031, 90.73688742197031], "calls": [0.0, 1.1049723756906076, 2.2673459137547534, 3.1283633493578247, 4.348138049795509, 5.43158498959604, 6.220850972232188, 7.447800817966564, 8.394919997129941, 9.442491210447011, 10.38961038961039, 11.401305876443999, 12.334074765013991, 13.575374901341753, 14.716223003515822, 15.720743345052737, 16.897467173710268, 17.94503838702734, 18.884982420894023, 19.70294898471694, 20.607017292100167, 21.66893879601062, 22.637583411064075, 23.57752744493076, 24.488770897610674, 25.292387170840207, 26.25385664059697, 27.14357465738681, 28.069168400660118, 29.080863887493724, 30.013632776063716, 30.996627681710553, 31.814594245533474, 32.639735954653084, 33.63708115089331, 34.799454688957454, 35.87572648346129, 36.693693047284206, 37.71256367941451, 38.717084020951425, 39.700078926598266, 40.80505130228887, 41.744995336155554, 42.7208150965057, 43.47420535265839, 44.277821625887924, 45.296692258018226, 46.30121259955514, 47.190930616344986, 48.15240008610174, 49.45110138480304, 50.29776852981273, 51.345339743129806, 52.313984358183255, 53.16065150319294, 54.114945827653, 55.219918203343624, 56.669297553275456, 57.666642749515674, 58.628112219272445, 59.52500538135897, 60.72325464590658, 61.87845303867403, 63.02647628614479, 63.88031857645118, 65.04269211451532, 65.982636148382, 67.05173279758915, 68.0203774126426, 69.08229891655307, 70.23032216402382, 71.19896677907728, 72.1891368300208, 73.09320513740403, 73.97574800889718, 75.10224582047786, 76.22874363205855, 77.00365932410132, 77.8646767597044, 78.92659826361484, 80.22529956231614, 81.32309679271005, 82.1912893736098, 83.13840855277319, 83.99942598837626, 84.7025902274521, 85.77886202195594, 86.639879457559, 87.35021884193155, 88.30451316639162, 89.2588074908517, 90.24180239649853, 91.30372390040898, 92.35129511372605, 93.38451603644974, 94.18813230967928, 95.47965846308388, 96.37655162517042, 97.60350147090479, 98.85915189782594, 100.0], "rescued": [0, 99, 181, 248, 319, 372, 413, 483, 540, 590, 631, 666, 714, 769, 818, 850, 906, 944, 982, 1014, 1047, 1093, 1117, 1148, 1185, 1211, 1243, 1274, 1306, 1341, 1369, 1387, 1411, 1436, 1465, 1488, 1523, 1549, 1579, 1598, 1621, 1654, 1673, 1692, 1704, 1725, 1751, 1767, 1777, 1795, 1819, 1831, 1844, 1862, 1869, 1887, 1904, 1921, 1929, 1938, 1944, 1955, 1977, 1992, 1997, 2011, 2021, 2025, 2032, 2043, 2050, 2059, 2069, 2075, 2081, 2087, 2095, 2101, 2107, 2114, 2131, 2139, 2145, 2149, 2153, 2157, 2161, 2166, 2171, 2176, 2179, 2185, 2191, 2192, 2198, 2202, 2211, 2214, 2217, 2222, 2225], "harmed": [0, 8, 16, 22, 28, 35, 42, 51, 64, 75, 86, 96, 105, 116, 128, 133, 143, 153, 164, 180, 191, 202, 212, 222, 230, 239, 249, 259, 270, 275, 284, 303, 311, 315, 324, 333, 340, 350, 365, 373, 380, 385, 393, 402, 407, 413, 417, 423, 430, 434, 442, 446, 458, 466, 476, 487, 495, 503, 508, 513, 516, 522, 527, 540, 546, 555, 561, 564, 570, 578, 585, 590, 595, 605, 611, 616, 621, 622, 626, 631, 635, 641, 650, 652, 654, 658, 663, 665, 667, 670, 673, 679, 686, 695, 697, 701, 709, 711, 718, 726, 729], "exact_score": [80.00287005811867, 80.57688168185406, 81.08631699791921, 81.5598765875009, 81.99038530530244, 82.29891655306021, 82.60744780081797, 82.90162875798235, 83.23886058692689, 83.5976178517615, 83.7913467747722, 84.01377627896966, 84.30078209083734, 84.56626246681496, 84.75999138982566, 85.01112147520988, 85.21202554351726, 85.49903135538494, 85.70711056898904, 85.88648920140633, 86.03716725263688, 86.23807132094426, 86.43180024395495, 86.5753031498888, 86.74033149171271, 86.94841070531677, 87.10626390184402, 87.25694195307454, 87.42914544019517, 87.63004950850255, 87.78790270502978, 87.78072755973308, 87.9529310468537, 88.12513453397432, 88.22558656812801, 88.33321374757838, 88.49824208940231, 88.64892014063285, 88.70632130300639, 88.81394848245677, 88.92875080720385, 89.15118031140132, 89.19423118318146, 89.28033292674176, 89.40231039678554, 89.52428786682931, 89.6175647556863, 89.67496591805984, 89.73954222573008, 89.85434455047715, 89.91174571285069, 89.94762143933414, 90.01219774700438, 90.00502260170768, 90.06242376408122, 90.10547463586138, 90.14135036234484, 90.19157637942169, 90.19157637942169, 90.24180239649853, 90.24180239649853, 90.29202841357538, 90.39248044772907, 90.40683073832245, 90.42118102891584, 90.44270646480591, 90.47858219128938, 90.48575733658608, 90.49293248188276, 90.51445791777284, 90.50010762717945, 90.54315849895961, 90.57185908014638, 90.54315849895961, 90.5503336442563, 90.53598335366291, 90.58620937073975, 90.61490995192652, 90.62926024251992, 90.64361053311329, 90.71536198608023, 90.73688742197031, 90.73688742197031, 90.75123771256368, 90.75841285786038, 90.75123771256368, 90.75841285786038, 90.77993829375045, 90.80863887493722, 90.79428858434385, 90.80146372964053, 90.77993829375045, 90.77276314845376, 90.77276314845376, 90.77993829375045, 90.78711343904715, 90.78711343904715, 90.79428858434385, 90.744062567267, 90.73688742197031, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.45693673890568, "threshold_area100": 88.4776494224008, "threshold_area25": 84.22329052163307}, "selected_budget": 94, "selected": {"val": {"score": 89.97421942136923, "calls": 93.9988541965053, "rescued": 1082, "harmed": 373}, "test": {"score": 90.77276314845376, "calls": 93.38451603644974, "rescued": 2198, "harmed": 697}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.729196548461914, "operator": "ge"}, {"budget": 0.02, "threshold": 2.292466640472412, "operator": "ge"}, {"budget": 0.03, "threshold": 1.9612164497375488, "operator": "ge"}, {"budget": 0.04, "threshold": 1.6937556266784668, "operator": "ge"}, {"budget": 0.05, "threshold": 1.492558479309082, "operator": "ge"}, {"budget": 0.06, "threshold": 1.3439345359802246, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1971080303192139, "operator": "ge"}, {"budget": 0.08, "threshold": 1.0871461629867554, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9742640256881714, "operator": "ge"}, {"budget": 0.1, "threshold": 0.8874463438987732, "operator": "ge"}, {"budget": 0.11, "threshold": 0.781869649887085, "operator": "ge"}, {"budget": 0.12, "threshold": 0.7063971757888794, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6300367116928101, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5737432837486267, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4987463057041168, "operator": "ge"}, {"budget": 0.16, "threshold": 0.4317557215690613, "operator": "ge"}, {"budget": 0.17, "threshold": 0.3657715916633606, "operator": "ge"}, {"budget": 0.18, "threshold": 0.31940171122550964, "operator": "ge"}, {"budget": 0.19, "threshold": 0.26194071769714355, "operator": "ge"}, {"budget": 0.2, "threshold": 0.2119487226009369, "operator": "ge"}, {"budget": 0.21, "threshold": 0.14895135164260864, "operator": "ge"}, {"budget": 0.22, "threshold": 0.09037423133850098, "operator": "ge"}, {"budget": 0.23, "threshold": 0.03105863928794861, "operator": "ge"}, {"budget": 0.24, "threshold": -0.017712175846099854, "operator": "ge"}, {"budget": 0.25, "threshold": -0.06474751234054565, "operator": "ge"}, {"budget": 0.26, "threshold": -0.11822515726089478, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1780996322631836, "operator": "ge"}, {"budget": 0.28, "threshold": -0.22830182313919067, "operator": "ge"}, {"budget": 0.29, "threshold": -0.2808113098144531, "operator": "ge"}, {"budget": 0.3, "threshold": -0.3420931100845337, "operator": "ge"}, {"budget": 0.31, "threshold": -0.3957923650741577, "operator": "ge"}, {"budget": 0.32, "threshold": -0.44090643525123596, "operator": "ge"}, {"budget": 0.33, "threshold": -0.49470269680023193, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5482072830200195, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6197293400764465, "operator": "ge"}, {"budget": 0.36, "threshold": -0.6815557479858398, "operator": "ge"}, {"budget": 0.37, "threshold": -0.7407472133636475, "operator": "ge"}, {"budget": 0.38, "threshold": -0.7922652363777161, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8583377003669739, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9162749648094177, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9720733165740967, "operator": "ge"}, {"budget": 0.42, "threshold": -1.0368788242340088, "operator": "ge"}, {"budget": 0.43, "threshold": -1.1000691652297974, "operator": "ge"}, {"budget": 0.44, "threshold": -1.16574227809906, "operator": "ge"}, {"budget": 0.45, "threshold": -1.2199088335037231, "operator": "ge"}, {"budget": 0.46, "threshold": -1.2851166725158691, "operator": "ge"}, {"budget": 0.47, "threshold": -1.3530114889144897, "operator": "ge"}, {"budget": 0.48, "threshold": -1.416986346244812, "operator": "ge"}, {"budget": 0.49, "threshold": -1.488965392112732, "operator": "ge"}, {"budget": 0.5, "threshold": -1.5438803434371948, "operator": "ge"}, {"budget": 0.51, "threshold": -1.6297813653945923, "operator": "ge"}, {"budget": 0.52, "threshold": -1.6903578042984009, "operator": "ge"}, {"budget": 0.53, "threshold": -1.7749121189117432, "operator": "ge"}, {"budget": 0.54, "threshold": -1.844225287437439, "operator": "ge"}, {"budget": 0.55, "threshold": -1.9067436456680298, "operator": "ge"}, {"budget": 0.56, "threshold": -1.9956629276275635, "operator": "ge"}, {"budget": 0.57, "threshold": -2.080540657043457, "operator": "ge"}, {"budget": 0.58, "threshold": -2.142606735229492, "operator": "ge"}, {"budget": 0.59, "threshold": -2.224008321762085, "operator": "ge"}, {"budget": 0.6, "threshold": -2.2911736965179443, "operator": "ge"}, {"budget": 0.61, "threshold": -2.3624866008758545, "operator": "ge"}, {"budget": 0.62, "threshold": -2.4563021659851074, "operator": "ge"}, {"budget": 0.63, "threshold": -2.537595748901367, "operator": "ge"}, {"budget": 0.64, "threshold": -2.635287046432495, "operator": "ge"}, {"budget": 0.65, "threshold": -2.715846300125122, "operator": "ge"}, {"budget": 0.66, "threshold": -2.7880311012268066, "operator": "ge"}, {"budget": 0.67, "threshold": -2.8616013526916504, "operator": "ge"}, {"budget": 0.68, "threshold": -2.9365944862365723, "operator": "ge"}, {"budget": 0.69, "threshold": -3.0004923343658447, "operator": "ge"}, {"budget": 0.7, "threshold": -3.1070096492767334, "operator": "ge"}, {"budget": 0.71, "threshold": -3.1740078926086426, "operator": "ge"}, {"budget": 0.72, "threshold": -3.2405552864074707, "operator": "ge"}, {"budget": 0.73, "threshold": -3.3227806091308594, "operator": "ge"}, {"budget": 0.74, "threshold": -3.3862690925598145, "operator": "ge"}, {"budget": 0.75, "threshold": -3.463373899459839, "operator": "ge"}, {"budget": 0.76, "threshold": -3.514726161956787, "operator": "ge"}, {"budget": 0.77, "threshold": -3.5845789909362793, "operator": "ge"}, {"budget": 0.78, "threshold": -3.6471922397613525, "operator": "ge"}, {"budget": 0.79, "threshold": -3.714676856994629, "operator": "ge"}, {"budget": 0.8, "threshold": -3.7882080078125, "operator": "ge"}, {"budget": 0.81, "threshold": -3.8578484058380127, "operator": "ge"}, {"budget": 0.82, "threshold": -3.9253592491149902, "operator": "ge"}, {"budget": 0.83, "threshold": -3.991793394088745, "operator": "ge"}, {"budget": 0.84, "threshold": -4.079067230224609, "operator": "ge"}, {"budget": 0.85, "threshold": -4.148083686828613, "operator": "ge"}, {"budget": 0.86, "threshold": -4.226552963256836, "operator": "ge"}, {"budget": 0.87, "threshold": -4.295672416687012, "operator": "ge"}, {"budget": 0.88, "threshold": -4.374350547790527, "operator": "ge"}, {"budget": 0.89, "threshold": -4.459078311920166, "operator": "ge"}, {"budget": 0.9, "threshold": -4.531355381011963, "operator": "ge"}, {"budget": 0.91, "threshold": -4.638496398925781, "operator": "ge"}, {"budget": 0.92, "threshold": -4.743917465209961, "operator": "ge"}, {"budget": 0.93, "threshold": -4.8369951248168945, "operator": "ge"}, {"budget": 0.94, "threshold": -4.9491119384765625, "operator": "ge"}, {"budget": 0.95, "threshold": -5.065619945526123, "operator": "ge"}, {"budget": 0.96, "threshold": -5.207189083099365, "operator": "ge"}, {"budget": 0.97, "threshold": -5.380016803741455, "operator": "ge"}, {"budget": 0.98, "threshold": -5.576135635375977, "operator": "ge"}, {"budget": 0.99, "threshold": -5.931278228759766, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.2492122600974, 80.53566313377254, 80.86508163849899, 81.26611286164422, 81.60985391005443, 82.03953022056717, 82.31165855055858, 82.59810942423374, 82.92752792896017, 83.27126897737037, 83.51475221999426, 83.75823546261816, 84.0160412489258, 84.23087940418218, 84.44571755943855, 84.60326553995989, 84.90403895731882, 85.09023202520767, 85.26210254941276, 85.44829561730162, 85.63448868519048, 85.9066170151819, 86.06416499570322, 86.23603551990833, 86.40790604411343, 86.50816384989973, 86.59409911200228, 86.70867946147234, 86.89487252936121, 86.95216270409625, 87.03809796619879, 87.10971068461758, 87.22429103408766, 87.32454883987396, 87.46777427671154, 87.61099971354912, 87.81151532512173, 87.86880549985678, 88.05499856774563, 88.1695789172157, 88.31280435405327, 88.32712689773703, 88.5133199656259, 88.68519048983099, 88.85706101403609, 88.85706101403609, 88.84273847035233, 88.91435118877112, 88.94299627613864, 89.00028645087367, 89.0575766256087, 88.98596390718991, 89.0575766256087, 89.08622171297623, 89.15783443139502, 89.25809223718132, 89.37267258665138, 89.31538241191636, 89.37267258665138, 89.42996276138642, 89.48725293612145, 89.48725293612145, 89.50157547980521, 89.55886565454024, 89.53022056717272, 89.51589802348897, 89.51589802348897, 89.55886565454024, 89.58751074190776, 89.55886565454024, 89.64480091664278, 89.6734460040103, 89.70209109137782, 89.70209109137782, 89.71641363506157, 89.73073617874535, 89.73073617874535, 89.74505872242909, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.78802635348038, 89.85963907189917, 89.9455743340017, 89.88828415926669, 89.90260670295044, 89.9169292466342, 89.95989687768548, 89.95989687768548, 89.9169292466342, 89.90260670295044, 89.9455743340017, 89.95989687768548, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.9169292466342, 89.93125179031796, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 37, 65, 92, 122, 150, 182, 207, 233, 261, 288, 308, 330, 352, 378, 402, 418, 444, 463, 481, 498, 514, 537, 553, 570, 589, 601, 613, 627, 641, 652, 663, 674, 687, 701, 716, 729, 747, 758, 774, 789, 803, 811, 825, 839, 853, 860, 864, 874, 880, 887, 896, 899, 906, 914, 924, 933, 941, 945, 951, 959, 964, 967, 972, 977, 980, 985, 988, 993, 996, 997, 1003, 1007, 1013, 1015, 1020, 1023, 1026, 1031, 1036, 1038, 1043, 1046, 1052, 1059, 1061, 1065, 1068, 1072, 1073, 1073, 1075, 1079, 1080, 1082, 1082, 1083, 1084, 1087, 1087, 1087], "harmed": [0, 7, 15, 19, 21, 25, 27, 33, 39, 44, 47, 50, 55, 59, 70, 79, 84, 89, 95, 101, 105, 108, 112, 117, 122, 129, 134, 140, 146, 147, 154, 159, 165, 170, 177, 182, 185, 189, 196, 199, 206, 210, 217, 218, 220, 222, 229, 234, 239, 243, 246, 251, 259, 261, 267, 272, 274, 274, 282, 284, 288, 289, 292, 296, 297, 302, 308, 311, 313, 314, 317, 317, 319, 323, 325, 329, 331, 334, 338, 341, 343, 346, 350, 351, 352, 358, 361, 363, 364, 365, 368, 371, 372, 372, 373, 374, 377, 379, 381, 383, 383], "exact_score": [79.81953594958465, 80.2492122600974, 80.53566313377254, 80.86508163849899, 81.26611286164422, 81.60985391005443, 82.03953022056717, 82.31165855055858, 82.59810942423374, 82.92752792896017, 83.27126897737037, 83.51475221999426, 83.75823546261816, 84.0160412489258, 84.23087940418218, 84.44571755943855, 84.60326553995989, 84.90403895731882, 85.09023202520767, 85.26210254941276, 85.44829561730162, 85.63448868519048, 85.9066170151819, 86.06416499570322, 86.23603551990833, 86.40790604411343, 86.50816384989973, 86.59409911200228, 86.70867946147234, 86.89487252936121, 86.95216270409625, 87.03809796619879, 87.10971068461758, 87.22429103408766, 87.32454883987396, 87.46777427671154, 87.61099971354912, 87.81151532512173, 87.86880549985678, 88.05499856774563, 88.1695789172157, 88.31280435405327, 88.32712689773703, 88.5133199656259, 88.68519048983099, 88.85706101403609, 88.85706101403609, 88.84273847035233, 88.91435118877112, 88.94299627613864, 89.00028645087367, 89.0575766256087, 88.98596390718991, 89.0575766256087, 89.08622171297623, 89.15783443139502, 89.25809223718132, 89.37267258665138, 89.31538241191636, 89.37267258665138, 89.42996276138642, 89.48725293612145, 89.48725293612145, 89.50157547980521, 89.55886565454024, 89.53022056717272, 89.51589802348897, 89.51589802348897, 89.55886565454024, 89.58751074190776, 89.55886565454024, 89.64480091664278, 89.6734460040103, 89.70209109137782, 89.70209109137782, 89.71641363506157, 89.73073617874535, 89.73073617874535, 89.74505872242909, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.78802635348038, 89.85963907189917, 89.9455743340017, 89.88828415926669, 89.90260670295044, 89.9169292466342, 89.95989687768548, 89.95989687768548, 89.9169292466342, 89.90260670295044, 89.9455743340017, 89.95989687768548, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.9169292466342, 89.93125179031796, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.67670863925495, "threshold_area100": 87.67602406187339, "threshold_area25": 83.57777141220281}, "test": {"score": [80.00287005811867, 80.62710769893091, 81.11501757910598, 81.60292745928105, 82.00473559589582, 82.34914257013705, 82.6791992537849, 82.98055535624597, 83.26038602281697, 83.65501901413504, 83.89179880892588, 84.25773121905719, 84.45146014206787, 84.68823993685872, 84.8891440051661, 85.18332496233049, 85.51338164597834, 85.73581115017579, 85.8290880390328, 86.0084666714501, 86.17349501327402, 86.35287364569133, 86.49637655162518, 86.66858003874579, 86.8479586711631, 86.96993614120686, 87.13496448303079, 87.29999282485471, 87.52959747434886, 87.5941737820191, 87.72332639735954, 87.82377843151325, 87.80942814091985, 87.98880677333716, 88.16818540575447, 88.34038889287508, 88.44801607232546, 88.54129296118248, 88.72784673889646, 88.81394848245677, 88.93592595250054, 89.02202769606085, 89.1440051661046, 89.251632345555, 89.344909234412, 89.41666068737892, 89.58168902920285, 89.68214106335654, 89.71084164454331, 89.8184688239937, 89.86151969577384, 89.91892085814737, 89.904570567554, 89.96197172992754, 89.99067231111431, 90.03372318289445, 90.10547463586138, 90.18440123412499, 90.22745210590514, 90.277678122982, 90.28485326827868, 90.34225443065222, 90.42118102891584, 90.39965559302576, 90.41400588361914, 90.46423190069599, 90.47858219128938, 90.49293248188276, 90.47858219128938, 90.47858219128938, 90.51445791777284, 90.5503336442563, 90.54315849895961, 90.61490995192652, 90.59338451603645, 90.57903422544307, 90.58620937073975, 90.59338451603645, 90.62208509722322, 90.6364353878166, 90.70818684078354, 90.75123771256368, 90.744062567267, 90.75841285786038, 90.77993829375045, 90.75841285786038, 90.77993829375045, 90.81581402023392, 90.80146372964053, 90.81581402023392, 90.80863887493722, 90.80863887493722, 90.80146372964053, 90.80146372964053, 90.78711343904715, 90.77276314845376, 90.78711343904715, 90.77993829375045, 90.76558800315708, 90.7297122766736, 90.73688742197031], "calls": [0.0, 1.0977972303939154, 2.195594460787831, 3.171414221137978, 4.283561742125278, 5.266556647772117, 6.256726698715649, 7.512377125636795, 8.495372031283633, 9.499892372820549, 10.468536987874003, 11.774413431871995, 12.850685226375836, 13.97718303795652, 14.723398148812514, 15.871421396283274, 17.026619789050727, 18.038315275884337, 18.748654660256868, 19.745999856497093, 20.657243309177012, 21.654588505417234, 22.630408265767382, 23.61340317141422, 24.49594604290737, 25.292387170840207, 26.239506350003587, 27.122049221496734, 27.975891511803113, 28.923010690966493, 29.877305015426565, 30.81724904929325, 31.628040467819474, 32.582334792279546, 33.56532969792638, 34.70617780010045, 35.753749013417526, 36.72956877376767, 37.56188562818397, 38.609456841501036, 39.46329913180742, 40.37454258448734, 41.56561670373825, 42.433809284638016, 43.4670302073617, 44.19889502762431, 45.30386740331492, 46.236636291884906, 47.20528090693836, 48.360479299705815, 49.02059266700151, 50.40539570926311, 51.20901198249265, 52.407261247040246, 53.26827868264332, 54.100595537059625, 55.420822271651005, 56.74822415153907, 57.6522924589223, 58.77879027050298, 59.54653081724906, 60.508000287005814, 61.92867905575088, 63.033651431441484, 64.12427351653871, 65.17184472985578, 66.15483963550261, 67.15935997703953, 68.13517973738968, 68.93162086532251, 70.27337303580397, 71.23484250556074, 72.18196168472412, 73.23670804333787, 74.15512664131448, 75.20987299992825, 76.01348927315779, 77.04671019588146, 77.88620219559445, 78.81179593886776, 80.08179665638228, 81.01456554495229, 82.04061132237928, 82.90880390327904, 84.08552773193657, 84.82456769749588, 85.87931405610964, 86.64705460285572, 87.42914544019517, 88.34756403817178, 89.20858147377484, 90.40683073832245, 91.2965487551123, 92.24366793427566, 93.34146516466959, 94.23835832675613, 95.35050584774342, 96.55593025758772, 97.62502690679486, 98.85197675252924, 100.0], "rescued": [0, 93, 170, 246, 305, 359, 418, 471, 519, 580, 625, 690, 724, 768, 803, 855, 907, 947, 974, 1014, 1049, 1085, 1115, 1151, 1185, 1211, 1244, 1277, 1317, 1336, 1363, 1386, 1402, 1434, 1466, 1500, 1525, 1548, 1580, 1604, 1627, 1645, 1668, 1690, 1710, 1726, 1756, 1772, 1784, 1804, 1817, 1835, 1841, 1860, 1875, 1887, 1908, 1924, 1932, 1943, 1950, 1962, 1983, 1990, 2001, 2014, 2021, 2026, 2034, 2042, 2053, 2062, 2067, 2081, 2087, 2090, 2094, 2100, 2108, 2115, 2128, 2138, 2144, 2150, 2155, 2159, 2166, 2171, 2172, 2178, 2181, 2187, 2192, 2195, 2199, 2204, 2211, 2215, 2218, 2221, 2225], "harmed": [0, 6, 15, 23, 26, 32, 45, 56, 65, 71, 83, 97, 104, 115, 122, 133, 139, 148, 162, 177, 189, 200, 210, 222, 231, 240, 250, 260, 268, 278, 287, 296, 314, 321, 328, 338, 348, 358, 364, 376, 382, 388, 394, 401, 408, 414, 421, 423, 431, 436, 443, 453, 461, 472, 483, 489, 500, 505, 507, 511, 517, 521, 531, 541, 550, 556, 561, 564, 574, 582, 588, 592, 598, 602, 611, 616, 619, 624, 628, 633, 636, 640, 647, 651, 653, 660, 664, 664, 667, 671, 675, 681, 687, 690, 696, 703, 708, 713, 718, 726, 729], "exact_score": [80.00287005811867, 80.54100595537061, 81.02174069024898, 81.50247542512736, 81.90428356174213, 82.21998995479659, 82.58592236492788, 82.84422759560881, 83.1312334074765, 83.5043409629045, 83.80569706536558, 84.01377627896966, 84.32230752672743, 84.48733586855134, 84.6954150821554, 84.95372031283634, 85.20485039822057, 85.51338164597834, 85.73581115017579, 85.85778862021957, 86.04434239793356, 86.21654588505417, 86.43897538925164, 86.5609528592954, 86.74033149171271, 86.94123556002009, 87.1206141924374, 87.25694195307454, 87.52959747434886, 87.60852407261247, 87.73767668795294, 87.80225299562316, 87.88835473918347, 88.01033220922724, 88.21841142283131, 88.36191432876515, 88.51259237999571, 88.6058692688527, 88.77089761067663, 88.85699935423692, 89.01485255076416, 89.12965487551124, 89.19423118318146, 89.28750807203846, 89.37360981559877, 89.56016359331277, 89.66061562746646, 89.71084164454331, 89.76824280691685, 89.86151969577384, 89.91174571285069, 89.91174571285069, 89.92609600344407, 90.02654803759776, 90.01937289230108, 90.0839491999713, 90.15570065293824, 90.20592667001507, 90.24180239649853, 90.27050297768531, 90.34225443065222, 90.3637798665423, 90.41400588361914, 90.40683073832245, 90.42118102891584, 90.44270646480591, 90.47140704599268, 90.49293248188276, 90.48575733658608, 90.47858219128938, 90.50010762717945, 90.5503336442563, 90.5503336442563, 90.61490995192652, 90.58620937073975, 90.58620937073975, 90.58620937073975, 90.59338451603645, 90.62926024251992, 90.66513596900337, 90.70101169548683, 90.744062567267, 90.744062567267, 90.75841285786038, 90.77276314845376, 90.75841285786038, 90.78711343904715, 90.81581402023392, 90.82298916553061, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.80863887493722, 90.79428858434385, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.77993829375045, 90.75123771256368, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.45492505201132, "threshold_area100": 88.47564038171774, "threshold_area25": 84.21683289086606}, "selected_budget": 94, "selected": {"val": {"score": 89.97421942136923, "calls": 93.9988541965053, "rescued": 1082, "harmed": 373}, "test": {"score": 90.78711343904715, "calls": 93.34146516466959, "rescued": 2199, "harmed": 696}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.83099398453164, "calls": 98.99742194213692}, "test": {"score": 90.67948625959676, "calls": 99.10310683791347}}, "old_normalized_interior": {"val": {"score": 89.97421942136923, "calls": 93.9988541965053}, "test": {"score": 90.78711343904715, "calls": 93.34146516466959}}, "old_normalized_endpoints": {"val": {"score": 89.97421942136923, "calls": 93.9988541965053}, "test": {"score": 90.78711343904715, "calls": 93.34146516466959}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.2492122600974, 80.53566313377254, 80.86508163849899, 81.26611286164422, 81.60985391005443, 82.03953022056717, 82.31165855055858, 82.59810942423374, 82.92752792896017, 83.27126897737037, 83.51475221999426, 83.75823546261816, 84.0160412489258, 84.23087940418218, 84.44571755943855, 84.60326553995989, 84.90403895731882, 85.09023202520767, 85.26210254941276, 85.44829561730162, 85.63448868519048, 85.9066170151819, 86.06416499570322, 86.23603551990833, 86.40790604411343, 86.50816384989973, 86.59409911200228, 86.70867946147234, 86.89487252936121, 86.95216270409625, 87.03809796619879, 87.10971068461758, 87.22429103408766, 87.32454883987396, 87.46777427671154, 87.61099971354912, 87.81151532512173, 87.86880549985678, 88.05499856774563, 88.1695789172157, 88.31280435405327, 88.32712689773703, 88.5133199656259, 88.68519048983099, 88.85706101403609, 88.85706101403609, 88.84273847035233, 88.91435118877112, 88.94299627613864, 89.00028645087367, 89.0575766256087, 88.98596390718991, 89.0575766256087, 89.08622171297623, 89.15783443139502, 89.25809223718132, 89.37267258665138, 89.31538241191636, 89.37267258665138, 89.42996276138642, 89.48725293612145, 89.48725293612145, 89.50157547980521, 89.55886565454024, 89.53022056717272, 89.51589802348897, 89.51589802348897, 89.55886565454024, 89.58751074190776, 89.55886565454024, 89.64480091664278, 89.6734460040103, 89.70209109137782, 89.70209109137782, 89.71641363506157, 89.73073617874535, 89.73073617874535, 89.74505872242909, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.78802635348038, 89.85963907189917, 89.9455743340017, 89.88828415926669, 89.90260670295044, 89.9169292466342, 89.95989687768548, 89.95989687768548, 89.9169292466342, 89.90260670295044, 89.9455743340017, 89.95989687768548, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.9169292466342, 89.93125179031796, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.2492122600974, 80.53566313377254, 80.86508163849899, 81.26611286164422, 81.60985391005443, 82.03953022056717, 82.31165855055858, 82.59810942423374, 82.92752792896017, 83.27126897737037, 83.51475221999426, 83.75823546261816, 84.0160412489258, 84.23087940418218, 84.44571755943855, 84.60326553995989, 84.90403895731882, 85.09023202520767, 85.26210254941276, 85.44829561730162, 85.63448868519048, 85.9066170151819, 86.06416499570322, 86.23603551990833, 86.40790604411343, 86.50816384989973, 86.59409911200228, 86.70867946147234, 86.89487252936121, 86.95216270409625, 87.03809796619879, 87.10971068461758, 87.22429103408766, 87.32454883987396, 87.46777427671154, 87.61099971354912, 87.81151532512173, 87.86880549985678, 88.05499856774563, 88.1695789172157, 88.31280435405327, 88.32712689773703, 88.5133199656259, 88.68519048983099, 88.85706101403609, 88.85706101403609, 88.84273847035233, 88.91435118877112, 88.94299627613864, 89.00028645087367, 89.0575766256087, 88.98596390718991, 89.0575766256087, 89.08622171297623, 89.15783443139502, 89.25809223718132, 89.37267258665138, 89.31538241191636, 89.37267258665138, 89.42996276138642, 89.48725293612145, 89.48725293612145, 89.50157547980521, 89.55886565454024, 89.53022056717272, 89.51589802348897, 89.51589802348897, 89.55886565454024, 89.58751074190776, 89.55886565454024, 89.64480091664278, 89.6734460040103, 89.70209109137782, 89.70209109137782, 89.71641363506157, 89.73073617874535, 89.73073617874535, 89.74505872242909, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.78802635348038, 89.85963907189917, 89.9455743340017, 89.88828415926669, 89.90260670295044, 89.9169292466342, 89.95989687768548, 89.95989687768548, 89.9169292466342, 89.90260670295044, 89.9455743340017, 89.95989687768548, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.9169292466342, 89.93125179031796, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 83.57777141220281, "exact_area100": 87.67670863925495}, "test": {"score": [80.00287005811867, 80.62710769893091, 81.11501757910598, 81.60292745928105, 82.00473559589582, 82.34914257013705, 82.6791992537849, 82.98055535624597, 83.26038602281697, 83.65501901413504, 83.89179880892588, 84.25773121905719, 84.45146014206787, 84.68823993685872, 84.8891440051661, 85.18332496233049, 85.51338164597834, 85.73581115017579, 85.8290880390328, 86.0084666714501, 86.17349501327402, 86.35287364569133, 86.49637655162518, 86.66858003874579, 86.8479586711631, 86.96993614120686, 87.13496448303079, 87.29999282485471, 87.52959747434886, 87.5941737820191, 87.72332639735954, 87.82377843151325, 87.80942814091985, 87.98880677333716, 88.16818540575447, 88.34038889287508, 88.44801607232546, 88.54129296118248, 88.72784673889646, 88.81394848245677, 88.93592595250054, 89.02202769606085, 89.1440051661046, 89.251632345555, 89.344909234412, 89.41666068737892, 89.58168902920285, 89.68214106335654, 89.71084164454331, 89.8184688239937, 89.86151969577384, 89.91892085814737, 89.904570567554, 89.96197172992754, 89.99067231111431, 90.03372318289445, 90.10547463586138, 90.18440123412499, 90.22745210590514, 90.277678122982, 90.28485326827868, 90.34225443065222, 90.42118102891584, 90.39965559302576, 90.41400588361914, 90.46423190069599, 90.47858219128938, 90.49293248188276, 90.47858219128938, 90.47858219128938, 90.51445791777284, 90.5503336442563, 90.54315849895961, 90.61490995192652, 90.59338451603645, 90.57903422544307, 90.58620937073975, 90.59338451603645, 90.62208509722322, 90.6364353878166, 90.70818684078354, 90.75123771256368, 90.744062567267, 90.75841285786038, 90.77993829375045, 90.75841285786038, 90.77993829375045, 90.81581402023392, 90.80146372964053, 90.81581402023392, 90.80863887493722, 90.80863887493722, 90.80146372964053, 90.80146372964053, 90.78711343904715, 90.77276314845376, 90.78711343904715, 90.77993829375045, 90.76558800315708, 90.7297122766736, 90.73688742197031], "calls": [0.0, 1.0977972303939154, 2.195594460787831, 3.171414221137978, 4.283561742125278, 5.266556647772117, 6.256726698715649, 7.512377125636795, 8.495372031283633, 9.499892372820549, 10.468536987874003, 11.774413431871995, 12.850685226375836, 13.97718303795652, 14.723398148812514, 15.871421396283274, 17.026619789050727, 18.038315275884337, 18.748654660256868, 19.745999856497093, 20.657243309177012, 21.654588505417234, 22.630408265767382, 23.61340317141422, 24.49594604290737, 25.292387170840207, 26.239506350003587, 27.122049221496734, 27.975891511803113, 28.923010690966493, 29.877305015426565, 30.81724904929325, 31.628040467819474, 32.582334792279546, 33.56532969792638, 34.70617780010045, 35.753749013417526, 36.72956877376767, 37.56188562818397, 38.609456841501036, 39.46329913180742, 40.37454258448734, 41.56561670373825, 42.433809284638016, 43.4670302073617, 44.19889502762431, 45.30386740331492, 46.236636291884906, 47.20528090693836, 48.360479299705815, 49.02059266700151, 50.40539570926311, 51.20901198249265, 52.407261247040246, 53.26827868264332, 54.100595537059625, 55.420822271651005, 56.74822415153907, 57.6522924589223, 58.77879027050298, 59.54653081724906, 60.508000287005814, 61.92867905575088, 63.033651431441484, 64.12427351653871, 65.17184472985578, 66.15483963550261, 67.15935997703953, 68.13517973738968, 68.93162086532251, 70.27337303580397, 71.23484250556074, 72.18196168472412, 73.23670804333787, 74.15512664131448, 75.20987299992825, 76.01348927315779, 77.04671019588146, 77.88620219559445, 78.81179593886776, 80.08179665638228, 81.01456554495229, 82.04061132237928, 82.90880390327904, 84.08552773193657, 84.82456769749588, 85.87931405610964, 86.64705460285572, 87.42914544019517, 88.34756403817178, 89.20858147377484, 90.40683073832245, 91.2965487551123, 92.24366793427566, 93.34146516466959, 94.23835832675613, 95.35050584774342, 96.55593025758772, 97.62502690679486, 98.85197675252924, 100.0], "exact_score": [80.00287005811867, 80.54100595537061, 81.02174069024898, 81.50247542512736, 81.90428356174213, 82.21998995479659, 82.58592236492788, 82.84422759560881, 83.1312334074765, 83.5043409629045, 83.80569706536558, 84.01377627896966, 84.32230752672743, 84.48733586855134, 84.6954150821554, 84.95372031283634, 85.20485039822057, 85.51338164597834, 85.73581115017579, 85.85778862021957, 86.04434239793356, 86.21654588505417, 86.43897538925164, 86.5609528592954, 86.74033149171271, 86.94123556002009, 87.1206141924374, 87.25694195307454, 87.52959747434886, 87.60852407261247, 87.73767668795294, 87.80225299562316, 87.88835473918347, 88.01033220922724, 88.21841142283131, 88.36191432876515, 88.51259237999571, 88.6058692688527, 88.77089761067663, 88.85699935423692, 89.01485255076416, 89.12965487551124, 89.19423118318146, 89.28750807203846, 89.37360981559877, 89.56016359331277, 89.66061562746646, 89.71084164454331, 89.76824280691685, 89.86151969577384, 89.91174571285069, 89.91174571285069, 89.92609600344407, 90.02654803759776, 90.01937289230108, 90.0839491999713, 90.15570065293824, 90.20592667001507, 90.24180239649853, 90.27050297768531, 90.34225443065222, 90.3637798665423, 90.41400588361914, 90.40683073832245, 90.42118102891584, 90.44270646480591, 90.47140704599268, 90.49293248188276, 90.48575733658608, 90.47858219128938, 90.50010762717945, 90.5503336442563, 90.5503336442563, 90.61490995192652, 90.58620937073975, 90.58620937073975, 90.58620937073975, 90.59338451603645, 90.62926024251992, 90.66513596900337, 90.70101169548683, 90.744062567267, 90.744062567267, 90.75841285786038, 90.77276314845376, 90.75841285786038, 90.78711343904715, 90.81581402023392, 90.82298916553061, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.80863887493722, 90.79428858434385, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.77993829375045, 90.75123771256368, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 84.21683289086606, "exact_area100": 88.45492505201132}}, "old_normalized_endpoint_budget": 94}, {"seed": 2028, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 1.0, "validation_area100": 0.8770866563535599, "training_seconds": 28.150454159011133, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6200053630195573, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5441053446720271, "operator": "ge"}, {"budget": 0.03, "threshold": 0.4957792041834302, "operator": "ge"}, {"budget": 0.04, "threshold": 0.44636117039399964, "operator": "ge"}, {"budget": 0.05, "threshold": 0.404890928764958, "operator": "ge"}, {"budget": 0.06, "threshold": 0.35787210287601406, "operator": "ge"}, {"budget": 0.07, "threshold": 0.3172382995470398, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2577903812189785, "operator": "ge"}, {"budget": 0.09, "threshold": 0.20834797509212866, "operator": "ge"}, {"budget": 0.1, "threshold": 0.1579059206160088, "operator": "ge"}, {"budget": 0.11, "threshold": 0.1121535412504586, "operator": "ge"}, {"budget": 0.12, "threshold": 0.06911235901309844, "operator": "ge"}, {"budget": 0.13, "threshold": 0.03533880736125441, "operator": "ge"}, {"budget": 0.14, "threshold": 0.002201428456134957, "operator": "ge"}, {"budget": 0.15, "threshold": -0.031191106203446145, "operator": "ge"}, {"budget": 0.16, "threshold": -0.06590722031076465, "operator": "ge"}, {"budget": 0.17, "threshold": -0.09533426200149853, "operator": "ge"}, {"budget": 0.18, "threshold": -0.12380938886483951, "operator": "ge"}, {"budget": 0.19, "threshold": -0.1495550008059936, "operator": "ge"}, {"budget": 0.2, "threshold": -0.1758257549481801, "operator": "ge"}, {"budget": 0.21, "threshold": -0.20676059823476886, "operator": "ge"}, {"budget": 0.22, "threshold": -0.23315130853046187, "operator": "ge"}, {"budget": 0.23, "threshold": -0.2647031480681456, "operator": "ge"}, {"budget": 0.24, "threshold": -0.29024236768395933, "operator": "ge"}, {"budget": 0.25, "threshold": -0.3179816162630177, "operator": "ge"}, {"budget": 0.26, "threshold": -0.3453457730667474, "operator": "ge"}, {"budget": 0.27, "threshold": -0.3716744070521514, "operator": "ge"}, {"budget": 0.28, "threshold": -0.4011198124708279, "operator": "ge"}, {"budget": 0.29, "threshold": -0.4313089512602375, "operator": "ge"}, {"budget": 0.3, "threshold": -0.4552822276566209, "operator": "ge"}, {"budget": 0.31, "threshold": -0.4848591680744551, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5168094818461765, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5402341561714763, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5620742164890699, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5816917995548381, "operator": "ge"}, {"budget": 0.36, "threshold": -0.6047766713706841, "operator": "ge"}, {"budget": 0.37, "threshold": -0.6275594857900187, "operator": "ge"}, {"budget": 0.38, "threshold": -0.6485976492982269, "operator": "ge"}, {"budget": 0.39, "threshold": -0.6719066963686483, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6900684206198082, "operator": "ge"}, {"budget": 0.41, "threshold": -0.7116263983510108, "operator": "ge"}, {"budget": 0.42, "threshold": -0.7289843733299617, "operator": "ge"}, {"budget": 0.43, "threshold": -0.7476895789684467, "operator": "ge"}, {"budget": 0.44, "threshold": -0.7621769378030855, "operator": "ge"}, {"budget": 0.45, "threshold": -0.7766313041110344, "operator": "ge"}, {"budget": 0.46, "threshold": -0.7875618386124067, "operator": "ge"}, {"budget": 0.47, "threshold": -0.7987073231804308, "operator": "ge"}, {"budget": 0.48, "threshold": -0.8091402249280454, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8199432469924113, "operator": "ge"}, {"budget": 0.5, "threshold": -0.8290716996344379, "operator": "ge"}, {"budget": 0.51, "threshold": -0.837559416855474, "operator": "ge"}, {"budget": 0.52, "threshold": -0.8461804029558837, "operator": "ge"}, {"budget": 0.53, "threshold": -0.8538760513850505, "operator": "ge"}, {"budget": 0.54, "threshold": -0.8608304442303495, "operator": "ge"}, {"budget": 0.55, "threshold": -0.8685768987991065, "operator": "ge"}, {"budget": 0.56, "threshold": -0.8746282613995175, "operator": "ge"}, {"budget": 0.57, "threshold": -0.8799349065259928, "operator": "ge"}, {"budget": 0.58, "threshold": -0.8848888700401211, "operator": "ge"}, {"budget": 0.59, "threshold": -0.8905841792736139, "operator": "ge"}, {"budget": 0.6, "threshold": -0.8952128300212848, "operator": "ge"}, {"budget": 0.61, "threshold": -0.901069744442701, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9065997908640795, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9106475028981331, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9141458706535006, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9177928656122862, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9217349545936838, "operator": "ge"}, {"budget": 0.67, "threshold": -0.925007644036853, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9281440351811325, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9314276372208451, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9345664132833049, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9376683665590593, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9400406974185691, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9421133110538343, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9449586646230794, "operator": "ge"}, {"budget": 0.75, "threshold": -0.947292010824217, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9497573404601283, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9516741473210847, "operator": "ge"}, {"budget": 0.78, "threshold": -0.95347810585297, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9555495346859426, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9574183794915575, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9592954103188434, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9608139709188633, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9627321217254231, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9643614577817287, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9660211398841161, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9677780979293459, "operator": "ge"}, {"budget": 0.87, "threshold": -0.969512882122815, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9710196441157335, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9725181506990436, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9742714344096325, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9758994435981133, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9773874937376431, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9786755412296919, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9802528886131359, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9818062978351754, "operator": "ge"}, {"budget": 0.96, "threshold": -0.983536438706806, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9851259598337826, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9869992413943374, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9894310179178005, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.1775995416786, 80.60727585219134, 80.99398453165281, 81.33772558006301, 81.71011171584072, 82.05385276425092, 82.29733600687481, 82.68404468633629, 82.81294757949011, 83.11372099684904, 83.42881695789171, 83.65797765683185, 83.7439129189344, 84.00171870524204, 84.35978229733601, 84.63191062732741, 84.90403895731882, 84.97565167573761, 85.17616728731022, 85.43397307361786, 85.69177885992552, 85.87797192781437, 86.09281008307075, 86.26468060727585, 86.36493841306215, 86.40790604411343, 86.56545402463478, 86.70867946147234, 86.80893726725866, 87.00945287883127, 87.16700085935261, 87.3102262961902, 87.45345173302778, 87.53938699513033, 87.69693497565167, 87.76854769407046, 87.96906330564308, 88.08364365511314, 88.19822400458321, 88.31280435405327, 88.34144944142079, 88.42738470352333, 88.48467487825837, 88.57061014036091, 88.6135777714122, 88.68519048983099, 88.82841592666857, 88.85706101403609, 88.91435118877112, 89.0575766256087, 89.11486680034373, 89.15783443139502, 89.22944714981381, 89.35835004296763, 89.38699513033515, 89.42996276138642, 89.55886565454024, 89.55886565454024, 89.55886565454024, 89.573188198224, 89.53022056717272, 89.5445431108565, 89.58751074190776, 89.573188198224, 89.5445431108565, 89.60183328559151, 89.55886565454024, 89.58751074190776, 89.60183328559151, 89.58751074190776, 89.5445431108565, 89.60183328559151, 89.63047837295903, 89.6734460040103, 89.70209109137782, 89.68776854769406, 89.65912346032655, 89.68776854769406, 89.74505872242909, 89.77370380979661, 89.85963907189917, 89.8453165282154, 89.90260670295044, 89.9169292466342, 89.97421942136923, 89.98854196505299, 89.97421942136923, 89.97421942136923, 89.97421942136923, 90.03150959610426, 90.0171870524205, 90.07447722715554, 90.03150959610426, 90.0171870524205, 90.00286450873675, 90.0171870524205, 89.95989687768548, 89.9455743340017, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 35, 69, 100, 132, 161, 192, 217, 251, 268, 290, 316, 338, 355, 382, 410, 435, 456, 470, 491, 514, 535, 554, 575, 592, 602, 612, 627, 642, 656, 675, 691, 704, 716, 727, 742, 752, 768, 784, 795, 808, 815, 825, 834, 842, 849, 858, 870, 876, 886, 898, 904, 910, 918, 928, 934, 939, 950, 955, 958, 962, 963, 968, 973, 976, 979, 985, 987, 991, 995, 996, 996, 1002, 1006, 1013, 1019, 1020, 1023, 1027, 1031, 1034, 1041, 1044, 1049, 1052, 1056, 1058, 1061, 1064, 1066, 1072, 1073, 1077, 1080, 1081, 1083, 1084, 1085, 1086, 1087, 1087], "harmed": [0, 10, 14, 18, 26, 29, 36, 44, 51, 59, 60, 64, 70, 81, 90, 93, 99, 101, 110, 117, 122, 125, 131, 137, 142, 145, 152, 156, 161, 168, 173, 178, 181, 183, 188, 192, 197, 199, 207, 210, 215, 220, 224, 229, 231, 235, 239, 241, 245, 251, 253, 255, 258, 261, 262, 266, 268, 270, 275, 278, 281, 285, 289, 291, 295, 300, 302, 307, 309, 312, 314, 317, 319, 321, 325, 329, 331, 336, 338, 338, 339, 340, 344, 345, 347, 347, 348, 352, 355, 357, 359, 361, 361, 367, 369, 372, 372, 377, 379, 383, 383], "exact_score": [79.81953594958465, 80.1775995416786, 80.60727585219134, 80.99398453165281, 81.33772558006301, 81.71011171584072, 82.05385276425092, 82.29733600687481, 82.68404468633629, 82.81294757949011, 83.11372099684904, 83.42881695789171, 83.65797765683185, 83.7439129189344, 84.00171870524204, 84.35978229733601, 84.63191062732741, 84.90403895731882, 84.97565167573761, 85.17616728731022, 85.43397307361786, 85.69177885992552, 85.87797192781437, 86.09281008307075, 86.26468060727585, 86.36493841306215, 86.40790604411343, 86.56545402463478, 86.70867946147234, 86.80893726725866, 87.00945287883127, 87.16700085935261, 87.3102262961902, 87.45345173302778, 87.53938699513033, 87.69693497565167, 87.76854769407046, 87.96906330564308, 88.08364365511314, 88.19822400458321, 88.31280435405327, 88.34144944142079, 88.42738470352333, 88.48467487825837, 88.57061014036091, 88.6135777714122, 88.68519048983099, 88.82841592666857, 88.85706101403609, 88.91435118877112, 89.0575766256087, 89.11486680034373, 89.15783443139502, 89.22944714981381, 89.35835004296763, 89.38699513033515, 89.42996276138642, 89.55886565454024, 89.55886565454024, 89.55886565454024, 89.573188198224, 89.53022056717272, 89.5445431108565, 89.58751074190776, 89.573188198224, 89.5445431108565, 89.60183328559151, 89.55886565454024, 89.58751074190776, 89.60183328559151, 89.58751074190776, 89.5445431108565, 89.60183328559151, 89.63047837295903, 89.6734460040103, 89.70209109137782, 89.68776854769406, 89.65912346032655, 89.68776854769406, 89.74505872242909, 89.77370380979661, 89.85963907189917, 89.8453165282154, 89.90260670295044, 89.9169292466342, 89.97421942136923, 89.98854196505299, 89.97421942136923, 89.97421942136923, 89.97421942136923, 90.03150959610426, 90.0171870524205, 90.07447722715554, 90.03150959610426, 90.0171870524205, 90.00286450873675, 90.0171870524205, 89.95989687768548, 89.9455743340017, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.70866563535598, "threshold_area100": 87.70796333428817, "threshold_area25": 83.54196505299342}, "test": {"score": [80.00287005811867, 80.71320944249122, 81.19394417736959, 81.63162804046782, 82.06931190356606, 82.37066800602713, 82.69354954437827, 83.00208079213604, 83.2173351510368, 83.52586639879458, 83.92049939011265, 84.16445433020019, 84.40840927028772, 84.60213819329842, 84.78869197101243, 85.04699720169334, 85.3627035947478, 85.5492573724618, 85.78603716725264, 86.0084666714501, 86.32417306450456, 86.5753031498888, 86.7546817823061, 86.86230896175648, 86.96276099591017, 86.99146157709694, 87.17801535481094, 87.35021884193155, 87.4937217478654, 87.55829805553563, 87.665925234986, 87.88117959388678, 88.12513453397432, 88.26146229461148, 88.41931549113869, 88.52694267058908, 88.66327043122624, 88.78524790127, 88.84982420894023, 88.92875080720385, 89.04355313195093, 89.10095429432447, 89.20140632847816, 89.32338379852193, 89.4310109779723, 89.48841214034584, 89.54581330271938, 89.58168902920285, 89.65344048216977, 89.811293678697, 89.83999425988375, 89.91174571285069, 89.91892085814737, 89.94044629403746, 89.97632202052091, 90.07677405467462, 90.13417521704815, 90.1772260888283, 90.25615268709191, 90.33507928535553, 90.3566047212456, 90.37813015713569, 90.40683073832245, 90.40683073832245, 90.44270646480591, 90.45705675539931, 90.45705675539931, 90.51445791777284, 90.52880820836621, 90.48575733658608, 90.49293248188276, 90.50010762717945, 90.52880820836621, 90.56468393484968, 90.59338451603645, 90.62208509722322, 90.62926024251992, 90.64361053311329, 90.6364353878166, 90.65796082370669, 90.69383655019014, 90.72253713137691, 90.73688742197031, 90.77993829375045, 90.73688742197031, 90.72253713137691, 90.7297122766736, 90.75841285786038, 90.77276314845376, 90.79428858434385, 90.81581402023392, 90.85168974671738, 90.85168974671738, 90.8301643108273, 90.8301643108273, 90.85886489201407, 90.80146372964053, 90.79428858434385, 90.77993829375045, 90.75841285786038, 90.73688742197031], "calls": [0.0, 1.2125995551409916, 2.339097366721676, 3.264691109994977, 4.31943746860874, 5.151754323025042, 6.299777570495803, 7.038817536055104, 8.186840783525867, 9.169835689172706, 10.296333500753391, 11.329554423477076, 12.334074765013991, 13.324244815957522, 14.192437396857285, 15.168257157207435, 16.14407691755758, 17.04097007964411, 18.002439549400876, 18.985434455047713, 19.98995479658463, 21.07340173638516, 21.99899547965846, 23.132668436535837, 23.964985290952143, 24.826002726555213, 25.859223649278896, 26.677190213101813, 27.574083375188348, 28.679055750878955, 29.48984716940518, 30.50154265623879, 31.75001793786324, 32.776063715290235, 33.54380426203631, 34.218267919925374, 35.1223362273086, 36.22013345770252, 37.160077491569204, 38.28657530314989, 39.2982707899835, 40.37454258448734, 41.46516466958456, 42.792566549472625, 43.675109420965775, 44.73703092487623, 45.51194661691899, 46.5236421037526, 47.61426418884982, 48.68336083805697, 49.62330487192366, 50.383870273373034, 51.45296692258018, 52.49336299060056, 53.40460644328048, 54.538279400157855, 55.650426921145154, 56.561670373825066, 57.52313984358183, 58.74291454401952, 59.67568343258951, 61.02461074836766, 62.194159431728494, 63.08387744851834, 63.96642032001149, 64.95659037095501, 65.95393556719523, 66.85082872928176, 67.91275023319223, 68.98902202769605, 69.9791920786396, 71.05546387314344, 71.93083159933988, 72.64834612900911, 73.97574800889718, 74.9659180598407, 76.07806558082801, 76.85298127287078, 77.663772691397, 78.79744564827438, 79.71586424625099, 80.70603429719452, 81.40202339097367, 82.57157207433451, 83.44693980053096, 84.55191217622156, 85.58513309894525, 86.61117887637224, 87.52242232905216, 88.48389179880893, 89.65344048216977, 90.78711343904715, 91.77010834469398, 92.66700150678051, 93.7863241730645, 94.65451675396427, 95.58011049723757, 96.36937647987371, 97.53175001793787, 98.91655306019946, 100.0], "rescued": [0, 106, 184, 253, 319, 367, 428, 478, 526, 580, 644, 690, 737, 776, 810, 855, 906, 945, 988, 1029, 1076, 1119, 1157, 1188, 1215, 1236, 1275, 1305, 1333, 1361, 1385, 1421, 1463, 1489, 1514, 1536, 1563, 1586, 1607, 1632, 1655, 1673, 1697, 1722, 1741, 1759, 1773, 1788, 1802, 1828, 1843, 1854, 1861, 1875, 1888, 1909, 1922, 1932, 1945, 1961, 1971, 1982, 1993, 1997, 2006, 2015, 2021, 2031, 2040, 2045, 2050, 2055, 2062, 2070, 2080, 2086, 2091, 2096, 2098, 2106, 2114, 2122, 2127, 2139, 2140, 2145, 2150, 2156, 2163, 2171, 2178, 2188, 2192, 2195, 2201, 2208, 2209, 2214, 2217, 2220, 2225], "harmed": [0, 7, 18, 26, 31, 37, 53, 60, 78, 89, 98, 110, 123, 135, 143, 152, 159, 172, 182, 192, 195, 203, 216, 232, 245, 262, 275, 281, 289, 308, 317, 323, 331, 338, 341, 348, 356, 362, 374, 388, 395, 405, 415, 423, 427, 437, 443, 453, 457, 461, 472, 473, 479, 490, 498, 505, 510, 514, 516, 521, 528, 536, 543, 547, 551, 558, 564, 566, 573, 584, 588, 592, 595, 598, 604, 606, 610, 613, 616, 621, 624, 628, 631, 637, 644, 651, 655, 657, 662, 667, 671, 676, 680, 686, 692, 695, 704, 710, 715, 721, 729], "exact_score": [80.00287005811867, 80.56253139126068, 80.97868981846884, 81.50247542512736, 81.9329841429289, 82.33479227954366, 82.58592236492788, 82.97338021094927, 83.19580971514674, 83.43976465523427, 83.81287221066228, 84.0640022960465, 84.30795723613403, 84.5375618856282, 84.78151682571571, 85.01829662050658, 85.2766018511875, 85.53490708186841, 85.77886202195594, 86.0156418167468, 86.32417306450456, 86.5681280045921, 86.7546817823061, 86.8479586711631, 86.95558585061349, 87.0273373035804, 87.2138910812944, 87.41479514960179, 87.51524718375548, 87.60134892731578, 87.80942814091985, 88.00315706393054, 88.18971084164454, 88.28298773050155, 88.505417234699, 88.64892014063285, 88.75654732008323, 88.83547391834685, 88.92157566190716, 89.00050226017078, 89.05790342254431, 89.16553060199469, 89.28750807203846, 89.36643467030207, 89.45971155915908, 89.47406184975246, 89.56733873860946, 89.60321446509292, 89.73236708043338, 89.83281911458707, 89.86869484107054, 89.8973954222573, 89.95479658463084, 89.97632202052091, 90.00502260170768, 90.09112434526799, 90.14852550764154, 90.20592667001507, 90.25615268709191, 90.33507928535553, 90.37813015713569, 90.37813015713569, 90.41400588361914, 90.39248044772907, 90.44270646480591, 90.45705675539931, 90.45705675539931, 90.52163306306953, 90.53598335366291, 90.48575733658608, 90.49293248188276, 90.50010762717945, 90.52880820836621, 90.56468393484968, 90.58620937073975, 90.62926024251992, 90.6364353878166, 90.64361053311329, 90.65078567840999, 90.65796082370669, 90.70818684078354, 90.7297122766736, 90.73688742197031, 90.75123771256368, 90.71536198608023, 90.73688742197031, 90.744062567267, 90.77276314845376, 90.77993829375045, 90.80146372964053, 90.85168974671738, 90.86604003731075, 90.84451460142068, 90.8301643108273, 90.83733945612398, 90.84451460142068, 90.80863887493722, 90.77993829375045, 90.77993829375045, 90.75841285786038, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.50535983147734, "threshold_area100": 88.50254717658032, "threshold_area25": 84.2319006959891}, "selected_budget": 92, "selected": {"val": {"score": 90.07447722715554, "calls": 91.99369808077915, "rescued": 1077, "harmed": 361}, "test": {"score": 90.85168974671738, "calls": 91.77010834469398, "rescued": 2192, "harmed": 680}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.4649602174758911, "operator": "ge"}, {"budget": 0.02, "threshold": 0.4010542035102844, "operator": "ge"}, {"budget": 0.03, "threshold": 0.34482118487358093, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2852729260921478, "operator": "ge"}, {"budget": 0.05, "threshold": 0.24432912468910217, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2065240442752838, "operator": "ge"}, {"budget": 0.07, "threshold": 0.16513192653656006, "operator": "ge"}, {"budget": 0.08, "threshold": 0.12669458985328674, "operator": "ge"}, {"budget": 0.09, "threshold": 0.09437468647956848, "operator": "ge"}, {"budget": 0.1, "threshold": 0.06078802049160004, "operator": "ge"}, {"budget": 0.11, "threshold": 0.04365609586238861, "operator": "ge"}, {"budget": 0.12, "threshold": 0.0254167839884758, "operator": "ge"}, {"budget": 0.13, "threshold": 0.012068897485733032, "operator": "ge"}, {"budget": 0.14, "threshold": 0.0008150190114974976, "operator": "ge"}, {"budget": 0.15, "threshold": -0.004590723197907209, "operator": "ge"}, {"budget": 0.16, "threshold": -0.005532305687665939, "operator": "ge"}, {"budget": 0.17, "threshold": -0.006467640865594149, "operator": "ge"}, {"budget": 0.18, "threshold": -0.007095312234014273, "operator": "ge"}, {"budget": 0.19, "threshold": -0.007739413995295763, "operator": "ge"}, {"budget": 0.2, "threshold": -0.00831425841897726, "operator": "ge"}, {"budget": 0.21, "threshold": -0.009008481167256832, "operator": "ge"}, {"budget": 0.22, "threshold": -0.009542585350573063, "operator": "ge"}, {"budget": 0.23, "threshold": -0.010241554118692875, "operator": "ge"}, {"budget": 0.24, "threshold": -0.010866543278098106, "operator": "ge"}, {"budget": 0.25, "threshold": -0.011640070006251335, "operator": "ge"}, {"budget": 0.26, "threshold": -0.012228946201503277, "operator": "ge"}, {"budget": 0.27, "threshold": -0.012866944074630737, "operator": "ge"}, {"budget": 0.28, "threshold": -0.013427271507680416, "operator": "ge"}, {"budget": 0.29, "threshold": -0.014198670163750648, "operator": "ge"}, {"budget": 0.3, "threshold": -0.014862102456390858, "operator": "ge"}, {"budget": 0.31, "threshold": -0.01584070734679699, "operator": "ge"}, {"budget": 0.32, "threshold": -0.0165980514138937, "operator": "ge"}, {"budget": 0.33, "threshold": -0.01758480630815029, "operator": "ge"}, {"budget": 0.34, "threshold": -0.018676143139600754, "operator": "ge"}, {"budget": 0.35, "threshold": -0.019606132060289383, "operator": "ge"}, {"budget": 0.36, "threshold": -0.020585788413882256, "operator": "ge"}, {"budget": 0.37, "threshold": -0.021508067846298218, "operator": "ge"}, {"budget": 0.38, "threshold": -0.022577710449695587, "operator": "ge"}, {"budget": 0.39, "threshold": -0.02349216677248478, "operator": "ge"}, {"budget": 0.4, "threshold": -0.024493636563420296, "operator": "ge"}, {"budget": 0.41, "threshold": -0.025804713368415833, "operator": "ge"}, {"budget": 0.42, "threshold": -0.02697097510099411, "operator": "ge"}, {"budget": 0.43, "threshold": -0.02812340296804905, "operator": "ge"}, {"budget": 0.44, "threshold": -0.02924203686416149, "operator": "ge"}, {"budget": 0.45, "threshold": -0.03035632148385048, "operator": "ge"}, {"budget": 0.46, "threshold": -0.03160352259874344, "operator": "ge"}, {"budget": 0.47, "threshold": -0.032684169709682465, "operator": "ge"}, {"budget": 0.48, "threshold": -0.033832743763923645, "operator": "ge"}, {"budget": 0.49, "threshold": -0.034955255687236786, "operator": "ge"}, {"budget": 0.5, "threshold": -0.03619430959224701, "operator": "ge"}, {"budget": 0.51, "threshold": -0.0374264232814312, "operator": "ge"}, {"budget": 0.52, "threshold": -0.03862692415714264, "operator": "ge"}, {"budget": 0.53, "threshold": -0.03966085612773895, "operator": "ge"}, {"budget": 0.54, "threshold": -0.04083188623189926, "operator": "ge"}, {"budget": 0.55, "threshold": -0.04199898615479469, "operator": "ge"}, {"budget": 0.56, "threshold": -0.04353182017803192, "operator": "ge"}, {"budget": 0.57, "threshold": -0.04464484006166458, "operator": "ge"}, {"budget": 0.58, "threshold": -0.045649126172065735, "operator": "ge"}, {"budget": 0.59, "threshold": -0.04691600799560547, "operator": "ge"}, {"budget": 0.6, "threshold": -0.048324331641197205, "operator": "ge"}, {"budget": 0.61, "threshold": -0.049507059156894684, "operator": "ge"}, {"budget": 0.62, "threshold": -0.05063147842884064, "operator": "ge"}, {"budget": 0.63, "threshold": -0.05202854424715042, "operator": "ge"}, {"budget": 0.64, "threshold": -0.05340275168418884, "operator": "ge"}, {"budget": 0.65, "threshold": -0.054546233266592026, "operator": "ge"}, {"budget": 0.66, "threshold": -0.05602727085351944, "operator": "ge"}, {"budget": 0.67, "threshold": -0.05743863806128502, "operator": "ge"}, {"budget": 0.68, "threshold": -0.05886063724756241, "operator": "ge"}, {"budget": 0.69, "threshold": -0.06034873425960541, "operator": "ge"}, {"budget": 0.7, "threshold": -0.062235601246356964, "operator": "ge"}, {"budget": 0.71, "threshold": -0.06372645497322083, "operator": "ge"}, {"budget": 0.72, "threshold": -0.06555299460887909, "operator": "ge"}, {"budget": 0.73, "threshold": -0.06705564260482788, "operator": "ge"}, {"budget": 0.74, "threshold": -0.0686950534582138, "operator": "ge"}, {"budget": 0.75, "threshold": -0.07059355080127716, "operator": "ge"}, {"budget": 0.76, "threshold": -0.07297766208648682, "operator": "ge"}, {"budget": 0.77, "threshold": -0.07482147961854935, "operator": "ge"}, {"budget": 0.78, "threshold": -0.07723726332187653, "operator": "ge"}, {"budget": 0.79, "threshold": -0.07940966635942459, "operator": "ge"}, {"budget": 0.8, "threshold": -0.08244139701128006, "operator": "ge"}, {"budget": 0.81, "threshold": -0.08485665917396545, "operator": "ge"}, {"budget": 0.82, "threshold": -0.08756150305271149, "operator": "ge"}, {"budget": 0.83, "threshold": -0.09079577773809433, "operator": "ge"}, {"budget": 0.84, "threshold": -0.09355565905570984, "operator": "ge"}, {"budget": 0.85, "threshold": -0.09653805196285248, "operator": "ge"}, {"budget": 0.86, "threshold": -0.09954611212015152, "operator": "ge"}, {"budget": 0.87, "threshold": -0.10277721285820007, "operator": "ge"}, {"budget": 0.88, "threshold": -0.1059189960360527, "operator": "ge"}, {"budget": 0.89, "threshold": -0.10914932191371918, "operator": "ge"}, {"budget": 0.9, "threshold": -0.11255854368209839, "operator": "ge"}, {"budget": 0.91, "threshold": -0.11583595722913742, "operator": "ge"}, {"budget": 0.92, "threshold": -0.12062112241983414, "operator": "ge"}, {"budget": 0.93, "threshold": -0.12469060719013214, "operator": "ge"}, {"budget": 0.94, "threshold": -0.12899774312973022, "operator": "ge"}, {"budget": 0.95, "threshold": -0.13411888480186462, "operator": "ge"}, {"budget": 0.96, "threshold": -0.14038732647895813, "operator": "ge"}, {"budget": 0.97, "threshold": -0.14775890111923218, "operator": "ge"}, {"budget": 0.98, "threshold": -0.15909816324710846, "operator": "ge"}, {"budget": 0.99, "threshold": -0.17348173260688782, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.1059868232598, 80.57863076482383, 80.99398453165281, 81.35204812374677, 81.62417645373819, 81.96791750214838, 82.25436837582353, 82.55514179318246, 82.89888284159265, 83.14236608421656, 83.37152678315668, 83.52907476367803, 83.71526783156689, 84.0160412489258, 84.2595244915497, 84.30249212260097, 84.33113720996849, 84.37410484101976, 84.40274992838728, 84.38842738470352, 84.41707247207103, 84.4313950157548, 84.46004010312231, 84.46004010312231, 84.47436264680607, 84.46004010312231, 84.44571755943855, 84.50300773417358, 84.54597536522486, 84.63191062732741, 84.68920080206243, 84.7035233457462, 84.73216843311371, 84.8037811515325, 84.86107132626755, 84.91836150100256, 84.93268404468634, 84.96132913205385, 85.03294185047264, 85.10455456889143, 85.23345746204527, 85.27642509309653, 85.3193927241478, 85.36236035519907, 85.43397307361786, 85.44829561730162, 85.53423087940418, 85.67745631624176, 85.72042394729303, 85.77771412202807, 85.8063592093956, 85.84932684044686, 85.87797192781437, 85.94958464623316, 85.96390718991692, 85.99255227728445, 86.078487539387, 86.12145517043827, 86.22171297622458, 86.3506158693784, 86.3506158693784, 86.42222858779719, 86.56545402463478, 86.6227441993698, 86.63706674305357, 86.75164709252363, 86.79461472357491, 86.85190489830994, 86.89487252936121, 86.92351761672873, 87.05242050988255, 87.06674305356631, 87.12403322830134, 87.22429103408766, 87.32454883987396, 87.45345173302778, 87.46777427671154, 87.65396734460039, 87.69693497565167, 87.7542251503867, 87.854482956173, 87.86880549985678, 87.95474076195931, 88.0263534803781, 88.15525637353193, 88.22686909195072, 88.38441707247206, 88.59925522772843, 88.69951303351475, 88.95731881982239, 88.92867373245488, 89.01460899455743, 89.17215697507876, 89.28673732454884, 89.37267258665138, 89.53022056717272, 89.64480091664278, 89.78802635348038, 89.83099398453164, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 70, 103, 134, 159, 192, 218, 244, 270, 293, 315, 334, 355, 383, 401, 404, 406, 409, 414, 416, 419, 421, 424, 425, 427, 427, 429, 435, 440, 446, 450, 451, 456, 461, 468, 472, 476, 480, 488, 493, 505, 510, 516, 520, 528, 534, 541, 554, 560, 568, 573, 579, 584, 592, 596, 602, 610, 615, 625, 637, 643, 652, 666, 672, 678, 688, 695, 700, 706, 710, 720, 727, 738, 748, 757, 770, 775, 792, 800, 809, 821, 827, 838, 849, 862, 872, 885, 905, 916, 938, 948, 959, 981, 995, 1011, 1026, 1042, 1059, 1073, 1087], "harmed": [0, 11, 17, 21, 27, 33, 42, 48, 53, 55, 61, 67, 75, 83, 90, 91, 91, 91, 91, 94, 97, 98, 99, 100, 101, 102, 103, 106, 108, 110, 110, 110, 110, 113, 113, 116, 116, 119, 121, 124, 124, 127, 129, 132, 133, 136, 141, 142, 145, 148, 152, 155, 158, 161, 164, 167, 171, 173, 175, 178, 181, 187, 191, 195, 197, 202, 204, 208, 209, 212, 214, 215, 221, 228, 231, 233, 237, 241, 245, 250, 255, 260, 265, 270, 276, 280, 285, 287, 292, 296, 300, 312, 317, 328, 334, 344, 348, 356, 363, 374, 383], "exact_score": [79.81953594958465, 80.1059868232598, 80.57863076482383, 80.99398453165281, 81.35204812374677, 81.62417645373819, 81.96791750214838, 82.25436837582353, 82.55514179318246, 82.89888284159265, 83.14236608421656, 83.37152678315668, 83.52907476367803, 83.71526783156689, 84.0160412489258, 84.2595244915497, 84.30249212260097, 84.33113720996849, 84.37410484101976, 84.40274992838728, 84.38842738470352, 84.41707247207103, 84.4313950157548, 84.46004010312231, 84.46004010312231, 84.47436264680607, 84.46004010312231, 84.44571755943855, 84.50300773417358, 84.54597536522486, 84.63191062732741, 84.68920080206243, 84.7035233457462, 84.73216843311371, 84.8037811515325, 84.86107132626755, 84.91836150100256, 84.93268404468634, 84.96132913205385, 85.03294185047264, 85.10455456889143, 85.23345746204527, 85.27642509309653, 85.3193927241478, 85.36236035519907, 85.43397307361786, 85.44829561730162, 85.53423087940418, 85.67745631624176, 85.72042394729303, 85.77771412202807, 85.8063592093956, 85.84932684044686, 85.87797192781437, 85.94958464623316, 85.96390718991692, 85.99255227728445, 86.078487539387, 86.12145517043827, 86.22171297622458, 86.3506158693784, 86.3506158693784, 86.42222858779719, 86.56545402463478, 86.6227441993698, 86.63706674305357, 86.75164709252363, 86.79461472357491, 86.85190489830994, 86.89487252936121, 86.92351761672873, 87.05242050988255, 87.06674305356631, 87.12403322830134, 87.22429103408766, 87.32454883987396, 87.45345173302778, 87.46777427671154, 87.65396734460039, 87.69693497565167, 87.7542251503867, 87.854482956173, 87.86880549985678, 87.95474076195931, 88.0263534803781, 88.15525637353193, 88.22686909195072, 88.38441707247206, 88.59925522772843, 88.69951303351475, 88.95731881982239, 88.92867373245488, 89.01460899455743, 89.17215697507876, 89.28673732454884, 89.37267258665138, 89.53022056717272, 89.64480091664278, 89.78802635348038, 89.83099398453164, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.79592500451913, "threshold_area100": 85.79518762532224, "threshold_area25": 83.11228874248067}, "test": {"score": [80.00287005811867, 80.52665566477721, 81.03609098084237, 81.48094998923729, 81.94733443352227, 82.24869053598336, 82.57157207433451, 82.87292817679558, 83.15993398866327, 83.45411494582765, 83.8774485183325, 84.15010403960682, 84.39405897969434, 84.59496304800173, 84.78869197101243, 84.96089545813304, 84.98242089402311, 84.9895960393198, 85.01112147520988, 85.03264691109995, 85.04699720169334, 85.04699720169334, 85.05417234699003, 85.10439836406688, 85.11874865466027, 85.15462438114372, 85.19767525292387, 85.19050010762719, 85.22637583411064, 85.24072612470403, 85.29812728707756, 85.3627035947478, 85.37705388534118, 85.47750591949487, 85.58513309894525, 85.67123484250556, 85.75016144076918, 85.75016144076918, 85.80756260314271, 85.81473774843941, 85.9080146372964, 85.9151897825931, 86.02999210734018, 86.09456841501041, 86.16631986797732, 86.22372103035086, 86.36722393628472, 86.42462509865825, 86.50355169692186, 86.4676759704384, 86.5753031498888, 86.70445576522926, 86.74750663700941, 86.76185692760279, 86.85513381645978, 86.89818468823994, 86.91971012413002, 87.0416875941738, 87.09908875654733, 87.19954079070101, 87.27129224366794, 87.32151826074478, 87.3932697137117, 87.4937217478654, 87.56547320083232, 87.63004950850255, 87.69462581617277, 87.75920212384301, 87.80225299562316, 87.89552988448015, 87.92423046566694, 87.98880677333716, 88.12513453397432, 88.20406113223794, 88.30451316639162, 88.36908947406185, 88.5197675252924, 88.54846810647916, 88.68479586711632, 88.84264906364353, 88.89287508072039, 88.97897682428069, 89.04355313195093, 89.19423118318146, 89.23728205496163, 89.30185836263185, 89.344909234412, 89.47406184975246, 89.57451388390615, 89.63191504627969, 89.76106766162016, 89.8973954222573, 90.06242376408122, 90.14852550764154, 90.21310181531177, 90.30637870416876, 90.44988161010261, 90.50010762717945, 90.55750878955298, 90.59338451603645, 90.73688742197031], "calls": [0.0, 1.0188706321303005, 2.102317571930832, 3.2503408194015924, 4.369663485685585, 5.216330630695271, 6.091698356891727, 7.0172921001650295, 7.971586424625099, 8.940231039678553, 10.253282628973237, 11.064074047499462, 12.276673602640454, 13.295544234770754, 14.1780871062639, 15.031929396570282, 15.778144507426276, 16.83289086604004, 17.844586352873645, 18.892157566190715, 19.946903924804477, 20.922723685154622, 21.769390830164312, 22.924589222931765, 23.943459855062066, 25.191935136686517, 26.045777426992895, 27.028772332639733, 27.73911171701227, 28.585778862021954, 29.446796297625028, 30.53024323742556, 31.26210805768817, 32.24510296333501, 33.364425629619, 34.397646552342685, 35.380641457989526, 36.37081150893305, 37.41120757695343, 38.35115161082012, 39.34849680706034, 40.55392121690464, 41.59431728492502, 42.57013704527517, 43.56748224151539, 44.5720025830523, 45.626748941666065, 46.51646695845591, 47.44206070172921, 48.460931333859506, 49.51567769247327, 50.627825213460575, 51.58211953792065, 52.543589007677404, 53.46918275095071, 54.38760134892732, 55.671952357035224, 56.59754610030853, 57.401162373538064, 58.41285786037167, 59.39585276601851, 60.22099447513812, 61.146588218411424, 62.100882542871496, 63.16280404678195, 63.987945755901556, 65.17901987515246, 66.00416158427208, 67.04455765229245, 67.97015139556576, 69.1468752242233, 70.0581186769032, 71.12004018081366, 72.08868479586712, 73.0286288297338, 74.03314917127072, 75.4466527947191, 76.44399799095932, 77.663772691397, 78.74004448590084, 80.23247470761284, 81.33744708330343, 82.3850182966205, 83.55456697998135, 84.6954150821554, 85.77168687665925, 86.69728061993254, 87.64439979909594, 88.64892014063285, 89.54581330271938, 90.38530530243237, 91.13869555858506, 92.17191648130874, 93.1405610963622, 93.84372533543804, 94.63299131807419, 95.70926311257803, 96.7640094711918, 98.03401018870632, 99.03135538494654, 100.0], "rescued": [0, 81, 160, 235, 308, 358, 411, 465, 520, 571, 642, 686, 736, 774, 810, 836, 839, 840, 846, 850, 853, 856, 859, 868, 872, 881, 889, 891, 896, 902, 910, 923, 928, 944, 962, 978, 993, 997, 1009, 1018, 1035, 1044, 1064, 1078, 1095, 1109, 1130, 1141, 1159, 1166, 1184, 1205, 1215, 1224, 1241, 1255, 1272, 1293, 1309, 1330, 1348, 1360, 1373, 1391, 1412, 1425, 1441, 1455, 1471, 1489, 1506, 1525, 1552, 1570, 1590, 1605, 1635, 1650, 1677, 1706, 1733, 1750, 1771, 1798, 1818, 1843, 1863, 1886, 1916, 1933, 1962, 1988, 2018, 2043, 2057, 2077, 2114, 2139, 2170, 2190, 2225], "harmed": [0, 8, 16, 29, 37, 45, 53, 65, 80, 90, 102, 108, 124, 134, 143, 145, 145, 145, 148, 149, 150, 153, 155, 157, 159, 163, 165, 168, 168, 172, 172, 176, 179, 181, 184, 188, 192, 196, 200, 208, 212, 220, 224, 229, 236, 242, 243, 246, 253, 265, 268, 271, 275, 282, 286, 294, 308, 312, 320, 327, 335, 340, 343, 347, 358, 362, 369, 374, 384, 389, 402, 412, 420, 427, 433, 439, 448, 459, 467, 474, 494, 499, 511, 517, 531, 547, 561, 566, 582, 591, 602, 609, 616, 629, 634, 641, 658, 676, 699, 714, 729], "exact_score": [80.00287005811867, 80.51230537418384, 81.04326612613906, 81.3804979550836, 81.75360551051159, 82.17693908301644, 82.53569634785106, 82.87292817679558, 83.15993398866327, 83.46846523642104, 83.79852192006888, 84.12857860371673, 84.32230752672743, 84.5303867403315, 84.75999138982566, 84.96089545813304, 84.98242089402311, 85.01112147520988, 85.01829662050658, 85.03264691109995, 85.03982205639664, 85.04699720169334, 85.06852263758341, 85.11157350936357, 85.11874865466027, 85.16179952644042, 85.19767525292387, 85.19050010762719, 85.24072612470403, 85.2694267058908, 85.34117815885772, 85.3698787400445, 85.4416301930114, 85.56360766305518, 85.64253426131879, 85.72863600487909, 85.74298629547249, 85.77886202195594, 85.80756260314271, 85.85778862021957, 85.90083949199972, 85.95824065437326, 86.07304297912033, 86.11609385090048, 86.18067015857072, 86.32417306450456, 86.38874937217479, 86.45332567984502, 86.4676759704384, 86.54660256870201, 86.67575518404249, 86.72598120111932, 86.7546817823061, 86.80490779938295, 86.8479586711631, 86.88383439764655, 86.94841070531677, 87.0416875941738, 87.14213962832747, 87.2067159359977, 87.29281767955801, 87.36456913252493, 87.47219631197532, 87.55829805553563, 87.63004950850255, 87.70180096146947, 87.75920212384301, 87.79507785032648, 87.88835473918347, 87.90988017507355, 87.96728133744709, 88.11795938867762, 88.20406113223794, 88.31168831168831, 88.36908947406185, 88.46954150821554, 88.51259237999571, 88.59869412355602, 88.73502188419316, 88.84982420894023, 88.87134964483032, 88.971801678984, 89.05072827724761, 89.1440051661046, 89.19423118318146, 89.2588074908517, 89.29468321733516, 89.38796010619215, 89.52428786682931, 89.6175647556863, 89.69649135394992, 89.84716940518047, 90.04089832819116, 90.13417521704815, 90.24180239649853, 90.37813015713569, 90.47140704599268, 90.50010762717945, 90.55750878955298, 90.59338451603645, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.65302016188029, "threshold_area100": 86.64131448661838, "threshold_area25": 83.74255578675469}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1087, "harmed": 383}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2225, "harmed": 729}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.1324548721313477, "operator": "ge"}, {"budget": 0.02, "threshold": 1.8237948417663574, "operator": "ge"}, {"budget": 0.03, "threshold": 1.5963833332061768, "operator": "ge"}, {"budget": 0.04, "threshold": 1.4160006046295166, "operator": "ge"}, {"budget": 0.05, "threshold": 1.207900047302246, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0839831829071045, "operator": "ge"}, {"budget": 0.07, "threshold": 0.8993838429450989, "operator": "ge"}, {"budget": 0.08, "threshold": 0.7312130928039551, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5585439801216125, "operator": "ge"}, {"budget": 0.1, "threshold": 0.40747562050819397, "operator": "ge"}, {"budget": 0.11, "threshold": 0.29142850637435913, "operator": "ge"}, {"budget": 0.12, "threshold": 0.17688187956809998, "operator": "ge"}, {"budget": 0.13, "threshold": 0.09132224321365356, "operator": "ge"}, {"budget": 0.14, "threshold": 0.005643904209136963, "operator": "ge"}, {"budget": 0.15, "threshold": -0.0757501870393753, "operator": "ge"}, {"budget": 0.16, "threshold": -0.1636357605457306, "operator": "ge"}, {"budget": 0.17, "threshold": -0.2415892332792282, "operator": "ge"}, {"budget": 0.18, "threshold": -0.30558711290359497, "operator": "ge"}, {"budget": 0.19, "threshold": -0.36052992939949036, "operator": "ge"}, {"budget": 0.2, "threshold": -0.42675209045410156, "operator": "ge"}, {"budget": 0.21, "threshold": -0.4946632385253906, "operator": "ge"}, {"budget": 0.22, "threshold": -0.5783985257148743, "operator": "ge"}, {"budget": 0.23, "threshold": -0.6470075249671936, "operator": "ge"}, {"budget": 0.24, "threshold": -0.7115668058395386, "operator": "ge"}, {"budget": 0.25, "threshold": -0.7698681950569153, "operator": "ge"}, {"budget": 0.26, "threshold": -0.8314762115478516, "operator": "ge"}, {"budget": 0.27, "threshold": -0.8961973190307617, "operator": "ge"}, {"budget": 0.28, "threshold": -0.9795791506767273, "operator": "ge"}, {"budget": 0.29, "threshold": -1.0522916316986084, "operator": "ge"}, {"budget": 0.3, "threshold": -1.1239874362945557, "operator": "ge"}, {"budget": 0.31, "threshold": -1.1970298290252686, "operator": "ge"}, {"budget": 0.32, "threshold": -1.2711583375930786, "operator": "ge"}, {"budget": 0.33, "threshold": -1.3310461044311523, "operator": "ge"}, {"budget": 0.34, "threshold": -1.3921908140182495, "operator": "ge"}, {"budget": 0.35, "threshold": -1.464553713798523, "operator": "ge"}, {"budget": 0.36, "threshold": -1.5306110382080078, "operator": "ge"}, {"budget": 0.37, "threshold": -1.5984904766082764, "operator": "ge"}, {"budget": 0.38, "threshold": -1.6798137426376343, "operator": "ge"}, {"budget": 0.39, "threshold": -1.7509591579437256, "operator": "ge"}, {"budget": 0.4, "threshold": -1.8361823558807373, "operator": "ge"}, {"budget": 0.41, "threshold": -1.9023512601852417, "operator": "ge"}, {"budget": 0.42, "threshold": -1.9740777015686035, "operator": "ge"}, {"budget": 0.43, "threshold": -2.058562755584717, "operator": "ge"}, {"budget": 0.44, "threshold": -2.1224305629730225, "operator": "ge"}, {"budget": 0.45, "threshold": -2.1845524311065674, "operator": "ge"}, {"budget": 0.46, "threshold": -2.2557239532470703, "operator": "ge"}, {"budget": 0.47, "threshold": -2.3139286041259766, "operator": "ge"}, {"budget": 0.48, "threshold": -2.3724939823150635, "operator": "ge"}, {"budget": 0.49, "threshold": -2.4273548126220703, "operator": "ge"}, {"budget": 0.5, "threshold": -2.4858908653259277, "operator": "ge"}, {"budget": 0.51, "threshold": -2.5431602001190186, "operator": "ge"}, {"budget": 0.52, "threshold": -2.6057190895080566, "operator": "ge"}, {"budget": 0.53, "threshold": -2.666250467300415, "operator": "ge"}, {"budget": 0.54, "threshold": -2.723875045776367, "operator": "ge"}, {"budget": 0.55, "threshold": -2.775522232055664, "operator": "ge"}, {"budget": 0.56, "threshold": -2.820657730102539, "operator": "ge"}, {"budget": 0.57, "threshold": -2.87185001373291, "operator": "ge"}, {"budget": 0.58, "threshold": -2.926534414291382, "operator": "ge"}, {"budget": 0.59, "threshold": -2.9717960357666016, "operator": "ge"}, {"budget": 0.6, "threshold": -3.019324779510498, "operator": "ge"}, {"budget": 0.61, "threshold": -3.0813302993774414, "operator": "ge"}, {"budget": 0.62, "threshold": -3.1398727893829346, "operator": "ge"}, {"budget": 0.63, "threshold": -3.1852989196777344, "operator": "ge"}, {"budget": 0.64, "threshold": -3.227570056915283, "operator": "ge"}, {"budget": 0.65, "threshold": -3.2696428298950195, "operator": "ge"}, {"budget": 0.66, "threshold": -3.3251490592956543, "operator": "ge"}, {"budget": 0.67, "threshold": -3.3729653358459473, "operator": "ge"}, {"budget": 0.68, "threshold": -3.426719903945923, "operator": "ge"}, {"budget": 0.69, "threshold": -3.4641804695129395, "operator": "ge"}, {"budget": 0.7, "threshold": -3.5037600994110107, "operator": "ge"}, {"budget": 0.71, "threshold": -3.553506851196289, "operator": "ge"}, {"budget": 0.72, "threshold": -3.6047205924987793, "operator": "ge"}, {"budget": 0.73, "threshold": -3.6568684577941895, "operator": "ge"}, {"budget": 0.74, "threshold": -3.7020792961120605, "operator": "ge"}, {"budget": 0.75, "threshold": -3.7421090602874756, "operator": "ge"}, {"budget": 0.76, "threshold": -3.7794711589813232, "operator": "ge"}, {"budget": 0.77, "threshold": -3.8288118839263916, "operator": "ge"}, {"budget": 0.78, "threshold": -3.8780667781829834, "operator": "ge"}, {"budget": 0.79, "threshold": -3.922792673110962, "operator": "ge"}, {"budget": 0.8, "threshold": -3.963148593902588, "operator": "ge"}, {"budget": 0.81, "threshold": -4.01010799407959, "operator": "ge"}, {"budget": 0.82, "threshold": -4.0597333908081055, "operator": "ge"}, {"budget": 0.83, "threshold": -4.108328819274902, "operator": "ge"}, {"budget": 0.84, "threshold": -4.153230667114258, "operator": "ge"}, {"budget": 0.85, "threshold": -4.203420639038086, "operator": "ge"}, {"budget": 0.86, "threshold": -4.255467414855957, "operator": "ge"}, {"budget": 0.87, "threshold": -4.311679840087891, "operator": "ge"}, {"budget": 0.88, "threshold": -4.366223335266113, "operator": "ge"}, {"budget": 0.89, "threshold": -4.433051109313965, "operator": "ge"}, {"budget": 0.9, "threshold": -4.492229461669922, "operator": "ge"}, {"budget": 0.91, "threshold": -4.551447868347168, "operator": "ge"}, {"budget": 0.92, "threshold": -4.630932331085205, "operator": "ge"}, {"budget": 0.93, "threshold": -4.695374965667725, "operator": "ge"}, {"budget": 0.94, "threshold": -4.7690606117248535, "operator": "ge"}, {"budget": 0.95, "threshold": -4.852752208709717, "operator": "ge"}, {"budget": 0.96, "threshold": -4.948860168457031, "operator": "ge"}, {"budget": 0.97, "threshold": -5.0679731369018555, "operator": "ge"}, {"budget": 0.98, "threshold": -5.19709587097168, "operator": "ge"}, {"budget": 0.99, "threshold": -5.440582275390625, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.07734173589229, 80.60727585219134, 80.99398453165281, 81.29475794901174, 81.59553136637066, 81.9965625895159, 82.31165855055858, 82.55514179318246, 82.87023775422514, 83.12804354053279, 83.38584932684044, 83.61501002578058, 83.72959037525064, 84.00171870524204, 84.34545975365224, 84.64623317101116, 84.88971641363506, 85.03294185047264, 85.27642509309653, 85.44829561730162, 85.63448868519048, 85.89229447149813, 86.1071326267545, 86.30764823832712, 86.43655113148094, 86.46519621884846, 86.57977656831852, 86.60842165568604, 86.72300200515612, 86.96648524778, 87.12403322830134, 87.2099684904039, 87.38183901460899, 87.5823546261816, 87.69693497565167, 87.854482956173, 87.98338584932684, 88.11228874248066, 88.28415926668576, 88.32712689773703, 88.29848181036952, 88.31280435405327, 88.45602979089085, 88.52764250930964, 88.6135777714122, 88.69951303351475, 88.81409338298482, 88.84273847035233, 88.91435118877112, 89.01460899455743, 89.10054425665997, 89.14351188771126, 89.20080206244629, 89.30105986823258, 89.3297049556001, 89.41564021770266, 89.50157547980521, 89.53022056717272, 89.53022056717272, 89.58751074190776, 89.61615582927529, 89.5445431108565, 89.61615582927529, 89.61615582927529, 89.58751074190776, 89.55886565454024, 89.53022056717272, 89.63047837295903, 89.61615582927529, 89.58751074190776, 89.573188198224, 89.53022056717272, 89.63047837295903, 89.65912346032655, 89.6734460040103, 89.64480091664278, 89.64480091664278, 89.64480091664278, 89.70209109137782, 89.71641363506157, 89.77370380979661, 89.83099398453164, 89.9169292466342, 89.9455743340017, 89.95989687768548, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.98854196505299, 90.03150959610426, 90.0171870524205, 90.04583213978802, 90.03150959610426, 90.00286450873675, 90.0171870524205, 90.0171870524205, 89.97421942136923, 89.9455743340017, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 30, 70, 102, 131, 158, 190, 220, 244, 270, 292, 314, 337, 354, 382, 410, 436, 456, 471, 496, 514, 533, 555, 572, 591, 604, 614, 627, 636, 650, 670, 686, 696, 712, 729, 741, 754, 766, 781, 796, 805, 811, 816, 828, 836, 847, 857, 869, 877, 883, 892, 900, 906, 913, 923, 927, 936, 945, 951, 955, 959, 963, 966, 973, 975, 979, 981, 983, 991, 993, 995, 997, 998, 1006, 1014, 1016, 1018, 1020, 1023, 1028, 1031, 1036, 1041, 1048, 1050, 1054, 1056, 1057, 1062, 1067, 1071, 1073, 1077, 1079, 1081, 1083, 1085, 1085, 1086, 1087, 1087], "harmed": [0, 12, 15, 20, 28, 34, 38, 46, 53, 57, 61, 65, 72, 81, 90, 94, 99, 102, 107, 115, 121, 127, 131, 133, 138, 142, 150, 155, 162, 168, 171, 176, 180, 184, 187, 191, 193, 196, 202, 205, 211, 219, 223, 225, 228, 233, 237, 241, 247, 248, 250, 252, 255, 258, 261, 263, 266, 269, 273, 277, 277, 279, 287, 289, 291, 297, 301, 305, 306, 309, 313, 316, 320, 321, 327, 328, 332, 334, 337, 338, 340, 341, 342, 343, 343, 346, 347, 349, 356, 357, 358, 361, 363, 366, 370, 371, 373, 376, 379, 383, 383], "exact_score": [79.81953594958465, 80.07734173589229, 80.60727585219134, 80.99398453165281, 81.29475794901174, 81.59553136637066, 81.9965625895159, 82.31165855055858, 82.55514179318246, 82.87023775422514, 83.12804354053279, 83.38584932684044, 83.61501002578058, 83.72959037525064, 84.00171870524204, 84.34545975365224, 84.64623317101116, 84.88971641363506, 85.03294185047264, 85.27642509309653, 85.44829561730162, 85.63448868519048, 85.89229447149813, 86.1071326267545, 86.30764823832712, 86.43655113148094, 86.46519621884846, 86.57977656831852, 86.60842165568604, 86.72300200515612, 86.96648524778, 87.12403322830134, 87.2099684904039, 87.38183901460899, 87.5823546261816, 87.69693497565167, 87.854482956173, 87.98338584932684, 88.11228874248066, 88.28415926668576, 88.32712689773703, 88.29848181036952, 88.31280435405327, 88.45602979089085, 88.52764250930964, 88.6135777714122, 88.69951303351475, 88.81409338298482, 88.84273847035233, 88.91435118877112, 89.01460899455743, 89.10054425665997, 89.14351188771126, 89.20080206244629, 89.30105986823258, 89.3297049556001, 89.41564021770266, 89.50157547980521, 89.53022056717272, 89.53022056717272, 89.58751074190776, 89.61615582927529, 89.5445431108565, 89.61615582927529, 89.61615582927529, 89.58751074190776, 89.55886565454024, 89.53022056717272, 89.63047837295903, 89.61615582927529, 89.58751074190776, 89.573188198224, 89.53022056717272, 89.63047837295903, 89.65912346032655, 89.6734460040103, 89.64480091664278, 89.64480091664278, 89.64480091664278, 89.70209109137782, 89.71641363506157, 89.77370380979661, 89.83099398453164, 89.9169292466342, 89.9455743340017, 89.95989687768548, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.98854196505299, 90.03150959610426, 90.0171870524205, 90.04583213978802, 90.03150959610426, 90.00286450873675, 90.0171870524205, 90.0171870524205, 89.97421942136923, 89.9455743340017, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.69736473401629, "threshold_area100": 87.696648524778, "threshold_area25": 83.52907476367804}, "test": {"score": [80.00287005811867, 80.62710769893091, 81.19394417736959, 81.53835115161083, 81.88275812585205, 82.34914257013705, 82.61462294611465, 82.92315419387243, 83.2245102963335, 83.47564038171772, 83.88462366362918, 84.13575374901342, 84.38688383439765, 84.57343761211165, 84.78869197101243, 85.05417234699003, 85.34117815885772, 85.55643251775848, 85.84343832962618, 86.03716725263688, 86.2882973380211, 86.48920140632848, 86.79773265408625, 86.89818468823994, 86.9340604147234, 87.08473846595393, 87.19236564540432, 87.35739398722825, 87.47937145727201, 87.69462581617277, 87.76637726913971, 88.01033220922724, 88.12513453397432, 88.26146229461148, 88.35473918346847, 88.505417234699, 88.63456985003947, 88.74937217478654, 88.85699935423692, 88.90722537131377, 89.02920284135753, 89.0722537131377, 89.21575661907154, 89.3377340891153, 89.40948554208224, 89.45253641386238, 89.531463012126, 89.57451388390615, 89.69649135394992, 89.74671737102676, 89.8184688239937, 89.85434455047715, 89.93327114874077, 89.94762143933414, 90.01219774700438, 90.04089832819116, 90.09112434526799, 90.19157637942169, 90.26332783238861, 90.32072899476215, 90.3566047212456, 90.34942957594892, 90.40683073832245, 90.39965559302576, 90.42118102891584, 90.45705675539931, 90.47858219128938, 90.51445791777284, 90.50010762717945, 90.48575733658608, 90.52163306306953, 90.51445791777284, 90.50728277247615, 90.51445791777284, 90.57903422544307, 90.60773480662984, 90.61490995192652, 90.62208509722322, 90.62208509722322, 90.65078567840999, 90.68666140489346, 90.69383655019014, 90.72253713137691, 90.744062567267, 90.75841285786038, 90.73688742197031, 90.7297122766736, 90.73688742197031, 90.78711343904715, 90.78711343904715, 90.80146372964053, 90.82298916553061, 90.85168974671738, 90.85886489201407, 90.8301643108273, 90.85168974671738, 90.83733945612398, 90.80146372964053, 90.77993829375045, 90.76558800315708, 90.73688742197031], "calls": [0.0, 1.1910741192509149, 2.396498529095214, 3.3220922723685153, 4.175934562674894, 5.302432374255579, 6.00559661333142, 7.038817536055104, 8.086388749372176, 9.076558800315707, 10.1958814665997, 11.229102389323383, 12.355600200904068, 13.266843653583985, 14.192437396857285, 15.182607447800816, 16.14407691755758, 17.141422113797805, 18.095716438257874, 18.856281839707254, 19.968429360694554, 20.894023103967854, 22.28600129152615, 23.22594532539284, 24.130013632776066, 24.962330487192368, 25.787472196311978, 26.62696419602497, 27.66736026404535, 28.808208366219418, 29.68357609241587, 30.752672741623016, 31.793068809643394, 32.639735954653084, 33.457702518476005, 34.39047140704599, 35.201262825572215, 36.184257731219056, 37.2533543804262, 38.207648704886275, 39.43459855062065, 40.39606802037741, 41.5225658319581, 42.727990241802395, 43.62488340388893, 44.47155054889861, 45.61239865107268, 46.6169189926096, 47.57121331706967, 48.468106479156205, 49.46545167539643, 50.283418239219344, 51.40991605080003, 52.56511444356748, 53.64138623807132, 54.64590657960824, 55.55715003228815, 56.590370955011835, 57.867546817823055, 58.8433665781732, 59.905288082083665, 61.13223792781803, 62.23721030350865, 63.14845375618856, 63.952070029418095, 64.84896319150462, 65.91088469541508, 66.87235416517184, 68.09212886560952, 68.99619717299275, 69.76393771973882, 70.76845806127574, 71.8232044198895, 73.06450455621726, 74.06902489775418, 74.8941666068738, 75.74800889718017, 76.7381789481237, 77.80010045203414, 78.81179593886776, 79.57953648561383, 80.51948051948052, 81.5598765875009, 82.41371887780727, 83.42541436464089, 84.48016072325466, 85.47750591949487, 86.48202626103179, 87.44349573078854, 88.59151897825932, 89.65344048216977, 90.56468393484968, 91.7557580541006, 92.52349860084666, 93.72174786539428, 94.6114658821841, 95.50835904427063, 96.45547822343403, 97.47434885556433, 99.00265480375977, 100.0], "rescued": [0, 96, 183, 241, 296, 370, 413, 469, 528, 576, 637, 685, 736, 771, 810, 855, 904, 948, 995, 1028, 1071, 1109, 1165, 1193, 1213, 1244, 1268, 1303, 1331, 1373, 1392, 1433, 1459, 1484, 1505, 1530, 1553, 1577, 1601, 1625, 1655, 1669, 1695, 1720, 1734, 1749, 1771, 1783, 1805, 1817, 1833, 1843, 1861, 1870, 1887, 1901, 1911, 1928, 1944, 1957, 1968, 1977, 1990, 1994, 2002, 2011, 2019, 2029, 2034, 2041, 2050, 2053, 2058, 2065, 2076, 2082, 2085, 2091, 2094, 2102, 2110, 2116, 2124, 2132, 2139, 2145, 2149, 2152, 2162, 2167, 2177, 2183, 2191, 2195, 2201, 2206, 2209, 2213, 2216, 2220, 2225], "harmed": [0, 9, 17, 27, 34, 43, 49, 62, 79, 92, 96, 109, 125, 134, 143, 151, 160, 174, 181, 187, 195, 205, 218, 232, 247, 257, 266, 278, 289, 301, 310, 317, 327, 333, 341, 345, 350, 358, 367, 384, 397, 405, 411, 419, 423, 432, 443, 449, 454, 459, 465, 470, 477, 484, 492, 502, 505, 508, 514, 519, 525, 535, 540, 545, 550, 554, 559, 564, 571, 580, 584, 588, 594, 600, 602, 604, 606, 611, 614, 618, 621, 626, 630, 635, 640, 649, 654, 656, 659, 664, 672, 675, 679, 682, 692, 694, 699, 708, 714, 720, 729], "exact_score": [80.00287005811867, 80.51948051948052, 81.02891583554567, 81.41637368156705, 81.82535696347851, 82.21281480949989, 82.60744780081797, 82.90880390327905, 83.18145942455335, 83.44693980053096, 83.83439764655235, 84.02095142426634, 84.32948267202411, 84.5375618856282, 84.78151682571571, 85.01112147520988, 85.31247757767095, 85.49903135538494, 85.80038745784603, 86.08021812441702, 86.2954724833178, 86.52507713281194, 86.76185692760279, 86.89100954294324, 86.9340604147234, 87.07756332065725, 87.24259166248117, 87.41479514960179, 87.52959747434886, 87.72332639735954, 87.8596541579967, 88.0533830810074, 88.14665996986439, 88.31168831168831, 88.42649063643539, 88.59151897825932, 88.73502188419316, 88.82829877305016, 88.8856999354237, 88.99332711487408, 89.0722537131377, 89.158355456698, 89.27315778144508, 89.35925952500538, 89.4238358326756, 89.46688670445577, 89.51711272153261, 89.61038961038962, 89.71801678984, 89.78976824280693, 89.83281911458707, 89.91174571285069, 89.95479658463084, 89.98349716581761, 90.01937289230108, 90.06959890937792, 90.10547463586138, 90.23462725120184, 90.25615268709191, 90.32072899476215, 90.3637798665423, 90.3566047212456, 90.39248044772907, 90.40683073832245, 90.42835617421254, 90.46423190069599, 90.48575733658608, 90.51445791777284, 90.50010762717945, 90.48575733658608, 90.52163306306953, 90.50010762717945, 90.50728277247615, 90.51445791777284, 90.56468393484968, 90.60773480662984, 90.6364353878166, 90.61490995192652, 90.62926024251992, 90.65796082370669, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.75841285786038, 90.7297122766736, 90.7297122766736, 90.71536198608023, 90.76558800315708, 90.77276314845376, 90.78711343904715, 90.80146372964053, 90.8301643108273, 90.85168974671738, 90.84451460142068, 90.85168974671738, 90.83733945612398, 90.80863887493722, 90.79428858434385, 90.78711343904715, 90.76558800315708, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.49732698423155, "threshold_area100": 88.4943675109421, "threshold_area25": 84.22400803616273}, "selected_budget": 92, "selected": {"val": {"score": 90.04583213978802, "calls": 91.99369808077915, "rescued": 1077, "harmed": 363}, "test": {"score": 90.85168974671738, "calls": 91.7557580541006, "rescued": 2191, "harmed": 679}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.450027585029602, "operator": "ge"}, {"budget": 0.02, "threshold": 1.2199382781982422, "operator": "ge"}, {"budget": 0.03, "threshold": 1.087388277053833, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9602937698364258, "operator": "ge"}, {"budget": 0.05, "threshold": 0.8589702844619751, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7488866448402405, "operator": "ge"}, {"budget": 0.07, "threshold": 0.657146692276001, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5274801254272461, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4228874444961548, "operator": "ge"}, {"budget": 0.1, "threshold": 0.31847673654556274, "operator": "ge"}, {"budget": 0.11, "threshold": 0.22525474429130554, "operator": "ge"}, {"budget": 0.12, "threshold": 0.13844546675682068, "operator": "ge"}, {"budget": 0.13, "threshold": 0.07070709764957428, "operator": "ge"}, {"budget": 0.14, "threshold": 0.004402875900268555, "operator": "ge"}, {"budget": 0.15, "threshold": -0.06240248680114746, "operator": "ge"}, {"budget": 0.16, "threshold": -0.13200567662715912, "operator": "ge"}, {"budget": 0.17, "threshold": -0.19124938547611237, "operator": "ge"}, {"budget": 0.18, "threshold": -0.2488957643508911, "operator": "ge"}, {"budget": 0.19, "threshold": -0.30137044191360474, "operator": "ge"}, {"budget": 0.2, "threshold": -0.3553439676761627, "operator": "ge"}, {"budget": 0.21, "threshold": -0.4195697009563446, "operator": "ge"}, {"budget": 0.22, "threshold": -0.47503861784935, "operator": "ge"}, {"budget": 0.23, "threshold": -0.5423184633255005, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5976619124412537, "operator": "ge"}, {"budget": 0.25, "threshold": -0.6588001847267151, "operator": "ge"}, {"budget": 0.26, "threshold": -0.720299243927002, "operator": "ge"}, {"budget": 0.27, "threshold": -0.7807288765907288, "operator": "ge"}, {"budget": 0.28, "threshold": -0.849965512752533, "operator": "ge"}, {"budget": 0.29, "threshold": -0.9230074882507324, "operator": "ge"}, {"budget": 0.3, "threshold": -0.9826875925064087, "operator": "ge"}, {"budget": 0.31, "threshold": -1.058634877204895, "operator": "ge"}, {"budget": 0.32, "threshold": -1.1439533233642578, "operator": "ge"}, {"budget": 0.33, "threshold": -1.208972692489624, "operator": "ge"}, {"budget": 0.34, "threshold": -1.2717206478118896, "operator": "ge"}, {"budget": 0.35, "threshold": -1.3300321102142334, "operator": "ge"}, {"budget": 0.36, "threshold": -1.4012889862060547, "operator": "ge"}, {"budget": 0.37, "threshold": -1.474759817123413, "operator": "ge"}, {"budget": 0.38, "threshold": -1.5457484722137451, "operator": "ge"}, {"budget": 0.39, "threshold": -1.6284223794937134, "operator": "ge"}, {"budget": 0.4, "threshold": -1.6961729526519775, "operator": "ge"}, {"budget": 0.41, "threshold": -1.780942678451538, "operator": "ge"}, {"budget": 0.42, "threshold": -1.8531132936477661, "operator": "ge"}, {"budget": 0.43, "threshold": -1.935389757156372, "operator": "ge"}, {"budget": 0.44, "threshold": -2.0027782917022705, "operator": "ge"}, {"budget": 0.45, "threshold": -2.073651075363159, "operator": "ge"}, {"budget": 0.46, "threshold": -2.12995982170105, "operator": "ge"}, {"budget": 0.47, "threshold": -2.190068006515503, "operator": "ge"}, {"budget": 0.48, "threshold": -2.24906849861145, "operator": "ge"}, {"budget": 0.49, "threshold": -2.313288450241089, "operator": "ge"}, {"budget": 0.5, "threshold": -2.3703246116638184, "operator": "ge"}, {"budget": 0.51, "threshold": -2.4258854389190674, "operator": "ge"}, {"budget": 0.52, "threshold": -2.485100030899048, "operator": "ge"}, {"budget": 0.53, "threshold": -2.540583610534668, "operator": "ge"}, {"budget": 0.54, "threshold": -2.593085765838623, "operator": "ge"}, {"budget": 0.55, "threshold": -2.6545116901397705, "operator": "ge"}, {"budget": 0.56, "threshold": -2.704885482788086, "operator": "ge"}, {"budget": 0.57, "threshold": -2.750960111618042, "operator": "ge"}, {"budget": 0.58, "threshold": -2.7957324981689453, "operator": "ge"}, {"budget": 0.59, "threshold": -2.849486827850342, "operator": "ge"}, {"budget": 0.6, "threshold": -2.8951613903045654, "operator": "ge"}, {"budget": 0.61, "threshold": -2.9557578563690186, "operator": "ge"}, {"budget": 0.62, "threshold": -3.0161850452423096, "operator": "ge"}, {"budget": 0.63, "threshold": -3.0626161098480225, "operator": "ge"}, {"budget": 0.64, "threshold": -3.1043782234191895, "operator": "ge"}, {"budget": 0.65, "threshold": -3.1497037410736084, "operator": "ge"}, {"budget": 0.66, "threshold": -3.2008965015411377, "operator": "ge"}, {"budget": 0.67, "threshold": -3.245302438735962, "operator": "ge"}, {"budget": 0.68, "threshold": -3.2896506786346436, "operator": "ge"}, {"budget": 0.69, "threshold": -3.338128089904785, "operator": "ge"}, {"budget": 0.7, "threshold": -3.386604070663452, "operator": "ge"}, {"budget": 0.71, "threshold": -3.4367737770080566, "operator": "ge"}, {"budget": 0.72, "threshold": -3.4768004417419434, "operator": "ge"}, {"budget": 0.73, "threshold": -3.513051748275757, "operator": "ge"}, {"budget": 0.74, "threshold": -3.5649147033691406, "operator": "ge"}, {"budget": 0.75, "threshold": -3.609434127807617, "operator": "ge"}, {"budget": 0.76, "threshold": -3.658602237701416, "operator": "ge"}, {"budget": 0.77, "threshold": -3.6985111236572266, "operator": "ge"}, {"budget": 0.78, "threshold": -3.737445116043091, "operator": "ge"}, {"budget": 0.79, "threshold": -3.7840535640716553, "operator": "ge"}, {"budget": 0.8, "threshold": -3.8279969692230225, "operator": "ge"}, {"budget": 0.81, "threshold": -3.874055862426758, "operator": "ge"}, {"budget": 0.82, "threshold": -3.9128530025482178, "operator": "ge"}, {"budget": 0.83, "threshold": -3.963965892791748, "operator": "ge"}, {"budget": 0.84, "threshold": -4.00949764251709, "operator": "ge"}, {"budget": 0.85, "threshold": -4.0580525398254395, "operator": "ge"}, {"budget": 0.86, "threshold": -4.1120219230651855, "operator": "ge"}, {"budget": 0.87, "threshold": -4.168242454528809, "operator": "ge"}, {"budget": 0.88, "threshold": -4.219691276550293, "operator": "ge"}, {"budget": 0.89, "threshold": -4.273544788360596, "operator": "ge"}, {"budget": 0.9, "threshold": -4.340367317199707, "operator": "ge"}, {"budget": 0.91, "threshold": -4.406552314758301, "operator": "ge"}, {"budget": 0.92, "threshold": -4.471044540405273, "operator": "ge"}, {"budget": 0.93, "threshold": -4.530369758605957, "operator": "ge"}, {"budget": 0.94, "threshold": -4.608028888702393, "operator": "ge"}, {"budget": 0.95, "threshold": -4.690717697143555, "operator": "ge"}, {"budget": 0.96, "threshold": -4.791511535644531, "operator": "ge"}, {"budget": 0.97, "threshold": -4.893845558166504, "operator": "ge"}, {"budget": 0.98, "threshold": -5.0293803215026855, "operator": "ge"}, {"budget": 0.99, "threshold": -5.237693786621094, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.1775995416786, 80.60727585219134, 80.99398453165281, 81.33772558006301, 81.71011171584072, 82.05385276425092, 82.29733600687481, 82.68404468633629, 82.81294757949011, 83.11372099684904, 83.42881695789171, 83.65797765683185, 83.7439129189344, 84.00171870524204, 84.35978229733601, 84.63191062732741, 84.90403895731882, 84.97565167573761, 85.17616728731022, 85.43397307361786, 85.69177885992552, 85.87797192781437, 86.09281008307075, 86.26468060727585, 86.36493841306215, 86.40790604411343, 86.56545402463478, 86.70867946147234, 86.80893726725866, 87.00945287883127, 87.16700085935261, 87.3102262961902, 87.45345173302778, 87.53938699513033, 87.69693497565167, 87.76854769407046, 87.96906330564308, 88.08364365511314, 88.19822400458321, 88.31280435405327, 88.34144944142079, 88.42738470352333, 88.48467487825837, 88.57061014036091, 88.6135777714122, 88.68519048983099, 88.82841592666857, 88.85706101403609, 88.91435118877112, 89.0575766256087, 89.11486680034373, 89.15783443139502, 89.22944714981381, 89.35835004296763, 89.38699513033515, 89.42996276138642, 89.55886565454024, 89.55886565454024, 89.55886565454024, 89.573188198224, 89.53022056717272, 89.5445431108565, 89.58751074190776, 89.573188198224, 89.5445431108565, 89.60183328559151, 89.55886565454024, 89.58751074190776, 89.60183328559151, 89.58751074190776, 89.5445431108565, 89.60183328559151, 89.63047837295903, 89.6734460040103, 89.70209109137782, 89.68776854769406, 89.65912346032655, 89.68776854769406, 89.74505872242909, 89.77370380979661, 89.85963907189917, 89.8453165282154, 89.90260670295044, 89.9169292466342, 89.97421942136923, 89.98854196505299, 89.97421942136923, 89.97421942136923, 89.97421942136923, 90.03150959610426, 90.0171870524205, 90.07447722715554, 90.03150959610426, 90.0171870524205, 90.00286450873675, 90.0171870524205, 89.95989687768548, 89.9455743340017, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 35, 69, 100, 132, 161, 192, 217, 251, 268, 290, 316, 338, 355, 382, 410, 435, 456, 470, 491, 514, 535, 554, 575, 592, 602, 612, 627, 642, 656, 675, 691, 704, 716, 727, 742, 752, 768, 784, 795, 808, 815, 825, 834, 842, 849, 858, 870, 876, 886, 898, 904, 910, 918, 928, 934, 939, 950, 955, 958, 962, 963, 968, 973, 976, 979, 985, 987, 991, 995, 996, 996, 1002, 1006, 1013, 1019, 1020, 1023, 1027, 1031, 1034, 1041, 1044, 1049, 1052, 1056, 1058, 1061, 1064, 1066, 1072, 1073, 1077, 1080, 1081, 1083, 1084, 1085, 1086, 1087, 1087], "harmed": [0, 10, 14, 18, 26, 29, 36, 44, 51, 59, 60, 64, 70, 81, 90, 93, 99, 101, 110, 117, 122, 125, 131, 137, 142, 145, 152, 156, 161, 168, 173, 178, 181, 183, 188, 192, 197, 199, 207, 210, 215, 220, 224, 229, 231, 235, 239, 241, 245, 251, 253, 255, 258, 261, 262, 266, 268, 270, 275, 278, 281, 285, 289, 291, 295, 300, 302, 307, 309, 312, 314, 317, 319, 321, 325, 329, 331, 336, 338, 338, 339, 340, 344, 345, 347, 347, 348, 352, 355, 357, 359, 361, 361, 367, 369, 372, 372, 377, 379, 383, 383], "exact_score": [79.81953594958465, 80.1775995416786, 80.60727585219134, 80.99398453165281, 81.33772558006301, 81.71011171584072, 82.05385276425092, 82.29733600687481, 82.68404468633629, 82.81294757949011, 83.11372099684904, 83.42881695789171, 83.65797765683185, 83.7439129189344, 84.00171870524204, 84.35978229733601, 84.63191062732741, 84.90403895731882, 84.97565167573761, 85.17616728731022, 85.43397307361786, 85.69177885992552, 85.87797192781437, 86.09281008307075, 86.26468060727585, 86.36493841306215, 86.40790604411343, 86.56545402463478, 86.70867946147234, 86.80893726725866, 87.00945287883127, 87.16700085935261, 87.3102262961902, 87.45345173302778, 87.53938699513033, 87.69693497565167, 87.76854769407046, 87.96906330564308, 88.08364365511314, 88.19822400458321, 88.31280435405327, 88.34144944142079, 88.42738470352333, 88.48467487825837, 88.57061014036091, 88.6135777714122, 88.68519048983099, 88.82841592666857, 88.85706101403609, 88.91435118877112, 89.0575766256087, 89.11486680034373, 89.15783443139502, 89.22944714981381, 89.35835004296763, 89.38699513033515, 89.42996276138642, 89.55886565454024, 89.55886565454024, 89.55886565454024, 89.573188198224, 89.53022056717272, 89.5445431108565, 89.58751074190776, 89.573188198224, 89.5445431108565, 89.60183328559151, 89.55886565454024, 89.58751074190776, 89.60183328559151, 89.58751074190776, 89.5445431108565, 89.60183328559151, 89.63047837295903, 89.6734460040103, 89.70209109137782, 89.68776854769406, 89.65912346032655, 89.68776854769406, 89.74505872242909, 89.77370380979661, 89.85963907189917, 89.8453165282154, 89.90260670295044, 89.9169292466342, 89.97421942136923, 89.98854196505299, 89.97421942136923, 89.97421942136923, 89.97421942136923, 90.03150959610426, 90.0171870524205, 90.07447722715554, 90.03150959610426, 90.0171870524205, 90.00286450873675, 90.0171870524205, 89.95989687768548, 89.9455743340017, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.70866563535598, "threshold_area100": 87.70796333428817, "threshold_area25": 83.54196505299342}, "test": {"score": [80.00287005811867, 80.71320944249122, 81.19394417736959, 81.63162804046782, 82.06931190356606, 82.37066800602713, 82.69354954437827, 83.00208079213604, 83.2173351510368, 83.52586639879458, 83.92049939011265, 84.16445433020019, 84.40840927028772, 84.60213819329842, 84.78869197101243, 85.04699720169334, 85.3627035947478, 85.5492573724618, 85.78603716725264, 86.0084666714501, 86.32417306450456, 86.5753031498888, 86.7546817823061, 86.86230896175648, 86.96276099591017, 86.99146157709694, 87.17801535481094, 87.35021884193155, 87.4937217478654, 87.55829805553563, 87.665925234986, 87.88117959388678, 88.12513453397432, 88.26146229461148, 88.41931549113869, 88.52694267058908, 88.66327043122624, 88.78524790127, 88.84982420894023, 88.92875080720385, 89.04355313195093, 89.10095429432447, 89.20140632847816, 89.32338379852193, 89.4310109779723, 89.48841214034584, 89.54581330271938, 89.58168902920285, 89.65344048216977, 89.811293678697, 89.83999425988375, 89.91174571285069, 89.91892085814737, 89.94044629403746, 89.97632202052091, 90.07677405467462, 90.13417521704815, 90.1772260888283, 90.25615268709191, 90.33507928535553, 90.3566047212456, 90.37813015713569, 90.40683073832245, 90.40683073832245, 90.44270646480591, 90.45705675539931, 90.45705675539931, 90.51445791777284, 90.52880820836621, 90.48575733658608, 90.49293248188276, 90.50010762717945, 90.52880820836621, 90.56468393484968, 90.59338451603645, 90.62208509722322, 90.62926024251992, 90.64361053311329, 90.6364353878166, 90.65796082370669, 90.69383655019014, 90.72253713137691, 90.73688742197031, 90.77993829375045, 90.73688742197031, 90.72253713137691, 90.7297122766736, 90.75841285786038, 90.77276314845376, 90.79428858434385, 90.81581402023392, 90.85168974671738, 90.85168974671738, 90.8301643108273, 90.8301643108273, 90.85886489201407, 90.80146372964053, 90.79428858434385, 90.77993829375045, 90.75841285786038, 90.73688742197031], "calls": [0.0, 1.2125995551409916, 2.339097366721676, 3.264691109994977, 4.31943746860874, 5.151754323025042, 6.299777570495803, 7.038817536055104, 8.186840783525867, 9.169835689172706, 10.296333500753391, 11.329554423477076, 12.334074765013991, 13.324244815957522, 14.192437396857285, 15.168257157207435, 16.14407691755758, 17.04097007964411, 18.002439549400876, 18.985434455047713, 19.98995479658463, 21.07340173638516, 21.99899547965846, 23.132668436535837, 23.964985290952143, 24.826002726555213, 25.859223649278896, 26.677190213101813, 27.574083375188348, 28.679055750878955, 29.48984716940518, 30.50154265623879, 31.75001793786324, 32.776063715290235, 33.54380426203631, 34.218267919925374, 35.1223362273086, 36.22013345770252, 37.160077491569204, 38.28657530314989, 39.2982707899835, 40.37454258448734, 41.46516466958456, 42.792566549472625, 43.675109420965775, 44.73703092487623, 45.51194661691899, 46.5236421037526, 47.61426418884982, 48.68336083805697, 49.62330487192366, 50.383870273373034, 51.45296692258018, 52.49336299060056, 53.40460644328048, 54.538279400157855, 55.650426921145154, 56.561670373825066, 57.52313984358183, 58.74291454401952, 59.67568343258951, 61.02461074836766, 62.194159431728494, 63.08387744851834, 63.96642032001149, 64.95659037095501, 65.95393556719523, 66.85082872928176, 67.91275023319223, 68.98902202769605, 69.9791920786396, 71.05546387314344, 71.93083159933988, 72.64834612900911, 73.97574800889718, 74.9659180598407, 76.07806558082801, 76.85298127287078, 77.663772691397, 78.79744564827438, 79.71586424625099, 80.70603429719452, 81.40202339097367, 82.57157207433451, 83.44693980053096, 84.55191217622156, 85.58513309894525, 86.61117887637224, 87.52242232905216, 88.48389179880893, 89.65344048216977, 90.78711343904715, 91.77010834469398, 92.66700150678051, 93.7863241730645, 94.65451675396427, 95.58011049723757, 96.36937647987371, 97.52457487264117, 98.91655306019946, 100.0], "rescued": [0, 106, 184, 253, 319, 367, 428, 478, 526, 580, 644, 690, 737, 776, 810, 855, 906, 945, 988, 1029, 1076, 1119, 1157, 1188, 1215, 1236, 1275, 1305, 1333, 1361, 1385, 1421, 1463, 1489, 1514, 1536, 1563, 1586, 1607, 1632, 1655, 1673, 1697, 1722, 1741, 1759, 1773, 1788, 1802, 1828, 1843, 1854, 1861, 1875, 1888, 1909, 1922, 1932, 1945, 1961, 1971, 1982, 1993, 1997, 2006, 2015, 2021, 2031, 2040, 2045, 2050, 2055, 2062, 2070, 2080, 2086, 2091, 2096, 2098, 2106, 2114, 2122, 2127, 2139, 2140, 2145, 2150, 2156, 2163, 2171, 2178, 2188, 2192, 2195, 2201, 2208, 2209, 2214, 2217, 2220, 2225], "harmed": [0, 7, 18, 26, 31, 37, 53, 60, 78, 89, 98, 110, 123, 135, 143, 152, 159, 172, 182, 192, 195, 203, 216, 232, 245, 262, 275, 281, 289, 308, 317, 323, 331, 338, 341, 348, 356, 362, 374, 388, 395, 405, 415, 423, 427, 437, 443, 453, 457, 461, 472, 473, 479, 490, 498, 505, 510, 514, 516, 521, 528, 536, 543, 547, 551, 558, 564, 566, 573, 584, 588, 592, 595, 598, 604, 606, 610, 613, 616, 621, 624, 628, 631, 637, 644, 651, 655, 657, 662, 667, 671, 676, 680, 686, 692, 695, 704, 710, 715, 721, 729], "exact_score": [80.00287005811867, 80.56253139126068, 80.97868981846884, 81.50247542512736, 81.9329841429289, 82.33479227954366, 82.58592236492788, 82.97338021094927, 83.19580971514674, 83.43976465523427, 83.81287221066228, 84.0640022960465, 84.30795723613403, 84.5375618856282, 84.78151682571571, 85.01829662050658, 85.2766018511875, 85.53490708186841, 85.77886202195594, 86.0156418167468, 86.32417306450456, 86.5681280045921, 86.7546817823061, 86.8479586711631, 86.95558585061349, 87.0273373035804, 87.2138910812944, 87.41479514960179, 87.51524718375548, 87.60134892731578, 87.80942814091985, 88.00315706393054, 88.18971084164454, 88.28298773050155, 88.505417234699, 88.64892014063285, 88.75654732008323, 88.83547391834685, 88.92157566190716, 89.00050226017078, 89.05790342254431, 89.16553060199469, 89.28750807203846, 89.36643467030207, 89.45971155915908, 89.47406184975246, 89.56733873860946, 89.60321446509292, 89.73236708043338, 89.83281911458707, 89.86869484107054, 89.8973954222573, 89.95479658463084, 89.97632202052091, 90.00502260170768, 90.09112434526799, 90.14852550764154, 90.20592667001507, 90.25615268709191, 90.33507928535553, 90.37813015713569, 90.37813015713569, 90.41400588361914, 90.39248044772907, 90.44270646480591, 90.45705675539931, 90.45705675539931, 90.52163306306953, 90.53598335366291, 90.48575733658608, 90.49293248188276, 90.50010762717945, 90.52880820836621, 90.56468393484968, 90.58620937073975, 90.62926024251992, 90.6364353878166, 90.64361053311329, 90.65078567840999, 90.65796082370669, 90.70818684078354, 90.7297122766736, 90.73688742197031, 90.75123771256368, 90.71536198608023, 90.73688742197031, 90.744062567267, 90.77276314845376, 90.77993829375045, 90.80146372964053, 90.85168974671738, 90.86604003731075, 90.84451460142068, 90.8301643108273, 90.83733945612398, 90.84451460142068, 90.80863887493722, 90.77993829375045, 90.77993829375045, 90.75841285786038, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.50535983147734, "threshold_area100": 88.50254717658032, "threshold_area25": 84.2319006959891}, "selected_budget": 92, "selected": {"val": {"score": 90.07447722715554, "calls": 91.99369808077915, "rescued": 1077, "harmed": 361}, "test": {"score": 90.85168974671738, "calls": 91.77010834469398, "rescued": 2192, "harmed": 680}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.8453165282154, "calls": 98.99742194213692}, "test": {"score": 90.80146372964053, "calls": 99.03853053024324}}, "old_normalized_interior": {"val": {"score": 89.9169292466342, "calls": 97.99484388427385}, "test": {"score": 90.80146372964053, "calls": 97.78288010332209}}, "old_normalized_endpoints": {"val": {"score": 89.9169292466342, "calls": 97.99484388427385}, "test": {"score": 90.80146372964053, "calls": 97.78288010332209}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.12030936694356, 80.53566313377254, 80.9653394442853, 81.38069321111429, 81.6528215411057, 82.05385276425092, 82.3403036379261, 82.54081924949871, 82.76997994843884, 83.05643082211401, 83.25694643368662, 83.47178458894298, 83.72959037525064, 83.9873961615583, 84.13062159839588, 84.38842738470352, 84.60326553995989, 84.8037811515325, 84.97565167573761, 85.30507018046404, 85.4912632483529, 85.63448868519048, 85.82068175307934, 86.02119736465195, 86.16442280148954, 86.3506158693784, 86.52248639358349, 86.60842165568604, 86.75164709252363, 86.86622744199369, 87.03809796619879, 87.08106559725006, 87.29590375250645, 87.39616155829275, 87.4820968203953, 87.4820968203953, 87.59667716986537, 87.59667716986537, 87.79719278143799, 87.854482956173, 87.88312804354052, 88.0263534803781, 88.12661128616442, 88.11228874248066, 88.28415926668576, 88.41306215983958, 88.49899742194212, 88.62790031509596, 88.69951303351475, 88.7854482956173, 88.8857061014036, 88.92867373245488, 89.08622171297623, 89.15783443139502, 89.24376969349757, 89.3297049556001, 89.31538241191636, 89.38699513033515, 89.37267258665138, 89.4013176740189, 89.48725293612145, 89.5445431108565, 89.60183328559151, 89.5445431108565, 89.573188198224, 89.53022056717272, 89.55886565454024, 89.58751074190776, 89.55886565454024, 89.53022056717272, 89.50157547980521, 89.5445431108565, 89.573188198224, 89.63047837295903, 89.65912346032655, 89.6734460040103, 89.65912346032655, 89.65912346032655, 89.6734460040103, 89.74505872242909, 89.78802635348038, 89.78802635348038, 89.80234889716414, 89.75938126611285, 89.78802635348038, 89.81667144084788, 89.85963907189917, 89.87396161558291, 89.8453165282154, 89.83099398453164, 89.81667144084788, 89.83099398453164, 89.85963907189917, 89.8453165282154, 89.8453165282154, 89.83099398453164, 89.87396161558291, 89.9169292466342, 89.9169292466342, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.12030936694356, 80.53566313377254, 80.9653394442853, 81.38069321111429, 81.6528215411057, 82.05385276425092, 82.3403036379261, 82.54081924949871, 82.76997994843884, 83.05643082211401, 83.25694643368662, 83.47178458894298, 83.72959037525064, 83.9873961615583, 84.13062159839588, 84.38842738470352, 84.60326553995989, 84.8037811515325, 84.97565167573761, 85.30507018046404, 85.4912632483529, 85.63448868519048, 85.82068175307934, 86.02119736465195, 86.16442280148954, 86.3506158693784, 86.52248639358349, 86.60842165568604, 86.75164709252363, 86.86622744199369, 87.03809796619879, 87.08106559725006, 87.29590375250645, 87.39616155829275, 87.4820968203953, 87.4820968203953, 87.59667716986537, 87.59667716986537, 87.79719278143799, 87.854482956173, 87.88312804354052, 88.0263534803781, 88.12661128616442, 88.11228874248066, 88.28415926668576, 88.41306215983958, 88.49899742194212, 88.62790031509596, 88.69951303351475, 88.7854482956173, 88.8857061014036, 88.92867373245488, 89.08622171297623, 89.15783443139502, 89.24376969349757, 89.3297049556001, 89.31538241191636, 89.38699513033515, 89.37267258665138, 89.4013176740189, 89.48725293612145, 89.5445431108565, 89.60183328559151, 89.5445431108565, 89.573188198224, 89.53022056717272, 89.55886565454024, 89.58751074190776, 89.55886565454024, 89.53022056717272, 89.50157547980521, 89.5445431108565, 89.573188198224, 89.63047837295903, 89.65912346032655, 89.6734460040103, 89.65912346032655, 89.65912346032655, 89.6734460040103, 89.74505872242909, 89.78802635348038, 89.78802635348038, 89.80234889716414, 89.75938126611285, 89.78802635348038, 89.81667144084788, 89.85963907189917, 89.87396161558291, 89.8453165282154, 89.83099398453164, 89.81667144084788, 89.83099398453164, 89.85963907189917, 89.8453165282154, 89.8453165282154, 89.83099398453164, 89.87396161558291, 89.9169292466342, 89.9169292466342, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 83.42738470352334, "exact_area100": 87.56144211234829}, "test": {"score": [80.00287005811867, 80.46925450240367, 80.97151467317214, 81.32309679271006, 81.63162804046782, 81.98321016000575, 82.49982062136758, 82.6720241084882, 83.02360622802613, 83.3249623304872, 83.63349357824497, 83.89179880892588, 84.18597976609027, 84.44428499677119, 84.68823993685872, 84.93219487694627, 85.1689746717371, 85.41292961182464, 85.61383368013202, 85.75016144076918, 85.93671521848317, 86.1878453038674, 86.41027480806487, 86.60400373107555, 86.76185692760279, 86.85513381645978, 86.92688526942672, 87.19236564540432, 87.4004448590084, 87.43632058549186, 87.51524718375548, 87.68745067087609, 87.78072755973308, 87.9529310468537, 88.08208366219417, 88.18253569634786, 88.26146229461148, 88.37626461935855, 88.46236636291886, 88.57716868766593, 88.72784673889646, 88.86417449953362, 88.98615196957739, 89.17988089258809, 89.28750807203846, 89.4238358326756, 89.48841214034584, 89.58886417449953, 89.65344048216977, 89.69649135394992, 89.82564396929038, 89.85434455047715, 89.96914687522423, 90.01219774700438, 90.07677405467462, 90.04089832819116, 90.12700007175145, 90.25615268709191, 90.32072899476215, 90.41400588361914, 90.47140704599268, 90.49293248188276, 90.52163306306953, 90.5503336442563, 90.60055966133315, 90.6364353878166, 90.6364353878166, 90.66513596900337, 90.67231111430006, 90.69383655019014, 90.67948625959676, 90.69383655019014, 90.7297122766736, 90.744062567267, 90.79428858434385, 90.75841285786038, 90.71536198608023, 90.70818684078354, 90.69383655019014, 90.72253713137691, 90.68666140489346, 90.70818684078354, 90.69383655019014, 90.67231111430006, 90.71536198608023, 90.72253713137691, 90.73688742197031, 90.79428858434385, 90.81581402023392, 90.85168974671738, 90.82298916553061, 90.83733945612398, 90.86604003731075, 90.87321518260745, 90.85168974671738, 90.85886489201407, 90.83733945612398, 90.81581402023392, 90.80146372964053, 90.75123771256368, 90.73688742197031], "calls": [0.0, 0.9471191791633782, 2.2171198966779078, 3.3077419817751306, 4.22616057975174, 5.2737317930688095, 6.522207074693262, 7.160795006098873, 8.380569706536557, 9.428140919853627, 10.504412714357466, 11.652435961828226, 12.656956303365144, 13.926957020879675, 15.146731721317359, 16.287579823491427, 17.299275310325037, 18.095716438257874, 19.006959890937793, 19.946903924804477, 20.88684795867116, 22.049221496735306, 23.053741838272224, 23.972160436248835, 24.861878453038674, 25.55069240152113, 26.490636435387817, 27.566908229891656, 28.320298486044344, 28.94453612685657, 30.14996053670087, 31.010977972303937, 31.8791705532037, 32.991318074191, 34.03171414221138, 35.32324029561599, 36.35646121833967, 37.325105833393124, 38.171772978402814, 38.99691468752242, 40.30996627681711, 41.27861089187056, 42.27595608811078, 43.409629044988165, 44.36392336944823, 45.33256798450169, 46.3370883260386, 47.29138265049867, 48.460931333859506, 49.307598478869195, 50.46279687163665, 51.46731721317357, 52.586639879457564, 53.47635789624739, 54.57415512664131, 55.671952357035224, 56.83432589509938, 57.709693621295834, 58.613761928679054, 59.876587500896896, 60.65150319293966, 61.821051876300494, 62.71076989309034, 63.7583411064074, 64.70546028557078, 65.5234268493937, 66.26964196024969, 67.2813374470833, 68.23563177154337, 69.34777929253067, 70.2518475999139, 71.31376910382436, 72.26088828298774, 73.15778144507426, 74.29862954724832, 75.54710482887278, 76.37942168328908, 77.3982923154194, 78.25213460572577, 79.44320872497667, 80.50513022888714, 81.48812513453397, 82.6720241084882, 83.75547104828873, 84.76716653512234, 85.64253426131879, 86.51072684221856, 87.27129224366793, 88.21841142283131, 89.158355456698, 90.00502260170768, 91.04541866972806, 92.22931764368228, 93.1333859510655, 93.82937504484465, 94.84824567697497, 95.63033651431442, 96.8501112147521, 97.78288010332209, 99.04570567553993, 100.0], "exact_score": [80.00287005811867, 80.49795508359044, 80.88541292961183, 81.20111932266629, 81.53117600631413, 81.88993327114873, 82.29174140776351, 82.62179809141136, 82.96620506565259, 83.16710913395997, 83.49716581760781, 83.74829590299204, 84.01377627896966, 84.23620578316712, 84.4371098514745, 84.66671450096864, 84.86044342397933, 85.06852263758341, 85.38422903063787, 85.61383368013202, 85.75733658606588, 85.96541579966994, 86.1878453038674, 86.40309966276818, 86.60400373107555, 86.76903207289948, 86.86948410705317, 87.0273373035804, 87.32151826074478, 87.43632058549186, 87.51524718375548, 87.6802755255794, 87.78072755973308, 87.9529310468537, 88.07490851689748, 88.14665996986439, 88.23993685872139, 88.36191432876515, 88.44801607232546, 88.57716868766593, 88.68479586711632, 88.82829877305016, 88.96462653368731, 89.09377914902777, 89.23010690966493, 89.39513525148884, 89.48123699504916, 89.56016359331277, 89.59603931979623, 89.70366649924661, 89.76106766162016, 89.8184688239937, 89.92609600344407, 89.97632202052091, 90.01219774700438, 90.05524861878453, 90.04807347348785, 90.15570065293824, 90.277678122982, 90.34225443065222, 90.42835617421254, 90.47140704599268, 90.49293248188276, 90.48575733658608, 90.53598335366291, 90.61490995192652, 90.62208509722322, 90.6364353878166, 90.66513596900337, 90.69383655019014, 90.67948625959676, 90.67231111430006, 90.7297122766736, 90.73688742197031, 90.77276314845376, 90.77993829375045, 90.75123771256368, 90.72253713137691, 90.68666140489346, 90.71536198608023, 90.72253713137691, 90.68666140489346, 90.69383655019014, 90.68666140489346, 90.66513596900337, 90.72253713137691, 90.72253713137691, 90.77276314845376, 90.80146372964053, 90.85168974671738, 90.82298916553061, 90.83733945612398, 90.85168974671738, 90.87321518260745, 90.85168974671738, 90.85168974671738, 90.83733945612398, 90.81581402023392, 90.80146372964053, 90.75841285786038, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 83.98292315419388, "exact_area100": 88.36537216498422}}, "old_normalized_endpoint_budget": 98}, {"seed": 2029, "selection": {"best_epoch": 9, "epochs": 17, "alpha": 1.0, "validation_area100": 0.8766698522804598, "training_seconds": 48.40590292797424, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9339406479612187, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8929210042921071, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8415322697157012, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7838137023357972, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7419238350726366, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7062922367537551, "operator": "ge"}, {"budget": 0.07, "threshold": 0.6622286004900831, "operator": "ge"}, {"budget": 0.08, "threshold": 0.6196136275732562, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5666875624326596, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5261862014729847, "operator": "ge"}, {"budget": 0.11, "threshold": 0.47614882399362585, "operator": "ge"}, {"budget": 0.12, "threshold": 0.4332491789333997, "operator": "ge"}, {"budget": 0.13, "threshold": 0.4031076755423651, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3699106403299344, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3369056200033365, "operator": "ge"}, {"budget": 0.16, "threshold": 0.2988614281231707, "operator": "ge"}, {"budget": 0.17, "threshold": 0.2703523153954322, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2466433255788667, "operator": "ge"}, {"budget": 0.19, "threshold": 0.21339842269862944, "operator": "ge"}, {"budget": 0.2, "threshold": 0.18548109628530213, "operator": "ge"}, {"budget": 0.21, "threshold": 0.15050214175267196, "operator": "ge"}, {"budget": 0.22, "threshold": 0.12477776991592858, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0828921119882148, "operator": "ge"}, {"budget": 0.24, "threshold": 0.058242884262569385, "operator": "ge"}, {"budget": 0.25, "threshold": 0.03115514449864885, "operator": "ge"}, {"budget": 0.26, "threshold": 0.0011534581988989223, "operator": "ge"}, {"budget": 0.27, "threshold": -0.032471320133305455, "operator": "ge"}, {"budget": 0.28, "threshold": -0.061549524343457104, "operator": "ge"}, {"budget": 0.29, "threshold": -0.09204701704016709, "operator": "ge"}, {"budget": 0.3, "threshold": -0.13151414142563034, "operator": "ge"}, {"budget": 0.31, "threshold": -0.16471466063272247, "operator": "ge"}, {"budget": 0.32, "threshold": -0.2037263098132393, "operator": "ge"}, {"budget": 0.33, "threshold": -0.2436173671380537, "operator": "ge"}, {"budget": 0.34, "threshold": -0.2831808120826812, "operator": "ge"}, {"budget": 0.35, "threshold": -0.3151117752811451, "operator": "ge"}, {"budget": 0.36, "threshold": -0.34648735052832313, "operator": "ge"}, {"budget": 0.37, "threshold": -0.3935133634668304, "operator": "ge"}, {"budget": 0.38, "threshold": -0.4348407204183451, "operator": "ge"}, {"budget": 0.39, "threshold": -0.472924425148193, "operator": "ge"}, {"budget": 0.4, "threshold": -0.511191337869416, "operator": "ge"}, {"budget": 0.41, "threshold": -0.542572079187449, "operator": "ge"}, {"budget": 0.42, "threshold": -0.5759584587475522, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6155512922599905, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6477111513185101, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6738906500992284, "operator": "ge"}, {"budget": 0.46, "threshold": -0.6962752159028445, "operator": "ge"}, {"budget": 0.47, "threshold": -0.7219252300992675, "operator": "ge"}, {"budget": 0.48, "threshold": -0.7507362755041136, "operator": "ge"}, {"budget": 0.49, "threshold": -0.7715502978468054, "operator": "ge"}, {"budget": 0.5, "threshold": -0.7881771814700558, "operator": "ge"}, {"budget": 0.51, "threshold": -0.8067499783708529, "operator": "ge"}, {"budget": 0.52, "threshold": -0.825690776108637, "operator": "ge"}, {"budget": 0.53, "threshold": -0.8436868781765003, "operator": "ge"}, {"budget": 0.54, "threshold": -0.8612359558305712, "operator": "ge"}, {"budget": 0.55, "threshold": -0.8760174216696363, "operator": "ge"}, {"budget": 0.56, "threshold": -0.8871262358892615, "operator": "ge"}, {"budget": 0.57, "threshold": -0.8978680611165493, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9078801506871024, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9178378812361606, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9265964679916457, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9326107183078365, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9398281283336279, "operator": "ge"}, {"budget": 0.63, "threshold": -0.947457730900078, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9528067347882826, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9578898014894084, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9618616497155078, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9651342428486779, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9692453002155833, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9725555358663525, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9754318778142532, "operator": "ge"}, {"budget": 0.71, "threshold": -0.977719985486779, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9802911380828407, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9822494823274015, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9841118877429554, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9857314807740007, "operator": "ge"}, {"budget": 0.76, "threshold": -0.987271676205729, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9886136814568143, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9898076501679521, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9912021113316735, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9922768146411803, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9932347004449256, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9939424347299218, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9947252986709583, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9954741443489745, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9960778132491145, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9966070937986906, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9970470896393155, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9975813891471584, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9979232049818962, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9982001645611629, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9985118892670425, "operator": "ge"}, {"budget": 0.92, "threshold": -0.99875096953053, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9989556863483838, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9991531257469267, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9993377894881212, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9995195003552104, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9996520131461029, "operator": "ge"}, {"budget": 0.98, "threshold": -0.999777167602359, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9998916608015959, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.14895445431108, 80.44972787167, 80.77914637639644, 81.10856488112287, 81.46662847321684, 81.79604697794328, 82.0681753079347, 82.2686909195073, 82.58378688054998, 82.87023775422514, 83.19965625895158, 83.38584932684044, 83.65797765683185, 83.93010598682325, 84.15926668576338, 84.44571755943855, 84.66055571469494, 84.94700658837009, 85.23345746204527, 85.4912632483529, 85.72042394729303, 86.02119736465195, 86.22171297622458, 86.37926095674591, 86.39358350042967, 86.50816384989973, 86.65138928673733, 86.76596963620739, 86.93784016041248, 87.03809796619879, 87.1383557719851, 87.2099684904039, 87.29590375250645, 87.39616155829275, 87.45345173302778, 87.65396734460039, 87.71125751933543, 87.86880549985678, 87.95474076195931, 88.05499856774563, 88.22686909195072, 88.31280435405327, 88.4417072472071, 88.59925522772843, 88.79977083930105, 88.84273847035233, 88.82841592666857, 88.94299627613864, 89.07189916929245, 89.11486680034373, 89.1291893440275, 89.20080206244629, 89.28673732454884, 89.35835004296763, 89.38699513033515, 89.53022056717272, 89.55886565454024, 89.60183328559151, 89.573188198224, 89.48725293612145, 89.51589802348897, 89.50157547980521, 89.573188198224, 89.63047837295903, 89.63047837295903, 89.6734460040103, 89.63047837295903, 89.70209109137782, 89.78802635348038, 89.80234889716414, 89.78802635348038, 89.81667144084788, 89.83099398453164, 89.80234889716414, 89.78802635348038, 89.81667144084788, 89.77370380979661, 89.81667144084788, 89.80234889716414, 89.81667144084788, 89.88828415926669, 89.90260670295044, 89.8453165282154, 89.9169292466342, 89.83099398453164, 89.87396161558291, 89.83099398453164, 89.81667144084788, 89.80234889716414, 89.78802635348038, 89.78802635348038, 89.77370380979661, 89.77370380979661, 89.78802635348038, 89.81667144084788, 89.8453165282154, 89.87396161558291, 89.88828415926669, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 30, 60, 88, 115, 146, 173, 201, 220, 244, 264, 288, 307, 331, 352, 374, 397, 420, 445, 471, 493, 511, 535, 553, 569, 577, 590, 609, 626, 642, 653, 668, 679, 690, 702, 712, 729, 740, 758, 770, 781, 797, 806, 820, 834, 848, 855, 862, 875, 886, 891, 898, 906, 913, 923, 931, 942, 950, 955, 957, 960, 966, 968, 976, 982, 985, 991, 998, 1004, 1013, 1016, 1018, 1022, 1030, 1032, 1033, 1036, 1037, 1041, 1045, 1047, 1053, 1055, 1057, 1063, 1064, 1067, 1067, 1068, 1069, 1070, 1071, 1071, 1073, 1074, 1076, 1079, 1081, 1084, 1086, 1087], "harmed": [0, 7, 16, 21, 25, 31, 35, 44, 49, 51, 51, 52, 58, 63, 65, 71, 74, 82, 87, 93, 97, 99, 102, 106, 111, 118, 123, 132, 141, 145, 149, 157, 163, 168, 173, 179, 182, 189, 196, 202, 206, 210, 213, 218, 221, 221, 225, 233, 238, 240, 242, 248, 251, 252, 257, 263, 264, 270, 272, 276, 285, 289, 292, 295, 297, 300, 303, 313, 314, 317, 319, 322, 324, 331, 335, 337, 338, 342, 343, 348, 349, 350, 351, 357, 358, 365, 365, 368, 370, 372, 374, 375, 376, 378, 378, 378, 379, 379, 381, 382, 383], "exact_score": [79.81953594958465, 80.14895445431108, 80.44972787167, 80.77914637639644, 81.10856488112287, 81.46662847321684, 81.79604697794328, 82.0681753079347, 82.2686909195073, 82.58378688054998, 82.87023775422514, 83.19965625895158, 83.38584932684044, 83.65797765683185, 83.93010598682325, 84.15926668576338, 84.44571755943855, 84.66055571469494, 84.94700658837009, 85.23345746204527, 85.4912632483529, 85.72042394729303, 86.02119736465195, 86.22171297622458, 86.37926095674591, 86.39358350042967, 86.50816384989973, 86.65138928673733, 86.76596963620739, 86.93784016041248, 87.03809796619879, 87.1383557719851, 87.2099684904039, 87.29590375250645, 87.39616155829275, 87.45345173302778, 87.65396734460039, 87.71125751933543, 87.86880549985678, 87.95474076195931, 88.05499856774563, 88.22686909195072, 88.31280435405327, 88.4417072472071, 88.59925522772843, 88.79977083930105, 88.84273847035233, 88.82841592666857, 88.94299627613864, 89.07189916929245, 89.11486680034373, 89.1291893440275, 89.20080206244629, 89.28673732454884, 89.35835004296763, 89.38699513033515, 89.53022056717272, 89.55886565454024, 89.60183328559151, 89.573188198224, 89.48725293612145, 89.51589802348897, 89.50157547980521, 89.573188198224, 89.63047837295903, 89.63047837295903, 89.6734460040103, 89.63047837295903, 89.70209109137782, 89.78802635348038, 89.80234889716414, 89.78802635348038, 89.81667144084788, 89.83099398453164, 89.80234889716414, 89.78802635348038, 89.81667144084788, 89.77370380979661, 89.81667144084788, 89.80234889716414, 89.81667144084788, 89.88828415926669, 89.90260670295044, 89.8453165282154, 89.9169292466342, 89.83099398453164, 89.87396161558291, 89.83099398453164, 89.81667144084788, 89.80234889716414, 89.78802635348038, 89.78802635348038, 89.77370380979661, 89.77370380979661, 89.78802635348038, 89.81667144084788, 89.8453165282154, 89.87396161558291, 89.88828415926669, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.66698522804597, "threshold_area100": 87.66628473216842, "threshold_area25": 83.4187911773131}, "test": {"score": [80.00287005811867, 80.52665566477721, 80.87106263901845, 81.1867690320729, 81.7392552199182, 82.10518763004951, 82.37066800602713, 82.6648489631915, 83.05948195450958, 83.52586639879458, 83.80569706536558, 84.16445433020019, 84.4371098514745, 84.6164884838918, 84.8030422616058, 85.03264691109995, 85.20485039822057, 85.39857932123127, 85.61383368013202, 85.87931405610964, 86.05151754323025, 86.2954724833178, 86.54660256870201, 86.79055750878956, 86.95558585061349, 87.1134390471407, 87.24976680777786, 87.47937145727201, 87.63722465379924, 87.73050154265624, 87.90988017507355, 88.06773337160078, 88.22558656812801, 88.29016287579825, 88.49106694410563, 88.55564325177585, 88.65609528592955, 88.87134964483032, 89.01485255076416, 89.10812943962115, 89.18705603788477, 89.29468321733516, 89.39513525148884, 89.49558728564254, 89.54581330271938, 89.63191504627969, 89.73954222573008, 89.83999425988375, 89.86869484107054, 89.8902202769606, 89.904570567554, 89.96197172992754, 89.96914687522423, 90.02654803759776, 90.03372318289445, 90.09112434526799, 90.14135036234484, 90.16287579823492, 90.1772260888283, 90.19875152471838, 90.30637870416876, 90.30637870416876, 90.33507928535553, 90.39965559302576, 90.45705675539931, 90.47858219128938, 90.51445791777284, 90.54315849895961, 90.61490995192652, 90.69383655019014, 90.73688742197031, 90.7297122766736, 90.75841285786038, 90.7297122766736, 90.76558800315708, 90.77993829375045, 90.77993829375045, 90.78711343904715, 90.75123771256368, 90.73688742197031, 90.75841285786038, 90.75841285786038, 90.73688742197031, 90.744062567267, 90.73688742197031, 90.744062567267, 90.744062567267, 90.73688742197031, 90.744062567267, 90.744062567267, 90.75841285786038, 90.76558800315708, 90.77993829375045, 90.77993829375045, 90.77993829375045, 90.75841285786038, 90.76558800315708, 90.77276314845376, 90.75841285786038, 90.73688742197031, 90.73688742197031], "calls": [0.0, 1.033220922723685, 1.7937863241730645, 2.7911315204132885, 4.118533400301356, 5.11587859654158, 5.905144579177728, 6.98859151897826, 8.143789911745714, 9.557293535194088, 10.662265910884695, 11.867690320728995, 13.022888713496448, 13.783454114945828, 14.658821841142283, 15.562890148525508, 16.538709908875653, 17.34232618210519, 18.11724187414795, 19.143287651574944, 20.14063284781517, 21.28865609528593, 22.25012556504269, 23.656454043194376, 24.53182176939083, 25.371313769103825, 26.37583411064074, 27.29425270861735, 28.047642964770038, 28.951711272153265, 30.07820908373395, 30.989452536413864, 32.0657243309177, 32.991318074191, 34.16086675755184, 35.14386166319868, 35.92595250053814, 37.160077491569204, 38.26504986725981, 39.24086962760996, 40.431943746860874, 41.50104039606802, 42.58448733586855, 43.91188921575662, 44.93793499318362, 45.94245533472053, 46.50211666786252, 47.49946186410275, 48.66183540216689, 49.551553418956736, 50.391045418669734, 51.25206285427279, 52.28528377699648, 53.36155557150032, 54.445002511300856, 55.55715003228815, 56.39664203200114, 57.30071033938437, 58.26935495443782, 59.202123843007826, 60.29274592810504, 61.046136184257726, 62.129583124058264, 63.37088326038602, 64.3180024395494, 65.31534763578962, 66.262466814953, 67.13065939585276, 68.25715720743345, 69.2329769677836, 70.18727129224366, 70.96936212958312, 72.06715935997704, 72.83489990672311, 73.74614335940302, 74.62151108559948, 75.64038171772978, 76.62337662337663, 77.62072181961685, 78.96964913539499, 79.93829375044844, 81.05761641673243, 81.96885986941236, 83.0451316639162, 84.16445433020019, 85.1761498170338, 86.20937073975749, 87.22106622659109, 88.45519121762216, 89.4238358326756, 90.34225443065222, 91.44722680634283, 92.32976967783597, 93.03293391691182, 94.01592882255866, 94.95587285642534, 96.13977183037956, 97.00078926598263, 98.01248475281625, 99.08158140202339, 100.0], "rescued": [0, 81, 133, 190, 274, 329, 372, 421, 487, 563, 613, 673, 720, 750, 785, 824, 858, 892, 930, 973, 1010, 1054, 1099, 1151, 1181, 1216, 1249, 1286, 1315, 1338, 1376, 1405, 1438, 1461, 1498, 1520, 1540, 1582, 1610, 1631, 1654, 1676, 1699, 1723, 1737, 1759, 1776, 1796, 1812, 1823, 1834, 1853, 1866, 1880, 1893, 1908, 1919, 1927, 1936, 1947, 1968, 1976, 1985, 1998, 2011, 2020, 2029, 2038, 2053, 2068, 2077, 2081, 2089, 2094, 2105, 2109, 2116, 2121, 2124, 2131, 2138, 2142, 2144, 2151, 2158, 2165, 2170, 2175, 2182, 2186, 2191, 2195, 2199, 2203, 2207, 2210, 2216, 2219, 2221, 2222, 2225], "harmed": [0, 8, 12, 25, 32, 36, 42, 50, 61, 72, 83, 93, 102, 107, 116, 123, 133, 140, 148, 154, 167, 177, 187, 205, 212, 225, 239, 244, 251, 261, 274, 281, 292, 306, 315, 328, 334, 346, 354, 362, 374, 381, 390, 400, 407, 417, 419, 425, 437, 445, 454, 465, 477, 483, 495, 502, 506, 511, 518, 526, 532, 540, 545, 549, 554, 560, 564, 569, 574, 578, 581, 586, 590, 599, 605, 607, 614, 618, 626, 635, 639, 643, 648, 654, 662, 668, 673, 679, 685, 689, 692, 695, 697, 701, 705, 711, 716, 718, 722, 726, 729], "exact_score": [80.00287005811867, 80.50513022888714, 80.94998923728205, 81.25852048503982, 81.72490492932482, 82.05496161297266, 82.3921934419172, 82.6720241084882, 83.02360622802613, 83.33931262108058, 83.61914328765158, 83.95637511659612, 84.17880462079358, 84.4299347061778, 84.64518906507857, 84.88196885986942, 85.11157350936357, 85.29812728707756, 85.57795795364856, 85.8290880390328, 86.07304297912033, 86.26677190213103, 86.48920140632848, 86.66858003874579, 86.8407835258664, 87.01298701298703, 87.2138910812944, 87.44349573078856, 87.63722465379924, 87.73767668795294, 87.89552988448015, 88.06773337160078, 88.21123627753462, 88.29016287579825, 88.45519121762216, 88.55564325177585, 88.67762072181962, 88.85699935423692, 88.95745138839062, 89.12247973021454, 89.12965487551124, 89.21575661907154, 89.29468321733516, 89.40231039678554, 89.50993757623593, 89.54581330271938, 89.63909019157639, 89.76824280691685, 89.84716940518047, 89.86869484107054, 89.8902202769606, 89.95479658463084, 89.94762143933414, 90.00502260170768, 90.01219774700438, 90.05524861878453, 90.12700007175145, 90.16287579823492, 90.18440123412499, 90.22027696060846, 90.27050297768531, 90.31355384946545, 90.32790414005885, 90.37813015713569, 90.45705675539931, 90.46423190069599, 90.51445791777284, 90.55750878955298, 90.60773480662984, 90.67231111430006, 90.7297122766736, 90.7297122766736, 90.75841285786038, 90.73688742197031, 90.76558800315708, 90.78711343904715, 90.78711343904715, 90.77993829375045, 90.75841285786038, 90.73688742197031, 90.75841285786038, 90.75841285786038, 90.744062567267, 90.75123771256368, 90.7297122766736, 90.75123771256368, 90.75123771256368, 90.744062567267, 90.744062567267, 90.73688742197031, 90.744062567267, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.77993829375045, 90.75841285786038, 90.77276314845376, 90.77276314845376, 90.75841285786038, 90.73688742197031, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.48354634982464, "threshold_area100": 88.5128076343546, "threshold_area25": 84.110640740475}, "selected_budget": 84, "selected": {"val": {"score": 89.9169292466342, "calls": 83.9873961615583, "rescued": 1063, "harmed": 358}, "test": {"score": 90.73688742197031, "calls": 84.16445433020019, "rescued": 2158, "harmed": 662}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8443073034286499, "operator": "ge"}, {"budget": 0.02, "threshold": 0.7420724630355835, "operator": "ge"}, {"budget": 0.03, "threshold": 0.6209361553192139, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5233756303787231, "operator": "ge"}, {"budget": 0.05, "threshold": 0.45173168182373047, "operator": "ge"}, {"budget": 0.06, "threshold": 0.37479355931282043, "operator": "ge"}, {"budget": 0.07, "threshold": 0.33472496271133423, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2905581295490265, "operator": "ge"}, {"budget": 0.09, "threshold": 0.25384318828582764, "operator": "ge"}, {"budget": 0.1, "threshold": 0.22771885991096497, "operator": "ge"}, {"budget": 0.11, "threshold": 0.19559317827224731, "operator": "ge"}, {"budget": 0.12, "threshold": 0.1696920543909073, "operator": "ge"}, {"budget": 0.13, "threshold": 0.14348739385604858, "operator": "ge"}, {"budget": 0.14, "threshold": 0.12580378353595734, "operator": "ge"}, {"budget": 0.15, "threshold": 0.11008375138044357, "operator": "ge"}, {"budget": 0.16, "threshold": 0.0883764773607254, "operator": "ge"}, {"budget": 0.17, "threshold": 0.07540334761142731, "operator": "ge"}, {"budget": 0.18, "threshold": 0.060300663113594055, "operator": "ge"}, {"budget": 0.19, "threshold": 0.04783952236175537, "operator": "ge"}, {"budget": 0.2, "threshold": 0.03565627336502075, "operator": "ge"}, {"budget": 0.21, "threshold": 0.027529485523700714, "operator": "ge"}, {"budget": 0.22, "threshold": 0.02028101496398449, "operator": "ge"}, {"budget": 0.23, "threshold": 0.014209773391485214, "operator": "ge"}, {"budget": 0.24, "threshold": 0.008088789880275726, "operator": "ge"}, {"budget": 0.25, "threshold": 0.0034031756222248077, "operator": "ge"}, {"budget": 0.26, "threshold": 9.375158697366714e-05, "operator": "ge"}, {"budget": 0.27, "threshold": -0.0011635050177574158, "operator": "ge"}, {"budget": 0.28, "threshold": -0.001874609850347042, "operator": "ge"}, {"budget": 0.29, "threshold": -0.002415379509329796, "operator": "ge"}, {"budget": 0.3, "threshold": -0.002975569572299719, "operator": "ge"}, {"budget": 0.31, "threshold": -0.0034697349183261395, "operator": "ge"}, {"budget": 0.32, "threshold": -0.003962968476116657, "operator": "ge"}, {"budget": 0.33, "threshold": -0.004454713314771652, "operator": "ge"}, {"budget": 0.34, "threshold": -0.0050000702030956745, "operator": "ge"}, {"budget": 0.35, "threshold": -0.005573790520429611, "operator": "ge"}, {"budget": 0.36, "threshold": -0.00606617983430624, "operator": "ge"}, {"budget": 0.37, "threshold": -0.006581267341971397, "operator": "ge"}, {"budget": 0.38, "threshold": -0.0070378221571445465, "operator": "ge"}, {"budget": 0.39, "threshold": -0.007596184965223074, "operator": "ge"}, {"budget": 0.4, "threshold": -0.008070224896073341, "operator": "ge"}, {"budget": 0.41, "threshold": -0.008648112416267395, "operator": "ge"}, {"budget": 0.42, "threshold": -0.009206019341945648, "operator": "ge"}, {"budget": 0.43, "threshold": -0.009814813733100891, "operator": "ge"}, {"budget": 0.44, "threshold": -0.010427122004330158, "operator": "ge"}, {"budget": 0.45, "threshold": -0.011045912280678749, "operator": "ge"}, {"budget": 0.46, "threshold": -0.01191092561930418, "operator": "ge"}, {"budget": 0.47, "threshold": -0.012599743902683258, "operator": "ge"}, {"budget": 0.48, "threshold": -0.013257182203233242, "operator": "ge"}, {"budget": 0.49, "threshold": -0.013937130570411682, "operator": "ge"}, {"budget": 0.5, "threshold": -0.014697876758873463, "operator": "ge"}, {"budget": 0.51, "threshold": -0.01560432929545641, "operator": "ge"}, {"budget": 0.52, "threshold": -0.016236305236816406, "operator": "ge"}, {"budget": 0.53, "threshold": -0.017087087035179138, "operator": "ge"}, {"budget": 0.54, "threshold": -0.017900807783007622, "operator": "ge"}, {"budget": 0.55, "threshold": -0.018697798252105713, "operator": "ge"}, {"budget": 0.56, "threshold": -0.019520964473485947, "operator": "ge"}, {"budget": 0.57, "threshold": -0.02055223286151886, "operator": "ge"}, {"budget": 0.58, "threshold": -0.021284081041812897, "operator": "ge"}, {"budget": 0.59, "threshold": -0.022101767361164093, "operator": "ge"}, {"budget": 0.6, "threshold": -0.023113053292036057, "operator": "ge"}, {"budget": 0.61, "threshold": -0.024068422615528107, "operator": "ge"}, {"budget": 0.62, "threshold": -0.025039535015821457, "operator": "ge"}, {"budget": 0.63, "threshold": -0.02623005583882332, "operator": "ge"}, {"budget": 0.64, "threshold": -0.027429068461060524, "operator": "ge"}, {"budget": 0.65, "threshold": -0.02834664285182953, "operator": "ge"}, {"budget": 0.66, "threshold": -0.02948746830224991, "operator": "ge"}, {"budget": 0.67, "threshold": -0.030734626576304436, "operator": "ge"}, {"budget": 0.68, "threshold": -0.03194915130734444, "operator": "ge"}, {"budget": 0.69, "threshold": -0.03320233151316643, "operator": "ge"}, {"budget": 0.7, "threshold": -0.03490597382187843, "operator": "ge"}, {"budget": 0.71, "threshold": -0.03646910935640335, "operator": "ge"}, {"budget": 0.72, "threshold": -0.03807978332042694, "operator": "ge"}, {"budget": 0.73, "threshold": -0.03950527310371399, "operator": "ge"}, {"budget": 0.74, "threshold": -0.04159017279744148, "operator": "ge"}, {"budget": 0.75, "threshold": -0.043432679027318954, "operator": "ge"}, {"budget": 0.76, "threshold": -0.045576658099889755, "operator": "ge"}, {"budget": 0.77, "threshold": -0.04759138450026512, "operator": "ge"}, {"budget": 0.78, "threshold": -0.04992866516113281, "operator": "ge"}, {"budget": 0.79, "threshold": -0.05193387717008591, "operator": "ge"}, {"budget": 0.8, "threshold": -0.054592229425907135, "operator": "ge"}, {"budget": 0.81, "threshold": -0.05704823508858681, "operator": "ge"}, {"budget": 0.82, "threshold": -0.059807922691106796, "operator": "ge"}, {"budget": 0.83, "threshold": -0.06265593320131302, "operator": "ge"}, {"budget": 0.84, "threshold": -0.06535029411315918, "operator": "ge"}, {"budget": 0.85, "threshold": -0.06874626129865646, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0719851404428482, "operator": "ge"}, {"budget": 0.87, "threshold": -0.07583577185869217, "operator": "ge"}, {"budget": 0.88, "threshold": -0.08003440499305725, "operator": "ge"}, {"budget": 0.89, "threshold": -0.08489874005317688, "operator": "ge"}, {"budget": 0.9, "threshold": -0.09135189652442932, "operator": "ge"}, {"budget": 0.91, "threshold": -0.0976654514670372, "operator": "ge"}, {"budget": 0.92, "threshold": -0.10482896864414215, "operator": "ge"}, {"budget": 0.93, "threshold": -0.1119665876030922, "operator": "ge"}, {"budget": 0.94, "threshold": -0.12107885628938675, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1338147222995758, "operator": "ge"}, {"budget": 0.96, "threshold": -0.1447102576494217, "operator": "ge"}, {"budget": 0.97, "threshold": -0.1634274125099182, "operator": "ge"}, {"budget": 0.98, "threshold": -0.19666926562786102, "operator": "ge"}, {"budget": 0.99, "threshold": -0.2583232522010803, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.1059868232598, 80.50701804640504, 80.95101690060154, 81.20882268690919, 81.55256373531938, 81.89630478372959, 82.21140074477226, 82.58378688054998, 82.89888284159265, 83.14236608421656, 83.42881695789171, 83.75823546261816, 84.10197651102835, 84.44571755943855, 84.64623317101116, 84.86107132626755, 85.09023202520767, 85.30507018046404, 85.50558579203667, 85.74906903466055, 85.89229447149813, 86.1071326267545, 86.23603551990833, 86.36493841306215, 86.47951876253221, 86.52248639358349, 86.6227441993698, 86.6227441993698, 86.6227441993698, 86.66571183042107, 86.70867946147234, 86.68003437410484, 86.68003437410484, 86.73732454883987, 86.73732454883987, 86.80893726725866, 86.85190489830994, 86.88054998567745, 86.96648524778, 86.99513033514752, 86.98080779146376, 86.98080779146376, 86.99513033514752, 87.08106559725006, 87.09538814093382, 87.09538814093382, 87.10971068461758, 87.15267831566887, 87.16700085935261, 87.19564594672013, 87.19564594672013, 87.2386135777714, 87.26725866513893, 87.28158120882267, 87.32454883987396, 87.35319392724148, 87.43912918934402, 87.43912918934402, 87.42480664566027, 87.45345173302778, 87.49641936407906, 87.55370953881409, 87.65396734460039, 87.73990260670294, 87.72558006301918, 87.76854769407046, 87.8258378688055, 87.854482956173, 87.94041821827557, 87.91177313090805, 87.91177313090805, 87.94041821827557, 88.01203093669436, 88.06932111142939, 88.14093382984818, 88.19822400458321, 88.31280435405327, 88.41306215983958, 88.48467487825837, 88.62790031509596, 88.64222285877972, 88.6135777714122, 88.71383557719851, 88.75680320824978, 88.79977083930105, 88.98596390718991, 89.02893153824118, 89.0575766256087, 89.17215697507876, 89.24376969349757, 89.41564021770266, 89.48725293612145, 89.60183328559151, 89.61615582927529, 89.68776854769406, 89.71641363506157, 89.80234889716414, 89.80234889716414, 89.85963907189917, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 67, 104, 129, 158, 187, 214, 243, 270, 291, 315, 344, 372, 399, 417, 439, 459, 476, 497, 516, 532, 551, 560, 572, 585, 591, 599, 601, 604, 608, 612, 612, 613, 617, 620, 626, 629, 632, 638, 642, 644, 645, 647, 654, 660, 662, 666, 672, 674, 678, 681, 688, 691, 697, 701, 704, 712, 717, 719, 723, 729, 736, 745, 753, 757, 761, 768, 773, 780, 786, 793, 797, 804, 815, 824, 832, 842, 849, 856, 867, 874, 881, 891, 900, 906, 923, 934, 944, 955, 965, 982, 992, 1002, 1012, 1022, 1033, 1045, 1054, 1070, 1087], "harmed": [0, 12, 19, 25, 32, 37, 42, 47, 50, 55, 59, 63, 69, 73, 76, 80, 87, 91, 93, 100, 102, 108, 112, 112, 115, 120, 123, 124, 126, 129, 130, 131, 133, 134, 134, 137, 138, 138, 139, 139, 141, 144, 145, 146, 147, 152, 154, 157, 160, 161, 163, 166, 170, 171, 176, 177, 178, 180, 185, 188, 190, 193, 196, 198, 200, 205, 206, 209, 212, 213, 221, 228, 230, 232, 239, 243, 247, 249, 249, 251, 252, 258, 267, 270, 276, 279, 283, 291, 299, 302, 307, 312, 317, 319, 328, 333, 342, 348, 357, 369, 383], "exact_score": [79.81953594958465, 80.1059868232598, 80.50701804640504, 80.95101690060154, 81.20882268690919, 81.55256373531938, 81.89630478372959, 82.21140074477226, 82.58378688054998, 82.89888284159265, 83.14236608421656, 83.42881695789171, 83.75823546261816, 84.10197651102835, 84.44571755943855, 84.64623317101116, 84.86107132626755, 85.09023202520767, 85.30507018046404, 85.50558579203667, 85.74906903466055, 85.89229447149813, 86.1071326267545, 86.23603551990833, 86.36493841306215, 86.47951876253221, 86.52248639358349, 86.6227441993698, 86.6227441993698, 86.6227441993698, 86.66571183042107, 86.70867946147234, 86.68003437410484, 86.68003437410484, 86.73732454883987, 86.73732454883987, 86.80893726725866, 86.85190489830994, 86.88054998567745, 86.96648524778, 86.99513033514752, 86.98080779146376, 86.98080779146376, 86.99513033514752, 87.08106559725006, 87.09538814093382, 87.09538814093382, 87.10971068461758, 87.15267831566887, 87.16700085935261, 87.19564594672013, 87.19564594672013, 87.2386135777714, 87.26725866513893, 87.28158120882267, 87.32454883987396, 87.35319392724148, 87.43912918934402, 87.43912918934402, 87.42480664566027, 87.45345173302778, 87.49641936407906, 87.55370953881409, 87.65396734460039, 87.73990260670294, 87.72558006301918, 87.76854769407046, 87.8258378688055, 87.854482956173, 87.94041821827557, 87.91177313090805, 87.91177313090805, 87.94041821827557, 88.01203093669436, 88.06932111142939, 88.14093382984818, 88.19822400458321, 88.31280435405327, 88.41306215983958, 88.48467487825837, 88.62790031509596, 88.64222285877972, 88.6135777714122, 88.71383557719851, 88.75680320824978, 88.79977083930105, 88.98596390718991, 89.02893153824118, 89.0575766256087, 89.17215697507876, 89.24376969349757, 89.41564021770266, 89.48725293612145, 89.60183328559151, 89.61615582927529, 89.68776854769406, 89.71641363506157, 89.80234889716414, 89.80234889716414, 89.85963907189917, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 86.80923573905844, "threshold_area100": 86.80850759094815, "threshold_area25": 83.64795187625322}, "test": {"score": [80.00287005811867, 80.56253139126068, 80.97868981846884, 81.53835115161083, 81.85405754466528, 82.32044198895028, 82.65767381789482, 82.9518547750592, 83.31778718519051, 83.62631843294827, 83.85592308244242, 84.09987802252996, 84.31513238143073, 84.58778790270503, 84.86761856927603, 85.01829662050658, 85.29095214178088, 85.4416301930114, 85.75733658606588, 86.0084666714501, 86.25242161153763, 86.43897538925164, 86.66858003874579, 86.76903207289948, 86.96993614120686, 87.1277893377341, 87.26411709837124, 87.3143431154481, 87.34304369663487, 87.3932697137117, 87.41479514960179, 87.45784602138194, 87.52959747434886, 87.56547320083232, 87.63004950850255, 87.64439979909594, 87.665925234986, 87.73050154265624, 87.73767668795294, 87.78790270502978, 87.83812872210662, 87.8596541579967, 87.89552988448015, 87.88835473918347, 87.90988017507355, 87.97445648274378, 88.01750735452394, 88.0533830810074, 88.09643395278755, 88.11078424338093, 88.12513453397432, 88.18971084164454, 88.23993685872139, 88.30451316639162, 88.31886345698501, 88.3260386022817, 88.34038889287508, 88.36191432876515, 88.36191432876515, 88.36191432876515, 88.412140345842, 88.46954150821554, 88.46954150821554, 88.47671665351223, 88.49824208940231, 88.53411781588576, 88.53411781588576, 88.59151897825932, 88.691971012413, 88.76372246537993, 88.74219702948984, 88.81394848245677, 88.84982420894023, 88.91440051661046, 89.03637798665423, 89.11530458491785, 89.158355456698, 89.2659826361484, 89.24445720025831, 89.30185836263185, 89.33055894381862, 89.4238358326756, 89.50276243093923, 89.48841214034584, 89.60321446509292, 89.63191504627969, 89.63909019157639, 89.69649135394992, 89.71801678984, 89.811293678697, 89.87586998636723, 90.01937289230108, 90.14852550764154, 90.1772260888283, 90.25615268709191, 90.39248044772907, 90.43553131950922, 90.50728277247615, 90.59338451603645, 90.66513596900337, 90.73688742197031], "calls": [0.0, 1.054746358613762, 1.9372892301069096, 3.142713639951209, 4.1974599985649705, 5.201980340101887, 6.421755040539572, 7.211021023175719, 8.244241945899404, 9.162660543876013, 9.908875654732007, 10.99949773982923, 11.989667790772764, 13.03006385879314, 13.898256439692904, 14.601420678768745, 16.022099447513813, 16.804190284853266, 17.973738968214107, 19.02848532682787, 20.27696060845232, 21.245605223505777, 22.30035158211954, 23.068092128865608, 24.1874147951496, 25.292387170840207, 26.39735954653082, 27.351653870990887, 28.40640022960465, 29.23154193872426, 30.114084810217406, 31.05402884408409, 32.11595034799455, 33.077419817751306, 34.117815885771684, 35.201262825572215, 36.09098084236206, 37.14572720097582, 38.04979550835905, 39.118892157566194, 40.030135610246106, 41.04900624237641, 42.11092774628686, 43.237425557867546, 44.16301930114085, 45.00251130085384, 46.294037454258444, 47.30573294109205, 48.303078137332285, 49.26454760708904, 50.283418239219344, 51.567769247327256, 52.2709334864031, 53.23957810145655, 54.179522135323246, 55.20556791275023, 56.13116165602353, 57.32223577527444, 57.989524287866836, 58.83619143287652, 59.89093779149027, 60.83088182535696, 61.756475568630265, 62.969075123771255, 63.95924517471479, 64.71981057616418, 65.55212743058047, 66.40596972088684, 67.29568773767669, 68.25715720743345, 69.49845734376122, 70.41687594173783, 71.32811939441773, 72.17478653942743, 73.32280978689818, 74.37038100021526, 75.37490134175216, 76.29331993972878, 77.34089115304586, 78.23778431513239, 79.19207863959245, 80.13919781875583, 81.11501757910598, 82.13388821123627, 82.88010332209227, 83.87027337303581, 84.84609313338595, 85.81473774843941, 86.85513381645978, 87.98163162804047, 89.35925952500538, 90.65796082370669, 91.85621008825429, 92.80332926741767, 93.92982707899836, 95.11372605295257, 95.88146659969864, 96.97926383009255, 98.24926454760708, 99.23943459855063, 100.0], "rescued": [0, 88, 155, 240, 295, 368, 427, 474, 536, 588, 626, 672, 716, 765, 811, 839, 892, 918, 968, 1013, 1058, 1091, 1129, 1150, 1190, 1223, 1251, 1260, 1266, 1273, 1278, 1285, 1296, 1303, 1313, 1319, 1325, 1334, 1338, 1346, 1355, 1362, 1367, 1369, 1374, 1383, 1397, 1406, 1416, 1425, 1431, 1444, 1453, 1466, 1475, 1482, 1489, 1500, 1504, 1508, 1518, 1532, 1538, 1548, 1560, 1567, 1576, 1590, 1607, 1624, 1632, 1650, 1658, 1674, 1694, 1710, 1724, 1745, 1756, 1770, 1779, 1798, 1816, 1827, 1850, 1863, 1876, 1898, 1913, 1933, 1955, 1988, 2020, 2042, 2064, 2094, 2114, 2143, 2178, 2205, 2225], "harmed": [0, 10, 19, 26, 37, 45, 57, 63, 74, 83, 89, 101, 115, 126, 133, 140, 155, 160, 166, 176, 187, 194, 200, 207, 219, 230, 239, 241, 243, 243, 245, 246, 247, 249, 250, 254, 257, 257, 260, 261, 263, 267, 267, 270, 272, 272, 280, 284, 288, 295, 299, 303, 305, 309, 316, 322, 327, 335, 339, 343, 346, 352, 358, 367, 376, 378, 387, 393, 396, 403, 414, 422, 425, 432, 435, 440, 448, 454, 468, 474, 479, 485, 492, 505, 512, 521, 533, 547, 559, 566, 579, 592, 606, 624, 635, 646, 660, 679, 702, 719, 729], "exact_score": [80.00287005811867, 80.52665566477721, 81.02174069024898, 81.4594245533472, 81.80383152758843, 82.24151539068667, 82.56439692903783, 82.86575303149888, 83.20298486044344, 83.61196814235488, 83.8774485183325, 84.09987802252996, 84.31513238143073, 84.55908732151826, 84.88196885986942, 85.0900480734735, 85.2766018511875, 85.49185621008826, 85.75733658606588, 86.00129152615341, 86.20937073975749, 86.3815742268781, 86.63270431226233, 86.7546817823061, 86.91253497883332, 87.10626390184402, 87.19954079070101, 87.32151826074478, 87.33586855133817, 87.38609456841502, 87.41479514960179, 87.45784602138194, 87.51524718375548, 87.55829805553563, 87.63004950850255, 87.63004950850255, 87.665925234986, 87.70897610676617, 87.73767668795294, 87.78072755973308, 87.83812872210662, 87.85247901270002, 87.89552988448015, 87.90270502977687, 87.90988017507355, 87.97445648274378, 87.98880677333716, 88.0462079357107, 88.08208366219417, 88.09643395278755, 88.12513453397432, 88.17536055105116, 88.21841142283131, 88.30451316639162, 88.31168831168831, 88.31168831168831, 88.34038889287508, 88.36908947406185, 88.36191432876515, 88.37626461935855, 88.412140345842, 88.46954150821554, 88.46236636291886, 88.48389179880894, 88.49824208940231, 88.54129296118248, 88.53411781588576, 88.67762072181962, 88.76372246537993, 88.74219702948984, 88.79959819186338, 88.83547391834685, 88.91440051661046, 88.99332711487408, 89.09377914902777, 89.13683002080792, 89.24445720025831, 89.23010690966493, 89.28033292674176, 89.31620865322523, 89.40231039678554, 89.49558728564254, 89.48841214034584, 89.58168902920285, 89.63191504627969, 89.63191504627969, 89.68214106335654, 89.72519193513669, 89.811293678697, 89.86151969577384, 89.94762143933414, 90.02654803759776, 90.14852550764154, 90.1772260888283, 90.26332783238861, 90.37095501183899, 90.44988161010261, 90.50728277247615, 90.55750878955298, 90.65078567840999, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.57718500253672, "threshold_area100": 87.57501614407691, "threshold_area25": 84.20248260027266}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1087, "harmed": 383}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2225, "harmed": 729}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.385457992553711, "operator": "ge"}, {"budget": 0.02, "threshold": 3.513397693634033, "operator": "ge"}, {"budget": 0.03, "threshold": 3.0718793869018555, "operator": "ge"}, {"budget": 0.04, "threshold": 2.7520837783813477, "operator": "ge"}, {"budget": 0.05, "threshold": 2.440133810043335, "operator": "ge"}, {"budget": 0.06, "threshold": 2.2047533988952637, "operator": "ge"}, {"budget": 0.07, "threshold": 2.0206503868103027, "operator": "ge"}, {"budget": 0.08, "threshold": 1.834553599357605, "operator": "ge"}, {"budget": 0.09, "threshold": 1.6988266706466675, "operator": "ge"}, {"budget": 0.1, "threshold": 1.5725022554397583, "operator": "ge"}, {"budget": 0.11, "threshold": 1.4321672916412354, "operator": "ge"}, {"budget": 0.12, "threshold": 1.3117501735687256, "operator": "ge"}, {"budget": 0.13, "threshold": 1.2038137912750244, "operator": "ge"}, {"budget": 0.14, "threshold": 1.092176079750061, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9864388704299927, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8879580497741699, "operator": "ge"}, {"budget": 0.17, "threshold": 0.7933460474014282, "operator": "ge"}, {"budget": 0.18, "threshold": 0.6884670257568359, "operator": "ge"}, {"budget": 0.19, "threshold": 0.6152529716491699, "operator": "ge"}, {"budget": 0.2, "threshold": 0.5272305011749268, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4325484037399292, "operator": "ge"}, {"budget": 0.22, "threshold": 0.3550809621810913, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2627914547920227, "operator": "ge"}, {"budget": 0.24, "threshold": 0.1685611605644226, "operator": "ge"}, {"budget": 0.25, "threshold": 0.0812591016292572, "operator": "ge"}, {"budget": 0.26, "threshold": 0.0036629438400268555, "operator": "ge"}, {"budget": 0.27, "threshold": -0.09271445870399475, "operator": "ge"}, {"budget": 0.28, "threshold": -0.17075833678245544, "operator": "ge"}, {"budget": 0.29, "threshold": -0.26941144466400146, "operator": "ge"}, {"budget": 0.3, "threshold": -0.3489888906478882, "operator": "ge"}, {"budget": 0.31, "threshold": -0.4393448829650879, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5364158749580383, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6435440182685852, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7364423871040344, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8259167671203613, "operator": "ge"}, {"budget": 0.36, "threshold": -0.9394395351409912, "operator": "ge"}, {"budget": 0.37, "threshold": -1.038757562637329, "operator": "ge"}, {"budget": 0.38, "threshold": -1.1552891731262207, "operator": "ge"}, {"budget": 0.39, "threshold": -1.2627745866775513, "operator": "ge"}, {"budget": 0.4, "threshold": -1.3531602621078491, "operator": "ge"}, {"budget": 0.41, "threshold": -1.4453637599945068, "operator": "ge"}, {"budget": 0.42, "threshold": -1.554504156112671, "operator": "ge"}, {"budget": 0.43, "threshold": -1.6778020858764648, "operator": "ge"}, {"budget": 0.44, "threshold": -1.7762186527252197, "operator": "ge"}, {"budget": 0.45, "threshold": -1.8642747402191162, "operator": "ge"}, {"budget": 0.46, "threshold": -1.9597725868225098, "operator": "ge"}, {"budget": 0.47, "threshold": -2.0493268966674805, "operator": "ge"}, {"budget": 0.48, "threshold": -2.150824785232544, "operator": "ge"}, {"budget": 0.49, "threshold": -2.249236583709717, "operator": "ge"}, {"budget": 0.5, "threshold": -2.3641295433044434, "operator": "ge"}, {"budget": 0.51, "threshold": -2.4747767448425293, "operator": "ge"}, {"budget": 0.52, "threshold": -2.5774178504943848, "operator": "ge"}, {"budget": 0.53, "threshold": -2.6817545890808105, "operator": "ge"}, {"budget": 0.54, "threshold": -2.806298017501831, "operator": "ge"}, {"budget": 0.55, "threshold": -2.9267055988311768, "operator": "ge"}, {"budget": 0.56, "threshold": -3.0062994956970215, "operator": "ge"}, {"budget": 0.57, "threshold": -3.109555244445801, "operator": "ge"}, {"budget": 0.58, "threshold": -3.2469022274017334, "operator": "ge"}, {"budget": 0.59, "threshold": -3.358287811279297, "operator": "ge"}, {"budget": 0.6, "threshold": -3.4514455795288086, "operator": "ge"}, {"budget": 0.61, "threshold": -3.5592753887176514, "operator": "ge"}, {"budget": 0.62, "threshold": -3.667661428451538, "operator": "ge"}, {"budget": 0.63, "threshold": -3.802943706512451, "operator": "ge"}, {"budget": 0.64, "threshold": -3.9104509353637695, "operator": "ge"}, {"budget": 0.65, "threshold": -4.006215572357178, "operator": "ge"}, {"budget": 0.66, "threshold": -4.113356113433838, "operator": "ge"}, {"budget": 0.67, "threshold": -4.218702793121338, "operator": "ge"}, {"budget": 0.68, "threshold": -4.32921838760376, "operator": "ge"}, {"budget": 0.69, "threshold": -4.429149627685547, "operator": "ge"}, {"budget": 0.7, "threshold": -4.54303503036499, "operator": "ge"}, {"budget": 0.71, "threshold": -4.653964996337891, "operator": "ge"}, {"budget": 0.72, "threshold": -4.7778730392456055, "operator": "ge"}, {"budget": 0.73, "threshold": -4.869566917419434, "operator": "ge"}, {"budget": 0.74, "threshold": -4.9643354415893555, "operator": "ge"}, {"budget": 0.75, "threshold": -5.069992542266846, "operator": "ge"}, {"budget": 0.76, "threshold": -5.18526554107666, "operator": "ge"}, {"budget": 0.77, "threshold": -5.326603412628174, "operator": "ge"}, {"budget": 0.78, "threshold": -5.470766067504883, "operator": "ge"}, {"budget": 0.79, "threshold": -5.603207588195801, "operator": "ge"}, {"budget": 0.8, "threshold": -5.730011463165283, "operator": "ge"}, {"budget": 0.81, "threshold": -5.84907341003418, "operator": "ge"}, {"budget": 0.82, "threshold": -5.957300186157227, "operator": "ge"}, {"budget": 0.83, "threshold": -6.111337184906006, "operator": "ge"}, {"budget": 0.84, "threshold": -6.236602783203125, "operator": "ge"}, {"budget": 0.85, "threshold": -6.392948627471924, "operator": "ge"}, {"budget": 0.86, "threshold": -6.525938510894775, "operator": "ge"}, {"budget": 0.87, "threshold": -6.6980767250061035, "operator": "ge"}, {"budget": 0.88, "threshold": -6.848849773406982, "operator": "ge"}, {"budget": 0.89, "threshold": -6.989867210388184, "operator": "ge"}, {"budget": 0.9, "threshold": -7.160377502441406, "operator": "ge"}, {"budget": 0.91, "threshold": -7.3623881340026855, "operator": "ge"}, {"budget": 0.92, "threshold": -7.544134140014648, "operator": "ge"}, {"budget": 0.93, "threshold": -7.746288299560547, "operator": "ge"}, {"budget": 0.94, "threshold": -7.939512729644775, "operator": "ge"}, {"budget": 0.95, "threshold": -8.1593599319458, "operator": "ge"}, {"budget": 0.96, "threshold": -8.462350845336914, "operator": "ge"}, {"budget": 0.97, "threshold": -8.782501220703125, "operator": "ge"}, {"budget": 0.98, "threshold": -9.254377365112305, "operator": "ge"}, {"budget": 0.99, "threshold": -9.966873168945312, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.1059868232598, 80.53566313377254, 80.86508163849899, 81.28043540532798, 81.56688627900314, 81.78172443425952, 82.12546548266971, 82.49785161844743, 82.69836723002004, 82.8845602979089, 83.21397880263535, 83.44313950157547, 83.71526783156689, 84.00171870524204, 84.20223431681467, 84.5316528215411, 84.83242623890003, 85.06158693784016, 85.34803781151533, 85.46261816098539, 85.72042394729303, 85.94958464623316, 86.15010025780579, 86.3506158693784, 86.43655113148094, 86.50816384989973, 86.68003437410484, 86.78029217989113, 86.92351761672873, 86.99513033514752, 87.12403322830134, 87.22429103408766, 87.35319392724148, 87.46777427671154, 87.49641936407906, 87.61099971354912, 87.72558006301918, 87.86880549985678, 87.94041821827557, 88.01203093669436, 88.18390146089945, 88.35577198510455, 88.4417072472071, 88.5419650529934, 88.58493268404469, 88.75680320824978, 88.87138355771984, 88.92867373245488, 88.97164136350615, 89.11486680034373, 89.1291893440275, 89.18647951876252, 89.34402749928387, 89.3297049556001, 89.47293039243769, 89.44428530507018, 89.51589802348897, 89.60183328559151, 89.55886565454024, 89.51589802348897, 89.5445431108565, 89.58751074190776, 89.60183328559151, 89.58751074190776, 89.65912346032655, 89.61615582927529, 89.68776854769406, 89.68776854769406, 89.65912346032655, 89.71641363506157, 89.80234889716414, 89.75938126611285, 89.70209109137782, 89.74505872242909, 89.74505872242909, 89.78802635348038, 89.75938126611285, 89.78802635348038, 89.78802635348038, 89.80234889716414, 89.8453165282154, 89.87396161558291, 89.90260670295044, 89.85963907189917, 89.87396161558291, 89.85963907189917, 89.81667144084788, 89.83099398453164, 89.81667144084788, 89.80234889716414, 89.80234889716414, 89.77370380979661, 89.75938126611285, 89.80234889716414, 89.83099398453164, 89.85963907189917, 89.88828415926669, 89.90260670295044, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 65, 95, 127, 151, 172, 200, 229, 249, 267, 291, 312, 333, 357, 378, 405, 428, 451, 475, 491, 516, 532, 550, 567, 580, 590, 610, 625, 641, 654, 666, 678, 690, 704, 714, 727, 743, 756, 765, 778, 793, 808, 817, 828, 836, 850, 863, 870, 877, 889, 895, 902, 915, 919, 933, 937, 944, 951, 955, 959, 964, 971, 975, 978, 985, 990, 998, 1003, 1006, 1012, 1019, 1023, 1024, 1029, 1031, 1036, 1037, 1042, 1044, 1047, 1051, 1055, 1060, 1062, 1065, 1066, 1067, 1069, 1070, 1071, 1071, 1072, 1072, 1075, 1077, 1080, 1082, 1085, 1086, 1087], "harmed": [0, 11, 15, 22, 25, 29, 35, 39, 42, 48, 53, 54, 59, 61, 65, 72, 76, 78, 85, 89, 97, 104, 104, 108, 111, 118, 123, 131, 139, 145, 153, 156, 161, 164, 170, 178, 183, 191, 194, 198, 206, 209, 212, 215, 219, 224, 226, 231, 234, 238, 240, 245, 248, 250, 255, 259, 265, 267, 268, 275, 282, 285, 289, 292, 296, 298, 306, 309, 314, 319, 321, 322, 329, 334, 336, 338, 340, 343, 346, 348, 350, 351, 353, 356, 361, 363, 365, 369, 370, 372, 374, 374, 377, 378, 378, 378, 379, 379, 381, 382, 383], "exact_score": [79.81953594958465, 80.1059868232598, 80.53566313377254, 80.86508163849899, 81.28043540532798, 81.56688627900314, 81.78172443425952, 82.12546548266971, 82.49785161844743, 82.69836723002004, 82.8845602979089, 83.21397880263535, 83.44313950157547, 83.71526783156689, 84.00171870524204, 84.20223431681467, 84.5316528215411, 84.83242623890003, 85.06158693784016, 85.34803781151533, 85.46261816098539, 85.72042394729303, 85.94958464623316, 86.15010025780579, 86.3506158693784, 86.43655113148094, 86.50816384989973, 86.68003437410484, 86.78029217989113, 86.92351761672873, 86.99513033514752, 87.12403322830134, 87.22429103408766, 87.35319392724148, 87.46777427671154, 87.49641936407906, 87.61099971354912, 87.72558006301918, 87.86880549985678, 87.94041821827557, 88.01203093669436, 88.18390146089945, 88.35577198510455, 88.4417072472071, 88.5419650529934, 88.58493268404469, 88.75680320824978, 88.87138355771984, 88.92867373245488, 88.97164136350615, 89.11486680034373, 89.1291893440275, 89.18647951876252, 89.34402749928387, 89.3297049556001, 89.47293039243769, 89.44428530507018, 89.51589802348897, 89.60183328559151, 89.55886565454024, 89.51589802348897, 89.5445431108565, 89.58751074190776, 89.60183328559151, 89.58751074190776, 89.65912346032655, 89.61615582927529, 89.68776854769406, 89.68776854769406, 89.65912346032655, 89.71641363506157, 89.80234889716414, 89.75938126611285, 89.70209109137782, 89.74505872242909, 89.74505872242909, 89.78802635348038, 89.75938126611285, 89.78802635348038, 89.78802635348038, 89.80234889716414, 89.8453165282154, 89.87396161558291, 89.90260670295044, 89.85963907189917, 89.87396161558291, 89.85963907189917, 89.81667144084788, 89.83099398453164, 89.81667144084788, 89.80234889716414, 89.80234889716414, 89.77370380979661, 89.75938126611285, 89.80234889716414, 89.83099398453164, 89.85963907189917, 89.88828415926669, 89.90260670295044, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.67286850723318, "threshold_area100": 87.67215697507879, "threshold_area25": 83.44886851904899}, "test": {"score": [80.00287005811867, 80.53383081007391, 81.11501757910598, 81.45224940805052, 81.81100667288513, 82.24151539068667, 82.58592236492788, 82.89445361268565, 83.21016000574012, 83.45411494582765, 83.76264619358543, 84.08552773193657, 84.41558441558442, 84.60931333859511, 84.87479371457273, 85.06852263758341, 85.26225156059411, 85.49903135538494, 85.67840998780225, 85.80038745784603, 86.1017435603071, 86.30982277391116, 86.53942742340533, 86.74750663700941, 86.99146157709694, 87.1134390471407, 87.25694195307454, 87.50089689316209, 87.61569921790917, 87.72332639735954, 87.91705532037024, 88.0533830810074, 88.18971084164454, 88.28298773050155, 88.43366578173209, 88.54846810647916, 88.75654732008323, 88.94310109779724, 88.97897682428069, 89.065078567841, 89.13683002080792, 89.20858147377486, 89.33055894381862, 89.50993757623593, 89.60321446509292, 89.6175647556863, 89.63191504627969, 89.70366649924661, 89.75389251632346, 89.8184688239937, 89.84716940518047, 89.87586998636723, 89.94044629403746, 89.97632202052091, 90.04089832819116, 90.06959890937792, 90.05524861878453, 90.12700007175145, 90.1700509435316, 90.23462725120184, 90.25615268709191, 90.277678122982, 90.32790414005885, 90.40683073832245, 90.42835617421254, 90.44270646480591, 90.50010762717945, 90.60773480662984, 90.62926024251992, 90.67231111430006, 90.70101169548683, 90.69383655019014, 90.744062567267, 90.7297122766736, 90.75123771256368, 90.73688742197031, 90.76558800315708, 90.744062567267, 90.744062567267, 90.76558800315708, 90.76558800315708, 90.76558800315708, 90.76558800315708, 90.75123771256368, 90.76558800315708, 90.75841285786038, 90.76558800315708, 90.744062567267, 90.71536198608023, 90.73688742197031, 90.75123771256368, 90.76558800315708, 90.80146372964053, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.78711343904715, 90.77276314845376, 90.75841285786038, 90.73688742197031, 90.73688742197031], "calls": [0.0, 1.0188706321303005, 2.1453684437109852, 3.11401305876444, 3.9822056396642034, 5.201980340101887, 6.306952715792495, 7.282772476142642, 8.330343689459712, 9.255937432733013, 10.253282628973237, 11.379780440553922, 12.39147592738753, 13.245318217693908, 14.2713639951209, 15.304584917844585, 16.273229532898043, 17.270574729138268, 18.296620506565258, 19.12893736098156, 20.262610317858936, 21.310181531176006, 22.142498385592308, 23.268996197172992, 24.381143718160292, 25.306737461433592, 26.383009255937434, 27.423405323957812, 28.126569563033648, 29.145440195163953, 30.006457630767024, 30.996627681710553, 31.92939657028055, 33.07024467245462, 34.12499103106838, 35.036234483748295, 36.36363636363637, 37.46860873932697, 38.587931405610966, 39.71442921719165, 40.72612470402526, 41.72346990026548, 42.73516538709909, 43.91906436105331, 44.90205926670015, 45.633924086962764, 46.65279471909306, 47.57121331706967, 48.51833249623304, 49.25019731649566, 50.43409629044988, 51.330989452536414, 52.19918203343617, 53.25392839204994, 54.445002511300856, 55.514099160508, 56.324890579034225, 57.21460859582407, 58.29805553562459, 59.3169261677549, 60.235344765731504, 61.26856568845519, 62.3735380641458, 63.442634713352945, 64.39692903781301, 65.26512161871277, 66.24094137906292, 67.32438831886346, 68.26433235273014, 69.16122551481666, 70.11551983927674, 70.94066154839635, 71.91648130874651, 72.72727272727273, 73.43043696634857, 74.29862954724832, 75.36772619645548, 76.71665351223362, 78.00817966563824, 79.04857573365861, 80.11049723756905, 80.97868981846882, 81.88993327114873, 83.2173351510368, 84.0711774413432, 85.24072612470403, 86.18067015857072, 87.41479514960177, 88.33321374757838, 89.2588074908517, 90.28485326827868, 91.2965487551123, 92.24366793427566, 93.10468536987874, 93.9728779507785, 94.84107053167826, 96.03931979622587, 97.00796441127932, 98.06988591518979, 99.0600559661333, 100.0], "rescued": [0, 83, 170, 228, 282, 350, 408, 459, 511, 551, 600, 656, 710, 743, 789, 829, 868, 908, 942, 969, 1021, 1059, 1097, 1139, 1183, 1213, 1250, 1290, 1312, 1343, 1374, 1403, 1432, 1458, 1487, 1509, 1549, 1585, 1603, 1624, 1645, 1663, 1690, 1721, 1740, 1754, 1767, 1784, 1796, 1811, 1827, 1840, 1857, 1872, 1887, 1899, 1905, 1920, 1935, 1951, 1961, 1969, 1982, 1995, 2005, 2011, 2025, 2045, 2053, 2066, 2074, 2078, 2088, 2092, 2098, 2102, 2114, 2117, 2126, 2134, 2137, 2141, 2146, 2156, 2161, 2167, 2171, 2177, 2178, 2184, 2189, 2192, 2198, 2203, 2205, 2208, 2215, 2219, 2221, 2222, 2225], "harmed": [0, 9, 15, 26, 30, 38, 48, 56, 64, 70, 76, 87, 95, 101, 110, 123, 135, 142, 151, 161, 171, 180, 186, 199, 209, 222, 239, 245, 251, 267, 271, 281, 291, 304, 312, 318, 329, 339, 352, 361, 372, 380, 390, 396, 402, 414, 425, 432, 437, 443, 455, 464, 472, 482, 488, 496, 504, 509, 518, 525, 532, 537, 543, 545, 552, 556, 562, 567, 572, 579, 583, 588, 591, 597, 600, 606, 614, 620, 629, 634, 637, 641, 646, 658, 661, 668, 671, 680, 685, 688, 691, 692, 693, 703, 704, 706, 712, 718, 722, 726, 729], "exact_score": [80.00287005811867, 80.51230537418384, 81.03609098084237, 81.42354882686375, 81.82535696347851, 82.13388821123628, 82.46394489488412, 82.80835186912535, 83.1384085527732, 83.38953863815742, 83.64784386883835, 83.95637511659612, 84.27925665494726, 84.55191217622158, 84.76716653512234, 84.9895960393198, 85.21202554351726, 85.46315562890149, 85.62818397072542, 85.77886202195594, 86.03716725263688, 86.24524646624094, 86.50355169692186, 86.71163091052594, 86.86230896175648, 87.04886273947048, 87.19954079070101, 87.41479514960179, 87.5941737820191, 87.74485183324963, 87.90988017507355, 88.0533830810074, 88.18253569634786, 88.29016287579825, 88.39779005524862, 88.54129296118248, 88.71349644830309, 88.90722537131377, 88.99332711487408, 88.99332711487408, 89.09377914902777, 89.158355456698, 89.2659826361484, 89.3377340891153, 89.50993757623593, 89.61038961038962, 89.61038961038962, 89.68931620865322, 89.72519193513669, 89.78259309751023, 89.83999425988375, 89.84716940518047, 89.91892085814737, 89.98349716581761, 90.02654803759776, 90.06242376408122, 90.06242376408122, 90.11264978115807, 90.14852550764154, 90.22027696060846, 90.24897754179523, 90.28485326827868, 90.30637870416876, 90.38530530243239, 90.42835617421254, 90.44988161010261, 90.47858219128938, 90.56468393484968, 90.62208509722322, 90.66513596900337, 90.69383655019014, 90.70818684078354, 90.73688742197031, 90.7297122766736, 90.7297122766736, 90.76558800315708, 90.77276314845376, 90.72253713137691, 90.744062567267, 90.76558800315708, 90.76558800315708, 90.76558800315708, 90.75123771256368, 90.75841285786038, 90.77276314845376, 90.744062567267, 90.75841285786038, 90.76558800315708, 90.71536198608023, 90.7297122766736, 90.76558800315708, 90.76558800315708, 90.78711343904715, 90.76558800315708, 90.77276314845376, 90.78711343904715, 90.78711343904715, 90.77276314845376, 90.75841285786038, 90.73688742197031, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.48793319154618, "threshold_area100": 88.51596469828515, "threshold_area25": 84.14651646695846}, "selected_budget": 83, "selected": {"val": {"score": 89.90260670295044, "calls": 82.99914064737898, "rescued": 1060, "harmed": 356}, "test": {"score": 90.75123771256368, "calls": 83.2173351510368, "rescued": 2156, "harmed": 658}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.376762866973877, "operator": "ge"}, {"budget": 0.02, "threshold": 2.8723092079162598, "operator": "ge"}, {"budget": 0.03, "threshold": 2.4528024196624756, "operator": "ge"}, {"budget": 0.04, "threshold": 2.1103687286376953, "operator": "ge"}, {"budget": 0.05, "threshold": 1.9094908237457275, "operator": "ge"}, {"budget": 0.06, "threshold": 1.7594926357269287, "operator": "ge"}, {"budget": 0.07, "threshold": 1.5935451984405518, "operator": "ge"}, {"budget": 0.08, "threshold": 1.4487555027008057, "operator": "ge"}, {"budget": 0.09, "threshold": 1.2852599620819092, "operator": "ge"}, {"budget": 0.1, "threshold": 1.1697131395339966, "operator": "ge"}, {"budget": 0.11, "threshold": 1.0359840393066406, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9277796149253845, "operator": "ge"}, {"budget": 0.13, "threshold": 0.854708194732666, "operator": "ge"}, {"budget": 0.14, "threshold": 0.7766392827033997, "operator": "ge"}, {"budget": 0.15, "threshold": 0.7011957764625549, "operator": "ge"}, {"budget": 0.16, "threshold": 0.6165379881858826, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5544878244400024, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5036711096763611, "operator": "ge"}, {"budget": 0.19, "threshold": 0.43345847725868225, "operator": "ge"}, {"budget": 0.2, "threshold": 0.3753064274787903, "operator": "ge"}, {"budget": 0.21, "threshold": 0.303308367729187, "operator": "ge"}, {"budget": 0.22, "threshold": 0.2508629560470581, "operator": "ge"}, {"budget": 0.23, "threshold": 0.16616559028625488, "operator": "ge"}, {"budget": 0.24, "threshold": 0.11661791801452637, "operator": "ge"}, {"budget": 0.25, "threshold": 0.062330394983291626, "operator": "ge"}, {"budget": 0.26, "threshold": 0.0023070573806762695, "operator": "ge"}, {"budget": 0.27, "threshold": -0.06496551632881165, "operator": "ge"}, {"budget": 0.28, "threshold": -0.12325489521026611, "operator": "ge"}, {"budget": 0.29, "threshold": -0.18461665511131287, "operator": "ge"}, {"budget": 0.3, "threshold": -0.2645605504512787, "operator": "ge"}, {"budget": 0.31, "threshold": -0.33245849609375, "operator": "ge"}, {"budget": 0.32, "threshold": -0.413235068321228, "operator": "ge"}, {"budget": 0.33, "threshold": -0.49723267555236816, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5822737812995911, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6524214744567871, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7228930592536926, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8319012522697449, "operator": "ge"}, {"budget": 0.38, "threshold": -0.9317026138305664, "operator": "ge"}, {"budget": 0.39, "threshold": -1.027661681175232, "operator": "ge"}, {"budget": 0.4, "threshold": -1.1286826133728027, "operator": "ge"}, {"budget": 0.41, "threshold": -1.2155873775482178, "operator": "ge"}, {"budget": 0.42, "threshold": -1.3127888441085815, "operator": "ge"}, {"budget": 0.43, "threshold": -1.4356212615966797, "operator": "ge"}, {"budget": 0.44, "threshold": -1.5426914691925049, "operator": "ge"}, {"budget": 0.45, "threshold": -1.6356738805770874, "operator": "ge"}, {"budget": 0.46, "threshold": -1.720068335533142, "operator": "ge"}, {"budget": 0.47, "threshold": -1.823311686515808, "operator": "ge"}, {"budget": 0.48, "threshold": -1.9492806196212769, "operator": "ge"}, {"budget": 0.49, "threshold": -2.048304557800293, "operator": "ge"}, {"budget": 0.5, "threshold": -2.1332082748413086, "operator": "ge"}, {"budget": 0.51, "threshold": -2.2353005409240723, "operator": "ge"}, {"budget": 0.52, "threshold": -2.348883867263794, "operator": "ge"}, {"budget": 0.53, "threshold": -2.4676620960235596, "operator": "ge"}, {"budget": 0.54, "threshold": -2.596221923828125, "operator": "ge"}, {"budget": 0.55, "threshold": -2.716765880584717, "operator": "ge"}, {"budget": 0.56, "threshold": -2.816542148590088, "operator": "ge"}, {"budget": 0.57, "threshold": -2.9222567081451416, "operator": "ge"}, {"budget": 0.58, "threshold": -3.0306591987609863, "operator": "ge"}, {"budget": 0.59, "threshold": -3.1502718925476074, "operator": "ge"}, {"budget": 0.6, "threshold": -3.267650842666626, "operator": "ge"}, {"budget": 0.61, "threshold": -3.3561902046203613, "operator": "ge"}, {"budget": 0.62, "threshold": -3.4731640815734863, "operator": "ge"}, {"budget": 0.63, "threshold": -3.612673759460449, "operator": "ge"}, {"budget": 0.64, "threshold": -3.722792148590088, "operator": "ge"}, {"budget": 0.65, "threshold": -3.8393352031707764, "operator": "ge"}, {"budget": 0.66, "threshold": -3.940450668334961, "operator": "ge"}, {"budget": 0.67, "threshold": -4.031810760498047, "operator": "ge"}, {"budget": 0.68, "threshold": -4.159362316131592, "operator": "ge"}, {"budget": 0.69, "threshold": -4.274964809417725, "operator": "ge"}, {"budget": 0.7, "threshold": -4.387104511260986, "operator": "ge"}, {"budget": 0.71, "threshold": -4.486035346984863, "operator": "ge"}, {"budget": 0.72, "threshold": -4.609958171844482, "operator": "ge"}, {"budget": 0.73, "threshold": -4.715582847595215, "operator": "ge"}, {"budget": 0.74, "threshold": -4.827361583709717, "operator": "ge"}, {"budget": 0.75, "threshold": -4.935693740844727, "operator": "ge"}, {"budget": 0.76, "threshold": -5.050825119018555, "operator": "ge"}, {"budget": 0.77, "threshold": -5.16292667388916, "operator": "ge"}, {"budget": 0.78, "threshold": -5.274164199829102, "operator": "ge"}, {"budget": 0.79, "threshold": -5.4220147132873535, "operator": "ge"}, {"budget": 0.8, "threshold": -5.5529465675354, "operator": "ge"}, {"budget": 0.81, "threshold": -5.685887336730957, "operator": "ge"}, {"budget": 0.82, "threshold": -5.796698570251465, "operator": "ge"}, {"budget": 0.83, "threshold": -5.935623645782471, "operator": "ge"}, {"budget": 0.84, "threshold": -6.089098930358887, "operator": "ge"}, {"budget": 0.85, "threshold": -6.232483386993408, "operator": "ge"}, {"budget": 0.86, "threshold": -6.377546310424805, "operator": "ge"}, {"budget": 0.87, "threshold": -6.518847465515137, "operator": "ge"}, {"budget": 0.88, "threshold": -6.716582298278809, "operator": "ge"}, {"budget": 0.89, "threshold": -6.869524002075195, "operator": "ge"}, {"budget": 0.9, "threshold": -7.013357162475586, "operator": "ge"}, {"budget": 0.91, "threshold": -7.2027201652526855, "operator": "ge"}, {"budget": 0.92, "threshold": -7.378375053405762, "operator": "ge"}, {"budget": 0.93, "threshold": -7.559257507324219, "operator": "ge"}, {"budget": 0.94, "threshold": -7.769176959991455, "operator": "ge"}, {"budget": 0.95, "threshold": -8.013267517089844, "operator": "ge"}, {"budget": 0.96, "threshold": -8.335236549377441, "operator": "ge"}, {"budget": 0.97, "threshold": -8.657279968261719, "operator": "ge"}, {"budget": 0.98, "threshold": -9.104049682617188, "operator": "ge"}, {"budget": 0.99, "threshold": -9.831510543823242, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.14895445431108, 80.44972787167, 80.77914637639644, 81.10856488112287, 81.46662847321684, 81.79604697794328, 82.0681753079347, 82.2686909195073, 82.58378688054998, 82.87023775422514, 83.19965625895158, 83.38584932684044, 83.65797765683185, 83.93010598682325, 84.15926668576338, 84.44571755943855, 84.66055571469494, 84.94700658837009, 85.23345746204527, 85.4912632483529, 85.72042394729303, 86.02119736465195, 86.22171297622458, 86.37926095674591, 86.39358350042967, 86.50816384989973, 86.65138928673733, 86.76596963620739, 86.93784016041248, 87.03809796619879, 87.1383557719851, 87.2099684904039, 87.29590375250645, 87.39616155829275, 87.45345173302778, 87.65396734460039, 87.71125751933543, 87.86880549985678, 87.95474076195931, 88.05499856774563, 88.22686909195072, 88.31280435405327, 88.4417072472071, 88.59925522772843, 88.79977083930105, 88.84273847035233, 88.82841592666857, 88.94299627613864, 89.07189916929245, 89.11486680034373, 89.1291893440275, 89.20080206244629, 89.28673732454884, 89.35835004296763, 89.38699513033515, 89.53022056717272, 89.55886565454024, 89.60183328559151, 89.573188198224, 89.48725293612145, 89.51589802348897, 89.50157547980521, 89.573188198224, 89.63047837295903, 89.63047837295903, 89.6734460040103, 89.63047837295903, 89.70209109137782, 89.78802635348038, 89.80234889716414, 89.78802635348038, 89.81667144084788, 89.83099398453164, 89.80234889716414, 89.78802635348038, 89.81667144084788, 89.77370380979661, 89.81667144084788, 89.80234889716414, 89.81667144084788, 89.88828415926669, 89.90260670295044, 89.8453165282154, 89.9169292466342, 89.83099398453164, 89.87396161558291, 89.83099398453164, 89.81667144084788, 89.80234889716414, 89.78802635348038, 89.78802635348038, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.81667144084788, 89.8453165282154, 89.87396161558291, 89.88828415926669, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 30, 60, 88, 115, 146, 173, 201, 220, 244, 264, 288, 307, 331, 352, 374, 397, 420, 445, 471, 493, 511, 535, 553, 569, 577, 590, 609, 626, 642, 653, 668, 679, 690, 702, 712, 729, 740, 758, 770, 781, 797, 806, 820, 834, 848, 855, 862, 875, 886, 891, 898, 906, 913, 923, 931, 942, 950, 955, 957, 960, 966, 968, 976, 982, 985, 991, 998, 1004, 1013, 1016, 1018, 1022, 1030, 1032, 1033, 1036, 1037, 1041, 1045, 1047, 1053, 1055, 1057, 1063, 1064, 1067, 1067, 1068, 1069, 1070, 1071, 1071, 1073, 1075, 1076, 1079, 1081, 1084, 1086, 1087], "harmed": [0, 7, 16, 21, 25, 31, 35, 44, 49, 51, 51, 52, 58, 63, 65, 71, 74, 82, 87, 93, 97, 99, 102, 106, 111, 118, 123, 132, 141, 145, 149, 157, 163, 168, 173, 179, 182, 189, 196, 202, 206, 210, 213, 218, 221, 221, 225, 233, 238, 240, 242, 248, 251, 252, 257, 263, 264, 270, 272, 276, 285, 289, 292, 295, 297, 300, 303, 313, 314, 317, 319, 322, 324, 331, 335, 337, 338, 342, 343, 348, 349, 350, 351, 357, 358, 365, 365, 368, 370, 372, 374, 375, 376, 378, 378, 378, 379, 379, 381, 382, 383], "exact_score": [79.81953594958465, 80.14895445431108, 80.44972787167, 80.77914637639644, 81.10856488112287, 81.46662847321684, 81.79604697794328, 82.0681753079347, 82.2686909195073, 82.58378688054998, 82.87023775422514, 83.19965625895158, 83.38584932684044, 83.65797765683185, 83.93010598682325, 84.15926668576338, 84.44571755943855, 84.66055571469494, 84.94700658837009, 85.23345746204527, 85.4912632483529, 85.72042394729303, 86.02119736465195, 86.22171297622458, 86.37926095674591, 86.39358350042967, 86.50816384989973, 86.65138928673733, 86.76596963620739, 86.93784016041248, 87.03809796619879, 87.1383557719851, 87.2099684904039, 87.29590375250645, 87.39616155829275, 87.45345173302778, 87.65396734460039, 87.71125751933543, 87.86880549985678, 87.95474076195931, 88.05499856774563, 88.22686909195072, 88.31280435405327, 88.4417072472071, 88.59925522772843, 88.79977083930105, 88.84273847035233, 88.82841592666857, 88.94299627613864, 89.07189916929245, 89.11486680034373, 89.1291893440275, 89.20080206244629, 89.28673732454884, 89.35835004296763, 89.38699513033515, 89.53022056717272, 89.55886565454024, 89.60183328559151, 89.573188198224, 89.48725293612145, 89.51589802348897, 89.50157547980521, 89.573188198224, 89.63047837295903, 89.63047837295903, 89.6734460040103, 89.63047837295903, 89.70209109137782, 89.78802635348038, 89.80234889716414, 89.78802635348038, 89.81667144084788, 89.83099398453164, 89.80234889716414, 89.78802635348038, 89.81667144084788, 89.77370380979661, 89.81667144084788, 89.80234889716414, 89.81667144084788, 89.88828415926669, 89.90260670295044, 89.8453165282154, 89.9169292466342, 89.83099398453164, 89.87396161558291, 89.83099398453164, 89.81667144084788, 89.80234889716414, 89.78802635348038, 89.78802635348038, 89.77370380979661, 89.77370380979661, 89.80234889716414, 89.81667144084788, 89.8453165282154, 89.87396161558291, 89.88828415926669, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.66712779704999, "threshold_area100": 87.66642795760526, "threshold_area25": 83.4187911773131}, "test": {"score": [80.00287005811867, 80.52665566477721, 80.87106263901845, 81.1867690320729, 81.7392552199182, 82.10518763004951, 82.37066800602713, 82.6648489631915, 83.05948195450958, 83.52586639879458, 83.80569706536558, 84.16445433020019, 84.4371098514745, 84.6164884838918, 84.8030422616058, 85.03264691109995, 85.20485039822057, 85.39857932123127, 85.61383368013202, 85.87931405610964, 86.05151754323025, 86.2954724833178, 86.54660256870201, 86.79055750878956, 86.95558585061349, 87.1134390471407, 87.24976680777786, 87.47937145727201, 87.63722465379924, 87.73050154265624, 87.90988017507355, 88.06773337160078, 88.22558656812801, 88.29016287579825, 88.49106694410563, 88.55564325177585, 88.65609528592955, 88.87134964483032, 89.01485255076416, 89.10812943962115, 89.18705603788477, 89.29468321733516, 89.39513525148884, 89.49558728564254, 89.54581330271938, 89.63191504627969, 89.73954222573008, 89.83999425988375, 89.86869484107054, 89.8902202769606, 89.904570567554, 89.96197172992754, 89.96914687522423, 90.02654803759776, 90.03372318289445, 90.09112434526799, 90.14135036234484, 90.16287579823492, 90.1772260888283, 90.19875152471838, 90.30637870416876, 90.30637870416876, 90.33507928535553, 90.39965559302576, 90.45705675539931, 90.47858219128938, 90.51445791777284, 90.54315849895961, 90.61490995192652, 90.69383655019014, 90.73688742197031, 90.7297122766736, 90.75841285786038, 90.7297122766736, 90.76558800315708, 90.77993829375045, 90.77993829375045, 90.78711343904715, 90.75123771256368, 90.73688742197031, 90.75841285786038, 90.75841285786038, 90.73688742197031, 90.744062567267, 90.73688742197031, 90.744062567267, 90.744062567267, 90.744062567267, 90.744062567267, 90.744062567267, 90.75841285786038, 90.76558800315708, 90.77993829375045, 90.77993829375045, 90.77993829375045, 90.75841285786038, 90.76558800315708, 90.77276314845376, 90.75841285786038, 90.73688742197031, 90.73688742197031], "calls": [0.0, 1.033220922723685, 1.7937863241730645, 2.7911315204132885, 4.118533400301356, 5.11587859654158, 5.905144579177728, 6.98859151897826, 8.143789911745714, 9.557293535194088, 10.662265910884695, 11.867690320728995, 13.022888713496448, 13.783454114945828, 14.658821841142283, 15.562890148525508, 16.538709908875653, 17.34232618210519, 18.11724187414795, 19.143287651574944, 20.14063284781517, 21.28865609528593, 22.25012556504269, 23.656454043194376, 24.53182176939083, 25.371313769103825, 26.37583411064074, 27.29425270861735, 28.047642964770038, 28.951711272153265, 30.07820908373395, 30.989452536413864, 32.0657243309177, 32.991318074191, 34.16086675755184, 35.14386166319868, 35.92595250053814, 37.160077491569204, 38.26504986725981, 39.24086962760996, 40.431943746860874, 41.50104039606802, 42.58448733586855, 43.91188921575662, 44.93793499318362, 45.94245533472053, 46.50211666786252, 47.49946186410275, 48.66183540216689, 49.551553418956736, 50.391045418669734, 51.25206285427279, 52.28528377699648, 53.36155557150032, 54.445002511300856, 55.55715003228815, 56.39664203200114, 57.30071033938437, 58.26935495443782, 59.202123843007826, 60.29274592810504, 61.046136184257726, 62.129583124058264, 63.37088326038602, 64.3180024395494, 65.31534763578962, 66.262466814953, 67.13065939585276, 68.25715720743345, 69.2329769677836, 70.18727129224366, 70.96936212958312, 72.06715935997704, 72.83489990672311, 73.74614335940302, 74.62151108559948, 75.64038171772978, 76.61620147807993, 77.62072181961685, 78.96964913539499, 79.93829375044844, 81.05761641673243, 81.96885986941236, 83.0451316639162, 84.16445433020019, 85.1761498170338, 86.20937073975749, 87.23541651718448, 88.45519121762216, 89.4238358326756, 90.34225443065222, 91.44722680634283, 92.32976967783597, 93.03293391691182, 94.03027911315203, 94.95587285642534, 96.13977183037956, 97.00078926598263, 98.01965989811293, 99.08158140202339, 100.0], "rescued": [0, 81, 133, 190, 274, 329, 372, 421, 487, 563, 613, 673, 720, 750, 785, 824, 858, 892, 930, 973, 1010, 1054, 1099, 1151, 1181, 1216, 1249, 1286, 1315, 1338, 1376, 1405, 1438, 1461, 1498, 1520, 1540, 1582, 1610, 1631, 1654, 1676, 1699, 1723, 1737, 1759, 1776, 1796, 1812, 1823, 1834, 1853, 1866, 1880, 1893, 1908, 1919, 1927, 1936, 1947, 1968, 1976, 1985, 1998, 2011, 2020, 2029, 2038, 2053, 2068, 2077, 2081, 2089, 2094, 2105, 2109, 2116, 2121, 2124, 2131, 2138, 2142, 2144, 2151, 2158, 2165, 2170, 2176, 2182, 2186, 2191, 2195, 2199, 2203, 2207, 2210, 2216, 2219, 2221, 2222, 2225], "harmed": [0, 8, 12, 25, 32, 36, 42, 50, 61, 72, 83, 93, 102, 107, 116, 123, 133, 140, 148, 154, 167, 177, 187, 205, 212, 225, 239, 244, 251, 261, 274, 281, 292, 306, 315, 328, 334, 346, 354, 362, 374, 381, 390, 400, 407, 417, 419, 425, 437, 445, 454, 465, 477, 483, 495, 502, 506, 511, 518, 526, 532, 540, 545, 549, 554, 560, 564, 569, 574, 578, 581, 586, 590, 599, 605, 607, 614, 618, 626, 635, 639, 643, 648, 654, 662, 668, 673, 679, 685, 689, 692, 695, 697, 701, 705, 711, 716, 718, 722, 726, 729], "exact_score": [80.00287005811867, 80.50513022888714, 80.94998923728205, 81.25852048503982, 81.72490492932482, 82.05496161297266, 82.3921934419172, 82.6720241084882, 83.02360622802613, 83.33931262108058, 83.61914328765158, 83.95637511659612, 84.17880462079358, 84.4299347061778, 84.64518906507857, 84.88196885986942, 85.11157350936357, 85.29812728707756, 85.57795795364856, 85.8290880390328, 86.07304297912033, 86.26677190213103, 86.48920140632848, 86.66858003874579, 86.8407835258664, 87.01298701298703, 87.2138910812944, 87.44349573078856, 87.63722465379924, 87.73767668795294, 87.89552988448015, 88.06773337160078, 88.21123627753462, 88.29016287579825, 88.45519121762216, 88.55564325177585, 88.67762072181962, 88.85699935423692, 88.95745138839062, 89.12247973021454, 89.12965487551124, 89.21575661907154, 89.29468321733516, 89.40231039678554, 89.50993757623593, 89.54581330271938, 89.63909019157639, 89.76824280691685, 89.84716940518047, 89.86869484107054, 89.8902202769606, 89.95479658463084, 89.94762143933414, 90.00502260170768, 90.01219774700438, 90.05524861878453, 90.12700007175145, 90.16287579823492, 90.18440123412499, 90.22027696060846, 90.27050297768531, 90.31355384946545, 90.32790414005885, 90.37813015713569, 90.45705675539931, 90.46423190069599, 90.51445791777284, 90.55750878955298, 90.60773480662984, 90.67231111430006, 90.7297122766736, 90.7297122766736, 90.75841285786038, 90.73688742197031, 90.76558800315708, 90.78711343904715, 90.78711343904715, 90.77993829375045, 90.75841285786038, 90.73688742197031, 90.75841285786038, 90.75841285786038, 90.744062567267, 90.75123771256368, 90.7297122766736, 90.75123771256368, 90.75123771256368, 90.744062567267, 90.744062567267, 90.73688742197031, 90.744062567267, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.77993829375045, 90.75841285786038, 90.77276314845376, 90.77276314845376, 90.75841285786038, 90.73688742197031, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.48354634982464, "threshold_area100": 88.51287938580757, "threshold_area25": 84.110640740475}, "selected_budget": 84, "selected": {"val": {"score": 89.9169292466342, "calls": 83.9873961615583, "rescued": 1063, "harmed": 358}, "test": {"score": 90.73688742197031, "calls": 84.16445433020019, "rescued": 2158, "harmed": 662}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692}, "test": {"score": 90.60055966133315, "calls": 98.91655306019946}}, "old_normalized_interior": {"val": {"score": 90.04583213978802, "calls": 85.99255227728445}, "test": {"score": 90.78711343904715, "calls": 85.32682786826433}}, "old_normalized_endpoints": {"val": {"score": 90.04583213978802, "calls": 85.99255227728445}, "test": {"score": 90.78711343904715, "calls": 85.32682786826433}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.14895445431108, 80.47837295903753, 80.85075909481525, 81.22314523059295, 81.62417645373819, 81.85333715267831, 82.2686909195073, 82.51217416213119, 82.79862503580635, 83.05643082211401, 83.19965625895158, 83.52907476367803, 83.75823546261816, 84.0876539673446, 84.37410484101976, 84.67487825837868, 85.03294185047264, 85.24778000572901, 85.41965052993412, 85.64881122887425, 85.82068175307934, 85.84932684044686, 85.92093955886565, 86.03551990833571, 86.22171297622458, 86.39358350042967, 86.42222858779719, 86.6227441993698, 86.70867946147234, 86.95216270409625, 87.08106559725006, 87.2099684904039, 87.29590375250645, 87.42480664566027, 87.46777427671154, 87.4820968203953, 87.59667716986537, 87.69693497565167, 87.7542251503867, 87.854482956173, 87.96906330564308, 88.11228874248066, 88.269836723002, 88.37009452878831, 88.48467487825837, 88.5133199656259, 88.62790031509596, 88.68519048983099, 88.82841592666857, 88.87138355771984, 88.92867373245488, 88.97164136350615, 89.07189916929245, 89.0575766256087, 89.08622171297623, 89.15783443139502, 89.21512460613005, 89.20080206244629, 89.21512460613005, 89.18647951876252, 89.17215697507876, 89.21512460613005, 89.17215697507876, 89.31538241191636, 89.35835004296763, 89.44428530507018, 89.53022056717272, 89.573188198224, 89.58751074190776, 89.6734460040103, 89.65912346032655, 89.68776854769406, 89.70209109137782, 89.65912346032655, 89.75938126611285, 89.8453165282154, 89.87396161558291, 89.87396161558291, 89.9169292466342, 89.97421942136923, 89.95989687768548, 89.98854196505299, 90.03150959610426, 90.0171870524205, 89.98854196505299, 90.04583213978802, 90.00286450873675, 89.93125179031796, 89.9455743340017, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.14895445431108, 80.47837295903753, 80.85075909481525, 81.22314523059295, 81.62417645373819, 81.85333715267831, 82.2686909195073, 82.51217416213119, 82.79862503580635, 83.05643082211401, 83.19965625895158, 83.52907476367803, 83.75823546261816, 84.0876539673446, 84.37410484101976, 84.67487825837868, 85.03294185047264, 85.24778000572901, 85.41965052993412, 85.64881122887425, 85.82068175307934, 85.84932684044686, 85.92093955886565, 86.03551990833571, 86.22171297622458, 86.39358350042967, 86.42222858779719, 86.6227441993698, 86.70867946147234, 86.95216270409625, 87.08106559725006, 87.2099684904039, 87.29590375250645, 87.42480664566027, 87.46777427671154, 87.4820968203953, 87.59667716986537, 87.69693497565167, 87.7542251503867, 87.854482956173, 87.96906330564308, 88.11228874248066, 88.269836723002, 88.37009452878831, 88.48467487825837, 88.5133199656259, 88.62790031509596, 88.68519048983099, 88.82841592666857, 88.87138355771984, 88.92867373245488, 88.97164136350615, 89.07189916929245, 89.0575766256087, 89.08622171297623, 89.15783443139502, 89.21512460613005, 89.20080206244629, 89.21512460613005, 89.18647951876252, 89.17215697507876, 89.21512460613005, 89.17215697507876, 89.31538241191636, 89.35835004296763, 89.44428530507018, 89.53022056717272, 89.573188198224, 89.58751074190776, 89.6734460040103, 89.65912346032655, 89.68776854769406, 89.70209109137782, 89.65912346032655, 89.75938126611285, 89.8453165282154, 89.87396161558291, 89.87396161558291, 89.9169292466342, 89.97421942136923, 89.95989687768548, 89.98854196505299, 90.03150959610426, 90.0171870524205, 89.98854196505299, 90.04583213978802, 90.00286450873675, 89.93125179031796, 89.9455743340017, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.97421942136923, 89.95989687768548, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 83.54482956173017, "exact_area100": 87.61673426967458}, "test": {"score": [80.00287005811867, 80.54818110066729, 81.05761641673244, 81.5598765875009, 81.98321016000575, 82.34196742484036, 82.68637439908159, 82.89445361268565, 83.26038602281697, 83.62631843294827, 83.8774485183325, 84.08552773193657, 84.4299347061778, 84.63801391978187, 84.83174284279256, 85.0828729281768, 85.31247757767095, 85.55643251775848, 85.78603716725264, 86.04434239793356, 86.25242161153763, 86.43180024395495, 86.64705460285572, 86.82643323527302, 87.02016215828371, 87.22824137188778, 87.42197029489847, 87.50089689316209, 87.5869986367224, 87.65875008968932, 87.80225299562316, 87.97445648274378, 88.07490851689748, 88.28298773050155, 88.44084092702877, 88.54846810647916, 88.69914615770969, 88.78524790127, 88.83547391834685, 88.93592595250054, 89.0722537131377, 89.11530458491785, 89.17270574729139, 89.2659826361484, 89.344909234412, 89.38796010619215, 89.48123699504916, 89.56733873860946, 89.624739900983, 89.68214106335654, 89.73954222573008, 89.77541795221353, 89.87586998636723, 89.8902202769606, 89.97632202052091, 90.06242376408122, 90.13417521704815, 90.21310181531177, 90.27050297768531, 90.277678122982, 90.31355384946545, 90.277678122982, 90.30637870416876, 90.277678122982, 90.32790414005885, 90.34225443065222, 90.33507928535553, 90.39248044772907, 90.43553131950922, 90.44988161010261, 90.49293248188276, 90.50010762717945, 90.52163306306953, 90.57185908014638, 90.58620937073975, 90.60055966133315, 90.60055966133315, 90.64361053311329, 90.65796082370669, 90.70101169548683, 90.70818684078354, 90.69383655019014, 90.70101169548683, 90.70101169548683, 90.70818684078354, 90.73688742197031, 90.78711343904715, 90.79428858434385, 90.80146372964053, 90.85886489201407, 90.87321518260745, 90.90191576379422, 90.85168974671738, 90.85168974671738, 90.83733945612398, 90.82298916553061, 90.76558800315708, 90.76558800315708, 90.77276314845376, 90.744062567267, 90.73688742197031], "calls": [0.0, 1.054746358613762, 2.367797947908445, 3.4081940159288227, 4.627968716366507, 5.768816818540575, 6.89531463012126, 7.6917557580541, 8.911530458491784, 9.994977398292315, 10.992322594532538, 11.66678625242161, 12.692832029848605, 13.826504986725983, 15.211308028987586, 16.122551481667504, 17.22034871206142, 18.08854129296118, 19.12893736098156, 20.31283633493578, 21.32453182176939, 22.12097294970223, 23.11114300064576, 23.878883547391837, 24.826002726555213, 25.794647341608666, 26.81351797373897, 27.63865968285858, 28.449451101384803, 29.41809571643826, 30.379565186195016, 31.4199612542154, 32.524933629906, 33.794934347420536, 34.59855062065007, 35.538494654516754, 36.36363636363637, 37.260529525722895, 37.978044055392125, 39.061490995192656, 40.21668938796011, 41.19250914831025, 42.1683289086604, 43.30917701083447, 44.177369591734234, 45.14601420678769, 46.31556289014853, 47.377484394058975, 48.41788046207935, 49.4008753677262, 50.36951998277965, 51.29511372605295, 52.41443639233695, 53.569634785104405, 54.65308172490493, 55.47104828872784, 56.619071536198604, 57.78862021955945, 59.01557006529382, 59.833536629116736, 60.92415871421396, 61.74930042333357, 62.667719021310184, 63.57178732869341, 64.28212671306594, 65.41579966994333, 66.24094137906292, 67.11630910525939, 68.07777857501614, 69.05359833536629, 70.14422042046351, 71.24201765085742, 72.2895888641745, 73.33716007749157, 74.27710411135826, 75.21704814522494, 76.23591877735524, 77.1184616488484, 78.05123053741838, 78.79027050297769, 79.78044055392121, 80.64863313482098, 81.48094998923729, 82.36349286073043, 83.28191145870704, 84.16445433020019, 85.32682786826433, 86.33852335509793, 87.50807203845878, 88.56281839707253, 89.56016359331277, 90.58620937073975, 91.54050369519983, 92.56654947262682, 93.56389466886704, 94.64734160866757, 95.93886776207218, 97.07971586424625, 98.06271076989309, 98.973954222573, 100.0], "exact_score": [80.00287005811867, 80.52665566477721, 80.92846380139198, 81.3733228097869, 81.78230609169836, 82.05496161297266, 82.3993685872139, 82.70789983497167, 82.95902992035589, 83.26038602281697, 83.62631843294827, 83.8774485183325, 84.22185549257372, 84.49451101384804, 84.65953935567195, 84.79586711630911, 85.02547176580326, 85.26225156059411, 85.53490708186841, 85.73581115017579, 86.0156418167468, 86.17349501327402, 86.40309966276818, 86.63270431226233, 86.85513381645978, 87.04886273947048, 87.24259166248117, 87.47219631197532, 87.54394776494225, 87.63004950850255, 87.73767668795294, 87.84530386740332, 88.07490851689748, 88.17536055105116, 88.30451316639162, 88.48389179880894, 88.63456985003947, 88.74937217478654, 88.84264906364353, 88.92157566190716, 89.01485255076416, 89.11530458491785, 89.17270574729139, 89.2588074908517, 89.29468321733516, 89.37360981559877, 89.45971155915908, 89.56733873860946, 89.6175647556863, 89.624739900983, 89.70366649924661, 89.75389251632346, 89.8902202769606, 89.87586998636723, 89.904570567554, 90.04089832819116, 90.07677405467462, 90.14852550764154, 90.22027696060846, 90.27050297768531, 90.28485326827868, 90.31355384946545, 90.277678122982, 90.29920355887207, 90.32790414005885, 90.32072899476215, 90.32790414005885, 90.38530530243239, 90.43553131950922, 90.44270646480591, 90.48575733658608, 90.49293248188276, 90.51445791777284, 90.54315849895961, 90.59338451603645, 90.60773480662984, 90.60055966133315, 90.64361053311329, 90.65078567840999, 90.71536198608023, 90.70818684078354, 90.68666140489346, 90.72253713137691, 90.70818684078354, 90.744062567267, 90.77993829375045, 90.79428858434385, 90.80863887493722, 90.82298916553061, 90.85168974671738, 90.85168974671738, 90.86604003731075, 90.84451460142068, 90.85886489201407, 90.84451460142068, 90.81581402023392, 90.76558800315708, 90.77276314845376, 90.77276314845376, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 84.23405323957812, "exact_area100": 88.41832676569261}}, "old_normalized_endpoint_budget": 86}, {"seed": 2030, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 1.0, "validation_area100": 0.877116995858155, "training_seconds": 33.90957190899644, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8273405172131074, "operator": "ge"}, {"budget": 0.02, "threshold": 0.7129671850270264, "operator": "ge"}, {"budget": 0.03, "threshold": 0.6313765500365913, "operator": "ge"}, {"budget": 0.04, "threshold": 0.567781867091132, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5060529893019707, "operator": "ge"}, {"budget": 0.06, "threshold": 0.46740793316493495, "operator": "ge"}, {"budget": 0.07, "threshold": 0.43965569226714835, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4061616988641358, "operator": "ge"}, {"budget": 0.09, "threshold": 0.37062246064132465, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3421502249795549, "operator": "ge"}, {"budget": 0.11, "threshold": 0.30390686964770697, "operator": "ge"}, {"budget": 0.12, "threshold": 0.2769386622322546, "operator": "ge"}, {"budget": 0.13, "threshold": 0.2536886851879701, "operator": "ge"}, {"budget": 0.14, "threshold": 0.22864376135171643, "operator": "ge"}, {"budget": 0.15, "threshold": 0.20290264695805169, "operator": "ge"}, {"budget": 0.16, "threshold": 0.18070313232631502, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1623281506939469, "operator": "ge"}, {"budget": 0.18, "threshold": 0.14214841075865156, "operator": "ge"}, {"budget": 0.19, "threshold": 0.1118329675320872, "operator": "ge"}, {"budget": 0.2, "threshold": 0.09067559579084353, "operator": "ge"}, {"budget": 0.21, "threshold": 0.06755392855988818, "operator": "ge"}, {"budget": 0.22, "threshold": 0.04372879091934198, "operator": "ge"}, {"budget": 0.23, "threshold": 0.017096502832965068, "operator": "ge"}, {"budget": 0.24, "threshold": -0.013044121660472056, "operator": "ge"}, {"budget": 0.25, "threshold": -0.03916988237133897, "operator": "ge"}, {"budget": 0.26, "threshold": -0.06970529611450502, "operator": "ge"}, {"budget": 0.27, "threshold": -0.09834185203807035, "operator": "ge"}, {"budget": 0.28, "threshold": -0.1239892922813952, "operator": "ge"}, {"budget": 0.29, "threshold": -0.15223589554488107, "operator": "ge"}, {"budget": 0.3, "threshold": -0.1744314504422635, "operator": "ge"}, {"budget": 0.31, "threshold": -0.20288111328268832, "operator": "ge"}, {"budget": 0.32, "threshold": -0.23407415523004338, "operator": "ge"}, {"budget": 0.33, "threshold": -0.2582844397914433, "operator": "ge"}, {"budget": 0.34, "threshold": -0.286429632067865, "operator": "ge"}, {"budget": 0.35, "threshold": -0.31177469925296647, "operator": "ge"}, {"budget": 0.36, "threshold": -0.33578516204593345, "operator": "ge"}, {"budget": 0.37, "threshold": -0.3636344600075417, "operator": "ge"}, {"budget": 0.38, "threshold": -0.3899697422751145, "operator": "ge"}, {"budget": 0.39, "threshold": -0.41542256103679165, "operator": "ge"}, {"budget": 0.4, "threshold": -0.4401614302798508, "operator": "ge"}, {"budget": 0.41, "threshold": -0.45870401419667545, "operator": "ge"}, {"budget": 0.42, "threshold": -0.48783798739182693, "operator": "ge"}, {"budget": 0.43, "threshold": -0.5087209239199088, "operator": "ge"}, {"budget": 0.44, "threshold": -0.5291029674870461, "operator": "ge"}, {"budget": 0.45, "threshold": -0.5525623954682751, "operator": "ge"}, {"budget": 0.46, "threshold": -0.5697364464777451, "operator": "ge"}, {"budget": 0.47, "threshold": -0.5933652843967537, "operator": "ge"}, {"budget": 0.48, "threshold": -0.6115561828177283, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6311925124493285, "operator": "ge"}, {"budget": 0.5, "threshold": -0.6483382442191651, "operator": "ge"}, {"budget": 0.51, "threshold": -0.6634846091739668, "operator": "ge"}, {"budget": 0.52, "threshold": -0.6794707385814078, "operator": "ge"}, {"budget": 0.53, "threshold": -0.6954036117970601, "operator": "ge"}, {"budget": 0.54, "threshold": -0.7110963203494381, "operator": "ge"}, {"budget": 0.55, "threshold": -0.7262811044531231, "operator": "ge"}, {"budget": 0.56, "threshold": -0.7406559130126787, "operator": "ge"}, {"budget": 0.57, "threshold": -0.7555210285843554, "operator": "ge"}, {"budget": 0.58, "threshold": -0.7692144566952182, "operator": "ge"}, {"budget": 0.59, "threshold": -0.7780197408294942, "operator": "ge"}, {"budget": 0.6, "threshold": -0.7896124360988113, "operator": "ge"}, {"budget": 0.61, "threshold": -0.8025624081203738, "operator": "ge"}, {"budget": 0.62, "threshold": -0.8136540155614775, "operator": "ge"}, {"budget": 0.63, "threshold": -0.8245295620688988, "operator": "ge"}, {"budget": 0.64, "threshold": -0.8347642875008994, "operator": "ge"}, {"budget": 0.65, "threshold": -0.8429699393692404, "operator": "ge"}, {"budget": 0.66, "threshold": -0.8516660303450688, "operator": "ge"}, {"budget": 0.67, "threshold": -0.8601470365141909, "operator": "ge"}, {"budget": 0.68, "threshold": -0.867754360473464, "operator": "ge"}, {"budget": 0.69, "threshold": -0.8748684131275989, "operator": "ge"}, {"budget": 0.7, "threshold": -0.8818523565377712, "operator": "ge"}, {"budget": 0.71, "threshold": -0.8882812208257707, "operator": "ge"}, {"budget": 0.72, "threshold": -0.8947943181903601, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9011357989429182, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9056824428625613, "operator": "ge"}, {"budget": 0.75, "threshold": -0.911945161302092, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9167353616150435, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9227532853129209, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9282859058211952, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9321981767217702, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9370181291166882, "operator": "ge"}, {"budget": 0.81, "threshold": -0.942420569659195, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9464695799588174, "operator": "ge"}, {"budget": 0.83, "threshold": -0.949867381740924, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9545535895414102, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9574600295811745, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9613254867177733, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9643797402382894, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9671637491887735, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9706631617351488, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9737746121961671, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9763937791691334, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9790576549262332, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9818550458170433, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9844055484631499, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9866043941395292, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9890766633632463, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9914473681375945, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9934386226997753, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9954928159827926, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.1059868232598, 80.60727585219134, 80.93669435691778, 81.32340303637926, 81.63849899742193, 82.08249785161844, 82.38327126897737, 82.69836723002004, 82.94185047264394, 83.34288169578916, 83.60068748209682, 83.87281581208822, 84.0160412489258, 84.20223431681467, 84.41707247207103, 84.54597536522486, 84.7035233457462, 84.91836150100256, 85.17616728731022, 85.3910054425666, 85.56287596677168, 85.74906903466055, 85.97822973360068, 86.12145517043827, 86.1787453451733, 86.42222858779719, 86.60842165568604, 86.78029217989113, 87.08106559725006, 87.2099684904039, 87.32454883987396, 87.39616155829275, 87.52506445144657, 87.66828988828415, 87.7542251503867, 87.86880549985678, 87.96906330564308, 88.09796619879691, 88.14093382984818, 88.25551417931824, 88.31280435405327, 88.4417072472071, 88.5133199656259, 88.59925522772843, 88.62790031509596, 88.75680320824978, 88.90002864508736, 88.97164136350615, 88.97164136350615, 89.0575766256087, 89.1291893440275, 89.20080206244629, 89.20080206244629, 89.22944714981381, 89.24376969349757, 89.31538241191636, 89.34402749928387, 89.34402749928387, 89.45860784875393, 89.48725293612145, 89.53022056717272, 89.50157547980521, 89.48725293612145, 89.51589802348897, 89.50157547980521, 89.53022056717272, 89.53022056717272, 89.53022056717272, 89.58751074190776, 89.61615582927529, 89.61615582927529, 89.68776854769406, 89.75938126611285, 89.74505872242909, 89.75938126611285, 89.75938126611285, 89.74505872242909, 89.78802635348038, 89.83099398453164, 89.93125179031796, 89.9169292466342, 89.9455743340017, 89.93125179031796, 89.90260670295044, 89.88828415926669, 89.90260670295044, 89.88828415926669, 89.87396161558291, 89.88828415926669, 89.9455743340017, 89.97421942136923, 89.98854196505299, 89.97421942136923, 89.93125179031796, 89.93125179031796, 89.90260670295044, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 72, 98, 128, 154, 187, 211, 238, 260, 291, 312, 336, 353, 377, 397, 415, 434, 451, 473, 493, 512, 530, 550, 567, 581, 602, 618, 633, 656, 673, 687, 700, 714, 727, 738, 753, 764, 775, 781, 796, 802, 814, 823, 834, 843, 855, 867, 875, 881, 890, 898, 906, 914, 924, 931, 939, 944, 947, 957, 962, 966, 968, 971, 976, 979, 986, 987, 993, 999, 1002, 1004, 1009, 1015, 1016, 1018, 1022, 1025, 1029, 1035, 1046, 1049, 1053, 1054, 1055, 1058, 1061, 1063, 1064, 1067, 1072, 1076, 1078, 1080, 1080, 1084, 1085, 1087, 1087, 1087, 1087], "harmed": [0, 12, 17, 20, 23, 27, 29, 32, 37, 42, 45, 48, 53, 60, 71, 76, 85, 93, 95, 99, 104, 111, 116, 120, 127, 137, 141, 144, 147, 149, 157, 163, 171, 176, 179, 184, 191, 195, 197, 200, 207, 209, 212, 216, 221, 228, 231, 233, 236, 242, 245, 248, 251, 259, 267, 273, 276, 279, 282, 284, 287, 288, 292, 296, 299, 303, 308, 309, 315, 317, 318, 320, 320, 321, 323, 324, 328, 332, 333, 336, 340, 344, 346, 348, 351, 355, 357, 360, 362, 364, 365, 367, 368, 371, 374, 378, 381, 381, 381, 381, 383], "exact_score": [79.81953594958465, 80.1059868232598, 80.60727585219134, 80.93669435691778, 81.32340303637926, 81.63849899742193, 82.08249785161844, 82.38327126897737, 82.69836723002004, 82.94185047264394, 83.34288169578916, 83.60068748209682, 83.87281581208822, 84.0160412489258, 84.20223431681467, 84.41707247207103, 84.54597536522486, 84.7035233457462, 84.91836150100256, 85.17616728731022, 85.3910054425666, 85.56287596677168, 85.74906903466055, 85.97822973360068, 86.12145517043827, 86.1787453451733, 86.42222858779719, 86.60842165568604, 86.78029217989113, 87.08106559725006, 87.2099684904039, 87.32454883987396, 87.39616155829275, 87.52506445144657, 87.66828988828415, 87.7542251503867, 87.86880549985678, 87.96906330564308, 88.09796619879691, 88.14093382984818, 88.25551417931824, 88.31280435405327, 88.4417072472071, 88.5133199656259, 88.59925522772843, 88.62790031509596, 88.75680320824978, 88.90002864508736, 88.97164136350615, 88.97164136350615, 89.0575766256087, 89.1291893440275, 89.20080206244629, 89.20080206244629, 89.22944714981381, 89.24376969349757, 89.31538241191636, 89.34402749928387, 89.34402749928387, 89.45860784875393, 89.48725293612145, 89.53022056717272, 89.50157547980521, 89.48725293612145, 89.51589802348897, 89.50157547980521, 89.53022056717272, 89.53022056717272, 89.53022056717272, 89.58751074190776, 89.61615582927529, 89.61615582927529, 89.68776854769406, 89.75938126611285, 89.74505872242909, 89.75938126611285, 89.75938126611285, 89.74505872242909, 89.78802635348038, 89.83099398453164, 89.93125179031796, 89.9169292466342, 89.9455743340017, 89.93125179031796, 89.90260670295044, 89.88828415926669, 89.90260670295044, 89.88828415926669, 89.87396161558291, 89.88828415926669, 89.9455743340017, 89.97421942136923, 89.98854196505299, 89.97421942136923, 89.93125179031796, 89.93125179031796, 89.90260670295044, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.7116995858155, "threshold_area100": 87.71097106846176, "threshold_area25": 83.55771985104555}, "test": {"score": [80.00287005811867, 80.59840711774415, 81.16524359618282, 81.59575231398436, 81.91145870703882, 82.35631771543373, 82.57157207433451, 82.84422759560881, 83.23886058692689, 83.64784386883835, 83.74829590299204, 84.19315491138696, 84.38688383439765, 84.60931333859511, 84.8173925521992, 85.16179952644042, 85.4488053383081, 85.59230824424195, 85.77168687665926, 85.99411638085671, 86.13761928679055, 86.36004879098802, 86.5609528592954, 86.70445576522926, 87.01298701298703, 87.17801535481094, 87.33586855133817, 87.50807203845878, 87.60134892731578, 87.73050154265624, 87.83812872210662, 87.97445648274378, 88.14665996986439, 88.21123627753462, 88.31886345698501, 88.51259237999571, 88.68479586711632, 88.83547391834685, 88.97897682428069, 89.05790342254431, 89.18705603788477, 89.22293176436823, 89.35925952500538, 89.52428786682931, 89.58168902920285, 89.63191504627969, 89.70366649924661, 89.78976824280693, 89.82564396929038, 89.84716940518047, 89.904570567554, 89.92609600344407, 89.94762143933414, 90.06242376408122, 90.06242376408122, 90.09112434526799, 90.14852550764154, 90.13417521704815, 90.22745210590514, 90.22027696060846, 90.27050297768531, 90.34225443065222, 90.34942957594892, 90.37095501183899, 90.42118102891584, 90.40683073832245, 90.39965559302576, 90.43553131950922, 90.44988161010261, 90.47858219128938, 90.48575733658608, 90.47858219128938, 90.50728277247615, 90.51445791777284, 90.52880820836621, 90.59338451603645, 90.64361053311329, 90.66513596900337, 90.62926024251992, 90.64361053311329, 90.71536198608023, 90.75123771256368, 90.75841285786038, 90.75123771256368, 90.75841285786038, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.78711343904715, 90.78711343904715, 90.75123771256368, 90.77993829375045, 90.81581402023392, 90.80146372964053, 90.8301643108273, 90.78711343904715, 90.75841285786038, 90.744062567267, 90.72253713137691, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.0403960680203774, 2.3534476573150607, 3.444069742412284, 4.326612613905432, 5.539212169046423, 6.428930185836264, 7.211021023175719, 8.194015928822559, 9.385090048073472, 10.303508646050084, 11.803114013058764, 12.77175862811222, 13.56819975604506, 14.493793499318361, 15.706393054459353, 16.6822128148095, 17.435603070962184, 18.26791992537849, 19.41594317284925, 20.2052091554854, 21.10210231757193, 22.013345770251846, 23.003515821195382, 24.21611537633637, 25.127358829016288, 26.167754897036666, 27.179450383870275, 28.076343545956806, 28.958886417449953, 29.67640094711918, 30.774198177513096, 31.857645117313627, 32.82628973236708, 34.01736385161799, 34.87838128722107, 35.782449594604294, 36.82284566262467, 38.006744636578894, 39.0184401234125, 40.05166104613618, 40.79070101169548, 42.06070172921002, 43.14414866901055, 44.00516610461362, 45.06708760852407, 45.8778790270503, 47.12635430867475, 48.11652435961828, 49.12821984645189, 50.254717658032575, 51.10855994833895, 52.141780871062636, 53.23957810145655, 54.18669728061993, 55.16969218626677, 56.17421252780369, 57.228958886417445, 58.26217980914114, 58.95099375762359, 59.88376264619358, 61.146588218411424, 62.23003515821195, 63.342182679199254, 64.61218339671379, 65.45167539642678, 66.36291884910669, 67.48941666068737, 68.46523642103752, 69.29037813015714, 70.35229963406759, 71.27071823204419, 72.21783741120757, 73.2223577527445, 73.91117170122695, 74.85829088039033, 75.7121331706967, 76.77405467460716, 77.97230393915477, 78.84049652005453, 79.83066657099806, 81.15806845088613, 82.01908588648921, 82.7653009973452, 83.97790055248619, 84.77434168041903, 85.92236492788979, 86.8479586711631, 87.60134892731578, 88.76372246537993, 89.96914687522423, 90.8229891655306, 91.7629331993973, 92.81767955801105, 93.85807562603142, 94.74779364282126, 95.80254000143503, 96.97926383009255, 97.94790844514601, 98.9883045131664, 100.0], "rescued": [0, 87, 178, 244, 293, 363, 403, 448, 506, 572, 606, 680, 715, 755, 794, 852, 905, 935, 970, 1015, 1047, 1088, 1125, 1156, 1211, 1241, 1274, 1311, 1332, 1358, 1384, 1418, 1451, 1469, 1498, 1530, 1557, 1589, 1617, 1635, 1658, 1673, 1702, 1731, 1747, 1764, 1781, 1806, 1818, 1829, 1843, 1852, 1864, 1885, 1895, 1909, 1921, 1925, 1942, 1947, 1957, 1972, 1979, 1988, 1999, 2004, 2010, 2019, 2028, 2033, 2042, 2047, 2056, 2060, 2066, 2078, 2087, 2098, 2104, 2110, 2123, 2133, 2139, 2142, 2152, 2156, 2166, 2172, 2177, 2183, 2183, 2191, 2198, 2199, 2207, 2210, 2211, 2213, 2215, 2219, 2225], "harmed": [0, 4, 16, 22, 27, 35, 45, 52, 55, 64, 84, 96, 104, 113, 123, 133, 146, 156, 166, 180, 192, 202, 211, 222, 234, 241, 252, 265, 273, 281, 292, 307, 316, 325, 339, 344, 347, 358, 366, 373, 378, 388, 398, 404, 412, 422, 429, 442, 449, 457, 463, 469, 478, 483, 493, 503, 507, 513, 517, 523, 526, 531, 537, 543, 547, 554, 561, 565, 572, 573, 581, 587, 592, 595, 599, 602, 604, 612, 623, 627, 630, 635, 640, 644, 653, 656, 665, 670, 674, 680, 685, 689, 691, 694, 698, 707, 712, 716, 721, 726, 729], "exact_score": [80.00287005811867, 80.59123197244745, 81.0073903996556, 81.43789911745714, 81.77513094640166, 82.1267130659396, 82.50699576666428, 82.75812585204851, 83.15275884336658, 83.46846523642104, 83.71242017650857, 83.94919997129942, 84.22903063787042, 84.4371098514745, 84.68823993685872, 84.97524574872641, 85.24072612470403, 85.5492573724618, 85.69276027839564, 85.87213891081295, 86.11609385090048, 86.33134820980125, 86.5609528592954, 86.69728061993256, 86.95558585061349, 87.14931477362417, 87.3071679701514, 87.50807203845878, 87.5941737820191, 87.72332639735954, 87.88835473918347, 87.99598191863386, 88.15383511516109, 88.22558656812801, 88.31168831168831, 88.53411781588576, 88.74219702948984, 88.84264906364353, 88.97897682428069, 89.05790342254431, 89.17270574729139, 89.251632345555, 89.344909234412, 89.50993757623593, 89.58168902920285, 89.63191504627969, 89.71801678984, 89.79694338810361, 89.83281911458707, 89.84716940518047, 89.8902202769606, 89.93327114874077, 89.94762143933414, 90.04089832819116, 90.05524861878453, 90.09112434526799, 90.13417521704815, 90.13417521704815, 90.20592667001507, 90.22027696060846, 90.277678122982, 90.32790414005885, 90.34942957594892, 90.38530530243239, 90.40683073832245, 90.42118102891584, 90.43553131950922, 90.42835617421254, 90.44988161010261, 90.46423190069599, 90.48575733658608, 90.48575733658608, 90.50010762717945, 90.51445791777284, 90.53598335366291, 90.58620937073975, 90.66513596900337, 90.65796082370669, 90.62926024251992, 90.65078567840999, 90.71536198608023, 90.75841285786038, 90.75841285786038, 90.75123771256368, 90.75841285786038, 90.79428858434385, 90.77276314845376, 90.78711343904715, 90.80146372964053, 90.77993829375045, 90.75123771256368, 90.79428858434385, 90.81581402023392, 90.80146372964053, 90.81581402023392, 90.77276314845376, 90.75123771256368, 90.744062567267, 90.72253713137691, 90.71536198608023, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.48613824686102, "threshold_area100": 88.51058333931263, "threshold_area25": 84.1988950276243}, "selected_budget": 92, "selected": {"val": {"score": 89.98854196505299, "calls": 91.99369808077915, "rescued": 1078, "harmed": 368}, "test": {"score": 90.81581402023392, "calls": 91.7629331993973, "rescued": 2198, "harmed": 691}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.614740252494812, "operator": "ge"}, {"budget": 0.02, "threshold": 0.4599558711051941, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3557879328727722, "operator": "ge"}, {"budget": 0.04, "threshold": 0.29493632912635803, "operator": "ge"}, {"budget": 0.05, "threshold": 0.2592149078845978, "operator": "ge"}, {"budget": 0.06, "threshold": 0.23114633560180664, "operator": "ge"}, {"budget": 0.07, "threshold": 0.20653142035007477, "operator": "ge"}, {"budget": 0.08, "threshold": 0.1808924674987793, "operator": "ge"}, {"budget": 0.09, "threshold": 0.15721789002418518, "operator": "ge"}, {"budget": 0.1, "threshold": 0.14066898822784424, "operator": "ge"}, {"budget": 0.11, "threshold": 0.1250370442867279, "operator": "ge"}, {"budget": 0.12, "threshold": 0.1110701858997345, "operator": "ge"}, {"budget": 0.13, "threshold": 0.09624584019184113, "operator": "ge"}, {"budget": 0.14, "threshold": 0.08232586085796356, "operator": "ge"}, {"budget": 0.15, "threshold": 0.07040198892354965, "operator": "ge"}, {"budget": 0.16, "threshold": 0.0597362220287323, "operator": "ge"}, {"budget": 0.17, "threshold": 0.0492674857378006, "operator": "ge"}, {"budget": 0.18, "threshold": 0.042315926402807236, "operator": "ge"}, {"budget": 0.19, "threshold": 0.032909102737903595, "operator": "ge"}, {"budget": 0.2, "threshold": 0.023658111691474915, "operator": "ge"}, {"budget": 0.21, "threshold": 0.01723085343837738, "operator": "ge"}, {"budget": 0.22, "threshold": 0.010364092886447906, "operator": "ge"}, {"budget": 0.23, "threshold": 0.004422783851623535, "operator": "ge"}, {"budget": 0.24, "threshold": -0.001970284152776003, "operator": "ge"}, {"budget": 0.25, "threshold": -0.003125621471554041, "operator": "ge"}, {"budget": 0.26, "threshold": -0.0039084237068891525, "operator": "ge"}, {"budget": 0.27, "threshold": -0.004575839266180992, "operator": "ge"}, {"budget": 0.28, "threshold": -0.005328751634806395, "operator": "ge"}, {"budget": 0.29, "threshold": -0.006008798722177744, "operator": "ge"}, {"budget": 0.3, "threshold": -0.0066112312488257885, "operator": "ge"}, {"budget": 0.31, "threshold": -0.00720612145960331, "operator": "ge"}, {"budget": 0.32, "threshold": -0.007774017285555601, "operator": "ge"}, {"budget": 0.33, "threshold": -0.00821196660399437, "operator": "ge"}, {"budget": 0.34, "threshold": -0.00894700363278389, "operator": "ge"}, {"budget": 0.35, "threshold": -0.009569737128913403, "operator": "ge"}, {"budget": 0.36, "threshold": -0.010257299058139324, "operator": "ge"}, {"budget": 0.37, "threshold": -0.010935870930552483, "operator": "ge"}, {"budget": 0.38, "threshold": -0.011597714386880398, "operator": "ge"}, {"budget": 0.39, "threshold": -0.012253902852535248, "operator": "ge"}, {"budget": 0.4, "threshold": -0.012885247357189655, "operator": "ge"}, {"budget": 0.41, "threshold": -0.01364807691425085, "operator": "ge"}, {"budget": 0.42, "threshold": -0.014300665818154812, "operator": "ge"}, {"budget": 0.43, "threshold": -0.015095840208232403, "operator": "ge"}, {"budget": 0.44, "threshold": -0.01584518887102604, "operator": "ge"}, {"budget": 0.45, "threshold": -0.016573674976825714, "operator": "ge"}, {"budget": 0.46, "threshold": -0.01721058040857315, "operator": "ge"}, {"budget": 0.47, "threshold": -0.017980340868234634, "operator": "ge"}, {"budget": 0.48, "threshold": -0.018677419051527977, "operator": "ge"}, {"budget": 0.49, "threshold": -0.019487181678414345, "operator": "ge"}, {"budget": 0.5, "threshold": -0.02012680470943451, "operator": "ge"}, {"budget": 0.51, "threshold": -0.020814325660467148, "operator": "ge"}, {"budget": 0.52, "threshold": -0.021649664267897606, "operator": "ge"}, {"budget": 0.53, "threshold": -0.02256297506392002, "operator": "ge"}, {"budget": 0.54, "threshold": -0.023455718532204628, "operator": "ge"}, {"budget": 0.55, "threshold": -0.024215221405029297, "operator": "ge"}, {"budget": 0.56, "threshold": -0.025219034403562546, "operator": "ge"}, {"budget": 0.57, "threshold": -0.025990232825279236, "operator": "ge"}, {"budget": 0.58, "threshold": -0.02683066390454769, "operator": "ge"}, {"budget": 0.59, "threshold": -0.02779858559370041, "operator": "ge"}, {"budget": 0.6, "threshold": -0.02866947650909424, "operator": "ge"}, {"budget": 0.61, "threshold": -0.02958068624138832, "operator": "ge"}, {"budget": 0.62, "threshold": -0.030473604798316956, "operator": "ge"}, {"budget": 0.63, "threshold": -0.03146655857563019, "operator": "ge"}, {"budget": 0.64, "threshold": -0.03251677006483078, "operator": "ge"}, {"budget": 0.65, "threshold": -0.03343205899000168, "operator": "ge"}, {"budget": 0.66, "threshold": -0.0345095619559288, "operator": "ge"}, {"budget": 0.67, "threshold": -0.03550612926483154, "operator": "ge"}, {"budget": 0.68, "threshold": -0.03671105206012726, "operator": "ge"}, {"budget": 0.69, "threshold": -0.03774190694093704, "operator": "ge"}, {"budget": 0.7, "threshold": -0.03892599791288376, "operator": "ge"}, {"budget": 0.71, "threshold": -0.04018964618444443, "operator": "ge"}, {"budget": 0.72, "threshold": -0.04174225032329559, "operator": "ge"}, {"budget": 0.73, "threshold": -0.04329107701778412, "operator": "ge"}, {"budget": 0.74, "threshold": -0.044608067721128464, "operator": "ge"}, {"budget": 0.75, "threshold": -0.046023037284612656, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0474967435002327, "operator": "ge"}, {"budget": 0.77, "threshold": -0.04921906068921089, "operator": "ge"}, {"budget": 0.78, "threshold": -0.051030632108449936, "operator": "ge"}, {"budget": 0.79, "threshold": -0.05307430773973465, "operator": "ge"}, {"budget": 0.8, "threshold": -0.05496010184288025, "operator": "ge"}, {"budget": 0.81, "threshold": -0.056902531534433365, "operator": "ge"}, {"budget": 0.82, "threshold": -0.05935601517558098, "operator": "ge"}, {"budget": 0.83, "threshold": -0.06137143820524216, "operator": "ge"}, {"budget": 0.84, "threshold": -0.06404557079076767, "operator": "ge"}, {"budget": 0.85, "threshold": -0.06689807772636414, "operator": "ge"}, {"budget": 0.86, "threshold": -0.06968793272972107, "operator": "ge"}, {"budget": 0.87, "threshold": -0.07313328981399536, "operator": "ge"}, {"budget": 0.88, "threshold": -0.07647059857845306, "operator": "ge"}, {"budget": 0.89, "threshold": -0.079743392765522, "operator": "ge"}, {"budget": 0.9, "threshold": -0.0830182358622551, "operator": "ge"}, {"budget": 0.91, "threshold": -0.08704479038715363, "operator": "ge"}, {"budget": 0.92, "threshold": -0.09170300513505936, "operator": "ge"}, {"budget": 0.93, "threshold": -0.09693571925163269, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10392462462186813, "operator": "ge"}, {"budget": 0.95, "threshold": -0.11076820641756058, "operator": "ge"}, {"budget": 0.96, "threshold": -0.11836428195238113, "operator": "ge"}, {"budget": 0.97, "threshold": -0.1279306560754776, "operator": "ge"}, {"budget": 0.98, "threshold": -0.142754927277565, "operator": "ge"}, {"budget": 0.99, "threshold": -0.1725480854511261, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.12030936694356, 80.54998567745632, 80.90804926955026, 81.35204812374677, 81.66714408478946, 82.01088513319965, 82.42623890002864, 82.72701231738756, 83.02778573474649, 83.45746204525923, 83.62933256946434, 83.8441707247207, 84.03036379260956, 84.14494414207962, 84.34545975365224, 84.57462045259237, 84.8037811515325, 85.03294185047264, 85.24778000572901, 85.46261816098539, 85.57719851045546, 85.86364938413061, 85.94958464623316, 86.0068748209682, 86.04984245201948, 86.09281008307075, 86.16442280148954, 86.19306788885706, 86.15010025780579, 86.19306788885706, 86.22171297622458, 86.23603551990833, 86.23603551990833, 86.29332569464336, 86.30764823832712, 86.32197078201088, 86.29332569464336, 86.30764823832712, 86.3506158693784, 86.36493841306215, 86.32197078201088, 86.3506158693784, 86.40790604411343, 86.42222858779719, 86.40790604411343, 86.47951876253221, 86.47951876253221, 86.47951876253221, 86.52248639358349, 86.65138928673733, 86.6943569177886, 86.80893726725866, 86.83758235462618, 86.90919507304497, 86.96648524778, 87.08106559725006, 87.12403322830134, 87.12403322830134, 87.2386135777714, 87.3102262961902, 87.32454883987396, 87.39616155829275, 87.41048410197651, 87.46777427671154, 87.53938699513033, 87.5823546261816, 87.68261243196791, 87.7542251503867, 87.91177313090805, 87.94041821827557, 88.01203093669436, 88.15525637353193, 88.269836723002, 88.37009452878831, 88.38441707247206, 88.47035233457461, 88.5133199656259, 88.55628759667717, 88.59925522772843, 88.75680320824978, 88.82841592666857, 88.84273847035233, 88.91435118877112, 88.91435118877112, 88.92867373245488, 88.98596390718991, 89.08622171297623, 89.17215697507876, 89.24376969349757, 89.44428530507018, 89.53022056717272, 89.50157547980521, 89.61615582927529, 89.70209109137782, 89.60183328559151, 89.61615582927529, 89.73073617874535, 89.80234889716414, 89.85963907189917, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 33, 70, 100, 134, 159, 187, 220, 245, 274, 307, 330, 350, 372, 385, 406, 427, 446, 467, 487, 505, 518, 539, 551, 559, 563, 569, 574, 578, 578, 582, 585, 587, 589, 593, 596, 600, 601, 603, 607, 608, 609, 613, 618, 621, 624, 630, 633, 636, 641, 651, 655, 664, 670, 677, 684, 692, 697, 703, 713, 721, 725, 731, 733, 740, 745, 751, 759, 767, 778, 783, 791, 803, 814, 822, 828, 837, 845, 850, 860, 873, 883, 893, 902, 908, 916, 924, 938, 944, 955, 974, 983, 991, 1002, 1016, 1024, 1036, 1048, 1059, 1070, 1087], "harmed": [0, 12, 19, 24, 27, 30, 34, 38, 42, 50, 53, 64, 69, 78, 83, 90, 95, 98, 103, 108, 111, 116, 117, 123, 127, 128, 131, 131, 133, 136, 137, 138, 139, 141, 141, 143, 146, 149, 150, 151, 151, 155, 157, 158, 160, 164, 165, 168, 171, 173, 174, 175, 176, 180, 182, 185, 185, 187, 193, 195, 198, 201, 202, 203, 206, 206, 209, 210, 213, 213, 216, 219, 221, 224, 225, 230, 233, 238, 240, 247, 249, 254, 263, 267, 273, 280, 284, 291, 291, 297, 302, 305, 315, 318, 326, 341, 352, 356, 362, 369, 383], "exact_score": [79.81953594958465, 80.12030936694356, 80.54998567745632, 80.90804926955026, 81.35204812374677, 81.66714408478946, 82.01088513319965, 82.42623890002864, 82.72701231738756, 83.02778573474649, 83.45746204525923, 83.62933256946434, 83.8441707247207, 84.03036379260956, 84.14494414207962, 84.34545975365224, 84.57462045259237, 84.8037811515325, 85.03294185047264, 85.24778000572901, 85.46261816098539, 85.57719851045546, 85.86364938413061, 85.94958464623316, 86.0068748209682, 86.04984245201948, 86.09281008307075, 86.16442280148954, 86.19306788885706, 86.15010025780579, 86.19306788885706, 86.22171297622458, 86.23603551990833, 86.23603551990833, 86.29332569464336, 86.30764823832712, 86.32197078201088, 86.29332569464336, 86.30764823832712, 86.3506158693784, 86.36493841306215, 86.32197078201088, 86.3506158693784, 86.40790604411343, 86.42222858779719, 86.40790604411343, 86.47951876253221, 86.47951876253221, 86.47951876253221, 86.52248639358349, 86.65138928673733, 86.6943569177886, 86.80893726725866, 86.83758235462618, 86.90919507304497, 86.96648524778, 87.08106559725006, 87.12403322830134, 87.12403322830134, 87.2386135777714, 87.3102262961902, 87.32454883987396, 87.39616155829275, 87.41048410197651, 87.46777427671154, 87.53938699513033, 87.5823546261816, 87.68261243196791, 87.7542251503867, 87.91177313090805, 87.94041821827557, 88.01203093669436, 88.15525637353193, 88.269836723002, 88.37009452878831, 88.38441707247206, 88.47035233457461, 88.5133199656259, 88.55628759667717, 88.59925522772843, 88.75680320824978, 88.82841592666857, 88.84273847035233, 88.91435118877112, 88.91435118877112, 88.92867373245488, 88.98596390718991, 89.08622171297623, 89.17215697507876, 89.24376969349757, 89.44428530507018, 89.53022056717272, 89.50157547980521, 89.61615582927529, 89.70209109137782, 89.60183328559151, 89.61615582927529, 89.73073617874535, 89.80234889716414, 89.85963907189917, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 86.6350441094139, "threshold_area100": 86.63434545975366, "threshold_area25": 83.57347464909768}, "test": {"score": [80.00287005811867, 80.53383081007391, 81.14371816029275, 81.53117600631413, 81.9329841429289, 82.21281480949989, 82.57157207433451, 82.85140274090551, 83.17428427925665, 83.59044270646481, 83.76264619358543, 84.04247686015643, 84.30795723613403, 84.6236636291885, 84.96089545813304, 85.2766018511875, 85.47033077419817, 85.64253426131879, 85.76451173136256, 85.92236492788979, 86.08021812441702, 86.2954724833178, 86.51790198751526, 86.70445576522926, 86.97711128650356, 87.01298701298703, 87.01298701298703, 87.04886273947048, 87.07756332065725, 87.14213962832747, 87.17801535481094, 87.2067159359977, 87.24976680777786, 87.27846738896463, 87.3143431154481, 87.35021884193155, 87.37174427782163, 87.38609456841502, 87.46502116667862, 87.54394776494225, 87.57264834612901, 87.64439979909594, 87.665925234986, 87.68745067087609, 87.6802755255794, 87.68745067087609, 87.69462581617277, 87.72332639735954, 87.80942814091985, 87.84530386740332, 87.90270502977687, 87.88835473918347, 87.90270502977687, 87.92423046566694, 87.98880677333716, 87.99598191863386, 88.03185764511731, 88.10360909808423, 88.11078424338093, 88.08925880749085, 88.09643395278755, 88.11078424338093, 88.20406113223794, 88.23993685872139, 88.25428714931478, 88.29733802109493, 88.31168831168831, 88.34038889287508, 88.37626461935855, 88.44084092702877, 88.47671665351223, 88.51259237999571, 88.53411781588576, 88.64174499533615, 88.76372246537993, 88.82829877305016, 88.91440051661046, 88.89287508072039, 88.93592595250054, 89.01485255076416, 89.11530458491785, 89.17270574729139, 89.37360981559877, 89.4310109779723, 89.47406184975246, 89.55298844801608, 89.65344048216977, 89.75389251632346, 89.85434455047715, 89.85434455047715, 89.92609600344407, 89.93327114874077, 90.05524861878453, 90.10547463586138, 90.19875152471838, 90.27050297768531, 90.37095501183899, 90.47140704599268, 90.65078567840999, 90.76558800315708, 90.73688742197031], "calls": [0.0, 0.9686446150534548, 2.15971873430437, 3.1498887852479016, 4.312262323312047, 5.173279758915118, 6.170624955155342, 7.175145296692258, 8.158140202339096, 9.485542082227164, 10.2891583554567, 11.293678696993615, 12.16904642319007, 13.202267345913754, 14.414866901054745, 15.483963550261892, 16.36650642175504, 17.4212527803688, 18.138767310038027, 19.16481308746502, 20.133457702518477, 20.987299992824855, 21.977470043768385, 22.88153835115161, 24.2806916840066, 25.091483102532823, 26.05295257228959, 26.928320298486046, 27.93284064002296, 28.82973380210949, 29.77685298127287, 30.60199469039248, 31.513238143072396, 32.23075267274162, 33.22809786898185, 34.25414364640884, 35.165387099088754, 36.12685656884552, 37.160077491569204, 38.04979550835905, 38.95386381574227, 40.18081366147664, 41.0561813876731, 42.13245318217694, 43.12979837841716, 44.26347133529454, 44.95946042907369, 46.03573222357753, 46.95415082155414, 48.04477290665136, 48.92731577814451, 49.80268350434096, 50.83590442706465, 51.833249623304866, 52.952572289588865, 53.899691468752245, 55.09076558800315, 55.93025758771615, 56.9060773480663, 57.989524287866836, 58.922293176436824, 59.969864389753894, 60.88110784243381, 61.87845303867403, 62.91884910669441, 63.74399081581402, 64.71981057616418, 65.58082801176724, 66.54947262682069, 67.4678912247973, 68.54416301930114, 69.53433307024467, 70.69670660830883, 71.83755471048289, 72.76314845375619, 73.8752959747435, 74.8798163162804, 75.84128578603718, 76.88168185405755, 77.81445074262754, 78.78309535768099, 79.91676831455837, 81.07914185262251, 82.06213675826936, 83.22451029633349, 84.58778790270503, 85.6353591160221, 86.94841070531677, 87.99598191863384, 88.92157566190716, 89.78259309751022, 90.8229891655306, 91.90643610533114, 92.83920499390112, 93.99440338666858, 95.06350003587572, 95.94604290736888, 96.7568343258951, 97.89768242806916, 99.11745712850684, 100.0], "rescued": [0, 80, 174, 236, 301, 347, 405, 453, 508, 582, 617, 666, 713, 769, 830, 885, 919, 959, 984, 1017, 1049, 1084, 1123, 1160, 1205, 1213, 1216, 1221, 1226, 1235, 1242, 1246, 1253, 1258, 1266, 1275, 1279, 1284, 1295, 1306, 1313, 1327, 1333, 1341, 1344, 1350, 1355, 1361, 1373, 1380, 1389, 1392, 1403, 1412, 1427, 1435, 1446, 1459, 1467, 1473, 1480, 1490, 1503, 1513, 1519, 1529, 1540, 1551, 1562, 1578, 1593, 1606, 1620, 1641, 1662, 1679, 1695, 1700, 1717, 1732, 1756, 1777, 1814, 1832, 1851, 1873, 1892, 1915, 1938, 1953, 1973, 1991, 2022, 2041, 2069, 2092, 2117, 2139, 2173, 2208, 2225], "harmed": [0, 6, 15, 23, 32, 39, 47, 56, 66, 82, 93, 103, 113, 125, 139, 150, 157, 173, 181, 192, 202, 207, 215, 226, 233, 236, 239, 239, 240, 240, 242, 242, 243, 244, 247, 251, 252, 255, 255, 255, 258, 262, 265, 270, 274, 279, 283, 285, 285, 287, 288, 293, 302, 308, 314, 321, 327, 330, 337, 346, 352, 360, 360, 365, 369, 373, 382, 389, 395, 402, 412, 420, 431, 437, 441, 449, 453, 461, 472, 476, 486, 499, 508, 518, 531, 542, 547, 556, 565, 580, 590, 607, 621, 633, 648, 661, 672, 680, 689, 708, 729], "exact_score": [80.00287005811867, 80.55535624596398, 81.05044127143574, 81.50247542512736, 81.82535696347851, 82.15541364712637, 82.52134605725766, 82.80117672382866, 83.1312334074765, 83.43976465523427, 83.74112075769534, 83.94919997129942, 84.27208150965058, 84.57343761211165, 84.82456769749588, 85.18332496233049, 85.39857932123127, 85.59948338953865, 85.75733658606588, 85.88648920140633, 86.08021812441702, 86.30264762861448, 86.51790198751526, 86.71880605582263, 86.9340604147234, 87.01298701298703, 87.01298701298703, 87.04886273947048, 87.07756332065725, 87.15648991892087, 87.17801535481094, 87.22106622659109, 87.27129224366794, 87.32151826074478, 87.32151826074478, 87.37891942311832, 87.38609456841502, 87.46502116667862, 87.54394776494225, 87.57264834612901, 87.64439979909594, 87.64439979909594, 87.69462581617277, 87.6802755255794, 87.6731003802827, 87.69462581617277, 87.72332639735954, 87.80942814091985, 87.83812872210662, 87.90270502977687, 87.88117959388678, 87.88835473918347, 87.92423046566694, 87.98163162804047, 88.01033220922724, 88.03185764511731, 88.10360909808423, 88.11078424338093, 88.08925880749085, 88.09643395278755, 88.11078424338093, 88.20406113223794, 88.24711200401808, 88.26146229461148, 88.28298773050155, 88.31168831168831, 88.35473918346847, 88.40496520054532, 88.43366578173209, 88.49824208940231, 88.505417234699, 88.55564325177585, 88.67044557652292, 88.77807275597331, 88.84264906364353, 88.91440051661046, 88.90005022601709, 88.94310109779724, 89.03637798665423, 89.12965487551124, 89.20140632847816, 89.3520843797087, 89.4238358326756, 89.47406184975246, 89.531463012126, 89.60321446509292, 89.68931620865322, 89.75389251632346, 89.85434455047715, 89.85434455047715, 89.92609600344407, 89.96914687522423, 90.06242376408122, 90.14135036234484, 90.19875152471838, 90.27050297768531, 90.37813015713569, 90.48575733658608, 90.66513596900337, 90.76558800315708, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.42506878180156, "threshold_area100": 87.4102030566119, "threshold_area25": 84.16804190284853}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1087, "harmed": 383}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2225, "harmed": 729}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.8754732608795166, "operator": "ge"}, {"budget": 0.02, "threshold": 2.2099204063415527, "operator": "ge"}, {"budget": 0.03, "threshold": 1.8700568675994873, "operator": "ge"}, {"budget": 0.04, "threshold": 1.5984039306640625, "operator": "ge"}, {"budget": 0.05, "threshold": 1.4303698539733887, "operator": "ge"}, {"budget": 0.06, "threshold": 1.2865664958953857, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1601613759994507, "operator": "ge"}, {"budget": 0.08, "threshold": 1.062964916229248, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9804728031158447, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9080361127853394, "operator": "ge"}, {"budget": 0.11, "threshold": 0.8090803027153015, "operator": "ge"}, {"budget": 0.12, "threshold": 0.7308468818664551, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6477231979370117, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5894582271575928, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5287015438079834, "operator": "ge"}, {"budget": 0.16, "threshold": 0.46489080786705017, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4095263183116913, "operator": "ge"}, {"budget": 0.18, "threshold": 0.3478749692440033, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2914617657661438, "operator": "ge"}, {"budget": 0.2, "threshold": 0.23418137431144714, "operator": "ge"}, {"budget": 0.21, "threshold": 0.16203323006629944, "operator": "ge"}, {"budget": 0.22, "threshold": 0.10575076937675476, "operator": "ge"}, {"budget": 0.23, "threshold": 0.04510074853897095, "operator": "ge"}, {"budget": 0.24, "threshold": -0.028497546911239624, "operator": "ge"}, {"budget": 0.25, "threshold": -0.09391529858112335, "operator": "ge"}, {"budget": 0.26, "threshold": -0.17026053369045258, "operator": "ge"}, {"budget": 0.27, "threshold": -0.23257918655872345, "operator": "ge"}, {"budget": 0.28, "threshold": -0.2911774516105652, "operator": "ge"}, {"budget": 0.29, "threshold": -0.3496180772781372, "operator": "ge"}, {"budget": 0.3, "threshold": -0.39951467514038086, "operator": "ge"}, {"budget": 0.31, "threshold": -0.46034345030784607, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5301913022994995, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5862065553665161, "operator": "ge"}, {"budget": 0.34, "threshold": -0.6494947671890259, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7052647471427917, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7700599431991577, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8307527899742126, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8841680884361267, "operator": "ge"}, {"budget": 0.39, "threshold": -0.939216673374176, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9869716167449951, "operator": "ge"}, {"budget": 0.41, "threshold": -1.0583646297454834, "operator": "ge"}, {"budget": 0.42, "threshold": -1.1221239566802979, "operator": "ge"}, {"budget": 0.43, "threshold": -1.178920030593872, "operator": "ge"}, {"budget": 0.44, "threshold": -1.2417305707931519, "operator": "ge"}, {"budget": 0.45, "threshold": -1.2969189882278442, "operator": "ge"}, {"budget": 0.46, "threshold": -1.3608403205871582, "operator": "ge"}, {"budget": 0.47, "threshold": -1.4169676303863525, "operator": "ge"}, {"budget": 0.48, "threshold": -1.4746767282485962, "operator": "ge"}, {"budget": 0.49, "threshold": -1.5415781736373901, "operator": "ge"}, {"budget": 0.5, "threshold": -1.5976678133010864, "operator": "ge"}, {"budget": 0.51, "threshold": -1.651874303817749, "operator": "ge"}, {"budget": 0.52, "threshold": -1.7210644483566284, "operator": "ge"}, {"budget": 0.53, "threshold": -1.773832082748413, "operator": "ge"}, {"budget": 0.54, "threshold": -1.8325711488723755, "operator": "ge"}, {"budget": 0.55, "threshold": -1.8866766691207886, "operator": "ge"}, {"budget": 0.56, "threshold": -1.9591575860977173, "operator": "ge"}, {"budget": 0.57, "threshold": -2.020184278488159, "operator": "ge"}, {"budget": 0.58, "threshold": -2.0782546997070312, "operator": "ge"}, {"budget": 0.59, "threshold": -2.132996082305908, "operator": "ge"}, {"budget": 0.6, "threshold": -2.18689227104187, "operator": "ge"}, {"budget": 0.61, "threshold": -2.2503461837768555, "operator": "ge"}, {"budget": 0.62, "threshold": -2.3151192665100098, "operator": "ge"}, {"budget": 0.63, "threshold": -2.386552572250366, "operator": "ge"}, {"budget": 0.64, "threshold": -2.4434680938720703, "operator": "ge"}, {"budget": 0.65, "threshold": -2.5117311477661133, "operator": "ge"}, {"budget": 0.66, "threshold": -2.558736801147461, "operator": "ge"}, {"budget": 0.67, "threshold": -2.6215991973876953, "operator": "ge"}, {"budget": 0.68, "threshold": -2.6788101196289062, "operator": "ge"}, {"budget": 0.69, "threshold": -2.7375755310058594, "operator": "ge"}, {"budget": 0.7, "threshold": -2.7936196327209473, "operator": "ge"}, {"budget": 0.71, "threshold": -2.8609182834625244, "operator": "ge"}, {"budget": 0.72, "threshold": -2.925476551055908, "operator": "ge"}, {"budget": 0.73, "threshold": -2.9770073890686035, "operator": "ge"}, {"budget": 0.74, "threshold": -3.035151958465576, "operator": "ge"}, {"budget": 0.75, "threshold": -3.0997202396392822, "operator": "ge"}, {"budget": 0.76, "threshold": -3.1683542728424072, "operator": "ge"}, {"budget": 0.77, "threshold": -3.245243787765503, "operator": "ge"}, {"budget": 0.78, "threshold": -3.324800968170166, "operator": "ge"}, {"budget": 0.79, "threshold": -3.3930697441101074, "operator": "ge"}, {"budget": 0.8, "threshold": -3.46441388130188, "operator": "ge"}, {"budget": 0.81, "threshold": -3.558900833129883, "operator": "ge"}, {"budget": 0.82, "threshold": -3.6218957901000977, "operator": "ge"}, {"budget": 0.83, "threshold": -3.693636417388916, "operator": "ge"}, {"budget": 0.84, "threshold": -3.795258045196533, "operator": "ge"}, {"budget": 0.85, "threshold": -3.8700599670410156, "operator": "ge"}, {"budget": 0.86, "threshold": -3.9575958251953125, "operator": "ge"}, {"budget": 0.87, "threshold": -4.042234420776367, "operator": "ge"}, {"budget": 0.88, "threshold": -4.138129234313965, "operator": "ge"}, {"budget": 0.89, "threshold": -4.239836692810059, "operator": "ge"}, {"budget": 0.9, "threshold": -4.369413375854492, "operator": "ge"}, {"budget": 0.91, "threshold": -4.460034370422363, "operator": "ge"}, {"budget": 0.92, "threshold": -4.586945056915283, "operator": "ge"}, {"budget": 0.93, "threshold": -4.7400689125061035, "operator": "ge"}, {"budget": 0.94, "threshold": -4.870133399963379, "operator": "ge"}, {"budget": 0.95, "threshold": -5.035138130187988, "operator": "ge"}, {"budget": 0.96, "threshold": -5.233498573303223, "operator": "ge"}, {"budget": 0.97, "threshold": -5.491673946380615, "operator": "ge"}, {"budget": 0.98, "threshold": -5.7441606521606445, "operator": "ge"}, {"budget": 0.99, "threshold": -6.12484073638916, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.034374104841, 80.57863076482383, 80.9653394442853, 81.28043540532798, 81.60985391005443, 81.93927241478086, 82.38327126897737, 82.74133486107132, 83.11372099684904, 83.38584932684044, 83.68662274419937, 83.85849326840446, 84.00171870524204, 84.20223431681467, 84.46004010312231, 84.4313950157548, 84.66055571469494, 84.94700658837009, 85.19048983099398, 85.37668289888283, 85.57719851045546, 85.72042394729303, 85.97822973360068, 86.1071326267545, 86.19306788885706, 86.40790604411343, 86.55113148095101, 86.80893726725866, 86.99513033514752, 87.16700085935261, 87.33887138355772, 87.5107419077628, 87.5107419077628, 87.68261243196791, 87.78287023775422, 87.854482956173, 87.94041821827557, 88.12661128616442, 88.18390146089945, 88.2411916356345, 88.28415926668576, 88.4417072472071, 88.48467487825837, 88.5419650529934, 88.6135777714122, 88.82841592666857, 88.87138355771984, 88.98596390718991, 88.90002864508736, 89.08622171297623, 89.11486680034373, 89.18647951876252, 89.21512460613005, 89.21512460613005, 89.24376969349757, 89.31538241191636, 89.34402749928387, 89.4013176740189, 89.4013176740189, 89.45860784875393, 89.50157547980521, 89.53022056717272, 89.47293039243769, 89.53022056717272, 89.48725293612145, 89.50157547980521, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.60183328559151, 89.60183328559151, 89.70209109137782, 89.73073617874535, 89.74505872242909, 89.74505872242909, 89.75938126611285, 89.78802635348038, 89.77370380979661, 89.81667144084788, 89.87396161558291, 89.90260670295044, 89.88828415926669, 89.93125179031796, 89.88828415926669, 89.88828415926669, 89.9169292466342, 89.88828415926669, 89.85963907189917, 89.90260670295044, 89.93125179031796, 89.95989687768548, 89.9455743340017, 89.97421942136923, 89.9455743340017, 89.93125179031796, 89.9169292466342, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 29, 68, 99, 126, 151, 180, 212, 243, 271, 294, 318, 340, 359, 379, 402, 413, 432, 455, 475, 492, 514, 529, 550, 566, 581, 601, 613, 634, 652, 669, 687, 704, 713, 726, 740, 748, 759, 774, 783, 792, 800, 813, 819, 830, 840, 857, 864, 874, 877, 890, 895, 905, 913, 920, 927, 936, 942, 949, 954, 959, 964, 967, 970, 975, 978, 983, 987, 995, 998, 1002, 1003, 1010, 1013, 1016, 1017, 1021, 1026, 1028, 1033, 1043, 1048, 1050, 1054, 1055, 1057, 1061, 1063, 1064, 1068, 1071, 1075, 1076, 1080, 1080, 1084, 1085, 1087, 1087, 1087, 1087], "harmed": [0, 14, 15, 19, 24, 26, 32, 33, 39, 41, 45, 48, 58, 67, 73, 78, 91, 94, 97, 100, 104, 112, 117, 120, 127, 136, 141, 143, 146, 151, 156, 162, 167, 176, 177, 184, 187, 192, 194, 199, 204, 209, 211, 214, 221, 226, 228, 232, 234, 243, 243, 246, 251, 257, 264, 269, 273, 277, 280, 285, 286, 288, 289, 296, 297, 303, 307, 308, 313, 314, 319, 320, 320, 321, 323, 324, 327, 330, 333, 335, 341, 344, 347, 348, 352, 354, 356, 360, 363, 364, 365, 367, 369, 371, 373, 378, 380, 381, 381, 381, 383], "exact_score": [79.81953594958465, 80.034374104841, 80.57863076482383, 80.9653394442853, 81.28043540532798, 81.60985391005443, 81.93927241478086, 82.38327126897737, 82.74133486107132, 83.11372099684904, 83.38584932684044, 83.68662274419937, 83.85849326840446, 84.00171870524204, 84.20223431681467, 84.46004010312231, 84.4313950157548, 84.66055571469494, 84.94700658837009, 85.19048983099398, 85.37668289888283, 85.57719851045546, 85.72042394729303, 85.97822973360068, 86.1071326267545, 86.19306788885706, 86.40790604411343, 86.55113148095101, 86.80893726725866, 86.99513033514752, 87.16700085935261, 87.33887138355772, 87.5107419077628, 87.5107419077628, 87.68261243196791, 87.78287023775422, 87.854482956173, 87.94041821827557, 88.12661128616442, 88.18390146089945, 88.2411916356345, 88.28415926668576, 88.4417072472071, 88.48467487825837, 88.5419650529934, 88.6135777714122, 88.82841592666857, 88.87138355771984, 88.98596390718991, 88.90002864508736, 89.08622171297623, 89.11486680034373, 89.18647951876252, 89.21512460613005, 89.21512460613005, 89.24376969349757, 89.31538241191636, 89.34402749928387, 89.4013176740189, 89.4013176740189, 89.45860784875393, 89.50157547980521, 89.53022056717272, 89.47293039243769, 89.53022056717272, 89.48725293612145, 89.50157547980521, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.60183328559151, 89.60183328559151, 89.70209109137782, 89.73073617874535, 89.74505872242909, 89.74505872242909, 89.75938126611285, 89.78802635348038, 89.77370380979661, 89.81667144084788, 89.87396161558291, 89.90260670295044, 89.88828415926669, 89.93125179031796, 89.88828415926669, 89.88828415926669, 89.9169292466342, 89.88828415926669, 89.85963907189917, 89.90260670295044, 89.93125179031796, 89.95989687768548, 89.9455743340017, 89.97421942136923, 89.9455743340017, 89.93125179031796, 89.9169292466342, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.70770560235054, "threshold_area100": 87.70696075623033, "threshold_area25": 83.56774563162416}, "test": {"score": [80.00287005811867, 80.62710769893091, 81.17241874147952, 81.60292745928105, 81.99038530530244, 82.21998995479659, 82.60744780081797, 82.98055535624597, 83.2173351510368, 83.54021668938796, 83.77699648417881, 84.03530171485973, 84.32948267202411, 84.59496304800173, 84.85326827868265, 85.1761498170338, 85.4416301930114, 85.65688455191219, 85.76451173136256, 85.92954007318649, 86.08739326971371, 86.40309966276818, 86.53225227810863, 86.68293032933917, 86.99863672239363, 87.18519050010764, 87.4004448590084, 87.53677261964555, 87.62287436320587, 87.78790270502978, 87.82377843151325, 87.96728133744709, 88.15383511516109, 88.21123627753462, 88.36191432876515, 88.48389179880894, 88.64174499533615, 88.84982420894023, 88.971801678984, 89.03637798665423, 89.10812943962115, 89.20858147377486, 89.3377340891153, 89.48123699504916, 89.531463012126, 89.57451388390615, 89.65344048216977, 89.72519193513669, 89.77541795221353, 89.83999425988375, 89.88304513166392, 89.91892085814737, 89.96914687522423, 90.04807347348785, 90.0839491999713, 90.05524861878453, 90.15570065293824, 90.16287579823492, 90.19157637942169, 90.23462725120184, 90.29202841357538, 90.30637870416876, 90.34225443065222, 90.39248044772907, 90.39248044772907, 90.40683073832245, 90.40683073832245, 90.44270646480591, 90.47140704599268, 90.48575733658608, 90.50728277247615, 90.50010762717945, 90.50010762717945, 90.49293248188276, 90.50010762717945, 90.54315849895961, 90.62926024251992, 90.62208509722322, 90.6364353878166, 90.6364353878166, 90.68666140489346, 90.72253713137691, 90.73688742197031, 90.744062567267, 90.75841285786038, 90.76558800315708, 90.76558800315708, 90.77993829375045, 90.77993829375045, 90.77276314845376, 90.76558800315708, 90.77276314845376, 90.80863887493722, 90.80863887493722, 90.82298916553061, 90.78711343904715, 90.75841285786038, 90.73688742197031, 90.72253713137691, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.0906220850972232, 2.3247470761282916, 3.3220922723685153, 4.51316639161943, 5.3239578101456555, 6.457630767023033, 7.49802683504341, 8.409270287723327, 9.428140919853627, 10.203056611896391, 11.386955585850615, 12.36277534620076, 13.532324029561599, 14.52249408050513, 15.584415584415584, 16.61763650713927, 17.449953361555572, 18.318145942455335, 19.215039104541866, 20.126282557221785, 21.2240797876157, 22.013345770251846, 22.97481524000861, 24.165889359259523, 25.177584846093133, 26.28255722178374, 27.251201836837197, 28.155270144220417, 28.96606156274665, 29.719451818899334, 30.709621869842863, 31.95092200617062, 32.81911458707039, 34.06758986869484, 34.935782449594605, 36.019229389395136, 37.10985147449236, 38.078496089545816, 38.97538925163235, 39.829231541938725, 40.912678481739256, 42.00330056683648, 43.04369663485686, 44.08409270287723, 45.09578818971084, 46.14335940302791, 47.12635430867475, 48.130874650211666, 49.14974528234197, 50.21884193154912, 51.18748654660257, 52.407261247040246, 53.31132955442348, 54.236923297696784, 55.13381645978331, 56.30336514314415, 57.35811150175791, 58.26935495443782, 59.237999569491286, 60.12771758628113, 61.17528879959819, 62.29461146588219, 63.52156131161656, 64.49738107196671, 65.59517830236062, 66.35574370381, 67.475066370094, 68.35043409629044, 69.26167754897037, 70.18009614694698, 71.25636794145082, 72.26088828298774, 72.9497022314702, 73.88247112004018, 74.72196311975317, 75.67625744421325, 76.82428069168401, 78.01535481093492, 79.07727631484538, 79.8808925880749, 81.29439621152328, 81.97603501470905, 82.77965128793858, 83.99942598837626, 84.87479371457272, 85.9080146372964, 86.79055750878956, 87.665925234986, 88.85699935423692, 90.10547463586137, 90.75123771256368, 91.82750950706752, 92.9683576092416, 93.80067446365788, 94.76931907871135, 95.77383942024827, 96.98643897538926, 97.91203271866256, 98.99547965846308, 100.0], "rescued": [0, 93, 179, 246, 305, 344, 410, 470, 511, 564, 604, 658, 704, 754, 801, 856, 906, 944, 974, 1005, 1042, 1096, 1122, 1155, 1209, 1240, 1280, 1313, 1337, 1364, 1382, 1411, 1449, 1463, 1499, 1524, 1556, 1590, 1612, 1631, 1648, 1668, 1695, 1722, 1743, 1758, 1777, 1792, 1806, 1824, 1835, 1849, 1864, 1880, 1895, 1901, 1918, 1926, 1934, 1949, 1960, 1967, 1976, 1988, 1995, 2003, 2009, 2018, 2026, 2031, 2040, 2048, 2054, 2056, 2061, 2071, 2085, 2094, 2102, 2109, 2119, 2131, 2135, 2139, 2150, 2156, 2164, 2172, 2176, 2182, 2185, 2190, 2198, 2200, 2205, 2210, 2211, 2213, 2215, 2219, 2225], "harmed": [0, 6, 16, 23, 28, 35, 47, 55, 63, 71, 78, 96, 101, 114, 125, 135, 148, 156, 171, 179, 194, 204, 212, 224, 234, 239, 249, 263, 275, 279, 292, 301, 313, 319, 334, 342, 352, 357, 362, 372, 379, 385, 394, 401, 415, 424, 432, 437, 444, 453, 458, 467, 475, 480, 490, 500, 503, 510, 514, 523, 526, 531, 535, 540, 547, 553, 559, 563, 567, 570, 576, 585, 591, 594, 598, 602, 604, 614, 620, 627, 630, 637, 639, 642, 651, 656, 664, 670, 674, 681, 685, 689, 692, 694, 697, 707, 712, 717, 721, 726, 729], "exact_score": [80.00287005811867, 80.58405682715075, 81.06479156202914, 81.48812513453397, 81.81100667288513, 82.13388821123628, 82.42806916840067, 82.78682643323528, 83.09535768099305, 83.4110640740475, 83.69806988591519, 83.97072540718949, 84.20033005668365, 84.45863528736457, 84.74564109923226, 85.03264691109995, 85.33400301356103, 85.55643251775848, 85.72146085958241, 85.90083949199972, 86.07304297912033, 86.27394704742771, 86.53942742340533, 86.69010547463587, 86.95558585061349, 87.15648991892087, 87.32151826074478, 87.50807203845878, 87.5941737820191, 87.78072755973308, 87.84530386740332, 88.00315706393054, 88.16818540575447, 88.20406113223794, 88.34038889287508, 88.49824208940231, 88.64174499533615, 88.80677333716008, 88.96462653368731, 89.03637798665423, 89.15118031140132, 89.23010690966493, 89.3377340891153, 89.46688670445577, 89.54581330271938, 89.54581330271938, 89.64626533687307, 89.73236708043338, 89.78259309751023, 89.82564396929038, 89.86869484107054, 89.8902202769606, 89.93327114874077, 90.01937289230108, 90.07677405467462, 90.05524861878453, 90.14852550764154, 90.15570065293824, 90.1700509435316, 90.22745210590514, 90.28485326827868, 90.30637870416876, 90.33507928535553, 90.3566047212456, 90.42835617421254, 90.39965559302576, 90.41400588361914, 90.41400588361914, 90.45705675539931, 90.49293248188276, 90.51445791777284, 90.49293248188276, 90.52880820836621, 90.49293248188276, 90.50010762717945, 90.57903422544307, 90.64361053311329, 90.64361053311329, 90.6364353878166, 90.64361053311329, 90.70101169548683, 90.71536198608023, 90.73688742197031, 90.744062567267, 90.75841285786038, 90.75123771256368, 90.75841285786038, 90.77276314845376, 90.79428858434385, 90.77276314845376, 90.75123771256368, 90.78711343904715, 90.80863887493722, 90.80863887493722, 90.81581402023392, 90.77993829375045, 90.75123771256368, 90.73688742197031, 90.72253713137691, 90.71536198608023, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.47788659809765, "threshold_area100": 88.50283418239219, "threshold_area25": 84.17091196096722}, "selected_budget": 93, "selected": {"val": {"score": 89.97421942136923, "calls": 92.99627613864222, "rescued": 1080, "harmed": 371}, "test": {"score": 90.80863887493722, "calls": 92.9683576092416, "rescued": 2200, "harmed": 694}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.3592958450317383, "operator": "ge"}, {"budget": 0.02, "threshold": 1.7863857746124268, "operator": "ge"}, {"budget": 0.03, "threshold": 1.4874036312103271, "operator": "ge"}, {"budget": 0.04, "threshold": 1.2884865999221802, "operator": "ge"}, {"budget": 0.05, "threshold": 1.1148192882537842, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0134971141815186, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9436076879501343, "operator": "ge"}, {"budget": 0.08, "threshold": 0.8620121479034424, "operator": "ge"}, {"budget": 0.09, "threshold": 0.7782890796661377, "operator": "ge"}, {"budget": 0.1, "threshold": 0.7130517363548279, "operator": "ge"}, {"budget": 0.11, "threshold": 0.6276369094848633, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5687268972396851, "operator": "ge"}, {"budget": 0.13, "threshold": 0.518702507019043, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4655158519744873, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4115159511566162, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3654188811779022, "operator": "ge"}, {"budget": 0.17, "threshold": 0.32755398750305176, "operator": "ge"}, {"budget": 0.18, "threshold": 0.28623515367507935, "operator": "ge"}, {"budget": 0.19, "threshold": 0.22460559010505676, "operator": "ge"}, {"budget": 0.2, "threshold": 0.18185067176818848, "operator": "ge"}, {"budget": 0.21, "threshold": 0.13531392812728882, "operator": "ge"}, {"budget": 0.22, "threshold": 0.0875135064125061, "operator": "ge"}, {"budget": 0.23, "threshold": 0.03419625759124756, "operator": "ge"}, {"budget": 0.24, "threshold": -0.026089608669281006, "operator": "ge"}, {"budget": 0.25, "threshold": -0.07837986946105957, "operator": "ge"}, {"budget": 0.26, "threshold": -0.1396370232105255, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1973215937614441, "operator": "ge"}, {"budget": 0.28, "threshold": -0.24926108121871948, "operator": "ge"}, {"budget": 0.29, "threshold": -0.3068573474884033, "operator": "ge"}, {"budget": 0.3, "threshold": -0.3524671494960785, "operator": "ge"}, {"budget": 0.31, "threshold": -0.41147106885910034, "operator": "ge"}, {"budget": 0.32, "threshold": -0.4769911766052246, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5285386443138123, "operator": "ge"}, {"budget": 0.34, "threshold": -0.58934485912323, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6450202465057373, "operator": "ge"}, {"budget": 0.36, "threshold": -0.698668897151947, "operator": "ge"}, {"budget": 0.37, "threshold": -0.7621358633041382, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8235288858413696, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8842954635620117, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9448620080947876, "operator": "ge"}, {"budget": 0.41, "threshold": -0.9913381338119507, "operator": "ge"}, {"budget": 0.42, "threshold": -1.0664384365081787, "operator": "ge"}, {"budget": 0.43, "threshold": -1.1220054626464844, "operator": "ge"}, {"budget": 0.44, "threshold": -1.1777971982955933, "operator": "ge"}, {"budget": 0.45, "threshold": -1.2441248893737793, "operator": "ge"}, {"budget": 0.46, "threshold": -1.2942652702331543, "operator": "ge"}, {"budget": 0.47, "threshold": -1.3656883239746094, "operator": "ge"}, {"budget": 0.48, "threshold": -1.422806978225708, "operator": "ge"}, {"budget": 0.49, "threshold": -1.4867920875549316, "operator": "ge"}, {"budget": 0.5, "threshold": -1.5448534488677979, "operator": "ge"}, {"budget": 0.51, "threshold": -1.5980257987976074, "operator": "ge"}, {"budget": 0.52, "threshold": -1.6562604904174805, "operator": "ge"}, {"budget": 0.53, "threshold": -1.716688871383667, "operator": "ge"}, {"budget": 0.54, "threshold": -1.778796672821045, "operator": "ge"}, {"budget": 0.55, "threshold": -1.841623067855835, "operator": "ge"}, {"budget": 0.56, "threshold": -1.9038619995117188, "operator": "ge"}, {"budget": 0.57, "threshold": -1.9713919162750244, "operator": "ge"}, {"budget": 0.58, "threshold": -2.0368025302886963, "operator": "ge"}, {"budget": 0.59, "threshold": -2.080667495727539, "operator": "ge"}, {"budget": 0.6, "threshold": -2.1408066749572754, "operator": "ge"}, {"budget": 0.61, "threshold": -2.2115426063537598, "operator": "ge"}, {"budget": 0.62, "threshold": -2.2754976749420166, "operator": "ge"}, {"budget": 0.63, "threshold": -2.3416099548339844, "operator": "ge"}, {"budget": 0.64, "threshold": -2.4072985649108887, "operator": "ge"}, {"budget": 0.65, "threshold": -2.4626965522766113, "operator": "ge"}, {"budget": 0.66, "threshold": -2.5243749618530273, "operator": "ge"}, {"budget": 0.67, "threshold": -2.5878212451934814, "operator": "ge"}, {"budget": 0.68, "threshold": -2.6478381156921387, "operator": "ge"}, {"budget": 0.69, "threshold": -2.706932783126831, "operator": "ge"}, {"budget": 0.7, "threshold": -2.7680883407592773, "operator": "ge"}, {"budget": 0.71, "threshold": -2.8274378776550293, "operator": "ge"}, {"budget": 0.72, "threshold": -2.89094877243042, "operator": "ge"}, {"budget": 0.73, "threshold": -2.9564666748046875, "operator": "ge"}, {"budget": 0.74, "threshold": -3.005929708480835, "operator": "ge"}, {"budget": 0.75, "threshold": -3.077922821044922, "operator": "ge"}, {"budget": 0.76, "threshold": -3.1363613605499268, "operator": "ge"}, {"budget": 0.77, "threshold": -3.214510440826416, "operator": "ge"}, {"budget": 0.78, "threshold": -3.2917022705078125, "operator": "ge"}, {"budget": 0.79, "threshold": -3.3498268127441406, "operator": "ge"}, {"budget": 0.8, "threshold": -3.42606520652771, "operator": "ge"}, {"budget": 0.81, "threshold": -3.5185279846191406, "operator": "ge"}, {"budget": 0.82, "threshold": -3.5935254096984863, "operator": "ge"}, {"budget": 0.83, "threshold": -3.6608574390411377, "operator": "ge"}, {"budget": 0.84, "threshold": -3.7613892555236816, "operator": "ge"}, {"budget": 0.85, "threshold": -3.828963279724121, "operator": "ge"}, {"budget": 0.86, "threshold": -3.926206111907959, "operator": "ge"}, {"budget": 0.87, "threshold": -4.0100226402282715, "operator": "ge"}, {"budget": 0.88, "threshold": -4.09282112121582, "operator": "ge"}, {"budget": 0.89, "threshold": -4.207335472106934, "operator": "ge"}, {"budget": 0.9, "threshold": -4.3209967613220215, "operator": "ge"}, {"budget": 0.91, "threshold": -4.427523136138916, "operator": "ge"}, {"budget": 0.92, "threshold": -4.548720359802246, "operator": "ge"}, {"budget": 0.93, "threshold": -4.69340705871582, "operator": "ge"}, {"budget": 0.94, "threshold": -4.846169948577881, "operator": "ge"}, {"budget": 0.95, "threshold": -4.999330520629883, "operator": "ge"}, {"budget": 0.96, "threshold": -5.204551696777344, "operator": "ge"}, {"budget": 0.97, "threshold": -5.450430870056152, "operator": "ge"}, {"budget": 0.98, "threshold": -5.7164435386657715, "operator": "ge"}, {"budget": 0.99, "threshold": -6.093266487121582, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.1059868232598, 80.60727585219134, 80.93669435691778, 81.32340303637926, 81.63849899742193, 82.08249785161844, 82.38327126897737, 82.69836723002004, 82.94185047264394, 83.34288169578916, 83.60068748209682, 83.87281581208822, 84.0160412489258, 84.20223431681467, 84.41707247207103, 84.54597536522486, 84.7035233457462, 84.91836150100256, 85.17616728731022, 85.3910054425666, 85.56287596677168, 85.74906903466055, 85.97822973360068, 86.12145517043827, 86.1787453451733, 86.42222858779719, 86.60842165568604, 86.78029217989113, 87.08106559725006, 87.2099684904039, 87.32454883987396, 87.39616155829275, 87.52506445144657, 87.66828988828415, 87.7542251503867, 87.86880549985678, 87.96906330564308, 88.09796619879691, 88.14093382984818, 88.25551417931824, 88.31280435405327, 88.4417072472071, 88.5133199656259, 88.59925522772843, 88.62790031509596, 88.75680320824978, 88.90002864508736, 88.97164136350615, 88.97164136350615, 89.0575766256087, 89.1291893440275, 89.20080206244629, 89.20080206244629, 89.22944714981381, 89.24376969349757, 89.31538241191636, 89.34402749928387, 89.34402749928387, 89.45860784875393, 89.48725293612145, 89.53022056717272, 89.50157547980521, 89.48725293612145, 89.51589802348897, 89.50157547980521, 89.53022056717272, 89.53022056717272, 89.53022056717272, 89.58751074190776, 89.61615582927529, 89.61615582927529, 89.68776854769406, 89.75938126611285, 89.74505872242909, 89.75938126611285, 89.75938126611285, 89.74505872242909, 89.78802635348038, 89.83099398453164, 89.93125179031796, 89.9169292466342, 89.9455743340017, 89.93125179031796, 89.90260670295044, 89.88828415926669, 89.90260670295044, 89.88828415926669, 89.87396161558291, 89.88828415926669, 89.9455743340017, 89.97421942136923, 89.98854196505299, 89.97421942136923, 89.93125179031796, 89.93125179031796, 89.90260670295044, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 72, 98, 128, 154, 187, 211, 238, 260, 291, 312, 336, 353, 377, 397, 415, 434, 451, 473, 493, 512, 530, 550, 567, 581, 602, 618, 633, 656, 673, 687, 700, 714, 727, 738, 753, 764, 775, 781, 796, 802, 814, 823, 834, 843, 855, 867, 875, 881, 890, 898, 906, 914, 924, 931, 939, 944, 947, 957, 962, 966, 968, 971, 976, 979, 986, 987, 993, 999, 1002, 1004, 1009, 1015, 1016, 1018, 1022, 1025, 1029, 1035, 1046, 1049, 1053, 1054, 1055, 1058, 1061, 1063, 1064, 1067, 1072, 1076, 1078, 1080, 1080, 1084, 1085, 1087, 1087, 1087, 1087], "harmed": [0, 12, 17, 20, 23, 27, 29, 32, 37, 42, 45, 48, 53, 60, 71, 76, 85, 93, 95, 99, 104, 111, 116, 120, 127, 137, 141, 144, 147, 149, 157, 163, 171, 176, 179, 184, 191, 195, 197, 200, 207, 209, 212, 216, 221, 228, 231, 233, 236, 242, 245, 248, 251, 259, 267, 273, 276, 279, 282, 284, 287, 288, 292, 296, 299, 303, 308, 309, 315, 317, 318, 320, 320, 321, 323, 324, 328, 332, 333, 336, 340, 344, 346, 348, 351, 355, 357, 360, 362, 364, 365, 367, 368, 371, 374, 378, 381, 381, 381, 381, 383], "exact_score": [79.81953594958465, 80.1059868232598, 80.60727585219134, 80.93669435691778, 81.32340303637926, 81.63849899742193, 82.08249785161844, 82.38327126897737, 82.69836723002004, 82.94185047264394, 83.34288169578916, 83.60068748209682, 83.87281581208822, 84.0160412489258, 84.20223431681467, 84.41707247207103, 84.54597536522486, 84.7035233457462, 84.91836150100256, 85.17616728731022, 85.3910054425666, 85.56287596677168, 85.74906903466055, 85.97822973360068, 86.12145517043827, 86.1787453451733, 86.42222858779719, 86.60842165568604, 86.78029217989113, 87.08106559725006, 87.2099684904039, 87.32454883987396, 87.39616155829275, 87.52506445144657, 87.66828988828415, 87.7542251503867, 87.86880549985678, 87.96906330564308, 88.09796619879691, 88.14093382984818, 88.25551417931824, 88.31280435405327, 88.4417072472071, 88.5133199656259, 88.59925522772843, 88.62790031509596, 88.75680320824978, 88.90002864508736, 88.97164136350615, 88.97164136350615, 89.0575766256087, 89.1291893440275, 89.20080206244629, 89.20080206244629, 89.22944714981381, 89.24376969349757, 89.31538241191636, 89.34402749928387, 89.34402749928387, 89.45860784875393, 89.48725293612145, 89.53022056717272, 89.50157547980521, 89.48725293612145, 89.51589802348897, 89.50157547980521, 89.53022056717272, 89.53022056717272, 89.53022056717272, 89.58751074190776, 89.61615582927529, 89.61615582927529, 89.68776854769406, 89.75938126611285, 89.74505872242909, 89.75938126611285, 89.75938126611285, 89.74505872242909, 89.78802635348038, 89.83099398453164, 89.93125179031796, 89.9169292466342, 89.9455743340017, 89.93125179031796, 89.90260670295044, 89.88828415926669, 89.90260670295044, 89.88828415926669, 89.87396161558291, 89.88828415926669, 89.9455743340017, 89.97421942136923, 89.98854196505299, 89.97421942136923, 89.93125179031796, 89.93125179031796, 89.90260670295044, 89.93125179031796, 89.93125179031796, 89.93125179031796, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.7116995858155, "threshold_area100": 87.71097106846176, "threshold_area25": 83.55771985104555}, "test": {"score": [80.00287005811867, 80.59840711774415, 81.16524359618282, 81.59575231398436, 81.91145870703882, 82.35631771543373, 82.57157207433451, 82.84422759560881, 83.23886058692689, 83.64784386883835, 83.74829590299204, 84.19315491138696, 84.38688383439765, 84.60931333859511, 84.8173925521992, 85.16179952644042, 85.4488053383081, 85.59230824424195, 85.77168687665926, 85.99411638085671, 86.13761928679055, 86.36004879098802, 86.5609528592954, 86.70445576522926, 87.01298701298703, 87.17801535481094, 87.33586855133817, 87.50807203845878, 87.60134892731578, 87.73050154265624, 87.83812872210662, 87.97445648274378, 88.14665996986439, 88.21123627753462, 88.31886345698501, 88.51259237999571, 88.68479586711632, 88.83547391834685, 88.97897682428069, 89.05790342254431, 89.18705603788477, 89.22293176436823, 89.35925952500538, 89.52428786682931, 89.58168902920285, 89.63191504627969, 89.70366649924661, 89.78976824280693, 89.82564396929038, 89.84716940518047, 89.904570567554, 89.92609600344407, 89.94762143933414, 90.06242376408122, 90.06242376408122, 90.09112434526799, 90.14852550764154, 90.13417521704815, 90.22745210590514, 90.22027696060846, 90.27050297768531, 90.34225443065222, 90.34942957594892, 90.37095501183899, 90.42118102891584, 90.40683073832245, 90.39965559302576, 90.43553131950922, 90.44988161010261, 90.47858219128938, 90.48575733658608, 90.47858219128938, 90.50728277247615, 90.51445791777284, 90.52880820836621, 90.59338451603645, 90.64361053311329, 90.66513596900337, 90.62926024251992, 90.64361053311329, 90.71536198608023, 90.75123771256368, 90.75841285786038, 90.75123771256368, 90.75841285786038, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.78711343904715, 90.78711343904715, 90.75123771256368, 90.77993829375045, 90.81581402023392, 90.80146372964053, 90.8301643108273, 90.78711343904715, 90.75841285786038, 90.744062567267, 90.72253713137691, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.0403960680203774, 2.3534476573150607, 3.444069742412284, 4.326612613905432, 5.539212169046423, 6.428930185836264, 7.211021023175719, 8.194015928822559, 9.385090048073472, 10.303508646050084, 11.803114013058764, 12.77175862811222, 13.56819975604506, 14.493793499318361, 15.706393054459353, 16.6822128148095, 17.435603070962184, 18.26791992537849, 19.41594317284925, 20.2052091554854, 21.10210231757193, 22.013345770251846, 23.003515821195382, 24.21611537633637, 25.127358829016288, 26.167754897036666, 27.179450383870275, 28.076343545956806, 28.958886417449953, 29.67640094711918, 30.774198177513096, 31.857645117313627, 32.82628973236708, 34.01736385161799, 34.87838128722107, 35.782449594604294, 36.82284566262467, 38.006744636578894, 39.0184401234125, 40.05166104613618, 40.79070101169548, 42.06070172921002, 43.14414866901055, 44.00516610461362, 45.06708760852407, 45.8778790270503, 47.12635430867475, 48.11652435961828, 49.12821984645189, 50.254717658032575, 51.10855994833895, 52.141780871062636, 53.23957810145655, 54.18669728061993, 55.16969218626677, 56.17421252780369, 57.228958886417445, 58.26217980914114, 58.95099375762359, 59.88376264619358, 61.146588218411424, 62.23003515821195, 63.342182679199254, 64.61218339671379, 65.45167539642678, 66.36291884910669, 67.48941666068737, 68.46523642103752, 69.29037813015714, 70.35229963406759, 71.27071823204419, 72.21783741120757, 73.2223577527445, 73.91117170122695, 74.85829088039033, 75.7121331706967, 76.77405467460716, 77.97230393915477, 78.84049652005453, 79.83066657099806, 81.15806845088613, 82.01908588648921, 82.7653009973452, 83.97790055248619, 84.77434168041903, 85.92236492788979, 86.8479586711631, 87.60134892731578, 88.76372246537993, 89.96914687522423, 90.8229891655306, 91.7629331993973, 92.81767955801105, 93.85090048073474, 94.74779364282126, 95.80254000143503, 96.97926383009255, 97.94790844514601, 98.9883045131664, 100.0], "rescued": [0, 87, 178, 244, 293, 363, 403, 448, 506, 572, 606, 680, 715, 755, 794, 852, 905, 935, 970, 1015, 1047, 1088, 1125, 1156, 1211, 1241, 1274, 1311, 1332, 1358, 1384, 1418, 1451, 1469, 1498, 1530, 1557, 1589, 1617, 1635, 1658, 1673, 1702, 1731, 1747, 1764, 1781, 1806, 1818, 1829, 1843, 1852, 1864, 1885, 1895, 1909, 1921, 1925, 1942, 1947, 1957, 1972, 1979, 1988, 1999, 2004, 2010, 2019, 2028, 2033, 2042, 2047, 2056, 2060, 2066, 2078, 2087, 2098, 2104, 2110, 2123, 2133, 2139, 2142, 2152, 2156, 2166, 2172, 2177, 2183, 2183, 2191, 2198, 2199, 2207, 2210, 2211, 2213, 2215, 2219, 2225], "harmed": [0, 4, 16, 22, 27, 35, 45, 52, 55, 64, 84, 96, 104, 113, 123, 133, 146, 156, 166, 180, 192, 202, 211, 222, 234, 241, 252, 265, 273, 281, 292, 307, 316, 325, 339, 344, 347, 358, 366, 373, 378, 388, 398, 404, 412, 422, 429, 442, 449, 457, 463, 469, 478, 483, 493, 503, 507, 513, 517, 523, 526, 531, 537, 543, 547, 554, 561, 565, 572, 573, 581, 587, 592, 595, 599, 602, 604, 612, 623, 627, 630, 635, 640, 644, 653, 656, 665, 670, 674, 680, 685, 689, 691, 694, 698, 707, 712, 716, 721, 726, 729], "exact_score": [80.00287005811867, 80.59123197244745, 81.0073903996556, 81.43789911745714, 81.77513094640166, 82.1267130659396, 82.50699576666428, 82.75812585204851, 83.15275884336658, 83.46846523642104, 83.71242017650857, 83.94919997129942, 84.22903063787042, 84.4371098514745, 84.68823993685872, 84.97524574872641, 85.24072612470403, 85.5492573724618, 85.69276027839564, 85.87213891081295, 86.11609385090048, 86.33134820980125, 86.5609528592954, 86.69728061993256, 86.95558585061349, 87.14931477362417, 87.3071679701514, 87.50807203845878, 87.5941737820191, 87.72332639735954, 87.88835473918347, 87.99598191863386, 88.15383511516109, 88.22558656812801, 88.31168831168831, 88.53411781588576, 88.74219702948984, 88.84264906364353, 88.97897682428069, 89.05790342254431, 89.17270574729139, 89.251632345555, 89.344909234412, 89.50993757623593, 89.58168902920285, 89.63191504627969, 89.71801678984, 89.79694338810361, 89.83281911458707, 89.84716940518047, 89.8902202769606, 89.93327114874077, 89.94762143933414, 90.04089832819116, 90.05524861878453, 90.09112434526799, 90.13417521704815, 90.13417521704815, 90.20592667001507, 90.22027696060846, 90.277678122982, 90.32790414005885, 90.34942957594892, 90.38530530243239, 90.40683073832245, 90.42118102891584, 90.43553131950922, 90.42835617421254, 90.44988161010261, 90.46423190069599, 90.48575733658608, 90.48575733658608, 90.50010762717945, 90.51445791777284, 90.53598335366291, 90.58620937073975, 90.66513596900337, 90.65796082370669, 90.62926024251992, 90.65078567840999, 90.71536198608023, 90.75841285786038, 90.75841285786038, 90.75123771256368, 90.75841285786038, 90.79428858434385, 90.77276314845376, 90.78711343904715, 90.80146372964053, 90.77993829375045, 90.75123771256368, 90.79428858434385, 90.81581402023392, 90.80146372964053, 90.81581402023392, 90.77276314845376, 90.75123771256368, 90.744062567267, 90.72253713137691, 90.71536198608023, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.48613824686102, "threshold_area100": 88.51058333931263, "threshold_area25": 84.1988950276243}, "selected_budget": 92, "selected": {"val": {"score": 89.98854196505299, "calls": 91.99369808077915, "rescued": 1078, "harmed": 368}, "test": {"score": 90.81581402023392, "calls": 91.7629331993973, "rescued": 2198, "harmed": 691}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.83099398453164, "calls": 98.99742194213692}, "test": {"score": 90.75123771256368, "calls": 99.14615770969361}}, "old_normalized_interior": {"val": {"score": 90.00286450873675, "calls": 84.98997421942137}, "test": {"score": 90.744062567267, "calls": 84.4227595608811}}, "old_normalized_endpoints": {"val": {"score": 90.00286450873675, "calls": 84.98997421942137}, "test": {"score": 90.744062567267, "calls": 84.4227595608811}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.13463191062732, 80.46405041535377, 80.95101690060154, 81.39501575479805, 81.60985391005443, 81.95359495846462, 82.2686909195073, 82.52649670581495, 82.79862503580635, 83.02778573474649, 83.24262389000285, 83.60068748209682, 83.82984818103695, 84.0160412489258, 84.24520194786594, 84.31681466628473, 84.60326553995989, 84.78945860784874, 84.98997421942136, 85.20481237467774, 85.34803781151533, 85.5915210541392, 85.73474649097679, 86.078487539387, 86.15010025780579, 86.32197078201088, 86.36493841306215, 86.57977656831852, 86.73732454883987, 86.82325981094242, 87.03809796619879, 87.16700085935261, 87.35319392724148, 87.4820968203953, 87.62532225723288, 87.79719278143799, 87.7542251503867, 87.84016041248925, 87.9977083930106, 88.04067602406187, 88.0263534803781, 88.09796619879691, 88.18390146089945, 88.2411916356345, 88.38441707247206, 88.41306215983958, 88.6135777714122, 88.68519048983099, 88.77112575193354, 88.82841592666857, 88.92867373245488, 88.95731881982239, 89.0575766256087, 89.17215697507876, 89.14351188771126, 89.20080206244629, 89.24376969349757, 89.30105986823258, 89.35835004296763, 89.38699513033515, 89.50157547980521, 89.573188198224, 89.61615582927529, 89.61615582927529, 89.61615582927529, 89.64480091664278, 89.70209109137782, 89.73073617874535, 89.74505872242909, 89.75938126611285, 89.77370380979661, 89.8453165282154, 89.87396161558291, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.9455743340017, 89.88828415926669, 89.93125179031796, 89.90260670295044, 89.95989687768548, 89.9455743340017, 89.95989687768548, 89.97421942136923, 90.00286450873675, 89.98854196505299, 90.00286450873675, 89.98854196505299, 89.95989687768548, 89.95989687768548, 89.95989687768548, 89.93125179031796, 89.9455743340017, 89.95989687768548, 89.95989687768548, 89.9455743340017, 89.93125179031796, 89.9169292466342, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.13463191062732, 80.46405041535377, 80.95101690060154, 81.39501575479805, 81.60985391005443, 81.95359495846462, 82.2686909195073, 82.52649670581495, 82.79862503580635, 83.02778573474649, 83.24262389000285, 83.60068748209682, 83.82984818103695, 84.0160412489258, 84.24520194786594, 84.31681466628473, 84.60326553995989, 84.78945860784874, 84.98997421942136, 85.20481237467774, 85.34803781151533, 85.5915210541392, 85.73474649097679, 86.078487539387, 86.15010025780579, 86.32197078201088, 86.36493841306215, 86.57977656831852, 86.73732454883987, 86.82325981094242, 87.03809796619879, 87.16700085935261, 87.35319392724148, 87.4820968203953, 87.62532225723288, 87.79719278143799, 87.7542251503867, 87.84016041248925, 87.9977083930106, 88.04067602406187, 88.0263534803781, 88.09796619879691, 88.18390146089945, 88.2411916356345, 88.38441707247206, 88.41306215983958, 88.6135777714122, 88.68519048983099, 88.77112575193354, 88.82841592666857, 88.92867373245488, 88.95731881982239, 89.0575766256087, 89.17215697507876, 89.14351188771126, 89.20080206244629, 89.24376969349757, 89.30105986823258, 89.35835004296763, 89.38699513033515, 89.50157547980521, 89.573188198224, 89.61615582927529, 89.61615582927529, 89.61615582927529, 89.64480091664278, 89.70209109137782, 89.73073617874535, 89.74505872242909, 89.75938126611285, 89.77370380979661, 89.8453165282154, 89.87396161558291, 89.93125179031796, 89.9455743340017, 89.9455743340017, 89.9455743340017, 89.88828415926669, 89.93125179031796, 89.90260670295044, 89.95989687768548, 89.9455743340017, 89.95989687768548, 89.97421942136923, 90.00286450873675, 89.98854196505299, 90.00286450873675, 89.98854196505299, 89.95989687768548, 89.95989687768548, 89.95989687768548, 89.93125179031796, 89.9455743340017, 89.95989687768548, 89.95989687768548, 89.9455743340017, 89.93125179031796, 89.9169292466342, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 83.41449441420797, "exact_area100": 87.63406409623435}, "test": {"score": [80.00287005811867, 80.49795508359044, 81.1006672885126, 81.63162804046782, 81.94733443352227, 82.27739111717013, 82.57157207433451, 82.79400157853198, 83.08100739039966, 83.42541436464089, 83.66219415943173, 83.99225084307957, 84.25773121905719, 84.6236636291885, 84.83174284279256, 85.15462438114372, 85.38422903063787, 85.62100882542872, 85.80038745784603, 86.13044414149387, 86.31699791920788, 86.42462509865825, 86.5681280045921, 86.78338236349286, 86.99146157709694, 87.16366506421755, 87.29999282485471, 87.53677261964555, 87.72332639735954, 87.78790270502978, 87.92423046566694, 88.06773337160078, 88.16818540575447, 88.30451316639162, 88.412140345842, 88.49106694410563, 88.67044557652292, 88.79959819186338, 88.84982420894023, 88.86417449953362, 88.97897682428069, 89.05790342254431, 89.12247973021454, 89.22293176436823, 89.29468321733516, 89.39513525148884, 89.48841214034584, 89.531463012126, 89.64626533687307, 89.69649135394992, 89.75389251632346, 89.86151969577384, 89.91892085814737, 89.95479658463084, 90.01219774700438, 90.02654803759776, 90.07677405467462, 90.13417521704815, 90.19157637942169, 90.18440123412499, 90.24180239649853, 90.34225443065222, 90.40683073832245, 90.44270646480591, 90.48575733658608, 90.52880820836621, 90.52163306306953, 90.55750878955298, 90.52880820836621, 90.59338451603645, 90.59338451603645, 90.62208509722322, 90.62208509722322, 90.65796082370669, 90.64361053311329, 90.61490995192652, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.69383655019014, 90.68666140489346, 90.7297122766736, 90.73688742197031, 90.73688742197031, 90.73688742197031, 90.744062567267, 90.77276314845376, 90.78711343904715, 90.82298916553061, 90.8301643108273, 90.86604003731075, 90.86604003731075, 90.86604003731075, 90.88039032790415, 90.85168974671738, 90.84451460142068, 90.85168974671738, 90.82298916553061, 90.75123771256368, 90.7297122766736, 90.73688742197031], "calls": [0.0, 0.9829949056468393, 2.102317571930832, 3.472770323599053, 4.4342397933558155, 5.488986151969577, 6.306952715792495, 7.203845877879027, 8.12943962115233, 9.306163449809858, 10.160005740116237, 11.257802970510152, 12.377125636794146, 13.618425773121906, 14.622946114658822, 15.505488986151969, 16.56741049006242, 17.808710626390184, 18.734304369663484, 19.76752529238717, 20.764870488627395, 21.54696132596685, 22.43667934275669, 23.649278897897684, 24.833177871851905, 25.852048503982207, 26.734591375475354, 27.681710554638734, 28.951711272153265, 29.597474348855563, 30.587644399799096, 31.513238143072396, 32.58950993757623, 33.63708115089331, 34.60572576594676, 35.45239291095645, 36.47126354308675, 37.425557867546814, 38.31527588433666, 39.384372533543804, 40.55392121690464, 41.63736815670517, 42.62036306235201, 43.4670302073617, 44.414149386525075, 45.13883906149099, 46.164884838917985, 47.23398148812513, 48.23850182966205, 49.314773624165895, 50.08251417091196, 51.13008538422903, 52.206357178732866, 53.13195092200617, 54.22257300710339, 55.607376049365, 56.669297553275456, 57.53031498887853, 58.506134749228664, 59.46042907368875, 60.414723398148816, 61.419243739685726, 62.4668149530028, 63.52156131161656, 64.76286144794432, 65.55212743058047, 66.90105474635861, 67.754897036665, 68.55133816459784, 69.34060414723399, 70.27337303580397, 71.27071823204419, 72.24653799239434, 73.16495659037095, 73.99009829949057, 74.92286718806056, 75.96326325608094, 76.917557580541, 77.91490277678122, 78.81179593886776, 79.90959316926167, 80.87823778431513, 81.68185405754467, 82.57157207433451, 83.54021668938796, 84.4227595608811, 85.47750591949487, 86.51790198751524, 87.55112291023893, 88.7780727559733, 89.73236708043338, 90.49293248188276, 91.54050369519983, 92.65982636148382, 93.62847097653727, 94.49666355743703, 95.4653081724905, 96.58463083877449, 97.91203271866256, 99.04570567553993, 100.0], "exact_score": [80.00287005811867, 80.50513022888714, 81.06479156202914, 81.4665996986439, 81.81100667288513, 82.1195379206429, 82.4854703307742, 82.70072468967497, 83.0451316639162, 83.33931262108058, 83.5976178517615, 83.92049939011265, 84.12857860371673, 84.41558441558442, 84.7097653727488, 84.94654516753964, 85.26225156059411, 85.49185621008826, 85.62818397072542, 85.87213891081295, 86.17349501327402, 86.35287364569133, 86.51072684221856, 86.68293032933917, 86.85513381645978, 87.0273373035804, 87.2138910812944, 87.33586855133817, 87.55829805553563, 87.71615125206286, 87.85247901270002, 87.9601061921504, 88.11078424338093, 88.24711200401808, 88.36191432876515, 88.44084092702877, 88.56999354236925, 88.74219702948984, 88.79959819186338, 88.85699935423692, 88.91440051661046, 88.97897682428069, 89.09377914902777, 89.15118031140132, 89.28750807203846, 89.38796010619215, 89.48841214034584, 89.55298844801608, 89.63191504627969, 89.66779077276315, 89.74671737102676, 89.85434455047715, 89.904570567554, 89.95479658463084, 90.00502260170768, 90.01219774700438, 90.02654803759776, 90.10547463586138, 90.14852550764154, 90.19875152471838, 90.23462725120184, 90.31355384946545, 90.40683073832245, 90.43553131950922, 90.43553131950922, 90.50010762717945, 90.52163306306953, 90.52880820836621, 90.5503336442563, 90.56468393484968, 90.58620937073975, 90.60773480662984, 90.61490995192652, 90.65078567840999, 90.64361053311329, 90.62208509722322, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.67948625959676, 90.67948625959676, 90.72253713137691, 90.7297122766736, 90.72253713137691, 90.73688742197031, 90.744062567267, 90.77993829375045, 90.78711343904715, 90.84451460142068, 90.84451460142068, 90.88039032790415, 90.85886489201407, 90.88039032790415, 90.85886489201407, 90.85886489201407, 90.8301643108273, 90.8301643108273, 90.79428858434385, 90.75123771256368, 90.7297122766736, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 84.19889502762432, "exact_area100": 88.4517683896459}}, "old_normalized_endpoint_budget": 85}]}, {"expert": "nlvr2", "endpoint": "Qwen3.8", "task": "nlvr2", "small": 80.00287005811867, "large": 90.06242376408122, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 1.0, "validation_area100": 0.8855517166087595, "training_seconds": 40.37867002899293, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7868961484006395, "operator": "ge"}, {"budget": 0.02, "threshold": 0.725559951454113, "operator": "ge"}, {"budget": 0.03, "threshold": 0.6783955984571952, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6411981105132736, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5910459048886381, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5609715593887055, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5247843096785022, "operator": "ge"}, {"budget": 0.08, "threshold": 0.49051541479732363, "operator": "ge"}, {"budget": 0.09, "threshold": 0.45594222934766715, "operator": "ge"}, {"budget": 0.1, "threshold": 0.42542587725108666, "operator": "ge"}, {"budget": 0.11, "threshold": 0.39400431991831886, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3633539450667066, "operator": "ge"}, {"budget": 0.13, "threshold": 0.33864834907187236, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3136408563053354, "operator": "ge"}, {"budget": 0.15, "threshold": 0.2859752193460105, "operator": "ge"}, {"budget": 0.16, "threshold": 0.2636859095387526, "operator": "ge"}, {"budget": 0.17, "threshold": 0.23548707504931163, "operator": "ge"}, {"budget": 0.18, "threshold": 0.21160707632464512, "operator": "ge"}, {"budget": 0.19, "threshold": 0.17731226586066787, "operator": "ge"}, {"budget": 0.2, "threshold": 0.14305305492997525, "operator": "ge"}, {"budget": 0.21, "threshold": 0.11206258858874879, "operator": "ge"}, {"budget": 0.22, "threshold": 0.08354960503649343, "operator": "ge"}, {"budget": 0.23, "threshold": 0.058020330351050954, "operator": "ge"}, {"budget": 0.24, "threshold": 0.027993992310900318, "operator": "ge"}, {"budget": 0.25, "threshold": 0.003198711424704214, "operator": "ge"}, {"budget": 0.26, "threshold": -0.0278592858799527, "operator": "ge"}, {"budget": 0.27, "threshold": -0.06323774911966681, "operator": "ge"}, {"budget": 0.28, "threshold": -0.09370102436516721, "operator": "ge"}, {"budget": 0.29, "threshold": -0.13003343028324432, "operator": "ge"}, {"budget": 0.3, "threshold": -0.1631044116922162, "operator": "ge"}, {"budget": 0.31, "threshold": -0.19376172481522294, "operator": "ge"}, {"budget": 0.32, "threshold": -0.23491802552379693, "operator": "ge"}, {"budget": 0.33, "threshold": -0.2597349699227813, "operator": "ge"}, {"budget": 0.34, "threshold": -0.29338976624736074, "operator": "ge"}, {"budget": 0.35, "threshold": -0.3273386766340802, "operator": "ge"}, {"budget": 0.36, "threshold": -0.36411952637573286, "operator": "ge"}, {"budget": 0.37, "threshold": -0.39752016392682854, "operator": "ge"}, {"budget": 0.38, "threshold": -0.42632882602859673, "operator": "ge"}, {"budget": 0.39, "threshold": -0.45180080802378, "operator": "ge"}, {"budget": 0.4, "threshold": -0.47923448221193765, "operator": "ge"}, {"budget": 0.41, "threshold": -0.5029004728776605, "operator": "ge"}, {"budget": 0.42, "threshold": -0.5313398996958593, "operator": "ge"}, {"budget": 0.43, "threshold": -0.5568045176335943, "operator": "ge"}, {"budget": 0.44, "threshold": -0.5823605609881847, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6080489555586797, "operator": "ge"}, {"budget": 0.46, "threshold": -0.6287141866785826, "operator": "ge"}, {"budget": 0.47, "threshold": -0.6486867619095542, "operator": "ge"}, {"budget": 0.48, "threshold": -0.667486518978637, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6865621725659833, "operator": "ge"}, {"budget": 0.5, "threshold": -0.7034507088998777, "operator": "ge"}, {"budget": 0.51, "threshold": -0.7242928005312366, "operator": "ge"}, {"budget": 0.52, "threshold": -0.7412179331721088, "operator": "ge"}, {"budget": 0.53, "threshold": -0.7553752080493773, "operator": "ge"}, {"budget": 0.54, "threshold": -0.7687986008369089, "operator": "ge"}, {"budget": 0.55, "threshold": -0.7812340589001519, "operator": "ge"}, {"budget": 0.56, "threshold": -0.793718336363798, "operator": "ge"}, {"budget": 0.57, "threshold": -0.809287557572181, "operator": "ge"}, {"budget": 0.58, "threshold": -0.8205697962845352, "operator": "ge"}, {"budget": 0.59, "threshold": -0.8310704292121992, "operator": "ge"}, {"budget": 0.6, "threshold": -0.8429146591680637, "operator": "ge"}, {"budget": 0.61, "threshold": -0.8543336715211959, "operator": "ge"}, {"budget": 0.62, "threshold": -0.8631325382805596, "operator": "ge"}, {"budget": 0.63, "threshold": -0.8725151830688628, "operator": "ge"}, {"budget": 0.64, "threshold": -0.8816394929066874, "operator": "ge"}, {"budget": 0.65, "threshold": -0.8890166564429525, "operator": "ge"}, {"budget": 0.66, "threshold": -0.8970902379451784, "operator": "ge"}, {"budget": 0.67, "threshold": -0.903875444581299, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9101944684368728, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9162331429429583, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9213134921979329, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9260362226243486, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9312179741602457, "operator": "ge"}, {"budget": 0.73, "threshold": -0.934644984157515, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9386730559671087, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9426132303450577, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9462920389510464, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9496245207810564, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9535688505068357, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9565951277738423, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9598947419175062, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9634559824808707, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9659245952739256, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9688922507287869, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9711670406413094, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9734646185051644, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9754484860902428, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9774370823493389, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9790103149682468, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9806815829581461, "operator": "ge"}, {"budget": 0.9, "threshold": -0.982347912909589, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9840305894789517, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9858016205286728, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9871646925936206, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9890311542964018, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9905881467553921, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9920435696205507, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9935516024690796, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9953765159840464, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9971173555562127, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.3494700658837, 80.82211400744772, 81.25179031796046, 81.63849899742193, 82.05385276425092, 82.38327126897737, 82.79862503580635, 83.07075336579776, 83.32855915210541, 83.60068748209682, 83.93010598682325, 84.21655686049841, 84.4313950157548, 84.66055571469494, 84.90403895731882, 85.11887711257519, 85.24778000572901, 85.51990833572042, 85.84932684044686, 86.04984245201948, 86.37926095674591, 86.57977656831852, 86.68003437410484, 86.96648524778, 87.16700085935261, 87.39616155829275, 87.46777427671154, 87.62532225723288, 87.78287023775422, 88.04067602406187, 88.08364365511314, 88.18390146089945, 88.22686909195072, 88.38441707247206, 88.47035233457461, 88.47035233457461, 88.62790031509596, 88.81409338298482, 88.90002864508736, 88.97164136350615, 89.02893153824118, 89.07189916929245, 89.17215697507876, 89.35835004296763, 89.44428530507018, 89.573188198224, 89.6734460040103, 89.71641363506157, 89.8453165282154, 89.87396161558291, 89.98854196505299, 90.07447722715554, 90.17473503294184, 90.24634775136063, 90.27499283872815, 90.31796046977942, 90.31796046977942, 90.27499283872815, 90.3609281008307, 90.47550845030077, 90.53279862503581, 90.57576625608708, 90.61873388713835, 90.63305643082211, 90.66170151818963, 90.67602406187338, 90.69034660555714, 90.74763678029217, 90.79060441134345, 90.80492695502721, 90.83357204239474, 90.84789458607848, 90.81924949871096, 90.81924949871096, 90.90518476081351, 90.96247493554854, 91.03408765396735, 91.06273274133486, 91.03408765396735, 91.03408765396735, 91.06273274133486, 91.03408765396735, 91.07705528501862, 91.10570037238614, 91.14866800343741, 91.14866800343741, 91.13434545975365, 91.13434545975365, 91.10570037238614, 91.07705528501862, 91.01976511028357, 90.9767974792323, 90.91950730449727, 90.93382984818103, 90.89086221712975, 90.76195932397593, 90.71899169292466, 90.64737897450587, 90.59008879977084, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 38, 73, 103, 132, 161, 185, 215, 238, 261, 283, 309, 333, 353, 370, 393, 414, 432, 458, 487, 506, 534, 553, 568, 591, 613, 635, 648, 663, 679, 698, 710, 723, 732, 746, 759, 769, 784, 803, 814, 826, 833, 840, 852, 867, 875, 886, 897, 905, 917, 925, 934, 943, 951, 959, 964, 970, 976, 978, 988, 999, 1008, 1012, 1020, 1024, 1028, 1033, 1036, 1043, 1049, 1053, 1058, 1063, 1066, 1070, 1078, 1082, 1089, 1093, 1094, 1098, 1104, 1106, 1111, 1115, 1119, 1121, 1125, 1128, 1130, 1133, 1137, 1141, 1142, 1146, 1149, 1149, 1151, 1152, 1154, 1154], "harmed": [0, 1, 3, 3, 5, 5, 6, 7, 11, 16, 19, 22, 26, 31, 32, 38, 44, 53, 60, 66, 71, 76, 81, 89, 92, 100, 106, 114, 118, 123, 124, 133, 139, 145, 148, 155, 165, 169, 175, 180, 187, 190, 194, 199, 201, 203, 205, 209, 214, 217, 223, 224, 227, 228, 231, 234, 237, 243, 248, 252, 255, 260, 261, 266, 269, 271, 275, 277, 280, 283, 286, 289, 293, 298, 302, 304, 304, 306, 308, 311, 315, 319, 323, 325, 327, 328, 330, 335, 338, 342, 347, 355, 362, 367, 370, 376, 385, 390, 396, 402, 408], "exact_score": [79.81953594958465, 80.3494700658837, 80.82211400744772, 81.25179031796046, 81.63849899742193, 82.05385276425092, 82.38327126897737, 82.79862503580635, 83.07075336579776, 83.32855915210541, 83.60068748209682, 83.93010598682325, 84.21655686049841, 84.4313950157548, 84.66055571469494, 84.90403895731882, 85.11887711257519, 85.24778000572901, 85.51990833572042, 85.84932684044686, 86.04984245201948, 86.37926095674591, 86.57977656831852, 86.68003437410484, 86.96648524778, 87.16700085935261, 87.39616155829275, 87.46777427671154, 87.62532225723288, 87.78287023775422, 88.04067602406187, 88.08364365511314, 88.18390146089945, 88.22686909195072, 88.38441707247206, 88.47035233457461, 88.47035233457461, 88.62790031509596, 88.81409338298482, 88.90002864508736, 88.97164136350615, 89.02893153824118, 89.07189916929245, 89.17215697507876, 89.35835004296763, 89.44428530507018, 89.573188198224, 89.6734460040103, 89.71641363506157, 89.8453165282154, 89.87396161558291, 89.98854196505299, 90.07447722715554, 90.17473503294184, 90.24634775136063, 90.27499283872815, 90.31796046977942, 90.31796046977942, 90.27499283872815, 90.3609281008307, 90.47550845030077, 90.53279862503581, 90.57576625608708, 90.61873388713835, 90.63305643082211, 90.66170151818963, 90.67602406187338, 90.69034660555714, 90.74763678029217, 90.79060441134345, 90.80492695502721, 90.83357204239474, 90.84789458607848, 90.81924949871096, 90.81924949871096, 90.90518476081351, 90.96247493554854, 91.03408765396735, 91.06273274133486, 91.03408765396735, 91.03408765396735, 91.06273274133486, 91.03408765396735, 91.07705528501862, 91.10570037238614, 91.14866800343741, 91.14866800343741, 91.13434545975365, 91.13434545975365, 91.10570037238614, 91.07705528501862, 91.01976511028357, 90.9767974792323, 90.91950730449727, 90.93382984818103, 90.89086221712975, 90.76195932397593, 90.71899169292466, 90.64737897450587, 90.59008879977084, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.55517166087596, "threshold_area100": 88.55442566599827, "threshold_area25": 84.02033801203093}, "test": {"score": [80.00287005811867, 80.58405682715075, 81.1006672885126, 81.50247542512736, 81.85405754466528, 82.25586568128006, 82.55722178374111, 82.83705245031211, 83.33931262108058, 83.6908947406185, 83.95637511659612, 84.22185549257372, 84.39405897969434, 84.66671450096864, 84.8963191504628, 85.20485039822057, 85.38422903063787, 85.67123484250556, 85.87213891081295, 86.08021812441702, 86.34569850039465, 86.5753031498888, 86.78338236349286, 86.91971012413002, 87.08473846595393, 87.22106622659109, 87.27846738896463, 87.50089689316209, 87.72332639735954, 87.91705532037024, 88.1394848245677, 88.35473918346847, 88.505417234699, 88.62021955944608, 88.69914615770969, 88.85699935423692, 89.07942885843438, 89.23010690966493, 89.29468321733516, 89.41666068737892, 89.51711272153261, 89.71084164454331, 89.8184688239937, 89.82564396929038, 89.93327114874077, 90.04089832819116, 90.06959890937792, 90.1700509435316, 90.22027696060846, 90.32072899476215, 90.37095501183899, 90.34225443065222, 90.44270646480591, 90.46423190069599, 90.50010762717945, 90.52880820836621, 90.47140704599268, 90.49293248188276, 90.47858219128938, 90.50728277247615, 90.55750878955298, 90.59338451603645, 90.61490995192652, 90.65078567840999, 90.6364353878166, 90.62926024251992, 90.60773480662984, 90.6364353878166, 90.66513596900337, 90.71536198608023, 90.70101169548683, 90.7297122766736, 90.72253713137691, 90.73688742197031, 90.75123771256368, 90.70101169548683, 90.69383655019014, 90.65796082370669, 90.65078567840999, 90.60773480662984, 90.62926024251992, 90.60055966133315, 90.61490995192652, 90.64361053311329, 90.61490995192652, 90.62926024251992, 90.59338451603645, 90.55750878955298, 90.55750878955298, 90.55750878955298, 90.53598335366291, 90.52880820836621, 90.52163306306953, 90.50010762717945, 90.45705675539931, 90.40683073832245, 90.3566047212456, 90.30637870416876, 90.24897754179523, 90.12700007175145, 90.06242376408122], "calls": [0.0, 1.1838989739542225, 2.231470187271292, 3.0781373322809786, 3.874578460213819, 5.080002870058118, 5.897969433881036, 6.880964339527876, 8.043337877592021, 9.047858219128937, 10.002152543589007, 10.99949773982923, 12.047068953146303, 12.922436679342756, 13.75475353375906, 14.766449020592667, 15.5700652938222, 16.703738250699576, 17.557580541005954, 18.7199540790701, 20.14063284781517, 21.3173566764727, 22.343402453899692, 23.089617564755684, 24.115663342182682, 24.861878453038674, 25.73007103393844, 26.82069311903566, 27.674535409342038, 28.772332639735954, 29.791203271866255, 30.63787041687594, 31.764368228456625, 32.46035732223578, 33.42182679199254, 34.57702518475999, 35.83985075697783, 36.83002080792136, 37.848891440051666, 38.77448518332496, 39.829231541938725, 40.704599268135176, 41.78804620793571, 42.792566549472625, 43.840137762789695, 45.131663916194306, 46.07160795006099, 47.090478582191295, 48.130874650211666, 49.1210447011552, 49.96053670086819, 51.17313625600919, 52.19918203343617, 53.28262897323671, 54.136471263543086, 55.06206500681638, 56.00918418597976, 57.1859080146373, 58.20477864676759, 59.11602209944752, 60.17794360335797, 61.13223792781803, 62.021955944607875, 63.11975317500179, 64.38257874721963, 65.27229676400947, 66.41314486618354, 67.40331491712708, 68.46523642103752, 69.62043481380498, 70.68953146301212, 71.63665064217551, 72.64834612900911, 73.32998493219488, 74.0474994618641, 75.06637009399441, 76.01348927315779, 77.01800961469469, 78.19473344335222, 79.24230465666929, 80.28987586998637, 81.53117600631413, 82.37784315132382, 83.69089474061849, 84.5303867403315, 85.57078280835186, 86.51790198751524, 87.54394776494225, 88.29016287579825, 89.0722537131377, 89.93327114874076, 90.97366721676114, 91.94231183181459, 92.86790557508789, 94.04462940374543, 94.96304800172204, 95.8312405826218, 96.89316208653224, 98.1488125134534, 99.12463227380354, 100.0], "rescued": [0, 87, 162, 219, 271, 339, 386, 432, 506, 560, 607, 653, 687, 730, 768, 816, 849, 900, 932, 975, 1030, 1073, 1106, 1128, 1159, 1183, 1198, 1241, 1280, 1315, 1354, 1390, 1424, 1444, 1461, 1494, 1533, 1560, 1581, 1607, 1630, 1660, 1684, 1696, 1721, 1745, 1757, 1779, 1794, 1815, 1830, 1844, 1862, 1872, 1886, 1898, 1906, 1919, 1927, 1943, 1957, 1966, 1975, 1986, 1997, 2007, 2015, 2028, 2037, 2051, 2057, 2069, 2076, 2081, 2089, 2094, 2102, 2107, 2116, 2122, 2131, 2140, 2147, 2161, 2164, 2174, 2179, 2185, 2190, 2196, 2200, 2207, 2215, 2222, 2231, 2235, 2236, 2237, 2242, 2243, 2247], "harmed": [0, 6, 9, 10, 13, 25, 30, 37, 41, 46, 56, 65, 75, 80, 86, 91, 99, 110, 114, 128, 146, 157, 161, 164, 172, 177, 184, 196, 204, 212, 220, 226, 239, 243, 249, 260, 268, 274, 286, 295, 304, 307, 316, 327, 337, 346, 354, 362, 370, 377, 385, 403, 407, 414, 423, 431, 447, 457, 467, 479, 486, 490, 496, 502, 515, 526, 537, 546, 551, 558, 566, 574, 582, 585, 591, 603, 612, 622, 632, 644, 650, 663, 668, 678, 685, 693, 703, 714, 719, 725, 732, 740, 749, 759, 774, 785, 793, 801, 814, 832, 845], "exact_score": [80.00287005811867, 80.49077993829376, 80.96433952787544, 81.44507426275382, 81.88275812585205, 82.22716510009327, 82.5787472196312, 82.90162875798235, 83.31778718519051, 83.67654445002512, 83.95637511659612, 84.22185549257372, 84.37970868910097, 84.68823993685872, 84.96807060342972, 85.24790127000072, 85.51338164597834, 85.75733658606588, 85.93671521848317, 86.15914472268064, 86.33134820980125, 86.52507713281194, 86.73315634641602, 86.91971012413002, 87.08473846595393, 87.22106622659109, 87.34304369663487, 87.53677261964555, 87.75920212384301, 87.945755901557, 88.21841142283131, 88.43366578173209, 88.56281839707253, 88.66327043122624, 88.74937217478654, 88.91440051661046, 89.10812943962115, 89.23010690966493, 89.32338379852193, 89.438186123269, 89.56733873860946, 89.73954222573008, 89.811293678697, 89.82564396929038, 89.96197172992754, 90.04807347348785, 90.06959890937792, 90.15570065293824, 90.21310181531177, 90.31355384946545, 90.3637798665423, 90.33507928535553, 90.41400588361914, 90.44988161010261, 90.50010762717945, 90.52880820836621, 90.47140704599268, 90.48575733658608, 90.47858219128938, 90.49293248188276, 90.5503336442563, 90.59338451603645, 90.61490995192652, 90.65796082370669, 90.65078567840999, 90.6364353878166, 90.65078567840999, 90.62208509722322, 90.64361053311329, 90.69383655019014, 90.70101169548683, 90.67948625959676, 90.73688742197031, 90.73688742197031, 90.75123771256368, 90.70818684078354, 90.69383655019014, 90.65796082370669, 90.64361053311329, 90.60773480662984, 90.62208509722322, 90.61490995192652, 90.60773480662984, 90.62208509722322, 90.6364353878166, 90.6364353878166, 90.6364353878166, 90.57903422544307, 90.56468393484968, 90.55750878955298, 90.52880820836621, 90.52163306306953, 90.52163306306953, 90.50010762717945, 90.46423190069599, 90.40683073832245, 90.34942957594892, 90.31355384946545, 90.24897754179523, 90.12700007175145, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.62940330267455, "threshold_area100": 88.62122407978762, "threshold_area25": 84.27495156776925}, "selected_budget": 85, "selected": {"val": {"score": 91.14866800343741, "calls": 84.98997421942137, "rescued": 1119, "harmed": 328}, "test": {"score": 90.62926024251992, "calls": 85.57078280835186, "rescued": 2174, "harmed": 693}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5964714288711548, "operator": "ge"}, {"budget": 0.02, "threshold": 0.509648323059082, "operator": "ge"}, {"budget": 0.03, "threshold": 0.4168300926685333, "operator": "ge"}, {"budget": 0.04, "threshold": 0.369790643453598, "operator": "ge"}, {"budget": 0.05, "threshold": 0.33766719698905945, "operator": "ge"}, {"budget": 0.06, "threshold": 0.3102230131626129, "operator": "ge"}, {"budget": 0.07, "threshold": 0.28070539236068726, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2609180808067322, "operator": "ge"}, {"budget": 0.09, "threshold": 0.24116083979606628, "operator": "ge"}, {"budget": 0.1, "threshold": 0.22163711488246918, "operator": "ge"}, {"budget": 0.11, "threshold": 0.2015872299671173, "operator": "ge"}, {"budget": 0.12, "threshold": 0.17894965410232544, "operator": "ge"}, {"budget": 0.13, "threshold": 0.160380020737648, "operator": "ge"}, {"budget": 0.14, "threshold": 0.13986465334892273, "operator": "ge"}, {"budget": 0.15, "threshold": 0.12271307408809662, "operator": "ge"}, {"budget": 0.16, "threshold": 0.10659100860357285, "operator": "ge"}, {"budget": 0.17, "threshold": 0.0882742702960968, "operator": "ge"}, {"budget": 0.18, "threshold": 0.07471165060997009, "operator": "ge"}, {"budget": 0.19, "threshold": 0.0627129003405571, "operator": "ge"}, {"budget": 0.2, "threshold": 0.048571862280368805, "operator": "ge"}, {"budget": 0.21, "threshold": 0.036267541348934174, "operator": "ge"}, {"budget": 0.22, "threshold": 0.022452376782894135, "operator": "ge"}, {"budget": 0.23, "threshold": 0.013272270560264587, "operator": "ge"}, {"budget": 0.24, "threshold": 0.006053227931261063, "operator": "ge"}, {"budget": 0.25, "threshold": 0.0007501393556594849, "operator": "ge"}, {"budget": 0.26, "threshold": -0.0026658978313207626, "operator": "ge"}, {"budget": 0.27, "threshold": -0.0043951645493507385, "operator": "ge"}, {"budget": 0.28, "threshold": -0.005825965199619532, "operator": "ge"}, {"budget": 0.29, "threshold": -0.006856398191303015, "operator": "ge"}, {"budget": 0.3, "threshold": -0.007812866009771824, "operator": "ge"}, {"budget": 0.31, "threshold": -0.008936230093240738, "operator": "ge"}, {"budget": 0.32, "threshold": -0.01000085100531578, "operator": "ge"}, {"budget": 0.33, "threshold": -0.010812987573444843, "operator": "ge"}, {"budget": 0.34, "threshold": -0.011742641218006611, "operator": "ge"}, {"budget": 0.35, "threshold": -0.012763279490172863, "operator": "ge"}, {"budget": 0.36, "threshold": -0.013756271451711655, "operator": "ge"}, {"budget": 0.37, "threshold": -0.014712966978549957, "operator": "ge"}, {"budget": 0.38, "threshold": -0.01560598611831665, "operator": "ge"}, {"budget": 0.39, "threshold": -0.016587618738412857, "operator": "ge"}, {"budget": 0.4, "threshold": -0.017660334706306458, "operator": "ge"}, {"budget": 0.41, "threshold": -0.018632149323821068, "operator": "ge"}, {"budget": 0.42, "threshold": -0.019387835636734962, "operator": "ge"}, {"budget": 0.43, "threshold": -0.020391184836626053, "operator": "ge"}, {"budget": 0.44, "threshold": -0.021385809406638145, "operator": "ge"}, {"budget": 0.45, "threshold": -0.022310877218842506, "operator": "ge"}, {"budget": 0.46, "threshold": -0.023631509393453598, "operator": "ge"}, {"budget": 0.47, "threshold": -0.024579405784606934, "operator": "ge"}, {"budget": 0.48, "threshold": -0.02570054680109024, "operator": "ge"}, {"budget": 0.49, "threshold": -0.026918906718492508, "operator": "ge"}, {"budget": 0.5, "threshold": -0.02819862589240074, "operator": "ge"}, {"budget": 0.51, "threshold": -0.029554981738328934, "operator": "ge"}, {"budget": 0.52, "threshold": -0.030646050348877907, "operator": "ge"}, {"budget": 0.53, "threshold": -0.031813714653253555, "operator": "ge"}, {"budget": 0.54, "threshold": -0.03309047594666481, "operator": "ge"}, {"budget": 0.55, "threshold": -0.034555017948150635, "operator": "ge"}, {"budget": 0.56, "threshold": -0.03629998117685318, "operator": "ge"}, {"budget": 0.57, "threshold": -0.03766292333602905, "operator": "ge"}, {"budget": 0.58, "threshold": -0.03972860425710678, "operator": "ge"}, {"budget": 0.59, "threshold": -0.041245415806770325, "operator": "ge"}, {"budget": 0.6, "threshold": -0.04286649823188782, "operator": "ge"}, {"budget": 0.61, "threshold": -0.044553812593221664, "operator": "ge"}, {"budget": 0.62, "threshold": -0.0460750088095665, "operator": "ge"}, {"budget": 0.63, "threshold": -0.047520820051431656, "operator": "ge"}, {"budget": 0.64, "threshold": -0.049047231674194336, "operator": "ge"}, {"budget": 0.65, "threshold": -0.05114707350730896, "operator": "ge"}, {"budget": 0.66, "threshold": -0.05319993197917938, "operator": "ge"}, {"budget": 0.67, "threshold": -0.05571812763810158, "operator": "ge"}, {"budget": 0.68, "threshold": -0.05775493383407593, "operator": "ge"}, {"budget": 0.69, "threshold": -0.059800926595926285, "operator": "ge"}, {"budget": 0.7, "threshold": -0.06219836324453354, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0647609531879425, "operator": "ge"}, {"budget": 0.72, "threshold": -0.06754229217767715, "operator": "ge"}, {"budget": 0.73, "threshold": -0.06977970898151398, "operator": "ge"}, {"budget": 0.74, "threshold": -0.07242459058761597, "operator": "ge"}, {"budget": 0.75, "threshold": -0.07537168264389038, "operator": "ge"}, {"budget": 0.76, "threshold": -0.07870890200138092, "operator": "ge"}, {"budget": 0.77, "threshold": -0.08207070082426071, "operator": "ge"}, {"budget": 0.78, "threshold": -0.08572269231081009, "operator": "ge"}, {"budget": 0.79, "threshold": -0.08924134075641632, "operator": "ge"}, {"budget": 0.8, "threshold": -0.09302942454814911, "operator": "ge"}, {"budget": 0.81, "threshold": -0.0970994085073471, "operator": "ge"}, {"budget": 0.82, "threshold": -0.10144376754760742, "operator": "ge"}, {"budget": 0.83, "threshold": -0.10658526420593262, "operator": "ge"}, {"budget": 0.84, "threshold": -0.11154815554618835, "operator": "ge"}, {"budget": 0.85, "threshold": -0.1179526299238205, "operator": "ge"}, {"budget": 0.86, "threshold": -0.12434191256761551, "operator": "ge"}, {"budget": 0.87, "threshold": -0.13308009505271912, "operator": "ge"}, {"budget": 0.88, "threshold": -0.14032945036888123, "operator": "ge"}, {"budget": 0.89, "threshold": -0.15024346113204956, "operator": "ge"}, {"budget": 0.9, "threshold": -0.15845339000225067, "operator": "ge"}, {"budget": 0.91, "threshold": -0.16841578483581543, "operator": "ge"}, {"budget": 0.92, "threshold": -0.17978273332118988, "operator": "ge"}, {"budget": 0.93, "threshold": -0.19709116220474243, "operator": "ge"}, {"budget": 0.94, "threshold": -0.21208491921424866, "operator": "ge"}, {"budget": 0.95, "threshold": -0.23300471901893616, "operator": "ge"}, {"budget": 0.96, "threshold": -0.25904032588005066, "operator": "ge"}, {"budget": 0.97, "threshold": -0.29177480936050415, "operator": "ge"}, {"budget": 0.98, "threshold": -0.35389140248298645, "operator": "ge"}, {"budget": 0.99, "threshold": -0.49074235558509827, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.39243769693498, 80.7934689200802, 81.1372099684904, 81.53824119163563, 81.98224004583213, 82.39759381266111, 82.68404468633629, 82.91320538527643, 83.21397880263535, 83.42881695789171, 83.70094528788312, 84.07333142366083, 84.33113720996849, 84.58894299627613, 84.98997421942136, 85.24778000572901, 85.47694070466915, 85.69177885992552, 85.9352621025494, 86.09281008307075, 86.3506158693784, 86.46519621884846, 86.75164709252363, 86.96648524778, 87.15267831566887, 87.25293612145518, 87.35319392724148, 87.43912918934402, 87.5107419077628, 87.52506445144657, 87.53938699513033, 87.55370953881409, 87.55370953881409, 87.59667716986537, 87.62532225723288, 87.66828988828415, 87.78287023775422, 87.8258378688055, 87.86880549985678, 87.94041821827557, 87.95474076195931, 87.9977083930106, 88.05499856774563, 88.09796619879691, 88.11228874248066, 88.15525637353193, 88.28415926668576, 88.29848181036952, 88.34144944142079, 88.4417072472071, 88.48467487825837, 88.49899742194212, 88.58493268404469, 88.55628759667717, 88.58493268404469, 88.6135777714122, 88.72815812088226, 88.79977083930105, 88.84273847035233, 88.91435118877112, 88.95731881982239, 89.01460899455743, 89.10054425665997, 89.15783443139502, 89.21512460613005, 89.21512460613005, 89.21512460613005, 89.24376969349757, 89.25809223718132, 89.3297049556001, 89.4013176740189, 89.51589802348897, 89.573188198224, 89.60183328559151, 89.61615582927529, 89.63047837295903, 89.68776854769406, 89.68776854769406, 89.74505872242909, 89.83099398453164, 89.81667144084788, 89.97421942136923, 89.9455743340017, 90.13176740189057, 90.0887997708393, 90.13176740189057, 90.26067029504439, 90.30363792609567, 90.41821827556574, 90.50415353766829, 90.63305643082211, 90.71899169292466, 90.64737897450587, 90.59008879977084, 90.64737897450587, 90.81924949871096, 90.93382984818103, 90.86221712976223, 90.80492695502721, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 40, 70, 95, 124, 155, 187, 209, 232, 255, 276, 299, 329, 351, 376, 409, 431, 457, 479, 501, 516, 536, 551, 574, 594, 613, 622, 630, 636, 642, 644, 646, 649, 650, 653, 655, 659, 669, 672, 678, 684, 687, 691, 696, 700, 701, 705, 714, 719, 723, 731, 738, 744, 752, 755, 761, 765, 774, 780, 786, 792, 800, 807, 816, 823, 828, 833, 837, 842, 848, 858, 866, 875, 886, 890, 895, 905, 913, 920, 930, 939, 943, 957, 959, 975, 981, 990, 1005, 1014, 1025, 1038, 1052, 1066, 1071, 1077, 1089, 1108, 1121, 1133, 1146, 1154], "harmed": [0, 0, 2, 3, 4, 4, 7, 9, 16, 18, 24, 28, 32, 36, 43, 48, 52, 62, 69, 74, 78, 80, 87, 90, 95, 101, 103, 104, 104, 105, 106, 107, 109, 110, 110, 110, 111, 113, 113, 116, 117, 119, 120, 121, 122, 122, 123, 123, 127, 128, 129, 133, 138, 140, 145, 149, 151, 152, 153, 156, 157, 162, 165, 168, 171, 172, 177, 181, 184, 189, 194, 197, 198, 205, 207, 211, 220, 224, 231, 237, 240, 245, 248, 252, 255, 264, 270, 276, 282, 285, 292, 297, 305, 315, 325, 333, 340, 345, 362, 379, 408], "exact_score": [79.81953594958465, 80.39243769693498, 80.7934689200802, 81.1372099684904, 81.53824119163563, 81.98224004583213, 82.39759381266111, 82.68404468633629, 82.91320538527643, 83.21397880263535, 83.42881695789171, 83.70094528788312, 84.07333142366083, 84.33113720996849, 84.58894299627613, 84.98997421942136, 85.24778000572901, 85.47694070466915, 85.69177885992552, 85.9352621025494, 86.09281008307075, 86.3506158693784, 86.46519621884846, 86.75164709252363, 86.96648524778, 87.15267831566887, 87.25293612145518, 87.35319392724148, 87.43912918934402, 87.5107419077628, 87.52506445144657, 87.53938699513033, 87.55370953881409, 87.55370953881409, 87.59667716986537, 87.62532225723288, 87.66828988828415, 87.78287023775422, 87.8258378688055, 87.86880549985678, 87.94041821827557, 87.95474076195931, 87.9977083930106, 88.05499856774563, 88.09796619879691, 88.11228874248066, 88.15525637353193, 88.28415926668576, 88.29848181036952, 88.34144944142079, 88.4417072472071, 88.48467487825837, 88.49899742194212, 88.58493268404469, 88.55628759667717, 88.58493268404469, 88.6135777714122, 88.72815812088226, 88.79977083930105, 88.84273847035233, 88.91435118877112, 88.95731881982239, 89.01460899455743, 89.10054425665997, 89.15783443139502, 89.21512460613005, 89.21512460613005, 89.21512460613005, 89.24376969349757, 89.25809223718132, 89.3297049556001, 89.4013176740189, 89.51589802348897, 89.573188198224, 89.60183328559151, 89.61615582927529, 89.63047837295903, 89.68776854769406, 89.68776854769406, 89.74505872242909, 89.83099398453164, 89.81667144084788, 89.97421942136923, 89.9455743340017, 90.13176740189057, 90.0887997708393, 90.13176740189057, 90.26067029504439, 90.30363792609567, 90.41821827556574, 90.50415353766829, 90.63305643082211, 90.71899169292466, 90.64737897450587, 90.59008879977084, 90.64737897450587, 90.81924949871096, 90.93382984818103, 90.86221712976223, 90.80492695502721, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.78150403693878, "threshold_area100": 87.78072185620167, "threshold_area25": 83.99598968776854}, "test": {"score": [80.00287005811867, 80.59123197244745, 81.0073903996556, 81.5670517327976, 82.0262610317859, 82.42806916840067, 82.82270215971874, 83.20298486044344, 83.41823921934419, 83.67654445002512, 83.90614909951927, 84.16445433020019, 84.40840927028772, 84.67388964626534, 85.02547176580326, 85.1761498170338, 85.4559804836048, 85.72863600487909, 85.84343832962618, 86.0084666714501, 86.31699791920788, 86.55377771399871, 86.77620721819616, 86.91253497883332, 87.08473846595393, 87.2067159359977, 87.27846738896463, 87.35739398722825, 87.42914544019517, 87.47219631197532, 87.46502116667862, 87.50089689316209, 87.55829805553563, 87.5941737820191, 87.63722465379924, 87.65875008968932, 87.65157494439264, 87.69462581617277, 87.74485183324963, 87.78790270502978, 87.83095357680993, 87.88117959388678, 87.88835473918347, 87.945755901557, 88.01750735452394, 88.03903279041401, 88.09643395278755, 88.14665996986439, 88.16101026045777, 88.21841142283131, 88.28298773050155, 88.35473918346847, 88.39061490995192, 88.46954150821554, 88.55564325177585, 88.62021955944608, 88.70632130300639, 88.79959819186338, 88.89287508072039, 88.8856999354237, 88.90722537131377, 89.00050226017078, 89.03637798665423, 89.08660400373107, 89.19423118318146, 89.2588074908517, 89.32338379852193, 89.38796010619215, 89.39513525148884, 89.47406184975246, 89.58168902920285, 89.58886417449953, 89.63909019157639, 89.68931620865322, 89.74671737102676, 89.8041185334003, 89.86869484107054, 89.91892085814737, 89.997847456411, 90.00502260170768, 90.10547463586138, 90.14852550764154, 90.18440123412499, 90.25615268709191, 90.29920355887207, 90.34225443065222, 90.44270646480591, 90.50010762717945, 90.60055966133315, 90.73688742197031, 90.79428858434385, 90.77993829375045, 90.75123771256368, 90.7297122766736, 90.7297122766736, 90.71536198608023, 90.62926024251992, 90.52163306306953, 90.56468393484968, 90.48575733658608, 90.06242376408122], "calls": [0.0, 1.0045203415369162, 1.7722608882829876, 3.1283633493578247, 4.111358255004664, 5.087178015354811, 6.134749228671881, 7.225371313769104, 8.086388749372176, 8.976106766162015, 9.908875654732007, 10.841644543302003, 11.774413431871995, 12.685656884551912, 13.955657602066443, 14.787974456482743, 15.670517327975892, 16.725263686589653, 17.564755686302647, 18.411422831312336, 19.645547822343403, 20.78639592451747, 22.042046351438614, 22.97481524000861, 24.029561598622372, 24.81165243596183, 25.651144435674823, 26.641314486618356, 27.846738896462654, 28.87995981918634, 29.956231613690175, 31.247757767094786, 32.237927818038315, 33.01284351008108, 34.168041902848536, 35.25866398794575, 36.313410346559515, 37.296405252206355, 38.25787472196312, 39.30544593528019, 40.431943746860874, 41.52974097725479, 42.30465666929755, 43.1871995407907, 44.32087249766808, 45.12448877089761, 46.344263471335296, 47.2626820693119, 48.231326684365364, 49.44392623950635, 50.383870273373034, 51.38121546961326, 52.278108631699794, 53.20370237497309, 54.10777068235632, 55.12664131448661, 56.27466456195738, 57.401162373538064, 58.814665996986434, 59.72590944966636, 60.64432804764297, 61.59862237210304, 62.51704097007964, 63.22020520915549, 64.14579895242879, 65.21489560163593, 66.25529166965632, 67.48941666068737, 68.32173351510367, 69.22580182248691, 70.2446724546172, 71.22766736026405, 72.12456052235058, 72.99992824854704, 73.83942024826003, 74.77218913683002, 75.83411064074048, 76.77405467460716, 77.69247327258377, 78.53914041759346, 79.40733299849322, 80.22529956231614, 80.92846380139198, 81.88275812585205, 82.74377556145512, 83.81287221066228, 84.82456769749588, 86.11609385090048, 87.06321303006386, 88.3260386022817, 89.16553060199469, 90.1700509435316, 90.95931692616776, 92.33694482313267, 93.31993972877952, 94.5253641386238, 95.62316136901772, 96.62050656525794, 97.93355815455263, 98.95960393197963, 100.0], "rescued": [0, 86, 145, 230, 300, 362, 423, 484, 523, 569, 614, 660, 702, 744, 804, 832, 874, 922, 946, 978, 1029, 1067, 1106, 1132, 1162, 1182, 1194, 1205, 1215, 1222, 1225, 1234, 1242, 1249, 1256, 1262, 1266, 1273, 1281, 1291, 1301, 1310, 1315, 1323, 1334, 1339, 1353, 1362, 1370, 1381, 1391, 1405, 1412, 1427, 1443, 1456, 1473, 1492, 1512, 1518, 1527, 1548, 1560, 1571, 1590, 1603, 1620, 1635, 1644, 1659, 1681, 1693, 1706, 1718, 1730, 1745, 1764, 1780, 1793, 1807, 1827, 1842, 1853, 1874, 1889, 1908, 1929, 1952, 1976, 2007, 2026, 2045, 2056, 2083, 2101, 2125, 2140, 2159, 2196, 2226, 2247], "harmed": [0, 4, 5, 12, 18, 24, 30, 38, 47, 57, 70, 80, 88, 93, 104, 111, 114, 124, 132, 141, 149, 154, 162, 169, 175, 178, 180, 180, 180, 181, 185, 189, 189, 191, 192, 195, 200, 201, 202, 206, 210, 212, 216, 216, 217, 219, 225, 227, 233, 236, 237, 241, 243, 247, 251, 255, 260, 266, 273, 280, 286, 294, 301, 305, 309, 313, 321, 327, 335, 339, 346, 357, 363, 368, 372, 379, 389, 398, 400, 413, 419, 428, 434, 445, 454, 467, 474, 489, 499, 511, 522, 543, 558, 588, 606, 632, 659, 693, 724, 765, 845], "exact_score": [80.00287005811867, 80.58405682715075, 81.12219272440268, 81.50247542512736, 81.97603501470905, 82.41371887780727, 82.80117672382866, 83.10970797158643, 83.40388892875082, 83.69806988591519, 83.93484968070604, 84.20033005668365, 84.47298557795796, 84.78151682571571, 85.03982205639664, 85.21202554351726, 85.5492573724618, 85.80038745784603, 85.92954007318649, 86.15914472268064, 86.41027480806487, 86.58965344048217, 86.76903207289948, 86.91253497883332, 87.08473846595393, 87.22824137188778, 87.3143431154481, 87.35739398722825, 87.43632058549186, 87.47219631197532, 87.46502116667862, 87.50089689316209, 87.53677261964555, 87.5941737820191, 87.61569921790917, 87.65157494439264, 87.65157494439264, 87.69462581617277, 87.73050154265624, 87.78072755973308, 87.80225299562316, 87.85247901270002, 87.88835473918347, 87.93858075626032, 88.00315706393054, 88.03185764511731, 88.08208366219417, 88.1394848245677, 88.16818540575447, 88.18971084164454, 88.26146229461148, 88.34038889287508, 88.36191432876515, 88.46236636291886, 88.55564325177585, 88.61304441414939, 88.67762072181962, 88.74937217478654, 88.83547391834685, 88.8856999354237, 88.90005022601709, 88.92157566190716, 89.03637798665423, 89.065078567841, 89.17988089258809, 89.251632345555, 89.33055894381862, 89.37360981559877, 89.38796010619215, 89.44536126856569, 89.54581330271938, 89.59603931979623, 89.63909019157639, 89.68931620865322, 89.76824280691685, 89.82564396929038, 89.87586998636723, 89.92609600344407, 90.01219774700438, 90.04089832819116, 90.11982492645475, 90.19875152471838, 90.24180239649853, 90.34225443065222, 90.3637798665423, 90.47140704599268, 90.47858219128938, 90.60055966133315, 90.70101169548683, 90.80863887493722, 90.77993829375045, 90.744062567267, 90.744062567267, 90.7297122766736, 90.69383655019014, 90.70101169548683, 90.55750878955298, 90.56468393484968, 90.57903422544307, 90.46423190069599, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.90091111474592, "threshold_area100": 87.89251632345557, "threshold_area25": 84.28643180024396}, "selected_budget": 97, "selected": {"val": {"score": 90.93382984818103, "calls": 96.99226582641077, "rescued": 1121, "harmed": 345}, "test": {"score": 90.52163306306953, "calls": 96.62050656525794, "rescued": 2159, "harmed": 693}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.43576717376709, "operator": "ge"}, {"budget": 0.02, "threshold": 2.9020156860351562, "operator": "ge"}, {"budget": 0.03, "threshold": 2.575654983520508, "operator": "ge"}, {"budget": 0.04, "threshold": 2.319356918334961, "operator": "ge"}, {"budget": 0.05, "threshold": 2.1206259727478027, "operator": "ge"}, {"budget": 0.06, "threshold": 1.945447564125061, "operator": "ge"}, {"budget": 0.07, "threshold": 1.776586890220642, "operator": "ge"}, {"budget": 0.08, "threshold": 1.6422878503799438, "operator": "ge"}, {"budget": 0.09, "threshold": 1.4956446886062622, "operator": "ge"}, {"budget": 0.1, "threshold": 1.3776812553405762, "operator": "ge"}, {"budget": 0.11, "threshold": 1.2691789865493774, "operator": "ge"}, {"budget": 0.12, "threshold": 1.162294626235962, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0754289627075195, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9775873422622681, "operator": "ge"}, {"budget": 0.15, "threshold": 0.8565788269042969, "operator": "ge"}, {"budget": 0.16, "threshold": 0.7580100297927856, "operator": "ge"}, {"budget": 0.17, "threshold": 0.6760632395744324, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5890371203422546, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5129914283752441, "operator": "ge"}, {"budget": 0.2, "threshold": 0.411136269569397, "operator": "ge"}, {"budget": 0.21, "threshold": 0.3157396614551544, "operator": "ge"}, {"budget": 0.22, "threshold": 0.22846220433712006, "operator": "ge"}, {"budget": 0.23, "threshold": 0.1561899334192276, "operator": "ge"}, {"budget": 0.24, "threshold": 0.07340745627880096, "operator": "ge"}, {"budget": 0.25, "threshold": 0.007287085056304932, "operator": "ge"}, {"budget": 0.26, "threshold": -0.07846009731292725, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1588151454925537, "operator": "ge"}, {"budget": 0.28, "threshold": -0.23500171303749084, "operator": "ge"}, {"budget": 0.29, "threshold": -0.3216482698917389, "operator": "ge"}, {"budget": 0.3, "threshold": -0.4042986035346985, "operator": "ge"}, {"budget": 0.31, "threshold": -0.49803194403648376, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5715210437774658, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6361521482467651, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7155550718307495, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7998086810112, "operator": "ge"}, {"budget": 0.36, "threshold": -0.887599766254425, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9751696586608887, "operator": "ge"}, {"budget": 0.38, "threshold": -1.0417243242263794, "operator": "ge"}, {"budget": 0.39, "threshold": -1.1211484670639038, "operator": "ge"}, {"budget": 0.4, "threshold": -1.1915909051895142, "operator": "ge"}, {"budget": 0.41, "threshold": -1.2639024257659912, "operator": "ge"}, {"budget": 0.42, "threshold": -1.3398010730743408, "operator": "ge"}, {"budget": 0.43, "threshold": -1.4165122509002686, "operator": "ge"}, {"budget": 0.44, "threshold": -1.486159086227417, "operator": "ge"}, {"budget": 0.45, "threshold": -1.5720690488815308, "operator": "ge"}, {"budget": 0.46, "threshold": -1.6472618579864502, "operator": "ge"}, {"budget": 0.47, "threshold": -1.7251819372177124, "operator": "ge"}, {"budget": 0.48, "threshold": -1.798263430595398, "operator": "ge"}, {"budget": 0.49, "threshold": -1.886399507522583, "operator": "ge"}, {"budget": 0.5, "threshold": -1.9530200958251953, "operator": "ge"}, {"budget": 0.51, "threshold": -2.035001039505005, "operator": "ge"}, {"budget": 0.52, "threshold": -2.1030638217926025, "operator": "ge"}, {"budget": 0.53, "threshold": -2.1734933853149414, "operator": "ge"}, {"budget": 0.54, "threshold": -2.2379589080810547, "operator": "ge"}, {"budget": 0.55, "threshold": -2.3227202892303467, "operator": "ge"}, {"budget": 0.56, "threshold": -2.377025604248047, "operator": "ge"}, {"budget": 0.57, "threshold": -2.4482994079589844, "operator": "ge"}, {"budget": 0.58, "threshold": -2.5176210403442383, "operator": "ge"}, {"budget": 0.59, "threshold": -2.5803754329681396, "operator": "ge"}, {"budget": 0.6, "threshold": -2.659566879272461, "operator": "ge"}, {"budget": 0.61, "threshold": -2.737433910369873, "operator": "ge"}, {"budget": 0.62, "threshold": -2.8124547004699707, "operator": "ge"}, {"budget": 0.63, "threshold": -2.8863422870635986, "operator": "ge"}, {"budget": 0.64, "threshold": -2.956205368041992, "operator": "ge"}, {"budget": 0.65, "threshold": -3.0392768383026123, "operator": "ge"}, {"budget": 0.66, "threshold": -3.125171184539795, "operator": "ge"}, {"budget": 0.67, "threshold": -3.179508686065674, "operator": "ge"}, {"budget": 0.68, "threshold": -3.2552926540374756, "operator": "ge"}, {"budget": 0.69, "threshold": -3.3237552642822266, "operator": "ge"}, {"budget": 0.7, "threshold": -3.389599084854126, "operator": "ge"}, {"budget": 0.71, "threshold": -3.46317720413208, "operator": "ge"}, {"budget": 0.72, "threshold": -3.516721487045288, "operator": "ge"}, {"budget": 0.73, "threshold": -3.5831775665283203, "operator": "ge"}, {"budget": 0.74, "threshold": -3.6433839797973633, "operator": "ge"}, {"budget": 0.75, "threshold": -3.7066617012023926, "operator": "ge"}, {"budget": 0.76, "threshold": -3.781437635421753, "operator": "ge"}, {"budget": 0.77, "threshold": -3.848813056945801, "operator": "ge"}, {"budget": 0.78, "threshold": -3.91743803024292, "operator": "ge"}, {"budget": 0.79, "threshold": -3.9821114540100098, "operator": "ge"}, {"budget": 0.8, "threshold": -4.048892021179199, "operator": "ge"}, {"budget": 0.81, "threshold": -4.135839939117432, "operator": "ge"}, {"budget": 0.82, "threshold": -4.2221760749816895, "operator": "ge"}, {"budget": 0.83, "threshold": -4.296469211578369, "operator": "ge"}, {"budget": 0.84, "threshold": -4.380586624145508, "operator": "ge"}, {"budget": 0.85, "threshold": -4.45690393447876, "operator": "ge"}, {"budget": 0.86, "threshold": -4.544686794281006, "operator": "ge"}, {"budget": 0.87, "threshold": -4.613958835601807, "operator": "ge"}, {"budget": 0.88, "threshold": -4.6966166496276855, "operator": "ge"}, {"budget": 0.89, "threshold": -4.772329807281494, "operator": "ge"}, {"budget": 0.9, "threshold": -4.871453285217285, "operator": "ge"}, {"budget": 0.91, "threshold": -4.97259521484375, "operator": "ge"}, {"budget": 0.92, "threshold": -5.053301811218262, "operator": "ge"}, {"budget": 0.93, "threshold": -5.186931133270264, "operator": "ge"}, {"budget": 0.94, "threshold": -5.319652557373047, "operator": "ge"}, {"budget": 0.95, "threshold": -5.461123943328857, "operator": "ge"}, {"budget": 0.96, "threshold": -5.631709098815918, "operator": "ge"}, {"budget": 0.97, "threshold": -5.819836616516113, "operator": "ge"}, {"budget": 0.98, "threshold": -6.147180080413818, "operator": "ge"}, {"budget": 0.99, "threshold": -6.618288040161133, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.2492122600974, 80.67888857061013, 81.09424233743913, 81.52391864795187, 81.91062732741334, 82.2686909195073, 82.68404468633629, 82.8845602979089, 83.17101117158407, 83.5004296763105, 83.82984818103695, 84.1162990547121, 84.44571755943855, 84.73216843311371, 85.06158693784016, 85.21913491836149, 85.4912632483529, 85.69177885992552, 85.86364938413061, 86.1071326267545, 86.26468060727585, 86.46519621884846, 86.68003437410484, 86.95216270409625, 87.16700085935261, 87.41048410197651, 87.45345173302778, 87.5823546261816, 87.76854769407046, 87.98338584932684, 88.04067602406187, 88.15525637353193, 88.28415926668576, 88.28415926668576, 88.47035233457461, 88.52764250930964, 88.64222285877972, 88.81409338298482, 88.84273847035233, 88.90002864508736, 88.98596390718991, 89.04325408192494, 89.17215697507876, 89.22944714981381, 89.37267258665138, 89.48725293612145, 89.573188198224, 89.74505872242909, 89.78802635348038, 89.81667144084788, 89.97421942136923, 90.11744485820681, 90.18905757662561, 90.24634775136063, 90.31796046977942, 90.28931538241191, 90.33228301346318, 90.34660555714694, 90.4325408192495, 90.41821827556574, 90.48983099398453, 90.54712116871956, 90.6044113434546, 90.64737897450587, 90.7046691492409, 90.66170151818963, 90.7046691492409, 90.76195932397593, 90.77628186765969, 90.81924949871096, 90.76195932397593, 90.79060441134345, 90.79060441134345, 90.80492695502721, 90.876539673446, 90.93382984818103, 90.9481523918648, 91.00544256659983, 91.01976511028357, 91.0484101976511, 91.01976511028357, 91.03408765396735, 91.09137782870236, 91.09137782870236, 91.12002291606989, 91.12002291606989, 91.14866800343741, 91.09137782870236, 91.12002291606989, 91.07705528501862, 91.01976511028357, 90.91950730449727, 90.93382984818103, 90.91950730449727, 90.876539673446, 90.76195932397593, 90.74763678029217, 90.64737897450587, 90.57576625608708, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 30, 61, 92, 122, 151, 176, 205, 220, 244, 269, 296, 320, 347, 372, 398, 417, 439, 463, 485, 506, 528, 546, 566, 590, 613, 636, 647, 660, 677, 696, 708, 723, 736, 744, 762, 774, 787, 803, 813, 825, 835, 842, 853, 859, 872, 883, 895, 909, 917, 922, 935, 947, 953, 960, 967, 974, 981, 987, 996, 1000, 1006, 1014, 1021, 1027, 1033, 1034, 1039, 1044, 1048, 1054, 1057, 1061, 1064, 1069, 1076, 1081, 1085, 1089, 1095, 1101, 1102, 1106, 1111, 1113, 1117, 1121, 1125, 1127, 1130, 1134, 1134, 1136, 1142, 1145, 1148, 1148, 1151, 1152, 1154, 1154], "harmed": [0, 0, 1, 3, 3, 5, 5, 5, 6, 10, 12, 16, 20, 24, 29, 32, 40, 43, 53, 63, 67, 78, 82, 87, 92, 100, 106, 114, 118, 122, 126, 134, 141, 145, 153, 158, 166, 171, 175, 183, 191, 195, 198, 200, 202, 205, 208, 214, 216, 221, 224, 226, 228, 229, 232, 234, 243, 247, 252, 255, 260, 261, 265, 268, 271, 273, 277, 279, 280, 283, 286, 293, 295, 298, 302, 304, 305, 308, 308, 313, 317, 320, 323, 324, 326, 328, 332, 334, 340, 341, 348, 352, 361, 366, 370, 376, 384, 388, 396, 403, 408], "exact_score": [79.81953594958465, 80.2492122600974, 80.67888857061013, 81.09424233743913, 81.52391864795187, 81.91062732741334, 82.2686909195073, 82.68404468633629, 82.8845602979089, 83.17101117158407, 83.5004296763105, 83.82984818103695, 84.1162990547121, 84.44571755943855, 84.73216843311371, 85.06158693784016, 85.21913491836149, 85.4912632483529, 85.69177885992552, 85.86364938413061, 86.1071326267545, 86.26468060727585, 86.46519621884846, 86.68003437410484, 86.95216270409625, 87.16700085935261, 87.41048410197651, 87.45345173302778, 87.5823546261816, 87.76854769407046, 87.98338584932684, 88.04067602406187, 88.15525637353193, 88.28415926668576, 88.28415926668576, 88.47035233457461, 88.52764250930964, 88.64222285877972, 88.81409338298482, 88.84273847035233, 88.90002864508736, 88.98596390718991, 89.04325408192494, 89.17215697507876, 89.22944714981381, 89.37267258665138, 89.48725293612145, 89.573188198224, 89.74505872242909, 89.78802635348038, 89.81667144084788, 89.97421942136923, 90.11744485820681, 90.18905757662561, 90.24634775136063, 90.31796046977942, 90.28931538241191, 90.33228301346318, 90.34660555714694, 90.4325408192495, 90.41821827556574, 90.48983099398453, 90.54712116871956, 90.6044113434546, 90.64737897450587, 90.7046691492409, 90.66170151818963, 90.7046691492409, 90.76195932397593, 90.77628186765969, 90.81924949871096, 90.76195932397593, 90.79060441134345, 90.79060441134345, 90.80492695502721, 90.876539673446, 90.93382984818103, 90.9481523918648, 91.00544256659983, 91.01976511028357, 91.0484101976511, 91.01976511028357, 91.03408765396735, 91.09137782870236, 91.09137782870236, 91.12002291606989, 91.12002291606989, 91.14866800343741, 91.09137782870236, 91.12002291606989, 91.07705528501862, 91.01976511028357, 90.91950730449727, 90.93382984818103, 90.91950730449727, 90.876539673446, 90.76195932397593, 90.74763678029217, 90.64737897450587, 90.57576625608708, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.53342424654434, "threshold_area100": 88.53265539959897, "threshold_area25": 84.01460899455742}, "test": {"score": [80.00287005811867, 80.6342828442276, 81.23699504914975, 81.59575231398436, 82.04778646767598, 82.36349286073043, 82.70072468967497, 83.08818253569635, 83.40388892875082, 83.75547104828873, 83.98507569778289, 84.23620578316712, 84.47298557795796, 84.7097653727488, 84.88196885986942, 85.14027409055034, 85.4488053383081, 85.67840998780225, 85.9080146372964, 86.07304297912033, 86.3815742268781, 86.5609528592954, 86.79055750878956, 86.9340604147234, 87.09191361125063, 87.22106622659109, 87.28564253426131, 87.52959747434886, 87.6802755255794, 87.88835473918347, 88.08925880749085, 88.36908947406185, 88.49824208940231, 88.58434383296262, 88.72784673889646, 88.84264906364353, 89.10095429432447, 89.16553060199469, 89.28750807203846, 89.37360981559877, 89.45253641386238, 89.6175647556863, 89.76106766162016, 89.86869484107054, 89.93327114874077, 90.01219774700438, 90.10547463586138, 90.19875152471838, 90.24180239649853, 90.27050297768531, 90.33507928535553, 90.40683073832245, 90.40683073832245, 90.43553131950922, 90.39248044772907, 90.44270646480591, 90.42118102891584, 90.43553131950922, 90.50010762717945, 90.50728277247615, 90.54315849895961, 90.61490995192652, 90.60773480662984, 90.60055966133315, 90.58620937073975, 90.60773480662984, 90.59338451603645, 90.60773480662984, 90.65078567840999, 90.744062567267, 90.73688742197031, 90.744062567267, 90.744062567267, 90.744062567267, 90.69383655019014, 90.68666140489346, 90.67231111430006, 90.64361053311329, 90.62926024251992, 90.62926024251992, 90.57185908014638, 90.57185908014638, 90.59338451603645, 90.59338451603645, 90.64361053311329, 90.59338451603645, 90.60773480662984, 90.57903422544307, 90.57903422544307, 90.5503336442563, 90.54315849895961, 90.52163306306953, 90.53598335366291, 90.52163306306953, 90.47140704599268, 90.43553131950922, 90.37095501183899, 90.31355384946545, 90.24897754179523, 90.12700007175145, 90.06242376408122], "calls": [0.0, 1.1910741192509149, 2.389323383798522, 3.3938437253354383, 4.549042118102892, 5.43158498959604, 6.335653296979264, 7.275597330845949, 8.136614766449021, 9.234411996842937, 10.016502834182392, 11.049723756906078, 11.94661691899261, 12.70718232044199, 13.697352371385522, 15.010403960680204, 16.07950060988735, 17.012269498457343, 18.04549042118103, 18.798880677333717, 20.08323168544163, 21.238430078209085, 22.40080361627323, 23.19006959890938, 24.05826217980914, 24.87622874363206, 25.75877161512521, 26.799167683145587, 27.660185118748654, 28.69340604147234, 29.662050656525796, 30.723972160436247, 31.606515031929394, 32.42448159575231, 33.37160077491569, 34.54832460357322, 35.62459639807706, 36.71521848317428, 37.64798737174428, 38.70990887565473, 39.58527660185119, 40.604147233981486, 41.46516466958456, 42.591662481165244, 43.574657386812085, 44.66527947190931, 45.76307670230322, 47.04025256511444, 47.958671163091054, 49.207146444715505, 50.254717658032575, 51.48884264906365, 52.42161153763364, 53.53375905862093, 54.53110425486116, 55.63607663055177, 56.253139126067296, 57.236134031714144, 58.30523068092129, 59.28105044127143, 60.1564181674679, 61.1609385090048, 62.29461146588219, 63.449809858649644, 64.36822845662624, 65.50190141350363, 66.76472698572145, 67.61139413073116, 68.84551912176221, 69.9720169333429, 70.81150893305589, 71.80167898399942, 72.51919351366865, 73.39456123986511, 74.18382722250125, 75.0735452392911, 76.24309392265194, 77.24043911889216, 78.26648489631916, 79.24230465666929, 80.23964985290952, 81.33744708330343, 82.51417091196097, 83.44693980053096, 84.58061275740833, 85.50620650068164, 86.52507713281193, 87.40762000430509, 88.29733802109493, 88.95027624309392, 90.14852550764154, 91.1960967209586, 91.79880892588075, 93.04010906220851, 93.95852766018513, 94.84824567697497, 95.80254000143503, 96.7568343258951, 98.12728707756332, 99.13180741910024, 100.0], "rescued": [0, 90, 180, 236, 303, 352, 401, 458, 507, 562, 600, 643, 682, 720, 759, 811, 859, 899, 939, 970, 1024, 1064, 1107, 1130, 1159, 1183, 1199, 1245, 1276, 1312, 1350, 1399, 1425, 1443, 1469, 1497, 1539, 1560, 1584, 1605, 1623, 1654, 1678, 1706, 1720, 1741, 1764, 1791, 1803, 1821, 1836, 1855, 1862, 1881, 1891, 1904, 1908, 1920, 1937, 1948, 1957, 1971, 1982, 1992, 1999, 2011, 2024, 2031, 2043, 2061, 2070, 2075, 2079, 2087, 2089, 2096, 2103, 2108, 2116, 2124, 2129, 2136, 2144, 2151, 2165, 2169, 2178, 2183, 2193, 2195, 2203, 2207, 2214, 2226, 2233, 2235, 2236, 2237, 2242, 2243, 2247], "harmed": [0, 2, 8, 14, 18, 23, 25, 28, 33, 39, 45, 53, 59, 64, 79, 95, 100, 108, 116, 124, 135, 150, 161, 164, 171, 177, 184, 196, 206, 213, 223, 233, 241, 247, 253, 265, 271, 283, 290, 299, 306, 314, 318, 331, 336, 346, 356, 370, 376, 390, 396, 405, 412, 427, 443, 449, 456, 466, 474, 484, 488, 492, 504, 515, 524, 533, 548, 553, 559, 564, 574, 578, 582, 590, 599, 607, 616, 625, 635, 643, 656, 663, 668, 675, 682, 693, 700, 709, 719, 725, 734, 741, 746, 760, 774, 781, 791, 800, 814, 832, 845], "exact_score": [80.00287005811867, 80.54818110066729, 81.08631699791921, 81.4594245533472, 81.8325321087752, 82.21281480949989, 82.56439692903783, 82.98055535624597, 83.35366291167396, 83.6837195953218, 83.97790055248619, 84.22903063787042, 84.48016072325466, 84.76716653512234, 84.96089545813304, 85.14027409055034, 85.42010475712134, 85.66405969720887, 85.9080146372964, 86.14479443208725, 86.35287364569133, 86.53942742340533, 86.67575518404249, 86.86948410705317, 87.07756332065725, 87.22106622659109, 87.37174427782163, 87.53677261964555, 87.75202697854631, 87.945755901557, 88.18971084164454, 88.39061490995192, 88.55564325177585, 88.67762072181962, 88.77089761067663, 88.93592595250054, 89.13683002080792, 89.21575661907154, 89.3377340891153, 89.39513525148884, 89.5386381574227, 89.66779077276315, 89.79694338810361, 89.87586998636723, 89.97632202052091, 90.05524861878453, 90.10547463586138, 90.19157637942169, 90.24180239649853, 90.21310181531177, 90.29920355887207, 90.37095501183899, 90.42118102891584, 90.42835617421254, 90.41400588361914, 90.42835617421254, 90.44988161010261, 90.43553131950922, 90.47858219128938, 90.50010762717945, 90.5503336442563, 90.60773480662984, 90.60773480662984, 90.59338451603645, 90.61490995192652, 90.62208509722322, 90.61490995192652, 90.60773480662984, 90.6364353878166, 90.65796082370669, 90.744062567267, 90.744062567267, 90.73688742197031, 90.73688742197031, 90.68666140489346, 90.67948625959676, 90.67948625959676, 90.65796082370669, 90.62926024251992, 90.64361053311329, 90.56468393484968, 90.56468393484968, 90.60055966133315, 90.60055966133315, 90.64361053311329, 90.62208509722322, 90.61490995192652, 90.56468393484968, 90.57185908014638, 90.5503336442563, 90.52880820836621, 90.51445791777284, 90.53598335366291, 90.52880820836621, 90.45705675539931, 90.41400588361914, 90.3566047212456, 90.30637870416876, 90.25615268709191, 90.12700007175145, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.62294685600985, "threshold_area100": 88.62595967568345, "threshold_area25": 84.29647700365933}, "selected_budget": 87, "selected": {"val": {"score": 91.14866800343741, "calls": 86.99513033514752, "rescued": 1125, "harmed": 334}, "test": {"score": 90.57903422544307, "calls": 87.40762000430509, "rescued": 2183, "harmed": 709}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.126455783843994, "operator": "ge"}, {"budget": 0.02, "threshold": 1.8385742902755737, "operator": "ge"}, {"budget": 0.03, "threshold": 1.6522715091705322, "operator": "ge"}, {"budget": 0.04, "threshold": 1.520411491394043, "operator": "ge"}, {"budget": 0.05, "threshold": 1.3585439920425415, "operator": "ge"}, {"budget": 0.06, "threshold": 1.2684996128082275, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1658395528793335, "operator": "ge"}, {"budget": 0.08, "threshold": 1.0734776258468628, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9843531847000122, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9085967540740967, "operator": "ge"}, {"budget": 0.11, "threshold": 0.8330629467964172, "operator": "ge"}, {"budget": 0.12, "threshold": 0.7614891529083252, "operator": "ge"}, {"budget": 0.13, "threshold": 0.7051299810409546, "operator": "ge"}, {"budget": 0.14, "threshold": 0.6491568088531494, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5883549451828003, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5401314496994019, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4799816608428955, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4297063946723938, "operator": "ge"}, {"budget": 0.19, "threshold": 0.3584127426147461, "operator": "ge"}, {"budget": 0.2, "threshold": 0.28808215260505676, "operator": "ge"}, {"budget": 0.21, "threshold": 0.22507056593894958, "operator": "ge"}, {"budget": 0.22, "threshold": 0.16748970746994019, "operator": "ge"}, {"budget": 0.23, "threshold": 0.11617118120193481, "operator": "ge"}, {"budget": 0.24, "threshold": 0.056002676486968994, "operator": "ge"}, {"budget": 0.25, "threshold": 0.006397511810064316, "operator": "ge"}, {"budget": 0.26, "threshold": -0.05573296546936035, "operator": "ge"}, {"budget": 0.27, "threshold": -0.12664440274238586, "operator": "ge"}, {"budget": 0.28, "threshold": -0.18795347213745117, "operator": "ge"}, {"budget": 0.29, "threshold": -0.26154768466949463, "operator": "ge"}, {"budget": 0.3, "threshold": -0.329148530960083, "operator": "ge"}, {"budget": 0.31, "threshold": -0.3924853205680847, "operator": "ge"}, {"budget": 0.32, "threshold": -0.4787769317626953, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5316482782363892, "operator": "ge"}, {"budget": 0.34, "threshold": -0.60454261302948, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6796891689300537, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7632539868354797, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8414006233215332, "operator": "ge"}, {"budget": 0.38, "threshold": -0.910802960395813, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9739214181900024, "operator": "ge"}, {"budget": 0.4, "threshold": -1.0439825057983398, "operator": "ge"}, {"budget": 0.41, "threshold": -1.106361985206604, "operator": "ge"}, {"budget": 0.42, "threshold": -1.1840205192565918, "operator": "ge"}, {"budget": 0.43, "threshold": -1.2563797235488892, "operator": "ge"}, {"budget": 0.44, "threshold": -1.3320550918579102, "operator": "ge"}, {"budget": 0.45, "threshold": -1.41163969039917, "operator": "ge"}, {"budget": 0.46, "threshold": -1.4785746335983276, "operator": "ge"}, {"budget": 0.47, "threshold": -1.5460574626922607, "operator": "ge"}, {"budget": 0.48, "threshold": -1.6123926639556885, "operator": "ge"}, {"budget": 0.49, "threshold": -1.6828471422195435, "operator": "ge"}, {"budget": 0.5, "threshold": -1.748199224472046, "operator": "ge"}, {"budget": 0.51, "threshold": -1.8332328796386719, "operator": "ge"}, {"budget": 0.52, "threshold": -1.906355619430542, "operator": "ge"}, {"budget": 0.53, "threshold": -1.970712661743164, "operator": "ge"}, {"budget": 0.54, "threshold": -2.0347702503204346, "operator": "ge"}, {"budget": 0.55, "threshold": -2.0970606803894043, "operator": "ge"}, {"budget": 0.56, "threshold": -2.1628072261810303, "operator": "ge"}, {"budget": 0.57, "threshold": -2.249922275543213, "operator": "ge"}, {"budget": 0.58, "threshold": -2.3171212673187256, "operator": "ge"}, {"budget": 0.59, "threshold": -2.383174419403076, "operator": "ge"}, {"budget": 0.6, "threshold": -2.462315320968628, "operator": "ge"}, {"budget": 0.61, "threshold": -2.5439646244049072, "operator": "ge"}, {"budget": 0.62, "threshold": -2.6110048294067383, "operator": "ge"}, {"budget": 0.63, "threshold": -2.687049388885498, "operator": "ge"}, {"budget": 0.64, "threshold": -2.7661659717559814, "operator": "ge"}, {"budget": 0.65, "threshold": -2.834439277648926, "operator": "ge"}, {"budget": 0.66, "threshold": -2.9142308235168457, "operator": "ge"}, {"budget": 0.67, "threshold": -2.9860053062438965, "operator": "ge"}, {"budget": 0.68, "threshold": -3.0573172569274902, "operator": "ge"}, {"budget": 0.69, "threshold": -3.130084276199341, "operator": "ge"}, {"budget": 0.7, "threshold": -3.195298671722412, "operator": "ge"}, {"budget": 0.71, "threshold": -3.2596452236175537, "operator": "ge"}, {"budget": 0.72, "threshold": -3.3349640369415283, "operator": "ge"}, {"budget": 0.73, "threshold": -3.3878467082977295, "operator": "ge"}, {"budget": 0.74, "threshold": -3.4535555839538574, "operator": "ge"}, {"budget": 0.75, "threshold": -3.5219786167144775, "operator": "ge"}, {"budget": 0.76, "threshold": -3.5901315212249756, "operator": "ge"}, {"budget": 0.77, "threshold": -3.655898332595825, "operator": "ge"}, {"budget": 0.78, "threshold": -3.739445924758911, "operator": "ge"}, {"budget": 0.79, "threshold": -3.808391571044922, "operator": "ge"}, {"budget": 0.8, "threshold": -3.8891401290893555, "operator": "ge"}, {"budget": 0.81, "threshold": -3.9839444160461426, "operator": "ge"}, {"budget": 0.82, "threshold": -4.055143356323242, "operator": "ge"}, {"budget": 0.83, "threshold": -4.147771835327148, "operator": "ge"}, {"budget": 0.84, "threshold": -4.224863052368164, "operator": "ge"}, {"budget": 0.85, "threshold": -4.309074878692627, "operator": "ge"}, {"budget": 0.86, "threshold": -4.387787342071533, "operator": "ge"}, {"budget": 0.87, "threshold": -4.473258972167969, "operator": "ge"}, {"budget": 0.88, "threshold": -4.5464043617248535, "operator": "ge"}, {"budget": 0.89, "threshold": -4.630141258239746, "operator": "ge"}, {"budget": 0.9, "threshold": -4.721186637878418, "operator": "ge"}, {"budget": 0.91, "threshold": -4.822237014770508, "operator": "ge"}, {"budget": 0.92, "threshold": -4.940651893615723, "operator": "ge"}, {"budget": 0.93, "threshold": -5.042342185974121, "operator": "ge"}, {"budget": 0.94, "threshold": -5.200364112854004, "operator": "ge"}, {"budget": 0.95, "threshold": -5.3542656898498535, "operator": "ge"}, {"budget": 0.96, "threshold": -5.523026466369629, "operator": "ge"}, {"budget": 0.97, "threshold": -5.733850955963135, "operator": "ge"}, {"budget": 0.98, "threshold": -6.067453384399414, "operator": "ge"}, {"budget": 0.99, "threshold": -6.540841579437256, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.3494700658837, 80.82211400744772, 81.25179031796046, 81.63849899742193, 82.05385276425092, 82.38327126897737, 82.79862503580635, 83.07075336579776, 83.32855915210541, 83.60068748209682, 83.93010598682325, 84.21655686049841, 84.4313950157548, 84.66055571469494, 84.90403895731882, 85.11887711257519, 85.24778000572901, 85.51990833572042, 85.84932684044686, 86.04984245201948, 86.37926095674591, 86.57977656831852, 86.68003437410484, 86.96648524778, 87.16700085935261, 87.39616155829275, 87.46777427671154, 87.62532225723288, 87.78287023775422, 88.04067602406187, 88.08364365511314, 88.18390146089945, 88.22686909195072, 88.38441707247206, 88.47035233457461, 88.47035233457461, 88.62790031509596, 88.81409338298482, 88.90002864508736, 88.97164136350615, 89.02893153824118, 89.07189916929245, 89.17215697507876, 89.35835004296763, 89.44428530507018, 89.573188198224, 89.6734460040103, 89.71641363506157, 89.8453165282154, 89.87396161558291, 89.98854196505299, 90.07447722715554, 90.17473503294184, 90.24634775136063, 90.27499283872815, 90.31796046977942, 90.31796046977942, 90.27499283872815, 90.3609281008307, 90.47550845030077, 90.53279862503581, 90.57576625608708, 90.61873388713835, 90.63305643082211, 90.66170151818963, 90.67602406187338, 90.69034660555714, 90.74763678029217, 90.79060441134345, 90.80492695502721, 90.83357204239474, 90.84789458607848, 90.81924949871096, 90.81924949871096, 90.90518476081351, 90.96247493554854, 91.03408765396735, 91.06273274133486, 91.03408765396735, 91.03408765396735, 91.06273274133486, 91.03408765396735, 91.07705528501862, 91.10570037238614, 91.14866800343741, 91.14866800343741, 91.13434545975365, 91.13434545975365, 91.10570037238614, 91.07705528501862, 91.01976511028357, 90.9767974792323, 90.91950730449727, 90.93382984818103, 90.89086221712975, 90.76195932397593, 90.71899169292466, 90.64737897450587, 90.59008879977084, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 38, 73, 103, 132, 161, 185, 215, 238, 261, 283, 309, 333, 353, 370, 393, 414, 432, 458, 487, 506, 534, 553, 568, 591, 613, 635, 648, 663, 679, 698, 710, 723, 732, 746, 759, 769, 784, 803, 814, 826, 833, 840, 852, 867, 875, 886, 897, 905, 917, 925, 934, 943, 951, 959, 964, 970, 976, 978, 988, 999, 1008, 1012, 1020, 1024, 1028, 1033, 1036, 1043, 1049, 1053, 1058, 1063, 1066, 1070, 1078, 1082, 1089, 1093, 1094, 1098, 1104, 1106, 1111, 1115, 1119, 1121, 1125, 1128, 1130, 1133, 1137, 1141, 1142, 1146, 1149, 1149, 1151, 1152, 1154, 1154], "harmed": [0, 1, 3, 3, 5, 5, 6, 7, 11, 16, 19, 22, 26, 31, 32, 38, 44, 53, 60, 66, 71, 76, 81, 89, 92, 100, 106, 114, 118, 123, 124, 133, 139, 145, 148, 155, 165, 169, 175, 180, 187, 190, 194, 199, 201, 203, 205, 209, 214, 217, 223, 224, 227, 228, 231, 234, 237, 243, 248, 252, 255, 260, 261, 266, 269, 271, 275, 277, 280, 283, 286, 289, 293, 298, 302, 304, 304, 306, 308, 311, 315, 319, 323, 325, 327, 328, 330, 335, 338, 342, 347, 355, 362, 367, 370, 376, 385, 390, 396, 402, 408], "exact_score": [79.81953594958465, 80.3494700658837, 80.82211400744772, 81.25179031796046, 81.63849899742193, 82.05385276425092, 82.38327126897737, 82.79862503580635, 83.07075336579776, 83.32855915210541, 83.60068748209682, 83.93010598682325, 84.21655686049841, 84.4313950157548, 84.66055571469494, 84.90403895731882, 85.11887711257519, 85.24778000572901, 85.51990833572042, 85.84932684044686, 86.04984245201948, 86.37926095674591, 86.57977656831852, 86.68003437410484, 86.96648524778, 87.16700085935261, 87.39616155829275, 87.46777427671154, 87.62532225723288, 87.78287023775422, 88.04067602406187, 88.08364365511314, 88.18390146089945, 88.22686909195072, 88.38441707247206, 88.47035233457461, 88.47035233457461, 88.62790031509596, 88.81409338298482, 88.90002864508736, 88.97164136350615, 89.02893153824118, 89.07189916929245, 89.17215697507876, 89.35835004296763, 89.44428530507018, 89.573188198224, 89.6734460040103, 89.71641363506157, 89.8453165282154, 89.87396161558291, 89.98854196505299, 90.07447722715554, 90.17473503294184, 90.24634775136063, 90.27499283872815, 90.31796046977942, 90.31796046977942, 90.27499283872815, 90.3609281008307, 90.47550845030077, 90.53279862503581, 90.57576625608708, 90.61873388713835, 90.63305643082211, 90.66170151818963, 90.67602406187338, 90.69034660555714, 90.74763678029217, 90.79060441134345, 90.80492695502721, 90.83357204239474, 90.84789458607848, 90.81924949871096, 90.81924949871096, 90.90518476081351, 90.96247493554854, 91.03408765396735, 91.06273274133486, 91.03408765396735, 91.03408765396735, 91.06273274133486, 91.03408765396735, 91.07705528501862, 91.10570037238614, 91.14866800343741, 91.14866800343741, 91.13434545975365, 91.13434545975365, 91.10570037238614, 91.07705528501862, 91.01976511028357, 90.9767974792323, 90.91950730449727, 90.93382984818103, 90.89086221712975, 90.76195932397593, 90.71899169292466, 90.64737897450587, 90.59008879977084, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.55517166087596, "threshold_area100": 88.55442566599827, "threshold_area25": 84.02033801203093}, "test": {"score": [80.00287005811867, 80.58405682715075, 81.1006672885126, 81.50247542512736, 81.85405754466528, 82.25586568128006, 82.55722178374111, 82.83705245031211, 83.33931262108058, 83.6908947406185, 83.95637511659612, 84.22185549257372, 84.39405897969434, 84.66671450096864, 84.8963191504628, 85.20485039822057, 85.38422903063787, 85.67123484250556, 85.87213891081295, 86.08021812441702, 86.34569850039465, 86.5753031498888, 86.78338236349286, 86.91971012413002, 87.08473846595393, 87.22106622659109, 87.27846738896463, 87.50089689316209, 87.72332639735954, 87.91705532037024, 88.1394848245677, 88.35473918346847, 88.505417234699, 88.62021955944608, 88.69914615770969, 88.85699935423692, 89.07942885843438, 89.23010690966493, 89.29468321733516, 89.41666068737892, 89.51711272153261, 89.71084164454331, 89.8184688239937, 89.82564396929038, 89.93327114874077, 90.04089832819116, 90.06959890937792, 90.1700509435316, 90.22027696060846, 90.32072899476215, 90.37095501183899, 90.34225443065222, 90.44270646480591, 90.46423190069599, 90.50010762717945, 90.52880820836621, 90.47140704599268, 90.49293248188276, 90.47858219128938, 90.50728277247615, 90.55750878955298, 90.59338451603645, 90.61490995192652, 90.65078567840999, 90.6364353878166, 90.62926024251992, 90.60773480662984, 90.6364353878166, 90.66513596900337, 90.71536198608023, 90.70101169548683, 90.7297122766736, 90.72253713137691, 90.73688742197031, 90.75123771256368, 90.70101169548683, 90.69383655019014, 90.65796082370669, 90.65078567840999, 90.60773480662984, 90.62926024251992, 90.60055966133315, 90.61490995192652, 90.64361053311329, 90.61490995192652, 90.62926024251992, 90.59338451603645, 90.55750878955298, 90.55750878955298, 90.55750878955298, 90.53598335366291, 90.52880820836621, 90.52163306306953, 90.50010762717945, 90.45705675539931, 90.40683073832245, 90.3566047212456, 90.30637870416876, 90.24897754179523, 90.12700007175145, 90.06242376408122], "calls": [0.0, 1.1838989739542225, 2.231470187271292, 3.0781373322809786, 3.874578460213819, 5.080002870058118, 5.897969433881036, 6.880964339527876, 8.043337877592021, 9.047858219128937, 10.002152543589007, 10.99949773982923, 12.047068953146303, 12.922436679342756, 13.75475353375906, 14.766449020592667, 15.5700652938222, 16.703738250699576, 17.557580541005954, 18.7199540790701, 20.14063284781517, 21.3173566764727, 22.343402453899692, 23.089617564755684, 24.115663342182682, 24.861878453038674, 25.73007103393844, 26.82069311903566, 27.674535409342038, 28.772332639735954, 29.791203271866255, 30.63787041687594, 31.764368228456625, 32.46035732223578, 33.42182679199254, 34.57702518475999, 35.83985075697783, 36.83002080792136, 37.848891440051666, 38.77448518332496, 39.829231541938725, 40.704599268135176, 41.78804620793571, 42.792566549472625, 43.840137762789695, 45.131663916194306, 46.07160795006099, 47.090478582191295, 48.130874650211666, 49.1210447011552, 49.96053670086819, 51.17313625600919, 52.19918203343617, 53.28262897323671, 54.136471263543086, 55.06206500681638, 56.00918418597976, 57.1859080146373, 58.20477864676759, 59.11602209944752, 60.185118748654666, 61.13223792781803, 62.021955944607875, 63.11975317500179, 64.38257874721963, 65.27229676400947, 66.41314486618354, 67.40331491712708, 68.46523642103752, 69.62043481380498, 70.68953146301212, 71.63665064217551, 72.64834612900911, 73.32998493219488, 74.0474994618641, 75.06637009399441, 76.01348927315779, 77.01800961469469, 78.19473344335222, 79.24230465666929, 80.28987586998637, 81.53117600631413, 82.37784315132382, 83.69089474061849, 84.5303867403315, 85.57078280835186, 86.51790198751524, 87.54394776494225, 88.29016287579825, 89.0722537131377, 89.93327114874076, 90.97366721676114, 91.94231183181459, 92.86790557508789, 94.04462940374543, 94.96304800172204, 95.8312405826218, 96.89316208653224, 98.1488125134534, 99.12463227380354, 100.0], "rescued": [0, 87, 162, 219, 271, 339, 386, 432, 506, 560, 607, 653, 687, 730, 768, 816, 849, 900, 932, 975, 1030, 1073, 1106, 1128, 1159, 1183, 1198, 1241, 1280, 1315, 1354, 1390, 1424, 1444, 1461, 1494, 1533, 1560, 1581, 1607, 1630, 1660, 1684, 1696, 1721, 1745, 1757, 1779, 1794, 1815, 1830, 1844, 1862, 1872, 1886, 1898, 1906, 1919, 1927, 1943, 1957, 1966, 1975, 1986, 1997, 2007, 2015, 2028, 2037, 2051, 2057, 2069, 2076, 2081, 2089, 2094, 2102, 2107, 2116, 2122, 2131, 2140, 2147, 2161, 2164, 2174, 2179, 2185, 2190, 2196, 2200, 2207, 2215, 2222, 2231, 2235, 2236, 2237, 2242, 2243, 2247], "harmed": [0, 6, 9, 10, 13, 25, 30, 37, 41, 46, 56, 65, 75, 80, 86, 91, 99, 110, 114, 128, 146, 157, 161, 164, 172, 177, 184, 196, 204, 212, 220, 226, 239, 243, 249, 260, 268, 274, 286, 295, 304, 307, 316, 327, 337, 346, 354, 362, 370, 377, 385, 403, 407, 414, 423, 431, 447, 457, 467, 479, 486, 490, 496, 502, 515, 526, 537, 546, 551, 558, 566, 574, 582, 585, 591, 603, 612, 622, 632, 644, 650, 663, 668, 678, 685, 693, 703, 714, 719, 725, 732, 740, 749, 759, 774, 785, 793, 801, 814, 832, 845], "exact_score": [80.00287005811867, 80.49077993829376, 80.96433952787544, 81.44507426275382, 81.88275812585205, 82.22716510009327, 82.5787472196312, 82.90162875798235, 83.31778718519051, 83.67654445002512, 83.95637511659612, 84.22185549257372, 84.37970868910097, 84.68823993685872, 84.96807060342972, 85.24790127000072, 85.51338164597834, 85.75733658606588, 85.93671521848317, 86.15914472268064, 86.33134820980125, 86.52507713281194, 86.73315634641602, 86.91971012413002, 87.08473846595393, 87.22106622659109, 87.34304369663487, 87.53677261964555, 87.75920212384301, 87.945755901557, 88.21841142283131, 88.43366578173209, 88.56281839707253, 88.66327043122624, 88.74937217478654, 88.91440051661046, 89.10812943962115, 89.23010690966493, 89.32338379852193, 89.438186123269, 89.56733873860946, 89.73954222573008, 89.811293678697, 89.82564396929038, 89.96197172992754, 90.04807347348785, 90.06959890937792, 90.15570065293824, 90.21310181531177, 90.31355384946545, 90.3637798665423, 90.33507928535553, 90.41400588361914, 90.44988161010261, 90.50010762717945, 90.52880820836621, 90.47140704599268, 90.48575733658608, 90.47858219128938, 90.49293248188276, 90.5503336442563, 90.59338451603645, 90.61490995192652, 90.65796082370669, 90.65078567840999, 90.6364353878166, 90.65078567840999, 90.62208509722322, 90.64361053311329, 90.69383655019014, 90.70101169548683, 90.67948625959676, 90.73688742197031, 90.73688742197031, 90.75123771256368, 90.70818684078354, 90.69383655019014, 90.65796082370669, 90.64361053311329, 90.60773480662984, 90.62208509722322, 90.61490995192652, 90.60773480662984, 90.62208509722322, 90.6364353878166, 90.6364353878166, 90.6364353878166, 90.57903422544307, 90.56468393484968, 90.55750878955298, 90.52880820836621, 90.52163306306953, 90.52163306306953, 90.50010762717945, 90.46423190069599, 90.40683073832245, 90.34942957594892, 90.31355384946545, 90.24897754179523, 90.12700007175145, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.62940330267455, "threshold_area100": 88.62122407978762, "threshold_area25": 84.27495156776925}, "selected_budget": 85, "selected": {"val": {"score": 91.14866800343741, "calls": 84.98997421942137, "rescued": 1119, "harmed": 328}, "test": {"score": 90.62926024251992, "calls": 85.57078280835186, "rescued": 2174, "harmed": 693}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.99112002291606, "calls": 96.99226582641077}, "test": {"score": 90.67948625959676, "calls": 96.79988519767525}}, "old_normalized_interior": {"val": {"score": 91.06273274133486, "calls": 82.99914064737898}, "test": {"score": 90.62926024251992, "calls": 82.98055535624597}}, "old_normalized_endpoints": {"val": {"score": 91.06273274133486, "calls": 82.99914064737898}, "test": {"score": 90.62926024251992, "calls": 82.98055535624597}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.37811515325122, 80.87940418218274, 81.23746777427671, 81.60985391005443, 81.98224004583213, 82.39759381266111, 82.75565740475508, 83.05643082211401, 83.35720423947292, 83.64365511314809, 83.8441707247207, 84.13062159839588, 84.33113720996849, 84.57462045259237, 84.81810369521627, 84.98997421942136, 85.37668289888283, 85.62016614150673, 85.8063592093956, 86.078487539387, 86.23603551990833, 86.50816384989973, 86.78029217989113, 86.88054998567745, 87.02377542251504, 87.2386135777714, 87.39616155829275, 87.4820968203953, 87.68261243196791, 87.8258378688055, 87.94041821827557, 88.08364365511314, 88.2411916356345, 88.2411916356345, 88.41306215983958, 88.49899742194212, 88.58493268404469, 88.65654540246346, 88.72815812088226, 88.74248066456603, 88.81409338298482, 88.92867373245488, 89.10054425665997, 89.18647951876252, 89.22944714981381, 89.22944714981381, 89.30105986823258, 89.38699513033515, 89.55886565454024, 89.70209109137782, 89.75938126611285, 89.87396161558291, 89.95989687768548, 90.06015468347178, 90.17473503294184, 90.18905757662561, 90.23202520767688, 90.27499283872815, 90.31796046977942, 90.40389573188197, 90.48983099398453, 90.59008879977084, 90.69034660555714, 90.64737897450587, 90.7046691492409, 90.76195932397593, 90.67602406187338, 90.6044113434546, 90.59008879977084, 90.59008879977084, 90.66170151818963, 90.64737897450587, 90.66170151818963, 90.69034660555714, 90.73331423660842, 90.7046691492409, 90.69034660555714, 90.71899169292466, 90.81924949871096, 90.84789458607848, 90.90518476081351, 90.99112002291606, 91.06273274133486, 91.03408765396735, 90.99112002291606, 90.9767974792323, 90.99112002291606, 90.9767974792323, 90.96247493554854, 90.93382984818103, 90.83357204239474, 90.74763678029217, 90.67602406187338, 90.64737897450587, 90.66170151818963, 90.56144371240332, 90.56144371240332, 90.51847608135203, 90.54712116871956, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.37811515325122, 80.87940418218274, 81.23746777427671, 81.60985391005443, 81.98224004583213, 82.39759381266111, 82.75565740475508, 83.05643082211401, 83.35720423947292, 83.64365511314809, 83.8441707247207, 84.13062159839588, 84.33113720996849, 84.57462045259237, 84.81810369521627, 84.98997421942136, 85.37668289888283, 85.62016614150673, 85.8063592093956, 86.078487539387, 86.23603551990833, 86.50816384989973, 86.78029217989113, 86.88054998567745, 87.02377542251504, 87.2386135777714, 87.39616155829275, 87.4820968203953, 87.68261243196791, 87.8258378688055, 87.94041821827557, 88.08364365511314, 88.2411916356345, 88.2411916356345, 88.41306215983958, 88.49899742194212, 88.58493268404469, 88.65654540246346, 88.72815812088226, 88.74248066456603, 88.81409338298482, 88.92867373245488, 89.10054425665997, 89.18647951876252, 89.22944714981381, 89.22944714981381, 89.30105986823258, 89.38699513033515, 89.55886565454024, 89.70209109137782, 89.75938126611285, 89.87396161558291, 89.95989687768548, 90.06015468347178, 90.17473503294184, 90.18905757662561, 90.23202520767688, 90.27499283872815, 90.31796046977942, 90.40389573188197, 90.48983099398453, 90.59008879977084, 90.69034660555714, 90.64737897450587, 90.7046691492409, 90.76195932397593, 90.67602406187338, 90.6044113434546, 90.59008879977084, 90.59008879977084, 90.66170151818963, 90.64737897450587, 90.66170151818963, 90.69034660555714, 90.73331423660842, 90.7046691492409, 90.69034660555714, 90.71899169292466, 90.81924949871096, 90.84789458607848, 90.90518476081351, 90.99112002291606, 91.06273274133486, 91.03408765396735, 90.99112002291606, 90.9767974792323, 90.99112002291606, 90.9767974792323, 90.96247493554854, 90.93382984818103, 90.83357204239474, 90.74763678029217, 90.67602406187338, 90.64737897450587, 90.66170151818963, 90.56144371240332, 90.56144371240332, 90.51847608135203, 90.54712116871956, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 83.98882841592666, "exact_area100": 88.440709264179}, "test": {"score": [80.00287005811867, 80.61275740833752, 81.02174069024898, 81.4594245533472, 81.8468823993686, 82.18411422831313, 82.53569634785106, 82.87292817679558, 83.15993398866327, 83.58326756116811, 84.01377627896966, 84.25055607376049, 84.48733586855134, 84.78869197101243, 85.05417234699003, 85.18332496233049, 85.37705388534118, 85.57795795364856, 85.75733658606588, 85.97259094496664, 86.10891870560378, 86.31699791920788, 86.53942742340533, 86.66858003874579, 86.94841070531677, 87.09908875654733, 87.27129224366794, 87.41479514960179, 87.5941737820191, 87.6731003802827, 87.88117959388678, 88.06773337160078, 88.11795938867762, 88.29733802109493, 88.51259237999571, 88.67762072181962, 88.7924230465667, 88.8856999354237, 88.99332711487408, 89.12965487551124, 89.28033292674176, 89.3377340891153, 89.4310109779723, 89.51711272153261, 89.5386381574227, 89.63909019157639, 89.72519193513669, 89.8184688239937, 89.94044629403746, 89.97632202052091, 90.06959890937792, 90.14135036234484, 90.18440123412499, 90.1700509435316, 90.23462725120184, 90.30637870416876, 90.37813015713569, 90.3637798665423, 90.46423190069599, 90.50728277247615, 90.52163306306953, 90.52163306306953, 90.57903422544307, 90.60055966133315, 90.61490995192652, 90.65796082370669, 90.65078567840999, 90.6364353878166, 90.65078567840999, 90.68666140489346, 90.68666140489346, 90.67948625959676, 90.744062567267, 90.75123771256368, 90.81581402023392, 90.79428858434385, 90.79428858434385, 90.75123771256368, 90.70818684078354, 90.69383655019014, 90.67948625959676, 90.67948625959676, 90.65796082370669, 90.62926024251992, 90.60773480662984, 90.59338451603645, 90.61490995192652, 90.5503336442563, 90.56468393484968, 90.5503336442563, 90.52163306306953, 90.47140704599268, 90.42118102891584, 90.42118102891584, 90.41400588361914, 90.37095501183899, 90.33507928535553, 90.23462725120184, 90.1700509435316, 90.12700007175145, 90.06242376408122], "calls": [0.0, 1.140848102174069, 2.0520915548539858, 3.200114802324747, 4.276386596828586, 5.173279758915118, 6.249551553418957, 7.311473057329411, 8.315993398866327, 9.255937432733013, 10.568989022027695, 11.430006457630768, 12.291023893233838, 13.460572576594675, 14.472268063428285, 15.22565831958097, 16.237353806414582, 17.25622443854488, 18.27509507067518, 19.365717155772405, 20.30566118963909, 21.32453182176939, 22.393628470976537, 23.29052163306307, 24.287866829303294, 25.313912606730288, 26.124704025256513, 27.157924947980195, 28.02611752887996, 28.894310109779724, 29.99210734017364, 31.018153117600633, 31.92939657028055, 32.94826720241085, 33.93126210805769, 34.92860730429791, 35.83267561168114, 36.686517901987514, 37.604936499964126, 38.408552773193655, 39.46329913180742, 40.266915405036954, 41.25708545598049, 42.27595608811078, 43.51725622443855, 44.64375403601923, 45.46889574513884, 46.437540360192294, 47.30573294109205, 48.34612900911243, 49.472626820693115, 50.44127143574657, 51.45296692258018, 52.435961828227015, 53.64856138336801, 54.782234340245395, 55.63607663055177, 56.662122407978764, 57.70251847599914, 58.76443997990959, 59.79048575733658, 60.95285929540073, 61.792351295113725, 62.95472483317786, 63.90901915763795, 64.93506493506493, 65.97546100308531, 66.80777785750162, 67.86252421611537, 68.93162086532251, 69.6921862667719, 70.69670660830883, 71.6510009327689, 72.60529525722896, 73.6815670517328, 74.67891224797302, 75.65473200832317, 76.62337662337663, 77.9507785032647, 79.04857573365861, 80.06027122049221, 81.10784243380928, 82.08366219415943, 82.98055535624597, 83.6837195953218, 84.65236421037527, 85.72863600487909, 86.85513381645978, 87.80942814091985, 88.89287508072037, 89.85434455047715, 91.03106837913467, 92.0284135753749, 92.83920499390112, 93.8796010619215, 94.8051948051948, 95.7307885484681, 96.6707325823348, 97.80440553921217, 98.85915189782594, 100.0], "exact_score": [80.00287005811867, 80.55535624596398, 80.98586496376552, 81.40202339097367, 81.72490492932482, 82.09083733945613, 82.44959460429074, 82.72942527086174, 83.03795651861951, 83.46846523642104, 83.82722250125565, 84.11422831312333, 84.4227595608811, 84.65953935567195, 84.93219487694627, 85.14744923584703, 85.32682786826433, 85.52773193657171, 85.69993542369232, 85.90083949199972, 86.08021812441702, 86.23807132094426, 86.47485111573509, 86.61835402166894, 86.89100954294324, 87.0273373035804, 87.22824137188778, 87.36456913252493, 87.5798234914257, 87.70180096146947, 87.88117959388678, 88.06773337160078, 88.12513453397432, 88.29733802109493, 88.5197675252924, 88.67762072181962, 88.81394848245677, 88.91440051661046, 89.065078567841, 89.22293176436823, 89.3377340891153, 89.37360981559877, 89.50276243093923, 89.51711272153261, 89.58168902920285, 89.66779077276315, 89.79694338810361, 89.88304513166392, 89.96197172992754, 90.01937289230108, 90.11264978115807, 90.1772260888283, 90.20592667001507, 90.20592667001507, 90.26332783238861, 90.32072899476215, 90.39248044772907, 90.40683073832245, 90.48575733658608, 90.50728277247615, 90.50728277247615, 90.52880820836621, 90.59338451603645, 90.60055966133315, 90.61490995192652, 90.66513596900337, 90.65796082370669, 90.6364353878166, 90.65796082370669, 90.68666140489346, 90.67948625959676, 90.69383655019014, 90.744062567267, 90.77993829375045, 90.82298916553061, 90.79428858434385, 90.76558800315708, 90.73688742197031, 90.70818684078354, 90.70101169548683, 90.68666140489346, 90.69383655019014, 90.67231111430006, 90.62926024251992, 90.57903422544307, 90.60773480662984, 90.59338451603645, 90.55750878955298, 90.5503336442563, 90.54315849895961, 90.52163306306953, 90.45705675539931, 90.42118102891584, 90.42835617421254, 90.40683073832245, 90.3566047212456, 90.32790414005885, 90.21310181531177, 90.16287579823492, 90.11982492645475, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 84.20822271651002, "exact_area100": 88.50683995939066}}, "old_normalized_endpoint_budget": 83}, {"seed": 2027, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 1.0, "validation_area100": 0.8854917760864969, "training_seconds": 33.01176838495303, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8436836559738317, "operator": "ge"}, {"budget": 0.02, "threshold": 0.7948945653977728, "operator": "ge"}, {"budget": 0.03, "threshold": 0.7486351468494248, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7092448312899717, "operator": "ge"}, {"budget": 0.05, "threshold": 0.6709603241093741, "operator": "ge"}, {"budget": 0.06, "threshold": 0.6370129233472946, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5967327015357714, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5670147314730578, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5353058001580681, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4917291549081901, "operator": "ge"}, {"budget": 0.11, "threshold": 0.46433211162101634, "operator": "ge"}, {"budget": 0.12, "threshold": 0.4306625981708827, "operator": "ge"}, {"budget": 0.13, "threshold": 0.39962760400912506, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3708307598417911, "operator": "ge"}, {"budget": 0.15, "threshold": 0.33647802092552775, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3075955972982703, "operator": "ge"}, {"budget": 0.17, "threshold": 0.2814320500865325, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2481302001400487, "operator": "ge"}, {"budget": 0.19, "threshold": 0.21538300954825018, "operator": "ge"}, {"budget": 0.2, "threshold": 0.18114884287208008, "operator": "ge"}, {"budget": 0.21, "threshold": 0.1475932298098975, "operator": "ge"}, {"budget": 0.22, "threshold": 0.11265290146872634, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0821716039172629, "operator": "ge"}, {"budget": 0.24, "threshold": 0.055715470483894476, "operator": "ge"}, {"budget": 0.25, "threshold": 0.027256636734390875, "operator": "ge"}, {"budget": 0.26, "threshold": -0.007496535272242817, "operator": "ge"}, {"budget": 0.27, "threshold": -0.03545685223738373, "operator": "ge"}, {"budget": 0.28, "threshold": -0.06663070997049249, "operator": "ge"}, {"budget": 0.29, "threshold": -0.09834275149661484, "operator": "ge"}, {"budget": 0.3, "threshold": -0.13170838683859065, "operator": "ge"}, {"budget": 0.31, "threshold": -0.16703339326696182, "operator": "ge"}, {"budget": 0.32, "threshold": -0.19422057289345565, "operator": "ge"}, {"budget": 0.33, "threshold": -0.2265740465165206, "operator": "ge"}, {"budget": 0.34, "threshold": -0.25551707497557147, "operator": "ge"}, {"budget": 0.35, "threshold": -0.28915902120896647, "operator": "ge"}, {"budget": 0.36, "threshold": -0.31356951992071724, "operator": "ge"}, {"budget": 0.37, "threshold": -0.34747259353515797, "operator": "ge"}, {"budget": 0.38, "threshold": -0.37965294635787106, "operator": "ge"}, {"budget": 0.39, "threshold": -0.41065423482303787, "operator": "ge"}, {"budget": 0.4, "threshold": -0.44622410555642467, "operator": "ge"}, {"budget": 0.41, "threshold": -0.4732048013553065, "operator": "ge"}, {"budget": 0.42, "threshold": -0.501073631867993, "operator": "ge"}, {"budget": 0.43, "threshold": -0.5242925381557026, "operator": "ge"}, {"budget": 0.44, "threshold": -0.5500141481111509, "operator": "ge"}, {"budget": 0.45, "threshold": -0.5722307147662846, "operator": "ge"}, {"budget": 0.46, "threshold": -0.5920537914818997, "operator": "ge"}, {"budget": 0.47, "threshold": -0.6122904490260488, "operator": "ge"}, {"budget": 0.48, "threshold": -0.6319649610327861, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6530350846647437, "operator": "ge"}, {"budget": 0.5, "threshold": -0.6701852904092562, "operator": "ge"}, {"budget": 0.51, "threshold": -0.687162608818831, "operator": "ge"}, {"budget": 0.52, "threshold": -0.7016313281948511, "operator": "ge"}, {"budget": 0.53, "threshold": -0.716312996950337, "operator": "ge"}, {"budget": 0.54, "threshold": -0.7332029850777466, "operator": "ge"}, {"budget": 0.55, "threshold": -0.7477122469234089, "operator": "ge"}, {"budget": 0.56, "threshold": -0.7639199660328508, "operator": "ge"}, {"budget": 0.57, "threshold": -0.7763418366676084, "operator": "ge"}, {"budget": 0.58, "threshold": -0.789621684387459, "operator": "ge"}, {"budget": 0.59, "threshold": -0.8014125620099147, "operator": "ge"}, {"budget": 0.6, "threshold": -0.8121591213645182, "operator": "ge"}, {"budget": 0.61, "threshold": -0.8254702346987811, "operator": "ge"}, {"budget": 0.62, "threshold": -0.8366296948255778, "operator": "ge"}, {"budget": 0.63, "threshold": -0.8449124125989802, "operator": "ge"}, {"budget": 0.64, "threshold": -0.8563890724031343, "operator": "ge"}, {"budget": 0.65, "threshold": -0.8651398161309785, "operator": "ge"}, {"budget": 0.66, "threshold": -0.8729894017518277, "operator": "ge"}, {"budget": 0.67, "threshold": -0.8814255886982573, "operator": "ge"}, {"budget": 0.68, "threshold": -0.8889792540522926, "operator": "ge"}, {"budget": 0.69, "threshold": -0.896391208803726, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9040441818854572, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9100529801904372, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9161849526002004, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9217648979847141, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9277219753039858, "operator": "ge"}, {"budget": 0.75, "threshold": -0.932586494430257, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9369572855579406, "operator": "ge"}, {"budget": 0.77, "threshold": -0.941356657519905, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9464026931577166, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9487407074812001, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9525435654767349, "operator": "ge"}, {"budget": 0.81, "threshold": -0.955670310695961, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9589147722303067, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9618297393755251, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9650531753342987, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9676498942911033, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9706460020068123, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9735161511271326, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9759337556703725, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9781121253625141, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9796696357296862, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9820595168256485, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9834706389830068, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9853639686910005, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9871268187987736, "operator": "ge"}, {"budget": 0.95, "threshold": -0.988827347227563, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9906621936033638, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9921663732849353, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9937750697486329, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9959589490936995, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.33514752219993, 80.77914637639644, 81.25179031796046, 81.62417645373819, 81.93927241478086, 82.3403036379261, 82.62675451160125, 82.98481810369522, 83.31423660842165, 83.60068748209682, 83.88713835577198, 84.17358922944715, 84.41707247207103, 84.66055571469494, 84.96132913205385, 85.23345746204527, 85.47694070466915, 85.79203666571182, 85.89229447149813, 86.09281008307075, 86.22171297622458, 86.46519621884846, 86.63706674305357, 86.83758235462618, 87.19564594672013, 87.3102262961902, 87.46777427671154, 87.4820968203953, 87.73990260670294, 87.89745058722428, 88.0263534803781, 88.1695789172157, 88.21254654826697, 88.22686909195072, 88.31280435405327, 88.5133199656259, 88.58493268404469, 88.71383557719851, 88.7854482956173, 88.98596390718991, 89.08622171297623, 89.25809223718132, 89.42996276138642, 89.573188198224, 89.60183328559151, 89.65912346032655, 89.71641363506157, 89.75938126611285, 89.75938126611285, 89.87396161558291, 89.9169292466342, 89.98854196505299, 90.0171870524205, 90.18905757662561, 90.11744485820681, 90.17473503294184, 90.23202520767688, 90.24634775136063, 90.37525064451447, 90.4325408192495, 90.46118590661702, 90.57576625608708, 90.59008879977084, 90.6044113434546, 90.66170151818963, 90.7046691492409, 90.7046691492409, 90.74763678029217, 90.79060441134345, 90.80492695502721, 90.876539673446, 90.90518476081351, 90.9767974792323, 91.07705528501862, 91.06273274133486, 91.13434545975365, 91.12002291606989, 91.12002291606989, 91.14866800343741, 91.13434545975365, 91.10570037238614, 91.13434545975365, 91.12002291606989, 91.10570037238614, 91.14866800343741, 91.12002291606989, 91.07705528501862, 91.09137782870236, 91.09137782870236, 91.01976511028357, 90.99112002291606, 90.9481523918648, 90.876539673446, 90.81924949871096, 90.77628186765969, 90.7046691492409, 90.63305643082211, 90.57576625608708, 90.56144371240332, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 37, 69, 103, 130, 153, 181, 201, 230, 253, 277, 299, 320, 340, 363, 386, 410, 433, 458, 473, 492, 510, 527, 543, 561, 590, 604, 622, 629, 649, 665, 678, 692, 702, 714, 724, 744, 755, 770, 780, 796, 811, 826, 839, 854, 864, 875, 882, 891, 900, 912, 915, 924, 932, 948, 950, 955, 964, 969, 982, 993, 999, 1009, 1012, 1017, 1023, 1033, 1038, 1045, 1052, 1055, 1064, 1068, 1073, 1081, 1088, 1095, 1098, 1102, 1108, 1109, 1114, 1121, 1122, 1127, 1134, 1135, 1137, 1141, 1143, 1143, 1143, 1145, 1147, 1148, 1149, 1149, 1150, 1152, 1153, 1154], "harmed": [0, 1, 2, 3, 4, 5, 5, 5, 9, 9, 13, 15, 16, 19, 25, 27, 32, 38, 41, 49, 54, 63, 63, 67, 71, 75, 81, 88, 94, 96, 101, 105, 109, 116, 127, 131, 137, 143, 149, 154, 156, 164, 167, 168, 173, 181, 188, 191, 197, 206, 210, 210, 214, 220, 224, 231, 232, 237, 241, 245, 252, 256, 258, 260, 264, 266, 273, 278, 282, 286, 288, 292, 294, 294, 295, 303, 305, 309, 313, 317, 319, 326, 331, 333, 339, 343, 346, 351, 354, 356, 361, 363, 368, 375, 380, 384, 389, 395, 401, 403, 408], "exact_score": [79.81953594958465, 80.33514752219993, 80.77914637639644, 81.25179031796046, 81.62417645373819, 81.93927241478086, 82.3403036379261, 82.62675451160125, 82.98481810369522, 83.31423660842165, 83.60068748209682, 83.88713835577198, 84.17358922944715, 84.41707247207103, 84.66055571469494, 84.96132913205385, 85.23345746204527, 85.47694070466915, 85.79203666571182, 85.89229447149813, 86.09281008307075, 86.22171297622458, 86.46519621884846, 86.63706674305357, 86.83758235462618, 87.19564594672013, 87.3102262961902, 87.46777427671154, 87.4820968203953, 87.73990260670294, 87.89745058722428, 88.0263534803781, 88.1695789172157, 88.21254654826697, 88.22686909195072, 88.31280435405327, 88.5133199656259, 88.58493268404469, 88.71383557719851, 88.7854482956173, 88.98596390718991, 89.08622171297623, 89.25809223718132, 89.42996276138642, 89.573188198224, 89.60183328559151, 89.65912346032655, 89.71641363506157, 89.75938126611285, 89.75938126611285, 89.87396161558291, 89.9169292466342, 89.98854196505299, 90.0171870524205, 90.18905757662561, 90.11744485820681, 90.17473503294184, 90.23202520767688, 90.24634775136063, 90.37525064451447, 90.4325408192495, 90.46118590661702, 90.57576625608708, 90.59008879977084, 90.6044113434546, 90.66170151818963, 90.7046691492409, 90.7046691492409, 90.74763678029217, 90.79060441134345, 90.80492695502721, 90.876539673446, 90.90518476081351, 90.9767974792323, 91.07705528501862, 91.06273274133486, 91.13434545975365, 91.12002291606989, 91.12002291606989, 91.14866800343741, 91.13434545975365, 91.10570037238614, 91.13434545975365, 91.12002291606989, 91.10570037238614, 91.14866800343741, 91.12002291606989, 91.07705528501862, 91.09137782870236, 91.09137782870236, 91.01976511028357, 90.99112002291606, 90.9481523918648, 90.876539673446, 90.81924949871096, 90.77628186765969, 90.7046691492409, 90.63305643082211, 90.57576625608708, 90.56144371240332, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.54917760864969, "threshold_area100": 88.54841019765108, "threshold_area25": 84.02033801203093}, "test": {"score": [80.00287005811867, 80.64145798952428, 81.01456554495229, 81.41637368156705, 81.76795580110497, 82.21281480949989, 82.60027265552128, 82.90162875798235, 83.19580971514674, 83.54739183468466, 84.1572791849035, 84.3510081079142, 84.66671450096864, 84.83174284279256, 85.03264691109995, 85.19767525292387, 85.59948338953865, 85.70711056898904, 85.92236492788979, 86.13761928679055, 86.3815742268781, 86.5681280045921, 86.77620721819616, 86.94123556002009, 87.14213962832747, 87.27129224366794, 87.52242232905216, 87.65157494439264, 87.84530386740332, 88.02468249982063, 88.1394848245677, 88.35473918346847, 88.45519121762216, 88.56281839707253, 88.69914615770969, 88.8856999354237, 88.95745138839062, 89.0722537131377, 89.13683002080792, 89.30903350792853, 89.50276243093923, 89.58168902920285, 89.65344048216977, 89.69649135394992, 89.84716940518047, 89.93327114874077, 90.01219774700438, 90.03372318289445, 90.14135036234484, 90.15570065293824, 90.20592667001507, 90.277678122982, 90.41400588361914, 90.46423190069599, 90.49293248188276, 90.51445791777284, 90.57185908014638, 90.57185908014638, 90.59338451603645, 90.64361053311329, 90.65796082370669, 90.70818684078354, 90.70101169548683, 90.73688742197031, 90.80863887493722, 90.82298916553061, 90.81581402023392, 90.82298916553061, 90.85886489201407, 90.77993829375045, 90.77276314845376, 90.76558800315708, 90.744062567267, 90.70101169548683, 90.70101169548683, 90.65796082370669, 90.65796082370669, 90.66513596900337, 90.67948625959676, 90.67231111430006, 90.65796082370669, 90.70101169548683, 90.70101169548683, 90.67948625959676, 90.67948625959676, 90.57185908014638, 90.5503336442563, 90.57903422544307, 90.51445791777284, 90.47858219128938, 90.48575733658608, 90.45705675539931, 90.45705675539931, 90.39965559302576, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.26332783238861, 90.19157637942169, 90.14852550764154, 90.06242376408122], "calls": [0.0, 1.3058764439979909, 2.1525435890076774, 3.1068379134677477, 3.9319796225873573, 5.02260170768458, 6.048647485111573, 7.153619860802181, 8.03616273229533, 8.9904570567554, 10.411135825500466, 11.351079859367152, 12.463227380354452, 13.3457702518476, 14.192437396857285, 15.426562387888353, 16.52435961828227, 17.299275310325037, 18.49034942957595, 19.437468608739326, 20.52809069383655, 21.582837052450312, 22.694984573437612, 23.61340317141422, 24.395494008753676, 25.177584846093133, 26.096003444069744, 26.863743990815813, 27.79651287938581, 28.743632058549185, 29.68357609241587, 30.78854846810648, 31.606515031929394, 32.546459065796085, 33.414651646695845, 34.49092344119968, 35.35194087680276, 36.44973810719667, 37.4399081581402, 38.473129080863885, 39.7574800889718, 40.76917557580541, 41.85979766090263, 42.864318002439546, 43.89036377986654, 44.90205926670015, 45.82047786467676, 46.78194733443352, 47.78646767597044, 48.75511229102389, 49.6735308890005, 50.7856784099878, 51.704097007964414, 52.586639879457564, 53.74183827222502, 54.75353375905863, 55.93025758771615, 56.64777211738537, 57.65946760421898, 58.57071105689891, 59.611107124919286, 60.97438473129081, 62.07218196168473, 62.99060055966134, 64.10992322594532, 65.07139269570209, 66.09026332783239, 67.08760852407262, 68.14953002798306, 69.34777929253067, 70.56037884767167, 71.4572720097582, 72.51201836837195, 73.42326182105188, 74.49235847025902, 75.39642677764225, 76.42247255506925, 77.30501542656239, 78.45303867403315, 79.02705029776853, 80.08179665638228, 80.89258807490852, 82.08366219415943, 83.05230680921288, 84.1501040396068, 85.09722321877018, 86.23807132094426, 87.4004448590084, 88.38343976465524, 89.28750807203846, 89.98349716581761, 91.03106837913467, 91.67683145583698, 92.73875295974744, 93.76479873717443, 94.64734160866757, 95.7379636937648, 96.78553490708187, 97.66807777857501, 98.93807849608955, 100.0], "rescued": [0, 95, 148, 208, 261, 329, 389, 438, 484, 538, 627, 661, 718, 746, 777, 812, 871, 895, 934, 969, 1009, 1042, 1078, 1107, 1136, 1162, 1201, 1225, 1262, 1290, 1312, 1348, 1369, 1392, 1417, 1449, 1471, 1494, 1520, 1549, 1582, 1600, 1624, 1643, 1670, 1692, 1710, 1726, 1750, 1770, 1786, 1808, 1831, 1845, 1864, 1877, 1896, 1905, 1918, 1933, 1946, 1963, 1971, 1983, 2000, 2012, 2023, 2031, 2045, 2056, 2071, 2078, 2092, 2095, 2103, 2105, 2112, 2118, 2131, 2136, 2142, 2151, 2160, 2164, 2173, 2175, 2180, 2190, 2196, 2200, 2207, 2214, 2218, 2223, 2228, 2233, 2237, 2239, 2240, 2246, 2247], "harmed": [0, 6, 7, 11, 15, 21, 27, 34, 39, 44, 48, 55, 68, 73, 76, 88, 91, 100, 109, 114, 120, 127, 134, 140, 141, 149, 153, 159, 169, 172, 178, 184, 191, 199, 205, 211, 223, 230, 247, 252, 258, 265, 279, 292, 298, 308, 315, 328, 337, 355, 364, 376, 380, 387, 402, 412, 423, 432, 442, 450, 461, 471, 480, 487, 494, 504, 516, 523, 532, 554, 570, 578, 595, 604, 612, 620, 627, 632, 643, 649, 657, 660, 669, 676, 685, 702, 710, 716, 731, 740, 746, 757, 761, 774, 780, 789, 801, 809, 820, 832, 845], "exact_score": [80.00287005811867, 80.51948051948052, 80.94281409198537, 81.35897251919351, 81.78230609169836, 82.1984645189065, 82.58592236492788, 82.8585778862022, 83.18863456985004, 83.55456697998135, 83.9635502618928, 84.27208150965058, 84.54473703092488, 84.77434168041903, 84.99677118461649, 85.15462438114372, 85.4488053383081, 85.69993542369232, 85.8290880390328, 86.02999210734018, 86.2954724833178, 86.4676759704384, 86.67575518404249, 86.80490779938295, 87.06321303006386, 87.24259166248117, 87.4937217478654, 87.665925234986, 87.8596541579967, 88.08208366219417, 88.16818540575447, 88.37626461935855, 88.48389179880894, 88.62739470474278, 88.79959819186338, 88.93592595250054, 89.05072827724761, 89.10095429432447, 89.23728205496163, 89.39513525148884, 89.51711272153261, 89.59603931979623, 89.63909019157639, 89.73954222573008, 89.86151969577384, 89.92609600344407, 90.03372318289445, 90.05524861878453, 90.14135036234484, 90.1700509435316, 90.22027696060846, 90.32790414005885, 90.44988161010261, 90.47140704599268, 90.48575733658608, 90.52880820836621, 90.57185908014638, 90.59338451603645, 90.60055966133315, 90.65796082370669, 90.65796082370669, 90.70818684078354, 90.69383655019014, 90.73688742197031, 90.78711343904715, 90.81581402023392, 90.81581402023392, 90.80146372964053, 90.87321518260745, 90.80863887493722, 90.77993829375045, 90.78711343904715, 90.75123771256368, 90.71536198608023, 90.69383655019014, 90.66513596900337, 90.67231111430006, 90.66513596900337, 90.68666140489346, 90.67948625959676, 90.65796082370669, 90.70818684078354, 90.70101169548683, 90.67948625959676, 90.67948625959676, 90.57903422544307, 90.56468393484968, 90.5503336442563, 90.53598335366291, 90.50010762717945, 90.48575733658608, 90.46423190069599, 90.44988161010261, 90.39248044772907, 90.40683073832245, 90.34225443065222, 90.30637870416876, 90.24180239649853, 90.1772260888283, 90.14135036234484, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.62546127156763, "threshold_area100": 88.63248905790344, "threshold_area25": 84.31728492501975}, "selected_budget": 79, "selected": {"val": {"score": 91.14866800343741, "calls": 78.98882841592668, "rescued": 1108, "harmed": 317}, "test": {"score": 90.67231111430006, "calls": 79.02705029776853, "rescued": 2136, "harmed": 649}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7229817509651184, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6144948601722717, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5405305624008179, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4716658890247345, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4117107391357422, "operator": "ge"}, {"budget": 0.06, "threshold": 0.36499300599098206, "operator": "ge"}, {"budget": 0.07, "threshold": 0.32603737711906433, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2881035804748535, "operator": "ge"}, {"budget": 0.09, "threshold": 0.2583020031452179, "operator": "ge"}, {"budget": 0.1, "threshold": 0.22714146971702576, "operator": "ge"}, {"budget": 0.11, "threshold": 0.20176872611045837, "operator": "ge"}, {"budget": 0.12, "threshold": 0.17630238831043243, "operator": "ge"}, {"budget": 0.13, "threshold": 0.1570243537425995, "operator": "ge"}, {"budget": 0.14, "threshold": 0.13233894109725952, "operator": "ge"}, {"budget": 0.15, "threshold": 0.1159592792391777, "operator": "ge"}, {"budget": 0.16, "threshold": 0.1010880321264267, "operator": "ge"}, {"budget": 0.17, "threshold": 0.08099313080310822, "operator": "ge"}, {"budget": 0.18, "threshold": 0.061356425285339355, "operator": "ge"}, {"budget": 0.19, "threshold": 0.050820812582969666, "operator": "ge"}, {"budget": 0.2, "threshold": 0.03980666399002075, "operator": "ge"}, {"budget": 0.21, "threshold": 0.03072980046272278, "operator": "ge"}, {"budget": 0.22, "threshold": 0.022987548261880875, "operator": "ge"}, {"budget": 0.23, "threshold": 0.016089392825961113, "operator": "ge"}, {"budget": 0.24, "threshold": 0.009807683527469635, "operator": "ge"}, {"budget": 0.25, "threshold": 0.004239611327648163, "operator": "ge"}, {"budget": 0.26, "threshold": -0.0011116806417703629, "operator": "ge"}, {"budget": 0.27, "threshold": -0.0030949003994464874, "operator": "ge"}, {"budget": 0.28, "threshold": -0.004600305110216141, "operator": "ge"}, {"budget": 0.29, "threshold": -0.006082186941057444, "operator": "ge"}, {"budget": 0.3, "threshold": -0.007290152832865715, "operator": "ge"}, {"budget": 0.31, "threshold": -0.008085721172392368, "operator": "ge"}, {"budget": 0.32, "threshold": -0.008868848904967308, "operator": "ge"}, {"budget": 0.33, "threshold": -0.009657947346568108, "operator": "ge"}, {"budget": 0.34, "threshold": -0.010367552749812603, "operator": "ge"}, {"budget": 0.35, "threshold": -0.011088070459663868, "operator": "ge"}, {"budget": 0.36, "threshold": -0.011984280310571194, "operator": "ge"}, {"budget": 0.37, "threshold": -0.012900014407932758, "operator": "ge"}, {"budget": 0.38, "threshold": -0.013708338141441345, "operator": "ge"}, {"budget": 0.39, "threshold": -0.01446924451738596, "operator": "ge"}, {"budget": 0.4, "threshold": -0.01524405274540186, "operator": "ge"}, {"budget": 0.41, "threshold": -0.015883974730968475, "operator": "ge"}, {"budget": 0.42, "threshold": -0.016859492287039757, "operator": "ge"}, {"budget": 0.43, "threshold": -0.01763000898063183, "operator": "ge"}, {"budget": 0.44, "threshold": -0.01846274361014366, "operator": "ge"}, {"budget": 0.45, "threshold": -0.01924724504351616, "operator": "ge"}, {"budget": 0.46, "threshold": -0.020265845581889153, "operator": "ge"}, {"budget": 0.47, "threshold": -0.02140893042087555, "operator": "ge"}, {"budget": 0.48, "threshold": -0.02243868261575699, "operator": "ge"}, {"budget": 0.49, "threshold": -0.023691289126873016, "operator": "ge"}, {"budget": 0.5, "threshold": -0.024581179022789, "operator": "ge"}, {"budget": 0.51, "threshold": -0.02557707577943802, "operator": "ge"}, {"budget": 0.52, "threshold": -0.02666814811527729, "operator": "ge"}, {"budget": 0.53, "threshold": -0.027780599892139435, "operator": "ge"}, {"budget": 0.54, "threshold": -0.028780218213796616, "operator": "ge"}, {"budget": 0.55, "threshold": -0.02988184243440628, "operator": "ge"}, {"budget": 0.56, "threshold": -0.03121570125222206, "operator": "ge"}, {"budget": 0.57, "threshold": -0.03232140466570854, "operator": "ge"}, {"budget": 0.58, "threshold": -0.03361494094133377, "operator": "ge"}, {"budget": 0.59, "threshold": -0.03501221910119057, "operator": "ge"}, {"budget": 0.6, "threshold": -0.03631855547428131, "operator": "ge"}, {"budget": 0.61, "threshold": -0.0372471809387207, "operator": "ge"}, {"budget": 0.62, "threshold": -0.03830727934837341, "operator": "ge"}, {"budget": 0.63, "threshold": -0.03949554264545441, "operator": "ge"}, {"budget": 0.64, "threshold": -0.04088011011481285, "operator": "ge"}, {"budget": 0.65, "threshold": -0.04239172860980034, "operator": "ge"}, {"budget": 0.66, "threshold": -0.04375188797712326, "operator": "ge"}, {"budget": 0.67, "threshold": -0.045586880296468735, "operator": "ge"}, {"budget": 0.68, "threshold": -0.04721198230981827, "operator": "ge"}, {"budget": 0.69, "threshold": -0.04870620369911194, "operator": "ge"}, {"budget": 0.7, "threshold": -0.050252798944711685, "operator": "ge"}, {"budget": 0.71, "threshold": -0.052381083369255066, "operator": "ge"}, {"budget": 0.72, "threshold": -0.053644005209207535, "operator": "ge"}, {"budget": 0.73, "threshold": -0.05576597526669502, "operator": "ge"}, {"budget": 0.74, "threshold": -0.05782356485724449, "operator": "ge"}, {"budget": 0.75, "threshold": -0.05987654998898506, "operator": "ge"}, {"budget": 0.76, "threshold": -0.06207664683461189, "operator": "ge"}, {"budget": 0.77, "threshold": -0.06403116881847382, "operator": "ge"}, {"budget": 0.78, "threshold": -0.06617997586727142, "operator": "ge"}, {"budget": 0.79, "threshold": -0.06856991350650787, "operator": "ge"}, {"budget": 0.8, "threshold": -0.07129254937171936, "operator": "ge"}, {"budget": 0.81, "threshold": -0.0738721564412117, "operator": "ge"}, {"budget": 0.82, "threshold": -0.07701568305492401, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0794377475976944, "operator": "ge"}, {"budget": 0.84, "threshold": -0.08273598551750183, "operator": "ge"}, {"budget": 0.85, "threshold": -0.08600757271051407, "operator": "ge"}, {"budget": 0.86, "threshold": -0.08989743888378143, "operator": "ge"}, {"budget": 0.87, "threshold": -0.0942760482430458, "operator": "ge"}, {"budget": 0.88, "threshold": -0.09846457093954086, "operator": "ge"}, {"budget": 0.89, "threshold": -0.10275097191333771, "operator": "ge"}, {"budget": 0.9, "threshold": -0.10884015262126923, "operator": "ge"}, {"budget": 0.91, "threshold": -0.11531335115432739, "operator": "ge"}, {"budget": 0.92, "threshold": -0.12285300344228745, "operator": "ge"}, {"budget": 0.93, "threshold": -0.13226382434368134, "operator": "ge"}, {"budget": 0.94, "threshold": -0.14274726808071136, "operator": "ge"}, {"budget": 0.95, "threshold": -0.15359556674957275, "operator": "ge"}, {"budget": 0.96, "threshold": -0.17008216679096222, "operator": "ge"}, {"budget": 0.97, "threshold": -0.19073767960071564, "operator": "ge"}, {"budget": 0.98, "threshold": -0.22272783517837524, "operator": "ge"}, {"budget": 0.99, "threshold": -0.30530402064323425, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.37811515325122, 80.82211400744772, 81.28043540532798, 81.60985391005443, 82.08249785161844, 82.35462618160985, 82.66972214265253, 83.02778573474649, 83.34288169578916, 83.5720423947293, 83.80120309366943, 84.13062159839588, 84.46004010312231, 84.78945860784874, 85.16184474362646, 85.3910054425666, 85.4912632483529, 85.64881122887425, 85.94958464623316, 86.1787453451733, 86.4508736751647, 86.56545402463478, 86.63706674305357, 86.98080779146376, 87.18132340303637, 87.29590375250645, 87.35319392724148, 87.43912918934402, 87.42480664566027, 87.42480664566027, 87.4820968203953, 87.52506445144657, 87.55370953881409, 87.62532225723288, 87.65396734460039, 87.69693497565167, 87.73990260670294, 87.81151532512173, 87.81151532512173, 87.86880549985678, 87.91177313090805, 87.96906330564308, 87.98338584932684, 88.0263534803781, 88.09796619879691, 88.1695789172157, 88.12661128616442, 88.18390146089945, 88.22686909195072, 88.25551417931824, 88.29848181036952, 88.34144944142079, 88.41306215983958, 88.41306215983958, 88.52764250930964, 88.52764250930964, 88.59925522772843, 88.62790031509596, 88.62790031509596, 88.64222285877972, 88.71383557719851, 88.82841592666857, 88.85706101403609, 88.94299627613864, 89.02893153824118, 89.11486680034373, 89.0575766256087, 89.11486680034373, 89.22944714981381, 89.35835004296763, 89.42996276138642, 89.47293039243769, 89.53022056717272, 89.573188198224, 89.573188198224, 89.63047837295903, 89.70209109137782, 89.73073617874535, 89.80234889716414, 89.9169292466342, 90.04583213978802, 90.13176740189057, 90.10312231452305, 90.11744485820681, 90.10312231452305, 90.23202520767688, 90.37525064451447, 90.40389573188197, 90.47550845030077, 90.44686336293324, 90.48983099398453, 90.57576625608708, 90.63305643082211, 90.61873388713835, 90.67602406187338, 90.77628186765969, 90.80492695502721, 90.91950730449727, 90.80492695502721, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 40, 72, 105, 128, 162, 183, 205, 231, 254, 273, 293, 322, 348, 375, 402, 425, 442, 457, 482, 503, 523, 537, 548, 574, 591, 603, 610, 616, 618, 622, 627, 631, 634, 639, 641, 644, 648, 655, 656, 660, 665, 670, 672, 676, 681, 687, 689, 694, 699, 703, 707, 711, 718, 722, 731, 735, 741, 746, 751, 756, 765, 774, 781, 788, 797, 805, 809, 816, 829, 842, 850, 860, 866, 875, 880, 888, 897, 903, 913, 924, 934, 950, 957, 961, 969, 982, 998, 1007, 1021, 1029, 1037, 1052, 1065, 1074, 1088, 1100, 1114, 1130, 1143, 1154], "harmed": [0, 1, 2, 3, 3, 4, 6, 6, 7, 8, 11, 15, 21, 24, 28, 29, 36, 46, 50, 54, 59, 60, 66, 72, 74, 77, 81, 84, 84, 87, 91, 92, 93, 94, 94, 94, 94, 95, 97, 98, 98, 100, 101, 102, 103, 103, 104, 109, 110, 112, 114, 115, 116, 118, 122, 123, 127, 128, 131, 136, 140, 144, 145, 150, 151, 154, 156, 164, 167, 172, 176, 179, 186, 188, 194, 199, 203, 207, 211, 216, 219, 220, 230, 239, 242, 251, 255, 261, 268, 277, 287, 292, 301, 310, 320, 330, 335, 347, 355, 376, 408], "exact_score": [79.81953594958465, 80.37811515325122, 80.82211400744772, 81.28043540532798, 81.60985391005443, 82.08249785161844, 82.35462618160985, 82.66972214265253, 83.02778573474649, 83.34288169578916, 83.5720423947293, 83.80120309366943, 84.13062159839588, 84.46004010312231, 84.78945860784874, 85.16184474362646, 85.3910054425666, 85.4912632483529, 85.64881122887425, 85.94958464623316, 86.1787453451733, 86.4508736751647, 86.56545402463478, 86.63706674305357, 86.98080779146376, 87.18132340303637, 87.29590375250645, 87.35319392724148, 87.43912918934402, 87.42480664566027, 87.42480664566027, 87.4820968203953, 87.52506445144657, 87.55370953881409, 87.62532225723288, 87.65396734460039, 87.69693497565167, 87.73990260670294, 87.81151532512173, 87.81151532512173, 87.86880549985678, 87.91177313090805, 87.96906330564308, 87.98338584932684, 88.0263534803781, 88.09796619879691, 88.1695789172157, 88.12661128616442, 88.18390146089945, 88.22686909195072, 88.25551417931824, 88.29848181036952, 88.34144944142079, 88.41306215983958, 88.41306215983958, 88.52764250930964, 88.52764250930964, 88.59925522772843, 88.62790031509596, 88.62790031509596, 88.64222285877972, 88.71383557719851, 88.82841592666857, 88.85706101403609, 88.94299627613864, 89.02893153824118, 89.11486680034373, 89.0575766256087, 89.11486680034373, 89.22944714981381, 89.35835004296763, 89.42996276138642, 89.47293039243769, 89.53022056717272, 89.573188198224, 89.573188198224, 89.63047837295903, 89.70209109137782, 89.73073617874535, 89.80234889716414, 89.9169292466342, 90.04583213978802, 90.13176740189057, 90.10312231452305, 90.11744485820681, 90.10312231452305, 90.23202520767688, 90.37525064451447, 90.40389573188197, 90.47550845030077, 90.44686336293324, 90.48983099398453, 90.57576625608708, 90.63305643082211, 90.61873388713835, 90.67602406187338, 90.77628186765969, 90.80492695502721, 90.91950730449727, 90.80492695502721, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.76216593620735, "threshold_area100": 87.76138642222858, "threshold_area25": 84.0991120022916}, "test": {"score": [80.00287005811867, 80.64863313482098, 81.1867690320729, 81.64597833106122, 82.09083733945613, 82.34196742484036, 82.70072468967497, 83.0451316639162, 83.43976465523427, 83.73394561239866, 84.11422831312333, 84.37253354380427, 84.71694051804549, 84.86761856927603, 85.1761498170338, 85.38422903063787, 85.54208222716511, 85.75733658606588, 86.0084666714501, 86.15914472268064, 86.31699791920788, 86.54660256870201, 86.76903207289948, 86.99863672239363, 87.13496448303079, 87.29999282485471, 87.50089689316209, 87.55829805553563, 87.56547320083232, 87.60852407261247, 87.68745067087609, 87.72332639735954, 87.78072755973308, 87.81660328621655, 87.84530386740332, 87.92423046566694, 87.98880677333716, 88.01750735452394, 88.07490851689748, 88.09643395278755, 88.11795938867762, 88.16818540575447, 88.20406113223794, 88.21841142283131, 88.26146229461148, 88.31886345698501, 88.36908947406185, 88.41931549113869, 88.46954150821554, 88.5197675252924, 88.53411781588576, 88.56999354236925, 88.59151897825932, 88.61304441414939, 88.67044557652292, 88.7924230465667, 88.87134964483032, 88.89287508072039, 88.95745138839062, 88.97897682428069, 89.01485255076416, 89.04355313195093, 89.1440051661046, 89.20140632847816, 89.30185836263185, 89.32338379852193, 89.35925952500538, 89.38078496089547, 89.39513525148884, 89.41666068737892, 89.47406184975246, 89.60321446509292, 89.67496591805984, 89.73236708043338, 89.78259309751023, 89.83999425988375, 89.8902202769606, 89.8973954222573, 89.95479658463084, 89.98349716581761, 90.01219774700438, 90.00502260170768, 90.0839491999713, 90.13417521704815, 90.16287579823492, 90.22745210590514, 90.27050297768531, 90.25615268709191, 90.29202841357538, 90.29202841357538, 90.34225443065222, 90.34942957594892, 90.41400588361914, 90.42835617421254, 90.43553131950922, 90.41400588361914, 90.41400588361914, 90.44988161010261, 90.47858219128938, 90.37813015713569, 90.06242376408122], "calls": [0.0, 1.0619215039104541, 2.166893879601062, 3.1498887852479016, 4.262036306235202, 5.237856066585348, 6.306952715792495, 7.2971227667360266, 8.32316854416302, 9.205711415656168, 10.167180885412929, 11.164526081653154, 12.197747004376838, 13.087465021166677, 14.42921719164813, 15.340460644328047, 16.11537633637081, 17.442778216258876, 18.863456985003946, 19.588146659969865, 20.571141565616706, 21.60436248834039, 22.479730214536843, 23.340747650139914, 24.316567410490062, 25.113008538422903, 26.1318791705532, 26.756116811365434, 27.703235990528807, 28.987586998636722, 30.401090622085096, 31.45583698069886, 32.48188275812585, 33.32854990313554, 34.168041902848536, 35.04340962904499, 35.99052880820837, 37.20312836334936, 38.22917414077635, 39.16194302934635, 40.05883619143288, 40.79070101169548, 41.960249695056326, 42.80691684006601, 43.78273660041616, 44.58635287364569, 45.45454545454545, 46.68149530027983, 47.93714572720098, 49.15692042763866, 50.16861591447227, 51.223362273086025, 52.184831742842796, 53.28262897323671, 54.22257300710339, 55.349070818684076, 56.662122407978764, 57.48008897180168, 58.69986367223936, 59.85506206500681, 60.96003444069742, 61.59144722680634, 62.395063500035874, 63.205854918562096, 64.18884982420894, 65.25794647341608, 66.16201478079931, 67.4607160795006, 68.43653583985075, 69.30472842075052, 70.1585707110569, 71.41422113797805, 72.05280906938366, 73.15778144507426, 74.11207576953433, 75.1668221281481, 76.04936499964124, 76.95343330702447, 77.83597617851761, 78.70416875941739, 79.5149601779436, 80.36880246824998, 81.33744708330343, 82.18411422831312, 83.15275884336658, 84.13575374901342, 85.21202554351726, 86.3743990815814, 87.25694195307454, 88.11795938867762, 89.30185836263185, 90.40683073832245, 91.46157709693621, 92.5952500538136, 93.5997703953505, 94.6186410274808, 95.67338738609456, 96.69225801822486, 97.71112865035518, 98.99547965846308, 100.0], "rescued": [0, 92, 171, 239, 305, 350, 404, 459, 519, 564, 621, 667, 720, 755, 810, 848, 873, 915, 957, 982, 1010, 1046, 1080, 1114, 1140, 1167, 1198, 1208, 1211, 1219, 1231, 1238, 1247, 1256, 1262, 1275, 1285, 1292, 1300, 1305, 1310, 1318, 1326, 1331, 1339, 1347, 1355, 1366, 1377, 1391, 1399, 1408, 1415, 1423, 1436, 1455, 1472, 1481, 1496, 1505, 1518, 1527, 1543, 1555, 1573, 1584, 1592, 1607, 1616, 1627, 1638, 1662, 1674, 1695, 1710, 1726, 1741, 1750, 1768, 1783, 1800, 1816, 1836, 1854, 1871, 1895, 1916, 1931, 1952, 1969, 1990, 2008, 2033, 2056, 2077, 2099, 2119, 2152, 2179, 2216, 2247], "harmed": [0, 2, 6, 10, 14, 24, 28, 35, 40, 44, 48, 58, 63, 77, 89, 98, 101, 113, 120, 124, 130, 134, 137, 139, 146, 150, 153, 155, 157, 159, 160, 162, 163, 167, 169, 171, 172, 175, 175, 177, 179, 180, 183, 186, 188, 188, 189, 193, 197, 204, 210, 214, 218, 223, 228, 230, 236, 242, 248, 254, 262, 267, 269, 273, 277, 285, 288, 300, 307, 315, 318, 324, 326, 339, 347, 355, 363, 371, 381, 392, 405, 422, 431, 442, 455, 470, 485, 502, 518, 535, 549, 566, 582, 603, 623, 648, 668, 696, 719, 770, 845], "exact_score": [80.00287005811867, 80.61275740833752, 81.10784243380928, 81.59575231398436, 81.99756045059912, 82.27739111717013, 82.5787472196312, 82.93750448446582, 83.31778718519051, 83.65501901413504, 84.04965200545311, 84.3366578173208, 84.67388964626534, 84.8173925521992, 85.07569778288011, 85.30530243237426, 85.51338164597834, 85.68558513309895, 85.84343832962618, 86.06586783382365, 86.23807132094426, 86.43180024395495, 86.62552916696563, 86.91253497883332, 87.09191361125063, 87.27846738896463, 87.48654660256871, 87.55829805553563, 87.5798234914257, 87.60852407261247, 87.63722465379924, 87.70897610676617, 87.7735524144364, 87.80942814091985, 87.83812872210662, 87.92423046566694, 87.98880677333716, 88.01033220922724, 88.0533830810074, 88.08208366219417, 88.11078424338093, 88.18971084164454, 88.21123627753462, 88.22558656812801, 88.26146229461148, 88.36191432876515, 88.36191432876515, 88.42649063643539, 88.47671665351223, 88.5197675252924, 88.53411781588576, 88.56281839707253, 88.57716868766593, 88.59151897825932, 88.67762072181962, 88.74219702948984, 88.83547391834685, 88.878524790127, 88.92875080720385, 88.95745138839062, 88.98615196957739, 89.01485255076416, 89.08660400373107, 89.17988089258809, 89.28033292674176, 89.33055894381862, 89.35925952500538, 89.40231039678554, 89.38796010619215, 89.40948554208224, 89.45971155915908, 89.56016359331277, 89.68214106335654, 89.72519193513669, 89.77541795221353, 89.83999425988375, 89.87586998636723, 89.8902202769606, 89.95479658463084, 90.01937289230108, 90.01937289230108, 90.05524861878453, 90.12700007175145, 90.1700509435316, 90.22745210590514, 90.24897754179523, 90.26332783238861, 90.28485326827868, 90.29202841357538, 90.34942957594892, 90.32790414005885, 90.37095501183899, 90.40683073832245, 90.42118102891584, 90.46423190069599, 90.44270646480591, 90.44270646480591, 90.48575733658608, 90.48575733658608, 90.37813015713569, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.93068305112838, "threshold_area100": 87.950132740188, "threshold_area25": 84.36177082585922}, "selected_budget": 98, "selected": {"val": {"score": 90.91950730449727, "calls": 97.99484388427385, "rescued": 1130, "harmed": 355}, "test": {"score": 90.47858219128938, "calls": 97.71112865035518, "rescued": 2179, "harmed": 719}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.693002700805664, "operator": "ge"}, {"budget": 0.02, "threshold": 3.315732717514038, "operator": "ge"}, {"budget": 0.03, "threshold": 2.9000306129455566, "operator": "ge"}, {"budget": 0.04, "threshold": 2.6082215309143066, "operator": "ge"}, {"budget": 0.05, "threshold": 2.414426803588867, "operator": "ge"}, {"budget": 0.06, "threshold": 2.236410140991211, "operator": "ge"}, {"budget": 0.07, "threshold": 2.0561742782592773, "operator": "ge"}, {"budget": 0.08, "threshold": 1.9035016298294067, "operator": "ge"}, {"budget": 0.09, "threshold": 1.7555347681045532, "operator": "ge"}, {"budget": 0.1, "threshold": 1.599146842956543, "operator": "ge"}, {"budget": 0.11, "threshold": 1.4639248847961426, "operator": "ge"}, {"budget": 0.12, "threshold": 1.3201568126678467, "operator": "ge"}, {"budget": 0.13, "threshold": 1.1911001205444336, "operator": "ge"}, {"budget": 0.14, "threshold": 1.0824189186096191, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9687632322311401, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8479628562927246, "operator": "ge"}, {"budget": 0.17, "threshold": 0.757366418838501, "operator": "ge"}, {"budget": 0.18, "threshold": 0.6560664176940918, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5419195890426636, "operator": "ge"}, {"budget": 0.2, "threshold": 0.4609470069408417, "operator": "ge"}, {"budget": 0.21, "threshold": 0.37009185552597046, "operator": "ge"}, {"budget": 0.22, "threshold": 0.2708299160003662, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2092747688293457, "operator": "ge"}, {"budget": 0.24, "threshold": 0.12723246216773987, "operator": "ge"}, {"budget": 0.25, "threshold": 0.06727484613656998, "operator": "ge"}, {"budget": 0.26, "threshold": -0.016710281372070312, "operator": "ge"}, {"budget": 0.27, "threshold": -0.09340560436248779, "operator": "ge"}, {"budget": 0.28, "threshold": -0.16254055500030518, "operator": "ge"}, {"budget": 0.29, "threshold": -0.22705745697021484, "operator": "ge"}, {"budget": 0.3, "threshold": -0.3067619800567627, "operator": "ge"}, {"budget": 0.31, "threshold": -0.38544517755508423, "operator": "ge"}, {"budget": 0.32, "threshold": -0.4584655165672302, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5222674012184143, "operator": "ge"}, {"budget": 0.34, "threshold": -0.589932918548584, "operator": "ge"}, {"budget": 0.35, "threshold": -0.6620469093322754, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7298672795295715, "operator": "ge"}, {"budget": 0.37, "threshold": -0.8090435266494751, "operator": "ge"}, {"budget": 0.38, "threshold": -0.8798527717590332, "operator": "ge"}, {"budget": 0.39, "threshold": -0.9606488347053528, "operator": "ge"}, {"budget": 0.4, "threshold": -1.0357364416122437, "operator": "ge"}, {"budget": 0.41, "threshold": -1.117357611656189, "operator": "ge"}, {"budget": 0.42, "threshold": -1.1851160526275635, "operator": "ge"}, {"budget": 0.43, "threshold": -1.257131814956665, "operator": "ge"}, {"budget": 0.44, "threshold": -1.3290451765060425, "operator": "ge"}, {"budget": 0.45, "threshold": -1.396632194519043, "operator": "ge"}, {"budget": 0.46, "threshold": -1.463733434677124, "operator": "ge"}, {"budget": 0.47, "threshold": -1.5292025804519653, "operator": "ge"}, {"budget": 0.48, "threshold": -1.588843822479248, "operator": "ge"}, {"budget": 0.49, "threshold": -1.6597195863723755, "operator": "ge"}, {"budget": 0.5, "threshold": -1.7250726222991943, "operator": "ge"}, {"budget": 0.51, "threshold": -1.7883490324020386, "operator": "ge"}, {"budget": 0.52, "threshold": -1.8409394025802612, "operator": "ge"}, {"budget": 0.53, "threshold": -1.9137221574783325, "operator": "ge"}, {"budget": 0.54, "threshold": -1.9771580696105957, "operator": "ge"}, {"budget": 0.55, "threshold": -2.0427496433258057, "operator": "ge"}, {"budget": 0.56, "threshold": -2.1149117946624756, "operator": "ge"}, {"budget": 0.57, "threshold": -2.1782519817352295, "operator": "ge"}, {"budget": 0.58, "threshold": -2.241020679473877, "operator": "ge"}, {"budget": 0.59, "threshold": -2.310438871383667, "operator": "ge"}, {"budget": 0.6, "threshold": -2.3754632472991943, "operator": "ge"}, {"budget": 0.61, "threshold": -2.455995798110962, "operator": "ge"}, {"budget": 0.62, "threshold": -2.5148508548736572, "operator": "ge"}, {"budget": 0.63, "threshold": -2.5877528190612793, "operator": "ge"}, {"budget": 0.64, "threshold": -2.653224468231201, "operator": "ge"}, {"budget": 0.65, "threshold": -2.734686851501465, "operator": "ge"}, {"budget": 0.66, "threshold": -2.7996108531951904, "operator": "ge"}, {"budget": 0.67, "threshold": -2.8674042224884033, "operator": "ge"}, {"budget": 0.68, "threshold": -2.9427576065063477, "operator": "ge"}, {"budget": 0.69, "threshold": -3.0110368728637695, "operator": "ge"}, {"budget": 0.7, "threshold": -3.085392475128174, "operator": "ge"}, {"budget": 0.71, "threshold": -3.157522201538086, "operator": "ge"}, {"budget": 0.72, "threshold": -3.227357864379883, "operator": "ge"}, {"budget": 0.73, "threshold": -3.3037848472595215, "operator": "ge"}, {"budget": 0.74, "threshold": -3.3807005882263184, "operator": "ge"}, {"budget": 0.75, "threshold": -3.4469285011291504, "operator": "ge"}, {"budget": 0.76, "threshold": -3.5162134170532227, "operator": "ge"}, {"budget": 0.77, "threshold": -3.6038622856140137, "operator": "ge"}, {"budget": 0.78, "threshold": -3.6716513633728027, "operator": "ge"}, {"budget": 0.79, "threshold": -3.736501455307007, "operator": "ge"}, {"budget": 0.8, "threshold": -3.8060035705566406, "operator": "ge"}, {"budget": 0.81, "threshold": -3.8849973678588867, "operator": "ge"}, {"budget": 0.82, "threshold": -3.955817699432373, "operator": "ge"}, {"budget": 0.83, "threshold": -4.038460731506348, "operator": "ge"}, {"budget": 0.84, "threshold": -4.113241195678711, "operator": "ge"}, {"budget": 0.85, "threshold": -4.201733589172363, "operator": "ge"}, {"budget": 0.86, "threshold": -4.288350582122803, "operator": "ge"}, {"budget": 0.87, "threshold": -4.4009785652160645, "operator": "ge"}, {"budget": 0.88, "threshold": -4.496504306793213, "operator": "ge"}, {"budget": 0.89, "threshold": -4.581015586853027, "operator": "ge"}, {"budget": 0.9, "threshold": -4.666928291320801, "operator": "ge"}, {"budget": 0.91, "threshold": -4.765248775482178, "operator": "ge"}, {"budget": 0.92, "threshold": -4.868922233581543, "operator": "ge"}, {"budget": 0.93, "threshold": -4.992138385772705, "operator": "ge"}, {"budget": 0.94, "threshold": -5.112354278564453, "operator": "ge"}, {"budget": 0.95, "threshold": -5.264832496643066, "operator": "ge"}, {"budget": 0.96, "threshold": -5.425079345703125, "operator": "ge"}, {"budget": 0.97, "threshold": -5.624449253082275, "operator": "ge"}, {"budget": 0.98, "threshold": -5.826086521148682, "operator": "ge"}, {"budget": 0.99, "threshold": -6.23495626449585, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.2492122600974, 80.75050128902893, 81.12288742480665, 81.49527356058435, 81.85333715267831, 82.21140074477226, 82.56946433686623, 82.94185047264394, 83.2283013463191, 83.54339730736177, 83.88713835577198, 84.15926668576338, 84.4313950157548, 84.74649097679747, 84.96132913205385, 85.21913491836149, 85.46261816098539, 85.73474649097679, 85.96390718991692, 86.02119736465195, 86.26468060727585, 86.4508736751647, 86.6943569177886, 86.90919507304497, 87.22429103408766, 87.3102262961902, 87.45345173302778, 87.49641936407906, 87.7542251503867, 87.94041821827557, 88.0263534803781, 88.08364365511314, 88.09796619879691, 88.19822400458321, 88.38441707247206, 88.48467487825837, 88.62790031509596, 88.72815812088226, 88.82841592666857, 89.02893153824118, 89.15783443139502, 89.24376969349757, 89.3297049556001, 89.51589802348897, 89.5445431108565, 89.6734460040103, 89.71641363506157, 89.73073617874535, 89.88828415926669, 89.88828415926669, 89.90260670295044, 90.00286450873675, 90.03150959610426, 90.10312231452305, 90.14608994557433, 90.14608994557433, 90.23202520767688, 90.30363792609567, 90.34660555714694, 90.4325408192495, 90.46118590661702, 90.53279862503581, 90.56144371240332, 90.59008879977084, 90.6044113434546, 90.67602406187338, 90.7046691492409, 90.76195932397593, 90.79060441134345, 90.81924949871096, 90.876539673446, 90.90518476081351, 91.01976511028357, 91.0484101976511, 91.0484101976511, 91.07705528501862, 91.07705528501862, 91.13434545975365, 91.17731309080493, 91.13434545975365, 91.06273274133486, 91.10570037238614, 91.14866800343741, 91.14866800343741, 91.16299054712115, 91.12002291606989, 91.06273274133486, 91.09137782870236, 91.06273274133486, 91.01976511028357, 90.9767974792323, 90.96247493554854, 90.876539673446, 90.80492695502721, 90.76195932397593, 90.69034660555714, 90.61873388713835, 90.57576625608708, 90.56144371240332, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 30, 66, 94, 121, 147, 174, 199, 226, 247, 269, 294, 314, 335, 362, 380, 402, 424, 449, 471, 486, 508, 526, 544, 563, 592, 604, 623, 630, 650, 668, 678, 689, 700, 713, 731, 743, 756, 772, 784, 800, 816, 825, 835, 853, 864, 876, 886, 891, 906, 911, 914, 926, 934, 944, 949, 954, 961, 971, 980, 993, 999, 1006, 1010, 1015, 1022, 1030, 1036, 1044, 1051, 1056, 1064, 1068, 1076, 1082, 1085, 1090, 1094, 1102, 1107, 1111, 1113, 1119, 1124, 1128, 1132, 1134, 1136, 1140, 1141, 1143, 1143, 1146, 1147, 1148, 1148, 1149, 1150, 1152, 1153, 1154], "harmed": [0, 0, 1, 3, 4, 5, 7, 7, 8, 9, 9, 10, 11, 13, 18, 21, 25, 30, 36, 42, 53, 58, 63, 64, 68, 75, 81, 90, 94, 96, 101, 105, 112, 122, 128, 133, 138, 141, 150, 155, 157, 164, 167, 171, 176, 185, 188, 195, 199, 203, 208, 210, 215, 221, 226, 228, 233, 234, 239, 245, 252, 256, 258, 260, 263, 269, 272, 276, 280, 285, 288, 292, 294, 294, 298, 301, 304, 308, 312, 314, 321, 328, 331, 333, 337, 340, 345, 351, 353, 356, 361, 364, 368, 375, 381, 384, 390, 396, 401, 403, 408], "exact_score": [79.81953594958465, 80.2492122600974, 80.75050128902893, 81.12288742480665, 81.49527356058435, 81.85333715267831, 82.21140074477226, 82.56946433686623, 82.94185047264394, 83.2283013463191, 83.54339730736177, 83.88713835577198, 84.15926668576338, 84.4313950157548, 84.74649097679747, 84.96132913205385, 85.21913491836149, 85.46261816098539, 85.73474649097679, 85.96390718991692, 86.02119736465195, 86.26468060727585, 86.4508736751647, 86.6943569177886, 86.90919507304497, 87.22429103408766, 87.3102262961902, 87.45345173302778, 87.49641936407906, 87.7542251503867, 87.94041821827557, 88.0263534803781, 88.08364365511314, 88.09796619879691, 88.19822400458321, 88.38441707247206, 88.48467487825837, 88.62790031509596, 88.72815812088226, 88.82841592666857, 89.02893153824118, 89.15783443139502, 89.24376969349757, 89.3297049556001, 89.51589802348897, 89.5445431108565, 89.6734460040103, 89.71641363506157, 89.73073617874535, 89.88828415926669, 89.88828415926669, 89.90260670295044, 90.00286450873675, 90.03150959610426, 90.10312231452305, 90.14608994557433, 90.14608994557433, 90.23202520767688, 90.30363792609567, 90.34660555714694, 90.4325408192495, 90.46118590661702, 90.53279862503581, 90.56144371240332, 90.59008879977084, 90.6044113434546, 90.67602406187338, 90.7046691492409, 90.76195932397593, 90.79060441134345, 90.81924949871096, 90.876539673446, 90.90518476081351, 91.01976511028357, 91.0484101976511, 91.0484101976511, 91.07705528501862, 91.07705528501862, 91.13434545975365, 91.17731309080493, 91.13434545975365, 91.06273274133486, 91.10570037238614, 91.14866800343741, 91.14866800343741, 91.16299054712115, 91.12002291606989, 91.06273274133486, 91.09137782870236, 91.06273274133486, 91.01976511028357, 90.9767974792323, 90.96247493554854, 90.876539673446, 90.80492695502721, 90.76195932397593, 90.69034660555714, 90.61873388713835, 90.57576625608708, 90.56144371240332, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.53930855140784, "threshold_area100": 88.53852764250931, "threshold_area25": 83.98023488971641}, "test": {"score": [80.00287005811867, 80.8208366219416, 81.17241874147952, 81.71055463873144, 82.04778646767598, 82.34914257013705, 82.6648489631915, 82.97338021094927, 83.23886058692689, 83.5976178517615, 83.94202482600272, 84.31513238143073, 84.60213819329842, 84.91066944105619, 85.13309894525365, 85.2766018511875, 85.4559804836048, 85.66405969720887, 85.93671521848317, 86.16631986797732, 86.35287364569133, 86.55377771399871, 86.80490779938295, 86.90535983353664, 87.13496448303079, 87.27129224366794, 87.51524718375548, 87.65157494439264, 87.83095357680993, 88.02468249982063, 88.08925880749085, 88.33321374757838, 88.42649063643539, 88.56999354236925, 88.70632130300639, 88.81394848245677, 88.971801678984, 89.04355313195093, 89.18705603788477, 89.33055894381862, 89.48123699504916, 89.54581330271938, 89.66779077276315, 89.75389251632346, 89.86869484107054, 89.94044629403746, 90.00502260170768, 90.11264978115807, 90.16287579823492, 90.22745210590514, 90.22027696060846, 90.28485326827868, 90.38530530243239, 90.42835617421254, 90.50728277247615, 90.48575733658608, 90.51445791777284, 90.53598335366291, 90.62926024251992, 90.67231111430006, 90.67948625959676, 90.73688742197031, 90.70818684078354, 90.72253713137691, 90.77276314845376, 90.78711343904715, 90.80863887493722, 90.8301643108273, 90.82298916553061, 90.78711343904715, 90.77276314845376, 90.72253713137691, 90.71536198608023, 90.70101169548683, 90.68666140489346, 90.67948625959676, 90.67231111430006, 90.66513596900337, 90.67948625959676, 90.69383655019014, 90.71536198608023, 90.67948625959676, 90.67231111430006, 90.66513596900337, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.5503336442563, 90.52880820836621, 90.50728277247615, 90.47140704599268, 90.45705675539931, 90.44988161010261, 90.39965559302576, 90.40683073832245, 90.3637798665423, 90.31355384946545, 90.25615268709191, 90.22027696060846, 90.14852550764154, 90.06242376408122], "calls": [0.0, 1.4924302217119898, 2.317571930831599, 3.55887206715936, 4.570567553992968, 5.410059553705962, 6.29260242519911, 7.311473057329411, 8.215541364712635, 9.21288656095286, 10.353734663126929, 11.487407620004305, 12.448877089761067, 13.439047140704599, 14.465092918131592, 15.462438114371816, 16.667862524216115, 17.6078065580828, 18.61950204491641, 19.982779651287938, 20.800746215110856, 21.719164813087467, 22.93893951352515, 23.58470259022745, 24.452895171127214, 25.156059410203056, 26.074478008179664, 26.971371170266195, 27.79651287938581, 28.671880605582263, 29.568773767668795, 30.645045562172633, 31.721317356676472, 32.68996197172992, 33.457702518476005, 34.433522278826146, 35.39499174858291, 36.291884910669445, 37.31793068809643, 38.3655019014135, 39.484824567697494, 40.62567266987156, 41.66606873789194, 42.67058907942886, 43.804262036306234, 44.67962976250269, 45.75590155700653, 46.90392480447729, 47.70036593241013, 48.733586855133815, 49.75963263256081, 50.82872928176796, 51.66104613618426, 52.73014278539141, 53.820764870488624, 54.75353375905863, 55.786754681782305, 56.70517327975892, 57.616416732438836, 58.5635359116022, 59.67568343258951, 61.0676616201478, 61.97172992753103, 63.06235201262825, 64.04534691827509, 65.2507713281194, 66.23376623376623, 67.34591375475354, 68.40783525866398, 69.36930472842076, 70.42405108703451, 71.47162230035158, 72.50484322307527, 73.64569132524935, 74.5067087608524, 75.41077706823563, 76.27896964913539, 77.44851833249623, 78.20908373394562, 79.04140058836191, 80.01722034871206, 81.08631699791921, 82.04778646767596, 83.10253282628973, 84.00660113367296, 85.1617995264404, 86.14479443208725, 87.50089689316208, 88.45519121762216, 89.26598263614838, 89.9906723111143, 90.85886489201407, 91.8346846523642, 92.80332926741767, 93.67869699361412, 94.73344335222788, 95.71643825787473, 96.85728636004879, 97.61067661620147, 98.8878524790127, 100.0], "rescued": [0, 118, 171, 251, 302, 346, 393, 444, 485, 541, 593, 651, 696, 744, 785, 815, 850, 885, 930, 974, 1004, 1038, 1080, 1100, 1135, 1161, 1200, 1228, 1262, 1289, 1307, 1346, 1372, 1397, 1418, 1444, 1474, 1496, 1526, 1552, 1581, 1600, 1630, 1650, 1676, 1693, 1714, 1739, 1756, 1778, 1793, 1809, 1831, 1848, 1866, 1874, 1893, 1905, 1923, 1937, 1949, 1966, 1971, 1984, 1998, 2012, 2024, 2034, 2049, 2057, 2068, 2077, 2087, 2096, 2100, 2107, 2111, 2118, 2125, 2136, 2142, 2150, 2157, 2163, 2172, 2176, 2177, 2190, 2198, 2202, 2204, 2212, 2217, 2221, 2226, 2233, 2237, 2239, 2240, 2245, 2247], "harmed": [0, 4, 8, 13, 17, 19, 22, 30, 34, 40, 44, 50, 55, 60, 70, 80, 90, 96, 103, 115, 119, 125, 132, 138, 141, 148, 153, 162, 171, 171, 180, 185, 198, 203, 205, 216, 224, 236, 246, 252, 260, 270, 283, 291, 301, 308, 320, 330, 340, 353, 369, 376, 384, 395, 402, 413, 428, 437, 442, 450, 461, 470, 479, 490, 497, 509, 518, 525, 541, 554, 567, 583, 594, 605, 611, 619, 624, 632, 637, 646, 649, 662, 670, 677, 692, 701, 707, 720, 731, 738, 745, 755, 761, 772, 776, 789, 800, 810, 816, 831, 845], "exact_score": [80.00287005811867, 80.56970653655738, 81.06479156202914, 81.48094998923729, 81.8397072540719, 82.21281480949989, 82.55722178374111, 82.88010332209228, 83.20298486044344, 83.51869125349789, 83.82004735595896, 84.1572791849035, 84.46581043266126, 84.75999138982566, 85.03264691109995, 85.21202554351726, 85.34835330415441, 85.52055679127503, 85.77886202195594, 86.00129152615341, 86.16631986797732, 86.42462509865825, 86.5753031498888, 86.81208294467963, 87.0345124488771, 87.23541651718448, 87.48654660256871, 87.65875008968932, 87.85247901270002, 88.03185764511731, 88.16101026045777, 88.34756403817178, 88.47671665351223, 88.61304441414939, 88.80677333716008, 88.90005022601709, 89.01485255076416, 89.15118031140132, 89.30903350792853, 89.40231039678554, 89.51711272153261, 89.58168902920285, 89.68931620865322, 89.811293678697, 89.86869484107054, 89.96197172992754, 90.01219774700438, 90.12700007175145, 90.19157637942169, 90.19875152471838, 90.22027696060846, 90.29920355887207, 90.39248044772907, 90.46423190069599, 90.50010762717945, 90.51445791777284, 90.52880820836621, 90.57185908014638, 90.62926024251992, 90.64361053311329, 90.70818684078354, 90.73688742197031, 90.70101169548683, 90.72253713137691, 90.77276314845376, 90.79428858434385, 90.80863887493722, 90.80863887493722, 90.81581402023392, 90.79428858434385, 90.79428858434385, 90.73688742197031, 90.72253713137691, 90.73688742197031, 90.68666140489346, 90.67948625959676, 90.67231111430006, 90.67948625959676, 90.69383655019014, 90.69383655019014, 90.71536198608023, 90.69383655019014, 90.67231111430006, 90.66513596900337, 90.62208509722322, 90.60055966133315, 90.5503336442563, 90.57185908014638, 90.52163306306953, 90.50010762717945, 90.47140704599268, 90.44988161010261, 90.43553131950922, 90.40683073832245, 90.39248044772907, 90.33507928535553, 90.31355384946545, 90.24180239649853, 90.16287579823492, 90.14852550764154, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.61906197071109, "threshold_area100": 88.64167324388319, "threshold_area25": 84.31154480878239}, "selected_budget": 79, "selected": {"val": {"score": 91.17731309080493, "calls": 78.98882841592668, "rescued": 1107, "harmed": 314}, "test": {"score": 90.69383655019014, "calls": 79.04140058836191, "rescued": 2136, "harmed": 646}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.4676389694213867, "operator": "ge"}, {"budget": 0.02, "threshold": 2.169177532196045, "operator": "ge"}, {"budget": 0.03, "threshold": 1.939685583114624, "operator": "ge"}, {"budget": 0.04, "threshold": 1.7713255882263184, "operator": "ge"}, {"budget": 0.05, "threshold": 1.6249754428863525, "operator": "ge"}, {"budget": 0.06, "threshold": 1.5062614679336548, "operator": "ge"}, {"budget": 0.07, "threshold": 1.376115083694458, "operator": "ge"}, {"budget": 0.08, "threshold": 1.286224126815796, "operator": "ge"}, {"budget": 0.09, "threshold": 1.1951054334640503, "operator": "ge"}, {"budget": 0.1, "threshold": 1.076676845550537, "operator": "ge"}, {"budget": 0.11, "threshold": 1.0056402683258057, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9214198589324951, "operator": "ge"}, {"budget": 0.13, "threshold": 0.8464112877845764, "operator": "ge"}, {"budget": 0.14, "threshold": 0.7787719964981079, "operator": "ge"}, {"budget": 0.15, "threshold": 0.7002310752868652, "operator": "ge"}, {"budget": 0.16, "threshold": 0.6357752084732056, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5784733295440674, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5068387985229492, "operator": "ge"}, {"budget": 0.19, "threshold": 0.43761885166168213, "operator": "ge"}, {"budget": 0.2, "threshold": 0.3663405776023865, "operator": "ge"}, {"budget": 0.21, "threshold": 0.29735836386680603, "operator": "ge"}, {"budget": 0.22, "threshold": 0.22626620531082153, "operator": "ge"}, {"budget": 0.23, "threshold": 0.16471454501152039, "operator": "ge"}, {"budget": 0.24, "threshold": 0.11154663562774658, "operator": "ge"}, {"budget": 0.25, "threshold": 0.054526835680007935, "operator": "ge"}, {"budget": 0.26, "threshold": -0.014993250370025635, "operator": "ge"}, {"budget": 0.27, "threshold": -0.07094335556030273, "operator": "ge"}, {"budget": 0.28, "threshold": -0.13345932960510254, "operator": "ge"}, {"budget": 0.29, "threshold": -0.19732332229614258, "operator": "ge"}, {"budget": 0.3, "threshold": -0.2649560570716858, "operator": "ge"}, {"budget": 0.31, "threshold": -0.33722662925720215, "operator": "ge"}, {"budget": 0.32, "threshold": -0.3934389352798462, "operator": "ge"}, {"budget": 0.33, "threshold": -0.46115046739578247, "operator": "ge"}, {"budget": 0.34, "threshold": -0.522612988948822, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5952965617179871, "operator": "ge"}, {"budget": 0.36, "threshold": -0.6489987373352051, "operator": "ge"}, {"budget": 0.37, "threshold": -0.7251330018043518, "operator": "ge"}, {"budget": 0.38, "threshold": -0.7993083596229553, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8727961778640747, "operator": "ge"}, {"budget": 0.4, "threshold": -0.9599514603614807, "operator": "ge"}, {"budget": 0.41, "threshold": -1.0283836126327515, "operator": "ge"}, {"budget": 0.42, "threshold": -1.1014779806137085, "operator": "ge"}, {"budget": 0.43, "threshold": -1.164482831954956, "operator": "ge"}, {"budget": 0.44, "threshold": -1.2368032932281494, "operator": "ge"}, {"budget": 0.45, "threshold": -1.3016667366027832, "operator": "ge"}, {"budget": 0.46, "threshold": -1.3616447448730469, "operator": "ge"}, {"budget": 0.47, "threshold": -1.4251549243927002, "operator": "ge"}, {"budget": 0.48, "threshold": -1.4893622398376465, "operator": "ge"}, {"budget": 0.49, "threshold": -1.5611447095870972, "operator": "ge"}, {"budget": 0.5, "threshold": -1.622161626815796, "operator": "ge"}, {"budget": 0.51, "threshold": -1.6851199865341187, "operator": "ge"}, {"budget": 0.52, "threshold": -1.741013526916504, "operator": "ge"}, {"budget": 0.53, "threshold": -1.800062656402588, "operator": "ge"}, {"budget": 0.54, "threshold": -1.8712389469146729, "operator": "ge"}, {"budget": 0.55, "threshold": -1.9354926347732544, "operator": "ge"}, {"budget": 0.56, "threshold": -2.0111236572265625, "operator": "ge"}, {"budget": 0.57, "threshold": -2.072193145751953, "operator": "ge"}, {"budget": 0.58, "threshold": -2.140852689743042, "operator": "ge"}, {"budget": 0.59, "threshold": -2.2050974369049072, "operator": "ge"}, {"budget": 0.6, "threshold": -2.266679286956787, "operator": "ge"}, {"budget": 0.61, "threshold": -2.3474984169006348, "operator": "ge"}, {"budget": 0.62, "threshold": -2.419668197631836, "operator": "ge"}, {"budget": 0.63, "threshold": -2.4761974811553955, "operator": "ge"}, {"budget": 0.64, "threshold": -2.559281826019287, "operator": "ge"}, {"budget": 0.65, "threshold": -2.6268584728240967, "operator": "ge"}, {"budget": 0.66, "threshold": -2.6910219192504883, "operator": "ge"}, {"budget": 0.67, "threshold": -2.7642457485198975, "operator": "ge"}, {"budget": 0.68, "threshold": -2.8340771198272705, "operator": "ge"}, {"budget": 0.69, "threshold": -2.9070873260498047, "operator": "ge"}, {"budget": 0.7, "threshold": -2.987852096557617, "operator": "ge"}, {"budget": 0.71, "threshold": -3.055666923522949, "operator": "ge"}, {"budget": 0.72, "threshold": -3.1294829845428467, "operator": "ge"}, {"budget": 0.73, "threshold": -3.201284408569336, "operator": "ge"}, {"budget": 0.74, "threshold": -3.283581256866455, "operator": "ge"}, {"budget": 0.75, "threshold": -3.3557732105255127, "operator": "ge"}, {"budget": 0.76, "threshold": -3.4250707626342773, "operator": "ge"}, {"budget": 0.77, "threshold": -3.4996795654296875, "operator": "ge"}, {"budget": 0.78, "threshold": -3.5922610759735107, "operator": "ge"}, {"budget": 0.79, "threshold": -3.6380457878112793, "operator": "ge"}, {"budget": 0.8, "threshold": -3.717101573944092, "operator": "ge"}, {"budget": 0.81, "threshold": -3.7868332862854004, "operator": "ge"}, {"budget": 0.82, "threshold": -3.864501953125, "operator": "ge"}, {"budget": 0.83, "threshold": -3.9395816326141357, "operator": "ge"}, {"budget": 0.84, "threshold": -4.029452323913574, "operator": "ge"}, {"budget": 0.85, "threshold": -4.107977867126465, "operator": "ge"}, {"budget": 0.86, "threshold": -4.206707000732422, "operator": "ge"}, {"budget": 0.87, "threshold": -4.3110504150390625, "operator": "ge"}, {"budget": 0.88, "threshold": -4.407991409301758, "operator": "ge"}, {"budget": 0.89, "threshold": -4.503981113433838, "operator": "ge"}, {"budget": 0.9, "threshold": -4.57857608795166, "operator": "ge"}, {"budget": 0.91, "threshold": -4.704875469207764, "operator": "ge"}, {"budget": 0.92, "threshold": -4.7874674797058105, "operator": "ge"}, {"budget": 0.93, "threshold": -4.910084247589111, "operator": "ge"}, {"budget": 0.94, "threshold": -5.039321422576904, "operator": "ge"}, {"budget": 0.95, "threshold": -5.1818695068359375, "operator": "ge"}, {"budget": 0.96, "threshold": -5.362264156341553, "operator": "ge"}, {"budget": 0.97, "threshold": -5.538600921630859, "operator": "ge"}, {"budget": 0.98, "threshold": -5.769251823425293, "operator": "ge"}, {"budget": 0.99, "threshold": -6.2025861740112305, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.33514752219993, 80.77914637639644, 81.25179031796046, 81.62417645373819, 81.93927241478086, 82.3403036379261, 82.62675451160125, 82.98481810369522, 83.31423660842165, 83.60068748209682, 83.88713835577198, 84.17358922944715, 84.41707247207103, 84.66055571469494, 84.96132913205385, 85.23345746204527, 85.47694070466915, 85.79203666571182, 85.89229447149813, 86.09281008307075, 86.22171297622458, 86.46519621884846, 86.63706674305357, 86.83758235462618, 87.19564594672013, 87.3102262961902, 87.46777427671154, 87.4820968203953, 87.73990260670294, 87.89745058722428, 88.0263534803781, 88.1695789172157, 88.21254654826697, 88.22686909195072, 88.31280435405327, 88.5133199656259, 88.58493268404469, 88.71383557719851, 88.7854482956173, 88.98596390718991, 89.08622171297623, 89.25809223718132, 89.42996276138642, 89.573188198224, 89.60183328559151, 89.65912346032655, 89.71641363506157, 89.75938126611285, 89.75938126611285, 89.87396161558291, 89.9169292466342, 89.98854196505299, 90.0171870524205, 90.18905757662561, 90.11744485820681, 90.17473503294184, 90.23202520767688, 90.24634775136063, 90.37525064451447, 90.4325408192495, 90.46118590661702, 90.57576625608708, 90.59008879977084, 90.6044113434546, 90.66170151818963, 90.7046691492409, 90.7046691492409, 90.74763678029217, 90.79060441134345, 90.80492695502721, 90.876539673446, 90.90518476081351, 90.9767974792323, 91.07705528501862, 91.06273274133486, 91.13434545975365, 91.12002291606989, 91.12002291606989, 91.14866800343741, 91.13434545975365, 91.10570037238614, 91.13434545975365, 91.12002291606989, 91.10570037238614, 91.14866800343741, 91.12002291606989, 91.07705528501862, 91.09137782870236, 91.09137782870236, 91.01976511028357, 90.99112002291606, 90.9481523918648, 90.876539673446, 90.81924949871096, 90.77628186765969, 90.7046691492409, 90.63305643082211, 90.57576625608708, 90.56144371240332, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 37, 69, 103, 130, 153, 181, 201, 230, 253, 277, 299, 320, 340, 363, 386, 410, 433, 458, 473, 492, 510, 527, 543, 561, 590, 604, 622, 629, 649, 665, 678, 692, 702, 714, 724, 744, 755, 770, 780, 796, 811, 826, 839, 854, 864, 875, 882, 891, 900, 912, 915, 924, 932, 948, 950, 955, 964, 969, 982, 993, 999, 1009, 1012, 1017, 1023, 1033, 1038, 1045, 1052, 1055, 1064, 1068, 1073, 1081, 1088, 1095, 1098, 1102, 1108, 1109, 1114, 1121, 1122, 1127, 1134, 1135, 1137, 1141, 1143, 1143, 1143, 1145, 1147, 1148, 1149, 1149, 1150, 1152, 1153, 1154], "harmed": [0, 1, 2, 3, 4, 5, 5, 5, 9, 9, 13, 15, 16, 19, 25, 27, 32, 38, 41, 49, 54, 63, 63, 67, 71, 75, 81, 88, 94, 96, 101, 105, 109, 116, 127, 131, 137, 143, 149, 154, 156, 164, 167, 168, 173, 181, 188, 191, 197, 206, 210, 210, 214, 220, 224, 231, 232, 237, 241, 245, 252, 256, 258, 260, 264, 266, 273, 278, 282, 286, 288, 292, 294, 294, 295, 303, 305, 309, 313, 317, 319, 326, 331, 333, 339, 343, 346, 351, 354, 356, 361, 363, 368, 375, 380, 384, 389, 395, 401, 403, 408], "exact_score": [79.81953594958465, 80.33514752219993, 80.77914637639644, 81.25179031796046, 81.62417645373819, 81.93927241478086, 82.3403036379261, 82.62675451160125, 82.98481810369522, 83.31423660842165, 83.60068748209682, 83.88713835577198, 84.17358922944715, 84.41707247207103, 84.66055571469494, 84.96132913205385, 85.23345746204527, 85.47694070466915, 85.79203666571182, 85.89229447149813, 86.09281008307075, 86.22171297622458, 86.46519621884846, 86.63706674305357, 86.83758235462618, 87.19564594672013, 87.3102262961902, 87.46777427671154, 87.4820968203953, 87.73990260670294, 87.89745058722428, 88.0263534803781, 88.1695789172157, 88.21254654826697, 88.22686909195072, 88.31280435405327, 88.5133199656259, 88.58493268404469, 88.71383557719851, 88.7854482956173, 88.98596390718991, 89.08622171297623, 89.25809223718132, 89.42996276138642, 89.573188198224, 89.60183328559151, 89.65912346032655, 89.71641363506157, 89.75938126611285, 89.75938126611285, 89.87396161558291, 89.9169292466342, 89.98854196505299, 90.0171870524205, 90.18905757662561, 90.11744485820681, 90.17473503294184, 90.23202520767688, 90.24634775136063, 90.37525064451447, 90.4325408192495, 90.46118590661702, 90.57576625608708, 90.59008879977084, 90.6044113434546, 90.66170151818963, 90.7046691492409, 90.7046691492409, 90.74763678029217, 90.79060441134345, 90.80492695502721, 90.876539673446, 90.90518476081351, 90.9767974792323, 91.07705528501862, 91.06273274133486, 91.13434545975365, 91.12002291606989, 91.12002291606989, 91.14866800343741, 91.13434545975365, 91.10570037238614, 91.13434545975365, 91.12002291606989, 91.10570037238614, 91.14866800343741, 91.12002291606989, 91.07705528501862, 91.09137782870236, 91.09137782870236, 91.01976511028357, 90.99112002291606, 90.9481523918648, 90.876539673446, 90.81924949871096, 90.77628186765969, 90.7046691492409, 90.63305643082211, 90.57576625608708, 90.56144371240332, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.54917760864969, "threshold_area100": 88.54841019765108, "threshold_area25": 84.02033801203093}, "test": {"score": [80.00287005811867, 80.64145798952428, 81.01456554495229, 81.41637368156705, 81.76795580110497, 82.21281480949989, 82.60027265552128, 82.90162875798235, 83.19580971514674, 83.54739183468466, 84.1572791849035, 84.3510081079142, 84.66671450096864, 84.83174284279256, 85.03264691109995, 85.19767525292387, 85.59948338953865, 85.70711056898904, 85.92236492788979, 86.13761928679055, 86.3815742268781, 86.5681280045921, 86.77620721819616, 86.94123556002009, 87.14213962832747, 87.27129224366794, 87.52242232905216, 87.65157494439264, 87.84530386740332, 88.02468249982063, 88.1394848245677, 88.35473918346847, 88.45519121762216, 88.56281839707253, 88.69914615770969, 88.8856999354237, 88.95745138839062, 89.0722537131377, 89.13683002080792, 89.30903350792853, 89.50276243093923, 89.58168902920285, 89.65344048216977, 89.69649135394992, 89.84716940518047, 89.93327114874077, 90.01219774700438, 90.03372318289445, 90.14135036234484, 90.15570065293824, 90.20592667001507, 90.277678122982, 90.41400588361914, 90.46423190069599, 90.49293248188276, 90.51445791777284, 90.57185908014638, 90.57185908014638, 90.59338451603645, 90.64361053311329, 90.65796082370669, 90.70818684078354, 90.70101169548683, 90.73688742197031, 90.80863887493722, 90.82298916553061, 90.81581402023392, 90.82298916553061, 90.85886489201407, 90.77993829375045, 90.77276314845376, 90.76558800315708, 90.744062567267, 90.70101169548683, 90.70101169548683, 90.65796082370669, 90.65796082370669, 90.66513596900337, 90.67948625959676, 90.67231111430006, 90.65796082370669, 90.70101169548683, 90.70101169548683, 90.67948625959676, 90.67948625959676, 90.57185908014638, 90.5503336442563, 90.57903422544307, 90.51445791777284, 90.47858219128938, 90.48575733658608, 90.45705675539931, 90.45705675539931, 90.39965559302576, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.26332783238861, 90.19157637942169, 90.14852550764154, 90.06242376408122], "calls": [0.0, 1.3058764439979909, 2.1525435890076774, 3.1068379134677477, 3.9319796225873573, 5.02260170768458, 6.048647485111573, 7.153619860802181, 8.03616273229533, 8.9904570567554, 10.411135825500466, 11.351079859367152, 12.463227380354452, 13.3457702518476, 14.192437396857285, 15.426562387888353, 16.52435961828227, 17.299275310325037, 18.49034942957595, 19.437468608739326, 20.52809069383655, 21.582837052450312, 22.694984573437612, 23.61340317141422, 24.395494008753676, 25.177584846093133, 26.096003444069744, 26.863743990815813, 27.79651287938581, 28.743632058549185, 29.68357609241587, 30.78854846810648, 31.606515031929394, 32.546459065796085, 33.414651646695845, 34.49092344119968, 35.35194087680276, 36.44973810719667, 37.4399081581402, 38.473129080863885, 39.7574800889718, 40.76917557580541, 41.85979766090263, 42.864318002439546, 43.89036377986654, 44.90205926670015, 45.82047786467676, 46.78194733443352, 47.78646767597044, 48.75511229102389, 49.6735308890005, 50.7856784099878, 51.704097007964414, 52.586639879457564, 53.74183827222502, 54.75353375905863, 55.93025758771615, 56.64777211738537, 57.65946760421898, 58.57071105689891, 59.611107124919286, 60.97438473129081, 62.07218196168473, 62.99060055966134, 64.10992322594532, 65.07139269570209, 66.09026332783239, 67.08760852407262, 68.14953002798306, 69.34777929253067, 70.56755399296836, 71.4572720097582, 72.51201836837195, 73.42326182105188, 74.49235847025902, 75.39642677764225, 76.42247255506925, 77.30501542656239, 78.45303867403315, 79.02705029776853, 80.08179665638228, 80.89258807490852, 82.08366219415943, 83.05230680921288, 84.1501040396068, 85.09722321877018, 86.23807132094426, 87.4004448590084, 88.38343976465524, 89.28750807203846, 89.98349716581761, 91.03106837913467, 91.67683145583698, 92.73875295974744, 93.76479873717443, 94.64734160866757, 95.7379636937648, 96.78553490708187, 97.66807777857501, 98.93807849608955, 100.0], "rescued": [0, 95, 148, 208, 261, 329, 389, 438, 484, 538, 627, 661, 718, 746, 777, 812, 871, 895, 934, 969, 1009, 1042, 1078, 1107, 1136, 1162, 1201, 1225, 1262, 1290, 1312, 1348, 1369, 1392, 1417, 1449, 1471, 1494, 1520, 1549, 1582, 1600, 1624, 1643, 1670, 1692, 1710, 1726, 1750, 1770, 1786, 1808, 1831, 1845, 1864, 1877, 1896, 1905, 1918, 1933, 1946, 1963, 1971, 1983, 2000, 2012, 2023, 2031, 2045, 2056, 2071, 2078, 2092, 2095, 2103, 2105, 2112, 2118, 2131, 2136, 2142, 2151, 2160, 2164, 2173, 2175, 2180, 2190, 2196, 2200, 2207, 2214, 2218, 2223, 2228, 2233, 2237, 2239, 2240, 2246, 2247], "harmed": [0, 6, 7, 11, 15, 21, 27, 34, 39, 44, 48, 55, 68, 73, 76, 88, 91, 100, 109, 114, 120, 127, 134, 140, 141, 149, 153, 159, 169, 172, 178, 184, 191, 199, 205, 211, 223, 230, 247, 252, 258, 265, 279, 292, 298, 308, 315, 328, 337, 355, 364, 376, 380, 387, 402, 412, 423, 432, 442, 450, 461, 471, 480, 487, 494, 504, 516, 523, 532, 554, 570, 578, 595, 604, 612, 620, 627, 632, 643, 649, 657, 660, 669, 676, 685, 702, 710, 716, 731, 740, 746, 757, 761, 774, 780, 789, 801, 809, 820, 832, 845], "exact_score": [80.00287005811867, 80.51948051948052, 80.94281409198537, 81.35897251919351, 81.78230609169836, 82.1984645189065, 82.58592236492788, 82.8585778862022, 83.18863456985004, 83.55456697998135, 83.9635502618928, 84.27208150965058, 84.54473703092488, 84.77434168041903, 84.99677118461649, 85.15462438114372, 85.4488053383081, 85.69993542369232, 85.8290880390328, 86.02999210734018, 86.2954724833178, 86.4676759704384, 86.67575518404249, 86.80490779938295, 87.06321303006386, 87.24259166248117, 87.4937217478654, 87.665925234986, 87.8596541579967, 88.08208366219417, 88.16818540575447, 88.37626461935855, 88.48389179880894, 88.62739470474278, 88.79959819186338, 88.93592595250054, 89.05072827724761, 89.10095429432447, 89.23728205496163, 89.39513525148884, 89.51711272153261, 89.59603931979623, 89.63909019157639, 89.73954222573008, 89.86151969577384, 89.92609600344407, 90.03372318289445, 90.05524861878453, 90.14135036234484, 90.1700509435316, 90.22027696060846, 90.32790414005885, 90.44988161010261, 90.47140704599268, 90.48575733658608, 90.52880820836621, 90.57185908014638, 90.59338451603645, 90.60055966133315, 90.65796082370669, 90.65796082370669, 90.70818684078354, 90.69383655019014, 90.73688742197031, 90.78711343904715, 90.81581402023392, 90.81581402023392, 90.80146372964053, 90.87321518260745, 90.80863887493722, 90.77993829375045, 90.78711343904715, 90.75123771256368, 90.71536198608023, 90.69383655019014, 90.66513596900337, 90.67231111430006, 90.66513596900337, 90.68666140489346, 90.67948625959676, 90.65796082370669, 90.70818684078354, 90.70101169548683, 90.67948625959676, 90.67948625959676, 90.57903422544307, 90.56468393484968, 90.5503336442563, 90.53598335366291, 90.50010762717945, 90.48575733658608, 90.46423190069599, 90.44988161010261, 90.39248044772907, 90.40683073832245, 90.34225443065222, 90.30637870416876, 90.24180239649853, 90.1772260888283, 90.14135036234484, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.62546127156763, "threshold_area100": 88.63248905790344, "threshold_area25": 84.31728492501975}, "selected_budget": 79, "selected": {"val": {"score": 91.14866800343741, "calls": 78.98882841592668, "rescued": 1108, "harmed": 317}, "test": {"score": 90.67231111430006, "calls": 79.02705029776853, "rescued": 2136, "harmed": 649}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.84789458607848, "calls": 97.99484388427385}, "test": {"score": 90.39248044772907, "calls": 97.71830379565186}}, "old_normalized_interior": {"val": {"score": 91.12002291606989, "calls": 80.99398453165281}, "test": {"score": 90.6364353878166, "calls": 80.49795508359044}}, "old_normalized_endpoints": {"val": {"score": 91.12002291606989, "calls": 80.99398453165281}, "test": {"score": 90.6364353878166, "calls": 80.49795508359044}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.2778573474649, 80.65024348324262, 81.07991979375537, 81.45230592953308, 81.89630478372959, 82.15411057003723, 82.54081924949871, 82.91320538527643, 83.4001718705242, 83.72959037525064, 84.1162990547121, 84.31681466628473, 84.57462045259237, 84.83242623890003, 85.13319965625895, 85.3193927241478, 85.47694070466915, 85.70610140360928, 85.9352621025494, 86.1071326267545, 86.40790604411343, 86.63706674305357, 86.66571183042107, 86.79461472357491, 87.05242050988255, 87.15267831566887, 87.19564594672013, 87.38183901460899, 87.46777427671154, 87.65396734460039, 87.79719278143799, 87.9977083930106, 88.14093382984818, 88.269836723002, 88.35577198510455, 88.47035233457461, 88.55628759667717, 88.75680320824978, 88.8857061014036, 89.01460899455743, 89.0575766256087, 89.17215697507876, 89.34402749928387, 89.48725293612145, 89.48725293612145, 89.60183328559151, 89.68776854769406, 89.6734460040103, 89.73073617874535, 89.74505872242909, 89.80234889716414, 89.85963907189917, 89.98854196505299, 90.06015468347178, 90.13176740189057, 90.21770266399312, 90.24634775136063, 90.21770266399312, 90.31796046977942, 90.30363792609567, 90.37525064451447, 90.4325408192495, 90.48983099398453, 90.50415353766829, 90.53279862503581, 90.51847608135203, 90.56144371240332, 90.54712116871956, 90.59008879977084, 90.57576625608708, 90.57576625608708, 90.66170151818963, 90.69034660555714, 90.80492695502721, 90.80492695502721, 90.876539673446, 90.96247493554854, 91.01976511028357, 91.0484101976511, 91.09137782870236, 91.12002291606989, 91.12002291606989, 91.06273274133486, 91.07705528501862, 91.09137782870236, 91.06273274133486, 91.03408765396735, 90.99112002291606, 90.90518476081351, 90.86221712976223, 90.83357204239474, 90.86221712976223, 90.91950730449727, 90.876539673446, 90.80492695502721, 90.76195932397593, 90.71899169292466, 90.66170151818963, 90.61873388713835, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.2778573474649, 80.65024348324262, 81.07991979375537, 81.45230592953308, 81.89630478372959, 82.15411057003723, 82.54081924949871, 82.91320538527643, 83.4001718705242, 83.72959037525064, 84.1162990547121, 84.31681466628473, 84.57462045259237, 84.83242623890003, 85.13319965625895, 85.3193927241478, 85.47694070466915, 85.70610140360928, 85.9352621025494, 86.1071326267545, 86.40790604411343, 86.63706674305357, 86.66571183042107, 86.79461472357491, 87.05242050988255, 87.15267831566887, 87.19564594672013, 87.38183901460899, 87.46777427671154, 87.65396734460039, 87.79719278143799, 87.9977083930106, 88.14093382984818, 88.269836723002, 88.35577198510455, 88.47035233457461, 88.55628759667717, 88.75680320824978, 88.8857061014036, 89.01460899455743, 89.0575766256087, 89.17215697507876, 89.34402749928387, 89.48725293612145, 89.48725293612145, 89.60183328559151, 89.68776854769406, 89.6734460040103, 89.73073617874535, 89.74505872242909, 89.80234889716414, 89.85963907189917, 89.98854196505299, 90.06015468347178, 90.13176740189057, 90.21770266399312, 90.24634775136063, 90.21770266399312, 90.31796046977942, 90.30363792609567, 90.37525064451447, 90.4325408192495, 90.48983099398453, 90.50415353766829, 90.53279862503581, 90.51847608135203, 90.56144371240332, 90.54712116871956, 90.59008879977084, 90.57576625608708, 90.57576625608708, 90.66170151818963, 90.69034660555714, 90.80492695502721, 90.80492695502721, 90.876539673446, 90.96247493554854, 91.01976511028357, 91.0484101976511, 91.09137782870236, 91.12002291606989, 91.12002291606989, 91.06273274133486, 91.07705528501862, 91.09137782870236, 91.06273274133486, 91.03408765396735, 90.99112002291606, 90.90518476081351, 90.86221712976223, 90.83357204239474, 90.86221712976223, 90.91950730449727, 90.876539673446, 90.80492695502721, 90.76195932397593, 90.71899169292466, 90.66170151818963, 90.61873388713835, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 84.06044113434547, "exact_area100": 88.48685033766493}, "test": {"score": [80.00287005811867, 80.53383081007391, 81.09349214321591, 81.49530027983067, 81.95450957881899, 82.34196742484036, 82.6720241084882, 82.91597904857574, 83.26756116811366, 83.5043409629045, 83.85592308244242, 84.09987802252996, 84.4227595608811, 84.76716653512234, 85.06134749228671, 85.26225156059411, 85.58513309894525, 85.77886202195594, 85.87931405610964, 86.18067015857072, 86.39592451747149, 86.5609528592954, 86.87665925234987, 86.98428643180026, 87.1134390471407, 87.24976680777786, 87.37174427782163, 87.52959747434886, 87.74485183324963, 87.93858075626032, 87.97445648274378, 88.08925880749085, 88.24711200401808, 88.39779005524862, 88.49106694410563, 88.64892014063285, 88.80677333716008, 88.94310109779724, 89.01485255076416, 89.12247973021454, 89.22293176436823, 89.3520843797087, 89.45971155915908, 89.56016359331277, 89.60321446509292, 89.66061562746646, 89.76824280691685, 89.91892085814737, 90.01219774700438, 90.10547463586138, 90.19157637942169, 90.18440123412499, 90.28485326827868, 90.29202841357538, 90.34942957594892, 90.37813015713569, 90.39248044772907, 90.42835617421254, 90.46423190069599, 90.5503336442563, 90.57185908014638, 90.57185908014638, 90.57903422544307, 90.56468393484968, 90.59338451603645, 90.68666140489346, 90.67231111430006, 90.67948625959676, 90.67231111430006, 90.69383655019014, 90.7297122766736, 90.744062567267, 90.77276314845376, 90.75841285786038, 90.73688742197031, 90.744062567267, 90.71536198608023, 90.68666140489346, 90.67948625959676, 90.64361053311329, 90.6364353878166, 90.6364353878166, 90.65078567840999, 90.64361053311329, 90.65796082370669, 90.66513596900337, 90.65078567840999, 90.62208509722322, 90.6364353878166, 90.61490995192652, 90.57903422544307, 90.56468393484968, 90.55750878955298, 90.50010762717945, 90.47858219128938, 90.39248044772907, 90.3637798665423, 90.29202841357538, 90.29202841357538, 90.19875152471838, 90.06242376408122], "calls": [0.0, 1.0906220850972232, 2.3319222214249837, 3.2288153835115163, 4.41988950276243, 5.474635861376194, 6.407404749946187, 7.34017363851618, 8.35904427064648, 9.177010834469398, 10.2891583554567, 11.257802970510152, 12.06859438903638, 13.302719380067446, 14.278539140417593, 15.326110353734663, 16.517184472985576, 17.7010834469398, 18.61950204491641, 19.896677907727632, 20.772045633924087, 21.72633995838416, 22.694984573437612, 23.426849393700223, 24.208940231039676, 24.91927961541221, 25.83052306809213, 26.512161871277897, 27.545382794001576, 28.578603716725265, 29.166965631054033, 30.322164023821486, 31.204706895314633, 32.28815383511516, 32.941092057114155, 33.924086962760995, 35.1223362273086, 36.02640453469183, 37.23900408983282, 38.39420248260027, 39.18346846523642, 40.1305876443998, 41.37188778072756, 42.31183181459425, 43.201549831384085, 44.09126784817393, 44.988161010260455, 46.1505345483246, 47.284207505201984, 48.51115735093636, 49.4869771112865, 50.57759919638373, 51.42426634139341, 52.622515605941025, 53.69878740044486, 54.88986151969577, 55.715003228815384, 56.805625313912614, 57.96799885197675, 59.07297122766736, 60.04879098801751, 61.10353734663126, 62.194159431728494, 62.99777570495802, 64.01664633708832, 64.94224008036163, 65.96111071249193, 66.93693047284206, 67.8409987802253, 68.86704455765229, 69.94331635215613, 70.96936212958312, 72.08150965057042, 73.05732941092057, 74.0403243165674, 75.02331922221424, 76.02783956375117, 76.89603214465093, 77.73552414436392, 78.71851905001076, 79.57236134031714, 80.49795508359044, 81.8325321087752, 83.1312334074765, 84.03530171485973, 85.14744923584703, 86.05151754323025, 87.23541651718448, 88.14665996986439, 89.08660400373107, 89.98349716581761, 91.04541866972806, 91.85621008825429, 92.85355528449452, 93.67869699361412, 94.6186410274808, 95.58011049723757, 96.92186266771901, 97.9550835904427, 98.8878524790127, 100.0], "exact_score": [80.00287005811867, 80.49077993829376, 80.95716438257875, 81.40202339097367, 81.75360551051159, 82.16258879242305, 82.56439692903783, 82.81552701442205, 83.14558369806988, 83.43258950993759, 83.74112075769534, 84.02812656956303, 84.41558441558442, 84.65953935567195, 84.96807060342972, 85.19767525292387, 85.43445504771472, 85.67840998780225, 85.8219128937361, 85.94389036377987, 86.19502044916409, 86.44615053454832, 86.61835402166894, 86.89818468823994, 87.05603788476716, 87.25694195307454, 87.4004448590084, 87.63004950850255, 87.83812872210662, 87.93858075626032, 88.06055822630408, 88.23276171342471, 88.3260386022817, 88.505417234699, 88.64892014063285, 88.7924230465667, 88.92157566190716, 88.96462653368731, 89.07942885843438, 89.19423118318146, 89.32338379852193, 89.44536126856569, 89.52428786682931, 89.56733873860946, 89.64626533687307, 89.76824280691685, 89.8902202769606, 89.997847456411, 90.09112434526799, 90.1700509435316, 90.1772260888283, 90.23462725120184, 90.27050297768531, 90.32072899476215, 90.38530530243239, 90.37095501183899, 90.37813015713569, 90.42835617421254, 90.47140704599268, 90.52880820836621, 90.56468393484968, 90.57185908014638, 90.58620937073975, 90.56468393484968, 90.58620937073975, 90.68666140489346, 90.66513596900337, 90.67231111430006, 90.67231111430006, 90.70818684078354, 90.7297122766736, 90.75841285786038, 90.77276314845376, 90.73688742197031, 90.73688742197031, 90.73688742197031, 90.71536198608023, 90.68666140489346, 90.65796082370669, 90.64361053311329, 90.64361053311329, 90.6364353878166, 90.6364353878166, 90.65078567840999, 90.65796082370669, 90.67231111430006, 90.65078567840999, 90.60773480662984, 90.65078567840999, 90.61490995192652, 90.57903422544307, 90.55750878955298, 90.5503336442563, 90.49293248188276, 90.46423190069599, 90.39248044772907, 90.32790414005885, 90.28485326827868, 90.29202841357538, 90.19157637942169, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 84.29647700365932, "exact_area100": 88.56683173173282}}, "old_normalized_endpoint_budget": 81}, {"seed": 2028, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 1.0, "validation_area100": 0.8856807261722474, "training_seconds": 33.70334718597587, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8017870066167821, "operator": "ge"}, {"budget": 0.02, "threshold": 0.7494557728049791, "operator": "ge"}, {"budget": 0.03, "threshold": 0.7038093371650661, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6674455654826331, "operator": "ge"}, {"budget": 0.05, "threshold": 0.6273106846345151, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5933569210624183, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5609937025990208, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5228826193534313, "operator": "ge"}, {"budget": 0.09, "threshold": 0.49241857579793447, "operator": "ge"}, {"budget": 0.1, "threshold": 0.46113205971762183, "operator": "ge"}, {"budget": 0.11, "threshold": 0.4336749724649526, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3996928632038799, "operator": "ge"}, {"budget": 0.13, "threshold": 0.37363985979849995, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3458119363097979, "operator": "ge"}, {"budget": 0.15, "threshold": 0.31787093092714724, "operator": "ge"}, {"budget": 0.16, "threshold": 0.2901951117019249, "operator": "ge"}, {"budget": 0.17, "threshold": 0.25343208437949993, "operator": "ge"}, {"budget": 0.18, "threshold": 0.22081612552612703, "operator": "ge"}, {"budget": 0.19, "threshold": 0.1855524557608162, "operator": "ge"}, {"budget": 0.2, "threshold": 0.15250905523660238, "operator": "ge"}, {"budget": 0.21, "threshold": 0.11070525398134877, "operator": "ge"}, {"budget": 0.22, "threshold": 0.07862422222453298, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0463298058634323, "operator": "ge"}, {"budget": 0.24, "threshold": 0.01684920565144354, "operator": "ge"}, {"budget": 0.25, "threshold": -0.021877629933056194, "operator": "ge"}, {"budget": 0.26, "threshold": -0.058596617374411276, "operator": "ge"}, {"budget": 0.27, "threshold": -0.09472922400023367, "operator": "ge"}, {"budget": 0.28, "threshold": -0.13569968692191975, "operator": "ge"}, {"budget": 0.29, "threshold": -0.17101425772316164, "operator": "ge"}, {"budget": 0.3, "threshold": -0.21120392527771914, "operator": "ge"}, {"budget": 0.31, "threshold": -0.2415336824539028, "operator": "ge"}, {"budget": 0.32, "threshold": -0.2800671484508277, "operator": "ge"}, {"budget": 0.33, "threshold": -0.31162284611293983, "operator": "ge"}, {"budget": 0.34, "threshold": -0.3486049238862223, "operator": "ge"}, {"budget": 0.35, "threshold": -0.38092665875083337, "operator": "ge"}, {"budget": 0.36, "threshold": -0.41281422867368667, "operator": "ge"}, {"budget": 0.37, "threshold": -0.44942905284109724, "operator": "ge"}, {"budget": 0.38, "threshold": -0.481369961175206, "operator": "ge"}, {"budget": 0.39, "threshold": -0.5133655313979779, "operator": "ge"}, {"budget": 0.4, "threshold": -0.5480677718872051, "operator": "ge"}, {"budget": 0.41, "threshold": -0.5757329041680334, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6026815601755675, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6248757408971102, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6474022143908695, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6711465077289474, "operator": "ge"}, {"budget": 0.46, "threshold": -0.6951450976025642, "operator": "ge"}, {"budget": 0.47, "threshold": -0.7192631232823731, "operator": "ge"}, {"budget": 0.48, "threshold": -0.7354602772035825, "operator": "ge"}, {"budget": 0.49, "threshold": -0.7551254839836272, "operator": "ge"}, {"budget": 0.5, "threshold": -0.7705416134758971, "operator": "ge"}, {"budget": 0.51, "threshold": -0.7874655980123991, "operator": "ge"}, {"budget": 0.52, "threshold": -0.8027807454999919, "operator": "ge"}, {"budget": 0.53, "threshold": -0.8176602204041556, "operator": "ge"}, {"budget": 0.54, "threshold": -0.8332057076972909, "operator": "ge"}, {"budget": 0.55, "threshold": -0.8472219840509884, "operator": "ge"}, {"budget": 0.56, "threshold": -0.8591668657320606, "operator": "ge"}, {"budget": 0.57, "threshold": -0.8695434164077402, "operator": "ge"}, {"budget": 0.58, "threshold": -0.8793850310165334, "operator": "ge"}, {"budget": 0.59, "threshold": -0.886401651206604, "operator": "ge"}, {"budget": 0.6, "threshold": -0.8954252082164335, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9044108017006097, "operator": "ge"}, {"budget": 0.62, "threshold": -0.91143542164769, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9182378313823168, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9239412200003718, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9297444980225684, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9356634208826834, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9408189817943173, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9457998893259616, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9501065143235208, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9541989654935691, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9572911682292145, "operator": "ge"}, {"budget": 0.72, "threshold": -0.960388929748291, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9638106460027142, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9669144402842187, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9693690216691409, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9718402343517852, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9740624272161694, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9755919371774915, "operator": "ge"}, {"budget": 0.79, "threshold": -0.977332169135984, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9790312361709629, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9807372902790111, "operator": "ge"}, {"budget": 0.82, "threshold": -0.982451598738498, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9836206765527595, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9850203557617473, "operator": "ge"}, {"budget": 0.85, "threshold": -0.986248087715809, "operator": "ge"}, {"budget": 0.86, "threshold": -0.987239473688364, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9883600162774965, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9895362191722898, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9906119153114747, "operator": "ge"}, {"budget": 0.9, "threshold": -0.991499670495478, "operator": "ge"}, {"budget": 0.91, "threshold": -0.992536705141395, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9935272972486524, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9945561261264659, "operator": "ge"}, {"budget": 0.94, "threshold": -0.995263563064657, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9959343865751165, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9965856899167473, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9972816475459026, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9980087901106819, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9986221624352369, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.32082497851619, 80.7934689200802, 81.10856488112287, 81.53824119163563, 82.03953022056717, 82.41191635634489, 82.81294757949011, 83.09939845316528, 83.37152678315668, 83.54339730736177, 83.87281581208822, 84.1162990547121, 84.41707247207103, 84.61758808364365, 85.0472643941564, 85.30507018046404, 85.62016614150673, 85.8063592093956, 86.04984245201948, 86.30764823832712, 86.43655113148094, 86.56545402463478, 86.75164709252363, 86.86622744199369, 87.05242050988255, 87.2099684904039, 87.5107419077628, 87.69693497565167, 87.89745058722428, 87.94041821827557, 88.06932111142939, 88.1695789172157, 88.31280435405327, 88.42738470352333, 88.57061014036091, 88.64222285877972, 88.71383557719851, 88.79977083930105, 88.92867373245488, 89.01460899455743, 89.17215697507876, 89.30105986823258, 89.4013176740189, 89.47293039243769, 89.573188198224, 89.73073617874535, 89.75938126611285, 89.83099398453164, 89.87396161558291, 89.97421942136923, 90.10312231452305, 90.14608994557433, 90.16041248925809, 90.21770266399312, 90.34660555714694, 90.38957318819823, 90.38957318819823, 90.41821827556574, 90.47550845030077, 90.37525064451447, 90.44686336293324, 90.50415353766829, 90.59008879977084, 90.61873388713835, 90.61873388713835, 90.61873388713835, 90.63305643082211, 90.59008879977084, 90.63305643082211, 90.74763678029217, 90.76195932397593, 90.77628186765969, 90.84789458607848, 90.90518476081351, 90.90518476081351, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.9481523918648, 91.00544256659983, 90.99112002291606, 90.91950730449727, 90.89086221712975, 90.90518476081351, 90.99112002291606, 90.99112002291606, 91.06273274133486, 91.01976511028357, 91.09137782870236, 91.07705528501862, 90.99112002291606, 90.9767974792323, 90.96247493554854, 90.91950730449727, 90.89086221712975, 90.81924949871096, 90.74763678029217, 90.69034660555714, 90.67602406187338, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 35, 68, 92, 123, 159, 186, 215, 238, 258, 272, 299, 322, 347, 365, 395, 418, 444, 461, 483, 505, 521, 538, 557, 573, 592, 612, 636, 654, 674, 683, 699, 711, 724, 736, 754, 763, 775, 785, 798, 811, 826, 839, 854, 866, 877, 890, 897, 908, 912, 925, 940, 946, 953, 961, 972, 980, 986, 993, 1000, 1001, 1008, 1014, 1021, 1028, 1030, 1035, 1038, 1041, 1045, 1054, 1057, 1060, 1067, 1075, 1079, 1085, 1090, 1093, 1094, 1098, 1099, 1100, 1104, 1108, 1115, 1121, 1127, 1129, 1135, 1138, 1140, 1141, 1144, 1148, 1150, 1150, 1151, 1153, 1154, 1154], "harmed": [0, 0, 0, 2, 3, 4, 5, 6, 9, 10, 12, 16, 22, 26, 30, 30, 35, 39, 43, 48, 52, 59, 67, 73, 81, 87, 96, 99, 104, 110, 116, 123, 128, 131, 135, 143, 147, 154, 158, 162, 169, 173, 177, 185, 192, 196, 198, 203, 209, 210, 216, 222, 225, 231, 235, 237, 242, 248, 253, 256, 264, 266, 268, 269, 274, 276, 281, 283, 289, 290, 291, 293, 295, 297, 301, 305, 307, 311, 314, 317, 317, 319, 325, 331, 334, 335, 341, 342, 347, 348, 352, 360, 362, 366, 373, 377, 382, 388, 394, 396, 408], "exact_score": [79.81953594958465, 80.32082497851619, 80.7934689200802, 81.10856488112287, 81.53824119163563, 82.03953022056717, 82.41191635634489, 82.81294757949011, 83.09939845316528, 83.37152678315668, 83.54339730736177, 83.87281581208822, 84.1162990547121, 84.41707247207103, 84.61758808364365, 85.0472643941564, 85.30507018046404, 85.62016614150673, 85.8063592093956, 86.04984245201948, 86.30764823832712, 86.43655113148094, 86.56545402463478, 86.75164709252363, 86.86622744199369, 87.05242050988255, 87.2099684904039, 87.5107419077628, 87.69693497565167, 87.89745058722428, 87.94041821827557, 88.06932111142939, 88.1695789172157, 88.31280435405327, 88.42738470352333, 88.57061014036091, 88.64222285877972, 88.71383557719851, 88.79977083930105, 88.92867373245488, 89.01460899455743, 89.17215697507876, 89.30105986823258, 89.4013176740189, 89.47293039243769, 89.573188198224, 89.73073617874535, 89.75938126611285, 89.83099398453164, 89.87396161558291, 89.97421942136923, 90.10312231452305, 90.14608994557433, 90.16041248925809, 90.21770266399312, 90.34660555714694, 90.38957318819823, 90.38957318819823, 90.41821827556574, 90.47550845030077, 90.37525064451447, 90.44686336293324, 90.50415353766829, 90.59008879977084, 90.61873388713835, 90.61873388713835, 90.61873388713835, 90.63305643082211, 90.59008879977084, 90.63305643082211, 90.74763678029217, 90.76195932397593, 90.77628186765969, 90.84789458607848, 90.90518476081351, 90.90518476081351, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.9481523918648, 91.00544256659983, 90.99112002291606, 90.91950730449727, 90.89086221712975, 90.90518476081351, 90.99112002291606, 90.99112002291606, 91.06273274133486, 91.01976511028357, 91.09137782870236, 91.07705528501862, 90.99112002291606, 90.9767974792323, 90.96247493554854, 90.91950730449727, 90.89086221712975, 90.81924949871096, 90.74763678029217, 90.69034660555714, 90.67602406187338, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.56807261722473, "threshold_area100": 88.56731595531367, "threshold_area25": 84.07476367802921}, "test": {"score": [80.00287005811867, 80.71320944249122, 81.10784243380928, 81.53117600631413, 81.88275812585205, 82.27739111717013, 82.5787472196312, 82.8585778862022, 83.30343689459711, 83.62631843294827, 83.99225084307957, 84.27925665494726, 84.5375618856282, 84.73129080863887, 84.91066944105619, 85.14744923584703, 85.39857932123127, 85.67123484250556, 85.9080146372964, 86.13044414149387, 86.25242161153763, 86.5753031498888, 86.71880605582263, 86.92688526942672, 87.08473846595393, 87.27846738896463, 87.44349573078856, 87.60852407261247, 87.85247901270002, 87.98880677333716, 88.06055822630408, 88.14665996986439, 88.31886345698501, 88.412140345842, 88.62739470474278, 88.79959819186338, 88.93592595250054, 89.09377914902777, 89.158355456698, 89.2588074908517, 89.4238358326756, 89.531463012126, 89.63909019157639, 89.70366649924661, 89.74671737102676, 89.77541795221353, 89.92609600344407, 90.02654803759776, 90.06242376408122, 90.21310181531177, 90.31355384946545, 90.277678122982, 90.34942957594892, 90.41400588361914, 90.47140704599268, 90.51445791777284, 90.53598335366291, 90.55750878955298, 90.53598335366291, 90.52163306306953, 90.49293248188276, 90.45705675539931, 90.46423190069599, 90.50010762717945, 90.52880820836621, 90.53598335366291, 90.53598335366291, 90.56468393484968, 90.58620937073975, 90.59338451603645, 90.6364353878166, 90.67231111430006, 90.67231111430006, 90.70101169548683, 90.67948625959676, 90.68666140489346, 90.70818684078354, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.68666140489346, 90.70818684078354, 90.75123771256368, 90.72253713137691, 90.70818684078354, 90.64361053311329, 90.62208509722322, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.53598335366291, 90.50728277247615, 90.49293248188276, 90.46423190069599, 90.41400588361914, 90.43553131950922, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.19875152471838, 90.06242376408122], "calls": [0.0, 1.3704527516682212, 2.274521059051446, 3.2431656741049, 4.053957092631126, 5.05130228887135, 5.948195450957882, 6.845088613044414, 8.10073903996556, 9.062208509722321, 10.145655449522852, 10.970797158642462, 12.054244098442995, 13.015713568199756, 13.934132166176367, 14.730573294109206, 15.720743345052737, 16.976393771973882, 17.966563822917415, 19.03566047212456, 20.033005668364783, 21.32453182176939, 22.350577599196384, 23.23312047068953, 24.122838487479374, 25.141709119609672, 26.103178589366433, 27.10769893090335, 28.11939441773696, 29.109564468680492, 30.193011408481023, 30.910525938150247, 31.943746860873933, 32.804764296477, 33.84516036449738, 34.8496807060343, 35.73939872282414, 36.80132022673459, 37.68386309822774, 38.67403314917127, 39.865107268422186, 40.92702877233264, 41.816746789122476, 42.562961899978475, 43.38810360909808, 44.42132453182177, 45.64109923225945, 46.85369878740045, 47.650139915333284, 48.84838917988089, 49.73810719667073, 50.756977828801034, 51.96240223864533, 52.97409772547894, 54.17234699002654, 55.24144363923369, 56.31054028844083, 57.200258305230676, 58.26935495443782, 59.05144579177728, 59.969864389753894, 61.2613905431585, 62.22286001291526, 63.26325608093563, 64.26060127717587, 65.34404821697639, 66.43467030207361, 67.4678912247973, 68.55851330989452, 69.48410705316783, 70.66083088182535, 71.62947549687881, 72.64834612900911, 73.64569132524935, 74.80088971801679, 75.71930831599339, 76.82428069168401, 77.97230393915477, 78.6826433235273, 79.6154122120973, 80.56253139126068, 81.52400086101743, 82.55004663844443, 83.29626174930043, 84.37253354380427, 85.25507641529741, 86.06586783382365, 86.98428643180024, 88.03185764511731, 88.94310109779722, 89.98349716581761, 91.04541866972806, 92.05711415656167, 93.29123914759275, 94.28858434383297, 95.1854775059195, 96.09672095859942, 96.95773839420248, 98.01248475281625, 98.92372820549616, 100.0], "rescued": [0, 103, 162, 226, 279, 337, 387, 430, 496, 545, 603, 649, 690, 727, 760, 795, 838, 884, 923, 961, 992, 1047, 1081, 1118, 1147, 1182, 1210, 1241, 1279, 1307, 1334, 1357, 1395, 1417, 1457, 1489, 1515, 1544, 1563, 1585, 1623, 1645, 1667, 1684, 1699, 1712, 1742, 1768, 1781, 1807, 1826, 1839, 1859, 1876, 1897, 1915, 1926, 1940, 1946, 1952, 1958, 1968, 1980, 1989, 2000, 2007, 2015, 2025, 2035, 2044, 2055, 2071, 2081, 2092, 2102, 2109, 2117, 2127, 2131, 2138, 2142, 2149, 2161, 2164, 2170, 2171, 2176, 2183, 2187, 2191, 2194, 2200, 2204, 2212, 2219, 2226, 2231, 2236, 2242, 2245, 2247], "harmed": [0, 4, 8, 13, 17, 20, 28, 32, 36, 40, 47, 53, 58, 68, 76, 78, 86, 94, 100, 107, 121, 131, 145, 153, 160, 168, 173, 181, 185, 194, 211, 222, 236, 245, 255, 263, 270, 277, 287, 295, 310, 317, 324, 332, 341, 350, 359, 371, 379, 384, 389, 407, 417, 425, 438, 450, 458, 469, 478, 486, 496, 511, 522, 526, 533, 539, 547, 553, 560, 568, 573, 584, 594, 601, 614, 620, 625, 639, 641, 646, 653, 657, 663, 670, 678, 688, 696, 703, 712, 721, 726, 736, 742, 754, 768, 772, 783, 792, 806, 824, 845], "exact_score": [80.00287005811867, 80.56970653655738, 80.98586496376552, 81.40202339097367, 81.86123268996197, 82.24869053598336, 82.60027265552128, 82.91597904857574, 83.27473631341034, 83.6047929970582, 83.94919997129942, 84.28643180024396, 84.5232115950348, 84.72411566334218, 84.91784458635287, 85.18332496233049, 85.4488053383081, 85.67123484250556, 85.9080146372964, 86.11609385090048, 86.23807132094426, 86.48920140632848, 86.71163091052594, 86.8479586711631, 87.05603788476716, 87.25694195307454, 87.41479514960179, 87.56547320083232, 87.82377843151325, 87.98880677333716, 88.0533830810074, 88.15383511516109, 88.31168831168831, 88.47671665351223, 88.66327043122624, 88.7924230465667, 89.00767740546746, 89.09377914902777, 89.18705603788477, 89.344909234412, 89.44536126856569, 89.5386381574227, 89.66061562746646, 89.76106766162016, 89.75389251632346, 89.88304513166392, 89.94044629403746, 90.04089832819116, 90.09112434526799, 90.23462725120184, 90.26332783238861, 90.31355384946545, 90.34942957594892, 90.41400588361914, 90.46423190069599, 90.53598335366291, 90.5503336442563, 90.55750878955298, 90.54315849895961, 90.52163306306953, 90.49293248188276, 90.46423190069599, 90.48575733658608, 90.50010762717945, 90.53598335366291, 90.53598335366291, 90.54315849895961, 90.54315849895961, 90.58620937073975, 90.60773480662984, 90.61490995192652, 90.65078567840999, 90.66513596900337, 90.67231111430006, 90.68666140489346, 90.68666140489346, 90.67231111430006, 90.72253713137691, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.69383655019014, 90.71536198608023, 90.71536198608023, 90.69383655019014, 90.66513596900337, 90.62926024251992, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.53598335366291, 90.50728277247615, 90.49293248188276, 90.48575733658608, 90.42118102891584, 90.42118102891584, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.19875152471838, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.59345447074284, "threshold_area100": 88.59510655090766, "threshold_area25": 84.26203630623522}, "selected_budget": 89, "selected": {"val": {"score": 91.09137782870236, "calls": 88.98596390718991, "rescued": 1135, "harmed": 348}, "test": {"score": 90.5503336442563, "calls": 88.94310109779722, "rescued": 2191, "harmed": 721}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.653666079044342, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5734273195266724, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5053702592849731, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4460807740688324, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4003133773803711, "operator": "ge"}, {"budget": 0.06, "threshold": 0.3600628674030304, "operator": "ge"}, {"budget": 0.07, "threshold": 0.3216365575790405, "operator": "ge"}, {"budget": 0.08, "threshold": 0.28738468885421753, "operator": "ge"}, {"budget": 0.09, "threshold": 0.2611674666404724, "operator": "ge"}, {"budget": 0.1, "threshold": 0.24114814400672913, "operator": "ge"}, {"budget": 0.11, "threshold": 0.2153022736310959, "operator": "ge"}, {"budget": 0.12, "threshold": 0.1914307028055191, "operator": "ge"}, {"budget": 0.13, "threshold": 0.1698422133922577, "operator": "ge"}, {"budget": 0.14, "threshold": 0.1476001739501953, "operator": "ge"}, {"budget": 0.15, "threshold": 0.12549036741256714, "operator": "ge"}, {"budget": 0.16, "threshold": 0.10766968876123428, "operator": "ge"}, {"budget": 0.17, "threshold": 0.09152001142501831, "operator": "ge"}, {"budget": 0.18, "threshold": 0.0723060891032219, "operator": "ge"}, {"budget": 0.19, "threshold": 0.05816023051738739, "operator": "ge"}, {"budget": 0.2, "threshold": 0.04380321502685547, "operator": "ge"}, {"budget": 0.21, "threshold": 0.030765950679779053, "operator": "ge"}, {"budget": 0.22, "threshold": 0.020957492291927338, "operator": "ge"}, {"budget": 0.23, "threshold": 0.011039400473237038, "operator": "ge"}, {"budget": 0.24, "threshold": 0.002666078507900238, "operator": "ge"}, {"budget": 0.25, "threshold": -0.0030365977436304092, "operator": "ge"}, {"budget": 0.26, "threshold": -0.007861586287617683, "operator": "ge"}, {"budget": 0.27, "threshold": -0.01004628837108612, "operator": "ge"}, {"budget": 0.28, "threshold": -0.01165139488875866, "operator": "ge"}, {"budget": 0.29, "threshold": -0.01327027752995491, "operator": "ge"}, {"budget": 0.3, "threshold": -0.015010440722107887, "operator": "ge"}, {"budget": 0.31, "threshold": -0.01635495014488697, "operator": "ge"}, {"budget": 0.32, "threshold": -0.017677173018455505, "operator": "ge"}, {"budget": 0.33, "threshold": -0.01917165145277977, "operator": "ge"}, {"budget": 0.34, "threshold": -0.020582811906933784, "operator": "ge"}, {"budget": 0.35, "threshold": -0.0221808310598135, "operator": "ge"}, {"budget": 0.36, "threshold": -0.023603500798344612, "operator": "ge"}, {"budget": 0.37, "threshold": -0.02482941560447216, "operator": "ge"}, {"budget": 0.38, "threshold": -0.026092713698744774, "operator": "ge"}, {"budget": 0.39, "threshold": -0.027423083782196045, "operator": "ge"}, {"budget": 0.4, "threshold": -0.02889844961464405, "operator": "ge"}, {"budget": 0.41, "threshold": -0.030401041731238365, "operator": "ge"}, {"budget": 0.42, "threshold": -0.0318879596889019, "operator": "ge"}, {"budget": 0.43, "threshold": -0.03378017619252205, "operator": "ge"}, {"budget": 0.44, "threshold": -0.03508686274290085, "operator": "ge"}, {"budget": 0.45, "threshold": -0.03614652901887894, "operator": "ge"}, {"budget": 0.46, "threshold": -0.03739761933684349, "operator": "ge"}, {"budget": 0.47, "threshold": -0.03862467408180237, "operator": "ge"}, {"budget": 0.48, "threshold": -0.0397663451731205, "operator": "ge"}, {"budget": 0.49, "threshold": -0.04105134308338165, "operator": "ge"}, {"budget": 0.5, "threshold": -0.04248419031500816, "operator": "ge"}, {"budget": 0.51, "threshold": -0.04373247176408768, "operator": "ge"}, {"budget": 0.52, "threshold": -0.04526583477854729, "operator": "ge"}, {"budget": 0.53, "threshold": -0.04683913290500641, "operator": "ge"}, {"budget": 0.54, "threshold": -0.04840004816651344, "operator": "ge"}, {"budget": 0.55, "threshold": -0.05007832869887352, "operator": "ge"}, {"budget": 0.56, "threshold": -0.05184450373053551, "operator": "ge"}, {"budget": 0.57, "threshold": -0.05312887579202652, "operator": "ge"}, {"budget": 0.58, "threshold": -0.054915398359298706, "operator": "ge"}, {"budget": 0.59, "threshold": -0.05676915869116783, "operator": "ge"}, {"budget": 0.6, "threshold": -0.05840567871928215, "operator": "ge"}, {"budget": 0.61, "threshold": -0.05994183570146561, "operator": "ge"}, {"budget": 0.62, "threshold": -0.06181029602885246, "operator": "ge"}, {"budget": 0.63, "threshold": -0.0638088658452034, "operator": "ge"}, {"budget": 0.64, "threshold": -0.06533844023942947, "operator": "ge"}, {"budget": 0.65, "threshold": -0.0670749694108963, "operator": "ge"}, {"budget": 0.66, "threshold": -0.06876607984304428, "operator": "ge"}, {"budget": 0.67, "threshold": -0.07074587047100067, "operator": "ge"}, {"budget": 0.68, "threshold": -0.07297981530427933, "operator": "ge"}, {"budget": 0.69, "threshold": -0.07560395449399948, "operator": "ge"}, {"budget": 0.7, "threshold": -0.07791174948215485, "operator": "ge"}, {"budget": 0.71, "threshold": -0.08012533187866211, "operator": "ge"}, {"budget": 0.72, "threshold": -0.08277352899312973, "operator": "ge"}, {"budget": 0.73, "threshold": -0.08469771593809128, "operator": "ge"}, {"budget": 0.74, "threshold": -0.08749176561832428, "operator": "ge"}, {"budget": 0.75, "threshold": -0.09010442346334457, "operator": "ge"}, {"budget": 0.76, "threshold": -0.09254242479801178, "operator": "ge"}, {"budget": 0.77, "threshold": -0.09587102383375168, "operator": "ge"}, {"budget": 0.78, "threshold": -0.09863974153995514, "operator": "ge"}, {"budget": 0.79, "threshold": -0.10226594656705856, "operator": "ge"}, {"budget": 0.8, "threshold": -0.1058390885591507, "operator": "ge"}, {"budget": 0.81, "threshold": -0.1094055324792862, "operator": "ge"}, {"budget": 0.82, "threshold": -0.113491490483284, "operator": "ge"}, {"budget": 0.83, "threshold": -0.11687052994966507, "operator": "ge"}, {"budget": 0.84, "threshold": -0.1207970604300499, "operator": "ge"}, {"budget": 0.85, "threshold": -0.1256982833147049, "operator": "ge"}, {"budget": 0.86, "threshold": -0.12931115925312042, "operator": "ge"}, {"budget": 0.87, "threshold": -0.13549725711345673, "operator": "ge"}, {"budget": 0.88, "threshold": -0.1428607702255249, "operator": "ge"}, {"budget": 0.89, "threshold": -0.14895233511924744, "operator": "ge"}, {"budget": 0.9, "threshold": -0.15680158138275146, "operator": "ge"}, {"budget": 0.91, "threshold": -0.16576792299747467, "operator": "ge"}, {"budget": 0.92, "threshold": -0.17377682030200958, "operator": "ge"}, {"budget": 0.93, "threshold": -0.1846802681684494, "operator": "ge"}, {"budget": 0.94, "threshold": -0.19542744755744934, "operator": "ge"}, {"budget": 0.95, "threshold": -0.21047207713127136, "operator": "ge"}, {"budget": 0.96, "threshold": -0.23020051419734955, "operator": "ge"}, {"budget": 0.97, "threshold": -0.2550819516181946, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2835085391998291, "operator": "ge"}, {"budget": 0.99, "threshold": -0.326565146446228, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.36379260956745, 80.77914637639644, 81.20882268690919, 81.59553136637066, 81.93927241478086, 82.3403036379261, 82.65539959896877, 82.91320538527643, 83.18533371526783, 83.5720423947293, 83.82984818103695, 84.05900887997709, 84.4313950157548, 84.71784588942995, 85.06158693784016, 85.3910054425666, 85.69177885992552, 85.89229447149813, 86.1071326267545, 86.29332569464336, 86.47951876253221, 86.57977656831852, 86.75164709252363, 86.86622744199369, 86.98080779146376, 87.05242050988255, 87.19564594672013, 87.3102262961902, 87.3102262961902, 87.3102262961902, 87.33887138355772, 87.36751647092524, 87.43912918934402, 87.46777427671154, 87.5823546261816, 87.62532225723288, 87.72558006301918, 87.7542251503867, 87.81151532512173, 87.854482956173, 87.854482956173, 87.92609567459179, 87.96906330564308, 88.01203093669436, 88.06932111142939, 88.11228874248066, 88.11228874248066, 88.19822400458321, 88.25551417931824, 88.35577198510455, 88.45602979089085, 88.45602979089085, 88.5419650529934, 88.62790031509596, 88.75680320824978, 88.77112575193354, 88.71383557719851, 88.74248066456603, 88.8857061014036, 89.04325408192494, 89.10054425665997, 89.11486680034373, 89.20080206244629, 89.22944714981381, 89.35835004296763, 89.45860784875393, 89.44428530507018, 89.5445431108565, 89.573188198224, 89.60183328559151, 89.63047837295903, 89.70209109137782, 89.71641363506157, 89.74505872242909, 89.80234889716414, 89.88828415926669, 89.98854196505299, 90.0171870524205, 90.10312231452305, 90.24634775136063, 90.27499283872815, 90.37525064451447, 90.44686336293324, 90.4325408192495, 90.47550845030077, 90.53279862503581, 90.53279862503581, 90.57576625608708, 90.6044113434546, 90.56144371240332, 90.57576625608708, 90.64737897450587, 90.66170151818963, 90.74763678029217, 90.79060441134345, 90.86221712976223, 90.80492695502721, 90.69034660555714, 90.74763678029217, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 38, 67, 98, 127, 152, 180, 204, 225, 247, 274, 296, 316, 348, 372, 400, 428, 451, 471, 492, 511, 528, 544, 562, 574, 584, 592, 604, 613, 616, 617, 621, 627, 633, 636, 645, 648, 655, 660, 666, 669, 672, 678, 682, 687, 693, 698, 700, 708, 714, 721, 729, 733, 739, 746, 756, 759, 761, 766, 779, 793, 798, 803, 812, 819, 828, 837, 840, 848, 853, 858, 866, 876, 882, 886, 894, 902, 911, 919, 929, 944, 953, 966, 976, 985, 996, 1005, 1012, 1023, 1031, 1037, 1044, 1057, 1068, 1081, 1092, 1107, 1118, 1127, 1144, 1154], "harmed": [0, 0, 0, 1, 3, 4, 4, 6, 9, 12, 12, 16, 20, 26, 30, 34, 39, 41, 47, 53, 59, 63, 72, 78, 82, 84, 87, 89, 90, 93, 94, 96, 100, 101, 102, 103, 103, 103, 106, 108, 108, 111, 112, 113, 115, 117, 119, 121, 123, 125, 125, 126, 130, 130, 131, 132, 134, 140, 143, 146, 149, 150, 154, 157, 162, 162, 164, 168, 169, 172, 175, 181, 186, 191, 193, 197, 199, 201, 207, 211, 216, 223, 229, 234, 244, 252, 257, 264, 272, 278, 287, 293, 301, 311, 318, 326, 336, 351, 368, 381, 408], "exact_score": [79.81953594958465, 80.36379260956745, 80.77914637639644, 81.20882268690919, 81.59553136637066, 81.93927241478086, 82.3403036379261, 82.65539959896877, 82.91320538527643, 83.18533371526783, 83.5720423947293, 83.82984818103695, 84.05900887997709, 84.4313950157548, 84.71784588942995, 85.06158693784016, 85.3910054425666, 85.69177885992552, 85.89229447149813, 86.1071326267545, 86.29332569464336, 86.47951876253221, 86.57977656831852, 86.75164709252363, 86.86622744199369, 86.98080779146376, 87.05242050988255, 87.19564594672013, 87.3102262961902, 87.3102262961902, 87.3102262961902, 87.33887138355772, 87.36751647092524, 87.43912918934402, 87.46777427671154, 87.5823546261816, 87.62532225723288, 87.72558006301918, 87.7542251503867, 87.81151532512173, 87.854482956173, 87.854482956173, 87.92609567459179, 87.96906330564308, 88.01203093669436, 88.06932111142939, 88.11228874248066, 88.11228874248066, 88.19822400458321, 88.25551417931824, 88.35577198510455, 88.45602979089085, 88.45602979089085, 88.5419650529934, 88.62790031509596, 88.75680320824978, 88.77112575193354, 88.71383557719851, 88.74248066456603, 88.8857061014036, 89.04325408192494, 89.10054425665997, 89.11486680034373, 89.20080206244629, 89.22944714981381, 89.35835004296763, 89.45860784875393, 89.44428530507018, 89.5445431108565, 89.573188198224, 89.60183328559151, 89.63047837295903, 89.70209109137782, 89.71641363506157, 89.74505872242909, 89.80234889716414, 89.88828415926669, 89.98854196505299, 90.0171870524205, 90.10312231452305, 90.24634775136063, 90.27499283872815, 90.37525064451447, 90.44686336293324, 90.4325408192495, 90.47550845030077, 90.53279862503581, 90.53279862503581, 90.57576625608708, 90.6044113434546, 90.56144371240332, 90.57576625608708, 90.64737897450587, 90.66170151818963, 90.74763678029217, 90.79060441134345, 90.86221712976223, 90.80492695502721, 90.69034660555714, 90.74763678029217, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.84733399244658, "threshold_area100": 87.84660555714694, "threshold_area25": 84.05327986250359}, "test": {"score": [80.00287005811867, 80.77061060486476, 81.25134533974312, 81.60292745928105, 81.95450957881899, 82.32761713424696, 82.72942527086174, 83.14558369806988, 83.59044270646481, 83.7841716294755, 84.00660113367296, 84.25055607376049, 84.44428499677119, 84.7025902274521, 84.93937002224295, 85.14027409055034, 85.34835330415441, 85.59948338953865, 85.80756260314271, 85.98694123556002, 86.32417306450456, 86.63987945755902, 86.72598120111932, 86.95558585061349, 87.09191361125063, 87.24259166248117, 87.32869340604148, 87.37174427782163, 87.42197029489847, 87.50807203845878, 87.55829805553563, 87.60134892731578, 87.65157494439264, 87.70180096146947, 87.72332639735954, 87.74485183324963, 87.76637726913971, 87.82377843151325, 87.83812872210662, 87.88835473918347, 87.93140561096362, 88.01750735452394, 88.08925880749085, 88.18971084164454, 88.29016287579825, 88.31168831168831, 88.34756403817178, 88.40496520054532, 88.45519121762216, 88.505417234699, 88.56281839707253, 88.64174499533615, 88.74219702948984, 88.80677333716008, 88.84264906364353, 88.89287508072039, 88.92875080720385, 88.94310109779724, 89.01485255076416, 89.04355313195093, 89.05072827724761, 89.1440051661046, 89.20140632847816, 89.3377340891153, 89.35925952500538, 89.39513525148884, 89.49558728564254, 89.50276243093923, 89.6175647556863, 89.71084164454331, 89.77541795221353, 89.78259309751023, 89.85434455047715, 89.88304513166392, 89.94762143933414, 90.06242376408122, 90.07677405467462, 90.13417521704815, 90.18440123412499, 90.22027696060846, 90.22745210590514, 90.28485326827868, 90.31355384946545, 90.32790414005885, 90.37095501183899, 90.42118102891584, 90.41400588361914, 90.40683073832245, 90.48575733658608, 90.57185908014638, 90.58620937073975, 90.6364353878166, 90.62208509722322, 90.60773480662984, 90.6364353878166, 90.59338451603645, 90.59338451603645, 90.55750878955298, 90.54315849895961, 90.43553131950922, 90.06242376408122], "calls": [0.0, 1.277175862811222, 2.231470187271292, 3.1785893664346707, 4.183109707971586, 5.123053741838272, 6.07017292100165, 7.124919279615412, 8.29446796297625, 9.184185979766092, 9.858649637655162, 10.956446868049078, 11.94661691899261, 12.893736098155989, 13.833680132022675, 14.902776781229822, 15.85707110568989, 16.768314558369806, 17.887637224653798, 18.856281839707254, 20.076056540144936, 21.231254932912393, 22.07792207792208, 23.13984358183253, 24.180239649852908, 25.026906794862597, 26.39735954653082, 27.473631341034654, 28.183970725407185, 29.181315921647418, 30.257587716151253, 31.176006314127864, 32.00832316854416, 33.09894525364139, 34.117815885771684, 35.208437970868914, 36.284709765372746, 37.18877807275597, 38.06414579895243, 38.97538925163235, 40.266915405036954, 41.44363923369448, 42.62036306235201, 44.19889502762431, 45.016861591447224, 45.77742699289661, 46.83934849680706, 47.81516825715721, 48.75511229102389, 49.62330487192366, 50.599124632273806, 51.41709119609672, 52.32115950347994, 53.31850469972017, 54.208222716510015, 55.270144220420455, 56.21008825428715, 56.898902202769605, 57.9321231254933, 59.123197244744205, 60.25687020162158, 61.03178589366435, 62.194159431728494, 63.32783238860586, 64.12427351653871, 64.98529095214178, 65.76738178948123, 66.64274951567769, 67.71184616488483, 68.83834397646552, 69.76393771973882, 70.83303436894597, 71.9236564540432, 72.6698715648992, 73.6815670517328, 74.68608739326972, 75.47535337590587, 76.6520772045634, 77.40546746071608, 78.38846236636292, 79.32123125493291, 80.29705101528306, 81.28722106622659, 82.04778646767596, 83.03078137332281, 84.27208150965056, 84.87479371457272, 86.16631986797732, 87.55112291023893, 88.56999354236923, 89.6175647556863, 90.68666140489346, 91.50462796871636, 92.56654947262682, 93.46344263471336, 94.71191791633781, 95.95321805266556, 97.07971586424625, 97.91203271866256, 98.83045131663917, 100.0], "rescued": [0, 110, 181, 239, 294, 349, 405, 469, 537, 569, 605, 647, 685, 732, 770, 807, 842, 884, 923, 958, 1012, 1065, 1084, 1127, 1150, 1174, 1190, 1197, 1204, 1217, 1226, 1234, 1242, 1250, 1258, 1265, 1270, 1280, 1284, 1295, 1304, 1317, 1330, 1349, 1363, 1370, 1381, 1391, 1400, 1409, 1420, 1432, 1448, 1463, 1472, 1484, 1495, 1502, 1515, 1525, 1533, 1550, 1562, 1588, 1595, 1607, 1622, 1630, 1651, 1671, 1685, 1695, 1712, 1723, 1741, 1765, 1776, 1798, 1813, 1827, 1837, 1855, 1871, 1885, 1900, 1928, 1937, 1953, 1976, 2002, 2022, 2051, 2065, 2095, 2112, 2129, 2154, 2179, 2204, 2226, 2247], "harmed": [0, 3, 7, 16, 22, 25, 25, 31, 37, 42, 47, 55, 66, 77, 82, 91, 97, 104, 114, 124, 131, 140, 147, 158, 162, 165, 169, 170, 170, 171, 173, 175, 176, 177, 182, 186, 188, 190, 192, 196, 199, 200, 203, 208, 208, 212, 218, 220, 222, 224, 227, 228, 230, 236, 240, 245, 251, 256, 259, 265, 272, 276, 280, 287, 291, 298, 299, 306, 311, 318, 323, 332, 339, 346, 355, 363, 372, 386, 394, 403, 412, 422, 434, 446, 455, 476, 486, 503, 515, 529, 547, 569, 585, 617, 630, 653, 678, 708, 735, 772, 845], "exact_score": [80.00287005811867, 80.6342828442276, 81.12219272440268, 81.53117600631413, 81.87558298055536, 82.23434024538997, 82.72225012556504, 83.05948195450958, 83.44693980053096, 83.75547104828873, 84.02095142426634, 84.26490636435389, 84.46581043266126, 84.71694051804549, 85.01829662050658, 85.1689746717371, 85.3698787400445, 85.6353591160221, 85.83626318432948, 86.0156418167468, 86.30982277391116, 86.5609528592954, 86.70445576522926, 86.94123556002009, 87.07038817536056, 87.23541651718448, 87.3143431154481, 87.34304369663487, 87.42197029489847, 87.47937145727201, 87.52242232905216, 87.5798234914257, 87.65157494439264, 87.70180096146947, 87.73050154265624, 87.75202697854631, 87.75202697854631, 87.80225299562316, 87.83812872210662, 87.88835473918347, 87.92423046566694, 87.97445648274378, 88.06773337160078, 88.10360909808423, 88.17536055105116, 88.28298773050155, 88.33321374757838, 88.35473918346847, 88.40496520054532, 88.46954150821554, 88.51259237999571, 88.62739470474278, 88.72784673889646, 88.79959819186338, 88.83547391834685, 88.89287508072039, 88.92875080720385, 88.93592595250054, 89.01485255076416, 89.03637798665423, 89.065078567841, 89.13683002080792, 89.21575661907154, 89.30903350792853, 89.35925952500538, 89.39513525148884, 89.48841214034584, 89.56016359331277, 89.66061562746646, 89.71801678984, 89.76824280691685, 89.78259309751023, 89.85434455047715, 89.8902202769606, 89.98349716581761, 90.04807347348785, 90.09112434526799, 90.13417521704815, 90.21310181531177, 90.22745210590514, 90.24897754179523, 90.30637870416876, 90.32790414005885, 90.34942957594892, 90.44270646480591, 90.41400588361914, 90.38530530243239, 90.46423190069599, 90.50728277247615, 90.57185908014638, 90.62926024251992, 90.62926024251992, 90.60773480662984, 90.60773480662984, 90.64361053311329, 90.55750878955298, 90.58620937073975, 90.57903422544307, 90.52880820836621, 90.39248044772907, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.9764607046452, "threshold_area100": 87.98658247829519, "threshold_area25": 84.28427925665495}, "selected_budget": 96, "selected": {"val": {"score": 90.86221712976223, "calls": 95.9896877685477, "rescued": 1107, "harmed": 336}, "test": {"score": 90.59338451603645, "calls": 95.95321805266556, "rescued": 2154, "harmed": 678}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.765038013458252, "operator": "ge"}, {"budget": 0.02, "threshold": 3.31158447265625, "operator": "ge"}, {"budget": 0.03, "threshold": 2.9754066467285156, "operator": "ge"}, {"budget": 0.04, "threshold": 2.686948776245117, "operator": "ge"}, {"budget": 0.05, "threshold": 2.435880184173584, "operator": "ge"}, {"budget": 0.06, "threshold": 2.19919490814209, "operator": "ge"}, {"budget": 0.07, "threshold": 2.016608953475952, "operator": "ge"}, {"budget": 0.08, "threshold": 1.828273892402649, "operator": "ge"}, {"budget": 0.09, "threshold": 1.667983055114746, "operator": "ge"}, {"budget": 0.1, "threshold": 1.4844512939453125, "operator": "ge"}, {"budget": 0.11, "threshold": 1.3664612770080566, "operator": "ge"}, {"budget": 0.12, "threshold": 1.2520277500152588, "operator": "ge"}, {"budget": 0.13, "threshold": 1.1531522274017334, "operator": "ge"}, {"budget": 0.14, "threshold": 1.030269742012024, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9206018447875977, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8181940913200378, "operator": "ge"}, {"budget": 0.17, "threshold": 0.7357290983200073, "operator": "ge"}, {"budget": 0.18, "threshold": 0.6112534999847412, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5168734788894653, "operator": "ge"}, {"budget": 0.2, "threshold": 0.3984813690185547, "operator": "ge"}, {"budget": 0.21, "threshold": 0.3103705644607544, "operator": "ge"}, {"budget": 0.22, "threshold": 0.20450454950332642, "operator": "ge"}, {"budget": 0.23, "threshold": 0.11503258347511292, "operator": "ge"}, {"budget": 0.24, "threshold": 0.04106573760509491, "operator": "ge"}, {"budget": 0.25, "threshold": -0.057812824845314026, "operator": "ge"}, {"budget": 0.26, "threshold": -0.13853982090950012, "operator": "ge"}, {"budget": 0.27, "threshold": -0.23606911301612854, "operator": "ge"}, {"budget": 0.28, "threshold": -0.33139747381210327, "operator": "ge"}, {"budget": 0.29, "threshold": -0.41194939613342285, "operator": "ge"}, {"budget": 0.3, "threshold": -0.5080811977386475, "operator": "ge"}, {"budget": 0.31, "threshold": -0.5784786939620972, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6674528121948242, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7419869899749756, "operator": "ge"}, {"budget": 0.34, "threshold": -0.8343074321746826, "operator": "ge"}, {"budget": 0.35, "threshold": -0.9176257252693176, "operator": "ge"}, {"budget": 0.36, "threshold": -0.9967954158782959, "operator": "ge"}, {"budget": 0.37, "threshold": -1.1015958786010742, "operator": "ge"}, {"budget": 0.38, "threshold": -1.1908011436462402, "operator": "ge"}, {"budget": 0.39, "threshold": -1.2809752225875854, "operator": "ge"}, {"budget": 0.4, "threshold": -1.3692402839660645, "operator": "ge"}, {"budget": 0.41, "threshold": -1.4540313482284546, "operator": "ge"}, {"budget": 0.42, "threshold": -1.5443590879440308, "operator": "ge"}, {"budget": 0.43, "threshold": -1.624683141708374, "operator": "ge"}, {"budget": 0.44, "threshold": -1.706303596496582, "operator": "ge"}, {"budget": 0.45, "threshold": -1.8091822862625122, "operator": "ge"}, {"budget": 0.46, "threshold": -1.8969539403915405, "operator": "ge"}, {"budget": 0.47, "threshold": -1.9881796836853027, "operator": "ge"}, {"budget": 0.48, "threshold": -2.0732734203338623, "operator": "ge"}, {"budget": 0.49, "threshold": -2.1416983604431152, "operator": "ge"}, {"budget": 0.5, "threshold": -2.2368898391723633, "operator": "ge"}, {"budget": 0.51, "threshold": -2.306936502456665, "operator": "ge"}, {"budget": 0.52, "threshold": -2.411635160446167, "operator": "ge"}, {"budget": 0.53, "threshold": -2.5081424713134766, "operator": "ge"}, {"budget": 0.54, "threshold": -2.589890956878662, "operator": "ge"}, {"budget": 0.55, "threshold": -2.672551155090332, "operator": "ge"}, {"budget": 0.56, "threshold": -2.774576187133789, "operator": "ge"}, {"budget": 0.57, "threshold": -2.858059883117676, "operator": "ge"}, {"budget": 0.58, "threshold": -2.934464931488037, "operator": "ge"}, {"budget": 0.59, "threshold": -3.028687000274658, "operator": "ge"}, {"budget": 0.6, "threshold": -3.1176702976226807, "operator": "ge"}, {"budget": 0.61, "threshold": -3.197835922241211, "operator": "ge"}, {"budget": 0.62, "threshold": -3.270911693572998, "operator": "ge"}, {"budget": 0.63, "threshold": -3.3419229984283447, "operator": "ge"}, {"budget": 0.64, "threshold": -3.438180446624756, "operator": "ge"}, {"budget": 0.65, "threshold": -3.5156006813049316, "operator": "ge"}, {"budget": 0.66, "threshold": -3.6084091663360596, "operator": "ge"}, {"budget": 0.67, "threshold": -3.6909337043762207, "operator": "ge"}, {"budget": 0.68, "threshold": -3.790832042694092, "operator": "ge"}, {"budget": 0.69, "threshold": -3.857120990753174, "operator": "ge"}, {"budget": 0.7, "threshold": -3.936276435852051, "operator": "ge"}, {"budget": 0.71, "threshold": -4.014188289642334, "operator": "ge"}, {"budget": 0.72, "threshold": -4.109676361083984, "operator": "ge"}, {"budget": 0.73, "threshold": -4.194304943084717, "operator": "ge"}, {"budget": 0.74, "threshold": -4.284149646759033, "operator": "ge"}, {"budget": 0.75, "threshold": -4.366723537445068, "operator": "ge"}, {"budget": 0.76, "threshold": -4.428588390350342, "operator": "ge"}, {"budget": 0.77, "threshold": -4.528055667877197, "operator": "ge"}, {"budget": 0.78, "threshold": -4.576888084411621, "operator": "ge"}, {"budget": 0.79, "threshold": -4.665833473205566, "operator": "ge"}, {"budget": 0.8, "threshold": -4.748727321624756, "operator": "ge"}, {"budget": 0.81, "threshold": -4.827517509460449, "operator": "ge"}, {"budget": 0.82, "threshold": -4.9027814865112305, "operator": "ge"}, {"budget": 0.83, "threshold": -4.989965438842773, "operator": "ge"}, {"budget": 0.84, "threshold": -5.068756103515625, "operator": "ge"}, {"budget": 0.85, "threshold": -5.141115665435791, "operator": "ge"}, {"budget": 0.86, "threshold": -5.215929985046387, "operator": "ge"}, {"budget": 0.87, "threshold": -5.3249335289001465, "operator": "ge"}, {"budget": 0.88, "threshold": -5.4122700691223145, "operator": "ge"}, {"budget": 0.89, "threshold": -5.5232343673706055, "operator": "ge"}, {"budget": 0.9, "threshold": -5.6334099769592285, "operator": "ge"}, {"budget": 0.91, "threshold": -5.737174034118652, "operator": "ge"}, {"budget": 0.92, "threshold": -5.896483421325684, "operator": "ge"}, {"budget": 0.93, "threshold": -6.077735900878906, "operator": "ge"}, {"budget": 0.94, "threshold": -6.223499298095703, "operator": "ge"}, {"budget": 0.95, "threshold": -6.374080657958984, "operator": "ge"}, {"budget": 0.96, "threshold": -6.5355610847473145, "operator": "ge"}, {"budget": 0.97, "threshold": -6.767317295074463, "operator": "ge"}, {"budget": 0.98, "threshold": -7.097081184387207, "operator": "ge"}, {"budget": 0.99, "threshold": -7.46957540512085, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.26353480378114, 80.63592093955886, 81.15153251217416, 81.53824119163563, 81.81036952162704, 82.21140074477226, 82.56946433686623, 83.02778573474649, 83.4001718705242, 83.64365511314809, 83.85849326840446, 84.10197651102835, 84.35978229733601, 84.61758808364365, 84.88971641363506, 85.23345746204527, 85.57719851045546, 85.83500429676309, 86.078487539387, 86.32197078201088, 86.49384130621598, 86.6227441993698, 86.75164709252363, 86.88054998567745, 87.08106559725006, 87.22429103408766, 87.49641936407906, 87.71125751933543, 87.89745058722428, 87.96906330564308, 88.05499856774563, 88.2411916356345, 88.34144944142079, 88.4417072472071, 88.55628759667717, 88.72815812088226, 88.74248066456603, 88.8857061014036, 88.91435118877112, 89.04325408192494, 89.15783443139502, 89.21512460613005, 89.31538241191636, 89.44428530507018, 89.55886565454024, 89.63047837295903, 89.64480091664278, 89.77370380979661, 89.93125179031796, 89.95989687768548, 90.0171870524205, 90.07447722715554, 90.11744485820681, 90.21770266399312, 90.34660555714694, 90.37525064451447, 90.4325408192495, 90.46118590661702, 90.41821827556574, 90.41821827556574, 90.53279862503581, 90.53279862503581, 90.6044113434546, 90.66170151818963, 90.63305643082211, 90.63305643082211, 90.63305643082211, 90.67602406187338, 90.67602406187338, 90.63305643082211, 90.69034660555714, 90.79060441134345, 90.80492695502721, 90.86221712976223, 90.84789458607848, 90.90518476081351, 90.99112002291606, 90.96247493554854, 90.9767974792323, 90.9481523918648, 90.90518476081351, 90.91950730449727, 90.91950730449727, 90.91950730449727, 90.9481523918648, 90.9767974792323, 91.00544256659983, 91.0484101976511, 91.09137782870236, 91.06273274133486, 90.99112002291606, 90.96247493554854, 90.876539673446, 90.86221712976223, 90.86221712976223, 90.81924949871096, 90.71899169292466, 90.67602406187338, 90.67602406187338, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 59, 95, 123, 142, 171, 197, 229, 255, 276, 294, 315, 337, 360, 384, 410, 438, 460, 481, 503, 521, 539, 556, 574, 594, 614, 636, 657, 675, 688, 701, 717, 728, 741, 753, 768, 775, 790, 798, 813, 826, 839, 849, 864, 876, 885, 895, 904, 917, 926, 934, 943, 953, 962, 975, 981, 987, 994, 997, 1002, 1011, 1015, 1023, 1030, 1034, 1035, 1039, 1044, 1045, 1047, 1052, 1063, 1065, 1071, 1074, 1082, 1089, 1091, 1094, 1094, 1095, 1100, 1105, 1109, 1115, 1119, 1123, 1130, 1134, 1135, 1138, 1139, 1140, 1144, 1148, 1149, 1151, 1151, 1154, 1154], "harmed": [0, 0, 2, 2, 3, 3, 4, 5, 5, 5, 9, 12, 16, 20, 25, 30, 32, 36, 40, 44, 49, 55, 64, 72, 81, 87, 97, 100, 106, 111, 119, 126, 129, 133, 139, 143, 146, 152, 157, 163, 169, 174, 183, 186, 192, 196, 200, 209, 209, 211, 218, 222, 227, 234, 236, 240, 244, 246, 251, 257, 262, 263, 267, 270, 273, 279, 280, 284, 286, 287, 292, 293, 297, 298, 300, 304, 308, 309, 313, 315, 317, 321, 325, 330, 334, 338, 340, 342, 346, 347, 350, 358, 361, 368, 373, 377, 381, 390, 393, 396, 408], "exact_score": [79.81953594958465, 80.26353480378114, 80.63592093955886, 81.15153251217416, 81.53824119163563, 81.81036952162704, 82.21140074477226, 82.56946433686623, 83.02778573474649, 83.4001718705242, 83.64365511314809, 83.85849326840446, 84.10197651102835, 84.35978229733601, 84.61758808364365, 84.88971641363506, 85.23345746204527, 85.57719851045546, 85.83500429676309, 86.078487539387, 86.32197078201088, 86.49384130621598, 86.6227441993698, 86.75164709252363, 86.88054998567745, 87.08106559725006, 87.22429103408766, 87.49641936407906, 87.71125751933543, 87.89745058722428, 87.96906330564308, 88.05499856774563, 88.2411916356345, 88.34144944142079, 88.4417072472071, 88.55628759667717, 88.72815812088226, 88.74248066456603, 88.8857061014036, 88.91435118877112, 89.04325408192494, 89.15783443139502, 89.21512460613005, 89.31538241191636, 89.44428530507018, 89.55886565454024, 89.63047837295903, 89.64480091664278, 89.77370380979661, 89.93125179031796, 89.95989687768548, 90.0171870524205, 90.07447722715554, 90.11744485820681, 90.21770266399312, 90.34660555714694, 90.37525064451447, 90.4325408192495, 90.46118590661702, 90.41821827556574, 90.41821827556574, 90.53279862503581, 90.53279862503581, 90.6044113434546, 90.66170151818963, 90.63305643082211, 90.63305643082211, 90.63305643082211, 90.67602406187338, 90.67602406187338, 90.63305643082211, 90.69034660555714, 90.79060441134345, 90.80492695502721, 90.86221712976223, 90.84789458607848, 90.90518476081351, 90.99112002291606, 90.96247493554854, 90.9767974792323, 90.9481523918648, 90.90518476081351, 90.91950730449727, 90.91950730449727, 90.91950730449727, 90.9481523918648, 90.9767974792323, 91.00544256659983, 91.0484101976511, 91.09137782870236, 91.06273274133486, 90.99112002291606, 90.96247493554854, 90.876539673446, 90.86221712976223, 90.86221712976223, 90.81924949871096, 90.71899169292466, 90.67602406187338, 90.67602406187338, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.55188436837335, "threshold_area100": 88.55113148095101, "threshold_area25": 84.0232025207677}, "test": {"score": [80.00287005811867, 80.74191002367799, 81.16524359618282, 81.61727774987443, 81.94733443352227, 82.27739111717013, 82.71507498026835, 83.0523068092129, 83.38236349286073, 83.66219415943173, 84.0568271507498, 84.26490636435389, 84.49451101384804, 84.7097653727488, 84.99677118461649, 85.1689746717371, 85.37705388534118, 85.54208222716511, 85.87931405610964, 86.08021812441702, 86.27394704742771, 86.44615053454832, 86.77620721819616, 86.90535983353664, 87.08473846595393, 87.29999282485471, 87.42914544019517, 87.61569921790917, 87.85247901270002, 87.9529310468537, 88.03903279041401, 88.09643395278755, 88.28298773050155, 88.41931549113869, 88.59869412355602, 88.76372246537993, 88.878524790127, 89.065078567841, 89.15118031140132, 89.37360981559877, 89.438186123269, 89.56016359331277, 89.61038961038962, 89.66779077276315, 89.69649135394992, 89.77541795221353, 89.95479658463084, 90.00502260170768, 90.12700007175145, 90.1772260888283, 90.24897754179523, 90.29920355887207, 90.34942957594892, 90.40683073832245, 90.44270646480591, 90.47140704599268, 90.49293248188276, 90.51445791777284, 90.48575733658608, 90.44270646480591, 90.44988161010261, 90.45705675539931, 90.44988161010261, 90.45705675539931, 90.47140704599268, 90.52880820836621, 90.55750878955298, 90.58620937073975, 90.65078567840999, 90.65796082370669, 90.65796082370669, 90.64361053311329, 90.64361053311329, 90.66513596900337, 90.66513596900337, 90.64361053311329, 90.65078567840999, 90.62926024251992, 90.66513596900337, 90.67948625959676, 90.69383655019014, 90.75123771256368, 90.75841285786038, 90.72253713137691, 90.68666140489346, 90.65078567840999, 90.67231111430006, 90.61490995192652, 90.60055966133315, 90.57185908014638, 90.53598335366291, 90.52163306306953, 90.52163306306953, 90.47858219128938, 90.44270646480591, 90.42835617421254, 90.38530530243239, 90.34942957594892, 90.29920355887207, 90.18440123412499, 90.06242376408122], "calls": [0.0, 1.3058764439979909, 2.2242950419746, 3.3077419817751306, 4.1974599985649705, 5.130228887134964, 6.271076989309034, 7.311473057329411, 8.330343689459712, 9.284638013919782, 10.547463586137619, 11.415656167037383, 12.312549329123915, 13.080289875869985, 14.199612542153977, 15.182607447800816, 15.98622372103035, 16.74678912247973, 17.937863241730646, 18.985434455047713, 20.219559446078783, 21.06622659108847, 22.386453325679845, 23.247470761282916, 24.122838487479374, 25.2062854272799, 26.074478008179664, 27.08617349501327, 28.162445289517112, 29.052163306306955, 30.17866111788764, 30.917701083446943, 31.75001793786324, 32.64691109994977, 33.77340891153045, 34.64877663772691, 35.4021668938796, 36.62911673961398, 37.51165961110712, 38.58075626031427, 39.44894884121403, 40.54674607160795, 41.55844155844156, 42.37640812226447, 43.30917701083447, 44.60070316423908, 45.75590155700653, 46.82499820621368, 47.86539427423405, 48.62595967568343, 49.817033794934346, 50.68522637583411, 51.92652651216187, 53.13912606730287, 54.036019229389396, 55.04053957092631, 56.19573796369376, 57.17873286934061, 58.04692545024037, 59.266700150678055, 60.371672526368656, 61.34031714142212, 62.23003515821195, 63.10540288440841, 64.27495156776925, 65.236421037526, 66.36291884910669, 67.44636578890723, 68.73789194231183, 69.54150821554137, 70.45992681351797, 71.47879744564828, 72.68422185549257, 73.62416588935926, 74.9730932051374, 75.89151180311401, 76.81710554638731, 78.07275597330846, 78.51761498170337, 79.73738968214107, 80.76343545956806, 81.5670517327976, 82.42806916840067, 83.46846523642104, 84.47298557795794, 85.2694267058908, 86.0084666714501, 87.14931477362417, 88.00315706393054, 88.99332711487408, 89.98349716581761, 90.9090909090909, 92.05711415656167, 93.26253856640598, 94.29575948912967, 95.23570352299635, 96.02496950563248, 96.97926383009255, 98.0483604792997, 99.02418023964985, 100.0], "rescued": [0, 105, 170, 240, 290, 339, 403, 453, 503, 547, 611, 643, 678, 717, 765, 799, 834, 862, 915, 953, 990, 1027, 1086, 1115, 1146, 1185, 1208, 1242, 1283, 1306, 1336, 1360, 1392, 1421, 1458, 1485, 1509, 1546, 1564, 1605, 1621, 1645, 1664, 1680, 1692, 1721, 1751, 1767, 1791, 1804, 1829, 1844, 1863, 1884, 1895, 1911, 1924, 1936, 1942, 1950, 1958, 1968, 1973, 1985, 1996, 2009, 2018, 2028, 2042, 2049, 2059, 2067, 2079, 2088, 2098, 2104, 2113, 2122, 2128, 2137, 2142, 2154, 2159, 2162, 2168, 2173, 2180, 2184, 2187, 2190, 2194, 2197, 2204, 2211, 2217, 2224, 2230, 2236, 2241, 2245, 2247], "harmed": [0, 2, 8, 15, 19, 22, 25, 28, 32, 37, 46, 49, 52, 61, 69, 79, 85, 90, 96, 106, 116, 129, 142, 153, 159, 168, 173, 181, 189, 198, 216, 232, 238, 248, 260, 264, 272, 283, 289, 299, 306, 313, 325, 333, 341, 359, 364, 373, 380, 386, 401, 409, 421, 434, 440, 452, 462, 471, 481, 495, 502, 511, 517, 528, 537, 542, 547, 553, 558, 564, 574, 584, 596, 602, 612, 621, 629, 641, 642, 649, 652, 656, 660, 668, 679, 689, 693, 705, 710, 717, 726, 731, 738, 751, 762, 771, 783, 794, 806, 826, 845], "exact_score": [80.00287005811867, 80.58405682715075, 81.07914185262251, 81.50965057042406, 81.90428356174213, 82.21998995479659, 82.63614838200473, 82.93750448446582, 83.3249623304872, 83.57609241587143, 83.8774485183325, 84.16445433020019, 84.40123412499103, 84.68106479156204, 84.93937002224295, 85.13309894525365, 85.37705388534118, 85.62818397072542, 85.88648920140633, 86.08739326971371, 86.23089617564756, 86.43897538925164, 86.72598120111932, 86.87665925234987, 87.06321303006386, 87.26411709837124, 87.41479514960179, 87.60852407261247, 87.82377843151325, 87.9529310468537, 88.06055822630408, 88.10360909808423, 88.31168831168831, 88.47671665351223, 88.64174499533615, 88.82829877305016, 88.99332711487408, 89.10095429432447, 89.2659826361484, 89.38796010619215, 89.47406184975246, 89.59603931979623, 89.63909019157639, 89.71084164454331, 89.71801678984, 89.86151969577384, 89.97632202052091, 90.01937289230108, 90.11982492645475, 90.20592667001507, 90.27050297768531, 90.31355384946545, 90.34942957594892, 90.39248044772907, 90.44270646480591, 90.47858219128938, 90.50010762717945, 90.49293248188276, 90.49293248188276, 90.43553131950922, 90.45705675539931, 90.47140704599268, 90.45705675539931, 90.48575733658608, 90.45705675539931, 90.50010762717945, 90.55750878955298, 90.57903422544307, 90.62926024251992, 90.64361053311329, 90.65078567840999, 90.65078567840999, 90.61490995192652, 90.65796082370669, 90.65078567840999, 90.66513596900337, 90.64361053311329, 90.6364353878166, 90.62208509722322, 90.67231111430006, 90.67948625959676, 90.70101169548683, 90.75123771256368, 90.73688742197031, 90.68666140489346, 90.65796082370669, 90.67231111430006, 90.62926024251992, 90.60773480662984, 90.57185908014638, 90.53598335366291, 90.51445791777284, 90.52880820836621, 90.51445791777284, 90.44988161010261, 90.42835617421254, 90.39965559302576, 90.34942957594892, 90.29920355887207, 90.19157637942169, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.58685618921203, "threshold_area100": 88.59316926167757, "threshold_area25": 84.28571428571429}, "selected_budget": 89, "selected": {"val": {"score": 91.09137782870236, "calls": 88.98596390718991, "rescued": 1134, "harmed": 347}, "test": {"score": 90.57185908014638, "calls": 88.99332711487408, "rescued": 2190, "harmed": 717}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.2071921825408936, "operator": "ge"}, {"budget": 0.02, "threshold": 1.9434247016906738, "operator": "ge"}, {"budget": 0.03, "threshold": 1.7496185302734375, "operator": "ge"}, {"budget": 0.04, "threshold": 1.612244725227356, "operator": "ge"}, {"budget": 0.05, "threshold": 1.4739389419555664, "operator": "ge"}, {"budget": 0.06, "threshold": 1.3656625747680664, "operator": "ge"}, {"budget": 0.07, "threshold": 1.268563985824585, "operator": "ge"}, {"budget": 0.08, "threshold": 1.160597801208496, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0784962177276611, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9974961876869202, "operator": "ge"}, {"budget": 0.11, "threshold": 0.928828239440918, "operator": "ge"}, {"budget": 0.12, "threshold": 0.8465667366981506, "operator": "ge"}, {"budget": 0.13, "threshold": 0.7852937579154968, "operator": "ge"}, {"budget": 0.14, "threshold": 0.7213577032089233, "operator": "ge"}, {"budget": 0.15, "threshold": 0.6585538983345032, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5975586175918579, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5181541442871094, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4490278661251068, "operator": "ge"}, {"budget": 0.19, "threshold": 0.3754540979862213, "operator": "ge"}, {"budget": 0.2, "threshold": 0.3074164390563965, "operator": "ge"}, {"budget": 0.21, "threshold": 0.22232168912887573, "operator": "ge"}, {"budget": 0.22, "threshold": 0.1575736254453659, "operator": "ge"}, {"budget": 0.23, "threshold": 0.09272587299346924, "operator": "ge"}, {"budget": 0.24, "threshold": 0.033701539039611816, "operator": "ge"}, {"budget": 0.25, "threshold": -0.04376229643821716, "operator": "ge"}, {"budget": 0.26, "threshold": -0.11732769012451172, "operator": "ge"}, {"budget": 0.27, "threshold": -0.19002822041511536, "operator": "ge"}, {"budget": 0.28, "threshold": -0.2730840742588043, "operator": "ge"}, {"budget": 0.29, "threshold": -0.3454226851463318, "operator": "ge"}, {"budget": 0.3, "threshold": -0.42886224389076233, "operator": "ge"}, {"budget": 0.31, "threshold": -0.4928044378757477, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5755099058151245, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6446835994720459, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7277097702026367, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8022863268852234, "operator": "ge"}, {"budget": 0.36, "threshold": -0.877997636795044, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9679691195487976, "operator": "ge"}, {"budget": 0.38, "threshold": -1.0495319366455078, "operator": "ge"}, {"budget": 0.39, "threshold": -1.134577989578247, "operator": "ge"}, {"budget": 0.4, "threshold": -1.231230616569519, "operator": "ge"}, {"budget": 0.41, "threshold": -1.3121130466461182, "operator": "ge"}, {"budget": 0.42, "threshold": -1.3946952819824219, "operator": "ge"}, {"budget": 0.43, "threshold": -1.465929627418518, "operator": "ge"}, {"budget": 0.44, "threshold": -1.5416269302368164, "operator": "ge"}, {"budget": 0.45, "threshold": -1.6256530284881592, "operator": "ge"}, {"budget": 0.46, "threshold": -1.7156882286071777, "operator": "ge"}, {"budget": 0.47, "threshold": -1.8122330904006958, "operator": "ge"}, {"budget": 0.48, "threshold": -1.8810367584228516, "operator": "ge"}, {"budget": 0.49, "threshold": -1.9695531129837036, "operator": "ge"}, {"budget": 0.5, "threshold": -2.0433192253112793, "operator": "ge"}, {"budget": 0.51, "threshold": -2.129450559616089, "operator": "ge"}, {"budget": 0.52, "threshold": -2.2127697467803955, "operator": "ge"}, {"budget": 0.53, "threshold": -2.299433946609497, "operator": "ge"}, {"budget": 0.54, "threshold": -2.397061347961426, "operator": "ge"}, {"budget": 0.55, "threshold": -2.492452621459961, "operator": "ge"}, {"budget": 0.56, "threshold": -2.5803093910217285, "operator": "ge"}, {"budget": 0.57, "threshold": -2.662411689758301, "operator": "ge"}, {"budget": 0.58, "threshold": -2.7460978031158447, "operator": "ge"}, {"budget": 0.59, "threshold": -2.809758186340332, "operator": "ge"}, {"budget": 0.6, "threshold": -2.897298812866211, "operator": "ge"}, {"budget": 0.61, "threshold": -2.991870403289795, "operator": "ge"}, {"budget": 0.62, "threshold": -3.0718889236450195, "operator": "ge"}, {"budget": 0.63, "threshold": -3.1553494930267334, "operator": "ge"}, {"budget": 0.64, "threshold": -3.2306272983551025, "operator": "ge"}, {"budget": 0.65, "threshold": -3.3130061626434326, "operator": "ge"}, {"budget": 0.66, "threshold": -3.4040799140930176, "operator": "ge"}, {"budget": 0.67, "threshold": -3.490267753601074, "operator": "ge"}, {"budget": 0.68, "threshold": -3.580754280090332, "operator": "ge"}, {"budget": 0.69, "threshold": -3.66575026512146, "operator": "ge"}, {"budget": 0.7, "threshold": -3.753431558609009, "operator": "ge"}, {"budget": 0.71, "threshold": -3.8249239921569824, "operator": "ge"}, {"budget": 0.72, "threshold": -3.9017958641052246, "operator": "ge"}, {"budget": 0.73, "threshold": -3.993893623352051, "operator": "ge"}, {"budget": 0.74, "threshold": -4.08512544631958, "operator": "ge"}, {"budget": 0.75, "threshold": -4.163467884063721, "operator": "ge"}, {"budget": 0.76, "threshold": -4.248836517333984, "operator": "ge"}, {"budget": 0.77, "threshold": -4.332169532775879, "operator": "ge"}, {"budget": 0.78, "threshold": -4.393720626831055, "operator": "ge"}, {"budget": 0.79, "threshold": -4.468562602996826, "operator": "ge"}, {"budget": 0.8, "threshold": -4.547331809997559, "operator": "ge"}, {"budget": 0.81, "threshold": -4.633058547973633, "operator": "ge"}, {"budget": 0.82, "threshold": -4.727134704589844, "operator": "ge"}, {"budget": 0.83, "threshold": -4.796659469604492, "operator": "ge"}, {"budget": 0.84, "threshold": -4.886703014373779, "operator": "ge"}, {"budget": 0.85, "threshold": -4.972859859466553, "operator": "ge"}, {"budget": 0.86, "threshold": -5.048172950744629, "operator": "ge"}, {"budget": 0.87, "threshold": -5.140628814697266, "operator": "ge"}, {"budget": 0.88, "threshold": -5.247747421264648, "operator": "ge"}, {"budget": 0.89, "threshold": -5.3567795753479, "operator": "ge"}, {"budget": 0.9, "threshold": -5.456545829772949, "operator": "ge"}, {"budget": 0.91, "threshold": -5.587186336517334, "operator": "ge"}, {"budget": 0.92, "threshold": -5.73008918762207, "operator": "ge"}, {"budget": 0.93, "threshold": -5.903720378875732, "operator": "ge"}, {"budget": 0.94, "threshold": -6.043275833129883, "operator": "ge"}, {"budget": 0.95, "threshold": -6.196364879608154, "operator": "ge"}, {"budget": 0.96, "threshold": -6.371243476867676, "operator": "ge"}, {"budget": 0.97, "threshold": -6.599544525146484, "operator": "ge"}, {"budget": 0.98, "threshold": -6.911219596862793, "operator": "ge"}, {"budget": 0.99, "threshold": -7.279867172241211, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.32082497851619, 80.7934689200802, 81.10856488112287, 81.53824119163563, 82.03953022056717, 82.41191635634489, 82.81294757949011, 83.09939845316528, 83.37152678315668, 83.54339730736177, 83.87281581208822, 84.1162990547121, 84.41707247207103, 84.61758808364365, 85.0472643941564, 85.30507018046404, 85.62016614150673, 85.8063592093956, 86.04984245201948, 86.30764823832712, 86.43655113148094, 86.56545402463478, 86.75164709252363, 86.86622744199369, 87.05242050988255, 87.2099684904039, 87.5107419077628, 87.69693497565167, 87.89745058722428, 87.94041821827557, 88.06932111142939, 88.1695789172157, 88.31280435405327, 88.42738470352333, 88.57061014036091, 88.64222285877972, 88.71383557719851, 88.79977083930105, 88.92867373245488, 89.01460899455743, 89.17215697507876, 89.30105986823258, 89.4013176740189, 89.47293039243769, 89.573188198224, 89.73073617874535, 89.75938126611285, 89.83099398453164, 89.87396161558291, 89.97421942136923, 90.10312231452305, 90.14608994557433, 90.16041248925809, 90.21770266399312, 90.34660555714694, 90.38957318819823, 90.38957318819823, 90.41821827556574, 90.47550845030077, 90.37525064451447, 90.44686336293324, 90.50415353766829, 90.59008879977084, 90.61873388713835, 90.61873388713835, 90.61873388713835, 90.63305643082211, 90.59008879977084, 90.63305643082211, 90.74763678029217, 90.76195932397593, 90.77628186765969, 90.84789458607848, 90.90518476081351, 90.90518476081351, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.9481523918648, 91.00544256659983, 90.99112002291606, 90.91950730449727, 90.89086221712975, 90.90518476081351, 90.99112002291606, 90.99112002291606, 91.06273274133486, 91.01976511028357, 91.09137782870236, 91.07705528501862, 90.99112002291606, 90.9767974792323, 90.96247493554854, 90.91950730449727, 90.89086221712975, 90.81924949871096, 90.74763678029217, 90.69034660555714, 90.67602406187338, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 35, 68, 92, 123, 159, 186, 215, 238, 258, 272, 299, 322, 347, 365, 395, 418, 444, 461, 483, 505, 521, 538, 557, 573, 592, 612, 636, 654, 674, 683, 699, 711, 724, 736, 754, 763, 775, 785, 798, 811, 826, 839, 854, 866, 877, 890, 897, 908, 912, 925, 940, 946, 953, 961, 972, 980, 986, 993, 1000, 1001, 1008, 1014, 1021, 1028, 1030, 1035, 1038, 1041, 1045, 1054, 1057, 1060, 1067, 1075, 1079, 1085, 1090, 1093, 1094, 1098, 1099, 1100, 1104, 1108, 1115, 1121, 1127, 1129, 1135, 1138, 1140, 1141, 1144, 1148, 1150, 1150, 1151, 1153, 1154, 1154], "harmed": [0, 0, 0, 2, 3, 4, 5, 6, 9, 10, 12, 16, 22, 26, 30, 30, 35, 39, 43, 48, 52, 59, 67, 73, 81, 87, 96, 99, 104, 110, 116, 123, 128, 131, 135, 143, 147, 154, 158, 162, 169, 173, 177, 185, 192, 196, 198, 203, 209, 210, 216, 222, 225, 231, 235, 237, 242, 248, 253, 256, 264, 266, 268, 269, 274, 276, 281, 283, 289, 290, 291, 293, 295, 297, 301, 305, 307, 311, 314, 317, 317, 319, 325, 331, 334, 335, 341, 342, 347, 348, 352, 360, 362, 366, 373, 377, 382, 388, 394, 396, 408], "exact_score": [79.81953594958465, 80.32082497851619, 80.7934689200802, 81.10856488112287, 81.53824119163563, 82.03953022056717, 82.41191635634489, 82.81294757949011, 83.09939845316528, 83.37152678315668, 83.54339730736177, 83.87281581208822, 84.1162990547121, 84.41707247207103, 84.61758808364365, 85.0472643941564, 85.30507018046404, 85.62016614150673, 85.8063592093956, 86.04984245201948, 86.30764823832712, 86.43655113148094, 86.56545402463478, 86.75164709252363, 86.86622744199369, 87.05242050988255, 87.2099684904039, 87.5107419077628, 87.69693497565167, 87.89745058722428, 87.94041821827557, 88.06932111142939, 88.1695789172157, 88.31280435405327, 88.42738470352333, 88.57061014036091, 88.64222285877972, 88.71383557719851, 88.79977083930105, 88.92867373245488, 89.01460899455743, 89.17215697507876, 89.30105986823258, 89.4013176740189, 89.47293039243769, 89.573188198224, 89.73073617874535, 89.75938126611285, 89.83099398453164, 89.87396161558291, 89.97421942136923, 90.10312231452305, 90.14608994557433, 90.16041248925809, 90.21770266399312, 90.34660555714694, 90.38957318819823, 90.38957318819823, 90.41821827556574, 90.47550845030077, 90.37525064451447, 90.44686336293324, 90.50415353766829, 90.59008879977084, 90.61873388713835, 90.61873388713835, 90.61873388713835, 90.63305643082211, 90.59008879977084, 90.63305643082211, 90.74763678029217, 90.76195932397593, 90.77628186765969, 90.84789458607848, 90.90518476081351, 90.90518476081351, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.9481523918648, 91.00544256659983, 90.99112002291606, 90.91950730449727, 90.89086221712975, 90.90518476081351, 90.99112002291606, 90.99112002291606, 91.06273274133486, 91.01976511028357, 91.09137782870236, 91.07705528501862, 90.99112002291606, 90.9767974792323, 90.96247493554854, 90.91950730449727, 90.89086221712975, 90.81924949871096, 90.74763678029217, 90.69034660555714, 90.67602406187338, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.56807261722473, "threshold_area100": 88.56731595531367, "threshold_area25": 84.07476367802921}, "test": {"score": [80.00287005811867, 80.71320944249122, 81.10784243380928, 81.53117600631413, 81.88275812585205, 82.27739111717013, 82.5787472196312, 82.8585778862022, 83.30343689459711, 83.62631843294827, 83.99225084307957, 84.27925665494726, 84.5375618856282, 84.73129080863887, 84.91066944105619, 85.14744923584703, 85.39857932123127, 85.67123484250556, 85.9080146372964, 86.13044414149387, 86.25242161153763, 86.5753031498888, 86.71880605582263, 86.92688526942672, 87.08473846595393, 87.27846738896463, 87.44349573078856, 87.60852407261247, 87.85247901270002, 87.98880677333716, 88.06055822630408, 88.14665996986439, 88.31886345698501, 88.412140345842, 88.62739470474278, 88.79959819186338, 88.93592595250054, 89.09377914902777, 89.158355456698, 89.2588074908517, 89.4238358326756, 89.531463012126, 89.63909019157639, 89.70366649924661, 89.74671737102676, 89.77541795221353, 89.92609600344407, 90.02654803759776, 90.06242376408122, 90.21310181531177, 90.31355384946545, 90.277678122982, 90.34942957594892, 90.41400588361914, 90.47140704599268, 90.51445791777284, 90.53598335366291, 90.55750878955298, 90.53598335366291, 90.52163306306953, 90.49293248188276, 90.45705675539931, 90.46423190069599, 90.50010762717945, 90.52880820836621, 90.53598335366291, 90.53598335366291, 90.56468393484968, 90.58620937073975, 90.59338451603645, 90.6364353878166, 90.67231111430006, 90.67231111430006, 90.70101169548683, 90.67948625959676, 90.68666140489346, 90.70818684078354, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.68666140489346, 90.70818684078354, 90.75123771256368, 90.72253713137691, 90.70818684078354, 90.64361053311329, 90.62208509722322, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.53598335366291, 90.50728277247615, 90.49293248188276, 90.46423190069599, 90.41400588361914, 90.43553131950922, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.19875152471838, 90.06242376408122], "calls": [0.0, 1.3704527516682212, 2.274521059051446, 3.2431656741049, 4.053957092631126, 5.05130228887135, 5.948195450957882, 6.845088613044414, 8.10073903996556, 9.062208509722321, 10.145655449522852, 10.970797158642462, 12.054244098442995, 13.015713568199756, 13.934132166176367, 14.730573294109206, 15.720743345052737, 16.976393771973882, 17.966563822917415, 19.03566047212456, 20.033005668364783, 21.32453182176939, 22.350577599196384, 23.23312047068953, 24.122838487479374, 25.141709119609672, 26.103178589366433, 27.10769893090335, 28.11939441773696, 29.109564468680492, 30.193011408481023, 30.910525938150247, 31.943746860873933, 32.804764296477, 33.84516036449738, 34.8496807060343, 35.73939872282414, 36.80132022673459, 37.68386309822774, 38.67403314917127, 39.865107268422186, 40.92702877233264, 41.816746789122476, 42.562961899978475, 43.38810360909808, 44.42132453182177, 45.64109923225945, 46.85369878740045, 47.650139915333284, 48.84838917988089, 49.73810719667073, 50.756977828801034, 51.96240223864533, 52.97409772547894, 54.17234699002654, 55.24144363923369, 56.31054028844083, 57.200258305230676, 58.26935495443782, 59.05144579177728, 59.969864389753894, 61.2613905431585, 62.22286001291526, 63.26325608093563, 64.26060127717587, 65.34404821697639, 66.43467030207361, 67.4678912247973, 68.55851330989452, 69.48410705316783, 70.66083088182535, 71.62947549687881, 72.64834612900911, 73.64569132524935, 74.80088971801679, 75.71930831599339, 76.82428069168401, 77.97230393915477, 78.6826433235273, 79.6154122120973, 80.56253139126068, 81.52400086101743, 82.55004663844443, 83.29626174930043, 84.37253354380427, 85.25507641529741, 86.06586783382365, 86.98428643180024, 88.03185764511731, 88.94310109779722, 89.98349716581761, 91.04541866972806, 92.05711415656167, 93.29123914759275, 94.28858434383297, 95.1854775059195, 96.09672095859942, 96.95773839420248, 98.01248475281625, 98.92372820549616, 100.0], "rescued": [0, 103, 162, 226, 279, 337, 387, 430, 496, 545, 603, 649, 690, 727, 760, 795, 838, 884, 923, 961, 992, 1047, 1081, 1118, 1147, 1182, 1210, 1241, 1279, 1307, 1334, 1357, 1395, 1417, 1457, 1489, 1515, 1544, 1563, 1585, 1623, 1645, 1667, 1684, 1699, 1712, 1742, 1768, 1781, 1807, 1826, 1839, 1859, 1876, 1897, 1915, 1926, 1940, 1946, 1952, 1958, 1968, 1980, 1989, 2000, 2007, 2015, 2025, 2035, 2044, 2055, 2071, 2081, 2092, 2102, 2109, 2117, 2127, 2131, 2138, 2142, 2149, 2161, 2164, 2170, 2171, 2176, 2183, 2187, 2191, 2194, 2200, 2204, 2212, 2219, 2226, 2231, 2236, 2242, 2245, 2247], "harmed": [0, 4, 8, 13, 17, 20, 28, 32, 36, 40, 47, 53, 58, 68, 76, 78, 86, 94, 100, 107, 121, 131, 145, 153, 160, 168, 173, 181, 185, 194, 211, 222, 236, 245, 255, 263, 270, 277, 287, 295, 310, 317, 324, 332, 341, 350, 359, 371, 379, 384, 389, 407, 417, 425, 438, 450, 458, 469, 478, 486, 496, 511, 522, 526, 533, 539, 547, 553, 560, 568, 573, 584, 594, 601, 614, 620, 625, 639, 641, 646, 653, 657, 663, 670, 678, 688, 696, 703, 712, 721, 726, 736, 742, 754, 768, 772, 783, 792, 806, 824, 845], "exact_score": [80.00287005811867, 80.56970653655738, 80.98586496376552, 81.40202339097367, 81.86123268996197, 82.24869053598336, 82.60027265552128, 82.91597904857574, 83.27473631341034, 83.6047929970582, 83.94919997129942, 84.28643180024396, 84.5232115950348, 84.72411566334218, 84.91784458635287, 85.18332496233049, 85.4488053383081, 85.67123484250556, 85.9080146372964, 86.11609385090048, 86.23807132094426, 86.48920140632848, 86.71163091052594, 86.8479586711631, 87.05603788476716, 87.25694195307454, 87.41479514960179, 87.56547320083232, 87.82377843151325, 87.98880677333716, 88.0533830810074, 88.15383511516109, 88.31168831168831, 88.47671665351223, 88.66327043122624, 88.7924230465667, 89.00767740546746, 89.09377914902777, 89.18705603788477, 89.344909234412, 89.44536126856569, 89.5386381574227, 89.66061562746646, 89.76106766162016, 89.75389251632346, 89.88304513166392, 89.94044629403746, 90.04089832819116, 90.09112434526799, 90.23462725120184, 90.26332783238861, 90.31355384946545, 90.34942957594892, 90.41400588361914, 90.46423190069599, 90.53598335366291, 90.5503336442563, 90.55750878955298, 90.54315849895961, 90.52163306306953, 90.49293248188276, 90.46423190069599, 90.48575733658608, 90.50010762717945, 90.53598335366291, 90.53598335366291, 90.54315849895961, 90.54315849895961, 90.58620937073975, 90.60773480662984, 90.61490995192652, 90.65078567840999, 90.66513596900337, 90.67231111430006, 90.68666140489346, 90.68666140489346, 90.67231111430006, 90.72253713137691, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.69383655019014, 90.71536198608023, 90.71536198608023, 90.69383655019014, 90.66513596900337, 90.62926024251992, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.53598335366291, 90.50728277247615, 90.49293248188276, 90.48575733658608, 90.42118102891584, 90.42118102891584, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.19875152471838, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.59345447074284, "threshold_area100": 88.59510655090766, "threshold_area25": 84.26203630623522}, "selected_budget": 89, "selected": {"val": {"score": 91.09137782870236, "calls": 88.98596390718991, "rescued": 1135, "harmed": 348}, "test": {"score": 90.5503336442563, "calls": 88.94310109779722, "rescued": 2191, "harmed": 721}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.86221712976223, "calls": 95.9896877685477}, "test": {"score": 90.59338451603645, "calls": 95.95321805266556}}, "old_normalized_interior": {"val": {"score": 91.09137782870236, "calls": 88.98596390718991}, "test": {"score": 90.5503336442563, "calls": 88.94310109779722}}, "old_normalized_endpoints": {"val": {"score": 91.09137782870236, "calls": 88.98596390718991}, "test": {"score": 90.5503336442563, "calls": 88.94310109779722}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.32082497851619, 80.7934689200802, 81.10856488112287, 81.53824119163563, 82.03953022056717, 82.41191635634489, 82.81294757949011, 83.09939845316528, 83.37152678315668, 83.54339730736177, 83.87281581208822, 84.1162990547121, 84.41707247207103, 84.61758808364365, 85.0472643941564, 85.30507018046404, 85.62016614150673, 85.8063592093956, 86.04984245201948, 86.30764823832712, 86.43655113148094, 86.56545402463478, 86.75164709252363, 86.86622744199369, 87.05242050988255, 87.2099684904039, 87.5107419077628, 87.69693497565167, 87.89745058722428, 87.94041821827557, 88.06932111142939, 88.1695789172157, 88.31280435405327, 88.42738470352333, 88.57061014036091, 88.64222285877972, 88.71383557719851, 88.79977083930105, 88.92867373245488, 89.01460899455743, 89.17215697507876, 89.30105986823258, 89.4013176740189, 89.47293039243769, 89.573188198224, 89.73073617874535, 89.75938126611285, 89.83099398453164, 89.87396161558291, 89.97421942136923, 90.10312231452305, 90.14608994557433, 90.16041248925809, 90.21770266399312, 90.34660555714694, 90.38957318819823, 90.38957318819823, 90.41821827556574, 90.47550845030077, 90.37525064451447, 90.44686336293324, 90.50415353766829, 90.59008879977084, 90.61873388713835, 90.61873388713835, 90.61873388713835, 90.63305643082211, 90.59008879977084, 90.63305643082211, 90.74763678029217, 90.76195932397593, 90.77628186765969, 90.84789458607848, 90.90518476081351, 90.90518476081351, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.9481523918648, 91.00544256659983, 90.99112002291606, 90.91950730449727, 90.89086221712975, 90.90518476081351, 90.99112002291606, 90.99112002291606, 91.06273274133486, 91.01976511028357, 91.09137782870236, 91.07705528501862, 90.99112002291606, 90.9767974792323, 90.96247493554854, 90.91950730449727, 90.89086221712975, 90.81924949871096, 90.74763678029217, 90.69034660555714, 90.67602406187338, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.32082497851619, 80.7934689200802, 81.10856488112287, 81.53824119163563, 82.03953022056717, 82.41191635634489, 82.81294757949011, 83.09939845316528, 83.37152678315668, 83.54339730736177, 83.87281581208822, 84.1162990547121, 84.41707247207103, 84.61758808364365, 85.0472643941564, 85.30507018046404, 85.62016614150673, 85.8063592093956, 86.04984245201948, 86.30764823832712, 86.43655113148094, 86.56545402463478, 86.75164709252363, 86.86622744199369, 87.05242050988255, 87.2099684904039, 87.5107419077628, 87.69693497565167, 87.89745058722428, 87.94041821827557, 88.06932111142939, 88.1695789172157, 88.31280435405327, 88.42738470352333, 88.57061014036091, 88.64222285877972, 88.71383557719851, 88.79977083930105, 88.92867373245488, 89.01460899455743, 89.17215697507876, 89.30105986823258, 89.4013176740189, 89.47293039243769, 89.573188198224, 89.73073617874535, 89.75938126611285, 89.83099398453164, 89.87396161558291, 89.97421942136923, 90.10312231452305, 90.14608994557433, 90.16041248925809, 90.21770266399312, 90.34660555714694, 90.38957318819823, 90.38957318819823, 90.41821827556574, 90.47550845030077, 90.37525064451447, 90.44686336293324, 90.50415353766829, 90.59008879977084, 90.61873388713835, 90.61873388713835, 90.61873388713835, 90.63305643082211, 90.59008879977084, 90.63305643082211, 90.74763678029217, 90.76195932397593, 90.77628186765969, 90.84789458607848, 90.90518476081351, 90.90518476081351, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.9481523918648, 91.00544256659983, 90.99112002291606, 90.91950730449727, 90.89086221712975, 90.90518476081351, 90.99112002291606, 90.99112002291606, 91.06273274133486, 91.01976511028357, 91.09137782870236, 91.07705528501862, 90.99112002291606, 90.9767974792323, 90.96247493554854, 90.91950730449727, 90.89086221712975, 90.81924949871096, 90.74763678029217, 90.69034660555714, 90.67602406187338, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 84.07476367802921, "exact_area100": 88.56807261722473}, "test": {"score": [80.00287005811867, 80.71320944249122, 81.10784243380928, 81.53117600631413, 81.88275812585205, 82.27739111717013, 82.5787472196312, 82.8585778862022, 83.30343689459711, 83.62631843294827, 83.99225084307957, 84.27925665494726, 84.5375618856282, 84.73129080863887, 84.91066944105619, 85.14744923584703, 85.39857932123127, 85.67123484250556, 85.9080146372964, 86.13044414149387, 86.25242161153763, 86.5753031498888, 86.71880605582263, 86.92688526942672, 87.08473846595393, 87.27846738896463, 87.44349573078856, 87.60852407261247, 87.85247901270002, 87.98880677333716, 88.06055822630408, 88.14665996986439, 88.31886345698501, 88.412140345842, 88.62739470474278, 88.79959819186338, 88.93592595250054, 89.09377914902777, 89.158355456698, 89.2588074908517, 89.4238358326756, 89.531463012126, 89.63909019157639, 89.70366649924661, 89.74671737102676, 89.77541795221353, 89.92609600344407, 90.02654803759776, 90.06242376408122, 90.21310181531177, 90.31355384946545, 90.277678122982, 90.34942957594892, 90.41400588361914, 90.47140704599268, 90.51445791777284, 90.53598335366291, 90.55750878955298, 90.53598335366291, 90.52163306306953, 90.49293248188276, 90.45705675539931, 90.46423190069599, 90.50010762717945, 90.52880820836621, 90.53598335366291, 90.53598335366291, 90.56468393484968, 90.58620937073975, 90.59338451603645, 90.6364353878166, 90.67231111430006, 90.67231111430006, 90.70101169548683, 90.67948625959676, 90.68666140489346, 90.70818684078354, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.68666140489346, 90.70818684078354, 90.75123771256368, 90.72253713137691, 90.70818684078354, 90.64361053311329, 90.62208509722322, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.53598335366291, 90.50728277247615, 90.49293248188276, 90.46423190069599, 90.41400588361914, 90.43553131950922, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.19875152471838, 90.06242376408122], "calls": [0.0, 1.3704527516682212, 2.274521059051446, 3.2431656741049, 4.053957092631126, 5.05130228887135, 5.948195450957882, 6.845088613044414, 8.10073903996556, 9.062208509722321, 10.145655449522852, 10.970797158642462, 12.054244098442995, 13.015713568199756, 13.934132166176367, 14.730573294109206, 15.720743345052737, 16.976393771973882, 17.966563822917415, 19.03566047212456, 20.033005668364783, 21.32453182176939, 22.350577599196384, 23.23312047068953, 24.122838487479374, 25.141709119609672, 26.103178589366433, 27.10769893090335, 28.11939441773696, 29.109564468680492, 30.193011408481023, 30.910525938150247, 31.943746860873933, 32.804764296477, 33.84516036449738, 34.8496807060343, 35.73939872282414, 36.80132022673459, 37.68386309822774, 38.67403314917127, 39.865107268422186, 40.92702877233264, 41.816746789122476, 42.562961899978475, 43.38810360909808, 44.42132453182177, 45.64109923225945, 46.85369878740045, 47.650139915333284, 48.84838917988089, 49.73810719667073, 50.756977828801034, 51.96240223864533, 52.97409772547894, 54.17234699002654, 55.24144363923369, 56.31054028844083, 57.200258305230676, 58.26935495443782, 59.05144579177728, 59.969864389753894, 61.2613905431585, 62.22286001291526, 63.26325608093563, 64.26060127717587, 65.34404821697639, 66.43467030207361, 67.4678912247973, 68.55851330989452, 69.48410705316783, 70.66083088182535, 71.62947549687881, 72.64834612900911, 73.64569132524935, 74.80088971801679, 75.71930831599339, 76.82428069168401, 77.97230393915477, 78.6826433235273, 79.6154122120973, 80.56253139126068, 81.52400086101743, 82.55004663844443, 83.29626174930043, 84.37253354380427, 85.25507641529741, 86.06586783382365, 86.98428643180024, 88.03185764511731, 88.94310109779722, 89.98349716581761, 91.04541866972806, 92.05711415656167, 93.29123914759275, 94.28858434383297, 95.1854775059195, 96.09672095859942, 96.95773839420248, 98.01248475281625, 98.92372820549616, 100.0], "exact_score": [80.00287005811867, 80.56970653655738, 80.98586496376552, 81.40202339097367, 81.86123268996197, 82.24869053598336, 82.60027265552128, 82.91597904857574, 83.27473631341034, 83.6047929970582, 83.94919997129942, 84.28643180024396, 84.5232115950348, 84.72411566334218, 84.91784458635287, 85.18332496233049, 85.4488053383081, 85.67123484250556, 85.9080146372964, 86.11609385090048, 86.23807132094426, 86.48920140632848, 86.71163091052594, 86.8479586711631, 87.05603788476716, 87.25694195307454, 87.41479514960179, 87.56547320083232, 87.82377843151325, 87.98880677333716, 88.0533830810074, 88.15383511516109, 88.31168831168831, 88.47671665351223, 88.66327043122624, 88.7924230465667, 89.00767740546746, 89.09377914902777, 89.18705603788477, 89.344909234412, 89.44536126856569, 89.5386381574227, 89.66061562746646, 89.76106766162016, 89.75389251632346, 89.88304513166392, 89.94044629403746, 90.04089832819116, 90.09112434526799, 90.23462725120184, 90.26332783238861, 90.31355384946545, 90.34942957594892, 90.41400588361914, 90.46423190069599, 90.53598335366291, 90.5503336442563, 90.55750878955298, 90.54315849895961, 90.52163306306953, 90.49293248188276, 90.46423190069599, 90.48575733658608, 90.50010762717945, 90.53598335366291, 90.53598335366291, 90.54315849895961, 90.54315849895961, 90.58620937073975, 90.60773480662984, 90.61490995192652, 90.65078567840999, 90.66513596900337, 90.67231111430006, 90.68666140489346, 90.68666140489346, 90.67231111430006, 90.72253713137691, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.69383655019014, 90.71536198608023, 90.71536198608023, 90.69383655019014, 90.66513596900337, 90.62926024251992, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.53598335366291, 90.50728277247615, 90.49293248188276, 90.48575733658608, 90.42118102891584, 90.42118102891584, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.19875152471838, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 84.26203630623522, "exact_area100": 88.59345447074284}}, "old_normalized_endpoint_budget": 89}, {"seed": 2029, "selection": {"best_epoch": 8, "epochs": 16, "alpha": 1.0, "validation_area100": 0.8848358355868807, "training_seconds": 49.04504462494515, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8803408347327412, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8168046538344066, "operator": "ge"}, {"budget": 0.03, "threshold": 0.7671628192073273, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7154617577313155, "operator": "ge"}, {"budget": 0.05, "threshold": 0.6698242263697192, "operator": "ge"}, {"budget": 0.06, "threshold": 0.6318310576103069, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5936824607089403, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5541048850656477, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5183599931729466, "operator": "ge"}, {"budget": 0.1, "threshold": 0.47735352012060234, "operator": "ge"}, {"budget": 0.11, "threshold": 0.43575108372063037, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3965405670413272, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3611052677816315, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3291694884856133, "operator": "ge"}, {"budget": 0.15, "threshold": 0.29067338567085277, "operator": "ge"}, {"budget": 0.16, "threshold": 0.2562345426106126, "operator": "ge"}, {"budget": 0.17, "threshold": 0.21439309687533442, "operator": "ge"}, {"budget": 0.18, "threshold": 0.18063806865794635, "operator": "ge"}, {"budget": 0.19, "threshold": 0.14349345600718674, "operator": "ge"}, {"budget": 0.2, "threshold": 0.1115766835871982, "operator": "ge"}, {"budget": 0.21, "threshold": 0.07823254376704318, "operator": "ge"}, {"budget": 0.22, "threshold": 0.039353898364490714, "operator": "ge"}, {"budget": 0.23, "threshold": 0.002285667353446123, "operator": "ge"}, {"budget": 0.24, "threshold": -0.041593545017324124, "operator": "ge"}, {"budget": 0.25, "threshold": -0.08991838482776847, "operator": "ge"}, {"budget": 0.26, "threshold": -0.12990219274006845, "operator": "ge"}, {"budget": 0.27, "threshold": -0.16432603980303073, "operator": "ge"}, {"budget": 0.28, "threshold": -0.20863550373655726, "operator": "ge"}, {"budget": 0.29, "threshold": -0.25499087980252355, "operator": "ge"}, {"budget": 0.3, "threshold": -0.29745057906085676, "operator": "ge"}, {"budget": 0.31, "threshold": -0.32955921231231033, "operator": "ge"}, {"budget": 0.32, "threshold": -0.3809697851073907, "operator": "ge"}, {"budget": 0.33, "threshold": -0.42013297670999333, "operator": "ge"}, {"budget": 0.34, "threshold": -0.4604848798102375, "operator": "ge"}, {"budget": 0.35, "threshold": -0.4964543901968352, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5280746989460772, "operator": "ge"}, {"budget": 0.37, "threshold": -0.5593001811446944, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5905917257095602, "operator": "ge"}, {"budget": 0.39, "threshold": -0.6181782188921037, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6435414944456463, "operator": "ge"}, {"budget": 0.41, "threshold": -0.6736341802217034, "operator": "ge"}, {"budget": 0.42, "threshold": -0.699424416545209, "operator": "ge"}, {"budget": 0.43, "threshold": -0.726465170820239, "operator": "ge"}, {"budget": 0.44, "threshold": -0.7435473840894412, "operator": "ge"}, {"budget": 0.45, "threshold": -0.7610191170909176, "operator": "ge"}, {"budget": 0.46, "threshold": -0.7821417206242841, "operator": "ge"}, {"budget": 0.47, "threshold": -0.8014016208308252, "operator": "ge"}, {"budget": 0.48, "threshold": -0.8194816405677307, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8351939552214834, "operator": "ge"}, {"budget": 0.5, "threshold": -0.8493938558005462, "operator": "ge"}, {"budget": 0.51, "threshold": -0.8623957702660158, "operator": "ge"}, {"budget": 0.52, "threshold": -0.8759876184051366, "operator": "ge"}, {"budget": 0.53, "threshold": -0.8863994638227403, "operator": "ge"}, {"budget": 0.54, "threshold": -0.8969002568221538, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9068275808461379, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9160746595297761, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9244210866313398, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9315114530476104, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9384887414562945, "operator": "ge"}, {"budget": 0.6, "threshold": -0.944313192409181, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9497496201111725, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9556548585171807, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9594441934341086, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9633970370055364, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9674712320642792, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9707589771893613, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9730104778834118, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9756330375442853, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9784371896197844, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9806239069196621, "operator": "ge"}, {"budget": 0.71, "threshold": -0.982302867270833, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9844023848001764, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9860699329592703, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9876953996132971, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9887469763908985, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9897469063442639, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9908935374433329, "operator": "ge"}, {"budget": 0.78, "threshold": -0.991810580009287, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9926379206871516, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9933056988621827, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9942108901260929, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9948301155311453, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9954966629818827, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9960023443665875, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9964538454190239, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9968962799370216, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9973248951083998, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9976319417220958, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9979659119647936, "operator": "ge"}, {"budget": 0.9, "threshold": -0.998283329499277, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9985576612451343, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9988047705672751, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9990159998302302, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9992185001771513, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9994154339436716, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9995554843321193, "operator": "ge"}, {"budget": 0.97, "threshold": -0.999700904148607, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9998314285004156, "operator": "ge"}, {"budget": 0.99, "threshold": -0.999928412933641, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.29217989114866, 80.76482383271268, 81.15153251217416, 81.63849899742193, 81.93927241478086, 82.31165855055858, 82.58378688054998, 82.91320538527643, 83.21397880263535, 83.51475221999426, 83.7439129189344, 84.00171870524204, 84.27384703523344, 84.64623317101116, 84.91836150100256, 85.1475221999427, 85.44829561730162, 85.63448868519048, 85.89229447149813, 86.15010025780579, 86.27900315095961, 86.37926095674591, 86.55113148095101, 86.83758235462618, 86.98080779146376, 87.19564594672013, 87.36751647092524, 87.4820968203953, 87.72558006301918, 87.81151532512173, 87.854482956173, 88.05499856774563, 88.1695789172157, 88.35577198510455, 88.41306215983958, 88.52764250930964, 88.67086794614723, 88.84273847035233, 88.91435118877112, 89.04325408192494, 89.21512460613005, 89.34402749928387, 89.53022056717272, 89.573188198224, 89.64480091664278, 89.70209109137782, 89.77370380979661, 90.0171870524205, 90.0887997708393, 90.16041248925809, 90.18905757662561, 90.18905757662561, 90.23202520767688, 90.18905757662561, 90.21770266399312, 90.28931538241191, 90.26067029504439, 90.34660555714694, 90.40389573188197, 90.38957318819823, 90.41821827556574, 90.48983099398453, 90.48983099398453, 90.48983099398453, 90.50415353766829, 90.56144371240332, 90.53279862503581, 90.56144371240332, 90.56144371240332, 90.63305643082211, 90.64737897450587, 90.74763678029217, 90.79060441134345, 90.76195932397593, 90.80492695502721, 90.76195932397593, 90.73331423660842, 90.81924949871096, 90.89086221712975, 90.90518476081351, 90.876539673446, 90.93382984818103, 90.90518476081351, 90.89086221712975, 90.93382984818103, 90.93382984818103, 90.90518476081351, 90.83357204239474, 90.79060441134345, 90.77628186765969, 90.71899169292466, 90.7046691492409, 90.69034660555714, 90.63305643082211, 90.63305643082211, 90.64737897450587, 90.63305643082211, 90.59008879977084, 90.59008879977084, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 34, 67, 94, 131, 152, 180, 202, 227, 249, 273, 293, 315, 334, 361, 381, 401, 424, 443, 463, 486, 500, 514, 531, 554, 574, 593, 610, 626, 646, 657, 669, 689, 704, 723, 733, 747, 761, 778, 786, 800, 819, 830, 848, 857, 865, 876, 888, 908, 920, 929, 938, 944, 954, 957, 964, 973, 977, 988, 995, 999, 1003, 1009, 1015, 1022, 1028, 1034, 1036, 1043, 1045, 1054, 1059, 1067, 1074, 1077, 1082, 1085, 1088, 1094, 1101, 1104, 1107, 1114, 1115, 1119, 1123, 1126, 1133, 1133, 1134, 1137, 1137, 1140, 1141, 1142, 1146, 1149, 1150, 1151, 1154, 1154], "harmed": [0, 1, 1, 1, 4, 4, 6, 9, 11, 12, 15, 19, 23, 23, 24, 25, 29, 31, 37, 39, 44, 49, 56, 61, 64, 74, 78, 83, 91, 94, 99, 108, 114, 121, 127, 133, 139, 143, 148, 151, 156, 163, 165, 170, 176, 179, 186, 193, 196, 203, 207, 214, 220, 227, 233, 238, 242, 248, 253, 256, 261, 263, 264, 270, 277, 282, 284, 288, 293, 295, 299, 303, 304, 308, 313, 315, 321, 326, 326, 328, 330, 335, 338, 341, 346, 347, 350, 359, 364, 368, 372, 376, 380, 382, 387, 391, 393, 395, 399, 402, 408], "exact_score": [79.81953594958465, 80.29217989114866, 80.76482383271268, 81.15153251217416, 81.63849899742193, 81.93927241478086, 82.31165855055858, 82.58378688054998, 82.91320538527643, 83.21397880263535, 83.51475221999426, 83.7439129189344, 84.00171870524204, 84.27384703523344, 84.64623317101116, 84.91836150100256, 85.1475221999427, 85.44829561730162, 85.63448868519048, 85.89229447149813, 86.15010025780579, 86.27900315095961, 86.37926095674591, 86.55113148095101, 86.83758235462618, 86.98080779146376, 87.19564594672013, 87.36751647092524, 87.4820968203953, 87.72558006301918, 87.81151532512173, 87.854482956173, 88.05499856774563, 88.1695789172157, 88.35577198510455, 88.41306215983958, 88.52764250930964, 88.67086794614723, 88.84273847035233, 88.91435118877112, 89.04325408192494, 89.21512460613005, 89.34402749928387, 89.53022056717272, 89.573188198224, 89.64480091664278, 89.70209109137782, 89.77370380979661, 90.0171870524205, 90.0887997708393, 90.16041248925809, 90.18905757662561, 90.18905757662561, 90.23202520767688, 90.18905757662561, 90.21770266399312, 90.28931538241191, 90.26067029504439, 90.34660555714694, 90.40389573188197, 90.38957318819823, 90.41821827556574, 90.48983099398453, 90.48983099398453, 90.48983099398453, 90.50415353766829, 90.56144371240332, 90.53279862503581, 90.56144371240332, 90.56144371240332, 90.63305643082211, 90.64737897450587, 90.74763678029217, 90.79060441134345, 90.76195932397593, 90.80492695502721, 90.76195932397593, 90.73331423660842, 90.81924949871096, 90.89086221712975, 90.90518476081351, 90.876539673446, 90.93382984818103, 90.90518476081351, 90.89086221712975, 90.93382984818103, 90.93382984818103, 90.90518476081351, 90.83357204239474, 90.79060441134345, 90.77628186765969, 90.71899169292466, 90.7046691492409, 90.69034660555714, 90.63305643082211, 90.63305643082211, 90.64737897450587, 90.63305643082211, 90.59008879977084, 90.59008879977084, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.48358355868807, "threshold_area100": 88.48281294757949, "threshold_area25": 83.98453165282153}, "test": {"score": [80.00287005811867, 80.51230537418384, 80.97868981846884, 81.39484824567698, 81.72490492932482, 82.10518763004951, 82.36349286073043, 82.65767381789482, 83.0451316639162, 83.35366291167396, 83.64066872354165, 83.97072540718949, 84.28643180024396, 84.58061275740833, 84.8173925521992, 85.06852263758341, 85.29095214178088, 85.59948338953865, 85.86496376551625, 86.04434239793356, 86.26677190213103, 86.43180024395495, 86.51072684221856, 86.69010547463587, 86.8407835258664, 87.05603788476716, 87.27846738896463, 87.47937145727201, 87.64439979909594, 87.82377843151325, 88.02468249982063, 88.09643395278755, 88.29016287579825, 88.42649063643539, 88.61304441414939, 88.75654732008323, 88.92157566190716, 89.10095429432447, 89.158355456698, 89.28750807203846, 89.40231039678554, 89.51711272153261, 89.57451388390615, 89.69649135394992, 89.76824280691685, 89.86869484107054, 89.95479658463084, 90.03372318289445, 90.0839491999713, 90.10547463586138, 90.19157637942169, 90.23462725120184, 90.30637870416876, 90.28485326827868, 90.32072899476215, 90.34225443065222, 90.3566047212456, 90.3566047212456, 90.37813015713569, 90.44270646480591, 90.46423190069599, 90.50728277247615, 90.50010762717945, 90.49293248188276, 90.47858219128938, 90.45705675539931, 90.47140704599268, 90.42118102891584, 90.40683073832245, 90.39965559302576, 90.37813015713569, 90.3566047212456, 90.37095501183899, 90.41400588361914, 90.38530530243239, 90.3566047212456, 90.41400588361914, 90.3566047212456, 90.37095501183899, 90.37813015713569, 90.39248044772907, 90.40683073832245, 90.41400588361914, 90.3637798665423, 90.37813015713569, 90.34942957594892, 90.32072899476215, 90.27050297768531, 90.23462725120184, 90.24180239649853, 90.27050297768531, 90.24897754179523, 90.24897754179523, 90.25615268709191, 90.21310181531177, 90.22745210590514, 90.24897754179523, 90.23462725120184, 90.1772260888283, 90.09112434526799, 90.06242376408122], "calls": [0.0, 1.1838989739542225, 2.260170768458061, 3.200114802324747, 4.13288369089474, 5.072827724761426, 5.919494869771113, 6.909664920714645, 7.985936715218483, 8.8039032790414, 10.04520341536916, 11.171701226949846, 12.269498457343762, 13.295544234770754, 14.213962832747363, 15.326110353734663, 16.151252062854272, 17.31362560091842, 18.30379565186195, 19.243739685728638, 20.31283633493578, 21.25278036880247, 22.221424983855922, 23.11114300064576, 24.151539068666143, 25.33543804262036, 26.37583411064074, 27.287077563320654, 28.392049939011265, 29.425270861734948, 30.429791203271865, 31.039678553490706, 32.273803544521776, 33.30702446724546, 34.48374829590299, 35.41651718447299, 36.25600918418598, 37.31075554279974, 38.243524431369735, 39.2049939011265, 40.2238645332568, 41.285786037167256, 42.41228384874794, 43.58183253210878, 44.32804764296477, 45.17471478797446, 46.12183396713783, 47.23398148812513, 48.24567697495875, 49.22149673530889, 50.16144076917558, 51.07985936715218, 52.09872999928249, 53.074549759632625, 54.114945827653, 55.19121762215685, 56.13116165602353, 57.2791849034943, 58.27653009973452, 59.41737820190859, 60.42907368874219, 61.51252062854273, 62.811221927244034, 63.77986654229748, 64.8704886273947, 66.11178876372247, 67.13783454114946, 67.94862595967568, 68.89574513883906, 70.02224295041974, 70.9908875654732, 71.90930616344981, 72.96405252206357, 73.8752959747435, 75.03766951280764, 75.89151180311401, 76.77405467460716, 77.81445074262754, 78.73286934060415, 79.65128793858077, 80.41185334003013, 81.54552629690751, 82.55722178374111, 83.66219415943172, 84.51603644973811, 85.31965272296765, 86.23807132094426, 87.24259166248116, 88.01750735452393, 88.84264906364353, 89.92609600344407, 90.87321518260745, 91.90643610533114, 92.98988304513166, 94.21683289086604, 95.05632489057903, 95.93169261677549, 97.07254071894957, 98.17033794934348, 99.23943459855063, 100.0], "rescued": [0, 75, 143, 205, 254, 310, 350, 397, 456, 500, 548, 596, 647, 692, 732, 773, 810, 858, 900, 934, 972, 1006, 1032, 1065, 1100, 1140, 1175, 1209, 1243, 1279, 1317, 1334, 1371, 1399, 1442, 1470, 1503, 1535, 1555, 1580, 1605, 1632, 1654, 1683, 1702, 1723, 1745, 1769, 1786, 1800, 1823, 1836, 1857, 1865, 1881, 1897, 1910, 1922, 1936, 1957, 1970, 1983, 1998, 2007, 2015, 2025, 2038, 2047, 2053, 2059, 2067, 2073, 2083, 2093, 2102, 2107, 2120, 2123, 2129, 2134, 2140, 2149, 2155, 2160, 2167, 2171, 2174, 2179, 2182, 2188, 2199, 2205, 2207, 2213, 2217, 2225, 2233, 2241, 2243, 2245, 2247], "harmed": [0, 4, 7, 11, 14, 17, 21, 27, 32, 33, 41, 43, 50, 54, 61, 67, 73, 78, 83, 92, 99, 110, 125, 133, 147, 157, 161, 167, 178, 189, 199, 206, 216, 225, 242, 250, 260, 267, 279, 286, 295, 306, 320, 332, 341, 348, 358, 371, 381, 392, 403, 410, 421, 432, 443, 456, 467, 479, 490, 502, 512, 519, 535, 545, 555, 568, 579, 595, 603, 610, 621, 630, 638, 642, 655, 664, 669, 680, 684, 688, 692, 699, 704, 716, 721, 729, 736, 748, 756, 761, 768, 777, 779, 784, 794, 800, 805, 815, 825, 839, 845], "exact_score": [80.00287005811867, 80.4405539212169, 80.87823778431513, 81.34462222860013, 81.66750376695128, 82.06213675826936, 82.40654373251058, 82.68637439908159, 83.0523068092129, 83.4110640740475, 83.64066872354165, 83.89897395422258, 84.21468034727704, 84.49451101384804, 84.77434168041903, 84.97524574872641, 85.2694267058908, 85.52055679127503, 85.79321231254933, 85.97976609026333, 86.19502044916409, 86.41027480806487, 86.47485111573509, 86.66140489344909, 86.81208294467963, 86.96276099591017, 87.18519050010764, 87.42914544019517, 87.55112291023893, 87.79507785032648, 87.93140561096362, 88.08208366219417, 88.22558656812801, 88.39779005524862, 88.54129296118248, 88.68479586711632, 88.87134964483032, 89.05790342254431, 89.11530458491785, 89.251632345555, 89.40231039678554, 89.50276243093923, 89.54581330271938, 89.63191504627969, 89.71084164454331, 89.85434455047715, 89.93327114874077, 89.99067231111431, 90.0839491999713, 90.0982994905647, 90.19875152471838, 90.22745210590514, 90.277678122982, 90.28485326827868, 90.33507928535553, 90.34225443065222, 90.37095501183899, 90.34942957594892, 90.37813015713569, 90.41400588361914, 90.47140704599268, 90.53598335366291, 90.51445791777284, 90.50728277247615, 90.50010762717945, 90.50010762717945, 90.46423190069599, 90.47140704599268, 90.41400588361914, 90.40683073832245, 90.39965559302576, 90.37813015713569, 90.34942957594892, 90.37095501183899, 90.42118102891584, 90.38530530243239, 90.3566047212456, 90.40683073832245, 90.3637798665423, 90.38530530243239, 90.37813015713569, 90.38530530243239, 90.39965559302576, 90.39965559302576, 90.37095501183899, 90.34942957594892, 90.34225443065222, 90.28485326827868, 90.24180239649853, 90.24180239649853, 90.28485326827868, 90.24897754179523, 90.24180239649853, 90.25615268709191, 90.22027696060846, 90.22745210590514, 90.24180239649853, 90.22745210590514, 90.19157637942169, 90.11264978115807, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.4239808543215, "threshold_area100": 88.44220420463516, "threshold_area25": 84.12212097294972}, "selected_budget": 82, "selected": {"val": {"score": 90.93382984818103, "calls": 81.9965625895159, "rescued": 1114, "harmed": 338}, "test": {"score": 90.41400588361914, "calls": 82.55722178374111, "rescued": 2155, "harmed": 704}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7996295690536499, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6492824554443359, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5699692368507385, "operator": "ge"}, {"budget": 0.04, "threshold": 0.49496009945869446, "operator": "ge"}, {"budget": 0.05, "threshold": 0.43852347135543823, "operator": "ge"}, {"budget": 0.06, "threshold": 0.3871402442455292, "operator": "ge"}, {"budget": 0.07, "threshold": 0.3376520872116089, "operator": "ge"}, {"budget": 0.08, "threshold": 0.30039358139038086, "operator": "ge"}, {"budget": 0.09, "threshold": 0.2644583582878113, "operator": "ge"}, {"budget": 0.1, "threshold": 0.23447202146053314, "operator": "ge"}, {"budget": 0.11, "threshold": 0.20443417131900787, "operator": "ge"}, {"budget": 0.12, "threshold": 0.17909280955791473, "operator": "ge"}, {"budget": 0.13, "threshold": 0.15265579521656036, "operator": "ge"}, {"budget": 0.14, "threshold": 0.12664887309074402, "operator": "ge"}, {"budget": 0.15, "threshold": 0.10788560658693314, "operator": "ge"}, {"budget": 0.16, "threshold": 0.08850033581256866, "operator": "ge"}, {"budget": 0.17, "threshold": 0.07030875235795975, "operator": "ge"}, {"budget": 0.18, "threshold": 0.05661039799451828, "operator": "ge"}, {"budget": 0.19, "threshold": 0.043292056769132614, "operator": "ge"}, {"budget": 0.2, "threshold": 0.03042873740196228, "operator": "ge"}, {"budget": 0.21, "threshold": 0.01873808354139328, "operator": "ge"}, {"budget": 0.22, "threshold": 0.007343992590904236, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0005430756136775017, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0026374920271337032, "operator": "ge"}, {"budget": 0.25, "threshold": -0.0042986683547496796, "operator": "ge"}, {"budget": 0.26, "threshold": -0.005604641046375036, "operator": "ge"}, {"budget": 0.27, "threshold": -0.006755332462489605, "operator": "ge"}, {"budget": 0.28, "threshold": -0.007591861300170422, "operator": "ge"}, {"budget": 0.29, "threshold": -0.008959535509347916, "operator": "ge"}, {"budget": 0.3, "threshold": -0.010335635393857956, "operator": "ge"}, {"budget": 0.31, "threshold": -0.01115699578076601, "operator": "ge"}, {"budget": 0.32, "threshold": -0.012242908589541912, "operator": "ge"}, {"budget": 0.33, "threshold": -0.013346719555556774, "operator": "ge"}, {"budget": 0.34, "threshold": -0.014391440898180008, "operator": "ge"}, {"budget": 0.35, "threshold": -0.015520082786679268, "operator": "ge"}, {"budget": 0.36, "threshold": -0.017124947160482407, "operator": "ge"}, {"budget": 0.37, "threshold": -0.01849978044629097, "operator": "ge"}, {"budget": 0.38, "threshold": -0.01984531246125698, "operator": "ge"}, {"budget": 0.39, "threshold": -0.02108384110033512, "operator": "ge"}, {"budget": 0.4, "threshold": -0.02220170944929123, "operator": "ge"}, {"budget": 0.41, "threshold": -0.023270558565855026, "operator": "ge"}, {"budget": 0.42, "threshold": -0.024534108117222786, "operator": "ge"}, {"budget": 0.43, "threshold": -0.02599925734102726, "operator": "ge"}, {"budget": 0.44, "threshold": -0.027368752285838127, "operator": "ge"}, {"budget": 0.45, "threshold": -0.028405025601387024, "operator": "ge"}, {"budget": 0.46, "threshold": -0.02989652007818222, "operator": "ge"}, {"budget": 0.47, "threshold": -0.031635552644729614, "operator": "ge"}, {"budget": 0.48, "threshold": -0.03332666680216789, "operator": "ge"}, {"budget": 0.49, "threshold": -0.03495518118143082, "operator": "ge"}, {"budget": 0.5, "threshold": -0.0365118607878685, "operator": "ge"}, {"budget": 0.51, "threshold": -0.03858334943652153, "operator": "ge"}, {"budget": 0.52, "threshold": -0.04043017700314522, "operator": "ge"}, {"budget": 0.53, "threshold": -0.04213841259479523, "operator": "ge"}, {"budget": 0.54, "threshold": -0.04455908015370369, "operator": "ge"}, {"budget": 0.55, "threshold": -0.04614521935582161, "operator": "ge"}, {"budget": 0.56, "threshold": -0.04794734716415405, "operator": "ge"}, {"budget": 0.57, "threshold": -0.05020992457866669, "operator": "ge"}, {"budget": 0.58, "threshold": -0.05271271988749504, "operator": "ge"}, {"budget": 0.59, "threshold": -0.05438341572880745, "operator": "ge"}, {"budget": 0.6, "threshold": -0.05708564072847366, "operator": "ge"}, {"budget": 0.61, "threshold": -0.05956941470503807, "operator": "ge"}, {"budget": 0.62, "threshold": -0.06205864995718002, "operator": "ge"}, {"budget": 0.63, "threshold": -0.06490637362003326, "operator": "ge"}, {"budget": 0.64, "threshold": -0.06789271533489227, "operator": "ge"}, {"budget": 0.65, "threshold": -0.07091140002012253, "operator": "ge"}, {"budget": 0.66, "threshold": -0.07412830740213394, "operator": "ge"}, {"budget": 0.67, "threshold": -0.07720686495304108, "operator": "ge"}, {"budget": 0.68, "threshold": -0.08033360540866852, "operator": "ge"}, {"budget": 0.69, "threshold": -0.08364588022232056, "operator": "ge"}, {"budget": 0.7, "threshold": -0.08670847862958908, "operator": "ge"}, {"budget": 0.71, "threshold": -0.09098724275827408, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0951983779668808, "operator": "ge"}, {"budget": 0.73, "threshold": -0.09963851422071457, "operator": "ge"}, {"budget": 0.74, "threshold": -0.10368170589208603, "operator": "ge"}, {"budget": 0.75, "threshold": -0.10824595391750336, "operator": "ge"}, {"budget": 0.76, "threshold": -0.11390902101993561, "operator": "ge"}, {"budget": 0.77, "threshold": -0.11891405284404755, "operator": "ge"}, {"budget": 0.78, "threshold": -0.12420386075973511, "operator": "ge"}, {"budget": 0.79, "threshold": -0.1299891471862793, "operator": "ge"}, {"budget": 0.8, "threshold": -0.13677316904067993, "operator": "ge"}, {"budget": 0.81, "threshold": -0.1438128650188446, "operator": "ge"}, {"budget": 0.82, "threshold": -0.15139812231063843, "operator": "ge"}, {"budget": 0.83, "threshold": -0.15799736976623535, "operator": "ge"}, {"budget": 0.84, "threshold": -0.1661277711391449, "operator": "ge"}, {"budget": 0.85, "threshold": -0.17406421899795532, "operator": "ge"}, {"budget": 0.86, "threshold": -0.18181763589382172, "operator": "ge"}, {"budget": 0.87, "threshold": -0.19146943092346191, "operator": "ge"}, {"budget": 0.88, "threshold": -0.20173300802707672, "operator": "ge"}, {"budget": 0.89, "threshold": -0.21220268309116364, "operator": "ge"}, {"budget": 0.9, "threshold": -0.22601337730884552, "operator": "ge"}, {"budget": 0.91, "threshold": -0.24346624314785004, "operator": "ge"}, {"budget": 0.92, "threshold": -0.2587955892086029, "operator": "ge"}, {"budget": 0.93, "threshold": -0.2780372202396393, "operator": "ge"}, {"budget": 0.94, "threshold": -0.3021227717399597, "operator": "ge"}, {"budget": 0.95, "threshold": -0.33348187804222107, "operator": "ge"}, {"budget": 0.96, "threshold": -0.37200450897216797, "operator": "ge"}, {"budget": 0.97, "threshold": -0.4389360547065735, "operator": "ge"}, {"budget": 0.98, "threshold": -0.5497942566871643, "operator": "ge"}, {"budget": 0.99, "threshold": -0.7004361152648926, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.30650243483241, 80.73617874534517, 81.09424233743913, 81.4093382984818, 81.89630478372959, 82.19707820108852, 82.54081924949871, 82.87023775422514, 83.18533371526783, 83.42881695789171, 83.81552563735319, 84.17358922944715, 84.50300773417358, 84.8037811515325, 85.10455456889143, 85.33371526783156, 85.43397307361786, 85.62016614150673, 85.82068175307934, 86.04984245201948, 86.26468060727585, 86.39358350042967, 86.55113148095101, 86.70867946147234, 86.78029217989113, 86.79461472357491, 86.85190489830994, 86.92351761672873, 86.92351761672873, 86.98080779146376, 87.00945287883127, 87.08106559725006, 87.12403322830134, 87.16700085935261, 87.22429103408766, 87.25293612145518, 87.28158120882267, 87.36751647092524, 87.39616155829275, 87.36751647092524, 87.39616155829275, 87.55370953881409, 87.55370953881409, 87.56803208249785, 87.66828988828415, 87.73990260670294, 87.76854769407046, 87.8258378688055, 87.86880549985678, 87.91177313090805, 87.94041821827557, 87.96906330564308, 88.06932111142939, 88.11228874248066, 88.1695789172157, 88.29848181036952, 88.31280435405327, 88.37009452878831, 88.48467487825837, 88.57061014036091, 88.64222285877972, 88.65654540246346, 88.75680320824978, 88.85706101403609, 88.90002864508736, 88.98596390718991, 89.07189916929245, 89.02893153824118, 89.0575766256087, 89.10054425665997, 89.08622171297623, 89.15783443139502, 89.28673732454884, 89.35835004296763, 89.45860784875393, 89.50157547980521, 89.48725293612145, 89.64480091664278, 89.73073617874535, 89.78802635348038, 89.87396161558291, 89.98854196505299, 90.0171870524205, 90.10312231452305, 90.23202520767688, 90.18905757662561, 90.28931538241191, 90.37525064451447, 90.51847608135203, 90.57576625608708, 90.53279862503581, 90.6044113434546, 90.64737897450587, 90.71899169292466, 90.79060441134345, 90.76195932397593, 90.84789458607848, 90.93382984818103, 90.79060441134345, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 34, 65, 92, 115, 150, 173, 200, 225, 250, 271, 299, 326, 350, 374, 398, 419, 434, 449, 468, 486, 504, 520, 531, 543, 549, 551, 555, 560, 562, 566, 568, 573, 578, 582, 589, 591, 594, 600, 603, 604, 609, 622, 623, 626, 634, 641, 646, 653, 658, 662, 667, 672, 681, 688, 693, 702, 707, 713, 723, 730, 738, 741, 749, 758, 764, 774, 784, 788, 795, 802, 809, 818, 828, 839, 850, 860, 870, 886, 897, 904, 915, 926, 935, 947, 961, 968, 981, 994, 1012, 1029, 1036, 1048, 1060, 1076, 1088, 1097, 1114, 1131, 1142, 1154], "harmed": [0, 0, 1, 3, 4, 5, 7, 10, 12, 15, 19, 20, 22, 23, 26, 29, 34, 42, 44, 49, 51, 54, 61, 61, 62, 63, 64, 64, 64, 66, 66, 66, 66, 68, 69, 72, 72, 73, 73, 74, 77, 80, 82, 83, 85, 86, 88, 91, 94, 96, 97, 100, 103, 105, 109, 110, 110, 114, 116, 118, 119, 122, 124, 125, 127, 130, 134, 138, 145, 150, 154, 162, 166, 167, 173, 177, 184, 195, 200, 205, 208, 213, 216, 223, 229, 234, 244, 250, 257, 265, 278, 288, 295, 304, 315, 322, 333, 344, 355, 376, 408], "exact_score": [79.81953594958465, 80.30650243483241, 80.73617874534517, 81.09424233743913, 81.4093382984818, 81.89630478372959, 82.19707820108852, 82.54081924949871, 82.87023775422514, 83.18533371526783, 83.42881695789171, 83.81552563735319, 84.17358922944715, 84.50300773417358, 84.8037811515325, 85.10455456889143, 85.33371526783156, 85.43397307361786, 85.62016614150673, 85.82068175307934, 86.04984245201948, 86.26468060727585, 86.39358350042967, 86.55113148095101, 86.70867946147234, 86.78029217989113, 86.79461472357491, 86.85190489830994, 86.92351761672873, 86.92351761672873, 86.98080779146376, 87.00945287883127, 87.08106559725006, 87.12403322830134, 87.16700085935261, 87.22429103408766, 87.25293612145518, 87.28158120882267, 87.36751647092524, 87.39616155829275, 87.36751647092524, 87.39616155829275, 87.55370953881409, 87.55370953881409, 87.56803208249785, 87.66828988828415, 87.73990260670294, 87.76854769407046, 87.8258378688055, 87.86880549985678, 87.91177313090805, 87.94041821827557, 87.96906330564308, 88.06932111142939, 88.11228874248066, 88.1695789172157, 88.29848181036952, 88.31280435405327, 88.37009452878831, 88.48467487825837, 88.57061014036091, 88.64222285877972, 88.65654540246346, 88.75680320824978, 88.85706101403609, 88.90002864508736, 88.98596390718991, 89.07189916929245, 89.02893153824118, 89.0575766256087, 89.10054425665997, 89.08622171297623, 89.15783443139502, 89.28673732454884, 89.35835004296763, 89.45860784875393, 89.50157547980521, 89.48725293612145, 89.64480091664278, 89.73073617874535, 89.78802635348038, 89.87396161558291, 89.98854196505299, 90.0171870524205, 90.10312231452305, 90.23202520767688, 90.18905757662561, 90.28931538241191, 90.37525064451447, 90.51847608135203, 90.57576625608708, 90.53279862503581, 90.6044113434546, 90.64737897450587, 90.71899169292466, 90.79060441134345, 90.76195932397593, 90.84789458607848, 90.93382984818103, 90.79060441134345, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.56506377532077, "threshold_area100": 87.56430822114007, "threshold_area25": 83.95588656545402}, "test": {"score": [80.00287005811867, 80.49077993829376, 80.95716438257875, 81.35179737389683, 81.71055463873144, 82.09083733945613, 82.52852120255434, 82.93032933916912, 83.2173351510368, 83.56891727057473, 83.79852192006888, 84.09270287723328, 84.27208150965058, 84.68106479156204, 84.95372031283634, 85.19767525292387, 85.4488053383081, 85.69993542369232, 85.83626318432948, 86.03716725263688, 86.15196957738395, 86.33134820980125, 86.50355169692186, 86.67575518404249, 86.79055750878956, 86.81925808997633, 86.8407835258664, 86.86230896175648, 86.89100954294324, 86.92688526942672, 86.97711128650356, 87.01298701298703, 87.0416875941738, 87.06321303006386, 87.07756332065725, 87.13496448303079, 87.17801535481094, 87.19954079070101, 87.24259166248117, 87.26411709837124, 87.25694195307454, 87.26411709837124, 87.35021884193155, 87.41479514960179, 87.4937217478654, 87.53677261964555, 87.63004950850255, 87.69462581617277, 87.70897610676617, 87.74485183324963, 87.80942814091985, 87.81660328621655, 87.91705532037024, 87.96728133744709, 88.10360909808423, 88.15383511516109, 88.17536055105116, 88.25428714931478, 88.27581258520485, 88.36191432876515, 88.43366578173209, 88.51259237999571, 88.59869412355602, 88.67762072181962, 88.75654732008323, 88.84264906364353, 88.90005022601709, 89.02920284135753, 89.09377914902777, 89.23010690966493, 89.251632345555, 89.38078496089547, 89.40231039678554, 89.50993757623593, 89.57451388390615, 89.63909019157639, 89.69649135394992, 89.73954222573008, 89.86151969577384, 89.94044629403746, 89.96197172992754, 90.02654803759776, 90.07677405467462, 90.13417521704815, 90.26332783238861, 90.29202841357538, 90.3566047212456, 90.33507928535553, 90.47858219128938, 90.57185908014638, 90.65796082370669, 90.7297122766736, 90.67231111430006, 90.72253713137691, 90.75123771256368, 90.75841285786038, 90.69383655019014, 90.64361053311329, 90.52163306306953, 90.3637798665423, 90.06242376408122], "calls": [0.0, 0.9973451962402239, 2.1812441701944465, 3.0207361699074404, 4.046781947334434, 4.950850254717658, 5.9840711774413435, 7.010116954868336, 7.964411279328407, 9.0837339456124, 9.88735021884193, 10.89904570567554, 11.752887995981919, 13.022888713496448, 14.335940302791132, 15.22565831958097, 16.208653225227813, 17.177297840281266, 18.04549042118103, 18.94238358326756, 19.86797732654086, 20.958599411638083, 22.164023821482385, 23.068092128865608, 24.065437325105833, 25.321087752026976, 26.24668149530028, 27.093348640309966, 27.975891511803113, 29.238717084020948, 30.537418382722247, 31.333859510655092, 32.32402956159862, 33.213747578388464, 34.168041902848536, 35.22996340675898, 36.557365286647055, 37.51165961110712, 38.50182966205066, 39.37002224295042, 40.28126569563034, 41.02748080648633, 41.953074549759634, 43.04369663485686, 44.019516395207006, 44.77290665135969, 45.806127574083376, 46.83934849680706, 47.81516825715721, 48.61160938509005, 49.479801965989815, 50.599124632273806, 51.68257157207433, 52.42161153763364, 53.770538853411786, 54.48087823778431, 55.29166965631054, 56.18138767310038, 57.34376121116453, 58.02540001435029, 59.15189782593097, 60.142067876874506, 61.038961038961034, 62.24438544880534, 63.3852335509794, 64.48303078137332, 65.66692975532754, 66.64274951567769, 67.57551840424767, 68.5800387457846, 69.39083016431083, 70.69670660830883, 71.83037956518619, 72.9353519408768, 73.89682141063356, 74.91569204276387, 76.0995910167181, 77.03953505058477, 77.97947908445147, 78.71851905001076, 79.78044055392121, 80.89258807490852, 82.03343617708259, 83.08100739039965, 84.10705316782665, 85.01112147520988, 85.77168687665925, 86.69010547463586, 87.7735524144364, 88.70632130300639, 89.71801678984, 90.88756547320084, 91.6553060199469, 92.5808997632202, 93.64282126713066, 94.60429073688742, 95.70926311257803, 96.95056324890578, 98.2420894023104, 99.1676831455837, 100.0], "rescued": [0, 69, 143, 200, 258, 312, 376, 435, 484, 541, 579, 627, 655, 718, 766, 805, 846, 883, 908, 942, 970, 1003, 1034, 1063, 1081, 1087, 1094, 1099, 1103, 1110, 1118, 1123, 1129, 1135, 1139, 1149, 1158, 1164, 1172, 1177, 1180, 1186, 1198, 1209, 1222, 1232, 1248, 1260, 1268, 1275, 1286, 1295, 1314, 1323, 1349, 1359, 1367, 1379, 1386, 1398, 1414, 1428, 1445, 1465, 1482, 1501, 1519, 1544, 1564, 1587, 1596, 1620, 1629, 1645, 1660, 1682, 1702, 1720, 1745, 1762, 1777, 1797, 1820, 1840, 1868, 1887, 1907, 1922, 1954, 1981, 2010, 2037, 2048, 2072, 2096, 2122, 2150, 2181, 2208, 2228, 2247], "harmed": [0, 1, 10, 12, 20, 21, 24, 27, 36, 44, 50, 57, 60, 66, 76, 81, 87, 89, 95, 101, 113, 121, 128, 133, 135, 137, 141, 143, 143, 145, 146, 146, 148, 151, 153, 155, 158, 161, 163, 165, 169, 174, 174, 176, 178, 182, 185, 188, 194, 196, 198, 206, 211, 213, 220, 223, 228, 229, 233, 233, 239, 242, 247, 256, 262, 269, 279, 286, 297, 301, 307, 313, 319, 320, 326, 339, 351, 363, 371, 377, 389, 400, 416, 428, 438, 453, 464, 482, 494, 508, 525, 542, 561, 578, 598, 623, 660, 698, 742, 784, 845], "exact_score": [80.00287005811867, 80.49077993829376, 80.89258807490852, 81.34462222860013, 81.69620434813805, 82.09801248475283, 82.52852120255434, 82.91597904857574, 83.2245102963335, 83.56174212527804, 83.79852192006888, 84.12140345842005, 84.3510081079142, 84.67388964626534, 84.8963191504628, 85.11157350936357, 85.39140417593457, 85.62100882542872, 85.83626318432948, 86.03716725263688, 86.15196957738395, 86.34569850039465, 86.48920140632848, 86.66140489344909, 86.78338236349286, 86.81925808997633, 86.8407835258664, 86.86230896175648, 86.89100954294324, 86.92688526942672, 86.94841070531677, 86.99146157709694, 87.0416875941738, 87.05603788476716, 87.07038817536056, 87.1134390471407, 87.17801535481094, 87.18519050010764, 87.22106622659109, 87.25694195307454, 87.26411709837124, 87.26411709837124, 87.35021884193155, 87.41479514960179, 87.4937217478654, 87.56547320083232, 87.63722465379924, 87.69462581617277, 87.70897610676617, 87.78790270502978, 87.80942814091985, 87.86682930329339, 87.92423046566694, 88.03185764511731, 88.11795938867762, 88.16101026045777, 88.25428714931478, 88.27581258520485, 88.35473918346847, 88.44084092702877, 88.51259237999571, 88.59151897825932, 88.63456985003947, 88.69914615770969, 88.81394848245677, 88.87134964483032, 88.92875080720385, 89.07942885843438, 89.158355456698, 89.24445720025831, 89.344909234412, 89.38078496089547, 89.40948554208224, 89.51711272153261, 89.57451388390615, 89.63191504627969, 89.67496591805984, 89.73954222573008, 89.86869484107054, 89.96914687522423, 89.95479658463084, 90.02654803759776, 90.05524861878453, 90.14135036234484, 90.24897754179523, 90.29202841357538, 90.3637798665423, 90.37095501183899, 90.48575733658608, 90.60055966133315, 90.68666140489346, 90.71536198608023, 90.68666140489346, 90.744062567267, 90.73688742197031, 90.77993829375045, 90.65796082370669, 90.6364353878166, 90.57185908014638, 90.41400588361914, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.61728050934771, "threshold_area100": 87.60895458133027, "threshold_area25": 84.13001363277608}, "selected_budget": 98, "selected": {"val": {"score": 90.93382984818103, "calls": 97.99484388427385, "rescued": 1131, "harmed": 355}, "test": {"score": 90.52163306306953, "calls": 98.2420894023104, "rescued": 2208, "harmed": 742}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 4.580524444580078, "operator": "ge"}, {"budget": 0.02, "threshold": 3.843111515045166, "operator": "ge"}, {"budget": 0.03, "threshold": 3.408071517944336, "operator": "ge"}, {"budget": 0.04, "threshold": 2.957204818725586, "operator": "ge"}, {"budget": 0.05, "threshold": 2.715981960296631, "operator": "ge"}, {"budget": 0.06, "threshold": 2.4690394401550293, "operator": "ge"}, {"budget": 0.07, "threshold": 2.215864419937134, "operator": "ge"}, {"budget": 0.08, "threshold": 1.9262912273406982, "operator": "ge"}, {"budget": 0.09, "threshold": 1.7544564008712769, "operator": "ge"}, {"budget": 0.1, "threshold": 1.5855249166488647, "operator": "ge"}, {"budget": 0.11, "threshold": 1.4236533641815186, "operator": "ge"}, {"budget": 0.12, "threshold": 1.2618485689163208, "operator": "ge"}, {"budget": 0.13, "threshold": 1.1372792720794678, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9987629652023315, "operator": "ge"}, {"budget": 0.15, "threshold": 0.8654505610466003, "operator": "ge"}, {"budget": 0.16, "threshold": 0.7485981583595276, "operator": "ge"}, {"budget": 0.17, "threshold": 0.6199667453765869, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5167773365974426, "operator": "ge"}, {"budget": 0.19, "threshold": 0.4041445851325989, "operator": "ge"}, {"budget": 0.2, "threshold": 0.29451316595077515, "operator": "ge"}, {"budget": 0.21, "threshold": 0.19520536065101624, "operator": "ge"}, {"budget": 0.22, "threshold": 0.09742063283920288, "operator": "ge"}, {"budget": 0.23, "threshold": 0.005463816225528717, "operator": "ge"}, {"budget": 0.24, "threshold": -0.10349706560373306, "operator": "ge"}, {"budget": 0.25, "threshold": -0.21261483430862427, "operator": "ge"}, {"budget": 0.26, "threshold": -0.30486661195755005, "operator": "ge"}, {"budget": 0.27, "threshold": -0.38872021436691284, "operator": "ge"}, {"budget": 0.28, "threshold": -0.49401718378067017, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6201298236846924, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6904577612876892, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8016340732574463, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9081307053565979, "operator": "ge"}, {"budget": 0.33, "threshold": -1.0046223402023315, "operator": "ge"}, {"budget": 0.34, "threshold": -1.1137125492095947, "operator": "ge"}, {"budget": 0.35, "threshold": -1.2019104957580566, "operator": "ge"}, {"budget": 0.36, "threshold": -1.2895660400390625, "operator": "ge"}, {"budget": 0.37, "threshold": -1.3920297622680664, "operator": "ge"}, {"budget": 0.38, "threshold": -1.4772882461547852, "operator": "ge"}, {"budget": 0.39, "threshold": -1.5745869874954224, "operator": "ge"}, {"budget": 0.4, "threshold": -1.678564190864563, "operator": "ge"}, {"budget": 0.41, "threshold": -1.7614953517913818, "operator": "ge"}, {"budget": 0.42, "threshold": -1.8748581409454346, "operator": "ge"}, {"budget": 0.43, "threshold": -1.9512478113174438, "operator": "ge"}, {"budget": 0.44, "threshold": -2.04483699798584, "operator": "ge"}, {"budget": 0.45, "threshold": -2.122375965118408, "operator": "ge"}, {"budget": 0.46, "threshold": -2.211337089538574, "operator": "ge"}, {"budget": 0.47, "threshold": -2.3277385234832764, "operator": "ge"}, {"budget": 0.48, "threshold": -2.4291679859161377, "operator": "ge"}, {"budget": 0.49, "threshold": -2.5493359565734863, "operator": "ge"}, {"budget": 0.5, "threshold": -2.6546425819396973, "operator": "ge"}, {"budget": 0.51, "threshold": -2.751892566680908, "operator": "ge"}, {"budget": 0.52, "threshold": -2.845564842224121, "operator": "ge"}, {"budget": 0.53, "threshold": -2.9366531372070312, "operator": "ge"}, {"budget": 0.54, "threshold": -3.0520148277282715, "operator": "ge"}, {"budget": 0.55, "threshold": -3.156062126159668, "operator": "ge"}, {"budget": 0.56, "threshold": -3.264482259750366, "operator": "ge"}, {"budget": 0.57, "threshold": -3.3620848655700684, "operator": "ge"}, {"budget": 0.58, "threshold": -3.4702329635620117, "operator": "ge"}, {"budget": 0.59, "threshold": -3.5751776695251465, "operator": "ge"}, {"budget": 0.6, "threshold": -3.6804966926574707, "operator": "ge"}, {"budget": 0.61, "threshold": -3.798738718032837, "operator": "ge"}, {"budget": 0.62, "threshold": -3.9183921813964844, "operator": "ge"}, {"budget": 0.63, "threshold": -3.9997036457061768, "operator": "ge"}, {"budget": 0.64, "threshold": -4.096379280090332, "operator": "ge"}, {"budget": 0.65, "threshold": -4.220254898071289, "operator": "ge"}, {"budget": 0.66, "threshold": -4.322229385375977, "operator": "ge"}, {"budget": 0.67, "threshold": -4.417308807373047, "operator": "ge"}, {"budget": 0.68, "threshold": -4.531821250915527, "operator": "ge"}, {"budget": 0.69, "threshold": -4.643789291381836, "operator": "ge"}, {"budget": 0.7, "threshold": -4.7427978515625, "operator": "ge"}, {"budget": 0.71, "threshold": -4.850761890411377, "operator": "ge"}, {"budget": 0.72, "threshold": -4.97305965423584, "operator": "ge"}, {"budget": 0.73, "threshold": -5.093940258026123, "operator": "ge"}, {"budget": 0.74, "threshold": -5.187809944152832, "operator": "ge"}, {"budget": 0.75, "threshold": -5.290140628814697, "operator": "ge"}, {"budget": 0.76, "threshold": -5.402032375335693, "operator": "ge"}, {"budget": 0.77, "threshold": -5.494227409362793, "operator": "ge"}, {"budget": 0.78, "threshold": -5.602240085601807, "operator": "ge"}, {"budget": 0.79, "threshold": -5.698952674865723, "operator": "ge"}, {"budget": 0.8, "threshold": -5.823517322540283, "operator": "ge"}, {"budget": 0.81, "threshold": -5.951419353485107, "operator": "ge"}, {"budget": 0.82, "threshold": -6.074920654296875, "operator": "ge"}, {"budget": 0.83, "threshold": -6.180683135986328, "operator": "ge"}, {"budget": 0.84, "threshold": -6.308517932891846, "operator": "ge"}, {"budget": 0.85, "threshold": -6.422390937805176, "operator": "ge"}, {"budget": 0.86, "threshold": -6.575046062469482, "operator": "ge"}, {"budget": 0.87, "threshold": -6.708390712738037, "operator": "ge"}, {"budget": 0.88, "threshold": -6.827009201049805, "operator": "ge"}, {"budget": 0.89, "threshold": -6.990633487701416, "operator": "ge"}, {"budget": 0.9, "threshold": -7.144850254058838, "operator": "ge"}, {"budget": 0.91, "threshold": -7.313961029052734, "operator": "ge"}, {"budget": 0.92, "threshold": -7.490289688110352, "operator": "ge"}, {"budget": 0.93, "threshold": -7.6794281005859375, "operator": "ge"}, {"budget": 0.94, "threshold": -7.902985572814941, "operator": "ge"}, {"budget": 0.95, "threshold": -8.199743270874023, "operator": "ge"}, {"budget": 0.96, "threshold": -8.480213165283203, "operator": "ge"}, {"budget": 0.97, "threshold": -8.876498222351074, "operator": "ge"}, {"budget": 0.98, "threshold": -9.393453598022461, "operator": "ge"}, {"budget": 0.99, "threshold": -10.309208869934082, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.23488971641363, 80.59295330850759, 80.97966198796905, 81.39501575479805, 81.6528215411057, 82.03953022056717, 82.4405614437124, 82.82727012317388, 83.14236608421656, 83.55771985104555, 83.81552563735319, 84.05900887997709, 84.33113720996849, 84.5316528215411, 84.81810369521627, 85.10455456889143, 85.43397307361786, 85.67745631624176, 85.89229447149813, 86.06416499570322, 86.27900315095961, 86.36493841306215, 86.55113148095101, 86.83758235462618, 86.98080779146376, 87.16700085935261, 87.32454883987396, 87.45345173302778, 87.61099971354912, 87.76854769407046, 87.86880549985678, 87.96906330564308, 88.18390146089945, 88.35577198510455, 88.4417072472071, 88.55628759667717, 88.72815812088226, 88.8857061014036, 88.98596390718991, 89.10054425665997, 89.28673732454884, 89.41564021770266, 89.51589802348897, 89.60183328559151, 89.65912346032655, 89.68776854769406, 89.85963907189917, 90.04583213978802, 90.11744485820681, 90.18905757662561, 90.16041248925809, 90.20338012030936, 90.18905757662561, 90.16041248925809, 90.18905757662561, 90.18905757662561, 90.30363792609567, 90.31796046977942, 90.38957318819823, 90.44686336293324, 90.46118590661702, 90.41821827556574, 90.44686336293324, 90.44686336293324, 90.51847608135203, 90.53279862503581, 90.50415353766829, 90.54712116871956, 90.6044113434546, 90.63305643082211, 90.66170151818963, 90.71899169292466, 90.73331423660842, 90.77628186765969, 90.77628186765969, 90.77628186765969, 90.79060441134345, 90.79060441134345, 90.86221712976223, 90.83357204239474, 90.89086221712975, 90.93382984818103, 90.93382984818103, 90.90518476081351, 90.90518476081351, 90.90518476081351, 90.876539673446, 90.80492695502721, 90.74763678029217, 90.76195932397593, 90.73331423660842, 90.7046691492409, 90.69034660555714, 90.64737897450587, 90.63305643082211, 90.64737897450587, 90.63305643082211, 90.59008879977084, 90.59008879977084, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 29, 55, 83, 113, 132, 161, 189, 220, 244, 274, 292, 311, 333, 353, 375, 395, 420, 440, 460, 478, 500, 514, 531, 554, 575, 592, 610, 626, 642, 658, 672, 687, 709, 726, 737, 751, 768, 782, 793, 809, 823, 837, 849, 858, 868, 879, 895, 911, 922, 932, 940, 949, 953, 957, 964, 968, 980, 988, 996, 1002, 1006, 1009, 1015, 1021, 1029, 1033, 1038, 1042, 1049, 1056, 1059, 1068, 1072, 1078, 1083, 1087, 1090, 1093, 1099, 1102, 1108, 1114, 1116, 1120, 1123, 1127, 1133, 1133, 1134, 1137, 1138, 1141, 1141, 1142, 1146, 1149, 1150, 1151, 1154, 1154], "harmed": [0, 0, 1, 2, 3, 4, 6, 6, 10, 12, 13, 13, 15, 18, 24, 26, 26, 28, 31, 36, 42, 49, 57, 61, 64, 75, 79, 86, 93, 98, 103, 110, 118, 125, 130, 135, 141, 146, 149, 153, 161, 162, 167, 172, 175, 181, 190, 194, 197, 203, 208, 218, 224, 229, 235, 240, 244, 248, 255, 258, 260, 263, 269, 273, 279, 282, 285, 292, 293, 296, 301, 302, 307, 310, 313, 318, 322, 324, 327, 328, 333, 335, 338, 340, 346, 349, 353, 361, 366, 371, 373, 376, 381, 382, 386, 391, 393, 395, 399, 402, 408], "exact_score": [79.81953594958465, 80.23488971641363, 80.59295330850759, 80.97966198796905, 81.39501575479805, 81.6528215411057, 82.03953022056717, 82.4405614437124, 82.82727012317388, 83.14236608421656, 83.55771985104555, 83.81552563735319, 84.05900887997709, 84.33113720996849, 84.5316528215411, 84.81810369521627, 85.10455456889143, 85.43397307361786, 85.67745631624176, 85.89229447149813, 86.06416499570322, 86.27900315095961, 86.36493841306215, 86.55113148095101, 86.83758235462618, 86.98080779146376, 87.16700085935261, 87.32454883987396, 87.45345173302778, 87.61099971354912, 87.76854769407046, 87.86880549985678, 87.96906330564308, 88.18390146089945, 88.35577198510455, 88.4417072472071, 88.55628759667717, 88.72815812088226, 88.8857061014036, 88.98596390718991, 89.10054425665997, 89.28673732454884, 89.41564021770266, 89.51589802348897, 89.60183328559151, 89.65912346032655, 89.68776854769406, 89.85963907189917, 90.04583213978802, 90.11744485820681, 90.18905757662561, 90.16041248925809, 90.20338012030936, 90.18905757662561, 90.16041248925809, 90.18905757662561, 90.18905757662561, 90.30363792609567, 90.31796046977942, 90.38957318819823, 90.44686336293324, 90.46118590661702, 90.41821827556574, 90.44686336293324, 90.44686336293324, 90.51847608135203, 90.53279862503581, 90.50415353766829, 90.54712116871956, 90.6044113434546, 90.63305643082211, 90.66170151818963, 90.71899169292466, 90.73331423660842, 90.77628186765969, 90.77628186765969, 90.77628186765969, 90.79060441134345, 90.79060441134345, 90.86221712976223, 90.83357204239474, 90.89086221712975, 90.93382984818103, 90.93382984818103, 90.90518476081351, 90.90518476081351, 90.90518476081351, 90.876539673446, 90.80492695502721, 90.74763678029217, 90.76195932397593, 90.73331423660842, 90.7046691492409, 90.69034660555714, 90.64737897450587, 90.63305643082211, 90.64737897450587, 90.63305643082211, 90.59008879977084, 90.59008879977084, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.46540036945679, "threshold_area100": 88.46462331710111, "threshold_area25": 83.89859639071899}, "test": {"score": [80.00287005811867, 80.53383081007391, 81.0073903996556, 81.35897251919351, 81.97603501470905, 82.26304082657674, 82.52134605725766, 82.90880390327905, 83.31061203989381, 83.56174212527804, 83.71959532180527, 83.95637511659612, 84.25773121905719, 84.4299347061778, 84.68823993685872, 84.9895960393198, 85.13309894525365, 85.51338164597834, 85.74298629547249, 86.05869268852695, 86.26677190213103, 86.41744995336155, 86.51790198751526, 86.69010547463587, 86.8479586711631, 87.07038817536056, 87.26411709837124, 87.45784602138194, 87.69462581617277, 87.86682930329339, 88.01750735452394, 88.13230967927102, 88.36908947406185, 88.49824208940231, 88.63456985003947, 88.76372246537993, 89.00050226017078, 89.08660400373107, 89.19423118318146, 89.33055894381862, 89.44536126856569, 89.47406184975246, 89.59603931979623, 89.65344048216977, 89.83281911458707, 89.8902202769606, 89.91892085814737, 89.997847456411, 89.997847456411, 90.13417521704815, 90.22027696060846, 90.20592667001507, 90.24180239649853, 90.277678122982, 90.29202841357538, 90.34942957594892, 90.33507928535553, 90.3566047212456, 90.40683073832245, 90.43553131950922, 90.47858219128938, 90.49293248188276, 90.46423190069599, 90.47140704599268, 90.45705675539931, 90.48575733658608, 90.43553131950922, 90.37813015713569, 90.37813015713569, 90.3566047212456, 90.3566047212456, 90.32072899476215, 90.3566047212456, 90.3637798665423, 90.33507928535553, 90.3566047212456, 90.34225443065222, 90.34225443065222, 90.3637798665423, 90.3566047212456, 90.3566047212456, 90.38530530243239, 90.38530530243239, 90.37095501183899, 90.3566047212456, 90.34225443065222, 90.29920355887207, 90.24897754179523, 90.23462725120184, 90.24897754179523, 90.27050297768531, 90.25615268709191, 90.25615268709191, 90.23462725120184, 90.21310181531177, 90.22745210590514, 90.24180239649853, 90.24180239649853, 90.1772260888283, 90.09112434526799, 90.06242376408122], "calls": [0.0, 1.2125995551409916, 2.3103967855349072, 3.214465092918132, 4.778646767597044, 5.517686733156346, 6.371529023462726, 7.311473057329411, 8.63887493721748, 9.442491210447011, 10.339384372533544, 11.329554423477076, 12.45605223505776, 13.338595106550907, 14.307239721604361, 15.469613259668508, 16.323455549974884, 17.41407763507211, 18.432948267202413, 19.430293463442634, 20.42763865968286, 21.396283274736312, 22.28600129152615, 23.125493291239145, 24.151539068666143, 25.278036880246823, 26.354308674750666, 27.16510009327689, 28.27724761426419, 29.511372605295254, 30.135610246107486, 31.26928320298486, 32.33120470689531, 33.450527373179305, 34.591375475353374, 35.373466312692834, 36.3349357824496, 37.28205496161297, 38.26504986725981, 39.26239506350004, 40.439118892157566, 41.22838487479372, 42.50556073760494, 43.35222788261463, 44.313697352371385, 45.052737317930685, 45.92092989883045, 46.918275095070676, 47.965846308387746, 49.286073042979126, 50.14709047858219, 51.07985936715218, 52.04132883690895, 52.99562316136902, 54.100595537059625, 55.11946616918992, 56.21008825428715, 57.0854559804836, 58.29805553562459, 59.39585276601851, 60.35014709047858, 61.620147807993106, 62.84709765372749, 63.643538781660325, 64.66240941379063, 65.86065867833824, 66.87952931046853, 67.76207218196168, 68.97467173710268, 70.09399440338667, 71.00523785606659, 72.03845877879027, 72.96405252206357, 74.11925091483103, 75.03766951280764, 75.95608811078425, 77.00365932410132, 77.71399870847384, 78.73286934060415, 79.6082370668006, 80.60558226304083, 81.77513094640166, 82.75095070675181, 83.58326756116811, 84.45863528736457, 85.24790127000071, 86.30982277391116, 87.24259166248116, 88.02468249982061, 88.90005022601709, 89.85434455047715, 90.88039032790414, 91.8490349429576, 92.9611824639449, 94.10920571141565, 95.10655090765589, 95.98909377914903, 97.05101528305948, 98.15598765875009, 99.27531032503408, 100.0], "rescued": [0, 77, 144, 198, 291, 333, 373, 428, 489, 525, 555, 593, 639, 668, 711, 760, 786, 846, 883, 933, 973, 1005, 1032, 1065, 1102, 1142, 1176, 1206, 1249, 1290, 1313, 1343, 1384, 1416, 1447, 1475, 1515, 1540, 1565, 1589, 1616, 1633, 1664, 1684, 1716, 1729, 1742, 1768, 1780, 1813, 1833, 1845, 1856, 1870, 1885, 1903, 1916, 1928, 1945, 1960, 1973, 1989, 1998, 2007, 2013, 2031, 2039, 2045, 2054, 2061, 2067, 2073, 2084, 2095, 2100, 2110, 2119, 2124, 2131, 2134, 2140, 2150, 2155, 2160, 2165, 2171, 2176, 2179, 2183, 2189, 2200, 2205, 2207, 2212, 2217, 2226, 2233, 2241, 2243, 2245, 2247], "harmed": [0, 3, 4, 9, 16, 18, 22, 23, 28, 29, 37, 42, 46, 51, 58, 65, 71, 78, 83, 89, 100, 111, 124, 133, 148, 157, 164, 167, 177, 194, 196, 210, 218, 232, 244, 254, 261, 274, 284, 289, 300, 313, 327, 339, 346, 351, 360, 375, 387, 401, 409, 423, 429, 438, 451, 461, 476, 485, 495, 506, 513, 527, 540, 548, 556, 570, 585, 599, 608, 618, 624, 635, 641, 651, 660, 667, 678, 683, 687, 691, 697, 703, 708, 715, 722, 730, 741, 751, 757, 761, 769, 776, 778, 786, 794, 801, 806, 814, 825, 839, 845], "exact_score": [80.00287005811867, 80.46925450240367, 80.89258807490852, 81.28722106622659, 81.68902920284135, 82.05496161297266, 82.3993685872139, 82.78682643323528, 83.08100739039966, 83.40388892875082, 83.70524503121189, 83.84874793714573, 84.11422831312333, 84.36535839850757, 84.60931333859511, 84.86761856927603, 85.0828729281768, 85.34117815885772, 85.66405969720887, 85.87213891081295, 86.21654588505417, 86.35287364569133, 86.49637655162518, 86.6542297481524, 86.81925808997633, 86.98428643180026, 87.2067159359977, 87.3932697137117, 87.5941737820191, 87.78072755973308, 88.00315706393054, 88.11795938867762, 88.30451316639162, 88.48389179880894, 88.6058692688527, 88.72784673889646, 88.93592595250054, 89.065078567841, 89.16553060199469, 89.28033292674176, 89.40948554208224, 89.438186123269, 89.531463012126, 89.63191504627969, 89.78259309751023, 89.88304513166392, 89.93327114874077, 89.997847456411, 89.997847456411, 90.0982994905647, 90.19157637942169, 90.21310181531177, 90.22745210590514, 90.277678122982, 90.28485326827868, 90.33507928535553, 90.32072899476215, 90.34942957594892, 90.40683073832245, 90.45705675539931, 90.48575733658608, 90.52163306306953, 90.50010762717945, 90.44988161010261, 90.47140704599268, 90.46423190069599, 90.47858219128938, 90.43553131950922, 90.37813015713569, 90.37813015713569, 90.3566047212456, 90.34942957594892, 90.32072899476215, 90.3637798665423, 90.34225443065222, 90.33507928535553, 90.3566047212456, 90.34225443065222, 90.3637798665423, 90.3637798665423, 90.3566047212456, 90.37095501183899, 90.38530530243239, 90.37813015713569, 90.37813015713569, 90.33507928535553, 90.34942957594892, 90.27050297768531, 90.23462725120184, 90.24180239649853, 90.28485326827868, 90.25615268709191, 90.24897754179523, 90.23462725120184, 90.19875152471838, 90.22745210590514, 90.24180239649853, 90.24180239649853, 90.19157637942169, 90.11264978115807, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.41156065052708, "threshold_area100": 88.44478725694196, "threshold_area25": 84.1551266413145}, "selected_budget": 82, "selected": {"val": {"score": 90.93382984818103, "calls": 81.9965625895159, "rescued": 1114, "harmed": 338}, "test": {"score": 90.38530530243239, "calls": 82.75095070675181, "rescued": 2155, "harmed": 708}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.7545604705810547, "operator": "ge"}, {"budget": 0.02, "threshold": 2.2942817211151123, "operator": "ge"}, {"budget": 0.03, "threshold": 2.0267910957336426, "operator": "ge"}, {"budget": 0.04, "threshold": 1.7965703010559082, "operator": "ge"}, {"budget": 0.05, "threshold": 1.620848536491394, "operator": "ge"}, {"budget": 0.06, "threshold": 1.4889161586761475, "operator": "ge"}, {"budget": 0.07, "threshold": 1.3666677474975586, "operator": "ge"}, {"budget": 0.08, "threshold": 1.2485713958740234, "operator": "ge"}, {"budget": 0.09, "threshold": 1.1481890678405762, "operator": "ge"}, {"budget": 0.1, "threshold": 1.03910231590271, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9339478015899658, "operator": "ge"}, {"budget": 0.12, "threshold": 0.8390746116638184, "operator": "ge"}, {"budget": 0.13, "threshold": 0.7563127279281616, "operator": "ge"}, {"budget": 0.14, "threshold": 0.6837930083274841, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5986034274101257, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5241484045982361, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4355432093143463, "operator": "ge"}, {"budget": 0.18, "threshold": 0.36528444290161133, "operator": "ge"}, {"budget": 0.19, "threshold": 0.28898119926452637, "operator": "ge"}, {"budget": 0.2, "threshold": 0.22408640384674072, "operator": "ge"}, {"budget": 0.21, "threshold": 0.15678542852401733, "operator": "ge"}, {"budget": 0.22, "threshold": 0.07874858379364014, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0045713260769844055, "operator": "ge"}, {"budget": 0.24, "threshold": -0.08323518186807632, "operator": "ge"}, {"budget": 0.25, "threshold": -0.1803237497806549, "operator": "ge"}, {"budget": 0.26, "threshold": -0.2612808346748352, "operator": "ge"}, {"budget": 0.27, "threshold": -0.331659197807312, "operator": "ge"}, {"budget": 0.28, "threshold": -0.4234887361526489, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5214870572090149, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6134412288665771, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6846674084663391, "operator": "ge"}, {"budget": 0.32, "threshold": -0.8023872375488281, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8957070708274841, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9958529472351074, "operator": "ge"}, {"budget": 0.35, "threshold": -1.0891797542572021, "operator": "ge"}, {"budget": 0.36, "threshold": -1.174943208694458, "operator": "ge"}, {"budget": 0.37, "threshold": -1.2636284828186035, "operator": "ge"}, {"budget": 0.38, "threshold": -1.3571486473083496, "operator": "ge"}, {"budget": 0.39, "threshold": -1.4441026449203491, "operator": "ge"}, {"budget": 0.4, "threshold": -1.5283911228179932, "operator": "ge"}, {"budget": 0.41, "threshold": -1.6347336769104004, "operator": "ge"}, {"budget": 0.42, "threshold": -1.7323458194732666, "operator": "ge"}, {"budget": 0.43, "threshold": -1.842402458190918, "operator": "ge"}, {"budget": 0.44, "threshold": -1.9167336225509644, "operator": "ge"}, {"budget": 0.45, "threshold": -1.9972645044326782, "operator": "ge"}, {"budget": 0.46, "threshold": -2.101726531982422, "operator": "ge"}, {"budget": 0.47, "threshold": -2.20503568649292, "operator": "ge"}, {"budget": 0.48, "threshold": -2.310476064682007, "operator": "ge"}, {"budget": 0.49, "threshold": -2.4101381301879883, "operator": "ge"}, {"budget": 0.5, "threshold": -2.5079452991485596, "operator": "ge"}, {"budget": 0.51, "threshold": -2.605238914489746, "operator": "ge"}, {"budget": 0.52, "threshold": -2.7165095806121826, "operator": "ge"}, {"budget": 0.53, "threshold": -2.809737205505371, "operator": "ge"}, {"budget": 0.54, "threshold": -2.912280559539795, "operator": "ge"}, {"budget": 0.55, "threshold": -3.0187454223632812, "operator": "ge"}, {"budget": 0.56, "threshold": -3.1281070709228516, "operator": "ge"}, {"budget": 0.57, "threshold": -3.23720383644104, "operator": "ge"}, {"budget": 0.58, "threshold": -3.3393919467926025, "operator": "ge"}, {"budget": 0.59, "threshold": -3.4504454135894775, "operator": "ge"}, {"budget": 0.6, "threshold": -3.5529236793518066, "operator": "ge"}, {"budget": 0.61, "threshold": -3.6584434509277344, "operator": "ge"}, {"budget": 0.62, "threshold": -3.7866902351379395, "operator": "ge"}, {"budget": 0.63, "threshold": -3.877739667892456, "operator": "ge"}, {"budget": 0.64, "threshold": -3.9823100566864014, "operator": "ge"}, {"budget": 0.65, "threshold": -4.102382659912109, "operator": "ge"}, {"budget": 0.66, "threshold": -4.210604667663574, "operator": "ge"}, {"budget": 0.67, "threshold": -4.291871070861816, "operator": "ge"}, {"budget": 0.68, "threshold": -4.395419597625732, "operator": "ge"}, {"budget": 0.69, "threshold": -4.519103527069092, "operator": "ge"}, {"budget": 0.7, "threshold": -4.627143859863281, "operator": "ge"}, {"budget": 0.71, "threshold": -4.71861457824707, "operator": "ge"}, {"budget": 0.72, "threshold": -4.845953941345215, "operator": "ge"}, {"budget": 0.73, "threshold": -4.959919452667236, "operator": "ge"}, {"budget": 0.74, "threshold": -5.084787845611572, "operator": "ge"}, {"budget": 0.75, "threshold": -5.174656391143799, "operator": "ge"}, {"budget": 0.76, "threshold": -5.268582344055176, "operator": "ge"}, {"budget": 0.77, "threshold": -5.387368679046631, "operator": "ge"}, {"budget": 0.78, "threshold": -5.493995189666748, "operator": "ge"}, {"budget": 0.79, "threshold": -5.600886344909668, "operator": "ge"}, {"budget": 0.8, "threshold": -5.696295261383057, "operator": "ge"}, {"budget": 0.81, "threshold": -5.842170238494873, "operator": "ge"}, {"budget": 0.82, "threshold": -5.955549716949463, "operator": "ge"}, {"budget": 0.83, "threshold": -6.0938401222229, "operator": "ge"}, {"budget": 0.84, "threshold": -6.213283538818359, "operator": "ge"}, {"budget": 0.85, "threshold": -6.333283424377441, "operator": "ge"}, {"budget": 0.86, "threshold": -6.4668707847595215, "operator": "ge"}, {"budget": 0.87, "threshold": -6.615630626678467, "operator": "ge"}, {"budget": 0.88, "threshold": -6.737789630889893, "operator": "ge"}, {"budget": 0.89, "threshold": -6.8900041580200195, "operator": "ge"}, {"budget": 0.9, "threshold": -7.059760093688965, "operator": "ge"}, {"budget": 0.91, "threshold": -7.234053611755371, "operator": "ge"}, {"budget": 0.92, "threshold": -7.4222612380981445, "operator": "ge"}, {"budget": 0.93, "threshold": -7.617498874664307, "operator": "ge"}, {"budget": 0.94, "threshold": -7.847224712371826, "operator": "ge"}, {"budget": 0.95, "threshold": -8.13821792602539, "operator": "ge"}, {"budget": 0.96, "threshold": -8.414820671081543, "operator": "ge"}, {"budget": 0.97, "threshold": -8.808725357055664, "operator": "ge"}, {"budget": 0.98, "threshold": -9.381540298461914, "operator": "ge"}, {"budget": 0.99, "threshold": -10.238024711608887, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.29217989114866, 80.76482383271268, 81.15153251217416, 81.63849899742193, 81.93927241478086, 82.31165855055858, 82.58378688054998, 82.91320538527643, 83.21397880263535, 83.51475221999426, 83.7439129189344, 84.00171870524204, 84.27384703523344, 84.64623317101116, 84.91836150100256, 85.1475221999427, 85.44829561730162, 85.63448868519048, 85.89229447149813, 86.15010025780579, 86.27900315095961, 86.37926095674591, 86.55113148095101, 86.83758235462618, 86.98080779146376, 87.19564594672013, 87.36751647092524, 87.4820968203953, 87.72558006301918, 87.81151532512173, 87.854482956173, 88.05499856774563, 88.1695789172157, 88.35577198510455, 88.41306215983958, 88.52764250930964, 88.67086794614723, 88.84273847035233, 88.91435118877112, 89.04325408192494, 89.21512460613005, 89.34402749928387, 89.53022056717272, 89.573188198224, 89.64480091664278, 89.70209109137782, 89.77370380979661, 90.0171870524205, 90.0887997708393, 90.16041248925809, 90.18905757662561, 90.18905757662561, 90.23202520767688, 90.18905757662561, 90.21770266399312, 90.28931538241191, 90.26067029504439, 90.34660555714694, 90.40389573188197, 90.38957318819823, 90.41821827556574, 90.48983099398453, 90.48983099398453, 90.48983099398453, 90.50415353766829, 90.56144371240332, 90.53279862503581, 90.56144371240332, 90.56144371240332, 90.63305643082211, 90.64737897450587, 90.74763678029217, 90.79060441134345, 90.76195932397593, 90.80492695502721, 90.76195932397593, 90.73331423660842, 90.81924949871096, 90.89086221712975, 90.90518476081351, 90.876539673446, 90.93382984818103, 90.90518476081351, 90.89086221712975, 90.93382984818103, 90.93382984818103, 90.90518476081351, 90.83357204239474, 90.79060441134345, 90.77628186765969, 90.71899169292466, 90.7046691492409, 90.69034660555714, 90.63305643082211, 90.63305643082211, 90.64737897450587, 90.63305643082211, 90.59008879977084, 90.59008879977084, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 34, 67, 94, 131, 152, 180, 202, 227, 249, 273, 293, 315, 334, 361, 381, 401, 424, 443, 463, 486, 500, 514, 531, 554, 574, 593, 610, 626, 646, 657, 669, 689, 704, 723, 733, 747, 761, 778, 786, 800, 819, 830, 848, 857, 865, 876, 888, 908, 920, 929, 938, 944, 954, 957, 964, 973, 977, 988, 995, 999, 1003, 1009, 1015, 1022, 1028, 1034, 1036, 1043, 1045, 1054, 1059, 1067, 1074, 1077, 1082, 1085, 1088, 1094, 1101, 1104, 1107, 1114, 1115, 1119, 1123, 1126, 1133, 1133, 1134, 1137, 1137, 1140, 1141, 1142, 1146, 1149, 1150, 1151, 1154, 1154], "harmed": [0, 1, 1, 1, 4, 4, 6, 9, 11, 12, 15, 19, 23, 23, 24, 25, 29, 31, 37, 39, 44, 49, 56, 61, 64, 74, 78, 83, 91, 94, 99, 108, 114, 121, 127, 133, 139, 143, 148, 151, 156, 163, 165, 170, 176, 179, 186, 193, 196, 203, 207, 214, 220, 227, 233, 238, 242, 248, 253, 256, 261, 263, 264, 270, 277, 282, 284, 288, 293, 295, 299, 303, 304, 308, 313, 315, 321, 326, 326, 328, 330, 335, 338, 341, 346, 347, 350, 359, 364, 368, 372, 376, 380, 382, 387, 391, 393, 395, 399, 402, 408], "exact_score": [79.81953594958465, 80.29217989114866, 80.76482383271268, 81.15153251217416, 81.63849899742193, 81.93927241478086, 82.31165855055858, 82.58378688054998, 82.91320538527643, 83.21397880263535, 83.51475221999426, 83.7439129189344, 84.00171870524204, 84.27384703523344, 84.64623317101116, 84.91836150100256, 85.1475221999427, 85.44829561730162, 85.63448868519048, 85.89229447149813, 86.15010025780579, 86.27900315095961, 86.37926095674591, 86.55113148095101, 86.83758235462618, 86.98080779146376, 87.19564594672013, 87.36751647092524, 87.4820968203953, 87.72558006301918, 87.81151532512173, 87.854482956173, 88.05499856774563, 88.1695789172157, 88.35577198510455, 88.41306215983958, 88.52764250930964, 88.67086794614723, 88.84273847035233, 88.91435118877112, 89.04325408192494, 89.21512460613005, 89.34402749928387, 89.53022056717272, 89.573188198224, 89.64480091664278, 89.70209109137782, 89.77370380979661, 90.0171870524205, 90.0887997708393, 90.16041248925809, 90.18905757662561, 90.18905757662561, 90.23202520767688, 90.18905757662561, 90.21770266399312, 90.28931538241191, 90.26067029504439, 90.34660555714694, 90.40389573188197, 90.38957318819823, 90.41821827556574, 90.48983099398453, 90.48983099398453, 90.48983099398453, 90.50415353766829, 90.56144371240332, 90.53279862503581, 90.56144371240332, 90.56144371240332, 90.63305643082211, 90.64737897450587, 90.74763678029217, 90.79060441134345, 90.76195932397593, 90.80492695502721, 90.76195932397593, 90.73331423660842, 90.81924949871096, 90.89086221712975, 90.90518476081351, 90.876539673446, 90.93382984818103, 90.90518476081351, 90.89086221712975, 90.93382984818103, 90.93382984818103, 90.90518476081351, 90.83357204239474, 90.79060441134345, 90.77628186765969, 90.71899169292466, 90.7046691492409, 90.69034660555714, 90.63305643082211, 90.63305643082211, 90.64737897450587, 90.63305643082211, 90.59008879977084, 90.59008879977084, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.48358355868807, "threshold_area100": 88.48281294757949, "threshold_area25": 83.98453165282153}, "test": {"score": [80.00287005811867, 80.51230537418384, 80.97868981846884, 81.39484824567698, 81.72490492932482, 82.10518763004951, 82.36349286073043, 82.65767381789482, 83.0451316639162, 83.35366291167396, 83.64066872354165, 83.97072540718949, 84.28643180024396, 84.58061275740833, 84.8173925521992, 85.06852263758341, 85.29095214178088, 85.59948338953865, 85.86496376551625, 86.04434239793356, 86.26677190213103, 86.43180024395495, 86.51072684221856, 86.69010547463587, 86.8407835258664, 87.05603788476716, 87.27846738896463, 87.47937145727201, 87.64439979909594, 87.82377843151325, 88.02468249982063, 88.09643395278755, 88.29016287579825, 88.42649063643539, 88.61304441414939, 88.75654732008323, 88.92157566190716, 89.10095429432447, 89.158355456698, 89.28750807203846, 89.40231039678554, 89.51711272153261, 89.57451388390615, 89.69649135394992, 89.76824280691685, 89.86869484107054, 89.95479658463084, 90.03372318289445, 90.0839491999713, 90.10547463586138, 90.19157637942169, 90.23462725120184, 90.30637870416876, 90.28485326827868, 90.32072899476215, 90.34225443065222, 90.3566047212456, 90.3566047212456, 90.37813015713569, 90.44270646480591, 90.46423190069599, 90.50728277247615, 90.50010762717945, 90.49293248188276, 90.47858219128938, 90.45705675539931, 90.47140704599268, 90.42118102891584, 90.40683073832245, 90.39965559302576, 90.37813015713569, 90.3566047212456, 90.37095501183899, 90.41400588361914, 90.38530530243239, 90.3566047212456, 90.41400588361914, 90.3566047212456, 90.37095501183899, 90.37813015713569, 90.39248044772907, 90.40683073832245, 90.41400588361914, 90.3637798665423, 90.37813015713569, 90.34942957594892, 90.32072899476215, 90.27050297768531, 90.23462725120184, 90.24180239649853, 90.27050297768531, 90.24897754179523, 90.24897754179523, 90.25615268709191, 90.21310181531177, 90.23462725120184, 90.24897754179523, 90.23462725120184, 90.1772260888283, 90.09112434526799, 90.06242376408122], "calls": [0.0, 1.1838989739542225, 2.260170768458061, 3.200114802324747, 4.13288369089474, 5.072827724761426, 5.919494869771113, 6.909664920714645, 7.985936715218483, 8.8039032790414, 10.04520341536916, 11.171701226949846, 12.269498457343762, 13.295544234770754, 14.213962832747363, 15.326110353734663, 16.151252062854272, 17.31362560091842, 18.30379565186195, 19.243739685728638, 20.31283633493578, 21.25278036880247, 22.221424983855922, 23.11114300064576, 24.151539068666143, 25.33543804262036, 26.37583411064074, 27.287077563320654, 28.392049939011265, 29.425270861734948, 30.429791203271865, 31.039678553490706, 32.273803544521776, 33.30702446724546, 34.48374829590299, 35.41651718447299, 36.25600918418598, 37.31075554279974, 38.243524431369735, 39.2049939011265, 40.2238645332568, 41.285786037167256, 42.41228384874794, 43.58183253210878, 44.32804764296477, 45.17471478797446, 46.12183396713783, 47.23398148812513, 48.24567697495875, 49.22149673530889, 50.16144076917558, 51.07985936715218, 52.09872999928249, 53.074549759632625, 54.114945827653, 55.19121762215685, 56.13116165602353, 57.2791849034943, 58.27653009973452, 59.41737820190859, 60.42907368874219, 61.51252062854273, 62.81839707254072, 63.77986654229748, 64.8704886273947, 66.11178876372247, 67.13783454114946, 67.94862595967568, 68.89574513883906, 70.02224295041974, 70.9908875654732, 71.90930616344981, 72.96405252206357, 73.8752959747435, 75.03766951280764, 75.89151180311401, 76.78122981990386, 77.81445074262754, 78.73286934060415, 79.65128793858077, 80.41185334003013, 81.5527014422042, 82.56439692903781, 83.66219415943172, 84.51603644973811, 85.31965272296765, 86.23807132094426, 87.24259166248116, 88.01750735452393, 88.84264906364353, 89.92609600344407, 90.87321518260745, 91.90643610533114, 93.00423333572505, 94.20965774556936, 95.06350003587572, 95.94604290736888, 97.07254071894957, 98.17033794934348, 99.23943459855063, 100.0], "rescued": [0, 75, 143, 205, 254, 310, 350, 397, 456, 500, 548, 596, 647, 692, 732, 773, 810, 858, 900, 934, 972, 1006, 1032, 1065, 1100, 1140, 1175, 1209, 1243, 1279, 1317, 1334, 1371, 1399, 1442, 1470, 1503, 1535, 1555, 1580, 1605, 1632, 1654, 1683, 1702, 1723, 1745, 1769, 1786, 1800, 1823, 1836, 1857, 1865, 1881, 1897, 1910, 1922, 1936, 1957, 1970, 1983, 1998, 2007, 2015, 2025, 2038, 2047, 2053, 2059, 2067, 2073, 2083, 2093, 2102, 2107, 2120, 2123, 2129, 2134, 2140, 2149, 2155, 2160, 2167, 2171, 2174, 2179, 2182, 2188, 2199, 2205, 2207, 2213, 2217, 2226, 2233, 2241, 2243, 2245, 2247], "harmed": [0, 4, 7, 11, 14, 17, 21, 27, 32, 33, 41, 43, 50, 54, 61, 67, 73, 78, 83, 92, 99, 110, 125, 133, 147, 157, 161, 167, 178, 189, 199, 206, 216, 225, 242, 250, 260, 267, 279, 286, 295, 306, 320, 332, 341, 348, 358, 371, 381, 392, 403, 410, 421, 432, 443, 456, 467, 479, 490, 502, 512, 519, 535, 545, 555, 568, 579, 595, 603, 610, 621, 630, 638, 642, 655, 664, 669, 680, 684, 688, 692, 699, 704, 716, 721, 729, 736, 748, 756, 761, 768, 777, 779, 784, 794, 800, 805, 815, 825, 839, 845], "exact_score": [80.00287005811867, 80.4405539212169, 80.87823778431513, 81.34462222860013, 81.66750376695128, 82.06213675826936, 82.40654373251058, 82.68637439908159, 83.0523068092129, 83.4110640740475, 83.64066872354165, 83.89897395422258, 84.21468034727704, 84.49451101384804, 84.77434168041903, 84.97524574872641, 85.2694267058908, 85.52055679127503, 85.79321231254933, 85.97976609026333, 86.19502044916409, 86.41027480806487, 86.47485111573509, 86.66140489344909, 86.81208294467963, 86.96276099591017, 87.18519050010764, 87.42914544019517, 87.55112291023893, 87.79507785032648, 87.93140561096362, 88.08208366219417, 88.22558656812801, 88.39779005524862, 88.54129296118248, 88.68479586711632, 88.87134964483032, 89.05790342254431, 89.11530458491785, 89.251632345555, 89.40231039678554, 89.50276243093923, 89.54581330271938, 89.63191504627969, 89.71084164454331, 89.85434455047715, 89.93327114874077, 89.99067231111431, 90.0839491999713, 90.0982994905647, 90.19875152471838, 90.22745210590514, 90.277678122982, 90.28485326827868, 90.33507928535553, 90.34225443065222, 90.37095501183899, 90.34942957594892, 90.37813015713569, 90.41400588361914, 90.47140704599268, 90.53598335366291, 90.51445791777284, 90.50728277247615, 90.50010762717945, 90.50010762717945, 90.46423190069599, 90.47140704599268, 90.41400588361914, 90.40683073832245, 90.39965559302576, 90.37813015713569, 90.34942957594892, 90.37095501183899, 90.42118102891584, 90.38530530243239, 90.3566047212456, 90.40683073832245, 90.3637798665423, 90.38530530243239, 90.37813015713569, 90.38530530243239, 90.39965559302576, 90.39965559302576, 90.37095501183899, 90.34942957594892, 90.34225443065222, 90.28485326827868, 90.24180239649853, 90.24180239649853, 90.28485326827868, 90.24897754179523, 90.24180239649853, 90.25615268709191, 90.22027696060846, 90.22745210590514, 90.24180239649853, 90.22745210590514, 90.19157637942169, 90.11264978115807, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.4239808543215, "threshold_area100": 88.44227595608811, "threshold_area25": 84.12212097294972}, "selected_budget": 82, "selected": {"val": {"score": 90.93382984818103, "calls": 81.9965625895159, "rescued": 1114, "harmed": 338}, "test": {"score": 90.41400588361914, "calls": 82.56439692903781, "rescued": 2155, "harmed": 704}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.93382984818103, "calls": 97.99484388427385}, "test": {"score": 90.66513596900337, "calls": 97.92638300925593}}, "old_normalized_interior": {"val": {"score": 91.07705528501862, "calls": 87.9977083930106}, "test": {"score": 90.72253713137691, "calls": 88.11078424338092}}, "old_normalized_endpoints": {"val": {"score": 91.07705528501862, "calls": 87.9977083930106}, "test": {"score": 90.72253713137691, "calls": 88.11078424338092}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.2778573474649, 80.77914637639644, 81.00830707533657, 81.39501575479805, 81.71011171584072, 82.03953022056717, 82.39759381266111, 82.64107705528502, 82.87023775422514, 83.2283013463191, 83.48610713262676, 83.75823546261816, 84.0876539673446, 84.28816957891722, 84.44571755943855, 84.68920080206243, 84.86107132626755, 85.16184474362646, 85.41965052993412, 85.56287596677168, 85.8063592093956, 86.04984245201948, 86.15010025780579, 86.33629332569464, 86.49384130621598, 86.65138928673733, 86.83758235462618, 87.03809796619879, 87.25293612145518, 87.36751647092524, 87.65396734460039, 87.76854769407046, 87.88312804354052, 87.91177313090805, 88.05499856774563, 88.1695789172157, 88.32712689773703, 88.49899742194212, 88.6135777714122, 88.67086794614723, 88.75680320824978, 88.85706101403609, 88.95731881982239, 89.02893153824118, 89.07189916929245, 89.22944714981381, 89.31538241191636, 89.37267258665138, 89.50157547980521, 89.58751074190776, 89.58751074190776, 89.75938126611285, 89.77370380979661, 89.8453165282154, 89.95989687768548, 90.0171870524205, 90.0887997708393, 90.13176740189057, 90.23202520767688, 90.3609281008307, 90.37525064451447, 90.3609281008307, 90.44686336293324, 90.53279862503581, 90.59008879977084, 90.56144371240332, 90.59008879977084, 90.7046691492409, 90.74763678029217, 90.74763678029217, 90.71899169292466, 90.76195932397593, 90.83357204239474, 90.83357204239474, 90.81924949871096, 90.73331423660842, 90.79060441134345, 90.86221712976223, 90.90518476081351, 90.93382984818103, 90.93382984818103, 91.01976511028357, 90.99112002291606, 91.01976511028357, 91.0484101976511, 91.0484101976511, 91.06273274133486, 91.07705528501862, 91.03408765396735, 90.9481523918648, 90.89086221712975, 90.876539673446, 90.876539673446, 90.86221712976223, 90.84789458607848, 90.81924949871096, 90.74763678029217, 90.61873388713835, 90.56144371240332, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.2778573474649, 80.77914637639644, 81.00830707533657, 81.39501575479805, 81.71011171584072, 82.03953022056717, 82.39759381266111, 82.64107705528502, 82.87023775422514, 83.2283013463191, 83.48610713262676, 83.75823546261816, 84.0876539673446, 84.28816957891722, 84.44571755943855, 84.68920080206243, 84.86107132626755, 85.16184474362646, 85.41965052993412, 85.56287596677168, 85.8063592093956, 86.04984245201948, 86.15010025780579, 86.33629332569464, 86.49384130621598, 86.65138928673733, 86.83758235462618, 87.03809796619879, 87.25293612145518, 87.36751647092524, 87.65396734460039, 87.76854769407046, 87.88312804354052, 87.91177313090805, 88.05499856774563, 88.1695789172157, 88.32712689773703, 88.49899742194212, 88.6135777714122, 88.67086794614723, 88.75680320824978, 88.85706101403609, 88.95731881982239, 89.02893153824118, 89.07189916929245, 89.22944714981381, 89.31538241191636, 89.37267258665138, 89.50157547980521, 89.58751074190776, 89.58751074190776, 89.75938126611285, 89.77370380979661, 89.8453165282154, 89.95989687768548, 90.0171870524205, 90.0887997708393, 90.13176740189057, 90.23202520767688, 90.3609281008307, 90.37525064451447, 90.3609281008307, 90.44686336293324, 90.53279862503581, 90.59008879977084, 90.56144371240332, 90.59008879977084, 90.7046691492409, 90.74763678029217, 90.74763678029217, 90.71899169292466, 90.76195932397593, 90.83357204239474, 90.83357204239474, 90.81924949871096, 90.73331423660842, 90.79060441134345, 90.86221712976223, 90.90518476081351, 90.93382984818103, 90.93382984818103, 91.01976511028357, 90.99112002291606, 91.01976511028357, 91.0484101976511, 91.0484101976511, 91.06273274133486, 91.07705528501862, 91.03408765396735, 90.9481523918648, 90.89086221712975, 90.876539673446, 90.876539673446, 90.86221712976223, 90.84789458607848, 90.81924949871096, 90.74763678029217, 90.61873388713835, 90.56144371240332, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 83.62073904325408, "exact_area100": 88.29452372557465}, "test": {"score": [80.00287005811867, 80.4549042118103, 80.80648633134821, 81.33027193800675, 81.66032862165459, 81.99038530530244, 82.21281480949989, 82.54287149314774, 82.86575303149888, 83.01643108272943, 83.2245102963335, 83.5043409629045, 83.83439764655235, 84.17880462079358, 84.48016072325466, 84.82456769749588, 84.9895960393198, 85.21202554351726, 85.47033077419817, 85.61383368013202, 85.80756260314271, 86.0084666714501, 86.23807132094426, 86.45332567984502, 86.55377771399871, 86.76185692760279, 86.96276099591017, 87.1277893377341, 87.32869340604148, 87.43632058549186, 87.5798234914257, 87.70180096146947, 87.88117959388678, 88.10360909808423, 88.22558656812801, 88.412140345842, 88.56281839707253, 88.69914615770969, 88.85699935423692, 88.89287508072039, 89.01485255076416, 89.10812943962115, 89.18705603788477, 89.27315778144508, 89.36643467030207, 89.45971155915908, 89.48123699504916, 89.61038961038962, 89.70366649924661, 89.83999425988375, 89.8973954222573, 89.94762143933414, 90.01937289230108, 90.04089832819116, 90.13417521704815, 90.20592667001507, 90.25615268709191, 90.39965559302576, 90.49293248188276, 90.57185908014638, 90.60773480662984, 90.70101169548683, 90.70818684078354, 90.75841285786038, 90.75841285786038, 90.78711343904715, 90.85886489201407, 90.94496663557437, 90.99519265265121, 90.97366721676114, 90.95931692616776, 90.94496663557437, 90.90191576379422, 90.88756547320084, 90.90191576379422, 90.90909090909092, 90.90191576379422, 90.88039032790415, 90.86604003731075, 90.78711343904715, 90.80146372964053, 90.80863887493722, 90.77993829375045, 90.77993829375045, 90.744062567267, 90.70101169548683, 90.744062567267, 90.73688742197031, 90.72253713137691, 90.7297122766736, 90.67948625959676, 90.65078567840999, 90.52880820836621, 90.48575733658608, 90.42835617421254, 90.42118102891584, 90.37095501183899, 90.29920355887207, 90.1700509435316, 90.11982492645475, 90.06242376408122], "calls": [0.0, 1.3130515892946832, 2.166893879601062, 3.4512448877089765, 4.391188921575662, 5.488986151969577, 6.271076989309034, 7.34017363851618, 8.380569706536557, 9.11960967209586, 10.016502834182392, 11.071249192796154, 12.255148166750377, 13.50362344837483, 14.694697567625745, 15.76379421683289, 16.502834182392192, 17.57193083159934, 18.705603788476715, 19.423118318145942, 20.291310899045705, 21.346057257659467, 22.573007103393845, 23.52012628255722, 24.237640812226445, 25.42153978618067, 26.390184401234123, 27.6171342469685, 28.37769964841788, 29.210016502834186, 30.106909664920718, 30.88900050226017, 31.721317356676472, 32.82628973236708, 34.04606443280476, 35.172562244385446, 36.44973810719667, 37.38968214106336, 38.473129080863885, 39.23369448231327, 40.06601133672957, 40.876802755255795, 42.118102891583554, 43.15132381430724, 44.22759560881108, 45.20341536916123, 46.12183396713783, 47.212456052235055, 48.01607232546459, 49.243022171198966, 50.16144076917558, 51.194661691899256, 52.14895601635933, 53.0099734519624, 53.8422903063787, 54.61003085312478, 55.49974886991461, 56.59754610030853, 57.573365860658676, 58.60658678338236, 59.79048575733658, 60.56540144937935, 61.526870919136115, 62.4668149530028, 63.42828442275956, 64.5045562172634, 65.34404821697639, 66.60687378919422, 67.59704384013776, 68.55133816459784, 69.51998277965129, 70.8976106766162, 71.83037956518619, 72.64117098371243, 73.51653870990887, 74.85829088039033, 75.89151180311401, 76.86733156346416, 78.07275597330846, 79.37863241730645, 80.2611752887996, 81.2800459209299, 82.46394489488412, 83.38953863815742, 84.55191217622156, 85.4416301930114, 86.47485111573509, 87.26411709837124, 88.11078424338092, 88.84982420894023, 89.76824280691685, 90.57185908014638, 91.59790485757337, 92.64547607089042, 93.62129583124059, 94.47513812154696, 95.41508215541364, 96.4770036593241, 97.74700437683863, 98.95960393197963, 100.0], "exact_score": [80.00287005811867, 80.36162732295328, 80.74191002367799, 81.15806845088613, 81.50965057042406, 81.81100667288513, 82.14106335653297, 82.44959460429074, 82.75812585204851, 82.98773050154266, 83.2245102963335, 83.48281552701442, 83.74112075769534, 84.0568271507498, 84.27925665494726, 84.55908732151826, 84.85326827868265, 85.09722321877018, 85.30530243237426, 85.52773193657171, 85.75016144076918, 85.9151897825931, 86.13044414149387, 86.30264762861448, 86.51072684221856, 86.68293032933917, 86.85513381645978, 87.0416875941738, 87.24259166248117, 87.40762000430509, 87.57264834612901, 87.73050154265624, 87.945755901557, 88.12513453397432, 88.21123627753462, 88.36908947406185, 88.49106694410563, 88.64174499533615, 88.81394848245677, 88.878524790127, 89.00050226017078, 89.10812943962115, 89.17988089258809, 89.251632345555, 89.344909234412, 89.45971155915908, 89.46688670445577, 89.6175647556863, 89.71084164454331, 89.79694338810361, 89.91174571285069, 89.94762143933414, 90.01937289230108, 90.04807347348785, 90.16287579823492, 90.23462725120184, 90.33507928535553, 90.43553131950922, 90.53598335366291, 90.55750878955298, 90.6364353878166, 90.71536198608023, 90.75123771256368, 90.75123771256368, 90.77993829375045, 90.82298916553061, 90.90191576379422, 90.97366721676114, 91.0095429432446, 90.98801750735453, 90.97366721676114, 90.93779149027769, 90.89474061849752, 90.88756547320084, 90.90191576379422, 90.90191576379422, 90.89474061849752, 90.89474061849752, 90.86604003731075, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.78711343904715, 90.75123771256368, 90.73688742197031, 90.70818684078354, 90.72253713137691, 90.75123771256368, 90.7297122766736, 90.70101169548683, 90.65078567840999, 90.62208509722322, 90.51445791777284, 90.48575733658608, 90.42118102891584, 90.38530530243239, 90.34225443065222, 90.25615268709191, 90.14852550764154, 90.12700007175145, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 83.84587787902706, "exact_area100": 88.38602085032254}}, "old_normalized_endpoint_budget": 88}, {"seed": 2030, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 1.0, "validation_area100": 0.8852309363497295, "training_seconds": 27.83179755101446, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6868091064811961, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6289386938547201, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5807773366521347, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5382219374818199, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4986090828163304, "operator": "ge"}, {"budget": 0.06, "threshold": 0.46341959106646424, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4309026160941876, "operator": "ge"}, {"budget": 0.08, "threshold": 0.3953670501800402, "operator": "ge"}, {"budget": 0.09, "threshold": 0.3643756929293275, "operator": "ge"}, {"budget": 0.1, "threshold": 0.33761398158055367, "operator": "ge"}, {"budget": 0.11, "threshold": 0.304232066576866, "operator": "ge"}, {"budget": 0.12, "threshold": 0.28011554883363504, "operator": "ge"}, {"budget": 0.13, "threshold": 0.25510857038307155, "operator": "ge"}, {"budget": 0.14, "threshold": 0.2361680207777997, "operator": "ge"}, {"budget": 0.15, "threshold": 0.20796245803259886, "operator": "ge"}, {"budget": 0.16, "threshold": 0.1794019478650099, "operator": "ge"}, {"budget": 0.17, "threshold": 0.15175764086353358, "operator": "ge"}, {"budget": 0.18, "threshold": 0.12937821668330318, "operator": "ge"}, {"budget": 0.19, "threshold": 0.1017972420646642, "operator": "ge"}, {"budget": 0.2, "threshold": 0.06954401979775343, "operator": "ge"}, {"budget": 0.21, "threshold": 0.041470176736381585, "operator": "ge"}, {"budget": 0.22, "threshold": 0.004811196787691832, "operator": "ge"}, {"budget": 0.23, "threshold": -0.038391225216576445, "operator": "ge"}, {"budget": 0.24, "threshold": -0.07463339923002119, "operator": "ge"}, {"budget": 0.25, "threshold": -0.10987805965130173, "operator": "ge"}, {"budget": 0.26, "threshold": -0.1441805039937663, "operator": "ge"}, {"budget": 0.27, "threshold": -0.18004882135695752, "operator": "ge"}, {"budget": 0.28, "threshold": -0.21175297538934956, "operator": "ge"}, {"budget": 0.29, "threshold": -0.24638431463116034, "operator": "ge"}, {"budget": 0.3, "threshold": -0.28211826377970445, "operator": "ge"}, {"budget": 0.31, "threshold": -0.315553752559438, "operator": "ge"}, {"budget": 0.32, "threshold": -0.3438567095035934, "operator": "ge"}, {"budget": 0.33, "threshold": -0.3726099380132944, "operator": "ge"}, {"budget": 0.34, "threshold": -0.4022338792033109, "operator": "ge"}, {"budget": 0.35, "threshold": -0.42953564083182294, "operator": "ge"}, {"budget": 0.36, "threshold": -0.4538122692580842, "operator": "ge"}, {"budget": 0.37, "threshold": -0.47899541873356016, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5063203561624537, "operator": "ge"}, {"budget": 0.39, "threshold": -0.5281747962937748, "operator": "ge"}, {"budget": 0.4, "threshold": -0.5521093562967119, "operator": "ge"}, {"budget": 0.41, "threshold": -0.5787507784516319, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6004304934811915, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6181331544108025, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6339241332095366, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6530101919134816, "operator": "ge"}, {"budget": 0.46, "threshold": -0.6733364409432882, "operator": "ge"}, {"budget": 0.47, "threshold": -0.691578230909327, "operator": "ge"}, {"budget": 0.48, "threshold": -0.7098516640415631, "operator": "ge"}, {"budget": 0.49, "threshold": -0.724697478872535, "operator": "ge"}, {"budget": 0.5, "threshold": -0.7392790637106812, "operator": "ge"}, {"budget": 0.51, "threshold": -0.7553114580913332, "operator": "ge"}, {"budget": 0.52, "threshold": -0.7678595171236469, "operator": "ge"}, {"budget": 0.53, "threshold": -0.7800470736410908, "operator": "ge"}, {"budget": 0.54, "threshold": -0.7916086227778973, "operator": "ge"}, {"budget": 0.55, "threshold": -0.8030766929775579, "operator": "ge"}, {"budget": 0.56, "threshold": -0.8157113803430587, "operator": "ge"}, {"budget": 0.57, "threshold": -0.824803020114839, "operator": "ge"}, {"budget": 0.58, "threshold": -0.8344664840970998, "operator": "ge"}, {"budget": 0.59, "threshold": -0.8433659464174409, "operator": "ge"}, {"budget": 0.6, "threshold": -0.8540070158554253, "operator": "ge"}, {"budget": 0.61, "threshold": -0.8620719399630297, "operator": "ge"}, {"budget": 0.62, "threshold": -0.868160963047324, "operator": "ge"}, {"budget": 0.63, "threshold": -0.8758881758939191, "operator": "ge"}, {"budget": 0.64, "threshold": -0.8836586823629515, "operator": "ge"}, {"budget": 0.65, "threshold": -0.8897176497881957, "operator": "ge"}, {"budget": 0.66, "threshold": -0.8962608747944726, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9012837274142442, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9067609937805192, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9127155515012257, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9180093676094297, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9230661952737026, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9278262836787431, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9322704337666985, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9358378498097986, "operator": "ge"}, {"budget": 0.75, "threshold": -0.939616181786768, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9437695972301601, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9481727700953335, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9515364410610524, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9549094781447632, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9584218441131783, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9613478203800051, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9645375607318704, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9667879024610521, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9694576977198941, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9722078385182513, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9742640089127141, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9764979248829839, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9788295778481212, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9809820768702079, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9831643935442489, "operator": "ge"}, {"budget": 0.91, "threshold": -0.985006954999673, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9870358406869448, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9887405042498442, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9902117372878053, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9917569785603775, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9930350351740691, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9940903334486955, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9952746788522456, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9966349216449538, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.23488971641363, 80.75050128902893, 81.09424233743913, 81.53824119163563, 81.89630478372959, 82.28301346319105, 82.6124319679175, 82.98481810369522, 83.32855915210541, 83.55771985104555, 83.88713835577198, 84.10197651102835, 84.41707247207103, 84.73216843311371, 85.00429676310513, 85.13319965625895, 85.30507018046404, 85.46261816098539, 85.74906903466055, 85.97822973360068, 86.26468060727585, 86.47951876253221, 86.78029217989113, 86.85190489830994, 87.02377542251504, 87.25293612145518, 87.38183901460899, 87.55370953881409, 87.78287023775422, 87.92609567459179, 88.08364365511314, 88.18390146089945, 88.34144944142079, 88.42738470352333, 88.49899742194212, 88.57061014036091, 88.64222285877972, 88.84273847035233, 88.90002864508736, 88.98596390718991, 89.17215697507876, 89.17215697507876, 89.28673732454884, 89.41564021770266, 89.573188198224, 89.60183328559151, 89.60183328559151, 89.63047837295903, 89.71641363506157, 89.78802635348038, 89.9169292466342, 89.98854196505299, 90.00286450873675, 90.06015468347178, 90.24634775136063, 90.27499283872815, 90.30363792609567, 90.31796046977942, 90.30363792609567, 90.33228301346318, 90.41821827556574, 90.56144371240332, 90.6044113434546, 90.59008879977084, 90.63305643082211, 90.59008879977084, 90.69034660555714, 90.71899169292466, 90.76195932397593, 90.84789458607848, 90.81924949871096, 90.81924949871096, 90.81924949871096, 90.86221712976223, 90.89086221712975, 90.93382984818103, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.96247493554854, 90.99112002291606, 90.9481523918648, 90.9481523918648, 91.01976511028357, 91.09137782870236, 91.12002291606989, 91.10570037238614, 91.07705528501862, 91.01976511028357, 90.99112002291606, 90.99112002291606, 90.96247493554854, 90.9481523918648, 90.91950730449727, 90.91950730449727, 90.83357204239474, 90.79060441134345, 90.74763678029217, 90.67602406187338, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 67, 93, 124, 150, 177, 200, 228, 254, 272, 299, 320, 348, 371, 391, 404, 424, 439, 462, 483, 509, 526, 549, 559, 577, 596, 611, 627, 649, 665, 680, 695, 711, 725, 740, 751, 761, 776, 784, 795, 808, 814, 825, 836, 848, 857, 867, 874, 885, 894, 906, 915, 921, 927, 943, 947, 952, 959, 963, 968, 977, 989, 997, 1001, 1007, 1009, 1020, 1024, 1032, 1041, 1043, 1048, 1051, 1056, 1063, 1067, 1075, 1079, 1085, 1088, 1093, 1096, 1101, 1108, 1115, 1121, 1124, 1128, 1129, 1130, 1136, 1138, 1141, 1143, 1145, 1148, 1151, 1153, 1154, 1154], "harmed": [0, 2, 2, 4, 4, 5, 5, 5, 7, 9, 11, 15, 21, 27, 28, 29, 33, 41, 45, 48, 53, 59, 61, 63, 68, 74, 77, 83, 87, 93, 99, 103, 111, 116, 124, 134, 140, 145, 146, 150, 155, 155, 161, 164, 166, 167, 174, 184, 189, 194, 198, 201, 205, 210, 212, 215, 217, 220, 226, 231, 234, 237, 239, 244, 249, 252, 257, 261, 263, 268, 271, 275, 280, 283, 285, 290, 291, 297, 300, 306, 310, 313, 319, 324, 326, 328, 332, 336, 342, 347, 350, 356, 360, 364, 368, 370, 379, 385, 390, 396, 408], "exact_score": [79.81953594958465, 80.23488971641363, 80.75050128902893, 81.09424233743913, 81.53824119163563, 81.89630478372959, 82.28301346319105, 82.6124319679175, 82.98481810369522, 83.32855915210541, 83.55771985104555, 83.88713835577198, 84.10197651102835, 84.41707247207103, 84.73216843311371, 85.00429676310513, 85.13319965625895, 85.30507018046404, 85.46261816098539, 85.74906903466055, 85.97822973360068, 86.26468060727585, 86.47951876253221, 86.78029217989113, 86.85190489830994, 87.02377542251504, 87.25293612145518, 87.38183901460899, 87.55370953881409, 87.78287023775422, 87.92609567459179, 88.08364365511314, 88.18390146089945, 88.34144944142079, 88.42738470352333, 88.49899742194212, 88.57061014036091, 88.64222285877972, 88.84273847035233, 88.90002864508736, 88.98596390718991, 89.17215697507876, 89.17215697507876, 89.28673732454884, 89.41564021770266, 89.573188198224, 89.60183328559151, 89.60183328559151, 89.63047837295903, 89.71641363506157, 89.78802635348038, 89.9169292466342, 89.98854196505299, 90.00286450873675, 90.06015468347178, 90.24634775136063, 90.27499283872815, 90.30363792609567, 90.31796046977942, 90.30363792609567, 90.33228301346318, 90.41821827556574, 90.56144371240332, 90.6044113434546, 90.59008879977084, 90.63305643082211, 90.59008879977084, 90.69034660555714, 90.71899169292466, 90.76195932397593, 90.84789458607848, 90.81924949871096, 90.81924949871096, 90.81924949871096, 90.86221712976223, 90.89086221712975, 90.93382984818103, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.96247493554854, 90.99112002291606, 90.9481523918648, 90.9481523918648, 91.01976511028357, 91.09137782870236, 91.12002291606989, 91.10570037238614, 91.07705528501862, 91.01976511028357, 90.99112002291606, 90.99112002291606, 90.96247493554854, 90.9481523918648, 90.91950730449727, 90.91950730449727, 90.83357204239474, 90.79060441134345, 90.74763678029217, 90.67602406187338, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.52309363497295, "threshold_area100": 88.52234316814668, "threshold_area25": 83.97164136350615}, "test": {"score": [80.00287005811867, 80.49077993829376, 80.98586496376552, 81.54552629690751, 81.9186338523355, 82.26304082657674, 82.61462294611465, 82.98773050154266, 83.27473631341034, 83.54739183468466, 83.82004735595896, 84.27208150965058, 84.57343761211165, 84.92501973164957, 85.07569778288011, 85.33400301356103, 85.61383368013202, 85.8290880390328, 86.00129152615341, 86.21654588505417, 86.47485111573509, 86.58965344048217, 86.81208294467963, 87.0345124488771, 87.2138910812944, 87.35739398722825, 87.4937217478654, 87.6802755255794, 87.88117959388678, 88.02468249982063, 88.16818540575447, 88.28298773050155, 88.39779005524862, 88.53411781588576, 88.61304441414939, 88.78524790127, 88.91440051661046, 89.03637798665423, 89.15118031140132, 89.19423118318146, 89.3520843797087, 89.40231039678554, 89.55298844801608, 89.59603931979623, 89.69649135394992, 89.8184688239937, 89.92609600344407, 89.94762143933414, 90.07677405467462, 90.1700509435316, 90.24180239649853, 90.37095501183899, 90.44270646480591, 90.53598335366291, 90.55750878955298, 90.5503336442563, 90.58620937073975, 90.60773480662984, 90.62926024251992, 90.65796082370669, 90.65078567840999, 90.6364353878166, 90.65796082370669, 90.6364353878166, 90.64361053311329, 90.61490995192652, 90.66513596900337, 90.71536198608023, 90.70818684078354, 90.71536198608023, 90.71536198608023, 90.80146372964053, 90.78711343904715, 90.84451460142068, 90.85886489201407, 90.85886489201407, 90.90191576379422, 90.90909090909092, 90.9162660543876, 90.87321518260745, 90.88756547320084, 90.88039032790415, 90.82298916553061, 90.86604003731075, 90.84451460142068, 90.78711343904715, 90.73688742197031, 90.75123771256368, 90.80863887493722, 90.75841285786038, 90.69383655019014, 90.67231111430006, 90.64361053311329, 90.60773480662984, 90.55750878955298, 90.50010762717945, 90.44988161010261, 90.41400588361914, 90.29920355887207, 90.22027696060846, 90.06242376408122], "calls": [0.0, 1.1121475209873, 2.138193298414293, 3.221640238214824, 4.211810289158356, 5.158929468321734, 6.199325536342111, 7.332998493219487, 8.373394561239865, 9.34203917629332, 10.396785534907082, 11.580684508861303, 12.599555140991606, 13.647126354308675, 14.242663413934132, 15.240008610174355, 16.438257874721963, 17.528879959819186, 18.38989739542226, 19.301140848102175, 20.284135753749013, 21.008825428714932, 21.93441917198823, 23.132668436535837, 24.02238645332568, 24.78295185477506, 25.794647341608666, 26.827868264332356, 27.710411135825503, 28.815383511516107, 29.77685298127287, 30.874650211666786, 31.678266484896316, 32.546459065796085, 33.56532969792638, 34.57702518475999, 35.39499174858291, 36.27035947477936, 37.360981559876585, 38.08567123484251, 39.05431584989596, 40.28126569563034, 41.37188778072756, 42.125278036880246, 42.94324460070316, 44.041041831097075, 45.20341536916123, 46.243811437181606, 47.248331778718516, 48.26720241084882, 49.4008753677262, 50.54889861519696, 51.48884264906365, 52.62969075123771, 53.53375905862093, 54.581330271938015, 55.715003228815384, 56.64777211738537, 57.6522924589223, 58.59223649278897, 59.833536629116736, 60.75913037239003, 61.412068594389034, 62.38788835473918, 63.47133529453971, 64.33235273014279, 65.41579966994333, 66.21224079787615, 67.04455765229245, 68.15670517327976, 69.26167754897037, 70.2518475999139, 71.19896677907728, 72.24653799239434, 73.08602999210734, 74.12642606012771, 75.36055105115878, 76.59467604218986, 77.477218913683, 78.44586352873645, 79.73021453684437, 80.78496089545814, 81.90428356174213, 82.71507498026835, 83.63349357824495, 84.75281624452894, 85.60665853483533, 86.5537777139987, 87.42197029489847, 88.53411781588576, 89.68214106335654, 90.75123771256368, 91.8346846523642, 92.89660615627466, 93.86525077132812, 94.97739829231541, 96.04649494152257, 96.92903781301571, 97.94073329984933, 99.00265480375977, 100.0], "rescued": [0, 72, 144, 223, 278, 333, 385, 442, 486, 529, 574, 641, 686, 742, 768, 809, 856, 896, 929, 962, 1005, 1025, 1061, 1101, 1133, 1161, 1194, 1227, 1258, 1291, 1316, 1344, 1368, 1396, 1419, 1447, 1469, 1495, 1523, 1533, 1560, 1585, 1616, 1628, 1646, 1670, 1696, 1711, 1731, 1752, 1770, 1797, 1818, 1838, 1851, 1863, 1876, 1889, 1902, 1911, 1921, 1929, 1935, 1942, 1951, 1957, 1970, 1979, 1988, 1997, 2005, 2021, 2032, 2049, 2056, 2060, 2075, 2087, 2096, 2105, 2118, 2127, 2134, 2144, 2156, 2158, 2161, 2171, 2183, 2187, 2194, 2200, 2205, 2211, 2215, 2221, 2224, 2230, 2236, 2244, 2247], "harmed": [0, 4, 7, 8, 11, 18, 21, 26, 30, 35, 42, 46, 49, 56, 61, 66, 74, 84, 93, 96, 103, 107, 112, 121, 128, 136, 150, 157, 160, 173, 178, 190, 198, 207, 219, 223, 227, 236, 248, 252, 257, 275, 285, 291, 295, 302, 313, 325, 327, 335, 343, 352, 363, 370, 380, 393, 401, 411, 421, 426, 437, 447, 450, 460, 468, 478, 484, 486, 496, 504, 512, 516, 529, 538, 543, 547, 556, 567, 575, 590, 601, 611, 626, 630, 645, 655, 665, 673, 677, 688, 704, 713, 722, 733, 744, 758, 768, 779, 801, 820, 845], "exact_score": [80.00287005811867, 80.43337877592022, 80.89976322020522, 81.4665996986439, 81.87558298055536, 82.21998995479659, 82.50699576666428, 82.88727846738897, 83.15275884336658, 83.47564038171772, 83.7841716294755, 84.03530171485973, 84.38688383439765, 84.68823993685872, 84.99677118461649, 85.29095214178088, 85.50620650068164, 85.73581115017579, 85.9151897825931, 86.15196957738395, 86.39592451747149, 86.58965344048217, 86.81208294467963, 86.98428643180026, 87.2067159359977, 87.41479514960179, 87.52242232905216, 87.71615125206286, 87.86682930329339, 88.06773337160078, 88.18971084164454, 88.33321374757838, 88.42649063643539, 88.56999354236925, 88.67762072181962, 88.87134964483032, 88.99332711487408, 89.12247973021454, 89.18705603788477, 89.3377340891153, 89.41666068737892, 89.531463012126, 89.57451388390615, 89.70366649924661, 89.8041185334003, 89.904570567554, 89.93327114874077, 90.04807347348785, 90.14135036234484, 90.23462725120184, 90.33507928535553, 90.3637798665423, 90.47858219128938, 90.55750878955298, 90.54315849895961, 90.56468393484968, 90.57903422544307, 90.61490995192652, 90.62926024251992, 90.65796082370669, 90.6364353878166, 90.62926024251992, 90.64361053311329, 90.65078567840999, 90.62208509722322, 90.62926024251992, 90.70101169548683, 90.71536198608023, 90.70818684078354, 90.70818684078354, 90.77993829375045, 90.79428858434385, 90.81581402023392, 90.85168974671738, 90.85886489201407, 90.89474061849752, 90.90191576379422, 90.93061634498099, 90.88756547320084, 90.87321518260745, 90.90191576379422, 90.85168974671738, 90.82298916553061, 90.84451460142068, 90.81581402023392, 90.77993829375045, 90.73688742197031, 90.79428858434385, 90.79428858434385, 90.73688742197031, 90.69383655019014, 90.67231111430006, 90.64361053311329, 90.60055966133315, 90.54315849895961, 90.50010762717945, 90.44270646480591, 90.39965559302576, 90.28485326827868, 90.22027696060846, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.66692055536727, "threshold_area100": 88.66994331635216, "threshold_area25": 84.31441486690106}, "selected_budget": 86, "selected": {"val": {"score": 91.12002291606989, "calls": 85.99255227728445, "rescued": 1121, "harmed": 332}, "test": {"score": 90.73688742197031, "calls": 85.60665853483533, "rescued": 2161, "harmed": 665}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5450479984283447, "operator": "ge"}, {"budget": 0.02, "threshold": 0.44866690039634705, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3822011947631836, "operator": "ge"}, {"budget": 0.04, "threshold": 0.31616151332855225, "operator": "ge"}, {"budget": 0.05, "threshold": 0.27680838108062744, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2525078058242798, "operator": "ge"}, {"budget": 0.07, "threshold": 0.22654283046722412, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2041226029396057, "operator": "ge"}, {"budget": 0.09, "threshold": 0.1826368272304535, "operator": "ge"}, {"budget": 0.1, "threshold": 0.16365739703178406, "operator": "ge"}, {"budget": 0.11, "threshold": 0.1459392011165619, "operator": "ge"}, {"budget": 0.12, "threshold": 0.12741364538669586, "operator": "ge"}, {"budget": 0.13, "threshold": 0.10535850375890732, "operator": "ge"}, {"budget": 0.14, "threshold": 0.08853459358215332, "operator": "ge"}, {"budget": 0.15, "threshold": 0.07403448224067688, "operator": "ge"}, {"budget": 0.16, "threshold": 0.06059907376766205, "operator": "ge"}, {"budget": 0.17, "threshold": 0.049349211156368256, "operator": "ge"}, {"budget": 0.18, "threshold": 0.03568431735038757, "operator": "ge"}, {"budget": 0.19, "threshold": 0.02629062533378601, "operator": "ge"}, {"budget": 0.2, "threshold": 0.017906108871102333, "operator": "ge"}, {"budget": 0.21, "threshold": 0.0075860172510147095, "operator": "ge"}, {"budget": 0.22, "threshold": 0.0007969290018081665, "operator": "ge"}, {"budget": 0.23, "threshold": -0.0027504940517246723, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0036017820239067078, "operator": "ge"}, {"budget": 0.25, "threshold": -0.004298527725040913, "operator": "ge"}, {"budget": 0.26, "threshold": -0.004843590315431356, "operator": "ge"}, {"budget": 0.27, "threshold": -0.005447747185826302, "operator": "ge"}, {"budget": 0.28, "threshold": -0.006010666023939848, "operator": "ge"}, {"budget": 0.29, "threshold": -0.0065215821377933025, "operator": "ge"}, {"budget": 0.3, "threshold": -0.00712765846401453, "operator": "ge"}, {"budget": 0.31, "threshold": -0.007734772749245167, "operator": "ge"}, {"budget": 0.32, "threshold": -0.00826684758067131, "operator": "ge"}, {"budget": 0.33, "threshold": -0.008896857500076294, "operator": "ge"}, {"budget": 0.34, "threshold": -0.009645300917327404, "operator": "ge"}, {"budget": 0.35, "threshold": -0.01023318525403738, "operator": "ge"}, {"budget": 0.36, "threshold": -0.010937191545963287, "operator": "ge"}, {"budget": 0.37, "threshold": -0.011612320318818092, "operator": "ge"}, {"budget": 0.38, "threshold": -0.012424354441463947, "operator": "ge"}, {"budget": 0.39, "threshold": -0.013040806166827679, "operator": "ge"}, {"budget": 0.4, "threshold": -0.013830265030264854, "operator": "ge"}, {"budget": 0.41, "threshold": -0.01466621458530426, "operator": "ge"}, {"budget": 0.42, "threshold": -0.015659702941775322, "operator": "ge"}, {"budget": 0.43, "threshold": -0.016754012554883957, "operator": "ge"}, {"budget": 0.44, "threshold": -0.017880264669656754, "operator": "ge"}, {"budget": 0.45, "threshold": -0.01887194812297821, "operator": "ge"}, {"budget": 0.46, "threshold": -0.020049303770065308, "operator": "ge"}, {"budget": 0.47, "threshold": -0.021183684468269348, "operator": "ge"}, {"budget": 0.48, "threshold": -0.022350836545228958, "operator": "ge"}, {"budget": 0.49, "threshold": -0.0235259011387825, "operator": "ge"}, {"budget": 0.5, "threshold": -0.024588480591773987, "operator": "ge"}, {"budget": 0.51, "threshold": -0.025538042187690735, "operator": "ge"}, {"budget": 0.52, "threshold": -0.026420120149850845, "operator": "ge"}, {"budget": 0.53, "threshold": -0.027546614408493042, "operator": "ge"}, {"budget": 0.54, "threshold": -0.02836635150015354, "operator": "ge"}, {"budget": 0.55, "threshold": -0.029583370313048363, "operator": "ge"}, {"budget": 0.56, "threshold": -0.030813084915280342, "operator": "ge"}, {"budget": 0.57, "threshold": -0.03197440505027771, "operator": "ge"}, {"budget": 0.58, "threshold": -0.03337150812149048, "operator": "ge"}, {"budget": 0.59, "threshold": -0.03459979593753815, "operator": "ge"}, {"budget": 0.6, "threshold": -0.03591132536530495, "operator": "ge"}, {"budget": 0.61, "threshold": -0.03761451691389084, "operator": "ge"}, {"budget": 0.62, "threshold": -0.039053451269865036, "operator": "ge"}, {"budget": 0.63, "threshold": -0.04063161835074425, "operator": "ge"}, {"budget": 0.64, "threshold": -0.042250894010066986, "operator": "ge"}, {"budget": 0.65, "threshold": -0.043920230120420456, "operator": "ge"}, {"budget": 0.66, "threshold": -0.04575689136981964, "operator": "ge"}, {"budget": 0.67, "threshold": -0.04770984500646591, "operator": "ge"}, {"budget": 0.68, "threshold": -0.04968325048685074, "operator": "ge"}, {"budget": 0.69, "threshold": -0.05195293575525284, "operator": "ge"}, {"budget": 0.7, "threshold": -0.05401977151632309, "operator": "ge"}, {"budget": 0.71, "threshold": -0.05611192062497139, "operator": "ge"}, {"budget": 0.72, "threshold": -0.058290034532547, "operator": "ge"}, {"budget": 0.73, "threshold": -0.06008421629667282, "operator": "ge"}, {"budget": 0.74, "threshold": -0.0620439276099205, "operator": "ge"}, {"budget": 0.75, "threshold": -0.06506276875734329, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0676497220993042, "operator": "ge"}, {"budget": 0.77, "threshold": -0.07128432393074036, "operator": "ge"}, {"budget": 0.78, "threshold": -0.07495281100273132, "operator": "ge"}, {"budget": 0.79, "threshold": -0.07797320187091827, "operator": "ge"}, {"budget": 0.8, "threshold": -0.08221407234668732, "operator": "ge"}, {"budget": 0.81, "threshold": -0.08611778914928436, "operator": "ge"}, {"budget": 0.82, "threshold": -0.09148325026035309, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0958746150135994, "operator": "ge"}, {"budget": 0.84, "threshold": -0.10081078112125397, "operator": "ge"}, {"budget": 0.85, "threshold": -0.10716839134693146, "operator": "ge"}, {"budget": 0.86, "threshold": -0.1134929358959198, "operator": "ge"}, {"budget": 0.87, "threshold": -0.1205015555024147, "operator": "ge"}, {"budget": 0.88, "threshold": -0.12990669906139374, "operator": "ge"}, {"budget": 0.89, "threshold": -0.13961949944496155, "operator": "ge"}, {"budget": 0.9, "threshold": -0.15027828514575958, "operator": "ge"}, {"budget": 0.91, "threshold": -0.16266211867332458, "operator": "ge"}, {"budget": 0.92, "threshold": -0.17933420836925507, "operator": "ge"}, {"budget": 0.93, "threshold": -0.19552993774414062, "operator": "ge"}, {"budget": 0.94, "threshold": -0.21202006936073303, "operator": "ge"}, {"budget": 0.95, "threshold": -0.23394060134887695, "operator": "ge"}, {"budget": 0.96, "threshold": -0.26798373460769653, "operator": "ge"}, {"budget": 0.97, "threshold": -0.31653717160224915, "operator": "ge"}, {"budget": 0.98, "threshold": -0.38330143690109253, "operator": "ge"}, {"budget": 0.99, "threshold": -0.5097091197967529, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.2778573474649, 80.75050128902893, 81.18017759954166, 81.50959610426811, 81.81036952162704, 82.22572328845602, 82.6124319679175, 82.99914064737897, 83.27126897737037, 83.5720423947293, 83.87281581208822, 84.24520194786594, 84.41707247207103, 84.58894299627613, 84.78945860784874, 85.09023202520767, 85.36236035519907, 85.57719851045546, 85.74906903466055, 85.99255227728445, 86.25035806359209, 86.52248639358349, 86.57977656831852, 86.59409911200228, 86.65138928673733, 86.6943569177886, 86.75164709252363, 86.75164709252363, 86.75164709252363, 86.78029217989113, 86.80893726725866, 86.85190489830994, 86.89487252936121, 86.90919507304497, 86.96648524778, 87.06674305356631, 87.09538814093382, 87.26725866513893, 87.38183901460899, 87.38183901460899, 87.45345173302778, 87.53938699513033, 87.59667716986537, 87.61099971354912, 87.71125751933543, 87.69693497565167, 87.73990260670294, 87.79719278143799, 87.854482956173, 87.95474076195931, 87.96906330564308, 88.0263534803781, 88.11228874248066, 88.1695789172157, 88.25551417931824, 88.28415926668576, 88.39873961615584, 88.47035233457461, 88.5419650529934, 88.57061014036091, 88.62790031509596, 88.69951303351475, 88.71383557719851, 88.74248066456603, 88.81409338298482, 88.94299627613864, 89.04325408192494, 89.0575766256087, 89.15783443139502, 89.20080206244629, 89.30105986823258, 89.44428530507018, 89.50157547980521, 89.5445431108565, 89.573188198224, 89.573188198224, 89.58751074190776, 89.5445431108565, 89.61615582927529, 89.68776854769406, 89.75938126611285, 89.87396161558291, 89.8453165282154, 89.9169292466342, 90.03150959610426, 90.03150959610426, 90.06015468347178, 90.13176740189057, 90.23202520767688, 90.30363792609567, 90.38957318819823, 90.46118590661702, 90.50415353766829, 90.57576625608708, 90.67602406187338, 90.6044113434546, 90.66170151818963, 90.80492695502721, 90.71899169292466, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 33, 66, 96, 121, 145, 176, 203, 233, 253, 276, 298, 326, 346, 363, 382, 407, 428, 450, 469, 489, 509, 529, 534, 536, 540, 543, 547, 549, 550, 552, 554, 558, 561, 562, 566, 574, 580, 592, 601, 603, 609, 616, 623, 627, 635, 639, 646, 652, 658, 667, 671, 677, 684, 689, 697, 701, 711, 718, 724, 730, 737, 747, 751, 755, 763, 777, 786, 791, 800, 808, 819, 831, 840, 849, 857, 863, 872, 878, 885, 894, 905, 917, 922, 934, 948, 957, 967, 979, 989, 1000, 1014, 1032, 1046, 1061, 1079, 1090, 1107, 1127, 1138, 1154], "harmed": [0, 1, 1, 1, 3, 6, 8, 8, 11, 12, 14, 15, 17, 25, 30, 35, 39, 41, 48, 55, 58, 60, 61, 62, 63, 63, 63, 63, 65, 66, 66, 66, 67, 67, 67, 67, 68, 72, 72, 73, 75, 76, 77, 80, 83, 84, 89, 93, 95, 97, 99, 102, 104, 105, 106, 108, 110, 112, 114, 115, 119, 122, 127, 130, 132, 135, 140, 142, 146, 148, 153, 157, 159, 164, 170, 176, 182, 190, 199, 201, 205, 211, 215, 222, 229, 235, 244, 252, 259, 262, 268, 276, 289, 300, 310, 321, 337, 350, 360, 377, 408], "exact_score": [79.81953594958465, 80.2778573474649, 80.75050128902893, 81.18017759954166, 81.50959610426811, 81.81036952162704, 82.22572328845602, 82.6124319679175, 82.99914064737897, 83.27126897737037, 83.5720423947293, 83.87281581208822, 84.24520194786594, 84.41707247207103, 84.58894299627613, 84.78945860784874, 85.09023202520767, 85.36236035519907, 85.57719851045546, 85.74906903466055, 85.99255227728445, 86.25035806359209, 86.52248639358349, 86.57977656831852, 86.59409911200228, 86.65138928673733, 86.6943569177886, 86.75164709252363, 86.75164709252363, 86.75164709252363, 86.78029217989113, 86.80893726725866, 86.85190489830994, 86.89487252936121, 86.90919507304497, 86.96648524778, 87.06674305356631, 87.09538814093382, 87.26725866513893, 87.38183901460899, 87.38183901460899, 87.45345173302778, 87.53938699513033, 87.59667716986537, 87.61099971354912, 87.71125751933543, 87.69693497565167, 87.73990260670294, 87.79719278143799, 87.854482956173, 87.95474076195931, 87.96906330564308, 88.0263534803781, 88.11228874248066, 88.1695789172157, 88.25551417931824, 88.28415926668576, 88.39873961615584, 88.47035233457461, 88.5419650529934, 88.57061014036091, 88.62790031509596, 88.69951303351475, 88.71383557719851, 88.74248066456603, 88.81409338298482, 88.94299627613864, 89.04325408192494, 89.0575766256087, 89.15783443139502, 89.20080206244629, 89.30105986823258, 89.44428530507018, 89.50157547980521, 89.5445431108565, 89.573188198224, 89.573188198224, 89.58751074190776, 89.5445431108565, 89.61615582927529, 89.68776854769406, 89.75938126611285, 89.87396161558291, 89.8453165282154, 89.9169292466342, 90.03150959610426, 90.03150959610426, 90.06015468347178, 90.13176740189057, 90.23202520767688, 90.30363792609567, 90.38957318819823, 90.46118590661702, 90.50415353766829, 90.57576625608708, 90.67602406187338, 90.6044113434546, 90.66170151818963, 90.80492695502721, 90.71899169292466, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 87.51792471880675, "threshold_area100": 87.51718705242051, "threshold_area25": 83.8799770839301}, "test": {"score": [80.00287005811867, 80.61993255363421, 81.1867690320729, 81.5598765875009, 82.14106335653297, 82.54287149314774, 82.83705245031211, 83.21016000574012, 83.48999067231112, 83.7841716294755, 84.00660113367296, 84.3510081079142, 84.58778790270503, 84.8891440051661, 85.05417234699003, 85.29095214178088, 85.57795795364856, 85.72863600487909, 85.95824065437326, 86.27394704742771, 86.36004879098802, 86.62552916696563, 86.80490779938295, 86.89818468823994, 86.9340604147234, 86.98428643180026, 87.01298701298703, 87.07756332065725, 87.10626390184402, 87.14931477362417, 87.19236564540432, 87.23541651718448, 87.27129224366794, 87.33586855133817, 87.43632058549186, 87.42914544019517, 87.46502116667862, 87.52242232905216, 87.52242232905216, 87.5941737820191, 87.61569921790917, 87.6802755255794, 87.74485183324963, 87.78790270502978, 87.87400444859009, 87.92423046566694, 88.00315706393054, 88.0533830810074, 88.08925880749085, 88.1394848245677, 88.18971084164454, 88.21123627753462, 88.21123627753462, 88.29016287579825, 88.34756403817178, 88.43366578173209, 88.48389179880894, 88.56999354236925, 88.61304441414939, 88.691971012413, 88.67762072181962, 88.74219702948984, 88.79959819186338, 88.98615196957739, 89.02920284135753, 89.09377914902777, 89.158355456698, 89.17988089258809, 89.19423118318146, 89.24445720025831, 89.32338379852193, 89.38796010619215, 89.40948554208224, 89.45253641386238, 89.56733873860946, 89.61038961038962, 89.63191504627969, 89.71084164454331, 89.70366649924661, 89.78976824280693, 89.78976824280693, 89.84716940518047, 89.98349716581761, 89.94762143933414, 89.98349716581761, 90.05524861878453, 90.07677405467462, 90.1772260888283, 90.21310181531177, 90.277678122982, 90.22745210590514, 90.28485326827868, 90.24897754179523, 90.30637870416876, 90.24180239649853, 90.32790414005885, 90.33507928535553, 90.39965559302576, 90.42118102891584, 90.32790414005885, 90.06242376408122], "calls": [0.0, 0.9901700509435316, 2.231470187271292, 3.1570639305445938, 4.635143861663198, 5.632489057903422, 6.364353878166034, 7.368874219702949, 8.258592236492788, 9.220061706249552, 10.246107483676544, 11.193226662839923, 12.183396713783454, 13.266843653583985, 14.321590012197746, 15.326110353734663, 16.287579823491427, 17.12707182320442, 18.210518763004952, 19.143287651574944, 19.86797732654086, 20.965774556934775, 21.920068881394847, 23.02504125708546, 23.85735811150176, 24.976680777785752, 25.87357393987228, 26.964196024969507, 27.96154122120973, 28.923010690966493, 29.977757049580255, 31.032503408194017, 31.9724474420607, 33.02001865537777, 34.16086675755184, 35.07928535552845, 36.04075482528521, 36.959173423261824, 38.02827007246897, 38.81036090980842, 39.65702805481811, 40.4965200545311, 41.47233981488125, 42.58448733586855, 43.69663485685585, 44.5720025830523, 45.75590155700653, 46.817823060916986, 47.80799311186051, 48.86991461577097, 49.946186410274805, 50.67087608524072, 51.531893520843795, 52.50053813589726, 53.14630121259955, 54.15799669943316, 55.27731936571716, 56.167037382507004, 57.32223577527444, 58.15455262969075, 59.05144579177728, 60.213819329841435, 61.18963909019157, 62.3735380641458, 63.406758986869484, 64.35387816603286, 65.34404821697639, 66.42032001148023, 67.4678912247973, 68.70919136112506, 69.64196024969506, 70.45992681351797, 71.36399512090121, 72.18196168472412, 73.03580397503049, 74.35603070962186, 75.26727416230179, 76.53009973451962, 77.63507211021023, 78.44586352873645, 79.45755901557007, 80.46925450240367, 81.70337949343475, 82.50699576666428, 83.42541436464089, 84.54473703092488, 85.45598048360479, 86.40309966276817, 87.57264834612901, 88.6847958671163, 89.84716940518047, 90.77993829375045, 91.8490349429576, 92.73157781445074, 93.57106981416374, 94.51101384803042, 95.58011049723757, 96.6707325823348, 97.75417952213532, 99.08158140202339, 100.0], "rescued": [0, 88, 172, 230, 318, 378, 420, 475, 516, 566, 605, 658, 697, 746, 781, 818, 861, 888, 929, 975, 991, 1029, 1061, 1081, 1086, 1093, 1098, 1108, 1115, 1122, 1130, 1139, 1145, 1154, 1171, 1174, 1179, 1190, 1192, 1202, 1207, 1218, 1229, 1240, 1255, 1267, 1282, 1292, 1304, 1315, 1327, 1332, 1338, 1352, 1362, 1378, 1387, 1403, 1414, 1429, 1435, 1453, 1465, 1495, 1508, 1522, 1537, 1549, 1560, 1577, 1594, 1610, 1623, 1636, 1655, 1677, 1689, 1711, 1724, 1739, 1752, 1775, 1807, 1819, 1839, 1861, 1876, 1898, 1921, 1946, 1966, 1993, 2013, 2039, 2051, 2077, 2107, 2139, 2175, 2217, 2247], "harmed": [0, 2, 7, 13, 20, 24, 25, 28, 30, 39, 47, 52, 58, 65, 77, 81, 84, 90, 99, 101, 105, 106, 113, 120, 120, 120, 121, 122, 125, 126, 128, 131, 132, 132, 135, 139, 139, 142, 144, 144, 146, 148, 150, 155, 158, 163, 167, 170, 177, 181, 186, 188, 194, 197, 199, 203, 205, 209, 214, 218, 226, 235, 239, 243, 250, 255, 261, 270, 279, 289, 295, 302, 312, 319, 322, 338, 347, 358, 372, 375, 388, 403, 416, 433, 448, 460, 472, 480, 498, 514, 541, 560, 585, 603, 624, 638, 667, 690, 723, 778, 845], "exact_score": [80.00287005811867, 80.61993255363421, 81.1006672885126, 81.47377484394059, 81.86123268996197, 82.29891655306021, 82.68637439908159, 83.09535768099305, 83.4110640740475, 83.71242017650857, 83.93484968070604, 84.29360694554066, 84.55191217622158, 84.8030422616058, 84.98242089402311, 85.22637583411064, 85.51338164597834, 85.72146085958241, 85.9080146372964, 86.23807132094426, 86.3815742268781, 86.62552916696563, 86.80490779938295, 86.89818468823994, 86.9340604147234, 86.98428643180026, 87.01298701298703, 87.07038817536056, 87.10626390184402, 87.15648991892087, 87.19236564540432, 87.23541651718448, 87.27129224366794, 87.33586855133817, 87.41479514960179, 87.42197029489847, 87.45784602138194, 87.52242232905216, 87.52242232905216, 87.60852407261247, 87.64439979909594, 87.72332639735954, 87.78790270502978, 87.85247901270002, 87.88835473918347, 87.9529310468537, 88.03185764511731, 88.07490851689748, 88.11078424338093, 88.15383511516109, 88.19688598694124, 88.21841142283131, 88.26863743990816, 88.34038889287508, 88.42649063643539, 88.48389179880894, 88.55564325177585, 88.59869412355602, 88.691971012413, 88.67762072181962, 88.74219702948984, 88.80677333716008, 88.91440051661046, 89.02202769606085, 89.05790342254431, 89.11530458491785, 89.1440051661046, 89.20140632847816, 89.19423118318146, 89.28750807203846, 89.36643467030207, 89.40948554208224, 89.438186123269, 89.56733873860946, 89.60321446509292, 89.63909019157639, 89.66779077276315, 89.69649135394992, 89.76106766162016, 89.79694338810361, 89.84716940518047, 89.94762143933414, 89.94762143933414, 89.94762143933414, 90.01219774700438, 90.09112434526799, 90.14852550764154, 90.20592667001507, 90.20592667001507, 90.25615268709191, 90.24897754179523, 90.25615268709191, 90.277678122982, 90.27050297768531, 90.22027696060846, 90.34225443065222, 90.38530530243239, 90.40683073832245, 90.44270646480591, 90.34225443065222, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 87.73372178616854, "threshold_area100": 87.73200832316856, "threshold_area25": 84.3388103609098}, "selected_budget": 98, "selected": {"val": {"score": 90.80492695502721, "calls": 97.99484388427385, "rescued": 1127, "harmed": 360}, "test": {"score": 90.42118102891584, "calls": 97.75417952213532, "rescued": 2175, "harmed": 723}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.368499994277954, "operator": "ge"}, {"budget": 0.02, "threshold": 2.845097541809082, "operator": "ge"}, {"budget": 0.03, "threshold": 2.484431743621826, "operator": "ge"}, {"budget": 0.04, "threshold": 2.2396326065063477, "operator": "ge"}, {"budget": 0.05, "threshold": 2.049095630645752, "operator": "ge"}, {"budget": 0.06, "threshold": 1.8817672729492188, "operator": "ge"}, {"budget": 0.07, "threshold": 1.730617642402649, "operator": "ge"}, {"budget": 0.08, "threshold": 1.5673842430114746, "operator": "ge"}, {"budget": 0.09, "threshold": 1.4358793497085571, "operator": "ge"}, {"budget": 0.1, "threshold": 1.2841124534606934, "operator": "ge"}, {"budget": 0.11, "threshold": 1.1364469528198242, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0113316774368286, "operator": "ge"}, {"budget": 0.13, "threshold": 0.8939421772956848, "operator": "ge"}, {"budget": 0.14, "threshold": 0.7680280804634094, "operator": "ge"}, {"budget": 0.15, "threshold": 0.6536218523979187, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5401996374130249, "operator": "ge"}, {"budget": 0.17, "threshold": 0.43814167380332947, "operator": "ge"}, {"budget": 0.18, "threshold": 0.3600166141986847, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2889218330383301, "operator": "ge"}, {"budget": 0.2, "threshold": 0.191896453499794, "operator": "ge"}, {"budget": 0.21, "threshold": 0.11024469137191772, "operator": "ge"}, {"budget": 0.22, "threshold": 0.01319095492362976, "operator": "ge"}, {"budget": 0.23, "threshold": -0.10484865307807922, "operator": "ge"}, {"budget": 0.24, "threshold": -0.17739981412887573, "operator": "ge"}, {"budget": 0.25, "threshold": -0.2715942859649658, "operator": "ge"}, {"budget": 0.26, "threshold": -0.34213924407958984, "operator": "ge"}, {"budget": 0.27, "threshold": -0.43966302275657654, "operator": "ge"}, {"budget": 0.28, "threshold": -0.5018633604049683, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5933927893638611, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6650259494781494, "operator": "ge"}, {"budget": 0.31, "threshold": -0.7353493571281433, "operator": "ge"}, {"budget": 0.32, "threshold": -0.807865560054779, "operator": "ge"}, {"budget": 0.33, "threshold": -0.882608950138092, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9614303112030029, "operator": "ge"}, {"budget": 0.35, "threshold": -1.0360790491104126, "operator": "ge"}, {"budget": 0.36, "threshold": -1.0952446460723877, "operator": "ge"}, {"budget": 0.37, "threshold": -1.1708252429962158, "operator": "ge"}, {"budget": 0.38, "threshold": -1.2371629476547241, "operator": "ge"}, {"budget": 0.39, "threshold": -1.3008060455322266, "operator": "ge"}, {"budget": 0.4, "threshold": -1.375016689300537, "operator": "ge"}, {"budget": 0.41, "threshold": -1.466152548789978, "operator": "ge"}, {"budget": 0.42, "threshold": -1.5363309383392334, "operator": "ge"}, {"budget": 0.43, "threshold": -1.5989654064178467, "operator": "ge"}, {"budget": 0.44, "threshold": -1.6593413352966309, "operator": "ge"}, {"budget": 0.45, "threshold": -1.7226049900054932, "operator": "ge"}, {"budget": 0.46, "threshold": -1.7861897945404053, "operator": "ge"}, {"budget": 0.47, "threshold": -1.866382122039795, "operator": "ge"}, {"budget": 0.48, "threshold": -1.9353758096694946, "operator": "ge"}, {"budget": 0.49, "threshold": -2.015366554260254, "operator": "ge"}, {"budget": 0.5, "threshold": -2.080451726913452, "operator": "ge"}, {"budget": 0.51, "threshold": -2.1525676250457764, "operator": "ge"}, {"budget": 0.52, "threshold": -2.2092947959899902, "operator": "ge"}, {"budget": 0.53, "threshold": -2.2736644744873047, "operator": "ge"}, {"budget": 0.54, "threshold": -2.3553035259246826, "operator": "ge"}, {"budget": 0.55, "threshold": -2.4320476055145264, "operator": "ge"}, {"budget": 0.56, "threshold": -2.4937596321105957, "operator": "ge"}, {"budget": 0.57, "threshold": -2.5547075271606445, "operator": "ge"}, {"budget": 0.58, "threshold": -2.632605791091919, "operator": "ge"}, {"budget": 0.59, "threshold": -2.69541072845459, "operator": "ge"}, {"budget": 0.6, "threshold": -2.7529258728027344, "operator": "ge"}, {"budget": 0.61, "threshold": -2.8102588653564453, "operator": "ge"}, {"budget": 0.62, "threshold": -2.883305072784424, "operator": "ge"}, {"budget": 0.63, "threshold": -2.9391942024230957, "operator": "ge"}, {"budget": 0.64, "threshold": -3.0085787773132324, "operator": "ge"}, {"budget": 0.65, "threshold": -3.0634989738464355, "operator": "ge"}, {"budget": 0.66, "threshold": -3.126070499420166, "operator": "ge"}, {"budget": 0.67, "threshold": -3.18276309967041, "operator": "ge"}, {"budget": 0.68, "threshold": -3.2530336380004883, "operator": "ge"}, {"budget": 0.69, "threshold": -3.3157968521118164, "operator": "ge"}, {"budget": 0.7, "threshold": -3.392275810241699, "operator": "ge"}, {"budget": 0.71, "threshold": -3.452895402908325, "operator": "ge"}, {"budget": 0.72, "threshold": -3.5081794261932373, "operator": "ge"}, {"budget": 0.73, "threshold": -3.57645845413208, "operator": "ge"}, {"budget": 0.74, "threshold": -3.6465892791748047, "operator": "ge"}, {"budget": 0.75, "threshold": -3.7184247970581055, "operator": "ge"}, {"budget": 0.76, "threshold": -3.778714656829834, "operator": "ge"}, {"budget": 0.77, "threshold": -3.8561744689941406, "operator": "ge"}, {"budget": 0.78, "threshold": -3.9261655807495117, "operator": "ge"}, {"budget": 0.79, "threshold": -4.007455348968506, "operator": "ge"}, {"budget": 0.8, "threshold": -4.071496486663818, "operator": "ge"}, {"budget": 0.81, "threshold": -4.129145622253418, "operator": "ge"}, {"budget": 0.82, "threshold": -4.210762977600098, "operator": "ge"}, {"budget": 0.83, "threshold": -4.302441596984863, "operator": "ge"}, {"budget": 0.84, "threshold": -4.375946044921875, "operator": "ge"}, {"budget": 0.85, "threshold": -4.4469170570373535, "operator": "ge"}, {"budget": 0.86, "threshold": -4.532223701477051, "operator": "ge"}, {"budget": 0.87, "threshold": -4.616732597351074, "operator": "ge"}, {"budget": 0.88, "threshold": -4.707438945770264, "operator": "ge"}, {"budget": 0.89, "threshold": -4.805290222167969, "operator": "ge"}, {"budget": 0.9, "threshold": -4.904852390289307, "operator": "ge"}, {"budget": 0.91, "threshold": -5.025387763977051, "operator": "ge"}, {"budget": 0.92, "threshold": -5.162154197692871, "operator": "ge"}, {"budget": 0.93, "threshold": -5.292247295379639, "operator": "ge"}, {"budget": 0.94, "threshold": -5.411787033081055, "operator": "ge"}, {"budget": 0.95, "threshold": -5.568326950073242, "operator": "ge"}, {"budget": 0.96, "threshold": -5.741003513336182, "operator": "ge"}, {"budget": 0.97, "threshold": -5.914728164672852, "operator": "ge"}, {"budget": 0.98, "threshold": -6.1456708908081055, "operator": "ge"}, {"budget": 0.99, "threshold": -6.4879536628723145, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.26353480378114, 80.72185620166141, 81.10856488112287, 81.4093382984818, 81.81036952162704, 82.28301346319105, 82.6124319679175, 82.9561730163277, 83.34288169578916, 83.70094528788312, 83.95875107419077, 84.15926668576338, 84.41707247207103, 84.66055571469494, 84.88971641363506, 85.17616728731022, 85.36236035519907, 85.50558579203667, 85.72042394729303, 86.0068748209682, 86.26468060727585, 86.47951876253221, 86.76596963620739, 86.88054998567745, 87.05242050988255, 87.29590375250645, 87.42480664566027, 87.61099971354912, 87.7542251503867, 87.94041821827557, 88.08364365511314, 88.21254654826697, 88.38441707247206, 88.42738470352333, 88.41306215983958, 88.52764250930964, 88.69951303351475, 88.85706101403609, 88.90002864508736, 89.0575766256087, 89.1291893440275, 89.22944714981381, 89.41564021770266, 89.50157547980521, 89.51589802348897, 89.5445431108565, 89.61615582927529, 89.64480091664278, 89.74505872242909, 89.88828415926669, 89.85963907189917, 89.97421942136923, 90.04583213978802, 90.16041248925809, 90.18905757662561, 90.16041248925809, 90.18905757662561, 90.21770266399312, 90.30363792609567, 90.44686336293324, 90.50415353766829, 90.51847608135203, 90.50415353766829, 90.54712116871956, 90.59008879977084, 90.64737897450587, 90.73331423660842, 90.77628186765969, 90.76195932397593, 90.79060441134345, 90.83357204239474, 90.74763678029217, 90.80492695502721, 90.83357204239474, 90.90518476081351, 90.96247493554854, 91.00544256659983, 91.0484101976511, 91.01976511028357, 90.93382984818103, 90.96247493554854, 91.00544256659983, 91.09137782870236, 91.14866800343741, 91.13434545975365, 91.13434545975365, 91.06273274133486, 91.0484101976511, 91.03408765396735, 90.9481523918648, 90.9767974792323, 90.9481523918648, 90.93382984818103, 90.90518476081351, 90.91950730449727, 90.84789458607848, 90.80492695502721, 90.71899169292466, 90.67602406187338, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 64, 92, 116, 145, 178, 201, 227, 255, 280, 299, 316, 336, 360, 381, 403, 423, 441, 461, 485, 509, 526, 548, 561, 578, 598, 614, 634, 648, 668, 683, 697, 714, 728, 736, 752, 765, 779, 788, 800, 810, 819, 834, 842, 849, 858, 868, 876, 889, 901, 908, 918, 928, 938, 944, 949, 955, 959, 966, 979, 987, 995, 997, 1004, 1011, 1016, 1024, 1031, 1036, 1041, 1045, 1048, 1055, 1060, 1067, 1075, 1081, 1086, 1087, 1090, 1095, 1100, 1108, 1117, 1119, 1120, 1125, 1126, 1129, 1129, 1136, 1137, 1140, 1143, 1146, 1149, 1151, 1151, 1154, 1154], "harmed": [0, 0, 1, 2, 5, 6, 6, 6, 8, 9, 9, 10, 13, 15, 22, 27, 29, 36, 44, 49, 53, 59, 61, 63, 68, 73, 76, 83, 90, 94, 101, 106, 111, 116, 127, 136, 144, 145, 148, 154, 155, 160, 162, 164, 166, 172, 179, 184, 190, 196, 198, 207, 209, 214, 216, 220, 227, 231, 233, 234, 237, 241, 248, 251, 255, 259, 260, 262, 266, 272, 275, 276, 285, 288, 291, 293, 297, 300, 302, 305, 314, 317, 319, 321, 326, 329, 330, 340, 342, 346, 352, 357, 360, 364, 369, 371, 379, 384, 390, 396, 408], "exact_score": [79.81953594958465, 80.26353480378114, 80.72185620166141, 81.10856488112287, 81.4093382984818, 81.81036952162704, 82.28301346319105, 82.6124319679175, 82.9561730163277, 83.34288169578916, 83.70094528788312, 83.95875107419077, 84.15926668576338, 84.41707247207103, 84.66055571469494, 84.88971641363506, 85.17616728731022, 85.36236035519907, 85.50558579203667, 85.72042394729303, 86.0068748209682, 86.26468060727585, 86.47951876253221, 86.76596963620739, 86.88054998567745, 87.05242050988255, 87.29590375250645, 87.42480664566027, 87.61099971354912, 87.7542251503867, 87.94041821827557, 88.08364365511314, 88.21254654826697, 88.38441707247206, 88.42738470352333, 88.41306215983958, 88.52764250930964, 88.69951303351475, 88.85706101403609, 88.90002864508736, 89.0575766256087, 89.1291893440275, 89.22944714981381, 89.41564021770266, 89.50157547980521, 89.51589802348897, 89.5445431108565, 89.61615582927529, 89.64480091664278, 89.74505872242909, 89.88828415926669, 89.85963907189917, 89.97421942136923, 90.04583213978802, 90.16041248925809, 90.18905757662561, 90.16041248925809, 90.18905757662561, 90.21770266399312, 90.30363792609567, 90.44686336293324, 90.50415353766829, 90.51847608135203, 90.50415353766829, 90.54712116871956, 90.59008879977084, 90.64737897450587, 90.73331423660842, 90.77628186765969, 90.76195932397593, 90.79060441134345, 90.83357204239474, 90.74763678029217, 90.80492695502721, 90.83357204239474, 90.90518476081351, 90.96247493554854, 91.00544256659983, 91.0484101976511, 91.01976511028357, 90.93382984818103, 90.96247493554854, 91.00544256659983, 91.09137782870236, 91.14866800343741, 91.13434545975365, 91.13434545975365, 91.06273274133486, 91.0484101976511, 91.03408765396735, 90.9481523918648, 90.9767974792323, 90.9481523918648, 90.93382984818103, 90.90518476081351, 90.91950730449727, 90.84789458607848, 90.80492695502721, 90.71899169292466, 90.67602406187338, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.52879229242835, "threshold_area100": 88.52807218562016, "threshold_area25": 83.9687768547694}, "test": {"score": [80.00287005811867, 80.56970653655738, 81.19394417736959, 81.77513094640166, 82.07648704886275, 82.4854703307742, 82.85140274090551, 83.1384085527732, 83.49716581760781, 83.69806988591519, 84.00660113367296, 84.25055607376049, 84.48733586855134, 84.68823993685872, 85.0828729281768, 85.34835330415441, 85.56360766305518, 85.81473774843941, 85.99411638085671, 86.13761928679055, 86.44615053454832, 86.61835402166894, 86.80490779938295, 87.07038817536056, 87.19954079070101, 87.35739398722825, 87.50089689316209, 87.72332639735954, 87.83095357680993, 88.0533830810074, 88.16818540575447, 88.28298773050155, 88.39779005524862, 88.5197675252924, 88.65609528592955, 88.75654732008323, 89.00767740546746, 89.04355313195093, 89.10812943962115, 89.18705603788477, 89.32338379852193, 89.47406184975246, 89.56733873860946, 89.61038961038962, 89.73236708043338, 89.82564396929038, 89.87586998636723, 90.02654803759776, 90.04807347348785, 90.18440123412499, 90.29920355887207, 90.37095501183899, 90.47858219128938, 90.47140704599268, 90.50728277247615, 90.52880820836621, 90.53598335366291, 90.57185908014638, 90.57903422544307, 90.60773480662984, 90.60773480662984, 90.60773480662984, 90.57903422544307, 90.57903422544307, 90.62926024251992, 90.67231111430006, 90.67231111430006, 90.70818684078354, 90.744062567267, 90.73688742197031, 90.76558800315708, 90.8301643108273, 90.84451460142068, 90.88756547320084, 90.90909090909092, 90.92344119968429, 90.94496663557437, 90.93061634498099, 90.92344119968429, 90.85886489201407, 90.81581402023392, 90.8301643108273, 90.89474061849752, 90.81581402023392, 90.80146372964053, 90.76558800315708, 90.77993829375045, 90.79428858434385, 90.75841285786038, 90.67948625959676, 90.67948625959676, 90.65796082370669, 90.60773480662984, 90.57185908014638, 90.55750878955298, 90.49293248188276, 90.44270646480591, 90.41400588361914, 90.29920355887207, 90.21310181531177, 90.06242376408122], "calls": [0.0, 1.1049723756906076, 2.339097366721676, 3.6664992466097437, 4.6207935710698145, 5.539212169046423, 6.601133672956877, 7.519552270933487, 8.531247757767096, 9.485542082227164, 10.597689603214464, 11.551983927674534, 12.398651072684222, 13.381645978331061, 14.594245533472053, 15.620291310899045, 16.545885054172345, 17.57910597689603, 18.382722250125568, 19.222214249838558, 20.26978546315563, 21.051876300495085, 21.920068881394847, 23.2618210518763, 23.936284709765374, 24.840353017148598, 25.722895888641744, 26.835043409629044, 27.595608811078424, 28.743632058549185, 29.748152400086102, 30.645045562172633, 31.534763578962473, 32.48188275812585, 33.550979407332996, 34.55549974886992, 35.45239291095645, 36.442562961899974, 37.224653799239434, 38.07132094424912, 39.10454186697281, 40.467819473344335, 41.37188778072756, 42.297481524000865, 43.101097797230395, 44.02669154050369, 44.86618354021669, 46.19358542010475, 47.06895314630121, 48.439405897969436, 49.42957594891297, 50.656525794647344, 51.52471837554711, 52.42161153763364, 53.60551051158786, 54.739183468465235, 55.66477721173854, 56.604721245605226, 57.80297051015283, 58.74291454401952, 59.568056253139126, 60.515175432302506, 61.49817033794934, 62.42376408122264, 63.3852335509794, 64.3180024395494, 65.20772045633923, 65.95393556719523, 67.18088541292961, 68.07777857501614, 69.15405036951998, 70.12269498457344, 70.9908875654732, 72.16043624883403, 73.24388318863457, 74.24840353017149, 75.27444930759847, 76.59467604218986, 77.76422472555069, 78.91224797302145, 79.83066657099806, 80.66298342541437, 81.58857716868766, 82.91597904857574, 83.84157279184905, 84.75999138982564, 85.71428571428571, 86.60400373107555, 87.61569921790915, 88.54846810647916, 89.59603931979623, 90.71536198608023, 91.95666212240798, 92.94683217335151, 93.7863241730645, 94.94152256583196, 95.98909377914903, 96.92186266771901, 97.99813446222286, 99.00265480375977, 100.0], "rescued": [0, 83, 171, 260, 309, 368, 421, 466, 518, 552, 599, 637, 673, 706, 769, 811, 847, 891, 924, 951, 1002, 1028, 1060, 1106, 1131, 1162, 1195, 1233, 1256, 1295, 1316, 1344, 1368, 1395, 1423, 1442, 1483, 1501, 1518, 1534, 1563, 1602, 1623, 1633, 1658, 1678, 1694, 1723, 1733, 1764, 1787, 1807, 1830, 1840, 1856, 1872, 1885, 1897, 1910, 1918, 1927, 1935, 1941, 1952, 1965, 1975, 1982, 1992, 2004, 2011, 2026, 2040, 2047, 2056, 2070, 2079, 2091, 2103, 2112, 2117, 2123, 2130, 2144, 2150, 2155, 2159, 2168, 2176, 2182, 2184, 2195, 2199, 2205, 2211, 2215, 2221, 2223, 2230, 2237, 2244, 2247], "harmed": [0, 4, 5, 13, 20, 22, 24, 29, 31, 37, 41, 45, 48, 53, 61, 66, 72, 81, 89, 96, 104, 106, 112, 121, 128, 137, 150, 157, 165, 173, 178, 190, 198, 208, 217, 222, 228, 241, 249, 254, 264, 282, 290, 294, 302, 309, 318, 326, 333, 345, 352, 362, 370, 381, 392, 405, 417, 424, 436, 440, 449, 457, 467, 478, 484, 488, 495, 500, 507, 515, 526, 531, 536, 539, 550, 557, 566, 580, 590, 604, 616, 621, 626, 643, 650, 659, 666, 672, 683, 696, 707, 714, 727, 738, 744, 759, 768, 779, 802, 821, 845], "exact_score": [80.00287005811867, 80.49077993829376, 81.07196670732583, 81.48812513453397, 81.86123268996197, 82.24869053598336, 82.62897323670805, 83.00925593743274, 83.29626174930043, 83.61914328765158, 83.84874793714573, 84.12857860371673, 84.40123412499103, 84.64518906507857, 84.83174284279256, 85.18332496233049, 85.41292961182464, 85.67123484250556, 85.92236492788979, 86.09456841501041, 86.36722393628472, 86.61117887637225, 86.81208294467963, 86.98428643180026, 87.24259166248117, 87.4004448590084, 87.53677261964555, 87.74485183324963, 87.89552988448015, 88.07490851689748, 88.22558656812801, 88.3260386022817, 88.46954150821554, 88.58434383296262, 88.67762072181962, 88.87134964483032, 89.02202769606085, 89.065078567841, 89.18705603788477, 89.32338379852193, 89.39513525148884, 89.5386381574227, 89.624739900983, 89.69649135394992, 89.82564396929038, 89.8973954222573, 90.01219774700438, 90.03372318289445, 90.14852550764154, 90.24180239649853, 90.32072899476215, 90.42835617421254, 90.50010762717945, 90.50010762717945, 90.54315849895961, 90.50010762717945, 90.5503336442563, 90.58620937073975, 90.58620937073975, 90.62208509722322, 90.6364353878166, 90.58620937073975, 90.57185908014638, 90.59338451603645, 90.65796082370669, 90.65796082370669, 90.70818684078354, 90.70818684078354, 90.744062567267, 90.75841285786038, 90.8301643108273, 90.84451460142068, 90.87321518260745, 90.90191576379422, 90.90191576379422, 90.94496663557437, 90.95214178087106, 90.9162660543876, 90.93061634498099, 90.86604003731075, 90.8301643108273, 90.84451460142068, 90.85886489201407, 90.81581402023392, 90.80146372964053, 90.75123771256368, 90.80146372964053, 90.80863887493722, 90.72253713137691, 90.66513596900337, 90.67231111430006, 90.65796082370669, 90.60055966133315, 90.57185908014638, 90.54315849895961, 90.49293248188276, 90.44270646480591, 90.40683073832245, 90.29920355887207, 90.21310181531177, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.67281687014685, "threshold_area100": 88.68271507498027, "threshold_area25": 84.39334146516467}, "selected_budget": 84, "selected": {"val": {"score": 91.14866800343741, "calls": 83.9873961615583, "rescued": 1117, "harmed": 326}, "test": {"score": 90.80146372964053, "calls": 83.84157279184905, "rescued": 2155, "harmed": 650}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.6837810277938843, "operator": "ge"}, {"budget": 0.02, "threshold": 1.4793167114257812, "operator": "ge"}, {"budget": 0.03, "threshold": 1.3272697925567627, "operator": "ge"}, {"budget": 0.04, "threshold": 1.2032980918884277, "operator": "ge"}, {"budget": 0.05, "threshold": 1.0949066877365112, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0033148527145386, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9220091700553894, "operator": "ge"}, {"budget": 0.08, "threshold": 0.8362913131713867, "operator": "ge"}, {"budget": 0.09, "threshold": 0.7638445496559143, "operator": "ge"}, {"budget": 0.1, "threshold": 0.7027941942214966, "operator": "ge"}, {"budget": 0.11, "threshold": 0.628353476524353, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5756150484085083, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5217390656471252, "operator": "ge"}, {"budget": 0.14, "threshold": 0.48142367601394653, "operator": "ge"}, {"budget": 0.15, "threshold": 0.42208153009414673, "operator": "ge"}, {"budget": 0.16, "threshold": 0.36272940039634705, "operator": "ge"}, {"budget": 0.17, "threshold": 0.30587807297706604, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2602148652076721, "operator": "ge"}, {"budget": 0.19, "threshold": 0.20430219173431396, "operator": "ge"}, {"budget": 0.2, "threshold": 0.1393129825592041, "operator": "ge"}, {"budget": 0.21, "threshold": 0.08298802375793457, "operator": "ge"}, {"budget": 0.22, "threshold": 0.009622454643249512, "operator": "ge"}, {"budget": 0.23, "threshold": -0.07682019472122192, "operator": "ge"}, {"budget": 0.24, "threshold": -0.1495448350906372, "operator": "ge"}, {"budget": 0.25, "threshold": -0.2206469178199768, "operator": "ge"}, {"budget": 0.26, "threshold": -0.2903843820095062, "operator": "ge"}, {"budget": 0.27, "threshold": -0.36406636238098145, "operator": "ge"}, {"budget": 0.28, "threshold": -0.4300117492675781, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5031196475028992, "operator": "ge"}, {"budget": 0.3, "threshold": -0.5799641609191895, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6534028053283691, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7169197797775269, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7829010486602783, "operator": "ge"}, {"budget": 0.34, "threshold": -0.852622389793396, "operator": "ge"}, {"budget": 0.35, "threshold": -0.9186542630195618, "operator": "ge"}, {"budget": 0.36, "threshold": -0.9789819717407227, "operator": "ge"}, {"budget": 0.37, "threshold": -1.043359637260437, "operator": "ge"}, {"budget": 0.38, "threshold": -1.1155383586883545, "operator": "ge"}, {"budget": 0.39, "threshold": -1.1752212047576904, "operator": "ge"}, {"budget": 0.4, "threshold": -1.242821216583252, "operator": "ge"}, {"budget": 0.41, "threshold": -1.321165680885315, "operator": "ge"}, {"budget": 0.42, "threshold": -1.3876404762268066, "operator": "ge"}, {"budget": 0.43, "threshold": -1.4439563751220703, "operator": "ge"}, {"budget": 0.44, "threshold": -1.4959009885787964, "operator": "ge"}, {"budget": 0.45, "threshold": -1.5610579252243042, "operator": "ge"}, {"budget": 0.46, "threshold": -1.6336461305618286, "operator": "ge"}, {"budget": 0.47, "threshold": -1.7019492387771606, "operator": "ge"}, {"budget": 0.48, "threshold": -1.7737715244293213, "operator": "ge"}, {"budget": 0.49, "threshold": -1.8349366188049316, "operator": "ge"}, {"budget": 0.5, "threshold": -1.897775650024414, "operator": "ge"}, {"budget": 0.51, "threshold": -1.970415711402893, "operator": "ge"}, {"budget": 0.52, "threshold": -2.030182361602783, "operator": "ge"}, {"budget": 0.53, "threshold": -2.090982437133789, "operator": "ge"}, {"budget": 0.54, "threshold": -2.151451587677002, "operator": "ge"}, {"budget": 0.55, "threshold": -2.2144360542297363, "operator": "ge"}, {"budget": 0.56, "threshold": -2.2877306938171387, "operator": "ge"}, {"budget": 0.57, "threshold": -2.3433167934417725, "operator": "ge"}, {"budget": 0.58, "threshold": -2.4053354263305664, "operator": "ge"}, {"budget": 0.59, "threshold": -2.4654412269592285, "operator": "ge"}, {"budget": 0.6, "threshold": -2.541548490524292, "operator": "ge"}, {"budget": 0.61, "threshold": -2.6027145385742188, "operator": "ge"}, {"budget": 0.62, "threshold": -2.6511285305023193, "operator": "ge"}, {"budget": 0.63, "threshold": -2.7156550884246826, "operator": "ge"}, {"budget": 0.64, "threshold": -2.7844433784484863, "operator": "ge"}, {"budget": 0.65, "threshold": -2.8411388397216797, "operator": "ge"}, {"budget": 0.66, "threshold": -2.905762195587158, "operator": "ge"}, {"budget": 0.67, "threshold": -2.958040237426758, "operator": "ge"}, {"budget": 0.68, "threshold": -3.017996072769165, "operator": "ge"}, {"budget": 0.69, "threshold": -3.0871083736419678, "operator": "ge"}, {"budget": 0.7, "threshold": -3.1524500846862793, "operator": "ge"}, {"budget": 0.71, "threshold": -3.2187306880950928, "operator": "ge"}, {"budget": 0.72, "threshold": -3.2850770950317383, "operator": "ge"}, {"budget": 0.73, "threshold": -3.3509345054626465, "operator": "ge"}, {"budget": 0.74, "threshold": -3.4068877696990967, "operator": "ge"}, {"budget": 0.75, "threshold": -3.4695286750793457, "operator": "ge"}, {"budget": 0.76, "threshold": -3.542926788330078, "operator": "ge"}, {"budget": 0.77, "threshold": -3.6267619132995605, "operator": "ge"}, {"budget": 0.78, "threshold": -3.695563554763794, "operator": "ge"}, {"budget": 0.79, "threshold": -3.7694272994995117, "operator": "ge"}, {"budget": 0.8, "threshold": -3.8523340225219727, "operator": "ge"}, {"budget": 0.81, "threshold": -3.926804542541504, "operator": "ge"}, {"budget": 0.82, "threshold": -4.014542102813721, "operator": "ge"}, {"budget": 0.83, "threshold": -4.081248760223389, "operator": "ge"}, {"budget": 0.84, "threshold": -4.166403293609619, "operator": "ge"}, {"budget": 0.85, "threshold": -4.262168884277344, "operator": "ge"}, {"budget": 0.86, "threshold": -4.340071678161621, "operator": "ge"}, {"budget": 0.87, "threshold": -4.431997776031494, "operator": "ge"}, {"budget": 0.88, "threshold": -4.537659645080566, "operator": "ge"}, {"budget": 0.89, "threshold": -4.645967483520508, "operator": "ge"}, {"budget": 0.9, "threshold": -4.768975257873535, "operator": "ge"}, {"budget": 0.91, "threshold": -4.885798454284668, "operator": "ge"}, {"budget": 0.92, "threshold": -5.032238483428955, "operator": "ge"}, {"budget": 0.93, "threshold": -5.174057960510254, "operator": "ge"}, {"budget": 0.94, "threshold": -5.31486701965332, "operator": "ge"}, {"budget": 0.95, "threshold": -5.487431526184082, "operator": "ge"}, {"budget": 0.96, "threshold": -5.656537055969238, "operator": "ge"}, {"budget": 0.97, "threshold": -5.821414947509766, "operator": "ge"}, {"budget": 0.98, "threshold": -6.045616626739502, "operator": "ge"}, {"budget": 0.99, "threshold": -6.385791301727295, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [79.81953594958465, 80.23488971641363, 80.75050128902893, 81.09424233743913, 81.53824119163563, 81.89630478372959, 82.28301346319105, 82.6124319679175, 82.98481810369522, 83.32855915210541, 83.55771985104555, 83.88713835577198, 84.10197651102835, 84.41707247207103, 84.73216843311371, 85.00429676310513, 85.13319965625895, 85.30507018046404, 85.46261816098539, 85.74906903466055, 85.97822973360068, 86.26468060727585, 86.47951876253221, 86.78029217989113, 86.85190489830994, 87.02377542251504, 87.25293612145518, 87.38183901460899, 87.55370953881409, 87.78287023775422, 87.92609567459179, 88.08364365511314, 88.18390146089945, 88.34144944142079, 88.42738470352333, 88.49899742194212, 88.57061014036091, 88.64222285877972, 88.84273847035233, 88.90002864508736, 88.98596390718991, 89.17215697507876, 89.17215697507876, 89.28673732454884, 89.41564021770266, 89.573188198224, 89.60183328559151, 89.60183328559151, 89.63047837295903, 89.71641363506157, 89.78802635348038, 89.9169292466342, 89.98854196505299, 90.00286450873675, 90.06015468347178, 90.24634775136063, 90.27499283872815, 90.30363792609567, 90.31796046977942, 90.30363792609567, 90.33228301346318, 90.41821827556574, 90.56144371240332, 90.6044113434546, 90.59008879977084, 90.63305643082211, 90.59008879977084, 90.69034660555714, 90.71899169292466, 90.76195932397593, 90.84789458607848, 90.81924949871096, 90.81924949871096, 90.81924949871096, 90.86221712976223, 90.89086221712975, 90.93382984818103, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.96247493554854, 90.99112002291606, 90.9481523918648, 90.9481523918648, 91.01976511028357, 91.09137782870236, 91.12002291606989, 91.10570037238614, 91.07705528501862, 91.01976511028357, 90.99112002291606, 90.99112002291606, 90.96247493554854, 90.9481523918648, 90.91950730449727, 90.91950730449727, 90.83357204239474, 90.79060441134345, 90.74763678029217, 90.67602406187338, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 67, 93, 124, 150, 177, 200, 228, 254, 272, 299, 320, 348, 371, 391, 404, 424, 439, 462, 483, 509, 526, 549, 559, 577, 596, 611, 627, 649, 665, 680, 695, 711, 725, 740, 751, 761, 776, 784, 795, 808, 814, 825, 836, 848, 857, 867, 874, 885, 894, 906, 915, 921, 927, 943, 947, 952, 959, 963, 968, 977, 989, 997, 1001, 1007, 1009, 1020, 1024, 1032, 1041, 1043, 1048, 1051, 1056, 1063, 1067, 1075, 1079, 1085, 1088, 1093, 1096, 1101, 1108, 1115, 1121, 1124, 1128, 1129, 1130, 1136, 1138, 1141, 1143, 1145, 1148, 1151, 1153, 1154, 1154], "harmed": [0, 2, 2, 4, 4, 5, 5, 5, 7, 9, 11, 15, 21, 27, 28, 29, 33, 41, 45, 48, 53, 59, 61, 63, 68, 74, 77, 83, 87, 93, 99, 103, 111, 116, 124, 134, 140, 145, 146, 150, 155, 155, 161, 164, 166, 167, 174, 184, 189, 194, 198, 201, 205, 210, 212, 215, 217, 220, 226, 231, 234, 237, 239, 244, 249, 252, 257, 261, 263, 268, 271, 275, 280, 283, 285, 290, 291, 297, 300, 306, 310, 313, 319, 324, 326, 328, 332, 336, 342, 347, 350, 356, 360, 364, 368, 370, 379, 385, 390, 396, 408], "exact_score": [79.81953594958465, 80.23488971641363, 80.75050128902893, 81.09424233743913, 81.53824119163563, 81.89630478372959, 82.28301346319105, 82.6124319679175, 82.98481810369522, 83.32855915210541, 83.55771985104555, 83.88713835577198, 84.10197651102835, 84.41707247207103, 84.73216843311371, 85.00429676310513, 85.13319965625895, 85.30507018046404, 85.46261816098539, 85.74906903466055, 85.97822973360068, 86.26468060727585, 86.47951876253221, 86.78029217989113, 86.85190489830994, 87.02377542251504, 87.25293612145518, 87.38183901460899, 87.55370953881409, 87.78287023775422, 87.92609567459179, 88.08364365511314, 88.18390146089945, 88.34144944142079, 88.42738470352333, 88.49899742194212, 88.57061014036091, 88.64222285877972, 88.84273847035233, 88.90002864508736, 88.98596390718991, 89.17215697507876, 89.17215697507876, 89.28673732454884, 89.41564021770266, 89.573188198224, 89.60183328559151, 89.60183328559151, 89.63047837295903, 89.71641363506157, 89.78802635348038, 89.9169292466342, 89.98854196505299, 90.00286450873675, 90.06015468347178, 90.24634775136063, 90.27499283872815, 90.30363792609567, 90.31796046977942, 90.30363792609567, 90.33228301346318, 90.41821827556574, 90.56144371240332, 90.6044113434546, 90.59008879977084, 90.63305643082211, 90.59008879977084, 90.69034660555714, 90.71899169292466, 90.76195932397593, 90.84789458607848, 90.81924949871096, 90.81924949871096, 90.81924949871096, 90.86221712976223, 90.89086221712975, 90.93382984818103, 90.96247493554854, 90.9767974792323, 90.9767974792323, 90.96247493554854, 90.99112002291606, 90.9481523918648, 90.9481523918648, 91.01976511028357, 91.09137782870236, 91.12002291606989, 91.10570037238614, 91.07705528501862, 91.01976511028357, 90.99112002291606, 90.99112002291606, 90.96247493554854, 90.9481523918648, 90.91950730449727, 90.91950730449727, 90.83357204239474, 90.79060441134345, 90.74763678029217, 90.67602406187338, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 88.52309363497295, "threshold_area100": 88.52234316814668, "threshold_area25": 83.97164136350615}, "test": {"score": [80.00287005811867, 80.49077993829376, 80.98586496376552, 81.54552629690751, 81.9186338523355, 82.26304082657674, 82.61462294611465, 82.98773050154266, 83.27473631341034, 83.54739183468466, 83.82004735595896, 84.27208150965058, 84.57343761211165, 84.92501973164957, 85.07569778288011, 85.33400301356103, 85.61383368013202, 85.8290880390328, 86.00129152615341, 86.21654588505417, 86.47485111573509, 86.58965344048217, 86.81208294467963, 87.0345124488771, 87.2138910812944, 87.35739398722825, 87.4937217478654, 87.6802755255794, 87.88117959388678, 88.02468249982063, 88.16818540575447, 88.28298773050155, 88.39779005524862, 88.53411781588576, 88.61304441414939, 88.78524790127, 88.91440051661046, 89.03637798665423, 89.15118031140132, 89.19423118318146, 89.3520843797087, 89.40231039678554, 89.55298844801608, 89.59603931979623, 89.69649135394992, 89.8184688239937, 89.92609600344407, 89.94762143933414, 90.07677405467462, 90.1700509435316, 90.24180239649853, 90.37095501183899, 90.44270646480591, 90.53598335366291, 90.55750878955298, 90.5503336442563, 90.58620937073975, 90.60773480662984, 90.62926024251992, 90.65796082370669, 90.65078567840999, 90.6364353878166, 90.65796082370669, 90.6364353878166, 90.64361053311329, 90.61490995192652, 90.66513596900337, 90.71536198608023, 90.70818684078354, 90.71536198608023, 90.71536198608023, 90.80146372964053, 90.78711343904715, 90.84451460142068, 90.85886489201407, 90.85886489201407, 90.90191576379422, 90.90909090909092, 90.9162660543876, 90.87321518260745, 90.88756547320084, 90.88039032790415, 90.82298916553061, 90.86604003731075, 90.84451460142068, 90.78711343904715, 90.73688742197031, 90.75123771256368, 90.80863887493722, 90.75841285786038, 90.69383655019014, 90.67231111430006, 90.64361053311329, 90.60773480662984, 90.55750878955298, 90.50010762717945, 90.44988161010261, 90.41400588361914, 90.29920355887207, 90.22027696060846, 90.06242376408122], "calls": [0.0, 1.1121475209873, 2.138193298414293, 3.221640238214824, 4.211810289158356, 5.158929468321734, 6.199325536342111, 7.332998493219487, 8.373394561239865, 9.34203917629332, 10.396785534907082, 11.580684508861303, 12.599555140991606, 13.647126354308675, 14.242663413934132, 15.240008610174355, 16.438257874721963, 17.528879959819186, 18.38989739542226, 19.301140848102175, 20.284135753749013, 21.008825428714932, 21.93441917198823, 23.132668436535837, 24.02238645332568, 24.78295185477506, 25.794647341608666, 26.827868264332356, 27.710411135825503, 28.815383511516107, 29.77685298127287, 30.874650211666786, 31.678266484896316, 32.546459065796085, 33.56532969792638, 34.57702518475999, 35.39499174858291, 36.27035947477936, 37.360981559876585, 38.08567123484251, 39.05431584989596, 40.28126569563034, 41.37188778072756, 42.125278036880246, 42.94324460070316, 44.041041831097075, 45.20341536916123, 46.243811437181606, 47.248331778718516, 48.26720241084882, 49.4008753677262, 50.54889861519696, 51.48884264906365, 52.62969075123771, 53.53375905862093, 54.581330271938015, 55.715003228815384, 56.64777211738537, 57.6522924589223, 58.59223649278897, 59.833536629116736, 60.75913037239003, 61.412068594389034, 62.38788835473918, 63.47133529453971, 64.33235273014279, 65.41579966994333, 66.21224079787615, 67.04455765229245, 68.15670517327976, 69.26167754897037, 70.2518475999139, 71.19896677907728, 72.24653799239434, 73.08602999210734, 74.12642606012771, 75.36055105115878, 76.59467604218986, 77.477218913683, 78.44586352873645, 79.73021453684437, 80.78496089545814, 81.90428356174213, 82.71507498026835, 83.63349357824495, 84.75999138982564, 85.60665853483533, 86.5537777139987, 87.42197029489847, 88.53411781588576, 89.68214106335654, 90.75123771256368, 91.8346846523642, 92.89660615627466, 93.86525077132812, 94.97739829231541, 96.04649494152257, 96.92903781301571, 97.94073329984933, 99.00265480375977, 100.0], "rescued": [0, 72, 144, 223, 278, 333, 385, 442, 486, 529, 574, 641, 686, 742, 768, 809, 856, 896, 929, 962, 1005, 1025, 1061, 1101, 1133, 1161, 1194, 1227, 1258, 1291, 1316, 1344, 1368, 1396, 1419, 1447, 1469, 1495, 1523, 1533, 1560, 1585, 1616, 1628, 1646, 1670, 1696, 1711, 1731, 1752, 1770, 1797, 1818, 1838, 1851, 1863, 1876, 1889, 1902, 1911, 1921, 1929, 1935, 1942, 1951, 1957, 1970, 1979, 1988, 1997, 2005, 2021, 2032, 2049, 2056, 2060, 2075, 2087, 2096, 2105, 2118, 2127, 2134, 2144, 2156, 2158, 2161, 2171, 2183, 2187, 2194, 2200, 2205, 2211, 2215, 2221, 2224, 2230, 2236, 2244, 2247], "harmed": [0, 4, 7, 8, 11, 18, 21, 26, 30, 35, 42, 46, 49, 56, 61, 66, 74, 84, 93, 96, 103, 107, 112, 121, 128, 136, 150, 157, 160, 173, 178, 190, 198, 207, 219, 223, 227, 236, 248, 252, 257, 275, 285, 291, 295, 302, 313, 325, 327, 335, 343, 352, 363, 370, 380, 393, 401, 411, 421, 426, 437, 447, 450, 460, 468, 478, 484, 486, 496, 504, 512, 516, 529, 538, 543, 547, 556, 567, 575, 590, 601, 611, 626, 630, 645, 655, 665, 673, 677, 688, 704, 713, 722, 733, 744, 758, 768, 779, 801, 820, 845], "exact_score": [80.00287005811867, 80.43337877592022, 80.89976322020522, 81.4665996986439, 81.87558298055536, 82.21998995479659, 82.50699576666428, 82.88727846738897, 83.15275884336658, 83.47564038171772, 83.7841716294755, 84.03530171485973, 84.38688383439765, 84.68823993685872, 84.99677118461649, 85.29095214178088, 85.50620650068164, 85.73581115017579, 85.9151897825931, 86.15196957738395, 86.39592451747149, 86.58965344048217, 86.81208294467963, 86.98428643180026, 87.2067159359977, 87.41479514960179, 87.52242232905216, 87.71615125206286, 87.86682930329339, 88.06773337160078, 88.18971084164454, 88.33321374757838, 88.42649063643539, 88.56999354236925, 88.67762072181962, 88.87134964483032, 88.99332711487408, 89.12247973021454, 89.18705603788477, 89.3377340891153, 89.41666068737892, 89.531463012126, 89.57451388390615, 89.70366649924661, 89.8041185334003, 89.904570567554, 89.93327114874077, 90.04807347348785, 90.14135036234484, 90.23462725120184, 90.33507928535553, 90.3637798665423, 90.47858219128938, 90.55750878955298, 90.54315849895961, 90.56468393484968, 90.57903422544307, 90.61490995192652, 90.62926024251992, 90.65796082370669, 90.6364353878166, 90.62926024251992, 90.64361053311329, 90.65078567840999, 90.62208509722322, 90.62926024251992, 90.70101169548683, 90.71536198608023, 90.70818684078354, 90.70818684078354, 90.77993829375045, 90.79428858434385, 90.81581402023392, 90.85168974671738, 90.85886489201407, 90.89474061849752, 90.90191576379422, 90.93061634498099, 90.88756547320084, 90.87321518260745, 90.90191576379422, 90.85168974671738, 90.82298916553061, 90.84451460142068, 90.81581402023392, 90.77993829375045, 90.73688742197031, 90.79428858434385, 90.79428858434385, 90.73688742197031, 90.69383655019014, 90.67231111430006, 90.64361053311329, 90.60055966133315, 90.54315849895961, 90.50010762717945, 90.44270646480591, 90.39965559302576, 90.28485326827868, 90.22027696060846, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 88.66692055536727, "threshold_area100": 88.66994331635216, "threshold_area25": 84.31441486690106}, "selected_budget": 86, "selected": {"val": {"score": 91.12002291606989, "calls": 85.99255227728445, "rescued": 1121, "harmed": 332}, "test": {"score": 90.73688742197031, "calls": 85.60665853483533, "rescued": 2161, "harmed": 665}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.96247493554854, "calls": 95.9896877685477}, "test": {"score": 90.72253713137691, "calls": 95.98909377914903}}, "old_normalized_interior": {"val": {"score": 91.00544256659983, "calls": 76.99799484388427}, "test": {"score": 90.5503336442563, "calls": 76.86015641816746}}, "old_normalized_endpoints": {"val": {"score": 91.00544256659983, "calls": 76.99799484388427}, "test": {"score": 90.5503336442563, "calls": 76.86015641816746}}}, "baseline_old_normalized": {"val": {"score": [79.81953594958465, 80.37811515325122, 80.92237181323402, 81.3090804926955, 81.6814666284732, 82.03953022056717, 82.39759381266111, 82.75565740475508, 82.99914064737897, 83.34288169578916, 83.65797765683185, 83.9157834431395, 84.20223431681467, 84.4313950157548, 84.76081352048124, 85.03294185047264, 85.36236035519907, 85.4912632483529, 85.8063592093956, 85.94958464623316, 86.16442280148954, 86.37926095674591, 86.59409911200228, 86.72300200515612, 86.92351761672873, 86.98080779146376, 87.1383557719851, 87.26725866513893, 87.35319392724148, 87.42480664566027, 87.53938699513033, 87.65396734460039, 87.79719278143799, 87.95474076195931, 88.11228874248066, 88.22686909195072, 88.32712689773703, 88.4417072472071, 88.58493268404469, 88.74248066456603, 88.81409338298482, 88.92867373245488, 89.10054425665997, 89.17215697507876, 89.24376969349757, 89.31538241191636, 89.42996276138642, 89.51589802348897, 89.60183328559151, 89.73073617874535, 89.73073617874535, 89.81667144084788, 89.93125179031796, 90.03150959610426, 90.10312231452305, 90.11744485820681, 90.26067029504439, 90.34660555714694, 90.40389573188197, 90.46118590661702, 90.53279862503581, 90.56144371240332, 90.6044113434546, 90.57576625608708, 90.57576625608708, 90.61873388713835, 90.69034660555714, 90.76195932397593, 90.84789458607848, 90.86221712976223, 90.876539673446, 90.84789458607848, 90.86221712976223, 90.86221712976223, 90.86221712976223, 90.93382984818103, 90.96247493554854, 91.00544256659983, 90.9481523918648, 90.90518476081351, 90.96247493554854, 90.9767974792323, 90.93382984818103, 90.9767974792323, 90.96247493554854, 90.9767974792323, 90.9481523918648, 90.93382984818103, 90.89086221712975, 90.84789458607848, 90.81924949871096, 90.80492695502721, 90.76195932397593, 90.79060441134345, 90.71899169292466, 90.73331423660842, 90.73331423660842, 90.63305643082211, 90.59008879977084, 90.50415353766829, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [79.81953594958465, 80.37811515325122, 80.92237181323402, 81.3090804926955, 81.6814666284732, 82.03953022056717, 82.39759381266111, 82.75565740475508, 82.99914064737897, 83.34288169578916, 83.65797765683185, 83.9157834431395, 84.20223431681467, 84.4313950157548, 84.76081352048124, 85.03294185047264, 85.36236035519907, 85.4912632483529, 85.8063592093956, 85.94958464623316, 86.16442280148954, 86.37926095674591, 86.59409911200228, 86.72300200515612, 86.92351761672873, 86.98080779146376, 87.1383557719851, 87.26725866513893, 87.35319392724148, 87.42480664566027, 87.53938699513033, 87.65396734460039, 87.79719278143799, 87.95474076195931, 88.11228874248066, 88.22686909195072, 88.32712689773703, 88.4417072472071, 88.58493268404469, 88.74248066456603, 88.81409338298482, 88.92867373245488, 89.10054425665997, 89.17215697507876, 89.24376969349757, 89.31538241191636, 89.42996276138642, 89.51589802348897, 89.60183328559151, 89.73073617874535, 89.73073617874535, 89.81667144084788, 89.93125179031796, 90.03150959610426, 90.10312231452305, 90.11744485820681, 90.26067029504439, 90.34660555714694, 90.40389573188197, 90.46118590661702, 90.53279862503581, 90.56144371240332, 90.6044113434546, 90.57576625608708, 90.57576625608708, 90.61873388713835, 90.69034660555714, 90.76195932397593, 90.84789458607848, 90.86221712976223, 90.876539673446, 90.84789458607848, 90.86221712976223, 90.86221712976223, 90.86221712976223, 90.93382984818103, 90.96247493554854, 91.00544256659983, 90.9481523918648, 90.90518476081351, 90.96247493554854, 90.9767974792323, 90.93382984818103, 90.9767974792323, 90.96247493554854, 90.9767974792323, 90.9481523918648, 90.93382984818103, 90.89086221712975, 90.84789458607848, 90.81924949871096, 90.80492695502721, 90.76195932397593, 90.79060441134345, 90.71899169292466, 90.73331423660842, 90.73331423660842, 90.63305643082211, 90.59008879977084, 90.50415353766829, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 84.05900887997709, "exact_area100": 88.48256198510538}, "test": {"score": [80.00287005811867, 80.66298342541437, 81.02891583554567, 81.45224940805052, 81.99038530530244, 82.32044198895028, 82.77965128793858, 83.0523068092129, 83.41823921934419, 83.66219415943173, 83.8774485183325, 84.24338092846381, 84.48016072325466, 84.79586711630911, 85.02547176580326, 85.29812728707756, 85.48468106479157, 85.69276027839564, 85.87931405610964, 86.19502044916409, 86.40309966276818, 86.52507713281194, 86.72598120111932, 86.90535983353664, 87.01298701298703, 87.17084020951424, 87.42914544019517, 87.53677261964555, 87.68745067087609, 87.86682930329339, 88.01033220922724, 88.15383511516109, 88.28298773050155, 88.46954150821554, 88.53411781588576, 88.69914615770969, 88.878524790127, 88.971801678984, 89.12965487551124, 89.2588074908517, 89.38796010619215, 89.41666068737892, 89.48123699504916, 89.59603931979623, 89.73236708043338, 89.78259309751023, 89.8973954222573, 89.95479658463084, 90.01937289230108, 90.0839491999713, 90.14852550764154, 90.24897754179523, 90.30637870416876, 90.3566047212456, 90.42118102891584, 90.47858219128938, 90.56468393484968, 90.56468393484968, 90.60055966133315, 90.62926024251992, 90.6364353878166, 90.67948625959676, 90.744062567267, 90.77276314845376, 90.70818684078354, 90.73688742197031, 90.67948625959676, 90.64361053311329, 90.66513596900337, 90.67948625959676, 90.70818684078354, 90.65078567840999, 90.69383655019014, 90.65078567840999, 90.62208509722322, 90.59338451603645, 90.59338451603645, 90.5503336442563, 90.54315849895961, 90.51445791777284, 90.50728277247615, 90.54315849895961, 90.54315849895961, 90.52163306306953, 90.54315849895961, 90.52880820836621, 90.51445791777284, 90.52880820836621, 90.51445791777284, 90.46423190069599, 90.41400588361914, 90.37813015713569, 90.38530530243239, 90.3566047212456, 90.30637870416876, 90.27050297768531, 90.24897754179523, 90.19157637942169, 90.1700509435316, 90.13417521704815, 90.06242376408122], "calls": [0.0, 1.2341249910310683, 2.080792136040755, 3.200114802324747, 4.534691827509507, 5.474635861376194, 6.565257946473416, 7.440625672669872, 8.402095142426633, 9.449666355743704, 10.267632919566621, 11.336729568773768, 12.276673602640454, 13.553849465451675, 14.436392336944822, 15.448087823778431, 16.380856712348425, 17.098371242017652, 18.08136614766449, 19.4661691899261, 20.506565257946473, 21.281480949989238, 22.207074693262538, 23.347922795436606, 24.302217119896678, 25.062782521346055, 26.167754897036666, 26.928320298486046, 27.95436607591304, 28.87995981918634, 30.257587716151253, 31.66391619430293, 32.474707612829164, 33.414651646695845, 34.31871995407907, 35.24431369735237, 36.083805697065365, 37.4470833034369, 38.408552773193655, 39.39872282413719, 40.48934490923441, 41.32883690894741, 42.326182105187634, 43.28765157494439, 44.42849967711846, 45.40431943746861, 46.19358542010475, 47.183755471048286, 48.18827581258521, 49.22867188060558, 50.30494367510943, 51.230537418382724, 52.33550979407333, 53.225227810863174, 54.15799669943316, 54.96878811795939, 56.23161369017723, 57.52313984358183, 58.692688526942675, 59.64698285140274, 60.36449738107197, 61.13941307311473, 61.89997847456411, 63.13410346559518, 64.05252206357179, 65.13596900337232, 66.42032001148023, 67.37461433594031, 68.50828729281768, 69.75676257444213, 70.63213030063858, 71.47162230035158, 72.22501255650427, 73.07167970151396, 74.14077635072111, 75.1596469828514, 76.00631412786109, 76.86015641816746, 77.78575016144077, 78.84049652005453, 80.03874578460214, 80.97868981846882, 81.81100667288513, 83.00925593743274, 83.99942598837626, 84.86044342397933, 86.08739326971371, 86.90535983353664, 88.07490851689747, 88.92157566190716, 89.8041185334003, 90.77276314845376, 91.70553203702376, 92.74592810504413, 93.66434670302074, 94.69039248044773, 95.72361340317141, 96.81423548826864, 97.94790844514601, 98.89502762430939, 100.0], "exact_score": [80.00287005811867, 80.56970653655738, 81.01456554495229, 81.38767310038028, 81.78230609169836, 82.1267130659396, 82.52852120255434, 82.90162875798235, 83.24603573222358, 83.56174212527804, 83.81287221066228, 84.12857860371673, 84.41558441558442, 84.63801391978187, 84.94654516753964, 85.14744923584703, 85.43445504771472, 85.65688455191219, 85.87213891081295, 86.11609385090048, 86.24524646624094, 86.47485111573509, 86.70445576522926, 86.82643323527302, 86.99146157709694, 87.14931477362417, 87.38609456841502, 87.53677261964555, 87.70897610676617, 87.91705532037024, 87.96728133744709, 88.08925880749085, 88.23276171342471, 88.40496520054532, 88.52694267058908, 88.64174499533615, 88.878524790127, 88.95027624309392, 89.10095429432447, 89.20858147377486, 89.33055894381862, 89.4310109779723, 89.47406184975246, 89.55298844801608, 89.68214106335654, 89.75389251632346, 89.86151969577384, 89.93327114874077, 90.00502260170768, 90.06242376408122, 90.12700007175145, 90.20592667001507, 90.28485326827868, 90.32790414005885, 90.39248044772907, 90.47140704599268, 90.54315849895961, 90.55750878955298, 90.59338451603645, 90.60773480662984, 90.60773480662984, 90.67231111430006, 90.744062567267, 90.77276314845376, 90.70101169548683, 90.744062567267, 90.67948625959676, 90.62926024251992, 90.65078567840999, 90.65796082370669, 90.66513596900337, 90.66513596900337, 90.67231111430006, 90.65078567840999, 90.62926024251992, 90.60773480662984, 90.58620937073975, 90.57185908014638, 90.53598335366291, 90.50728277247615, 90.50728277247615, 90.54315849895961, 90.5503336442563, 90.52163306306953, 90.54315849895961, 90.52163306306953, 90.52163306306953, 90.52880820836621, 90.50010762717945, 90.47140704599268, 90.39965559302576, 90.3637798665423, 90.38530530243239, 90.3566047212456, 90.30637870416876, 90.26332783238861, 90.24180239649853, 90.18440123412499, 90.1700509435316, 90.12700007175145, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 84.297194518189, "exact_area100": 88.5345492409958}}, "old_normalized_endpoint_budget": 77}]}, {"expert": "vilt", "endpoint": "MiniCPM", "task": "nlvr2", "small": 76.42247255506925, "large": 90.7368874219703, "runs": [{"seed": 2026, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 1.0, "validation_area100": 0.8550970490800871, "training_seconds": 37.25677033094689, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6662795029022436, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6084758338384576, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5644565887510075, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5287704560987748, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4988894902026254, "operator": "ge"}, {"budget": 0.06, "threshold": 0.47372354111112097, "operator": "ge"}, {"budget": 0.07, "threshold": 0.45116666237308317, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4254612921867018, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4004955851042756, "operator": "ge"}, {"budget": 0.1, "threshold": 0.38347901887930935, "operator": "ge"}, {"budget": 0.11, "threshold": 0.365883803267281, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3497192085102476, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3302660673634835, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3143926009002725, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3008564337300634, "operator": "ge"}, {"budget": 0.16, "threshold": 0.27889469357443347, "operator": "ge"}, {"budget": 0.17, "threshold": 0.26282757745274343, "operator": "ge"}, {"budget": 0.18, "threshold": 0.24614215063276826, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2275097277354351, "operator": "ge"}, {"budget": 0.2, "threshold": 0.2088431798173098, "operator": "ge"}, {"budget": 0.21, "threshold": 0.18590716130660234, "operator": "ge"}, {"budget": 0.22, "threshold": 0.16650508981212628, "operator": "ge"}, {"budget": 0.23, "threshold": 0.1425795631950353, "operator": "ge"}, {"budget": 0.24, "threshold": 0.12264342243249865, "operator": "ge"}, {"budget": 0.25, "threshold": 0.10316697723522943, "operator": "ge"}, {"budget": 0.26, "threshold": 0.08125514833121175, "operator": "ge"}, {"budget": 0.27, "threshold": 0.05746528486579257, "operator": "ge"}, {"budget": 0.28, "threshold": 0.02548894687793129, "operator": "ge"}, {"budget": 0.29, "threshold": -0.014248376164476785, "operator": "ge"}, {"budget": 0.3, "threshold": -0.06196124347209871, "operator": "ge"}, {"budget": 0.31, "threshold": -0.11588252548198659, "operator": "ge"}, {"budget": 0.32, "threshold": -0.16519928028992054, "operator": "ge"}, {"budget": 0.33, "threshold": -0.22685416262488772, "operator": "ge"}, {"budget": 0.34, "threshold": -0.2892055478164411, "operator": "ge"}, {"budget": 0.35, "threshold": -0.34227165155786543, "operator": "ge"}, {"budget": 0.36, "threshold": -0.39936255445580404, "operator": "ge"}, {"budget": 0.37, "threshold": -0.44946916529079317, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5011826954887892, "operator": "ge"}, {"budget": 0.39, "threshold": -0.5474301867702167, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6012170808775232, "operator": "ge"}, {"budget": 0.41, "threshold": -0.6499122670318379, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6846884600074143, "operator": "ge"}, {"budget": 0.43, "threshold": -0.7234269275797508, "operator": "ge"}, {"budget": 0.44, "threshold": -0.7545164430045844, "operator": "ge"}, {"budget": 0.45, "threshold": -0.7894929497991287, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8159134239637637, "operator": "ge"}, {"budget": 0.47, "threshold": -0.8380439018356297, "operator": "ge"}, {"budget": 0.48, "threshold": -0.859786305286076, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8798466297057604, "operator": "ge"}, {"budget": 0.5, "threshold": -0.8968909527630095, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9094494916920371, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9170007464203809, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9250487530311261, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9305704648422145, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9363167613437092, "operator": "ge"}, {"budget": 0.56, "threshold": -0.940483447937465, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9447336387589872, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9482340220572016, "operator": "ge"}, {"budget": 0.59, "threshold": -0.951254052875842, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9544968732158275, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9570257993947276, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9593393585801494, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9614523483259385, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9637002062084078, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9656988978084292, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9671503453455308, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9687521277312108, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9703890581013968, "operator": "ge"}, {"budget": 0.69, "threshold": -0.972381480050404, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9740021906853304, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9755188571858187, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9768470135897483, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9781407780712258, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9794198331945606, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9807374428627847, "operator": "ge"}, {"budget": 0.76, "threshold": -0.982037710002637, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9833127309622364, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9844949449659115, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9855400551598458, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9867727710576544, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9876360149870175, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9883759819145878, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9893129770038692, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9900914224938269, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9909715069567363, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9917778947928944, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9924040353034418, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9930012233045197, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9935441151221288, "operator": "ge"}, {"budget": 0.9, "threshold": -0.99404512873852, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9946124387144317, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9951507296061015, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9956242548958307, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9960665830827504, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9964182521416918, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9967985498670368, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9972584241234621, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9977306385982262, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9982047405415322, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.88083643655112, 76.15296476654252, 76.56831853337152, 76.7831566886279, 77.25580063019191, 77.59954167860211, 77.97192781437982, 78.31566886279003, 78.6307648238327, 78.90289315382411, 79.23231165855056, 79.51876253222572, 79.83385849326841, 80.09166427957605, 80.32082497851619, 80.44972787167, 80.50701804640504, 80.93669435691778, 81.23746777427671, 81.45230592953308, 81.69578917215698, 81.95359495846462, 82.08249785161844, 82.22572328845604, 82.45488398739616, 82.76997994843884, 82.97049556001146, 83.05643082211401, 83.28559152105413, 83.51475221999426, 83.80120309366943, 83.94442853050701, 84.23087940418218, 84.44571755943855, 84.68920080206244, 85.00429676310513, 85.17616728731022, 85.36236035519909, 85.51990833572042, 85.72042394729303, 85.97822973360068, 86.15010025780579, 86.22171297622458, 86.36493841306215, 86.55113148095101, 86.63706674305357, 86.65138928673733, 86.76596963620739, 86.86622744199369, 86.98080779146376, 87.12403322830134, 87.19564594672013, 87.29590375250645, 87.35319392724148, 87.49641936407906, 87.6539673446004, 87.7255800630192, 87.89745058722428, 87.9260956745918, 87.9260956745918, 87.9260956745918, 87.9260956745918, 88.02635348037812, 88.11228874248066, 88.15525637353194, 88.25551417931825, 88.34144944142079, 88.4417072472071, 88.52764250930964, 88.6135777714122, 88.64222285877972, 88.69951303351475, 88.85706101403609, 88.92867373245488, 88.98596390718991, 89.01460899455743, 89.10054425665999, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.4013176740189, 89.44428530507018, 89.51589802348897, 89.60183328559151, 89.573188198224, 89.5445431108565, 89.53022056717272, 89.55886565454024, 89.6734460040103, 89.75938126611285, 89.7450587224291, 89.78802635348038, 89.83099398453164, 89.84531652821542, 89.85963907189917, 89.85963907189917, 89.88828415926669, 89.85963907189917, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 26, 52, 83, 103, 140, 166, 192, 220, 246, 271, 300, 327, 353, 374, 397, 415, 427, 459, 487, 510, 535, 562, 580, 598, 623, 647, 665, 678, 704, 725, 752, 767, 787, 806, 827, 850, 869, 883, 899, 915, 939, 955, 969, 986, 1001, 1011, 1020, 1033, 1044, 1054, 1067, 1076, 1088, 1095, 1109, 1123, 1132, 1145, 1151, 1156, 1162, 1164, 1171, 1180, 1186, 1197, 1204, 1213, 1220, 1228, 1236, 1243, 1256, 1261, 1267, 1272, 1281, 1288, 1297, 1306, 1312, 1320, 1326, 1334, 1342, 1342, 1343, 1344, 1349, 1358, 1364, 1366, 1370, 1374, 1376, 1379, 1382, 1385, 1386, 1389], "harmed": [0, 4, 11, 13, 18, 22, 24, 24, 28, 32, 38, 44, 51, 55, 58, 65, 74, 82, 84, 91, 99, 107, 116, 125, 133, 142, 144, 148, 155, 165, 170, 177, 182, 182, 186, 190, 191, 198, 199, 204, 206, 212, 216, 225, 232, 234, 238, 246, 251, 255, 257, 260, 264, 269, 272, 276, 279, 283, 284, 288, 293, 299, 301, 301, 304, 307, 311, 312, 314, 315, 317, 323, 326, 328, 328, 330, 333, 336, 339, 343, 349, 351, 354, 357, 360, 362, 364, 367, 369, 372, 373, 373, 376, 377, 378, 379, 381, 384, 385, 388, 388], "exact_score": [75.56574047550845, 75.88083643655112, 76.15296476654252, 76.56831853337152, 76.7831566886279, 77.25580063019191, 77.59954167860211, 77.97192781437982, 78.31566886279003, 78.6307648238327, 78.90289315382411, 79.23231165855056, 79.51876253222572, 79.83385849326841, 80.09166427957605, 80.32082497851619, 80.44972787167, 80.50701804640504, 80.93669435691778, 81.23746777427671, 81.45230592953308, 81.69578917215698, 81.95359495846462, 82.08249785161844, 82.22572328845604, 82.45488398739616, 82.76997994843884, 82.97049556001146, 83.05643082211401, 83.28559152105413, 83.51475221999426, 83.80120309366943, 83.94442853050701, 84.23087940418218, 84.44571755943855, 84.68920080206244, 85.00429676310513, 85.17616728731022, 85.36236035519909, 85.51990833572042, 85.72042394729303, 85.97822973360068, 86.15010025780579, 86.22171297622458, 86.36493841306215, 86.55113148095101, 86.63706674305357, 86.65138928673733, 86.76596963620739, 86.86622744199369, 86.98080779146376, 87.12403322830134, 87.19564594672013, 87.29590375250645, 87.35319392724148, 87.49641936407906, 87.6539673446004, 87.7255800630192, 87.89745058722428, 87.9260956745918, 87.9260956745918, 87.9260956745918, 87.9260956745918, 88.02635348037812, 88.11228874248066, 88.15525637353194, 88.25551417931825, 88.34144944142079, 88.4417072472071, 88.52764250930964, 88.6135777714122, 88.64222285877972, 88.69951303351475, 88.85706101403609, 88.92867373245488, 88.98596390718991, 89.01460899455743, 89.10054425665999, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.4013176740189, 89.44428530507018, 89.51589802348897, 89.60183328559151, 89.573188198224, 89.5445431108565, 89.53022056717272, 89.55886565454024, 89.6734460040103, 89.75938126611285, 89.7450587224291, 89.78802635348038, 89.83099398453164, 89.84531652821542, 89.85963907189917, 89.85963907189917, 89.88828415926669, 89.85963907189917, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.50970490800871, "threshold_area100": 85.50866513892868, "threshold_area25": 79.38842738470352}, "test": {"score": [76.42247255506925, 76.82428069168401, 77.12563679414508, 77.49156920427639, 77.89337734089116, 78.2162588792423, 78.38128722106623, 78.6826433235273, 79.01270000717516, 79.3355815455263, 79.50778503264692, 79.83066657099806, 80.06027122049221, 80.36880246825, 80.59840711774415, 80.84236205783168, 81.17959388677622, 81.36614766449021, 81.58857716868766, 81.89710841644543, 82.1984645189065, 82.42089402310397, 82.58592236492788, 82.87292817679558, 83.03078137332281, 83.18145942455335, 83.43258950993759, 83.66219415943173, 83.89897395422258, 84.1429288943101, 84.37970868910097, 84.6954150821554, 84.91784458635287, 85.19050010762719, 85.27660185118748, 85.49185621008826, 85.73581115017579, 85.97259094496664, 86.23089617564756, 86.43897538925164, 86.69010547463587, 86.90535983353664, 87.1134390471407, 87.26411709837124, 87.46502116667862, 87.62287436320587, 87.75202697854631, 87.84530386740332, 87.96010619215039, 88.06055822630408, 88.21841142283131, 88.31168831168831, 88.39061490995192, 88.505417234699, 88.56999354236923, 88.65609528592955, 88.77089761067663, 88.86417449953362, 88.96462653368731, 89.04355313195093, 89.12965487551124, 89.20858147377486, 89.20858147377486, 89.28750807203846, 89.33055894381862, 89.3520843797087, 89.46688670445577, 89.49558728564253, 89.56016359331277, 89.63909019157639, 89.70366649924661, 89.79694338810361, 89.84716940518047, 89.96197172992754, 90.00502260170768, 90.04807347348785, 90.10547463586138, 90.15570065293824, 90.24180239649853, 90.277678122982, 90.30637870416876, 90.30637870416876, 90.33507928535553, 90.39248044772907, 90.42835617421254, 90.49293248188276, 90.60773480662984, 90.60055966133315, 90.60773480662984, 90.65078567840999, 90.66513596900337, 90.67948625959676, 90.71536198608023, 90.744062567267, 90.70818684078353, 90.70818684078353, 90.69383655019014, 90.66513596900337, 90.67948625959676, 90.744062567267, 90.73688742197031], "calls": [0.0, 1.1049723756906076, 2.131018153117601, 3.1211882040611325, 4.305087178015355, 5.316782664848963, 6.07017292100165, 6.974241228384876, 8.021812441701945, 9.105259381502476, 10.095429432446007, 10.884695415082156, 11.702661978905073, 12.628255722178375, 13.424696850111214, 14.192437396857285, 15.340460644328047, 16.244528951711274, 17.198823276171343, 18.23204419889503, 19.22938939513525, 20.320011480232473, 21.0375260099017, 22.18554925737246, 23.154193872425914, 23.979335581545527, 24.854703307741982, 25.80899763220205, 26.89244457200258, 27.825213460572577, 29.04498816101026, 30.214536844371096, 31.154480878237784, 32.367080433378774, 33.57250484322308, 34.61290091124346, 35.674822415153905, 36.67934275669082, 37.71256367941451, 38.79601061921504, 40.2238645332568, 41.39341321661764, 42.41228384874794, 43.3737533185047, 44.36392336944823, 45.497596326325606, 46.30838774485183, 47.00437683863098, 47.958671163091054, 49.13539499174858, 50.254717658032575, 51.46014206787688, 52.14895601635933, 53.23957810145655, 54.14364640883977, 55.21274305804692, 56.06658534835331, 57.31506062997775, 58.40568271507498, 59.424553347205276, 60.6156274664562, 61.383368013202265, 62.25156059410203, 63.26325608093563, 64.39692903781301, 65.35122336227307, 66.11178876372247, 67.01585707110569, 68.01320226734592, 69.20427638659683, 70.29489847169405, 71.20614192437397, 72.13173566764726, 73.10755542799741, 74.12642606012771, 75.2457487264117, 76.38659682858578, 77.49156920427639, 78.91224797302145, 79.90241802396498, 81.0934921432159, 82.1051876300495, 82.95902992035589, 84.03530171485973, 84.92501973164957, 86.03716725263688, 87.1206141924374, 87.90270502977685, 88.67762072181962, 89.58168902920283, 90.39248044772906, 91.49745282341968, 92.60242519911029, 93.42039176293319, 94.43208724976681, 95.25722895888642, 96.12542153978619, 97.08689100954294, 98.06988591518979, 99.03853053024324, 100.0], "rescued": [0, 58, 107, 163, 226, 274, 304, 353, 409, 461, 502, 551, 594, 643, 681, 723, 781, 819, 861, 917, 972, 1024, 1057, 1114, 1147, 1177, 1220, 1261, 1305, 1348, 1394, 1446, 1484, 1537, 1570, 1607, 1650, 1689, 1735, 1774, 1823, 1861, 1896, 1923, 1962, 1988, 2013, 2033, 2056, 2080, 2112, 2136, 2151, 2175, 2192, 2212, 2233, 2256, 2277, 2292, 2309, 2323, 2330, 2349, 2359, 2369, 2386, 2395, 2406, 2420, 2436, 2455, 2466, 2483, 2493, 2503, 2515, 2527, 2546, 2561, 2573, 2582, 2590, 2602, 2611, 2623, 2640, 2646, 2652, 2664, 2668, 2673, 2681, 2687, 2689, 2692, 2692, 2694, 2696, 2706, 2706], "harmed": [0, 2, 9, 14, 21, 24, 31, 38, 48, 55, 72, 76, 87, 93, 99, 107, 118, 130, 141, 154, 167, 188, 198, 215, 226, 235, 243, 252, 263, 272, 285, 293, 300, 315, 336, 343, 352, 358, 368, 378, 392, 400, 406, 412, 423, 427, 434, 441, 448, 458, 468, 479, 483, 491, 499, 507, 512, 522, 529, 533, 538, 541, 548, 556, 560, 567, 568, 573, 575, 578, 585, 591, 595, 596, 600, 604, 608, 613, 620, 630, 638, 647, 651, 655, 659, 662, 663, 670, 675, 681, 683, 686, 689, 691, 698, 701, 703, 709, 709, 710, 711], "exact_score": [76.42247255506925, 76.77405467460716, 77.07541077706824, 77.44851833249623, 77.78575016144077, 78.10863169979191, 78.35976178517615, 78.68981846882399, 78.99117457128507, 79.32123125493291, 79.5149601779436, 79.86654229748153, 80.1678983999426, 80.51230537418384, 80.76343545956806, 81.08631699791921, 81.2800459209299, 81.50965057042406, 81.82535696347851, 82.10518763004951, 82.3993685872139, 82.58592236492788, 82.82987730501543, 83.00208079213604, 83.17428427925665, 83.46129009112435, 83.6837195953218, 83.93484968070604, 84.18597976609027, 84.37970868910097, 84.6093133385951, 84.9034942957595, 85.0828729281768, 85.25507641529741, 85.3627035947478, 85.59948338953865, 85.80038745784603, 86.05869268852695, 86.2882973380211, 86.46050082514172, 86.6542297481524, 86.8479586711631, 87.02016215828371, 87.22824137188778, 87.37891942311832, 87.52242232905216, 87.73767668795294, 87.84530386740332, 87.96728133744709, 88.0462079357107, 88.15383511516109, 88.26146229461148, 88.36908947406185, 88.49824208940231, 88.56281839707253, 88.63456985003947, 88.76372246537993, 88.84264906364353, 88.95027624309392, 88.98615196957739, 89.07942885843438, 89.17270574729139, 89.21575661907154, 89.2659826361484, 89.30903350792853, 89.32338379852193, 89.45253641386238, 89.49558728564253, 89.56016359331277, 89.6175647556863, 89.66061562746646, 89.76106766162016, 89.82564396929038, 89.95479658463084, 89.997847456411, 90.04807347348785, 90.06959890937792, 90.14135036234484, 90.18440123412499, 90.24897754179523, 90.29920355887207, 90.30637870416876, 90.29920355887207, 90.32072899476215, 90.39248044772907, 90.43553131950922, 90.49293248188276, 90.58620937073975, 90.58620937073975, 90.6364353878166, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.72253713137691, 90.70818684078353, 90.71536198608023, 90.69383655019014, 90.67231111430006, 90.67948625959676, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.48837407917831, "threshold_area100": 86.46068020377413, "threshold_area25": 80.11336729568775}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.375366747379303, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3379790484905243, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3110315203666687, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2908889353275299, "operator": "ge"}, {"budget": 0.05, "threshold": 0.2680003345012665, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2515815198421478, "operator": "ge"}, {"budget": 0.07, "threshold": 0.23431086540222168, "operator": "ge"}, {"budget": 0.08, "threshold": 0.21818141639232635, "operator": "ge"}, {"budget": 0.09, "threshold": 0.20448338985443115, "operator": "ge"}, {"budget": 0.1, "threshold": 0.19382721185684204, "operator": "ge"}, {"budget": 0.11, "threshold": 0.18085405230522156, "operator": "ge"}, {"budget": 0.12, "threshold": 0.17157630622386932, "operator": "ge"}, {"budget": 0.13, "threshold": 0.16061067581176758, "operator": "ge"}, {"budget": 0.14, "threshold": 0.1493714302778244, "operator": "ge"}, {"budget": 0.15, "threshold": 0.13927814364433289, "operator": "ge"}, {"budget": 0.16, "threshold": 0.1275421380996704, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1171954795718193, "operator": "ge"}, {"budget": 0.18, "threshold": 0.10923883318901062, "operator": "ge"}, {"budget": 0.19, "threshold": 0.09942708909511566, "operator": "ge"}, {"budget": 0.2, "threshold": 0.09038028120994568, "operator": "ge"}, {"budget": 0.21, "threshold": 0.08172013610601425, "operator": "ge"}, {"budget": 0.22, "threshold": 0.07266479730606079, "operator": "ge"}, {"budget": 0.23, "threshold": 0.06220176815986633, "operator": "ge"}, {"budget": 0.24, "threshold": 0.05044066160917282, "operator": "ge"}, {"budget": 0.25, "threshold": 0.039996638894081116, "operator": "ge"}, {"budget": 0.26, "threshold": 0.02922394871711731, "operator": "ge"}, {"budget": 0.27, "threshold": 0.017132416367530823, "operator": "ge"}, {"budget": 0.28, "threshold": 0.007575593888759613, "operator": "ge"}, {"budget": 0.29, "threshold": -0.00367804616689682, "operator": "ge"}, {"budget": 0.3, "threshold": -0.0076773762702941895, "operator": "ge"}, {"budget": 0.31, "threshold": -0.00953109934926033, "operator": "ge"}, {"budget": 0.32, "threshold": -0.010324697941541672, "operator": "ge"}, {"budget": 0.33, "threshold": -0.011308201588690281, "operator": "ge"}, {"budget": 0.34, "threshold": -0.012119412422180176, "operator": "ge"}, {"budget": 0.35, "threshold": -0.012884044088423252, "operator": "ge"}, {"budget": 0.36, "threshold": -0.01385439746081829, "operator": "ge"}, {"budget": 0.37, "threshold": -0.014586996287107468, "operator": "ge"}, {"budget": 0.38, "threshold": -0.01526931393891573, "operator": "ge"}, {"budget": 0.39, "threshold": -0.016274865716695786, "operator": "ge"}, {"budget": 0.4, "threshold": -0.016931600868701935, "operator": "ge"}, {"budget": 0.41, "threshold": -0.01803322695195675, "operator": "ge"}, {"budget": 0.42, "threshold": -0.018792442977428436, "operator": "ge"}, {"budget": 0.43, "threshold": -0.01978888548910618, "operator": "ge"}, {"budget": 0.44, "threshold": -0.02066819742321968, "operator": "ge"}, {"budget": 0.45, "threshold": -0.021639127284288406, "operator": "ge"}, {"budget": 0.46, "threshold": -0.022603079676628113, "operator": "ge"}, {"budget": 0.47, "threshold": -0.023502200841903687, "operator": "ge"}, {"budget": 0.48, "threshold": -0.024518044665455818, "operator": "ge"}, {"budget": 0.49, "threshold": -0.02534373104572296, "operator": "ge"}, {"budget": 0.5, "threshold": -0.026531841605901718, "operator": "ge"}, {"budget": 0.51, "threshold": -0.027688052505254745, "operator": "ge"}, {"budget": 0.52, "threshold": -0.0286659337580204, "operator": "ge"}, {"budget": 0.53, "threshold": -0.029751474037766457, "operator": "ge"}, {"budget": 0.54, "threshold": -0.03104998916387558, "operator": "ge"}, {"budget": 0.55, "threshold": -0.032201915979385376, "operator": "ge"}, {"budget": 0.56, "threshold": -0.03323090821504593, "operator": "ge"}, {"budget": 0.57, "threshold": -0.03425837308168411, "operator": "ge"}, {"budget": 0.58, "threshold": -0.035372547805309296, "operator": "ge"}, {"budget": 0.59, "threshold": -0.036719001829624176, "operator": "ge"}, {"budget": 0.6, "threshold": -0.03794762119650841, "operator": "ge"}, {"budget": 0.61, "threshold": -0.03895614296197891, "operator": "ge"}, {"budget": 0.62, "threshold": -0.040449146181344986, "operator": "ge"}, {"budget": 0.63, "threshold": -0.04196557402610779, "operator": "ge"}, {"budget": 0.64, "threshold": -0.04319366440176964, "operator": "ge"}, {"budget": 0.65, "threshold": -0.04450373351573944, "operator": "ge"}, {"budget": 0.66, "threshold": -0.0457957461476326, "operator": "ge"}, {"budget": 0.67, "threshold": -0.04726768285036087, "operator": "ge"}, {"budget": 0.68, "threshold": -0.04843372479081154, "operator": "ge"}, {"budget": 0.69, "threshold": -0.049873583018779755, "operator": "ge"}, {"budget": 0.7, "threshold": -0.051163945347070694, "operator": "ge"}, {"budget": 0.71, "threshold": -0.0522572360932827, "operator": "ge"}, {"budget": 0.72, "threshold": -0.053603798151016235, "operator": "ge"}, {"budget": 0.73, "threshold": -0.054774440824985504, "operator": "ge"}, {"budget": 0.74, "threshold": -0.056161895394325256, "operator": "ge"}, {"budget": 0.75, "threshold": -0.05751712620258331, "operator": "ge"}, {"budget": 0.76, "threshold": -0.059120841324329376, "operator": "ge"}, {"budget": 0.77, "threshold": -0.060963574796915054, "operator": "ge"}, {"budget": 0.78, "threshold": -0.06269712001085281, "operator": "ge"}, {"budget": 0.79, "threshold": -0.06453341245651245, "operator": "ge"}, {"budget": 0.8, "threshold": -0.06634536385536194, "operator": "ge"}, {"budget": 0.81, "threshold": -0.06808751821517944, "operator": "ge"}, {"budget": 0.82, "threshold": -0.06980098783969879, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0719824880361557, "operator": "ge"}, {"budget": 0.84, "threshold": -0.07428515702486038, "operator": "ge"}, {"budget": 0.85, "threshold": -0.0763547345995903, "operator": "ge"}, {"budget": 0.86, "threshold": -0.07884011417627335, "operator": "ge"}, {"budget": 0.87, "threshold": -0.08145882189273834, "operator": "ge"}, {"budget": 0.88, "threshold": -0.08389105647802353, "operator": "ge"}, {"budget": 0.89, "threshold": -0.08740770816802979, "operator": "ge"}, {"budget": 0.9, "threshold": -0.09116537123918533, "operator": "ge"}, {"budget": 0.91, "threshold": -0.09495160728693008, "operator": "ge"}, {"budget": 0.92, "threshold": -0.09899584949016571, "operator": "ge"}, {"budget": 0.93, "threshold": -0.10334943234920502, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10894119739532471, "operator": "ge"}, {"budget": 0.95, "threshold": -0.11354709416627884, "operator": "ge"}, {"budget": 0.96, "threshold": -0.12010059505701065, "operator": "ge"}, {"budget": 0.97, "threshold": -0.12864530086517334, "operator": "ge"}, {"budget": 0.98, "threshold": -0.14023108780384064, "operator": "ge"}, {"budget": 0.99, "threshold": -0.15563204884529114, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.86651389286737, 76.1672873102263, 76.539673446004, 76.89773703809797, 77.19851045545688, 77.71412202807218, 77.97192781437982, 78.30134631910627, 78.57347464909768, 78.81695789172156, 79.20366657118304, 79.38985963907189, 79.56173016327699, 79.81953594958465, 80.13463191062733, 80.40676024061874, 80.76482383271268, 80.90804926955026, 81.23746777427671, 81.5525637353194, 81.73875680320825, 81.91062732741334, 82.12546548266972, 82.39759381266113, 82.54081924949871, 82.66972214265253, 82.92752792896017, 83.04210827843025, 83.28559152105413, 83.3715267831567, 83.38584932684044, 83.4001718705242, 83.48610713262676, 83.5004296763105, 83.54339730736179, 83.58636493841306, 83.57204239472931, 83.60068748209682, 83.6436551131481, 83.68662274419937, 83.75823546261816, 83.82984818103695, 83.95875107419077, 84.00171870524204, 84.00171870524204, 84.00171870524204, 84.07333142366083, 84.15926668576338, 84.21655686049843, 84.31681466628473, 84.34545975365224, 84.4313950157548, 84.48868519048983, 84.61758808364365, 84.61758808364365, 84.71784588942997, 84.90403895731882, 84.96132913205385, 85.06158693784016, 85.20481237467774, 85.29074763678028, 85.37668289888283, 85.47694070466915, 85.60584359782297, 85.66313377255801, 85.7347464909768, 85.7347464909768, 85.79203666571182, 85.92093955886565, 85.92093955886565, 86.09281008307075, 86.15010025780579, 86.26468060727585, 86.33629332569464, 86.39358350042967, 86.65138928673733, 86.78029217989115, 86.95216270409625, 87.10971068461758, 87.16700085935261, 87.38183901460899, 87.5823546261816, 87.68261243196791, 87.82583786880551, 87.95474076195933, 88.04067602406187, 88.12661128616442, 88.14093382984818, 88.2411916356345, 88.38441707247206, 88.59925522772845, 88.72815812088226, 88.85706101403609, 89.02893153824118, 89.24376969349757, 89.35835004296763, 89.51589802348897, 89.51589802348897, 89.7450587224291, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 28, 52, 82, 111, 138, 175, 198, 225, 250, 274, 305, 328, 348, 373, 397, 422, 451, 469, 499, 527, 550, 573, 591, 611, 625, 642, 665, 680, 703, 711, 714, 717, 723, 725, 729, 734, 735, 739, 744, 747, 754, 760, 770, 776, 780, 783, 792, 799, 806, 817, 821, 830, 835, 845, 846, 854, 868, 876, 884, 897, 907, 918, 928, 938, 943, 951, 956, 964, 974, 978, 994, 1004, 1015, 1023, 1030, 1049, 1061, 1076, 1090, 1097, 1117, 1132, 1145, 1158, 1170, 1181, 1191, 1201, 1211, 1223, 1242, 1253, 1269, 1285, 1305, 1323, 1340, 1348, 1373, 1389], "harmed": [0, 7, 10, 14, 18, 24, 25, 30, 34, 40, 47, 51, 61, 69, 76, 78, 84, 88, 96, 103, 109, 119, 130, 133, 134, 138, 146, 151, 158, 164, 166, 168, 170, 170, 171, 172, 174, 176, 178, 180, 180, 182, 183, 184, 187, 191, 194, 198, 199, 202, 206, 208, 211, 212, 213, 214, 215, 216, 220, 221, 224, 228, 233, 236, 237, 238, 241, 246, 250, 251, 255, 259, 265, 268, 271, 274, 275, 278, 281, 284, 287, 292, 293, 299, 302, 305, 310, 314, 323, 326, 328, 332, 334, 341, 345, 350, 360, 366, 374, 383, 388], "exact_score": [75.56574047550845, 75.86651389286737, 76.1672873102263, 76.539673446004, 76.89773703809797, 77.19851045545688, 77.71412202807218, 77.97192781437982, 78.30134631910627, 78.57347464909768, 78.81695789172156, 79.20366657118304, 79.38985963907189, 79.56173016327699, 79.81953594958465, 80.13463191062733, 80.40676024061874, 80.76482383271268, 80.90804926955026, 81.23746777427671, 81.5525637353194, 81.73875680320825, 81.91062732741334, 82.12546548266972, 82.39759381266113, 82.54081924949871, 82.66972214265253, 82.92752792896017, 83.04210827843025, 83.28559152105413, 83.3715267831567, 83.38584932684044, 83.4001718705242, 83.48610713262676, 83.5004296763105, 83.54339730736179, 83.58636493841306, 83.57204239472931, 83.60068748209682, 83.6436551131481, 83.68662274419937, 83.75823546261816, 83.82984818103695, 83.95875107419077, 84.00171870524204, 84.00171870524204, 84.00171870524204, 84.07333142366083, 84.15926668576338, 84.21655686049843, 84.31681466628473, 84.34545975365224, 84.4313950157548, 84.48868519048983, 84.61758808364365, 84.61758808364365, 84.71784588942997, 84.90403895731882, 84.96132913205385, 85.06158693784016, 85.20481237467774, 85.29074763678028, 85.37668289888283, 85.47694070466915, 85.60584359782297, 85.66313377255801, 85.7347464909768, 85.7347464909768, 85.79203666571182, 85.92093955886565, 85.92093955886565, 86.09281008307075, 86.15010025780579, 86.26468060727585, 86.33629332569464, 86.39358350042967, 86.65138928673733, 86.78029217989115, 86.95216270409625, 87.10971068461758, 87.16700085935261, 87.38183901460899, 87.5823546261816, 87.68261243196791, 87.82583786880551, 87.95474076195933, 88.04067602406187, 88.12661128616442, 88.14093382984818, 88.2411916356345, 88.38441707247206, 88.59925522772845, 88.72815812088226, 88.85706101403609, 89.02893153824118, 89.24376969349757, 89.35835004296763, 89.51589802348897, 89.51589802348897, 89.7450587224291, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 84.13312527421456, "threshold_area100": 84.13212546548266, "threshold_area25": 79.35118877112576}, "test": {"score": [76.42247255506925, 76.81710554638732, 77.15433737533185, 77.44134318719954, 77.72834899906724, 78.02252995623161, 78.26648489631916, 78.64676759704385, 78.90507282772477, 79.2423046566693, 79.52213532324029, 79.8952428786683, 80.15354810934922, 80.49795508359044, 80.74908516897467, 80.92128865609529, 81.25852048503982, 81.4594245533472, 81.70337949343475, 81.86123268996197, 82.17693908301644, 82.36349286073043, 82.56439692903783, 82.77965128793858, 82.88010332209228, 83.13840855277319, 83.38953863815742, 83.61914328765158, 83.85592308244242, 84.15010403960682, 84.24338092846381, 84.30078209083734, 84.32230752672743, 84.36535839850757, 84.40123412499103, 84.45146014206787, 84.50886130444142, 84.54473703092488, 84.54473703092488, 84.56626246681496, 84.57343761211165, 84.6093133385951, 84.63801391978187, 84.67388964626534, 84.68823993685872, 84.75281624452894, 84.79586711630911, 84.85326827868265, 84.96089545813304, 85.00394632991318, 85.11874865466027, 85.24072612470403, 85.27660185118748, 85.3627035947478, 85.43445504771472, 85.54208222716511, 85.67123484250556, 85.78603716725264, 85.78603716725264, 85.8290880390328, 85.9080146372964, 86.0084666714501, 86.05151754323025, 86.15914472268064, 86.2811221927244, 86.35287364569133, 86.42462509865825, 86.47485111573509, 86.55377771399871, 86.67575518404249, 86.81925808997633, 86.86230896175647, 86.96276099591017, 87.0416875941738, 87.15648991892087, 87.24976680777786, 87.3932697137117, 87.52242232905216, 87.63004950850255, 87.73050154265624, 87.83095357680993, 87.96010619215039, 88.06055822630408, 88.23276171342471, 88.36191432876515, 88.46236636291886, 88.54846810647916, 88.74219702948984, 88.81394848245677, 89.02202769606085, 89.18705603788477, 89.33055894381862, 89.531463012126, 89.68214106335654, 89.88304513166392, 89.904570567554, 90.13417521704815, 90.27050297768531, 90.43553131950922, 90.56468393484968, 90.73688742197031], "calls": [0.0, 1.1551983927674536, 2.138193298414293, 3.0637870416875943, 3.9965559302575877, 4.922149673530889, 5.754466527947192, 6.809212886560953, 7.741981775130946, 8.789552988448015, 9.679271005237856, 10.798593671521848, 11.66678625242161, 12.714357465738683, 13.63277606371529, 14.436392336944822, 15.82119537920643, 16.775489703666498, 17.493004233335725, 18.45447370309249, 19.408768027552558, 20.219559446078783, 20.922723685154622, 22.02769606084523, 22.895888641744996, 23.83583267561168, 24.81882758125852, 26.00272655521274, 26.806342828442276, 27.83956375116596, 28.801033220922722, 29.94188132309679, 30.573294109205712, 31.463012125995547, 32.202052091554854, 33.005668364784384, 34.06758986869484, 34.97883332137476, 35.90442706464806, 37.10267632919567, 37.848891440051666, 39.190643610533115, 39.95120901198249, 41.106407404749945, 42.04635143861663, 43.18002439549401, 44.15584415584416, 45.081437899117454, 46.236636291884906, 47.14070459926813, 48.41070531678267, 49.63765516251704, 50.46997201693334, 51.69692186266772, 52.93822199899548, 53.96426777642247, 54.94726268206931, 55.89438186123269, 56.78409987802253, 57.89624739900982, 58.90076774054675, 59.905288082083665, 61.10353734663126, 62.29461146588219, 63.306306952715794, 64.39692903781301, 65.236421037526, 66.23376623376623, 67.14500968644616, 68.19975604505991, 69.36212958312406, 70.07964411279329, 71.01958814665997, 71.83755471048289, 72.82772476142641, 73.61699074406258, 74.62868623089618, 75.75518404247687, 76.79558011049724, 77.80010045203414, 78.74004448590084, 79.71586424625099, 80.64863313482098, 81.68185405754467, 82.81552701442205, 83.81287221066228, 84.8891440051661, 86.05151754323025, 87.0273373035804, 88.36191432876515, 89.6103896103896, 90.66513596900337, 91.81315921647413, 92.92530673746143, 94.14508143789911, 94.94869771112864, 95.9245174714788, 96.96491353949918, 97.99813446222286, 98.9883045131664, 100.0], "rescued": [0, 63, 114, 159, 204, 250, 290, 353, 400, 455, 505, 569, 612, 667, 713, 750, 815, 854, 895, 937, 990, 1027, 1060, 1105, 1132, 1177, 1217, 1263, 1299, 1349, 1365, 1377, 1382, 1388, 1395, 1403, 1412, 1418, 1422, 1432, 1434, 1442, 1450, 1459, 1468, 1481, 1490, 1501, 1520, 1528, 1548, 1566, 1577, 1593, 1611, 1629, 1650, 1668, 1678, 1694, 1713, 1734, 1744, 1765, 1788, 1803, 1819, 1835, 1851, 1875, 1903, 1915, 1933, 1951, 1972, 1988, 2013, 2042, 2060, 2080, 2102, 2129, 2149, 2175, 2202, 2226, 2247, 2281, 2304, 2341, 2377, 2409, 2448, 2476, 2513, 2533, 2574, 2608, 2641, 2669, 2706], "harmed": [0, 8, 12, 17, 22, 27, 33, 43, 54, 62, 73, 85, 92, 99, 110, 123, 141, 152, 159, 179, 188, 199, 204, 219, 232, 241, 246, 260, 263, 272, 275, 279, 281, 281, 283, 284, 285, 286, 290, 297, 298, 301, 305, 309, 316, 320, 323, 326, 330, 332, 336, 337, 343, 347, 355, 358, 361, 363, 373, 383, 391, 398, 402, 408, 414, 419, 425, 434, 439, 446, 454, 460, 464, 471, 476, 479, 484, 495, 498, 504, 512, 521, 527, 529, 538, 548, 557, 564, 577, 585, 598, 610, 621, 628, 637, 654, 663, 678, 688, 698, 711], "exact_score": [76.42247255506925, 76.7381789481237, 77.08258592236493, 77.42699289660617, 77.72834899906724, 78.04405539212169, 78.28083518691254, 78.74004448590084, 78.95529884480162, 79.28535552844946, 79.622587357394, 79.95981918633852, 80.2540001435029, 80.59840711774415, 80.86388749372175, 81.1006672885126, 81.2800459209299, 81.5598765875009, 81.78948123699506, 82.09083733945613, 82.29174140776351, 82.58592236492788, 82.77247614264189, 82.89445361268565, 83.19580971514672, 83.42541436464089, 83.61914328765158, 83.94202482600272, 84.17880462079357, 84.25773121905719, 84.30078209083734, 84.32948267202411, 84.38688383439765, 84.45146014206787, 84.50886130444142, 84.55191217622158, 84.55191217622158, 84.56626246681496, 84.56626246681496, 84.60213819329842, 84.63801391978187, 84.68106479156204, 84.68823993685872, 84.74564109923226, 84.8030422616058, 84.84609313338596, 84.93937002224295, 84.98242089402311, 85.07569778288011, 85.1689746717371, 85.25507641529741, 85.33400301356102, 85.3627035947478, 85.4416301930114, 85.5492573724618, 85.67123484250556, 85.79321231254933, 85.79321231254933, 85.8290880390328, 85.9151897825931, 86.02281696204348, 86.04434239793356, 86.14479443208725, 86.23089617564756, 86.33852335509793, 86.39592451747149, 86.43897538925164, 86.5609528592954, 86.64705460285572, 86.80490779938295, 86.85513381645978, 86.95558585061349, 87.05603788476716, 87.16366506421755, 87.3071679701514, 87.40762000430509, 87.56547320083232, 87.665925234986, 87.72332639735954, 87.85247901270002, 88.00315706393054, 88.11078424338093, 88.29733802109493, 88.38343976465524, 88.46954150821554, 88.56999354236923, 88.73502188419316, 88.7924230465667, 88.94310109779724, 89.12247973021454, 89.23728205496163, 89.39513525148884, 89.56016359331277, 89.68931620865322, 89.88304513166392, 89.92609600344407, 90.13417521704815, 90.28485326827868, 90.43553131950922, 90.56468393484968, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 84.99244251835731, "threshold_area100": 84.8980770610605, "threshold_area25": 80.08466671450097}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.8948988914489746, "operator": "ge"}, {"budget": 0.02, "threshold": 1.680652141571045, "operator": "ge"}, {"budget": 0.03, "threshold": 1.5378721952438354, "operator": "ge"}, {"budget": 0.04, "threshold": 1.4286407232284546, "operator": "ge"}, {"budget": 0.05, "threshold": 1.335425615310669, "operator": "ge"}, {"budget": 0.06, "threshold": 1.2569234371185303, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1711064577102661, "operator": "ge"}, {"budget": 0.08, "threshold": 1.0924508571624756, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0339574813842773, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9849340915679932, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9354583024978638, "operator": "ge"}, {"budget": 0.12, "threshold": 0.8945333957672119, "operator": "ge"}, {"budget": 0.13, "threshold": 0.8497880101203918, "operator": "ge"}, {"budget": 0.14, "threshold": 0.8123354315757751, "operator": "ge"}, {"budget": 0.15, "threshold": 0.7654257416725159, "operator": "ge"}, {"budget": 0.16, "threshold": 0.7185759544372559, "operator": "ge"}, {"budget": 0.17, "threshold": 0.6712232828140259, "operator": "ge"}, {"budget": 0.18, "threshold": 0.6321802139282227, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5828415155410767, "operator": "ge"}, {"budget": 0.2, "threshold": 0.5357187390327454, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4878029227256775, "operator": "ge"}, {"budget": 0.22, "threshold": 0.4320698380470276, "operator": "ge"}, {"budget": 0.23, "threshold": 0.37963318824768066, "operator": "ge"}, {"budget": 0.24, "threshold": 0.32992878556251526, "operator": "ge"}, {"budget": 0.25, "threshold": 0.2779650390148163, "operator": "ge"}, {"budget": 0.26, "threshold": 0.2143431156873703, "operator": "ge"}, {"budget": 0.27, "threshold": 0.14032559096813202, "operator": "ge"}, {"budget": 0.28, "threshold": 0.06269954890012741, "operator": "ge"}, {"budget": 0.29, "threshold": -0.032694801688194275, "operator": "ge"}, {"budget": 0.3, "threshold": -0.1500592827796936, "operator": "ge"}, {"budget": 0.31, "threshold": -0.27016597986221313, "operator": "ge"}, {"budget": 0.32, "threshold": -0.39405226707458496, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5196232795715332, "operator": "ge"}, {"budget": 0.34, "threshold": -0.6895826458930969, "operator": "ge"}, {"budget": 0.35, "threshold": -0.81254643201828, "operator": "ge"}, {"budget": 0.36, "threshold": -0.9672350883483887, "operator": "ge"}, {"budget": 0.37, "threshold": -1.1054835319519043, "operator": "ge"}, {"budget": 0.38, "threshold": -1.2370492219924927, "operator": "ge"}, {"budget": 0.39, "threshold": -1.3981825113296509, "operator": "ge"}, {"budget": 0.4, "threshold": -1.5706900358200073, "operator": "ge"}, {"budget": 0.41, "threshold": -1.7287119626998901, "operator": "ge"}, {"budget": 0.42, "threshold": -1.8808832168579102, "operator": "ge"}, {"budget": 0.43, "threshold": -2.0410079956054688, "operator": "ge"}, {"budget": 0.44, "threshold": -2.2168514728546143, "operator": "ge"}, {"budget": 0.45, "threshold": -2.374051332473755, "operator": "ge"}, {"budget": 0.46, "threshold": -2.5350828170776367, "operator": "ge"}, {"budget": 0.47, "threshold": -2.692284107208252, "operator": "ge"}, {"budget": 0.48, "threshold": -2.859738826751709, "operator": "ge"}, {"budget": 0.49, "threshold": -3.063248872756958, "operator": "ge"}, {"budget": 0.5, "threshold": -3.237269401550293, "operator": "ge"}, {"budget": 0.51, "threshold": -3.3854622840881348, "operator": "ge"}, {"budget": 0.52, "threshold": -3.507446050643921, "operator": "ge"}, {"budget": 0.53, "threshold": -3.6144561767578125, "operator": "ge"}, {"budget": 0.54, "threshold": -3.740755796432495, "operator": "ge"}, {"budget": 0.55, "threshold": -3.81425404548645, "operator": "ge"}, {"budget": 0.56, "threshold": -3.9051170349121094, "operator": "ge"}, {"budget": 0.57, "threshold": -3.987632989883423, "operator": "ge"}, {"budget": 0.58, "threshold": -4.0651535987854, "operator": "ge"}, {"budget": 0.59, "threshold": -4.12681245803833, "operator": "ge"}, {"budget": 0.6, "threshold": -4.199270725250244, "operator": "ge"}, {"budget": 0.61, "threshold": -4.251153945922852, "operator": "ge"}, {"budget": 0.62, "threshold": -4.320351600646973, "operator": "ge"}, {"budget": 0.63, "threshold": -4.3770365715026855, "operator": "ge"}, {"budget": 0.64, "threshold": -4.438748359680176, "operator": "ge"}, {"budget": 0.65, "threshold": -4.4984307289123535, "operator": "ge"}, {"budget": 0.66, "threshold": -4.559154033660889, "operator": "ge"}, {"budget": 0.67, "threshold": -4.619077682495117, "operator": "ge"}, {"budget": 0.68, "threshold": -4.682483673095703, "operator": "ge"}, {"budget": 0.69, "threshold": -4.767012119293213, "operator": "ge"}, {"budget": 0.7, "threshold": -4.84267520904541, "operator": "ge"}, {"budget": 0.71, "threshold": -4.898303031921387, "operator": "ge"}, {"budget": 0.72, "threshold": -4.967787265777588, "operator": "ge"}, {"budget": 0.73, "threshold": -5.039646625518799, "operator": "ge"}, {"budget": 0.74, "threshold": -5.118212699890137, "operator": "ge"}, {"budget": 0.75, "threshold": -5.197194576263428, "operator": "ge"}, {"budget": 0.76, "threshold": -5.279525279998779, "operator": "ge"}, {"budget": 0.77, "threshold": -5.360673427581787, "operator": "ge"}, {"budget": 0.78, "threshold": -5.42772912979126, "operator": "ge"}, {"budget": 0.79, "threshold": -5.500107288360596, "operator": "ge"}, {"budget": 0.8, "threshold": -5.5782389640808105, "operator": "ge"}, {"budget": 0.81, "threshold": -5.669565200805664, "operator": "ge"}, {"budget": 0.82, "threshold": -5.740218162536621, "operator": "ge"}, {"budget": 0.83, "threshold": -5.834672451019287, "operator": "ge"}, {"budget": 0.84, "threshold": -5.913476943969727, "operator": "ge"}, {"budget": 0.85, "threshold": -6.0002522468566895, "operator": "ge"}, {"budget": 0.86, "threshold": -6.109414577484131, "operator": "ge"}, {"budget": 0.87, "threshold": -6.193826198577881, "operator": "ge"}, {"budget": 0.88, "threshold": -6.296875953674316, "operator": "ge"}, {"budget": 0.89, "threshold": -6.38218879699707, "operator": "ge"}, {"budget": 0.9, "threshold": -6.487048149108887, "operator": "ge"}, {"budget": 0.91, "threshold": -6.576032638549805, "operator": "ge"}, {"budget": 0.92, "threshold": -6.6794562339782715, "operator": "ge"}, {"budget": 0.93, "threshold": -6.786842346191406, "operator": "ge"}, {"budget": 0.94, "threshold": -6.8874101638793945, "operator": "ge"}, {"budget": 0.95, "threshold": -7.004393100738525, "operator": "ge"}, {"budget": 0.96, "threshold": -7.188556671142578, "operator": "ge"}, {"budget": 0.97, "threshold": -7.350561141967773, "operator": "ge"}, {"budget": 0.98, "threshold": -7.513500213623047, "operator": "ge"}, {"budget": 0.99, "threshold": -7.775004863739014, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.86651389286737, 76.18160985391005, 76.5110283586365, 76.85476940704669, 77.16986536808938, 77.5708965912346, 78.00057290174735, 78.27270123173875, 78.55915210541392, 78.9745058722429, 79.28960183328559, 79.47579490117445, 79.77656831853336, 80.07734173589229, 80.29217989114866, 80.53566313377254, 80.70753365797765, 80.99398453165281, 81.33772558006301, 81.59553136637066, 81.73875680320825, 81.83901460899456, 82.0968203953022, 82.22572328845604, 82.51217416213119, 82.74133486107132, 82.94185047264394, 83.05643082211401, 83.29991406473789, 83.51475221999426, 83.7725580063019, 83.95875107419077, 84.23087940418218, 84.46004010312231, 84.67487825837868, 84.90403895731882, 85.26210254941277, 85.37668289888283, 85.5628759667717, 85.72042394729303, 85.97822973360068, 86.13577771412201, 86.27900315095961, 86.40790604411343, 86.56545402463478, 86.60842165568606, 86.75164709252363, 86.76596963620739, 86.86622744199369, 87.00945287883127, 87.08106559725006, 87.12403322830134, 87.28158120882267, 87.38183901460899, 87.5107419077628, 87.68261243196791, 87.71125751933543, 87.76854769407046, 87.75422515038672, 87.95474076195933, 88.01203093669436, 88.04067602406187, 88.14093382984818, 88.14093382984818, 88.269836723002, 88.269836723002, 88.37009452878831, 88.38441707247206, 88.48467487825837, 88.58493268404469, 88.74248066456603, 88.75680320824978, 88.77112575193354, 88.82841592666857, 88.85706101403609, 88.94299627613864, 89.00028645087367, 89.10054425665999, 89.22944714981381, 89.3297049556001, 89.38699513033515, 89.41564021770266, 89.44428530507018, 89.47293039243769, 89.51589802348897, 89.51589802348897, 89.53022056717272, 89.573188198224, 89.573188198224, 89.61615582927529, 89.63047837295903, 89.6734460040103, 89.70209109137782, 89.73073617874535, 89.77370380979661, 89.80234889716414, 89.80234889716414, 89.85963907189917, 89.84531652821542, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 26, 52, 80, 108, 135, 165, 197, 218, 244, 274, 304, 325, 351, 377, 398, 418, 438, 466, 494, 518, 540, 559, 584, 603, 626, 646, 663, 678, 704, 725, 749, 767, 787, 806, 826, 845, 872, 884, 902, 915, 938, 953, 970, 985, 1000, 1009, 1024, 1031, 1042, 1056, 1066, 1074, 1088, 1096, 1106, 1120, 1125, 1132, 1138, 1152, 1159, 1165, 1173, 1179, 1190, 1194, 1205, 1210, 1219, 1229, 1240, 1243, 1247, 1253, 1260, 1268, 1274, 1284, 1295, 1303, 1310, 1318, 1323, 1329, 1333, 1337, 1341, 1345, 1349, 1353, 1355, 1359, 1362, 1366, 1371, 1375, 1377, 1381, 1383, 1389], "harmed": [0, 5, 9, 14, 18, 23, 25, 27, 29, 35, 36, 44, 52, 57, 62, 68, 71, 79, 87, 91, 97, 109, 121, 128, 138, 141, 145, 148, 155, 164, 170, 176, 181, 182, 185, 190, 193, 195, 199, 204, 206, 211, 215, 222, 228, 232, 238, 243, 249, 253, 257, 262, 267, 270, 271, 272, 274, 277, 280, 287, 287, 290, 294, 295, 301, 303, 307, 311, 315, 317, 320, 320, 322, 325, 327, 332, 334, 336, 339, 341, 342, 345, 351, 354, 358, 359, 363, 366, 367, 371, 372, 373, 374, 375, 377, 379, 381, 383, 383, 386, 388], "exact_score": [75.56574047550845, 75.86651389286737, 76.18160985391005, 76.5110283586365, 76.85476940704669, 77.16986536808938, 77.5708965912346, 78.00057290174735, 78.27270123173875, 78.55915210541392, 78.9745058722429, 79.28960183328559, 79.47579490117445, 79.77656831853336, 80.07734173589229, 80.29217989114866, 80.53566313377254, 80.70753365797765, 80.99398453165281, 81.33772558006301, 81.59553136637066, 81.73875680320825, 81.83901460899456, 82.0968203953022, 82.22572328845604, 82.51217416213119, 82.74133486107132, 82.94185047264394, 83.05643082211401, 83.29991406473789, 83.51475221999426, 83.7725580063019, 83.95875107419077, 84.23087940418218, 84.46004010312231, 84.67487825837868, 84.90403895731882, 85.26210254941277, 85.37668289888283, 85.5628759667717, 85.72042394729303, 85.97822973360068, 86.13577771412201, 86.27900315095961, 86.40790604411343, 86.56545402463478, 86.60842165568606, 86.75164709252363, 86.76596963620739, 86.86622744199369, 87.00945287883127, 87.08106559725006, 87.12403322830134, 87.28158120882267, 87.38183901460899, 87.5107419077628, 87.68261243196791, 87.71125751933543, 87.76854769407046, 87.75422515038672, 87.95474076195933, 88.01203093669436, 88.04067602406187, 88.14093382984818, 88.14093382984818, 88.269836723002, 88.269836723002, 88.37009452878831, 88.38441707247206, 88.48467487825837, 88.58493268404469, 88.74248066456603, 88.75680320824978, 88.77112575193354, 88.82841592666857, 88.85706101403609, 88.94299627613864, 89.00028645087367, 89.10054425665999, 89.22944714981381, 89.3297049556001, 89.38699513033515, 89.41564021770266, 89.44428530507018, 89.47293039243769, 89.51589802348897, 89.51589802348897, 89.53022056717272, 89.573188198224, 89.573188198224, 89.61615582927529, 89.63047837295903, 89.6734460040103, 89.70209109137782, 89.73073617874535, 89.77370380979661, 89.80234889716414, 89.80234889716414, 89.85963907189917, 89.84531652821542, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.5051026985051, "threshold_area100": 85.50408192494987, "threshold_area25": 79.41420796333429}, "test": {"score": [76.42247255506925, 76.78122981990386, 77.14716223003516, 77.41981775130947, 77.77857501614407, 78.11580684508861, 78.40281265695631, 78.74004448590084, 79.00552486187846, 79.31405610963623, 79.60106192150391, 79.74456482743776, 80.03874578460214, 80.34010188706321, 80.53383081007391, 80.85671234842506, 81.12936786969937, 81.44507426275382, 81.67467891224798, 81.98321016000574, 82.15541364712635, 82.35631771543373, 82.60744780081797, 82.81552701442205, 83.00208079213604, 83.14558369806988, 83.38236349286073, 83.65501901413504, 83.88462366362918, 84.13575374901342, 84.38688383439765, 84.68106479156204, 84.93219487694627, 85.11874865466027, 85.30530243237426, 85.47750591949487, 85.75733658606588, 86.00129152615341, 86.17349501327402, 86.48920140632848, 86.7546817823061, 86.91253497883332, 87.1134390471407, 87.26411709837124, 87.43632058549186, 87.63722465379924, 87.73767668795294, 87.8596541579967, 87.945755901557, 88.03903279041401, 88.17536055105116, 88.33321374757838, 88.412140345842, 88.505417234699, 88.56281839707253, 88.66327043122624, 88.77807275597331, 88.89287508072039, 88.97897682428069, 89.065078567841, 89.10812943962115, 89.16553060199469, 89.251632345555, 89.30185836263185, 89.33055894381862, 89.38796010619215, 89.47406184975246, 89.52428786682931, 89.58168902920285, 89.65344048216977, 89.75389251632346, 89.77541795221353, 89.83281911458707, 89.87586998636723, 89.96914687522423, 90.10547463586138, 90.14852550764154, 90.1772260888283, 90.24180239649853, 90.23462725120184, 90.24897754179523, 90.27050297768531, 90.29920355887207, 90.37095501183899, 90.44988161010261, 90.49293248188276, 90.55750878955298, 90.56468393484968, 90.60773480662984, 90.64361053311329, 90.65078567840999, 90.65796082370669, 90.67231111430006, 90.70818684078353, 90.72253713137691, 90.70101169548683, 90.70101169548683, 90.67948625959676, 90.67948625959676, 90.69383655019014, 90.73688742197031], "calls": [0.0, 1.0475712133170696, 2.1812441701944465, 3.0924876228743634, 4.118533400301356, 5.101528305948196, 5.948195450957882, 7.060342971945182, 8.10073903996556, 9.155485398579321, 10.066728851259239, 10.812943962115233, 11.59503479945469, 12.420176508574299, 13.202267345913754, 14.285714285714285, 15.361986080218124, 16.36650642175504, 17.270574729138268, 18.310970797158642, 19.301140848102175, 20.17650857429863, 21.145153189352083, 22.12097294970223, 22.94611465882184, 23.914759273875298, 24.861878453038674, 25.84487335868551, 26.84221855492574, 27.8036880246825, 29.080863887493724, 30.17866111788764, 31.183181459424553, 32.18770180096147, 33.694482313266846, 34.57702518475999, 35.66764726985722, 36.722393628470975, 37.55471048288727, 38.917988089258806, 40.302791131520415, 41.37188778072756, 42.3477075410777, 43.36657817320801, 44.46437540360193, 45.38279400157853, 46.272512018368374, 47.047427710411135, 47.92279543660759, 48.99189208581474, 50.240367367439184, 51.27358829016288, 52.07720456339241, 53.09607519552271, 54.33737533185047, 55.234268493937, 56.296189997847456, 57.3652866470546, 58.319580971514675, 59.20929898830452, 60.26404534691827, 61.082011910741194, 62.11523283346487, 63.033651431441484, 63.93771973882472, 64.8704886273947, 65.93241013130516, 66.728851259238, 67.6687952931047, 68.90292028413575, 70.01506780512305, 70.7971586424625, 71.7371026763292, 72.69857214608597, 73.92552199182033, 75.33185046997201, 76.63055176867331, 77.850326469111, 78.876372246538, 79.7947908445146, 80.74908516897467, 81.88275812585205, 82.75095070675181, 83.81287221066228, 84.73129080863887, 85.76451173136256, 87.07756332065725, 87.84530386740332, 88.89287508072037, 89.60321446509292, 90.64361053311329, 91.4831025328263, 92.32976967783597, 93.16208653225227, 93.94417736959173, 94.81236995049149, 96.24022386453326, 97.25909449666355, 98.12728707756332, 99.06723111143, 100.0], "rescued": [0, 52, 111, 153, 211, 259, 306, 361, 409, 463, 513, 542, 589, 638, 679, 730, 776, 832, 874, 933, 977, 1017, 1066, 1106, 1141, 1175, 1217, 1260, 1303, 1347, 1395, 1444, 1486, 1526, 1574, 1604, 1652, 1690, 1722, 1781, 1829, 1862, 1893, 1922, 1956, 1987, 2010, 2033, 2051, 2076, 2103, 2130, 2147, 2170, 2192, 2211, 2231, 2252, 2268, 2285, 2299, 2312, 2327, 2340, 2351, 2363, 2379, 2390, 2403, 2418, 2438, 2445, 2457, 2469, 2485, 2507, 2519, 2529, 2543, 2548, 2557, 2570, 2578, 2592, 2607, 2617, 2634, 2639, 2650, 2659, 2665, 2668, 2673, 2681, 2685, 2686, 2691, 2692, 2695, 2698, 2706], "harmed": [0, 2, 10, 14, 22, 23, 30, 38, 49, 60, 70, 79, 85, 92, 106, 112, 120, 132, 142, 158, 178, 190, 204, 215, 224, 238, 247, 252, 263, 272, 285, 293, 300, 314, 336, 342, 351, 355, 363, 378, 389, 400, 403, 411, 421, 424, 433, 439, 445, 457, 465, 470, 476, 486, 500, 505, 509, 514, 518, 523, 531, 536, 539, 545, 552, 556, 560, 564, 569, 574, 580, 584, 588, 594, 597, 600, 606, 612, 617, 623, 630, 640, 644, 648, 652, 656, 664, 668, 673, 677, 682, 684, 687, 690, 692, 696, 701, 705, 708, 709, 711], "exact_score": [76.42247255506925, 76.75252923871709, 77.06823563177154, 77.3911171701227, 77.7498744349573, 78.06558082801178, 78.41716294754968, 78.72569419530745, 78.96964913539499, 79.27100523785607, 79.57236134031714, 79.83784171629476, 80.23964985290952, 80.51230537418384, 80.75626031427137, 81.03609098084237, 81.33744708330343, 81.63162804046782, 81.87558298055536, 82.14823850182967, 82.3060916983569, 82.56439692903783, 82.81552701442205, 82.99490564683936, 83.17428427925665, 83.40388892875082, 83.66936930472842, 83.93484968070604, 84.19315491138696, 84.36535839850757, 84.6093133385951, 84.85326827868265, 85.06134749228671, 85.23355097940734, 85.3555284494511, 85.57078280835188, 85.8290880390328, 86.03716725263688, 86.30982277391116, 86.48920140632848, 86.67575518404249, 86.86948410705317, 87.06321303006386, 87.2067159359977, 87.34304369663487, 87.55112291023893, 87.74485183324963, 87.86682930329339, 87.9529310468537, 88.03903279041401, 88.15383511516109, 88.30451316639162, 88.40496520054532, 88.5125923799957, 88.55564325177585, 88.64174499533615, 88.74937217478654, 88.8856999354237, 88.93592595250054, 89.04355313195093, 89.07942885843438, 89.13683002080792, 89.23728205496163, 89.28750807203846, 89.32338379852193, 89.39513525148884, 89.47406184975246, 89.51711272153261, 89.60321446509292, 89.65344048216977, 89.75389251632346, 89.79694338810361, 89.86151969577384, 89.88304513166392, 89.98349716581761, 90.07677405467462, 90.13417521704815, 90.14852550764154, 90.19157637942169, 90.24180239649853, 90.22027696060846, 90.25615268709191, 90.26332783238861, 90.32072899476215, 90.39965559302576, 90.46423190069599, 90.49293248188276, 90.55750878955298, 90.56468393484968, 90.60773480662984, 90.64361053311329, 90.67231111430006, 90.67231111430006, 90.70101169548683, 90.7297122766736, 90.72253713137691, 90.70101169548683, 90.67231111430006, 90.68666140489346, 90.69383655019014, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.49082130979953, "threshold_area100": 86.45429432446008, "threshold_area25": 80.1026045777427}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.6080448627471924, "operator": "ge"}, {"budget": 0.02, "threshold": 1.4129951000213623, "operator": "ge"}, {"budget": 0.03, "threshold": 1.2786991596221924, "operator": "ge"}, {"budget": 0.04, "threshold": 1.1768736839294434, "operator": "ge"}, {"budget": 0.05, "threshold": 1.0956531763076782, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0297209024429321, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9723283052444458, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9086832404136658, "operator": "ge"}, {"budget": 0.09, "threshold": 0.8484781384468079, "operator": "ge"}, {"budget": 0.1, "threshold": 0.8082643151283264, "operator": "ge"}, {"budget": 0.11, "threshold": 0.767324686050415, "operator": "ge"}, {"budget": 0.12, "threshold": 0.7302474975585938, "operator": "ge"}, {"budget": 0.13, "threshold": 0.6862537860870361, "operator": "ge"}, {"budget": 0.14, "threshold": 0.650824785232544, "operator": "ge"}, {"budget": 0.15, "threshold": 0.6209220886230469, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5729662775993347, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5382868051528931, "operator": "ge"}, {"budget": 0.18, "threshold": 0.5026038885116577, "operator": "ge"}, {"budget": 0.19, "threshold": 0.4631235599517822, "operator": "ge"}, {"budget": 0.2, "threshold": 0.42392289638519287, "operator": "ge"}, {"budget": 0.21, "threshold": 0.3761889636516571, "operator": "ge"}, {"budget": 0.22, "threshold": 0.33613982796669006, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2871153950691223, "operator": "ge"}, {"budget": 0.24, "threshold": 0.2465277910232544, "operator": "ge"}, {"budget": 0.25, "threshold": 0.20707058906555176, "operator": "ge"}, {"budget": 0.26, "threshold": 0.1628694236278534, "operator": "ge"}, {"budget": 0.27, "threshold": 0.11505740880966187, "operator": "ge"}, {"budget": 0.28, "threshold": 0.05098895728588104, "operator": "ge"}, {"budget": 0.29, "threshold": -0.028498739004135132, "operator": "ge"}, {"budget": 0.3, "threshold": -0.12408149242401123, "operator": "ge"}, {"budget": 0.31, "threshold": -0.23281104862689972, "operator": "ge"}, {"budget": 0.32, "threshold": -0.3334542512893677, "operator": "ge"}, {"budget": 0.33, "threshold": -0.4617409110069275, "operator": "ge"}, {"budget": 0.34, "threshold": -0.59539794921875, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7133267521858215, "operator": "ge"}, {"budget": 0.36, "threshold": -0.845780611038208, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9680700302124023, "operator": "ge"}, {"budget": 0.38, "threshold": -1.101768970489502, "operator": "ge"}, {"budget": 0.39, "threshold": -1.2294089794158936, "operator": "ge"}, {"budget": 0.4, "threshold": -1.3901021480560303, "operator": "ge"}, {"budget": 0.41, "threshold": -1.5502936840057373, "operator": "ge"}, {"budget": 0.42, "threshold": -1.6757749319076538, "operator": "ge"}, {"budget": 0.43, "threshold": -1.8295950889587402, "operator": "ge"}, {"budget": 0.44, "threshold": -1.966719388961792, "operator": "ge"}, {"budget": 0.45, "threshold": -2.1401686668395996, "operator": "ge"}, {"budget": 0.46, "threshold": -2.288938522338867, "operator": "ge"}, {"budget": 0.47, "threshold": -2.4291324615478516, "operator": "ge"}, {"budget": 0.48, "threshold": -2.585049629211426, "operator": "ge"}, {"budget": 0.49, "threshold": -2.7501778602600098, "operator": "ge"}, {"budget": 0.5, "threshold": -2.9121856689453125, "operator": "ge"}, {"budget": 0.51, "threshold": -3.048664093017578, "operator": "ge"}, {"budget": 0.52, "threshold": -3.1396896839141846, "operator": "ge"}, {"budget": 0.53, "threshold": -3.2458698749542236, "operator": "ge"}, {"budget": 0.54, "threshold": -3.3252604007720947, "operator": "ge"}, {"budget": 0.55, "threshold": -3.414623975753784, "operator": "ge"}, {"budget": 0.56, "threshold": -3.4844398498535156, "operator": "ge"}, {"budget": 0.57, "threshold": -3.560717821121216, "operator": "ge"}, {"budget": 0.58, "threshold": -3.6279473304748535, "operator": "ge"}, {"budget": 0.59, "threshold": -3.6896138191223145, "operator": "ge"}, {"budget": 0.6, "threshold": -3.76011061668396, "operator": "ge"}, {"budget": 0.61, "threshold": -3.8185863494873047, "operator": "ge"}, {"budget": 0.62, "threshold": -3.875123977661133, "operator": "ge"}, {"budget": 0.63, "threshold": -3.929565191268921, "operator": "ge"}, {"budget": 0.64, "threshold": -3.9907875061035156, "operator": "ge"}, {"budget": 0.65, "threshold": -4.048440933227539, "operator": "ge"}, {"budget": 0.66, "threshold": -4.09241247177124, "operator": "ge"}, {"budget": 0.67, "threshold": -4.143209934234619, "operator": "ge"}, {"budget": 0.68, "threshold": -4.19785213470459, "operator": "ge"}, {"budget": 0.69, "threshold": -4.268547058105469, "operator": "ge"}, {"budget": 0.7, "threshold": -4.329818248748779, "operator": "ge"}, {"budget": 0.71, "threshold": -4.390700340270996, "operator": "ge"}, {"budget": 0.72, "threshold": -4.4471540451049805, "operator": "ge"}, {"budget": 0.73, "threshold": -4.505295753479004, "operator": "ge"}, {"budget": 0.74, "threshold": -4.566239833831787, "operator": "ge"}, {"budget": 0.75, "threshold": -4.63307523727417, "operator": "ge"}, {"budget": 0.76, "threshold": -4.703637599945068, "operator": "ge"}, {"budget": 0.77, "threshold": -4.777893543243408, "operator": "ge"}, {"budget": 0.78, "threshold": -4.851978302001953, "operator": "ge"}, {"budget": 0.79, "threshold": -4.922276496887207, "operator": "ge"}, {"budget": 0.8, "threshold": -5.011997699737549, "operator": "ge"}, {"budget": 0.81, "threshold": -5.079921722412109, "operator": "ge"}, {"budget": 0.82, "threshold": -5.142080783843994, "operator": "ge"}, {"budget": 0.83, "threshold": -5.226562023162842, "operator": "ge"}, {"budget": 0.84, "threshold": -5.30255126953125, "operator": "ge"}, {"budget": 0.85, "threshold": -5.396022319793701, "operator": "ge"}, {"budget": 0.86, "threshold": -5.490142345428467, "operator": "ge"}, {"budget": 0.87, "threshold": -5.569498062133789, "operator": "ge"}, {"budget": 0.88, "threshold": -5.651677131652832, "operator": "ge"}, {"budget": 0.89, "threshold": -5.732728004455566, "operator": "ge"}, {"budget": 0.9, "threshold": -5.813833236694336, "operator": "ge"}, {"budget": 0.91, "threshold": -5.914156913757324, "operator": "ge"}, {"budget": 0.92, "threshold": -6.0197434425354, "operator": "ge"}, {"budget": 0.93, "threshold": -6.122668266296387, "operator": "ge"}, {"budget": 0.94, "threshold": -6.229682922363281, "operator": "ge"}, {"budget": 0.95, "threshold": -6.32346248626709, "operator": "ge"}, {"budget": 0.96, "threshold": -6.435874938964844, "operator": "ge"}, {"budget": 0.97, "threshold": -6.591158390045166, "operator": "ge"}, {"budget": 0.98, "threshold": -6.780459880828857, "operator": "ge"}, {"budget": 0.99, "threshold": -7.0151047706604, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.88083643655112, 76.15296476654252, 76.56831853337152, 76.7831566886279, 77.25580063019191, 77.59954167860211, 77.97192781437982, 78.31566886279003, 78.6307648238327, 78.90289315382411, 79.23231165855056, 79.51876253222572, 79.83385849326841, 80.09166427957605, 80.32082497851619, 80.44972787167, 80.50701804640504, 80.93669435691778, 81.23746777427671, 81.45230592953308, 81.69578917215698, 81.95359495846462, 82.08249785161844, 82.22572328845604, 82.45488398739616, 82.76997994843884, 82.97049556001146, 83.05643082211401, 83.28559152105413, 83.51475221999426, 83.80120309366943, 83.94442853050701, 84.23087940418218, 84.44571755943855, 84.68920080206244, 85.00429676310513, 85.17616728731022, 85.36236035519909, 85.51990833572042, 85.72042394729303, 85.97822973360068, 86.15010025780579, 86.22171297622458, 86.36493841306215, 86.55113148095101, 86.63706674305357, 86.65138928673733, 86.76596963620739, 86.86622744199369, 86.98080779146376, 87.12403322830134, 87.19564594672013, 87.29590375250645, 87.35319392724148, 87.49641936407906, 87.6539673446004, 87.7255800630192, 87.89745058722428, 87.9260956745918, 87.9260956745918, 87.9260956745918, 87.9260956745918, 88.02635348037812, 88.11228874248066, 88.15525637353194, 88.25551417931825, 88.34144944142079, 88.4417072472071, 88.52764250930964, 88.6135777714122, 88.64222285877972, 88.69951303351475, 88.85706101403609, 88.92867373245488, 88.98596390718991, 89.01460899455743, 89.10054425665999, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.4013176740189, 89.44428530507018, 89.51589802348897, 89.60183328559151, 89.573188198224, 89.5445431108565, 89.53022056717272, 89.55886565454024, 89.6734460040103, 89.75938126611285, 89.7450587224291, 89.78802635348038, 89.83099398453164, 89.84531652821542, 89.85963907189917, 89.85963907189917, 89.88828415926669, 89.85963907189917, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 26, 52, 83, 103, 140, 166, 192, 220, 246, 271, 300, 327, 353, 374, 397, 415, 427, 459, 487, 510, 535, 562, 580, 598, 623, 647, 665, 678, 704, 725, 752, 767, 787, 806, 827, 850, 869, 883, 899, 915, 939, 955, 969, 986, 1001, 1011, 1020, 1033, 1044, 1054, 1067, 1076, 1088, 1095, 1109, 1123, 1132, 1145, 1151, 1156, 1162, 1164, 1171, 1180, 1186, 1197, 1204, 1213, 1220, 1228, 1236, 1243, 1256, 1261, 1267, 1272, 1281, 1288, 1297, 1306, 1312, 1320, 1326, 1334, 1342, 1342, 1343, 1344, 1349, 1358, 1364, 1366, 1370, 1374, 1376, 1379, 1382, 1385, 1386, 1389], "harmed": [0, 4, 11, 13, 18, 22, 24, 24, 28, 32, 38, 44, 51, 55, 58, 65, 74, 82, 84, 91, 99, 107, 116, 125, 133, 142, 144, 148, 155, 165, 170, 177, 182, 182, 186, 190, 191, 198, 199, 204, 206, 212, 216, 225, 232, 234, 238, 246, 251, 255, 257, 260, 264, 269, 272, 276, 279, 283, 284, 288, 293, 299, 301, 301, 304, 307, 311, 312, 314, 315, 317, 323, 326, 328, 328, 330, 333, 336, 339, 343, 349, 351, 354, 357, 360, 362, 364, 367, 369, 372, 373, 373, 376, 377, 378, 379, 381, 384, 385, 388, 388], "exact_score": [75.56574047550845, 75.88083643655112, 76.15296476654252, 76.56831853337152, 76.7831566886279, 77.25580063019191, 77.59954167860211, 77.97192781437982, 78.31566886279003, 78.6307648238327, 78.90289315382411, 79.23231165855056, 79.51876253222572, 79.83385849326841, 80.09166427957605, 80.32082497851619, 80.44972787167, 80.50701804640504, 80.93669435691778, 81.23746777427671, 81.45230592953308, 81.69578917215698, 81.95359495846462, 82.08249785161844, 82.22572328845604, 82.45488398739616, 82.76997994843884, 82.97049556001146, 83.05643082211401, 83.28559152105413, 83.51475221999426, 83.80120309366943, 83.94442853050701, 84.23087940418218, 84.44571755943855, 84.68920080206244, 85.00429676310513, 85.17616728731022, 85.36236035519909, 85.51990833572042, 85.72042394729303, 85.97822973360068, 86.15010025780579, 86.22171297622458, 86.36493841306215, 86.55113148095101, 86.63706674305357, 86.65138928673733, 86.76596963620739, 86.86622744199369, 86.98080779146376, 87.12403322830134, 87.19564594672013, 87.29590375250645, 87.35319392724148, 87.49641936407906, 87.6539673446004, 87.7255800630192, 87.89745058722428, 87.9260956745918, 87.9260956745918, 87.9260956745918, 87.9260956745918, 88.02635348037812, 88.11228874248066, 88.15525637353194, 88.25551417931825, 88.34144944142079, 88.4417072472071, 88.52764250930964, 88.6135777714122, 88.64222285877972, 88.69951303351475, 88.85706101403609, 88.92867373245488, 88.98596390718991, 89.01460899455743, 89.10054425665999, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.4013176740189, 89.44428530507018, 89.51589802348897, 89.60183328559151, 89.573188198224, 89.5445431108565, 89.53022056717272, 89.55886565454024, 89.6734460040103, 89.75938126611285, 89.7450587224291, 89.78802635348038, 89.83099398453164, 89.84531652821542, 89.85963907189917, 89.85963907189917, 89.88828415926669, 89.85963907189917, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.50970490800871, "threshold_area100": 85.50866513892868, "threshold_area25": 79.38842738470352}, "test": {"score": [76.42247255506925, 76.82428069168401, 77.12563679414508, 77.49156920427639, 77.89337734089116, 78.2162588792423, 78.38128722106623, 78.6826433235273, 79.01270000717516, 79.3355815455263, 79.50778503264692, 79.83066657099806, 80.06027122049221, 80.36880246825, 80.59840711774415, 80.84236205783168, 81.17959388677622, 81.36614766449021, 81.58857716868766, 81.89710841644543, 82.1984645189065, 82.42089402310397, 82.58592236492788, 82.87292817679558, 83.03078137332281, 83.18145942455335, 83.43258950993759, 83.66219415943173, 83.89897395422258, 84.1429288943101, 84.37970868910097, 84.6954150821554, 84.91784458635287, 85.19050010762719, 85.27660185118748, 85.49185621008826, 85.73581115017579, 85.97259094496664, 86.23089617564756, 86.43897538925164, 86.69010547463587, 86.90535983353664, 87.1134390471407, 87.26411709837124, 87.46502116667862, 87.62287436320587, 87.75202697854631, 87.84530386740332, 87.96010619215039, 88.06055822630408, 88.21841142283131, 88.31168831168831, 88.39061490995192, 88.505417234699, 88.56999354236923, 88.65609528592955, 88.77089761067663, 88.86417449953362, 88.96462653368731, 89.04355313195093, 89.12965487551124, 89.20858147377486, 89.20858147377486, 89.28750807203846, 89.33055894381862, 89.3520843797087, 89.46688670445577, 89.49558728564253, 89.56016359331277, 89.63909019157639, 89.70366649924661, 89.79694338810361, 89.84716940518047, 89.96197172992754, 90.00502260170768, 90.04807347348785, 90.10547463586138, 90.15570065293824, 90.24180239649853, 90.277678122982, 90.30637870416876, 90.30637870416876, 90.33507928535553, 90.39248044772907, 90.42835617421254, 90.49293248188276, 90.60773480662984, 90.60055966133315, 90.60773480662984, 90.65078567840999, 90.66513596900337, 90.67948625959676, 90.71536198608023, 90.744062567267, 90.70818684078353, 90.70818684078353, 90.69383655019014, 90.66513596900337, 90.67948625959676, 90.744062567267, 90.73688742197031], "calls": [0.0, 1.1049723756906076, 2.131018153117601, 3.1211882040611325, 4.305087178015355, 5.316782664848963, 6.07017292100165, 6.974241228384876, 8.021812441701945, 9.105259381502476, 10.095429432446007, 10.884695415082156, 11.702661978905073, 12.628255722178375, 13.424696850111214, 14.192437396857285, 15.340460644328047, 16.244528951711274, 17.198823276171343, 18.23204419889503, 19.22938939513525, 20.320011480232473, 21.0375260099017, 22.18554925737246, 23.154193872425914, 23.979335581545527, 24.854703307741982, 25.80899763220205, 26.89244457200258, 27.825213460572577, 29.04498816101026, 30.214536844371096, 31.154480878237784, 32.367080433378774, 33.57250484322308, 34.61290091124346, 35.674822415153905, 36.67934275669082, 37.71256367941451, 38.79601061921504, 40.2238645332568, 41.39341321661764, 42.41228384874794, 43.3737533185047, 44.36392336944823, 45.497596326325606, 46.30838774485183, 47.00437683863098, 47.958671163091054, 49.13539499174858, 50.254717658032575, 51.46014206787688, 52.14895601635933, 53.23957810145655, 54.14364640883977, 55.21274305804692, 56.06658534835331, 57.31506062997775, 58.40568271507498, 59.424553347205276, 60.6156274664562, 61.383368013202265, 62.25156059410203, 63.26325608093563, 64.39692903781301, 65.35122336227307, 66.11178876372247, 67.01585707110569, 68.01320226734592, 69.20427638659683, 70.29489847169405, 71.20614192437397, 72.13891081294396, 73.10755542799741, 74.12642606012771, 75.2457487264117, 76.38659682858578, 77.49156920427639, 78.91224797302145, 79.90241802396498, 81.0934921432159, 82.1051876300495, 82.95902992035589, 84.03530171485973, 84.92501973164957, 86.03716725263688, 87.1206141924374, 87.90270502977685, 88.67762072181962, 89.58168902920283, 90.39248044772906, 91.49745282341968, 92.60242519911029, 93.42039176293319, 94.43208724976681, 95.25722895888642, 96.12542153978619, 97.08689100954294, 98.06988591518979, 99.03853053024324, 100.0], "rescued": [0, 58, 107, 163, 226, 274, 304, 353, 409, 461, 502, 551, 594, 643, 681, 723, 781, 819, 861, 917, 972, 1024, 1057, 1114, 1147, 1177, 1220, 1261, 1305, 1348, 1394, 1446, 1484, 1537, 1570, 1607, 1650, 1689, 1735, 1774, 1823, 1861, 1896, 1923, 1962, 1988, 2013, 2033, 2056, 2080, 2112, 2136, 2151, 2175, 2192, 2212, 2233, 2256, 2277, 2292, 2309, 2323, 2330, 2349, 2359, 2369, 2386, 2395, 2406, 2420, 2436, 2455, 2466, 2483, 2493, 2503, 2515, 2527, 2546, 2561, 2573, 2582, 2590, 2602, 2611, 2623, 2640, 2646, 2652, 2664, 2668, 2673, 2681, 2687, 2689, 2692, 2692, 2694, 2696, 2706, 2706], "harmed": [0, 2, 9, 14, 21, 24, 31, 38, 48, 55, 72, 76, 87, 93, 99, 107, 118, 130, 141, 154, 167, 188, 198, 215, 226, 235, 243, 252, 263, 272, 285, 293, 300, 315, 336, 343, 352, 358, 368, 378, 392, 400, 406, 412, 423, 427, 434, 441, 448, 458, 468, 479, 483, 491, 499, 507, 512, 522, 529, 533, 538, 541, 548, 556, 560, 567, 568, 573, 575, 578, 585, 591, 595, 596, 600, 604, 608, 613, 620, 630, 638, 647, 651, 655, 659, 662, 663, 670, 675, 681, 683, 686, 689, 691, 698, 701, 703, 709, 709, 710, 711], "exact_score": [76.42247255506925, 76.77405467460716, 77.07541077706824, 77.44851833249623, 77.78575016144077, 78.10863169979191, 78.35976178517615, 78.68981846882399, 78.99117457128507, 79.32123125493291, 79.5149601779436, 79.86654229748153, 80.1678983999426, 80.51230537418384, 80.76343545956806, 81.08631699791921, 81.2800459209299, 81.50965057042406, 81.82535696347851, 82.10518763004951, 82.3993685872139, 82.58592236492788, 82.82987730501543, 83.00208079213604, 83.17428427925665, 83.46129009112435, 83.6837195953218, 83.93484968070604, 84.18597976609027, 84.37970868910097, 84.6093133385951, 84.9034942957595, 85.0828729281768, 85.25507641529741, 85.3627035947478, 85.59948338953865, 85.80038745784603, 86.05869268852695, 86.2882973380211, 86.46050082514172, 86.6542297481524, 86.8479586711631, 87.02016215828371, 87.22824137188778, 87.37891942311832, 87.52242232905216, 87.73767668795294, 87.84530386740332, 87.96728133744709, 88.0462079357107, 88.15383511516109, 88.26146229461148, 88.36908947406185, 88.49824208940231, 88.56281839707253, 88.63456985003947, 88.76372246537993, 88.84264906364353, 88.95027624309392, 88.98615196957739, 89.07942885843438, 89.17270574729139, 89.21575661907154, 89.2659826361484, 89.30903350792853, 89.32338379852193, 89.45253641386238, 89.49558728564253, 89.56016359331277, 89.6175647556863, 89.66061562746646, 89.76106766162016, 89.82564396929038, 89.95479658463084, 89.997847456411, 90.04807347348785, 90.06959890937792, 90.14135036234484, 90.18440123412499, 90.24897754179523, 90.29920355887207, 90.30637870416876, 90.29920355887207, 90.32072899476215, 90.39248044772907, 90.43553131950922, 90.49293248188276, 90.58620937073975, 90.58620937073975, 90.6364353878166, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.72253713137691, 90.70818684078353, 90.71536198608023, 90.69383655019014, 90.67231111430006, 90.67948625959676, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.48837407917831, "threshold_area100": 86.46068020377413, "threshold_area25": 80.11336729568775}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.7450587224291, "calls": 98.99742194213692}, "test": {"score": 90.56468393484968, "calls": 98.9883045131664}}, "old_normalized_interior": {"val": {"score": 89.88828415926669, "calls": 97.99484388427385}, "test": {"score": 90.67948625959676, "calls": 98.06988591518979}}, "old_normalized_endpoints": {"val": {"score": 89.90260670295044, "calls": 100.0}, "test": {"score": 90.73688742197031, "calls": 100.0}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 75.88083643655112, 76.15296476654252, 76.56831853337152, 76.7831566886279, 77.25580063019191, 77.59954167860211, 77.97192781437982, 78.31566886279003, 78.6307648238327, 78.90289315382411, 79.23231165855056, 79.51876253222572, 79.83385849326841, 80.09166427957605, 80.32082497851619, 80.44972787167, 80.50701804640504, 80.93669435691778, 81.23746777427671, 81.45230592953308, 81.69578917215698, 81.95359495846462, 82.08249785161844, 82.22572328845604, 82.45488398739616, 82.76997994843884, 82.97049556001146, 83.05643082211401, 83.28559152105413, 83.51475221999426, 83.80120309366943, 83.94442853050701, 84.23087940418218, 84.44571755943855, 84.68920080206244, 85.00429676310513, 85.17616728731022, 85.36236035519909, 85.51990833572042, 85.72042394729303, 85.97822973360068, 86.15010025780579, 86.22171297622458, 86.36493841306215, 86.55113148095101, 86.63706674305357, 86.65138928673733, 86.76596963620739, 86.86622744199369, 86.98080779146376, 87.12403322830134, 87.19564594672013, 87.29590375250645, 87.35319392724148, 87.49641936407906, 87.6539673446004, 87.7255800630192, 87.89745058722428, 87.9260956745918, 87.9260956745918, 87.9260956745918, 87.9260956745918, 88.02635348037812, 88.11228874248066, 88.15525637353194, 88.25551417931825, 88.34144944142079, 88.4417072472071, 88.52764250930964, 88.6135777714122, 88.64222285877972, 88.69951303351475, 88.85706101403609, 88.92867373245488, 88.98596390718991, 89.01460899455743, 89.10054425665999, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.4013176740189, 89.44428530507018, 89.51589802348897, 89.60183328559151, 89.573188198224, 89.5445431108565, 89.53022056717272, 89.55886565454024, 89.6734460040103, 89.75938126611285, 89.7450587224291, 89.78802635348038, 89.83099398453164, 89.84531652821542, 89.85963907189917, 89.85963907189917, 89.88828415926669, 89.85963907189917, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 75.88083643655112, 76.15296476654252, 76.56831853337152, 76.7831566886279, 77.25580063019191, 77.59954167860211, 77.97192781437982, 78.31566886279003, 78.6307648238327, 78.90289315382411, 79.23231165855056, 79.51876253222572, 79.83385849326841, 80.09166427957605, 80.32082497851619, 80.44972787167, 80.50701804640504, 80.93669435691778, 81.23746777427671, 81.45230592953308, 81.69578917215698, 81.95359495846462, 82.08249785161844, 82.22572328845604, 82.45488398739616, 82.76997994843884, 82.97049556001146, 83.05643082211401, 83.28559152105413, 83.51475221999426, 83.80120309366943, 83.94442853050701, 84.23087940418218, 84.44571755943855, 84.68920080206244, 85.00429676310513, 85.17616728731022, 85.36236035519909, 85.51990833572042, 85.72042394729303, 85.97822973360068, 86.15010025780579, 86.22171297622458, 86.36493841306215, 86.55113148095101, 86.63706674305357, 86.65138928673733, 86.76596963620739, 86.86622744199369, 86.98080779146376, 87.12403322830134, 87.19564594672013, 87.29590375250645, 87.35319392724148, 87.49641936407906, 87.6539673446004, 87.7255800630192, 87.89745058722428, 87.9260956745918, 87.9260956745918, 87.9260956745918, 87.9260956745918, 88.02635348037812, 88.11228874248066, 88.15525637353194, 88.25551417931825, 88.34144944142079, 88.4417072472071, 88.52764250930964, 88.6135777714122, 88.64222285877972, 88.69951303351475, 88.85706101403609, 88.92867373245488, 88.98596390718991, 89.01460899455743, 89.10054425665999, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.4013176740189, 89.44428530507018, 89.51589802348897, 89.60183328559151, 89.573188198224, 89.5445431108565, 89.53022056717272, 89.55886565454024, 89.6734460040103, 89.75938126611285, 89.7450587224291, 89.78802635348038, 89.83099398453164, 89.84531652821542, 89.85963907189917, 89.85963907189917, 89.88828415926669, 89.85963907189917, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.38842738470352, "exact_area100": 85.50970490800871}, "test": {"score": [76.42247255506925, 76.82428069168401, 77.12563679414508, 77.49156920427639, 77.89337734089116, 78.2162588792423, 78.38128722106623, 78.6826433235273, 79.01270000717516, 79.3355815455263, 79.50778503264692, 79.83066657099806, 80.06027122049221, 80.36880246825, 80.59840711774415, 80.84236205783168, 81.17959388677622, 81.36614766449021, 81.58857716868766, 81.89710841644543, 82.1984645189065, 82.42089402310397, 82.58592236492788, 82.87292817679558, 83.03078137332281, 83.18145942455335, 83.43258950993759, 83.66219415943173, 83.89897395422258, 84.1429288943101, 84.37970868910097, 84.6954150821554, 84.91784458635287, 85.19050010762719, 85.27660185118748, 85.49185621008826, 85.73581115017579, 85.97259094496664, 86.23089617564756, 86.43897538925164, 86.69010547463587, 86.90535983353664, 87.1134390471407, 87.26411709837124, 87.46502116667862, 87.62287436320587, 87.75202697854631, 87.84530386740332, 87.96010619215039, 88.06055822630408, 88.21841142283131, 88.31168831168831, 88.39061490995192, 88.505417234699, 88.56999354236923, 88.65609528592955, 88.77089761067663, 88.86417449953362, 88.96462653368731, 89.04355313195093, 89.12965487551124, 89.20858147377486, 89.20858147377486, 89.28750807203846, 89.33055894381862, 89.3520843797087, 89.46688670445577, 89.49558728564253, 89.56016359331277, 89.63909019157639, 89.70366649924661, 89.79694338810361, 89.84716940518047, 89.96197172992754, 90.00502260170768, 90.04807347348785, 90.10547463586138, 90.15570065293824, 90.24180239649853, 90.277678122982, 90.30637870416876, 90.30637870416876, 90.33507928535553, 90.39248044772907, 90.42835617421254, 90.49293248188276, 90.60773480662984, 90.60055966133315, 90.60773480662984, 90.65078567840999, 90.66513596900337, 90.67948625959676, 90.71536198608023, 90.744062567267, 90.70818684078353, 90.70818684078353, 90.69383655019014, 90.66513596900337, 90.67948625959676, 90.744062567267, 90.73688742197031], "calls": [0.0, 1.1049723756906076, 2.131018153117601, 3.1211882040611325, 4.305087178015355, 5.316782664848963, 6.07017292100165, 6.974241228384876, 8.021812441701945, 9.105259381502476, 10.095429432446007, 10.884695415082156, 11.702661978905073, 12.628255722178375, 13.424696850111214, 14.192437396857285, 15.340460644328047, 16.244528951711274, 17.198823276171343, 18.23204419889503, 19.22938939513525, 20.320011480232473, 21.0375260099017, 22.18554925737246, 23.154193872425914, 23.979335581545527, 24.854703307741982, 25.80899763220205, 26.89244457200258, 27.825213460572577, 29.04498816101026, 30.214536844371096, 31.154480878237784, 32.367080433378774, 33.57250484322308, 34.61290091124346, 35.674822415153905, 36.67934275669082, 37.71256367941451, 38.79601061921504, 40.2238645332568, 41.39341321661764, 42.41228384874794, 43.3737533185047, 44.36392336944823, 45.497596326325606, 46.30838774485183, 47.00437683863098, 47.958671163091054, 49.13539499174858, 50.254717658032575, 51.46014206787688, 52.14895601635933, 53.23957810145655, 54.14364640883977, 55.21274305804692, 56.06658534835331, 57.31506062997775, 58.40568271507498, 59.424553347205276, 60.6156274664562, 61.383368013202265, 62.25156059410203, 63.26325608093563, 64.39692903781301, 65.35122336227307, 66.11178876372247, 67.01585707110569, 68.01320226734592, 69.20427638659683, 70.29489847169405, 71.20614192437397, 72.13173566764726, 73.10755542799741, 74.12642606012771, 75.2457487264117, 76.38659682858578, 77.49156920427639, 78.91224797302145, 79.90241802396498, 81.0934921432159, 82.1051876300495, 82.95902992035589, 84.03530171485973, 84.92501973164957, 86.03716725263688, 87.1206141924374, 87.90270502977685, 88.67762072181962, 89.58168902920283, 90.39248044772906, 91.49745282341968, 92.60242519911029, 93.42039176293319, 94.43208724976681, 95.25722895888642, 96.12542153978619, 97.08689100954294, 98.06988591518979, 99.03853053024324, 100.0], "exact_score": [76.42247255506925, 76.77405467460716, 77.07541077706824, 77.44851833249623, 77.78575016144077, 78.10863169979191, 78.35976178517615, 78.68981846882399, 78.99117457128507, 79.32123125493291, 79.5149601779436, 79.86654229748153, 80.1678983999426, 80.51230537418384, 80.76343545956806, 81.08631699791921, 81.2800459209299, 81.50965057042406, 81.82535696347851, 82.10518763004951, 82.3993685872139, 82.58592236492788, 82.82987730501543, 83.00208079213604, 83.17428427925665, 83.46129009112435, 83.6837195953218, 83.93484968070604, 84.18597976609027, 84.37970868910097, 84.6093133385951, 84.9034942957595, 85.0828729281768, 85.25507641529741, 85.3627035947478, 85.59948338953865, 85.80038745784603, 86.05869268852695, 86.2882973380211, 86.46050082514172, 86.6542297481524, 86.8479586711631, 87.02016215828371, 87.22824137188778, 87.37891942311832, 87.52242232905216, 87.73767668795294, 87.84530386740332, 87.96728133744709, 88.0462079357107, 88.15383511516109, 88.26146229461148, 88.36908947406185, 88.49824208940231, 88.56281839707253, 88.63456985003947, 88.76372246537993, 88.84264906364353, 88.95027624309392, 88.98615196957739, 89.07942885843438, 89.17270574729139, 89.21575661907154, 89.2659826361484, 89.30903350792853, 89.32338379852193, 89.45253641386238, 89.49558728564253, 89.56016359331277, 89.6175647556863, 89.66061562746646, 89.76106766162016, 89.82564396929038, 89.95479658463084, 89.997847456411, 90.04807347348785, 90.06959890937792, 90.14135036234484, 90.18440123412499, 90.24897754179523, 90.29920355887207, 90.30637870416876, 90.29920355887207, 90.32072899476215, 90.39248044772907, 90.43553131950922, 90.49293248188276, 90.58620937073975, 90.58620937073975, 90.6364353878166, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.72253713137691, 90.70818684078353, 90.71536198608023, 90.69383655019014, 90.67231111430006, 90.67948625959676, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 80.11336729568775, "exact_area100": 86.48837407917831}}, "old_normalized_endpoint_budget": 100}, {"seed": 2027, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 1.0, "validation_area100": 0.8555034527956283, "training_seconds": 33.4665404780535, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7119576927885296, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6500407320953824, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5902553658844094, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5453773062992064, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5005332029602471, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4593412724800027, "operator": "ge"}, {"budget": 0.07, "threshold": 0.42653651104370777, "operator": "ge"}, {"budget": 0.08, "threshold": 0.38342517492489325, "operator": "ge"}, {"budget": 0.09, "threshold": 0.34768134971375164, "operator": "ge"}, {"budget": 0.1, "threshold": 0.31763712612327194, "operator": "ge"}, {"budget": 0.11, "threshold": 0.280923116577133, "operator": "ge"}, {"budget": 0.12, "threshold": 0.252822772582416, "operator": "ge"}, {"budget": 0.13, "threshold": 0.22145257624448986, "operator": "ge"}, {"budget": 0.14, "threshold": 0.19016593992355818, "operator": "ge"}, {"budget": 0.15, "threshold": 0.15717014855768255, "operator": "ge"}, {"budget": 0.16, "threshold": 0.12373441969077062, "operator": "ge"}, {"budget": 0.17, "threshold": 0.09123605999507015, "operator": "ge"}, {"budget": 0.18, "threshold": 0.04986608915055082, "operator": "ge"}, {"budget": 0.19, "threshold": 0.014070119296616583, "operator": "ge"}, {"budget": 0.2, "threshold": -0.03657188518901295, "operator": "ge"}, {"budget": 0.21, "threshold": -0.08648619985437797, "operator": "ge"}, {"budget": 0.22, "threshold": -0.13211791054749938, "operator": "ge"}, {"budget": 0.23, "threshold": -0.16987089918181555, "operator": "ge"}, {"budget": 0.24, "threshold": -0.21656300092276948, "operator": "ge"}, {"budget": 0.25, "threshold": -0.26669269014681884, "operator": "ge"}, {"budget": 0.26, "threshold": -0.31079439110403867, "operator": "ge"}, {"budget": 0.27, "threshold": -0.3574558154860297, "operator": "ge"}, {"budget": 0.28, "threshold": -0.4026053454600452, "operator": "ge"}, {"budget": 0.29, "threshold": -0.45028227363769696, "operator": "ge"}, {"budget": 0.3, "threshold": -0.49793679433045557, "operator": "ge"}, {"budget": 0.31, "threshold": -0.546029084141893, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5955632036481199, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6331230421788031, "operator": "ge"}, {"budget": 0.34, "threshold": -0.6736163062943163, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7063765058624893, "operator": "ge"}, {"budget": 0.36, "threshold": -0.7349983293462932, "operator": "ge"}, {"budget": 0.37, "threshold": -0.759322315869683, "operator": "ge"}, {"budget": 0.38, "threshold": -0.7852015685723384, "operator": "ge"}, {"budget": 0.39, "threshold": -0.8084195977749414, "operator": "ge"}, {"budget": 0.4, "threshold": -0.8310613337465235, "operator": "ge"}, {"budget": 0.41, "threshold": -0.846012860242791, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8611624793427464, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8792841000198441, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8903596110568811, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9028192262428762, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9124480138124559, "operator": "ge"}, {"budget": 0.47, "threshold": -0.922709171821996, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9308527538549721, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9393826617835972, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9477096671302258, "operator": "ge"}, {"budget": 0.51, "threshold": -0.954137021783715, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9583715400217764, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9628091132978659, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9676708107695006, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9704201958418454, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9738720417131876, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9766809454916291, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9789939114319645, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9806450604662242, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9828359611724883, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9842498927180107, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9855787021042522, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9869116500614044, "operator": "ge"}, {"budget": 0.64, "threshold": -0.988097053221402, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9891216615058704, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9903950350358247, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9911613085130377, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9921370694081347, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9927854068586657, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9934083379671371, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9940963770043024, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9946771075756802, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9951389298382903, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9956042459164262, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9960358332721945, "operator": "ge"}, {"budget": 0.76, "threshold": -0.996405817296816, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9967932356794246, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9971109274301394, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9973975776097322, "operator": "ge"}, {"budget": 0.8, "threshold": -0.997683557996094, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9978786882811025, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9980926481222406, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9982711372456227, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9984515464539022, "operator": "ge"}, {"budget": 0.85, "threshold": -0.998604170247788, "operator": "ge"}, {"budget": 0.86, "threshold": -0.998768618190161, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9988873771520639, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9990305830060533, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9991399387644673, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9992385136889167, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9993230714417536, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9994226789125377, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9995110804993652, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9995840202272089, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9996432517530086, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9997190479511485, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9997740358724506, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9998495431430441, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9999145060525583, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.00973932970496, 76.36780292179891, 76.76883414494414, 77.14122028072185, 77.47063878544829, 77.69979948438842, 77.94328272701232, 78.22973360068748, 78.4302492122601, 78.6021197364652, 78.8742480664566, 79.10340876539674, 79.40418218275566, 79.81953594958465, 80.1775995416786, 80.40676024061874, 80.69321111429389, 81.0655972500716, 81.20882268690919, 81.38069321111429, 81.81036952162704, 81.9965625895159, 82.19707820108852, 82.46920653107992, 82.64107705528502, 82.87023775422514, 83.21397880263535, 83.3715267831567, 83.51475221999426, 83.70094528788312, 83.90146089945574, 84.0876539673446, 84.35978229733601, 84.56029790890862, 84.74649097679747, 84.97565167573761, 85.29074763678028, 85.44829561730162, 85.57719851045546, 85.79203666571182, 86.04984245201948, 86.22171297622458, 86.37926095674592, 86.50816384989974, 86.6227441993698, 86.72300200515612, 86.83758235462618, 86.88054998567746, 86.92351761672873, 87.03809796619879, 87.12403322830134, 87.2099684904039, 87.3102262961902, 87.42480664566027, 87.45345173302779, 87.49641936407906, 87.6539673446004, 87.73990260670294, 87.82583786880551, 87.94041821827557, 87.98338584932684, 88.05499856774563, 88.12661128616442, 88.14093382984818, 88.1695789172157, 88.269836723002, 88.35577198510455, 88.39873961615584, 88.45602979089085, 88.58493268404469, 88.68519048983099, 88.82841592666857, 88.92867373245488, 88.97164136350615, 89.0575766256087, 89.15783443139502, 89.18647951876252, 89.24376969349757, 89.25809223718132, 89.31538241191636, 89.35835004296763, 89.34402749928387, 89.38699513033515, 89.41564021770266, 89.41564021770266, 89.51589802348897, 89.60183328559151, 89.63047837295903, 89.70209109137782, 89.73073617874535, 89.80234889716414, 89.80234889716414, 89.90260670295044, 89.97421942136923, 90.00286450873675, 89.95989687768548, 89.94557433400172, 89.95989687768548, 89.88828415926669, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 34, 62, 93, 124, 152, 174, 198, 223, 246, 267, 291, 310, 335, 365, 395, 419, 447, 477, 495, 514, 547, 568, 588, 615, 634, 655, 682, 699, 718, 738, 754, 768, 791, 812, 829, 848, 870, 886, 898, 916, 936, 952, 967, 980, 994, 1007, 1019, 1029, 1037, 1046, 1057, 1068, 1078, 1089, 1095, 1102, 1115, 1125, 1136, 1146, 1151, 1159, 1166, 1168, 1173, 1183, 1192, 1199, 1204, 1215, 1225, 1235, 1244, 1252, 1261, 1269, 1275, 1279, 1283, 1289, 1297, 1298, 1304, 1310, 1314, 1324, 1333, 1338, 1348, 1353, 1359, 1360, 1369, 1374, 1377, 1378, 1380, 1384, 1385, 1389], "harmed": [0, 3, 6, 9, 14, 19, 25, 32, 37, 46, 55, 60, 63, 67, 68, 73, 81, 89, 93, 101, 108, 111, 119, 125, 133, 140, 145, 148, 154, 163, 170, 172, 173, 177, 184, 188, 191, 191, 196, 199, 202, 204, 208, 212, 216, 222, 228, 232, 239, 244, 245, 250, 255, 258, 261, 265, 269, 271, 275, 280, 282, 284, 287, 289, 290, 293, 296, 299, 303, 304, 306, 309, 309, 311, 316, 319, 320, 324, 324, 327, 329, 334, 336, 339, 343, 347, 350, 353, 356, 361, 364, 365, 366, 368, 368, 369, 373, 376, 379, 385, 388], "exact_score": [75.56574047550845, 76.00973932970496, 76.36780292179891, 76.76883414494414, 77.14122028072185, 77.47063878544829, 77.69979948438842, 77.94328272701232, 78.22973360068748, 78.4302492122601, 78.6021197364652, 78.8742480664566, 79.10340876539674, 79.40418218275566, 79.81953594958465, 80.1775995416786, 80.40676024061874, 80.69321111429389, 81.0655972500716, 81.20882268690919, 81.38069321111429, 81.81036952162704, 81.9965625895159, 82.19707820108852, 82.46920653107992, 82.64107705528502, 82.87023775422514, 83.21397880263535, 83.3715267831567, 83.51475221999426, 83.70094528788312, 83.90146089945574, 84.0876539673446, 84.35978229733601, 84.56029790890862, 84.74649097679747, 84.97565167573761, 85.29074763678028, 85.44829561730162, 85.57719851045546, 85.79203666571182, 86.04984245201948, 86.22171297622458, 86.37926095674592, 86.50816384989974, 86.6227441993698, 86.72300200515612, 86.83758235462618, 86.88054998567746, 86.92351761672873, 87.03809796619879, 87.12403322830134, 87.2099684904039, 87.3102262961902, 87.42480664566027, 87.45345173302779, 87.49641936407906, 87.6539673446004, 87.73990260670294, 87.82583786880551, 87.94041821827557, 87.98338584932684, 88.05499856774563, 88.12661128616442, 88.14093382984818, 88.1695789172157, 88.269836723002, 88.35577198510455, 88.39873961615584, 88.45602979089085, 88.58493268404469, 88.68519048983099, 88.82841592666857, 88.92867373245488, 88.97164136350615, 89.0575766256087, 89.15783443139502, 89.18647951876252, 89.24376969349757, 89.25809223718132, 89.31538241191636, 89.35835004296763, 89.34402749928387, 89.38699513033515, 89.41564021770266, 89.41564021770266, 89.51589802348897, 89.60183328559151, 89.63047837295903, 89.70209109137782, 89.73073617874535, 89.80234889716414, 89.80234889716414, 89.90260670295044, 89.97421942136923, 90.00286450873675, 89.95989687768548, 89.94557433400172, 89.95989687768548, 89.88828415926669, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.55034527956283, "threshold_area100": 85.54934116299054, "threshold_area25": 79.34689200802062}, "test": {"score": [76.42247255506925, 76.81710554638732, 77.15433737533185, 77.65659754610031, 77.92207792207793, 78.31671091339601, 78.53196527229677, 78.86202195594461, 79.21360407548254, 79.49343474205352, 79.74456482743776, 80.06027122049221, 80.31857645117314, 80.55535624596398, 80.78496089545814, 81.17241874147952, 81.4594245533472, 81.73208007462152, 81.90428356174213, 82.15541364712635, 82.44241945899404, 82.6791992537849, 82.97338021094927, 83.07383224510296, 83.30343689459711, 83.54739183468466, 83.63349357824497, 83.85592308244242, 84.04965200545311, 84.17162947549687, 84.50168615914473, 84.72411566334218, 85.06134749228671, 85.32682786826433, 85.5492573724618, 85.8219128937361, 85.89366434670302, 86.08739326971371, 86.34569850039465, 86.50355169692186, 86.66858003874579, 86.89100954294324, 87.09191361125063, 87.32151826074478, 87.45067087608524, 87.60134892731578, 87.63722465379924, 87.71615125206286, 87.83812872210662, 87.93858075626032, 88.11795938867762, 88.19688598694124, 88.25428714931478, 88.33321374757838, 88.46954150821554, 88.48389179880894, 88.56999354236923, 88.64174499533615, 88.76372246537993, 88.7924230465667, 88.86417449953362, 88.95027624309392, 89.05790342254431, 89.10095429432447, 89.19423118318146, 89.27315778144508, 89.38796010619215, 89.43818612326899, 89.50993757623593, 89.56016359331277, 89.6175647556863, 89.73236708043338, 89.77541795221353, 89.86151969577384, 89.87586998636723, 89.87586998636723, 89.91174571285069, 89.96914687522423, 89.98349716581761, 90.03372318289445, 90.0839491999713, 90.11264978115807, 90.1772260888283, 90.21310181531177, 90.27050297768531, 90.32790414005885, 90.38530530243239, 90.39965559302576, 90.42118102891584, 90.47858219128938, 90.50010762717945, 90.52880820836621, 90.56468393484968, 90.57185908014638, 90.57903422544307, 90.58620937073975, 90.6364353878166, 90.67231111430006, 90.71536198608023, 90.70101169548683, 90.73688742197031], "calls": [0.0, 1.2125995551409916, 2.1525435890076774, 3.3005668364784384, 4.269211451531894, 5.460285570782808, 6.350003587572648, 7.232546459065796, 8.41644543302002, 9.399440338666858, 10.124130013632776, 11.099949773982923, 11.94661691899261, 12.951137260529524, 13.733228097868983, 14.70904785821913, 15.641816746789122, 16.4956590370955, 17.392552199182035, 18.289445361268566, 19.516395207002944, 20.68594389036378, 21.64023821482385, 22.40080361627323, 23.527301427853914, 24.574872641170984, 25.428714931477366, 26.404534691827507, 27.29425270861735, 28.37769964841788, 29.547248331778718, 30.75984788691971, 31.993972877950778, 33.07024467245462, 34.261318791705534, 35.072110210231756, 35.954653081724906, 36.80849537203129, 37.89194231183182, 38.961038961038966, 40.030135610246106, 40.98442993470618, 41.917198823276166, 43.136973523713856, 44.19171988232762, 45.232115950347996, 45.963980770610604, 46.918275095070676, 47.851043983640665, 48.82686374399082, 49.98923728205496, 51.12291023893234, 51.8117241874148, 52.73014278539141, 53.84946545167539, 54.473703092487625, 55.57867546817823, 56.604721245605226, 57.573365860658676, 58.3841572791849, 59.532180526655665, 60.371672526368656, 61.30444141493865, 62.24438544880534, 63.21303006385879, 64.23907584128578, 65.40862452464663, 66.33421826791992, 67.4607160795006, 68.32173351510367, 69.24732725837698, 70.33077419817751, 71.63665064217551, 72.48331778718519, 73.60981559876588, 74.82241515390686, 75.81976035014709, 76.95343330702447, 77.88620219559445, 78.86202195594461, 80.06744636578891, 80.8208366219416, 81.71772978402812, 82.53569634785104, 83.46129009112434, 84.49451101384803, 85.66405969720887, 86.52507713281193, 87.75202697854631, 88.64174499533615, 89.54581330271938, 90.3566047212456, 91.44722680634283, 92.5808997632202, 93.57106981416374, 94.39621152328334, 95.59446078783095, 96.59180598407117, 98.00530960751955, 99.13180741910024, 100.0], "rescued": [0, 65, 118, 194, 237, 300, 340, 391, 446, 497, 537, 588, 632, 678, 717, 781, 831, 874, 912, 957, 1012, 1058, 1108, 1134, 1179, 1228, 1256, 1296, 1332, 1357, 1413, 1457, 1510, 1559, 1605, 1646, 1670, 1704, 1747, 1784, 1811, 1846, 1880, 1919, 1946, 1975, 1987, 2007, 2030, 2052, 2083, 2107, 2120, 2137, 2161, 2169, 2185, 2202, 2223, 2234, 2251, 2264, 2280, 2293, 2310, 2327, 2347, 2359, 2372, 2385, 2399, 2419, 2430, 2448, 2458, 2468, 2478, 2491, 2498, 2511, 2526, 2533, 2544, 2553, 2567, 2578, 2593, 2600, 2612, 2623, 2630, 2638, 2645, 2655, 2663, 2666, 2678, 2685, 2697, 2699, 2706], "harmed": [0, 10, 16, 22, 28, 36, 46, 51, 57, 69, 74, 81, 89, 102, 109, 119, 129, 134, 148, 158, 173, 186, 195, 207, 220, 235, 251, 260, 269, 277, 287, 300, 306, 318, 333, 336, 350, 357, 364, 379, 383, 387, 393, 400, 409, 417, 424, 433, 439, 447, 453, 466, 471, 477, 482, 488, 492, 499, 503, 510, 517, 518, 519, 526, 530, 536, 540, 545, 548, 554, 560, 564, 569, 575, 583, 593, 598, 603, 608, 614, 622, 625, 627, 631, 637, 640, 647, 652, 661, 664, 668, 672, 674, 683, 690, 692, 697, 699, 705, 709, 711], "exact_score": [76.42247255506925, 76.7381789481237, 77.10411135825501, 77.53462007605654, 77.850326469111, 78.18755829805554, 78.46021381932985, 78.76156992179092, 79.07727631484538, 79.37863241730645, 79.74456482743776, 80.03157063930544, 80.32575159646983, 80.55535624596398, 80.87106263901845, 81.21546961325967, 81.58140202339098, 81.85405754466528, 82.09801248475281, 82.32044198895028, 82.55004663844443, 82.84422759560881, 83.00208079213604, 83.2245102963335, 83.42541436464089, 83.61196814235488, 83.74829590299204, 84.00660113367296, 84.12857860371673, 84.36535839850757, 84.59496304800173, 84.75281624452894, 85.06134749228671, 85.30530243237426, 85.49903135538494, 85.8219128937361, 85.9080146372964, 86.16631986797732, 86.38874937217479, 86.51790198751526, 86.66140489344909, 86.89100954294324, 87.09908875654733, 87.32151826074478, 87.41479514960177, 87.54394776494225, 87.63722465379924, 87.73050154265624, 87.84530386740332, 87.96010619215039, 88.11795938867762, 88.18253569634786, 88.26146229461148, 88.34756403817178, 88.48389179880894, 88.53411781588576, 88.6058692688527, 88.70632130300639, 88.77807275597331, 88.85699935423692, 88.92157566190716, 89.02920284135753, 89.07942885843438, 89.16553060199469, 89.2659826361484, 89.3520843797087, 89.40231039678554, 89.48123699504916, 89.54581330271938, 89.60321446509292, 89.69649135394992, 89.74671737102676, 89.79694338810361, 89.84716940518047, 89.8902202769606, 89.87586998636723, 89.91174571285069, 89.96914687522423, 89.99067231111431, 90.03372318289445, 90.0839491999713, 90.11264978115807, 90.1772260888283, 90.24180239649853, 90.29920355887207, 90.34225443065222, 90.39965559302576, 90.44270646480592, 90.44270646480592, 90.47858219128938, 90.51445791777284, 90.54315849895961, 90.57185908014638, 90.57903422544307, 90.57903422544307, 90.61490995192652, 90.66513596900337, 90.70101169548683, 90.71536198608023, 90.70818684078353, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.4804251487499, "threshold_area100": 86.456590370955, "threshold_area25": 80.33220922723686}, "selected_budget": 95, "selected": {"val": {"score": 90.00286450873675, "calls": 94.98710971068462, "rescued": 1377, "harmed": 369}, "test": {"score": 90.58620937073975, "calls": 94.39621152328334, "rescued": 2666, "harmed": 692}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.46198296546936035, "operator": "ge"}, {"budget": 0.02, "threshold": 0.39724791049957275, "operator": "ge"}, {"budget": 0.03, "threshold": 0.36188194155693054, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3177708387374878, "operator": "ge"}, {"budget": 0.05, "threshold": 0.28387904167175293, "operator": "ge"}, {"budget": 0.06, "threshold": 0.25668007135391235, "operator": "ge"}, {"budget": 0.07, "threshold": 0.22727155685424805, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2068760097026825, "operator": "ge"}, {"budget": 0.09, "threshold": 0.1846446841955185, "operator": "ge"}, {"budget": 0.1, "threshold": 0.16637784242630005, "operator": "ge"}, {"budget": 0.11, "threshold": 0.1434454619884491, "operator": "ge"}, {"budget": 0.12, "threshold": 0.1209203377366066, "operator": "ge"}, {"budget": 0.13, "threshold": 0.10018935799598694, "operator": "ge"}, {"budget": 0.14, "threshold": 0.07798901945352554, "operator": "ge"}, {"budget": 0.15, "threshold": 0.06351056694984436, "operator": "ge"}, {"budget": 0.16, "threshold": 0.04875655472278595, "operator": "ge"}, {"budget": 0.17, "threshold": 0.03372848033905029, "operator": "ge"}, {"budget": 0.18, "threshold": 0.017923712730407715, "operator": "ge"}, {"budget": 0.19, "threshold": 0.0037861019372940063, "operator": "ge"}, {"budget": 0.2, "threshold": -0.006360923405736685, "operator": "ge"}, {"budget": 0.21, "threshold": -0.009048117324709892, "operator": "ge"}, {"budget": 0.22, "threshold": -0.011263343505561352, "operator": "ge"}, {"budget": 0.23, "threshold": -0.012932364828884602, "operator": "ge"}, {"budget": 0.24, "threshold": -0.014666274189949036, "operator": "ge"}, {"budget": 0.25, "threshold": -0.016069136559963226, "operator": "ge"}, {"budget": 0.26, "threshold": -0.017752280458807945, "operator": "ge"}, {"budget": 0.27, "threshold": -0.019011398777365685, "operator": "ge"}, {"budget": 0.28, "threshold": -0.020563293248414993, "operator": "ge"}, {"budget": 0.29, "threshold": -0.022140173241496086, "operator": "ge"}, {"budget": 0.3, "threshold": -0.023880857974290848, "operator": "ge"}, {"budget": 0.31, "threshold": -0.02530716173350811, "operator": "ge"}, {"budget": 0.32, "threshold": -0.026668675243854523, "operator": "ge"}, {"budget": 0.33, "threshold": -0.028265824541449547, "operator": "ge"}, {"budget": 0.34, "threshold": -0.0297149196267128, "operator": "ge"}, {"budget": 0.35, "threshold": -0.031087951734662056, "operator": "ge"}, {"budget": 0.36, "threshold": -0.03228934109210968, "operator": "ge"}, {"budget": 0.37, "threshold": -0.033819522708654404, "operator": "ge"}, {"budget": 0.38, "threshold": -0.03545701503753662, "operator": "ge"}, {"budget": 0.39, "threshold": -0.03696536272764206, "operator": "ge"}, {"budget": 0.4, "threshold": -0.03872750326991081, "operator": "ge"}, {"budget": 0.41, "threshold": -0.04048309847712517, "operator": "ge"}, {"budget": 0.42, "threshold": -0.042328156530857086, "operator": "ge"}, {"budget": 0.43, "threshold": -0.044025272130966187, "operator": "ge"}, {"budget": 0.44, "threshold": -0.04587769880890846, "operator": "ge"}, {"budget": 0.45, "threshold": -0.047252215445041656, "operator": "ge"}, {"budget": 0.46, "threshold": -0.048932917416095734, "operator": "ge"}, {"budget": 0.47, "threshold": -0.05023840442299843, "operator": "ge"}, {"budget": 0.48, "threshold": -0.05198487639427185, "operator": "ge"}, {"budget": 0.49, "threshold": -0.05345945060253143, "operator": "ge"}, {"budget": 0.5, "threshold": -0.05493202805519104, "operator": "ge"}, {"budget": 0.51, "threshold": -0.056769516319036484, "operator": "ge"}, {"budget": 0.52, "threshold": -0.058477696031332016, "operator": "ge"}, {"budget": 0.53, "threshold": -0.06022944673895836, "operator": "ge"}, {"budget": 0.54, "threshold": -0.06178067624568939, "operator": "ge"}, {"budget": 0.55, "threshold": -0.0632920041680336, "operator": "ge"}, {"budget": 0.56, "threshold": -0.06546900421380997, "operator": "ge"}, {"budget": 0.57, "threshold": -0.06763727217912674, "operator": "ge"}, {"budget": 0.58, "threshold": -0.0691157877445221, "operator": "ge"}, {"budget": 0.59, "threshold": -0.07097700238227844, "operator": "ge"}, {"budget": 0.6, "threshold": -0.07249714434146881, "operator": "ge"}, {"budget": 0.61, "threshold": -0.07429603487253189, "operator": "ge"}, {"budget": 0.62, "threshold": -0.07652604579925537, "operator": "ge"}, {"budget": 0.63, "threshold": -0.07856475561857224, "operator": "ge"}, {"budget": 0.64, "threshold": -0.08012310415506363, "operator": "ge"}, {"budget": 0.65, "threshold": -0.08202812820672989, "operator": "ge"}, {"budget": 0.66, "threshold": -0.08403224498033524, "operator": "ge"}, {"budget": 0.67, "threshold": -0.08591047674417496, "operator": "ge"}, {"budget": 0.68, "threshold": -0.08797825872898102, "operator": "ge"}, {"budget": 0.69, "threshold": -0.09023299813270569, "operator": "ge"}, {"budget": 0.7, "threshold": -0.09215577691793442, "operator": "ge"}, {"budget": 0.71, "threshold": -0.09470850229263306, "operator": "ge"}, {"budget": 0.72, "threshold": -0.09682674705982208, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0988280400633812, "operator": "ge"}, {"budget": 0.74, "threshold": -0.10153363645076752, "operator": "ge"}, {"budget": 0.75, "threshold": -0.10371583700180054, "operator": "ge"}, {"budget": 0.76, "threshold": -0.10643763095140457, "operator": "ge"}, {"budget": 0.77, "threshold": -0.10888375341892242, "operator": "ge"}, {"budget": 0.78, "threshold": -0.11194464564323425, "operator": "ge"}, {"budget": 0.79, "threshold": -0.11449815332889557, "operator": "ge"}, {"budget": 0.8, "threshold": -0.11748920381069183, "operator": "ge"}, {"budget": 0.81, "threshold": -0.12045203149318695, "operator": "ge"}, {"budget": 0.82, "threshold": -0.12362923473119736, "operator": "ge"}, {"budget": 0.83, "threshold": -0.12696315348148346, "operator": "ge"}, {"budget": 0.84, "threshold": -0.12997350096702576, "operator": "ge"}, {"budget": 0.85, "threshold": -0.13382413983345032, "operator": "ge"}, {"budget": 0.86, "threshold": -0.13822974264621735, "operator": "ge"}, {"budget": 0.87, "threshold": -0.14194755256175995, "operator": "ge"}, {"budget": 0.88, "threshold": -0.14556275308132172, "operator": "ge"}, {"budget": 0.89, "threshold": -0.14963912963867188, "operator": "ge"}, {"budget": 0.9, "threshold": -0.15406814217567444, "operator": "ge"}, {"budget": 0.91, "threshold": -0.15950234234333038, "operator": "ge"}, {"budget": 0.92, "threshold": -0.16427233815193176, "operator": "ge"}, {"budget": 0.93, "threshold": -0.17048445343971252, "operator": "ge"}, {"budget": 0.94, "threshold": -0.1766338348388672, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1840505748987198, "operator": "ge"}, {"budget": 0.96, "threshold": -0.19264698028564453, "operator": "ge"}, {"budget": 0.97, "threshold": -0.20320914685726166, "operator": "ge"}, {"budget": 0.98, "threshold": -0.21501114964485168, "operator": "ge"}, {"budget": 0.99, "threshold": -0.23972861468791962, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.05270696075623, 76.35348037811515, 76.81180177599542, 77.09825264967057, 77.44199369808078, 77.69979948438842, 78.04354053279863, 78.31566886279003, 78.50186193067888, 78.67373245488398, 78.91721569750787, 79.17502148381553, 79.40418218275566, 79.61902033801202, 80.0057290174735, 80.26353480378114, 80.72185620166141, 80.9653394442853, 81.22314523059295, 81.32340303637926, 81.4093382984818, 81.42366084216557, 81.48095101690059, 81.5525637353194, 81.65282154110571, 81.66714408478946, 81.69578917215698, 81.753079346892, 81.83901460899456, 81.92494987109711, 82.02520767688341, 82.0681753079347, 82.15411057003723, 82.15411057003723, 82.24004583213978, 82.3689487252936, 82.4405614437124, 82.55514179318246, 82.66972214265253, 82.68404468633629, 82.76997994843884, 82.78430249212259, 82.82727012317388, 82.87023775422514, 82.9561730163277, 83.07075336579777, 83.19965625895158, 83.29991406473789, 83.35720423947292, 83.471784588943, 83.62933256946434, 83.68662274419937, 83.81552563735319, 83.87281581208822, 84.03036379260956, 84.0876539673446, 84.23087940418218, 84.34545975365224, 84.38842738470352, 84.57462045259237, 84.68920080206244, 84.76081352048124, 84.78945860784876, 84.88971641363506, 85.01861930678888, 85.13319965625895, 85.29074763678028, 85.36236035519909, 85.50558579203667, 85.51990833572042, 85.63448868519049, 85.87797192781437, 85.97822973360068, 86.25035806359209, 86.3506158693784, 86.50816384989974, 86.72300200515612, 86.85190489830994, 86.92351761672873, 87.05242050988255, 87.12403322830134, 87.22429103408766, 87.38183901460899, 87.45345173302779, 87.61099971354912, 87.78287023775422, 87.91177313090805, 88.04067602406187, 88.1695789172157, 88.29848181036952, 88.48467487825837, 88.62790031509596, 88.84273847035233, 89.07189916929245, 89.22944714981381, 89.35835004296763, 89.48725293612145, 89.68776854769406, 89.7450587224291, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 36, 63, 96, 124, 154, 180, 209, 232, 254, 272, 294, 316, 338, 360, 392, 415, 451, 473, 496, 509, 515, 518, 522, 527, 535, 537, 540, 545, 553, 561, 570, 575, 582, 587, 595, 606, 613, 624, 632, 638, 647, 653, 658, 665, 675, 685, 697, 707, 714, 723, 735, 740, 752, 759, 772, 782, 794, 806, 812, 827, 838, 847, 857, 868, 881, 891, 904, 911, 923, 931, 945, 965, 976, 997, 1008, 1022, 1040, 1052, 1063, 1079, 1091, 1103, 1118, 1129, 1142, 1161, 1174, 1190, 1200, 1213, 1230, 1244, 1264, 1285, 1301, 1315, 1331, 1355, 1370, 1389], "harmed": [0, 2, 8, 9, 17, 23, 31, 36, 40, 49, 55, 60, 64, 70, 77, 82, 87, 91, 96, 101, 107, 107, 109, 109, 109, 110, 111, 112, 113, 115, 117, 119, 121, 122, 127, 129, 131, 133, 136, 136, 141, 144, 149, 151, 155, 159, 161, 164, 167, 170, 171, 172, 173, 176, 179, 181, 187, 189, 193, 196, 198, 201, 205, 213, 217, 221, 223, 225, 227, 229, 236, 242, 245, 249, 251, 255, 258, 261, 264, 270, 277, 284, 289, 293, 299, 301, 308, 312, 319, 320, 324, 328, 332, 337, 342, 347, 352, 359, 369, 380, 388], "exact_score": [75.56574047550845, 76.05270696075623, 76.35348037811515, 76.81180177599542, 77.09825264967057, 77.44199369808078, 77.69979948438842, 78.04354053279863, 78.31566886279003, 78.50186193067888, 78.67373245488398, 78.91721569750787, 79.17502148381553, 79.40418218275566, 79.61902033801202, 80.0057290174735, 80.26353480378114, 80.72185620166141, 80.9653394442853, 81.22314523059295, 81.32340303637926, 81.4093382984818, 81.42366084216557, 81.48095101690059, 81.5525637353194, 81.65282154110571, 81.66714408478946, 81.69578917215698, 81.753079346892, 81.83901460899456, 81.92494987109711, 82.02520767688341, 82.0681753079347, 82.15411057003723, 82.15411057003723, 82.24004583213978, 82.3689487252936, 82.4405614437124, 82.55514179318246, 82.66972214265253, 82.68404468633629, 82.76997994843884, 82.78430249212259, 82.82727012317388, 82.87023775422514, 82.9561730163277, 83.07075336579777, 83.19965625895158, 83.29991406473789, 83.35720423947292, 83.471784588943, 83.62933256946434, 83.68662274419937, 83.81552563735319, 83.87281581208822, 84.03036379260956, 84.0876539673446, 84.23087940418218, 84.34545975365224, 84.38842738470352, 84.57462045259237, 84.68920080206244, 84.76081352048124, 84.78945860784876, 84.88971641363506, 85.01861930678888, 85.13319965625895, 85.29074763678028, 85.36236035519909, 85.50558579203667, 85.51990833572042, 85.63448868519049, 85.87797192781437, 85.97822973360068, 86.25035806359209, 86.3506158693784, 86.50816384989974, 86.72300200515612, 86.85190489830994, 86.92351761672873, 87.05242050988255, 87.12403322830134, 87.22429103408766, 87.38183901460899, 87.45345173302779, 87.61099971354912, 87.78287023775422, 87.91177313090805, 88.04067602406187, 88.1695789172157, 88.29848181036952, 88.48467487825837, 88.62790031509596, 88.84273847035233, 89.07189916929245, 89.22944714981381, 89.35835004296763, 89.48725293612145, 89.68776854769406, 89.7450587224291, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 83.65117332033816, "threshold_area100": 83.6501718705242, "threshold_area25": 79.21082784302492}, "test": {"score": [76.42247255506925, 76.79558011049724, 77.16151252062855, 77.41264260601277, 77.80010045203416, 78.18038315275885, 78.4960895458133, 78.89789768242808, 79.10597689603215, 79.4288584343833, 79.6943388103609, 79.94546889574514, 80.20377412642607, 80.48360479299707, 80.9141135107986, 81.15806845088613, 81.43072397216044, 81.72490492932482, 81.9329841429289, 82.1267130659396, 82.37066800602713, 82.42089402310397, 82.4854703307742, 82.52134605725766, 82.57157207433451, 82.60744780081797, 82.65049867259813, 82.69354954437827, 82.76530099734521, 82.82987730501543, 82.8585778862022, 82.93032933916912, 82.95902992035589, 82.99490564683936, 83.08818253569635, 83.08100739039965, 83.11688311688312, 83.13840855277319, 83.2245102963335, 83.31778718519051, 83.39671378345412, 83.47564038171772, 83.55456697998135, 83.65501901413504, 83.71242017650857, 83.76982133888211, 83.84157279184905, 83.94919997129942, 84.13575374901342, 84.21468034727704, 84.26490636435389, 84.35100810791418, 84.41558441558442, 84.50168615914473, 84.6164884838918, 84.71694051804549, 84.8173925521992, 84.96807060342972, 85.09722321877018, 85.1689746717371, 85.31247757767095, 85.40575446652795, 85.54208222716511, 85.71428571428572, 85.80038745784603, 85.9080146372964, 86.08021812441702, 86.18067015857072, 86.35287364569133, 86.53225227810863, 86.68293032933917, 86.82643323527302, 86.94123556002009, 87.08473846595393, 87.22106622659109, 87.35021884193155, 87.47937145727201, 87.60134892731578, 87.75202697854631, 87.86682930329339, 87.93858075626032, 88.03903279041401, 88.16101026045777, 88.29016287579825, 88.40496520054532, 88.5125923799957, 88.69914615770969, 88.77807275597331, 88.81394848245677, 89.00767740546746, 89.20140632847816, 89.31620865322523, 89.45253641386238, 89.624739900983, 89.79694338810361, 90.04089832819116, 90.12700007175145, 90.29202841357538, 90.38530530243239, 90.56468393484968, 90.73688742197031], "calls": [0.0, 1.0260457774269929, 2.166893879601062, 2.8485326827868267, 4.018081366147665, 5.108703451244888, 5.998421468034728, 7.275597330845949, 8.021812441701945, 9.033507928535553, 9.794073329984933, 10.71249192796154, 11.717012269498458, 12.814809499892373, 14.0848102174069, 14.909951926526514, 15.684867618569276, 16.603286216545886, 17.435603070962184, 18.368371959532183, 19.423118318145942, 20.348712061419246, 21.28865609528593, 22.17837411207577, 23.22594532539284, 24.043911889215757, 25.04125708545598, 25.687020162158287, 26.74894166606874, 27.674535409342038, 28.793858075626034, 29.626174930042332, 30.53024323742556, 31.506062997775704, 32.51775848460932, 33.39312621080577, 34.23979335581546, 35.22996340675898, 36.3277606371529, 37.296405252206355, 38.444428499677116, 39.556576020664416, 40.84810217406903, 41.93154911386955, 43.12262323312047, 44.00516610461362, 45.03121188204061, 45.76307670230322, 47.03307741981775, 48.04477290665136, 48.78381287221066, 50.12556504269211, 51.13008538422903, 52.235057759919634, 53.088900050226016, 53.96426777642247, 55.29884480160723, 56.590370955011835, 57.59489129654875, 58.628112219272445, 59.62545741551266, 60.66585348353304, 61.79952644041041, 63.01212599555141, 63.607663055176864, 64.59783310612039, 65.53777713998709, 66.48489631915047, 67.69032072899476, 68.80964339527875, 69.58455908732152, 70.76845806127574, 71.69405180454905, 72.63399583841573, 73.77484394058979, 74.72913826504987, 75.84846093133386, 76.89603214465093, 78.05123053741838, 78.9552988448016, 79.98851976752529, 80.95716438257875, 81.95450957881897, 82.93032933916912, 83.70524503121189, 84.72411566334218, 86.05151754323025, 86.9268852694267, 87.88117959388677, 88.76372246537993, 89.78976824280693, 90.95931692616776, 91.8346846523642, 92.92530673746143, 93.92265193370166, 94.95587285642534, 95.89581689029202, 96.89316208653224, 97.74700437683863, 99.00265480375977, 100.0], "rescued": [0, 58, 120, 162, 229, 290, 337, 403, 435, 488, 533, 579, 627, 677, 743, 785, 830, 879, 919, 958, 998, 1008, 1017, 1025, 1033, 1039, 1047, 1055, 1067, 1080, 1089, 1100, 1109, 1121, 1139, 1144, 1153, 1164, 1182, 1196, 1212, 1225, 1246, 1264, 1277, 1289, 1303, 1320, 1351, 1366, 1380, 1404, 1420, 1436, 1458, 1474, 1495, 1526, 1549, 1570, 1597, 1618, 1644, 1676, 1690, 1711, 1737, 1758, 1788, 1818, 1842, 1871, 1893, 1916, 1945, 1973, 2001, 2031, 2061, 2086, 2108, 2128, 2153, 2178, 2198, 2222, 2260, 2284, 2306, 2337, 2373, 2404, 2438, 2471, 2502, 2547, 2574, 2608, 2631, 2671, 2706], "harmed": [0, 6, 17, 24, 37, 45, 48, 58, 61, 69, 77, 88, 100, 111, 117, 125, 132, 140, 151, 163, 169, 172, 172, 175, 176, 177, 179, 181, 183, 187, 192, 193, 198, 205, 210, 216, 220, 228, 234, 235, 240, 242, 252, 256, 261, 265, 269, 271, 276, 280, 287, 299, 306, 310, 316, 318, 325, 335, 340, 351, 358, 366, 373, 381, 383, 389, 391, 398, 404, 409, 412, 421, 427, 430, 440, 450, 460, 473, 482, 491, 503, 509, 517, 524, 528, 537, 549, 562, 579, 583, 592, 607, 622, 631, 638, 649, 664, 675, 685, 700, 711], "exact_score": [76.42247255506925, 76.79558011049724, 77.1112865035517, 77.44851833249623, 77.78575016144077, 78.18755829805554, 78.4960895458133, 78.80462079357106, 79.09880175073546, 79.41450814378992, 79.74456482743776, 80.01004520341537, 80.27552557939298, 80.53383081007391, 80.89976322020522, 81.17241874147952, 81.54552629690751, 81.78230609169836, 82.06931190356606, 82.28456626246683, 82.40654373251058, 82.46394489488412, 82.53569634785106, 82.55004663844443, 82.60744780081797, 82.65049867259813, 82.72225012556504, 82.78682643323528, 82.84422759560881, 82.8585778862022, 82.93750448446582, 82.97338021094927, 83.02360622802611, 83.09535768099305, 83.12405826217982, 83.13840855277319, 83.19580971514672, 83.28191145870704, 83.37518834756405, 83.43258950993759, 83.55456697998135, 83.56174212527804, 83.65501901413504, 83.70524503121189, 83.76982133888211, 83.83439764655235, 83.99225084307957, 84.12857860371673, 84.20750520198035, 84.28643180024396, 84.32948267202411, 84.40840927028772, 84.48733586855134, 84.59496304800173, 84.73129080863887, 84.78869197101243, 84.91066944105619, 85.00394632991318, 85.10439836406688, 85.21920068881394, 85.31965272296765, 85.4416301930114, 85.55643251775848, 85.71428571428572, 85.84343832962618, 85.97976609026333, 86.12326899619718, 86.2811221927244, 86.38874937217479, 86.55377771399871, 86.74033149171271, 86.8407835258664, 86.96993614120686, 87.1134390471407, 87.23541651718448, 87.38609456841502, 87.46502116667862, 87.60852407261247, 87.75202697854631, 87.88117959388678, 87.93858075626032, 88.03903279041401, 88.17536055105116, 88.29016287579825, 88.44801607232546, 88.54129296118248, 88.6847958671163, 88.78524790127, 88.83547391834685, 89.03637798665423, 89.20858147377486, 89.30903350792853, 89.46688670445577, 89.64626533687307, 89.8184688239937, 90.04089832819116, 90.13417521704815, 90.30637870416876, 90.42835617421254, 90.55750878955298, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 84.57254490191976, "threshold_area100": 84.52805481811006, "threshold_area25": 80.18368371959532}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.2265777587890625, "operator": "ge"}, {"budget": 0.02, "threshold": 1.957787275314331, "operator": "ge"}, {"budget": 0.03, "threshold": 1.7860629558563232, "operator": "ge"}, {"budget": 0.04, "threshold": 1.6073048114776611, "operator": "ge"}, {"budget": 0.05, "threshold": 1.4978840351104736, "operator": "ge"}, {"budget": 0.06, "threshold": 1.3681035041809082, "operator": "ge"}, {"budget": 0.07, "threshold": 1.2137371301651, "operator": "ge"}, {"budget": 0.08, "threshold": 1.1064670085906982, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0043338537216187, "operator": "ge"}, {"budget": 0.1, "threshold": 0.8959108591079712, "operator": "ge"}, {"budget": 0.11, "threshold": 0.8046845197677612, "operator": "ge"}, {"budget": 0.12, "threshold": 0.7209985256195068, "operator": "ge"}, {"budget": 0.13, "threshold": 0.628438413143158, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5295190811157227, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4417848587036133, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3314165472984314, "operator": "ge"}, {"budget": 0.17, "threshold": 0.24259155988693237, "operator": "ge"}, {"budget": 0.18, "threshold": 0.13945022225379944, "operator": "ge"}, {"budget": 0.19, "threshold": 0.037661463022232056, "operator": "ge"}, {"budget": 0.2, "threshold": -0.09502506256103516, "operator": "ge"}, {"budget": 0.21, "threshold": -0.21885794401168823, "operator": "ge"}, {"budget": 0.22, "threshold": -0.34268850088119507, "operator": "ge"}, {"budget": 0.23, "threshold": -0.44629019498825073, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5636487007141113, "operator": "ge"}, {"budget": 0.25, "threshold": -0.6679275035858154, "operator": "ge"}, {"budget": 0.26, "threshold": -0.792151927947998, "operator": "ge"}, {"budget": 0.27, "threshold": -0.9225456714630127, "operator": "ge"}, {"budget": 0.28, "threshold": -1.026136875152588, "operator": "ge"}, {"budget": 0.29, "threshold": -1.139541506767273, "operator": "ge"}, {"budget": 0.3, "threshold": -1.2911955118179321, "operator": "ge"}, {"budget": 0.31, "threshold": -1.4281843900680542, "operator": "ge"}, {"budget": 0.32, "threshold": -1.5600090026855469, "operator": "ge"}, {"budget": 0.33, "threshold": -1.700549602508545, "operator": "ge"}, {"budget": 0.34, "threshold": -1.8484140634536743, "operator": "ge"}, {"budget": 0.35, "threshold": -1.970489740371704, "operator": "ge"}, {"budget": 0.36, "threshold": -2.108614921569824, "operator": "ge"}, {"budget": 0.37, "threshold": -2.237855911254883, "operator": "ge"}, {"budget": 0.38, "threshold": -2.3786509037017822, "operator": "ge"}, {"budget": 0.39, "threshold": -2.515089750289917, "operator": "ge"}, {"budget": 0.4, "threshold": -2.6655755043029785, "operator": "ge"}, {"budget": 0.41, "threshold": -2.781961441040039, "operator": "ge"}, {"budget": 0.42, "threshold": -2.9165472984313965, "operator": "ge"}, {"budget": 0.43, "threshold": -3.02980375289917, "operator": "ge"}, {"budget": 0.44, "threshold": -3.1952872276306152, "operator": "ge"}, {"budget": 0.45, "threshold": -3.3160033226013184, "operator": "ge"}, {"budget": 0.46, "threshold": -3.4499354362487793, "operator": "ge"}, {"budget": 0.47, "threshold": -3.6002728939056396, "operator": "ge"}, {"budget": 0.48, "threshold": -3.742741584777832, "operator": "ge"}, {"budget": 0.49, "threshold": -3.9018189907073975, "operator": "ge"}, {"budget": 0.5, "threshold": -4.024624347686768, "operator": "ge"}, {"budget": 0.51, "threshold": -4.1741251945495605, "operator": "ge"}, {"budget": 0.52, "threshold": -4.295970916748047, "operator": "ge"}, {"budget": 0.53, "threshold": -4.435344696044922, "operator": "ge"}, {"budget": 0.54, "threshold": -4.552717685699463, "operator": "ge"}, {"budget": 0.55, "threshold": -4.685269832611084, "operator": "ge"}, {"budget": 0.56, "threshold": -4.8092498779296875, "operator": "ge"}, {"budget": 0.57, "threshold": -4.907227993011475, "operator": "ge"}, {"budget": 0.58, "threshold": -5.044401168823242, "operator": "ge"}, {"budget": 0.59, "threshold": -5.16937255859375, "operator": "ge"}, {"budget": 0.6, "threshold": -5.284322261810303, "operator": "ge"}, {"budget": 0.61, "threshold": -5.416153907775879, "operator": "ge"}, {"budget": 0.62, "threshold": -5.526625633239746, "operator": "ge"}, {"budget": 0.63, "threshold": -5.623592376708984, "operator": "ge"}, {"budget": 0.64, "threshold": -5.721132755279541, "operator": "ge"}, {"budget": 0.65, "threshold": -5.8392653465271, "operator": "ge"}, {"budget": 0.66, "threshold": -5.959394454956055, "operator": "ge"}, {"budget": 0.67, "threshold": -6.080674648284912, "operator": "ge"}, {"budget": 0.68, "threshold": -6.173195838928223, "operator": "ge"}, {"budget": 0.69, "threshold": -6.262954235076904, "operator": "ge"}, {"budget": 0.7, "threshold": -6.375019073486328, "operator": "ge"}, {"budget": 0.71, "threshold": -6.477879047393799, "operator": "ge"}, {"budget": 0.72, "threshold": -6.590500831604004, "operator": "ge"}, {"budget": 0.73, "threshold": -6.6842145919799805, "operator": "ge"}, {"budget": 0.74, "threshold": -6.789634704589844, "operator": "ge"}, {"budget": 0.75, "threshold": -6.894872188568115, "operator": "ge"}, {"budget": 0.76, "threshold": -7.010485649108887, "operator": "ge"}, {"budget": 0.77, "threshold": -7.088004112243652, "operator": "ge"}, {"budget": 0.78, "threshold": -7.188697338104248, "operator": "ge"}, {"budget": 0.79, "threshold": -7.304052352905273, "operator": "ge"}, {"budget": 0.8, "threshold": -7.420321464538574, "operator": "ge"}, {"budget": 0.81, "threshold": -7.5166015625, "operator": "ge"}, {"budget": 0.82, "threshold": -7.6079535484313965, "operator": "ge"}, {"budget": 0.83, "threshold": -7.703011989593506, "operator": "ge"}, {"budget": 0.84, "threshold": -7.804083824157715, "operator": "ge"}, {"budget": 0.85, "threshold": -7.919240474700928, "operator": "ge"}, {"budget": 0.86, "threshold": -8.028585433959961, "operator": "ge"}, {"budget": 0.87, "threshold": -8.125134468078613, "operator": "ge"}, {"budget": 0.88, "threshold": -8.251703262329102, "operator": "ge"}, {"budget": 0.89, "threshold": -8.341910362243652, "operator": "ge"}, {"budget": 0.9, "threshold": -8.45736312866211, "operator": "ge"}, {"budget": 0.91, "threshold": -8.598941802978516, "operator": "ge"}, {"budget": 0.92, "threshold": -8.754629135131836, "operator": "ge"}, {"budget": 0.93, "threshold": -8.904452323913574, "operator": "ge"}, {"budget": 0.94, "threshold": -9.036402702331543, "operator": "ge"}, {"budget": 0.95, "threshold": -9.218911170959473, "operator": "ge"}, {"budget": 0.96, "threshold": -9.414280891418457, "operator": "ge"}, {"budget": 0.97, "threshold": -9.67419147491455, "operator": "ge"}, {"budget": 0.98, "threshold": -10.072924613952637, "operator": "ge"}, {"budget": 0.99, "threshold": -10.635442733764648, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.00973932970496, 76.42509309653394, 76.74018905757663, 77.0552850186193, 77.44199369808078, 77.80005729017473, 78.10083070753365, 78.27270123173875, 78.51618447436263, 78.6021197364652, 78.83128043540533, 79.13205385276424, 79.27527928960183, 79.7622457748496, 80.12030936694356, 80.40676024061874, 80.67888857061014, 81.02262961902034, 81.22314523059295, 81.39501575479805, 81.78172443425952, 82.08249785161844, 82.31165855055858, 82.46920653107992, 82.6124319679175, 82.97049556001146, 83.18533371526783, 83.42881695789171, 83.60068748209682, 83.70094528788312, 83.9157834431395, 84.15926668576338, 84.37410484101976, 84.60326553995989, 84.78945860784876, 84.94700658837009, 85.20481237467774, 85.36236035519909, 85.62016614150673, 85.86364938413061, 86.06416499570322, 86.26468060727585, 86.50816384989974, 86.55113148095101, 86.63706674305357, 86.6943569177886, 86.73732454883987, 86.86622744199369, 86.93784016041248, 87.10971068461758, 87.15267831566887, 87.2099684904039, 87.29590375250645, 87.28158120882267, 87.3102262961902, 87.42480664566027, 87.53938699513033, 87.7255800630192, 87.84016041248925, 87.9260956745918, 87.98338584932684, 88.06932111142939, 88.08364365511314, 88.14093382984818, 88.25551417931825, 88.34144944142079, 88.4417072472071, 88.49899742194214, 88.57061014036093, 88.68519048983099, 88.77112575193354, 88.91435118877112, 89.07189916929245, 89.14351188771126, 89.17215697507876, 89.24376969349757, 89.28673732454884, 89.31538241191636, 89.31538241191636, 89.31538241191636, 89.31538241191636, 89.35835004296763, 89.41564021770266, 89.51589802348897, 89.55886565454024, 89.51589802348897, 89.5445431108565, 89.60183328559151, 89.6448009166428, 89.68776854769406, 89.80234889716414, 89.85963907189917, 89.9169292466342, 90.03150959610426, 89.94557433400172, 89.97421942136923, 89.97421942136923, 89.97421942136923, 89.88828415926669, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 33, 65, 93, 119, 149, 179, 207, 226, 249, 266, 287, 312, 328, 362, 393, 419, 447, 476, 496, 514, 546, 570, 591, 613, 632, 660, 677, 703, 722, 737, 754, 773, 793, 811, 831, 847, 866, 881, 901, 918, 937, 953, 974, 983, 994, 1004, 1014, 1027, 1035, 1049, 1059, 1066, 1075, 1080, 1087, 1098, 1109, 1125, 1135, 1144, 1148, 1157, 1163, 1168, 1179, 1188, 1199, 1204, 1211, 1222, 1229, 1241, 1253, 1260, 1265, 1274, 1280, 1284, 1287, 1290, 1294, 1302, 1306, 1313, 1318, 1322, 1328, 1333, 1340, 1345, 1354, 1360, 1364, 1374, 1375, 1379, 1382, 1383, 1385, 1389], "harmed": [0, 2, 5, 11, 15, 18, 23, 30, 37, 43, 54, 59, 63, 69, 69, 75, 81, 90, 95, 101, 107, 112, 115, 120, 131, 140, 143, 145, 154, 161, 169, 171, 173, 178, 180, 187, 192, 193, 197, 199, 199, 204, 206, 210, 216, 221, 227, 234, 238, 241, 243, 250, 253, 256, 262, 267, 270, 273, 276, 278, 281, 281, 284, 289, 290, 293, 296, 300, 301, 303, 306, 307, 309, 310, 312, 315, 319, 322, 324, 327, 330, 334, 339, 339, 339, 341, 348, 352, 353, 357, 359, 360, 362, 362, 364, 371, 373, 376, 377, 385, 388], "exact_score": [75.56574047550845, 76.00973932970496, 76.42509309653394, 76.74018905757663, 77.0552850186193, 77.44199369808078, 77.80005729017473, 78.10083070753365, 78.27270123173875, 78.51618447436263, 78.6021197364652, 78.83128043540533, 79.13205385276424, 79.27527928960183, 79.7622457748496, 80.12030936694356, 80.40676024061874, 80.67888857061014, 81.02262961902034, 81.22314523059295, 81.39501575479805, 81.78172443425952, 82.08249785161844, 82.31165855055858, 82.46920653107992, 82.6124319679175, 82.97049556001146, 83.18533371526783, 83.42881695789171, 83.60068748209682, 83.70094528788312, 83.9157834431395, 84.15926668576338, 84.37410484101976, 84.60326553995989, 84.78945860784876, 84.94700658837009, 85.20481237467774, 85.36236035519909, 85.62016614150673, 85.86364938413061, 86.06416499570322, 86.26468060727585, 86.50816384989974, 86.55113148095101, 86.63706674305357, 86.6943569177886, 86.73732454883987, 86.86622744199369, 86.93784016041248, 87.10971068461758, 87.15267831566887, 87.2099684904039, 87.29590375250645, 87.28158120882267, 87.3102262961902, 87.42480664566027, 87.53938699513033, 87.7255800630192, 87.84016041248925, 87.9260956745918, 87.98338584932684, 88.06932111142939, 88.08364365511314, 88.14093382984818, 88.25551417931825, 88.34144944142079, 88.4417072472071, 88.49899742194214, 88.57061014036093, 88.68519048983099, 88.77112575193354, 88.91435118877112, 89.07189916929245, 89.14351188771126, 89.17215697507876, 89.24376969349757, 89.28673732454884, 89.31538241191636, 89.31538241191636, 89.31538241191636, 89.31538241191636, 89.35835004296763, 89.41564021770266, 89.51589802348897, 89.55886565454024, 89.51589802348897, 89.5445431108565, 89.60183328559151, 89.6448009166428, 89.68776854769406, 89.80234889716414, 89.85963907189917, 89.9169292466342, 90.03150959610426, 89.94557433400172, 89.97421942136923, 89.97421942136923, 89.97421942136923, 89.88828415926669, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.5685212890601, "threshold_area100": 85.56753079346892, "threshold_area25": 79.3297049556001}, "test": {"score": [76.42247255506925, 76.74535409342039, 77.24043911889216, 77.59202123843008, 77.97230393915477, 78.16603286216545, 78.50326469110999, 78.89072253713138, 79.09880175073546, 79.41450814378992, 79.67281337447083, 80.03874578460214, 80.21812441701944, 80.48360479299707, 80.79931118605153, 81.05044127143574, 81.52400086101744, 81.71772978402812, 81.95450957881899, 82.13388821123628, 82.44959460429074, 82.68637439908159, 82.95902992035589, 83.14558369806988, 83.35366291167396, 83.57609241587143, 83.71242017650857, 83.89179880892588, 83.99225084307957, 84.25055607376049, 84.51603644973811, 84.73129080863887, 85.05417234699003, 85.3555284494511, 85.59948338953865, 85.80756260314271, 85.97259094496664, 86.13044414149387, 86.39592451747149, 86.48920140632848, 86.72598120111932, 86.87665925234987, 87.1277893377341, 87.28564253426131, 87.4937217478654, 87.60134892731578, 87.65875008968932, 87.79507785032648, 87.88835473918347, 88.02468249982063, 88.132309679271, 88.22558656812801, 88.29016287579825, 88.37626461935855, 88.44084092702877, 88.49824208940231, 88.56281839707253, 88.66327043122624, 88.79959819186338, 88.83547391834685, 88.90005022601709, 89.04355313195093, 89.08660400373107, 89.15118031140132, 89.24445720025831, 89.344909234412, 89.41666068737892, 89.5386381574227, 89.6103896103896, 89.69649135394992, 89.76106766162016, 89.78976824280693, 89.83281911458707, 89.8902202769606, 89.8902202769606, 89.8973954222573, 89.91892085814737, 89.93327114874077, 89.98349716581761, 90.01937289230108, 90.06242376408122, 90.15570065293824, 90.21310181531177, 90.25615268709191, 90.29920355887207, 90.37095501183899, 90.38530530243239, 90.3637798665423, 90.3566047212456, 90.40683073832245, 90.45705675539931, 90.50728277247615, 90.53598335366291, 90.5503336442563, 90.59338451603645, 90.65078567840999, 90.66513596900337, 90.69383655019014, 90.72253713137691, 90.72253713137691, 90.73688742197031], "calls": [0.0, 1.0906220850972232, 2.2242950419746, 3.264691109994977, 4.362488340388893, 5.108703451244888, 6.192150391045419, 7.397574800889719, 8.308818253569635, 9.126784817392553, 10.07390399655593, 11.021023175719307, 11.81746430365215, 12.750233192222144, 13.675826935495444, 14.601420678768745, 15.76379421683289, 16.517184472985576, 17.34232618210519, 18.289445361268566, 19.480519480519483, 20.664418454473704, 21.72633995838416, 22.65910884695415, 23.728205496161298, 24.59639807706106, 25.486116093850903, 26.62696419602497, 27.308602999210734, 28.38487479371457, 29.640525220635716, 30.731147305732943, 31.92939657028055, 33.04871923656454, 34.12499103106838, 34.971658176078066, 35.9618282270216, 36.887421970294895, 37.999569491282195, 38.982564396929035, 40.13776278969649, 40.977254789409486, 41.960249695056326, 42.88584343832962, 44.15584415584416, 45.024036736743916, 46.007031642390764, 46.9900265480376, 47.86539427423405, 48.99906723111143, 49.75245748726412, 50.81437899117457, 51.66822128148095, 52.715792494798016, 53.46918275095071, 54.46652794719093, 55.442347707541074, 56.24596398077061, 57.45138839061491, 58.40568271507498, 59.24517471478797, 60.36449738107197, 61.32596685082873, 62.194159431728494, 63.19150462796872, 64.19602496950563, 65.39427423405324, 66.66427495156778, 67.51094209657745, 68.501112147521, 69.54150821554137, 70.5316782664849, 71.55054889861519, 72.52636865896534, 73.64569132524935, 74.6932625385664, 75.95608811078425, 76.68077778575017, 77.59202123843008, 78.76156992179092, 79.9669943316352, 80.9069383655019, 81.9329841429289, 82.74377556145512, 83.74829590299203, 84.75281624452894, 85.57795795364856, 86.4676759704384, 87.48654660256871, 88.24711200401808, 89.33055894381862, 90.32072899476215, 91.43287651574944, 92.3082442419459, 93.0472842075052, 94.23835832675613, 95.22852837769965, 96.45547822343403, 97.89768242806916, 99.11745712850684, 100.0], "rescued": [0, 55, 129, 183, 243, 280, 335, 396, 436, 484, 528, 586, 622, 668, 720, 765, 839, 874, 915, 955, 1011, 1056, 1105, 1146, 1189, 1231, 1261, 1302, 1322, 1367, 1416, 1456, 1509, 1560, 1608, 1643, 1679, 1708, 1752, 1780, 1816, 1843, 1880, 1907, 1942, 1966, 1985, 2011, 2035, 2063, 2083, 2102, 2117, 2135, 2150, 2167, 2183, 2199, 2225, 2235, 2250, 2277, 2290, 2303, 2320, 2338, 2351, 2374, 2388, 2402, 2419, 2430, 2440, 2451, 2462, 2468, 2478, 2486, 2496, 2508, 2520, 2535, 2551, 2559, 2568, 2581, 2587, 2591, 2598, 2607, 2621, 2632, 2641, 2647, 2655, 2667, 2672, 2685, 2694, 2700, 2706], "harmed": [0, 10, 15, 20, 27, 37, 45, 52, 63, 67, 75, 82, 93, 102, 110, 120, 128, 136, 144, 159, 171, 183, 194, 209, 223, 234, 245, 261, 267, 276, 288, 298, 306, 315, 329, 335, 348, 355, 362, 377, 380, 386, 388, 393, 399, 408, 419, 426, 437, 446, 451, 457, 463, 469, 475, 484, 491, 493, 500, 505, 511, 518, 525, 529, 533, 537, 540, 546, 550, 552, 560, 567, 571, 574, 585, 590, 597, 603, 606, 613, 619, 621, 629, 631, 634, 637, 641, 648, 656, 658, 665, 669, 674, 678, 680, 684, 687, 696, 701, 707, 711], "exact_score": [76.42247255506925, 76.70230322164025, 77.13281193944178, 77.49156920427639, 77.84315132381431, 78.15168257157208, 78.48891440051662, 78.74721963119752, 79.03422544306522, 79.36428212671306, 79.64411279328407, 80.03874578460214, 80.26835043409629, 80.56970653655738, 80.89258807490852, 81.20829446796299, 81.57422687809428, 81.8468823993686, 82.10518763004951, 82.33479227954366, 82.57157207433451, 82.82270215971874, 83.00925593743274, 83.24603573222358, 83.38236349286073, 83.63349357824497, 83.79852192006888, 83.95637511659612, 84.13575374901342, 84.4227595608811, 84.57343761211165, 84.8173925521992, 85.0828729281768, 85.34835330415441, 85.53490708186841, 85.80038745784603, 85.97259094496664, 86.17349501327402, 86.39592451747149, 86.49637655162518, 86.70445576522926, 86.87665925234987, 87.1277893377341, 87.28564253426131, 87.45784602138194, 87.60134892731578, 87.65875008968932, 87.79507785032648, 87.89552988448015, 88.02468249982063, 88.16101026045777, 88.23993685872139, 88.3260386022817, 88.39779005524862, 88.48389179880894, 88.5197675252924, 88.64174499533615, 88.74219702948984, 88.82829877305016, 88.878524790127, 88.98615196957739, 89.07942885843438, 89.158355456698, 89.24445720025831, 89.30903350792853, 89.38796010619215, 89.46688670445577, 89.57451388390615, 89.67496591805984, 89.75389251632346, 89.8041185334003, 89.78259309751022, 89.87586998636723, 89.86869484107054, 89.8973954222573, 89.91174571285069, 89.91892085814737, 89.95479658463084, 90.01219774700438, 90.01937289230108, 90.06242376408122, 90.15570065293824, 90.20592667001507, 90.27050297768531, 90.31355384946545, 90.39248044772907, 90.39248044772907, 90.39248044772907, 90.37813015713569, 90.43553131950922, 90.48575733658608, 90.52880820836621, 90.5503336442563, 90.60055966133315, 90.6364353878166, 90.65078567840999, 90.70101169548683, 90.71536198608023, 90.71536198608023, 90.71536198608023, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.49871850010436, "threshold_area100": 86.47086891009543, "threshold_area25": 80.26763291956664}, "selected_budget": 94, "selected": {"val": {"score": 90.03150959610426, "calls": 93.9988541965053, "rescued": 1374, "harmed": 364}, "test": {"score": 90.59338451603645, "calls": 93.0472842075052, "rescued": 2655, "harmed": 680}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.7822855710983276, "operator": "ge"}, {"budget": 0.02, "threshold": 1.5507383346557617, "operator": "ge"}, {"budget": 0.03, "threshold": 1.3561159372329712, "operator": "ge"}, {"budget": 0.04, "threshold": 1.223555564880371, "operator": "ge"}, {"budget": 0.05, "threshold": 1.1000347137451172, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9929520487785339, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9113104939460754, "operator": "ge"}, {"budget": 0.08, "threshold": 0.8081380128860474, "operator": "ge"}, {"budget": 0.09, "threshold": 0.725607693195343, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6580336093902588, "operator": "ge"}, {"budget": 0.11, "threshold": 0.5773678421974182, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5168522000312805, "operator": "ge"}, {"budget": 0.13, "threshold": 0.45036613941192627, "operator": "ge"}, {"budget": 0.14, "threshold": 0.38501864671707153, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3169676661491394, "operator": "ge"}, {"budget": 0.16, "threshold": 0.24874351918697357, "operator": "ge"}, {"budget": 0.17, "threshold": 0.18298104405403137, "operator": "ge"}, {"budget": 0.18, "threshold": 0.09981493651866913, "operator": "ge"}, {"budget": 0.19, "threshold": 0.02814200520515442, "operator": "ge"}, {"budget": 0.2, "threshold": -0.07317648082971573, "operator": "ge"}, {"budget": 0.21, "threshold": -0.17340552806854248, "operator": "ge"}, {"budget": 0.22, "threshold": -0.2657895088195801, "operator": "ge"}, {"budget": 0.23, "threshold": -0.3430674076080322, "operator": "ge"}, {"budget": 0.24, "threshold": -0.4400944113731384, "operator": "ge"}, {"budget": 0.25, "threshold": -0.5465996265411377, "operator": "ge"}, {"budget": 0.26, "threshold": -0.6428492069244385, "operator": "ge"}, {"budget": 0.27, "threshold": -0.7479320168495178, "operator": "ge"}, {"budget": 0.28, "threshold": -0.8535086512565613, "operator": "ge"}, {"budget": 0.29, "threshold": -0.9701086282730103, "operator": "ge"}, {"budget": 0.3, "threshold": -1.0931179523468018, "operator": "ge"}, {"budget": 0.31, "threshold": -1.2254118919372559, "operator": "ge"}, {"budget": 0.32, "threshold": -1.37248694896698, "operator": "ge"}, {"budget": 0.33, "threshold": -1.4932231903076172, "operator": "ge"}, {"budget": 0.34, "threshold": -1.6346684694290161, "operator": "ge"}, {"budget": 0.35, "threshold": -1.7598295211791992, "operator": "ge"}, {"budget": 0.36, "threshold": -1.8790262937545776, "operator": "ge"}, {"budget": 0.37, "threshold": -1.9892256259918213, "operator": "ge"}, {"budget": 0.38, "threshold": -2.117587089538574, "operator": "ge"}, {"budget": 0.39, "threshold": -2.244901180267334, "operator": "ge"}, {"budget": 0.4, "threshold": -2.383115768432617, "operator": "ge"}, {"budget": 0.41, "threshold": -2.483914375305176, "operator": "ge"}, {"budget": 0.42, "threshold": -2.5956523418426514, "operator": "ge"}, {"budget": 0.43, "threshold": -2.7452070713043213, "operator": "ge"}, {"budget": 0.44, "threshold": -2.8473193645477295, "operator": "ge"}, {"budget": 0.45, "threshold": -2.974519968032837, "operator": "ge"}, {"budget": 0.46, "threshold": -3.0839085578918457, "operator": "ge"}, {"budget": 0.47, "threshold": -3.213916063308716, "operator": "ge"}, {"budget": 0.48, "threshold": -3.3294801712036133, "operator": "ge"}, {"budget": 0.49, "threshold": -3.465546131134033, "operator": "ge"}, {"budget": 0.5, "threshold": -3.61759877204895, "operator": "ge"}, {"budget": 0.51, "threshold": -3.7520461082458496, "operator": "ge"}, {"budget": 0.52, "threshold": -3.8510866165161133, "operator": "ge"}, {"budget": 0.53, "threshold": -3.9660744667053223, "operator": "ge"}, {"budget": 0.54, "threshold": -4.10863733291626, "operator": "ge"}, {"budget": 0.55, "threshold": -4.198923110961914, "operator": "ge"}, {"budget": 0.56, "threshold": -4.324748516082764, "operator": "ge"}, {"budget": 0.57, "threshold": -4.439909934997559, "operator": "ge"}, {"budget": 0.58, "threshold": -4.545541763305664, "operator": "ge"}, {"budget": 0.59, "threshold": -4.628268241882324, "operator": "ge"}, {"budget": 0.6, "threshold": -4.74946928024292, "operator": "ge"}, {"budget": 0.61, "threshold": -4.83615779876709, "operator": "ge"}, {"budget": 0.62, "threshold": -4.9249653816223145, "operator": "ge"}, {"budget": 0.63, "threshold": -5.02262020111084, "operator": "ge"}, {"budget": 0.64, "threshold": -5.118165969848633, "operator": "ge"}, {"budget": 0.65, "threshold": -5.208711624145508, "operator": "ge"}, {"budget": 0.66, "threshold": -5.333867073059082, "operator": "ge"}, {"budget": 0.67, "threshold": -5.4173712730407715, "operator": "ge"}, {"budget": 0.68, "threshold": -5.5348124504089355, "operator": "ge"}, {"budget": 0.69, "threshold": -5.621229648590088, "operator": "ge"}, {"budget": 0.7, "threshold": -5.711812496185303, "operator": "ge"}, {"budget": 0.71, "threshold": -5.822430610656738, "operator": "ge"}, {"budget": 0.72, "threshold": -5.926287651062012, "operator": "ge"}, {"budget": 0.73, "threshold": -6.017222881317139, "operator": "ge"}, {"budget": 0.74, "threshold": -6.1182026863098145, "operator": "ge"}, {"budget": 0.75, "threshold": -6.221706390380859, "operator": "ge"}, {"budget": 0.76, "threshold": -6.319812297821045, "operator": "ge"}, {"budget": 0.77, "threshold": -6.4340620040893555, "operator": "ge"}, {"budget": 0.78, "threshold": -6.538553714752197, "operator": "ge"}, {"budget": 0.79, "threshold": -6.643221378326416, "operator": "ge"}, {"budget": 0.8, "threshold": -6.759832382202148, "operator": "ge"}, {"budget": 0.81, "threshold": -6.847866058349609, "operator": "ge"}, {"budget": 0.82, "threshold": -6.954311370849609, "operator": "ge"}, {"budget": 0.83, "threshold": -7.052628993988037, "operator": "ge"}, {"budget": 0.84, "threshold": -7.162949562072754, "operator": "ge"}, {"budget": 0.85, "threshold": -7.267045974731445, "operator": "ge"}, {"budget": 0.86, "threshold": -7.392378807067871, "operator": "ge"}, {"budget": 0.87, "threshold": -7.493752956390381, "operator": "ge"}, {"budget": 0.88, "threshold": -7.631547927856445, "operator": "ge"}, {"budget": 0.89, "threshold": -7.7515153884887695, "operator": "ge"}, {"budget": 0.9, "threshold": -7.873290538787842, "operator": "ge"}, {"budget": 0.91, "threshold": -7.991521835327148, "operator": "ge"}, {"budget": 0.92, "threshold": -8.150282859802246, "operator": "ge"}, {"budget": 0.93, "threshold": -8.316862106323242, "operator": "ge"}, {"budget": 0.94, "threshold": -8.478327751159668, "operator": "ge"}, {"budget": 0.95, "threshold": -8.631654739379883, "operator": "ge"}, {"budget": 0.96, "threshold": -8.872366905212402, "operator": "ge"}, {"budget": 0.97, "threshold": -9.089679718017578, "operator": "ge"}, {"budget": 0.98, "threshold": -9.4988431930542, "operator": "ge"}, {"budget": 0.99, "threshold": -10.06451416015625, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.00973932970496, 76.36780292179891, 76.76883414494414, 77.14122028072185, 77.47063878544829, 77.69979948438842, 77.94328272701232, 78.22973360068748, 78.4302492122601, 78.6021197364652, 78.8742480664566, 79.10340876539674, 79.40418218275566, 79.81953594958465, 80.1775995416786, 80.40676024061874, 80.69321111429389, 81.0655972500716, 81.20882268690919, 81.38069321111429, 81.81036952162704, 81.9965625895159, 82.19707820108852, 82.46920653107992, 82.64107705528502, 82.87023775422514, 83.21397880263535, 83.3715267831567, 83.51475221999426, 83.70094528788312, 83.90146089945574, 84.0876539673446, 84.35978229733601, 84.56029790890862, 84.74649097679747, 84.97565167573761, 85.29074763678028, 85.44829561730162, 85.57719851045546, 85.79203666571182, 86.04984245201948, 86.22171297622458, 86.37926095674592, 86.50816384989974, 86.6227441993698, 86.72300200515612, 86.83758235462618, 86.88054998567746, 86.92351761672873, 87.03809796619879, 87.12403322830134, 87.2099684904039, 87.3102262961902, 87.42480664566027, 87.45345173302779, 87.49641936407906, 87.6539673446004, 87.73990260670294, 87.82583786880551, 87.94041821827557, 87.98338584932684, 88.05499856774563, 88.12661128616442, 88.14093382984818, 88.1695789172157, 88.269836723002, 88.35577198510455, 88.39873961615584, 88.45602979089085, 88.58493268404469, 88.68519048983099, 88.82841592666857, 88.92867373245488, 88.97164136350615, 89.0575766256087, 89.15783443139502, 89.18647951876252, 89.24376969349757, 89.25809223718132, 89.31538241191636, 89.35835004296763, 89.34402749928387, 89.38699513033515, 89.41564021770266, 89.41564021770266, 89.51589802348897, 89.60183328559151, 89.63047837295903, 89.70209109137782, 89.73073617874535, 89.80234889716414, 89.80234889716414, 89.90260670295044, 89.97421942136923, 90.00286450873675, 89.95989687768548, 89.94557433400172, 89.95989687768548, 89.88828415926669, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 34, 62, 93, 124, 152, 174, 198, 223, 246, 267, 291, 310, 335, 365, 395, 419, 447, 477, 495, 514, 547, 568, 588, 615, 634, 655, 682, 699, 718, 738, 754, 768, 791, 812, 829, 848, 870, 886, 898, 916, 936, 952, 967, 980, 994, 1007, 1019, 1029, 1037, 1046, 1057, 1068, 1078, 1089, 1095, 1102, 1115, 1125, 1136, 1146, 1151, 1159, 1166, 1168, 1173, 1183, 1192, 1199, 1204, 1215, 1225, 1235, 1244, 1252, 1261, 1269, 1275, 1279, 1283, 1289, 1297, 1298, 1304, 1310, 1314, 1324, 1333, 1338, 1348, 1353, 1359, 1360, 1369, 1374, 1377, 1378, 1380, 1384, 1385, 1389], "harmed": [0, 3, 6, 9, 14, 19, 25, 32, 37, 46, 55, 60, 63, 67, 68, 73, 81, 89, 93, 101, 108, 111, 119, 125, 133, 140, 145, 148, 154, 163, 170, 172, 173, 177, 184, 188, 191, 191, 196, 199, 202, 204, 208, 212, 216, 222, 228, 232, 239, 244, 245, 250, 255, 258, 261, 265, 269, 271, 275, 280, 282, 284, 287, 289, 290, 293, 296, 299, 303, 304, 306, 309, 309, 311, 316, 319, 320, 324, 324, 327, 329, 334, 336, 339, 343, 347, 350, 353, 356, 361, 364, 365, 366, 368, 368, 369, 373, 376, 379, 385, 388], "exact_score": [75.56574047550845, 76.00973932970496, 76.36780292179891, 76.76883414494414, 77.14122028072185, 77.47063878544829, 77.69979948438842, 77.94328272701232, 78.22973360068748, 78.4302492122601, 78.6021197364652, 78.8742480664566, 79.10340876539674, 79.40418218275566, 79.81953594958465, 80.1775995416786, 80.40676024061874, 80.69321111429389, 81.0655972500716, 81.20882268690919, 81.38069321111429, 81.81036952162704, 81.9965625895159, 82.19707820108852, 82.46920653107992, 82.64107705528502, 82.87023775422514, 83.21397880263535, 83.3715267831567, 83.51475221999426, 83.70094528788312, 83.90146089945574, 84.0876539673446, 84.35978229733601, 84.56029790890862, 84.74649097679747, 84.97565167573761, 85.29074763678028, 85.44829561730162, 85.57719851045546, 85.79203666571182, 86.04984245201948, 86.22171297622458, 86.37926095674592, 86.50816384989974, 86.6227441993698, 86.72300200515612, 86.83758235462618, 86.88054998567746, 86.92351761672873, 87.03809796619879, 87.12403322830134, 87.2099684904039, 87.3102262961902, 87.42480664566027, 87.45345173302779, 87.49641936407906, 87.6539673446004, 87.73990260670294, 87.82583786880551, 87.94041821827557, 87.98338584932684, 88.05499856774563, 88.12661128616442, 88.14093382984818, 88.1695789172157, 88.269836723002, 88.35577198510455, 88.39873961615584, 88.45602979089085, 88.58493268404469, 88.68519048983099, 88.82841592666857, 88.92867373245488, 88.97164136350615, 89.0575766256087, 89.15783443139502, 89.18647951876252, 89.24376969349757, 89.25809223718132, 89.31538241191636, 89.35835004296763, 89.34402749928387, 89.38699513033515, 89.41564021770266, 89.41564021770266, 89.51589802348897, 89.60183328559151, 89.63047837295903, 89.70209109137782, 89.73073617874535, 89.80234889716414, 89.80234889716414, 89.90260670295044, 89.97421942136923, 90.00286450873675, 89.95989687768548, 89.94557433400172, 89.95989687768548, 89.88828415926669, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.55034527956283, "threshold_area100": 85.54934116299054, "threshold_area25": 79.34689200802062}, "test": {"score": [76.42247255506925, 76.81710554638732, 77.15433737533185, 77.65659754610031, 77.92207792207793, 78.31671091339601, 78.53196527229677, 78.86202195594461, 79.21360407548254, 79.49343474205352, 79.74456482743776, 80.06027122049221, 80.31857645117314, 80.55535624596398, 80.78496089545814, 81.17241874147952, 81.4594245533472, 81.73208007462152, 81.90428356174213, 82.15541364712635, 82.44241945899404, 82.6791992537849, 82.97338021094927, 83.07383224510296, 83.30343689459711, 83.54739183468466, 83.63349357824497, 83.85592308244242, 84.04965200545311, 84.17162947549687, 84.50168615914473, 84.72411566334218, 85.06134749228671, 85.32682786826433, 85.5492573724618, 85.8219128937361, 85.89366434670302, 86.08739326971371, 86.34569850039465, 86.50355169692186, 86.66858003874579, 86.89100954294324, 87.09191361125063, 87.32151826074478, 87.45067087608524, 87.60134892731578, 87.63722465379924, 87.71615125206286, 87.83812872210662, 87.93858075626032, 88.11795938867762, 88.19688598694124, 88.25428714931478, 88.33321374757838, 88.46954150821554, 88.48389179880894, 88.56999354236923, 88.64174499533615, 88.76372246537993, 88.7924230465667, 88.86417449953362, 88.95027624309392, 89.05790342254431, 89.10095429432447, 89.19423118318146, 89.27315778144508, 89.38796010619215, 89.43818612326899, 89.50993757623593, 89.56016359331277, 89.6175647556863, 89.73236708043338, 89.77541795221353, 89.86151969577384, 89.87586998636723, 89.87586998636723, 89.91174571285069, 89.96914687522423, 89.98349716581761, 90.03372318289445, 90.0839491999713, 90.11264978115807, 90.1772260888283, 90.21310181531177, 90.27050297768531, 90.32790414005885, 90.38530530243239, 90.39965559302576, 90.42118102891584, 90.47858219128938, 90.50010762717945, 90.52880820836621, 90.56468393484968, 90.57185908014638, 90.57903422544307, 90.58620937073975, 90.6364353878166, 90.67231111430006, 90.71536198608023, 90.70101169548683, 90.73688742197031], "calls": [0.0, 1.2125995551409916, 2.1525435890076774, 3.3005668364784384, 4.269211451531894, 5.460285570782808, 6.350003587572648, 7.232546459065796, 8.41644543302002, 9.399440338666858, 10.124130013632776, 11.099949773982923, 11.94661691899261, 12.951137260529524, 13.733228097868983, 14.70904785821913, 15.641816746789122, 16.4956590370955, 17.392552199182035, 18.289445361268566, 19.516395207002944, 20.68594389036378, 21.64023821482385, 22.40080361627323, 23.527301427853914, 24.574872641170984, 25.428714931477366, 26.404534691827507, 27.29425270861735, 28.37769964841788, 29.547248331778718, 30.75984788691971, 31.993972877950778, 33.07024467245462, 34.261318791705534, 35.072110210231756, 35.954653081724906, 36.80849537203129, 37.89194231183182, 38.961038961038966, 40.030135610246106, 40.98442993470618, 41.917198823276166, 43.136973523713856, 44.19171988232762, 45.232115950347996, 45.963980770610604, 46.918275095070676, 47.851043983640665, 48.82686374399082, 49.98923728205496, 51.12291023893234, 51.8117241874148, 52.73014278539141, 53.84946545167539, 54.473703092487625, 55.57867546817823, 56.604721245605226, 57.573365860658676, 58.3841572791849, 59.532180526655665, 60.371672526368656, 61.30444141493865, 62.24438544880534, 63.21303006385879, 64.23907584128578, 65.40862452464663, 66.33421826791992, 67.4607160795006, 68.32173351510367, 69.24732725837698, 70.33077419817751, 71.63665064217551, 72.48331778718519, 73.60981559876588, 74.82241515390686, 75.81976035014709, 76.95343330702447, 77.88620219559445, 78.86202195594461, 80.06744636578891, 80.8208366219416, 81.71772978402812, 82.52852120255434, 83.46129009112434, 84.49451101384803, 85.66405969720887, 86.52507713281193, 87.75202697854631, 88.64174499533615, 89.54581330271938, 90.3566047212456, 91.44722680634283, 92.57372461792352, 93.59259525005382, 94.39621152328334, 95.59446078783095, 96.59180598407117, 98.01248475281625, 99.13180741910024, 100.0], "rescued": [0, 65, 118, 194, 237, 300, 340, 391, 446, 497, 537, 588, 632, 678, 717, 781, 831, 874, 912, 957, 1012, 1058, 1108, 1134, 1179, 1228, 1256, 1296, 1332, 1357, 1413, 1457, 1510, 1559, 1605, 1646, 1670, 1704, 1747, 1784, 1811, 1846, 1880, 1919, 1946, 1975, 1987, 2007, 2030, 2052, 2083, 2107, 2120, 2137, 2161, 2169, 2185, 2202, 2223, 2234, 2251, 2264, 2280, 2293, 2310, 2327, 2347, 2359, 2372, 2385, 2399, 2419, 2430, 2448, 2458, 2468, 2478, 2491, 2498, 2511, 2526, 2533, 2544, 2553, 2567, 2578, 2593, 2600, 2612, 2623, 2630, 2638, 2645, 2655, 2663, 2666, 2678, 2685, 2697, 2699, 2706], "harmed": [0, 10, 16, 22, 28, 36, 46, 51, 57, 69, 74, 81, 89, 102, 109, 119, 129, 134, 148, 158, 173, 186, 195, 207, 220, 235, 251, 260, 269, 277, 287, 300, 306, 318, 333, 336, 350, 357, 364, 379, 383, 387, 393, 400, 409, 417, 424, 433, 439, 447, 453, 466, 471, 477, 482, 488, 492, 499, 503, 510, 517, 518, 519, 526, 530, 536, 540, 545, 548, 554, 560, 564, 569, 575, 583, 593, 598, 603, 608, 614, 622, 625, 627, 631, 637, 640, 647, 652, 661, 664, 668, 672, 674, 683, 690, 692, 697, 699, 705, 709, 711], "exact_score": [76.42247255506925, 76.7381789481237, 77.10411135825501, 77.53462007605654, 77.850326469111, 78.18755829805554, 78.46021381932985, 78.76156992179092, 79.07727631484538, 79.37863241730645, 79.74456482743776, 80.03157063930544, 80.32575159646983, 80.55535624596398, 80.87106263901845, 81.21546961325967, 81.58140202339098, 81.85405754466528, 82.09801248475281, 82.32044198895028, 82.55004663844443, 82.84422759560881, 83.00208079213604, 83.2245102963335, 83.42541436464089, 83.61196814235488, 83.74829590299204, 84.00660113367296, 84.12857860371673, 84.36535839850757, 84.59496304800173, 84.75281624452894, 85.06134749228671, 85.30530243237426, 85.49903135538494, 85.8219128937361, 85.9080146372964, 86.16631986797732, 86.38874937217479, 86.51790198751526, 86.66140489344909, 86.89100954294324, 87.09908875654733, 87.32151826074478, 87.41479514960177, 87.54394776494225, 87.63722465379924, 87.73050154265624, 87.84530386740332, 87.96010619215039, 88.11795938867762, 88.18253569634786, 88.26146229461148, 88.34756403817178, 88.48389179880894, 88.53411781588576, 88.6058692688527, 88.70632130300639, 88.77807275597331, 88.85699935423692, 88.92157566190716, 89.02920284135753, 89.07942885843438, 89.16553060199469, 89.2659826361484, 89.3520843797087, 89.40231039678554, 89.48123699504916, 89.54581330271938, 89.60321446509292, 89.69649135394992, 89.74671737102676, 89.79694338810361, 89.84716940518047, 89.8902202769606, 89.87586998636723, 89.91174571285069, 89.96914687522423, 89.99067231111431, 90.03372318289445, 90.0839491999713, 90.11264978115807, 90.1772260888283, 90.24180239649853, 90.29920355887207, 90.34225443065222, 90.39965559302576, 90.44270646480592, 90.44270646480592, 90.47858219128938, 90.51445791777284, 90.54315849895961, 90.57185908014638, 90.57903422544307, 90.57903422544307, 90.61490995192652, 90.66513596900337, 90.70101169548683, 90.71536198608023, 90.70818684078353, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.4804251487499, "threshold_area100": 86.456590370955, "threshold_area25": 80.33220922723686}, "selected_budget": 95, "selected": {"val": {"score": 90.00286450873675, "calls": 94.98710971068462, "rescued": 1377, "harmed": 369}, "test": {"score": 90.58620937073975, "calls": 94.39621152328334, "rescued": 2666, "harmed": 692}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.83099398453164, "calls": 98.99742194213692}, "test": {"score": 90.57185908014638, "calls": 98.82327617134247}}, "old_normalized_interior": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692}, "test": {"score": 90.70818684078353, "calls": 98.89502762430939}}, "old_normalized_endpoints": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692}, "test": {"score": 90.70818684078353, "calls": 98.89502762430939}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 75.98109424233743, 76.39644800916643, 76.66857633915784, 76.89773703809797, 77.21283299914064, 77.58521913491836, 77.85734746490976, 78.22973360068748, 78.57347464909768, 78.90289315382411, 79.11773130908048, 79.37553709538814, 79.67631051274707, 79.9054712116872, 80.12030936694356, 80.42108278430248, 80.69321111429389, 80.93669435691778, 81.19450014322544, 81.43798338584932, 81.66714408478946, 81.96791750214838, 82.08249785161844, 82.29733600687481, 82.51217416213119, 82.7126897737038, 82.97049556001146, 83.07075336579777, 83.21397880263535, 83.38584932684044, 83.55771985104555, 83.8441707247207, 84.20223431681467, 84.37410484101976, 84.56029790890862, 84.86107132626755, 85.06158693784016, 85.30507018046404, 85.47694070466915, 85.70610140360928, 85.96390718991692, 86.09281008307075, 86.16442280148954, 86.3506158693784, 86.50816384989974, 86.59409911200228, 86.70867946147236, 86.78029217989115, 86.79461472357491, 86.89487252936121, 86.99513033514752, 87.08106559725006, 87.19564594672013, 87.19564594672013, 87.32454883987396, 87.38183901460899, 87.45345173302779, 87.49641936407906, 87.56803208249785, 87.66828988828415, 87.7255800630192, 87.78287023775422, 87.84016041248925, 87.88312804354052, 87.9977083930106, 88.05499856774563, 88.15525637353194, 88.15525637353194, 88.269836723002, 88.38441707247206, 88.47035233457461, 88.48467487825837, 88.52764250930964, 88.58493268404469, 88.64222285877972, 88.69951303351475, 88.79977083930106, 88.85706101403609, 88.84273847035233, 88.92867373245488, 89.02893153824118, 89.04325408192494, 89.11486680034373, 89.18647951876252, 89.28673732454884, 89.34402749928387, 89.50157547980521, 89.48725293612145, 89.58751074190776, 89.63047837295903, 89.6448009166428, 89.68776854769406, 89.73073617874535, 89.77370380979661, 89.8166714408479, 89.8166714408479, 89.84531652821542, 89.87396161558291, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 75.98109424233743, 76.39644800916643, 76.66857633915784, 76.89773703809797, 77.21283299914064, 77.58521913491836, 77.85734746490976, 78.22973360068748, 78.57347464909768, 78.90289315382411, 79.11773130908048, 79.37553709538814, 79.67631051274707, 79.9054712116872, 80.12030936694356, 80.42108278430248, 80.69321111429389, 80.93669435691778, 81.19450014322544, 81.43798338584932, 81.66714408478946, 81.96791750214838, 82.08249785161844, 82.29733600687481, 82.51217416213119, 82.7126897737038, 82.97049556001146, 83.07075336579777, 83.21397880263535, 83.38584932684044, 83.55771985104555, 83.8441707247207, 84.20223431681467, 84.37410484101976, 84.56029790890862, 84.86107132626755, 85.06158693784016, 85.30507018046404, 85.47694070466915, 85.70610140360928, 85.96390718991692, 86.09281008307075, 86.16442280148954, 86.3506158693784, 86.50816384989974, 86.59409911200228, 86.70867946147236, 86.78029217989115, 86.79461472357491, 86.89487252936121, 86.99513033514752, 87.08106559725006, 87.19564594672013, 87.19564594672013, 87.32454883987396, 87.38183901460899, 87.45345173302779, 87.49641936407906, 87.56803208249785, 87.66828988828415, 87.7255800630192, 87.78287023775422, 87.84016041248925, 87.88312804354052, 87.9977083930106, 88.05499856774563, 88.15525637353194, 88.15525637353194, 88.269836723002, 88.38441707247206, 88.47035233457461, 88.48467487825837, 88.52764250930964, 88.58493268404469, 88.64222285877972, 88.69951303351475, 88.79977083930106, 88.85706101403609, 88.84273847035233, 88.92867373245488, 89.02893153824118, 89.04325408192494, 89.11486680034373, 89.18647951876252, 89.28673732454884, 89.34402749928387, 89.50157547980521, 89.48725293612145, 89.58751074190776, 89.63047837295903, 89.6448009166428, 89.68776854769406, 89.73073617874535, 89.77370380979661, 89.8166714408479, 89.8166714408479, 89.84531652821542, 89.87396161558291, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.34259524491549, "exact_area100": 85.39451120411852}, "test": {"score": [76.42247255506925, 76.82428069168401, 77.21891368300209, 77.51309464016647, 77.77139987084739, 78.16603286216545, 78.50326469110999, 78.80462079357106, 79.13467747721891, 79.36428212671306, 79.67281337447083, 79.8952428786683, 80.12484752816245, 80.36162732295328, 80.71320944249122, 80.92846380139198, 81.05761641673243, 81.27287077563321, 81.51682571572074, 81.71772978402812, 81.9258089976322, 82.17693908301644, 82.53569634785106, 82.72225012556504, 83.03795651861951, 83.23886058692689, 83.46129009112435, 83.71959532180527, 83.99942598837626, 84.27208150965058, 84.48016072325466, 84.71694051804549, 84.83174284279256, 85.14744923584703, 85.38422903063787, 85.57078280835188, 85.87213891081295, 86.05869268852695, 86.38874937217479, 86.61117887637225, 86.76903207289948, 86.89818468823994, 87.05603788476716, 87.17801535481094, 87.43632058549186, 87.56547320083232, 87.62287436320587, 87.6802755255794, 87.7735524144364, 87.86682930329339, 87.99598191863386, 88.11795938867762, 88.20406113223794, 88.34756403817178, 88.45519121762216, 88.5125923799957, 88.56281839707253, 88.67762072181962, 88.74219702948984, 88.82112362775347, 88.90722537131377, 89.00050226017078, 89.02920284135753, 89.0722537131377, 89.11530458491785, 89.16553060199469, 89.22293176436823, 89.30185836263185, 89.3520843797087, 89.4238358326756, 89.531463012126, 89.59603931979623, 89.70366649924661, 89.73954222573008, 89.74671737102676, 89.78259309751022, 89.83281911458707, 89.93327114874077, 89.95479658463084, 89.94762143933414, 90.03372318289445, 90.04807347348785, 90.0982994905647, 90.14135036234484, 90.20592667001507, 90.24180239649853, 90.277678122982, 90.29202841357538, 90.32072899476215, 90.32072899476215, 90.34942957594892, 90.37095501183899, 90.41400588361914, 90.47858219128938, 90.48575733658608, 90.48575733658608, 90.54315849895961, 90.6364353878166, 90.66513596900337, 90.70818684078353, 90.73688742197031], "calls": [0.0, 1.140848102174069, 2.339097366721676, 3.192939657028055, 4.018081366147665, 5.065652579464734, 5.962545741551266, 7.031642390758412, 8.079213604075482, 9.011982492645476, 10.095429432446007, 10.855994833895387, 11.717012269498458, 12.578029705101528, 13.63277606371529, 14.52249408050513, 15.261534046064432, 16.179952644041045, 16.962043481380498, 17.844586352873645, 18.698428643180023, 20.068881394848244, 21.231254932912393, 22.164023821482385, 23.154193872425914, 23.929109564468682, 24.983855923082444, 26.11035373466313, 27.157924947980195, 28.176795580110497, 29.382219989954795, 30.594819545095785, 31.59216474133601, 32.65408624524647, 33.924086962760995, 34.87120614192437, 36.019229389395136, 37.04527516682213, 38.3655019014135, 39.31262108057688, 40.30996627681711, 41.27861089187056, 42.47686015641817, 43.538781660328624, 44.60787830953577, 45.51912176221569, 46.49494152256583, 47.183755471048286, 48.15240008610174, 49.14974528234197, 50.025113008538426, 50.91483102532827, 51.833249623304866, 52.8305948195451, 53.90686661404893, 54.91138695558585, 56.18138767310038, 57.071105689890224, 58.26935495443782, 59.30257587716151, 60.31427136399512, 61.37619286790558, 62.2802611752888, 63.19867977326541, 63.93771973882472, 65.04269211451532, 66.3485685585133, 67.58269354954439, 68.5728636004879, 69.62043481380498, 70.63213030063858, 71.78732869340604, 72.9497022314702, 73.91834684652365, 74.77936428212672, 75.89151180311401, 76.89603214465093, 77.78575016144077, 78.74004448590084, 79.45038387027338, 80.31857645117314, 81.20829446796297, 82.1195379206429, 83.10253282628973, 84.09270287723326, 85.19767525292387, 86.34569850039463, 87.17084020951424, 88.18253569634786, 88.92875080720385, 89.78976824280693, 90.65078567840999, 91.51180311401306, 92.72440266915405, 93.72174786539428, 94.58276530099734, 95.94604290736888, 96.9433881036091, 97.83310612039894, 98.89502762430939, 100.0], "exact_score": [76.42247255506925, 76.78122981990386, 77.10411135825501, 77.44134318719954, 77.77857501614407, 78.130157135682, 78.52479012700007, 78.79027050297769, 79.11315204132883, 79.35710698141638, 79.65846308387745, 79.90959316926168, 80.1607232546459, 80.49795508359044, 80.78496089545814, 81.03609098084237, 81.23699504914975, 81.51682571572074, 81.77513094640166, 81.97603501470905, 82.16976393771974, 82.46394489488412, 82.6791992537849, 82.98773050154266, 83.25321087752027, 83.47564038171772, 83.6908947406185, 83.94202482600272, 84.25055607376049, 84.41558441558442, 84.6236636291885, 84.75999138982566, 85.00394632991318, 85.24072612470403, 85.39140417593455, 85.58513309894525, 85.86496376551625, 86.04434239793356, 86.2954724833178, 86.54660256870201, 86.70445576522926, 86.88383439764655, 86.96276099591017, 87.10626390184402, 87.3071679701514, 87.4937217478654, 87.61569921790917, 87.6802755255794, 87.7735524144364, 87.85247901270002, 87.99598191863386, 88.132309679271, 88.22558656812801, 88.38343976465524, 88.44801607232546, 88.52694267058908, 88.55564325177585, 88.6847958671163, 88.72067159359977, 88.81394848245677, 88.85699935423692, 88.96462653368731, 89.04355313195093, 89.0722537131377, 89.11530458491785, 89.158355456698, 89.20140632847816, 89.28033292674176, 89.3520843797087, 89.40231039678554, 89.46688670445577, 89.52428786682931, 89.6103896103896, 89.70366649924661, 89.74671737102676, 89.73954222573008, 89.78259309751022, 89.83281911458707, 89.93327114874077, 89.95479658463084, 89.99067231111431, 90.05524861878453, 90.07677405467462, 90.12700007175145, 90.19157637942169, 90.22745210590514, 90.28485326827868, 90.29202841357538, 90.31355384946545, 90.32790414005885, 90.34942957594892, 90.39248044772907, 90.46423190069599, 90.48575733658608, 90.49293248188276, 90.52163306306953, 90.54315849895961, 90.64361053311329, 90.67948625959676, 90.70818684078353, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 80.10475712133172, "exact_area100": 86.37667333029692}}, "old_normalized_endpoint_budget": 99}, {"seed": 2028, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 1.0, "validation_area100": 0.8550838999100767, "training_seconds": 34.595371076022275, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7146629438692113, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6473636127810821, "operator": "ge"}, {"budget": 0.03, "threshold": 0.6000752155150472, "operator": "ge"}, {"budget": 0.04, "threshold": 0.554152482699981, "operator": "ge"}, {"budget": 0.05, "threshold": 0.512953410382494, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4745468400147805, "operator": "ge"}, {"budget": 0.07, "threshold": 0.44242522805403006, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4121306741520423, "operator": "ge"}, {"budget": 0.09, "threshold": 0.3745841317254203, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3471593184257441, "operator": "ge"}, {"budget": 0.11, "threshold": 0.31704070869444295, "operator": "ge"}, {"budget": 0.12, "threshold": 0.2827820957485095, "operator": "ge"}, {"budget": 0.13, "threshold": 0.25012102925459206, "operator": "ge"}, {"budget": 0.14, "threshold": 0.21843962001724745, "operator": "ge"}, {"budget": 0.15, "threshold": 0.18969019043305707, "operator": "ge"}, {"budget": 0.16, "threshold": 0.15386533022804463, "operator": "ge"}, {"budget": 0.17, "threshold": 0.12623384256213094, "operator": "ge"}, {"budget": 0.18, "threshold": 0.09318441543431034, "operator": "ge"}, {"budget": 0.19, "threshold": 0.05387503908527966, "operator": "ge"}, {"budget": 0.2, "threshold": 0.01856577697401358, "operator": "ge"}, {"budget": 0.21, "threshold": -0.015142783636614001, "operator": "ge"}, {"budget": 0.22, "threshold": -0.05581022551505462, "operator": "ge"}, {"budget": 0.23, "threshold": -0.09244127332588915, "operator": "ge"}, {"budget": 0.24, "threshold": -0.13582547533898875, "operator": "ge"}, {"budget": 0.25, "threshold": -0.1655126950221416, "operator": "ge"}, {"budget": 0.26, "threshold": -0.202005078958578, "operator": "ge"}, {"budget": 0.27, "threshold": -0.24717632775434406, "operator": "ge"}, {"budget": 0.28, "threshold": -0.28341315767253233, "operator": "ge"}, {"budget": 0.29, "threshold": -0.32264778793770893, "operator": "ge"}, {"budget": 0.3, "threshold": -0.3540770275110279, "operator": "ge"}, {"budget": 0.31, "threshold": -0.3919975598875691, "operator": "ge"}, {"budget": 0.32, "threshold": -0.4259781035335662, "operator": "ge"}, {"budget": 0.33, "threshold": -0.4641570875580167, "operator": "ge"}, {"budget": 0.34, "threshold": -0.49421325755110157, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5283116597370189, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5613887968637142, "operator": "ge"}, {"budget": 0.37, "threshold": -0.5955439616550329, "operator": "ge"}, {"budget": 0.38, "threshold": -0.6262762036282187, "operator": "ge"}, {"budget": 0.39, "threshold": -0.6630266193597476, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6953128392070015, "operator": "ge"}, {"budget": 0.41, "threshold": -0.7281937960816137, "operator": "ge"}, {"budget": 0.42, "threshold": -0.7598180893598345, "operator": "ge"}, {"budget": 0.43, "threshold": -0.7863281180684712, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8098548206570562, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8331864820125454, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8499361195255396, "operator": "ge"}, {"budget": 0.47, "threshold": -0.868416961210583, "operator": "ge"}, {"budget": 0.48, "threshold": -0.8829708414931284, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8943305722669709, "operator": "ge"}, {"budget": 0.5, "threshold": -0.903916330026017, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9147613850227978, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9243523563748812, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9304870559309788, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9376608918604943, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9444946996676223, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9494541493329126, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9540244636341974, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9574249598918577, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9607434718751963, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9635672463128024, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9665007034546325, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9688907256327046, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9710252996278276, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9730182814311843, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9745616682011814, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9759986736965831, "operator": "ge"}, {"budget": 0.67, "threshold": -0.977473014370962, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9786702043539479, "operator": "ge"}, {"budget": 0.69, "threshold": -0.979997716152329, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9813060361128151, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9822852572193531, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9834850202533896, "operator": "ge"}, {"budget": 0.73, "threshold": -0.984500796639029, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9855619044965277, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9863737269110975, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9872418015790377, "operator": "ge"}, {"budget": 0.77, "threshold": -0.987977388652244, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9886871322099376, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9894085588873558, "operator": "ge"}, {"budget": 0.8, "threshold": -0.990202816292211, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9908683807488862, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9913788181466966, "operator": "ge"}, {"budget": 0.83, "threshold": -0.991952632929993, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9924500729815008, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9930439497466403, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9935733031176853, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9939573931825009, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9943152976706442, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9947482165548528, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9951828678764921, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9955683428944396, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9959330402120289, "operator": "ge"}, {"budget": 0.93, "threshold": -0.996259290904795, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9966525947894617, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9969763421773609, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9972805354266595, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9976304842486612, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9980580462347883, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9985302423544025, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.96677169865367, 76.39644800916643, 76.65425379547408, 76.94070466914924, 77.25580063019191, 77.6711543970209, 78.05786307648238, 78.32999140647378, 78.58779719278144, 78.8742480664566, 79.17502148381553, 79.51876253222572, 79.77656831853336, 80.02005156115726, 80.23488971641363, 80.56430822114007, 80.75050128902893, 80.99398453165281, 81.19450014322544, 81.49527356058435, 81.81036952162704, 81.92494987109711, 82.168433113721, 82.35462618160985, 82.62675451160125, 82.85591521054138, 83.05643082211401, 83.31423660842165, 83.45746204525923, 83.60068748209682, 83.70094528788312, 83.93010598682325, 84.14494414207962, 84.35978229733601, 84.60326553995989, 84.83242623890003, 85.16184474362647, 85.40532798625036, 85.70610140360928, 85.82068175307934, 86.02119736465195, 86.12145517043827, 86.23603551990833, 86.40790604411343, 86.47951876253222, 86.53680893726725, 86.63706674305357, 86.66571183042107, 86.79461472357491, 86.99513033514752, 87.05242050988255, 87.12403322830134, 87.22429103408766, 87.32454883987396, 87.35319392724148, 87.45345173302779, 87.6539673446004, 87.76854769407046, 87.85448295617302, 87.9260956745918, 88.01203093669436, 88.09796619879691, 88.18390146089945, 88.269836723002, 88.35577198510455, 88.31280435405327, 88.38441707247206, 88.47035233457461, 88.57061014036093, 88.6135777714122, 88.75680320824978, 88.79977083930106, 88.82841592666857, 88.87138355771985, 88.8857061014036, 88.91435118877112, 89.02893153824118, 89.08622171297623, 89.11486680034373, 89.18647951876252, 89.21512460613005, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.37267258665138, 89.4013176740189, 89.48725293612145, 89.50157547980521, 89.5445431108565, 89.61615582927529, 89.68776854769406, 89.73073617874535, 89.78802635348038, 89.8166714408479, 89.80234889716414, 89.84531652821542, 89.85963907189917, 89.9169292466342, 89.9169292466342, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 63, 89, 117, 141, 172, 203, 228, 252, 277, 302, 332, 355, 383, 405, 433, 451, 473, 495, 521, 548, 567, 590, 610, 631, 652, 672, 695, 713, 726, 739, 757, 777, 796, 815, 835, 858, 878, 904, 922, 940, 949, 962, 979, 990, 1002, 1012, 1024, 1036, 1050, 1057, 1067, 1078, 1088, 1095, 1107, 1122, 1132, 1140, 1147, 1156, 1163, 1172, 1180, 1186, 1191, 1200, 1210, 1221, 1228, 1241, 1249, 1254, 1259, 1263, 1267, 1277, 1283, 1288, 1296, 1300, 1304, 1308, 1312, 1317, 1323, 1330, 1336, 1341, 1347, 1354, 1358, 1364, 1368, 1369, 1374, 1379, 1384, 1386, 1389], "harmed": [0, 3, 5, 13, 21, 23, 25, 29, 35, 41, 46, 50, 56, 61, 72, 79, 84, 89, 94, 102, 107, 112, 123, 129, 136, 138, 143, 149, 154, 162, 165, 171, 173, 178, 182, 184, 188, 188, 191, 196, 206, 210, 212, 217, 222, 228, 236, 239, 249, 252, 252, 255, 260, 264, 267, 272, 277, 278, 280, 282, 284, 287, 288, 291, 293, 293, 301, 305, 309, 313, 317, 320, 325, 328, 330, 333, 335, 337, 339, 342, 345, 347, 350, 351, 351, 353, 357, 358, 363, 365, 366, 368, 369, 371, 373, 375, 377, 381, 382, 384, 388], "exact_score": [75.56574047550845, 75.96677169865367, 76.39644800916643, 76.65425379547408, 76.94070466914924, 77.25580063019191, 77.6711543970209, 78.05786307648238, 78.32999140647378, 78.58779719278144, 78.8742480664566, 79.17502148381553, 79.51876253222572, 79.77656831853336, 80.02005156115726, 80.23488971641363, 80.56430822114007, 80.75050128902893, 80.99398453165281, 81.19450014322544, 81.49527356058435, 81.81036952162704, 81.92494987109711, 82.168433113721, 82.35462618160985, 82.62675451160125, 82.85591521054138, 83.05643082211401, 83.31423660842165, 83.45746204525923, 83.60068748209682, 83.70094528788312, 83.93010598682325, 84.14494414207962, 84.35978229733601, 84.60326553995989, 84.83242623890003, 85.16184474362647, 85.40532798625036, 85.70610140360928, 85.82068175307934, 86.02119736465195, 86.12145517043827, 86.23603551990833, 86.40790604411343, 86.47951876253222, 86.53680893726725, 86.63706674305357, 86.66571183042107, 86.79461472357491, 86.99513033514752, 87.05242050988255, 87.12403322830134, 87.22429103408766, 87.32454883987396, 87.35319392724148, 87.45345173302779, 87.6539673446004, 87.76854769407046, 87.85448295617302, 87.9260956745918, 88.01203093669436, 88.09796619879691, 88.18390146089945, 88.269836723002, 88.35577198510455, 88.31280435405327, 88.38441707247206, 88.47035233457461, 88.57061014036093, 88.6135777714122, 88.75680320824978, 88.79977083930106, 88.82841592666857, 88.87138355771985, 88.8857061014036, 88.91435118877112, 89.02893153824118, 89.08622171297623, 89.11486680034373, 89.18647951876252, 89.21512460613005, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.37267258665138, 89.4013176740189, 89.48725293612145, 89.50157547980521, 89.5445431108565, 89.61615582927529, 89.68776854769406, 89.73073617874535, 89.78802635348038, 89.8166714408479, 89.80234889716414, 89.84531652821542, 89.85963907189917, 89.9169292466342, 89.9169292466342, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.50838999100768, "threshold_area100": 85.50737610999713, "threshold_area25": 79.39129189344027}, "test": {"score": [76.42247255506925, 76.90320728994763, 77.18303795651862, 77.52026978546316, 77.90055248618785, 78.18038315275885, 78.46021381932985, 78.74004448590084, 78.98399942598839, 79.29970581904284, 79.622587357394, 79.86654229748153, 80.15354810934922, 80.31140130587644, 80.56253139126068, 80.84953720312836, 81.09349214321591, 81.25852048503982, 81.49530027983067, 81.71055463873144, 81.98321016000574, 82.19128937360982, 82.45676974958744, 82.72942527086174, 83.00208079213604, 83.16710913395997, 83.39671378345412, 83.65501901413504, 83.92049939011265, 84.19315491138696, 84.31513238143073, 84.6954150821554, 84.8030422616058, 85.19050010762719, 85.42010475712134, 85.71428571428572, 85.9151897825931, 86.14479443208725, 86.36004879098802, 86.55377771399871, 86.71163091052594, 86.89818468823994, 87.10626390184402, 87.25694195307454, 87.41479514960177, 87.5798234914257, 87.76637726913971, 87.93140561096362, 88.01033220922724, 88.07490851689747, 88.12513453397432, 88.18253569634786, 88.31168831168831, 88.39061490995192, 88.44084092702877, 88.46954150821554, 88.55564325177585, 88.67044557652292, 88.72784673889646, 88.83547391834685, 88.89287508072039, 89.00050226017078, 89.05072827724761, 89.17988089258809, 89.21575661907154, 89.28750807203846, 89.30903350792853, 89.36643467030207, 89.43818612326899, 89.51711272153261, 89.63191504627969, 89.75389251632346, 89.85434455047715, 89.91892085814737, 89.92609600344407, 89.96914687522423, 90.01219774700438, 90.13417521704815, 90.18440123412499, 90.1700509435316, 90.24897754179523, 90.277678122982, 90.37095501183899, 90.41400588361914, 90.45705675539931, 90.47858219128938, 90.49293248188276, 90.52880820836621, 90.54315849895961, 90.57903422544307, 90.57903422544307, 90.60055966133315, 90.62926024251992, 90.64361053311329, 90.69383655019014, 90.70818684078353, 90.71536198608023, 90.70818684078353, 90.70101169548683, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.2341249910310683, 2.367797947908445, 3.3292674176652075, 4.348138049795509, 5.359833536629117, 6.192150391045419, 7.124919279615412, 7.835258663987946, 8.796728133744708, 9.621869842864319, 10.38961038961039, 11.401305876443999, 12.097294970223146, 13.080289875869985, 13.991533328549904, 15.060629977757051, 15.878596541579967, 16.818540575446654, 17.75848460931334, 18.734304369663484, 19.710124130013632, 20.922723685154622, 21.877018009614694, 22.981990385305302, 23.656454043194376, 24.682499820621366, 25.87357393987228, 27.0574729138265, 28.262897323670806, 29.13826504986726, 30.28628829733802, 31.226232331204706, 32.65408624524647, 33.744708330343684, 34.86403099662768, 35.954653081724906, 36.930472842075055, 38.02109492717228, 39.19781875582981, 40.195163952070025, 41.27143574657387, 42.3548826863744, 43.2804764296477, 44.33522278826146, 45.28951711272153, 46.172059984214684, 47.23398148812513, 48.25285212025543, 49.042118102891585, 49.781158068450885, 50.64217550405395, 51.675396426777645, 52.35703522996341, 53.462007605654016, 54.581330271938015, 55.55715003228815, 56.697998134462225, 57.58771615125207, 58.549185621008824, 59.467604218985436, 60.52235057759919, 61.47664490205926, 62.52421611537634, 63.50721102102318, 64.42562961899978, 65.2435961828227, 66.11178876372247, 67.08043337877592, 67.92710052378561, 68.91009542943245, 69.7711128650355, 70.96218698428643, 71.93800674463657, 73.05732941092057, 74.0474994618641, 75.03766951280764, 76.00631412786109, 77.03235990528809, 78.17320800746215, 79.46473416086675, 80.69168400660114, 81.60292745928105, 82.70789983497166, 83.81287221066228, 85.02547176580326, 86.11609385090048, 86.99863672239363, 87.90270502977685, 89.05072827724761, 90.04807347348783, 90.94496663557437, 92.04993901126498, 92.99705819042836, 94.03027911315203, 94.99174858290881, 95.98909377914903, 96.99361412068595, 98.09141135107986, 99.12463227380354, 100.0], "rescued": [0, 73, 121, 175, 235, 280, 321, 371, 409, 466, 514, 555, 607, 635, 684, 732, 777, 810, 855, 900, 946, 987, 1037, 1085, 1133, 1168, 1207, 1259, 1308, 1357, 1386, 1445, 1472, 1537, 1579, 1626, 1664, 1705, 1750, 1790, 1823, 1856, 1891, 1915, 1944, 1974, 2005, 2036, 2054, 2070, 2082, 2099, 2120, 2136, 2153, 2170, 2187, 2213, 2224, 2244, 2257, 2277, 2290, 2315, 2329, 2343, 2356, 2368, 2381, 2395, 2417, 2435, 2452, 2465, 2472, 2480, 2492, 2509, 2521, 2531, 2546, 2557, 2573, 2586, 2597, 2607, 2614, 2622, 2628, 2637, 2641, 2649, 2660, 2665, 2674, 2680, 2686, 2690, 2696, 2702, 2706], "harmed": [0, 6, 15, 22, 29, 35, 37, 48, 52, 65, 68, 75, 87, 93, 107, 115, 126, 136, 148, 163, 171, 183, 196, 206, 216, 228, 235, 251, 263, 274, 286, 292, 304, 315, 325, 331, 341, 350, 365, 378, 389, 396, 402, 405, 412, 419, 424, 432, 439, 446, 451, 460, 463, 468, 478, 491, 496, 506, 509, 514, 519, 524, 530, 537, 546, 550, 560, 564, 567, 570, 576, 577, 580, 584, 590, 592, 598, 598, 603, 615, 619, 626, 629, 636, 641, 648, 653, 656, 660, 664, 668, 673, 680, 683, 685, 689, 694, 699, 706, 710, 711], "exact_score": [76.42247255506925, 76.81710554638732, 77.1112865035517, 77.41981775130947, 77.76422472555069, 78.07275597330846, 78.409987802253, 78.69699361412069, 79.01270000717516, 79.37145727200976, 79.73738968214107, 80.06027122049221, 80.27552557939298, 80.56970653655738, 80.84236205783168, 81.07914185262251, 81.30874650211666, 81.54552629690751, 81.78948123699506, 82.0262610317859, 82.27021597187343, 82.4782951854775, 82.76530099734521, 83.00925593743274, 83.23886058692689, 83.45411494582765, 83.67654445002512, 83.90614909951927, 84.11422831312333, 84.27208150965058, 84.6164884838918, 84.78151682571571, 85.00394632991318, 85.27660185118748, 85.43445504771472, 85.72863600487909, 85.92236492788979, 86.16631986797732, 86.35287364569133, 86.53942742340533, 86.68293032933917, 86.87665925234987, 87.02016215828371, 87.22824137188778, 87.38609456841502, 87.52959747434885, 87.75202697854631, 87.88835473918347, 88.00315706393054, 88.07490851689747, 88.1394848245677, 88.22558656812801, 88.34038889287508, 88.39779005524862, 88.44801607232546, 88.5197675252924, 88.59151897825932, 88.6847958671163, 88.7924230465667, 88.87134964483032, 88.92157566190716, 89.05790342254431, 89.12965487551124, 89.22293176436823, 89.23010690966493, 89.30903350792853, 89.36643467030207, 89.4238358326756, 89.52428786682931, 89.64626533687307, 89.76106766162016, 89.85434455047715, 89.91892085814737, 89.92609600344407, 89.96197172992754, 90.01937289230108, 90.13417521704815, 90.18440123412499, 90.18440123412499, 90.22745210590514, 90.25615268709191, 90.32072899476215, 90.37813015713569, 90.42118102891584, 90.46423190069599, 90.47858219128938, 90.49293248188276, 90.52880820836621, 90.54315849895961, 90.57903422544307, 90.57903422544307, 90.59338451603645, 90.6364353878166, 90.64361053311329, 90.69383655019014, 90.70818684078353, 90.71536198608023, 90.70818684078353, 90.70101169548683, 90.69383655019014, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.48142442815157, "threshold_area100": 86.41841859797661, "threshold_area25": 80.08610174356032}, "selected_budget": 98, "selected": {"val": {"score": 89.9169292466342, "calls": 97.99484388427385, "rescued": 1384, "harmed": 382}, "test": {"score": 90.70101169548683, "calls": 98.09141135107986, "rescued": 2696, "harmed": 706}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.37935662269592285, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3130576014518738, "operator": "ge"}, {"budget": 0.03, "threshold": 0.28283005952835083, "operator": "ge"}, {"budget": 0.04, "threshold": 0.25471755862236023, "operator": "ge"}, {"budget": 0.05, "threshold": 0.23189586400985718, "operator": "ge"}, {"budget": 0.06, "threshold": 0.20653560757637024, "operator": "ge"}, {"budget": 0.07, "threshold": 0.18848156929016113, "operator": "ge"}, {"budget": 0.08, "threshold": 0.16791997849941254, "operator": "ge"}, {"budget": 0.09, "threshold": 0.15060698986053467, "operator": "ge"}, {"budget": 0.1, "threshold": 0.13530071079730988, "operator": "ge"}, {"budget": 0.11, "threshold": 0.11860465258359909, "operator": "ge"}, {"budget": 0.12, "threshold": 0.10149314999580383, "operator": "ge"}, {"budget": 0.13, "threshold": 0.08444730937480927, "operator": "ge"}, {"budget": 0.14, "threshold": 0.07248879969120026, "operator": "ge"}, {"budget": 0.15, "threshold": 0.05950671061873436, "operator": "ge"}, {"budget": 0.16, "threshold": 0.04833132028579712, "operator": "ge"}, {"budget": 0.17, "threshold": 0.03855311870574951, "operator": "ge"}, {"budget": 0.18, "threshold": 0.02643565833568573, "operator": "ge"}, {"budget": 0.19, "threshold": 0.015617460012435913, "operator": "ge"}, {"budget": 0.2, "threshold": 0.004985429346561432, "operator": "ge"}, {"budget": 0.21, "threshold": -0.0028748000040650368, "operator": "ge"}, {"budget": 0.22, "threshold": -0.0047187963500618935, "operator": "ge"}, {"budget": 0.23, "threshold": -0.005887750070542097, "operator": "ge"}, {"budget": 0.24, "threshold": -0.006816945970058441, "operator": "ge"}, {"budget": 0.25, "threshold": -0.007673380430787802, "operator": "ge"}, {"budget": 0.26, "threshold": -0.00879391934722662, "operator": "ge"}, {"budget": 0.27, "threshold": -0.009685968980193138, "operator": "ge"}, {"budget": 0.28, "threshold": -0.01057250052690506, "operator": "ge"}, {"budget": 0.29, "threshold": -0.011273435316979885, "operator": "ge"}, {"budget": 0.3, "threshold": -0.012112271040678024, "operator": "ge"}, {"budget": 0.31, "threshold": -0.013190538622438908, "operator": "ge"}, {"budget": 0.32, "threshold": -0.013945600017905235, "operator": "ge"}, {"budget": 0.33, "threshold": -0.014789935201406479, "operator": "ge"}, {"budget": 0.34, "threshold": -0.01574079692363739, "operator": "ge"}, {"budget": 0.35, "threshold": -0.01676885038614273, "operator": "ge"}, {"budget": 0.36, "threshold": -0.017777279019355774, "operator": "ge"}, {"budget": 0.37, "threshold": -0.018889185041189194, "operator": "ge"}, {"budget": 0.38, "threshold": -0.01998099498450756, "operator": "ge"}, {"budget": 0.39, "threshold": -0.021131442859768867, "operator": "ge"}, {"budget": 0.4, "threshold": -0.022267863154411316, "operator": "ge"}, {"budget": 0.41, "threshold": -0.02331492118537426, "operator": "ge"}, {"budget": 0.42, "threshold": -0.024530069902539253, "operator": "ge"}, {"budget": 0.43, "threshold": -0.025664659217000008, "operator": "ge"}, {"budget": 0.44, "threshold": -0.02681991457939148, "operator": "ge"}, {"budget": 0.45, "threshold": -0.027945952489972115, "operator": "ge"}, {"budget": 0.46, "threshold": -0.02892877161502838, "operator": "ge"}, {"budget": 0.47, "threshold": -0.030040591955184937, "operator": "ge"}, {"budget": 0.48, "threshold": -0.031050562858581543, "operator": "ge"}, {"budget": 0.49, "threshold": -0.03267833963036537, "operator": "ge"}, {"budget": 0.5, "threshold": -0.0339660570025444, "operator": "ge"}, {"budget": 0.51, "threshold": -0.035266533493995667, "operator": "ge"}, {"budget": 0.52, "threshold": -0.03625601530075073, "operator": "ge"}, {"budget": 0.53, "threshold": -0.037769775837659836, "operator": "ge"}, {"budget": 0.54, "threshold": -0.038903020322322845, "operator": "ge"}, {"budget": 0.55, "threshold": -0.04033960402011871, "operator": "ge"}, {"budget": 0.56, "threshold": -0.041802436113357544, "operator": "ge"}, {"budget": 0.57, "threshold": -0.043019577860832214, "operator": "ge"}, {"budget": 0.58, "threshold": -0.04454854503273964, "operator": "ge"}, {"budget": 0.59, "threshold": -0.04622175171971321, "operator": "ge"}, {"budget": 0.6, "threshold": -0.04713770002126694, "operator": "ge"}, {"budget": 0.61, "threshold": -0.04840948432683945, "operator": "ge"}, {"budget": 0.62, "threshold": -0.049482155591249466, "operator": "ge"}, {"budget": 0.63, "threshold": -0.05091208964586258, "operator": "ge"}, {"budget": 0.64, "threshold": -0.05215434357523918, "operator": "ge"}, {"budget": 0.65, "threshold": -0.05328764766454697, "operator": "ge"}, {"budget": 0.66, "threshold": -0.05491914227604866, "operator": "ge"}, {"budget": 0.67, "threshold": -0.056464001536369324, "operator": "ge"}, {"budget": 0.68, "threshold": -0.05806313455104828, "operator": "ge"}, {"budget": 0.69, "threshold": -0.05961256101727486, "operator": "ge"}, {"budget": 0.7, "threshold": -0.061363231390714645, "operator": "ge"}, {"budget": 0.71, "threshold": -0.06299355626106262, "operator": "ge"}, {"budget": 0.72, "threshold": -0.06459663808345795, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0661502331495285, "operator": "ge"}, {"budget": 0.74, "threshold": -0.06800451874732971, "operator": "ge"}, {"budget": 0.75, "threshold": -0.06984568387269974, "operator": "ge"}, {"budget": 0.76, "threshold": -0.07162370532751083, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0734582394361496, "operator": "ge"}, {"budget": 0.78, "threshold": -0.07519975304603577, "operator": "ge"}, {"budget": 0.79, "threshold": -0.07709374278783798, "operator": "ge"}, {"budget": 0.8, "threshold": -0.07962089031934738, "operator": "ge"}, {"budget": 0.81, "threshold": -0.08206433057785034, "operator": "ge"}, {"budget": 0.82, "threshold": -0.08424125611782074, "operator": "ge"}, {"budget": 0.83, "threshold": -0.08671829849481583, "operator": "ge"}, {"budget": 0.84, "threshold": -0.08917007595300674, "operator": "ge"}, {"budget": 0.85, "threshold": -0.0918852910399437, "operator": "ge"}, {"budget": 0.86, "threshold": -0.09428446739912033, "operator": "ge"}, {"budget": 0.87, "threshold": -0.09746803343296051, "operator": "ge"}, {"budget": 0.88, "threshold": -0.10113058239221573, "operator": "ge"}, {"budget": 0.89, "threshold": -0.10453324764966965, "operator": "ge"}, {"budget": 0.9, "threshold": -0.10824505239725113, "operator": "ge"}, {"budget": 0.91, "threshold": -0.11277112364768982, "operator": "ge"}, {"budget": 0.92, "threshold": -0.11780354380607605, "operator": "ge"}, {"budget": 0.93, "threshold": -0.12321008741855621, "operator": "ge"}, {"budget": 0.94, "threshold": -0.12967044115066528, "operator": "ge"}, {"budget": 0.95, "threshold": -0.13648708164691925, "operator": "ge"}, {"budget": 0.96, "threshold": -0.14333570003509521, "operator": "ge"}, {"budget": 0.97, "threshold": -0.15267229080200195, "operator": "ge"}, {"budget": 0.98, "threshold": -0.16742494702339172, "operator": "ge"}, {"budget": 0.99, "threshold": -0.1868419051170349, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.00973932970496, 76.41077055285018, 76.82612431967917, 77.0552850186193, 77.3274133486107, 77.5708965912346, 77.85734746490976, 78.12947579490117, 78.5305070180464, 78.78831280435405, 79.10340876539674, 79.38985963907189, 79.7336006874821, 79.9054712116872, 80.10598682325981, 80.40676024061874, 80.75050128902893, 80.99398453165281, 81.18017759954168, 81.48095101690059, 81.73875680320825, 81.76740189057577, 81.85333715267831, 81.89630478372959, 81.96791750214838, 81.96791750214838, 81.98224004583213, 82.0681753079347, 82.03953022056717, 82.08249785161844, 82.11114293898596, 82.12546548266972, 82.22572328845604, 82.24004583213978, 82.31165855055858, 82.3403036379261, 82.4405614437124, 82.46920653107992, 82.56946433686623, 82.62675451160125, 82.75565740475508, 82.89888284159265, 82.98481810369522, 83.01346319106273, 83.05643082211401, 83.17101117158407, 83.25694643368662, 83.32855915210541, 83.4001718705242, 83.45746204525923, 83.54339730736179, 83.72959037525064, 83.90146089945574, 84.01604124892582, 84.14494414207962, 84.2595244915497, 84.33113720996849, 84.38842738470352, 84.46004010312231, 84.60326553995989, 84.73216843311371, 84.86107132626755, 84.94700658837009, 85.13319965625895, 85.21913491836149, 85.30507018046404, 85.3910054425666, 85.53423087940418, 85.60584359782297, 85.7347464909768, 85.9066170151819, 86.02119736465195, 86.1787453451733, 86.33629332569464, 86.49384130621598, 86.50816384989974, 86.57977656831854, 86.6227441993698, 86.82325981094242, 86.86622744199369, 87.00945287883127, 87.15267831566887, 87.36751647092524, 87.45345173302779, 87.5823546261816, 87.76854769407046, 87.94041821827557, 88.02635348037812, 88.1695789172157, 88.25551417931825, 88.41306215983958, 88.64222285877972, 88.85706101403609, 88.97164136350615, 89.1291893440275, 89.35835004296763, 89.47293039243769, 89.65912346032655, 89.65912346032655, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 35, 69, 102, 120, 144, 170, 198, 225, 256, 280, 306, 330, 358, 378, 399, 426, 454, 475, 496, 522, 541, 547, 553, 558, 564, 566, 569, 576, 578, 583, 587, 590, 598, 600, 606, 611, 620, 627, 636, 640, 651, 662, 670, 675, 680, 691, 700, 708, 716, 724, 734, 748, 761, 771, 780, 789, 799, 807, 817, 831, 842, 852, 865, 881, 893, 903, 911, 925, 933, 943, 958, 968, 980, 996, 1012, 1023, 1033, 1043, 1057, 1064, 1079, 1095, 1114, 1125, 1137, 1153, 1169, 1183, 1198, 1208, 1223, 1245, 1264, 1279, 1295, 1317, 1334, 1354, 1367, 1389], "harmed": [0, 4, 10, 14, 16, 21, 30, 38, 46, 49, 55, 59, 63, 67, 75, 82, 88, 92, 96, 104, 109, 110, 114, 114, 116, 117, 119, 121, 122, 126, 128, 130, 132, 133, 134, 135, 138, 140, 145, 147, 147, 149, 150, 152, 155, 157, 160, 163, 166, 169, 173, 177, 178, 179, 181, 181, 182, 187, 191, 196, 200, 202, 203, 210, 213, 219, 223, 225, 229, 232, 233, 236, 238, 239, 244, 249, 259, 264, 271, 271, 275, 280, 286, 290, 295, 298, 301, 305, 313, 318, 322, 326, 332, 336, 343, 348, 354, 363, 370, 383, 388], "exact_score": [75.56574047550845, 76.00973932970496, 76.41077055285018, 76.82612431967917, 77.0552850186193, 77.3274133486107, 77.5708965912346, 77.85734746490976, 78.12947579490117, 78.5305070180464, 78.78831280435405, 79.10340876539674, 79.38985963907189, 79.7336006874821, 79.9054712116872, 80.10598682325981, 80.40676024061874, 80.75050128902893, 80.99398453165281, 81.18017759954168, 81.48095101690059, 81.73875680320825, 81.76740189057577, 81.85333715267831, 81.89630478372959, 81.96791750214838, 81.96791750214838, 81.98224004583213, 82.0681753079347, 82.03953022056717, 82.08249785161844, 82.11114293898596, 82.12546548266972, 82.22572328845604, 82.24004583213978, 82.31165855055858, 82.3403036379261, 82.4405614437124, 82.46920653107992, 82.56946433686623, 82.62675451160125, 82.75565740475508, 82.89888284159265, 82.98481810369522, 83.01346319106273, 83.05643082211401, 83.17101117158407, 83.25694643368662, 83.32855915210541, 83.4001718705242, 83.45746204525923, 83.54339730736179, 83.72959037525064, 83.90146089945574, 84.01604124892582, 84.14494414207962, 84.2595244915497, 84.33113720996849, 84.38842738470352, 84.46004010312231, 84.60326553995989, 84.73216843311371, 84.86107132626755, 84.94700658837009, 85.13319965625895, 85.21913491836149, 85.30507018046404, 85.3910054425666, 85.53423087940418, 85.60584359782297, 85.7347464909768, 85.9066170151819, 86.02119736465195, 86.1787453451733, 86.33629332569464, 86.49384130621598, 86.50816384989974, 86.57977656831854, 86.6227441993698, 86.82325981094242, 86.86622744199369, 87.00945287883127, 87.15267831566887, 87.36751647092524, 87.45345173302779, 87.5823546261816, 87.76854769407046, 87.94041821827557, 88.02635348037812, 88.1695789172157, 88.25551417931825, 88.41306215983958, 88.64222285877972, 88.85706101403609, 88.97164136350615, 89.1291893440275, 89.35835004296763, 89.47293039243769, 89.65912346032655, 89.65912346032655, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 83.71606683339513, "threshold_area100": 83.71505299341162, "threshold_area25": 79.29389859639073}, "test": {"score": [76.42247255506925, 76.86733156346416, 77.15433737533185, 77.40546746071608, 77.7498744349573, 78.036880246825, 78.33106120398938, 78.596541579967, 78.92659826361485, 79.20642893018584, 79.47190930616345, 79.72303939154767, 80.10332209227236, 80.34727703235991, 80.59123197244745, 80.87823778431513, 81.12219272440268, 81.28722106622659, 81.49530027983067, 81.70337949343475, 81.97603501470905, 82.16258879242305, 82.27021597187343, 82.2989165530602, 82.35631771543373, 82.3921934419172, 82.45676974958744, 82.4782951854775, 82.4782951854775, 82.49982062136758, 82.52134605725766, 82.5787472196312, 82.65049867259813, 82.74377556145512, 82.79400157853198, 82.88010332209228, 82.93750448446582, 83.00208079213604, 83.08818253569635, 83.15275884336658, 83.2173351510368, 83.26038602281697, 83.34648776637728, 83.38953863815742, 83.46846523642104, 83.58326756116811, 83.67654445002512, 83.77699648417881, 83.84157279184905, 83.97072540718949, 84.04247686015643, 84.1572791849035, 84.23620578316711, 84.35818325321088, 84.44428499677119, 84.56626246681496, 84.66671450096864, 84.77434168041903, 84.91066944105619, 85.06134749228671, 85.11157350936357, 85.21920068881394, 85.25507641529741, 85.3555284494511, 85.3698787400445, 85.42727990241804, 85.6353591160221, 85.75733658606588, 85.9151897825931, 86.13044414149387, 86.24524646624094, 86.41027480806487, 86.49637655162518, 86.66140489344909, 86.88383439764655, 87.01298701298703, 87.14931477362417, 87.27129224366794, 87.3932697137117, 87.50807203845878, 87.78072755973308, 87.93140561096362, 88.14665996986439, 88.30451316639162, 88.39061490995192, 88.47671665351223, 88.62021955944608, 88.70632130300639, 88.87134964483032, 89.065078567841, 89.19423118318146, 89.38796010619215, 89.50993757623593, 89.63909019157639, 89.82564396929038, 89.97632202052091, 90.14852550764154, 90.277678122982, 90.43553131950922, 90.5503336442563, 90.73688742197031], "calls": [0.0, 1.0762717945038387, 2.1740690248977543, 2.977685298127287, 3.874578460213819, 4.771471622300352, 5.661189639090192, 6.5796082370668, 7.6200043050871775, 8.51689746717371, 9.356389466886704, 10.317858936643468, 11.415656167037383, 12.463227380354452, 13.274018798880677, 14.257013704527516, 15.060629977757051, 15.814020233909737, 16.696563105402884, 17.708258592236493, 18.648202626103178, 19.688598694123556, 20.693119035660473, 21.532611035373467, 22.37210303508646, 23.118318145942453, 24.25199110281983, 25.213460572576597, 26.11035373466313, 26.84939370022243, 27.71758628112219, 28.94453612685657, 29.76967783597618, 30.71679701513956, 31.6926167754897, 32.740187988806774, 33.830810073904, 35.0290593384516, 36.019229389395136, 37.15290234627251, 38.15742268780943, 39.14041759345627, 40.28844084092703, 41.32166176365072, 42.24725550692401, 43.33787759202124, 44.26347133529454, 45.24646624094138, 46.164884838917985, 47.52816244528952, 48.64748511157351, 49.809858649637654, 50.71392695702088, 51.9911028198321, 52.852120255435175, 53.95709263112578, 55.13381645978331, 56.11681136543015, 57.15720743345053, 58.506134749228664, 59.20929898830452, 60.142067876874506, 60.93133385951065, 62.086532252278104, 63.02647628614479, 63.7655162517041, 64.9781158068451, 66.0830881825357, 67.23828657530315, 68.2141063356533, 69.29755327545382, 70.23032216402382, 71.23484250556074, 72.26088828298774, 73.29410920571141, 74.32015498313841, 75.23857358111502, 76.12829159790486, 77.06106048647486, 78.07993111860515, 79.3284064002296, 80.49795508359044, 81.50247542512736, 82.55004663844443, 83.49716581760781, 84.48016072325466, 85.29812728707756, 86.38874937217479, 87.54394776494225, 88.67044557652292, 89.71084164454331, 90.84451460142068, 91.96383726770468, 93.0544593528019, 94.20965774556936, 95.11372605295257, 96.00344406974241, 96.98643897538926, 98.12728707756332, 98.96677907727631, 100.0], "rescued": [0, 65, 120, 162, 213, 263, 312, 356, 409, 457, 501, 547, 609, 659, 701, 749, 789, 821, 860, 900, 942, 979, 996, 1003, 1012, 1019, 1029, 1036, 1040, 1044, 1050, 1060, 1072, 1088, 1100, 1116, 1129, 1144, 1160, 1171, 1183, 1199, 1214, 1229, 1244, 1263, 1277, 1296, 1311, 1336, 1355, 1377, 1392, 1417, 1432, 1451, 1468, 1488, 1515, 1545, 1558, 1576, 1591, 1612, 1624, 1637, 1672, 1694, 1722, 1758, 1782, 1813, 1835, 1864, 1901, 1925, 1952, 1977, 2003, 2027, 2071, 2101, 2140, 2172, 2197, 2221, 2244, 2267, 2302, 2339, 2366, 2409, 2440, 2476, 2515, 2545, 2577, 2610, 2641, 2667, 2706], "harmed": [0, 3, 18, 25, 28, 38, 46, 53, 60, 69, 76, 87, 96, 112, 120, 128, 134, 143, 153, 164, 168, 179, 181, 184, 185, 187, 188, 192, 196, 197, 200, 202, 204, 207, 212, 216, 221, 227, 231, 233, 236, 246, 249, 258, 262, 265, 266, 271, 277, 284, 293, 299, 303, 311, 314, 316, 319, 324, 332, 341, 347, 350, 360, 367, 377, 382, 388, 393, 399, 405, 413, 421, 431, 437, 443, 449, 457, 465, 474, 482, 488, 497, 506, 516, 529, 541, 544, 555, 567, 577, 586, 602, 616, 634, 647, 656, 664, 679, 688, 698, 711], "exact_score": [76.42247255506925, 76.8386309822774, 77.09693621295831, 77.41264260601277, 77.77139987084739, 78.07993111860515, 78.41716294754968, 78.74721963119752, 79.062926024252, 79.34993183611968, 79.6082370668006, 79.95264404104184, 80.24682499820621, 80.51948051948052, 80.8208366219416, 81.12936786969937, 81.33027193800675, 81.5598765875009, 81.80383152758843, 82.01908588648921, 82.21998995479659, 82.27021597187343, 82.32044198895028, 82.37784315132382, 82.41371887780727, 82.4782951854775, 82.47112004018081, 82.50699576666428, 82.52134605725766, 82.58592236492788, 82.68637439908159, 82.75812585204851, 82.78682643323528, 82.90162875798235, 82.96620506565257, 82.99490564683936, 83.08100739039965, 83.14558369806988, 83.2173351510368, 83.25321087752027, 83.33931262108058, 83.36801320226735, 83.44693980053096, 83.54021668938796, 83.66219415943173, 83.76982133888211, 83.81287221066228, 83.8774485183325, 83.99225084307957, 84.0711774413432, 84.18597976609027, 84.26490636435389, 84.35818325321088, 84.47298557795796, 84.57343761211165, 84.66671450096864, 84.74564109923226, 84.9034942957595, 84.97524574872641, 85.11157350936357, 85.21920068881394, 85.2694267058908, 85.34117815885772, 85.3627035947478, 85.4416301930114, 85.64253426131879, 85.75733658606588, 85.88648920140633, 86.11609385090048, 86.21654588505417, 86.3743990815814, 86.45332567984502, 86.61835402166894, 86.81208294467963, 87.00581186769031, 87.16366506421755, 87.25694195307454, 87.37174427782163, 87.4937217478654, 87.6802755255794, 87.88117959388678, 88.00315706393054, 88.23276171342471, 88.34756403817178, 88.41931549113869, 88.58434383296262, 88.691971012413, 88.77807275597331, 88.93592595250054, 89.10095429432447, 89.20140632847816, 89.4238358326756, 89.50993757623593, 89.66779077276314, 89.79694338810361, 89.94044629403746, 90.14135036234484, 90.277678122982, 90.39965559302576, 90.5503336442563, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 84.42859718537126, "threshold_area100": 84.32492645476071, "threshold_area25": 79.95407907010119}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.3687095642089844, "operator": "ge"}, {"budget": 0.02, "threshold": 2.054736375808716, "operator": "ge"}, {"budget": 0.03, "threshold": 1.843550443649292, "operator": "ge"}, {"budget": 0.04, "threshold": 1.6819183826446533, "operator": "ge"}, {"budget": 0.05, "threshold": 1.5356124639511108, "operator": "ge"}, {"budget": 0.06, "threshold": 1.4073903560638428, "operator": "ge"}, {"budget": 0.07, "threshold": 1.291198968887329, "operator": "ge"}, {"budget": 0.08, "threshold": 1.181732177734375, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0857033729553223, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9778388738632202, "operator": "ge"}, {"budget": 0.11, "threshold": 0.8812270164489746, "operator": "ge"}, {"budget": 0.12, "threshold": 0.8008946180343628, "operator": "ge"}, {"budget": 0.13, "threshold": 0.708441972732544, "operator": "ge"}, {"budget": 0.14, "threshold": 0.620721697807312, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5267055034637451, "operator": "ge"}, {"budget": 0.16, "threshold": 0.4376007318496704, "operator": "ge"}, {"budget": 0.17, "threshold": 0.3363415598869324, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2511863112449646, "operator": "ge"}, {"budget": 0.19, "threshold": 0.14195144176483154, "operator": "ge"}, {"budget": 0.2, "threshold": 0.05445101857185364, "operator": "ge"}, {"budget": 0.21, "threshold": -0.045082807540893555, "operator": "ge"}, {"budget": 0.22, "threshold": -0.16077473759651184, "operator": "ge"}, {"budget": 0.23, "threshold": -0.24666613340377808, "operator": "ge"}, {"budget": 0.24, "threshold": -0.3474770188331604, "operator": "ge"}, {"budget": 0.25, "threshold": -0.4326431155204773, "operator": "ge"}, {"budget": 0.26, "threshold": -0.543546736240387, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6353045701980591, "operator": "ge"}, {"budget": 0.28, "threshold": -0.7398222088813782, "operator": "ge"}, {"budget": 0.29, "threshold": -0.8362875580787659, "operator": "ge"}, {"budget": 0.3, "threshold": -0.9280846118927002, "operator": "ge"}, {"budget": 0.31, "threshold": -1.0117948055267334, "operator": "ge"}, {"budget": 0.32, "threshold": -1.1117143630981445, "operator": "ge"}, {"budget": 0.33, "threshold": -1.1879420280456543, "operator": "ge"}, {"budget": 0.34, "threshold": -1.2983132600784302, "operator": "ge"}, {"budget": 0.35, "threshold": -1.394158124923706, "operator": "ge"}, {"budget": 0.36, "threshold": -1.500007152557373, "operator": "ge"}, {"budget": 0.37, "threshold": -1.5863418579101562, "operator": "ge"}, {"budget": 0.38, "threshold": -1.701102614402771, "operator": "ge"}, {"budget": 0.39, "threshold": -1.810516595840454, "operator": "ge"}, {"budget": 0.4, "threshold": -1.9572476148605347, "operator": "ge"}, {"budget": 0.41, "threshold": -2.062697410583496, "operator": "ge"}, {"budget": 0.42, "threshold": -2.2168145179748535, "operator": "ge"}, {"budget": 0.43, "threshold": -2.3882484436035156, "operator": "ge"}, {"budget": 0.44, "threshold": -2.527621269226074, "operator": "ge"}, {"budget": 0.45, "threshold": -2.666400194168091, "operator": "ge"}, {"budget": 0.46, "threshold": -2.812565326690674, "operator": "ge"}, {"budget": 0.47, "threshold": -2.9476754665374756, "operator": "ge"}, {"budget": 0.48, "threshold": -3.063755989074707, "operator": "ge"}, {"budget": 0.49, "threshold": -3.169231653213501, "operator": "ge"}, {"budget": 0.5, "threshold": -3.3043999671936035, "operator": "ge"}, {"budget": 0.51, "threshold": -3.4119133949279785, "operator": "ge"}, {"budget": 0.52, "threshold": -3.5435030460357666, "operator": "ge"}, {"budget": 0.53, "threshold": -3.644130229949951, "operator": "ge"}, {"budget": 0.54, "threshold": -3.7641706466674805, "operator": "ge"}, {"budget": 0.55, "threshold": -3.8859848976135254, "operator": "ge"}, {"budget": 0.56, "threshold": -3.9938559532165527, "operator": "ge"}, {"budget": 0.57, "threshold": -4.093439102172852, "operator": "ge"}, {"budget": 0.58, "threshold": -4.179982662200928, "operator": "ge"}, {"budget": 0.59, "threshold": -4.26503324508667, "operator": "ge"}, {"budget": 0.6, "threshold": -4.359347343444824, "operator": "ge"}, {"budget": 0.61, "threshold": -4.437066078186035, "operator": "ge"}, {"budget": 0.62, "threshold": -4.5263471603393555, "operator": "ge"}, {"budget": 0.63, "threshold": -4.6085100173950195, "operator": "ge"}, {"budget": 0.64, "threshold": -4.6746826171875, "operator": "ge"}, {"budget": 0.65, "threshold": -4.744001388549805, "operator": "ge"}, {"budget": 0.66, "threshold": -4.799456596374512, "operator": "ge"}, {"budget": 0.67, "threshold": -4.857142925262451, "operator": "ge"}, {"budget": 0.68, "threshold": -4.926192283630371, "operator": "ge"}, {"budget": 0.69, "threshold": -4.997354507446289, "operator": "ge"}, {"budget": 0.7, "threshold": -5.0673041343688965, "operator": "ge"}, {"budget": 0.71, "threshold": -5.148287773132324, "operator": "ge"}, {"budget": 0.72, "threshold": -5.213038921356201, "operator": "ge"}, {"budget": 0.73, "threshold": -5.295984268188477, "operator": "ge"}, {"budget": 0.74, "threshold": -5.369488716125488, "operator": "ge"}, {"budget": 0.75, "threshold": -5.421236515045166, "operator": "ge"}, {"budget": 0.76, "threshold": -5.497439384460449, "operator": "ge"}, {"budget": 0.77, "threshold": -5.560326099395752, "operator": "ge"}, {"budget": 0.78, "threshold": -5.6307244300842285, "operator": "ge"}, {"budget": 0.79, "threshold": -5.71870231628418, "operator": "ge"}, {"budget": 0.8, "threshold": -5.802583694458008, "operator": "ge"}, {"budget": 0.81, "threshold": -5.871333122253418, "operator": "ge"}, {"budget": 0.82, "threshold": -5.9365997314453125, "operator": "ge"}, {"budget": 0.83, "threshold": -6.021441459655762, "operator": "ge"}, {"budget": 0.84, "threshold": -6.099334239959717, "operator": "ge"}, {"budget": 0.85, "threshold": -6.1823530197143555, "operator": "ge"}, {"budget": 0.86, "threshold": -6.257395267486572, "operator": "ge"}, {"budget": 0.87, "threshold": -6.320274353027344, "operator": "ge"}, {"budget": 0.88, "threshold": -6.403284072875977, "operator": "ge"}, {"budget": 0.89, "threshold": -6.4735426902771, "operator": "ge"}, {"budget": 0.9, "threshold": -6.563778400421143, "operator": "ge"}, {"budget": 0.91, "threshold": -6.651202201843262, "operator": "ge"}, {"budget": 0.92, "threshold": -6.759310722351074, "operator": "ge"}, {"budget": 0.93, "threshold": -6.853146553039551, "operator": "ge"}, {"budget": 0.94, "threshold": -6.967232704162598, "operator": "ge"}, {"budget": 0.95, "threshold": -7.060482978820801, "operator": "ge"}, {"budget": 0.96, "threshold": -7.196783542633057, "operator": "ge"}, {"budget": 0.97, "threshold": -7.35296106338501, "operator": "ge"}, {"budget": 0.98, "threshold": -7.549773216247559, "operator": "ge"}, {"budget": 0.99, "threshold": -7.835921287536621, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.00973932970496, 76.36780292179891, 76.74018905757663, 77.01231738756803, 77.27012317387569, 77.64250930965339, 78.00057290174735, 78.38728158120882, 78.65940991120023, 78.90289315382411, 79.07476367802921, 79.33256946433687, 79.71927814379833, 80.04869664852478, 80.23488971641363, 80.44972787167, 80.76482383271268, 80.99398453165281, 81.18017759954168, 81.48095101690059, 81.83901460899456, 81.91062732741334, 82.168433113721, 82.39759381266113, 82.58378688054998, 82.8845602979089, 83.09939845316528, 83.32855915210541, 83.48610713262676, 83.61501002578058, 83.78688054998568, 84.00171870524204, 84.14494414207962, 84.34545975365224, 84.63191062732741, 84.86107132626755, 85.20481237467774, 85.34803781151533, 85.66313377255801, 85.86364938413061, 85.99255227728445, 86.19306788885706, 86.27900315095961, 86.3506158693784, 86.43655113148094, 86.56545402463478, 86.66571183042107, 86.75164709252363, 86.88054998567746, 86.98080779146376, 87.09538814093382, 87.19564594672013, 87.2099684904039, 87.35319392724148, 87.42480664566027, 87.53938699513033, 87.5823546261816, 87.6539673446004, 87.78287023775422, 87.88312804354052, 87.9260956745918, 88.01203093669436, 88.14093382984818, 88.1695789172157, 88.269836723002, 88.34144944142079, 88.39873961615584, 88.48467487825837, 88.6135777714122, 88.6135777714122, 88.65654540246346, 88.71383557719851, 88.71383557719851, 88.7854482956173, 88.81409338298482, 88.87138355771985, 88.94299627613864, 89.01460899455743, 89.0575766256087, 89.1291893440275, 89.14351188771126, 89.18647951876252, 89.22944714981381, 89.3297049556001, 89.4013176740189, 89.38699513033515, 89.42996276138642, 89.53022056717272, 89.55886565454024, 89.5445431108565, 89.55886565454024, 89.68776854769406, 89.70209109137782, 89.6734460040103, 89.75938126611285, 89.75938126611285, 89.87396161558291, 89.88828415926669, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 33, 62, 93, 120, 142, 172, 200, 231, 254, 278, 300, 324, 353, 383, 405, 425, 452, 474, 495, 520, 549, 566, 588, 610, 628, 652, 672, 693, 710, 727, 741, 761, 777, 794, 815, 834, 862, 875, 902, 921, 937, 953, 964, 974, 985, 1000, 1014, 1027, 1039, 1047, 1059, 1069, 1074, 1087, 1095, 1108, 1116, 1122, 1132, 1140, 1148, 1156, 1167, 1173, 1183, 1191, 1199, 1208, 1219, 1227, 1234, 1241, 1246, 1252, 1256, 1262, 1268, 1277, 1283, 1290, 1294, 1298, 1304, 1312, 1318, 1322, 1326, 1333, 1337, 1341, 1345, 1354, 1357, 1358, 1364, 1368, 1378, 1381, 1384, 1389], "harmed": [0, 2, 6, 11, 19, 23, 27, 30, 34, 38, 45, 55, 61, 63, 70, 79, 84, 89, 95, 103, 107, 111, 123, 127, 133, 138, 141, 146, 151, 157, 165, 167, 172, 178, 181, 182, 185, 189, 192, 197, 202, 209, 211, 216, 221, 226, 232, 239, 246, 249, 250, 254, 257, 261, 264, 267, 272, 277, 278, 279, 280, 285, 287, 289, 293, 296, 299, 303, 306, 308, 316, 320, 323, 328, 329, 331, 333, 334, 338, 341, 343, 346, 347, 350, 351, 352, 357, 358, 358, 360, 365, 368, 368, 370, 373, 373, 377, 379, 381, 383, 388], "exact_score": [75.56574047550845, 76.00973932970496, 76.36780292179891, 76.74018905757663, 77.01231738756803, 77.27012317387569, 77.64250930965339, 78.00057290174735, 78.38728158120882, 78.65940991120023, 78.90289315382411, 79.07476367802921, 79.33256946433687, 79.71927814379833, 80.04869664852478, 80.23488971641363, 80.44972787167, 80.76482383271268, 80.99398453165281, 81.18017759954168, 81.48095101690059, 81.83901460899456, 81.91062732741334, 82.168433113721, 82.39759381266113, 82.58378688054998, 82.8845602979089, 83.09939845316528, 83.32855915210541, 83.48610713262676, 83.61501002578058, 83.78688054998568, 84.00171870524204, 84.14494414207962, 84.34545975365224, 84.63191062732741, 84.86107132626755, 85.20481237467774, 85.34803781151533, 85.66313377255801, 85.86364938413061, 85.99255227728445, 86.19306788885706, 86.27900315095961, 86.3506158693784, 86.43655113148094, 86.56545402463478, 86.66571183042107, 86.75164709252363, 86.88054998567746, 86.98080779146376, 87.09538814093382, 87.19564594672013, 87.2099684904039, 87.35319392724148, 87.42480664566027, 87.53938699513033, 87.5823546261816, 87.6539673446004, 87.78287023775422, 87.88312804354052, 87.9260956745918, 88.01203093669436, 88.14093382984818, 88.1695789172157, 88.269836723002, 88.34144944142079, 88.39873961615584, 88.48467487825837, 88.6135777714122, 88.6135777714122, 88.65654540246346, 88.71383557719851, 88.71383557719851, 88.7854482956173, 88.81409338298482, 88.87138355771985, 88.94299627613864, 89.01460899455743, 89.0575766256087, 89.1291893440275, 89.14351188771126, 89.18647951876252, 89.22944714981381, 89.3297049556001, 89.4013176740189, 89.38699513033515, 89.42996276138642, 89.53022056717272, 89.55886565454024, 89.5445431108565, 89.55886565454024, 89.68776854769406, 89.70209109137782, 89.6734460040103, 89.75938126611285, 89.75938126611285, 89.87396161558291, 89.88828415926669, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.49309818323188, "threshold_area100": 85.49205098825551, "threshold_area25": 79.39272414780865}, "test": {"score": [76.42247255506925, 76.8458061275741, 77.21173853770539, 77.49874434957307, 77.80727559733084, 78.11580684508861, 78.40281265695631, 78.71134390471407, 79.02705029776853, 79.2351295113726, 79.57236134031714, 79.83784171629476, 80.0746215110856, 80.30422616057974, 80.54100595537061, 80.77778575016144, 81.08631699791921, 81.28722106622659, 81.51682571572074, 81.71772978402812, 81.96885986941236, 82.21998995479659, 82.4854703307742, 82.74377556145512, 82.95902992035589, 83.2316854416302, 83.46846523642104, 83.64066872354165, 83.9635502618928, 84.17162947549687, 84.49451101384804, 84.67388964626534, 84.9034942957595, 85.1689746717371, 85.49903135538494, 85.75733658606588, 85.95106550907656, 86.15914472268064, 86.33134820980125, 86.53942742340533, 86.74750663700941, 86.91971012413, 87.05603788476716, 87.27129224366794, 87.43632058549186, 87.61569921790917, 87.72332639735954, 87.92423046566694, 87.97445648274378, 88.03185764511731, 88.09643395278755, 88.23276171342471, 88.34038889287508, 88.39061490995192, 88.412140345842, 88.49106694410563, 88.58434383296262, 88.63456985003947, 88.72784673889646, 88.83547391834685, 88.94310109779724, 89.03637798665423, 89.12965487551124, 89.15118031140132, 89.158355456698, 89.22293176436823, 89.28033292674176, 89.33055894381862, 89.41666068737892, 89.47406184975246, 89.57451388390615, 89.66061562746646, 89.8041185334003, 89.86151969577384, 89.8973954222573, 89.91892085814737, 89.997847456411, 90.02654803759776, 90.0839491999713, 90.0982994905647, 90.22027696060846, 90.29920355887207, 90.31355384946545, 90.38530530243239, 90.38530530243239, 90.44988161010261, 90.52163306306953, 90.52163306306953, 90.50728277247615, 90.52163306306953, 90.55750878955298, 90.57185908014638, 90.62208509722322, 90.6364353878166, 90.66513596900337, 90.70818684078353, 90.744062567267, 90.70818684078353, 90.70818684078353, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.1121475209873, 2.2242950419746, 3.1785893664346707, 4.1041831097079715, 5.123053741838272, 6.091698356891727, 7.038817536055104, 7.935710698141637, 8.70345124488771, 9.585994116380856, 10.42548611609385, 11.257802970510152, 12.11164526081653, 12.987012987012985, 13.869555858506136, 14.909951926526514, 15.921647413360121, 16.825715720743347, 17.80153548109349, 18.65537777139987, 19.76752529238717, 21.123627753462006, 21.963119753175, 22.903063787041688, 23.80713209442491, 24.833177871851905, 25.780297051015282, 27.07899834971658, 28.155270144220417, 29.260242519911028, 30.25041257085456, 31.649565903709547, 32.639735954653084, 33.924086962760995, 35.04340962904499, 36.047929970581905, 36.92329769677836, 38.013919781875586, 39.09736672167611, 40.18081366147664, 41.03465595178302, 42.11092774628686, 43.28765157494439, 44.284996771184616, 45.21059051445791, 46.21511085599484, 47.19810576164167, 47.87974456482743, 48.69053598335366, 49.644830307813734, 50.48432230752673, 51.55341895673388, 52.479012700007175, 53.39025615268709, 54.473703092487625, 55.49257372461792, 56.52579464734161, 57.47291382650499, 58.42003300566837, 59.48195450957881, 60.479299705819045, 61.37619286790558, 62.366362918849106, 63.306306952715794, 64.17449953361556, 64.98529095214178, 65.59517830236062, 66.52794719093062, 67.59704384013776, 68.69484107053168, 69.82851402740906, 70.76845806127574, 72.12456052235058, 73.13625600918418, 73.92552199182033, 74.9802683504341, 75.95608811078425, 76.9247327258377, 78.20190858864892, 79.56518619502046, 80.39032790414005, 81.28722106622659, 82.61462294611465, 83.79852192006888, 85.06852263758341, 86.23089617564756, 87.08473846595393, 88.17536055105116, 89.00767740546746, 89.93327114874076, 90.99519265265121, 91.99253784889144, 92.98988304513166, 93.9800530960752, 94.92717227523858, 96.00344406974241, 96.9433881036091, 97.92638300925593, 98.973954222573, 100.0], "rescued": [0, 63, 120, 167, 220, 270, 316, 367, 420, 456, 510, 554, 594, 640, 684, 726, 776, 818, 860, 902, 944, 990, 1044, 1088, 1129, 1174, 1218, 1254, 1312, 1351, 1406, 1442, 1488, 1533, 1587, 1634, 1672, 1710, 1748, 1788, 1823, 1853, 1881, 1915, 1943, 1974, 1998, 2030, 2042, 2059, 2077, 2100, 2118, 2131, 2144, 2166, 2185, 2199, 2215, 2235, 2256, 2274, 2293, 2305, 2317, 2331, 2345, 2354, 2370, 2385, 2403, 2419, 2442, 2453, 2461, 2469, 2484, 2491, 2505, 2512, 2533, 2546, 2556, 2572, 2585, 2597, 2610, 2617, 2623, 2629, 2636, 2642, 2650, 2660, 2668, 2678, 2686, 2688, 2694, 2698, 2706], "harmed": [0, 4, 10, 17, 27, 34, 40, 48, 57, 64, 71, 78, 85, 99, 110, 119, 126, 140, 150, 164, 171, 182, 199, 207, 218, 225, 236, 248, 261, 271, 281, 292, 306, 314, 322, 333, 344, 353, 367, 378, 384, 390, 399, 403, 408, 414, 423, 427, 432, 441, 450, 454, 457, 463, 473, 484, 490, 497, 500, 505, 511, 516, 522, 531, 542, 547, 553, 555, 559, 566, 570, 574, 577, 580, 583, 588, 592, 595, 601, 606, 610, 612, 620, 626, 639, 642, 645, 652, 660, 664, 666, 670, 671, 679, 683, 687, 690, 697, 703, 706, 711], "exact_score": [76.42247255506925, 76.79558011049724, 77.15433737533185, 77.43416804190285, 77.76422472555069, 78.07993111860515, 78.36693693047285, 78.69699361412069, 79.03422544306522, 79.342756690823, 79.70151395565762, 80.00287005811867, 80.2540001435029, 80.54100595537061, 80.81366147664491, 81.07196670732583, 81.30874650211666, 81.55270144220421, 81.76078065580829, 82.03343617708259, 82.27739111717013, 82.45676974958744, 82.75812585204851, 83.01643108272943, 83.30343689459711, 83.51151610820119, 83.71242017650857, 83.94919997129942, 84.15010403960682, 84.41558441558442, 84.63801391978187, 84.76716653512234, 84.99677118461649, 85.29095214178088, 85.54208222716511, 85.74298629547249, 85.94389036377987, 86.1878453038674, 86.33134820980125, 86.50355169692186, 86.69728061993256, 86.92688526942672, 87.06321303006386, 87.2067159359977, 87.37891942311832, 87.55112291023893, 87.6731003802827, 87.89552988448015, 87.99598191863386, 88.07490851689747, 88.15383511516109, 88.29016287579825, 88.35473918346847, 88.412140345842, 88.43366578173209, 88.56281839707253, 88.61304441414939, 88.67762072181962, 88.78524790127, 88.86417449953362, 89.00767740546746, 89.0722537131377, 89.1440051661046, 89.17270574729139, 89.19423118318146, 89.28033292674176, 89.38078496089545, 89.44536126856569, 89.52428786682931, 89.6103896103896, 89.69649135394992, 89.8041185334003, 89.86869484107054, 89.8973954222573, 89.92609600344407, 89.997847456411, 90.02654803759776, 90.0839491999713, 90.0982994905647, 90.1700509435316, 90.277678122982, 90.31355384946545, 90.32072899476215, 90.37813015713569, 90.40683073832245, 90.44988161010261, 90.49293248188276, 90.52163306306953, 90.52163306306953, 90.52163306306953, 90.56468393484968, 90.57185908014638, 90.62208509722322, 90.6364353878166, 90.66513596900337, 90.71536198608023, 90.744062567267, 90.70818684078353, 90.69383655019014, 90.72253713137691, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.47697246080182, "threshold_area100": 86.40758412857859, "threshold_area25": 80.05237856066586}, "selected_budget": 99, "selected": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692, "rescued": 1384, "harmed": 383}, "test": {"score": 90.71536198608023, "calls": 98.973954222573, "rescued": 2698, "harmed": 706}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.793300747871399, "operator": "ge"}, {"budget": 0.02, "threshold": 1.5414938926696777, "operator": "ge"}, {"budget": 0.03, "threshold": 1.3865294456481934, "operator": "ge"}, {"budget": 0.04, "threshold": 1.2487084865570068, "operator": "ge"}, {"budget": 0.05, "threshold": 1.133459210395813, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0318448543548584, "operator": "ge"}, {"budget": 0.07, "threshold": 0.950484573841095, "operator": "ge"}, {"budget": 0.08, "threshold": 0.8763502836227417, "operator": "ge"}, {"budget": 0.09, "threshold": 0.787489652633667, "operator": "ge"}, {"budget": 0.1, "threshold": 0.7244203686714172, "operator": "ge"}, {"budget": 0.11, "threshold": 0.6567074060440063, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5814065933227539, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5110837817192078, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4440340995788574, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3840314447879791, "operator": "ge"}, {"budget": 0.16, "threshold": 0.31019431352615356, "operator": "ge"}, {"budget": 0.17, "threshold": 0.2538216710090637, "operator": "ge"}, {"budget": 0.18, "threshold": 0.18691104650497437, "operator": "ge"}, {"budget": 0.19, "threshold": 0.10785448551177979, "operator": "ge"}, {"budget": 0.2, "threshold": 0.0371360182762146, "operator": "ge"}, {"budget": 0.21, "threshold": -0.030287861824035645, "operator": "ge"}, {"budget": 0.22, "threshold": -0.11173644661903381, "operator": "ge"}, {"budget": 0.23, "threshold": -0.18541184067726135, "operator": "ge"}, {"budget": 0.24, "threshold": -0.2733401656150818, "operator": "ge"}, {"budget": 0.25, "threshold": -0.33409881591796875, "operator": "ge"}, {"budget": 0.26, "threshold": -0.4096441864967346, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5048065185546875, "operator": "ge"}, {"budget": 0.28, "threshold": -0.5827787518501282, "operator": "ge"}, {"budget": 0.29, "threshold": -0.6691994071006775, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7401950359344482, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8283160328865051, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9099457859992981, "operator": "ge"}, {"budget": 0.33, "threshold": -1.005194067955017, "operator": "ge"}, {"budget": 0.34, "threshold": -1.083240032196045, "operator": "ge"}, {"budget": 0.35, "threshold": -1.175600528717041, "operator": "ge"}, {"budget": 0.36, "threshold": -1.269718050956726, "operator": "ge"}, {"budget": 0.37, "threshold": -1.3724271059036255, "operator": "ge"}, {"budget": 0.38, "threshold": -1.4705312252044678, "operator": "ge"}, {"budget": 0.39, "threshold": -1.596390724182129, "operator": "ge"}, {"budget": 0.4, "threshold": -1.7163372039794922, "operator": "ge"}, {"budget": 0.41, "threshold": -1.8497428894042969, "operator": "ge"}, {"budget": 0.42, "threshold": -1.9915695190429688, "operator": "ge"}, {"budget": 0.43, "threshold": -2.123476266860962, "operator": "ge"}, {"budget": 0.44, "threshold": -2.2532150745391846, "operator": "ge"}, {"budget": 0.45, "threshold": -2.396935224533081, "operator": "ge"}, {"budget": 0.46, "threshold": -2.5118463039398193, "operator": "ge"}, {"budget": 0.47, "threshold": -2.6532113552093506, "operator": "ge"}, {"budget": 0.48, "threshold": -2.7781834602355957, "operator": "ge"}, {"budget": 0.49, "threshold": -2.8863070011138916, "operator": "ge"}, {"budget": 0.5, "threshold": -2.986449718475342, "operator": "ge"}, {"budget": 0.51, "threshold": -3.1118977069854736, "operator": "ge"}, {"budget": 0.52, "threshold": -3.2362611293792725, "operator": "ge"}, {"budget": 0.53, "threshold": -3.32401967048645, "operator": "ge"}, {"budget": 0.54, "threshold": -3.4366507530212402, "operator": "ge"}, {"budget": 0.55, "threshold": -3.5562820434570312, "operator": "ge"}, {"budget": 0.56, "threshold": -3.6524269580841064, "operator": "ge"}, {"budget": 0.57, "threshold": -3.7496092319488525, "operator": "ge"}, {"budget": 0.58, "threshold": -3.828132390975952, "operator": "ge"}, {"budget": 0.59, "threshold": -3.9109652042388916, "operator": "ge"}, {"budget": 0.6, "threshold": -3.987074613571167, "operator": "ge"}, {"budget": 0.61, "threshold": -4.072493553161621, "operator": "ge"}, {"budget": 0.62, "threshold": -4.14772891998291, "operator": "ge"}, {"budget": 0.63, "threshold": -4.219964027404785, "operator": "ge"}, {"budget": 0.64, "threshold": -4.2921671867370605, "operator": "ge"}, {"budget": 0.65, "threshold": -4.351853370666504, "operator": "ge"}, {"budget": 0.66, "threshold": -4.410730838775635, "operator": "ge"}, {"budget": 0.67, "threshold": -4.474864959716797, "operator": "ge"}, {"budget": 0.68, "threshold": -4.530084609985352, "operator": "ge"}, {"budget": 0.69, "threshold": -4.59501838684082, "operator": "ge"}, {"budget": 0.7, "threshold": -4.663322448730469, "operator": "ge"}, {"budget": 0.71, "threshold": -4.717628002166748, "operator": "ge"}, {"budget": 0.72, "threshold": -4.788351058959961, "operator": "ge"}, {"budget": 0.73, "threshold": -4.852355480194092, "operator": "ge"}, {"budget": 0.74, "threshold": -4.923792839050293, "operator": "ge"}, {"budget": 0.75, "threshold": -4.982077598571777, "operator": "ge"}, {"budget": 0.76, "threshold": -5.048352241516113, "operator": "ge"}, {"budget": 0.77, "threshold": -5.108119487762451, "operator": "ge"}, {"budget": 0.78, "threshold": -5.169318675994873, "operator": "ge"}, {"budget": 0.79, "threshold": -5.235565185546875, "operator": "ge"}, {"budget": 0.8, "threshold": -5.3139166831970215, "operator": "ge"}, {"budget": 0.81, "threshold": -5.384610652923584, "operator": "ge"}, {"budget": 0.82, "threshold": -5.442378997802734, "operator": "ge"}, {"budget": 0.83, "threshold": -5.5115461349487305, "operator": "ge"}, {"budget": 0.84, "threshold": -5.575592994689941, "operator": "ge"}, {"budget": 0.85, "threshold": -5.657831192016602, "operator": "ge"}, {"budget": 0.86, "threshold": -5.737244129180908, "operator": "ge"}, {"budget": 0.87, "threshold": -5.799070358276367, "operator": "ge"}, {"budget": 0.88, "threshold": -5.86030912399292, "operator": "ge"}, {"budget": 0.89, "threshold": -5.939730644226074, "operator": "ge"}, {"budget": 0.9, "threshold": -6.026468753814697, "operator": "ge"}, {"budget": 0.91, "threshold": -6.109968185424805, "operator": "ge"}, {"budget": 0.92, "threshold": -6.196216583251953, "operator": "ge"}, {"budget": 0.93, "threshold": -6.280068397521973, "operator": "ge"}, {"budget": 0.94, "threshold": -6.392003536224365, "operator": "ge"}, {"budget": 0.95, "threshold": -6.49304723739624, "operator": "ge"}, {"budget": 0.96, "threshold": -6.5992326736450195, "operator": "ge"}, {"budget": 0.97, "threshold": -6.737806797027588, "operator": "ge"}, {"budget": 0.98, "threshold": -6.936276435852051, "operator": "ge"}, {"budget": 0.99, "threshold": -7.215461254119873, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.96677169865367, 76.39644800916643, 76.65425379547408, 76.94070466914924, 77.25580063019191, 77.6711543970209, 78.05786307648238, 78.32999140647378, 78.58779719278144, 78.8742480664566, 79.17502148381553, 79.51876253222572, 79.77656831853336, 80.02005156115726, 80.23488971641363, 80.56430822114007, 80.75050128902893, 80.99398453165281, 81.19450014322544, 81.49527356058435, 81.81036952162704, 81.92494987109711, 82.168433113721, 82.35462618160985, 82.62675451160125, 82.85591521054138, 83.05643082211401, 83.31423660842165, 83.45746204525923, 83.60068748209682, 83.70094528788312, 83.93010598682325, 84.14494414207962, 84.35978229733601, 84.60326553995989, 84.83242623890003, 85.16184474362647, 85.40532798625036, 85.70610140360928, 85.82068175307934, 86.02119736465195, 86.12145517043827, 86.23603551990833, 86.40790604411343, 86.47951876253222, 86.53680893726725, 86.63706674305357, 86.66571183042107, 86.79461472357491, 86.99513033514752, 87.05242050988255, 87.12403322830134, 87.22429103408766, 87.32454883987396, 87.35319392724148, 87.45345173302779, 87.6539673446004, 87.76854769407046, 87.85448295617302, 87.9260956745918, 88.01203093669436, 88.09796619879691, 88.18390146089945, 88.269836723002, 88.35577198510455, 88.31280435405327, 88.38441707247206, 88.47035233457461, 88.57061014036093, 88.6135777714122, 88.75680320824978, 88.79977083930106, 88.82841592666857, 88.87138355771985, 88.8857061014036, 88.91435118877112, 89.02893153824118, 89.08622171297623, 89.11486680034373, 89.18647951876252, 89.21512460613005, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.37267258665138, 89.4013176740189, 89.48725293612145, 89.50157547980521, 89.5445431108565, 89.61615582927529, 89.68776854769406, 89.73073617874535, 89.78802635348038, 89.8166714408479, 89.80234889716414, 89.84531652821542, 89.85963907189917, 89.9169292466342, 89.9169292466342, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 63, 89, 117, 141, 172, 203, 228, 252, 277, 302, 332, 355, 383, 405, 433, 451, 473, 495, 521, 548, 567, 590, 610, 631, 652, 672, 695, 713, 726, 739, 757, 777, 796, 815, 835, 858, 878, 904, 922, 940, 949, 962, 979, 990, 1002, 1012, 1024, 1036, 1050, 1057, 1067, 1078, 1088, 1095, 1107, 1122, 1132, 1140, 1147, 1156, 1163, 1172, 1180, 1186, 1191, 1200, 1210, 1221, 1228, 1241, 1249, 1254, 1259, 1263, 1267, 1277, 1283, 1288, 1296, 1300, 1304, 1308, 1312, 1317, 1323, 1330, 1336, 1341, 1347, 1354, 1358, 1364, 1368, 1369, 1374, 1379, 1384, 1386, 1389], "harmed": [0, 3, 5, 13, 21, 23, 25, 29, 35, 41, 46, 50, 56, 61, 72, 79, 84, 89, 94, 102, 107, 112, 123, 129, 136, 138, 143, 149, 154, 162, 165, 171, 173, 178, 182, 184, 188, 188, 191, 196, 206, 210, 212, 217, 222, 228, 236, 239, 249, 252, 252, 255, 260, 264, 267, 272, 277, 278, 280, 282, 284, 287, 288, 291, 293, 293, 301, 305, 309, 313, 317, 320, 325, 328, 330, 333, 335, 337, 339, 342, 345, 347, 350, 351, 351, 353, 357, 358, 363, 365, 366, 368, 369, 371, 373, 375, 377, 381, 382, 384, 388], "exact_score": [75.56574047550845, 75.96677169865367, 76.39644800916643, 76.65425379547408, 76.94070466914924, 77.25580063019191, 77.6711543970209, 78.05786307648238, 78.32999140647378, 78.58779719278144, 78.8742480664566, 79.17502148381553, 79.51876253222572, 79.77656831853336, 80.02005156115726, 80.23488971641363, 80.56430822114007, 80.75050128902893, 80.99398453165281, 81.19450014322544, 81.49527356058435, 81.81036952162704, 81.92494987109711, 82.168433113721, 82.35462618160985, 82.62675451160125, 82.85591521054138, 83.05643082211401, 83.31423660842165, 83.45746204525923, 83.60068748209682, 83.70094528788312, 83.93010598682325, 84.14494414207962, 84.35978229733601, 84.60326553995989, 84.83242623890003, 85.16184474362647, 85.40532798625036, 85.70610140360928, 85.82068175307934, 86.02119736465195, 86.12145517043827, 86.23603551990833, 86.40790604411343, 86.47951876253222, 86.53680893726725, 86.63706674305357, 86.66571183042107, 86.79461472357491, 86.99513033514752, 87.05242050988255, 87.12403322830134, 87.22429103408766, 87.32454883987396, 87.35319392724148, 87.45345173302779, 87.6539673446004, 87.76854769407046, 87.85448295617302, 87.9260956745918, 88.01203093669436, 88.09796619879691, 88.18390146089945, 88.269836723002, 88.35577198510455, 88.31280435405327, 88.38441707247206, 88.47035233457461, 88.57061014036093, 88.6135777714122, 88.75680320824978, 88.79977083930106, 88.82841592666857, 88.87138355771985, 88.8857061014036, 88.91435118877112, 89.02893153824118, 89.08622171297623, 89.11486680034373, 89.18647951876252, 89.21512460613005, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.37267258665138, 89.4013176740189, 89.48725293612145, 89.50157547980521, 89.5445431108565, 89.61615582927529, 89.68776854769406, 89.73073617874535, 89.78802635348038, 89.8166714408479, 89.80234889716414, 89.84531652821542, 89.85963907189917, 89.9169292466342, 89.9169292466342, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.50838999100768, "threshold_area100": 85.50737610999713, "threshold_area25": 79.39129189344027}, "test": {"score": [76.42247255506925, 76.90320728994763, 77.18303795651862, 77.52026978546316, 77.90055248618785, 78.18038315275885, 78.46021381932985, 78.74004448590084, 78.98399942598839, 79.29970581904284, 79.622587357394, 79.86654229748153, 80.15354810934922, 80.31140130587644, 80.56253139126068, 80.84953720312836, 81.09349214321591, 81.25852048503982, 81.49530027983067, 81.71055463873144, 81.98321016000574, 82.19128937360982, 82.45676974958744, 82.72942527086174, 83.00208079213604, 83.16710913395997, 83.39671378345412, 83.65501901413504, 83.92049939011265, 84.19315491138696, 84.31513238143073, 84.6954150821554, 84.8030422616058, 85.19050010762719, 85.42010475712134, 85.71428571428572, 85.9151897825931, 86.14479443208725, 86.36004879098802, 86.55377771399871, 86.71163091052594, 86.89818468823994, 87.10626390184402, 87.25694195307454, 87.41479514960177, 87.5798234914257, 87.76637726913971, 87.93140561096362, 88.01033220922724, 88.07490851689747, 88.12513453397432, 88.18253569634786, 88.31168831168831, 88.39061490995192, 88.44084092702877, 88.46954150821554, 88.55564325177585, 88.67044557652292, 88.72784673889646, 88.83547391834685, 88.89287508072039, 89.00050226017078, 89.05072827724761, 89.17988089258809, 89.21575661907154, 89.28750807203846, 89.30903350792853, 89.36643467030207, 89.43818612326899, 89.51711272153261, 89.63191504627969, 89.75389251632346, 89.85434455047715, 89.91892085814737, 89.92609600344407, 89.96914687522423, 90.01219774700438, 90.13417521704815, 90.18440123412499, 90.1700509435316, 90.24897754179523, 90.277678122982, 90.37095501183899, 90.41400588361914, 90.45705675539931, 90.47858219128938, 90.49293248188276, 90.52880820836621, 90.54315849895961, 90.57903422544307, 90.57903422544307, 90.60055966133315, 90.62926024251992, 90.64361053311329, 90.69383655019014, 90.70818684078353, 90.71536198608023, 90.70818684078353, 90.70101169548683, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.2341249910310683, 2.367797947908445, 3.3292674176652075, 4.348138049795509, 5.359833536629117, 6.192150391045419, 7.124919279615412, 7.835258663987946, 8.796728133744708, 9.621869842864319, 10.38961038961039, 11.401305876443999, 12.097294970223146, 13.080289875869985, 13.991533328549904, 15.060629977757051, 15.878596541579967, 16.818540575446654, 17.75848460931334, 18.734304369663484, 19.710124130013632, 20.922723685154622, 21.877018009614694, 22.981990385305302, 23.656454043194376, 24.682499820621366, 25.87357393987228, 27.0574729138265, 28.262897323670806, 29.13826504986726, 30.28628829733802, 31.226232331204706, 32.65408624524647, 33.744708330343684, 34.86403099662768, 35.954653081724906, 36.930472842075055, 38.02109492717228, 39.19781875582981, 40.195163952070025, 41.27143574657387, 42.3548826863744, 43.2804764296477, 44.33522278826146, 45.28951711272153, 46.172059984214684, 47.23398148812513, 48.25285212025543, 49.042118102891585, 49.781158068450885, 50.64217550405395, 51.675396426777645, 52.35703522996341, 53.462007605654016, 54.581330271938015, 55.55715003228815, 56.697998134462225, 57.58771615125207, 58.549185621008824, 59.467604218985436, 60.52235057759919, 61.47664490205926, 62.52421611537634, 63.50721102102318, 64.42562961899978, 65.2435961828227, 66.11178876372247, 67.08043337877592, 67.92710052378561, 68.91009542943245, 69.7711128650355, 70.96218698428643, 71.93800674463657, 73.05732941092057, 74.0474994618641, 75.03766951280764, 76.00631412786109, 77.03235990528809, 78.17320800746215, 79.46473416086675, 80.69168400660114, 81.60292745928105, 82.70789983497166, 83.81287221066228, 85.02547176580326, 86.11609385090048, 86.99863672239363, 87.90270502977685, 89.05072827724761, 90.04807347348783, 90.94496663557437, 92.04993901126498, 93.00423333572505, 94.03027911315203, 94.99174858290881, 95.98191863385233, 96.99361412068595, 98.09141135107986, 99.12463227380354, 100.0], "rescued": [0, 73, 121, 175, 235, 280, 321, 371, 409, 466, 514, 555, 607, 635, 684, 732, 777, 810, 855, 900, 946, 987, 1037, 1085, 1133, 1168, 1207, 1259, 1308, 1357, 1386, 1445, 1472, 1537, 1579, 1626, 1664, 1705, 1750, 1790, 1823, 1856, 1891, 1915, 1944, 1974, 2005, 2036, 2054, 2070, 2082, 2099, 2120, 2136, 2153, 2170, 2187, 2213, 2224, 2244, 2257, 2277, 2290, 2315, 2329, 2343, 2356, 2368, 2381, 2395, 2417, 2435, 2452, 2465, 2472, 2480, 2492, 2509, 2521, 2531, 2546, 2557, 2573, 2586, 2597, 2607, 2614, 2622, 2628, 2637, 2641, 2649, 2660, 2665, 2674, 2680, 2686, 2690, 2696, 2702, 2706], "harmed": [0, 6, 15, 22, 29, 35, 37, 48, 52, 65, 68, 75, 87, 93, 107, 115, 126, 136, 148, 163, 171, 183, 196, 206, 216, 228, 235, 251, 263, 274, 286, 292, 304, 315, 325, 331, 341, 350, 365, 378, 389, 396, 402, 405, 412, 419, 424, 432, 439, 446, 451, 460, 463, 468, 478, 491, 496, 506, 509, 514, 519, 524, 530, 537, 546, 550, 560, 564, 567, 570, 576, 577, 580, 584, 590, 592, 598, 598, 603, 615, 619, 626, 629, 636, 641, 648, 653, 656, 660, 664, 668, 673, 680, 683, 685, 689, 694, 699, 706, 710, 711], "exact_score": [76.42247255506925, 76.81710554638732, 77.1112865035517, 77.41981775130947, 77.76422472555069, 78.07275597330846, 78.409987802253, 78.69699361412069, 79.01270000717516, 79.37145727200976, 79.73738968214107, 80.06027122049221, 80.27552557939298, 80.56970653655738, 80.84236205783168, 81.07914185262251, 81.30874650211666, 81.54552629690751, 81.78948123699506, 82.0262610317859, 82.27021597187343, 82.4782951854775, 82.76530099734521, 83.00925593743274, 83.23886058692689, 83.45411494582765, 83.67654445002512, 83.90614909951927, 84.11422831312333, 84.27208150965058, 84.6164884838918, 84.78151682571571, 85.00394632991318, 85.27660185118748, 85.43445504771472, 85.72863600487909, 85.92236492788979, 86.16631986797732, 86.35287364569133, 86.53942742340533, 86.68293032933917, 86.87665925234987, 87.02016215828371, 87.22824137188778, 87.38609456841502, 87.52959747434885, 87.75202697854631, 87.88835473918347, 88.00315706393054, 88.07490851689747, 88.1394848245677, 88.22558656812801, 88.34038889287508, 88.39779005524862, 88.44801607232546, 88.5197675252924, 88.59151897825932, 88.6847958671163, 88.7924230465667, 88.87134964483032, 88.92157566190716, 89.05790342254431, 89.12965487551124, 89.22293176436823, 89.23010690966493, 89.30903350792853, 89.36643467030207, 89.4238358326756, 89.52428786682931, 89.64626533687307, 89.76106766162016, 89.85434455047715, 89.91892085814737, 89.92609600344407, 89.96197172992754, 90.01937289230108, 90.13417521704815, 90.18440123412499, 90.18440123412499, 90.22745210590514, 90.25615268709191, 90.32072899476215, 90.37813015713569, 90.42118102891584, 90.46423190069599, 90.47858219128938, 90.49293248188276, 90.52880820836621, 90.54315849895961, 90.57903422544307, 90.57903422544307, 90.59338451603645, 90.6364353878166, 90.64361053311329, 90.69383655019014, 90.70818684078353, 90.71536198608023, 90.70818684078353, 90.70101169548683, 90.69383655019014, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.48142442815157, "threshold_area100": 86.41841859797661, "threshold_area25": 80.08610174356032}, "selected_budget": 98, "selected": {"val": {"score": 89.9169292466342, "calls": 97.99484388427385, "rescued": 1384, "harmed": 382}, "test": {"score": 90.70101169548683, "calls": 98.09141135107986, "rescued": 2696, "harmed": 706}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.77370380979661, "calls": 98.99742194213692}, "test": {"score": 90.6364353878166, "calls": 99.08875654732007}}, "old_normalized_interior": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692}, "test": {"score": 90.70818684078353, "calls": 98.81610102604577}}, "old_normalized_endpoints": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692}, "test": {"score": 90.70818684078353, "calls": 98.81610102604577}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 76.00973932970496, 76.32483529074764, 76.63993125179032, 76.98367230020051, 77.29876826124318, 77.62818676596963, 77.87167000859353, 78.18676596963621, 78.37295903752506, 78.71670008593526, 79.10340876539674, 79.40418218275566, 79.66198796906329, 79.99140647378974, 80.14895445431108, 80.30650243483242, 80.59295330850759, 80.93669435691778, 81.20882268690919, 81.43798338584932, 81.69578917215698, 82.11114293898596, 82.28301346319105, 82.49785161844744, 82.7126897737038, 82.87023775422514, 83.07075336579777, 83.2283013463191, 83.42881695789171, 83.6436551131481, 83.87281581208822, 84.13062159839588, 84.30249212260097, 84.41707247207104, 84.58894299627615, 84.83242623890003, 84.98997421942136, 85.16184474362647, 85.33371526783156, 85.4912632483529, 85.62016614150673, 85.9066170151819, 86.04984245201948, 86.19306788885706, 86.33629332569464, 86.49384130621598, 86.57977656831854, 86.72300200515612, 86.86622744199369, 86.86622744199369, 86.96648524778, 86.98080779146376, 87.09538814093382, 87.16700085935261, 87.33887138355772, 87.45345173302779, 87.53938699513033, 87.56803208249785, 87.66828988828415, 87.79719278143799, 87.81151532512173, 87.94041821827557, 88.02635348037812, 88.04067602406187, 88.05499856774563, 88.19822400458321, 88.31280435405327, 88.47035233457461, 88.5133199656259, 88.5419650529934, 88.62790031509596, 88.69951303351475, 88.77112575193354, 88.82841592666857, 88.85706101403609, 88.87138355771985, 88.95731881982239, 89.00028645087367, 89.01460899455743, 89.02893153824118, 89.0575766256087, 89.1291893440275, 89.21512460613005, 89.24376969349757, 89.3010598682326, 89.31538241191636, 89.35835004296763, 89.50157547980521, 89.5445431108565, 89.63047837295903, 89.6734460040103, 89.7450587224291, 89.80234889716414, 89.7450587224291, 89.80234889716414, 89.84531652821542, 89.87396161558291, 89.84531652821542, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 76.00973932970496, 76.32483529074764, 76.63993125179032, 76.98367230020051, 77.29876826124318, 77.62818676596963, 77.87167000859353, 78.18676596963621, 78.37295903752506, 78.71670008593526, 79.10340876539674, 79.40418218275566, 79.66198796906329, 79.99140647378974, 80.14895445431108, 80.30650243483242, 80.59295330850759, 80.93669435691778, 81.20882268690919, 81.43798338584932, 81.69578917215698, 82.11114293898596, 82.28301346319105, 82.49785161844744, 82.7126897737038, 82.87023775422514, 83.07075336579777, 83.2283013463191, 83.42881695789171, 83.6436551131481, 83.87281581208822, 84.13062159839588, 84.30249212260097, 84.41707247207104, 84.58894299627615, 84.83242623890003, 84.98997421942136, 85.16184474362647, 85.33371526783156, 85.4912632483529, 85.62016614150673, 85.9066170151819, 86.04984245201948, 86.19306788885706, 86.33629332569464, 86.49384130621598, 86.57977656831854, 86.72300200515612, 86.86622744199369, 86.86622744199369, 86.96648524778, 86.98080779146376, 87.09538814093382, 87.16700085935261, 87.33887138355772, 87.45345173302779, 87.53938699513033, 87.56803208249785, 87.66828988828415, 87.79719278143799, 87.81151532512173, 87.94041821827557, 88.02635348037812, 88.04067602406187, 88.05499856774563, 88.19822400458321, 88.31280435405327, 88.47035233457461, 88.5133199656259, 88.5419650529934, 88.62790031509596, 88.69951303351475, 88.77112575193354, 88.82841592666857, 88.85706101403609, 88.87138355771985, 88.95731881982239, 89.00028645087367, 89.01460899455743, 89.02893153824118, 89.0575766256087, 89.1291893440275, 89.21512460613005, 89.24376969349757, 89.3010598682326, 89.31538241191636, 89.35835004296763, 89.50157547980521, 89.5445431108565, 89.63047837295903, 89.6734460040103, 89.7450587224291, 89.80234889716414, 89.7450587224291, 89.80234889716414, 89.84531652821542, 89.87396161558291, 89.84531652821542, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.348324262389, "exact_area100": 85.43633315475624}, "test": {"score": [76.42247255506925, 76.8314558369807, 77.21173853770539, 77.46286862308962, 77.79292530673746, 78.04405539212169, 78.3238860586927, 78.61806701585708, 78.89072253713138, 79.156202913109, 79.34993183611968, 79.67281337447083, 79.8880677333716, 80.23247470761284, 80.4405539212169, 80.74191002367799, 81.00021525435889, 81.25134533974312, 81.55270144220421, 81.76078065580829, 81.96885986941236, 82.18411422831312, 82.40654373251058, 82.72942527086174, 82.98773050154266, 83.18863456985004, 83.45411494582765, 83.74112075769534, 83.94202482600272, 84.11422831312333, 84.32948267202411, 84.60213819329842, 84.91066944105619, 85.14027409055034, 85.37705388534118, 85.59230824424195, 85.72863600487909, 85.93671521848317, 86.14479443208725, 86.36004879098802, 86.6542297481524, 86.79773265408625, 86.96993614120686, 87.09908875654733, 87.25694195307454, 87.38609456841502, 87.52959747434885, 87.65875008968932, 87.78790270502978, 87.81660328621655, 87.9529310468537, 88.0533830810074, 88.10360909808423, 88.11795938867762, 88.23276171342471, 88.27581258520485, 88.33321374757838, 88.46954150821554, 88.52694267058908, 88.67044557652292, 88.71349644830309, 88.76372246537993, 88.84264906364353, 88.92157566190716, 89.05072827724761, 89.17270574729139, 89.23010690966493, 89.30185836263185, 89.35925952500538, 89.45253641386238, 89.48841214034584, 89.56733873860946, 89.65344048216977, 89.69649135394992, 89.76824280691685, 89.8184688239937, 89.92609600344407, 89.94762143933414, 90.00502260170768, 90.05524861878453, 90.11264978115807, 90.1700509435316, 90.22745210590514, 90.25615268709191, 90.29202841357538, 90.37095501183899, 90.38530530243239, 90.42118102891584, 90.46423190069599, 90.46423190069599, 90.48575733658608, 90.50010762717945, 90.54315849895961, 90.59338451603645, 90.61490995192652, 90.62926024251992, 90.65796082370669, 90.65796082370669, 90.65078567840999, 90.70818684078353, 90.73688742197031], "calls": [0.0, 1.0977972303939154, 2.202769606084523, 3.1068379134677477, 4.16158427208151, 5.080002870058118, 5.948195450957882, 6.8737891942311835, 7.820908373394561, 8.768027552557939, 9.478366936930472, 10.368084953720313, 11.099949773982923, 12.118820406113223, 12.893736098155989, 13.869555858506136, 14.795149601779437, 15.713568199756045, 16.83289086604004, 17.873286934060413, 18.54057544665279, 19.559446078783097, 20.563966420320014, 21.776565975461004, 22.84566262466815, 23.77125636794145, 25.034081940159286, 26.174930042333354, 27.416230178661117, 28.241371887780726, 29.20284135753749, 30.422616057975173, 31.649565903709547, 32.632560809356384, 33.61555571500323, 34.60572576594676, 35.56719523570352, 36.413862380713205, 37.4470833034369, 38.63098227739112, 39.95120901198249, 40.79787615699218, 41.680419028485325, 42.74951567769247, 43.81143718160293, 44.686804907799385, 45.69850039463299, 46.78194733443352, 47.729066513596905, 48.72641170983712, 49.70223147018727, 50.70675181172418, 51.46731721317357, 52.44313697352372, 53.73466312692832, 54.55980483604793, 55.485398579321235, 56.389466886704454, 57.1859080146373, 58.319580971514675, 59.295400731864824, 60.12771758628113, 60.823706680060276, 61.91432876515749, 63.069527157924945, 64.17449953361556, 65.22207074693263, 66.05438760134894, 66.91540503695201, 68.19975604505991, 69.33342900193729, 70.37382506995766, 71.22766736026405, 72.17478653942743, 72.97840281265697, 73.98292315419387, 74.99461864102747, 75.88433665781731, 77.03235990528809, 78.16603286216545, 79.29253067374614, 80.15354810934922, 81.1006672885126, 82.34196742484036, 83.43976465523427, 84.5375618856282, 85.49903135538494, 86.66858003874577, 87.6731003802827, 88.67762072181962, 89.59603931979623, 90.62208509722322, 91.57637942168328, 92.71005237856068, 93.7934993183612, 94.66169189926096, 95.86711630910526, 96.77835976178517, 97.69677835976178, 98.81610102604577, 100.0], "exact_score": [76.42247255506925, 76.80275525579393, 77.13281193944178, 77.46286862308962, 77.71399870847385, 78.036880246825, 78.35258663987946, 78.66111788763723, 78.94094855420822, 79.19925378488915, 79.53648561383369, 79.8880677333716, 80.20377412642607, 80.48360479299707, 80.75626031427137, 81.04326612613906, 81.31592164741336, 81.60292745928105, 81.81818181818183, 82.06213675826936, 82.2989165530602, 82.53569634785106, 82.78682643323528, 83.03078137332281, 83.21016000574012, 83.45411494582765, 83.6908947406185, 83.8774485183325, 84.0568271507498, 84.22903063787042, 84.48733586855134, 84.73846595393557, 84.99677118461649, 85.21920068881394, 85.49185621008826, 85.60665853483533, 85.8290880390328, 86.08021812441702, 86.27394704742771, 86.46050082514172, 86.66140489344909, 86.8407835258664, 86.99863672239363, 87.13496448303079, 87.26411709837124, 87.4004448590084, 87.54394776494225, 87.6802755255794, 87.79507785032648, 87.83812872210662, 87.98163162804047, 88.08925880749085, 88.11795938867762, 88.1394848245677, 88.24711200401808, 88.30451316639162, 88.40496520054532, 88.5125923799957, 88.62739470474277, 88.67044557652292, 88.76372246537993, 88.87134964483032, 88.92875080720385, 89.03637798665423, 89.1440051661046, 89.23010690966493, 89.29468321733516, 89.36643467030207, 89.4238358326756, 89.47406184975246, 89.55298844801607, 89.63909019157639, 89.69649135394992, 89.76824280691685, 89.8184688239937, 89.92609600344407, 89.96197172992754, 89.997847456411, 90.06242376408122, 90.09112434526799, 90.1700509435316, 90.22027696060846, 90.24897754179523, 90.26332783238861, 90.33507928535553, 90.37095501183899, 90.39248044772907, 90.44988161010261, 90.47140704599268, 90.47140704599268, 90.50010762717945, 90.51445791777284, 90.5503336442563, 90.58620937073975, 90.64361053311329, 90.62208509722322, 90.66513596900337, 90.67231111430006, 90.65796082370669, 90.71536198608023, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 79.98206213675829, "exact_area100": 86.38820957596013}}, "old_normalized_endpoint_budget": 99}, {"seed": 2029, "selection": {"best_epoch": 5, "epochs": 13, "alpha": 1.0, "validation_area100": 0.8550680634681921, "training_seconds": 38.44985482795164, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7069262843334879, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6399100784786743, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5736518379618832, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5205850704057791, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4703972859175223, "operator": "ge"}, {"budget": 0.06, "threshold": 0.43284158053746624, "operator": "ge"}, {"budget": 0.07, "threshold": 0.3983865425438262, "operator": "ge"}, {"budget": 0.08, "threshold": 0.36789086606466703, "operator": "ge"}, {"budget": 0.09, "threshold": 0.3318853412865212, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3001458760722544, "operator": "ge"}, {"budget": 0.11, "threshold": 0.2670632657661273, "operator": "ge"}, {"budget": 0.12, "threshold": 0.23148062240245487, "operator": "ge"}, {"budget": 0.13, "threshold": 0.1982082513733733, "operator": "ge"}, {"budget": 0.14, "threshold": 0.16609314438743367, "operator": "ge"}, {"budget": 0.15, "threshold": 0.13596937348355934, "operator": "ge"}, {"budget": 0.16, "threshold": 0.11072993435315846, "operator": "ge"}, {"budget": 0.17, "threshold": 0.08098156091596719, "operator": "ge"}, {"budget": 0.18, "threshold": 0.05034521481211284, "operator": "ge"}, {"budget": 0.19, "threshold": 0.015109371750251477, "operator": "ge"}, {"budget": 0.2, "threshold": -0.01391818288009837, "operator": "ge"}, {"budget": 0.21, "threshold": -0.03965587301965075, "operator": "ge"}, {"budget": 0.22, "threshold": -0.06854535356699461, "operator": "ge"}, {"budget": 0.23, "threshold": -0.10391215398686249, "operator": "ge"}, {"budget": 0.24, "threshold": -0.1472918304623766, "operator": "ge"}, {"budget": 0.25, "threshold": -0.18602615295040095, "operator": "ge"}, {"budget": 0.26, "threshold": -0.21750126791218383, "operator": "ge"}, {"budget": 0.27, "threshold": -0.2571534513363113, "operator": "ge"}, {"budget": 0.28, "threshold": -0.30191916870230195, "operator": "ge"}, {"budget": 0.29, "threshold": -0.3403995910831462, "operator": "ge"}, {"budget": 0.3, "threshold": -0.37829176855609015, "operator": "ge"}, {"budget": 0.31, "threshold": -0.4189564632158599, "operator": "ge"}, {"budget": 0.32, "threshold": -0.45690355591507237, "operator": "ge"}, {"budget": 0.33, "threshold": -0.4979854533308063, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5364240427778268, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5710642615257184, "operator": "ge"}, {"budget": 0.36, "threshold": -0.6056136680615701, "operator": "ge"}, {"budget": 0.37, "threshold": -0.6507060647828801, "operator": "ge"}, {"budget": 0.38, "threshold": -0.6796980467520395, "operator": "ge"}, {"budget": 0.39, "threshold": -0.7106308187170062, "operator": "ge"}, {"budget": 0.4, "threshold": -0.7485833450617793, "operator": "ge"}, {"budget": 0.41, "threshold": -0.7744333015146151, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8039932748060341, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8255656920427885, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8513205242007341, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8722989716001679, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8897003660497871, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9049748681424981, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9203678033294764, "operator": "ge"}, {"budget": 0.49, "threshold": -0.931549878746776, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9416535398992584, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9489208166954831, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9562458137991944, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9615686045989713, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9667059475072822, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9708536980962279, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9744442070102234, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9775804897111201, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9800905590885993, "operator": "ge"}, {"budget": 0.59, "threshold": -0.982404437669458, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9847046772690489, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9868716823509764, "operator": "ge"}, {"budget": 0.62, "threshold": -0.988409484456284, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9898991498283094, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9909371626949565, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9919008308191889, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9928406838595486, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9936251046320147, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9942933246516981, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9948776891782397, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9953897081089524, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9958473214220157, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9962992719665668, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9966828346816642, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9971328405128006, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9973811228374819, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9976788592088534, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9979549444860217, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9981804612971805, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9983645542758083, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9985167478603846, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9986611454971452, "operator": "ge"}, {"budget": 0.82, "threshold": -0.998812391770449, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9989283892528603, "operator": "ge"}, {"budget": 0.84, "threshold": -0.999042131862897, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9991423120618863, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9992278977623252, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9993213599002129, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9993926444855812, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9994688546920029, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9995387388540681, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9996016379941608, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9996702897267666, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9997245804419007, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9997754541993896, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9998191133470838, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9998557279752776, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9998974034524637, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9999306243492979, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9999548796893659, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.35348037811515, 76.75451160126038, 77.16986536808938, 77.42767115439702, 77.7427671154397, 78.10083070753365, 78.38728158120882, 78.71670008593526, 79.00315095961042, 79.3182469206531, 79.6333428816958, 79.84818103695216, 80.12030936694356, 80.39243769693498, 80.67888857061014, 80.9653394442853, 81.20882268690919, 81.32340303637926, 81.63849899742193, 81.753079346892, 81.98224004583213, 82.0968203953022, 82.22572328845604, 82.45488398739616, 82.6124319679175, 82.92752792896017, 83.05643082211401, 83.15668862790031, 83.52907476367803, 83.81552563735319, 83.9873961615583, 84.1879117731309, 84.35978229733601, 84.64623317101116, 84.8037811515325, 84.96132913205385, 85.17616728731022, 85.33371526783156, 85.51990833572042, 85.8063592093956, 85.9352621025494, 86.09281008307075, 86.23603551990833, 86.3506158693784, 86.42222858779719, 86.60842165568606, 86.75164709252363, 86.89487252936121, 87.02377542251504, 87.05242050988255, 87.05242050988255, 87.15267831566887, 87.2099684904039, 87.29590375250645, 87.35319392724148, 87.42480664566027, 87.5823546261816, 87.66828988828415, 87.73990260670294, 87.85448295617302, 87.98338584932684, 88.14093382984818, 88.269836723002, 88.29848181036952, 88.38441707247206, 88.42738470352333, 88.5419650529934, 88.62790031509596, 88.74248066456603, 88.81409338298482, 88.85706101403609, 88.90002864508737, 88.91435118877112, 89.00028645087367, 89.02893153824118, 89.08622171297623, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.47293039243769, 89.48725293612145, 89.51589802348897, 89.45860784875393, 89.50157547980521, 89.5445431108565, 89.55886565454024, 89.60183328559151, 89.6448009166428, 89.61615582927529, 89.6734460040103, 89.73073617874535, 89.75938126611285, 89.84531652821542, 89.84531652821542, 89.87396161558291, 89.9169292466342, 89.93125179031796, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 34, 62, 95, 125, 150, 175, 205, 230, 257, 282, 307, 333, 354, 378, 402, 426, 453, 477, 495, 523, 542, 565, 584, 602, 622, 641, 665, 683, 695, 723, 748, 763, 780, 796, 817, 834, 848, 870, 886, 901, 925, 936, 951, 964, 976, 987, 1005, 1018, 1029, 1042, 1051, 1056, 1066, 1073, 1083, 1091, 1102, 1114, 1124, 1137, 1148, 1160, 1172, 1183, 1189, 1198, 1205, 1215, 1225, 1235, 1240, 1245, 1250, 1253, 1261, 1265, 1271, 1278, 1285, 1290, 1296, 1308, 1313, 1320, 1324, 1331, 1334, 1338, 1343, 1349, 1350, 1356, 1362, 1367, 1375, 1377, 1380, 1384, 1387, 1389], "harmed": [0, 5, 7, 12, 13, 20, 23, 28, 33, 37, 42, 45, 49, 55, 60, 65, 69, 76, 83, 93, 99, 110, 117, 128, 137, 141, 149, 151, 160, 165, 167, 172, 175, 178, 182, 183, 189, 192, 199, 204, 206, 210, 212, 216, 219, 223, 229, 234, 237, 238, 242, 249, 254, 257, 260, 264, 268, 274, 275, 279, 287, 290, 293, 294, 296, 300, 303, 307, 309, 313, 315, 315, 317, 319, 321, 323, 325, 327, 329, 331, 333, 335, 337, 341, 346, 354, 358, 358, 361, 363, 366, 369, 371, 373, 376, 378, 380, 381, 382, 384, 388], "exact_score": [75.56574047550845, 75.98109424233743, 76.35348037811515, 76.75451160126038, 77.16986536808938, 77.42767115439702, 77.7427671154397, 78.10083070753365, 78.38728158120882, 78.71670008593526, 79.00315095961042, 79.3182469206531, 79.6333428816958, 79.84818103695216, 80.12030936694356, 80.39243769693498, 80.67888857061014, 80.9653394442853, 81.20882268690919, 81.32340303637926, 81.63849899742193, 81.753079346892, 81.98224004583213, 82.0968203953022, 82.22572328845604, 82.45488398739616, 82.6124319679175, 82.92752792896017, 83.05643082211401, 83.15668862790031, 83.52907476367803, 83.81552563735319, 83.9873961615583, 84.1879117731309, 84.35978229733601, 84.64623317101116, 84.8037811515325, 84.96132913205385, 85.17616728731022, 85.33371526783156, 85.51990833572042, 85.8063592093956, 85.9352621025494, 86.09281008307075, 86.23603551990833, 86.3506158693784, 86.42222858779719, 86.60842165568606, 86.75164709252363, 86.89487252936121, 87.02377542251504, 87.05242050988255, 87.05242050988255, 87.15267831566887, 87.2099684904039, 87.29590375250645, 87.35319392724148, 87.42480664566027, 87.5823546261816, 87.66828988828415, 87.73990260670294, 87.85448295617302, 87.98338584932684, 88.14093382984818, 88.269836723002, 88.29848181036952, 88.38441707247206, 88.42738470352333, 88.5419650529934, 88.62790031509596, 88.74248066456603, 88.81409338298482, 88.85706101403609, 88.90002864508737, 88.91435118877112, 89.00028645087367, 89.02893153824118, 89.08622171297623, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.47293039243769, 89.48725293612145, 89.51589802348897, 89.45860784875393, 89.50157547980521, 89.5445431108565, 89.55886565454024, 89.60183328559151, 89.6448009166428, 89.61615582927529, 89.6734460040103, 89.73073617874535, 89.75938126611285, 89.84531652821542, 89.84531652821542, 89.87396161558291, 89.9169292466342, 89.93125179031796, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.5068063468192, "threshold_area100": 85.50580063019191, "threshold_area25": 79.49441420796333}, "test": {"score": [76.42247255506925, 76.87450670876086, 77.18303795651862, 77.54179522135324, 77.8646767597044, 78.19473344335222, 78.47456410992322, 78.73286934060415, 78.89072253713138, 79.19925378488915, 79.49343474205352, 79.68716366506422, 80.01004520341537, 80.34010188706321, 80.67733371600775, 80.92128865609529, 81.1006672885126, 81.38767310038028, 81.68902920284135, 81.96168472411567, 82.16976393771974, 82.44241945899404, 82.6720241084882, 82.90162875798235, 83.21016000574012, 83.36801320226735, 83.56891727057473, 83.81287221066228, 84.10705316782665, 84.3366578173208, 84.48016072325466, 84.78869197101243, 84.9034942957595, 85.0828729281768, 85.38422903063787, 85.60665853483533, 85.83626318432948, 86.09456841501041, 86.33134820980125, 86.5681280045921, 86.79773265408625, 86.92688526942672, 87.07756332065725, 87.15648991892087, 87.29281767955801, 87.38609456841502, 87.52959747434885, 87.60852407261247, 87.78072755973308, 87.88117959388678, 87.99598191863386, 88.14665996986439, 88.21123627753462, 88.29733802109493, 88.36908947406185, 88.41931549113869, 88.5197675252924, 88.61304441414939, 88.72784673889646, 88.7924230465667, 88.84264906364353, 88.90005022601709, 88.95027624309392, 89.065078567841, 89.11530458491785, 89.16553060199469, 89.23010690966493, 89.2659826361484, 89.28033292674176, 89.33055894381862, 89.40948554208224, 89.49558728564253, 89.56733873860946, 89.65344048216977, 89.79694338810361, 89.8902202769606, 89.98349716581761, 90.01937289230108, 90.10547463586138, 90.0982994905647, 90.14852550764154, 90.18440123412499, 90.20592667001507, 90.22027696060846, 90.24897754179523, 90.29202841357538, 90.29920355887207, 90.34942957594892, 90.38530530243239, 90.46423190069599, 90.51445791777284, 90.52163306306953, 90.52880820836621, 90.55750878955298, 90.58620937073975, 90.60773480662984, 90.65078567840999, 90.69383655019014, 90.70101169548683, 90.7297122766736, 90.73688742197031], "calls": [0.0, 1.0977972303939154, 1.973164956590371, 2.9920355887206718, 3.9319796225873573, 5.015426562387888, 5.926670015067805, 6.809212886560953, 7.447800817966564, 8.459496304800172, 9.399440338666858, 10.210231757193084, 11.286503551696923, 12.283848747937146, 13.2524933629906, 14.156561670373824, 14.909951926526514, 16.036449738107198, 17.06249551553419, 18.23921934419172, 19.437468608739326, 20.43481380497955, 21.367582693549544, 22.40080361627323, 23.570352299634067, 24.603573222357753, 25.44306522207075, 26.426060127717587, 27.674535409342038, 28.621654588505418, 29.640525220635716, 30.66657099806271, 31.513238143072396, 32.51058333931262, 33.64425629619, 34.59855062065007, 35.610246107483675, 36.887421970294895, 37.74843940589797, 38.70273373035804, 40.0373107555428, 40.89832819114587, 41.945899404462935, 42.79974169476932, 44.01234124991031, 44.99533615555715, 45.84917844586353, 46.8967496591806, 48.06629834254144, 48.963191504627964, 49.83855923082442, 50.73545239291095, 51.76149817033795, 52.608165315347634, 53.46918275095071, 54.35172562244386, 55.3347205280907, 56.1598622372103, 57.05675539929683, 57.881897108416446, 58.97969433881036, 60.16359331276458, 61.117887637224655, 62.194159431728494, 62.98342541436463, 63.92336944823133, 64.93506493506493, 65.982636148382, 67.05173279758915, 68.01320226734592, 68.93162086532251, 69.86438975389252, 70.78280835186914, 71.93800674463657, 73.41608667575518, 74.2340532395781, 75.29597474348856, 76.47987371744279, 77.64942240080362, 78.73286934060415, 79.622587357394, 80.59123197244745, 81.54552629690751, 82.45676974958744, 83.39671378345412, 84.40840927028772, 85.27660185118748, 86.43897538925164, 87.12778933773409, 88.26146229461148, 89.31620865322523, 90.50728277247615, 91.55485398579322, 92.62395063500037, 93.61412068594389, 94.63299131807419, 95.62316136901772, 96.79988519767525, 97.86898184688239, 98.74434957307886, 100.0], "rescued": [0, 66, 116, 173, 220, 274, 321, 366, 391, 445, 495, 533, 588, 639, 693, 740, 774, 829, 879, 934, 973, 1023, 1064, 1110, 1166, 1204, 1237, 1281, 1334, 1373, 1409, 1457, 1489, 1525, 1575, 1615, 1651, 1697, 1740, 1776, 1822, 1852, 1880, 1902, 1927, 1948, 1975, 1992, 2024, 2043, 2069, 2095, 2113, 2128, 2143, 2156, 2178, 2195, 2215, 2228, 2240, 2259, 2272, 2291, 2305, 2321, 2334, 2345, 2354, 2367, 2382, 2399, 2417, 2435, 2457, 2471, 2487, 2499, 2518, 2525, 2535, 2546, 2554, 2561, 2570, 2580, 2587, 2598, 2605, 2619, 2630, 2636, 2640, 2647, 2655, 2663, 2673, 2688, 2692, 2699, 2706], "harmed": [0, 3, 10, 17, 19, 27, 35, 44, 47, 58, 67, 78, 88, 93, 100, 113, 122, 137, 145, 162, 172, 184, 193, 207, 220, 236, 241, 251, 263, 270, 286, 291, 307, 318, 326, 335, 339, 349, 359, 362, 376, 388, 395, 406, 412, 420, 427, 433, 441, 446, 456, 461, 470, 473, 478, 484, 492, 496, 500, 504, 509, 520, 526, 529, 536, 545, 549, 555, 562, 568, 572, 577, 585, 591, 593, 594, 597, 604, 611, 619, 622, 628, 633, 638, 643, 647, 653, 657, 659, 662, 666, 671, 674, 677, 681, 686, 690, 699, 702, 705, 711], "exact_score": [76.42247255506925, 76.8314558369807, 77.197388247112, 77.54179522135324, 77.89337734089116, 78.20190858864893, 78.50326469110999, 78.76874506708761, 79.04857573365861, 79.41450814378992, 79.6082370668006, 79.90959316926168, 80.23964985290952, 80.60558226304083, 80.92128865609529, 81.11501757910598, 81.36614766449021, 81.67467891224798, 81.94015928822559, 82.10518763004951, 82.2989165530602, 82.58592236492788, 82.80835186912535, 83.06665709980628, 83.28908660400374, 83.45411494582765, 83.71959532180527, 83.97790055248619, 84.19315491138696, 84.44428499677119, 84.58061275740833, 84.83174284279256, 84.94654516753964, 85.19767525292387, 85.48468106479157, 85.69993542369232, 85.93671521848317, 86.12326899619718, 86.41027480806487, 86.60400373107555, 86.81208294467963, 86.94841070531677, 87.08473846595393, 87.19236564540432, 87.29281767955801, 87.38609456841502, 87.54394776494225, 87.63004950850255, 87.7735524144364, 87.88117959388678, 88.03185764511731, 88.14665996986439, 88.23276171342471, 88.29733802109493, 88.40496520054532, 88.5197675252924, 88.56999354236923, 88.71349644830309, 88.79959819186338, 88.84264906364353, 88.91440051661046, 88.94310109779724, 89.04355313195093, 89.10812943962115, 89.17270574729139, 89.23010690966493, 89.27315778144508, 89.28033292674176, 89.33055894381862, 89.40948554208224, 89.51711272153261, 89.58168902920285, 89.66061562746646, 89.74671737102676, 89.86869484107054, 89.98349716581761, 90.00502260170768, 90.04807347348785, 90.11264978115807, 90.11264978115807, 90.1700509435316, 90.19875152471838, 90.21310181531177, 90.23462725120184, 90.29920355887207, 90.30637870416876, 90.32072899476215, 90.38530530243239, 90.44988161010261, 90.50010762717945, 90.52163306306953, 90.52880820836621, 90.52163306306953, 90.57185908014638, 90.59338451603645, 90.62926024251992, 90.68666140489346, 90.69383655019014, 90.69383655019014, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.43880060545109, "threshold_area100": 86.37056037884769, "threshold_area25": 80.13489273157782}, "selected_budget": 99, "selected": {"val": {"score": 89.93125179031796, "calls": 98.99742194213692, "rescued": 1387, "harmed": 384}, "test": {"score": 90.7297122766736, "calls": 98.74434957307886, "rescued": 2699, "harmed": 705}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.44174715876579285, "operator": "ge"}, {"budget": 0.02, "threshold": 0.370986670255661, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3181401491165161, "operator": "ge"}, {"budget": 0.04, "threshold": 0.28240782022476196, "operator": "ge"}, {"budget": 0.05, "threshold": 0.24824678897857666, "operator": "ge"}, {"budget": 0.06, "threshold": 0.22292497754096985, "operator": "ge"}, {"budget": 0.07, "threshold": 0.19398784637451172, "operator": "ge"}, {"budget": 0.08, "threshold": 0.16748127341270447, "operator": "ge"}, {"budget": 0.09, "threshold": 0.15200695395469666, "operator": "ge"}, {"budget": 0.1, "threshold": 0.1296093761920929, "operator": "ge"}, {"budget": 0.11, "threshold": 0.1113237738609314, "operator": "ge"}, {"budget": 0.12, "threshold": 0.09368545562028885, "operator": "ge"}, {"budget": 0.13, "threshold": 0.08167165517807007, "operator": "ge"}, {"budget": 0.14, "threshold": 0.06767672300338745, "operator": "ge"}, {"budget": 0.15, "threshold": 0.055825695395469666, "operator": "ge"}, {"budget": 0.16, "threshold": 0.041486337780952454, "operator": "ge"}, {"budget": 0.17, "threshold": 0.02746446430683136, "operator": "ge"}, {"budget": 0.18, "threshold": 0.014469146728515625, "operator": "ge"}, {"budget": 0.19, "threshold": 0.005531273782253265, "operator": "ge"}, {"budget": 0.2, "threshold": -0.0016693053767085075, "operator": "ge"}, {"budget": 0.21, "threshold": -0.0027391952462494373, "operator": "ge"}, {"budget": 0.22, "threshold": -0.0038344089407473803, "operator": "ge"}, {"budget": 0.23, "threshold": -0.004735791590064764, "operator": "ge"}, {"budget": 0.24, "threshold": -0.005607517436146736, "operator": "ge"}, {"budget": 0.25, "threshold": -0.006498164962977171, "operator": "ge"}, {"budget": 0.26, "threshold": -0.00719708064571023, "operator": "ge"}, {"budget": 0.27, "threshold": -0.007917433977127075, "operator": "ge"}, {"budget": 0.28, "threshold": -0.008672141470015049, "operator": "ge"}, {"budget": 0.29, "threshold": -0.009580261074006557, "operator": "ge"}, {"budget": 0.3, "threshold": -0.010269257239997387, "operator": "ge"}, {"budget": 0.31, "threshold": -0.011006507091224194, "operator": "ge"}, {"budget": 0.32, "threshold": -0.012006902135908604, "operator": "ge"}, {"budget": 0.33, "threshold": -0.012747131288051605, "operator": "ge"}, {"budget": 0.34, "threshold": -0.01352767925709486, "operator": "ge"}, {"budget": 0.35, "threshold": -0.014265047386288643, "operator": "ge"}, {"budget": 0.36, "threshold": -0.014978672377765179, "operator": "ge"}, {"budget": 0.37, "threshold": -0.01573834754526615, "operator": "ge"}, {"budget": 0.38, "threshold": -0.016579782590270042, "operator": "ge"}, {"budget": 0.39, "threshold": -0.017280783504247665, "operator": "ge"}, {"budget": 0.4, "threshold": -0.01799611747264862, "operator": "ge"}, {"budget": 0.41, "threshold": -0.01890011504292488, "operator": "ge"}, {"budget": 0.42, "threshold": -0.019778870046138763, "operator": "ge"}, {"budget": 0.43, "threshold": -0.02069132775068283, "operator": "ge"}, {"budget": 0.44, "threshold": -0.021669667214155197, "operator": "ge"}, {"budget": 0.45, "threshold": -0.0226052887737751, "operator": "ge"}, {"budget": 0.46, "threshold": -0.0234666895121336, "operator": "ge"}, {"budget": 0.47, "threshold": -0.024529054760932922, "operator": "ge"}, {"budget": 0.48, "threshold": -0.02564431168138981, "operator": "ge"}, {"budget": 0.49, "threshold": -0.02656453661620617, "operator": "ge"}, {"budget": 0.5, "threshold": -0.027502045035362244, "operator": "ge"}, {"budget": 0.51, "threshold": -0.028523974120616913, "operator": "ge"}, {"budget": 0.52, "threshold": -0.029816418886184692, "operator": "ge"}, {"budget": 0.53, "threshold": -0.031022312119603157, "operator": "ge"}, {"budget": 0.54, "threshold": -0.031922124326229095, "operator": "ge"}, {"budget": 0.55, "threshold": -0.03317449241876602, "operator": "ge"}, {"budget": 0.56, "threshold": -0.03425666689872742, "operator": "ge"}, {"budget": 0.57, "threshold": -0.03538912534713745, "operator": "ge"}, {"budget": 0.58, "threshold": -0.03691762313246727, "operator": "ge"}, {"budget": 0.59, "threshold": -0.03824945539236069, "operator": "ge"}, {"budget": 0.6, "threshold": -0.03941546753048897, "operator": "ge"}, {"budget": 0.61, "threshold": -0.04078070446848869, "operator": "ge"}, {"budget": 0.62, "threshold": -0.0419003963470459, "operator": "ge"}, {"budget": 0.63, "threshold": -0.04319772496819496, "operator": "ge"}, {"budget": 0.64, "threshold": -0.04449520632624626, "operator": "ge"}, {"budget": 0.65, "threshold": -0.045778412371873856, "operator": "ge"}, {"budget": 0.66, "threshold": -0.04764179512858391, "operator": "ge"}, {"budget": 0.67, "threshold": -0.049162283539772034, "operator": "ge"}, {"budget": 0.68, "threshold": -0.05087859928607941, "operator": "ge"}, {"budget": 0.69, "threshold": -0.05260346084833145, "operator": "ge"}, {"budget": 0.7, "threshold": -0.05414769798517227, "operator": "ge"}, {"budget": 0.71, "threshold": -0.05608813464641571, "operator": "ge"}, {"budget": 0.72, "threshold": -0.05794382467865944, "operator": "ge"}, {"budget": 0.73, "threshold": -0.05966620519757271, "operator": "ge"}, {"budget": 0.74, "threshold": -0.061905208975076675, "operator": "ge"}, {"budget": 0.75, "threshold": -0.06399639695882797, "operator": "ge"}, {"budget": 0.76, "threshold": -0.06583628803491592, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0679098442196846, "operator": "ge"}, {"budget": 0.78, "threshold": -0.07034799456596375, "operator": "ge"}, {"budget": 0.79, "threshold": -0.07250462472438812, "operator": "ge"}, {"budget": 0.8, "threshold": -0.07519977539777756, "operator": "ge"}, {"budget": 0.81, "threshold": -0.07801888138055801, "operator": "ge"}, {"budget": 0.82, "threshold": -0.08092060685157776, "operator": "ge"}, {"budget": 0.83, "threshold": -0.08420781791210175, "operator": "ge"}, {"budget": 0.84, "threshold": -0.08696140348911285, "operator": "ge"}, {"budget": 0.85, "threshold": -0.09085702896118164, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0944456160068512, "operator": "ge"}, {"budget": 0.87, "threshold": -0.0977877676486969, "operator": "ge"}, {"budget": 0.88, "threshold": -0.10152147710323334, "operator": "ge"}, {"budget": 0.89, "threshold": -0.1054815948009491, "operator": "ge"}, {"budget": 0.9, "threshold": -0.11081470549106598, "operator": "ge"}, {"budget": 0.91, "threshold": -0.11569221317768097, "operator": "ge"}, {"budget": 0.92, "threshold": -0.124468594789505, "operator": "ge"}, {"budget": 0.93, "threshold": -0.13173222541809082, "operator": "ge"}, {"budget": 0.94, "threshold": -0.14143061637878418, "operator": "ge"}, {"budget": 0.95, "threshold": -0.15081898868083954, "operator": "ge"}, {"budget": 0.96, "threshold": -0.1585751175880432, "operator": "ge"}, {"budget": 0.97, "threshold": -0.1717226505279541, "operator": "ge"}, {"budget": 0.98, "threshold": -0.1915823370218277, "operator": "ge"}, {"budget": 0.99, "threshold": -0.22513195872306824, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.06702950443999, 76.46806072758523, 76.7831566886279, 77.16986536808938, 77.44199369808078, 77.80005729017473, 78.12947579490117, 78.32999140647378, 78.50186193067888, 78.81695789172156, 79.18934402749929, 79.46147235749069, 79.84818103695216, 80.10598682325981, 80.40676024061874, 80.65024348324262, 80.90804926955026, 81.0655972500716, 81.28043540532799, 81.49527356058435, 81.5525637353194, 81.63849899742193, 81.66714408478946, 81.76740189057577, 81.78172443425952, 81.86765969636207, 81.95359495846462, 81.95359495846462, 82.02520767688341, 82.02520767688341, 82.11114293898596, 82.11114293898596, 82.13978802635347, 82.11114293898596, 82.13978802635347, 82.21140074477226, 82.21140074477226, 82.2686909195073, 82.31165855055858, 82.42623890002865, 82.51217416213119, 82.59810942423374, 82.64107705528502, 82.66972214265253, 82.81294757949011, 82.84159266685764, 82.97049556001146, 82.99914064737897, 83.04210827843025, 83.19965625895158, 83.32855915210541, 83.44313950157549, 83.54339730736179, 83.60068748209682, 83.75823546261816, 83.9157834431395, 84.01604124892582, 84.10197651102835, 84.28816957891722, 84.41707247207104, 84.48868519048983, 84.51733027785735, 84.56029790890862, 84.71784588942997, 84.93268404468634, 85.03294185047264, 85.27642509309653, 85.3910054425666, 85.53423087940418, 85.62016614150673, 85.72042394729303, 85.79203666571182, 85.86364938413061, 85.9066170151819, 86.02119736465195, 86.13577771412201, 86.25035806359209, 86.40790604411343, 86.36493841306215, 86.60842165568606, 86.75164709252363, 86.90919507304497, 87.15267831566887, 87.26725866513893, 87.46777427671154, 87.71125751933543, 87.86880549985678, 88.08364365511314, 88.14093382984818, 88.269836723002, 88.25551417931825, 88.35577198510455, 88.5133199656259, 88.79977083930106, 88.92867373245488, 89.14351188771126, 89.35835004296763, 89.50157547980521, 89.70209109137782, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 38, 70, 98, 126, 150, 179, 208, 229, 248, 274, 304, 327, 357, 381, 408, 429, 452, 473, 493, 511, 515, 522, 524, 531, 534, 542, 549, 553, 560, 565, 573, 577, 581, 583, 586, 592, 596, 602, 607, 615, 623, 632, 640, 647, 658, 663, 674, 680, 688, 702, 713, 726, 736, 744, 756, 771, 782, 790, 805, 816, 824, 830, 837, 849, 868, 879, 899, 909, 919, 932, 943, 951, 962, 969, 982, 997, 1007, 1024, 1031, 1050, 1066, 1080, 1099, 1110, 1129, 1147, 1161, 1180, 1190, 1205, 1216, 1231, 1246, 1272, 1287, 1308, 1326, 1342, 1365, 1389], "harmed": [0, 3, 7, 13, 14, 19, 23, 29, 36, 43, 47, 51, 55, 58, 64, 70, 74, 79, 89, 94, 97, 97, 98, 98, 98, 100, 102, 103, 107, 109, 114, 116, 120, 122, 126, 127, 128, 132, 134, 136, 136, 138, 141, 146, 151, 152, 155, 157, 161, 166, 169, 171, 176, 179, 183, 184, 188, 192, 194, 196, 198, 201, 205, 209, 210, 214, 218, 221, 223, 223, 230, 234, 237, 243, 247, 252, 259, 261, 267, 277, 279, 285, 288, 290, 293, 298, 299, 302, 306, 312, 318, 330, 338, 342, 348, 354, 360, 363, 369, 378, 388], "exact_score": [75.56574047550845, 76.06702950443999, 76.46806072758523, 76.7831566886279, 77.16986536808938, 77.44199369808078, 77.80005729017473, 78.12947579490117, 78.32999140647378, 78.50186193067888, 78.81695789172156, 79.18934402749929, 79.46147235749069, 79.84818103695216, 80.10598682325981, 80.40676024061874, 80.65024348324262, 80.90804926955026, 81.0655972500716, 81.28043540532799, 81.49527356058435, 81.5525637353194, 81.63849899742193, 81.66714408478946, 81.76740189057577, 81.78172443425952, 81.86765969636207, 81.95359495846462, 81.95359495846462, 82.02520767688341, 82.02520767688341, 82.11114293898596, 82.11114293898596, 82.13978802635347, 82.11114293898596, 82.13978802635347, 82.21140074477226, 82.21140074477226, 82.2686909195073, 82.31165855055858, 82.42623890002865, 82.51217416213119, 82.59810942423374, 82.64107705528502, 82.66972214265253, 82.81294757949011, 82.84159266685764, 82.97049556001146, 82.99914064737897, 83.04210827843025, 83.19965625895158, 83.32855915210541, 83.44313950157549, 83.54339730736179, 83.60068748209682, 83.75823546261816, 83.9157834431395, 84.01604124892582, 84.10197651102835, 84.28816957891722, 84.41707247207104, 84.48868519048983, 84.51733027785735, 84.56029790890862, 84.71784588942997, 84.93268404468634, 85.03294185047264, 85.27642509309653, 85.3910054425666, 85.53423087940418, 85.62016614150673, 85.72042394729303, 85.79203666571182, 85.86364938413061, 85.9066170151819, 86.02119736465195, 86.13577771412201, 86.25035806359209, 86.40790604411343, 86.36493841306215, 86.60842165568606, 86.75164709252363, 86.90919507304497, 87.15267831566887, 87.26725866513893, 87.46777427671154, 87.71125751933543, 87.86880549985678, 88.08364365511314, 88.14093382984818, 88.269836723002, 88.25551417931825, 88.35577198510455, 88.5133199656259, 88.79977083930106, 88.92867373245488, 89.14351188771126, 89.35835004296763, 89.50157547980521, 89.70209109137782, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 83.57051618841297, "threshold_area100": 83.56953594958465, "threshold_area25": 79.36694356917788}, "test": {"score": [76.42247255506925, 76.80275525579393, 77.15433737533185, 77.49156920427639, 77.80727559733084, 78.130157135682, 78.42433809284638, 78.67546817823062, 79.01987515247184, 79.26383009255937, 79.47908445146015, 79.90241802396498, 80.1678983999426, 80.3544521776566, 80.69885915189784, 80.89258807490852, 81.17241874147952, 81.45224940805052, 81.80383152758843, 81.94015928822559, 82.14106335653297, 82.14823850182967, 82.22716510009327, 82.26304082657674, 82.3132668436536, 82.36349286073043, 82.42089402310397, 82.4782951854775, 82.52852120255434, 82.5930975102246, 82.64332352730143, 82.70789983497167, 82.75812585204851, 82.79400157853198, 82.88727846738897, 82.9518547750592, 82.98773050154266, 83.06665709980628, 83.15275884336658, 83.2245102963335, 83.29626174930043, 83.38236349286073, 83.47564038171772, 83.61196814235488, 83.66936930472842, 83.76982133888211, 83.86309822773912, 83.97072540718949, 84.0711774413432, 84.15010403960682, 84.20750520198035, 84.27208150965058, 84.32230752672743, 84.40840927028772, 84.5232115950348, 84.64518906507857, 84.73846595393557, 84.94654516753964, 85.09722321877018, 85.1617995264404, 85.14744923584703, 85.2694267058908, 85.28377699648418, 85.38422903063787, 85.46315562890149, 85.52055679127503, 85.67840998780225, 85.74298629547249, 85.9080146372964, 86.00129152615341, 86.08739326971371, 86.23089617564756, 86.3815742268781, 86.53942742340533, 86.70445576522926, 86.85513381645978, 87.00581186769031, 87.08473846595393, 87.23541651718448, 87.44349573078856, 87.665925234986, 87.80225299562316, 88.01750735452393, 88.17536055105116, 88.23276171342471, 88.39779005524862, 88.58434383296262, 88.691971012413, 88.79959819186338, 88.90722537131377, 89.065078567841, 89.23728205496163, 89.37360981559877, 89.49558728564253, 89.74671737102676, 89.93327114874077, 90.11264978115807, 90.29202841357538, 90.49293248188276, 90.59338451603645, 90.73688742197031], "calls": [0.0, 0.9758197603501471, 2.0090406830738323, 2.8987586998636723, 3.7597761354667436, 4.642319006959891, 5.567912750233192, 6.543732510583339, 7.5984788691971, 8.35904427064648, 9.420965774556935, 10.490062423764082, 11.487407620004305, 12.255148166750377, 13.266843653583985, 14.091985362703593, 15.096505704240512, 16.25887924230466, 17.27774987443496, 18.1674678912248, 19.394417736959173, 20.226734591375475, 21.16667862524216, 22.070746932625383, 23.118318145942453, 24.151539068666143, 25.04125708545598, 25.852048503982207, 26.763291956662123, 27.868264332352734, 28.801033220922722, 29.877305015426565, 31.312334074765012, 32.09442491210447, 33.19222214249839, 34.13934132166177, 34.98600846667145, 35.9618282270216, 36.94482313266844, 37.75561455119466, 38.68838343976466, 39.81488125134534, 40.75482528521203, 41.83827222501256, 43.07957236134032, 44.16301930114085, 45.17471478797446, 46.272512018368374, 47.39183468465236, 48.39635502618928, 49.243022171198966, 50.34799454688957, 51.68257157207433, 52.90234627251201, 53.71313769103825, 54.82528521202554, 55.7078280835187, 56.68364784386883, 57.99669943316352, 59.14472268063429, 60.063141278610885, 61.00308531247758, 61.74212527803689, 62.68924445720025, 63.53591160220995, 64.41127932840641, 65.6167037382507, 66.728851259238, 67.8409987802253, 68.90292028413575, 69.79263830092559, 70.76128291597905, 71.67970151395566, 72.69139700078927, 73.81071966707326, 74.87264117098371, 75.74083375188347, 76.61620147807993, 77.69247327258377, 78.58936643467031, 79.7015139556576, 80.85671234842505, 81.91145870703882, 83.03795651861951, 83.7913467747722, 85.1617995264404, 86.14479443208725, 86.99863672239363, 87.8596541579967, 88.74937217478654, 89.83999425988375, 90.6364353878166, 92.07863959245175, 93.0544593528019, 94.36033579679989, 95.32180526655665, 96.04649494152257, 97.20169333429001, 98.2420894023104, 99.14615770969361, 100.0], "rescued": [0, 60, 114, 164, 215, 264, 314, 363, 419, 459, 507, 571, 617, 651, 708, 743, 789, 841, 897, 931, 967, 971, 983, 991, 999, 1011, 1020, 1031, 1040, 1054, 1064, 1073, 1087, 1093, 1111, 1125, 1131, 1143, 1159, 1172, 1185, 1200, 1216, 1237, 1256, 1276, 1292, 1308, 1327, 1341, 1353, 1367, 1384, 1401, 1424, 1443, 1459, 1490, 1517, 1536, 1546, 1568, 1578, 1597, 1614, 1629, 1662, 1682, 1716, 1739, 1758, 1786, 1812, 1838, 1870, 1899, 1926, 1946, 1977, 2011, 2049, 2077, 2117, 2149, 2168, 2210, 2246, 2271, 2293, 2319, 2351, 2380, 2413, 2448, 2492, 2526, 2557, 2597, 2637, 2670, 2706], "harmed": [0, 7, 12, 15, 22, 26, 35, 49, 57, 63, 81, 86, 95, 103, 112, 120, 127, 140, 147, 162, 170, 173, 174, 177, 178, 183, 184, 187, 189, 194, 197, 197, 204, 205, 210, 215, 216, 217, 221, 224, 227, 230, 233, 235, 246, 252, 255, 256, 261, 264, 268, 273, 283, 288, 295, 297, 300, 302, 308, 318, 330, 335, 343, 348, 354, 361, 372, 383, 394, 404, 411, 419, 424, 428, 437, 445, 451, 460, 470, 475, 482, 491, 501, 511, 522, 541, 551, 561, 568, 579, 589, 594, 608, 626, 635, 643, 649, 664, 676, 695, 711], "exact_score": [76.42247255506925, 76.82428069168401, 77.14716223003516, 77.54179522135324, 77.88620219559446, 78.28801033220924, 78.57501614407693, 78.84049652005454, 79.14185262251561, 79.35710698141638, 79.68716366506422, 80.04592092989883, 80.29705101528306, 80.57688168185406, 80.85671234842506, 81.17241874147952, 81.40202339097367, 81.68185405754467, 81.89710841644543, 82.09083733945613, 82.15541364712635, 82.21281480949989, 82.26304082657674, 82.3060916983569, 82.36349286073043, 82.42089402310397, 82.47112004018081, 82.53569634785106, 82.5930975102246, 82.64332352730143, 82.71507498026835, 82.74377556145512, 82.78682643323528, 82.88010332209228, 82.9518547750592, 82.99490564683936, 83.06665709980628, 83.15275884336658, 83.2316854416302, 83.31061203989381, 83.38236349286073, 83.52586639879458, 83.62631843294827, 83.67654445002512, 83.74829590299204, 83.84874793714573, 83.94202482600272, 84.01377627896966, 84.12140345842003, 84.18597976609027, 84.23620578316711, 84.3366578173208, 84.3438329626175, 84.4299347061778, 84.5232115950348, 84.65953935567195, 84.77434168041903, 85.00394632991318, 85.09722321877018, 85.15462438114372, 85.14027409055034, 85.26225156059411, 85.29812728707756, 85.39857932123127, 85.53490708186841, 85.64253426131879, 85.69993542369232, 85.80756260314271, 85.9080146372964, 85.98694123556002, 86.11609385090048, 86.26677190213103, 86.43180024395494, 86.5681280045921, 86.73315634641602, 86.85513381645978, 87.0273373035804, 87.09908875654733, 87.3071679701514, 87.54394776494225, 87.68745067087609, 87.83812872210662, 88.0462079357107, 88.18253569634786, 88.23993685872139, 88.39061490995192, 88.54129296118248, 88.691971012413, 88.84264906364353, 88.92875080720385, 89.10095429432447, 89.30903350792853, 89.36643467030207, 89.49558728564253, 89.68214106335654, 89.91174571285069, 90.0839491999713, 90.23462725120184, 90.44988161010261, 90.60773480662984, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 84.40629487538686, "threshold_area100": 84.350326469111, "threshold_area25": 80.0071751452967}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.3710076808929443, "operator": "ge"}, {"budget": 0.02, "threshold": 1.995640516281128, "operator": "ge"}, {"budget": 0.03, "threshold": 1.7504571676254272, "operator": "ge"}, {"budget": 0.04, "threshold": 1.5650334358215332, "operator": "ge"}, {"budget": 0.05, "threshold": 1.4011592864990234, "operator": "ge"}, {"budget": 0.06, "threshold": 1.279715657234192, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1655771732330322, "operator": "ge"}, {"budget": 0.08, "threshold": 1.0564374923706055, "operator": "ge"}, {"budget": 0.09, "threshold": 0.9645521640777588, "operator": "ge"}, {"budget": 0.1, "threshold": 0.855009138584137, "operator": "ge"}, {"budget": 0.11, "threshold": 0.7604106664657593, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6762897372245789, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5686159729957581, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4909829795360565, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4124797582626343, "operator": "ge"}, {"budget": 0.16, "threshold": 0.32492154836654663, "operator": "ge"}, {"budget": 0.17, "threshold": 0.23912574350833893, "operator": "ge"}, {"budget": 0.18, "threshold": 0.1415514498949051, "operator": "ge"}, {"budget": 0.19, "threshold": 0.05002915859222412, "operator": "ge"}, {"budget": 0.2, "threshold": -0.040365397930145264, "operator": "ge"}, {"budget": 0.21, "threshold": -0.11265118420124054, "operator": "ge"}, {"budget": 0.22, "threshold": -0.19711846113204956, "operator": "ge"}, {"budget": 0.23, "threshold": -0.28995412588119507, "operator": "ge"}, {"budget": 0.24, "threshold": -0.39270398020744324, "operator": "ge"}, {"budget": 0.25, "threshold": -0.4878239631652832, "operator": "ge"}, {"budget": 0.26, "threshold": -0.5908448696136475, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6998181343078613, "operator": "ge"}, {"budget": 0.28, "threshold": -0.7971129417419434, "operator": "ge"}, {"budget": 0.29, "threshold": -0.9147021770477295, "operator": "ge"}, {"budget": 0.3, "threshold": -0.9857653379440308, "operator": "ge"}, {"budget": 0.31, "threshold": -1.1046186685562134, "operator": "ge"}, {"budget": 0.32, "threshold": -1.2257702350616455, "operator": "ge"}, {"budget": 0.33, "threshold": -1.3411145210266113, "operator": "ge"}, {"budget": 0.34, "threshold": -1.4522053003311157, "operator": "ge"}, {"budget": 0.35, "threshold": -1.5791652202606201, "operator": "ge"}, {"budget": 0.36, "threshold": -1.7229844331741333, "operator": "ge"}, {"budget": 0.37, "threshold": -1.836168646812439, "operator": "ge"}, {"budget": 0.38, "threshold": -1.9823286533355713, "operator": "ge"}, {"budget": 0.39, "threshold": -2.1440157890319824, "operator": "ge"}, {"budget": 0.4, "threshold": -2.2716147899627686, "operator": "ge"}, {"budget": 0.41, "threshold": -2.4409468173980713, "operator": "ge"}, {"budget": 0.42, "threshold": -2.60603666305542, "operator": "ge"}, {"budget": 0.43, "threshold": -2.7468039989471436, "operator": "ge"}, {"budget": 0.44, "threshold": -2.9094302654266357, "operator": "ge"}, {"budget": 0.45, "threshold": -3.024315595626831, "operator": "ge"}, {"budget": 0.46, "threshold": -3.2228691577911377, "operator": "ge"}, {"budget": 0.47, "threshold": -3.386110782623291, "operator": "ge"}, {"budget": 0.48, "threshold": -3.5829010009765625, "operator": "ge"}, {"budget": 0.49, "threshold": -3.695906639099121, "operator": "ge"}, {"budget": 0.5, "threshold": -3.9098732471466064, "operator": "ge"}, {"budget": 0.51, "threshold": -4.025958061218262, "operator": "ge"}, {"budget": 0.52, "threshold": -4.200861930847168, "operator": "ge"}, {"budget": 0.53, "threshold": -4.3657002449035645, "operator": "ge"}, {"budget": 0.54, "threshold": -4.501424312591553, "operator": "ge"}, {"budget": 0.55, "threshold": -4.620879173278809, "operator": "ge"}, {"budget": 0.56, "threshold": -4.737017631530762, "operator": "ge"}, {"budget": 0.57, "threshold": -4.886433124542236, "operator": "ge"}, {"budget": 0.58, "threshold": -5.001624584197998, "operator": "ge"}, {"budget": 0.59, "threshold": -5.177721977233887, "operator": "ge"}, {"budget": 0.6, "threshold": -5.3050618171691895, "operator": "ge"}, {"budget": 0.61, "threshold": -5.43849515914917, "operator": "ge"}, {"budget": 0.62, "threshold": -5.565981388092041, "operator": "ge"}, {"budget": 0.63, "threshold": -5.6947245597839355, "operator": "ge"}, {"budget": 0.64, "threshold": -5.839589595794678, "operator": "ge"}, {"budget": 0.65, "threshold": -5.976497173309326, "operator": "ge"}, {"budget": 0.66, "threshold": -6.088998317718506, "operator": "ge"}, {"budget": 0.67, "threshold": -6.183928966522217, "operator": "ge"}, {"budget": 0.68, "threshold": -6.290885925292969, "operator": "ge"}, {"budget": 0.69, "threshold": -6.384757041931152, "operator": "ge"}, {"budget": 0.7, "threshold": -6.489990234375, "operator": "ge"}, {"budget": 0.71, "threshold": -6.619457244873047, "operator": "ge"}, {"budget": 0.72, "threshold": -6.752349853515625, "operator": "ge"}, {"budget": 0.73, "threshold": -6.8552703857421875, "operator": "ge"}, {"budget": 0.74, "threshold": -6.968081951141357, "operator": "ge"}, {"budget": 0.75, "threshold": -7.097877502441406, "operator": "ge"}, {"budget": 0.76, "threshold": -7.22809362411499, "operator": "ge"}, {"budget": 0.77, "threshold": -7.361289024353027, "operator": "ge"}, {"budget": 0.78, "threshold": -7.47624397277832, "operator": "ge"}, {"budget": 0.79, "threshold": -7.576362609863281, "operator": "ge"}, {"budget": 0.8, "threshold": -7.709856986999512, "operator": "ge"}, {"budget": 0.81, "threshold": -7.827696323394775, "operator": "ge"}, {"budget": 0.82, "threshold": -7.941395282745361, "operator": "ge"}, {"budget": 0.83, "threshold": -8.043011665344238, "operator": "ge"}, {"budget": 0.84, "threshold": -8.140785217285156, "operator": "ge"}, {"budget": 0.85, "threshold": -8.258099555969238, "operator": "ge"}, {"budget": 0.86, "threshold": -8.366324424743652, "operator": "ge"}, {"budget": 0.87, "threshold": -8.491289138793945, "operator": "ge"}, {"budget": 0.88, "threshold": -8.635003089904785, "operator": "ge"}, {"budget": 0.89, "threshold": -8.784337043762207, "operator": "ge"}, {"budget": 0.9, "threshold": -8.910804748535156, "operator": "ge"}, {"budget": 0.91, "threshold": -9.057848930358887, "operator": "ge"}, {"budget": 0.92, "threshold": -9.237421035766602, "operator": "ge"}, {"budget": 0.93, "threshold": -9.396744728088379, "operator": "ge"}, {"budget": 0.94, "threshold": -9.575535774230957, "operator": "ge"}, {"budget": 0.95, "threshold": -9.770130157470703, "operator": "ge"}, {"budget": 0.96, "threshold": -10.038204193115234, "operator": "ge"}, {"budget": 0.97, "threshold": -10.315329551696777, "operator": "ge"}, {"budget": 0.98, "threshold": -10.750017166137695, "operator": "ge"}, {"budget": 0.99, "threshold": -11.320954322814941, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.95244915496991, 76.29619020338012, 76.7831566886279, 77.18418791177312, 77.44199369808078, 77.77141220280721, 78.02921798911487, 78.40160412489257, 78.77399026067029, 78.94586078487539, 79.18934402749929, 79.56173016327699, 79.87682612431968, 80.19192208536235, 80.43540532798625, 80.70753365797765, 80.93669435691778, 81.10856488112287, 81.33772558006301, 81.60985391005443, 81.83901460899456, 81.9965625895159, 82.0968203953022, 82.19707820108852, 82.38327126897737, 82.64107705528502, 82.81294757949011, 83.01346319106273, 83.29991406473789, 83.52907476367803, 83.7725580063019, 84.01604124892582, 84.27384703523344, 84.37410484101976, 84.68920080206244, 84.96132913205385, 85.07590948152391, 85.20481237467774, 85.44829561730162, 85.57719851045546, 85.82068175307934, 85.94958464623316, 86.078487539387, 86.20739043254082, 86.3506158693784, 86.47951876253222, 86.6227441993698, 86.82325981094242, 86.88054998567746, 86.89487252936121, 87.06674305356631, 87.00945287883127, 87.03809796619879, 87.1383557719851, 87.15267831566887, 87.28158120882267, 87.33887138355772, 87.4820968203953, 87.52506445144658, 87.59667716986537, 87.66828988828415, 87.88312804354052, 87.98338584932684, 88.09796619879691, 88.25551417931825, 88.34144944142079, 88.38441707247206, 88.52764250930964, 88.6135777714122, 88.71383557719851, 88.75680320824978, 88.81409338298482, 88.87138355771985, 88.94299627613864, 88.94299627613864, 89.00028645087367, 89.04325408192494, 89.17215697507876, 89.27241478086508, 89.34402749928387, 89.3297049556001, 89.31538241191636, 89.4013176740189, 89.44428530507018, 89.45860784875393, 89.55886565454024, 89.573188198224, 89.63047837295903, 89.6734460040103, 89.71641363506158, 89.70209109137782, 89.68776854769406, 89.75938126611285, 89.78802635348038, 89.84531652821542, 89.8166714408479, 89.85963907189917, 89.85963907189917, 89.9169292466342, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 61, 97, 128, 148, 174, 199, 230, 259, 277, 299, 328, 354, 379, 400, 428, 452, 474, 495, 521, 547, 565, 583, 601, 619, 640, 663, 679, 704, 727, 746, 766, 787, 796, 821, 842, 858, 870, 891, 904, 924, 939, 951, 963, 977, 987, 999, 1016, 1025, 1032, 1049, 1054, 1060, 1070, 1077, 1090, 1095, 1109, 1117, 1127, 1136, 1151, 1160, 1172, 1184, 1193, 1201, 1213, 1222, 1230, 1235, 1240, 1245, 1254, 1256, 1262, 1267, 1277, 1286, 1293, 1297, 1301, 1308, 1313, 1320, 1329, 1332, 1339, 1343, 1348, 1351, 1357, 1363, 1366, 1373, 1374, 1379, 1380, 1387, 1389], "harmed": [0, 5, 10, 12, 15, 17, 20, 27, 32, 35, 41, 46, 49, 53, 56, 60, 69, 77, 87, 92, 99, 109, 116, 127, 138, 143, 146, 157, 159, 164, 171, 173, 176, 179, 181, 184, 186, 194, 197, 201, 205, 208, 214, 217, 220, 224, 225, 227, 230, 235, 241, 246, 255, 259, 262, 268, 272, 273, 277, 282, 287, 291, 291, 293, 297, 298, 301, 306, 308, 311, 312, 314, 315, 316, 320, 322, 324, 326, 327, 329, 331, 336, 341, 342, 344, 350, 352, 354, 357, 358, 360, 364, 371, 372, 373, 376, 379, 381, 382, 385, 388], "exact_score": [75.56574047550845, 75.95244915496991, 76.29619020338012, 76.7831566886279, 77.18418791177312, 77.44199369808078, 77.77141220280721, 78.02921798911487, 78.40160412489257, 78.77399026067029, 78.94586078487539, 79.18934402749929, 79.56173016327699, 79.87682612431968, 80.19192208536235, 80.43540532798625, 80.70753365797765, 80.93669435691778, 81.10856488112287, 81.33772558006301, 81.60985391005443, 81.83901460899456, 81.9965625895159, 82.0968203953022, 82.19707820108852, 82.38327126897737, 82.64107705528502, 82.81294757949011, 83.01346319106273, 83.29991406473789, 83.52907476367803, 83.7725580063019, 84.01604124892582, 84.27384703523344, 84.37410484101976, 84.68920080206244, 84.96132913205385, 85.07590948152391, 85.20481237467774, 85.44829561730162, 85.57719851045546, 85.82068175307934, 85.94958464623316, 86.078487539387, 86.20739043254082, 86.3506158693784, 86.47951876253222, 86.6227441993698, 86.82325981094242, 86.88054998567746, 86.89487252936121, 87.06674305356631, 87.00945287883127, 87.03809796619879, 87.1383557719851, 87.15267831566887, 87.28158120882267, 87.33887138355772, 87.4820968203953, 87.52506445144658, 87.59667716986537, 87.66828988828415, 87.88312804354052, 87.98338584932684, 88.09796619879691, 88.25551417931825, 88.34144944142079, 88.38441707247206, 88.52764250930964, 88.6135777714122, 88.71383557719851, 88.75680320824978, 88.81409338298482, 88.87138355771985, 88.94299627613864, 88.94299627613864, 89.00028645087367, 89.04325408192494, 89.17215697507876, 89.27241478086508, 89.34402749928387, 89.3297049556001, 89.31538241191636, 89.4013176740189, 89.44428530507018, 89.45860784875393, 89.55886565454024, 89.573188198224, 89.63047837295903, 89.6734460040103, 89.71641363506158, 89.70209109137782, 89.68776854769406, 89.75938126611285, 89.78802635348038, 89.84531652821542, 89.8166714408479, 89.85963907189917, 89.85963907189917, 89.9169292466342, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.49279868575582, "threshold_area100": 85.49176453738183, "threshold_area25": 79.48152391864795}, "test": {"score": [76.42247255506925, 76.78122981990386, 77.21891368300209, 77.5848460931334, 77.90055248618785, 78.18038315275885, 78.48173925521992, 78.67546817823062, 78.91942311831815, 79.27818038315276, 79.50060988735022, 79.73738968214107, 79.91676831455837, 80.39032790414005, 80.64145798952428, 80.9069383655019, 81.19394417736959, 81.32309679271006, 81.66750376695128, 81.94015928822559, 82.16976393771974, 82.41371887780727, 82.64332352730143, 82.9446796297625, 83.2316854416302, 83.40388892875082, 83.5976178517615, 83.90614909951927, 84.10705316782665, 84.41558441558442, 84.58061275740833, 84.73846595393557, 84.92501973164957, 85.19767525292387, 85.38422903063787, 85.58513309894525, 85.90083949199972, 86.05151754323025, 86.30264762861448, 86.54660256870201, 86.72598120111932, 86.86948410705317, 87.06321303006386, 87.19236564540432, 87.3071679701514, 87.37891942311832, 87.47937145727201, 87.665925234986, 87.79507785032648, 87.90988017507355, 88.01750735452393, 88.10360909808423, 88.22558656812801, 88.29016287579825, 88.34756403817178, 88.40496520054532, 88.49824208940231, 88.59151897825932, 88.67044557652292, 88.77089761067663, 88.84982420894023, 88.90722537131377, 88.95745138839062, 88.98615196957739, 89.08660400373107, 89.15118031140132, 89.20858147377486, 89.22293176436823, 89.24445720025831, 89.3377340891153, 89.4310109779723, 89.4310109779723, 89.49558728564253, 89.64626533687307, 89.76106766162016, 89.86869484107054, 89.92609600344407, 89.94044629403746, 90.01937289230108, 90.06242376408122, 90.10547463586138, 90.12700007175145, 90.18440123412499, 90.22027696060846, 90.28485326827868, 90.31355384946545, 90.3566047212456, 90.40683073832245, 90.41400588361914, 90.49293248188276, 90.47140704599268, 90.49293248188276, 90.54315849895961, 90.59338451603645, 90.62208509722322, 90.61490995192652, 90.65078567840999, 90.70101169548683, 90.69383655019014, 90.744062567267, 90.73688742197031], "calls": [0.0, 0.9255937432733014, 2.131018153117601, 3.1068379134677477, 3.9965559302575877, 4.95802540001435, 5.840568271507498, 6.608308818253569, 7.505201980340102, 8.466671450096865, 9.399440338666858, 10.2891583554567, 11.085599483389538, 12.334074765013991, 13.180741910023677, 13.998708473846596, 14.981703379493435, 15.849895960393198, 17.098371242017652, 18.153117600631415, 19.401592882255866, 20.36306235201263, 21.331706967066083, 22.486905359833536, 23.491425701370453, 24.603573222357753, 25.529166965631056, 26.569563033651434, 27.602783956375117, 28.772332639735954, 29.482672024108485, 30.55176867331563, 31.59216474133601, 32.51775848460932, 33.49357824495946, 34.533974312979836, 35.82550046638445, 36.73674391906437, 37.82736600416158, 39.00408983281912, 39.857932123125494, 40.85527731936572, 42.0750520198034, 43.02217119896678, 43.94776494224008, 44.55765229245892, 45.74155126641315, 46.68149530027983, 47.90844514601421, 48.61160938509005, 49.745282341967425, 50.43409629044988, 51.68974671737102, 52.651216187127794, 53.47635789624739, 54.30149960536701, 55.047714716223005, 55.994833895386385, 56.776924732725846, 57.90342254430653, 58.814665996986434, 59.74025974025974, 60.85958240654373, 61.89997847456411, 63.069527157924945, 64.13862380713209, 65.20772045633923, 65.91088469541508, 66.85082872928176, 67.68314558369806, 68.59438903637799, 69.67783597617851, 70.87608524072613, 71.9236564540432, 72.90665135969003, 74.12642606012771, 75.27444930759847, 76.55880031570639, 77.62072181961685, 78.63959245174715, 79.8019659898113, 80.85671234842505, 81.7392552199182, 82.50699576666428, 83.39671378345412, 84.36535839850757, 85.34117815885772, 86.3815742268781, 87.44349573078854, 88.691971012413, 89.624739900983, 90.60773480662984, 91.61225514816675, 92.45174714787974, 93.44909234411996, 94.40338666858004, 95.45095788189711, 96.3837267704671, 97.79005524861878, 98.84480160723254, 100.0], "rescued": [0, 55, 124, 176, 224, 274, 319, 354, 398, 455, 498, 536, 571, 646, 690, 735, 785, 816, 880, 931, 973, 1015, 1062, 1113, 1164, 1206, 1244, 1293, 1334, 1389, 1419, 1457, 1494, 1537, 1571, 1609, 1661, 1690, 1738, 1777, 1809, 1838, 1878, 1902, 1925, 1938, 1961, 1992, 2022, 2043, 2065, 2082, 2107, 2121, 2136, 2151, 2167, 2185, 2201, 2222, 2237, 2253, 2268, 2281, 2302, 2321, 2337, 2342, 2348, 2365, 2383, 2391, 2405, 2433, 2451, 2470, 2481, 2495, 2510, 2519, 2531, 2541, 2551, 2558, 2570, 2582, 2593, 2602, 2607, 2620, 2623, 2630, 2638, 2648, 2655, 2659, 2673, 2684, 2690, 2701, 2706], "harmed": [0, 5, 13, 14, 18, 29, 32, 40, 50, 57, 69, 74, 84, 93, 102, 110, 120, 133, 149, 162, 172, 180, 195, 204, 215, 233, 244, 250, 263, 275, 282, 298, 309, 314, 322, 332, 340, 348, 361, 366, 373, 382, 395, 401, 408, 411, 420, 425, 437, 442, 449, 454, 462, 467, 474, 481, 484, 489, 494, 501, 505, 513, 521, 530, 537, 547, 555, 558, 561, 565, 570, 578, 583, 590, 592, 596, 599, 611, 615, 618, 624, 631, 633, 635, 638, 646, 651, 653, 657, 659, 665, 669, 670, 673, 676, 681, 690, 694, 701, 705, 711], "exact_score": [76.42247255506925, 76.80993040109063, 77.2045633924087, 77.53462007605654, 77.90055248618785, 78.18755829805554, 78.52479012700007, 78.77592021238429, 79.04140058836192, 79.38580756260315, 79.66563822917414, 79.91676831455837, 80.23964985290952, 80.57688168185406, 80.9069383655019, 81.1867690320729, 81.38767310038028, 81.6531534763579, 81.9329841429289, 82.1195379206429, 82.33479227954366, 82.56439692903783, 82.84422759560881, 83.07383224510296, 83.34648776637728, 83.49716581760781, 83.74829590299204, 83.99942598837626, 84.20033005668365, 84.49451101384804, 84.67388964626534, 84.8030422616058, 85.06134749228671, 85.29812728707756, 85.49185621008826, 85.73581115017579, 85.93671521848317, 86.09456841501041, 86.33852335509793, 86.54660256870201, 86.76185692760279, 86.92688526942672, 87.05603788476716, 87.18519050010764, 87.3143431154481, 87.38609456841502, 87.53677261964555, 87.71615125206286, 87.83812872210662, 87.9529310468537, 88.06773337160078, 88.132309679271, 88.26146229461148, 88.28298773050155, 88.36908947406185, 88.48389179880894, 88.59151897825932, 88.691971012413, 88.7924230465667, 88.85699935423692, 88.92157566190716, 88.96462653368731, 88.99332711487408, 89.08660400373107, 89.13683002080792, 89.17270574729139, 89.22293176436823, 89.251632345555, 89.38078496089545, 89.4310109779723, 89.44536126856569, 89.52428786682931, 89.66779077276314, 89.76824280691685, 89.85434455047715, 89.904570567554, 89.94044629403746, 89.96914687522423, 90.02654803759776, 90.0982994905647, 90.09112434526799, 90.12700007175145, 90.1772260888283, 90.25615268709191, 90.29920355887207, 90.34225443065222, 90.38530530243239, 90.42118102891584, 90.46423190069599, 90.47858219128938, 90.47140704599268, 90.51445791777284, 90.56468393484968, 90.60055966133315, 90.59338451603645, 90.6364353878166, 90.68666140489346, 90.70101169548683, 90.68666140489346, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.43872775741639, "threshold_area100": 86.36740331491714, "threshold_area25": 80.13417521704815}, "selected_budget": 99, "selected": {"val": {"score": 89.9169292466342, "calls": 98.99742194213692, "rescued": 1387, "harmed": 385}, "test": {"score": 90.744062567267, "calls": 98.84480160723254, "rescued": 2701, "harmed": 705}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.7620253562927246, "operator": "ge"}, {"budget": 0.02, "threshold": 1.5160430669784546, "operator": "ge"}, {"budget": 0.03, "threshold": 1.3058979511260986, "operator": "ge"}, {"budget": 0.04, "threshold": 1.154284119606018, "operator": "ge"}, {"budget": 0.05, "threshold": 1.0211608409881592, "operator": "ge"}, {"budget": 0.06, "threshold": 0.9267762899398804, "operator": "ge"}, {"budget": 0.07, "threshold": 0.8434592485427856, "operator": "ge"}, {"budget": 0.08, "threshold": 0.7719632983207703, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6898909211158752, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6193597912788391, "operator": "ge"}, {"budget": 0.11, "threshold": 0.5473977327346802, "operator": "ge"}, {"budget": 0.12, "threshold": 0.4715067744255066, "operator": "ge"}, {"budget": 0.13, "threshold": 0.4017336964607239, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3352925181388855, "operator": "ge"}, {"budget": 0.15, "threshold": 0.2736333906650543, "operator": "ge"}, {"budget": 0.16, "threshold": 0.22237172722816467, "operator": "ge"}, {"budget": 0.17, "threshold": 0.16231858730316162, "operator": "ge"}, {"budget": 0.18, "threshold": 0.1007758378982544, "operator": "ge"}, {"budget": 0.19, "threshold": 0.03022100031375885, "operator": "ge"}, {"budget": 0.2, "threshold": -0.027838096022605896, "operator": "ge"}, {"budget": 0.21, "threshold": -0.07935333251953125, "operator": "ge"}, {"budget": 0.22, "threshold": -0.13730594515800476, "operator": "ge"}, {"budget": 0.23, "threshold": -0.20857718586921692, "operator": "ge"}, {"budget": 0.24, "threshold": -0.2967420518398285, "operator": "ge"}, {"budget": 0.25, "threshold": -0.3764354884624481, "operator": "ge"}, {"budget": 0.26, "threshold": -0.44206345081329346, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5261158347129822, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6232597827911377, "operator": "ge"}, {"budget": 0.29, "threshold": -0.709088921546936, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7961294651031494, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8928515315055847, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9867817163467407, "operator": "ge"}, {"budget": 0.33, "threshold": -1.0932472944259644, "operator": "ge"}, {"budget": 0.34, "threshold": -1.1982427835464478, "operator": "ge"}, {"budget": 0.35, "threshold": -1.298201560974121, "operator": "ge"}, {"budget": 0.36, "threshold": -1.403930425643921, "operator": "ge"}, {"budget": 0.37, "threshold": -1.5530447959899902, "operator": "ge"}, {"budget": 0.38, "threshold": -1.6571054458618164, "operator": "ge"}, {"budget": 0.39, "threshold": -1.776914358139038, "operator": "ge"}, {"budget": 0.4, "threshold": -1.9394527673721313, "operator": "ge"}, {"budget": 0.41, "threshold": -2.0626208782196045, "operator": "ge"}, {"budget": 0.42, "threshold": -2.219609498977661, "operator": "ge"}, {"budget": 0.43, "threshold": -2.3480982780456543, "operator": "ge"}, {"budget": 0.44, "threshold": -2.521862030029297, "operator": "ge"}, {"budget": 0.45, "threshold": -2.685232162475586, "operator": "ge"}, {"budget": 0.46, "threshold": -2.840987205505371, "operator": "ge"}, {"budget": 0.47, "threshold": -2.99808406829834, "operator": "ge"}, {"budget": 0.48, "threshold": -3.1828560829162598, "operator": "ge"}, {"budget": 0.49, "threshold": -3.3399758338928223, "operator": "ge"}, {"budget": 0.5, "threshold": -3.5048985481262207, "operator": "ge"}, {"budget": 0.51, "threshold": -3.6416571140289307, "operator": "ge"}, {"budget": 0.52, "threshold": -3.8001999855041504, "operator": "ge"}, {"budget": 0.53, "threshold": -3.9326303005218506, "operator": "ge"}, {"budget": 0.54, "threshold": -4.078739643096924, "operator": "ge"}, {"budget": 0.55, "threshold": -4.2139129638671875, "operator": "ge"}, {"budget": 0.56, "threshold": -4.347181797027588, "operator": "ge"}, {"budget": 0.57, "threshold": -4.479809761047363, "operator": "ge"}, {"budget": 0.58, "threshold": -4.599714279174805, "operator": "ge"}, {"budget": 0.59, "threshold": -4.724423885345459, "operator": "ge"}, {"budget": 0.6, "threshold": -4.865686416625977, "operator": "ge"}, {"budget": 0.61, "threshold": -5.019604682922363, "operator": "ge"}, {"budget": 0.62, "threshold": -5.14496374130249, "operator": "ge"}, {"budget": 0.63, "threshold": -5.283251762390137, "operator": "ge"}, {"budget": 0.64, "threshold": -5.392236709594727, "operator": "ge"}, {"budget": 0.65, "threshold": -5.505191802978516, "operator": "ge"}, {"budget": 0.66, "threshold": -5.628913879394531, "operator": "ge"}, {"budget": 0.67, "threshold": -5.745421886444092, "operator": "ge"}, {"budget": 0.68, "threshold": -5.856860160827637, "operator": "ge"}, {"budget": 0.69, "threshold": -5.965182304382324, "operator": "ge"}, {"budget": 0.7, "threshold": -6.0703125, "operator": "ge"}, {"budget": 0.71, "threshold": -6.17510461807251, "operator": "ge"}, {"budget": 0.72, "threshold": -6.290804386138916, "operator": "ge"}, {"budget": 0.73, "threshold": -6.40019416809082, "operator": "ge"}, {"budget": 0.74, "threshold": -6.546962738037109, "operator": "ge"}, {"budget": 0.75, "threshold": -6.637095928192139, "operator": "ge"}, {"budget": 0.76, "threshold": -6.75797176361084, "operator": "ge"}, {"budget": 0.77, "threshold": -6.884551525115967, "operator": "ge"}, {"budget": 0.78, "threshold": -7.00244140625, "operator": "ge"}, {"budget": 0.79, "threshold": -7.108240604400635, "operator": "ge"}, {"budget": 0.8, "threshold": -7.206040859222412, "operator": "ge"}, {"budget": 0.81, "threshold": -7.308839321136475, "operator": "ge"}, {"budget": 0.82, "threshold": -7.431347370147705, "operator": "ge"}, {"budget": 0.83, "threshold": -7.531482219696045, "operator": "ge"}, {"budget": 0.84, "threshold": -7.644360065460205, "operator": "ge"}, {"budget": 0.85, "threshold": -7.754792213439941, "operator": "ge"}, {"budget": 0.86, "threshold": -7.8595123291015625, "operator": "ge"}, {"budget": 0.87, "threshold": -7.989416122436523, "operator": "ge"}, {"budget": 0.88, "threshold": -8.10034465789795, "operator": "ge"}, {"budget": 0.89, "threshold": -8.233537673950195, "operator": "ge"}, {"budget": 0.9, "threshold": -8.375521659851074, "operator": "ge"}, {"budget": 0.91, "threshold": -8.523049354553223, "operator": "ge"}, {"budget": 0.92, "threshold": -8.712282180786133, "operator": "ge"}, {"budget": 0.93, "threshold": -8.891668319702148, "operator": "ge"}, {"budget": 0.94, "threshold": -9.102977752685547, "operator": "ge"}, {"budget": 0.95, "threshold": -9.316192626953125, "operator": "ge"}, {"budget": 0.96, "threshold": -9.538707733154297, "operator": "ge"}, {"budget": 0.97, "threshold": -9.885919570922852, "operator": "ge"}, {"budget": 0.98, "threshold": -10.276147842407227, "operator": "ge"}, {"budget": 0.99, "threshold": -10.706032752990723, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.35348037811515, 76.75451160126038, 77.16986536808938, 77.42767115439702, 77.7427671154397, 78.10083070753365, 78.38728158120882, 78.71670008593526, 79.00315095961042, 79.3182469206531, 79.6333428816958, 79.84818103695216, 80.12030936694356, 80.39243769693498, 80.67888857061014, 80.9653394442853, 81.20882268690919, 81.32340303637926, 81.63849899742193, 81.753079346892, 81.98224004583213, 82.0968203953022, 82.22572328845604, 82.45488398739616, 82.6124319679175, 82.92752792896017, 83.05643082211401, 83.15668862790031, 83.52907476367803, 83.81552563735319, 83.9873961615583, 84.1879117731309, 84.35978229733601, 84.64623317101116, 84.8037811515325, 84.96132913205385, 85.17616728731022, 85.33371526783156, 85.51990833572042, 85.8063592093956, 85.9352621025494, 86.09281008307075, 86.23603551990833, 86.3506158693784, 86.42222858779719, 86.60842165568606, 86.75164709252363, 86.89487252936121, 87.02377542251504, 87.05242050988255, 87.05242050988255, 87.15267831566887, 87.2099684904039, 87.29590375250645, 87.35319392724148, 87.42480664566027, 87.5823546261816, 87.66828988828415, 87.73990260670294, 87.85448295617302, 87.98338584932684, 88.14093382984818, 88.269836723002, 88.29848181036952, 88.38441707247206, 88.42738470352333, 88.5419650529934, 88.62790031509596, 88.74248066456603, 88.81409338298482, 88.85706101403609, 88.90002864508737, 88.91435118877112, 89.00028645087367, 89.02893153824118, 89.08622171297623, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.47293039243769, 89.48725293612145, 89.51589802348897, 89.47293039243769, 89.50157547980521, 89.5445431108565, 89.55886565454024, 89.60183328559151, 89.6448009166428, 89.63047837295903, 89.6734460040103, 89.73073617874535, 89.75938126611285, 89.84531652821542, 89.84531652821542, 89.87396161558291, 89.9169292466342, 89.9169292466342, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 34, 62, 95, 125, 150, 175, 205, 230, 257, 282, 307, 333, 354, 378, 402, 426, 453, 477, 495, 523, 542, 565, 584, 602, 622, 641, 665, 683, 695, 723, 748, 763, 780, 796, 817, 834, 848, 870, 886, 901, 925, 936, 951, 964, 976, 987, 1005, 1018, 1029, 1042, 1051, 1056, 1066, 1073, 1083, 1091, 1102, 1114, 1124, 1137, 1148, 1160, 1172, 1183, 1189, 1198, 1205, 1215, 1225, 1235, 1240, 1245, 1250, 1253, 1261, 1265, 1271, 1278, 1285, 1290, 1296, 1308, 1313, 1320, 1325, 1331, 1334, 1338, 1343, 1349, 1351, 1356, 1362, 1367, 1375, 1377, 1380, 1384, 1387, 1389], "harmed": [0, 5, 7, 12, 13, 20, 23, 28, 33, 37, 42, 45, 49, 55, 60, 65, 69, 76, 83, 93, 99, 110, 117, 128, 137, 141, 149, 151, 160, 165, 167, 172, 175, 178, 182, 183, 189, 192, 199, 204, 206, 210, 212, 216, 219, 223, 229, 234, 237, 238, 242, 249, 254, 257, 260, 264, 268, 274, 275, 279, 287, 290, 293, 294, 296, 300, 303, 307, 309, 313, 315, 315, 317, 319, 321, 323, 325, 327, 329, 331, 333, 335, 337, 341, 346, 354, 358, 358, 361, 363, 366, 369, 371, 373, 376, 378, 380, 381, 382, 385, 388], "exact_score": [75.56574047550845, 75.98109424233743, 76.35348037811515, 76.75451160126038, 77.16986536808938, 77.42767115439702, 77.7427671154397, 78.10083070753365, 78.38728158120882, 78.71670008593526, 79.00315095961042, 79.3182469206531, 79.6333428816958, 79.84818103695216, 80.12030936694356, 80.39243769693498, 80.67888857061014, 80.9653394442853, 81.20882268690919, 81.32340303637926, 81.63849899742193, 81.753079346892, 81.98224004583213, 82.0968203953022, 82.22572328845604, 82.45488398739616, 82.6124319679175, 82.92752792896017, 83.05643082211401, 83.15668862790031, 83.52907476367803, 83.81552563735319, 83.9873961615583, 84.1879117731309, 84.35978229733601, 84.64623317101116, 84.8037811515325, 84.96132913205385, 85.17616728731022, 85.33371526783156, 85.51990833572042, 85.8063592093956, 85.9352621025494, 86.09281008307075, 86.23603551990833, 86.3506158693784, 86.42222858779719, 86.60842165568606, 86.75164709252363, 86.89487252936121, 87.02377542251504, 87.05242050988255, 87.05242050988255, 87.15267831566887, 87.2099684904039, 87.29590375250645, 87.35319392724148, 87.42480664566027, 87.5823546261816, 87.66828988828415, 87.73990260670294, 87.85448295617302, 87.98338584932684, 88.14093382984818, 88.269836723002, 88.29848181036952, 88.38441707247206, 88.42738470352333, 88.5419650529934, 88.62790031509596, 88.74248066456603, 88.81409338298482, 88.85706101403609, 88.90002864508737, 88.91435118877112, 89.00028645087367, 89.02893153824118, 89.08622171297623, 89.15783443139502, 89.22944714981381, 89.27241478086508, 89.3297049556001, 89.47293039243769, 89.48725293612145, 89.51589802348897, 89.47293039243769, 89.50157547980521, 89.5445431108565, 89.55886565454024, 89.60183328559151, 89.6448009166428, 89.63047837295903, 89.6734460040103, 89.73073617874535, 89.75938126611285, 89.84531652821542, 89.84531652821542, 89.87396161558291, 89.9169292466342, 89.9169292466342, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.50694994149951, "threshold_area100": 85.50594385562874, "threshold_area25": 79.49441420796333}, "test": {"score": [76.42247255506925, 76.87450670876086, 77.18303795651862, 77.54179522135324, 77.8646767597044, 78.19473344335222, 78.47456410992322, 78.73286934060415, 78.89072253713138, 79.19925378488915, 79.49343474205352, 79.68716366506422, 80.01004520341537, 80.34010188706321, 80.67733371600775, 80.92128865609529, 81.1006672885126, 81.38767310038028, 81.68902920284135, 81.96168472411567, 82.16976393771974, 82.44241945899404, 82.6720241084882, 82.90162875798235, 83.21016000574012, 83.36801320226735, 83.56891727057473, 83.81287221066228, 84.10705316782665, 84.3366578173208, 84.48016072325466, 84.78869197101243, 84.9034942957595, 85.0828729281768, 85.38422903063787, 85.60665853483533, 85.83626318432948, 86.09456841501041, 86.33134820980125, 86.5681280045921, 86.79773265408625, 86.92688526942672, 87.07756332065725, 87.15648991892087, 87.29281767955801, 87.38609456841502, 87.52959747434885, 87.60852407261247, 87.78072755973308, 87.88117959388678, 87.99598191863386, 88.14665996986439, 88.21123627753462, 88.29733802109493, 88.36908947406185, 88.41931549113869, 88.5197675252924, 88.61304441414939, 88.72784673889646, 88.7924230465667, 88.84264906364353, 88.90005022601709, 88.95027624309392, 89.065078567841, 89.11530458491785, 89.16553060199469, 89.23010690966493, 89.2659826361484, 89.28033292674176, 89.33055894381862, 89.40948554208224, 89.49558728564253, 89.56733873860946, 89.65344048216977, 89.79694338810361, 89.8902202769606, 89.98349716581761, 90.01937289230108, 90.10547463586138, 90.0982994905647, 90.14852550764154, 90.18440123412499, 90.19875152471838, 90.22027696060846, 90.24897754179523, 90.29202841357538, 90.29920355887207, 90.34942957594892, 90.38530530243239, 90.46423190069599, 90.51445791777284, 90.52163306306953, 90.52880820836621, 90.55750878955298, 90.59338451603645, 90.60773480662984, 90.65796082370669, 90.69383655019014, 90.70101169548683, 90.7297122766736, 90.73688742197031], "calls": [0.0, 1.0977972303939154, 1.973164956590371, 2.9920355887206718, 3.9319796225873573, 5.015426562387888, 5.926670015067805, 6.809212886560953, 7.447800817966564, 8.459496304800172, 9.399440338666858, 10.210231757193084, 11.286503551696923, 12.283848747937146, 13.2524933629906, 14.156561670373824, 14.909951926526514, 16.036449738107198, 17.06249551553419, 18.23921934419172, 19.437468608739326, 20.43481380497955, 21.367582693549544, 22.40080361627323, 23.570352299634067, 24.603573222357753, 25.44306522207075, 26.426060127717587, 27.674535409342038, 28.621654588505418, 29.640525220635716, 30.66657099806271, 31.513238143072396, 32.51058333931262, 33.64425629619, 34.59855062065007, 35.610246107483675, 36.887421970294895, 37.74843940589797, 38.70273373035804, 40.0373107555428, 40.89832819114587, 41.945899404462935, 42.79974169476932, 44.01234124991031, 44.99533615555715, 45.84917844586353, 46.8967496591806, 48.06629834254144, 48.963191504627964, 49.83855923082442, 50.73545239291095, 51.76149817033795, 52.608165315347634, 53.46918275095071, 54.35172562244386, 55.3347205280907, 56.1598622372103, 57.05675539929683, 57.881897108416446, 58.97969433881036, 60.16359331276458, 61.117887637224655, 62.194159431728494, 62.98342541436463, 63.92336944823133, 64.93506493506493, 65.982636148382, 67.05890794288585, 68.01320226734592, 68.93162086532251, 69.86438975389252, 70.78280835186914, 71.93800674463657, 73.42326182105188, 74.2340532395781, 75.29597474348856, 76.47987371744279, 77.64942240080362, 78.73286934060415, 79.622587357394, 80.58405682715075, 81.5670517327976, 82.44241945899404, 83.38953863815742, 84.40840927028772, 85.26225156059411, 86.45332567984502, 87.1206141924374, 88.23993685872139, 89.30903350792853, 90.51445791777284, 91.55485398579322, 92.62395063500037, 93.64999641242736, 94.63299131807419, 95.59446078783095, 96.79988519767525, 97.8618067015857, 98.7156489918921, 100.0], "rescued": [0, 66, 116, 173, 220, 274, 321, 366, 391, 445, 495, 533, 588, 639, 693, 740, 774, 829, 879, 934, 973, 1023, 1064, 1110, 1166, 1204, 1237, 1281, 1334, 1373, 1409, 1457, 1489, 1525, 1575, 1615, 1651, 1697, 1740, 1776, 1822, 1852, 1880, 1902, 1927, 1948, 1975, 1992, 2024, 2043, 2069, 2095, 2113, 2128, 2143, 2156, 2178, 2195, 2215, 2228, 2240, 2259, 2272, 2291, 2305, 2321, 2334, 2345, 2354, 2367, 2382, 2399, 2417, 2435, 2457, 2471, 2487, 2499, 2518, 2525, 2535, 2546, 2554, 2561, 2570, 2580, 2587, 2598, 2605, 2619, 2630, 2636, 2640, 2647, 2656, 2663, 2673, 2688, 2692, 2699, 2706], "harmed": [0, 3, 10, 17, 19, 27, 35, 44, 47, 58, 67, 78, 88, 93, 100, 113, 122, 137, 145, 162, 172, 184, 193, 207, 220, 236, 241, 251, 263, 270, 286, 291, 307, 318, 326, 335, 339, 349, 359, 362, 376, 388, 395, 406, 412, 420, 427, 433, 441, 446, 456, 461, 470, 473, 478, 484, 492, 496, 500, 504, 509, 520, 526, 529, 536, 545, 549, 555, 562, 568, 572, 577, 585, 591, 593, 594, 597, 604, 611, 619, 622, 628, 634, 638, 643, 647, 653, 657, 659, 662, 666, 671, 674, 677, 681, 686, 689, 699, 702, 705, 711], "exact_score": [76.42247255506925, 76.8314558369807, 77.197388247112, 77.54179522135324, 77.89337734089116, 78.20190858864893, 78.50326469110999, 78.76874506708761, 79.04857573365861, 79.41450814378992, 79.6082370668006, 79.90959316926168, 80.23964985290952, 80.60558226304083, 80.92128865609529, 81.11501757910598, 81.36614766449021, 81.67467891224798, 81.94015928822559, 82.10518763004951, 82.2989165530602, 82.58592236492788, 82.80835186912535, 83.06665709980628, 83.28908660400374, 83.45411494582765, 83.71959532180527, 83.97790055248619, 84.19315491138696, 84.44428499677119, 84.58061275740833, 84.83174284279256, 84.94654516753964, 85.19767525292387, 85.48468106479157, 85.69993542369232, 85.93671521848317, 86.12326899619718, 86.41027480806487, 86.60400373107555, 86.81208294467963, 86.94841070531677, 87.08473846595393, 87.19236564540432, 87.29281767955801, 87.38609456841502, 87.54394776494225, 87.63004950850255, 87.7735524144364, 87.88117959388678, 88.03185764511731, 88.14665996986439, 88.23276171342471, 88.29733802109493, 88.40496520054532, 88.5197675252924, 88.56999354236923, 88.71349644830309, 88.79959819186338, 88.84264906364353, 88.91440051661046, 88.94310109779724, 89.04355313195093, 89.10812943962115, 89.17270574729139, 89.23010690966493, 89.27315778144508, 89.28033292674176, 89.33055894381862, 89.40948554208224, 89.51711272153261, 89.58168902920285, 89.66061562746646, 89.74671737102676, 89.86869484107054, 89.98349716581761, 90.00502260170768, 90.04807347348785, 90.11264978115807, 90.11264978115807, 90.1700509435316, 90.19875152471838, 90.21310181531177, 90.23462725120184, 90.29920355887207, 90.30637870416876, 90.32072899476215, 90.38530530243239, 90.44988161010261, 90.50010762717945, 90.52163306306953, 90.52880820836621, 90.52880820836621, 90.57185908014638, 90.59338451603645, 90.62926024251992, 90.68666140489346, 90.69383655019014, 90.69383655019014, 90.744062567267, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.43887242383158, "threshold_area100": 86.37063213030063, "threshold_area25": 80.13489273157782}, "selected_budget": 98, "selected": {"val": {"score": 89.9169292466342, "calls": 97.99484388427385, "rescued": 1384, "harmed": 382}, "test": {"score": 90.70101169548683, "calls": 97.8618067015857, "rescued": 2692, "harmed": 702}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.73073617874535, "calls": 98.99742194213692}, "test": {"score": 90.61490995192652, "calls": 99.0600559661333}}, "old_normalized_interior": {"val": {"score": 89.88828415926669, "calls": 97.99484388427385}, "test": {"score": 90.66513596900337, "calls": 97.86898184688239}}, "old_normalized_endpoints": {"val": {"score": 89.90260670295044, "calls": 100.0}, "test": {"score": 90.73688742197031, "calls": 100.0}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 75.90948152391864, 76.32483529074764, 76.65425379547408, 76.96934975651676, 77.37038097966199, 77.68547694070466, 77.97192781437982, 78.22973360068748, 78.55915210541392, 78.91721569750787, 79.24663420223432, 79.46147235749069, 79.61902033801202, 79.94843884273847, 80.22056717272987, 80.56430822114007, 80.72185620166141, 81.0369521627041, 81.3090804926955, 81.53824119163563, 81.71011171584072, 81.98224004583213, 82.01088513319965, 82.21140074477226, 82.32598109424234, 82.62675451160125, 82.7126897737038, 82.98481810369522, 83.05643082211401, 83.28559152105413, 83.471784588943, 83.70094528788312, 83.82984818103695, 84.13062159839588, 84.27384703523344, 84.35978229733601, 84.54597536522486, 84.74649097679747, 84.91836150100256, 85.1475221999427, 85.27642509309653, 85.53423087940418, 85.63448868519049, 85.87797192781437, 86.078487539387, 86.33629332569464, 86.40790604411343, 86.55113148095101, 86.56545402463478, 86.72300200515612, 86.86622744199369, 86.93784016041248, 86.96648524778, 87.02377542251504, 87.09538814093382, 87.18132340303637, 87.2386135777714, 87.38183901460899, 87.45345173302779, 87.5823546261816, 87.61099971354912, 87.63964480091664, 87.73990260670294, 87.89745058722428, 87.91177313090805, 87.91177313090805, 88.08364365511314, 88.08364365511314, 88.1695789172157, 88.21254654826697, 88.31280435405327, 88.38441707247206, 88.47035233457461, 88.58493268404469, 88.64222285877972, 88.72815812088226, 88.79977083930106, 88.82841592666857, 89.00028645087367, 89.0575766256087, 89.0575766256087, 89.04325408192494, 89.10054425665999, 89.18647951876252, 89.25809223718132, 89.31538241191636, 89.45860784875393, 89.51589802348897, 89.48725293612145, 89.48725293612145, 89.55886565454024, 89.65912346032655, 89.6448009166428, 89.71641363506158, 89.80234889716414, 89.8166714408479, 89.83099398453164, 89.88828415926669, 89.85963907189917, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 75.90948152391864, 76.32483529074764, 76.65425379547408, 76.96934975651676, 77.37038097966199, 77.68547694070466, 77.97192781437982, 78.22973360068748, 78.55915210541392, 78.91721569750787, 79.24663420223432, 79.46147235749069, 79.61902033801202, 79.94843884273847, 80.22056717272987, 80.56430822114007, 80.72185620166141, 81.0369521627041, 81.3090804926955, 81.53824119163563, 81.71011171584072, 81.98224004583213, 82.01088513319965, 82.21140074477226, 82.32598109424234, 82.62675451160125, 82.7126897737038, 82.98481810369522, 83.05643082211401, 83.28559152105413, 83.471784588943, 83.70094528788312, 83.82984818103695, 84.13062159839588, 84.27384703523344, 84.35978229733601, 84.54597536522486, 84.74649097679747, 84.91836150100256, 85.1475221999427, 85.27642509309653, 85.53423087940418, 85.63448868519049, 85.87797192781437, 86.078487539387, 86.33629332569464, 86.40790604411343, 86.55113148095101, 86.56545402463478, 86.72300200515612, 86.86622744199369, 86.93784016041248, 86.96648524778, 87.02377542251504, 87.09538814093382, 87.18132340303637, 87.2386135777714, 87.38183901460899, 87.45345173302779, 87.5823546261816, 87.61099971354912, 87.63964480091664, 87.73990260670294, 87.89745058722428, 87.91177313090805, 87.91177313090805, 88.08364365511314, 88.08364365511314, 88.1695789172157, 88.21254654826697, 88.31280435405327, 88.38441707247206, 88.47035233457461, 88.58493268404469, 88.64222285877972, 88.72815812088226, 88.79977083930106, 88.82841592666857, 89.00028645087367, 89.0575766256087, 89.0575766256087, 89.04325408192494, 89.10054425665999, 89.18647951876252, 89.25809223718132, 89.31538241191636, 89.45860784875393, 89.51589802348897, 89.48725293612145, 89.48725293612145, 89.55886565454024, 89.65912346032655, 89.6448009166428, 89.71641363506158, 89.80234889716414, 89.8166714408479, 89.83099398453164, 89.88828415926669, 89.85963907189917, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.39845316528216, "exact_area100": 85.28595362581079}, "test": {"score": [76.42247255506925, 76.79558011049724, 77.08976106766163, 77.41981775130947, 77.63507211021023, 77.9364282126713, 78.22343402453899, 78.42433809284638, 78.61089187056038, 78.89789768242808, 79.19207863959245, 79.39298270789983, 79.65846308387745, 79.95264404104184, 80.27552557939298, 80.47642964770037, 80.78496089545814, 81.11501757910598, 81.49530027983067, 81.91145870703882, 82.23434024538997, 82.42089402310397, 82.69354954437827, 82.82987730501543, 83.00925593743274, 83.28191145870704, 83.46846523642104, 83.73394561239866, 83.9635502618928, 84.0640022960465, 84.23620578316711, 84.46581043266126, 84.71694051804549, 84.87479371457273, 85.1617995264404, 85.33400301356102, 85.59230824424195, 85.77168687665926, 85.97259094496664, 86.20219559446079, 86.35287364569133, 86.5753031498888, 86.64705460285572, 86.80490779938295, 87.02016215828371, 87.07756332065725, 87.2067159359977, 87.3143431154481, 87.37891942311832, 87.52242232905216, 87.63722465379924, 87.72332639735954, 87.88117959388678, 87.92423046566694, 87.98880677333716, 88.01750735452393, 88.10360909808423, 88.20406113223794, 88.29016287579825, 88.412140345842, 88.43366578173209, 88.55564325177585, 88.59151897825932, 88.66327043122624, 88.72067159359977, 88.77807275597331, 88.93592595250054, 89.02202769606085, 89.07942885843438, 89.1440051661046, 89.18705603788477, 89.30185836263185, 89.38078496089545, 89.4238358326756, 89.44536126856569, 89.52428786682931, 89.6103896103896, 89.66779077276314, 89.68931620865322, 89.77541795221353, 89.88304513166392, 89.8902202769606, 89.92609600344407, 89.94044629403746, 89.96197172992754, 89.99067231111431, 90.04089832819116, 90.13417521704815, 90.19157637942169, 90.21310181531177, 90.29920355887207, 90.33507928535553, 90.3566047212456, 90.39965559302576, 90.44270646480592, 90.47858219128938, 90.54315849895961, 90.60773480662984, 90.66513596900337, 90.71536198608023, 90.73688742197031], "calls": [0.0, 0.9112434526799168, 1.6502834182392192, 2.712204922149674, 3.888928750807204, 4.771471622300352, 5.661189639090192, 6.357178732869341, 7.139269570208796, 7.985936715218483, 8.875654732008323, 9.88735021884193, 10.942096577455693, 11.939441773695918, 12.922436679342756, 13.704527516682214, 14.960177943603359, 16.187127789337737, 17.213173566764727, 18.375547104828875, 19.401592882255866, 20.291310899045705, 21.331706967066083, 22.343402453899692, 23.39097366721676, 24.49594604290737, 25.306737461433592, 26.49781158068451, 27.50950706751812, 28.11939441773696, 29.095214178087108, 30.164310827294255, 31.197531750017937, 32.08724976680778, 33.40030135610246, 34.333070244672456, 35.473918346846524, 36.40668723541652, 37.48295902992035, 38.645332567984504, 39.73595465308172, 40.79787615699218, 41.59431728492502, 42.57731219057186, 43.78991174571285, 44.36392336944823, 45.33974312979838, 46.48059123197245, 47.47793642821267, 48.633134820980125, 49.63048001722035, 50.74262753820765, 51.88347564038172, 52.98844801607232, 53.95709263112578, 54.775059194948696, 55.67912750233192, 56.54014493793499, 57.5590155700653, 58.53483533041543, 59.4102030566119, 60.479299705819045, 61.412068594389034, 62.3807132094425, 63.37088326038602, 64.15297409772548, 65.37274879816316, 66.29834254143645, 67.27416230178662, 68.32173351510367, 69.32625385664059, 70.47427710411137, 71.72275238573582, 72.6555212743058, 73.63851617995265, 74.6071607950061, 75.50405395709264, 76.80275525579393, 77.67812298199038, 78.596541579967, 79.83066657099806, 80.79213604075483, 81.87558298055536, 82.61462294611465, 83.5976178517615, 84.37253354380427, 85.5492573724618, 86.91971012413, 87.60852407261247, 88.74937217478654, 89.81129367869698, 90.81581402023392, 91.92796154122121, 92.80332926741767, 93.76479873717443, 94.72626820693118, 95.7451388390615, 96.7640094711918, 97.86898184688239, 98.973954222573, 100.0], "exact_score": [76.42247255506925, 76.80993040109063, 77.24043911889216, 77.4700437683863, 77.663772691397, 78.00100452034154, 78.30236062280261, 78.56784099878023, 78.90507282772477, 79.22795436607592, 79.40015785319653, 79.66563822917414, 79.96699433163522, 80.29705101528306, 80.58405682715075, 80.77778575016144, 81.06479156202914, 81.39484824567698, 81.76078065580829, 82.10518763004951, 82.36349286073043, 82.62897323670805, 82.80117672382866, 82.91597904857574, 83.18863456985004, 83.38953863815742, 83.58326756116811, 83.84874793714573, 84.04965200545311, 84.22903063787042, 84.41558441558442, 84.68106479156204, 84.85326827868265, 85.06134749228671, 85.28377699648418, 85.52055679127503, 85.66405969720887, 85.87931405610964, 86.06586783382365, 86.21654588505417, 86.39592451747149, 86.5753031498888, 86.69728061993256, 86.85513381645978, 87.0416875941738, 87.14213962832747, 87.27846738896463, 87.34304369663487, 87.41479514960177, 87.57264834612901, 87.6731003802827, 87.74485183324963, 87.88835473918347, 87.92423046566694, 87.98880677333716, 88.0462079357107, 88.1394848245677, 88.26146229461148, 88.35473918346847, 88.43366578173209, 88.47671665351223, 88.54846810647916, 88.62739470474277, 88.72067159359977, 88.76372246537993, 88.90005022601709, 88.99332711487408, 89.065078567841, 89.158355456698, 89.17988089258809, 89.251632345555, 89.3377340891153, 89.38796010619215, 89.44536126856569, 89.46688670445577, 89.55298844801607, 89.63909019157639, 89.67496591805984, 89.73954222573008, 89.84716940518047, 89.8973954222573, 89.8902202769606, 89.93327114874077, 89.98349716581761, 89.96914687522423, 90.02654803759776, 90.06242376408122, 90.14852550764154, 90.24180239649853, 90.24180239649853, 90.31355384946545, 90.33507928535553, 90.3637798665423, 90.42118102891584, 90.44270646480592, 90.49293248188276, 90.54315849895961, 90.59338451603645, 90.67231111430006, 90.72253713137691, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 79.93829375044845, "exact_area100": 86.19320336691365}}, "old_normalized_endpoint_budget": 100}, {"seed": 2030, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 1.0, "validation_area100": 0.8555663164953117, "training_seconds": 28.21637722395826, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5842243490022546, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5360357270520772, "operator": "ge"}, {"budget": 0.03, "threshold": 0.49843719857106267, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4651323666151262, "operator": "ge"}, {"budget": 0.05, "threshold": 0.43056718837852104, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4030753597165743, "operator": "ge"}, {"budget": 0.07, "threshold": 0.38149228184402395, "operator": "ge"}, {"budget": 0.08, "threshold": 0.3589522539259127, "operator": "ge"}, {"budget": 0.09, "threshold": 0.3383755450044739, "operator": "ge"}, {"budget": 0.1, "threshold": 0.31413476024323705, "operator": "ge"}, {"budget": 0.11, "threshold": 0.2935480480450447, "operator": "ge"}, {"budget": 0.12, "threshold": 0.27019315398771127, "operator": "ge"}, {"budget": 0.13, "threshold": 0.25007490227704166, "operator": "ge"}, {"budget": 0.14, "threshold": 0.2294966694501547, "operator": "ge"}, {"budget": 0.15, "threshold": 0.20576449833176508, "operator": "ge"}, {"budget": 0.16, "threshold": 0.19024321127986352, "operator": "ge"}, {"budget": 0.17, "threshold": 0.17467779328540511, "operator": "ge"}, {"budget": 0.18, "threshold": 0.15995230316485984, "operator": "ge"}, {"budget": 0.19, "threshold": 0.14017446907409128, "operator": "ge"}, {"budget": 0.2, "threshold": 0.11772692012175441, "operator": "ge"}, {"budget": 0.21, "threshold": 0.09186171474103072, "operator": "ge"}, {"budget": 0.22, "threshold": 0.06627271009465188, "operator": "ge"}, {"budget": 0.23, "threshold": 0.034640325405081114, "operator": "ge"}, {"budget": 0.24, "threshold": 0.007896499599893874, "operator": "ge"}, {"budget": 0.25, "threshold": -0.02257365562230805, "operator": "ge"}, {"budget": 0.26, "threshold": -0.05377440673578487, "operator": "ge"}, {"budget": 0.27, "threshold": -0.08902128924411616, "operator": "ge"}, {"budget": 0.28, "threshold": -0.12407271283569261, "operator": "ge"}, {"budget": 0.29, "threshold": -0.16143978746751883, "operator": "ge"}, {"budget": 0.3, "threshold": -0.195255225609809, "operator": "ge"}, {"budget": 0.31, "threshold": -0.2319984440950185, "operator": "ge"}, {"budget": 0.32, "threshold": -0.2621115867064383, "operator": "ge"}, {"budget": 0.33, "threshold": -0.30235544146950827, "operator": "ge"}, {"budget": 0.34, "threshold": -0.33148139829507245, "operator": "ge"}, {"budget": 0.35, "threshold": -0.36917189595950317, "operator": "ge"}, {"budget": 0.36, "threshold": -0.4047597125839881, "operator": "ge"}, {"budget": 0.37, "threshold": -0.44577921922090546, "operator": "ge"}, {"budget": 0.38, "threshold": -0.4899850069895278, "operator": "ge"}, {"budget": 0.39, "threshold": -0.5338131808056281, "operator": "ge"}, {"budget": 0.4, "threshold": -0.5768678123161051, "operator": "ge"}, {"budget": 0.41, "threshold": -0.6251378109484753, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6709570039597595, "operator": "ge"}, {"budget": 0.43, "threshold": -0.7268145712354559, "operator": "ge"}, {"budget": 0.44, "threshold": -0.7614586594925796, "operator": "ge"}, {"budget": 0.45, "threshold": -0.7997926157408592, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8276005599246962, "operator": "ge"}, {"budget": 0.47, "threshold": -0.847790970766833, "operator": "ge"}, {"budget": 0.48, "threshold": -0.8692226621128811, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8841470829919669, "operator": "ge"}, {"budget": 0.5, "threshold": -0.897399350662401, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9063839111325864, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9137548996354882, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9204234187814876, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9271955812422351, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9325270751106957, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9367878116847781, "operator": "ge"}, {"budget": 0.57, "threshold": -0.940616434354865, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9440581619526935, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9466620181336647, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9495013709076773, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9520978487846831, "operator": "ge"}, {"budget": 0.62, "threshold": -0.954646699249766, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9565975351461004, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9584169200120854, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9603381206571087, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9621685086687249, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9637286544830254, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9652401214603803, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9663371146166871, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9676739359745198, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9688758388278582, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9700319399950385, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9712515958780408, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9722483776080899, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9732892462357665, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9743799243309085, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9753090579243102, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9762765621605721, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9771572980800807, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9778911976217817, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9788168417002052, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9797284664687528, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9803040505380016, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9808911076167953, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9816320278128717, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9823719530979281, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9831261582362764, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9838833812356735, "operator": "ge"}, {"budget": 0.89, "threshold": -0.984645391736119, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9853648432799272, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9859914137946284, "operator": "ge"}, {"budget": 0.92, "threshold": -0.986548107784097, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9871360928447386, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9879046230223795, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9887455762020881, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9896125108981162, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9903389102981285, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9914367001565308, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9927928274576011, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.7115439702091, 76.99799484388427, 77.35605843597823, 77.54225150386709, 77.85734746490976, 78.21541105700372, 78.6307648238327, 78.93153824119163, 79.2179891148668, 79.50443998854196, 79.74792323116586, 79.94843884273847, 80.20624462904613, 80.57863076482383, 80.80779146376396, 81.10856488112287, 81.29475794901174, 81.49527356058435, 81.6814666284732, 81.92494987109711, 82.0681753079347, 82.32598109424234, 82.51217416213119, 82.78430249212259, 82.94185047264394, 83.21397880263535, 83.32855915210541, 83.51475221999426, 83.72959037525064, 83.88713835577198, 84.20223431681467, 84.48868519048983, 84.73216843311371, 84.98997421942136, 85.17616728731022, 85.40532798625036, 85.72042394729303, 85.82068175307934, 86.078487539387, 86.25035806359209, 86.36493841306215, 86.65138928673733, 86.76596963620739, 86.92351761672873, 86.92351761672873, 87.09538814093382, 87.2386135777714, 87.22429103408766, 87.26725866513893, 87.32454883987396, 87.39616155829275, 87.5107419077628, 87.68261243196791, 87.73990260670294, 87.84016041248925, 87.94041821827557, 87.95474076195933, 88.02635348037812, 88.06932111142939, 88.14093382984818, 88.12661128616442, 88.19822400458321, 88.31280435405327, 88.35577198510455, 88.42738470352333, 88.52764250930964, 88.59925522772845, 88.62790031509596, 88.75680320824978, 88.84273847035233, 88.92867373245488, 88.95731881982239, 88.92867373245488, 88.95731881982239, 88.97164136350615, 89.10054425665999, 89.1291893440275, 89.18647951876252, 89.27241478086508, 89.31538241191636, 89.35835004296763, 89.45860784875393, 89.47293039243769, 89.47293039243769, 89.48725293612145, 89.5445431108565, 89.55886565454024, 89.573188198224, 89.61615582927529, 89.65912346032655, 89.7450587224291, 89.7450587224291, 89.84531652821542, 89.83099398453164, 89.84531652821542, 89.87396161558291, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 60, 89, 113, 141, 162, 187, 217, 249, 274, 299, 321, 346, 371, 395, 426, 450, 474, 495, 518, 541, 564, 584, 607, 624, 647, 667, 688, 703, 719, 743, 760, 788, 809, 828, 849, 867, 887, 910, 923, 941, 956, 972, 996, 1008, 1021, 1029, 1047, 1059, 1064, 1075, 1085, 1097, 1111, 1124, 1132, 1143, 1151, 1157, 1168, 1173, 1180, 1186, 1192, 1200, 1204, 1213, 1222, 1230, 1235, 1247, 1257, 1266, 1274, 1276, 1281, 1287, 1297, 1302, 1307, 1315, 1319, 1324, 1331, 1335, 1338, 1342, 1346, 1348, 1352, 1356, 1360, 1366, 1369, 1378, 1380, 1382, 1385, 1387, 1389], "harmed": [0, 2, 7, 9, 13, 16, 24, 27, 32, 35, 39, 44, 46, 54, 65, 71, 76, 84, 87, 95, 104, 114, 120, 130, 135, 139, 143, 152, 154, 161, 164, 173, 179, 185, 186, 188, 191, 196, 200, 201, 207, 207, 210, 218, 222, 226, 228, 236, 242, 244, 250, 258, 264, 271, 277, 278, 282, 286, 287, 292, 298, 300, 302, 309, 310, 310, 311, 315, 317, 320, 323, 326, 330, 333, 339, 343, 346, 351, 352, 355, 356, 358, 359, 361, 361, 364, 367, 370, 370, 371, 374, 375, 376, 376, 379, 381, 384, 385, 386, 386, 388], "exact_score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.7115439702091, 76.99799484388427, 77.35605843597823, 77.54225150386709, 77.85734746490976, 78.21541105700372, 78.6307648238327, 78.93153824119163, 79.2179891148668, 79.50443998854196, 79.74792323116586, 79.94843884273847, 80.20624462904613, 80.57863076482383, 80.80779146376396, 81.10856488112287, 81.29475794901174, 81.49527356058435, 81.6814666284732, 81.92494987109711, 82.0681753079347, 82.32598109424234, 82.51217416213119, 82.78430249212259, 82.94185047264394, 83.21397880263535, 83.32855915210541, 83.51475221999426, 83.72959037525064, 83.88713835577198, 84.20223431681467, 84.48868519048983, 84.73216843311371, 84.98997421942136, 85.17616728731022, 85.40532798625036, 85.72042394729303, 85.82068175307934, 86.078487539387, 86.25035806359209, 86.36493841306215, 86.65138928673733, 86.76596963620739, 86.92351761672873, 86.92351761672873, 87.09538814093382, 87.2386135777714, 87.22429103408766, 87.26725866513893, 87.32454883987396, 87.39616155829275, 87.5107419077628, 87.68261243196791, 87.73990260670294, 87.84016041248925, 87.94041821827557, 87.95474076195933, 88.02635348037812, 88.06932111142939, 88.14093382984818, 88.12661128616442, 88.19822400458321, 88.31280435405327, 88.35577198510455, 88.42738470352333, 88.52764250930964, 88.59925522772845, 88.62790031509596, 88.75680320824978, 88.84273847035233, 88.92867373245488, 88.95731881982239, 88.92867373245488, 88.95731881982239, 88.97164136350615, 89.10054425665999, 89.1291893440275, 89.18647951876252, 89.27241478086508, 89.31538241191636, 89.35835004296763, 89.45860784875393, 89.47293039243769, 89.47293039243769, 89.48725293612145, 89.5445431108565, 89.55886565454024, 89.573188198224, 89.61615582927529, 89.65912346032655, 89.7450587224291, 89.7450587224291, 89.84531652821542, 89.83099398453164, 89.84531652821542, 89.87396161558291, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.55663164953117, "threshold_area100": 85.55564308221139, "threshold_area25": 79.40561443712404}, "test": {"score": [76.42247255506925, 76.74535409342039, 77.05388534117816, 77.32654086245246, 77.67812298199038, 78.00100452034154, 78.34541149458278, 78.55349070818684, 78.81179593886777, 79.11315204132883, 79.40015785319653, 79.75173997273444, 80.0746215110856, 80.50513022888714, 80.71320944249122, 81.05761641673243, 81.22981990385306, 81.45224940805052, 81.63162804046782, 81.86123268996197, 82.10518763004951, 82.2989165530602, 82.52852120255434, 82.88010332209228, 83.07383224510296, 83.23886058692689, 83.46129009112435, 83.75547104828873, 84.04965200545311, 84.40840927028772, 84.63083877448518, 84.83174284279256, 84.95372031283634, 85.19050010762719, 85.39857932123127, 85.71428571428572, 85.88648920140633, 86.1878453038674, 86.36722393628472, 86.5753031498888, 86.66858003874579, 86.81925808997633, 86.99863672239363, 87.2067159359977, 87.41479514960177, 87.63004950850255, 87.79507785032648, 87.98163162804047, 88.16818540575447, 88.27581258520485, 88.39779005524862, 88.46954150821554, 88.52694267058908, 88.61304441414939, 88.75654732008323, 88.89287508072039, 88.91440051661046, 88.96462653368731, 89.04355313195093, 89.10812943962115, 89.16553060199469, 89.24445720025831, 89.33055894381862, 89.40948554208224, 89.45971155915908, 89.56016359331277, 89.60321446509292, 89.71084164454331, 89.76106766162016, 89.82564396929038, 89.8902202769606, 89.93327114874077, 89.96197172992754, 90.03372318289445, 90.0982994905647, 90.14852550764154, 90.14135036234484, 90.16287579823492, 90.19875152471838, 90.24897754179523, 90.25615268709191, 90.32072899476215, 90.29202841357538, 90.3637798665423, 90.39965559302576, 90.42835617421254, 90.42118102891584, 90.44270646480592, 90.52163306306953, 90.58620937073975, 90.62926024251992, 90.66513596900337, 90.66513596900337, 90.65796082370669, 90.65078567840999, 90.65796082370669, 90.71536198608023, 90.7297122766736, 90.72253713137691, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.054746358613762, 1.9301140848102174, 2.812656956303365, 3.84587787902705, 4.871923656454044, 5.783167109133959, 6.651359690033723, 7.569778288010333, 8.45232115950348, 9.399440338666858, 10.32503408194016, 11.372605295257229, 12.54932912391476, 13.517973738968214, 14.780799311186051, 15.627466456195737, 16.316280404678192, 17.12707182320442, 18.188993327114876, 19.28679055750879, 20.162158283705246, 21.202554351725624, 22.422329052163306, 23.32639735954653, 24.22329052163306, 25.09865824782952, 26.239506350003587, 27.36600416158427, 28.628829733802107, 29.690751237712565, 30.781373322809785, 31.785893664346705, 32.95544234770754, 33.80928463801392, 34.99318361196814, 35.98335366291167, 37.08115089330558, 38.171772978402814, 39.03996555930257, 39.91533328549903, 40.81940159288225, 41.75934562674894, 42.97912032718663, 44.03386668580039, 45.09578818971084, 46.164884838917985, 47.10482887278468, 48.26720241084882, 49.2143215900122, 50.3264691109995, 51.25923799956949, 52.141780871062636, 53.01714859725909, 54.165171844729855, 55.16969218626677, 56.08811078424338, 57.30071033938437, 58.24782951854775, 59.04427064648059, 59.9626892444572, 61.010260457774265, 62.29461146588219, 63.29195666212241, 64.38257874721963, 65.4301499605367, 66.57817320800746, 67.5611681136543, 68.58721389108129, 69.38365501901413, 70.40252565114443, 71.36399512090121, 72.40439118892158, 73.33716007749157, 74.31297983784172, 75.33185046997201, 76.37942168328908, 77.43416804190285, 78.596541579967, 79.55801104972376, 80.49795508359044, 81.71055463873144, 83.02360622802611, 83.7769964841788, 84.66671450096864, 85.69993542369232, 86.71880605582263, 87.80225299562316, 88.95027624309392, 90.09829949056468, 91.13152041328837, 92.10016502834183, 92.84638013919782, 93.75762359187773, 94.72626820693118, 95.87429145440196, 96.69943316352156, 97.31649565903709, 98.27796512879385, 99.06723111143, 100.0], "rescued": [0, 49, 98, 142, 197, 249, 303, 341, 387, 438, 487, 540, 594, 660, 699, 759, 793, 832, 872, 921, 971, 1008, 1054, 1116, 1153, 1186, 1224, 1279, 1330, 1387, 1428, 1469, 1499, 1547, 1581, 1635, 1673, 1720, 1757, 1789, 1814, 1842, 1873, 1911, 1945, 1984, 2012, 2042, 2080, 2104, 2127, 2148, 2163, 2184, 2213, 2238, 2250, 2265, 2282, 2295, 2308, 2329, 2349, 2364, 2379, 2397, 2412, 2432, 2449, 2462, 2477, 2488, 2499, 2510, 2524, 2534, 2540, 2550, 2563, 2572, 2578, 2589, 2594, 2605, 2615, 2621, 2628, 2635, 2646, 2655, 2665, 2673, 2676, 2677, 2683, 2687, 2695, 2698, 2702, 2702, 2706], "harmed": [0, 4, 10, 16, 22, 29, 35, 44, 54, 63, 72, 76, 85, 91, 101, 113, 123, 131, 146, 163, 179, 189, 203, 216, 226, 236, 243, 257, 267, 274, 284, 297, 310, 325, 330, 340, 354, 359, 371, 374, 386, 393, 399, 408, 413, 422, 427, 431, 443, 452, 458, 469, 476, 485, 494, 500, 509, 517, 523, 527, 532, 542, 550, 554, 562, 566, 575, 580, 590, 594, 600, 605, 612, 613, 618, 621, 628, 635, 643, 645, 650, 652, 661, 662, 667, 669, 677, 681, 681, 681, 685, 688, 691, 693, 700, 703, 703, 704, 709, 710, 711], "exact_score": [76.42247255506925, 76.71665351223362, 77.08258592236493, 77.3982923154194, 77.73552414436392, 78.02970510152831, 78.409987802253, 78.67546817823062, 78.99834971658176, 79.29970581904284, 79.65128793858077, 79.9741694769319, 80.26835043409629, 80.61993255363421, 80.87823778431513, 81.13654301499605, 81.33027193800675, 81.61010260457775, 81.81818181818183, 82.06931190356606, 82.27021597187343, 82.49264547607089, 82.74377556145512, 83.02360622802611, 83.18145942455335, 83.41823921934419, 83.69806988591519, 83.97072540718949, 84.17880462079357, 84.50168615914473, 84.68823993685872, 84.8891440051661, 84.99677118461649, 85.19767525292387, 85.44880533830809, 85.71428571428572, 85.90083949199972, 86.16631986797732, 86.33134820980125, 86.5609528592954, 86.66858003874579, 86.89818468823994, 87.0416875941738, 87.2067159359977, 87.40762000430509, 87.5941737820191, 87.75920212384301, 87.945755901557, 88.15383511516109, 88.24711200401808, 88.36908947406185, 88.46954150821554, 88.5125923799957, 88.6058692688527, 88.74937217478654, 88.90005022601709, 88.90722537131377, 88.95745138839062, 89.02202769606085, 89.08660400373107, 89.16553060199469, 89.23728205496163, 89.27315778144508, 89.40231039678554, 89.4310109779723, 89.54581330271938, 89.56016359331277, 89.66779077276314, 89.74671737102676, 89.78976824280693, 89.83999425988375, 89.91174571285069, 89.96197172992754, 90.01937289230108, 90.0839491999713, 90.13417521704815, 90.16287579823492, 90.14135036234484, 90.16287579823492, 90.23462725120184, 90.24180239649853, 90.26332783238861, 90.30637870416876, 90.29202841357538, 90.37095501183899, 90.41400588361914, 90.44270646480592, 90.42835617421254, 90.46423190069599, 90.52163306306953, 90.58620937073975, 90.62926024251992, 90.65078567840999, 90.67231111430006, 90.65796082370669, 90.64361053311329, 90.65796082370669, 90.70818684078353, 90.72253713137691, 90.71536198608023, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.53071938041754, "threshold_area100": 86.49799095931692, "threshold_area25": 80.0789265982636}, "selected_budget": 99, "selected": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692, "rescued": 1387, "harmed": 386}, "test": {"score": 90.71536198608023, "calls": 99.06723111143, "rescued": 2702, "harmed": 710}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.36802196502685547, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3364613652229309, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3076219856739044, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2866649627685547, "operator": "ge"}, {"budget": 0.05, "threshold": 0.2669260501861572, "operator": "ge"}, {"budget": 0.06, "threshold": 0.24608178436756134, "operator": "ge"}, {"budget": 0.07, "threshold": 0.22900252044200897, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2145807445049286, "operator": "ge"}, {"budget": 0.09, "threshold": 0.200770765542984, "operator": "ge"}, {"budget": 0.1, "threshold": 0.18697942793369293, "operator": "ge"}, {"budget": 0.11, "threshold": 0.17438936233520508, "operator": "ge"}, {"budget": 0.12, "threshold": 0.16209855675697327, "operator": "ge"}, {"budget": 0.13, "threshold": 0.1464288830757141, "operator": "ge"}, {"budget": 0.14, "threshold": 0.13338233530521393, "operator": "ge"}, {"budget": 0.15, "threshold": 0.12043939530849457, "operator": "ge"}, {"budget": 0.16, "threshold": 0.10855291783809662, "operator": "ge"}, {"budget": 0.17, "threshold": 0.09373757243156433, "operator": "ge"}, {"budget": 0.18, "threshold": 0.07969363778829575, "operator": "ge"}, {"budget": 0.19, "threshold": 0.06750082224607468, "operator": "ge"}, {"budget": 0.2, "threshold": 0.054041698575019836, "operator": "ge"}, {"budget": 0.21, "threshold": 0.03793124854564667, "operator": "ge"}, {"budget": 0.22, "threshold": 0.027431800961494446, "operator": "ge"}, {"budget": 0.23, "threshold": 0.014453276991844177, "operator": "ge"}, {"budget": 0.24, "threshold": 0.0029264092445373535, "operator": "ge"}, {"budget": 0.25, "threshold": -0.006324693560600281, "operator": "ge"}, {"budget": 0.26, "threshold": -0.014724187552928925, "operator": "ge"}, {"budget": 0.27, "threshold": -0.018076110631227493, "operator": "ge"}, {"budget": 0.28, "threshold": -0.019747110083699226, "operator": "ge"}, {"budget": 0.29, "threshold": -0.021329104900360107, "operator": "ge"}, {"budget": 0.3, "threshold": -0.022485317662358284, "operator": "ge"}, {"budget": 0.31, "threshold": -0.02381652593612671, "operator": "ge"}, {"budget": 0.32, "threshold": -0.02503300830721855, "operator": "ge"}, {"budget": 0.33, "threshold": -0.026052650064229965, "operator": "ge"}, {"budget": 0.34, "threshold": -0.02745874598622322, "operator": "ge"}, {"budget": 0.35, "threshold": -0.028679482638835907, "operator": "ge"}, {"budget": 0.36, "threshold": -0.029836203902959824, "operator": "ge"}, {"budget": 0.37, "threshold": -0.030947692692279816, "operator": "ge"}, {"budget": 0.38, "threshold": -0.03214825317263603, "operator": "ge"}, {"budget": 0.39, "threshold": -0.033256061375141144, "operator": "ge"}, {"budget": 0.4, "threshold": -0.03432794660329819, "operator": "ge"}, {"budget": 0.41, "threshold": -0.035573795437812805, "operator": "ge"}, {"budget": 0.42, "threshold": -0.03675921633839607, "operator": "ge"}, {"budget": 0.43, "threshold": -0.037895869463682175, "operator": "ge"}, {"budget": 0.44, "threshold": -0.03896808251738548, "operator": "ge"}, {"budget": 0.45, "threshold": -0.040379829704761505, "operator": "ge"}, {"budget": 0.46, "threshold": -0.041477229446172714, "operator": "ge"}, {"budget": 0.47, "threshold": -0.04274400696158409, "operator": "ge"}, {"budget": 0.48, "threshold": -0.043990880250930786, "operator": "ge"}, {"budget": 0.49, "threshold": -0.04539339989423752, "operator": "ge"}, {"budget": 0.5, "threshold": -0.04677155613899231, "operator": "ge"}, {"budget": 0.51, "threshold": -0.0482286661863327, "operator": "ge"}, {"budget": 0.52, "threshold": -0.04933500289916992, "operator": "ge"}, {"budget": 0.53, "threshold": -0.050618771463632584, "operator": "ge"}, {"budget": 0.54, "threshold": -0.05195322260260582, "operator": "ge"}, {"budget": 0.55, "threshold": -0.05332297086715698, "operator": "ge"}, {"budget": 0.56, "threshold": -0.05446885526180267, "operator": "ge"}, {"budget": 0.57, "threshold": -0.055942125618457794, "operator": "ge"}, {"budget": 0.58, "threshold": -0.057401496917009354, "operator": "ge"}, {"budget": 0.59, "threshold": -0.05900324508547783, "operator": "ge"}, {"budget": 0.6, "threshold": -0.06036965548992157, "operator": "ge"}, {"budget": 0.61, "threshold": -0.06167006120085716, "operator": "ge"}, {"budget": 0.62, "threshold": -0.06296135485172272, "operator": "ge"}, {"budget": 0.63, "threshold": -0.06412673741579056, "operator": "ge"}, {"budget": 0.64, "threshold": -0.06562349200248718, "operator": "ge"}, {"budget": 0.65, "threshold": -0.0669652670621872, "operator": "ge"}, {"budget": 0.66, "threshold": -0.06832138448953629, "operator": "ge"}, {"budget": 0.67, "threshold": -0.07005409896373749, "operator": "ge"}, {"budget": 0.68, "threshold": -0.07149534672498703, "operator": "ge"}, {"budget": 0.69, "threshold": -0.07303737848997116, "operator": "ge"}, {"budget": 0.7, "threshold": -0.07462871819734573, "operator": "ge"}, {"budget": 0.71, "threshold": -0.07616723328828812, "operator": "ge"}, {"budget": 0.72, "threshold": -0.07762795686721802, "operator": "ge"}, {"budget": 0.73, "threshold": -0.07967174798250198, "operator": "ge"}, {"budget": 0.74, "threshold": -0.08125492930412292, "operator": "ge"}, {"budget": 0.75, "threshold": -0.08286802470684052, "operator": "ge"}, {"budget": 0.76, "threshold": -0.08449137955904007, "operator": "ge"}, {"budget": 0.77, "threshold": -0.08594485372304916, "operator": "ge"}, {"budget": 0.78, "threshold": -0.08778201043605804, "operator": "ge"}, {"budget": 0.79, "threshold": -0.0895085334777832, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0909561887383461, "operator": "ge"}, {"budget": 0.81, "threshold": -0.09249917417764664, "operator": "ge"}, {"budget": 0.82, "threshold": -0.09467490762472153, "operator": "ge"}, {"budget": 0.83, "threshold": -0.09680451452732086, "operator": "ge"}, {"budget": 0.84, "threshold": -0.09865304082632065, "operator": "ge"}, {"budget": 0.85, "threshold": -0.10093680024147034, "operator": "ge"}, {"budget": 0.86, "threshold": -0.10289013385772705, "operator": "ge"}, {"budget": 0.87, "threshold": -0.10518992692232132, "operator": "ge"}, {"budget": 0.88, "threshold": -0.10720427334308624, "operator": "ge"}, {"budget": 0.89, "threshold": -0.10918629169464111, "operator": "ge"}, {"budget": 0.9, "threshold": -0.11191540956497192, "operator": "ge"}, {"budget": 0.91, "threshold": -0.11502347141504288, "operator": "ge"}, {"budget": 0.92, "threshold": -0.11830385774374008, "operator": "ge"}, {"budget": 0.93, "threshold": -0.1212974339723587, "operator": "ge"}, {"budget": 0.94, "threshold": -0.12537498772144318, "operator": "ge"}, {"budget": 0.95, "threshold": -0.12893524765968323, "operator": "ge"}, {"budget": 0.96, "threshold": -0.13330699503421783, "operator": "ge"}, {"budget": 0.97, "threshold": -0.13946491479873657, "operator": "ge"}, {"budget": 0.98, "threshold": -0.14633698761463165, "operator": "ge"}, {"budget": 0.99, "threshold": -0.15542080998420715, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.9238040676024, 76.28186765969636, 76.65425379547408, 76.86909195073045, 77.29876826124318, 77.65683185333715, 78.02921798911487, 78.30134631910627, 78.61644228014895, 79.00315095961042, 79.27527928960183, 79.54740761959323, 79.71927814379833, 80.03437410484102, 80.33514752219993, 80.47837295903753, 80.63592093955886, 80.80779146376396, 81.10856488112287, 81.32340303637926, 81.59553136637066, 81.92494987109711, 82.12546548266972, 82.3689487252936, 82.54081924949871, 82.75565740475508, 82.8845602979089, 82.8845602979089, 82.91320538527643, 82.92752792896017, 82.91320538527643, 82.92752792896017, 82.97049556001146, 82.97049556001146, 83.04210827843025, 83.08507590948152, 83.2283013463191, 83.29991406473789, 83.34288169578917, 83.42881695789171, 83.55771985104555, 83.61501002578058, 83.70094528788312, 83.75823546261816, 83.85849326840446, 83.88713835577198, 83.9873961615583, 84.07333142366083, 84.11629905471212, 84.30249212260097, 84.38842738470352, 84.46004010312231, 84.58894299627615, 84.66055571469494, 84.68920080206244, 84.81810369521627, 84.8753938699513, 85.00429676310513, 85.1475221999427, 85.23345746204527, 85.3910054425666, 85.5628759667717, 85.63448868519049, 85.74906903466055, 85.9352621025494, 86.12145517043827, 86.20739043254082, 86.30764823832713, 86.42222858779719, 86.55113148095101, 86.59409911200228, 86.72300200515612, 86.79461472357491, 86.88054998567746, 86.95216270409625, 87.03809796619879, 87.10971068461758, 87.2099684904039, 87.32454883987396, 87.39616155829275, 87.53938699513033, 87.59667716986537, 87.79719278143799, 87.9977083930106, 88.11228874248066, 88.22686909195072, 88.31280435405327, 88.34144944142079, 88.48467487825837, 88.62790031509596, 88.84273847035233, 88.91435118877112, 89.11486680034373, 89.14351188771126, 89.18647951876252, 89.28673732454884, 89.48725293612145, 89.58751074190776, 89.71641363506158, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 30, 59, 88, 109, 142, 168, 200, 223, 252, 287, 314, 337, 357, 383, 408, 428, 445, 465, 492, 513, 540, 567, 589, 609, 627, 644, 653, 656, 660, 661, 661, 663, 667, 670, 677, 681, 691, 698, 703, 711, 721, 728, 735, 742, 751, 758, 767, 777, 781, 794, 804, 811, 822, 830, 837, 848, 857, 869, 881, 891, 906, 918, 928, 941, 956, 970, 979, 991, 1003, 1016, 1023, 1034, 1044, 1055, 1067, 1075, 1085, 1097, 1108, 1117, 1130, 1136, 1150, 1168, 1180, 1192, 1204, 1212, 1227, 1243, 1260, 1274, 1290, 1299, 1311, 1323, 1341, 1356, 1371, 1389], "harmed": [0, 5, 9, 12, 18, 21, 22, 28, 32, 39, 47, 55, 59, 67, 71, 75, 85, 91, 99, 105, 111, 119, 123, 131, 134, 140, 142, 142, 145, 147, 147, 148, 149, 150, 153, 155, 156, 156, 158, 160, 162, 163, 166, 167, 170, 172, 177, 179, 183, 184, 184, 188, 190, 192, 195, 200, 202, 207, 210, 212, 216, 220, 220, 225, 230, 232, 233, 236, 241, 245, 249, 253, 255, 260, 265, 272, 274, 279, 284, 287, 291, 294, 296, 296, 300, 304, 308, 314, 320, 325, 331, 333, 342, 344, 351, 360, 365, 369, 377, 383, 388], "exact_score": [75.56574047550845, 75.9238040676024, 76.28186765969636, 76.65425379547408, 76.86909195073045, 77.29876826124318, 77.65683185333715, 78.02921798911487, 78.30134631910627, 78.61644228014895, 79.00315095961042, 79.27527928960183, 79.54740761959323, 79.71927814379833, 80.03437410484102, 80.33514752219993, 80.47837295903753, 80.63592093955886, 80.80779146376396, 81.10856488112287, 81.32340303637926, 81.59553136637066, 81.92494987109711, 82.12546548266972, 82.3689487252936, 82.54081924949871, 82.75565740475508, 82.8845602979089, 82.8845602979089, 82.91320538527643, 82.92752792896017, 82.91320538527643, 82.92752792896017, 82.97049556001146, 82.97049556001146, 83.04210827843025, 83.08507590948152, 83.2283013463191, 83.29991406473789, 83.34288169578917, 83.42881695789171, 83.55771985104555, 83.61501002578058, 83.70094528788312, 83.75823546261816, 83.85849326840446, 83.88713835577198, 83.9873961615583, 84.07333142366083, 84.11629905471212, 84.30249212260097, 84.38842738470352, 84.46004010312231, 84.58894299627615, 84.66055571469494, 84.68920080206244, 84.81810369521627, 84.8753938699513, 85.00429676310513, 85.1475221999427, 85.23345746204527, 85.3910054425666, 85.5628759667717, 85.63448868519049, 85.74906903466055, 85.9352621025494, 86.12145517043827, 86.20739043254082, 86.30764823832713, 86.42222858779719, 86.55113148095101, 86.59409911200228, 86.72300200515612, 86.79461472357491, 86.88054998567746, 86.95216270409625, 87.03809796619879, 87.10971068461758, 87.2099684904039, 87.32454883987396, 87.39616155829275, 87.53938699513033, 87.59667716986537, 87.79719278143799, 87.9977083930106, 88.11228874248066, 88.22686909195072, 88.31280435405327, 88.34144944142079, 88.48467487825837, 88.62790031509596, 88.84273847035233, 88.91435118877112, 89.11486680034373, 89.14351188771126, 89.18647951876252, 89.28673732454884, 89.48725293612145, 89.58751074190776, 89.71641363506158, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 84.17807964023542, "threshold_area100": 84.17709825264966, "threshold_area25": 79.4027499283873}, "test": {"score": [76.42247255506925, 76.72382865753032, 77.02518475999139, 77.3911171701227, 77.63507211021023, 77.95795364856139, 78.26648489631916, 78.6037167252637, 78.89072253713138, 79.12750233192223, 79.50060988735022, 79.72303939154767, 80.0817966563823, 80.34727703235991, 80.56253139126068, 80.74908516897467, 81.06479156202914, 81.34462222860013, 81.59575231398436, 81.8397072540719, 81.99756045059912, 82.2989165530602, 82.55004663844443, 82.8585778862022, 83.06665709980628, 83.21016000574012, 83.45411494582765, 83.53304154409128, 83.61196814235488, 83.64784386883835, 83.72677046710196, 83.74829590299204, 83.79852192006888, 83.87027337303581, 83.95637511659612, 84.02812656956303, 84.08552773193657, 84.12140345842003, 84.17880462079357, 84.27208150965058, 84.3438329626175, 84.44428499677119, 84.56626246681496, 84.6093133385951, 84.63083877448518, 84.74564109923226, 84.8173925521992, 84.8963191504628, 85.05417234699003, 85.14027409055034, 85.21202554351726, 85.31965272296765, 85.3555284494511, 85.41292961182464, 85.43445504771472, 85.47750591949487, 85.55643251775848, 85.62818397072542, 85.80756260314271, 86.00129152615341, 86.08739326971371, 86.19502044916409, 86.22372103035086, 86.35287364569133, 86.42462509865825, 86.48202626103179, 86.5753031498888, 86.67575518404249, 86.73315634641602, 86.91971012413, 87.07038817536056, 87.2138910812944, 87.33586855133817, 87.43632058549186, 87.57264834612901, 87.61569921790917, 87.74485183324963, 87.85247901270002, 87.88835473918347, 87.98163162804047, 88.15383511516109, 88.28298773050155, 88.48389179880894, 88.64174499533615, 88.73502188419316, 88.92875080720385, 89.04355313195093, 89.24445720025831, 89.38796010619215, 89.4238358326756, 89.56016359331277, 89.71801678984, 89.78976824280693, 89.904570567554, 89.96197172992754, 90.0982994905647, 90.24897754179523, 90.40683073832245, 90.5503336442563, 90.70101169548683, 90.73688742197031], "calls": [0.0, 0.9184185979766091, 1.77943603357968, 2.834182392193442, 3.774126426060128, 4.692545024036736, 5.811867690320729, 6.658534835330415, 7.584128578603718, 8.35904427064648, 9.385090048073472, 10.353734663126929, 11.322379278180383, 12.369950491497454, 13.3457702518476, 14.221137978044055, 15.247183755471047, 16.273229532898043, 17.25622443854488, 18.160292745928107, 18.992609600344405, 20.341536916122553, 21.116452608165314, 22.149673530889, 23.319222214249837, 24.165889359259523, 25.242161153763366, 26.311257802970513, 27.179450383870275, 28.083518691253502, 28.714931477362416, 29.640525220635716, 30.594819545095785, 31.283633493578243, 32.467532467532465, 33.57250484322308, 34.50527373179307, 35.45239291095645, 36.43538781660329, 37.41120757695343, 38.343976465523426, 39.484824567697494, 40.62567266987156, 41.63736815670517, 42.64906364353878, 43.832962617493, 44.81595752313984, 45.899404462940375, 47.01872712922437, 48.31025328262897, 49.53002798306665, 50.613474922867184, 51.43144148669011, 52.44313697352372, 53.41178158857717, 54.35172562244386, 55.19121762215685, 56.389466886704454, 57.51596469828514, 58.77879027050298, 59.71873430436966, 60.608452321159504, 61.490995192652655, 62.395063500035874, 63.4856855851331, 64.3108272942527, 65.21489560163593, 66.3485685585133, 67.28851259238, 68.37195953218053, 69.54868336083806, 70.50297768529813, 71.39269570208798, 72.6555212743058, 73.69591734232618, 74.77936428212672, 75.66190715361985, 76.45117313625602, 77.41264260601277, 78.38846236636292, 79.28535552844946, 80.13202267345913, 81.24417019444644, 82.32044198895028, 83.2173351510368, 84.37253354380427, 85.26225156059411, 86.52507713281193, 87.42914544019517, 88.25428714931476, 89.32338379852192, 90.56468393484968, 91.69118174643036, 92.64547607089042, 93.80067446365788, 94.85542082227165, 96.07519552270934, 97.24474420607018, 98.17033794934348, 99.08158140202339, 100.0], "rescued": [0, 46, 95, 152, 193, 242, 298, 351, 400, 440, 499, 545, 602, 650, 697, 733, 786, 834, 881, 926, 959, 1014, 1058, 1111, 1153, 1181, 1221, 1238, 1251, 1260, 1273, 1279, 1289, 1302, 1317, 1329, 1337, 1346, 1355, 1373, 1386, 1402, 1424, 1435, 1449, 1470, 1483, 1500, 1524, 1543, 1557, 1577, 1587, 1601, 1612, 1626, 1643, 1664, 1692, 1721, 1739, 1759, 1771, 1795, 1810, 1823, 1839, 1861, 1878, 1909, 1936, 1960, 1982, 2007, 2036, 2052, 2077, 2096, 2110, 2133, 2162, 2184, 2216, 2245, 2268, 2305, 2326, 2363, 2392, 2407, 2433, 2467, 2487, 2512, 2533, 2564, 2601, 2632, 2660, 2686, 2706], "harmed": [0, 4, 11, 17, 24, 28, 41, 47, 56, 63, 70, 85, 92, 103, 120, 130, 139, 148, 160, 171, 182, 195, 204, 214, 227, 235, 241, 247, 249, 253, 255, 258, 261, 264, 267, 269, 269, 273, 274, 279, 282, 284, 289, 294, 305, 310, 313, 319, 321, 328, 332, 337, 342, 348, 356, 364, 370, 381, 384, 386, 392, 397, 405, 411, 416, 421, 424, 432, 441, 446, 452, 456, 461, 472, 482, 492, 499, 503, 512, 522, 527, 531, 535, 542, 552, 562, 567, 576, 585, 595, 602, 614, 624, 633, 646, 658, 674, 683, 691, 696, 711], "exact_score": [76.42247255506925, 76.75970438401379, 77.08258592236493, 77.43416804190285, 77.72117385377054, 78.01535481093492, 78.30236062280261, 78.68981846882399, 79.03422544306522, 79.37145727200976, 79.7086891009543, 79.93829375044845, 80.2540001435029, 80.46207935710699, 80.68450886130445, 80.96433952787544, 81.30157135681998, 81.53835115161083, 81.80383152758843, 81.99756045059912, 82.21281480949989, 82.49264547607089, 82.80117672382866, 83.03078137332281, 83.18145942455335, 83.42541436464089, 83.51869125349789, 83.59044270646481, 83.64784386883835, 83.73394561239866, 83.76264619358543, 83.84157279184905, 83.89897395422258, 83.98507569778289, 84.0711774413432, 84.08552773193657, 84.1572791849035, 84.23620578316711, 84.32230752672743, 84.38688383439765, 84.48016072325466, 84.58778790270503, 84.6164884838918, 84.65236421037527, 84.73846595393557, 84.85326827868265, 84.91066944105619, 85.05417234699003, 85.11874865466027, 85.1761498170338, 85.24790127000072, 85.31247757767095, 85.39140417593455, 85.43445504771472, 85.4559804836048, 85.53490708186841, 85.6353591160221, 85.71428571428572, 85.87213891081295, 86.02281696204348, 86.09456841501041, 86.20219559446079, 86.2954724833178, 86.41027480806487, 86.48920140632848, 86.5681280045921, 86.64705460285572, 86.71880605582263, 86.8479586711631, 86.99863672239363, 87.14213962832747, 87.3071679701514, 87.4004448590084, 87.43632058549186, 87.58699863672238, 87.62287436320587, 87.79507785032648, 87.89552988448015, 87.96010619215039, 88.09643395278755, 88.24711200401808, 88.39779005524862, 88.59151897825932, 88.72784673889646, 88.84982420894023, 89.01485255076416, 89.158355456698, 89.3520843797087, 89.4238358326756, 89.48123699504916, 89.60321446509292, 89.72519193513668, 89.8184688239937, 89.8902202769606, 90.00502260170768, 90.11264978115807, 90.23462725120184, 90.37813015713569, 90.52880820836621, 90.70101169548683, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 85.02644092781955, "threshold_area100": 84.94127143574659, "threshold_area25": 80.00430508717803}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.9551740884780884, "operator": "ge"}, {"budget": 0.02, "threshold": 1.7678292989730835, "operator": "ge"}, {"budget": 0.03, "threshold": 1.6108415126800537, "operator": "ge"}, {"budget": 0.04, "threshold": 1.4882237911224365, "operator": "ge"}, {"budget": 0.05, "threshold": 1.3846940994262695, "operator": "ge"}, {"budget": 0.06, "threshold": 1.2969385385513306, "operator": "ge"}, {"budget": 0.07, "threshold": 1.213577389717102, "operator": "ge"}, {"budget": 0.08, "threshold": 1.126125454902649, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0642040967941284, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9976176023483276, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9387578964233398, "operator": "ge"}, {"budget": 0.12, "threshold": 0.8820607662200928, "operator": "ge"}, {"budget": 0.13, "threshold": 0.819525957107544, "operator": "ge"}, {"budget": 0.14, "threshold": 0.7585870027542114, "operator": "ge"}, {"budget": 0.15, "threshold": 0.6738402843475342, "operator": "ge"}, {"budget": 0.16, "threshold": 0.6187916994094849, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5571905970573425, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4902919828891754, "operator": "ge"}, {"budget": 0.19, "threshold": 0.4224497377872467, "operator": "ge"}, {"budget": 0.2, "threshold": 0.36229825019836426, "operator": "ge"}, {"budget": 0.21, "threshold": 0.28380298614501953, "operator": "ge"}, {"budget": 0.22, "threshold": 0.19113042950630188, "operator": "ge"}, {"budget": 0.23, "threshold": 0.10104548931121826, "operator": "ge"}, {"budget": 0.24, "threshold": 0.02226387709379196, "operator": "ge"}, {"budget": 0.25, "threshold": -0.06074419617652893, "operator": "ge"}, {"budget": 0.26, "threshold": -0.14921638369560242, "operator": "ge"}, {"budget": 0.27, "threshold": -0.2433508038520813, "operator": "ge"}, {"budget": 0.28, "threshold": -0.3243705928325653, "operator": "ge"}, {"budget": 0.29, "threshold": -0.4323245882987976, "operator": "ge"}, {"budget": 0.3, "threshold": -0.5161692500114441, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6081064343452454, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6869038343429565, "operator": "ge"}, {"budget": 0.33, "threshold": -0.7818587422370911, "operator": "ge"}, {"budget": 0.34, "threshold": -0.8773102760314941, "operator": "ge"}, {"budget": 0.35, "threshold": -0.978078305721283, "operator": "ge"}, {"budget": 0.36, "threshold": -1.0691471099853516, "operator": "ge"}, {"budget": 0.37, "threshold": -1.1751402616500854, "operator": "ge"}, {"budget": 0.38, "threshold": -1.3258165121078491, "operator": "ge"}, {"budget": 0.39, "threshold": -1.453472375869751, "operator": "ge"}, {"budget": 0.4, "threshold": -1.587626338005066, "operator": "ge"}, {"budget": 0.41, "threshold": -1.7628635168075562, "operator": "ge"}, {"budget": 0.42, "threshold": -1.9936904907226562, "operator": "ge"}, {"budget": 0.43, "threshold": -2.210188627243042, "operator": "ge"}, {"budget": 0.44, "threshold": -2.3956055641174316, "operator": "ge"}, {"budget": 0.45, "threshold": -2.6187615394592285, "operator": "ge"}, {"budget": 0.46, "threshold": -2.812175750732422, "operator": "ge"}, {"budget": 0.47, "threshold": -2.9766459465026855, "operator": "ge"}, {"budget": 0.48, "threshold": -3.145658016204834, "operator": "ge"}, {"budget": 0.49, "threshold": -3.297700881958008, "operator": "ge"}, {"budget": 0.5, "threshold": -3.4285731315612793, "operator": "ge"}, {"budget": 0.51, "threshold": -3.5391829013824463, "operator": "ge"}, {"budget": 0.52, "threshold": -3.6553707122802734, "operator": "ge"}, {"budget": 0.53, "threshold": -3.7484617233276367, "operator": "ge"}, {"budget": 0.54, "threshold": -3.8393449783325195, "operator": "ge"}, {"budget": 0.55, "threshold": -3.9139585494995117, "operator": "ge"}, {"budget": 0.56, "threshold": -4.0066728591918945, "operator": "ge"}, {"budget": 0.57, "threshold": -4.073347091674805, "operator": "ge"}, {"budget": 0.58, "threshold": -4.1487274169921875, "operator": "ge"}, {"budget": 0.59, "threshold": -4.209892272949219, "operator": "ge"}, {"budget": 0.6, "threshold": -4.260292053222656, "operator": "ge"}, {"budget": 0.61, "threshold": -4.317992210388184, "operator": "ge"}, {"budget": 0.62, "threshold": -4.372204780578613, "operator": "ge"}, {"budget": 0.63, "threshold": -4.4196977615356445, "operator": "ge"}, {"budget": 0.64, "threshold": -4.472421646118164, "operator": "ge"}, {"budget": 0.65, "threshold": -4.5265984535217285, "operator": "ge"}, {"budget": 0.66, "threshold": -4.5726542472839355, "operator": "ge"}, {"budget": 0.67, "threshold": -4.6155686378479, "operator": "ge"}, {"budget": 0.68, "threshold": -4.657099723815918, "operator": "ge"}, {"budget": 0.69, "threshold": -4.704586982727051, "operator": "ge"}, {"budget": 0.7, "threshold": -4.737147331237793, "operator": "ge"}, {"budget": 0.71, "threshold": -4.77189826965332, "operator": "ge"}, {"budget": 0.72, "threshold": -4.810626029968262, "operator": "ge"}, {"budget": 0.73, "threshold": -4.845527648925781, "operator": "ge"}, {"budget": 0.74, "threshold": -4.886590957641602, "operator": "ge"}, {"budget": 0.75, "threshold": -4.920016288757324, "operator": "ge"}, {"budget": 0.76, "threshold": -4.96240234375, "operator": "ge"}, {"budget": 0.77, "threshold": -5.005867004394531, "operator": "ge"}, {"budget": 0.78, "threshold": -5.038171768188477, "operator": "ge"}, {"budget": 0.79, "threshold": -5.074487686157227, "operator": "ge"}, {"budget": 0.8, "threshold": -5.113455772399902, "operator": "ge"}, {"budget": 0.81, "threshold": -5.154217720031738, "operator": "ge"}, {"budget": 0.82, "threshold": -5.19495964050293, "operator": "ge"}, {"budget": 0.83, "threshold": -5.228305816650391, "operator": "ge"}, {"budget": 0.84, "threshold": -5.263183116912842, "operator": "ge"}, {"budget": 0.85, "threshold": -5.301562309265137, "operator": "ge"}, {"budget": 0.86, "threshold": -5.337179183959961, "operator": "ge"}, {"budget": 0.87, "threshold": -5.375956058502197, "operator": "ge"}, {"budget": 0.88, "threshold": -5.420205116271973, "operator": "ge"}, {"budget": 0.89, "threshold": -5.457891464233398, "operator": "ge"}, {"budget": 0.9, "threshold": -5.510141372680664, "operator": "ge"}, {"budget": 0.91, "threshold": -5.551543235778809, "operator": "ge"}, {"budget": 0.92, "threshold": -5.60804557800293, "operator": "ge"}, {"budget": 0.93, "threshold": -5.650325775146484, "operator": "ge"}, {"budget": 0.94, "threshold": -5.706973075866699, "operator": "ge"}, {"budget": 0.95, "threshold": -5.778480529785156, "operator": "ge"}, {"budget": 0.96, "threshold": -5.8519768714904785, "operator": "ge"}, {"budget": 0.97, "threshold": -5.944544792175293, "operator": "ge"}, {"budget": 0.98, "threshold": -6.0438971519470215, "operator": "ge"}, {"budget": 0.99, "threshold": -6.187008857727051, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.9954167860212, 76.29619020338012, 76.69722142652535, 76.96934975651676, 77.2271555428244, 77.54225150386709, 77.95760527069608, 78.258378688055, 78.64508736751647, 78.9745058722429, 79.17502148381553, 79.43282727012317, 79.74792323116586, 80.07734173589229, 80.2492122600974, 80.53566313377254, 80.73617874534517, 80.93669435691778, 81.25179031796047, 81.42366084216557, 81.6814666284732, 81.92494987109711, 82.08249785161844, 82.3403036379261, 82.51217416213119, 82.79862503580635, 82.97049556001146, 83.14236608421656, 83.34288169578917, 83.5004296763105, 83.72959037525064, 83.88713835577198, 84.27384703523344, 84.54597536522486, 84.71784588942997, 84.91836150100256, 85.1475221999427, 85.43397307361788, 85.69177885992552, 85.86364938413061, 86.06416499570322, 86.26468060727585, 86.42222858779719, 86.6943569177886, 86.86622744199369, 86.99513033514752, 87.02377542251504, 87.10971068461758, 87.19564594672013, 87.22429103408766, 87.28158120882267, 87.35319392724148, 87.38183901460899, 87.4820968203953, 87.52506445144658, 87.62532225723288, 87.76854769407046, 87.78287023775422, 87.84016041248925, 87.95474076195933, 88.04067602406187, 88.11228874248066, 88.18390146089945, 88.21254654826697, 88.21254654826697, 88.28415926668576, 88.35577198510455, 88.5133199656259, 88.64222285877972, 88.69951303351475, 88.77112575193354, 88.87138355771985, 88.90002864508737, 88.94299627613864, 89.07189916929245, 89.18647951876252, 89.22944714981381, 89.27241478086508, 89.3010598682326, 89.37267258665138, 89.3297049556001, 89.34402749928387, 89.34402749928387, 89.37267258665138, 89.41564021770266, 89.47293039243769, 89.50157547980521, 89.48725293612145, 89.50157547980521, 89.53022056717272, 89.63047837295903, 89.58751074190776, 89.6448009166428, 89.6734460040103, 89.7450587224291, 89.75938126611285, 89.80234889716414, 89.8166714408479, 89.87396161558291, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 59, 89, 114, 136, 160, 193, 220, 249, 280, 299, 322, 347, 376, 401, 427, 445, 468, 495, 518, 541, 564, 585, 608, 624, 648, 668, 682, 703, 718, 742, 761, 790, 813, 828, 844, 865, 888, 908, 924, 940, 957, 974, 997, 1012, 1025, 1033, 1043, 1055, 1065, 1074, 1085, 1093, 1105, 1112, 1124, 1135, 1143, 1151, 1159, 1168, 1177, 1184, 1191, 1195, 1201, 1211, 1223, 1236, 1244, 1250, 1257, 1262, 1267, 1277, 1287, 1293, 1298, 1301, 1307, 1309, 1313, 1314, 1321, 1325, 1330, 1334, 1337, 1341, 1347, 1355, 1356, 1363, 1367, 1372, 1374, 1377, 1381, 1385, 1389], "harmed": [0, 2, 8, 10, 16, 20, 22, 26, 32, 34, 42, 47, 52, 55, 61, 74, 80, 84, 93, 98, 109, 114, 120, 130, 135, 139, 143, 151, 153, 160, 164, 172, 180, 182, 186, 189, 191, 196, 199, 201, 205, 207, 210, 216, 220, 223, 227, 233, 237, 243, 251, 256, 262, 268, 273, 277, 282, 283, 290, 294, 294, 297, 301, 303, 308, 312, 313, 318, 319, 323, 327, 328, 328, 331, 333, 334, 336, 339, 341, 342, 343, 348, 351, 352, 357, 358, 359, 361, 365, 368, 372, 373, 377, 380, 382, 382, 383, 383, 386, 386, 388], "exact_score": [75.56574047550845, 75.9954167860212, 76.29619020338012, 76.69722142652535, 76.96934975651676, 77.2271555428244, 77.54225150386709, 77.95760527069608, 78.258378688055, 78.64508736751647, 78.9745058722429, 79.17502148381553, 79.43282727012317, 79.74792323116586, 80.07734173589229, 80.2492122600974, 80.53566313377254, 80.73617874534517, 80.93669435691778, 81.25179031796047, 81.42366084216557, 81.6814666284732, 81.92494987109711, 82.08249785161844, 82.3403036379261, 82.51217416213119, 82.79862503580635, 82.97049556001146, 83.14236608421656, 83.34288169578917, 83.5004296763105, 83.72959037525064, 83.88713835577198, 84.27384703523344, 84.54597536522486, 84.71784588942997, 84.91836150100256, 85.1475221999427, 85.43397307361788, 85.69177885992552, 85.86364938413061, 86.06416499570322, 86.26468060727585, 86.42222858779719, 86.6943569177886, 86.86622744199369, 86.99513033514752, 87.02377542251504, 87.10971068461758, 87.19564594672013, 87.22429103408766, 87.28158120882267, 87.35319392724148, 87.38183901460899, 87.4820968203953, 87.52506445144658, 87.62532225723288, 87.76854769407046, 87.78287023775422, 87.84016041248925, 87.95474076195933, 88.04067602406187, 88.11228874248066, 88.18390146089945, 88.21254654826697, 88.21254654826697, 88.28415926668576, 88.35577198510455, 88.5133199656259, 88.64222285877972, 88.69951303351475, 88.77112575193354, 88.87138355771985, 88.90002864508737, 88.94299627613864, 89.07189916929245, 89.18647951876252, 89.22944714981381, 89.27241478086508, 89.3010598682326, 89.37267258665138, 89.3297049556001, 89.34402749928387, 89.34402749928387, 89.37267258665138, 89.41564021770266, 89.47293039243769, 89.50157547980521, 89.48725293612145, 89.50157547980521, 89.53022056717272, 89.63047837295903, 89.58751074190776, 89.6448009166428, 89.6734460040103, 89.7450587224291, 89.75938126611285, 89.80234889716414, 89.8166714408479, 89.87396161558291, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.55336589623062, "threshold_area100": 85.55234889716414, "threshold_area25": 79.38269836723002}, "test": {"score": [76.42247255506925, 76.75970438401379, 77.03235990528809, 77.36959173423263, 77.72117385377054, 78.00817966563824, 78.23778431513239, 78.57501614407693, 78.94812369950492, 79.16337805840568, 79.38580756260315, 79.74456482743776, 80.05309607519553, 80.34010188706321, 80.6342828442276, 80.93563894668867, 81.15089330558945, 81.33744708330343, 81.60292745928105, 81.9186338523355, 82.09083733945613, 82.24869053598336, 82.52134605725766, 82.89445361268565, 83.07383224510296, 83.2245102963335, 83.45411494582765, 83.77699648417881, 84.0568271507498, 84.37970868910097, 84.66671450096864, 84.84609313338596, 84.99677118461649, 85.20485039822057, 85.51338164597834, 85.83626318432948, 85.9151897825931, 86.15914472268064, 86.40309966276818, 86.52507713281194, 86.64705460285572, 86.82643323527302, 87.04886273947048, 87.2138910812944, 87.42914544019517, 87.68745067087609, 87.80942814091985, 87.99598191863386, 88.21123627753462, 88.34038889287508, 88.39779005524862, 88.42649063643539, 88.54846810647916, 88.66327043122624, 88.71349644830309, 88.82829877305016, 88.91440051661046, 89.01485255076416, 89.13683002080792, 89.20140632847816, 89.33055894381862, 89.35925952500538, 89.36643467030207, 89.38078496089545, 89.46688670445577, 89.531463012126, 89.58168902920285, 89.67496591805984, 89.73236708043338, 89.76824280691685, 89.78259309751022, 89.811293678697, 89.85434455047715, 89.8902202769606, 89.94044629403746, 89.94762143933414, 89.99067231111431, 90.04807347348785, 90.0982994905647, 90.15570065293824, 90.19875152471838, 90.24897754179523, 90.24897754179523, 90.28485326827868, 90.277678122982, 90.32790414005885, 90.38530530243239, 90.44270646480592, 90.50728277247615, 90.57185908014638, 90.60055966133315, 90.58620937073975, 90.61490995192652, 90.65796082370669, 90.69383655019014, 90.73688742197031, 90.72253713137691, 90.70818684078353, 90.7297122766736, 90.72253713137691, 90.73688742197031], "calls": [0.0, 1.0045203415369162, 1.8224869053598336, 2.8485326827868267, 3.9032790414005882, 4.89344909234412, 5.6540144937934995, 6.6370093994403385, 7.777857501614408, 8.502547176580325, 9.370739757480088, 10.382435244313697, 11.243452679916768, 12.061419243739687, 13.09464016646337, 14.422042046351438, 15.247183755471047, 16.244528951711274, 17.31362560091842, 18.38989739542226, 19.107411925091483, 20.14063284781517, 21.25278036880247, 22.314701872712924, 23.347922795436606, 24.1874147951496, 25.07713281193944, 26.225156059410203, 27.337303580397503, 28.64318002439549, 29.6548755112291, 30.78854846810648, 31.757193083159933, 32.905216330630694, 33.96713783454115, 35.15821195379206, 36.02640453469183, 37.059625457415514, 38.26504986725981, 39.03996555930257, 39.82205639664203, 40.668723541651715, 41.83827222501256, 42.87866829303294, 43.94776494224008, 45.160364497381075, 46.236636291884906, 47.183755471048286, 48.3748295902992, 49.34347420535266, 50.254717658032575, 51.09420965774557, 52.12025543517256, 53.182176939083014, 54.064719810576165, 55.004663844442845, 56.11681136543015, 57.0854559804836, 58.15455262969075, 59.02274521059051, 59.79048575733658, 60.85958240654373, 61.71342469685012, 62.6533687307168, 63.77269139700079, 64.84178804620794, 65.94676042189855, 66.90105474635861, 67.89839994259884, 69.01772260888282, 69.85003946329913, 70.85455980483604, 71.98105761641673, 72.86360048790989, 73.8752959747435, 74.83676544450026, 76.05654014493794, 77.37676687952931, 78.30236062280261, 79.21360407548252, 80.20377412642607, 81.33027193800675, 82.43524431369735, 83.5976178517615, 84.55191217622156, 85.62100882542872, 86.53942742340533, 87.68745067087609, 88.83547391834684, 89.70366649924661, 90.9090909090909, 91.64095572935352, 92.81050441271435, 93.73609815598766, 94.73344335222788, 95.78101456554495, 96.7568343258951, 97.73265408624525, 98.36406687235416, 99.13180741910024, 100.0], "rescued": [0, 50, 92, 146, 200, 251, 287, 341, 406, 444, 484, 544, 592, 638, 689, 748, 786, 828, 875, 932, 965, 1004, 1054, 1116, 1154, 1183, 1223, 1281, 1330, 1385, 1433, 1469, 1502, 1548, 1596, 1649, 1675, 1716, 1760, 1784, 1809, 1839, 1878, 1909, 1946, 1990, 2013, 2045, 2081, 2107, 2128, 2145, 2166, 2185, 2203, 2225, 2247, 2271, 2291, 2308, 2329, 2341, 2350, 2361, 2375, 2389, 2399, 2415, 2428, 2442, 2450, 2461, 2473, 2482, 2494, 2504, 2514, 2527, 2538, 2549, 2558, 2567, 2572, 2581, 2586, 2597, 2607, 2618, 2631, 2641, 2648, 2652, 2660, 2673, 2683, 2692, 2695, 2697, 2700, 2702, 2706], "harmed": [0, 3, 7, 14, 19, 30, 34, 41, 54, 62, 71, 81, 86, 92, 102, 119, 127, 143, 153, 166, 175, 192, 204, 214, 227, 235, 243, 256, 266, 276, 284, 295, 307, 324, 329, 337, 352, 359, 369, 376, 384, 389, 397, 405, 412, 420, 426, 432, 438, 446, 459, 472, 476, 479, 490, 496, 506, 516, 519, 527, 530, 538, 546, 555, 557, 562, 565, 568, 573, 582, 588, 595, 601, 605, 610, 619, 623, 628, 632, 635, 638, 640, 645, 649, 655, 659, 661, 664, 668, 669, 672, 678, 682, 689, 694, 697, 702, 706, 706, 709, 711], "exact_score": [76.42247255506925, 76.75252923871709, 77.08976106766163, 77.41264260601277, 77.74269928966062, 78.036880246825, 78.37411207576955, 78.64676759704385, 79.00552486187846, 79.29253067374614, 79.57953648561383, 79.95981918633852, 80.30422616057974, 80.58405682715075, 80.82801176723828, 81.07914185262251, 81.28722106622659, 81.55270144220421, 81.8397072540719, 82.04778646767598, 82.21998995479659, 82.4854703307742, 82.77247614264189, 83.01643108272943, 83.18145942455335, 83.41823921934419, 83.6837195953218, 83.95637511659612, 84.18597976609027, 84.50168615914473, 84.72411566334218, 84.86761856927603, 84.99677118461649, 85.24072612470403, 85.51338164597834, 85.77886202195594, 85.9080146372964, 86.13761928679055, 86.3815742268781, 86.51072684221856, 86.68293032933917, 86.89100954294324, 87.07038817536056, 87.24976680777786, 87.42914544019517, 87.665925234986, 87.78790270502978, 87.93858075626032, 88.132309679271, 88.30451316639162, 88.37626461935855, 88.42649063643539, 88.52694267058908, 88.64174499533615, 88.69914615770969, 88.82829877305016, 88.91440051661046, 89.02920284135753, 89.10812943962115, 89.20858147377486, 89.3377340891153, 89.38078496089545, 89.37360981559877, 89.40948554208224, 89.48123699504916, 89.531463012126, 89.58886417449953, 89.67496591805984, 89.73236708043338, 89.76106766162016, 89.78976824280693, 89.82564396929038, 89.85434455047715, 89.904570567554, 89.92609600344407, 89.96197172992754, 89.99067231111431, 90.03372318289445, 90.07677405467462, 90.13417521704815, 90.20592667001507, 90.23462725120184, 90.25615268709191, 90.28485326827868, 90.277678122982, 90.30637870416876, 90.33507928535553, 90.42118102891584, 90.44988161010261, 90.51445791777284, 90.58620937073975, 90.59338451603645, 90.60773480662984, 90.62208509722322, 90.66513596900337, 90.71536198608023, 90.7297122766736, 90.71536198608023, 90.71536198608023, 90.7297122766736, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.52174903302212, "threshold_area100": 86.48321016000575, "threshold_area25": 80.04879098801752}, "selected_budget": 100, "selected": {"val": {"score": 89.90260670295044, "calls": 100.0, "rescued": 1389, "harmed": 388}, "test": {"score": 90.73688742197031, "calls": 100.0, "rescued": 2706, "harmed": 711}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.3377043008804321, "operator": "ge"}, {"budget": 0.02, "threshold": 1.1971527338027954, "operator": "ge"}, {"budget": 0.03, "threshold": 1.0944490432739258, "operator": "ge"}, {"budget": 0.04, "threshold": 1.0076814889907837, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9211853742599487, "operator": "ge"}, {"budget": 0.06, "threshold": 0.8546310663223267, "operator": "ge"}, {"budget": 0.07, "threshold": 0.8036100268363953, "operator": "ge"}, {"budget": 0.08, "threshold": 0.7513654828071594, "operator": "ge"}, {"budget": 0.09, "threshold": 0.7045139074325562, "operator": "ge"}, {"budget": 0.1, "threshold": 0.6502526998519897, "operator": "ge"}, {"budget": 0.11, "threshold": 0.604888916015625, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5541443824768066, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5109854936599731, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4673161506652832, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4174891710281372, "operator": "ge"}, {"budget": 0.16, "threshold": 0.38517892360687256, "operator": "ge"}, {"budget": 0.17, "threshold": 0.35297536849975586, "operator": "ge"}, {"budget": 0.18, "threshold": 0.32267552614212036, "operator": "ge"}, {"budget": 0.19, "threshold": 0.28220701217651367, "operator": "ge"}, {"budget": 0.2, "threshold": 0.23655080795288086, "operator": "ge"}, {"budget": 0.21, "threshold": 0.18424275517463684, "operator": "ge"}, {"budget": 0.22, "threshold": 0.13273990154266357, "operator": "ge"}, {"budget": 0.23, "threshold": 0.06930848956108093, "operator": "ge"}, {"budget": 0.24, "threshold": 0.015793412923812866, "operator": "ge"}, {"budget": 0.25, "threshold": -0.04515501856803894, "operator": "ge"}, {"budget": 0.26, "threshold": -0.10765272378921509, "operator": "ge"}, {"budget": 0.27, "threshold": -0.17851513624191284, "operator": "ge"}, {"budget": 0.28, "threshold": -0.24943062663078308, "operator": "ge"}, {"budget": 0.29, "threshold": -0.3257291913032532, "operator": "ge"}, {"budget": 0.3, "threshold": -0.3955898582935333, "operator": "ge"}, {"budget": 0.31, "threshold": -0.4726012945175171, "operator": "ge"}, {"budget": 0.32, "threshold": -0.5367488265037537, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6242200136184692, "operator": "ge"}, {"budget": 0.34, "threshold": -0.6889832019805908, "operator": "ge"}, {"budget": 0.35, "threshold": -0.7749279737472534, "operator": "ge"}, {"budget": 0.36, "threshold": -0.858656644821167, "operator": "ge"}, {"budget": 0.37, "threshold": -0.9588406085968018, "operator": "ge"}, {"budget": 0.38, "threshold": -1.072081446647644, "operator": "ge"}, {"budget": 0.39, "threshold": -1.1909258365631104, "operator": "ge"}, {"budget": 0.4, "threshold": -1.3155112266540527, "operator": "ge"}, {"budget": 0.41, "threshold": -1.4667894840240479, "operator": "ge"}, {"budget": 0.42, "threshold": -1.624963641166687, "operator": "ge"}, {"budget": 0.43, "threshold": -1.8438830375671387, "operator": "ge"}, {"budget": 0.44, "threshold": -1.9993550777435303, "operator": "ge"}, {"budget": 0.45, "threshold": -2.196073532104492, "operator": "ge"}, {"budget": 0.46, "threshold": -2.3609459400177, "operator": "ge"}, {"budget": 0.47, "threshold": -2.4964916706085205, "operator": "ge"}, {"budget": 0.48, "threshold": -2.6597819328308105, "operator": "ge"}, {"budget": 0.49, "threshold": -2.788909912109375, "operator": "ge"}, {"budget": 0.5, "threshold": -2.917396306991577, "operator": "ge"}, {"budget": 0.51, "threshold": -3.013762950897217, "operator": "ge"}, {"budget": 0.52, "threshold": -3.0996298789978027, "operator": "ge"}, {"budget": 0.53, "threshold": -3.183582305908203, "operator": "ge"}, {"budget": 0.54, "threshold": -3.27604603767395, "operator": "ge"}, {"budget": 0.55, "threshold": -3.3548595905303955, "operator": "ge"}, {"budget": 0.56, "threshold": -3.422290325164795, "operator": "ge"}, {"budget": 0.57, "threshold": -3.486745834350586, "operator": "ge"}, {"budget": 0.58, "threshold": -3.5482230186462402, "operator": "ge"}, {"budget": 0.59, "threshold": -3.597224235534668, "operator": "ge"}, {"budget": 0.6, "threshold": -3.6533854007720947, "operator": "ge"}, {"budget": 0.61, "threshold": -3.7075018882751465, "operator": "ge"}, {"budget": 0.62, "threshold": -3.7634835243225098, "operator": "ge"}, {"budget": 0.63, "threshold": -3.808448076248169, "operator": "ge"}, {"budget": 0.64, "threshold": -3.8522090911865234, "operator": "ge"}, {"budget": 0.65, "threshold": -3.9004857540130615, "operator": "ge"}, {"budget": 0.66, "threshold": -3.94866681098938, "operator": "ge"}, {"budget": 0.67, "threshold": -3.991576671600342, "operator": "ge"}, {"budget": 0.68, "threshold": -4.034909248352051, "operator": "ge"}, {"budget": 0.69, "threshold": -4.06753396987915, "operator": "ge"}, {"budget": 0.7, "threshold": -4.108736038208008, "operator": "ge"}, {"budget": 0.71, "threshold": -4.147240161895752, "operator": "ge"}, {"budget": 0.72, "threshold": -4.185674667358398, "operator": "ge"}, {"budget": 0.73, "threshold": -4.227846622467041, "operator": "ge"}, {"budget": 0.74, "threshold": -4.263641834259033, "operator": "ge"}, {"budget": 0.75, "threshold": -4.30239725112915, "operator": "ge"}, {"budget": 0.76, "threshold": -4.344640254974365, "operator": "ge"}, {"budget": 0.77, "threshold": -4.382053852081299, "operator": "ge"}, {"budget": 0.78, "threshold": -4.42251443862915, "operator": "ge"}, {"budget": 0.79, "threshold": -4.460787773132324, "operator": "ge"}, {"budget": 0.8, "threshold": -4.493817329406738, "operator": "ge"}, {"budget": 0.81, "threshold": -4.537059783935547, "operator": "ge"}, {"budget": 0.82, "threshold": -4.581508159637451, "operator": "ge"}, {"budget": 0.83, "threshold": -4.610601425170898, "operator": "ge"}, {"budget": 0.84, "threshold": -4.641160011291504, "operator": "ge"}, {"budget": 0.85, "threshold": -4.681087017059326, "operator": "ge"}, {"budget": 0.86, "threshold": -4.722567558288574, "operator": "ge"}, {"budget": 0.87, "threshold": -4.766685485839844, "operator": "ge"}, {"budget": 0.88, "threshold": -4.812972068786621, "operator": "ge"}, {"budget": 0.89, "threshold": -4.86181116104126, "operator": "ge"}, {"budget": 0.9, "threshold": -4.910142421722412, "operator": "ge"}, {"budget": 0.91, "threshold": -4.954220771789551, "operator": "ge"}, {"budget": 0.92, "threshold": -4.995080947875977, "operator": "ge"}, {"budget": 0.93, "threshold": -5.0400495529174805, "operator": "ge"}, {"budget": 0.94, "threshold": -5.1020402908325195, "operator": "ge"}, {"budget": 0.95, "threshold": -5.174513816833496, "operator": "ge"}, {"budget": 0.96, "threshold": -5.2551422119140625, "operator": "ge"}, {"budget": 0.97, "threshold": -5.327973365783691, "operator": "ge"}, {"budget": 0.98, "threshold": -5.449141502380371, "operator": "ge"}, {"budget": 0.99, "threshold": -5.622258186340332, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.7115439702091, 76.99799484388427, 77.35605843597823, 77.54225150386709, 77.85734746490976, 78.21541105700372, 78.6307648238327, 78.93153824119163, 79.2179891148668, 79.50443998854196, 79.74792323116586, 79.94843884273847, 80.20624462904613, 80.57863076482383, 80.80779146376396, 81.10856488112287, 81.29475794901174, 81.49527356058435, 81.6814666284732, 81.92494987109711, 82.0681753079347, 82.32598109424234, 82.51217416213119, 82.78430249212259, 82.94185047264394, 83.21397880263535, 83.32855915210541, 83.51475221999426, 83.72959037525064, 83.88713835577198, 84.20223431681467, 84.48868519048983, 84.73216843311371, 84.98997421942136, 85.17616728731022, 85.40532798625036, 85.72042394729303, 85.82068175307934, 86.078487539387, 86.25035806359209, 86.36493841306215, 86.65138928673733, 86.76596963620739, 86.92351761672873, 86.92351761672873, 87.09538814093382, 87.2386135777714, 87.22429103408766, 87.26725866513893, 87.32454883987396, 87.39616155829275, 87.5107419077628, 87.68261243196791, 87.73990260670294, 87.84016041248925, 87.94041821827557, 87.95474076195933, 88.02635348037812, 88.06932111142939, 88.14093382984818, 88.12661128616442, 88.19822400458321, 88.31280435405327, 88.35577198510455, 88.42738470352333, 88.52764250930964, 88.59925522772845, 88.62790031509596, 88.75680320824978, 88.84273847035233, 88.92867373245488, 88.95731881982239, 88.92867373245488, 88.95731881982239, 88.97164136350615, 89.10054425665999, 89.1291893440275, 89.18647951876252, 89.27241478086508, 89.31538241191636, 89.35835004296763, 89.45860784875393, 89.47293039243769, 89.47293039243769, 89.48725293612145, 89.5445431108565, 89.55886565454024, 89.573188198224, 89.61615582927529, 89.65912346032655, 89.7450587224291, 89.7450587224291, 89.84531652821542, 89.83099398453164, 89.84531652821542, 89.87396161558291, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 60, 89, 113, 141, 162, 187, 217, 249, 274, 299, 321, 346, 371, 395, 426, 450, 474, 495, 518, 541, 564, 584, 607, 624, 647, 667, 688, 703, 719, 743, 760, 788, 809, 828, 849, 867, 887, 910, 923, 941, 956, 972, 996, 1008, 1021, 1029, 1047, 1059, 1064, 1075, 1085, 1097, 1111, 1124, 1132, 1143, 1151, 1157, 1168, 1173, 1180, 1186, 1192, 1200, 1204, 1213, 1222, 1230, 1235, 1247, 1257, 1266, 1274, 1276, 1281, 1287, 1297, 1302, 1307, 1315, 1319, 1324, 1331, 1335, 1338, 1342, 1346, 1348, 1352, 1356, 1360, 1366, 1369, 1378, 1380, 1382, 1385, 1387, 1389], "harmed": [0, 2, 7, 9, 13, 16, 24, 27, 32, 35, 39, 44, 46, 54, 65, 71, 76, 84, 87, 95, 104, 114, 120, 130, 135, 139, 143, 152, 154, 161, 164, 173, 179, 185, 186, 188, 191, 196, 200, 201, 207, 207, 210, 218, 222, 226, 228, 236, 242, 244, 250, 258, 264, 271, 277, 278, 282, 286, 287, 292, 298, 300, 302, 309, 310, 310, 311, 315, 317, 320, 323, 326, 330, 333, 339, 343, 346, 351, 352, 355, 356, 358, 359, 361, 361, 364, 367, 370, 370, 371, 374, 375, 376, 376, 379, 381, 384, 385, 386, 386, 388], "exact_score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.7115439702091, 76.99799484388427, 77.35605843597823, 77.54225150386709, 77.85734746490976, 78.21541105700372, 78.6307648238327, 78.93153824119163, 79.2179891148668, 79.50443998854196, 79.74792323116586, 79.94843884273847, 80.20624462904613, 80.57863076482383, 80.80779146376396, 81.10856488112287, 81.29475794901174, 81.49527356058435, 81.6814666284732, 81.92494987109711, 82.0681753079347, 82.32598109424234, 82.51217416213119, 82.78430249212259, 82.94185047264394, 83.21397880263535, 83.32855915210541, 83.51475221999426, 83.72959037525064, 83.88713835577198, 84.20223431681467, 84.48868519048983, 84.73216843311371, 84.98997421942136, 85.17616728731022, 85.40532798625036, 85.72042394729303, 85.82068175307934, 86.078487539387, 86.25035806359209, 86.36493841306215, 86.65138928673733, 86.76596963620739, 86.92351761672873, 86.92351761672873, 87.09538814093382, 87.2386135777714, 87.22429103408766, 87.26725866513893, 87.32454883987396, 87.39616155829275, 87.5107419077628, 87.68261243196791, 87.73990260670294, 87.84016041248925, 87.94041821827557, 87.95474076195933, 88.02635348037812, 88.06932111142939, 88.14093382984818, 88.12661128616442, 88.19822400458321, 88.31280435405327, 88.35577198510455, 88.42738470352333, 88.52764250930964, 88.59925522772845, 88.62790031509596, 88.75680320824978, 88.84273847035233, 88.92867373245488, 88.95731881982239, 88.92867373245488, 88.95731881982239, 88.97164136350615, 89.10054425665999, 89.1291893440275, 89.18647951876252, 89.27241478086508, 89.31538241191636, 89.35835004296763, 89.45860784875393, 89.47293039243769, 89.47293039243769, 89.48725293612145, 89.5445431108565, 89.55886565454024, 89.573188198224, 89.61615582927529, 89.65912346032655, 89.7450587224291, 89.7450587224291, 89.84531652821542, 89.83099398453164, 89.84531652821542, 89.87396161558291, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.55663164953117, "threshold_area100": 85.55564308221139, "threshold_area25": 79.40561443712404}, "test": {"score": [76.42247255506925, 76.74535409342039, 77.05388534117816, 77.32654086245246, 77.67812298199038, 78.00100452034154, 78.34541149458278, 78.55349070818684, 78.81179593886777, 79.11315204132883, 79.40015785319653, 79.75173997273444, 80.0746215110856, 80.50513022888714, 80.71320944249122, 81.05761641673243, 81.22981990385306, 81.45224940805052, 81.63162804046782, 81.86123268996197, 82.10518763004951, 82.2989165530602, 82.52852120255434, 82.88010332209228, 83.07383224510296, 83.23886058692689, 83.46129009112435, 83.75547104828873, 84.04965200545311, 84.40840927028772, 84.63083877448518, 84.83174284279256, 84.95372031283634, 85.19050010762719, 85.39857932123127, 85.71428571428572, 85.88648920140633, 86.1878453038674, 86.36722393628472, 86.5753031498888, 86.66858003874579, 86.81925808997633, 86.99863672239363, 87.2067159359977, 87.41479514960177, 87.63004950850255, 87.79507785032648, 87.98163162804047, 88.16818540575447, 88.27581258520485, 88.39779005524862, 88.46954150821554, 88.52694267058908, 88.61304441414939, 88.75654732008323, 88.89287508072039, 88.91440051661046, 88.96462653368731, 89.04355313195093, 89.10812943962115, 89.16553060199469, 89.24445720025831, 89.33055894381862, 89.40948554208224, 89.45971155915908, 89.56016359331277, 89.60321446509292, 89.71084164454331, 89.76106766162016, 89.82564396929038, 89.8902202769606, 89.93327114874077, 89.96197172992754, 90.03372318289445, 90.0982994905647, 90.14852550764154, 90.14135036234484, 90.16287579823492, 90.19875152471838, 90.24897754179523, 90.25615268709191, 90.32072899476215, 90.29202841357538, 90.3637798665423, 90.39965559302576, 90.42835617421254, 90.42118102891584, 90.44270646480592, 90.52163306306953, 90.58620937073975, 90.62926024251992, 90.66513596900337, 90.66513596900337, 90.65796082370669, 90.65078567840999, 90.65796082370669, 90.71536198608023, 90.7297122766736, 90.72253713137691, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.054746358613762, 1.9301140848102174, 2.812656956303365, 3.84587787902705, 4.871923656454044, 5.783167109133959, 6.651359690033723, 7.569778288010333, 8.45232115950348, 9.399440338666858, 10.32503408194016, 11.372605295257229, 12.54932912391476, 13.517973738968214, 14.780799311186051, 15.627466456195737, 16.316280404678192, 17.12707182320442, 18.188993327114876, 19.28679055750879, 20.162158283705246, 21.202554351725624, 22.422329052163306, 23.32639735954653, 24.22329052163306, 25.09865824782952, 26.239506350003587, 27.36600416158427, 28.628829733802107, 29.690751237712565, 30.781373322809785, 31.785893664346705, 32.95544234770754, 33.80928463801392, 34.99318361196814, 35.98335366291167, 37.08115089330558, 38.171772978402814, 39.03996555930257, 39.91533328549903, 40.81940159288225, 41.75934562674894, 42.97912032718663, 44.03386668580039, 45.09578818971084, 46.164884838917985, 47.10482887278468, 48.26720241084882, 49.2143215900122, 50.3264691109995, 51.25923799956949, 52.141780871062636, 53.01714859725909, 54.165171844729855, 55.16969218626677, 56.08811078424338, 57.30071033938437, 58.24782951854775, 59.04427064648059, 59.9626892444572, 61.010260457774265, 62.29461146588219, 63.29195666212241, 64.38975389251632, 65.4301499605367, 66.57817320800746, 67.5611681136543, 68.58721389108129, 69.38365501901413, 70.40252565114443, 71.36399512090121, 72.40439118892158, 73.33716007749157, 74.31297983784172, 75.33185046997201, 76.37942168328908, 77.43416804190285, 78.596541579967, 79.55801104972376, 80.49795508359044, 81.71055463873144, 83.02360622802611, 83.7769964841788, 84.66671450096864, 85.69993542369232, 86.71880605582263, 87.80225299562316, 88.95027624309392, 90.09829949056468, 91.13152041328837, 92.10016502834183, 92.84638013919782, 93.75762359187773, 94.72626820693118, 95.87429145440196, 96.69943316352156, 97.31649565903709, 98.27796512879385, 99.06723111143, 100.0], "rescued": [0, 49, 98, 142, 197, 249, 303, 341, 387, 438, 487, 540, 594, 660, 699, 759, 793, 832, 872, 921, 971, 1008, 1054, 1116, 1153, 1186, 1224, 1279, 1330, 1387, 1428, 1469, 1499, 1547, 1581, 1635, 1673, 1720, 1757, 1789, 1814, 1842, 1873, 1911, 1945, 1984, 2012, 2042, 2080, 2104, 2127, 2148, 2163, 2184, 2213, 2238, 2250, 2265, 2282, 2295, 2308, 2329, 2349, 2364, 2379, 2397, 2412, 2432, 2449, 2462, 2477, 2488, 2499, 2510, 2524, 2534, 2540, 2550, 2563, 2572, 2578, 2589, 2594, 2605, 2615, 2621, 2628, 2635, 2646, 2655, 2665, 2673, 2676, 2677, 2683, 2687, 2695, 2698, 2702, 2702, 2706], "harmed": [0, 4, 10, 16, 22, 29, 35, 44, 54, 63, 72, 76, 85, 91, 101, 113, 123, 131, 146, 163, 179, 189, 203, 216, 226, 236, 243, 257, 267, 274, 284, 297, 310, 325, 330, 340, 354, 359, 371, 374, 386, 393, 399, 408, 413, 422, 427, 431, 443, 452, 458, 469, 476, 485, 494, 500, 509, 517, 523, 527, 532, 542, 550, 554, 562, 566, 575, 580, 590, 594, 600, 605, 612, 613, 618, 621, 628, 635, 643, 645, 650, 652, 661, 662, 667, 669, 677, 681, 681, 681, 685, 688, 691, 693, 700, 703, 703, 704, 709, 710, 711], "exact_score": [76.42247255506925, 76.71665351223362, 77.08258592236493, 77.3982923154194, 77.73552414436392, 78.02970510152831, 78.409987802253, 78.67546817823062, 78.99834971658176, 79.29970581904284, 79.65128793858077, 79.9741694769319, 80.26835043409629, 80.61993255363421, 80.87823778431513, 81.13654301499605, 81.33027193800675, 81.61010260457775, 81.81818181818183, 82.06931190356606, 82.27021597187343, 82.49264547607089, 82.74377556145512, 83.02360622802611, 83.18145942455335, 83.41823921934419, 83.69806988591519, 83.97072540718949, 84.17880462079357, 84.50168615914473, 84.68823993685872, 84.8891440051661, 84.99677118461649, 85.19767525292387, 85.44880533830809, 85.71428571428572, 85.90083949199972, 86.16631986797732, 86.33134820980125, 86.5609528592954, 86.66858003874579, 86.89818468823994, 87.0416875941738, 87.2067159359977, 87.40762000430509, 87.5941737820191, 87.75920212384301, 87.945755901557, 88.15383511516109, 88.24711200401808, 88.36908947406185, 88.46954150821554, 88.5125923799957, 88.6058692688527, 88.74937217478654, 88.90005022601709, 88.90722537131377, 88.95745138839062, 89.02202769606085, 89.08660400373107, 89.16553060199469, 89.23728205496163, 89.27315778144508, 89.40231039678554, 89.4310109779723, 89.54581330271938, 89.56016359331277, 89.66779077276314, 89.74671737102676, 89.78976824280693, 89.83999425988375, 89.91174571285069, 89.96197172992754, 90.01937289230108, 90.0839491999713, 90.13417521704815, 90.16287579823492, 90.14135036234484, 90.16287579823492, 90.23462725120184, 90.24180239649853, 90.26332783238861, 90.30637870416876, 90.29202841357538, 90.37095501183899, 90.41400588361914, 90.44270646480592, 90.42835617421254, 90.46423190069599, 90.52163306306953, 90.58620937073975, 90.62926024251992, 90.65078567840999, 90.67231111430006, 90.65796082370669, 90.64361053311329, 90.65796082370669, 90.70818684078353, 90.72253713137691, 90.71536198608023, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.53071938041754, "threshold_area100": 86.49799095931692, "threshold_area25": 80.0789265982636}, "selected_budget": 99, "selected": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692, "rescued": 1387, "harmed": 386}, "test": {"score": 90.71536198608023, "calls": 99.06723111143, "rescued": 2702, "harmed": 710}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 89.71641363506158, "calls": 98.99742194213692}, "test": {"score": 90.70101169548683, "calls": 99.08158140202339}}, "old_normalized_interior": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692}, "test": {"score": 90.71536198608023, "calls": 99.06723111143}}, "old_normalized_endpoints": {"val": {"score": 89.90260670295044, "calls": 98.99742194213692}, "test": {"score": 90.71536198608023, "calls": 99.06723111143}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.7115439702091, 76.99799484388427, 77.35605843597823, 77.54225150386709, 77.85734746490976, 78.21541105700372, 78.6307648238327, 78.93153824119163, 79.2179891148668, 79.50443998854196, 79.74792323116586, 79.94843884273847, 80.20624462904613, 80.57863076482383, 80.80779146376396, 81.10856488112287, 81.29475794901174, 81.49527356058435, 81.6814666284732, 81.92494987109711, 82.0681753079347, 82.32598109424234, 82.51217416213119, 82.78430249212259, 82.94185047264394, 83.21397880263535, 83.32855915210541, 83.51475221999426, 83.72959037525064, 83.88713835577198, 84.20223431681467, 84.48868519048983, 84.73216843311371, 84.98997421942136, 85.17616728731022, 85.40532798625036, 85.72042394729303, 85.82068175307934, 86.078487539387, 86.25035806359209, 86.36493841306215, 86.65138928673733, 86.76596963620739, 86.92351761672873, 86.92351761672873, 87.09538814093382, 87.2386135777714, 87.22429103408766, 87.26725866513893, 87.32454883987396, 87.39616155829275, 87.5107419077628, 87.68261243196791, 87.73990260670294, 87.84016041248925, 87.94041821827557, 87.95474076195933, 88.02635348037812, 88.06932111142939, 88.14093382984818, 88.12661128616442, 88.19822400458321, 88.31280435405327, 88.35577198510455, 88.42738470352333, 88.52764250930964, 88.59925522772845, 88.62790031509596, 88.75680320824978, 88.84273847035233, 88.92867373245488, 88.95731881982239, 88.92867373245488, 88.95731881982239, 88.97164136350615, 89.10054425665999, 89.1291893440275, 89.18647951876252, 89.27241478086508, 89.31538241191636, 89.35835004296763, 89.45860784875393, 89.47293039243769, 89.47293039243769, 89.48725293612145, 89.5445431108565, 89.55886565454024, 89.573188198224, 89.61615582927529, 89.65912346032655, 89.7450587224291, 89.7450587224291, 89.84531652821542, 89.83099398453164, 89.84531652821542, 89.87396161558291, 89.90260670295044, 89.90260670295044], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.7115439702091, 76.99799484388427, 77.35605843597823, 77.54225150386709, 77.85734746490976, 78.21541105700372, 78.6307648238327, 78.93153824119163, 79.2179891148668, 79.50443998854196, 79.74792323116586, 79.94843884273847, 80.20624462904613, 80.57863076482383, 80.80779146376396, 81.10856488112287, 81.29475794901174, 81.49527356058435, 81.6814666284732, 81.92494987109711, 82.0681753079347, 82.32598109424234, 82.51217416213119, 82.78430249212259, 82.94185047264394, 83.21397880263535, 83.32855915210541, 83.51475221999426, 83.72959037525064, 83.88713835577198, 84.20223431681467, 84.48868519048983, 84.73216843311371, 84.98997421942136, 85.17616728731022, 85.40532798625036, 85.72042394729303, 85.82068175307934, 86.078487539387, 86.25035806359209, 86.36493841306215, 86.65138928673733, 86.76596963620739, 86.92351761672873, 86.92351761672873, 87.09538814093382, 87.2386135777714, 87.22429103408766, 87.26725866513893, 87.32454883987396, 87.39616155829275, 87.5107419077628, 87.68261243196791, 87.73990260670294, 87.84016041248925, 87.94041821827557, 87.95474076195933, 88.02635348037812, 88.06932111142939, 88.14093382984818, 88.12661128616442, 88.19822400458321, 88.31280435405327, 88.35577198510455, 88.42738470352333, 88.52764250930964, 88.59925522772845, 88.62790031509596, 88.75680320824978, 88.84273847035233, 88.92867373245488, 88.95731881982239, 88.92867373245488, 88.95731881982239, 88.97164136350615, 89.10054425665999, 89.1291893440275, 89.18647951876252, 89.27241478086508, 89.31538241191636, 89.35835004296763, 89.45860784875393, 89.47293039243769, 89.47293039243769, 89.48725293612145, 89.5445431108565, 89.55886565454024, 89.573188198224, 89.61615582927529, 89.65912346032655, 89.7450587224291, 89.7450587224291, 89.84531652821542, 89.83099398453164, 89.84531652821542, 89.87396161558291, 89.90260670295044, 89.90260670295044], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.40561443712404, "exact_area100": 85.55663164953117}, "test": {"score": [76.42247255506925, 76.74535409342039, 77.05388534117816, 77.32654086245246, 77.67812298199038, 78.00100452034154, 78.34541149458278, 78.55349070818684, 78.81179593886777, 79.11315204132883, 79.40015785319653, 79.75173997273444, 80.0746215110856, 80.50513022888714, 80.71320944249122, 81.05761641673243, 81.22981990385306, 81.45224940805052, 81.63162804046782, 81.86123268996197, 82.10518763004951, 82.2989165530602, 82.52852120255434, 82.88010332209228, 83.07383224510296, 83.23886058692689, 83.46129009112435, 83.75547104828873, 84.04965200545311, 84.40840927028772, 84.63083877448518, 84.83174284279256, 84.95372031283634, 85.19050010762719, 85.39857932123127, 85.71428571428572, 85.88648920140633, 86.1878453038674, 86.36722393628472, 86.5753031498888, 86.66858003874579, 86.81925808997633, 86.99863672239363, 87.2067159359977, 87.41479514960177, 87.63004950850255, 87.79507785032648, 87.98163162804047, 88.16818540575447, 88.27581258520485, 88.39779005524862, 88.46954150821554, 88.52694267058908, 88.61304441414939, 88.75654732008323, 88.89287508072039, 88.91440051661046, 88.96462653368731, 89.04355313195093, 89.10812943962115, 89.16553060199469, 89.24445720025831, 89.33055894381862, 89.40948554208224, 89.45971155915908, 89.56016359331277, 89.60321446509292, 89.71084164454331, 89.76106766162016, 89.82564396929038, 89.8902202769606, 89.93327114874077, 89.96197172992754, 90.03372318289445, 90.0982994905647, 90.14852550764154, 90.14135036234484, 90.16287579823492, 90.19875152471838, 90.24897754179523, 90.25615268709191, 90.32072899476215, 90.29202841357538, 90.3637798665423, 90.39965559302576, 90.42835617421254, 90.42118102891584, 90.44270646480592, 90.52163306306953, 90.58620937073975, 90.62926024251992, 90.66513596900337, 90.66513596900337, 90.65796082370669, 90.65078567840999, 90.65796082370669, 90.71536198608023, 90.7297122766736, 90.72253713137691, 90.71536198608023, 90.73688742197031], "calls": [0.0, 1.054746358613762, 1.9301140848102174, 2.812656956303365, 3.84587787902705, 4.871923656454044, 5.783167109133959, 6.651359690033723, 7.569778288010333, 8.45232115950348, 9.399440338666858, 10.32503408194016, 11.372605295257229, 12.54932912391476, 13.517973738968214, 14.780799311186051, 15.627466456195737, 16.316280404678192, 17.12707182320442, 18.188993327114876, 19.28679055750879, 20.162158283705246, 21.202554351725624, 22.422329052163306, 23.32639735954653, 24.22329052163306, 25.09865824782952, 26.239506350003587, 27.36600416158427, 28.628829733802107, 29.690751237712565, 30.781373322809785, 31.785893664346705, 32.95544234770754, 33.80928463801392, 34.99318361196814, 35.98335366291167, 37.08115089330558, 38.171772978402814, 39.03996555930257, 39.91533328549903, 40.81940159288225, 41.75934562674894, 42.97912032718663, 44.03386668580039, 45.09578818971084, 46.164884838917985, 47.10482887278468, 48.26720241084882, 49.2143215900122, 50.3264691109995, 51.25923799956949, 52.141780871062636, 53.01714859725909, 54.165171844729855, 55.16969218626677, 56.08811078424338, 57.30071033938437, 58.24782951854775, 59.04427064648059, 59.9626892444572, 61.010260457774265, 62.29461146588219, 63.29195666212241, 64.38257874721963, 65.4301499605367, 66.57817320800746, 67.5611681136543, 68.58721389108129, 69.38365501901413, 70.40252565114443, 71.36399512090121, 72.40439118892158, 73.33716007749157, 74.31297983784172, 75.33185046997201, 76.37942168328908, 77.43416804190285, 78.596541579967, 79.55801104972376, 80.49795508359044, 81.71055463873144, 83.02360622802611, 83.7769964841788, 84.66671450096864, 85.69993542369232, 86.71880605582263, 87.80225299562316, 88.95027624309392, 90.09829949056468, 91.13152041328837, 92.10016502834183, 92.84638013919782, 93.75762359187773, 94.72626820693118, 95.87429145440196, 96.69943316352156, 97.31649565903709, 98.27796512879385, 99.06723111143, 100.0], "exact_score": [76.42247255506925, 76.71665351223362, 77.08258592236493, 77.3982923154194, 77.73552414436392, 78.02970510152831, 78.409987802253, 78.67546817823062, 78.99834971658176, 79.29970581904284, 79.65128793858077, 79.9741694769319, 80.26835043409629, 80.61993255363421, 80.87823778431513, 81.13654301499605, 81.33027193800675, 81.61010260457775, 81.81818181818183, 82.06931190356606, 82.27021597187343, 82.49264547607089, 82.74377556145512, 83.02360622802611, 83.18145942455335, 83.41823921934419, 83.69806988591519, 83.97072540718949, 84.17880462079357, 84.50168615914473, 84.68823993685872, 84.8891440051661, 84.99677118461649, 85.19767525292387, 85.44880533830809, 85.71428571428572, 85.90083949199972, 86.16631986797732, 86.33134820980125, 86.5609528592954, 86.66858003874579, 86.89818468823994, 87.0416875941738, 87.2067159359977, 87.40762000430509, 87.5941737820191, 87.75920212384301, 87.945755901557, 88.15383511516109, 88.24711200401808, 88.36908947406185, 88.46954150821554, 88.5125923799957, 88.6058692688527, 88.74937217478654, 88.90005022601709, 88.90722537131377, 88.95745138839062, 89.02202769606085, 89.08660400373107, 89.16553060199469, 89.23728205496163, 89.27315778144508, 89.40231039678554, 89.4310109779723, 89.54581330271938, 89.56016359331277, 89.66779077276314, 89.74671737102676, 89.78976824280693, 89.83999425988375, 89.91174571285069, 89.96197172992754, 90.01937289230108, 90.0839491999713, 90.13417521704815, 90.16287579823492, 90.14135036234484, 90.16287579823492, 90.23462725120184, 90.24180239649853, 90.26332783238861, 90.30637870416876, 90.29202841357538, 90.37095501183899, 90.41400588361914, 90.44270646480592, 90.42835617421254, 90.46423190069599, 90.52163306306953, 90.58620937073975, 90.62926024251992, 90.65078567840999, 90.67231111430006, 90.65796082370669, 90.64361053311329, 90.65796082370669, 90.70818684078353, 90.72253713137691, 90.71536198608023, 90.73688742197031], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 80.0789265982636, "exact_area100": 86.53071938041754}}, "old_normalized_endpoint_budget": 99}]}, {"expert": "vilt", "endpoint": "Qwen3.8", "task": "nlvr2", "small": 76.42247255506925, "large": 90.06242376408122, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 1.0, "validation_area100": 0.8597261825862977, "training_seconds": 40.2262916759355, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7302389727397903, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6610749554774192, "operator": "ge"}, {"budget": 0.03, "threshold": 0.6086360564217613, "operator": "ge"}, {"budget": 0.04, "threshold": 0.566590496886246, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5243278060704287, "operator": "ge"}, {"budget": 0.06, "threshold": 0.49624034862888955, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4626228570096569, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4267541034131966, "operator": "ge"}, {"budget": 0.09, "threshold": 0.39402400490623085, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3618759015111528, "operator": "ge"}, {"budget": 0.11, "threshold": 0.33900174686192625, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3130592117280817, "operator": "ge"}, {"budget": 0.13, "threshold": 0.2895875139340566, "operator": "ge"}, {"budget": 0.14, "threshold": 0.25984247697861884, "operator": "ge"}, {"budget": 0.15, "threshold": 0.23304153408675718, "operator": "ge"}, {"budget": 0.16, "threshold": 0.20641316368008958, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1784434847781496, "operator": "ge"}, {"budget": 0.18, "threshold": 0.15528788789149886, "operator": "ge"}, {"budget": 0.19, "threshold": 0.13136018390088106, "operator": "ge"}, {"budget": 0.2, "threshold": 0.09623225833784337, "operator": "ge"}, {"budget": 0.21, "threshold": 0.06716553582058854, "operator": "ge"}, {"budget": 0.22, "threshold": 0.034623239315138714, "operator": "ge"}, {"budget": 0.23, "threshold": 0.00638062197346528, "operator": "ge"}, {"budget": 0.24, "threshold": -0.027779982563586644, "operator": "ge"}, {"budget": 0.25, "threshold": -0.0659564745238215, "operator": "ge"}, {"budget": 0.26, "threshold": -0.1097407048195411, "operator": "ge"}, {"budget": 0.27, "threshold": -0.15043576588392554, "operator": "ge"}, {"budget": 0.28, "threshold": -0.1920666493478377, "operator": "ge"}, {"budget": 0.29, "threshold": -0.23582462707959678, "operator": "ge"}, {"budget": 0.3, "threshold": -0.27869055463227127, "operator": "ge"}, {"budget": 0.31, "threshold": -0.32508786998462263, "operator": "ge"}, {"budget": 0.32, "threshold": -0.36872760835472235, "operator": "ge"}, {"budget": 0.33, "threshold": -0.41334620112956333, "operator": "ge"}, {"budget": 0.34, "threshold": -0.46015484755290903, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5123534112200507, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5683339339566286, "operator": "ge"}, {"budget": 0.37, "threshold": -0.6114458010674945, "operator": "ge"}, {"budget": 0.38, "threshold": -0.6624890536945166, "operator": "ge"}, {"budget": 0.39, "threshold": -0.7065684887352506, "operator": "ge"}, {"budget": 0.4, "threshold": -0.7374357925036754, "operator": "ge"}, {"budget": 0.41, "threshold": -0.7708649392347112, "operator": "ge"}, {"budget": 0.42, "threshold": -0.804487521779047, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8325756616866009, "operator": "ge"}, {"budget": 0.44, "threshold": -0.857275224128716, "operator": "ge"}, {"budget": 0.45, "threshold": -0.875545956043365, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8939458634763495, "operator": "ge"}, {"budget": 0.47, "threshold": -0.9093846329491432, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9226129267148185, "operator": "ge"}, {"budget": 0.49, "threshold": -0.933483102022109, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9426358931266547, "operator": "ge"}, {"budget": 0.51, "threshold": -0.949868638253088, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9556574011088842, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9615608215016144, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9664600473820728, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9701643390618867, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9727083369840168, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9753037456370495, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9778830604366795, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9801385040209409, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9817836424030181, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9834618548743487, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9853066041100378, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9865672118225435, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9878763877874052, "operator": "ge"}, {"budget": 0.65, "threshold": -0.988973515683954, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9899720266706361, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9907900754088487, "operator": "ge"}, {"budget": 0.68, "threshold": -0.991429212233239, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9924547257147975, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9930212432717419, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9936767110455621, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9941356410674087, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9946380659641965, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9951316746341468, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9954898158376925, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9958375065025488, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9962675282553591, "operator": "ge"}, {"budget": 0.78, "threshold": -0.996585963851069, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9968266703700008, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9970851497906832, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9972854679275678, "operator": "ge"}, {"budget": 0.82, "threshold": -0.997508636839176, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9977680347073143, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9979580965568494, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9981600845359413, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9983272807654392, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9984763060454153, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9986605100522791, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9988188161604481, "operator": "ge"}, {"budget": 0.9, "threshold": -0.998952686047838, "operator": "ge"}, {"budget": 0.91, "threshold": -0.999078980881845, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9991958058261221, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9993122766371466, "operator": "ge"}, {"budget": 0.94, "threshold": -0.999397837372627, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9995035216946815, "operator": "ge"}, {"budget": 0.96, "threshold": -0.999597845521075, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9996964547296433, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9997853246188473, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9998572001355173, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.65425379547408, 77.02663993125178, 77.3990260670295, 77.64250930965339, 77.95760527069608, 78.31566886279003, 78.64508736751647, 78.91721569750787, 79.24663420223432, 79.57605270696075, 79.93411629905471, 80.26353480378114, 80.57863076482383, 80.92237181323402, 81.0655972500716, 81.35204812374678, 81.62417645373819, 81.91062732741334, 82.0681753079347, 82.38327126897737, 82.64107705528502, 82.8845602979089, 83.12804354053279, 83.4001718705242, 83.54339730736179, 83.82984818103695, 84.03036379260956, 84.14494414207962, 84.33113720996849, 84.50300773417358, 84.7035233457462, 84.93268404468634, 85.16184474362647, 85.37668289888283, 85.63448868519049, 85.9066170151819, 86.0068748209682, 86.078487539387, 86.30764823832713, 86.47951876253222, 86.6943569177886, 86.70867946147236, 86.90919507304497, 86.99513033514752, 87.12403322830134, 87.19564594672013, 87.35319392724148, 87.45345173302779, 87.5823546261816, 87.69693497565167, 87.89745058722428, 87.9977083930106, 88.08364365511314, 88.19822400458321, 88.34144944142079, 88.45602979089085, 88.59925522772845, 88.74248066456603, 88.84273847035233, 88.8857061014036, 88.95731881982239, 88.98596390718991, 89.07189916929245, 89.04325408192494, 89.07189916929245, 89.08622171297623, 89.2008020624463, 89.22944714981381, 89.3297049556001, 89.37267258665138, 89.42996276138642, 89.42996276138642, 89.42996276138642, 89.41564021770266, 89.50157547980521, 89.58751074190776, 89.63047837295903, 89.65912346032655, 89.70209109137782, 89.75938126611285, 89.77370380979661, 89.87396161558291, 89.9169292466342, 89.95989687768548, 90.03150959610426, 90.0887997708393, 90.10312231452305, 90.17473503294184, 90.24634775136064, 90.26067029504439, 90.27499283872815, 90.23202520767688, 90.24634775136064, 90.31796046977942, 90.28931538241191, 90.40389573188197, 90.46118590661702, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 59, 84, 112, 140, 163, 189, 218, 244, 265, 293, 321, 352, 380, 407, 436, 453, 475, 500, 526, 548, 572, 597, 622, 642, 669, 684, 709, 729, 743, 760, 779, 800, 821, 844, 867, 890, 911, 926, 940, 958, 977, 995, 1005, 1024, 1038, 1050, 1060, 1073, 1085, 1097, 1108, 1123, 1136, 1146, 1157, 1169, 1180, 1190, 1201, 1212, 1221, 1231, 1238, 1246, 1254, 1260, 1263, 1273, 1280, 1290, 1295, 1300, 1307, 1311, 1315, 1325, 1333, 1338, 1345, 1350, 1356, 1361, 1370, 1376, 1381, 1388, 1394, 1395, 1401, 1409, 1412, 1417, 1419, 1423, 1428, 1430, 1440, 1446, 1449], "harmed": [0, 3, 6, 8, 10, 12, 18, 22, 26, 29, 31, 36, 41, 47, 52, 57, 62, 69, 71, 77, 83, 94, 96, 103, 111, 114, 122, 127, 132, 138, 144, 148, 155, 162, 167, 174, 182, 187, 189, 197, 206, 208, 215, 218, 227, 232, 240, 243, 248, 250, 255, 258, 261, 262, 268, 272, 275, 277, 280, 280, 281, 285, 291, 296, 301, 303, 313, 317, 319, 321, 326, 329, 331, 332, 339, 343, 348, 352, 354, 356, 361, 363, 365, 369, 371, 374, 376, 378, 380, 380, 381, 384, 386, 390, 395, 398, 398, 402, 404, 406, 406], "exact_score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.65425379547408, 77.02663993125178, 77.3990260670295, 77.64250930965339, 77.95760527069608, 78.31566886279003, 78.64508736751647, 78.91721569750787, 79.24663420223432, 79.57605270696075, 79.93411629905471, 80.26353480378114, 80.57863076482383, 80.92237181323402, 81.0655972500716, 81.35204812374678, 81.62417645373819, 81.91062732741334, 82.0681753079347, 82.38327126897737, 82.64107705528502, 82.8845602979089, 83.12804354053279, 83.4001718705242, 83.54339730736179, 83.82984818103695, 84.03036379260956, 84.14494414207962, 84.33113720996849, 84.50300773417358, 84.7035233457462, 84.93268404468634, 85.16184474362647, 85.37668289888283, 85.63448868519049, 85.9066170151819, 86.0068748209682, 86.078487539387, 86.30764823832713, 86.47951876253222, 86.6943569177886, 86.70867946147236, 86.90919507304497, 86.99513033514752, 87.12403322830134, 87.19564594672013, 87.35319392724148, 87.45345173302779, 87.5823546261816, 87.69693497565167, 87.89745058722428, 87.9977083930106, 88.08364365511314, 88.19822400458321, 88.34144944142079, 88.45602979089085, 88.59925522772845, 88.74248066456603, 88.84273847035233, 88.8857061014036, 88.95731881982239, 88.98596390718991, 89.07189916929245, 89.04325408192494, 89.07189916929245, 89.08622171297623, 89.2008020624463, 89.22944714981381, 89.3297049556001, 89.37267258665138, 89.42996276138642, 89.42996276138642, 89.42996276138642, 89.41564021770266, 89.50157547980521, 89.58751074190776, 89.63047837295903, 89.65912346032655, 89.70209109137782, 89.75938126611285, 89.77370380979661, 89.87396161558291, 89.9169292466342, 89.95989687768548, 90.03150959610426, 90.0887997708393, 90.10312231452305, 90.17473503294184, 90.24634775136064, 90.26067029504439, 90.27499283872815, 90.23202520767688, 90.24634775136064, 90.31796046977942, 90.28931538241191, 90.40389573188197, 90.46118590661702, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.97261825862978, "threshold_area100": 85.97156975078774, "threshold_area25": 79.63047837295905}, "test": {"score": [76.42247255506925, 76.89603214465093, 77.34806629834254, 77.72117385377054, 78.02970510152831, 78.36693693047285, 78.58219128937361, 78.79027050297769, 79.07727631484538, 79.32840640022961, 79.64411279328407, 79.84501686159145, 80.13202267345913, 80.36880246825, 80.7275597330846, 81.04326612613906, 81.30157135681998, 81.5598765875009, 81.8397072540719, 82.09083733945613, 82.4782951854775, 82.70072468967497, 82.93750448446582, 83.1312334074765, 83.38236349286073, 83.61914328765158, 83.82722250125565, 84.01377627896966, 84.21468034727704, 84.3366578173208, 84.5232115950348, 84.73129080863887, 84.86761856927603, 84.96807060342972, 85.14744923584703, 85.34117815885772, 85.57795795364856, 85.85061347492287, 86.16631986797732, 86.39592451747149, 86.4676759704384, 86.66858003874579, 86.8336083805697, 86.94841070531677, 87.17801535481094, 87.32151826074478, 87.52242232905216, 87.70180096146947, 87.80942814091985, 87.88117959388678, 87.97445648274378, 88.0533830810074, 88.1394848245677, 88.29016287579825, 88.40496520054532, 88.48389179880894, 88.54129296118248, 88.58434383296262, 88.66327043122624, 88.71349644830309, 88.75654732008323, 88.77807275597331, 88.878524790127, 88.92157566190716, 88.97897682428069, 89.01485255076416, 89.00767740546746, 89.09377914902777, 89.13683002080792, 89.20140632847816, 89.33055894381862, 89.40231039678554, 89.43818612326899, 89.49558728564253, 89.57451388390615, 89.56733873860946, 89.58168902920285, 89.67496591805984, 89.68931620865322, 89.75389251632346, 89.8902202769606, 89.91892085814737, 89.92609600344407, 90.00502260170768, 89.99067231111431, 90.02654803759776, 90.05524861878453, 90.05524861878453, 90.11264978115807, 90.14135036234484, 90.07677405467462, 90.0839491999713, 90.10547463586138, 90.06959890937792, 90.04089832819116, 90.07677405467462, 90.0839491999713, 90.05524861878453, 90.06959890937792, 90.0839491999713, 90.06242376408122], "calls": [0.0, 1.2843510081079141, 2.4538996914687523, 3.501470904785822, 4.3553131950922, 5.474635861376194, 6.098873502188419, 6.89531463012126, 7.749156920427638, 8.63887493721748, 9.672095859941164, 10.353734663126929, 11.193226662839923, 11.867690320728995, 13.037239004089832, 14.063284781516824, 14.945827653009975, 15.928822558656814, 16.854416301930115, 17.76565975461003, 19.050010762717946, 19.997129941881322, 21.06622659108847, 21.812441701944465, 22.895888641744996, 24.008036162732296, 25.191935136686517, 26.217980914113507, 27.215326110353733, 28.37769964841788, 29.30329339169118, 30.350864605008248, 31.4199612542154, 32.24510296333501, 33.213747578388464, 34.31154480878238, 35.5887206715936, 36.413862380713205, 37.633637081150894, 38.59510655090765, 39.326971371170266, 40.34584200330057, 41.46516466958456, 42.562961899978475, 43.58900767740547, 44.514601420678765, 45.66979981344622, 46.717371026763296, 47.76494224008036, 48.67618569276028, 49.62330487192366, 50.584774341680415, 51.194661691899256, 52.37856066585348, 53.49788333213748, 54.437827366004164, 55.16969218626677, 56.00200904068308, 56.898902202769605, 57.95364856138336, 58.83619143287652, 59.69003372318289, 60.93133385951065, 61.97890507282773, 63.112578029705105, 63.9448948841214, 65.22207074693263, 66.18354021668938, 66.99433163521562, 68.51546243811437, 69.5056324890579, 70.68953146301212, 71.47162230035158, 72.60529525722896, 73.70309248762288, 74.53540934203917, 75.53992968357609, 76.78122981990386, 77.88620219559445, 78.89789768242807, 80.06027122049221, 80.80648633134821, 81.71772978402812, 82.95902992035589, 83.81287221066228, 84.83174284279256, 85.67123484250556, 86.6542297481524, 87.6731003802827, 88.6847958671163, 89.76106766162016, 90.5503336442563, 91.51897825930975, 92.63112578029705, 93.48496807060343, 94.58994044629404, 95.7379636937648, 96.92186266771901, 98.19903853053025, 99.14615770969361, 100.0], "rescued": [0, 73, 145, 207, 255, 315, 348, 387, 435, 479, 531, 564, 614, 651, 712, 762, 806, 848, 894, 940, 1003, 1041, 1086, 1123, 1170, 1213, 1262, 1302, 1346, 1383, 1425, 1468, 1508, 1536, 1576, 1620, 1664, 1711, 1762, 1804, 1825, 1864, 1899, 1935, 1972, 2003, 2041, 2073, 2097, 2115, 2138, 2161, 2178, 2209, 2233, 2248, 2263, 2276, 2295, 2315, 2333, 2345, 2367, 2381, 2400, 2413, 2426, 2444, 2456, 2477, 2499, 2513, 2525, 2539, 2557, 2563, 2574, 2596, 2606, 2618, 2640, 2650, 2658, 2676, 2682, 2692, 2702, 2709, 2721, 2732, 2736, 2742, 2754, 2758, 2759, 2771, 2778, 2781, 2789, 2791, 2791], "harmed": [0, 7, 16, 26, 31, 44, 47, 57, 65, 74, 82, 87, 97, 101, 112, 118, 126, 132, 139, 150, 159, 166, 178, 188, 200, 210, 230, 244, 260, 280, 296, 310, 331, 345, 360, 377, 388, 397, 404, 414, 425, 436, 448, 468, 473, 484, 494, 501, 510, 518, 528, 540, 545, 555, 563, 567, 574, 581, 589, 602, 614, 623, 631, 639, 650, 658, 672, 678, 684, 696, 700, 704, 711, 717, 724, 731, 740, 749, 757, 760, 763, 769, 776, 783, 791, 796, 802, 809, 813, 820, 833, 838, 847, 856, 861, 868, 874, 881, 887, 887, 890], "exact_score": [76.42247255506925, 76.78840496520056, 77.17586281122193, 77.54179522135324, 77.88620219559446, 78.20190858864893, 78.54631556289016, 78.81179593886777, 79.17055320370238, 79.42168328908662, 79.71586424625099, 80.088971801679, 80.40467819473345, 80.7203845877879, 81.01456554495229, 81.30874650211666, 81.58857716868766, 81.8468823993686, 82.17693908301644, 82.45676974958744, 82.70072468967497, 82.90162875798235, 83.15993398866327, 83.41823921934419, 83.62631843294827, 83.82004735595896, 83.99225084307957, 84.20750520198035, 84.30078209083734, 84.4371098514745, 84.67388964626534, 84.75281624452894, 84.96807060342972, 85.13309894525365, 85.28377699648418, 85.47033077419817, 85.73581115017579, 86.02281696204348, 86.23807132094426, 86.4676759704384, 86.61117887637225, 86.76185692760279, 86.85513381645978, 87.0273373035804, 87.23541651718448, 87.38609456841502, 87.56547320083232, 87.69462581617277, 87.82377843151325, 87.93858075626032, 88.01750735452393, 88.07490851689747, 88.21841142283131, 88.39779005524862, 88.45519121762216, 88.54129296118248, 88.58434383296262, 88.67762072181962, 88.72067159359977, 88.75654732008323, 88.79959819186338, 88.87134964483032, 88.92157566190716, 88.971801678984, 89.01485255076416, 89.00050226017078, 89.09377914902777, 89.13683002080792, 89.21575661907154, 89.23010690966493, 89.37360981559877, 89.44536126856569, 89.4238358326756, 89.52428786682931, 89.57451388390615, 89.57451388390615, 89.59603931979623, 89.69649135394992, 89.71084164454331, 89.75389251632346, 89.87586998636723, 89.91892085814737, 89.94762143933414, 89.997847456411, 90.01219774700438, 90.01219774700438, 90.06242376408122, 90.05524861878453, 90.11982492645475, 90.12700007175145, 90.10547463586138, 90.0839491999713, 90.09112434526799, 90.06959890937792, 90.0839491999713, 90.07677405467462, 90.0839491999713, 90.06242376408122, 90.05524861878453, 90.0839491999713, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.33667332391308, "threshold_area100": 86.26472698572147, "threshold_area25": 80.31068379134678}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.39467117190361023, "operator": "ge"}, {"budget": 0.02, "threshold": 0.34863367676734924, "operator": "ge"}, {"budget": 0.03, "threshold": 0.319738507270813, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2944691479206085, "operator": "ge"}, {"budget": 0.05, "threshold": 0.2675919532775879, "operator": "ge"}, {"budget": 0.06, "threshold": 0.24756276607513428, "operator": "ge"}, {"budget": 0.07, "threshold": 0.2291007936000824, "operator": "ge"}, {"budget": 0.08, "threshold": 0.21113619208335876, "operator": "ge"}, {"budget": 0.09, "threshold": 0.19014352560043335, "operator": "ge"}, {"budget": 0.1, "threshold": 0.1720106154680252, "operator": "ge"}, {"budget": 0.11, "threshold": 0.15313386917114258, "operator": "ge"}, {"budget": 0.12, "threshold": 0.13781185448169708, "operator": "ge"}, {"budget": 0.13, "threshold": 0.12406764924526215, "operator": "ge"}, {"budget": 0.14, "threshold": 0.10936620086431503, "operator": "ge"}, {"budget": 0.15, "threshold": 0.0942024290561676, "operator": "ge"}, {"budget": 0.16, "threshold": 0.08193294703960419, "operator": "ge"}, {"budget": 0.17, "threshold": 0.06927821040153503, "operator": "ge"}, {"budget": 0.18, "threshold": 0.05608811974525452, "operator": "ge"}, {"budget": 0.19, "threshold": 0.0434211827814579, "operator": "ge"}, {"budget": 0.2, "threshold": 0.03222271800041199, "operator": "ge"}, {"budget": 0.21, "threshold": 0.01962694525718689, "operator": "ge"}, {"budget": 0.22, "threshold": 0.010674823075532913, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0016347244381904602, "operator": "ge"}, {"budget": 0.24, "threshold": -0.002354629570618272, "operator": "ge"}, {"budget": 0.25, "threshold": -0.003529152600094676, "operator": "ge"}, {"budget": 0.26, "threshold": -0.00484571373090148, "operator": "ge"}, {"budget": 0.27, "threshold": -0.005690908525139093, "operator": "ge"}, {"budget": 0.28, "threshold": -0.007019669748842716, "operator": "ge"}, {"budget": 0.29, "threshold": -0.008040695451200008, "operator": "ge"}, {"budget": 0.3, "threshold": -0.009083347395062447, "operator": "ge"}, {"budget": 0.31, "threshold": -0.010214975103735924, "operator": "ge"}, {"budget": 0.32, "threshold": -0.01126069389283657, "operator": "ge"}, {"budget": 0.33, "threshold": -0.012269329279661179, "operator": "ge"}, {"budget": 0.34, "threshold": -0.013432656414806843, "operator": "ge"}, {"budget": 0.35, "threshold": -0.014307080768048763, "operator": "ge"}, {"budget": 0.36, "threshold": -0.015403063036501408, "operator": "ge"}, {"budget": 0.37, "threshold": -0.016633784398436546, "operator": "ge"}, {"budget": 0.38, "threshold": -0.01806660369038582, "operator": "ge"}, {"budget": 0.39, "threshold": -0.01931958645582199, "operator": "ge"}, {"budget": 0.4, "threshold": -0.020537663251161575, "operator": "ge"}, {"budget": 0.41, "threshold": -0.021741513162851334, "operator": "ge"}, {"budget": 0.42, "threshold": -0.02270733192563057, "operator": "ge"}, {"budget": 0.43, "threshold": -0.023804862052202225, "operator": "ge"}, {"budget": 0.44, "threshold": -0.024936234578490257, "operator": "ge"}, {"budget": 0.45, "threshold": -0.02618533931672573, "operator": "ge"}, {"budget": 0.46, "threshold": -0.02742629684507847, "operator": "ge"}, {"budget": 0.47, "threshold": -0.028761517256498337, "operator": "ge"}, {"budget": 0.48, "threshold": -0.030105778947472572, "operator": "ge"}, {"budget": 0.49, "threshold": -0.03141267970204353, "operator": "ge"}, {"budget": 0.5, "threshold": -0.03257499262690544, "operator": "ge"}, {"budget": 0.51, "threshold": -0.033791009336709976, "operator": "ge"}, {"budget": 0.52, "threshold": -0.03489704802632332, "operator": "ge"}, {"budget": 0.53, "threshold": -0.03619113191962242, "operator": "ge"}, {"budget": 0.54, "threshold": -0.037400417029857635, "operator": "ge"}, {"budget": 0.55, "threshold": -0.03857851028442383, "operator": "ge"}, {"budget": 0.56, "threshold": -0.039531558752059937, "operator": "ge"}, {"budget": 0.57, "threshold": -0.04089195653796196, "operator": "ge"}, {"budget": 0.58, "threshold": -0.04233691841363907, "operator": "ge"}, {"budget": 0.59, "threshold": -0.04362523555755615, "operator": "ge"}, {"budget": 0.6, "threshold": -0.04494483396410942, "operator": "ge"}, {"budget": 0.61, "threshold": -0.0465385764837265, "operator": "ge"}, {"budget": 0.62, "threshold": -0.04816737771034241, "operator": "ge"}, {"budget": 0.63, "threshold": -0.04967299848794937, "operator": "ge"}, {"budget": 0.64, "threshold": -0.05127239227294922, "operator": "ge"}, {"budget": 0.65, "threshold": -0.052859097719192505, "operator": "ge"}, {"budget": 0.66, "threshold": -0.054650913923978806, "operator": "ge"}, {"budget": 0.67, "threshold": -0.05659874901175499, "operator": "ge"}, {"budget": 0.68, "threshold": -0.05853817239403725, "operator": "ge"}, {"budget": 0.69, "threshold": -0.06051012873649597, "operator": "ge"}, {"budget": 0.7, "threshold": -0.06245966628193855, "operator": "ge"}, {"budget": 0.71, "threshold": -0.06426383554935455, "operator": "ge"}, {"budget": 0.72, "threshold": -0.06627590209245682, "operator": "ge"}, {"budget": 0.73, "threshold": -0.06840798258781433, "operator": "ge"}, {"budget": 0.74, "threshold": -0.07025939971208572, "operator": "ge"}, {"budget": 0.75, "threshold": -0.07238515466451645, "operator": "ge"}, {"budget": 0.76, "threshold": -0.07486971467733383, "operator": "ge"}, {"budget": 0.77, "threshold": -0.07749264687299728, "operator": "ge"}, {"budget": 0.78, "threshold": -0.08041618019342422, "operator": "ge"}, {"budget": 0.79, "threshold": -0.08304587006568909, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0861554816365242, "operator": "ge"}, {"budget": 0.81, "threshold": -0.08976656198501587, "operator": "ge"}, {"budget": 0.82, "threshold": -0.09268432855606079, "operator": "ge"}, {"budget": 0.83, "threshold": -0.09640207886695862, "operator": "ge"}, {"budget": 0.84, "threshold": -0.10052569955587387, "operator": "ge"}, {"budget": 0.85, "threshold": -0.10464103519916534, "operator": "ge"}, {"budget": 0.86, "threshold": -0.10831522941589355, "operator": "ge"}, {"budget": 0.87, "threshold": -0.11374153941869736, "operator": "ge"}, {"budget": 0.88, "threshold": -0.1194714903831482, "operator": "ge"}, {"budget": 0.89, "threshold": -0.126680389046669, "operator": "ge"}, {"budget": 0.9, "threshold": -0.13533106446266174, "operator": "ge"}, {"budget": 0.91, "threshold": -0.1401822417974472, "operator": "ge"}, {"budget": 0.92, "threshold": -0.14983277022838593, "operator": "ge"}, {"budget": 0.93, "threshold": -0.15959738194942474, "operator": "ge"}, {"budget": 0.94, "threshold": -0.17320725321769714, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1909855455160141, "operator": "ge"}, {"budget": 0.96, "threshold": -0.20874068140983582, "operator": "ge"}, {"budget": 0.97, "threshold": -0.2467743158340454, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2847498655319214, "operator": "ge"}, {"budget": 0.99, "threshold": -0.35713425278663635, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.90948152391864, 76.31051274706387, 76.75451160126038, 77.11257519335433, 77.35605843597823, 77.69979948438842, 78.07218562016614, 78.28702377542251, 78.54482956173015, 78.84560297908908, 79.18934402749929, 79.51876253222572, 79.89114866800342, 80.1775995416786, 80.39243769693498, 80.65024348324262, 80.97966198796907, 81.29475794901174, 81.50959610426811, 81.85333715267831, 82.0968203953022, 82.39759381266113, 82.6124319679175, 82.76997994843884, 82.78430249212259, 82.85591521054138, 82.92752792896017, 83.02778573474649, 83.11372099684904, 83.19965625895158, 83.29991406473789, 83.4001718705242, 83.471784588943, 83.54339730736179, 83.60068748209682, 83.65797765683185, 83.7439129189344, 83.81552563735319, 83.93010598682325, 84.00171870524204, 84.07333142366083, 84.1879117731309, 84.2595244915497, 84.28816957891722, 84.4313950157548, 84.50300773417358, 84.68920080206244, 84.83242623890003, 84.93268404468634, 85.11887711257519, 85.16184474362647, 85.23345746204527, 85.33371526783156, 85.47694070466915, 85.53423087940418, 85.76339157834431, 85.9066170151819, 85.99255227728445, 86.04984245201948, 86.16442280148954, 86.20739043254082, 86.37926095674592, 86.4508736751647, 86.56545402463478, 86.68003437410484, 86.88054998567746, 86.99513033514752, 87.15267831566887, 87.3102262961902, 87.45345173302779, 87.61099971354912, 87.7255800630192, 87.85448295617302, 87.94041821827557, 88.06932111142939, 88.269836723002, 88.39873961615584, 88.49899742194214, 88.65654540246346, 88.79977083930106, 88.95731881982239, 88.98596390718991, 89.1291893440275, 89.28673732454884, 89.47293039243769, 89.55886565454024, 89.6448009166428, 89.84531652821542, 89.98854196505299, 90.0887997708393, 90.20338012030936, 90.17473503294184, 90.24634775136064, 90.34660555714696, 90.47550845030077, 90.61873388713835, 90.57576625608708, 90.66170151818963, 90.71899169292466, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 28, 59, 91, 119, 142, 170, 199, 221, 244, 272, 300, 331, 359, 383, 406, 428, 454, 478, 501, 528, 550, 574, 596, 609, 612, 617, 623, 631, 638, 644, 651, 660, 667, 673, 678, 684, 691, 697, 706, 712, 718, 728, 735, 738, 751, 759, 772, 785, 795, 810, 815, 823, 834, 849, 854, 871, 881, 887, 894, 904, 910, 926, 933, 944, 956, 973, 986, 998, 1011, 1026, 1039, 1050, 1062, 1072, 1084, 1099, 1110, 1122, 1140, 1156, 1169, 1179, 1197, 1214, 1230, 1242, 1254, 1272, 1289, 1301, 1319, 1327, 1342, 1356, 1375, 1393, 1404, 1420, 1436, 1449], "harmed": [0, 4, 7, 8, 11, 17, 21, 24, 31, 36, 43, 47, 55, 57, 61, 69, 73, 76, 78, 86, 89, 94, 97, 104, 106, 108, 108, 109, 110, 111, 111, 111, 113, 115, 116, 117, 119, 120, 121, 122, 123, 124, 126, 128, 129, 132, 135, 135, 138, 141, 143, 145, 148, 152, 157, 158, 159, 159, 159, 162, 164, 167, 171, 173, 176, 180, 183, 188, 189, 191, 196, 198, 201, 204, 208, 211, 212, 214, 219, 226, 232, 234, 242, 250, 256, 259, 265, 271, 275, 282, 287, 297, 307, 317, 324, 334, 342, 356, 366, 378, 406], "exact_score": [75.56574047550845, 75.90948152391864, 76.31051274706387, 76.75451160126038, 77.11257519335433, 77.35605843597823, 77.69979948438842, 78.07218562016614, 78.28702377542251, 78.54482956173015, 78.84560297908908, 79.18934402749929, 79.51876253222572, 79.89114866800342, 80.1775995416786, 80.39243769693498, 80.65024348324262, 80.97966198796907, 81.29475794901174, 81.50959610426811, 81.85333715267831, 82.0968203953022, 82.39759381266113, 82.6124319679175, 82.76997994843884, 82.78430249212259, 82.85591521054138, 82.92752792896017, 83.02778573474649, 83.11372099684904, 83.19965625895158, 83.29991406473789, 83.4001718705242, 83.471784588943, 83.54339730736179, 83.60068748209682, 83.65797765683185, 83.7439129189344, 83.81552563735319, 83.93010598682325, 84.00171870524204, 84.07333142366083, 84.1879117731309, 84.2595244915497, 84.28816957891722, 84.4313950157548, 84.50300773417358, 84.68920080206244, 84.83242623890003, 84.93268404468634, 85.11887711257519, 85.16184474362647, 85.23345746204527, 85.33371526783156, 85.47694070466915, 85.53423087940418, 85.76339157834431, 85.9066170151819, 85.99255227728445, 86.04984245201948, 86.16442280148954, 86.20739043254082, 86.37926095674592, 86.4508736751647, 86.56545402463478, 86.68003437410484, 86.88054998567746, 86.99513033514752, 87.15267831566887, 87.3102262961902, 87.45345173302779, 87.61099971354912, 87.7255800630192, 87.85448295617302, 87.94041821827557, 88.06932111142939, 88.269836723002, 88.39873961615584, 88.49899742194214, 88.65654540246346, 88.79977083930106, 88.95731881982239, 88.98596390718991, 89.1291893440275, 89.28673732454884, 89.47293039243769, 89.55886565454024, 89.6448009166428, 89.84531652821542, 89.98854196505299, 90.0887997708393, 90.20338012030936, 90.17473503294184, 90.24634775136064, 90.34660555714696, 90.47550845030077, 90.61873388713835, 90.57576625608708, 90.66170151818963, 90.71899169292466, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 84.88555832196406, "threshold_area100": 84.88448868519049, "threshold_area25": 79.52449154969923}, "test": {"score": [76.42247255506925, 76.81710554638732, 77.16868766592523, 77.4843940589797, 77.81445074262754, 78.1445074262754, 78.43868838343977, 78.65394274234053, 78.94094855420822, 79.22077922077922, 79.52931046853699, 79.8880677333716, 80.09614694697568, 80.28270072468968, 80.56970653655738, 80.87106263901845, 81.1867690320729, 81.49530027983067, 81.74643036521489, 82.06213675826936, 82.32044198895028, 82.60744780081797, 82.85140274090551, 83.12405826217982, 83.2316854416302, 83.33213747578388, 83.43258950993759, 83.5043409629045, 83.61196814235488, 83.66936930472842, 83.74112075769534, 83.85592308244242, 83.94919997129942, 84.02095142426634, 84.0711774413432, 84.15010403960682, 84.22903063787042, 84.35100810791418, 84.44428499677119, 84.50886130444142, 84.58778790270503, 84.7025902274521, 84.78869197101243, 84.82456769749588, 84.88196885986942, 84.96807060342972, 85.0828729281768, 85.1689746717371, 85.29095214178088, 85.39140417593455, 85.47750591949487, 85.55643251775848, 85.6497094066155, 85.75733658606588, 85.80756260314271, 85.88648920140633, 85.92236492788979, 86.02281696204348, 86.16631986797732, 86.27394704742771, 86.32417306450456, 86.45332567984502, 86.51790198751526, 86.63987945755902, 86.72598120111932, 86.80490779938295, 86.9340604147234, 87.04886273947048, 87.19236564540432, 87.32151826074478, 87.42197029489847, 87.50807203845878, 87.62287436320587, 87.73767668795294, 87.81660328621655, 87.87400444859009, 88.00315706393054, 88.16818540575447, 88.26863743990816, 88.39061490995192, 88.47671665351223, 88.62021955944608, 88.67044557652292, 88.77807275597331, 88.86417449953362, 89.02920284135753, 89.0722537131377, 89.27315778144508, 89.27315778144508, 89.48841214034584, 89.6103896103896, 89.70366649924661, 89.83281911458707, 89.94762143933414, 90.01219774700438, 90.09112434526799, 90.16287579823492, 90.23462725120184, 90.30637870416876, 90.29202841357538, 90.06242376408122], "calls": [0.0, 1.2054244098442994, 2.3247470761282916, 3.192939657028055, 4.068307383224511, 4.965200545311042, 5.7472913826504985, 6.565257946473416, 7.325823347922795, 8.35904427064648, 9.298988304513166, 10.396785534907082, 11.221927244026691, 12.01119322666284, 12.857860371672528, 13.998708473846596, 15.103680849537204, 16.00774915692043, 16.84006601133673, 17.823060916983568, 18.84193154911387, 19.997129941881322, 20.87967281337447, 21.776565975461004, 22.78108631699792, 23.66362918849107, 24.775776709478368, 25.428714931477366, 26.63413934132166, 27.653009973451965, 28.528377699648416, 29.67640094711918, 30.731147305732943, 31.56346416014924, 32.51775848460932, 33.213747578388464, 34.23261821051876, 35.42369232976968, 36.57889072253713, 37.66233766233766, 38.59510655090765, 39.54222573007103, 40.32431656741049, 41.24991031068379, 42.218554925737244, 43.330702446724544, 44.414149386525075, 45.47607089043553, 46.5236421037526, 47.43488555643252, 48.389179880892584, 49.27172275238574, 50.06098873502188, 51.07268422185549, 51.99827796512879, 52.75166822128148, 53.48353304154409, 54.437827366004164, 55.60020090406831, 56.626246681495296, 57.50161440769176, 58.57071105689891, 59.67568343258951, 60.71607950060989, 61.58427208150965, 62.52421611537634, 63.81574226878094, 64.94941522565833, 66.09026332783239, 67.31003802827007, 68.32173351510367, 69.32625385664059, 70.42405108703451, 71.41422113797805, 72.2895888641745, 73.19365717155773, 74.35603070962186, 75.53992968357609, 76.84580612757408, 77.80727559733084, 78.91224797302145, 79.98851976752529, 80.98586496376552, 82.01191074119251, 83.21016000574012, 84.21468034727704, 84.93937002224295, 86.04434239793356, 87.0345124488771, 88.29016287579825, 89.59603931979623, 90.19157637942168, 91.21044701155199, 92.10734017363852, 93.22666283992251, 94.48948841214035, 95.4653081724905, 96.97208868479586, 97.84745641099232, 98.81610102604577, 100.0], "rescued": [0, 66, 128, 180, 233, 283, 333, 376, 417, 468, 520, 582, 619, 654, 701, 749, 803, 852, 892, 942, 985, 1039, 1077, 1123, 1142, 1157, 1172, 1183, 1198, 1207, 1220, 1240, 1255, 1267, 1275, 1289, 1305, 1325, 1342, 1358, 1372, 1392, 1407, 1415, 1426, 1444, 1464, 1479, 1499, 1516, 1533, 1549, 1564, 1583, 1595, 1611, 1622, 1641, 1662, 1684, 1700, 1721, 1739, 1760, 1781, 1800, 1824, 1848, 1876, 1902, 1924, 1945, 1971, 1997, 2015, 2035, 2064, 2100, 2126, 2144, 2166, 2194, 2217, 2237, 2269, 2305, 2320, 2360, 2377, 2420, 2459, 2477, 2509, 2540, 2567, 2605, 2637, 2689, 2725, 2751, 2791], "harmed": [0, 11, 24, 32, 39, 43, 52, 65, 66, 78, 87, 99, 107, 116, 123, 129, 139, 145, 150, 156, 163, 177, 181, 189, 193, 194, 195, 196, 196, 197, 200, 204, 206, 208, 209, 212, 217, 220, 224, 231, 234, 238, 241, 244, 247, 253, 257, 260, 263, 266, 271, 276, 278, 282, 287, 292, 298, 303, 304, 311, 320, 323, 332, 336, 345, 353, 359, 367, 375, 383, 391, 400, 410, 420, 427, 439, 450, 463, 475, 476, 486, 494, 510, 515, 535, 548, 557, 569, 586, 599, 621, 626, 640, 655, 673, 700, 722, 764, 790, 818, 890], "exact_score": [76.42247255506925, 76.76687952931047, 77.08976106766163, 77.40546746071608, 77.79292530673746, 78.15168257157208, 78.50326469110999, 78.81897108416446, 79.10597689603215, 79.45755901557007, 79.75173997273444, 80.06744636578891, 80.28987586998637, 80.59840711774415, 80.87106263901845, 81.14371816029275, 81.50247542512736, 81.78230609169836, 82.09801248475281, 82.35631771543373, 82.60744780081797, 82.88727846738897, 83.15993398866327, 83.24603573222358, 83.35366291167396, 83.46129009112435, 83.56174212527804, 83.63349357824497, 83.6837195953218, 83.7913467747722, 83.89897395422258, 83.97790055248619, 84.0568271507498, 84.15010403960682, 84.21468034727704, 84.31513238143073, 84.39405897969434, 84.45146014206787, 84.55908732151826, 84.63801391978187, 84.73129080863887, 84.8173925521992, 84.86761856927603, 84.91784458635287, 85.03982205639664, 85.12592379995695, 85.20485039822057, 85.33400301356102, 85.4416301930114, 85.54208222716511, 85.62100882542872, 85.75016144076918, 85.80756260314271, 85.9151897825931, 85.95106550907656, 86.09456841501041, 86.1878453038674, 86.30982277391116, 86.3743990815814, 86.49637655162518, 86.5681280045921, 86.67575518404249, 86.74750663700941, 86.85513381645978, 86.98428643180026, 87.04886273947048, 87.15648991892087, 87.26411709837124, 87.38609456841502, 87.50089689316209, 87.57264834612901, 87.6802755255794, 87.78790270502978, 87.87400444859009, 87.97445648274378, 88.07490851689747, 88.21841142283131, 88.27581258520485, 88.39779005524862, 88.47671665351223, 88.62021955944608, 88.67044557652292, 88.77807275597331, 88.83547391834685, 88.97897682428069, 89.10812943962115, 89.2659826361484, 89.2659826361484, 89.43818612326899, 89.57451388390615, 89.67496591805984, 89.83281911458707, 89.93327114874077, 89.94762143933414, 90.06242376408122, 90.14135036234484, 90.20592667001507, 90.23462725120184, 90.29920355887207, 90.29920355887207, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 85.26105046896913, "threshold_area100": 85.11383368013202, "threshold_area25": 80.14852550764154}, "selected_budget": 99, "selected": {"val": {"score": 90.71899169292466, "calls": 98.99742194213692, "rescued": 1436, "harmed": 378}, "test": {"score": 90.29202841357538, "calls": 98.81610102604577, "rescued": 2751, "harmed": 818}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.502920627593994, "operator": "ge"}, {"budget": 0.02, "threshold": 2.205763816833496, "operator": "ge"}, {"budget": 0.03, "threshold": 2.0081160068511963, "operator": "ge"}, {"budget": 0.04, "threshold": 1.8320002555847168, "operator": "ge"}, {"budget": 0.05, "threshold": 1.6879205703735352, "operator": "ge"}, {"budget": 0.06, "threshold": 1.5879255533218384, "operator": "ge"}, {"budget": 0.07, "threshold": 1.4922882318496704, "operator": "ge"}, {"budget": 0.08, "threshold": 1.3716920614242554, "operator": "ge"}, {"budget": 0.09, "threshold": 1.2853920459747314, "operator": "ge"}, {"budget": 0.1, "threshold": 1.1817607879638672, "operator": "ge"}, {"budget": 0.11, "threshold": 1.0990800857543945, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0044273138046265, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9231438040733337, "operator": "ge"}, {"budget": 0.14, "threshold": 0.8529651165008545, "operator": "ge"}, {"budget": 0.15, "threshold": 0.7536962032318115, "operator": "ge"}, {"budget": 0.16, "threshold": 0.656705379486084, "operator": "ge"}, {"budget": 0.17, "threshold": 0.5674192905426025, "operator": "ge"}, {"budget": 0.18, "threshold": 0.47759491205215454, "operator": "ge"}, {"budget": 0.19, "threshold": 0.3826315402984619, "operator": "ge"}, {"budget": 0.2, "threshold": 0.2776743769645691, "operator": "ge"}, {"budget": 0.21, "threshold": 0.1868211030960083, "operator": "ge"}, {"budget": 0.22, "threshold": 0.09562836587429047, "operator": "ge"}, {"budget": 0.23, "threshold": 0.015730436891317368, "operator": "ge"}, {"budget": 0.24, "threshold": -0.07699011266231537, "operator": "ge"}, {"budget": 0.25, "threshold": -0.16840192675590515, "operator": "ge"}, {"budget": 0.26, "threshold": -0.2872666120529175, "operator": "ge"}, {"budget": 0.27, "threshold": -0.3884469270706177, "operator": "ge"}, {"budget": 0.28, "threshold": -0.500806450843811, "operator": "ge"}, {"budget": 0.29, "threshold": -0.5988213419914246, "operator": "ge"}, {"budget": 0.3, "threshold": -0.7133344411849976, "operator": "ge"}, {"budget": 0.31, "threshold": -0.8315320014953613, "operator": "ge"}, {"budget": 0.32, "threshold": -0.9508777856826782, "operator": "ge"}, {"budget": 0.33, "threshold": -1.0687050819396973, "operator": "ge"}, {"budget": 0.34, "threshold": -1.1999574899673462, "operator": "ge"}, {"budget": 0.35, "threshold": -1.3550785779953003, "operator": "ge"}, {"budget": 0.36, "threshold": -1.5048353672027588, "operator": "ge"}, {"budget": 0.37, "threshold": -1.6863383054733276, "operator": "ge"}, {"budget": 0.38, "threshold": -1.8713970184326172, "operator": "ge"}, {"budget": 0.39, "threshold": -2.0201022624969482, "operator": "ge"}, {"budget": 0.4, "threshold": -2.1472456455230713, "operator": "ge"}, {"budget": 0.41, "threshold": -2.3191113471984863, "operator": "ge"}, {"budget": 0.42, "threshold": -2.4953055381774902, "operator": "ge"}, {"budget": 0.43, "threshold": -2.7024126052856445, "operator": "ge"}, {"budget": 0.44, "threshold": -2.858583927154541, "operator": "ge"}, {"budget": 0.45, "threshold": -3.011399507522583, "operator": "ge"}, {"budget": 0.46, "threshold": -3.1812005043029785, "operator": "ge"}, {"budget": 0.47, "threshold": -3.3575074672698975, "operator": "ge"}, {"budget": 0.48, "threshold": -3.4998817443847656, "operator": "ge"}, {"budget": 0.49, "threshold": -3.6577236652374268, "operator": "ge"}, {"budget": 0.5, "threshold": -3.8136534690856934, "operator": "ge"}, {"budget": 0.51, "threshold": -3.9317891597747803, "operator": "ge"}, {"budget": 0.52, "threshold": -4.077884197235107, "operator": "ge"}, {"budget": 0.53, "threshold": -4.204491138458252, "operator": "ge"}, {"budget": 0.54, "threshold": -4.331018924713135, "operator": "ge"}, {"budget": 0.55, "threshold": -4.443381309509277, "operator": "ge"}, {"budget": 0.56, "threshold": -4.549281120300293, "operator": "ge"}, {"budget": 0.57, "threshold": -4.650546073913574, "operator": "ge"}, {"budget": 0.58, "threshold": -4.749022006988525, "operator": "ge"}, {"budget": 0.59, "threshold": -4.858872890472412, "operator": "ge"}, {"budget": 0.6, "threshold": -4.951343536376953, "operator": "ge"}, {"budget": 0.61, "threshold": -5.046069622039795, "operator": "ge"}, {"budget": 0.62, "threshold": -5.156416893005371, "operator": "ge"}, {"budget": 0.63, "threshold": -5.263864994049072, "operator": "ge"}, {"budget": 0.64, "threshold": -5.347527980804443, "operator": "ge"}, {"budget": 0.65, "threshold": -5.439070224761963, "operator": "ge"}, {"budget": 0.66, "threshold": -5.532366752624512, "operator": "ge"}, {"budget": 0.67, "threshold": -5.615549087524414, "operator": "ge"}, {"budget": 0.68, "threshold": -5.7139434814453125, "operator": "ge"}, {"budget": 0.69, "threshold": -5.800436496734619, "operator": "ge"}, {"budget": 0.7, "threshold": -5.895418643951416, "operator": "ge"}, {"budget": 0.71, "threshold": -5.983736991882324, "operator": "ge"}, {"budget": 0.72, "threshold": -6.079428672790527, "operator": "ge"}, {"budget": 0.73, "threshold": -6.172077655792236, "operator": "ge"}, {"budget": 0.74, "threshold": -6.255557537078857, "operator": "ge"}, {"budget": 0.75, "threshold": -6.354596138000488, "operator": "ge"}, {"budget": 0.76, "threshold": -6.452750205993652, "operator": "ge"}, {"budget": 0.77, "threshold": -6.530435085296631, "operator": "ge"}, {"budget": 0.78, "threshold": -6.599440574645996, "operator": "ge"}, {"budget": 0.79, "threshold": -6.677258491516113, "operator": "ge"}, {"budget": 0.8, "threshold": -6.778475761413574, "operator": "ge"}, {"budget": 0.81, "threshold": -6.866765022277832, "operator": "ge"}, {"budget": 0.82, "threshold": -6.967536449432373, "operator": "ge"}, {"budget": 0.83, "threshold": -7.057764053344727, "operator": "ge"}, {"budget": 0.84, "threshold": -7.154922008514404, "operator": "ge"}, {"budget": 0.85, "threshold": -7.241004943847656, "operator": "ge"}, {"budget": 0.86, "threshold": -7.339784622192383, "operator": "ge"}, {"budget": 0.87, "threshold": -7.457113265991211, "operator": "ge"}, {"budget": 0.88, "threshold": -7.564226150512695, "operator": "ge"}, {"budget": 0.89, "threshold": -7.670220851898193, "operator": "ge"}, {"budget": 0.9, "threshold": -7.788367748260498, "operator": "ge"}, {"budget": 0.91, "threshold": -7.931129455566406, "operator": "ge"}, {"budget": 0.92, "threshold": -8.068185806274414, "operator": "ge"}, {"budget": 0.93, "threshold": -8.188619613647461, "operator": "ge"}, {"budget": 0.94, "threshold": -8.354957580566406, "operator": "ge"}, {"budget": 0.95, "threshold": -8.519783020019531, "operator": "ge"}, {"budget": 0.96, "threshold": -8.758870124816895, "operator": "ge"}, {"budget": 0.97, "threshold": -9.046317100524902, "operator": "ge"}, {"budget": 0.98, "threshold": -9.381691932678223, "operator": "ge"}, {"budget": 0.99, "threshold": -9.838029861450195, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.9954167860212, 76.36780292179891, 76.7115439702091, 77.08393010598682, 77.42767115439702, 77.80005729017473, 78.10083070753365, 78.40160412489257, 78.73102262961902, 79.00315095961042, 79.3182469206531, 79.61902033801202, 79.80521340590089, 80.14895445431108, 80.5213405900888, 80.86508163849899, 81.07991979375538, 81.48095101690059, 81.59553136637066, 81.89630478372959, 82.0968203953022, 82.39759381266113, 82.65539959896877, 82.87023775422514, 83.09939845316528, 83.38584932684044, 83.67230020051562, 83.78688054998568, 84.01604124892582, 84.13062159839588, 84.35978229733601, 84.56029790890862, 84.775136064165, 85.01861930678888, 85.17616728731022, 85.37668289888283, 85.66313377255801, 85.8063592093956, 85.8350042967631, 86.1071326267545, 86.3506158693784, 86.46519621884846, 86.55113148095101, 86.80893726725867, 86.86622744199369, 86.99513033514752, 87.12403322830134, 87.18132340303637, 87.28158120882267, 87.41048410197651, 87.53938699513033, 87.73990260670294, 87.88312804354052, 87.94041821827557, 88.01203093669436, 88.14093382984818, 88.29848181036952, 88.45602979089085, 88.59925522772845, 88.62790031509596, 88.72815812088226, 88.87138355771985, 88.92867373245488, 89.00028645087367, 88.98596390718991, 89.02893153824118, 89.04325408192494, 89.14351188771126, 89.22944714981381, 89.24376969349757, 89.22944714981381, 89.3297049556001, 89.38699513033515, 89.37267258665138, 89.38699513033515, 89.44428530507018, 89.47293039243769, 89.53022056717272, 89.55886565454024, 89.63047837295903, 89.65912346032655, 89.77370380979661, 89.85963907189917, 89.85963907189917, 89.94557433400172, 89.98854196505299, 90.00286450873675, 90.06015468347178, 90.13176740189057, 90.14608994557433, 90.24634775136064, 90.24634775136064, 90.23202520767688, 90.23202520767688, 90.26067029504439, 90.26067029504439, 90.31796046977942, 90.3609281008307, 90.46118590661702, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 60, 87, 113, 137, 166, 191, 217, 244, 265, 292, 316, 335, 362, 389, 422, 445, 477, 495, 524, 548, 573, 598, 621, 643, 670, 692, 709, 731, 744, 764, 786, 806, 828, 849, 869, 892, 909, 922, 943, 963, 978, 993, 1014, 1025, 1039, 1053, 1061, 1071, 1085, 1096, 1112, 1125, 1136, 1143, 1156, 1168, 1180, 1190, 1195, 1206, 1217, 1228, 1239, 1244, 1252, 1257, 1266, 1274, 1281, 1286, 1293, 1300, 1305, 1310, 1317, 1323, 1329, 1333, 1340, 1346, 1356, 1366, 1370, 1378, 1384, 1386, 1391, 1397, 1401, 1409, 1412, 1415, 1419, 1423, 1425, 1431, 1437, 1446, 1449], "harmed": [0, 1, 4, 7, 7, 7, 10, 14, 19, 23, 25, 30, 33, 39, 42, 43, 52, 60, 64, 74, 82, 92, 96, 103, 111, 117, 124, 126, 135, 141, 146, 150, 158, 163, 168, 178, 184, 187, 194, 205, 207, 210, 217, 226, 229, 236, 241, 246, 250, 253, 258, 260, 262, 265, 272, 274, 278, 279, 280, 280, 283, 287, 288, 295, 301, 307, 312, 316, 318, 320, 326, 332, 332, 335, 341, 345, 348, 352, 354, 356, 358, 362, 364, 368, 372, 374, 377, 378, 379, 380, 383, 384, 387, 391, 395, 397, 399, 401, 404, 406, 406], "exact_score": [75.56574047550845, 75.9954167860212, 76.36780292179891, 76.7115439702091, 77.08393010598682, 77.42767115439702, 77.80005729017473, 78.10083070753365, 78.40160412489257, 78.73102262961902, 79.00315095961042, 79.3182469206531, 79.61902033801202, 79.80521340590089, 80.14895445431108, 80.5213405900888, 80.86508163849899, 81.07991979375538, 81.48095101690059, 81.59553136637066, 81.89630478372959, 82.0968203953022, 82.39759381266113, 82.65539959896877, 82.87023775422514, 83.09939845316528, 83.38584932684044, 83.67230020051562, 83.78688054998568, 84.01604124892582, 84.13062159839588, 84.35978229733601, 84.56029790890862, 84.775136064165, 85.01861930678888, 85.17616728731022, 85.37668289888283, 85.66313377255801, 85.8063592093956, 85.8350042967631, 86.1071326267545, 86.3506158693784, 86.46519621884846, 86.55113148095101, 86.80893726725867, 86.86622744199369, 86.99513033514752, 87.12403322830134, 87.18132340303637, 87.28158120882267, 87.41048410197651, 87.53938699513033, 87.73990260670294, 87.88312804354052, 87.94041821827557, 88.01203093669436, 88.14093382984818, 88.29848181036952, 88.45602979089085, 88.59925522772845, 88.62790031509596, 88.72815812088226, 88.87138355771985, 88.92867373245488, 89.00028645087367, 88.98596390718991, 89.02893153824118, 89.04325408192494, 89.14351188771126, 89.22944714981381, 89.24376969349757, 89.22944714981381, 89.3297049556001, 89.38699513033515, 89.37267258665138, 89.38699513033515, 89.44428530507018, 89.47293039243769, 89.53022056717272, 89.55886565454024, 89.63047837295903, 89.65912346032655, 89.77370380979661, 89.85963907189917, 89.85963907189917, 89.94557433400172, 89.98854196505299, 90.00286450873675, 90.06015468347178, 90.13176740189057, 90.14608994557433, 90.24634775136064, 90.24634775136064, 90.23202520767688, 90.23202520767688, 90.26067029504439, 90.26067029504439, 90.31796046977942, 90.3609281008307, 90.46118590661702, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.96745910690181, "threshold_area100": 85.9664136350616, "threshold_area25": 79.63620739043253}, "test": {"score": [76.42247255506925, 76.88885699935423, 77.25478940948555, 77.5848460931334, 78.01535481093492, 78.40281265695631, 78.61089187056038, 78.876372246538, 79.18490349429577, 79.39298270789983, 79.56518619502046, 79.80914113510799, 80.13919781875583, 80.46925450240367, 80.68450886130445, 81.06479156202914, 81.38767310038028, 81.60292745928105, 81.86123268996197, 82.16258879242305, 82.49982062136758, 82.6791992537849, 82.90880390327904, 83.1312334074765, 83.38953863815742, 83.6047929970582, 83.76982133888211, 83.97072540718949, 84.1572791849035, 84.32230752672743, 84.56626246681496, 84.75281624452894, 84.91066944105619, 85.06852263758341, 85.1761498170338, 85.3627035947478, 85.59230824424195, 85.89366434670302, 86.19502044916409, 86.2811221927244, 86.46050082514172, 86.66858003874579, 86.78338236349286, 86.99863672239363, 87.18519050010764, 87.32869340604148, 87.52242232905216, 87.6802755255794, 87.75202697854631, 87.82377843151325, 87.90988017507355, 87.99598191863386, 88.15383511516109, 88.26146229461148, 88.38343976465524, 88.44084092702877, 88.47671665351223, 88.54129296118248, 88.64174499533615, 88.66327043122624, 88.70632130300639, 88.72784673889646, 88.84264906364353, 88.90722537131377, 88.96462653368731, 89.00050226017078, 89.065078567841, 89.065078567841, 89.16553060199469, 89.20858147377486, 89.30185836263185, 89.3377340891153, 89.38796010619215, 89.4238358326756, 89.48123699504916, 89.56016359331277, 89.58168902920285, 89.63909019157639, 89.72519193513668, 89.79694338810361, 89.84716940518047, 89.8902202769606, 89.93327114874077, 89.97632202052091, 90.01219774700438, 89.98349716581761, 90.02654803759776, 90.04089832819116, 90.09112434526799, 90.0982994905647, 90.0982994905647, 90.05524861878453, 90.02654803759776, 90.02654803759776, 90.04089832819116, 90.07677405467462, 90.07677405467462, 90.05524861878453, 90.05524861878453, 90.0839491999713, 90.06242376408122], "calls": [0.0, 1.3704527516682212, 2.3821482385018298, 3.4368945971155918, 4.455765229245892, 5.395709263112578, 6.127574083375189, 6.89531463012126, 7.842433809284638, 8.681925808997633, 9.542943244600703, 10.382435244313697, 11.372605295257229, 12.448877089761067, 13.1592164741336, 14.335940302791132, 15.405036951998277, 16.301930114084808, 17.12707182320442, 18.153117600631415, 19.222214249838558, 20.047355958958168, 20.994475138121548, 21.819616847241157, 22.895888641744996, 23.900408983281913, 25.148884264906364, 26.18210518763005, 27.32295328980412, 28.298773050154264, 29.210016502834186, 30.465666929755326, 31.463012125995547, 32.345554997488705, 33.29984932194877, 34.44069742412284, 35.50261892803329, 36.65064217550405, 37.75561455119466, 38.67403314917127, 39.34849680706034, 40.38171772978403, 41.34318719954079, 42.7208150965057, 43.71816029274593, 44.79443208724977, 45.89222931764368, 46.83934849680706, 47.64296477003659, 48.633134820980125, 49.63765516251704, 50.398220563966426, 51.38839061490995, 52.32833464877664, 53.43330702446725, 54.38760134892732, 55.25579392982708, 56.04505991246322, 56.898902202769605, 57.975173997273444, 58.850541723469895, 59.6828585778862, 60.809356389466885, 61.89997847456411, 62.79687163665064, 63.8587931405611, 65.00681638803187, 65.97546100308531, 67.29568773767669, 68.2069311903566, 69.26167754897037, 70.34512448877089, 71.49314773624165, 72.69139700078927, 73.58111501757911, 74.65738681208295, 75.74800889718017, 76.70947836693693, 77.64224725550693, 78.61806701585708, 79.9741694769319, 81.03609098084236, 82.05496161297266, 82.99490564683934, 83.90614909951927, 84.73129080863887, 85.76451173136256, 86.91971012413, 87.73767668795293, 88.52694267058908, 89.58886417449953, 90.57903422544307, 91.5692042763866, 92.33694482313267, 93.5136686517902, 94.47513812154696, 95.6375116596111, 96.97926383009255, 98.19186338523355, 99.2681351797374, 100.0], "rescued": [0, 70, 126, 181, 247, 305, 340, 385, 436, 470, 505, 544, 600, 653, 689, 750, 803, 845, 889, 942, 1001, 1035, 1081, 1123, 1173, 1211, 1259, 1303, 1351, 1388, 1432, 1482, 1515, 1551, 1586, 1626, 1672, 1720, 1770, 1801, 1831, 1871, 1900, 1949, 1983, 2013, 2049, 2077, 2094, 2115, 2134, 2153, 2183, 2208, 2232, 2247, 2259, 2274, 2300, 2316, 2329, 2339, 2361, 2381, 2395, 2411, 2433, 2443, 2460, 2474, 2492, 2508, 2521, 2536, 2550, 2564, 2580, 2594, 2608, 2622, 2635, 2652, 2665, 2678, 2687, 2692, 2701, 2706, 2718, 2728, 2735, 2741, 2747, 2751, 2762, 2769, 2777, 2781, 2787, 2791, 2791], "harmed": [0, 5, 10, 19, 25, 29, 35, 43, 51, 56, 67, 72, 82, 89, 95, 103, 111, 123, 131, 142, 154, 163, 177, 188, 202, 210, 235, 251, 273, 287, 297, 321, 332, 346, 366, 380, 394, 400, 408, 427, 432, 443, 456, 475, 483, 493, 502, 508, 515, 526, 533, 540, 548, 558, 565, 572, 579, 585, 597, 610, 617, 624, 630, 641, 647, 658, 671, 681, 684, 692, 697, 708, 714, 724, 730, 733, 746, 752, 754, 758, 764, 775, 782, 789, 793, 802, 805, 808, 813, 822, 829, 841, 851, 855, 864, 866, 874, 881, 887, 887, 890], "exact_score": [76.42247255506925, 76.75970438401379, 77.08976106766163, 77.44134318719954, 77.76422472555069, 78.2306091698357, 78.56784099878023, 78.91224797302145, 79.20642893018584, 79.45755901557007, 79.7086891009543, 80.04592092989883, 80.3544521776566, 80.66298342541437, 80.93563894668867, 81.28722106622659, 81.51682571572074, 81.81818181818183, 82.1123627753462, 82.3993685872139, 82.6720241084882, 82.90880390327904, 83.16710913395997, 83.41823921934419, 83.58326756116811, 83.75547104828873, 83.94202482600272, 84.10705316782665, 84.25055607376049, 84.51603644973811, 84.65953935567195, 84.84609313338596, 84.98242089402311, 85.11157350936357, 85.27660185118748, 85.44880533830809, 85.74298629547249, 85.99411638085671, 86.24524646624094, 86.36722393628472, 86.58965344048217, 86.74750663700941, 86.8479586711631, 87.04886273947048, 87.22824137188778, 87.35739398722825, 87.53677261964555, 87.68745067087609, 87.76637726913971, 87.85247901270002, 87.945755901557, 88.11795938867762, 88.23993685872139, 88.36191432876515, 88.41931549113869, 88.49106694410563, 88.54129296118248, 88.64174499533615, 88.66327043122624, 88.69914615770969, 88.74937217478654, 88.86417449953362, 88.90722537131377, 88.94310109779724, 89.01485255076416, 89.065078567841, 89.07942885843438, 89.13683002080792, 89.18705603788477, 89.28033292674176, 89.32338379852193, 89.31620865322523, 89.40231039678554, 89.45253641386238, 89.49558728564253, 89.59603931979623, 89.6175647556863, 89.68214106335654, 89.73236708043338, 89.82564396929038, 89.84716940518047, 89.8973954222573, 89.93327114874077, 89.97632202052091, 90.01937289230108, 89.997847456411, 90.02654803759776, 90.04089832819116, 90.0839491999713, 90.10547463586138, 90.07677405467462, 90.03372318289445, 90.04089832819116, 90.05524861878453, 90.06959890937792, 90.06242376408122, 90.07677405467462, 90.05524861878453, 90.05524861878453, 90.07677405467462, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.31543516154495, "threshold_area100": 86.25489703666499, "threshold_area25": 80.30924876228744}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.8584784269332886, "operator": "ge"}, {"budget": 0.02, "threshold": 1.5894412994384766, "operator": "ge"}, {"budget": 0.03, "threshold": 1.4135041236877441, "operator": "ge"}, {"budget": 0.04, "threshold": 1.2849738597869873, "operator": "ge"}, {"budget": 0.05, "threshold": 1.16457998752594, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0886114835739136, "operator": "ge"}, {"budget": 0.07, "threshold": 1.0012866258621216, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9118425846099854, "operator": "ge"}, {"budget": 0.09, "threshold": 0.8331094980239868, "operator": "ge"}, {"budget": 0.1, "threshold": 0.7580855488777161, "operator": "ge"}, {"budget": 0.11, "threshold": 0.7059284448623657, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6478668451309204, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5962318778038025, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5318789482116699, "operator": "ge"}, {"budget": 0.15, "threshold": 0.474806547164917, "operator": "ge"}, {"budget": 0.16, "threshold": 0.4188440442085266, "operator": "ge"}, {"budget": 0.17, "threshold": 0.36074909567832947, "operator": "ge"}, {"budget": 0.18, "threshold": 0.3131091296672821, "operator": "ge"}, {"budget": 0.19, "threshold": 0.26424741744995117, "operator": "ge"}, {"budget": 0.2, "threshold": 0.19306181371212006, "operator": "ge"}, {"budget": 0.21, "threshold": 0.1345335841178894, "operator": "ge"}, {"budget": 0.22, "threshold": 0.06927421689033508, "operator": "ge"}, {"budget": 0.23, "threshold": 0.012761414051055908, "operator": "ge"}, {"budget": 0.24, "threshold": -0.05557435750961304, "operator": "ge"}, {"budget": 0.25, "threshold": -0.13210460543632507, "operator": "ge"}, {"budget": 0.26, "threshold": -0.2203691005706787, "operator": "ge"}, {"budget": 0.27, "threshold": -0.3031727373600006, "operator": "ge"}, {"budget": 0.28, "threshold": -0.3889642357826233, "operator": "ge"}, {"budget": 0.29, "threshold": -0.480696439743042, "operator": "ge"}, {"budget": 0.3, "threshold": -0.5725237131118774, "operator": "ge"}, {"budget": 0.31, "threshold": -0.674651563167572, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7738996148109436, "operator": "ge"}, {"budget": 0.33, "threshold": -0.879280686378479, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9950155019760132, "operator": "ge"}, {"budget": 0.35, "threshold": -1.1318315267562866, "operator": "ge"}, {"budget": 0.36, "threshold": -1.290116786956787, "operator": "ge"}, {"budget": 0.37, "threshold": -1.422454595565796, "operator": "ge"}, {"budget": 0.38, "threshold": -1.5944736003875732, "operator": "ge"}, {"budget": 0.39, "threshold": -1.7605962753295898, "operator": "ge"}, {"budget": 0.4, "threshold": -1.8896747827529907, "operator": "ge"}, {"budget": 0.41, "threshold": -2.0449118614196777, "operator": "ge"}, {"budget": 0.42, "threshold": -2.222407817840576, "operator": "ge"}, {"budget": 0.43, "threshold": -2.392946720123291, "operator": "ge"}, {"budget": 0.44, "threshold": -2.5659477710723877, "operator": "ge"}, {"budget": 0.45, "threshold": -2.712751865386963, "operator": "ge"}, {"budget": 0.46, "threshold": -2.8824679851531982, "operator": "ge"}, {"budget": 0.47, "threshold": -3.0479137897491455, "operator": "ge"}, {"budget": 0.48, "threshold": -3.21262264251709, "operator": "ge"}, {"budget": 0.49, "threshold": -3.3696272373199463, "operator": "ge"}, {"budget": 0.5, "threshold": -3.52238392829895, "operator": "ge"}, {"budget": 0.51, "threshold": -3.6608712673187256, "operator": "ge"}, {"budget": 0.52, "threshold": -3.7865395545959473, "operator": "ge"}, {"budget": 0.53, "threshold": -3.9324300289154053, "operator": "ge"}, {"budget": 0.54, "threshold": -4.07125997543335, "operator": "ge"}, {"budget": 0.55, "threshold": -4.190174579620361, "operator": "ge"}, {"budget": 0.56, "threshold": -4.280588150024414, "operator": "ge"}, {"budget": 0.57, "threshold": -4.381833076477051, "operator": "ge"}, {"budget": 0.58, "threshold": -4.493464469909668, "operator": "ge"}, {"budget": 0.59, "threshold": -4.602156639099121, "operator": "ge"}, {"budget": 0.6, "threshold": -4.689562797546387, "operator": "ge"}, {"budget": 0.61, "threshold": -4.786945819854736, "operator": "ge"}, {"budget": 0.62, "threshold": -4.9061479568481445, "operator": "ge"}, {"budget": 0.63, "threshold": -4.996505260467529, "operator": "ge"}, {"budget": 0.64, "threshold": -5.0996785163879395, "operator": "ge"}, {"budget": 0.65, "threshold": -5.195086479187012, "operator": "ge"}, {"budget": 0.66, "threshold": -5.290567398071289, "operator": "ge"}, {"budget": 0.67, "threshold": -5.376016616821289, "operator": "ge"}, {"budget": 0.68, "threshold": -5.44826602935791, "operator": "ge"}, {"budget": 0.69, "threshold": -5.576233863830566, "operator": "ge"}, {"budget": 0.7, "threshold": -5.654571533203125, "operator": "ge"}, {"budget": 0.71, "threshold": -5.753533363342285, "operator": "ge"}, {"budget": 0.72, "threshold": -5.829071998596191, "operator": "ge"}, {"budget": 0.73, "threshold": -5.918971538543701, "operator": "ge"}, {"budget": 0.74, "threshold": -6.015758514404297, "operator": "ge"}, {"budget": 0.75, "threshold": -6.092360496520996, "operator": "ge"}, {"budget": 0.76, "threshold": -6.172721862792969, "operator": "ge"}, {"budget": 0.77, "threshold": -6.28202486038208, "operator": "ge"}, {"budget": 0.78, "threshold": -6.371387004852295, "operator": "ge"}, {"budget": 0.79, "threshold": -6.44459342956543, "operator": "ge"}, {"budget": 0.8, "threshold": -6.529719829559326, "operator": "ge"}, {"budget": 0.81, "threshold": -6.601215362548828, "operator": "ge"}, {"budget": 0.82, "threshold": -6.686908721923828, "operator": "ge"}, {"budget": 0.83, "threshold": -6.798628330230713, "operator": "ge"}, {"budget": 0.84, "threshold": -6.886336803436279, "operator": "ge"}, {"budget": 0.85, "threshold": -6.9905009269714355, "operator": "ge"}, {"budget": 0.86, "threshold": -7.085839748382568, "operator": "ge"}, {"budget": 0.87, "threshold": -7.179080009460449, "operator": "ge"}, {"budget": 0.88, "threshold": -7.308583736419678, "operator": "ge"}, {"budget": 0.89, "threshold": -7.4340033531188965, "operator": "ge"}, {"budget": 0.9, "threshold": -7.554991722106934, "operator": "ge"}, {"budget": 0.91, "threshold": -7.6837849617004395, "operator": "ge"}, {"budget": 0.92, "threshold": -7.818763732910156, "operator": "ge"}, {"budget": 0.93, "threshold": -7.975096702575684, "operator": "ge"}, {"budget": 0.94, "threshold": -8.107943534851074, "operator": "ge"}, {"budget": 0.95, "threshold": -8.301252365112305, "operator": "ge"}, {"budget": 0.96, "threshold": -8.513957023620605, "operator": "ge"}, {"budget": 0.97, "threshold": -8.793746948242188, "operator": "ge"}, {"budget": 0.98, "threshold": -9.1408052444458, "operator": "ge"}, {"budget": 0.99, "threshold": -9.54840087890625, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.65425379547408, 77.02663993125178, 77.3990260670295, 77.64250930965339, 77.95760527069608, 78.31566886279003, 78.64508736751647, 78.91721569750787, 79.24663420223432, 79.57605270696075, 79.93411629905471, 80.26353480378114, 80.57863076482383, 80.92237181323402, 81.0655972500716, 81.35204812374678, 81.62417645373819, 81.91062732741334, 82.0681753079347, 82.38327126897737, 82.64107705528502, 82.8845602979089, 83.12804354053279, 83.4001718705242, 83.54339730736179, 83.82984818103695, 84.03036379260956, 84.14494414207962, 84.33113720996849, 84.50300773417358, 84.7035233457462, 84.93268404468634, 85.16184474362647, 85.37668289888283, 85.63448868519049, 85.9066170151819, 86.0068748209682, 86.078487539387, 86.30764823832713, 86.47951876253222, 86.6943569177886, 86.70867946147236, 86.90919507304497, 86.99513033514752, 87.12403322830134, 87.19564594672013, 87.35319392724148, 87.45345173302779, 87.5823546261816, 87.69693497565167, 87.89745058722428, 87.9977083930106, 88.08364365511314, 88.19822400458321, 88.34144944142079, 88.45602979089085, 88.59925522772845, 88.74248066456603, 88.84273847035233, 88.8857061014036, 88.95731881982239, 88.98596390718991, 89.07189916929245, 89.04325408192494, 89.07189916929245, 89.08622171297623, 89.2008020624463, 89.22944714981381, 89.3297049556001, 89.37267258665138, 89.42996276138642, 89.42996276138642, 89.42996276138642, 89.41564021770266, 89.50157547980521, 89.58751074190776, 89.63047837295903, 89.65912346032655, 89.70209109137782, 89.75938126611285, 89.77370380979661, 89.87396161558291, 89.9169292466342, 89.95989687768548, 90.03150959610426, 90.0887997708393, 90.10312231452305, 90.17473503294184, 90.24634775136064, 90.26067029504439, 90.27499283872815, 90.23202520767688, 90.24634775136064, 90.31796046977942, 90.28931538241191, 90.40389573188197, 90.46118590661702, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 59, 84, 112, 140, 163, 189, 218, 244, 265, 293, 321, 352, 380, 407, 436, 453, 475, 500, 526, 548, 572, 597, 622, 642, 669, 684, 709, 729, 743, 760, 779, 800, 821, 844, 867, 890, 911, 926, 940, 958, 977, 995, 1005, 1024, 1038, 1050, 1060, 1073, 1085, 1097, 1108, 1123, 1136, 1146, 1157, 1169, 1180, 1190, 1201, 1212, 1221, 1231, 1238, 1246, 1254, 1260, 1263, 1273, 1280, 1290, 1295, 1300, 1307, 1311, 1315, 1325, 1333, 1338, 1345, 1350, 1356, 1361, 1370, 1376, 1381, 1388, 1394, 1395, 1401, 1409, 1412, 1417, 1419, 1423, 1428, 1430, 1440, 1446, 1449], "harmed": [0, 3, 6, 8, 10, 12, 18, 22, 26, 29, 31, 36, 41, 47, 52, 57, 62, 69, 71, 77, 83, 94, 96, 103, 111, 114, 122, 127, 132, 138, 144, 148, 155, 162, 167, 174, 182, 187, 189, 197, 206, 208, 215, 218, 227, 232, 240, 243, 248, 250, 255, 258, 261, 262, 268, 272, 275, 277, 280, 280, 281, 285, 291, 296, 301, 303, 313, 317, 319, 321, 326, 329, 331, 332, 339, 343, 348, 352, 354, 356, 361, 363, 365, 369, 371, 374, 376, 378, 380, 380, 381, 384, 386, 390, 395, 398, 398, 402, 404, 406, 406], "exact_score": [75.56574047550845, 75.98109424233743, 76.32483529074764, 76.65425379547408, 77.02663993125178, 77.3990260670295, 77.64250930965339, 77.95760527069608, 78.31566886279003, 78.64508736751647, 78.91721569750787, 79.24663420223432, 79.57605270696075, 79.93411629905471, 80.26353480378114, 80.57863076482383, 80.92237181323402, 81.0655972500716, 81.35204812374678, 81.62417645373819, 81.91062732741334, 82.0681753079347, 82.38327126897737, 82.64107705528502, 82.8845602979089, 83.12804354053279, 83.4001718705242, 83.54339730736179, 83.82984818103695, 84.03036379260956, 84.14494414207962, 84.33113720996849, 84.50300773417358, 84.7035233457462, 84.93268404468634, 85.16184474362647, 85.37668289888283, 85.63448868519049, 85.9066170151819, 86.0068748209682, 86.078487539387, 86.30764823832713, 86.47951876253222, 86.6943569177886, 86.70867946147236, 86.90919507304497, 86.99513033514752, 87.12403322830134, 87.19564594672013, 87.35319392724148, 87.45345173302779, 87.5823546261816, 87.69693497565167, 87.89745058722428, 87.9977083930106, 88.08364365511314, 88.19822400458321, 88.34144944142079, 88.45602979089085, 88.59925522772845, 88.74248066456603, 88.84273847035233, 88.8857061014036, 88.95731881982239, 88.98596390718991, 89.07189916929245, 89.04325408192494, 89.07189916929245, 89.08622171297623, 89.2008020624463, 89.22944714981381, 89.3297049556001, 89.37267258665138, 89.42996276138642, 89.42996276138642, 89.42996276138642, 89.41564021770266, 89.50157547980521, 89.58751074190776, 89.63047837295903, 89.65912346032655, 89.70209109137782, 89.75938126611285, 89.77370380979661, 89.87396161558291, 89.9169292466342, 89.95989687768548, 90.03150959610426, 90.0887997708393, 90.10312231452305, 90.17473503294184, 90.24634775136064, 90.26067029504439, 90.27499283872815, 90.23202520767688, 90.24634775136064, 90.31796046977942, 90.28931538241191, 90.40389573188197, 90.46118590661702, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.97261825862978, "threshold_area100": 85.97156975078774, "threshold_area25": 79.63047837295905}, "test": {"score": [76.42247255506925, 76.89603214465093, 77.34806629834254, 77.72117385377054, 78.02970510152831, 78.36693693047285, 78.58219128937361, 78.79027050297769, 79.07727631484538, 79.32840640022961, 79.64411279328407, 79.84501686159145, 80.13202267345913, 80.36880246825, 80.7275597330846, 81.04326612613906, 81.30157135681998, 81.5598765875009, 81.8397072540719, 82.09083733945613, 82.4782951854775, 82.70072468967497, 82.93750448446582, 83.1312334074765, 83.38236349286073, 83.61914328765158, 83.82722250125565, 84.01377627896966, 84.21468034727704, 84.3366578173208, 84.5232115950348, 84.73129080863887, 84.86761856927603, 84.96807060342972, 85.14744923584703, 85.34117815885772, 85.57795795364856, 85.85061347492287, 86.16631986797732, 86.39592451747149, 86.4676759704384, 86.66858003874579, 86.8336083805697, 86.94841070531677, 87.17801535481094, 87.32151826074478, 87.52242232905216, 87.70180096146947, 87.80942814091985, 87.88117959388678, 87.97445648274378, 88.0533830810074, 88.1394848245677, 88.29016287579825, 88.40496520054532, 88.48389179880894, 88.54129296118248, 88.58434383296262, 88.66327043122624, 88.71349644830309, 88.75654732008323, 88.77807275597331, 88.878524790127, 88.92157566190716, 88.97897682428069, 89.01485255076416, 89.00767740546746, 89.09377914902777, 89.13683002080792, 89.20140632847816, 89.33055894381862, 89.40231039678554, 89.43818612326899, 89.49558728564253, 89.57451388390615, 89.56733873860946, 89.58168902920285, 89.67496591805984, 89.68931620865322, 89.75389251632346, 89.8902202769606, 89.91892085814737, 89.92609600344407, 90.00502260170768, 89.99067231111431, 90.02654803759776, 90.05524861878453, 90.05524861878453, 90.10547463586138, 90.14135036234484, 90.07677405467462, 90.0839491999713, 90.10547463586138, 90.06959890937792, 90.04089832819116, 90.07677405467462, 90.0839491999713, 90.05524861878453, 90.06959890937792, 90.0839491999713, 90.06242376408122], "calls": [0.0, 1.2843510081079141, 2.4538996914687523, 3.501470904785822, 4.3553131950922, 5.474635861376194, 6.098873502188419, 6.89531463012126, 7.749156920427638, 8.63887493721748, 9.672095859941164, 10.353734663126929, 11.193226662839923, 11.867690320728995, 13.037239004089832, 14.063284781516824, 14.945827653009975, 15.928822558656814, 16.854416301930115, 17.76565975461003, 19.050010762717946, 19.997129941881322, 21.06622659108847, 21.812441701944465, 22.895888641744996, 24.008036162732296, 25.191935136686517, 26.217980914113507, 27.215326110353733, 28.37769964841788, 29.30329339169118, 30.350864605008248, 31.4199612542154, 32.24510296333501, 33.213747578388464, 34.31154480878238, 35.5887206715936, 36.413862380713205, 37.633637081150894, 38.59510655090765, 39.326971371170266, 40.34584200330057, 41.46516466958456, 42.562961899978475, 43.58900767740547, 44.514601420678765, 45.66979981344622, 46.717371026763296, 47.76494224008036, 48.67618569276028, 49.62330487192366, 50.584774341680415, 51.194661691899256, 52.37856066585348, 53.49788333213748, 54.437827366004164, 55.16969218626677, 56.00200904068308, 56.898902202769605, 57.95364856138336, 58.83619143287652, 59.69003372318289, 60.93133385951065, 61.97890507282773, 63.112578029705105, 63.9448948841214, 65.22207074693263, 66.18354021668938, 66.99433163521562, 68.51546243811437, 69.49845734376122, 70.68953146301212, 71.47162230035158, 72.60529525722896, 73.69591734232618, 74.53540934203917, 75.53992968357609, 76.78122981990386, 77.88620219559445, 78.89789768242807, 80.06027122049221, 80.80648633134821, 81.71772978402812, 82.95902992035589, 83.80569706536556, 84.82456769749588, 85.67123484250556, 86.6542297481524, 87.68027552557939, 88.6847958671163, 89.76106766162016, 90.5503336442563, 91.51180311401306, 92.62395063500037, 93.48496807060343, 94.58276530099734, 95.7379636937648, 96.92186266771901, 98.19186338523355, 99.14615770969361, 100.0], "rescued": [0, 73, 145, 207, 255, 315, 348, 387, 435, 479, 531, 564, 614, 651, 712, 762, 806, 848, 894, 940, 1003, 1041, 1086, 1123, 1170, 1213, 1262, 1302, 1346, 1383, 1425, 1468, 1508, 1536, 1576, 1620, 1664, 1711, 1762, 1804, 1825, 1864, 1899, 1935, 1972, 2003, 2041, 2073, 2097, 2115, 2138, 2161, 2178, 2209, 2233, 2248, 2263, 2276, 2295, 2315, 2333, 2345, 2367, 2381, 2400, 2413, 2426, 2444, 2456, 2477, 2499, 2513, 2525, 2539, 2557, 2563, 2574, 2596, 2606, 2618, 2640, 2650, 2658, 2676, 2682, 2692, 2702, 2709, 2721, 2732, 2736, 2742, 2754, 2758, 2759, 2771, 2778, 2781, 2789, 2791, 2791], "harmed": [0, 7, 16, 26, 31, 44, 47, 57, 65, 74, 82, 87, 97, 101, 112, 118, 126, 132, 139, 150, 159, 166, 178, 188, 200, 210, 230, 244, 260, 280, 296, 310, 331, 345, 360, 377, 388, 397, 404, 414, 425, 436, 448, 468, 473, 484, 494, 501, 510, 518, 528, 540, 545, 555, 563, 567, 574, 581, 589, 602, 614, 623, 631, 639, 650, 658, 672, 678, 684, 696, 700, 704, 711, 717, 724, 731, 740, 749, 757, 760, 763, 769, 776, 783, 791, 796, 802, 809, 814, 820, 833, 838, 847, 856, 861, 868, 874, 881, 887, 887, 890], "exact_score": [76.42247255506925, 76.78840496520056, 77.17586281122193, 77.54179522135324, 77.88620219559446, 78.20190858864893, 78.54631556289016, 78.81179593886777, 79.17055320370238, 79.42168328908662, 79.71586424625099, 80.088971801679, 80.40467819473345, 80.7203845877879, 81.01456554495229, 81.30874650211666, 81.58857716868766, 81.8468823993686, 82.17693908301644, 82.45676974958744, 82.70072468967497, 82.90162875798235, 83.15993398866327, 83.41823921934419, 83.62631843294827, 83.82004735595896, 83.99225084307957, 84.20750520198035, 84.30078209083734, 84.4371098514745, 84.67388964626534, 84.75281624452894, 84.96807060342972, 85.13309894525365, 85.28377699648418, 85.47033077419817, 85.73581115017579, 86.02281696204348, 86.23807132094426, 86.4676759704384, 86.61117887637225, 86.76185692760279, 86.85513381645978, 87.0273373035804, 87.23541651718448, 87.38609456841502, 87.56547320083232, 87.69462581617277, 87.82377843151325, 87.93858075626032, 88.01750735452393, 88.07490851689747, 88.21841142283131, 88.39779005524862, 88.45519121762216, 88.54129296118248, 88.58434383296262, 88.67762072181962, 88.72067159359977, 88.75654732008323, 88.79959819186338, 88.87134964483032, 88.92157566190716, 88.971801678984, 89.01485255076416, 89.00050226017078, 89.09377914902777, 89.13683002080792, 89.21575661907154, 89.23010690966493, 89.37360981559877, 89.44536126856569, 89.4238358326756, 89.52428786682931, 89.57451388390615, 89.57451388390615, 89.59603931979623, 89.69649135394992, 89.71084164454331, 89.75389251632346, 89.87586998636723, 89.91892085814737, 89.94762143933414, 89.997847456411, 90.01219774700438, 90.01219774700438, 90.06242376408122, 90.05524861878453, 90.11982492645475, 90.12700007175145, 90.10547463586138, 90.0839491999713, 90.09112434526799, 90.06959890937792, 90.0839491999713, 90.07677405467462, 90.0839491999713, 90.06242376408122, 90.05524861878453, 90.0839491999713, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.33667332391308, "threshold_area100": 86.26465523426849, "threshold_area25": 80.31068379134678}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.54712116871956, "calls": 98.99742194213692}, "test": {"score": 90.28485326827868, "calls": 98.9883045131664}}, "old_normalized_interior": {"val": {"score": 90.53279862503581, "calls": 98.99742194213692}, "test": {"score": 90.02654803759776, "calls": 99.06723111143}}, "old_normalized_endpoints": {"val": {"score": 90.53279862503581, "calls": 98.99742194213692}, "test": {"score": 90.02654803759776, "calls": 99.06723111143}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 75.88083643655112, 76.21025494127757, 76.72586651389285, 77.12689773703809, 77.47063878544829, 77.94328272701232, 78.15812088226869, 78.57347464909768, 78.94586078487539, 79.33256946433687, 79.70495560011457, 79.99140647378974, 80.26353480378114, 80.42108278430248, 80.70753365797765, 81.0655972500716, 81.32340303637926, 81.5812088226869, 81.79604697794328, 82.0968203953022, 82.3689487252936, 82.62675451160125, 82.84159266685764, 82.91320538527643, 83.05643082211401, 83.27126897737038, 83.471784588943, 83.68662274419937, 83.82984818103695, 83.97307361787453, 84.0876539673446, 84.37410484101976, 84.67487825837868, 84.8037811515325, 84.83242623890003, 84.97565167573761, 85.17616728731022, 85.3910054425666, 85.50558579203667, 85.57719851045546, 85.79203666571182, 86.02119736465195, 86.20739043254082, 86.33629332569464, 86.46519621884846, 86.55113148095101, 86.70867946147236, 86.80893726725867, 86.89487252936121, 87.00945287883127, 87.08106559725006, 87.26725866513893, 87.3102262961902, 87.46777427671154, 87.5823546261816, 87.69693497565167, 87.84016041248925, 87.9977083930106, 87.96906330564308, 88.08364365511314, 88.18390146089945, 88.21254654826697, 88.31280435405327, 88.52764250930964, 88.65654540246346, 88.77112575193354, 88.7854482956173, 88.85706101403609, 88.90002864508737, 88.8857061014036, 88.90002864508737, 89.0575766256087, 89.11486680034373, 89.17215697507876, 89.2008020624463, 89.28673732454884, 89.38699513033515, 89.35835004296763, 89.38699513033515, 89.48725293612145, 89.60183328559151, 89.6448009166428, 89.7450587224291, 89.75938126611285, 89.75938126611285, 89.80234889716414, 89.87396161558291, 89.94557433400172, 90.01718705242051, 90.0887997708393, 90.14608994557433, 90.17473503294184, 90.21770266399312, 90.18905757662561, 90.21770266399312, 90.24634775136064, 90.3609281008307, 90.41821827556574, 90.53279862503581, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 75.88083643655112, 76.21025494127757, 76.72586651389285, 77.12689773703809, 77.47063878544829, 77.94328272701232, 78.15812088226869, 78.57347464909768, 78.94586078487539, 79.33256946433687, 79.70495560011457, 79.99140647378974, 80.26353480378114, 80.42108278430248, 80.70753365797765, 81.0655972500716, 81.32340303637926, 81.5812088226869, 81.79604697794328, 82.0968203953022, 82.3689487252936, 82.62675451160125, 82.84159266685764, 82.91320538527643, 83.05643082211401, 83.27126897737038, 83.471784588943, 83.68662274419937, 83.82984818103695, 83.97307361787453, 84.0876539673446, 84.37410484101976, 84.67487825837868, 84.8037811515325, 84.83242623890003, 84.97565167573761, 85.17616728731022, 85.3910054425666, 85.50558579203667, 85.57719851045546, 85.79203666571182, 86.02119736465195, 86.20739043254082, 86.33629332569464, 86.46519621884846, 86.55113148095101, 86.70867946147236, 86.80893726725867, 86.89487252936121, 87.00945287883127, 87.08106559725006, 87.26725866513893, 87.3102262961902, 87.46777427671154, 87.5823546261816, 87.69693497565167, 87.84016041248925, 87.9977083930106, 87.96906330564308, 88.08364365511314, 88.18390146089945, 88.21254654826697, 88.31280435405327, 88.52764250930964, 88.65654540246346, 88.77112575193354, 88.7854482956173, 88.85706101403609, 88.90002864508737, 88.8857061014036, 88.90002864508737, 89.0575766256087, 89.11486680034373, 89.17215697507876, 89.2008020624463, 89.28673732454884, 89.38699513033515, 89.35835004296763, 89.38699513033515, 89.48725293612145, 89.60183328559151, 89.6448009166428, 89.7450587224291, 89.75938126611285, 89.75938126611285, 89.80234889716414, 89.87396161558291, 89.94557433400172, 90.01718705242051, 90.0887997708393, 90.14608994557433, 90.17473503294184, 90.21770266399312, 90.18905757662561, 90.21770266399312, 90.24634775136064, 90.3609281008307, 90.41821827556574, 90.53279862503581, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.78372959037526, "exact_area100": 85.80170161337239}, "test": {"score": [76.42247255506925, 76.87450670876086, 77.22608882829877, 77.62072181961685, 78.01535481093492, 78.31671091339601, 78.55349070818684, 78.78309535768099, 79.02705029776853, 79.4288584343833, 79.622587357394, 79.8880677333716, 80.11767238286576, 80.56970653655738, 80.82801176723828, 81.10784243380928, 81.35179737389683, 81.57422687809428, 81.90428356174213, 82.16258879242305, 82.37066800602713, 82.5787472196312, 82.75812585204851, 82.98055535624597, 83.15993398866327, 83.35366291167396, 83.43258950993759, 83.65501901413504, 83.84157279184905, 84.11422831312333, 84.25055607376049, 84.57343761211165, 84.76716653512234, 84.9034942957595, 85.01829662050658, 85.27660185118748, 85.42727990241804, 85.56360766305518, 85.79321231254933, 85.92954007318649, 86.12326899619718, 86.22372103035086, 86.45332567984502, 86.61117887637225, 86.71880605582263, 86.8336083805697, 86.88383439764655, 87.02016215828371, 87.08473846595393, 87.17801535481094, 87.27129224366794, 87.38609456841502, 87.45784602138194, 87.54394776494225, 87.65875008968932, 87.78790270502978, 87.8596541579967, 87.91705532037024, 87.945755901557, 88.03185764511731, 88.08925880749085, 88.18253569634786, 88.21123627753462, 88.23276171342471, 88.30451316639162, 88.3260386022817, 88.42649063643539, 88.43366578173209, 88.49106694410563, 88.53411781588576, 88.57716868766593, 88.62739470474277, 88.72784673889646, 88.80677333716008, 88.89287508072039, 88.91440051661046, 88.95745138839062, 89.08660400373107, 89.18705603788477, 89.20858147377486, 89.27315778144508, 89.37360981559877, 89.38796010619215, 89.4310109779723, 89.48123699504916, 89.50993757623593, 89.56733873860946, 89.58886417449953, 89.6103896103896, 89.6175647556863, 89.66779077276314, 89.73954222573008, 89.76106766162016, 89.78976824280693, 89.811293678697, 89.83999425988375, 89.84716940518047, 89.86869484107054, 89.91892085814737, 90.02654803759776, 90.06242376408122], "calls": [0.0, 1.2843510081079141, 2.374973093205137, 3.4584200330056687, 4.51316639161943, 5.209155485398579, 6.192150391045419, 6.98859151897826, 8.028987586998637, 9.227236851546245, 10.160005740116237, 11.164526081653154, 12.061419243739687, 13.274018798880677, 14.163736815670516, 14.99605367008682, 15.964698285140274, 16.71808854129296, 17.744134318719954, 18.748654660256868, 19.875152471837552, 20.894023103967854, 21.848317428427926, 22.738035445217765, 23.692329769677837, 24.797302145368445, 25.65831958097152, 26.490636435387817, 27.373179306880964, 28.334648776637728, 29.159790485757338, 30.25041257085456, 31.276458348281555, 32.19487694625816, 33.02719380067447, 34.117815885771684, 34.95013274018799, 35.818325321087755, 36.91612255148166, 37.906292602425204, 38.961038961038966, 39.90815814020234, 40.79787615699218, 41.85979766090263, 42.727990241802395, 43.639233694482314, 44.442849967711844, 45.526296907512375, 46.738896462653365, 47.88691971012413, 48.71923656454043, 49.66635574370381, 50.6062997775705, 51.610820119107416, 52.586639879457564, 53.68443710985148, 54.775059194948696, 55.84415584415584, 56.963478510439835, 57.75274449307598, 58.785965415799666, 59.81201119322667, 60.83805697065365, 61.885628183970724, 62.85427279902418, 63.74399081581402, 64.89918920858148, 65.94676042189855, 67.01585707110569, 67.62574442132453, 68.68766592523498, 69.92896606156275, 71.12721532611035, 72.1963119753175, 73.23670804333787, 74.22687809428142, 75.05201980340101, 76.15699217909163, 77.44134318719954, 78.61089187056038, 79.6943388103609, 80.82801176723828, 81.76795580110497, 82.72942527086174, 83.79852192006888, 84.72411566334218, 85.85061347492287, 86.66140489344909, 87.73767668795293, 88.6847958671163, 89.72519193513668, 90.60773480662984, 91.7485829088039, 92.58807490851689, 93.57106981416374, 94.76931907871135, 95.58011049723757, 96.74248403530171, 98.12728707756332, 99.06723111143, 100.0], "exact_score": [76.42247255506925, 76.78122981990386, 77.1112865035517, 77.44134318719954, 77.80727559733084, 78.24495946042907, 78.53196527229677, 78.78309535768099, 79.01987515247184, 79.37145727200976, 79.57953648561383, 79.8019659898113, 80.10332209227236, 80.4549042118103, 80.75626031427137, 81.10784243380928, 81.35179737389683, 81.6531534763579, 81.96168472411567, 82.24869053598336, 82.37784315132382, 82.60027265552128, 82.80835186912535, 83.02360622802611, 83.20298486044344, 83.37518834756405, 83.51869125349789, 83.80569706536558, 84.02812656956303, 84.22903063787042, 84.48016072325466, 84.75281624452894, 84.83891798808926, 85.01829662050658, 85.31965272296765, 85.43445504771472, 85.60665853483533, 85.80756260314271, 85.95106550907656, 86.13044414149387, 86.25959675683433, 86.44615053454832, 86.63270431226233, 86.74033149171271, 86.8479586711631, 86.9340604147234, 87.0273373035804, 87.10626390184402, 87.18519050010764, 87.28564253426131, 87.42197029489847, 87.47937145727201, 87.58699863672238, 87.70897610676617, 87.81660328621655, 87.88835473918347, 87.92423046566694, 87.9529310468537, 88.0533830810074, 88.11078424338093, 88.21841142283131, 88.21841142283131, 88.21841142283131, 88.3260386022817, 88.31886345698501, 88.43366578173209, 88.43366578173209, 88.49824208940231, 88.56281839707253, 88.59151897825932, 88.63456985003947, 88.72067159359977, 88.78524790127, 88.90005022601709, 88.89287508072039, 88.93592595250054, 89.05072827724761, 89.16553060199469, 89.19423118318146, 89.24445720025831, 89.30185836263185, 89.35925952500538, 89.38796010619215, 89.45253641386238, 89.48123699504916, 89.52428786682931, 89.57451388390615, 89.58168902920285, 89.6103896103896, 89.66061562746646, 89.69649135394992, 89.73954222573008, 89.77541795221353, 89.8041185334003, 89.811293678697, 89.83999425988375, 89.87586998636723, 89.88304513166392, 89.91892085814737, 90.02654803759776, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 80.26117528879962, "exact_area100": 85.94770754931494}}, "old_normalized_endpoint_budget": 99}, {"seed": 2027, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 1.0, "validation_area100": 0.859014086309921, "training_seconds": 30.928948140935972, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5296270744747568, "operator": "ge"}, {"budget": 0.02, "threshold": 0.4724657859807682, "operator": "ge"}, {"budget": 0.03, "threshold": 0.4370185782685187, "operator": "ge"}, {"budget": 0.04, "threshold": 0.41219101795524726, "operator": "ge"}, {"budget": 0.05, "threshold": 0.38630884598311893, "operator": "ge"}, {"budget": 0.06, "threshold": 0.3667167829508332, "operator": "ge"}, {"budget": 0.07, "threshold": 0.34730069127977253, "operator": "ge"}, {"budget": 0.08, "threshold": 0.3271724249518503, "operator": "ge"}, {"budget": 0.09, "threshold": 0.30945682006561875, "operator": "ge"}, {"budget": 0.1, "threshold": 0.29120992377179117, "operator": "ge"}, {"budget": 0.11, "threshold": 0.2743999684967959, "operator": "ge"}, {"budget": 0.12, "threshold": 0.259749247782707, "operator": "ge"}, {"budget": 0.13, "threshold": 0.23994360005759313, "operator": "ge"}, {"budget": 0.14, "threshold": 0.2216546856273118, "operator": "ge"}, {"budget": 0.15, "threshold": 0.20116175599190456, "operator": "ge"}, {"budget": 0.16, "threshold": 0.1795548227841528, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1581698674974678, "operator": "ge"}, {"budget": 0.18, "threshold": 0.13731044745732005, "operator": "ge"}, {"budget": 0.19, "threshold": 0.11122982613949729, "operator": "ge"}, {"budget": 0.2, "threshold": 0.08168611173160815, "operator": "ge"}, {"budget": 0.21, "threshold": 0.05412927253906807, "operator": "ge"}, {"budget": 0.22, "threshold": 0.020705713813692878, "operator": "ge"}, {"budget": 0.23, "threshold": -0.016706169147554944, "operator": "ge"}, {"budget": 0.24, "threshold": -0.0465291501795537, "operator": "ge"}, {"budget": 0.25, "threshold": -0.08529688284630484, "operator": "ge"}, {"budget": 0.26, "threshold": -0.1169837970073076, "operator": "ge"}, {"budget": 0.27, "threshold": -0.1559375080392602, "operator": "ge"}, {"budget": 0.28, "threshold": -0.193152489254947, "operator": "ge"}, {"budget": 0.29, "threshold": -0.22941432668652448, "operator": "ge"}, {"budget": 0.3, "threshold": -0.26846863532804144, "operator": "ge"}, {"budget": 0.31, "threshold": -0.30898540095805777, "operator": "ge"}, {"budget": 0.32, "threshold": -0.33950352606518625, "operator": "ge"}, {"budget": 0.33, "threshold": -0.38150509530179055, "operator": "ge"}, {"budget": 0.34, "threshold": -0.41865480370532, "operator": "ge"}, {"budget": 0.35, "threshold": -0.4672513717802958, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5070315001804916, "operator": "ge"}, {"budget": 0.37, "threshold": -0.5444295428500103, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5788093447369177, "operator": "ge"}, {"budget": 0.39, "threshold": -0.61578130433741, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6590928377791715, "operator": "ge"}, {"budget": 0.41, "threshold": -0.7032209928676033, "operator": "ge"}, {"budget": 0.42, "threshold": -0.7321588810550906, "operator": "ge"}, {"budget": 0.43, "threshold": -0.7696795369858497, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8071348024805857, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8345099325486238, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8598067310597821, "operator": "ge"}, {"budget": 0.47, "threshold": -0.8810059161463933, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9012451505124852, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9150246134728555, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9280864705228343, "operator": "ge"}, {"budget": 0.51, "threshold": -0.936277746993313, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9434027929717337, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9491605649853228, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9554657781879522, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9598743608492618, "operator": "ge"}, {"budget": 0.56, "threshold": -0.963818209745177, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9686249409194791, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9723703104022061, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9754366006247949, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9779324326657244, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9795676964773301, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9811238661829151, "operator": "ge"}, {"budget": 0.63, "threshold": -0.98268634040875, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9844279472345302, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9856112192140905, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9866978391875098, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9877213760440704, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9885062566252399, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9892269880459365, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9900222538167, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9906735326048142, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9912958241330098, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9918364918997543, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9923613138062044, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9928977535885724, "operator": "ge"}, {"budget": 0.76, "threshold": -0.993390969299774, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9938056287412278, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9942033465693092, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9945729086571191, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9949239598095658, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9952641164503423, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9955663739551173, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9958892133283588, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9962000361640331, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9964897677579886, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9967421385907189, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9969895473090403, "operator": "ge"}, {"budget": 0.88, "threshold": -0.997181857791156, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9974071361792696, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9976180347405642, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9977978478372072, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9980300978489558, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9981947174541486, "operator": "ge"}, {"budget": 0.94, "threshold": -0.998368049318754, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9985405174855929, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9986989921849574, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9989090209845896, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9990876687346774, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9993325796649873, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.09567459180751, 76.4394156402177, 76.7115439702091, 77.09825264967057, 77.44199369808078, 77.78573474649097, 78.17244342595244, 78.57347464909768, 78.96018332855915, 79.20366657118304, 79.50443998854196, 79.74792323116586, 80.0057290174735, 80.23488971641363, 80.46405041535377, 80.86508163849899, 81.12288742480665, 81.33772558006301, 81.59553136637066, 81.88198224004583, 82.12546548266972, 82.38327126897737, 82.59810942423374, 82.84159266685764, 83.01346319106273, 83.24262389000286, 83.44313950157549, 83.72959037525064, 83.9157834431395, 84.04468633629332, 84.2595244915497, 84.46004010312231, 84.63191062732741, 84.8037811515325, 85.07590948152391, 85.10455456889143, 85.3193927241478, 85.5915210541392, 85.74906903466055, 86.02119736465195, 86.20739043254082, 86.3506158693784, 86.50816384989974, 86.68003437410484, 86.82325981094242, 86.93784016041248, 87.00945287883127, 87.09538814093382, 87.12403322830134, 87.26725866513893, 87.42480664566027, 87.53938699513033, 87.68261243196791, 87.75422515038672, 87.84016041248925, 87.9260956745918, 88.08364365511314, 88.15525637353194, 88.31280435405327, 88.4417072472071, 88.5419650529934, 88.58493268404469, 88.6135777714122, 88.59925522772845, 88.69951303351475, 88.84273847035233, 88.87138355771985, 88.91435118877112, 88.95731881982239, 89.02893153824118, 89.08622171297623, 89.25809223718132, 89.31538241191636, 89.4013176740189, 89.38699513033515, 89.47293039243769, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.68776854769406, 89.70209109137782, 89.73073617874535, 89.83099398453164, 89.90260670295044, 89.87396161558291, 89.94557433400172, 90.00286450873675, 90.11744485820681, 90.16041248925809, 90.20338012030936, 90.16041248925809, 90.14608994557433, 90.26067029504439, 90.30363792609567, 90.26067029504439, 90.33228301346318, 90.40389573188197, 90.41821827556574, 90.47550845030077, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 37, 64, 89, 117, 142, 169, 198, 228, 255, 276, 300, 318, 344, 365, 389, 421, 446, 466, 491, 516, 539, 566, 589, 615, 635, 656, 676, 699, 716, 732, 755, 774, 792, 815, 838, 853, 876, 902, 916, 937, 956, 973, 988, 1005, 1023, 1037, 1048, 1059, 1064, 1076, 1089, 1102, 1113, 1121, 1132, 1141, 1155, 1162, 1175, 1186, 1197, 1205, 1212, 1217, 1227, 1238, 1242, 1249, 1254, 1264, 1271, 1283, 1289, 1296, 1300, 1308, 1316, 1322, 1327, 1334, 1337, 1343, 1350, 1356, 1359, 1367, 1373, 1382, 1386, 1395, 1395, 1397, 1408, 1414, 1417, 1425, 1433, 1436, 1443, 1449], "harmed": [0, 0, 3, 9, 10, 11, 14, 16, 18, 18, 22, 25, 26, 34, 39, 47, 51, 58, 63, 70, 75, 81, 90, 98, 107, 115, 120, 126, 129, 133, 140, 148, 153, 159, 170, 174, 187, 195, 202, 205, 207, 213, 220, 224, 229, 237, 243, 249, 254, 257, 259, 261, 266, 267, 270, 275, 278, 281, 283, 285, 287, 291, 296, 301, 307, 310, 311, 313, 317, 319, 324, 327, 327, 329, 330, 335, 337, 340, 343, 346, 348, 350, 354, 354, 355, 360, 363, 365, 366, 367, 373, 376, 379, 382, 385, 391, 394, 397, 399, 402, 406], "exact_score": [75.56574047550845, 76.09567459180751, 76.4394156402177, 76.7115439702091, 77.09825264967057, 77.44199369808078, 77.78573474649097, 78.17244342595244, 78.57347464909768, 78.96018332855915, 79.20366657118304, 79.50443998854196, 79.74792323116586, 80.0057290174735, 80.23488971641363, 80.46405041535377, 80.86508163849899, 81.12288742480665, 81.33772558006301, 81.59553136637066, 81.88198224004583, 82.12546548266972, 82.38327126897737, 82.59810942423374, 82.84159266685764, 83.01346319106273, 83.24262389000286, 83.44313950157549, 83.72959037525064, 83.9157834431395, 84.04468633629332, 84.2595244915497, 84.46004010312231, 84.63191062732741, 84.8037811515325, 85.07590948152391, 85.10455456889143, 85.3193927241478, 85.5915210541392, 85.74906903466055, 86.02119736465195, 86.20739043254082, 86.3506158693784, 86.50816384989974, 86.68003437410484, 86.82325981094242, 86.93784016041248, 87.00945287883127, 87.09538814093382, 87.12403322830134, 87.26725866513893, 87.42480664566027, 87.53938699513033, 87.68261243196791, 87.75422515038672, 87.84016041248925, 87.9260956745918, 88.08364365511314, 88.15525637353194, 88.31280435405327, 88.4417072472071, 88.5419650529934, 88.58493268404469, 88.6135777714122, 88.59925522772845, 88.69951303351475, 88.84273847035233, 88.87138355771985, 88.91435118877112, 88.95731881982239, 89.02893153824118, 89.08622171297623, 89.25809223718132, 89.31538241191636, 89.4013176740189, 89.38699513033515, 89.47293039243769, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.68776854769406, 89.70209109137782, 89.73073617874535, 89.83099398453164, 89.90260670295044, 89.87396161558291, 89.94557433400172, 90.00286450873675, 90.11744485820681, 90.16041248925809, 90.20338012030936, 90.16041248925809, 90.14608994557433, 90.26067029504439, 90.30363792609567, 90.26067029504439, 90.33228301346318, 90.40389573188197, 90.41821827556574, 90.47550845030077, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.90140863099211, "threshold_area100": 85.90038670867946, "threshold_area25": 79.6562589515898}, "test": {"score": [76.42247255506925, 76.87450670876086, 77.32654086245246, 77.663772691397, 77.9364282126713, 78.31671091339601, 78.6037167252637, 78.88354739183468, 79.30688096433953, 79.48625959675684, 79.85936715218483, 80.12484752816245, 80.38315275884337, 80.77061060486476, 81.07914185262251, 81.35179737389683, 81.54552629690751, 81.81100667288513, 82.06931190356606, 82.34914257013705, 82.61462294611465, 82.76530099734521, 83.03078137332281, 83.31061203989381, 83.53304154409128, 83.77699648417881, 83.90614909951927, 84.17880462079357, 84.37253354380427, 84.47298557795796, 84.66671450096864, 84.96089545813304, 85.10439836406688, 85.31965272296765, 85.48468106479157, 85.73581115017579, 85.9151897825931, 86.1017435603071, 86.23089617564756, 86.36004879098802, 86.62552916696563, 86.77620721819616, 86.89100954294324, 86.96276099591017, 87.1277893377341, 87.22106622659109, 87.34304369663487, 87.46502116667862, 87.65157494439264, 87.71615125206286, 87.75202697854631, 87.83812872210662, 87.945755901557, 88.0462079357107, 88.08925880749085, 88.15383511516109, 88.20406113223794, 88.36191432876515, 88.38343976465524, 88.53411781588576, 88.59151897825932, 88.67762072181962, 88.6847958671163, 88.79959819186338, 88.83547391834685, 88.8856999354237, 88.971801678984, 89.01485255076416, 89.05072827724761, 89.07942885843438, 89.10812943962115, 89.17270574729139, 89.24445720025831, 89.24445720025831, 89.30903350792853, 89.38078496089545, 89.47406184975246, 89.50276243093923, 89.55298844801607, 89.56733873860946, 89.55298844801607, 89.63191504627969, 89.66779077276314, 89.75389251632346, 89.78259309751022, 89.84716940518047, 89.86151969577384, 89.8973954222573, 89.91174571285069, 89.87586998636723, 89.85434455047715, 89.93327114874077, 89.94762143933414, 89.97632202052091, 89.97632202052091, 89.98349716581761, 90.03372318289445, 90.05524861878453, 90.07677405467462, 90.06242376408122, 90.06242376408122], "calls": [0.0, 1.3202267345913754, 2.44672454617206, 3.3292674176652075, 4.1974599985649705, 5.173279758915118, 5.94102030566119, 6.866614048934491, 7.914185262251561, 8.73932697137117, 9.78689818468824, 10.71249192796154, 11.551983927674534, 12.556504269211452, 13.496448303078138, 14.529669225801822, 15.591590729712276, 16.53153476357896, 17.363851617995266, 18.296620506565258, 19.423118318145942, 20.320011480232473, 21.231254932912393, 22.40797876156992, 23.29052163306307, 24.481595752313982, 25.292387170840207, 26.58391332424482, 27.55973308459496, 28.578603716725265, 29.733802109492718, 31.00380282700725, 31.77871851905001, 32.94826720241085, 33.87386094568415, 35.007533902561526, 36.047929970581905, 37.030924876228745, 37.899117457128504, 38.87493721747865, 39.92250843079572, 41.01313051589295, 41.83827222501256, 42.89301858362632, 44.048216976393775, 44.85900839492, 46.04290736887422, 46.911099949773984, 48.07347348783813, 49.04929324818828, 49.91748582908804, 50.69957666642749, 51.574944392623955, 52.3642103752601, 53.36155557150032, 54.057544665279465, 54.803759776135465, 56.138336801320236, 57.228958886417445, 58.49178445863529, 59.611107124919286, 60.436248834038885, 61.2613905431585, 62.21568486761857, 63.442634713352945, 64.57630767023032, 65.57365286647054, 66.60687378919422, 67.53964267776422, 68.36478438688384, 69.4195307454976, 70.45992681351797, 71.42139628327475, 72.31111430006457, 73.39456123986511, 74.5067087608524, 75.60450599124631, 76.66642749515677, 77.54179522135323, 78.62524216115376, 79.55801104972376, 80.56970653655736, 81.58140202339096, 82.68637439908159, 83.71959532180526, 84.8891440051661, 85.86496376551625, 86.94841070531677, 87.78072755973308, 88.99332711487408, 89.9906723111143, 90.94496663557437, 92.15756619071536, 92.99705819042836, 93.93700222429504, 94.86977111286504, 95.81689029202842, 97.00078926598263, 97.91920786395924, 98.99547965846308, 100.0], "rescued": [0, 70, 140, 191, 232, 294, 338, 384, 452, 483, 541, 587, 628, 685, 738, 784, 821, 863, 907, 957, 1010, 1046, 1091, 1145, 1189, 1236, 1266, 1323, 1363, 1395, 1440, 1501, 1531, 1577, 1614, 1663, 1700, 1741, 1769, 1803, 1848, 1882, 1912, 1943, 1982, 2003, 2031, 2054, 2087, 2111, 2124, 2142, 2163, 2183, 2203, 2217, 2230, 2257, 2273, 2300, 2314, 2329, 2337, 2356, 2370, 2385, 2403, 2417, 2428, 2438, 2450, 2464, 2482, 2492, 2506, 2522, 2539, 2552, 2565, 2577, 2583, 2602, 2613, 2629, 2638, 2656, 2664, 2676, 2684, 2689, 2695, 2712, 2722, 2732, 2740, 2749, 2759, 2771, 2779, 2784, 2791], "harmed": [0, 7, 14, 18, 21, 30, 34, 41, 50, 56, 62, 71, 76, 79, 89, 97, 107, 112, 120, 131, 147, 162, 170, 185, 198, 211, 223, 242, 255, 273, 291, 311, 321, 337, 351, 365, 377, 392, 402, 418, 426, 439, 453, 474, 490, 498, 509, 515, 522, 537, 545, 551, 557, 563, 577, 582, 588, 593, 606, 612, 618, 621, 628, 631, 640, 648, 654, 662, 668, 674, 682, 687, 695, 705, 710, 716, 720, 729, 735, 745, 753, 761, 767, 771, 776, 785, 791, 798, 804, 814, 823, 829, 837, 843, 851, 859, 862, 871, 876, 883, 890], "exact_score": [76.42247255506925, 76.78122981990386, 77.1184616488484, 77.55614551194661, 77.84315132381431, 78.24495946042907, 78.62524216115376, 78.91942311831815, 79.31405610963623, 79.59388677620723, 79.8952428786683, 80.20377412642607, 80.56970653655738, 80.87106263901845, 81.22981990385306, 81.44507426275382, 81.61010260457775, 81.9186338523355, 82.24151539068666, 82.50699576666428, 82.6791992537849, 82.90880390327904, 83.25321087752027, 83.43258950993759, 83.63349357824497, 83.87027337303581, 84.01377627896966, 84.27925665494726, 84.37970868910097, 84.51603644973811, 84.73129080863887, 84.95372031283634, 85.14027409055034, 85.32682786826433, 85.51338164597834, 85.73581115017579, 85.9151897825931, 86.09456841501041, 86.24524646624094, 86.39592451747149, 86.64705460285572, 86.76903207289948, 86.91253497883332, 86.96993614120686, 87.1206141924374, 87.23541651718448, 87.33586855133817, 87.46502116667862, 87.63004950850255, 87.72332639735954, 87.76637726913971, 87.84530386740332, 87.98880677333716, 88.0533830810074, 88.1394848245677, 88.24711200401808, 88.35473918346847, 88.39061490995192, 88.505417234699, 88.55564325177585, 88.61304441414939, 88.67762072181962, 88.7924230465667, 88.84264906364353, 88.85699935423692, 88.93592595250054, 88.97897682428069, 89.05790342254431, 89.0722537131377, 89.07942885843438, 89.12965487551124, 89.21575661907154, 89.251632345555, 89.28033292674176, 89.33055894381862, 89.4310109779723, 89.47406184975246, 89.51711272153261, 89.56016359331277, 89.55298844801607, 89.6103896103896, 89.65344048216977, 89.70366649924661, 89.76106766162016, 89.77541795221353, 89.85434455047715, 89.86869484107054, 89.904570567554, 89.8973954222573, 89.87586998636723, 89.85434455047715, 89.94044629403746, 89.95479658463084, 89.97632202052091, 89.97632202052091, 89.99067231111431, 90.02654803759776, 90.04807347348785, 90.06959890937792, 90.06242376408122, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.26908965373651, "threshold_area100": 86.23746143359405, "threshold_area25": 80.44844658104327}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.2619883418083191, "operator": "ge"}, {"budget": 0.02, "threshold": 0.2256968915462494, "operator": "ge"}, {"budget": 0.03, "threshold": 0.20567458868026733, "operator": "ge"}, {"budget": 0.04, "threshold": 0.19212891161441803, "operator": "ge"}, {"budget": 0.05, "threshold": 0.18139171600341797, "operator": "ge"}, {"budget": 0.06, "threshold": 0.16776221990585327, "operator": "ge"}, {"budget": 0.07, "threshold": 0.15599431097507477, "operator": "ge"}, {"budget": 0.08, "threshold": 0.14448311924934387, "operator": "ge"}, {"budget": 0.09, "threshold": 0.1349351704120636, "operator": "ge"}, {"budget": 0.1, "threshold": 0.12296487390995026, "operator": "ge"}, {"budget": 0.11, "threshold": 0.11389874666929245, "operator": "ge"}, {"budget": 0.12, "threshold": 0.1037798821926117, "operator": "ge"}, {"budget": 0.13, "threshold": 0.09446622431278229, "operator": "ge"}, {"budget": 0.14, "threshold": 0.08420136570930481, "operator": "ge"}, {"budget": 0.15, "threshold": 0.07322114706039429, "operator": "ge"}, {"budget": 0.16, "threshold": 0.06367747485637665, "operator": "ge"}, {"budget": 0.17, "threshold": 0.05240854620933533, "operator": "ge"}, {"budget": 0.18, "threshold": 0.04325215518474579, "operator": "ge"}, {"budget": 0.19, "threshold": 0.03236702084541321, "operator": "ge"}, {"budget": 0.2, "threshold": 0.023087501525878906, "operator": "ge"}, {"budget": 0.21, "threshold": 0.013620540499687195, "operator": "ge"}, {"budget": 0.22, "threshold": 0.00392497330904007, "operator": "ge"}, {"budget": 0.23, "threshold": -0.0023291674442589283, "operator": "ge"}, {"budget": 0.24, "threshold": -0.004673302173614502, "operator": "ge"}, {"budget": 0.25, "threshold": -0.00616145133972168, "operator": "ge"}, {"budget": 0.26, "threshold": -0.007144510746002197, "operator": "ge"}, {"budget": 0.27, "threshold": -0.008165041916072369, "operator": "ge"}, {"budget": 0.28, "threshold": -0.009046947583556175, "operator": "ge"}, {"budget": 0.29, "threshold": -0.010079195722937584, "operator": "ge"}, {"budget": 0.3, "threshold": -0.010926103219389915, "operator": "ge"}, {"budget": 0.31, "threshold": -0.011894334107637405, "operator": "ge"}, {"budget": 0.32, "threshold": -0.012727484107017517, "operator": "ge"}, {"budget": 0.33, "threshold": -0.013624770566821098, "operator": "ge"}, {"budget": 0.34, "threshold": -0.014523397199809551, "operator": "ge"}, {"budget": 0.35, "threshold": -0.01540815643966198, "operator": "ge"}, {"budget": 0.36, "threshold": -0.01632646843791008, "operator": "ge"}, {"budget": 0.37, "threshold": -0.01726466231048107, "operator": "ge"}, {"budget": 0.38, "threshold": -0.018147386610507965, "operator": "ge"}, {"budget": 0.39, "threshold": -0.018853606656193733, "operator": "ge"}, {"budget": 0.4, "threshold": -0.019868940114974976, "operator": "ge"}, {"budget": 0.41, "threshold": -0.020734835416078568, "operator": "ge"}, {"budget": 0.42, "threshold": -0.021707503125071526, "operator": "ge"}, {"budget": 0.43, "threshold": -0.02263258397579193, "operator": "ge"}, {"budget": 0.44, "threshold": -0.023514356464147568, "operator": "ge"}, {"budget": 0.45, "threshold": -0.024484217166900635, "operator": "ge"}, {"budget": 0.46, "threshold": -0.025681043043732643, "operator": "ge"}, {"budget": 0.47, "threshold": -0.02692312002182007, "operator": "ge"}, {"budget": 0.48, "threshold": -0.027998443692922592, "operator": "ge"}, {"budget": 0.49, "threshold": -0.029299624264240265, "operator": "ge"}, {"budget": 0.5, "threshold": -0.03041158989071846, "operator": "ge"}, {"budget": 0.51, "threshold": -0.032091543078422546, "operator": "ge"}, {"budget": 0.52, "threshold": -0.033269576728343964, "operator": "ge"}, {"budget": 0.53, "threshold": -0.034501686692237854, "operator": "ge"}, {"budget": 0.54, "threshold": -0.035980165004730225, "operator": "ge"}, {"budget": 0.55, "threshold": -0.037351593375205994, "operator": "ge"}, {"budget": 0.56, "threshold": -0.03898220881819725, "operator": "ge"}, {"budget": 0.57, "threshold": -0.040743470191955566, "operator": "ge"}, {"budget": 0.58, "threshold": -0.04237254709005356, "operator": "ge"}, {"budget": 0.59, "threshold": -0.04415878280997276, "operator": "ge"}, {"budget": 0.6, "threshold": -0.045645952224731445, "operator": "ge"}, {"budget": 0.61, "threshold": -0.04710298776626587, "operator": "ge"}, {"budget": 0.62, "threshold": -0.04823341593146324, "operator": "ge"}, {"budget": 0.63, "threshold": -0.0500960499048233, "operator": "ge"}, {"budget": 0.64, "threshold": -0.05168551951646805, "operator": "ge"}, {"budget": 0.65, "threshold": -0.05377189815044403, "operator": "ge"}, {"budget": 0.66, "threshold": -0.055634986609220505, "operator": "ge"}, {"budget": 0.67, "threshold": -0.05732155963778496, "operator": "ge"}, {"budget": 0.68, "threshold": -0.05948815867304802, "operator": "ge"}, {"budget": 0.69, "threshold": -0.06148936599493027, "operator": "ge"}, {"budget": 0.7, "threshold": -0.06390643864870071, "operator": "ge"}, {"budget": 0.71, "threshold": -0.06580998748540878, "operator": "ge"}, {"budget": 0.72, "threshold": -0.06800060719251633, "operator": "ge"}, {"budget": 0.73, "threshold": -0.07002560049295425, "operator": "ge"}, {"budget": 0.74, "threshold": -0.07238217443227768, "operator": "ge"}, {"budget": 0.75, "threshold": -0.07476625591516495, "operator": "ge"}, {"budget": 0.76, "threshold": -0.07732628285884857, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0800284743309021, "operator": "ge"}, {"budget": 0.78, "threshold": -0.08304869383573532, "operator": "ge"}, {"budget": 0.79, "threshold": -0.08561491966247559, "operator": "ge"}, {"budget": 0.8, "threshold": -0.08902665227651596, "operator": "ge"}, {"budget": 0.81, "threshold": -0.09253410249948502, "operator": "ge"}, {"budget": 0.82, "threshold": -0.09585403650999069, "operator": "ge"}, {"budget": 0.83, "threshold": -0.09941156953573227, "operator": "ge"}, {"budget": 0.84, "threshold": -0.10293514281511307, "operator": "ge"}, {"budget": 0.85, "threshold": -0.10739076137542725, "operator": "ge"}, {"budget": 0.86, "threshold": -0.11143749952316284, "operator": "ge"}, {"budget": 0.87, "threshold": -0.11540102958679199, "operator": "ge"}, {"budget": 0.88, "threshold": -0.1216730922460556, "operator": "ge"}, {"budget": 0.89, "threshold": -0.127266526222229, "operator": "ge"}, {"budget": 0.9, "threshold": -0.1343318521976471, "operator": "ge"}, {"budget": 0.91, "threshold": -0.14016446471214294, "operator": "ge"}, {"budget": 0.92, "threshold": -0.148186594247818, "operator": "ge"}, {"budget": 0.93, "threshold": -0.15881256759166718, "operator": "ge"}, {"budget": 0.94, "threshold": -0.16770979762077332, "operator": "ge"}, {"budget": 0.95, "threshold": -0.17965710163116455, "operator": "ge"}, {"budget": 0.96, "threshold": -0.19270825386047363, "operator": "ge"}, {"budget": 0.97, "threshold": -0.21013490855693817, "operator": "ge"}, {"budget": 0.98, "threshold": -0.23748798668384552, "operator": "ge"}, {"budget": 0.99, "threshold": -0.2945282459259033, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.49670581495273, 76.86909195073045, 77.25580063019191, 77.58521913491836, 77.88599255227729, 78.07218562016614, 78.38728158120882, 78.61644228014895, 78.83128043540533, 79.146376396448, 79.53308507590948, 79.91979375537095, 80.22056717272987, 80.47837295903753, 80.77914637639645, 81.05127470638786, 81.25179031796047, 81.53824119163563, 81.91062732741334, 82.12546548266972, 82.38327126897737, 82.59810942423374, 82.72701231738756, 82.78430249212259, 82.89888284159265, 82.97049556001146, 83.05643082211401, 83.17101117158407, 83.21397880263535, 83.27126897737038, 83.34288169578917, 83.4001718705242, 83.48610713262676, 83.58636493841306, 83.61501002578058, 83.71526783156689, 83.8441707247207, 83.94442853050701, 84.01604124892582, 84.11629905471212, 84.17358922944715, 84.23087940418218, 84.40274992838728, 84.56029790890862, 84.74649097679747, 84.88971641363506, 85.06158693784016, 85.10455456889143, 85.1475221999427, 85.21913491836149, 85.36236035519909, 85.44829561730162, 85.63448868519049, 85.76339157834431, 85.87797192781437, 85.97822973360068, 86.13577771412201, 86.32197078201088, 86.39358350042967, 86.46519621884846, 86.49384130621598, 86.53680893726725, 86.65138928673733, 86.79461472357491, 86.88054998567746, 86.92351761672873, 87.00945287883127, 87.12403322830134, 87.3102262961902, 87.52506445144658, 87.6539673446004, 87.78287023775422, 87.95474076195933, 88.19822400458321, 88.47035233457461, 88.67086794614724, 88.77112575193354, 88.91435118877112, 89.02893153824118, 89.14351188771126, 89.2008020624463, 89.31538241191636, 89.42996276138642, 89.47293039243769, 89.6734460040103, 89.78802635348038, 89.87396161558291, 89.97421942136923, 89.97421942136923, 90.01718705242051, 90.10312231452305, 90.30363792609567, 90.3609281008307, 90.4325408192495, 90.4325408192495, 90.48983099398453, 90.57576625608708, 90.6044113434546, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 68, 97, 127, 151, 177, 198, 223, 243, 263, 289, 319, 350, 377, 402, 428, 451, 470, 494, 525, 546, 567, 586, 598, 603, 611, 617, 624, 632, 636, 642, 648, 653, 662, 669, 672, 681, 691, 701, 707, 715, 721, 727, 740, 754, 769, 779, 794, 801, 806, 814, 826, 833, 846, 858, 868, 878, 893, 908, 916, 925, 930, 936, 946, 960, 969, 976, 987, 1001, 1015, 1030, 1039, 1051, 1065, 1084, 1105, 1122, 1135, 1148, 1161, 1175, 1186, 1201, 1213, 1223, 1240, 1258, 1272, 1285, 1294, 1308, 1322, 1340, 1355, 1368, 1381, 1395, 1415, 1433, 1449], "harmed": [0, 2, 3, 6, 9, 10, 15, 23, 26, 30, 35, 39, 42, 46, 52, 59, 64, 68, 73, 77, 82, 88, 91, 95, 98, 99, 99, 100, 101, 101, 102, 104, 105, 106, 109, 109, 110, 112, 113, 116, 117, 118, 120, 122, 123, 126, 128, 128, 131, 135, 137, 140, 142, 143, 143, 146, 148, 151, 155, 157, 160, 164, 167, 170, 172, 176, 179, 183, 188, 194, 195, 195, 195, 198, 200, 202, 204, 207, 213, 216, 221, 227, 234, 241, 245, 252, 255, 265, 273, 279, 288, 299, 307, 311, 322, 330, 343, 353, 367, 383, 406], "exact_score": [75.56574047550845, 75.98109424233743, 76.49670581495273, 76.86909195073045, 77.25580063019191, 77.58521913491836, 77.88599255227729, 78.07218562016614, 78.38728158120882, 78.61644228014895, 78.83128043540533, 79.146376396448, 79.53308507590948, 79.91979375537095, 80.22056717272987, 80.47837295903753, 80.77914637639645, 81.05127470638786, 81.25179031796047, 81.53824119163563, 81.91062732741334, 82.12546548266972, 82.38327126897737, 82.59810942423374, 82.72701231738756, 82.78430249212259, 82.89888284159265, 82.97049556001146, 83.05643082211401, 83.17101117158407, 83.21397880263535, 83.27126897737038, 83.34288169578917, 83.4001718705242, 83.48610713262676, 83.58636493841306, 83.61501002578058, 83.71526783156689, 83.8441707247207, 83.94442853050701, 84.01604124892582, 84.11629905471212, 84.17358922944715, 84.23087940418218, 84.40274992838728, 84.56029790890862, 84.74649097679747, 84.88971641363506, 85.06158693784016, 85.10455456889143, 85.1475221999427, 85.21913491836149, 85.36236035519909, 85.44829561730162, 85.63448868519049, 85.76339157834431, 85.87797192781437, 85.97822973360068, 86.13577771412201, 86.32197078201088, 86.39358350042967, 86.46519621884846, 86.49384130621598, 86.53680893726725, 86.65138928673733, 86.79461472357491, 86.88054998567746, 86.92351761672873, 87.00945287883127, 87.12403322830134, 87.3102262961902, 87.52506445144658, 87.6539673446004, 87.78287023775422, 87.95474076195933, 88.19822400458321, 88.47035233457461, 88.67086794614724, 88.77112575193354, 88.91435118877112, 89.02893153824118, 89.14351188771126, 89.2008020624463, 89.31538241191636, 89.42996276138642, 89.47293039243769, 89.6734460040103, 89.78802635348038, 89.87396161558291, 89.97421942136923, 89.97421942136923, 90.01718705242051, 90.10312231452305, 90.30363792609567, 90.3609281008307, 90.4325408192495, 90.4325408192495, 90.48983099398453, 90.57576625608708, 90.6044113434546, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 84.93995916659615, "threshold_area100": 84.93891435118877, "threshold_area25": 79.59610426811801}, "test": {"score": [76.42247255506925, 76.79558011049724, 77.21891368300209, 77.60637152902346, 77.88620219559446, 78.09428140919854, 78.43868838343977, 78.79744564827438, 79.156202913109, 79.45755901557007, 79.72303939154767, 79.93829375044845, 80.2611752887996, 80.43337877592022, 80.66298342541437, 80.97868981846884, 81.23699504914975, 81.63880318576452, 81.89710841644543, 82.21998995479659, 82.42089402310397, 82.68637439908159, 83.01643108272943, 83.27473631341034, 83.4110640740475, 83.51869125349789, 83.61196814235488, 83.73394561239866, 83.76264619358543, 83.87027337303581, 83.91332424481595, 83.9635502618928, 84.02095142426634, 84.09987802252996, 84.23620578316711, 84.29360694554065, 84.36535839850757, 84.40123412499103, 84.50168615914473, 84.55908732151826, 84.72411566334218, 84.78869197101243, 84.8963191504628, 84.9895960393198, 85.0828729281768, 85.15462438114372, 85.24072612470403, 85.32682786826433, 85.39857932123127, 85.49903135538494, 85.57795795364856, 85.71428571428572, 85.80038745784603, 85.89366434670302, 86.02281696204348, 86.15914472268064, 86.31699791920786, 86.45332567984502, 86.59682858577887, 86.67575518404249, 86.77620721819616, 86.8407835258664, 86.94841070531677, 87.10626390184402, 87.22824137188778, 87.35739398722825, 87.47219631197531, 87.50089689316209, 87.5941737820191, 87.69462581617277, 87.74485183324963, 87.88835473918347, 87.98163162804047, 88.0462079357107, 88.16818540575447, 88.25428714931478, 88.3260386022817, 88.37626461935855, 88.46954150821554, 88.58434383296262, 88.69914615770969, 88.74937217478654, 88.74937217478654, 88.85699935423692, 88.98615196957739, 89.18705603788477, 89.30903350792853, 89.40948554208224, 89.55298844801607, 89.67496591805984, 89.83999425988375, 89.904570567554, 89.96197172992754, 90.04089832819116, 90.1772260888283, 90.25615268709191, 90.29920355887207, 90.3566047212456, 90.33507928535553, 90.32790414005885, 90.06242376408122], "calls": [0.0, 1.1767238286575303, 2.281696204348138, 3.3723182894453614, 4.147233981488125, 4.857573365860659, 5.9840711774413435, 6.952715792494798, 7.842433809284638, 8.70345124488771, 9.614694697567625, 10.368084953720313, 11.264978115806844, 12.097294970223146, 13.108990457056754, 14.170911960967208, 15.067805123053743, 16.251704097007966, 17.076845806127576, 18.181818181818183, 19.12893736098156, 20.17650857429863, 21.281480949989238, 22.350577599196384, 23.591877735524143, 24.75425127358829, 25.586568128004593, 26.612613905431587, 27.36600416158427, 28.327473631341032, 29.181315921647418, 30.329339169118175, 31.297983784171628, 32.40295615986224, 33.32854990313554, 34.30436966348569, 35.179737389682145, 36.22013345770252, 37.066800602712206, 37.669512807634355, 38.609456841501036, 39.628327473631344, 40.5897969433881, 41.43646408839779, 42.22573007103394, 43.14414866901055, 44.06256726698716, 45.20341536916123, 46.136184257731216, 47.1694051804549, 47.93714572720098, 49.04929324818828, 49.93183611968142, 50.850254717658025, 51.79019875152472, 52.80189423835833, 54.00014350290593, 55.06206500681638, 56.14551194661692, 57.16438257874722, 58.211953792064286, 59.0370955011839, 59.64698285140274, 60.75913037239003, 61.691899260960035, 62.91884910669441, 63.952070029418095, 64.79156202913109, 65.982636148382, 67.11630910525939, 68.35043409629044, 69.36212958312406, 70.4455765229246, 71.39987084738466, 72.48331778718519, 73.58111501757911, 74.85829088039033, 75.87716151252063, 77.010834469398, 77.90055248618785, 79.14185262251561, 80.24682499820621, 81.22981990385306, 82.3132668436536, 83.26756116811366, 84.4227595608811, 85.45598048360479, 86.25959675683433, 87.78072755973308, 88.82112362775347, 89.95479658463084, 90.77993829375045, 91.82033436177083, 92.88943101097797, 93.75044844658105, 94.8195450957882, 95.68773767668796, 96.6635574370381, 97.79005524861878, 98.92372820549616, 100.0], "rescued": [0, 64, 127, 193, 239, 274, 328, 384, 439, 489, 536, 570, 620, 655, 700, 750, 795, 862, 906, 956, 994, 1040, 1093, 1138, 1159, 1176, 1190, 1207, 1213, 1231, 1239, 1251, 1260, 1275, 1298, 1310, 1321, 1333, 1347, 1357, 1383, 1395, 1416, 1430, 1444, 1459, 1474, 1491, 1503, 1521, 1533, 1555, 1571, 1590, 1615, 1634, 1661, 1685, 1713, 1731, 1747, 1761, 1777, 1804, 1827, 1850, 1874, 1887, 1909, 1933, 1957, 1983, 2005, 2023, 2049, 2072, 2097, 2118, 2142, 2163, 2191, 2214, 2229, 2255, 2283, 2324, 2353, 2374, 2413, 2443, 2478, 2502, 2527, 2557, 2591, 2625, 2656, 2691, 2724, 2765, 2791], "harmed": [0, 12, 16, 28, 35, 41, 47, 53, 58, 66, 76, 80, 85, 96, 109, 115, 124, 135, 143, 148, 158, 167, 174, 183, 185, 187, 188, 188, 190, 193, 195, 200, 201, 205, 209, 213, 214, 221, 221, 223, 226, 229, 235, 236, 237, 242, 245, 250, 252, 256, 257, 260, 264, 270, 277, 277, 282, 287, 295, 302, 304, 309, 310, 315, 321, 326, 334, 343, 352, 362, 379, 385, 394, 403, 412, 423, 438, 452, 463, 468, 480, 496, 511, 522, 532, 545, 557, 564, 583, 596, 608, 623, 640, 659, 674, 697, 722, 749, 785, 827, 890], "exact_score": [76.42247255506925, 76.7381789481237, 77.1184616488484, 77.49156920427639, 77.83597617851761, 78.130157135682, 78.43868838343977, 78.80462079357106, 79.18490349429577, 79.52931046853699, 79.8019659898113, 80.1678983999426, 80.41185334003013, 80.64863313482098, 80.92128865609529, 81.20829446796299, 81.58140202339098, 81.8397072540719, 82.15541364712635, 82.3921934419172, 82.64332352730143, 82.96620506565257, 83.24603573222358, 83.34648776637728, 83.46846523642104, 83.53304154409128, 83.64784386883835, 83.75547104828873, 83.82004735595896, 83.91332424481595, 83.94919997129942, 83.99942598837626, 84.04965200545311, 84.16445433020019, 84.27208150965058, 84.35100810791418, 84.38688383439765, 84.48733586855134, 84.6236636291885, 84.78151682571571, 84.83891798808926, 84.96089545813304, 85.03982205639664, 85.14027409055034, 85.21920068881394, 85.32682786826433, 85.37705388534118, 85.47750591949487, 85.58513309894525, 85.70711056898902, 85.80038745784603, 85.92236492788979, 86.03716725263688, 86.1878453038674, 86.31699791920786, 86.45332567984502, 86.54660256870201, 86.64705460285572, 86.7546817823061, 86.8407835258664, 87.02016215828371, 87.15648991892087, 87.24976680777786, 87.35739398722825, 87.45784602138194, 87.53677261964555, 87.5941737820191, 87.6731003802827, 87.74485183324963, 87.84530386740332, 87.9529310468537, 88.01033220922724, 88.10360909808423, 88.19688598694124, 88.25428714931478, 88.34038889287508, 88.37626461935855, 88.46236636291886, 88.59151897825932, 88.67044557652292, 88.75654732008323, 88.73502188419316, 88.83547391834685, 88.94310109779724, 89.12965487551124, 89.23728205496163, 89.39513525148884, 89.45253641386238, 89.56733873860946, 89.69649135394992, 89.85434455047715, 89.91892085814737, 89.96197172992754, 90.0839491999713, 90.21310181531177, 90.22745210590514, 90.31355384946545, 90.3637798665423, 90.39248044772907, 90.31355384946545, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 85.42263667680054, "threshold_area100": 85.30691684006601, "threshold_area25": 80.23749730932053}, "selected_budget": 99, "selected": {"val": {"score": 90.6044113434546, "calls": 98.99742194213692, "rescued": 1433, "harmed": 383}, "test": {"score": 90.32790414005885, "calls": 98.92372820549616, "rescued": 2765, "harmed": 827}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.147176504135132, "operator": "ge"}, {"budget": 0.02, "threshold": 1.8203661441802979, "operator": "ge"}, {"budget": 0.03, "threshold": 1.6325929164886475, "operator": "ge"}, {"budget": 0.04, "threshold": 1.474632740020752, "operator": "ge"}, {"budget": 0.05, "threshold": 1.3454395532608032, "operator": "ge"}, {"budget": 0.06, "threshold": 1.231242060661316, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1352529525756836, "operator": "ge"}, {"budget": 0.08, "threshold": 1.069762945175171, "operator": "ge"}, {"budget": 0.09, "threshold": 0.987147331237793, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9066791534423828, "operator": "ge"}, {"budget": 0.11, "threshold": 0.8306971788406372, "operator": "ge"}, {"budget": 0.12, "threshold": 0.7611117959022522, "operator": "ge"}, {"budget": 0.13, "threshold": 0.700110912322998, "operator": "ge"}, {"budget": 0.14, "threshold": 0.6365636587142944, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5672776699066162, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5023202300071716, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4464421272277832, "operator": "ge"}, {"budget": 0.18, "threshold": 0.3726169168949127, "operator": "ge"}, {"budget": 0.19, "threshold": 0.29869526624679565, "operator": "ge"}, {"budget": 0.2, "threshold": 0.21529129147529602, "operator": "ge"}, {"budget": 0.21, "threshold": 0.13628330826759338, "operator": "ge"}, {"budget": 0.22, "threshold": 0.05235111713409424, "operator": "ge"}, {"budget": 0.23, "threshold": -0.04093825817108154, "operator": "ge"}, {"budget": 0.24, "threshold": -0.11735682189464569, "operator": "ge"}, {"budget": 0.25, "threshold": -0.21346604824066162, "operator": "ge"}, {"budget": 0.26, "threshold": -0.30311739444732666, "operator": "ge"}, {"budget": 0.27, "threshold": -0.39423197507858276, "operator": "ge"}, {"budget": 0.28, "threshold": -0.4848424196243286, "operator": "ge"}, {"budget": 0.29, "threshold": -0.576433539390564, "operator": "ge"}, {"budget": 0.3, "threshold": -0.6741632223129272, "operator": "ge"}, {"budget": 0.31, "threshold": -0.7698931694030762, "operator": "ge"}, {"budget": 0.32, "threshold": -0.8480938673019409, "operator": "ge"}, {"budget": 0.33, "threshold": -0.9461542367935181, "operator": "ge"}, {"budget": 0.34, "threshold": -1.053098440170288, "operator": "ge"}, {"budget": 0.35, "threshold": -1.1784043312072754, "operator": "ge"}, {"budget": 0.36, "threshold": -1.3082542419433594, "operator": "ge"}, {"budget": 0.37, "threshold": -1.4018571376800537, "operator": "ge"}, {"budget": 0.38, "threshold": -1.4978324174880981, "operator": "ge"}, {"budget": 0.39, "threshold": -1.6357076168060303, "operator": "ge"}, {"budget": 0.4, "threshold": -1.7728997468948364, "operator": "ge"}, {"budget": 0.41, "threshold": -1.947937250137329, "operator": "ge"}, {"budget": 0.42, "threshold": -2.128434658050537, "operator": "ge"}, {"budget": 0.43, "threshold": -2.2827610969543457, "operator": "ge"}, {"budget": 0.44, "threshold": -2.4765899181365967, "operator": "ge"}, {"budget": 0.45, "threshold": -2.6645290851593018, "operator": "ge"}, {"budget": 0.46, "threshold": -2.849733829498291, "operator": "ge"}, {"budget": 0.47, "threshold": -3.033475160598755, "operator": "ge"}, {"budget": 0.48, "threshold": -3.2252354621887207, "operator": "ge"}, {"budget": 0.49, "threshold": -3.3881664276123047, "operator": "ge"}, {"budget": 0.5, "threshold": -3.5464329719543457, "operator": "ge"}, {"budget": 0.51, "threshold": -3.668625831604004, "operator": "ge"}, {"budget": 0.52, "threshold": -3.8008382320404053, "operator": "ge"}, {"budget": 0.53, "threshold": -3.946143627166748, "operator": "ge"}, {"budget": 0.54, "threshold": -4.054897308349609, "operator": "ge"}, {"budget": 0.55, "threshold": -4.159765243530273, "operator": "ge"}, {"budget": 0.56, "threshold": -4.283224105834961, "operator": "ge"}, {"budget": 0.57, "threshold": -4.386868000030518, "operator": "ge"}, {"budget": 0.58, "threshold": -4.512913227081299, "operator": "ge"}, {"budget": 0.59, "threshold": -4.65394401550293, "operator": "ge"}, {"budget": 0.6, "threshold": -4.744760513305664, "operator": "ge"}, {"budget": 0.61, "threshold": -4.8323655128479, "operator": "ge"}, {"budget": 0.62, "threshold": -4.916426658630371, "operator": "ge"}, {"budget": 0.63, "threshold": -4.995678901672363, "operator": "ge"}, {"budget": 0.64, "threshold": -5.100462913513184, "operator": "ge"}, {"budget": 0.65, "threshold": -5.188089370727539, "operator": "ge"}, {"budget": 0.66, "threshold": -5.266417503356934, "operator": "ge"}, {"budget": 0.67, "threshold": -5.343132495880127, "operator": "ge"}, {"budget": 0.68, "threshold": -5.414971828460693, "operator": "ge"}, {"budget": 0.69, "threshold": -5.487094879150391, "operator": "ge"}, {"budget": 0.7, "threshold": -5.550174713134766, "operator": "ge"}, {"budget": 0.71, "threshold": -5.620702743530273, "operator": "ge"}, {"budget": 0.72, "threshold": -5.698033809661865, "operator": "ge"}, {"budget": 0.73, "threshold": -5.763031482696533, "operator": "ge"}, {"budget": 0.74, "threshold": -5.833500385284424, "operator": "ge"}, {"budget": 0.75, "threshold": -5.900059223175049, "operator": "ge"}, {"budget": 0.76, "threshold": -5.972609043121338, "operator": "ge"}, {"budget": 0.77, "threshold": -6.042103290557861, "operator": "ge"}, {"budget": 0.78, "threshold": -6.105071067810059, "operator": "ge"}, {"budget": 0.79, "threshold": -6.171163082122803, "operator": "ge"}, {"budget": 0.8, "threshold": -6.2440009117126465, "operator": "ge"}, {"budget": 0.81, "threshold": -6.30693244934082, "operator": "ge"}, {"budget": 0.82, "threshold": -6.381460189819336, "operator": "ge"}, {"budget": 0.83, "threshold": -6.452460289001465, "operator": "ge"}, {"budget": 0.84, "threshold": -6.53818416595459, "operator": "ge"}, {"budget": 0.85, "threshold": -6.611065864562988, "operator": "ge"}, {"budget": 0.86, "threshold": -6.689425945281982, "operator": "ge"}, {"budget": 0.87, "threshold": -6.763959884643555, "operator": "ge"}, {"budget": 0.88, "threshold": -6.829411029815674, "operator": "ge"}, {"budget": 0.89, "threshold": -6.897873878479004, "operator": "ge"}, {"budget": 0.9, "threshold": -6.97963285446167, "operator": "ge"}, {"budget": 0.91, "threshold": -7.064486503601074, "operator": "ge"}, {"budget": 0.92, "threshold": -7.186668395996094, "operator": "ge"}, {"budget": 0.93, "threshold": -7.277047634124756, "operator": "ge"}, {"budget": 0.94, "threshold": -7.37044620513916, "operator": "ge"}, {"budget": 0.95, "threshold": -7.466160774230957, "operator": "ge"}, {"budget": 0.96, "threshold": -7.617124557495117, "operator": "ge"}, {"budget": 0.97, "threshold": -7.783701419830322, "operator": "ge"}, {"budget": 0.98, "threshold": -7.963056564331055, "operator": "ge"}, {"budget": 0.99, "threshold": -8.281852722167969, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.93812661128617, 76.36780292179891, 76.63993125179032, 77.08393010598682, 77.3990260670295, 77.72844457175594, 78.0865081638499, 78.38728158120882, 78.67373245488398, 79.04611859066169, 79.4901174448582, 79.71927814379833, 80.0057290174735, 80.33514752219993, 80.56430822114007, 80.80779146376396, 81.0655972500716, 81.38069321111429, 81.59553136637066, 81.85333715267831, 82.12546548266972, 82.39759381266113, 82.6124319679175, 82.84159266685764, 83.02778573474649, 83.27126897737038, 83.4001718705242, 83.72959037525064, 83.9157834431395, 84.0876539673446, 84.24520194786594, 84.48868519048983, 84.5316528215411, 84.86107132626755, 84.93268404468634, 85.16184474362647, 85.3193927241478, 85.5628759667717, 85.7347464909768, 85.9352621025494, 86.23603551990833, 86.33629332569464, 86.52248639358349, 86.63706674305357, 86.82325981094242, 86.92351761672873, 87.05242050988255, 87.03809796619879, 87.12403322830134, 87.29590375250645, 87.42480664566027, 87.55370953881409, 87.63964480091664, 87.78287023775422, 87.85448295617302, 87.94041821827557, 88.08364365511314, 88.1695789172157, 88.31280435405327, 88.31280435405327, 88.52764250930964, 88.55628759667717, 88.55628759667717, 88.58493268404469, 88.72815812088226, 88.81409338298482, 88.87138355771985, 88.94299627613864, 88.97164136350615, 89.0575766256087, 89.1291893440275, 89.28673732454884, 89.31538241191636, 89.35835004296763, 89.41564021770266, 89.53022056717272, 89.51589802348897, 89.55886565454024, 89.65912346032655, 89.65912346032655, 89.65912346032655, 89.7450587224291, 89.8166714408479, 89.85963907189917, 89.93125179031796, 89.98854196505299, 90.00286450873675, 90.0887997708393, 90.11744485820681, 90.0887997708393, 90.14608994557433, 90.17473503294184, 90.20338012030936, 90.27499283872815, 90.28931538241191, 90.3609281008307, 90.38957318819823, 90.41821827556574, 90.47550845030077, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 26, 57, 78, 109, 136, 163, 190, 214, 237, 266, 297, 317, 342, 369, 390, 414, 438, 466, 490, 514, 538, 567, 590, 615, 636, 658, 673, 699, 716, 734, 756, 776, 788, 821, 835, 860, 877, 900, 916, 933, 960, 973, 989, 1004, 1025, 1037, 1052, 1058, 1065, 1078, 1089, 1103, 1112, 1124, 1132, 1145, 1156, 1163, 1175, 1181, 1197, 1205, 1211, 1217, 1227, 1235, 1241, 1249, 1256, 1264, 1271, 1284, 1289, 1294, 1300, 1310, 1315, 1321, 1329, 1332, 1335, 1342, 1349, 1357, 1362, 1368, 1372, 1381, 1385, 1388, 1394, 1400, 1405, 1412, 1419, 1428, 1432, 1436, 1443, 1449], "harmed": [0, 0, 1, 3, 3, 8, 12, 14, 17, 20, 23, 23, 27, 32, 36, 41, 48, 54, 60, 69, 75, 80, 90, 98, 107, 115, 120, 126, 129, 133, 139, 150, 153, 162, 172, 181, 190, 196, 202, 206, 209, 215, 221, 224, 231, 239, 244, 250, 257, 258, 259, 261, 266, 269, 271, 274, 281, 282, 283, 285, 291, 292, 298, 304, 308, 308, 310, 312, 315, 320, 322, 324, 326, 329, 331, 333, 335, 341, 344, 345, 348, 351, 352, 354, 359, 359, 361, 364, 367, 369, 374, 376, 380, 383, 385, 391, 395, 397, 399, 402, 406], "exact_score": [75.56574047550845, 75.93812661128617, 76.36780292179891, 76.63993125179032, 77.08393010598682, 77.3990260670295, 77.72844457175594, 78.0865081638499, 78.38728158120882, 78.67373245488398, 79.04611859066169, 79.4901174448582, 79.71927814379833, 80.0057290174735, 80.33514752219993, 80.56430822114007, 80.80779146376396, 81.0655972500716, 81.38069321111429, 81.59553136637066, 81.85333715267831, 82.12546548266972, 82.39759381266113, 82.6124319679175, 82.84159266685764, 83.02778573474649, 83.27126897737038, 83.4001718705242, 83.72959037525064, 83.9157834431395, 84.0876539673446, 84.24520194786594, 84.48868519048983, 84.5316528215411, 84.86107132626755, 84.93268404468634, 85.16184474362647, 85.3193927241478, 85.5628759667717, 85.7347464909768, 85.9352621025494, 86.23603551990833, 86.33629332569464, 86.52248639358349, 86.63706674305357, 86.82325981094242, 86.92351761672873, 87.05242050988255, 87.03809796619879, 87.12403322830134, 87.29590375250645, 87.42480664566027, 87.55370953881409, 87.63964480091664, 87.78287023775422, 87.85448295617302, 87.94041821827557, 88.08364365511314, 88.1695789172157, 88.31280435405327, 88.31280435405327, 88.52764250930964, 88.55628759667717, 88.55628759667717, 88.58493268404469, 88.72815812088226, 88.81409338298482, 88.87138355771985, 88.94299627613864, 88.97164136350615, 89.0575766256087, 89.1291893440275, 89.28673732454884, 89.31538241191636, 89.35835004296763, 89.41564021770266, 89.53022056717272, 89.51589802348897, 89.55886565454024, 89.65912346032655, 89.65912346032655, 89.65912346032655, 89.7450587224291, 89.8166714408479, 89.85963907189917, 89.93125179031796, 89.98854196505299, 90.00286450873675, 90.0887997708393, 90.11744485820681, 90.0887997708393, 90.14608994557433, 90.17473503294184, 90.20338012030936, 90.27499283872815, 90.28931538241191, 90.3609281008307, 90.38957318819823, 90.41821827556574, 90.47550845030077, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.88623990437087, "threshold_area100": 85.8852048123747, "threshold_area25": 79.63191062732741}, "test": {"score": [76.42247255506925, 76.78840496520056, 77.13281193944178, 77.41264260601277, 77.72117385377054, 78.07993111860515, 78.38846236636292, 78.72569419530745, 79.03422544306522, 79.4288584343833, 79.79479084451461, 80.11049723756906, 80.52665566477721, 80.83518691253498, 81.08631699791921, 81.38767310038028, 81.59575231398436, 81.78948123699506, 82.04061132237928, 82.36349286073043, 82.60027265552128, 82.76530099734521, 83.0523068092129, 83.31778718519051, 83.54021668938796, 83.74829590299204, 83.90614909951927, 84.18597976609027, 84.32948267202411, 84.48733586855134, 84.71694051804549, 84.93937002224295, 85.0900480734735, 85.34117815885772, 85.48468106479157, 85.75016144076918, 85.96541579966994, 86.05151754323025, 86.18067015857072, 86.41744995336155, 86.5681280045921, 86.76185692760279, 86.86948410705317, 86.98428643180026, 87.14213962832747, 87.2138910812944, 87.35021884193155, 87.45067087608524, 87.60852407261247, 87.71615125206286, 87.75920212384301, 87.79507785032648, 87.945755901557, 88.01750735452393, 88.08208366219417, 88.16818540575447, 88.20406113223794, 88.26863743990816, 88.36191432876515, 88.505417234699, 88.58434383296262, 88.67762072181962, 88.74219702948984, 88.80677333716008, 88.82112362775347, 88.87134964483032, 88.97897682428069, 89.02202769606085, 89.01485255076416, 89.05072827724761, 89.09377914902777, 89.158355456698, 89.20858147377486, 89.27315778144508, 89.35925952500538, 89.40231039678554, 89.45253641386238, 89.50276243093923, 89.50993757623593, 89.5386381574227, 89.55298844801607, 89.60321446509292, 89.69649135394992, 89.71801678984, 89.811293678697, 89.84716940518047, 89.8902202769606, 89.88304513166392, 89.86151969577384, 89.86151969577384, 89.87586998636723, 89.8973954222573, 89.92609600344407, 89.93327114874077, 89.98349716581761, 90.00502260170768, 90.01937289230108, 90.04089832819116, 90.06242376408122, 90.06242376408122, 90.06242376408122], "calls": [0.0, 1.0762717945038387, 2.138193298414293, 2.9059338451603645, 3.9104541866972804, 4.950850254717658, 5.962545741551266, 6.945540647198106, 7.648704886273946, 8.768027552557939, 9.937576235918776, 10.985147449235846, 12.097294970223146, 13.051589294683216, 13.912606730286289, 14.874076200043051, 15.785319652722968, 16.502834182392192, 17.435603070962184, 18.382722250125568, 19.45181889933271, 20.420463514386167, 21.25278036880247, 22.393628470976537, 23.312047068953145, 24.40266915405037, 25.45024036736744, 26.63413934132166, 27.624309392265197, 28.722106622659112, 29.877305015426565, 30.98227739111717, 31.922221424983853, 32.912391475927386, 33.90256152687092, 35.007533902561526, 36.24165889359259, 36.9806988591519, 37.769964841788045, 38.77448518332496, 39.83640668723542, 40.93420391762933, 42.10375260099017, 42.84279256654947, 44.09844299347061, 45.00968644615053, 46.07878309535768, 46.95415082155414, 48.06629834254144, 48.970366649924664, 49.91748582908804, 50.72827724761426, 51.62517040970079, 52.622515605941025, 53.37590586209371, 54.1221209729497, 55.069240152113075, 55.815455262969074, 56.94195307454976, 58.47025902274521, 59.331276458348285, 60.29274592810504, 61.16811365430151, 62.15110855994834, 63.313482098012486, 64.53325679845017, 65.47320083231686, 66.48489631915047, 67.48224151539068, 68.36478438688384, 69.32625385664059, 70.4455765229246, 71.44292171916481, 72.3828657530315, 73.52371385520556, 74.72196311975317, 75.76235918777355, 76.731003802827, 77.78575016144077, 78.74721963119752, 79.77326540862452, 80.59123197244745, 81.61010260457773, 82.75095070675181, 83.87027337303581, 84.97524574872641, 86.0156418167468, 86.95558585061347, 87.89552988448015, 88.7924230465667, 89.76106766162016, 90.81581402023392, 92.20061706249551, 93.03293391691182, 93.93700222429504, 94.70474277104111, 95.79536485613833, 96.97208868479586, 97.99813446222286, 99.11745712850684, 100.0], "rescued": [0, 53, 103, 145, 192, 251, 302, 353, 397, 461, 526, 575, 638, 692, 733, 784, 825, 856, 902, 954, 1008, 1046, 1092, 1145, 1190, 1232, 1272, 1326, 1362, 1405, 1449, 1500, 1534, 1580, 1618, 1666, 1713, 1739, 1770, 1809, 1844, 1886, 1921, 1946, 1988, 2006, 2034, 2056, 2088, 2111, 2125, 2139, 2166, 2188, 2204, 2219, 2235, 2249, 2268, 2300, 2312, 2330, 2343, 2357, 2369, 2385, 2403, 2417, 2426, 2439, 2451, 2466, 2478, 2495, 2512, 2527, 2543, 2556, 2567, 2576, 2586, 2598, 2616, 2627, 2644, 2656, 2669, 2675, 2682, 2689, 2700, 2710, 2724, 2730, 2740, 2750, 2760, 2768, 2778, 2783, 2791], "harmed": [0, 2, 4, 7, 11, 20, 28, 32, 33, 42, 56, 61, 66, 77, 83, 92, 104, 108, 119, 126, 147, 162, 168, 184, 198, 211, 229, 244, 260, 281, 293, 313, 326, 337, 355, 366, 383, 397, 410, 416, 430, 445, 465, 474, 494, 502, 511, 519, 529, 537, 545, 554, 560, 572, 579, 582, 593, 598, 604, 616, 617, 622, 626, 631, 641, 650, 653, 661, 671, 679, 685, 691, 696, 704, 709, 718, 727, 733, 743, 748, 756, 761, 766, 774, 778, 785, 792, 799, 809, 816, 825, 832, 842, 847, 850, 857, 865, 870, 877, 882, 890], "exact_score": [76.42247255506925, 76.74535409342039, 77.08258592236493, 77.41981775130947, 77.7498744349573, 78.09428140919854, 78.41716294754968, 78.75439477649422, 79.17772834899907, 79.50778503264692, 79.82349142570138, 80.11767238286576, 80.49077993829376, 80.8208366219416, 81.12936786969937, 81.40919853627037, 81.63880318576452, 81.90428356174213, 82.21281480949989, 82.49264547607089, 82.6648489631915, 82.93032933916912, 83.24603573222358, 83.44693980053096, 83.64784386883835, 83.85592308244242, 84.01377627896966, 84.25055607376049, 84.40840927028772, 84.55191217622158, 84.73846595393557, 84.94654516753964, 85.09722321877018, 85.34835330415441, 85.49185621008826, 85.75016144076918, 85.94389036377987, 86.04434239793356, 86.23089617564756, 86.46050082514172, 86.63270431226233, 86.76903207289948, 86.88383439764655, 86.99863672239363, 87.14213962832747, 87.2138910812944, 87.34304369663487, 87.46502116667862, 87.58699863672238, 87.70897610676617, 87.74485183324963, 87.84530386740332, 87.97445648274378, 88.03903279041401, 88.132309679271, 88.21841142283131, 88.26863743990816, 88.36908947406185, 88.47671665351223, 88.56281839707253, 88.63456985003947, 88.72067159359977, 88.77807275597331, 88.81394848245677, 88.82829877305016, 88.92875080720385, 88.97897682428069, 89.02202769606085, 89.03637798665423, 89.0722537131377, 89.12247973021454, 89.19423118318146, 89.24445720025831, 89.33055894381862, 89.40948554208224, 89.4238358326756, 89.48123699504916, 89.50276243093923, 89.52428786682931, 89.54581330271938, 89.56016359331277, 89.624739900983, 89.71801678984, 89.74671737102676, 89.8184688239937, 89.85434455047715, 89.8902202769606, 89.88304513166392, 89.84716940518047, 89.86869484107054, 89.8902202769606, 89.91174571285069, 89.94762143933414, 89.93327114874077, 89.97632202052091, 90.00502260170768, 90.01219774700438, 90.04089832819116, 90.06242376408122, 90.06242376408122, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.24971748219338, "threshold_area100": 86.21729927531034, "threshold_area25": 80.3896103896104}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.179253339767456, "operator": "ge"}, {"budget": 0.02, "threshold": 1.026479959487915, "operator": "ge"}, {"budget": 0.03, "threshold": 0.9370793104171753, "operator": "ge"}, {"budget": 0.04, "threshold": 0.8764957189559937, "operator": "ge"}, {"budget": 0.05, "threshold": 0.8149082660675049, "operator": "ge"}, {"budget": 0.06, "threshold": 0.7692488431930542, "operator": "ge"}, {"budget": 0.07, "threshold": 0.7247418165206909, "operator": "ge"}, {"budget": 0.08, "threshold": 0.6793168783187866, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6398891806602478, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5997755527496338, "operator": "ge"}, {"budget": 0.11, "threshold": 0.5632318258285522, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5316790342330933, "operator": "ge"}, {"budget": 0.13, "threshold": 0.4894285798072815, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4507911801338196, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4078860282897949, "operator": "ge"}, {"budget": 0.16, "threshold": 0.36304527521133423, "operator": "ge"}, {"budget": 0.17, "threshold": 0.31901815533638, "operator": "ge"}, {"budget": 0.18, "threshold": 0.27636662125587463, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2233840823173523, "operator": "ge"}, {"budget": 0.2, "threshold": 0.16373711824417114, "operator": "ge"}, {"budget": 0.21, "threshold": 0.10836434364318848, "operator": "ge"}, {"budget": 0.22, "threshold": 0.04141730070114136, "operator": "ge"}, {"budget": 0.23, "threshold": -0.033415377140045166, "operator": "ge"}, {"budget": 0.24, "threshold": -0.09312553703784943, "operator": "ge"}, {"budget": 0.25, "threshold": -0.17100930213928223, "operator": "ge"}, {"budget": 0.26, "threshold": -0.2350437492132187, "operator": "ge"}, {"budget": 0.27, "threshold": -0.31444042921066284, "operator": "ge"}, {"budget": 0.28, "threshold": -0.3912196159362793, "operator": "ge"}, {"budget": 0.29, "threshold": -0.46714240312576294, "operator": "ge"}, {"budget": 0.3, "threshold": -0.55042564868927, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6388466358184814, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7070624828338623, "operator": "ge"}, {"budget": 0.33, "threshold": -0.8036398887634277, "operator": "ge"}, {"budget": 0.34, "threshold": -0.892119824886322, "operator": "ge"}, {"budget": 0.35, "threshold": -1.0130966901779175, "operator": "ge"}, {"budget": 0.36, "threshold": -1.1174519062042236, "operator": "ge"}, {"budget": 0.37, "threshold": -1.2208595275878906, "operator": "ge"}, {"budget": 0.38, "threshold": -1.3213410377502441, "operator": "ge"}, {"budget": 0.39, "threshold": -1.4363620281219482, "operator": "ge"}, {"budget": 0.4, "threshold": -1.5824166536331177, "operator": "ge"}, {"budget": 0.41, "threshold": -1.7472890615463257, "operator": "ge"}, {"budget": 0.42, "threshold": -1.8667306900024414, "operator": "ge"}, {"budget": 0.43, "threshold": -2.039083480834961, "operator": "ge"}, {"budget": 0.44, "threshold": -2.237506866455078, "operator": "ge"}, {"budget": 0.45, "threshold": -2.405622959136963, "operator": "ge"}, {"budget": 0.46, "threshold": -2.5852062702178955, "operator": "ge"}, {"budget": 0.47, "threshold": -2.760488510131836, "operator": "ge"}, {"budget": 0.48, "threshold": -2.9576239585876465, "operator": "ge"}, {"budget": 0.49, "threshold": -3.1151256561279297, "operator": "ge"}, {"budget": 0.5, "threshold": -3.2888240814208984, "operator": "ge"}, {"budget": 0.51, "threshold": -3.413990020751953, "operator": "ge"}, {"budget": 0.52, "threshold": -3.5362555980682373, "operator": "ge"}, {"budget": 0.53, "threshold": -3.646482467651367, "operator": "ge"}, {"budget": 0.54, "threshold": -3.7821264266967773, "operator": "ge"}, {"budget": 0.55, "threshold": -3.888624668121338, "operator": "ge"}, {"budget": 0.56, "threshold": -3.99411678314209, "operator": "ge"}, {"budget": 0.57, "threshold": -4.139089107513428, "operator": "ge"}, {"budget": 0.58, "threshold": -4.2681427001953125, "operator": "ge"}, {"budget": 0.59, "threshold": -4.387368202209473, "operator": "ge"}, {"budget": 0.6, "threshold": -4.495703220367432, "operator": "ge"}, {"budget": 0.61, "threshold": -4.573524475097656, "operator": "ge"}, {"budget": 0.62, "threshold": -4.653537750244141, "operator": "ge"}, {"budget": 0.63, "threshold": -4.740723609924316, "operator": "ge"}, {"budget": 0.64, "threshold": -4.847683906555176, "operator": "ge"}, {"budget": 0.65, "threshold": -4.927243232727051, "operator": "ge"}, {"budget": 0.66, "threshold": -5.006317615509033, "operator": "ge"}, {"budget": 0.67, "threshold": -5.086899757385254, "operator": "ge"}, {"budget": 0.68, "threshold": -5.153372764587402, "operator": "ge"}, {"budget": 0.69, "threshold": -5.218508243560791, "operator": "ge"}, {"budget": 0.7, "threshold": -5.295578479766846, "operator": "ge"}, {"budget": 0.71, "threshold": -5.36342716217041, "operator": "ge"}, {"budget": 0.72, "threshold": -5.432769298553467, "operator": "ge"}, {"budget": 0.73, "threshold": -5.497146129608154, "operator": "ge"}, {"budget": 0.74, "threshold": -5.563870429992676, "operator": "ge"}, {"budget": 0.75, "threshold": -5.6369428634643555, "operator": "ge"}, {"budget": 0.76, "threshold": -5.709224700927734, "operator": "ge"}, {"budget": 0.77, "threshold": -5.774171352386475, "operator": "ge"}, {"budget": 0.78, "threshold": -5.840801239013672, "operator": "ge"}, {"budget": 0.79, "threshold": -5.906811237335205, "operator": "ge"}, {"budget": 0.8, "threshold": -5.97393274307251, "operator": "ge"}, {"budget": 0.81, "threshold": -6.043397426605225, "operator": "ge"}, {"budget": 0.82, "threshold": -6.109482765197754, "operator": "ge"}, {"budget": 0.83, "threshold": -6.185263633728027, "operator": "ge"}, {"budget": 0.84, "threshold": -6.264157295227051, "operator": "ge"}, {"budget": 0.85, "threshold": -6.34352970123291, "operator": "ge"}, {"budget": 0.86, "threshold": -6.418215751647949, "operator": "ge"}, {"budget": 0.87, "threshold": -6.497392177581787, "operator": "ge"}, {"budget": 0.88, "threshold": -6.563438415527344, "operator": "ge"}, {"budget": 0.89, "threshold": -6.646900653839111, "operator": "ge"}, {"budget": 0.9, "threshold": -6.731915473937988, "operator": "ge"}, {"budget": 0.91, "threshold": -6.810434341430664, "operator": "ge"}, {"budget": 0.92, "threshold": -6.922167778015137, "operator": "ge"}, {"budget": 0.93, "threshold": -7.009565353393555, "operator": "ge"}, {"budget": 0.94, "threshold": -7.1103715896606445, "operator": "ge"}, {"budget": 0.95, "threshold": -7.222413063049316, "operator": "ge"}, {"budget": 0.96, "threshold": -7.337399482727051, "operator": "ge"}, {"budget": 0.97, "threshold": -7.513645648956299, "operator": "ge"}, {"budget": 0.98, "threshold": -7.6926140785217285, "operator": "ge"}, {"budget": 0.99, "threshold": -8.004934310913086, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.09567459180751, 76.4394156402177, 76.7115439702091, 77.09825264967057, 77.44199369808078, 77.78573474649097, 78.17244342595244, 78.57347464909768, 78.96018332855915, 79.20366657118304, 79.50443998854196, 79.74792323116586, 80.0057290174735, 80.23488971641363, 80.46405041535377, 80.86508163849899, 81.12288742480665, 81.33772558006301, 81.59553136637066, 81.88198224004583, 82.12546548266972, 82.38327126897737, 82.59810942423374, 82.84159266685764, 83.01346319106273, 83.24262389000286, 83.44313950157549, 83.72959037525064, 83.9157834431395, 84.04468633629332, 84.2595244915497, 84.46004010312231, 84.63191062732741, 84.8037811515325, 85.07590948152391, 85.10455456889143, 85.3193927241478, 85.5915210541392, 85.74906903466055, 86.02119736465195, 86.20739043254082, 86.3506158693784, 86.50816384989974, 86.68003437410484, 86.82325981094242, 86.93784016041248, 87.00945287883127, 87.09538814093382, 87.12403322830134, 87.26725866513893, 87.42480664566027, 87.53938699513033, 87.68261243196791, 87.75422515038672, 87.84016041248925, 87.9260956745918, 88.08364365511314, 88.15525637353194, 88.31280435405327, 88.4417072472071, 88.5419650529934, 88.58493268404469, 88.6135777714122, 88.59925522772845, 88.69951303351475, 88.84273847035233, 88.87138355771985, 88.91435118877112, 88.95731881982239, 89.02893153824118, 89.08622171297623, 89.25809223718132, 89.31538241191636, 89.4013176740189, 89.38699513033515, 89.47293039243769, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.68776854769406, 89.70209109137782, 89.73073617874535, 89.83099398453164, 89.90260670295044, 89.87396161558291, 89.94557433400172, 90.00286450873675, 90.11744485820681, 90.16041248925809, 90.20338012030936, 90.16041248925809, 90.14608994557433, 90.26067029504439, 90.30363792609567, 90.26067029504439, 90.33228301346318, 90.40389573188197, 90.41821827556574, 90.47550845030077, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 37, 64, 89, 117, 142, 169, 198, 228, 255, 276, 300, 318, 344, 365, 389, 421, 446, 466, 491, 516, 539, 566, 589, 615, 635, 656, 676, 699, 716, 732, 755, 774, 792, 815, 838, 853, 876, 902, 916, 937, 956, 973, 988, 1005, 1023, 1037, 1048, 1059, 1064, 1076, 1089, 1102, 1113, 1121, 1132, 1141, 1155, 1162, 1175, 1186, 1197, 1205, 1212, 1217, 1227, 1238, 1242, 1249, 1254, 1264, 1271, 1283, 1289, 1296, 1300, 1308, 1316, 1322, 1327, 1334, 1337, 1343, 1350, 1356, 1359, 1367, 1373, 1382, 1386, 1395, 1395, 1397, 1408, 1414, 1417, 1425, 1433, 1436, 1443, 1449], "harmed": [0, 0, 3, 9, 10, 11, 14, 16, 18, 18, 22, 25, 26, 34, 39, 47, 51, 58, 63, 70, 75, 81, 90, 98, 107, 115, 120, 126, 129, 133, 140, 148, 153, 159, 170, 174, 187, 195, 202, 205, 207, 213, 220, 224, 229, 237, 243, 249, 254, 257, 259, 261, 266, 267, 270, 275, 278, 281, 283, 285, 287, 291, 296, 301, 307, 310, 311, 313, 317, 319, 324, 327, 327, 329, 330, 335, 337, 340, 343, 346, 348, 350, 354, 354, 355, 360, 363, 365, 366, 367, 373, 376, 379, 382, 385, 391, 394, 397, 399, 402, 406], "exact_score": [75.56574047550845, 76.09567459180751, 76.4394156402177, 76.7115439702091, 77.09825264967057, 77.44199369808078, 77.78573474649097, 78.17244342595244, 78.57347464909768, 78.96018332855915, 79.20366657118304, 79.50443998854196, 79.74792323116586, 80.0057290174735, 80.23488971641363, 80.46405041535377, 80.86508163849899, 81.12288742480665, 81.33772558006301, 81.59553136637066, 81.88198224004583, 82.12546548266972, 82.38327126897737, 82.59810942423374, 82.84159266685764, 83.01346319106273, 83.24262389000286, 83.44313950157549, 83.72959037525064, 83.9157834431395, 84.04468633629332, 84.2595244915497, 84.46004010312231, 84.63191062732741, 84.8037811515325, 85.07590948152391, 85.10455456889143, 85.3193927241478, 85.5915210541392, 85.74906903466055, 86.02119736465195, 86.20739043254082, 86.3506158693784, 86.50816384989974, 86.68003437410484, 86.82325981094242, 86.93784016041248, 87.00945287883127, 87.09538814093382, 87.12403322830134, 87.26725866513893, 87.42480664566027, 87.53938699513033, 87.68261243196791, 87.75422515038672, 87.84016041248925, 87.9260956745918, 88.08364365511314, 88.15525637353194, 88.31280435405327, 88.4417072472071, 88.5419650529934, 88.58493268404469, 88.6135777714122, 88.59925522772845, 88.69951303351475, 88.84273847035233, 88.87138355771985, 88.91435118877112, 88.95731881982239, 89.02893153824118, 89.08622171297623, 89.25809223718132, 89.31538241191636, 89.4013176740189, 89.38699513033515, 89.47293039243769, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.68776854769406, 89.70209109137782, 89.73073617874535, 89.83099398453164, 89.90260670295044, 89.87396161558291, 89.94557433400172, 90.00286450873675, 90.11744485820681, 90.16041248925809, 90.20338012030936, 90.16041248925809, 90.14608994557433, 90.26067029504439, 90.30363792609567, 90.26067029504439, 90.33228301346318, 90.40389573188197, 90.41821827556574, 90.47550845030077, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.90140863099211, "threshold_area100": 85.90038670867946, "threshold_area25": 79.6562589515898}, "test": {"score": [76.42247255506925, 76.87450670876086, 77.32654086245246, 77.663772691397, 77.9364282126713, 78.31671091339601, 78.6037167252637, 78.88354739183468, 79.30688096433953, 79.48625959675684, 79.85936715218483, 80.12484752816245, 80.38315275884337, 80.77061060486476, 81.07914185262251, 81.35179737389683, 81.54552629690751, 81.81100667288513, 82.06931190356606, 82.34914257013705, 82.61462294611465, 82.76530099734521, 83.03078137332281, 83.31061203989381, 83.53304154409128, 83.77699648417881, 83.90614909951927, 84.17880462079357, 84.37253354380427, 84.47298557795796, 84.66671450096864, 84.96089545813304, 85.10439836406688, 85.31965272296765, 85.48468106479157, 85.73581115017579, 85.9151897825931, 86.1017435603071, 86.23089617564756, 86.36004879098802, 86.62552916696563, 86.77620721819616, 86.89100954294324, 86.96276099591017, 87.1277893377341, 87.22106622659109, 87.34304369663487, 87.46502116667862, 87.65157494439264, 87.71615125206286, 87.75202697854631, 87.83812872210662, 87.945755901557, 88.0462079357107, 88.08925880749085, 88.15383511516109, 88.20406113223794, 88.36191432876515, 88.38343976465524, 88.53411781588576, 88.59151897825932, 88.67762072181962, 88.6847958671163, 88.79959819186338, 88.83547391834685, 88.8856999354237, 88.971801678984, 89.01485255076416, 89.05072827724761, 89.07942885843438, 89.10812943962115, 89.17270574729139, 89.24445720025831, 89.24445720025831, 89.30903350792853, 89.38078496089545, 89.47406184975246, 89.50276243093923, 89.55298844801607, 89.56733873860946, 89.55298844801607, 89.63191504627969, 89.66779077276314, 89.75389251632346, 89.78259309751022, 89.84716940518047, 89.86151969577384, 89.8973954222573, 89.91174571285069, 89.87586998636723, 89.85434455047715, 89.93327114874077, 89.94762143933414, 89.97632202052091, 89.97632202052091, 89.98349716581761, 90.03372318289445, 90.05524861878453, 90.07677405467462, 90.06242376408122, 90.06242376408122], "calls": [0.0, 1.3202267345913754, 2.44672454617206, 3.3292674176652075, 4.1974599985649705, 5.173279758915118, 5.94102030566119, 6.866614048934491, 7.914185262251561, 8.73932697137117, 9.78689818468824, 10.71249192796154, 11.551983927674534, 12.556504269211452, 13.496448303078138, 14.529669225801822, 15.591590729712276, 16.53153476357896, 17.363851617995266, 18.296620506565258, 19.423118318145942, 20.320011480232473, 21.231254932912393, 22.40797876156992, 23.29052163306307, 24.481595752313982, 25.292387170840207, 26.58391332424482, 27.55973308459496, 28.578603716725265, 29.733802109492718, 31.00380282700725, 31.77871851905001, 32.94826720241085, 33.87386094568415, 35.007533902561526, 36.047929970581905, 37.030924876228745, 37.899117457128504, 38.87493721747865, 39.92250843079572, 41.01313051589295, 41.83827222501256, 42.89301858362632, 44.048216976393775, 44.85900839492, 46.04290736887422, 46.911099949773984, 48.07347348783813, 49.04929324818828, 49.91748582908804, 50.69957666642749, 51.574944392623955, 52.3642103752601, 53.36155557150032, 54.057544665279465, 54.803759776135465, 56.138336801320236, 57.228958886417445, 58.49178445863529, 59.611107124919286, 60.436248834038885, 61.2613905431585, 62.21568486761857, 63.442634713352945, 64.57630767023032, 65.57365286647054, 66.60687378919422, 67.53964267776422, 68.36478438688384, 69.4195307454976, 70.45992681351797, 71.42139628327475, 72.31111430006457, 73.39456123986511, 74.5067087608524, 75.60450599124631, 76.66642749515677, 77.54179522135323, 78.62524216115376, 79.55801104972376, 80.56970653655736, 81.58140202339096, 82.68637439908159, 83.72677046710196, 84.8891440051661, 85.86496376551625, 86.94841070531677, 87.78072755973308, 88.99332711487408, 89.9906723111143, 90.94496663557437, 92.15756619071536, 92.99705819042836, 93.93700222429504, 94.86977111286504, 95.81689029202842, 97.00078926598263, 97.92638300925593, 98.99547965846308, 100.0], "rescued": [0, 70, 140, 191, 232, 294, 338, 384, 452, 483, 541, 587, 628, 685, 738, 784, 821, 863, 907, 957, 1010, 1046, 1091, 1145, 1189, 1236, 1266, 1323, 1363, 1395, 1440, 1501, 1531, 1577, 1614, 1663, 1700, 1741, 1769, 1803, 1848, 1882, 1912, 1943, 1982, 2003, 2031, 2054, 2087, 2111, 2124, 2142, 2163, 2183, 2203, 2217, 2230, 2257, 2273, 2300, 2314, 2329, 2337, 2356, 2370, 2385, 2403, 2417, 2428, 2438, 2450, 2464, 2482, 2492, 2506, 2522, 2539, 2552, 2565, 2577, 2583, 2602, 2613, 2629, 2638, 2656, 2664, 2676, 2684, 2689, 2695, 2712, 2722, 2732, 2740, 2749, 2759, 2771, 2779, 2784, 2791], "harmed": [0, 7, 14, 18, 21, 30, 34, 41, 50, 56, 62, 71, 76, 79, 89, 97, 107, 112, 120, 131, 147, 162, 170, 185, 198, 211, 223, 242, 255, 273, 291, 311, 321, 337, 351, 365, 377, 392, 402, 418, 426, 439, 453, 474, 490, 498, 509, 515, 522, 537, 545, 551, 557, 563, 577, 582, 588, 593, 606, 612, 618, 621, 628, 631, 640, 648, 654, 662, 668, 674, 682, 687, 695, 705, 710, 716, 720, 729, 735, 745, 753, 761, 767, 771, 776, 785, 791, 798, 804, 814, 823, 829, 837, 843, 851, 859, 862, 871, 876, 883, 890], "exact_score": [76.42247255506925, 76.78122981990386, 77.1184616488484, 77.55614551194661, 77.84315132381431, 78.24495946042907, 78.62524216115376, 78.91942311831815, 79.31405610963623, 79.59388677620723, 79.8952428786683, 80.20377412642607, 80.56970653655738, 80.87106263901845, 81.22981990385306, 81.44507426275382, 81.61010260457775, 81.9186338523355, 82.24151539068666, 82.50699576666428, 82.6791992537849, 82.90880390327904, 83.25321087752027, 83.43258950993759, 83.63349357824497, 83.87027337303581, 84.01377627896966, 84.27925665494726, 84.37970868910097, 84.51603644973811, 84.73129080863887, 84.95372031283634, 85.14027409055034, 85.32682786826433, 85.51338164597834, 85.73581115017579, 85.9151897825931, 86.09456841501041, 86.24524646624094, 86.39592451747149, 86.64705460285572, 86.76903207289948, 86.91253497883332, 86.96993614120686, 87.1206141924374, 87.23541651718448, 87.33586855133817, 87.46502116667862, 87.63004950850255, 87.72332639735954, 87.76637726913971, 87.84530386740332, 87.98880677333716, 88.0533830810074, 88.1394848245677, 88.24711200401808, 88.35473918346847, 88.39061490995192, 88.505417234699, 88.55564325177585, 88.61304441414939, 88.67762072181962, 88.7924230465667, 88.84264906364353, 88.85699935423692, 88.93592595250054, 88.97897682428069, 89.05790342254431, 89.0722537131377, 89.07942885843438, 89.12965487551124, 89.21575661907154, 89.251632345555, 89.28033292674176, 89.33055894381862, 89.4310109779723, 89.47406184975246, 89.51711272153261, 89.56016359331277, 89.55298844801607, 89.6103896103896, 89.65344048216977, 89.70366649924661, 89.76106766162016, 89.77541795221353, 89.85434455047715, 89.86869484107054, 89.904570567554, 89.8973954222573, 89.87586998636723, 89.85434455047715, 89.94044629403746, 89.95479658463084, 89.97632202052091, 89.97632202052091, 89.99067231111431, 90.02654803759776, 90.04807347348785, 90.06959890937792, 90.06242376408122, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.26908965373651, "threshold_area100": 86.23746143359405, "threshold_area25": 80.44844658104327}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.67602406187338, "calls": 98.99742194213692}, "test": {"score": 90.34942957594892, "calls": 98.90937791490278}}, "old_normalized_interior": {"val": {"score": 90.46118590661702, "calls": 97.99484388427385}, "test": {"score": 89.97632202052091, "calls": 97.8618067015857}}, "old_normalized_endpoints": {"val": {"score": 90.50415353766829, "calls": 100.0}, "test": {"score": 90.06242376408122, "calls": 100.0}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 76.02406187338872, 76.38212546548266, 76.69722142652535, 77.11257519335433, 77.31309080492696, 77.68547694070466, 77.98625035806359, 78.30134631910627, 78.71670008593526, 79.07476367802921, 79.41850472643941, 79.74792323116586, 80.06301919220853, 80.39243769693498, 80.60727585219135, 80.95101690060154, 81.19450014322544, 81.52391864795187, 81.83901460899456, 81.95359495846462, 82.0968203953022, 82.29733600687481, 82.46920653107992, 82.66972214265253, 82.81294757949011, 82.94185047264394, 83.11372099684904, 83.28559152105413, 83.60068748209682, 83.8441707247207, 83.97307361787453, 84.11629905471212, 84.23087940418218, 84.37410484101976, 84.57462045259237, 84.76081352048124, 84.86107132626755, 85.17616728731022, 85.27642509309653, 85.4912632483529, 85.70610140360928, 85.84932684044686, 85.94958464623316, 86.12145517043827, 86.25035806359209, 86.50816384989974, 86.47951876253222, 86.6227441993698, 86.82325981094242, 86.95216270409625, 87.1383557719851, 87.19564594672013, 87.28158120882267, 87.35319392724148, 87.46777427671154, 87.55370953881409, 87.59667716986537, 87.7255800630192, 87.81151532512173, 87.82583786880551, 87.88312804354052, 88.01203093669436, 88.12661128616442, 88.2411916356345, 88.32712689773703, 88.39873961615584, 88.49899742194214, 88.58493268404469, 88.67086794614724, 88.84273847035233, 88.91435118877112, 89.01460899455743, 89.02893153824118, 89.15783443139502, 89.22944714981381, 89.28673732454884, 89.34402749928387, 89.45860784875393, 89.50157547980521, 89.5445431108565, 89.6734460040103, 89.75938126611285, 89.85963907189917, 89.90260670295044, 90.04583213978803, 90.00286450873675, 90.06015468347178, 90.01718705242051, 90.00286450873675, 90.00286450873675, 90.10312231452305, 90.16041248925809, 90.21770266399312, 90.24634775136064, 90.21770266399312, 90.28931538241191, 90.38957318819823, 90.46118590661702, 90.44686336293324, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 76.02406187338872, 76.38212546548266, 76.69722142652535, 77.11257519335433, 77.31309080492696, 77.68547694070466, 77.98625035806359, 78.30134631910627, 78.71670008593526, 79.07476367802921, 79.41850472643941, 79.74792323116586, 80.06301919220853, 80.39243769693498, 80.60727585219135, 80.95101690060154, 81.19450014322544, 81.52391864795187, 81.83901460899456, 81.95359495846462, 82.0968203953022, 82.29733600687481, 82.46920653107992, 82.66972214265253, 82.81294757949011, 82.94185047264394, 83.11372099684904, 83.28559152105413, 83.60068748209682, 83.8441707247207, 83.97307361787453, 84.11629905471212, 84.23087940418218, 84.37410484101976, 84.57462045259237, 84.76081352048124, 84.86107132626755, 85.17616728731022, 85.27642509309653, 85.4912632483529, 85.70610140360928, 85.84932684044686, 85.94958464623316, 86.12145517043827, 86.25035806359209, 86.50816384989974, 86.47951876253222, 86.6227441993698, 86.82325981094242, 86.95216270409625, 87.1383557719851, 87.19564594672013, 87.28158120882267, 87.35319392724148, 87.46777427671154, 87.55370953881409, 87.59667716986537, 87.7255800630192, 87.81151532512173, 87.82583786880551, 87.88312804354052, 88.01203093669436, 88.12661128616442, 88.2411916356345, 88.32712689773703, 88.39873961615584, 88.49899742194214, 88.58493268404469, 88.67086794614724, 88.84273847035233, 88.91435118877112, 89.01460899455743, 89.02893153824118, 89.15783443139502, 89.22944714981381, 89.28673732454884, 89.34402749928387, 89.45860784875393, 89.50157547980521, 89.5445431108565, 89.6734460040103, 89.75938126611285, 89.85963907189917, 89.90260670295044, 90.04583213978803, 90.00286450873675, 90.06015468347178, 90.01718705242051, 90.00286450873675, 90.00286450873675, 90.10312231452305, 90.16041248925809, 90.21770266399312, 90.24634775136064, 90.21770266399312, 90.28931538241191, 90.38957318819823, 90.46118590661702, 90.44686336293324, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.6276138642223, "exact_area100": 85.68232827860912}, "test": {"score": [76.42247255506925, 76.94625816172777, 77.33371600774916, 77.663772691397, 78.02970510152831, 78.31671091339601, 78.48173925521992, 78.80462079357106, 79.19925378488915, 79.41450814378992, 79.67281337447083, 79.93111860515177, 80.19659898112937, 80.41185334003013, 80.73473487838129, 81.04326612613906, 81.43789911745714, 81.76078065580829, 81.94015928822559, 82.10518763004951, 82.37066800602713, 82.65767381789482, 82.88727846738897, 83.08818253569635, 83.27473631341034, 83.46129009112435, 83.64066872354165, 83.79852192006888, 83.93484968070604, 84.1429288943101, 84.4227595608811, 84.75999138982566, 84.91784458635287, 85.10439836406688, 85.31247757767095, 85.53490708186841, 85.62100882542872, 85.69276027839564, 85.85778862021957, 86.11609385090048, 86.19502044916409, 86.3815742268781, 86.52507713281194, 86.7546817823061, 86.87665925234987, 87.0273373035804, 87.10626390184402, 87.15648991892087, 87.24976680777786, 87.37174427782163, 87.47937145727201, 87.5941737820191, 87.68745067087609, 87.73050154265624, 87.85247901270002, 87.93858075626032, 87.96010619215039, 88.0533830810074, 88.07490851689747, 88.17536055105116, 88.23276171342471, 88.26146229461148, 88.34756403817178, 88.412140345842, 88.48389179880894, 88.5125923799957, 88.59151897825932, 88.65609528592955, 88.72067159359977, 88.77807275597331, 88.84264906364353, 88.878524790127, 88.92875080720385, 88.98615196957739, 89.01485255076416, 89.11530458491785, 89.15118031140132, 89.21575661907154, 89.27315778144508, 89.33055894381862, 89.3520843797087, 89.36643467030207, 89.40948554208224, 89.4310109779723, 89.49558728564253, 89.56733873860946, 89.57451388390615, 89.64626533687307, 89.75389251632346, 89.811293678697, 89.84716940518047, 89.83999425988375, 89.86151969577384, 89.88304513166392, 89.88304513166392, 89.86869484107054, 89.91174571285069, 89.94044629403746, 89.97632202052091, 90.04089832819116, 90.06242376408122], "calls": [0.0, 1.3274018798880678, 2.532826289732367, 3.522996340675898, 4.455765229245892, 5.517686733156346, 6.3141278610891876, 7.203845877879027, 8.29446796297625, 9.255937432733013, 10.124130013632776, 10.956446868049078, 11.774413431871995, 12.73588290162876, 13.66147664490206, 14.68034727703236, 15.842720815096506, 16.696563105402884, 17.57193083159934, 18.23204419889503, 19.337016574585636, 20.413288369089475, 21.489560163593314, 22.37210303508646, 23.570352299634067, 24.790127000071752, 25.629618999784746, 26.404534691827507, 27.394704742771044, 28.21984645189065, 29.059338451603644, 30.25041257085456, 31.56346416014924, 32.48905790342255, 33.37160077491569, 34.426347133529454, 35.19408768027553, 36.09815598765875, 36.98787400444859, 38.006744636578894, 38.9323383798522, 39.91533328549903, 40.89115304584918, 41.90284853268278, 43.0006457630767, 44.12714357465738, 45.31104254861161, 46.14335940302791, 47.248331778718516, 48.18827581258521, 49.16409557293535, 50.383870273373034, 51.330989452536414, 52.4574872641171, 53.54810934921432, 54.62438114371816, 55.442347707541074, 56.61189639090192, 57.74556934777929, 58.63528736456913, 59.67568343258951, 60.55105115878596, 61.56992179091627, 62.48116524359618, 63.50721102102318, 64.53325679845017, 65.54495228528377, 66.72167611394131, 68.18540575446653, 69.0392480447729, 70.02224295041974, 71.04111358255004, 71.93800674463657, 72.69857214608597, 73.71026763291957, 74.66456195737963, 75.5255793929827, 76.63055176867331, 77.82880103322091, 78.9552988448016, 79.78044055392121, 80.83518691253498, 81.58857716868766, 82.4854703307742, 83.53304154409128, 84.6236636291885, 85.49185621008826, 86.5537777139987, 87.75920212384301, 88.81394848245677, 89.78259309751022, 90.77276314845376, 91.7485829088039, 93.00423333572505, 94.02310396785535, 94.85542082227165, 95.8240654373251, 96.7640094711918, 97.8618067015857, 99.01700509435317, 100.0], "exact_score": [76.42247255506925, 76.81710554638732, 77.13281193944178, 77.50591949486977, 77.81445074262754, 78.18755829805554, 78.41716294754968, 78.69699361412069, 79.0557508789553, 79.32840640022961, 79.63693764798737, 79.94546889574514, 80.2540001435029, 80.52665566477721, 80.87106263901845, 81.16524359618282, 81.50247542512736, 81.79665638229174, 82.04778646767598, 82.3060916983569, 82.52852120255434, 82.80117672382866, 82.97338021094927, 83.18863456985004, 83.31061203989381, 83.49716581760781, 83.69806988591519, 83.85592308244242, 84.09270287723328, 84.40123412499103, 84.65953935567195, 84.84609313338596, 85.04699720169334, 85.23355097940734, 85.44880533830809, 85.59230824424195, 85.67840998780225, 85.85778862021957, 86.11609385090048, 86.21654588505417, 86.38874937217479, 86.53225227810863, 86.76185692760279, 86.86948410705317, 87.00581186769031, 87.10626390184402, 87.16366506421755, 87.24259166248117, 87.3071679701514, 87.4937217478654, 87.57264834612901, 87.65875008968932, 87.71615125206286, 87.80942814091985, 87.89552988448015, 87.96728133744709, 88.00315706393054, 88.08208366219417, 88.07490851689747, 88.20406113223794, 88.22558656812801, 88.29016287579825, 88.39061490995192, 88.45519121762216, 88.5125923799957, 88.54846810647916, 88.62739470474277, 88.691971012413, 88.71349644830309, 88.78524790127, 88.84264906364353, 88.87134964483032, 88.92875080720385, 88.97897682428069, 89.02920284135753, 89.158355456698, 89.15118031140132, 89.20858147377486, 89.29468321733516, 89.3377340891153, 89.3520843797087, 89.38796010619215, 89.45253641386238, 89.45971155915908, 89.51711272153261, 89.56733873860946, 89.6103896103896, 89.69649135394992, 89.76106766162016, 89.84716940518047, 89.82564396929038, 89.83999425988375, 89.83999425988375, 89.88304513166392, 89.88304513166392, 89.86151969577384, 89.93327114874077, 89.94044629403746, 90.00502260170768, 90.04089832819116, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 80.26906794862596, "exact_area100": 86.04945540328005}}, "old_normalized_endpoint_budget": 100}, {"seed": 2028, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 1.0, "validation_area100": 0.8590087322796985, "training_seconds": 29.63543311902322, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.3240878220366937, "operator": "ge"}, {"budget": 0.02, "threshold": 0.2620133574346226, "operator": "ge"}, {"budget": 0.03, "threshold": 0.2255127621683145, "operator": "ge"}, {"budget": 0.04, "threshold": 0.19267696525996375, "operator": "ge"}, {"budget": 0.05, "threshold": 0.16724052849981844, "operator": "ge"}, {"budget": 0.06, "threshold": 0.14420323415526104, "operator": "ge"}, {"budget": 0.07, "threshold": 0.12201452920191971, "operator": "ge"}, {"budget": 0.08, "threshold": 0.09824976331090486, "operator": "ge"}, {"budget": 0.09, "threshold": 0.0793003024198582, "operator": "ge"}, {"budget": 0.1, "threshold": 0.06095987067454016, "operator": "ge"}, {"budget": 0.11, "threshold": 0.0441155675122175, "operator": "ge"}, {"budget": 0.12, "threshold": 0.02586482612675626, "operator": "ge"}, {"budget": 0.13, "threshold": 0.009070849646753656, "operator": "ge"}, {"budget": 0.14, "threshold": -0.014887920881937385, "operator": "ge"}, {"budget": 0.15, "threshold": -0.034140464307120075, "operator": "ge"}, {"budget": 0.16, "threshold": -0.054490828801646414, "operator": "ge"}, {"budget": 0.17, "threshold": -0.07596927286209193, "operator": "ge"}, {"budget": 0.18, "threshold": -0.10118714269171081, "operator": "ge"}, {"budget": 0.19, "threshold": -0.12767748069735296, "operator": "ge"}, {"budget": 0.2, "threshold": -0.15411120526434788, "operator": "ge"}, {"budget": 0.21, "threshold": -0.1822786752533044, "operator": "ge"}, {"budget": 0.22, "threshold": -0.20778569321955426, "operator": "ge"}, {"budget": 0.23, "threshold": -0.24250100161656643, "operator": "ge"}, {"budget": 0.24, "threshold": -0.2721191947928602, "operator": "ge"}, {"budget": 0.25, "threshold": -0.30532776359927416, "operator": "ge"}, {"budget": 0.26, "threshold": -0.3367520547022355, "operator": "ge"}, {"budget": 0.27, "threshold": -0.361724751840564, "operator": "ge"}, {"budget": 0.28, "threshold": -0.38832293037177334, "operator": "ge"}, {"budget": 0.29, "threshold": -0.4176886446622701, "operator": "ge"}, {"budget": 0.3, "threshold": -0.43978458129612935, "operator": "ge"}, {"budget": 0.31, "threshold": -0.46657448845901134, "operator": "ge"}, {"budget": 0.32, "threshold": -0.49462748181136246, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5127434903105746, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5380924528560538, "operator": "ge"}, {"budget": 0.35, "threshold": -0.5612953289827608, "operator": "ge"}, {"budget": 0.36, "threshold": -0.5803352878428444, "operator": "ge"}, {"budget": 0.37, "threshold": -0.5989442449704563, "operator": "ge"}, {"budget": 0.38, "threshold": -0.6213741810192529, "operator": "ge"}, {"budget": 0.39, "threshold": -0.6355555664967532, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6490216091462888, "operator": "ge"}, {"budget": 0.41, "threshold": -0.659978586632905, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6720890497476812, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6864302339300059, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6985251640038759, "operator": "ge"}, {"budget": 0.45, "threshold": -0.7126746100428446, "operator": "ge"}, {"budget": 0.46, "threshold": -0.726752945879698, "operator": "ge"}, {"budget": 0.47, "threshold": -0.7400369686139302, "operator": "ge"}, {"budget": 0.48, "threshold": -0.7523432006007231, "operator": "ge"}, {"budget": 0.49, "threshold": -0.7635783892646232, "operator": "ge"}, {"budget": 0.5, "threshold": -0.7725022993274507, "operator": "ge"}, {"budget": 0.51, "threshold": -0.783299236317724, "operator": "ge"}, {"budget": 0.52, "threshold": -0.7927400620407397, "operator": "ge"}, {"budget": 0.53, "threshold": -0.8019443780683696, "operator": "ge"}, {"budget": 0.54, "threshold": -0.8113973313077862, "operator": "ge"}, {"budget": 0.55, "threshold": -0.8197628793687921, "operator": "ge"}, {"budget": 0.56, "threshold": -0.8284844694120506, "operator": "ge"}, {"budget": 0.57, "threshold": -0.8369111382073495, "operator": "ge"}, {"budget": 0.58, "threshold": -0.843899494433805, "operator": "ge"}, {"budget": 0.59, "threshold": -0.8506523001624816, "operator": "ge"}, {"budget": 0.6, "threshold": -0.8559692670579705, "operator": "ge"}, {"budget": 0.61, "threshold": -0.8613931757140982, "operator": "ge"}, {"budget": 0.62, "threshold": -0.8665133316007221, "operator": "ge"}, {"budget": 0.63, "threshold": -0.872023394139841, "operator": "ge"}, {"budget": 0.64, "threshold": -0.8770127610319158, "operator": "ge"}, {"budget": 0.65, "threshold": -0.8819383616129419, "operator": "ge"}, {"budget": 0.66, "threshold": -0.8866808551467621, "operator": "ge"}, {"budget": 0.67, "threshold": -0.8916071726068019, "operator": "ge"}, {"budget": 0.68, "threshold": -0.8969997941692255, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9011453909680089, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9052448546275716, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9095033749550578, "operator": "ge"}, {"budget": 0.72, "threshold": -0.913689092921953, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9169443827592801, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9204133770624419, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9243087294688179, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9277412927236228, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9307633116282213, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9334082359461293, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9368205108627128, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9396481511896879, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9426224779170506, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9451909076259505, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9473381516725468, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9500066628166338, "operator": "ge"}, {"budget": 0.85, "threshold": -0.952967475633249, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9554042238951144, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9576899138545137, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9597003720694786, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9621085427704468, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9643413996236843, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9665515116397488, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9692617879718044, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9717649296205236, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9741589259993201, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9768039430722238, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9795452119865204, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9821803724331525, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9856563930770151, "operator": "ge"}, {"budget": 0.99, "threshold": -0.989411737201357, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.36780292179891, 76.68289888284158, 77.12689773703809, 77.59954167860211, 77.88599255227729, 78.15812088226869, 78.44457175594385, 78.71670008593526, 78.91721569750787, 79.23231165855056, 79.46147235749069, 79.81953594958465, 80.09166427957605, 80.39243769693498, 80.62159839587511, 80.92237181323402, 81.33772558006301, 81.62417645373819, 81.88198224004583, 82.24004583213978, 82.3689487252936, 82.62675451160125, 82.82727012317388, 82.98481810369522, 83.28559152105413, 83.44313950157549, 83.72959037525064, 83.95875107419077, 84.13062159839588, 84.31681466628473, 84.46004010312231, 84.7035233457462, 84.90403895731882, 84.98997421942136, 85.26210254941277, 85.51990833572042, 85.60584359782297, 85.7347464909768, 85.82068175307934, 85.94958464623316, 86.02119736465195, 86.16442280148954, 86.25035806359209, 86.55113148095101, 86.65138928673733, 86.83758235462618, 86.99513033514752, 87.12403322830134, 87.2386135777714, 87.39616155829275, 87.49641936407906, 87.69693497565167, 87.73990260670294, 87.75422515038672, 87.79719278143799, 87.96906330564308, 88.14093382984818, 88.25551417931825, 88.35577198510455, 88.45602979089085, 88.55628759667717, 88.67086794614724, 88.81409338298482, 88.91435118877112, 88.8857061014036, 88.92867373245488, 89.02893153824118, 89.11486680034373, 89.18647951876252, 89.25809223718132, 89.3297049556001, 89.4013176740189, 89.47293039243769, 89.51589802348897, 89.60183328559151, 89.68776854769406, 89.7450587224291, 89.73073617874535, 89.7450587224291, 89.8166714408479, 89.95989687768548, 90.01718705242051, 90.03150959610426, 90.10312231452305, 90.11744485820681, 90.17473503294184, 90.23202520767688, 90.20338012030936, 90.27499283872815, 90.30363792609567, 90.30363792609567, 90.31796046977942, 90.34660555714696, 90.40389573188197, 90.41821827556574, 90.40389573188197, 90.46118590661702, 90.44686336293324, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 64, 88, 122, 157, 182, 203, 229, 250, 272, 298, 318, 344, 368, 394, 417, 441, 472, 495, 518, 545, 557, 579, 600, 618, 645, 663, 687, 711, 726, 742, 761, 782, 799, 810, 830, 850, 862, 882, 893, 909, 921, 937, 948, 970, 980, 998, 1012, 1029, 1043, 1060, 1073, 1088, 1097, 1107, 1115, 1130, 1144, 1154, 1163, 1175, 1185, 1197, 1210, 1218, 1223, 1229, 1242, 1250, 1259, 1267, 1273, 1283, 1289, 1297, 1306, 1314, 1319, 1324, 1328, 1337, 1349, 1357, 1361, 1370, 1376, 1383, 1389, 1392, 1400, 1403, 1408, 1411, 1416, 1423, 1430, 1433, 1440, 1443, 1449], "harmed": [0, 3, 8, 10, 13, 15, 20, 22, 28, 30, 38, 42, 46, 47, 52, 57, 64, 67, 69, 72, 77, 79, 82, 86, 93, 100, 106, 113, 117, 125, 128, 131, 140, 144, 147, 152, 153, 155, 161, 172, 177, 184, 191, 197, 202, 203, 206, 211, 214, 222, 228, 234, 240, 241, 247, 256, 261, 264, 266, 268, 270, 275, 278, 282, 285, 286, 293, 296, 302, 304, 308, 311, 312, 317, 318, 323, 326, 328, 329, 335, 338, 342, 344, 348, 351, 355, 360, 363, 365, 370, 373, 374, 379, 381, 384, 387, 393, 397, 400, 404, 406], "exact_score": [75.56574047550845, 75.98109424233743, 76.36780292179891, 76.68289888284158, 77.12689773703809, 77.59954167860211, 77.88599255227729, 78.15812088226869, 78.44457175594385, 78.71670008593526, 78.91721569750787, 79.23231165855056, 79.46147235749069, 79.81953594958465, 80.09166427957605, 80.39243769693498, 80.62159839587511, 80.92237181323402, 81.33772558006301, 81.62417645373819, 81.88198224004583, 82.24004583213978, 82.3689487252936, 82.62675451160125, 82.82727012317388, 82.98481810369522, 83.28559152105413, 83.44313950157549, 83.72959037525064, 83.95875107419077, 84.13062159839588, 84.31681466628473, 84.46004010312231, 84.7035233457462, 84.90403895731882, 84.98997421942136, 85.26210254941277, 85.51990833572042, 85.60584359782297, 85.7347464909768, 85.82068175307934, 85.94958464623316, 86.02119736465195, 86.16442280148954, 86.25035806359209, 86.55113148095101, 86.65138928673733, 86.83758235462618, 86.99513033514752, 87.12403322830134, 87.2386135777714, 87.39616155829275, 87.49641936407906, 87.69693497565167, 87.73990260670294, 87.75422515038672, 87.79719278143799, 87.96906330564308, 88.14093382984818, 88.25551417931825, 88.35577198510455, 88.45602979089085, 88.55628759667717, 88.67086794614724, 88.81409338298482, 88.91435118877112, 88.8857061014036, 88.92867373245488, 89.02893153824118, 89.11486680034373, 89.18647951876252, 89.25809223718132, 89.3297049556001, 89.4013176740189, 89.47293039243769, 89.51589802348897, 89.60183328559151, 89.68776854769406, 89.7450587224291, 89.73073617874535, 89.7450587224291, 89.8166714408479, 89.95989687768548, 90.01718705242051, 90.03150959610426, 90.10312231452305, 90.11744485820681, 90.17473503294184, 90.23202520767688, 90.20338012030936, 90.27499283872815, 90.30363792609567, 90.30363792609567, 90.31796046977942, 90.34660555714696, 90.40389573188197, 90.41821827556574, 90.40389573188197, 90.46118590661702, 90.44686336293324, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.90087322796985, "threshold_area100": 85.89981380693213, "threshold_area25": 79.61329132053851}, "test": {"score": [76.42247255506925, 76.86015641816746, 77.22608882829877, 77.52744493075984, 77.87185190500108, 78.18038315275885, 78.45303867403315, 78.72569419530745, 79.12032718662553, 79.40733299849322, 79.65846308387745, 79.8880677333716, 80.29705101528306, 80.58405682715075, 80.97151467317214, 81.22264475855636, 81.4594245533472, 81.60292745928105, 81.87558298055536, 82.09083733945613, 82.32044198895028, 82.54287149314774, 82.74377556145512, 83.05948195450958, 83.31778718519051, 83.56174212527804, 83.74829590299204, 83.94919997129942, 84.20033005668365, 84.45863528736457, 84.6236636291885, 84.73846595393557, 84.98242089402311, 85.14027409055034, 85.3627035947478, 85.58513309894525, 85.73581115017579, 85.97976609026333, 86.17349501327402, 86.24524646624094, 86.47485111573509, 86.5681280045921, 86.67575518404249, 86.88383439764655, 86.99863672239363, 87.08473846595393, 87.2138910812944, 87.27846738896463, 87.3932697137117, 87.45784602138194, 87.63722465379924, 87.71615125206286, 87.78072755973308, 87.93858075626032, 87.945755901557, 88.01750735452393, 88.14665996986439, 88.25428714931478, 88.26863743990816, 88.38343976465524, 88.46954150821554, 88.56281839707253, 88.61304441414939, 88.67762072181962, 88.6847958671163, 88.72067159359977, 88.82829877305016, 88.90722537131377, 88.97897682428069, 89.02202769606085, 89.08660400373107, 89.158355456698, 89.17270574729139, 89.23728205496163, 89.33055894381862, 89.33055894381862, 89.39513525148884, 89.43818612326899, 89.48841214034584, 89.56733873860946, 89.624739900983, 89.71801678984, 89.72519193513668, 89.77541795221353, 89.84716940518047, 89.91174571285069, 89.97632202052091, 90.00502260170768, 90.06959890937792, 90.0982994905647, 90.10547463586138, 90.10547463586138, 90.06242376408122, 90.13417521704815, 90.11264978115807, 90.12700007175145, 90.13417521704815, 90.12700007175145, 90.09112434526799, 90.06959890937792, 90.06242376408122], "calls": [0.0, 1.33457702518476, 2.274521059051446, 3.0135610246107487, 3.9678553490708186, 4.836047929970582, 5.754466527947192, 6.6370093994403385, 7.799382937504484, 8.760852407261247, 9.765372748798164, 10.67661620147808, 11.767238286575303, 12.649781158068452, 14.056109636220132, 14.917127071823206, 15.749443926239506, 16.545885054172345, 17.47865394274234, 18.468823993685874, 19.50922006170625, 20.441988950276244, 21.202554351725624, 22.515605941020304, 23.469900265480376, 24.52464662409414, 25.464590657960823, 26.311257802970513, 27.29425270861735, 28.506852263758343, 29.20284135753749, 30.25041257085456, 31.4199612542154, 32.173351510368086, 33.285499031355386, 34.49092344119968, 35.610246107483675, 36.686517901987514, 37.96369376479874, 39.03279041400589, 39.97273444787257, 40.704599268135176, 41.544091267848174, 42.64906364353878, 43.52443136973524, 44.55765229245892, 45.74155126641315, 46.738896462653365, 47.59991389825644, 48.51115735093636, 49.4008753677262, 50.2331922221425, 51.05833393126211, 52.012628255722184, 52.93104685369878, 53.94274234053239, 54.90421181028916, 56.06658534835331, 57.01370452751669, 57.92494798019659, 58.69986367223936, 59.38867762072182, 60.28557078280835, 61.32596685082873, 62.13675826935495, 63.05517686733156, 64.00229604649495, 65.04269211451532, 66.28399225084308, 67.22393628470977, 68.24998206213677, 69.51998277965129, 70.7254071894956, 71.60795006098873, 72.69857214608597, 73.8681208294468, 75.0735452392911, 76.23591877735524, 77.21891368300207, 78.71134390471407, 79.65846308387745, 80.84236205783168, 81.9258089976322, 82.72225012556504, 83.88462366362918, 85.23355097940734, 86.33134820980125, 87.23541651718448, 88.01750735452393, 89.00767740546746, 89.85434455047715, 90.76558800315706, 91.85621008825429, 92.91813159216474, 93.77197388247112, 94.87694625816174, 95.98191863385233, 96.77118461648848, 97.85463155628902, 98.85197675252924, 100.0], "rescued": [0, 69, 130, 173, 228, 279, 325, 367, 430, 479, 524, 561, 622, 667, 731, 776, 817, 847, 892, 934, 977, 1014, 1048, 1110, 1155, 1201, 1239, 1276, 1321, 1374, 1405, 1436, 1484, 1513, 1553, 1595, 1625, 1667, 1706, 1734, 1776, 1798, 1822, 1862, 1887, 1906, 1943, 1966, 1988, 2007, 2041, 2062, 2078, 2108, 2124, 2147, 2171, 2198, 2212, 2233, 2249, 2268, 2283, 2303, 2312, 2332, 2356, 2372, 2393, 2408, 2426, 2443, 2454, 2468, 2487, 2502, 2521, 2533, 2545, 2564, 2579, 2595, 2603, 2615, 2631, 2649, 2661, 2670, 2683, 2698, 2705, 2713, 2719, 2732, 2737, 2745, 2753, 2760, 2769, 2779, 2791], "harmed": [0, 8, 18, 19, 26, 34, 42, 46, 54, 63, 73, 78, 82, 87, 97, 107, 115, 125, 132, 144, 155, 161, 167, 185, 194, 206, 218, 227, 237, 254, 262, 277, 291, 298, 307, 318, 327, 335, 347, 365, 375, 384, 393, 404, 413, 420, 439, 453, 459, 469, 478, 488, 495, 503, 518, 531, 537, 549, 561, 566, 570, 576, 584, 595, 603, 618, 627, 632, 643, 652, 661, 668, 677, 682, 688, 703, 713, 719, 724, 732, 739, 742, 749, 754, 760, 769, 772, 777, 781, 792, 798, 806, 818, 821, 829, 835, 842, 850, 864, 877, 890], "exact_score": [76.42247255506925, 76.69512807634355, 77.13281193944178, 77.52026978546316, 77.88620219559446, 78.2306091698357, 78.51043983640669, 78.876372246538, 79.17772834899907, 79.43603357967999, 79.73021453684437, 79.9741694769319, 80.36162732295328, 80.69168400660114, 80.94281409198535, 81.24417019444644, 81.50965057042406, 81.75360551051159, 82.01908588648921, 82.2056396642032, 82.47112004018081, 82.68637439908159, 82.9446796297625, 83.15993398866327, 83.41823921934419, 83.64066872354165, 83.86309822773912, 84.10705316782665, 84.3366578173208, 84.55908732151826, 84.72411566334218, 84.91784458635287, 85.09722321877018, 85.29095214178088, 85.42727990241804, 85.66405969720887, 85.79321231254933, 86.06586783382365, 86.18067015857072, 86.23807132094426, 86.4676759704384, 86.59682858577887, 86.79773265408625, 86.95558585061349, 87.05603788476716, 87.14931477362417, 87.23541651718448, 87.32151826074478, 87.43632058549186, 87.5798234914257, 87.69462581617277, 87.78072755973308, 87.93858075626032, 87.96010619215039, 88.02468249982063, 88.14665996986439, 88.23276171342471, 88.26863743990816, 88.39061490995192, 88.52694267058908, 88.59151897825932, 88.64174499533615, 88.67762072181962, 88.73502188419316, 88.82829877305016, 88.89287508072039, 88.95745138839062, 89.02920284135753, 89.04355313195093, 89.13683002080792, 89.158355456698, 89.16553060199469, 89.2588074908517, 89.3520843797087, 89.3377340891153, 89.38796010619215, 89.4238358326756, 89.45971155915908, 89.5386381574227, 89.59603931979623, 89.66779077276314, 89.72519193513668, 89.73236708043338, 89.79694338810361, 89.84716940518047, 89.8973954222573, 89.96914687522423, 90.00502260170768, 90.06242376408122, 90.0982994905647, 90.11264978115807, 90.11264978115807, 90.06959890937792, 90.14135036234484, 90.11264978115807, 90.12700007175145, 90.13417521704815, 90.13417521704815, 90.09112434526799, 90.06959890937792, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.21455993951483, "threshold_area100": 86.15344048216977, "threshold_area25": 80.27480806486331}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.19621776044368744, "operator": "ge"}, {"budget": 0.02, "threshold": 0.16494610905647278, "operator": "ge"}, {"budget": 0.03, "threshold": 0.14261195063591003, "operator": "ge"}, {"budget": 0.04, "threshold": 0.12494911253452301, "operator": "ge"}, {"budget": 0.05, "threshold": 0.10825667530298233, "operator": "ge"}, {"budget": 0.06, "threshold": 0.09344153106212616, "operator": "ge"}, {"budget": 0.07, "threshold": 0.07594668865203857, "operator": "ge"}, {"budget": 0.08, "threshold": 0.06188488006591797, "operator": "ge"}, {"budget": 0.09, "threshold": 0.04509425163269043, "operator": "ge"}, {"budget": 0.1, "threshold": 0.0327327623963356, "operator": "ge"}, {"budget": 0.11, "threshold": 0.021744661033153534, "operator": "ge"}, {"budget": 0.12, "threshold": 0.012372896075248718, "operator": "ge"}, {"budget": 0.13, "threshold": 0.0040642134845256805, "operator": "ge"}, {"budget": 0.14, "threshold": -0.004869244992733002, "operator": "ge"}, {"budget": 0.15, "threshold": -0.007708340883255005, "operator": "ge"}, {"budget": 0.16, "threshold": -0.009758958593010902, "operator": "ge"}, {"budget": 0.17, "threshold": -0.011484116315841675, "operator": "ge"}, {"budget": 0.18, "threshold": -0.013551086187362671, "operator": "ge"}, {"budget": 0.19, "threshold": -0.015245321206748486, "operator": "ge"}, {"budget": 0.2, "threshold": -0.017380325123667717, "operator": "ge"}, {"budget": 0.21, "threshold": -0.0192456915974617, "operator": "ge"}, {"budget": 0.22, "threshold": -0.020850909873843193, "operator": "ge"}, {"budget": 0.23, "threshold": -0.02242724597454071, "operator": "ge"}, {"budget": 0.24, "threshold": -0.024139706045389175, "operator": "ge"}, {"budget": 0.25, "threshold": -0.025741776451468468, "operator": "ge"}, {"budget": 0.26, "threshold": -0.02760748378932476, "operator": "ge"}, {"budget": 0.27, "threshold": -0.029294604435563087, "operator": "ge"}, {"budget": 0.28, "threshold": -0.031221959739923477, "operator": "ge"}, {"budget": 0.29, "threshold": -0.03332192450761795, "operator": "ge"}, {"budget": 0.3, "threshold": -0.03498038277029991, "operator": "ge"}, {"budget": 0.31, "threshold": -0.037239331752061844, "operator": "ge"}, {"budget": 0.32, "threshold": -0.03914574161171913, "operator": "ge"}, {"budget": 0.33, "threshold": -0.04166673123836517, "operator": "ge"}, {"budget": 0.34, "threshold": -0.043801188468933105, "operator": "ge"}, {"budget": 0.35, "threshold": -0.04599952697753906, "operator": "ge"}, {"budget": 0.36, "threshold": -0.04796290025115013, "operator": "ge"}, {"budget": 0.37, "threshold": -0.050400421023368835, "operator": "ge"}, {"budget": 0.38, "threshold": -0.052510764449834824, "operator": "ge"}, {"budget": 0.39, "threshold": -0.05425805598497391, "operator": "ge"}, {"budget": 0.4, "threshold": -0.05652887001633644, "operator": "ge"}, {"budget": 0.41, "threshold": -0.05897572264075279, "operator": "ge"}, {"budget": 0.42, "threshold": -0.06155781075358391, "operator": "ge"}, {"budget": 0.43, "threshold": -0.06398937106132507, "operator": "ge"}, {"budget": 0.44, "threshold": -0.06712834537029266, "operator": "ge"}, {"budget": 0.45, "threshold": -0.06922703981399536, "operator": "ge"}, {"budget": 0.46, "threshold": -0.07138389348983765, "operator": "ge"}, {"budget": 0.47, "threshold": -0.07408379018306732, "operator": "ge"}, {"budget": 0.48, "threshold": -0.07674439996480942, "operator": "ge"}, {"budget": 0.49, "threshold": -0.07878315448760986, "operator": "ge"}, {"budget": 0.5, "threshold": -0.08119391649961472, "operator": "ge"}, {"budget": 0.51, "threshold": -0.08372557908296585, "operator": "ge"}, {"budget": 0.52, "threshold": -0.08577344566583633, "operator": "ge"}, {"budget": 0.53, "threshold": -0.08895888924598694, "operator": "ge"}, {"budget": 0.54, "threshold": -0.09167656302452087, "operator": "ge"}, {"budget": 0.55, "threshold": -0.09375953674316406, "operator": "ge"}, {"budget": 0.56, "threshold": -0.09600233286619186, "operator": "ge"}, {"budget": 0.57, "threshold": -0.09911605715751648, "operator": "ge"}, {"budget": 0.58, "threshold": -0.10191789269447327, "operator": "ge"}, {"budget": 0.59, "threshold": -0.10433895885944366, "operator": "ge"}, {"budget": 0.6, "threshold": -0.10699912905693054, "operator": "ge"}, {"budget": 0.61, "threshold": -0.10941236466169357, "operator": "ge"}, {"budget": 0.62, "threshold": -0.11248452216386795, "operator": "ge"}, {"budget": 0.63, "threshold": -0.11586908996105194, "operator": "ge"}, {"budget": 0.64, "threshold": -0.1186208724975586, "operator": "ge"}, {"budget": 0.65, "threshold": -0.12164050340652466, "operator": "ge"}, {"budget": 0.66, "threshold": -0.12453514337539673, "operator": "ge"}, {"budget": 0.67, "threshold": -0.1275503933429718, "operator": "ge"}, {"budget": 0.68, "threshold": -0.13112536072731018, "operator": "ge"}, {"budget": 0.69, "threshold": -0.1339987963438034, "operator": "ge"}, {"budget": 0.7, "threshold": -0.13678821921348572, "operator": "ge"}, {"budget": 0.71, "threshold": -0.14049845933914185, "operator": "ge"}, {"budget": 0.72, "threshold": -0.14380738139152527, "operator": "ge"}, {"budget": 0.73, "threshold": -0.1468200832605362, "operator": "ge"}, {"budget": 0.74, "threshold": -0.15049247443675995, "operator": "ge"}, {"budget": 0.75, "threshold": -0.15460161864757538, "operator": "ge"}, {"budget": 0.76, "threshold": -0.15886913239955902, "operator": "ge"}, {"budget": 0.77, "threshold": -0.16272079944610596, "operator": "ge"}, {"budget": 0.78, "threshold": -0.16777297854423523, "operator": "ge"}, {"budget": 0.79, "threshold": -0.17282558977603912, "operator": "ge"}, {"budget": 0.8, "threshold": -0.17819687724113464, "operator": "ge"}, {"budget": 0.81, "threshold": -0.1826077699661255, "operator": "ge"}, {"budget": 0.82, "threshold": -0.18893536925315857, "operator": "ge"}, {"budget": 0.83, "threshold": -0.19488981366157532, "operator": "ge"}, {"budget": 0.84, "threshold": -0.199966698884964, "operator": "ge"}, {"budget": 0.85, "threshold": -0.20641764998435974, "operator": "ge"}, {"budget": 0.86, "threshold": -0.21525588631629944, "operator": "ge"}, {"budget": 0.87, "threshold": -0.22493109107017517, "operator": "ge"}, {"budget": 0.88, "threshold": -0.2358090579509735, "operator": "ge"}, {"budget": 0.89, "threshold": -0.24661575257778168, "operator": "ge"}, {"budget": 0.9, "threshold": -0.25826510787010193, "operator": "ge"}, {"budget": 0.91, "threshold": -0.2730156481266022, "operator": "ge"}, {"budget": 0.92, "threshold": -0.28802478313446045, "operator": "ge"}, {"budget": 0.93, "threshold": -0.3098578453063965, "operator": "ge"}, {"budget": 0.94, "threshold": -0.33238592743873596, "operator": "ge"}, {"budget": 0.95, "threshold": -0.36259204149246216, "operator": "ge"}, {"budget": 0.96, "threshold": -0.40614861249923706, "operator": "ge"}, {"budget": 0.97, "threshold": -0.4530940055847168, "operator": "ge"}, {"budget": 0.98, "threshold": -0.50173419713974, "operator": "ge"}, {"budget": 0.99, "threshold": -0.5647693276405334, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.96677169865367, 76.28186765969636, 76.7115439702091, 77.15554282440561, 77.49928387281581, 77.7427671154397, 78.04354053279863, 78.32999140647378, 78.61644228014895, 78.90289315382411, 79.2179891148668, 79.53308507590948, 79.83385849326841, 80.10598682325981, 80.14895445431108, 80.36379260956745, 80.49269550272128, 80.60727585219135, 80.77914637639645, 80.85075909481525, 80.90804926955026, 81.0369521627041, 81.12288742480665, 81.23746777427671, 81.3090804926955, 81.42366084216557, 81.60985391005443, 81.71011171584072, 81.8246920653108, 81.9965625895159, 82.0968203953022, 82.21140074477226, 82.39759381266113, 82.54081924949871, 82.64107705528502, 82.69836723002004, 82.82727012317388, 82.94185047264394, 83.07075336579777, 83.08507590948152, 83.17101117158407, 83.25694643368662, 83.29991406473789, 83.35720423947292, 83.52907476367803, 83.70094528788312, 83.85849326840446, 84.04468633629332, 84.10197651102835, 84.24520194786594, 84.40274992838728, 84.48868519048983, 84.58894299627615, 84.71784588942997, 84.94700658837009, 85.1475221999427, 85.16184474362647, 85.36236035519909, 85.4912632483529, 85.63448868519049, 85.79203666571182, 86.0068748209682, 86.1071326267545, 86.26468060727585, 86.29332569464336, 86.47951876253222, 86.59409911200228, 86.73732454883987, 86.86622744199369, 87.09538814093382, 87.25293612145518, 87.45345173302779, 87.55370953881409, 87.63964480091664, 87.81151532512173, 87.98338584932684, 88.22686909195072, 88.37009452878831, 88.48467487825837, 88.67086794614724, 88.94299627613864, 89.07189916929245, 89.14351188771126, 89.21512460613005, 89.41564021770266, 89.41564021770266, 89.51589802348897, 89.75938126611285, 89.87396161558291, 89.97421942136923, 90.13176740189057, 90.20338012030936, 90.37525064451447, 90.46118590661702, 90.3609281008307, 90.44686336293324, 90.50415353766829, 90.67602406187338, 90.73331423660842, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 58, 90, 124, 153, 174, 198, 225, 251, 274, 299, 322, 345, 368, 374, 389, 398, 406, 418, 427, 433, 444, 450, 459, 467, 475, 488, 497, 509, 521, 528, 537, 551, 562, 569, 575, 585, 596, 608, 613, 620, 628, 632, 639, 652, 665, 677, 693, 699, 713, 725, 735, 748, 759, 775, 791, 798, 812, 825, 837, 852, 868, 879, 893, 900, 916, 927, 942, 956, 974, 989, 1009, 1021, 1034, 1049, 1063, 1084, 1098, 1110, 1125, 1147, 1159, 1173, 1185, 1203, 1212, 1226, 1250, 1264, 1279, 1293, 1313, 1331, 1350, 1359, 1378, 1398, 1419, 1435, 1449], "harmed": [0, 4, 8, 10, 13, 18, 22, 25, 32, 38, 41, 44, 45, 47, 51, 54, 54, 54, 54, 54, 58, 60, 62, 62, 63, 66, 66, 66, 68, 72, 72, 72, 73, 74, 75, 75, 77, 78, 81, 84, 88, 89, 91, 92, 95, 96, 97, 98, 101, 103, 107, 108, 112, 118, 120, 120, 122, 128, 128, 132, 134, 138, 139, 143, 146, 151, 154, 157, 162, 167, 169, 173, 179, 184, 191, 194, 196, 200, 204, 208, 210, 213, 216, 225, 232, 236, 245, 252, 259, 265, 273, 276, 291, 297, 310, 326, 339, 355, 364, 376, 406], "exact_score": [75.56574047550845, 75.96677169865367, 76.28186765969636, 76.7115439702091, 77.15554282440561, 77.49928387281581, 77.7427671154397, 78.04354053279863, 78.32999140647378, 78.61644228014895, 78.90289315382411, 79.2179891148668, 79.53308507590948, 79.83385849326841, 80.10598682325981, 80.14895445431108, 80.36379260956745, 80.49269550272128, 80.60727585219135, 80.77914637639645, 80.85075909481525, 80.90804926955026, 81.0369521627041, 81.12288742480665, 81.23746777427671, 81.3090804926955, 81.42366084216557, 81.60985391005443, 81.71011171584072, 81.8246920653108, 81.9965625895159, 82.0968203953022, 82.21140074477226, 82.39759381266113, 82.54081924949871, 82.64107705528502, 82.69836723002004, 82.82727012317388, 82.94185047264394, 83.07075336579777, 83.08507590948152, 83.17101117158407, 83.25694643368662, 83.29991406473789, 83.35720423947292, 83.52907476367803, 83.70094528788312, 83.85849326840446, 84.04468633629332, 84.10197651102835, 84.24520194786594, 84.40274992838728, 84.48868519048983, 84.58894299627615, 84.71784588942997, 84.94700658837009, 85.1475221999427, 85.16184474362647, 85.36236035519909, 85.4912632483529, 85.63448868519049, 85.79203666571182, 86.0068748209682, 86.1071326267545, 86.26468060727585, 86.29332569464336, 86.47951876253222, 86.59409911200228, 86.73732454883987, 86.86622744199369, 87.09538814093382, 87.25293612145518, 87.45345173302779, 87.55370953881409, 87.63964480091664, 87.81151532512173, 87.98338584932684, 88.22686909195072, 88.37009452878831, 88.48467487825837, 88.67086794614724, 88.94299627613864, 89.07189916929245, 89.14351188771126, 89.21512460613005, 89.41564021770266, 89.41564021770266, 89.51589802348897, 89.75938126611285, 89.87396161558291, 89.97421942136923, 90.13176740189057, 90.20338012030936, 90.37525064451447, 90.46118590661702, 90.3609281008307, 90.44686336293324, 90.50415353766829, 90.67602406187338, 90.73331423660842, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 84.39425835440004, "threshold_area100": 84.39322543683757, "threshold_area25": 79.16786021197365}, "test": {"score": [76.42247255506925, 76.81710554638732, 77.06106048647486, 77.41264260601277, 77.71399870847385, 78.02252995623161, 78.25930975102247, 78.66111788763723, 78.98399942598839, 79.28535552844946, 79.57236134031714, 79.87371744277823, 80.24682499820621, 80.56253139126068, 80.94281409198535, 81.04326612613906, 81.17959388677622, 81.25852048503982, 81.39484824567698, 81.53117600631413, 81.63880318576452, 81.80383152758843, 81.87558298055536, 81.96168472411567, 82.01908588648921, 82.07648704886275, 82.2056396642032, 82.27739111717013, 82.37066800602713, 82.4782951854775, 82.60027265552128, 82.6648489631915, 82.78682643323528, 82.98055535624597, 83.06665709980628, 83.15993398866327, 83.28191145870704, 83.40388892875082, 83.48281552701442, 83.54021668938796, 83.63349357824497, 83.72677046710196, 83.90614909951927, 84.04965200545311, 84.12857860371673, 84.23620578316711, 84.37970868910097, 84.48016072325466, 84.58061275740833, 84.63083877448518, 84.73129080863887, 84.86044342397933, 84.98242089402311, 85.13309894525365, 85.26225156059411, 85.3698787400445, 85.43445504771472, 85.56360766305518, 85.72863600487909, 85.75016144076918, 85.85778862021957, 85.97259094496664, 86.14479443208725, 86.33852335509793, 86.45332567984502, 86.60400373107555, 86.78338236349286, 86.91253497883332, 87.07038817536056, 87.14213962832747, 87.2067159359977, 87.3071679701514, 87.40762000430509, 87.50089689316209, 87.60852407261247, 87.78072755973308, 87.89552988448015, 87.97445648274378, 88.14665996986439, 88.29016287579825, 88.48389179880894, 88.63456985003947, 88.77807275597331, 88.92875080720385, 89.05790342254431, 89.17988089258809, 89.31620865322523, 89.48123699504916, 89.59603931979623, 89.68214106335654, 89.8041185334003, 89.8184688239937, 89.87586998636723, 89.94044629403746, 90.03372318289445, 90.13417521704815, 90.14852550764154, 90.24180239649853, 90.30637870416876, 90.26332783238861, 90.06242376408122], "calls": [0.0, 1.2915261534046065, 2.023390973667217, 2.8198321016000576, 3.587572648346129, 4.527516682212815, 5.3741838272225015, 6.500681638803185, 7.462151108559949, 8.667575518404249, 9.736672167611395, 10.820119107411925, 11.745712850685226, 12.606730286288299, 14.135036234483747, 15.132381430723974, 16.129726626964196, 16.93334290019373, 18.08136614766449, 19.179163378058405, 20.291310899045705, 21.36040754825285, 22.264475855636075, 23.204419889502763, 23.936284709765374, 24.75425127358829, 25.744421324531825, 26.63413934132166, 27.437755614551197, 28.38487479371457, 29.20284135753749, 30.228887134964484, 30.989452536413864, 32.23075267274162, 33.07024467245462, 34.110640740475, 34.95730788548468, 35.92595250053814, 36.930472842075055, 37.65516251704097, 38.62380713209442, 39.56375116596111, 40.68307383224511, 41.644543302001864, 42.84279256654947, 43.58183253210878, 44.45720025830523, 45.389969146875224, 46.372964052522065, 47.06895314630121, 48.01607232546459, 48.963191504627964, 49.781158068450885, 50.91483102532827, 51.82607447800818, 52.787543947764945, 53.69161225514817, 54.87551122910239, 55.82980555356246, 56.76974958742914, 57.70251847599914, 58.46308387744852, 59.5895816890292, 60.586926885269435, 61.648848389179875, 62.71076989309034, 63.751165961110715, 64.85613833680132, 65.9898112936787, 66.95845590873215, 67.90557508789553, 69.22580182248691, 70.32359905288082, 71.2635430867475, 72.26088828298774, 73.3084594963048, 74.48518332496234, 75.48970366649924, 76.80275525579393, 78.00817966563824, 79.12750233192222, 80.01722034871206, 81.08631699791921, 82.16258879242304, 83.02360622802611, 84.02095142426634, 85.14744923584703, 86.38874937217479, 87.40762000430509, 88.49824208940231, 89.4310109779723, 90.42835617421254, 91.23914759273876, 92.60242519911029, 93.64999641242736, 94.81236995049149, 96.06802037741264, 97.13711702661979, 97.97660902633278, 98.76587500896893, 100.0], "rescued": [0, 66, 106, 158, 205, 259, 297, 361, 415, 467, 513, 561, 617, 664, 724, 741, 764, 778, 799, 821, 841, 867, 881, 895, 905, 914, 934, 947, 960, 977, 996, 1009, 1027, 1057, 1072, 1091, 1112, 1130, 1143, 1153, 1171, 1189, 1217, 1241, 1260, 1276, 1298, 1317, 1340, 1351, 1371, 1393, 1415, 1440, 1462, 1481, 1499, 1529, 1553, 1570, 1593, 1611, 1640, 1669, 1694, 1718, 1750, 1775, 1805, 1823, 1844, 1875, 1901, 1921, 1944, 1977, 2005, 2025, 2064, 2099, 2132, 2161, 2196, 2228, 2254, 2283, 2315, 2356, 2388, 2422, 2454, 2473, 2498, 2540, 2574, 2613, 2649, 2688, 2726, 2753, 2791], "harmed": [0, 11, 17, 20, 25, 36, 41, 49, 58, 68, 74, 80, 84, 87, 94, 97, 101, 104, 106, 109, 114, 117, 121, 123, 125, 126, 128, 131, 131, 133, 135, 139, 140, 143, 146, 152, 156, 157, 159, 161, 166, 171, 174, 178, 186, 187, 189, 194, 203, 207, 213, 217, 222, 226, 230, 234, 243, 255, 256, 270, 278, 280, 285, 287, 296, 299, 306, 313, 321, 329, 341, 358, 370, 377, 385, 394, 406, 415, 430, 445, 451, 459, 474, 485, 493, 505, 518, 536, 552, 574, 589, 606, 623, 656, 677, 702, 736, 762, 791, 824, 890], "exact_score": [76.42247255506925, 76.71665351223362, 77.05388534117816, 77.46286862308962, 77.87185190500108, 78.13733228097868, 78.48173925521992, 78.79744564827438, 79.12032718662553, 79.40015785319653, 79.66563822917414, 79.95981918633852, 80.36880246825, 80.69885915189784, 80.92128865609529, 81.04326612613906, 81.15806845088613, 81.25852048503982, 81.3804979550836, 81.50247542512736, 81.63880318576452, 81.74643036521489, 81.8468823993686, 81.9186338523355, 82.03343617708259, 82.09801248475281, 82.24869053598336, 82.32044198895028, 82.45676974958744, 82.57157207433451, 82.62897323670805, 82.78682643323528, 82.96620506565257, 83.07383224510296, 83.15993398866327, 83.30343689459711, 83.4110640740475, 83.47564038171772, 83.56174212527804, 83.67654445002512, 83.81287221066228, 83.95637511659612, 84.09987802252996, 84.13575374901342, 84.30078209083734, 84.41558441558442, 84.55908732151826, 84.6236636291885, 84.73129080863887, 84.86044342397933, 85.01112147520988, 85.13309894525365, 85.2694267058908, 85.40575446652795, 85.49185621008826, 85.58513309894525, 85.74298629547249, 85.75733658606588, 85.89366434670302, 86.06586783382365, 86.25242161153763, 86.35287364569133, 86.50355169692186, 86.6542297481524, 86.8336083805697, 86.95558585061349, 87.07038817536056, 87.14213962832747, 87.2138910812944, 87.29281767955801, 87.35739398722825, 87.47219631197531, 87.5941737820191, 87.73767668795294, 87.86682930329339, 87.90270502977687, 88.02468249982063, 88.16101026045777, 88.29016287579825, 88.48389179880894, 88.62021955944608, 88.74219702948984, 88.93592595250054, 89.05072827724761, 89.158355456698, 89.30185836263185, 89.4310109779723, 89.48123699504916, 89.67496591805984, 89.76824280691685, 89.78259309751022, 89.86151969577384, 89.904570567554, 89.99067231111431, 90.07677405467462, 90.15570065293824, 90.14135036234484, 90.21310181531177, 90.29920355887207, 90.27050297768531, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 84.86089029390239, "threshold_area100": 84.73412499103108, "threshold_area25": 79.90528808208369}, "selected_budget": 99, "selected": {"val": {"score": 90.73331423660842, "calls": 98.99742194213692, "rescued": 1435, "harmed": 376}, "test": {"score": 90.26332783238861, "calls": 98.76587500896893, "rescued": 2753, "harmed": 824}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.4648289680480957, "operator": "ge"}, {"budget": 0.02, "threshold": 1.1971707344055176, "operator": "ge"}, {"budget": 0.03, "threshold": 0.9614917039871216, "operator": "ge"}, {"budget": 0.04, "threshold": 0.8335264921188354, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7179217338562012, "operator": "ge"}, {"budget": 0.06, "threshold": 0.6184903383255005, "operator": "ge"}, {"budget": 0.07, "threshold": 0.523453950881958, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4434037506580353, "operator": "ge"}, {"budget": 0.09, "threshold": 0.35213780403137207, "operator": "ge"}, {"budget": 0.1, "threshold": 0.2602345943450928, "operator": "ge"}, {"budget": 0.11, "threshold": 0.18307530879974365, "operator": "ge"}, {"budget": 0.12, "threshold": 0.10868129879236221, "operator": "ge"}, {"budget": 0.13, "threshold": 0.03171160817146301, "operator": "ge"}, {"budget": 0.14, "threshold": -0.04638013243675232, "operator": "ge"}, {"budget": 0.15, "threshold": -0.12109357118606567, "operator": "ge"}, {"budget": 0.16, "threshold": -0.18175065517425537, "operator": "ge"}, {"budget": 0.17, "threshold": -0.25840985774993896, "operator": "ge"}, {"budget": 0.18, "threshold": -0.3412567675113678, "operator": "ge"}, {"budget": 0.19, "threshold": -0.42545515298843384, "operator": "ge"}, {"budget": 0.2, "threshold": -0.5048110485076904, "operator": "ge"}, {"budget": 0.21, "threshold": -0.5908660292625427, "operator": "ge"}, {"budget": 0.22, "threshold": -0.6799954175949097, "operator": "ge"}, {"budget": 0.23, "threshold": -0.763521671295166, "operator": "ge"}, {"budget": 0.24, "threshold": -0.8597939014434814, "operator": "ge"}, {"budget": 0.25, "threshold": -0.9425904154777527, "operator": "ge"}, {"budget": 0.26, "threshold": -1.0270155668258667, "operator": "ge"}, {"budget": 0.27, "threshold": -1.0926637649536133, "operator": "ge"}, {"budget": 0.28, "threshold": -1.1589020490646362, "operator": "ge"}, {"budget": 0.29, "threshold": -1.240799069404602, "operator": "ge"}, {"budget": 0.3, "threshold": -1.3220716714859009, "operator": "ge"}, {"budget": 0.31, "threshold": -1.3909763097763062, "operator": "ge"}, {"budget": 0.32, "threshold": -1.4573379755020142, "operator": "ge"}, {"budget": 0.33, "threshold": -1.5168026685714722, "operator": "ge"}, {"budget": 0.34, "threshold": -1.5850410461425781, "operator": "ge"}, {"budget": 0.35, "threshold": -1.6482374668121338, "operator": "ge"}, {"budget": 0.36, "threshold": -1.706092119216919, "operator": "ge"}, {"budget": 0.37, "threshold": -1.7590047121047974, "operator": "ge"}, {"budget": 0.38, "threshold": -1.8137919902801514, "operator": "ge"}, {"budget": 0.39, "threshold": -1.8607361316680908, "operator": "ge"}, {"budget": 0.4, "threshold": -1.908111572265625, "operator": "ge"}, {"budget": 0.41, "threshold": -1.9585492610931396, "operator": "ge"}, {"budget": 0.42, "threshold": -2.025007486343384, "operator": "ge"}, {"budget": 0.43, "threshold": -2.0759458541870117, "operator": "ge"}, {"budget": 0.44, "threshold": -2.130749225616455, "operator": "ge"}, {"budget": 0.45, "threshold": -2.1817703247070312, "operator": "ge"}, {"budget": 0.46, "threshold": -2.23649263381958, "operator": "ge"}, {"budget": 0.47, "threshold": -2.3059401512145996, "operator": "ge"}, {"budget": 0.48, "threshold": -2.3494791984558105, "operator": "ge"}, {"budget": 0.49, "threshold": -2.411576747894287, "operator": "ge"}, {"budget": 0.5, "threshold": -2.463501453399658, "operator": "ge"}, {"budget": 0.51, "threshold": -2.511265277862549, "operator": "ge"}, {"budget": 0.52, "threshold": -2.5496037006378174, "operator": "ge"}, {"budget": 0.53, "threshold": -2.6092281341552734, "operator": "ge"}, {"budget": 0.54, "threshold": -2.674391746520996, "operator": "ge"}, {"budget": 0.55, "threshold": -2.7383713722229004, "operator": "ge"}, {"budget": 0.56, "threshold": -2.7953569889068604, "operator": "ge"}, {"budget": 0.57, "threshold": -2.837301015853882, "operator": "ge"}, {"budget": 0.58, "threshold": -2.878894567489624, "operator": "ge"}, {"budget": 0.59, "threshold": -2.9241831302642822, "operator": "ge"}, {"budget": 0.6, "threshold": -2.9622511863708496, "operator": "ge"}, {"budget": 0.61, "threshold": -3.0077786445617676, "operator": "ge"}, {"budget": 0.62, "threshold": -3.0562005043029785, "operator": "ge"}, {"budget": 0.63, "threshold": -3.1068918704986572, "operator": "ge"}, {"budget": 0.64, "threshold": -3.1535422801971436, "operator": "ge"}, {"budget": 0.65, "threshold": -3.1984000205993652, "operator": "ge"}, {"budget": 0.66, "threshold": -3.2430596351623535, "operator": "ge"}, {"budget": 0.67, "threshold": -3.285109043121338, "operator": "ge"}, {"budget": 0.68, "threshold": -3.342445135116577, "operator": "ge"}, {"budget": 0.69, "threshold": -3.3901031017303467, "operator": "ge"}, {"budget": 0.7, "threshold": -3.432929515838623, "operator": "ge"}, {"budget": 0.71, "threshold": -3.478071689605713, "operator": "ge"}, {"budget": 0.72, "threshold": -3.519418716430664, "operator": "ge"}, {"budget": 0.73, "threshold": -3.5730507373809814, "operator": "ge"}, {"budget": 0.74, "threshold": -3.6110408306121826, "operator": "ge"}, {"budget": 0.75, "threshold": -3.6572113037109375, "operator": "ge"}, {"budget": 0.76, "threshold": -3.6986184120178223, "operator": "ge"}, {"budget": 0.77, "threshold": -3.738680601119995, "operator": "ge"}, {"budget": 0.78, "threshold": -3.7752060890197754, "operator": "ge"}, {"budget": 0.79, "threshold": -3.821441888809204, "operator": "ge"}, {"budget": 0.8, "threshold": -3.8797261714935303, "operator": "ge"}, {"budget": 0.81, "threshold": -3.9237473011016846, "operator": "ge"}, {"budget": 0.82, "threshold": -3.9857094287872314, "operator": "ge"}, {"budget": 0.83, "threshold": -4.0243353843688965, "operator": "ge"}, {"budget": 0.84, "threshold": -4.067356109619141, "operator": "ge"}, {"budget": 0.85, "threshold": -4.121035575866699, "operator": "ge"}, {"budget": 0.86, "threshold": -4.168330669403076, "operator": "ge"}, {"budget": 0.87, "threshold": -4.227099418640137, "operator": "ge"}, {"budget": 0.88, "threshold": -4.289048194885254, "operator": "ge"}, {"budget": 0.89, "threshold": -4.353620529174805, "operator": "ge"}, {"budget": 0.9, "threshold": -4.40568208694458, "operator": "ge"}, {"budget": 0.91, "threshold": -4.489944934844971, "operator": "ge"}, {"budget": 0.92, "threshold": -4.5738983154296875, "operator": "ge"}, {"budget": 0.93, "threshold": -4.652194976806641, "operator": "ge"}, {"budget": 0.94, "threshold": -4.73099946975708, "operator": "ge"}, {"budget": 0.95, "threshold": -4.814966678619385, "operator": "ge"}, {"budget": 0.96, "threshold": -4.922561168670654, "operator": "ge"}, {"budget": 0.97, "threshold": -5.067933559417725, "operator": "ge"}, {"budget": 0.98, "threshold": -5.256481170654297, "operator": "ge"}, {"budget": 0.99, "threshold": -5.523816108703613, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.93812661128617, 76.21025494127757, 76.69722142652535, 76.98367230020051, 77.3990260670295, 77.68547694070466, 78.10083070753365, 78.41592666857635, 78.64508736751647, 79.00315095961042, 79.26095674591808, 79.59037525064451, 79.81953594958465, 80.07734173589229, 80.29217989114866, 80.57863076482383, 80.97966198796907, 81.39501575479805, 81.69578917215698, 81.96791750214838, 82.18275565740475, 82.46920653107992, 82.62675451160125, 82.76997994843884, 83.08507590948152, 83.32855915210541, 83.52907476367803, 83.72959037525064, 83.93010598682325, 84.21655686049843, 84.50300773417358, 84.54597536522486, 84.76081352048124, 84.90403895731882, 85.11887711257519, 85.27642509309653, 85.37668289888283, 85.5628759667717, 85.66313377255801, 85.79203666571182, 85.89229447149813, 86.04984245201948, 86.20739043254082, 86.32197078201088, 86.40790604411343, 86.53680893726725, 86.73732454883987, 86.95216270409625, 87.1383557719851, 87.38183901460899, 87.52506445144658, 87.5823546261816, 87.6539673446004, 87.63964480091664, 87.75422515038672, 87.78287023775422, 87.9260956745918, 88.04067602406187, 88.18390146089945, 88.25551417931825, 88.28415926668576, 88.4417072472071, 88.48467487825837, 88.65654540246346, 88.67086794614724, 88.77112575193354, 88.85706101403609, 88.8857061014036, 88.95731881982239, 89.04325408192494, 89.14351188771126, 89.21512460613005, 89.35835004296763, 89.37267258665138, 89.44428530507018, 89.47293039243769, 89.5445431108565, 89.573188198224, 89.60183328559151, 89.68776854769406, 89.7450587224291, 89.83099398453164, 89.8166714408479, 89.90260670295044, 89.95989687768548, 90.03150959610426, 90.04583213978803, 90.10312231452305, 90.23202520767688, 90.24634775136064, 90.30363792609567, 90.34660555714696, 90.37525064451447, 90.3609281008307, 90.33228301346318, 90.3609281008307, 90.4325408192495, 90.44686336293324, 90.4325408192495, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 28, 50, 85, 110, 142, 165, 196, 222, 243, 272, 296, 325, 344, 368, 390, 416, 447, 479, 503, 527, 544, 571, 586, 608, 632, 655, 675, 694, 713, 737, 761, 772, 788, 802, 822, 838, 854, 873, 885, 898, 912, 927, 945, 959, 972, 988, 1005, 1023, 1038, 1059, 1069, 1081, 1091, 1100, 1110, 1116, 1130, 1142, 1153, 1163, 1170, 1182, 1189, 1203, 1208, 1220, 1228, 1237, 1246, 1255, 1264, 1273, 1284, 1288, 1294, 1300, 1308, 1313, 1317, 1327, 1336, 1344, 1348, 1359, 1366, 1374, 1382, 1388, 1397, 1399, 1403, 1410, 1415, 1419, 1422, 1428, 1435, 1439, 1443, 1449], "harmed": [0, 2, 5, 6, 11, 14, 17, 19, 23, 28, 32, 38, 44, 47, 53, 60, 66, 69, 72, 75, 80, 82, 89, 93, 105, 107, 113, 119, 124, 129, 133, 137, 145, 146, 150, 155, 160, 169, 175, 180, 184, 191, 195, 202, 208, 215, 222, 225, 228, 230, 234, 234, 242, 247, 257, 259, 263, 267, 271, 272, 277, 282, 283, 287, 289, 293, 298, 300, 307, 311, 314, 316, 320, 321, 324, 325, 329, 332, 335, 337, 341, 346, 348, 353, 358, 361, 364, 371, 373, 373, 374, 374, 378, 381, 386, 391, 395, 397, 400, 405, 406], "exact_score": [75.56574047550845, 75.93812661128617, 76.21025494127757, 76.69722142652535, 76.98367230020051, 77.3990260670295, 77.68547694070466, 78.10083070753365, 78.41592666857635, 78.64508736751647, 79.00315095961042, 79.26095674591808, 79.59037525064451, 79.81953594958465, 80.07734173589229, 80.29217989114866, 80.57863076482383, 80.97966198796907, 81.39501575479805, 81.69578917215698, 81.96791750214838, 82.18275565740475, 82.46920653107992, 82.62675451160125, 82.76997994843884, 83.08507590948152, 83.32855915210541, 83.52907476367803, 83.72959037525064, 83.93010598682325, 84.21655686049843, 84.50300773417358, 84.54597536522486, 84.76081352048124, 84.90403895731882, 85.11887711257519, 85.27642509309653, 85.37668289888283, 85.5628759667717, 85.66313377255801, 85.79203666571182, 85.89229447149813, 86.04984245201948, 86.20739043254082, 86.32197078201088, 86.40790604411343, 86.53680893726725, 86.73732454883987, 86.95216270409625, 87.1383557719851, 87.38183901460899, 87.52506445144658, 87.5823546261816, 87.6539673446004, 87.63964480091664, 87.75422515038672, 87.78287023775422, 87.9260956745918, 88.04067602406187, 88.18390146089945, 88.25551417931825, 88.28415926668576, 88.4417072472071, 88.48467487825837, 88.65654540246346, 88.67086794614724, 88.77112575193354, 88.85706101403609, 88.8857061014036, 88.95731881982239, 89.04325408192494, 89.14351188771126, 89.21512460613005, 89.35835004296763, 89.37267258665138, 89.44428530507018, 89.47293039243769, 89.5445431108565, 89.573188198224, 89.60183328559151, 89.68776854769406, 89.7450587224291, 89.83099398453164, 89.8166714408479, 89.90260670295044, 89.95989687768548, 90.03150959610426, 90.04583213978803, 90.10312231452305, 90.23202520767688, 90.24634775136064, 90.30363792609567, 90.34660555714696, 90.37525064451447, 90.3609281008307, 90.33228301346318, 90.3609281008307, 90.4325408192495, 90.44686336293324, 90.4325408192495, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.85990464000362, "threshold_area100": 85.85885133199656, "threshold_area25": 79.5975365224864}, "test": {"score": [76.42247255506925, 76.78122981990386, 76.97495874291455, 77.31936571715578, 77.67812298199038, 78.07993111860515, 78.409987802253, 78.68981846882399, 78.94812369950492, 79.29970581904284, 79.6082370668006, 79.86654229748153, 80.20377412642607, 80.61275740833752, 80.97151467317214, 81.22264475855636, 81.40202339097367, 81.62445289517113, 81.88275812585205, 82.1195379206429, 82.26304082657674, 82.52852120255434, 82.70789983497167, 83.07383224510296, 83.44693980053096, 83.64066872354165, 83.82004735595896, 83.99225084307957, 84.24338092846381, 84.41558441558442, 84.7097653727488, 84.91784458635287, 85.01112147520988, 85.21920068881394, 85.4559804836048, 85.58513309894525, 85.70711056898902, 85.9151897825931, 86.14479443208725, 86.30264762861448, 86.33852335509793, 86.38874937217479, 86.60400373107555, 86.71880605582263, 86.89100954294324, 86.97711128650354, 87.13496448303079, 87.32151826074478, 87.42914544019517, 87.51524718375548, 87.56547320083232, 87.60852407261247, 87.65157494439264, 87.78790270502978, 87.91705532037024, 87.99598191863386, 88.1394848245677, 88.21123627753462, 88.23993685872139, 88.33321374757838, 88.42649063643539, 88.56281839707253, 88.64174499533615, 88.65609528592955, 88.72784673889646, 88.74937217478654, 88.80677333716008, 88.83547391834685, 88.82829877305016, 88.89287508072039, 88.95027624309392, 88.971801678984, 89.03637798665423, 89.12965487551124, 89.20140632847816, 89.27315778144508, 89.3377340891153, 89.41666068737892, 89.44536126856569, 89.49558728564253, 89.531463012126, 89.58886417449953, 89.65344048216977, 89.68931620865322, 89.72519193513668, 89.82564396929038, 89.86869484107054, 89.87586998636723, 89.91174571285069, 89.96197172992754, 90.03372318289445, 90.01937289230108, 90.05524861878453, 90.10547463586138, 90.13417521704815, 90.0982994905647, 90.12700007175145, 90.06959890937792, 90.04807347348785, 90.09112434526799, 90.06242376408122], "calls": [0.0, 1.0403960680203774, 1.7937863241730645, 3.0422616057975174, 3.9104541866972804, 4.871923656454044, 5.862093707397575, 6.794862595967569, 7.777857501614408, 8.861304441414939, 9.829949056468394, 10.734017363851619, 11.673961397718303, 12.664131448661836, 13.934132166176367, 14.917127071823206, 15.6633421826792, 16.538709908875653, 17.47865394274234, 18.447298557795797, 19.329841429288944, 20.341536916122553, 21.32453182176939, 22.40080361627323, 23.606228026117527, 24.72555069240152, 25.70854559804836, 26.63413934132166, 27.416230178661117, 28.535552844945112, 29.76967783597618, 30.7670230322164, 31.678266484896316, 32.48905790342255, 33.51510368084954, 34.670302073616995, 35.674822415153905, 36.52866470546029, 37.4399081581402, 38.42290306378704, 39.18346846523642, 40.06601133672957, 41.43646408839779, 42.45533472052809, 43.574657386812085, 44.45720025830523, 45.38279400157853, 46.65279471909306, 47.29855779579537, 48.50398220563966, 49.51567769247327, 50.31211882040611, 51.022458204778644, 52.12025543517256, 53.232402956159866, 54.4952285283777, 55.47822343402454, 56.23161369017723, 56.963478510439835, 57.8316710913396, 58.57071105689891, 59.496304800172204, 60.47212456052235, 61.56274664561957, 62.53856640596972, 63.442634713352945, 64.43997990959318, 65.39427423405324, 66.74320154983138, 67.90557508789553, 68.86704455765229, 69.8859151897826, 70.74693262538567, 71.96670732582334, 72.85642534261318, 74.09055033364426, 75.02331922221424, 76.02066441845447, 76.84580612757408, 78.036880246825, 79.3284064002296, 80.47642964770037, 81.87558298055536, 82.6720241084882, 83.61914328765158, 84.72411566334218, 85.75733658606588, 86.8407835258664, 87.91705532037024, 88.971801678984, 89.84716940518047, 91.13152041328837, 92.07863959245175, 93.04010906220851, 93.93700222429504, 94.8984716940518, 95.79536485613833, 96.7496591805984, 97.85463155628902, 98.87350218841931, 100.0], "rescued": [0, 51, 80, 135, 190, 248, 302, 352, 397, 452, 507, 554, 608, 670, 731, 778, 816, 856, 902, 945, 979, 1026, 1066, 1127, 1190, 1230, 1266, 1308, 1346, 1384, 1439, 1474, 1498, 1533, 1576, 1604, 1638, 1677, 1715, 1743, 1763, 1783, 1829, 1853, 1884, 1910, 1938, 1974, 1993, 2022, 2042, 2060, 2076, 2106, 2136, 2160, 2191, 2208, 2221, 2241, 2255, 2277, 2298, 2316, 2333, 2347, 2365, 2377, 2395, 2414, 2427, 2438, 2452, 2474, 2491, 2510, 2523, 2537, 2544, 2559, 2573, 2585, 2603, 2612, 2621, 2639, 2651, 2666, 2676, 2690, 2704, 2712, 2722, 2733, 2744, 2745, 2754, 2758, 2768, 2783, 2791], "harmed": [0, 1, 3, 10, 15, 17, 25, 36, 45, 51, 63, 74, 81, 86, 97, 109, 122, 131, 141, 151, 165, 175, 190, 200, 211, 224, 235, 253, 256, 270, 284, 290, 301, 307, 317, 327, 344, 354, 360, 366, 381, 394, 410, 418, 425, 439, 445, 455, 459, 476, 489, 501, 511, 522, 534, 547, 558, 565, 574, 581, 582, 585, 595, 611, 618, 629, 639, 647, 666, 676, 681, 689, 694, 703, 710, 719, 723, 726, 729, 737, 746, 750, 759, 763, 767, 771, 777, 791, 796, 803, 807, 817, 822, 826, 833, 839, 844, 856, 869, 878, 890], "exact_score": [76.42247255506925, 76.75252923871709, 77.05388534117816, 77.29784028126569, 77.72117385377054, 78.12298199038531, 78.45303867403315, 78.73286934060415, 79.04140058836192, 79.3355815455263, 79.67281337447083, 79.9741694769319, 80.34010188706321, 80.71320944249122, 80.97868981846884, 81.25134533974312, 81.48094998923729, 81.77513094640166, 82.0262610317859, 82.2056396642032, 82.44241945899404, 82.69354954437827, 82.92315419387243, 83.2316854416302, 83.54021668938796, 83.67654445002512, 83.89179880892588, 84.09987802252996, 84.31513238143073, 84.5375618856282, 84.76716653512234, 84.99677118461649, 85.11157350936357, 85.3627035947478, 85.50620650068164, 85.66405969720887, 85.76451173136256, 86.02281696204348, 86.23807132094426, 86.33852335509793, 86.41027480806487, 86.51072684221856, 86.67575518404249, 86.82643323527302, 86.91971012413, 87.07038817536056, 87.24976680777786, 87.37174427782163, 87.47937145727201, 87.55829805553563, 87.57264834612901, 87.65157494439264, 87.76637726913971, 87.88835473918347, 87.945755901557, 88.06773337160078, 88.16818540575447, 88.23993685872139, 88.36908947406185, 88.5197675252924, 88.59869412355602, 88.66327043122624, 88.691971012413, 88.76372246537993, 88.77089761067663, 88.82112362775347, 88.81394848245677, 88.84264906364353, 88.8856999354237, 88.94310109779724, 88.96462653368731, 89.04355313195093, 89.12965487551124, 89.21575661907154, 89.28033292674176, 89.3377340891153, 89.40948554208224, 89.44536126856569, 89.48841214034584, 89.51711272153261, 89.58886417449953, 89.624739900983, 89.65344048216977, 89.69649135394992, 89.77541795221353, 89.83999425988375, 89.84716940518047, 89.87586998636723, 89.91892085814737, 89.96914687522423, 90.04089832819116, 90.01219774700438, 90.04089832819116, 90.0982994905647, 90.12700007175145, 90.09112434526799, 90.12700007175145, 90.06959890937792, 90.05524861878453, 90.0982994905647, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.16783989499092, "threshold_area100": 86.1060845232116, "threshold_area25": 80.24108488196886}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6724159717559814, "operator": "ge"}, {"budget": 0.02, "threshold": 0.536537766456604, "operator": "ge"}, {"budget": 0.03, "threshold": 0.4589134454727173, "operator": "ge"}, {"budget": 0.04, "threshold": 0.390231728553772, "operator": "ge"}, {"budget": 0.05, "threshold": 0.33765289187431335, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2904309332370758, "operator": "ge"}, {"budget": 0.07, "threshold": 0.2452508509159088, "operator": "ge"}, {"budget": 0.08, "threshold": 0.19713547825813293, "operator": "ge"}, {"budget": 0.09, "threshold": 0.1589343547821045, "operator": "ge"}, {"budget": 0.1, "threshold": 0.12207111716270447, "operator": "ge"}, {"budget": 0.11, "threshold": 0.08828845620155334, "operator": "ge"}, {"budget": 0.12, "threshold": 0.051741182804107666, "operator": "ge"}, {"budget": 0.13, "threshold": 0.018142282962799072, "operator": "ge"}, {"budget": 0.14, "threshold": -0.029778093099594116, "operator": "ge"}, {"budget": 0.15, "threshold": -0.06830748915672302, "operator": "ge"}, {"budget": 0.16, "threshold": -0.10908973217010498, "operator": "ge"}, {"budget": 0.17, "threshold": -0.15223178267478943, "operator": "ge"}, {"budget": 0.18, "threshold": -0.20306932926177979, "operator": "ge"}, {"budget": 0.19, "threshold": -0.25675615668296814, "operator": "ge"}, {"budget": 0.2, "threshold": -0.3106979727745056, "operator": "ge"}, {"budget": 0.21, "threshold": -0.36867737770080566, "operator": "ge"}, {"budget": 0.22, "threshold": -0.42171210050582886, "operator": "ge"}, {"budget": 0.23, "threshold": -0.49485933780670166, "operator": "ge"}, {"budget": 0.24, "threshold": -0.5583021640777588, "operator": "ge"}, {"budget": 0.25, "threshold": -0.6307693123817444, "operator": "ge"}, {"budget": 0.26, "threshold": -0.7008493542671204, "operator": "ge"}, {"budget": 0.27, "threshold": -0.7577376961708069, "operator": "ge"}, {"budget": 0.28, "threshold": -0.8196473717689514, "operator": "ge"}, {"budget": 0.29, "threshold": -0.889778196811676, "operator": "ge"}, {"budget": 0.3, "threshold": -0.9439274072647095, "operator": "ge"}, {"budget": 0.31, "threshold": -1.0113654136657715, "operator": "ge"}, {"budget": 0.32, "threshold": -1.0843364000320435, "operator": "ge"}, {"budget": 0.33, "threshold": -1.132889747619629, "operator": "ge"}, {"budget": 0.34, "threshold": -1.20293390750885, "operator": "ge"}, {"budget": 0.35, "threshold": -1.2694447040557861, "operator": "ge"}, {"budget": 0.36, "threshold": -1.3259363174438477, "operator": "ge"}, {"budget": 0.37, "threshold": -1.3829983472824097, "operator": "ge"}, {"budget": 0.38, "threshold": -1.4544808864593506, "operator": "ge"}, {"budget": 0.39, "threshold": -1.5013635158538818, "operator": "ge"}, {"budget": 0.4, "threshold": -1.5472129583358765, "operator": "ge"}, {"budget": 0.41, "threshold": -1.5855515003204346, "operator": "ge"}, {"budget": 0.42, "threshold": -1.629087209701538, "operator": "ge"}, {"budget": 0.43, "threshold": -1.6823476552963257, "operator": "ge"}, {"budget": 0.44, "threshold": -1.7288293838500977, "operator": "ge"}, {"budget": 0.45, "threshold": -1.7851964235305786, "operator": "ge"}, {"budget": 0.46, "threshold": -1.8436214923858643, "operator": "ge"}, {"budget": 0.47, "threshold": -1.9011263847351074, "operator": "ge"}, {"budget": 0.48, "threshold": -1.956665277481079, "operator": "ge"}, {"budget": 0.49, "threshold": -2.009483575820923, "operator": "ge"}, {"budget": 0.5, "threshold": -2.0530076026916504, "operator": "ge"}, {"budget": 0.51, "threshold": -2.1077053546905518, "operator": "ge"}, {"budget": 0.52, "threshold": -2.1575276851654053, "operator": "ge"}, {"budget": 0.53, "threshold": -2.208073854446411, "operator": "ge"}, {"budget": 0.54, "threshold": -2.262211561203003, "operator": "ge"}, {"budget": 0.55, "threshold": -2.3121883869171143, "operator": "ge"}, {"budget": 0.56, "threshold": -2.366569757461548, "operator": "ge"}, {"budget": 0.57, "threshold": -2.42154598236084, "operator": "ge"}, {"budget": 0.58, "threshold": -2.4691379070281982, "operator": "ge"}, {"budget": 0.59, "threshold": -2.5170164108276367, "operator": "ge"}, {"budget": 0.6, "threshold": -2.556135654449463, "operator": "ge"}, {"budget": 0.61, "threshold": -2.597439765930176, "operator": "ge"}, {"budget": 0.62, "threshold": -2.637826442718506, "operator": "ge"}, {"budget": 0.63, "threshold": -2.6829276084899902, "operator": "ge"}, {"budget": 0.64, "threshold": -2.725356340408325, "operator": "ge"}, {"budget": 0.65, "threshold": -2.7688512802124023, "operator": "ge"}, {"budget": 0.66, "threshold": -2.812366247177124, "operator": "ge"}, {"budget": 0.67, "threshold": -2.8594207763671875, "operator": "ge"}, {"budget": 0.68, "threshold": -2.9132988452911377, "operator": "ge"}, {"budget": 0.69, "threshold": -2.9565627574920654, "operator": "ge"}, {"budget": 0.7, "threshold": -3.0010695457458496, "operator": "ge"}, {"budget": 0.71, "threshold": -3.049286127090454, "operator": "ge"}, {"budget": 0.72, "threshold": -3.098834991455078, "operator": "ge"}, {"budget": 0.73, "threshold": -3.138977527618408, "operator": "ge"}, {"budget": 0.74, "threshold": -3.18345046043396, "operator": "ge"}, {"budget": 0.75, "threshold": -3.2356650829315186, "operator": "ge"}, {"budget": 0.76, "threshold": -3.2838523387908936, "operator": "ge"}, {"budget": 0.77, "threshold": -3.328141689300537, "operator": "ge"}, {"budget": 0.78, "threshold": -3.368462085723877, "operator": "ge"}, {"budget": 0.79, "threshold": -3.4228246212005615, "operator": "ge"}, {"budget": 0.8, "threshold": -3.47007155418396, "operator": "ge"}, {"budget": 0.81, "threshold": -3.522144317626953, "operator": "ge"}, {"budget": 0.82, "threshold": -3.569261074066162, "operator": "ge"}, {"budget": 0.83, "threshold": -3.6103286743164062, "operator": "ge"}, {"budget": 0.84, "threshold": -3.6636979579925537, "operator": "ge"}, {"budget": 0.85, "threshold": -3.726275682449341, "operator": "ge"}, {"budget": 0.86, "threshold": -3.780717372894287, "operator": "ge"}, {"budget": 0.87, "threshold": -3.8344974517822266, "operator": "ge"}, {"budget": 0.88, "threshold": -3.884215831756592, "operator": "ge"}, {"budget": 0.89, "threshold": -3.947050094604492, "operator": "ge"}, {"budget": 0.9, "threshold": -4.00892972946167, "operator": "ge"}, {"budget": 0.91, "threshold": -4.074031829833984, "operator": "ge"}, {"budget": 0.92, "threshold": -4.159913539886475, "operator": "ge"}, {"budget": 0.93, "threshold": -4.246121883392334, "operator": "ge"}, {"budget": 0.94, "threshold": -4.335931777954102, "operator": "ge"}, {"budget": 0.95, "threshold": -4.445259094238281, "operator": "ge"}, {"budget": 0.96, "threshold": -4.5724077224731445, "operator": "ge"}, {"budget": 0.97, "threshold": -4.711653709411621, "operator": "ge"}, {"budget": 0.98, "threshold": -4.930403709411621, "operator": "ge"}, {"budget": 0.99, "threshold": -5.23585319519043, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.36780292179891, 76.68289888284158, 77.12689773703809, 77.59954167860211, 77.88599255227729, 78.15812088226869, 78.44457175594385, 78.71670008593526, 78.91721569750787, 79.23231165855056, 79.46147235749069, 79.81953594958465, 80.09166427957605, 80.39243769693498, 80.62159839587511, 80.92237181323402, 81.33772558006301, 81.62417645373819, 81.88198224004583, 82.24004583213978, 82.3689487252936, 82.62675451160125, 82.82727012317388, 82.98481810369522, 83.28559152105413, 83.44313950157549, 83.72959037525064, 83.95875107419077, 84.13062159839588, 84.31681466628473, 84.46004010312231, 84.7035233457462, 84.90403895731882, 84.98997421942136, 85.26210254941277, 85.51990833572042, 85.60584359782297, 85.7347464909768, 85.82068175307934, 85.94958464623316, 86.02119736465195, 86.16442280148954, 86.25035806359209, 86.55113148095101, 86.65138928673733, 86.83758235462618, 86.99513033514752, 87.12403322830134, 87.2386135777714, 87.39616155829275, 87.49641936407906, 87.69693497565167, 87.73990260670294, 87.75422515038672, 87.79719278143799, 87.96906330564308, 88.14093382984818, 88.25551417931825, 88.35577198510455, 88.45602979089085, 88.55628759667717, 88.67086794614724, 88.81409338298482, 88.91435118877112, 88.8857061014036, 88.92867373245488, 89.02893153824118, 89.11486680034373, 89.18647951876252, 89.25809223718132, 89.3297049556001, 89.4013176740189, 89.47293039243769, 89.51589802348897, 89.60183328559151, 89.68776854769406, 89.7450587224291, 89.73073617874535, 89.7450587224291, 89.8166714408479, 89.95989687768548, 90.01718705242051, 90.03150959610426, 90.10312231452305, 90.11744485820681, 90.17473503294184, 90.23202520767688, 90.20338012030936, 90.27499283872815, 90.30363792609567, 90.30363792609567, 90.31796046977942, 90.34660555714696, 90.40389573188197, 90.41821827556574, 90.40389573188197, 90.46118590661702, 90.44686336293324, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 64, 88, 122, 157, 182, 203, 229, 250, 272, 298, 318, 344, 368, 394, 417, 441, 472, 495, 518, 545, 557, 579, 600, 618, 645, 663, 687, 711, 726, 742, 761, 782, 799, 810, 830, 850, 862, 882, 893, 909, 921, 937, 948, 970, 980, 998, 1012, 1029, 1043, 1060, 1073, 1088, 1097, 1107, 1115, 1130, 1144, 1154, 1163, 1175, 1185, 1197, 1210, 1218, 1223, 1229, 1242, 1250, 1259, 1267, 1273, 1283, 1289, 1297, 1306, 1314, 1319, 1324, 1328, 1337, 1349, 1357, 1361, 1370, 1376, 1383, 1389, 1392, 1400, 1403, 1408, 1411, 1416, 1423, 1430, 1433, 1440, 1443, 1449], "harmed": [0, 3, 8, 10, 13, 15, 20, 22, 28, 30, 38, 42, 46, 47, 52, 57, 64, 67, 69, 72, 77, 79, 82, 86, 93, 100, 106, 113, 117, 125, 128, 131, 140, 144, 147, 152, 153, 155, 161, 172, 177, 184, 191, 197, 202, 203, 206, 211, 214, 222, 228, 234, 240, 241, 247, 256, 261, 264, 266, 268, 270, 275, 278, 282, 285, 286, 293, 296, 302, 304, 308, 311, 312, 317, 318, 323, 326, 328, 329, 335, 338, 342, 344, 348, 351, 355, 360, 363, 365, 370, 373, 374, 379, 381, 384, 387, 393, 397, 400, 404, 406], "exact_score": [75.56574047550845, 75.98109424233743, 76.36780292179891, 76.68289888284158, 77.12689773703809, 77.59954167860211, 77.88599255227729, 78.15812088226869, 78.44457175594385, 78.71670008593526, 78.91721569750787, 79.23231165855056, 79.46147235749069, 79.81953594958465, 80.09166427957605, 80.39243769693498, 80.62159839587511, 80.92237181323402, 81.33772558006301, 81.62417645373819, 81.88198224004583, 82.24004583213978, 82.3689487252936, 82.62675451160125, 82.82727012317388, 82.98481810369522, 83.28559152105413, 83.44313950157549, 83.72959037525064, 83.95875107419077, 84.13062159839588, 84.31681466628473, 84.46004010312231, 84.7035233457462, 84.90403895731882, 84.98997421942136, 85.26210254941277, 85.51990833572042, 85.60584359782297, 85.7347464909768, 85.82068175307934, 85.94958464623316, 86.02119736465195, 86.16442280148954, 86.25035806359209, 86.55113148095101, 86.65138928673733, 86.83758235462618, 86.99513033514752, 87.12403322830134, 87.2386135777714, 87.39616155829275, 87.49641936407906, 87.69693497565167, 87.73990260670294, 87.75422515038672, 87.79719278143799, 87.96906330564308, 88.14093382984818, 88.25551417931825, 88.35577198510455, 88.45602979089085, 88.55628759667717, 88.67086794614724, 88.81409338298482, 88.91435118877112, 88.8857061014036, 88.92867373245488, 89.02893153824118, 89.11486680034373, 89.18647951876252, 89.25809223718132, 89.3297049556001, 89.4013176740189, 89.47293039243769, 89.51589802348897, 89.60183328559151, 89.68776854769406, 89.7450587224291, 89.73073617874535, 89.7450587224291, 89.8166714408479, 89.95989687768548, 90.01718705242051, 90.03150959610426, 90.10312231452305, 90.11744485820681, 90.17473503294184, 90.23202520767688, 90.20338012030936, 90.27499283872815, 90.30363792609567, 90.30363792609567, 90.31796046977942, 90.34660555714696, 90.40389573188197, 90.41821827556574, 90.40389573188197, 90.46118590661702, 90.44686336293324, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.90087322796985, "threshold_area100": 85.89981380693213, "threshold_area25": 79.61329132053851}, "test": {"score": [76.42247255506925, 76.86015641816746, 77.22608882829877, 77.52744493075984, 77.87185190500108, 78.18038315275885, 78.45303867403315, 78.72569419530745, 79.12032718662553, 79.40733299849322, 79.65846308387745, 79.8880677333716, 80.29705101528306, 80.58405682715075, 80.97151467317214, 81.22264475855636, 81.4594245533472, 81.60292745928105, 81.87558298055536, 82.09083733945613, 82.32044198895028, 82.54287149314774, 82.74377556145512, 83.05948195450958, 83.31778718519051, 83.56174212527804, 83.74829590299204, 83.94919997129942, 84.20033005668365, 84.45863528736457, 84.6236636291885, 84.73846595393557, 84.98242089402311, 85.14027409055034, 85.3627035947478, 85.58513309894525, 85.73581115017579, 85.97976609026333, 86.17349501327402, 86.24524646624094, 86.47485111573509, 86.5681280045921, 86.67575518404249, 86.88383439764655, 86.99863672239363, 87.08473846595393, 87.2138910812944, 87.27846738896463, 87.3932697137117, 87.45784602138194, 87.63722465379924, 87.71615125206286, 87.78072755973308, 87.93858075626032, 87.945755901557, 88.01750735452393, 88.14665996986439, 88.25428714931478, 88.26863743990816, 88.38343976465524, 88.46954150821554, 88.56281839707253, 88.61304441414939, 88.67762072181962, 88.6847958671163, 88.72067159359977, 88.82829877305016, 88.90722537131377, 88.97897682428069, 89.02202769606085, 89.08660400373107, 89.158355456698, 89.17270574729139, 89.23728205496163, 89.33055894381862, 89.33055894381862, 89.39513525148884, 89.43818612326899, 89.48841214034584, 89.56733873860946, 89.624739900983, 89.71801678984, 89.72519193513668, 89.77541795221353, 89.84716940518047, 89.91174571285069, 89.97632202052091, 90.00502260170768, 90.06959890937792, 90.0982994905647, 90.10547463586138, 90.10547463586138, 90.06242376408122, 90.13417521704815, 90.11264978115807, 90.12700007175145, 90.13417521704815, 90.12700007175145, 90.09112434526799, 90.06959890937792, 90.06242376408122], "calls": [0.0, 1.33457702518476, 2.274521059051446, 3.0135610246107487, 3.9678553490708186, 4.836047929970582, 5.754466527947192, 6.6370093994403385, 7.799382937504484, 8.760852407261247, 9.765372748798164, 10.67661620147808, 11.767238286575303, 12.649781158068452, 14.056109636220132, 14.917127071823206, 15.749443926239506, 16.545885054172345, 17.47865394274234, 18.468823993685874, 19.50922006170625, 20.441988950276244, 21.202554351725624, 22.515605941020304, 23.469900265480376, 24.52464662409414, 25.464590657960823, 26.311257802970513, 27.29425270861735, 28.506852263758343, 29.20284135753749, 30.25041257085456, 31.4199612542154, 32.173351510368086, 33.285499031355386, 34.49092344119968, 35.610246107483675, 36.686517901987514, 37.96369376479874, 39.03279041400589, 39.97273444787257, 40.704599268135176, 41.544091267848174, 42.64906364353878, 43.52443136973524, 44.55765229245892, 45.74155126641315, 46.738896462653365, 47.59991389825644, 48.51115735093636, 49.4008753677262, 50.2331922221425, 51.05833393126211, 52.012628255722184, 52.93104685369878, 53.94274234053239, 54.90421181028916, 56.06658534835331, 57.01370452751669, 57.92494798019659, 58.69986367223936, 59.38867762072182, 60.28557078280835, 61.32596685082873, 62.13675826935495, 63.05517686733156, 64.00229604649495, 65.04269211451532, 66.28399225084308, 67.22393628470977, 68.24998206213677, 69.51998277965129, 70.7254071894956, 71.60795006098873, 72.69857214608597, 73.8681208294468, 75.0735452392911, 76.23591877735524, 77.21891368300207, 78.71134390471407, 79.65846308387745, 80.84236205783168, 81.9258089976322, 82.72225012556504, 83.88462366362918, 85.23355097940734, 86.33134820980125, 87.23541651718448, 88.01750735452393, 89.00767740546746, 89.85434455047715, 90.76558800315706, 91.85621008825429, 92.91813159216474, 93.77197388247112, 94.87694625816174, 95.98191863385233, 96.77118461648848, 97.85463155628902, 98.85197675252924, 100.0], "rescued": [0, 69, 130, 173, 228, 279, 325, 367, 430, 479, 524, 561, 622, 667, 731, 776, 817, 847, 892, 934, 977, 1014, 1048, 1110, 1155, 1201, 1239, 1276, 1321, 1374, 1405, 1436, 1484, 1513, 1553, 1595, 1625, 1667, 1706, 1734, 1776, 1798, 1822, 1862, 1887, 1906, 1943, 1966, 1988, 2007, 2041, 2062, 2078, 2108, 2124, 2147, 2171, 2198, 2212, 2233, 2249, 2268, 2283, 2303, 2312, 2332, 2356, 2372, 2393, 2408, 2426, 2443, 2454, 2468, 2487, 2502, 2521, 2533, 2545, 2564, 2579, 2595, 2603, 2615, 2631, 2649, 2661, 2670, 2683, 2698, 2705, 2713, 2719, 2732, 2737, 2745, 2753, 2760, 2769, 2779, 2791], "harmed": [0, 8, 18, 19, 26, 34, 42, 46, 54, 63, 73, 78, 82, 87, 97, 107, 115, 125, 132, 144, 155, 161, 167, 185, 194, 206, 218, 227, 237, 254, 262, 277, 291, 298, 307, 318, 327, 335, 347, 365, 375, 384, 393, 404, 413, 420, 439, 453, 459, 469, 478, 488, 495, 503, 518, 531, 537, 549, 561, 566, 570, 576, 584, 595, 603, 618, 627, 632, 643, 652, 661, 668, 677, 682, 688, 703, 713, 719, 724, 732, 739, 742, 749, 754, 760, 769, 772, 777, 781, 792, 798, 806, 818, 821, 829, 835, 842, 850, 864, 877, 890], "exact_score": [76.42247255506925, 76.69512807634355, 77.13281193944178, 77.52026978546316, 77.88620219559446, 78.2306091698357, 78.51043983640669, 78.876372246538, 79.17772834899907, 79.43603357967999, 79.73021453684437, 79.9741694769319, 80.36162732295328, 80.69168400660114, 80.94281409198535, 81.24417019444644, 81.50965057042406, 81.75360551051159, 82.01908588648921, 82.2056396642032, 82.47112004018081, 82.68637439908159, 82.9446796297625, 83.15993398866327, 83.41823921934419, 83.64066872354165, 83.86309822773912, 84.10705316782665, 84.3366578173208, 84.55908732151826, 84.72411566334218, 84.91784458635287, 85.09722321877018, 85.29095214178088, 85.42727990241804, 85.66405969720887, 85.79321231254933, 86.06586783382365, 86.18067015857072, 86.23807132094426, 86.4676759704384, 86.59682858577887, 86.79773265408625, 86.95558585061349, 87.05603788476716, 87.14931477362417, 87.23541651718448, 87.32151826074478, 87.43632058549186, 87.5798234914257, 87.69462581617277, 87.78072755973308, 87.93858075626032, 87.96010619215039, 88.02468249982063, 88.14665996986439, 88.23276171342471, 88.26863743990816, 88.39061490995192, 88.52694267058908, 88.59151897825932, 88.64174499533615, 88.67762072181962, 88.73502188419316, 88.82829877305016, 88.89287508072039, 88.95745138839062, 89.02920284135753, 89.04355313195093, 89.13683002080792, 89.158355456698, 89.16553060199469, 89.2588074908517, 89.3520843797087, 89.3377340891153, 89.38796010619215, 89.4238358326756, 89.45971155915908, 89.5386381574227, 89.59603931979623, 89.66779077276314, 89.72519193513668, 89.73236708043338, 89.79694338810361, 89.84716940518047, 89.8973954222573, 89.96914687522423, 90.00502260170768, 90.06242376408122, 90.0982994905647, 90.11264978115807, 90.11264978115807, 90.06959890937792, 90.14135036234484, 90.11264978115807, 90.12700007175145, 90.13417521704815, 90.13417521704815, 90.09112434526799, 90.06959890937792, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.21455993951483, "threshold_area100": 86.15344048216977, "threshold_area25": 80.27480806486331}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.71899169292466, "calls": 98.99742194213692}, "test": {"score": 90.34225443065222, "calls": 98.81610102604577}}, "old_normalized_interior": {"val": {"score": 90.48983099398453, "calls": 98.99742194213692}, "test": {"score": 90.00502260170768, "calls": 99.06723111143}}, "old_normalized_endpoints": {"val": {"score": 90.50415353766829, "calls": 100.0}, "test": {"score": 90.06242376408122, "calls": 100.0}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 75.90948152391864, 76.31051274706387, 76.74018905757663, 77.16986536808938, 77.48496132913205, 77.87167000859353, 78.15812088226869, 78.51618447436263, 78.88857061014036, 79.23231165855056, 79.4901174448582, 79.79089086221713, 80.1775995416786, 80.36379260956745, 80.57863076482383, 80.80779146376396, 81.10856488112287, 81.36637066743053, 81.6814666284732, 81.753079346892, 81.96791750214838, 82.22572328845604, 82.51217416213119, 82.75565740475508, 82.9561730163277, 83.17101117158407, 83.42881695789171, 83.71526783156689, 83.9157834431395, 84.0876539673446, 84.27384703523344, 84.5316528215411, 84.7035233457462, 84.86107132626755, 85.11887711257519, 85.33371526783156, 85.51990833572042, 85.7347464909768, 85.87797192781437, 86.03551990833571, 86.12145517043827, 86.23603551990833, 86.30764823832713, 86.42222858779719, 86.59409911200228, 86.75164709252363, 86.88054998567746, 86.86622744199369, 87.03809796619879, 87.15267831566887, 87.26725866513893, 87.3102262961902, 87.49641936407906, 87.6539673446004, 87.78287023775422, 87.88312804354052, 87.9260956745918, 87.95474076195933, 88.12661128616442, 88.14093382984818, 88.1695789172157, 88.32712689773703, 88.38441707247206, 88.38441707247206, 88.42738470352333, 88.57061014036093, 88.69951303351475, 88.81409338298482, 88.92867373245488, 89.04325408192494, 89.0575766256087, 89.08622171297623, 89.08622171297623, 89.2008020624463, 89.22944714981381, 89.35835004296763, 89.41564021770266, 89.573188198224, 89.61615582927529, 89.63047837295903, 89.63047837295903, 89.73073617874535, 89.77370380979661, 89.78802635348038, 89.83099398453164, 89.93125179031796, 89.97421942136923, 90.07447722715554, 90.13176740189057, 90.14608994557433, 90.20338012030936, 90.21770266399312, 90.21770266399312, 90.23202520767688, 90.30363792609567, 90.30363792609567, 90.4325408192495, 90.44686336293324, 90.48983099398453, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 75.90948152391864, 76.31051274706387, 76.74018905757663, 77.16986536808938, 77.48496132913205, 77.87167000859353, 78.15812088226869, 78.51618447436263, 78.88857061014036, 79.23231165855056, 79.4901174448582, 79.79089086221713, 80.1775995416786, 80.36379260956745, 80.57863076482383, 80.80779146376396, 81.10856488112287, 81.36637066743053, 81.6814666284732, 81.753079346892, 81.96791750214838, 82.22572328845604, 82.51217416213119, 82.75565740475508, 82.9561730163277, 83.17101117158407, 83.42881695789171, 83.71526783156689, 83.9157834431395, 84.0876539673446, 84.27384703523344, 84.5316528215411, 84.7035233457462, 84.86107132626755, 85.11887711257519, 85.33371526783156, 85.51990833572042, 85.7347464909768, 85.87797192781437, 86.03551990833571, 86.12145517043827, 86.23603551990833, 86.30764823832713, 86.42222858779719, 86.59409911200228, 86.75164709252363, 86.88054998567746, 86.86622744199369, 87.03809796619879, 87.15267831566887, 87.26725866513893, 87.3102262961902, 87.49641936407906, 87.6539673446004, 87.78287023775422, 87.88312804354052, 87.9260956745918, 87.95474076195933, 88.12661128616442, 88.14093382984818, 88.1695789172157, 88.32712689773703, 88.38441707247206, 88.38441707247206, 88.42738470352333, 88.57061014036093, 88.69951303351475, 88.81409338298482, 88.92867373245488, 89.04325408192494, 89.0575766256087, 89.08622171297623, 89.08622171297623, 89.2008020624463, 89.22944714981381, 89.35835004296763, 89.41564021770266, 89.573188198224, 89.61615582927529, 89.63047837295903, 89.63047837295903, 89.73073617874535, 89.77370380979661, 89.78802635348038, 89.83099398453164, 89.93125179031796, 89.97421942136923, 90.07447722715554, 90.13176740189057, 90.14608994557433, 90.20338012030936, 90.21770266399312, 90.21770266399312, 90.23202520767688, 90.30363792609567, 90.30363792609567, 90.4325408192495, 90.44686336293324, 90.48983099398453, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.66198796906329, "exact_area100": 85.8404321756785}, "test": {"score": [76.42247255506925, 76.88885699935423, 77.26196455478224, 77.54897036664993, 77.98665422974815, 78.23778431513239, 78.53914041759346, 78.76156992179092, 79.04857573365861, 79.40733299849322, 79.76609026332784, 79.99569491282199, 80.19659898112937, 80.52665566477721, 80.84236205783168, 81.05761641673243, 81.35897251919351, 81.5670517327976, 81.78230609169836, 81.99756045059912, 82.27739111717013, 82.45676974958744, 82.80835186912535, 83.05948195450958, 83.28908660400374, 83.55456697998135, 83.8774485183325, 84.22903063787042, 84.3366578173208, 84.57343761211165, 84.78869197101243, 84.96807060342972, 85.19767525292387, 85.42010475712134, 85.52773193657171, 85.70711056898902, 85.90083949199972, 86.10891870560378, 86.26677190213103, 86.32417306450456, 86.46050082514172, 86.59682858577887, 86.85513381645978, 86.99863672239363, 87.13496448303079, 87.2138910812944, 87.33586855133817, 87.45067087608524, 87.55829805553563, 87.5941737820191, 87.65157494439264, 87.74485183324963, 87.84530386740332, 87.8596541579967, 87.87400444859009, 87.97445648274378, 88.0533830810074, 88.09643395278755, 88.21841142283131, 88.23993685872139, 88.31168831168831, 88.40496520054532, 88.48389179880894, 88.57716868766593, 88.66327043122624, 88.74219702948984, 88.85699935423692, 88.85699935423692, 88.971801678984, 89.03637798665423, 89.0722537131377, 89.10812943962115, 89.18705603788477, 89.23010690966493, 89.2659826361484, 89.22293176436823, 89.31620865322523, 89.36643467030207, 89.4310109779723, 89.45971155915908, 89.50276243093923, 89.56016359331277, 89.58886417449953, 89.59603931979623, 89.60321446509292, 89.58886417449953, 89.624739900983, 89.6103896103896, 89.69649135394992, 89.71084164454331, 89.78259309751022, 89.74671737102676, 89.78259309751022, 89.83281911458707, 89.83281911458707, 89.86869484107054, 89.85434455047715, 89.91892085814737, 89.96197172992754, 90.00502260170768, 90.06242376408122], "calls": [0.0, 1.2700007175145296, 2.3319222214249837, 3.2288153835115163, 4.427064648059123, 5.158929468321734, 6.163449809858649, 6.98859151897826, 7.892659826361484, 8.796728133744708, 9.822773911171701, 10.71249192796154, 11.544808782377842, 12.441701944464375, 13.230967927100522, 13.934132166176367, 14.909951926526514, 15.69921790916266, 16.603286216545886, 17.493004233335725, 18.72712922436679, 19.63837267704671, 21.008825428714932, 22.056396642032, 23.068092128865608, 24.15871421396283, 25.33543804262036, 26.619789050728276, 27.351653870990887, 28.506852263758343, 29.590299203558875, 30.587644399799096, 31.6926167754897, 32.94826720241085, 33.622730860299924, 34.54832460357322, 35.63177154337375, 36.57171557724044, 37.74843940589797, 38.40137762789696, 39.24804477290665, 40.34584200330057, 41.651718447298556, 42.74234053239578, 43.6535839850757, 44.35674822415154, 45.389969146875224, 46.45906579608237, 47.41336012054244, 48.39635502618928, 49.415225658319585, 50.398220563966426, 51.395565760206644, 52.40008610174356, 53.13912606730287, 54.45217765659754, 55.449522852837774, 56.4396929037813, 57.43703810002152, 58.11867690320729, 59.194948697711126, 60.134892731577814, 61.074836765444495, 62.100882542871496, 63.16997919207864, 64.33952787543947, 65.48755112291023, 66.57817320800746, 67.43919064361053, 68.46523642103752, 69.35495443782736, 70.23749730932052, 71.18461648848388, 72.15326110353735, 73.17930688096433, 74.30580469254502, 75.51122910238932, 76.56597546100309, 77.5776709478367, 78.60371672526368, 79.65128793858077, 80.69168400660114, 81.5670517327976, 82.32044198895028, 83.05948195450958, 84.1501040396068, 85.27660185118748, 86.25242161153763, 87.27846738896463, 88.7780727559733, 89.94044629403746, 91.09564468680492, 91.99253784889144, 93.1333859510655, 93.9872282413719, 95.11372605295257, 95.98909377914903, 97.05101528305948, 97.91203271866256, 99.06723111143, 100.0], "exact_score": [76.42247255506925, 76.80275525579393, 77.197388247112, 77.44851833249623, 77.82162588792423, 78.18755829805554, 78.46738896462654, 78.76874506708761, 79.09880175073546, 79.45038387027338, 79.79479084451461, 80.03874578460214, 80.34727703235991, 80.75626031427137, 81.06479156202914, 81.3733228097869, 81.61727774987443, 81.87558298055536, 82.08366219415943, 82.33479227954366, 82.56439692903783, 82.80835186912535, 83.06665709980628, 83.25321087752027, 83.51869125349789, 83.81287221066228, 84.01377627896966, 84.28643180024396, 84.47298557795796, 84.66671450096864, 84.8891440051661, 85.03264691109995, 85.25507641529741, 85.43445504771472, 85.57795795364856, 85.79321231254933, 85.97976609026333, 86.14479443208725, 86.26677190213103, 86.41027480806487, 86.52507713281194, 86.68293032933917, 86.89100954294324, 87.0273373035804, 87.18519050010764, 87.32869340604148, 87.40762000430509, 87.50089689316209, 87.5798234914257, 87.62287436320587, 87.71615125206286, 87.80942814091985, 87.83095357680993, 87.87400444859009, 87.91705532037024, 88.01750735452393, 88.07490851689747, 88.16818540575447, 88.22558656812801, 88.29733802109493, 88.39779005524862, 88.47671665351223, 88.58434383296262, 88.64892014063285, 88.72067159359977, 88.83547391834685, 88.89287508072039, 88.89287508072039, 89.01485255076416, 89.05790342254431, 89.09377914902777, 89.18705603788477, 89.22293176436823, 89.2659826361484, 89.23728205496163, 89.27315778144508, 89.3377340891153, 89.39513525148884, 89.4310109779723, 89.50993757623593, 89.52428786682931, 89.56016359331277, 89.6103896103896, 89.6103896103896, 89.58886417449953, 89.60321446509292, 89.6103896103896, 89.68214106335654, 89.75389251632346, 89.72519193513668, 89.78259309751022, 89.75389251632346, 89.78259309751022, 89.82564396929038, 89.83281911458707, 89.86151969577384, 89.86151969577384, 89.91892085814737, 89.96197172992754, 90.00502260170768, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 80.26548037597762, "exact_area100": 86.18083902185963}}, "old_normalized_endpoint_budget": 100}, {"seed": 2029, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 1.0, "validation_area100": 0.8585301312102546, "training_seconds": 29.65872742200736, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6185448512780537, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5673151902937962, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5257380123779777, "operator": "ge"}, {"budget": 0.04, "threshold": 0.49158557921130647, "operator": "ge"}, {"budget": 0.05, "threshold": 0.45973139251625356, "operator": "ge"}, {"budget": 0.06, "threshold": 0.43572262589304034, "operator": "ge"}, {"budget": 0.07, "threshold": 0.41260295912656353, "operator": "ge"}, {"budget": 0.08, "threshold": 0.39164688313891144, "operator": "ge"}, {"budget": 0.09, "threshold": 0.36255260255010174, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3415761821232295, "operator": "ge"}, {"budget": 0.11, "threshold": 0.3203681706528533, "operator": "ge"}, {"budget": 0.12, "threshold": 0.30086304494684674, "operator": "ge"}, {"budget": 0.13, "threshold": 0.2757853436687906, "operator": "ge"}, {"budget": 0.14, "threshold": 0.24760226440816818, "operator": "ge"}, {"budget": 0.15, "threshold": 0.22122888428862883, "operator": "ge"}, {"budget": 0.16, "threshold": 0.192058700738665, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1604480375214011, "operator": "ge"}, {"budget": 0.18, "threshold": 0.1304747806648062, "operator": "ge"}, {"budget": 0.19, "threshold": 0.09991223461110817, "operator": "ge"}, {"budget": 0.2, "threshold": 0.06758076867069314, "operator": "ge"}, {"budget": 0.21, "threshold": 0.022478714201468, "operator": "ge"}, {"budget": 0.22, "threshold": -0.02039884768317784, "operator": "ge"}, {"budget": 0.23, "threshold": -0.06758802227059399, "operator": "ge"}, {"budget": 0.24, "threshold": -0.11890010383904868, "operator": "ge"}, {"budget": 0.25, "threshold": -0.16352179643538603, "operator": "ge"}, {"budget": 0.26, "threshold": -0.221191528155548, "operator": "ge"}, {"budget": 0.27, "threshold": -0.2625340859766082, "operator": "ge"}, {"budget": 0.28, "threshold": -0.3061623561500208, "operator": "ge"}, {"budget": 0.29, "threshold": -0.3488252222261257, "operator": "ge"}, {"budget": 0.3, "threshold": -0.39028129837379333, "operator": "ge"}, {"budget": 0.31, "threshold": -0.43808527384873625, "operator": "ge"}, {"budget": 0.32, "threshold": -0.4795571118192419, "operator": "ge"}, {"budget": 0.33, "threshold": -0.5153917879513481, "operator": "ge"}, {"budget": 0.34, "threshold": -0.5487269314602268, "operator": "ge"}, {"budget": 0.35, "threshold": -0.586815847795797, "operator": "ge"}, {"budget": 0.36, "threshold": -0.6238576750754146, "operator": "ge"}, {"budget": 0.37, "threshold": -0.6562767294634979, "operator": "ge"}, {"budget": 0.38, "threshold": -0.6847659357425134, "operator": "ge"}, {"budget": 0.39, "threshold": -0.7134921833014646, "operator": "ge"}, {"budget": 0.4, "threshold": -0.7427517348664455, "operator": "ge"}, {"budget": 0.41, "threshold": -0.7673106470428065, "operator": "ge"}, {"budget": 0.42, "threshold": -0.7911693931784066, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8163346397991031, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8376991525386763, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8546101765799872, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8715200236547099, "operator": "ge"}, {"budget": 0.47, "threshold": -0.8889088667284039, "operator": "ge"}, {"budget": 0.48, "threshold": -0.9005687853897345, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9122482271241293, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9225900635059283, "operator": "ge"}, {"budget": 0.51, "threshold": -0.9304228862191094, "operator": "ge"}, {"budget": 0.52, "threshold": -0.9366522005118104, "operator": "ge"}, {"budget": 0.53, "threshold": -0.9424745845357356, "operator": "ge"}, {"budget": 0.54, "threshold": -0.9482148652042545, "operator": "ge"}, {"budget": 0.55, "threshold": -0.9538508168799504, "operator": "ge"}, {"budget": 0.56, "threshold": -0.9580760833667046, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9614709180515539, "operator": "ge"}, {"budget": 0.58, "threshold": -0.966187276845509, "operator": "ge"}, {"budget": 0.59, "threshold": -0.969483469618869, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9717370898474914, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9736736303706104, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9757610859131312, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9775824049521314, "operator": "ge"}, {"budget": 0.64, "threshold": -0.9792958197655675, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9805533877030455, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9819296328059262, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9831802479596112, "operator": "ge"}, {"budget": 0.68, "threshold": -0.984426045817527, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9854327311055735, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9862646474861748, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9871018696633609, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9879766144377217, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9887187168217212, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9895072135219575, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9900449296191759, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9907204650147494, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9912592156475017, "operator": "ge"}, {"budget": 0.78, "threshold": -0.991736450568025, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9922009854387429, "operator": "ge"}, {"budget": 0.8, "threshold": -0.9926974796600855, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9931981874748614, "operator": "ge"}, {"budget": 0.82, "threshold": -0.993547411659426, "operator": "ge"}, {"budget": 0.83, "threshold": -0.993943348001885, "operator": "ge"}, {"budget": 0.84, "threshold": -0.994287807009529, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9947206896965254, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9950305381436277, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9953113200858236, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9956165100439216, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9960028172897586, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9963423526922328, "operator": "ge"}, {"budget": 0.91, "threshold": -0.9966326701785727, "operator": "ge"}, {"budget": 0.92, "threshold": -0.996967959868241, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9972720102145545, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9976001195431784, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9978775544326387, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9981504042859731, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9983787291379013, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9986718987255754, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9989649318172695, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.00973932970496, 76.42509309653394, 76.82612431967917, 77.19851045545688, 77.45631624176454, 77.6711543970209, 77.97192781437982, 78.17244342595244, 78.51618447436263, 78.96018332855915, 79.30392437696935, 79.70495560011457, 79.96276138642222, 80.2492122600974, 80.49269550272128, 80.70753365797765, 80.87940418218275, 81.12288742480665, 81.38069321111429, 81.69578917215698, 82.01088513319965, 82.2686909195073, 82.55514179318246, 82.89888284159265, 83.15668862790031, 83.28559152105413, 83.58636493841306, 83.82984818103695, 84.07333142366083, 84.30249212260097, 84.51733027785735, 84.57462045259237, 84.73216843311371, 84.88971641363506, 85.0472643941564, 85.34803781151533, 85.54855342308794, 85.7347464909768, 85.9066170151819, 86.02119736465195, 86.19306788885706, 86.3506158693784, 86.56545402463478, 86.73732454883987, 86.96648524778, 86.99513033514752, 87.02377542251504, 87.1383557719851, 87.16700085935261, 87.22429103408766, 87.28158120882267, 87.38183901460899, 87.38183901460899, 87.5107419077628, 87.69693497565167, 87.78287023775422, 87.9260956745918, 88.11228874248066, 88.19822400458321, 88.21254654826697, 88.37009452878831, 88.41306215983958, 88.47035233457461, 88.55628759667717, 88.57061014036093, 88.6135777714122, 88.71383557719851, 88.72815812088226, 88.77112575193354, 88.90002864508737, 89.00028645087367, 89.02893153824118, 89.15783443139502, 89.15783443139502, 89.18647951876252, 89.3297049556001, 89.4013176740189, 89.50157547980521, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.68776854769406, 89.73073617874535, 89.75938126611285, 89.7450587224291, 89.8166714408479, 90.00286450873675, 90.14608994557433, 90.21770266399312, 90.24634775136064, 90.27499283872815, 90.26067029504439, 90.28931538241191, 90.31796046977942, 90.33228301346318, 90.46118590661702, 90.4325408192495, 90.50415353766829, 90.47550845030077, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 64, 94, 122, 145, 167, 193, 212, 243, 276, 306, 337, 360, 388, 410, 429, 447, 469, 493, 518, 543, 569, 595, 623, 645, 660, 686, 708, 729, 749, 770, 784, 802, 821, 840, 866, 884, 901, 918, 934, 952, 966, 986, 1002, 1020, 1029, 1036, 1048, 1057, 1067, 1075, 1085, 1091, 1104, 1118, 1128, 1140, 1154, 1163, 1167, 1181, 1186, 1195, 1205, 1213, 1220, 1230, 1236, 1240, 1251, 1260, 1265, 1276, 1280, 1289, 1300, 1306, 1313, 1319, 1326, 1329, 1334, 1338, 1342, 1346, 1353, 1367, 1378, 1385, 1391, 1396, 1400, 1405, 1412, 1416, 1426, 1431, 1439, 1444, 1449], "harmed": [0, 1, 4, 6, 8, 13, 20, 25, 30, 37, 39, 45, 48, 53, 61, 66, 70, 76, 81, 87, 90, 93, 101, 107, 111, 115, 121, 126, 131, 135, 139, 145, 155, 162, 170, 178, 183, 187, 191, 196, 204, 210, 213, 218, 222, 224, 231, 236, 240, 247, 253, 257, 260, 266, 270, 271, 275, 277, 278, 281, 284, 287, 289, 294, 298, 305, 309, 312, 317, 318, 320, 322, 325, 327, 331, 338, 339, 340, 340, 343, 347, 348, 348, 349, 351, 356, 358, 359, 360, 362, 366, 369, 374, 377, 382, 385, 386, 393, 396, 403, 406], "exact_score": [75.56574047550845, 76.00973932970496, 76.42509309653394, 76.82612431967917, 77.19851045545688, 77.45631624176454, 77.6711543970209, 77.97192781437982, 78.17244342595244, 78.51618447436263, 78.96018332855915, 79.30392437696935, 79.70495560011457, 79.96276138642222, 80.2492122600974, 80.49269550272128, 80.70753365797765, 80.87940418218275, 81.12288742480665, 81.38069321111429, 81.69578917215698, 82.01088513319965, 82.2686909195073, 82.55514179318246, 82.89888284159265, 83.15668862790031, 83.28559152105413, 83.58636493841306, 83.82984818103695, 84.07333142366083, 84.30249212260097, 84.51733027785735, 84.57462045259237, 84.73216843311371, 84.88971641363506, 85.0472643941564, 85.34803781151533, 85.54855342308794, 85.7347464909768, 85.9066170151819, 86.02119736465195, 86.19306788885706, 86.3506158693784, 86.56545402463478, 86.73732454883987, 86.96648524778, 86.99513033514752, 87.02377542251504, 87.1383557719851, 87.16700085935261, 87.22429103408766, 87.28158120882267, 87.38183901460899, 87.38183901460899, 87.5107419077628, 87.69693497565167, 87.78287023775422, 87.9260956745918, 88.11228874248066, 88.19822400458321, 88.21254654826697, 88.37009452878831, 88.41306215983958, 88.47035233457461, 88.55628759667717, 88.57061014036093, 88.6135777714122, 88.71383557719851, 88.72815812088226, 88.77112575193354, 88.90002864508737, 89.00028645087367, 89.02893153824118, 89.15783443139502, 89.15783443139502, 89.18647951876252, 89.3297049556001, 89.4013176740189, 89.50157547980521, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.68776854769406, 89.73073617874535, 89.75938126611285, 89.7450587224291, 89.8166714408479, 90.00286450873675, 90.14608994557433, 90.21770266399312, 90.24634775136064, 90.27499283872815, 90.26067029504439, 90.28931538241191, 90.31796046977942, 90.33228301346318, 90.46118590661702, 90.4325408192495, 90.50415353766829, 90.47550845030077, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.85301312102546, "threshold_area100": 85.85197651102835, "threshold_area25": 79.59323975938126}, "test": {"score": [76.42247255506925, 76.917557580541, 77.27631484537562, 77.69247327258377, 78.02970510152831, 78.38128722106623, 78.68981846882399, 78.92659826361485, 79.24947980196599, 79.54366075913038, 79.87371744277823, 80.12484752816245, 80.32575159646983, 80.64145798952428, 80.89976322020522, 81.15089330558945, 81.45224940805052, 81.7392552199182, 81.91145870703882, 82.18411422831312, 82.3850182966205, 82.72225012556504, 82.98773050154266, 83.18863456985004, 83.36801320226735, 83.6047929970582, 83.83439764655235, 84.04965200545311, 84.29360694554065, 84.55191217622158, 84.78151682571571, 85.0900480734735, 85.22637583411064, 85.39857932123127, 85.52055679127503, 85.64253426131879, 85.77886202195594, 85.95824065437326, 86.12326899619718, 86.34569850039465, 86.63270431226233, 86.79773265408625, 86.94123556002009, 87.09908875654733, 87.2138910812944, 87.24259166248117, 87.36456913252493, 87.50089689316209, 87.55112291023893, 87.72332639735954, 87.82377843151325, 87.84530386740332, 87.90270502977687, 87.97445648274378, 88.03185764511731, 88.12513453397432, 88.15383511516109, 88.21123627753462, 88.37626461935855, 88.46954150821554, 88.5125923799957, 88.57716868766593, 88.66327043122624, 88.75654732008323, 88.76372246537993, 88.84982420894023, 88.89287508072039, 88.96462653368731, 89.03637798665423, 89.12965487551124, 89.158355456698, 89.17988089258809, 89.28750807203846, 89.30185836263185, 89.38078496089545, 89.4238358326756, 89.46688670445577, 89.54581330271938, 89.6175647556863, 89.67496591805984, 89.73954222573008, 89.76824280691685, 89.73236708043338, 89.8041185334003, 89.82564396929038, 89.84716940518047, 89.83999425988375, 89.86151969577384, 89.8902202769606, 89.94762143933414, 89.99067231111431, 90.04089832819116, 90.04089832819116, 90.0982994905647, 90.10547463586138, 90.09112434526799, 90.07677405467462, 90.0839491999713, 90.07677405467462, 90.05524861878453, 90.06242376408122], "calls": [0.0, 1.363277606371529, 2.346272512018368, 3.3651431441486688, 4.312262323312047, 5.345483246035732, 6.106048647485111, 6.89531463012126, 7.878309535768099, 8.904355313195092, 9.758197603501472, 10.590514457917772, 11.171701226949846, 12.161871277893377, 13.130515892946832, 14.005883619143288, 15.031929396570282, 16.179952644041045, 17.026619789050727, 18.002439549400876, 18.920858147377484, 20.05453110425486, 21.03035086460501, 22.020520915548538, 23.268996197172992, 24.287866829303294, 25.629618999784746, 26.52651216187128, 27.444930759847885, 28.44227595608811, 29.532898041185334, 30.723972160436247, 31.843294826720243, 32.912391475927386, 33.70165745856354, 34.799454688957454, 35.79679988519768, 36.858721389108126, 37.7197388247112, 38.65968285857789, 39.857932123125494, 40.7835258663988, 41.79522135323241, 42.88584343832962, 43.98364066872354, 44.72985577957954, 45.69850039463299, 46.88957451388391, 47.74341680419029, 48.6546602568702, 49.601779436033574, 50.46997201693334, 51.266413144866185, 52.15613116165603, 53.1032503408194, 54.00731864820263, 54.767884049652004, 55.650426921145154, 56.94195307454976, 58.054100595537065, 58.879242304656664, 59.840711774413435, 61.10353734663126, 62.20133457702518, 63.205854918562096, 64.19602496950563, 65.18619502044916, 66.19071536198608, 67.36743919064361, 68.4867618569276, 69.57020879672814, 70.54602855707827, 71.66535122336228, 72.7487981631628, 74.08337518834756, 74.6932625385664, 75.77670947836694, 76.63772691397, 77.49156920427639, 78.38128722106623, 79.48625959675684, 80.61993255363421, 81.45224940805052, 82.34914257013705, 83.19580971514672, 84.30795723613403, 85.18332496233049, 86.08021812441702, 87.02016215828371, 88.20406113223794, 89.43818612326899, 90.56468393484968, 91.77010834469398, 92.88943101097797, 94.20965774556936, 95.23570352299635, 96.27609959101672, 97.08689100954294, 98.08423620578317, 98.95242878668293, 100.0], "rescued": [0, 78, 133, 199, 255, 312, 359, 401, 457, 509, 563, 607, 643, 693, 742, 784, 832, 882, 913, 960, 996, 1050, 1097, 1139, 1185, 1226, 1274, 1315, 1356, 1402, 1448, 1503, 1542, 1581, 1608, 1640, 1676, 1716, 1747, 1789, 1840, 1875, 1906, 1939, 1970, 1986, 2014, 2043, 2059, 2091, 2110, 2121, 2139, 2157, 2174, 2191, 2200, 2215, 2243, 2265, 2278, 2295, 2318, 2338, 2349, 2366, 2377, 2395, 2414, 2431, 2443, 2456, 2479, 2490, 2512, 2523, 2535, 2550, 2567, 2579, 2591, 2604, 2611, 2623, 2632, 2644, 2652, 2659, 2670, 2685, 2703, 2717, 2728, 2742, 2750, 2757, 2763, 2769, 2777, 2781, 2791], "harmed": [0, 9, 14, 22, 31, 39, 43, 52, 63, 74, 82, 91, 99, 105, 118, 125, 131, 141, 148, 157, 165, 172, 182, 196, 217, 225, 241, 252, 259, 269, 283, 295, 315, 330, 340, 355, 372, 387, 395, 406, 417, 429, 440, 451, 466, 478, 489, 499, 508, 516, 521, 529, 539, 547, 556, 560, 565, 572, 577, 586, 593, 601, 612, 619, 629, 634, 639, 647, 656, 660, 668, 678, 686, 695, 706, 711, 717, 721, 728, 732, 735, 744, 756, 758, 764, 773, 782, 786, 793, 800, 812, 819, 830, 836, 843, 852, 860, 865, 874, 881, 890], "exact_score": [76.42247255506925, 76.77405467460716, 77.13998708473846, 77.5848460931334, 77.92925306737462, 78.25930975102247, 78.63959245174715, 78.95529884480162, 79.26383009255937, 79.58671163091053, 79.95264404104184, 80.2611752887996, 80.59840711774415, 80.84236205783168, 81.15089330558945, 81.43789911745714, 81.68185405754467, 81.91145870703882, 82.18411422831312, 82.40654373251058, 82.70789983497167, 82.98055535624597, 83.18863456985004, 83.3249623304872, 83.55456697998135, 83.74112075769534, 83.93484968070604, 84.19315491138696, 84.44428499677119, 84.65953935567195, 84.91784458635287, 85.13309894525365, 85.22637583411064, 85.41292961182464, 85.58513309894525, 85.67840998780225, 85.78603716725264, 85.98694123556002, 86.14479443208725, 86.43180024395494, 86.64705460285572, 86.82643323527302, 86.97711128650354, 87.09908875654733, 87.22106622659109, 87.29999282485471, 87.37174427782163, 87.52242232905216, 87.60134892731578, 87.80225299562316, 87.83095357680993, 87.88835473918347, 87.96728133744709, 88.03185764511731, 88.12513453397432, 88.18253569634786, 88.24711200401808, 88.36908947406185, 88.47671665351223, 88.505417234699, 88.59151897825932, 88.64174499533615, 88.74219702948984, 88.74219702948984, 88.84264906364353, 88.878524790127, 88.93592595250054, 89.03637798665423, 89.07942885843438, 89.13683002080792, 89.16553060199469, 89.20858147377486, 89.2659826361484, 89.30903350792853, 89.38796010619215, 89.4310109779723, 89.50993757623593, 89.56016359331277, 89.66061562746646, 89.69649135394992, 89.76824280691685, 89.73954222573008, 89.78976824280693, 89.811293678697, 89.85434455047715, 89.84716940518047, 89.86151969577384, 89.8902202769606, 89.96197172992754, 89.98349716581761, 90.03372318289445, 90.03372318289445, 90.07677405467462, 90.10547463586138, 90.0982994905647, 90.09112434526799, 90.07677405467462, 90.07677405467462, 90.07677405467462, 90.04089832819116, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.28916739582058, "threshold_area100": 86.23186482026262, "threshold_area25": 80.36090980842363}, "selected_budget": 98, "selected": {"val": {"score": 90.50415353766829, "calls": 97.99484388427385, "rescued": 1439, "harmed": 396}, "test": {"score": 90.07677405467462, "calls": 98.08423620578317, "rescued": 2777, "harmed": 874}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.4021409749984741, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3470430374145508, "operator": "ge"}, {"budget": 0.03, "threshold": 0.31270095705986023, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2876023054122925, "operator": "ge"}, {"budget": 0.05, "threshold": 0.267993301153183, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2483767569065094, "operator": "ge"}, {"budget": 0.07, "threshold": 0.2309420108795166, "operator": "ge"}, {"budget": 0.08, "threshold": 0.21320512890815735, "operator": "ge"}, {"budget": 0.09, "threshold": 0.19672422111034393, "operator": "ge"}, {"budget": 0.1, "threshold": 0.18205265700817108, "operator": "ge"}, {"budget": 0.11, "threshold": 0.1673576533794403, "operator": "ge"}, {"budget": 0.12, "threshold": 0.1517435908317566, "operator": "ge"}, {"budget": 0.13, "threshold": 0.1356879472732544, "operator": "ge"}, {"budget": 0.14, "threshold": 0.11987419426441193, "operator": "ge"}, {"budget": 0.15, "threshold": 0.09969654679298401, "operator": "ge"}, {"budget": 0.16, "threshold": 0.08238351345062256, "operator": "ge"}, {"budget": 0.17, "threshold": 0.06718334555625916, "operator": "ge"}, {"budget": 0.18, "threshold": 0.05488840863108635, "operator": "ge"}, {"budget": 0.19, "threshold": 0.039264529943466187, "operator": "ge"}, {"budget": 0.2, "threshold": 0.022691398859024048, "operator": "ge"}, {"budget": 0.21, "threshold": 0.007322847843170166, "operator": "ge"}, {"budget": 0.22, "threshold": -0.006318695843219757, "operator": "ge"}, {"budget": 0.23, "threshold": -0.010030281729996204, "operator": "ge"}, {"budget": 0.24, "threshold": -0.012382805347442627, "operator": "ge"}, {"budget": 0.25, "threshold": -0.013887130655348301, "operator": "ge"}, {"budget": 0.26, "threshold": -0.01553991436958313, "operator": "ge"}, {"budget": 0.27, "threshold": -0.01695038564503193, "operator": "ge"}, {"budget": 0.28, "threshold": -0.018173756077885628, "operator": "ge"}, {"budget": 0.29, "threshold": -0.019599568098783493, "operator": "ge"}, {"budget": 0.3, "threshold": -0.02087537571787834, "operator": "ge"}, {"budget": 0.31, "threshold": -0.02216392755508423, "operator": "ge"}, {"budget": 0.32, "threshold": -0.023215746507048607, "operator": "ge"}, {"budget": 0.33, "threshold": -0.024245116859674454, "operator": "ge"}, {"budget": 0.34, "threshold": -0.02545556239783764, "operator": "ge"}, {"budget": 0.35, "threshold": -0.02661783993244171, "operator": "ge"}, {"budget": 0.36, "threshold": -0.027897827327251434, "operator": "ge"}, {"budget": 0.37, "threshold": -0.0289685670286417, "operator": "ge"}, {"budget": 0.38, "threshold": -0.03006790578365326, "operator": "ge"}, {"budget": 0.39, "threshold": -0.031052222475409508, "operator": "ge"}, {"budget": 0.4, "threshold": -0.03217502310872078, "operator": "ge"}, {"budget": 0.41, "threshold": -0.0333661325275898, "operator": "ge"}, {"budget": 0.42, "threshold": -0.0344071239233017, "operator": "ge"}, {"budget": 0.43, "threshold": -0.03570104017853737, "operator": "ge"}, {"budget": 0.44, "threshold": -0.036948103457689285, "operator": "ge"}, {"budget": 0.45, "threshold": -0.038164690136909485, "operator": "ge"}, {"budget": 0.46, "threshold": -0.03947262838482857, "operator": "ge"}, {"budget": 0.47, "threshold": -0.040584199130535126, "operator": "ge"}, {"budget": 0.48, "threshold": -0.041998960077762604, "operator": "ge"}, {"budget": 0.49, "threshold": -0.04333873093128204, "operator": "ge"}, {"budget": 0.5, "threshold": -0.04482933506369591, "operator": "ge"}, {"budget": 0.51, "threshold": -0.04598228633403778, "operator": "ge"}, {"budget": 0.52, "threshold": -0.047209661453962326, "operator": "ge"}, {"budget": 0.53, "threshold": -0.04870576038956642, "operator": "ge"}, {"budget": 0.54, "threshold": -0.05030615255236626, "operator": "ge"}, {"budget": 0.55, "threshold": -0.051591843366622925, "operator": "ge"}, {"budget": 0.56, "threshold": -0.053208861500024796, "operator": "ge"}, {"budget": 0.57, "threshold": -0.0549536757171154, "operator": "ge"}, {"budget": 0.58, "threshold": -0.05627026408910751, "operator": "ge"}, {"budget": 0.59, "threshold": -0.057413291186094284, "operator": "ge"}, {"budget": 0.6, "threshold": -0.05881470814347267, "operator": "ge"}, {"budget": 0.61, "threshold": -0.06020348146557808, "operator": "ge"}, {"budget": 0.62, "threshold": -0.06185191497206688, "operator": "ge"}, {"budget": 0.63, "threshold": -0.06335818767547607, "operator": "ge"}, {"budget": 0.64, "threshold": -0.06527286022901535, "operator": "ge"}, {"budget": 0.65, "threshold": -0.06728887557983398, "operator": "ge"}, {"budget": 0.66, "threshold": -0.06936945021152496, "operator": "ge"}, {"budget": 0.67, "threshold": -0.07111427187919617, "operator": "ge"}, {"budget": 0.68, "threshold": -0.0734831839799881, "operator": "ge"}, {"budget": 0.69, "threshold": -0.07535192370414734, "operator": "ge"}, {"budget": 0.7, "threshold": -0.07736611366271973, "operator": "ge"}, {"budget": 0.71, "threshold": -0.07971882820129395, "operator": "ge"}, {"budget": 0.72, "threshold": -0.08209744840860367, "operator": "ge"}, {"budget": 0.73, "threshold": -0.08455898612737656, "operator": "ge"}, {"budget": 0.74, "threshold": -0.08671771734952927, "operator": "ge"}, {"budget": 0.75, "threshold": -0.08942101150751114, "operator": "ge"}, {"budget": 0.76, "threshold": -0.09248260408639908, "operator": "ge"}, {"budget": 0.77, "threshold": -0.09497105330228806, "operator": "ge"}, {"budget": 0.78, "threshold": -0.09744115173816681, "operator": "ge"}, {"budget": 0.79, "threshold": -0.10064176470041275, "operator": "ge"}, {"budget": 0.8, "threshold": -0.10300792008638382, "operator": "ge"}, {"budget": 0.81, "threshold": -0.1071971133351326, "operator": "ge"}, {"budget": 0.82, "threshold": -0.11136626452207565, "operator": "ge"}, {"budget": 0.83, "threshold": -0.11699751764535904, "operator": "ge"}, {"budget": 0.84, "threshold": -0.12153826653957367, "operator": "ge"}, {"budget": 0.85, "threshold": -0.1259147971868515, "operator": "ge"}, {"budget": 0.86, "threshold": -0.13120536506175995, "operator": "ge"}, {"budget": 0.87, "threshold": -0.13779950141906738, "operator": "ge"}, {"budget": 0.88, "threshold": -0.14568598568439484, "operator": "ge"}, {"budget": 0.89, "threshold": -0.15363675355911255, "operator": "ge"}, {"budget": 0.9, "threshold": -0.1627320945262909, "operator": "ge"}, {"budget": 0.91, "threshold": -0.17194229364395142, "operator": "ge"}, {"budget": 0.92, "threshold": -0.1819506287574768, "operator": "ge"}, {"budget": 0.93, "threshold": -0.19478069245815277, "operator": "ge"}, {"budget": 0.94, "threshold": -0.21061213314533234, "operator": "ge"}, {"budget": 0.95, "threshold": -0.234638974070549, "operator": "ge"}, {"budget": 0.96, "threshold": -0.25330108404159546, "operator": "ge"}, {"budget": 0.97, "threshold": -0.2802375555038452, "operator": "ge"}, {"budget": 0.98, "threshold": -0.3215874433517456, "operator": "ge"}, {"budget": 0.99, "threshold": -0.38329172134399414, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.93812661128617, 76.2675451160126, 76.5110283586365, 76.82612431967917, 77.18418791177312, 77.54225150386709, 77.92896018332856, 78.37295903752506, 78.73102262961902, 79.16069894013177, 79.27527928960183, 79.66198796906329, 79.89114866800342, 80.12030936694356, 80.3494700658837, 80.62159839587511, 80.93669435691778, 81.15153251217416, 81.33772558006301, 81.66714408478946, 82.03953022056717, 82.31165855055858, 82.32598109424234, 82.39759381266113, 82.42623890002865, 82.51217416213119, 82.62675451160125, 82.76997994843884, 82.91320538527643, 82.94185047264394, 82.94185047264394, 83.02778573474649, 83.07075336579777, 83.19965625895158, 83.27126897737038, 83.32855915210541, 83.4001718705242, 83.44313950157549, 83.52907476367803, 83.57204239472931, 83.6436551131481, 83.70094528788312, 83.75823546261816, 83.82984818103695, 83.90146089945574, 83.94442853050701, 84.10197651102835, 84.23087940418218, 84.34545975365224, 84.51733027785735, 84.67487825837868, 84.83242623890003, 84.8753938699513, 84.90403895731882, 85.01861930678888, 85.19048983099398, 85.34803781151533, 85.46261816098539, 85.60584359782297, 85.77771412202807, 85.86364938413061, 85.97822973360068, 86.1071326267545, 86.1787453451733, 86.36493841306215, 86.55113148095101, 86.70867946147236, 86.80893726725867, 86.98080779146376, 87.2099684904039, 87.35319392724148, 87.5107419077628, 87.56803208249785, 87.66828988828415, 87.86880549985678, 88.12661128616442, 88.269836723002, 88.35577198510455, 88.5133199656259, 88.55628759667717, 88.72815812088226, 88.87138355771985, 88.94299627613864, 89.14351188771126, 89.25809223718132, 89.31538241191636, 89.58751074190776, 89.71641363506158, 89.78802635348038, 89.97421942136923, 90.10312231452305, 90.18905757662561, 90.23202520767688, 90.31796046977942, 90.41821827556574, 90.4325408192495, 90.57576625608708, 90.66170151818963, 90.77628186765969, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 29, 55, 80, 108, 137, 164, 194, 229, 260, 293, 308, 340, 365, 386, 406, 427, 452, 472, 491, 518, 545, 569, 574, 579, 582, 588, 597, 607, 618, 621, 624, 630, 636, 645, 651, 658, 664, 672, 679, 683, 688, 692, 699, 705, 710, 715, 726, 735, 746, 760, 772, 783, 789, 795, 807, 820, 834, 844, 856, 871, 880, 893, 907, 915, 929, 942, 957, 968, 985, 1005, 1019, 1033, 1041, 1051, 1069, 1089, 1102, 1113, 1127, 1138, 1153, 1168, 1180, 1198, 1211, 1223, 1242, 1257, 1270, 1285, 1300, 1314, 1327, 1345, 1362, 1376, 1400, 1417, 1436, 1449], "harmed": [0, 3, 6, 14, 20, 24, 26, 29, 33, 39, 42, 49, 54, 63, 68, 72, 74, 77, 82, 88, 92, 93, 98, 102, 102, 103, 103, 104, 104, 105, 106, 109, 109, 112, 112, 113, 116, 117, 122, 123, 124, 124, 124, 127, 128, 128, 130, 130, 130, 133, 135, 136, 136, 139, 143, 147, 148, 151, 153, 155, 158, 161, 166, 171, 174, 175, 175, 179, 183, 188, 192, 196, 199, 203, 206, 210, 212, 215, 220, 223, 231, 234, 239, 246, 250, 255, 263, 263, 269, 277, 279, 285, 293, 303, 315, 325, 338, 352, 363, 374, 406], "exact_score": [75.56574047550845, 75.93812661128617, 76.2675451160126, 76.5110283586365, 76.82612431967917, 77.18418791177312, 77.54225150386709, 77.92896018332856, 78.37295903752506, 78.73102262961902, 79.16069894013177, 79.27527928960183, 79.66198796906329, 79.89114866800342, 80.12030936694356, 80.3494700658837, 80.62159839587511, 80.93669435691778, 81.15153251217416, 81.33772558006301, 81.66714408478946, 82.03953022056717, 82.31165855055858, 82.32598109424234, 82.39759381266113, 82.42623890002865, 82.51217416213119, 82.62675451160125, 82.76997994843884, 82.91320538527643, 82.94185047264394, 82.94185047264394, 83.02778573474649, 83.07075336579777, 83.19965625895158, 83.27126897737038, 83.32855915210541, 83.4001718705242, 83.44313950157549, 83.52907476367803, 83.57204239472931, 83.6436551131481, 83.70094528788312, 83.75823546261816, 83.82984818103695, 83.90146089945574, 83.94442853050701, 84.10197651102835, 84.23087940418218, 84.34545975365224, 84.51733027785735, 84.67487825837868, 84.83242623890003, 84.8753938699513, 84.90403895731882, 85.01861930678888, 85.19048983099398, 85.34803781151533, 85.46261816098539, 85.60584359782297, 85.77771412202807, 85.86364938413061, 85.97822973360068, 86.1071326267545, 86.1787453451733, 86.36493841306215, 86.55113148095101, 86.70867946147236, 86.80893726725867, 86.98080779146376, 87.2099684904039, 87.35319392724148, 87.5107419077628, 87.56803208249785, 87.66828988828415, 87.86880549985678, 88.12661128616442, 88.269836723002, 88.35577198510455, 88.5133199656259, 88.55628759667717, 88.72815812088226, 88.87138355771985, 88.94299627613864, 89.14351188771126, 89.25809223718132, 89.31538241191636, 89.58751074190776, 89.71641363506158, 89.78802635348038, 89.97421942136923, 90.10312231452305, 90.18905757662561, 90.23202520767688, 90.31796046977942, 90.41821827556574, 90.4325408192495, 90.57576625608708, 90.66170151818963, 90.77628186765969, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 84.63905035996724, "threshold_area100": 84.63799770839302, "threshold_area25": 79.47149813806932}, "test": {"score": [76.42247255506925, 76.88885699935423, 77.16151252062855, 77.60637152902346, 77.80010045203416, 78.12298199038531, 78.44586352873645, 78.76874506708761, 79.1490277678123, 79.50060988735022, 79.78044055392121, 79.96699433163522, 80.1750735452393, 80.48360479299707, 80.7275597330846, 81.12219272440268, 81.38767310038028, 81.63880318576452, 81.89710841644543, 82.09801248475281, 82.41371887780727, 82.72942527086174, 83.03078137332281, 83.1312334074765, 83.17428427925665, 83.21016000574012, 83.26038602281697, 83.3249623304872, 83.39671378345412, 83.46129009112435, 83.5043409629045, 83.56891727057473, 83.64066872354165, 83.65501901413504, 83.67654445002512, 83.74112075769534, 83.84157279184905, 83.88462366362918, 83.89897395422258, 83.94202482600272, 84.02812656956303, 84.1429288943101, 84.19315491138696, 84.31513238143073, 84.39405897969434, 84.45146014206787, 84.5375618856282, 84.58778790270503, 84.65953935567195, 84.7097653727488, 84.75999138982566, 84.83891798808926, 84.96089545813304, 85.02547176580326, 85.13309894525365, 85.26225156059411, 85.32682786826433, 85.42010475712134, 85.59230824424195, 85.69993542369232, 85.85061347492287, 85.98694123556002, 86.11609385090048, 86.20937073975749, 86.30264762861448, 86.48920140632848, 86.61835402166894, 86.71880605582263, 86.8407835258664, 86.96993614120686, 87.09191361125063, 87.23541651718448, 87.3932697137117, 87.47937145727201, 87.5941737820191, 87.76637726913971, 87.85247901270002, 87.96010619215039, 88.11795938867762, 88.24711200401808, 88.31168831168831, 88.45519121762216, 88.57716868766593, 88.76372246537993, 88.90005022601709, 89.05072827724761, 89.15118031140132, 89.2588074908517, 89.38796010619215, 89.48841214034584, 89.50276243093923, 89.64626533687307, 89.71801678984, 89.87586998636723, 89.92609600344407, 90.11264978115807, 90.23462725120184, 90.27050297768531, 90.27050297768531, 90.25615268709191, 90.06242376408122], "calls": [0.0, 1.1551983927674536, 2.1740690248977543, 3.1068379134677477, 3.9104541866972804, 4.685369878740044, 5.689890220276961, 6.543732510583339, 7.6200043050871775, 8.617349501327402, 9.535768099304011, 10.36090980842362, 11.27932840640023, 12.06859438903638, 12.951137260529524, 14.213962832747363, 15.283059481954508, 16.151252062854272, 16.90464231900696, 17.844586352873645, 19.03566047212456, 20.08323168544163, 21.245605223505777, 22.257300710339383, 23.182894453612686, 24.000861017435604, 24.904929324818827, 25.916624811652433, 26.806342828442276, 27.846738896462654, 28.96606156274665, 29.977757049580255, 30.896175647556863, 31.721317356676472, 32.67561168113654, 33.61555571500323, 34.74922867188061, 35.66764726985722, 36.72956877376767, 37.51883475640382, 38.437253354380424, 39.34849680706034, 40.238214823850186, 41.379062926024254, 42.591662481165244, 43.6535839850757, 44.79443208724977, 45.655449522852834, 46.710195881466596, 47.750591949486974, 48.733586855133815, 49.65918059840712, 50.5130228887135, 51.52471837554711, 52.62969075123771, 53.598335366291174, 54.63873143431155, 55.89438186123269, 56.74104900624237, 57.51596469828514, 58.506134749228664, 59.32410131305159, 60.25687020162158, 61.153763363708116, 62.23003515821195, 63.36370811508934, 64.55478223434025, 65.54495228528377, 66.63557437038101, 67.5468178230609, 68.60873932697137, 69.73523713855205, 70.63930544593529, 71.70122694984573, 72.4689674965918, 73.69591734232618, 74.86546602568703, 75.81258520485041, 76.6520772045634, 77.68529812728707, 78.28801033220923, 79.44320872497667, 80.56253139126068, 81.96885986941236, 82.77965128793858, 83.64066872354165, 84.6954150821554, 85.99411638085671, 87.15648991892085, 88.28298773050153, 89.4238358326756, 90.59338451603645, 91.53332854990313, 92.60242519911029, 93.65717155772404, 95.07785032646912, 95.9245174714788, 96.87881179593887, 98.00530960751955, 98.85197675252924, 100.0], "rescued": [0, 74, 126, 196, 234, 283, 333, 384, 446, 507, 551, 588, 627, 675, 718, 785, 832, 873, 915, 956, 1005, 1052, 1102, 1118, 1127, 1136, 1144, 1156, 1167, 1179, 1187, 1197, 1208, 1212, 1222, 1232, 1248, 1256, 1264, 1271, 1286, 1304, 1313, 1334, 1351, 1364, 1381, 1393, 1405, 1417, 1431, 1445, 1465, 1478, 1500, 1526, 1543, 1565, 1593, 1615, 1638, 1659, 1682, 1699, 1720, 1750, 1778, 1799, 1824, 1846, 1872, 1900, 1928, 1948, 1969, 2006, 2032, 2055, 2079, 2106, 2122, 2151, 2182, 2217, 2242, 2269, 2300, 2337, 2368, 2398, 2424, 2459, 2482, 2524, 2556, 2613, 2646, 2677, 2715, 2747, 2791], "harmed": [0, 9, 23, 31, 42, 46, 51, 57, 66, 78, 83, 94, 104, 109, 118, 130, 140, 146, 152, 165, 170, 173, 181, 183, 186, 190, 191, 194, 195, 198, 200, 201, 202, 204, 211, 212, 214, 216, 222, 223, 226, 228, 230, 234, 240, 245, 250, 255, 257, 262, 269, 272, 275, 279, 286, 294, 302, 311, 315, 322, 324, 326, 331, 335, 343, 347, 357, 364, 372, 376, 385, 393, 399, 407, 412, 425, 439, 447, 449, 458, 465, 474, 488, 497, 503, 509, 526, 548, 561, 577, 601, 616, 629, 649, 674, 705, 721, 747, 785, 819, 890], "exact_score": [76.42247255506925, 76.8314558369807, 77.1112865035517, 77.52026978546316, 77.82880103322093, 78.23778431513239, 78.53914041759346, 78.9624739900983, 79.29970581904284, 79.6082370668006, 79.86654229748153, 80.12484752816245, 80.43337877592022, 80.73473487838129, 81.05761641673243, 81.30874650211666, 81.58140202339098, 81.94015928822559, 82.1195379206429, 82.3993685872139, 82.70789983497167, 82.98773050154266, 83.10970797158643, 83.16710913395997, 83.21016000574012, 83.26756116811366, 83.33213747578388, 83.4110640740475, 83.46129009112435, 83.5043409629045, 83.56891727057473, 83.63349357824497, 83.66219415943173, 83.6908947406185, 83.80569706536558, 83.84874793714573, 83.89179880892588, 83.92049939011265, 83.97072540718949, 84.12140345842003, 84.17880462079357, 84.27208150965058, 84.35818325321088, 84.4299347061778, 84.47298557795796, 84.56626246681496, 84.6093133385951, 84.66671450096864, 84.7097653727488, 84.8030422616058, 84.88196885986942, 84.99677118461649, 85.03982205639664, 85.22637583411064, 85.27660185118748, 85.3698787400445, 85.43445504771472, 85.62818397072542, 85.79321231254933, 85.92954007318649, 86.08021812441702, 86.17349501327402, 86.27394704742771, 86.40309966276818, 86.58247829518548, 86.67575518404249, 86.77620721819616, 86.87665925234987, 87.04886273947048, 87.14931477362417, 87.29281767955801, 87.43632058549186, 87.50089689316209, 87.665925234986, 87.81660328621655, 87.87400444859009, 88.00315706393054, 88.16101026045777, 88.29016287579825, 88.39061490995192, 88.5197675252924, 88.62021955944608, 88.77089761067663, 88.92157566190716, 89.065078567841, 89.20858147377486, 89.2588074908517, 89.38078496089545, 89.44536126856569, 89.52428786682931, 89.58168902920285, 89.68214106335654, 89.78976824280693, 89.86869484107054, 89.98349716581761, 90.11264978115807, 90.20592667001507, 90.25615268709191, 90.27050297768531, 90.24897754179523, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 85.02899395540989, "threshold_area100": 84.90489344909234, "threshold_area25": 80.25113008538423}, "selected_budget": 99, "selected": {"val": {"score": 90.77628186765969, "calls": 98.99742194213692, "rescued": 1436, "harmed": 374}, "test": {"score": 90.25615268709191, "calls": 98.85197675252924, "rescued": 2747, "harmed": 819}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.124847650527954, "operator": "ge"}, {"budget": 0.02, "threshold": 1.7993535995483398, "operator": "ge"}, {"budget": 0.03, "threshold": 1.64516019821167, "operator": "ge"}, {"budget": 0.04, "threshold": 1.506714105606079, "operator": "ge"}, {"budget": 0.05, "threshold": 1.390610694885254, "operator": "ge"}, {"budget": 0.06, "threshold": 1.3066484928131104, "operator": "ge"}, {"budget": 0.07, "threshold": 1.2305556535720825, "operator": "ge"}, {"budget": 0.08, "threshold": 1.149672269821167, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0763243436813354, "operator": "ge"}, {"budget": 0.1, "threshold": 1.0088071823120117, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9352731108665466, "operator": "ge"}, {"budget": 0.12, "threshold": 0.854698121547699, "operator": "ge"}, {"budget": 0.13, "threshold": 0.7819557785987854, "operator": "ge"}, {"budget": 0.14, "threshold": 0.7089284062385559, "operator": "ge"}, {"budget": 0.15, "threshold": 0.6354802250862122, "operator": "ge"}, {"budget": 0.16, "threshold": 0.546513020992279, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4653134047985077, "operator": "ge"}, {"budget": 0.18, "threshold": 0.36913084983825684, "operator": "ge"}, {"budget": 0.19, "threshold": 0.28826841711997986, "operator": "ge"}, {"budget": 0.2, "threshold": 0.18103748559951782, "operator": "ge"}, {"budget": 0.21, "threshold": 0.062212228775024414, "operator": "ge"}, {"budget": 0.22, "threshold": -0.05477464199066162, "operator": "ge"}, {"budget": 0.23, "threshold": -0.17328450083732605, "operator": "ge"}, {"budget": 0.24, "threshold": -0.31760746240615845, "operator": "ge"}, {"budget": 0.25, "threshold": -0.4512089490890503, "operator": "ge"}, {"budget": 0.26, "threshold": -0.5680163502693176, "operator": "ge"}, {"budget": 0.27, "threshold": -0.6711066961288452, "operator": "ge"}, {"budget": 0.28, "threshold": -0.7992178201675415, "operator": "ge"}, {"budget": 0.29, "threshold": -0.9030927419662476, "operator": "ge"}, {"budget": 0.3, "threshold": -1.012667179107666, "operator": "ge"}, {"budget": 0.31, "threshold": -1.1372171640396118, "operator": "ge"}, {"budget": 0.32, "threshold": -1.224507212638855, "operator": "ge"}, {"budget": 0.33, "threshold": -1.3290302753448486, "operator": "ge"}, {"budget": 0.34, "threshold": -1.4326426982879639, "operator": "ge"}, {"budget": 0.35, "threshold": -1.5448095798492432, "operator": "ge"}, {"budget": 0.36, "threshold": -1.6647772789001465, "operator": "ge"}, {"budget": 0.37, "threshold": -1.7814574241638184, "operator": "ge"}, {"budget": 0.38, "threshold": -1.8945894241333008, "operator": "ge"}, {"budget": 0.39, "threshold": -2.0091426372528076, "operator": "ge"}, {"budget": 0.4, "threshold": -2.159242868423462, "operator": "ge"}, {"budget": 0.41, "threshold": -2.2818479537963867, "operator": "ge"}, {"budget": 0.42, "threshold": -2.411890983581543, "operator": "ge"}, {"budget": 0.43, "threshold": -2.526790142059326, "operator": "ge"}, {"budget": 0.44, "threshold": -2.6885457038879395, "operator": "ge"}, {"budget": 0.45, "threshold": -2.824396848678589, "operator": "ge"}, {"budget": 0.46, "threshold": -2.937390089035034, "operator": "ge"}, {"budget": 0.47, "threshold": -3.0722570419311523, "operator": "ge"}, {"budget": 0.48, "threshold": -3.215824604034424, "operator": "ge"}, {"budget": 0.49, "threshold": -3.36179780960083, "operator": "ge"}, {"budget": 0.5, "threshold": -3.4919090270996094, "operator": "ge"}, {"budget": 0.51, "threshold": -3.5958070755004883, "operator": "ge"}, {"budget": 0.52, "threshold": -3.71413516998291, "operator": "ge"}, {"budget": 0.53, "threshold": -3.8309850692749023, "operator": "ge"}, {"budget": 0.54, "threshold": -3.9406704902648926, "operator": "ge"}, {"budget": 0.55, "threshold": -4.043003082275391, "operator": "ge"}, {"budget": 0.56, "threshold": -4.16398286819458, "operator": "ge"}, {"budget": 0.57, "threshold": -4.2854533195495605, "operator": "ge"}, {"budget": 0.58, "threshold": -4.383467674255371, "operator": "ge"}, {"budget": 0.59, "threshold": -4.465221405029297, "operator": "ge"}, {"budget": 0.6, "threshold": -4.543985843658447, "operator": "ge"}, {"budget": 0.61, "threshold": -4.641172409057617, "operator": "ge"}, {"budget": 0.62, "threshold": -4.7243242263793945, "operator": "ge"}, {"budget": 0.63, "threshold": -4.812397003173828, "operator": "ge"}, {"budget": 0.64, "threshold": -4.884191513061523, "operator": "ge"}, {"budget": 0.65, "threshold": -4.952435493469238, "operator": "ge"}, {"budget": 0.66, "threshold": -5.0384626388549805, "operator": "ge"}, {"budget": 0.67, "threshold": -5.111513137817383, "operator": "ge"}, {"budget": 0.68, "threshold": -5.1843767166137695, "operator": "ge"}, {"budget": 0.69, "threshold": -5.25035285949707, "operator": "ge"}, {"budget": 0.7, "threshold": -5.328371524810791, "operator": "ge"}, {"budget": 0.71, "threshold": -5.400135040283203, "operator": "ge"}, {"budget": 0.72, "threshold": -5.456247806549072, "operator": "ge"}, {"budget": 0.73, "threshold": -5.525753974914551, "operator": "ge"}, {"budget": 0.74, "threshold": -5.595252990722656, "operator": "ge"}, {"budget": 0.75, "threshold": -5.657398223876953, "operator": "ge"}, {"budget": 0.76, "threshold": -5.74017333984375, "operator": "ge"}, {"budget": 0.77, "threshold": -5.806314945220947, "operator": "ge"}, {"budget": 0.78, "threshold": -5.865735054016113, "operator": "ge"}, {"budget": 0.79, "threshold": -5.91912841796875, "operator": "ge"}, {"budget": 0.8, "threshold": -5.986212730407715, "operator": "ge"}, {"budget": 0.81, "threshold": -6.050565242767334, "operator": "ge"}, {"budget": 0.82, "threshold": -6.122864246368408, "operator": "ge"}, {"budget": 0.83, "threshold": -6.181477069854736, "operator": "ge"}, {"budget": 0.84, "threshold": -6.248091220855713, "operator": "ge"}, {"budget": 0.85, "threshold": -6.307513236999512, "operator": "ge"}, {"budget": 0.86, "threshold": -6.386740207672119, "operator": "ge"}, {"budget": 0.87, "threshold": -6.4462127685546875, "operator": "ge"}, {"budget": 0.88, "threshold": -6.521955966949463, "operator": "ge"}, {"budget": 0.89, "threshold": -6.596214771270752, "operator": "ge"}, {"budget": 0.9, "threshold": -6.675212383270264, "operator": "ge"}, {"budget": 0.91, "threshold": -6.775523662567139, "operator": "ge"}, {"budget": 0.92, "threshold": -6.873295307159424, "operator": "ge"}, {"budget": 0.93, "threshold": -6.973400115966797, "operator": "ge"}, {"budget": 0.94, "threshold": -7.107568264007568, "operator": "ge"}, {"budget": 0.95, "threshold": -7.2061767578125, "operator": "ge"}, {"budget": 0.96, "threshold": -7.346907138824463, "operator": "ge"}, {"budget": 0.97, "threshold": -7.493220329284668, "operator": "ge"}, {"budget": 0.98, "threshold": -7.663825035095215, "operator": "ge"}, {"budget": 0.99, "threshold": -7.879819869995117, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.9954167860212, 76.35348037811515, 76.74018905757663, 77.11257519335433, 77.42767115439702, 77.85734746490976, 78.17244342595244, 78.47321684331136, 78.75966771698654, 79.01747350329418, 79.3182469206531, 79.69063305643081, 79.93411629905471, 80.2778573474649, 80.53566313377254, 80.76482383271268, 80.99398453165281, 81.09424233743913, 81.36637066743053, 81.66714408478946, 81.98224004583213, 82.2686909195073, 82.59810942423374, 82.82727012317388, 83.19965625895158, 83.34288169578917, 83.61501002578058, 83.88713835577198, 84.04468633629332, 84.30249212260097, 84.51733027785735, 84.58894299627615, 84.66055571469494, 84.86107132626755, 85.07590948152391, 85.21913491836149, 85.53423087940418, 85.77771412202807, 85.9066170151819, 86.04984245201948, 86.27900315095961, 86.49384130621598, 86.6227441993698, 86.78029217989115, 86.86622744199369, 86.99513033514752, 87.03809796619879, 87.08106559725006, 87.05242050988255, 87.1383557719851, 87.16700085935261, 87.26725866513893, 87.42480664566027, 87.49641936407906, 87.61099971354912, 87.71125751933543, 87.86880549985678, 87.96906330564308, 88.06932111142939, 88.18390146089945, 88.28415926668576, 88.32712689773703, 88.52764250930964, 88.55628759667717, 88.62790031509596, 88.65654540246346, 88.69951303351475, 88.7854482956173, 88.8857061014036, 88.92867373245488, 88.95731881982239, 89.10054425665999, 89.10054425665999, 89.21512460613005, 89.31538241191636, 89.37267258665138, 89.41564021770266, 89.41564021770266, 89.42996276138642, 89.47293039243769, 89.573188198224, 89.61615582927529, 89.70209109137782, 89.8166714408479, 89.95989687768548, 90.01718705242051, 90.10312231452305, 90.10312231452305, 90.13176740189057, 90.16041248925809, 90.24634775136064, 90.24634775136064, 90.33228301346318, 90.33228301346318, 90.3609281008307, 90.41821827556574, 90.44686336293324, 90.44686336293324, 90.47550845030077, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 31, 59, 86, 115, 142, 173, 198, 222, 247, 272, 303, 330, 355, 384, 409, 431, 449, 464, 489, 516, 541, 570, 597, 620, 649, 666, 689, 713, 731, 752, 773, 788, 801, 823, 846, 859, 885, 907, 924, 939, 958, 976, 992, 1006, 1018, 1031, 1040, 1047, 1054, 1064, 1072, 1085, 1097, 1106, 1115, 1124, 1138, 1148, 1158, 1167, 1181, 1187, 1201, 1209, 1217, 1226, 1232, 1239, 1248, 1253, 1262, 1274, 1277, 1288, 1298, 1304, 1310, 1314, 1316, 1320, 1328, 1334, 1341, 1350, 1361, 1368, 1378, 1382, 1386, 1392, 1402, 1404, 1413, 1415, 1420, 1428, 1434, 1438, 1444, 1449], "harmed": [0, 1, 4, 4, 7, 12, 13, 16, 19, 24, 31, 41, 42, 50, 55, 62, 68, 70, 78, 84, 90, 93, 102, 106, 113, 116, 123, 127, 132, 139, 142, 148, 158, 166, 174, 182, 185, 189, 194, 202, 207, 210, 213, 220, 223, 229, 233, 239, 243, 252, 256, 262, 268, 269, 273, 274, 276, 279, 282, 285, 286, 293, 296, 296, 302, 305, 312, 315, 316, 318, 320, 327, 329, 332, 335, 338, 340, 343, 347, 348, 349, 350, 353, 354, 355, 356, 359, 363, 367, 369, 373, 377, 379, 382, 384, 387, 391, 395, 399, 403, 406], "exact_score": [75.56574047550845, 75.9954167860212, 76.35348037811515, 76.74018905757663, 77.11257519335433, 77.42767115439702, 77.85734746490976, 78.17244342595244, 78.47321684331136, 78.75966771698654, 79.01747350329418, 79.3182469206531, 79.69063305643081, 79.93411629905471, 80.2778573474649, 80.53566313377254, 80.76482383271268, 80.99398453165281, 81.09424233743913, 81.36637066743053, 81.66714408478946, 81.98224004583213, 82.2686909195073, 82.59810942423374, 82.82727012317388, 83.19965625895158, 83.34288169578917, 83.61501002578058, 83.88713835577198, 84.04468633629332, 84.30249212260097, 84.51733027785735, 84.58894299627615, 84.66055571469494, 84.86107132626755, 85.07590948152391, 85.21913491836149, 85.53423087940418, 85.77771412202807, 85.9066170151819, 86.04984245201948, 86.27900315095961, 86.49384130621598, 86.6227441993698, 86.78029217989115, 86.86622744199369, 86.99513033514752, 87.03809796619879, 87.08106559725006, 87.05242050988255, 87.1383557719851, 87.16700085935261, 87.26725866513893, 87.42480664566027, 87.49641936407906, 87.61099971354912, 87.71125751933543, 87.86880549985678, 87.96906330564308, 88.06932111142939, 88.18390146089945, 88.28415926668576, 88.32712689773703, 88.52764250930964, 88.55628759667717, 88.62790031509596, 88.65654540246346, 88.69951303351475, 88.7854482956173, 88.8857061014036, 88.92867373245488, 88.95731881982239, 89.10054425665999, 89.10054425665999, 89.21512460613005, 89.31538241191636, 89.37267258665138, 89.41564021770266, 89.41564021770266, 89.42996276138642, 89.47293039243769, 89.573188198224, 89.61615582927529, 89.70209109137782, 89.8166714408479, 89.95989687768548, 90.01718705242051, 90.10312231452305, 90.10312231452305, 90.13176740189057, 90.16041248925809, 90.24634775136064, 90.24634775136064, 90.33228301346318, 90.33228301346318, 90.3609281008307, 90.41821827556574, 90.44686336293324, 90.44686336293324, 90.47550845030077, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.85633631219814, "threshold_area100": 85.85527069607564, "threshold_area25": 79.60613004869666}, "test": {"score": [76.42247255506925, 76.98213388821124, 77.29066513596901, 77.61354667432016, 77.96512879385807, 78.40281265695631, 78.65394274234053, 78.94812369950492, 79.12750233192223, 79.45755901557007, 79.78044055392121, 80.10332209227236, 80.48360479299707, 80.66298342541437, 80.89976322020522, 81.12936786969937, 81.41637368156705, 81.66750376695128, 81.94733443352227, 82.18411422831312, 82.42806916840067, 82.75812585204851, 82.99490564683936, 83.13840855277319, 83.4110640740475, 83.59044270646481, 83.80569706536558, 84.0711774413432, 84.35100810791418, 84.5375618856282, 84.85326827868265, 85.0828729281768, 85.19050010762719, 85.31965272296765, 85.4559804836048, 85.59948338953865, 85.77886202195594, 85.92236492788979, 86.10891870560378, 86.38874937217479, 86.61117887637225, 86.86230896175647, 86.99863672239363, 87.04886273947048, 87.1206141924374, 87.27129224366794, 87.40762000430509, 87.47219631197531, 87.5941737820191, 87.69462581617277, 87.73767668795294, 87.84530386740332, 87.88117959388678, 87.97445648274378, 88.02468249982063, 88.06773337160078, 88.18253569634786, 88.26146229461148, 88.26146229461148, 88.31168831168831, 88.37626461935855, 88.55564325177585, 88.62021955944608, 88.73502188419316, 88.80677333716008, 88.84264906364353, 88.92157566190716, 88.94310109779724, 88.99332711487408, 89.0722537131377, 89.17270574729139, 89.2659826361484, 89.30903350792853, 89.31620865322523, 89.3520843797087, 89.41666068737892, 89.48841214034584, 89.531463012126, 89.57451388390615, 89.58886417449953, 89.68931620865322, 89.71801678984, 89.76824280691685, 89.82564396929038, 89.82564396929038, 89.82564396929038, 89.83281911458707, 89.86869484107054, 89.91892085814737, 89.95479658463084, 89.97632202052091, 89.93327114874077, 89.99067231111431, 89.98349716581761, 90.02654803759776, 90.01219774700438, 90.05524861878453, 90.04089832819116, 90.04089832819116, 90.04089832819116, 90.06242376408122], "calls": [0.0, 1.363277606371529, 2.3247470761282916, 3.200114802324747, 4.3409629044988165, 5.316782664848963, 6.170624955155342, 6.967066083088183, 7.7850326469111, 8.732151826074478, 9.60034440697424, 10.547463586137619, 11.551983927674534, 12.413001363277607, 13.245318217693908, 14.264188849824208, 15.268709191361124, 16.101026045777427, 17.11989667790773, 17.91633780584057, 18.956733873860944, 20.10475712133171, 21.016000574011624, 22.006170624955153, 23.276171342469684, 24.417019444643753, 25.500466384444287, 26.454760708904356, 27.674535409342038, 28.69340604147234, 29.712276673602638, 31.07555427997417, 31.785893664346705, 32.66126139054315, 33.62990600559662, 34.69900265480376, 35.753749013417526, 36.722393628470975, 37.57623591877736, 38.62380713209442, 39.994259883762645, 40.93420391762933, 42.0750520198034, 42.87866829303294, 43.86883834397646, 44.95946042907369, 45.8778790270503, 46.78912247973022, 47.64296477003659, 48.597259094496664, 49.56590370955012, 50.383870273373034, 51.266413144866185, 52.2709334864031, 53.13195092200617, 53.91404175934563, 54.91856210088254, 56.17421252780369, 57.0854559804836, 57.91777283489991, 58.80749085168975, 60.02726555212743, 61.01743560307096, 62.17980914113511, 63.19150462796872, 63.9448948841214, 65.22924589222932, 66.21941594317285, 67.35308890005022, 68.3002080792136, 69.55585850613475, 70.69670660830883, 71.52902346272512, 72.51201836837195, 73.56676472698572, 74.46365788907225, 75.50405395709264, 76.70230322164024, 77.52026978546316, 78.45303867403315, 79.40733299849322, 80.36162732295328, 81.41637368156705, 82.49264547607089, 83.29626174930043, 84.24338092846381, 85.52055679127503, 86.30264762861448, 87.40762000430509, 88.29016287579825, 89.35925952500538, 90.60773480662984, 91.71270718232044, 92.65265121618714, 94.0589796943388, 94.94152256583196, 96.07519552270934, 97.02231470187272, 97.94790844514601, 98.84480160723254, 100.0], "rescued": [0, 85, 134, 181, 236, 302, 345, 394, 428, 481, 533, 588, 652, 686, 732, 772, 823, 866, 915, 952, 998, 1055, 1098, 1136, 1192, 1233, 1273, 1318, 1372, 1414, 1467, 1520, 1545, 1574, 1606, 1642, 1685, 1716, 1751, 1797, 1848, 1892, 1922, 1941, 1966, 1998, 2026, 2045, 2070, 2090, 2104, 2127, 2143, 2162, 2177, 2189, 2209, 2229, 2240, 2256, 2271, 2300, 2317, 2342, 2358, 2371, 2391, 2404, 2418, 2432, 2455, 2478, 2490, 2499, 2510, 2526, 2540, 2557, 2569, 2578, 2597, 2607, 2620, 2633, 2642, 2650, 2668, 2677, 2689, 2700, 2709, 2714, 2728, 2735, 2749, 2753, 2765, 2769, 2775, 2783, 2791], "harmed": [0, 7, 13, 15, 21, 26, 34, 42, 51, 58, 65, 75, 86, 95, 108, 116, 127, 135, 145, 149, 161, 172, 182, 200, 218, 234, 244, 252, 267, 283, 292, 313, 323, 334, 347, 363, 381, 392, 401, 408, 428, 437, 448, 460, 475, 486, 495, 505, 513, 519, 527, 535, 546, 552, 560, 566, 570, 579, 590, 599, 605, 609, 617, 626, 632, 640, 649, 659, 666, 669, 678, 688, 694, 702, 708, 715, 719, 730, 736, 743, 748, 754, 760, 765, 774, 782, 799, 803, 808, 814, 820, 831, 837, 845, 853, 859, 865, 871, 877, 885, 890], "exact_score": [76.42247255506925, 76.86015641816746, 77.21173853770539, 77.52026978546316, 77.8646767597044, 78.2306091698357, 78.61089187056038, 78.95529884480162, 79.18490349429577, 79.55083590442706, 79.95981918633852, 80.23247470761284, 80.56253139126068, 80.84236205783168, 81.07196670732583, 81.3733228097869, 81.61727774987443, 81.91145870703882, 82.21998995479659, 82.44241945899404, 82.70789983497167, 82.99490564683936, 83.1312334074765, 83.33213747578388, 83.51869125349789, 83.72677046710196, 83.93484968070604, 84.23620578316711, 84.4227595608811, 84.65236421037527, 84.92501973164957, 85.05417234699003, 85.25507641529741, 85.39857932123127, 85.48468106479157, 85.62818397072542, 85.81473774843941, 85.96541579966994, 86.20219559446079, 86.42462509865825, 86.61117887637225, 86.86230896175647, 86.98428643180026, 87.0416875941738, 87.13496448303079, 87.27129224366794, 87.41479514960177, 87.4937217478654, 87.61569921790917, 87.69462581617277, 87.75920212384301, 87.86682930329339, 87.945755901557, 88.03903279041401, 88.07490851689747, 88.17536055105116, 88.26146229461148, 88.25428714931478, 88.31886345698501, 88.412140345842, 88.55564325177585, 88.62021955944608, 88.72784673889646, 88.78524790127, 88.84264906364353, 88.8856999354237, 88.95745138839062, 88.97897682428069, 89.05072827724761, 89.13683002080792, 89.21575661907154, 89.2659826361484, 89.30185836263185, 89.3377340891153, 89.39513525148884, 89.45971155915908, 89.50993757623593, 89.55298844801607, 89.59603931979623, 89.65344048216977, 89.73236708043338, 89.73236708043338, 89.79694338810361, 89.82564396929038, 89.83281911458707, 89.85434455047715, 89.84716940518047, 89.8973954222573, 89.94762143933414, 89.98349716581761, 89.93327114874077, 89.94762143933414, 89.98349716581761, 89.99067231111431, 90.02654803759776, 90.02654803759776, 90.05524861878453, 90.04807347348785, 90.03372318289445, 90.04807347348785, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.26929815871213, "threshold_area100": 86.215864246251, "threshold_area25": 80.34942957594892}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.4452896118164062, "operator": "ge"}, {"budget": 0.02, "threshold": 1.2871097326278687, "operator": "ge"}, {"budget": 0.03, "threshold": 1.168473720550537, "operator": "ge"}, {"budget": 0.04, "threshold": 1.0762979984283447, "operator": "ge"}, {"budget": 0.05, "threshold": 0.9939413666725159, "operator": "ge"}, {"budget": 0.06, "threshold": 0.93387770652771, "operator": "ge"}, {"budget": 0.07, "threshold": 0.8774884939193726, "operator": "ge"}, {"budget": 0.08, "threshold": 0.8274877071380615, "operator": "ge"}, {"budget": 0.09, "threshold": 0.7596433162689209, "operator": "ge"}, {"budget": 0.1, "threshold": 0.7117515802383423, "operator": "ge"}, {"budget": 0.11, "threshold": 0.6641146540641785, "operator": "ge"}, {"budget": 0.12, "threshold": 0.6209366321563721, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5662293434143066, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5057137608528137, "operator": "ge"}, {"budget": 0.15, "threshold": 0.44989582896232605, "operator": "ge"}, {"budget": 0.16, "threshold": 0.38894763588905334, "operator": "ge"}, {"budget": 0.17, "threshold": 0.3236929774284363, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2624456286430359, "operator": "ge"}, {"budget": 0.19, "threshold": 0.20049333572387695, "operator": "ge"}, {"budget": 0.2, "threshold": 0.13536791503429413, "operator": "ge"}, {"budget": 0.21, "threshold": 0.04496502876281738, "operator": "ge"}, {"budget": 0.22, "threshold": -0.040803372859954834, "operator": "ge"}, {"budget": 0.23, "threshold": -0.1353825330734253, "operator": "ge"}, {"budget": 0.24, "threshold": -0.238930344581604, "operator": "ge"}, {"budget": 0.25, "threshold": -0.33000630140304565, "operator": "ge"}, {"budget": 0.26, "threshold": -0.4498171806335449, "operator": "ge"}, {"budget": 0.27, "threshold": -0.5376562476158142, "operator": "ge"}, {"budget": 0.28, "threshold": -0.6326106786727905, "operator": "ge"}, {"budget": 0.29, "threshold": -0.7282113432884216, "operator": "ge"}, {"budget": 0.3, "threshold": -0.8242640495300293, "operator": "ge"}, {"budget": 0.31, "threshold": -0.9397178888320923, "operator": "ge"}, {"budget": 0.32, "threshold": -1.0448179244995117, "operator": "ge"}, {"budget": 0.33, "threshold": -1.1400885581970215, "operator": "ge"}, {"budget": 0.34, "threshold": -1.2331161499023438, "operator": "ge"}, {"budget": 0.35, "threshold": -1.3455913066864014, "operator": "ge"}, {"budget": 0.36, "threshold": -1.462592601776123, "operator": "ge"}, {"budget": 0.37, "threshold": -1.5724906921386719, "operator": "ge"}, {"budget": 0.38, "threshold": -1.6760666370391846, "operator": "ge"}, {"budget": 0.39, "threshold": -1.7885229587554932, "operator": "ge"}, {"budget": 0.4, "threshold": -1.9131793975830078, "operator": "ge"}, {"budget": 0.41, "threshold": -2.027509927749634, "operator": "ge"}, {"budget": 0.42, "threshold": -2.1491007804870605, "operator": "ge"}, {"budget": 0.43, "threshold": -2.2914609909057617, "operator": "ge"}, {"budget": 0.44, "threshold": -2.426819324493408, "operator": "ge"}, {"budget": 0.45, "threshold": -2.5460124015808105, "operator": "ge"}, {"budget": 0.46, "threshold": -2.6787335872650146, "operator": "ge"}, {"budget": 0.47, "threshold": -2.833404302597046, "operator": "ge"}, {"budget": 0.48, "threshold": -2.9504427909851074, "operator": "ge"}, {"budget": 0.49, "threshold": -3.0815234184265137, "operator": "ge"}, {"budget": 0.5, "threshold": -3.2123143672943115, "operator": "ge"}, {"budget": 0.51, "threshold": -3.3230605125427246, "operator": "ge"}, {"budget": 0.52, "threshold": -3.4200809001922607, "operator": "ge"}, {"budget": 0.53, "threshold": -3.519496440887451, "operator": "ge"}, {"budget": 0.54, "threshold": -3.627567768096924, "operator": "ge"}, {"budget": 0.55, "threshold": -3.745685338973999, "operator": "ge"}, {"budget": 0.56, "threshold": -3.843869209289551, "operator": "ge"}, {"budget": 0.57, "threshold": -3.930041790008545, "operator": "ge"}, {"budget": 0.58, "threshold": -4.063046932220459, "operator": "ge"}, {"budget": 0.59, "threshold": -4.167261123657227, "operator": "ge"}, {"budget": 0.6, "threshold": -4.24513053894043, "operator": "ge"}, {"budget": 0.61, "threshold": -4.317085266113281, "operator": "ge"}, {"budget": 0.62, "threshold": -4.400759220123291, "operator": "ge"}, {"budget": 0.63, "threshold": -4.479795932769775, "operator": "ge"}, {"budget": 0.64, "threshold": -4.560164928436279, "operator": "ge"}, {"budget": 0.65, "threshold": -4.623470306396484, "operator": "ge"}, {"budget": 0.66, "threshold": -4.697566986083984, "operator": "ge"}, {"budget": 0.67, "threshold": -4.7699151039123535, "operator": "ge"}, {"budget": 0.68, "threshold": -4.8475236892700195, "operator": "ge"}, {"budget": 0.69, "threshold": -4.914820671081543, "operator": "ge"}, {"budget": 0.7, "threshold": -4.974063873291016, "operator": "ge"}, {"budget": 0.71, "threshold": -5.037360191345215, "operator": "ge"}, {"budget": 0.72, "threshold": -5.108020305633545, "operator": "ge"}, {"budget": 0.73, "threshold": -5.172116279602051, "operator": "ge"}, {"budget": 0.74, "threshold": -5.245003700256348, "operator": "ge"}, {"budget": 0.75, "threshold": -5.297847747802734, "operator": "ge"}, {"budget": 0.76, "threshold": -5.368494987487793, "operator": "ge"}, {"budget": 0.77, "threshold": -5.4285359382629395, "operator": "ge"}, {"budget": 0.78, "threshold": -5.48492956161499, "operator": "ge"}, {"budget": 0.79, "threshold": -5.543028354644775, "operator": "ge"}, {"budget": 0.8, "threshold": -5.609044551849365, "operator": "ge"}, {"budget": 0.81, "threshold": -5.68038272857666, "operator": "ge"}, {"budget": 0.82, "threshold": -5.733260154724121, "operator": "ge"}, {"budget": 0.83, "threshold": -5.7967376708984375, "operator": "ge"}, {"budget": 0.84, "threshold": -5.855445861816406, "operator": "ge"}, {"budget": 0.85, "threshold": -5.934520244598389, "operator": "ge"}, {"budget": 0.86, "threshold": -5.995140552520752, "operator": "ge"}, {"budget": 0.87, "threshold": -6.053427219390869, "operator": "ge"}, {"budget": 0.88, "threshold": -6.120926856994629, "operator": "ge"}, {"budget": 0.89, "threshold": -6.213360786437988, "operator": "ge"}, {"budget": 0.9, "threshold": -6.302362442016602, "operator": "ge"}, {"budget": 0.91, "threshold": -6.385165214538574, "operator": "ge"}, {"budget": 0.92, "threshold": -6.490178108215332, "operator": "ge"}, {"budget": 0.93, "threshold": -6.596169948577881, "operator": "ge"}, {"budget": 0.94, "threshold": -6.724307060241699, "operator": "ge"}, {"budget": 0.95, "threshold": -6.847312927246094, "operator": "ge"}, {"budget": 0.96, "threshold": -6.985147953033447, "operator": "ge"}, {"budget": 0.97, "threshold": -7.1170759201049805, "operator": "ge"}, {"budget": 0.98, "threshold": -7.3165411949157715, "operator": "ge"}, {"budget": 0.99, "threshold": -7.5659685134887695, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.00973932970496, 76.42509309653394, 76.82612431967917, 77.19851045545688, 77.45631624176454, 77.6711543970209, 77.97192781437982, 78.17244342595244, 78.51618447436263, 78.96018332855915, 79.30392437696935, 79.70495560011457, 79.96276138642222, 80.2492122600974, 80.49269550272128, 80.70753365797765, 80.87940418218275, 81.12288742480665, 81.38069321111429, 81.69578917215698, 82.01088513319965, 82.2686909195073, 82.55514179318246, 82.89888284159265, 83.15668862790031, 83.28559152105413, 83.58636493841306, 83.82984818103695, 84.07333142366083, 84.30249212260097, 84.51733027785735, 84.57462045259237, 84.73216843311371, 84.88971641363506, 85.0472643941564, 85.34803781151533, 85.54855342308794, 85.7347464909768, 85.9066170151819, 86.02119736465195, 86.19306788885706, 86.3506158693784, 86.56545402463478, 86.73732454883987, 86.96648524778, 86.99513033514752, 87.02377542251504, 87.1383557719851, 87.16700085935261, 87.22429103408766, 87.28158120882267, 87.38183901460899, 87.38183901460899, 87.5107419077628, 87.69693497565167, 87.78287023775422, 87.9260956745918, 88.11228874248066, 88.19822400458321, 88.21254654826697, 88.37009452878831, 88.41306215983958, 88.47035233457461, 88.55628759667717, 88.57061014036093, 88.6135777714122, 88.71383557719851, 88.72815812088226, 88.77112575193354, 88.90002864508737, 89.00028645087367, 89.02893153824118, 89.15783443139502, 89.15783443139502, 89.18647951876252, 89.3297049556001, 89.4013176740189, 89.50157547980521, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.68776854769406, 89.73073617874535, 89.75938126611285, 89.7450587224291, 89.8166714408479, 90.00286450873675, 90.16041248925809, 90.21770266399312, 90.24634775136064, 90.27499283872815, 90.26067029504439, 90.28931538241191, 90.31796046977942, 90.33228301346318, 90.46118590661702, 90.4325408192495, 90.50415353766829, 90.47550845030077, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 32, 64, 94, 122, 145, 167, 193, 212, 243, 276, 306, 337, 360, 388, 410, 429, 447, 469, 493, 518, 543, 569, 595, 623, 645, 660, 686, 708, 729, 749, 770, 784, 802, 821, 840, 866, 884, 901, 918, 934, 952, 966, 986, 1002, 1020, 1029, 1036, 1048, 1057, 1067, 1075, 1085, 1091, 1104, 1118, 1128, 1140, 1154, 1163, 1167, 1181, 1186, 1195, 1205, 1213, 1220, 1230, 1236, 1240, 1251, 1260, 1265, 1276, 1280, 1289, 1300, 1306, 1313, 1319, 1326, 1329, 1334, 1338, 1342, 1346, 1353, 1367, 1379, 1385, 1391, 1396, 1400, 1405, 1412, 1416, 1426, 1431, 1439, 1444, 1449], "harmed": [0, 1, 4, 6, 8, 13, 20, 25, 30, 37, 39, 45, 48, 53, 61, 66, 70, 76, 81, 87, 90, 93, 101, 107, 111, 115, 121, 126, 131, 135, 139, 145, 155, 162, 170, 178, 183, 187, 191, 196, 204, 210, 213, 218, 222, 224, 231, 236, 240, 247, 253, 257, 260, 266, 270, 271, 275, 277, 278, 281, 284, 287, 289, 294, 298, 305, 309, 312, 317, 318, 320, 322, 325, 327, 331, 338, 339, 340, 340, 343, 347, 348, 348, 349, 351, 356, 358, 359, 360, 362, 366, 369, 374, 377, 382, 385, 386, 393, 396, 403, 406], "exact_score": [75.56574047550845, 76.00973932970496, 76.42509309653394, 76.82612431967917, 77.19851045545688, 77.45631624176454, 77.6711543970209, 77.97192781437982, 78.17244342595244, 78.51618447436263, 78.96018332855915, 79.30392437696935, 79.70495560011457, 79.96276138642222, 80.2492122600974, 80.49269550272128, 80.70753365797765, 80.87940418218275, 81.12288742480665, 81.38069321111429, 81.69578917215698, 82.01088513319965, 82.2686909195073, 82.55514179318246, 82.89888284159265, 83.15668862790031, 83.28559152105413, 83.58636493841306, 83.82984818103695, 84.07333142366083, 84.30249212260097, 84.51733027785735, 84.57462045259237, 84.73216843311371, 84.88971641363506, 85.0472643941564, 85.34803781151533, 85.54855342308794, 85.7347464909768, 85.9066170151819, 86.02119736465195, 86.19306788885706, 86.3506158693784, 86.56545402463478, 86.73732454883987, 86.96648524778, 86.99513033514752, 87.02377542251504, 87.1383557719851, 87.16700085935261, 87.22429103408766, 87.28158120882267, 87.38183901460899, 87.38183901460899, 87.5107419077628, 87.69693497565167, 87.78287023775422, 87.9260956745918, 88.11228874248066, 88.19822400458321, 88.21254654826697, 88.37009452878831, 88.41306215983958, 88.47035233457461, 88.55628759667717, 88.57061014036093, 88.6135777714122, 88.71383557719851, 88.72815812088226, 88.77112575193354, 88.90002864508737, 89.00028645087367, 89.02893153824118, 89.15783443139502, 89.15783443139502, 89.18647951876252, 89.3297049556001, 89.4013176740189, 89.50157547980521, 89.5445431108565, 89.58751074190776, 89.61615582927529, 89.68776854769406, 89.73073617874535, 89.75938126611285, 89.7450587224291, 89.8166714408479, 90.00286450873675, 90.16041248925809, 90.21770266399312, 90.24634775136064, 90.27499283872815, 90.26067029504439, 90.28931538241191, 90.31796046977942, 90.33228301346318, 90.46118590661702, 90.4325408192495, 90.50415353766829, 90.47550845030077, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.85315569002947, "threshold_area100": 85.8521197364652, "threshold_area25": 79.59323975938126}, "test": {"score": [76.42247255506925, 76.917557580541, 77.27631484537562, 77.69247327258377, 78.02970510152831, 78.38128722106623, 78.68981846882399, 78.92659826361485, 79.24947980196599, 79.54366075913038, 79.87371744277823, 80.12484752816245, 80.32575159646983, 80.64145798952428, 80.89976322020522, 81.15089330558945, 81.45224940805052, 81.7392552199182, 81.91145870703882, 82.18411422831312, 82.3850182966205, 82.72225012556504, 82.98773050154266, 83.18863456985004, 83.36801320226735, 83.6047929970582, 83.83439764655235, 84.04965200545311, 84.29360694554065, 84.55191217622158, 84.78151682571571, 85.0900480734735, 85.22637583411064, 85.39857932123127, 85.52055679127503, 85.64253426131879, 85.77886202195594, 85.95824065437326, 86.12326899619718, 86.34569850039465, 86.63270431226233, 86.79773265408625, 86.94123556002009, 87.09908875654733, 87.2138910812944, 87.24259166248117, 87.36456913252493, 87.50089689316209, 87.55112291023893, 87.72332639735954, 87.82377843151325, 87.84530386740332, 87.90270502977687, 87.97445648274378, 88.03185764511731, 88.12513453397432, 88.15383511516109, 88.21123627753462, 88.37626461935855, 88.46954150821554, 88.5125923799957, 88.57716868766593, 88.66327043122624, 88.75654732008323, 88.76372246537993, 88.84982420894023, 88.89287508072039, 88.96462653368731, 89.03637798665423, 89.12965487551124, 89.158355456698, 89.17988089258809, 89.28750807203846, 89.30185836263185, 89.38078496089545, 89.4238358326756, 89.46688670445577, 89.54581330271938, 89.6175647556863, 89.67496591805984, 89.73954222573008, 89.76824280691685, 89.73236708043338, 89.8041185334003, 89.82564396929038, 89.84716940518047, 89.83999425988375, 89.86151969577384, 89.8902202769606, 89.94762143933414, 89.99067231111431, 90.04089832819116, 90.04089832819116, 90.0982994905647, 90.10547463586138, 90.09112434526799, 90.07677405467462, 90.0839491999713, 90.07677405467462, 90.05524861878453, 90.06242376408122], "calls": [0.0, 1.363277606371529, 2.346272512018368, 3.3651431441486688, 4.312262323312047, 5.345483246035732, 6.106048647485111, 6.89531463012126, 7.878309535768099, 8.904355313195092, 9.758197603501472, 10.590514457917772, 11.171701226949846, 12.161871277893377, 13.130515892946832, 14.005883619143288, 15.031929396570282, 16.179952644041045, 17.026619789050727, 18.002439549400876, 18.920858147377484, 20.05453110425486, 21.03035086460501, 22.020520915548538, 23.268996197172992, 24.287866829303294, 25.629618999784746, 26.52651216187128, 27.444930759847885, 28.44227595608811, 29.532898041185334, 30.723972160436247, 31.843294826720243, 32.912391475927386, 33.70165745856354, 34.799454688957454, 35.79679988519768, 36.858721389108126, 37.7197388247112, 38.65968285857789, 39.857932123125494, 40.7835258663988, 41.79522135323241, 42.88584343832962, 43.98364066872354, 44.72985577957954, 45.69850039463299, 46.88957451388391, 47.74341680419029, 48.6546602568702, 49.601779436033574, 50.46997201693334, 51.266413144866185, 52.15613116165603, 53.1032503408194, 54.00731864820263, 54.767884049652004, 55.650426921145154, 56.94195307454976, 58.054100595537065, 58.879242304656664, 59.840711774413435, 61.10353734663126, 62.20133457702518, 63.205854918562096, 64.19602496950563, 65.18619502044916, 66.19071536198608, 67.36743919064361, 68.4867618569276, 69.57020879672814, 70.54602855707827, 71.66535122336228, 72.7487981631628, 74.08337518834756, 74.6932625385664, 75.77670947836694, 76.63772691397, 77.49156920427639, 78.38128722106623, 79.48625959675684, 80.61993255363421, 81.45224940805052, 82.34914257013705, 83.19580971514672, 84.30795723613403, 85.18332496233049, 86.08021812441702, 87.02016215828371, 88.20406113223794, 89.43818612326899, 90.56468393484968, 91.77010834469398, 92.88943101097797, 94.20248260027265, 95.23570352299635, 96.27609959101672, 97.08689100954294, 98.08423620578317, 98.95242878668293, 100.0], "rescued": [0, 78, 133, 199, 255, 312, 359, 401, 457, 509, 563, 607, 643, 693, 742, 784, 832, 882, 913, 960, 996, 1050, 1097, 1139, 1185, 1226, 1274, 1315, 1356, 1402, 1448, 1503, 1542, 1581, 1608, 1640, 1676, 1716, 1747, 1789, 1840, 1875, 1906, 1939, 1970, 1986, 2014, 2043, 2059, 2091, 2110, 2121, 2139, 2157, 2174, 2191, 2200, 2215, 2243, 2265, 2278, 2295, 2318, 2338, 2349, 2366, 2377, 2395, 2414, 2431, 2443, 2456, 2479, 2490, 2512, 2523, 2535, 2550, 2567, 2579, 2591, 2604, 2611, 2623, 2632, 2644, 2652, 2659, 2670, 2685, 2703, 2717, 2728, 2742, 2750, 2757, 2763, 2769, 2777, 2781, 2791], "harmed": [0, 9, 14, 22, 31, 39, 43, 52, 63, 74, 82, 91, 99, 105, 118, 125, 131, 141, 148, 157, 165, 172, 182, 196, 217, 225, 241, 252, 259, 269, 283, 295, 315, 330, 340, 355, 372, 387, 395, 406, 417, 429, 440, 451, 466, 478, 489, 499, 508, 516, 521, 529, 539, 547, 556, 560, 565, 572, 577, 586, 593, 601, 612, 619, 629, 634, 639, 647, 656, 660, 668, 678, 686, 695, 706, 711, 717, 721, 728, 732, 735, 744, 756, 758, 764, 773, 782, 786, 793, 800, 812, 819, 830, 836, 843, 852, 860, 865, 874, 881, 890], "exact_score": [76.42247255506925, 76.77405467460716, 77.13998708473846, 77.5848460931334, 77.92925306737462, 78.25930975102247, 78.63959245174715, 78.95529884480162, 79.26383009255937, 79.58671163091053, 79.95264404104184, 80.2611752887996, 80.59840711774415, 80.84236205783168, 81.15089330558945, 81.43789911745714, 81.68185405754467, 81.91145870703882, 82.18411422831312, 82.40654373251058, 82.70789983497167, 82.98055535624597, 83.18863456985004, 83.3249623304872, 83.55456697998135, 83.74112075769534, 83.93484968070604, 84.19315491138696, 84.44428499677119, 84.65953935567195, 84.91784458635287, 85.13309894525365, 85.22637583411064, 85.41292961182464, 85.58513309894525, 85.67840998780225, 85.78603716725264, 85.98694123556002, 86.14479443208725, 86.43180024395494, 86.64705460285572, 86.82643323527302, 86.97711128650354, 87.09908875654733, 87.22106622659109, 87.29999282485471, 87.37174427782163, 87.52242232905216, 87.60134892731578, 87.80225299562316, 87.83095357680993, 87.88835473918347, 87.96728133744709, 88.03185764511731, 88.12513453397432, 88.18253569634786, 88.24711200401808, 88.36908947406185, 88.47671665351223, 88.505417234699, 88.59151897825932, 88.64174499533615, 88.74219702948984, 88.74219702948984, 88.84264906364353, 88.878524790127, 88.93592595250054, 89.03637798665423, 89.07942885843438, 89.13683002080792, 89.16553060199469, 89.20858147377486, 89.2659826361484, 89.30903350792853, 89.38796010619215, 89.4310109779723, 89.50993757623593, 89.56016359331277, 89.66061562746646, 89.69649135394992, 89.76824280691685, 89.73954222573008, 89.78976824280693, 89.811293678697, 89.85434455047715, 89.84716940518047, 89.86151969577384, 89.8902202769606, 89.96197172992754, 89.98349716581761, 90.03372318289445, 90.03372318289445, 90.07677405467462, 90.10547463586138, 90.0982994905647, 90.09112434526799, 90.07677405467462, 90.07677405467462, 90.07677405467462, 90.04089832819116, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.28916739582058, "threshold_area100": 86.23186482026262, "threshold_area25": 80.36090980842363}, "selected_budget": 98, "selected": {"val": {"score": 90.50415353766829, "calls": 97.99484388427385, "rescued": 1439, "harmed": 396}, "test": {"score": 90.07677405467462, "calls": 98.08423620578317, "rescued": 2777, "harmed": 874}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.64737897450587, "calls": 97.99484388427385}, "test": {"score": 90.37813015713569, "calls": 97.76852981272872}}, "old_normalized_interior": {"val": {"score": 90.48983099398453, "calls": 92.99627613864222}, "test": {"score": 89.95479658463084, "calls": 92.75310325034081}}, "old_normalized_endpoints": {"val": {"score": 90.50415353766829, "calls": 100.0}, "test": {"score": 90.06242376408122, "calls": 100.0}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 76.03838441707246, 76.33915783443139, 76.74018905757663, 77.0552850186193, 77.47063878544829, 77.9146376396448, 78.20108851331996, 78.61644228014895, 78.85992552277284, 79.08908622171298, 79.37553709538814, 79.70495560011457, 80.03437410484102, 80.23488971641363, 80.50701804640504, 80.82211400744772, 81.07991979375538, 81.33772558006301, 81.63849899742193, 81.86765969636207, 82.15411057003723, 82.35462618160985, 82.48352907476367, 82.75565740475508, 83.01346319106273, 83.17101117158407, 83.28559152105413, 83.61501002578058, 83.7725580063019, 83.94442853050701, 84.11629905471212, 84.34545975365224, 84.4313950157548, 84.67487825837868, 84.86107132626755, 84.96132913205385, 85.11887711257519, 85.44829561730162, 85.5628759667717, 85.69177885992552, 85.82068175307934, 85.99255227728445, 86.15010025780579, 86.30764823832713, 86.52248639358349, 86.72300200515612, 86.89487252936121, 87.02377542251504, 87.10971068461758, 87.09538814093382, 87.19564594672013, 87.39616155829275, 87.43912918934402, 87.63964480091664, 87.81151532512173, 87.95474076195933, 88.05499856774563, 88.06932111142939, 88.22686909195072, 88.41306215983958, 88.37009452878831, 88.39873961615584, 88.41306215983958, 88.41306215983958, 88.5133199656259, 88.64222285877972, 88.71383557719851, 88.79977083930106, 88.94299627613864, 88.94299627613864, 89.02893153824118, 89.14351188771126, 89.21512460613005, 89.22944714981381, 89.3297049556001, 89.35835004296763, 89.42996276138642, 89.60183328559151, 89.6448009166428, 89.71641363506158, 89.80234889716414, 89.85963907189917, 89.88828415926669, 89.97421942136923, 90.07447722715554, 90.07447722715554, 90.17473503294184, 90.18905757662561, 90.21770266399312, 90.31796046977942, 90.40389573188197, 90.46118590661702, 90.48983099398453, 90.4325408192495, 90.41821827556574, 90.41821827556574, 90.46118590661702, 90.41821827556574, 90.4325408192495, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 76.03838441707246, 76.33915783443139, 76.74018905757663, 77.0552850186193, 77.47063878544829, 77.9146376396448, 78.20108851331996, 78.61644228014895, 78.85992552277284, 79.08908622171298, 79.37553709538814, 79.70495560011457, 80.03437410484102, 80.23488971641363, 80.50701804640504, 80.82211400744772, 81.07991979375538, 81.33772558006301, 81.63849899742193, 81.86765969636207, 82.15411057003723, 82.35462618160985, 82.48352907476367, 82.75565740475508, 83.01346319106273, 83.17101117158407, 83.28559152105413, 83.61501002578058, 83.7725580063019, 83.94442853050701, 84.11629905471212, 84.34545975365224, 84.4313950157548, 84.67487825837868, 84.86107132626755, 84.96132913205385, 85.11887711257519, 85.44829561730162, 85.5628759667717, 85.69177885992552, 85.82068175307934, 85.99255227728445, 86.15010025780579, 86.30764823832713, 86.52248639358349, 86.72300200515612, 86.89487252936121, 87.02377542251504, 87.10971068461758, 87.09538814093382, 87.19564594672013, 87.39616155829275, 87.43912918934402, 87.63964480091664, 87.81151532512173, 87.95474076195933, 88.05499856774563, 88.06932111142939, 88.22686909195072, 88.41306215983958, 88.37009452878831, 88.39873961615584, 88.41306215983958, 88.41306215983958, 88.5133199656259, 88.64222285877972, 88.71383557719851, 88.79977083930106, 88.94299627613864, 88.94299627613864, 89.02893153824118, 89.14351188771126, 89.21512460613005, 89.22944714981381, 89.3297049556001, 89.35835004296763, 89.42996276138642, 89.60183328559151, 89.6448009166428, 89.71641363506158, 89.80234889716414, 89.85963907189917, 89.88828415926669, 89.97421942136923, 90.07447722715554, 90.07447722715554, 90.17473503294184, 90.18905757662561, 90.21770266399312, 90.31796046977942, 90.40389573188197, 90.46118590661702, 90.48983099398453, 90.4325408192495, 90.41821827556574, 90.41821827556574, 90.46118590661702, 90.41821827556574, 90.4325408192495, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.6448009166428, "exact_area100": 85.84029370937964}, "test": {"score": [76.42247255506925, 76.96060845232117, 77.37676687952931, 77.73552414436392, 78.07993111860515, 78.409987802253, 78.65394274234053, 78.86919710124131, 79.14185262251561, 79.43603357967999, 79.67998851976753, 80.03874578460214, 80.34727703235991, 80.71320944249122, 81.02174069024898, 81.28722106622659, 81.53835115161083, 81.68185405754467, 81.94015928822559, 82.1267130659396, 82.44959460429074, 82.71507498026835, 83.03795651861951, 83.28908660400374, 83.57609241587143, 83.7841716294755, 83.90614909951927, 84.11422831312333, 84.31513238143073, 84.39405897969434, 84.58061275740833, 84.8102174069025, 85.06852263758341, 85.24072612470403, 85.51338164597834, 85.66405969720887, 85.88648920140633, 86.05151754323025, 86.11609385090048, 86.26677190213103, 86.43897538925164, 86.54660256870201, 86.61117887637225, 86.79773265408625, 86.98428643180026, 87.10626390184402, 87.2138910812944, 87.36456913252493, 87.51524718375548, 87.5798234914257, 87.6731003802827, 87.79507785032648, 87.90270502977687, 88.00315706393054, 88.08208366219417, 88.11078424338093, 88.21123627753462, 88.23993685872139, 88.35473918346847, 88.44084092702877, 88.49824208940231, 88.52694267058908, 88.56999354236923, 88.61304441414939, 88.65609528592955, 88.6847958671163, 88.77807275597331, 88.84982420894023, 88.94310109779724, 89.00050226017078, 89.03637798665423, 89.10812943962115, 89.13683002080792, 89.08660400373107, 89.158355456698, 89.20858147377486, 89.28750807203846, 89.31620865322523, 89.4238358326756, 89.48123699504916, 89.56733873860946, 89.68214106335654, 89.71801678984, 89.78259309751022, 89.8184688239937, 89.87586998636723, 89.88304513166392, 89.8973954222573, 89.904570567554, 89.8902202769606, 89.94762143933414, 89.93327114874077, 89.96197172992754, 89.95479658463084, 89.96914687522423, 89.99067231111431, 90.00502260170768, 90.00502260170768, 90.05524861878453, 90.04089832819116, 90.06242376408122], "calls": [0.0, 1.4278539140417594, 2.303221640238215, 3.422544306522207, 4.549042118102892, 5.582263040826577, 6.407404749946187, 7.096218698428642, 8.043337877592021, 8.9904570567554, 9.679271005237856, 10.762717945038387, 11.645260816531534, 12.764583482815528, 13.72605295257229, 14.587070388175361, 15.527014422042045, 16.26605438760135, 17.22752385735811, 18.038315275884337, 19.171988232761713, 20.3917629331994, 21.625887924230465, 22.50125565042692, 23.591877735524143, 24.653799239434598, 25.486116093850903, 26.461935854201045, 27.55255793929827, 28.56425342613188, 29.482672024108485, 30.40826576738179, 31.35538494654517, 32.367080433378774, 33.54380426203631, 34.55549974886992, 35.4954437827366, 36.413862380713205, 37.05245031211882, 38.3655019014135, 39.37719738824711, 40.3171414221138, 41.09923225945325, 42.25443065222071, 43.35940302791131, 44.32087249766808, 45.33256798450169, 46.51646695845591, 47.549687881179594, 48.34612900911243, 49.1210447011552, 50.333644256296196, 51.3381645978331, 52.184831742842796, 52.98127287077563, 53.95709263112578, 54.983138408552776, 56.07376049364999, 57.21460859582407, 57.96799885197675, 58.879242304656664, 59.94116380856712, 60.823706680060276, 61.792351295113725, 62.94754968788118, 63.83009255937433, 64.9781158068451, 66.01851187486547, 67.1880605582263, 68.49393700222429, 69.60608452321159, 70.59625457415513, 71.80885412929612, 72.85642534261318, 73.86094568415011, 74.82241515390686, 75.69778288010333, 76.44399799095932, 77.26196455478224, 78.316710913396, 79.3284064002296, 80.24682499820621, 81.21546961325967, 82.09801248475281, 83.03795651861951, 84.0640022960465, 85.18332496233049, 86.25242161153763, 87.25694195307454, 88.52694267058908, 89.48123699504916, 90.7297122766736, 91.79163378058406, 92.75310325034081, 93.75762359187773, 94.7190930616345, 95.96756834325895, 96.8357609241587, 97.79723039391548, 98.95960393197963, 100.0], "exact_score": [76.42247255506925, 76.82428069168401, 77.24761426418885, 77.61354667432016, 77.8575016144077, 78.24495946042907, 78.53914041759346, 78.83332137475783, 79.12750233192223, 79.44320872497669, 79.82349142570138, 80.13202267345913, 80.42620363062353, 80.78496089545814, 81.07196670732583, 81.42354882686375, 81.63162804046782, 81.89710841644543, 82.1195379206429, 82.40654373251058, 82.65049867259813, 82.91597904857574, 83.17428427925665, 83.45411494582765, 83.64066872354165, 83.82004735595896, 84.01377627896966, 84.25055607376049, 84.35100810791418, 84.45863528736457, 84.7025902274521, 84.96089545813304, 85.21202554351726, 85.39857932123127, 85.57078280835188, 85.72863600487909, 85.99411638085671, 86.12326899619718, 86.19502044916409, 86.39592451747149, 86.51790198751526, 86.62552916696563, 86.74750663700941, 86.88383439764655, 87.05603788476716, 87.17801535481094, 87.3143431154481, 87.4937217478654, 87.55829805553563, 87.65157494439264, 87.74485183324963, 87.84530386740332, 87.99598191863386, 88.08925880749085, 88.11795938867762, 88.21841142283131, 88.23993685872139, 88.34038889287508, 88.44084092702877, 88.505417234699, 88.52694267058908, 88.55564325177585, 88.61304441414939, 88.65609528592955, 88.6847958671163, 88.77807275597331, 88.84982420894023, 88.96462653368731, 88.99332711487408, 89.03637798665423, 89.065078567841, 89.11530458491785, 89.1440051661046, 89.11530458491785, 89.158355456698, 89.21575661907154, 89.32338379852193, 89.38078496089545, 89.46688670445577, 89.54581330271938, 89.63909019157639, 89.69649135394992, 89.77541795221353, 89.82564396929038, 89.86869484107054, 89.8902202769606, 89.87586998636723, 89.91174571285069, 89.904570567554, 89.904570567554, 89.94762143933414, 89.93327114874077, 89.94762143933414, 89.95479658463084, 89.99067231111431, 90.01219774700438, 90.01219774700438, 90.00502260170768, 90.04807347348785, 90.04089832819116, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 80.38602281696205, "exact_area100": 86.22689236528865}}, "old_normalized_endpoint_budget": 100}, {"seed": 2030, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 1.0, "validation_area100": 0.8597602555525806, "training_seconds": 27.30348897795193, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6650238303033241, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6002982135336561, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5634988926003479, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5342824027107976, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5080915456464113, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4824384863904464, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4577211522415304, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4384396654191203, "operator": "ge"}, {"budget": 0.09, "threshold": 0.42011807721707345, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3984836234550206, "operator": "ge"}, {"budget": 0.11, "threshold": 0.37970855272569665, "operator": "ge"}, {"budget": 0.12, "threshold": 0.36066300868733764, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3418918173558024, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3214512363102797, "operator": "ge"}, {"budget": 0.15, "threshold": 0.2937183980879684, "operator": "ge"}, {"budget": 0.16, "threshold": 0.2706668673015285, "operator": "ge"}, {"budget": 0.17, "threshold": 0.24900493211355157, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2253399095911671, "operator": "ge"}, {"budget": 0.19, "threshold": 0.20233302623393273, "operator": "ge"}, {"budget": 0.2, "threshold": 0.17693272407126454, "operator": "ge"}, {"budget": 0.21, "threshold": 0.13934288496882208, "operator": "ge"}, {"budget": 0.22, "threshold": 0.10520604624541252, "operator": "ge"}, {"budget": 0.23, "threshold": 0.07216492696783675, "operator": "ge"}, {"budget": 0.24, "threshold": 0.03723840399722914, "operator": "ge"}, {"budget": 0.25, "threshold": -0.0021436106309314197, "operator": "ge"}, {"budget": 0.26, "threshold": -0.04965551610808558, "operator": "ge"}, {"budget": 0.27, "threshold": -0.0910104652563973, "operator": "ge"}, {"budget": 0.28, "threshold": -0.13054286930509568, "operator": "ge"}, {"budget": 0.29, "threshold": -0.1698745947947033, "operator": "ge"}, {"budget": 0.3, "threshold": -0.20992371851601285, "operator": "ge"}, {"budget": 0.31, "threshold": -0.24869614937876156, "operator": "ge"}, {"budget": 0.32, "threshold": -0.2935913320526448, "operator": "ge"}, {"budget": 0.33, "threshold": -0.33617399302144524, "operator": "ge"}, {"budget": 0.34, "threshold": -0.37519190394941543, "operator": "ge"}, {"budget": 0.35, "threshold": -0.41097979535133783, "operator": "ge"}, {"budget": 0.36, "threshold": -0.463098527005686, "operator": "ge"}, {"budget": 0.37, "threshold": -0.508242106002382, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5490592747897776, "operator": "ge"}, {"budget": 0.39, "threshold": -0.5809163114738706, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6148834301659655, "operator": "ge"}, {"budget": 0.41, "threshold": -0.6421370723877827, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6760939654903839, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6974598640814285, "operator": "ge"}, {"budget": 0.44, "threshold": -0.7266585730470588, "operator": "ge"}, {"budget": 0.45, "threshold": -0.7521996611907996, "operator": "ge"}, {"budget": 0.46, "threshold": -0.773203617465656, "operator": "ge"}, {"budget": 0.47, "threshold": -0.7921557543122633, "operator": "ge"}, {"budget": 0.48, "threshold": -0.809769843566569, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8265105193557775, "operator": "ge"}, {"budget": 0.5, "threshold": -0.8380314695318375, "operator": "ge"}, {"budget": 0.51, "threshold": -0.8521321710659532, "operator": "ge"}, {"budget": 0.52, "threshold": -0.8626184095025448, "operator": "ge"}, {"budget": 0.53, "threshold": -0.8729685134068486, "operator": "ge"}, {"budget": 0.54, "threshold": -0.8805326185422356, "operator": "ge"}, {"budget": 0.55, "threshold": -0.8880816655489883, "operator": "ge"}, {"budget": 0.56, "threshold": -0.8953384902753707, "operator": "ge"}, {"budget": 0.57, "threshold": -0.9011452182535321, "operator": "ge"}, {"budget": 0.58, "threshold": -0.9072366560357381, "operator": "ge"}, {"budget": 0.59, "threshold": -0.9115688363831453, "operator": "ge"}, {"budget": 0.6, "threshold": -0.9161268309522663, "operator": "ge"}, {"budget": 0.61, "threshold": -0.9208025657652888, "operator": "ge"}, {"budget": 0.62, "threshold": -0.9250375764248449, "operator": "ge"}, {"budget": 0.63, "threshold": -0.9295073016477308, "operator": "ge"}, {"budget": 0.64, "threshold": -0.932879698606348, "operator": "ge"}, {"budget": 0.65, "threshold": -0.9363286095068547, "operator": "ge"}, {"budget": 0.66, "threshold": -0.9392566047237529, "operator": "ge"}, {"budget": 0.67, "threshold": -0.9417990483132223, "operator": "ge"}, {"budget": 0.68, "threshold": -0.9448035078490523, "operator": "ge"}, {"budget": 0.69, "threshold": -0.9477106456218523, "operator": "ge"}, {"budget": 0.7, "threshold": -0.9501729654947431, "operator": "ge"}, {"budget": 0.71, "threshold": -0.9529096633725359, "operator": "ge"}, {"budget": 0.72, "threshold": -0.9551200615251162, "operator": "ge"}, {"budget": 0.73, "threshold": -0.9570972857702436, "operator": "ge"}, {"budget": 0.74, "threshold": -0.9592000111309873, "operator": "ge"}, {"budget": 0.75, "threshold": -0.9612145260082995, "operator": "ge"}, {"budget": 0.76, "threshold": -0.9628437308622081, "operator": "ge"}, {"budget": 0.77, "threshold": -0.9644156504614099, "operator": "ge"}, {"budget": 0.78, "threshold": -0.9663298425509332, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9676677513143245, "operator": "ge"}, {"budget": 0.8, "threshold": -0.969106185387413, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9706833277116035, "operator": "ge"}, {"budget": 0.82, "threshold": -0.9721232973840347, "operator": "ge"}, {"budget": 0.83, "threshold": -0.9735384792268514, "operator": "ge"}, {"budget": 0.84, "threshold": -0.9750591898376313, "operator": "ge"}, {"budget": 0.85, "threshold": -0.9760532499288758, "operator": "ge"}, {"budget": 0.86, "threshold": -0.9773176896667524, "operator": "ge"}, {"budget": 0.87, "threshold": -0.9784712508020792, "operator": "ge"}, {"budget": 0.88, "threshold": -0.9794992490563896, "operator": "ge"}, {"budget": 0.89, "threshold": -0.9808253380706367, "operator": "ge"}, {"budget": 0.9, "threshold": -0.9819508379525997, "operator": "ge"}, {"budget": 0.91, "threshold": -0.982942328976874, "operator": "ge"}, {"budget": 0.92, "threshold": -0.9838741408399201, "operator": "ge"}, {"budget": 0.93, "threshold": -0.9854064295067682, "operator": "ge"}, {"budget": 0.94, "threshold": -0.9864789793249041, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9877365336562957, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9893253496449619, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9909744575120284, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9928973987000463, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9949203409208137, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.33915783443139, 76.76883414494414, 77.15554282440561, 77.51360641649957, 77.84302492122602, 78.12947579490117, 78.40160412489257, 78.75966771698654, 79.08908622171298, 79.30392437696935, 79.56173016327699, 79.81953594958465, 80.06301919220853, 80.36379260956745, 80.59295330850759, 80.80779146376396, 81.15153251217416, 81.45230592953308, 81.71011171584072, 82.0681753079347, 82.49785161844744, 82.78430249212259, 82.98481810369522, 83.19965625895158, 83.35720423947292, 83.70094528788312, 83.87281581208822, 84.07333142366083, 84.23087940418218, 84.33113720996849, 84.46004010312231, 84.7035233457462, 85.0472643941564, 85.17616728731022, 85.46261816098539, 85.69177885992552, 85.77771412202807, 85.94958464623316, 86.09281008307075, 86.25035806359209, 86.47951876253222, 86.72300200515612, 86.92351761672873, 87.03809796619879, 87.09538814093382, 87.2386135777714, 87.25293612145518, 87.3102262961902, 87.42480664566027, 87.62532225723288, 87.7255800630192, 87.89745058722428, 88.05499856774563, 88.14093382984818, 88.19822400458321, 88.25551417931825, 88.4417072472071, 88.5419650529934, 88.6135777714122, 88.5419650529934, 88.65654540246346, 88.69951303351475, 88.74248066456603, 88.81409338298482, 88.8857061014036, 88.97164136350615, 88.94299627613864, 88.97164136350615, 89.1291893440275, 89.18647951876252, 89.22944714981381, 89.3297049556001, 89.35835004296763, 89.4013176740189, 89.5445431108565, 89.60183328559151, 89.71641363506158, 89.73073617874535, 89.7450587224291, 89.78802635348038, 89.9169292466342, 89.93125179031796, 90.06015468347178, 90.07447722715554, 90.17473503294184, 90.17473503294184, 90.27499283872815, 90.26067029504439, 90.21770266399312, 90.18905757662561, 90.21770266399312, 90.28931538241191, 90.31796046977942, 90.24634775136064, 90.30363792609567, 90.40389573188197, 90.4325408192495, 90.48983099398453, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 29, 59, 91, 119, 147, 175, 198, 219, 249, 275, 299, 322, 344, 364, 392, 415, 438, 463, 490, 511, 539, 571, 597, 619, 640, 658, 684, 704, 725, 740, 756, 772, 795, 822, 839, 862, 880, 893, 910, 927, 945, 966, 985, 1004, 1018, 1028, 1040, 1046, 1057, 1072, 1087, 1097, 1112, 1126, 1134, 1141, 1150, 1166, 1174, 1181, 1185, 1195, 1202, 1210, 1216, 1224, 1233, 1238, 1244, 1257, 1264, 1269, 1278, 1283, 1290, 1300, 1307, 1318, 1323, 1329, 1334, 1344, 1348, 1359, 1364, 1373, 1378, 1387, 1392, 1396, 1399, 1408, 1415, 1419, 1422, 1429, 1436, 1441, 1447, 1449], "harmed": [0, 0, 5, 7, 8, 11, 16, 19, 21, 26, 29, 38, 43, 47, 50, 57, 64, 72, 73, 79, 82, 85, 87, 93, 101, 107, 114, 116, 124, 131, 135, 144, 151, 157, 160, 168, 171, 173, 180, 185, 192, 199, 204, 206, 211, 217, 223, 225, 230, 237, 244, 245, 248, 251, 254, 256, 259, 264, 267, 268, 270, 279, 281, 285, 290, 291, 294, 297, 304, 308, 310, 313, 315, 317, 320, 324, 324, 327, 330, 334, 339, 341, 342, 345, 347, 351, 353, 358, 360, 366, 373, 378, 385, 387, 389, 397, 400, 400, 403, 405, 406], "exact_score": [75.56574047550845, 75.98109424233743, 76.33915783443139, 76.76883414494414, 77.15554282440561, 77.51360641649957, 77.84302492122602, 78.12947579490117, 78.40160412489257, 78.75966771698654, 79.08908622171298, 79.30392437696935, 79.56173016327699, 79.81953594958465, 80.06301919220853, 80.36379260956745, 80.59295330850759, 80.80779146376396, 81.15153251217416, 81.45230592953308, 81.71011171584072, 82.0681753079347, 82.49785161844744, 82.78430249212259, 82.98481810369522, 83.19965625895158, 83.35720423947292, 83.70094528788312, 83.87281581208822, 84.07333142366083, 84.23087940418218, 84.33113720996849, 84.46004010312231, 84.7035233457462, 85.0472643941564, 85.17616728731022, 85.46261816098539, 85.69177885992552, 85.77771412202807, 85.94958464623316, 86.09281008307075, 86.25035806359209, 86.47951876253222, 86.72300200515612, 86.92351761672873, 87.03809796619879, 87.09538814093382, 87.2386135777714, 87.25293612145518, 87.3102262961902, 87.42480664566027, 87.62532225723288, 87.7255800630192, 87.89745058722428, 88.05499856774563, 88.14093382984818, 88.19822400458321, 88.25551417931825, 88.4417072472071, 88.5419650529934, 88.6135777714122, 88.5419650529934, 88.65654540246346, 88.69951303351475, 88.74248066456603, 88.81409338298482, 88.8857061014036, 88.97164136350615, 88.94299627613864, 88.97164136350615, 89.1291893440275, 89.18647951876252, 89.22944714981381, 89.3297049556001, 89.35835004296763, 89.4013176740189, 89.5445431108565, 89.60183328559151, 89.71641363506158, 89.73073617874535, 89.7450587224291, 89.78802635348038, 89.9169292466342, 89.93125179031796, 90.06015468347178, 90.07447722715554, 90.17473503294184, 90.17473503294184, 90.27499283872815, 90.26067029504439, 90.21770266399312, 90.18905757662561, 90.21770266399312, 90.28931538241191, 90.31796046977942, 90.24634775136064, 90.30363792609567, 90.40389573188197, 90.4325408192495, 90.48983099398453, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.97602555525806, "threshold_area100": 85.97500716127183, "threshold_area25": 79.61185906617015}, "test": {"score": [76.42247255506925, 76.9247327258377, 77.3911171701227, 77.68529812728708, 78.00100452034154, 78.25213460572577, 78.596541579967, 78.98399942598839, 79.26383009255937, 79.49343474205352, 79.79479084451461, 80.13202267345913, 80.4405539212169, 80.73473487838129, 81.00021525435889, 81.35897251919351, 81.69620434813805, 81.94015928822559, 82.16976393771974, 82.34914257013705, 82.61462294611465, 82.8585778862022, 83.08818253569635, 83.33931262108058, 83.58326756116811, 83.84874793714573, 84.09987802252996, 84.30078209083734, 84.50886130444142, 84.6093133385951, 84.8030422616058, 84.96089545813304, 85.12592379995695, 85.27660185118748, 85.43445504771472, 85.69276027839564, 85.87931405610964, 86.21654588505417, 86.43180024395494, 86.53225227810863, 86.62552916696563, 86.77620721819616, 86.94841070531677, 87.07756332065725, 87.26411709837124, 87.33586855133817, 87.4937217478654, 87.63004950850255, 87.69462581617277, 87.945755901557, 88.0462079357107, 88.19688598694124, 88.27581258520485, 88.412140345842, 88.48389179880894, 88.49824208940231, 88.54846810647916, 88.6058692688527, 88.691971012413, 88.72067159359977, 88.81394848245677, 88.90005022601709, 88.98615196957739, 88.99332711487408, 88.98615196957739, 89.01485255076416, 89.02202769606085, 89.05790342254431, 89.0722537131377, 89.11530458491785, 89.17988089258809, 89.30903350792853, 89.40231039678554, 89.39513525148884, 89.4310109779723, 89.45253641386238, 89.43818612326899, 89.49558728564253, 89.54581330271938, 89.64626533687307, 89.66779077276314, 89.69649135394992, 89.73236708043338, 89.73954222573008, 89.78976824280693, 89.8041185334003, 89.86869484107054, 89.95479658463084, 89.997847456411, 90.00502260170768, 90.03372318289445, 90.0839491999713, 90.0839491999713, 90.14852550764154, 90.1772260888283, 90.16287579823492, 90.15570065293824, 90.0982994905647, 90.04807347348785, 90.06959890937792, 90.06242376408122], "calls": [0.0, 1.2843510081079141, 2.374973093205137, 3.3077419817751306, 4.269211451531894, 5.123053741838272, 6.098873502188419, 7.146444715505488, 8.107914185262253, 8.875654732008323, 9.78689818468824, 10.705316782664848, 11.638085671234842, 12.556504269211452, 13.668651790198751, 14.852550764152975, 15.907297122766737, 16.75396426777642, 17.715433737533186, 18.310970797158642, 19.236564540431942, 20.348712061419246, 21.582837052450312, 22.59453253928392, 23.54882686374399, 24.62509865824783, 25.70854559804836, 26.74894166606874, 27.71758628112219, 28.657530314988875, 29.626174930042332, 30.680921288656094, 31.814594245533474, 32.81911458707039, 33.80928463801392, 34.81380497955084, 36.16273229532898, 37.55471048288727, 38.63098227739112, 39.30544593528019, 40.21668938796011, 41.02030566118964, 42.053526583913325, 42.792566549472625, 43.904714070459924, 44.765731506063, 45.84200330056684, 46.8034727703236, 48.0088971801679, 49.27172275238574, 50.03228815383511, 51.20901198249265, 52.29963406758987, 53.41178158857717, 54.29432446007032, 55.10511587859654, 56.23878883547392, 57.028054818110064, 57.946473416086675, 58.879242304656664, 59.79048575733658, 60.90263327832388, 61.78517614981703, 62.82557221783741, 63.65788907225372, 64.66958455908733, 65.67410490062424, 66.47772117385377, 67.51811724187414, 68.64461505345483, 69.59890937791491, 70.7254071894956, 71.77297840281265, 72.61247040252566, 73.50936356461219, 74.54258448733587, 75.27444930759847, 76.20004305087178, 77.4700437683863, 78.37411207576953, 79.31405610963623, 80.41185334003013, 81.43789911745712, 82.56439692903781, 83.89179880892587, 84.8030422616058, 85.84343832962618, 86.94841070531677, 87.83095357680993, 89.04355313195093, 90.01219774700438, 90.85168974671737, 91.72705747291383, 92.89660615627466, 93.81502475425127, 94.8984716940518, 96.16847241156633, 97.18016789839994, 98.40711774413433, 99.22508430795723, 100.0], "rescued": [0, 75, 147, 193, 244, 286, 342, 407, 452, 494, 541, 590, 642, 687, 738, 795, 852, 895, 938, 966, 1010, 1056, 1104, 1151, 1190, 1241, 1285, 1327, 1368, 1398, 1437, 1475, 1514, 1553, 1591, 1638, 1685, 1744, 1789, 1812, 1838, 1867, 1899, 1923, 1959, 1984, 2016, 2043, 2064, 2108, 2126, 2158, 2172, 2198, 2212, 2225, 2244, 2258, 2276, 2284, 2302, 2320, 2337, 2347, 2356, 2368, 2379, 2388, 2398, 2415, 2431, 2456, 2473, 2481, 2493, 2504, 2511, 2522, 2543, 2561, 2573, 2590, 2601, 2611, 2628, 2634, 2648, 2665, 2678, 2689, 2703, 2711, 2718, 2732, 2742, 2755, 2764, 2768, 2776, 2786, 2791], "harmed": [0, 5, 12, 17, 24, 31, 39, 50, 56, 66, 71, 73, 82, 86, 100, 107, 117, 126, 137, 140, 147, 159, 175, 187, 192, 206, 215, 229, 241, 257, 269, 285, 301, 319, 335, 346, 367, 379, 394, 403, 416, 424, 432, 438, 448, 463, 473, 481, 493, 502, 506, 517, 520, 527, 531, 542, 554, 560, 566, 570, 575, 581, 586, 595, 605, 613, 623, 627, 635, 646, 653, 660, 664, 673, 680, 688, 697, 700, 714, 718, 727, 740, 746, 755, 765, 769, 774, 779, 786, 796, 806, 807, 814, 819, 825, 840, 850, 862, 877, 884, 890], "exact_score": [76.42247255506925, 76.79558011049724, 77.25478940948555, 77.5776709478367, 77.943603357968, 78.20908373394562, 78.55349070818684, 78.92659826361485, 79.20642893018584, 79.55801104972376, 79.87371744277823, 80.21812441701944, 80.56253139126068, 80.85671234842506, 81.11501757910598, 81.39484824567698, 81.71055463873144, 82.00473559589582, 82.22716510009327, 82.57157207433451, 82.78682643323528, 82.9446796297625, 83.2245102963335, 83.43976465523427, 83.71242017650857, 83.89897395422258, 84.16445433020019, 84.37970868910097, 84.5303867403315, 84.67388964626534, 84.88196885986942, 85.00394632991318, 85.1689746717371, 85.29812728707756, 85.46315562890149, 85.69276027839564, 85.86496376551625, 86.06586783382365, 86.32417306450456, 86.4676759704384, 86.5609528592954, 86.76903207289948, 86.9340604147234, 87.09191361125063, 87.26411709837124, 87.3932697137117, 87.50089689316209, 87.63004950850255, 87.69462581617277, 87.88117959388678, 88.03903279041401, 88.18971084164454, 88.25428714931478, 88.36191432876515, 88.48389179880894, 88.49824208940231, 88.54129296118248, 88.6058692688527, 88.691971012413, 88.72784673889646, 88.82112362775347, 88.90722537131377, 88.99332711487408, 88.97897682428069, 89.01485255076416, 89.00050226017078, 89.0722537131377, 89.07942885843438, 89.07942885843438, 89.1440051661046, 89.20858147377486, 89.3520843797087, 89.39513525148884, 89.41666068737892, 89.44536126856569, 89.45253641386238, 89.45971155915908, 89.5386381574227, 89.56733873860946, 89.65344048216977, 89.67496591805984, 89.71084164454331, 89.74671737102676, 89.76106766162016, 89.78976824280693, 89.8041185334003, 89.87586998636723, 89.96914687522423, 89.99067231111431, 89.997847456411, 90.03372318289445, 90.06959890937792, 90.07677405467462, 90.15570065293824, 90.1772260888283, 90.1772260888283, 90.14852550764154, 90.10547463586138, 90.06959890937792, 90.06242376408122, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.34930048820179, "threshold_area100": 86.33504340962904, "threshold_area25": 80.43122623233121}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.4195428490638733, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3577759861946106, "operator": "ge"}, {"budget": 0.03, "threshold": 0.32592153549194336, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3033381402492523, "operator": "ge"}, {"budget": 0.05, "threshold": 0.2856070399284363, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2716599404811859, "operator": "ge"}, {"budget": 0.07, "threshold": 0.255342036485672, "operator": "ge"}, {"budget": 0.08, "threshold": 0.24203793704509735, "operator": "ge"}, {"budget": 0.09, "threshold": 0.22746394574642181, "operator": "ge"}, {"budget": 0.1, "threshold": 0.21264564990997314, "operator": "ge"}, {"budget": 0.11, "threshold": 0.19764669239521027, "operator": "ge"}, {"budget": 0.12, "threshold": 0.17845234274864197, "operator": "ge"}, {"budget": 0.13, "threshold": 0.16557157039642334, "operator": "ge"}, {"budget": 0.14, "threshold": 0.15250879526138306, "operator": "ge"}, {"budget": 0.15, "threshold": 0.13906270265579224, "operator": "ge"}, {"budget": 0.16, "threshold": 0.12552422285079956, "operator": "ge"}, {"budget": 0.17, "threshold": 0.11085377633571625, "operator": "ge"}, {"budget": 0.18, "threshold": 0.0905931144952774, "operator": "ge"}, {"budget": 0.19, "threshold": 0.07282404601573944, "operator": "ge"}, {"budget": 0.2, "threshold": 0.059527069330215454, "operator": "ge"}, {"budget": 0.21, "threshold": 0.04310189187526703, "operator": "ge"}, {"budget": 0.22, "threshold": 0.02965838462114334, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0191546231508255, "operator": "ge"}, {"budget": 0.24, "threshold": 0.008235376328229904, "operator": "ge"}, {"budget": 0.25, "threshold": -0.0006741434335708618, "operator": "ge"}, {"budget": 0.26, "threshold": -0.005701487883925438, "operator": "ge"}, {"budget": 0.27, "threshold": -0.006713406648486853, "operator": "ge"}, {"budget": 0.28, "threshold": -0.007751162629574537, "operator": "ge"}, {"budget": 0.29, "threshold": -0.008815724402666092, "operator": "ge"}, {"budget": 0.3, "threshold": -0.009706987999379635, "operator": "ge"}, {"budget": 0.31, "threshold": -0.010625495575368404, "operator": "ge"}, {"budget": 0.32, "threshold": -0.011485332623124123, "operator": "ge"}, {"budget": 0.33, "threshold": -0.012333311140537262, "operator": "ge"}, {"budget": 0.34, "threshold": -0.01323747355490923, "operator": "ge"}, {"budget": 0.35, "threshold": -0.013944287784397602, "operator": "ge"}, {"budget": 0.36, "threshold": -0.01497446745634079, "operator": "ge"}, {"budget": 0.37, "threshold": -0.01603955775499344, "operator": "ge"}, {"budget": 0.38, "threshold": -0.017156099900603294, "operator": "ge"}, {"budget": 0.39, "threshold": -0.01819514110684395, "operator": "ge"}, {"budget": 0.4, "threshold": -0.019273659214377403, "operator": "ge"}, {"budget": 0.41, "threshold": -0.02034337818622589, "operator": "ge"}, {"budget": 0.42, "threshold": -0.021321484819054604, "operator": "ge"}, {"budget": 0.43, "threshold": -0.022694934159517288, "operator": "ge"}, {"budget": 0.44, "threshold": -0.023634344339370728, "operator": "ge"}, {"budget": 0.45, "threshold": -0.024863505735993385, "operator": "ge"}, {"budget": 0.46, "threshold": -0.026473745703697205, "operator": "ge"}, {"budget": 0.47, "threshold": -0.027793440967798233, "operator": "ge"}, {"budget": 0.48, "threshold": -0.029262233525514603, "operator": "ge"}, {"budget": 0.49, "threshold": -0.030554329976439476, "operator": "ge"}, {"budget": 0.5, "threshold": -0.03227918967604637, "operator": "ge"}, {"budget": 0.51, "threshold": -0.033882878720760345, "operator": "ge"}, {"budget": 0.52, "threshold": -0.035284463316202164, "operator": "ge"}, {"budget": 0.53, "threshold": -0.03720303624868393, "operator": "ge"}, {"budget": 0.54, "threshold": -0.03920431807637215, "operator": "ge"}, {"budget": 0.55, "threshold": -0.04063429683446884, "operator": "ge"}, {"budget": 0.56, "threshold": -0.04234058037400246, "operator": "ge"}, {"budget": 0.57, "threshold": -0.0440569743514061, "operator": "ge"}, {"budget": 0.58, "threshold": -0.04606040567159653, "operator": "ge"}, {"budget": 0.59, "threshold": -0.047944046556949615, "operator": "ge"}, {"budget": 0.6, "threshold": -0.04920473322272301, "operator": "ge"}, {"budget": 0.61, "threshold": -0.05092432349920273, "operator": "ge"}, {"budget": 0.62, "threshold": -0.05283576250076294, "operator": "ge"}, {"budget": 0.63, "threshold": -0.05469706282019615, "operator": "ge"}, {"budget": 0.64, "threshold": -0.05671772360801697, "operator": "ge"}, {"budget": 0.65, "threshold": -0.0582314170897007, "operator": "ge"}, {"budget": 0.66, "threshold": -0.06007430702447891, "operator": "ge"}, {"budget": 0.67, "threshold": -0.061938825994729996, "operator": "ge"}, {"budget": 0.68, "threshold": -0.0637274757027626, "operator": "ge"}, {"budget": 0.69, "threshold": -0.06594229489564896, "operator": "ge"}, {"budget": 0.7, "threshold": -0.06870242953300476, "operator": "ge"}, {"budget": 0.71, "threshold": -0.07088995724916458, "operator": "ge"}, {"budget": 0.72, "threshold": -0.07269151508808136, "operator": "ge"}, {"budget": 0.73, "threshold": -0.07455538958311081, "operator": "ge"}, {"budget": 0.74, "threshold": -0.07683230936527252, "operator": "ge"}, {"budget": 0.75, "threshold": -0.07970023155212402, "operator": "ge"}, {"budget": 0.76, "threshold": -0.08274900168180466, "operator": "ge"}, {"budget": 0.77, "threshold": -0.08602140843868256, "operator": "ge"}, {"budget": 0.78, "threshold": -0.08944143354892731, "operator": "ge"}, {"budget": 0.79, "threshold": -0.0924767479300499, "operator": "ge"}, {"budget": 0.8, "threshold": -0.0952652245759964, "operator": "ge"}, {"budget": 0.81, "threshold": -0.09809818118810654, "operator": "ge"}, {"budget": 0.82, "threshold": -0.10105592757463455, "operator": "ge"}, {"budget": 0.83, "threshold": -0.10490161925554276, "operator": "ge"}, {"budget": 0.84, "threshold": -0.10912524908781052, "operator": "ge"}, {"budget": 0.85, "threshold": -0.11232984811067581, "operator": "ge"}, {"budget": 0.86, "threshold": -0.11684951931238174, "operator": "ge"}, {"budget": 0.87, "threshold": -0.12186364829540253, "operator": "ge"}, {"budget": 0.88, "threshold": -0.12718091905117035, "operator": "ge"}, {"budget": 0.89, "threshold": -0.1336061954498291, "operator": "ge"}, {"budget": 0.9, "threshold": -0.14057105779647827, "operator": "ge"}, {"budget": 0.91, "threshold": -0.14889512956142426, "operator": "ge"}, {"budget": 0.92, "threshold": -0.15904457867145538, "operator": "ge"}, {"budget": 0.93, "threshold": -0.16786713898181915, "operator": "ge"}, {"budget": 0.94, "threshold": -0.18052425980567932, "operator": "ge"}, {"budget": 0.95, "threshold": -0.19654884934425354, "operator": "ge"}, {"budget": 0.96, "threshold": -0.22077396512031555, "operator": "ge"}, {"budget": 0.97, "threshold": -0.25456786155700684, "operator": "ge"}, {"budget": 0.98, "threshold": -0.30932602286338806, "operator": "ge"}, {"budget": 0.99, "threshold": -0.39227294921875, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.8951589802349, 76.32483529074764, 76.69722142652535, 76.94070466914924, 77.24147808650817, 77.5708965912346, 77.88599255227729, 78.24405614437123, 78.67373245488398, 78.93153824119163, 79.20366657118304, 79.44714981380693, 79.87682612431968, 80.13463191062733, 80.40676024061874, 80.76482383271268, 80.9653394442853, 81.16585505585792, 81.50959610426811, 81.89630478372959, 82.2686909195073, 82.51217416213119, 82.66972214265253, 82.92752792896017, 83.21397880263535, 83.35720423947292, 83.41449441420797, 83.471784588943, 83.60068748209682, 83.65797765683185, 83.7725580063019, 83.85849326840446, 83.95875107419077, 83.9873961615583, 84.0876539673446, 84.14494414207962, 84.1879117731309, 84.24520194786594, 84.31681466628473, 84.46004010312231, 84.56029790890862, 84.57462045259237, 84.68920080206244, 84.775136064165, 84.84674878258377, 84.96132913205385, 85.09023202520767, 85.19048983099398, 85.36236035519909, 85.47694070466915, 85.62016614150673, 85.82068175307934, 85.94958464623316, 86.09281008307075, 86.25035806359209, 86.37926095674592, 86.53680893726725, 86.66571183042107, 86.70867946147236, 86.79461472357491, 86.90919507304497, 86.99513033514752, 87.09538814093382, 87.22429103408766, 87.36751647092524, 87.43912918934402, 87.62532225723288, 87.73990260670294, 87.88312804354052, 87.9977083930106, 88.06932111142939, 88.08364365511314, 88.12661128616442, 88.2411916356345, 88.45602979089085, 88.5419650529934, 88.65654540246346, 88.75680320824978, 88.90002864508737, 89.02893153824118, 89.14351188771126, 89.21512460613005, 89.31538241191636, 89.44428530507018, 89.5445431108565, 89.48725293612145, 89.48725293612145, 89.70209109137782, 89.75938126611285, 89.87396161558291, 90.04583213978803, 90.10312231452305, 90.24634775136064, 90.34660555714696, 90.33228301346318, 90.50415353766829, 90.66170151818963, 90.74763678029217, 90.73331423660842, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 25, 59, 88, 113, 138, 163, 188, 218, 254, 275, 300, 322, 359, 382, 408, 434, 452, 469, 497, 527, 555, 577, 594, 616, 640, 651, 656, 661, 671, 675, 683, 691, 699, 701, 708, 714, 718, 723, 729, 741, 750, 756, 765, 773, 779, 788, 800, 807, 820, 830, 842, 857, 867, 878, 890, 900, 914, 928, 933, 942, 952, 959, 970, 981, 993, 1003, 1019, 1030, 1042, 1052, 1060, 1067, 1075, 1087, 1104, 1114, 1126, 1138, 1150, 1167, 1180, 1192, 1204, 1215, 1231, 1237, 1249, 1267, 1279, 1291, 1308, 1320, 1337, 1351, 1363, 1380, 1400, 1417, 1435, 1449], "harmed": [0, 2, 6, 9, 17, 21, 23, 26, 31, 37, 40, 46, 51, 58, 63, 70, 71, 75, 78, 82, 85, 87, 92, 98, 102, 106, 107, 108, 109, 110, 110, 110, 112, 113, 113, 113, 115, 116, 117, 118, 120, 122, 127, 128, 130, 131, 132, 135, 135, 136, 138, 140, 141, 142, 143, 144, 145, 148, 153, 155, 158, 160, 161, 165, 167, 169, 174, 177, 180, 182, 184, 187, 193, 198, 202, 204, 208, 212, 217, 219, 227, 232, 239, 244, 246, 255, 265, 277, 280, 288, 292, 297, 305, 312, 319, 332, 337, 346, 357, 376, 406], "exact_score": [75.56574047550845, 75.8951589802349, 76.32483529074764, 76.69722142652535, 76.94070466914924, 77.24147808650817, 77.5708965912346, 77.88599255227729, 78.24405614437123, 78.67373245488398, 78.93153824119163, 79.20366657118304, 79.44714981380693, 79.87682612431968, 80.13463191062733, 80.40676024061874, 80.76482383271268, 80.9653394442853, 81.16585505585792, 81.50959610426811, 81.89630478372959, 82.2686909195073, 82.51217416213119, 82.66972214265253, 82.92752792896017, 83.21397880263535, 83.35720423947292, 83.41449441420797, 83.471784588943, 83.60068748209682, 83.65797765683185, 83.7725580063019, 83.85849326840446, 83.95875107419077, 83.9873961615583, 84.0876539673446, 84.14494414207962, 84.1879117731309, 84.24520194786594, 84.31681466628473, 84.46004010312231, 84.56029790890862, 84.57462045259237, 84.68920080206244, 84.775136064165, 84.84674878258377, 84.96132913205385, 85.09023202520767, 85.19048983099398, 85.36236035519909, 85.47694070466915, 85.62016614150673, 85.82068175307934, 85.94958464623316, 86.09281008307075, 86.25035806359209, 86.37926095674592, 86.53680893726725, 86.66571183042107, 86.70867946147236, 86.79461472357491, 86.90919507304497, 86.99513033514752, 87.09538814093382, 87.22429103408766, 87.36751647092524, 87.43912918934402, 87.62532225723288, 87.73990260670294, 87.88312804354052, 87.9977083930106, 88.06932111142939, 88.08364365511314, 88.12661128616442, 88.2411916356345, 88.45602979089085, 88.5419650529934, 88.65654540246346, 88.75680320824978, 88.90002864508737, 89.02893153824118, 89.14351188771126, 89.21512460613005, 89.31538241191636, 89.44428530507018, 89.5445431108565, 89.48725293612145, 89.48725293612145, 89.70209109137782, 89.75938126611285, 89.87396161558291, 90.04583213978803, 90.10312231452305, 90.24634775136064, 90.34660555714696, 90.33228301346318, 90.50415353766829, 90.66170151818963, 90.74763678029217, 90.73331423660842, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.15213158918037, "threshold_area100": 85.15103122314524, "threshold_area25": 79.57318819822402}, "test": {"score": [76.42247255506925, 76.78122981990386, 77.14716223003516, 77.46286862308962, 77.757049580254, 78.04405539212169, 78.35258663987946, 78.66829303293392, 78.92659826361485, 79.37145727200976, 79.67998851976753, 80.06027122049221, 80.46207935710699, 80.6342828442276, 80.93563894668867, 81.07196670732583, 81.28722106622659, 81.50965057042406, 81.86840783525867, 82.2056396642032, 82.42089402310397, 82.69354954437827, 82.93750448446582, 83.25321087752027, 83.58326756116811, 83.87027337303581, 84.00660113367296, 84.0711774413432, 84.1572791849035, 84.22185549257372, 84.29360694554065, 84.39405897969434, 84.49451101384804, 84.58061275740833, 84.65236421037527, 84.7097653727488, 84.8030422616058, 84.86044342397933, 84.93937002224295, 85.01829662050658, 85.0900480734735, 85.19050010762719, 85.25507641529741, 85.32682786826433, 85.39140417593455, 85.47033077419817, 85.59230824424195, 85.6497094066155, 85.70711056898902, 85.80756260314271, 85.86496376551625, 85.97259094496664, 86.05869268852695, 86.16631986797732, 86.30982277391116, 86.44615053454832, 86.52507713281194, 86.59682858577887, 86.68293032933917, 86.74033149171271, 86.81208294467963, 86.91253497883332, 87.00581186769031, 87.1277893377341, 87.22106622659109, 87.34304369663487, 87.4004448590084, 87.48654660256871, 87.5941737820191, 87.665925234986, 87.80942814091985, 87.88117959388678, 87.93140561096362, 88.02468249982063, 88.18253569634786, 88.29733802109493, 88.412140345842, 88.54846810647916, 88.62739470474277, 88.65609528592955, 88.6847958671163, 88.76372246537993, 88.79959819186338, 88.91440051661046, 89.05790342254431, 89.09377914902777, 89.19423118318146, 89.31620865322523, 89.43818612326899, 89.56733873860946, 89.69649135394992, 89.8041185334003, 89.811293678697, 89.84716940518047, 89.94762143933414, 89.997847456411, 90.15570065293824, 90.18440123412499, 90.28485326827868, 90.28485326827868, 90.06242376408122], "calls": [0.0, 0.9255937432733014, 2.0090406830738323, 3.0494367510942095, 4.046781947334434, 4.95802540001435, 5.754466527947192, 6.694410561813877, 7.5984788691971, 8.646050082514172, 9.492717227523856, 10.604864748511158, 11.903566047212456, 12.814809499892373, 13.63277606371529, 14.350290593384516, 15.175432302504124, 15.957523139843582, 17.22752385735811, 18.210518763004952, 18.999784745641097, 20.097581976035013, 21.195379206428928, 22.235775274449306, 23.462725120183684, 24.63227380354452, 25.744421324531825, 26.576738178948123, 27.473631341034654, 28.707756332065728, 29.626174930042332, 30.652220707469326, 31.491712707182316, 32.603860228169616, 33.479227954366074, 34.117815885771684, 35.22996340675898, 36.39951209011982, 37.497309320513736, 38.616631986797735, 39.93685872138911, 41.06335653296979, 41.823921934419175, 42.83561742125278, 43.49573078854847, 44.38544880533831, 45.425844873358685, 46.236636291884906, 47.20528090693836, 48.07347348783813, 49.09951926526512, 49.98923728205496, 50.82872928176796, 51.969577383942024, 53.189352084379706, 54.114945827653, 55.02618928033293, 56.07376049364999, 56.963478510439835, 57.975173997273444, 58.57071105689891, 59.53935567195236, 60.53670086819258, 61.43359403027912, 62.23721030350865, 63.12692832029848, 63.995120901198256, 64.94224008036163, 65.77455693477793, 66.88670445576523, 68.34325895099376, 69.17557580541006, 69.85003946329913, 70.69670660830883, 71.70840209514243, 72.9497022314702, 74.1479514960178, 75.48252852120255, 76.80275525579393, 77.7426992896606, 78.53196527229677, 79.3284064002296, 80.31140130587644, 81.52400086101743, 82.5787472196312, 83.49716581760781, 84.57343761211165, 85.70711056898902, 86.94841070531677, 87.96728133744708, 89.0579034225443, 90.20592667001507, 91.52615340460645, 92.32976967783597, 93.49931836119681, 94.5397144292172, 95.8312405826218, 96.85728636004879, 97.97660902633278, 98.94525364138623, 100.0], "rescued": [0, 57, 113, 169, 220, 269, 318, 370, 412, 483, 534, 593, 662, 696, 746, 771, 809, 849, 906, 959, 998, 1047, 1093, 1141, 1194, 1242, 1265, 1275, 1289, 1300, 1313, 1331, 1345, 1360, 1371, 1380, 1393, 1406, 1419, 1431, 1447, 1465, 1477, 1488, 1500, 1513, 1531, 1541, 1556, 1573, 1587, 1605, 1623, 1643, 1665, 1687, 1703, 1716, 1733, 1751, 1767, 1786, 1807, 1828, 1846, 1872, 1886, 1906, 1923, 1943, 1974, 1992, 2006, 2025, 2054, 2083, 2109, 2142, 2169, 2181, 2198, 2215, 2230, 2258, 2287, 2311, 2344, 2372, 2406, 2436, 2465, 2493, 2522, 2541, 2574, 2605, 2647, 2677, 2721, 2755, 2791], "harmed": [0, 7, 12, 24, 34, 43, 49, 57, 63, 72, 80, 86, 99, 109, 117, 123, 131, 140, 147, 153, 162, 173, 185, 189, 196, 204, 208, 209, 211, 213, 216, 220, 220, 223, 224, 225, 225, 230, 232, 233, 239, 243, 246, 247, 250, 252, 253, 255, 262, 265, 271, 274, 280, 285, 287, 290, 295, 298, 303, 313, 319, 324, 332, 336, 341, 350, 356, 364, 366, 376, 387, 395, 402, 408, 415, 428, 438, 452, 468, 476, 489, 495, 505, 517, 526, 545, 564, 575, 592, 604, 615, 628, 656, 670, 689, 713, 733, 759, 789, 823, 890], "exact_score": [76.42247255506925, 76.79558011049724, 77.13998708473846, 77.44134318719954, 77.73552414436392, 78.06558082801178, 78.45303867403315, 78.77592021238429, 79.062926024252, 79.49343474205352, 79.84501686159145, 80.1750735452393, 80.46925450240367, 80.69885915189784, 81.02891583554567, 81.24417019444644, 81.51682571572074, 81.77513094640166, 82.1123627753462, 82.42089402310397, 82.65767381789482, 82.91597904857574, 83.18863456985004, 83.5043409629045, 83.71242017650857, 83.92767453540935, 84.02095142426634, 84.10705316782665, 84.1572791849035, 84.25773121905719, 84.35100810791418, 84.44428499677119, 84.5375618856282, 84.6093133385951, 84.71694051804549, 84.8030422616058, 84.83891798808926, 84.91066944105619, 84.99677118461649, 85.03264691109995, 85.10439836406688, 85.18332496233049, 85.2694267058908, 85.33400301356102, 85.4416301930114, 85.53490708186841, 85.6353591160221, 85.71428571428572, 85.78603716725264, 85.86496376551625, 85.97259094496664, 86.08021812441702, 86.16631986797732, 86.2882973380211, 86.41744995336155, 86.52507713281194, 86.5753031498888, 86.69010547463587, 86.74033149171271, 86.85513381645978, 86.91971012413, 87.09908875654733, 87.19236564540432, 87.32151826074478, 87.4004448590084, 87.48654660256871, 87.63722465379924, 87.69462581617277, 87.7735524144364, 87.8596541579967, 87.96010619215039, 88.06773337160078, 88.20406113223794, 88.30451316639162, 88.39061490995192, 88.49824208940231, 88.55564325177585, 88.64892014063285, 88.67044557652292, 88.72067159359977, 88.77807275597331, 88.90722537131377, 88.98615196957739, 89.0722537131377, 89.11530458491785, 89.2659826361484, 89.32338379852193, 89.45253641386238, 89.58168902920285, 89.69649135394992, 89.8184688239937, 89.85434455047715, 89.83999425988375, 89.86869484107054, 89.97632202052091, 90.05524861878453, 90.14135036234484, 90.22027696060846, 90.28485326827868, 90.31355384946545, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 85.51368693844881, "threshold_area100": 85.41059768960322, "threshold_area25": 80.27265552127432}, "selected_budget": 98, "selected": {"val": {"score": 90.74763678029217, "calls": 97.99484388427385, "rescued": 1417, "harmed": 357}, "test": {"score": 90.28485326827868, "calls": 97.97660902633278, "rescued": 2721, "harmed": 789}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.181845188140869, "operator": "ge"}, {"budget": 0.02, "threshold": 2.7537126541137695, "operator": "ge"}, {"budget": 0.03, "threshold": 2.478766441345215, "operator": "ge"}, {"budget": 0.04, "threshold": 2.216701030731201, "operator": "ge"}, {"budget": 0.05, "threshold": 1.9961944818496704, "operator": "ge"}, {"budget": 0.06, "threshold": 1.798870325088501, "operator": "ge"}, {"budget": 0.07, "threshold": 1.644870400428772, "operator": "ge"}, {"budget": 0.08, "threshold": 1.5401604175567627, "operator": "ge"}, {"budget": 0.09, "threshold": 1.4415647983551025, "operator": "ge"}, {"budget": 0.1, "threshold": 1.3249520063400269, "operator": "ge"}, {"budget": 0.11, "threshold": 1.2372207641601562, "operator": "ge"}, {"budget": 0.12, "threshold": 1.1615020036697388, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0754449367523193, "operator": "ge"}, {"budget": 0.14, "threshold": 1.0018646717071533, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9212947487831116, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8384131193161011, "operator": "ge"}, {"budget": 0.17, "threshold": 0.750100314617157, "operator": "ge"}, {"budget": 0.18, "threshold": 0.6651239395141602, "operator": "ge"}, {"budget": 0.19, "threshold": 0.5810633897781372, "operator": "ge"}, {"budget": 0.2, "threshold": 0.505038857460022, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4168356657028198, "operator": "ge"}, {"budget": 0.22, "threshold": 0.30652838945388794, "operator": "ge"}, {"budget": 0.23, "threshold": 0.20258396863937378, "operator": "ge"}, {"budget": 0.24, "threshold": 0.10349893569946289, "operator": "ge"}, {"budget": 0.25, "threshold": -0.005213052034378052, "operator": "ge"}, {"budget": 0.26, "threshold": -0.13080298900604248, "operator": "ge"}, {"budget": 0.27, "threshold": -0.24459204077720642, "operator": "ge"}, {"budget": 0.28, "threshold": -0.33760640025138855, "operator": "ge"}, {"budget": 0.29, "threshold": -0.4317869544029236, "operator": "ge"}, {"budget": 0.3, "threshold": -0.5435092449188232, "operator": "ge"}, {"budget": 0.31, "threshold": -0.6370534896850586, "operator": "ge"}, {"budget": 0.32, "threshold": -0.7441383600234985, "operator": "ge"}, {"budget": 0.33, "threshold": -0.855013370513916, "operator": "ge"}, {"budget": 0.34, "threshold": -0.9418022036552429, "operator": "ge"}, {"budget": 0.35, "threshold": -1.0528393983840942, "operator": "ge"}, {"budget": 0.36, "threshold": -1.1770706176757812, "operator": "ge"}, {"budget": 0.37, "threshold": -1.292776346206665, "operator": "ge"}, {"budget": 0.38, "threshold": -1.4204550981521606, "operator": "ge"}, {"budget": 0.39, "threshold": -1.535463571548462, "operator": "ge"}, {"budget": 0.4, "threshold": -1.6410467624664307, "operator": "ge"}, {"budget": 0.41, "threshold": -1.760849952697754, "operator": "ge"}, {"budget": 0.42, "threshold": -1.865075945854187, "operator": "ge"}, {"budget": 0.43, "threshold": -1.9788787364959717, "operator": "ge"}, {"budget": 0.44, "threshold": -2.0831339359283447, "operator": "ge"}, {"budget": 0.45, "threshold": -2.1851720809936523, "operator": "ge"}, {"budget": 0.46, "threshold": -2.288536548614502, "operator": "ge"}, {"budget": 0.47, "threshold": -2.3948936462402344, "operator": "ge"}, {"budget": 0.48, "threshold": -2.4956648349761963, "operator": "ge"}, {"budget": 0.49, "threshold": -2.5896949768066406, "operator": "ge"}, {"budget": 0.5, "threshold": -2.677926540374756, "operator": "ge"}, {"budget": 0.51, "threshold": -2.7811224460601807, "operator": "ge"}, {"budget": 0.52, "threshold": -2.860732078552246, "operator": "ge"}, {"budget": 0.53, "threshold": -2.937295913696289, "operator": "ge"}, {"budget": 0.54, "threshold": -3.007779121398926, "operator": "ge"}, {"budget": 0.55, "threshold": -3.0822744369506836, "operator": "ge"}, {"budget": 0.56, "threshold": -3.1530394554138184, "operator": "ge"}, {"budget": 0.57, "threshold": -3.2209038734436035, "operator": "ge"}, {"budget": 0.58, "threshold": -3.2697372436523438, "operator": "ge"}, {"budget": 0.59, "threshold": -3.328434944152832, "operator": "ge"}, {"budget": 0.6, "threshold": -3.3949995040893555, "operator": "ge"}, {"budget": 0.61, "threshold": -3.45743465423584, "operator": "ge"}, {"budget": 0.62, "threshold": -3.5109052658081055, "operator": "ge"}, {"budget": 0.63, "threshold": -3.551637649536133, "operator": "ge"}, {"budget": 0.64, "threshold": -3.6030993461608887, "operator": "ge"}, {"budget": 0.65, "threshold": -3.6566290855407715, "operator": "ge"}, {"budget": 0.66, "threshold": -3.701897144317627, "operator": "ge"}, {"budget": 0.67, "threshold": -3.7493810653686523, "operator": "ge"}, {"budget": 0.68, "threshold": -3.803884983062744, "operator": "ge"}, {"budget": 0.69, "threshold": -3.839494228363037, "operator": "ge"}, {"budget": 0.7, "threshold": -3.8946919441223145, "operator": "ge"}, {"budget": 0.71, "threshold": -3.944248676300049, "operator": "ge"}, {"budget": 0.72, "threshold": -3.9937729835510254, "operator": "ge"}, {"budget": 0.73, "threshold": -4.037806510925293, "operator": "ge"}, {"budget": 0.74, "threshold": -4.075500965118408, "operator": "ge"}, {"budget": 0.75, "threshold": -4.115624904632568, "operator": "ge"}, {"budget": 0.76, "threshold": -4.164362907409668, "operator": "ge"}, {"budget": 0.77, "threshold": -4.203606128692627, "operator": "ge"}, {"budget": 0.78, "threshold": -4.247547626495361, "operator": "ge"}, {"budget": 0.79, "threshold": -4.29812479019165, "operator": "ge"}, {"budget": 0.8, "threshold": -4.343725204467773, "operator": "ge"}, {"budget": 0.81, "threshold": -4.393411636352539, "operator": "ge"}, {"budget": 0.82, "threshold": -4.44258975982666, "operator": "ge"}, {"budget": 0.83, "threshold": -4.495233058929443, "operator": "ge"}, {"budget": 0.84, "threshold": -4.549990653991699, "operator": "ge"}, {"budget": 0.85, "threshold": -4.5951008796691895, "operator": "ge"}, {"budget": 0.86, "threshold": -4.6456170082092285, "operator": "ge"}, {"budget": 0.87, "threshold": -4.701045036315918, "operator": "ge"}, {"budget": 0.88, "threshold": -4.747764587402344, "operator": "ge"}, {"budget": 0.89, "threshold": -4.807066917419434, "operator": "ge"}, {"budget": 0.9, "threshold": -4.868743896484375, "operator": "ge"}, {"budget": 0.91, "threshold": -4.917361259460449, "operator": "ge"}, {"budget": 0.92, "threshold": -4.99676513671875, "operator": "ge"}, {"budget": 0.93, "threshold": -5.075278282165527, "operator": "ge"}, {"budget": 0.94, "threshold": -5.153229236602783, "operator": "ge"}, {"budget": 0.95, "threshold": -5.248754501342773, "operator": "ge"}, {"budget": 0.96, "threshold": -5.364862442016602, "operator": "ge"}, {"budget": 0.97, "threshold": -5.544410228729248, "operator": "ge"}, {"budget": 0.98, "threshold": -5.779386520385742, "operator": "ge"}, {"budget": 0.99, "threshold": -6.104294300079346, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 76.05270696075623, 76.45373818390145, 76.68289888284158, 76.98367230020051, 77.3274133486107, 77.75708965912345, 78.15812088226869, 78.51618447436263, 78.78831280435405, 79.08908622171298, 79.3182469206531, 79.70495560011457, 79.91979375537095, 80.20624462904613, 80.44972787167, 80.66456602692638, 80.9653394442853, 81.16585505585792, 81.36637066743053, 81.753079346892, 81.9965625895159, 82.38327126897737, 82.76997994843884, 82.99914064737897, 83.19965625895158, 83.3715267831567, 83.58636493841306, 83.85849326840446, 84.0876539673446, 84.1879117731309, 84.34545975365224, 84.57462045259237, 84.71784588942997, 85.03294185047264, 85.23345746204527, 85.4912632483529, 85.64881122887425, 85.79203666571182, 85.9352621025494, 86.12145517043827, 86.32197078201088, 86.56545402463478, 86.6943569177886, 86.85190489830994, 86.92351761672873, 87.08106559725006, 87.19564594672013, 87.3102262961902, 87.36751647092524, 87.53938699513033, 87.6539673446004, 87.75422515038672, 87.94041821827557, 88.01203093669436, 88.18390146089945, 88.25551417931825, 88.35577198510455, 88.39873961615584, 88.47035233457461, 88.49899742194214, 88.55628759667717, 88.6135777714122, 88.69951303351475, 88.74248066456603, 88.85706101403609, 88.91435118877112, 88.98596390718991, 88.98596390718991, 89.0575766256087, 89.07189916929245, 89.17215697507876, 89.24376969349757, 89.25809223718132, 89.3010598682326, 89.34402749928387, 89.42996276138642, 89.55886565454024, 89.573188198224, 89.6734460040103, 89.73073617874535, 89.80234889716414, 89.9169292466342, 90.03150959610426, 90.04583213978803, 90.04583213978803, 90.20338012030936, 90.20338012030936, 90.21770266399312, 90.24634775136064, 90.23202520767688, 90.20338012030936, 90.27499283872815, 90.30363792609567, 90.28931538241191, 90.26067029504439, 90.31796046977942, 90.38957318819823, 90.44686336293324, 90.50415353766829, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 34, 64, 81, 107, 133, 166, 196, 223, 244, 269, 290, 320, 341, 366, 391, 411, 435, 458, 479, 510, 533, 563, 595, 619, 640, 658, 680, 705, 724, 740, 758, 779, 797, 822, 844, 865, 880, 897, 914, 934, 953, 973, 988, 1004, 1015, 1028, 1039, 1053, 1064, 1077, 1089, 1098, 1113, 1123, 1138, 1148, 1157, 1163, 1173, 1178, 1189, 1195, 1204, 1209, 1220, 1228, 1236, 1239, 1248, 1252, 1263, 1269, 1274, 1279, 1287, 1295, 1305, 1310, 1319, 1326, 1333, 1341, 1353, 1357, 1363, 1375, 1380, 1385, 1391, 1395, 1400, 1409, 1415, 1417, 1422, 1429, 1435, 1441, 1448, 1449], "harmed": [0, 0, 2, 3, 8, 10, 13, 15, 17, 19, 23, 28, 31, 37, 42, 50, 55, 58, 67, 74, 78, 84, 87, 92, 100, 107, 113, 120, 126, 129, 138, 145, 150, 158, 161, 169, 172, 176, 183, 190, 197, 202, 205, 211, 216, 222, 224, 227, 233, 240, 241, 245, 247, 249, 254, 257, 262, 264, 267, 272, 275, 282, 284, 287, 289, 292, 296, 299, 302, 306, 309, 313, 314, 318, 320, 325, 327, 328, 332, 334, 337, 339, 339, 343, 346, 352, 353, 358, 362, 366, 371, 378, 382, 386, 389, 396, 399, 400, 402, 405, 406], "exact_score": [75.56574047550845, 76.05270696075623, 76.45373818390145, 76.68289888284158, 76.98367230020051, 77.3274133486107, 77.75708965912345, 78.15812088226869, 78.51618447436263, 78.78831280435405, 79.08908622171298, 79.3182469206531, 79.70495560011457, 79.91979375537095, 80.20624462904613, 80.44972787167, 80.66456602692638, 80.9653394442853, 81.16585505585792, 81.36637066743053, 81.753079346892, 81.9965625895159, 82.38327126897737, 82.76997994843884, 82.99914064737897, 83.19965625895158, 83.3715267831567, 83.58636493841306, 83.85849326840446, 84.0876539673446, 84.1879117731309, 84.34545975365224, 84.57462045259237, 84.71784588942997, 85.03294185047264, 85.23345746204527, 85.4912632483529, 85.64881122887425, 85.79203666571182, 85.9352621025494, 86.12145517043827, 86.32197078201088, 86.56545402463478, 86.6943569177886, 86.85190489830994, 86.92351761672873, 87.08106559725006, 87.19564594672013, 87.3102262961902, 87.36751647092524, 87.53938699513033, 87.6539673446004, 87.75422515038672, 87.94041821827557, 88.01203093669436, 88.18390146089945, 88.25551417931825, 88.35577198510455, 88.39873961615584, 88.47035233457461, 88.49899742194214, 88.55628759667717, 88.6135777714122, 88.69951303351475, 88.74248066456603, 88.85706101403609, 88.91435118877112, 88.98596390718991, 88.98596390718991, 89.0575766256087, 89.07189916929245, 89.17215697507876, 89.24376969349757, 89.25809223718132, 89.3010598682326, 89.34402749928387, 89.42996276138642, 89.55886565454024, 89.573188198224, 89.6734460040103, 89.73073617874535, 89.80234889716414, 89.9169292466342, 90.03150959610426, 90.04583213978803, 90.04583213978803, 90.20338012030936, 90.20338012030936, 90.21770266399312, 90.24634775136064, 90.23202520767688, 90.20338012030936, 90.27499283872815, 90.30363792609567, 90.28931538241191, 90.26067029504439, 90.31796046977942, 90.38957318819823, 90.44686336293324, 90.50415353766829, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.97877129068068, "threshold_area100": 85.97772844457174, "threshold_area25": 79.60040103122314}, "test": {"score": [76.42247255506925, 76.75252923871709, 77.05388534117816, 77.45569347779293, 77.83597617851761, 78.11580684508861, 78.61806701585708, 78.97682428069169, 79.27100523785607, 79.47908445146015, 79.90959316926168, 80.31140130587644, 80.51230537418384, 80.83518691253498, 81.09349214321591, 81.44507426275382, 81.73208007462152, 81.9186338523355, 82.1984645189065, 82.42089402310397, 82.5930975102246, 82.8585778862022, 83.06665709980628, 83.34648776637728, 83.55456697998135, 83.82722250125565, 84.09270287723328, 84.35818325321088, 84.50886130444142, 84.6236636291885, 84.79586711630911, 84.9895960393198, 85.12592379995695, 85.29095214178088, 85.49185621008826, 85.65688455191219, 85.94389036377987, 86.26677190213103, 86.41027480806487, 86.51790198751526, 86.67575518404249, 86.82643323527302, 86.98428643180026, 87.18519050010764, 87.24259166248117, 87.34304369663487, 87.4937217478654, 87.53677261964555, 87.70897610676617, 87.89552988448015, 88.01750735452393, 88.14665996986439, 88.22558656812801, 88.36191432876515, 88.39061490995192, 88.47671665351223, 88.5125923799957, 88.6058692688527, 88.67044557652292, 88.73502188419316, 88.89287508072039, 88.97897682428069, 88.96462653368731, 88.93592595250054, 88.99332711487408, 89.00050226017078, 89.05790342254431, 89.13683002080792, 89.17270574729139, 89.21575661907154, 89.28750807203846, 89.344909234412, 89.3377340891153, 89.38078496089545, 89.40948554208224, 89.40948554208224, 89.4238358326756, 89.47406184975246, 89.531463012126, 89.6175647556863, 89.6175647556863, 89.65344048216977, 89.68214106335654, 89.70366649924661, 89.8184688239937, 89.84716940518047, 89.87586998636723, 89.94044629403746, 89.96197172992754, 89.96197172992754, 90.03372318289445, 90.09112434526799, 90.11264978115807, 90.14852550764154, 90.1772260888283, 90.1700509435316, 90.12700007175145, 90.10547463586138, 90.04089832819116, 90.07677405467462, 90.06242376408122], "calls": [0.0, 0.9973451962402239, 1.922938939513525, 2.8485326827868267, 3.946329913180742, 4.943675109420965, 6.220850972232188, 7.2971227667360266, 8.10073903996556, 8.93305589438186, 10.23175719308316, 11.444356748224152, 12.255148166750377, 13.259668508287293, 14.149386525077132, 15.074980268350435, 16.029274592810506, 16.962043481380498, 17.91633780584057, 18.741479514960176, 19.523570352299636, 20.355887206715938, 21.42498385592308, 22.55865681280046, 23.527301427853914, 24.603573222357753, 25.65831958097152, 26.756116811365434, 27.74628686230896, 28.578603716725265, 29.740977254789406, 30.860299921073402, 31.92939657028055, 32.991318074191, 33.81645978331061, 34.9429575948913, 36.19860802181244, 37.375331850469976, 38.50182966205066, 39.42742340532396, 40.302791131520415, 41.29296118246394, 42.06070172921002, 43.04369663485686, 43.76121116452608, 44.622228600129155, 45.605223505775996, 46.72454617205998, 47.93714572720098, 48.963191504627964, 49.99641242735166, 51.25206285427279, 52.22070746932626, 53.1032503408194, 53.94274234053239, 55.011838989739545, 56.059410203056615, 57.135681997560454, 57.982349142570136, 58.857716868766595, 60.04879098801751, 61.082011910741194, 62.036306235201266, 62.89014852550764, 63.80856712348425, 64.90636435387816, 65.81760780655809, 66.70732582334792, 67.81229819903854, 68.4867618569276, 69.6850111214752, 70.68235631771543, 71.65817607806558, 72.49766807777857, 73.24388318863457, 74.11207576953433, 75.06637009399441, 75.94173782019085, 76.97495874291455, 78.07275597330846, 79.11315204132883, 80.22529956231614, 81.28722106622659, 82.51417091196097, 83.79852192006888, 84.68106479156204, 85.67123484250556, 86.85513381645978, 87.80942814091985, 88.90722537131377, 89.98349716581761, 90.71536198608023, 91.87773552414437, 92.87508072038459, 93.77914902776782, 94.94869771112864, 96.03931979622587, 97.28061993255363, 98.45734376121116, 99.24660974384732, 100.0], "rescued": [0, 49, 93, 150, 205, 249, 324, 381, 426, 467, 537, 604, 640, 694, 738, 794, 843, 885, 931, 974, 1008, 1049, 1093, 1146, 1186, 1238, 1284, 1336, 1374, 1401, 1443, 1483, 1519, 1562, 1603, 1642, 1697, 1749, 1788, 1817, 1848, 1877, 1905, 1942, 1963, 1989, 2019, 2036, 2070, 2101, 2128, 2152, 2169, 2193, 2205, 2227, 2244, 2260, 2272, 2285, 2309, 2329, 2339, 2346, 2361, 2373, 2386, 2400, 2415, 2427, 2447, 2461, 2468, 2479, 2488, 2494, 2503, 2521, 2536, 2556, 2569, 2583, 2595, 2606, 2628, 2636, 2648, 2662, 2675, 2688, 2702, 2712, 2724, 2731, 2741, 2755, 2762, 2770, 2777, 2787, 2791], "harmed": [0, 3, 5, 6, 8, 13, 18, 25, 29, 41, 51, 62, 70, 79, 87, 94, 103, 119, 126, 138, 148, 152, 167, 181, 192, 206, 215, 230, 247, 258, 276, 289, 306, 326, 339, 355, 370, 377, 396, 410, 419, 427, 433, 442, 455, 467, 476, 487, 497, 502, 512, 518, 524, 529, 537, 547, 559, 562, 565, 569, 571, 579, 591, 602, 609, 620, 625, 628, 638, 644, 654, 660, 668, 673, 678, 684, 691, 702, 709, 717, 730, 739, 747, 755, 761, 765, 773, 778, 788, 801, 805, 807, 816, 818, 824, 839, 852, 863, 879, 884, 890], "exact_score": [76.42247255506925, 76.75252923871709, 77.08976106766163, 77.49874434957307, 77.84315132381431, 78.130157135682, 78.54631556289016, 78.89072253713138, 79.2351295113726, 79.52213532324029, 79.84501686159145, 80.18942383583267, 80.46207935710699, 80.74191002367799, 81.06479156202914, 81.40919853627037, 81.71772978402812, 81.9329841429289, 82.22716510009327, 82.4782951854775, 82.77965128793858, 82.99490564683936, 83.2245102963335, 83.43976465523427, 83.6908947406185, 83.89179880892588, 84.17162947549687, 84.41558441558442, 84.5232115950348, 84.73846595393557, 84.83174284279256, 84.98242089402311, 85.13309894525365, 85.29095214178088, 85.51338164597834, 85.67123484250556, 85.90083949199972, 86.17349501327402, 86.33134820980125, 86.52507713281194, 86.61835402166894, 86.77620721819616, 86.97711128650354, 87.17801535481094, 87.29281767955801, 87.38609456841502, 87.50089689316209, 87.55829805553563, 87.70897610676617, 87.90988017507355, 88.01750735452393, 88.10360909808423, 88.21841142283131, 88.34038889287508, 88.39779005524862, 88.47671665351223, 88.505417234699, 88.57716868766593, 88.67044557652292, 88.77089761067663, 88.8856999354237, 88.96462653368731, 88.96462653368731, 88.95027624309392, 89.00050226017078, 89.00767740546746, 89.07942885843438, 89.12247973021454, 89.18705603788477, 89.24445720025831, 89.32338379852193, 89.33055894381862, 89.3520843797087, 89.38796010619215, 89.41666068737892, 89.43818612326899, 89.48841214034584, 89.52428786682931, 89.6103896103896, 89.6175647556863, 89.63909019157639, 89.68931620865322, 89.72519193513668, 89.71084164454331, 89.811293678697, 89.84716940518047, 89.87586998636723, 89.94044629403746, 89.96914687522423, 89.96914687522423, 90.03372318289445, 90.0982994905647, 90.10547463586138, 90.14852550764154, 90.1700509435316, 90.1700509435316, 90.14135036234484, 90.12700007175145, 90.06242376408122, 90.06242376408122, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.34269963253547, "threshold_area100": 86.33246035732225, "threshold_area25": 80.43768386309824}, "selected_budget": 99, "selected": {"val": {"score": 90.50415353766829, "calls": 98.99742194213692, "rescued": 1448, "harmed": 405}, "test": {"score": 90.07677405467462, "calls": 99.24660974384732, "rescued": 2787, "harmed": 884}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.6035351753234863, "operator": "ge"}, {"budget": 0.02, "threshold": 1.3872263431549072, "operator": "ge"}, {"budget": 0.03, "threshold": 1.275890588760376, "operator": "ge"}, {"budget": 0.04, "threshold": 1.1922385692596436, "operator": "ge"}, {"budget": 0.05, "threshold": 1.1203076839447021, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0523152351379395, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9888492822647095, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9405950903892517, "operator": "ge"}, {"budget": 0.09, "threshold": 0.8956708908081055, "operator": "ge"}, {"budget": 0.1, "threshold": 0.8436900973320007, "operator": "ge"}, {"budget": 0.11, "threshold": 0.7994382381439209, "operator": "ge"}, {"budget": 0.12, "threshold": 0.7552956342697144, "operator": "ge"}, {"budget": 0.13, "threshold": 0.7124663591384888, "operator": "ge"}, {"budget": 0.14, "threshold": 0.6665294170379639, "operator": "ge"}, {"budget": 0.15, "threshold": 0.6052618622779846, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5551666021347046, "operator": "ge"}, {"budget": 0.17, "threshold": 0.508703351020813, "operator": "ge"}, {"budget": 0.18, "threshold": 0.45854926109313965, "operator": "ge"}, {"budget": 0.19, "threshold": 0.4103279709815979, "operator": "ge"}, {"budget": 0.2, "threshold": 0.35762906074523926, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2805108428001404, "operator": "ge"}, {"budget": 0.22, "threshold": 0.2111935019493103, "operator": "ge"}, {"budget": 0.23, "threshold": 0.14458119869232178, "operator": "ge"}, {"budget": 0.24, "threshold": 0.07451128959655762, "operator": "ge"}, {"budget": 0.25, "threshold": -0.004287242889404297, "operator": "ge"}, {"budget": 0.26, "threshold": -0.099392831325531, "operator": "ge"}, {"budget": 0.27, "threshold": -0.18252596259117126, "operator": "ge"}, {"budget": 0.28, "threshold": -0.2625841498374939, "operator": "ge"}, {"budget": 0.29, "threshold": -0.3430752754211426, "operator": "ge"}, {"budget": 0.3, "threshold": -0.4261830449104309, "operator": "ge"}, {"budget": 0.31, "threshold": -0.5080450177192688, "operator": "ge"}, {"budget": 0.32, "threshold": -0.6049838066101074, "operator": "ge"}, {"budget": 0.33, "threshold": -0.6995455026626587, "operator": "ge"}, {"budget": 0.34, "threshold": -0.7889041900634766, "operator": "ge"}, {"budget": 0.35, "threshold": -0.8735790252685547, "operator": "ge"}, {"budget": 0.36, "threshold": -1.002497673034668, "operator": "ge"}, {"budget": 0.37, "threshold": -1.1207135915756226, "operator": "ge"}, {"budget": 0.38, "threshold": -1.2340681552886963, "operator": "ge"}, {"budget": 0.39, "threshold": -1.3276901245117188, "operator": "ge"}, {"budget": 0.4, "threshold": -1.4334731101989746, "operator": "ge"}, {"budget": 0.41, "threshold": -1.523603916168213, "operator": "ge"}, {"budget": 0.42, "threshold": -1.643768072128296, "operator": "ge"}, {"budget": 0.43, "threshold": -1.7246742248535156, "operator": "ge"}, {"budget": 0.44, "threshold": -1.8432217836380005, "operator": "ge"}, {"budget": 0.45, "threshold": -1.9560047388076782, "operator": "ge"}, {"budget": 0.46, "threshold": -2.0564937591552734, "operator": "ge"}, {"budget": 0.47, "threshold": -2.1543874740600586, "operator": "ge"}, {"budget": 0.48, "threshold": -2.252723217010498, "operator": "ge"}, {"budget": 0.49, "threshold": -2.3540470600128174, "operator": "ge"}, {"budget": 0.5, "threshold": -2.429048776626587, "operator": "ge"}, {"budget": 0.51, "threshold": -2.527773857116699, "operator": "ge"}, {"budget": 0.52, "threshold": -2.606978416442871, "operator": "ge"}, {"budget": 0.53, "threshold": -2.6908459663391113, "operator": "ge"}, {"budget": 0.54, "threshold": -2.75626802444458, "operator": "ge"}, {"budget": 0.55, "threshold": -2.825547456741333, "operator": "ge"}, {"budget": 0.56, "threshold": -2.8964266777038574, "operator": "ge"}, {"budget": 0.57, "threshold": -2.9565629959106445, "operator": "ge"}, {"budget": 0.58, "threshold": -3.023364543914795, "operator": "ge"}, {"budget": 0.59, "threshold": -3.0734572410583496, "operator": "ge"}, {"budget": 0.6, "threshold": -3.128756284713745, "operator": "ge"}, {"budget": 0.61, "threshold": -3.1885571479797363, "operator": "ge"}, {"budget": 0.62, "threshold": -3.2457146644592285, "operator": "ge"}, {"budget": 0.63, "threshold": -3.309512138366699, "operator": "ge"}, {"budget": 0.64, "threshold": -3.360281467437744, "operator": "ge"}, {"budget": 0.65, "threshold": -3.4148216247558594, "operator": "ge"}, {"budget": 0.66, "threshold": -3.4634077548980713, "operator": "ge"}, {"budget": 0.67, "threshold": -3.5074806213378906, "operator": "ge"}, {"budget": 0.68, "threshold": -3.5620200634002686, "operator": "ge"}, {"budget": 0.69, "threshold": -3.6176204681396484, "operator": "ge"}, {"budget": 0.7, "threshold": -3.6671180725097656, "operator": "ge"}, {"budget": 0.71, "threshold": -3.7250125408172607, "operator": "ge"}, {"budget": 0.72, "threshold": -3.774223566055298, "operator": "ge"}, {"budget": 0.73, "threshold": -3.8203020095825195, "operator": "ge"}, {"budget": 0.74, "threshold": -3.8716115951538086, "operator": "ge"}, {"budget": 0.75, "threshold": -3.923274040222168, "operator": "ge"}, {"budget": 0.76, "threshold": -3.9670286178588867, "operator": "ge"}, {"budget": 0.77, "threshold": -4.011075019836426, "operator": "ge"}, {"budget": 0.78, "threshold": -4.067317008972168, "operator": "ge"}, {"budget": 0.79, "threshold": -4.108560085296631, "operator": "ge"}, {"budget": 0.8, "threshold": -4.154782772064209, "operator": "ge"}, {"budget": 0.81, "threshold": -4.207982540130615, "operator": "ge"}, {"budget": 0.82, "threshold": -4.259081840515137, "operator": "ge"}, {"budget": 0.83, "threshold": -4.311897277832031, "operator": "ge"}, {"budget": 0.84, "threshold": -4.37185525894165, "operator": "ge"}, {"budget": 0.85, "threshold": -4.413028717041016, "operator": "ge"}, {"budget": 0.86, "threshold": -4.467913627624512, "operator": "ge"}, {"budget": 0.87, "threshold": -4.520697116851807, "operator": "ge"}, {"budget": 0.88, "threshold": -4.57014274597168, "operator": "ge"}, {"budget": 0.89, "threshold": -4.637689590454102, "operator": "ge"}, {"budget": 0.9, "threshold": -4.698744773864746, "operator": "ge"}, {"budget": 0.91, "threshold": -4.755743026733398, "operator": "ge"}, {"budget": 0.92, "threshold": -4.812396049499512, "operator": "ge"}, {"budget": 0.93, "threshold": -4.9130144119262695, "operator": "ge"}, {"budget": 0.94, "threshold": -4.98988151550293, "operator": "ge"}, {"budget": 0.95, "threshold": -5.0881476402282715, "operator": "ge"}, {"budget": 0.96, "threshold": -5.227680206298828, "operator": "ge"}, {"budget": 0.97, "threshold": -5.396324157714844, "operator": "ge"}, {"budget": 0.98, "threshold": -5.636923789978027, "operator": "ge"}, {"budget": 0.99, "threshold": -5.973151206970215, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [75.56574047550845, 75.98109424233743, 76.33915783443139, 76.76883414494414, 77.15554282440561, 77.51360641649957, 77.84302492122602, 78.12947579490117, 78.40160412489257, 78.75966771698654, 79.08908622171298, 79.30392437696935, 79.56173016327699, 79.81953594958465, 80.06301919220853, 80.36379260956745, 80.59295330850759, 80.80779146376396, 81.15153251217416, 81.45230592953308, 81.71011171584072, 82.0681753079347, 82.49785161844744, 82.78430249212259, 82.98481810369522, 83.19965625895158, 83.35720423947292, 83.70094528788312, 83.87281581208822, 84.07333142366083, 84.23087940418218, 84.33113720996849, 84.46004010312231, 84.7035233457462, 85.0472643941564, 85.17616728731022, 85.46261816098539, 85.69177885992552, 85.77771412202807, 85.94958464623316, 86.09281008307075, 86.25035806359209, 86.47951876253222, 86.72300200515612, 86.92351761672873, 87.03809796619879, 87.09538814093382, 87.2386135777714, 87.25293612145518, 87.3102262961902, 87.42480664566027, 87.62532225723288, 87.7255800630192, 87.89745058722428, 88.05499856774563, 88.14093382984818, 88.19822400458321, 88.25551417931825, 88.4417072472071, 88.5419650529934, 88.6135777714122, 88.5419650529934, 88.65654540246346, 88.69951303351475, 88.74248066456603, 88.81409338298482, 88.8857061014036, 88.97164136350615, 88.94299627613864, 88.97164136350615, 89.1291893440275, 89.21512460613005, 89.22944714981381, 89.3297049556001, 89.35835004296763, 89.4013176740189, 89.5445431108565, 89.60183328559151, 89.71641363506158, 89.73073617874535, 89.7450587224291, 89.78802635348038, 89.9169292466342, 89.93125179031796, 90.06015468347178, 90.07447722715554, 90.17473503294184, 90.17473503294184, 90.27499283872815, 90.26067029504439, 90.21770266399312, 90.18905757662561, 90.21770266399312, 90.28931538241191, 90.31796046977942, 90.24634775136064, 90.30363792609567, 90.40389573188197, 90.4325408192495, 90.48983099398453, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "rescued": [0, 29, 59, 91, 119, 147, 175, 198, 219, 249, 275, 299, 322, 344, 364, 392, 415, 438, 463, 490, 511, 539, 571, 597, 619, 640, 658, 684, 704, 725, 740, 756, 772, 795, 822, 839, 862, 880, 893, 910, 927, 945, 966, 985, 1004, 1018, 1028, 1040, 1046, 1057, 1072, 1087, 1097, 1112, 1126, 1134, 1141, 1150, 1166, 1174, 1181, 1185, 1195, 1202, 1210, 1216, 1224, 1233, 1238, 1244, 1257, 1265, 1269, 1278, 1283, 1290, 1300, 1307, 1318, 1323, 1329, 1334, 1344, 1348, 1359, 1364, 1373, 1378, 1387, 1392, 1396, 1399, 1408, 1415, 1419, 1422, 1429, 1436, 1441, 1447, 1449], "harmed": [0, 0, 5, 7, 8, 11, 16, 19, 21, 26, 29, 38, 43, 47, 50, 57, 64, 72, 73, 79, 82, 85, 87, 93, 101, 107, 114, 116, 124, 131, 135, 144, 151, 157, 160, 168, 171, 173, 180, 185, 192, 199, 204, 206, 211, 217, 223, 225, 230, 237, 244, 245, 248, 251, 254, 256, 259, 264, 267, 268, 270, 279, 281, 285, 290, 291, 294, 297, 304, 308, 310, 312, 315, 317, 320, 324, 324, 327, 330, 334, 339, 341, 342, 345, 347, 351, 353, 358, 360, 366, 373, 378, 385, 387, 389, 397, 400, 400, 403, 405, 406], "exact_score": [75.56574047550845, 75.98109424233743, 76.33915783443139, 76.76883414494414, 77.15554282440561, 77.51360641649957, 77.84302492122602, 78.12947579490117, 78.40160412489257, 78.75966771698654, 79.08908622171298, 79.30392437696935, 79.56173016327699, 79.81953594958465, 80.06301919220853, 80.36379260956745, 80.59295330850759, 80.80779146376396, 81.15153251217416, 81.45230592953308, 81.71011171584072, 82.0681753079347, 82.49785161844744, 82.78430249212259, 82.98481810369522, 83.19965625895158, 83.35720423947292, 83.70094528788312, 83.87281581208822, 84.07333142366083, 84.23087940418218, 84.33113720996849, 84.46004010312231, 84.7035233457462, 85.0472643941564, 85.17616728731022, 85.46261816098539, 85.69177885992552, 85.77771412202807, 85.94958464623316, 86.09281008307075, 86.25035806359209, 86.47951876253222, 86.72300200515612, 86.92351761672873, 87.03809796619879, 87.09538814093382, 87.2386135777714, 87.25293612145518, 87.3102262961902, 87.42480664566027, 87.62532225723288, 87.7255800630192, 87.89745058722428, 88.05499856774563, 88.14093382984818, 88.19822400458321, 88.25551417931825, 88.4417072472071, 88.5419650529934, 88.6135777714122, 88.5419650529934, 88.65654540246346, 88.69951303351475, 88.74248066456603, 88.81409338298482, 88.8857061014036, 88.97164136350615, 88.94299627613864, 88.97164136350615, 89.1291893440275, 89.21512460613005, 89.22944714981381, 89.3297049556001, 89.35835004296763, 89.4013176740189, 89.5445431108565, 89.60183328559151, 89.71641363506158, 89.73073617874535, 89.7450587224291, 89.78802635348038, 89.9169292466342, 89.93125179031796, 90.06015468347178, 90.07447722715554, 90.17473503294184, 90.17473503294184, 90.27499283872815, 90.26067029504439, 90.21770266399312, 90.18905757662561, 90.21770266399312, 90.28931538241191, 90.31796046977942, 90.24634775136064, 90.30363792609567, 90.40389573188197, 90.4325408192495, 90.48983099398453, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_area100": 85.97631274461867, "threshold_area100": 85.9752936121455, "threshold_area25": 79.61185906617015}, "test": {"score": [76.42247255506925, 76.9247327258377, 77.3911171701227, 77.68529812728708, 78.00100452034154, 78.25213460572577, 78.596541579967, 78.98399942598839, 79.26383009255937, 79.49343474205352, 79.79479084451461, 80.13202267345913, 80.4405539212169, 80.73473487838129, 81.00021525435889, 81.35897251919351, 81.69620434813805, 81.94015928822559, 82.16976393771974, 82.34914257013705, 82.61462294611465, 82.8585778862022, 83.08818253569635, 83.33931262108058, 83.58326756116811, 83.84874793714573, 84.09987802252996, 84.30078209083734, 84.50886130444142, 84.6093133385951, 84.8030422616058, 84.96089545813304, 85.12592379995695, 85.27660185118748, 85.43445504771472, 85.69276027839564, 85.87931405610964, 86.21654588505417, 86.43180024395494, 86.53225227810863, 86.62552916696563, 86.77620721819616, 86.94841070531677, 87.07756332065725, 87.26411709837124, 87.33586855133817, 87.4937217478654, 87.63004950850255, 87.69462581617277, 87.945755901557, 88.0462079357107, 88.19688598694124, 88.27581258520485, 88.412140345842, 88.48389179880894, 88.49824208940231, 88.54846810647916, 88.6058692688527, 88.691971012413, 88.72067159359977, 88.81394848245677, 88.90005022601709, 88.98615196957739, 88.99332711487408, 88.98615196957739, 89.01485255076416, 89.02202769606085, 89.05790342254431, 89.0722537131377, 89.11530458491785, 89.17988089258809, 89.30903350792853, 89.40231039678554, 89.39513525148884, 89.4310109779723, 89.45253641386238, 89.43818612326899, 89.49558728564253, 89.54581330271938, 89.64626533687307, 89.66779077276314, 89.69649135394992, 89.73236708043338, 89.73954222573008, 89.78976824280693, 89.8041185334003, 89.86869484107054, 89.95479658463084, 89.997847456411, 90.00502260170768, 90.03372318289445, 90.0839491999713, 90.0839491999713, 90.14852550764154, 90.1772260888283, 90.16287579823492, 90.15570065293824, 90.0982994905647, 90.04807347348785, 90.06959890937792, 90.06242376408122], "calls": [0.0, 1.2843510081079141, 2.374973093205137, 3.3077419817751306, 4.269211451531894, 5.123053741838272, 6.098873502188419, 7.146444715505488, 8.107914185262253, 8.875654732008323, 9.78689818468824, 10.705316782664848, 11.638085671234842, 12.556504269211452, 13.668651790198751, 14.852550764152975, 15.907297122766737, 16.75396426777642, 17.715433737533186, 18.310970797158642, 19.236564540431942, 20.348712061419246, 21.582837052450312, 22.59453253928392, 23.54882686374399, 24.62509865824783, 25.70854559804836, 26.74894166606874, 27.71758628112219, 28.657530314988875, 29.626174930042332, 30.680921288656094, 31.814594245533474, 32.81911458707039, 33.80928463801392, 34.81380497955084, 36.16273229532898, 37.55471048288727, 38.63098227739112, 39.30544593528019, 40.21668938796011, 41.02030566118964, 42.053526583913325, 42.792566549472625, 43.904714070459924, 44.765731506063, 45.84200330056684, 46.8034727703236, 48.0088971801679, 49.27172275238574, 50.03228815383511, 51.20901198249265, 52.29963406758987, 53.41178158857717, 54.29432446007032, 55.10511587859654, 56.23878883547392, 57.028054818110064, 57.946473416086675, 58.879242304656664, 59.79048575733658, 60.90263327832388, 61.78517614981703, 62.82557221783741, 63.65788907225372, 64.66958455908733, 65.67410490062424, 66.47772117385377, 67.51811724187414, 68.64461505345483, 69.59890937791491, 70.7254071894956, 71.77297840281265, 72.61247040252566, 73.50936356461219, 74.54258448733587, 75.27444930759847, 76.20004305087178, 77.4700437683863, 78.37411207576953, 79.31405610963623, 80.41185334003013, 81.43789911745712, 82.56439692903781, 83.88462366362918, 84.8030422616058, 85.84343832962618, 86.94841070531677, 87.83095357680993, 89.04355313195093, 90.01219774700438, 90.85168974671737, 91.72705747291383, 92.89660615627466, 93.81502475425127, 94.8984716940518, 96.16847241156633, 97.18016789839994, 98.40711774413433, 99.22508430795723, 100.0], "rescued": [0, 75, 147, 193, 244, 286, 342, 407, 452, 494, 541, 590, 642, 687, 738, 795, 852, 895, 938, 966, 1010, 1056, 1104, 1151, 1190, 1241, 1285, 1327, 1368, 1398, 1437, 1475, 1514, 1553, 1591, 1638, 1685, 1744, 1789, 1812, 1838, 1867, 1899, 1923, 1959, 1984, 2016, 2043, 2064, 2108, 2126, 2158, 2172, 2198, 2212, 2225, 2244, 2258, 2276, 2284, 2302, 2320, 2337, 2347, 2356, 2368, 2379, 2388, 2398, 2415, 2431, 2456, 2473, 2481, 2493, 2504, 2511, 2522, 2543, 2561, 2573, 2590, 2601, 2611, 2628, 2634, 2648, 2665, 2678, 2689, 2703, 2711, 2718, 2732, 2742, 2755, 2764, 2768, 2776, 2786, 2791], "harmed": [0, 5, 12, 17, 24, 31, 39, 50, 56, 66, 71, 73, 82, 86, 100, 107, 117, 126, 137, 140, 147, 159, 175, 187, 192, 206, 215, 229, 241, 257, 269, 285, 301, 319, 335, 346, 367, 379, 394, 403, 416, 424, 432, 438, 448, 463, 473, 481, 493, 502, 506, 517, 520, 527, 531, 542, 554, 560, 566, 570, 575, 581, 586, 595, 605, 613, 623, 627, 635, 646, 653, 660, 664, 673, 680, 688, 697, 700, 714, 718, 727, 740, 746, 755, 765, 769, 774, 779, 786, 796, 806, 807, 814, 819, 825, 840, 850, 862, 877, 884, 890], "exact_score": [76.42247255506925, 76.79558011049724, 77.25478940948555, 77.5776709478367, 77.943603357968, 78.20908373394562, 78.55349070818684, 78.92659826361485, 79.20642893018584, 79.55801104972376, 79.87371744277823, 80.21812441701944, 80.56253139126068, 80.85671234842506, 81.11501757910598, 81.39484824567698, 81.71055463873144, 82.00473559589582, 82.22716510009327, 82.57157207433451, 82.78682643323528, 82.9446796297625, 83.2245102963335, 83.43976465523427, 83.71242017650857, 83.89897395422258, 84.16445433020019, 84.37970868910097, 84.5303867403315, 84.67388964626534, 84.88196885986942, 85.00394632991318, 85.1689746717371, 85.29812728707756, 85.46315562890149, 85.69276027839564, 85.86496376551625, 86.06586783382365, 86.32417306450456, 86.4676759704384, 86.5609528592954, 86.76903207289948, 86.9340604147234, 87.09191361125063, 87.26411709837124, 87.3932697137117, 87.50089689316209, 87.63004950850255, 87.69462581617277, 87.88117959388678, 88.03903279041401, 88.18971084164454, 88.25428714931478, 88.36191432876515, 88.48389179880894, 88.49824208940231, 88.54129296118248, 88.6058692688527, 88.691971012413, 88.72784673889646, 88.82112362775347, 88.90722537131377, 88.99332711487408, 88.97897682428069, 89.01485255076416, 89.00050226017078, 89.0722537131377, 89.07942885843438, 89.07942885843438, 89.1440051661046, 89.20858147377486, 89.3520843797087, 89.39513525148884, 89.41666068737892, 89.44536126856569, 89.45253641386238, 89.45971155915908, 89.5386381574227, 89.56733873860946, 89.65344048216977, 89.67496591805984, 89.71084164454331, 89.74671737102676, 89.76106766162016, 89.78976824280693, 89.8041185334003, 89.87586998636723, 89.96914687522423, 89.99067231111431, 89.997847456411, 90.03372318289445, 90.06959890937792, 90.07677405467462, 90.15570065293824, 90.1772260888283, 90.1772260888283, 90.14852550764154, 90.10547463586138, 90.06959890937792, 90.06242376408122, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "exact_area100": 86.34930048820179, "threshold_area100": 86.33504340962904, "threshold_area25": 80.43122623233121}, "selected_budget": 100, "selected": {"val": {"score": 90.50415353766829, "calls": 100.0, "rescued": 1449, "harmed": 406}, "test": {"score": 90.06242376408122, "calls": 100.0, "rescued": 2791, "harmed": 890}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 90.71899169292466, "calls": 98.99742194213692}, "test": {"score": 90.1772260888283, "calls": 99.11745712850684}}, "old_normalized_interior": {"val": {"score": 90.53279862503581, "calls": 96.99226582641077}, "test": {"score": 90.04089832819116, "calls": 97.05819042835617}}, "old_normalized_endpoints": {"val": {"score": 90.53279862503581, "calls": 96.99226582641077}, "test": {"score": 90.04089832819116, "calls": 97.05819042835617}}}, "baseline_old_normalized": {"val": {"score": [75.56574047550845, 75.95244915496991, 76.38212546548266, 76.75451160126038, 77.0552850186193, 77.45631624176454, 77.77141220280721, 78.12947579490117, 78.57347464909768, 78.96018332855915, 79.24663420223432, 79.61902033801202, 79.81953594958465, 80.16327699799484, 80.47837295903753, 80.80779146376396, 81.09424233743913, 81.48095101690059, 81.63849899742193, 81.86765969636207, 82.05385276425092, 82.25436837582353, 82.41191635634489, 82.52649670581495, 82.75565740475508, 82.99914064737897, 83.14236608421656, 83.32855915210541, 83.52907476367803, 83.71526783156689, 83.8441707247207, 83.95875107419077, 84.11629905471212, 84.27384703523344, 84.44571755943855, 84.73216843311371, 84.91836150100256, 85.03294185047264, 85.17616728731022, 85.3910054425666, 85.51990833572042, 85.69177885992552, 85.8350042967631, 86.078487539387, 86.37926095674592, 86.49384130621598, 86.59409911200228, 86.75164709252363, 86.80893726725867, 86.88054998567746, 87.02377542251504, 87.15267831566887, 87.29590375250645, 87.42480664566027, 87.56803208249785, 87.61099971354912, 87.66828988828415, 87.78287023775422, 87.95474076195933, 88.05499856774563, 88.15525637353194, 88.22686909195072, 88.32712689773703, 88.37009452878831, 88.47035233457461, 88.65654540246346, 88.75680320824978, 88.7854482956173, 88.84273847035233, 88.94299627613864, 89.0575766256087, 89.2008020624463, 89.24376969349757, 89.27241478086508, 89.37267258665138, 89.44428530507018, 89.47293039243769, 89.51589802348897, 89.6448009166428, 89.70209109137782, 89.7450587224291, 89.78802635348038, 89.8166714408479, 89.84531652821542, 89.9169292466342, 90.00286450873675, 90.0887997708393, 90.21770266399312, 90.24634775136064, 90.27499283872815, 90.28931538241191, 90.34660555714696, 90.38957318819823, 90.41821827556574, 90.4325408192495, 90.47550845030077, 90.48983099398453, 90.53279862503581, 90.51847608135203, 90.53279862503581, 90.50415353766829], "calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "exact_score": [75.56574047550845, 75.95244915496991, 76.38212546548266, 76.75451160126038, 77.0552850186193, 77.45631624176454, 77.77141220280721, 78.12947579490117, 78.57347464909768, 78.96018332855915, 79.24663420223432, 79.61902033801202, 79.81953594958465, 80.16327699799484, 80.47837295903753, 80.80779146376396, 81.09424233743913, 81.48095101690059, 81.63849899742193, 81.86765969636207, 82.05385276425092, 82.25436837582353, 82.41191635634489, 82.52649670581495, 82.75565740475508, 82.99914064737897, 83.14236608421656, 83.32855915210541, 83.52907476367803, 83.71526783156689, 83.8441707247207, 83.95875107419077, 84.11629905471212, 84.27384703523344, 84.44571755943855, 84.73216843311371, 84.91836150100256, 85.03294185047264, 85.17616728731022, 85.3910054425666, 85.51990833572042, 85.69177885992552, 85.8350042967631, 86.078487539387, 86.37926095674592, 86.49384130621598, 86.59409911200228, 86.75164709252363, 86.80893726725867, 86.88054998567746, 87.02377542251504, 87.15267831566887, 87.29590375250645, 87.42480664566027, 87.56803208249785, 87.61099971354912, 87.66828988828415, 87.78287023775422, 87.95474076195933, 88.05499856774563, 88.15525637353194, 88.22686909195072, 88.32712689773703, 88.37009452878831, 88.47035233457461, 88.65654540246346, 88.75680320824978, 88.7854482956173, 88.84273847035233, 88.94299627613864, 89.0575766256087, 89.2008020624463, 89.24376969349757, 89.27241478086508, 89.37267258665138, 89.44428530507018, 89.47293039243769, 89.51589802348897, 89.6448009166428, 89.70209109137782, 89.7450587224291, 89.78802635348038, 89.8166714408479, 89.84531652821542, 89.9169292466342, 90.00286450873675, 90.0887997708393, 90.21770266399312, 90.24634775136064, 90.27499283872815, 90.28931538241191, 90.34660555714696, 90.38957318819823, 90.41821827556574, 90.4325408192495, 90.47550845030077, 90.48983099398453, 90.53279862503581, 90.51847608135203, 90.53279862503581, 90.50415353766829], "exact_calls": [0.0, 0.9882555141793182, 1.9908335720423949, 2.9934116299054714, 3.9959896877685477, 4.998567745631624, 5.986823259810943, 6.989401317674019, 7.991979375537095, 8.994557433400171, 9.997135491263249, 10.999713549126325, 11.987969063305643, 12.990547121168719, 13.993125179031797, 14.995703236894872, 15.998281294757948, 16.986536808937267, 17.989114866800342, 18.99169292466342, 19.994270982526498, 20.996849040389574, 21.99942709825265, 22.987682612431968, 23.990260670295044, 24.99283872815812, 25.995416786021195, 26.99799484388428, 27.986250358063593, 28.98882841592667, 29.991406473789745, 30.993984531652824, 31.996562589515896, 32.99914064737898, 33.987396161558294, 34.98997421942137, 35.992552277284446, 36.99513033514752, 37.9977083930106, 38.98596390718992, 39.988541965052995, 40.99112002291607, 41.99369808077915, 42.99627613864222, 43.9988541965053, 44.98710971068462, 45.9896877685477, 46.99226582641077, 47.99484388427385, 48.997421942136924, 50.0, 50.988255514179315, 51.99083357204239, 52.993411629905474, 53.99598968776856, 54.99856774563162, 55.98682325981095, 56.989401317674016, 57.99197937553709, 58.994557433400175, 59.99713549126325, 60.99971354912632, 61.98796906330565, 62.99054712116872, 63.99312517903179, 64.99570323689487, 65.99828129475796, 66.98653680893727, 67.98911486680035, 68.99169292466341, 69.9942709825265, 70.99684904038958, 71.99942709825265, 72.98768261243197, 73.99026067029504, 74.99283872815812, 75.9954167860212, 76.99799484388427, 77.98625035806359, 78.98882841592668, 79.99140647378975, 80.99398453165281, 81.9965625895159, 82.99914064737898, 83.9873961615583, 84.98997421942137, 85.99255227728445, 86.99513033514752, 87.9977083930106, 88.98596390718991, 89.98854196505299, 90.99112002291608, 91.99369808077915, 92.99627613864222, 93.9988541965053, 94.98710971068462, 95.9896877685477, 96.99226582641077, 97.99484388427385, 98.99742194213692, 100.0], "threshold_area25": 79.76940704669148, "exact_area100": 85.83399092859071}, "test": {"score": [76.42247255506925, 76.75252923871709, 77.25478940948555, 77.65659754610031, 78.10145655449523, 78.38128722106623, 78.63959245174715, 79.04857573365861, 79.17055320370238, 79.49343474205352, 79.71586424625099, 79.95264404104184, 80.26835043409629, 80.59123197244745, 80.74908516897467, 81.02891583554567, 81.22264475855636, 81.49530027983067, 81.78948123699506, 81.99038530530244, 82.26304082657674, 82.41371887780727, 82.68637439908159, 83.03078137332281, 83.24603573222358, 83.43258950993759, 83.56174212527804, 83.80569706536558, 83.98507569778289, 84.20750520198035, 84.44428499677119, 84.63083877448518, 84.85326827868265, 85.03982205639664, 85.1761498170338, 85.38422903063787, 85.52773193657171, 85.62100882542872, 85.88648920140633, 86.0156418167468, 86.24524646624094, 86.3743990815814, 86.48202626103179, 86.61117887637225, 86.80490779938295, 86.94123556002009, 86.99146157709694, 87.1206141924374, 87.15648991892087, 87.45067087608524, 87.5941737820191, 87.6731003802827, 87.80225299562316, 87.91705532037024, 87.96010619215039, 88.02468249982063, 88.132309679271, 88.1394848245677, 88.19688598694124, 88.28298773050155, 88.3260386022817, 88.39779005524862, 88.47671665351223, 88.52694267058908, 88.6058692688527, 88.75654732008323, 88.79959819186338, 88.87134964483032, 89.00050226017078, 89.05790342254431, 89.1440051661046, 89.16553060199469, 89.20140632847816, 89.29468321733516, 89.3377340891153, 89.38078496089545, 89.39513525148884, 89.37360981559877, 89.43818612326899, 89.47406184975246, 89.48123699504916, 89.531463012126, 89.56016359331277, 89.6103896103896, 89.66061562746646, 89.71084164454331, 89.76106766162016, 89.82564396929038, 89.82564396929038, 89.83999425988375, 89.8902202769606, 89.91892085814737, 90.00502260170768, 89.97632202052091, 90.01219774700438, 90.01937289230108, 90.01937289230108, 90.04089832819116, 90.06959890937792, 90.07677405467462, 90.06242376408122], "calls": [0.0, 1.033220922723685, 2.4897754179522136, 3.6234483748295903, 4.721245605223506, 5.388534117815886, 6.3141278610891876, 7.34017363851618, 8.122264475855637, 8.940231039678553, 9.94475138121547, 10.798593671521848, 11.925091483102532, 12.872210662265912, 13.654301499605367, 14.594245533472053, 15.390686661404892, 16.409557293535194, 17.335151036808497, 18.533400301356103, 19.4661691899261, 20.406113223792783, 21.38910812943962, 22.666283992250843, 23.656454043194376, 24.804477290665137, 25.665494726268207, 26.576738178948123, 27.523857358111503, 28.44227595608811, 29.439621152328332, 30.221711989667792, 31.10425486116094, 31.96527229676401, 33.09894525364139, 34.04606443280476, 35.08646050082514, 35.918777355241446, 36.966348568558516, 37.698213388821124, 38.70990887565473, 39.764655234268496, 40.53957092631126, 41.44363923369448, 42.54861160938509, 43.73251058333931, 44.622228600129155, 45.69850039463299, 46.638444428499675, 48.15240008610174, 49.286073042979126, 50.211666786252415, 51.30946401664633, 52.50771328119394, 53.31850469972017, 54.66743201549832, 55.614551194661686, 56.432517758484614, 57.200258305230676, 58.176078065580825, 58.98686948410705, 60.0559661333142, 60.83088182535696, 61.95020449164096, 62.940374542584486, 63.80139197818756, 64.94224008036163, 66.09743847312907, 67.26698715648992, 68.46523642103752, 69.36212958312406, 70.4455765229246, 71.30659395852767, 72.36851546243811, 73.42326182105188, 74.6071607950061, 75.48252852120255, 76.56597546100309, 77.42699289660617, 78.56066585348354, 79.342756690823, 80.09614694697568, 81.05761641673243, 81.87558298055536, 82.89445361268565, 84.0568271507498, 85.11874865466027, 86.5681280045921, 87.7663772691397, 88.95745138839062, 89.997847456411, 91.08129439621152, 92.3871708402095, 93.5208437970869, 94.3459855062065, 95.09937576235919, 96.22587357393986, 97.05819042835617, 97.86898184688239, 99.1676831455837, 100.0], "exact_score": [76.42247255506925, 76.74535409342039, 77.10411135825501, 77.42699289660617, 77.81445074262754, 78.23778431513239, 78.54631556289016, 78.90507282772477, 79.17055320370238, 79.50778503264692, 79.75173997273444, 80.01722034871206, 80.28270072468968, 80.59840711774415, 80.84236205783168, 81.11501757910598, 81.42354882686375, 81.68902920284135, 81.88993327114873, 82.13388821123628, 82.36349286073043, 82.53569634785106, 82.81552701442205, 83.13840855277319, 83.29626174930043, 83.45411494582765, 83.5976178517615, 83.85592308244242, 84.13575374901342, 84.32230752672743, 84.57343761211165, 84.83174284279256, 85.05417234699003, 85.18332496233049, 85.37705388534118, 85.51338164597834, 85.62100882542872, 85.87213891081295, 86.12326899619718, 86.30264762861448, 86.39592451747149, 86.52507713281194, 86.69010547463587, 86.89100954294324, 86.95558585061349, 87.04886273947048, 87.1206141924374, 87.22824137188778, 87.43632058549186, 87.52959747434885, 87.665925234986, 87.7735524144364, 87.88117959388678, 87.98163162804047, 87.98880677333716, 88.0533830810074, 88.12513453397432, 88.18253569634786, 88.26146229461148, 88.31886345698501, 88.39061490995192, 88.48389179880894, 88.5125923799957, 88.62021955944608, 88.74219702948984, 88.79959819186338, 88.87134964483032, 88.94310109779724, 89.04355313195093, 89.11530458491785, 89.15118031140132, 89.20140632847816, 89.2659826361484, 89.31620865322523, 89.3377340891153, 89.39513525148884, 89.38078496089545, 89.39513525148884, 89.47406184975246, 89.48123699504916, 89.531463012126, 89.54581330271938, 89.6103896103896, 89.66779077276314, 89.71084164454331, 89.75389251632346, 89.78976824280693, 89.811293678697, 89.83999425988375, 89.83999425988375, 89.8902202769606, 89.91174571285069, 89.99067231111431, 90.00502260170768, 90.01219774700438, 90.01937289230108, 90.03372318289445, 90.03372318289445, 90.06959890937792, 90.07677405467462, 90.06242376408122], "exact_calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "threshold_area25": 80.26332783238863, "exact_area100": 86.12579499536474}}, "old_normalized_endpoint_budget": 97}]}, {"expert": "yolo26x", "endpoint": "MiniCPM", "task": "construction", "small": 18.129624718318375, "large": 22.361115727328066, "runs": [{"seed": 2026, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.5, "validation_area100": 0.24820179625573882, "training_seconds": 5.142528719035909, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.3346830844430999, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3180826937380077, "operator": "ge"}, {"budget": 0.03, "threshold": 0.2960649314028587, "operator": "ge"}, {"budget": 0.04, "threshold": 0.26926586546014847, "operator": "ge"}, {"budget": 0.05, "threshold": 0.23586783383478888, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2175596998306078, "operator": "ge"}, {"budget": 0.07, "threshold": 0.1915239977591481, "operator": "ge"}, {"budget": 0.08, "threshold": 0.16867875499874907, "operator": "ge"}, {"budget": 0.09, "threshold": 0.14852250411677637, "operator": "ge"}, {"budget": 0.1, "threshold": 0.08060658000156996, "operator": "ge"}, {"budget": 0.11, "threshold": 0.0014228986816543167, "operator": "ge"}, {"budget": 0.12, "threshold": -0.008574556380122443, "operator": "ge"}, {"budget": 0.13, "threshold": -0.01492546963892291, "operator": "ge"}, {"budget": 0.14, "threshold": -0.01995066938207865, "operator": "ge"}, {"budget": 0.15, "threshold": -0.025570005596881395, "operator": "ge"}, {"budget": 0.16, "threshold": -0.030638490307985676, "operator": "ge"}, {"budget": 0.17, "threshold": -0.034442768976397996, "operator": "ge"}, {"budget": 0.18, "threshold": -0.03792279544895927, "operator": "ge"}, {"budget": 0.19, "threshold": -0.03890040909964789, "operator": "ge"}, {"budget": 0.2, "threshold": -0.042058647121907065, "operator": "ge"}, {"budget": 0.21, "threshold": -0.04807498262749599, "operator": "ge"}, {"budget": 0.22, "threshold": -0.05349816072156135, "operator": "ge"}, {"budget": 0.23, "threshold": -0.05610377439220817, "operator": "ge"}, {"budget": 0.24, "threshold": -0.059391320086534444, "operator": "ge"}, {"budget": 0.25, "threshold": -0.0629914296746294, "operator": "ge"}, {"budget": 0.26, "threshold": -0.06586441665521951, "operator": "ge"}, {"budget": 0.27, "threshold": -0.06859419706730177, "operator": "ge"}, {"budget": 0.28, "threshold": -0.0733954696043712, "operator": "ge"}, {"budget": 0.29, "threshold": -0.07571750528164707, "operator": "ge"}, {"budget": 0.3, "threshold": -0.07768063627146876, "operator": "ge"}, {"budget": 0.31, "threshold": -0.079105638907849, "operator": "ge"}, {"budget": 0.32, "threshold": -0.08297073070077074, "operator": "ge"}, {"budget": 0.33, "threshold": -0.08579907718457228, "operator": "ge"}, {"budget": 0.34, "threshold": -0.08990302034297061, "operator": "ge"}, {"budget": 0.35, "threshold": -0.09629052428184884, "operator": "ge"}, {"budget": 0.36, "threshold": -0.09929329570954519, "operator": "ge"}, {"budget": 0.37, "threshold": -0.10564677527210165, "operator": "ge"}, {"budget": 0.38, "threshold": -0.10786723446075945, "operator": "ge"}, {"budget": 0.39, "threshold": -0.11249164214896092, "operator": "ge"}, {"budget": 0.4, "threshold": -0.12138909632603587, "operator": "ge"}, {"budget": 0.41, "threshold": -0.13034631533547675, "operator": "ge"}, {"budget": 0.42, "threshold": -0.13802887608222977, "operator": "ge"}, {"budget": 0.43, "threshold": -0.14680807243161081, "operator": "ge"}, {"budget": 0.44, "threshold": -0.17476482649719544, "operator": "ge"}, {"budget": 0.45, "threshold": -0.23661273220781207, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3365704427855552, "operator": "ge"}, {"budget": 0.47, "threshold": -0.34847194909437196, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3524123045550812, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3639279487388174, "operator": "ge"}, {"budget": 0.5, "threshold": -0.37121761398491154, "operator": "ge"}, {"budget": 0.51, "threshold": -0.37999895585911225, "operator": "ge"}, {"budget": 0.52, "threshold": -0.3853421893318952, "operator": "ge"}, {"budget": 0.53, "threshold": -0.39761746772530415, "operator": "ge"}, {"budget": 0.54, "threshold": -0.4049981220185869, "operator": "ge"}, {"budget": 0.55, "threshold": -0.41657265505376107, "operator": "ge"}, {"budget": 0.56, "threshold": -0.4266926183610899, "operator": "ge"}, {"budget": 0.57, "threshold": -0.43507394707734415, "operator": "ge"}, {"budget": 0.58, "threshold": -0.4401324289161748, "operator": "ge"}, {"budget": 0.59, "threshold": -0.4446414207446072, "operator": "ge"}, {"budget": 0.6, "threshold": -0.4507535051987675, "operator": "ge"}, {"budget": 0.61, "threshold": -0.4530847867765632, "operator": "ge"}, {"budget": 0.62, "threshold": -0.4545289269400208, "operator": "ge"}, {"budget": 0.63, "threshold": -0.4574137086819433, "operator": "ge"}, {"budget": 0.64, "threshold": -0.46042748139849865, "operator": "ge"}, {"budget": 0.65, "threshold": -0.46326582396550986, "operator": "ge"}, {"budget": 0.66, "threshold": -0.4678538384863775, "operator": "ge"}, {"budget": 0.67, "threshold": -0.46917989460294945, "operator": "ge"}, {"budget": 0.68, "threshold": -0.4709624447445655, "operator": "ge"}, {"budget": 0.69, "threshold": -0.4729388117157893, "operator": "ge"}, {"budget": 0.7, "threshold": -0.47551676686826705, "operator": "ge"}, {"budget": 0.71, "threshold": -0.47767098634011124, "operator": "ge"}, {"budget": 0.72, "threshold": -0.4804087267892626, "operator": "ge"}, {"budget": 0.73, "threshold": -0.4818965072239871, "operator": "ge"}, {"budget": 0.74, "threshold": -0.48386663561239746, "operator": "ge"}, {"budget": 0.75, "threshold": -0.4859288718483971, "operator": "ge"}, {"budget": 0.76, "threshold": -0.48799710135614144, "operator": "ge"}, {"budget": 0.77, "threshold": -0.49078907847350367, "operator": "ge"}, {"budget": 0.78, "threshold": -0.49255559836637025, "operator": "ge"}, {"budget": 0.79, "threshold": -0.4939538959629259, "operator": "ge"}, {"budget": 0.8, "threshold": -0.49629580354890107, "operator": "ge"}, {"budget": 0.81, "threshold": -0.49755937480637263, "operator": "ge"}, {"budget": 0.82, "threshold": -0.4987189341748684, "operator": "ge"}, {"budget": 0.83, "threshold": -0.5007613206845215, "operator": "ge"}, {"budget": 0.84, "threshold": -0.5018879874833267, "operator": "ge"}, {"budget": 0.85, "threshold": -0.5040714583402874, "operator": "ge"}, {"budget": 0.86, "threshold": -0.5051167105591194, "operator": "ge"}, {"budget": 0.87, "threshold": -0.5058769842311478, "operator": "ge"}, {"budget": 0.88, "threshold": -0.5075403676683711, "operator": "ge"}, {"budget": 0.89, "threshold": -0.5097234704168284, "operator": "ge"}, {"budget": 0.9, "threshold": -0.5118273369531179, "operator": "ge"}, {"budget": 0.91, "threshold": -0.5132885259885954, "operator": "ge"}, {"budget": 0.92, "threshold": -0.5169568959365931, "operator": "ge"}, {"budget": 0.93, "threshold": -0.5198967864420649, "operator": "ge"}, {"budget": 0.94, "threshold": -0.5211616620786012, "operator": "ge"}, {"budget": 0.95, "threshold": -0.5239375467167025, "operator": "ge"}, {"budget": 0.96, "threshold": -0.5273060910003506, "operator": "ge"}, {"budget": 0.97, "threshold": -0.5357380015398889, "operator": "ge"}, {"budget": 0.98, "threshold": -0.5414272093691888, "operator": "ge"}, {"budget": 0.99, "threshold": -0.549153356063018, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.450958204581394, 18.73127093384612, 19.667374587587354, 20.227756980355966, 20.606650531411134, 21.033389179408395, 21.406179492519698, 21.87467398219998, 22.339935974389757, 22.60171840354767, 22.633668852156248, 22.537217045234723, 22.726528013434464, 24.00372304062949, 24.127714523561377, 23.915522183716252, 23.910167230656914, 24.251315910979777, 23.79706723947468, 24.102156334231804, 24.65642979174095, 24.56099375279737, 24.833616342778082, 25.352847945297867, 25.54490926037612, 25.681106975461816, 25.94965786901271, 26.090476190476185, 26.031198895614317, 26.512402352867948, 26.63298072881231, 26.988846283027318, 27.52275826532653, 28.132465788111993, 28.245932365433173, 28.319327731092436, 26.473016485958677, 26.655337815027043, 26.91266336003178, 26.97102125722864, 27.076413604487076, 27.004518685877255, 27.560069129836574, 27.68686868686869, 28.55822749811899, 28.85771696562181, 28.841174368434054, 28.695004445451776, 28.611522237732217, 28.43172872905101, 28.384555209120194, 28.4054362485735, 27.914438502673793, 27.731500976417102, 27.679458301423605, 27.626311038609135, 27.35085376543837, 27.444573911883175, 27.497884907015717, 27.446880138804364, 27.58222989572709, 27.437608702639576, 27.382747970983267, 27.21277465955689, 25.92759438138063, 25.867558621923497, 25.82265258199626, 25.38038102177721, 25.328313959400866, 25.28316330279366, 25.22255427949023, 25.26858235783868, 25.184353561130813, 24.469712677259846, 24.430954458795924, 24.604733420083807, 24.557109557109555, 24.515541833977586, 24.046282657765218, 23.91011861394218, 23.793521845405056, 23.520442456799596, 23.405725613129263, 23.366734143049932, 23.30727783341316, 23.254979868616232, 23.19404951587204, 22.487206688388774, 22.434331311025037, 22.50263059179573, 22.380466609044067, 21.739868285176257, 21.61211846906735, 21.48175129962405, 21.349718297701433, 21.220691659809948, 21.09042080867144, 20.96737216279762, 20.78078099917753, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 23, 35, 48, 60, 70, 80, 90, 101, 110, 115, 119, 124, 132, 136, 139, 144, 148, 154, 159, 164, 170, 174, 178, 184, 191, 196, 201, 205, 212, 216, 219, 224, 229, 234, 239, 243, 247, 251, 252, 256, 259, 264, 266, 270, 274, 274, 274, 274, 275, 275, 275, 275, 276, 276, 277, 277, 279, 280, 280, 281, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 283, 283, 283, 283, 284, 284, 284, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 3, 5, 9, 10, 12, 14, 18, 20, 23, 25, 28, 33, 39, 45, 50, 53, 58, 60, 71, 77, 81, 85, 88, 96, 103, 107, 112, 119, 123, 130, 133, 138, 145, 152, 156, 162, 169, 173, 175, 176, 184, 190, 195, 198, 204, 208, 209, 213, 218, 223, 226, 234, 243, 248, 252, 257, 262, 267, 270, 274, 278, 284, 289, 296, 303, 308, 312, 318, 322, 327, 333, 337, 345, 351, 360, 365, 370, 375, 381, 387, 391, 395, 399, 404, 411, 417, 425, 432, 439, 444, 450, 457, 465, 472, 480, 487, 495, 502, 511, 518], "exact_score": [18.254072168965788, 18.450958204581394, 18.73127093384612, 19.667374587587354, 20.227756980355966, 20.606650531411134, 21.033389179408395, 21.406179492519698, 21.87467398219998, 22.339935974389757, 22.60171840354767, 22.633668852156248, 22.537217045234723, 22.726528013434464, 24.00372304062949, 24.127714523561377, 23.915522183716252, 23.910167230656914, 24.251315910979777, 23.79706723947468, 24.102156334231804, 24.65642979174095, 24.56099375279737, 24.833616342778082, 25.352847945297867, 25.54490926037612, 25.681106975461816, 25.94965786901271, 26.090476190476185, 26.031198895614317, 26.512402352867948, 26.63298072881231, 26.988846283027318, 27.52275826532653, 28.132465788111993, 28.245932365433173, 28.319327731092436, 26.473016485958677, 26.655337815027043, 26.91266336003178, 26.97102125722864, 27.076413604487076, 27.004518685877255, 27.560069129836574, 27.68686868686869, 28.55822749811899, 28.85771696562181, 28.841174368434054, 28.695004445451776, 28.611522237732217, 28.43172872905101, 28.384555209120194, 28.4054362485735, 27.914438502673793, 27.731500976417102, 27.679458301423605, 27.626311038609135, 27.35085376543837, 27.444573911883175, 27.497884907015717, 27.446880138804364, 27.58222989572709, 27.437608702639576, 27.382747970983267, 27.21277465955689, 25.92759438138063, 25.867558621923497, 25.82265258199626, 25.38038102177721, 25.328313959400866, 25.28316330279366, 25.22255427949023, 25.26858235783868, 25.184353561130813, 24.469712677259846, 24.430954458795924, 24.604733420083807, 24.557109557109555, 24.515541833977586, 24.046282657765218, 23.91011861394218, 23.793521845405056, 23.520442456799596, 23.405725613129263, 23.366734143049932, 23.30727783341316, 23.254979868616232, 23.19404951587204, 22.487206688388774, 22.434331311025037, 22.50263059179573, 22.380466609044067, 21.739868285176257, 21.61211846906735, 21.48175129962405, 21.349718297701433, 21.220691659809948, 21.09042080867144, 20.96737216279762, 20.78078099917753, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.820179625573882, "threshold_area100": 24.8260321694227, "threshold_area25": 22.667546101484586}, "test": {"score": [18.129624718318375, 18.238843178495635, 18.34755613296779, 18.82721908149278, 19.165615777207034, 19.43607985663063, 19.83558050909899, 20.239844988054408, 20.617354052148848, 20.922679213800276, 21.07571883624894, 20.679831681476415, 20.89269059930863, 21.20717958441772, 21.22889132424066, 21.323735102050463, 21.55669022054009, 21.642620031743856, 21.89090331490534, 21.98290366608791, 22.02648879809414, 22.27207345201864, 22.595592334657034, 22.926844707017118, 23.45011343395643, 23.869664940217422, 24.065314705021624, 24.28020431770867, 24.445297085252506, 24.713646981007106, 24.71084793934257, 24.758503962901475, 24.843409041795272, 24.8744641853353, 24.962243913862714, 25.095229519884448, 25.144476250951225, 25.47337613472955, 25.724000420349714, 25.845512755625116, 25.773310059142716, 26.047289653588475, 26.072393971808683, 26.07938358400268, 26.299792192023908, 25.923967529910858, 25.92463564955129, 25.761318045407776, 25.761914478722005, 25.57865729123383, 25.545183581564253, 25.91573299267027, 25.865910161873156, 25.262049109408093, 25.19746481417885, 25.151952148127975, 24.833763965121953, 24.865985580384084, 24.72409351095787, 24.28202967515945, 24.555245836452066, 24.350085736362935, 24.197977172518996, 24.150604481358233, 24.025614891861192, 23.972130056299562, 23.769857498561276, 23.683138475249553, 23.64776758870048, 24.026198743977943, 23.951004137459975, 24.46168430493389, 24.411544505287146, 24.291125173979296, 24.34599443592801, 24.146533458835858, 23.950393554703638, 23.62240288912983, 23.44271497394585, 23.371667815720805, 23.247534954273675, 23.15868593494286, 23.149845537399727, 22.89837788473718, 22.919145541813585, 22.623054222192053, 22.55297821837654, 22.504657610283918, 22.844119292563626, 22.706874032482467, 22.92489733274623, 22.81988719597151, 22.68826415153772, 22.461481222682707, 22.399447596561377, 22.25093841193937, 22.032613686712047, 22.107813377883648, 22.458141533832197, 22.40734249883748, 22.361115727328066], "calls": [0.0, 0.8322237017310253, 1.2982689747003995, 2.0639147802929427, 3.262316910785619, 4.527296937416778, 5.625832223701731, 7.223701731025299, 8.388814913448735, 9.221038615179761, 10.85219707057257, 12.51664447403462, 13.482023968042608, 14.214380825565911, 15.07989347536618, 15.978695073235686, 17.14380825565912, 17.976031957390145, 18.874833555259656, 19.207723035952064, 19.973368841544605, 21.937416777629824, 23.834886817576564, 24.70039946737683, 25.732356857523303, 27.19707057256991, 28.395472703062584, 29.59387483355526, 30.825565912117174, 31.724367509986685, 32.256990679094535, 32.52330226364847, 33.45539280958722, 34.25432756324901, 34.9866844207723, 36.484687083888154, 36.8508655126498, 37.88282290279627, 38.548601864181094, 39.114513981358186, 40.279627163781626, 41.37816245006658, 41.84420772303595, 42.24367509986685, 43.40878828229028, 44.54061251664447, 46.17177097203729, 47.17043941411451, 47.969374167776294, 49.60053262316911, 50.36617842876166, 51.33155792276965, 51.83089214380826, 52.92942743009321, 53.52862849533955, 54.460719041278296, 55.49267643142477, 56.22503328894807, 57.02396804260985, 58.089214380825574, 59.92010652463382, 60.685752330226364, 61.35153129161118, 62.61651131824234, 63.61517976031957, 64.68042609853528, 66.81091877496671, 67.27696404793608, 68.10918774966711, 68.84154460719041, 69.84021304926765, 70.93874833555259, 71.97070572569906, 72.56990679094541, 73.43541944074568, 74.56724367509987, 75.43275632490013, 77.1970705725699, 78.39547270306258, 78.728362183755, 80.39280958721704, 81.02529960053262, 81.29161118508655, 82.58988015978696, 83.22237017310253, 84.68708388814915, 85.21970705725698, 85.48601864181093, 86.58455392809587, 87.84953395472704, 88.8814913448735, 89.61384820239681, 91.6444740346205, 93.30892143808256, 93.9081225033289, 95.07323568575234, 96.13848202396804, 97.76964047936086, 98.63515312916113, 99.33422103861518, 100.0], "rescued": [0, 45, 67, 106, 162, 213, 252, 309, 353, 382, 429, 476, 499, 520, 533, 555, 584, 601, 624, 633, 649, 693, 736, 757, 780, 814, 841, 865, 891, 912, 923, 929, 950, 963, 977, 999, 1008, 1028, 1043, 1052, 1063, 1081, 1087, 1092, 1110, 1130, 1144, 1147, 1149, 1151, 1152, 1153, 1153, 1154, 1154, 1156, 1158, 1163, 1163, 1163, 1165, 1165, 1165, 1167, 1167, 1167, 1167, 1167, 1168, 1169, 1170, 1175, 1176, 1176, 1178, 1178, 1178, 1180, 1180, 1180, 1181, 1182, 1182, 1182, 1183, 1183, 1184, 1184, 1187, 1187, 1188, 1189, 1191, 1192, 1192, 1192, 1192, 1196, 1198, 1198, 1198], "harmed": [0, 6, 14, 22, 41, 60, 79, 90, 103, 109, 131, 158, 174, 190, 205, 222, 247, 260, 276, 282, 297, 331, 371, 390, 411, 441, 465, 485, 513, 533, 544, 550, 572, 586, 601, 632, 639, 655, 668, 678, 698, 715, 722, 731, 753, 780, 813, 831, 844, 881, 895, 911, 925, 945, 952, 967, 988, 1001, 1018, 1049, 1079, 1092, 1105, 1130, 1155, 1174, 1222, 1237, 1257, 1275, 1297, 1322, 1343, 1359, 1380, 1407, 1429, 1475, 1504, 1512, 1554, 1571, 1579, 1610, 1624, 1665, 1682, 1690, 1718, 1752, 1783, 1803, 1866, 1909, 1926, 1958, 1993, 2044, 2067, 2089, 2109], "exact_score": [18.129624718318375, 18.288257862443537, 18.808548254467997, 19.074344648869303, 19.306328592042878, 19.504287151018342, 19.931529767176013, 20.16366601750135, 20.519049868300897, 20.858715903463896, 21.010287022992554, 21.07907300899299, 20.514789747918584, 20.785446911992665, 21.164958404156174, 21.21002165079685, 21.323735102050463, 21.502188363504395, 21.642620031743856, 21.931694420909263, 22.02648879809414, 22.023547756815397, 22.27586967388339, 22.274533254179918, 22.61018572846921, 23.372991176029025, 23.543026765306184, 23.825827974513835, 23.997325924279806, 24.137801928587717, 24.279780858135855, 24.510474241554874, 24.202084867011934, 24.89552090786747, 24.82581753077974, 24.962243913862714, 25.021185833318782, 25.157404101637816, 25.477542222772875, 25.816593249169284, 25.76740245304583, 25.970972800508406, 26.087677616681397, 26.141264214951505, 26.306335214200953, 25.920681345209644, 25.996018225316586, 25.77444796597176, 25.761914478722005, 25.601355268316716, 25.562536616130018, 25.932531007916687, 25.854963216198957, 25.25944389898725, 25.202457009484153, 24.949731772295785, 24.923786985661224, 24.77719447661421, 24.345287391364884, 24.685968490339942, 24.54940003026916, 24.31757857442873, 24.259022652301628, 24.103244455265816, 24.005622217464218, 23.95035470745611, 23.840721402903743, 23.70681700384906, 23.65413851180742, 24.016572018464178, 24.032249538748452, 24.46168430493389, 24.411544505287146, 24.374909571100748, 24.324894061159807, 24.05498230187712, 23.7777324417746, 23.614982989189272, 23.51375174775157, 23.321293740297516, 23.331982200124035, 23.213630989887914, 22.96105088668933, 22.93225132018405, 22.720559286550486, 22.561346295856122, 22.88881261556652, 22.829724876920153, 22.701879561084834, 22.965828760996494, 22.846259873695704, 22.705447230994903, 22.620240028618248, 22.46473576338186, 22.396347066098688, 22.272138016141554, 22.037841530054646, 22.09614615199458, 22.149001768916772, 22.424037938339534, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.323958961645214, "threshold_area100": 23.398312262173423, "threshold_area25": 20.972333484458417}, "selected_budget": 46, "selected": {"val": {"score": 28.85771696562181, "calls": 45.93437945791726, "rescued": 274, "harmed": 208}, "test": {"score": 25.92463564955129, "calls": 46.17177097203729, "rescued": 1144, "harmed": 813}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.29099422693252563, "operator": "ge"}, {"budget": 0.02, "threshold": 0.2688666582107544, "operator": "ge"}, {"budget": 0.03, "threshold": 0.25210094451904297, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2294207066297531, "operator": "ge"}, {"budget": 0.05, "threshold": 0.2079160511493683, "operator": "ge"}, {"budget": 0.06, "threshold": 0.18820054829120636, "operator": "ge"}, {"budget": 0.07, "threshold": 0.17078328132629395, "operator": "ge"}, {"budget": 0.08, "threshold": 0.14593808352947235, "operator": "ge"}, {"budget": 0.09, "threshold": 0.11705677211284637, "operator": "ge"}, {"budget": 0.1, "threshold": 0.06260053813457489, "operator": "ge"}, {"budget": 0.11, "threshold": 0.0010594874620437622, "operator": "ge"}, {"budget": 0.12, "threshold": -0.005846068263053894, "operator": "ge"}, {"budget": 0.13, "threshold": -0.01028585433959961, "operator": "ge"}, {"budget": 0.14, "threshold": -0.014254823327064514, "operator": "ge"}, {"budget": 0.15, "threshold": -0.01737402379512787, "operator": "ge"}, {"budget": 0.16, "threshold": -0.021291926503181458, "operator": "ge"}, {"budget": 0.17, "threshold": -0.02437037229537964, "operator": "ge"}, {"budget": 0.18, "threshold": -0.026894867420196533, "operator": "ge"}, {"budget": 0.19, "threshold": -0.028013229370117188, "operator": "ge"}, {"budget": 0.2, "threshold": -0.02925439178943634, "operator": "ge"}, {"budget": 0.21, "threshold": -0.03334254026412964, "operator": "ge"}, {"budget": 0.22, "threshold": -0.036628663539886475, "operator": "ge"}, {"budget": 0.23, "threshold": -0.038582026958465576, "operator": "ge"}, {"budget": 0.24, "threshold": -0.04132561385631561, "operator": "ge"}, {"budget": 0.25, "threshold": -0.04438810050487518, "operator": "ge"}, {"budget": 0.26, "threshold": -0.04609835147857666, "operator": "ge"}, {"budget": 0.27, "threshold": -0.04794429987668991, "operator": "ge"}, {"budget": 0.28, "threshold": -0.049966007471084595, "operator": "ge"}, {"budget": 0.29, "threshold": -0.05223727226257324, "operator": "ge"}, {"budget": 0.3, "threshold": -0.05492784082889557, "operator": "ge"}, {"budget": 0.31, "threshold": -0.05657121539115906, "operator": "ge"}, {"budget": 0.32, "threshold": -0.05789059400558472, "operator": "ge"}, {"budget": 0.33, "threshold": -0.061395470052957535, "operator": "ge"}, {"budget": 0.34, "threshold": -0.06427799165248871, "operator": "ge"}, {"budget": 0.35, "threshold": -0.06741350889205933, "operator": "ge"}, {"budget": 0.36, "threshold": -0.06948055326938629, "operator": "ge"}, {"budget": 0.37, "threshold": -0.07224392145872116, "operator": "ge"}, {"budget": 0.38, "threshold": -0.07595627009868622, "operator": "ge"}, {"budget": 0.39, "threshold": -0.08021591603755951, "operator": "ge"}, {"budget": 0.4, "threshold": -0.08564135432243347, "operator": "ge"}, {"budget": 0.41, "threshold": -0.08916689455509186, "operator": "ge"}, {"budget": 0.42, "threshold": -0.09659984707832336, "operator": "ge"}, {"budget": 0.43, "threshold": -0.10320758819580078, "operator": "ge"}, {"budget": 0.44, "threshold": -0.11368893831968307, "operator": "ge"}, {"budget": 0.45, "threshold": -0.12078028917312622, "operator": "ge"}, {"budget": 0.46, "threshold": -0.12336002290248871, "operator": "ge"}, {"budget": 0.47, "threshold": -0.1268751323223114, "operator": "ge"}, {"budget": 0.48, "threshold": -0.13460838794708252, "operator": "ge"}, {"budget": 0.49, "threshold": -0.1402255743741989, "operator": "ge"}, {"budget": 0.5, "threshold": -0.14679628610610962, "operator": "ge"}, {"budget": 0.51, "threshold": -0.15539589524269104, "operator": "ge"}, {"budget": 0.52, "threshold": -0.17174676060676575, "operator": "ge"}, {"budget": 0.53, "threshold": -0.17890405654907227, "operator": "ge"}, {"budget": 0.54, "threshold": -0.19128155708312988, "operator": "ge"}, {"budget": 0.55, "threshold": -0.1990281194448471, "operator": "ge"}, {"budget": 0.56, "threshold": -0.2027525156736374, "operator": "ge"}, {"budget": 0.57, "threshold": -0.2049560397863388, "operator": "ge"}, {"budget": 0.58, "threshold": -0.2065984159708023, "operator": "ge"}, {"budget": 0.59, "threshold": -0.20905962586402893, "operator": "ge"}, {"budget": 0.6, "threshold": -0.2122252732515335, "operator": "ge"}, {"budget": 0.61, "threshold": -0.21521694958209991, "operator": "ge"}, {"budget": 0.62, "threshold": -0.2176799774169922, "operator": "ge"}, {"budget": 0.63, "threshold": -0.21943767368793488, "operator": "ge"}, {"budget": 0.64, "threshold": -0.22138144075870514, "operator": "ge"}, {"budget": 0.65, "threshold": -0.22270402312278748, "operator": "ge"}, {"budget": 0.66, "threshold": -0.22543573379516602, "operator": "ge"}, {"budget": 0.67, "threshold": -0.22673365473747253, "operator": "ge"}, {"budget": 0.68, "threshold": -0.22924938797950745, "operator": "ge"}, {"budget": 0.69, "threshold": -0.23222588002681732, "operator": "ge"}, {"budget": 0.7, "threshold": -0.2335834503173828, "operator": "ge"}, {"budget": 0.71, "threshold": -0.2356836199760437, "operator": "ge"}, {"budget": 0.72, "threshold": -0.23839110136032104, "operator": "ge"}, {"budget": 0.73, "threshold": -0.2404613047838211, "operator": "ge"}, {"budget": 0.74, "threshold": -0.24173136055469513, "operator": "ge"}, {"budget": 0.75, "threshold": -0.2442944347858429, "operator": "ge"}, {"budget": 0.76, "threshold": -0.24746951460838318, "operator": "ge"}, {"budget": 0.77, "threshold": -0.2484801709651947, "operator": "ge"}, {"budget": 0.78, "threshold": -0.24995526671409607, "operator": "ge"}, {"budget": 0.79, "threshold": -0.25114524364471436, "operator": "ge"}, {"budget": 0.8, "threshold": -0.2529514729976654, "operator": "ge"}, {"budget": 0.81, "threshold": -0.2543599009513855, "operator": "ge"}, {"budget": 0.82, "threshold": -0.2563100755214691, "operator": "ge"}, {"budget": 0.83, "threshold": -0.2577177584171295, "operator": "ge"}, {"budget": 0.84, "threshold": -0.26004520058631897, "operator": "ge"}, {"budget": 0.85, "threshold": -0.2607360780239105, "operator": "ge"}, {"budget": 0.86, "threshold": -0.26227498054504395, "operator": "ge"}, {"budget": 0.87, "threshold": -0.2639511227607727, "operator": "ge"}, {"budget": 0.88, "threshold": -0.2654750645160675, "operator": "ge"}, {"budget": 0.89, "threshold": -0.2686779201030731, "operator": "ge"}, {"budget": 0.9, "threshold": -0.27026891708374023, "operator": "ge"}, {"budget": 0.91, "threshold": -0.2724815011024475, "operator": "ge"}, {"budget": 0.92, "threshold": -0.2758895754814148, "operator": "ge"}, {"budget": 0.93, "threshold": -0.27778398990631104, "operator": "ge"}, {"budget": 0.94, "threshold": -0.27966487407684326, "operator": "ge"}, {"budget": 0.95, "threshold": -0.2836954593658447, "operator": "ge"}, {"budget": 0.96, "threshold": -0.29172155261039734, "operator": "ge"}, {"budget": 0.97, "threshold": -0.29586857557296753, "operator": "ge"}, {"budget": 0.98, "threshold": -0.30264925956726074, "operator": "ge"}, {"budget": 0.99, "threshold": -0.3117719292640686, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.445282460810414, 18.745206273868416, 19.166954356405714, 20.236173816949844, 20.606650531411134, 21.07598099619376, 21.406179492519698, 21.79840247566926, 22.339935974389757, 22.60171840354767, 22.633668852156248, 22.537217045234723, 22.709462511291783, 24.00372304062949, 24.008485832904114, 23.915522183716252, 24.025757381782732, 24.18493569036116, 24.109497605613033, 24.102156334231804, 24.449833043538437, 24.575185951997547, 25.014695473365656, 25.43195445591705, 25.564264099085797, 25.560954156342735, 25.52935294870779, 25.87060469777717, 26.24259259259259, 26.539479752337776, 26.680283032185205, 27.02823764580836, 27.31382124768248, 28.04352379636603, 28.251642745121398, 26.38866448146019, 26.338839050252417, 26.790074681105075, 26.91266336003178, 26.787167634625263, 27.64536849222447, 27.89616411973243, 27.560069129836574, 27.5281100374748, 28.141410206466695, 28.071895424836597, 27.91018426569568, 28.22249436118837, 28.20581379404909, 28.1980010939287, 28.19537457383975, 28.227650488577932, 28.27689214712461, 28.207516974920555, 28.1694168987204, 27.90638454113147, 27.765785796097088, 27.536200392699627, 27.4872087563349, 27.35412831562468, 26.72039366322713, 26.242573379444885, 25.84580209221505, 25.193770207102222, 25.143291979444538, 25.105865439133996, 25.16040639753749, 25.1024878649307, 25.046515937372053, 24.64273210310394, 24.701687881918303, 24.65714273245048, 24.289145813536056, 24.330517579725118, 24.517486441449517, 24.47452710697364, 24.168446895599125, 24.09320418729021, 24.03511974590406, 23.924863447069367, 23.635954973378283, 23.301857287786177, 23.343267192727087, 23.291070577014782, 23.245038389106185, 23.3886164550065, 22.679232430785227, 22.559977142328545, 22.5066328730105, 22.577117434833678, 22.523361857300745, 22.03440719442667, 21.897223447571367, 21.843238247826868, 21.631604020976376, 21.577465714952428, 21.454828890925523, 21.121986163908694, 21.010169013534817, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 10, 23, 36, 49, 60, 71, 80, 91, 101, 110, 115, 119, 125, 132, 136, 139, 143, 148, 153, 159, 164, 169, 172, 178, 184, 190, 196, 202, 205, 210, 214, 219, 226, 230, 234, 238, 241, 247, 251, 254, 259, 261, 264, 264, 267, 267, 267, 269, 269, 270, 271, 272, 274, 274, 274, 274, 274, 274, 274, 274, 274, 274, 275, 275, 275, 275, 275, 275, 275, 275, 276, 276, 276, 277, 278, 278, 279, 281, 281, 281, 282, 282, 283, 283, 283, 284, 284, 284, 284, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286], "harmed": [0, 2, 5, 7, 10, 12, 15, 18, 21, 23, 25, 28, 33, 36, 45, 50, 53, 57, 61, 68, 77, 81, 84, 86, 94, 102, 108, 113, 118, 121, 126, 130, 135, 141, 152, 155, 162, 167, 171, 175, 181, 187, 192, 195, 197, 200, 204, 209, 214, 215, 221, 227, 231, 235, 240, 243, 248, 253, 259, 263, 268, 275, 282, 287, 296, 301, 305, 311, 317, 323, 329, 332, 337, 343, 353, 358, 363, 367, 372, 378, 382, 388, 393, 399, 405, 411, 418, 425, 431, 438, 443, 449, 456, 464, 471, 479, 487, 494, 501, 509, 518], "exact_score": [18.254072168965788, 18.445282460810414, 18.745206273868416, 19.166954356405714, 20.236173816949844, 20.606650531411134, 21.07598099619376, 21.406179492519698, 21.79840247566926, 22.339935974389757, 22.60171840354767, 22.633668852156248, 22.537217045234723, 22.709462511291783, 24.00372304062949, 24.008485832904114, 23.915522183716252, 24.025757381782732, 24.18493569036116, 24.109497605613033, 24.102156334231804, 24.449833043538437, 24.575185951997547, 25.014695473365656, 25.43195445591705, 25.564264099085797, 25.560954156342735, 25.52935294870779, 25.87060469777717, 26.24259259259259, 26.539479752337776, 26.680283032185205, 27.02823764580836, 27.31382124768248, 28.04352379636603, 28.251642745121398, 26.38866448146019, 26.338839050252417, 26.790074681105075, 26.91266336003178, 26.787167634625263, 27.64536849222447, 27.89616411973243, 27.560069129836574, 27.5281100374748, 28.141410206466695, 28.071895424836597, 27.91018426569568, 28.22249436118837, 28.20581379404909, 28.1980010939287, 28.19537457383975, 28.227650488577932, 28.27689214712461, 28.207516974920555, 28.1694168987204, 27.90638454113147, 27.765785796097088, 27.536200392699627, 27.4872087563349, 27.35412831562468, 26.72039366322713, 26.242573379444885, 25.84580209221505, 25.193770207102222, 25.143291979444538, 25.105865439133996, 25.16040639753749, 25.1024878649307, 25.046515937372053, 24.64273210310394, 24.701687881918303, 24.65714273245048, 24.289145813536056, 24.330517579725118, 24.517486441449517, 24.47452710697364, 24.168446895599125, 24.09320418729021, 24.03511974590406, 23.924863447069367, 23.635954973378283, 23.301857287786177, 23.343267192727087, 23.291070577014782, 23.245038389106185, 23.3886164550065, 22.679232430785227, 22.559977142328545, 22.5066328730105, 22.577117434833678, 22.523361857300745, 22.03440719442667, 21.897223447571367, 21.843238247826868, 21.631604020976376, 21.577465714952428, 21.454828890925523, 21.121986163908694, 21.010169013534817, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.684024718099728, "threshold_area100": 24.689518906356184, "threshold_area25": 22.645635847224106}, "test": {"score": [18.129624718318375, 18.320016897034343, 18.693172239367893, 18.853709572746542, 19.17637352127297, 19.400125927625986, 19.871971708302894, 20.16524125621332, 20.61497622780968, 21.000060167914764, 21.084964146443692, 20.65688566779228, 20.924489945772542, 21.19769923141883, 21.239594984999872, 21.273496648730745, 21.556084551966407, 21.677705712265933, 21.896490074213652, 21.987832532637785, 22.03300653594771, 22.241733773650644, 22.635407964355334, 22.636720836454007, 23.375270270796925, 23.898750201964134, 24.10446792698706, 24.28503762853341, 24.361271251081824, 24.559147519265277, 24.77517424270616, 24.888676999991883, 24.90069492808286, 24.92393238169763, 24.97013271846512, 25.074963351278644, 25.214695129912517, 25.361506267891265, 25.479438147479833, 25.797816007849235, 25.870852569413515, 25.99420687557068, 26.09778491316853, 26.209037291305105, 26.09291734435556, 26.01519012952236, 26.006746439559773, 25.952321554880648, 25.29117175549495, 25.27538429726407, 25.339417243654623, 25.301714508027484, 25.41496029294202, 25.29870870127755, 25.196976842377666, 25.43832585844733, 25.293691442563276, 25.15901668693379, 24.55608031804399, 24.802585592313612, 24.724873717863126, 24.71079522087125, 24.652140183870408, 24.6211495509937, 24.49748009130426, 24.450180659564843, 24.332165357798807, 24.788201024295375, 25.203751655907514, 25.08882245822322, 24.96583596817957, 24.901083031802923, 24.682222309592575, 24.487622682391958, 24.39256421701331, 24.1171165891733, 23.974782845293692, 23.839105884075064, 23.70455288297549, 23.70889754151017, 23.616623890261263, 23.63494649480414, 23.396365798538337, 23.301949882755853, 22.95862160810852, 22.939980301546754, 22.88495299054763, 23.063006748998298, 22.913161517038073, 23.288304624473074, 23.223848863712494, 23.07997804843094, 22.99969664648287, 22.903206364569336, 22.84135884600881, 22.639304847190292, 22.506770488304014, 22.51114369681152, 22.799949449496832, 22.65505795027275, 22.361115727328066], "calls": [0.0, 0.9986684420772304, 1.5645805592543276, 2.330226364846871, 3.462050599201065, 4.427430093209055, 5.692410119840213, 6.890812250332889, 8.355525965379494, 9.653794940079893, 10.885486018641812, 12.549933422103862, 13.448735019973368, 14.181091877496671, 15.11318242343542, 15.84553928095872, 17.14380825565912, 18.07589880159787, 19.174434087882823, 19.607190412782955, 19.973368841544605, 21.70439414114514, 23.535286284953397, 24.134487350199734, 25.499334221038616, 27.396804260985352, 28.56191744340879, 29.460719041278296, 30.525965379494007, 31.324900133155793, 32.35685752330226, 32.889480692410125, 33.45539280958722, 34.58721704394141, 35.21970705725699, 36.21837549933422, 36.651131824234355, 37.31691078561917, 38.38215712383489, 39.4474034620506, 40.31291611185087, 41.24500665778961, 41.91078561917443, 42.543275632490015, 43.9081225033289, 44.84021304926764, 45.63914780292942, 46.07190412782956, 47.70306258322237, 48.568575233022635, 49.80026631158455, 50.665778961384824, 51.93075898801598, 52.663115845539274, 53.695073235685754, 55.1930758988016, 56.25832223701731, 57.090545938748335, 57.78961384820239, 58.95472703062583, 60.38615179760319, 61.58455392809587, 62.4167776298269, 63.182423435419445, 64.0479360852197, 64.78029294274302, 65.77896138482025, 66.24500665778962, 67.57656458055925, 68.97470039946738, 69.60719041278296, 70.57256990679095, 71.43808255659121, 72.66977363515313, 73.5352862849534, 74.40079893475367, 76.2982689747004, 76.8641810918775, 77.3635153129161, 77.99600532623168, 79.19440745672436, 79.82689747003995, 81.0918774966711, 82.09054593874833, 83.288948069241, 83.65512649800266, 84.45406125166444, 85.41944074567243, 86.18508655126497, 87.71637816245007, 88.74833555259653, 89.81358189081226, 91.6444740346205, 92.7762982689747, 93.6750998668442, 94.87350199733689, 96.93741677762982, 97.60319573901465, 98.66844207723035, 99.33422103861518, 100.0], "rescued": [0, 54, 81, 119, 169, 207, 253, 296, 352, 395, 430, 476, 499, 519, 535, 551, 585, 605, 630, 641, 649, 689, 730, 742, 775, 817, 843, 859, 885, 902, 925, 936, 948, 971, 983, 997, 1006, 1020, 1035, 1053, 1063, 1078, 1088, 1096, 1107, 1112, 1114, 1114, 1118, 1122, 1129, 1131, 1137, 1139, 1144, 1146, 1146, 1146, 1147, 1151, 1152, 1154, 1155, 1156, 1156, 1156, 1157, 1158, 1160, 1163, 1163, 1163, 1164, 1166, 1167, 1167, 1171, 1171, 1172, 1174, 1176, 1177, 1178, 1179, 1179, 1180, 1180, 1181, 1181, 1185, 1186, 1186, 1188, 1189, 1190, 1191, 1196, 1197, 1198, 1198, 1198], "harmed": [0, 3, 11, 28, 44, 57, 77, 87, 101, 115, 131, 161, 173, 190, 206, 219, 245, 260, 282, 291, 296, 324, 359, 378, 406, 442, 460, 479, 505, 525, 546, 554, 566, 593, 606, 627, 636, 650, 668, 687, 695, 714, 724, 734, 755, 773, 786, 795, 836, 850, 871, 892, 907, 923, 946, 979, 992, 1007, 1020, 1041, 1073, 1099, 1115, 1134, 1154, 1172, 1197, 1206, 1233, 1264, 1279, 1298, 1324, 1359, 1380, 1404, 1442, 1455, 1472, 1488, 1525, 1541, 1573, 1599, 1630, 1641, 1664, 1690, 1717, 1754, 1778, 1811, 1862, 1893, 1915, 1950, 2015, 2033, 2062, 2084, 2109], "exact_score": [18.129624718318375, 18.320016897034343, 18.759329364251307, 19.03395804809371, 19.307935991161532, 19.482813647450914, 19.94108442066776, 20.173944568245574, 20.509077076523646, 20.83232174259205, 21.059565265380318, 21.087412022223628, 20.514789747918584, 20.785446911992665, 21.164062678789186, 21.21002165079685, 21.2504909770684, 21.472856519760793, 21.65641819943599, 21.844474585908173, 22.03300653594771, 22.118552307231553, 22.29743867461373, 22.32143115986882, 22.661349865309646, 23.342047794523506, 23.53098896530139, 23.848511491824013, 24.01541807866089, 24.21519875787065, 24.404982282408234, 24.41516769202191, 24.676647173161857, 24.91951343838325, 24.87062779804488, 24.9992296827513, 25.032358017876135, 25.288346612681913, 25.484095635054306, 25.602174267782413, 25.78515065496075, 25.950878468786225, 26.178512477484677, 26.270449388663902, 26.107796360493197, 26.011565971442156, 25.977226015527265, 25.361626498363055, 25.335183686965774, 25.336431512125042, 25.320595318110005, 25.33076381535534, 25.410821234674646, 25.301089307379936, 25.15591309833735, 24.984598380106437, 25.351649332062298, 25.165666983639333, 24.521614171477932, 24.800501738016703, 24.744256682974843, 24.72876512311547, 24.682356705302915, 24.639274011362197, 24.50439763419303, 24.43646633814653, 24.795101570177845, 25.208931745157475, 25.18599383902528, 25.08882245822322, 24.942772349785898, 24.802883299725206, 24.620097764857444, 24.468651079582806, 24.285605027034673, 24.09786895697297, 23.98025066574812, 23.828362653806025, 23.70889754151017, 23.60710960672891, 23.58082466703156, 23.400171600323773, 23.30460613690625, 22.97083827881907, 22.921046671568018, 23.144979281513056, 22.96849146791386, 23.246560079250873, 23.275169853669087, 23.216111187000962, 23.07466311550224, 23.008345727608408, 22.985453365791585, 22.876377025305906, 22.735728159643486, 22.61711920022372, 22.532188021281698, 22.506770488304014, 22.47740204908576, 22.738604081471703, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.4836273668419, "threshold_area100": 23.559627689099134, "threshold_area25": 20.96115614377788}, "selected_budget": 53, "selected": {"val": {"score": 28.27689214712461, "calls": 52.92439372325249, "rescued": 274, "harmed": 235}, "test": {"score": 25.29870870127755, "calls": 52.663115845539274, "rescued": 1139, "harmed": 923}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.209426760673523, "operator": "ge"}, {"budget": 0.02, "threshold": 1.0645873546600342, "operator": "ge"}, {"budget": 0.03, "threshold": 0.9852919578552246, "operator": "ge"}, {"budget": 0.04, "threshold": 0.9355100393295288, "operator": "ge"}, {"budget": 0.05, "threshold": 0.8878711462020874, "operator": "ge"}, {"budget": 0.06, "threshold": 0.8519248366355896, "operator": "ge"}, {"budget": 0.07, "threshold": 0.8052138090133667, "operator": "ge"}, {"budget": 0.08, "threshold": 0.6786444783210754, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5540452003479004, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3276498317718506, "operator": "ge"}, {"budget": 0.11, "threshold": 0.007795780897140503, "operator": "ge"}, {"budget": 0.12, "threshold": -0.03267556428909302, "operator": "ge"}, {"budget": 0.13, "threshold": -0.05945086479187012, "operator": "ge"}, {"budget": 0.14, "threshold": -0.07106417417526245, "operator": "ge"}, {"budget": 0.15, "threshold": -0.09639465063810349, "operator": "ge"}, {"budget": 0.16, "threshold": -0.11632376909255981, "operator": "ge"}, {"budget": 0.17, "threshold": -0.12518754601478577, "operator": "ge"}, {"budget": 0.18, "threshold": -0.13312214612960815, "operator": "ge"}, {"budget": 0.19, "threshold": -0.15335381031036377, "operator": "ge"}, {"budget": 0.2, "threshold": -0.16525772213935852, "operator": "ge"}, {"budget": 0.21, "threshold": -0.17404338717460632, "operator": "ge"}, {"budget": 0.22, "threshold": -0.18516969680786133, "operator": "ge"}, {"budget": 0.23, "threshold": -0.1999192237854004, "operator": "ge"}, {"budget": 0.24, "threshold": -0.2098887860774994, "operator": "ge"}, {"budget": 0.25, "threshold": -0.2233046591281891, "operator": "ge"}, {"budget": 0.26, "threshold": -0.23965054750442505, "operator": "ge"}, {"budget": 0.27, "threshold": -0.25295689702033997, "operator": "ge"}, {"budget": 0.28, "threshold": -0.26246732473373413, "operator": "ge"}, {"budget": 0.29, "threshold": -0.26964303851127625, "operator": "ge"}, {"budget": 0.3, "threshold": -0.28175026178359985, "operator": "ge"}, {"budget": 0.31, "threshold": -0.3001018762588501, "operator": "ge"}, {"budget": 0.32, "threshold": -0.3167969882488251, "operator": "ge"}, {"budget": 0.33, "threshold": -0.33611804246902466, "operator": "ge"}, {"budget": 0.34, "threshold": -0.3640175461769104, "operator": "ge"}, {"budget": 0.35, "threshold": -0.40984290838241577, "operator": "ge"}, {"budget": 0.36, "threshold": -0.44987136125564575, "operator": "ge"}, {"budget": 0.37, "threshold": -0.4990055561065674, "operator": "ge"}, {"budget": 0.38, "threshold": -0.5503662824630737, "operator": "ge"}, {"budget": 0.39, "threshold": -0.6229844093322754, "operator": "ge"}, {"budget": 0.4, "threshold": -0.6814095973968506, "operator": "ge"}, {"budget": 0.41, "threshold": -0.744638979434967, "operator": "ge"}, {"budget": 0.42, "threshold": -0.8415294885635376, "operator": "ge"}, {"budget": 0.43, "threshold": -0.8903891444206238, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9936566352844238, "operator": "ge"}, {"budget": 0.45, "threshold": -1.1899847984313965, "operator": "ge"}, {"budget": 0.46, "threshold": -1.6876076459884644, "operator": "ge"}, {"budget": 0.47, "threshold": -1.9432010650634766, "operator": "ge"}, {"budget": 0.48, "threshold": -2.2234461307525635, "operator": "ge"}, {"budget": 0.49, "threshold": -2.640923500061035, "operator": "ge"}, {"budget": 0.5, "threshold": -2.735163450241089, "operator": "ge"}, {"budget": 0.51, "threshold": -2.882460594177246, "operator": "ge"}, {"budget": 0.52, "threshold": -4.578181743621826, "operator": "ge"}, {"budget": 0.53, "threshold": -4.68320369720459, "operator": "ge"}, {"budget": 0.54, "threshold": -4.727412223815918, "operator": "ge"}, {"budget": 0.55, "threshold": -4.783528804779053, "operator": "ge"}, {"budget": 0.56, "threshold": -4.814594268798828, "operator": "ge"}, {"budget": 0.57, "threshold": -4.84147834777832, "operator": "ge"}, {"budget": 0.58, "threshold": -4.855273246765137, "operator": "ge"}, {"budget": 0.59, "threshold": -4.884262561798096, "operator": "ge"}, {"budget": 0.6, "threshold": -4.908026218414307, "operator": "ge"}, {"budget": 0.61, "threshold": -4.936931610107422, "operator": "ge"}, {"budget": 0.62, "threshold": -4.9578537940979, "operator": "ge"}, {"budget": 0.63, "threshold": -4.969408988952637, "operator": "ge"}, {"budget": 0.64, "threshold": -4.980615615844727, "operator": "ge"}, {"budget": 0.65, "threshold": -4.998864650726318, "operator": "ge"}, {"budget": 0.66, "threshold": -5.013849258422852, "operator": "ge"}, {"budget": 0.67, "threshold": -5.027232646942139, "operator": "ge"}, {"budget": 0.68, "threshold": -5.034890651702881, "operator": "ge"}, {"budget": 0.69, "threshold": -5.048084259033203, "operator": "ge"}, {"budget": 0.7, "threshold": -5.059701442718506, "operator": "ge"}, {"budget": 0.71, "threshold": -5.065311431884766, "operator": "ge"}, {"budget": 0.72, "threshold": -5.074363708496094, "operator": "ge"}, {"budget": 0.73, "threshold": -5.081503868103027, "operator": "ge"}, {"budget": 0.74, "threshold": -5.098762512207031, "operator": "ge"}, {"budget": 0.75, "threshold": -5.112545013427734, "operator": "ge"}, {"budget": 0.76, "threshold": -5.128900051116943, "operator": "ge"}, {"budget": 0.77, "threshold": -5.143597602844238, "operator": "ge"}, {"budget": 0.78, "threshold": -5.161372661590576, "operator": "ge"}, {"budget": 0.79, "threshold": -5.18238639831543, "operator": "ge"}, {"budget": 0.8, "threshold": -5.232397556304932, "operator": "ge"}, {"budget": 0.81, "threshold": -5.352535724639893, "operator": "ge"}, {"budget": 0.82, "threshold": -13.620952606201172, "operator": "ge"}, {"budget": 0.83, "threshold": -13.63210678100586, "operator": "ge"}, {"budget": 0.84, "threshold": -13.652885437011719, "operator": "ge"}, {"budget": 0.85, "threshold": -13.70086669921875, "operator": "ge"}, {"budget": 0.86, "threshold": -13.751321792602539, "operator": "ge"}, {"budget": 0.87, "threshold": -13.785215377807617, "operator": "ge"}, {"budget": 0.88, "threshold": -13.807631492614746, "operator": "ge"}, {"budget": 0.89, "threshold": -13.84128475189209, "operator": "ge"}, {"budget": 0.9, "threshold": -13.876569747924805, "operator": "ge"}, {"budget": 0.91, "threshold": -13.893288612365723, "operator": "ge"}, {"budget": 0.92, "threshold": -13.926886558532715, "operator": "ge"}, {"budget": 0.93, "threshold": -13.937383651733398, "operator": "ge"}, {"budget": 0.94, "threshold": -13.962080955505371, "operator": "ge"}, {"budget": 0.95, "threshold": -14.010053634643555, "operator": "ge"}, {"budget": 0.96, "threshold": -14.068700790405273, "operator": "ge"}, {"budget": 0.97, "threshold": -14.108865737915039, "operator": "ge"}, {"budget": 0.98, "threshold": -14.13460922241211, "operator": "ge"}, {"budget": 0.99, "threshold": -14.211509704589844, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.47397025171625, 18.893401783956506, 19.03210254074051, 20.05787802012191, 20.42992630180017, 20.911590647584326, 21.378905393945743, 21.85853256615402, 22.339935974389757, 22.678687796941286, 22.633668852156248, 22.69015987489445, 22.735013866639374, 24.159964933828178, 24.283978657437448, 24.16051554135005, 23.80659453169326, 23.998429274245225, 24.186059992832902, 24.78819735805701, 24.467936222322187, 24.75344130643706, 25.415749485613066, 25.567048312949954, 25.93912780133253, 26.269774457188955, 26.551727371537538, 26.87417648260243, 27.344630196256965, 27.577094474153295, 28.06376959507247, 28.398275872755303, 28.462033462033464, 29.24559326493028, 27.349224257053407, 27.234917230380024, 27.428288917291415, 27.20948985654868, 27.52459958342311, 27.637550621421585, 27.577613388309487, 27.55568356374808, 28.388166396773528, 28.664674809015388, 28.87603198393683, 28.73982322742959, 28.881026424906885, 28.27112939733938, 28.52850803860986, 27.952676522947744, 27.837735083560577, 27.73109243697479, 27.605730816916935, 27.51823246152394, 26.857076857076855, 26.927628129376867, 27.0948797196379, 26.92940620942037, 26.827459896425417, 26.244002393761377, 26.17659440594261, 26.02219647651624, 25.948158108165792, 25.889101264050048, 24.999621470209707, 24.475591590909325, 24.333188934958848, 24.191562537588567, 24.137390905455916, 23.99231797965975, 23.933824369946425, 23.514211576399447, 23.452151242401307, 23.152519752192298, 23.013814016172507, 22.884531957967802, 22.774979317952017, 22.739328694066096, 22.471786827109163, 22.43307398886389, 22.3799135582747, 22.286913865861234, 22.18301186207182, 22.15296993432365, 22.123453315486877, 22.088758913009524, 22.072049544623653, 21.96545071183119, 21.849861419302314, 21.754607408440112, 21.632813462998165, 21.591124522977402, 21.285082660515187, 21.255416106379798, 21.15548575435261, 20.93302984999317, 20.90935686418692, 20.889161741314176, 20.68874825884421, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 10, 21, 34, 47, 56, 68, 78, 90, 101, 110, 115, 120, 125, 131, 136, 141, 146, 152, 157, 163, 168, 173, 179, 185, 191, 195, 200, 205, 210, 216, 220, 226, 231, 236, 240, 242, 247, 252, 257, 261, 261, 261, 267, 271, 272, 274, 276, 277, 281, 281, 283, 283, 283, 283, 283, 284, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 2, 2, 7, 10, 10, 13, 15, 20, 23, 25, 28, 33, 42, 44, 49, 54, 63, 69, 75, 80, 86, 89, 97, 102, 109, 114, 119, 123, 129, 136, 144, 150, 156, 160, 165, 169, 175, 183, 186, 191, 194, 195, 200, 205, 205, 212, 215, 224, 230, 238, 243, 250, 258, 264, 271, 278, 283, 289, 296, 302, 311, 317, 323, 328, 335, 340, 346, 352, 357, 364, 370, 376, 383, 387, 394, 401, 407, 412, 418, 423, 429, 432, 436, 440, 444, 449, 452, 457, 464, 468, 475, 481, 486, 491, 497, 502, 506, 510, 514, 518], "exact_score": [18.254072168965788, 18.47397025171625, 18.893401783956506, 19.03210254074051, 20.05787802012191, 20.42992630180017, 20.911590647584326, 21.378905393945743, 21.85853256615402, 22.339935974389757, 22.678687796941286, 22.633668852156248, 22.69015987489445, 22.735013866639374, 24.159964933828178, 24.283978657437448, 24.16051554135005, 23.80659453169326, 23.998429274245225, 24.186059992832902, 24.78819735805701, 24.467936222322187, 24.75344130643706, 25.415749485613066, 25.567048312949954, 25.93912780133253, 26.269774457188955, 26.551727371537538, 26.87417648260243, 27.344630196256965, 27.577094474153295, 28.06376959507247, 28.398275872755303, 28.462033462033464, 29.24559326493028, 27.349224257053407, 27.234917230380024, 27.428288917291415, 27.20948985654868, 27.52459958342311, 27.637550621421585, 27.577613388309487, 27.55568356374808, 28.388166396773528, 28.664674809015388, 28.87603198393683, 28.73982322742959, 28.881026424906885, 28.27112939733938, 28.52850803860986, 27.952676522947744, 27.837735083560577, 27.73109243697479, 27.605730816916935, 27.51823246152394, 26.857076857076855, 26.927628129376867, 27.0948797196379, 26.92940620942037, 26.827459896425417, 26.244002393761377, 26.17659440594261, 26.02219647651624, 25.948158108165792, 25.889101264050048, 24.999621470209707, 24.475591590909325, 24.333188934958848, 24.191562537588567, 24.137390905455916, 23.99231797965975, 23.933824369946425, 23.514211576399447, 23.452151242401307, 23.152519752192298, 23.013814016172507, 22.884531957967802, 22.774979317952017, 22.739328694066096, 22.471786827109163, 22.43307398886389, 22.3799135582747, 22.286913865861234, 22.18301186207182, 22.15296993432365, 22.123453315486877, 22.088758913009524, 22.072049544623653, 21.96545071183119, 21.849861419302314, 21.754607408440112, 21.632813462998165, 21.591124522977402, 21.285082660515187, 21.255416106379798, 21.15548575435261, 20.93302984999317, 20.90935686418692, 20.889161741314176, 20.68874825884421, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.484698381171018, "threshold_area100": 24.49013740377092, "threshold_area25": 22.855478019877015}, "test": {"score": [18.129624718318375, 18.63093753710135, 18.861141736232717, 19.10620631108389, 19.526529506984332, 19.71023712084364, 19.929326369233525, 20.26512063781198, 20.75764879348379, 20.940281240651608, 21.077093431232854, 20.669908137278682, 20.953421987419198, 21.059040444169618, 21.310931832137253, 21.453722013214595, 21.66878806001497, 21.820756839125423, 22.014843129678955, 22.137625422515722, 22.24464477163128, 22.481128853014397, 22.93384038587153, 23.537367502087825, 23.847371072445185, 24.023953954323083, 24.38750297309602, 24.704900008298065, 24.728328142878837, 24.70985035071574, 24.9470528315787, 24.9356126517773, 25.11046676933897, 25.245348163978104, 25.681288924328705, 25.96886365848261, 25.96572064220652, 26.1107506181498, 26.157578644585968, 26.22429034681508, 26.42390845466674, 26.394572578965935, 26.358897330764336, 26.412235226207496, 25.966697993782205, 26.043738121137505, 25.954689797526676, 26.046957014351907, 26.527923669467786, 25.650176582648633, 25.390067180680653, 24.505361645786156, 24.16034408577572, 24.046549312158955, 24.102182692070844, 23.907686763538745, 23.913412922581674, 23.779031046828003, 23.68749084896576, 23.56939704712989, 23.370919378956902, 23.277985885319705, 23.620387073310418, 23.63341283749214, 23.571811911754647, 23.551502654191694, 23.929376798517037, 23.812558265865224, 23.739048132738176, 23.619771924787493, 23.502409196755572, 23.449251522250254, 23.404014524018127, 23.323198227465312, 23.116388644780617, 23.119360860095135, 23.074954356098367, 23.27242123775437, 23.050828429206895, 22.903152703765883, 22.995088962990895, 22.846494666090877, 22.74873617799316, 22.737367986560983, 22.65657724197109, 22.605347541645443, 22.557065986711493, 22.49197647171624, 22.471389698303298, 22.36718181111991, 22.682273186238977, 22.638773041269584, 22.455506715593295, 22.439654391460962, 22.422747125811107, 22.319224460488684, 22.27231110875776, 22.215502665681637, 22.179674808191812, 22.109465946131028, 22.361115727328066], "calls": [0.0, 1.2982689747003995, 2.1637816245006656, 3.6284953395472703, 4.360852197070573, 5.159786950732357, 6.125166444740346, 7.223701731025299, 9.087882822902795, 10.053262316910786, 11.051930758988016, 12.48335552596538, 13.415446071904128, 14.314247669773636, 14.81358189081225, 15.945406125166445, 17.44340878828229, 17.976031957390145, 18.575233022636482, 20.00665778961385, 21.071904127829562, 21.870838881491345, 22.969374167776298, 23.901464713715047, 25.0, 26.564580559254324, 27.862849533954726, 28.99467376830892, 29.59387483355526, 30.25965379494008, 31.158455392809586, 31.990679094540614, 32.98934753661784, 33.85486018641811, 34.92010652463382, 36.21837549933422, 37.11717709720373, 37.982689747003995, 38.81491344873502, 39.61384820239681, 40.279627163781626, 40.94540612516644, 42.0439414114514, 42.543275632490015, 42.97603195739015, 44.14114513981358, 45.43941411451398, 46.03861517976032, 46.6711051930759, 48.402130492676434, 48.968042609853526, 50.36617842876166, 51.53129161118508, 52.762982689747005, 53.56191744340879, 55.25965379494008, 56.524633821571236, 57.52330226364847, 58.22237017310253, 59.45406125166445, 60.65246338215713, 62.28362183754993, 63.84820239680427, 64.61384820239681, 65.37949400798935, 66.6444740346205, 67.7762982689747, 68.37549933422103, 68.97470039946738, 70.07323568575234, 71.23834886817576, 71.90412782956058, 72.60319573901465, 73.33555259653795, 74.90013315579228, 75.93209054593875, 76.96404793608522, 78.16245006657789, 79.02796271637817, 79.79360852197071, 81.19174434087883, 81.92410119840213, 83.02263648468708, 83.45539280958721, 84.55392809587217, 86.21837549933421, 87.94940079893475, 88.81491344873503, 89.51398135818907, 90.5126498002663, 92.0439414114514, 92.80958721704394, 94.2410119840213, 94.67376830892144, 95.3728362183755, 96.271637816245, 97.27030625832224, 97.7363515312916, 98.33555259653795, 99.56724367509987, 100.0], "rescued": [0, 68, 108, 163, 194, 227, 262, 306, 371, 399, 433, 475, 499, 522, 534, 560, 594, 605, 622, 653, 679, 699, 727, 750, 776, 812, 842, 872, 885, 897, 917, 937, 956, 976, 1001, 1029, 1041, 1055, 1071, 1087, 1098, 1104, 1115, 1122, 1126, 1138, 1147, 1153, 1157, 1164, 1165, 1167, 1171, 1172, 1174, 1175, 1177, 1177, 1177, 1178, 1178, 1179, 1182, 1183, 1184, 1186, 1188, 1188, 1188, 1188, 1189, 1189, 1189, 1189, 1190, 1191, 1191, 1193, 1193, 1193, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1197, 1198], "harmed": [0, 4, 13, 36, 42, 53, 65, 78, 99, 117, 134, 157, 173, 195, 203, 226, 258, 266, 276, 303, 329, 344, 367, 389, 419, 447, 471, 499, 519, 535, 549, 570, 593, 611, 630, 655, 670, 687, 700, 718, 727, 740, 757, 764, 771, 787, 815, 825, 833, 874, 888, 934, 971, 1002, 1018, 1056, 1091, 1119, 1137, 1170, 1199, 1242, 1277, 1294, 1313, 1348, 1381, 1395, 1411, 1442, 1471, 1490, 1509, 1529, 1574, 1596, 1616, 1648, 1668, 1690, 1726, 1744, 1763, 1769, 1787, 1819, 1846, 1865, 1877, 1902, 1929, 1946, 1976, 1987, 1999, 2022, 2044, 2051, 2067, 2100, 2109], "exact_score": [18.129624718318375, 18.31106396081482, 18.840822855930327, 19.04499918854957, 19.202040563734045, 19.6820921583209, 19.88386122753326, 20.2171566720824, 20.449403397069762, 20.636799993476785, 20.951651635217484, 21.066845516391787, 20.514789747918584, 20.818369609940614, 20.991574393895267, 21.256072254881182, 21.42110538175661, 21.630467422089552, 21.820756839125423, 21.994628285953556, 22.13365004198437, 22.277807186746145, 22.493904747298068, 22.93384038587153, 23.53770795955523, 23.847371072445185, 24.028536328349567, 24.177321744096066, 24.37532603721068, 24.704900008298065, 24.635971164985964, 24.92189890497859, 24.9356126517773, 25.11046676933897, 25.258079924330996, 25.699470581254147, 26.012048356112476, 25.946569049865897, 26.1107506181498, 26.139257718795605, 26.344140926362066, 26.39578677255762, 26.358897330764336, 25.966697993782205, 25.98740312438826, 26.081727116183423, 26.050276626621194, 26.4592702414673, 25.827693607717727, 25.390067180680653, 24.735498551867778, 24.227294016505862, 24.123958399004895, 24.028251916053122, 24.06619136831004, 23.93107794369759, 23.91980830688139, 23.838192160162595, 23.70592837193552, 23.60873407879188, 23.4495545079512, 23.352364804913776, 23.319952308424526, 23.09472515096488, 23.68983357906493, 23.546780164780415, 23.514844563428657, 23.51530799651314, 23.91607963439883, 23.739048132738176, 23.625360997844805, 23.52126145187702, 23.44520228350714, 23.36177069128523, 23.17489290075557, 23.109605897840392, 23.117005359358505, 23.015890782215205, 23.281989005147576, 23.052594414909475, 22.794133244098198, 23.053097770700678, 22.8430429838889, 22.74873617799316, 22.719055263442954, 22.645475611573353, 22.61052966927467, 22.589428125694475, 22.557065986711493, 22.486559432995755, 22.453485317074584, 22.3530102095552, 22.6855290034316, 22.634230473597885, 22.508682187663755, 22.433460205397722, 22.368396213301956, 22.279995665020138, 22.207367207444197, 22.140394423952518, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.183101957143727, "threshold_area100": 23.27481226913529, "threshold_area25": 21.112497334648623}, "selected_budget": 34, "selected": {"val": {"score": 29.24559326493028, "calls": 33.951497860199716, "rescued": 236, "harmed": 160}, "test": {"score": 25.681288924328705, "calls": 34.92010652463382, "rescued": 1001, "harmed": 630}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.8441710472106934, "operator": "ge"}, {"budget": 0.02, "threshold": 0.7710989713668823, "operator": "ge"}, {"budget": 0.03, "threshold": 0.6922498941421509, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6497902274131775, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5781593918800354, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5140243768692017, "operator": "ge"}, {"budget": 0.07, "threshold": 0.45106300711631775, "operator": "ge"}, {"budget": 0.08, "threshold": 0.39470428228378296, "operator": "ge"}, {"budget": 0.09, "threshold": 0.33793243765830994, "operator": "ge"}, {"budget": 0.1, "threshold": 0.208333820104599, "operator": "ge"}, {"budget": 0.11, "threshold": 0.0038219094276428223, "operator": "ge"}, {"budget": 0.12, "threshold": -0.025027155876159668, "operator": "ge"}, {"budget": 0.13, "threshold": -0.04280509054660797, "operator": "ge"}, {"budget": 0.14, "threshold": -0.05627387762069702, "operator": "ge"}, {"budget": 0.15, "threshold": -0.07461848855018616, "operator": "ge"}, {"budget": 0.16, "threshold": -0.09057345986366272, "operator": "ge"}, {"budget": 0.17, "threshold": -0.0994509607553482, "operator": "ge"}, {"budget": 0.18, "threshold": -0.10721037536859512, "operator": "ge"}, {"budget": 0.19, "threshold": -0.11130152642726898, "operator": "ge"}, {"budget": 0.2, "threshold": -0.1240692287683487, "operator": "ge"}, {"budget": 0.21, "threshold": -0.13885658979415894, "operator": "ge"}, {"budget": 0.22, "threshold": -0.14987768232822418, "operator": "ge"}, {"budget": 0.23, "threshold": -0.16273580491542816, "operator": "ge"}, {"budget": 0.24, "threshold": -0.17064009606838226, "operator": "ge"}, {"budget": 0.25, "threshold": -0.1808520257472992, "operator": "ge"}, {"budget": 0.26, "threshold": -0.19020192325115204, "operator": "ge"}, {"budget": 0.27, "threshold": -0.19612735509872437, "operator": "ge"}, {"budget": 0.28, "threshold": -0.20698288083076477, "operator": "ge"}, {"budget": 0.29, "threshold": -0.2144189029932022, "operator": "ge"}, {"budget": 0.3, "threshold": -0.2245190143585205, "operator": "ge"}, {"budget": 0.31, "threshold": -0.22884532809257507, "operator": "ge"}, {"budget": 0.32, "threshold": -0.23774999380111694, "operator": "ge"}, {"budget": 0.33, "threshold": -0.2458992302417755, "operator": "ge"}, {"budget": 0.34, "threshold": -0.25598055124282837, "operator": "ge"}, {"budget": 0.35, "threshold": -0.2754178047180176, "operator": "ge"}, {"budget": 0.36, "threshold": -0.28322795033454895, "operator": "ge"}, {"budget": 0.37, "threshold": -0.2962822914123535, "operator": "ge"}, {"budget": 0.38, "threshold": -0.3129757344722748, "operator": "ge"}, {"budget": 0.39, "threshold": -0.3289611041545868, "operator": "ge"}, {"budget": 0.4, "threshold": -0.35182520747184753, "operator": "ge"}, {"budget": 0.41, "threshold": -0.36766505241394043, "operator": "ge"}, {"budget": 0.42, "threshold": -0.404396116733551, "operator": "ge"}, {"budget": 0.43, "threshold": -0.44894054532051086, "operator": "ge"}, {"budget": 0.44, "threshold": -0.5364125370979309, "operator": "ge"}, {"budget": 0.45, "threshold": -0.8055166602134705, "operator": "ge"}, {"budget": 0.46, "threshold": -1.426302194595337, "operator": "ge"}, {"budget": 0.47, "threshold": -1.7054346799850464, "operator": "ge"}, {"budget": 0.48, "threshold": -1.9455355405807495, "operator": "ge"}, {"budget": 0.49, "threshold": -2.2043628692626953, "operator": "ge"}, {"budget": 0.5, "threshold": -2.2667014598846436, "operator": "ge"}, {"budget": 0.51, "threshold": -2.5512053966522217, "operator": "ge"}, {"budget": 0.52, "threshold": -3.881025552749634, "operator": "ge"}, {"budget": 0.53, "threshold": -4.061973571777344, "operator": "ge"}, {"budget": 0.54, "threshold": -4.137342929840088, "operator": "ge"}, {"budget": 0.55, "threshold": -4.208991050720215, "operator": "ge"}, {"budget": 0.56, "threshold": -4.239169120788574, "operator": "ge"}, {"budget": 0.57, "threshold": -4.293961048126221, "operator": "ge"}, {"budget": 0.58, "threshold": -4.354779243469238, "operator": "ge"}, {"budget": 0.59, "threshold": -4.3841047286987305, "operator": "ge"}, {"budget": 0.6, "threshold": -4.396336555480957, "operator": "ge"}, {"budget": 0.61, "threshold": -4.425755023956299, "operator": "ge"}, {"budget": 0.62, "threshold": -4.442431926727295, "operator": "ge"}, {"budget": 0.63, "threshold": -4.461883544921875, "operator": "ge"}, {"budget": 0.64, "threshold": -4.487256050109863, "operator": "ge"}, {"budget": 0.65, "threshold": -4.504961967468262, "operator": "ge"}, {"budget": 0.66, "threshold": -4.528165817260742, "operator": "ge"}, {"budget": 0.67, "threshold": -4.551025390625, "operator": "ge"}, {"budget": 0.68, "threshold": -4.574006080627441, "operator": "ge"}, {"budget": 0.69, "threshold": -4.583413600921631, "operator": "ge"}, {"budget": 0.7, "threshold": -4.611461162567139, "operator": "ge"}, {"budget": 0.71, "threshold": -4.6232404708862305, "operator": "ge"}, {"budget": 0.72, "threshold": -4.633475303649902, "operator": "ge"}, {"budget": 0.73, "threshold": -4.656976699829102, "operator": "ge"}, {"budget": 0.74, "threshold": -4.663482666015625, "operator": "ge"}, {"budget": 0.75, "threshold": -4.683252334594727, "operator": "ge"}, {"budget": 0.76, "threshold": -4.69879150390625, "operator": "ge"}, {"budget": 0.77, "threshold": -4.713472843170166, "operator": "ge"}, {"budget": 0.78, "threshold": -4.757115364074707, "operator": "ge"}, {"budget": 0.79, "threshold": -4.76962947845459, "operator": "ge"}, {"budget": 0.8, "threshold": -4.81850528717041, "operator": "ge"}, {"budget": 0.81, "threshold": -4.8970232009887695, "operator": "ge"}, {"budget": 0.82, "threshold": -11.807112693786621, "operator": "ge"}, {"budget": 0.83, "threshold": -11.881204605102539, "operator": "ge"}, {"budget": 0.84, "threshold": -11.976737976074219, "operator": "ge"}, {"budget": 0.85, "threshold": -12.039937973022461, "operator": "ge"}, {"budget": 0.86, "threshold": -12.089849472045898, "operator": "ge"}, {"budget": 0.87, "threshold": -12.189900398254395, "operator": "ge"}, {"budget": 0.88, "threshold": -12.267346382141113, "operator": "ge"}, {"budget": 0.89, "threshold": -12.292236328125, "operator": "ge"}, {"budget": 0.9, "threshold": -12.3140230178833, "operator": "ge"}, {"budget": 0.91, "threshold": -12.331256866455078, "operator": "ge"}, {"budget": 0.92, "threshold": -12.349028587341309, "operator": "ge"}, {"budget": 0.93, "threshold": -12.388426780700684, "operator": "ge"}, {"budget": 0.94, "threshold": -12.414846420288086, "operator": "ge"}, {"budget": 0.95, "threshold": -12.485769271850586, "operator": "ge"}, {"budget": 0.96, "threshold": -12.538629531860352, "operator": "ge"}, {"budget": 0.97, "threshold": -12.577099800109863, "operator": "ge"}, {"budget": 0.98, "threshold": -12.607933044433594, "operator": "ge"}, {"budget": 0.99, "threshold": -12.718658447265625, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.32946042975213, 19.297405718125237, 19.771459937574342, 20.227756980355966, 20.68641968055271, 21.033389179408395, 21.476060694476374, 22.01391701504969, 22.306123855663028, 22.60171840354767, 22.633668852156248, 22.678985007031986, 22.726528013434464, 24.00372304062949, 24.127714523561377, 24.11937322351656, 23.8967301933496, 23.73704210366317, 24.077453553599916, 24.102156334231804, 24.65642979174095, 24.526875566366197, 24.978382247161683, 25.415749485613066, 25.748442881990584, 25.84472614462987, 25.924099017109576, 26.260159448394738, 26.245558785529717, 26.584692762686956, 26.786691752088554, 27.389338102933852, 27.561327561327566, 27.67237664296488, 27.676458588331293, 28.300495988122766, 28.53817461779333, 28.680797412455235, 26.71558080261403, 26.98579521566457, 27.252021721295467, 27.004518685877255, 27.105385122520985, 27.68686868686869, 28.649117147391873, 28.9518962411386, 28.864483827719123, 28.27112939733938, 28.758556198224582, 28.30404607300426, 27.837735083560577, 27.740358372699518, 27.876784650040303, 27.781374845019975, 27.59114307783099, 27.37855694986268, 27.107536501260466, 26.471385638735327, 25.94205867232183, 25.472812141673483, 25.388803327232417, 25.024051670635266, 24.63766572761954, 24.590894192664102, 24.160801798747638, 23.933017567649294, 23.61783658856896, 23.688321494702972, 23.615978542618357, 23.576580577534965, 23.51674899660389, 23.398524891276846, 23.11056619379809, 23.063011388884398, 23.016602287921557, 22.961494916007126, 22.846773721773722, 22.57766272189349, 22.46321570390954, 22.343740231671266, 22.308619426490953, 22.28077513044073, 22.17694058528231, 21.843035661552374, 21.80221509463621, 21.768137647051528, 21.464211747485127, 21.4367719908957, 21.341320689833196, 21.303675447563318, 21.261905340506438, 21.251905090500188, 21.22951673398898, 21.212296029134194, 21.114958352847392, 21.018969117687643, 20.80293917830173, 20.779547772720647, 20.752322468902534, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 24, 37, 48, 59, 70, 81, 92, 101, 110, 115, 119, 124, 132, 136, 140, 143, 148, 154, 159, 164, 171, 175, 180, 186, 191, 196, 202, 207, 210, 215, 221, 225, 228, 232, 238, 244, 246, 250, 253, 256, 259, 262, 266, 271, 275, 276, 277, 281, 282, 283, 283, 284, 284, 284, 284, 284, 284, 284, 284, 284, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 4, 7, 9, 10, 11, 14, 18, 21, 23, 25, 28, 32, 39, 45, 50, 53, 57, 63, 69, 77, 81, 87, 91, 98, 102, 108, 112, 119, 125, 133, 136, 142, 146, 151, 157, 163, 168, 169, 175, 176, 181, 190, 195, 198, 204, 208, 216, 224, 229, 236, 243, 249, 257, 263, 270, 279, 287, 293, 301, 308, 315, 322, 328, 336, 342, 349, 354, 359, 366, 370, 376, 381, 386, 391, 396, 402, 407, 413, 418, 424, 429, 433, 437, 444, 450, 455, 461, 465, 469, 474, 480, 482, 486, 489, 494, 499, 505, 509, 514, 518], "exact_score": [18.254072168965788, 18.32946042975213, 19.297405718125237, 19.771459937574342, 20.227756980355966, 20.68641968055271, 21.033389179408395, 21.476060694476374, 22.01391701504969, 22.306123855663028, 22.60171840354767, 22.633668852156248, 22.678985007031986, 22.726528013434464, 24.00372304062949, 24.127714523561377, 24.11937322351656, 23.8967301933496, 23.73704210366317, 24.077453553599916, 24.102156334231804, 24.65642979174095, 24.526875566366197, 24.978382247161683, 25.415749485613066, 25.748442881990584, 25.84472614462987, 25.924099017109576, 26.260159448394738, 26.245558785529717, 26.584692762686956, 26.786691752088554, 27.389338102933852, 27.561327561327566, 27.67237664296488, 27.676458588331293, 28.300495988122766, 28.53817461779333, 28.680797412455235, 26.71558080261403, 26.98579521566457, 27.252021721295467, 27.004518685877255, 27.105385122520985, 27.68686868686869, 28.649117147391873, 28.9518962411386, 28.864483827719123, 28.27112939733938, 28.758556198224582, 28.30404607300426, 27.837735083560577, 27.740358372699518, 27.876784650040303, 27.781374845019975, 27.59114307783099, 27.37855694986268, 27.107536501260466, 26.471385638735327, 25.94205867232183, 25.472812141673483, 25.388803327232417, 25.024051670635266, 24.63766572761954, 24.590894192664102, 24.160801798747638, 23.933017567649294, 23.61783658856896, 23.688321494702972, 23.615978542618357, 23.576580577534965, 23.51674899660389, 23.398524891276846, 23.11056619379809, 23.063011388884398, 23.016602287921557, 22.961494916007126, 22.846773721773722, 22.57766272189349, 22.46321570390954, 22.343740231671266, 22.308619426490953, 22.28077513044073, 22.17694058528231, 21.843035661552374, 21.80221509463621, 21.768137647051528, 21.464211747485127, 21.4367719908957, 21.341320689833196, 21.303675447563318, 21.261905340506438, 21.251905090500188, 21.22951673398898, 21.212296029134194, 21.114958352847392, 21.018969117687643, 20.80293917830173, 20.779547772720647, 20.752322468902534, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.260221160827655, "threshold_area100": 24.265294091534173, "threshold_area25": 22.703853293474353}, "test": {"score": [18.129624718318375, 18.12668953167634, 18.30509128151973, 18.940164701784557, 19.070640798797122, 19.37665756762129, 19.836620185323632, 20.177292607872477, 20.617354052148848, 20.957654995140157, 21.067281237735276, 20.679831681476415, 20.89269059930863, 21.217417519826633, 21.24337464430899, 21.334155274713144, 21.524140747210623, 21.66034399256373, 21.8170266826751, 21.968690970873425, 21.966476097965394, 22.172554401394294, 22.29549458885729, 22.97023318399039, 23.450774712740426, 23.84531073808137, 24.03620325881317, 24.248275635531446, 24.43997748535965, 24.074353432110694, 24.19523284756394, 24.725448976372093, 24.730214286454753, 24.80819226282599, 24.906769462506396, 25.045242637003472, 25.18386364656944, 25.58306854678286, 25.674068735756684, 25.688489609733146, 25.777208284860908, 25.85078454644573, 26.021599794193325, 26.08840990466861, 26.18457378778495, 25.878316799499174, 25.969557633503452, 26.475296297727347, 26.514785747120378, 25.651007468569592, 25.496189173619033, 24.34163711707309, 24.16163337779552, 24.066262293858088, 23.980821995592397, 23.83695644297153, 23.77603710336783, 23.705961670987953, 23.47053607936478, 23.388870920545056, 23.300384689986462, 23.63134268800732, 23.478198261117193, 23.25630324479819, 22.994378038578127, 23.024549195430524, 23.212183728401705, 23.072234175454312, 23.09018356914841, 23.02917434347772, 22.861858414178265, 22.728406365028828, 22.717069819444426, 22.630029380588017, 22.619263214418037, 22.507589528613092, 22.46713136990895, 22.4344686920179, 22.99340235029658, 22.948153707436916, 22.940821102008776, 22.851705549724404, 22.77642460433388, 22.649634435386588, 22.57041121239725, 22.520034681022153, 22.452834294616395, 22.30442499385428, 22.948329343614358, 22.830510975455688, 22.790547033031196, 22.762435283831508, 22.689847389371078, 22.697928279080664, 22.681306139736193, 22.649822806494527, 22.45240461627223, 22.437816065983814, 22.426733939144714, 22.365108622324207, 22.361115727328066], "calls": [0.0, 0.4993342210386152, 1.1984021304926764, 2.2969374167776295, 2.9960053262316912, 4.194407456724368, 5.59254327563249, 7.057256990679095, 8.355525965379494, 9.55392809587217, 10.785619174434089, 12.51664447403462, 13.482023968042608, 14.247669773635153, 14.980026631158456, 16.07856191744341, 17.509986684420774, 18.142476697736353, 18.608521970705727, 19.10785619174434, 20.47270306258322, 22.03728362183755, 23.40213049267643, 25.099866844207725, 25.998668442077232, 27.230359520639148, 28.59520639147803, 29.394141145139812, 30.459387483355528, 31.458055925432753, 31.990679094540614, 32.49001331557923, 33.3555259653795, 34.22103861517976, 35.05326231691079, 36.41810918774967, 37.150466045272964, 37.949400798934754, 38.81491344873502, 39.480692410119836, 40.37949400798935, 40.84553928095872, 41.67776298268975, 42.177097203728366, 43.37549933422104, 44.607190412782955, 45.87217043941412, 46.40479360852197, 47.003994673768304, 48.23568575233023, 48.60186418109188, 50.699067909454065, 51.53129161118508, 52.46338215712384, 53.3288948069241, 54.19440745672437, 54.76031957390146, 56.15845539280959, 57.72303595206392, 58.95472703062583, 59.42077230359521, 61.284953395472705, 61.95073235685752, 63.24900133155792, 64.64713715046605, 65.21304926764314, 66.51131824234355, 67.70972037283622, 68.77496671105193, 69.20772303595206, 70.87217043941412, 71.80426098535287, 72.60319573901465, 73.8681757656458, 74.20106524633822, 75.56591211717709, 76.23169107856191, 76.93075898801598, 79.22769640479362, 79.82689747003995, 81.0918774966711, 81.75765645805593, 82.78961384820239, 83.65512649800266, 84.4873501997337, 85.25299600532622, 85.98535286284952, 87.21704394141145, 88.91478029294274, 90.11318242343542, 91.17842876165113, 92.0439414114514, 92.70972037283622, 94.6404793608522, 95.3728362183755, 96.47137150466045, 97.60319573901465, 98.30226364846871, 98.73501997336885, 99.83355525965379, 100.0], "rescued": [0, 27, 63, 119, 150, 200, 251, 307, 354, 390, 427, 476, 499, 521, 533, 557, 591, 604, 619, 630, 658, 695, 726, 765, 781, 813, 843, 863, 884, 905, 916, 927, 945, 962, 974, 998, 1014, 1029, 1042, 1052, 1066, 1071, 1084, 1091, 1108, 1130, 1148, 1154, 1159, 1163, 1164, 1168, 1172, 1173, 1174, 1175, 1175, 1176, 1177, 1178, 1178, 1181, 1181, 1182, 1182, 1183, 1184, 1185, 1188, 1188, 1188, 1188, 1190, 1191, 1191, 1191, 1191, 1191, 1194, 1194, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1197, 1197, 1197, 1197, 1197, 1198, 1198, 1198, 1198, 1198, 1198, 1198, 1198], "harmed": [0, 5, 18, 27, 40, 54, 79, 93, 104, 112, 130, 158, 174, 190, 204, 223, 254, 262, 270, 281, 310, 336, 366, 398, 416, 442, 470, 487, 509, 531, 541, 548, 572, 591, 602, 631, 644, 653, 670, 683, 697, 707, 722, 728, 754, 784, 817, 829, 837, 872, 883, 944, 968, 993, 1018, 1042, 1059, 1106, 1156, 1188, 1199, 1248, 1262, 1296, 1329, 1349, 1384, 1419, 1442, 1453, 1491, 1513, 1532, 1562, 1567, 1597, 1615, 1634, 1685, 1699, 1728, 1741, 1767, 1785, 1801, 1819, 1830, 1863, 1899, 1921, 1936, 1954, 1970, 2000, 2011, 2036, 2063, 2073, 2085, 2106, 2109], "exact_score": [18.129624718318375, 18.225144273827468, 18.602105702391636, 19.070640798797122, 19.30893941598328, 19.478623103003375, 19.93075623594754, 20.152428912276697, 20.494215610137935, 20.767884429759263, 21.02175985509319, 21.11668452361286, 20.514789747918584, 20.785446911992665, 21.164958404156174, 21.24337464430899, 21.334510748332864, 21.58195885999751, 21.622168079907695, 21.90207962584476, 22.003608710854206, 22.047432681031477, 22.165854631243885, 22.3038400150588, 22.56842751708389, 22.99212864430256, 23.450774712740426, 23.746796717098032, 24.030875415889767, 24.099828425753913, 24.299956940692738, 24.46254380857522, 24.19523284756394, 24.834084274768188, 24.828107776122483, 24.88534172197293, 25.04004717292515, 25.165718832587807, 25.596563121441022, 25.68408062858535, 25.814940040852726, 25.92394562795054, 26.04206266275232, 26.169676015439435, 26.337208246158315, 25.920681345209644, 26.05114989749208, 26.514785747120378, 25.73504137621758, 25.388878252998047, 24.950432561764934, 24.126487564892958, 24.15007423624953, 24.015293552836535, 23.859130033140186, 23.762179325818373, 23.736565432028293, 23.601475347316047, 23.45187856280826, 23.385718236034194, 23.829224613605465, 23.71326769599387, 23.478198261117193, 23.31477848613447, 23.142459228726565, 23.03566427324964, 22.812466813635186, 23.15854297375236, 23.087957421123704, 23.0446155285398, 22.925698066541866, 22.856758493510366, 22.76897271767329, 22.69096746011394, 22.627950127788303, 22.578657849605843, 22.477284096855687, 22.4344686920179, 23.166364823504956, 23.025898750399644, 22.940785557842432, 22.94476088994347, 22.8430429838889, 22.717151378994078, 22.642616530123483, 22.54752623014514, 22.452834294616395, 22.313360655882207, 22.645352722133367, 22.948329343614358, 22.835242816201838, 22.798954905746367, 22.765153237859252, 22.68354242444888, 22.712599460283244, 22.689199176386953, 22.664096338499004, 22.582296839546657, 22.444848400026967, 22.421476040862217, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 22.996979425570398, "threshold_area100": 23.05530566425376, "threshold_area25": 20.946407581246994}, "selected_budget": 46, "selected": {"val": {"score": 28.9518962411386, "calls": 45.93437945791726, "rescued": 275, "harmed": 208}, "test": {"score": 25.969557633503452, "calls": 45.87217043941412, "rescued": 1148, "harmed": 817}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 28.284479991066817, "calls": 46.932952924393724}, "test": {"score": 26.215437873972274, "calls": 46.63781624500666}}, "old_normalized_interior": {"val": {"score": 28.688125190338976, "calls": 46.932952924393724}, "test": {"score": 26.238122931940755, "calls": 47.070572569906794}}, "old_normalized_endpoints": {"val": {"score": 28.688125190338976, "calls": 46.932952924393724}, "test": {"score": 26.238122931940755, "calls": 47.070572569906794}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.515988130077112, 19.30434700184981, 19.752071969634592, 20.15959508031001, 20.499882299642902, 20.97243386202813, 21.39924575944401, 21.978655433653714, 22.381538636960162, 22.79585326953748, 22.648858646357937, 24.105801018398633, 23.92512298774508, 24.00075558620285, 24.10675483363776, 24.581643960731384, 24.4070358660164, 24.637424499084577, 24.85356817745521, 24.957744452835616, 25.404509283819632, 26.544527701531003, 26.71353419416547, 27.123344542699378, 27.41549183767394, 27.659535556401078, 25.9404975169505, 26.267375668272585, 26.37987698245191, 26.154269437069686, 26.090462969937946, 26.71156609195402, 26.866397721879448, 27.066519312969618, 27.196593144869002, 26.843761077632045, 26.789131037638114, 26.791990076005774, 26.531535050393106, 26.45374806704856, 26.940623880221192, 27.034248182031433, 27.615244939426624, 27.83212934296525, 28.404360110946936, 28.628390338768657, 28.688125190338976, 28.623203299488907, 28.65328152412551, 28.406215747168435, 28.141768268892342, 28.081031355684598, 28.051513168050345, 28.008778125315303, 27.47577509879664, 27.40862533692723, 26.91682079082366, 27.12300503617534, 26.97557126192273, 26.808072934121668, 26.639156285656817, 26.599494730923873, 26.770816606262883, 26.63034482891146, 26.203025369692035, 26.033955528962533, 25.98029246155139, 25.911293548402643, 25.64325521222073, 25.12985480776892, 24.671618489683528, 24.70991311424232, 24.737802889858393, 24.676866942704486, 23.985772985976254, 23.93474013171254, 23.414273877071707, 23.342424374426724, 23.076823783038474, 22.936673209684084, 22.895492687663253, 22.582916352598353, 22.540897338994696, 22.494438110938052, 22.443658043347927, 22.389033179795618, 22.341767338535867, 22.287839485064044, 22.16698420583817, 21.821153919903267, 21.610310496251444, 21.652086646188913, 21.539503212799218, 21.509760724620477, 21.395875251509054, 21.282811846786693, 21.06591915271207, 20.883125319294887, 20.699372617573584, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.515988130077112, 19.30434700184981, 19.752071969634592, 20.15959508031001, 20.499882299642902, 20.97243386202813, 21.39924575944401, 21.978655433653714, 22.381538636960162, 22.79585326953748, 22.648858646357937, 24.105801018398633, 23.92512298774508, 24.00075558620285, 24.10675483363776, 24.581643960731384, 24.4070358660164, 24.637424499084577, 24.85356817745521, 24.957744452835616, 25.404509283819632, 26.544527701531003, 26.71353419416547, 27.123344542699378, 27.41549183767394, 27.659535556401078, 25.9404975169505, 26.267375668272585, 26.37987698245191, 26.154269437069686, 26.090462969937946, 26.71156609195402, 26.866397721879448, 27.066519312969618, 27.196593144869002, 26.843761077632045, 26.789131037638114, 26.791990076005774, 26.531535050393106, 26.45374806704856, 26.940623880221192, 27.034248182031433, 27.615244939426624, 27.83212934296525, 28.404360110946936, 28.628390338768657, 28.688125190338976, 28.623203299488907, 28.65328152412551, 28.406215747168435, 28.141768268892342, 28.081031355684598, 28.051513168050345, 28.008778125315303, 27.47577509879664, 27.40862533692723, 26.91682079082366, 27.12300503617534, 26.97557126192273, 26.808072934121668, 26.639156285656817, 26.599494730923873, 26.770816606262883, 26.63034482891146, 26.203025369692035, 26.033955528962533, 25.98029246155139, 25.911293548402643, 25.64325521222073, 25.12985480776892, 24.671618489683528, 24.70991311424232, 24.737802889858393, 24.676866942704486, 23.985772985976254, 23.93474013171254, 23.414273877071707, 23.342424374426724, 23.076823783038474, 22.936673209684084, 22.895492687663253, 22.582916352598353, 22.540897338994696, 22.494438110938052, 22.443658043347927, 22.389033179795618, 22.341767338535867, 22.287839485064044, 22.16698420583817, 21.821153919903267, 21.610310496251444, 21.652086646188913, 21.539503212799218, 21.509760724620477, 21.395875251509054, 21.282811846786693, 21.06591915271207, 20.883125319294887, 20.699372617573584, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 23.039003371794724, "exact_area100": 24.742571798637606}, "test": {"score": [18.129624718318375, 18.3155537051833, 18.373362488769338, 18.985507482355317, 19.201661853694933, 19.41375285002371, 19.945460481878293, 20.24633040591968, 20.471498771498773, 20.783979601413552, 21.025692094115048, 21.505208849402223, 21.71068320908274, 21.939287148821833, 22.296461766909434, 22.74188910218322, 23.050808564138116, 23.278173907957296, 23.437723902513696, 23.48903458574948, 23.800778223663926, 24.080294753583793, 24.211681088037157, 24.798864265542907, 24.80136353715138, 25.403172373211564, 25.659317416069555, 25.76593633404779, 26.05204343411397, 26.36408803418758, 26.08024217863226, 26.311557552603137, 26.55418809646941, 26.719070179877924, 26.823156323282227, 26.334926294297063, 26.34032486078998, 26.468100358530293, 26.502714630212378, 26.548558636126916, 26.731320794286294, 26.358219863747646, 26.341558204600123, 26.343097230725316, 26.56935460332923, 26.58283883493646, 26.406845211941665, 26.238122931940755, 26.160486635327857, 26.152763717029494, 26.112067016788558, 25.988741675853475, 25.915236489729732, 25.372057515590086, 25.272804367859692, 25.17016597471421, 25.457041068998247, 25.336275172635396, 25.193381685664978, 24.93387387725627, 24.653253256861284, 24.609008900424552, 24.455916374240385, 24.266813128942765, 24.083993144135032, 24.446770704659002, 24.213623796772993, 24.10047697800265, 23.897822134432182, 23.80881599245309, 24.20036776383648, 24.149904060371135, 24.128006004851635, 23.96974536201568, 23.933960992274834, 23.73767002835864, 23.573327587165657, 23.48526108559778, 23.302675109246955, 23.254454260955683, 23.23732852408233, 23.188431934650865, 23.16649494192722, 23.00892341819515, 22.905787562835947, 22.761990750072794, 22.63034935249534, 22.541300623419314, 22.878368161033084, 22.5720544956771, 22.552476421334386, 22.753814435215318, 23.095170246528724, 22.890887768562756, 22.818270620654328, 22.7521971457705, 22.725858454541928, 22.628931005394506, 22.62993556040118, 22.530918730664798, 22.361115727328066], "calls": [0.0, 0.5326231691078562, 1.2649800266311584, 2.4966711051930757, 3.262316910785619, 4.161118508655127, 4.826897470039946, 5.758988015978695, 6.824234354194407, 8.754993342210387, 10.486018641810919, 11.684420772303596, 12.21704394141145, 12.916111850865514, 13.81491344873502, 14.680426098535287, 15.978695073235686, 17.743009320905458, 18.841544607190414, 19.67376830892144, 21.13848202396804, 21.604527296937416, 22.270306258322236, 23.601864181091877, 24.167776298268976, 24.96671105193076, 25.998668442077232, 26.93075898801598, 28.66178428761651, 29.860186418109187, 30.792276964047936, 31.89081225033289, 33.68841544607191, 34.42077230359521, 35.38615179760319, 35.81890812250333, 36.81757656458056, 37.68308921438083, 38.18242343541944, 39.18109187749667, 40.37949400798935, 42.01065246338216, 42.50998668442077, 43.54194407456725, 44.14114513981358, 44.573901464713714, 45.87217043941412, 47.070572569906794, 47.969374167776294, 48.46870838881491, 49.20106524633822, 50.53262316910786, 50.93209054593875, 52.46338215712384, 53.22902796271638, 55.12649800266311, 56.79094540612517, 57.55659121171771, 58.48868175765646, 59.42077230359521, 61.31824234354194, 61.98402130492676, 62.38348868175766, 63.61517976031957, 64.84687083888149, 66.04527296937417, 67.74300932090546, 68.97470039946738, 70.0399467376831, 70.83888149134488, 71.43808255659121, 73.36884154460719, 74.30093209054593, 75.43275632490013, 75.99866844207723, 77.13049267643143, 78.4953395472703, 79.02796271637817, 80.32623169107856, 80.95872170439414, 81.72436750998668, 82.82290279627163, 83.38881491344874, 83.92143808255659, 84.52063914780294, 85.58588548601864, 86.45139813581892, 88.11584553928095, 89.01464713715048, 90.07989347536618, 90.54593874833556, 92.54327563249002, 93.04260985352862, 93.84154460719041, 94.44074567243675, 95.20639147802929, 95.90545938748336, 96.77097203728363, 97.47003994673769, 98.66844207723035, 100.0], "exact_score": [18.129624718318375, 18.280139245770282, 18.849281493873654, 19.11848207282174, 19.370473845067465, 19.994339574129505, 20.27044554600355, 20.47752324422003, 20.647966037619096, 20.819867810632246, 20.88569376842354, 21.15087463080124, 21.604666030048342, 21.959945438244407, 22.319962887677473, 22.79260726145417, 23.050808564138116, 23.100730484461394, 23.28442692573127, 23.442809492531435, 23.58327677142426, 23.815694946606587, 24.101100739856722, 24.626129730940548, 24.810566812751947, 25.411963471422226, 25.659317416069555, 25.79135680456166, 25.671599117847304, 26.090348128476293, 26.375721512601473, 26.10244354685662, 26.329196103419804, 26.549428143997766, 26.622499601475567, 26.74376437793763, 26.339956561521984, 26.36083418503804, 26.44116319923087, 26.529626037581355, 26.70371486005801, 26.73924482968893, 26.359685890546814, 26.312345193073373, 26.51774100453592, 25.88080975195145, 26.44196729732227, 26.30343118690418, 26.160486635327857, 26.10666449618163, 26.053153386375627, 25.915236489729732, 25.898081184769733, 25.282143446927147, 25.248851439155352, 25.19858907597814, 25.1232284454188, 25.450189828748087, 25.28759899393079, 25.093138036369737, 24.82632608140383, 24.69132906230442, 24.609008900424552, 24.381276487153126, 24.228092800860903, 24.059738964859353, 24.50195326816422, 24.34504898077988, 24.20734004615934, 24.10047697800265, 23.900814951670814, 23.804290819984704, 24.168043720575074, 24.174051043209477, 24.146414456988964, 24.088306661527124, 23.933960992274834, 23.69873273171577, 23.634319353060047, 23.486636268497104, 23.31628797445934, 23.254454260955683, 23.182368806735195, 23.18083121665232, 23.00297768890359, 22.845628731594356, 22.70502675475007, 22.60728723886719, 22.543349477558237, 22.878368161033084, 22.626320171419557, 22.49417523917801, 22.80944699894553, 23.08932438202462, 22.885021747423036, 22.778643402808495, 22.72281724147902, 22.65001517954417, 22.595191025434502, 22.428251854708854, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.749702163150175, "exact_area100": 23.933462989323864}}, "old_normalized_endpoint_budget": 47}, {"seed": 2027, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.5, "validation_area100": 0.2477446419752862, "training_seconds": 3.831651362008415, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.32428159089113795, "operator": "ge"}, {"budget": 0.02, "threshold": 0.289368052691958, "operator": "ge"}, {"budget": 0.03, "threshold": 0.2786715501095895, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2603252794431717, "operator": "ge"}, {"budget": 0.05, "threshold": 0.25184104357751425, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2314334342149778, "operator": "ge"}, {"budget": 0.07, "threshold": 0.1991055083280256, "operator": "ge"}, {"budget": 0.08, "threshold": 0.1681239574105767, "operator": "ge"}, {"budget": 0.09, "threshold": 0.13577276034044172, "operator": "ge"}, {"budget": 0.1, "threshold": 0.11882500566504886, "operator": "ge"}, {"budget": 0.11, "threshold": 0.09915887153244295, "operator": "ge"}, {"budget": 0.12, "threshold": 0.08442114718318514, "operator": "ge"}, {"budget": 0.13, "threshold": 0.07673277206078345, "operator": "ge"}, {"budget": 0.14, "threshold": 0.07208206118465771, "operator": "ge"}, {"budget": 0.15, "threshold": 0.06956535144764045, "operator": "ge"}, {"budget": 0.16, "threshold": 0.06525019068877834, "operator": "ge"}, {"budget": 0.17, "threshold": 0.06053347310936504, "operator": "ge"}, {"budget": 0.18, "threshold": 0.0545368076877669, "operator": "ge"}, {"budget": 0.19, "threshold": 0.04942660727282963, "operator": "ge"}, {"budget": 0.2, "threshold": 0.046547437580242934, "operator": "ge"}, {"budget": 0.21, "threshold": 0.04239045440879809, "operator": "ge"}, {"budget": 0.22, "threshold": 0.03760735174720705, "operator": "ge"}, {"budget": 0.23, "threshold": 0.034876420104549305, "operator": "ge"}, {"budget": 0.24, "threshold": 0.03001299091118253, "operator": "ge"}, {"budget": 0.25, "threshold": 0.025297318578218857, "operator": "ge"}, {"budget": 0.26, "threshold": 0.022616299989678085, "operator": "ge"}, {"budget": 0.27, "threshold": 0.02021604927216131, "operator": "ge"}, {"budget": 0.28, "threshold": 0.009651876629863606, "operator": "ge"}, {"budget": 0.29, "threshold": 0.005301286922823947, "operator": "ge"}, {"budget": 0.3, "threshold": 0.0035480071592933864, "operator": "ge"}, {"budget": 0.31, "threshold": -0.0022210739139446284, "operator": "ge"}, {"budget": 0.32, "threshold": -0.005687413009547701, "operator": "ge"}, {"budget": 0.33, "threshold": -0.00992703492788382, "operator": "ge"}, {"budget": 0.34, "threshold": -0.014349691172977803, "operator": "ge"}, {"budget": 0.35, "threshold": -0.01909043899928848, "operator": "ge"}, {"budget": 0.36, "threshold": -0.023193037724505424, "operator": "ge"}, {"budget": 0.37, "threshold": -0.028362537377171846, "operator": "ge"}, {"budget": 0.38, "threshold": -0.0335834357751191, "operator": "ge"}, {"budget": 0.39, "threshold": -0.04251485610479216, "operator": "ge"}, {"budget": 0.4, "threshold": -0.04988544575294298, "operator": "ge"}, {"budget": 0.41, "threshold": -0.05706847858105782, "operator": "ge"}, {"budget": 0.42, "threshold": -0.0713458158056152, "operator": "ge"}, {"budget": 0.43, "threshold": -0.09056668283644485, "operator": "ge"}, {"budget": 0.44, "threshold": -0.12056702509908677, "operator": "ge"}, {"budget": 0.45, "threshold": -0.15157535371222589, "operator": "ge"}, {"budget": 0.46, "threshold": -0.27194246924330945, "operator": "ge"}, {"budget": 0.47, "threshold": -0.3083505885625456, "operator": "ge"}, {"budget": 0.48, "threshold": -0.31268960395772477, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3206152415904336, "operator": "ge"}, {"budget": 0.5, "threshold": -0.324019860148319, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3282090509954158, "operator": "ge"}, {"budget": 0.52, "threshold": -0.3385980818257768, "operator": "ge"}, {"budget": 0.53, "threshold": -0.34444460924345344, "operator": "ge"}, {"budget": 0.54, "threshold": -0.34922636477863406, "operator": "ge"}, {"budget": 0.55, "threshold": -0.3557478978294564, "operator": "ge"}, {"budget": 0.56, "threshold": -0.36024667460823834, "operator": "ge"}, {"budget": 0.57, "threshold": -0.36298863332529846, "operator": "ge"}, {"budget": 0.58, "threshold": -0.3652331065862104, "operator": "ge"}, {"budget": 0.59, "threshold": -0.3702937194741188, "operator": "ge"}, {"budget": 0.6, "threshold": -0.3773905448642255, "operator": "ge"}, {"budget": 0.61, "threshold": -0.38076704906117603, "operator": "ge"}, {"budget": 0.62, "threshold": -0.38601827311306136, "operator": "ge"}, {"budget": 0.63, "threshold": -0.38938436218151345, "operator": "ge"}, {"budget": 0.64, "threshold": -0.392535287976218, "operator": "ge"}, {"budget": 0.65, "threshold": -0.39529168430050715, "operator": "ge"}, {"budget": 0.66, "threshold": -0.39771556241584705, "operator": "ge"}, {"budget": 0.67, "threshold": -0.4003129683972162, "operator": "ge"}, {"budget": 0.68, "threshold": -0.40203435211468685, "operator": "ge"}, {"budget": 0.69, "threshold": -0.40362136101519963, "operator": "ge"}, {"budget": 0.7, "threshold": -0.40484123390069743, "operator": "ge"}, {"budget": 0.71, "threshold": -0.40590939252751185, "operator": "ge"}, {"budget": 0.72, "threshold": -0.40780116307304126, "operator": "ge"}, {"budget": 0.73, "threshold": -0.4087549659862676, "operator": "ge"}, {"budget": 0.74, "threshold": -0.4093952543491555, "operator": "ge"}, {"budget": 0.75, "threshold": -0.41039087869411794, "operator": "ge"}, {"budget": 0.76, "threshold": -0.4114494681317808, "operator": "ge"}, {"budget": 0.77, "threshold": -0.41210599396305697, "operator": "ge"}, {"budget": 0.78, "threshold": -0.41437528515821137, "operator": "ge"}, {"budget": 0.79, "threshold": -0.41518338885246653, "operator": "ge"}, {"budget": 0.8, "threshold": -0.4162065440578851, "operator": "ge"}, {"budget": 0.81, "threshold": -0.41752649558615407, "operator": "ge"}, {"budget": 0.82, "threshold": -0.4184373385869481, "operator": "ge"}, {"budget": 0.83, "threshold": -0.41974153716424506, "operator": "ge"}, {"budget": 0.84, "threshold": -0.42061165980135057, "operator": "ge"}, {"budget": 0.85, "threshold": -0.42152409126798895, "operator": "ge"}, {"budget": 0.86, "threshold": -0.4219648915566577, "operator": "ge"}, {"budget": 0.87, "threshold": -0.4228040977298024, "operator": "ge"}, {"budget": 0.88, "threshold": -0.4246973739891789, "operator": "ge"}, {"budget": 0.89, "threshold": -0.4254765302938322, "operator": "ge"}, {"budget": 0.9, "threshold": -0.42672290539353647, "operator": "ge"}, {"budget": 0.91, "threshold": -0.4276776080164763, "operator": "ge"}, {"budget": 0.92, "threshold": -0.42886327029481125, "operator": "ge"}, {"budget": 0.93, "threshold": -0.4307383921410664, "operator": "ge"}, {"budget": 0.94, "threshold": -0.43087573009839225, "operator": "ge"}, {"budget": 0.95, "threshold": -0.4319749683009219, "operator": "ge"}, {"budget": 0.96, "threshold": -0.4329183812117159, "operator": "ge"}, {"budget": 0.97, "threshold": -0.43523563616034516, "operator": "ge"}, {"budget": 0.98, "threshold": -0.43761833424731894, "operator": "ge"}, {"budget": 0.99, "threshold": -0.4408071967842974, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.507340074066676, 18.882101214632783, 19.723935673560206, 20.170190037575235, 20.685260125821728, 21.036291512310264, 21.479167236365956, 21.841305754173412, 22.23979591836735, 22.753253475303165, 24.005474596657887, 23.8549796510357, 24.007846464033396, 23.576111997164627, 23.398930049938908, 23.446975826940395, 23.526917844373088, 23.836085509472607, 24.146387438183908, 24.366080634806682, 24.445253022839232, 24.636798253416853, 25.06030855539972, 25.623799708351857, 26.10047904165551, 26.22529428816088, 26.931289539357678, 26.90484601258814, 26.480258490863974, 26.54721210157655, 26.673899802040506, 27.061116536398444, 27.606743532181078, 25.759846815314148, 26.179939549247656, 27.285927321246373, 27.437456419678753, 27.415526595117345, 27.558027079303677, 27.716190733119124, 27.97490603572908, 28.188131313131315, 28.485982194596133, 28.22488038277512, 28.250905774248718, 28.64514777488093, 28.576460271259567, 28.559779704120285, 28.510780112450217, 28.478477796890196, 28.264523200584584, 28.002916715691434, 27.738063624570096, 27.66699031213869, 27.597259232360017, 27.6109315694397, 27.65219971332733, 27.591127107985926, 27.042854686706576, 27.008531225409865, 26.927859825948996, 27.041292117654255, 26.70581054626146, 26.665009670276852, 26.560378856269086, 26.508148633325586, 25.9410118037028, 25.887372192423737, 25.754293370280852, 25.704858033793958, 25.17688118420719, 25.219146238377004, 24.8301680963951, 24.559355874011047, 24.203057811753464, 24.070098340774877, 23.677525356289745, 23.758979805194745, 23.48205781948113, 23.202893764532323, 23.164434653173753, 23.02016563555884, 22.972039816552016, 22.634001726011412, 22.50607420224835, 22.076358199370315, 22.044738480029192, 22.187744166126233, 22.16014735335901, 22.05378799745576, 21.941487437438855, 21.833482540839082, 21.59907328387248, 21.37932438703988, 21.33794046889004, 21.219065887480653, 21.182862336562962, 21.098805740256644, 21.05246594434204, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 24, 37, 48, 59, 71, 82, 90, 101, 109, 116, 122, 129, 132, 136, 141, 147, 153, 159, 167, 172, 177, 182, 189, 192, 198, 203, 207, 209, 211, 215, 221, 227, 230, 233, 241, 244, 245, 248, 253, 256, 260, 262, 267, 271, 274, 274, 274, 274, 274, 275, 275, 275, 275, 275, 276, 277, 277, 277, 277, 277, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 279, 279, 280, 280, 280, 280, 281, 282, 282, 282, 282, 282, 282, 282, 282, 282, 283, 284, 284, 284, 284, 284, 284, 284, 284, 284, 285, 285, 286], "harmed": [0, 2, 4, 6, 11, 12, 15, 19, 20, 24, 24, 31, 36, 39, 46, 57, 62, 66, 69, 72, 77, 82, 88, 93, 99, 101, 107, 116, 121, 126, 127, 131, 136, 142, 149, 153, 161, 165, 167, 172, 176, 178, 181, 189, 194, 204, 209, 213, 214, 217, 223, 230, 234, 238, 243, 248, 254, 258, 263, 269, 272, 279, 285, 292, 296, 298, 303, 309, 314, 320, 326, 333, 337, 342, 351, 358, 364, 370, 377, 382, 389, 395, 402, 408, 416, 422, 431, 436, 443, 449, 454, 459, 465, 471, 477, 483, 491, 497, 503, 510, 518], "exact_score": [18.254072168965788, 18.507340074066676, 18.882101214632783, 19.723935673560206, 20.170190037575235, 20.685260125821728, 21.036291512310264, 21.479167236365956, 21.841305754173412, 22.23979591836735, 22.753253475303165, 24.005474596657887, 23.8549796510357, 24.007846464033396, 23.576111997164627, 23.398930049938908, 23.446975826940395, 23.526917844373088, 23.836085509472607, 24.146387438183908, 24.366080634806682, 24.445253022839232, 24.636798253416853, 25.06030855539972, 25.623799708351857, 26.10047904165551, 26.22529428816088, 26.931289539357678, 26.90484601258814, 26.480258490863974, 26.54721210157655, 26.673899802040506, 27.061116536398444, 27.606743532181078, 25.759846815314148, 26.179939549247656, 27.285927321246373, 27.437456419678753, 27.415526595117345, 27.558027079303677, 27.716190733119124, 27.97490603572908, 28.188131313131315, 28.485982194596133, 28.22488038277512, 28.250905774248718, 28.64514777488093, 28.576460271259567, 28.559779704120285, 28.510780112450217, 28.478477796890196, 28.264523200584584, 28.002916715691434, 27.738063624570096, 27.66699031213869, 27.597259232360017, 27.6109315694397, 27.65219971332733, 27.591127107985926, 27.042854686706576, 27.008531225409865, 26.927859825948996, 27.041292117654255, 26.70581054626146, 26.665009670276852, 26.560378856269086, 26.508148633325586, 25.9410118037028, 25.887372192423737, 25.754293370280852, 25.704858033793958, 25.17688118420719, 25.219146238377004, 24.8301680963951, 24.559355874011047, 24.203057811753464, 24.070098340774877, 23.677525356289745, 23.758979805194745, 23.48205781948113, 23.202893764532323, 23.164434653173753, 23.02016563555884, 22.972039816552016, 22.634001726011412, 22.50607420224835, 22.076358199370315, 22.044738480029192, 22.187744166126233, 22.16014735335901, 22.05378799745576, 21.941487437438855, 21.833482540839082, 21.59907328387248, 21.37932438703988, 21.33794046889004, 21.219065887480653, 21.182862336562962, 21.098805740256644, 21.05246594434204, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.77446419752862, "threshold_area100": 24.780057372947965, "threshold_area25": 22.676159978236228}, "test": {"score": [18.129624718318375, 18.17787677753799, 18.66103186004181, 18.72844901942819, 18.996709929044613, 19.20881619327964, 19.416133784894026, 19.809073101710997, 20.50748988436245, 20.90305790174854, 21.02779953802219, 21.365545051652955, 21.580438291934023, 22.135661450748916, 22.181886679881156, 22.18349003837692, 22.300620993753782, 22.548911216937174, 22.416180256169824, 22.631549607913858, 22.814062209718582, 23.104847450144177, 23.631625533996804, 23.37382330403698, 23.385083051321097, 23.583403854369735, 23.552052666676946, 23.580275868741275, 24.24258884768557, 25.034494867840927, 24.49497484614827, 24.713463003706902, 24.905081172018775, 25.027444198162534, 25.036481327572556, 25.144997462703895, 25.835126114972034, 25.89002757833675, 26.15171887101328, 26.26537930736424, 26.310547970984178, 26.335805050870732, 26.374156430016626, 26.34891817723598, 25.76100392220855, 25.79925194504586, 25.93611863895939, 25.826974980561314, 25.696102476985722, 25.63865324427389, 25.5806436593087, 25.531119851713907, 25.437315330634952, 25.22893080772917, 25.150362130551212, 25.10149364872713, 25.080911625434272, 24.99714140100763, 24.932856718942627, 25.151701748582894, 25.03867262592836, 24.957472080767467, 25.350457457663556, 25.19430934879155, 25.068566617791998, 24.878994459384323, 24.799285182047267, 24.704511814564263, 24.52136994965679, 24.381709372221707, 24.239746584270836, 24.198266700507602, 24.123811181260177, 24.086785121796485, 23.998173218390605, 24.363537724787538, 24.22971909128888, 24.138890922942316, 24.380629312977526, 24.19280304551702, 24.135156068827467, 23.99657565766593, 23.890680606522487, 23.769975180910293, 23.600999779818657, 23.92320784714431, 23.883663926892133, 23.599207110070683, 23.550006962492446, 23.424845914170543, 22.905985634829527, 22.865585318481223, 22.874619815480727, 22.75046786750957, 22.743550683392368, 23.12118931746083, 23.053733933877066, 22.859026379264584, 22.702192085558526, 22.594314544287343, 22.361115727328066], "calls": [0.0, 0.6324900133155792, 1.8641810918774968, 2.330226364846871, 3.561917443408788, 4.194407456724368, 5.25965379494008, 6.55792276964048, 8.189081225033288, 9.587217043941411, 10.286284953395473, 11.351531291611185, 12.649800266311583, 13.615179760319574, 14.314247669773636, 14.580559254327563, 15.412782956058587, 16.4780292942743, 18.242343541944077, 19.474034620505993, 19.973368841544605, 21.00532623169108, 22.470039946737682, 23.302263648468706, 24.267643142476697, 25.099866844207725, 25.499334221038616, 25.96537949400799, 28.19573901464714, 29.26098535286285, 29.6271637816245, 31.391478029294273, 32.456724367509985, 33.58854860186418, 34.25432756324901, 35.21970705725699, 36.1185086551265, 37.31691078561917, 38.115845539280954, 39.314247669773636, 40.07989347536618, 40.54593874833555, 41.6444740346205, 42.543275632490015, 43.70838881491345, 44.374167776298265, 45.50599201065246, 47.237017310252995, 47.969374167776294, 48.8681757656458, 49.30093209054594, 49.70039946737683, 51.16511318242344, 52.197070572569906, 52.89613848202397, 54.627163781624496, 55.72569906790945, 56.55792276964048, 57.090545938748335, 58.68841544607191, 60.75233022636485, 61.18508655126498, 62.35019973368841, 63.58189081225033, 64.4474034620506, 65.31291611185085, 66.57789613848203, 67.90945406125167, 68.94141145139814, 70.30625832223701, 71.271637816245, 71.80426098535287, 73.23568575233023, 74.13448735019973, 74.80026631158455, 75.56591211717709, 76.33155792276965, 77.03062583222369, 78.69507323568575, 79.36085219707057, 80.62583222370174, 81.8242343541944, 82.78961384820239, 84.05459387483354, 85.08655126498003, 85.68575233022636, 86.18508655126497, 87.1837549933422, 88.91478029294274, 89.78029294274302, 91.44474034620505, 92.17709720372837, 93.20905459387482, 94.50732356857523, 94.6404793608522, 95.50599201065246, 95.93874833555259, 97.07057256990679, 97.83621837549934, 98.50199733688414, 100.0], "rescued": [0, 37, 94, 113, 164, 191, 233, 278, 344, 392, 411, 444, 478, 507, 524, 529, 548, 571, 614, 643, 655, 681, 717, 735, 751, 768, 778, 786, 839, 864, 869, 906, 929, 952, 966, 988, 1005, 1024, 1043, 1062, 1069, 1077, 1088, 1098, 1116, 1126, 1145, 1150, 1151, 1151, 1152, 1152, 1152, 1152, 1152, 1155, 1157, 1158, 1159, 1161, 1163, 1164, 1167, 1168, 1169, 1169, 1169, 1169, 1169, 1170, 1170, 1171, 1173, 1174, 1175, 1176, 1176, 1176, 1177, 1177, 1177, 1178, 1178, 1179, 1179, 1181, 1181, 1183, 1184, 1185, 1185, 1185, 1188, 1189, 1189, 1191, 1191, 1192, 1193, 1194, 1198], "harmed": [0, 7, 26, 33, 52, 58, 75, 86, 103, 118, 126, 139, 167, 183, 198, 204, 216, 236, 273, 301, 311, 333, 362, 376, 400, 416, 427, 436, 478, 503, 511, 543, 560, 583, 603, 625, 637, 661, 673, 695, 705, 712, 724, 741, 769, 783, 810, 839, 849, 865, 875, 887, 913, 932, 940, 975, 991, 1005, 1017, 1052, 1088, 1099, 1127, 1156, 1176, 1199, 1223, 1255, 1281, 1316, 1349, 1362, 1395, 1417, 1433, 1454, 1471, 1485, 1533, 1549, 1579, 1614, 1637, 1666, 1695, 1712, 1728, 1749, 1796, 1814, 1859, 1879, 1909, 1944, 1948, 1974, 1987, 2020, 2045, 2066, 2109], "exact_score": [18.129624718318375, 18.28578577315594, 18.704077928870753, 18.881680777548095, 19.136533234370184, 19.407234650405442, 19.59075209293343, 19.993811798026986, 20.455598200016883, 20.755587549705197, 20.985990192902584, 21.28039681912009, 21.47193576138749, 21.646116924631514, 22.096182494415224, 22.235349411066192, 22.473162993565122, 22.56840105455414, 22.39091381100727, 22.50352474359722, 22.814062209718582, 23.070585463890563, 23.479942874767616, 23.783032534011696, 23.458029743956637, 23.528522283983282, 23.586854360888843, 24.189925666644914, 24.196004775001644, 24.978862312505818, 24.695618265364697, 24.8403472468891, 24.843926731927617, 24.95312554975385, 25.077536790630745, 25.05703329375329, 25.808419283722685, 25.85210987567812, 26.059544427383695, 26.202184800066224, 26.298631927306417, 26.349602249261828, 26.345867632122626, 26.338780804035377, 25.765517334583592, 25.823969499016574, 25.890193548375752, 25.822770979020977, 25.696102476985722, 25.635691860715493, 25.50814164880066, 25.447036304393748, 25.238375468713016, 25.143670761303145, 25.038326575740832, 25.044251062851785, 25.066221453416578, 24.940828670687917, 24.73834920021802, 25.219500157000834, 25.05841392872189, 24.96863980812006, 25.379647622783764, 25.274923940302045, 25.16431617811023, 24.89826108128304, 24.823552597112933, 24.78366851449878, 24.702787438788295, 24.518245457386783, 24.438381130579963, 24.275460357882768, 24.190803559966955, 24.197121485509797, 24.099836504779024, 23.937237178043098, 24.323884145767302, 24.14161394905665, 24.493915503578055, 24.345267137206513, 24.17081991415366, 24.09433936336197, 23.928411580318564, 23.82625257933949, 23.772314608452376, 23.653717635371528, 23.909702363348348, 23.721188522216547, 23.374594917647688, 23.547866588183787, 23.366744483753994, 23.312344297503373, 22.86973329312548, 22.883704629907722, 22.76867251341726, 22.725619714052257, 23.051881677762967, 22.861333823078326, 22.748947002861094, 22.609489167553647, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.76166972505993, "threshold_area100": 23.74105749543781, "threshold_area25": 21.21813645314828}, "selected_budget": 46, "selected": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726, "rescued": 274, "harmed": 209}, "test": {"score": 25.93611863895939, "calls": 45.50599201065246, "rescued": 1145, "harmed": 810}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.2868528962135315, "operator": "ge"}, {"budget": 0.02, "threshold": 0.25111040472984314, "operator": "ge"}, {"budget": 0.03, "threshold": 0.2317826747894287, "operator": "ge"}, {"budget": 0.04, "threshold": 0.22151030600070953, "operator": "ge"}, {"budget": 0.05, "threshold": 0.2155287116765976, "operator": "ge"}, {"budget": 0.06, "threshold": 0.20707790553569794, "operator": "ge"}, {"budget": 0.07, "threshold": 0.17042846977710724, "operator": "ge"}, {"budget": 0.08, "threshold": 0.14630725979804993, "operator": "ge"}, {"budget": 0.09, "threshold": 0.12520667910575867, "operator": "ge"}, {"budget": 0.1, "threshold": 0.07985508441925049, "operator": "ge"}, {"budget": 0.11, "threshold": 0.07119356095790863, "operator": "ge"}, {"budget": 0.12, "threshold": 0.058609738945961, "operator": "ge"}, {"budget": 0.13, "threshold": 0.05263875424861908, "operator": "ge"}, {"budget": 0.14, "threshold": 0.04938949644565582, "operator": "ge"}, {"budget": 0.15, "threshold": 0.04607081413269043, "operator": "ge"}, {"budget": 0.16, "threshold": 0.04448631405830383, "operator": "ge"}, {"budget": 0.17, "threshold": 0.04202960431575775, "operator": "ge"}, {"budget": 0.18, "threshold": 0.037329643964767456, "operator": "ge"}, {"budget": 0.19, "threshold": 0.034348875284194946, "operator": "ge"}, {"budget": 0.2, "threshold": 0.031656742095947266, "operator": "ge"}, {"budget": 0.21, "threshold": 0.02938924729824066, "operator": "ge"}, {"budget": 0.22, "threshold": 0.027489349246025085, "operator": "ge"}, {"budget": 0.23, "threshold": 0.024091482162475586, "operator": "ge"}, {"budget": 0.24, "threshold": 0.02113249897956848, "operator": "ge"}, {"budget": 0.25, "threshold": 0.017551064491271973, "operator": "ge"}, {"budget": 0.26, "threshold": 0.015892788767814636, "operator": "ge"}, {"budget": 0.27, "threshold": 0.013979136943817139, "operator": "ge"}, {"budget": 0.28, "threshold": 0.007272511720657349, "operator": "ge"}, {"budget": 0.29, "threshold": 0.003955721855163574, "operator": "ge"}, {"budget": 0.3, "threshold": 0.0025006234645843506, "operator": "ge"}, {"budget": 0.31, "threshold": -0.0015078634023666382, "operator": "ge"}, {"budget": 0.32, "threshold": -0.004067152738571167, "operator": "ge"}, {"budget": 0.33, "threshold": -0.006924822926521301, "operator": "ge"}, {"budget": 0.34, "threshold": -0.010259643197059631, "operator": "ge"}, {"budget": 0.35, "threshold": -0.01297256350517273, "operator": "ge"}, {"budget": 0.36, "threshold": -0.017449960112571716, "operator": "ge"}, {"budget": 0.37, "threshold": -0.02065350115299225, "operator": "ge"}, {"budget": 0.38, "threshold": -0.02730512246489525, "operator": "ge"}, {"budget": 0.39, "threshold": -0.03156419098377228, "operator": "ge"}, {"budget": 0.4, "threshold": -0.03478717803955078, "operator": "ge"}, {"budget": 0.41, "threshold": -0.04201299324631691, "operator": "ge"}, {"budget": 0.42, "threshold": -0.04681025445461273, "operator": "ge"}, {"budget": 0.43, "threshold": -0.06325958669185638, "operator": "ge"}, {"budget": 0.44, "threshold": -0.08185030519962311, "operator": "ge"}, {"budget": 0.45, "threshold": -0.08855905383825302, "operator": "ge"}, {"budget": 0.46, "threshold": -0.09560920298099518, "operator": "ge"}, {"budget": 0.47, "threshold": -0.09814518690109253, "operator": "ge"}, {"budget": 0.48, "threshold": -0.10414987802505493, "operator": "ge"}, {"budget": 0.49, "threshold": -0.10747270286083221, "operator": "ge"}, {"budget": 0.5, "threshold": -0.11789597570896149, "operator": "ge"}, {"budget": 0.51, "threshold": -0.12134447693824768, "operator": "ge"}, {"budget": 0.52, "threshold": -0.1277044117450714, "operator": "ge"}, {"budget": 0.53, "threshold": -0.1319359689950943, "operator": "ge"}, {"budget": 0.54, "threshold": -0.13549524545669556, "operator": "ge"}, {"budget": 0.55, "threshold": -0.13887032866477966, "operator": "ge"}, {"budget": 0.56, "threshold": -0.1413278877735138, "operator": "ge"}, {"budget": 0.57, "threshold": -0.14473745226860046, "operator": "ge"}, {"budget": 0.58, "threshold": -0.14783068001270294, "operator": "ge"}, {"budget": 0.59, "threshold": -0.15258193016052246, "operator": "ge"}, {"budget": 0.6, "threshold": -0.15583351254463196, "operator": "ge"}, {"budget": 0.61, "threshold": -0.15766556560993195, "operator": "ge"}, {"budget": 0.62, "threshold": -0.15994460880756378, "operator": "ge"}, {"budget": 0.63, "threshold": -0.16162380576133728, "operator": "ge"}, {"budget": 0.64, "threshold": -0.16321274638175964, "operator": "ge"}, {"budget": 0.65, "threshold": -0.16430076956748962, "operator": "ge"}, {"budget": 0.66, "threshold": -0.16508600115776062, "operator": "ge"}, {"budget": 0.67, "threshold": -0.166667178273201, "operator": "ge"}, {"budget": 0.68, "threshold": -0.16762253642082214, "operator": "ge"}, {"budget": 0.69, "threshold": -0.16941075026988983, "operator": "ge"}, {"budget": 0.7, "threshold": -0.17062726616859436, "operator": "ge"}, {"budget": 0.71, "threshold": -0.1719721406698227, "operator": "ge"}, {"budget": 0.72, "threshold": -0.1726587861776352, "operator": "ge"}, {"budget": 0.73, "threshold": -0.17364980280399323, "operator": "ge"}, {"budget": 0.74, "threshold": -0.17486421763896942, "operator": "ge"}, {"budget": 0.75, "threshold": -0.17535844445228577, "operator": "ge"}, {"budget": 0.76, "threshold": -0.17632989585399628, "operator": "ge"}, {"budget": 0.77, "threshold": -0.17763325572013855, "operator": "ge"}, {"budget": 0.78, "threshold": -0.1785346269607544, "operator": "ge"}, {"budget": 0.79, "threshold": -0.17888763546943665, "operator": "ge"}, {"budget": 0.8, "threshold": -0.1798924058675766, "operator": "ge"}, {"budget": 0.81, "threshold": -0.1811179667711258, "operator": "ge"}, {"budget": 0.82, "threshold": -0.1813768446445465, "operator": "ge"}, {"budget": 0.83, "threshold": -0.18239717185497284, "operator": "ge"}, {"budget": 0.84, "threshold": -0.18321768939495087, "operator": "ge"}, {"budget": 0.85, "threshold": -0.18393081426620483, "operator": "ge"}, {"budget": 0.86, "threshold": -0.18491533398628235, "operator": "ge"}, {"budget": 0.87, "threshold": -0.18576973676681519, "operator": "ge"}, {"budget": 0.88, "threshold": -0.18724000453948975, "operator": "ge"}, {"budget": 0.89, "threshold": -0.18809263408184052, "operator": "ge"}, {"budget": 0.9, "threshold": -0.1892458200454712, "operator": "ge"}, {"budget": 0.91, "threshold": -0.1903112530708313, "operator": "ge"}, {"budget": 0.92, "threshold": -0.19162309169769287, "operator": "ge"}, {"budget": 0.93, "threshold": -0.19266730546951294, "operator": "ge"}, {"budget": 0.94, "threshold": -0.19528406858444214, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1962403804063797, "operator": "ge"}, {"budget": 0.96, "threshold": -0.20026125013828278, "operator": "ge"}, {"budget": 0.97, "threshold": -0.20587587356567383, "operator": "ge"}, {"budget": 0.98, "threshold": -0.21659140288829803, "operator": "ge"}, {"budget": 0.99, "threshold": -0.23178313672542572, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.507340074066676, 18.855784161490686, 19.71427248966948, 20.167639088417747, 20.595002153304225, 21.040736650890462, 21.479167236365956, 21.849899938715854, 22.248472559755612, 22.753253475303165, 23.83096569749333, 23.842503076515328, 23.791125122135576, 23.674014095066724, 23.644924476967823, 23.38849368412973, 23.526917844373088, 23.836085509472607, 24.143024032042725, 24.320852231369326, 24.445253022839232, 24.60347147192873, 24.895730706075533, 25.597740307679455, 26.1218435196501, 26.476894539761137, 26.931289539357678, 26.90484601258814, 26.480258490863974, 26.54721210157655, 26.673899802040506, 27.061116536398444, 27.606743532181078, 25.759846815314148, 26.581465919701213, 27.34442717612423, 27.441867224741305, 27.415526595117345, 27.57864138563227, 27.7372300766916, 27.994884059902486, 28.188131313131315, 28.485982194596133, 28.256655622710696, 28.34867397974194, 28.17897416669865, 28.14458315626086, 28.11170869560748, 27.936699499799715, 27.888429186042675, 28.286864248700166, 28.24141658925463, 28.260857044643124, 28.29837150955763, 28.243919268587202, 27.70076177052921, 27.63847905466402, 27.48425280067639, 27.331466903895084, 27.218949787739255, 26.95646980139329, 27.192282102461746, 27.15944407257535, 26.87588918261255, 26.380317367876916, 26.24106382806787, 26.1996393621817, 25.756644707479452, 25.70975850662266, 25.31881292360334, 25.112287518557718, 24.37582175231926, 24.11779789669855, 24.071662058185765, 24.002143341608225, 23.35760700401168, 23.23487225956846, 23.26888394978645, 23.226086602045683, 23.177221743398217, 22.987740100802135, 22.661273353097354, 22.622164657048373, 22.300521757634343, 22.181108333443557, 21.924441922952667, 21.69511959343618, 21.646891497400944, 21.603549605387464, 21.844046870591637, 21.651223774569186, 21.613741799688942, 21.392573756700354, 21.4122129142585, 21.296319970289794, 21.32126354739973, 21.208468483395443, 21.05038934455562, 21.029534816825997, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 22, 35, 49, 59, 71, 82, 91, 100, 109, 117, 122, 128, 132, 136, 141, 147, 153, 160, 167, 172, 176, 182, 188, 193, 198, 203, 207, 209, 211, 215, 221, 227, 230, 235, 241, 244, 245, 248, 253, 257, 260, 262, 267, 269, 271, 271, 271, 271, 271, 272, 272, 273, 274, 274, 274, 274, 274, 274, 274, 274, 275, 275, 275, 275, 275, 275, 275, 275, 275, 275, 275, 275, 275, 275, 275, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 278, 278, 278, 278, 279, 279, 280, 280, 281, 284, 286], "harmed": [0, 2, 3, 5, 8, 13, 15, 19, 21, 23, 24, 30, 36, 40, 50, 56, 61, 66, 69, 73, 78, 82, 87, 94, 98, 101, 107, 116, 121, 126, 127, 131, 136, 142, 149, 154, 161, 165, 167, 171, 175, 178, 181, 189, 196, 201, 208, 210, 212, 218, 221, 227, 230, 235, 239, 243, 248, 253, 259, 265, 268, 274, 277, 280, 288, 294, 300, 304, 310, 315, 320, 326, 332, 338, 347, 354, 362, 368, 376, 381, 388, 393, 400, 405, 411, 417, 424, 429, 436, 443, 448, 455, 460, 466, 474, 481, 488, 495, 503, 510, 518], "exact_score": [18.254072168965788, 18.507340074066676, 18.855784161490686, 19.71427248966948, 20.167639088417747, 20.595002153304225, 21.040736650890462, 21.479167236365956, 21.849899938715854, 22.248472559755612, 22.753253475303165, 23.83096569749333, 23.842503076515328, 23.791125122135576, 23.674014095066724, 23.644924476967823, 23.38849368412973, 23.526917844373088, 23.836085509472607, 24.143024032042725, 24.320852231369326, 24.445253022839232, 24.60347147192873, 24.895730706075533, 25.597740307679455, 26.1218435196501, 26.476894539761137, 26.931289539357678, 26.90484601258814, 26.480258490863974, 26.54721210157655, 26.673899802040506, 27.061116536398444, 27.606743532181078, 25.759846815314148, 26.581465919701213, 27.34442717612423, 27.441867224741305, 27.415526595117345, 27.57864138563227, 27.7372300766916, 27.994884059902486, 28.188131313131315, 28.485982194596133, 28.256655622710696, 28.34867397974194, 28.17897416669865, 28.14458315626086, 28.11170869560748, 27.936699499799715, 27.888429186042675, 28.286864248700166, 28.24141658925463, 28.260857044643124, 28.29837150955763, 28.243919268587202, 27.70076177052921, 27.63847905466402, 27.48425280067639, 27.331466903895084, 27.218949787739255, 26.95646980139329, 27.192282102461746, 27.15944407257535, 26.87588918261255, 26.380317367876916, 26.24106382806787, 26.1996393621817, 25.756644707479452, 25.70975850662266, 25.31881292360334, 25.112287518557718, 24.37582175231926, 24.11779789669855, 24.071662058185765, 24.002143341608225, 23.35760700401168, 23.23487225956846, 23.26888394978645, 23.226086602045683, 23.177221743398217, 22.987740100802135, 22.661273353097354, 22.622164657048373, 22.300521757634343, 22.181108333443557, 21.924441922952667, 21.69511959343618, 21.646891497400944, 21.603549605387464, 21.844046870591637, 21.651223774569186, 21.613741799688942, 21.392573756700354, 21.4122129142585, 21.296319970289794, 21.32126354739973, 21.208468483395443, 21.05038934455562, 21.029534816825997, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.697432249078282, "threshold_area100": 24.70293175823378, "threshold_area25": 22.700398036250498}, "test": {"score": [18.129624718318375, 18.21707961483475, 18.721180244454303, 18.919285346145145, 19.090808186466237, 19.249501328084147, 19.42466644464845, 19.927974230600302, 20.534112680483776, 20.871952777478118, 21.193442466413682, 21.366553127431843, 21.593705527140706, 21.757023331764415, 22.09389512679229, 22.196522218839966, 22.319121646103856, 22.513767840788244, 22.763204798822645, 22.654360370365378, 22.85202514810231, 23.279032349255502, 23.49484413189184, 23.359187261230822, 23.455263724069212, 23.561052885475576, 23.586451365419016, 23.588896304751902, 24.209914684462664, 25.066890777589947, 24.49497484614827, 24.713463003706902, 24.90333391088786, 25.03784585581488, 25.043118037005275, 25.024405005223223, 25.825213783843676, 25.93778878612314, 26.168861793533775, 26.246967711033175, 26.276454251803123, 26.3373458137996, 26.349742588294777, 26.35851498163025, 25.71359815614346, 25.684540687279878, 25.679293751221767, 25.59287048588157, 25.525064533599334, 25.40958875191963, 25.277729786852017, 25.202360032911486, 25.149509818619585, 25.08911396739476, 25.10460223719469, 25.077078942133284, 25.030295276604647, 25.022272287149207, 24.91186506429121, 24.832516119265595, 25.193217686131142, 25.126418608786814, 25.15377171153508, 25.053813284341313, 24.99067763967173, 24.95408242544281, 24.864161263620037, 24.683985942100474, 24.648772990983506, 24.34282226361879, 24.661220061303254, 24.537959728468824, 24.352631877736663, 24.302947482852545, 24.25978155615128, 24.235070484641067, 24.630600500574783, 24.248336601023006, 24.028921764331788, 23.856792218808934, 23.774490276263112, 24.053596593549965, 24.035542964827307, 23.914928219811262, 24.213785648767082, 24.11457651903887, 24.01790799738427, 23.83609323603414, 23.65812381221846, 23.505191216852335, 23.49144520778873, 23.40790912799467, 23.365760966711253, 23.57483086743712, 23.22273415640124, 23.264435418718605, 23.31417102943007, 23.210422161571067, 23.46617896254252, 22.94715204884871, 22.361115727328066], "calls": [0.0, 0.7323568575233023, 2.030625832223702, 3.029294274300932, 3.894806924101198, 4.327563249001331, 5.093209054593874, 6.824234354194407, 8.255659121171771, 9.52063914780293, 10.952063914780293, 11.51797603195739, 12.81624500665779, 13.615179760319574, 14.380825565912117, 15.179760319573901, 15.745672436750999, 16.67776298268975, 18.17576564580559, 19.507323568575234, 20.306258322237017, 21.271637816245008, 21.937416777629824, 23.40213049267643, 24.167776298268976, 25.233022636484687, 25.532623169107854, 26.065246338215715, 28.02929427430093, 29.19440745672437, 29.66045272969374, 31.391478029294273, 32.52330226364847, 33.55525965379494, 34.25432756324901, 34.9866844207723, 36.451398135818906, 37.28362183754994, 38.68175765645805, 39.41411451398136, 39.88015978695073, 40.645805592543276, 41.31158455392809, 42.80958721704394, 43.67509986684421, 44.40745672436751, 45.63914780292942, 46.37150466045273, 47.603195739014645, 48.23568575233023, 50.09986684420772, 50.632490013315575, 52.06391478029294, 53.39547270306259, 54.29427430093209, 55.15978695073236, 55.9254327563249, 57.02396804260985, 57.922769640479366, 59.35419440745673, 60.51930758988016, 61.35153129161118, 62.28362183754993, 63.182423435419445, 64.08122503328894, 64.88015978695073, 65.77896138482025, 67.11051930758988, 67.84287616511318, 69.57390146471371, 71.00532623169109, 72.33688415446072, 73.10252996005326, 74.06790945406125, 75.09986684420772, 75.56591211717709, 76.56458055925432, 77.92942743009321, 78.99467376830891, 79.36085219707057, 80.45938748335553, 81.89081225033289, 82.25699067909454, 83.58854860186418, 84.4207723035952, 85.08655126498003, 86.05193075898802, 86.81757656458056, 88.58189081225034, 89.24766977363515, 90.41278295605859, 91.34487350199734, 92.37683089214381, 93.04260985352862, 94.57390146471371, 94.84021304926765, 95.73901464713715, 96.70439414114514, 97.50332889480693, 98.26897470039947, 100.0], "rescued": [0, 40, 104, 144, 176, 197, 229, 286, 346, 390, 434, 451, 483, 502, 522, 542, 551, 571, 612, 643, 662, 690, 704, 735, 750, 771, 778, 788, 835, 863, 869, 906, 930, 953, 966, 984, 1013, 1026, 1054, 1064, 1069, 1078, 1086, 1102, 1115, 1118, 1123, 1127, 1130, 1132, 1134, 1137, 1139, 1141, 1142, 1144, 1144, 1146, 1147, 1147, 1149, 1149, 1151, 1152, 1153, 1153, 1154, 1154, 1154, 1154, 1157, 1159, 1159, 1159, 1161, 1161, 1163, 1163, 1163, 1163, 1163, 1165, 1165, 1165, 1166, 1167, 1168, 1168, 1171, 1171, 1174, 1175, 1177, 1178, 1179, 1180, 1182, 1184, 1189, 1195, 1198], "harmed": [0, 4, 27, 39, 51, 56, 68, 87, 103, 117, 132, 143, 168, 186, 200, 215, 217, 237, 273, 301, 317, 341, 353, 379, 396, 419, 425, 438, 474, 500, 511, 543, 561, 584, 604, 620, 650, 661, 684, 699, 706, 715, 721, 748, 770, 777, 795, 805, 831, 852, 886, 898, 921, 942, 957, 973, 987, 1012, 1034, 1062, 1084, 1099, 1117, 1139, 1163, 1178, 1199, 1229, 1245, 1285, 1333, 1367, 1382, 1404, 1433, 1445, 1466, 1503, 1532, 1543, 1569, 1603, 1613, 1642, 1661, 1686, 1711, 1732, 1780, 1800, 1831, 1852, 1883, 1904, 1959, 1966, 1990, 2018, 2039, 2059, 2109], "exact_score": [18.129624718318375, 18.251019987058363, 18.745095544272484, 18.940430030117206, 19.123672796961998, 19.399696105119236, 19.615880991696212, 19.98731120439293, 20.512475054387398, 20.7558439881183, 20.99364498742483, 21.19456726696391, 21.437627747561297, 21.63809237746043, 22.07972717712839, 22.18965193162359, 22.30417401931253, 22.52741270308192, 22.715671359514257, 22.534649264117974, 22.78759871046756, 23.183085715085006, 23.507091398887585, 23.26196506951125, 23.496941274226224, 23.549806935967638, 23.580275868741275, 24.191869783188412, 24.196004775001644, 24.978862312505818, 24.695618265364697, 24.8403472468891, 24.843926731927617, 24.95144394763672, 25.069378298765148, 25.024405005223223, 25.958366676689298, 25.885854101027366, 26.06114484855603, 26.227715597583067, 26.28787497394453, 26.326747716183508, 26.34591848764814, 26.361141265192973, 25.686076226382543, 25.668801683188935, 25.628734331326196, 25.589539425177342, 25.500803064983092, 25.34492963897405, 25.28066604741126, 25.18397481440946, 25.153670937128243, 25.055941406875604, 25.05482646957336, 25.10637256711034, 25.023936004559822, 25.024489037597675, 24.91186506429121, 24.85996979350649, 25.291039627622574, 25.16922306946428, 25.16290285989811, 25.087271263218007, 24.99809835413934, 24.946863103438783, 24.850589081973943, 24.690369811849106, 24.620317621324382, 24.42034346037359, 24.302598020257136, 24.66420356428676, 24.561304504158237, 24.358473205449815, 24.30717245963639, 24.266665351283923, 24.71776402609294, 24.543682107251392, 24.243110023977344, 24.028921764331788, 23.816105598290996, 24.091301246171724, 24.046352429534544, 23.99137586702535, 24.292314905055047, 24.11916358148408, 24.039210845400817, 23.829605663220075, 23.728539732246148, 23.572920520687184, 23.445522786312985, 23.467285422038415, 23.362164484479244, 23.21693392042396, 23.40928701427766, 23.25887864302447, 23.284685591610543, 23.588346066220687, 23.392156208617813, 22.750783673952025, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.808287068339816, "threshold_area100": 23.796426789543553, "threshold_area25": 21.26736599266742}, "selected_budget": 43, "selected": {"val": {"score": 28.485982194596133, "calls": 42.93865905848788, "rescued": 262, "harmed": 189}, "test": {"score": 26.35851498163025, "calls": 42.80958721704394, "rescued": 1102, "harmed": 748}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.4623864889144897, "operator": "ge"}, {"budget": 0.02, "threshold": 1.2923039197921753, "operator": "ge"}, {"budget": 0.03, "threshold": 1.1672215461730957, "operator": "ge"}, {"budget": 0.04, "threshold": 1.1367366313934326, "operator": "ge"}, {"budget": 0.05, "threshold": 1.0582969188690186, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0046074390411377, "operator": "ge"}, {"budget": 0.07, "threshold": 0.9207277297973633, "operator": "ge"}, {"budget": 0.08, "threshold": 0.8711613416671753, "operator": "ge"}, {"budget": 0.09, "threshold": 0.6611010432243347, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5539144277572632, "operator": "ge"}, {"budget": 0.11, "threshold": 0.46589910984039307, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3914957642555237, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3448988199234009, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3113781213760376, "operator": "ge"}, {"budget": 0.15, "threshold": 0.26765671372413635, "operator": "ge"}, {"budget": 0.16, "threshold": 0.25804075598716736, "operator": "ge"}, {"budget": 0.17, "threshold": 0.23669204115867615, "operator": "ge"}, {"budget": 0.18, "threshold": 0.22532635927200317, "operator": "ge"}, {"budget": 0.19, "threshold": 0.20351237058639526, "operator": "ge"}, {"budget": 0.2, "threshold": 0.18706822395324707, "operator": "ge"}, {"budget": 0.21, "threshold": 0.16976594924926758, "operator": "ge"}, {"budget": 0.22, "threshold": 0.15721717476844788, "operator": "ge"}, {"budget": 0.23, "threshold": 0.13707736134529114, "operator": "ge"}, {"budget": 0.24, "threshold": 0.12189997732639313, "operator": "ge"}, {"budget": 0.25, "threshold": 0.10716450959444046, "operator": "ge"}, {"budget": 0.26, "threshold": 0.08994907140731812, "operator": "ge"}, {"budget": 0.27, "threshold": 0.08102013170719147, "operator": "ge"}, {"budget": 0.28, "threshold": 0.06308788061141968, "operator": "ge"}, {"budget": 0.29, "threshold": 0.03139645233750343, "operator": "ge"}, {"budget": 0.3, "threshold": 0.016538113355636597, "operator": "ge"}, {"budget": 0.31, "threshold": -0.008841484785079956, "operator": "ge"}, {"budget": 0.32, "threshold": -0.026164531707763672, "operator": "ge"}, {"budget": 0.33, "threshold": -0.05319476127624512, "operator": "ge"}, {"budget": 0.34, "threshold": -0.0633169561624527, "operator": "ge"}, {"budget": 0.35, "threshold": -0.09340846538543701, "operator": "ge"}, {"budget": 0.36, "threshold": -0.1373755931854248, "operator": "ge"}, {"budget": 0.37, "threshold": -0.1721809208393097, "operator": "ge"}, {"budget": 0.38, "threshold": -0.24941325187683105, "operator": "ge"}, {"budget": 0.39, "threshold": -0.28511571884155273, "operator": "ge"}, {"budget": 0.4, "threshold": -0.33048200607299805, "operator": "ge"}, {"budget": 0.41, "threshold": -0.421067476272583, "operator": "ge"}, {"budget": 0.42, "threshold": -0.4979217052459717, "operator": "ge"}, {"budget": 0.43, "threshold": -0.6041804552078247, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6482442617416382, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9231481552124023, "operator": "ge"}, {"budget": 0.46, "threshold": -1.5683776140213013, "operator": "ge"}, {"budget": 0.47, "threshold": -1.7568371295928955, "operator": "ge"}, {"budget": 0.48, "threshold": -2.3067708015441895, "operator": "ge"}, {"budget": 0.49, "threshold": -2.5759613513946533, "operator": "ge"}, {"budget": 0.5, "threshold": -2.676337480545044, "operator": "ge"}, {"budget": 0.51, "threshold": -2.738311290740967, "operator": "ge"}, {"budget": 0.52, "threshold": -3.8040764331817627, "operator": "ge"}, {"budget": 0.53, "threshold": -3.902332067489624, "operator": "ge"}, {"budget": 0.54, "threshold": -4.021499156951904, "operator": "ge"}, {"budget": 0.55, "threshold": -4.081480026245117, "operator": "ge"}, {"budget": 0.56, "threshold": -4.150885581970215, "operator": "ge"}, {"budget": 0.57, "threshold": -4.205293655395508, "operator": "ge"}, {"budget": 0.58, "threshold": -4.252251148223877, "operator": "ge"}, {"budget": 0.59, "threshold": -4.299348831176758, "operator": "ge"}, {"budget": 0.6, "threshold": -4.336150169372559, "operator": "ge"}, {"budget": 0.61, "threshold": -4.391183376312256, "operator": "ge"}, {"budget": 0.62, "threshold": -4.42464542388916, "operator": "ge"}, {"budget": 0.63, "threshold": -4.484387397766113, "operator": "ge"}, {"budget": 0.64, "threshold": -4.554937839508057, "operator": "ge"}, {"budget": 0.65, "threshold": -4.578028678894043, "operator": "ge"}, {"budget": 0.66, "threshold": -4.6222028732299805, "operator": "ge"}, {"budget": 0.67, "threshold": -4.658541679382324, "operator": "ge"}, {"budget": 0.68, "threshold": -4.688871383666992, "operator": "ge"}, {"budget": 0.69, "threshold": -4.712714672088623, "operator": "ge"}, {"budget": 0.7, "threshold": -4.7230143547058105, "operator": "ge"}, {"budget": 0.71, "threshold": -4.743936538696289, "operator": "ge"}, {"budget": 0.72, "threshold": -4.769416332244873, "operator": "ge"}, {"budget": 0.73, "threshold": -4.815711498260498, "operator": "ge"}, {"budget": 0.74, "threshold": -4.838210582733154, "operator": "ge"}, {"budget": 0.75, "threshold": -4.868735313415527, "operator": "ge"}, {"budget": 0.76, "threshold": -4.8883867263793945, "operator": "ge"}, {"budget": 0.77, "threshold": -4.9129486083984375, "operator": "ge"}, {"budget": 0.78, "threshold": -4.933052062988281, "operator": "ge"}, {"budget": 0.79, "threshold": -4.962169647216797, "operator": "ge"}, {"budget": 0.8, "threshold": -4.992321014404297, "operator": "ge"}, {"budget": 0.81, "threshold": -5.043473243713379, "operator": "ge"}, {"budget": 0.82, "threshold": -12.404533386230469, "operator": "ge"}, {"budget": 0.83, "threshold": -12.528003692626953, "operator": "ge"}, {"budget": 0.84, "threshold": -12.589437484741211, "operator": "ge"}, {"budget": 0.85, "threshold": -12.660003662109375, "operator": "ge"}, {"budget": 0.86, "threshold": -12.715003967285156, "operator": "ge"}, {"budget": 0.87, "threshold": -12.766263961791992, "operator": "ge"}, {"budget": 0.88, "threshold": -12.818546295166016, "operator": "ge"}, {"budget": 0.89, "threshold": -12.894281387329102, "operator": "ge"}, {"budget": 0.9, "threshold": -12.958368301391602, "operator": "ge"}, {"budget": 0.91, "threshold": -12.988161087036133, "operator": "ge"}, {"budget": 0.92, "threshold": -13.018159866333008, "operator": "ge"}, {"budget": 0.93, "threshold": -13.07958984375, "operator": "ge"}, {"budget": 0.94, "threshold": -13.113737106323242, "operator": "ge"}, {"budget": 0.95, "threshold": -13.135679244995117, "operator": "ge"}, {"budget": 0.96, "threshold": -13.161890029907227, "operator": "ge"}, {"budget": 0.97, "threshold": -13.187295913696289, "operator": "ge"}, {"budget": 0.98, "threshold": -13.219831466674805, "operator": "ge"}, {"budget": 0.99, "threshold": -13.232305526733398, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.506283263452552, 18.893401783956506, 19.127073450156516, 19.56160988769684, 20.46185290858484, 20.78383033553796, 21.253090387499633, 21.736300979722028, 22.264521193092623, 22.529420613010547, 22.474643314485704, 23.707817624034696, 23.75642442663719, 23.799224850821723, 23.626456512692467, 23.4155843179736, 23.50579935635216, 23.644024361542638, 23.93547972495341, 24.220914722749104, 24.37538273269236, 24.30193811087871, 24.884156486805693, 25.3123973727422, 25.926557324363902, 26.035903980133824, 26.39020156029502, 26.90484601258814, 26.674289333623243, 26.319730034175304, 26.673899802040506, 27.598309549529066, 27.689126670823526, 26.535595424209347, 27.811049073454708, 27.476798580469787, 27.613770685579198, 27.547518718497184, 27.93295827204315, 28.309833233320646, 28.3776331438833, 28.073871697353482, 28.18343957246797, 28.152399171050536, 28.892789420049102, 28.64514777488093, 28.58528386101916, 28.493651037058466, 28.272633431375176, 27.948090897906464, 27.48433292404308, 27.742622015067838, 27.66608377455028, 27.816588330247356, 27.15288122768438, 26.962766037849928, 26.90693107392673, 26.829883288362705, 26.67276849976433, 26.592695074709464, 26.455191254750194, 25.882796115492212, 25.708896988922614, 25.278280039045352, 25.112861721152314, 24.967070849344992, 24.46961326340675, 24.40637639313136, 24.02913807198856, 23.87777600346788, 23.977579237291152, 23.841987219667544, 23.786189239039164, 23.106862580861318, 23.046137605563835, 22.999174115818114, 22.941908835247247, 23.00528856638712, 22.631768095280524, 22.580242629860948, 22.2993982603681, 22.250712250712247, 22.238502238502235, 22.20298069562775, 22.176975900380153, 21.94659131831907, 21.84429416868067, 21.808048469832066, 21.802761660421545, 21.627805813166873, 21.537687546723692, 21.499915485188446, 21.46410691279823, 21.43090266938856, 21.2106630386647, 21.178556253292054, 20.966829447266143, 20.79993311780587, 20.705583275679228, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 10, 22, 33, 44, 56, 69, 79, 89, 100, 107, 114, 121, 125, 130, 134, 139, 143, 151, 156, 161, 168, 173, 180, 186, 193, 197, 203, 207, 210, 210, 215, 221, 227, 231, 239, 242, 245, 251, 253, 256, 257, 261, 265, 270, 273, 274, 275, 277, 280, 280, 281, 283, 283, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 285, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 2, 3, 5, 7, 9, 12, 17, 20, 23, 24, 28, 30, 37, 40, 46, 52, 62, 68, 70, 72, 78, 84, 89, 96, 100, 104, 112, 121, 125, 127, 131, 136, 141, 150, 157, 162, 167, 174, 181, 183, 184, 190, 196, 202, 205, 209, 215, 220, 229, 236, 244, 249, 254, 262, 269, 276, 280, 286, 292, 298, 303, 308, 316, 324, 331, 337, 344, 350, 356, 364, 367, 373, 379, 385, 392, 397, 404, 410, 417, 423, 430, 436, 438, 443, 447, 451, 455, 460, 461, 466, 469, 474, 480, 485, 490, 495, 500, 505, 510, 518], "exact_score": [18.254072168965788, 18.506283263452552, 18.893401783956506, 19.127073450156516, 19.56160988769684, 20.46185290858484, 20.78383033553796, 21.253090387499633, 21.736300979722028, 22.264521193092623, 22.529420613010547, 22.474643314485704, 23.707817624034696, 23.75642442663719, 23.799224850821723, 23.626456512692467, 23.4155843179736, 23.50579935635216, 23.644024361542638, 23.93547972495341, 24.220914722749104, 24.37538273269236, 24.30193811087871, 24.884156486805693, 25.3123973727422, 25.926557324363902, 26.035903980133824, 26.39020156029502, 26.90484601258814, 26.674289333623243, 26.319730034175304, 26.673899802040506, 27.598309549529066, 27.689126670823526, 26.535595424209347, 27.811049073454708, 27.476798580469787, 27.613770685579198, 27.547518718497184, 27.93295827204315, 28.309833233320646, 28.3776331438833, 28.073871697353482, 28.18343957246797, 28.152399171050536, 28.892789420049102, 28.64514777488093, 28.58528386101916, 28.493651037058466, 28.272633431375176, 27.948090897906464, 27.48433292404308, 27.742622015067838, 27.66608377455028, 27.816588330247356, 27.15288122768438, 26.962766037849928, 26.90693107392673, 26.829883288362705, 26.67276849976433, 26.592695074709464, 26.455191254750194, 25.882796115492212, 25.708896988922614, 25.278280039045352, 25.112861721152314, 24.967070849344992, 24.46961326340675, 24.40637639313136, 24.02913807198856, 23.87777600346788, 23.977579237291152, 23.841987219667544, 23.786189239039164, 23.106862580861318, 23.046137605563835, 22.999174115818114, 22.941908835247247, 23.00528856638712, 22.631768095280524, 22.580242629860948, 22.2993982603681, 22.250712250712247, 22.238502238502235, 22.20298069562775, 22.176975900380153, 21.94659131831907, 21.84429416868067, 21.808048469832066, 21.802761660421545, 21.627805813166873, 21.537687546723692, 21.499915485188446, 21.46410691279823, 21.43090266938856, 21.2106630386647, 21.178556253292054, 20.966829447266143, 20.79993311780587, 20.705583275679228, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.409959183607384, "threshold_area100": 24.415279410913033, "threshold_area25": 22.58579190074036}, "test": {"score": [18.129624718318375, 18.570904344414803, 18.779255200038282, 19.09864151823461, 19.20734540299758, 19.475069143412156, 19.837989212103274, 20.28195725016488, 20.530074551387013, 20.93454476052062, 21.02779219051675, 21.182942766592312, 21.35102782456797, 21.4360795765741, 21.61314402255473, 22.182157114963953, 22.227209052249652, 22.331005060609517, 22.641297200729486, 22.50070825876328, 22.713411838392222, 23.08744388538438, 23.386027974321134, 23.94157783455648, 23.513740264744083, 23.435986874888354, 23.55704720578843, 23.6258240556712, 23.98968427782721, 24.39462459604821, 25.06938914409475, 24.7734083702102, 24.980488508286786, 25.073103226374304, 25.162900386245084, 25.793894858417676, 26.05114191480009, 26.15321808739258, 26.264797827486547, 26.370735626385972, 26.424282501975483, 26.464940356790112, 25.874760517408912, 25.84696243665134, 25.92058131153679, 25.941740690214736, 26.030736437409434, 26.029737553440523, 26.295228183049403, 25.72200387905907, 25.386634423436195, 25.019626780649862, 24.11545671715793, 24.115797235009374, 24.043640946459206, 24.008185837069846, 23.925535696294, 23.84281346412346, 23.785950292074666, 23.645498172040437, 23.510497652410816, 23.954065377719225, 23.88707984698453, 24.280751785169233, 24.24109399849569, 24.158902097234993, 23.963308522926084, 23.865523111267734, 23.67713656631114, 23.684979727831497, 23.67640003999048, 23.579011759201283, 23.469198983078154, 23.340243778434388, 23.35678210383488, 23.25439683293001, 23.146888017319778, 23.593433888922586, 23.542628137302945, 23.269769273249103, 23.053745445980336, 23.251806859710864, 22.832349288090647, 22.782159720293873, 22.671373868449788, 22.639343846596493, 22.594438551033154, 22.630728657375858, 22.513638531427866, 22.355481034754437, 22.230745702266972, 22.217052387867625, 22.19322304132457, 22.473448583433893, 22.41196683520389, 22.394643379757824, 22.346403644042976, 22.267038530849774, 22.213383458998493, 22.562481926615614, 22.361115727328066], "calls": [0.0, 1.3315579227696404, 2.330226364846871, 3.861517976031957, 4.327563249001331, 5.525965379494008, 6.55792276964048, 8.022636484687084, 8.88814913448735, 10.219707057256992, 10.685752330226366, 11.418109187749668, 12.450066577896138, 13.11584553928096, 13.914780292942744, 15.912117177097205, 16.278295605858855, 17.310252996005325, 18.07589880159787, 19.640479360852197, 20.70572569906791, 21.604527296937416, 22.436750998668444, 24.00133155792277, 24.833555259653796, 25.499334221038616, 26.098535286284953, 26.2982689747004, 27.097203728362185, 29.027962716378163, 29.69374167776298, 31.25832223701731, 32.58988015978695, 33.72170439414114, 34.0878828229028, 35.21970705725699, 36.884154460719046, 37.81624500665779, 39.18109187749667, 39.74700399467377, 40.279627163781626, 41.27829560585886, 41.944074567243675, 42.60985352862849, 42.876165113182424, 44.00798934753662, 45.572569906790946, 45.97203728362184, 47.33688415446072, 48.83488681757657, 49.63382157123835, 50.09986684420772, 51.797603195739015, 52.529960053262315, 53.76165113182424, 54.89347536617842, 56.0918774966711, 57.75632490013316, 58.82157123834887, 60.153129161118514, 61.051930758988014, 62.4167776298269, 63.215712383488686, 64.84687083888149, 66.67776298268974, 67.31025299600533, 68.47536617842877, 69.70705725699068, 70.5392809587217, 71.20505992010652, 71.43808255659121, 72.03728362183755, 72.96937416777631, 74.36750998668442, 75.06657789613848, 76.23169107856191, 77.29693741677764, 78.16245006657789, 78.76165113182424, 79.49400798934754, 80.42609853528629, 81.22503328894807, 82.39014647137151, 83.288948069241, 84.0878828229028, 85.05326231691079, 86.01864181091877, 86.55126498002663, 87.28362183754993, 88.84820239680427, 90.1797603195739, 90.84553928095872, 91.6444740346205, 93.14247669773636, 94.2410119840213, 94.74034620505992, 95.80559254327564, 96.90412782956058, 98.06924101198402, 98.36884154460719, 100.0], "rescued": [0, 67, 104, 161, 180, 225, 265, 323, 356, 404, 420, 444, 465, 478, 498, 550, 559, 580, 597, 637, 664, 686, 708, 743, 758, 774, 784, 788, 808, 854, 871, 904, 933, 958, 968, 995, 1030, 1049, 1067, 1076, 1083, 1100, 1108, 1117, 1121, 1135, 1149, 1152, 1159, 1163, 1167, 1167, 1171, 1172, 1173, 1174, 1176, 1177, 1178, 1178, 1178, 1181, 1182, 1184, 1186, 1186, 1186, 1186, 1186, 1187, 1187, 1188, 1188, 1189, 1190, 1192, 1192, 1194, 1194, 1194, 1194, 1195, 1195, 1195, 1195, 1195, 1195, 1196, 1196, 1196, 1196, 1196, 1196, 1197, 1197, 1197, 1197, 1197, 1197, 1198, 1198], "harmed": [0, 8, 16, 30, 36, 51, 68, 89, 102, 119, 126, 137, 153, 164, 180, 219, 225, 243, 249, 286, 312, 331, 349, 377, 397, 419, 431, 434, 451, 491, 507, 541, 564, 593, 602, 628, 662, 681, 700, 711, 719, 738, 750, 766, 768, 790, 818, 824, 848, 884, 911, 931, 978, 995, 1023, 1046, 1072, 1108, 1135, 1169, 1193, 1232, 1251, 1293, 1337, 1354, 1390, 1421, 1444, 1462, 1466, 1481, 1501, 1543, 1562, 1593, 1623, 1644, 1661, 1678, 1709, 1725, 1752, 1769, 1790, 1811, 1836, 1844, 1857, 1892, 1920, 1928, 1943, 1969, 1987, 1998, 2019, 2041, 2071, 2077, 2109], "exact_score": [18.129624718318375, 18.30134767632201, 18.711697495914056, 18.890343154054936, 19.13842779383719, 19.38536460538497, 19.512825638242987, 19.94535222954012, 20.28195725016488, 20.554425003625887, 20.901013903216096, 21.086241013500835, 21.317900870482795, 21.401246062047523, 21.631739509811993, 22.0644641454873, 22.179632589232014, 22.3511373317563, 22.609964824182214, 22.43581593239434, 22.595387254303034, 23.00091075333889, 23.237539099572707, 23.601175535515363, 23.94157783455648, 23.49778034628898, 23.548585621573427, 23.9899378256146, 24.497270135821243, 24.38490739856637, 24.61446787534576, 24.801623828902073, 24.8534618396016, 24.98382074253309, 25.125141094673225, 25.77718768746861, 25.810654444844744, 26.144902105332378, 26.136781282561337, 26.22991053999285, 26.3849806264505, 26.402554666028944, 25.874760517408912, 25.952258925110954, 25.945330858676492, 25.968557851725176, 26.029737553440523, 26.463873481364658, 26.026086822918543, 25.661562846591636, 25.078885780063587, 24.309711114085218, 24.112143419530252, 24.081430313133414, 24.0294971980611, 23.976661990422876, 23.925535696294, 23.85890310632198, 23.819377441590696, 23.747407447943747, 23.661479983383863, 23.516529536196856, 23.477443808312742, 23.904521659195176, 24.368810622297545, 24.271974235518957, 24.32225177827981, 24.197267258850825, 24.02173243550507, 23.917278356030746, 23.84571157205094, 23.70517529546757, 23.581667978715124, 23.469198983078154, 23.370768778010394, 23.35867154864042, 23.28488550226532, 23.214869206154397, 23.115147121562167, 23.350274092814484, 23.167944843843554, 23.33631723453325, 22.843468854544604, 22.810700147866566, 22.67464693578146, 22.641398054200582, 22.59645117304836, 22.57007933180017, 22.497136089898746, 22.351105730913893, 22.257527135083134, 22.212817970523453, 22.186741360252093, 22.47980240570598, 22.439028373488394, 22.3880105627249, 22.34209413536326, 22.26532285088291, 22.21751898809314, 22.484351121976843, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.301423932505045, "threshold_area100": 23.33320619658074, "threshold_area25": 21.23624721677769}, "selected_budget": 45, "selected": {"val": {"score": 28.892789420049102, "calls": 44.935805991440795, "rescued": 273, "harmed": 205}, "test": {"score": 25.941740690214736, "calls": 44.00798934753662, "rescued": 1135, "harmed": 790}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.7890532612800598, "operator": "ge"}, {"budget": 0.02, "threshold": 0.7171701192855835, "operator": "ge"}, {"budget": 0.03, "threshold": 0.6855732202529907, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6316301822662354, "operator": "ge"}, {"budget": 0.05, "threshold": 0.6131452322006226, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5257330536842346, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4427303373813629, "operator": "ge"}, {"budget": 0.08, "threshold": 0.38015851378440857, "operator": "ge"}, {"budget": 0.09, "threshold": 0.3521468937397003, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3092775046825409, "operator": "ge"}, {"budget": 0.11, "threshold": 0.27603957056999207, "operator": "ge"}, {"budget": 0.12, "threshold": 0.24671322107315063, "operator": "ge"}, {"budget": 0.13, "threshold": 0.22230368852615356, "operator": "ge"}, {"budget": 0.14, "threshold": 0.2149742841720581, "operator": "ge"}, {"budget": 0.15, "threshold": 0.20480141043663025, "operator": "ge"}, {"budget": 0.16, "threshold": 0.1877152919769287, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1771550178527832, "operator": "ge"}, {"budget": 0.18, "threshold": 0.16028356552124023, "operator": "ge"}, {"budget": 0.19, "threshold": 0.1439853012561798, "operator": "ge"}, {"budget": 0.2, "threshold": 0.1304306834936142, "operator": "ge"}, {"budget": 0.21, "threshold": 0.12157046794891357, "operator": "ge"}, {"budget": 0.22, "threshold": 0.11072897911071777, "operator": "ge"}, {"budget": 0.23, "threshold": 0.09776708483695984, "operator": "ge"}, {"budget": 0.24, "threshold": 0.08681125938892365, "operator": "ge"}, {"budget": 0.25, "threshold": 0.07432542741298676, "operator": "ge"}, {"budget": 0.26, "threshold": 0.0641661286354065, "operator": "ge"}, {"budget": 0.27, "threshold": 0.05644538998603821, "operator": "ge"}, {"budget": 0.28, "threshold": 0.025620728731155396, "operator": "ge"}, {"budget": 0.29, "threshold": 0.014561295509338379, "operator": "ge"}, {"budget": 0.3, "threshold": 0.009541928768157959, "operator": "ge"}, {"budget": 0.31, "threshold": -0.006543219089508057, "operator": "ge"}, {"budget": 0.32, "threshold": -0.015906602144241333, "operator": "ge"}, {"budget": 0.33, "threshold": -0.028791576623916626, "operator": "ge"}, {"budget": 0.34, "threshold": -0.04014578461647034, "operator": "ge"}, {"budget": 0.35, "threshold": -0.056201785802841187, "operator": "ge"}, {"budget": 0.36, "threshold": -0.06324043869972229, "operator": "ge"}, {"budget": 0.37, "threshold": -0.07725480198860168, "operator": "ge"}, {"budget": 0.38, "threshold": -0.08492535352706909, "operator": "ge"}, {"budget": 0.39, "threshold": -0.11314928531646729, "operator": "ge"}, {"budget": 0.4, "threshold": -0.1392759531736374, "operator": "ge"}, {"budget": 0.41, "threshold": -0.16216781735420227, "operator": "ge"}, {"budget": 0.42, "threshold": -0.1976059079170227, "operator": "ge"}, {"budget": 0.43, "threshold": -0.27371928095817566, "operator": "ge"}, {"budget": 0.44, "threshold": -0.36921703815460205, "operator": "ge"}, {"budget": 0.45, "threshold": -0.5111805200576782, "operator": "ge"}, {"budget": 0.46, "threshold": -0.9813427925109863, "operator": "ge"}, {"budget": 0.47, "threshold": -1.4441677331924438, "operator": "ge"}, {"budget": 0.48, "threshold": -1.7967774868011475, "operator": "ge"}, {"budget": 0.49, "threshold": -1.9836251735687256, "operator": "ge"}, {"budget": 0.5, "threshold": -2.126303195953369, "operator": "ge"}, {"budget": 0.51, "threshold": -2.256047487258911, "operator": "ge"}, {"budget": 0.52, "threshold": -3.3274378776550293, "operator": "ge"}, {"budget": 0.53, "threshold": -3.4637012481689453, "operator": "ge"}, {"budget": 0.54, "threshold": -3.5167157649993896, "operator": "ge"}, {"budget": 0.55, "threshold": -3.574604034423828, "operator": "ge"}, {"budget": 0.56, "threshold": -3.5986557006835938, "operator": "ge"}, {"budget": 0.57, "threshold": -3.6478846073150635, "operator": "ge"}, {"budget": 0.58, "threshold": -3.7034361362457275, "operator": "ge"}, {"budget": 0.59, "threshold": -3.719276189804077, "operator": "ge"}, {"budget": 0.6, "threshold": -3.752640724182129, "operator": "ge"}, {"budget": 0.61, "threshold": -3.7792978286743164, "operator": "ge"}, {"budget": 0.62, "threshold": -3.810351848602295, "operator": "ge"}, {"budget": 0.63, "threshold": -3.844977855682373, "operator": "ge"}, {"budget": 0.64, "threshold": -3.8606784343719482, "operator": "ge"}, {"budget": 0.65, "threshold": -3.878148078918457, "operator": "ge"}, {"budget": 0.66, "threshold": -3.908081531524658, "operator": "ge"}, {"budget": 0.67, "threshold": -3.9279723167419434, "operator": "ge"}, {"budget": 0.68, "threshold": -3.958385944366455, "operator": "ge"}, {"budget": 0.69, "threshold": -3.9749960899353027, "operator": "ge"}, {"budget": 0.7, "threshold": -3.9898364543914795, "operator": "ge"}, {"budget": 0.71, "threshold": -4.007785797119141, "operator": "ge"}, {"budget": 0.72, "threshold": -4.023406505584717, "operator": "ge"}, {"budget": 0.73, "threshold": -4.044723033905029, "operator": "ge"}, {"budget": 0.74, "threshold": -4.062218189239502, "operator": "ge"}, {"budget": 0.75, "threshold": -4.083617687225342, "operator": "ge"}, {"budget": 0.76, "threshold": -4.103228569030762, "operator": "ge"}, {"budget": 0.77, "threshold": -4.12237548828125, "operator": "ge"}, {"budget": 0.78, "threshold": -4.144104957580566, "operator": "ge"}, {"budget": 0.79, "threshold": -4.168252468109131, "operator": "ge"}, {"budget": 0.8, "threshold": -4.2098846435546875, "operator": "ge"}, {"budget": 0.81, "threshold": -4.275156497955322, "operator": "ge"}, {"budget": 0.82, "threshold": -10.578392028808594, "operator": "ge"}, {"budget": 0.83, "threshold": -10.651162147521973, "operator": "ge"}, {"budget": 0.84, "threshold": -10.687799453735352, "operator": "ge"}, {"budget": 0.85, "threshold": -10.712631225585938, "operator": "ge"}, {"budget": 0.86, "threshold": -10.747430801391602, "operator": "ge"}, {"budget": 0.87, "threshold": -10.762008666992188, "operator": "ge"}, {"budget": 0.88, "threshold": -10.774459838867188, "operator": "ge"}, {"budget": 0.89, "threshold": -10.790678024291992, "operator": "ge"}, {"budget": 0.9, "threshold": -10.801925659179688, "operator": "ge"}, {"budget": 0.91, "threshold": -10.81551742553711, "operator": "ge"}, {"budget": 0.92, "threshold": -10.833667755126953, "operator": "ge"}, {"budget": 0.93, "threshold": -10.854703903198242, "operator": "ge"}, {"budget": 0.94, "threshold": -10.873933792114258, "operator": "ge"}, {"budget": 0.95, "threshold": -10.896025657653809, "operator": "ge"}, {"budget": 0.96, "threshold": -10.913753509521484, "operator": "ge"}, {"budget": 0.97, "threshold": -10.937765121459961, "operator": "ge"}, {"budget": 0.98, "threshold": -10.994222640991211, "operator": "ge"}, {"budget": 0.99, "threshold": -11.05534553527832, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.49927175050685, 18.777955830985853, 19.738682784282798, 20.14463942082295, 20.65747575625995, 21.05195956610476, 21.406703468250015, 21.841305754173412, 23.34609712411596, 23.728773398277518, 23.98138776350711, 24.193764462306664, 23.947840463433337, 23.463041401046794, 23.600001090019237, 23.491058403509765, 23.6190018448083, 23.765317850683704, 24.045198563366633, 24.306463725706, 24.231370057058125, 24.79286541132267, 25.097343609301337, 25.623799708351857, 26.10047904165551, 26.05486498929122, 26.931289539357678, 26.90484601258814, 26.656673361808796, 26.52986303842596, 26.673899802040506, 27.061116536398444, 27.606743532181078, 25.759846815314148, 26.179939549247656, 26.866115716235562, 27.373585105257593, 27.415526595117345, 27.558027079303677, 27.696492121871213, 27.77717833501977, 28.187354312354312, 28.48714658277104, 28.204678362573098, 28.250905774248718, 28.64514777488093, 28.55198382771913, 28.493651037058466, 28.174390968508618, 28.118246807644326, 28.086742562597294, 27.745335422304517, 27.603045350371552, 27.49992470106322, 27.57299316122846, 26.989405261656568, 26.798193482289, 26.96908599901329, 26.815715915215787, 26.654320341334447, 26.579242440169594, 26.07634816336831, 25.99316410916214, 25.843898158798368, 25.721531139487798, 25.43344813081655, 24.92926483358251, 24.435438344430523, 24.358643165153108, 24.21369459522444, 24.14722041116872, 23.776916072513558, 23.306285066798253, 22.91980419001982, 23.065640754875954, 23.009126788467945, 22.954267054461607, 22.437847199806455, 22.38826309635558, 22.355966905585227, 22.308619426490953, 22.27776177214379, 22.244573515291748, 22.00993867862176, 21.99812358483945, 21.892526807964625, 21.86706393817819, 21.637317570399723, 21.541817068992888, 21.42564955347449, 21.396463798797388, 21.357142857142858, 21.328575408134363, 21.283237775408022, 21.261483513036307, 21.243167236735278, 21.04204348703488, 20.937494317220118, 20.84283612215802, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 23, 38, 47, 59, 73, 81, 90, 100, 107, 116, 123, 127, 132, 137, 141, 148, 154, 160, 166, 171, 177, 183, 189, 192, 198, 203, 207, 209, 210, 215, 221, 227, 230, 233, 240, 244, 245, 248, 252, 255, 260, 263, 267, 271, 274, 275, 277, 279, 282, 283, 283, 283, 283, 284, 284, 284, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 2, 5, 7, 10, 13, 17, 19, 20, 23, 26, 32, 35, 39, 46, 55, 62, 66, 69, 73, 78, 83, 87, 94, 99, 101, 107, 116, 121, 125, 127, 131, 136, 142, 149, 153, 160, 165, 167, 172, 176, 178, 181, 190, 195, 204, 209, 217, 220, 229, 236, 242, 249, 258, 265, 272, 279, 287, 292, 297, 303, 309, 315, 322, 328, 332, 341, 345, 352, 359, 366, 373, 379, 383, 389, 394, 400, 406, 413, 419, 423, 429, 433, 437, 441, 443, 448, 452, 457, 460, 466, 471, 477, 481, 488, 492, 496, 500, 506, 510, 518], "exact_score": [18.254072168965788, 18.49927175050685, 18.777955830985853, 19.738682784282798, 20.14463942082295, 20.65747575625995, 21.05195956610476, 21.406703468250015, 21.841305754173412, 23.34609712411596, 23.728773398277518, 23.98138776350711, 24.193764462306664, 23.947840463433337, 23.463041401046794, 23.600001090019237, 23.491058403509765, 23.6190018448083, 23.765317850683704, 24.045198563366633, 24.306463725706, 24.231370057058125, 24.79286541132267, 25.097343609301337, 25.623799708351857, 26.10047904165551, 26.05486498929122, 26.931289539357678, 26.90484601258814, 26.656673361808796, 26.52986303842596, 26.673899802040506, 27.061116536398444, 27.606743532181078, 25.759846815314148, 26.179939549247656, 26.866115716235562, 27.373585105257593, 27.415526595117345, 27.558027079303677, 27.696492121871213, 27.77717833501977, 28.187354312354312, 28.48714658277104, 28.204678362573098, 28.250905774248718, 28.64514777488093, 28.55198382771913, 28.493651037058466, 28.174390968508618, 28.118246807644326, 28.086742562597294, 27.745335422304517, 27.603045350371552, 27.49992470106322, 27.57299316122846, 26.989405261656568, 26.798193482289, 26.96908599901329, 26.815715915215787, 26.654320341334447, 26.579242440169594, 26.07634816336831, 25.99316410916214, 25.843898158798368, 25.721531139487798, 25.43344813081655, 24.92926483358251, 24.435438344430523, 24.358643165153108, 24.21369459522444, 24.14722041116872, 23.776916072513558, 23.306285066798253, 22.91980419001982, 23.065640754875954, 23.009126788467945, 22.954267054461607, 22.437847199806455, 22.38826309635558, 22.355966905585227, 22.308619426490953, 22.27776177214379, 22.244573515291748, 22.00993867862176, 21.99812358483945, 21.892526807964625, 21.86706393817819, 21.637317570399723, 21.541817068992888, 21.42564955347449, 21.396463798797388, 21.357142857142858, 21.328575408134363, 21.283237775408022, 21.261483513036307, 21.243167236735278, 21.04204348703488, 20.937494317220118, 20.84283612215802, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.45119855298459, "threshold_area100": 24.45647965592758, "threshold_area25": 22.89399791511467}, "test": {"score": [18.129624718318375, 18.180471940908323, 18.458678116128457, 18.644890377957292, 18.866786929286842, 19.015643196768785, 19.460167593018863, 19.796220243950422, 20.46309671380401, 20.62259767695928, 21.156287301029117, 21.25785603795054, 21.57073066841812, 22.05258906048843, 22.07934636585729, 22.161818681962906, 22.326381764890886, 22.57170599464741, 22.400454230220728, 22.553094980226316, 23.035142686981132, 23.13893232960518, 23.491033717299338, 23.744672729300824, 23.34346265317523, 23.515485665999485, 23.552052666676946, 23.613657467874695, 24.27580975986341, 25.034494867840927, 24.596237935031937, 24.731481225403506, 24.895501159095986, 25.054542760728328, 25.04047717189446, 25.160010689255525, 25.77207124354983, 25.755490409746418, 26.0134966044393, 26.270584252870606, 26.245545667212088, 26.32414180594565, 26.342302239857535, 26.32028789923527, 25.767013497581825, 25.827727954877282, 25.85981383405553, 25.977991497110526, 26.40823856055644, 26.1210313009595, 24.84258151574777, 24.428708187355763, 24.158703270600764, 23.895889390722036, 23.884479070547773, 24.015684093476423, 23.951140707369152, 23.68185592571079, 23.67904134898029, 23.708813904786904, 23.500142569479856, 23.321308611213908, 23.228074685678294, 23.677377546150314, 23.713558457954665, 23.67239746366614, 23.427514312922153, 23.32733374141042, 23.683980999088845, 23.632091013387946, 23.448184844080313, 23.381053223121427, 23.265256190052863, 23.223561057921238, 23.578929419556744, 23.440892591694144, 23.22293245737933, 23.112808400833377, 22.896947196660864, 22.82098640993084, 22.7664891165446, 22.92188560965844, 22.69271774208325, 22.686039134914886, 22.617616828347334, 22.542562432245134, 22.799529619997084, 22.729493053723438, 22.670654771552, 22.55597094812782, 22.47600668099323, 22.436372615603638, 22.378036266524134, 22.35487191266755, 22.3319890566438, 22.30277020451891, 22.214780386686307, 22.16408478646491, 22.085674018832655, 22.37834139290342, 22.361115727328066], "calls": [0.0, 0.7656458055925432, 1.2982689747003995, 2.097203728362184, 3.1957390146471374, 3.695073235685752, 5.525965379494008, 6.724367509986684, 8.122503328894808, 8.88814913448735, 10.286284953395473, 11.18508655126498, 12.350199733688415, 13.548601864181093, 13.914780292942744, 14.247669773635153, 15.479360852197072, 16.378162450066576, 17.80958721704394, 19.241011984021306, 20.206391478029296, 21.00532623169108, 22.103861517976032, 23.235685752330227, 24.167776298268976, 24.900133155792275, 25.466045272969374, 25.998668442077232, 28.462050599201067, 29.26098535286285, 29.793608521970704, 31.458055925432753, 32.42343541944074, 33.655126498002666, 34.32090545938748, 35.65246338215712, 36.25166444740346, 37.217043941411454, 37.74966711051931, 39.14780292942743, 40.07989347536618, 40.77896138482024, 41.54460719041278, 42.7762982689747, 43.77496671105193, 44.47403462050599, 45.339547270306255, 46.23834886817576, 47.13715046604528, 47.969374167776294, 49.367509986684425, 50.59920106524633, 51.53129161118508, 53.06258322237017, 53.76165113182424, 54.89347536617842, 55.59254327563249, 56.924101198402134, 58.45539280958722, 59.22103861517976, 60.51930758988016, 61.65113182423435, 62.782956058588546, 64.0479360852197, 64.71371504660452, 65.5792276964048, 67.21038615179761, 68.50865512649801, 70.47270306258322, 71.1051930758988, 72.27030625832224, 73.20239680426099, 73.73501997336885, 74.86684420772303, 75.56591211717709, 76.63115845539281, 77.49667110519307, 78.26231691078563, 79.16111850865512, 79.72703062583223, 80.62583222370174, 81.69107856191744, 82.9560585885486, 83.85486018641811, 84.62050599201065, 85.58588548601864, 87.35019973368841, 87.98268974700399, 88.81491344873503, 89.58055925432757, 90.1797603195739, 90.81225033288948, 91.6444740346205, 92.50998668442078, 93.40878828229027, 94.3075898801598, 95.10652463382158, 96.07190412782957, 98.06924101198402, 99.20106524633822, 100.0], "rescued": [0, 42, 66, 102, 146, 168, 237, 284, 344, 368, 411, 433, 469, 505, 513, 521, 549, 571, 608, 639, 664, 683, 707, 733, 748, 764, 777, 789, 845, 864, 874, 908, 928, 953, 967, 992, 1003, 1022, 1033, 1058, 1069, 1077, 1085, 1102, 1118, 1130, 1141, 1152, 1159, 1162, 1164, 1168, 1171, 1171, 1172, 1176, 1176, 1176, 1178, 1180, 1180, 1180, 1181, 1183, 1186, 1188, 1188, 1188, 1190, 1190, 1190, 1190, 1191, 1192, 1193, 1193, 1193, 1193, 1193, 1193, 1194, 1195, 1195, 1196, 1196, 1196, 1197, 1197, 1197, 1197, 1197, 1197, 1197, 1197, 1197, 1197, 1197, 1197, 1197, 1198, 1198], "harmed": [0, 13, 23, 34, 50, 57, 76, 91, 107, 115, 129, 144, 163, 186, 193, 198, 216, 234, 268, 296, 316, 333, 355, 377, 400, 414, 426, 439, 485, 503, 515, 545, 559, 588, 603, 628, 638, 661, 667, 689, 706, 711, 722, 747, 769, 786, 807, 826, 842, 865, 906, 940, 971, 1011, 1026, 1053, 1071, 1112, 1150, 1168, 1200, 1229, 1258, 1289, 1304, 1323, 1375, 1410, 1458, 1472, 1507, 1528, 1544, 1571, 1595, 1616, 1641, 1659, 1679, 1691, 1713, 1739, 1767, 1792, 1804, 1820, 1850, 1863, 1878, 1897, 1908, 1922, 1938, 1957, 1975, 1994, 2012, 2034, 2076, 2096, 2109], "exact_score": [18.129624718318375, 18.21331420640158, 18.652621736088033, 18.832031820264454, 19.08996684407024, 19.28380956657901, 19.649542303215792, 20.02743676430226, 20.437703855334142, 20.763008808059357, 21.077670390340128, 21.268596270878703, 21.437371642391813, 21.71582899811564, 22.09916758405261, 22.30971126280735, 22.512399381248265, 22.66334382857894, 22.43845468191545, 22.52830576900757, 22.8251801347506, 23.137293226807294, 23.43781347947088, 23.807890997044964, 23.423763336037428, 23.54018654930266, 23.613657467874695, 24.18355173717534, 24.215487994894115, 24.986871324219717, 24.68962076032679, 24.792750663201364, 24.834444325874074, 24.957248071087896, 25.091470300337694, 25.168081653719643, 25.758848301838142, 25.71196926779134, 26.06150680642959, 26.19282981968188, 26.33948793091162, 26.353313523542127, 26.363765341967948, 26.196274533377416, 25.7452217063995, 25.823969499016574, 26.070853419796475, 26.443650259549838, 26.1210313009595, 25.596574792831138, 24.48929094515105, 24.255993116887897, 24.092439568060172, 23.903714585852434, 23.920672641680248, 24.00792984725119, 23.917888414268084, 23.678257363983942, 23.72155047662825, 23.762451129907788, 23.58565683872634, 23.440699842235762, 23.248164983984644, 23.213438704446197, 23.683612231469276, 23.636638843245695, 23.628599886564654, 23.446512117071155, 23.380732101916724, 23.34613159011779, 23.728665598305795, 23.642354462501437, 23.483574568690486, 23.40729403541535, 23.252270724932924, 23.2186144765831, 23.50728026716691, 23.290715848280033, 23.142894728868093, 22.90228766961315, 22.873724481945366, 23.053097770700678, 22.8430429838889, 22.690602379260756, 22.682328100348734, 22.60620840002229, 22.528148737154392, 22.806577043157382, 22.729493053723438, 22.62023815495471, 22.525262153929727, 22.43351332536117, 22.370584452374416, 22.343392680779132, 22.313496222867514, 22.26172368408346, 22.165356940111995, 22.137924487942772, 22.090280762383312, 22.402599850931047, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.25182018653601, "threshold_area100": 23.230410355378357, "threshold_area25": 21.238289411780176}, "selected_budget": 46, "selected": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726, "rescued": 274, "harmed": 209}, "test": {"score": 25.85981383405553, "calls": 45.339547270306255, "rescued": 1141, "harmed": 807}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 28.485982194596133, "calls": 42.93865905848788}, "test": {"score": 26.35851498163025, "calls": 42.80958721704394}}, "old_normalized_interior": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726}, "test": {"score": 25.93611863895939, "calls": 45.50599201065246}}, "old_normalized_endpoints": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726}, "test": {"score": 25.93611863895939, "calls": 45.50599201065246}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.507340074066676, 18.882101214632783, 19.723935673560206, 20.170190037575235, 20.685260125821728, 21.036291512310264, 21.479167236365956, 21.841305754173412, 22.23979591836735, 22.753253475303165, 24.005474596657887, 23.8549796510357, 24.007846464033396, 23.576111997164627, 23.398930049938908, 23.446975826940395, 23.526917844373088, 23.836085509472607, 24.146387438183908, 24.366080634806682, 24.445253022839232, 24.636798253416853, 25.06030855539972, 25.623799708351857, 26.10047904165551, 26.22529428816088, 26.931289539357678, 26.90484601258814, 26.480258490863974, 26.54721210157655, 26.673899802040506, 27.061116536398444, 27.606743532181078, 25.759846815314148, 26.179939549247656, 27.285927321246373, 27.437456419678753, 27.415526595117345, 27.558027079303677, 27.716190733119124, 27.97490603572908, 28.188131313131315, 28.485982194596133, 28.22488038277512, 28.250905774248718, 28.64514777488093, 28.576460271259567, 28.559779704120285, 28.510780112450217, 28.478477796890196, 28.264523200584584, 28.002916715691434, 27.738063624570096, 27.66699031213869, 27.597259232360017, 27.6109315694397, 27.65219971332733, 27.591127107985926, 27.042854686706576, 27.008531225409865, 26.927859825948996, 27.041292117654255, 26.70581054626146, 26.665009670276852, 26.560378856269086, 26.508148633325586, 25.9410118037028, 25.887372192423737, 25.754293370280852, 25.704858033793958, 25.17688118420719, 25.219146238377004, 24.8301680963951, 24.559355874011047, 24.203057811753464, 24.070098340774877, 23.677525356289745, 23.758979805194745, 23.48205781948113, 23.202893764532323, 23.164434653173753, 23.02016563555884, 22.972039816552016, 22.634001726011412, 22.50607420224835, 22.076358199370315, 22.044738480029192, 22.187744166126233, 22.16014735335901, 22.05378799745576, 21.941487437438855, 21.833482540839082, 21.59907328387248, 21.37932438703988, 21.33794046889004, 21.219065887480653, 21.182862336562962, 21.098805740256644, 21.05246594434204, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.507340074066676, 18.882101214632783, 19.723935673560206, 20.170190037575235, 20.685260125821728, 21.036291512310264, 21.479167236365956, 21.841305754173412, 22.23979591836735, 22.753253475303165, 24.005474596657887, 23.8549796510357, 24.007846464033396, 23.576111997164627, 23.398930049938908, 23.446975826940395, 23.526917844373088, 23.836085509472607, 24.146387438183908, 24.366080634806682, 24.445253022839232, 24.636798253416853, 25.06030855539972, 25.623799708351857, 26.10047904165551, 26.22529428816088, 26.931289539357678, 26.90484601258814, 26.480258490863974, 26.54721210157655, 26.673899802040506, 27.061116536398444, 27.606743532181078, 25.759846815314148, 26.179939549247656, 27.285927321246373, 27.437456419678753, 27.415526595117345, 27.558027079303677, 27.716190733119124, 27.97490603572908, 28.188131313131315, 28.485982194596133, 28.22488038277512, 28.250905774248718, 28.64514777488093, 28.576460271259567, 28.559779704120285, 28.510780112450217, 28.478477796890196, 28.264523200584584, 28.002916715691434, 27.738063624570096, 27.66699031213869, 27.597259232360017, 27.6109315694397, 27.65219971332733, 27.591127107985926, 27.042854686706576, 27.008531225409865, 26.927859825948996, 27.041292117654255, 26.70581054626146, 26.665009670276852, 26.560378856269086, 26.508148633325586, 25.9410118037028, 25.887372192423737, 25.754293370280852, 25.704858033793958, 25.17688118420719, 25.219146238377004, 24.8301680963951, 24.559355874011047, 24.203057811753464, 24.070098340774877, 23.677525356289745, 23.758979805194745, 23.48205781948113, 23.202893764532323, 23.164434653173753, 23.02016563555884, 22.972039816552016, 22.634001726011412, 22.50607420224835, 22.076358199370315, 22.044738480029192, 22.187744166126233, 22.16014735335901, 22.05378799745576, 21.941487437438855, 21.833482540839082, 21.59907328387248, 21.37932438703988, 21.33794046889004, 21.219065887480653, 21.182862336562962, 21.098805740256644, 21.05246594434204, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 22.676159978236228, "exact_area100": 24.77446419752862}, "test": {"score": [18.129624718318375, 18.17787677753799, 18.66103186004181, 18.72844901942819, 18.996709929044613, 19.20881619327964, 19.416133784894026, 19.809073101710997, 20.50748988436245, 20.90305790174854, 21.02779953802219, 21.365545051652955, 21.580438291934023, 22.135661450748916, 22.181886679881156, 22.18349003837692, 22.300620993753782, 22.548911216937174, 22.416180256169824, 22.631549607913858, 22.814062209718582, 23.104847450144177, 23.631625533996804, 23.37382330403698, 23.385083051321097, 23.583403854369735, 23.552052666676946, 23.580275868741275, 24.24258884768557, 25.034494867840927, 24.49497484614827, 24.713463003706902, 24.905081172018775, 25.027444198162534, 25.036481327572556, 25.144997462703895, 25.835126114972034, 25.89002757833675, 26.15171887101328, 26.26537930736424, 26.310547970984178, 26.335805050870732, 26.374156430016626, 26.34891817723598, 25.76100392220855, 25.79925194504586, 25.93611863895939, 25.826974980561314, 25.696102476985722, 25.63865324427389, 25.5806436593087, 25.531119851713907, 25.437315330634952, 25.22893080772917, 25.150362130551212, 25.10149364872713, 25.080911625434272, 24.99714140100763, 24.932856718942627, 25.151701748582894, 25.03867262592836, 24.957472080767467, 25.350457457663556, 25.19430934879155, 25.068566617791998, 24.878994459384323, 24.799285182047267, 24.704511814564263, 24.52136994965679, 24.381709372221707, 24.239746584270836, 24.198266700507602, 24.123811181260177, 24.086785121796485, 23.998173218390605, 24.363537724787538, 24.22971909128888, 24.138890922942316, 24.380629312977526, 24.19280304551702, 24.135156068827467, 23.99657565766593, 23.890680606522487, 23.769975180910293, 23.600999779818657, 23.92320784714431, 23.883663926892133, 23.599207110070683, 23.550006962492446, 23.424845914170543, 22.905985634829527, 22.865585318481223, 22.874619815480727, 22.75046786750957, 22.743550683392368, 23.12118931746083, 23.053733933877066, 22.859026379264584, 22.702192085558526, 22.594314544287343, 22.361115727328066], "calls": [0.0, 0.6324900133155792, 1.8641810918774968, 2.330226364846871, 3.561917443408788, 4.194407456724368, 5.25965379494008, 6.55792276964048, 8.189081225033288, 9.587217043941411, 10.286284953395473, 11.351531291611185, 12.649800266311583, 13.615179760319574, 14.314247669773636, 14.580559254327563, 15.412782956058587, 16.4780292942743, 18.242343541944077, 19.474034620505993, 19.973368841544605, 21.00532623169108, 22.470039946737682, 23.302263648468706, 24.267643142476697, 25.099866844207725, 25.499334221038616, 25.96537949400799, 28.19573901464714, 29.26098535286285, 29.6271637816245, 31.391478029294273, 32.456724367509985, 33.58854860186418, 34.25432756324901, 35.21970705725699, 36.1185086551265, 37.31691078561917, 38.115845539280954, 39.314247669773636, 40.07989347536618, 40.54593874833555, 41.6444740346205, 42.543275632490015, 43.70838881491345, 44.374167776298265, 45.50599201065246, 47.237017310252995, 47.969374167776294, 48.8681757656458, 49.30093209054594, 49.70039946737683, 51.16511318242344, 52.197070572569906, 52.89613848202397, 54.627163781624496, 55.72569906790945, 56.55792276964048, 57.090545938748335, 58.68841544607191, 60.75233022636485, 61.18508655126498, 62.35019973368841, 63.58189081225033, 64.4474034620506, 65.31291611185085, 66.57789613848203, 67.90945406125167, 68.94141145139814, 70.30625832223701, 71.271637816245, 71.80426098535287, 73.23568575233023, 74.13448735019973, 74.80026631158455, 75.56591211717709, 76.33155792276965, 77.03062583222369, 78.69507323568575, 79.36085219707057, 80.62583222370174, 81.8242343541944, 82.78961384820239, 84.05459387483354, 85.08655126498003, 85.68575233022636, 86.18508655126497, 87.1837549933422, 88.91478029294274, 89.78029294274302, 91.44474034620505, 92.17709720372837, 93.20905459387482, 94.50732356857523, 94.6404793608522, 95.50599201065246, 95.93874833555259, 97.07057256990679, 97.83621837549934, 98.50199733688414, 100.0], "exact_score": [18.129624718318375, 18.28578577315594, 18.704077928870753, 18.881680777548095, 19.136533234370184, 19.407234650405442, 19.59075209293343, 19.993811798026986, 20.455598200016883, 20.755587549705197, 20.985990192902584, 21.28039681912009, 21.47193576138749, 21.646116924631514, 22.096182494415224, 22.235349411066192, 22.473162993565122, 22.56840105455414, 22.39091381100727, 22.50352474359722, 22.814062209718582, 23.070585463890563, 23.479942874767616, 23.783032534011696, 23.458029743956637, 23.528522283983282, 23.586854360888843, 24.189925666644914, 24.196004775001644, 24.978862312505818, 24.695618265364697, 24.8403472468891, 24.843926731927617, 24.95312554975385, 25.077536790630745, 25.05703329375329, 25.808419283722685, 25.85210987567812, 26.059544427383695, 26.202184800066224, 26.298631927306417, 26.349602249261828, 26.345867632122626, 26.338780804035377, 25.765517334583592, 25.823969499016574, 25.890193548375752, 25.822770979020977, 25.696102476985722, 25.635691860715493, 25.50814164880066, 25.447036304393748, 25.238375468713016, 25.143670761303145, 25.038326575740832, 25.044251062851785, 25.066221453416578, 24.940828670687917, 24.73834920021802, 25.219500157000834, 25.05841392872189, 24.96863980812006, 25.379647622783764, 25.274923940302045, 25.16431617811023, 24.89826108128304, 24.823552597112933, 24.78366851449878, 24.702787438788295, 24.518245457386783, 24.438381130579963, 24.275460357882768, 24.190803559966955, 24.197121485509797, 24.099836504779024, 23.937237178043098, 24.323884145767302, 24.14161394905665, 24.493915503578055, 24.345267137206513, 24.17081991415366, 24.09433936336197, 23.928411580318564, 23.82625257933949, 23.772314608452376, 23.653717635371528, 23.909702363348348, 23.721188522216547, 23.374594917647688, 23.547866588183787, 23.366744483753994, 23.312344297503373, 22.86973329312548, 22.883704629907722, 22.76867251341726, 22.725619714052257, 23.051881677762967, 22.861333823078326, 22.748947002861094, 22.609489167553647, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.21813645314828, "exact_area100": 23.76166972505993}}, "old_normalized_endpoint_budget": 46}, {"seed": 2028, "selection": {"best_epoch": 5, "epochs": 13, "alpha": 0.75, "validation_area100": 0.24813931142670065, "training_seconds": 4.417721334961243, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.4115027927969035, "operator": "ge"}, {"budget": 0.02, "threshold": 0.39260264832542696, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3628935899866111, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3437955406180572, "operator": "ge"}, {"budget": 0.05, "threshold": 0.32676555259571444, "operator": "ge"}, {"budget": 0.06, "threshold": 0.3043959630588544, "operator": "ge"}, {"budget": 0.07, "threshold": 0.29161725481016765, "operator": "ge"}, {"budget": 0.08, "threshold": 0.25852529206583824, "operator": "ge"}, {"budget": 0.09, "threshold": 0.22163556406544835, "operator": "ge"}, {"budget": 0.1, "threshold": 0.1279206721701493, "operator": "ge"}, {"budget": 0.11, "threshold": 0.11220493614501946, "operator": "ge"}, {"budget": 0.12, "threshold": 0.1052052173668165, "operator": "ge"}, {"budget": 0.13, "threshold": 0.09612782425716211, "operator": "ge"}, {"budget": 0.14, "threshold": 0.08955172927506883, "operator": "ge"}, {"budget": 0.15, "threshold": 0.08639754460226368, "operator": "ge"}, {"budget": 0.16, "threshold": 0.08309047264911913, "operator": "ge"}, {"budget": 0.17, "threshold": 0.079437347059857, "operator": "ge"}, {"budget": 0.18, "threshold": 0.07039787088174246, "operator": "ge"}, {"budget": 0.19, "threshold": 0.06392176665396397, "operator": "ge"}, {"budget": 0.2, "threshold": 0.05780561717884243, "operator": "ge"}, {"budget": 0.21, "threshold": 0.04864266112264597, "operator": "ge"}, {"budget": 0.22, "threshold": 0.04008801705237824, "operator": "ge"}, {"budget": 0.23, "threshold": 0.0346246403275599, "operator": "ge"}, {"budget": 0.24, "threshold": 0.03320023835831557, "operator": "ge"}, {"budget": 0.25, "threshold": 0.023953036973074025, "operator": "ge"}, {"budget": 0.26, "threshold": 0.015014322473524418, "operator": "ge"}, {"budget": 0.27, "threshold": -3.243071385902255e-05, "operator": "ge"}, {"budget": 0.28, "threshold": -0.008514521027975591, "operator": "ge"}, {"budget": 0.29, "threshold": -0.015167503184273984, "operator": "ge"}, {"budget": 0.3, "threshold": -0.026598177491302374, "operator": "ge"}, {"budget": 0.31, "threshold": -0.032311564581759274, "operator": "ge"}, {"budget": 0.32, "threshold": -0.0396351908840306, "operator": "ge"}, {"budget": 0.33, "threshold": -0.04901908672293935, "operator": "ge"}, {"budget": 0.34, "threshold": -0.057858305972897275, "operator": "ge"}, {"budget": 0.35, "threshold": -0.06375858217398687, "operator": "ge"}, {"budget": 0.36, "threshold": -0.07191473513052131, "operator": "ge"}, {"budget": 0.37, "threshold": -0.08048228496760038, "operator": "ge"}, {"budget": 0.38, "threshold": -0.08811748730752388, "operator": "ge"}, {"budget": 0.39, "threshold": -0.09771983918904872, "operator": "ge"}, {"budget": 0.4, "threshold": -0.1084546450415285, "operator": "ge"}, {"budget": 0.41, "threshold": -0.12162505418026433, "operator": "ge"}, {"budget": 0.42, "threshold": -0.14016762094393456, "operator": "ge"}, {"budget": 0.43, "threshold": -0.155394237235881, "operator": "ge"}, {"budget": 0.44, "threshold": -0.18149034865795055, "operator": "ge"}, {"budget": 0.45, "threshold": -0.21528634559799156, "operator": "ge"}, {"budget": 0.46, "threshold": -0.305393338456888, "operator": "ge"}, {"budget": 0.47, "threshold": -0.457510538743241, "operator": "ge"}, {"budget": 0.48, "threshold": -0.49816408617754565, "operator": "ge"}, {"budget": 0.49, "threshold": -0.5226171492916752, "operator": "ge"}, {"budget": 0.5, "threshold": -0.5431040577859461, "operator": "ge"}, {"budget": 0.51, "threshold": -0.5488918660882105, "operator": "ge"}, {"budget": 0.52, "threshold": -0.5528793609228423, "operator": "ge"}, {"budget": 0.53, "threshold": -0.5599270187463063, "operator": "ge"}, {"budget": 0.54, "threshold": -0.5692466027238038, "operator": "ge"}, {"budget": 0.55, "threshold": -0.5729895866108415, "operator": "ge"}, {"budget": 0.56, "threshold": -0.5768459605733179, "operator": "ge"}, {"budget": 0.57, "threshold": -0.5850092583043616, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5907125579954983, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5972781426758086, "operator": "ge"}, {"budget": 0.6, "threshold": -0.6038705301882965, "operator": "ge"}, {"budget": 0.61, "threshold": -0.6058987001210462, "operator": "ge"}, {"budget": 0.62, "threshold": -0.6129304848486948, "operator": "ge"}, {"budget": 0.63, "threshold": -0.6169061499396128, "operator": "ge"}, {"budget": 0.64, "threshold": -0.6211400151197658, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6244538928804774, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6277172043687573, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6297473252302989, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6315304614752101, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6340428577038614, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6355417701033261, "operator": "ge"}, {"budget": 0.71, "threshold": -0.6377943722804739, "operator": "ge"}, {"budget": 0.72, "threshold": -0.6402532628334889, "operator": "ge"}, {"budget": 0.73, "threshold": -0.6428776568243562, "operator": "ge"}, {"budget": 0.74, "threshold": -0.6447361494620908, "operator": "ge"}, {"budget": 0.75, "threshold": -0.6452934442881909, "operator": "ge"}, {"budget": 0.76, "threshold": -0.6463527864185452, "operator": "ge"}, {"budget": 0.77, "threshold": -0.6486311771761815, "operator": "ge"}, {"budget": 0.78, "threshold": -0.6496208353649472, "operator": "ge"}, {"budget": 0.79, "threshold": -0.6514296353123403, "operator": "ge"}, {"budget": 0.8, "threshold": -0.6523602899316283, "operator": "ge"}, {"budget": 0.81, "threshold": -0.6543698471670786, "operator": "ge"}, {"budget": 0.82, "threshold": -0.655710674235104, "operator": "ge"}, {"budget": 0.83, "threshold": -0.6571167168482293, "operator": "ge"}, {"budget": 0.84, "threshold": -0.6589705321075926, "operator": "ge"}, {"budget": 0.85, "threshold": -0.6610899623960598, "operator": "ge"}, {"budget": 0.86, "threshold": -0.663667103699903, "operator": "ge"}, {"budget": 0.87, "threshold": -0.6660643335353217, "operator": "ge"}, {"budget": 0.88, "threshold": -0.6694409094683792, "operator": "ge"}, {"budget": 0.89, "threshold": -0.6704302229187092, "operator": "ge"}, {"budget": 0.9, "threshold": -0.6728210623610504, "operator": "ge"}, {"budget": 0.91, "threshold": -0.6754727990926787, "operator": "ge"}, {"budget": 0.92, "threshold": -0.6764396662011138, "operator": "ge"}, {"budget": 0.93, "threshold": -0.6796567962638791, "operator": "ge"}, {"budget": 0.94, "threshold": -0.6829836810240905, "operator": "ge"}, {"budget": 0.95, "threshold": -0.6853556978122379, "operator": "ge"}, {"budget": 0.96, "threshold": -0.6901984508325426, "operator": "ge"}, {"budget": 0.97, "threshold": -0.6947581950263434, "operator": "ge"}, {"budget": 0.98, "threshold": -0.6981169050348344, "operator": "ge"}, {"budget": 0.99, "threshold": -0.709184789790697, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.399938738527112, 19.315622249257483, 19.861515128584706, 20.16864027799642, 20.556114122515588, 21.058693122857772, 21.46064490871175, 22.086247086247084, 22.5063829359495, 22.507779080189067, 22.556191020875037, 22.646259017551547, 23.99875950350035, 23.957636439488756, 24.107265651143642, 24.0960504263727, 24.56193398348314, 24.4907381175725, 24.647631121687724, 24.605877438694773, 24.359850124181705, 24.486863711001643, 24.701223554583184, 25.160492262307542, 23.697010717091235, 23.824493035353605, 23.681940647168506, 24.675398542017142, 24.891873909450844, 25.715137303204006, 25.78942441851879, 25.848794063079776, 26.096712706172358, 27.03977329220327, 27.2275646142657, 27.362085585316077, 27.415038517103067, 26.934435935831004, 26.96738253680177, 26.99732016321463, 27.324852704244627, 27.953494553861752, 28.149788663076837, 28.21981424148607, 28.73963106534258, 28.64514777488093, 28.351130477013452, 28.608949244505023, 28.417174760127033, 28.43739250086, 28.39127277319946, 28.27654340500858, 28.219072140640765, 28.46651637282444, 28.37788044994808, 28.30513164464068, 28.253310157334926, 27.691219721754074, 27.636314995865558, 27.563661828367714, 27.050724186155996, 26.99358270432472, 26.944208639879264, 26.90667110234173, 26.707659509332558, 26.551495318607397, 26.502173897724628, 26.382758706135746, 26.250963948332366, 25.70589406610912, 25.132589861391004, 25.301591744056996, 25.223643333638435, 25.168788535369778, 25.122125011791756, 24.906538195412626, 24.76894680661804, 24.413057447703512, 24.0676289947604, 24.03157876276256, 23.90532383887235, 23.58188425092473, 23.29889254756276, 22.780043005517687, 22.712136001751716, 22.8694153552603, 22.837799521267264, 22.801397329121855, 22.69273212557758, 22.63284307255169, 22.59414182271928, 22.183406069248374, 21.9343321551941, 21.628544434222068, 21.50521215526864, 21.401023977270558, 20.973887617686852, 20.846789430208442, 20.718424558894434, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 24, 38, 47, 58, 70, 80, 92, 101, 109, 115, 123, 131, 137, 142, 148, 154, 158, 163, 166, 169, 174, 178, 182, 186, 191, 195, 202, 207, 214, 217, 220, 225, 231, 235, 240, 246, 248, 252, 255, 258, 263, 264, 266, 272, 274, 276, 278, 279, 280, 280, 280, 280, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 283, 283, 283, 284, 284, 284, 284, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 3, 7, 8, 10, 13, 16, 19, 21, 22, 26, 30, 34, 36, 40, 44, 50, 57, 61, 63, 69, 74, 80, 84, 87, 95, 101, 109, 117, 122, 128, 134, 140, 148, 154, 158, 162, 167, 174, 178, 182, 187, 191, 197, 199, 204, 209, 217, 222, 227, 232, 235, 237, 241, 244, 250, 255, 259, 265, 269, 275, 282, 287, 291, 295, 300, 305, 310, 315, 320, 327, 333, 339, 346, 351, 355, 361, 367, 371, 377, 381, 390, 397, 405, 412, 419, 424, 428, 432, 437, 444, 449, 456, 465, 472, 479, 483, 493, 501, 510, 518], "exact_score": [18.254072168965788, 18.399938738527112, 19.315622249257483, 19.861515128584706, 20.16864027799642, 20.556114122515588, 21.058693122857772, 21.46064490871175, 22.086247086247084, 22.5063829359495, 22.507779080189067, 22.556191020875037, 22.646259017551547, 23.99875950350035, 23.957636439488756, 24.107265651143642, 24.0960504263727, 24.56193398348314, 24.4907381175725, 24.647631121687724, 24.605877438694773, 24.359850124181705, 24.486863711001643, 24.701223554583184, 25.160492262307542, 23.697010717091235, 23.824493035353605, 23.681940647168506, 24.675398542017142, 24.891873909450844, 25.715137303204006, 25.78942441851879, 25.848794063079776, 26.096712706172358, 27.03977329220327, 27.2275646142657, 27.362085585316077, 27.415038517103067, 26.934435935831004, 26.96738253680177, 26.99732016321463, 27.324852704244627, 27.953494553861752, 28.149788663076837, 28.21981424148607, 28.73963106534258, 28.64514777488093, 28.351130477013452, 28.608949244505023, 28.417174760127033, 28.43739250086, 28.39127277319946, 28.27654340500858, 28.219072140640765, 28.46651637282444, 28.37788044994808, 28.30513164464068, 28.253310157334926, 27.691219721754074, 27.636314995865558, 27.563661828367714, 27.050724186155996, 26.99358270432472, 26.944208639879264, 26.90667110234173, 26.707659509332558, 26.551495318607397, 26.502173897724628, 26.382758706135746, 26.250963948332366, 25.70589406610912, 25.132589861391004, 25.301591744056996, 25.223643333638435, 25.168788535369778, 25.122125011791756, 24.906538195412626, 24.76894680661804, 24.413057447703512, 24.0676289947604, 24.03157876276256, 23.90532383887235, 23.58188425092473, 23.29889254756276, 22.780043005517687, 22.712136001751716, 22.8694153552603, 22.837799521267264, 22.801397329121855, 22.69273212557758, 22.63284307255169, 22.59414182271928, 22.183406069248374, 21.9343321551941, 21.628544434222068, 21.50521215526864, 21.401023977270558, 20.973887617686852, 20.846789430208442, 20.718424558894434, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.813931142670064, "threshold_area100": 24.81981930042922, "threshold_area25": 22.55051554711432}, "test": {"score": [18.129624718318375, 18.264376448117307, 18.46405562908922, 18.94263188287708, 19.1593278777861, 19.374271659928375, 19.672136063447997, 20.028888310481605, 20.429052814749195, 20.70047905414106, 21.1268530669724, 21.232749140767044, 21.579638662226476, 21.868590747908414, 22.25507935860907, 22.375372023809522, 22.447292989958136, 22.616712389543324, 22.991878172442416, 23.22959596010948, 23.418016265403736, 23.359395682679484, 23.50405405305274, 23.579518400913187, 23.6606190387703, 24.55278989200102, 24.72833469429115, 24.76811534443013, 24.98105179530155, 25.341564880297863, 25.651133636566843, 25.762934696067934, 25.93176175420617, 25.75358448287524, 25.58607651625936, 25.69404883630344, 25.8766281157142, 25.992625308674604, 26.131310141166146, 26.183130016236028, 25.76069641190848, 25.901310625925113, 25.95977736852505, 25.979500333599955, 25.864396838752317, 25.707853728930235, 25.92057423882758, 26.391636164570997, 26.263621545802916, 26.07589366593906, 25.12897766801891, 24.981708794314788, 24.880783373983956, 24.63395501054528, 24.578141483170338, 24.480352122470673, 24.36254306236379, 24.2043457328479, 24.146163048542842, 24.576967603683723, 24.468086816919392, 24.430001179566073, 24.438009620314777, 24.822762692890443, 24.810227830387138, 24.618953715086807, 24.54784122855743, 24.353671969772684, 24.138889061021228, 23.830491448775437, 23.73731395637825, 23.370854081889586, 23.30181215406055, 22.90488389309294, 22.699292094995847, 22.69192398402897, 22.623255996979115, 22.428342977193054, 22.36499524821813, 22.296204337352755, 22.243814560838697, 22.183687183208338, 22.079510915119265, 22.00756471381738, 22.194132961997155, 22.42866691706968, 22.364391114014115, 22.354515590202567, 22.307999647182093, 22.29279803171166, 22.191679966366383, 22.1738067815974, 22.1550698124792, 22.380575923359043, 22.571855570661643, 22.652855025692084, 22.574297823635288, 22.43916310749901, 22.46914013935519, 22.480300068919473, 22.361115727328066], "calls": [0.0, 0.8322237017310253, 1.3648468708388815, 2.3635153129161117, 3.029294274300932, 4.094540612516644, 5.159786950732357, 5.89214380825566, 7.090545938748336, 8.189081225033288, 11.051930758988016, 11.717709720372836, 12.350199733688415, 13.182423435419441, 13.914780292942744, 14.480692410119842, 15.11318242343542, 15.545938748335553, 17.0439414114514, 18.342210386151798, 19.607190412782955, 21.50466045272969, 22.969374167776298, 23.93475366178429, 24.101198402130493, 25.299600532623167, 26.830892143808256, 28.86151797603196, 30.326231691078565, 31.091877496671106, 32.822902796271634, 33.38881491344873, 34.25432756324901, 35.38615179760319, 36.38482023968043, 37.150466045272964, 37.88282290279627, 38.44873501997337, 38.98135818908123, 39.280958721704394, 40.31291611185087, 41.211717709720375, 42.2103861517976, 42.876165113182424, 43.57523302263648, 44.17443408788282, 45.173102529960055, 46.17177097203729, 46.70439414114514, 47.969374167776294, 49.60053262316911, 50.13315579227696, 50.76564580559254, 51.96404793608522, 53.76165113182424, 54.693741677762986, 55.52596537949401, 57.35685752330226, 58.3555259653795, 59.78695073235686, 61.085219707057256, 61.55126498002663, 62.849533954727036, 63.98135818908123, 64.81358189081226, 65.97869507323568, 67.0439414114514, 68.14247669773636, 69.0745672436751, 70.33954727030626, 71.1051930758988, 72.56990679094541, 74.2676431424767, 75.89880159786951, 77.1970705725699, 77.3635153129161, 77.92942743009321, 79.26098535286285, 79.76031957390146, 80.32623169107856, 80.85885486018641, 81.8242343541944, 82.45672436750998, 83.15579227696405, 83.72170439414116, 84.68708388814915, 85.98535286284952, 87.31691078561917, 88.48202396804261, 88.78162450066579, 89.84687083888149, 90.77896138482025, 91.14513981358189, 92.37683089214381, 93.60852197070572, 94.47403462050599, 95.73901464713715, 97.30359520639148, 97.76964047936086, 99.10119840213049, 100.0], "rescued": [0, 46, 74, 122, 154, 197, 241, 267, 311, 354, 437, 453, 473, 497, 517, 533, 547, 560, 597, 625, 652, 697, 731, 752, 758, 788, 818, 863, 892, 908, 949, 962, 983, 1005, 1024, 1038, 1051, 1060, 1068, 1073, 1089, 1099, 1106, 1112, 1119, 1124, 1141, 1154, 1156, 1158, 1160, 1161, 1163, 1163, 1164, 1165, 1165, 1165, 1165, 1167, 1169, 1170, 1173, 1174, 1175, 1175, 1176, 1176, 1177, 1177, 1177, 1177, 1178, 1178, 1178, 1178, 1178, 1179, 1179, 1179, 1180, 1181, 1181, 1181, 1182, 1183, 1184, 1185, 1186, 1186, 1187, 1188, 1188, 1189, 1190, 1193, 1193, 1195, 1196, 1198, 1198], "harmed": [0, 5, 11, 26, 36, 49, 66, 73, 92, 104, 136, 144, 153, 171, 185, 195, 205, 212, 247, 276, 298, 337, 366, 381, 384, 411, 442, 484, 518, 531, 572, 583, 604, 628, 646, 661, 675, 683, 695, 698, 722, 732, 740, 751, 768, 782, 800, 828, 840, 859, 890, 905, 919, 944, 976, 992, 1011, 1045, 1063, 1087, 1113, 1127, 1150, 1169, 1184, 1210, 1240, 1267, 1289, 1322, 1342, 1377, 1414, 1460, 1489, 1493, 1504, 1540, 1553, 1569, 1582, 1608, 1627, 1642, 1658, 1686, 1717, 1751, 1778, 1790, 1822, 1849, 1859, 1887, 1922, 1947, 1979, 2025, 2042, 2084, 2109], "exact_score": [18.129624718318375, 18.30817076832232, 18.852368660156365, 19.15172729591374, 19.342402110567527, 19.64197864995814, 20.025056427999708, 20.38507212003019, 20.667416346176662, 20.83896978859977, 21.026160735937047, 21.1078217496855, 21.3345572861091, 21.821455408758254, 22.23842438245832, 22.434620299922027, 22.84240973179, 22.985499606628615, 23.200550339979735, 23.403975325785627, 23.55930325708287, 23.375303407673385, 23.358874477282473, 23.50405405305274, 23.591994293361616, 24.146399245245544, 24.723346906295465, 24.736834091342285, 25.029115534490522, 24.74016540885146, 24.92596052791989, 25.33197374957513, 25.529095376856585, 25.68452356152113, 25.803241502145518, 26.120660706761576, 25.386569168479717, 25.65277052191827, 25.903239877187577, 26.131310141166146, 25.726094446615633, 25.92544412926243, 25.95863445419534, 26.00983371071272, 25.746274338280916, 25.917157737034874, 26.443500078277566, 26.142412395273375, 26.07589366593906, 25.754652487344863, 25.099726962552243, 24.81935805515589, 24.63395501054528, 24.57806285864519, 24.586379830905525, 24.46357081825451, 24.335188227952052, 24.27675286957763, 24.16935677950162, 24.105292279213874, 24.555811732052167, 24.47411992111549, 24.392208723604195, 24.432198110950644, 24.822762692890443, 24.80371022518532, 24.618953715086807, 24.5512175081674, 24.362291780357257, 24.14281637944821, 23.88670232453568, 23.793874651378992, 23.458811035543675, 23.350149179782488, 23.335918565978275, 23.18977155061253, 22.85175042243498, 22.75474450165541, 22.623255996979115, 22.480675224367886, 22.33502708906716, 22.241346746078793, 22.1748974709501, 22.014712990568754, 22.164158320583333, 22.41461858701708, 22.364391114014115, 22.36332288529347, 22.326397880859265, 22.269460905721953, 22.187910264715125, 22.164408647598908, 22.410524356498332, 22.6499381785668, 22.587357637487635, 22.616204952137135, 22.585812928397655, 22.518067827593054, 22.46323997216704, 22.487157435690634, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.461149436471242, "threshold_area100": 23.42683884991929, "threshold_area25": 21.527144064254745}, "selected_budget": 45, "selected": {"val": {"score": 28.73963106534258, "calls": 44.935805991440795, "rescued": 272, "harmed": 204}, "test": {"score": 25.707853728930235, "calls": 44.17443408788282, "rescued": 1124, "harmed": 782}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.3308299481868744, "operator": "ge"}, {"budget": 0.02, "threshold": 0.28251340985298157, "operator": "ge"}, {"budget": 0.03, "threshold": 0.2749890387058258, "operator": "ge"}, {"budget": 0.04, "threshold": 0.262128084897995, "operator": "ge"}, {"budget": 0.05, "threshold": 0.24537649750709534, "operator": "ge"}, {"budget": 0.06, "threshold": 0.23267678916454315, "operator": "ge"}, {"budget": 0.07, "threshold": 0.21530349552631378, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2059350609779358, "operator": "ge"}, {"budget": 0.09, "threshold": 0.1774962991476059, "operator": "ge"}, {"budget": 0.1, "threshold": 0.08884723484516144, "operator": "ge"}, {"budget": 0.11, "threshold": 0.058810755610466, "operator": "ge"}, {"budget": 0.12, "threshold": 0.05232292413711548, "operator": "ge"}, {"budget": 0.13, "threshold": 0.04931269586086273, "operator": "ge"}, {"budget": 0.14, "threshold": 0.04670526087284088, "operator": "ge"}, {"budget": 0.15, "threshold": 0.0454326868057251, "operator": "ge"}, {"budget": 0.16, "threshold": 0.043865516781806946, "operator": "ge"}, {"budget": 0.17, "threshold": 0.04250766336917877, "operator": "ge"}, {"budget": 0.18, "threshold": 0.03834165632724762, "operator": "ge"}, {"budget": 0.19, "threshold": 0.03473776578903198, "operator": "ge"}, {"budget": 0.2, "threshold": 0.030873656272888184, "operator": "ge"}, {"budget": 0.21, "threshold": 0.025220274925231934, "operator": "ge"}, {"budget": 0.22, "threshold": 0.021761834621429443, "operator": "ge"}, {"budget": 0.23, "threshold": 0.019833698868751526, "operator": "ge"}, {"budget": 0.24, "threshold": 0.017736494541168213, "operator": "ge"}, {"budget": 0.25, "threshold": 0.014020167291164398, "operator": "ge"}, {"budget": 0.26, "threshold": 0.008498236536979675, "operator": "ge"}, {"budget": 0.27, "threshold": -1.8551945686340332e-05, "operator": "ge"}, {"budget": 0.28, "threshold": -0.0047393739223480225, "operator": "ge"}, {"budget": 0.29, "threshold": -0.008754327893257141, "operator": "ge"}, {"budget": 0.3, "threshold": -0.01278848946094513, "operator": "ge"}, {"budget": 0.31, "threshold": -0.019203394651412964, "operator": "ge"}, {"budget": 0.32, "threshold": -0.02355913817882538, "operator": "ge"}, {"budget": 0.33, "threshold": -0.02679157257080078, "operator": "ge"}, {"budget": 0.34, "threshold": -0.031072348356246948, "operator": "ge"}, {"budget": 0.35, "threshold": -0.03542228043079376, "operator": "ge"}, {"budget": 0.36, "threshold": -0.04178282618522644, "operator": "ge"}, {"budget": 0.37, "threshold": -0.04553802311420441, "operator": "ge"}, {"budget": 0.38, "threshold": -0.04877835512161255, "operator": "ge"}, {"budget": 0.39, "threshold": -0.05528506636619568, "operator": "ge"}, {"budget": 0.4, "threshold": -0.06423117220401764, "operator": "ge"}, {"budget": 0.41, "threshold": -0.06775818020105362, "operator": "ge"}, {"budget": 0.42, "threshold": -0.0746002197265625, "operator": "ge"}, {"budget": 0.43, "threshold": -0.08289097249507904, "operator": "ge"}, {"budget": 0.44, "threshold": -0.08979165554046631, "operator": "ge"}, {"budget": 0.45, "threshold": -0.09178914874792099, "operator": "ge"}, {"budget": 0.46, "threshold": -0.0938408225774765, "operator": "ge"}, {"budget": 0.47, "threshold": -0.09923499077558517, "operator": "ge"}, {"budget": 0.48, "threshold": -0.10381849110126495, "operator": "ge"}, {"budget": 0.49, "threshold": -0.10631191730499268, "operator": "ge"}, {"budget": 0.5, "threshold": -0.10954432189464569, "operator": "ge"}, {"budget": 0.51, "threshold": -0.11497385799884796, "operator": "ge"}, {"budget": 0.52, "threshold": -0.11880984902381897, "operator": "ge"}, {"budget": 0.53, "threshold": -0.12497489899396896, "operator": "ge"}, {"budget": 0.54, "threshold": -0.13171370327472687, "operator": "ge"}, {"budget": 0.55, "threshold": -0.1340555101633072, "operator": "ge"}, {"budget": 0.56, "threshold": -0.13827651739120483, "operator": "ge"}, {"budget": 0.57, "threshold": -0.1418142467737198, "operator": "ge"}, {"budget": 0.58, "threshold": -0.14489807188510895, "operator": "ge"}, {"budget": 0.59, "threshold": -0.14742255210876465, "operator": "ge"}, {"budget": 0.6, "threshold": -0.15259508788585663, "operator": "ge"}, {"budget": 0.61, "threshold": -0.1562899947166443, "operator": "ge"}, {"budget": 0.62, "threshold": -0.15818259119987488, "operator": "ge"}, {"budget": 0.63, "threshold": -0.16099680960178375, "operator": "ge"}, {"budget": 0.64, "threshold": -0.16477172076702118, "operator": "ge"}, {"budget": 0.65, "threshold": -0.16732005774974823, "operator": "ge"}, {"budget": 0.66, "threshold": -0.1697973906993866, "operator": "ge"}, {"budget": 0.67, "threshold": -0.17120549082756042, "operator": "ge"}, {"budget": 0.68, "threshold": -0.17282381653785706, "operator": "ge"}, {"budget": 0.69, "threshold": -0.1737048327922821, "operator": "ge"}, {"budget": 0.7, "threshold": -0.17586630582809448, "operator": "ge"}, {"budget": 0.71, "threshold": -0.17776507139205933, "operator": "ge"}, {"budget": 0.72, "threshold": -0.17915716767311096, "operator": "ge"}, {"budget": 0.73, "threshold": -0.18153832852840424, "operator": "ge"}, {"budget": 0.74, "threshold": -0.18367737531661987, "operator": "ge"}, {"budget": 0.75, "threshold": -0.18530672788619995, "operator": "ge"}, {"budget": 0.76, "threshold": -0.18665507435798645, "operator": "ge"}, {"budget": 0.77, "threshold": -0.18812023103237152, "operator": "ge"}, {"budget": 0.78, "threshold": -0.19010013341903687, "operator": "ge"}, {"budget": 0.79, "threshold": -0.19093501567840576, "operator": "ge"}, {"budget": 0.8, "threshold": -0.19238603115081787, "operator": "ge"}, {"budget": 0.81, "threshold": -0.1944231390953064, "operator": "ge"}, {"budget": 0.82, "threshold": -0.19854019582271576, "operator": "ge"}, {"budget": 0.83, "threshold": -0.20096665620803833, "operator": "ge"}, {"budget": 0.84, "threshold": -0.2025768756866455, "operator": "ge"}, {"budget": 0.85, "threshold": -0.20596633851528168, "operator": "ge"}, {"budget": 0.86, "threshold": -0.20817846059799194, "operator": "ge"}, {"budget": 0.87, "threshold": -0.21155302226543427, "operator": "ge"}, {"budget": 0.88, "threshold": -0.2133837789297104, "operator": "ge"}, {"budget": 0.89, "threshold": -0.21528767049312592, "operator": "ge"}, {"budget": 0.9, "threshold": -0.21759267151355743, "operator": "ge"}, {"budget": 0.91, "threshold": -0.22076308727264404, "operator": "ge"}, {"budget": 0.92, "threshold": -0.22571790218353271, "operator": "ge"}, {"budget": 0.93, "threshold": -0.22945308685302734, "operator": "ge"}, {"budget": 0.94, "threshold": -0.23532819747924805, "operator": "ge"}, {"budget": 0.95, "threshold": -0.24474529922008514, "operator": "ge"}, {"budget": 0.96, "threshold": -0.24882633984088898, "operator": "ge"}, {"budget": 0.97, "threshold": -0.25874069333076477, "operator": "ge"}, {"budget": 0.98, "threshold": -0.273191899061203, "operator": "ge"}, {"budget": 0.99, "threshold": -0.28613048791885376, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.509057971014496, 18.7223804870932, 19.1817617866005, 20.203462950234716, 20.654761904761905, 21.00696585927804, 21.578022434186817, 21.990093240093238, 22.339935974389757, 22.507779080189067, 22.66812670423929, 22.455419322895054, 22.60831141479907, 23.92583447145256, 23.74066945950693, 24.23302707537198, 24.44886219521876, 24.643874643874643, 24.67770953091551, 24.448092805854298, 24.35794217668604, 24.295949946649102, 24.33523756267236, 24.75000970182073, 23.697010717091235, 23.647810019687537, 23.681940647168506, 24.675398542017142, 24.891873909450844, 25.715137303204006, 25.78942441851879, 25.848794063079776, 26.096712706172358, 26.61190357810277, 27.163906120639297, 27.270555764078875, 27.15734138132369, 27.587045570916537, 27.343141117593326, 27.629322583916437, 27.375503160923166, 27.68022455459128, 27.950720208807866, 27.859900497419392, 27.803878088455807, 27.768275062198576, 27.750968035545753, 27.86525651850729, 27.81698620475025, 28.003980252481252, 28.018133790247727, 27.959331880900507, 27.87205492572415, 27.405481161429147, 27.663033084719828, 27.612030618690532, 27.327789169894434, 27.267575448466303, 27.00734737161036, 26.74571814792258, 26.623121390258053, 26.26101887163182, 26.29202920491487, 25.552791987218214, 25.47756855481813, 25.434742562402135, 25.31146423729495, 25.27504755865567, 24.890072492524183, 24.858083731528495, 24.6507659007659, 24.605692530819432, 24.550543024227235, 24.731358060465563, 24.681732352589776, 24.228055148539156, 23.62006171991676, 23.56444528491538, 23.199530609459146, 23.234434431893824, 23.18574872421026, 22.91059390550224, 22.886178796368085, 22.847577779821705, 23.01208087319509, 23.05046144331859, 22.937930698614462, 22.764158508932027, 22.80243075922539, 22.830583268216152, 22.521457917960632, 22.2237609451615, 22.014811129396456, 21.903109743425585, 21.959564951928012, 21.832692382339708, 21.62790736574473, 21.58027208443852, 21.071478673596136, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 22, 36, 48, 61, 71, 81, 91, 101, 109, 116, 122, 129, 136, 141, 148, 154, 158, 163, 166, 170, 172, 176, 182, 186, 190, 195, 202, 207, 214, 217, 220, 225, 229, 234, 239, 243, 248, 251, 257, 259, 262, 265, 265, 266, 266, 266, 268, 268, 268, 269, 270, 270, 270, 272, 272, 272, 272, 273, 273, 273, 274, 275, 275, 275, 275, 276, 276, 276, 277, 277, 277, 277, 278, 278, 278, 278, 278, 278, 279, 279, 279, 279, 279, 280, 281, 281, 282, 282, 283, 283, 283, 283, 283, 284, 284, 284, 284, 285, 286], "harmed": [0, 2, 5, 7, 10, 14, 17, 19, 21, 23, 26, 29, 35, 38, 41, 46, 47, 55, 59, 64, 70, 75, 79, 83, 88, 95, 100, 109, 117, 122, 128, 134, 140, 148, 152, 157, 162, 168, 174, 178, 183, 189, 191, 193, 198, 202, 204, 205, 209, 212, 218, 223, 228, 234, 237, 243, 247, 254, 259, 265, 273, 277, 281, 286, 293, 300, 304, 310, 314, 318, 322, 328, 333, 339, 345, 351, 357, 363, 373, 379, 384, 391, 397, 400, 405, 409, 415, 421, 429, 438, 445, 451, 457, 465, 470, 477, 485, 494, 501, 510, 518], "exact_score": [18.254072168965788, 18.509057971014496, 18.7223804870932, 19.1817617866005, 20.203462950234716, 20.654761904761905, 21.00696585927804, 21.578022434186817, 21.990093240093238, 22.339935974389757, 22.507779080189067, 22.66812670423929, 22.455419322895054, 22.60831141479907, 23.92583447145256, 23.74066945950693, 24.23302707537198, 24.44886219521876, 24.643874643874643, 24.67770953091551, 24.448092805854298, 24.35794217668604, 24.295949946649102, 24.33523756267236, 24.75000970182073, 23.697010717091235, 23.647810019687537, 23.681940647168506, 24.675398542017142, 24.891873909450844, 25.715137303204006, 25.78942441851879, 25.848794063079776, 26.096712706172358, 26.61190357810277, 27.163906120639297, 27.270555764078875, 27.15734138132369, 27.587045570916537, 27.343141117593326, 27.629322583916437, 27.375503160923166, 27.68022455459128, 27.950720208807866, 27.859900497419392, 27.803878088455807, 27.768275062198576, 27.750968035545753, 27.86525651850729, 27.81698620475025, 28.003980252481252, 28.018133790247727, 27.959331880900507, 27.87205492572415, 27.405481161429147, 27.663033084719828, 27.612030618690532, 27.327789169894434, 27.267575448466303, 27.00734737161036, 26.74571814792258, 26.623121390258053, 26.26101887163182, 26.29202920491487, 25.552791987218214, 25.47756855481813, 25.434742562402135, 25.31146423729495, 25.27504755865567, 24.890072492524183, 24.858083731528495, 24.6507659007659, 24.605692530819432, 24.550543024227235, 24.731358060465563, 24.681732352589776, 24.228055148539156, 23.62006171991676, 23.56444528491538, 23.199530609459146, 23.234434431893824, 23.18574872421026, 22.91059390550224, 22.886178796368085, 22.847577779821705, 23.01208087319509, 23.05046144331859, 22.937930698614462, 22.764158508932027, 22.80243075922539, 22.830583268216152, 22.521457917960632, 22.2237609451615, 22.014811129396456, 21.903109743425585, 21.959564951928012, 21.832692382339708, 21.62790736574473, 21.58027208443852, 21.071478673596136, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.51788893567568, "threshold_area100": 24.523101994485774, "threshold_area25": 22.465368938668146}, "test": {"score": [18.129624718318375, 18.374080840331644, 18.877504968841222, 18.996295989295476, 19.234546575586016, 19.396045992253825, 19.616032584996812, 20.08922125117373, 20.427712654139665, 20.66117937216702, 21.131471041558132, 21.63343914244439, 21.786746254073424, 22.076542278901467, 22.3205802026367, 22.34213871734084, 22.424985496992452, 22.627226649015206, 22.59511639198338, 22.88685843139526, 22.985801620970587, 23.355359766363925, 23.54690792974303, 23.575356359602065, 23.71221034741968, 24.185474302401193, 24.721873076302003, 24.76811534443013, 25.04123327481591, 25.371786527472988, 25.521761763141072, 25.817844940466895, 25.518094232268478, 25.64793716224076, 25.953361651263634, 25.622144937766745, 25.9273459998272, 25.952232573187818, 26.00270739256818, 26.20491505643229, 25.926935761939635, 25.936026533934825, 25.828710338811927, 25.674644068499635, 25.677652486165304, 25.653100095140147, 25.620251528290517, 25.42150585061111, 25.309978264373505, 25.309827312139067, 25.284099464255007, 25.254282530852585, 25.209691520175394, 25.114280514075617, 24.966615810025655, 24.943845973938725, 25.37684414190438, 25.33994763949064, 25.774782232634337, 25.64156724347769, 26.24801237572977, 26.179711498187075, 26.044713241214545, 25.822635292201568, 25.682424119814375, 25.326781844324998, 25.166915029471337, 24.988163653919674, 24.861678974463285, 24.690347447208243, 24.518436819538408, 24.509289027091896, 24.427487824132342, 24.105105431736217, 23.872069225737363, 23.71109120319704, 23.660253327968466, 23.496152846370038, 23.268185593589124, 23.64468551480568, 23.561123206945993, 23.07529475494039, 23.088057169009048, 22.97836875953297, 22.970130230854092, 23.025838496294465, 22.907385786720734, 22.845130586197058, 22.825302928228997, 22.770236216155677, 22.70987436164578, 23.07440202094238, 23.332723764505964, 23.232226324759555, 23.191006961175095, 23.063816405419278, 23.096044930096422, 23.022564394675253, 22.861458538605813, 22.858091815000765, 22.361115727328066], "calls": [0.0, 1.1318242343541944, 2.0639147802929427, 2.4966711051930757, 3.2290279627163785, 4.360852197070573, 5.3928095872170445, 6.391478029294275, 7.023968042609853, 8.189081225033288, 11.085219707057258, 12.050599201065246, 12.949400798934754, 13.81491344873502, 14.480692410119842, 14.81358189081225, 15.446071904127828, 15.84553928095872, 17.14380825565912, 18.40878828229028, 20.03994673768309, 22.07057256990679, 23.10252996005326, 23.768308921438084, 24.400798934753663, 25.133155792276963, 26.864181091877498, 28.86151797603196, 30.326231691078565, 31.19174434087883, 32.123834886817576, 33.5219707057257, 34.52063914780293, 35.18641810918775, 36.18508655126498, 36.98402130492676, 37.949400798934754, 38.28229027962716, 38.68175765645805, 39.41411451398136, 40.44607190412783, 40.87882822902797, 42.07723035952064, 43.30892143808256, 44.30758988015979, 44.77363515312916, 45.372836218375504, 46.43808255659121, 47.86950732356858, 48.368841544607186, 49.20106524633822, 50.665778961384824, 51.86418109187749, 53.695073235685754, 55.22636484687084, 55.49267643142477, 56.657789613848195, 57.2237017310253, 57.98934753661784, 58.72170439414115, 60.48601864181092, 61.31824234354194, 62.11717709720372, 63.24900133155792, 65.21304926764314, 66.44474034620505, 68.20905459387482, 69.04127829560586, 70.0399467376831, 70.50599201065246, 71.5046604527297, 72.4034620505992, 73.00266311584554, 74.36750998668442, 75.46604527296937, 76.2982689747004, 76.93075898801598, 77.3635153129161, 78.12916111850865, 78.52862849533955, 79.52729693741678, 80.22636484687084, 82.02396804260985, 82.98934753661784, 83.55525965379495, 84.85352862849534, 85.51930758988016, 86.75099866844208, 87.35019973368841, 88.08255659121171, 88.54860186418108, 89.54727030625833, 90.77896138482025, 92.11051930758988, 93.27563249001332, 95.23968042609854, 95.63914780292943, 96.93741677762982, 97.86950732356857, 98.5352862849534, 100.0], "rescued": [0, 61, 107, 128, 161, 207, 243, 284, 310, 349, 437, 464, 489, 512, 532, 540, 556, 565, 598, 626, 661, 708, 734, 749, 764, 783, 819, 863, 893, 912, 935, 967, 990, 1002, 1020, 1032, 1052, 1058, 1063, 1074, 1091, 1095, 1104, 1107, 1110, 1111, 1114, 1115, 1118, 1120, 1121, 1127, 1128, 1131, 1134, 1134, 1139, 1140, 1144, 1145, 1153, 1153, 1153, 1153, 1156, 1156, 1157, 1158, 1159, 1159, 1160, 1162, 1162, 1163, 1163, 1163, 1163, 1163, 1164, 1166, 1167, 1167, 1168, 1169, 1170, 1174, 1174, 1175, 1176, 1176, 1176, 1178, 1179, 1181, 1185, 1189, 1190, 1193, 1195, 1196, 1198], "harmed": [0, 4, 14, 23, 34, 55, 70, 81, 91, 101, 134, 150, 164, 183, 195, 202, 212, 219, 246, 273, 306, 348, 367, 379, 392, 406, 444, 484, 517, 535, 556, 587, 612, 623, 638, 656, 675, 681, 687, 698, 719, 725, 742, 755, 769, 777, 786, 807, 838, 849, 861, 890, 905, 935, 964, 970, 994, 1007, 1019, 1035, 1065, 1083, 1100, 1127, 1171, 1202, 1244, 1263, 1287, 1303, 1325, 1344, 1360, 1395, 1426, 1454, 1469, 1482, 1503, 1516, 1541, 1556, 1608, 1633, 1648, 1687, 1707, 1737, 1753, 1773, 1787, 1810, 1839, 1881, 1910, 1980, 1990, 2031, 2057, 2071, 2109], "exact_score": [18.129624718318375, 18.324642189256497, 18.857138099769504, 19.14267971524976, 19.32367244067469, 19.445851728857484, 19.782107206893787, 20.419301575943138, 20.65674143141489, 20.841032540024454, 21.0649480340202, 21.143585409050292, 21.601377963420436, 21.796460381189718, 22.122682028301433, 22.420247368237916, 22.66228891461774, 22.522871715177608, 22.846424927020802, 22.97070987890488, 22.96036721084368, 23.2405117512008, 23.375535577375146, 23.535584402464895, 23.581439904114323, 24.153075102860367, 24.722056162766133, 24.736834091342285, 25.036171230456432, 24.74016540885146, 24.965012538894147, 25.317319464941747, 25.489689724777293, 25.690178696377863, 25.90586617180944, 25.666443774416436, 25.859732077003017, 25.622144937766745, 25.946333088606828, 25.97333560413079, 26.39086899522943, 25.932092909311542, 25.818657561732262, 25.76231959093726, 25.692047268610313, 25.66495791547278, 25.518580681358095, 25.39879665719549, 25.302336306534624, 25.278408866946428, 25.303414802740953, 25.23100475297035, 25.216272502164166, 25.12578043062048, 25.048371435391775, 24.964819550484552, 25.38507180324951, 25.35658269669101, 25.774782232634337, 26.149996093597405, 26.21898394121312, 26.197275479245853, 26.124287683310875, 25.85624833061937, 25.730403688556002, 25.697074013216763, 25.400082892543658, 25.227608286364045, 25.1818038823658, 25.051965256106122, 24.86328908682766, 24.582080218212166, 24.458530691100314, 24.430464113818186, 24.186810690748384, 23.954701682029114, 23.817608124735777, 23.65757147126543, 23.22144993933677, 23.558093009310653, 23.132010488094608, 23.39235431619863, 23.090463124456413, 22.97836875953297, 22.979665396846944, 22.99883759435448, 22.890599566127005, 22.816517368881776, 22.793098620061407, 22.675915517429058, 23.366058847856454, 23.38196798141957, 23.236084846596242, 23.21636920862562, 23.09487916479302, 23.09857432416635, 23.08044048575711, 22.97315318476644, 22.856795089465038, 22.65254172943048, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.83157769956668, "threshold_area100": 23.811533665697777, "threshold_area25": 21.402601376496634}, "selected_budget": 51, "selected": {"val": {"score": 28.018133790247727, "calls": 50.92724679029958, "rescued": 269, "harmed": 223}, "test": {"score": 25.254282530852585, "calls": 50.665778961384824, "rescued": 1127, "harmed": 890}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.5440630912780762, "operator": "ge"}, {"budget": 0.02, "threshold": 1.2957404851913452, "operator": "ge"}, {"budget": 0.03, "threshold": 1.2218142747879028, "operator": "ge"}, {"budget": 0.04, "threshold": 1.160203218460083, "operator": "ge"}, {"budget": 0.05, "threshold": 1.116869568824768, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0604050159454346, "operator": "ge"}, {"budget": 0.07, "threshold": 1.016830325126648, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9357538223266602, "operator": "ge"}, {"budget": 0.09, "threshold": 0.8331810235977173, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4856545031070709, "operator": "ge"}, {"budget": 0.11, "threshold": 0.36163800954818726, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3155252933502197, "operator": "ge"}, {"budget": 0.13, "threshold": 0.2998811602592468, "operator": "ge"}, {"budget": 0.14, "threshold": 0.27729421854019165, "operator": "ge"}, {"budget": 0.15, "threshold": 0.26115620136260986, "operator": "ge"}, {"budget": 0.16, "threshold": 0.2504185438156128, "operator": "ge"}, {"budget": 0.17, "threshold": 0.23552396893501282, "operator": "ge"}, {"budget": 0.18, "threshold": 0.22404024004936218, "operator": "ge"}, {"budget": 0.19, "threshold": 0.20618999004364014, "operator": "ge"}, {"budget": 0.2, "threshold": 0.17121979594230652, "operator": "ge"}, {"budget": 0.21, "threshold": 0.14636445045471191, "operator": "ge"}, {"budget": 0.22, "threshold": 0.12680763006210327, "operator": "ge"}, {"budget": 0.23, "threshold": 0.11114436388015747, "operator": "ge"}, {"budget": 0.24, "threshold": 0.09705954790115356, "operator": "ge"}, {"budget": 0.25, "threshold": 0.08496683835983276, "operator": "ge"}, {"budget": 0.26, "threshold": 0.043646156787872314, "operator": "ge"}, {"budget": 0.27, "threshold": -8.803606033325195e-05, "operator": "ge"}, {"budget": 0.28, "threshold": -0.0242730975151062, "operator": "ge"}, {"budget": 0.29, "threshold": -0.049327552318573, "operator": "ge"}, {"budget": 0.3, "threshold": -0.07809120416641235, "operator": "ge"}, {"budget": 0.31, "threshold": -0.10097050666809082, "operator": "ge"}, {"budget": 0.32, "threshold": -0.12711495161056519, "operator": "ge"}, {"budget": 0.33, "threshold": -0.1376391053199768, "operator": "ge"}, {"budget": 0.34, "threshold": -0.16626673936843872, "operator": "ge"}, {"budget": 0.35, "threshold": -0.204233318567276, "operator": "ge"}, {"budget": 0.36, "threshold": -0.24742674827575684, "operator": "ge"}, {"budget": 0.37, "threshold": -0.28081220388412476, "operator": "ge"}, {"budget": 0.38, "threshold": -0.36999082565307617, "operator": "ge"}, {"budget": 0.39, "threshold": -0.4664260149002075, "operator": "ge"}, {"budget": 0.4, "threshold": -0.509406566619873, "operator": "ge"}, {"budget": 0.41, "threshold": -0.5702195167541504, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6838445067405701, "operator": "ge"}, {"budget": 0.43, "threshold": -0.845156192779541, "operator": "ge"}, {"budget": 0.44, "threshold": -0.9985588788986206, "operator": "ge"}, {"budget": 0.45, "threshold": -1.2389718294143677, "operator": "ge"}, {"budget": 0.46, "threshold": -1.4227447509765625, "operator": "ge"}, {"budget": 0.47, "threshold": -1.706304669380188, "operator": "ge"}, {"budget": 0.48, "threshold": -1.9934935569763184, "operator": "ge"}, {"budget": 0.49, "threshold": -2.226430892944336, "operator": "ge"}, {"budget": 0.5, "threshold": -2.4796807765960693, "operator": "ge"}, {"budget": 0.51, "threshold": -2.8224148750305176, "operator": "ge"}, {"budget": 0.52, "threshold": -4.309603691101074, "operator": "ge"}, {"budget": 0.53, "threshold": -4.37282657623291, "operator": "ge"}, {"budget": 0.54, "threshold": -4.424750328063965, "operator": "ge"}, {"budget": 0.55, "threshold": -4.465745449066162, "operator": "ge"}, {"budget": 0.56, "threshold": -4.488899230957031, "operator": "ge"}, {"budget": 0.57, "threshold": -4.557817459106445, "operator": "ge"}, {"budget": 0.58, "threshold": -4.588666915893555, "operator": "ge"}, {"budget": 0.59, "threshold": -4.652579307556152, "operator": "ge"}, {"budget": 0.6, "threshold": -4.746213912963867, "operator": "ge"}, {"budget": 0.61, "threshold": -4.8229827880859375, "operator": "ge"}, {"budget": 0.62, "threshold": -4.883676528930664, "operator": "ge"}, {"budget": 0.63, "threshold": -4.904263019561768, "operator": "ge"}, {"budget": 0.64, "threshold": -4.947712421417236, "operator": "ge"}, {"budget": 0.65, "threshold": -4.964178562164307, "operator": "ge"}, {"budget": 0.66, "threshold": -4.991442680358887, "operator": "ge"}, {"budget": 0.67, "threshold": -5.0391998291015625, "operator": "ge"}, {"budget": 0.68, "threshold": -5.077427864074707, "operator": "ge"}, {"budget": 0.69, "threshold": -5.093221664428711, "operator": "ge"}, {"budget": 0.7, "threshold": -5.11414098739624, "operator": "ge"}, {"budget": 0.71, "threshold": -5.146929740905762, "operator": "ge"}, {"budget": 0.72, "threshold": -5.17294979095459, "operator": "ge"}, {"budget": 0.73, "threshold": -5.189030170440674, "operator": "ge"}, {"budget": 0.74, "threshold": -5.21529483795166, "operator": "ge"}, {"budget": 0.75, "threshold": -5.259051322937012, "operator": "ge"}, {"budget": 0.76, "threshold": -5.273904800415039, "operator": "ge"}, {"budget": 0.77, "threshold": -5.3360772132873535, "operator": "ge"}, {"budget": 0.78, "threshold": -5.376209259033203, "operator": "ge"}, {"budget": 0.79, "threshold": -5.397269248962402, "operator": "ge"}, {"budget": 0.8, "threshold": -5.464372158050537, "operator": "ge"}, {"budget": 0.81, "threshold": -5.570054054260254, "operator": "ge"}, {"budget": 0.82, "threshold": -13.346918106079102, "operator": "ge"}, {"budget": 0.83, "threshold": -13.379218101501465, "operator": "ge"}, {"budget": 0.84, "threshold": -13.398651123046875, "operator": "ge"}, {"budget": 0.85, "threshold": -13.458649635314941, "operator": "ge"}, {"budget": 0.86, "threshold": -13.503206253051758, "operator": "ge"}, {"budget": 0.87, "threshold": -13.558369636535645, "operator": "ge"}, {"budget": 0.88, "threshold": -13.67621898651123, "operator": "ge"}, {"budget": 0.89, "threshold": -13.767268180847168, "operator": "ge"}, {"budget": 0.9, "threshold": -13.806638717651367, "operator": "ge"}, {"budget": 0.91, "threshold": -13.848135948181152, "operator": "ge"}, {"budget": 0.92, "threshold": -13.892945289611816, "operator": "ge"}, {"budget": 0.93, "threshold": -13.930668830871582, "operator": "ge"}, {"budget": 0.94, "threshold": -13.959216117858887, "operator": "ge"}, {"budget": 0.95, "threshold": -13.988587379455566, "operator": "ge"}, {"budget": 0.96, "threshold": -14.034542083740234, "operator": "ge"}, {"budget": 0.97, "threshold": -14.08267879486084, "operator": "ge"}, {"budget": 0.98, "threshold": -14.13174819946289, "operator": "ge"}, {"budget": 0.99, "threshold": -14.173405647277832, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.60983981693364, 18.83301531052656, 19.31709232368603, 20.09761869433221, 20.351032219222777, 20.894375797011516, 21.42707443801643, 22.06771244550552, 22.392601149624237, 22.507779080189067, 22.680823556918675, 22.614184131196723, 22.49484931994141, 23.628692363332178, 23.80782088856623, 23.574690783902717, 23.85205654560493, 24.088055377097763, 24.105063801665743, 24.135198818743124, 24.093959468579772, 24.196539440176004, 24.512582788215244, 23.273117549864274, 23.697010717091235, 23.647810019687537, 23.681940647168506, 24.316624895572264, 25.0846100110806, 25.32559570656859, 25.809218250746234, 26.024402506319593, 26.52203425229741, 27.163906120639297, 27.402471981357724, 27.508470114765117, 27.43767902433968, 27.781782206204507, 28.59399001536098, 28.899335225860717, 29.09897704766602, 28.615202551426037, 28.59231872861715, 28.5117683398264, 28.368764051338967, 28.474182158719774, 28.446664272093464, 28.690897800978753, 28.272633431375176, 28.340437624325094, 28.104997400129246, 27.740358372699518, 27.41479732388823, 27.31768957753424, 27.26190476190476, 27.21181368722352, 27.146639592291766, 27.052761403068686, 27.004433548539776, 26.92247987324426, 26.7614750593474, 26.51559214512278, 26.444994499449948, 26.284089148500616, 26.126686909581647, 26.0715473517918, 25.855579901325104, 24.893384691771793, 24.64653524320066, 24.542502068302277, 24.917245435196595, 24.835099403282083, 24.77994442987649, 24.203184341221153, 23.745106700402843, 23.401615615781203, 23.073595885608704, 22.779003967215093, 22.43404692986815, 22.368557368557365, 22.308619426490953, 22.286913865861234, 22.247451708418247, 22.13814685903184, 22.100197942979783, 22.074503773890275, 22.05257214574103, 22.030164584216312, 21.989323349877367, 21.94346911653983, 21.92249984005033, 21.751410868773828, 21.654189801843053, 21.55164395385602, 21.45077504375399, 21.4178933615, 21.173160487676615, 21.07189002129632, 20.862068779145307, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 20, 32, 43, 54, 66, 78, 91, 100, 109, 116, 122, 127, 134, 139, 143, 149, 155, 161, 164, 167, 170, 175, 179, 186, 190, 195, 201, 209, 212, 218, 223, 227, 233, 238, 244, 246, 250, 254, 257, 263, 266, 268, 270, 273, 274, 276, 279, 280, 282, 283, 283, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 1, 3, 6, 8, 13, 15, 16, 20, 21, 26, 28, 32, 37, 40, 41, 47, 50, 60, 64, 67, 73, 76, 81, 89, 95, 100, 109, 117, 125, 131, 138, 145, 149, 156, 161, 167, 173, 181, 188, 189, 195, 199, 202, 206, 210, 212, 220, 223, 229, 236, 241, 249, 257, 263, 267, 270, 275, 282, 286, 292, 298, 304, 310, 317, 324, 329, 334, 341, 350, 353, 358, 365, 374, 382, 389, 395, 402, 408, 414, 422, 429, 432, 437, 442, 447, 451, 455, 458, 464, 470, 473, 478, 482, 488, 494, 500, 506, 512, 515, 518], "exact_score": [18.254072168965788, 18.60983981693364, 18.83301531052656, 19.31709232368603, 20.09761869433221, 20.351032219222777, 20.894375797011516, 21.42707443801643, 22.06771244550552, 22.392601149624237, 22.507779080189067, 22.680823556918675, 22.614184131196723, 22.49484931994141, 23.628692363332178, 23.80782088856623, 23.574690783902717, 23.85205654560493, 24.088055377097763, 24.105063801665743, 24.135198818743124, 24.093959468579772, 24.196539440176004, 24.512582788215244, 23.273117549864274, 23.697010717091235, 23.647810019687537, 23.681940647168506, 24.316624895572264, 25.0846100110806, 25.32559570656859, 25.809218250746234, 26.024402506319593, 26.52203425229741, 27.163906120639297, 27.402471981357724, 27.508470114765117, 27.43767902433968, 27.781782206204507, 28.59399001536098, 28.899335225860717, 29.09897704766602, 28.615202551426037, 28.59231872861715, 28.5117683398264, 28.368764051338967, 28.474182158719774, 28.446664272093464, 28.690897800978753, 28.272633431375176, 28.340437624325094, 28.104997400129246, 27.740358372699518, 27.41479732388823, 27.31768957753424, 27.26190476190476, 27.21181368722352, 27.146639592291766, 27.052761403068686, 27.004433548539776, 26.92247987324426, 26.7614750593474, 26.51559214512278, 26.444994499449948, 26.284089148500616, 26.126686909581647, 26.0715473517918, 25.855579901325104, 24.893384691771793, 24.64653524320066, 24.542502068302277, 24.917245435196595, 24.835099403282083, 24.77994442987649, 24.203184341221153, 23.745106700402843, 23.401615615781203, 23.073595885608704, 22.779003967215093, 22.43404692986815, 22.368557368557365, 22.308619426490953, 22.286913865861234, 22.247451708418247, 22.13814685903184, 22.100197942979783, 22.074503773890275, 22.05257214574103, 22.030164584216312, 21.989323349877367, 21.94346911653983, 21.92249984005033, 21.751410868773828, 21.654189801843053, 21.55164395385602, 21.45077504375399, 21.4178933615, 21.173160487676615, 21.07189002129632, 20.862068779145307, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.426858465157654, "threshold_area100": 24.43209105893447, "threshold_area25": 22.35547448994994}, "test": {"score": [18.129624718318375, 18.38640402215815, 18.82771445410679, 18.968950692768082, 19.44510553233284, 19.601590800766548, 19.95262488875736, 20.085666021247803, 20.525321962553484, 20.71802411802589, 21.326696425146597, 21.44247712958992, 21.586416504957963, 21.641337795121935, 21.90866456633385, 22.09080598420433, 22.189526653054813, 22.35646543351083, 22.55398375744643, 22.82224374904482, 23.34943498799145, 23.284868856026243, 23.334373077830115, 23.44087107379087, 23.6058612375722, 23.730773350067633, 24.76409723538453, 24.76811534443013, 24.98105179530155, 25.499042589958275, 25.622393425783258, 25.776436011785993, 26.16919434191941, 26.330302946561346, 25.669417362531888, 25.83062769392853, 26.130184785237798, 26.250844168210907, 25.909240815885703, 25.986740647951343, 25.92671890735499, 25.908442127258873, 25.751350658010413, 25.950368971189807, 25.9918753444761, 26.08350947415762, 26.610109085208798, 26.509881765265764, 26.355331511814562, 26.013445216176017, 25.12755733278701, 24.469596986981745, 24.161350054841446, 24.152664039459157, 24.107464171454758, 24.073292587021093, 24.016700176447948, 24.395247965969777, 24.32727506426798, 24.21819758067081, 24.12984963118559, 23.906881225718607, 23.890761113851752, 23.842773639950654, 23.63186745286761, 23.5445033380696, 23.3858500309023, 23.718820061605363, 23.837075134613716, 23.770688048861217, 23.637761561037028, 23.551506252847574, 23.383119882868513, 23.259214391346887, 23.219213324207807, 22.738915180798635, 22.670300075911882, 22.413960910352888, 22.24302125502032, 22.168541463786507, 22.122429366006138, 22.845617705446585, 22.796499245395335, 22.766324999671184, 22.743578462595774, 22.6919238535633, 22.606272261356974, 22.5306703355241, 22.453485317074584, 22.34768278476464, 22.263580162847283, 22.244222616626736, 22.213106751741577, 22.565967410409897, 22.505860354159502, 22.441000623130403, 22.345756389874634, 22.258247355759426, 22.193380361612718, 22.169141781679265, 22.361115727328066], "calls": [0.0, 1.1984021304926764, 2.1637816245006656, 2.862849533954727, 4.094540612516644, 4.8934753661784285, 5.958721704394141, 6.55792276964048, 8.022636484687084, 9.087882822902795, 11.18508655126498, 11.78428761651132, 12.81624500665779, 13.415446071904128, 14.280958721704396, 15.07989347536618, 15.579227696404793, 16.577896138482025, 17.11051930758988, 18.142476697736353, 20.505992010652463, 22.03728362183755, 23.10252996005326, 23.768308921438084, 24.33422103861518, 25.0, 26.897470039946736, 28.86151797603196, 30.25965379494008, 31.424766977363518, 32.7896138482024, 33.55525965379494, 34.42077230359521, 34.78695073235686, 35.81890812250333, 36.61784287616511, 37.51664447403462, 38.115845539280954, 39.4474034620506, 40.21304926764314, 40.47936085219707, 40.812250332889484, 41.577896138482025, 42.543275632490015, 43.40878828229028, 44.17443408788282, 44.87350199733689, 45.67243675099867, 46.47137150466045, 47.37017310252996, 48.8681757656458, 50.3994673768309, 51.398135818908116, 52.4300932090546, 53.99467376830892, 55.12649800266311, 55.59254327563249, 57.35685752330226, 57.98934753661784, 59.08788282290279, 60.61917443408789, 62.150466045272964, 63.1491344873502, 63.81491344873502, 65.34620505992011, 66.04527296937417, 66.91078561917443, 68.94141145139814, 70.3728362183755, 70.87217043941412, 71.53794940079894, 72.83621837549934, 73.90146471371504, 74.66711051930758, 75.63249001331558, 77.29693741677764, 77.62982689747004, 79.39414114513981, 80.22636484687084, 80.52596537949401, 81.25832223701731, 81.95739014647138, 82.72303595206392, 83.62183754993342, 84.32090545938749, 86.01864181091877, 87.0838881491345, 88.2157123834887, 89.81358189081226, 91.17842876165113, 91.711051930759, 92.44340878828228, 93.47536617842877, 94.0745672436751, 94.8069241011984, 95.87217043941412, 96.90412782956058, 97.66977363515313, 98.33555259653795, 98.73501997336885, 100.0], "rescued": [0, 64, 101, 129, 178, 208, 253, 275, 329, 367, 439, 452, 477, 491, 515, 538, 550, 574, 590, 613, 667, 705, 730, 746, 761, 777, 818, 863, 891, 919, 949, 968, 988, 997, 1021, 1041, 1059, 1071, 1091, 1098, 1102, 1105, 1115, 1128, 1137, 1145, 1152, 1154, 1158, 1160, 1166, 1169, 1171, 1172, 1174, 1175, 1175, 1176, 1176, 1177, 1178, 1178, 1179, 1179, 1179, 1179, 1179, 1182, 1188, 1188, 1188, 1189, 1190, 1190, 1190, 1190, 1191, 1192, 1192, 1192, 1192, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1196, 1196, 1196, 1196, 1196, 1197, 1197, 1198], "harmed": [0, 4, 9, 16, 31, 41, 56, 63, 86, 100, 134, 147, 158, 167, 181, 201, 207, 225, 235, 253, 301, 336, 363, 377, 387, 401, 439, 484, 517, 539, 573, 594, 613, 620, 648, 669, 688, 700, 721, 730, 740, 745, 765, 772, 790, 802, 811, 821, 839, 869, 906, 942, 966, 985, 1016, 1038, 1049, 1089, 1104, 1130, 1173, 1214, 1241, 1258, 1302, 1323, 1344, 1395, 1437, 1458, 1472, 1506, 1532, 1553, 1579, 1625, 1630, 1683, 1704, 1711, 1730, 1745, 1762, 1778, 1794, 1823, 1845, 1862, 1888, 1913, 1926, 1939, 1958, 1976, 1994, 2019, 2040, 2061, 2077, 2082, 2109], "exact_score": [18.129624718318375, 18.31238862853331, 18.771738454518246, 18.96704446012019, 19.405903189561528, 19.634854587562174, 19.960742943310684, 20.189346338894985, 20.5342680305436, 20.691658628060793, 20.91539809094026, 21.06851726152068, 21.490474844371864, 21.588859339388193, 21.683935360909523, 22.063873788420345, 22.31606251265228, 22.497209528194176, 22.804675828060315, 23.069445300554044, 23.238861067431934, 23.41379436425251, 23.205025196728325, 23.330145493301398, 23.51975482583294, 23.730773350067633, 24.663352758557185, 24.727062074147828, 25.009455140149186, 24.739308947265563, 24.97477815740464, 25.3814533258958, 25.636841652599852, 25.724356237034367, 26.004795235796546, 25.875685825841988, 25.724968558510568, 26.048929139167075, 26.190049986032648, 25.828254405779095, 25.96624082979074, 25.806380516192952, 25.829051477382393, 25.893701975760674, 26.06746300515485, 26.632018020244047, 26.423354053429104, 26.221583289906192, 25.76871333312502, 25.068272358948334, 24.52253762474808, 24.321218099936356, 24.174163812676472, 24.073107333045954, 24.107464171454758, 24.083258577118407, 23.96187353269137, 23.88152424001828, 24.32727506426798, 24.22438918321397, 24.20887018393883, 24.08944654269131, 23.914699991490068, 23.903063317050005, 23.811875617962862, 23.696316381841324, 23.570210615082615, 23.383817923607033, 23.28563157377971, 23.716876552924358, 23.85289510858556, 23.767990746163917, 23.594184711013902, 23.59830055444246, 23.376729283514955, 23.242606935432875, 23.20218310364575, 22.755446846486233, 22.637989565345162, 22.47284899413373, 22.299814565944843, 22.153534922015066, 22.843468854544604, 22.785569049626172, 22.758859422694037, 22.717030391458948, 22.69348733073653, 22.611305006274062, 22.537097658555176, 22.4945996486817, 22.399369061575612, 22.369269393803243, 22.25429992200557, 22.22348903237154, 22.572231096615997, 22.50023627850915, 22.439227115283895, 22.345756389874634, 22.203290840986, 22.139235597553764, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.339768743040715, "threshold_area100": 23.373090135979602, "threshold_area25": 21.45974544646039}, "selected_budget": 41, "selected": {"val": {"score": 29.09897704766602, "calls": 40.94151212553495, "rescued": 263, "harmed": 195}, "test": {"score": 25.908442127258873, "calls": 40.812250332889484, "rescued": 1105, "harmed": 745}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.0026021003723145, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8950729370117188, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8392665982246399, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7947367429733276, "operator": "ge"}, {"budget": 0.05, "threshold": 0.7404524087905884, "operator": "ge"}, {"budget": 0.06, "threshold": 0.6841170787811279, "operator": "ge"}, {"budget": 0.07, "threshold": 0.6639848947525024, "operator": "ge"}, {"budget": 0.08, "threshold": 0.565403938293457, "operator": "ge"}, {"budget": 0.09, "threshold": 0.503646731376648, "operator": "ge"}, {"budget": 0.1, "threshold": 0.32563120126724243, "operator": "ge"}, {"budget": 0.11, "threshold": 0.27536651492118835, "operator": "ge"}, {"budget": 0.12, "threshold": 0.2565228343009949, "operator": "ge"}, {"budget": 0.13, "threshold": 0.24165880680084229, "operator": "ge"}, {"budget": 0.14, "threshold": 0.2252940833568573, "operator": "ge"}, {"budget": 0.15, "threshold": 0.21246761083602905, "operator": "ge"}, {"budget": 0.16, "threshold": 0.2054876685142517, "operator": "ge"}, {"budget": 0.17, "threshold": 0.19509071111679077, "operator": "ge"}, {"budget": 0.18, "threshold": 0.17283505201339722, "operator": "ge"}, {"budget": 0.19, "threshold": 0.15378475189208984, "operator": "ge"}, {"budget": 0.2, "threshold": 0.1418527364730835, "operator": "ge"}, {"budget": 0.21, "threshold": 0.11823511123657227, "operator": "ge"}, {"budget": 0.22, "threshold": 0.10003858804702759, "operator": "ge"}, {"budget": 0.23, "threshold": 0.08552873134613037, "operator": "ge"}, {"budget": 0.24, "threshold": 0.080780029296875, "operator": "ge"}, {"budget": 0.25, "threshold": 0.056996703147888184, "operator": "ge"}, {"budget": 0.26, "threshold": 0.037813663482666016, "operator": "ge"}, {"budget": 0.27, "threshold": -7.82012939453125e-05, "operator": "ge"}, {"budget": 0.28, "threshold": -0.02129039168357849, "operator": "ge"}, {"budget": 0.29, "threshold": -0.03655219078063965, "operator": "ge"}, {"budget": 0.3, "threshold": -0.06606048345565796, "operator": "ge"}, {"budget": 0.31, "threshold": -0.08245301246643066, "operator": "ge"}, {"budget": 0.32, "threshold": -0.09434962272644043, "operator": "ge"}, {"budget": 0.33, "threshold": -0.11762619018554688, "operator": "ge"}, {"budget": 0.34, "threshold": -0.13705366849899292, "operator": "ge"}, {"budget": 0.35, "threshold": -0.15302696824073792, "operator": "ge"}, {"budget": 0.36, "threshold": -0.1733154058456421, "operator": "ge"}, {"budget": 0.37, "threshold": -0.1918889284133911, "operator": "ge"}, {"budget": 0.38, "threshold": -0.20870083570480347, "operator": "ge"}, {"budget": 0.39, "threshold": -0.2430400848388672, "operator": "ge"}, {"budget": 0.4, "threshold": -0.2663728594779968, "operator": "ge"}, {"budget": 0.41, "threshold": -0.3001757860183716, "operator": "ge"}, {"budget": 0.42, "threshold": -0.3371405601501465, "operator": "ge"}, {"budget": 0.43, "threshold": -0.3721202611923218, "operator": "ge"}, {"budget": 0.44, "threshold": -0.4464093744754791, "operator": "ge"}, {"budget": 0.45, "threshold": -0.5356491804122925, "operator": "ge"}, {"budget": 0.46, "threshold": -0.8699411749839783, "operator": "ge"}, {"budget": 0.47, "threshold": -1.4225068092346191, "operator": "ge"}, {"budget": 0.48, "threshold": -1.6857082843780518, "operator": "ge"}, {"budget": 0.49, "threshold": -2.0091352462768555, "operator": "ge"}, {"budget": 0.5, "threshold": -2.1919944286346436, "operator": "ge"}, {"budget": 0.51, "threshold": -2.49438214302063, "operator": "ge"}, {"budget": 0.52, "threshold": -4.1258440017700195, "operator": "ge"}, {"budget": 0.53, "threshold": -4.18553352355957, "operator": "ge"}, {"budget": 0.54, "threshold": -4.230530738830566, "operator": "ge"}, {"budget": 0.55, "threshold": -4.283965110778809, "operator": "ge"}, {"budget": 0.56, "threshold": -4.31519079208374, "operator": "ge"}, {"budget": 0.57, "threshold": -4.355724334716797, "operator": "ge"}, {"budget": 0.58, "threshold": -4.383542060852051, "operator": "ge"}, {"budget": 0.59, "threshold": -4.427788257598877, "operator": "ge"}, {"budget": 0.6, "threshold": -4.456594467163086, "operator": "ge"}, {"budget": 0.61, "threshold": -4.483898162841797, "operator": "ge"}, {"budget": 0.62, "threshold": -4.513663291931152, "operator": "ge"}, {"budget": 0.63, "threshold": -4.53985595703125, "operator": "ge"}, {"budget": 0.64, "threshold": -4.5677947998046875, "operator": "ge"}, {"budget": 0.65, "threshold": -4.627599239349365, "operator": "ge"}, {"budget": 0.66, "threshold": -4.672073841094971, "operator": "ge"}, {"budget": 0.67, "threshold": -4.714056491851807, "operator": "ge"}, {"budget": 0.68, "threshold": -4.739791393280029, "operator": "ge"}, {"budget": 0.69, "threshold": -4.764472484588623, "operator": "ge"}, {"budget": 0.7, "threshold": -4.793038845062256, "operator": "ge"}, {"budget": 0.71, "threshold": -4.8004021644592285, "operator": "ge"}, {"budget": 0.72, "threshold": -4.819255828857422, "operator": "ge"}, {"budget": 0.73, "threshold": -4.856290817260742, "operator": "ge"}, {"budget": 0.74, "threshold": -4.913658142089844, "operator": "ge"}, {"budget": 0.75, "threshold": -4.932194709777832, "operator": "ge"}, {"budget": 0.76, "threshold": -4.962860107421875, "operator": "ge"}, {"budget": 0.77, "threshold": -4.995879173278809, "operator": "ge"}, {"budget": 0.78, "threshold": -5.030535697937012, "operator": "ge"}, {"budget": 0.79, "threshold": -5.062804698944092, "operator": "ge"}, {"budget": 0.8, "threshold": -5.106877326965332, "operator": "ge"}, {"budget": 0.81, "threshold": -5.225955963134766, "operator": "ge"}, {"budget": 0.82, "threshold": -12.821975708007812, "operator": "ge"}, {"budget": 0.83, "threshold": -12.856985092163086, "operator": "ge"}, {"budget": 0.84, "threshold": -12.883783340454102, "operator": "ge"}, {"budget": 0.85, "threshold": -12.917034149169922, "operator": "ge"}, {"budget": 0.86, "threshold": -12.936763763427734, "operator": "ge"}, {"budget": 0.87, "threshold": -13.000905990600586, "operator": "ge"}, {"budget": 0.88, "threshold": -13.029715538024902, "operator": "ge"}, {"budget": 0.89, "threshold": -13.058515548706055, "operator": "ge"}, {"budget": 0.9, "threshold": -13.078203201293945, "operator": "ge"}, {"budget": 0.91, "threshold": -13.112083435058594, "operator": "ge"}, {"budget": 0.92, "threshold": -13.163244247436523, "operator": "ge"}, {"budget": 0.93, "threshold": -13.222599029541016, "operator": "ge"}, {"budget": 0.94, "threshold": -13.31731128692627, "operator": "ge"}, {"budget": 0.95, "threshold": -13.41817855834961, "operator": "ge"}, {"budget": 0.96, "threshold": -13.520444869995117, "operator": "ge"}, {"budget": 0.97, "threshold": -13.62855052947998, "operator": "ge"}, {"budget": 0.98, "threshold": -13.687875747680664, "operator": "ge"}, {"budget": 0.99, "threshold": -13.784873962402344, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.32946042975213, 19.30434700184981, 19.861515128584706, 20.196797072141635, 20.49233861231151, 20.982935547100194, 21.46064490871175, 22.086247086247084, 22.5063829359495, 22.27125465570844, 22.503731343283583, 22.68765257991712, 23.99875950350035, 23.944989232165348, 24.27577355052693, 24.459909350452442, 24.56193398348314, 24.4907381175725, 24.647631121687724, 24.605877438694773, 24.517971272885344, 24.486863711001643, 24.701223554583184, 23.631408381606505, 23.697010717091235, 23.824493035353605, 23.681940647168506, 24.675398542017142, 24.891873909450844, 25.364582061344414, 25.78942441851879, 25.848794063079776, 26.107487625812233, 27.03977329220327, 27.598749899100504, 27.428595283343093, 27.481823929192345, 26.94319600499376, 26.96738253680177, 26.99732016321463, 27.01749319780276, 27.953494553861752, 28.169590643274855, 28.468413465997266, 28.73963106534258, 28.722189653104074, 28.665595885818483, 28.59302694532502, 28.797308337749516, 28.271880945622897, 28.086742562597294, 27.740358372699518, 27.428571428571423, 27.321838485303505, 27.22451209992193, 27.170678053830226, 27.094967130840462, 27.028935976304393, 26.972547950186343, 26.743826092750194, 26.675659077948854, 26.599340704423525, 26.41641302342741, 26.496368430859267, 26.33253948185495, 26.101300892506206, 25.603666194870232, 25.072303389748786, 25.23168283394812, 25.082017691187996, 24.59832891986403, 24.54292079325111, 24.392082685368933, 23.93119159633823, 23.89091142460415, 23.669706636538816, 23.624457111079185, 23.28075047953964, 22.710370305043554, 22.35974598608942, 22.30574625757498, 22.186341665130463, 22.144086512156097, 22.117641051610097, 22.004450098983675, 21.895771317023687, 21.80778334101303, 21.771918136724583, 21.7368043633071, 21.715763359651227, 21.684641541238662, 21.66199929819061, 21.622750950752607, 21.60589110719529, 21.178556253292054, 21.15010223344012, 21.061308912978454, 20.955782103494684, 20.685604348672708, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 24, 38, 49, 57, 70, 80, 92, 101, 109, 115, 122, 131, 137, 143, 148, 154, 158, 163, 166, 170, 174, 178, 183, 186, 191, 195, 202, 207, 213, 217, 220, 225, 231, 235, 240, 245, 249, 252, 255, 258, 263, 264, 266, 272, 275, 276, 278, 281, 281, 283, 283, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 4, 7, 8, 12, 13, 17, 19, 21, 22, 28, 30, 33, 36, 41, 44, 50, 57, 61, 63, 69, 74, 80, 84, 92, 95, 101, 109, 117, 122, 130, 134, 140, 148, 154, 158, 162, 166, 173, 178, 182, 187, 191, 196, 201, 204, 210, 216, 223, 228, 234, 242, 249, 256, 263, 269, 273, 278, 283, 287, 292, 297, 303, 311, 316, 323, 330, 336, 343, 348, 355, 361, 367, 374, 381, 385, 397, 402, 408, 415, 422, 429, 435, 441, 445, 450, 455, 458, 463, 468, 472, 477, 481, 488, 491, 495, 500, 504, 510, 514, 518], "exact_score": [18.254072168965788, 18.32946042975213, 19.30434700184981, 19.861515128584706, 20.196797072141635, 20.49233861231151, 20.982935547100194, 21.46064490871175, 22.086247086247084, 22.5063829359495, 22.27125465570844, 22.503731343283583, 22.68765257991712, 23.99875950350035, 23.944989232165348, 24.27577355052693, 24.459909350452442, 24.56193398348314, 24.4907381175725, 24.647631121687724, 24.605877438694773, 24.517971272885344, 24.486863711001643, 24.701223554583184, 23.631408381606505, 23.697010717091235, 23.824493035353605, 23.681940647168506, 24.675398542017142, 24.891873909450844, 25.364582061344414, 25.78942441851879, 25.848794063079776, 26.107487625812233, 27.03977329220327, 27.598749899100504, 27.428595283343093, 27.481823929192345, 26.94319600499376, 26.96738253680177, 26.99732016321463, 27.01749319780276, 27.953494553861752, 28.169590643274855, 28.468413465997266, 28.73963106534258, 28.722189653104074, 28.665595885818483, 28.59302694532502, 28.797308337749516, 28.271880945622897, 28.086742562597294, 27.740358372699518, 27.428571428571423, 27.321838485303505, 27.22451209992193, 27.170678053830226, 27.094967130840462, 27.028935976304393, 26.972547950186343, 26.743826092750194, 26.675659077948854, 26.599340704423525, 26.41641302342741, 26.496368430859267, 26.33253948185495, 26.101300892506206, 25.603666194870232, 25.072303389748786, 25.23168283394812, 25.082017691187996, 24.59832891986403, 24.54292079325111, 24.392082685368933, 23.93119159633823, 23.89091142460415, 23.669706636538816, 23.624457111079185, 23.28075047953964, 22.710370305043554, 22.35974598608942, 22.30574625757498, 22.186341665130463, 22.144086512156097, 22.117641051610097, 22.004450098983675, 21.895771317023687, 21.80778334101303, 21.771918136724583, 21.7368043633071, 21.715763359651227, 21.684641541238662, 21.66199929819061, 21.622750950752607, 21.60589110719529, 21.178556253292054, 21.15010223344012, 21.061308912978454, 20.955782103494684, 20.685604348672708, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.42385146114671, "threshold_area100": 24.42920580522528, "threshold_area25": 22.524157140054037}, "test": {"score": [18.129624718318375, 18.167549017163424, 18.53117756994723, 18.905878486252778, 19.1213726694963, 19.367388670744504, 19.930429143967668, 19.980032311825678, 20.437701959796883, 20.65728135005672, 21.076867551017532, 21.30077599050859, 21.352744467512494, 21.899163690163714, 22.05526646956586, 22.432073283389517, 22.490256265764998, 22.638090153385896, 23.044842486123013, 23.294552024880755, 23.426649558040246, 23.385324094534226, 23.485566518108094, 23.64034514146563, 23.615904523679017, 24.55939572356768, 24.76532198853594, 24.76811534443013, 24.975207943034135, 25.360902255639097, 25.672298605066352, 25.82336922499061, 25.88501414160667, 26.235933704146092, 25.568994951145978, 25.684658932751383, 25.82805815619651, 25.991470328184324, 26.117139695207918, 25.660845703808103, 25.789592693261827, 25.901310625925113, 25.96252965530757, 26.005874575363702, 25.875425127727418, 25.707853728930235, 25.874058848300546, 26.43908488345559, 26.361281752527493, 25.349415780856955, 25.067239054325324, 24.483673605710717, 24.155599970603685, 24.20588522114275, 24.16111851399666, 24.009950419169645, 23.92508243078229, 23.829902710290384, 23.773690104164686, 24.0767004324011, 23.92235525898211, 23.903303519159664, 23.92056083952926, 23.87631589200457, 23.607776361298235, 23.428291689648628, 23.23542026090389, 23.1987124527674, 23.182872314267644, 23.01814023500503, 23.3046971117989, 23.304383714639698, 23.132049005574327, 22.94208485741603, 22.569092579977138, 22.47221052540896, 22.361661926947622, 22.26106606344255, 22.24274025402827, 22.53142008617241, 22.4543667447396, 22.39954662912799, 22.793111627940537, 22.75840008553996, 22.730423589247355, 22.682879209779987, 22.654191288935728, 22.49806668739097, 22.44411520981277, 22.350929330801268, 22.331106429033923, 22.216116085070723, 22.474076110831938, 22.445318428303118, 22.29075032812417, 22.218419313477547, 22.15620092004035, 22.171000634349916, 22.072803018711554, 22.3696228699906, 22.361115727328066], "calls": [0.0, 0.4993342210386152, 1.5645805592543276, 2.1970705725699067, 2.929427430093209, 4.094540612516644, 5.25965379494008, 5.725699067909454, 7.157123834886818, 7.956058588548602, 10.785619174434089, 11.78428761651132, 12.383488681757656, 13.11584553928096, 13.81491344873502, 14.647137150466044, 15.046604527296935, 15.612516644474034, 17.11051930758988, 18.575233022636482, 19.474034620505993, 21.604527296937416, 22.836218375499335, 23.801597869507322, 24.06790945406125, 25.332889480692412, 26.73102529960053, 28.86151797603196, 30.359520639147803, 31.158455392809586, 32.92276964047936, 33.68841544607191, 34.187749667110516, 35.319573901464715, 36.284953395472705, 37.18375499334221, 37.71637816245006, 38.41544607190413, 38.8814913448735, 39.647137150466044, 40.54593874833555, 41.31158455392809, 42.07723035952064, 42.743009320905465, 43.60852197070572, 44.17443408788282, 45.23968042609853, 46.27163781624501, 46.60452729693741, 47.93608521970706, 49.134487350199734, 50.499334221038616, 51.46471371504661, 52.36351531291611, 53.22902796271638, 54.76031957390146, 55.89214380825565, 57.05725699067909, 57.922769640479366, 59.45406125166445, 60.38615179760319, 61.01864181091877, 62.083888149134495, 62.88282290279628, 63.914780292942744, 66.17842876165113, 67.90945406125167, 69.8069241011984, 70.57256990679095, 71.47137150466045, 72.53661784287615, 72.83621837549934, 73.7017310252996, 74.76697736351531, 76.73102529960053, 77.33022636484687, 78.36218375499334, 79.19440745672436, 79.69374167776299, 80.15978695073235, 80.89214380825565, 81.69107856191744, 82.68974700399467, 83.48868175765645, 84.15446071904128, 85.01997336884153, 85.81890812250333, 88.44873501997337, 89.24766977363515, 90.21304926764314, 90.87882822902796, 91.51131824234355, 92.90945406125167, 94.00798934753661, 95.27296937416777, 96.33821571238349, 97.17043941411451, 98.60186418109188, 99.16777629826898, 99.73368841544608, 100.0], "rescued": [0, 29, 86, 114, 150, 197, 245, 262, 313, 346, 428, 456, 472, 496, 514, 537, 547, 562, 598, 633, 649, 701, 727, 749, 756, 789, 817, 863, 892, 910, 952, 970, 981, 1004, 1022, 1038, 1050, 1060, 1067, 1079, 1091, 1101, 1105, 1111, 1121, 1124, 1141, 1154, 1156, 1160, 1165, 1170, 1171, 1173, 1174, 1174, 1175, 1175, 1175, 1176, 1176, 1177, 1179, 1181, 1181, 1182, 1182, 1185, 1186, 1186, 1189, 1190, 1190, 1191, 1191, 1191, 1191, 1192, 1192, 1193, 1193, 1193, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1196, 1196, 1196, 1196, 1196, 1197, 1197, 1198, 1198], "harmed": [0, 2, 19, 26, 37, 50, 65, 72, 93, 101, 131, 144, 155, 169, 183, 198, 205, 214, 248, 282, 297, 339, 363, 378, 383, 411, 439, 484, 519, 533, 573, 591, 602, 625, 645, 660, 674, 683, 694, 713, 724, 734, 738, 747, 767, 782, 804, 827, 837, 878, 907, 946, 967, 987, 1005, 1038, 1064, 1092, 1109, 1149, 1172, 1191, 1222, 1241, 1271, 1330, 1376, 1428, 1451, 1475, 1503, 1513, 1538, 1570, 1618, 1634, 1656, 1678, 1691, 1704, 1720, 1740, 1763, 1781, 1796, 1813, 1833, 1881, 1894, 1912, 1924, 1939, 1961, 1980, 2010, 2029, 2047, 2080, 2094, 2103, 2109], "exact_score": [18.129624718318375, 18.36012202893026, 18.838027860733106, 19.10107689951704, 19.33637397161169, 19.632868474099624, 20.045990084830827, 20.394254118416626, 20.66563118425287, 20.83941459271143, 21.035093485769796, 21.115556788237157, 21.31062492314311, 21.844511137070718, 22.069849814819193, 22.481037503608796, 22.869740779736592, 22.973933146554675, 23.188783008452198, 23.39755519687032, 23.55927651841733, 23.73175424904053, 23.441791511986303, 23.480246984661267, 23.591305294712896, 24.55124112602639, 24.723346906295465, 24.751184271732217, 25.029887624689117, 24.74016540885146, 24.92596052791989, 25.33197374957513, 25.579284583842004, 25.672298605066352, 25.803241502145518, 26.037306529822224, 25.457951750889457, 25.643568378290045, 25.902597956281216, 26.145543871529924, 25.672873077262746, 25.92446061756757, 25.96252965530757, 26.017309787267738, 25.746274338280916, 25.917157737034874, 26.530899569266563, 26.229436154322826, 25.346013340172842, 25.109117175648105, 24.779597625696994, 24.325480705458517, 24.229884565825778, 24.173741167204113, 24.063238569727055, 23.992103850236358, 23.914393879835412, 23.83642505178362, 23.766343904207, 24.17354379536655, 24.02957137054357, 23.906625531076383, 23.926101211179468, 23.80631606210544, 23.602356094536265, 23.453450523680925, 23.440983659040324, 23.347293960490816, 23.23049073992916, 23.21688818365458, 23.188837052151502, 23.12145821478655, 22.95645441840167, 23.239529962619873, 23.118284654366494, 22.927978522309875, 22.763546183574206, 22.508036378404746, 22.380280628208208, 22.28997078570388, 22.634521847460427, 22.450481540275653, 22.843902876586505, 22.781742920503223, 22.737788739243197, 22.682879209779987, 22.65063789638336, 22.617662041697457, 22.50341610787735, 22.469062549809998, 22.40678721903769, 22.279932165698295, 22.20827138550632, 22.47272789317568, 22.445318428303118, 22.394549663726888, 22.27338366302261, 22.16212701648822, 22.140984449053697, 22.1113853301337, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.313126677558937, "threshold_area100": 23.29261199942991, "threshold_area25": 21.529497856826968}, "selected_budget": 49, "selected": {"val": {"score": 28.797308337749516, "calls": 48.93009985734665, "rescued": 281, "harmed": 228}, "test": {"score": 25.349415780856955, "calls": 47.93608521970706, "rescued": 1160, "harmed": 878}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 28.728969883242918, "calls": 46.932952924393724}, "test": {"score": 26.237361710647345, "calls": 46.23834886817576}}, "old_normalized_interior": {"val": {"score": 28.68713969706174, "calls": 44.935805991440795}, "test": {"score": 26.4896588669812, "calls": 44.40745672436751}}, "old_normalized_endpoints": {"val": {"score": 28.68713969706174, "calls": 44.935805991440795}, "test": {"score": 26.4896588669812, "calls": 44.40745672436751}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.588924171039547, 18.911363596073453, 19.93346519952813, 20.220819570105927, 20.651489334209305, 20.97540684391932, 21.489575045031632, 21.93198438781478, 22.37418168817232, 22.39912914649129, 22.599232414801502, 22.79074852903036, 22.814310001369957, 24.05671491566761, 24.15560338332065, 24.52104062329461, 24.799672399672403, 25.596350920925076, 26.035118534040013, 26.38096394924104, 26.780038331121926, 25.278321619435147, 25.043284014482825, 25.138123278824466, 25.637123624206147, 26.27555349062702, 26.201447790411514, 26.66159643288103, 26.870406153710107, 26.768216497763138, 26.931257163008027, 27.021993258804073, 26.58510387466925, 26.337669570752276, 26.940372945199563, 26.61157193886352, 26.773161049476833, 26.864389233954455, 26.63275178476261, 26.977099932302245, 27.01749319780276, 27.08169934640523, 27.804539487048263, 27.800918716910616, 28.68713969706174, 28.64514777488093, 28.665595885818483, 28.575719918672192, 28.388487788051364, 28.30888807066103, 28.086742562597294, 28.016319825485354, 27.887136259462466, 27.79081551319715, 27.140890229203286, 27.051000580412342, 27.00737858218173, 26.827421594418666, 26.745888794717697, 26.56481481481482, 26.491968301563507, 26.260621064857265, 26.068171310889753, 25.987345987345982, 25.828748853880136, 25.324402477692242, 25.39280498400902, 25.164434748484133, 25.028742659405275, 24.99971990935248, 24.493980963045914, 24.087566844919785, 23.945793786841005, 23.568501306108242, 23.226808254858746, 22.910910372912195, 22.617804197400222, 22.763137330723186, 22.63540617028989, 22.355966905585227, 22.30574625757498, 22.186341665130463, 22.132519831906333, 22.0973837967703, 21.995948032327473, 21.819100999389512, 21.774854025056246, 21.763773385703356, 21.71304705336558, 21.68419655336215, 21.673993252594347, 21.43275895041578, 21.415444697236687, 21.195311125580147, 21.17101057290386, 21.153209059775243, 21.058171660629537, 20.84590333885647, 20.818908134736922, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.588924171039547, 18.911363596073453, 19.93346519952813, 20.220819570105927, 20.651489334209305, 20.97540684391932, 21.489575045031632, 21.93198438781478, 22.37418168817232, 22.39912914649129, 22.599232414801502, 22.79074852903036, 22.814310001369957, 24.05671491566761, 24.15560338332065, 24.52104062329461, 24.799672399672403, 25.596350920925076, 26.035118534040013, 26.38096394924104, 26.780038331121926, 25.278321619435147, 25.043284014482825, 25.138123278824466, 25.637123624206147, 26.27555349062702, 26.201447790411514, 26.66159643288103, 26.870406153710107, 26.768216497763138, 26.931257163008027, 27.021993258804073, 26.58510387466925, 26.337669570752276, 26.940372945199563, 26.61157193886352, 26.773161049476833, 26.864389233954455, 26.63275178476261, 26.977099932302245, 27.01749319780276, 27.08169934640523, 27.804539487048263, 27.800918716910616, 28.68713969706174, 28.64514777488093, 28.665595885818483, 28.575719918672192, 28.388487788051364, 28.30888807066103, 28.086742562597294, 28.016319825485354, 27.887136259462466, 27.79081551319715, 27.140890229203286, 27.051000580412342, 27.00737858218173, 26.827421594418666, 26.745888794717697, 26.56481481481482, 26.491968301563507, 26.260621064857265, 26.068171310889753, 25.987345987345982, 25.828748853880136, 25.324402477692242, 25.39280498400902, 25.164434748484133, 25.028742659405275, 24.99971990935248, 24.493980963045914, 24.087566844919785, 23.945793786841005, 23.568501306108242, 23.226808254858746, 22.910910372912195, 22.617804197400222, 22.763137330723186, 22.63540617028989, 22.355966905585227, 22.30574625757498, 22.186341665130463, 22.132519831906333, 22.0973837967703, 21.995948032327473, 21.819100999389512, 21.774854025056246, 21.763773385703356, 21.71304705336558, 21.68419655336215, 21.673993252594347, 21.43275895041578, 21.415444697236687, 21.195311125580147, 21.17101057290386, 21.153209059775243, 21.058171660629537, 20.84590333885647, 20.818908134736922, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 23.10655674196965, "exact_area100": 24.51960526794477}, "test": {"score": [18.129624718318375, 18.428059472167597, 18.495650550668955, 18.49599471531067, 18.833664299280198, 19.326764107480265, 20.002547756840087, 20.257735490753177, 20.63631599773279, 20.739643136865325, 21.079059712250224, 21.413259814992593, 21.57949028809353, 22.121494417485856, 22.368571016255185, 22.56178452572832, 22.911045487679097, 23.136159311625438, 23.205301979573825, 23.844927208155358, 24.297928319058105, 24.42360137540365, 24.255238323009497, 24.45582901698779, 24.972426369379722, 25.240151120528598, 25.472724814967385, 25.805444937936556, 26.07359768905153, 26.411767531524227, 26.795382591980477, 26.91041690611141, 26.940194805039297, 27.36658294974402, 27.594757433038687, 27.649591557580294, 27.322562348696493, 27.339964952706197, 27.33505654170802, 27.38421779370055, 27.459756532597, 26.926475290976725, 27.087503240860773, 26.62250744073979, 26.592814598294034, 26.4896588669812, 26.506834941186725, 26.485022184589642, 26.345451077975696, 25.43970274423821, 25.14117458450078, 24.50472730476848, 24.17269327314186, 24.149916132804087, 24.076699644634672, 24.019480304887598, 23.940367313496104, 23.75314890624898, 23.691298036393718, 23.619877217695716, 23.49322919724246, 23.35164284691979, 23.77012881171457, 23.650165368787935, 24.08238679492665, 23.90315442035307, 23.84249382076205, 23.988999369043682, 23.91983762158605, 23.62654841971568, 23.471713490892153, 23.358813946640698, 23.27336996236721, 23.138212193355052, 23.054422271521176, 22.886719769310847, 22.794205686730933, 22.762128748383557, 22.379661120868917, 22.31352035642172, 22.55003763687067, 22.851645947148867, 22.804848698792306, 22.786904569374393, 22.68982454209754, 22.61327617196493, 22.56912720424754, 22.341352652809388, 22.251443242707346, 22.207088550357977, 22.46450512630542, 22.43769823592691, 22.410503514313916, 22.399742462459677, 22.287707117460407, 22.223925645010787, 22.18745373127515, 22.463677017933634, 22.406670536928935, 22.37145553760741, 22.361115727328066], "calls": [0.0, 0.7989347536617843, 1.1318242343541944, 1.6977363515312915, 2.862849533954727, 3.5286284953395475, 4.760319573901465, 5.625832223701731, 7.157123834886818, 8.322237017310254, 9.786950732356857, 11.55126498002663, 12.017310252996005, 12.88282290279627, 13.84820239680426, 14.547270306258323, 15.446071904127828, 16.711051930758988, 17.842876165113182, 18.508655126498002, 19.973368841544605, 20.272969374167776, 21.637816245006658, 22.70306258322237, 24.533954727030626, 26.59786950732357, 28.095872170439414, 28.894806924101196, 29.993342210386153, 30.825565912117174, 31.790945406125164, 32.52330226364847, 33.75499334221039, 34.85352862849534, 35.719041278295606, 36.01864181091877, 37.08388814913449, 37.383488681757655, 38.41544607190413, 39.41411451398136, 40.046604527296935, 41.577896138482025, 42.24367509986685, 42.94274300932091, 44.14114513981358, 44.40745672436751, 45.23968042609853, 46.27163781624501, 46.47137150466045, 47.836218375499335, 49.334221038615176, 50.56591211717709, 51.76431424766977, 52.629826897470046, 53.8948069241012, 54.460719041278296, 55.15978695073236, 56.724367509986685, 58.32223701731025, 59.35419440745673, 59.92010652463382, 61.784287616511314, 62.64980026631159, 63.61517976031957, 64.71371504660452, 65.47936085219706, 66.17842876165113, 67.34354194407457, 68.64181091877496, 71.40479360852197, 72.76964047936086, 74.06790945406125, 75.03328894806924, 75.79893475366178, 76.43142476697736, 77.16378162450066, 77.89613848202397, 78.52862849533955, 79.22769640479362, 79.76031957390146, 80.5925432756325, 81.8242343541944, 82.62316910785619, 83.02263648468708, 84.52063914780294, 85.48601864181093, 86.45139813581892, 88.01597869507324, 89.34753661784288, 90.34620505992011, 91.21171770972038, 92.0439414114514, 93.00932090545939, 93.47536617842877, 95.50599201065246, 96.37150466045273, 97.43675099866844, 98.33555259653795, 98.73501997336885, 99.50066577896138, 100.0], "exact_score": [18.129624718318375, 18.456305741513486, 18.621752709149998, 18.888525284100158, 19.42199202508682, 20.08391794175031, 20.349935268145032, 20.593918572863814, 20.708750639147734, 20.890084251281866, 21.055927694465733, 21.235136288059014, 21.55369049449188, 22.131125720432763, 22.47133546968657, 22.601142037230247, 22.99029480207976, 23.141162369107025, 23.44737329748332, 23.984185238035526, 24.297928319058105, 24.127226326900004, 24.26191028576414, 24.558127627774226, 24.848517803986862, 25.138097295092308, 25.314014007783648, 25.32617477575766, 25.375514305724735, 25.8259978136588, 26.07359768905153, 26.435106473232977, 26.832282497089555, 26.787701580282274, 27.070243938549837, 27.398935025568065, 27.634807145755385, 27.289455450686102, 27.319737454016995, 27.362733334487245, 27.459756532597, 26.94423027437623, 27.0911968424064, 26.62250744073979, 26.592814598294034, 26.45681093800789, 26.504369453402326, 26.249672529757902, 25.450986441684186, 25.19580289784053, 24.783116245404575, 24.268293170605578, 24.145117152759386, 24.16250563377585, 24.067287202493507, 23.958069578326917, 23.834347761418904, 23.76544703120031, 23.706449521957545, 23.644906713762165, 23.489894774228404, 23.47255619941018, 23.339832847440054, 23.7329796335493, 23.593918142511665, 24.047100639740886, 23.852527900608738, 23.926243098795265, 23.91320906163717, 23.89969239284308, 23.830996092340474, 23.727884732159822, 23.557754454861488, 23.445860180684665, 23.36291901167519, 23.27336996236721, 23.173310621859947, 22.893213426641445, 22.790543487946785, 22.482989513338218, 22.62219153041747, 22.983508186654795, 22.843902876586505, 22.78897292559261, 22.73219076566638, 22.65575754401757, 22.596329330264158, 22.554016727486218, 22.341352652809388, 22.26049898112601, 22.21325216679261, 22.099313894040513, 22.43906809894061, 22.41183658357258, 22.370267892977036, 22.320502691928123, 22.326216920890833, 22.206834583980008, 22.115862595483108, 22.383084599107534, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.790084916788082, "exact_area100": 23.66159701691065}}, "old_normalized_endpoint_budget": 45}, {"seed": 2029, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.75, "validation_area100": 0.24947427789076, "training_seconds": 4.358241431997158, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.39894906204399055, "operator": "ge"}, {"budget": 0.02, "threshold": 0.36966603801947534, "operator": "ge"}, {"budget": 0.03, "threshold": 0.34743858977365977, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3290496628159303, "operator": "ge"}, {"budget": 0.05, "threshold": 0.3031831772752764, "operator": "ge"}, {"budget": 0.06, "threshold": 0.28838840437573815, "operator": "ge"}, {"budget": 0.07, "threshold": 0.27816498972893455, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2566957683101098, "operator": "ge"}, {"budget": 0.09, "threshold": 0.19786742025363327, "operator": "ge"}, {"budget": 0.1, "threshold": 0.14119331657894207, "operator": "ge"}, {"budget": 0.11, "threshold": 0.11802992932879613, "operator": "ge"}, {"budget": 0.12, "threshold": 0.10921767028499606, "operator": "ge"}, {"budget": 0.13, "threshold": 0.10012786297291622, "operator": "ge"}, {"budget": 0.14, "threshold": 0.09183289558527337, "operator": "ge"}, {"budget": 0.15, "threshold": 0.08635438126436586, "operator": "ge"}, {"budget": 0.16, "threshold": 0.08216864658487032, "operator": "ge"}, {"budget": 0.17, "threshold": 0.0745714106907882, "operator": "ge"}, {"budget": 0.18, "threshold": 0.06920042361576317, "operator": "ge"}, {"budget": 0.19, "threshold": 0.06279087857146967, "operator": "ge"}, {"budget": 0.2, "threshold": 0.05567187890202357, "operator": "ge"}, {"budget": 0.21, "threshold": 0.050601635427305384, "operator": "ge"}, {"budget": 0.22, "threshold": 0.04552174597191395, "operator": "ge"}, {"budget": 0.23, "threshold": 0.04223446456795884, "operator": "ge"}, {"budget": 0.24, "threshold": 0.037750252551875675, "operator": "ge"}, {"budget": 0.25, "threshold": 0.034191841599248186, "operator": "ge"}, {"budget": 0.26, "threshold": 0.029891476020819072, "operator": "ge"}, {"budget": 0.27, "threshold": 0.02709784479338401, "operator": "ge"}, {"budget": 0.28, "threshold": 0.02508407932577918, "operator": "ge"}, {"budget": 0.29, "threshold": 0.020456684007004122, "operator": "ge"}, {"budget": 0.3, "threshold": 0.015313307692292036, "operator": "ge"}, {"budget": 0.31, "threshold": 0.013044303217277493, "operator": "ge"}, {"budget": 0.32, "threshold": 0.005788905754392302, "operator": "ge"}, {"budget": 0.33, "threshold": 0.003044422243236884, "operator": "ge"}, {"budget": 0.34, "threshold": -0.0026868108173418673, "operator": "ge"}, {"budget": 0.35, "threshold": -0.014466291532278703, "operator": "ge"}, {"budget": 0.36, "threshold": -0.01831853550532344, "operator": "ge"}, {"budget": 0.37, "threshold": -0.025854373310623728, "operator": "ge"}, {"budget": 0.38, "threshold": -0.03203828564426059, "operator": "ge"}, {"budget": 0.39, "threshold": -0.04090046417519664, "operator": "ge"}, {"budget": 0.4, "threshold": -0.0466752612695551, "operator": "ge"}, {"budget": 0.41, "threshold": -0.05953937608329526, "operator": "ge"}, {"budget": 0.42, "threshold": -0.07948912663835477, "operator": "ge"}, {"budget": 0.43, "threshold": -0.08900163561422615, "operator": "ge"}, {"budget": 0.44, "threshold": -0.10940047835913021, "operator": "ge"}, {"budget": 0.45, "threshold": -0.16162280672997031, "operator": "ge"}, {"budget": 0.46, "threshold": -0.32222263090817316, "operator": "ge"}, {"budget": 0.47, "threshold": -0.49334338989501886, "operator": "ge"}, {"budget": 0.48, "threshold": -0.5072059265748275, "operator": "ge"}, {"budget": 0.49, "threshold": -0.5220401359688275, "operator": "ge"}, {"budget": 0.5, "threshold": -0.5317373284465783, "operator": "ge"}, {"budget": 0.51, "threshold": -0.536449610949579, "operator": "ge"}, {"budget": 0.52, "threshold": -0.5447098909656733, "operator": "ge"}, {"budget": 0.53, "threshold": -0.5558075318393003, "operator": "ge"}, {"budget": 0.54, "threshold": -0.5642179399288021, "operator": "ge"}, {"budget": 0.55, "threshold": -0.5668304872299919, "operator": "ge"}, {"budget": 0.56, "threshold": -0.5730429022097763, "operator": "ge"}, {"budget": 0.57, "threshold": -0.5774810373439522, "operator": "ge"}, {"budget": 0.58, "threshold": -0.5823165428573944, "operator": "ge"}, {"budget": 0.59, "threshold": -0.5878233525867856, "operator": "ge"}, {"budget": 0.6, "threshold": -0.5975164934844174, "operator": "ge"}, {"budget": 0.61, "threshold": -0.6021921450492583, "operator": "ge"}, {"budget": 0.62, "threshold": -0.6078672589065018, "operator": "ge"}, {"budget": 0.63, "threshold": -0.6117029751360276, "operator": "ge"}, {"budget": 0.64, "threshold": -0.615640686824606, "operator": "ge"}, {"budget": 0.65, "threshold": -0.6183301188038125, "operator": "ge"}, {"budget": 0.66, "threshold": -0.6205596288242167, "operator": "ge"}, {"budget": 0.67, "threshold": -0.6245676915031826, "operator": "ge"}, {"budget": 0.68, "threshold": -0.6274882746579825, "operator": "ge"}, {"budget": 0.69, "threshold": -0.6306578750060787, "operator": "ge"}, {"budget": 0.7, "threshold": -0.6326575527755847, "operator": "ge"}, {"budget": 0.71, "threshold": -0.6332573743836509, "operator": "ge"}, {"budget": 0.72, "threshold": -0.6349493571145525, "operator": "ge"}, {"budget": 0.73, "threshold": -0.6375218906445286, "operator": "ge"}, {"budget": 0.74, "threshold": -0.6391670091419435, "operator": "ge"}, {"budget": 0.75, "threshold": -0.6411013312735109, "operator": "ge"}, {"budget": 0.76, "threshold": -0.6430904448830695, "operator": "ge"}, {"budget": 0.77, "threshold": -0.6443883055471157, "operator": "ge"}, {"budget": 0.78, "threshold": -0.6459555467034925, "operator": "ge"}, {"budget": 0.79, "threshold": -0.6481580960524196, "operator": "ge"}, {"budget": 0.8, "threshold": -0.6495160252415381, "operator": "ge"}, {"budget": 0.81, "threshold": -0.6512073349142413, "operator": "ge"}, {"budget": 0.82, "threshold": -0.654204997599507, "operator": "ge"}, {"budget": 0.83, "threshold": -0.6559819278421337, "operator": "ge"}, {"budget": 0.84, "threshold": -0.6578484025047984, "operator": "ge"}, {"budget": 0.85, "threshold": -0.6614843274174648, "operator": "ge"}, {"budget": 0.86, "threshold": -0.6632148716186885, "operator": "ge"}, {"budget": 0.87, "threshold": -0.6648092486236035, "operator": "ge"}, {"budget": 0.88, "threshold": -0.6670125969059402, "operator": "ge"}, {"budget": 0.89, "threshold": -0.6690624024767383, "operator": "ge"}, {"budget": 0.9, "threshold": -0.6722059444589682, "operator": "ge"}, {"budget": 0.91, "threshold": -0.674414079458403, "operator": "ge"}, {"budget": 0.92, "threshold": -0.6750104793081816, "operator": "ge"}, {"budget": 0.93, "threshold": -0.6802412396761458, "operator": "ge"}, {"budget": 0.94, "threshold": -0.6823808590680738, "operator": "ge"}, {"budget": 0.95, "threshold": -0.6884835113185962, "operator": "ge"}, {"budget": 0.96, "threshold": -0.6915198627350321, "operator": "ge"}, {"budget": 0.97, "threshold": -0.6938301756976778, "operator": "ge"}, {"budget": 0.98, "threshold": -0.6980989785573092, "operator": "ge"}, {"budget": 0.99, "threshold": -0.7059644326655182, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.362915413832557, 19.239686081111195, 19.794380518164715, 20.26842849939917, 20.61591271224418, 20.913914788914788, 21.342667358674113, 21.715932656472802, 22.339935974389757, 22.67253835840906, 23.989185417756847, 24.060047119195183, 24.576344707923656, 24.728951589244147, 25.000036969342066, 25.47390278578937, 25.253648750730463, 25.32682492859484, 25.397946339709115, 25.48247481197673, 25.50775256399903, 25.70851151761716, 25.76950320358028, 25.49279071393609, 25.64555192357883, 25.614014408856704, 26.065809045529097, 26.25593918276845, 27.08342325388673, 27.37291541089009, 27.320053604126592, 27.405741804973033, 27.35745227307668, 27.819483115585065, 28.559575345289627, 26.661858487701185, 26.850770093872, 27.835358870402676, 27.36474476609564, 27.65229650195841, 27.97426276352199, 27.806611330893134, 27.950379813679007, 28.071097352299596, 28.55822749811899, 28.64514777488093, 28.585833451559704, 28.39603591544427, 28.363988485247578, 28.236050852588036, 27.951330687397913, 27.90555711400761, 27.845850117582334, 27.745929949795645, 27.731500976417102, 27.853427106393898, 27.982764497380487, 27.768293392888932, 26.959057891261278, 26.88936271025823, 26.826154380898902, 26.775287844258756, 26.448306595365416, 26.396510207228687, 26.350331799386417, 26.191808191808192, 25.718350180392346, 25.22482828160954, 25.268342768342766, 25.392648472632995, 25.335976044007925, 25.4647101717125, 25.3854212837974, 24.958521870286575, 24.82338270218784, 24.465612939218715, 24.407369795455587, 24.124311442747196, 24.063264083982315, 23.862944247093942, 23.00957986387803, 22.958035950881296, 22.840855524149717, 22.775675992020833, 22.430500199282584, 22.461435358125737, 22.411361204147706, 22.090194109772423, 22.056215477444354, 22.011554621848738, 21.962669153203134, 21.915439840058163, 21.792097439147014, 21.562266597530655, 21.271331441802285, 21.15134764994428, 20.958198405030146, 20.913830233262082, 20.714311025369135, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 10, 24, 38, 47, 59, 68, 80, 91, 101, 111, 117, 122, 128, 133, 140, 145, 150, 156, 161, 167, 171, 176, 180, 185, 190, 195, 202, 209, 214, 220, 223, 227, 231, 235, 241, 243, 247, 252, 254, 256, 257, 259, 265, 268, 270, 274, 275, 276, 276, 276, 276, 276, 276, 276, 276, 277, 279, 280, 280, 280, 280, 280, 280, 280, 281, 281, 281, 281, 282, 283, 283, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 3, 5, 6, 9, 13, 16, 18, 22, 23, 26, 32, 39, 41, 45, 52, 56, 62, 66, 71, 76, 81, 84, 87, 97, 102, 107, 113, 120, 127, 131, 135, 140, 149, 152, 160, 166, 169, 176, 181, 186, 186, 189, 195, 199, 204, 209, 215, 223, 225, 228, 233, 236, 240, 247, 248, 254, 258, 263, 271, 276, 281, 285, 294, 298, 303, 310, 314, 317, 322, 326, 331, 334, 341, 349, 355, 359, 365, 371, 377, 383, 390, 396, 400, 407, 413, 419, 429, 435, 440, 445, 451, 458, 464, 471, 480, 487, 494, 501, 511, 518], "exact_score": [18.254072168965788, 18.362915413832557, 19.239686081111195, 19.794380518164715, 20.26842849939917, 20.61591271224418, 20.913914788914788, 21.342667358674113, 21.715932656472802, 22.339935974389757, 22.67253835840906, 23.989185417756847, 24.060047119195183, 24.576344707923656, 24.728951589244147, 25.000036969342066, 25.47390278578937, 25.253648750730463, 25.32682492859484, 25.397946339709115, 25.48247481197673, 25.50775256399903, 25.70851151761716, 25.76950320358028, 25.49279071393609, 25.64555192357883, 25.614014408856704, 26.065809045529097, 26.25593918276845, 27.08342325388673, 27.37291541089009, 27.320053604126592, 27.405741804973033, 27.35745227307668, 27.819483115585065, 28.559575345289627, 26.661858487701185, 26.850770093872, 27.835358870402676, 27.36474476609564, 27.65229650195841, 27.97426276352199, 27.806611330893134, 27.950379813679007, 28.071097352299596, 28.55822749811899, 28.64514777488093, 28.585833451559704, 28.39603591544427, 28.363988485247578, 28.236050852588036, 27.951330687397913, 27.90555711400761, 27.845850117582334, 27.745929949795645, 27.731500976417102, 27.853427106393898, 27.982764497380487, 27.768293392888932, 26.959057891261278, 26.88936271025823, 26.826154380898902, 26.775287844258756, 26.448306595365416, 26.396510207228687, 26.350331799386417, 26.191808191808192, 25.718350180392346, 25.22482828160954, 25.268342768342766, 25.392648472632995, 25.335976044007925, 25.4647101717125, 25.3854212837974, 24.958521870286575, 24.82338270218784, 24.465612939218715, 24.407369795455587, 24.124311442747196, 24.063264083982315, 23.862944247093942, 23.00957986387803, 22.958035950881296, 22.840855524149717, 22.775675992020833, 22.430500199282584, 22.461435358125737, 22.411361204147706, 22.090194109772423, 22.056215477444354, 22.011554621848738, 21.962669153203134, 21.915439840058163, 21.792097439147014, 21.562266597530655, 21.271331441802285, 21.15134764994428, 20.958198405030146, 20.913830233262082, 20.714311025369135, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.947427789076, "threshold_area100": 24.95350959456825, "threshold_area25": 23.144154979648874}, "test": {"score": [18.129624718318375, 18.25473191669012, 18.540054981520164, 18.709240175353944, 18.85937469448245, 19.211667434869938, 20.11816839728448, 20.351798235909502, 20.657769454743384, 21.0308506461535, 21.21685410558354, 21.510691519520975, 21.758134178954396, 21.792148068220968, 21.94474147134234, 22.223564314453725, 22.26388393660102, 22.604985918193147, 22.322939682312445, 22.71969610542679, 23.71871523393486, 23.6996476389805, 24.07969201528287, 24.313727577689654, 24.384726378283005, 24.058411844853154, 24.245221683583296, 24.440354254113505, 24.52608830011628, 25.212759722513617, 25.469030815133014, 25.472145285642632, 25.228772199191962, 25.187492216706502, 25.405400079710706, 25.74952740356573, 25.83908960906416, 25.90916785828259, 25.93288959684454, 26.054068749682212, 26.095431747792365, 26.193190080145406, 26.42831188705189, 26.457508624363857, 26.470259579216826, 25.82413778641553, 25.93111532108809, 26.35785503839928, 26.354162281463527, 26.201763173246302, 26.17364620582222, 25.610413512447593, 25.394333276828302, 25.323917450330164, 25.061171514813918, 25.05636593904622, 24.97728131277764, 24.91941514266471, 24.890034199927698, 24.843426452036052, 24.898306008866296, 25.3232093223743, 25.158815305217107, 25.192070420481393, 25.005133170480253, 24.788126588458663, 24.679245639945073, 24.37220583743758, 24.1787460931302, 24.00005671458696, 23.84886652566246, 23.835729260359546, 24.152424580410727, 23.926333370326294, 23.79513068808062, 23.61047515356255, 23.437354485756927, 23.420123279456682, 23.239412158361656, 23.03326975908888, 22.948369597755228, 22.967082302193102, 22.72217674465862, 22.535276822097796, 22.445854608417974, 22.214950864093613, 22.04320991262316, 22.0832611697449, 22.354870137702953, 22.262759135433797, 22.105027574570325, 22.08274314268284, 22.071123214350383, 22.703228438472994, 22.554111579331547, 22.392924752130632, 22.516082375071385, 22.451048452869827, 22.43567630443131, 22.414167784326374, 22.361115727328066], "calls": [0.0, 0.9986684420772304, 1.7643142476697737, 2.2303595206391478, 3.029294274300932, 4.327563249001331, 5.625832223701731, 6.524633821571238, 7.92276964047936, 9.820239680426099, 10.985352862849535, 12.083888149134488, 12.616511318242344, 13.781624500665778, 14.747003994673769, 15.379494007989347, 15.81225033288948, 17.14380825565912, 17.909454061251665, 19.37416777629827, 20.938748335552596, 22.203728362183757, 23.768308921438084, 24.60053262316911, 25.59920106524634, 26.43142476697736, 27.66311584553928, 28.395472703062584, 29.127829560585884, 30.492676431424766, 31.52463382157124, 31.990679094540614, 33.18908122503329, 33.85486018641811, 34.720372836218374, 36.8508655126498, 37.45006657789614, 38.315579227696404, 38.68175765645805, 39.58055925432756, 40.046604527296935, 41.011984021304926, 42.24367509986685, 42.709720372836216, 43.508655126498006, 44.50732356857524, 45.339547270306255, 46.63781624500666, 47.270306258322236, 48.10252996005326, 49.03462050599201, 50.0, 51.19840213049268, 52.82956058588548, 53.96138482023968, 54.460719041278296, 55.1930758988016, 55.9254327563249, 56.524633821571236, 57.889480692410125, 60.11984021304927, 61.31824234354194, 62.71637816245007, 64.14780292942743, 65.11318242343542, 65.94540612516646, 66.67776298268974, 68.70838881491345, 69.6404793608522, 70.97203728362183, 71.6711051930759, 71.97070572569906, 72.56990679094541, 73.66844207723035, 74.36750998668442, 75.29960053262317, 76.39813581890812, 76.89747003994674, 77.56324900133156, 78.4953395472703, 79.16111850865512, 79.79360852197071, 81.45805592543276, 82.35685752330227, 83.32223701731026, 85.31957390146472, 86.15179760319573, 86.98402130492676, 87.94940079893475, 88.71504660452729, 89.98002663115847, 90.47936085219706, 90.77896138482025, 92.54327563249002, 93.37549933422103, 95.57256990679095, 96.63781624500666, 97.27030625832224, 98.10252996005326, 99.20106524633822, 100.0], "rescued": [0, 52, 92, 117, 147, 210, 261, 290, 339, 401, 433, 465, 481, 512, 534, 551, 562, 594, 613, 650, 689, 717, 756, 778, 803, 822, 848, 859, 874, 902, 926, 936, 962, 976, 995, 1038, 1046, 1057, 1060, 1074, 1081, 1093, 1106, 1111, 1120, 1130, 1142, 1154, 1155, 1156, 1157, 1158, 1160, 1161, 1161, 1162, 1162, 1163, 1163, 1165, 1167, 1169, 1170, 1174, 1175, 1176, 1177, 1177, 1177, 1178, 1178, 1178, 1179, 1179, 1179, 1179, 1179, 1179, 1179, 1179, 1179, 1180, 1181, 1182, 1182, 1182, 1182, 1184, 1185, 1185, 1185, 1185, 1185, 1189, 1189, 1191, 1196, 1196, 1197, 1198, 1198], "harmed": [0, 10, 18, 25, 35, 59, 71, 84, 99, 118, 131, 155, 164, 186, 207, 222, 232, 258, 275, 306, 338, 371, 401, 419, 444, 465, 489, 500, 514, 539, 552, 563, 588, 602, 622, 668, 675, 687, 691, 704, 712, 725, 740, 749, 760, 787, 805, 831, 838, 851, 865, 889, 921, 947, 968, 976, 986, 998, 1007, 1027, 1076, 1096, 1130, 1157, 1179, 1201, 1219, 1265, 1289, 1316, 1335, 1341, 1356, 1386, 1407, 1430, 1461, 1470, 1488, 1511, 1530, 1546, 1584, 1610, 1631, 1684, 1710, 1731, 1759, 1787, 1816, 1829, 1836, 1883, 1910, 1973, 2004, 2024, 2048, 2085, 2109], "exact_score": [18.129624718318375, 18.25473191669012, 18.62472798196278, 18.851856485380004, 19.138919570963992, 19.77140939247638, 20.226394846735293, 20.40783443118738, 20.636601003963907, 20.846058450608997, 21.058171514978266, 21.21685410558354, 21.504606166801516, 21.603920335712356, 21.842269135946044, 22.20354274360772, 22.3197501922649, 22.582974015846606, 22.331023094375062, 22.636307297347823, 22.712311942702705, 23.71871523393486, 23.6457177912966, 23.81749011486021, 24.101028638752943, 24.349358279152227, 23.99882613855205, 24.172788276850543, 24.313318374415907, 24.506303737828226, 24.68260262025534, 25.37216623127934, 25.472145285642632, 25.170088468920245, 25.253250479097762, 25.448015786547707, 25.742207298767244, 25.74952740356573, 25.827627472855823, 25.94928631476424, 26.081111748808777, 26.193190080145406, 26.282976182735606, 26.44988703062998, 26.263259031449255, 25.8983279318343, 26.002974862751167, 26.349961991841326, 26.204882294901644, 26.1592105695022, 25.610413512447593, 25.408216559026503, 25.300927690670704, 25.312943706923647, 25.05714016981999, 25.032658958037658, 24.91301019956773, 24.94083933956537, 24.81928728773369, 24.66820172600539, 24.977615749575666, 25.298888119745133, 25.25470704219935, 25.221008366931112, 25.182387482542996, 25.086633121845942, 24.788126588458663, 24.644283969863448, 24.46517647627834, 24.293656362123873, 24.146685893352775, 24.00005671458696, 23.835729260359546, 24.06976765098909, 23.88963840336223, 23.687364102652307, 23.454526156894193, 23.417172300425992, 23.161397003656546, 23.007903911976733, 22.94103887666967, 22.7426500071968, 22.563381124774928, 22.50178838302836, 22.33254443555918, 22.26365561616815, 22.049656405301377, 22.0832611697449, 22.33616335429029, 22.25308187482678, 22.105027574570325, 22.06593104848043, 22.343323728440296, 22.68620499772119, 22.45398621376221, 22.43274242525135, 22.423814447642787, 22.4810643608716, 22.442243673246715, 22.424486136190772, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.571311589121954, "threshold_area100": 23.620263137244958, "threshold_area25": 21.492963874085564}, "selected_budget": 46, "selected": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726, "rescued": 274, "harmed": 209}, "test": {"score": 25.93111532108809, "calls": 45.339547270306255, "rescued": 1142, "harmed": 805}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.3206043839454651, "operator": "ge"}, {"budget": 0.02, "threshold": 0.2954588532447815, "operator": "ge"}, {"budget": 0.03, "threshold": 0.2745589017868042, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2597978711128235, "operator": "ge"}, {"budget": 0.05, "threshold": 0.24697500467300415, "operator": "ge"}, {"budget": 0.06, "threshold": 0.23037508130073547, "operator": "ge"}, {"budget": 0.07, "threshold": 0.2174025923013687, "operator": "ge"}, {"budget": 0.08, "threshold": 0.20624655485153198, "operator": "ge"}, {"budget": 0.09, "threshold": 0.1498858630657196, "operator": "ge"}, {"budget": 0.1, "threshold": 0.0899314135313034, "operator": "ge"}, {"budget": 0.11, "threshold": 0.06548376381397247, "operator": "ge"}, {"budget": 0.12, "threshold": 0.06011086702346802, "operator": "ge"}, {"budget": 0.13, "threshold": 0.05424703657627106, "operator": "ge"}, {"budget": 0.14, "threshold": 0.05027337372303009, "operator": "ge"}, {"budget": 0.15, "threshold": 0.04634174704551697, "operator": "ge"}, {"budget": 0.16, "threshold": 0.04419626295566559, "operator": "ge"}, {"budget": 0.17, "threshold": 0.04052847623825073, "operator": "ge"}, {"budget": 0.18, "threshold": 0.03870188444852829, "operator": "ge"}, {"budget": 0.19, "threshold": 0.035489827394485474, "operator": "ge"}, {"budget": 0.2, "threshold": 0.033267028629779816, "operator": "ge"}, {"budget": 0.21, "threshold": 0.0286407470703125, "operator": "ge"}, {"budget": 0.22, "threshold": 0.025687173008918762, "operator": "ge"}, {"budget": 0.23, "threshold": 0.02337522804737091, "operator": "ge"}, {"budget": 0.24, "threshold": 0.0219946950674057, "operator": "ge"}, {"budget": 0.25, "threshold": 0.018641844391822815, "operator": "ge"}, {"budget": 0.26, "threshold": 0.01715800166130066, "operator": "ge"}, {"budget": 0.27, "threshold": 0.015887513756752014, "operator": "ge"}, {"budget": 0.28, "threshold": 0.014748796820640564, "operator": "ge"}, {"budget": 0.29, "threshold": 0.011905521154403687, "operator": "ge"}, {"budget": 0.3, "threshold": 0.009116485714912415, "operator": "ge"}, {"budget": 0.31, "threshold": 0.007464244961738586, "operator": "ge"}, {"budget": 0.32, "threshold": 0.003461286425590515, "operator": "ge"}, {"budget": 0.33, "threshold": 0.0017232373356819153, "operator": "ge"}, {"budget": 0.34, "threshold": -0.0015555694699287415, "operator": "ge"}, {"budget": 0.35, "threshold": -0.008427128195762634, "operator": "ge"}, {"budget": 0.36, "threshold": -0.01085694134235382, "operator": "ge"}, {"budget": 0.37, "threshold": -0.01496751606464386, "operator": "ge"}, {"budget": 0.38, "threshold": -0.017183467745780945, "operator": "ge"}, {"budget": 0.39, "threshold": -0.024410076439380646, "operator": "ge"}, {"budget": 0.4, "threshold": -0.029820390045642853, "operator": "ge"}, {"budget": 0.41, "threshold": -0.03590814769268036, "operator": "ge"}, {"budget": 0.42, "threshold": -0.043082863092422485, "operator": "ge"}, {"budget": 0.43, "threshold": -0.051084063947200775, "operator": "ge"}, {"budget": 0.44, "threshold": -0.06102542579174042, "operator": "ge"}, {"budget": 0.45, "threshold": -0.07046637684106827, "operator": "ge"}, {"budget": 0.46, "threshold": -0.07427139580249786, "operator": "ge"}, {"budget": 0.47, "threshold": -0.0789092481136322, "operator": "ge"}, {"budget": 0.48, "threshold": -0.08281754702329636, "operator": "ge"}, {"budget": 0.49, "threshold": -0.08824026584625244, "operator": "ge"}, {"budget": 0.5, "threshold": -0.09629657119512558, "operator": "ge"}, {"budget": 0.51, "threshold": -0.10029807686805725, "operator": "ge"}, {"budget": 0.52, "threshold": -0.10743754357099533, "operator": "ge"}, {"budget": 0.53, "threshold": -0.11172406375408173, "operator": "ge"}, {"budget": 0.54, "threshold": -0.11845997720956802, "operator": "ge"}, {"budget": 0.55, "threshold": -0.12622690200805664, "operator": "ge"}, {"budget": 0.56, "threshold": -0.13091008365154266, "operator": "ge"}, {"budget": 0.57, "threshold": -0.1370542049407959, "operator": "ge"}, {"budget": 0.58, "threshold": -0.14107373356819153, "operator": "ge"}, {"budget": 0.59, "threshold": -0.1434464156627655, "operator": "ge"}, {"budget": 0.6, "threshold": -0.1486809253692627, "operator": "ge"}, {"budget": 0.61, "threshold": -0.15027667582035065, "operator": "ge"}, {"budget": 0.62, "threshold": -0.15422430634498596, "operator": "ge"}, {"budget": 0.63, "threshold": -0.1568869799375534, "operator": "ge"}, {"budget": 0.64, "threshold": -0.16040849685668945, "operator": "ge"}, {"budget": 0.65, "threshold": -0.16146253049373627, "operator": "ge"}, {"budget": 0.66, "threshold": -0.16357746720314026, "operator": "ge"}, {"budget": 0.67, "threshold": -0.16629579663276672, "operator": "ge"}, {"budget": 0.68, "threshold": -0.16799624264240265, "operator": "ge"}, {"budget": 0.69, "threshold": -0.17095893621444702, "operator": "ge"}, {"budget": 0.7, "threshold": -0.17293739318847656, "operator": "ge"}, {"budget": 0.71, "threshold": -0.17451322078704834, "operator": "ge"}, {"budget": 0.72, "threshold": -0.17657577991485596, "operator": "ge"}, {"budget": 0.73, "threshold": -0.17882636189460754, "operator": "ge"}, {"budget": 0.74, "threshold": -0.18123586475849152, "operator": "ge"}, {"budget": 0.75, "threshold": -0.18306408822536469, "operator": "ge"}, {"budget": 0.76, "threshold": -0.1850607693195343, "operator": "ge"}, {"budget": 0.77, "threshold": -0.1861141473054886, "operator": "ge"}, {"budget": 0.78, "threshold": -0.18863600492477417, "operator": "ge"}, {"budget": 0.79, "threshold": -0.19193018972873688, "operator": "ge"}, {"budget": 0.8, "threshold": -0.19608944654464722, "operator": "ge"}, {"budget": 0.81, "threshold": -0.19744820892810822, "operator": "ge"}, {"budget": 0.82, "threshold": -0.1991225630044937, "operator": "ge"}, {"budget": 0.83, "threshold": -0.20244437456130981, "operator": "ge"}, {"budget": 0.84, "threshold": -0.2040291577577591, "operator": "ge"}, {"budget": 0.85, "threshold": -0.2065638154745102, "operator": "ge"}, {"budget": 0.86, "threshold": -0.20908673107624054, "operator": "ge"}, {"budget": 0.87, "threshold": -0.2111976444721222, "operator": "ge"}, {"budget": 0.88, "threshold": -0.21530218422412872, "operator": "ge"}, {"budget": 0.89, "threshold": -0.2175392508506775, "operator": "ge"}, {"budget": 0.9, "threshold": -0.2195645272731781, "operator": "ge"}, {"budget": 0.91, "threshold": -0.22441114485263824, "operator": "ge"}, {"budget": 0.92, "threshold": -0.22783109545707703, "operator": "ge"}, {"budget": 0.93, "threshold": -0.23497170209884644, "operator": "ge"}, {"budget": 0.94, "threshold": -0.23895320296287537, "operator": "ge"}, {"budget": 0.95, "threshold": -0.24213622510433197, "operator": "ge"}, {"budget": 0.96, "threshold": -0.24488139152526855, "operator": "ge"}, {"budget": 0.97, "threshold": -0.25053173303604126, "operator": "ge"}, {"budget": 0.98, "threshold": -0.25618380308151245, "operator": "ge"}, {"budget": 0.99, "threshold": -0.27307817339897156, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.540459613418374, 18.92227352319281, 19.824875943171776, 20.179466600745226, 20.61591271224418, 20.907334211682038, 21.342667358674113, 21.715932656472802, 22.339935974389757, 22.67253835840906, 22.86253196930946, 22.84593697144919, 24.34654795076685, 24.73822746731459, 24.74131114436213, 24.979316050744618, 24.93581088408815, 24.910158261928174, 25.479066022544284, 25.248605205593922, 25.627361787917778, 25.622146396904505, 25.73872246747257, 25.49279071393609, 25.80439946705262, 25.595559684447256, 26.00321403524563, 26.513888292955023, 27.066590126291622, 27.26107226107226, 27.320053604126592, 27.405741804973033, 27.294193730707363, 27.819483115585065, 26.480550879143806, 26.592476611509586, 27.275195588406635, 27.835358870402676, 27.30083655083655, 27.65229650195841, 27.97426276352199, 27.875739074624317, 28.020202020202024, 28.071097352299596, 28.01666783455234, 28.470473719342497, 28.3994662953939, 28.329216387356027, 28.278239327033173, 28.272210075688665, 28.286264223112635, 28.166151897042557, 28.18389958051811, 28.130136139657893, 27.55854994173632, 27.61022029293645, 27.562871808087962, 27.5106772432858, 27.46004370262597, 27.228023114894622, 26.744805855217752, 26.611878706248238, 26.548596933109327, 26.4174432219545, 26.396625827865215, 25.678253119429588, 25.263842910901737, 25.46637484611679, 24.904691091136865, 24.853377630491817, 25.052596975673897, 25.08668213098904, 24.75561421688387, 24.717667193727383, 24.667444206279153, 24.61355903725795, 24.545317077717552, 24.129585038817364, 23.28474172642907, 23.216996626732232, 22.87466827810399, 22.413271426429322, 22.36825674325674, 22.397198159294764, 22.36876602322182, 22.041847041847042, 22.00261815190969, 21.77429149797571, 21.728929644100255, 21.675582381639664, 21.739857055391038, 21.51055917117278, 21.654489769561447, 21.371244202460172, 21.33234856481695, 21.285657568165654, 21.232951789635713, 20.970580785238425, 20.63372217564812, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 10, 23, 38, 48, 59, 69, 80, 91, 101, 111, 116, 120, 127, 133, 138, 145, 149, 155, 161, 165, 171, 174, 179, 185, 191, 195, 202, 209, 214, 219, 223, 227, 230, 235, 240, 243, 247, 252, 252, 256, 257, 260, 266, 268, 268, 269, 270, 270, 270, 271, 272, 272, 273, 273, 273, 274, 274, 274, 274, 274, 274, 274, 274, 274, 274, 274, 274, 275, 275, 275, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 277, 277, 278, 278, 278, 278, 278, 279, 279, 280, 280, 282, 282, 282, 283, 284, 285, 285, 286], "harmed": [0, 1, 3, 6, 8, 13, 15, 18, 22, 23, 26, 33, 39, 42, 47, 54, 58, 62, 67, 69, 76, 80, 83, 89, 97, 103, 108, 114, 120, 128, 131, 135, 140, 148, 152, 163, 166, 169, 176, 179, 186, 186, 190, 196, 199, 202, 204, 209, 213, 216, 222, 227, 230, 235, 239, 246, 249, 253, 257, 261, 267, 272, 277, 283, 288, 290, 296, 303, 308, 313, 318, 323, 331, 340, 344, 350, 356, 363, 370, 377, 384, 389, 396, 402, 408, 416, 423, 428, 433, 438, 445, 451, 458, 464, 473, 478, 486, 494, 503, 512, 518], "exact_score": [18.254072168965788, 18.540459613418374, 18.92227352319281, 19.824875943171776, 20.179466600745226, 20.61591271224418, 20.907334211682038, 21.342667358674113, 21.715932656472802, 22.339935974389757, 22.67253835840906, 22.86253196930946, 22.84593697144919, 24.34654795076685, 24.73822746731459, 24.74131114436213, 24.979316050744618, 24.93581088408815, 24.910158261928174, 25.479066022544284, 25.248605205593922, 25.627361787917778, 25.622146396904505, 25.73872246747257, 25.49279071393609, 25.80439946705262, 25.595559684447256, 26.00321403524563, 26.513888292955023, 27.066590126291622, 27.26107226107226, 27.320053604126592, 27.405741804973033, 27.294193730707363, 27.819483115585065, 26.480550879143806, 26.592476611509586, 27.275195588406635, 27.835358870402676, 27.30083655083655, 27.65229650195841, 27.97426276352199, 27.875739074624317, 28.020202020202024, 28.071097352299596, 28.01666783455234, 28.470473719342497, 28.3994662953939, 28.329216387356027, 28.278239327033173, 28.272210075688665, 28.286264223112635, 28.166151897042557, 28.18389958051811, 28.130136139657893, 27.55854994173632, 27.61022029293645, 27.562871808087962, 27.5106772432858, 27.46004370262597, 27.228023114894622, 26.744805855217752, 26.611878706248238, 26.548596933109327, 26.4174432219545, 26.396625827865215, 25.678253119429588, 25.263842910901737, 25.46637484611679, 24.904691091136865, 24.853377630491817, 25.052596975673897, 25.08668213098904, 24.75561421688387, 24.717667193727383, 24.667444206279153, 24.61355903725795, 24.545317077717552, 24.129585038817364, 23.28474172642907, 23.216996626732232, 22.87466827810399, 22.413271426429322, 22.36825674325674, 22.397198159294764, 22.36876602322182, 22.041847041847042, 22.00261815190969, 21.77429149797571, 21.728929644100255, 21.675582381639664, 21.739857055391038, 21.51055917117278, 21.654489769561447, 21.371244202460172, 21.33234856481695, 21.285657568165654, 21.232951789635713, 20.970580785238425, 20.63372217564812, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.804547081185707, "threshold_area100": 24.81048233484506, "threshold_area25": 23.0615206477237}, "test": {"score": [18.129624718318375, 18.387286181399634, 18.536647271145636, 18.76359887583908, 18.904519927028392, 19.3269998917523, 19.94809530139239, 20.422662692642373, 20.60919693914311, 21.02165177285688, 21.11745395939326, 21.537481502114026, 21.835391756541984, 22.049886192786076, 21.92834930585428, 21.93615730369401, 21.988867836118505, 22.052246561269484, 22.596407149367103, 22.775326766133347, 23.22455960519305, 23.712086659290133, 24.00642375267525, 24.25707104240626, 24.398886372298477, 24.18958291026422, 24.223005370157267, 24.307176771194637, 24.42957340760133, 24.86806261763099, 25.458112804540118, 25.482309075427455, 25.2172706776482, 25.2010344249965, 25.423010525381777, 25.74952740356573, 25.79211058823303, 25.888896476750915, 25.99685098861259, 26.173663729469194, 26.166568445168327, 26.31893864380312, 26.275128215962688, 26.438734524050144, 26.390726176429574, 25.722655606382403, 25.535310475057425, 25.533459231912648, 25.454289021573377, 25.367274293701193, 25.264236157415887, 25.27208309088757, 25.18851922842887, 25.189960606277474, 25.15524652188449, 25.552043820505883, 25.540272885722043, 25.972289069224253, 25.885671111137725, 25.746820881771114, 25.748328892578883, 25.560112323296558, 25.538140396245733, 25.390811847742402, 25.366038171387924, 25.406485688545644, 25.82631242753572, 25.592132375918897, 25.14927429393866, 24.970479543219643, 24.782670591256956, 25.064216732862693, 24.865864813586484, 24.725905458636, 24.724685068618502, 24.55613589721616, 24.336079442720163, 24.3181644905424, 24.176867677451806, 24.08157601369473, 23.7950625798501, 23.70171192125544, 23.628167378019064, 23.312671916710386, 23.236740809495252, 23.568871413702393, 23.381509327124235, 23.33756139254532, 23.092837902531187, 22.976626596399086, 22.94791603219501, 23.50559911104537, 23.333980978336662, 23.207136887657978, 23.125227112610773, 23.07247066988578, 23.13882499030618, 22.96393886160558, 22.819390708385015, 22.6056783328693, 22.361115727328066], "calls": [0.0, 1.1651131824234355, 1.7310252996005324, 2.563249001331558, 3.262316910785619, 4.527296937416778, 6.25832223701731, 7.157123834886818, 8.022636484687084, 9.820239680426099, 11.018641810918774, 12.283621837549934, 13.182423435419441, 14.181091877496671, 15.013315579227696, 16.145139813581892, 16.67776298268975, 17.509986684420774, 18.10918774966711, 19.507323568575234, 20.33954727030626, 22.436750998668444, 23.834886817576564, 24.800266311584554, 25.432756324900136, 27.06391478029294, 27.563249001331556, 28.262316910785618, 28.928095872170438, 30.525965379494007, 31.591211717709722, 32.05725699067909, 33.15579227696405, 33.88814913448735, 34.78695073235686, 36.8508655126498, 37.48335552596538, 38.04926764314248, 38.515312916111846, 39.71371504660453, 40.47936085219707, 41.17842876165113, 41.87749667110519, 42.60985352862849, 43.841544607190414, 44.90679094540613, 45.572569906790946, 46.6711051930759, 47.53661784287617, 48.63515312916112, 50.06657789613848, 50.466045272969374, 51.564580559254324, 52.06391478029294, 52.99600532623169, 54.49400798934754, 55.32623169107856, 57.39014647137151, 58.55525965379494, 59.254327563249, 60.885486018641814, 61.684420772303596, 63.01597869507324, 64.0479360852197, 65.31291611185085, 65.64580559254327, 66.44474034620505, 67.44340878828228, 68.07589880159787, 69.27430093209055, 69.94007989347537, 70.63914780292943, 71.60452729693742, 72.6364846870839, 73.63515312916113, 74.66711051930758, 75.46604527296937, 75.86551264980027, 76.89747003994674, 78.09587217043942, 80.09320905459387, 80.6924101198402, 81.49134487350199, 82.75632490013317, 83.5219707057257, 84.55392809587217, 85.68575233022636, 86.31824234354194, 87.61651131824235, 88.24900133155792, 88.81491344873503, 90.24633821571238, 91.34487350199734, 93.40878828229027, 94.74034620505992, 95.43941411451398, 96.13848202396804, 97.07057256990679, 98.1358189081225, 99.30093209054593, 100.0], "rescued": [0, 63, 91, 126, 157, 211, 278, 310, 338, 400, 432, 473, 496, 521, 543, 574, 584, 602, 620, 650, 673, 724, 759, 783, 799, 835, 844, 856, 868, 904, 927, 937, 961, 976, 996, 1038, 1046, 1054, 1061, 1078, 1086, 1095, 1102, 1110, 1120, 1121, 1122, 1129, 1129, 1130, 1131, 1132, 1134, 1135, 1136, 1139, 1140, 1144, 1145, 1145, 1150, 1150, 1152, 1153, 1156, 1157, 1158, 1158, 1160, 1160, 1160, 1161, 1161, 1162, 1164, 1164, 1164, 1164, 1164, 1166, 1171, 1171, 1171, 1172, 1173, 1176, 1177, 1177, 1179, 1179, 1179, 1182, 1183, 1189, 1190, 1191, 1194, 1196, 1197, 1198, 1198], "harmed": [0, 4, 15, 25, 38, 55, 80, 87, 98, 118, 132, 156, 174, 195, 213, 245, 256, 274, 282, 309, 325, 380, 406, 427, 441, 479, 484, 497, 509, 538, 555, 564, 587, 602, 622, 668, 677, 685, 691, 705, 716, 725, 738, 749, 764, 778, 793, 812, 836, 860, 885, 888, 907, 914, 931, 952, 966, 1001, 1023, 1041, 1076, 1094, 1119, 1141, 1167, 1174, 1193, 1223, 1238, 1267, 1286, 1304, 1326, 1356, 1374, 1403, 1421, 1432, 1456, 1484, 1534, 1553, 1576, 1615, 1637, 1663, 1691, 1709, 1745, 1759, 1776, 1812, 1844, 1901, 1941, 1962, 1986, 2012, 2044, 2085, 2109], "exact_score": [18.129624718318375, 18.310534061820757, 18.613405808091464, 18.869130856708733, 19.106161534167747, 19.690668248937683, 19.963344590909173, 20.417936753953896, 20.601399972163268, 20.81779515985875, 21.066037788331265, 21.10872541705983, 21.445816422858456, 21.830685037539542, 22.07528856900634, 21.92562769798803, 22.27207032165203, 21.984120140445174, 22.546542899766848, 22.709586699368796, 22.928689715378617, 23.6361418170134, 23.702619610404174, 23.781385567296507, 24.027446667662748, 24.30085033122837, 24.084186452578844, 24.172233641340256, 24.250530905900337, 24.44448483604628, 24.667162516999653, 25.35613586199114, 25.471502967956983, 25.19738720099446, 25.211999734612256, 25.448015786547707, 25.765590247878507, 25.781394457633954, 25.888896476750915, 26.009442210161666, 26.147938800399256, 26.19054948904781, 26.270065266625934, 26.435085233039196, 26.341933367071746, 25.602316324180297, 25.55314587108323, 25.510807628252714, 25.40535221356391, 25.30350734311219, 25.268556756931243, 25.264325957526395, 25.192706907221357, 25.15524652188449, 25.08355355467062, 25.555743415462075, 25.53860547758779, 25.497796470460415, 25.935501728252103, 25.852443739729345, 25.674463325033365, 25.725514690990064, 25.54145000790129, 25.538140396245733, 25.45372651257882, 25.338273394505347, 25.387240904533975, 25.754766795386196, 25.15101977895076, 24.985019632496666, 24.782670591256956, 24.990930870742297, 24.77040572522064, 24.733093404491324, 24.711770392977368, 24.446816111413522, 24.312750422628334, 24.192978504482536, 24.08420748184344, 23.955362958229532, 23.797559982551288, 23.647741770845222, 23.50944948289233, 23.319752319975688, 23.245877931243765, 23.506989236960884, 23.348638028033577, 23.27855484032613, 23.035249441893246, 22.94061116197606, 23.158660216112683, 23.291195134051172, 23.296796376500147, 23.18586201964818, 23.168236422085613, 23.098768577608624, 23.19634339294399, 22.985945596305957, 22.829389278359578, 22.66779993914999, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.86907138530425, "threshold_area100": 23.928852476894544, "threshold_area25": 21.352954914864785}, "selected_budget": 46, "selected": {"val": {"score": 28.470473719342497, "calls": 45.93437945791726, "rescued": 269, "harmed": 204}, "test": {"score": 25.535310475057425, "calls": 45.572569906790946, "rescued": 1122, "harmed": 793}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.630393385887146, "operator": "ge"}, {"budget": 0.02, "threshold": 1.3315526247024536, "operator": "ge"}, {"budget": 0.03, "threshold": 1.252586841583252, "operator": "ge"}, {"budget": 0.04, "threshold": 1.1833916902542114, "operator": "ge"}, {"budget": 0.05, "threshold": 1.12424898147583, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0905468463897705, "operator": "ge"}, {"budget": 0.07, "threshold": 1.018612265586853, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9566188454627991, "operator": "ge"}, {"budget": 0.09, "threshold": 0.8625038862228394, "operator": "ge"}, {"budget": 0.1, "threshold": 0.62535560131073, "operator": "ge"}, {"budget": 0.11, "threshold": 0.3578774631023407, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3230434060096741, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3062124252319336, "operator": "ge"}, {"budget": 0.14, "threshold": 0.28674083948135376, "operator": "ge"}, {"budget": 0.15, "threshold": 0.2653653919696808, "operator": "ge"}, {"budget": 0.16, "threshold": 0.2561017870903015, "operator": "ge"}, {"budget": 0.17, "threshold": 0.24657988548278809, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2352091372013092, "operator": "ge"}, {"budget": 0.19, "threshold": 0.20840826630592346, "operator": "ge"}, {"budget": 0.2, "threshold": 0.19822874665260315, "operator": "ge"}, {"budget": 0.21, "threshold": 0.17987924814224243, "operator": "ge"}, {"budget": 0.22, "threshold": 0.14610400795936584, "operator": "ge"}, {"budget": 0.23, "threshold": 0.12980562448501587, "operator": "ge"}, {"budget": 0.24, "threshold": 0.11913269013166428, "operator": "ge"}, {"budget": 0.25, "threshold": 0.10357318073511124, "operator": "ge"}, {"budget": 0.26, "threshold": 0.09346256405115128, "operator": "ge"}, {"budget": 0.27, "threshold": 0.08518471568822861, "operator": "ge"}, {"budget": 0.28, "threshold": 0.07794297486543655, "operator": "ge"}, {"budget": 0.29, "threshold": 0.07038978487253189, "operator": "ge"}, {"budget": 0.3, "threshold": 0.05690775066614151, "operator": "ge"}, {"budget": 0.31, "threshold": 0.04087770730257034, "operator": "ge"}, {"budget": 0.32, "threshold": 0.017828047275543213, "operator": "ge"}, {"budget": 0.33, "threshold": 0.010317064821720123, "operator": "ge"}, {"budget": 0.34, "threshold": -0.009262852370738983, "operator": "ge"}, {"budget": 0.35, "threshold": -0.04415314644575119, "operator": "ge"}, {"budget": 0.36, "threshold": -0.07671788334846497, "operator": "ge"}, {"budget": 0.37, "threshold": -0.10357576608657837, "operator": "ge"}, {"budget": 0.38, "threshold": -0.15218454599380493, "operator": "ge"}, {"budget": 0.39, "threshold": -0.202407568693161, "operator": "ge"}, {"budget": 0.4, "threshold": -0.27963510155677795, "operator": "ge"}, {"budget": 0.41, "threshold": -0.3095020353794098, "operator": "ge"}, {"budget": 0.42, "threshold": -0.4527289867401123, "operator": "ge"}, {"budget": 0.43, "threshold": -0.5261328816413879, "operator": "ge"}, {"budget": 0.44, "threshold": -0.6571394205093384, "operator": "ge"}, {"budget": 0.45, "threshold": -0.9343714714050293, "operator": "ge"}, {"budget": 0.46, "threshold": -1.5519126653671265, "operator": "ge"}, {"budget": 0.47, "threshold": -1.990483045578003, "operator": "ge"}, {"budget": 0.48, "threshold": -2.6142427921295166, "operator": "ge"}, {"budget": 0.49, "threshold": -2.833251476287842, "operator": "ge"}, {"budget": 0.5, "threshold": -3.0881123542785645, "operator": "ge"}, {"budget": 0.51, "threshold": -3.200615882873535, "operator": "ge"}, {"budget": 0.52, "threshold": -4.8070292472839355, "operator": "ge"}, {"budget": 0.53, "threshold": -4.915364742279053, "operator": "ge"}, {"budget": 0.54, "threshold": -4.9557576179504395, "operator": "ge"}, {"budget": 0.55, "threshold": -4.977481365203857, "operator": "ge"}, {"budget": 0.56, "threshold": -5.026656150817871, "operator": "ge"}, {"budget": 0.57, "threshold": -5.067741870880127, "operator": "ge"}, {"budget": 0.58, "threshold": -5.096231937408447, "operator": "ge"}, {"budget": 0.59, "threshold": -5.127989292144775, "operator": "ge"}, {"budget": 0.6, "threshold": -5.156412124633789, "operator": "ge"}, {"budget": 0.61, "threshold": -5.175171375274658, "operator": "ge"}, {"budget": 0.62, "threshold": -5.231557846069336, "operator": "ge"}, {"budget": 0.63, "threshold": -5.238612651824951, "operator": "ge"}, {"budget": 0.64, "threshold": -5.258386135101318, "operator": "ge"}, {"budget": 0.65, "threshold": -5.272547721862793, "operator": "ge"}, {"budget": 0.66, "threshold": -5.304098606109619, "operator": "ge"}, {"budget": 0.67, "threshold": -5.320165634155273, "operator": "ge"}, {"budget": 0.68, "threshold": -5.34406852722168, "operator": "ge"}, {"budget": 0.69, "threshold": -5.367645263671875, "operator": "ge"}, {"budget": 0.7, "threshold": -5.382306098937988, "operator": "ge"}, {"budget": 0.71, "threshold": -5.4086713790893555, "operator": "ge"}, {"budget": 0.72, "threshold": -5.429797649383545, "operator": "ge"}, {"budget": 0.73, "threshold": -5.467480182647705, "operator": "ge"}, {"budget": 0.74, "threshold": -5.489755630493164, "operator": "ge"}, {"budget": 0.75, "threshold": -5.520005702972412, "operator": "ge"}, {"budget": 0.76, "threshold": -5.546342849731445, "operator": "ge"}, {"budget": 0.77, "threshold": -5.580183029174805, "operator": "ge"}, {"budget": 0.78, "threshold": -5.617748737335205, "operator": "ge"}, {"budget": 0.79, "threshold": -5.654303550720215, "operator": "ge"}, {"budget": 0.8, "threshold": -5.701037406921387, "operator": "ge"}, {"budget": 0.81, "threshold": -5.912628173828125, "operator": "ge"}, {"budget": 0.82, "threshold": -15.007793426513672, "operator": "ge"}, {"budget": 0.83, "threshold": -15.057239532470703, "operator": "ge"}, {"budget": 0.84, "threshold": -15.078920364379883, "operator": "ge"}, {"budget": 0.85, "threshold": -15.104199409484863, "operator": "ge"}, {"budget": 0.86, "threshold": -15.16373348236084, "operator": "ge"}, {"budget": 0.87, "threshold": -15.19938850402832, "operator": "ge"}, {"budget": 0.88, "threshold": -15.239335060119629, "operator": "ge"}, {"budget": 0.89, "threshold": -15.260358810424805, "operator": "ge"}, {"budget": 0.9, "threshold": -15.276138305664062, "operator": "ge"}, {"budget": 0.91, "threshold": -15.300268173217773, "operator": "ge"}, {"budget": 0.92, "threshold": -15.327590942382812, "operator": "ge"}, {"budget": 0.93, "threshold": -15.357640266418457, "operator": "ge"}, {"budget": 0.94, "threshold": -15.38984489440918, "operator": "ge"}, {"budget": 0.95, "threshold": -15.43847370147705, "operator": "ge"}, {"budget": 0.96, "threshold": -15.473119735717773, "operator": "ge"}, {"budget": 0.97, "threshold": -15.539250373840332, "operator": "ge"}, {"budget": 0.98, "threshold": -15.590731620788574, "operator": "ge"}, {"budget": 0.99, "threshold": -15.6785888671875, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.60983981693364, 19.003857430637026, 19.25578786043902, 19.36212428233705, 20.260338736482435, 20.67898290123156, 21.355824229076088, 21.87178266681346, 22.348014115441313, 22.678687796941286, 22.647425991673266, 23.791737264918357, 24.01069305266696, 24.401513211823854, 24.391658625580465, 24.610601655110713, 24.892590563075736, 25.044624128589437, 25.203484116527598, 24.933708654133568, 25.043661984979828, 25.26976007794422, 25.46391854880324, 25.40088451042251, 25.396206690561527, 25.519754761545805, 25.710166326950954, 26.10911705303829, 26.612969129358895, 27.169541900885186, 27.15027036455608, 27.394265607580827, 27.35364786593729, 28.20056903588768, 28.571909347771417, 26.88968633705476, 27.852158677236076, 28.2521769943596, 28.2521769943596, 28.82292101237658, 27.96265148107854, 27.92485470139379, 27.957676861291326, 28.573281987707194, 28.971423275273057, 28.64514777488093, 28.665595885818483, 28.27112939733938, 28.371234509928524, 27.894044839896033, 27.845429821284558, 27.757358236700608, 27.66608377455028, 27.115534738485557, 27.278974904673227, 27.174899696821154, 27.002160531572294, 26.84110537342611, 26.777462452003537, 26.607233800588396, 26.47957063837013, 26.235202705790943, 26.151458820338114, 26.013168016652333, 25.51403077718867, 25.462238860817223, 25.408383478896486, 25.248111000747254, 25.197634506495014, 25.077455231310992, 24.545938515318458, 24.038046528297617, 23.901693940044076, 23.402315435030694, 23.07112966749981, 22.94006264312546, 22.833954451345758, 22.768968456947995, 22.71228339318713, 22.352558690957284, 22.308619426490953, 22.29024616665066, 22.256467965423184, 22.217674239172712, 22.115020124526747, 22.08313018397654, 21.962345881789837, 21.943166149465682, 21.92426608521922, 21.753408601921105, 21.724797717876417, 21.685756608132134, 21.655515327646427, 21.62134907934716, 21.311950129479968, 21.272762821615462, 21.065615545192067, 20.889161741314176, 20.68874825884421, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 20, 30, 42, 55, 67, 79, 92, 99, 110, 115, 122, 127, 134, 136, 142, 148, 153, 159, 163, 168, 173, 179, 183, 188, 193, 199, 205, 212, 217, 222, 227, 231, 236, 240, 244, 248, 251, 253, 259, 263, 264, 265, 269, 273, 274, 276, 277, 279, 280, 283, 283, 283, 283, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 1, 2, 4, 8, 11, 15, 17, 21, 22, 25, 33, 37, 42, 46, 48, 52, 58, 62, 67, 72, 78, 83, 87, 92, 101, 105, 109, 115, 122, 129, 134, 138, 146, 153, 159, 169, 173, 179, 181, 187, 194, 197, 200, 204, 205, 209, 216, 224, 229, 236, 243, 248, 254, 259, 266, 273, 279, 285, 290, 297, 301, 308, 315, 320, 326, 335, 340, 348, 353, 357, 364, 369, 375, 382, 389, 396, 402, 409, 416, 424, 429, 431, 436, 441, 446, 450, 457, 460, 463, 467, 472, 478, 483, 489, 494, 500, 503, 510, 514, 518], "exact_score": [18.254072168965788, 18.60983981693364, 19.003857430637026, 19.25578786043902, 19.36212428233705, 20.260338736482435, 20.67898290123156, 21.355824229076088, 21.87178266681346, 22.348014115441313, 22.678687796941286, 22.647425991673266, 23.791737264918357, 24.01069305266696, 24.401513211823854, 24.391658625580465, 24.610601655110713, 24.892590563075736, 25.044624128589437, 25.203484116527598, 24.933708654133568, 25.043661984979828, 25.26976007794422, 25.46391854880324, 25.40088451042251, 25.396206690561527, 25.519754761545805, 25.710166326950954, 26.10911705303829, 26.612969129358895, 27.169541900885186, 27.15027036455608, 27.394265607580827, 27.35364786593729, 28.20056903588768, 28.571909347771417, 26.88968633705476, 27.852158677236076, 28.2521769943596, 28.2521769943596, 28.82292101237658, 27.96265148107854, 27.92485470139379, 27.957676861291326, 28.573281987707194, 28.971423275273057, 28.64514777488093, 28.665595885818483, 28.27112939733938, 28.371234509928524, 27.894044839896033, 27.845429821284558, 27.757358236700608, 27.66608377455028, 27.115534738485557, 27.278974904673227, 27.174899696821154, 27.002160531572294, 26.84110537342611, 26.777462452003537, 26.607233800588396, 26.47957063837013, 26.235202705790943, 26.151458820338114, 26.013168016652333, 25.51403077718867, 25.462238860817223, 25.408383478896486, 25.248111000747254, 25.197634506495014, 25.077455231310992, 24.545938515318458, 24.038046528297617, 23.901693940044076, 23.402315435030694, 23.07112966749981, 22.94006264312546, 22.833954451345758, 22.768968456947995, 22.71228339318713, 22.352558690957284, 22.308619426490953, 22.29024616665066, 22.256467965423184, 22.217674239172712, 22.115020124526747, 22.08313018397654, 21.962345881789837, 21.943166149465682, 21.92426608521922, 21.753408601921105, 21.724797717876417, 21.685756608132134, 21.655515327646427, 21.62134907934716, 21.311950129479968, 21.272762821615462, 21.065615545192067, 20.889161741314176, 20.68874825884421, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.621131839414748, "threshold_area100": 24.626765766954996, "threshold_area25": 22.817906648580284}, "test": {"score": [18.129624718318375, 18.382593870614528, 18.59065767854782, 19.09122347478594, 19.665490789655806, 20.069908073550717, 20.493046731140023, 20.58079142817783, 20.727507437321513, 20.889737928369957, 21.06689403577989, 21.450499344045582, 21.705100333803585, 21.63212725574914, 21.951757164335405, 22.084839836132062, 22.378644869046987, 22.3554183341513, 22.340615957905126, 22.322489197341593, 22.390456499057628, 22.783601383802456, 23.655079133055573, 24.036951547993873, 24.239726254391197, 24.025574249335197, 24.16841736694678, 24.243896644412462, 24.477015580096577, 24.602414963368624, 25.332728852219468, 25.459404535579615, 25.206702469911196, 25.14790519649826, 25.403188456161057, 25.81898917879365, 25.864233457199703, 25.92895309825077, 26.136454122142478, 26.25472766255252, 26.36679521437471, 26.445077685105012, 26.28560313545672, 25.800798885222914, 25.80706917263671, 26.04612003548174, 25.99931002468609, 26.58559995094512, 26.387548550151102, 25.942901704818865, 25.108654057065387, 24.94504158092616, 24.153775819019796, 24.179181776507278, 24.064296574299274, 24.016701601955624, 23.919921138876862, 23.732174409059922, 24.207521657387467, 24.171866039591595, 24.631438896891964, 24.52109417552582, 24.190378907362348, 24.229324967645834, 24.17156275702414, 24.105986575051666, 23.977713016332764, 23.93663103637701, 23.85560233822062, 23.755873376587736, 23.686425983471956, 23.42671984050658, 23.508533286960347, 23.490948530421, 23.395301721115924, 23.329404218654176, 23.19303026923527, 22.874614129606506, 22.789003032520096, 22.643662880243294, 23.128008770614404, 22.846911703255994, 22.794818949663288, 22.756042184263066, 22.732278245816104, 22.70979522375675, 22.527171642510528, 22.48215302509947, 22.457526651449776, 22.440187988065027, 22.416089230741278, 22.759946764177496, 22.737479727781775, 22.560583086765664, 22.49044486927873, 22.38238768465272, 22.313136840273913, 22.22031271188317, 22.196567623285777, 22.063570325884506, 22.361115727328066], "calls": [0.0, 1.1984021304926764, 2.097203728362184, 3.362183754993342, 5.193075898801598, 6.291611185086551, 7.290279627163781, 8.322237017310254, 9.087882822902795, 9.88681757656458, 10.785619174434089, 12.416777629826898, 13.11584553928096, 13.81491344873502, 14.713715046604529, 15.679094540612518, 16.11185086551265, 16.378162450066576, 16.977363515312916, 18.77496671105193, 19.573901464713714, 21.271637816245008, 23.93475366178429, 25.59920106524634, 26.531291611185086, 27.82956058588549, 28.428761651131822, 29.094540612516646, 29.69374167776298, 30.459387483355528, 31.557922769640477, 32.19041278295606, 33.422103861517975, 33.788282290279625, 34.85352862849534, 36.38482023968043, 37.48335552596538, 37.849533954727036, 38.81491344873502, 39.58055925432756, 40.41278295605859, 40.87882822902797, 42.143808255659124, 42.709720372836216, 43.00932090545938, 44.0745672436751, 45.47270306258323, 46.03861517976032, 47.470039946737685, 48.402130492676434, 49.533954727030626, 50.03328894806924, 51.431424766977365, 52.86284953395472, 54.19440745672437, 54.72703062583223, 56.02529960053262, 57.98934753661784, 59.58721704394141, 60.918774966711055, 62.18375499334221, 63.08255659121171, 65.47936085219706, 65.71238348868175, 66.41145139813582, 66.94407456724367, 68.07589880159787, 68.64181091877496, 69.47403462050599, 70.60585885486019, 71.17177097203728, 72.10386151797603, 73.30226364846871, 75.26631158455392, 75.99866844207723, 77.3635153129161, 78.22902796271637, 79.39414114513981, 79.9600532623169, 80.62583222370174, 81.22503328894807, 81.92410119840213, 82.75632490013317, 84.02130492676432, 85.01997336884153, 85.81890812250333, 88.04926764314249, 88.81491344873503, 89.94673768308921, 90.5126498002663, 91.01198402130493, 92.01065246338216, 92.90945406125167, 93.74167776298269, 94.3075898801598, 95.63914780292943, 96.33821571238349, 98.03595206391478, 98.7017310252996, 99.53395472703063, 100.0], "rescued": [0, 64, 94, 141, 210, 258, 299, 334, 363, 394, 422, 471, 489, 509, 533, 553, 565, 571, 583, 628, 647, 691, 750, 787, 811, 841, 854, 868, 883, 901, 927, 940, 966, 974, 998, 1032, 1051, 1057, 1071, 1085, 1096, 1103, 1116, 1125, 1127, 1137, 1149, 1154, 1162, 1165, 1168, 1169, 1171, 1173, 1174, 1174, 1175, 1176, 1178, 1179, 1183, 1183, 1183, 1184, 1184, 1184, 1185, 1185, 1185, 1185, 1185, 1185, 1188, 1191, 1191, 1192, 1192, 1192, 1192, 1193, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1195, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1197, 1197, 1198], "harmed": [0, 4, 9, 20, 40, 54, 66, 82, 94, 111, 125, 153, 168, 183, 201, 221, 230, 238, 249, 285, 300, 334, 397, 427, 445, 477, 488, 498, 511, 527, 554, 566, 592, 601, 627, 661, 680, 688, 702, 715, 733, 737, 762, 776, 777, 785, 815, 823, 855, 882, 912, 923, 968, 995, 1030, 1039, 1067, 1112, 1148, 1178, 1214, 1239, 1300, 1305, 1329, 1341, 1371, 1386, 1406, 1441, 1455, 1480, 1513, 1569, 1589, 1625, 1645, 1671, 1691, 1710, 1728, 1744, 1763, 1784, 1801, 1814, 1855, 1871, 1887, 1897, 1912, 1933, 1947, 1966, 1980, 2003, 2022, 2061, 2081, 2101, 2109], "exact_score": [18.129624718318375, 18.31569007222758, 18.563598370360886, 19.01666616982306, 19.416495546692158, 19.642747802489914, 19.977574590530725, 20.438689942093074, 20.528511053886454, 20.68702367893193, 20.847279857762786, 21.057161200816267, 21.30480430343109, 21.71311028796059, 21.716796283698102, 22.017444121581832, 22.361367481925857, 22.340615957905126, 22.616149569736827, 22.42265583645516, 22.528121130650565, 22.6986784413264, 23.319506828595518, 23.59722006139545, 23.655079133055573, 23.959066100147183, 24.16958643044923, 24.27443524596021, 24.058326736325768, 24.249195499859685, 24.45468190536885, 25.150502648202362, 25.48238327324367, 25.165160053608155, 25.23697250713848, 25.572251472251473, 25.787989863548315, 25.989465422736064, 25.976004469583973, 26.15606482537257, 26.345107023815473, 26.460540311294118, 26.317772391698487, 25.80220600987881, 26.049761369666584, 26.0782979388894, 26.58559995094512, 26.347918851613418, 26.112451034947647, 25.22262340442976, 24.999978673334113, 24.32465236011591, 24.117110318523405, 24.165519139843042, 24.09500711237156, 23.98753453668589, 23.919921138876862, 23.86964555914885, 23.732174409059922, 24.184416034455023, 24.177534243475215, 24.215503975498045, 24.587989962462085, 24.52916219261366, 24.46763885891261, 24.24588507179655, 24.196695593469496, 24.102881542138316, 23.98274276919391, 23.917002758328902, 23.81895981236492, 23.71691654706951, 23.43655939893976, 23.52812876528128, 23.570797164944953, 23.50942469637511, 23.395301721115924, 23.415527666678322, 23.226049414868985, 22.943502626877706, 22.78682825988269, 22.885383722242636, 22.8430429838889, 22.791471071659963, 22.758132039062936, 22.732278245816104, 22.656600580527808, 22.63046816670785, 22.529170682971106, 22.479232720219645, 22.457526651449776, 22.416089230741278, 22.761826438188315, 22.717866286138268, 22.502683553744323, 22.42962687612082, 22.328133727632014, 22.294394396831507, 22.223238498662127, 22.154919404444783, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.378319078168623, "threshold_area100": 23.4209575681759, "threshold_area25": 21.337939585669417}, "selected_budget": 45, "selected": {"val": {"score": 28.971423275273057, "calls": 44.935805991440795, "rescued": 273, "harmed": 205}, "test": {"score": 26.04612003548174, "calls": 44.0745672436751, "rescued": 1137, "harmed": 785}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9221049547195435, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8365728259086609, "operator": "ge"}, {"budget": 0.03, "threshold": 0.7859477996826172, "operator": "ge"}, {"budget": 0.04, "threshold": 0.7421367764472961, "operator": "ge"}, {"budget": 0.05, "threshold": 0.6796948909759521, "operator": "ge"}, {"budget": 0.06, "threshold": 0.6401472091674805, "operator": "ge"}, {"budget": 0.07, "threshold": 0.6107075214385986, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5656745433807373, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4411364197731018, "operator": "ge"}, {"budget": 0.1, "threshold": 0.33175912499427795, "operator": "ge"}, {"budget": 0.11, "threshold": 0.29554903507232666, "operator": "ge"}, {"budget": 0.12, "threshold": 0.2619883418083191, "operator": "ge"}, {"budget": 0.13, "threshold": 0.2495841383934021, "operator": "ge"}, {"budget": 0.14, "threshold": 0.22584131360054016, "operator": "ge"}, {"budget": 0.15, "threshold": 0.2159239649772644, "operator": "ge"}, {"budget": 0.16, "threshold": 0.19584470987319946, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1791682243347168, "operator": "ge"}, {"budget": 0.18, "threshold": 0.16830039024353027, "operator": "ge"}, {"budget": 0.19, "threshold": 0.15583252906799316, "operator": "ge"}, {"budget": 0.2, "threshold": 0.13171344995498657, "operator": "ge"}, {"budget": 0.21, "threshold": 0.12352046370506287, "operator": "ge"}, {"budget": 0.22, "threshold": 0.11289764940738678, "operator": "ge"}, {"budget": 0.23, "threshold": 0.10183548927307129, "operator": "ge"}, {"budget": 0.24, "threshold": 0.09349328279495239, "operator": "ge"}, {"budget": 0.25, "threshold": 0.08419087529182434, "operator": "ge"}, {"budget": 0.26, "threshold": 0.07196560502052307, "operator": "ge"}, {"budget": 0.27, "threshold": 0.06521463394165039, "operator": "ge"}, {"budget": 0.28, "threshold": 0.060171812772750854, "operator": "ge"}, {"budget": 0.29, "threshold": 0.04970255494117737, "operator": "ge"}, {"budget": 0.3, "threshold": 0.0377526730298996, "operator": "ge"}, {"budget": 0.31, "threshold": 0.03142660856246948, "operator": "ge"}, {"budget": 0.32, "threshold": 0.013733386993408203, "operator": "ge"}, {"budget": 0.33, "threshold": 0.007284939289093018, "operator": "ge"}, {"budget": 0.34, "threshold": -0.006447315216064453, "operator": "ge"}, {"budget": 0.35, "threshold": -0.03464615345001221, "operator": "ge"}, {"budget": 0.36, "threshold": -0.04505756497383118, "operator": "ge"}, {"budget": 0.37, "threshold": -0.061937540769577026, "operator": "ge"}, {"budget": 0.38, "threshold": -0.07352334260940552, "operator": "ge"}, {"budget": 0.39, "threshold": -0.1000504195690155, "operator": "ge"}, {"budget": 0.4, "threshold": -0.11327677965164185, "operator": "ge"}, {"budget": 0.41, "threshold": -0.14117521047592163, "operator": "ge"}, {"budget": 0.42, "threshold": -0.19040736556053162, "operator": "ge"}, {"budget": 0.43, "threshold": -0.22556966543197632, "operator": "ge"}, {"budget": 0.44, "threshold": -0.26130568981170654, "operator": "ge"}, {"budget": 0.45, "threshold": -0.4274575412273407, "operator": "ge"}, {"budget": 0.46, "threshold": -1.0016696453094482, "operator": "ge"}, {"budget": 0.47, "threshold": -1.6876800060272217, "operator": "ge"}, {"budget": 0.48, "threshold": -1.9831135272979736, "operator": "ge"}, {"budget": 0.49, "threshold": -2.2545363903045654, "operator": "ge"}, {"budget": 0.5, "threshold": -2.4646239280700684, "operator": "ge"}, {"budget": 0.51, "threshold": -2.732424736022949, "operator": "ge"}, {"budget": 0.52, "threshold": -4.288111209869385, "operator": "ge"}, {"budget": 0.53, "threshold": -4.321200847625732, "operator": "ge"}, {"budget": 0.54, "threshold": -4.351518154144287, "operator": "ge"}, {"budget": 0.55, "threshold": -4.412347793579102, "operator": "ge"}, {"budget": 0.56, "threshold": -4.459195613861084, "operator": "ge"}, {"budget": 0.57, "threshold": -4.4801530838012695, "operator": "ge"}, {"budget": 0.58, "threshold": -4.511013984680176, "operator": "ge"}, {"budget": 0.59, "threshold": -4.523858070373535, "operator": "ge"}, {"budget": 0.6, "threshold": -4.564365386962891, "operator": "ge"}, {"budget": 0.61, "threshold": -4.5864787101745605, "operator": "ge"}, {"budget": 0.62, "threshold": -4.594547271728516, "operator": "ge"}, {"budget": 0.63, "threshold": -4.630983352661133, "operator": "ge"}, {"budget": 0.64, "threshold": -4.642690658569336, "operator": "ge"}, {"budget": 0.65, "threshold": -4.671256065368652, "operator": "ge"}, {"budget": 0.66, "threshold": -4.681416988372803, "operator": "ge"}, {"budget": 0.67, "threshold": -4.693335056304932, "operator": "ge"}, {"budget": 0.68, "threshold": -4.700372695922852, "operator": "ge"}, {"budget": 0.69, "threshold": -4.716997146606445, "operator": "ge"}, {"budget": 0.7, "threshold": -4.7276506423950195, "operator": "ge"}, {"budget": 0.71, "threshold": -4.751594543457031, "operator": "ge"}, {"budget": 0.72, "threshold": -4.76206111907959, "operator": "ge"}, {"budget": 0.73, "threshold": -4.792182922363281, "operator": "ge"}, {"budget": 0.74, "threshold": -4.802939414978027, "operator": "ge"}, {"budget": 0.75, "threshold": -4.813704490661621, "operator": "ge"}, {"budget": 0.76, "threshold": -4.833223819732666, "operator": "ge"}, {"budget": 0.77, "threshold": -4.854098320007324, "operator": "ge"}, {"budget": 0.78, "threshold": -4.877076148986816, "operator": "ge"}, {"budget": 0.79, "threshold": -4.914061546325684, "operator": "ge"}, {"budget": 0.8, "threshold": -4.968216419219971, "operator": "ge"}, {"budget": 0.81, "threshold": -5.144789218902588, "operator": "ge"}, {"budget": 0.82, "threshold": -12.450653076171875, "operator": "ge"}, {"budget": 0.83, "threshold": -12.520683288574219, "operator": "ge"}, {"budget": 0.84, "threshold": -12.612448692321777, "operator": "ge"}, {"budget": 0.85, "threshold": -12.688196182250977, "operator": "ge"}, {"budget": 0.86, "threshold": -12.737610816955566, "operator": "ge"}, {"budget": 0.87, "threshold": -12.79923152923584, "operator": "ge"}, {"budget": 0.88, "threshold": -12.858390808105469, "operator": "ge"}, {"budget": 0.89, "threshold": -12.881017684936523, "operator": "ge"}, {"budget": 0.9, "threshold": -12.920239448547363, "operator": "ge"}, {"budget": 0.91, "threshold": -12.971824645996094, "operator": "ge"}, {"budget": 0.92, "threshold": -13.041620254516602, "operator": "ge"}, {"budget": 0.93, "threshold": -13.085198402404785, "operator": "ge"}, {"budget": 0.94, "threshold": -13.159390449523926, "operator": "ge"}, {"budget": 0.95, "threshold": -13.311628341674805, "operator": "ge"}, {"budget": 0.96, "threshold": -13.448036193847656, "operator": "ge"}, {"budget": 0.97, "threshold": -13.526050567626953, "operator": "ge"}, {"budget": 0.98, "threshold": -13.618658065795898, "operator": "ge"}, {"budget": 0.99, "threshold": -13.660520553588867, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.399938738527112, 19.226878453976603, 19.750728097553658, 20.315459313364002, 20.714049177821195, 20.913914788914788, 21.406442868878194, 21.715932656472802, 22.405168361050716, 22.678687796941286, 23.989185417756847, 24.345835895333746, 24.604350682531575, 24.654895666131623, 25.025521169138194, 25.304913776231114, 25.44820127991334, 25.32682492859484, 25.447737380586744, 25.48247481197673, 25.50775256399903, 25.70851151761716, 25.76950320358028, 25.68280669522321, 25.64555192357883, 25.614014408856704, 26.045925899799737, 26.25593918276845, 27.08342325388673, 27.37291541089009, 27.320053604126592, 27.405741804973033, 27.35745227307668, 27.819483115585065, 28.559575345289627, 26.661858487701185, 26.850770093872, 27.74233616891733, 27.36474476609564, 27.65229650195841, 27.97426276352199, 28.011612003026485, 27.950379813679007, 28.088757396449704, 28.55822749811899, 28.64514777488093, 28.665595885818483, 28.27112939733938, 28.431758812603668, 27.894044839896033, 27.728039511926983, 27.742622015067838, 27.61681945505475, 27.523163111398407, 27.92674931934372, 27.152293064876957, 26.886303380733068, 26.734197433883217, 26.63845207531023, 26.57439108890714, 26.442683943806532, 25.92472237321338, 25.854002097646827, 25.42190380425675, 25.19754676063974, 25.13349347975292, 24.97051468667189, 24.83374299343362, 24.7085284158383, 24.651520272351693, 24.531971333218532, 24.029247267883633, 23.96109733188529, 23.917857142857144, 23.892863291103662, 23.454733672124974, 23.299219547790212, 22.986562723184797, 22.703490360083116, 22.427049062691314, 22.308619426490953, 22.213593810363434, 22.156162712983317, 21.95904095904096, 21.915714073987456, 21.88175341175454, 21.431046790089344, 21.406876398042474, 21.401643535666757, 21.290573549609697, 21.259714586847746, 21.249911386102703, 21.223063789757525, 21.122607931532134, 21.102804143054698, 21.018969117687643, 20.98766824234114, 20.776559394721243, 20.748171368861023, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 24, 34, 47, 59, 68, 80, 91, 102, 110, 117, 123, 128, 133, 139, 146, 151, 156, 162, 167, 171, 176, 180, 185, 190, 195, 202, 209, 214, 220, 223, 227, 231, 235, 241, 243, 247, 251, 254, 256, 257, 259, 265, 268, 270, 274, 276, 277, 280, 280, 282, 283, 283, 283, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 3, 5, 7, 9, 13, 16, 18, 22, 24, 25, 32, 37, 40, 45, 50, 57, 61, 66, 71, 76, 81, 84, 87, 95, 102, 107, 113, 120, 127, 131, 135, 140, 149, 152, 160, 166, 169, 176, 181, 186, 186, 186, 195, 198, 204, 209, 216, 224, 230, 236, 244, 249, 257, 263, 270, 279, 286, 291, 298, 303, 307, 314, 320, 324, 330, 336, 344, 350, 355, 361, 366, 374, 381, 386, 393, 399, 405, 411, 416, 423, 429, 432, 439, 446, 452, 458, 465, 469, 470, 476, 481, 483, 488, 493, 496, 499, 504, 509, 515, 518], "exact_score": [18.254072168965788, 18.399938738527112, 19.226878453976603, 19.750728097553658, 20.315459313364002, 20.714049177821195, 20.913914788914788, 21.406442868878194, 21.715932656472802, 22.405168361050716, 22.678687796941286, 23.989185417756847, 24.345835895333746, 24.604350682531575, 24.654895666131623, 25.025521169138194, 25.304913776231114, 25.44820127991334, 25.32682492859484, 25.447737380586744, 25.48247481197673, 25.50775256399903, 25.70851151761716, 25.76950320358028, 25.68280669522321, 25.64555192357883, 25.614014408856704, 26.045925899799737, 26.25593918276845, 27.08342325388673, 27.37291541089009, 27.320053604126592, 27.405741804973033, 27.35745227307668, 27.819483115585065, 28.559575345289627, 26.661858487701185, 26.850770093872, 27.74233616891733, 27.36474476609564, 27.65229650195841, 27.97426276352199, 28.011612003026485, 27.950379813679007, 28.088757396449704, 28.55822749811899, 28.64514777488093, 28.665595885818483, 28.27112939733938, 28.431758812603668, 27.894044839896033, 27.728039511926983, 27.742622015067838, 27.61681945505475, 27.523163111398407, 27.92674931934372, 27.152293064876957, 26.886303380733068, 26.734197433883217, 26.63845207531023, 26.57439108890714, 26.442683943806532, 25.92472237321338, 25.854002097646827, 25.42190380425675, 25.19754676063974, 25.13349347975292, 24.97051468667189, 24.83374299343362, 24.7085284158383, 24.651520272351693, 24.531971333218532, 24.029247267883633, 23.96109733188529, 23.917857142857144, 23.892863291103662, 23.454733672124974, 23.299219547790212, 22.986562723184797, 22.703490360083116, 22.427049062691314, 22.308619426490953, 22.213593810363434, 22.156162712983317, 21.95904095904096, 21.915714073987456, 21.88175341175454, 21.431046790089344, 21.406876398042474, 21.401643535666757, 21.290573549609697, 21.259714586847746, 21.249911386102703, 21.223063789757525, 21.122607931532134, 21.102804143054698, 21.018969117687643, 20.98766824234114, 20.776559394721243, 20.748171368861023, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.622896335140094, "threshold_area100": 24.62849033830995, "threshold_area25": 23.170109000429946}, "test": {"score": [18.129624718318375, 18.197590535431267, 18.51105230546925, 18.696794381852673, 18.780836075343498, 19.152548690122355, 20.134826154543564, 20.395883928869548, 20.556904467641345, 21.0308506461535, 21.244784053448562, 21.55407366734665, 21.578321445862407, 21.691144683187865, 22.22295930794951, 22.185600507758107, 22.367934371412286, 22.648955694420987, 22.77557753613079, 22.53555266816639, 23.5114160387794, 23.716617054196888, 24.000000000000004, 24.326793009963733, 24.32956750578575, 24.050028286644196, 24.259743706399757, 24.46432289913415, 24.54112932215373, 25.208119552209478, 25.46356330671305, 25.472145285642632, 25.183287162246078, 25.187492216706502, 25.405400079710706, 25.777640468411235, 25.89628289177744, 25.851738819007274, 25.93288959684454, 26.057904576302853, 26.095431747792365, 26.122640990366815, 26.417470022121186, 26.45368860038586, 26.39956722580416, 25.78183359180582, 25.99724753470164, 26.600884614650216, 26.42491644334311, 25.452559958509614, 25.389662661807723, 24.559668730628857, 24.01495646501403, 24.022727027347845, 23.949176769304092, 23.85051433492404, 23.852618414409854, 23.85914097998395, 23.79267126211971, 23.75945659944745, 23.463792022409084, 23.36069361261129, 23.286772571869893, 23.5705596761551, 23.501411881096928, 23.76596321641762, 23.747161776395647, 23.809744088091726, 23.748479292260765, 23.57364028200902, 23.403796423477814, 23.259252098243426, 23.14878711854594, 23.03873808969995, 23.018255161419628, 22.93778198457226, 22.80926080417816, 22.576622379218996, 22.455439543682708, 22.638021195706816, 22.452510173203038, 22.846494666090877, 22.795219655308752, 22.752905788786357, 22.577521558362477, 22.539939428891547, 22.46386736047753, 22.789147170687336, 22.704161869333454, 22.65055190403367, 22.616674294956535, 22.54654883214096, 22.457425809118075, 22.39512626475882, 22.277919741519394, 22.599443023191366, 22.551344324687427, 22.480838762909467, 22.414559549720046, 22.39147221998011, 22.361115727328066], "calls": [0.0, 0.8322237017310253, 1.7310252996005324, 2.2303595206391478, 2.796271637816245, 4.094540612516644, 5.59254327563249, 6.55792276964048, 7.689747003994674, 9.820239680426099, 10.818908122503329, 11.750998668442076, 12.81624500665779, 13.382157123834887, 14.547270306258323, 15.013315579227696, 16.045272969374167, 17.14380825565912, 17.7762982689747, 18.841544607190414, 20.938748335552596, 22.00399467376831, 23.302263648468706, 24.63382157123835, 25.432756324900136, 26.331557922769637, 27.729693741677764, 28.495339547270305, 29.127829560585884, 30.392809587217045, 31.458055925432753, 31.990679094540614, 33.222370173102526, 33.85486018641811, 34.720372836218374, 36.78428761651132, 37.48335552596538, 38.28229027962716, 38.68175765645805, 39.647137150466044, 40.1797603195739, 40.97869507323569, 42.2103861517976, 42.80958721704394, 43.37549933422104, 44.50732356857524, 45.572569906790946, 46.33821571238349, 46.904127829560586, 48.1358189081225, 49.23435419440746, 50.76564580559254, 51.997336884154464, 52.330226364846865, 52.82956058588548, 54.061251664447404, 55.22636484687084, 56.05858854860186, 57.25699067909454, 57.689747003994675, 60.21970705725699, 62.150466045272964, 62.849533954727036, 65.47936085219706, 66.04527296937417, 67.97603195739015, 68.80825565912117, 69.6404793608522, 70.23968042609854, 71.47137150466045, 72.30359520639148, 73.63515312916113, 74.36750998668442, 75.76564580559254, 76.16511318242344, 76.56458055925432, 77.46338215712385, 78.12916111850865, 78.76165113182424, 79.8601864181092, 80.89214380825565, 81.92410119840213, 82.75632490013317, 83.45539280958721, 84.45406125166444, 85.41944074567243, 86.21837549933421, 87.01731025299601, 88.08255659121171, 88.64846870838882, 89.64713715046605, 91.44474034620505, 93.07589880159787, 93.97470039946738, 94.87350199733689, 96.47137150466045, 97.6364846870839, 98.36884154460719, 99.2676431424767, 99.50066577896138, 100.0], "rescued": [0, 44, 90, 115, 139, 198, 259, 294, 331, 401, 430, 457, 487, 502, 529, 541, 567, 595, 611, 635, 688, 712, 745, 779, 799, 820, 849, 862, 875, 901, 925, 936, 962, 976, 995, 1036, 1047, 1057, 1060, 1075, 1081, 1091, 1105, 1111, 1117, 1130, 1146, 1155, 1158, 1161, 1168, 1170, 1171, 1172, 1172, 1173, 1175, 1177, 1179, 1179, 1180, 1182, 1182, 1185, 1185, 1186, 1187, 1189, 1189, 1189, 1189, 1189, 1189, 1191, 1191, 1191, 1192, 1192, 1192, 1193, 1193, 1195, 1195, 1195, 1195, 1195, 1195, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1197, 1198, 1198, 1198, 1198, 1198], "harmed": [0, 11, 21, 25, 34, 55, 72, 85, 99, 118, 129, 145, 169, 181, 201, 214, 234, 258, 269, 294, 339, 367, 393, 419, 438, 463, 489, 502, 514, 537, 552, 563, 590, 602, 622, 667, 676, 686, 691, 704, 712, 725, 738, 750, 760, 790, 810, 825, 843, 875, 899, 944, 983, 995, 1010, 1040, 1067, 1088, 1117, 1131, 1202, 1259, 1279, 1336, 1349, 1403, 1427, 1446, 1464, 1490, 1510, 1548, 1568, 1598, 1611, 1622, 1644, 1656, 1672, 1697, 1720, 1744, 1766, 1780, 1805, 1825, 1842, 1862, 1883, 1896, 1916, 1950, 1981, 1991, 2007, 2033, 2059, 2079, 2094, 2099, 2109], "exact_score": [18.129624718318375, 18.23325860872402, 18.629029874826227, 18.84689922168456, 19.120652230181378, 20.00028755343271, 20.234627793929448, 20.44378111375328, 20.627185990676885, 20.825324312802948, 21.087331996316724, 21.249894501179188, 21.65165097709865, 21.57174516644092, 21.835550500398337, 22.177820424770555, 22.3980421477634, 22.559887831595145, 22.797690149882538, 22.640974981589366, 22.775569297888648, 23.56376707628178, 23.716944649357462, 23.893820737679945, 24.169476351029584, 24.28079407225884, 23.997812826172513, 24.378940994481034, 24.32790222590934, 24.565192388516145, 25.117790938351032, 25.318734158337854, 25.472145285642632, 25.170088468920245, 25.19564059480312, 25.448015786547707, 25.743401973255796, 25.74952740356573, 25.813959144472175, 25.912534598480203, 26.081111748808777, 26.122640990366815, 26.27223871387292, 26.469911965444208, 26.263259031449255, 25.899549705067876, 26.026146686582365, 26.443767246837425, 25.557969833424675, 25.50684062999436, 24.960464198015746, 24.320401213210744, 24.01495646501403, 23.896405337678818, 23.853676043288058, 23.87127691313212, 23.882523845159103, 23.66093586362287, 23.764826262631942, 23.642829753328787, 23.510518519094127, 23.45849372123297, 23.39230324329549, 23.275570240953876, 23.207849767610142, 23.570360397042208, 23.50424290830043, 23.381455050928388, 23.76596321641762, 23.735163668158833, 23.78670413094689, 23.674010636569182, 23.423215430285047, 23.354433773874078, 23.186787750023765, 23.166837789102722, 23.029428402970908, 22.845488218153296, 22.578455956418097, 22.44333196418465, 22.58444857508166, 22.449074738757997, 22.8430429838889, 22.7859407460296, 22.66299754652584, 22.556417727764387, 22.476514714527955, 22.791093236853584, 22.70907980221998, 22.63855428951589, 22.60863634705955, 22.560812423597778, 22.53606499716557, 22.458746942726204, 22.39512626475882, 22.27663914439525, 22.251085341652818, 22.633197193007355, 22.48955241003318, 22.41978952957946, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.29779389341124, "threshold_area100": 23.343583227132516, "threshold_area25": 21.436835158517948}, "selected_budget": 47, "selected": {"val": {"score": 28.665595885818483, "calls": 46.932952924393724, "rescued": 276, "harmed": 216}, "test": {"score": 26.600884614650216, "calls": 46.33821571238349, "rescued": 1155, "harmed": 825}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 28.07737471705083, "calls": 45.93437945791726}, "test": {"score": 25.574754185006398, "calls": 45.80559254327563}}, "old_normalized_interior": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726}, "test": {"score": 26.45767198021416, "calls": 45.572569906790946}}, "old_normalized_endpoints": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726}, "test": {"score": 26.45767198021416, "calls": 45.572569906790946}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.360708626971523, 19.30434700184981, 19.78379620768573, 20.315459313364002, 20.75187036597774, 21.000618715278087, 21.41668676151435, 21.79169023223038, 22.461889614145253, 22.804363392954436, 22.865196078431374, 22.800729636436014, 22.883470193693977, 23.012060654957693, 24.314715288187124, 24.663680167762305, 24.695773002799204, 25.258164568509393, 26.040583554376656, 26.47570092221384, 24.861604546070744, 25.348487606433057, 25.745555399193172, 25.777004609121402, 26.248772344997036, 26.17638370933796, 26.350523046747306, 26.361796441850334, 26.50507906558273, 26.831601331128528, 26.9098338597813, 26.97636846573017, 27.175189067537154, 27.056395518488614, 26.593000863741274, 26.51003480162443, 26.47408598532281, 26.46118608852198, 27.162211679395202, 27.43278298839883, 27.0632183908046, 27.966976264189885, 27.98417612659098, 28.060572753427486, 28.424588342888292, 28.64514777488093, 28.36767307420121, 28.41302538197502, 27.889159269873616, 27.761578044596913, 27.745198028216894, 27.979303390757455, 27.883324406531152, 27.75790804126529, 27.667288374071532, 27.59605522682446, 27.035929551072908, 26.88551087589054, 26.816757435637484, 26.819295019185073, 26.399369611569483, 25.573494507920735, 25.541439237091414, 25.462362933030263, 25.666187297294655, 25.527022674028686, 25.44956195453708, 25.40767115260279, 25.33653944950398, 25.356568806738405, 25.006851578961864, 24.586485140826415, 24.531286551268057, 24.240011905041627, 23.981163763772457, 23.848072173079522, 23.778329322311194, 23.66314219450053, 23.697343625103194, 23.64382171424935, 23.388969056559333, 23.32329317269076, 23.274214038912177, 23.04383003608477, 23.006748947971847, 22.962408668379965, 22.61643593496596, 22.286037559664283, 22.330516973374117, 22.112653129807867, 22.07652957781625, 21.915813811373294, 21.728606939916872, 21.605542068336987, 21.259999464249503, 21.129504381374336, 21.017605563679375, 20.89598342774386, 20.85388041438221, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.360708626971523, 19.30434700184981, 19.78379620768573, 20.315459313364002, 20.75187036597774, 21.000618715278087, 21.41668676151435, 21.79169023223038, 22.461889614145253, 22.804363392954436, 22.865196078431374, 22.800729636436014, 22.883470193693977, 23.012060654957693, 24.314715288187124, 24.663680167762305, 24.695773002799204, 25.258164568509393, 26.040583554376656, 26.47570092221384, 24.861604546070744, 25.348487606433057, 25.745555399193172, 25.777004609121402, 26.248772344997036, 26.17638370933796, 26.350523046747306, 26.361796441850334, 26.50507906558273, 26.831601331128528, 26.9098338597813, 26.97636846573017, 27.175189067537154, 27.056395518488614, 26.593000863741274, 26.51003480162443, 26.47408598532281, 26.46118608852198, 27.162211679395202, 27.43278298839883, 27.0632183908046, 27.966976264189885, 27.98417612659098, 28.060572753427486, 28.424588342888292, 28.64514777488093, 28.36767307420121, 28.41302538197502, 27.889159269873616, 27.761578044596913, 27.745198028216894, 27.979303390757455, 27.883324406531152, 27.75790804126529, 27.667288374071532, 27.59605522682446, 27.035929551072908, 26.88551087589054, 26.816757435637484, 26.819295019185073, 26.399369611569483, 25.573494507920735, 25.541439237091414, 25.462362933030263, 25.666187297294655, 25.527022674028686, 25.44956195453708, 25.40767115260279, 25.33653944950398, 25.356568806738405, 25.006851578961864, 24.586485140826415, 24.531286551268057, 24.240011905041627, 23.981163763772457, 23.848072173079522, 23.778329322311194, 23.66314219450053, 23.697343625103194, 23.64382171424935, 23.388969056559333, 23.32329317269076, 23.274214038912177, 23.04383003608477, 23.006748947971847, 22.962408668379965, 22.61643593496596, 22.286037559664283, 22.330516973374117, 22.112653129807867, 22.07652957781625, 21.915813811373294, 21.728606939916872, 21.605542068336987, 21.259999464249503, 21.129504381374336, 21.017605563679375, 20.89598342774386, 20.85388041438221, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 23.31961444526291, "exact_area100": 24.711429335259158}, "test": {"score": [18.129624718318375, 18.33830057333354, 18.46223123589801, 18.67742387529821, 18.7654659370892, 19.614076946587485, 20.045582004954365, 20.29771205473379, 20.550929031685428, 20.874559489544772, 21.008736816384666, 21.214637819988994, 21.529036036605852, 21.502653230849074, 21.606270063166615, 22.406545747226332, 23.37042883615884, 23.61891643173981, 24.11531947011147, 24.182057340790912, 24.525325995751196, 24.798258794994506, 24.992885749447325, 24.9746019386308, 25.161373856099246, 25.404990963814495, 25.442031947735266, 25.19336176040916, 25.413096827323038, 25.89306262718283, 26.363655897854837, 26.294101165367884, 26.459680648619837, 26.57928362774499, 26.174531212078143, 25.785568252498848, 25.963550746945508, 26.04358731552759, 26.141632802790728, 26.312434259503398, 26.232978886531356, 26.220447557537142, 26.192245476812182, 26.12899181436153, 25.6416187854058, 25.835823508655682, 26.45767198021416, 26.46053443017892, 25.83213446518704, 25.65121496269409, 25.466662030388537, 25.313606877000655, 25.25025914304841, 24.991447158026304, 25.183839169530593, 25.152474316312883, 25.105630396103695, 25.17138802639626, 25.074464531411948, 24.875668884355278, 24.655994542733257, 24.418253413854867, 24.692556777734236, 24.617326519079548, 24.398781054595283, 24.29095963785663, 24.122491493703986, 24.416640450806113, 24.31195120709875, 24.19177583333739, 24.041239229876513, 23.910998469685254, 23.931107262297445, 23.864622840404788, 23.808416755340005, 23.772481719604567, 23.73653910723352, 23.716007939894972, 23.633409163413074, 23.98249610035233, 23.940654890083987, 23.810453661428458, 23.792691908553966, 23.655500899925325, 23.63880549893024, 23.514559020322352, 23.59237943898776, 23.50203139969027, 23.497713273877334, 23.374552818565746, 23.695535731679858, 23.615240214135476, 23.488311338348286, 23.461083671175075, 23.378363751541677, 23.190550074337022, 23.109844548243874, 22.994799468190866, 22.781100047583966, 22.671329078997488, 22.361115727328066], "calls": [0.0, 0.6657789613848202, 1.5312916111850865, 2.2303595206391478, 2.563249001331558, 4.360852197070573, 4.993342210386151, 5.9920106524633825, 7.490013315579228, 9.320905459387484, 10.053262316910786, 11.018641810918774, 12.583222370173102, 13.182423435419441, 14.047936085219707, 16.4780292942743, 18.009320905459386, 18.77496671105193, 19.806924101198405, 20.439414114513983, 21.304926764314246, 22.603195739014648, 23.26897470039947, 23.501997336884152, 24.833555259653796, 25.865512649800266, 26.697736351531294, 27.62982689747004, 28.22902796271638, 29.494007989347537, 30.892143808255657, 31.724367509986685, 32.55659121171771, 33.38881491344873, 34.25432756324901, 35.45272969374168, 37.25033288948069, 38.01597869507324, 38.81491344873502, 39.61384820239681, 41.07856191744341, 41.344873501997334, 41.87749667110519, 42.47669773635153, 43.37549933422104, 44.607190412782955, 45.572569906790946, 46.737683089214386, 47.70306258322237, 48.26897470039947, 49.60053262316911, 50.0, 51.1318242343542, 52.330226364846865, 54.22769640479361, 55.22636484687084, 55.89214380825565, 56.724367509986685, 57.72303595206392, 59.08788282290279, 60.086551264980024, 61.35153129161118, 62.68308921438083, 63.28229027962716, 64.58055925432757, 65.41278295605859, 66.61118508655126, 67.34354194407457, 68.6750998668442, 70.43941411451398, 71.73768308921437, 72.43675099866844, 73.16910785619174, 73.66844207723035, 74.40079893475367, 75.69906790945407, 76.63115845539281, 77.16378162450066, 77.79627163781625, 78.76165113182424, 79.22769640479362, 80.79227696404794, 81.49134487350199, 82.42343541944075, 83.02263648468708, 83.88814913448735, 85.61917443408788, 87.0838881491345, 87.54993342210386, 88.38215712383489, 89.01464713715048, 89.64713715046605, 90.87882822902796, 91.54460719041279, 92.11051930758988, 94.0745672436751, 95.00665778961384, 96.17177097203728, 97.37017310252996, 98.76830892143809, 100.0], "exact_score": [18.129624718318375, 18.28126641345032, 18.62883459112026, 18.89628833535866, 19.19785089539956, 20.045582004954365, 20.29771205473379, 20.414148535162365, 20.661720998813106, 20.784355733295246, 20.988423856844907, 21.2134636297776, 21.50379929285465, 21.55059121262177, 21.587243797508698, 22.221777497033735, 22.448592874573333, 22.89873072547624, 23.334280519369443, 23.60160158506329, 24.13621994685205, 24.478761558119363, 24.58446571349797, 24.883822098334736, 25.076181771836513, 25.223713646995467, 25.427586052057816, 25.4254930859915, 25.28957971597272, 25.791743806449684, 26.139860361704443, 26.390508447391603, 26.366381353973768, 26.525602456107872, 26.68868421469534, 25.763152946254348, 25.725266318143667, 25.934727372894017, 26.04358731552759, 26.15275961605193, 26.255105016348917, 26.273145631567, 26.192245476812182, 25.670174834807995, 25.761683646289757, 25.920681345209644, 26.472188179922117, 26.449386043820244, 25.71390350416468, 25.61682448417117, 25.313606877000655, 25.25585480081798, 24.987918892022865, 24.94529590374381, 25.195555096268883, 25.170184317743665, 25.105630396103695, 25.158307956475127, 25.040869260099193, 24.899090977524846, 24.661904228114178, 24.437408664340083, 24.73346429427331, 24.679677321894154, 24.530765243208243, 24.374114459610297, 24.226268903945638, 24.03450298656188, 24.353683267878388, 24.288360279918606, 24.221913898046846, 24.144757967535654, 23.97835044672191, 23.86837413719975, 23.85067356233797, 23.784865020876804, 23.759612685165, 23.720189940564094, 23.621838316740362, 24.016745591201584, 23.94068813410294, 23.801084749312576, 23.749143799259578, 23.641112955030884, 23.510059286845028, 23.697009062855305, 23.63883412233617, 23.506199108284083, 23.38335953044062, 23.695535731679858, 23.53833265228707, 23.484446888389567, 23.382136583523682, 23.36560436835817, 23.196846150135375, 23.111122321559588, 22.963274070624205, 22.795761488855515, 22.74374792009879, 22.568387097993163, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.864398669403222, "exact_area100": 23.932864167016483}}, "old_normalized_endpoint_budget": 46}, {"seed": 2030, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.5, "validation_area100": 0.24631318513876224, "training_seconds": 3.9875611340394244, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.38545658260953736, "operator": "ge"}, {"budget": 0.02, "threshold": 0.36305348160932616, "operator": "ge"}, {"budget": 0.03, "threshold": 0.33931173613411747, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3203744372362838, "operator": "ge"}, {"budget": 0.05, "threshold": 0.29249458652837024, "operator": "ge"}, {"budget": 0.06, "threshold": 0.27255274754586645, "operator": "ge"}, {"budget": 0.07, "threshold": 0.23930467556263432, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2166040708237066, "operator": "ge"}, {"budget": 0.09, "threshold": 0.1871597039970576, "operator": "ge"}, {"budget": 0.1, "threshold": 0.1329431745878077, "operator": "ge"}, {"budget": 0.11, "threshold": 0.12139984963807293, "operator": "ge"}, {"budget": 0.12, "threshold": 0.114213409043764, "operator": "ge"}, {"budget": 0.13, "threshold": 0.10650311846861724, "operator": "ge"}, {"budget": 0.14, "threshold": 0.09856023137605413, "operator": "ge"}, {"budget": 0.15, "threshold": 0.09118741787882903, "operator": "ge"}, {"budget": 0.16, "threshold": 0.08103414398457329, "operator": "ge"}, {"budget": 0.17, "threshold": 0.07885388051672883, "operator": "ge"}, {"budget": 0.18, "threshold": 0.07289835994865429, "operator": "ge"}, {"budget": 0.19, "threshold": 0.07075646465091982, "operator": "ge"}, {"budget": 0.2, "threshold": 0.06098453542312409, "operator": "ge"}, {"budget": 0.21, "threshold": 0.05429770611057958, "operator": "ge"}, {"budget": 0.22, "threshold": 0.049110891116714475, "operator": "ge"}, {"budget": 0.23, "threshold": 0.04359226351545688, "operator": "ge"}, {"budget": 0.24, "threshold": 0.03696531775043204, "operator": "ge"}, {"budget": 0.25, "threshold": 0.03281453155839995, "operator": "ge"}, {"budget": 0.26, "threshold": 0.026378098952028774, "operator": "ge"}, {"budget": 0.27, "threshold": 0.018101570463102443, "operator": "ge"}, {"budget": 0.28, "threshold": 0.012093708307723397, "operator": "ge"}, {"budget": 0.29, "threshold": 0.007681945841528667, "operator": "ge"}, {"budget": 0.3, "threshold": 0.00604468449396076, "operator": "ge"}, {"budget": 0.31, "threshold": 0.00047359535350592304, "operator": "ge"}, {"budget": 0.32, "threshold": -0.006585465273108975, "operator": "ge"}, {"budget": 0.33, "threshold": -0.017640621692430875, "operator": "ge"}, {"budget": 0.34, "threshold": -0.02696041169179786, "operator": "ge"}, {"budget": 0.35, "threshold": -0.03412195199619444, "operator": "ge"}, {"budget": 0.36, "threshold": -0.03999892922420159, "operator": "ge"}, {"budget": 0.37, "threshold": -0.048731036404378036, "operator": "ge"}, {"budget": 0.38, "threshold": -0.05588229300202057, "operator": "ge"}, {"budget": 0.39, "threshold": -0.06988543500745316, "operator": "ge"}, {"budget": 0.4, "threshold": -0.08013397683068248, "operator": "ge"}, {"budget": 0.41, "threshold": -0.09283128942945175, "operator": "ge"}, {"budget": 0.42, "threshold": -0.1004479874742872, "operator": "ge"}, {"budget": 0.43, "threshold": -0.11462041769762572, "operator": "ge"}, {"budget": 0.44, "threshold": -0.1576724967947805, "operator": "ge"}, {"budget": 0.45, "threshold": -0.1883921299264719, "operator": "ge"}, {"budget": 0.46, "threshold": -0.27538920052695537, "operator": "ge"}, {"budget": 0.47, "threshold": -0.3231728607827706, "operator": "ge"}, {"budget": 0.48, "threshold": -0.3390351804619515, "operator": "ge"}, {"budget": 0.49, "threshold": -0.3469724267892572, "operator": "ge"}, {"budget": 0.5, "threshold": -0.35340418730975964, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3593966889790478, "operator": "ge"}, {"budget": 0.52, "threshold": -0.36369919860705713, "operator": "ge"}, {"budget": 0.53, "threshold": -0.3686998038018857, "operator": "ge"}, {"budget": 0.54, "threshold": -0.37322333404124475, "operator": "ge"}, {"budget": 0.55, "threshold": -0.381871425430159, "operator": "ge"}, {"budget": 0.56, "threshold": -0.38603563927209916, "operator": "ge"}, {"budget": 0.57, "threshold": -0.39344492788801255, "operator": "ge"}, {"budget": 0.58, "threshold": -0.39715637789024866, "operator": "ge"}, {"budget": 0.59, "threshold": -0.4022696681620214, "operator": "ge"}, {"budget": 0.6, "threshold": -0.4069435981764002, "operator": "ge"}, {"budget": 0.61, "threshold": -0.41135844928729065, "operator": "ge"}, {"budget": 0.62, "threshold": -0.4170625939127948, "operator": "ge"}, {"budget": 0.63, "threshold": -0.4193446069712377, "operator": "ge"}, {"budget": 0.64, "threshold": -0.4240607217165259, "operator": "ge"}, {"budget": 0.65, "threshold": -0.4284773367837192, "operator": "ge"}, {"budget": 0.66, "threshold": -0.4324650233247164, "operator": "ge"}, {"budget": 0.67, "threshold": -0.43664199864810993, "operator": "ge"}, {"budget": 0.68, "threshold": -0.43870201157870276, "operator": "ge"}, {"budget": 0.69, "threshold": -0.4408665771114865, "operator": "ge"}, {"budget": 0.7, "threshold": -0.44524354753328826, "operator": "ge"}, {"budget": 0.71, "threshold": -0.44747347725070535, "operator": "ge"}, {"budget": 0.72, "threshold": -0.4493312731938671, "operator": "ge"}, {"budget": 0.73, "threshold": -0.45093282499247406, "operator": "ge"}, {"budget": 0.74, "threshold": -0.4526200595910789, "operator": "ge"}, {"budget": 0.75, "threshold": -0.4536786592739308, "operator": "ge"}, {"budget": 0.76, "threshold": -0.45628626945069906, "operator": "ge"}, {"budget": 0.77, "threshold": -0.4591718796537484, "operator": "ge"}, {"budget": 0.78, "threshold": -0.46128809447564856, "operator": "ge"}, {"budget": 0.79, "threshold": -0.46403606510578327, "operator": "ge"}, {"budget": 0.8, "threshold": -0.4656382632085656, "operator": "ge"}, {"budget": 0.81, "threshold": -0.46897619380410904, "operator": "ge"}, {"budget": 0.82, "threshold": -0.47230264888839935, "operator": "ge"}, {"budget": 0.83, "threshold": -0.47718190745153655, "operator": "ge"}, {"budget": 0.84, "threshold": -0.4815296079513205, "operator": "ge"}, {"budget": 0.85, "threshold": -0.48427683164093444, "operator": "ge"}, {"budget": 0.86, "threshold": -0.48690288671786636, "operator": "ge"}, {"budget": 0.87, "threshold": -0.48861714485128643, "operator": "ge"}, {"budget": 0.88, "threshold": -0.4905669231731996, "operator": "ge"}, {"budget": 0.89, "threshold": -0.49297895438203265, "operator": "ge"}, {"budget": 0.9, "threshold": -0.49603577987466246, "operator": "ge"}, {"budget": 0.91, "threshold": -0.4999648754082013, "operator": "ge"}, {"budget": 0.92, "threshold": -0.5040166787476037, "operator": "ge"}, {"budget": 0.93, "threshold": -0.5066151852872335, "operator": "ge"}, {"budget": 0.94, "threshold": -0.5085102800496211, "operator": "ge"}, {"budget": 0.95, "threshold": -0.5134224808663067, "operator": "ge"}, {"budget": 0.96, "threshold": -0.5197553723954013, "operator": "ge"}, {"budget": 0.97, "threshold": -0.5255476561536998, "operator": "ge"}, {"budget": 0.98, "threshold": -0.5280651194965438, "operator": "ge"}, {"budget": 0.99, "threshold": -0.5365584598598132, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.450958204581394, 18.81987003168782, 19.77617595528369, 20.201956178328523, 20.586850695500196, 20.936961727949264, 21.543701387117064, 22.05458089668616, 22.5063829359495, 22.459135147091352, 22.692270845185806, 23.087364106185397, 24.43267259056733, 24.55606213969928, 24.72533044463554, 25.08740944250928, 25.310183863566678, 25.333350030054095, 25.33440475252483, 25.21064172393667, 25.11949911068846, 24.835741990696647, 25.082541467188936, 25.409272656213698, 25.51876171739016, 24.100974346416145, 24.120479559748425, 24.269931932603168, 24.44876559214127, 24.55942334739803, 24.770284925564425, 24.556457181789035, 24.66206960887812, 25.339409676914244, 25.593478932848917, 26.250170508260485, 26.116710070872255, 26.80962171910273, 27.265624320891895, 26.732024680300544, 26.80488159822123, 27.675777690063946, 28.29541776692891, 28.094960475215526, 28.575427360520088, 28.64514777488093, 28.45687513206695, 28.42355380207058, 28.358445788354587, 28.10916418662973, 27.984747674813637, 27.705944666001997, 27.848085849639798, 27.774082119509586, 27.482849494643997, 27.413453704199302, 26.87258374544469, 26.939449402146, 26.889354919889275, 26.84056948106904, 26.791682612578132, 26.768332773632565, 26.774756774756774, 27.073556695405436, 26.579455173882415, 26.724366605858414, 26.187901908075055, 26.13975531521902, 25.837377413810536, 25.67986679505697, 24.96332583603464, 25.15068068569249, 25.02384347044541, 24.789516436457692, 24.817178715092133, 24.68882791170736, 24.48881673881674, 24.425398048279405, 24.346644580436962, 24.28798858146684, 24.247284889596934, 23.911110511741647, 23.43644958217022, 23.1280745277979, 23.075532737074685, 22.485213931469165, 22.22716049382716, 22.182827789146593, 22.032548581099455, 21.76719846593722, 21.734474470323526, 21.600640886743257, 21.29786600456336, 21.19475381582211, 21.087841000585666, 20.97057592612318, 20.85528416347382, 20.812502813408255, 20.78489453270283, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 25, 38, 48, 58, 68, 81, 91, 101, 109, 115, 121, 128, 134, 139, 146, 152, 156, 162, 166, 170, 174, 178, 184, 189, 193, 197, 201, 207, 211, 217, 218, 224, 229, 233, 239, 240, 244, 251, 254, 255, 259, 263, 267, 270, 274, 275, 275, 275, 275, 275, 275, 277, 277, 277, 277, 277, 277, 277, 278, 278, 279, 280, 282, 282, 283, 283, 283, 283, 283, 283, 284, 284, 284, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 3, 5, 6, 10, 13, 17, 19, 21, 22, 27, 30, 36, 40, 46, 49, 53, 61, 65, 70, 76, 81, 87, 89, 94, 97, 104, 112, 117, 124, 128, 134, 139, 146, 154, 158, 164, 166, 171, 176, 183, 184, 188, 191, 196, 203, 209, 214, 216, 220, 226, 229, 234, 237, 242, 248, 253, 258, 263, 267, 272, 276, 282, 290, 295, 301, 303, 313, 318, 324, 331, 336, 341, 346, 353, 359, 364, 368, 374, 381, 387, 392, 398, 405, 412, 419, 429, 436, 441, 450, 456, 461, 468, 475, 479, 485, 492, 499, 505, 510, 518], "exact_score": [18.254072168965788, 18.450958204581394, 18.81987003168782, 19.77617595528369, 20.201956178328523, 20.586850695500196, 20.936961727949264, 21.543701387117064, 22.05458089668616, 22.5063829359495, 22.459135147091352, 22.692270845185806, 23.087364106185397, 24.43267259056733, 24.55606213969928, 24.72533044463554, 25.08740944250928, 25.310183863566678, 25.333350030054095, 25.33440475252483, 25.21064172393667, 25.11949911068846, 24.835741990696647, 25.082541467188936, 25.409272656213698, 25.51876171739016, 24.100974346416145, 24.120479559748425, 24.269931932603168, 24.44876559214127, 24.55942334739803, 24.770284925564425, 24.556457181789035, 24.66206960887812, 25.339409676914244, 25.593478932848917, 26.250170508260485, 26.116710070872255, 26.80962171910273, 27.265624320891895, 26.732024680300544, 26.80488159822123, 27.675777690063946, 28.29541776692891, 28.094960475215526, 28.575427360520088, 28.64514777488093, 28.45687513206695, 28.42355380207058, 28.358445788354587, 28.10916418662973, 27.984747674813637, 27.705944666001997, 27.848085849639798, 27.774082119509586, 27.482849494643997, 27.413453704199302, 26.87258374544469, 26.939449402146, 26.889354919889275, 26.84056948106904, 26.791682612578132, 26.768332773632565, 26.774756774756774, 27.073556695405436, 26.579455173882415, 26.724366605858414, 26.187901908075055, 26.13975531521902, 25.837377413810536, 25.67986679505697, 24.96332583603464, 25.15068068569249, 25.02384347044541, 24.789516436457692, 24.817178715092133, 24.68882791170736, 24.48881673881674, 24.425398048279405, 24.346644580436962, 24.28798858146684, 24.247284889596934, 23.911110511741647, 23.43644958217022, 23.1280745277979, 23.075532737074685, 22.485213931469165, 22.22716049382716, 22.182827789146593, 22.032548581099455, 21.76719846593722, 21.734474470323526, 21.600640886743257, 21.29786600456336, 21.19475381582211, 21.087841000585666, 20.97057592612318, 20.85528416347382, 20.812502813408255, 20.78489453270283, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.631318513876224, "threshold_area100": 24.63689831789867, "threshold_area25": 22.97367499086835}, "test": {"score": [18.129624718318375, 18.48642895213548, 18.37083030134489, 18.583852392992465, 19.01027096287319, 19.72338509430686, 19.87515389132074, 20.177933088682323, 20.515454401797733, 20.680969588838266, 21.190833835457923, 21.340349367919544, 21.588040266128065, 21.78177642891311, 22.2543321643197, 22.94596001146803, 23.400150258112557, 23.413398935815916, 23.58746844996972, 23.588007022399456, 23.391884321425916, 23.731071941585117, 23.859135163568318, 24.185591615683816, 24.304086296903137, 24.437243907866417, 24.5491983065167, 24.243230238480045, 24.430156468332786, 24.57787283949755, 24.59862804930969, 24.69895686293006, 25.017706561315688, 25.374265830520482, 25.59907964526872, 25.906885653869573, 26.22693374966817, 26.29535227362355, 26.365962460789593, 25.973893558253415, 25.961035484995133, 26.223321509289736, 26.207681575325008, 26.269774370941985, 25.77379044273787, 25.816230706783937, 25.902457757296464, 26.35636521334262, 26.087002278167738, 25.97201572569383, 25.455636502474572, 25.410829916076583, 25.225482622229457, 25.05278459247455, 24.858743115655678, 24.710442724055735, 24.635215606258363, 24.460717587467336, 24.868263130167893, 24.765455080218413, 24.61305509738271, 24.524486991024496, 24.45840888983419, 24.536379358725764, 24.340227159122765, 24.272453841374368, 24.00674702256325, 23.766842470959347, 23.694990268914868, 23.671978932143894, 23.213343819450184, 23.246775312504187, 23.109029824039105, 23.10682793049379, 22.94539008333729, 22.926520974913668, 22.883862002304305, 22.74967211790081, 22.574570341270388, 22.45716573099995, 22.825536794931452, 22.965517114161376, 22.753190412384892, 22.833686004941505, 22.63687633044111, 22.55198289995753, 22.467633476935855, 22.44047033458652, 22.4161439138354, 22.363736736117875, 22.5925619444141, 22.553292476348147, 22.38407765474965, 22.594917280749176, 22.513447629286908, 22.47870044657073, 22.429917150867478, 22.414799205599735, 22.447846431062544, 22.406974506249554, 22.361115727328066], "calls": [0.0, 0.8988015978695073, 1.1984021304926764, 1.9973368841544608, 2.929427430093209, 4.327563249001331, 5.292942743009322, 6.857523302263649, 7.723035952063914, 8.322237017310254, 10.452729693741677, 11.484687083888149, 12.583222370173102, 13.715046604527299, 15.046604527296935, 16.378162450066576, 18.342210386151798, 18.541944074567244, 19.241011984021306, 19.640479360852197, 21.338215712383487, 22.33688415446072, 23.33555259653795, 24.866844207723034, 26.03195739014647, 26.63115845539281, 27.62982689747004, 29.094540612516646, 30.059920106524636, 30.825565912117174, 30.99201065246338, 32.123834886817576, 33.12250332889481, 34.6537949400799, 35.519307589880164, 36.18508655126498, 36.684420772303596, 37.45006657789614, 38.21571238348868, 39.4474034620506, 40.246338215712385, 41.37816245006658, 41.67776298268975, 42.41011984021305, 43.9081225033289, 44.44074567243675, 45.40612516644474, 46.304926764314246, 47.436750998668444, 48.00266311584554, 49.03462050599201, 49.70039946737683, 50.699067909454065, 51.73102529960053, 52.69640479360852, 54.460719041278296, 55.559254327563245, 57.65645805592543, 58.72170439414115, 59.8202396804261, 61.085219707057256, 62.217043941411454, 63.81491344873502, 64.61384820239681, 65.74567243675101, 66.87749667110519, 68.1757656458056, 69.47403462050599, 70.0399467376831, 70.47270306258322, 71.97070572569906, 72.43675099866844, 73.03595206391478, 73.66844207723035, 74.53395472703063, 74.80026631158455, 75.89880159786951, 76.89747003994674, 77.62982689747004, 78.92809587217043, 79.39414114513981, 80.6924101198402, 81.92410119840213, 83.65512649800266, 85.41944074567243, 86.75099866844208, 87.84953395472704, 88.48202396804261, 89.0479360852197, 90.04660452729694, 91.21171770972038, 92.84287616511318, 94.10785619174435, 94.94007989347537, 95.30625832223701, 96.1051930758988, 97.13715046604527, 97.93608521970705, 98.33555259653795, 99.23435419440746, 100.0], "rescued": [0, 51, 65, 105, 145, 208, 242, 297, 326, 349, 418, 449, 475, 505, 540, 580, 630, 634, 656, 665, 706, 732, 754, 780, 808, 822, 845, 875, 892, 905, 909, 930, 951, 985, 1004, 1017, 1029, 1040, 1050, 1068, 1076, 1088, 1090, 1096, 1119, 1127, 1143, 1150, 1150, 1151, 1154, 1154, 1155, 1155, 1155, 1159, 1161, 1164, 1165, 1166, 1166, 1168, 1170, 1172, 1173, 1174, 1176, 1176, 1176, 1176, 1176, 1177, 1177, 1178, 1178, 1178, 1180, 1180, 1181, 1181, 1182, 1184, 1184, 1185, 1186, 1187, 1187, 1187, 1187, 1188, 1192, 1194, 1194, 1195, 1195, 1195, 1195, 1196, 1197, 1198, 1198], "harmed": [0, 8, 11, 24, 37, 51, 63, 82, 94, 100, 127, 148, 168, 198, 222, 250, 291, 295, 311, 321, 356, 377, 395, 423, 448, 459, 484, 519, 534, 542, 546, 566, 586, 615, 637, 645, 655, 665, 676, 696, 706, 719, 724, 734, 769, 781, 807, 821, 840, 848, 866, 878, 896, 914, 939, 975, 991, 1036, 1054, 1077, 1107, 1131, 1164, 1182, 1211, 1238, 1272, 1303, 1317, 1327, 1369, 1375, 1390, 1405, 1430, 1438, 1468, 1495, 1514, 1547, 1562, 1595, 1626, 1671, 1719, 1750, 1773, 1788, 1801, 1824, 1852, 1899, 1932, 1957, 1968, 1994, 2027, 2049, 2060, 2088, 2109], "exact_score": [18.129624718318375, 18.32043510007531, 18.583852392992465, 19.023425185030412, 19.623575268347064, 19.870757185381738, 20.007997820240423, 20.126981947592224, 20.583781083195575, 20.743830319217864, 21.000140038180696, 21.261441603243593, 21.47028995284713, 21.69703517545612, 21.879735301159865, 22.22004638080076, 22.64860801485431, 23.44206952050089, 23.207276733858958, 23.496925320844053, 23.122264405188023, 23.2855299175478, 23.541460915947475, 23.667331393696582, 24.16374961617757, 24.23816613708872, 24.296599389171185, 24.48270754701302, 24.104989447622252, 24.238253200989725, 24.475083162371806, 24.59862804930969, 24.654976777174433, 25.010217255556455, 25.333295831697413, 25.333540056110227, 25.766103865711, 26.27899989665628, 26.3482240667275, 25.925787492123053, 25.936445078887104, 26.32521136711287, 26.21652033415437, 25.691639956560042, 25.790424234310134, 25.944457398252048, 26.460004686535676, 26.178987875217462, 25.97201572569383, 25.458894862495423, 25.40010557881582, 25.218084451260115, 25.01991289433389, 24.84471672020048, 24.776536861952465, 24.641512362763713, 24.64728296398687, 24.532131561633946, 24.912063813510578, 24.800109520750695, 24.6930277774646, 24.619000523989165, 24.497667419847627, 24.497175067417714, 24.44705854995187, 24.470235860436027, 24.318327029645484, 24.267138350343384, 23.972677924485154, 23.797574814593744, 23.699880573282286, 23.50851177757335, 23.213343819450184, 23.11197497493478, 23.036258970918492, 22.964485031576118, 22.80961351482742, 22.745383981665896, 22.55768349764083, 22.45308085670704, 23.17222905008906, 22.951293376734377, 22.748345518856087, 22.915625036049438, 22.86618740029434, 22.67416279896552, 22.6515661754838, 22.525436221237676, 22.463210959442343, 22.41735655440065, 22.367026878756754, 22.494375679817463, 22.62423246064408, 22.547192701458886, 22.407305156305256, 22.592987337682494, 22.485529992048633, 22.43717780847561, 22.413031330169293, 22.368315272454957, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.469265121022126, "threshold_area100": 23.507103842381323, "threshold_area25": 21.707099515150226}, "selected_budget": 46, "selected": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726, "rescued": 274, "harmed": 209}, "test": {"score": 25.902457757296464, "calls": 45.40612516644474, "rescued": 1143, "harmed": 807}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.3336018919944763, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3074936270713806, "operator": "ge"}, {"budget": 0.03, "threshold": 0.289029985666275, "operator": "ge"}, {"budget": 0.04, "threshold": 0.2636280655860901, "operator": "ge"}, {"budget": 0.05, "threshold": 0.24964028596878052, "operator": "ge"}, {"budget": 0.06, "threshold": 0.23097990453243256, "operator": "ge"}, {"budget": 0.07, "threshold": 0.20602774620056152, "operator": "ge"}, {"budget": 0.08, "threshold": 0.18017688393592834, "operator": "ge"}, {"budget": 0.09, "threshold": 0.15456841886043549, "operator": "ge"}, {"budget": 0.1, "threshold": 0.0944749265909195, "operator": "ge"}, {"budget": 0.11, "threshold": 0.0853106826543808, "operator": "ge"}, {"budget": 0.12, "threshold": 0.07851894199848175, "operator": "ge"}, {"budget": 0.13, "threshold": 0.07284504175186157, "operator": "ge"}, {"budget": 0.14, "threshold": 0.06910267472267151, "operator": "ge"}, {"budget": 0.15, "threshold": 0.05905112624168396, "operator": "ge"}, {"budget": 0.16, "threshold": 0.055705949664115906, "operator": "ge"}, {"budget": 0.17, "threshold": 0.05308067798614502, "operator": "ge"}, {"budget": 0.18, "threshold": 0.050673291087150574, "operator": "ge"}, {"budget": 0.19, "threshold": 0.0467500314116478, "operator": "ge"}, {"budget": 0.2, "threshold": 0.041115567088127136, "operator": "ge"}, {"budget": 0.21, "threshold": 0.037346452474594116, "operator": "ge"}, {"budget": 0.22, "threshold": 0.033383503556251526, "operator": "ge"}, {"budget": 0.23, "threshold": 0.030645348131656647, "operator": "ge"}, {"budget": 0.24, "threshold": 0.026611536741256714, "operator": "ge"}, {"budget": 0.25, "threshold": 0.021564200520515442, "operator": "ge"}, {"budget": 0.26, "threshold": 0.018027551472187042, "operator": "ge"}, {"budget": 0.27, "threshold": 0.012212231755256653, "operator": "ge"}, {"budget": 0.28, "threshold": 0.008188173174858093, "operator": "ge"}, {"budget": 0.29, "threshold": 0.005287840962409973, "operator": "ge"}, {"budget": 0.3, "threshold": 0.003848925232887268, "operator": "ge"}, {"budget": 0.31, "threshold": 0.0002939179539680481, "operator": "ge"}, {"budget": 0.32, "threshold": -0.004845224320888519, "operator": "ge"}, {"budget": 0.33, "threshold": -0.011593729257583618, "operator": "ge"}, {"budget": 0.34, "threshold": -0.018205970525741577, "operator": "ge"}, {"budget": 0.35, "threshold": -0.023832201957702637, "operator": "ge"}, {"budget": 0.36, "threshold": -0.026462018489837646, "operator": "ge"}, {"budget": 0.37, "threshold": -0.03198763728141785, "operator": "ge"}, {"budget": 0.38, "threshold": -0.038036197423934937, "operator": "ge"}, {"budget": 0.39, "threshold": -0.04771490395069122, "operator": "ge"}, {"budget": 0.4, "threshold": -0.05385317653417587, "operator": "ge"}, {"budget": 0.41, "threshold": -0.06356310099363327, "operator": "ge"}, {"budget": 0.42, "threshold": -0.06821301579475403, "operator": "ge"}, {"budget": 0.43, "threshold": -0.07381212711334229, "operator": "ge"}, {"budget": 0.44, "threshold": -0.1048860102891922, "operator": "ge"}, {"budget": 0.45, "threshold": -0.11113355308771133, "operator": "ge"}, {"budget": 0.46, "threshold": -0.12039311975240707, "operator": "ge"}, {"budget": 0.47, "threshold": -0.1248975470662117, "operator": "ge"}, {"budget": 0.48, "threshold": -0.13040094077587128, "operator": "ge"}, {"budget": 0.49, "threshold": -0.1335795521736145, "operator": "ge"}, {"budget": 0.5, "threshold": -0.13840903341770172, "operator": "ge"}, {"budget": 0.51, "threshold": -0.1446637064218521, "operator": "ge"}, {"budget": 0.52, "threshold": -0.14745087921619415, "operator": "ge"}, {"budget": 0.53, "threshold": -0.14930489659309387, "operator": "ge"}, {"budget": 0.54, "threshold": -0.1577746421098709, "operator": "ge"}, {"budget": 0.55, "threshold": -0.1618909239768982, "operator": "ge"}, {"budget": 0.56, "threshold": -0.16478809714317322, "operator": "ge"}, {"budget": 0.57, "threshold": -0.1684103012084961, "operator": "ge"}, {"budget": 0.58, "threshold": -0.17144067585468292, "operator": "ge"}, {"budget": 0.59, "threshold": -0.17697332799434662, "operator": "ge"}, {"budget": 0.6, "threshold": -0.1803922802209854, "operator": "ge"}, {"budget": 0.61, "threshold": -0.18285757303237915, "operator": "ge"}, {"budget": 0.62, "threshold": -0.18484966456890106, "operator": "ge"}, {"budget": 0.63, "threshold": -0.18898753821849823, "operator": "ge"}, {"budget": 0.64, "threshold": -0.19308537244796753, "operator": "ge"}, {"budget": 0.65, "threshold": -0.1956465244293213, "operator": "ge"}, {"budget": 0.66, "threshold": -0.19966676831245422, "operator": "ge"}, {"budget": 0.67, "threshold": -0.2009340226650238, "operator": "ge"}, {"budget": 0.68, "threshold": -0.20334027707576752, "operator": "ge"}, {"budget": 0.69, "threshold": -0.20603545010089874, "operator": "ge"}, {"budget": 0.7, "threshold": -0.20804044604301453, "operator": "ge"}, {"budget": 0.71, "threshold": -0.20887801051139832, "operator": "ge"}, {"budget": 0.72, "threshold": -0.209943026304245, "operator": "ge"}, {"budget": 0.73, "threshold": -0.21180807054042816, "operator": "ge"}, {"budget": 0.74, "threshold": -0.2133612185716629, "operator": "ge"}, {"budget": 0.75, "threshold": -0.21538834273815155, "operator": "ge"}, {"budget": 0.76, "threshold": -0.21796926856040955, "operator": "ge"}, {"budget": 0.77, "threshold": -0.2208152413368225, "operator": "ge"}, {"budget": 0.78, "threshold": -0.22245380282402039, "operator": "ge"}, {"budget": 0.79, "threshold": -0.22448943555355072, "operator": "ge"}, {"budget": 0.8, "threshold": -0.22706033289432526, "operator": "ge"}, {"budget": 0.81, "threshold": -0.2298104614019394, "operator": "ge"}, {"budget": 0.82, "threshold": -0.23560573160648346, "operator": "ge"}, {"budget": 0.83, "threshold": -0.23770661652088165, "operator": "ge"}, {"budget": 0.84, "threshold": -0.24048832058906555, "operator": "ge"}, {"budget": 0.85, "threshold": -0.24188849329948425, "operator": "ge"}, {"budget": 0.86, "threshold": -0.2443358600139618, "operator": "ge"}, {"budget": 0.87, "threshold": -0.24554109573364258, "operator": "ge"}, {"budget": 0.88, "threshold": -0.24830202758312225, "operator": "ge"}, {"budget": 0.89, "threshold": -0.2507593631744385, "operator": "ge"}, {"budget": 0.9, "threshold": -0.25315549969673157, "operator": "ge"}, {"budget": 0.91, "threshold": -0.2558209300041199, "operator": "ge"}, {"budget": 0.92, "threshold": -0.2582601010799408, "operator": "ge"}, {"budget": 0.93, "threshold": -0.26247522234916687, "operator": "ge"}, {"budget": 0.94, "threshold": -0.2676481306552887, "operator": "ge"}, {"budget": 0.95, "threshold": -0.273738294839859, "operator": "ge"}, {"budget": 0.96, "threshold": -0.2763708233833313, "operator": "ge"}, {"budget": 0.97, "threshold": -0.2789314389228821, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2849220633506775, "operator": "ge"}, {"budget": 0.99, "threshold": -0.2936684787273407, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.53992216854536, 18.745206273868416, 19.10718660320571, 20.167639088417747, 20.556114122515588, 21.00112945540468, 21.421685166247965, 21.983669352090406, 22.339935974389757, 22.52922077922078, 22.603618362916304, 23.003117527410776, 23.253638918623842, 24.652680497187202, 24.978554961960207, 25.137873754152825, 25.211758666716282, 25.322116922471245, 25.273908684769236, 25.21064172393667, 25.10036030081849, 25.00383695930095, 25.075616631188236, 25.238496481380736, 25.629607048018944, 24.100974346416145, 24.120479559748425, 24.319036622726163, 24.39652870836735, 24.55942334739803, 24.770284925564425, 24.556457181789035, 24.66206960887812, 25.272600561883397, 25.79791564891711, 26.250170508260485, 26.382250380307216, 27.29951176464946, 27.265624320891895, 26.732024680300544, 27.215179349117115, 27.53900751051866, 28.31405535895202, 28.366089703299004, 28.200777500576113, 28.294930875576036, 28.26041882645656, 28.117571270437512, 28.051433704299946, 28.319460695670674, 28.424105629679847, 28.34902625160366, 28.291855486942058, 27.535107169253507, 27.481788626366942, 27.426515539780226, 27.146996451900936, 27.09653354775306, 27.071802553556417, 26.78593220641887, 26.70034728943516, 26.569012241846558, 26.60641326285498, 25.92649606823937, 25.687333399281755, 25.622296325111193, 25.640416676947876, 25.391735228122663, 24.987171511218683, 24.547421523366538, 24.512097853891422, 24.544295175572078, 24.51120424031646, 24.2219537969165, 24.162165209362094, 24.181519358322863, 24.226014295781738, 24.086652030584716, 24.030828132871665, 23.913870455450972, 23.300367078830828, 22.941659704817603, 22.740240184779612, 22.70561486357103, 22.030962487997886, 21.972584214408656, 21.93758976697181, 21.89349060702369, 21.859767226630485, 21.884615236145834, 21.92476088999525, 21.88364246529866, 21.772069522692224, 21.57510530449061, 21.53636812843443, 21.425362369540014, 21.304324551986948, 21.04541545283328, 21.014282547060116, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 23, 35, 48, 60, 69, 79, 91, 101, 109, 114, 121, 126, 133, 139, 145, 150, 157, 162, 167, 171, 175, 177, 184, 190, 193, 197, 202, 206, 211, 217, 218, 224, 229, 235, 239, 241, 245, 251, 254, 258, 260, 264, 265, 268, 271, 271, 271, 271, 271, 273, 273, 273, 273, 273, 274, 274, 274, 274, 274, 274, 274, 275, 276, 276, 276, 277, 277, 277, 277, 277, 278, 278, 278, 278, 279, 279, 279, 279, 279, 279, 279, 280, 280, 281, 281, 281, 281, 283, 284, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 2, 5, 7, 11, 15, 17, 18, 21, 23, 26, 30, 38, 40, 44, 50, 56, 61, 66, 71, 77, 83, 87, 88, 94, 97, 104, 112, 118, 123, 128, 134, 139, 146, 154, 159, 164, 166, 170, 176, 183, 185, 186, 191, 193, 199, 205, 207, 211, 215, 219, 224, 229, 233, 239, 243, 248, 254, 258, 260, 267, 275, 280, 285, 290, 300, 306, 313, 317, 323, 328, 331, 338, 342, 348, 353, 359, 366, 373, 380, 386, 393, 399, 406, 411, 420, 428, 433, 440, 447, 454, 460, 465, 470, 477, 483, 489, 496, 503, 508, 518], "exact_score": [18.254072168965788, 18.53992216854536, 18.745206273868416, 19.10718660320571, 20.167639088417747, 20.556114122515588, 21.00112945540468, 21.421685166247965, 21.983669352090406, 22.339935974389757, 22.52922077922078, 22.603618362916304, 23.003117527410776, 23.253638918623842, 24.652680497187202, 24.978554961960207, 25.137873754152825, 25.211758666716282, 25.322116922471245, 25.273908684769236, 25.21064172393667, 25.10036030081849, 25.00383695930095, 25.075616631188236, 25.238496481380736, 25.629607048018944, 24.100974346416145, 24.120479559748425, 24.319036622726163, 24.39652870836735, 24.55942334739803, 24.770284925564425, 24.556457181789035, 24.66206960887812, 25.272600561883397, 25.79791564891711, 26.250170508260485, 26.382250380307216, 27.29951176464946, 27.265624320891895, 26.732024680300544, 27.215179349117115, 27.53900751051866, 28.31405535895202, 28.366089703299004, 28.200777500576113, 28.294930875576036, 28.26041882645656, 28.117571270437512, 28.051433704299946, 28.319460695670674, 28.424105629679847, 28.34902625160366, 28.291855486942058, 27.535107169253507, 27.481788626366942, 27.426515539780226, 27.146996451900936, 27.09653354775306, 27.071802553556417, 26.78593220641887, 26.70034728943516, 26.569012241846558, 26.60641326285498, 25.92649606823937, 25.687333399281755, 25.622296325111193, 25.640416676947876, 25.391735228122663, 24.987171511218683, 24.547421523366538, 24.512097853891422, 24.544295175572078, 24.51120424031646, 24.2219537969165, 24.162165209362094, 24.181519358322863, 24.226014295781738, 24.086652030584716, 24.030828132871665, 23.913870455450972, 23.300367078830828, 22.941659704817603, 22.740240184779612, 22.70561486357103, 22.030962487997886, 21.972584214408656, 21.93758976697181, 21.89349060702369, 21.859767226630485, 21.884615236145834, 21.92476088999525, 21.88364246529866, 21.772069522692224, 21.57510530449061, 21.53636812843443, 21.425362369540014, 21.304324551986948, 21.04541545283328, 21.014282547060116, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.496166998595672, "threshold_area100": 24.501389880443174, "threshold_area25": 23.043274239225124}, "test": {"score": [18.129624718318375, 18.2417041809344, 18.38078824231247, 18.819154267494635, 19.38331630220814, 19.73171364006423, 19.979337715263142, 20.086325297863407, 20.506215023480397, 20.65828346710524, 21.273187226322328, 21.397588083806625, 21.542435585975632, 21.973610345359674, 22.33556864874642, 23.561559744555673, 23.387808621018007, 23.03417455500673, 23.15945172101447, 23.084959969110088, 23.465340568243885, 23.736585151190265, 23.894805912562315, 24.29213570092776, 24.314751592733057, 24.474916009150956, 24.557264526184035, 24.23939251784508, 24.490288565992117, 24.567262954859604, 24.596130871258698, 24.69895686293006, 25.046728044121338, 25.339734320472502, 25.539327650241322, 25.972018151423924, 26.066287991004366, 26.272276718264443, 26.316695394326867, 25.954505526501176, 26.16466322855174, 26.21427279549069, 26.281008170519282, 26.303220657781672, 25.803749556901295, 25.855862467666924, 25.7742399495039, 25.7132305745014, 25.662513882983635, 25.630645535909473, 25.527998178528417, 25.305307441491287, 25.246039662280857, 25.222150715399987, 25.062049595503446, 25.527872266746147, 25.3429230510299, 25.20532539961139, 25.182061882314503, 25.175474160759876, 25.56017127591869, 25.5253389204237, 25.450353158452078, 25.33351228152043, 25.038156353724084, 24.864949245300853, 24.206065468847477, 24.189199092427906, 23.871863038371348, 23.60158862530516, 23.550043234041162, 23.50870901995156, 23.53605262421798, 23.428190347395986, 23.38660942483201, 23.17997165800661, 23.579734567635814, 23.431385279887092, 23.701028337468998, 23.687429652741706, 23.613748276182797, 23.596129073647713, 23.316452470023407, 23.173452552194988, 23.099642167207584, 23.059578297671575, 22.855336611310626, 23.135181506772952, 23.007259807676647, 22.99761711248609, 22.801244464976374, 22.819720948895082, 22.746879787460163, 22.990664397238188, 22.825390638394815, 22.69472935665916, 22.629103658094266, 22.548056656991594, 22.49670246998972, 22.5163583134802, 22.361115727328066], "calls": [0.0, 0.8322237017310253, 1.3648468708388815, 2.130492676431425, 3.5952063914780292, 4.360852197070573, 5.426098535286285, 6.55792276964048, 7.723035952063914, 8.388814913448735, 10.652463382157123, 11.717709720372836, 13.11584553928096, 14.347536617842877, 15.246338215712383, 17.7762982689747, 18.375499334221036, 18.874833555259656, 19.37416777629827, 20.00665778961385, 21.604527296937416, 22.270306258322236, 23.40213049267643, 24.833555259653796, 25.699067909454058, 26.93075898801598, 27.62982689747004, 29.0612516644474, 30.09320905459387, 30.758988015978694, 31.05858854860186, 32.123834886817576, 33.222370173102526, 34.52063914780293, 35.41944074567244, 36.38482023968043, 36.55126498002663, 37.383488681757655, 38.315579227696404, 39.38082556591212, 40.1797603195739, 41.27829560585886, 41.6444740346205, 42.0439414114514, 44.27430093209054, 45.10652463382157, 46.03861517976032, 46.77097203728362, 47.636484687083886, 48.10252996005326, 49.334221038615176, 50.76564580559254, 51.398135818908116, 51.89747003994674, 54.36085219707058, 55.559254327563245, 56.69107856191744, 57.75632490013316, 58.42210386151797, 60.41944074567244, 61.61784287616512, 62.54993342210386, 63.08255659121171, 64.48069241011984, 65.9121171770972, 66.6444740346205, 68.14247669773636, 68.50865512649801, 69.54061251664447, 70.73901464713715, 71.47137150466045, 71.80426098535287, 72.10386151797603, 72.86950732356857, 73.46870838881492, 74.36750998668442, 75.56591211717709, 76.59786950732357, 77.56324900133156, 78.59520639147803, 79.6271637816245, 81.05858854860186, 83.45539280958721, 84.4207723035952, 85.75233022636485, 86.38482023968042, 87.51664447403462, 88.1491344873502, 89.34753661784288, 90.34620505992011, 91.17842876165113, 92.0439414114514, 93.00932090545939, 94.27430093209055, 95.63914780292943, 96.40479360852197, 96.77097203728363, 97.47003994673769, 98.36884154460719, 98.96804260985353, 100.0], "rescued": [0, 47, 71, 110, 175, 209, 249, 286, 325, 350, 427, 454, 490, 523, 548, 616, 632, 645, 659, 669, 713, 729, 755, 779, 800, 829, 844, 874, 893, 904, 911, 930, 954, 981, 1001, 1019, 1025, 1041, 1050, 1066, 1076, 1089, 1091, 1094, 1119, 1124, 1132, 1134, 1134, 1136, 1139, 1141, 1142, 1142, 1143, 1146, 1146, 1147, 1148, 1152, 1155, 1157, 1157, 1159, 1161, 1161, 1162, 1163, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1169, 1172, 1172, 1174, 1176, 1178, 1179, 1182, 1182, 1184, 1184, 1184, 1185, 1186, 1189, 1190, 1192, 1193, 1195, 1195, 1195, 1195, 1195, 1197, 1198, 1198], "harmed": [0, 8, 13, 24, 43, 50, 60, 80, 93, 100, 130, 148, 182, 211, 228, 284, 295, 305, 313, 330, 364, 375, 396, 421, 441, 465, 483, 517, 532, 541, 548, 566, 588, 613, 634, 648, 652, 664, 679, 695, 704, 722, 723, 727, 770, 778, 796, 808, 821, 833, 862, 894, 908, 915, 957, 978, 1005, 1027, 1041, 1075, 1105, 1125, 1137, 1165, 1204, 1222, 1254, 1262, 1294, 1326, 1335, 1341, 1350, 1374, 1390, 1419, 1452, 1482, 1504, 1533, 1560, 1595, 1655, 1677, 1711, 1723, 1751, 1767, 1798, 1820, 1851, 1876, 1901, 1935, 1980, 2004, 2014, 2034, 2061, 2080, 2109], "exact_score": [18.129624718318375, 18.301936903738063, 18.569186157296826, 19.025824371766305, 19.59118337115855, 19.879631281693563, 19.982551232754805, 20.13499797965635, 20.583589041567585, 20.832132744331744, 20.974765280895046, 21.287170635347792, 21.38605605164589, 21.508244885420222, 21.851441118454257, 22.216141791096085, 22.407846124526635, 23.417077906522465, 23.2557172929376, 23.05694307340169, 23.07949251848774, 23.333167034071778, 23.70545105652944, 23.83355290505721, 23.908916448335763, 24.260336253427067, 24.370886566008515, 24.474770943595132, 24.095566872379354, 24.238253200989725, 24.475083162371806, 24.619567104286745, 24.654976777174433, 25.010217255556455, 25.26196717990807, 25.292718091629453, 25.822793208114625, 26.23724936390488, 26.39584286967577, 25.9680114090001, 26.110245787813867, 26.075056948577952, 26.303220657781672, 25.617588926884405, 25.851261088294503, 25.840077072486867, 25.756700267059262, 25.701954572758012, 25.642971989572384, 25.532166866729956, 25.511098185287683, 25.3123516134034, 25.212472036156296, 25.1666954141071, 25.10790737824873, 25.03340673442821, 25.38977947856732, 25.317697676725142, 25.195365669064174, 25.17824698615838, 25.23122589816886, 25.616987179487182, 25.591713304839192, 25.459542657192756, 25.335903581712277, 25.23594342423782, 24.98609355186094, 24.805777381379286, 24.212944795774987, 24.01149455397398, 23.821941112970563, 23.599920415544464, 23.495018249760765, 23.44482791779354, 23.30514572049459, 23.6087447056697, 23.520164489830904, 23.34659682222359, 23.680381838218807, 23.66977281646678, 23.442870825341192, 23.597462193681846, 23.43202062081073, 23.35167000031305, 23.22470926822863, 23.08480112200032, 23.089073046746304, 23.019910064361866, 23.19064285532434, 23.02941053167535, 22.953227605733574, 22.806698302645405, 22.821810803694955, 22.748930480594446, 23.05217483438253, 22.906705329299704, 22.758551405228104, 22.622443415612516, 22.530520622090673, 22.5163583134802, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.68588305392979, "threshold_area100": 23.72372167237999, "threshold_area25": 21.866814308584157}, "selected_budget": 51, "selected": {"val": {"score": 28.424105629679847, "calls": 50.92724679029958, "rescued": 273, "harmed": 224}, "test": {"score": 25.305307441491287, "calls": 50.76564580559254, "rescued": 1141, "harmed": 894}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.4536597728729248, "operator": "ge"}, {"budget": 0.02, "threshold": 1.353060007095337, "operator": "ge"}, {"budget": 0.03, "threshold": 1.2632066011428833, "operator": "ge"}, {"budget": 0.04, "threshold": 1.1995081901550293, "operator": "ge"}, {"budget": 0.05, "threshold": 1.1515969038009644, "operator": "ge"}, {"budget": 0.06, "threshold": 1.1062941551208496, "operator": "ge"}, {"budget": 0.07, "threshold": 1.0245683193206787, "operator": "ge"}, {"budget": 0.08, "threshold": 0.9395146369934082, "operator": "ge"}, {"budget": 0.09, "threshold": 0.7942070960998535, "operator": "ge"}, {"budget": 0.1, "threshold": 0.47588080167770386, "operator": "ge"}, {"budget": 0.11, "threshold": 0.44281846284866333, "operator": "ge"}, {"budget": 0.12, "threshold": 0.4120001494884491, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3902233839035034, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3687281906604767, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3420814871788025, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3098480701446533, "operator": "ge"}, {"budget": 0.17, "threshold": 0.28544217348098755, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2750304937362671, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2632319927215576, "operator": "ge"}, {"budget": 0.2, "threshold": 0.24600011110305786, "operator": "ge"}, {"budget": 0.21, "threshold": 0.219828262925148, "operator": "ge"}, {"budget": 0.22, "threshold": 0.19549614191055298, "operator": "ge"}, {"budget": 0.23, "threshold": 0.1783430576324463, "operator": "ge"}, {"budget": 0.24, "threshold": 0.1605531871318817, "operator": "ge"}, {"budget": 0.25, "threshold": 0.12813466787338257, "operator": "ge"}, {"budget": 0.26, "threshold": 0.10189175605773926, "operator": "ge"}, {"budget": 0.27, "threshold": 0.07692323625087738, "operator": "ge"}, {"budget": 0.28, "threshold": 0.05479177460074425, "operator": "ge"}, {"budget": 0.29, "threshold": 0.03229641914367676, "operator": "ge"}, {"budget": 0.3, "threshold": 0.021976903080940247, "operator": "ge"}, {"budget": 0.31, "threshold": 0.0027684569358825684, "operator": "ge"}, {"budget": 0.32, "threshold": -0.039409175515174866, "operator": "ge"}, {"budget": 0.33, "threshold": -0.06492845714092255, "operator": "ge"}, {"budget": 0.34, "threshold": -0.10780173540115356, "operator": "ge"}, {"budget": 0.35, "threshold": -0.13622906804084778, "operator": "ge"}, {"budget": 0.36, "threshold": -0.18767698109149933, "operator": "ge"}, {"budget": 0.37, "threshold": -0.24048347771167755, "operator": "ge"}, {"budget": 0.38, "threshold": -0.30652058124542236, "operator": "ge"}, {"budget": 0.39, "threshold": -0.35719263553619385, "operator": "ge"}, {"budget": 0.4, "threshold": -0.48992282152175903, "operator": "ge"}, {"budget": 0.41, "threshold": -0.5453184247016907, "operator": "ge"}, {"budget": 0.42, "threshold": -0.6309343576431274, "operator": "ge"}, {"budget": 0.43, "threshold": -0.71136474609375, "operator": "ge"}, {"budget": 0.44, "threshold": -0.8708391785621643, "operator": "ge"}, {"budget": 0.45, "threshold": -1.0721553564071655, "operator": "ge"}, {"budget": 0.46, "threshold": -1.3405003547668457, "operator": "ge"}, {"budget": 0.47, "threshold": -1.7058391571044922, "operator": "ge"}, {"budget": 0.48, "threshold": -1.974297046661377, "operator": "ge"}, {"budget": 0.49, "threshold": -2.1627044677734375, "operator": "ge"}, {"budget": 0.5, "threshold": -2.3121681213378906, "operator": "ge"}, {"budget": 0.51, "threshold": -2.5590648651123047, "operator": "ge"}, {"budget": 0.52, "threshold": -3.9427289962768555, "operator": "ge"}, {"budget": 0.53, "threshold": -4.030449867248535, "operator": "ge"}, {"budget": 0.54, "threshold": -4.060711860656738, "operator": "ge"}, {"budget": 0.55, "threshold": -4.071979522705078, "operator": "ge"}, {"budget": 0.56, "threshold": -4.104611396789551, "operator": "ge"}, {"budget": 0.57, "threshold": -4.130474090576172, "operator": "ge"}, {"budget": 0.58, "threshold": -4.153443813323975, "operator": "ge"}, {"budget": 0.59, "threshold": -4.175743103027344, "operator": "ge"}, {"budget": 0.6, "threshold": -4.202032089233398, "operator": "ge"}, {"budget": 0.61, "threshold": -4.231728553771973, "operator": "ge"}, {"budget": 0.62, "threshold": -4.253023147583008, "operator": "ge"}, {"budget": 0.63, "threshold": -4.282868385314941, "operator": "ge"}, {"budget": 0.64, "threshold": -4.311151504516602, "operator": "ge"}, {"budget": 0.65, "threshold": -4.322361946105957, "operator": "ge"}, {"budget": 0.66, "threshold": -4.374330043792725, "operator": "ge"}, {"budget": 0.67, "threshold": -4.410955429077148, "operator": "ge"}, {"budget": 0.68, "threshold": -4.438023090362549, "operator": "ge"}, {"budget": 0.69, "threshold": -4.489955902099609, "operator": "ge"}, {"budget": 0.7, "threshold": -4.5384931564331055, "operator": "ge"}, {"budget": 0.71, "threshold": -4.589663505554199, "operator": "ge"}, {"budget": 0.72, "threshold": -4.619632244110107, "operator": "ge"}, {"budget": 0.73, "threshold": -4.670443534851074, "operator": "ge"}, {"budget": 0.74, "threshold": -4.720406532287598, "operator": "ge"}, {"budget": 0.75, "threshold": -4.742088317871094, "operator": "ge"}, {"budget": 0.76, "threshold": -4.784712791442871, "operator": "ge"}, {"budget": 0.77, "threshold": -4.807441711425781, "operator": "ge"}, {"budget": 0.78, "threshold": -4.837223052978516, "operator": "ge"}, {"budget": 0.79, "threshold": -4.866865634918213, "operator": "ge"}, {"budget": 0.8, "threshold": -4.908908843994141, "operator": "ge"}, {"budget": 0.81, "threshold": -4.981313705444336, "operator": "ge"}, {"budget": 0.82, "threshold": -11.407302856445312, "operator": "ge"}, {"budget": 0.83, "threshold": -11.443475723266602, "operator": "ge"}, {"budget": 0.84, "threshold": -11.464986801147461, "operator": "ge"}, {"budget": 0.85, "threshold": -11.50373649597168, "operator": "ge"}, {"budget": 0.86, "threshold": -11.555044174194336, "operator": "ge"}, {"budget": 0.87, "threshold": -11.633504867553711, "operator": "ge"}, {"budget": 0.88, "threshold": -11.723186492919922, "operator": "ge"}, {"budget": 0.89, "threshold": -11.810492515563965, "operator": "ge"}, {"budget": 0.9, "threshold": -11.853092193603516, "operator": "ge"}, {"budget": 0.91, "threshold": -11.893962860107422, "operator": "ge"}, {"budget": 0.92, "threshold": -11.932427406311035, "operator": "ge"}, {"budget": 0.93, "threshold": -11.986421585083008, "operator": "ge"}, {"budget": 0.94, "threshold": -12.065515518188477, "operator": "ge"}, {"budget": 0.95, "threshold": -12.146320343017578, "operator": "ge"}, {"budget": 0.96, "threshold": -12.474004745483398, "operator": "ge"}, {"budget": 0.97, "threshold": -12.591371536254883, "operator": "ge"}, {"budget": 0.98, "threshold": -12.739151000976562, "operator": "ge"}, {"budget": 0.99, "threshold": -12.871965408325195, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.580620555596568, 18.7605853995819, 19.158709875910855, 19.365391685616633, 20.342300677436164, 20.914052101317214, 21.422445501491787, 21.642282252021808, 22.25171186668814, 22.459135147091352, 22.692270845185806, 23.044740243681066, 23.0947163959695, 24.37978371438448, 24.428601993420504, 24.616459534219217, 24.931736221180664, 24.951962008541948, 24.93796574744612, 24.994724509685078, 24.883816293056313, 24.90124159875614, 24.799014294105458, 25.096456554030734, 25.403705807509287, 24.081729465979944, 24.057365023474176, 24.220203289970733, 24.414737582490332, 24.55942334739803, 24.770284925564425, 24.705572853600877, 24.66206960887812, 25.562049895792494, 26.00125938769986, 26.28139629542705, 27.73810825587753, 27.38112522686026, 27.432323367658128, 27.887449065355923, 28.35341473112371, 28.23403343765335, 28.265324973809392, 28.387492459987374, 28.586572840518322, 28.64514777488093, 28.473417729254713, 28.096569685772486, 28.713951489686785, 28.577390611711305, 28.086742562597294, 27.740358372699518, 27.564776712334044, 26.99817795323413, 26.957527546730063, 26.85476295968099, 26.77319456553327, 26.66925298504246, 26.5886944458373, 26.746195632598447, 26.612886835155997, 26.481541860107978, 26.68873403019745, 26.546757714665524, 26.471975716361513, 25.987905385246407, 25.5224857538888, 25.45431300878855, 25.296544385191172, 25.10539714065037, 25.04259520828322, 24.88069301794792, 24.725879916217984, 24.196327910526875, 23.98182296844887, 23.631625381940637, 23.430497570044682, 23.358549435952337, 23.043805132317562, 22.657818293460547, 22.308619426490953, 22.25966172062826, 22.23569009312419, 22.211798798514664, 22.115020124526747, 22.08313018397654, 22.044008245872785, 22.013343291047054, 21.83007891955528, 21.726753976956196, 21.632813462998165, 21.548062173497957, 21.295973674022456, 21.261628302695502, 21.161937505009195, 21.134600531896165, 20.90935686418692, 20.889161741314176, 20.87347732288362, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 20, 32, 43, 54, 66, 77, 88, 100, 109, 115, 119, 125, 132, 137, 142, 149, 154, 159, 164, 168, 172, 176, 181, 186, 192, 196, 202, 206, 211, 217, 220, 224, 230, 235, 240, 246, 248, 249, 254, 260, 263, 265, 268, 272, 274, 275, 276, 280, 282, 283, 283, 283, 283, 283, 283, 283, 283, 283, 284, 284, 284, 285, 285, 285, 285, 285, 285, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 2, 4, 8, 11, 14, 15, 16, 21, 23, 27, 30, 34, 38, 43, 50, 51, 58, 64, 69, 74, 79, 82, 86, 90, 95, 103, 112, 117, 120, 128, 134, 140, 146, 155, 160, 166, 170, 177, 179, 183, 191, 197, 201, 205, 206, 209, 213, 220, 227, 233, 242, 249, 255, 261, 264, 271, 277, 285, 291, 298, 302, 306, 308, 314, 320, 325, 332, 339, 346, 351, 356, 364, 371, 381, 388, 395, 401, 408, 414, 422, 429, 435, 438, 442, 446, 450, 456, 460, 466, 471, 475, 478, 485, 491, 497, 501, 506, 510, 513, 518], "exact_score": [18.254072168965788, 18.580620555596568, 18.7605853995819, 19.158709875910855, 19.365391685616633, 20.342300677436164, 20.914052101317214, 21.422445501491787, 21.642282252021808, 22.25171186668814, 22.459135147091352, 22.692270845185806, 23.044740243681066, 23.0947163959695, 24.37978371438448, 24.428601993420504, 24.616459534219217, 24.931736221180664, 24.951962008541948, 24.93796574744612, 24.994724509685078, 24.883816293056313, 24.90124159875614, 24.799014294105458, 25.096456554030734, 25.403705807509287, 24.081729465979944, 24.057365023474176, 24.220203289970733, 24.414737582490332, 24.55942334739803, 24.770284925564425, 24.705572853600877, 24.66206960887812, 25.562049895792494, 26.00125938769986, 26.28139629542705, 27.73810825587753, 27.38112522686026, 27.432323367658128, 27.887449065355923, 28.35341473112371, 28.23403343765335, 28.265324973809392, 28.387492459987374, 28.586572840518322, 28.64514777488093, 28.473417729254713, 28.096569685772486, 28.713951489686785, 28.577390611711305, 28.086742562597294, 27.740358372699518, 27.564776712334044, 26.99817795323413, 26.957527546730063, 26.85476295968099, 26.77319456553327, 26.66925298504246, 26.5886944458373, 26.746195632598447, 26.612886835155997, 26.481541860107978, 26.68873403019745, 26.546757714665524, 26.471975716361513, 25.987905385246407, 25.5224857538888, 25.45431300878855, 25.296544385191172, 25.10539714065037, 25.04259520828322, 24.88069301794792, 24.725879916217984, 24.196327910526875, 23.98182296844887, 23.631625381940637, 23.430497570044682, 23.358549435952337, 23.043805132317562, 22.657818293460547, 22.308619426490953, 22.25966172062826, 22.23569009312419, 22.211798798514664, 22.115020124526747, 22.08313018397654, 22.044008245872785, 22.013343291047054, 21.83007891955528, 21.726753976956196, 21.632813462998165, 21.548062173497957, 21.295973674022456, 21.261628302695502, 21.161937505009195, 21.134600531896165, 20.90935686418692, 20.889161741314176, 20.87347732288362, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.407162147055566, "threshold_area100": 24.41235845427527, "threshold_area25": 22.81073026317413}, "test": {"score": [18.129624718318375, 18.483584626824683, 19.009103271918597, 19.350063082241387, 19.525691298794023, 19.70666881003263, 19.84082896723027, 20.039367161110107, 20.332830464919212, 20.652967498759317, 21.138315813634552, 21.22365873587211, 21.374240801184875, 21.63681378311133, 21.980289877800583, 22.246130658075217, 23.551293263227066, 23.577884829970465, 23.701694883721068, 23.942044410362126, 23.936426513938557, 23.5490707630546, 23.747229780158776, 23.806767882140626, 24.166512790177382, 24.312729251127607, 24.51758574870166, 24.26856189728315, 24.38029707618767, 24.511115591588847, 24.609718130341214, 24.671200586855786, 25.24908710599758, 25.331608371560726, 25.708139341470915, 26.193802700526614, 26.406005614852912, 25.968261844628255, 26.25193668937415, 26.237595023038413, 25.844585820281964, 25.810747105439464, 25.813117944792936, 25.866263848705902, 25.9489513659954, 25.893134958196434, 25.969072909379648, 26.455645724025096, 25.770283280909517, 25.56066597365839, 25.024996390404546, 24.567534373689096, 24.18636622337654, 23.99676926740891, 23.88643308210386, 23.85785024394707, 23.722360745516525, 23.593196839006836, 23.978362060041327, 23.996277244218252, 23.85936749468689, 23.854394471365513, 23.782174109818598, 23.692984517329865, 23.599993976519045, 23.628900045857655, 23.513089604469467, 23.360952291872458, 23.155061258451042, 22.780637494206253, 22.511411313738233, 22.429134546769525, 22.36683285590769, 22.23040994495523, 22.032040461463286, 21.991931226972294, 22.73986948471249, 22.72595375348308, 22.544215471019932, 22.48982908500075, 22.443339095663948, 22.467401937401764, 22.794818949663288, 22.7597624586553, 22.687777790647793, 22.648989630861234, 22.61111489029925, 22.53775405777002, 22.82713391155726, 22.770471001091476, 22.673256053014317, 22.643272903303487, 22.62337836534709, 22.562202248995806, 22.53339748613188, 22.396688291629584, 22.251189254020325, 22.22207803341944, 22.15748452847682, 22.071392992397723, 22.361115727328066], "calls": [0.0, 1.0652463382157125, 2.1970705725699067, 3.2956058588548602, 4.194407456724368, 4.793608521970706, 5.359520639147803, 6.458055925432757, 7.456724367509987, 8.988015978695072, 11.384820239680426, 12.083888149134488, 12.81624500665779, 13.914780292942744, 15.013315579227696, 16.344873501997338, 18.042609853528628, 19.37416777629827, 19.70705725699068, 20.206391478029296, 21.171770972037287, 22.436750998668444, 23.735019973368843, 24.60053262316911, 25.665778961384824, 26.897470039946736, 27.929427430093206, 29.027962716378163, 29.793608521970704, 30.725699067909456, 31.19174434087883, 32.05725699067909, 33.75499334221039, 34.52063914780293, 35.519307589880164, 36.01864181091877, 36.651131824234355, 37.54993342210386, 38.548601864181094, 39.34753661784288, 40.44607190412783, 41.07856191744341, 41.944074567243675, 42.64314247669774, 43.3422103861518, 43.97470039946737, 44.973368841544605, 46.07190412782956, 46.737683089214386, 47.70306258322237, 48.934753661784285, 50.29960053262317, 51.33155792276965, 52.729693741677764, 53.46205059920106, 53.728362183754996, 55.29294274300932, 56.55792276964048, 57.75632490013316, 58.788282290279625, 60.21970705725699, 61.58455392809587, 62.45006657789614, 63.34886817576565, 64.21438082556591, 64.68042609853528, 66.37816245006658, 67.24367509986683, 68.07589880159787, 69.84021304926765, 71.30492676431425, 72.7363515312916, 73.5352862849534, 74.63382157123834, 76.26498002663115, 76.93075898801598, 78.26231691078563, 78.728362183755, 79.4607190412783, 80.15978695073235, 80.82556591211718, 81.6577896138482, 83.288948069241, 84.45406125166444, 85.15312916111851, 86.25166444740347, 87.61651131824235, 88.58189081225034, 90.0133155792277, 91.21171770972038, 92.11051930758988, 92.74300932090546, 93.30892143808256, 93.77496671105193, 94.8069241011984, 95.57256990679095, 97.00399467376832, 97.47003994673769, 98.50199733688414, 99.20106524633822, 100.0], "rescued": [0, 54, 104, 149, 184, 207, 233, 271, 308, 362, 440, 460, 479, 505, 535, 569, 617, 655, 665, 680, 699, 728, 762, 781, 800, 826, 846, 868, 885, 901, 911, 928, 967, 983, 1007, 1022, 1035, 1047, 1066, 1075, 1088, 1101, 1112, 1121, 1134, 1138, 1145, 1152, 1156, 1158, 1163, 1169, 1171, 1171, 1171, 1171, 1171, 1171, 1172, 1174, 1174, 1176, 1177, 1177, 1177, 1178, 1180, 1180, 1180, 1182, 1182, 1185, 1185, 1187, 1187, 1188, 1192, 1193, 1193, 1193, 1193, 1194, 1195, 1195, 1195, 1195, 1195, 1195, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1198], "harmed": [0, 3, 14, 28, 37, 40, 51, 67, 79, 103, 140, 153, 168, 193, 216, 240, 276, 300, 306, 317, 337, 370, 402, 417, 436, 462, 482, 510, 522, 539, 546, 565, 599, 617, 641, 649, 659, 679, 694, 704, 719, 741, 757, 766, 785, 797, 805, 823, 842, 861, 895, 937, 964, 1001, 1023, 1031, 1068, 1098, 1129, 1150, 1184, 1211, 1231, 1255, 1276, 1285, 1322, 1344, 1370, 1418, 1458, 1488, 1507, 1542, 1600, 1619, 1651, 1662, 1682, 1700, 1715, 1738, 1763, 1782, 1803, 1829, 1851, 1866, 1891, 1916, 1936, 1945, 1958, 1966, 1985, 2005, 2038, 2046, 2073, 2092, 2109], "exact_score": [18.129624718318375, 18.47019441487749, 18.947501056143246, 19.293255219208984, 19.451852368842626, 19.765438367433337, 19.885881608103833, 20.147081342027004, 20.4628200289411, 20.652967498759317, 20.98560036301108, 21.019391202486954, 21.21227241252229, 21.37909334704956, 21.6618996486218, 21.977668325751644, 22.2564879697914, 22.379328862413047, 23.556064289395646, 23.492390600723468, 23.927773964359485, 23.896185141580396, 23.475549911701393, 23.597497814483656, 23.803353430350754, 24.207255642739515, 24.234365162130068, 24.320678076294445, 24.520776010325726, 24.259923011038218, 24.428100378853117, 24.562269578679473, 24.660393504026953, 25.044269140132503, 25.28623832891716, 25.42846634246818, 26.13492544905403, 26.578011658353667, 26.00492385081331, 26.12185371705521, 26.27119928627966, 25.860508309423068, 25.792801878674947, 25.86881880496894, 25.893134958196434, 25.969072909379648, 26.507512872063817, 25.76250431142517, 25.577768241836946, 25.043067119887773, 24.70984898706286, 24.257736207969284, 24.101438707837115, 23.9546541688688, 23.82839176016023, 23.741994906559523, 23.651418878735765, 23.557576646942785, 23.966048573151568, 23.966726314304545, 23.888482362059857, 23.829729458872002, 23.81618332640846, 23.733120875560296, 23.61453515129282, 23.690110723584006, 23.528186667780417, 23.392438431676993, 23.165399558344635, 22.990469222492308, 22.72184520090448, 22.586014055288818, 22.44867200538518, 22.39552456239238, 22.29712260970241, 22.20690435427251, 22.04624507036752, 21.98961877551196, 22.30532693475592, 22.69041021088758, 22.51512943934229, 22.437238674047173, 22.843468854544604, 22.818741847274136, 22.772166493480935, 22.69670383213856, 22.656146185909947, 22.627150962861535, 22.600667762151016, 22.86263169394444, 22.82856857297717, 22.791528851004404, 22.728547031766887, 22.635547653087627, 22.557763408464275, 22.529755444221077, 22.36589339763433, 22.298608302929686, 22.187617465422907, 22.142614107696204, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.21268253006761, "threshold_area100": 23.29097197854663, "threshold_area25": 21.64974375608079}, "selected_budget": 49, "selected": {"val": {"score": 28.713951489686785, "calls": 48.93009985734665, "rescued": 280, "harmed": 227}, "test": {"score": 25.56066597365839, "calls": 47.70306258322237, "rescued": 1158, "harmed": 861}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.9627993106842041, "operator": "ge"}, {"budget": 0.02, "threshold": 0.8974404335021973, "operator": "ge"}, {"budget": 0.03, "threshold": 0.8432167768478394, "operator": "ge"}, {"budget": 0.04, "threshold": 0.8052730560302734, "operator": "ge"}, {"budget": 0.05, "threshold": 0.709918737411499, "operator": "ge"}, {"budget": 0.06, "threshold": 0.6738080978393555, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5978015661239624, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5359371304512024, "operator": "ge"}, {"budget": 0.09, "threshold": 0.46213412284851074, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3830212354660034, "operator": "ge"}, {"budget": 0.11, "threshold": 0.3519654870033264, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3302079141139984, "operator": "ge"}, {"budget": 0.13, "threshold": 0.309420645236969, "operator": "ge"}, {"budget": 0.14, "threshold": 0.2968369722366333, "operator": "ge"}, {"budget": 0.15, "threshold": 0.2683740556240082, "operator": "ge"}, {"budget": 0.16, "threshold": 0.24319365620613098, "operator": "ge"}, {"budget": 0.17, "threshold": 0.22898831963539124, "operator": "ge"}, {"budget": 0.18, "threshold": 0.21929267048835754, "operator": "ge"}, {"budget": 0.19, "threshold": 0.20190277695655823, "operator": "ge"}, {"budget": 0.2, "threshold": 0.1855747103691101, "operator": "ge"}, {"budget": 0.21, "threshold": 0.16313189268112183, "operator": "ge"}, {"budget": 0.22, "threshold": 0.13917456567287445, "operator": "ge"}, {"budget": 0.23, "threshold": 0.12589234113693237, "operator": "ge"}, {"budget": 0.24, "threshold": 0.1112169623374939, "operator": "ge"}, {"budget": 0.25, "threshold": 0.09576781094074249, "operator": "ge"}, {"budget": 0.26, "threshold": 0.07588908821344376, "operator": "ge"}, {"budget": 0.27, "threshold": 0.053674936294555664, "operator": "ge"}, {"budget": 0.28, "threshold": 0.03494769334793091, "operator": "ge"}, {"budget": 0.29, "threshold": 0.02358507364988327, "operator": "ge"}, {"budget": 0.3, "threshold": 0.018569111824035645, "operator": "ge"}, {"budget": 0.31, "threshold": 0.001526176929473877, "operator": "ge"}, {"budget": 0.32, "threshold": -0.01969355344772339, "operator": "ge"}, {"budget": 0.33, "threshold": -0.05297945439815521, "operator": "ge"}, {"budget": 0.34, "threshold": -0.08130058646202087, "operator": "ge"}, {"budget": 0.35, "threshold": -0.09599852561950684, "operator": "ge"}, {"budget": 0.36, "threshold": -0.11356650292873383, "operator": "ge"}, {"budget": 0.37, "threshold": -0.14723935723304749, "operator": "ge"}, {"budget": 0.38, "threshold": -0.17097359895706177, "operator": "ge"}, {"budget": 0.39, "threshold": -0.21247807145118713, "operator": "ge"}, {"budget": 0.4, "threshold": -0.23240751028060913, "operator": "ge"}, {"budget": 0.41, "threshold": -0.27519068121910095, "operator": "ge"}, {"budget": 0.42, "threshold": -0.3075917363166809, "operator": "ge"}, {"budget": 0.43, "threshold": -0.3667084276676178, "operator": "ge"}, {"budget": 0.44, "threshold": -0.45845821499824524, "operator": "ge"}, {"budget": 0.45, "threshold": -0.6547278761863708, "operator": "ge"}, {"budget": 0.46, "threshold": -1.0456180572509766, "operator": "ge"}, {"budget": 0.47, "threshold": -1.4178836345672607, "operator": "ge"}, {"budget": 0.48, "threshold": -1.6630620956420898, "operator": "ge"}, {"budget": 0.49, "threshold": -1.793367862701416, "operator": "ge"}, {"budget": 0.5, "threshold": -1.9688243865966797, "operator": "ge"}, {"budget": 0.51, "threshold": -2.2059030532836914, "operator": "ge"}, {"budget": 0.52, "threshold": -3.627060890197754, "operator": "ge"}, {"budget": 0.53, "threshold": -3.7210545539855957, "operator": "ge"}, {"budget": 0.54, "threshold": -3.7480740547180176, "operator": "ge"}, {"budget": 0.55, "threshold": -3.778897285461426, "operator": "ge"}, {"budget": 0.56, "threshold": -3.8276946544647217, "operator": "ge"}, {"budget": 0.57, "threshold": -3.8493549823760986, "operator": "ge"}, {"budget": 0.58, "threshold": -3.85530948638916, "operator": "ge"}, {"budget": 0.59, "threshold": -3.8776662349700928, "operator": "ge"}, {"budget": 0.6, "threshold": -3.8920185565948486, "operator": "ge"}, {"budget": 0.61, "threshold": -3.9266304969787598, "operator": "ge"}, {"budget": 0.62, "threshold": -3.9438328742980957, "operator": "ge"}, {"budget": 0.63, "threshold": -3.9579918384552, "operator": "ge"}, {"budget": 0.64, "threshold": -3.9956283569335938, "operator": "ge"}, {"budget": 0.65, "threshold": -4.018506050109863, "operator": "ge"}, {"budget": 0.66, "threshold": -4.043986797332764, "operator": "ge"}, {"budget": 0.67, "threshold": -4.077042102813721, "operator": "ge"}, {"budget": 0.68, "threshold": -4.098222732543945, "operator": "ge"}, {"budget": 0.69, "threshold": -4.1222147941589355, "operator": "ge"}, {"budget": 0.7, "threshold": -4.143379211425781, "operator": "ge"}, {"budget": 0.71, "threshold": -4.208569526672363, "operator": "ge"}, {"budget": 0.72, "threshold": -4.244857311248779, "operator": "ge"}, {"budget": 0.73, "threshold": -4.289419174194336, "operator": "ge"}, {"budget": 0.74, "threshold": -4.324270725250244, "operator": "ge"}, {"budget": 0.75, "threshold": -4.373085975646973, "operator": "ge"}, {"budget": 0.76, "threshold": -4.417248725891113, "operator": "ge"}, {"budget": 0.77, "threshold": -4.458086013793945, "operator": "ge"}, {"budget": 0.78, "threshold": -4.512639999389648, "operator": "ge"}, {"budget": 0.79, "threshold": -4.549860954284668, "operator": "ge"}, {"budget": 0.8, "threshold": -4.612930774688721, "operator": "ge"}, {"budget": 0.81, "threshold": -4.688554763793945, "operator": "ge"}, {"budget": 0.82, "threshold": -11.034050941467285, "operator": "ge"}, {"budget": 0.83, "threshold": -11.063652038574219, "operator": "ge"}, {"budget": 0.84, "threshold": -11.103889465332031, "operator": "ge"}, {"budget": 0.85, "threshold": -11.131792068481445, "operator": "ge"}, {"budget": 0.86, "threshold": -11.147787094116211, "operator": "ge"}, {"budget": 0.87, "threshold": -11.172905921936035, "operator": "ge"}, {"budget": 0.88, "threshold": -11.203827857971191, "operator": "ge"}, {"budget": 0.89, "threshold": -11.217547416687012, "operator": "ge"}, {"budget": 0.9, "threshold": -11.258858680725098, "operator": "ge"}, {"budget": 0.91, "threshold": -11.297635078430176, "operator": "ge"}, {"budget": 0.92, "threshold": -11.32463264465332, "operator": "ge"}, {"budget": 0.93, "threshold": -11.415425300598145, "operator": "ge"}, {"budget": 0.94, "threshold": -11.578113555908203, "operator": "ge"}, {"budget": 0.95, "threshold": -11.716915130615234, "operator": "ge"}, {"budget": 0.96, "threshold": -11.929435729980469, "operator": "ge"}, {"budget": 0.97, "threshold": -12.035179138183594, "operator": "ge"}, {"budget": 0.98, "threshold": -12.219362258911133, "operator": "ge"}, {"budget": 0.99, "threshold": -12.317176818847656, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.30829772550402, 18.76878953872259, 19.86042183622829, 20.26068025645083, 20.65976719288762, 20.936961727949264, 21.543701387117064, 22.08458969034937, 22.5063829359495, 22.495282992645137, 22.8037084398977, 23.07551122881223, 24.44228368385672, 24.437623062809152, 24.72533044463554, 24.95778994975871, 25.21553747457362, 25.23834387556192, 25.067297114521303, 25.074316221841386, 24.859371964208638, 25.00502529978512, 25.09149502592125, 25.15622559846507, 25.51876171739016, 24.280032206119163, 24.120479559748425, 24.239337527008757, 24.44876559214127, 24.55942334739803, 24.770284925564425, 24.556457181789035, 25.077017471595788, 25.339409676914244, 25.537344963881694, 26.02822447223877, 26.116710070872255, 26.80962171910273, 27.273123790104805, 26.732024680300544, 26.90623109497883, 27.268701460876155, 28.277032034257466, 28.094960475215526, 28.575427360520088, 28.64514777488093, 28.45687513206695, 28.30136645587832, 28.779981030620416, 28.574773953027016, 28.083698849622245, 27.740358372699518, 27.577765622897466, 27.404058013569326, 27.315062895720786, 26.659508247743542, 26.47080102837452, 26.3920222529494, 26.2364887706942, 26.423902753439283, 26.588461723492756, 26.73345092589293, 25.838020901466592, 25.78174407982573, 25.737646686239096, 25.671725396440408, 25.251174438480938, 25.1937656641604, 25.12078403859067, 25.075056153708253, 25.018232389692514, 24.857152528305736, 24.475458498347987, 24.14847398305822, 24.10890401750786, 23.67450517856297, 23.280635038825654, 23.227262338394134, 23.01484674329502, 22.894904653428263, 22.53016749113733, 22.1953579221354, 22.068787076054807, 22.04795737382703, 21.998699998356912, 21.89283405110743, 21.79124894947864, 21.76999044607728, 21.740026144272353, 21.721678092483327, 21.706751976756177, 21.666718508134515, 21.424640871177015, 21.32416611768253, 20.972424969024463, 20.936187833483466, 20.91370374529972, 20.897656474579552, 20.700243493746637, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 24, 37, 50, 60, 68, 81, 91, 101, 108, 114, 121, 128, 134, 139, 146, 153, 157, 161, 166, 169, 175, 179, 184, 189, 194, 197, 201, 207, 211, 217, 218, 225, 229, 232, 239, 240, 245, 250, 254, 255, 257, 263, 267, 270, 274, 275, 276, 281, 282, 283, 283, 283, 283, 283, 283, 283, 283, 283, 284, 285, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286, 286], "harmed": [0, 5, 5, 7, 11, 14, 17, 19, 21, 22, 27, 29, 36, 40, 46, 49, 53, 60, 63, 72, 76, 81, 86, 90, 93, 97, 105, 112, 116, 124, 128, 134, 139, 145, 154, 157, 164, 166, 172, 175, 183, 185, 188, 192, 196, 203, 209, 214, 220, 228, 233, 242, 249, 254, 260, 266, 273, 281, 287, 293, 297, 302, 308, 315, 320, 324, 330, 336, 341, 348, 352, 358, 366, 373, 382, 386, 393, 403, 409, 417, 423, 429, 434, 441, 444, 451, 456, 461, 465, 469, 472, 475, 482, 487, 492, 496, 502, 505, 508, 512, 518], "exact_score": [18.254072168965788, 18.30829772550402, 18.76878953872259, 19.86042183622829, 20.26068025645083, 20.65976719288762, 20.936961727949264, 21.543701387117064, 22.08458969034937, 22.5063829359495, 22.495282992645137, 22.8037084398977, 23.07551122881223, 24.44228368385672, 24.437623062809152, 24.72533044463554, 24.95778994975871, 25.21553747457362, 25.23834387556192, 25.067297114521303, 25.074316221841386, 24.859371964208638, 25.00502529978512, 25.09149502592125, 25.15622559846507, 25.51876171739016, 24.280032206119163, 24.120479559748425, 24.239337527008757, 24.44876559214127, 24.55942334739803, 24.770284925564425, 24.556457181789035, 25.077017471595788, 25.339409676914244, 25.537344963881694, 26.02822447223877, 26.116710070872255, 26.80962171910273, 27.273123790104805, 26.732024680300544, 26.90623109497883, 27.268701460876155, 28.277032034257466, 28.094960475215526, 28.575427360520088, 28.64514777488093, 28.45687513206695, 28.30136645587832, 28.779981030620416, 28.574773953027016, 28.083698849622245, 27.740358372699518, 27.577765622897466, 27.404058013569326, 27.315062895720786, 26.659508247743542, 26.47080102837452, 26.3920222529494, 26.2364887706942, 26.423902753439283, 26.588461723492756, 26.73345092589293, 25.838020901466592, 25.78174407982573, 25.737646686239096, 25.671725396440408, 25.251174438480938, 25.1937656641604, 25.12078403859067, 25.075056153708253, 25.018232389692514, 24.857152528305736, 24.475458498347987, 24.14847398305822, 24.10890401750786, 23.67450517856297, 23.280635038825654, 23.227262338394134, 23.01484674329502, 22.894904653428263, 22.53016749113733, 22.1953579221354, 22.068787076054807, 22.04795737382703, 21.998699998356912, 21.89283405110743, 21.79124894947864, 21.76999044607728, 21.740026144272353, 21.721678092483327, 21.706751976756177, 21.666718508134515, 21.424640871177015, 21.32416611768253, 20.972424969024463, 20.936187833483466, 20.91370374529972, 20.897656474579552, 20.700243493746637, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 24.350934365527916, "threshold_area100": 24.356174085794823, "threshold_area25": 22.96822275903753}, "test": {"score": [18.129624718318375, 18.39335406394741, 18.592739416723333, 18.60073685162977, 19.00803185404581, 19.669582757136777, 19.79341884763814, 20.12828915548711, 20.364141346545225, 20.638543643639938, 21.057306279564035, 21.234895773434783, 21.695945979164584, 21.789569905306244, 21.916326342549223, 22.849617601386807, 23.15547640762412, 23.349671073830976, 23.49307233079947, 23.59154912515881, 23.727346158563677, 23.508125261701725, 24.19040087004087, 24.18268965391465, 24.299684683288337, 24.41217339718089, 24.597909336358377, 24.31165089728693, 24.445335545730604, 24.567262954859604, 24.59862804930969, 24.68746106051382, 25.017706561315688, 25.386876905690364, 25.618605962246164, 25.879270033455114, 26.179333938283804, 26.307880059550076, 26.413831590062415, 25.9379580645216, 25.97540520915706, 26.107206445306325, 26.260958448494975, 26.280645543103414, 25.717267808672666, 25.821743201786017, 25.89773632198013, 26.452311056824882, 25.808997436766834, 25.68512728525437, 24.960795356487083, 24.56321377417374, 24.15219315697643, 23.985466886794903, 23.87003671101874, 23.755317484494494, 23.496350416119625, 23.382635538422818, 23.337281576456213, 23.158946405726784, 23.08155313359673, 22.785104948880466, 22.836387913424662, 22.816117890129846, 23.20385791092285, 23.01249514355509, 23.110295831509813, 22.86123554485496, 22.785347850893665, 22.805292737254184, 22.57242501974163, 22.396410672261105, 22.17263579860311, 22.14862229847925, 22.602974719149604, 22.474569625391382, 22.364278798781562, 22.186420881353662, 22.50706445172774, 22.523973618442692, 22.443584169643273, 22.407556505171904, 22.81957944923385, 22.809459162396006, 22.713988986296787, 22.64865201784828, 23.001205646804994, 22.962188764811486, 22.88611318853728, 22.816799504680922, 22.66141912799693, 22.581984691769197, 22.573685987048638, 22.44036027088908, 22.37452223258983, 22.288558206522254, 22.17908385203364, 22.127111589704803, 22.470784220605182, 22.36744040113833, 22.361115727328066], "calls": [0.0, 0.6990679094540613, 1.2982689747003995, 1.9640479360852199, 2.6631158455392807, 4.427430093209055, 5.226364846870839, 6.524633821571238, 7.490013315579228, 8.388814913448735, 10.119840213049267, 11.218375499334222, 12.316910785619175, 13.182423435419441, 14.047936085219707, 16.11185086551265, 17.643142476697736, 18.47536617842876, 18.908122503328894, 19.77363515312916, 20.838881491344875, 22.270306258322236, 23.768308921438084, 24.766977363515313, 25.699067909454058, 26.59786950732357, 27.729693741677764, 29.161118508655125, 30.126498002663116, 30.758988015978694, 30.99201065246338, 32.090545938748335, 33.12250332889481, 34.68708388814913, 35.685752330226364, 35.98535286284953, 36.55126498002663, 37.54993342210386, 38.515312916111846, 39.54727030625832, 40.14647137150466, 41.17842876165113, 41.777629826897474, 42.50998668442077, 43.54194407456725, 44.54061251664447, 45.47270306258323, 46.27163781624501, 47.003994673768304, 47.802929427430094, 49.10119840213049, 50.3994673768309, 51.46471371504661, 52.197070572569906, 52.92942743009321, 53.8948069241012, 55.59254327563249, 56.62450066577897, 56.99067909454061, 58.1225033288948, 58.854860186418115, 60.885486018641814, 62.017310252996005, 63.04926764314248, 65.21304926764314, 66.41145139813582, 67.11051930758988, 68.24234354194407, 68.9081225033289, 69.54061251664447, 70.00665778961384, 71.6711051930759, 73.06924101198402, 74.2676431424767, 75.06657789613848, 76.23169107856191, 77.29693741677764, 78.36218375499334, 79.29427430093209, 79.89347536617842, 81.05858854860186, 81.55792276964048, 82.49001331557922, 82.92276964047936, 84.38748335552596, 85.81890812250333, 86.65113182423435, 87.9161118508655, 89.08122503328894, 89.68042609853528, 91.17842876165113, 92.24367509986683, 92.47669773635153, 94.10785619174435, 95.73901464713715, 96.63781624500666, 97.27030625832224, 97.6364846870839, 99.03462050599201, 99.76697736351531, 100.0], "rescued": [0, 38, 71, 105, 133, 208, 240, 288, 317, 350, 407, 437, 470, 490, 514, 569, 611, 634, 646, 666, 692, 729, 760, 781, 802, 820, 847, 877, 893, 904, 909, 929, 951, 986, 1008, 1015, 1024, 1040, 1055, 1069, 1076, 1085, 1093, 1098, 1114, 1130, 1143, 1152, 1156, 1159, 1163, 1169, 1171, 1171, 1171, 1172, 1173, 1173, 1173, 1173, 1173, 1173, 1176, 1177, 1180, 1180, 1183, 1184, 1184, 1185, 1185, 1185, 1185, 1186, 1188, 1189, 1189, 1189, 1191, 1192, 1193, 1193, 1195, 1195, 1195, 1195, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1196, 1198, 1198, 1198], "harmed": [0, 8, 16, 25, 31, 55, 66, 79, 92, 101, 128, 146, 164, 185, 202, 243, 275, 294, 305, 323, 343, 373, 402, 422, 443, 456, 487, 518, 534, 541, 546, 565, 586, 616, 642, 646, 651, 665, 680, 699, 706, 717, 728, 737, 766, 787, 808, 824, 843, 857, 902, 935, 968, 991, 1013, 1040, 1089, 1118, 1127, 1155, 1175, 1230, 1259, 1284, 1330, 1356, 1372, 1392, 1406, 1423, 1432, 1477, 1513, 1541, 1562, 1592, 1624, 1651, 1679, 1692, 1723, 1737, 1758, 1764, 1789, 1816, 1834, 1860, 1886, 1901, 1926, 1946, 1951, 1985, 2017, 2032, 2044, 2050, 2083, 2104, 2109], "exact_score": [18.129624718318375, 18.483194754954912, 18.608099118916517, 19.068484434163704, 19.565566786941528, 19.787087918302014, 20.016145857614827, 20.33037140578586, 20.555027578740255, 20.77885587866109, 21.034668369816682, 21.203149129830866, 21.531282631649564, 21.744254427969764, 21.910829211248057, 22.212501292802383, 22.648510584165532, 22.981123860050616, 23.196535193991156, 23.509499085374884, 23.606976130989835, 23.746102736285064, 23.421838381685685, 23.73283914201823, 24.207491854550675, 24.204733206968804, 24.296859145610647, 24.4666353933268, 24.599888840767477, 24.244003999711772, 24.440818550676426, 24.59862804930969, 24.654976777174433, 25.010217255556455, 25.332485823529804, 25.330109302367738, 25.879270033455114, 26.223025322814486, 26.401097355590387, 25.9828885899542, 25.936445078887104, 26.139716212168036, 26.260958448494975, 25.732454679823103, 25.727406992056217, 25.8983279318343, 26.50097092665313, 25.816777773340547, 25.54150722375685, 25.052131090914674, 24.72569962569234, 24.252684495294318, 24.024943660491036, 23.866169162221794, 23.74499803156047, 23.610618513200425, 23.442891840889043, 23.337281576456213, 23.170385253961864, 23.052338074826476, 22.96220509563476, 22.776610176082407, 22.838928482298147, 22.82152214607561, 23.181151541815222, 23.217647082790542, 23.113934263552945, 23.0159246910874, 22.92526709732362, 22.780516147276394, 22.57242501974163, 22.479700957343947, 22.33020652470577, 22.17839442138301, 22.176270187047532, 22.60772741882012, 22.481158305817484, 22.379404701789138, 22.20660806701885, 22.58675920642706, 22.51659972061462, 22.446922649800182, 22.843902876586505, 22.78723666392128, 22.740922640226326, 22.674561586967172, 22.641451943482895, 22.988147315232464, 22.95919834696467, 22.889500953299102, 22.740895092357892, 22.664193606494063, 22.591152239728125, 22.494333298636686, 22.446600933865206, 22.394796032637984, 22.370788412476987, 22.280316931232573, 22.099197634630926, 22.474852623387946, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 23.153471207997566, "threshold_area100": 23.213285487910042, "threshold_area25": 21.71495037088019}, "selected_budget": 49, "selected": {"val": {"score": 28.779981030620416, "calls": 48.93009985734665, "rescued": 281, "harmed": 228}, "test": {"score": 25.68512728525437, "calls": 47.802929427430094, "rescued": 1159, "harmed": 857}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 28.424105629679847, "calls": 50.92724679029958}, "test": {"score": 25.305307441491287, "calls": 50.76564580559254}}, "old_normalized_interior": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726}, "test": {"score": 25.902457757296464, "calls": 45.40612516644474}}, "old_normalized_endpoints": {"val": {"score": 28.64514777488093, "calls": 45.93437945791726}, "test": {"score": 25.902457757296464, "calls": 45.40612516644474}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.450958204581394, 18.81987003168782, 19.77617595528369, 20.201956178328523, 20.586850695500196, 20.936961727949264, 21.543701387117064, 22.05458089668616, 22.5063829359495, 22.459135147091352, 22.692270845185806, 23.087364106185397, 24.43267259056733, 24.55606213969928, 24.72533044463554, 25.08740944250928, 25.310183863566678, 25.333350030054095, 25.33440475252483, 25.21064172393667, 25.11949911068846, 24.835741990696647, 25.082541467188936, 25.409272656213698, 25.51876171739016, 24.100974346416145, 24.120479559748425, 24.269931932603168, 24.44876559214127, 24.55942334739803, 24.770284925564425, 24.556457181789035, 24.66206960887812, 25.339409676914244, 25.593478932848917, 26.250170508260485, 26.116710070872255, 26.80962171910273, 27.265624320891895, 26.732024680300544, 26.80488159822123, 27.675777690063946, 28.29541776692891, 28.094960475215526, 28.575427360520088, 28.64514777488093, 28.45687513206695, 28.42355380207058, 28.358445788354587, 28.10916418662973, 27.984747674813637, 27.705944666001997, 27.848085849639798, 27.774082119509586, 27.482849494643997, 27.413453704199302, 26.87258374544469, 26.939449402146, 26.889354919889275, 26.84056948106904, 26.791682612578132, 26.768332773632565, 26.774756774756774, 27.073556695405436, 26.579455173882415, 26.724366605858414, 26.187901908075055, 26.13975531521902, 25.837377413810536, 25.67986679505697, 24.96332583603464, 25.15068068569249, 25.02384347044541, 24.789516436457692, 24.817178715092133, 24.68882791170736, 24.48881673881674, 24.425398048279405, 24.346644580436962, 24.28798858146684, 24.247284889596934, 23.911110511741647, 23.43644958217022, 23.1280745277979, 23.075532737074685, 22.485213931469165, 22.22716049382716, 22.182827789146593, 22.032548581099455, 21.76719846593722, 21.734474470323526, 21.600640886743257, 21.29786600456336, 21.19475381582211, 21.087841000585666, 20.97057592612318, 20.85528416347382, 20.812502813408255, 20.78489453270283, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.450958204581394, 18.81987003168782, 19.77617595528369, 20.201956178328523, 20.586850695500196, 20.936961727949264, 21.543701387117064, 22.05458089668616, 22.5063829359495, 22.459135147091352, 22.692270845185806, 23.087364106185397, 24.43267259056733, 24.55606213969928, 24.72533044463554, 25.08740944250928, 25.310183863566678, 25.333350030054095, 25.33440475252483, 25.21064172393667, 25.11949911068846, 24.835741990696647, 25.082541467188936, 25.409272656213698, 25.51876171739016, 24.100974346416145, 24.120479559748425, 24.269931932603168, 24.44876559214127, 24.55942334739803, 24.770284925564425, 24.556457181789035, 24.66206960887812, 25.339409676914244, 25.593478932848917, 26.250170508260485, 26.116710070872255, 26.80962171910273, 27.265624320891895, 26.732024680300544, 26.80488159822123, 27.675777690063946, 28.29541776692891, 28.094960475215526, 28.575427360520088, 28.64514777488093, 28.45687513206695, 28.42355380207058, 28.358445788354587, 28.10916418662973, 27.984747674813637, 27.705944666001997, 27.848085849639798, 27.774082119509586, 27.482849494643997, 27.413453704199302, 26.87258374544469, 26.939449402146, 26.889354919889275, 26.84056948106904, 26.791682612578132, 26.768332773632565, 26.774756774756774, 27.073556695405436, 26.579455173882415, 26.724366605858414, 26.187901908075055, 26.13975531521902, 25.837377413810536, 25.67986679505697, 24.96332583603464, 25.15068068569249, 25.02384347044541, 24.789516436457692, 24.817178715092133, 24.68882791170736, 24.48881673881674, 24.425398048279405, 24.346644580436962, 24.28798858146684, 24.247284889596934, 23.911110511741647, 23.43644958217022, 23.1280745277979, 23.075532737074685, 22.485213931469165, 22.22716049382716, 22.182827789146593, 22.032548581099455, 21.76719846593722, 21.734474470323526, 21.600640886743257, 21.29786600456336, 21.19475381582211, 21.087841000585666, 20.97057592612318, 20.85528416347382, 20.812502813408255, 20.78489453270283, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 22.97367499086835, "exact_area100": 24.631318513876224}, "test": {"score": [18.129624718318375, 18.48642895213548, 18.37083030134489, 18.583852392992465, 19.01027096287319, 19.72338509430686, 19.87515389132074, 20.177933088682323, 20.515454401797733, 20.680969588838266, 21.190833835457923, 21.340349367919544, 21.588040266128065, 21.78177642891311, 22.2543321643197, 22.94596001146803, 23.400150258112557, 23.413398935815916, 23.58746844996972, 23.588007022399456, 23.391884321425916, 23.731071941585117, 23.859135163568318, 24.185591615683816, 24.304086296903137, 24.437243907866417, 24.5491983065167, 24.243230238480045, 24.430156468332786, 24.57787283949755, 24.59862804930969, 24.69895686293006, 25.017706561315688, 25.374265830520482, 25.59907964526872, 25.906885653869573, 26.22693374966817, 26.29535227362355, 26.365962460789593, 25.973893558253415, 25.961035484995133, 26.223321509289736, 26.207681575325008, 26.269774370941985, 25.77379044273787, 25.816230706783937, 25.902457757296464, 26.35636521334262, 26.087002278167738, 25.97201572569383, 25.455636502474572, 25.410829916076583, 25.225482622229457, 25.05278459247455, 24.858743115655678, 24.710442724055735, 24.635215606258363, 24.460717587467336, 24.868263130167893, 24.765455080218413, 24.61305509738271, 24.524486991024496, 24.45840888983419, 24.536379358725764, 24.340227159122765, 24.272453841374368, 24.00674702256325, 23.766842470959347, 23.694990268914868, 23.671978932143894, 23.213343819450184, 23.246775312504187, 23.109029824039105, 23.10682793049379, 22.94539008333729, 22.926520974913668, 22.883862002304305, 22.74967211790081, 22.574570341270388, 22.45716573099995, 22.825536794931452, 22.965517114161376, 22.753190412384892, 22.833686004941505, 22.63687633044111, 22.55198289995753, 22.467633476935855, 22.44047033458652, 22.4161439138354, 22.363736736117875, 22.5925619444141, 22.553292476348147, 22.38407765474965, 22.594917280749176, 22.513447629286908, 22.47870044657073, 22.429917150867478, 22.414799205599735, 22.447846431062544, 22.406974506249554, 22.361115727328066], "calls": [0.0, 0.8988015978695073, 1.1984021304926764, 1.9973368841544608, 2.929427430093209, 4.327563249001331, 5.292942743009322, 6.857523302263649, 7.723035952063914, 8.322237017310254, 10.452729693741677, 11.484687083888149, 12.583222370173102, 13.715046604527299, 15.046604527296935, 16.378162450066576, 18.342210386151798, 18.541944074567244, 19.241011984021306, 19.640479360852197, 21.338215712383487, 22.33688415446072, 23.33555259653795, 24.866844207723034, 26.03195739014647, 26.63115845539281, 27.62982689747004, 29.094540612516646, 30.059920106524636, 30.825565912117174, 30.99201065246338, 32.123834886817576, 33.12250332889481, 34.6537949400799, 35.519307589880164, 36.18508655126498, 36.684420772303596, 37.45006657789614, 38.21571238348868, 39.4474034620506, 40.246338215712385, 41.37816245006658, 41.67776298268975, 42.41011984021305, 43.9081225033289, 44.44074567243675, 45.40612516644474, 46.304926764314246, 47.436750998668444, 48.00266311584554, 49.03462050599201, 49.70039946737683, 50.699067909454065, 51.73102529960053, 52.69640479360852, 54.460719041278296, 55.559254327563245, 57.65645805592543, 58.72170439414115, 59.8202396804261, 61.085219707057256, 62.217043941411454, 63.81491344873502, 64.61384820239681, 65.74567243675101, 66.87749667110519, 68.1757656458056, 69.47403462050599, 70.0399467376831, 70.47270306258322, 71.97070572569906, 72.43675099866844, 73.03595206391478, 73.66844207723035, 74.53395472703063, 74.80026631158455, 75.89880159786951, 76.89747003994674, 77.62982689747004, 78.92809587217043, 79.39414114513981, 80.6924101198402, 81.92410119840213, 83.65512649800266, 85.41944074567243, 86.75099866844208, 87.84953395472704, 88.48202396804261, 89.0479360852197, 90.04660452729694, 91.21171770972038, 92.84287616511318, 94.10785619174435, 94.94007989347537, 95.30625832223701, 96.1051930758988, 97.13715046604527, 97.93608521970705, 98.33555259653795, 99.23435419440746, 100.0], "exact_score": [18.129624718318375, 18.32043510007531, 18.583852392992465, 19.023425185030412, 19.623575268347064, 19.870757185381738, 20.007997820240423, 20.126981947592224, 20.583781083195575, 20.743830319217864, 21.000140038180696, 21.261441603243593, 21.47028995284713, 21.69703517545612, 21.879735301159865, 22.22004638080076, 22.64860801485431, 23.44206952050089, 23.207276733858958, 23.496925320844053, 23.122264405188023, 23.2855299175478, 23.541460915947475, 23.667331393696582, 24.16374961617757, 24.23816613708872, 24.296599389171185, 24.48270754701302, 24.104989447622252, 24.238253200989725, 24.475083162371806, 24.59862804930969, 24.654976777174433, 25.010217255556455, 25.333295831697413, 25.333540056110227, 25.766103865711, 26.27899989665628, 26.3482240667275, 25.925787492123053, 25.936445078887104, 26.32521136711287, 26.21652033415437, 25.691639956560042, 25.790424234310134, 25.944457398252048, 26.460004686535676, 26.178987875217462, 25.97201572569383, 25.458894862495423, 25.40010557881582, 25.218084451260115, 25.01991289433389, 24.84471672020048, 24.776536861952465, 24.641512362763713, 24.64728296398687, 24.532131561633946, 24.912063813510578, 24.800109520750695, 24.6930277774646, 24.619000523989165, 24.497667419847627, 24.497175067417714, 24.44705854995187, 24.470235860436027, 24.318327029645484, 24.267138350343384, 23.972677924485154, 23.797574814593744, 23.699880573282286, 23.50851177757335, 23.213343819450184, 23.11197497493478, 23.036258970918492, 22.964485031576118, 22.80961351482742, 22.745383981665896, 22.55768349764083, 22.45308085670704, 23.17222905008906, 22.951293376734377, 22.748345518856087, 22.915625036049438, 22.86618740029434, 22.67416279896552, 22.6515661754838, 22.525436221237676, 22.463210959442343, 22.41735655440065, 22.367026878756754, 22.494375679817463, 22.62423246064408, 22.547192701458886, 22.407305156305256, 22.592987337682494, 22.485529992048633, 22.43717780847561, 22.413031330169293, 22.368315272454957, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.707099515150226, "exact_area100": 23.469265121022126}}, "old_normalized_endpoint_budget": 46}]}, {"expert": "yolo26x", "endpoint": "Qwen3.8", "task": "construction", "small": 18.129624718318375, "large": 27.001009067886244, "runs": [{"seed": 2026, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.75, "validation_area100": 0.28042240724382755, "training_seconds": 5.145923204021528, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5767569489553759, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5412700503256116, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5178315465253231, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4993041790041114, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4890451700361891, "operator": "ge"}, {"budget": 0.06, "threshold": 0.46046963834394283, "operator": "ge"}, {"budget": 0.07, "threshold": 0.442966722242126, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4369329268521253, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4244260759750869, "operator": "ge"}, {"budget": 0.1, "threshold": 0.41518182442074175, "operator": "ge"}, {"budget": 0.11, "threshold": 0.40715794446363757, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3940500596261666, "operator": "ge"}, {"budget": 0.13, "threshold": 0.38775216201063745, "operator": "ge"}, {"budget": 0.14, "threshold": 0.37655580372366737, "operator": "ge"}, {"budget": 0.15, "threshold": 0.37248310002927265, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3599187780701946, "operator": "ge"}, {"budget": 0.17, "threshold": 0.3472245449625865, "operator": "ge"}, {"budget": 0.18, "threshold": 0.34020227500958955, "operator": "ge"}, {"budget": 0.19, "threshold": 0.33457196615659324, "operator": "ge"}, {"budget": 0.2, "threshold": 0.31936553586688377, "operator": "ge"}, {"budget": 0.21, "threshold": 0.3104633758916613, "operator": "ge"}, {"budget": 0.22, "threshold": 0.3033149952508489, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2948517425044238, "operator": "ge"}, {"budget": 0.24, "threshold": 0.29024285668546806, "operator": "ge"}, {"budget": 0.25, "threshold": 0.2816648462726659, "operator": "ge"}, {"budget": 0.26, "threshold": 0.27862158929804715, "operator": "ge"}, {"budget": 0.27, "threshold": 0.27155698258306255, "operator": "ge"}, {"budget": 0.28, "threshold": 0.2635131653315581, "operator": "ge"}, {"budget": 0.29, "threshold": 0.2571840838220835, "operator": "ge"}, {"budget": 0.3, "threshold": 0.243554910001795, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2339148827038077, "operator": "ge"}, {"budget": 0.32, "threshold": 0.22599605676783116, "operator": "ge"}, {"budget": 0.33, "threshold": 0.21933767197276266, "operator": "ge"}, {"budget": 0.34, "threshold": 0.21386527064699548, "operator": "ge"}, {"budget": 0.35, "threshold": 0.20595300750229678, "operator": "ge"}, {"budget": 0.36, "threshold": 0.1971168906340148, "operator": "ge"}, {"budget": 0.37, "threshold": 0.18582700770658564, "operator": "ge"}, {"budget": 0.38, "threshold": 0.17587087695392006, "operator": "ge"}, {"budget": 0.39, "threshold": 0.16735674839070394, "operator": "ge"}, {"budget": 0.4, "threshold": 0.1533794972550758, "operator": "ge"}, {"budget": 0.41, "threshold": 0.14981905179652122, "operator": "ge"}, {"budget": 0.42, "threshold": 0.1420025853637456, "operator": "ge"}, {"budget": 0.43, "threshold": 0.09943296147653857, "operator": "ge"}, {"budget": 0.44, "threshold": 0.07346894120283932, "operator": "ge"}, {"budget": 0.45, "threshold": 0.015573756125582155, "operator": "ge"}, {"budget": 0.46, "threshold": -0.0518085115333748, "operator": "ge"}, {"budget": 0.47, "threshold": -0.12859324126301463, "operator": "ge"}, {"budget": 0.48, "threshold": -0.17031100765674467, "operator": "ge"}, {"budget": 0.49, "threshold": -0.2543430593043022, "operator": "ge"}, {"budget": 0.5, "threshold": -0.3139048811947463, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3533419813698783, "operator": "ge"}, {"budget": 0.52, "threshold": -0.3605068891190945, "operator": "ge"}, {"budget": 0.53, "threshold": -0.373451262798054, "operator": "ge"}, {"budget": 0.54, "threshold": -0.38017316227437914, "operator": "ge"}, {"budget": 0.55, "threshold": -0.38530867490348747, "operator": "ge"}, {"budget": 0.56, "threshold": -0.3878670157178773, "operator": "ge"}, {"budget": 0.57, "threshold": -0.3955888295678394, "operator": "ge"}, {"budget": 0.58, "threshold": -0.404911218649825, "operator": "ge"}, {"budget": 0.59, "threshold": -0.4096542667802907, "operator": "ge"}, {"budget": 0.6, "threshold": -0.414717420101763, "operator": "ge"}, {"budget": 0.61, "threshold": -0.41778835001106274, "operator": "ge"}, {"budget": 0.62, "threshold": -0.4230754209649138, "operator": "ge"}, {"budget": 0.63, "threshold": -0.43128917021794666, "operator": "ge"}, {"budget": 0.64, "threshold": -0.43739756889848, "operator": "ge"}, {"budget": 0.65, "threshold": -0.4447347614099298, "operator": "ge"}, {"budget": 0.66, "threshold": -0.450344881351038, "operator": "ge"}, {"budget": 0.67, "threshold": -0.46099584008626, "operator": "ge"}, {"budget": 0.68, "threshold": -0.46861009007037446, "operator": "ge"}, {"budget": 0.69, "threshold": -0.4722762496428581, "operator": "ge"}, {"budget": 0.7, "threshold": -0.4765043062892239, "operator": "ge"}, {"budget": 0.71, "threshold": -0.48907082254619716, "operator": "ge"}, {"budget": 0.72, "threshold": -0.49416798181999433, "operator": "ge"}, {"budget": 0.73, "threshold": -0.5008760278571068, "operator": "ge"}, {"budget": 0.74, "threshold": -0.5062834235161603, "operator": "ge"}, {"budget": 0.75, "threshold": -0.5119980870773462, "operator": "ge"}, {"budget": 0.76, "threshold": -0.5158359278823604, "operator": "ge"}, {"budget": 0.77, "threshold": -0.5203648296013472, "operator": "ge"}, {"budget": 0.78, "threshold": -0.5235391276253115, "operator": "ge"}, {"budget": 0.79, "threshold": -0.5261229447918406, "operator": "ge"}, {"budget": 0.8, "threshold": -0.5313482840099643, "operator": "ge"}, {"budget": 0.81, "threshold": -0.5343904379933875, "operator": "ge"}, {"budget": 0.82, "threshold": -0.5398593936667728, "operator": "ge"}, {"budget": 0.83, "threshold": -0.5440163023395661, "operator": "ge"}, {"budget": 0.84, "threshold": -0.5478636509222338, "operator": "ge"}, {"budget": 0.85, "threshold": -0.551530891195087, "operator": "ge"}, {"budget": 0.86, "threshold": -0.556840562776176, "operator": "ge"}, {"budget": 0.87, "threshold": -0.5654447553115265, "operator": "ge"}, {"budget": 0.88, "threshold": -0.5696638656376866, "operator": "ge"}, {"budget": 0.89, "threshold": -0.5713163602598944, "operator": "ge"}, {"budget": 0.9, "threshold": -0.5740740663933696, "operator": "ge"}, {"budget": 0.91, "threshold": -0.5768454573946012, "operator": "ge"}, {"budget": 0.92, "threshold": -0.5804244057125174, "operator": "ge"}, {"budget": 0.93, "threshold": -0.5848051616117161, "operator": "ge"}, {"budget": 0.94, "threshold": -0.5905234008781014, "operator": "ge"}, {"budget": 0.95, "threshold": -0.5932003249943009, "operator": "ge"}, {"budget": 0.96, "threshold": -0.6028231081786356, "operator": "ge"}, {"budget": 0.97, "threshold": -0.6065927001046937, "operator": "ge"}, {"budget": 0.98, "threshold": -0.6140958177336873, "operator": "ge"}, {"budget": 0.99, "threshold": -0.6205923254919529, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.570977475658456, 18.939804972392004, 19.532085561497325, 20.007060092011546, 20.41223481384454, 20.68854291844614, 21.108170766781008, 21.47239478006324, 21.574214206856478, 22.49433231284411, 22.878220334050724, 23.03179988755957, 23.412178370768306, 23.81782317823178, 24.035336356764926, 24.154125197463884, 24.404939614299217, 24.770447713969308, 25.06962233670308, 25.249003984063744, 25.488080612246662, 25.809940997675668, 26.123945805415232, 26.247801446159862, 26.650873115203527, 27.127116308183624, 27.35608202090905, 27.60619080547289, 27.878231492361927, 28.27911031235839, 27.05717893217893, 27.205111402359105, 27.640250137628865, 27.9032521653865, 28.655804191837543, 29.377535572136697, 29.46077426275446, 29.86853866647381, 29.754638757627845, 30.09374854536145, 30.12972050990484, 30.740794293425864, 30.822739839288936, 32.45950771152384, 32.81285807463818, 32.69493177387914, 32.417701264450486, 32.101422912119006, 32.002077423572956, 31.905241072074897, 31.819116101773165, 32.401866684523746, 32.20173026624639, 32.015185025353325, 31.78372124676739, 31.70178231283604, 32.15596429187511, 32.15596429187511, 32.08651984743066, 32.08651984743066, 31.98571339581776, 31.80994918297595, 31.490182376436916, 31.490182376436916, 31.424064245376094, 31.795603879849814, 31.643170813600058, 31.483596345514954, 31.272927007547423, 31.036184583305005, 30.894627960990316, 30.530024509803923, 30.941514376442754, 30.866439301367677, 30.80074412697685, 30.374922955568117, 30.240884535397416, 30.030914850578217, 29.970607833034357, 29.838478576948468, 29.640065878535772, 29.442399946707198, 29.25854700854701, 29.079808348904045, 28.72194854953476, 28.152576334312666, 27.878680293286536, 27.663268302162326, 27.663268302162326, 27.506158832378347, 27.301661752881266, 27.021117138428913, 26.914328312562123, 26.57322352444304, 26.506000357289228, 26.379173042690734, 26.174861635940307, 25.65261473292303, 25.580499348307644, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 23, 37, 47, 56, 63, 73, 80, 87, 96, 104, 110, 116, 124, 131, 136, 143, 151, 157, 163, 169, 175, 181, 187, 194, 202, 207, 212, 218, 223, 227, 231, 238, 244, 250, 257, 261, 267, 272, 276, 281, 287, 291, 296, 298, 299, 299, 300, 301, 302, 302, 303, 303, 303, 303, 303, 304, 304, 304, 304, 304, 304, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 1, 2, 2, 4, 5, 5, 8, 9, 13, 15, 15, 16, 19, 20, 21, 23, 23, 25, 27, 28, 29, 32, 33, 36, 37, 38, 42, 43, 46, 49, 54, 56, 59, 60, 63, 66, 71, 73, 78, 79, 82, 84, 84, 85, 90, 92, 93, 95, 96, 96, 98, 99, 102, 103, 104, 104, 105, 105, 106, 108, 110, 110, 111, 117, 119, 121, 124, 125, 127, 130, 131, 132, 133, 137, 139, 142, 143, 145, 146, 149, 152, 155, 161, 168, 171, 173, 177, 180, 182, 186, 188, 194, 203, 205, 208, 215, 217, 220], "exact_score": [18.254072168965788, 18.570977475658456, 18.939804972392004, 19.532085561497325, 20.007060092011546, 20.41223481384454, 20.68854291844614, 21.108170766781008, 21.47239478006324, 21.574214206856478, 22.49433231284411, 22.878220334050724, 23.03179988755957, 23.412178370768306, 23.81782317823178, 24.035336356764926, 24.154125197463884, 24.404939614299217, 24.770447713969308, 25.06962233670308, 25.249003984063744, 25.488080612246662, 25.809940997675668, 26.123945805415232, 26.247801446159862, 26.650873115203527, 27.127116308183624, 27.35608202090905, 27.60619080547289, 27.878231492361927, 28.27911031235839, 27.05717893217893, 27.205111402359105, 27.640250137628865, 27.9032521653865, 28.655804191837543, 29.377535572136697, 29.46077426275446, 29.86853866647381, 29.754638757627845, 30.09374854536145, 30.12972050990484, 30.740794293425864, 30.822739839288936, 32.45950771152384, 32.81285807463818, 32.69493177387914, 32.417701264450486, 32.101422912119006, 32.002077423572956, 31.905241072074897, 31.819116101773165, 32.401866684523746, 32.20173026624639, 32.015185025353325, 31.78372124676739, 31.70178231283604, 32.15596429187511, 32.15596429187511, 32.08651984743066, 32.08651984743066, 31.98571339581776, 31.80994918297595, 31.490182376436916, 31.490182376436916, 31.424064245376094, 31.795603879849814, 31.643170813600058, 31.483596345514954, 31.272927007547423, 31.036184583305005, 30.894627960990316, 30.530024509803923, 30.941514376442754, 30.866439301367677, 30.80074412697685, 30.374922955568117, 30.240884535397416, 30.030914850578217, 29.970607833034357, 29.838478576948468, 29.640065878535772, 29.442399946707198, 29.25854700854701, 29.079808348904045, 28.72194854953476, 28.152576334312666, 27.878680293286536, 27.663268302162326, 27.663268302162326, 27.506158832378347, 27.301661752881266, 27.021117138428913, 26.914328312562123, 26.57322352444304, 26.506000357289228, 26.379173042690734, 26.174861635940307, 25.65261473292303, 25.580499348307644, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 28.042240724382754, "threshold_area100": 28.045851692867004, "threshold_area25": 22.928676021920396}, "test": {"score": [18.129624718318375, 18.34658029461874, 18.62089513751095, 18.66341878796323, 18.945120681660867, 18.868537688102375, 19.40965505310129, 19.82335406378871, 19.833554615719958, 20.37655968034874, 21.03003107523692, 21.255550029330344, 21.67540912918265, 21.767839053763787, 21.85834371217866, 21.84959586537697, 22.265059133979527, 22.291169141310753, 22.468932748538013, 22.621288846946104, 23.19956758370792, 23.676207048662683, 24.084904119372542, 24.317174147608164, 24.643336278875495, 24.6470959873525, 24.739407001976833, 25.181661381425364, 25.276627437912335, 25.422715737961944, 25.881992673937397, 26.144447837057054, 26.519920731627632, 26.97331733005322, 27.036699029708544, 27.24836667667884, 27.294897044080212, 28.13910201998482, 28.55629660507968, 28.894659191054966, 29.63109793926727, 29.660846855637878, 29.89057578222604, 30.272504331600693, 30.267238540011533, 29.94456979113117, 29.94439002291685, 29.875993589336698, 29.523954675288195, 29.878586296941478, 30.1851248107429, 30.10935699810804, 30.069245261964717, 29.95817675144782, 29.852531490011668, 29.70010643394537, 29.660922679685903, 29.56339556431047, 29.373784578410916, 29.664086468122253, 29.53948676595847, 29.467781869279573, 29.3966142005075, 29.120078501980817, 29.071873501825863, 29.05719126438792, 29.0223165550105, 28.84903113438537, 28.730437080505627, 29.02782168324402, 28.987450662196334, 29.025280341771886, 29.043130477389205, 29.323940975767986, 29.22416121081169, 29.114595621013763, 28.97499361788646, 28.75523204380303, 28.967722112384553, 29.166573212045954, 29.03457404914652, 28.947535532960554, 28.825828765782802, 28.84466375656476, 29.29313710222587, 29.13026976906668, 29.088436688603892, 28.87144047946985, 28.592071791972867, 28.523074044683582, 28.39599626812198, 28.21466260158584, 28.137181602927463, 28.06327299305599, 27.757768860176295, 27.78670284058215, 27.57207730941641, 27.467068813876626, 27.318185180400928, 27.327787555917077, 27.001009067886244], "calls": [0.0, 0.7656458055925432, 1.4980026631158456, 2.330226364846871, 3.129161118508655, 3.7283621837549936, 5.625832223701731, 6.990679094540613, 7.390146471371504, 8.788282290279627, 10.086551264980026, 10.885486018641812, 12.117177097203728, 12.78295605858855, 14.314247669773636, 14.913448735019974, 16.411451398135817, 17.7762982689747, 18.375499334221036, 19.0745672436751, 20.805592543275633, 21.937416777629824, 22.836218375499335, 23.868175765645805, 24.63382157123835, 25.63249001331558, 25.798934753661783, 26.564580559254324, 27.563249001331556, 28.428761651131822, 30.22636484687084, 31.22503328894807, 32.39014647137151, 33.32223701731025, 33.82157123834887, 35.05326231691079, 36.051930758988014, 37.416777629826896, 38.48202396804261, 39.38082556591212, 40.31291611185087, 40.54593874833555, 41.14513981358189, 43.20905459387483, 43.74167776298269, 44.87350199733689, 45.372836218375504, 46.03861517976032, 46.47137150466045, 48.46870838881491, 49.30093209054594, 50.3994673768309, 50.96537949400799, 53.16245006657789, 54.49400798934754, 55.35952063914781, 55.59254327563249, 57.25699067909454, 59.08788282290279, 60.086551264980024, 61.051930758988014, 61.75099866844208, 62.64980026631159, 64.18109187749667, 65.1797603195739, 66.41145139813582, 67.47669773635153, 68.94141145139814, 70.0399467376831, 70.5392809587217, 70.93874833555259, 72.7363515312916, 73.8681757656458, 75.0, 75.86551264980027, 76.89747003994674, 77.43009320905459, 78.36218375499334, 78.8948069241012, 79.36085219707057, 80.29294274300932, 81.0918774966711, 82.02396804260985, 83.12250332889481, 83.98801597869507, 84.95339547270306, 86.1185086551265, 88.48202396804261, 89.58055925432757, 90.07989347536618, 90.77896138482025, 91.81091877496671, 92.50998668442078, 93.37549933422103, 94.50732356857523, 94.90679094540613, 96.271637816245, 96.77097203728363, 97.6364846870839, 98.50199733688414, 100.0], "rescued": [0, 38, 76, 122, 158, 180, 259, 306, 320, 374, 417, 440, 481, 501, 545, 560, 603, 638, 655, 673, 718, 744, 762, 788, 808, 831, 837, 857, 875, 890, 933, 958, 987, 1009, 1018, 1049, 1068, 1101, 1123, 1141, 1164, 1168, 1180, 1214, 1221, 1234, 1238, 1242, 1244, 1251, 1253, 1254, 1254, 1254, 1254, 1254, 1254, 1255, 1256, 1257, 1257, 1257, 1257, 1257, 1258, 1259, 1259, 1260, 1260, 1261, 1261, 1264, 1267, 1269, 1269, 1270, 1270, 1270, 1271, 1272, 1272, 1272, 1272, 1273, 1275, 1275, 1276, 1277, 1277, 1277, 1277, 1277, 1279, 1279, 1279, 1280, 1281, 1281, 1281, 1282, 1282], "harmed": [0, 2, 5, 12, 13, 19, 28, 35, 37, 44, 50, 53, 59, 60, 68, 74, 79, 93, 95, 99, 105, 114, 122, 130, 135, 147, 147, 154, 161, 165, 177, 186, 197, 210, 215, 231, 243, 256, 261, 271, 281, 282, 284, 303, 310, 316, 318, 324, 327, 336, 338, 343, 345, 348, 353, 359, 361, 367, 377, 380, 386, 390, 394, 407, 416, 420, 423, 434, 442, 444, 447, 459, 469, 476, 482, 493, 499, 514, 517, 521, 528, 537, 544, 555, 562, 571, 581, 608, 623, 628, 636, 648, 658, 664, 685, 690, 709, 718, 732, 748, 777], "exact_score": [18.129624718318375, 18.441092378980578, 18.58016011713009, 18.911756220564367, 18.96641705999187, 19.189441588075585, 19.517183764442812, 19.82335406378871, 20.06954509202395, 20.41330416741251, 21.012881027080493, 21.298214601057, 21.614659926293232, 21.844084587728656, 22.074295376824146, 21.85916013983491, 22.107595062933573, 22.402149641371246, 22.322186425804176, 22.592065599120954, 22.831055300515338, 23.18685581188112, 23.676207048662683, 24.054385406480975, 24.3171712561297, 24.686330564634172, 24.71375574054778, 25.243725073572023, 25.340280529910913, 25.58919952882801, 25.824533048259045, 26.087233151246547, 26.480658607492195, 26.917418330958377, 27.0500618059537, 27.23200467392409, 27.278202163454445, 28.003193416527527, 28.308293364243976, 28.767064832302157, 29.53994925891369, 29.853873693472295, 30.11939515163906, 30.2514432675565, 30.35811430318645, 29.98263974088433, 29.875993589336698, 29.836480195635488, 29.796953497804836, 30.225258485105535, 30.116092465102735, 30.069245261964717, 30.069245261964717, 29.95817675144782, 29.88525230762732, 29.75955946605, 29.61492730951353, 29.56339556431047, 29.499292403918737, 29.373784578410916, 29.664086468122253, 29.53948676595847, 29.467781869279573, 29.356845499375734, 29.13994605164969, 29.071873501825863, 29.08073914507866, 29.045507201449773, 28.895181046343836, 28.84903113438537, 28.748725937652296, 28.987450662196334, 28.907504257409744, 29.00713956937811, 29.002020508221683, 29.323940975767986, 29.31434571033817, 29.055733636418967, 28.87956673351401, 28.967722112384553, 29.08001341179015, 28.947535532960554, 28.825828765782802, 28.560897628849585, 29.29313710222587, 29.13026976906668, 29.146329466554178, 29.162702146818088, 28.925801375597615, 28.783461136993072, 28.53205715643349, 28.37835980599628, 28.141687978593854, 28.104127018876856, 27.85744706701676, 27.738287738287738, 27.638803281413317, 27.42849466196493, 27.29642977190519, 27.2612699463711, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.79605213429762, "threshold_area100": 26.78498545195682, "threshold_area25": 21.267218513051926}, "selected_budget": 45, "selected": {"val": {"score": 32.81285807463818, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.94456979113117, "calls": 44.87350199733689, "rescued": 1234, "harmed": 316}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.4445183277130127, "operator": "ge"}, {"budget": 0.02, "threshold": 0.37748005986213684, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3587694764137268, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3536924421787262, "operator": "ge"}, {"budget": 0.05, "threshold": 0.3364779055118561, "operator": "ge"}, {"budget": 0.06, "threshold": 0.31278932094573975, "operator": "ge"}, {"budget": 0.07, "threshold": 0.2913793921470642, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2596673369407654, "operator": "ge"}, {"budget": 0.09, "threshold": 0.2235143631696701, "operator": "ge"}, {"budget": 0.1, "threshold": 0.2030419409275055, "operator": "ge"}, {"budget": 0.11, "threshold": 0.1932654082775116, "operator": "ge"}, {"budget": 0.12, "threshold": 0.182695209980011, "operator": "ge"}, {"budget": 0.13, "threshold": 0.17519184947013855, "operator": "ge"}, {"budget": 0.14, "threshold": 0.17200447618961334, "operator": "ge"}, {"budget": 0.15, "threshold": 0.1665722131729126, "operator": "ge"}, {"budget": 0.16, "threshold": 0.16295740008354187, "operator": "ge"}, {"budget": 0.17, "threshold": 0.1584639847278595, "operator": "ge"}, {"budget": 0.18, "threshold": 0.15553981065750122, "operator": "ge"}, {"budget": 0.19, "threshold": 0.1530124992132187, "operator": "ge"}, {"budget": 0.2, "threshold": 0.1495186984539032, "operator": "ge"}, {"budget": 0.21, "threshold": 0.14531216025352478, "operator": "ge"}, {"budget": 0.22, "threshold": 0.14251172542572021, "operator": "ge"}, {"budget": 0.23, "threshold": 0.13924874365329742, "operator": "ge"}, {"budget": 0.24, "threshold": 0.13551479578018188, "operator": "ge"}, {"budget": 0.25, "threshold": 0.13358381390571594, "operator": "ge"}, {"budget": 0.26, "threshold": 0.13093715906143188, "operator": "ge"}, {"budget": 0.27, "threshold": 0.12889857590198517, "operator": "ge"}, {"budget": 0.28, "threshold": 0.12356464564800262, "operator": "ge"}, {"budget": 0.29, "threshold": 0.12060867995023727, "operator": "ge"}, {"budget": 0.3, "threshold": 0.11767058074474335, "operator": "ge"}, {"budget": 0.31, "threshold": 0.11471492797136307, "operator": "ge"}, {"budget": 0.32, "threshold": 0.1107991635799408, "operator": "ge"}, {"budget": 0.33, "threshold": 0.10733550786972046, "operator": "ge"}, {"budget": 0.34, "threshold": 0.10259245336055756, "operator": "ge"}, {"budget": 0.35, "threshold": 0.09825846552848816, "operator": "ge"}, {"budget": 0.36, "threshold": 0.09314630180597305, "operator": "ge"}, {"budget": 0.37, "threshold": 0.09094487130641937, "operator": "ge"}, {"budget": 0.38, "threshold": 0.0850234255194664, "operator": "ge"}, {"budget": 0.39, "threshold": 0.08054026961326599, "operator": "ge"}, {"budget": 0.4, "threshold": 0.07732946425676346, "operator": "ge"}, {"budget": 0.41, "threshold": 0.07059865444898605, "operator": "ge"}, {"budget": 0.42, "threshold": 0.06512843072414398, "operator": "ge"}, {"budget": 0.43, "threshold": 0.05548107624053955, "operator": "ge"}, {"budget": 0.44, "threshold": 0.035460494458675385, "operator": "ge"}, {"budget": 0.45, "threshold": 0.009329468011856079, "operator": "ge"}, {"budget": 0.46, "threshold": -0.01451866514980793, "operator": "ge"}, {"budget": 0.47, "threshold": -0.016858508810400963, "operator": "ge"}, {"budget": 0.48, "threshold": -0.01898224465548992, "operator": "ge"}, {"budget": 0.49, "threshold": -0.02039104327559471, "operator": "ge"}, {"budget": 0.5, "threshold": -0.02159114181995392, "operator": "ge"}, {"budget": 0.51, "threshold": -0.02241542935371399, "operator": "ge"}, {"budget": 0.52, "threshold": -0.0257584135979414, "operator": "ge"}, {"budget": 0.53, "threshold": -0.029321983456611633, "operator": "ge"}, {"budget": 0.54, "threshold": -0.031141269952058792, "operator": "ge"}, {"budget": 0.55, "threshold": -0.03267061710357666, "operator": "ge"}, {"budget": 0.56, "threshold": -0.03475663438439369, "operator": "ge"}, {"budget": 0.57, "threshold": -0.03660355135798454, "operator": "ge"}, {"budget": 0.58, "threshold": -0.038238417357206345, "operator": "ge"}, {"budget": 0.59, "threshold": -0.039737679064273834, "operator": "ge"}, {"budget": 0.6, "threshold": -0.04138291999697685, "operator": "ge"}, {"budget": 0.61, "threshold": -0.04369465261697769, "operator": "ge"}, {"budget": 0.62, "threshold": -0.0453919842839241, "operator": "ge"}, {"budget": 0.63, "threshold": -0.04832033812999725, "operator": "ge"}, {"budget": 0.64, "threshold": -0.049424830824136734, "operator": "ge"}, {"budget": 0.65, "threshold": -0.05202291160821915, "operator": "ge"}, {"budget": 0.66, "threshold": -0.05633050948381424, "operator": "ge"}, {"budget": 0.67, "threshold": -0.05777600035071373, "operator": "ge"}, {"budget": 0.68, "threshold": -0.059294819831848145, "operator": "ge"}, {"budget": 0.69, "threshold": -0.06148127093911171, "operator": "ge"}, {"budget": 0.7, "threshold": -0.06447886675596237, "operator": "ge"}, {"budget": 0.71, "threshold": -0.06713595986366272, "operator": "ge"}, {"budget": 0.72, "threshold": -0.07080332934856415, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0727929100394249, "operator": "ge"}, {"budget": 0.74, "threshold": -0.07521030306816101, "operator": "ge"}, {"budget": 0.75, "threshold": -0.0774284079670906, "operator": "ge"}, {"budget": 0.76, "threshold": -0.07955319434404373, "operator": "ge"}, {"budget": 0.77, "threshold": -0.08133120089769363, "operator": "ge"}, {"budget": 0.78, "threshold": -0.08414655178785324, "operator": "ge"}, {"budget": 0.79, "threshold": -0.08706767857074738, "operator": "ge"}, {"budget": 0.8, "threshold": -0.08929240703582764, "operator": "ge"}, {"budget": 0.81, "threshold": -0.09029213339090347, "operator": "ge"}, {"budget": 0.82, "threshold": -0.09323795884847641, "operator": "ge"}, {"budget": 0.83, "threshold": -0.09626581519842148, "operator": "ge"}, {"budget": 0.84, "threshold": -0.10115911066532135, "operator": "ge"}, {"budget": 0.85, "threshold": -0.10327890515327454, "operator": "ge"}, {"budget": 0.86, "threshold": -0.10574319958686829, "operator": "ge"}, {"budget": 0.87, "threshold": -0.10903394967317581, "operator": "ge"}, {"budget": 0.88, "threshold": -0.11123649030923843, "operator": "ge"}, {"budget": 0.89, "threshold": -0.11484389752149582, "operator": "ge"}, {"budget": 0.9, "threshold": -0.1188347339630127, "operator": "ge"}, {"budget": 0.91, "threshold": -0.12283867597579956, "operator": "ge"}, {"budget": 0.92, "threshold": -0.12639909982681274, "operator": "ge"}, {"budget": 0.93, "threshold": -0.13018852472305298, "operator": "ge"}, {"budget": 0.94, "threshold": -0.13386113941669464, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1371196210384369, "operator": "ge"}, {"budget": 0.96, "threshold": -0.14040440320968628, "operator": "ge"}, {"budget": 0.97, "threshold": -0.14805355668067932, "operator": "ge"}, {"budget": 0.98, "threshold": -0.15379509329795837, "operator": "ge"}, {"budget": 0.99, "threshold": -0.16002704203128815, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.606411530247485, 19.102056225934355, 19.49886919597526, 20.006229235880397, 20.603658862360053, 21.002598907220143, 22.160030209416533, 22.963942864582275, 23.55829085321694, 23.741697966102787, 24.059476777208467, 24.31697523995313, 24.75729319941142, 25.060912472580032, 25.199071996848293, 25.55715674501058, 25.784158617776033, 25.614965385181915, 25.827505827505824, 26.23747618169525, 26.419861678825825, 26.49382773868344, 26.859887117900715, 26.966497501409712, 27.045094096724533, 27.556439363722905, 27.845337323081548, 27.619149358279792, 27.863162701872383, 26.663100253363947, 27.057156239852233, 27.4182466118878, 27.431544845337953, 27.544501274083856, 27.78191856452726, 28.24102368220015, 29.139576883384933, 29.564338747687707, 30.009303561935134, 30.207508227812795, 30.380593544940016, 30.54160125588697, 30.704656168573692, 32.31757548571856, 32.81285807463818, 32.69493177387914, 32.61088658236633, 32.61088658236633, 32.3954417293233, 32.3954417293233, 32.31313387094679, 32.18998116158719, 32.18998116158719, 32.18998116158719, 31.871105191634047, 31.691647074190183, 31.59022163559369, 31.475279106858057, 31.20646190255698, 31.09138710393788, 30.80314194916613, 30.49574607329843, 30.416666666666664, 30.36265432098765, 30.36265432098765, 30.68798687448806, 30.76624680540343, 30.45124445024564, 30.260396567249202, 30.260396567249202, 30.205677005005704, 29.840272404731532, 30.279548882490058, 30.128566193436306, 29.86453672378042, 29.450119169220297, 29.393939393939394, 29.24224296083593, 29.044794343301806, 29.418888334417076, 29.362897292983703, 28.65188577047202, 28.445413903773154, 28.34127977902048, 28.380169751528978, 28.085158548545273, 27.98725022633789, 27.75758365841715, 27.35057286751567, 27.259656262837602, 27.171957945050785, 26.997510405726068, 26.7445490714477, 26.665543110088798, 26.48184768069753, 26.379173042690734, 26.379173042690734, 25.78954417562284, 25.616277523620706, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 25, 36, 48, 58, 69, 80, 91, 101, 106, 114, 121, 128, 135, 141, 147, 153, 157, 162, 168, 172, 178, 185, 191, 196, 204, 210, 215, 220, 225, 231, 238, 241, 245, 250, 256, 263, 269, 272, 277, 281, 286, 290, 294, 298, 299, 299, 299, 299, 299, 299, 299, 299, 299, 299, 299, 300, 300, 300, 300, 300, 300, 300, 300, 300, 301, 302, 302, 302, 302, 302, 302, 303, 303, 303, 303, 303, 303, 303, 304, 304, 304, 304, 304, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 2, 2, 2, 4, 5, 6, 8, 9, 10, 12, 12, 15, 16, 19, 20, 24, 25, 25, 26, 28, 28, 30, 32, 34, 35, 40, 44, 47, 48, 51, 54, 58, 60, 63, 66, 68, 71, 75, 79, 81, 83, 84, 84, 85, 86, 86, 88, 92, 93, 94, 94, 94, 97, 98, 99, 100, 105, 107, 110, 112, 113, 114, 114, 117, 117, 119, 122, 122, 123, 126, 127, 129, 131, 135, 136, 138, 141, 143, 144, 151, 154, 156, 164, 167, 169, 171, 175, 181, 183, 186, 191, 197, 199, 205, 205, 212, 216, 220], "exact_score": [18.254072168965788, 18.606411530247485, 19.102056225934355, 19.49886919597526, 20.006229235880397, 20.603658862360053, 21.002598907220143, 22.160030209416533, 22.963942864582275, 23.55829085321694, 23.741697966102787, 24.059476777208467, 24.31697523995313, 24.75729319941142, 25.060912472580032, 25.199071996848293, 25.55715674501058, 25.784158617776033, 25.614965385181915, 25.827505827505824, 26.23747618169525, 26.419861678825825, 26.49382773868344, 26.859887117900715, 26.966497501409712, 27.045094096724533, 27.556439363722905, 27.845337323081548, 27.619149358279792, 27.863162701872383, 26.663100253363947, 27.057156239852233, 27.4182466118878, 27.431544845337953, 27.544501274083856, 27.78191856452726, 28.24102368220015, 29.139576883384933, 29.564338747687707, 30.009303561935134, 30.207508227812795, 30.380593544940016, 30.54160125588697, 30.704656168573692, 32.31757548571856, 32.81285807463818, 32.69493177387914, 32.61088658236633, 32.61088658236633, 32.3954417293233, 32.3954417293233, 32.31313387094679, 32.18998116158719, 32.18998116158719, 32.18998116158719, 31.871105191634047, 31.691647074190183, 31.59022163559369, 31.475279106858057, 31.20646190255698, 31.09138710393788, 30.80314194916613, 30.49574607329843, 30.416666666666664, 30.36265432098765, 30.36265432098765, 30.68798687448806, 30.76624680540343, 30.45124445024564, 30.260396567249202, 30.260396567249202, 30.205677005005704, 29.840272404731532, 30.279548882490058, 30.128566193436306, 29.86453672378042, 29.450119169220297, 29.393939393939394, 29.24224296083593, 29.044794343301806, 29.418888334417076, 29.362897292983703, 28.65188577047202, 28.445413903773154, 28.34127977902048, 28.380169751528978, 28.085158548545273, 27.98725022633789, 27.75758365841715, 27.35057286751567, 27.259656262837602, 27.171957945050785, 26.997510405726068, 26.7445490714477, 26.665543110088798, 26.48184768069753, 26.379173042690734, 26.379173042690734, 25.78954417562284, 25.616277523620706, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.997426803501124, "threshold_area100": 28.00094819625889, "threshold_area25": 23.686297627970312}, "test": {"score": [18.129624718318375, 18.31601143421796, 18.908079839982015, 18.871744804758922, 19.00379219172901, 19.206691488445795, 19.750187372423326, 20.045627005219607, 20.03539673888298, 20.221349007417658, 20.41696535244922, 20.677676219948022, 21.262338165525204, 21.653570908428495, 22.111009973227794, 22.732463996142453, 22.96802148879638, 23.220153914644893, 23.383420500749885, 23.631977336107298, 23.735864385149984, 24.013043860812136, 24.14895360259364, 24.406056467996308, 24.83781033149449, 25.19769139613234, 25.296344451228947, 25.356149384385112, 25.785041145602207, 25.918619937276656, 26.078135644546883, 26.608446547397453, 26.779853303734484, 26.8335345635652, 27.50380155002508, 28.105410671200147, 28.426466529506104, 28.643423497651277, 28.8629773004773, 29.09653951328227, 29.659699505862474, 29.993906087170743, 29.982347288457024, 30.15759736292656, 30.32331278429203, 29.925601605640118, 29.882018218028854, 29.878623379880803, 29.7699363478052, 29.75089968511021, 29.687237250698296, 29.668245066187175, 29.543325526932087, 29.45115084245314, 29.356561757490834, 29.323068101133, 29.14670433047391, 29.387196398146187, 29.31056856890147, 29.229770726659034, 29.173541260543466, 29.400519829187203, 29.283346793189313, 28.940118417658216, 28.934728638121122, 28.821398598515795, 28.709404303473434, 28.60562438414101, 28.555174410101046, 28.67979619993037, 29.136976184416056, 29.027143607618694, 29.200584431580097, 29.20660412998067, 29.02784065651544, 29.04456452804645, 28.952195714954705, 29.0810740008318, 29.112160715009104, 29.206889986545015, 29.362267175441193, 29.590717828630243, 29.398443319418462, 29.219420466987607, 29.268090129442403, 29.27132142499072, 29.018027268493036, 28.903996318431197, 28.876930967067814, 28.68781789231749, 28.348220424637915, 28.107607084285917, 28.01404774911947, 27.922722696174684, 27.86116379409075, 27.865612399448935, 27.741903185173243, 27.4395568755686, 27.306388783426787, 27.51209921789381, 27.001009067886244], "calls": [0.0, 0.7323568575233023, 2.396804260985353, 2.9960053262316912, 3.3288948069241013, 4.760319573901465, 6.225033288948069, 7.589880159786951, 8.88814913448735, 9.753661784287617, 10.319573901464715, 11.218375499334222, 12.916111850865514, 14.047936085219707, 14.613848202396804, 15.479360852197072, 16.511318242343542, 17.842876165113182, 18.808255659121173, 19.840213049267643, 20.838881491344875, 22.203728362183757, 23.00266311584554, 24.167776298268976, 25.233022636484687, 26.065246338215715, 26.764314247669773, 27.163781624500665, 29.360852197070574, 30.29294274300932, 30.9254327563249, 31.724367509986685, 32.9560585885486, 33.988015978695074, 35.18641810918775, 36.284953395472705, 37.71637816245006, 38.08255659121172, 39.14780292942743, 39.81358189081225, 40.37949400798935, 41.211717709720375, 41.87749667110519, 42.743009320905465, 43.77496671105193, 44.77363515312916, 45.572569906790946, 46.304926764314246, 47.70306258322237, 49.03462050599201, 49.56724367509987, 50.03328894806924, 51.93075898801598, 53.628495339547264, 54.460719041278296, 55.15978695073236, 56.0918774966711, 57.02396804260985, 58.18908122503329, 58.95472703062583, 60.21970705725699, 61.75099866844208, 62.849533954727036, 64.4474034620506, 64.98002663115847, 66.24500665778962, 67.84287616511318, 68.37549933422103, 68.80825565912117, 70.00665778961384, 71.17177097203728, 72.13715046604527, 73.50199733688414, 74.20106524633822, 74.96671105193076, 76.06524633821572, 76.99733688415446, 77.72969374167776, 78.69507323568575, 79.82689747003995, 80.5925432756325, 80.99201065246339, 81.89081225033289, 82.92276964047936, 84.4873501997337, 85.25299600532622, 86.15179760319573, 87.0838881491345, 88.11584553928095, 89.41411451398136, 90.87882822902796, 92.31025299600533, 93.1757656458056, 93.97470039946738, 94.54061251664447, 94.84021304926765, 95.90545938748336, 97.03728362183755, 97.83621837549934, 98.16910785619174, 100.0], "rescued": [0, 49, 134, 161, 176, 238, 301, 354, 405, 439, 459, 484, 536, 568, 582, 603, 628, 658, 680, 709, 731, 766, 783, 813, 841, 859, 873, 883, 926, 946, 963, 982, 1009, 1027, 1054, 1081, 1110, 1118, 1138, 1153, 1164, 1182, 1191, 1204, 1223, 1233, 1237, 1238, 1238, 1238, 1238, 1238, 1239, 1242, 1242, 1243, 1243, 1244, 1245, 1245, 1247, 1249, 1249, 1252, 1253, 1253, 1255, 1255, 1255, 1256, 1260, 1261, 1263, 1266, 1266, 1267, 1267, 1270, 1271, 1272, 1273, 1274, 1274, 1274, 1275, 1276, 1276, 1276, 1277, 1278, 1278, 1278, 1279, 1279, 1279, 1280, 1281, 1281, 1281, 1282, 1282], "harmed": [0, 1, 6, 14, 14, 27, 32, 41, 50, 56, 57, 61, 68, 72, 76, 77, 81, 88, 94, 99, 107, 115, 123, 131, 136, 145, 150, 154, 170, 177, 184, 188, 204, 217, 233, 246, 258, 261, 270, 278, 281, 289, 295, 300, 309, 316, 320, 322, 325, 326, 330, 331, 338, 345, 350, 352, 360, 365, 369, 374, 379, 385, 391, 403, 405, 411, 419, 425, 427, 438, 445, 452, 463, 467, 476, 482, 487, 492, 507, 518, 523, 527, 535, 544, 559, 566, 580, 590, 605, 618, 639, 658, 668, 673, 678, 684, 697, 718, 731, 733, 777], "exact_score": [18.129624718318375, 18.338699267186577, 18.527187211232558, 18.871744804758922, 19.213766537443693, 19.33898121335364, 19.668170325979677, 19.84985133713403, 19.85904065145679, 20.088988630436454, 20.301169407706176, 20.620966897407307, 20.91645645760922, 21.313907622678673, 21.62647731236938, 22.25055850687484, 22.827112369574, 23.052030091229906, 23.290692743927995, 23.43367067599932, 23.64878300418357, 23.768597514131606, 23.944157487284663, 24.137381116736936, 24.3561250405254, 24.802045076435324, 25.169020466854526, 25.361673532635624, 25.53046149467334, 25.853053951037985, 25.883913649222567, 26.078135644546883, 26.68482839640353, 26.78006743992808, 26.8335345635652, 27.502266062198494, 28.08369434091388, 28.164590885782133, 28.62287153053442, 28.843576062245273, 29.554185319182142, 29.944337777944018, 29.995319700460833, 30.157454714782904, 30.382496604563197, 29.98263974088433, 29.902475496756175, 29.878623379880803, 29.7699363478052, 29.75089968511021, 29.668245066187175, 29.611349488398673, 29.499580994998574, 29.51854918048963, 29.397227954115333, 29.30421255104282, 29.14670433047391, 29.387196398146187, 29.31056856890147, 29.229770726659034, 29.20443709806341, 29.48466603248606, 29.358986932475982, 29.283346793189313, 28.940118417658216, 28.934728638121122, 28.880930409486893, 28.796754030396766, 28.67962691610503, 28.518994771009275, 28.67979619993037, 29.023509850251468, 29.027143607618694, 29.17203292901848, 29.179260138508273, 29.02784065651544, 29.06203347214583, 28.952195714954705, 28.977665463120143, 29.044005290137953, 29.50219199996783, 29.590717828630243, 29.32545757060418, 29.181605006435102, 28.995871741281988, 29.29262662193698, 29.046186002700765, 28.913942325819658, 28.91173778920495, 28.749643539159543, 28.570674212373337, 28.348220424637915, 28.255482309436125, 28.043746014740755, 27.922722696174684, 27.857426787850493, 27.733967263129838, 27.44719643069776, 27.51209921789381, 27.275179726887703, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.802108380717126, "threshold_area100": 26.87859771341905, "threshold_area25": 21.551128655882565}, "selected_budget": 45, "selected": {"val": {"score": 32.81285807463818, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.925601605640118, "calls": 44.77363515312916, "rescued": 1233, "harmed": 316}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.7818996906280518, "operator": "ge"}, {"budget": 0.02, "threshold": 2.574815273284912, "operator": "ge"}, {"budget": 0.03, "threshold": 2.371493101119995, "operator": "ge"}, {"budget": 0.04, "threshold": 2.269347667694092, "operator": "ge"}, {"budget": 0.05, "threshold": 2.1717700958251953, "operator": "ge"}, {"budget": 0.06, "threshold": 2.076301097869873, "operator": "ge"}, {"budget": 0.07, "threshold": 2.0229010581970215, "operator": "ge"}, {"budget": 0.08, "threshold": 1.9544034004211426, "operator": "ge"}, {"budget": 0.09, "threshold": 1.9134628772735596, "operator": "ge"}, {"budget": 0.1, "threshold": 1.8791277408599854, "operator": "ge"}, {"budget": 0.11, "threshold": 1.8455530405044556, "operator": "ge"}, {"budget": 0.12, "threshold": 1.8077380657196045, "operator": "ge"}, {"budget": 0.13, "threshold": 1.7670834064483643, "operator": "ge"}, {"budget": 0.14, "threshold": 1.7102168798446655, "operator": "ge"}, {"budget": 0.15, "threshold": 1.65541672706604, "operator": "ge"}, {"budget": 0.16, "threshold": 1.5953835248947144, "operator": "ge"}, {"budget": 0.17, "threshold": 1.5456634759902954, "operator": "ge"}, {"budget": 0.18, "threshold": 1.5057117938995361, "operator": "ge"}, {"budget": 0.19, "threshold": 1.4670501947402954, "operator": "ge"}, {"budget": 0.2, "threshold": 1.427680492401123, "operator": "ge"}, {"budget": 0.21, "threshold": 1.3981815576553345, "operator": "ge"}, {"budget": 0.22, "threshold": 1.3608214855194092, "operator": "ge"}, {"budget": 0.23, "threshold": 1.3349496126174927, "operator": "ge"}, {"budget": 0.24, "threshold": 1.2921987771987915, "operator": "ge"}, {"budget": 0.25, "threshold": 1.250209093093872, "operator": "ge"}, {"budget": 0.26, "threshold": 1.1849827766418457, "operator": "ge"}, {"budget": 0.27, "threshold": 1.149476408958435, "operator": "ge"}, {"budget": 0.28, "threshold": 1.1168053150177002, "operator": "ge"}, {"budget": 0.29, "threshold": 1.0585103034973145, "operator": "ge"}, {"budget": 0.3, "threshold": 1.0321176052093506, "operator": "ge"}, {"budget": 0.31, "threshold": 0.9930316805839539, "operator": "ge"}, {"budget": 0.32, "threshold": 0.9493133425712585, "operator": "ge"}, {"budget": 0.33, "threshold": 0.9151377081871033, "operator": "ge"}, {"budget": 0.34, "threshold": 0.8753581643104553, "operator": "ge"}, {"budget": 0.35, "threshold": 0.8445616960525513, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7763196229934692, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7460830807685852, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7362204194068909, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6984133124351501, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6409314870834351, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5807450413703918, "operator": "ge"}, {"budget": 0.42, "threshold": 0.5317443609237671, "operator": "ge"}, {"budget": 0.43, "threshold": 0.46536245942115784, "operator": "ge"}, {"budget": 0.44, "threshold": 0.35390979051589966, "operator": "ge"}, {"budget": 0.45, "threshold": 0.10543298721313477, "operator": "ge"}, {"budget": 0.46, "threshold": -0.3055853247642517, "operator": "ge"}, {"budget": 0.47, "threshold": -0.6521859169006348, "operator": "ge"}, {"budget": 0.48, "threshold": -1.051513910293579, "operator": "ge"}, {"budget": 0.49, "threshold": -1.656231164932251, "operator": "ge"}, {"budget": 0.5, "threshold": -1.8259453773498535, "operator": "ge"}, {"budget": 0.51, "threshold": -1.9925074577331543, "operator": "ge"}, {"budget": 0.52, "threshold": -2.9266796112060547, "operator": "ge"}, {"budget": 0.53, "threshold": -3.132575750350952, "operator": "ge"}, {"budget": 0.54, "threshold": -3.2258245944976807, "operator": "ge"}, {"budget": 0.55, "threshold": -3.2987873554229736, "operator": "ge"}, {"budget": 0.56, "threshold": -3.36602783203125, "operator": "ge"}, {"budget": 0.57, "threshold": -3.468090534210205, "operator": "ge"}, {"budget": 0.58, "threshold": -3.536349058151245, "operator": "ge"}, {"budget": 0.59, "threshold": -3.698761463165283, "operator": "ge"}, {"budget": 0.6, "threshold": -3.826336145401001, "operator": "ge"}, {"budget": 0.61, "threshold": -3.8937110900878906, "operator": "ge"}, {"budget": 0.62, "threshold": -3.9727306365966797, "operator": "ge"}, {"budget": 0.63, "threshold": -4.013580799102783, "operator": "ge"}, {"budget": 0.64, "threshold": -4.08123254776001, "operator": "ge"}, {"budget": 0.65, "threshold": -4.130321502685547, "operator": "ge"}, {"budget": 0.66, "threshold": -4.221787452697754, "operator": "ge"}, {"budget": 0.67, "threshold": -4.299808025360107, "operator": "ge"}, {"budget": 0.68, "threshold": -4.402392387390137, "operator": "ge"}, {"budget": 0.69, "threshold": -4.433801174163818, "operator": "ge"}, {"budget": 0.7, "threshold": -4.507187843322754, "operator": "ge"}, {"budget": 0.71, "threshold": -4.544757843017578, "operator": "ge"}, {"budget": 0.72, "threshold": -4.592101573944092, "operator": "ge"}, {"budget": 0.73, "threshold": -4.611885070800781, "operator": "ge"}, {"budget": 0.74, "threshold": -4.661886692047119, "operator": "ge"}, {"budget": 0.75, "threshold": -4.709540367126465, "operator": "ge"}, {"budget": 0.76, "threshold": -4.760960102081299, "operator": "ge"}, {"budget": 0.77, "threshold": -4.820700168609619, "operator": "ge"}, {"budget": 0.78, "threshold": -4.860047340393066, "operator": "ge"}, {"budget": 0.79, "threshold": -4.906013011932373, "operator": "ge"}, {"budget": 0.8, "threshold": -5.005242347717285, "operator": "ge"}, {"budget": 0.81, "threshold": -5.139169216156006, "operator": "ge"}, {"budget": 0.82, "threshold": -13.320686340332031, "operator": "ge"}, {"budget": 0.83, "threshold": -13.398002624511719, "operator": "ge"}, {"budget": 0.84, "threshold": -13.448490142822266, "operator": "ge"}, {"budget": 0.85, "threshold": -13.472400665283203, "operator": "ge"}, {"budget": 0.86, "threshold": -13.54831600189209, "operator": "ge"}, {"budget": 0.87, "threshold": -13.693071365356445, "operator": "ge"}, {"budget": 0.88, "threshold": -13.867557525634766, "operator": "ge"}, {"budget": 0.89, "threshold": -14.077868461608887, "operator": "ge"}, {"budget": 0.9, "threshold": -14.164002418518066, "operator": "ge"}, {"budget": 0.91, "threshold": -14.323781967163086, "operator": "ge"}, {"budget": 0.92, "threshold": -14.417237281799316, "operator": "ge"}, {"budget": 0.93, "threshold": -14.492993354797363, "operator": "ge"}, {"budget": 0.94, "threshold": -14.574166297912598, "operator": "ge"}, {"budget": 0.95, "threshold": -14.628684997558594, "operator": "ge"}, {"budget": 0.96, "threshold": -14.695955276489258, "operator": "ge"}, {"budget": 0.97, "threshold": -14.788618087768555, "operator": "ge"}, {"budget": 0.98, "threshold": -14.883031845092773, "operator": "ge"}, {"budget": 0.99, "threshold": -14.939950942993164, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.53992216854536, 19.065032901239796, 19.452393792799256, 19.894755393003106, 20.429595884031645, 20.923594847775178, 21.000528476333333, 21.154330059824606, 21.376825876422327, 21.77578111425277, 21.992180343473446, 22.372408293460925, 23.19131892441501, 23.67557021904848, 24.265401366025863, 24.569788046071015, 24.578135169389924, 25.364816069792116, 25.716955133924145, 26.133558537051186, 26.389817290552585, 26.78094273297762, 27.14311792625046, 25.77709176069832, 26.105382528993644, 26.171309310844194, 26.484359434150228, 26.482371794871796, 26.631295533734555, 27.053897622311062, 27.151484120749636, 27.482069513742935, 27.61931929916745, 27.953126423257558, 28.549821983662255, 28.471049141421194, 29.445103318987275, 29.826861779194434, 29.793181197739198, 29.713237121544832, 30.22095275616402, 30.538768416479044, 32.281992810426466, 32.32723257924871, 32.81285807463818, 32.69493177387914, 32.59974700344241, 32.101422912119006, 32.0283937814345, 31.840739905256033, 32.30033777864701, 32.48909307244844, 32.35189795049722, 31.848317181770557, 32.214286406262104, 32.214286406262104, 32.214286406262104, 32.214286406262104, 32.05676039374294, 31.840900057811822, 31.63181267130508, 31.57076261025502, 31.57076261025502, 31.13182516408323, 30.772448535579873, 30.444932597591208, 30.182250205046557, 30.05050505050505, 30.31279178338002, 30.080196127316544, 29.470877127127125, 29.2711277436521, 28.92573696145125, 28.851925711062766, 28.68487506645402, 28.506462934429038, 28.463733978439866, 28.080549404463877, 27.756297018161046, 27.50612892472189, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.152805996181673, 27.152805996181673, 27.152805996181673, 27.08398536448072, 27.016508981214866, 27.016508981214866, 26.948359770310994, 26.87276921912989, 26.805281651050322, 26.805281651050322, 26.575274061687104, 26.50315867707172, 26.307303807303807, 26.178049175679508, 25.718607688310684, 25.483324348821334, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 24, 34, 45, 55, 63, 68, 77, 83, 90, 97, 104, 112, 121, 130, 138, 144, 153, 160, 168, 173, 179, 184, 190, 195, 199, 205, 209, 214, 220, 224, 230, 234, 241, 248, 253, 259, 265, 270, 276, 281, 286, 292, 295, 298, 299, 299, 300, 302, 302, 303, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 2, 2, 2, 3, 5, 7, 11, 11, 13, 13, 14, 14, 14, 16, 20, 20, 21, 23, 25, 25, 26, 29, 30, 32, 34, 36, 39, 40, 42, 46, 49, 51, 52, 56, 59, 61, 66, 72, 73, 76, 81, 84, 84, 85, 89, 92, 94, 95, 96, 100, 102, 106, 108, 108, 108, 108, 110, 113, 116, 117, 117, 121, 124, 129, 131, 133, 136, 144, 150, 153, 160, 161, 164, 168, 169, 174, 177, 181, 186, 186, 187, 190, 190, 190, 191, 192, 196, 197, 199, 200, 200, 203, 205, 208, 210, 216, 219, 220], "exact_score": [18.254072168965788, 18.53992216854536, 19.065032901239796, 19.452393792799256, 19.894755393003106, 20.429595884031645, 20.923594847775178, 21.000528476333333, 21.154330059824606, 21.376825876422327, 21.77578111425277, 21.992180343473446, 22.372408293460925, 23.19131892441501, 23.67557021904848, 24.265401366025863, 24.569788046071015, 24.578135169389924, 25.364816069792116, 25.716955133924145, 26.133558537051186, 26.389817290552585, 26.78094273297762, 27.14311792625046, 25.77709176069832, 26.105382528993644, 26.171309310844194, 26.484359434150228, 26.482371794871796, 26.631295533734555, 27.053897622311062, 27.151484120749636, 27.482069513742935, 27.61931929916745, 27.953126423257558, 28.549821983662255, 28.471049141421194, 29.445103318987275, 29.826861779194434, 29.793181197739198, 29.713237121544832, 30.22095275616402, 30.538768416479044, 32.281992810426466, 32.32723257924871, 32.81285807463818, 32.69493177387914, 32.59974700344241, 32.101422912119006, 32.0283937814345, 31.840739905256033, 32.30033777864701, 32.48909307244844, 32.35189795049722, 31.848317181770557, 32.214286406262104, 32.214286406262104, 32.214286406262104, 32.214286406262104, 32.05676039374294, 31.840900057811822, 31.63181267130508, 31.57076261025502, 31.57076261025502, 31.13182516408323, 30.772448535579873, 30.444932597591208, 30.182250205046557, 30.05050505050505, 30.31279178338002, 30.080196127316544, 29.470877127127125, 29.2711277436521, 28.92573696145125, 28.851925711062766, 28.68487506645402, 28.506462934429038, 28.463733978439866, 28.080549404463877, 27.756297018161046, 27.50612892472189, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.152805996181673, 27.152805996181673, 27.152805996181673, 27.08398536448072, 27.016508981214866, 27.016508981214866, 26.948359770310994, 26.87276921912989, 26.805281651050322, 26.805281651050322, 26.575274061687104, 26.50315867707172, 26.307303807303807, 26.178049175679508, 25.718607688310684, 25.483324348821334, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.615050636784094, "threshold_area100": 27.618120744428538, "threshold_area25": 22.956812850068236}, "test": {"score": [18.129624718318375, 18.64321850184275, 18.611710787550447, 19.12870765090125, 19.502487909126337, 19.832473295729777, 20.044274449164508, 20.334026157894964, 20.72652133438814, 21.0526886344877, 20.567804194809614, 20.600359730229766, 20.74829528957154, 20.965511267235406, 21.471095533240288, 21.761195615440236, 21.979118056256556, 22.210539252088566, 22.514497307990627, 22.71564444000634, 23.038650846215294, 23.158450077552633, 23.318813572664517, 23.442932104363344, 23.687089208342112, 23.85161793499526, 24.828117515813325, 25.014745914435828, 25.19518527138142, 25.971706142067184, 26.77037304686066, 26.99129262252764, 27.199351015305172, 27.36953551912568, 27.632669106756268, 28.067498827851555, 28.294463032484124, 28.47886717117448, 28.48813706226706, 28.811829250956905, 29.176647369566066, 29.568458281283622, 29.96628431548788, 30.284042261197747, 30.106632900952214, 29.94456979113117, 29.930849323443002, 29.914556425102333, 29.522434026750616, 29.844461278398533, 29.746270007566235, 29.61237830982438, 29.834312227910896, 29.618476888127955, 29.503205105329915, 29.448663381528757, 29.347980411682332, 29.47870198732498, 29.58075876036269, 29.811699480140913, 29.747659082885125, 29.699484850745, 29.54075505734676, 29.498222817064345, 29.283255341023377, 29.30699307955219, 29.35446473725994, 29.076735385712617, 29.178106602951136, 29.140269934643186, 28.826479773714002, 29.055148685546868, 28.88815371102827, 28.76980085794129, 28.847316418264818, 28.863311092377263, 28.577838182489344, 28.378199682940714, 28.15982340864692, 28.09664198042659, 27.792026493367278, 27.84367961397069, 27.75925486404407, 27.725671019760128, 27.629808458936715, 27.629808458936715, 27.570588509506187, 27.459028456142313, 27.40945092565315, 27.351276174805587, 27.351276174805587, 27.30094327938783, 27.255790610219645, 27.41849900331912, 27.3364569189965, 27.27530138146116, 27.193781206681905, 27.11943254316816, 27.0966772762403, 27.348973275429096, 27.001009067886244], "calls": [0.0, 0.8988015978695073, 1.7976031957390146, 3.2290279627163785, 4.161118508655127, 5.226364846870839, 6.291611185086551, 7.157123834886818, 8.388814913448735, 9.454061251664447, 10.119840213049267, 10.952063914780293, 11.950732356857523, 12.849533954727029, 14.347536617842877, 15.745672436750999, 16.644474034620508, 17.509986684420774, 18.64181091877497, 19.67376830892144, 20.70572569906791, 21.171770972037287, 21.870838881491345, 22.303595206391478, 23.501997336884152, 24.467376830892142, 26.264980026631157, 27.030625832223702, 27.996005326231693, 29.560585885486017, 30.525965379494007, 31.591211717709722, 33.08921438082557, 33.95472703062583, 34.92010652463382, 35.719041278295606, 37.11717709720373, 37.982689747003995, 38.18242343541944, 39.21438082556591, 40.47936085219707, 41.24500665778961, 42.01065246338216, 43.142476697736356, 43.70838881491345, 44.84021304926764, 45.372836218375504, 46.03861517976032, 46.6711051930759, 48.83488681757657, 49.500665778961384, 50.19973368841545, 51.33155792276965, 52.46338215712384, 53.59520639147804, 54.89347536617842, 56.25832223701731, 57.75632490013316, 58.655126498002666, 60.58588548601864, 62.11717709720372, 62.782956058588546, 63.78162450066578, 64.4474034620506, 65.5792276964048, 66.24500665778962, 67.64314247669773, 68.87483355525966, 71.00532623169109, 71.5046604527297, 72.70306258322236, 73.5352862849534, 74.30093209054593, 74.73368841544608, 75.69906790945407, 76.49800266311586, 77.69640479360852, 78.69507323568575, 79.39414114513981, 79.82689747003995, 80.99201065246339, 81.99067909454061, 82.39014647137151, 84.02130492676432, 85.7190412782956, 86.21837549933421, 87.68308921438083, 89.24766977363515, 90.11318242343542, 90.9121171770972, 91.44474034620505, 92.21038615179761, 92.84287616511318, 93.44207723035952, 94.27430093209055, 94.8069241011984, 96.13848202396804, 97.07057256990679, 97.86950732356857, 98.43541944074568, 100.0], "rescued": [0, 52, 99, 166, 211, 253, 289, 319, 362, 400, 417, 439, 469, 496, 542, 584, 610, 630, 661, 689, 712, 723, 737, 747, 776, 797, 842, 862, 883, 922, 943, 966, 998, 1018, 1038, 1058, 1086, 1108, 1112, 1133, 1159, 1175, 1191, 1213, 1219, 1234, 1237, 1244, 1245, 1252, 1252, 1253, 1255, 1255, 1255, 1256, 1256, 1257, 1259, 1263, 1264, 1265, 1265, 1266, 1266, 1267, 1268, 1268, 1271, 1271, 1271, 1273, 1273, 1273, 1275, 1276, 1276, 1276, 1277, 1277, 1277, 1278, 1278, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1280, 1280, 1280, 1280, 1280, 1280, 1282, 1282], "harmed": [0, 1, 7, 11, 14, 20, 25, 28, 36, 44, 48, 55, 62, 67, 71, 79, 85, 89, 93, 103, 108, 110, 114, 114, 119, 125, 140, 146, 153, 165, 170, 178, 190, 200, 210, 221, 240, 254, 257, 267, 276, 283, 288, 300, 308, 316, 318, 324, 328, 338, 342, 347, 353, 366, 373, 379, 389, 398, 401, 417, 424, 432, 446, 454, 467, 471, 486, 501, 530, 533, 547, 557, 570, 577, 589, 594, 610, 626, 640, 645, 667, 680, 684, 688, 693, 693, 697, 707, 711, 715, 715, 718, 720, 725, 730, 733, 738, 747, 749, 751, 777], "exact_score": [18.129624718318375, 18.584052194177175, 18.674604662760547, 19.024519816336966, 19.427841605614002, 19.758652385603785, 19.94290075582252, 20.269741765615738, 20.62952378560638, 20.925289082545966, 21.072878080628477, 20.600359730229766, 20.757071874459488, 21.01483702946001, 21.32611661665049, 21.544517356291017, 21.805292393545717, 22.0781732631752, 22.34013327242702, 22.558706424962537, 22.913391225364258, 23.09979539190953, 23.329857117472503, 23.56110807182852, 23.808405884188897, 23.972108836729777, 24.5233072048998, 24.998773387182705, 25.19518527138142, 25.819082299450347, 26.051277541837447, 26.858747093306278, 27.074288445248364, 27.199916601249218, 27.384615384615387, 27.61765436500983, 28.151979843366636, 28.23692402306075, 28.47886717117448, 28.7294198050012, 29.010230744385673, 29.453728461081408, 29.95651604444134, 30.2920858850977, 30.130286380286382, 29.98263974088433, 29.895252215945423, 29.896668597519938, 29.877463532155158, 29.82539809757065, 29.65272926259509, 29.860608986835363, 29.735834674779007, 29.60584584101415, 29.511939626974076, 29.436424478134498, 29.379155147446024, 29.53353020523745, 29.47870198732498, 29.53599614545586, 29.769244406360617, 29.832667608585677, 29.747659082885125, 29.680934119873086, 29.558570848699194, 29.408269122536566, 29.35056606430165, 29.136842932502997, 29.28905098945544, 29.034875836810812, 29.080626522464875, 29.188252606415627, 28.985003758410073, 28.861125184654597, 28.935385904491334, 29.01831655880675, 28.91510921480569, 28.824688654618726, 28.54302107352394, 28.25891300873821, 28.09664198042659, 27.792026493367278, 27.84367961397069, 27.711781111087515, 27.725671019760128, 27.664942352030796, 27.629808458936715, 27.605954017448354, 27.570588509506187, 27.489917233315076, 27.416702679565724, 27.351276174805587, 27.30094327938783, 27.2486181099194, 27.352362838050766, 27.252342798963287, 27.193781206681905, 27.12647581176894, 27.127002945545094, 27.260838692188976, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.59248986723568, "threshold_area100": 26.58984579404702, "threshold_area25": 21.23814905577035}, "selected_budget": 45, "selected": {"val": {"score": 32.81285807463818, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.94456979113117, "calls": 44.84021304926764, "rescued": 1234, "harmed": 316}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.5642495155334473, "operator": "ge"}, {"budget": 0.02, "threshold": 1.4615094661712646, "operator": "ge"}, {"budget": 0.03, "threshold": 1.409281611442566, "operator": "ge"}, {"budget": 0.04, "threshold": 1.3789288997650146, "operator": "ge"}, {"budget": 0.05, "threshold": 1.3178099393844604, "operator": "ge"}, {"budget": 0.06, "threshold": 1.281389594078064, "operator": "ge"}, {"budget": 0.07, "threshold": 1.2357102632522583, "operator": "ge"}, {"budget": 0.08, "threshold": 1.216941237449646, "operator": "ge"}, {"budget": 0.09, "threshold": 1.2006967067718506, "operator": "ge"}, {"budget": 0.1, "threshold": 1.1567953824996948, "operator": "ge"}, {"budget": 0.11, "threshold": 1.1266307830810547, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0955356359481812, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0743920803070068, "operator": "ge"}, {"budget": 0.14, "threshold": 1.0530091524124146, "operator": "ge"}, {"budget": 0.15, "threshold": 1.0298537015914917, "operator": "ge"}, {"budget": 0.16, "threshold": 0.998663067817688, "operator": "ge"}, {"budget": 0.17, "threshold": 0.9663087129592896, "operator": "ge"}, {"budget": 0.18, "threshold": 0.9263192415237427, "operator": "ge"}, {"budget": 0.19, "threshold": 0.885640561580658, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8675780296325684, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8395991921424866, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8009056448936462, "operator": "ge"}, {"budget": 0.23, "threshold": 0.7914966344833374, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7645233869552612, "operator": "ge"}, {"budget": 0.25, "threshold": 0.7450307011604309, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7306718826293945, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7169496417045593, "operator": "ge"}, {"budget": 0.28, "threshold": 0.6917357444763184, "operator": "ge"}, {"budget": 0.29, "threshold": 0.6791010499000549, "operator": "ge"}, {"budget": 0.3, "threshold": 0.6437761783599854, "operator": "ge"}, {"budget": 0.31, "threshold": 0.6163479089736938, "operator": "ge"}, {"budget": 0.32, "threshold": 0.5927486419677734, "operator": "ge"}, {"budget": 0.33, "threshold": 0.5785368084907532, "operator": "ge"}, {"budget": 0.34, "threshold": 0.5600566864013672, "operator": "ge"}, {"budget": 0.35, "threshold": 0.5363327860832214, "operator": "ge"}, {"budget": 0.36, "threshold": 0.5035138726234436, "operator": "ge"}, {"budget": 0.37, "threshold": 0.479546457529068, "operator": "ge"}, {"budget": 0.38, "threshold": 0.4566110372543335, "operator": "ge"}, {"budget": 0.39, "threshold": 0.43856295943260193, "operator": "ge"}, {"budget": 0.4, "threshold": 0.39246562123298645, "operator": "ge"}, {"budget": 0.41, "threshold": 0.3775423765182495, "operator": "ge"}, {"budget": 0.42, "threshold": 0.34546375274658203, "operator": "ge"}, {"budget": 0.43, "threshold": 0.2771179676055908, "operator": "ge"}, {"budget": 0.44, "threshold": 0.18464389443397522, "operator": "ge"}, {"budget": 0.45, "threshold": 0.036953091621398926, "operator": "ge"}, {"budget": 0.46, "threshold": -0.15086477994918823, "operator": "ge"}, {"budget": 0.47, "threshold": -0.421322226524353, "operator": "ge"}, {"budget": 0.48, "threshold": -0.5889834761619568, "operator": "ge"}, {"budget": 0.49, "threshold": -0.939889669418335, "operator": "ge"}, {"budget": 0.5, "threshold": -1.2150077819824219, "operator": "ge"}, {"budget": 0.51, "threshold": -1.5427449941635132, "operator": "ge"}, {"budget": 0.52, "threshold": -2.6551175117492676, "operator": "ge"}, {"budget": 0.53, "threshold": -2.882021903991699, "operator": "ge"}, {"budget": 0.54, "threshold": -2.9397923946380615, "operator": "ge"}, {"budget": 0.55, "threshold": -3.0650432109832764, "operator": "ge"}, {"budget": 0.56, "threshold": -3.1190080642700195, "operator": "ge"}, {"budget": 0.57, "threshold": -3.1806583404541016, "operator": "ge"}, {"budget": 0.58, "threshold": -3.2552661895751953, "operator": "ge"}, {"budget": 0.59, "threshold": -3.3568572998046875, "operator": "ge"}, {"budget": 0.6, "threshold": -3.4575979709625244, "operator": "ge"}, {"budget": 0.61, "threshold": -3.550027847290039, "operator": "ge"}, {"budget": 0.62, "threshold": -3.581683397293091, "operator": "ge"}, {"budget": 0.63, "threshold": -3.6477293968200684, "operator": "ge"}, {"budget": 0.64, "threshold": -3.693671941757202, "operator": "ge"}, {"budget": 0.65, "threshold": -3.7449398040771484, "operator": "ge"}, {"budget": 0.66, "threshold": -3.766817569732666, "operator": "ge"}, {"budget": 0.67, "threshold": -3.7928659915924072, "operator": "ge"}, {"budget": 0.68, "threshold": -3.8215785026550293, "operator": "ge"}, {"budget": 0.69, "threshold": -3.8873236179351807, "operator": "ge"}, {"budget": 0.7, "threshold": -3.9177069664001465, "operator": "ge"}, {"budget": 0.71, "threshold": -3.948049306869507, "operator": "ge"}, {"budget": 0.72, "threshold": -3.9986841678619385, "operator": "ge"}, {"budget": 0.73, "threshold": -4.061189651489258, "operator": "ge"}, {"budget": 0.74, "threshold": -4.1003313064575195, "operator": "ge"}, {"budget": 0.75, "threshold": -4.145229816436768, "operator": "ge"}, {"budget": 0.76, "threshold": -4.185693740844727, "operator": "ge"}, {"budget": 0.77, "threshold": -4.263922691345215, "operator": "ge"}, {"budget": 0.78, "threshold": -4.284391403198242, "operator": "ge"}, {"budget": 0.79, "threshold": -4.357933044433594, "operator": "ge"}, {"budget": 0.8, "threshold": -4.459328651428223, "operator": "ge"}, {"budget": 0.81, "threshold": -4.578476905822754, "operator": "ge"}, {"budget": 0.82, "threshold": -11.485343933105469, "operator": "ge"}, {"budget": 0.83, "threshold": -11.530786514282227, "operator": "ge"}, {"budget": 0.84, "threshold": -11.561084747314453, "operator": "ge"}, {"budget": 0.85, "threshold": -11.606943130493164, "operator": "ge"}, {"budget": 0.86, "threshold": -11.63900089263916, "operator": "ge"}, {"budget": 0.87, "threshold": -11.785499572753906, "operator": "ge"}, {"budget": 0.88, "threshold": -11.861824989318848, "operator": "ge"}, {"budget": 0.89, "threshold": -11.956964492797852, "operator": "ge"}, {"budget": 0.9, "threshold": -11.995750427246094, "operator": "ge"}, {"budget": 0.91, "threshold": -12.011837005615234, "operator": "ge"}, {"budget": 0.92, "threshold": -12.064563751220703, "operator": "ge"}, {"budget": 0.93, "threshold": -12.10456657409668, "operator": "ge"}, {"budget": 0.94, "threshold": -12.246570587158203, "operator": "ge"}, {"budget": 0.95, "threshold": -12.391814231872559, "operator": "ge"}, {"budget": 0.96, "threshold": -12.617250442504883, "operator": "ge"}, {"budget": 0.97, "threshold": -12.723514556884766, "operator": "ge"}, {"budget": 0.98, "threshold": -12.873955726623535, "operator": "ge"}, {"budget": 0.99, "threshold": -12.932849884033203, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.440056184417088, 18.734978294970958, 19.067830557440875, 19.315045666563496, 19.63662543250021, 20.14578558696206, 20.27927093577246, 20.703122464741654, 21.088310881986434, 21.21430939037778, 21.535197294320003, 21.774558424422352, 22.005378278465535, 22.287827829322087, 22.583108715184185, 23.19792566983578, 24.023517201175366, 24.346153846153847, 24.731276472041426, 25.023624128815186, 25.129652585890668, 25.321386537478364, 25.48707125870943, 25.853768299522216, 26.230488730488734, 26.527792169361632, 26.87656171914043, 27.0050913719036, 27.359512041120347, 27.923847588674615, 28.517875798294817, 27.47282608695652, 27.525307608893236, 27.569217304882482, 28.68713422592733, 29.339523165588073, 29.527015492788795, 29.55340247062023, 29.75318834580898, 29.970328740633306, 30.267597588557226, 30.740794293425864, 30.822739839288936, 32.45950771152384, 32.81285807463818, 32.69493177387914, 32.417701264450486, 32.101422912119006, 32.0283937814345, 31.905241072074897, 32.645865771631385, 32.574663239985824, 32.10033331080282, 31.783773291925467, 32.214286406262104, 32.214286406262104, 32.214286406262104, 31.989933342152455, 31.989933342152455, 31.77705940408353, 31.77705940408353, 31.511039724445176, 31.257476076555022, 31.14895735707591, 30.85189306218942, 30.483405483405484, 30.11204481792717, 30.01777634130575, 29.682539682539684, 29.273317820438237, 29.52896992984258, 29.11315331669314, 28.815576186265844, 28.61892849697728, 28.50090084263001, 28.132547592680485, 27.903703446974514, 27.824380337508618, 27.544838602141247, 27.473230561940237, 27.434520884520886, 27.434520884520886, 27.292501606798545, 27.292501606798545, 27.222308020275708, 27.152805996181673, 27.08532961291582, 27.016508981214866, 26.948359770310994, 26.880872202231426, 26.880872202231426, 26.71651433787356, 26.71651433787356, 26.612183510506004, 26.612183510506004, 26.316192865105908, 26.070064687085964, 25.87792977796992, 25.668767695229906, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 9, 19, 28, 37, 45, 56, 61, 70, 78, 84, 91, 98, 104, 113, 120, 131, 138, 146, 154, 161, 166, 174, 179, 186, 192, 197, 203, 208, 214, 220, 226, 231, 235, 241, 249, 257, 261, 266, 272, 277, 281, 287, 291, 296, 298, 299, 299, 300, 302, 302, 304, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 2, 2, 4, 4, 5, 10, 10, 10, 12, 12, 13, 14, 17, 18, 19, 20, 21, 23, 25, 27, 31, 32, 34, 36, 37, 39, 42, 43, 46, 46, 49, 52, 56, 58, 59, 63, 66, 70, 74, 77, 79, 82, 84, 84, 85, 90, 92, 94, 95, 98, 99, 105, 107, 108, 108, 108, 111, 111, 114, 114, 118, 122, 124, 128, 131, 135, 137, 140, 145, 150, 155, 159, 165, 167, 171, 175, 177, 180, 181, 186, 186, 188, 188, 189, 190, 191, 192, 193, 194, 194, 196, 196, 198, 202, 207, 210, 213, 216, 220], "exact_score": [18.254072168965788, 18.440056184417088, 18.734978294970958, 19.067830557440875, 19.315045666563496, 19.63662543250021, 20.14578558696206, 20.27927093577246, 20.703122464741654, 21.088310881986434, 21.21430939037778, 21.535197294320003, 21.774558424422352, 22.005378278465535, 22.287827829322087, 22.583108715184185, 23.19792566983578, 24.023517201175366, 24.346153846153847, 24.731276472041426, 25.023624128815186, 25.129652585890668, 25.321386537478364, 25.48707125870943, 25.853768299522216, 26.230488730488734, 26.527792169361632, 26.87656171914043, 27.0050913719036, 27.359512041120347, 27.923847588674615, 28.517875798294817, 27.47282608695652, 27.525307608893236, 27.569217304882482, 28.68713422592733, 29.339523165588073, 29.527015492788795, 29.55340247062023, 29.75318834580898, 29.970328740633306, 30.267597588557226, 30.740794293425864, 30.822739839288936, 32.45950771152384, 32.81285807463818, 32.69493177387914, 32.417701264450486, 32.101422912119006, 32.0283937814345, 31.905241072074897, 32.645865771631385, 32.574663239985824, 32.10033331080282, 31.783773291925467, 32.214286406262104, 32.214286406262104, 32.214286406262104, 31.989933342152455, 31.989933342152455, 31.77705940408353, 31.77705940408353, 31.511039724445176, 31.257476076555022, 31.14895735707591, 30.85189306218942, 30.483405483405484, 30.11204481792717, 30.01777634130575, 29.682539682539684, 29.273317820438237, 29.52896992984258, 29.11315331669314, 28.815576186265844, 28.61892849697728, 28.50090084263001, 28.132547592680485, 27.903703446974514, 27.824380337508618, 27.544838602141247, 27.473230561940237, 27.434520884520886, 27.434520884520886, 27.292501606798545, 27.292501606798545, 27.222308020275708, 27.152805996181673, 27.08532961291582, 27.016508981214866, 26.948359770310994, 26.880872202231426, 26.880872202231426, 26.71651433787356, 26.71651433787356, 26.612183510506004, 26.612183510506004, 26.316192865105908, 26.070064687085964, 25.87792977796992, 25.668767695229906, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.4190927397474, "threshold_area100": 27.42175044800579, "threshold_area25": 22.139989623320925}, "test": {"score": [18.129624718318375, 18.52198378384991, 18.839118964433467, 18.835850267878236, 18.979762153598667, 19.168867603153355, 19.41362412829557, 19.651512444052415, 19.786443748568168, 20.01057403086436, 20.4045276685764, 20.710755609247798, 20.934589181970324, 21.165013461464106, 21.25920266505305, 21.62290212490103, 21.90404031476628, 22.17508170772684, 22.692127661234498, 22.654409277984975, 22.98597354262047, 23.210859074861638, 23.631223285000853, 23.720520224585268, 24.702789687012725, 24.742910731466935, 25.01940912872767, 25.158207264541417, 25.246174901690573, 25.43425269781581, 25.84609567063147, 25.899838727726166, 26.527398795782208, 26.807534930504783, 26.91265138178282, 27.367606570472912, 27.61105449173461, 27.979027023781534, 28.197242041910688, 28.490403069901642, 29.1134299835461, 29.63532691136146, 30.00335023070184, 30.160254358449095, 30.267238540011533, 29.94456979113117, 29.94439002291685, 29.88178653419719, 29.523954675288195, 29.88000217008051, 30.150419410443625, 30.036445884422115, 29.834312227910896, 29.618476888127955, 29.60584584101415, 29.424249153459176, 29.323818527055334, 29.627633149651334, 29.525886731107903, 29.742842699848165, 29.690304162001674, 29.44525975399032, 29.337177537755966, 29.175922027470335, 29.008837618910334, 29.123455931368287, 28.9972510176294, 28.848902638838304, 28.72522641757751, 28.67243907520797, 28.608977027387283, 28.815534400629026, 28.85424773255838, 28.6060690014158, 28.447482718483197, 28.212041490331597, 28.22952784471362, 28.08314689561737, 28.074929246400153, 27.864531155498206, 27.727457614753288, 27.795211566550094, 27.66216040529016, 27.63823744764999, 27.614378198101825, 27.539927345719406, 27.539927345719406, 27.400521487330842, 27.39326973341827, 27.506637277618427, 27.506637277618427, 27.47878988134998, 27.46447101007663, 27.45735062590707, 27.33832492252971, 27.192692809098578, 27.09571689380404, 27.010784971256, 27.170826112727255, 27.137183933714397, 27.001009067886244], "calls": [0.0, 0.7989347536617843, 1.7310252996005324, 2.62982689747004, 2.9960053262316912, 4.2942743009320905, 5.292942743009322, 6.624500665778961, 7.023968042609853, 7.55659121171771, 8.85486018641811, 10.053262316910786, 11.351531291611185, 12.050599201065246, 12.849533954727029, 13.84820239680426, 15.179760319573901, 16.444740346205062, 18.07589880159787, 19.873501997336884, 20.605858854860188, 21.80426098535286, 23.33555259653795, 23.834886817576564, 24.766977363515313, 25.432756324900136, 26.098535286284953, 26.73102529960053, 27.563249001331556, 28.095872170439414, 29.66045272969374, 30.725699067909456, 31.957390146471372, 32.65645805592543, 33.45539280958722, 34.58721704394141, 35.885486018641814, 37.150466045272964, 37.949400798934754, 38.61517976031957, 40.07989347536618, 40.57922769640479, 41.344873501997334, 43.00932090545938, 43.80825565912117, 44.87350199733689, 45.372836218375504, 46.105193075898804, 46.504660452729695, 48.33555259653795, 49.30093209054594, 50.23302263648469, 51.398135818908116, 52.46338215712384, 52.79627163781625, 55.05992010652463, 56.35818908122503, 57.65645805592543, 58.655126498002666, 60.31957390146472, 61.91744340878829, 63.315579227696404, 63.914780292942744, 65.0133155792277, 65.84553928095872, 67.07723035952064, 67.50998668442078, 68.24234354194407, 68.87483355525966, 70.83888149134488, 71.60452729693742, 72.13715046604527, 73.36884154460719, 74.83355525965379, 75.79893475366178, 76.66444740346205, 77.62982689747004, 78.92809587217043, 79.16111850865512, 80.32623169107856, 81.15845539280959, 81.95739014647138, 83.15579227696405, 84.32090545938749, 85.18641810918774, 86.45139813581892, 86.8508655126498, 89.14780292942743, 90.04660452729694, 91.54460719041279, 91.94407456724367, 92.34354194407457, 92.87616511318242, 93.20905459387482, 94.6404793608522, 95.90545938748336, 96.93741677762982, 97.53661784287615, 98.83488681757656, 99.10119840213049, 100.0], "rescued": [0, 36, 79, 107, 124, 170, 198, 246, 263, 289, 334, 371, 411, 435, 461, 494, 527, 568, 620, 673, 691, 724, 765, 779, 805, 819, 837, 853, 868, 884, 919, 940, 970, 988, 1009, 1039, 1065, 1092, 1107, 1122, 1154, 1165, 1185, 1210, 1221, 1234, 1238, 1243, 1244, 1250, 1253, 1253, 1255, 1255, 1255, 1256, 1256, 1258, 1259, 1262, 1264, 1265, 1265, 1265, 1266, 1267, 1267, 1267, 1267, 1270, 1270, 1271, 1272, 1273, 1274, 1274, 1275, 1277, 1277, 1277, 1277, 1278, 1278, 1278, 1278, 1278, 1278, 1279, 1279, 1280, 1280, 1280, 1280, 1280, 1280, 1280, 1280, 1280, 1282, 1282, 1282], "harmed": [0, 1, 5, 11, 11, 20, 22, 34, 36, 38, 43, 48, 54, 57, 61, 68, 74, 83, 91, 102, 108, 117, 126, 131, 134, 143, 149, 153, 161, 165, 174, 181, 194, 202, 213, 227, 239, 254, 260, 265, 279, 281, 284, 304, 310, 316, 318, 325, 327, 336, 340, 344, 353, 366, 367, 381, 391, 398, 405, 415, 430, 445, 452, 468, 482, 492, 501, 511, 518, 547, 552, 557, 567, 586, 598, 612, 627, 644, 645, 660, 670, 682, 688, 690, 692, 698, 698, 711, 712, 720, 720, 721, 723, 724, 731, 745, 752, 759, 766, 768, 777], "exact_score": [18.129624718318375, 18.59039698993343, 18.931688849882853, 18.979762153598667, 19.09916504711257, 19.322544641061988, 19.662535331481436, 19.774676199017243, 20.1138461045091, 20.451704517434084, 20.690447484547043, 20.94527132532559, 21.14716692051248, 21.289464296322024, 21.66350438289804, 21.866646436188923, 22.0396453365502, 22.34146463374706, 22.633215237908836, 22.782737148774228, 22.66539513406147, 23.085885002386824, 23.265832031032726, 23.518627756900468, 23.716123614822298, 24.704086889629494, 25.023645508720133, 25.22587056779541, 25.423577994332707, 25.6783811786332, 25.90339796821815, 25.924020555265017, 26.542533289946846, 26.89684461789237, 26.83896156576412, 27.446216253685847, 27.61452886007982, 27.86187227700069, 28.197242041910688, 28.609848887536828, 29.09534321820111, 29.806421228785258, 30.126356420461082, 30.141458429687688, 30.349971849266527, 29.98263974088433, 29.935025466550396, 29.981144311615182, 29.979805326905527, 29.87907986767316, 30.09007620665647, 29.874882555597654, 29.67053883488953, 29.586506529207142, 29.446900729202536, 29.436424478134498, 29.361862424940462, 29.285954293386908, 29.549583890060386, 29.84284224162939, 29.753449992886615, 29.648084147542498, 29.690304162001674, 29.49763677204148, 29.337177537755966, 29.175922027470335, 29.008837618910334, 29.133832576997854, 28.888470035303616, 28.667988673828205, 28.741128982307533, 28.654879187082273, 28.82479491470049, 28.91974161539956, 28.756895096259193, 28.587149152774593, 28.37810184179122, 28.17769141970934, 28.196035893233752, 28.08314689561737, 27.92601775062986, 27.73521369154927, 27.795211566550094, 27.706173322822313, 27.645791326127405, 27.621903837781197, 27.563632228887236, 27.539927345719406, 27.49950925936382, 27.407799951202577, 27.39326973341827, 27.557379054960652, 27.506637277618427, 27.46447101007663, 27.415631452552034, 27.315333005841865, 27.192692809098578, 27.09571689380404, 26.97136831271906, 27.170826112727255, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.522314632721006, "threshold_area100": 26.455228812180177, "threshold_area25": 21.123707732828784}, "selected_budget": 45, "selected": {"val": {"score": 32.81285807463818, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.94456979113117, "calls": 44.87350199733689, "rescued": 1234, "harmed": 316}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 32.81285807463818, "calls": 44.935805991440795}, "test": {"score": 29.925601605640118, "calls": 44.77363515312916}}, "old_normalized_interior": {"val": {"score": 32.81285807463818, "calls": 44.935805991440795}, "test": {"score": 29.94456979113117, "calls": 44.87350199733689}}, "old_normalized_endpoints": {"val": {"score": 32.81285807463818, "calls": 44.935805991440795}, "test": {"score": 29.94456979113117, "calls": 44.87350199733689}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.570977475658456, 18.939804972392004, 19.532085561497325, 20.007060092011546, 20.41223481384454, 20.68854291844614, 21.108170766781008, 21.47239478006324, 21.574214206856478, 22.49433231284411, 22.878220334050724, 23.03179988755957, 23.412178370768306, 23.81782317823178, 24.035336356764926, 24.154125197463884, 24.404939614299217, 24.770447713969308, 25.06962233670308, 25.249003984063744, 25.488080612246662, 25.809940997675668, 26.123945805415232, 26.247801446159862, 26.650873115203527, 27.127116308183624, 27.35608202090905, 27.60619080547289, 27.878231492361927, 28.27911031235839, 27.05717893217893, 27.205111402359105, 27.640250137628865, 27.9032521653865, 28.655804191837543, 29.377535572136697, 29.46077426275446, 29.86853866647381, 29.754638757627845, 30.09374854536145, 30.12972050990484, 30.740794293425864, 30.822739839288936, 32.45950771152384, 32.81285807463818, 32.69493177387914, 32.417701264450486, 32.101422912119006, 32.002077423572956, 31.905241072074897, 31.819116101773165, 32.401866684523746, 32.20173026624639, 32.015185025353325, 31.78372124676739, 31.70178231283604, 32.15596429187511, 32.15596429187511, 32.08651984743066, 32.08651984743066, 31.98571339581776, 31.80994918297595, 31.490182376436916, 31.490182376436916, 31.424064245376094, 31.795603879849814, 31.643170813600058, 31.483596345514954, 31.272927007547423, 31.036184583305005, 30.894627960990316, 30.530024509803923, 30.941514376442754, 30.866439301367677, 30.80074412697685, 30.374922955568117, 30.240884535397416, 30.030914850578217, 29.970607833034357, 29.838478576948468, 29.640065878535772, 29.442399946707198, 29.25854700854701, 29.079808348904045, 28.72194854953476, 28.152576334312666, 27.878680293286536, 27.663268302162326, 27.663268302162326, 27.506158832378347, 27.301661752881266, 27.021117138428913, 26.914328312562123, 26.57322352444304, 26.506000357289228, 26.379173042690734, 26.174861635940307, 25.65261473292303, 25.580499348307644, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.570977475658456, 18.939804972392004, 19.532085561497325, 20.007060092011546, 20.41223481384454, 20.68854291844614, 21.108170766781008, 21.47239478006324, 21.574214206856478, 22.49433231284411, 22.878220334050724, 23.03179988755957, 23.412178370768306, 23.81782317823178, 24.035336356764926, 24.154125197463884, 24.404939614299217, 24.770447713969308, 25.06962233670308, 25.249003984063744, 25.488080612246662, 25.809940997675668, 26.123945805415232, 26.247801446159862, 26.650873115203527, 27.127116308183624, 27.35608202090905, 27.60619080547289, 27.878231492361927, 28.27911031235839, 27.05717893217893, 27.205111402359105, 27.640250137628865, 27.9032521653865, 28.655804191837543, 29.377535572136697, 29.46077426275446, 29.86853866647381, 29.754638757627845, 30.09374854536145, 30.12972050990484, 30.740794293425864, 30.822739839288936, 32.45950771152384, 32.81285807463818, 32.69493177387914, 32.417701264450486, 32.101422912119006, 32.002077423572956, 31.905241072074897, 31.819116101773165, 32.401866684523746, 32.20173026624639, 32.015185025353325, 31.78372124676739, 31.70178231283604, 32.15596429187511, 32.15596429187511, 32.08651984743066, 32.08651984743066, 31.98571339581776, 31.80994918297595, 31.490182376436916, 31.490182376436916, 31.424064245376094, 31.795603879849814, 31.643170813600058, 31.483596345514954, 31.272927007547423, 31.036184583305005, 30.894627960990316, 30.530024509803923, 30.941514376442754, 30.866439301367677, 30.80074412697685, 30.374922955568117, 30.240884535397416, 30.030914850578217, 29.970607833034357, 29.838478576948468, 29.640065878535772, 29.442399946707198, 29.25854700854701, 29.079808348904045, 28.72194854953476, 28.152576334312666, 27.878680293286536, 27.663268302162326, 27.663268302162326, 27.506158832378347, 27.301661752881266, 27.021117138428913, 26.914328312562123, 26.57322352444304, 26.506000357289228, 26.379173042690734, 26.174861635940307, 25.65261473292303, 25.580499348307644, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 22.928676021920396, "exact_area100": 28.042240724382754}, "test": {"score": [18.129624718318375, 18.34658029461874, 18.62089513751095, 18.66341878796323, 18.945120681660867, 18.868537688102375, 19.40965505310129, 19.82335406378871, 19.833554615719958, 20.37655968034874, 21.03003107523692, 21.255550029330344, 21.67540912918265, 21.767839053763787, 21.85834371217866, 21.84959586537697, 22.265059133979527, 22.291169141310753, 22.468932748538013, 22.621288846946104, 23.19956758370792, 23.676207048662683, 24.084904119372542, 24.317174147608164, 24.643336278875495, 24.6470959873525, 24.739407001976833, 25.181661381425364, 25.276627437912335, 25.422715737961944, 25.881992673937397, 26.144447837057054, 26.519920731627632, 26.97331733005322, 27.036699029708544, 27.24836667667884, 27.294897044080212, 28.13910201998482, 28.55629660507968, 28.894659191054966, 29.63109793926727, 29.660846855637878, 29.89057578222604, 30.272504331600693, 30.267238540011533, 29.94456979113117, 29.94439002291685, 29.875993589336698, 29.523954675288195, 29.878586296941478, 30.1851248107429, 30.10935699810804, 30.069245261964717, 29.95817675144782, 29.852531490011668, 29.70010643394537, 29.660922679685903, 29.56339556431047, 29.373784578410916, 29.664086468122253, 29.53948676595847, 29.467781869279573, 29.3966142005075, 29.120078501980817, 29.071873501825863, 29.05719126438792, 29.0223165550105, 28.84903113438537, 28.730437080505627, 29.02782168324402, 28.987450662196334, 29.025280341771886, 29.043130477389205, 29.323940975767986, 29.22416121081169, 29.114595621013763, 28.97499361788646, 28.75523204380303, 28.967722112384553, 29.166573212045954, 29.03457404914652, 28.947535532960554, 28.825828765782802, 28.84466375656476, 29.29313710222587, 29.13026976906668, 29.088436688603892, 28.87144047946985, 28.592071791972867, 28.523074044683582, 28.39599626812198, 28.21466260158584, 28.137181602927463, 28.06327299305599, 27.757768860176295, 27.78670284058215, 27.57207730941641, 27.467068813876626, 27.318185180400928, 27.327787555917077, 27.001009067886244], "calls": [0.0, 0.7656458055925432, 1.4980026631158456, 2.330226364846871, 3.129161118508655, 3.7283621837549936, 5.625832223701731, 6.990679094540613, 7.390146471371504, 8.788282290279627, 10.086551264980026, 10.885486018641812, 12.117177097203728, 12.78295605858855, 14.314247669773636, 14.913448735019974, 16.411451398135817, 17.7762982689747, 18.375499334221036, 19.0745672436751, 20.805592543275633, 21.937416777629824, 22.836218375499335, 23.868175765645805, 24.63382157123835, 25.63249001331558, 25.798934753661783, 26.564580559254324, 27.563249001331556, 28.428761651131822, 30.22636484687084, 31.22503328894807, 32.39014647137151, 33.32223701731025, 33.82157123834887, 35.05326231691079, 36.051930758988014, 37.416777629826896, 38.48202396804261, 39.38082556591212, 40.31291611185087, 40.54593874833555, 41.14513981358189, 43.20905459387483, 43.74167776298269, 44.87350199733689, 45.372836218375504, 46.03861517976032, 46.47137150466045, 48.46870838881491, 49.30093209054594, 50.3994673768309, 50.96537949400799, 53.16245006657789, 54.49400798934754, 55.35952063914781, 55.59254327563249, 57.25699067909454, 59.08788282290279, 60.086551264980024, 61.051930758988014, 61.75099866844208, 62.64980026631159, 64.18109187749667, 65.1797603195739, 66.41145139813582, 67.47669773635153, 68.94141145139814, 70.0399467376831, 70.5392809587217, 70.93874833555259, 72.7363515312916, 73.8681757656458, 75.0, 75.86551264980027, 76.89747003994674, 77.43009320905459, 78.36218375499334, 78.8948069241012, 79.36085219707057, 80.29294274300932, 81.0918774966711, 82.02396804260985, 83.12250332889481, 83.98801597869507, 84.95339547270306, 86.1185086551265, 88.48202396804261, 89.58055925432757, 90.07989347536618, 90.77896138482025, 91.81091877496671, 92.50998668442078, 93.37549933422103, 94.50732356857523, 94.90679094540613, 96.271637816245, 96.77097203728363, 97.6364846870839, 98.50199733688414, 100.0], "exact_score": [18.129624718318375, 18.441092378980578, 18.58016011713009, 18.911756220564367, 18.96641705999187, 19.189441588075585, 19.517183764442812, 19.82335406378871, 20.06954509202395, 20.41330416741251, 21.012881027080493, 21.298214601057, 21.614659926293232, 21.844084587728656, 22.074295376824146, 21.85916013983491, 22.107595062933573, 22.402149641371246, 22.322186425804176, 22.592065599120954, 22.831055300515338, 23.18685581188112, 23.676207048662683, 24.054385406480975, 24.3171712561297, 24.686330564634172, 24.71375574054778, 25.243725073572023, 25.340280529910913, 25.58919952882801, 25.824533048259045, 26.087233151246547, 26.480658607492195, 26.917418330958377, 27.0500618059537, 27.23200467392409, 27.278202163454445, 28.003193416527527, 28.308293364243976, 28.767064832302157, 29.53994925891369, 29.853873693472295, 30.11939515163906, 30.2514432675565, 30.35811430318645, 29.98263974088433, 29.875993589336698, 29.836480195635488, 29.796953497804836, 30.225258485105535, 30.116092465102735, 30.069245261964717, 30.069245261964717, 29.95817675144782, 29.88525230762732, 29.75955946605, 29.61492730951353, 29.56339556431047, 29.499292403918737, 29.373784578410916, 29.664086468122253, 29.53948676595847, 29.467781869279573, 29.356845499375734, 29.13994605164969, 29.071873501825863, 29.08073914507866, 29.045507201449773, 28.895181046343836, 28.84903113438537, 28.748725937652296, 28.987450662196334, 28.907504257409744, 29.00713956937811, 29.002020508221683, 29.323940975767986, 29.31434571033817, 29.055733636418967, 28.87956673351401, 28.967722112384553, 29.08001341179015, 28.947535532960554, 28.825828765782802, 28.560897628849585, 29.29313710222587, 29.13026976906668, 29.146329466554178, 29.162702146818088, 28.925801375597615, 28.783461136993072, 28.53205715643349, 28.37835980599628, 28.141687978593854, 28.104127018876856, 27.85744706701676, 27.738287738287738, 27.638803281413317, 27.42849466196493, 27.29642977190519, 27.2612699463711, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.267218513051926, "exact_area100": 26.79605213429762}}, "old_normalized_endpoint_budget": 45}, {"seed": 2027, "selection": {"best_epoch": 3, "epochs": 11, "alpha": 0.5, "validation_area100": 0.2818567624070939, "training_seconds": 4.042568775010295, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.518089016947248, "operator": "ge"}, {"budget": 0.02, "threshold": 0.4880370372805, "operator": "ge"}, {"budget": 0.03, "threshold": 0.46088990450435424, "operator": "ge"}, {"budget": 0.04, "threshold": 0.44469159767369154, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4243510173070925, "operator": "ge"}, {"budget": 0.06, "threshold": 0.39316711946182725, "operator": "ge"}, {"budget": 0.07, "threshold": 0.36782118828932453, "operator": "ge"}, {"budget": 0.08, "threshold": 0.34260315433564686, "operator": "ge"}, {"budget": 0.09, "threshold": 0.3357443876669164, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3198698203019351, "operator": "ge"}, {"budget": 0.11, "threshold": 0.31171891420917075, "operator": "ge"}, {"budget": 0.12, "threshold": 0.30402654432505666, "operator": "ge"}, {"budget": 0.13, "threshold": 0.2973861740749848, "operator": "ge"}, {"budget": 0.14, "threshold": 0.29051659724549883, "operator": "ge"}, {"budget": 0.15, "threshold": 0.28445193876481806, "operator": "ge"}, {"budget": 0.16, "threshold": 0.28042241249260813, "operator": "ge"}, {"budget": 0.17, "threshold": 0.27641092445242194, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2731982427916503, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2637649677177246, "operator": "ge"}, {"budget": 0.2, "threshold": 0.25538424898511763, "operator": "ge"}, {"budget": 0.21, "threshold": 0.24972452419800117, "operator": "ge"}, {"budget": 0.22, "threshold": 0.24258355432379625, "operator": "ge"}, {"budget": 0.23, "threshold": 0.23276241715730173, "operator": "ge"}, {"budget": 0.24, "threshold": 0.23100984552417103, "operator": "ge"}, {"budget": 0.25, "threshold": 0.22586245433270408, "operator": "ge"}, {"budget": 0.26, "threshold": 0.22152773112071367, "operator": "ge"}, {"budget": 0.27, "threshold": 0.21619122050769388, "operator": "ge"}, {"budget": 0.28, "threshold": 0.2112604517563722, "operator": "ge"}, {"budget": 0.29, "threshold": 0.2069938868636774, "operator": "ge"}, {"budget": 0.3, "threshold": 0.2006651454419196, "operator": "ge"}, {"budget": 0.31, "threshold": 0.1956818716797945, "operator": "ge"}, {"budget": 0.32, "threshold": 0.18646654923191622, "operator": "ge"}, {"budget": 0.33, "threshold": 0.17458925039983827, "operator": "ge"}, {"budget": 0.34, "threshold": 0.17026067762347244, "operator": "ge"}, {"budget": 0.35, "threshold": 0.1646278835660649, "operator": "ge"}, {"budget": 0.36, "threshold": 0.16072306262737282, "operator": "ge"}, {"budget": 0.37, "threshold": 0.1537220406743105, "operator": "ge"}, {"budget": 0.38, "threshold": 0.1438294684775272, "operator": "ge"}, {"budget": 0.39, "threshold": 0.12850514924425901, "operator": "ge"}, {"budget": 0.4, "threshold": 0.11584505402881254, "operator": "ge"}, {"budget": 0.41, "threshold": 0.10552088885850477, "operator": "ge"}, {"budget": 0.42, "threshold": 0.09495978531138638, "operator": "ge"}, {"budget": 0.43, "threshold": 0.08276522993553266, "operator": "ge"}, {"budget": 0.44, "threshold": 0.06601462062397177, "operator": "ge"}, {"budget": 0.45, "threshold": 0.023982490021399305, "operator": "ge"}, {"budget": 0.46, "threshold": -0.025407040006105067, "operator": "ge"}, {"budget": 0.47, "threshold": -0.05695836275149561, "operator": "ge"}, {"budget": 0.48, "threshold": -0.0813802744311484, "operator": "ge"}, {"budget": 0.49, "threshold": -0.09861065156034024, "operator": "ge"}, {"budget": 0.5, "threshold": -0.12235111836988154, "operator": "ge"}, {"budget": 0.51, "threshold": -0.1313841907296093, "operator": "ge"}, {"budget": 0.52, "threshold": -0.14223272319969057, "operator": "ge"}, {"budget": 0.53, "threshold": -0.144891101227107, "operator": "ge"}, {"budget": 0.54, "threshold": -0.147974861863235, "operator": "ge"}, {"budget": 0.55, "threshold": -0.15365870173655055, "operator": "ge"}, {"budget": 0.56, "threshold": -0.16124583849263316, "operator": "ge"}, {"budget": 0.57, "threshold": -0.16480854875993117, "operator": "ge"}, {"budget": 0.58, "threshold": -0.16895737795951668, "operator": "ge"}, {"budget": 0.59, "threshold": -0.1715856809641172, "operator": "ge"}, {"budget": 0.6, "threshold": -0.17567501256728066, "operator": "ge"}, {"budget": 0.61, "threshold": -0.17691864268934546, "operator": "ge"}, {"budget": 0.62, "threshold": -0.18110956504896508, "operator": "ge"}, {"budget": 0.63, "threshold": -0.1865744818004091, "operator": "ge"}, {"budget": 0.64, "threshold": -0.1903050649968313, "operator": "ge"}, {"budget": 0.65, "threshold": -0.19518057704656816, "operator": "ge"}, {"budget": 0.66, "threshold": -0.2017921645184441, "operator": "ge"}, {"budget": 0.67, "threshold": -0.20608424680313037, "operator": "ge"}, {"budget": 0.68, "threshold": -0.21315906896943326, "operator": "ge"}, {"budget": 0.69, "threshold": -0.2172966115452106, "operator": "ge"}, {"budget": 0.7, "threshold": -0.2229268569821037, "operator": "ge"}, {"budget": 0.71, "threshold": -0.23091135668140264, "operator": "ge"}, {"budget": 0.72, "threshold": -0.23466244372591893, "operator": "ge"}, {"budget": 0.73, "threshold": -0.24130662295272753, "operator": "ge"}, {"budget": 0.74, "threshold": -0.24472548824159476, "operator": "ge"}, {"budget": 0.75, "threshold": -0.24961294913208062, "operator": "ge"}, {"budget": 0.76, "threshold": -0.25221715687486107, "operator": "ge"}, {"budget": 0.77, "threshold": -0.25626412026863105, "operator": "ge"}, {"budget": 0.78, "threshold": -0.2589083244075888, "operator": "ge"}, {"budget": 0.79, "threshold": -0.26185266573220567, "operator": "ge"}, {"budget": 0.8, "threshold": -0.26344553771884044, "operator": "ge"}, {"budget": 0.81, "threshold": -0.2676955858040149, "operator": "ge"}, {"budget": 0.82, "threshold": -0.2699986322715279, "operator": "ge"}, {"budget": 0.83, "threshold": -0.27619057063648955, "operator": "ge"}, {"budget": 0.84, "threshold": -0.28059860049831303, "operator": "ge"}, {"budget": 0.85, "threshold": -0.28222678614474456, "operator": "ge"}, {"budget": 0.86, "threshold": -0.2862875094226447, "operator": "ge"}, {"budget": 0.87, "threshold": -0.289410720775781, "operator": "ge"}, {"budget": 0.88, "threshold": -0.2923447529546737, "operator": "ge"}, {"budget": 0.89, "threshold": -0.29707518521824555, "operator": "ge"}, {"budget": 0.9, "threshold": -0.29922696733046156, "operator": "ge"}, {"budget": 0.91, "threshold": -0.3014213599940868, "operator": "ge"}, {"budget": 0.92, "threshold": -0.3048532097749386, "operator": "ge"}, {"budget": 0.93, "threshold": -0.30839893079220443, "operator": "ge"}, {"budget": 0.94, "threshold": -0.3109551663235511, "operator": "ge"}, {"budget": 0.95, "threshold": -0.31494938744968587, "operator": "ge"}, {"budget": 0.96, "threshold": -0.32113970672036785, "operator": "ge"}, {"budget": 0.97, "threshold": -0.32886862546196594, "operator": "ge"}, {"budget": 0.98, "threshold": -0.33600946515749586, "operator": "ge"}, {"budget": 0.99, "threshold": -0.34100761543703995, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.516669362447082, 19.05083364977509, 19.529349314388906, 20.018565445394714, 20.47463582565153, 21.023830190168074, 22.192632019186835, 22.54209857106126, 22.766035988361313, 23.060141924471196, 23.137641358588276, 23.470758937626407, 23.93780330920805, 23.988095238095237, 24.135221145936214, 24.752164502164504, 25.15638220097423, 25.300448114106054, 25.60353322072072, 25.914965385181908, 25.91849281678742, 26.181562390793445, 26.01129729135186, 26.338837860725683, 26.958724561364356, 27.126165960111837, 27.221311606228138, 27.465896672570373, 28.16446829604724, 26.82052793365833, 27.05499994232389, 27.159405549276993, 27.42607347430983, 28.393832149660835, 28.785937827175356, 29.11466077648618, 29.044653976160827, 29.81020733652313, 30.05710283454186, 30.149630541871918, 30.930079510497734, 30.681617835995713, 30.791986889692584, 31.321916425543368, 32.78813352826511, 32.69493177387914, 32.333789556633896, 32.37024011642008, 32.06400109494191, 31.87892471421335, 31.819116101773165, 31.618979683495812, 31.534701857282503, 31.534701857282503, 31.310560864438695, 31.253761124094652, 31.253761124094652, 31.823381377259206, 31.823381377259206, 31.823381377259206, 31.76165298219748, 31.955118219935553, 31.608405121206328, 31.608405121206328, 32.04997276688453, 31.89918730650155, 31.715394491129782, 31.631360877684404, 31.631360877684404, 31.562961424880033, 31.404231266149875, 31.24798126614987, 31.10424359279773, 31.53099942726232, 31.385437674061272, 30.94718083848519, 30.808330174689623, 30.808330174689623, 30.549707096344154, 30.549707096344154, 29.98949681733348, 29.91788877713247, 29.73474835678523, 29.4000152911044, 29.32889153577012, 28.8864174523497, 28.44427231563154, 28.051352790440454, 27.85549792067254, 27.386363636363637, 27.341318591318593, 26.890622632934402, 26.80886863813693, 26.623850863205966, 26.149341850561363, 26.02856890370146, 25.816399874646635, 25.532278214246535, 25.71513941371808, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 25, 34, 46, 57, 68, 77, 87, 93, 99, 104, 111, 118, 124, 130, 139, 147, 153, 159, 166, 172, 180, 183, 189, 197, 202, 207, 213, 219, 223, 228, 235, 240, 246, 252, 256, 259, 263, 270, 273, 277, 284, 289, 294, 298, 299, 299, 300, 301, 301, 302, 302, 302, 302, 302, 302, 302, 303, 303, 303, 303, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307], "harmed": [0, 1, 1, 2, 2, 2, 3, 3, 5, 6, 7, 9, 10, 13, 15, 16, 16, 19, 20, 22, 23, 26, 30, 33, 34, 36, 37, 39, 41, 41, 45, 47, 52, 56, 57, 59, 61, 64, 67, 69, 71, 76, 81, 83, 83, 84, 85, 87, 91, 93, 94, 96, 98, 99, 99, 102, 103, 103, 103, 103, 103, 104, 108, 110, 110, 115, 117, 118, 119, 119, 120, 122, 124, 126, 127, 129, 133, 135, 135, 139, 139, 145, 146, 149, 152, 153, 158, 163, 168, 171, 176, 177, 188, 190, 192, 197, 198, 203, 208, 214, 220], "exact_score": [18.254072168965788, 18.516669362447082, 19.05083364977509, 19.529349314388906, 20.018565445394714, 20.47463582565153, 21.023830190168074, 22.192632019186835, 22.54209857106126, 22.766035988361313, 23.060141924471196, 23.137641358588276, 23.470758937626407, 23.93780330920805, 23.988095238095237, 24.135221145936214, 24.752164502164504, 25.15638220097423, 25.300448114106054, 25.60353322072072, 25.914965385181908, 25.91849281678742, 26.181562390793445, 26.01129729135186, 26.338837860725683, 26.958724561364356, 27.126165960111837, 27.221311606228138, 27.465896672570373, 28.16446829604724, 26.82052793365833, 27.05499994232389, 27.159405549276993, 27.42607347430983, 28.393832149660835, 28.785937827175356, 29.11466077648618, 29.044653976160827, 29.81020733652313, 30.05710283454186, 30.149630541871918, 30.930079510497734, 30.681617835995713, 30.791986889692584, 31.321916425543368, 32.78813352826511, 32.69493177387914, 32.333789556633896, 32.37024011642008, 32.06400109494191, 31.87892471421335, 31.819116101773165, 31.618979683495812, 31.534701857282503, 31.534701857282503, 31.310560864438695, 31.253761124094652, 31.253761124094652, 31.823381377259206, 31.823381377259206, 31.823381377259206, 31.76165298219748, 31.955118219935553, 31.608405121206328, 31.608405121206328, 32.04997276688453, 31.89918730650155, 31.715394491129782, 31.631360877684404, 31.631360877684404, 31.562961424880033, 31.404231266149875, 31.24798126614987, 31.10424359279773, 31.53099942726232, 31.385437674061272, 30.94718083848519, 30.808330174689623, 30.808330174689623, 30.549707096344154, 30.549707096344154, 29.98949681733348, 29.91788877713247, 29.73474835678523, 29.4000152911044, 29.32889153577012, 28.8864174523497, 28.44427231563154, 28.051352790440454, 27.85549792067254, 27.386363636363637, 27.341318591318593, 26.890622632934402, 26.80886863813693, 26.623850863205966, 26.149341850561363, 26.02856890370146, 25.816399874646635, 25.532278214246535, 25.71513941371808, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 28.18567624070939, "threshold_area100": 28.1893959455988, "threshold_area25": 23.238272529281183}, "test": {"score": [18.129624718318375, 18.403616782813277, 18.92614316741908, 19.22545900795615, 19.361074056836085, 19.63003723858802, 20.328764353744383, 19.992336202460496, 19.925170526881768, 20.053621762398265, 20.704741912845293, 20.81154337943367, 21.106292580976604, 21.252331709504148, 21.602566161552595, 21.79248379554339, 22.081402860130957, 22.141881396073295, 22.27568457721449, 22.6677582587694, 22.969298521111988, 23.15713035463071, 23.70059642532806, 23.95322024643278, 24.06659312542673, 24.217552495626528, 24.747232147571253, 24.91110011559429, 25.326518263818553, 25.58781516333149, 25.75520275568415, 26.6714927522255, 27.0135705438763, 27.5223526421471, 27.910681307280548, 28.037871010569937, 28.17449559952682, 28.73654953499451, 29.17547316881545, 29.46984997192741, 29.67924732160576, 29.802527272713654, 29.860887841007873, 30.038254265574686, 29.798507009023435, 29.9648658140793, 29.84385884962839, 29.82121245698599, 29.770679457531056, 29.857072184449883, 30.50688253843853, 30.504324346981694, 30.34663020723101, 30.31960804906131, 30.159392947737167, 29.761794663780282, 29.69035839600458, 29.66462113058903, 29.55559201696157, 29.4788578561398, 29.284232434450907, 29.271929453028413, 29.19128490557062, 29.08953192496598, 29.03464099536829, 28.96168980319428, 29.15815311338274, 29.00099374417497, 29.18346060095493, 29.21141015461607, 29.03975238689781, 28.920765412958318, 28.82136269994499, 28.7280794418465, 28.68213514531907, 28.912025190881245, 28.89519143358662, 28.93131319424728, 28.832290247729176, 28.768900947508996, 28.670928296160064, 28.68290623964203, 28.988299536621202, 28.622525970345684, 28.39203906250835, 28.26944663787549, 28.685456360873076, 28.82316872814952, 28.575951232244652, 28.344940429626774, 28.234930317536566, 28.118722963585707, 27.88721361043734, 27.773708181473904, 27.74083369052278, 27.497077812123294, 27.278392095850023, 27.124091912728606, 27.194962518798505, 27.361496253216, 27.001009067886244], "calls": [0.0, 1.0652463382157125, 2.2969374167776295, 3.561917443408788, 4.227696404793608, 5.126498002663116, 6.391478029294275, 7.423435419440745, 8.954727030625833, 9.387483355525966, 10.85219707057257, 11.617842876165113, 12.78295605858855, 13.84820239680426, 15.14647137150466, 16.011984021304926, 17.0439414114514, 17.676431424766978, 18.209054593874836, 19.640479360852197, 21.171770972037287, 21.837549933422103, 23.00266311584554, 24.267643142476697, 24.667110519307588, 25.499334221038616, 26.364846870838882, 27.19707057256991, 27.862849533954726, 28.56191744340879, 29.6271637816245, 30.9254327563249, 32.52330226364847, 34.720372836218374, 35.61917443408788, 36.18508655126498, 36.51797603195739, 37.68308921438083, 38.68175765645805, 39.81358189081225, 40.41278295605859, 41.24500665778961, 41.6444740346205, 42.310252996005325, 43.142476697736356, 44.47403462050599, 45.173102529960055, 45.70572569906791, 47.10386151797603, 47.836218375499335, 48.63515312916112, 49.60053262316911, 52.729693741677764, 53.295605858854856, 54.427430093209054, 55.692410119840204, 57.65645805592543, 58.28894806924101, 59.42077230359521, 60.05326231691078, 61.41810918774967, 61.75099866844208, 62.5832223701731, 64.14780292942743, 64.88015978695073, 66.14513981358189, 67.50998668442078, 68.60852197070572, 69.90679094540613, 70.33954727030626, 71.03861517976033, 72.50332889480693, 73.16910785619174, 74.13448735019973, 74.63382157123834, 75.49933422103862, 75.99866844207723, 76.83089214380826, 77.5965379494008, 78.46205059920106, 79.12782956058588, 80.25965379494008, 81.19174434087883, 83.05592543275633, 84.4873501997337, 84.88681757656458, 85.98535286284952, 86.7842876165113, 87.48335552596538, 88.91478029294274, 89.31424766977364, 90.04660452729694, 91.04527296937417, 92.07723035952064, 92.50998668442078, 93.57523302263648, 95.47270306258322, 96.60452729693742, 97.70306258322236, 98.16910785619174, 100.0], "rescued": [0, 60, 116, 176, 204, 245, 306, 350, 402, 417, 461, 484, 516, 541, 575, 600, 628, 641, 653, 692, 725, 743, 773, 802, 812, 831, 853, 871, 889, 909, 935, 967, 999, 1044, 1068, 1082, 1091, 1117, 1134, 1151, 1163, 1174, 1181, 1194, 1208, 1229, 1235, 1237, 1242, 1244, 1248, 1249, 1252, 1253, 1253, 1253, 1254, 1254, 1254, 1254, 1254, 1254, 1254, 1255, 1255, 1256, 1257, 1257, 1259, 1260, 1260, 1263, 1263, 1263, 1263, 1264, 1264, 1265, 1265, 1265, 1265, 1266, 1269, 1269, 1269, 1269, 1272, 1274, 1275, 1276, 1276, 1277, 1277, 1279, 1279, 1279, 1279, 1279, 1280, 1281, 1282], "harmed": [0, 2, 11, 14, 17, 25, 27, 38, 49, 52, 59, 62, 66, 77, 80, 88, 91, 98, 99, 105, 119, 122, 133, 143, 145, 152, 157, 166, 171, 173, 184, 194, 208, 232, 239, 246, 250, 260, 271, 281, 285, 291, 297, 301, 307, 315, 320, 323, 329, 332, 335, 337, 344, 347, 355, 360, 364, 366, 371, 375, 387, 388, 393, 404, 407, 412, 418, 427, 434, 435, 442, 454, 461, 467, 470, 476, 477, 482, 487, 491, 499, 516, 528, 543, 554, 561, 578, 588, 601, 620, 626, 635, 653, 666, 673, 688, 708, 723, 738, 743, 777], "exact_score": [18.129624718318375, 18.39442127210876, 18.850102585349827, 19.191582010357422, 19.247847166001055, 19.595631609739975, 20.121998922453557, 19.9642833075005, 19.799845325747267, 19.924141055381707, 20.487116715350833, 20.746381381519896, 20.861287005588476, 21.15899757031383, 21.28491809977401, 21.570879251987762, 21.78247915551765, 22.050715352622902, 22.210352746015925, 22.517325712963995, 22.76006409160813, 22.91323754663689, 23.19640919215645, 23.70213939408893, 23.91642647585128, 24.096987260921974, 24.656331106238564, 24.85656007369391, 25.35414099824805, 25.636122328704914, 26.143244323464305, 26.6714927522255, 26.91503975832371, 27.039677459678302, 27.337705644422318, 27.64322997375979, 28.010332223931243, 28.3255036263954, 28.950364322412913, 28.98386480137231, 29.514284315211352, 29.750358027816056, 29.97591752541635, 29.78754915330707, 29.955429758779882, 29.89268995504254, 29.853825162996333, 29.81948140077279, 30.555405956385783, 30.50688253843853, 30.523674300710056, 30.481828373287446, 30.45440116395098, 30.31960804906131, 30.237891065705085, 29.827258295854236, 29.761794663780282, 29.735779907632487, 29.67745535518796, 29.568290250496194, 29.4788578561398, 29.371863898029083, 29.24090331675445, 29.172628746052993, 29.14562149903757, 29.03464099536829, 28.9222844296126, 29.247373893042898, 29.104310747917623, 29.040350609051064, 29.18346060095493, 29.09471734410401, 28.961877867869678, 28.83934243188353, 28.774975038649032, 28.64234842478442, 28.89519143358662, 28.93131319424728, 28.803854072191243, 28.692165888934813, 28.780349992673948, 28.92900184135912, 28.807722344874048, 28.66226106968343, 28.418775104559195, 28.804165914501517, 28.685456360873076, 28.695704462477124, 28.429791218108925, 28.344940429626774, 28.12724368348263, 27.925312319782186, 27.762585540554667, 27.679912110634724, 27.40651310188339, 27.317358720622114, 27.217333749841487, 27.267771364342853, 27.382436921323027, 27.182424788731748, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.72153985192856, "threshold_area100": 26.765642324766365, "threshold_area25": 21.254030015012226}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.9648658140793, "calls": 44.47403462050599, "rescued": 1229, "harmed": 315}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.43508368730545044, "operator": "ge"}, {"budget": 0.02, "threshold": 0.39439457654953003, "operator": "ge"}, {"budget": 0.03, "threshold": 0.38270026445388794, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3687681257724762, "operator": "ge"}, {"budget": 0.05, "threshold": 0.35730674862861633, "operator": "ge"}, {"budget": 0.06, "threshold": 0.331734836101532, "operator": "ge"}, {"budget": 0.07, "threshold": 0.3003591001033783, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2473229169845581, "operator": "ge"}, {"budget": 0.09, "threshold": 0.2351711392402649, "operator": "ge"}, {"budget": 0.1, "threshold": 0.22199943661689758, "operator": "ge"}, {"budget": 0.11, "threshold": 0.20388758182525635, "operator": "ge"}, {"budget": 0.12, "threshold": 0.19730007648468018, "operator": "ge"}, {"budget": 0.13, "threshold": 0.1895771622657776, "operator": "ge"}, {"budget": 0.14, "threshold": 0.182353675365448, "operator": "ge"}, {"budget": 0.15, "threshold": 0.17708566784858704, "operator": "ge"}, {"budget": 0.16, "threshold": 0.1728474646806717, "operator": "ge"}, {"budget": 0.17, "threshold": 0.17069965600967407, "operator": "ge"}, {"budget": 0.18, "threshold": 0.1687675416469574, "operator": "ge"}, {"budget": 0.19, "threshold": 0.1631973534822464, "operator": "ge"}, {"budget": 0.2, "threshold": 0.16082823276519775, "operator": "ge"}, {"budget": 0.21, "threshold": 0.1578754484653473, "operator": "ge"}, {"budget": 0.22, "threshold": 0.1527038812637329, "operator": "ge"}, {"budget": 0.23, "threshold": 0.14874011278152466, "operator": "ge"}, {"budget": 0.24, "threshold": 0.14589807391166687, "operator": "ge"}, {"budget": 0.25, "threshold": 0.14389944076538086, "operator": "ge"}, {"budget": 0.26, "threshold": 0.1405252069234848, "operator": "ge"}, {"budget": 0.27, "threshold": 0.13775424659252167, "operator": "ge"}, {"budget": 0.28, "threshold": 0.13388068974018097, "operator": "ge"}, {"budget": 0.29, "threshold": 0.13160309195518494, "operator": "ge"}, {"budget": 0.3, "threshold": 0.12806296348571777, "operator": "ge"}, {"budget": 0.31, "threshold": 0.12324130535125732, "operator": "ge"}, {"budget": 0.32, "threshold": 0.11826478689908981, "operator": "ge"}, {"budget": 0.33, "threshold": 0.11469864845275879, "operator": "ge"}, {"budget": 0.34, "threshold": 0.11028627306222916, "operator": "ge"}, {"budget": 0.35, "threshold": 0.10504066199064255, "operator": "ge"}, {"budget": 0.36, "threshold": 0.10353726148605347, "operator": "ge"}, {"budget": 0.37, "threshold": 0.09927493333816528, "operator": "ge"}, {"budget": 0.38, "threshold": 0.09250859171152115, "operator": "ge"}, {"budget": 0.39, "threshold": 0.08360899239778519, "operator": "ge"}, {"budget": 0.4, "threshold": 0.07872214168310165, "operator": "ge"}, {"budget": 0.41, "threshold": 0.07109049707651138, "operator": "ge"}, {"budget": 0.42, "threshold": 0.05620367079973221, "operator": "ge"}, {"budget": 0.43, "threshold": 0.04776056110858917, "operator": "ge"}, {"budget": 0.44, "threshold": 0.040211163461208344, "operator": "ge"}, {"budget": 0.45, "threshold": 0.013585388660430908, "operator": "ge"}, {"budget": 0.46, "threshold": -0.01009707897901535, "operator": "ge"}, {"budget": 0.47, "threshold": -0.014972377568483353, "operator": "ge"}, {"budget": 0.48, "threshold": -0.016675198450684547, "operator": "ge"}, {"budget": 0.49, "threshold": -0.018703671172261238, "operator": "ge"}, {"budget": 0.5, "threshold": -0.0203506201505661, "operator": "ge"}, {"budget": 0.51, "threshold": -0.021486401557922363, "operator": "ge"}, {"budget": 0.52, "threshold": -0.022623423486948013, "operator": "ge"}, {"budget": 0.53, "threshold": -0.024670051410794258, "operator": "ge"}, {"budget": 0.54, "threshold": -0.027217170223593712, "operator": "ge"}, {"budget": 0.55, "threshold": -0.030365074053406715, "operator": "ge"}, {"budget": 0.56, "threshold": -0.032790109515190125, "operator": "ge"}, {"budget": 0.57, "threshold": -0.03454376757144928, "operator": "ge"}, {"budget": 0.58, "threshold": -0.03547851741313934, "operator": "ge"}, {"budget": 0.59, "threshold": -0.03588720038533211, "operator": "ge"}, {"budget": 0.6, "threshold": -0.0380115769803524, "operator": "ge"}, {"budget": 0.61, "threshold": -0.039439857006073, "operator": "ge"}, {"budget": 0.62, "threshold": -0.04094216972589493, "operator": "ge"}, {"budget": 0.63, "threshold": -0.04200148954987526, "operator": "ge"}, {"budget": 0.64, "threshold": -0.04403527453541756, "operator": "ge"}, {"budget": 0.65, "threshold": -0.045748915523290634, "operator": "ge"}, {"budget": 0.66, "threshold": -0.04765385389328003, "operator": "ge"}, {"budget": 0.67, "threshold": -0.049774736166000366, "operator": "ge"}, {"budget": 0.68, "threshold": -0.05171993747353554, "operator": "ge"}, {"budget": 0.69, "threshold": -0.05433857440948486, "operator": "ge"}, {"budget": 0.7, "threshold": -0.05538832023739815, "operator": "ge"}, {"budget": 0.71, "threshold": -0.058838069438934326, "operator": "ge"}, {"budget": 0.72, "threshold": -0.062271907925605774, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0636187493801117, "operator": "ge"}, {"budget": 0.74, "threshold": -0.06667265295982361, "operator": "ge"}, {"budget": 0.75, "threshold": -0.06834901869297028, "operator": "ge"}, {"budget": 0.76, "threshold": -0.06984257698059082, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0710829645395279, "operator": "ge"}, {"budget": 0.78, "threshold": -0.07251127809286118, "operator": "ge"}, {"budget": 0.79, "threshold": -0.0737060084939003, "operator": "ge"}, {"budget": 0.8, "threshold": -0.07548533380031586, "operator": "ge"}, {"budget": 0.81, "threshold": -0.07699192315340042, "operator": "ge"}, {"budget": 0.82, "threshold": -0.08011180907487869, "operator": "ge"}, {"budget": 0.83, "threshold": -0.08307629823684692, "operator": "ge"}, {"budget": 0.84, "threshold": -0.08461720496416092, "operator": "ge"}, {"budget": 0.85, "threshold": -0.08574878424406052, "operator": "ge"}, {"budget": 0.86, "threshold": -0.08713366836309433, "operator": "ge"}, {"budget": 0.87, "threshold": -0.08954668045043945, "operator": "ge"}, {"budget": 0.88, "threshold": -0.09281641244888306, "operator": "ge"}, {"budget": 0.89, "threshold": -0.09433876723051071, "operator": "ge"}, {"budget": 0.9, "threshold": -0.09634554386138916, "operator": "ge"}, {"budget": 0.91, "threshold": -0.09767800569534302, "operator": "ge"}, {"budget": 0.92, "threshold": -0.10018672049045563, "operator": "ge"}, {"budget": 0.93, "threshold": -0.10294808447360992, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10626194626092911, "operator": "ge"}, {"budget": 0.95, "threshold": -0.10906388610601425, "operator": "ge"}, {"budget": 0.96, "threshold": -0.11220439523458481, "operator": "ge"}, {"budget": 0.97, "threshold": -0.1165541559457779, "operator": "ge"}, {"budget": 0.98, "threshold": -0.12092119455337524, "operator": "ge"}, {"budget": 0.99, "threshold": -0.1286080777645111, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.541758214321383, 19.077389651520214, 19.589735787818853, 20.107091147098785, 20.426558902574605, 21.023830190168074, 22.160030209416533, 22.73354385867717, 23.596595245453912, 24.14049483161934, 24.379721835552225, 24.616402116402117, 24.930725474203733, 24.975361070671838, 24.98303573580086, 25.240608498687354, 25.258904779121305, 25.54643404936947, 25.415117575466837, 25.712759691826303, 25.944963144963147, 26.12798212472045, 26.613947904743295, 26.562711462123666, 26.946324634822155, 27.235424150317773, 27.526821005081874, 27.353930977850442, 27.738297500621446, 28.203361384802978, 26.798763764309474, 27.147335079383883, 27.42607347430983, 28.02979706194154, 28.515114627713533, 28.553571428571427, 29.324300376931955, 29.988203238203237, 29.6953564727955, 29.895992288233668, 29.91346153846154, 30.7323449773721, 31.125356125356124, 31.239121703039224, 32.70447636073488, 32.69493177387914, 32.515316067947644, 32.4317738791423, 32.118490671122245, 31.831898531307505, 31.831898531307505, 31.831898531307505, 31.636513157894736, 31.716956002571873, 31.716956002571873, 31.307887341153474, 31.14208540115798, 31.222528245835125, 31.303817857298334, 31.303817857298334, 30.939853510948588, 30.883053770604548, 30.631037641572284, 30.575657894736842, 30.575657894736842, 30.964320708539727, 30.845326212035197, 30.845326212035197, 30.68798687448806, 30.619747899159666, 30.485465559280712, 30.33135369532428, 29.986430067565973, 30.930829941782633, 30.930829941782633, 30.868407969248302, 31.272774844203415, 31.043956043956044, 30.840378169767053, 30.22868274354929, 30.15725769059623, 30.028678137921837, 29.97198879551821, 29.291709666322674, 28.68293159126052, 28.37015825157939, 28.25097169979532, 27.953406242879925, 27.85549792067254, 27.650375939849624, 27.07970383275261, 27.03624618420647, 26.806418219461698, 26.586236760655364, 26.187857256723827, 25.990690115822673, 25.91678028581528, 25.566648917908573, 25.71513941371808, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 25, 36, 47, 57, 68, 80, 89, 101, 107, 113, 119, 127, 133, 138, 144, 151, 157, 160, 167, 174, 179, 186, 190, 196, 201, 207, 212, 218, 224, 230, 236, 241, 247, 252, 256, 261, 265, 269, 273, 278, 285, 289, 293, 297, 299, 299, 299, 299, 299, 299, 299, 299, 300, 300, 300, 300, 300, 301, 301, 301, 301, 301, 301, 301, 302, 302, 302, 302, 303, 303, 303, 303, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307], "harmed": [0, 1, 1, 2, 2, 3, 3, 5, 5, 7, 7, 8, 9, 10, 12, 14, 16, 18, 23, 25, 26, 30, 31, 31, 33, 34, 36, 37, 42, 44, 45, 53, 55, 57, 59, 61, 64, 65, 66, 70, 71, 77, 82, 82, 83, 84, 85, 86, 87, 89, 92, 92, 92, 94, 94, 94, 97, 99, 102, 102, 106, 109, 110, 111, 112, 112, 114, 116, 116, 118, 120, 122, 124, 127, 128, 128, 129, 130, 133, 136, 143, 144, 146, 147, 153, 159, 162, 164, 169, 171, 175, 180, 185, 188, 191, 196, 199, 201, 207, 214, 220], "exact_score": [18.254072168965788, 18.541758214321383, 19.077389651520214, 19.589735787818853, 20.107091147098785, 20.426558902574605, 21.023830190168074, 22.160030209416533, 22.73354385867717, 23.596595245453912, 24.14049483161934, 24.379721835552225, 24.616402116402117, 24.930725474203733, 24.975361070671838, 24.98303573580086, 25.240608498687354, 25.258904779121305, 25.54643404936947, 25.415117575466837, 25.712759691826303, 25.944963144963147, 26.12798212472045, 26.613947904743295, 26.562711462123666, 26.946324634822155, 27.235424150317773, 27.526821005081874, 27.353930977850442, 27.738297500621446, 28.203361384802978, 26.798763764309474, 27.147335079383883, 27.42607347430983, 28.02979706194154, 28.515114627713533, 28.553571428571427, 29.324300376931955, 29.988203238203237, 29.6953564727955, 29.895992288233668, 29.91346153846154, 30.7323449773721, 31.125356125356124, 31.239121703039224, 32.70447636073488, 32.69493177387914, 32.515316067947644, 32.4317738791423, 32.118490671122245, 31.831898531307505, 31.831898531307505, 31.831898531307505, 31.636513157894736, 31.716956002571873, 31.716956002571873, 31.307887341153474, 31.14208540115798, 31.222528245835125, 31.303817857298334, 31.303817857298334, 30.939853510948588, 30.883053770604548, 30.631037641572284, 30.575657894736842, 30.575657894736842, 30.964320708539727, 30.845326212035197, 30.845326212035197, 30.68798687448806, 30.619747899159666, 30.485465559280712, 30.33135369532428, 29.986430067565973, 30.930829941782633, 30.930829941782633, 30.868407969248302, 31.272774844203415, 31.043956043956044, 30.840378169767053, 30.22868274354929, 30.15725769059623, 30.028678137921837, 29.97198879551821, 29.291709666322674, 28.68293159126052, 28.37015825157939, 28.25097169979532, 27.953406242879925, 27.85549792067254, 27.650375939849624, 27.07970383275261, 27.03624618420647, 26.806418219461698, 26.586236760655364, 26.187857256723827, 25.990690115822673, 25.91678028581528, 25.566648917908573, 25.71513941371808, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 28.114811782445887, "threshold_area100": 28.118430252394926, "threshold_area25": 23.572609512743018}, "test": {"score": [18.129624718318375, 18.480405079336347, 19.008571793301623, 19.19773409517283, 19.39452684865529, 19.65579902868569, 20.105168396841595, 20.166327140323396, 20.0114842296364, 20.074070119906363, 20.233993099507767, 21.092990314330898, 21.257071939045378, 21.60384266652929, 21.922811741263953, 22.209972461602266, 22.808321242541517, 22.949368692200498, 23.027757964652963, 23.184307514813984, 23.320366620380774, 23.50098386222515, 23.76885114520933, 24.046815146578886, 24.250955585552035, 24.318679931375346, 24.421028254034464, 24.65410681463445, 25.17454674859075, 25.199157054942212, 26.090412665833462, 26.364488239475214, 26.875948483185592, 27.72720927250688, 28.04359445158961, 28.404032469232774, 28.59188251293514, 28.66365463452125, 29.025541953283945, 29.545446972139782, 29.578538732457226, 29.69856450340085, 29.965775210080324, 30.087916610066557, 29.75700382375225, 29.94603011846414, 29.84385884962839, 29.82000673275302, 29.780419691508964, 29.780419691508964, 29.65843048651434, 29.60338088664899, 29.494645585672103, 29.457150550246165, 29.3098151139114, 29.473283363305985, 29.43789398236445, 29.40483295665584, 29.269891257536607, 29.211108647951097, 29.21464973216827, 29.09333924548896, 29.294232436441693, 29.274882591240452, 29.256647197987945, 29.577266948400972, 29.526176738353215, 29.467491560248014, 29.32214728472553, 29.545246067770236, 29.50382050991062, 29.40090352098832, 29.670553338392214, 29.633551907348277, 29.497901399658033, 29.337606747082912, 28.98828412017861, 29.01580728142032, 28.932801525512907, 28.910523020752343, 29.06795546430172, 28.95136377989116, 29.388575868645432, 29.09129575630543, 28.68995272184537, 28.617614352106873, 28.513757383454813, 28.813478213922405, 28.417174562457692, 28.229214798861886, 28.144369942367298, 28.077261582664693, 27.87712336834224, 27.763788288356583, 27.616774461596872, 27.42479126078983, 27.35869445960917, 27.15354988602004, 27.46140276165866, 27.214025858093656, 27.001009067886244], "calls": [0.0, 1.3648468708388815, 3.3288948069241013, 3.7283621837549936, 4.2942743009320905, 4.993342210386151, 6.125166444740346, 7.789613848202396, 9.387483355525966, 9.620505992010653, 10.252996005326231, 11.8508655126498, 12.683089214380825, 14.247669773635153, 15.412782956058587, 16.81091877496671, 17.80958721704394, 18.308921438082557, 18.77496671105193, 20.539280958721704, 21.13848202396804, 21.904127829560586, 22.969374167776298, 24.201065246338217, 25.06657789613848, 25.5659121171771, 26.131824234354195, 27.230359520639148, 28.1624500665779, 28.728362183754992, 29.727030625832224, 31.19174434087883, 32.7896138482024, 33.988015978695074, 35.086551264980024, 36.41810918774967, 37.050599201065246, 37.68308921438083, 38.84820239680426, 39.74700399467377, 40.046604527296935, 40.77896138482024, 41.977363515312916, 42.543275632490015, 43.142476697736356, 44.44074567243675, 45.10652463382157, 45.40612516644474, 46.23834886817576, 47.802929427430094, 49.367509986684425, 50.29960053262317, 51.46471371504661, 53.06258322237017, 54.29427430093209, 56.15845539280959, 57.39014647137151, 58.22237017310253, 58.88814913448736, 59.28761651131824, 60.852197070572565, 62.18375499334221, 63.04926764314248, 63.48202396804261, 64.78029294274302, 65.9121171770972, 66.84420772303595, 67.74300932090546, 68.54194407456724, 69.2410119840213, 69.54061251664447, 71.03861517976033, 72.43675099866844, 73.03595206391478, 74.36750998668442, 75.46604527296937, 76.1318242343542, 76.63115845539281, 77.16378162450066, 77.86284953395473, 78.69507323568575, 79.69374167776299, 81.49134487350199, 83.15579227696405, 83.92143808255659, 84.32090545938749, 84.88681757656458, 86.01864181091877, 87.64980026631159, 88.54860186418108, 89.24766977363515, 89.61384820239681, 90.87882822902796, 92.07723035952064, 93.00932090545939, 93.94141145139814, 95.1398135818908, 96.271637816245, 97.33688415446072, 98.76830892143809, 100.0], "rescued": [0, 82, 163, 184, 210, 241, 293, 363, 424, 431, 453, 501, 522, 562, 596, 631, 655, 668, 678, 716, 733, 750, 775, 804, 827, 840, 849, 874, 898, 911, 938, 974, 1007, 1036, 1063, 1089, 1103, 1111, 1135, 1149, 1155, 1167, 1187, 1197, 1206, 1228, 1235, 1235, 1236, 1236, 1236, 1236, 1236, 1237, 1238, 1239, 1241, 1242, 1242, 1242, 1243, 1243, 1245, 1246, 1247, 1251, 1251, 1253, 1254, 1257, 1257, 1259, 1261, 1261, 1261, 1262, 1262, 1263, 1263, 1263, 1264, 1264, 1268, 1269, 1269, 1269, 1269, 1271, 1274, 1275, 1276, 1276, 1278, 1279, 1279, 1279, 1279, 1279, 1281, 1281, 1282], "harmed": [0, 2, 15, 15, 18, 21, 26, 41, 51, 52, 57, 65, 71, 79, 85, 93, 94, 96, 99, 117, 123, 128, 131, 143, 148, 156, 157, 162, 175, 182, 187, 207, 215, 220, 232, 245, 257, 264, 271, 278, 282, 289, 298, 301, 307, 315, 320, 321, 325, 325, 329, 332, 338, 341, 349, 357, 362, 364, 371, 374, 380, 388, 401, 403, 405, 409, 412, 418, 424, 429, 432, 440, 449, 451, 460, 468, 473, 477, 481, 483, 491, 500, 520, 542, 557, 560, 567, 582, 606, 619, 627, 631, 649, 663, 674, 690, 700, 719, 733, 753, 777], "exact_score": [18.129624718318375, 18.42409659258952, 18.472017014889783, 19.003401593159253, 19.250546620619616, 19.65579902868569, 20.043749011521996, 20.23730027147096, 19.651560306940794, 19.97924406689619, 20.130568129393794, 20.505609651267918, 21.115406269947332, 21.34834122476443, 21.567673986925755, 21.775276052449964, 22.038216571557147, 22.263215935117234, 22.880637021085562, 23.033924603536768, 23.006485763598146, 23.289398219779855, 23.523957821976627, 23.76885114520933, 24.00881456628453, 24.252318704989744, 24.39507465344544, 24.58825757445405, 25.147255221925207, 25.267590383668033, 26.133392461050224, 26.316606965807516, 26.615821578913195, 27.096394302384237, 27.72720927250688, 28.011615770710197, 28.244010983719647, 28.584973985321792, 28.883430734305705, 29.1176760827363, 29.578538732457226, 29.788498134461722, 29.965775210080324, 29.746153049017142, 29.955429758779882, 29.89268995504254, 29.812922035952184, 29.780419691508964, 29.735258401186382, 29.67182177223081, 29.626165119959534, 29.539201419701623, 29.494645585672103, 29.457150550246165, 29.360360638040316, 29.579908384715154, 29.52530686956304, 29.41285260212025, 29.417178461481168, 29.25125419408956, 29.131826341647766, 29.21464973216827, 29.09333924548896, 29.294232436441693, 29.274882591240452, 29.238453581123185, 29.577266948400972, 29.482136808817106, 29.40596079799954, 29.551720109204087, 29.50647073912313, 29.39439395672649, 29.40957385714836, 29.633551907348277, 29.5572895521522, 29.397346053843233, 28.98828412017861, 28.95025164829147, 28.868069719383847, 29.03348138620429, 28.93422233211708, 28.85560069406361, 29.26069332294279, 29.084609499533443, 28.68995272184537, 28.75933204338907, 28.813478213922405, 28.616008029541273, 28.40406978649251, 28.169390394995837, 27.966230612893494, 27.95323555629515, 27.78002747769625, 27.616774461596872, 27.42479126078983, 27.36650573909675, 27.17687869091866, 27.283632511330367, 27.39620679646127, 27.171256822419615, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.73352276764931, "threshold_area100": 26.838525088194665, "threshold_area25": 21.32885670700467}, "selected_budget": 45, "selected": {"val": {"score": 32.70447636073488, "calls": 44.935805991440795, "rescued": 297, "harmed": 84}, "test": {"score": 29.94603011846414, "calls": 44.44074567243675, "rescued": 1228, "harmed": 315}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.0426948070526123, "operator": "ge"}, {"budget": 0.02, "threshold": 2.685128688812256, "operator": "ge"}, {"budget": 0.03, "threshold": 2.5958871841430664, "operator": "ge"}, {"budget": 0.04, "threshold": 2.433452606201172, "operator": "ge"}, {"budget": 0.05, "threshold": 2.326662063598633, "operator": "ge"}, {"budget": 0.06, "threshold": 2.24094557762146, "operator": "ge"}, {"budget": 0.07, "threshold": 2.1460022926330566, "operator": "ge"}, {"budget": 0.08, "threshold": 2.064455032348633, "operator": "ge"}, {"budget": 0.09, "threshold": 2.0215044021606445, "operator": "ge"}, {"budget": 0.1, "threshold": 1.959765076637268, "operator": "ge"}, {"budget": 0.11, "threshold": 1.9037253856658936, "operator": "ge"}, {"budget": 0.12, "threshold": 1.8463233709335327, "operator": "ge"}, {"budget": 0.13, "threshold": 1.8133851289749146, "operator": "ge"}, {"budget": 0.14, "threshold": 1.762868046760559, "operator": "ge"}, {"budget": 0.15, "threshold": 1.6988577842712402, "operator": "ge"}, {"budget": 0.16, "threshold": 1.6460609436035156, "operator": "ge"}, {"budget": 0.17, "threshold": 1.6200083494186401, "operator": "ge"}, {"budget": 0.18, "threshold": 1.570464015007019, "operator": "ge"}, {"budget": 0.19, "threshold": 1.5220651626586914, "operator": "ge"}, {"budget": 0.2, "threshold": 1.4898558855056763, "operator": "ge"}, {"budget": 0.21, "threshold": 1.4556061029434204, "operator": "ge"}, {"budget": 0.22, "threshold": 1.418637752532959, "operator": "ge"}, {"budget": 0.23, "threshold": 1.3715016841888428, "operator": "ge"}, {"budget": 0.24, "threshold": 1.2848047018051147, "operator": "ge"}, {"budget": 0.25, "threshold": 1.240536093711853, "operator": "ge"}, {"budget": 0.26, "threshold": 1.1898354291915894, "operator": "ge"}, {"budget": 0.27, "threshold": 1.1614941358566284, "operator": "ge"}, {"budget": 0.28, "threshold": 1.1386717557907104, "operator": "ge"}, {"budget": 0.29, "threshold": 1.117440104484558, "operator": "ge"}, {"budget": 0.3, "threshold": 1.0919796228408813, "operator": "ge"}, {"budget": 0.31, "threshold": 1.0681796073913574, "operator": "ge"}, {"budget": 0.32, "threshold": 1.016892671585083, "operator": "ge"}, {"budget": 0.33, "threshold": 0.9946056604385376, "operator": "ge"}, {"budget": 0.34, "threshold": 0.9486693143844604, "operator": "ge"}, {"budget": 0.35, "threshold": 0.8882340788841248, "operator": "ge"}, {"budget": 0.36, "threshold": 0.8218019008636475, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7999151945114136, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7785786986351013, "operator": "ge"}, {"budget": 0.39, "threshold": 0.7423438429832458, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6631040573120117, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5921413898468018, "operator": "ge"}, {"budget": 0.42, "threshold": 0.53154057264328, "operator": "ge"}, {"budget": 0.43, "threshold": 0.4742388427257538, "operator": "ge"}, {"budget": 0.44, "threshold": 0.3810046911239624, "operator": "ge"}, {"budget": 0.45, "threshold": 0.2047330141067505, "operator": "ge"}, {"budget": 0.46, "threshold": -0.190618634223938, "operator": "ge"}, {"budget": 0.47, "threshold": -0.47620630264282227, "operator": "ge"}, {"budget": 0.48, "threshold": -0.771338939666748, "operator": "ge"}, {"budget": 0.49, "threshold": -0.9619788527488708, "operator": "ge"}, {"budget": 0.5, "threshold": -1.2795436382293701, "operator": "ge"}, {"budget": 0.51, "threshold": -1.4993704557418823, "operator": "ge"}, {"budget": 0.52, "threshold": -2.1929731369018555, "operator": "ge"}, {"budget": 0.53, "threshold": -2.413578987121582, "operator": "ge"}, {"budget": 0.54, "threshold": -2.5522966384887695, "operator": "ge"}, {"budget": 0.55, "threshold": -2.6470885276794434, "operator": "ge"}, {"budget": 0.56, "threshold": -2.7687742710113525, "operator": "ge"}, {"budget": 0.57, "threshold": -2.8514139652252197, "operator": "ge"}, {"budget": 0.58, "threshold": -2.988232135772705, "operator": "ge"}, {"budget": 0.59, "threshold": -3.1171975135803223, "operator": "ge"}, {"budget": 0.6, "threshold": -3.2011280059814453, "operator": "ge"}, {"budget": 0.61, "threshold": -3.2842864990234375, "operator": "ge"}, {"budget": 0.62, "threshold": -3.319500207901001, "operator": "ge"}, {"budget": 0.63, "threshold": -3.334731101989746, "operator": "ge"}, {"budget": 0.64, "threshold": -3.3976809978485107, "operator": "ge"}, {"budget": 0.65, "threshold": -3.4635698795318604, "operator": "ge"}, {"budget": 0.66, "threshold": -3.5137319564819336, "operator": "ge"}, {"budget": 0.67, "threshold": -3.5500874519348145, "operator": "ge"}, {"budget": 0.68, "threshold": -3.583932399749756, "operator": "ge"}, {"budget": 0.69, "threshold": -3.6020188331604004, "operator": "ge"}, {"budget": 0.7, "threshold": -3.625671148300171, "operator": "ge"}, {"budget": 0.71, "threshold": -3.6464667320251465, "operator": "ge"}, {"budget": 0.72, "threshold": -3.6686339378356934, "operator": "ge"}, {"budget": 0.73, "threshold": -3.693894863128662, "operator": "ge"}, {"budget": 0.74, "threshold": -3.7328386306762695, "operator": "ge"}, {"budget": 0.75, "threshold": -3.7651584148406982, "operator": "ge"}, {"budget": 0.76, "threshold": -3.7921175956726074, "operator": "ge"}, {"budget": 0.77, "threshold": -3.8201725482940674, "operator": "ge"}, {"budget": 0.78, "threshold": -3.887235641479492, "operator": "ge"}, {"budget": 0.79, "threshold": -3.950103998184204, "operator": "ge"}, {"budget": 0.8, "threshold": -4.002713680267334, "operator": "ge"}, {"budget": 0.81, "threshold": -4.1588029861450195, "operator": "ge"}, {"budget": 0.82, "threshold": -10.118515014648438, "operator": "ge"}, {"budget": 0.83, "threshold": -10.349989891052246, "operator": "ge"}, {"budget": 0.84, "threshold": -10.436674118041992, "operator": "ge"}, {"budget": 0.85, "threshold": -10.573359489440918, "operator": "ge"}, {"budget": 0.86, "threshold": -10.636312484741211, "operator": "ge"}, {"budget": 0.87, "threshold": -10.754395484924316, "operator": "ge"}, {"budget": 0.88, "threshold": -10.864968299865723, "operator": "ge"}, {"budget": 0.89, "threshold": -11.021512031555176, "operator": "ge"}, {"budget": 0.9, "threshold": -11.134138107299805, "operator": "ge"}, {"budget": 0.91, "threshold": -11.231292724609375, "operator": "ge"}, {"budget": 0.92, "threshold": -11.283885955810547, "operator": "ge"}, {"budget": 0.93, "threshold": -11.363173484802246, "operator": "ge"}, {"budget": 0.94, "threshold": -11.397407531738281, "operator": "ge"}, {"budget": 0.95, "threshold": -11.432819366455078, "operator": "ge"}, {"budget": 0.96, "threshold": -11.49505615234375, "operator": "ge"}, {"budget": 0.97, "threshold": -11.545638084411621, "operator": "ge"}, {"budget": 0.98, "threshold": -11.580564498901367, "operator": "ge"}, {"budget": 0.99, "threshold": -11.670291900634766, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.53992216854536, 19.03311257437565, 19.510370708453774, 20.09558114080876, 20.43755388798751, 20.755020947102576, 21.69513623994177, 21.940993788819878, 22.163578742077036, 22.443258807588077, 22.757578054030812, 23.15779086890161, 23.44389423526114, 23.67949568679496, 23.99865929277694, 24.16890516890517, 24.41586944799549, 24.79988347948193, 25.130912830248377, 25.529063021485186, 26.008154470392476, 26.428832211964746, 25.226183664113456, 25.814798228591336, 26.051587301587304, 26.17259333111776, 26.20388757215354, 26.689780109945026, 26.94138005624054, 26.97021946698029, 26.861296522080558, 27.062758213653353, 27.470126469530875, 27.416858084490165, 28.00634181883067, 28.570811518279804, 28.85542618301239, 29.468607904626865, 29.851808989132934, 30.18843576071931, 29.97018505094563, 29.82429065187686, 30.799534806591254, 32.62146231220359, 32.78813352826511, 32.515316067947644, 32.417701264450486, 32.185700738332315, 32.002077423572956, 31.905241072074897, 32.753392653351824, 32.574663239985824, 32.35189795049722, 32.03578942095772, 32.214286406262104, 32.214286406262104, 32.14588695345773, 32.14588695345773, 31.924625087189025, 31.77705940408353, 31.653587370499135, 31.370662267839688, 31.23239987484356, 31.176408833410186, 31.012165831244776, 30.771683673469386, 30.210084033613445, 30.03267973856209, 29.637616052710392, 29.230844154111363, 29.189142736263157, 28.88077926418287, 28.53427891768252, 28.297069833320688, 28.142443242972643, 28.53274017549484, 28.387560152266033, 28.017414028242417, 27.97676912917725, 27.473230561940237, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.293431775609996, 27.152805996181673, 27.152805996181673, 27.015836153576846, 26.940245602395745, 26.940245602395745, 26.575274061687104, 26.575274061687104, 26.354061341422312, 26.222309035756965, 26.041740298296457, 25.852732343526885, 25.816954168213822, 25.816954168213822, 25.753839631939574, 25.656083007515633, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 23, 34, 44, 54, 62, 73, 78, 84, 92, 99, 106, 113, 120, 127, 133, 140, 146, 153, 159, 167, 175, 182, 190, 194, 200, 205, 212, 216, 220, 225, 231, 236, 240, 246, 252, 258, 261, 267, 271, 276, 282, 289, 293, 298, 299, 299, 300, 301, 302, 304, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 2, 2, 3, 3, 4, 6, 10, 11, 11, 11, 12, 14, 16, 18, 19, 19, 20, 21, 22, 24, 28, 29, 30, 32, 35, 38, 38, 41, 46, 49, 50, 53, 55, 61, 63, 66, 66, 69, 75, 79, 82, 82, 84, 86, 90, 92, 93, 95, 97, 99, 102, 106, 108, 108, 109, 109, 112, 114, 116, 120, 122, 123, 125, 129, 133, 136, 141, 146, 147, 151, 153, 160, 162, 167, 168, 173, 174, 181, 186, 186, 187, 188, 190, 190, 192, 194, 194, 203, 203, 206, 208, 210, 213, 214, 214, 215, 217, 220], "exact_score": [18.254072168965788, 18.53992216854536, 19.03311257437565, 19.510370708453774, 20.09558114080876, 20.43755388798751, 20.755020947102576, 21.69513623994177, 21.940993788819878, 22.163578742077036, 22.443258807588077, 22.757578054030812, 23.15779086890161, 23.44389423526114, 23.67949568679496, 23.99865929277694, 24.16890516890517, 24.41586944799549, 24.79988347948193, 25.130912830248377, 25.529063021485186, 26.008154470392476, 26.428832211964746, 25.226183664113456, 25.814798228591336, 26.051587301587304, 26.17259333111776, 26.20388757215354, 26.689780109945026, 26.94138005624054, 26.97021946698029, 26.861296522080558, 27.062758213653353, 27.470126469530875, 27.416858084490165, 28.00634181883067, 28.570811518279804, 28.85542618301239, 29.468607904626865, 29.851808989132934, 30.18843576071931, 29.97018505094563, 29.82429065187686, 30.799534806591254, 32.62146231220359, 32.78813352826511, 32.515316067947644, 32.417701264450486, 32.185700738332315, 32.002077423572956, 31.905241072074897, 32.753392653351824, 32.574663239985824, 32.35189795049722, 32.03578942095772, 32.214286406262104, 32.214286406262104, 32.14588695345773, 32.14588695345773, 31.924625087189025, 31.77705940408353, 31.653587370499135, 31.370662267839688, 31.23239987484356, 31.176408833410186, 31.012165831244776, 30.771683673469386, 30.210084033613445, 30.03267973856209, 29.637616052710392, 29.230844154111363, 29.189142736263157, 28.88077926418287, 28.53427891768252, 28.297069833320688, 28.142443242972643, 28.53274017549484, 28.387560152266033, 28.017414028242417, 27.97676912917725, 27.473230561940237, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.293431775609996, 27.152805996181673, 27.152805996181673, 27.015836153576846, 26.940245602395745, 26.940245602395745, 26.575274061687104, 26.575274061687104, 26.354061341422312, 26.222309035756965, 26.041740298296457, 25.852732343526885, 25.816954168213822, 25.816954168213822, 25.753839631939574, 25.656083007515633, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.498500349848953, "threshold_area100": 27.501280558041564, "threshold_area25": 22.912272949022853}, "test": {"score": [18.129624718318375, 18.390076902909776, 18.929470860924173, 19.172162961140057, 19.39243902726144, 19.44929870426276, 19.744421623015832, 19.91472816291028, 20.620767913523704, 20.73249549218323, 20.88288186606471, 21.160853301453542, 21.51241743147746, 21.39944323589701, 21.39777673958631, 21.761747799641494, 22.02672949283668, 22.029543857235296, 22.233789811960353, 22.52898930144933, 23.079167115620837, 23.22858169919421, 23.394299879095705, 23.66999386186912, 24.296373427670545, 24.33146871689432, 24.595992218756223, 25.1075930073892, 25.666667834093698, 25.816575131977217, 26.18450838942633, 26.645254360313675, 26.827606210151234, 26.96796671306398, 27.193772021302582, 27.72409040279421, 28.460186682244228, 28.756663975410092, 28.88923872364017, 29.124111525256712, 29.49595005478894, 29.66913666009512, 29.496606742980834, 29.670085584388804, 29.789135569013986, 29.951243073183832, 29.868083655829942, 29.921559784802167, 29.988635037522375, 30.294463213290012, 30.1369453480138, 29.94119353973334, 29.788771645603706, 29.657840601354906, 29.561084573124635, 29.485348958661568, 29.78456654441385, 29.729856428055967, 29.905858384082112, 29.72002371573954, 29.67773309373905, 29.59687579964523, 29.502523226933302, 29.427616547117697, 29.224874013032686, 28.952166221640564, 28.576682711000444, 28.80239486852942, 28.526450627767957, 28.508166665955308, 28.610527998011204, 28.502934633581074, 28.411396474422023, 28.600882824709657, 28.822494205582483, 28.566982341470464, 28.58437757372733, 28.45717683517931, 28.139029821185808, 27.91011532495521, 27.977302431637206, 27.83131723224603, 27.77118150504717, 27.737405780697088, 27.69306947682888, 27.657416712351424, 27.64105982979736, 27.598484382383454, 27.583628262180167, 27.51791516119565, 27.364682960766924, 27.346534483213823, 27.53238044941, 27.38815673411783, 27.319949239168594, 27.27396200682555, 27.167065686978088, 27.083448524081295, 27.220342088802035, 27.095207586479802, 27.001009067886244], "calls": [0.0, 0.8322237017310253, 2.263648468708389, 3.029294274300932, 4.194407456724368, 4.8934753661784285, 5.525965379494008, 6.691078561917443, 7.989347536617843, 8.721704394141145, 9.986684420772303, 11.51797603195739, 12.583222370173102, 13.315579227696405, 14.380825565912117, 15.645805592543274, 16.77762982689747, 17.41011984021305, 18.40878828229028, 19.573901464713714, 20.372836218375497, 21.13848202396804, 22.303595206391478, 23.468708388814914, 25.932090545938745, 26.79760319573902, 27.82956058588549, 28.462050599201067, 29.360852197070574, 29.96005326231691, 30.426098535286282, 31.091877496671106, 32.75632490013315, 33.255659121171774, 34.45406125166445, 36.35153129161118, 37.616511318242345, 37.91611185086551, 38.41544607190413, 39.114513981358186, 40.1797603195739, 41.11185086551265, 41.944074567243675, 42.44340878828229, 43.10918774966711, 44.34087882822903, 45.206391478029296, 45.80559254327563, 46.70439414114514, 47.836218375499335, 49.40079893475366, 50.36617842876166, 51.597869507323566, 52.46338215712384, 53.495339547270305, 54.52729693741678, 56.45805592543276, 57.490013315579226, 58.75499334221038, 60.78561917443409, 61.817576564580555, 63.44873501997337, 63.98135818908123, 64.31424766977364, 65.5792276964048, 67.01065246338216, 68.50865512649801, 69.57390146471371, 70.33954727030626, 70.97203728362183, 71.90412782956058, 72.53661784287615, 73.5352862849534, 74.13448735019973, 75.03328894806924, 76.26498002663115, 77.1970705725699, 77.86284953395473, 79.6271637816245, 80.62583222370174, 81.19174434087883, 81.79094540612518, 82.58988015978696, 84.32090545938749, 84.9866844207723, 86.05193075898802, 86.61784287616511, 87.51664447403462, 88.41544607190413, 89.78029294274302, 90.9121171770972, 91.97736351531292, 92.67643142476697, 93.80825565912117, 94.4074567243675, 95.07323568575234, 96.37150466045273, 97.33688415446072, 98.03595206391478, 99.2676431424767, 100.0], "rescued": [0, 44, 115, 147, 200, 230, 262, 307, 355, 379, 414, 452, 489, 507, 535, 569, 599, 613, 643, 678, 700, 717, 744, 769, 830, 850, 876, 895, 919, 935, 944, 962, 1000, 1012, 1040, 1081, 1108, 1115, 1126, 1137, 1154, 1168, 1184, 1193, 1206, 1227, 1235, 1242, 1245, 1247, 1253, 1253, 1255, 1255, 1255, 1255, 1258, 1259, 1260, 1261, 1263, 1265, 1265, 1265, 1266, 1266, 1266, 1268, 1268, 1269, 1270, 1271, 1272, 1273, 1275, 1275, 1276, 1277, 1277, 1277, 1278, 1278, 1278, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1280, 1281, 1281, 1281, 1281, 1281, 1281, 1282, 1282, 1282], "harmed": [0, 3, 7, 9, 17, 20, 23, 29, 36, 40, 49, 56, 61, 66, 70, 78, 84, 91, 99, 106, 108, 112, 122, 130, 143, 156, 164, 166, 173, 177, 182, 191, 213, 216, 229, 249, 262, 268, 271, 276, 282, 289, 297, 300, 306, 314, 319, 323, 327, 330, 341, 347, 356, 364, 370, 375, 389, 394, 404, 416, 430, 442, 446, 453, 472, 489, 512, 523, 535, 538, 550, 558, 566, 572, 580, 595, 612, 626, 645, 658, 670, 680, 684, 688, 690, 692, 693, 696, 698, 703, 713, 717, 720, 732, 736, 740, 750, 754, 760, 770, 777], "exact_score": [18.129624718318375, 18.468660577784412, 18.832199922810265, 19.15485006992745, 19.355869737285015, 19.487966161467547, 19.737373936881866, 20.026245016724832, 20.620767913523704, 20.79185433011529, 20.88288186606471, 21.029767651147832, 21.28917225422854, 21.624186346530145, 21.306825289932384, 21.55993453700209, 21.84829241878208, 22.060494839308802, 22.091466671212938, 22.357577071365526, 22.971781305114636, 23.204229217881092, 23.354022811836057, 23.608438283733808, 23.835071161900434, 24.09418496948473, 24.296373427670545, 24.391476804943498, 24.64535439794197, 25.525874053591696, 25.83446421963374, 26.637188553645792, 26.800196495293616, 26.877213987875155, 27.063423824753453, 27.242288370129064, 27.482610520662636, 28.01344521263134, 28.787162942291545, 29.13765222473056, 29.460745117109173, 29.640860792129292, 29.50117311680856, 29.775001710949127, 29.915366193017924, 29.89268995504254, 29.935025466550396, 29.96550295332159, 30.272351549387306, 29.838946193310523, 30.075862777165064, 29.814786401751498, 29.76303438018816, 29.6378625771815, 29.516945040871228, 29.455849497559516, 29.685826234441954, 29.70199146470454, 29.665338918762124, 29.905858384082112, 29.826448799006442, 29.67111631680589, 29.67773309373905, 29.622229628709473, 29.502523226933302, 29.21479173285927, 29.103105970090414, 28.96240072293168, 28.710771437399035, 28.4909039955687, 28.62402887376318, 28.508166665955308, 28.563927229034576, 28.42325057885419, 28.65398028598974, 28.839944328361046, 28.58454674428077, 28.663153110444423, 28.440224168488253, 28.358035419407923, 28.083920625619278, 28.059960345972417, 27.795211566550094, 27.75170098556665, 27.765619133037312, 27.69306947682888, 27.657416712351424, 27.64105982979736, 27.598484382383454, 27.583628262180167, 27.498871169575008, 27.364682960766924, 27.346534483213823, 27.474440082719013, 27.38815673411783, 27.289856972889794, 27.180875934181515, 27.125107543267703, 27.238555919170558, 27.101930748554913, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.535148702112394, "threshold_area100": 26.568637647320735, "threshold_area25": 21.28072844063923}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.951243073183832, "calls": 44.34087882822903, "rescued": 1227, "harmed": 314}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.5229766368865967, "operator": "ge"}, {"budget": 0.02, "threshold": 1.38344407081604, "operator": "ge"}, {"budget": 0.03, "threshold": 1.2971680164337158, "operator": "ge"}, {"budget": 0.04, "threshold": 1.2845852375030518, "operator": "ge"}, {"budget": 0.05, "threshold": 1.221800446510315, "operator": "ge"}, {"budget": 0.06, "threshold": 1.1948943138122559, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1683859825134277, "operator": "ge"}, {"budget": 0.08, "threshold": 1.1314830780029297, "operator": "ge"}, {"budget": 0.09, "threshold": 1.106157898902893, "operator": "ge"}, {"budget": 0.1, "threshold": 1.0789979696273804, "operator": "ge"}, {"budget": 0.11, "threshold": 1.0489403009414673, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0200045108795166, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9998811483383179, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9780311584472656, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9572279453277588, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9404582977294922, "operator": "ge"}, {"budget": 0.17, "threshold": 0.902900218963623, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8786701560020447, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8275414705276489, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8093934059143066, "operator": "ge"}, {"budget": 0.21, "threshold": 0.7945913076400757, "operator": "ge"}, {"budget": 0.22, "threshold": 0.7788383960723877, "operator": "ge"}, {"budget": 0.23, "threshold": 0.7657745480537415, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7504820823669434, "operator": "ge"}, {"budget": 0.25, "threshold": 0.7285891771316528, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7144769430160522, "operator": "ge"}, {"budget": 0.27, "threshold": 0.6959387063980103, "operator": "ge"}, {"budget": 0.28, "threshold": 0.6755775809288025, "operator": "ge"}, {"budget": 0.29, "threshold": 0.6678318977355957, "operator": "ge"}, {"budget": 0.3, "threshold": 0.6384396553039551, "operator": "ge"}, {"budget": 0.31, "threshold": 0.6212537288665771, "operator": "ge"}, {"budget": 0.32, "threshold": 0.5915743112564087, "operator": "ge"}, {"budget": 0.33, "threshold": 0.56186842918396, "operator": "ge"}, {"budget": 0.34, "threshold": 0.5442116260528564, "operator": "ge"}, {"budget": 0.35, "threshold": 0.5111484527587891, "operator": "ge"}, {"budget": 0.36, "threshold": 0.5004218816757202, "operator": "ge"}, {"budget": 0.37, "threshold": 0.4754229187965393, "operator": "ge"}, {"budget": 0.38, "threshold": 0.4614139199256897, "operator": "ge"}, {"budget": 0.39, "threshold": 0.4020252823829651, "operator": "ge"}, {"budget": 0.4, "threshold": 0.36853134632110596, "operator": "ge"}, {"budget": 0.41, "threshold": 0.34034284949302673, "operator": "ge"}, {"budget": 0.42, "threshold": 0.30720940232276917, "operator": "ge"}, {"budget": 0.43, "threshold": 0.26280227303504944, "operator": "ge"}, {"budget": 0.44, "threshold": 0.21054133772850037, "operator": "ge"}, {"budget": 0.45, "threshold": 0.09193474054336548, "operator": "ge"}, {"budget": 0.46, "threshold": -0.1093212366104126, "operator": "ge"}, {"budget": 0.47, "threshold": -0.2785923480987549, "operator": "ge"}, {"budget": 0.48, "threshold": -0.4629719853401184, "operator": "ge"}, {"budget": 0.49, "threshold": -0.5772367715835571, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9166885614395142, "operator": "ge"}, {"budget": 0.51, "threshold": -1.133985996246338, "operator": "ge"}, {"budget": 0.52, "threshold": -1.8460066318511963, "operator": "ge"}, {"budget": 0.53, "threshold": -2.142160415649414, "operator": "ge"}, {"budget": 0.54, "threshold": -2.2551040649414062, "operator": "ge"}, {"budget": 0.55, "threshold": -2.379361152648926, "operator": "ge"}, {"budget": 0.56, "threshold": -2.4720921516418457, "operator": "ge"}, {"budget": 0.57, "threshold": -2.5187907218933105, "operator": "ge"}, {"budget": 0.58, "threshold": -2.6102170944213867, "operator": "ge"}, {"budget": 0.59, "threshold": -2.68601655960083, "operator": "ge"}, {"budget": 0.6, "threshold": -2.7414257526397705, "operator": "ge"}, {"budget": 0.61, "threshold": -2.796489715576172, "operator": "ge"}, {"budget": 0.62, "threshold": -2.839552164077759, "operator": "ge"}, {"budget": 0.63, "threshold": -2.91774320602417, "operator": "ge"}, {"budget": 0.64, "threshold": -2.9330129623413086, "operator": "ge"}, {"budget": 0.65, "threshold": -2.9703750610351562, "operator": "ge"}, {"budget": 0.66, "threshold": -2.988419532775879, "operator": "ge"}, {"budget": 0.67, "threshold": -3.019507884979248, "operator": "ge"}, {"budget": 0.68, "threshold": -3.0471224784851074, "operator": "ge"}, {"budget": 0.69, "threshold": -3.095956325531006, "operator": "ge"}, {"budget": 0.7, "threshold": -3.1402060985565186, "operator": "ge"}, {"budget": 0.71, "threshold": -3.17036771774292, "operator": "ge"}, {"budget": 0.72, "threshold": -3.190143346786499, "operator": "ge"}, {"budget": 0.73, "threshold": -3.2181050777435303, "operator": "ge"}, {"budget": 0.74, "threshold": -3.2470383644104004, "operator": "ge"}, {"budget": 0.75, "threshold": -3.2758536338806152, "operator": "ge"}, {"budget": 0.76, "threshold": -3.3114194869995117, "operator": "ge"}, {"budget": 0.77, "threshold": -3.370337963104248, "operator": "ge"}, {"budget": 0.78, "threshold": -3.405747890472412, "operator": "ge"}, {"budget": 0.79, "threshold": -3.450901985168457, "operator": "ge"}, {"budget": 0.8, "threshold": -3.5218052864074707, "operator": "ge"}, {"budget": 0.81, "threshold": -3.6795997619628906, "operator": "ge"}, {"budget": 0.82, "threshold": -9.228294372558594, "operator": "ge"}, {"budget": 0.83, "threshold": -9.303159713745117, "operator": "ge"}, {"budget": 0.84, "threshold": -9.357259750366211, "operator": "ge"}, {"budget": 0.85, "threshold": -9.375672340393066, "operator": "ge"}, {"budget": 0.86, "threshold": -9.413708686828613, "operator": "ge"}, {"budget": 0.87, "threshold": -9.507954597473145, "operator": "ge"}, {"budget": 0.88, "threshold": -9.661394119262695, "operator": "ge"}, {"budget": 0.89, "threshold": -9.74758529663086, "operator": "ge"}, {"budget": 0.9, "threshold": -9.779101371765137, "operator": "ge"}, {"budget": 0.91, "threshold": -9.845609664916992, "operator": "ge"}, {"budget": 0.92, "threshold": -9.909482955932617, "operator": "ge"}, {"budget": 0.93, "threshold": -9.975411415100098, "operator": "ge"}, {"budget": 0.94, "threshold": -10.013114929199219, "operator": "ge"}, {"budget": 0.95, "threshold": -10.04423713684082, "operator": "ge"}, {"budget": 0.96, "threshold": -10.07896614074707, "operator": "ge"}, {"budget": 0.97, "threshold": -10.13426399230957, "operator": "ge"}, {"budget": 0.98, "threshold": -10.208490371704102, "operator": "ge"}, {"budget": 0.99, "threshold": -10.289209365844727, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.516669362447082, 18.88809102440146, 19.217944517469444, 19.49735748122845, 19.846914175506267, 20.15692640692641, 20.35189075630252, 20.722980823533625, 20.911635973528984, 21.31752359770734, 21.520090080571634, 21.64735743872434, 22.253325942350333, 23.219158528063268, 23.59829184895032, 23.961678185362395, 23.910254839500126, 24.327263622376403, 24.705286973088832, 24.916824616824616, 25.230240361819313, 25.56062101791179, 25.31247156247156, 25.787112662112666, 26.320971928694632, 26.829143674637233, 27.449305080884027, 27.612975697964327, 26.62330747394932, 26.959473934324436, 27.036807278742764, 27.208415821546218, 27.294791508442838, 28.128454426690787, 28.613130395274876, 29.1008941449999, 29.005696058327636, 29.934149184149184, 30.152254249815226, 30.48642672486427, 30.626513691516124, 31.084601791611256, 30.741259748316196, 31.321916425543368, 32.78813352826511, 32.69493177387914, 32.333789556633896, 32.28542613217535, 32.002077423572956, 32.48799165482548, 32.645865771631385, 32.50521879554137, 32.285779819436385, 31.783773291925467, 32.14588695345773, 32.07905990186725, 32.07905990186725, 31.77705940408353, 31.77705940408353, 31.63181267130508, 31.57076261025502, 31.45260120177103, 31.23239987484356, 31.096199444690154, 30.935082366279843, 30.73500233426704, 30.313280257786364, 29.91893727187845, 29.545454545454547, 29.226726726726724, 28.998820248820252, 28.976756793788716, 29.12728735937598, 28.931623931623935, 28.61892849697728, 28.430289108255213, 28.387560152266033, 28.017414028242417, 27.69224503095471, 27.473230561940237, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.152805996181673, 27.152805996181673, 27.08398536448072, 27.016508981214866, 27.016508981214866, 27.016508981214866, 26.880872202231426, 26.880872202231426, 26.575274061687104, 26.538936852384783, 26.287868476316405, 26.041740298296457, 25.795595009828396, 25.795595009828396, 25.731882231504155, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 21, 30, 38, 48, 54, 61, 70, 75, 84, 90, 96, 106, 116, 122, 128, 133, 143, 151, 158, 164, 171, 175, 182, 190, 197, 204, 210, 216, 222, 228, 233, 239, 245, 250, 254, 259, 264, 267, 273, 279, 284, 288, 294, 298, 299, 299, 299, 301, 303, 304, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 2, 3, 4, 4, 7, 9, 9, 9, 10, 11, 13, 13, 14, 14, 14, 18, 20, 23, 25, 25, 27, 33, 34, 35, 35, 36, 40, 41, 43, 46, 49, 54, 55, 57, 59, 65, 66, 70, 71, 76, 78, 82, 83, 84, 85, 87, 91, 93, 95, 98, 100, 103, 107, 109, 110, 110, 114, 114, 116, 117, 119, 122, 124, 127, 131, 136, 141, 144, 147, 149, 152, 156, 159, 161, 163, 164, 173, 178, 181, 186, 186, 186, 187, 190, 190, 191, 192, 192, 192, 194, 198, 203, 204, 207, 210, 214, 214, 215, 220], "exact_score": [18.254072168965788, 18.516669362447082, 18.88809102440146, 19.217944517469444, 19.49735748122845, 19.846914175506267, 20.15692640692641, 20.35189075630252, 20.722980823533625, 20.911635973528984, 21.31752359770734, 21.520090080571634, 21.64735743872434, 22.253325942350333, 23.219158528063268, 23.59829184895032, 23.961678185362395, 23.910254839500126, 24.327263622376403, 24.705286973088832, 24.916824616824616, 25.230240361819313, 25.56062101791179, 25.31247156247156, 25.787112662112666, 26.320971928694632, 26.829143674637233, 27.449305080884027, 27.612975697964327, 26.62330747394932, 26.959473934324436, 27.036807278742764, 27.208415821546218, 27.294791508442838, 28.128454426690787, 28.613130395274876, 29.1008941449999, 29.005696058327636, 29.934149184149184, 30.152254249815226, 30.48642672486427, 30.626513691516124, 31.084601791611256, 30.741259748316196, 31.321916425543368, 32.78813352826511, 32.69493177387914, 32.333789556633896, 32.28542613217535, 32.002077423572956, 32.48799165482548, 32.645865771631385, 32.50521879554137, 32.285779819436385, 31.783773291925467, 32.14588695345773, 32.07905990186725, 32.07905990186725, 31.77705940408353, 31.77705940408353, 31.63181267130508, 31.57076261025502, 31.45260120177103, 31.23239987484356, 31.096199444690154, 30.935082366279843, 30.73500233426704, 30.313280257786364, 29.91893727187845, 29.545454545454547, 29.226726726726724, 28.998820248820252, 28.976756793788716, 29.12728735937598, 28.931623931623935, 28.61892849697728, 28.430289108255213, 28.387560152266033, 28.017414028242417, 27.69224503095471, 27.473230561940237, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.152805996181673, 27.152805996181673, 27.08398536448072, 27.016508981214866, 27.016508981214866, 27.016508981214866, 26.880872202231426, 26.880872202231426, 26.575274061687104, 26.538936852384783, 26.287868476316405, 26.041740298296457, 25.795595009828396, 25.795595009828396, 25.731882231504155, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.441138152371202, "threshold_area100": 27.44378227226457, "threshold_area25": 22.39341525756375}, "test": {"score": [18.129624718318375, 18.377406588032862, 18.79354966911277, 19.080551213363865, 19.193715677650765, 19.772618290138098, 19.930271735035873, 20.186553594983828, 20.323890707114362, 20.503486183558962, 20.85154806884875, 21.004763932474255, 21.149749540821034, 21.387877182984855, 21.525808085888613, 21.716554667062542, 21.59951099437652, 22.028124063792156, 22.379953666755707, 22.640141878788032, 22.81976101046675, 22.91187534251663, 23.34663311859999, 23.491305362200542, 23.648392702563903, 24.217256391428585, 24.318491991849296, 24.843166435066372, 25.45654562114083, 25.51264796177938, 26.164178409114502, 26.345238256174248, 26.887280977146986, 27.363807705306233, 27.587516598044832, 27.954107614923558, 28.134598586007485, 28.26195319723668, 28.562898430674156, 29.476995703713705, 29.549217043172078, 29.80344445119229, 29.913426664792713, 29.669040047280546, 29.761686977278835, 29.9648658140793, 29.868083655829942, 29.929283347641938, 29.969330828365464, 30.25300159565894, 30.150419410443625, 30.049134338105905, 29.82126988376691, 29.657840601354906, 29.612667137556077, 29.382448819364132, 29.44069697347106, 29.728503237886923, 29.888953696279486, 29.61757269063995, 29.37494608100127, 29.374395501730394, 29.245082146298834, 28.973971068171565, 28.943778471184473, 28.72565560089192, 28.578876820432036, 28.438884980790174, 28.536234715128767, 28.583539131370785, 28.334578028072464, 28.08460230028858, 28.024534450764012, 28.24258019419107, 28.06306765524425, 28.09743353643291, 27.989828957428735, 28.115671113262035, 27.994958378568207, 27.896459043022944, 27.96368513894101, 27.847829350239955, 27.707047267990113, 27.662734260746923, 27.629808458936715, 27.629808458936715, 27.61345157638265, 27.532606974780453, 27.43553388915709, 27.34287573435321, 27.32425376042397, 27.271062271062274, 27.457877959927142, 27.386517747063788, 27.379423208208813, 27.288625030840528, 27.218224064503758, 27.307883199602944, 27.26952778859559, 27.184073463462173, 27.001009067886244], "calls": [0.0, 0.6657789613848202, 1.6311584553928096, 3.029294274300932, 3.3288948069241013, 4.560585885486019, 5.159786950732357, 6.058588548601864, 7.290279627163781, 8.022636484687084, 9.121171770972039, 10.685752330226366, 12.15046604527297, 12.949400798934754, 13.81491344873502, 14.713715046604529, 15.479360852197072, 17.509986684420774, 18.47536617842876, 21.1051930758988, 22.03728362183755, 22.436750998668444, 23.00266311584554, 23.635153129161118, 24.201065246338217, 25.532623169107854, 26.131824234354195, 26.964047936085215, 28.328894806924097, 28.79494007989348, 30.392809587217045, 31.091877496671106, 32.65645805592543, 34.25432756324901, 34.953395472703065, 36.085219707057256, 36.684420772303596, 37.416777629826896, 37.982689747003995, 39.78029294274301, 40.47936085219707, 41.11185086551265, 41.577896138482025, 42.44340878828229, 43.275632490013315, 44.44074567243675, 45.206391478029296, 45.80559254327563, 46.70439414114514, 47.636484687083886, 49.367509986684425, 50.466045272969374, 51.33155792276965, 52.49667110519307, 53.295605858854856, 54.860186418109194, 56.657789613848195, 57.55659121171771, 59.05459387483355, 60.5525965379494, 61.48468708388815, 62.54993342210386, 63.48202396804261, 65.34620505992011, 65.64580559254327, 66.4780292942743, 66.94407456724367, 67.74300932090546, 68.54194407456724, 70.5392809587217, 71.70439414114514, 72.86950732356857, 73.26897470039947, 74.46737683089214, 75.76564580559254, 76.36484687083887, 77.16378162450066, 78.99467376830891, 79.49400798934754, 80.09320905459387, 80.95872170439414, 81.8242343541944, 83.3555259653795, 84.62050599201065, 85.75233022636485, 85.98535286284952, 86.75099866844208, 88.41544607190413, 89.94673768308921, 90.97869507323568, 91.3115845539281, 91.97736351531292, 93.07589880159787, 94.27430093209055, 94.70705725699068, 95.5392809587217, 96.271637816245, 97.30359520639148, 98.33555259653795, 99.20106524633822, 100.0], "rescued": [0, 34, 73, 136, 150, 198, 220, 255, 297, 319, 357, 407, 453, 479, 501, 530, 547, 603, 630, 706, 730, 740, 757, 773, 790, 826, 838, 859, 890, 903, 942, 957, 997, 1037, 1054, 1080, 1094, 1108, 1119, 1151, 1162, 1174, 1182, 1196, 1209, 1229, 1235, 1242, 1244, 1246, 1253, 1253, 1255, 1255, 1255, 1255, 1257, 1258, 1260, 1261, 1261, 1262, 1263, 1265, 1265, 1265, 1265, 1265, 1266, 1269, 1270, 1270, 1270, 1272, 1272, 1273, 1274, 1276, 1276, 1277, 1278, 1278, 1278, 1278, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1280, 1280, 1280, 1280, 1281, 1282, 1282, 1282, 1282], "harmed": [0, 2, 6, 12, 13, 20, 23, 29, 37, 39, 43, 53, 61, 65, 71, 74, 80, 89, 92, 117, 125, 127, 130, 135, 140, 153, 163, 166, 171, 178, 191, 197, 210, 228, 235, 244, 250, 260, 267, 281, 288, 288, 294, 304, 310, 315, 319, 322, 327, 331, 340, 345, 354, 364, 367, 384, 391, 393, 406, 422, 441, 444, 452, 478, 481, 495, 503, 515, 525, 545, 561, 577, 582, 590, 601, 604, 616, 639, 646, 656, 669, 679, 687, 689, 693, 693, 694, 701, 709, 715, 716, 722, 726, 731, 732, 738, 745, 754, 757, 764, 777], "exact_score": [18.129624718318375, 18.481457132927407, 18.698727953885903, 19.072652319209045, 19.38442814745878, 19.91093766328458, 20.168580968107975, 20.218810555164275, 20.4929587596196, 20.826295809976937, 20.994701494180216, 21.08278794861363, 21.08681406046149, 21.3972598473124, 21.596423367955506, 21.735917419058325, 21.754957398531293, 21.914631012141385, 22.17491298208562, 22.530208696087865, 22.51976010112525, 22.784791982019176, 22.79509099080675, 23.333572788300025, 23.58046476641347, 23.806353837602696, 24.250686120268774, 24.843166435066372, 25.439354736086727, 25.546239953422905, 25.706262564732423, 26.302268741874325, 26.618930974695733, 26.907694385824303, 27.24758771362565, 27.60361465434525, 27.943379462962923, 28.091517092560803, 28.562898430674156, 29.53632133654675, 29.466936999441263, 29.761916052515474, 29.673270349304286, 29.7531947488902, 29.90851033363563, 29.89268995504254, 29.935025466550396, 30.301037453842095, 30.229096615621426, 30.146106977884468, 30.1369453480138, 29.814786401751498, 29.68330479233634, 29.625231530067698, 29.52931260487337, 29.37014583794163, 29.314600088455776, 29.40283273980263, 30.067434107873208, 29.888953696279486, 29.679525216012948, 29.486886272013624, 29.37494608100127, 29.392481849558227, 29.228949215585175, 28.99166900164976, 28.85343067624379, 28.578876820432036, 28.713888589655458, 28.433172435698516, 28.360640487789173, 28.476875112089584, 28.26277043615925, 28.08460230028858, 27.958374760021908, 28.079592278999222, 28.03759928538012, 27.89837839242329, 27.83832594171642, 28.115671113262035, 27.929882828976055, 27.9558995151858, 27.795211566550094, 27.72348763322643, 27.662734260746923, 27.63512600733222, 27.629808458936715, 27.61345157638265, 27.58216286606194, 27.482584904908613, 27.43553388915709, 27.34287573435321, 27.271062271062274, 27.457877959927142, 27.386517747063788, 27.36344913315929, 27.220842282996983, 27.126426543316857, 27.27639592046372, 27.19086853055951, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.47440666532004, "threshold_area100": 26.47418859042433, "threshold_area25": 21.266784518277927}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.9648658140793, "calls": 44.44074567243675, "rescued": 1229, "harmed": 315}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 32.70447636073488, "calls": 44.935805991440795}, "test": {"score": 29.94603011846414, "calls": 44.44074567243675}}, "old_normalized_interior": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795}, "test": {"score": 29.9648658140793, "calls": 44.47403462050599}}, "old_normalized_endpoints": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795}, "test": {"score": 29.9648658140793, "calls": 44.47403462050599}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.516669362447082, 19.05083364977509, 19.529349314388906, 20.018565445394714, 20.47463582565153, 21.023830190168074, 22.192632019186835, 22.54209857106126, 22.766035988361313, 23.060141924471196, 23.137641358588276, 23.470758937626407, 23.93780330920805, 23.988095238095237, 24.135221145936214, 24.752164502164504, 25.15638220097423, 25.300448114106054, 25.60353322072072, 25.914965385181908, 25.91849281678742, 26.181562390793445, 26.01129729135186, 26.338837860725683, 26.958724561364356, 27.126165960111837, 27.221311606228138, 27.465896672570373, 28.16446829604724, 26.82052793365833, 27.05499994232389, 27.159405549276993, 27.42607347430983, 28.393832149660835, 28.785937827175356, 29.11466077648618, 29.044653976160827, 29.81020733652313, 30.05710283454186, 30.149630541871918, 30.930079510497734, 30.681617835995713, 30.791986889692584, 31.321916425543368, 32.78813352826511, 32.69493177387914, 32.333789556633896, 32.37024011642008, 32.06400109494191, 31.87892471421335, 31.819116101773165, 31.618979683495812, 31.534701857282503, 31.534701857282503, 31.310560864438695, 31.253761124094652, 31.253761124094652, 31.823381377259206, 31.823381377259206, 31.823381377259206, 31.76165298219748, 31.955118219935553, 31.608405121206328, 31.608405121206328, 32.04997276688453, 31.89918730650155, 31.715394491129782, 31.631360877684404, 31.631360877684404, 31.562961424880033, 31.404231266149875, 31.24798126614987, 31.10424359279773, 31.53099942726232, 31.385437674061272, 30.94718083848519, 30.808330174689623, 30.808330174689623, 30.549707096344154, 30.549707096344154, 29.98949681733348, 29.91788877713247, 29.73474835678523, 29.4000152911044, 29.32889153577012, 28.8864174523497, 28.44427231563154, 28.051352790440454, 27.85549792067254, 27.386363636363637, 27.341318591318593, 26.890622632934402, 26.80886863813693, 26.623850863205966, 26.149341850561363, 26.02856890370146, 25.816399874646635, 25.532278214246535, 25.71513941371808, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.516669362447082, 19.05083364977509, 19.529349314388906, 20.018565445394714, 20.47463582565153, 21.023830190168074, 22.192632019186835, 22.54209857106126, 22.766035988361313, 23.060141924471196, 23.137641358588276, 23.470758937626407, 23.93780330920805, 23.988095238095237, 24.135221145936214, 24.752164502164504, 25.15638220097423, 25.300448114106054, 25.60353322072072, 25.914965385181908, 25.91849281678742, 26.181562390793445, 26.01129729135186, 26.338837860725683, 26.958724561364356, 27.126165960111837, 27.221311606228138, 27.465896672570373, 28.16446829604724, 26.82052793365833, 27.05499994232389, 27.159405549276993, 27.42607347430983, 28.393832149660835, 28.785937827175356, 29.11466077648618, 29.044653976160827, 29.81020733652313, 30.05710283454186, 30.149630541871918, 30.930079510497734, 30.681617835995713, 30.791986889692584, 31.321916425543368, 32.78813352826511, 32.69493177387914, 32.333789556633896, 32.37024011642008, 32.06400109494191, 31.87892471421335, 31.819116101773165, 31.618979683495812, 31.534701857282503, 31.534701857282503, 31.310560864438695, 31.253761124094652, 31.253761124094652, 31.823381377259206, 31.823381377259206, 31.823381377259206, 31.76165298219748, 31.955118219935553, 31.608405121206328, 31.608405121206328, 32.04997276688453, 31.89918730650155, 31.715394491129782, 31.631360877684404, 31.631360877684404, 31.562961424880033, 31.404231266149875, 31.24798126614987, 31.10424359279773, 31.53099942726232, 31.385437674061272, 30.94718083848519, 30.808330174689623, 30.808330174689623, 30.549707096344154, 30.549707096344154, 29.98949681733348, 29.91788877713247, 29.73474835678523, 29.4000152911044, 29.32889153577012, 28.8864174523497, 28.44427231563154, 28.051352790440454, 27.85549792067254, 27.386363636363637, 27.341318591318593, 26.890622632934402, 26.80886863813693, 26.623850863205966, 26.149341850561363, 26.02856890370146, 25.816399874646635, 25.532278214246535, 25.71513941371808, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 23.238272529281183, "exact_area100": 28.18567624070939}, "test": {"score": [18.129624718318375, 18.403616782813277, 18.92614316741908, 19.22545900795615, 19.361074056836085, 19.63003723858802, 20.328764353744383, 19.992336202460496, 19.925170526881768, 20.053621762398265, 20.704741912845293, 20.81154337943367, 21.106292580976604, 21.252331709504148, 21.602566161552595, 21.79248379554339, 22.081402860130957, 22.141881396073295, 22.27568457721449, 22.6677582587694, 22.969298521111988, 23.15713035463071, 23.70059642532806, 23.95322024643278, 24.06659312542673, 24.217552495626528, 24.747232147571253, 24.91110011559429, 25.326518263818553, 25.58781516333149, 25.75520275568415, 26.6714927522255, 27.0135705438763, 27.5223526421471, 27.910681307280548, 28.037871010569937, 28.17449559952682, 28.73654953499451, 29.17547316881545, 29.46984997192741, 29.67924732160576, 29.802527272713654, 29.860887841007873, 30.038254265574686, 29.798507009023435, 29.9648658140793, 29.84385884962839, 29.82121245698599, 29.770679457531056, 29.857072184449883, 30.50688253843853, 30.504324346981694, 30.34663020723101, 30.31960804906131, 30.159392947737167, 29.761794663780282, 29.69035839600458, 29.66462113058903, 29.55559201696157, 29.4788578561398, 29.284232434450907, 29.271929453028413, 29.19128490557062, 29.08953192496598, 29.03464099536829, 28.96168980319428, 29.15815311338274, 29.00099374417497, 29.18346060095493, 29.21141015461607, 29.03975238689781, 28.920765412958318, 28.82136269994499, 28.7280794418465, 28.68213514531907, 28.912025190881245, 28.89519143358662, 28.93131319424728, 28.832290247729176, 28.768900947508996, 28.670928296160064, 28.68290623964203, 28.988299536621202, 28.622525970345684, 28.39203906250835, 28.26944663787549, 28.685456360873076, 28.82316872814952, 28.575951232244652, 28.344940429626774, 28.234930317536566, 28.118722963585707, 27.88721361043734, 27.773708181473904, 27.74083369052278, 27.497077812123294, 27.278392095850023, 27.124091912728606, 27.194962518798505, 27.361496253216, 27.001009067886244], "calls": [0.0, 1.0652463382157125, 2.2969374167776295, 3.561917443408788, 4.227696404793608, 5.126498002663116, 6.391478029294275, 7.423435419440745, 8.954727030625833, 9.387483355525966, 10.85219707057257, 11.617842876165113, 12.78295605858855, 13.84820239680426, 15.14647137150466, 16.011984021304926, 17.0439414114514, 17.676431424766978, 18.209054593874836, 19.640479360852197, 21.171770972037287, 21.837549933422103, 23.00266311584554, 24.267643142476697, 24.667110519307588, 25.499334221038616, 26.364846870838882, 27.19707057256991, 27.862849533954726, 28.56191744340879, 29.6271637816245, 30.9254327563249, 32.52330226364847, 34.720372836218374, 35.61917443408788, 36.18508655126498, 36.51797603195739, 37.68308921438083, 38.68175765645805, 39.81358189081225, 40.41278295605859, 41.24500665778961, 41.6444740346205, 42.310252996005325, 43.142476697736356, 44.47403462050599, 45.173102529960055, 45.70572569906791, 47.10386151797603, 47.836218375499335, 48.63515312916112, 49.60053262316911, 52.729693741677764, 53.295605858854856, 54.427430093209054, 55.692410119840204, 57.65645805592543, 58.28894806924101, 59.42077230359521, 60.05326231691078, 61.41810918774967, 61.75099866844208, 62.5832223701731, 64.14780292942743, 64.88015978695073, 66.14513981358189, 67.50998668442078, 68.60852197070572, 69.90679094540613, 70.33954727030626, 71.03861517976033, 72.50332889480693, 73.16910785619174, 74.13448735019973, 74.63382157123834, 75.49933422103862, 75.99866844207723, 76.83089214380826, 77.5965379494008, 78.46205059920106, 79.12782956058588, 80.25965379494008, 81.19174434087883, 83.05592543275633, 84.4873501997337, 84.88681757656458, 85.98535286284952, 86.7842876165113, 87.48335552596538, 88.91478029294274, 89.31424766977364, 90.04660452729694, 91.04527296937417, 92.07723035952064, 92.50998668442078, 93.57523302263648, 95.47270306258322, 96.60452729693742, 97.70306258322236, 98.16910785619174, 100.0], "exact_score": [18.129624718318375, 18.39442127210876, 18.850102585349827, 19.191582010357422, 19.247847166001055, 19.595631609739975, 20.121998922453557, 19.9642833075005, 19.799845325747267, 19.924141055381707, 20.487116715350833, 20.746381381519896, 20.861287005588476, 21.15899757031383, 21.28491809977401, 21.570879251987762, 21.78247915551765, 22.050715352622902, 22.210352746015925, 22.517325712963995, 22.76006409160813, 22.91323754663689, 23.19640919215645, 23.70213939408893, 23.91642647585128, 24.096987260921974, 24.656331106238564, 24.85656007369391, 25.35414099824805, 25.636122328704914, 26.143244323464305, 26.6714927522255, 26.91503975832371, 27.039677459678302, 27.337705644422318, 27.64322997375979, 28.010332223931243, 28.3255036263954, 28.950364322412913, 28.98386480137231, 29.514284315211352, 29.750358027816056, 29.97591752541635, 29.78754915330707, 29.955429758779882, 29.89268995504254, 29.853825162996333, 29.81948140077279, 30.555405956385783, 30.50688253843853, 30.523674300710056, 30.481828373287446, 30.45440116395098, 30.31960804906131, 30.237891065705085, 29.827258295854236, 29.761794663780282, 29.735779907632487, 29.67745535518796, 29.568290250496194, 29.4788578561398, 29.371863898029083, 29.24090331675445, 29.172628746052993, 29.14562149903757, 29.03464099536829, 28.9222844296126, 29.247373893042898, 29.104310747917623, 29.040350609051064, 29.18346060095493, 29.09471734410401, 28.961877867869678, 28.83934243188353, 28.774975038649032, 28.64234842478442, 28.89519143358662, 28.93131319424728, 28.803854072191243, 28.692165888934813, 28.780349992673948, 28.92900184135912, 28.807722344874048, 28.66226106968343, 28.418775104559195, 28.804165914501517, 28.685456360873076, 28.695704462477124, 28.429791218108925, 28.344940429626774, 28.12724368348263, 27.925312319782186, 27.762585540554667, 27.679912110634724, 27.40651310188339, 27.317358720622114, 27.217333749841487, 27.267771364342853, 27.382436921323027, 27.182424788731748, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.254030015012226, "exact_area100": 26.72153985192856}}, "old_normalized_endpoint_budget": 45}, {"seed": 2028, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.25, "validation_area100": 0.28327291655297643, "training_seconds": 4.352487433003262, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.41438168532328246, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3771790782730625, "operator": "ge"}, {"budget": 0.03, "threshold": 0.37108935293651063, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3596568204129026, "operator": "ge"}, {"budget": 0.05, "threshold": 0.34828551660759066, "operator": "ge"}, {"budget": 0.06, "threshold": 0.31347523446297826, "operator": "ge"}, {"budget": 0.07, "threshold": 0.2879485089832723, "operator": "ge"}, {"budget": 0.08, "threshold": 0.26238220579704474, "operator": "ge"}, {"budget": 0.09, "threshold": 0.23990148445897244, "operator": "ge"}, {"budget": 0.1, "threshold": 0.22265845174912724, "operator": "ge"}, {"budget": 0.11, "threshold": 0.21589941343988286, "operator": "ge"}, {"budget": 0.12, "threshold": 0.21304045379298978, "operator": "ge"}, {"budget": 0.13, "threshold": 0.20931272709941823, "operator": "ge"}, {"budget": 0.14, "threshold": 0.20582009497493134, "operator": "ge"}, {"budget": 0.15, "threshold": 0.20189393833178795, "operator": "ge"}, {"budget": 0.16, "threshold": 0.19741231416828234, "operator": "ge"}, {"budget": 0.17, "threshold": 0.19518865947273006, "operator": "ge"}, {"budget": 0.18, "threshold": 0.19285360274375146, "operator": "ge"}, {"budget": 0.19, "threshold": 0.19044725713095695, "operator": "ge"}, {"budget": 0.2, "threshold": 0.18768484566998178, "operator": "ge"}, {"budget": 0.21, "threshold": 0.18643067207412584, "operator": "ge"}, {"budget": 0.22, "threshold": 0.1837468985853866, "operator": "ge"}, {"budget": 0.23, "threshold": 0.17836793121943004, "operator": "ge"}, {"budget": 0.24, "threshold": 0.17462160869512405, "operator": "ge"}, {"budget": 0.25, "threshold": 0.17238027959816624, "operator": "ge"}, {"budget": 0.26, "threshold": 0.17113511375484144, "operator": "ge"}, {"budget": 0.27, "threshold": 0.16539673617059802, "operator": "ge"}, {"budget": 0.28, "threshold": 0.16225472671037444, "operator": "ge"}, {"budget": 0.29, "threshold": 0.15789190630979788, "operator": "ge"}, {"budget": 0.3, "threshold": 0.15219023786539554, "operator": "ge"}, {"budget": 0.31, "threshold": 0.14447859017843115, "operator": "ge"}, {"budget": 0.32, "threshold": 0.14073306451114462, "operator": "ge"}, {"budget": 0.33, "threshold": 0.13582048910757566, "operator": "ge"}, {"budget": 0.34, "threshold": 0.1312900824958147, "operator": "ge"}, {"budget": 0.35, "threshold": 0.12591946696699288, "operator": "ge"}, {"budget": 0.36, "threshold": 0.1207631636648347, "operator": "ge"}, {"budget": 0.37, "threshold": 0.11582048644497762, "operator": "ge"}, {"budget": 0.38, "threshold": 0.10882333025183143, "operator": "ge"}, {"budget": 0.39, "threshold": 0.10463816216311791, "operator": "ge"}, {"budget": 0.4, "threshold": 0.0846329228747971, "operator": "ge"}, {"budget": 0.41, "threshold": 0.0694289938450112, "operator": "ge"}, {"budget": 0.42, "threshold": 0.059066608006786356, "operator": "ge"}, {"budget": 0.43, "threshold": 0.051898881390598006, "operator": "ge"}, {"budget": 0.44, "threshold": 0.034708927330356805, "operator": "ge"}, {"budget": 0.45, "threshold": 0.0036737146111202263, "operator": "ge"}, {"budget": 0.46, "threshold": -0.028658352427870838, "operator": "ge"}, {"budget": 0.47, "threshold": -0.044286855031602, "operator": "ge"}, {"budget": 0.48, "threshold": -0.053384446112972654, "operator": "ge"}, {"budget": 0.49, "threshold": -0.0709780877707558, "operator": "ge"}, {"budget": 0.5, "threshold": -0.07783391837601189, "operator": "ge"}, {"budget": 0.51, "threshold": -0.08209709078823785, "operator": "ge"}, {"budget": 0.52, "threshold": -0.08715243384142461, "operator": "ge"}, {"budget": 0.53, "threshold": -0.09362593230270275, "operator": "ge"}, {"budget": 0.54, "threshold": -0.09684443368042844, "operator": "ge"}, {"budget": 0.55, "threshold": -0.09945367268174395, "operator": "ge"}, {"budget": 0.56, "threshold": -0.10070383947108989, "operator": "ge"}, {"budget": 0.57, "threshold": -0.10279205110901259, "operator": "ge"}, {"budget": 0.58, "threshold": -0.10406634284249411, "operator": "ge"}, {"budget": 0.59, "threshold": -0.10723625505106585, "operator": "ge"}, {"budget": 0.6, "threshold": -0.10990266690174583, "operator": "ge"}, {"budget": 0.61, "threshold": -0.11315844909971914, "operator": "ge"}, {"budget": 0.62, "threshold": -0.11436601438463859, "operator": "ge"}, {"budget": 0.63, "threshold": -0.1168154065188721, "operator": "ge"}, {"budget": 0.64, "threshold": -0.12010508688719655, "operator": "ge"}, {"budget": 0.65, "threshold": -0.12160415910518811, "operator": "ge"}, {"budget": 0.66, "threshold": -0.12260729952517152, "operator": "ge"}, {"budget": 0.67, "threshold": -0.12416958378134244, "operator": "ge"}, {"budget": 0.68, "threshold": -0.12535782496442727, "operator": "ge"}, {"budget": 0.69, "threshold": -0.12662486041329032, "operator": "ge"}, {"budget": 0.7, "threshold": -0.12796492154302147, "operator": "ge"}, {"budget": 0.71, "threshold": -0.1297855984904357, "operator": "ge"}, {"budget": 0.72, "threshold": -0.13137412362756493, "operator": "ge"}, {"budget": 0.73, "threshold": -0.13372087667166993, "operator": "ge"}, {"budget": 0.74, "threshold": -0.1359181697001181, "operator": "ge"}, {"budget": 0.75, "threshold": -0.1376077857330246, "operator": "ge"}, {"budget": 0.76, "threshold": -0.14643837932921172, "operator": "ge"}, {"budget": 0.77, "threshold": -0.14879992063940434, "operator": "ge"}, {"budget": 0.78, "threshold": -0.15220045967501578, "operator": "ge"}, {"budget": 0.79, "threshold": -0.1532391956131991, "operator": "ge"}, {"budget": 0.8, "threshold": -0.15576257499599924, "operator": "ge"}, {"budget": 0.81, "threshold": -0.16273201718570524, "operator": "ge"}, {"budget": 0.82, "threshold": -0.16430440102569813, "operator": "ge"}, {"budget": 0.83, "threshold": -0.1680683537697161, "operator": "ge"}, {"budget": 0.84, "threshold": -0.17221712131011374, "operator": "ge"}, {"budget": 0.85, "threshold": -0.17452038970334666, "operator": "ge"}, {"budget": 0.86, "threshold": -0.17642739943698538, "operator": "ge"}, {"budget": 0.87, "threshold": -0.17871453151010094, "operator": "ge"}, {"budget": 0.88, "threshold": -0.1814461697871278, "operator": "ge"}, {"budget": 0.89, "threshold": -0.18368538879696555, "operator": "ge"}, {"budget": 0.9, "threshold": -0.18661714854005942, "operator": "ge"}, {"budget": 0.91, "threshold": -0.1891763541807938, "operator": "ge"}, {"budget": 0.92, "threshold": -0.1922384548611306, "operator": "ge"}, {"budget": 0.93, "threshold": -0.19424781432626742, "operator": "ge"}, {"budget": 0.94, "threshold": -0.19791731742888893, "operator": "ge"}, {"budget": 0.95, "threshold": -0.20150696925616463, "operator": "ge"}, {"budget": 0.96, "threshold": -0.2087485063470042, "operator": "ge"}, {"budget": 0.97, "threshold": -0.21314431236682516, "operator": "ge"}, {"budget": 0.98, "threshold": -0.22628004408801466, "operator": "ge"}, {"budget": 0.99, "threshold": -0.2413455309945057, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.577794250357417, 18.993542833592553, 19.491732612751722, 20.094788505488744, 20.55190058479532, 21.819062378922897, 22.344012893399217, 22.864668608066392, 23.415608603418306, 23.763374401099213, 23.93281069956777, 24.20404136950714, 24.62985487794648, 24.735858161235313, 24.866522366522364, 24.6841642228739, 24.87182514582894, 24.800862621173593, 25.071195692160543, 25.274443236366594, 25.545620302133788, 25.94804064086319, 26.060680507721695, 26.29854167464787, 26.684704184704184, 26.943133760596595, 27.276329414062552, 27.48998390576311, 27.60852185147521, 28.046047519731733, 28.912741581036894, 29.03032715720622, 29.14714234392555, 29.647053009121983, 29.849533617524166, 28.970250349962356, 28.929271708683473, 29.180918450287148, 29.496778665340916, 30.305642633228842, 32.359840487919314, 32.649283188076296, 32.78660848077481, 32.53777472527473, 32.436003171307036, 32.81285807463818, 32.271121186701144, 31.95680031543409, 31.918461590319215, 32.00419248250172, 32.09085523220794, 32.00419248250172, 32.47200053651667, 32.386269644334156, 32.32135565536954, 31.798343971019495, 31.697537519406588, 31.697537519406588, 31.697537519406588, 31.363031914893618, 31.28141534391534, 31.756213450292396, 31.756213450292396, 31.51947102604997, 31.51947102604997, 31.981564624665758, 31.592580213903744, 31.514320282988372, 31.445920830183994, 31.445920830183994, 31.28996721887872, 30.7636970872265, 30.489772399884764, 30.413895895436827, 30.197002658931595, 30.135952597881534, 30.364637041655673, 30.229230911379833, 29.832502152165517, 29.77219513462166, 29.640065878535772, 29.46996278575226, 29.39763143201387, 29.343175986469316, 29.289788294758967, 29.064118629908098, 28.688142791429932, 28.406139747487423, 27.924482127068334, 28.026276101767145, 27.692307692307693, 26.875853435831587, 26.792514315059307, 26.75186941599414, 26.52941622278957, 26.451367442301766, 26.272674577195854, 26.041740298296457, 25.691314951144523, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 23, 35, 47, 57, 70, 79, 90, 98, 107, 111, 118, 125, 130, 136, 139, 145, 149, 156, 162, 167, 175, 180, 186, 193, 200, 207, 213, 217, 223, 231, 234, 240, 247, 252, 257, 263, 269, 272, 275, 280, 286, 288, 292, 294, 298, 299, 299, 300, 302, 302, 302, 303, 303, 303, 303, 303, 303, 303, 303, 303, 304, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 1, 2, 2, 2, 4, 7, 8, 9, 9, 10, 11, 15, 16, 18, 20, 24, 26, 27, 28, 29, 33, 35, 36, 39, 41, 45, 50, 52, 52, 52, 56, 62, 65, 67, 70, 71, 72, 73, 78, 79, 79, 81, 84, 84, 87, 89, 90, 91, 94, 95, 96, 97, 98, 103, 104, 104, 104, 106, 107, 108, 108, 113, 113, 114, 119, 120, 121, 121, 123, 128, 130, 131, 134, 135, 137, 139, 143, 144, 146, 149, 150, 155, 156, 160, 166, 169, 174, 177, 182, 190, 192, 193, 200, 202, 206, 210, 216, 220], "exact_score": [18.254072168965788, 18.577794250357417, 18.993542833592553, 19.491732612751722, 20.094788505488744, 20.55190058479532, 21.819062378922897, 22.344012893399217, 22.864668608066392, 23.415608603418306, 23.763374401099213, 23.93281069956777, 24.20404136950714, 24.62985487794648, 24.735858161235313, 24.866522366522364, 24.6841642228739, 24.87182514582894, 24.800862621173593, 25.071195692160543, 25.274443236366594, 25.545620302133788, 25.94804064086319, 26.060680507721695, 26.29854167464787, 26.684704184704184, 26.943133760596595, 27.276329414062552, 27.48998390576311, 27.60852185147521, 28.046047519731733, 28.912741581036894, 29.03032715720622, 29.14714234392555, 29.647053009121983, 29.849533617524166, 28.970250349962356, 28.929271708683473, 29.180918450287148, 29.496778665340916, 30.305642633228842, 32.359840487919314, 32.649283188076296, 32.78660848077481, 32.53777472527473, 32.436003171307036, 32.81285807463818, 32.271121186701144, 31.95680031543409, 31.918461590319215, 32.00419248250172, 32.09085523220794, 32.00419248250172, 32.47200053651667, 32.386269644334156, 32.32135565536954, 31.798343971019495, 31.697537519406588, 31.697537519406588, 31.697537519406588, 31.363031914893618, 31.28141534391534, 31.756213450292396, 31.756213450292396, 31.51947102604997, 31.51947102604997, 31.981564624665758, 31.592580213903744, 31.514320282988372, 31.445920830183994, 31.445920830183994, 31.28996721887872, 30.7636970872265, 30.489772399884764, 30.413895895436827, 30.197002658931595, 30.135952597881534, 30.364637041655673, 30.229230911379833, 29.832502152165517, 29.77219513462166, 29.640065878535772, 29.46996278575226, 29.39763143201387, 29.343175986469316, 29.289788294758967, 29.064118629908098, 28.688142791429932, 28.406139747487423, 27.924482127068334, 28.026276101767145, 27.692307692307693, 26.875853435831587, 26.792514315059307, 26.75186941599414, 26.52941622278957, 26.451367442301766, 26.272674577195854, 26.041740298296457, 25.691314951144523, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 28.327291655297643, "threshold_area100": 28.331230685395454, "threshold_area25": 23.3851257531237}, "test": {"score": [18.129624718318375, 18.654903797574654, 18.469044290648547, 18.572773137508527, 18.597525406204763, 18.590534318449514, 19.380921235582115, 19.610186465308495, 19.93725781097111, 20.03727116007527, 20.558509313851616, 20.83137227301265, 20.9871291717073, 21.12860777198729, 21.30499148346543, 21.498067494054553, 21.87460656278987, 22.114290027301184, 22.325521436421056, 22.613227255183155, 22.88519937773669, 23.00741797857491, 23.52324184329529, 24.018408084811796, 24.458286450167886, 24.866760755870065, 25.182752547802302, 25.737431577794563, 26.010050838562528, 26.522912938556875, 26.796885701098773, 27.07594163289556, 27.4130923746731, 27.597399986834702, 27.650309786532695, 27.85650190819393, 28.579358810560322, 28.898801471585706, 29.194254367371812, 29.335968118282967, 29.478406542915142, 29.71689754534108, 29.54246536549675, 29.555916545050277, 29.695333295253324, 29.790343750051377, 29.232570406205237, 29.604322978347685, 29.905273326127084, 29.730271269357907, 30.028743377165362, 30.344478366813394, 30.3118981150858, 30.02494484911172, 30.292917662471332, 30.169941800536073, 30.08561529571604, 30.011025223739868, 30.011025223739868, 30.086065468399685, 29.90581949834287, 29.90257479207539, 29.79990577588321, 29.671139341062197, 29.619454228547, 29.63667295272845, 29.5807209189186, 29.507088098071232, 29.458091289972614, 29.44600569854194, 29.427710396457336, 29.576791438883767, 29.43994177865146, 29.340707870338857, 29.628527639458994, 29.587336441335115, 29.17784299584029, 29.0630043247828, 29.184363152148073, 29.184363152148073, 29.042326875570367, 28.244481669677178, 28.467197942018046, 28.38832985883468, 28.266074717661095, 28.186887741594617, 28.168702617026515, 28.672585033971178, 28.549385404627703, 28.425538380264882, 28.312063839437375, 28.075826488971074, 28.13670820888833, 28.064037191264763, 27.919274861774774, 27.768842490149286, 27.61284045232937, 27.500360390422866, 27.27820694090693, 27.18278905369069, 27.001009067886244], "calls": [0.0, 0.6324900133155792, 1.8308921438082555, 2.1637816245006656, 2.762982689747004, 3.5286284953395475, 5.9254327563249, 7.023968042609853, 7.989347536617843, 8.55525965379494, 9.85352862849534, 11.284953395472703, 12.050599201065246, 12.78295605858855, 13.81491344873502, 14.81358189081225, 16.74434087882823, 17.676431424766978, 18.442077230359523, 19.607190412782955, 20.938748335552596, 21.40479360852197, 22.503328894806923, 24.63382157123835, 25.76564580559254, 26.564580559254324, 27.06391478029294, 28.961384820239683, 29.66045272969374, 30.392809587217045, 31.391478029294273, 32.456724367509985, 33.288948069241016, 33.95472703062583, 34.62050599201065, 35.58588548601864, 36.484687083888154, 37.017310252996005, 38.348868175765645, 38.715046604527295, 40.37949400798935, 41.24500665778961, 41.87749667110519, 42.27696404793608, 43.04260985352863, 44.27430093209054, 45.30625832223702, 45.87217043941412, 46.504660452729695, 48.50199733688416, 49.40079893475366, 50.13315579227696, 51.69773635153129, 54.460719041278296, 55.992010652463385, 56.99067909454061, 57.58988015978696, 58.45539280958722, 59.15446071904128, 60.5525965379494, 62.017310252996005, 63.78162450066578, 64.51398135818907, 65.5126498002663, 66.84420772303595, 67.57656458055925, 68.3422103861518, 69.04127829560586, 69.44074567243675, 69.94007989347537, 70.30625832223701, 71.30492676431425, 72.03728362183755, 72.86950732356857, 73.7017310252996, 74.13448735019973, 77.13049267643143, 77.89613848202397, 78.82822902796272, 79.06125166444741, 79.9600532623169, 82.49001331557922, 83.08921438082557, 84.05459387483354, 85.81890812250333, 86.45139813581892, 87.15046604527296, 87.98268974700399, 88.58189081225034, 89.51398135818907, 90.5126498002663, 91.27829560585884, 92.47669773635153, 92.90945406125167, 93.6750998668442, 94.74034620505992, 95.93874833555259, 96.57123834886818, 98.10252996005326, 98.80159786950732, 100.0], "rescued": [0, 40, 97, 110, 142, 176, 282, 330, 370, 392, 428, 467, 487, 506, 534, 563, 611, 636, 658, 691, 722, 734, 760, 810, 834, 854, 869, 919, 940, 960, 979, 1004, 1023, 1039, 1052, 1069, 1086, 1095, 1120, 1127, 1153, 1173, 1186, 1194, 1208, 1224, 1234, 1241, 1243, 1246, 1248, 1250, 1252, 1252, 1254, 1254, 1254, 1254, 1254, 1255, 1255, 1257, 1257, 1257, 1261, 1262, 1262, 1262, 1263, 1263, 1263, 1265, 1265, 1266, 1268, 1268, 1269, 1269, 1270, 1270, 1270, 1271, 1273, 1273, 1274, 1274, 1274, 1278, 1278, 1278, 1279, 1279, 1280, 1280, 1280, 1280, 1280, 1280, 1281, 1282, 1282], "harmed": [0, 0, 7, 11, 14, 21, 32, 40, 46, 49, 54, 66, 70, 74, 83, 90, 101, 106, 111, 119, 130, 134, 138, 155, 166, 177, 180, 200, 204, 212, 217, 228, 230, 235, 243, 251, 252, 261, 266, 267, 279, 286, 294, 299, 302, 308, 316, 321, 325, 334, 336, 338, 344, 355, 359, 364, 367, 371, 371, 374, 383, 390, 395, 401, 409, 415, 423, 428, 431, 432, 433, 446, 453, 460, 463, 466, 491, 497, 505, 505, 513, 552, 559, 569, 594, 598, 600, 615, 621, 629, 640, 653, 673, 680, 688, 703, 717, 726, 751, 761, 777], "exact_score": [18.129624718318375, 18.61709593886889, 18.52228065234317, 18.689093447113358, 18.862052908895965, 19.084461448629526, 19.39526432221436, 19.59361140065425, 19.93725781097111, 20.064410267225497, 20.559058855788386, 20.769022354257906, 20.967832047106654, 21.160619595918057, 21.35677297244043, 21.53541233074394, 21.708257614102276, 21.92908882940126, 22.18767678405834, 22.456924403986044, 22.688003617695625, 22.895745152603006, 23.369244325318462, 23.714261386402626, 23.852930321492906, 24.361634613273335, 24.4889696656001, 25.139731067866606, 25.461862698320232, 25.7526591116876, 26.456880693718883, 26.6442508506797, 26.9641188758395, 27.30499269908805, 27.59180915769783, 27.75836073193662, 28.250660113462413, 28.883947044966575, 29.14206549216494, 29.413945153075588, 29.475668012178026, 29.62637119226522, 29.57376417537202, 29.734542006863148, 29.71005964058963, 29.569045975157216, 29.62374304060406, 29.903776875804848, 29.83679934287407, 29.703722064292386, 30.344478366813394, 30.357710067350958, 30.292548161357434, 30.190478675341726, 30.066633331626353, 29.965982584960777, 30.292917662471332, 30.169941800536073, 30.08561529571604, 30.011025223739868, 30.09903788040349, 30.067145775958203, 29.90581949834287, 29.94721473699442, 29.90257479207539, 29.74138351966892, 29.71818645802256, 29.600946439181737, 29.599100774596778, 29.507088098071232, 29.44600569854194, 29.565572685347764, 29.475663460159012, 29.305341231931454, 29.587336441335115, 29.342713812305444, 29.265549597210196, 29.189317070169345, 29.026232859439904, 29.184363152148073, 29.042326875570367, 28.693843398114392, 28.36116812545615, 28.448940220091508, 28.38832985883468, 28.420162585482966, 28.22628471514958, 28.168702617026515, 28.672585033971178, 28.499996557486334, 28.394208852156144, 28.222836570820885, 27.98427964089239, 28.064037191264763, 27.911433364571575, 27.696519881007557, 27.589488170394922, 27.520176579492546, 27.326019862521, 27.17594540714116, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.830284115887682, "threshold_area100": 26.862820122065962, "threshold_area25": 21.006100648237318}, "selected_budget": 46, "selected": {"val": {"score": 32.81285807463818, "calls": 45.93437945791726, "rescued": 298, "harmed": 84}, "test": {"score": 29.232570406205237, "calls": 45.30625832223702, "rescued": 1234, "harmed": 316}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.36257338523864746, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3378974199295044, "operator": "ge"}, {"budget": 0.03, "threshold": 0.32755300402641296, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3179941177368164, "operator": "ge"}, {"budget": 0.05, "threshold": 0.3086601793766022, "operator": "ge"}, {"budget": 0.06, "threshold": 0.28300443291664124, "operator": "ge"}, {"budget": 0.07, "threshold": 0.25784993171691895, "operator": "ge"}, {"budget": 0.08, "threshold": 0.23073849081993103, "operator": "ge"}, {"budget": 0.09, "threshold": 0.21446815133094788, "operator": "ge"}, {"budget": 0.1, "threshold": 0.1755557805299759, "operator": "ge"}, {"budget": 0.11, "threshold": 0.17155185341835022, "operator": "ge"}, {"budget": 0.12, "threshold": 0.16526135802268982, "operator": "ge"}, {"budget": 0.13, "threshold": 0.16298779845237732, "operator": "ge"}, {"budget": 0.14, "threshold": 0.16073936223983765, "operator": "ge"}, {"budget": 0.15, "threshold": 0.1566319316625595, "operator": "ge"}, {"budget": 0.16, "threshold": 0.15580923855304718, "operator": "ge"}, {"budget": 0.17, "threshold": 0.15246950089931488, "operator": "ge"}, {"budget": 0.18, "threshold": 0.14987695217132568, "operator": "ge"}, {"budget": 0.19, "threshold": 0.14893034100532532, "operator": "ge"}, {"budget": 0.2, "threshold": 0.14671674370765686, "operator": "ge"}, {"budget": 0.21, "threshold": 0.14337237179279327, "operator": "ge"}, {"budget": 0.22, "threshold": 0.1419830024242401, "operator": "ge"}, {"budget": 0.23, "threshold": 0.13930650055408478, "operator": "ge"}, {"budget": 0.24, "threshold": 0.13821056485176086, "operator": "ge"}, {"budget": 0.25, "threshold": 0.1366189569234848, "operator": "ge"}, {"budget": 0.26, "threshold": 0.13525938987731934, "operator": "ge"}, {"budget": 0.27, "threshold": 0.13224899768829346, "operator": "ge"}, {"budget": 0.28, "threshold": 0.13045895099639893, "operator": "ge"}, {"budget": 0.29, "threshold": 0.1260211020708084, "operator": "ge"}, {"budget": 0.3, "threshold": 0.1219017282128334, "operator": "ge"}, {"budget": 0.31, "threshold": 0.1172659620642662, "operator": "ge"}, {"budget": 0.32, "threshold": 0.11249668896198273, "operator": "ge"}, {"budget": 0.33, "threshold": 0.10760686546564102, "operator": "ge"}, {"budget": 0.34, "threshold": 0.1048518493771553, "operator": "ge"}, {"budget": 0.35, "threshold": 0.10247613489627838, "operator": "ge"}, {"budget": 0.36, "threshold": 0.09670776128768921, "operator": "ge"}, {"budget": 0.37, "threshold": 0.09134246408939362, "operator": "ge"}, {"budget": 0.38, "threshold": 0.08390402793884277, "operator": "ge"}, {"budget": 0.39, "threshold": 0.08181420713663101, "operator": "ge"}, {"budget": 0.4, "threshold": 0.06829439103603363, "operator": "ge"}, {"budget": 0.41, "threshold": 0.05633844435214996, "operator": "ge"}, {"budget": 0.42, "threshold": 0.04687471687793732, "operator": "ge"}, {"budget": 0.43, "threshold": 0.040903620421886444, "operator": "ge"}, {"budget": 0.44, "threshold": 0.0269756019115448, "operator": "ge"}, {"budget": 0.45, "threshold": 0.00278405100107193, "operator": "ge"}, {"budget": 0.46, "threshold": -0.017963441088795662, "operator": "ge"}, {"budget": 0.47, "threshold": -0.025895606726408005, "operator": "ge"}, {"budget": 0.48, "threshold": -0.03053750842809677, "operator": "ge"}, {"budget": 0.49, "threshold": -0.034778643399477005, "operator": "ge"}, {"budget": 0.5, "threshold": -0.037443965673446655, "operator": "ge"}, {"budget": 0.51, "threshold": -0.039858002215623856, "operator": "ge"}, {"budget": 0.52, "threshold": -0.043290380388498306, "operator": "ge"}, {"budget": 0.53, "threshold": -0.04479266703128815, "operator": "ge"}, {"budget": 0.54, "threshold": -0.04630490019917488, "operator": "ge"}, {"budget": 0.55, "threshold": -0.04747401922941208, "operator": "ge"}, {"budget": 0.56, "threshold": -0.04894972965121269, "operator": "ge"}, {"budget": 0.57, "threshold": -0.050046950578689575, "operator": "ge"}, {"budget": 0.58, "threshold": -0.051579590886831284, "operator": "ge"}, {"budget": 0.59, "threshold": -0.05335037037730217, "operator": "ge"}, {"budget": 0.6, "threshold": -0.055508218705654144, "operator": "ge"}, {"budget": 0.61, "threshold": -0.05689837411046028, "operator": "ge"}, {"budget": 0.62, "threshold": -0.05770818516612053, "operator": "ge"}, {"budget": 0.63, "threshold": -0.05907577648758888, "operator": "ge"}, {"budget": 0.64, "threshold": -0.06091047823429108, "operator": "ge"}, {"budget": 0.65, "threshold": -0.062082476913928986, "operator": "ge"}, {"budget": 0.66, "threshold": -0.06295967102050781, "operator": "ge"}, {"budget": 0.67, "threshold": -0.06361529231071472, "operator": "ge"}, {"budget": 0.68, "threshold": -0.06418365240097046, "operator": "ge"}, {"budget": 0.69, "threshold": -0.06557980179786682, "operator": "ge"}, {"budget": 0.7, "threshold": -0.06607134640216827, "operator": "ge"}, {"budget": 0.71, "threshold": -0.06715989857912064, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0683446153998375, "operator": "ge"}, {"budget": 0.73, "threshold": -0.06950148195028305, "operator": "ge"}, {"budget": 0.74, "threshold": -0.07069478929042816, "operator": "ge"}, {"budget": 0.75, "threshold": -0.07276506721973419, "operator": "ge"}, {"budget": 0.76, "threshold": -0.07854803651571274, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0806751698255539, "operator": "ge"}, {"budget": 0.78, "threshold": -0.08340266346931458, "operator": "ge"}, {"budget": 0.79, "threshold": -0.08415579050779343, "operator": "ge"}, {"budget": 0.8, "threshold": -0.08779232203960419, "operator": "ge"}, {"budget": 0.81, "threshold": -0.09210678935050964, "operator": "ge"}, {"budget": 0.82, "threshold": -0.09250827878713608, "operator": "ge"}, {"budget": 0.83, "threshold": -0.09557878971099854, "operator": "ge"}, {"budget": 0.84, "threshold": -0.09763003140687943, "operator": "ge"}, {"budget": 0.85, "threshold": -0.10005791485309601, "operator": "ge"}, {"budget": 0.86, "threshold": -0.10066606104373932, "operator": "ge"}, {"budget": 0.87, "threshold": -0.10272324830293655, "operator": "ge"}, {"budget": 0.88, "threshold": -0.10441628843545914, "operator": "ge"}, {"budget": 0.89, "threshold": -0.10704664885997772, "operator": "ge"}, {"budget": 0.9, "threshold": -0.10947511345148087, "operator": "ge"}, {"budget": 0.91, "threshold": -0.11109668761491776, "operator": "ge"}, {"budget": 0.92, "threshold": -0.11380542069673538, "operator": "ge"}, {"budget": 0.93, "threshold": -0.11573690921068192, "operator": "ge"}, {"budget": 0.94, "threshold": -0.11801697313785553, "operator": "ge"}, {"budget": 0.95, "threshold": -0.12107200920581818, "operator": "ge"}, {"budget": 0.96, "threshold": -0.12551432847976685, "operator": "ge"}, {"budget": 0.97, "threshold": -0.12973567843437195, "operator": "ge"}, {"budget": 0.98, "threshold": -0.13789010047912598, "operator": "ge"}, {"budget": 0.99, "threshold": -0.15303033590316772, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.577794250357417, 18.993542833592553, 19.505494505494504, 20.01466030036054, 21.07082610372084, 21.729492650545282, 22.449428311314634, 22.864668608066392, 23.67851446798815, 23.74925361994327, 24.188153695452964, 24.40133004739746, 24.520056890746545, 24.855365227027644, 25.106276226965885, 24.972820793510447, 25.09567896067369, 25.259125520874964, 25.310430627621006, 25.252789695398683, 25.493135358592177, 25.635365075550137, 25.909454487282034, 26.23015873015873, 26.63080079974745, 26.694746694746698, 26.815585475316194, 27.0023193033163, 27.473569084943495, 28.195415695415694, 28.38225201612431, 28.862840102213806, 29.42490147423733, 29.900981171848418, 29.886348453089273, 28.817645231438338, 28.88826735513483, 29.02021646319432, 29.54085741938146, 30.305642633228842, 32.27568070285694, 32.84484930626145, 32.76146267525578, 32.53777472527473, 32.436003171307036, 32.48576581206205, 32.31896538348151, 31.95680031543409, 31.95680031543409, 31.749735898257896, 31.749735898257896, 31.749735898257896, 31.749735898257896, 31.551769616285746, 31.551769616285746, 31.354544998674722, 31.354544998674722, 31.3813150708312, 31.436694817666645, 31.18467868863438, 31.564622241086592, 31.483332629623373, 31.483332629623373, 31.064721922107317, 31.540365656224456, 31.540365656224456, 31.292729120473023, 30.989110198954762, 30.83695842915593, 30.83695842915593, 30.83695842915593, 30.254292698511765, 30.489772399884764, 30.413895895436827, 30.197002658931595, 30.197002658931595, 30.48035052292462, 30.27196456647744, 30.011961722488035, 29.951654704944175, 29.76113261138009, 29.453186987569424, 29.39763143201387, 29.39763143201387, 29.289788294758967, 29.043589407886337, 28.75611942702013, 28.22833035915172, 27.899902164608047, 27.827260458839408, 27.624820124228126, 27.16561796494329, 26.833836629108898, 26.75186941599414, 26.52941622278957, 26.490077119721118, 26.217417334438608, 26.041740298296457, 25.665150070600163, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 23, 35, 45, 58, 70, 80, 90, 100, 107, 114, 119, 126, 133, 138, 143, 148, 152, 158, 164, 169, 175, 181, 188, 194, 199, 205, 210, 217, 225, 230, 235, 240, 246, 251, 257, 263, 267, 272, 275, 280, 284, 288, 292, 294, 297, 299, 299, 299, 299, 299, 300, 300, 300, 300, 300, 300, 301, 302, 302, 303, 303, 303, 303, 304, 304, 304, 304, 304, 304, 304, 304, 305, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 1, 2, 2, 3, 5, 7, 7, 10, 10, 10, 12, 12, 12, 18, 23, 23, 25, 29, 31, 34, 36, 38, 39, 43, 46, 48, 51, 51, 54, 55, 61, 61, 65, 68, 70, 71, 71, 73, 77, 78, 79, 81, 84, 85, 87, 89, 89, 91, 91, 92, 92, 94, 94, 97, 97, 98, 102, 103, 105, 106, 106, 112, 113, 113, 116, 120, 122, 122, 122, 128, 130, 131, 134, 134, 137, 140, 142, 143, 146, 149, 150, 154, 156, 160, 163, 170, 174, 179, 183, 187, 191, 193, 196, 201, 206, 210, 216, 220], "exact_score": [18.254072168965788, 18.577794250357417, 18.993542833592553, 19.505494505494504, 20.01466030036054, 21.07082610372084, 21.729492650545282, 22.449428311314634, 22.864668608066392, 23.67851446798815, 23.74925361994327, 24.188153695452964, 24.40133004739746, 24.520056890746545, 24.855365227027644, 25.106276226965885, 24.972820793510447, 25.09567896067369, 25.259125520874964, 25.310430627621006, 25.252789695398683, 25.493135358592177, 25.635365075550137, 25.909454487282034, 26.23015873015873, 26.63080079974745, 26.694746694746698, 26.815585475316194, 27.0023193033163, 27.473569084943495, 28.195415695415694, 28.38225201612431, 28.862840102213806, 29.42490147423733, 29.900981171848418, 29.886348453089273, 28.817645231438338, 28.88826735513483, 29.02021646319432, 29.54085741938146, 30.305642633228842, 32.27568070285694, 32.84484930626145, 32.76146267525578, 32.53777472527473, 32.436003171307036, 32.48576581206205, 32.31896538348151, 31.95680031543409, 31.95680031543409, 31.749735898257896, 31.749735898257896, 31.749735898257896, 31.749735898257896, 31.551769616285746, 31.551769616285746, 31.354544998674722, 31.354544998674722, 31.3813150708312, 31.436694817666645, 31.18467868863438, 31.564622241086592, 31.483332629623373, 31.483332629623373, 31.064721922107317, 31.540365656224456, 31.540365656224456, 31.292729120473023, 30.989110198954762, 30.83695842915593, 30.83695842915593, 30.83695842915593, 30.254292698511765, 30.489772399884764, 30.413895895436827, 30.197002658931595, 30.197002658931595, 30.48035052292462, 30.27196456647744, 30.011961722488035, 29.951654704944175, 29.76113261138009, 29.453186987569424, 29.39763143201387, 29.39763143201387, 29.289788294758967, 29.043589407886337, 28.75611942702013, 28.22833035915172, 27.899902164608047, 27.827260458839408, 27.624820124228126, 27.16561796494329, 26.833836629108898, 26.75186941599414, 26.52941622278957, 26.490077119721118, 26.217417334438608, 26.041740298296457, 25.665150070600163, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 28.246735264169427, "threshold_area100": 28.25057790290886, "threshold_area25": 23.524316426077057}, "test": {"score": [18.129624718318375, 18.543431989115195, 18.505510570501947, 18.781024644518897, 18.788160266746477, 18.912094049281038, 19.29049375014565, 19.633953769778977, 19.97720287000909, 20.003238244889154, 20.462961812172193, 20.970819317808182, 21.18069621193785, 21.352908440463008, 21.51636987328051, 21.807806369898472, 21.88481081453058, 22.32415231946949, 22.53353293604916, 22.62620873486703, 22.896328495259315, 23.516551068887708, 23.81185558023182, 24.772767907509497, 24.9270610601243, 25.12508465305124, 25.382501508788696, 25.855353715784247, 26.11174733040747, 26.275025967756093, 26.923801881263152, 27.127099555449934, 27.37968334523651, 27.428276519907303, 27.52511967001504, 28.236675416363894, 28.731000927056456, 28.92064353257026, 29.319110448895636, 29.299209204246313, 29.48355311158386, 29.767970423309404, 29.540646563137653, 29.59787000662772, 29.695333295253324, 29.804041771612805, 29.20050524513435, 29.20989754899656, 29.594822492157626, 29.915388754950083, 30.09064315467766, 29.986056916416675, 30.110526425565094, 29.971018319237974, 30.21561483758048, 30.150362177855822, 30.068504363273295, 30.046952639135366, 29.994022898260514, 29.793819769438677, 29.91525595958541, 29.80076691535644, 29.7223659870128, 29.61512270374243, 29.57926273864314, 29.65524551855904, 29.575510606104626, 29.513701266278584, 29.50156932320035, 29.414575674737094, 29.396382057872334, 29.509661813738507, 29.42815257748881, 29.336834733893557, 29.260912806091273, 29.535748668243777, 29.186796690974898, 29.126241993348906, 29.276183183024436, 29.2649258516874, 28.885569733055856, 28.62019185981522, 28.572732731893907, 28.686451860282634, 28.458434862933984, 28.472327900148585, 28.454064977989724, 28.348745871366788, 28.527601643165486, 28.41975534774021, 28.27567449226729, 28.15331816414412, 28.14971842585164, 28.068837645567214, 27.890201198471875, 27.829291783436805, 27.62024639683416, 27.516266309477132, 27.331311608435854, 27.189657185558826, 27.001009067886244], "calls": [0.0, 0.7989347536617843, 1.7976031957390146, 2.4966711051930757, 3.129161118508655, 3.8282290279627165, 5.625832223701731, 7.090545938748336, 8.089214380825565, 8.488681757656458, 10.51930758988016, 11.451398135818907, 12.916111850865514, 13.781624500665778, 14.647137150466044, 15.978695073235686, 16.4780292942743, 18.10918774966711, 19.77363515312916, 20.139813581890813, 21.304926764314246, 23.40213049267643, 24.30093209054594, 25.96537949400799, 26.531291611185086, 27.19707057256991, 27.82956058588549, 28.928095872170438, 29.6271637816245, 30.326231691078565, 31.25832223701731, 32.19041278295606, 33.38881491344873, 34.25432756324901, 34.78695073235686, 35.28628495339547, 36.284953395472705, 37.017310252996005, 38.48202396804261, 38.78162450066578, 40.246338215712385, 41.37816245006658, 41.87749667110519, 42.310252996005325, 43.04260985352863, 44.30758988015979, 45.27296937416778, 45.83888149134487, 46.8375499334221, 48.10252996005326, 49.63382157123835, 50.96537949400799, 53.728362183754996, 54.793608521970704, 55.89214380825565, 56.55792276964048, 57.856191744340876, 58.5219707057257, 59.720372836218374, 61.118508655126504, 62.749667110519304, 63.914780292942744, 64.51398135818907, 65.47936085219706, 66.57789613848203, 67.44340878828228, 68.20905459387482, 68.70838881491345, 69.00798934753661, 69.70705725699068, 69.94007989347537, 70.87217043941412, 71.53794940079894, 72.13715046604527, 72.76964047936086, 73.96804260985353, 76.69773635153129, 77.56324900133156, 78.52862849533955, 78.66178428761651, 80.5925432756325, 82.68974700399467, 82.9560585885486, 84.28761651131825, 85.15312916111851, 86.38482023968042, 86.6844207723036, 87.88282290279628, 88.51531291611185, 89.51398135818907, 90.5792276964048, 91.41145139813582, 92.41011984021304, 93.1757656458056, 93.84154460719041, 94.57390146471371, 95.87217043941412, 96.6711051930759, 97.86950732356857, 98.90146471371504, 100.0], "rescued": [0, 50, 100, 135, 166, 197, 271, 334, 374, 389, 458, 483, 519, 541, 562, 597, 611, 654, 694, 703, 731, 783, 807, 852, 862, 877, 893, 923, 942, 959, 984, 1001, 1026, 1041, 1049, 1060, 1079, 1094, 1123, 1128, 1151, 1174, 1185, 1194, 1208, 1225, 1233, 1236, 1240, 1243, 1246, 1246, 1247, 1249, 1251, 1251, 1251, 1251, 1251, 1251, 1253, 1254, 1254, 1255, 1257, 1259, 1259, 1259, 1259, 1259, 1259, 1260, 1261, 1263, 1264, 1267, 1269, 1269, 1270, 1270, 1270, 1272, 1272, 1274, 1274, 1275, 1275, 1276, 1278, 1278, 1279, 1279, 1280, 1280, 1280, 1280, 1280, 1280, 1281, 1282, 1282], "harmed": [0, 1, 6, 11, 15, 19, 32, 41, 47, 49, 59, 68, 79, 84, 89, 98, 103, 109, 128, 130, 138, 153, 164, 182, 187, 190, 198, 204, 207, 213, 219, 225, 235, 243, 246, 248, 253, 258, 265, 270, 278, 286, 291, 297, 302, 308, 317, 319, 321, 325, 335, 340, 349, 355, 360, 362, 366, 367, 370, 380, 381, 389, 392, 398, 404, 409, 418, 422, 423, 428, 429, 438, 446, 452, 459, 466, 488, 492, 500, 501, 520, 549, 552, 568, 583, 596, 598, 612, 622, 629, 641, 650, 670, 679, 695, 699, 716, 725, 745, 760, 777], "exact_score": [18.129624718318375, 18.643110486690905, 18.588491984333714, 18.732120168711464, 18.70268295813164, 19.08543760210975, 19.417599808712975, 19.630274418693126, 19.93518925776223, 20.047303050104674, 20.31689907597663, 20.917148325256356, 21.079035764606584, 21.180989383761702, 21.377289349185126, 21.56606678167989, 21.807806369898472, 22.01366401556969, 22.26538027227539, 22.44195019224528, 22.58033700348078, 22.83937603874851, 23.020090238400094, 23.29167437583019, 23.70665590171129, 23.93832153142029, 24.772767907509497, 25.052523082034195, 25.442974236312583, 25.88166347726293, 26.187472822323972, 26.51025900255229, 27.122541710551623, 27.305890178183223, 27.31491080038573, 27.59646903391944, 28.356399404577655, 28.92064353257026, 29.166273667705582, 29.35601119318576, 29.528283348679963, 29.613525754131626, 29.596344820533304, 29.682212727877438, 29.72360977609099, 29.693469517461217, 29.23525687659915, 29.594822492157626, 29.93502740381104, 29.84378927899237, 30.05811896348551, 29.986056916416675, 29.825186928982824, 30.129649269048777, 30.08351245802515, 29.958060731274465, 30.21561483758048, 30.09035751012644, 30.046952639135366, 30.046952639135366, 29.994022898260514, 29.814503718694052, 29.91525595958541, 29.89668642173056, 29.742500215200717, 29.63978394300377, 29.564618915967827, 29.587777126508026, 29.60613235109427, 29.50156932320035, 29.396382057872334, 29.409906749138017, 29.377944703061083, 29.260912806091273, 29.535748668243777, 29.386856615596425, 29.213077788178822, 29.157666632485835, 29.037597787272613, 29.25372174650326, 29.052592182453168, 28.796567751991997, 28.453980079599393, 28.572732731893907, 28.755900569120502, 28.458434862933984, 28.254306172050498, 28.40591362492757, 28.39484971028697, 28.470888339667678, 28.388246008786027, 28.23330654852394, 28.01916488536207, 28.092823573822635, 27.860961432390006, 27.768842490149286, 27.61284045232937, 27.500360390422866, 27.301395005797037, 27.156669650705616, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.795116963108086, "threshold_area100": 26.905487594495963, "threshold_area25": 21.141309082459163}, "selected_budget": 42, "selected": {"val": {"score": 32.84484930626145, "calls": 41.940085592011414, "rescued": 284, "harmed": 78}, "test": {"score": 29.540646563137653, "calls": 41.87749667110519, "rescued": 1185, "harmed": 291}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.499683380126953, "operator": "ge"}, {"budget": 0.02, "threshold": 2.322355270385742, "operator": "ge"}, {"budget": 0.03, "threshold": 2.2204337120056152, "operator": "ge"}, {"budget": 0.04, "threshold": 2.1401891708374023, "operator": "ge"}, {"budget": 0.05, "threshold": 2.0405402183532715, "operator": "ge"}, {"budget": 0.06, "threshold": 1.9485456943511963, "operator": "ge"}, {"budget": 0.07, "threshold": 1.8548309803009033, "operator": "ge"}, {"budget": 0.08, "threshold": 1.8049383163452148, "operator": "ge"}, {"budget": 0.09, "threshold": 1.759049415588379, "operator": "ge"}, {"budget": 0.1, "threshold": 1.704017162322998, "operator": "ge"}, {"budget": 0.11, "threshold": 1.6606910228729248, "operator": "ge"}, {"budget": 0.12, "threshold": 1.6291099786758423, "operator": "ge"}, {"budget": 0.13, "threshold": 1.5965338945388794, "operator": "ge"}, {"budget": 0.14, "threshold": 1.5348321199417114, "operator": "ge"}, {"budget": 0.15, "threshold": 1.4725661277770996, "operator": "ge"}, {"budget": 0.16, "threshold": 1.4365274906158447, "operator": "ge"}, {"budget": 0.17, "threshold": 1.3727549314498901, "operator": "ge"}, {"budget": 0.18, "threshold": 1.3621375560760498, "operator": "ge"}, {"budget": 0.19, "threshold": 1.3291966915130615, "operator": "ge"}, {"budget": 0.2, "threshold": 1.275852918624878, "operator": "ge"}, {"budget": 0.21, "threshold": 1.2421663999557495, "operator": "ge"}, {"budget": 0.22, "threshold": 1.215759515762329, "operator": "ge"}, {"budget": 0.23, "threshold": 1.1984808444976807, "operator": "ge"}, {"budget": 0.24, "threshold": 1.164886474609375, "operator": "ge"}, {"budget": 0.25, "threshold": 1.139054775238037, "operator": "ge"}, {"budget": 0.26, "threshold": 1.1109371185302734, "operator": "ge"}, {"budget": 0.27, "threshold": 1.0858144760131836, "operator": "ge"}, {"budget": 0.28, "threshold": 1.0559802055358887, "operator": "ge"}, {"budget": 0.29, "threshold": 1.0262086391448975, "operator": "ge"}, {"budget": 0.3, "threshold": 0.9961552023887634, "operator": "ge"}, {"budget": 0.31, "threshold": 0.98511803150177, "operator": "ge"}, {"budget": 0.32, "threshold": 0.9657154083251953, "operator": "ge"}, {"budget": 0.33, "threshold": 0.9562556743621826, "operator": "ge"}, {"budget": 0.34, "threshold": 0.9074990749359131, "operator": "ge"}, {"budget": 0.35, "threshold": 0.8753395676612854, "operator": "ge"}, {"budget": 0.36, "threshold": 0.8534667491912842, "operator": "ge"}, {"budget": 0.37, "threshold": 0.8288371562957764, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7928162813186646, "operator": "ge"}, {"budget": 0.39, "threshold": 0.7509881258010864, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6870537400245667, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5768749117851257, "operator": "ge"}, {"budget": 0.42, "threshold": 0.49105381965637207, "operator": "ge"}, {"budget": 0.43, "threshold": 0.370427668094635, "operator": "ge"}, {"budget": 0.44, "threshold": 0.26865845918655396, "operator": "ge"}, {"budget": 0.45, "threshold": 0.033308327198028564, "operator": "ge"}, {"budget": 0.46, "threshold": -0.4166622757911682, "operator": "ge"}, {"budget": 0.47, "threshold": -0.5662239193916321, "operator": "ge"}, {"budget": 0.48, "threshold": -0.876389741897583, "operator": "ge"}, {"budget": 0.49, "threshold": -1.1401395797729492, "operator": "ge"}, {"budget": 0.5, "threshold": -1.3140963315963745, "operator": "ge"}, {"budget": 0.51, "threshold": -1.5368452072143555, "operator": "ge"}, {"budget": 0.52, "threshold": -2.9123647212982178, "operator": "ge"}, {"budget": 0.53, "threshold": -3.08864688873291, "operator": "ge"}, {"budget": 0.54, "threshold": -3.110262155532837, "operator": "ge"}, {"budget": 0.55, "threshold": -3.1816225051879883, "operator": "ge"}, {"budget": 0.56, "threshold": -3.210136890411377, "operator": "ge"}, {"budget": 0.57, "threshold": -3.2475030422210693, "operator": "ge"}, {"budget": 0.58, "threshold": -3.2819743156433105, "operator": "ge"}, {"budget": 0.59, "threshold": -3.3073248863220215, "operator": "ge"}, {"budget": 0.6, "threshold": -3.3284504413604736, "operator": "ge"}, {"budget": 0.61, "threshold": -3.3499255180358887, "operator": "ge"}, {"budget": 0.62, "threshold": -3.368107318878174, "operator": "ge"}, {"budget": 0.63, "threshold": -3.3844122886657715, "operator": "ge"}, {"budget": 0.64, "threshold": -3.394352674484253, "operator": "ge"}, {"budget": 0.65, "threshold": -3.405998706817627, "operator": "ge"}, {"budget": 0.66, "threshold": -3.4266977310180664, "operator": "ge"}, {"budget": 0.67, "threshold": -3.4327805042266846, "operator": "ge"}, {"budget": 0.68, "threshold": -3.442146062850952, "operator": "ge"}, {"budget": 0.69, "threshold": -3.4579126834869385, "operator": "ge"}, {"budget": 0.7, "threshold": -3.4903810024261475, "operator": "ge"}, {"budget": 0.71, "threshold": -3.498537302017212, "operator": "ge"}, {"budget": 0.72, "threshold": -3.515758514404297, "operator": "ge"}, {"budget": 0.73, "threshold": -3.529651165008545, "operator": "ge"}, {"budget": 0.74, "threshold": -3.548436164855957, "operator": "ge"}, {"budget": 0.75, "threshold": -3.564058542251587, "operator": "ge"}, {"budget": 0.76, "threshold": -3.5883607864379883, "operator": "ge"}, {"budget": 0.77, "threshold": -3.6129417419433594, "operator": "ge"}, {"budget": 0.78, "threshold": -3.664200782775879, "operator": "ge"}, {"budget": 0.79, "threshold": -3.6976351737976074, "operator": "ge"}, {"budget": 0.8, "threshold": -3.7483606338500977, "operator": "ge"}, {"budget": 0.81, "threshold": -3.850067377090454, "operator": "ge"}, {"budget": 0.82, "threshold": -11.383857727050781, "operator": "ge"}, {"budget": 0.83, "threshold": -11.49301528930664, "operator": "ge"}, {"budget": 0.84, "threshold": -11.580784797668457, "operator": "ge"}, {"budget": 0.85, "threshold": -11.60324478149414, "operator": "ge"}, {"budget": 0.86, "threshold": -11.626802444458008, "operator": "ge"}, {"budget": 0.87, "threshold": -11.651589393615723, "operator": "ge"}, {"budget": 0.88, "threshold": -11.682388305664062, "operator": "ge"}, {"budget": 0.89, "threshold": -11.694304466247559, "operator": "ge"}, {"budget": 0.9, "threshold": -11.718650817871094, "operator": "ge"}, {"budget": 0.91, "threshold": -11.731147766113281, "operator": "ge"}, {"budget": 0.92, "threshold": -11.74159049987793, "operator": "ge"}, {"budget": 0.93, "threshold": -11.751596450805664, "operator": "ge"}, {"budget": 0.94, "threshold": -11.764939308166504, "operator": "ge"}, {"budget": 0.95, "threshold": -11.790252685546875, "operator": "ge"}, {"budget": 0.96, "threshold": -11.841817855834961, "operator": "ge"}, {"budget": 0.97, "threshold": -11.866107940673828, "operator": "ge"}, {"budget": 0.98, "threshold": -11.929412841796875, "operator": "ge"}, {"budget": 0.99, "threshold": -12.036128997802734, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.580444872651164, 18.967170703421203, 19.480906708401537, 19.966583377283616, 20.450660817481705, 21.30952380952381, 21.576339431518498, 21.803740014220867, 22.05499704316972, 22.593974827070912, 22.89918297509841, 23.35091586365933, 23.751820577133277, 23.940546734664384, 24.176737266130505, 24.26717918097229, 24.423484942116755, 24.678414594541167, 24.852163087457207, 25.247749039398983, 25.541033173744797, 25.804816876245447, 26.106138361777457, 26.422275000102545, 26.4712531434518, 27.03857284020328, 27.316962316962318, 27.572240184252134, 28.07751049039991, 28.548203722822514, 28.780036584351876, 29.19337606837607, 28.269872542318364, 28.31366318187592, 28.463563248498176, 28.95422624969264, 28.923644763197004, 29.295581141824588, 29.46182990838012, 29.40738451163314, 30.392416225749564, 32.16847498493068, 32.19150265542018, 32.53777472527473, 32.75458301636312, 32.63512316143895, 32.271121186701144, 32.259620324136456, 32.47200053651667, 32.55866328622289, 32.83351964780985, 32.574663239985824, 32.251091498884314, 32.18497336782349, 31.995844760370417, 31.482810210063505, 31.234209753231497, 30.87067878408617, 30.739219603651293, 30.45313507414571, 30.56465010246523, 30.84967320261438, 30.529578167900823, 30.339805825242717, 29.911732515456706, 29.911732515456706, 29.76892482980672, 29.188865235985656, 29.188865235985656, 29.188865235985656, 28.950892857142858, 28.552892417340164, 28.348669098524248, 27.971829757544043, 27.858180755453404, 27.780497139634196, 27.903703446974514, 27.65925278028477, 27.473230561940237, 27.473230561940237, 27.434520884520886, 27.270163020163018, 27.270163020163018, 27.149390073303113, 27.149390073303113, 26.962860878389616, 26.962860878389616, 26.962860878389616, 26.78404822344664, 26.578249184385793, 26.442612405402354, 26.442612405402354, 26.14301257253385, 26.041740298296457, 25.977420731607225, 25.977420731607225, 25.913707953282984, 25.731882231504155, 25.697184105202975, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 23, 35, 44, 54, 64, 72, 79, 85, 93, 101, 110, 117, 124, 131, 136, 141, 146, 153, 160, 165, 172, 178, 184, 191, 199, 206, 212, 218, 225, 230, 236, 240, 244, 249, 254, 258, 262, 267, 273, 279, 283, 287, 292, 296, 298, 299, 301, 302, 303, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 2, 2, 2, 3, 5, 6, 10, 10, 11, 12, 13, 14, 17, 18, 19, 19, 22, 22, 22, 24, 26, 27, 33, 34, 38, 40, 41, 42, 44, 47, 49, 53, 55, 55, 60, 63, 69, 76, 77, 77, 79, 81, 83, 85, 87, 92, 95, 95, 97, 99, 103, 104, 107, 112, 116, 121, 123, 125, 128, 130, 133, 136, 141, 141, 144, 148, 148, 148, 154, 159, 163, 168, 170, 172, 175, 178, 181, 185, 186, 188, 188, 189, 189, 191, 191, 191, 194, 197, 199, 199, 204, 206, 207, 207, 208, 211, 216, 220], "exact_score": [18.254072168965788, 18.580444872651164, 18.967170703421203, 19.480906708401537, 19.966583377283616, 20.450660817481705, 21.30952380952381, 21.576339431518498, 21.803740014220867, 22.05499704316972, 22.593974827070912, 22.89918297509841, 23.35091586365933, 23.751820577133277, 23.940546734664384, 24.176737266130505, 24.26717918097229, 24.423484942116755, 24.678414594541167, 24.852163087457207, 25.247749039398983, 25.541033173744797, 25.804816876245447, 26.106138361777457, 26.422275000102545, 26.4712531434518, 27.03857284020328, 27.316962316962318, 27.572240184252134, 28.07751049039991, 28.548203722822514, 28.780036584351876, 29.19337606837607, 28.269872542318364, 28.31366318187592, 28.463563248498176, 28.95422624969264, 28.923644763197004, 29.295581141824588, 29.46182990838012, 29.40738451163314, 30.392416225749564, 32.16847498493068, 32.19150265542018, 32.53777472527473, 32.75458301636312, 32.63512316143895, 32.271121186701144, 32.259620324136456, 32.47200053651667, 32.55866328622289, 32.83351964780985, 32.574663239985824, 32.251091498884314, 32.18497336782349, 31.995844760370417, 31.482810210063505, 31.234209753231497, 30.87067878408617, 30.739219603651293, 30.45313507414571, 30.56465010246523, 30.84967320261438, 30.529578167900823, 30.339805825242717, 29.911732515456706, 29.911732515456706, 29.76892482980672, 29.188865235985656, 29.188865235985656, 29.188865235985656, 28.950892857142858, 28.552892417340164, 28.348669098524248, 27.971829757544043, 27.858180755453404, 27.780497139634196, 27.903703446974514, 27.65925278028477, 27.473230561940237, 27.473230561940237, 27.434520884520886, 27.270163020163018, 27.270163020163018, 27.149390073303113, 27.149390073303113, 26.962860878389616, 26.962860878389616, 26.962860878389616, 26.78404822344664, 26.578249184385793, 26.442612405402354, 26.442612405402354, 26.14301257253385, 26.041740298296457, 25.977420731607225, 25.977420731607225, 25.913707953282984, 25.731882231504155, 25.697184105202975, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.510778291892578, "threshold_area100": 27.513546682075475, "threshold_area25": 22.96635692125818}, "test": {"score": [18.129624718318375, 18.66467009437208, 18.67152513314057, 18.789897174930058, 18.946424015287306, 19.28268589839667, 19.290836493701267, 20.200180641312247, 20.25595067619502, 20.019609887963945, 20.01085073228023, 20.310304575864336, 20.45950120787674, 20.629145196138936, 20.886743874479606, 21.073301896964043, 21.341667850336755, 21.88507371519095, 22.192224380410835, 22.4794658382105, 22.87671588875322, 23.227027707778433, 23.571078087169116, 23.77108463979533, 24.501434350231264, 24.932924893258807, 25.368826275790536, 25.51306544083768, 25.73215745249694, 26.11603907721321, 26.626448196987685, 26.752869599236668, 26.792304124538514, 26.817891307230045, 27.288306638061798, 27.464037742115654, 27.68923790399694, 28.162584887269702, 28.3647444623148, 28.66393174058757, 29.02995897093027, 29.534508500623748, 29.382050700827733, 29.6067069670397, 29.65325942856778, 29.781461126451518, 29.62835160418176, 29.619870623931472, 29.96383809927145, 29.85538178460153, 29.885988642032157, 30.002321319938236, 29.722795149392038, 29.88957597943455, 29.865161751364973, 29.504291729648024, 29.358684518853117, 29.20185571219272, 29.097000817419072, 29.268085962986124, 29.18440846189465, 29.487345374369355, 29.529784855985504, 29.675235174489323, 29.63332347810893, 29.481121246475507, 29.25862069289814, 29.22121666023443, 29.300694599393466, 29.48612134374942, 29.366272141505938, 29.273314200180028, 29.163020108140064, 29.001042636241998, 28.793770399323044, 28.622442676405946, 28.36557432799685, 28.23076916967681, 28.266502589297428, 28.106733963810797, 28.00678722369183, 27.884278749987224, 27.715059809581216, 27.828956612292277, 27.736444360824592, 27.709458177750324, 27.60724539295968, 27.5526293945131, 27.479484001742392, 27.47223224782982, 27.38462083445259, 27.32314359397097, 27.519206154758326, 27.46306368520934, 27.430278338421083, 27.423806068008755, 27.348278726998508, 27.27212249903843, 27.231377362715612, 27.16010468395627, 27.001009067886244], "calls": [0.0, 0.7656458055925432, 1.3315579227696404, 2.1970705725699067, 3.062583222370173, 4.061251664447404, 5.093209054593874, 6.391478029294275, 7.123834886817576, 7.789613848202396, 9.187749667110518, 10.5525965379494, 11.051930758988016, 11.78428761651132, 12.849533954727029, 14.347536617842877, 15.013315579227696, 17.177097203728362, 17.609853528628495, 18.841544607190414, 20.838881491344875, 22.270306258322236, 23.33555259653795, 24.101198402130493, 25.532623169107854, 26.73102529960053, 27.896138482023968, 28.86151797603196, 29.82689747003995, 30.9254327563249, 32.19041278295606, 32.62316910785619, 33.15579227696405, 33.48868175765646, 35.11984021304927, 35.78561917443409, 36.41810918774967, 36.95073235685752, 37.782956058588546, 38.84820239680426, 39.913448735019976, 41.11185086551265, 41.711051930758984, 42.57656458055926, 43.04260985352863, 44.34087882822903, 45.60585885486019, 45.938748335552596, 46.504660452729695, 47.802929427430094, 49.10119840213049, 50.466045272969374, 51.53129161118508, 53.59520639147804, 54.02796271637816, 56.29161118508655, 57.35685752330226, 58.854860186418115, 60.35286284953395, 61.51797603195739, 62.68308921438083, 63.715046604527295, 64.64713715046605, 65.61251664447404, 66.34487350199734, 67.01065246338216, 68.57523302263648, 68.94141145139814, 69.77363515312916, 70.73901464713715, 72.70306258322236, 73.43541944074568, 74.20106524633822, 75.13315579227697, 76.06524633821572, 76.66444740346205, 77.72969374167776, 78.39547270306258, 79.72703062583223, 80.45938748335553, 80.89214380825565, 81.6577896138482, 82.98934753661784, 83.85486018641811, 85.61917443408788, 86.25166444740347, 87.4167776298269, 88.94806924101198, 90.34620505992011, 90.84553928095872, 92.14380825565912, 92.74300932090546, 93.1757656458056, 93.57523302263648, 94.10785619174435, 95.33954727030626, 96.63781624500666, 97.13715046604527, 98.1358189081225, 99.20106524633822, 100.0], "rescued": [0, 42, 69, 108, 146, 186, 229, 297, 318, 337, 380, 425, 442, 460, 490, 528, 546, 606, 618, 652, 699, 737, 767, 784, 823, 850, 877, 899, 925, 951, 982, 994, 1006, 1013, 1052, 1065, 1077, 1089, 1106, 1125, 1143, 1164, 1177, 1195, 1204, 1225, 1240, 1243, 1248, 1248, 1251, 1254, 1255, 1258, 1258, 1258, 1258, 1259, 1261, 1262, 1263, 1265, 1267, 1269, 1270, 1270, 1271, 1271, 1272, 1274, 1275, 1275, 1275, 1276, 1276, 1277, 1277, 1277, 1278, 1278, 1278, 1278, 1278, 1279, 1280, 1280, 1280, 1280, 1280, 1280, 1280, 1280, 1281, 1281, 1281, 1282, 1282, 1282, 1282, 1282, 1282], "harmed": [0, 0, 3, 5, 12, 17, 25, 34, 38, 43, 51, 62, 63, 66, 72, 86, 88, 98, 101, 111, 123, 131, 134, 138, 144, 156, 163, 172, 187, 200, 214, 217, 222, 227, 240, 244, 247, 251, 259, 268, 275, 285, 289, 295, 300, 309, 319, 322, 325, 331, 337, 346, 358, 373, 375, 399, 408, 422, 435, 447, 466, 477, 487, 501, 505, 512, 530, 533, 534, 542, 567, 571, 577, 589, 602, 616, 630, 639, 653, 663, 669, 677, 690, 697, 704, 705, 710, 715, 720, 721, 727, 731, 733, 738, 741, 746, 754, 758, 761, 766, 777], "exact_score": [18.129624718318375, 18.658430999348212, 18.723791886462617, 18.939344968756732, 19.258616358523504, 19.24643346935533, 19.641130700996655, 20.21871867196832, 19.800767074005453, 19.977635198630278, 20.19107109793824, 20.446837931485096, 20.63332821560265, 20.917036524492303, 21.019860311634666, 21.331084582167286, 21.599623589428518, 21.864254977168635, 22.27011928120719, 22.511844459390247, 22.75258972957645, 22.89820115951849, 23.16588197093378, 23.4791544027422, 23.756914304900086, 24.333008540153806, 24.89074808809154, 24.995547176319747, 25.37797930624017, 25.540781627090453, 25.720710859794544, 26.14709470992859, 26.476598340854324, 26.782711545789585, 26.978235468490453, 27.253256350959383, 27.506193078324227, 28.179926096554368, 28.410135531308338, 28.640486892326233, 29.059046782364046, 29.547219608070396, 29.47734348071373, 29.65325942856778, 29.72505187700434, 29.51884633001553, 29.619870623931472, 29.916110529540802, 29.85538178460153, 29.885988642032157, 29.737496953185055, 29.874882555597654, 29.69263455261505, 30.010227539069433, 29.865161751364973, 29.710140669219953, 29.64267284877972, 29.421623598836273, 29.30179076794026, 29.20185571219272, 29.07877847745679, 29.034160961628828, 29.36448283517971, 29.248702469687117, 29.468800284080487, 29.731290890979356, 29.656069549242098, 29.481121246475507, 29.426979624512477, 29.22121666023443, 29.28106195807269, 29.4407694619337, 29.49959907404354, 29.366272141505938, 29.186024478970523, 29.001042636241998, 28.84386164789026, 28.543751373989885, 28.318082409076766, 28.352603161690794, 28.16085562557746, 27.977302431637206, 27.778735383524587, 27.715059809581216, 27.812516247055964, 27.763713121732092, 27.736444360824592, 27.6500009563338, 27.599858599858596, 27.5526293945131, 27.51292677749273, 27.456085853857804, 27.400629151892357, 27.29129773250617, 27.430278338421083, 27.460645007807226, 27.384637233080944, 27.305565959618338, 27.231377362715612, 27.16010468395627, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.50876534890685, "threshold_area100": 26.52035128000386, "threshold_area25": 20.954965844436554}, "selected_budget": 51, "selected": {"val": {"score": 32.83351964780985, "calls": 50.92724679029958, "rescued": 304, "harmed": 97}, "test": {"score": 30.002321319938236, "calls": 50.466045272969374, "rescued": 1254, "harmed": 346}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.3346339464187622, "operator": "ge"}, {"budget": 0.02, "threshold": 1.2787621021270752, "operator": "ge"}, {"budget": 0.03, "threshold": 1.1928962469100952, "operator": "ge"}, {"budget": 0.04, "threshold": 1.1429812908172607, "operator": "ge"}, {"budget": 0.05, "threshold": 1.1147050857543945, "operator": "ge"}, {"budget": 0.06, "threshold": 1.0996522903442383, "operator": "ge"}, {"budget": 0.07, "threshold": 1.0670967102050781, "operator": "ge"}, {"budget": 0.08, "threshold": 1.0480902194976807, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0114362239837646, "operator": "ge"}, {"budget": 0.1, "threshold": 0.9769895076751709, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9546025395393372, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9300914406776428, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9153347611427307, "operator": "ge"}, {"budget": 0.14, "threshold": 0.899898886680603, "operator": "ge"}, {"budget": 0.15, "threshold": 0.8881884813308716, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8702774047851562, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8455886840820312, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8200161457061768, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8054714202880859, "operator": "ge"}, {"budget": 0.2, "threshold": 0.7823399305343628, "operator": "ge"}, {"budget": 0.21, "threshold": 0.767507016658783, "operator": "ge"}, {"budget": 0.22, "threshold": 0.749751091003418, "operator": "ge"}, {"budget": 0.23, "threshold": 0.7299788594245911, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7170473337173462, "operator": "ge"}, {"budget": 0.25, "threshold": 0.703154981136322, "operator": "ge"}, {"budget": 0.26, "threshold": 0.687949538230896, "operator": "ge"}, {"budget": 0.27, "threshold": 0.6668555736541748, "operator": "ge"}, {"budget": 0.28, "threshold": 0.6514136791229248, "operator": "ge"}, {"budget": 0.29, "threshold": 0.629612147808075, "operator": "ge"}, {"budget": 0.3, "threshold": 0.6003066301345825, "operator": "ge"}, {"budget": 0.31, "threshold": 0.5703026652336121, "operator": "ge"}, {"budget": 0.32, "threshold": 0.543185830116272, "operator": "ge"}, {"budget": 0.33, "threshold": 0.5323420763015747, "operator": "ge"}, {"budget": 0.34, "threshold": 0.5082319974899292, "operator": "ge"}, {"budget": 0.35, "threshold": 0.49966609477996826, "operator": "ge"}, {"budget": 0.36, "threshold": 0.47534042596817017, "operator": "ge"}, {"budget": 0.37, "threshold": 0.4608129858970642, "operator": "ge"}, {"budget": 0.38, "threshold": 0.4413457214832306, "operator": "ge"}, {"budget": 0.39, "threshold": 0.4087059199810028, "operator": "ge"}, {"budget": 0.4, "threshold": 0.33452895283699036, "operator": "ge"}, {"budget": 0.41, "threshold": 0.29158544540405273, "operator": "ge"}, {"budget": 0.42, "threshold": 0.24494892358779907, "operator": "ge"}, {"budget": 0.43, "threshold": 0.21231889724731445, "operator": "ge"}, {"budget": 0.44, "threshold": 0.14145886898040771, "operator": "ge"}, {"budget": 0.45, "threshold": 0.016882330179214478, "operator": "ge"}, {"budget": 0.46, "threshold": -0.23698633909225464, "operator": "ge"}, {"budget": 0.47, "threshold": -0.3573624789714813, "operator": "ge"}, {"budget": 0.48, "threshold": -0.535602867603302, "operator": "ge"}, {"budget": 0.49, "threshold": -0.7364258170127869, "operator": "ge"}, {"budget": 0.5, "threshold": -0.9437620639801025, "operator": "ge"}, {"budget": 0.51, "threshold": -1.2210640907287598, "operator": "ge"}, {"budget": 0.52, "threshold": -2.6742959022521973, "operator": "ge"}, {"budget": 0.53, "threshold": -2.8396360874176025, "operator": "ge"}, {"budget": 0.54, "threshold": -2.8932652473449707, "operator": "ge"}, {"budget": 0.55, "threshold": -2.9160261154174805, "operator": "ge"}, {"budget": 0.56, "threshold": -2.9466471672058105, "operator": "ge"}, {"budget": 0.57, "threshold": -2.9741287231445312, "operator": "ge"}, {"budget": 0.58, "threshold": -3.014936923980713, "operator": "ge"}, {"budget": 0.59, "threshold": -3.0367016792297363, "operator": "ge"}, {"budget": 0.6, "threshold": -3.055617094039917, "operator": "ge"}, {"budget": 0.61, "threshold": -3.082225799560547, "operator": "ge"}, {"budget": 0.62, "threshold": -3.0914220809936523, "operator": "ge"}, {"budget": 0.63, "threshold": -3.113330841064453, "operator": "ge"}, {"budget": 0.64, "threshold": -3.13034725189209, "operator": "ge"}, {"budget": 0.65, "threshold": -3.1451053619384766, "operator": "ge"}, {"budget": 0.66, "threshold": -3.1522321701049805, "operator": "ge"}, {"budget": 0.67, "threshold": -3.172201633453369, "operator": "ge"}, {"budget": 0.68, "threshold": -3.203927516937256, "operator": "ge"}, {"budget": 0.69, "threshold": -3.212517738342285, "operator": "ge"}, {"budget": 0.7, "threshold": -3.2208404541015625, "operator": "ge"}, {"budget": 0.71, "threshold": -3.228907585144043, "operator": "ge"}, {"budget": 0.72, "threshold": -3.252606153488159, "operator": "ge"}, {"budget": 0.73, "threshold": -3.263774871826172, "operator": "ge"}, {"budget": 0.74, "threshold": -3.280294418334961, "operator": "ge"}, {"budget": 0.75, "threshold": -3.3042380809783936, "operator": "ge"}, {"budget": 0.76, "threshold": -3.3378419876098633, "operator": "ge"}, {"budget": 0.77, "threshold": -3.355386257171631, "operator": "ge"}, {"budget": 0.78, "threshold": -3.383418083190918, "operator": "ge"}, {"budget": 0.79, "threshold": -3.4291186332702637, "operator": "ge"}, {"budget": 0.8, "threshold": -3.487936019897461, "operator": "ge"}, {"budget": 0.81, "threshold": -3.667825937271118, "operator": "ge"}, {"budget": 0.82, "threshold": -10.926399230957031, "operator": "ge"}, {"budget": 0.83, "threshold": -11.007805824279785, "operator": "ge"}, {"budget": 0.84, "threshold": -11.112709999084473, "operator": "ge"}, {"budget": 0.85, "threshold": -11.160842895507812, "operator": "ge"}, {"budget": 0.86, "threshold": -11.202862739562988, "operator": "ge"}, {"budget": 0.87, "threshold": -11.262591361999512, "operator": "ge"}, {"budget": 0.88, "threshold": -11.292799949645996, "operator": "ge"}, {"budget": 0.89, "threshold": -11.32455825805664, "operator": "ge"}, {"budget": 0.9, "threshold": -11.346904754638672, "operator": "ge"}, {"budget": 0.91, "threshold": -11.376693725585938, "operator": "ge"}, {"budget": 0.92, "threshold": -11.399821281433105, "operator": "ge"}, {"budget": 0.93, "threshold": -11.419938087463379, "operator": "ge"}, {"budget": 0.94, "threshold": -11.442366600036621, "operator": "ge"}, {"budget": 0.95, "threshold": -11.461973190307617, "operator": "ge"}, {"budget": 0.96, "threshold": -11.488533973693848, "operator": "ge"}, {"budget": 0.97, "threshold": -11.523651123046875, "operator": "ge"}, {"budget": 0.98, "threshold": -11.587774276733398, "operator": "ge"}, {"budget": 0.99, "threshold": -11.664440155029297, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.446927397004202, 18.877241929998124, 19.3028239216231, 19.67959576655229, 19.93708136126169, 20.21548344128989, 20.406748788243647, 20.837590742125673, 21.149179767600817, 22.156064274708342, 22.502556419057104, 22.5487012987013, 22.779978848910158, 23.275750395425852, 23.705174690396365, 23.884595198605968, 24.27306729406087, 24.407505512156675, 24.37988874385964, 24.75228415280798, 25.11809702632123, 25.453977659860012, 25.703924581369307, 25.95243640258113, 26.315640880858272, 26.954059156702332, 27.271557271557274, 27.79919983444851, 28.004021682112256, 28.289951081584906, 28.535846771140893, 27.288065047127972, 28.07517756058723, 28.911123224119102, 28.89182371977917, 29.19748429228714, 29.274147715444375, 29.96686746987952, 30.305642633228842, 30.370713623725678, 30.655235569028672, 32.39943436597132, 32.62929439240409, 32.53777472527473, 32.436003171307036, 32.63512316143895, 32.271121186701144, 32.17388943195395, 32.214007941567544, 32.451136404502456, 32.83351964780985, 32.40441234392847, 32.3366616664217, 31.91120470334975, 31.651101309718566, 31.512096774193548, 31.20455718172436, 31.09603846224525, 30.94139928698752, 30.53947642886942, 30.08338784456839, 29.77109368618802, 29.647644671009154, 29.41668941668941, 29.117872330613224, 28.987044353050223, 29.09957952169994, 28.895865462897387, 28.619178559524894, 28.424786827278048, 28.273332064424327, 28.273332064424327, 27.828964621030284, 27.79044921486782, 27.715880926013813, 28.017414028242417, 27.944348346039682, 27.619913677216324, 27.548305637015314, 27.548305637015314, 27.434520884520886, 27.434520884520886, 27.270163020163018, 27.078494550915057, 26.93879894029819, 26.93879894029819, 26.785669954686885, 26.714546199352608, 26.645725567651645, 26.645725567651645, 26.4667607539109, 26.429851305092, 26.362363737012434, 26.362363737012434, 26.19171463016256, 25.960081585081586, 25.795595009828396, 25.760896883527217, 25.697184105202975, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 9, 21, 32, 41, 48, 56, 63, 72, 79, 89, 96, 100, 107, 114, 122, 128, 136, 143, 146, 153, 161, 167, 174, 181, 188, 195, 204, 211, 216, 223, 229, 233, 241, 249, 253, 258, 262, 269, 272, 277, 280, 285, 289, 292, 294, 298, 299, 301, 302, 303, 304, 304, 304, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 2, 2, 7, 8, 9, 9, 9, 9, 9, 12, 13, 14, 14, 16, 18, 21, 23, 24, 27, 29, 31, 35, 36, 36, 40, 41, 43, 47, 51, 55, 55, 59, 60, 63, 66, 69, 70, 76, 79, 79, 80, 81, 84, 85, 87, 89, 93, 96, 97, 101, 102, 108, 112, 114, 121, 123, 126, 130, 133, 138, 140, 142, 145, 147, 150, 152, 156, 162, 164, 164, 170, 171, 172, 173, 174, 179, 184, 184, 186, 186, 188, 190, 192, 192, 194, 195, 196, 196, 198, 199, 200, 200, 203, 207, 210, 211, 216, 220], "exact_score": [18.254072168965788, 18.446927397004202, 18.877241929998124, 19.3028239216231, 19.67959576655229, 19.93708136126169, 20.21548344128989, 20.406748788243647, 20.837590742125673, 21.149179767600817, 22.156064274708342, 22.502556419057104, 22.5487012987013, 22.779978848910158, 23.275750395425852, 23.705174690396365, 23.884595198605968, 24.27306729406087, 24.407505512156675, 24.37988874385964, 24.75228415280798, 25.11809702632123, 25.453977659860012, 25.703924581369307, 25.95243640258113, 26.315640880858272, 26.954059156702332, 27.271557271557274, 27.79919983444851, 28.004021682112256, 28.289951081584906, 28.535846771140893, 27.288065047127972, 28.07517756058723, 28.911123224119102, 28.89182371977917, 29.19748429228714, 29.274147715444375, 29.96686746987952, 30.305642633228842, 30.370713623725678, 30.655235569028672, 32.39943436597132, 32.62929439240409, 32.53777472527473, 32.436003171307036, 32.63512316143895, 32.271121186701144, 32.17388943195395, 32.214007941567544, 32.451136404502456, 32.83351964780985, 32.40441234392847, 32.3366616664217, 31.91120470334975, 31.651101309718566, 31.512096774193548, 31.20455718172436, 31.09603846224525, 30.94139928698752, 30.53947642886942, 30.08338784456839, 29.77109368618802, 29.647644671009154, 29.41668941668941, 29.117872330613224, 28.987044353050223, 29.09957952169994, 28.895865462897387, 28.619178559524894, 28.424786827278048, 28.273332064424327, 28.273332064424327, 27.828964621030284, 27.79044921486782, 27.715880926013813, 28.017414028242417, 27.944348346039682, 27.619913677216324, 27.548305637015314, 27.548305637015314, 27.434520884520886, 27.434520884520886, 27.270163020163018, 27.078494550915057, 26.93879894029819, 26.93879894029819, 26.785669954686885, 26.714546199352608, 26.645725567651645, 26.645725567651645, 26.4667607539109, 26.429851305092, 26.362363737012434, 26.362363737012434, 26.19171463016256, 25.960081585081586, 25.795595009828396, 25.760896883527217, 25.697184105202975, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.311715985310226, "threshold_area100": 27.31420000076944, "threshold_area25": 22.56709220081728}, "test": {"score": [18.129624718318375, 18.507421837371712, 18.544762439021973, 18.478742050790203, 18.863384568612496, 18.78182237325889, 18.923960179752548, 19.1928675687929, 19.317220918576883, 19.648416624026382, 20.068307266601373, 20.323530202753712, 20.575253663212408, 20.876827222741248, 21.209792994474974, 21.463221935377664, 21.669638559984328, 21.90224679741938, 22.27066310063392, 22.449760651081174, 23.085788436479117, 23.322485016352754, 23.524756277277664, 24.04923503826758, 24.12300431639711, 24.59353225221248, 24.805040366264254, 24.92548561569479, 25.155178361702802, 25.646102586125817, 26.33309479059791, 26.673451723640184, 27.12232909201258, 27.26337036963558, 27.539001535816325, 27.760922113573532, 28.082637278729646, 28.168608486157986, 28.4902353818446, 29.191243796685807, 29.67070660504313, 29.683941513492897, 29.401558844658283, 29.526649768507863, 29.703909771193203, 29.790343750051377, 29.606479338215003, 29.933850417556506, 29.964821950063453, 29.979805326905527, 29.841729116570043, 30.001217549337184, 29.70989210857542, 29.706010127768128, 29.532046773872, 29.447191571741417, 29.24752370708487, 29.230330434728323, 29.272867038067034, 28.905287320546464, 28.7940173500722, 28.844807166954816, 28.839004684606767, 28.969354724190904, 28.79181084198385, 29.57379912050889, 29.491530907456724, 29.21451475739237, 28.968245131686732, 28.887801859093592, 28.82973078414164, 28.79413860132571, 28.680473153156584, 28.455085153803427, 28.596271595606666, 28.46689389150153, 28.144427070622395, 28.31748369743736, 28.241778820003642, 28.05063071163915, 27.94933988652883, 27.795211566550094, 27.722613688058633, 27.679925916487136, 27.479797979797983, 27.422388123495132, 27.808546321525995, 27.746699346174452, 27.69126957214575, 27.584729907159577, 27.476560257855603, 27.428663529496266, 27.384267964512798, 27.377350466137607, 27.309231236014718, 27.302411944079243, 27.26896848349934, 27.26800739934565, 27.15340523673857, 27.104949495361915, 27.001009067886244], "calls": [0.0, 0.9320905459387484, 1.3315579227696404, 1.9973368841544608, 2.796271637816245, 3.3288948069241013, 3.9280958721704398, 4.860186418109188, 5.59254327563249, 7.623169107856191, 9.121171770972039, 10.086551264980026, 11.151797603195739, 12.21704394141145, 13.1491344873502, 14.08122503328895, 14.84687083888149, 16.378162450066576, 17.976031957390145, 18.608521970705727, 20.07323568575233, 20.838881491344875, 21.73768308921438, 22.869507323568573, 23.601864181091877, 24.467376830892142, 25.299600532623167, 26.2982689747004, 27.496671105193077, 29.127829560585884, 30.9254327563249, 32.256990679094535, 33.75499334221039, 34.15446071904128, 35.01997336884155, 35.41944074567244, 36.451398135818906, 36.98402130492676, 37.71637816245006, 38.8814913448735, 40.21304926764314, 41.17842876165113, 41.711051930758984, 42.143808255659124, 43.075898801597866, 44.27430093209054, 45.67243675099867, 46.00532623169108, 46.37150466045273, 47.636484687083886, 49.10119840213049, 50.53262316910786, 51.53129161118508, 53.96138482023968, 55.35952063914781, 55.825565912117185, 56.8242343541944, 57.9560585885486, 59.52063914780293, 60.719041278295606, 61.48468708388815, 62.88282290279628, 63.38215712383488, 64.58055925432757, 65.5126498002663, 66.34487350199734, 66.97736351531292, 68.30892143808256, 70.57256990679095, 71.07190412782957, 71.73768308921437, 72.237017310253, 73.63515312916113, 74.63382157123834, 75.43275632490013, 76.39813581890812, 77.79627163781625, 78.56191744340879, 79.42743009320905, 80.3595206391478, 81.12516644474034, 81.89081225033289, 82.62316910785619, 83.22237017310253, 84.92010652463382, 85.7190412782956, 86.8508655126498, 88.38215712383489, 89.21438082556591, 90.37949400798935, 91.6444740346205, 92.84287616511318, 93.97470039946738, 94.8069241011984, 95.70572569906791, 96.30492676431425, 97.00399467376832, 97.60319573901465, 98.5352862849534, 99.20106524633822, 100.0], "rescued": [0, 49, 67, 94, 128, 146, 163, 194, 217, 283, 334, 362, 395, 430, 460, 490, 516, 558, 602, 620, 665, 689, 711, 741, 757, 783, 804, 831, 863, 905, 952, 987, 1023, 1032, 1052, 1063, 1082, 1091, 1104, 1130, 1159, 1174, 1183, 1192, 1207, 1224, 1240, 1243, 1246, 1249, 1249, 1254, 1255, 1258, 1258, 1258, 1258, 1260, 1261, 1262, 1262, 1263, 1264, 1265, 1265, 1270, 1271, 1271, 1273, 1273, 1273, 1273, 1275, 1275, 1276, 1276, 1276, 1278, 1278, 1278, 1278, 1278, 1278, 1278, 1278, 1278, 1280, 1281, 1281, 1281, 1281, 1281, 1281, 1281, 1281, 1281, 1281, 1282, 1282, 1282, 1282], "harmed": [0, 2, 4, 11, 13, 17, 21, 29, 36, 43, 49, 53, 59, 64, 72, 79, 84, 94, 105, 107, 115, 119, 126, 139, 145, 149, 157, 168, 178, 195, 208, 222, 230, 233, 241, 243, 252, 255, 266, 273, 276, 288, 297, 299, 302, 308, 320, 323, 324, 331, 337, 346, 359, 390, 401, 405, 415, 425, 440, 465, 471, 487, 492, 502, 511, 519, 532, 547, 567, 573, 576, 579, 592, 606, 616, 624, 644, 650, 654, 664, 672, 682, 689, 692, 705, 709, 715, 723, 727, 732, 739, 743, 746, 747, 753, 754, 756, 759, 767, 770, 777], "exact_score": [18.129624718318375, 18.54237515137492, 18.478742050790203, 18.921712592237565, 18.94086355888333, 19.218603555480808, 19.38683594601886, 19.633501301548492, 19.747253658090095, 20.028440284660796, 20.294910846423942, 20.539409785020002, 20.808124929235337, 21.241650927209182, 21.437108952323726, 21.69984925195973, 21.92826927160448, 21.933141657311197, 22.27066310063392, 22.533755745658837, 23.085053828050217, 23.36879513056974, 23.858308002694713, 24.100167447327955, 24.32442229868593, 24.744204793408965, 24.860492769805166, 25.02253764463458, 25.229593350845846, 25.62378587106507, 26.005019822664053, 26.359195154495314, 26.596876842186884, 26.8366797338857, 27.180725720284194, 27.481958691418395, 27.982701146973653, 28.168608486157986, 28.5309015123108, 29.208934540143954, 29.579178039699165, 29.668529900498374, 29.492346820541336, 29.743051113882736, 29.71157478805555, 29.563266222106815, 29.9145675991742, 29.888709274464727, 29.979805326905527, 29.9408189721297, 30.1369453480138, 29.874882555597654, 29.699507710060768, 29.480361957262897, 29.706010127768128, 29.56588178433579, 29.43613587560108, 29.30733287939944, 29.230330434728323, 29.363423559247742, 29.18513652587445, 28.895488072704943, 28.746521641824003, 28.844807166954816, 29.033883125943944, 28.90199638735922, 29.203982588682642, 29.491530907456724, 29.251199873610762, 29.197067078790656, 29.09761742795245, 28.91453852430893, 28.820684094809156, 28.72682864797969, 28.610902621479458, 28.63853165880742, 28.531634865337562, 28.33065477489121, 28.364427897771428, 28.31748369743736, 28.117776812000137, 27.98619636502569, 27.778735383524587, 27.722613688058633, 27.58524793584034, 27.479797979797983, 27.63445700510606, 27.808546321525995, 27.71612161223407, 27.70734670550643, 27.636041295267987, 27.534493379541825, 27.435630928549998, 27.428663529496266, 27.384267964512798, 27.377350466137607, 27.309231236014718, 27.26896848349934, 27.200735336161365, 27.15340523673857, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.505145157601063, "threshold_area100": 26.42550962061184, "threshold_area25": 20.952143699396494}, "selected_budget": 51, "selected": {"val": {"score": 32.83351964780985, "calls": 50.92724679029958, "rescued": 304, "harmed": 97}, "test": {"score": 30.001217549337184, "calls": 50.53262316910786, "rescued": 1254, "harmed": 346}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 32.187620296097464, "calls": 40.94151212553495}, "test": {"score": 29.525228230782417, "calls": 41.344873501997334}}, "old_normalized_interior": {"val": {"score": 32.283988607018664, "calls": 47.93152639087018}, "test": {"score": 30.47616577775716, "calls": 47.17043941411451}}, "old_normalized_endpoints": {"val": {"score": 32.283988607018664, "calls": 47.93152639087018}, "test": {"score": 30.47616577775716, "calls": 47.17043941411451}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.54526693737273, 19.06837790151834, 19.504425351020235, 19.96642248754574, 20.91176033303598, 21.34224302828954, 21.636330003699594, 21.899564399564397, 22.2486701369372, 22.608900528190144, 23.068912962529986, 23.509816232042752, 23.796405100752928, 24.06067273528182, 24.402419719583897, 24.676361314292347, 25.536844932516633, 25.730652185147417, 26.422001780651517, 26.801876176470817, 27.064828738512947, 27.903225806451616, 28.09361315376353, 26.57487410132119, 26.987201864407616, 27.225835539695332, 27.285207700101317, 27.92292490118577, 28.20295874537053, 28.253166094913528, 28.490596672038258, 28.806632070596415, 28.771967836679668, 29.135841252388804, 29.20035265386175, 29.453847889866847, 29.81357583114429, 30.233725003649102, 30.274456666135286, 30.37298967992037, 32.187620296097464, 32.11577060199744, 31.554982817869416, 31.78526151413772, 31.71875, 31.919175332968436, 31.736196246420374, 32.283988607018664, 32.283988607018664, 32.1986546668662, 32.11422373137143, 31.99107102201183, 31.907528833206484, 31.572845302686215, 31.572845302686215, 31.167763157894733, 30.98778933590521, 30.84753787878788, 30.61079545454546, 30.53118621447637, 30.47291115620131, 30.13791526973484, 29.952419426407722, 29.952419426407722, 29.87443898760254, 29.87443898760254, 29.82301377455523, 29.846635506126272, 29.796434702913423, 29.796434702913423, 29.61922713247736, 29.2654265213238, 29.66985122157536, 29.453608761915, 29.318502625486985, 29.09090909090909, 28.920690408564564, 28.556906450523474, 28.55273720828697, 28.482015850436902, 28.361549989839464, 28.26588672453334, 28.104021978634986, 27.99012271209427, 27.946441873422167, 27.79292868396087, 27.50590243725255, 27.296780212241543, 26.987757935255885, 27.21184813576118, 26.657351657351658, 26.657351657351658, 26.141986668302458, 26.001431518672895, 25.79365079365079, 25.950088334746464, 25.761741811877293, 25.44860455974843, 25.38249879051766, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.54526693737273, 19.06837790151834, 19.504425351020235, 19.96642248754574, 20.91176033303598, 21.34224302828954, 21.636330003699594, 21.899564399564397, 22.2486701369372, 22.608900528190144, 23.068912962529986, 23.509816232042752, 23.796405100752928, 24.06067273528182, 24.402419719583897, 24.676361314292347, 25.536844932516633, 25.730652185147417, 26.422001780651517, 26.801876176470817, 27.064828738512947, 27.903225806451616, 28.09361315376353, 26.57487410132119, 26.987201864407616, 27.225835539695332, 27.285207700101317, 27.92292490118577, 28.20295874537053, 28.253166094913528, 28.490596672038258, 28.806632070596415, 28.771967836679668, 29.135841252388804, 29.20035265386175, 29.453847889866847, 29.81357583114429, 30.233725003649102, 30.274456666135286, 30.37298967992037, 32.187620296097464, 32.11577060199744, 31.554982817869416, 31.78526151413772, 31.71875, 31.919175332968436, 31.736196246420374, 32.283988607018664, 32.283988607018664, 32.1986546668662, 32.11422373137143, 31.99107102201183, 31.907528833206484, 31.572845302686215, 31.572845302686215, 31.167763157894733, 30.98778933590521, 30.84753787878788, 30.61079545454546, 30.53118621447637, 30.47291115620131, 30.13791526973484, 29.952419426407722, 29.952419426407722, 29.87443898760254, 29.87443898760254, 29.82301377455523, 29.846635506126272, 29.796434702913423, 29.796434702913423, 29.61922713247736, 29.2654265213238, 29.66985122157536, 29.453608761915, 29.318502625486985, 29.09090909090909, 28.920690408564564, 28.556906450523474, 28.55273720828697, 28.482015850436902, 28.361549989839464, 28.26588672453334, 28.104021978634986, 27.99012271209427, 27.946441873422167, 27.79292868396087, 27.50590243725255, 27.296780212241543, 26.987757935255885, 27.21184813576118, 26.657351657351658, 26.657351657351658, 26.141986668302458, 26.001431518672895, 25.79365079365079, 25.950088334746464, 25.761741811877293, 25.44860455974843, 25.38249879051766, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 23.46911875479351, "exact_area100": 27.82116297046202}, "test": {"score": [18.129624718318375, 18.44957513471071, 18.602101377690882, 19.010238728009604, 19.042521725480714, 19.079301178039795, 19.331373548199185, 19.56929976461683, 20.032957025707557, 20.167477476296714, 20.349609586232898, 20.630348106821273, 20.886445929329103, 21.21707107560935, 21.317239028712937, 21.957317012395073, 22.581850102211458, 22.775236127508855, 23.02701524545609, 23.09802298500333, 23.255254248270145, 23.315151975612917, 23.52653618094096, 23.685088572363515, 24.017279673890148, 24.16706042576295, 24.44471742158264, 25.123611676719353, 25.44702961410045, 25.78365502871994, 25.908599490359947, 26.551919388681483, 26.66180253367652, 27.12974419114848, 27.597124689666558, 28.38685182712685, 28.545128588876693, 28.773681746582515, 28.86144891292884, 28.92681210747509, 28.973667951438514, 29.59087476756414, 29.403695382073213, 29.4075296780563, 29.632689728662807, 30.071498255690454, 30.577986636395917, 30.557832784799277, 30.47616577775716, 30.365530720998635, 30.399478628519855, 30.31242964217298, 30.286158099507986, 30.034006734607743, 29.96316881143029, 30.298635197675928, 30.273035015630185, 30.219885349670516, 30.133797800411028, 30.061253042660756, 29.985150184042187, 29.75531966956385, 29.737480828266214, 29.70664367199607, 29.516347404553155, 29.46086599332344, 29.401253360161295, 29.365402252271462, 29.274568408555545, 29.14739623054227, 29.07604230438449, 29.017229034861625, 29.267021580316666, 29.221681823499623, 29.1389295611048, 28.94316634089685, 28.92903006240708, 28.71732609510768, 28.901508047094023, 29.02970627360024, 29.05221529392269, 28.925849334752197, 28.59473968755805, 28.449456164707176, 28.658708938120704, 28.500998083696476, 28.473506036492253, 28.45443689655814, 28.4937152851711, 28.33403597824626, 28.1282395633987, 28.0839336517963, 27.96235011524336, 27.863409505370207, 27.522019134219615, 27.135689611995684, 27.28042865640306, 27.145728765081877, 27.39727017313978, 27.179702321113545, 27.001009067886244], "calls": [0.0, 0.7656458055925432, 1.7976031957390146, 2.929427430093209, 4.094540612516644, 4.4607190412782955, 5.292942743009322, 6.191744340878828, 7.92276964047936, 8.488681757656458, 9.187749667110518, 10.219707057256992, 10.985352862849535, 12.117177097203728, 12.982689747003995, 14.347536617842877, 15.878828229027963, 16.711051930758988, 18.209054593874836, 18.675099866844207, 19.340878828229027, 19.77363515312916, 20.972037283621837, 21.571238348868178, 22.37017310252996, 22.93608521970706, 23.901464713715047, 25.39946737683089, 26.764314247669773, 28.428761651131822, 29.460719041278296, 30.9254327563249, 32.02396804260985, 33.655126498002666, 34.88681757656458, 36.085219707057256, 37.18375499334221, 38.08255659121172, 39.04793608521971, 39.34753661784288, 40.14647137150466, 41.577896138482025, 42.543275632490015, 42.876165113182424, 43.80825565912117, 44.77363515312916, 45.73901464713715, 46.33821571238349, 47.17043941411451, 48.10252996005326, 48.83488681757657, 49.60053262316911, 50.53262316910786, 52.0306258322237, 52.729693741677764, 54.593874833555255, 55.39280958721704, 56.59121171770972, 58.88814913448736, 59.92010652463382, 60.78561917443409, 61.817576564580555, 62.982689747003995, 63.74833555259654, 65.54593874833556, 66.54460719041279, 67.41011984021304, 67.9427430093209, 68.84154460719041, 69.70705725699068, 70.1398135818908, 70.57256990679095, 71.83754993342211, 72.60319573901465, 73.16910785619174, 74.06790945406125, 75.66577896138482, 76.53129161118508, 77.52996005326231, 78.09587217043942, 79.26098535286285, 80.05992010652463, 81.35818908122504, 82.39014647137151, 83.288948069241, 84.72037283621837, 86.21837549933421, 86.7842876165113, 87.7496671105193, 88.54860186418108, 89.84687083888149, 90.44607190412783, 91.4780292942743, 92.01065246338216, 93.87483355525966, 95.63914780292943, 96.57123834886818, 97.60319573901465, 98.33555259653795, 99.06790945406125, 100.0], "exact_score": [18.129624718318375, 18.338662718614795, 18.68930204646847, 19.018141812271086, 19.103111204126442, 19.236009975660842, 19.62443429875249, 19.854962754509035, 20.052735706746216, 20.30321771741218, 20.614727975607757, 20.886445929329103, 21.187403014053842, 21.317239028712937, 21.558144259176622, 22.30376947045362, 22.614513008143945, 22.836755789830328, 22.969500799022317, 23.195682534913008, 23.32212749330842, 23.52653618094096, 23.930921605332657, 24.178464074305626, 24.438151013020363, 24.69751375882492, 25.20792361621964, 25.51713804453673, 25.828719988931258, 25.784434607620483, 26.352314157691126, 26.56735808923208, 26.64722690860645, 26.927188891550625, 27.29283186399975, 27.612886210614345, 28.402985788031355, 28.456875898097362, 28.753541954286522, 28.827123539928472, 28.906723010074963, 29.41909143320155, 29.663282798413565, 29.444569644311702, 29.582213355320103, 30.149390823219974, 30.577986636395917, 30.47616577775716, 30.405138103317014, 30.399478628519855, 30.299257882425874, 30.262356131878658, 30.034006734607743, 29.941007873776393, 30.223438061728448, 30.28580097307699, 30.24143707380845, 30.176593333102453, 30.43412264623696, 30.133797800411028, 30.014277117822235, 29.888433537362115, 29.871865128952756, 29.737480828266214, 29.6883726064541, 29.559348823094783, 29.46086599332344, 29.499939704999896, 29.365402252271462, 29.22010701513807, 29.087947066289253, 29.342732745790784, 29.25540930797832, 29.171985073912786, 28.94316634089685, 28.83927227034706, 28.83333578090107, 28.653409813857127, 29.104930394284818, 28.801628908938316, 28.94432679225404, 28.69964060292337, 28.46987843440681, 28.67894555082307, 28.533383489000393, 28.45709981572895, 28.62170399801621, 28.510459534184974, 28.467486132679586, 28.291489309330693, 28.11888848476265, 27.978648370978238, 27.863409505370207, 27.668656940010873, 27.461050040409074, 27.25432390817678, 27.088736145356187, 27.249747079786914, 27.112271805466705, 27.205821724098623, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.157964919395717, "exact_area100": 26.867288337846052}}, "old_normalized_endpoint_budget": 48}, {"seed": 2029, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.25, "validation_area100": 0.2807720952518051, "training_seconds": 4.124964528018609, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.4620164804104601, "operator": "ge"}, {"budget": 0.02, "threshold": 0.42567584155200205, "operator": "ge"}, {"budget": 0.03, "threshold": 0.4118234735870916, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3952599939802581, "operator": "ge"}, {"budget": 0.05, "threshold": 0.36128917611941747, "operator": "ge"}, {"budget": 0.06, "threshold": 0.33524730219276283, "operator": "ge"}, {"budget": 0.07, "threshold": 0.30990978067553854, "operator": "ge"}, {"budget": 0.08, "threshold": 0.30051543875164866, "operator": "ge"}, {"budget": 0.09, "threshold": 0.2867971645389043, "operator": "ge"}, {"budget": 0.1, "threshold": 0.2760768004671629, "operator": "ge"}, {"budget": 0.11, "threshold": 0.2611113466929541, "operator": "ge"}, {"budget": 0.12, "threshold": 0.2557090286581199, "operator": "ge"}, {"budget": 0.13, "threshold": 0.25030392711107596, "operator": "ge"}, {"budget": 0.14, "threshold": 0.24241601930189927, "operator": "ge"}, {"budget": 0.15, "threshold": 0.23805046771838465, "operator": "ge"}, {"budget": 0.16, "threshold": 0.22895770601391202, "operator": "ge"}, {"budget": 0.17, "threshold": 0.22317408731061966, "operator": "ge"}, {"budget": 0.18, "threshold": 0.21840761024146002, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2157162054739702, "operator": "ge"}, {"budget": 0.2, "threshold": 0.2131222583601257, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2095399717087133, "operator": "ge"}, {"budget": 0.22, "threshold": 0.20565168644958384, "operator": "ge"}, {"budget": 0.23, "threshold": 0.20394487762179098, "operator": "ge"}, {"budget": 0.24, "threshold": 0.201404035705098, "operator": "ge"}, {"budget": 0.25, "threshold": 0.1980873053395986, "operator": "ge"}, {"budget": 0.26, "threshold": 0.19122337743411533, "operator": "ge"}, {"budget": 0.27, "threshold": 0.18796905065793457, "operator": "ge"}, {"budget": 0.28, "threshold": 0.182666369860232, "operator": "ge"}, {"budget": 0.29, "threshold": 0.1766991052266892, "operator": "ge"}, {"budget": 0.3, "threshold": 0.17266159362152153, "operator": "ge"}, {"budget": 0.31, "threshold": 0.16474604927333866, "operator": "ge"}, {"budget": 0.32, "threshold": 0.15926715329309774, "operator": "ge"}, {"budget": 0.33, "threshold": 0.1532704474186929, "operator": "ge"}, {"budget": 0.34, "threshold": 0.14878965332314592, "operator": "ge"}, {"budget": 0.35, "threshold": 0.1435824187557674, "operator": "ge"}, {"budget": 0.36, "threshold": 0.14024425965769358, "operator": "ge"}, {"budget": 0.37, "threshold": 0.1364147950705421, "operator": "ge"}, {"budget": 0.38, "threshold": 0.13365189119847068, "operator": "ge"}, {"budget": 0.39, "threshold": 0.13027179787304977, "operator": "ge"}, {"budget": 0.4, "threshold": 0.12623509054665263, "operator": "ge"}, {"budget": 0.41, "threshold": 0.12290691477334492, "operator": "ge"}, {"budget": 0.42, "threshold": 0.10874532340538466, "operator": "ge"}, {"budget": 0.43, "threshold": 0.09131834330711953, "operator": "ge"}, {"budget": 0.44, "threshold": 0.08243006759919397, "operator": "ge"}, {"budget": 0.45, "threshold": 0.046315565722561215, "operator": "ge"}, {"budget": 0.46, "threshold": 0.0003709330831331739, "operator": "ge"}, {"budget": 0.47, "threshold": -0.016176113541560237, "operator": "ge"}, {"budget": 0.48, "threshold": -0.028779091345293273, "operator": "ge"}, {"budget": 0.49, "threshold": -0.032212343113882144, "operator": "ge"}, {"budget": 0.5, "threshold": -0.03505739552251064, "operator": "ge"}, {"budget": 0.51, "threshold": -0.03921228926823121, "operator": "ge"}, {"budget": 0.52, "threshold": -0.04188705647072878, "operator": "ge"}, {"budget": 0.53, "threshold": -0.043427198824110134, "operator": "ge"}, {"budget": 0.54, "threshold": -0.04551838371500616, "operator": "ge"}, {"budget": 0.55, "threshold": -0.04790291209774876, "operator": "ge"}, {"budget": 0.56, "threshold": -0.051474718643142986, "operator": "ge"}, {"budget": 0.57, "threshold": -0.05478359752205729, "operator": "ge"}, {"budget": 0.58, "threshold": -0.05730000941192172, "operator": "ge"}, {"budget": 0.59, "threshold": -0.06054638368634891, "operator": "ge"}, {"budget": 0.6, "threshold": -0.06189618889417218, "operator": "ge"}, {"budget": 0.61, "threshold": -0.06422692225616036, "operator": "ge"}, {"budget": 0.62, "threshold": -0.06846378596609284, "operator": "ge"}, {"budget": 0.63, "threshold": -0.07029762787825175, "operator": "ge"}, {"budget": 0.64, "threshold": -0.07444404159346399, "operator": "ge"}, {"budget": 0.65, "threshold": -0.07759753465569955, "operator": "ge"}, {"budget": 0.66, "threshold": -0.07902609917468204, "operator": "ge"}, {"budget": 0.67, "threshold": -0.08046883276379663, "operator": "ge"}, {"budget": 0.68, "threshold": -0.08186147860906653, "operator": "ge"}, {"budget": 0.69, "threshold": -0.08394616092233338, "operator": "ge"}, {"budget": 0.7, "threshold": -0.08614181589835342, "operator": "ge"}, {"budget": 0.71, "threshold": -0.08780912941461373, "operator": "ge"}, {"budget": 0.72, "threshold": -0.09049407360666889, "operator": "ge"}, {"budget": 0.73, "threshold": -0.09250224136617806, "operator": "ge"}, {"budget": 0.74, "threshold": -0.09696521382269455, "operator": "ge"}, {"budget": 0.75, "threshold": -0.10036095238526647, "operator": "ge"}, {"budget": 0.76, "threshold": -0.10278443245580793, "operator": "ge"}, {"budget": 0.77, "threshold": -0.10552241331069502, "operator": "ge"}, {"budget": 0.78, "threshold": -0.10951047411315942, "operator": "ge"}, {"budget": 0.79, "threshold": -0.11286990233200943, "operator": "ge"}, {"budget": 0.8, "threshold": -0.11525404250465301, "operator": "ge"}, {"budget": 0.81, "threshold": -0.11960066709262032, "operator": "ge"}, {"budget": 0.82, "threshold": -0.12428627015047028, "operator": "ge"}, {"budget": 0.83, "threshold": -0.1254000829985197, "operator": "ge"}, {"budget": 0.84, "threshold": -0.13040264041248867, "operator": "ge"}, {"budget": 0.85, "threshold": -0.13651554708729466, "operator": "ge"}, {"budget": 0.86, "threshold": -0.13757534364269033, "operator": "ge"}, {"budget": 0.87, "threshold": -0.14190413361160914, "operator": "ge"}, {"budget": 0.88, "threshold": -0.14477200885667277, "operator": "ge"}, {"budget": 0.89, "threshold": -0.14742883954655686, "operator": "ge"}, {"budget": 0.9, "threshold": -0.15165831662608648, "operator": "ge"}, {"budget": 0.91, "threshold": -0.1571656875645512, "operator": "ge"}, {"budget": 0.92, "threshold": -0.16307977116046882, "operator": "ge"}, {"budget": 0.93, "threshold": -0.1672832122479983, "operator": "ge"}, {"budget": 0.94, "threshold": -0.16971993055734158, "operator": "ge"}, {"budget": 0.95, "threshold": -0.17207995065633028, "operator": "ge"}, {"budget": 0.96, "threshold": -0.17812142256066726, "operator": "ge"}, {"budget": 0.97, "threshold": -0.18077181139910545, "operator": "ge"}, {"budget": 0.98, "threshold": -0.18578268372106416, "operator": "ge"}, {"budget": 0.99, "threshold": -0.19154798675858495, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.606411530247485, 19.110729788963226, 19.58136726210538, 19.92304967964197, 20.403640775025227, 20.995811340638927, 21.528539372917884, 22.733344374221474, 23.222050474553257, 23.43231119785014, 23.877493962625003, 24.098303831399914, 24.436555689263272, 24.803888356084858, 24.89463601532567, 24.751958812145546, 24.689164890678246, 25.03545071123237, 25.317377369007797, 25.95172816626755, 26.03137351778656, 26.564584544863678, 26.854255107598934, 27.154611691899603, 26.787812509669234, 27.26617638743839, 28.13235620697313, 28.463856424599605, 28.37609622380749, 28.46671091381645, 28.769326304537575, 29.443221686667982, 29.83541586977772, 28.4492389331099, 29.069327680120345, 29.27186776573419, 29.27481556002683, 30.131857277263446, 30.790927482103953, 30.87236642777128, 30.787579593200253, 31.078903365792915, 31.32224419995483, 31.15926817435438, 32.78813352826511, 32.515316067947644, 32.417701264450486, 32.185700738332315, 32.185700738332315, 32.101422912119006, 32.01804645303097, 31.615845168476742, 31.615845168476742, 31.615845168476742, 31.428644535153943, 31.428644535153943, 31.428644535153943, 31.368835922713757, 31.22894429346042, 31.121417411739994, 31.04065996256149, 30.98386022221745, 30.98386022221745, 30.74468766001024, 30.74468766001024, 30.56451612903226, 30.51050378335325, 30.28515776229191, 30.515581232492995, 30.43704720107938, 30.198386997285965, 30.198386997285965, 30.350136676292138, 29.97802349413729, 29.917879147705857, 30.292319127358013, 30.02345005783249, 29.68247225006152, 29.216780368184082, 29.145172327983083, 28.88061579437296, 28.5740009811792, 28.520306823103596, 28.449665277251484, 28.2894711024942, 28.009049773755656, 27.823046694405917, 27.444888223770498, 27.304701307882645, 27.238508442776737, 27.172949002217294, 26.954408656307283, 26.86825577410177, 26.680836293123456, 26.7716870327657, 26.689719819650946, 26.163009030226775, 26.036746403964152, 25.727093126457586, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 25, 37, 47, 58, 71, 80, 91, 99, 105, 113, 118, 125, 130, 137, 143, 149, 156, 162, 172, 178, 186, 190, 194, 196, 203, 209, 215, 219, 224, 230, 237, 241, 244, 250, 254, 259, 266, 272, 277, 281, 288, 291, 295, 298, 299, 299, 300, 300, 300, 300, 300, 300, 300, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 302, 303, 303, 303, 303, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 2, 3, 4, 5, 5, 7, 8, 9, 10, 10, 10, 10, 13, 17, 20, 23, 25, 26, 29, 30, 31, 35, 40, 40, 43, 46, 48, 54, 55, 56, 57, 59, 60, 62, 65, 66, 68, 72, 77, 82, 82, 84, 84, 86, 90, 92, 92, 93, 94, 98, 98, 98, 103, 103, 103, 104, 106, 107, 108, 109, 109, 111, 111, 113, 114, 116, 120, 121, 124, 124, 129, 132, 133, 135, 139, 142, 145, 146, 150, 153, 154, 159, 162, 165, 168, 173, 176, 181, 182, 186, 188, 192, 196, 202, 210, 211, 215, 220], "exact_score": [18.254072168965788, 18.606411530247485, 19.110729788963226, 19.58136726210538, 19.92304967964197, 20.403640775025227, 20.995811340638927, 21.528539372917884, 22.733344374221474, 23.222050474553257, 23.43231119785014, 23.877493962625003, 24.098303831399914, 24.436555689263272, 24.803888356084858, 24.89463601532567, 24.751958812145546, 24.689164890678246, 25.03545071123237, 25.317377369007797, 25.95172816626755, 26.03137351778656, 26.564584544863678, 26.854255107598934, 27.154611691899603, 26.787812509669234, 27.26617638743839, 28.13235620697313, 28.463856424599605, 28.37609622380749, 28.46671091381645, 28.769326304537575, 29.443221686667982, 29.83541586977772, 28.4492389331099, 29.069327680120345, 29.27186776573419, 29.27481556002683, 30.131857277263446, 30.790927482103953, 30.87236642777128, 30.787579593200253, 31.078903365792915, 31.32224419995483, 31.15926817435438, 32.78813352826511, 32.515316067947644, 32.417701264450486, 32.185700738332315, 32.185700738332315, 32.101422912119006, 32.01804645303097, 31.615845168476742, 31.615845168476742, 31.615845168476742, 31.428644535153943, 31.428644535153943, 31.428644535153943, 31.368835922713757, 31.22894429346042, 31.121417411739994, 31.04065996256149, 30.98386022221745, 30.98386022221745, 30.74468766001024, 30.74468766001024, 30.56451612903226, 30.51050378335325, 30.28515776229191, 30.515581232492995, 30.43704720107938, 30.198386997285965, 30.198386997285965, 30.350136676292138, 29.97802349413729, 29.917879147705857, 30.292319127358013, 30.02345005783249, 29.68247225006152, 29.216780368184082, 29.145172327983083, 28.88061579437296, 28.5740009811792, 28.520306823103596, 28.449665277251484, 28.2894711024942, 28.009049773755656, 27.823046694405917, 27.444888223770498, 27.304701307882645, 27.238508442776737, 27.172949002217294, 26.954408656307283, 26.86825577410177, 26.680836293123456, 26.7716870327657, 26.689719819650946, 26.163009030226775, 26.036746403964152, 25.727093126457586, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 28.077209525180507, "threshold_area100": 28.08076573925292, "threshold_area25": 23.44065169875722}, "test": {"score": [18.129624718318375, 18.28431204720806, 18.762219293012993, 18.84103106584469, 19.154341545011135, 19.74749861537206, 20.308823292359016, 20.204938016411546, 19.97916673153004, 20.111589270307782, 20.307482603257256, 20.77993878900609, 21.086128379463524, 21.67625671041425, 21.980941672146017, 22.17745049812929, 22.311618325350317, 22.793412195017545, 23.023084973634425, 23.045474360298236, 23.150055221870538, 23.343776535059078, 23.615591030313205, 23.827626047718805, 23.91646650419158, 24.06878704993982, 24.883907962147724, 25.026636769874212, 25.289119538422653, 25.42637810209345, 25.614754546353712, 26.41310107241584, 26.833440153510573, 27.172372001554955, 27.786828299039023, 28.071609102654776, 28.21981870744501, 28.460722141509155, 28.6885063865702, 29.23045338776802, 29.28577457279646, 29.458591766913145, 30.115116769584056, 30.242611246665213, 30.355325767927855, 29.893105690852895, 29.910604830665388, 29.92809712457618, 29.854470696734058, 29.79585100758022, 29.81353678936694, 29.730484409911632, 29.632489740969962, 29.494687999877872, 29.494687999877872, 29.778231135192286, 29.68150077317186, 29.627068897341267, 29.879788546872398, 30.095942155539674, 30.059563138942163, 30.04062348317386, 29.875012696925236, 29.764891559675423, 29.271343015011308, 29.170722936369604, 29.169447994292092, 29.033883127607325, 28.948606798274373, 28.82943709101139, 28.767345425687758, 28.737758581374894, 28.625944191178988, 28.596669421607434, 29.041387793293467, 28.844817906600383, 28.694668128295493, 28.99044404937644, 28.98635365703896, 28.95107514208638, 28.890542582613403, 29.421321141533895, 29.12265541698777, 29.067203976456014, 28.986238948674803, 29.084572878357072, 29.018664047460224, 28.879524410852504, 28.996892340372337, 28.777135805737363, 28.687463568922755, 28.44300918231091, 28.35690218773227, 28.08820655942281, 27.9044704494771, 27.851447919008095, 27.820716926984467, 27.737850511677042, 27.558614890962634, 27.439983722174375, 27.001009067886244], "calls": [0.0, 1.0652463382157125, 2.896138482023968, 3.4287616511318246, 4.427430093209055, 6.25832223701731, 7.789613848202396, 8.88814913448735, 9.154460719041278, 9.786950732356857, 10.386151797603196, 11.717709720372836, 13.1491344873502, 14.14780292942743, 15.745672436750999, 16.444740346205062, 17.942743009320907, 19.00798934753662, 19.840213049267643, 20.33954727030626, 20.73901464713715, 21.604527296937416, 22.603195739014648, 23.135818908122506, 23.501997336884152, 24.33422103861518, 25.59920106524634, 26.23169107856192, 27.396804260985352, 28.728362183754992, 29.527296937416775, 30.758988015978694, 31.69107856191744, 32.9560585885486, 34.32090545938748, 35.75233022636485, 36.51797603195739, 37.383488681757655, 38.515312916111846, 39.314247669773636, 40.21304926764314, 40.645805592543276, 42.110519307589875, 43.10918774966711, 43.54194407456725, 44.54061251664447, 45.539280958721704, 46.07190412782956, 46.70439414114514, 47.10386151797603, 48.23568575233023, 50.23302263648469, 52.06391478029294, 53.16245006657789, 54.29427430093209, 55.29294274300932, 56.657789613848195, 58.089214380825574, 59.12117177097203, 60.01997336884154, 60.38615179760319, 61.18508655126498, 63.08255659121171, 63.914780292942744, 65.54593874833556, 67.17709720372837, 67.67643142476697, 68.47536617842877, 69.27430093209055, 70.0399467376831, 70.40612516644474, 71.07190412782957, 72.13715046604527, 72.56990679094541, 74.10119840213049, 75.33288948069242, 76.23169107856191, 77.16378162450066, 78.12916111850865, 79.12782956058588, 79.76031957390146, 80.89214380825565, 82.49001331557922, 82.98934753661784, 84.18774966711052, 85.81890812250333, 86.08521970705726, 87.35019973368841, 88.08255659121171, 88.68175765645806, 89.61384820239681, 91.04527296937417, 92.87616511318242, 93.97470039946738, 94.77363515312916, 95.30625832223701, 96.60452729693742, 96.90412782956058, 97.70306258322236, 98.5352862849534, 100.0], "rescued": [0, 71, 148, 172, 215, 297, 364, 410, 420, 438, 457, 498, 534, 561, 603, 620, 660, 686, 707, 720, 727, 746, 765, 778, 784, 803, 832, 845, 871, 901, 919, 949, 975, 1004, 1034, 1063, 1082, 1100, 1122, 1143, 1161, 1169, 1199, 1214, 1221, 1231, 1240, 1244, 1244, 1244, 1246, 1246, 1246, 1248, 1248, 1249, 1250, 1251, 1252, 1253, 1253, 1253, 1254, 1254, 1255, 1256, 1257, 1258, 1258, 1259, 1260, 1260, 1260, 1260, 1263, 1264, 1264, 1266, 1268, 1270, 1270, 1273, 1273, 1273, 1274, 1276, 1276, 1276, 1277, 1277, 1277, 1277, 1279, 1280, 1280, 1280, 1281, 1281, 1282, 1282, 1282], "harmed": [0, 3, 10, 12, 20, 28, 37, 50, 51, 55, 56, 62, 68, 71, 82, 83, 101, 113, 118, 125, 125, 130, 133, 137, 138, 144, 156, 157, 162, 174, 179, 190, 197, 207, 220, 235, 244, 250, 263, 271, 284, 285, 298, 305, 307, 315, 321, 323, 327, 330, 332, 335, 340, 347, 347, 350, 360, 364, 368, 374, 376, 377, 386, 392, 404, 411, 416, 424, 429, 434, 439, 441, 447, 449, 454, 467, 476, 480, 493, 507, 512, 522, 535, 543, 554, 573, 576, 585, 592, 601, 608, 623, 650, 670, 684, 688, 706, 713, 729, 739, 777], "exact_score": [18.129624718318375, 18.261871431910265, 18.424575518847288, 18.80106529601695, 19.00128533099596, 19.399773066002297, 19.80658100192139, 20.000614781671498, 20.09730600806181, 19.92436766541957, 20.185824082306073, 20.576920832961722, 20.84913369052736, 21.048914996446626, 21.621366735776135, 21.845301477183064, 22.005202786021762, 22.129415308461837, 22.311618325350317, 22.80729385387578, 22.984215941804894, 23.197486117221022, 23.51194077158398, 23.846253884498818, 24.01697970803662, 24.45374393124159, 24.936639326966308, 25.222210795740207, 25.358971563614368, 25.478470525868, 25.753076174962853, 26.460593198492653, 26.89662572591206, 27.172632912764584, 27.31402599479661, 27.925481838692523, 28.165767435339088, 28.35809345727555, 28.5760097932081, 29.130686877425255, 29.246017206752427, 29.559899174416206, 30.076198972619643, 30.242611246665213, 29.881319302428345, 29.948675264286393, 29.92809712457618, 29.818295177707032, 29.832910941310647, 29.81353678936694, 29.730484409911632, 29.658761283634956, 29.632489740969962, 29.51396784507824, 29.494687999877872, 29.778231135192286, 29.82653242363198, 29.662447215352973, 29.627068897341267, 29.879788546872398, 30.095942155539674, 30.04062348317386, 30.04062348317386, 29.89830968695413, 29.764891559675423, 29.63346715413804, 29.25120878682339, 29.210170799777202, 29.11990221468485, 28.966811423578005, 28.82943709101139, 28.737758581374894, 28.625944191178988, 28.58544747647028, 29.077214730966094, 28.944913453210347, 28.705726855959774, 29.001396825616986, 29.02123044607056, 28.97859738809506, 28.918527299403912, 29.393490817292367, 29.172855717380074, 29.067203976456014, 29.048199042219263, 28.85464385127645, 29.028874976071243, 28.889513850587644, 29.03017001141418, 28.75081842751178, 28.61705024799518, 28.44300918231091, 28.168363412004844, 28.308245141216943, 28.08820655942281, 27.896348353598587, 27.73053444425258, 27.737850511677042, 27.53532651236452, 27.361504783403696, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.802456928523938, "threshold_area100": 26.847823573652875, "threshold_area25": 21.29633856455165}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.893105690852895, "calls": 44.54061251664447, "rescued": 1231, "harmed": 315}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.40263181924819946, "operator": "ge"}, {"budget": 0.02, "threshold": 0.3787396550178528, "operator": "ge"}, {"budget": 0.03, "threshold": 0.36041221022605896, "operator": "ge"}, {"budget": 0.04, "threshold": 0.3410666286945343, "operator": "ge"}, {"budget": 0.05, "threshold": 0.3221871554851532, "operator": "ge"}, {"budget": 0.06, "threshold": 0.2919948995113373, "operator": "ge"}, {"budget": 0.07, "threshold": 0.27255770564079285, "operator": "ge"}, {"budget": 0.08, "threshold": 0.2618773579597473, "operator": "ge"}, {"budget": 0.09, "threshold": 0.23911961913108826, "operator": "ge"}, {"budget": 0.1, "threshold": 0.21386978030204773, "operator": "ge"}, {"budget": 0.11, "threshold": 0.20304088294506073, "operator": "ge"}, {"budget": 0.12, "threshold": 0.19593723118305206, "operator": "ge"}, {"budget": 0.13, "threshold": 0.18796387314796448, "operator": "ge"}, {"budget": 0.14, "threshold": 0.18441084027290344, "operator": "ge"}, {"budget": 0.15, "threshold": 0.1803877353668213, "operator": "ge"}, {"budget": 0.16, "threshold": 0.17438904941082, "operator": "ge"}, {"budget": 0.17, "threshold": 0.17200729250907898, "operator": "ge"}, {"budget": 0.18, "threshold": 0.1698751002550125, "operator": "ge"}, {"budget": 0.19, "threshold": 0.1662481427192688, "operator": "ge"}, {"budget": 0.2, "threshold": 0.16221734881401062, "operator": "ge"}, {"budget": 0.21, "threshold": 0.1578259915113449, "operator": "ge"}, {"budget": 0.22, "threshold": 0.15536624193191528, "operator": "ge"}, {"budget": 0.23, "threshold": 0.1532023549079895, "operator": "ge"}, {"budget": 0.24, "threshold": 0.14879241585731506, "operator": "ge"}, {"budget": 0.25, "threshold": 0.1471835821866989, "operator": "ge"}, {"budget": 0.26, "threshold": 0.14332377910614014, "operator": "ge"}, {"budget": 0.27, "threshold": 0.14006653428077698, "operator": "ge"}, {"budget": 0.28, "threshold": 0.13569346070289612, "operator": "ge"}, {"budget": 0.29, "threshold": 0.13180282711982727, "operator": "ge"}, {"budget": 0.3, "threshold": 0.12923680245876312, "operator": "ge"}, {"budget": 0.31, "threshold": 0.12562203407287598, "operator": "ge"}, {"budget": 0.32, "threshold": 0.12191376835107803, "operator": "ge"}, {"budget": 0.33, "threshold": 0.11883535981178284, "operator": "ge"}, {"budget": 0.34, "threshold": 0.11410661786794662, "operator": "ge"}, {"budget": 0.35, "threshold": 0.10867905616760254, "operator": "ge"}, {"budget": 0.36, "threshold": 0.10729843378067017, "operator": "ge"}, {"budget": 0.37, "threshold": 0.10438696295022964, "operator": "ge"}, {"budget": 0.38, "threshold": 0.10325554758310318, "operator": "ge"}, {"budget": 0.39, "threshold": 0.10163843631744385, "operator": "ge"}, {"budget": 0.4, "threshold": 0.0987635925412178, "operator": "ge"}, {"budget": 0.41, "threshold": 0.09430743753910065, "operator": "ge"}, {"budget": 0.42, "threshold": 0.08758499473333359, "operator": "ge"}, {"budget": 0.43, "threshold": 0.06738392263650894, "operator": "ge"}, {"budget": 0.44, "threshold": 0.056384578347206116, "operator": "ge"}, {"budget": 0.45, "threshold": 0.037758298218250275, "operator": "ge"}, {"budget": 0.46, "threshold": 0.00025928765535354614, "operator": "ge"}, {"budget": 0.47, "threshold": -0.009102358482778072, "operator": "ge"}, {"budget": 0.48, "threshold": -0.010346274822950363, "operator": "ge"}, {"budget": 0.49, "threshold": -0.011853275820612907, "operator": "ge"}, {"budget": 0.5, "threshold": -0.01330043375492096, "operator": "ge"}, {"budget": 0.51, "threshold": -0.014474837109446526, "operator": "ge"}, {"budget": 0.52, "threshold": -0.014839891344308853, "operator": "ge"}, {"budget": 0.53, "threshold": -0.015941957011818886, "operator": "ge"}, {"budget": 0.54, "threshold": -0.016947835683822632, "operator": "ge"}, {"budget": 0.55, "threshold": -0.019148485735058784, "operator": "ge"}, {"budget": 0.56, "threshold": -0.019983628764748573, "operator": "ge"}, {"budget": 0.57, "threshold": -0.02216266095638275, "operator": "ge"}, {"budget": 0.58, "threshold": -0.0240793339908123, "operator": "ge"}, {"budget": 0.59, "threshold": -0.025170426815748215, "operator": "ge"}, {"budget": 0.6, "threshold": -0.026246747002005577, "operator": "ge"}, {"budget": 0.61, "threshold": -0.027521656826138496, "operator": "ge"}, {"budget": 0.62, "threshold": -0.029260320588946342, "operator": "ge"}, {"budget": 0.63, "threshold": -0.0313926599919796, "operator": "ge"}, {"budget": 0.64, "threshold": -0.03320334106683731, "operator": "ge"}, {"budget": 0.65, "threshold": -0.03437620773911476, "operator": "ge"}, {"budget": 0.66, "threshold": -0.03505028784275055, "operator": "ge"}, {"budget": 0.67, "threshold": -0.03644160181283951, "operator": "ge"}, {"budget": 0.68, "threshold": -0.0377546101808548, "operator": "ge"}, {"budget": 0.69, "threshold": -0.03861498087644577, "operator": "ge"}, {"budget": 0.7, "threshold": -0.03986773267388344, "operator": "ge"}, {"budget": 0.71, "threshold": -0.040671274065971375, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0420946441590786, "operator": "ge"}, {"budget": 0.73, "threshold": -0.04522949084639549, "operator": "ge"}, {"budget": 0.74, "threshold": -0.04718104749917984, "operator": "ge"}, {"budget": 0.75, "threshold": -0.04869900643825531, "operator": "ge"}, {"budget": 0.76, "threshold": -0.05009004846215248, "operator": "ge"}, {"budget": 0.77, "threshold": -0.05233544111251831, "operator": "ge"}, {"budget": 0.78, "threshold": -0.054068416357040405, "operator": "ge"}, {"budget": 0.79, "threshold": -0.05639246106147766, "operator": "ge"}, {"budget": 0.8, "threshold": -0.05785028263926506, "operator": "ge"}, {"budget": 0.81, "threshold": -0.06055372953414917, "operator": "ge"}, {"budget": 0.82, "threshold": -0.06296940892934799, "operator": "ge"}, {"budget": 0.83, "threshold": -0.06533639878034592, "operator": "ge"}, {"budget": 0.84, "threshold": -0.06832332909107208, "operator": "ge"}, {"budget": 0.85, "threshold": -0.07179395109415054, "operator": "ge"}, {"budget": 0.86, "threshold": -0.07333727926015854, "operator": "ge"}, {"budget": 0.87, "threshold": -0.07635228335857391, "operator": "ge"}, {"budget": 0.88, "threshold": -0.07798361778259277, "operator": "ge"}, {"budget": 0.89, "threshold": -0.08023663610219955, "operator": "ge"}, {"budget": 0.9, "threshold": -0.08215727657079697, "operator": "ge"}, {"budget": 0.91, "threshold": -0.08687297999858856, "operator": "ge"}, {"budget": 0.92, "threshold": -0.09160295873880386, "operator": "ge"}, {"budget": 0.93, "threshold": -0.09488407522439957, "operator": "ge"}, {"budget": 0.94, "threshold": -0.09701299667358398, "operator": "ge"}, {"budget": 0.95, "threshold": -0.09878528118133545, "operator": "ge"}, {"budget": 0.96, "threshold": -0.10232005268335342, "operator": "ge"}, {"budget": 0.97, "threshold": -0.10515035688877106, "operator": "ge"}, {"budget": 0.98, "threshold": -0.10831507295370102, "operator": "ge"}, {"budget": 0.99, "threshold": -0.11278463155031204, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.67579173376275, 19.154247220284955, 19.571170380877724, 19.92304967964197, 20.46427782446629, 21.005660432489975, 21.455267265711704, 22.81069624819625, 23.222737929868657, 23.740811494321328, 24.009754555827286, 24.280968545555176, 24.56676497882154, 24.892047323554173, 24.964517217282342, 25.273994939903275, 25.270898641588296, 25.16753850624818, 25.614543114543114, 26.121379900068618, 26.233347352879065, 26.86183168919742, 26.808725688379486, 26.99662314591692, 27.261907467962978, 28.141105319173725, 28.043316203527468, 27.98775907076445, 28.319184744482218, 28.793026774978074, 28.827779249095137, 29.454871293580975, 29.567631941552886, 28.4492389331099, 28.879933740726404, 29.33723052031637, 29.272538325917118, 29.696077324501978, 30.57918176949963, 30.618144797249276, 30.787579593200253, 30.814353101242652, 31.36345136345136, 31.15926817435438, 32.78813352826511, 32.515316067947644, 32.417701264450486, 32.417701264450486, 32.333789556633896, 32.06783210982539, 31.935556977550256, 31.730787697212374, 31.730787697212374, 31.730787697212374, 31.730787697212374, 31.615845168476742, 31.556036556036553, 31.448509674316128, 31.22956534542415, 31.17276560508011, 30.98386022221745, 30.98386022221745, 30.928480475382003, 30.928480475382003, 30.928480475382003, 30.747762202330687, 30.747762202330687, 30.36265432098765, 30.112244292871438, 30.059549321477284, 29.637353768899683, 29.56278548004568, 29.646116935273565, 29.588744588744593, 29.381025746677537, 29.57097288676236, 29.457336523126, 28.788935779839964, 28.664733069232216, 28.61325743436606, 28.88061579437296, 28.692928106685272, 28.173806476603254, 28.39520983170693, 28.060394060536836, 27.873412994772217, 27.7736307015937, 27.39729390047524, 27.304701307882645, 27.238508442776737, 27.172949002217294, 26.954408656307283, 26.910951007761135, 26.680836293123456, 26.81370383948839, 26.689719819650946, 26.163009030226775, 26.036746403964152, 25.727093126457586, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 13, 25, 37, 47, 58, 70, 80, 93, 99, 106, 113, 119, 125, 132, 138, 145, 152, 159, 165, 173, 179, 185, 189, 194, 200, 205, 209, 214, 220, 226, 230, 235, 238, 244, 250, 254, 259, 264, 270, 277, 281, 287, 291, 295, 298, 299, 299, 299, 299, 299, 300, 300, 300, 300, 300, 300, 300, 300, 300, 300, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 302, 302, 302, 303, 303, 304, 304, 304, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 2, 3, 4, 5, 5, 7, 8, 9, 10, 10, 10, 11, 13, 14, 20, 24, 25, 26, 30, 32, 34, 35, 39, 41, 46, 50, 50, 50, 56, 57, 58, 59, 61, 62, 67, 68, 69, 73, 77, 82, 82, 84, 84, 86, 90, 90, 91, 93, 95, 97, 97, 101, 101, 102, 103, 104, 107, 108, 109, 109, 110, 110, 110, 112, 112, 115, 118, 119, 123, 124, 128, 129, 132, 137, 139, 144, 146, 147, 150, 151, 156, 160, 164, 167, 169, 174, 176, 181, 182, 186, 187, 192, 199, 202, 210, 211, 215, 220], "exact_score": [18.254072168965788, 18.67579173376275, 19.154247220284955, 19.571170380877724, 19.92304967964197, 20.46427782446629, 21.005660432489975, 21.455267265711704, 22.81069624819625, 23.222737929868657, 23.740811494321328, 24.009754555827286, 24.280968545555176, 24.56676497882154, 24.892047323554173, 24.964517217282342, 25.273994939903275, 25.270898641588296, 25.16753850624818, 25.614543114543114, 26.121379900068618, 26.233347352879065, 26.86183168919742, 26.808725688379486, 26.99662314591692, 27.261907467962978, 28.141105319173725, 28.043316203527468, 27.98775907076445, 28.319184744482218, 28.793026774978074, 28.827779249095137, 29.454871293580975, 29.567631941552886, 28.4492389331099, 28.879933740726404, 29.33723052031637, 29.272538325917118, 29.696077324501978, 30.57918176949963, 30.618144797249276, 30.787579593200253, 30.814353101242652, 31.36345136345136, 31.15926817435438, 32.78813352826511, 32.515316067947644, 32.417701264450486, 32.417701264450486, 32.333789556633896, 32.06783210982539, 31.935556977550256, 31.730787697212374, 31.730787697212374, 31.730787697212374, 31.730787697212374, 31.615845168476742, 31.556036556036553, 31.448509674316128, 31.22956534542415, 31.17276560508011, 30.98386022221745, 30.98386022221745, 30.928480475382003, 30.928480475382003, 30.928480475382003, 30.747762202330687, 30.747762202330687, 30.36265432098765, 30.112244292871438, 30.059549321477284, 29.637353768899683, 29.56278548004568, 29.646116935273565, 29.588744588744593, 29.381025746677537, 29.57097288676236, 29.457336523126, 28.788935779839964, 28.664733069232216, 28.61325743436606, 28.88061579437296, 28.692928106685272, 28.173806476603254, 28.39520983170693, 28.060394060536836, 27.873412994772217, 27.7736307015937, 27.39729390047524, 27.304701307882645, 27.238508442776737, 27.172949002217294, 26.954408656307283, 26.910951007761135, 26.680836293123456, 26.81370383948839, 26.689719819650946, 26.163009030226775, 26.036746403964152, 25.727093126457586, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 28.049960651302992, "threshold_area100": 28.05347793841273, "threshold_area25": 23.609795250920595}, "test": {"score": [18.129624718318375, 18.38733596054732, 18.66865830330849, 18.958958486545935, 19.35098965625592, 19.678808021555753, 20.28535731070281, 20.25591862244446, 19.978932160439925, 20.323816549623, 20.553857764172438, 21.11201315700732, 21.4383073532061, 21.94029967176681, 22.150545114273555, 22.24949488985346, 22.37752705696812, 22.459241041428278, 22.546504139483876, 22.75704817493241, 23.20820472394478, 23.425227685856438, 23.69284138389707, 23.91674947071702, 24.158400492116932, 24.224572870083705, 24.421818882779053, 24.975746912642517, 25.575555226026452, 25.870773235399984, 26.302923074596624, 26.41948870260475, 26.805912302591384, 26.891125031922332, 27.85401824540082, 28.16151746468848, 28.303111368895507, 28.743283115727785, 29.07121434813697, 29.170521164366665, 29.385435724017128, 29.57370959972196, 30.051562502324735, 30.350125724945045, 29.96054285396083, 29.893105690852895, 29.910604830665388, 29.90460916215692, 29.88147707795613, 29.8656211866198, 29.8656211866198, 29.780285424304566, 29.69014207030456, 29.60715671352255, 29.553123413967825, 29.300498072929372, 29.260612072393638, 29.183231849438062, 29.410598922580512, 29.3757781405601, 29.67877198625405, 29.6409769097843, 29.462769149800387, 29.341291535413827, 29.18274262036011, 29.232123214609135, 29.19260730306863, 29.012688342585246, 28.941693902380255, 29.229796863736812, 29.218112800798657, 29.218112800798657, 29.065612697575094, 29.000688398459417, 29.086834653210115, 28.844730704161936, 29.03095566485756, 29.094002532193, 28.91929352150716, 28.889278679807166, 28.911719890091575, 29.3428863075869, 29.18176575450433, 29.074470735163942, 28.902482739758117, 29.031428132819244, 28.990534358020483, 29.104207376613907, 29.03430068991596, 28.77791551042944, 28.643089199904644, 28.43592802884475, 28.308245141216943, 28.046467091358362, 27.85950723557805, 27.750467617488894, 27.820716926984467, 27.766186003905023, 27.59347421153624, 27.447208912885806, 27.001009067886244], "calls": [0.0, 1.3315579227696404, 2.596537949400799, 3.695073235685752, 4.826897470039946, 5.9920106524633825, 7.789613848202396, 8.721704394141145, 9.254327563249001, 10.086551264980026, 10.818908122503329, 11.717709720372836, 13.082556591211716, 15.246338215712383, 16.07856191744341, 16.977363515312916, 18.209054593874836, 18.808255659121173, 19.241011984021306, 20.10652463382157, 20.938748335552596, 21.770972037283624, 22.37017310252996, 23.202396804260985, 24.234354194407455, 24.70039946737683, 25.832223701731028, 26.73102529960053, 27.996005326231693, 29.161118508655125, 29.82689747003995, 30.525965379494007, 31.69107856191744, 32.323568575233026, 33.85486018641811, 35.95206391478029, 36.61784287616511, 37.91611185086551, 38.315579227696404, 38.948069241011986, 39.81358189081225, 40.97869507323569, 41.777629826897474, 43.3422103861518, 43.74167776298269, 44.47403462050599, 45.539280958721704, 46.23834886817576, 46.504660452729695, 47.66977363515313, 49.134487350199734, 50.665778961384824, 51.33155792276965, 52.46338215712384, 53.295605858854856, 54.96005326231691, 55.559254327563245, 57.456724367509985, 58.75499334221038, 59.28761651131824, 60.05326231691078, 61.01864181091877, 62.48335552596538, 63.84820239680427, 65.5126498002663, 66.51131824234355, 66.94407456724367, 68.10918774966711, 68.80825565912117, 69.27430093209055, 69.90679094540613, 70.57256990679095, 71.33821571238349, 73.10252996005326, 74.50066577896138, 75.33288948069242, 76.1318242343542, 77.49667110519307, 78.26231691078563, 78.96138482023967, 79.59387483355526, 81.05858854860186, 82.25699067909454, 83.12250332889481, 84.22103861517975, 85.58588548601864, 86.08521970705726, 87.45006657789614, 87.98268974700399, 88.81491344873503, 89.4474034620506, 91.04527296937417, 92.97603195739015, 94.20772303595206, 95.0399467376831, 95.67243675099867, 96.60452729693742, 97.00399467376832, 97.60319573901465, 98.50199733688414, 100.0], "rescued": [0, 84, 137, 186, 235, 287, 362, 403, 421, 450, 473, 499, 536, 591, 612, 637, 668, 681, 691, 714, 730, 753, 765, 784, 803, 811, 835, 855, 882, 911, 930, 942, 972, 986, 1025, 1067, 1079, 1108, 1120, 1134, 1155, 1177, 1193, 1219, 1221, 1231, 1240, 1244, 1244, 1245, 1245, 1245, 1245, 1245, 1245, 1245, 1246, 1247, 1249, 1249, 1250, 1250, 1250, 1251, 1252, 1254, 1254, 1254, 1254, 1255, 1257, 1257, 1257, 1259, 1264, 1264, 1265, 1267, 1268, 1269, 1271, 1273, 1273, 1273, 1274, 1275, 1276, 1277, 1277, 1277, 1277, 1277, 1279, 1280, 1280, 1280, 1281, 1281, 1282, 1282, 1282], "harmed": [0, 4, 10, 12, 21, 26, 38, 48, 52, 57, 59, 63, 69, 77, 79, 90, 105, 110, 114, 121, 124, 132, 133, 136, 142, 145, 156, 165, 172, 179, 185, 193, 199, 205, 214, 239, 246, 259, 261, 269, 277, 289, 295, 305, 310, 315, 321, 324, 325, 328, 328, 332, 336, 340, 343, 354, 357, 363, 368, 370, 372, 374, 384, 390, 401, 407, 413, 422, 425, 428, 431, 431, 438, 443, 454, 467, 474, 487, 498, 504, 513, 523, 530, 537, 556, 569, 576, 586, 591, 602, 610, 623, 653, 675, 687, 695, 706, 712, 727, 738, 777], "exact_score": [18.129624718318375, 18.254778867444585, 18.450679892017153, 18.702650290885586, 19.049312505907054, 19.39375894155655, 19.678808021555753, 20.0476493368559, 20.098836220163797, 19.92436766541957, 20.294767781760104, 20.595602456495445, 21.156206602260948, 21.404541039509176, 21.62571550359274, 21.891667335097996, 22.112751504020533, 22.24949488985346, 22.415205282589458, 22.48425847458381, 22.689586430081157, 23.219044557503732, 23.584113911749476, 23.820565719846115, 24.0671714434438, 24.282555246496322, 24.745095192147843, 24.974822894372412, 25.575555226026452, 25.801020942754842, 26.30038404414854, 26.61461054320292, 26.843963939952246, 27.1576437889462, 27.9130400013604, 28.021101076131878, 28.13766534781311, 28.384107296442373, 28.805546358603333, 29.15697306862679, 29.424786990603245, 29.57370959972196, 30.0462898318239, 30.370398431451456, 29.907125754041253, 29.948675264286393, 29.90460916215692, 29.898490290335005, 29.8656211866198, 29.8656211866198, 29.793602928832513, 29.703313830051663, 29.629268377425255, 29.57490990634255, 29.508035353353424, 29.279645190018304, 29.20838853968014, 29.176832450665845, 29.097005350919176, 29.388123645385427, 29.67877198625405, 29.6409769097843, 29.549679649263865, 29.345685382315573, 29.322703216476263, 29.205113985013355, 29.148126541355236, 29.19260730306863, 29.063359266949263, 29.259585804484733, 29.218112800798657, 29.17539236334138, 29.036265416777397, 29.000688398459417, 29.39264459482346, 28.89174960357589, 29.042121888793243, 29.179941867233765, 28.925898784671876, 28.889278679807166, 29.19246790510217, 29.07544424192893, 29.27970532309463, 29.08496765956509, 28.964085813338613, 28.848077414267166, 29.00079046773155, 28.909624170054432, 29.03430068991596, 28.768182473721986, 28.60057406496968, 28.43592802884475, 28.17719971321331, 28.308245141216943, 28.08820655942281, 27.867597850464783, 27.706482534068737, 27.773825559034183, 27.54268599754286, 27.361504783403696, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.79938337170164, "threshold_area100": 26.863418643958248, "threshold_area25": 21.373492838745495}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.893105690852895, "calls": 44.47403462050599, "rescued": 1231, "harmed": 315}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.9801769256591797, "operator": "ge"}, {"budget": 0.02, "threshold": 2.8248345851898193, "operator": "ge"}, {"budget": 0.03, "threshold": 2.676076889038086, "operator": "ge"}, {"budget": 0.04, "threshold": 2.47998046875, "operator": "ge"}, {"budget": 0.05, "threshold": 2.3737454414367676, "operator": "ge"}, {"budget": 0.06, "threshold": 2.2531962394714355, "operator": "ge"}, {"budget": 0.07, "threshold": 2.1819591522216797, "operator": "ge"}, {"budget": 0.08, "threshold": 2.0993306636810303, "operator": "ge"}, {"budget": 0.09, "threshold": 2.0801072120666504, "operator": "ge"}, {"budget": 0.1, "threshold": 2.042266368865967, "operator": "ge"}, {"budget": 0.11, "threshold": 2.006277322769165, "operator": "ge"}, {"budget": 0.12, "threshold": 1.9488117694854736, "operator": "ge"}, {"budget": 0.13, "threshold": 1.921006202697754, "operator": "ge"}, {"budget": 0.14, "threshold": 1.882568120956421, "operator": "ge"}, {"budget": 0.15, "threshold": 1.858638882637024, "operator": "ge"}, {"budget": 0.16, "threshold": 1.8051824569702148, "operator": "ge"}, {"budget": 0.17, "threshold": 1.7899045944213867, "operator": "ge"}, {"budget": 0.18, "threshold": 1.7444117069244385, "operator": "ge"}, {"budget": 0.19, "threshold": 1.6937634944915771, "operator": "ge"}, {"budget": 0.2, "threshold": 1.6745386123657227, "operator": "ge"}, {"budget": 0.21, "threshold": 1.6465981006622314, "operator": "ge"}, {"budget": 0.22, "threshold": 1.6191526651382446, "operator": "ge"}, {"budget": 0.23, "threshold": 1.603879690170288, "operator": "ge"}, {"budget": 0.24, "threshold": 1.5674419403076172, "operator": "ge"}, {"budget": 0.25, "threshold": 1.5364234447479248, "operator": "ge"}, {"budget": 0.26, "threshold": 1.500556468963623, "operator": "ge"}, {"budget": 0.27, "threshold": 1.4389668703079224, "operator": "ge"}, {"budget": 0.28, "threshold": 1.394315242767334, "operator": "ge"}, {"budget": 0.29, "threshold": 1.3559763431549072, "operator": "ge"}, {"budget": 0.3, "threshold": 1.2975537776947021, "operator": "ge"}, {"budget": 0.31, "threshold": 1.2337898015975952, "operator": "ge"}, {"budget": 0.32, "threshold": 1.1706331968307495, "operator": "ge"}, {"budget": 0.33, "threshold": 1.1322776079177856, "operator": "ge"}, {"budget": 0.34, "threshold": 1.0929806232452393, "operator": "ge"}, {"budget": 0.35, "threshold": 1.057739496231079, "operator": "ge"}, {"budget": 0.36, "threshold": 1.0235302448272705, "operator": "ge"}, {"budget": 0.37, "threshold": 0.9739536643028259, "operator": "ge"}, {"budget": 0.38, "threshold": 0.9014661908149719, "operator": "ge"}, {"budget": 0.39, "threshold": 0.8739567995071411, "operator": "ge"}, {"budget": 0.4, "threshold": 0.8354649543762207, "operator": "ge"}, {"budget": 0.41, "threshold": 0.7948399782180786, "operator": "ge"}, {"budget": 0.42, "threshold": 0.7351788878440857, "operator": "ge"}, {"budget": 0.43, "threshold": 0.7045107483863831, "operator": "ge"}, {"budget": 0.44, "threshold": 0.5541388392448425, "operator": "ge"}, {"budget": 0.45, "threshold": 0.41355258226394653, "operator": "ge"}, {"budget": 0.46, "threshold": 0.004558920860290527, "operator": "ge"}, {"budget": 0.47, "threshold": -0.26695895195007324, "operator": "ge"}, {"budget": 0.48, "threshold": -0.8683147430419922, "operator": "ge"}, {"budget": 0.49, "threshold": -1.2258585691452026, "operator": "ge"}, {"budget": 0.5, "threshold": -1.4257622957229614, "operator": "ge"}, {"budget": 0.51, "threshold": -1.6395134925842285, "operator": "ge"}, {"budget": 0.52, "threshold": -2.4910292625427246, "operator": "ge"}, {"budget": 0.53, "threshold": -2.6544289588928223, "operator": "ge"}, {"budget": 0.54, "threshold": -2.737025499343872, "operator": "ge"}, {"budget": 0.55, "threshold": -2.8025612831115723, "operator": "ge"}, {"budget": 0.56, "threshold": -2.881666660308838, "operator": "ge"}, {"budget": 0.57, "threshold": -2.909935474395752, "operator": "ge"}, {"budget": 0.58, "threshold": -2.9538583755493164, "operator": "ge"}, {"budget": 0.59, "threshold": -2.99627423286438, "operator": "ge"}, {"budget": 0.6, "threshold": -3.0786843299865723, "operator": "ge"}, {"budget": 0.61, "threshold": -3.111453056335449, "operator": "ge"}, {"budget": 0.62, "threshold": -3.2126803398132324, "operator": "ge"}, {"budget": 0.63, "threshold": -3.2614293098449707, "operator": "ge"}, {"budget": 0.64, "threshold": -3.344348907470703, "operator": "ge"}, {"budget": 0.65, "threshold": -3.4308409690856934, "operator": "ge"}, {"budget": 0.66, "threshold": -3.5200257301330566, "operator": "ge"}, {"budget": 0.67, "threshold": -3.549604892730713, "operator": "ge"}, {"budget": 0.68, "threshold": -3.6301076412200928, "operator": "ge"}, {"budget": 0.69, "threshold": -3.6705121994018555, "operator": "ge"}, {"budget": 0.7, "threshold": -3.7156267166137695, "operator": "ge"}, {"budget": 0.71, "threshold": -3.7393596172332764, "operator": "ge"}, {"budget": 0.72, "threshold": -3.7591965198516846, "operator": "ge"}, {"budget": 0.73, "threshold": -3.8232944011688232, "operator": "ge"}, {"budget": 0.74, "threshold": -3.8620686531066895, "operator": "ge"}, {"budget": 0.75, "threshold": -3.9241113662719727, "operator": "ge"}, {"budget": 0.76, "threshold": -3.9508609771728516, "operator": "ge"}, {"budget": 0.77, "threshold": -4.030733585357666, "operator": "ge"}, {"budget": 0.78, "threshold": -4.048396110534668, "operator": "ge"}, {"budget": 0.79, "threshold": -4.126426696777344, "operator": "ge"}, {"budget": 0.8, "threshold": -4.176178932189941, "operator": "ge"}, {"budget": 0.81, "threshold": -4.347057342529297, "operator": "ge"}, {"budget": 0.82, "threshold": -11.880592346191406, "operator": "ge"}, {"budget": 0.83, "threshold": -11.981819152832031, "operator": "ge"}, {"budget": 0.84, "threshold": -12.069401741027832, "operator": "ge"}, {"budget": 0.85, "threshold": -12.112104415893555, "operator": "ge"}, {"budget": 0.86, "threshold": -12.149356842041016, "operator": "ge"}, {"budget": 0.87, "threshold": -12.180400848388672, "operator": "ge"}, {"budget": 0.88, "threshold": -12.26838493347168, "operator": "ge"}, {"budget": 0.89, "threshold": -12.327887535095215, "operator": "ge"}, {"budget": 0.9, "threshold": -12.380696296691895, "operator": "ge"}, {"budget": 0.91, "threshold": -12.431303024291992, "operator": "ge"}, {"budget": 0.92, "threshold": -12.471699714660645, "operator": "ge"}, {"budget": 0.93, "threshold": -12.541738510131836, "operator": "ge"}, {"budget": 0.94, "threshold": -12.590069770812988, "operator": "ge"}, {"budget": 0.95, "threshold": -12.626360893249512, "operator": "ge"}, {"budget": 0.96, "threshold": -12.663838386535645, "operator": "ge"}, {"budget": 0.97, "threshold": -12.705436706542969, "operator": "ge"}, {"budget": 0.98, "threshold": -12.771904945373535, "operator": "ge"}, {"budget": 0.99, "threshold": -12.866218566894531, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.53469589215203, 19.02522326060557, 19.523099670661487, 20.074061994976535, 20.488355047178576, 20.62666569167268, 20.814556869056517, 20.809483726150393, 21.121617025249765, 21.364850427350426, 21.725070345613638, 22.074322600026047, 22.517795480215018, 22.746544391281233, 23.092991316675526, 23.340056563740774, 23.53973050265826, 24.566142367412784, 25.022741557957534, 25.464177009795776, 25.56981190255936, 26.07458130517284, 26.642005122206985, 26.832871984509776, 26.95513080569819, 27.38793070121665, 27.72272221660431, 26.452694592229477, 26.922481538917815, 27.034666080038132, 27.255468645603177, 27.260382986189434, 28.218537702712844, 28.48753397140494, 28.720672029495553, 29.171898644515277, 29.440174969129636, 29.667733221530103, 29.805355504955735, 30.02419676308531, 30.427305286750567, 30.971993213372524, 31.50173137944201, 31.03611546499478, 32.74039155480323, 32.515316067947644, 32.59974700344241, 32.31797587060745, 31.840739905256033, 32.07786836819094, 32.645865771631385, 32.645865771631385, 32.574663239985824, 32.251091498884314, 32.03578942095772, 31.79434221179952, 31.613860928377058, 31.500224564740698, 31.569553757053757, 31.33281133281134, 31.050272047714504, 30.934638375189067, 30.66856330014225, 30.585738539898134, 30.44897504456328, 30.342682527556477, 29.88095238095238, 29.782913165266105, 29.57742220221252, 29.373664529914535, 29.16195879672225, 29.22525618401907, 28.661364533170897, 28.661364533170897, 28.37951637641486, 28.26501113265819, 28.04902304902305, 27.828628371899438, 27.584177705209694, 27.544838602141247, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.363625362132826, 27.293431775609996, 27.293431775609996, 27.152805996181673, 27.015836153576846, 26.940245602395745, 26.87276921912989, 26.87276921912989, 26.805281651050322, 26.571241258741264, 26.444978632478634, 26.222309035756965, 26.222309035756965, 26.093054404132666, 25.795595009828396, 25.668767695229906, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 24, 35, 47, 54, 62, 69, 74, 82, 88, 95, 103, 112, 117, 125, 132, 139, 147, 155, 163, 167, 174, 181, 186, 191, 198, 204, 208, 215, 221, 226, 231, 237, 243, 247, 254, 260, 263, 270, 276, 282, 288, 292, 294, 298, 299, 299, 300, 302, 303, 304, 304, 304, 304, 304, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 1, 2, 3, 5, 7, 9, 9, 13, 13, 13, 13, 14, 15, 17, 20, 21, 21, 22, 24, 25, 25, 28, 31, 31, 36, 38, 39, 44, 46, 50, 52, 55, 57, 60, 62, 62, 67, 70, 73, 78, 80, 84, 84, 86, 89, 91, 95, 98, 98, 98, 99, 103, 106, 109, 112, 114, 117, 118, 120, 122, 128, 129, 131, 133, 138, 140, 143, 146, 150, 154, 161, 161, 165, 167, 173, 176, 183, 184, 186, 186, 187, 187, 188, 188, 190, 192, 194, 195, 199, 200, 204, 205, 208, 208, 210, 214, 216, 220], "exact_score": [18.254072168965788, 18.53469589215203, 19.02522326060557, 19.523099670661487, 20.074061994976535, 20.488355047178576, 20.62666569167268, 20.814556869056517, 20.809483726150393, 21.121617025249765, 21.364850427350426, 21.725070345613638, 22.074322600026047, 22.517795480215018, 22.746544391281233, 23.092991316675526, 23.340056563740774, 23.53973050265826, 24.566142367412784, 25.022741557957534, 25.464177009795776, 25.56981190255936, 26.07458130517284, 26.642005122206985, 26.832871984509776, 26.95513080569819, 27.38793070121665, 27.72272221660431, 26.452694592229477, 26.922481538917815, 27.034666080038132, 27.255468645603177, 27.260382986189434, 28.218537702712844, 28.48753397140494, 28.720672029495553, 29.171898644515277, 29.440174969129636, 29.667733221530103, 29.805355504955735, 30.02419676308531, 30.427305286750567, 30.971993213372524, 31.50173137944201, 31.03611546499478, 32.74039155480323, 32.515316067947644, 32.59974700344241, 32.31797587060745, 31.840739905256033, 32.07786836819094, 32.645865771631385, 32.645865771631385, 32.574663239985824, 32.251091498884314, 32.03578942095772, 31.79434221179952, 31.613860928377058, 31.500224564740698, 31.569553757053757, 31.33281133281134, 31.050272047714504, 30.934638375189067, 30.66856330014225, 30.585738539898134, 30.44897504456328, 30.342682527556477, 29.88095238095238, 29.782913165266105, 29.57742220221252, 29.373664529914535, 29.16195879672225, 29.22525618401907, 28.661364533170897, 28.661364533170897, 28.37951637641486, 28.26501113265819, 28.04902304902305, 27.828628371899438, 27.584177705209694, 27.544838602141247, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.363625362132826, 27.293431775609996, 27.293431775609996, 27.152805996181673, 27.015836153576846, 26.940245602395745, 26.87276921912989, 26.87276921912989, 26.805281651050322, 26.571241258741264, 26.444978632478634, 26.222309035756965, 26.222309035756965, 26.093054404132666, 25.795595009828396, 25.668767695229906, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.4655930087098, "threshold_area100": 27.468317145923855, "threshold_area25": 22.60299505766646}, "test": {"score": [18.129624718318375, 18.40499104701074, 18.487455526350928, 19.070388717428397, 19.64854195879823, 19.75240457721975, 20.162003281381804, 20.568237986562913, 20.646990079993476, 20.82588232590394, 20.999778489997624, 21.212901719445735, 21.53160288875662, 21.399069223967164, 21.687961454636103, 21.896945491336677, 22.1246375608102, 22.17749293011268, 22.5010575988236, 22.606724950999503, 22.675752090507615, 22.830730166710477, 23.27845845769075, 23.434382099534968, 23.559275422256913, 24.219779120769218, 24.390140599523548, 24.572995882512515, 25.241387696655703, 25.371011422724436, 26.275129994205486, 26.639649361010747, 26.93198391025542, 27.07925881688613, 27.338555503153962, 27.29383881239002, 27.516338099500548, 28.082622155776615, 28.580281599363833, 28.778867769760197, 29.485734011341968, 29.548574077806478, 29.977159332802554, 30.086119629042617, 29.867299239239998, 29.868249809476865, 29.910604830665388, 29.914556425102333, 30.3444055944056, 29.791953718909745, 29.5128727703456, 29.847425071212424, 29.750754297777583, 30.092166100418332, 30.037867786720383, 29.851467698300098, 29.786901851977188, 29.76279140073106, 29.573875318031895, 29.998679807977723, 30.21164027683419, 30.10325517554363, 29.764177999678555, 29.726181555150816, 29.523751275227877, 29.60860598523282, 29.37732790505656, 29.393953431902602, 29.16900491488901, 28.979726951599783, 28.80255548689946, 28.69956450170322, 28.5212787759158, 28.55614420833405, 28.53240770133815, 28.77353243945478, 28.711998029641844, 28.48418658590678, 28.328477195397127, 28.08541808480811, 28.065713651310574, 27.884278749987224, 27.744175345887278, 27.715961993547054, 27.69306947682888, 27.64906885955305, 27.605954017448354, 27.589632501126832, 27.525247489602116, 27.41578993864721, 27.385078528628014, 27.296909435702936, 27.263863497867156, 27.191177774920888, 27.386517747063788, 27.36687297708924, 27.31214394067617, 27.24358265222814, 27.28328866929195, 27.235819356721, 27.001009067886244], "calls": [0.0, 1.2316910785619175, 2.0639147802929427, 3.362183754993342, 5.0599201065246335, 5.9920106524633825, 7.456724367509987, 8.55525965379494, 9.820239680426099, 10.386151797603196, 11.251664447403462, 12.017310252996005, 13.182423435419441, 14.11451398135819, 15.11318242343542, 15.81225033288948, 17.70972037283622, 18.07589880159787, 19.44074567243675, 20.605858854860188, 21.238348868175766, 22.03728362183755, 22.63648468708389, 23.03595206391478, 23.735019973368843, 24.400798934753663, 25.299600532623167, 26.23169107856192, 27.529960053262315, 28.262316910785618, 29.6271637816245, 30.65912117177097, 31.924101198402134, 32.92276964047936, 33.82157123834887, 34.48735019973369, 35.5525965379494, 36.884154460719046, 38.81491344873502, 39.513981358189085, 40.67909454061252, 41.54460719041278, 42.84287616511318, 43.37549933422104, 44.17443408788282, 44.67376830892144, 45.572569906790946, 46.07190412782956, 47.237017310252995, 48.46870838881491, 49.80026631158455, 50.7323568575233, 52.16378162450066, 53.56191744340879, 54.56058588548602, 55.45938748335553, 57.05725699067909, 57.52330226364847, 58.72170439414115, 59.45406125166445, 61.41810918774967, 62.150466045272964, 63.84820239680427, 64.68042609853528, 66.01198402130493, 67.37683089214381, 68.80825565912117, 69.34087882822902, 70.57256990679095, 71.6711051930759, 72.50332889480693, 72.96937416777631, 73.5352862849534, 74.73368841544608, 75.89880159786951, 77.33022636484687, 77.99600532623168, 79.16111850865512, 79.59387483355526, 80.49267643142477, 80.82556591211718, 81.85752330226364, 82.85619174434088, 83.88814913448735, 85.5525965379494, 86.55126498002663, 87.21704394141145, 87.7496671105193, 89.21438082556591, 90.11318242343542, 90.9121171770972, 91.54460719041279, 92.14380825565912, 93.74167776298269, 94.74034620505992, 95.1398135818908, 96.07190412782957, 96.83754993342211, 98.03595206391478, 98.96804260985353, 100.0], "rescued": [0, 73, 112, 173, 242, 281, 332, 366, 405, 424, 448, 468, 504, 533, 562, 582, 636, 644, 679, 703, 714, 736, 750, 761, 781, 800, 815, 836, 866, 881, 916, 943, 970, 991, 1014, 1026, 1048, 1081, 1127, 1140, 1166, 1180, 1208, 1215, 1228, 1232, 1240, 1244, 1250, 1250, 1253, 1255, 1256, 1258, 1259, 1259, 1260, 1260, 1260, 1263, 1265, 1266, 1266, 1267, 1268, 1270, 1270, 1271, 1271, 1271, 1271, 1271, 1272, 1273, 1274, 1276, 1277, 1277, 1277, 1278, 1278, 1278, 1278, 1278, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1280, 1281, 1281, 1281, 1282, 1282, 1282], "harmed": [0, 3, 8, 13, 18, 23, 33, 39, 48, 50, 56, 57, 63, 71, 75, 77, 91, 94, 99, 111, 115, 121, 124, 124, 131, 134, 135, 140, 154, 160, 168, 178, 188, 200, 209, 219, 228, 238, 257, 266, 278, 287, 303, 309, 313, 316, 321, 324, 330, 338, 350, 352, 361, 368, 375, 388, 403, 405, 419, 421, 433, 440, 455, 463, 475, 488, 501, 505, 518, 533, 543, 552, 564, 580, 598, 618, 627, 643, 651, 665, 666, 677, 686, 687, 690, 692, 695, 696, 702, 709, 712, 716, 723, 728, 731, 735, 740, 743, 755, 759, 777], "exact_score": [18.129624718318375, 18.31968435316348, 18.453694237136286, 19.021941839229886, 19.312427308784898, 19.631318207971834, 19.75240457721975, 20.035052529469105, 20.281480804413658, 20.57455685971825, 20.669744344340565, 20.943017863277223, 21.212901719445735, 21.454103466060946, 21.341368980749547, 21.65908851602963, 21.66529168380895, 21.912640336335116, 22.165890778561188, 22.378323399266275, 22.562608835177496, 22.630776079584436, 22.823145697216365, 23.41132009009127, 23.63522281052835, 24.341886317573614, 24.52027710084492, 24.96112123189176, 25.326143485631462, 26.141269130414774, 26.4666999757372, 26.701185325412695, 26.947855981362295, 27.078139058336507, 27.317384172502667, 27.36137742152379, 27.658504597641365, 28.11469630298668, 28.314832231666028, 28.619378407291652, 28.886207415115905, 29.383380352481474, 29.687545633967904, 30.00322086102305, 29.809601036479776, 29.898674170884643, 29.951949241451555, 29.94493678712169, 29.851868612912604, 29.811389731554588, 29.854338728014245, 29.847425071212424, 29.76352025522439, 30.0472071291768, 30.08209046078906, 29.942892427589594, 29.797111969155708, 29.786901851977188, 29.635421427974624, 29.68621683624727, 30.05236209886231, 30.29002886650361, 30.08213213621674, 29.818282649731565, 29.701400108927743, 29.685155936006748, 29.53507411422263, 29.68295505457229, 29.575727778939452, 29.359241451810043, 29.29318203220642, 29.127337628173432, 28.951942933663933, 28.69956450170322, 28.44110117255687, 28.828636524627736, 28.523923058541612, 28.336294084971282, 28.711998029641844, 28.541957003495465, 28.314258497070803, 28.065713651310574, 27.83131723224603, 27.744175345887278, 27.715961993547054, 27.65520862106755, 27.665461223414177, 27.61345157638265, 27.589632501126832, 27.539993767881533, 27.41578993864721, 27.385078528628014, 27.271062271062274, 27.230512987095707, 27.15912678742881, 27.36687297708924, 27.319111339729897, 27.236640270765268, 27.31477594843118, 27.235819356721, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.599700981969914, "threshold_area100": 26.60577617540757, "threshold_area25": 21.299916513721094}, "selected_budget": 45, "selected": {"val": {"score": 32.74039155480323, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.868249809476865, "calls": 44.67376830892144, "rescued": 1232, "harmed": 316}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.683976650238037, "operator": "ge"}, {"budget": 0.02, "threshold": 1.575854778289795, "operator": "ge"}, {"budget": 0.03, "threshold": 1.4890551567077637, "operator": "ge"}, {"budget": 0.04, "threshold": 1.4627492427825928, "operator": "ge"}, {"budget": 0.05, "threshold": 1.431347370147705, "operator": "ge"}, {"budget": 0.06, "threshold": 1.3899245262145996, "operator": "ge"}, {"budget": 0.07, "threshold": 1.3668527603149414, "operator": "ge"}, {"budget": 0.08, "threshold": 1.3378398418426514, "operator": "ge"}, {"budget": 0.09, "threshold": 1.287129521369934, "operator": "ge"}, {"budget": 0.1, "threshold": 1.2722809314727783, "operator": "ge"}, {"budget": 0.11, "threshold": 1.255643606185913, "operator": "ge"}, {"budget": 0.12, "threshold": 1.23103666305542, "operator": "ge"}, {"budget": 0.13, "threshold": 1.20755934715271, "operator": "ge"}, {"budget": 0.14, "threshold": 1.1818342208862305, "operator": "ge"}, {"budget": 0.15, "threshold": 1.1603361368179321, "operator": "ge"}, {"budget": 0.16, "threshold": 1.1372902393341064, "operator": "ge"}, {"budget": 0.17, "threshold": 1.106074571609497, "operator": "ge"}, {"budget": 0.18, "threshold": 1.0681297779083252, "operator": "ge"}, {"budget": 0.19, "threshold": 1.0554020404815674, "operator": "ge"}, {"budget": 0.2, "threshold": 1.033046007156372, "operator": "ge"}, {"budget": 0.21, "threshold": 1.0227878093719482, "operator": "ge"}, {"budget": 0.22, "threshold": 0.9884446859359741, "operator": "ge"}, {"budget": 0.23, "threshold": 0.9658594131469727, "operator": "ge"}, {"budget": 0.24, "threshold": 0.943770706653595, "operator": "ge"}, {"budget": 0.25, "threshold": 0.9224339723587036, "operator": "ge"}, {"budget": 0.26, "threshold": 0.9072825908660889, "operator": "ge"}, {"budget": 0.27, "threshold": 0.8847324848175049, "operator": "ge"}, {"budget": 0.28, "threshold": 0.8550193309783936, "operator": "ge"}, {"budget": 0.29, "threshold": 0.8309628963470459, "operator": "ge"}, {"budget": 0.3, "threshold": 0.7916327714920044, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7669000029563904, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7279430031776428, "operator": "ge"}, {"budget": 0.33, "threshold": 0.6963598132133484, "operator": "ge"}, {"budget": 0.34, "threshold": 0.6831260323524475, "operator": "ge"}, {"budget": 0.35, "threshold": 0.6560971736907959, "operator": "ge"}, {"budget": 0.36, "threshold": 0.631113588809967, "operator": "ge"}, {"budget": 0.37, "threshold": 0.6135830879211426, "operator": "ge"}, {"budget": 0.38, "threshold": 0.5980089902877808, "operator": "ge"}, {"budget": 0.39, "threshold": 0.5669659376144409, "operator": "ge"}, {"budget": 0.4, "threshold": 0.5371390581130981, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5119112133979797, "operator": "ge"}, {"budget": 0.42, "threshold": 0.47489041090011597, "operator": "ge"}, {"budget": 0.43, "threshold": 0.4340418577194214, "operator": "ge"}, {"budget": 0.44, "threshold": 0.3298385739326477, "operator": "ge"}, {"budget": 0.45, "threshold": 0.20792287588119507, "operator": "ge"}, {"budget": 0.46, "threshold": 0.002172112464904785, "operator": "ge"}, {"budget": 0.47, "threshold": -0.14685416221618652, "operator": "ge"}, {"budget": 0.48, "threshold": -0.4426653981208801, "operator": "ge"}, {"budget": 0.49, "threshold": -0.6783138513565063, "operator": "ge"}, {"budget": 0.5, "threshold": -0.8856829404830933, "operator": "ge"}, {"budget": 0.51, "threshold": -1.1165763139724731, "operator": "ge"}, {"budget": 0.52, "threshold": -2.2926886081695557, "operator": "ge"}, {"budget": 0.53, "threshold": -2.334247350692749, "operator": "ge"}, {"budget": 0.54, "threshold": -2.4609427452087402, "operator": "ge"}, {"budget": 0.55, "threshold": -2.576019763946533, "operator": "ge"}, {"budget": 0.56, "threshold": -2.630326986312866, "operator": "ge"}, {"budget": 0.57, "threshold": -2.6750943660736084, "operator": "ge"}, {"budget": 0.58, "threshold": -2.7075729370117188, "operator": "ge"}, {"budget": 0.59, "threshold": -2.7346439361572266, "operator": "ge"}, {"budget": 0.6, "threshold": -2.7944529056549072, "operator": "ge"}, {"budget": 0.61, "threshold": -2.827171802520752, "operator": "ge"}, {"budget": 0.62, "threshold": -2.90688419342041, "operator": "ge"}, {"budget": 0.63, "threshold": -2.929354429244995, "operator": "ge"}, {"budget": 0.64, "threshold": -2.991450071334839, "operator": "ge"}, {"budget": 0.65, "threshold": -3.0268971920013428, "operator": "ge"}, {"budget": 0.66, "threshold": -3.1013989448547363, "operator": "ge"}, {"budget": 0.67, "threshold": -3.127811908721924, "operator": "ge"}, {"budget": 0.68, "threshold": -3.1638283729553223, "operator": "ge"}, {"budget": 0.69, "threshold": -3.1938531398773193, "operator": "ge"}, {"budget": 0.7, "threshold": -3.2362170219421387, "operator": "ge"}, {"budget": 0.71, "threshold": -3.2787082195281982, "operator": "ge"}, {"budget": 0.72, "threshold": -3.3175525665283203, "operator": "ge"}, {"budget": 0.73, "threshold": -3.3435447216033936, "operator": "ge"}, {"budget": 0.74, "threshold": -3.3926258087158203, "operator": "ge"}, {"budget": 0.75, "threshold": -3.423229694366455, "operator": "ge"}, {"budget": 0.76, "threshold": -3.4844489097595215, "operator": "ge"}, {"budget": 0.77, "threshold": -3.5152394771575928, "operator": "ge"}, {"budget": 0.78, "threshold": -3.5955448150634766, "operator": "ge"}, {"budget": 0.79, "threshold": -3.6774353981018066, "operator": "ge"}, {"budget": 0.8, "threshold": -3.768746852874756, "operator": "ge"}, {"budget": 0.81, "threshold": -3.864328384399414, "operator": "ge"}, {"budget": 0.82, "threshold": -10.018043518066406, "operator": "ge"}, {"budget": 0.83, "threshold": -10.1434326171875, "operator": "ge"}, {"budget": 0.84, "threshold": -10.217123031616211, "operator": "ge"}, {"budget": 0.85, "threshold": -10.262471199035645, "operator": "ge"}, {"budget": 0.86, "threshold": -10.310810089111328, "operator": "ge"}, {"budget": 0.87, "threshold": -10.345222473144531, "operator": "ge"}, {"budget": 0.88, "threshold": -10.373505592346191, "operator": "ge"}, {"budget": 0.89, "threshold": -10.398695945739746, "operator": "ge"}, {"budget": 0.9, "threshold": -10.432151794433594, "operator": "ge"}, {"budget": 0.91, "threshold": -10.487712860107422, "operator": "ge"}, {"budget": 0.92, "threshold": -10.524001121520996, "operator": "ge"}, {"budget": 0.93, "threshold": -10.572145462036133, "operator": "ge"}, {"budget": 0.94, "threshold": -10.627103805541992, "operator": "ge"}, {"budget": 0.95, "threshold": -10.680723190307617, "operator": "ge"}, {"budget": 0.96, "threshold": -10.8502779006958, "operator": "ge"}, {"budget": 0.97, "threshold": -10.9927978515625, "operator": "ge"}, {"budget": 0.98, "threshold": -11.076298713684082, "operator": "ge"}, {"budget": 0.99, "threshold": -11.153421401977539, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.516669362447082, 18.806380800989967, 19.167698005535357, 19.418367663056205, 19.650358627108037, 19.864933409417784, 20.15674054856842, 20.310185882699816, 20.62920115475163, 20.994155648627196, 21.272074207555757, 21.55384828349945, 21.695537958918866, 21.907040328092958, 22.20355620747359, 22.587980983583563, 22.768045204468503, 23.093395252837976, 23.57306087954921, 23.833591753524562, 23.87163561076604, 23.983110920568816, 24.32139406405804, 24.771126760563384, 25.810422684931417, 26.213022917006633, 26.501896833092587, 26.880424474818064, 27.81278717002983, 28.240388275599543, 28.721461187214608, 29.079463883077423, 29.522471256342225, 29.94094019909237, 28.90059451708205, 29.2806847923747, 29.48489010989011, 29.699477351916375, 29.97724948131503, 30.35924895186959, 30.647495900484984, 31.042163455956562, 31.258572328623952, 31.15926817435438, 32.78813352826511, 32.515316067947644, 32.417701264450486, 32.31797587060745, 32.0283937814345, 32.18539524991137, 32.645865771631385, 32.574663239985824, 32.40441234392847, 32.18497336782349, 32.057404267894356, 31.79434221179952, 31.361844799344794, 31.653587370499135, 31.593864484689295, 31.535425962015147, 31.45139234856977, 30.878647333755694, 30.770128614276583, 30.634705495101265, 30.42462146148782, 30.11006951759006, 29.887057387057386, 29.841269841269842, 29.672897196261683, 29.503521173532643, 28.90149528280575, 28.782682320392578, 28.62013514786355, 28.419354128117014, 28.37951637641486, 27.828964621030284, 27.754396332176277, 27.869273270964605, 27.544838602141247, 27.544838602141247, 27.434520884520886, 27.363625362132826, 27.363625362132826, 27.152805996181673, 27.152805996181673, 27.152805996181673, 27.152805996181673, 27.152805996181673, 26.978413743119628, 26.84277696413619, 26.84277696413619, 26.6790190247877, 26.575274061687104, 26.575274061687104, 26.509081196581196, 26.378586691086692, 26.070064687085964, 25.731882231504155, 25.545267404678757, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 20, 30, 36, 45, 51, 60, 64, 72, 81, 88, 95, 102, 110, 118, 126, 133, 140, 148, 155, 160, 166, 172, 178, 187, 193, 200, 207, 213, 220, 227, 232, 238, 245, 250, 255, 261, 266, 271, 276, 280, 286, 293, 295, 298, 299, 299, 300, 302, 303, 304, 304, 304, 304, 304, 304, 304, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 2, 4, 5, 5, 6, 10, 10, 11, 12, 13, 16, 19, 20, 20, 22, 24, 24, 27, 30, 33, 35, 38, 40, 43, 45, 46, 47, 49, 52, 54, 56, 58, 59, 61, 64, 66, 69, 71, 75, 79, 82, 84, 84, 86, 90, 91, 94, 97, 98, 99, 101, 104, 106, 109, 113, 116, 117, 118, 119, 123, 125, 127, 132, 137, 141, 142, 145, 150, 157, 159, 162, 164, 165, 170, 171, 175, 180, 184, 186, 187, 187, 194, 194, 194, 194, 194, 197, 199, 199, 201, 203, 203, 204, 206, 210, 215, 218, 220], "exact_score": [18.254072168965788, 18.516669362447082, 18.806380800989967, 19.167698005535357, 19.418367663056205, 19.650358627108037, 19.864933409417784, 20.15674054856842, 20.310185882699816, 20.62920115475163, 20.994155648627196, 21.272074207555757, 21.55384828349945, 21.695537958918866, 21.907040328092958, 22.20355620747359, 22.587980983583563, 22.768045204468503, 23.093395252837976, 23.57306087954921, 23.833591753524562, 23.87163561076604, 23.983110920568816, 24.32139406405804, 24.771126760563384, 25.810422684931417, 26.213022917006633, 26.501896833092587, 26.880424474818064, 27.81278717002983, 28.240388275599543, 28.721461187214608, 29.079463883077423, 29.522471256342225, 29.94094019909237, 28.90059451708205, 29.2806847923747, 29.48489010989011, 29.699477351916375, 29.97724948131503, 30.35924895186959, 30.647495900484984, 31.042163455956562, 31.258572328623952, 31.15926817435438, 32.78813352826511, 32.515316067947644, 32.417701264450486, 32.31797587060745, 32.0283937814345, 32.18539524991137, 32.645865771631385, 32.574663239985824, 32.40441234392847, 32.18497336782349, 32.057404267894356, 31.79434221179952, 31.361844799344794, 31.653587370499135, 31.593864484689295, 31.535425962015147, 31.45139234856977, 30.878647333755694, 30.770128614276583, 30.634705495101265, 30.42462146148782, 30.11006951759006, 29.887057387057386, 29.841269841269842, 29.672897196261683, 29.503521173532643, 28.90149528280575, 28.782682320392578, 28.62013514786355, 28.419354128117014, 28.37951637641486, 27.828964621030284, 27.754396332176277, 27.869273270964605, 27.544838602141247, 27.544838602141247, 27.434520884520886, 27.363625362132826, 27.363625362132826, 27.152805996181673, 27.152805996181673, 27.152805996181673, 27.152805996181673, 27.152805996181673, 26.978413743119628, 26.84277696413619, 26.84277696413619, 26.6790190247877, 26.575274061687104, 26.575274061687104, 26.509081196581196, 26.378586691086692, 26.070064687085964, 25.731882231504155, 25.545267404678757, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.286120896173312, "threshold_area100": 27.288676859148424, "threshold_area25": 21.7427819327364}, "test": {"score": [18.129624718318375, 18.326793794897117, 18.2910929253395, 18.975609602135382, 19.154867900057297, 19.263489895804202, 19.49936998692875, 19.887756359034096, 19.976736030777587, 20.414405261524003, 20.59115012373475, 20.79193302448093, 21.03779662877833, 21.227427980058057, 21.4778824272176, 21.786752620727, 21.911860290093045, 21.74920293318046, 22.555301465590272, 22.955493533515924, 23.22077138426182, 23.361054392698957, 23.9083429156805, 24.306246742123385, 24.267043523615794, 24.42128779127063, 24.510408770385745, 24.791095849256138, 24.824659629629707, 25.018332749018292, 25.453587085794123, 25.73057614483007, 26.295368535283316, 26.612308098853184, 27.13214360244666, 27.506344964418634, 27.606513271917322, 27.93077464333012, 28.762613749107445, 29.203167829049264, 29.581757501642237, 29.7927638315698, 30.032706829377677, 30.1376985953148, 30.279059624691552, 29.905102810790417, 29.910604830665388, 29.914556425102333, 30.294463213290012, 29.887631975867272, 30.083315025779438, 29.841083160675964, 29.738056064242958, 29.725425017129155, 29.946572187975317, 29.77962207208082, 29.744076904245503, 29.656105377229995, 29.518933134137036, 29.838360638127977, 30.097007409736463, 30.063071822427123, 29.841975573724234, 29.603759087298904, 29.397344995970414, 29.20355772838136, 28.97070394557207, 28.873202528808932, 29.068946807971198, 28.879841681571694, 28.639800816512572, 28.5275494498222, 28.276340515359998, 28.418533288963392, 28.18910033227151, 28.30943143396667, 28.331601151896034, 28.343400530211255, 28.26956834921558, 28.06406381583834, 27.9452939071524, 27.884278749987224, 27.739717941197178, 27.69659308688725, 27.61638643160383, 27.743763398461873, 27.641314639115905, 27.575618197989694, 27.53355209583304, 27.482856652892774, 27.444396071445254, 27.42849015239099, 27.42849015239099, 27.42136976822142, 27.351803198597946, 27.294633520892642, 27.176888309064594, 27.062414891147053, 27.07216530745942, 26.94259496234819, 27.001009067886244], "calls": [0.0, 0.4993342210386152, 1.3648468708388815, 2.929427430093209, 3.561917443408788, 4.693741677762983, 5.758988015978695, 6.790945406125166, 7.723035952063914, 9.487350199733688, 10.053262316910786, 10.752330226364846, 11.78428761651132, 12.88282290279627, 14.247669773635153, 15.446071904127828, 16.311584553928096, 17.310252996005325, 18.941411451398135, 19.37416777629827, 20.10652463382157, 20.605858854860188, 22.270306258322236, 23.302263648468706, 24.36750998668442, 25.06657789613848, 25.466045272969374, 26.2982689747004, 27.230359520639148, 27.896138482023968, 29.527296937416775, 30.692410119840215, 32.35685752330226, 33.38881491344873, 34.12117177097203, 35.21970705725699, 36.41810918774967, 37.150466045272964, 37.849533954727036, 38.98135818908123, 40.37949400798935, 41.211717709720375, 42.0439414114514, 42.84287616511318, 43.70838881491345, 44.70705725699068, 45.572569906790946, 46.07190412782956, 46.870838881491345, 48.368841544607186, 49.60053262316911, 50.53262316910786, 52.36351531291611, 52.762982689747005, 54.29427430093209, 55.992010652463385, 57.19041278295606, 58.25565912117177, 59.18774966711052, 59.92010652463382, 61.21837549933422, 62.18375499334221, 63.84820239680427, 64.48069241011984, 66.37816245006658, 67.01065246338216, 68.47536617842877, 69.2410119840213, 70.0399467376831, 70.97203728362183, 71.90412782956058, 72.83621837549934, 73.8681757656458, 74.56724367509987, 75.83222370173102, 76.4647137150466, 77.89613848202397, 78.59520639147803, 79.89347536617842, 80.65912117177098, 81.49134487350199, 81.8242343541944, 82.85619174434088, 84.0878828229028, 85.31957390146472, 86.31824234354194, 88.11584553928095, 89.41411451398136, 90.27962716378163, 90.84553928095872, 91.94407456724367, 92.87616511318242, 93.47536617842877, 94.10785619174435, 94.54061251664447, 95.10652463382158, 96.47137150466045, 97.50332889480693, 98.1358189081225, 98.76830892143809, 100.0], "rescued": [0, 26, 62, 125, 148, 193, 228, 263, 289, 341, 363, 386, 415, 448, 492, 529, 551, 578, 630, 645, 673, 686, 734, 759, 781, 799, 807, 830, 854, 872, 907, 943, 983, 1007, 1028, 1055, 1077, 1095, 1114, 1138, 1164, 1182, 1199, 1209, 1221, 1233, 1240, 1244, 1247, 1250, 1253, 1254, 1256, 1256, 1258, 1259, 1260, 1260, 1260, 1263, 1265, 1266, 1266, 1266, 1268, 1268, 1269, 1269, 1270, 1270, 1270, 1271, 1272, 1273, 1273, 1274, 1275, 1276, 1278, 1278, 1278, 1278, 1278, 1278, 1278, 1279, 1279, 1279, 1279, 1279, 1280, 1280, 1280, 1280, 1280, 1280, 1280, 1280, 1281, 1281, 1282], "harmed": [0, 1, 6, 11, 13, 21, 26, 30, 36, 44, 48, 51, 55, 60, 73, 80, 86, 97, 108, 111, 116, 119, 130, 136, 141, 147, 148, 155, 165, 168, 183, 192, 206, 211, 219, 229, 240, 248, 256, 267, 281, 291, 302, 306, 309, 316, 321, 324, 330, 336, 343, 351, 362, 363, 381, 392, 404, 414, 419, 425, 436, 444, 454, 465, 485, 494, 508, 516, 523, 534, 546, 561, 579, 587, 603, 612, 629, 637, 656, 666, 673, 677, 687, 690, 696, 703, 710, 715, 718, 721, 725, 726, 726, 727, 731, 739, 746, 753, 757, 766, 777], "exact_score": [18.129624718318375, 18.179179719432558, 18.469478154275834, 18.990421006819865, 19.078997660433632, 19.357949489905817, 19.466464908149653, 19.930082157976454, 19.992187731958495, 20.281941364614514, 20.569922925959297, 20.860252640525296, 21.10753027389852, 21.230246704051332, 21.40510893722658, 21.796255148966438, 21.78988587576594, 21.702383177958897, 22.168949318545245, 22.57494710348275, 23.144740626940134, 23.50631257139884, 23.757068644572094, 24.064668869559245, 24.184339515453082, 24.405923676624006, 24.671042233344913, 24.864004038753883, 25.04502098562962, 25.367437641290564, 25.591543572622072, 25.83042411911069, 26.030000716580787, 26.511906305451006, 27.151548951401487, 27.498147487110376, 27.604016193988386, 27.86135874668328, 28.797491024693976, 29.203167829049264, 29.35900732513077, 29.705351244157217, 29.979985701802224, 30.1826999432785, 30.002470301940253, 29.936744120637805, 29.92809712457618, 30.294463213290012, 29.887231366422807, 29.867033044170487, 30.06251826605803, 29.834312227910896, 29.750754297777583, 30.0472071291768, 29.959020879180486, 29.93063525217285, 29.77962207208082, 29.77529320816055, 29.656105377229995, 29.537867887205582, 29.787491159877135, 30.12882724126107, 29.965344350797263, 30.032898702748135, 29.811457719736072, 29.57061870463795, 29.49024175598177, 29.231907831992938, 29.174646980808244, 28.9296639315584, 29.097574979451146, 28.879841681571694, 28.585471769561156, 28.48318614685831, 28.276340515359998, 28.36637587895089, 28.148493844737633, 28.258785079545145, 28.331601151896034, 28.217217659714976, 28.230133699597047, 27.960718090119087, 27.83131723224603, 27.739717941197178, 27.69659308688725, 27.623773224704905, 27.805315625066708, 27.67926292693858, 27.664732709154272, 27.598913893420036, 27.552383305424407, 27.42277689832715, 27.444396071445254, 27.42849015239099, 27.42136976822142, 27.32888792548246, 27.19966020815231, 27.127740971375992, 27.04860464394363, 26.89793176983918, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.556649301120693, "threshold_area100": 26.517621992704825, "threshold_area25": 21.227524055864457}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.905102810790417, "calls": 44.70705725699068, "rescued": 1233, "harmed": 316}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 33.167859760757345, "calls": 40.94151212553495}, "test": {"score": 29.66589960709296, "calls": 41.84420772303595}}, "old_normalized_interior": {"val": {"score": 33.167859760757345, "calls": 40.94151212553495}, "test": {"score": 29.66589960709296, "calls": 41.84420772303595}}, "old_normalized_endpoints": {"val": {"score": 33.167859760757345, "calls": 40.94151212553495}, "test": {"score": 29.66589960709296, "calls": 41.84420772303595}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.640704014464504, 19.193336884465463, 19.693350489955964, 20.24725333530077, 20.757958151966065, 21.150781114252766, 21.48448216223953, 22.694566185461184, 23.55829085321694, 24.15299295702431, 24.698275862068964, 25.411074068664014, 25.761565623316773, 24.625889350631294, 25.369371169730883, 25.516677567037284, 25.42189317721362, 25.79497414531059, 26.191019541355985, 26.03283058339238, 26.319180397127163, 26.543927648578812, 26.873616644532976, 26.485714285714284, 26.802536832717916, 26.71237821652759, 26.753002761036086, 26.82671291145867, 27.089175192623465, 26.983283973614352, 27.098709048836927, 27.51424014581909, 28.148041474654377, 28.45075016307893, 28.874064951651157, 30.377183823445502, 30.444096201138453, 30.887515985857217, 31.791653394914267, 31.12898598395556, 33.167859760757345, 32.87402276901888, 33.00401291780602, 32.557430233003444, 32.502848029163815, 32.587782587782584, 32.37024011642008, 32.31797587060745, 32.04915866630637, 32.04915866630637, 31.964880840093063, 31.881504381005026, 31.799014905524313, 31.525314589830717, 31.321404431292617, 31.321404431292617, 31.321404431292617, 31.321404431292617, 31.321404431292617, 31.263129373017556, 30.931201422546884, 30.81625889381125, 30.45058622293078, 30.17373251748252, 30.04002215537681, 30.04002215537681, 29.874777866908214, 29.86686198221664, 29.574739811138777, 29.52072746545976, 29.245216094778613, 29.193790881731307, 29.117673152977495, 28.868547418967587, 28.44427780341486, 27.94042327750193, 28.03394009972957, 27.785235189005807, 27.737456880357936, 28.176864835954607, 27.908439835236763, 27.792638709702878, 27.76704663899786, 27.599549069918194, 27.331740266055032, 27.19168424364607, 27.146760613816777, 27.505863833526593, 27.331101035369333, 27.46742107001673, 27.36823931945883, 27.144536082314286, 27.144536082314286, 26.94410082586915, 26.629329390408056, 26.230369742030497, 26.151046632564608, 25.83783509766467, 25.763430335759907, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.640704014464504, 19.193336884465463, 19.693350489955964, 20.24725333530077, 20.757958151966065, 21.150781114252766, 21.48448216223953, 22.694566185461184, 23.55829085321694, 24.15299295702431, 24.698275862068964, 25.411074068664014, 25.761565623316773, 24.625889350631294, 25.369371169730883, 25.516677567037284, 25.42189317721362, 25.79497414531059, 26.191019541355985, 26.03283058339238, 26.319180397127163, 26.543927648578812, 26.873616644532976, 26.485714285714284, 26.802536832717916, 26.71237821652759, 26.753002761036086, 26.82671291145867, 27.089175192623465, 26.983283973614352, 27.098709048836927, 27.51424014581909, 28.148041474654377, 28.45075016307893, 28.874064951651157, 30.377183823445502, 30.444096201138453, 30.887515985857217, 31.791653394914267, 31.12898598395556, 33.167859760757345, 32.87402276901888, 33.00401291780602, 32.557430233003444, 32.502848029163815, 32.587782587782584, 32.37024011642008, 32.31797587060745, 32.04915866630637, 32.04915866630637, 31.964880840093063, 31.881504381005026, 31.799014905524313, 31.525314589830717, 31.321404431292617, 31.321404431292617, 31.321404431292617, 31.321404431292617, 31.321404431292617, 31.263129373017556, 30.931201422546884, 30.81625889381125, 30.45058622293078, 30.17373251748252, 30.04002215537681, 30.04002215537681, 29.874777866908214, 29.86686198221664, 29.574739811138777, 29.52072746545976, 29.245216094778613, 29.193790881731307, 29.117673152977495, 28.868547418967587, 28.44427780341486, 27.94042327750193, 28.03394009972957, 27.785235189005807, 27.737456880357936, 28.176864835954607, 27.908439835236763, 27.792638709702878, 27.76704663899786, 27.599549069918194, 27.331740266055032, 27.19168424364607, 27.146760613816777, 27.505863833526593, 27.331101035369333, 27.46742107001673, 27.36823931945883, 27.144536082314286, 27.144536082314286, 26.94410082586915, 26.629329390408056, 26.230369742030497, 26.151046632564608, 25.83783509766467, 25.763430335759907, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 23.768291472591102, "exact_area100": 27.90634171081015}, "test": {"score": [18.129624718318375, 18.253657382347534, 18.054429667836395, 18.364766996850566, 18.690857836457788, 19.494355229262535, 19.84841800027902, 20.292203738594313, 20.23714155043024, 20.358877102839624, 20.6379714425425, 20.934900704160416, 21.191626683345234, 22.58441570805682, 22.966941523895372, 23.57657304421778, 23.803986289734265, 23.87763660420585, 23.92374774614963, 24.781642672243503, 24.953990061602436, 24.92154947260996, 25.210374333686893, 25.304455706894736, 25.55746857949799, 25.640790158882126, 25.60813617079602, 25.855002640959224, 26.044198905067333, 25.99129479628031, 26.492013949188276, 26.55067592530802, 26.764544118841027, 27.379064999588532, 27.607555119683262, 28.077458520524033, 28.183153956431045, 28.29818813384959, 28.823737866837014, 28.925977947415287, 29.381778222934706, 29.66589960709296, 29.53361278649017, 29.717366809951805, 29.74261085147612, 30.29287022535189, 30.343803821797867, 30.299131523635264, 30.191356769414156, 30.155595094759395, 30.195559959570822, 30.189873249371797, 30.114195505986547, 30.10102374623944, 29.96627111922729, 29.867591489794602, 29.726472373427974, 29.586290491802302, 29.72111516080381, 29.841042921964146, 29.74560244423258, 30.04596979752259, 29.92165357724645, 29.882162324700772, 30.15513587834408, 30.136510078872725, 30.052288077207994, 29.978476750628648, 29.751595810503524, 29.239264984600798, 29.046242890237323, 28.947831369976697, 29.133417139591433, 29.027395084592577, 29.223664223414602, 28.867775436484045, 28.74588779179488, 28.63434920473026, 28.67676953028417, 28.556691891967056, 28.48789296635411, 28.48191135283432, 28.378823875400837, 28.596013257072723, 28.55057844447282, 28.733840719884853, 28.70850416964259, 28.546166750926382, 28.437346413075744, 28.335711136039386, 28.153125399071392, 28.315061107600997, 28.290463382787053, 28.05618657879212, 27.847828824830977, 27.772758766381617, 27.707751642420487, 27.704821138304297, 27.67336562553881, 27.356869552280706, 27.001009067886244], "calls": [0.0, 1.3315579227696404, 2.2969374167776295, 3.062583222370173, 4.027962716378163, 6.291611185086551, 7.023968042609853, 8.222370173102531, 9.354194407456724, 10.086551264980026, 11.118508655126497, 12.050599201065246, 12.88282290279627, 14.314247669773636, 15.346205059920107, 16.611185086551263, 17.243675099866845, 17.576564580559253, 17.942743009320907, 19.44074567243675, 20.938748335552596, 21.80426098535286, 22.836218375499335, 24.134487350199734, 25.299600532623167, 26.198402130492678, 26.830892143808256, 27.96271637816245, 28.961384820239683, 29.460719041278296, 30.392809587217045, 30.85885486018642, 31.790945406125164, 33.18908122503329, 34.12117177097203, 35.28628495339547, 36.25166444740346, 36.684420772303596, 37.31691078561917, 38.08255659121172, 40.67909454061252, 41.84420772303595, 42.41011984021305, 43.142476697736356, 43.70838881491345, 45.07323568575233, 45.60585885486019, 45.80559254327563, 46.6711051930759, 47.20372836218375, 48.435419440745676, 50.09986684420772, 51.398135818908116, 51.89747003994674, 54.76031957390146, 55.85885486018641, 56.524633821571236, 57.52330226364847, 59.18774966711052, 60.35286284953395, 60.9520639147803, 61.784287616511314, 62.81624500665779, 64.01464713715048, 64.74700399467378, 65.27962716378163, 66.24500665778962, 67.17709720372837, 67.90945406125167, 68.97470039946738, 69.94007989347537, 70.83888149134488, 71.60452729693742, 72.20372836218375, 73.1358189081225, 76.26498002663115, 77.33022636484687, 77.99600532623168, 78.728362183755, 79.79360852197071, 80.42609853528629, 81.05858854860186, 82.09054593874833, 82.68974700399467, 83.02263648468708, 83.48868175765645, 84.68708388814915, 85.9520639147803, 86.75099866844208, 87.98268974700399, 89.18109187749667, 90.0133155792277, 91.54460719041279, 92.67643142476697, 94.10785619174435, 95.63914780292943, 96.20505992010652, 96.80426098535287, 97.20372836218375, 98.76830892143809, 100.0], "exact_score": [18.129624718318375, 18.192410073930475, 18.288519263957014, 18.320525953391105, 18.67076102374577, 19.254739169974453, 19.41121616061463, 19.849850554457294, 20.215773970839106, 20.139340750940296, 20.312435870479682, 20.596897699142087, 20.92664314899305, 21.229087240431127, 22.07247406126482, 22.82014675024072, 23.243041365144236, 23.719805245224403, 23.932972492234587, 24.68758646266873, 24.929653931468813, 24.953990061602436, 24.961974082781506, 25.203374981309288, 25.275747083153817, 25.531881323144745, 25.575009330563304, 25.581679890382404, 25.867198462814134, 26.046052560982524, 26.140383139635997, 26.601901293185904, 27.113914043569082, 27.33368897585314, 27.56456901852264, 28.07404724089783, 28.261837968205995, 28.784186175530323, 28.94130683231402, 28.846958003852013, 29.144142137436756, 29.49046155177215, 29.613492013953913, 29.68619936302718, 29.78113106346244, 30.29287022535189, 30.308140949600027, 30.155595094759395, 30.195559959570822, 30.237492296990844, 30.20319075389975, 30.167429079244982, 30.08792396332156, 30.046348509146505, 29.979158476361732, 29.96627111922729, 29.867591489794602, 29.625967127223618, 29.527405083124425, 29.653216751386047, 29.841042921964146, 29.726801034126826, 29.989016464571755, 29.890229174003135, 29.938188489596612, 30.136510078872725, 30.07465944186124, 29.978476750628648, 29.751595810503524, 29.239264984600798, 29.046242890237323, 29.256203553348882, 29.056821955850726, 29.241595116570124, 29.137743429047774, 28.964648374105522, 28.887429524534358, 28.812828716752243, 28.63434920473026, 28.659327890264453, 28.50514016368455, 28.48191135283432, 28.378823875400837, 28.55057844447282, 28.77373387236068, 28.700863046615748, 28.546166750926382, 28.353258446603423, 28.335711136039386, 28.22138260131457, 28.315061107600997, 28.398353582665177, 28.20440641015801, 27.979699517033673, 27.764160482109318, 27.92607538523159, 27.779023798291536, 27.697094440587083, 27.558614890962634, 27.30182812037027, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 22.1096194432451, "exact_area100": 27.00230600331005}}, "old_normalized_endpoint_budget": 41}, {"seed": 2030, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.75, "validation_area100": 0.28196371825681893, "training_seconds": 4.165323151042685, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5736990311342672, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5477351931698626, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5008606137282251, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4862015400896909, "operator": "ge"}, {"budget": 0.05, "threshold": 0.46514860858883567, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4488911383221874, "operator": "ge"}, {"budget": 0.07, "threshold": 0.42827464602382986, "operator": "ge"}, {"budget": 0.08, "threshold": 0.40650575210291096, "operator": "ge"}, {"budget": 0.09, "threshold": 0.3985981557101067, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3893704183709079, "operator": "ge"}, {"budget": 0.11, "threshold": 0.38300868567076146, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3782306454396576, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3719832600903249, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3646224866238583, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3578904324451177, "operator": "ge"}, {"budget": 0.16, "threshold": 0.35531434468001466, "operator": "ge"}, {"budget": 0.17, "threshold": 0.35041316839810027, "operator": "ge"}, {"budget": 0.18, "threshold": 0.3450873696737082, "operator": "ge"}, {"budget": 0.19, "threshold": 0.3399485841166982, "operator": "ge"}, {"budget": 0.2, "threshold": 0.32635711924671773, "operator": "ge"}, {"budget": 0.21, "threshold": 0.3185626789310219, "operator": "ge"}, {"budget": 0.22, "threshold": 0.30335564035241913, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2992804095521561, "operator": "ge"}, {"budget": 0.24, "threshold": 0.28672571333201885, "operator": "ge"}, {"budget": 0.25, "threshold": 0.28348929386577815, "operator": "ge"}, {"budget": 0.26, "threshold": 0.2776497772419256, "operator": "ge"}, {"budget": 0.27, "threshold": 0.2688064791203146, "operator": "ge"}, {"budget": 0.28, "threshold": 0.26304394696499594, "operator": "ge"}, {"budget": 0.29, "threshold": 0.257251245680479, "operator": "ge"}, {"budget": 0.3, "threshold": 0.253942813693339, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2489711319617611, "operator": "ge"}, {"budget": 0.32, "threshold": 0.23984685195778888, "operator": "ge"}, {"budget": 0.33, "threshold": 0.22925623431280903, "operator": "ge"}, {"budget": 0.34, "threshold": 0.21590528316503926, "operator": "ge"}, {"budget": 0.35, "threshold": 0.20901549622380586, "operator": "ge"}, {"budget": 0.36, "threshold": 0.2012940921999584, "operator": "ge"}, {"budget": 0.37, "threshold": 0.1829906680710611, "operator": "ge"}, {"budget": 0.38, "threshold": 0.16560254857306955, "operator": "ge"}, {"budget": 0.39, "threshold": 0.1515247780284148, "operator": "ge"}, {"budget": 0.4, "threshold": 0.13834927481233386, "operator": "ge"}, {"budget": 0.41, "threshold": 0.12275287420384182, "operator": "ge"}, {"budget": 0.42, "threshold": 0.10490696346777639, "operator": "ge"}, {"budget": 0.43, "threshold": 0.08763842651266754, "operator": "ge"}, {"budget": 0.44, "threshold": 0.054692801668179816, "operator": "ge"}, {"budget": 0.45, "threshold": -0.005500473325143033, "operator": "ge"}, {"budget": 0.46, "threshold": -0.06324579016912173, "operator": "ge"}, {"budget": 0.47, "threshold": -0.14988867203627607, "operator": "ge"}, {"budget": 0.48, "threshold": -0.18171910278060283, "operator": "ge"}, {"budget": 0.49, "threshold": -0.22007538246534017, "operator": "ge"}, {"budget": 0.5, "threshold": -0.2806726848590609, "operator": "ge"}, {"budget": 0.51, "threshold": -0.3348694328176104, "operator": "ge"}, {"budget": 0.52, "threshold": -0.34769260538298224, "operator": "ge"}, {"budget": 0.53, "threshold": -0.362266437826361, "operator": "ge"}, {"budget": 0.54, "threshold": -0.36977385855790057, "operator": "ge"}, {"budget": 0.55, "threshold": -0.3735517929363447, "operator": "ge"}, {"budget": 0.56, "threshold": -0.37786346772935814, "operator": "ge"}, {"budget": 0.57, "threshold": -0.38640487971037957, "operator": "ge"}, {"budget": 0.58, "threshold": -0.3965713816445217, "operator": "ge"}, {"budget": 0.59, "threshold": -0.4034811045489398, "operator": "ge"}, {"budget": 0.6, "threshold": -0.40807416289669945, "operator": "ge"}, {"budget": 0.61, "threshold": -0.4112391813574003, "operator": "ge"}, {"budget": 0.62, "threshold": -0.41604589490231053, "operator": "ge"}, {"budget": 0.63, "threshold": -0.41951185793419465, "operator": "ge"}, {"budget": 0.64, "threshold": -0.42196451749938163, "operator": "ge"}, {"budget": 0.65, "threshold": -0.427132629414411, "operator": "ge"}, {"budget": 0.66, "threshold": -0.430876394762095, "operator": "ge"}, {"budget": 0.67, "threshold": -0.43532071048733717, "operator": "ge"}, {"budget": 0.68, "threshold": -0.4412809590237933, "operator": "ge"}, {"budget": 0.69, "threshold": -0.4451567644771933, "operator": "ge"}, {"budget": 0.7, "threshold": -0.4479033528028918, "operator": "ge"}, {"budget": 0.71, "threshold": -0.45395031883945414, "operator": "ge"}, {"budget": 0.72, "threshold": -0.45792414868910497, "operator": "ge"}, {"budget": 0.73, "threshold": -0.46060334813177717, "operator": "ge"}, {"budget": 0.74, "threshold": -0.46340127852870977, "operator": "ge"}, {"budget": 0.75, "threshold": -0.4669426729997828, "operator": "ge"}, {"budget": 0.76, "threshold": -0.4709667069669403, "operator": "ge"}, {"budget": 0.77, "threshold": -0.4728791843388411, "operator": "ge"}, {"budget": 0.78, "threshold": -0.4765588275345471, "operator": "ge"}, {"budget": 0.79, "threshold": -0.48101175686389874, "operator": "ge"}, {"budget": 0.8, "threshold": -0.4840055647807233, "operator": "ge"}, {"budget": 0.81, "threshold": -0.4922501597835771, "operator": "ge"}, {"budget": 0.82, "threshold": -0.4957720613719859, "operator": "ge"}, {"budget": 0.83, "threshold": -0.5013823976370132, "operator": "ge"}, {"budget": 0.84, "threshold": -0.5035153804111238, "operator": "ge"}, {"budget": 0.85, "threshold": -0.5097103356623064, "operator": "ge"}, {"budget": 0.86, "threshold": -0.5122347626840483, "operator": "ge"}, {"budget": 0.87, "threshold": -0.5151036769797468, "operator": "ge"}, {"budget": 0.88, "threshold": -0.5206993147606985, "operator": "ge"}, {"budget": 0.89, "threshold": -0.5265442948747446, "operator": "ge"}, {"budget": 0.9, "threshold": -0.5303085838642374, "operator": "ge"}, {"budget": 0.91, "threshold": -0.5351673724509317, "operator": "ge"}, {"budget": 0.92, "threshold": -0.5382819591351758, "operator": "ge"}, {"budget": 0.93, "threshold": -0.5425771128676735, "operator": "ge"}, {"budget": 0.94, "threshold": -0.5453674237253808, "operator": "ge"}, {"budget": 0.95, "threshold": -0.5514905155227564, "operator": "ge"}, {"budget": 0.96, "threshold": -0.5551697142828949, "operator": "ge"}, {"budget": 0.97, "threshold": -0.5618611405361322, "operator": "ge"}, {"budget": 0.98, "threshold": -0.5682400916714024, "operator": "ge"}, {"budget": 0.99, "threshold": -0.5736318005496532, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.62253743104807, 18.91000328455166, 19.438570527957946, 20.440729771187385, 20.889104168829647, 21.286963841311668, 21.83812243268251, 22.34936540922593, 22.69621678589217, 23.036209168143202, 23.116434511201096, 23.299206665012154, 23.49692792502596, 24.061349933442955, 24.35643115198132, 24.41317407466131, 24.8062558356676, 24.82572726471634, 25.204813108038916, 25.37432459322527, 25.884632922936618, 26.14150251633152, 26.563205140677315, 26.61844946062752, 27.13263058527376, 27.44211593405554, 27.790572773231393, 28.085718803671543, 28.34246834246834, 28.65868541978355, 29.194137799771603, 28.52228852228852, 28.733074396867504, 29.306113922758236, 29.636110214934725, 29.93921537399798, 30.464813976168053, 30.859355166285855, 30.678668404474852, 30.789144717716145, 30.381914288842786, 30.446924209810806, 30.61069139194139, 31.003383190883195, 32.78813352826511, 32.69493177387914, 32.59974700344241, 32.23369804439414, 32.002077423572956, 31.905241072074897, 32.753392653351824, 32.645865771631385, 32.560295604094, 32.560295604094, 32.291124066083235, 32.20829930583912, 32.055157840262204, 32.055157840262204, 32.055157840262204, 31.98571339581776, 31.91796271831098, 31.91796271831098, 31.836895475166145, 31.69056795538529, 31.69056795538529, 31.59587098568832, 31.19994588744589, 30.795542872075053, 30.43198529411765, 30.310281440162274, 30.149337204955234, 30.149337204955234, 29.935076967369266, 29.935076967369266, 30.301921084213383, 30.240871023163322, 30.50090982488064, 29.867519062461156, 29.804604397624736, 29.606191699212037, 29.41681815282562, 29.099873304731936, 28.89744808483438, 28.532655681014386, 28.251719824890557, 28.091525650133274, 27.872911502094013, 27.75758365841715, 27.503011891905913, 27.193463397731694, 27.149222728491022, 26.95555769786947, 27.026719475569323, 26.76008688335533, 26.614406891916335, 26.574422885518896, 26.0540886890804, 25.87593033575991, 25.580499348307644, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 22, 34, 46, 58, 67, 78, 88, 96, 102, 106, 112, 117, 126, 133, 138, 143, 149, 155, 162, 170, 177, 184, 190, 196, 202, 208, 214, 219, 227, 234, 239, 245, 251, 256, 260, 264, 268, 273, 278, 282, 285, 289, 294, 298, 299, 299, 300, 301, 302, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 0, 1, 1, 2, 5, 5, 5, 6, 7, 7, 11, 13, 14, 14, 15, 17, 17, 21, 22, 26, 28, 29, 29, 32, 33, 34, 36, 40, 42, 45, 46, 48, 51, 56, 60, 62, 66, 67, 71, 74, 79, 80, 82, 84, 84, 85, 89, 91, 93, 95, 97, 98, 99, 99, 102, 103, 105, 105, 109, 110, 111, 111, 112, 114, 114, 115, 118, 121, 124, 126, 128, 128, 131, 131, 133, 134, 137, 144, 145, 146, 149, 152, 155, 159, 162, 165, 169, 171, 174, 178, 183, 187, 195, 199, 202, 203, 209, 211, 217, 220], "exact_score": [18.254072168965788, 18.62253743104807, 18.91000328455166, 19.438570527957946, 20.440729771187385, 20.889104168829647, 21.286963841311668, 21.83812243268251, 22.34936540922593, 22.69621678589217, 23.036209168143202, 23.116434511201096, 23.299206665012154, 23.49692792502596, 24.061349933442955, 24.35643115198132, 24.41317407466131, 24.8062558356676, 24.82572726471634, 25.204813108038916, 25.37432459322527, 25.884632922936618, 26.14150251633152, 26.563205140677315, 26.61844946062752, 27.13263058527376, 27.44211593405554, 27.790572773231393, 28.085718803671543, 28.34246834246834, 28.65868541978355, 29.194137799771603, 28.52228852228852, 28.733074396867504, 29.306113922758236, 29.636110214934725, 29.93921537399798, 30.464813976168053, 30.859355166285855, 30.678668404474852, 30.789144717716145, 30.381914288842786, 30.446924209810806, 30.61069139194139, 31.003383190883195, 32.78813352826511, 32.69493177387914, 32.59974700344241, 32.23369804439414, 32.002077423572956, 31.905241072074897, 32.753392653351824, 32.645865771631385, 32.560295604094, 32.560295604094, 32.291124066083235, 32.20829930583912, 32.055157840262204, 32.055157840262204, 32.055157840262204, 31.98571339581776, 31.91796271831098, 31.91796271831098, 31.836895475166145, 31.69056795538529, 31.69056795538529, 31.59587098568832, 31.19994588744589, 30.795542872075053, 30.43198529411765, 30.310281440162274, 30.149337204955234, 30.149337204955234, 29.935076967369266, 29.935076967369266, 30.301921084213383, 30.240871023163322, 30.50090982488064, 29.867519062461156, 29.804604397624736, 29.606191699212037, 29.41681815282562, 29.099873304731936, 28.89744808483438, 28.532655681014386, 28.251719824890557, 28.091525650133274, 27.872911502094013, 27.75758365841715, 27.503011891905913, 27.193463397731694, 27.149222728491022, 26.95555769786947, 27.026719475569323, 26.76008688335533, 26.614406891916335, 26.574422885518896, 26.0540886890804, 25.87593033575991, 25.580499348307644, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 28.196371825681894, "threshold_area100": 28.200202981453714, "threshold_area25": 23.269884091859847}, "test": {"score": [18.129624718318375, 18.174864321306096, 18.651239235857346, 18.92072399247975, 19.023607606509426, 19.183076537728283, 19.235684703838167, 19.653077520883564, 20.14707364642207, 19.843323122658546, 20.087276969842076, 20.556567126160598, 20.73369109144455, 20.950413866069205, 21.273914478807736, 21.626892638966865, 21.789998883090178, 21.939483827754284, 21.868171583736853, 22.248534638648128, 22.973116248226514, 23.262486830591772, 23.785151864642724, 24.017016998289918, 24.46744667651456, 24.519460038762496, 24.72507544510116, 25.31678745392563, 25.878414691243517, 26.1327275845646, 26.251772637397195, 26.375606619831654, 26.64453311313737, 27.221101206818467, 28.441874365569213, 28.589390006489296, 28.755006507515745, 29.022489963415243, 29.1498004960941, 29.295241300368563, 29.39157846091063, 29.757360825875317, 29.711114545776713, 30.192006332177378, 30.345154536045026, 29.948804804083817, 29.963537047178963, 29.90879291541927, 29.515273345350234, 29.809169556645614, 30.13830130155132, 29.980662564753423, 29.94138416569317, 29.83583613793485, 29.758400216420345, 29.65939237745007, 29.607100373572276, 29.755679029665604, 29.50868539103066, 29.4581185246822, 29.3675743813704, 29.33449485025288, 29.33666308270978, 29.286788536873647, 29.242191154247077, 29.500719411655616, 29.463336645111, 29.330847088343724, 29.371379604568904, 29.31958139097699, 29.222896245671816, 29.08416697467241, 28.983228166386393, 28.954244306418225, 29.14664518279021, 29.108438858875434, 29.41388020920207, 29.255490536513957, 29.125303411899583, 28.988145489899193, 29.23113496604807, 29.144942646410488, 29.114057500909947, 28.91677741073374, 29.411044376839858, 29.09974222368094, 28.922497851124707, 28.797251063464397, 28.92263774557945, 28.667476066134075, 28.504235765824816, 28.402039941665393, 28.223478245587174, 28.0550759821912, 27.950253721069224, 27.828223023830432, 27.75991795728638, 27.514358950110978, 27.297478644406333, 27.43278494897926, 27.001009067886244], "calls": [0.0, 0.36617842876165113, 0.9320905459387484, 3.129161118508655, 4.027962716378163, 4.960053262316911, 5.492676431424767, 6.790945406125166, 8.222370173102531, 8.954727030625833, 10.086551264980026, 10.918774966711052, 11.617842876165113, 12.383488681757656, 13.482023968042608, 14.480692410119842, 14.980026631158456, 15.712383488681759, 16.145139813581892, 17.07723035952064, 18.908122503328894, 20.139813581890813, 22.33688415446072, 23.06924101198402, 25.133155792276963, 25.432756324900136, 26.165113182423433, 27.463382157123835, 28.362183754993342, 29.161118508655125, 29.727030625832224, 30.525965379494007, 31.957390146471372, 33.48868175765646, 35.61917443408788, 36.18508655126498, 36.8508655126498, 37.949400798934754, 38.948069241011986, 39.71371504660453, 40.47936085219707, 41.211717709720375, 41.777629826897474, 42.47669773635153, 43.275632490013315, 44.6404793608522, 45.27296937416778, 46.07190412782956, 46.60452729693741, 47.470039946737685, 49.4340878828229, 50.632490013315575, 51.364846870838875, 53.22902796271638, 54.52729693741678, 55.45938748335553, 56.29161118508655, 58.1225033288948, 60.11984021304927, 61.085219707057256, 61.884154460719046, 62.250332889480696, 63.08255659121171, 63.81491344873502, 64.2809587217044, 65.44607190412783, 66.11185086551265, 67.14380825565912, 68.87483355525966, 70.23968042609854, 71.03861517976033, 72.56990679094541, 73.43541944074568, 74.06790945406125, 74.73368841544608, 75.76564580559254, 76.93075898801598, 77.52996005326231, 78.22902796271637, 79.22769640479362, 80.22636484687084, 81.6577896138482, 82.49001331557922, 84.0878828229028, 84.68708388814915, 86.25166444740347, 87.11717709720372, 87.78295605858855, 89.11451398135819, 90.37949400798935, 91.27829560585884, 92.41011984021304, 93.27563249001332, 94.0745672436751, 94.57390146471371, 95.60585885486019, 95.90545938748336, 97.03728362183755, 97.96937416777631, 98.46870838881492, 100.0], "rescued": [0, 20, 54, 154, 198, 235, 256, 306, 364, 390, 427, 452, 472, 495, 528, 553, 569, 589, 596, 624, 671, 701, 752, 769, 820, 826, 844, 874, 896, 916, 932, 952, 990, 1031, 1083, 1091, 1106, 1127, 1144, 1155, 1166, 1180, 1188, 1197, 1212, 1230, 1239, 1243, 1245, 1248, 1252, 1254, 1255, 1256, 1256, 1256, 1256, 1257, 1257, 1257, 1257, 1257, 1258, 1258, 1258, 1259, 1259, 1259, 1262, 1263, 1263, 1264, 1264, 1264, 1265, 1267, 1269, 1269, 1269, 1270, 1272, 1273, 1274, 1274, 1276, 1277, 1278, 1278, 1279, 1279, 1279, 1279, 1279, 1279, 1280, 1281, 1281, 1281, 1281, 1282, 1282], "harmed": [0, 3, 3, 17, 24, 30, 32, 42, 47, 51, 61, 67, 69, 71, 75, 81, 84, 90, 97, 103, 110, 118, 132, 134, 149, 150, 152, 158, 163, 173, 179, 188, 211, 223, 242, 248, 252, 265, 274, 277, 283, 286, 293, 298, 301, 313, 318, 323, 328, 332, 340, 347, 350, 354, 357, 362, 365, 374, 384, 387, 392, 394, 396, 399, 402, 406, 408, 414, 428, 439, 445, 461, 466, 468, 471, 477, 490, 496, 503, 516, 523, 532, 536, 551, 555, 576, 586, 594, 606, 619, 630, 641, 653, 665, 674, 691, 696, 714, 732, 740, 777], "exact_score": [18.129624718318375, 18.665411518775144, 18.59579082253898, 18.900318761517518, 19.01032033571485, 19.19989686583788, 19.36301364192644, 19.724527096071718, 20.10259744166108, 19.85249594950795, 20.067087634782023, 20.575473279894663, 20.865744535268895, 21.10949803354204, 21.60383783917196, 21.789998883090178, 21.87780101804121, 22.2152897606572, 22.700140697663983, 22.996454037667966, 23.249741123821448, 23.54995763565165, 23.739710054963187, 23.966816195077065, 24.16425498516009, 24.402036546080847, 24.699924929950313, 24.96115216795653, 25.78562114260746, 26.07208907004519, 26.341341589949042, 26.458394932962857, 26.66644985762712, 27.0332544261637, 27.74625377366088, 28.161066940583428, 28.553224569374873, 28.76576817105451, 29.03553230755923, 29.1498004960941, 29.34894508400961, 29.73042601336344, 29.76152553763441, 30.4000114741764, 30.438635204830522, 29.98263974088433, 29.946261378134263, 29.829935277205234, 29.939665603505496, 29.867033044170487, 30.055066017443654, 29.980662564753423, 29.88125764956276, 29.83583613793485, 29.758400216420345, 29.738917650451803, 29.607100373572276, 29.89836921466853, 29.77490543298021, 29.73649834906994, 29.50868539103066, 29.4581185246822, 29.35546214127888, 29.35564259878794, 29.286788536873647, 29.500719411655616, 29.482006115858955, 29.385588023850808, 29.406940553731197, 29.353072986827172, 29.34972135892235, 29.222896245671816, 29.166252659898067, 29.00598262246738, 28.954244306418225, 29.088285643437565, 29.07150889207094, 29.402852044967258, 29.15357837545279, 28.988145489899193, 29.23113496604807, 29.284320829053268, 29.09728717230996, 29.067016608611212, 28.91677741073374, 29.307383388405846, 29.138480473218237, 28.96693073312259, 29.05226417551251, 28.93939136291962, 28.777550342392388, 28.54069473212631, 28.45484691868344, 28.257258286005783, 28.084774247812494, 27.89317556344693, 27.741903185173243, 27.548984688314405, 27.297478644406333, 27.282172279006613, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.882676939648427, "threshold_area100": 26.835152097463393, "threshold_area25": 21.038980954660836}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.948804804083817, "calls": 44.6404793608522, "rescued": 1230, "harmed": 313}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.3847352862358093, "operator": "ge"}, {"budget": 0.02, "threshold": 0.35956937074661255, "operator": "ge"}, {"budget": 0.03, "threshold": 0.3397795855998993, "operator": "ge"}, {"budget": 0.04, "threshold": 0.31935927271842957, "operator": "ge"}, {"budget": 0.05, "threshold": 0.3111794590950012, "operator": "ge"}, {"budget": 0.06, "threshold": 0.29503321647644043, "operator": "ge"}, {"budget": 0.07, "threshold": 0.27543625235557556, "operator": "ge"}, {"budget": 0.08, "threshold": 0.25538384914398193, "operator": "ge"}, {"budget": 0.09, "threshold": 0.2242567092180252, "operator": "ge"}, {"budget": 0.1, "threshold": 0.18833227455615997, "operator": "ge"}, {"budget": 0.11, "threshold": 0.16788458824157715, "operator": "ge"}, {"budget": 0.12, "threshold": 0.16250082850456238, "operator": "ge"}, {"budget": 0.13, "threshold": 0.1568671464920044, "operator": "ge"}, {"budget": 0.14, "threshold": 0.15418918430805206, "operator": "ge"}, {"budget": 0.15, "threshold": 0.14978669583797455, "operator": "ge"}, {"budget": 0.16, "threshold": 0.1476365327835083, "operator": "ge"}, {"budget": 0.17, "threshold": 0.14413140714168549, "operator": "ge"}, {"budget": 0.18, "threshold": 0.14194244146347046, "operator": "ge"}, {"budget": 0.19, "threshold": 0.13894174993038177, "operator": "ge"}, {"budget": 0.2, "threshold": 0.1369280219078064, "operator": "ge"}, {"budget": 0.21, "threshold": 0.13525405526161194, "operator": "ge"}, {"budget": 0.22, "threshold": 0.1317416876554489, "operator": "ge"}, {"budget": 0.23, "threshold": 0.13028597831726074, "operator": "ge"}, {"budget": 0.24, "threshold": 0.12684257328510284, "operator": "ge"}, {"budget": 0.25, "threshold": 0.1250026822090149, "operator": "ge"}, {"budget": 0.26, "threshold": 0.1227598488330841, "operator": "ge"}, {"budget": 0.27, "threshold": 0.11962556838989258, "operator": "ge"}, {"budget": 0.28, "threshold": 0.11658573150634766, "operator": "ge"}, {"budget": 0.29, "threshold": 0.11410702019929886, "operator": "ge"}, {"budget": 0.3, "threshold": 0.11063309013843536, "operator": "ge"}, {"budget": 0.31, "threshold": 0.10474616289138794, "operator": "ge"}, {"budget": 0.32, "threshold": 0.10196512192487717, "operator": "ge"}, {"budget": 0.33, "threshold": 0.09941009432077408, "operator": "ge"}, {"budget": 0.34, "threshold": 0.09433229267597198, "operator": "ge"}, {"budget": 0.35, "threshold": 0.08724735677242279, "operator": "ge"}, {"budget": 0.36, "threshold": 0.07987795025110245, "operator": "ge"}, {"budget": 0.37, "threshold": 0.0752011314034462, "operator": "ge"}, {"budget": 0.38, "threshold": 0.06815867125988007, "operator": "ge"}, {"budget": 0.39, "threshold": 0.0656191036105156, "operator": "ge"}, {"budget": 0.4, "threshold": 0.05738770216703415, "operator": "ge"}, {"budget": 0.41, "threshold": 0.05295491963624954, "operator": "ge"}, {"budget": 0.42, "threshold": 0.04488062113523483, "operator": "ge"}, {"budget": 0.43, "threshold": 0.03472083806991577, "operator": "ge"}, {"budget": 0.44, "threshold": 0.021890893578529358, "operator": "ge"}, {"budget": 0.45, "threshold": -0.0026861727237701416, "operator": "ge"}, {"budget": 0.46, "threshold": -0.013196769170463085, "operator": "ge"}, {"budget": 0.47, "threshold": -0.014699450694024563, "operator": "ge"}, {"budget": 0.48, "threshold": -0.017525194212794304, "operator": "ge"}, {"budget": 0.49, "threshold": -0.018698206171393394, "operator": "ge"}, {"budget": 0.5, "threshold": -0.019759660586714745, "operator": "ge"}, {"budget": 0.51, "threshold": -0.021415360271930695, "operator": "ge"}, {"budget": 0.52, "threshold": -0.023081399500370026, "operator": "ge"}, {"budget": 0.53, "threshold": -0.025069743394851685, "operator": "ge"}, {"budget": 0.54, "threshold": -0.027316084131598473, "operator": "ge"}, {"budget": 0.55, "threshold": -0.028865255415439606, "operator": "ge"}, {"budget": 0.56, "threshold": -0.03097427450120449, "operator": "ge"}, {"budget": 0.57, "threshold": -0.031704649329185486, "operator": "ge"}, {"budget": 0.58, "threshold": -0.034311819821596146, "operator": "ge"}, {"budget": 0.59, "threshold": -0.03571946173906326, "operator": "ge"}, {"budget": 0.6, "threshold": -0.036981772631406784, "operator": "ge"}, {"budget": 0.61, "threshold": -0.03829123079776764, "operator": "ge"}, {"budget": 0.62, "threshold": -0.040202219039201736, "operator": "ge"}, {"budget": 0.63, "threshold": -0.04239821061491966, "operator": "ge"}, {"budget": 0.64, "threshold": -0.04294104874134064, "operator": "ge"}, {"budget": 0.65, "threshold": -0.043995972722768784, "operator": "ge"}, {"budget": 0.66, "threshold": -0.0453513041138649, "operator": "ge"}, {"budget": 0.67, "threshold": -0.04667278006672859, "operator": "ge"}, {"budget": 0.68, "threshold": -0.04786902293562889, "operator": "ge"}, {"budget": 0.69, "threshold": -0.04868284612894058, "operator": "ge"}, {"budget": 0.7, "threshold": -0.04955533891916275, "operator": "ge"}, {"budget": 0.71, "threshold": -0.050303176045417786, "operator": "ge"}, {"budget": 0.72, "threshold": -0.0526600144803524, "operator": "ge"}, {"budget": 0.73, "threshold": -0.05390433594584465, "operator": "ge"}, {"budget": 0.74, "threshold": -0.05706202611327171, "operator": "ge"}, {"budget": 0.75, "threshold": -0.05816053971648216, "operator": "ge"}, {"budget": 0.76, "threshold": -0.05999547615647316, "operator": "ge"}, {"budget": 0.77, "threshold": -0.06121275573968887, "operator": "ge"}, {"budget": 0.78, "threshold": -0.06358970701694489, "operator": "ge"}, {"budget": 0.79, "threshold": -0.06445995718240738, "operator": "ge"}, {"budget": 0.8, "threshold": -0.06627664715051651, "operator": "ge"}, {"budget": 0.81, "threshold": -0.06792491674423218, "operator": "ge"}, {"budget": 0.82, "threshold": -0.06969699263572693, "operator": "ge"}, {"budget": 0.83, "threshold": -0.07296676188707352, "operator": "ge"}, {"budget": 0.84, "threshold": -0.07427690178155899, "operator": "ge"}, {"budget": 0.85, "threshold": -0.07578686624765396, "operator": "ge"}, {"budget": 0.86, "threshold": -0.08023705333471298, "operator": "ge"}, {"budget": 0.87, "threshold": -0.08429063856601715, "operator": "ge"}, {"budget": 0.88, "threshold": -0.0874733179807663, "operator": "ge"}, {"budget": 0.89, "threshold": -0.09036442637443542, "operator": "ge"}, {"budget": 0.9, "threshold": -0.0930374413728714, "operator": "ge"}, {"budget": 0.91, "threshold": -0.095845066010952, "operator": "ge"}, {"budget": 0.92, "threshold": -0.10035011172294617, "operator": "ge"}, {"budget": 0.93, "threshold": -0.10261721163988113, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10456136614084244, "operator": "ge"}, {"budget": 0.95, "threshold": -0.10716451704502106, "operator": "ge"}, {"budget": 0.96, "threshold": -0.10901672393083572, "operator": "ge"}, {"budget": 0.97, "threshold": -0.11363641917705536, "operator": "ge"}, {"budget": 0.98, "threshold": -0.11837372183799744, "operator": "ge"}, {"budget": 0.99, "threshold": -0.12238980829715729, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.611583368752658, 18.954111297487483, 19.474969474969473, 20.594020456333595, 21.023619336668354, 21.47544859409266, 22.159198433344667, 22.94175118906366, 23.596595245453912, 24.205113979977906, 24.076081485080476, 24.17260767101122, 24.435604449482955, 24.661286526250986, 24.87263962540475, 25.14640460081357, 25.48187090928325, 25.750548620488743, 26.139245530930644, 26.196996705018094, 26.666666666666668, 26.833325993873814, 27.05050871848669, 27.358808761247783, 27.77766072013859, 27.900132275132272, 27.876798790222928, 26.494246494246493, 26.87700609387356, 27.14834931140159, 27.414399597127233, 28.118717209339582, 29.020602389020446, 29.511018334350815, 29.641660651155906, 30.088084224438305, 30.34987144743242, 30.460192369545645, 30.263466091893342, 30.06985564456829, 30.33643182283826, 30.620009526938023, 30.415733273596175, 31.003383190883195, 32.78813352826511, 32.69493177387914, 32.61088658236633, 32.61088658236633, 32.527716861598435, 32.31313387094679, 32.23167454719272, 32.23167454719272, 32.31313387094679, 32.25332525850661, 32.25332525850661, 32.212480407523515, 32.07189749087194, 31.533560982558882, 31.354102865115017, 30.903948400779036, 30.903948400779036, 30.98386022221745, 30.903948400779036, 30.903948400779036, 30.76948924731183, 30.590422864783555, 30.512969737279683, 30.26095360824742, 30.035585556418887, 29.722966627728532, 29.425044043835726, 29.083849438546384, 28.876750700280112, 28.876750700280112, 28.876750700280112, 29.336601307189543, 28.946196766577625, 28.75022080904434, 28.625930943004114, 28.394370957439552, 28.104413768366165, 28.002546196209966, 27.859463053971062, 27.37949089370726, 27.599549069918194, 27.486912240624406, 27.7126550918376, 27.549732924865484, 27.549732924865484, 27.193463397731694, 27.149222728491022, 27.105765079944877, 26.890622632934402, 26.849393247627056, 26.69637410503109, 26.551292355642087, 26.292894422825547, 25.87593033575991, 25.65261473292303, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 22, 34, 47, 57, 70, 80, 91, 101, 108, 114, 121, 127, 134, 139, 145, 151, 157, 163, 166, 173, 180, 186, 192, 198, 205, 209, 214, 220, 225, 232, 239, 246, 251, 256, 261, 265, 270, 272, 276, 281, 286, 289, 294, 298, 299, 299, 299, 299, 299, 299, 299, 299, 299, 299, 300, 300, 300, 300, 300, 300, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 302, 302, 303, 303, 303, 303, 303, 303, 303, 305, 305, 306, 306, 306, 306, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 1, 2, 3, 5, 5, 6, 7, 7, 10, 12, 12, 14, 15, 16, 17, 22, 22, 23, 24, 27, 29, 32, 32, 37, 41, 43, 46, 47, 50, 53, 56, 59, 61, 62, 64, 67, 74, 78, 79, 81, 84, 84, 84, 85, 86, 86, 87, 89, 90, 90, 93, 98, 98, 99, 101, 106, 107, 109, 109, 109, 110, 110, 112, 114, 115, 116, 119, 121, 123, 128, 131, 131, 131, 132, 136, 143, 145, 147, 150, 152, 154, 159, 164, 166, 170, 173, 173, 178, 187, 188, 192, 193, 200, 203, 205, 211, 215, 220], "exact_score": [18.254072168965788, 18.611583368752658, 18.954111297487483, 19.474969474969473, 20.594020456333595, 21.023619336668354, 21.47544859409266, 22.159198433344667, 22.94175118906366, 23.596595245453912, 24.205113979977906, 24.076081485080476, 24.17260767101122, 24.435604449482955, 24.661286526250986, 24.87263962540475, 25.14640460081357, 25.48187090928325, 25.750548620488743, 26.139245530930644, 26.196996705018094, 26.666666666666668, 26.833325993873814, 27.05050871848669, 27.358808761247783, 27.77766072013859, 27.900132275132272, 27.876798790222928, 26.494246494246493, 26.87700609387356, 27.14834931140159, 27.414399597127233, 28.118717209339582, 29.020602389020446, 29.511018334350815, 29.641660651155906, 30.088084224438305, 30.34987144743242, 30.460192369545645, 30.263466091893342, 30.06985564456829, 30.33643182283826, 30.620009526938023, 30.415733273596175, 31.003383190883195, 32.78813352826511, 32.69493177387914, 32.61088658236633, 32.61088658236633, 32.527716861598435, 32.31313387094679, 32.23167454719272, 32.23167454719272, 32.31313387094679, 32.25332525850661, 32.25332525850661, 32.212480407523515, 32.07189749087194, 31.533560982558882, 31.354102865115017, 30.903948400779036, 30.903948400779036, 30.98386022221745, 30.903948400779036, 30.903948400779036, 30.76948924731183, 30.590422864783555, 30.512969737279683, 30.26095360824742, 30.035585556418887, 29.722966627728532, 29.425044043835726, 29.083849438546384, 28.876750700280112, 28.876750700280112, 28.876750700280112, 29.336601307189543, 28.946196766577625, 28.75022080904434, 28.625930943004114, 28.394370957439552, 28.104413768366165, 28.002546196209966, 27.859463053971062, 27.37949089370726, 27.599549069918194, 27.486912240624406, 27.7126550918376, 27.549732924865484, 27.549732924865484, 27.193463397731694, 27.149222728491022, 27.105765079944877, 26.890622632934402, 26.849393247627056, 26.69637410503109, 26.551292355642087, 26.292894422825547, 25.87593033575991, 25.65261473292303, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.99955569870053, "threshold_area100": 28.003054177587643, "threshold_area25": 23.862847218324262}, "test": {"score": [18.129624718318375, 18.25131797590403, 18.49704721969934, 18.82865000096959, 19.361429449720198, 19.172704834485355, 19.15432834405624, 19.657997629408747, 19.954894002748066, 20.103300000931622, 20.34025872380036, 20.668953946250337, 20.87117407214579, 21.463920107284114, 21.62010094271615, 22.073116887669947, 22.209730968631924, 22.45533737688331, 23.037448479947873, 23.543251590080803, 23.828193683641036, 23.917678709914576, 24.28735045686684, 24.43080746983435, 24.905445067581468, 25.030945763486688, 25.304815753869402, 25.993469905584167, 26.26094391141514, 26.442910052069667, 26.82361254675236, 27.581649217074737, 27.739033920998736, 27.937342428312245, 28.338847727893963, 28.81058168325769, 29.080395640617628, 29.181202645306943, 29.19388069792172, 29.34078130663284, 29.559518868392026, 29.569411036805104, 29.683525580892045, 30.233468800270934, 30.438630999213217, 29.948804804083817, 29.933036566281157, 29.991722012290545, 29.940772624609345, 29.81853016932721, 29.735598841694973, 29.69748088169141, 29.641828975013922, 29.564465498000963, 29.492563198071004, 29.4141706471877, 29.34752450439516, 29.58900291637747, 29.420236626118978, 29.34948384948385, 29.270926459765533, 29.176757343949568, 29.384992836164614, 29.21837005705387, 29.21837005705387, 29.13624963628904, 29.41588750562693, 28.922455134620666, 29.008241674999248, 28.889148977904348, 29.184774527697428, 29.156177752005853, 29.115361113445303, 29.006050838537494, 29.219889028320544, 29.276354784404663, 29.254305477131563, 29.189953714282137, 29.10712395387262, 29.324313641199012, 29.46127551224989, 29.600078004810225, 29.43324480075251, 29.262667413358614, 29.153248471113535, 29.05966464335234, 29.094401779921743, 28.74176088660832, 28.607821846943356, 28.668132118862832, 28.43606597884078, 28.299827756559147, 28.05060676028418, 27.938037674561993, 27.872514740192933, 27.831911645626583, 27.864726987117876, 27.56699946042754, 27.38582375744677, 27.431381477156503, 27.001009067886244], "calls": [0.0, 0.9320905459387484, 1.5312916111850865, 2.62982689747004, 4.194407456724368, 4.760319573901465, 5.625832223701731, 7.123834886817576, 8.288948069241012, 9.387483355525966, 10.319573901464715, 11.55126498002663, 12.416777629826898, 13.448735019973368, 14.380825565912117, 16.145139813581892, 16.577896138482025, 17.80958721704394, 18.908122503328894, 19.940079893475364, 20.572569906790946, 21.171770972037287, 22.536617842876165, 23.06924101198402, 24.201065246338217, 24.73368841544607, 25.96537949400799, 27.330226364846872, 28.462050599201067, 29.360852197070574, 30.892143808255657, 32.689747003994675, 33.48868175765646, 34.187749667110516, 35.352862849533956, 36.684420772303596, 37.64980026631159, 38.41544607190413, 39.4474034620506, 39.78029294274301, 40.71238348868176, 41.14513981358189, 41.81091877496671, 42.57656458055926, 43.30892143808256, 44.6404793608522, 45.23968042609853, 45.83888149134487, 47.90279627163782, 48.8681757656458, 49.766977363515316, 50.99866844207723, 52.16378162450066, 53.39547270306259, 54.260985352862846, 54.96005326231691, 55.958721704394144, 56.657789613848195, 58.022636484687084, 59.05459387483355, 59.720372836218374, 60.61917443408789, 62.28362183754993, 63.61517976031957, 64.01464713715048, 64.84687083888149, 65.74567243675101, 66.51131824234355, 67.84287616511318, 68.40878828229027, 69.04127829560586, 69.60719041278296, 71.43808255659121, 72.4034620505992, 74.46737683089214, 75.36617842876166, 76.23169107856191, 77.13049267643143, 78.46205059920106, 79.16111850865512, 80.09320905459387, 81.05858854860186, 81.75765645805593, 83.18908122503329, 83.58854860186418, 84.25432756324899, 86.48468708388815, 88.24900133155792, 89.41411451398136, 90.41278295605859, 91.4780292942743, 92.34354194407457, 93.9081225033289, 94.57390146471371, 95.10652463382158, 95.47270306258322, 95.87217043941412, 96.97070572569906, 97.83621837549934, 98.56857523302264, 100.0], "rescued": [0, 65, 97, 143, 212, 235, 270, 335, 387, 424, 453, 495, 516, 543, 569, 618, 630, 660, 687, 712, 728, 742, 779, 789, 815, 827, 856, 887, 913, 933, 968, 1014, 1032, 1049, 1075, 1103, 1121, 1134, 1150, 1157, 1169, 1177, 1186, 1198, 1215, 1230, 1235, 1238, 1239, 1239, 1239, 1240, 1240, 1241, 1242, 1244, 1244, 1245, 1245, 1245, 1245, 1245, 1248, 1248, 1248, 1249, 1250, 1250, 1251, 1251, 1253, 1254, 1258, 1258, 1263, 1264, 1266, 1267, 1269, 1270, 1271, 1273, 1273, 1274, 1274, 1274, 1276, 1277, 1278, 1279, 1279, 1279, 1280, 1280, 1280, 1280, 1281, 1281, 1281, 1282, 1282], "harmed": [0, 3, 5, 14, 21, 25, 33, 41, 47, 52, 59, 66, 69, 76, 85, 95, 96, 105, 113, 120, 124, 129, 136, 137, 146, 148, 156, 162, 171, 180, 190, 219, 228, 235, 245, 256, 261, 268, 279, 279, 285, 290, 293, 296, 301, 313, 317, 318, 320, 324, 329, 332, 335, 338, 343, 348, 352, 357, 364, 367, 372, 377, 385, 393, 393, 398, 401, 409, 411, 417, 424, 430, 443, 450, 461, 472, 479, 483, 493, 497, 506, 519, 528, 543, 549, 554, 569, 596, 609, 620, 635, 645, 663, 673, 680, 683, 689, 711, 724, 739, 777], "exact_score": [18.129624718318375, 18.261149896645325, 18.868964562926827, 18.954036640083157, 19.295704522875173, 19.15068912175876, 19.223895933082858, 19.591711765635527, 19.847692923720942, 19.827712401840948, 20.28752679764326, 20.532001229327, 20.79233526185913, 21.014764107542845, 21.576612360962542, 21.735154780185237, 22.020640491552918, 22.32411110731283, 22.48651451808818, 23.050411651515546, 23.545091575782113, 23.890263015521676, 24.122000551246725, 24.407379753130837, 24.894762342537682, 25.10738571756465, 25.277801348943267, 25.878800170302863, 26.16215121970663, 26.33887267049409, 26.680652733605935, 26.925474975583352, 27.450584631101716, 27.620793169603168, 27.885516521767183, 28.051168697276225, 28.549291225632594, 28.90881109587344, 29.137048498202766, 29.18234326852658, 29.353823650776828, 29.649370143823205, 29.70348609653347, 30.394640087461074, 30.380360146555464, 29.938585575289068, 29.991722012290545, 29.967497206088993, 29.940772624609345, 29.81853016932721, 29.735598841694973, 29.69748088169141, 29.641828975013922, 29.577565280918844, 29.53164896340751, 29.4141706471877, 29.34752450439516, 29.57007904907682, 29.420236626118978, 29.34948384948385, 29.258711078157013, 29.076655709990273, 29.34565436474521, 29.31756603580331, 29.21837005705387, 29.118080074781126, 29.059967963024242, 29.040488143278676, 28.957004715096303, 29.214825849516004, 29.143907179422186, 29.091906883427953, 29.036269571469987, 28.980332163923972, 29.175092297596183, 29.072071576286014, 29.199320912503072, 29.189953714282137, 29.074142480981724, 29.335341805433828, 29.17154041291214, 29.600078004810225, 29.358254910559538, 29.300482873911115, 29.08705677537831, 28.935230674151676, 29.177470465613204, 28.988744976346187, 28.822462436761093, 28.632240800438417, 28.794378108533174, 28.472984998728705, 28.33583521524702, 28.121003818404418, 28.05060676028418, 27.88878983580912, 27.848639471030364, 27.56699946042754, 27.321135061706208, 27.3158385141275, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.89658774742889, "threshold_area100": 26.958658416661812, "threshold_area25": 21.398911874099845}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.948804804083817, "calls": 44.6404793608522, "rescued": 1230, "harmed": 313}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.5324788093566895, "operator": "ge"}, {"budget": 0.02, "threshold": 2.3129172325134277, "operator": "ge"}, {"budget": 0.03, "threshold": 2.2002220153808594, "operator": "ge"}, {"budget": 0.04, "threshold": 2.1509270668029785, "operator": "ge"}, {"budget": 0.05, "threshold": 2.0702459812164307, "operator": "ge"}, {"budget": 0.06, "threshold": 2.0129785537719727, "operator": "ge"}, {"budget": 0.07, "threshold": 1.8921788930892944, "operator": "ge"}, {"budget": 0.08, "threshold": 1.8088432550430298, "operator": "ge"}, {"budget": 0.09, "threshold": 1.7563095092773438, "operator": "ge"}, {"budget": 0.1, "threshold": 1.7019627094268799, "operator": "ge"}, {"budget": 0.11, "threshold": 1.666378378868103, "operator": "ge"}, {"budget": 0.12, "threshold": 1.6291197538375854, "operator": "ge"}, {"budget": 0.13, "threshold": 1.5794410705566406, "operator": "ge"}, {"budget": 0.14, "threshold": 1.5545718669891357, "operator": "ge"}, {"budget": 0.15, "threshold": 1.5100868940353394, "operator": "ge"}, {"budget": 0.16, "threshold": 1.4716792106628418, "operator": "ge"}, {"budget": 0.17, "threshold": 1.4499640464782715, "operator": "ge"}, {"budget": 0.18, "threshold": 1.4155917167663574, "operator": "ge"}, {"budget": 0.19, "threshold": 1.3877416849136353, "operator": "ge"}, {"budget": 0.2, "threshold": 1.3529455661773682, "operator": "ge"}, {"budget": 0.21, "threshold": 1.2966687679290771, "operator": "ge"}, {"budget": 0.22, "threshold": 1.2577012777328491, "operator": "ge"}, {"budget": 0.23, "threshold": 1.2331860065460205, "operator": "ge"}, {"budget": 0.24, "threshold": 1.198682188987732, "operator": "ge"}, {"budget": 0.25, "threshold": 1.1577253341674805, "operator": "ge"}, {"budget": 0.26, "threshold": 1.1195135116577148, "operator": "ge"}, {"budget": 0.27, "threshold": 1.0863111019134521, "operator": "ge"}, {"budget": 0.28, "threshold": 1.0598088502883911, "operator": "ge"}, {"budget": 0.29, "threshold": 1.0341686010360718, "operator": "ge"}, {"budget": 0.3, "threshold": 1.0037251710891724, "operator": "ge"}, {"budget": 0.31, "threshold": 0.9761720895767212, "operator": "ge"}, {"budget": 0.32, "threshold": 0.9525953531265259, "operator": "ge"}, {"budget": 0.33, "threshold": 0.8904132843017578, "operator": "ge"}, {"budget": 0.34, "threshold": 0.8266855478286743, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7782485485076904, "operator": "ge"}, {"budget": 0.36, "threshold": 0.7461093664169312, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7268557548522949, "operator": "ge"}, {"budget": 0.38, "threshold": 0.6585972309112549, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6235252618789673, "operator": "ge"}, {"budget": 0.4, "threshold": 0.5731513500213623, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5078642964363098, "operator": "ge"}, {"budget": 0.42, "threshold": 0.4208953380584717, "operator": "ge"}, {"budget": 0.43, "threshold": 0.384257435798645, "operator": "ge"}, {"budget": 0.44, "threshold": 0.21609441936016083, "operator": "ge"}, {"budget": 0.45, "threshold": -0.03209555149078369, "operator": "ge"}, {"budget": 0.46, "threshold": -0.38215887546539307, "operator": "ge"}, {"budget": 0.47, "threshold": -0.724716067314148, "operator": "ge"}, {"budget": 0.48, "threshold": -1.1284236907958984, "operator": "ge"}, {"budget": 0.49, "threshold": -1.3658761978149414, "operator": "ge"}, {"budget": 0.5, "threshold": -1.5778675079345703, "operator": "ge"}, {"budget": 0.51, "threshold": -1.8367855548858643, "operator": "ge"}, {"budget": 0.52, "threshold": -2.83270001411438, "operator": "ge"}, {"budget": 0.53, "threshold": -3.060351848602295, "operator": "ge"}, {"budget": 0.54, "threshold": -3.187187433242798, "operator": "ge"}, {"budget": 0.55, "threshold": -3.2624428272247314, "operator": "ge"}, {"budget": 0.56, "threshold": -3.3315939903259277, "operator": "ge"}, {"budget": 0.57, "threshold": -3.3762927055358887, "operator": "ge"}, {"budget": 0.58, "threshold": -3.426316499710083, "operator": "ge"}, {"budget": 0.59, "threshold": -3.4653127193450928, "operator": "ge"}, {"budget": 0.6, "threshold": -3.498476028442383, "operator": "ge"}, {"budget": 0.61, "threshold": -3.5123679637908936, "operator": "ge"}, {"budget": 0.62, "threshold": -3.5435774326324463, "operator": "ge"}, {"budget": 0.63, "threshold": -3.5735397338867188, "operator": "ge"}, {"budget": 0.64, "threshold": -3.616664409637451, "operator": "ge"}, {"budget": 0.65, "threshold": -3.6376760005950928, "operator": "ge"}, {"budget": 0.66, "threshold": -3.687333822250366, "operator": "ge"}, {"budget": 0.67, "threshold": -3.73134708404541, "operator": "ge"}, {"budget": 0.68, "threshold": -3.7673535346984863, "operator": "ge"}, {"budget": 0.69, "threshold": -3.8012499809265137, "operator": "ge"}, {"budget": 0.7, "threshold": -3.8374569416046143, "operator": "ge"}, {"budget": 0.71, "threshold": -3.866091251373291, "operator": "ge"}, {"budget": 0.72, "threshold": -3.9096555709838867, "operator": "ge"}, {"budget": 0.73, "threshold": -3.9507293701171875, "operator": "ge"}, {"budget": 0.74, "threshold": -3.9730136394500732, "operator": "ge"}, {"budget": 0.75, "threshold": -4.029394149780273, "operator": "ge"}, {"budget": 0.76, "threshold": -4.065952777862549, "operator": "ge"}, {"budget": 0.77, "threshold": -4.103100299835205, "operator": "ge"}, {"budget": 0.78, "threshold": -4.220688343048096, "operator": "ge"}, {"budget": 0.79, "threshold": -4.270918846130371, "operator": "ge"}, {"budget": 0.8, "threshold": -4.353043556213379, "operator": "ge"}, {"budget": 0.81, "threshold": -4.576075553894043, "operator": "ge"}, {"budget": 0.82, "threshold": -12.578554153442383, "operator": "ge"}, {"budget": 0.83, "threshold": -12.675573348999023, "operator": "ge"}, {"budget": 0.84, "threshold": -12.769739151000977, "operator": "ge"}, {"budget": 0.85, "threshold": -12.799469947814941, "operator": "ge"}, {"budget": 0.86, "threshold": -12.826101303100586, "operator": "ge"}, {"budget": 0.87, "threshold": -12.880035400390625, "operator": "ge"}, {"budget": 0.88, "threshold": -12.913247108459473, "operator": "ge"}, {"budget": 0.89, "threshold": -12.987922668457031, "operator": "ge"}, {"budget": 0.9, "threshold": -13.052190780639648, "operator": "ge"}, {"budget": 0.91, "threshold": -13.074577331542969, "operator": "ge"}, {"budget": 0.92, "threshold": -13.101492881774902, "operator": "ge"}, {"budget": 0.93, "threshold": -13.143568992614746, "operator": "ge"}, {"budget": 0.94, "threshold": -13.20492935180664, "operator": "ge"}, {"budget": 0.95, "threshold": -13.303669929504395, "operator": "ge"}, {"budget": 0.96, "threshold": -13.369001388549805, "operator": "ge"}, {"budget": 0.97, "threshold": -13.409912109375, "operator": "ge"}, {"budget": 0.98, "threshold": -13.532988548278809, "operator": "ge"}, {"budget": 0.99, "threshold": -13.628738403320312, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.606411530247485, 19.07848964452738, 19.533297064440198, 19.95842149961399, 20.51876974904625, 20.913484950184912, 21.931133035784196, 22.535544851536223, 23.000857859809585, 23.171199137847857, 23.316302528722066, 23.680931614510953, 23.906356801093644, 24.19776921585324, 24.602560759835207, 24.639161787459766, 24.97095032613337, 25.172488759950063, 25.520534769552487, 25.67831507267427, 26.11151794120724, 26.518149970184858, 26.797564250095895, 25.65178646081287, 25.820753433972527, 26.107814247349133, 26.33526191894268, 26.6510528722187, 27.157040917230134, 27.293031566408576, 27.57273974709183, 28.093720996644535, 27.87883850017877, 29.146825396825392, 29.345838444414156, 29.816360532752174, 30.21772345301757, 30.731882387962244, 30.642857142857142, 30.47444117137411, 30.85165679174896, 30.7432945061811, 30.58665859183994, 31.003383190883195, 32.78813352826511, 32.69493177387914, 32.685080943594876, 32.04915866630637, 32.002077423572956, 32.18539524991137, 32.645865771631385, 32.48909307244844, 31.99926741257386, 31.865398604006256, 31.700048262548265, 31.700048262548265, 31.57546354494029, 31.01192848461055, 30.95315105514347, 30.406614258606666, 30.215979672501415, 30.163385002094678, 30.422732362821947, 30.590738013418427, 30.590738013418427, 30.065972222222225, 30.01151677667767, 30.18372685394935, 29.81667756491414, 29.707824154173935, 29.233437670528104, 29.08599337170766, 29.038585027839137, 28.99200840930164, 28.633546384265973, 28.486669451674757, 28.32515192179367, 27.935369381235077, 27.756297018161046, 27.676973908695157, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.293431775609996, 27.152805996181673, 27.152805996181673, 27.152805996181673, 27.08398536448072, 27.08398536448072, 27.08398536448072, 26.978413743119628, 26.777746354952235, 26.521336629108895, 26.390398550724637, 26.258646245059285, 26.078077507598785, 25.830826953457287, 25.668767695229906, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 24, 36, 45, 55, 67, 76, 85, 92, 99, 106, 114, 119, 125, 132, 138, 146, 152, 159, 164, 171, 177, 182, 189, 194, 201, 205, 212, 220, 225, 230, 236, 240, 247, 253, 258, 263, 268, 272, 274, 281, 285, 288, 294, 298, 299, 300, 300, 301, 303, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 305, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 1, 1, 1, 2, 2, 4, 4, 4, 5, 10, 11, 11, 11, 11, 12, 14, 17, 18, 21, 22, 24, 25, 28, 30, 32, 34, 35, 37, 38, 40, 42, 44, 49, 49, 57, 59, 65, 66, 68, 70, 74, 79, 81, 84, 84, 85, 89, 92, 93, 97, 98, 100, 104, 106, 108, 108, 110, 115, 116, 120, 123, 124, 127, 131, 131, 137, 138, 140, 144, 146, 152, 159, 160, 161, 164, 166, 169, 175, 181, 183, 186, 186, 186, 187, 188, 190, 190, 190, 191, 191, 191, 193, 196, 199, 205, 207, 209, 213, 216, 220], "exact_score": [18.254072168965788, 18.606411530247485, 19.07848964452738, 19.533297064440198, 19.95842149961399, 20.51876974904625, 20.913484950184912, 21.931133035784196, 22.535544851536223, 23.000857859809585, 23.171199137847857, 23.316302528722066, 23.680931614510953, 23.906356801093644, 24.19776921585324, 24.602560759835207, 24.639161787459766, 24.97095032613337, 25.172488759950063, 25.520534769552487, 25.67831507267427, 26.11151794120724, 26.518149970184858, 26.797564250095895, 25.65178646081287, 25.820753433972527, 26.107814247349133, 26.33526191894268, 26.6510528722187, 27.157040917230134, 27.293031566408576, 27.57273974709183, 28.093720996644535, 27.87883850017877, 29.146825396825392, 29.345838444414156, 29.816360532752174, 30.21772345301757, 30.731882387962244, 30.642857142857142, 30.47444117137411, 30.85165679174896, 30.7432945061811, 30.58665859183994, 31.003383190883195, 32.78813352826511, 32.69493177387914, 32.685080943594876, 32.04915866630637, 32.002077423572956, 32.18539524991137, 32.645865771631385, 32.48909307244844, 31.99926741257386, 31.865398604006256, 31.700048262548265, 31.700048262548265, 31.57546354494029, 31.01192848461055, 30.95315105514347, 30.406614258606666, 30.215979672501415, 30.163385002094678, 30.422732362821947, 30.590738013418427, 30.590738013418427, 30.065972222222225, 30.01151677667767, 30.18372685394935, 29.81667756491414, 29.707824154173935, 29.233437670528104, 29.08599337170766, 29.038585027839137, 28.99200840930164, 28.633546384265973, 28.486669451674757, 28.32515192179367, 27.935369381235077, 27.756297018161046, 27.676973908695157, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.363625362132826, 27.293431775609996, 27.152805996181673, 27.152805996181673, 27.152805996181673, 27.08398536448072, 27.08398536448072, 27.08398536448072, 26.978413743119628, 26.777746354952235, 26.521336629108895, 26.390398550724637, 26.258646245059285, 26.078077507598785, 25.830826953457287, 25.668767695229906, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.658946759494825, "threshold_area100": 27.66194711635286, "threshold_area25": 23.20165150417455}, "test": {"score": [18.129624718318375, 18.342899459398797, 18.791248494763103, 19.231412855651953, 19.44156822841526, 19.718635029864306, 19.815060886665005, 19.590142303780315, 19.7119503235735, 19.90312054050132, 20.384121055418614, 20.420034795284003, 20.596437330786056, 20.83167180361326, 21.036194382059044, 21.210721000555584, 21.54264629224058, 21.72728336806472, 22.02301096830725, 22.29970120800453, 22.501098425991948, 23.615558770718923, 24.113033364292953, 24.224948418993765, 24.717462849013096, 25.16851544365969, 25.547642769788826, 25.61953499515787, 25.70899718927962, 25.92003125248682, 26.531240908132325, 26.995270189049954, 27.059343471218426, 27.526502760459433, 27.879077242329963, 28.26361138861139, 28.573340514399483, 28.886872195817837, 29.0277204951436, 29.18477351447228, 29.274835353080192, 29.412500647936966, 30.119329591876344, 30.286958349859443, 30.34593145653287, 29.948804804083817, 30.00196656993146, 29.938408541977708, 29.88229195047377, 29.790757528031037, 29.750897587481745, 29.847425071212424, 29.722795149392038, 29.920914634048245, 30.26788004549491, 30.413880626872437, 30.1210727867381, 30.15758895475621, 29.981554582792185, 29.809072586365804, 29.706056849844785, 29.772052977833496, 29.730933231750754, 29.701063274666158, 29.899318974868294, 29.698531323608485, 29.823908972158492, 29.886682016244766, 29.67590979066414, 29.683825143466404, 29.59766829751812, 29.45518034145182, 29.345545045269244, 29.117635694211625, 29.05806293160372, 28.856664767701627, 28.71358192361236, 28.437852507576117, 28.2812503047502, 28.06248540647077, 27.931559096871695, 27.786317660041398, 27.75925486404407, 27.60919449485888, 27.551618625089624, 27.551618625089624, 27.4899522724912, 27.447602166990766, 27.431351039896462, 27.364803028342934, 27.279007506875775, 27.279007506875775, 27.271675178469312, 27.425106697930225, 27.34866235180417, 27.520124767972, 27.451105925971643, 27.38717603137872, 27.31285607368148, 27.27436667425225, 27.001009067886244], "calls": [0.0, 0.7656458055925432, 2.2969374167776295, 3.462050599201065, 4.061251664447404, 4.8934753661784285, 5.59254327563249, 7.123834886817576, 8.322237017310254, 8.954727030625833, 9.986684420772303, 10.818908122503329, 11.484687083888149, 13.015978695073235, 13.81491344873502, 14.580559254327563, 15.745672436750999, 16.544607190412783, 17.576564580559253, 18.47536617842876, 19.474034620505993, 21.571238348868178, 23.202396804260985, 23.601864181091877, 24.63382157123835, 25.96537949400799, 27.296937416777627, 27.996005326231693, 28.56191744340879, 29.427430093209058, 30.392809587217045, 31.22503328894807, 31.757656458055926, 33.68841544607191, 35.486018641810915, 36.651131824234355, 37.35019973368841, 37.849533954727036, 38.8814913448735, 39.38082556591212, 40.246338215712385, 41.07856191744341, 42.0439414114514, 42.47669773635153, 43.44207723035952, 44.6404793608522, 45.43941411451398, 46.03861517976032, 47.17043941411451, 48.23568575233023, 49.46737683089214, 50.8988015978695, 51.49800266311585, 52.629826897470046, 54.19440745672437, 55.52596537949401, 57.12383488681758, 58.25565912117177, 59.62050599201065, 61.18508655126498, 62.4167776298269, 62.68308921438083, 63.51531291611185, 64.4474034620506, 66.07856191744341, 67.01065246338216, 68.50865512649801, 69.84021304926765, 70.90545938748336, 71.53794940079894, 72.50332889480693, 73.26897470039947, 74.10119840213049, 74.96671105193076, 75.53262316910786, 76.53129161118508, 77.23035952063914, 78.12916111850865, 79.9600532623169, 80.72569906790945, 81.3249001331558, 82.02396804260985, 82.72303595206392, 84.25432756324899, 85.85219707057257, 86.31824234354194, 87.28362183754993, 88.68175765645806, 89.48069241011984, 90.81225033288948, 91.91078561917443, 92.17709720372837, 92.64314247669773, 93.24234354194407, 94.14114513981359, 95.63914780292943, 96.5046604527297, 97.00399467376832, 98.20239680426099, 98.73501997336885, 100.0], "rescued": [0, 51, 123, 178, 204, 238, 263, 324, 375, 395, 425, 450, 469, 509, 532, 554, 587, 607, 635, 656, 677, 736, 777, 786, 811, 844, 876, 891, 902, 921, 944, 966, 974, 1022, 1069, 1094, 1112, 1122, 1140, 1150, 1162, 1175, 1192, 1200, 1214, 1230, 1241, 1244, 1249, 1252, 1253, 1255, 1255, 1257, 1260, 1261, 1261, 1262, 1263, 1265, 1267, 1268, 1268, 1268, 1269, 1269, 1271, 1272, 1272, 1273, 1275, 1275, 1276, 1276, 1276, 1276, 1276, 1276, 1278, 1278, 1278, 1278, 1278, 1278, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1279, 1280, 1280, 1282, 1282, 1282, 1282, 1282, 1282], "harmed": [0, 4, 9, 11, 13, 17, 21, 37, 44, 44, 53, 62, 63, 75, 78, 84, 91, 93, 96, 99, 102, 117, 129, 130, 135, 143, 151, 158, 160, 164, 169, 178, 181, 204, 234, 246, 252, 258, 269, 274, 280, 288, 295, 295, 302, 313, 318, 323, 330, 340, 342, 352, 358, 367, 376, 383, 397, 410, 428, 444, 462, 464, 466, 468, 479, 488, 498, 511, 522, 525, 539, 549, 559, 571, 576, 590, 598, 616, 649, 663, 672, 682, 684, 692, 697, 697, 701, 704, 705, 709, 718, 718, 719, 725, 729, 737, 743, 748, 753, 756, 777], "exact_score": [18.129624718318375, 18.40890014522558, 18.802758230844006, 19.04307365545973, 19.42283227713815, 19.76831944754647, 20.10106921205896, 19.525128299174572, 19.8839270506363, 19.90312054050132, 20.384121055418614, 20.457388140784705, 20.68428890126772, 20.821892694138086, 21.065835310317482, 21.313590635936137, 21.595446595446596, 21.85714234031904, 22.125194197881346, 22.42312499937398, 22.671823649014094, 23.4604029546199, 23.659028340115555, 24.049180012918875, 24.326318813683937, 24.845107895887995, 25.134574590960362, 25.462187476761727, 25.61953499515787, 25.807702052044146, 26.079316101563837, 26.631678809263203, 27.01655418282762, 27.233074919067217, 27.652764151618275, 27.720986202619063, 27.932467185015476, 28.348550125119438, 28.82465641348976, 29.059004839712678, 29.348569604475117, 29.412500647936966, 29.71841319368011, 30.31413226290292, 30.438635204830522, 29.936744120637805, 29.976174047653103, 29.889536950445844, 29.8516908754211, 29.769960768309627, 29.67184292561047, 29.834312227910896, 29.959213049628612, 30.258554093825108, 30.30929244116364, 30.158026378277714, 30.357419288989256, 30.1210727867381, 30.21100313671443, 30.11713299277408, 29.890753273409338, 29.88500680358395, 29.733888094553528, 29.772052977833496, 29.730933231750754, 29.648833897440515, 29.59645810449275, 29.698531323608485, 29.92728444785407, 29.794904998639055, 29.886682016244766, 29.67590979066414, 29.552942339699996, 29.482670608510524, 29.365431829345017, 29.117635694211625, 28.93978365540417, 28.76324127489972, 28.495189607277542, 28.25364549304286, 28.2812503047502, 28.0379894905728, 27.786317660041398, 27.75170098556665, 27.616692053793184, 27.530137936580733, 27.551618625089624, 27.508783482082567, 27.4899522724912, 27.447602166990766, 27.415134936782977, 27.364803028342934, 27.279007506875775, 27.25709139292557, 27.37196467587629, 27.268929407965036, 27.490015590546562, 27.403037178366237, 27.353314714367468, 27.24798476470451, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.711342947721008, "threshold_area100": 26.712191905622053, "threshold_area25": 21.092729118563895}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.948804804083817, "calls": 44.6404793608522, "rescued": 1230, "harmed": 313}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.5964179039001465, "operator": "ge"}, {"budget": 0.02, "threshold": 1.462735891342163, "operator": "ge"}, {"budget": 0.03, "threshold": 1.3343665599822998, "operator": "ge"}, {"budget": 0.04, "threshold": 1.290655255317688, "operator": "ge"}, {"budget": 0.05, "threshold": 1.268282413482666, "operator": "ge"}, {"budget": 0.06, "threshold": 1.2345826625823975, "operator": "ge"}, {"budget": 0.07, "threshold": 1.220177173614502, "operator": "ge"}, {"budget": 0.08, "threshold": 1.1862542629241943, "operator": "ge"}, {"budget": 0.09, "threshold": 1.15302574634552, "operator": "ge"}, {"budget": 0.1, "threshold": 1.1300865411758423, "operator": "ge"}, {"budget": 0.11, "threshold": 1.1157441139221191, "operator": "ge"}, {"budget": 0.12, "threshold": 1.094364047050476, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0718317031860352, "operator": "ge"}, {"budget": 0.14, "threshold": 1.0486319065093994, "operator": "ge"}, {"budget": 0.15, "threshold": 1.0246915817260742, "operator": "ge"}, {"budget": 0.16, "threshold": 1.0084922313690186, "operator": "ge"}, {"budget": 0.17, "threshold": 0.9944540858268738, "operator": "ge"}, {"budget": 0.18, "threshold": 0.9600342512130737, "operator": "ge"}, {"budget": 0.19, "threshold": 0.9211609363555908, "operator": "ge"}, {"budget": 0.2, "threshold": 0.88383948802948, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8489726185798645, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8238308429718018, "operator": "ge"}, {"budget": 0.23, "threshold": 0.807952344417572, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7832770347595215, "operator": "ge"}, {"budget": 0.25, "threshold": 0.780734658241272, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7663542032241821, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7484604716300964, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7258608937263489, "operator": "ge"}, {"budget": 0.29, "threshold": 0.7063570022583008, "operator": "ge"}, {"budget": 0.3, "threshold": 0.6919735670089722, "operator": "ge"}, {"budget": 0.31, "threshold": 0.6726564168930054, "operator": "ge"}, {"budget": 0.32, "threshold": 0.6384164094924927, "operator": "ge"}, {"budget": 0.33, "threshold": 0.6128135919570923, "operator": "ge"}, {"budget": 0.34, "threshold": 0.5880014896392822, "operator": "ge"}, {"budget": 0.35, "threshold": 0.5743820071220398, "operator": "ge"}, {"budget": 0.36, "threshold": 0.5446160435676575, "operator": "ge"}, {"budget": 0.37, "threshold": 0.4984148144721985, "operator": "ge"}, {"budget": 0.38, "threshold": 0.4362010359764099, "operator": "ge"}, {"budget": 0.39, "threshold": 0.4089586138725281, "operator": "ge"}, {"budget": 0.4, "threshold": 0.3592396378517151, "operator": "ge"}, {"budget": 0.41, "threshold": 0.33360928297042847, "operator": "ge"}, {"budget": 0.42, "threshold": 0.2741151452064514, "operator": "ge"}, {"budget": 0.43, "threshold": 0.23931458592414856, "operator": "ge"}, {"budget": 0.44, "threshold": 0.1480170488357544, "operator": "ge"}, {"budget": 0.45, "threshold": -0.013969779014587402, "operator": "ge"}, {"budget": 0.46, "threshold": -0.20466268062591553, "operator": "ge"}, {"budget": 0.47, "threshold": -0.4419301450252533, "operator": "ge"}, {"budget": 0.48, "threshold": -0.681963324546814, "operator": "ge"}, {"budget": 0.49, "threshold": -0.8670604825019836, "operator": "ge"}, {"budget": 0.5, "threshold": -1.0659098625183105, "operator": "ge"}, {"budget": 0.51, "threshold": -1.3217800855636597, "operator": "ge"}, {"budget": 0.52, "threshold": -2.6380860805511475, "operator": "ge"}, {"budget": 0.53, "threshold": -2.8646492958068848, "operator": "ge"}, {"budget": 0.54, "threshold": -2.907158374786377, "operator": "ge"}, {"budget": 0.55, "threshold": -3.0288326740264893, "operator": "ge"}, {"budget": 0.56, "threshold": -3.084836959838867, "operator": "ge"}, {"budget": 0.57, "threshold": -3.122699499130249, "operator": "ge"}, {"budget": 0.58, "threshold": -3.1686744689941406, "operator": "ge"}, {"budget": 0.59, "threshold": -3.1883504390716553, "operator": "ge"}, {"budget": 0.6, "threshold": -3.226316452026367, "operator": "ge"}, {"budget": 0.61, "threshold": -3.2594175338745117, "operator": "ge"}, {"budget": 0.62, "threshold": -3.287710666656494, "operator": "ge"}, {"budget": 0.63, "threshold": -3.3147244453430176, "operator": "ge"}, {"budget": 0.64, "threshold": -3.3332579135894775, "operator": "ge"}, {"budget": 0.65, "threshold": -3.3726654052734375, "operator": "ge"}, {"budget": 0.66, "threshold": -3.403379440307617, "operator": "ge"}, {"budget": 0.67, "threshold": -3.42500901222229, "operator": "ge"}, {"budget": 0.68, "threshold": -3.456310272216797, "operator": "ge"}, {"budget": 0.69, "threshold": -3.4861254692077637, "operator": "ge"}, {"budget": 0.7, "threshold": -3.52512264251709, "operator": "ge"}, {"budget": 0.71, "threshold": -3.5653023719787598, "operator": "ge"}, {"budget": 0.72, "threshold": -3.6113719940185547, "operator": "ge"}, {"budget": 0.73, "threshold": -3.6289710998535156, "operator": "ge"}, {"budget": 0.74, "threshold": -3.6690802574157715, "operator": "ge"}, {"budget": 0.75, "threshold": -3.711195468902588, "operator": "ge"}, {"budget": 0.76, "threshold": -3.743368625640869, "operator": "ge"}, {"budget": 0.77, "threshold": -3.795414924621582, "operator": "ge"}, {"budget": 0.78, "threshold": -3.8724634647369385, "operator": "ge"}, {"budget": 0.79, "threshold": -3.947601318359375, "operator": "ge"}, {"budget": 0.8, "threshold": -4.04849910736084, "operator": "ge"}, {"budget": 0.81, "threshold": -4.285282135009766, "operator": "ge"}, {"budget": 0.82, "threshold": -10.980627059936523, "operator": "ge"}, {"budget": 0.83, "threshold": -11.053467750549316, "operator": "ge"}, {"budget": 0.84, "threshold": -11.094281196594238, "operator": "ge"}, {"budget": 0.85, "threshold": -11.146958351135254, "operator": "ge"}, {"budget": 0.86, "threshold": -11.197421073913574, "operator": "ge"}, {"budget": 0.87, "threshold": -11.246533393859863, "operator": "ge"}, {"budget": 0.88, "threshold": -11.276710510253906, "operator": "ge"}, {"budget": 0.89, "threshold": -11.31483268737793, "operator": "ge"}, {"budget": 0.9, "threshold": -11.366476058959961, "operator": "ge"}, {"budget": 0.91, "threshold": -11.436914443969727, "operator": "ge"}, {"budget": 0.92, "threshold": -11.484947204589844, "operator": "ge"}, {"budget": 0.93, "threshold": -11.566726684570312, "operator": "ge"}, {"budget": 0.94, "threshold": -11.649770736694336, "operator": "ge"}, {"budget": 0.95, "threshold": -11.692230224609375, "operator": "ge"}, {"budget": 0.96, "threshold": -11.818147659301758, "operator": "ge"}, {"budget": 0.97, "threshold": -11.878642082214355, "operator": "ge"}, {"budget": 0.98, "threshold": -12.001541137695312, "operator": "ge"}, {"budget": 0.99, "threshold": -12.201926231384277, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [18.254072168965788, 18.62253743104807, 18.91000328455166, 19.244132597880164, 19.70945720945721, 20.074137321172593, 20.893315615023706, 21.00302840944226, 21.09847478287059, 21.55353182585118, 21.670755293457, 21.927160289058065, 22.361922524548188, 22.814492131663624, 23.294709166802193, 23.788743173200682, 24.051349523966156, 24.154042743753667, 24.17435226588289, 24.502864846646446, 24.96082483622289, 25.34087748574607, 25.638250638250636, 25.934711839884255, 26.318570440749856, 26.83086308309251, 27.364077364077367, 27.88187985556407, 28.028510189500906, 28.353821683188002, 28.67330293118549, 29.660401758088316, 30.052183696570033, 28.579713052329687, 28.958182621975727, 29.092305543918446, 29.867216117216117, 30.40801423582401, 30.859355166285855, 30.678668404474852, 30.730444670536837, 30.326534542007344, 30.446924209810806, 30.61069139194139, 31.003383190883195, 32.78813352826511, 32.69493177387914, 32.59974700344241, 32.04915866630637, 32.002077423572956, 32.37304912608984, 32.645865771631385, 32.48909307244844, 32.267257893476554, 31.764592152393355, 31.63702305246423, 31.63702305246423, 31.101055044325342, 31.101055044325342, 30.984821026502797, 30.656939421971796, 30.911623837155755, 30.855632795722382, 30.28769841269841, 30.670103092783506, 30.555555555555557, 30.22305327017369, 30.01151677667767, 29.78394188292774, 29.603002351238928, 29.55203395164667, 29.476696917546754, 29.329252618726304, 28.90126430766825, 28.813596277276666, 28.579528616631777, 28.23918061302355, 28.08277581004854, 28.00770073497346, 27.756297018161046, 27.544838602141247, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.292501606798545, 27.152805996181673, 27.08532961291582, 27.08532961291582, 27.08532961291582, 26.948359770310994, 26.784001905953126, 26.595741391013654, 26.375776891120662, 26.172234803790968, 25.930924695459577, 25.830826953457287, 25.697184105202975, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 11, 22, 32, 42, 50, 62, 67, 71, 81, 86, 93, 102, 111, 119, 127, 132, 139, 144, 151, 160, 167, 174, 181, 188, 195, 202, 208, 215, 220, 226, 233, 239, 243, 249, 252, 258, 263, 268, 272, 278, 281, 285, 289, 294, 298, 299, 299, 300, 301, 303, 304, 304, 304, 304, 304, 304, 304, 304, 304, 304, 305, 305, 305, 306, 306, 306, 306, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307, 307], "harmed": [0, 0, 1, 2, 2, 2, 5, 6, 10, 11, 12, 14, 14, 14, 14, 15, 16, 19, 24, 25, 27, 27, 30, 31, 33, 33, 34, 34, 39, 40, 42, 45, 48, 53, 55, 59, 61, 66, 67, 70, 73, 79, 80, 82, 84, 84, 85, 89, 92, 93, 96, 98, 100, 103, 107, 109, 109, 114, 114, 116, 121, 124, 125, 129, 130, 132, 135, 138, 145, 148, 149, 150, 153, 159, 161, 167, 170, 173, 174, 177, 184, 186, 186, 186, 186, 186, 186, 188, 190, 191, 191, 191, 193, 195, 197, 200, 204, 207, 213, 216, 220], "exact_score": [18.254072168965788, 18.62253743104807, 18.91000328455166, 19.244132597880164, 19.70945720945721, 20.074137321172593, 20.893315615023706, 21.00302840944226, 21.09847478287059, 21.55353182585118, 21.670755293457, 21.927160289058065, 22.361922524548188, 22.814492131663624, 23.294709166802193, 23.788743173200682, 24.051349523966156, 24.154042743753667, 24.17435226588289, 24.502864846646446, 24.96082483622289, 25.34087748574607, 25.638250638250636, 25.934711839884255, 26.318570440749856, 26.83086308309251, 27.364077364077367, 27.88187985556407, 28.028510189500906, 28.353821683188002, 28.67330293118549, 29.660401758088316, 30.052183696570033, 28.579713052329687, 28.958182621975727, 29.092305543918446, 29.867216117216117, 30.40801423582401, 30.859355166285855, 30.678668404474852, 30.730444670536837, 30.326534542007344, 30.446924209810806, 30.61069139194139, 31.003383190883195, 32.78813352826511, 32.69493177387914, 32.59974700344241, 32.04915866630637, 32.002077423572956, 32.37304912608984, 32.645865771631385, 32.48909307244844, 32.267257893476554, 31.764592152393355, 31.63702305246423, 31.63702305246423, 31.101055044325342, 31.101055044325342, 30.984821026502797, 30.656939421971796, 30.911623837155755, 30.855632795722382, 30.28769841269841, 30.670103092783506, 30.555555555555557, 30.22305327017369, 30.01151677667767, 29.78394188292774, 29.603002351238928, 29.55203395164667, 29.476696917546754, 29.329252618726304, 28.90126430766825, 28.813596277276666, 28.579528616631777, 28.23918061302355, 28.08277581004854, 28.00770073497346, 27.756297018161046, 27.544838602141247, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.434520884520886, 27.292501606798545, 27.152805996181673, 27.08532961291582, 27.08532961291582, 27.08532961291582, 26.948359770310994, 26.784001905953126, 26.595741391013654, 26.375776891120662, 26.172234803790968, 25.930924695459577, 25.830826953457287, 25.697184105202975, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 27.606301721203664, "threshold_area100": 27.60920657342843, "threshold_area25": 22.60738565001646}, "test": {"score": [18.129624718318375, 18.255507312716656, 18.554747648996987, 18.962428331978085, 18.690930001341037, 18.912850055941067, 19.129186204609265, 19.157601838441465, 19.40093683734835, 19.87374901544422, 20.057086115835265, 20.199690447295378, 20.468621221756166, 20.834358567262555, 21.364236657215955, 21.534939275205666, 21.600381848422266, 21.721204886892746, 22.158071816645748, 22.682799576805348, 23.298602626572677, 23.674387023208233, 23.85563816895897, 24.019627595570597, 24.394731132878782, 24.420682610071218, 24.80852234708595, 24.926484204300724, 25.117289418870698, 25.414296074813, 25.51598183333774, 26.032953021706163, 26.65491315610659, 27.11025613377555, 27.552219144043345, 28.278388278388285, 28.704169971737812, 28.89698127411997, 29.203576016071203, 29.277420494261463, 29.446263678319923, 29.70889550155242, 29.748192780852722, 30.205126899553257, 30.345154536045026, 29.948804804083817, 29.963537047178963, 29.956869838496193, 29.870112055027654, 29.885958294813904, 29.834700062213354, 29.947316125545736, 29.722795149392038, 29.89571919442282, 29.84453803298665, 30.357419288989256, 30.06349872630508, 29.919728241288198, 29.608671484026882, 29.730746929904416, 29.622904082142643, 29.53989028810794, 29.63161813559532, 29.614019004039033, 29.821592255818995, 29.579387149743102, 29.394301131384665, 29.91813404385756, 29.702629169487377, 29.700862773477922, 29.616223226833633, 29.391619211794705, 29.182430704262714, 29.08899872172984, 28.9855290802077, 28.768405654581958, 28.526515747200964, 28.388936448451375, 28.073212369930122, 27.937942145546007, 27.96847416691507, 27.786317660041398, 27.679138924864375, 27.57326497529412, 27.53306472732827, 27.4872907000264, 27.404814207275948, 27.361884865042335, 27.31971811713191, 27.269090194148117, 27.204766948839914, 27.175599080977143, 27.128365742849798, 27.344682455309112, 27.50274769175277, 27.49570442315199, 27.40794210230188, 27.385070827419227, 27.307883199602944, 27.187140887318822, 27.001009067886244], "calls": [0.0, 0.3328894806924101, 0.9986684420772304, 2.896138482023968, 3.761651131824234, 4.327563249001331, 5.326231691078561, 5.825565912117177, 6.92410119840213, 7.989347536617843, 8.921438082556591, 9.52063914780293, 10.486018641810919, 11.318242343541945, 12.48335552596538, 13.21571238348868, 14.08122503328895, 14.84687083888149, 16.711051930758988, 18.608521970705727, 20.372836218375497, 21.870838881491345, 22.836218375499335, 23.635153129161118, 25.03328894806924, 25.099866844207725, 25.732356857523303, 26.531291611185086, 27.330226364846872, 28.29560585885486, 29.327563249001333, 30.25965379494008, 32.05725699067909, 33.68841544607191, 34.6537949400799, 35.486018641810915, 36.651131824234355, 37.849533954727036, 39.114513981358186, 39.74700399467377, 40.612516644474034, 40.97869507323569, 41.91078561917443, 42.34354194407457, 43.275632490013315, 44.6404793608522, 45.27296937416778, 45.97203728362184, 46.70439414114514, 47.93608521970706, 49.2676431424767, 50.632490013315575, 51.49800266311585, 52.99600532623169, 53.52862849533955, 55.85885486018641, 57.35685752330226, 58.45539280958722, 59.553928095872166, 60.38615179760319, 61.71770972037284, 62.916111850865505, 63.948069241011986, 65.04660452729694, 65.71238348868175, 66.94407456724367, 68.37549933422103, 68.94141145139814, 70.07323568575234, 70.87217043941412, 71.87083888149135, 73.26897470039947, 74.16777629826898, 74.56724367509987, 75.46604527296937, 76.63115845539281, 77.33022636484687, 78.29560585885486, 79.39414114513981, 80.29294274300932, 81.15845539280959, 82.02396804260985, 83.55525965379495, 84.35419440745673, 84.88681757656458, 86.41810918774966, 87.8162450066578, 89.0479360852197, 89.61384820239681, 90.31291611185085, 91.41145139813582, 92.37683089214381, 93.07589880159787, 94.27430093209055, 94.90679094540613, 95.33954727030626, 96.5046604527297, 97.20372836218375, 98.20239680426099, 99.06790945406125, 100.0], "rescued": [0, 18, 47, 125, 160, 188, 224, 243, 282, 322, 346, 360, 393, 417, 461, 483, 504, 525, 584, 643, 694, 737, 759, 774, 810, 812, 830, 845, 865, 886, 912, 937, 986, 1027, 1055, 1077, 1103, 1124, 1147, 1153, 1170, 1177, 1191, 1197, 1212, 1230, 1239, 1243, 1246, 1250, 1251, 1255, 1255, 1257, 1257, 1261, 1261, 1261, 1261, 1263, 1264, 1265, 1267, 1268, 1269, 1269, 1270, 1272, 1272, 1273, 1274, 1275, 1275, 1275, 1276, 1276, 1276, 1276, 1276, 1277, 1278, 1278, 1278, 1278, 1278, 1278, 1278, 1278, 1278, 1278, 1279, 1279, 1279, 1280, 1281, 1281, 1282, 1282, 1282, 1282, 1282], "harmed": [0, 2, 7, 17, 24, 26, 35, 38, 47, 52, 56, 56, 60, 67, 69, 73, 81, 88, 100, 110, 119, 126, 131, 134, 140, 140, 146, 152, 158, 169, 184, 191, 206, 223, 233, 244, 253, 263, 273, 276, 284, 285, 294, 297, 301, 313, 318, 321, 328, 335, 339, 349, 358, 369, 372, 387, 402, 414, 426, 437, 449, 457, 468, 478, 483, 495, 507, 511, 522, 526, 537, 551, 565, 571, 582, 597, 611, 623, 643, 653, 670, 682, 688, 693, 696, 699, 705, 707, 710, 713, 722, 726, 730, 731, 737, 738, 746, 748, 754, 762, 777], "exact_score": [18.129624718318375, 18.554747648996987, 18.67533238711697, 18.971272452034686, 18.791199402654513, 19.07276407085366, 19.206080849986705, 19.413284711694796, 19.87374901544422, 20.080213426144226, 20.342027382731757, 20.761216679862763, 21.09260934286409, 21.467412065073027, 21.590624499744806, 21.779401923877316, 22.209728827488068, 22.2030163665059, 22.475533498380308, 22.810323433803354, 23.067755525736573, 23.39295975141064, 23.7083115647254, 23.914541566172243, 24.13451557313954, 24.382143761714232, 24.876439216056212, 25.08030707341959, 25.342266396782676, 25.48044962284687, 25.923633137666602, 26.237024785501244, 26.605455140909424, 26.86116342208854, 27.218896745205708, 27.561740166716064, 28.499048179869035, 28.764692522383278, 28.96121296121297, 29.1726260749059, 29.331003459185524, 29.70889550155242, 29.794329591876345, 30.26560287202586, 30.438635204830522, 29.94456979113117, 29.956869838496193, 29.954665538450087, 29.885958294813904, 29.834700062213354, 30.06251826605803, 29.887995398899175, 29.62707839774568, 29.89571919442282, 30.237123900733376, 30.092096250425083, 30.287953253326354, 30.11739249843785, 29.942969447318347, 29.743412416428903, 29.475403644168995, 29.73188877586702, 29.61179180131776, 29.53989028810794, 29.63161813559532, 29.614019004039033, 29.81058211574853, 29.579387149743102, 29.414204289178947, 29.91813404385756, 29.723333730550678, 29.682981613223852, 29.645075749187043, 29.47210675719245, 29.2788675244328, 28.96464456145279, 28.8600335531017, 28.66016433981401, 28.423475017189155, 28.15632778466524, 27.977579891372994, 28.02140514905468, 27.786317660041398, 27.686692803341796, 27.624217693472552, 27.513804326711945, 27.4872907000264, 27.41222846057443, 27.404814207275948, 27.361884865042335, 27.2851787157403, 27.230672271702044, 27.182850834889717, 27.128365742849798, 27.344682455309112, 27.50274769175277, 27.45386205781194, 27.385070827419227, 27.339504467164044, 27.218561491379933, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 26.667977417859845, "threshold_area100": 26.582939269919194, "threshold_area25": 21.015726347549897}, "selected_budget": 45, "selected": {"val": {"score": 32.78813352826511, "calls": 44.935805991440795, "rescued": 298, "harmed": 84}, "test": {"score": 29.948804804083817, "calls": 44.6404793608522, "rescued": 1230, "harmed": 313}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 32.614453827508015, "calls": 43.93723252496434}, "test": {"score": 30.289712994994076, "calls": 44.10785619174435}}, "old_normalized_interior": {"val": {"score": 32.614453827508015, "calls": 43.93723252496434}, "test": {"score": 30.289712994994076, "calls": 44.10785619174435}}, "old_normalized_endpoints": {"val": {"score": 32.614453827508015, "calls": 43.93723252496434}, "test": {"score": 30.289712994994076, "calls": 44.10785619174435}}}, "baseline_old_normalized": {"val": {"score": [18.254072168965788, 18.55369268714164, 18.978116869272714, 19.443048610430218, 19.928684266026398, 20.82811519670685, 21.297523151257302, 22.214609993070244, 22.90528437080161, 23.5437123851758, 23.84903201015362, 24.460625587139322, 24.922191059917058, 25.39030484757621, 26.120588703708908, 26.70116620952306, 26.536903948057898, 26.916054143853607, 27.195692360265312, 27.546323067538363, 27.776249650824898, 26.039812422241365, 26.386554621848735, 26.619198312236293, 26.818895115710117, 26.963009889839157, 27.181906787169947, 27.31510224193151, 27.407990323591346, 27.78652079494202, 28.144262208974034, 28.246031746031747, 28.48986611472738, 29.016580903373356, 29.169397653536084, 29.11183643411998, 29.5716625513761, 29.705432254606134, 29.6516046801967, 29.774078654477165, 31.736391608356605, 32.334339369550634, 32.57037997868769, 32.60392919392779, 32.614453827508015, 32.547863264342105, 32.29276122352578, 32.29276122352578, 32.21506517048528, 31.940614083471225, 31.73976532107112, 31.662854076647186, 31.58671580955414, 31.471773280818503, 31.40871021775544, 31.396396396396398, 31.40871021775544, 31.13989301345437, 30.651134460179684, 30.651134460179684, 30.956456589031227, 30.824942533466537, 30.717415651746105, 30.654124512505597, 30.477941176470587, 30.071279057673305, 30.012507385619237, 29.917810415922265, 29.86043806939329, 29.77086132258546, 29.566532094621977, 29.496642246642246, 29.496642246642246, 29.343894082700057, 29.26881900762498, 29.21506841039612, 29.070471199406732, 28.89516550342583, 28.824041748091545, 28.636354060403857, 28.361549989839464, 27.97166713575382, 27.833650764156637, 28.238691268207038, 28.238691268207038, 28.12214893585199, 27.73562915072349, 27.52285425674506, 27.437113023786132, 27.08271617129127, 26.923076923076927, 26.66818983173887, 26.28754096145401, 26.054637452942536, 26.248615863893253, 26.089327510870884, 25.941478048505296, 26.102588283929645, 25.68952418174193, 25.65261473292303, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [18.254072168965788, 18.55369268714164, 18.978116869272714, 19.443048610430218, 19.928684266026398, 20.82811519670685, 21.297523151257302, 22.214609993070244, 22.90528437080161, 23.5437123851758, 23.84903201015362, 24.460625587139322, 24.922191059917058, 25.39030484757621, 26.120588703708908, 26.70116620952306, 26.536903948057898, 26.916054143853607, 27.195692360265312, 27.546323067538363, 27.776249650824898, 26.039812422241365, 26.386554621848735, 26.619198312236293, 26.818895115710117, 26.963009889839157, 27.181906787169947, 27.31510224193151, 27.407990323591346, 27.78652079494202, 28.144262208974034, 28.246031746031747, 28.48986611472738, 29.016580903373356, 29.169397653536084, 29.11183643411998, 29.5716625513761, 29.705432254606134, 29.6516046801967, 29.774078654477165, 31.736391608356605, 32.334339369550634, 32.57037997868769, 32.60392919392779, 32.614453827508015, 32.547863264342105, 32.29276122352578, 32.29276122352578, 32.21506517048528, 31.940614083471225, 31.73976532107112, 31.662854076647186, 31.58671580955414, 31.471773280818503, 31.40871021775544, 31.396396396396398, 31.40871021775544, 31.13989301345437, 30.651134460179684, 30.651134460179684, 30.956456589031227, 30.824942533466537, 30.717415651746105, 30.654124512505597, 30.477941176470587, 30.071279057673305, 30.012507385619237, 29.917810415922265, 29.86043806939329, 29.77086132258546, 29.566532094621977, 29.496642246642246, 29.496642246642246, 29.343894082700057, 29.26881900762498, 29.21506841039612, 29.070471199406732, 28.89516550342583, 28.824041748091545, 28.636354060403857, 28.361549989839464, 27.97166713575382, 27.833650764156637, 28.238691268207038, 28.238691268207038, 28.12214893585199, 27.73562915072349, 27.52285425674506, 27.437113023786132, 27.08271617129127, 26.923076923076927, 26.66818983173887, 26.28754096145401, 26.054637452942536, 26.248615863893253, 26.089327510870884, 25.941478048505296, 26.102588283929645, 25.68952418174193, 25.65261473292303, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 24.352620819322183, "exact_area100": 28.015522424427843}, "test": {"score": [18.129624718318375, 18.328808716359575, 19.039313517378954, 19.174899521090325, 19.139628293146664, 19.40314780039465, 19.693414354285704, 20.078100004570594, 20.020675566595596, 20.412318085941365, 20.756340812146433, 20.856890755210266, 21.025959969922482, 21.217732981195912, 21.480946127951952, 21.635851911546737, 21.8441898258989, 21.96383068867631, 22.540679889598778, 22.721759228922846, 22.906941293535247, 23.36174494667553, 23.573788127094105, 24.18018556536625, 24.322782260907573, 24.49907188999567, 24.841027736722097, 24.952667250772933, 24.929987239236063, 25.443912664673686, 25.7878315490689, 26.597323213479385, 27.006440323137237, 27.222369012949642, 27.21552900797007, 27.81039600290547, 28.36319048064987, 28.528899431310418, 28.83784778791181, 28.921389408826027, 28.968853681412867, 29.076096283790527, 29.693695488646927, 30.2706941250263, 30.289712994994076, 30.239707515030094, 30.213326385297005, 29.898340040422365, 29.922400727603254, 29.903180234468106, 30.13349430413013, 30.12155125483671, 29.988510385239497, 29.422767066670335, 29.46356586036788, 29.713443281747814, 29.588851074407767, 29.44629412107298, 29.30242247075195, 29.29033978159462, 29.30284270458756, 29.23486899460948, 29.441068551894606, 29.286885593472267, 29.260950563431482, 29.223311885983115, 29.138445975662602, 29.132336689742267, 29.097561817876706, 29.052434220530138, 29.308369628004332, 29.21495791148737, 29.4043755009767, 29.163893731625667, 29.089967476088606, 29.091879791967013, 28.89262396198186, 29.031888593365668, 29.01384820570142, 28.899632843500957, 28.790330533851872, 28.82619609606234, 28.956371365923683, 28.939320077604126, 28.858501009844357, 28.75093234290106, 28.625914129520748, 28.388983915174837, 28.296765015298362, 28.040364824161458, 27.92173142073523, 27.73608774294562, 27.59722040850313, 27.391992051790886, 27.304059174630314, 27.383184189683618, 27.237698161248446, 27.14660851919877, 27.31696804812782, 27.183759568196237, 27.001009067886244], "calls": [0.0, 0.9653794940079893, 2.396804260985353, 3.129161118508655, 3.661784287616511, 4.360852197070573, 5.359520639147803, 6.1584553928095875, 7.423435419440745, 9.221038615179761, 10.319573901464715, 11.750998668442076, 12.283621837549934, 13.1491344873502, 13.84820239680426, 14.447403462050598, 15.179760319573901, 15.612516644474034, 17.310252996005325, 18.042609853528628, 19.0745672436751, 20.505992010652463, 21.304926764314246, 22.103861517976032, 22.902796271637815, 24.467376830892142, 26.198402130492678, 26.59786950732357, 27.26364846870839, 29.26098535286285, 30.62583222370173, 32.05725699067909, 33.45539280958722, 34.68708388814913, 35.086551264980024, 35.81890812250333, 36.78428761651132, 37.64980026631159, 38.98135818908123, 40.1797603195739, 40.57922769640479, 41.51131824234354, 42.47669773635153, 43.60852197070572, 44.10785619174435, 44.87350199733689, 45.905459387483354, 46.57123834886818, 47.93608521970706, 48.568575233022635, 49.83355525965379, 51.26498002663116, 52.36351531291611, 53.96138482023968, 55.09320905459387, 55.792276964047936, 57.19041278295606, 57.822902796271634, 58.38881491344874, 59.52063914780293, 60.35286284953395, 61.284953395472705, 62.38348868175766, 63.41544607190413, 64.38082556591212, 65.5126498002663, 65.94540612516646, 66.97736351531292, 67.44340878828228, 68.37549933422103, 69.20772303595206, 69.94007989347537, 71.33821571238349, 72.20372836218375, 73.36884154460719, 74.56724367509987, 75.19973368841545, 76.66444740346205, 77.99600532623168, 79.06125166444741, 79.32756324900133, 80.5925432756325, 81.8242343541944, 82.42343541944075, 83.75499334221038, 84.38748335552596, 85.35286284953395, 87.0838881491345, 87.84953395472704, 89.41411451398136, 90.61251664447404, 91.711051930759, 92.41011984021304, 93.24234354194407, 94.10785619174435, 95.57256990679095, 96.13848202396804, 96.80426098535287, 97.6364846870839, 98.56857523302264, 100.0], "exact_score": [18.129624718318375, 18.33860367522075, 18.851511528806746, 19.131875018946456, 19.277738594253595, 19.531137658578483, 20.00323619769678, 19.846899744341055, 20.174829354590994, 20.347771934260138, 20.675154931784313, 20.854735645779776, 20.91539947752346, 21.155039279855785, 21.523560179723436, 21.813664108274615, 22.334740013938976, 22.47708067531922, 22.70623368941035, 22.880127008745042, 23.374277353924523, 23.48851084292608, 24.146505219468377, 24.334455945202848, 24.408432539532082, 24.649313817937774, 24.81217343222691, 24.986408580194215, 25.12694429906025, 25.39182390810494, 25.660445350166512, 25.894092439253345, 26.557599322103982, 26.845415270714597, 27.09704519331435, 27.275456125949677, 27.856891731712217, 28.42734473945654, 28.539129903871054, 28.83784778791181, 28.895299783288884, 28.98998441991902, 29.14933724417837, 29.79941747251885, 30.276261499504287, 30.239707515030094, 30.226480207643032, 29.930764483951965, 29.922400727603254, 30.230356106414284, 30.13349430413013, 30.108542577283565, 30.017524295370073, 29.928540368364438, 29.422767066670335, 29.46356586036788, 29.758331622001432, 29.664618400468612, 29.365992325742457, 29.281569587840885, 29.26978479701086, 29.255132419128227, 29.459268160505168, 29.37292340159207, 29.18536820322534, 29.24900290065265, 29.138445975662602, 29.132336689742267, 29.0813039844127, 29.018687835017477, 29.21495791148737, 29.087756583957635, 29.210710654514056, 29.1446957031897, 29.12700128577839, 28.933026111938386, 29.055450903613433, 28.96744809362548, 29.01384820570142, 28.899632843500957, 28.64831616021023, 28.77189807702514, 28.956371365923683, 28.828598113256987, 28.8036352459875, 28.645150910245494, 28.573200220259043, 28.398213690429806, 28.27896152136268, 28.156087016291707, 27.973773103065035, 27.871860534892345, 27.688896216792706, 27.436994012275196, 27.336112294063025, 27.35353434403861, 27.252609663218152, 27.14660851919877, 27.279846092770928, 27.16912611520569, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 21.20332602435602, "exact_area100": 26.71438579045997}}, "old_normalized_endpoint_budget": 44}]}, {"expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "task": "construction", "small": 7.8873978710439285, "large": 22.361115727328066, "runs": [{"seed": 2026, "selection": {"best_epoch": 5, "epochs": 13, "alpha": 0.0, "validation_area100": 0.16317435507140857, "training_seconds": 4.358948125969619, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.578953742980957, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5649577379226685, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5484484434127808, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5429777503013611, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5375469326972961, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5260171890258789, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5085981488227844, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4859265089035034, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4749895930290222, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4576142132282257, "operator": "ge"}, {"budget": 0.11, "threshold": 0.4467303156852722, "operator": "ge"}, {"budget": 0.12, "threshold": 0.42141738533973694, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3979756236076355, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3706069588661194, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3503580689430237, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3222936987876892, "operator": "ge"}, {"budget": 0.17, "threshold": 0.312674343585968, "operator": "ge"}, {"budget": 0.18, "threshold": 0.30041828751564026, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2967943549156189, "operator": "ge"}, {"budget": 0.2, "threshold": 0.29219377040863037, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2879028916358948, "operator": "ge"}, {"budget": 0.22, "threshold": 0.2831959128379822, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2792527377605438, "operator": "ge"}, {"budget": 0.24, "threshold": 0.27468663454055786, "operator": "ge"}, {"budget": 0.25, "threshold": 0.27072596549987793, "operator": "ge"}, {"budget": 0.26, "threshold": 0.2685645520687103, "operator": "ge"}, {"budget": 0.27, "threshold": 0.2652846872806549, "operator": "ge"}, {"budget": 0.28, "threshold": 0.26321473717689514, "operator": "ge"}, {"budget": 0.29, "threshold": 0.25911760330200195, "operator": "ge"}, {"budget": 0.3, "threshold": 0.256024032831192, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2529665231704712, "operator": "ge"}, {"budget": 0.32, "threshold": 0.25040575861930847, "operator": "ge"}, {"budget": 0.33, "threshold": 0.24813129007816315, "operator": "ge"}, {"budget": 0.34, "threshold": 0.2459103763103485, "operator": "ge"}, {"budget": 0.35, "threshold": 0.2435876727104187, "operator": "ge"}, {"budget": 0.36, "threshold": 0.24123118817806244, "operator": "ge"}, {"budget": 0.37, "threshold": 0.23786038160324097, "operator": "ge"}, {"budget": 0.38, "threshold": 0.2347448170185089, "operator": "ge"}, {"budget": 0.39, "threshold": 0.23272322118282318, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2296583652496338, "operator": "ge"}, {"budget": 0.41, "threshold": 0.22556355595588684, "operator": "ge"}, {"budget": 0.42, "threshold": 0.22094401717185974, "operator": "ge"}, {"budget": 0.43, "threshold": 0.21668371558189392, "operator": "ge"}, {"budget": 0.44, "threshold": 0.2016891986131668, "operator": "ge"}, {"budget": 0.45, "threshold": 0.19432994723320007, "operator": "ge"}, {"budget": 0.46, "threshold": 0.17742015421390533, "operator": "ge"}, {"budget": 0.47, "threshold": 0.15503302216529846, "operator": "ge"}, {"budget": 0.48, "threshold": 0.13898782432079315, "operator": "ge"}, {"budget": 0.49, "threshold": 0.11880485713481903, "operator": "ge"}, {"budget": 0.5, "threshold": 0.1043073982000351, "operator": "ge"}, {"budget": 0.51, "threshold": 0.07747133076190948, "operator": "ge"}, {"budget": 0.52, "threshold": 0.06294746696949005, "operator": "ge"}, {"budget": 0.53, "threshold": 0.05759607255458832, "operator": "ge"}, {"budget": 0.54, "threshold": 0.05323518067598343, "operator": "ge"}, {"budget": 0.55, "threshold": 0.05223384499549866, "operator": "ge"}, {"budget": 0.56, "threshold": 0.04865512251853943, "operator": "ge"}, {"budget": 0.57, "threshold": 0.04474249482154846, "operator": "ge"}, {"budget": 0.58, "threshold": 0.0438164621591568, "operator": "ge"}, {"budget": 0.59, "threshold": 0.040483370423316956, "operator": "ge"}, {"budget": 0.6, "threshold": 0.0375652015209198, "operator": "ge"}, {"budget": 0.61, "threshold": 0.035870522260665894, "operator": "ge"}, {"budget": 0.62, "threshold": 0.033949896693229675, "operator": "ge"}, {"budget": 0.63, "threshold": 0.03084079921245575, "operator": "ge"}, {"budget": 0.64, "threshold": 0.02867540717124939, "operator": "ge"}, {"budget": 0.65, "threshold": 0.026674732565879822, "operator": "ge"}, {"budget": 0.66, "threshold": 0.023068934679031372, "operator": "ge"}, {"budget": 0.67, "threshold": 0.02046121656894684, "operator": "ge"}, {"budget": 0.68, "threshold": 0.017972871661186218, "operator": "ge"}, {"budget": 0.69, "threshold": 0.016247302293777466, "operator": "ge"}, {"budget": 0.7, "threshold": 0.013392716646194458, "operator": "ge"}, {"budget": 0.71, "threshold": 0.011500507593154907, "operator": "ge"}, {"budget": 0.72, "threshold": 0.003542110323905945, "operator": "ge"}, {"budget": 0.73, "threshold": 0.0023379921913146973, "operator": "ge"}, {"budget": 0.74, "threshold": -0.003750711679458618, "operator": "ge"}, {"budget": 0.75, "threshold": -0.011926263570785522, "operator": "ge"}, {"budget": 0.76, "threshold": -0.01648801565170288, "operator": "ge"}, {"budget": 0.77, "threshold": -0.04836222529411316, "operator": "ge"}, {"budget": 0.78, "threshold": -0.11503497511148453, "operator": "ge"}, {"budget": 0.79, "threshold": -0.12718862295150757, "operator": "ge"}, {"budget": 0.8, "threshold": -0.13217774033546448, "operator": "ge"}, {"budget": 0.81, "threshold": -0.1382945328950882, "operator": "ge"}, {"budget": 0.82, "threshold": -0.14318853616714478, "operator": "ge"}, {"budget": 0.83, "threshold": -0.15112662315368652, "operator": "ge"}, {"budget": 0.84, "threshold": -0.16144882142543793, "operator": "ge"}, {"budget": 0.85, "threshold": -0.1672518253326416, "operator": "ge"}, {"budget": 0.86, "threshold": -0.17598070204257965, "operator": "ge"}, {"budget": 0.87, "threshold": -0.1793127954006195, "operator": "ge"}, {"budget": 0.88, "threshold": -0.18589811027050018, "operator": "ge"}, {"budget": 0.89, "threshold": -0.19236180186271667, "operator": "ge"}, {"budget": 0.9, "threshold": -0.1959327906370163, "operator": "ge"}, {"budget": 0.91, "threshold": -0.20105597376823425, "operator": "ge"}, {"budget": 0.92, "threshold": -0.205694317817688, "operator": "ge"}, {"budget": 0.93, "threshold": -0.2099069058895111, "operator": "ge"}, {"budget": 0.94, "threshold": -0.21267203986644745, "operator": "ge"}, {"budget": 0.95, "threshold": -0.2162652611732483, "operator": "ge"}, {"budget": 0.96, "threshold": -0.21916747093200684, "operator": "ge"}, {"budget": 0.97, "threshold": -0.22159728407859802, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2250598967075348, "operator": "ge"}, {"budget": 0.99, "threshold": -0.23190198838710785, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 10.692401330971284, 10.038668526804887, 9.706661602896387, 9.530192066325638, 9.44813497688483, 9.365555715865916, 9.385928026015828, 9.215979209309975, 9.26541023691032, 9.372384206822474, 9.472135527518365, 9.48827210076541, 9.338990395250352, 9.92524555024555, 10.670555737957471, 10.70700277422396, 10.798179770533377, 11.981064634368229, 12.052527151211361, 12.606905289832119, 12.658941528813491, 12.63757929557614, 12.675207171024406, 12.724988807992272, 12.72036099402643, 12.728428466813424, 12.816512230844642, 12.882810241349823, 12.851028249548074, 13.352531604338939, 13.851710307667886, 13.960366417331475, 14.466406741959842, 14.57882084737191, 15.072736532051131, 15.19843212095347, 15.331035651361477, 15.473264233647349, 15.639745690741897, 15.594570954159991, 15.652442795299937, 15.826129159462493, 15.840288897737786, 16.08145570432098, 16.35448347435029, 16.536834143143007, 16.593454745923818, 16.416093017874704, 16.405297605733764, 16.159940425445544, 16.160008895515674, 16.366610886904713, 16.55551519426569, 16.75037718120977, 16.87713358070501, 17.057478120241715, 17.177576598487686, 17.13392279334754, 17.315203159957875, 17.573224942150773, 17.72673186654018, 17.962287200043583, 17.8896999662991, 18.179153571028355, 18.581834470388113, 18.59515880839605, 18.86527155269527, 19.07778197412344, 19.40394607906341, 19.93031570406634, 20.41235536270583, 20.529063637870966, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.888546869331478, 21.919506151400395, 21.893448502718726, 21.880083092400632, 21.84351278855813, 21.814686618124686, 21.776514553540807, 21.750809318616877, 21.714046781415853, 21.674162399864695, 21.649903583222564, 21.43052721097708, 21.310625175217268, 21.26368578880485, 21.2401181245122, 21.060306223878946, 21.01611221137199, 20.97775619958726, 20.93941382381252, 20.823727652839175, 20.96151507315106, 20.921163363964894, 20.866595027535237, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 41, 61, 77, 97, 113, 131, 145, 165, 182, 196, 209, 224, 238, 253, 266, 277, 288, 298, 310, 322, 331, 343, 355, 367, 381, 393, 404, 413, 426, 441, 452, 463, 475, 484, 496, 506, 516, 528, 538, 546, 555, 564, 576, 588, 598, 608, 613, 619, 622, 628, 635, 640, 647, 653, 660, 666, 671, 677, 684, 691, 698, 701, 707, 714, 720, 727, 733, 740, 747, 752, 758, 764, 771, 775, 781, 783, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786], "harmed": [0, 3, 9, 15, 22, 28, 33, 37, 41, 46, 50, 51, 54, 61, 66, 69, 73, 74, 77, 78, 82, 84, 88, 93, 96, 102, 109, 112, 116, 120, 124, 129, 131, 133, 136, 137, 141, 143, 150, 155, 161, 163, 166, 171, 176, 181, 183, 186, 191, 198, 201, 204, 207, 210, 214, 219, 224, 230, 238, 242, 246, 252, 258, 265, 269, 276, 283, 288, 294, 301, 306, 310, 317, 323, 329, 335, 340, 346, 352, 355, 357, 363, 367, 376, 379, 384, 389, 392, 398, 404, 411, 415, 420, 427, 433, 440, 447, 452, 458, 466, 475], "exact_score": [7.641657905799261, 10.692401330971284, 10.038668526804887, 9.706661602896387, 9.530192066325638, 9.44813497688483, 9.365555715865916, 9.385928026015828, 9.215979209309975, 9.26541023691032, 9.372384206822474, 9.472135527518365, 9.48827210076541, 9.338990395250352, 9.92524555024555, 10.670555737957471, 10.70700277422396, 10.798179770533377, 11.981064634368229, 12.052527151211361, 12.606905289832119, 12.658941528813491, 12.63757929557614, 12.675207171024406, 12.724988807992272, 12.72036099402643, 12.728428466813424, 12.816512230844642, 12.882810241349823, 12.851028249548074, 13.352531604338939, 13.851710307667886, 13.960366417331475, 14.466406741959842, 14.57882084737191, 15.072736532051131, 15.19843212095347, 15.331035651361477, 15.473264233647349, 15.639745690741897, 15.594570954159991, 15.652442795299937, 15.826129159462493, 15.840288897737786, 16.08145570432098, 16.35448347435029, 16.536834143143007, 16.593454745923818, 16.416093017874704, 16.405297605733764, 16.159940425445544, 16.160008895515674, 16.366610886904713, 16.55551519426569, 16.75037718120977, 16.87713358070501, 17.057478120241715, 17.177576598487686, 17.13392279334754, 17.315203159957875, 17.573224942150773, 17.72673186654018, 17.962287200043583, 17.8896999662991, 18.179153571028355, 18.581834470388113, 18.59515880839605, 18.86527155269527, 19.07778197412344, 19.40394607906341, 19.93031570406634, 20.41235536270583, 20.529063637870966, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.888546869331478, 21.919506151400395, 21.893448502718726, 21.880083092400632, 21.84351278855813, 21.814686618124686, 21.776514553540807, 21.750809318616877, 21.714046781415853, 21.674162399864695, 21.649903583222564, 21.43052721097708, 21.310625175217268, 21.26368578880485, 21.2401181245122, 21.060306223878946, 21.01611221137199, 20.97775619958726, 20.93941382381252, 20.823727652839175, 20.96151507315106, 20.921163363964894, 20.866595027535237, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.317435507140857, "threshold_area100": 16.311079977943084, "threshold_area25": 10.455797932281948}, "test": {"score": [7.8873978710439285, 7.938582130219832, 8.604854115685127, 9.503107553953265, 10.279830127076144, 10.144309210955575, 10.628218173975938, 10.96399402303937, 10.806535197101043, 10.762677414146514, 11.209483336889363, 11.443832388188078, 11.386817223907993, 11.16398806115688, 11.023550882640368, 10.989925141268447, 11.124521682098901, 11.059530804175862, 11.022660729955449, 11.17656415174768, 11.230508774647543, 11.335696990164907, 11.874952085430415, 12.375236912300792, 12.50772579358577, 12.596705094520166, 12.690064648502936, 12.814514009447317, 12.91797611527957, 13.122183474467594, 13.346629052862918, 13.473266379179737, 13.516070244243341, 13.727676747489806, 13.866816953721232, 14.00011212427546, 14.065400890300007, 14.17521879535838, 14.370066416168031, 14.452907292962806, 14.577790501057395, 14.917887271520247, 15.071407350539182, 15.234798903742707, 15.491502671378477, 15.682606071645544, 15.80024412333792, 15.813316445096696, 15.615484169967054, 15.347458950683174, 15.228522451523855, 15.112308599206395, 15.34817683125107, 15.540771303425839, 15.719130512515466, 15.767132219804067, 15.99953981542824, 16.209356803568628, 16.23811940819816, 16.477515204845716, 16.824617434670113, 16.7053995984643, 16.597085045571117, 16.86697158237546, 17.26406772130991, 17.57186476162452, 18.46144460959927, 18.73527242554262, 19.071458947291575, 19.414148519906256, 19.781348791202362, 20.06745835588736, 20.221858888113104, 20.42465775114531, 21.542829896144216, 21.94691336506925, 22.151919702262795, 22.853583435826284, 23.04787640979075, 22.968085689042308, 22.942334005076958, 22.87911892522202, 22.8925318934632, 23.07271660897012, 22.87195990709154, 22.849682266652724, 22.723983140884034, 22.717592021146967, 22.639929072684254, 22.616302510549815, 22.553837477061776, 22.52634081944332, 22.527528629365662, 22.45890573840617, 22.460338995963273, 22.45146187645203, 22.39080675786611, 22.40400292526237, 22.37379772859531, 22.392198864273936, 22.361115727328066], "calls": [0.0, 0.4660452729693742, 1.2649800266311584, 2.1637816245006656, 2.5299600532623168, 3.029294274300932, 3.9946737683089215, 5.226364846870839, 6.491344873501998, 6.957390146471372, 7.8561917443408795, 8.222370173102531, 8.921438082556591, 9.820239680426099, 10.818908122503329, 11.584553928095872, 12.616511318242344, 13.282290279627162, 14.81358189081225, 15.645805592543274, 16.378162450066576, 17.41011984021305, 18.708388814913448, 19.940079893475364, 21.438082556591212, 22.836218375499335, 23.735019973368843, 24.93342210386152, 25.63249001331558, 27.496671105193077, 29.360852197070574, 30.492676431424766, 31.657789613848202, 32.856191744340876, 33.75499334221039, 34.720372836218374, 36.085219707057256, 37.017310252996005, 38.1491344873502, 38.78162450066578, 39.513981358189085, 40.51264980026631, 41.344873501997334, 42.07723035952064, 44.40745672436751, 45.13981358189081, 46.504660452729695, 47.76964047936085, 49.03462050599201, 50.36617842876166, 50.96537949400799, 51.73102529960053, 53.26231691078562, 54.460719041278296, 55.625832223701735, 55.9254327563249, 57.05725699067909, 58.32223701731025, 58.655126498002666, 59.85352862849535, 61.051930758988014, 61.884154460719046, 62.61651131824234, 63.914780292942744, 65.04660452729694, 66.21171770972038, 67.37683089214381, 68.40878828229027, 69.2410119840213, 70.20639147802929, 71.20505992010652, 71.87083888149135, 73.40213049267643, 73.76830892143809, 75.09986684420772, 76.2982689747004, 76.83089214380826, 78.19573901464713, 79.32756324900133, 80.3595206391478, 80.95872170439414, 81.59121171770973, 82.29027962716378, 83.5219707057257, 85.38615179760319, 85.98535286284952, 86.81757656458056, 87.01731025299601, 87.71637816245007, 89.01464713715048, 89.54727030625833, 91.24500665778962, 92.41011984021304, 93.74167776298269, 94.74034620505992, 95.97203728362183, 96.6711051930759, 97.27030625832224, 98.00266311584554, 99.20106524633822, 100.0], "rescued": [0, 39, 104, 178, 209, 248, 330, 427, 519, 555, 621, 649, 689, 739, 795, 835, 889, 921, 997, 1042, 1080, 1123, 1187, 1252, 1322, 1388, 1428, 1486, 1518, 1608, 1691, 1745, 1799, 1855, 1896, 1941, 1996, 2039, 2092, 2121, 2157, 2202, 2238, 2276, 2379, 2414, 2462, 2500, 2531, 2554, 2564, 2577, 2618, 2649, 2682, 2691, 2722, 2758, 2767, 2801, 2835, 2856, 2876, 2913, 2945, 2976, 3009, 3033, 3056, 3083, 3110, 3129, 3169, 3179, 3214, 3243, 3258, 3294, 3304, 3304, 3304, 3304, 3305, 3306, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3308, 3309, 3309, 3310, 3311, 3311, 3312, 3312, 3314, 3314], "harmed": [0, 7, 22, 42, 48, 59, 80, 103, 125, 133, 144, 147, 155, 174, 190, 200, 215, 228, 250, 266, 280, 301, 325, 346, 367, 392, 408, 427, 436, 468, 500, 521, 541, 557, 574, 588, 611, 628, 646, 657, 671, 685, 702, 719, 761, 775, 799, 817, 836, 854, 864, 877, 896, 917, 937, 943, 964, 990, 999, 1024, 1046, 1066, 1084, 1122, 1143, 1167, 1190, 1221, 1243, 1264, 1294, 1308, 1349, 1358, 1389, 1412, 1424, 1455, 1470, 1485, 1498, 1505, 1521, 1547, 1590, 1602, 1620, 1624, 1642, 1671, 1681, 1723, 1750, 1782, 1805, 1835, 1852, 1868, 1892, 1926, 1945], "exact_score": [7.8873978710439285, 8.613048712068878, 9.541242879608937, 10.15604085862786, 10.628218173975938, 10.998571380877701, 10.893674404366505, 10.756245210336905, 11.213335347925648, 11.298364887587041, 11.17660714931372, 11.028373350894578, 10.932839440738263, 11.088776205283777, 11.078969658106349, 11.014369381986064, 11.29116061464605, 11.228996423790067, 11.463328120546603, 11.868349346765266, 12.3796739204838, 12.519645804537035, 12.471982172378684, 12.719430193639711, 12.697000609704789, 12.811933864720304, 12.908258961253225, 13.116444028424251, 13.202688875991456, 13.353353252828063, 13.455254002346576, 13.514896801424708, 13.55812336160786, 13.73699983032969, 13.884845908854269, 14.018394023278494, 14.054454978717363, 14.17521879535838, 14.350271409122886, 14.487192882002844, 14.721589922068615, 15.00891361897925, 15.208699071026258, 15.40549229377858, 15.641805321239088, 15.65551412661453, 15.834678565425014, 15.767682062616359, 15.775310380499713, 15.614590943306297, 15.407245613011336, 15.228522451523855, 15.113822990899955, 15.287625540864191, 15.471849693094692, 15.619743409718428, 15.772616501344746, 15.978981750565898, 16.191338541505736, 16.308014367455858, 16.507928213763115, 16.78943399904086, 16.732562028394348, 16.696545386105697, 16.8941331818875, 17.246163211040802, 17.499546607088927, 18.36594352852505, 18.602318047681056, 18.92565246724082, 19.321298695186893, 19.72504867133783, 20.108346185773893, 20.046764730007258, 20.851285539493563, 21.53086718976187, 21.86178053447084, 22.260170006601125, 22.755023949481725, 23.053637998519264, 22.978255683501907, 22.942334005076958, 22.863179997204533, 23.154822016625495, 23.036384210871336, 22.934580838844713, 22.849682266652724, 22.719676573746916, 22.631647071798742, 22.617653704293932, 22.59940851904116, 22.53695367976944, 22.545836951050617, 22.503630772264312, 22.448201473092848, 22.43622195943248, 22.45146187645203, 22.416584230386537, 22.374987314619375, 22.382223856943504, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.392316751844053, "threshold_area100": 16.352742706462827, "threshold_area25": 10.763255589308596}, "selected_budget": 76, "selected": {"val": {"score": 22.072697083038882, "calls": 75.89158345221114, "rescued": 781, "harmed": 340}, "test": {"score": 22.151919702262795, "calls": 76.83089214380826, "rescued": 3258, "harmed": 1424}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.578953742980957, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5649577379226685, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5484484434127808, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5429777503013611, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5375469326972961, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5260171890258789, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5085981488227844, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4859265089035034, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4749895930290222, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4576142132282257, "operator": "ge"}, {"budget": 0.11, "threshold": 0.4467303156852722, "operator": "ge"}, {"budget": 0.12, "threshold": 0.42141738533973694, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3979756236076355, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3706069588661194, "operator": "ge"}, {"budget": 0.15, "threshold": 0.3503580689430237, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3222936987876892, "operator": "ge"}, {"budget": 0.17, "threshold": 0.312674343585968, "operator": "ge"}, {"budget": 0.18, "threshold": 0.30041828751564026, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2967943549156189, "operator": "ge"}, {"budget": 0.2, "threshold": 0.29219377040863037, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2879028916358948, "operator": "ge"}, {"budget": 0.22, "threshold": 0.2831959128379822, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2792527377605438, "operator": "ge"}, {"budget": 0.24, "threshold": 0.27468663454055786, "operator": "ge"}, {"budget": 0.25, "threshold": 0.27072596549987793, "operator": "ge"}, {"budget": 0.26, "threshold": 0.2685645520687103, "operator": "ge"}, {"budget": 0.27, "threshold": 0.2652846872806549, "operator": "ge"}, {"budget": 0.28, "threshold": 0.26321473717689514, "operator": "ge"}, {"budget": 0.29, "threshold": 0.25911760330200195, "operator": "ge"}, {"budget": 0.3, "threshold": 0.256024032831192, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2529665231704712, "operator": "ge"}, {"budget": 0.32, "threshold": 0.25040575861930847, "operator": "ge"}, {"budget": 0.33, "threshold": 0.24813129007816315, "operator": "ge"}, {"budget": 0.34, "threshold": 0.2459103763103485, "operator": "ge"}, {"budget": 0.35, "threshold": 0.2435876727104187, "operator": "ge"}, {"budget": 0.36, "threshold": 0.24123118817806244, "operator": "ge"}, {"budget": 0.37, "threshold": 0.23786038160324097, "operator": "ge"}, {"budget": 0.38, "threshold": 0.2347448170185089, "operator": "ge"}, {"budget": 0.39, "threshold": 0.23272322118282318, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2296583652496338, "operator": "ge"}, {"budget": 0.41, "threshold": 0.22556355595588684, "operator": "ge"}, {"budget": 0.42, "threshold": 0.22094401717185974, "operator": "ge"}, {"budget": 0.43, "threshold": 0.21668371558189392, "operator": "ge"}, {"budget": 0.44, "threshold": 0.2016891986131668, "operator": "ge"}, {"budget": 0.45, "threshold": 0.19432994723320007, "operator": "ge"}, {"budget": 0.46, "threshold": 0.17742015421390533, "operator": "ge"}, {"budget": 0.47, "threshold": 0.15503302216529846, "operator": "ge"}, {"budget": 0.48, "threshold": 0.13898782432079315, "operator": "ge"}, {"budget": 0.49, "threshold": 0.11880485713481903, "operator": "ge"}, {"budget": 0.5, "threshold": 0.1043073982000351, "operator": "ge"}, {"budget": 0.51, "threshold": 0.07747133076190948, "operator": "ge"}, {"budget": 0.52, "threshold": 0.06294746696949005, "operator": "ge"}, {"budget": 0.53, "threshold": 0.05759607255458832, "operator": "ge"}, {"budget": 0.54, "threshold": 0.05323518067598343, "operator": "ge"}, {"budget": 0.55, "threshold": 0.05223384499549866, "operator": "ge"}, {"budget": 0.56, "threshold": 0.04865512251853943, "operator": "ge"}, {"budget": 0.57, "threshold": 0.04474249482154846, "operator": "ge"}, {"budget": 0.58, "threshold": 0.0438164621591568, "operator": "ge"}, {"budget": 0.59, "threshold": 0.040483370423316956, "operator": "ge"}, {"budget": 0.6, "threshold": 0.0375652015209198, "operator": "ge"}, {"budget": 0.61, "threshold": 0.035870522260665894, "operator": "ge"}, {"budget": 0.62, "threshold": 0.033949896693229675, "operator": "ge"}, {"budget": 0.63, "threshold": 0.03084079921245575, "operator": "ge"}, {"budget": 0.64, "threshold": 0.02867540717124939, "operator": "ge"}, {"budget": 0.65, "threshold": 0.026674732565879822, "operator": "ge"}, {"budget": 0.66, "threshold": 0.023068934679031372, "operator": "ge"}, {"budget": 0.67, "threshold": 0.02046121656894684, "operator": "ge"}, {"budget": 0.68, "threshold": 0.017972871661186218, "operator": "ge"}, {"budget": 0.69, "threshold": 0.016247302293777466, "operator": "ge"}, {"budget": 0.7, "threshold": 0.013392716646194458, "operator": "ge"}, {"budget": 0.71, "threshold": 0.011500507593154907, "operator": "ge"}, {"budget": 0.72, "threshold": 0.003542110323905945, "operator": "ge"}, {"budget": 0.73, "threshold": 0.0023379921913146973, "operator": "ge"}, {"budget": 0.74, "threshold": -0.003750711679458618, "operator": "ge"}, {"budget": 0.75, "threshold": -0.011926263570785522, "operator": "ge"}, {"budget": 0.76, "threshold": -0.01648801565170288, "operator": "ge"}, {"budget": 0.77, "threshold": -0.04836222529411316, "operator": "ge"}, {"budget": 0.78, "threshold": -0.11503497511148453, "operator": "ge"}, {"budget": 0.79, "threshold": -0.12718862295150757, "operator": "ge"}, {"budget": 0.8, "threshold": -0.13217774033546448, "operator": "ge"}, {"budget": 0.81, "threshold": -0.1382945328950882, "operator": "ge"}, {"budget": 0.82, "threshold": -0.14318853616714478, "operator": "ge"}, {"budget": 0.83, "threshold": -0.15112662315368652, "operator": "ge"}, {"budget": 0.84, "threshold": -0.16144882142543793, "operator": "ge"}, {"budget": 0.85, "threshold": -0.1672518253326416, "operator": "ge"}, {"budget": 0.86, "threshold": -0.17598070204257965, "operator": "ge"}, {"budget": 0.87, "threshold": -0.1793127954006195, "operator": "ge"}, {"budget": 0.88, "threshold": -0.18589811027050018, "operator": "ge"}, {"budget": 0.89, "threshold": -0.19236180186271667, "operator": "ge"}, {"budget": 0.9, "threshold": -0.1959327906370163, "operator": "ge"}, {"budget": 0.91, "threshold": -0.20105597376823425, "operator": "ge"}, {"budget": 0.92, "threshold": -0.205694317817688, "operator": "ge"}, {"budget": 0.93, "threshold": -0.2099069058895111, "operator": "ge"}, {"budget": 0.94, "threshold": -0.21267203986644745, "operator": "ge"}, {"budget": 0.95, "threshold": -0.2162652611732483, "operator": "ge"}, {"budget": 0.96, "threshold": -0.21916747093200684, "operator": "ge"}, {"budget": 0.97, "threshold": -0.22159728407859802, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2250598967075348, "operator": "ge"}, {"budget": 0.99, "threshold": -0.23190198838710785, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 10.692401330971284, 10.038668526804887, 9.706661602896387, 9.530192066325638, 9.44813497688483, 9.365555715865916, 9.385928026015828, 9.215979209309975, 9.26541023691032, 9.372384206822474, 9.472135527518365, 9.48827210076541, 9.338990395250352, 9.92524555024555, 10.670555737957471, 10.70700277422396, 10.798179770533377, 11.981064634368229, 12.052527151211361, 12.606905289832119, 12.658941528813491, 12.63757929557614, 12.675207171024406, 12.724988807992272, 12.72036099402643, 12.728428466813424, 12.816512230844642, 12.882810241349823, 12.851028249548074, 13.352531604338939, 13.851710307667886, 13.960366417331475, 14.466406741959842, 14.57882084737191, 15.072736532051131, 15.19843212095347, 15.331035651361477, 15.473264233647349, 15.639745690741897, 15.594570954159991, 15.652442795299937, 15.826129159462493, 15.840288897737786, 16.08145570432098, 16.35448347435029, 16.536834143143007, 16.593454745923818, 16.416093017874704, 16.405297605733764, 16.159940425445544, 16.160008895515674, 16.366610886904713, 16.55551519426569, 16.75037718120977, 16.87713358070501, 17.057478120241715, 17.177576598487686, 17.13392279334754, 17.315203159957875, 17.573224942150773, 17.72673186654018, 17.962287200043583, 17.8896999662991, 18.179153571028355, 18.581834470388113, 18.59515880839605, 18.86527155269527, 19.07778197412344, 19.40394607906341, 19.93031570406634, 20.41235536270583, 20.529063637870966, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.888546869331478, 21.919506151400395, 21.893448502718726, 21.880083092400632, 21.84351278855813, 21.814686618124686, 21.776514553540807, 21.750809318616877, 21.714046781415853, 21.674162399864695, 21.649903583222564, 21.43052721097708, 21.310625175217268, 21.26368578880485, 21.2401181245122, 21.060306223878946, 21.01611221137199, 20.97775619958726, 20.93941382381252, 20.823727652839175, 20.96151507315106, 20.921163363964894, 20.866595027535237, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 41, 61, 77, 97, 113, 131, 145, 165, 182, 196, 209, 224, 238, 253, 266, 277, 288, 298, 310, 322, 331, 343, 355, 367, 381, 393, 404, 413, 426, 441, 452, 463, 475, 484, 496, 506, 516, 528, 538, 546, 555, 564, 576, 588, 598, 608, 613, 619, 622, 628, 635, 640, 647, 653, 660, 666, 671, 677, 684, 691, 698, 701, 707, 714, 720, 727, 733, 740, 747, 752, 758, 764, 771, 775, 781, 783, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786], "harmed": [0, 3, 9, 15, 22, 28, 33, 37, 41, 46, 50, 51, 54, 61, 66, 69, 73, 74, 77, 78, 82, 84, 88, 93, 96, 102, 109, 112, 116, 120, 124, 129, 131, 133, 136, 137, 141, 143, 150, 155, 161, 163, 166, 171, 176, 181, 183, 186, 191, 198, 201, 204, 207, 210, 214, 219, 224, 230, 238, 242, 246, 252, 258, 265, 269, 276, 283, 288, 294, 301, 306, 310, 317, 323, 329, 335, 340, 346, 352, 355, 357, 363, 367, 376, 379, 384, 389, 392, 398, 404, 411, 415, 420, 427, 433, 440, 447, 452, 458, 466, 475], "exact_score": [7.641657905799261, 10.692401330971284, 10.038668526804887, 9.706661602896387, 9.530192066325638, 9.44813497688483, 9.365555715865916, 9.385928026015828, 9.215979209309975, 9.26541023691032, 9.372384206822474, 9.472135527518365, 9.48827210076541, 9.338990395250352, 9.92524555024555, 10.670555737957471, 10.70700277422396, 10.798179770533377, 11.981064634368229, 12.052527151211361, 12.606905289832119, 12.658941528813491, 12.63757929557614, 12.675207171024406, 12.724988807992272, 12.72036099402643, 12.728428466813424, 12.816512230844642, 12.882810241349823, 12.851028249548074, 13.352531604338939, 13.851710307667886, 13.960366417331475, 14.466406741959842, 14.57882084737191, 15.072736532051131, 15.19843212095347, 15.331035651361477, 15.473264233647349, 15.639745690741897, 15.594570954159991, 15.652442795299937, 15.826129159462493, 15.840288897737786, 16.08145570432098, 16.35448347435029, 16.536834143143007, 16.593454745923818, 16.416093017874704, 16.405297605733764, 16.159940425445544, 16.160008895515674, 16.366610886904713, 16.55551519426569, 16.75037718120977, 16.87713358070501, 17.057478120241715, 17.177576598487686, 17.13392279334754, 17.315203159957875, 17.573224942150773, 17.72673186654018, 17.962287200043583, 17.8896999662991, 18.179153571028355, 18.581834470388113, 18.59515880839605, 18.86527155269527, 19.07778197412344, 19.40394607906341, 19.93031570406634, 20.41235536270583, 20.529063637870966, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.888546869331478, 21.919506151400395, 21.893448502718726, 21.880083092400632, 21.84351278855813, 21.814686618124686, 21.776514553540807, 21.750809318616877, 21.714046781415853, 21.674162399864695, 21.649903583222564, 21.43052721097708, 21.310625175217268, 21.26368578880485, 21.2401181245122, 21.060306223878946, 21.01611221137199, 20.97775619958726, 20.93941382381252, 20.823727652839175, 20.96151507315106, 20.921163363964894, 20.866595027535237, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.317435507140857, "threshold_area100": 16.311079977943084, "threshold_area25": 10.455797932281948}, "test": {"score": [7.8873978710439285, 7.938582130219832, 8.604854115685127, 9.503107553953265, 10.279830127076144, 10.144309210955575, 10.628218173975938, 10.96399402303937, 10.806535197101043, 10.762677414146514, 11.209483336889363, 11.443832388188078, 11.386817223907993, 11.16398806115688, 11.023550882640368, 10.989925141268447, 11.124521682098901, 11.059530804175862, 11.022660729955449, 11.17656415174768, 11.230508774647543, 11.335696990164907, 11.874952085430415, 12.375236912300792, 12.50772579358577, 12.596705094520166, 12.690064648502936, 12.814514009447317, 12.91797611527957, 13.122183474467594, 13.346629052862918, 13.473266379179737, 13.516070244243341, 13.727676747489806, 13.866816953721232, 14.00011212427546, 14.065400890300007, 14.17521879535838, 14.370066416168031, 14.452907292962806, 14.577790501057395, 14.917887271520247, 15.071407350539182, 15.234798903742707, 15.491502671378477, 15.682606071645544, 15.80024412333792, 15.813316445096696, 15.615484169967054, 15.347458950683174, 15.228522451523855, 15.112308599206395, 15.34817683125107, 15.540771303425839, 15.719130512515466, 15.767132219804067, 15.99953981542824, 16.209356803568628, 16.23811940819816, 16.477515204845716, 16.824617434670113, 16.7053995984643, 16.597085045571117, 16.86697158237546, 17.26406772130991, 17.57186476162452, 18.46144460959927, 18.73527242554262, 19.071458947291575, 19.414148519906256, 19.781348791202362, 20.06745835588736, 20.221858888113104, 20.42465775114531, 21.542829896144216, 21.94691336506925, 22.151919702262795, 22.853583435826284, 23.04787640979075, 22.968085689042308, 22.942334005076958, 22.87911892522202, 22.8925318934632, 23.07271660897012, 22.87195990709154, 22.849682266652724, 22.723983140884034, 22.717592021146967, 22.639929072684254, 22.616302510549815, 22.553837477061776, 22.52634081944332, 22.527528629365662, 22.45890573840617, 22.460338995963273, 22.45146187645203, 22.39080675786611, 22.40400292526237, 22.37379772859531, 22.392198864273936, 22.361115727328066], "calls": [0.0, 0.4660452729693742, 1.2649800266311584, 2.1637816245006656, 2.5299600532623168, 3.029294274300932, 3.9946737683089215, 5.226364846870839, 6.491344873501998, 6.957390146471372, 7.8561917443408795, 8.222370173102531, 8.921438082556591, 9.820239680426099, 10.818908122503329, 11.584553928095872, 12.616511318242344, 13.282290279627162, 14.81358189081225, 15.645805592543274, 16.378162450066576, 17.41011984021305, 18.708388814913448, 19.940079893475364, 21.438082556591212, 22.836218375499335, 23.735019973368843, 24.93342210386152, 25.63249001331558, 27.496671105193077, 29.360852197070574, 30.492676431424766, 31.657789613848202, 32.856191744340876, 33.75499334221039, 34.720372836218374, 36.085219707057256, 37.017310252996005, 38.1491344873502, 38.78162450066578, 39.513981358189085, 40.51264980026631, 41.344873501997334, 42.07723035952064, 44.40745672436751, 45.13981358189081, 46.504660452729695, 47.76964047936085, 49.03462050599201, 50.36617842876166, 50.96537949400799, 51.73102529960053, 53.26231691078562, 54.460719041278296, 55.625832223701735, 55.9254327563249, 57.05725699067909, 58.32223701731025, 58.655126498002666, 59.85352862849535, 61.051930758988014, 61.884154460719046, 62.61651131824234, 63.914780292942744, 65.04660452729694, 66.21171770972038, 67.37683089214381, 68.40878828229027, 69.2410119840213, 70.20639147802929, 71.20505992010652, 71.87083888149135, 73.40213049267643, 73.76830892143809, 75.09986684420772, 76.2982689747004, 76.83089214380826, 78.19573901464713, 79.32756324900133, 80.3595206391478, 80.95872170439414, 81.59121171770973, 82.29027962716378, 83.5219707057257, 85.38615179760319, 85.98535286284952, 86.81757656458056, 87.01731025299601, 87.71637816245007, 89.01464713715048, 89.54727030625833, 91.24500665778962, 92.41011984021304, 93.74167776298269, 94.74034620505992, 95.97203728362183, 96.6711051930759, 97.27030625832224, 98.00266311584554, 99.20106524633822, 100.0], "rescued": [0, 39, 104, 178, 209, 248, 330, 427, 519, 555, 621, 649, 689, 739, 795, 835, 889, 921, 997, 1042, 1080, 1123, 1187, 1252, 1322, 1388, 1428, 1486, 1518, 1608, 1691, 1745, 1799, 1855, 1896, 1941, 1996, 2039, 2092, 2121, 2157, 2202, 2238, 2276, 2379, 2414, 2462, 2500, 2531, 2554, 2564, 2577, 2618, 2649, 2682, 2691, 2722, 2758, 2767, 2801, 2835, 2856, 2876, 2913, 2945, 2976, 3009, 3033, 3056, 3083, 3110, 3129, 3169, 3179, 3214, 3243, 3258, 3294, 3304, 3304, 3304, 3304, 3305, 3306, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3308, 3309, 3309, 3310, 3311, 3311, 3312, 3312, 3314, 3314], "harmed": [0, 7, 22, 42, 48, 59, 80, 103, 125, 133, 144, 147, 155, 174, 190, 200, 215, 228, 250, 266, 280, 301, 325, 346, 367, 392, 408, 427, 436, 468, 500, 521, 541, 557, 574, 588, 611, 628, 646, 657, 671, 685, 702, 719, 761, 775, 799, 817, 836, 854, 864, 877, 896, 917, 937, 943, 964, 990, 999, 1024, 1046, 1066, 1084, 1122, 1143, 1167, 1190, 1221, 1243, 1264, 1294, 1308, 1349, 1358, 1389, 1412, 1424, 1455, 1470, 1485, 1498, 1505, 1521, 1547, 1590, 1602, 1620, 1624, 1642, 1671, 1681, 1723, 1750, 1782, 1805, 1835, 1852, 1868, 1892, 1926, 1945], "exact_score": [7.8873978710439285, 8.613048712068878, 9.541242879608937, 10.15604085862786, 10.628218173975938, 10.998571380877701, 10.893674404366505, 10.756245210336905, 11.213335347925648, 11.298364887587041, 11.17660714931372, 11.028373350894578, 10.932839440738263, 11.088776205283777, 11.078969658106349, 11.014369381986064, 11.29116061464605, 11.228996423790067, 11.463328120546603, 11.868349346765266, 12.3796739204838, 12.519645804537035, 12.471982172378684, 12.719430193639711, 12.697000609704789, 12.811933864720304, 12.908258961253225, 13.116444028424251, 13.202688875991456, 13.353353252828063, 13.455254002346576, 13.514896801424708, 13.55812336160786, 13.73699983032969, 13.884845908854269, 14.018394023278494, 14.054454978717363, 14.17521879535838, 14.350271409122886, 14.487192882002844, 14.721589922068615, 15.00891361897925, 15.208699071026258, 15.40549229377858, 15.641805321239088, 15.65551412661453, 15.834678565425014, 15.767682062616359, 15.775310380499713, 15.614590943306297, 15.407245613011336, 15.228522451523855, 15.113822990899955, 15.287625540864191, 15.471849693094692, 15.619743409718428, 15.772616501344746, 15.978981750565898, 16.191338541505736, 16.308014367455858, 16.507928213763115, 16.78943399904086, 16.732562028394348, 16.696545386105697, 16.8941331818875, 17.246163211040802, 17.499546607088927, 18.36594352852505, 18.602318047681056, 18.92565246724082, 19.321298695186893, 19.72504867133783, 20.108346185773893, 20.046764730007258, 20.851285539493563, 21.53086718976187, 21.86178053447084, 22.260170006601125, 22.755023949481725, 23.053637998519264, 22.978255683501907, 22.942334005076958, 22.863179997204533, 23.154822016625495, 23.036384210871336, 22.934580838844713, 22.849682266652724, 22.719676573746916, 22.631647071798742, 22.617653704293932, 22.59940851904116, 22.53695367976944, 22.545836951050617, 22.503630772264312, 22.448201473092848, 22.43622195943248, 22.45146187645203, 22.416584230386537, 22.374987314619375, 22.382223856943504, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.392316751844053, "threshold_area100": 16.352742706462827, "threshold_area25": 10.763255589308596}, "selected_budget": 76, "selected": {"val": {"score": 22.072697083038882, "calls": 75.89158345221114, "rescued": 781, "harmed": 340}, "test": {"score": 22.151919702262795, "calls": 76.83089214380826, "rescued": 3258, "harmed": 1424}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.0277857780456543, "operator": "ge"}, {"budget": 0.02, "threshold": 1.8846635818481445, "operator": "ge"}, {"budget": 0.03, "threshold": 1.8035449981689453, "operator": "ge"}, {"budget": 0.04, "threshold": 1.7487624883651733, "operator": "ge"}, {"budget": 0.05, "threshold": 1.7097535133361816, "operator": "ge"}, {"budget": 0.06, "threshold": 1.6794633865356445, "operator": "ge"}, {"budget": 0.07, "threshold": 1.644155740737915, "operator": "ge"}, {"budget": 0.08, "threshold": 1.6058008670806885, "operator": "ge"}, {"budget": 0.09, "threshold": 1.5863760709762573, "operator": "ge"}, {"budget": 0.1, "threshold": 1.5704386234283447, "operator": "ge"}, {"budget": 0.11, "threshold": 1.5294456481933594, "operator": "ge"}, {"budget": 0.12, "threshold": 1.4795693159103394, "operator": "ge"}, {"budget": 0.13, "threshold": 1.4439030885696411, "operator": "ge"}, {"budget": 0.14, "threshold": 1.4249083995819092, "operator": "ge"}, {"budget": 0.15, "threshold": 1.4028198719024658, "operator": "ge"}, {"budget": 0.16, "threshold": 1.3743493556976318, "operator": "ge"}, {"budget": 0.17, "threshold": 1.3367469310760498, "operator": "ge"}, {"budget": 0.18, "threshold": 1.2942719459533691, "operator": "ge"}, {"budget": 0.19, "threshold": 1.2818652391433716, "operator": "ge"}, {"budget": 0.2, "threshold": 1.2430654764175415, "operator": "ge"}, {"budget": 0.21, "threshold": 1.2279908657073975, "operator": "ge"}, {"budget": 0.22, "threshold": 1.1946624517440796, "operator": "ge"}, {"budget": 0.23, "threshold": 1.168605089187622, "operator": "ge"}, {"budget": 0.24, "threshold": 1.1538331508636475, "operator": "ge"}, {"budget": 0.25, "threshold": 1.1349096298217773, "operator": "ge"}, {"budget": 0.26, "threshold": 1.1258571147918701, "operator": "ge"}, {"budget": 0.27, "threshold": 1.1022288799285889, "operator": "ge"}, {"budget": 0.28, "threshold": 1.0835171937942505, "operator": "ge"}, {"budget": 0.29, "threshold": 1.0706053972244263, "operator": "ge"}, {"budget": 0.3, "threshold": 1.0569627285003662, "operator": "ge"}, {"budget": 0.31, "threshold": 1.036561369895935, "operator": "ge"}, {"budget": 0.32, "threshold": 1.0252941846847534, "operator": "ge"}, {"budget": 0.33, "threshold": 1.0053116083145142, "operator": "ge"}, {"budget": 0.34, "threshold": 0.9926157593727112, "operator": "ge"}, {"budget": 0.35, "threshold": 0.9824280142784119, "operator": "ge"}, {"budget": 0.36, "threshold": 0.9488325119018555, "operator": "ge"}, {"budget": 0.37, "threshold": 0.9347901344299316, "operator": "ge"}, {"budget": 0.38, "threshold": 0.9209038019180298, "operator": "ge"}, {"budget": 0.39, "threshold": 0.8890469074249268, "operator": "ge"}, {"budget": 0.4, "threshold": 0.8540542125701904, "operator": "ge"}, {"budget": 0.41, "threshold": 0.8336830139160156, "operator": "ge"}, {"budget": 0.42, "threshold": 0.8143072128295898, "operator": "ge"}, {"budget": 0.43, "threshold": 0.7933553457260132, "operator": "ge"}, {"budget": 0.44, "threshold": 0.7733566164970398, "operator": "ge"}, {"budget": 0.45, "threshold": 0.7551624774932861, "operator": "ge"}, {"budget": 0.46, "threshold": 0.7323389053344727, "operator": "ge"}, {"budget": 0.47, "threshold": 0.7001591920852661, "operator": "ge"}, {"budget": 0.48, "threshold": 0.6651203036308289, "operator": "ge"}, {"budget": 0.49, "threshold": 0.5665906071662903, "operator": "ge"}, {"budget": 0.5, "threshold": 0.5128138065338135, "operator": "ge"}, {"budget": 0.51, "threshold": 0.40892142057418823, "operator": "ge"}, {"budget": 0.52, "threshold": 0.3911091685295105, "operator": "ge"}, {"budget": 0.53, "threshold": 0.3560854494571686, "operator": "ge"}, {"budget": 0.54, "threshold": 0.32434743642807007, "operator": "ge"}, {"budget": 0.55, "threshold": 0.2815152108669281, "operator": "ge"}, {"budget": 0.56, "threshold": 0.2469562590122223, "operator": "ge"}, {"budget": 0.57, "threshold": 0.23945340514183044, "operator": "ge"}, {"budget": 0.58, "threshold": 0.22691893577575684, "operator": "ge"}, {"budget": 0.59, "threshold": 0.20367580652236938, "operator": "ge"}, {"budget": 0.6, "threshold": 0.19422465562820435, "operator": "ge"}, {"budget": 0.61, "threshold": 0.1843041479587555, "operator": "ge"}, {"budget": 0.62, "threshold": 0.17499932646751404, "operator": "ge"}, {"budget": 0.63, "threshold": 0.155111625790596, "operator": "ge"}, {"budget": 0.64, "threshold": 0.14942580461502075, "operator": "ge"}, {"budget": 0.65, "threshold": 0.13416004180908203, "operator": "ge"}, {"budget": 0.66, "threshold": 0.11822140216827393, "operator": "ge"}, {"budget": 0.67, "threshold": 0.10641172528266907, "operator": "ge"}, {"budget": 0.68, "threshold": 0.09713073074817657, "operator": "ge"}, {"budget": 0.69, "threshold": 0.08457355946302414, "operator": "ge"}, {"budget": 0.7, "threshold": 0.07136431336402893, "operator": "ge"}, {"budget": 0.71, "threshold": 0.05340999364852905, "operator": "ge"}, {"budget": 0.72, "threshold": 0.01991816610097885, "operator": "ge"}, {"budget": 0.73, "threshold": 0.012768268585205078, "operator": "ge"}, {"budget": 0.74, "threshold": -0.01804351806640625, "operator": "ge"}, {"budget": 0.75, "threshold": -0.04721742868423462, "operator": "ge"}, {"budget": 0.76, "threshold": -0.08936488628387451, "operator": "ge"}, {"budget": 0.77, "threshold": -0.21317560970783234, "operator": "ge"}, {"budget": 0.78, "threshold": -2.9811973571777344, "operator": "ge"}, {"budget": 0.79, "threshold": -3.120330572128296, "operator": "ge"}, {"budget": 0.8, "threshold": -3.2236580848693848, "operator": "ge"}, {"budget": 0.81, "threshold": -3.3448562622070312, "operator": "ge"}, {"budget": 0.82, "threshold": -3.4269886016845703, "operator": "ge"}, {"budget": 0.83, "threshold": -3.4732937812805176, "operator": "ge"}, {"budget": 0.84, "threshold": -3.5100841522216797, "operator": "ge"}, {"budget": 0.85, "threshold": -3.5526468753814697, "operator": "ge"}, {"budget": 0.86, "threshold": -3.5799615383148193, "operator": "ge"}, {"budget": 0.87, "threshold": -3.619614839553833, "operator": "ge"}, {"budget": 0.88, "threshold": -3.6750447750091553, "operator": "ge"}, {"budget": 0.89, "threshold": -3.7324771881103516, "operator": "ge"}, {"budget": 0.9, "threshold": -3.7798755168914795, "operator": "ge"}, {"budget": 0.91, "threshold": -3.835334300994873, "operator": "ge"}, {"budget": 0.92, "threshold": -3.897451400756836, "operator": "ge"}, {"budget": 0.93, "threshold": -3.9351372718811035, "operator": "ge"}, {"budget": 0.94, "threshold": -3.9764504432678223, "operator": "ge"}, {"budget": 0.95, "threshold": -3.9945974349975586, "operator": "ge"}, {"budget": 0.96, "threshold": -4.052567958831787, "operator": "ge"}, {"budget": 0.97, "threshold": -4.083635330200195, "operator": "ge"}, {"budget": 0.98, "threshold": -4.201062202453613, "operator": "ge"}, {"budget": 0.99, "threshold": -4.334722518920898, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.663519371825353, 10.71336789203794, 10.330619414177365, 10.03576699878824, 9.923244502454148, 9.762641519931014, 11.201096869942564, 11.084651197162282, 10.790734499827847, 10.68773411447747, 10.43774393988872, 10.404990798755112, 10.436318958577893, 10.392805914156462, 11.07553238390654, 11.03827894089774, 11.112872452372482, 11.049650976646435, 10.723571473338591, 10.775612248388146, 10.772189557618884, 10.835703502812317, 10.874222703178384, 11.537040472200054, 11.608083827044963, 12.216354245625594, 12.80552877010489, 12.937631721084902, 12.983751467356866, 12.991507592184776, 13.036351563979595, 13.046516273323883, 13.14309777291729, 13.2623991285427, 13.250463765169648, 13.782585746697324, 14.30355470117728, 14.45735294334851, 14.593067307051225, 15.013589246432502, 15.10308532622863, 15.066630659006137, 15.19524764657795, 15.154027576197388, 15.546388062311628, 15.776053954964988, 16.031156054655305, 16.184863729620762, 16.370871442848298, 16.27582899653406, 16.201750604369263, 16.33268365259677, 16.585347437946403, 16.68050360640124, 16.805555555555557, 16.759876044446116, 16.9989194286322, 17.172420409847753, 17.418210444526235, 17.346871461593093, 17.54420983960626, 17.834085895906732, 18.084033416380883, 18.41142876660459, 18.5743613162968, 18.82623634896473, 18.759227013989012, 19.121042852648387, 19.45647350651675, 19.995011412192525, 20.41235536270583, 20.61056760091662, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.959975803430105, 21.66478622977795, 21.597999687079536, 21.566757043535123, 21.328923829367227, 21.48106110230197, 21.429319959074338, 21.370797309078736, 21.330205095221665, 21.285597194236853, 21.182426933293623, 21.07817631946929, 21.038608326603008, 21.002925225540068, 20.795190859542487, 20.75803100322011, 20.659795538412922, 20.633598388182634, 20.614782278148432, 20.59164657497359, 20.571314737297005, 20.547135309264167, 20.525114123986395, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 14, 31, 44, 58, 72, 88, 107, 118, 138, 152, 169, 185, 195, 212, 230, 242, 253, 267, 278, 290, 304, 316, 328, 343, 354, 365, 376, 386, 396, 408, 420, 428, 437, 448, 458, 469, 481, 492, 503, 514, 527, 537, 550, 559, 569, 580, 594, 605, 614, 621, 627, 634, 640, 645, 652, 657, 664, 670, 677, 682, 687, 694, 701, 708, 714, 720, 726, 733, 740, 747, 752, 758, 764, 771, 775, 781, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786], "harmed": [0, 4, 5, 9, 13, 15, 18, 22, 24, 29, 32, 38, 41, 44, 49, 55, 58, 59, 64, 71, 74, 80, 83, 87, 89, 91, 94, 96, 96, 99, 105, 111, 112, 115, 118, 126, 128, 131, 134, 138, 141, 149, 156, 162, 167, 172, 176, 181, 186, 190, 195, 202, 206, 210, 214, 218, 225, 230, 233, 237, 244, 250, 256, 263, 270, 277, 282, 289, 296, 301, 306, 310, 317, 323, 329, 335, 340, 347, 357, 365, 369, 375, 380, 386, 393, 399, 405, 414, 419, 425, 430, 435, 441, 446, 451, 454, 458, 462, 467, 471, 475], "exact_score": [7.641657905799261, 7.663519371825353, 10.71336789203794, 10.330619414177365, 10.03576699878824, 9.923244502454148, 9.762641519931014, 11.201096869942564, 11.084651197162282, 10.790734499827847, 10.68773411447747, 10.43774393988872, 10.404990798755112, 10.436318958577893, 10.392805914156462, 11.07553238390654, 11.03827894089774, 11.112872452372482, 11.049650976646435, 10.723571473338591, 10.775612248388146, 10.772189557618884, 10.835703502812317, 10.874222703178384, 11.537040472200054, 11.608083827044963, 12.216354245625594, 12.80552877010489, 12.937631721084902, 12.983751467356866, 12.991507592184776, 13.036351563979595, 13.046516273323883, 13.14309777291729, 13.2623991285427, 13.250463765169648, 13.782585746697324, 14.30355470117728, 14.45735294334851, 14.593067307051225, 15.013589246432502, 15.10308532622863, 15.066630659006137, 15.19524764657795, 15.154027576197388, 15.546388062311628, 15.776053954964988, 16.031156054655305, 16.184863729620762, 16.370871442848298, 16.27582899653406, 16.201750604369263, 16.33268365259677, 16.585347437946403, 16.68050360640124, 16.805555555555557, 16.759876044446116, 16.9989194286322, 17.172420409847753, 17.418210444526235, 17.346871461593093, 17.54420983960626, 17.834085895906732, 18.084033416380883, 18.41142876660459, 18.5743613162968, 18.82623634896473, 18.759227013989012, 19.121042852648387, 19.45647350651675, 19.995011412192525, 20.41235536270583, 20.61056760091662, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.959975803430105, 21.66478622977795, 21.597999687079536, 21.566757043535123, 21.328923829367227, 21.48106110230197, 21.429319959074338, 21.370797309078736, 21.330205095221665, 21.285597194236853, 21.182426933293623, 21.07817631946929, 21.038608326603008, 21.002925225540068, 20.795190859542487, 20.75803100322011, 20.659795538412922, 20.633598388182634, 20.614782278148432, 20.59164657497359, 20.571314737297005, 20.547135309264167, 20.525114123986395, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.056162140931722, "threshold_area100": 16.04967727899904, "threshold_area25": 10.417398823129684}, "test": {"score": [7.8873978710439285, 8.640475816415814, 9.283920228276978, 9.720667840835146, 9.627724728715137, 9.55671014572256, 9.53127359803335, 9.502941150599037, 9.762113800929804, 10.4023371942644, 10.682054603357178, 10.664427872921872, 10.526562725368759, 10.700082161386016, 10.688877704003698, 11.07259750802238, 11.028221537563356, 11.100886169754725, 11.204359810224375, 11.201261804691088, 11.111051644432075, 11.11265841068532, 11.219449761106748, 11.195897920071783, 11.2986426093107, 11.299763787888283, 11.314773763687699, 11.303386274935841, 11.20558902703188, 11.231088402824513, 11.372512829478865, 11.702002849134704, 11.885545635029457, 12.066938550163284, 12.131892277877009, 12.233562447103992, 12.421397016155575, 12.515488834614407, 12.765043757836796, 13.245312881913346, 13.739678069861593, 13.799604633227805, 14.049608408229169, 14.355051170284879, 14.523380821591001, 14.680774884970765, 14.714919015997657, 14.812943401789317, 14.764021721752805, 15.052692670487557, 15.093326046227123, 15.330335778546209, 15.364210243635473, 15.48561227389746, 15.597933950736381, 15.838852524972669, 15.999490876595827, 15.580192321817632, 15.74191365207016, 16.10771650218591, 16.19283828986742, 16.31971782676036, 16.480511606132485, 17.48974265374702, 17.529207718298217, 17.9893612370151, 18.50210187567683, 18.891884452741415, 19.09708480925246, 19.306665416846876, 19.59048244426828, 19.739923641612066, 20.24667750262382, 20.390893288706874, 21.517434182752314, 21.875294708222125, 22.271604145525355, 22.8199235830854, 23.030063281765788, 22.963412544136897, 22.926218437778843, 22.862970176075255, 22.686365434714666, 22.640343281654253, 22.63982415668414, 22.602763221040323, 22.589412874029545, 22.5425370253881, 22.563420263209462, 22.48151901433563, 22.45516482841347, 22.45918627213538, 22.409258518732084, 22.32156523404722, 22.293712765813627, 22.321275559032635, 22.224852281840686, 22.2026116724076, 22.450757260593143, 22.364567603971842, 22.361115727328066], "calls": [0.0, 0.7656458055925432, 1.4647137150466045, 2.2969374167776295, 3.362183754993342, 4.061251664447404, 4.394141145139813, 4.960053262316911, 6.058588548601864, 6.657789613848203, 7.023968042609853, 7.92276964047936, 9.154460719041278, 10.219707057256992, 10.785619174434089, 11.484687083888149, 12.350199733688415, 13.81491344873502, 15.512649800266312, 16.045272969374167, 17.676431424766978, 18.275632490013315, 19.540612516644472, 20.572569906790946, 21.13848202396804, 21.904127829560586, 22.237017310252995, 23.235685752330227, 24.36750998668442, 24.93342210386152, 25.532623169107854, 26.59786950732357, 27.62982689747004, 28.961384820239683, 29.82689747003995, 30.492676431424766, 33.15579227696405, 34.58721704394141, 35.685752330226364, 38.21571238348868, 40.279627163781626, 41.67776298268975, 42.743009320905465, 43.77496671105193, 44.67376830892144, 45.63914780292942, 46.43808255659121, 47.73635153129161, 48.568575233022635, 50.53262316910786, 51.16511318242344, 52.69640479360852, 52.99600532623169, 53.728362183754996, 54.52729693741678, 55.85885486018641, 57.15712383488681, 57.623169107856185, 58.5219707057257, 59.92010652463382, 60.51930758988016, 61.18508655126498, 61.8508655126498, 63.78162450066578, 64.24766977363515, 65.84553928095872, 67.17709720372837, 68.37549933422103, 69.0745672436751, 70.00665778961384, 70.83888149134488, 72.03728362183755, 73.40213049267643, 73.73501997336885, 75.0, 76.03195739014647, 76.99733688415446, 78.06258322237018, 79.06125166444741, 79.99334221038616, 80.65912117177098, 82.09054593874833, 83.12250332889481, 83.75499334221038, 84.62050599201065, 85.48601864181093, 85.85219707057257, 86.61784287616511, 87.51664447403462, 88.71504660452729, 89.34753661784288, 90.31291611185085, 91.54460719041279, 92.21038615179761, 92.90945406125167, 93.24234354194407, 94.3075898801598, 95.00665778961384, 96.80426098535287, 98.30226364846871, 100.0], "rescued": [0, 51, 94, 150, 220, 271, 296, 337, 403, 443, 470, 517, 588, 650, 687, 727, 774, 856, 946, 975, 1052, 1083, 1141, 1190, 1218, 1252, 1265, 1307, 1353, 1379, 1407, 1459, 1508, 1569, 1604, 1634, 1761, 1826, 1879, 1997, 2097, 2159, 2209, 2259, 2295, 2339, 2372, 2428, 2455, 2533, 2557, 2601, 2608, 2628, 2651, 2689, 2723, 2733, 2758, 2797, 2814, 2833, 2851, 2907, 2919, 2965, 3005, 3035, 3053, 3076, 3099, 3132, 3169, 3178, 3212, 3237, 3263, 3292, 3305, 3305, 3306, 3307, 3307, 3307, 3308, 3309, 3309, 3309, 3310, 3310, 3310, 3311, 3311, 3311, 3311, 3312, 3312, 3312, 3313, 3313, 3314], "harmed": [0, 6, 11, 24, 35, 45, 50, 58, 73, 82, 88, 94, 113, 129, 140, 155, 166, 190, 210, 215, 239, 245, 264, 282, 291, 302, 304, 317, 336, 345, 355, 370, 384, 407, 424, 437, 491, 518, 539, 585, 626, 655, 674, 695, 714, 737, 755, 787, 809, 848, 861, 879, 885, 895, 908, 933, 962, 972, 990, 1019, 1033, 1052, 1062, 1104, 1116, 1151, 1181, 1217, 1233, 1259, 1280, 1312, 1348, 1356, 1387, 1408, 1429, 1454, 1475, 1504, 1522, 1554, 1583, 1599, 1615, 1637, 1648, 1665, 1682, 1710, 1726, 1744, 1775, 1791, 1809, 1816, 1838, 1852, 1888, 1916, 1945], "exact_score": [7.8873978710439285, 9.357065141155102, 9.801668759352426, 9.673989035495188, 9.563021705696283, 9.505750396330964, 9.797673501363784, 10.678470245223155, 10.659238349270854, 10.542673068229012, 10.463716123205138, 10.912153405154273, 11.040528991678437, 11.142842024527019, 11.096866716031867, 11.236030266777924, 11.205074848086513, 11.143082997480892, 11.0984639561343, 11.062911823574781, 11.20841785905179, 11.326480533397817, 11.300305683415932, 11.33121058021351, 11.21863418874424, 11.233100006623339, 11.548353318113744, 11.8658168564853, 12.030366713327773, 12.070127732456944, 12.155305779632927, 12.25216684847455, 12.379262426479453, 12.426472962969177, 12.489925542081215, 12.518673559420632, 12.799511613386299, 12.925110768707967, 13.21083765511381, 13.425756682148032, 13.732333419460208, 13.72543133922471, 13.849059796506472, 14.134548098739971, 14.31408388023511, 14.53153264213212, 14.710554471138023, 14.746197848858586, 14.840513281607533, 14.789350366730353, 14.94304502850216, 15.070262297825437, 15.214381479731317, 15.364210243635473, 15.54421531305952, 15.69569512109887, 15.865347725765524, 15.963403688474626, 15.635980123153187, 15.890477680492024, 16.119610927194287, 16.287313190155466, 16.508732147078447, 17.071807820000075, 17.531122993840572, 17.80314081527751, 18.02721141640901, 18.431223563121684, 18.735516593406913, 19.071371827717623, 19.297368075524364, 19.65677998607469, 19.698543007798115, 20.103140620049007, 20.851285539493563, 21.517434182752314, 21.859331954647136, 22.271604145525355, 22.78875332699201, 22.988094613756125, 22.963412544136897, 22.862173416059065, 22.868534744362, 22.694406920251218, 22.63083481606978, 22.62287247306675, 22.58215917066236, 22.529489454586894, 22.54708122413463, 22.473182381184266, 22.435074678855337, 22.432895456234196, 22.329294520037767, 22.289266198449987, 22.256450429087625, 22.2026116724076, 22.53508132999751, 22.445827249148472, 22.376106560590955, 22.35073989123565, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 15.926263977398136, "threshold_area100": 15.875691619413661, "threshold_area25": 10.40319894620006}, "selected_budget": 76, "selected": {"val": {"score": 22.072697083038882, "calls": 75.89158345221114, "rescued": 781, "harmed": 340}, "test": {"score": 22.271604145525355, "calls": 76.99733688415446, "rescued": 3263, "harmed": 1429}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.4014074802398682, "operator": "ge"}, {"budget": 0.02, "threshold": 1.3677167892456055, "operator": "ge"}, {"budget": 0.03, "threshold": 1.3396347761154175, "operator": "ge"}, {"budget": 0.04, "threshold": 1.3025915622711182, "operator": "ge"}, {"budget": 0.05, "threshold": 1.27433443069458, "operator": "ge"}, {"budget": 0.06, "threshold": 1.2541084289550781, "operator": "ge"}, {"budget": 0.07, "threshold": 1.244545340538025, "operator": "ge"}, {"budget": 0.08, "threshold": 1.2249457836151123, "operator": "ge"}, {"budget": 0.09, "threshold": 1.159066915512085, "operator": "ge"}, {"budget": 0.1, "threshold": 1.1421189308166504, "operator": "ge"}, {"budget": 0.11, "threshold": 1.1082710027694702, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0941038131713867, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0575897693634033, "operator": "ge"}, {"budget": 0.14, "threshold": 1.0388437509536743, "operator": "ge"}, {"budget": 0.15, "threshold": 1.0133930444717407, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9697048664093018, "operator": "ge"}, {"budget": 0.17, "threshold": 0.948307991027832, "operator": "ge"}, {"budget": 0.18, "threshold": 0.9320639371871948, "operator": "ge"}, {"budget": 0.19, "threshold": 0.9181658029556274, "operator": "ge"}, {"budget": 0.2, "threshold": 0.9030966758728027, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8811688423156738, "operator": "ge"}, {"budget": 0.22, "threshold": 0.8617434501647949, "operator": "ge"}, {"budget": 0.23, "threshold": 0.8440311551094055, "operator": "ge"}, {"budget": 0.24, "threshold": 0.8265466690063477, "operator": "ge"}, {"budget": 0.25, "threshold": 0.8172916173934937, "operator": "ge"}, {"budget": 0.26, "threshold": 0.8037440776824951, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7945261001586914, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7816344499588013, "operator": "ge"}, {"budget": 0.29, "threshold": 0.7708438634872437, "operator": "ge"}, {"budget": 0.3, "threshold": 0.765949547290802, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7573944330215454, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7484956979751587, "operator": "ge"}, {"budget": 0.33, "threshold": 0.737762451171875, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7262606024742126, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7153174877166748, "operator": "ge"}, {"budget": 0.36, "threshold": 0.710723876953125, "operator": "ge"}, {"budget": 0.37, "threshold": 0.6957316398620605, "operator": "ge"}, {"budget": 0.38, "threshold": 0.6866811513900757, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6769475936889648, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6653227210044861, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6502715349197388, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6323776245117188, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6178823709487915, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6039513349533081, "operator": "ge"}, {"budget": 0.45, "threshold": 0.5856349468231201, "operator": "ge"}, {"budget": 0.46, "threshold": 0.5426764488220215, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5230569839477539, "operator": "ge"}, {"budget": 0.48, "threshold": 0.48134851455688477, "operator": "ge"}, {"budget": 0.49, "threshold": 0.4480810761451721, "operator": "ge"}, {"budget": 0.5, "threshold": 0.4139590263366699, "operator": "ge"}, {"budget": 0.51, "threshold": 0.37471842765808105, "operator": "ge"}, {"budget": 0.52, "threshold": 0.3365747332572937, "operator": "ge"}, {"budget": 0.53, "threshold": 0.31084370613098145, "operator": "ge"}, {"budget": 0.54, "threshold": 0.2694913148880005, "operator": "ge"}, {"budget": 0.55, "threshold": 0.2418636679649353, "operator": "ge"}, {"budget": 0.56, "threshold": 0.22983327507972717, "operator": "ge"}, {"budget": 0.57, "threshold": 0.2194046974182129, "operator": "ge"}, {"budget": 0.58, "threshold": 0.2071569263935089, "operator": "ge"}, {"budget": 0.59, "threshold": 0.19142727553844452, "operator": "ge"}, {"budget": 0.6, "threshold": 0.17676275968551636, "operator": "ge"}, {"budget": 0.61, "threshold": 0.16573530435562134, "operator": "ge"}, {"budget": 0.62, "threshold": 0.15384241938591003, "operator": "ge"}, {"budget": 0.63, "threshold": 0.14626021683216095, "operator": "ge"}, {"budget": 0.64, "threshold": 0.136856347322464, "operator": "ge"}, {"budget": 0.65, "threshold": 0.11841115355491638, "operator": "ge"}, {"budget": 0.66, "threshold": 0.10436132550239563, "operator": "ge"}, {"budget": 0.67, "threshold": 0.08972683548927307, "operator": "ge"}, {"budget": 0.68, "threshold": 0.08590859919786453, "operator": "ge"}, {"budget": 0.69, "threshold": 0.07868508249521255, "operator": "ge"}, {"budget": 0.7, "threshold": 0.06237095594406128, "operator": "ge"}, {"budget": 0.71, "threshold": 0.050176508724689484, "operator": "ge"}, {"budget": 0.72, "threshold": 0.018374525010585785, "operator": "ge"}, {"budget": 0.73, "threshold": 0.011094659566879272, "operator": "ge"}, {"budget": 0.74, "threshold": -0.016820132732391357, "operator": "ge"}, {"budget": 0.75, "threshold": -0.04343201220035553, "operator": "ge"}, {"budget": 0.76, "threshold": -0.07900550961494446, "operator": "ge"}, {"budget": 0.77, "threshold": -0.19543872773647308, "operator": "ge"}, {"budget": 0.78, "threshold": -2.7997970581054688, "operator": "ge"}, {"budget": 0.79, "threshold": -2.9061849117279053, "operator": "ge"}, {"budget": 0.8, "threshold": -3.0630974769592285, "operator": "ge"}, {"budget": 0.81, "threshold": -3.1686885356903076, "operator": "ge"}, {"budget": 0.82, "threshold": -3.2175819873809814, "operator": "ge"}, {"budget": 0.83, "threshold": -3.264742136001587, "operator": "ge"}, {"budget": 0.84, "threshold": -3.292625665664673, "operator": "ge"}, {"budget": 0.85, "threshold": -3.3575618267059326, "operator": "ge"}, {"budget": 0.86, "threshold": -3.3915553092956543, "operator": "ge"}, {"budget": 0.87, "threshold": -3.410964012145996, "operator": "ge"}, {"budget": 0.88, "threshold": -3.455631971359253, "operator": "ge"}, {"budget": 0.89, "threshold": -3.5182130336761475, "operator": "ge"}, {"budget": 0.9, "threshold": -3.5697169303894043, "operator": "ge"}, {"budget": 0.91, "threshold": -3.6159791946411133, "operator": "ge"}, {"budget": 0.92, "threshold": -3.6561102867126465, "operator": "ge"}, {"budget": 0.93, "threshold": -3.6943116188049316, "operator": "ge"}, {"budget": 0.94, "threshold": -3.7194719314575195, "operator": "ge"}, {"budget": 0.95, "threshold": -3.7645158767700195, "operator": "ge"}, {"budget": 0.96, "threshold": -3.7964932918548584, "operator": "ge"}, {"budget": 0.97, "threshold": -3.8646445274353027, "operator": "ge"}, {"budget": 0.98, "threshold": -3.9296152591705322, "operator": "ge"}, {"budget": 0.99, "threshold": -4.058135032653809, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 10.692401330971284, 10.148908639039416, 9.822737268082896, 9.57542172445395, 9.471948495117832, 9.463582997186483, 9.242143709121581, 9.30587033988259, 9.348053968392433, 9.39454716148532, 9.267860945189703, 9.375793758778485, 10.17962929253252, 10.213414364780348, 10.265619012187733, 10.224784251250158, 10.218984663210428, 9.995202221337395, 10.730978711642493, 11.400054232520441, 11.425171108899875, 11.466001340415707, 12.088396564203016, 12.725598992428116, 13.378532262142711, 13.326786709485738, 13.394737058872746, 13.435477346278319, 14.000382278428367, 13.90832728035663, 13.909875594337084, 14.016769754507097, 14.0788755850346, 14.191056199935225, 14.35792560626247, 14.479701342906386, 14.561450796905664, 14.725242920432494, 14.938512892169156, 15.05814545198107, 15.502144272245522, 15.724470393356357, 15.889228129150398, 16.383157780184188, 16.511988639969076, 16.454194396550935, 16.42793205039804, 16.540527186223336, 16.426232907090373, 16.59550138208675, 16.592379128787908, 16.78449726801743, 16.969762484876792, 16.695205612665, 16.84712493787478, 16.85810775853149, 16.97453213276836, 17.26467331118494, 17.412894964220243, 17.576936935003815, 17.52833519137867, 17.810522001270392, 18.00393255595214, 18.317996600001692, 18.62846204608546, 18.864324272219008, 18.936714129318243, 19.072383294456483, 19.45647350651675, 20.0074239601501, 20.41235536270583, 20.61056760091662, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.959975803430105, 21.66478622977795, 21.60311346870699, 21.557850449416712, 21.328923829367227, 21.47179755436311, 21.420409950164327, 21.382010262937577, 21.333507758669796, 21.29948620558117, 21.24776450680065, 20.96829755483107, 20.853879262563595, 20.812963299124938, 20.795190859542487, 20.77310713769923, 20.667004304272208, 20.640697123027866, 20.610181615653033, 20.589627092442992, 20.566927040399108, 20.547135309264167, 20.685604348672708, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 39, 57, 78, 93, 111, 124, 143, 157, 176, 190, 205, 222, 234, 250, 262, 273, 285, 296, 309, 320, 331, 344, 357, 368, 379, 390, 400, 410, 422, 432, 441, 453, 465, 478, 489, 499, 508, 521, 530, 541, 553, 564, 575, 585, 593, 600, 609, 616, 625, 630, 637, 642, 646, 652, 658, 664, 671, 677, 683, 689, 695, 701, 708, 714, 721, 726, 733, 740, 747, 752, 758, 764, 771, 775, 781, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786], "harmed": [0, 3, 9, 14, 21, 26, 30, 35, 38, 42, 47, 53, 55, 58, 62, 67, 71, 74, 80, 82, 85, 88, 91, 94, 95, 95, 101, 103, 106, 107, 114, 122, 124, 129, 132, 136, 140, 144, 146, 151, 153, 158, 162, 166, 170, 174, 179, 183, 186, 191, 198, 201, 204, 208, 215, 219, 225, 229, 233, 237, 243, 252, 258, 263, 270, 276, 283, 290, 295, 301, 305, 310, 317, 323, 329, 335, 340, 347, 357, 365, 370, 375, 381, 387, 396, 402, 407, 414, 419, 426, 432, 435, 438, 445, 449, 455, 459, 463, 467, 471, 475], "exact_score": [7.641657905799261, 10.692401330971284, 10.148908639039416, 9.822737268082896, 9.57542172445395, 9.471948495117832, 9.463582997186483, 9.242143709121581, 9.30587033988259, 9.348053968392433, 9.39454716148532, 9.267860945189703, 9.375793758778485, 10.17962929253252, 10.213414364780348, 10.265619012187733, 10.224784251250158, 10.218984663210428, 9.995202221337395, 10.730978711642493, 11.400054232520441, 11.425171108899875, 11.466001340415707, 12.088396564203016, 12.725598992428116, 13.378532262142711, 13.326786709485738, 13.394737058872746, 13.435477346278319, 14.000382278428367, 13.90832728035663, 13.909875594337084, 14.016769754507097, 14.0788755850346, 14.191056199935225, 14.35792560626247, 14.479701342906386, 14.561450796905664, 14.725242920432494, 14.938512892169156, 15.05814545198107, 15.502144272245522, 15.724470393356357, 15.889228129150398, 16.383157780184188, 16.511988639969076, 16.454194396550935, 16.42793205039804, 16.540527186223336, 16.426232907090373, 16.59550138208675, 16.592379128787908, 16.78449726801743, 16.969762484876792, 16.695205612665, 16.84712493787478, 16.85810775853149, 16.97453213276836, 17.26467331118494, 17.412894964220243, 17.576936935003815, 17.52833519137867, 17.810522001270392, 18.00393255595214, 18.317996600001692, 18.62846204608546, 18.864324272219008, 18.936714129318243, 19.072383294456483, 19.45647350651675, 20.0074239601501, 20.41235536270583, 20.61056760091662, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.959975803430105, 21.66478622977795, 21.60311346870699, 21.557850449416712, 21.328923829367227, 21.47179755436311, 21.420409950164327, 21.382010262937577, 21.333507758669796, 21.29948620558117, 21.24776450680065, 20.96829755483107, 20.853879262563595, 20.812963299124938, 20.795190859542487, 20.77310713769923, 20.667004304272208, 20.640697123027866, 20.610181615653033, 20.589627092442992, 20.566927040399108, 20.547135309264167, 20.685604348672708, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.17535536335429, "threshold_area100": 16.16892614157886, "threshold_area25": 10.208452797056463}, "test": {"score": [7.8873978710439285, 7.954421562908079, 8.01565523946444, 8.625009829907645, 9.520383961533351, 10.165928972973392, 10.415051747168535, 10.685163227272252, 10.52934581879192, 11.241333983997933, 11.2044200709242, 11.295956992840491, 11.258585413294815, 11.385556601994125, 11.337736961603026, 11.18643456559593, 11.031896180910328, 11.032913460118722, 11.37888041942542, 11.563746809082215, 11.593798363861115, 11.502402155908074, 11.469731317252762, 11.63674466727867, 11.778298775519971, 11.800260692434625, 12.008247428369383, 12.039786384050064, 12.189053651501995, 12.287449567484384, 12.268159110817793, 12.42424092384717, 12.622377223015224, 12.837515113299613, 12.991769313012835, 13.33773889562979, 13.365496929190398, 13.79657660846311, 13.988993834123084, 14.324950768844303, 14.435464011946635, 14.54769555454763, 14.766243869885917, 14.75502900054152, 14.838490656309808, 14.889628543634363, 15.088631493093738, 15.325441728403325, 15.544444764371718, 15.47396133948694, 15.410622756209118, 15.385460422784778, 15.43939692825577, 15.532740418122176, 15.787607952449315, 15.870884681765835, 15.985935788284628, 16.07315249266166, 16.261073734012736, 16.04915459744084, 16.2516721064562, 16.460612602977953, 16.615120599169096, 17.081046478568258, 17.67786258853188, 18.006734642551013, 18.603173326086186, 18.991738767499854, 19.124947748712714, 19.26050908609048, 19.71008636909944, 19.72968344256824, 20.24667750262382, 20.42465775114531, 21.530110341489042, 21.875294708222125, 22.271604145525355, 22.835588741905664, 22.984615524911185, 22.991336059267514, 22.863833407811526, 22.713478273370267, 22.672004004657232, 22.61213441614754, 22.544404304551133, 22.58472387271178, 22.56211846140839, 22.54374903036173, 22.5645785034906, 22.507886271702784, 22.409229884137712, 22.394869963963497, 22.354552744119747, 22.28127036939228, 22.271749767256672, 22.267461973961513, 22.589641843826836, 22.544870623852027, 22.4515320129425, 22.37538180462975, 22.361115727328066], "calls": [0.0, 0.5992010652463382, 1.1984021304926764, 1.6644474034620507, 2.762982689747004, 3.561917443408788, 4.194407456724368, 4.4607190412782955, 4.993342210386151, 7.023968042609853, 7.456724367509987, 8.355525965379494, 8.721704394141145, 9.720372836218376, 10.51930758988016, 11.18508655126498, 12.916111850865514, 13.548601864181093, 14.181091877496671, 14.513981358189081, 15.213049267643141, 16.74434087882823, 17.80958721704394, 19.340878828229027, 20.33954727030626, 21.1051930758988, 22.436750998668444, 23.302263648468706, 24.766977363515313, 25.898801597869507, 26.43142476697736, 27.19707057256991, 28.22902796271638, 29.69374167776298, 31.29161118508655, 32.889480692410125, 33.45539280958722, 35.28628495339547, 36.21837549933422, 37.48335552596538, 38.48202396804261, 39.94673768308921, 41.4780292942743, 42.41011984021305, 43.40878828229028, 44.50732356857524, 46.737683089214386, 47.603195739014645, 49.06790945406125, 50.06657789613848, 50.79893475366178, 52.16378162450066, 53.16245006657789, 53.8948069241012, 55.45938748335553, 56.55792276964048, 57.02396804260985, 57.58988015978696, 58.25565912117177, 59.52063914780293, 60.78561917443409, 61.51797603195739, 62.45006657789614, 63.38215712383488, 64.4474034620506, 65.87882822902796, 67.57656458055925, 68.77496671105193, 69.14114513981359, 69.74034620505992, 71.00532623169109, 71.83754993342211, 73.40213049267643, 73.76830892143809, 75.03328894806924, 76.03195739014647, 76.99733688415446, 78.09587217043942, 79.02796271637817, 79.69374167776299, 81.02529960053262, 82.58988015978696, 83.22237017310253, 83.88814913448735, 84.28761651131825, 85.68575233022636, 86.18508655126497, 86.51797603195739, 87.48335552596538, 88.54860186418108, 89.74700399467378, 90.61251664447404, 91.37816245006658, 92.11051930758988, 92.44340878828228, 93.47536617842877, 93.94141145139814, 95.50599201065246, 96.57123834886818, 97.90279627163781, 100.0], "rescued": [0, 50, 98, 136, 226, 287, 335, 356, 398, 549, 580, 641, 661, 725, 774, 810, 904, 934, 967, 984, 1020, 1089, 1144, 1217, 1264, 1297, 1364, 1402, 1462, 1513, 1532, 1568, 1620, 1676, 1752, 1829, 1856, 1940, 1985, 2050, 2092, 2155, 2227, 2261, 2305, 2349, 2431, 2469, 2521, 2547, 2561, 2590, 2617, 2635, 2679, 2707, 2721, 2735, 2756, 2789, 2824, 2844, 2870, 2897, 2929, 2967, 3015, 3044, 3054, 3069, 3105, 3127, 3169, 3179, 3213, 3237, 3263, 3293, 3304, 3305, 3306, 3307, 3307, 3307, 3307, 3309, 3309, 3309, 3310, 3310, 3310, 3311, 3311, 3311, 3311, 3312, 3313, 3313, 3313, 3313, 3314], "harmed": [0, 7, 17, 26, 47, 62, 71, 77, 93, 119, 125, 142, 148, 164, 177, 192, 221, 227, 234, 238, 245, 273, 294, 313, 331, 339, 354, 366, 384, 397, 404, 419, 437, 460, 491, 517, 532, 567, 583, 609, 632, 660, 695, 711, 731, 751, 790, 807, 831, 843, 851, 869, 890, 897, 925, 952, 959, 972, 986, 1013, 1043, 1057, 1078, 1101, 1120, 1153, 1194, 1230, 1239, 1252, 1287, 1308, 1348, 1358, 1388, 1408, 1429, 1455, 1476, 1494, 1533, 1569, 1582, 1597, 1608, 1638, 1654, 1664, 1681, 1704, 1735, 1755, 1770, 1788, 1794, 1821, 1833, 1862, 1887, 1909, 1945], "exact_score": [7.8873978710439285, 7.9963604876418195, 9.160703824951288, 9.514654469031171, 10.455282097435942, 10.52934581879192, 11.0413187396933, 11.237079335873272, 11.039556598251417, 11.22059625222308, 11.349307928826171, 11.186177424998556, 11.155495622906814, 11.03860935941667, 11.008057516231574, 11.57941931358895, 11.538840789683011, 11.502375989984886, 11.477873795467339, 11.64917444535867, 11.769667348614934, 11.787839996239386, 11.897610743453416, 12.019308032437982, 12.06578231160944, 12.204861436834921, 12.267042092134513, 12.409019832605443, 12.623718157571354, 12.844435087094485, 12.855621394661028, 13.007474929786369, 13.049134232325924, 13.34267165495811, 13.516997782790199, 13.759203027875044, 13.972993600294256, 14.157112396261038, 14.32246031286089, 14.568607818812932, 14.553138251835648, 14.636433904889227, 14.76846559456112, 14.782954641402867, 14.90242804460881, 14.888690272338364, 15.023959537801366, 15.179471183997572, 15.402732601805674, 15.571038311853632, 15.462638721614603, 15.379915085993472, 15.411593309273133, 15.443308398241376, 15.550291428926203, 15.708238817514022, 15.814962389025453, 15.980966619014023, 16.128721305057, 16.206072241464575, 16.15672799538468, 16.293292937466862, 16.56411073914661, 16.756889686623925, 17.48544443808088, 17.815346483693688, 18.086736217784605, 18.449885494869356, 18.752413888679044, 19.050577416369123, 19.317109312379934, 19.701828285404574, 19.783259842119243, 20.103140620049007, 20.851285539493563, 21.517434182752314, 21.859331954647136, 22.271604145525355, 22.78875332699201, 22.98635271541698, 22.909603268274758, 22.86605470495033, 22.741227107224535, 22.695605351798335, 22.606783671872975, 22.564985587006873, 22.56936525459928, 22.533427496067596, 22.54630248768724, 22.43955658992584, 22.383512416252536, 22.36612728441098, 22.284025812047584, 22.286712663641843, 22.589641843826836, 22.560391802516325, 22.529938002607576, 22.447814400428662, 22.373694120672198, 22.34575523792584, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.23482475011074, "threshold_area100": 16.154598751448027, "threshold_area25": 10.798882251018783}, "selected_budget": 76, "selected": {"val": {"score": 22.072697083038882, "calls": 75.89158345221114, "rescued": 781, "harmed": 340}, "test": {"score": 22.271604145525355, "calls": 76.99733688415446, "rescued": 3263, "harmed": 1429}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 22.072697083038882, "calls": 75.89158345221114}, "test": {"score": 22.151919702262795, "calls": 76.83089214380826}}, "old_normalized_interior": {"val": {"score": 22.072697083038882, "calls": 75.89158345221114}, "test": {"score": 22.151919702262795, "calls": 76.83089214380826}}, "old_normalized_endpoints": {"val": {"score": 22.072697083038882, "calls": 75.89158345221114}, "test": {"score": 22.151919702262795, "calls": 76.83089214380826}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 10.692401330971284, 10.038668526804887, 9.706661602896387, 9.530192066325638, 9.44813497688483, 9.365555715865916, 9.385928026015828, 9.215979209309975, 9.26541023691032, 9.372384206822474, 9.472135527518365, 9.48827210076541, 9.338990395250352, 9.92524555024555, 10.670555737957471, 10.70700277422396, 10.798179770533377, 11.981064634368229, 12.052527151211361, 12.606905289832119, 12.658941528813491, 12.63757929557614, 12.675207171024406, 12.724988807992272, 12.72036099402643, 12.728428466813424, 12.816512230844642, 12.882810241349823, 12.851028249548074, 13.352531604338939, 13.851710307667886, 13.960366417331475, 14.466406741959842, 14.57882084737191, 15.072736532051131, 15.19843212095347, 15.331035651361477, 15.473264233647349, 15.639745690741897, 15.594570954159991, 15.652442795299937, 15.826129159462493, 15.840288897737786, 16.08145570432098, 16.35448347435029, 16.536834143143007, 16.593454745923818, 16.416093017874704, 16.405297605733764, 16.159940425445544, 16.160008895515674, 16.366610886904713, 16.55551519426569, 16.75037718120977, 16.87713358070501, 17.057478120241715, 17.177576598487686, 17.13392279334754, 17.315203159957875, 17.573224942150773, 17.72673186654018, 17.962287200043583, 17.8896999662991, 18.179153571028355, 18.581834470388113, 18.59515880839605, 18.86527155269527, 19.07778197412344, 19.40394607906341, 19.93031570406634, 20.41235536270583, 20.529063637870966, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.888546869331478, 21.919506151400395, 21.893448502718726, 21.880083092400632, 21.84351278855813, 21.814686618124686, 21.776514553540807, 21.750809318616877, 21.714046781415853, 21.674162399864695, 21.649903583222564, 21.43052721097708, 21.310625175217268, 21.26368578880485, 21.2401181245122, 21.060306223878946, 21.01611221137199, 20.97775619958726, 20.93941382381252, 20.823727652839175, 20.96151507315106, 20.921163363964894, 20.866595027535237, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 10.692401330971284, 10.038668526804887, 9.706661602896387, 9.530192066325638, 9.44813497688483, 9.365555715865916, 9.385928026015828, 9.215979209309975, 9.26541023691032, 9.372384206822474, 9.472135527518365, 9.48827210076541, 9.338990395250352, 9.92524555024555, 10.670555737957471, 10.70700277422396, 10.798179770533377, 11.981064634368229, 12.052527151211361, 12.606905289832119, 12.658941528813491, 12.63757929557614, 12.675207171024406, 12.724988807992272, 12.72036099402643, 12.728428466813424, 12.816512230844642, 12.882810241349823, 12.851028249548074, 13.352531604338939, 13.851710307667886, 13.960366417331475, 14.466406741959842, 14.57882084737191, 15.072736532051131, 15.19843212095347, 15.331035651361477, 15.473264233647349, 15.639745690741897, 15.594570954159991, 15.652442795299937, 15.826129159462493, 15.840288897737786, 16.08145570432098, 16.35448347435029, 16.536834143143007, 16.593454745923818, 16.416093017874704, 16.405297605733764, 16.159940425445544, 16.160008895515674, 16.366610886904713, 16.55551519426569, 16.75037718120977, 16.87713358070501, 17.057478120241715, 17.177576598487686, 17.13392279334754, 17.315203159957875, 17.573224942150773, 17.72673186654018, 17.962287200043583, 17.8896999662991, 18.179153571028355, 18.581834470388113, 18.59515880839605, 18.86527155269527, 19.07778197412344, 19.40394607906341, 19.93031570406634, 20.41235536270583, 20.529063637870966, 21.092229183027534, 21.366664359797806, 21.541058815295575, 22.072697083038882, 21.888546869331478, 21.919506151400395, 21.893448502718726, 21.880083092400632, 21.84351278855813, 21.814686618124686, 21.776514553540807, 21.750809318616877, 21.714046781415853, 21.674162399864695, 21.649903583222564, 21.43052721097708, 21.310625175217268, 21.26368578880485, 21.2401181245122, 21.060306223878946, 21.01611221137199, 20.97775619958726, 20.93941382381252, 20.823727652839175, 20.96151507315106, 20.921163363964894, 20.866595027535237, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 10.455797932281948, "exact_area100": 16.317435507140857}, "test": {"score": [7.8873978710439285, 7.938582130219832, 8.604854115685127, 9.503107553953265, 10.279830127076144, 10.144309210955575, 10.628218173975938, 10.96399402303937, 10.806535197101043, 10.762677414146514, 11.209483336889363, 11.443832388188078, 11.386817223907993, 11.16398806115688, 11.023550882640368, 10.989925141268447, 11.124521682098901, 11.059530804175862, 11.022660729955449, 11.17656415174768, 11.230508774647543, 11.335696990164907, 11.874952085430415, 12.375236912300792, 12.50772579358577, 12.596705094520166, 12.690064648502936, 12.814514009447317, 12.91797611527957, 13.122183474467594, 13.346629052862918, 13.473266379179737, 13.516070244243341, 13.727676747489806, 13.866816953721232, 14.00011212427546, 14.065400890300007, 14.17521879535838, 14.370066416168031, 14.452907292962806, 14.577790501057395, 14.917887271520247, 15.071407350539182, 15.234798903742707, 15.491502671378477, 15.682606071645544, 15.80024412333792, 15.813316445096696, 15.615484169967054, 15.347458950683174, 15.228522451523855, 15.112308599206395, 15.34817683125107, 15.540771303425839, 15.719130512515466, 15.767132219804067, 15.99953981542824, 16.209356803568628, 16.23811940819816, 16.477515204845716, 16.824617434670113, 16.7053995984643, 16.597085045571117, 16.86697158237546, 17.26406772130991, 17.57186476162452, 18.46144460959927, 18.73527242554262, 19.071458947291575, 19.414148519906256, 19.781348791202362, 20.06745835588736, 20.221858888113104, 20.42465775114531, 21.542829896144216, 21.94691336506925, 22.151919702262795, 22.853583435826284, 23.04787640979075, 22.968085689042308, 22.942334005076958, 22.87911892522202, 22.8925318934632, 23.07271660897012, 22.87195990709154, 22.849682266652724, 22.723983140884034, 22.717592021146967, 22.639929072684254, 22.616302510549815, 22.553837477061776, 22.52634081944332, 22.527528629365662, 22.45890573840617, 22.460338995963273, 22.45146187645203, 22.39080675786611, 22.40400292526237, 22.37379772859531, 22.392198864273936, 22.361115727328066], "calls": [0.0, 0.4660452729693742, 1.2649800266311584, 2.1637816245006656, 2.5299600532623168, 3.029294274300932, 3.9946737683089215, 5.226364846870839, 6.491344873501998, 6.957390146471372, 7.8561917443408795, 8.222370173102531, 8.921438082556591, 9.820239680426099, 10.818908122503329, 11.584553928095872, 12.616511318242344, 13.282290279627162, 14.81358189081225, 15.645805592543274, 16.378162450066576, 17.41011984021305, 18.708388814913448, 19.940079893475364, 21.438082556591212, 22.836218375499335, 23.735019973368843, 24.93342210386152, 25.63249001331558, 27.496671105193077, 29.360852197070574, 30.492676431424766, 31.657789613848202, 32.856191744340876, 33.75499334221039, 34.720372836218374, 36.085219707057256, 37.017310252996005, 38.1491344873502, 38.78162450066578, 39.513981358189085, 40.51264980026631, 41.344873501997334, 42.07723035952064, 44.40745672436751, 45.13981358189081, 46.504660452729695, 47.76964047936085, 49.03462050599201, 50.36617842876166, 50.96537949400799, 51.73102529960053, 53.26231691078562, 54.460719041278296, 55.625832223701735, 55.9254327563249, 57.05725699067909, 58.32223701731025, 58.655126498002666, 59.85352862849535, 61.051930758988014, 61.884154460719046, 62.61651131824234, 63.914780292942744, 65.04660452729694, 66.21171770972038, 67.37683089214381, 68.40878828229027, 69.2410119840213, 70.20639147802929, 71.20505992010652, 71.87083888149135, 73.40213049267643, 73.76830892143809, 75.09986684420772, 76.2982689747004, 76.83089214380826, 78.19573901464713, 79.32756324900133, 80.3595206391478, 80.95872170439414, 81.59121171770973, 82.29027962716378, 83.5219707057257, 85.38615179760319, 85.98535286284952, 86.81757656458056, 87.01731025299601, 87.71637816245007, 89.01464713715048, 89.54727030625833, 91.24500665778962, 92.41011984021304, 93.74167776298269, 94.74034620505992, 95.97203728362183, 96.6711051930759, 97.27030625832224, 98.00266311584554, 99.20106524633822, 100.0], "exact_score": [7.8873978710439285, 8.613048712068878, 9.541242879608937, 10.15604085862786, 10.628218173975938, 10.998571380877701, 10.893674404366505, 10.756245210336905, 11.213335347925648, 11.298364887587041, 11.17660714931372, 11.028373350894578, 10.932839440738263, 11.088776205283777, 11.078969658106349, 11.014369381986064, 11.29116061464605, 11.228996423790067, 11.463328120546603, 11.868349346765266, 12.3796739204838, 12.519645804537035, 12.471982172378684, 12.719430193639711, 12.697000609704789, 12.811933864720304, 12.908258961253225, 13.116444028424251, 13.202688875991456, 13.353353252828063, 13.455254002346576, 13.514896801424708, 13.55812336160786, 13.73699983032969, 13.884845908854269, 14.018394023278494, 14.054454978717363, 14.17521879535838, 14.350271409122886, 14.487192882002844, 14.721589922068615, 15.00891361897925, 15.208699071026258, 15.40549229377858, 15.641805321239088, 15.65551412661453, 15.834678565425014, 15.767682062616359, 15.775310380499713, 15.614590943306297, 15.407245613011336, 15.228522451523855, 15.113822990899955, 15.287625540864191, 15.471849693094692, 15.619743409718428, 15.772616501344746, 15.978981750565898, 16.191338541505736, 16.308014367455858, 16.507928213763115, 16.78943399904086, 16.732562028394348, 16.696545386105697, 16.8941331818875, 17.246163211040802, 17.499546607088927, 18.36594352852505, 18.602318047681056, 18.92565246724082, 19.321298695186893, 19.72504867133783, 20.108346185773893, 20.046764730007258, 20.851285539493563, 21.53086718976187, 21.86178053447084, 22.260170006601125, 22.755023949481725, 23.053637998519264, 22.978255683501907, 22.942334005076958, 22.863179997204533, 23.154822016625495, 23.036384210871336, 22.934580838844713, 22.849682266652724, 22.719676573746916, 22.631647071798742, 22.617653704293932, 22.59940851904116, 22.53695367976944, 22.545836951050617, 22.503630772264312, 22.448201473092848, 22.43622195943248, 22.45146187645203, 22.416584230386537, 22.374987314619375, 22.382223856943504, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 10.763255589308596, "exact_area100": 16.392316751844053}}, "old_normalized_endpoint_budget": 76}, {"seed": 2027, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.0, "validation_area100": 0.16320460446639076, "training_seconds": 4.067160057951696, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5426844954490662, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5270047187805176, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5149059295654297, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5082348585128784, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4876318573951721, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4827843904495239, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4726085066795349, "operator": "ge"}, {"budget": 0.08, "threshold": 0.44910871982574463, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4393736720085144, "operator": "ge"}, {"budget": 0.1, "threshold": 0.40735292434692383, "operator": "ge"}, {"budget": 0.11, "threshold": 0.37872883677482605, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3539283275604248, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3380585312843323, "operator": "ge"}, {"budget": 0.14, "threshold": 0.32391253113746643, "operator": "ge"}, {"budget": 0.15, "threshold": 0.31818974018096924, "operator": "ge"}, {"budget": 0.16, "threshold": 0.30115869641304016, "operator": "ge"}, {"budget": 0.17, "threshold": 0.29165932536125183, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2850906252861023, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2817002236843109, "operator": "ge"}, {"budget": 0.2, "threshold": 0.27978020906448364, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2754681408405304, "operator": "ge"}, {"budget": 0.22, "threshold": 0.2718718647956848, "operator": "ge"}, {"budget": 0.23, "threshold": 0.27121150493621826, "operator": "ge"}, {"budget": 0.24, "threshold": 0.26724565029144287, "operator": "ge"}, {"budget": 0.25, "threshold": 0.26565009355545044, "operator": "ge"}, {"budget": 0.26, "threshold": 0.2629561424255371, "operator": "ge"}, {"budget": 0.27, "threshold": 0.26052623987197876, "operator": "ge"}, {"budget": 0.28, "threshold": 0.2590908408164978, "operator": "ge"}, {"budget": 0.29, "threshold": 0.25725722312927246, "operator": "ge"}, {"budget": 0.3, "threshold": 0.2546693980693817, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2517916262149811, "operator": "ge"}, {"budget": 0.32, "threshold": 0.24886560440063477, "operator": "ge"}, {"budget": 0.33, "threshold": 0.24377061426639557, "operator": "ge"}, {"budget": 0.34, "threshold": 0.24020269513130188, "operator": "ge"}, {"budget": 0.35, "threshold": 0.2345752865076065, "operator": "ge"}, {"budget": 0.36, "threshold": 0.2324848473072052, "operator": "ge"}, {"budget": 0.37, "threshold": 0.22940993309020996, "operator": "ge"}, {"budget": 0.38, "threshold": 0.22276505827903748, "operator": "ge"}, {"budget": 0.39, "threshold": 0.2150501310825348, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2106122374534607, "operator": "ge"}, {"budget": 0.41, "threshold": 0.1978839486837387, "operator": "ge"}, {"budget": 0.42, "threshold": 0.1836877465248108, "operator": "ge"}, {"budget": 0.43, "threshold": 0.17422938346862793, "operator": "ge"}, {"budget": 0.44, "threshold": 0.16241896152496338, "operator": "ge"}, {"budget": 0.45, "threshold": 0.15406842529773712, "operator": "ge"}, {"budget": 0.46, "threshold": 0.13724160194396973, "operator": "ge"}, {"budget": 0.47, "threshold": 0.1279979646205902, "operator": "ge"}, {"budget": 0.48, "threshold": 0.11859403550624847, "operator": "ge"}, {"budget": 0.49, "threshold": 0.11529508233070374, "operator": "ge"}, {"budget": 0.5, "threshold": 0.11066114902496338, "operator": "ge"}, {"budget": 0.51, "threshold": 0.10685388743877411, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1050761267542839, "operator": "ge"}, {"budget": 0.53, "threshold": 0.10178354382514954, "operator": "ge"}, {"budget": 0.54, "threshold": 0.09797769784927368, "operator": "ge"}, {"budget": 0.55, "threshold": 0.09530256688594818, "operator": "ge"}, {"budget": 0.56, "threshold": 0.09328354895114899, "operator": "ge"}, {"budget": 0.57, "threshold": 0.0910453200340271, "operator": "ge"}, {"budget": 0.58, "threshold": 0.08741492033004761, "operator": "ge"}, {"budget": 0.59, "threshold": 0.0844215452671051, "operator": "ge"}, {"budget": 0.6, "threshold": 0.08227072656154633, "operator": "ge"}, {"budget": 0.61, "threshold": 0.08010168373584747, "operator": "ge"}, {"budget": 0.62, "threshold": 0.07787324488162994, "operator": "ge"}, {"budget": 0.63, "threshold": 0.07330247759819031, "operator": "ge"}, {"budget": 0.64, "threshold": 0.0718550980091095, "operator": "ge"}, {"budget": 0.65, "threshold": 0.06675291061401367, "operator": "ge"}, {"budget": 0.66, "threshold": 0.06311599910259247, "operator": "ge"}, {"budget": 0.67, "threshold": 0.05742239952087402, "operator": "ge"}, {"budget": 0.68, "threshold": 0.05470120906829834, "operator": "ge"}, {"budget": 0.69, "threshold": 0.048705294728279114, "operator": "ge"}, {"budget": 0.7, "threshold": 0.04292312264442444, "operator": "ge"}, {"budget": 0.71, "threshold": 0.037435874342918396, "operator": "ge"}, {"budget": 0.72, "threshold": 0.03282877802848816, "operator": "ge"}, {"budget": 0.73, "threshold": 0.026126623153686523, "operator": "ge"}, {"budget": 0.74, "threshold": 0.021005868911743164, "operator": "ge"}, {"budget": 0.75, "threshold": 0.013213291764259338, "operator": "ge"}, {"budget": 0.76, "threshold": 0.0030574798583984375, "operator": "ge"}, {"budget": 0.77, "threshold": -0.008658885955810547, "operator": "ge"}, {"budget": 0.78, "threshold": -0.09074939042329788, "operator": "ge"}, {"budget": 0.79, "threshold": -0.10961824655532837, "operator": "ge"}, {"budget": 0.8, "threshold": -0.11667399853467941, "operator": "ge"}, {"budget": 0.81, "threshold": -0.11903789639472961, "operator": "ge"}, {"budget": 0.82, "threshold": -0.12213939428329468, "operator": "ge"}, {"budget": 0.83, "threshold": -0.1256289780139923, "operator": "ge"}, {"budget": 0.84, "threshold": -0.1288442462682724, "operator": "ge"}, {"budget": 0.85, "threshold": -0.13026514649391174, "operator": "ge"}, {"budget": 0.86, "threshold": -0.13201916217803955, "operator": "ge"}, {"budget": 0.87, "threshold": -0.13581186532974243, "operator": "ge"}, {"budget": 0.88, "threshold": -0.1381959319114685, "operator": "ge"}, {"budget": 0.89, "threshold": -0.1402575969696045, "operator": "ge"}, {"budget": 0.9, "threshold": -0.14238020777702332, "operator": "ge"}, {"budget": 0.91, "threshold": -0.14441955089569092, "operator": "ge"}, {"budget": 0.92, "threshold": -0.14597465097904205, "operator": "ge"}, {"budget": 0.93, "threshold": -0.14858360588550568, "operator": "ge"}, {"budget": 0.94, "threshold": -0.1504906266927719, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1529204249382019, "operator": "ge"}, {"budget": 0.96, "threshold": -0.15487785637378693, "operator": "ge"}, {"budget": 0.97, "threshold": -0.15640202164649963, "operator": "ge"}, {"budget": 0.98, "threshold": -0.15996035933494568, "operator": "ge"}, {"budget": 0.99, "threshold": -0.16852501034736633, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.751224860383048, 7.876181366312142, 7.970885416231045, 7.871188629346748, 9.215434910654718, 9.199688313674494, 9.16618379321786, 9.191102937788239, 9.278572267399083, 9.378500811955695, 9.38432007623213, 9.448709190640695, 9.553116932872646, 10.421895235619687, 10.404777722322166, 10.421122383759013, 10.36104413505829, 10.465538336774845, 10.54049450659667, 11.132699469413465, 11.785588240555963, 11.730913664422445, 11.756653914481241, 11.864391756570065, 12.411438396487245, 12.468094405594405, 13.021349224674882, 13.086620034307566, 13.169765943701083, 13.291556525094428, 13.355625144170402, 13.419829065610667, 13.957597916500125, 14.455101037400489, 14.835194950071928, 14.932202087654927, 15.095449467041098, 15.543249944736798, 15.48287243509264, 15.691458420478796, 15.690430028306643, 15.89285802489906, 16.124697532092604, 16.16192020735517, 16.259685548395648, 16.483079904591534, 16.47129952157126, 16.605356436238246, 16.72077922077922, 16.89886195022234, 17.10201448781905, 17.30167424464784, 17.499584593107254, 17.695335710041594, 17.86042241818264, 17.994414357593868, 18.135256358775127, 18.23273220625956, 18.501518771742916, 18.782082033563142, 18.957061457061457, 19.230587476368726, 19.308041165184022, 19.387767671739795, 19.707327741926342, 19.781362881224183, 19.906842318755068, 19.70375682428056, 19.83164079682216, 20.10205514686158, 20.467145467145468, 20.831658004596605, 21.13978712890955, 21.33255491360587, 21.320900576561485, 21.757149177129527, 22.09509348306372, 21.943098196525597, 21.90598487228739, 21.8798590109504, 21.851357479614276, 21.826121190416796, 21.41812496042334, 21.40884883548237, 21.570142583373737, 21.532119592344554, 21.491128955783427, 21.447799041167126, 21.404624501553673, 21.367476568149744, 21.32631786866614, 21.30746417334186, 21.112263417163597, 20.9804345693785, 20.87099221623277, 20.76013776122802, 20.7243618450515, 20.675293266571156, 20.795821202798752, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 20, 41, 59, 77, 94, 112, 129, 148, 163, 180, 198, 211, 223, 237, 251, 262, 276, 289, 300, 314, 326, 335, 345, 357, 370, 383, 397, 409, 421, 430, 441, 453, 464, 476, 487, 497, 505, 517, 526, 537, 547, 558, 569, 577, 586, 595, 602, 607, 614, 621, 628, 636, 643, 651, 656, 664, 670, 678, 685, 692, 698, 704, 709, 715, 721, 726, 731, 736, 742, 749, 756, 761, 767, 771, 775, 780, 783, 784, 784, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786], "harmed": [0, 6, 12, 18, 26, 30, 35, 39, 44, 47, 49, 54, 57, 60, 65, 69, 73, 78, 81, 85, 90, 93, 96, 99, 100, 104, 109, 113, 118, 121, 122, 127, 132, 133, 138, 142, 145, 146, 150, 155, 160, 165, 168, 170, 175, 179, 182, 186, 187, 191, 195, 200, 205, 211, 216, 221, 227, 233, 236, 241, 244, 249, 253, 259, 264, 267, 273, 281, 291, 296, 301, 309, 314, 320, 324, 336, 343, 346, 349, 354, 358, 362, 365, 370, 371, 376, 382, 388, 393, 399, 408, 414, 418, 425, 434, 441, 447, 453, 460, 466, 475], "exact_score": [7.641657905799261, 7.751224860383048, 7.876181366312142, 7.970885416231045, 7.871188629346748, 9.215434910654718, 9.199688313674494, 9.16618379321786, 9.191102937788239, 9.278572267399083, 9.378500811955695, 9.38432007623213, 9.448709190640695, 9.553116932872646, 10.421895235619687, 10.404777722322166, 10.421122383759013, 10.36104413505829, 10.465538336774845, 10.54049450659667, 11.132699469413465, 11.785588240555963, 11.730913664422445, 11.756653914481241, 11.864391756570065, 12.411438396487245, 12.468094405594405, 13.021349224674882, 13.086620034307566, 13.169765943701083, 13.291556525094428, 13.355625144170402, 13.419829065610667, 13.957597916500125, 14.455101037400489, 14.835194950071928, 14.932202087654927, 15.095449467041098, 15.543249944736798, 15.48287243509264, 15.691458420478796, 15.690430028306643, 15.89285802489906, 16.124697532092604, 16.16192020735517, 16.259685548395648, 16.483079904591534, 16.47129952157126, 16.605356436238246, 16.72077922077922, 16.89886195022234, 17.10201448781905, 17.30167424464784, 17.499584593107254, 17.695335710041594, 17.86042241818264, 17.994414357593868, 18.135256358775127, 18.23273220625956, 18.501518771742916, 18.782082033563142, 18.957061457061457, 19.230587476368726, 19.308041165184022, 19.387767671739795, 19.707327741926342, 19.781362881224183, 19.906842318755068, 19.70375682428056, 19.83164079682216, 20.10205514686158, 20.467145467145468, 20.831658004596605, 21.13978712890955, 21.33255491360587, 21.320900576561485, 21.757149177129527, 22.09509348306372, 21.943098196525597, 21.90598487228739, 21.8798590109504, 21.851357479614276, 21.826121190416796, 21.41812496042334, 21.40884883548237, 21.570142583373737, 21.532119592344554, 21.491128955783427, 21.447799041167126, 21.404624501553673, 21.367476568149744, 21.32631786866614, 21.30746417334186, 21.112263417163597, 20.9804345693785, 20.87099221623277, 20.76013776122802, 20.7243618450515, 20.675293266571156, 20.795821202798752, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.320460446639075, "threshold_area100": 16.314159791515394, "threshold_area25": 10.03159221309786}, "test": {"score": [7.8873978710439285, 7.938123344978387, 8.599244213648, 9.088503057398686, 9.027922701948954, 10.05412489533252, 10.672029498946197, 11.216999626695603, 11.310256448036416, 11.223232070048148, 11.331546624217909, 11.305551240003844, 11.518250148103853, 11.457553230522814, 11.840099454718063, 11.808417781290368, 11.874719301307927, 11.835200105608585, 11.705024196994009, 11.73718413474445, 11.745832503190433, 12.020828320916847, 12.036365386033806, 12.041951783256732, 12.201541455371279, 12.317761945024175, 12.400436389944128, 12.574045255706997, 12.724767639233047, 12.842057609436994, 12.946448950821404, 13.439228025950362, 13.462445616510513, 13.852422220725813, 14.034054669689286, 14.050182019467488, 14.080949869721355, 14.28208568457015, 14.490422473269055, 14.779566411774688, 14.898735341846875, 15.084285753875296, 15.41050122144708, 15.579394062088452, 15.737113127740129, 15.632929277781813, 15.906524151426337, 15.971078323356888, 16.279041775650132, 16.464587723287806, 16.539719453380165, 16.806770580324145, 16.881923455052185, 16.907387805444316, 17.170611105132824, 17.249237278465568, 17.287394911782865, 17.409618785700452, 17.521362089901626, 17.642329258398114, 17.807046712850617, 17.917373831211332, 17.95103182978675, 18.63310195556981, 18.755340603194046, 18.76449152629602, 19.425134169875733, 20.00880095055646, 20.231362781542973, 20.53029228106648, 21.26683250366985, 21.48012935602043, 21.274244957297668, 21.49800031464571, 21.650712520308517, 22.18264419238977, 22.502941612090805, 22.8248953805867, 23.043888595952684, 22.993625783632478, 22.95462328825931, 22.934444621976667, 23.266957262934227, 23.247649741125155, 23.163583724944285, 23.095547093750305, 23.015536532222946, 22.85831370272955, 22.826852789912692, 22.74807271671718, 22.692281330761475, 22.67006429302199, 22.582693368407654, 22.547031158142268, 22.502566823539443, 22.522074517379682, 22.47942936031285, 22.511020130029127, 22.481618160123222, 22.412865960495463, 22.361115727328066], "calls": [0.0, 0.4327563249001331, 1.1984021304926764, 1.6311584553928096, 2.130492676431425, 3.561917443408788, 4.027962716378163, 4.793608521970706, 6.125166444740346, 6.458055925432757, 7.256990679094541, 7.623169107856191, 8.455392809587217, 9.221038615179761, 10.019973368841544, 10.286284953395473, 11.8508655126498, 13.382157123834887, 14.547270306258323, 15.346205059920107, 15.745672436750999, 17.11051930758988, 18.442077230359523, 18.708388814913448, 20.47270306258322, 21.371504660452732, 22.503328894806923, 24.00133155792277, 24.73368841544607, 25.59920106524634, 26.830892143808256, 28.19573901464714, 29.360852197070574, 30.99201065246338, 32.35685752330226, 33.5219707057257, 34.42077230359521, 35.21970705725699, 36.81757656458056, 38.24900133155793, 38.81491344873502, 40.67909454061252, 42.143808255659124, 43.44207723035952, 44.50732356857524, 45.173102529960055, 46.97070572569907, 48.23568575233023, 50.06657789613848, 50.79893475366178, 51.76431424766977, 52.962716378162455, 53.59520639147804, 54.76031957390146, 56.22503328894807, 56.79094540612517, 57.689747003994675, 58.854860186418115, 60.05326231691078, 61.284953395472705, 62.18375499334221, 63.115845539280954, 63.78162450066578, 65.24633821571238, 65.77896138482025, 67.21038615179761, 68.37549933422103, 69.57390146471371, 70.1398135818908, 71.23834886817576, 72.50332889480693, 73.33555259653795, 73.73501997336885, 74.43408788282291, 75.0, 75.96537949400799, 76.79760319573901, 77.46338215712385, 78.82822902796272, 80.89214380825565, 82.25699067909454, 82.82290279627163, 83.48868175765645, 84.15446071904128, 85.18641810918774, 85.58588548601864, 86.28495339547271, 87.78295605858855, 88.41544607190413, 89.21438082556591, 90.31291611185085, 91.37816245006658, 91.94407456724367, 93.27563249001332, 94.47403462050599, 95.57256990679095, 96.33821571238349, 96.83754993342211, 98.1358189081225, 99.23435419440746, 100.0], "rescued": [0, 36, 101, 139, 181, 295, 332, 394, 494, 517, 576, 599, 656, 704, 751, 766, 847, 936, 992, 1034, 1054, 1123, 1192, 1205, 1287, 1331, 1388, 1460, 1493, 1536, 1588, 1657, 1713, 1784, 1845, 1894, 1936, 1974, 2049, 2115, 2143, 2227, 2297, 2344, 2385, 2412, 2473, 2511, 2562, 2586, 2612, 2644, 2662, 2689, 2731, 2747, 2770, 2802, 2833, 2865, 2889, 2909, 2924, 2966, 2980, 3013, 3045, 3075, 3092, 3119, 3155, 3175, 3184, 3201, 3213, 3238, 3259, 3278, 3303, 3305, 3305, 3305, 3306, 3306, 3307, 3307, 3307, 3307, 3307, 3307, 3307, 3309, 3309, 3310, 3310, 3311, 3311, 3312, 3314, 3314, 3314], "harmed": [0, 5, 23, 34, 47, 71, 78, 88, 109, 114, 130, 135, 147, 163, 176, 180, 210, 238, 261, 272, 276, 297, 317, 321, 342, 357, 381, 401, 415, 430, 455, 478, 498, 524, 546, 570, 588, 597, 625, 643, 655, 692, 718, 737, 755, 767, 793, 817, 853, 863, 885, 906, 914, 935, 960, 971, 986, 1016, 1043, 1075, 1092, 1116, 1130, 1163, 1174, 1213, 1236, 1264, 1281, 1299, 1324, 1350, 1358, 1375, 1383, 1403, 1421, 1437, 1463, 1499, 1519, 1530, 1544, 1554, 1579, 1588, 1605, 1641, 1660, 1679, 1703, 1730, 1745, 1770, 1803, 1827, 1843, 1855, 1891, 1924, 1945], "exact_score": [7.8873978710439285, 8.605616114853026, 9.029473847807077, 10.149860322130793, 10.667461647204822, 11.19016172744321, 11.344325529700285, 11.353009507379554, 11.293881318909923, 11.49756017080554, 11.631118360147736, 11.959624353290451, 11.85696289392525, 11.846411893259361, 11.817305551331184, 11.694136121078843, 11.91627125858658, 12.020466416796136, 12.005126117537129, 12.039535475853377, 12.039752712789968, 12.345217124530066, 12.428714547574065, 12.427047241910632, 12.570131556336037, 12.719076517563874, 12.847866814247777, 13.061375383327473, 13.3142687163661, 13.46894273621152, 13.584720671928185, 13.852422220725813, 14.012967535070016, 14.007841684256434, 14.088494219165751, 14.253756995717534, 14.414658705716374, 14.592500031885228, 14.740809731929122, 14.933371857867675, 15.014593067273175, 15.198765487131919, 15.387244474490663, 15.529826157891847, 15.644116773924422, 15.590006222478817, 15.745539425052717, 15.906524151426337, 15.955265196940912, 16.01813987353869, 16.27600159496675, 16.436119530049222, 16.58852617710013, 16.814480604668134, 16.920185226171906, 16.94139351334307, 17.163173160218022, 17.28429264245799, 17.333178312534965, 17.392823997516604, 17.508237314581613, 17.65036184224966, 17.725896874048175, 17.88163160400949, 17.9536399292574, 18.512242468740265, 18.517137408195794, 18.712461097829156, 18.990376213566435, 19.855192500126844, 20.213643965269373, 20.49158965743563, 20.741364510764186, 21.38625789611747, 21.348721012625386, 21.650712520308517, 22.134090117691706, 22.595308233419203, 22.825716629621372, 23.04160272492153, 22.97374756672434, 22.989687133561333, 22.962356055137523, 22.926836349767452, 23.25415478805968, 23.14714014976884, 23.026315873687246, 22.937608368661547, 22.84586166274585, 22.757358424147583, 22.721563458690028, 22.67468150759923, 22.535074320788606, 22.55732611345393, 22.52226818374528, 22.485734580269582, 22.510962017422507, 22.50680251833556, 22.44932440651126, 22.442871847682976, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.790349574366523, "threshold_area100": 16.776272970217587, "threshold_area25": 11.008500342413056}, "selected_budget": 77, "selected": {"val": {"score": 22.09509348306372, "calls": 76.8901569186876, "rescued": 783, "harmed": 346}, "test": {"score": 22.8248953805867, "calls": 77.46338215712385, "rescued": 3278, "harmed": 1437}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5426844954490662, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5270047187805176, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5149059295654297, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5082348585128784, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4876318573951721, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4827843904495239, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4726085066795349, "operator": "ge"}, {"budget": 0.08, "threshold": 0.44910871982574463, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4393736720085144, "operator": "ge"}, {"budget": 0.1, "threshold": 0.40735292434692383, "operator": "ge"}, {"budget": 0.11, "threshold": 0.37872883677482605, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3539283275604248, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3380585312843323, "operator": "ge"}, {"budget": 0.14, "threshold": 0.32391253113746643, "operator": "ge"}, {"budget": 0.15, "threshold": 0.31818974018096924, "operator": "ge"}, {"budget": 0.16, "threshold": 0.30115869641304016, "operator": "ge"}, {"budget": 0.17, "threshold": 0.29165932536125183, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2850906252861023, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2817002236843109, "operator": "ge"}, {"budget": 0.2, "threshold": 0.27978020906448364, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2754681408405304, "operator": "ge"}, {"budget": 0.22, "threshold": 0.2718718647956848, "operator": "ge"}, {"budget": 0.23, "threshold": 0.27121150493621826, "operator": "ge"}, {"budget": 0.24, "threshold": 0.26724565029144287, "operator": "ge"}, {"budget": 0.25, "threshold": 0.26565009355545044, "operator": "ge"}, {"budget": 0.26, "threshold": 0.2629561424255371, "operator": "ge"}, {"budget": 0.27, "threshold": 0.26052623987197876, "operator": "ge"}, {"budget": 0.28, "threshold": 0.2590908408164978, "operator": "ge"}, {"budget": 0.29, "threshold": 0.25725722312927246, "operator": "ge"}, {"budget": 0.3, "threshold": 0.2546693980693817, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2517916262149811, "operator": "ge"}, {"budget": 0.32, "threshold": 0.24886560440063477, "operator": "ge"}, {"budget": 0.33, "threshold": 0.24377061426639557, "operator": "ge"}, {"budget": 0.34, "threshold": 0.24020269513130188, "operator": "ge"}, {"budget": 0.35, "threshold": 0.2345752865076065, "operator": "ge"}, {"budget": 0.36, "threshold": 0.2324848473072052, "operator": "ge"}, {"budget": 0.37, "threshold": 0.22940993309020996, "operator": "ge"}, {"budget": 0.38, "threshold": 0.22276505827903748, "operator": "ge"}, {"budget": 0.39, "threshold": 0.2150501310825348, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2106122374534607, "operator": "ge"}, {"budget": 0.41, "threshold": 0.1978839486837387, "operator": "ge"}, {"budget": 0.42, "threshold": 0.1836877465248108, "operator": "ge"}, {"budget": 0.43, "threshold": 0.17422938346862793, "operator": "ge"}, {"budget": 0.44, "threshold": 0.16241896152496338, "operator": "ge"}, {"budget": 0.45, "threshold": 0.15406842529773712, "operator": "ge"}, {"budget": 0.46, "threshold": 0.13724160194396973, "operator": "ge"}, {"budget": 0.47, "threshold": 0.1279979646205902, "operator": "ge"}, {"budget": 0.48, "threshold": 0.11859403550624847, "operator": "ge"}, {"budget": 0.49, "threshold": 0.11529508233070374, "operator": "ge"}, {"budget": 0.5, "threshold": 0.11066114902496338, "operator": "ge"}, {"budget": 0.51, "threshold": 0.10685388743877411, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1050761267542839, "operator": "ge"}, {"budget": 0.53, "threshold": 0.10178354382514954, "operator": "ge"}, {"budget": 0.54, "threshold": 0.09797769784927368, "operator": "ge"}, {"budget": 0.55, "threshold": 0.09530256688594818, "operator": "ge"}, {"budget": 0.56, "threshold": 0.09328354895114899, "operator": "ge"}, {"budget": 0.57, "threshold": 0.0910453200340271, "operator": "ge"}, {"budget": 0.58, "threshold": 0.08741492033004761, "operator": "ge"}, {"budget": 0.59, "threshold": 0.0844215452671051, "operator": "ge"}, {"budget": 0.6, "threshold": 0.08227072656154633, "operator": "ge"}, {"budget": 0.61, "threshold": 0.08010168373584747, "operator": "ge"}, {"budget": 0.62, "threshold": 0.07787324488162994, "operator": "ge"}, {"budget": 0.63, "threshold": 0.07330247759819031, "operator": "ge"}, {"budget": 0.64, "threshold": 0.0718550980091095, "operator": "ge"}, {"budget": 0.65, "threshold": 0.06675291061401367, "operator": "ge"}, {"budget": 0.66, "threshold": 0.06311599910259247, "operator": "ge"}, {"budget": 0.67, "threshold": 0.05742239952087402, "operator": "ge"}, {"budget": 0.68, "threshold": 0.05470120906829834, "operator": "ge"}, {"budget": 0.69, "threshold": 0.048705294728279114, "operator": "ge"}, {"budget": 0.7, "threshold": 0.04292312264442444, "operator": "ge"}, {"budget": 0.71, "threshold": 0.037435874342918396, "operator": "ge"}, {"budget": 0.72, "threshold": 0.03282877802848816, "operator": "ge"}, {"budget": 0.73, "threshold": 0.026126623153686523, "operator": "ge"}, {"budget": 0.74, "threshold": 0.021005868911743164, "operator": "ge"}, {"budget": 0.75, "threshold": 0.013213291764259338, "operator": "ge"}, {"budget": 0.76, "threshold": 0.0030574798583984375, "operator": "ge"}, {"budget": 0.77, "threshold": -0.008658885955810547, "operator": "ge"}, {"budget": 0.78, "threshold": -0.09074939042329788, "operator": "ge"}, {"budget": 0.79, "threshold": -0.10961824655532837, "operator": "ge"}, {"budget": 0.8, "threshold": -0.11667399853467941, "operator": "ge"}, {"budget": 0.81, "threshold": -0.11903789639472961, "operator": "ge"}, {"budget": 0.82, "threshold": -0.12213939428329468, "operator": "ge"}, {"budget": 0.83, "threshold": -0.1256289780139923, "operator": "ge"}, {"budget": 0.84, "threshold": -0.1288442462682724, "operator": "ge"}, {"budget": 0.85, "threshold": -0.13026514649391174, "operator": "ge"}, {"budget": 0.86, "threshold": -0.13201916217803955, "operator": "ge"}, {"budget": 0.87, "threshold": -0.13581186532974243, "operator": "ge"}, {"budget": 0.88, "threshold": -0.1381959319114685, "operator": "ge"}, {"budget": 0.89, "threshold": -0.1402575969696045, "operator": "ge"}, {"budget": 0.9, "threshold": -0.14238020777702332, "operator": "ge"}, {"budget": 0.91, "threshold": -0.14441955089569092, "operator": "ge"}, {"budget": 0.92, "threshold": -0.14597465097904205, "operator": "ge"}, {"budget": 0.93, "threshold": -0.14858360588550568, "operator": "ge"}, {"budget": 0.94, "threshold": -0.1504906266927719, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1529204249382019, "operator": "ge"}, {"budget": 0.96, "threshold": -0.15487785637378693, "operator": "ge"}, {"budget": 0.97, "threshold": -0.15640202164649963, "operator": "ge"}, {"budget": 0.98, "threshold": -0.15996035933494568, "operator": "ge"}, {"budget": 0.99, "threshold": -0.16852501034736633, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.751224860383048, 7.876181366312142, 7.970885416231045, 7.871188629346748, 9.215434910654718, 9.199688313674494, 9.16618379321786, 9.191102937788239, 9.278572267399083, 9.378500811955695, 9.38432007623213, 9.448709190640695, 9.553116932872646, 10.421895235619687, 10.404777722322166, 10.421122383759013, 10.36104413505829, 10.465538336774845, 10.54049450659667, 11.132699469413465, 11.785588240555963, 11.730913664422445, 11.756653914481241, 11.864391756570065, 12.411438396487245, 12.468094405594405, 13.021349224674882, 13.086620034307566, 13.169765943701083, 13.291556525094428, 13.355625144170402, 13.419829065610667, 13.957597916500125, 14.455101037400489, 14.835194950071928, 14.932202087654927, 15.095449467041098, 15.543249944736798, 15.48287243509264, 15.691458420478796, 15.690430028306643, 15.89285802489906, 16.124697532092604, 16.16192020735517, 16.259685548395648, 16.483079904591534, 16.47129952157126, 16.605356436238246, 16.72077922077922, 16.89886195022234, 17.10201448781905, 17.30167424464784, 17.499584593107254, 17.695335710041594, 17.86042241818264, 17.994414357593868, 18.135256358775127, 18.23273220625956, 18.501518771742916, 18.782082033563142, 18.957061457061457, 19.230587476368726, 19.308041165184022, 19.387767671739795, 19.707327741926342, 19.781362881224183, 19.906842318755068, 19.70375682428056, 19.83164079682216, 20.10205514686158, 20.467145467145468, 20.831658004596605, 21.13978712890955, 21.33255491360587, 21.320900576561485, 21.757149177129527, 22.09509348306372, 21.943098196525597, 21.90598487228739, 21.8798590109504, 21.851357479614276, 21.826121190416796, 21.41812496042334, 21.40884883548237, 21.570142583373737, 21.532119592344554, 21.491128955783427, 21.447799041167126, 21.404624501553673, 21.367476568149744, 21.32631786866614, 21.30746417334186, 21.112263417163597, 20.9804345693785, 20.87099221623277, 20.76013776122802, 20.7243618450515, 20.675293266571156, 20.795821202798752, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 20, 41, 59, 77, 94, 112, 129, 148, 163, 180, 198, 211, 223, 237, 251, 262, 276, 289, 300, 314, 326, 335, 345, 357, 370, 383, 397, 409, 421, 430, 441, 453, 464, 476, 487, 497, 505, 517, 526, 537, 547, 558, 569, 577, 586, 595, 602, 607, 614, 621, 628, 636, 643, 651, 656, 664, 670, 678, 685, 692, 698, 704, 709, 715, 721, 726, 731, 736, 742, 749, 756, 761, 767, 771, 775, 780, 783, 784, 784, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786], "harmed": [0, 6, 12, 18, 26, 30, 35, 39, 44, 47, 49, 54, 57, 60, 65, 69, 73, 78, 81, 85, 90, 93, 96, 99, 100, 104, 109, 113, 118, 121, 122, 127, 132, 133, 138, 142, 145, 146, 150, 155, 160, 165, 168, 170, 175, 179, 182, 186, 187, 191, 195, 200, 205, 211, 216, 221, 227, 233, 236, 241, 244, 249, 253, 259, 264, 267, 273, 281, 291, 296, 301, 309, 314, 320, 324, 336, 343, 346, 349, 354, 358, 362, 365, 370, 371, 376, 382, 388, 393, 399, 408, 414, 418, 425, 434, 441, 447, 453, 460, 466, 475], "exact_score": [7.641657905799261, 7.751224860383048, 7.876181366312142, 7.970885416231045, 7.871188629346748, 9.215434910654718, 9.199688313674494, 9.16618379321786, 9.191102937788239, 9.278572267399083, 9.378500811955695, 9.38432007623213, 9.448709190640695, 9.553116932872646, 10.421895235619687, 10.404777722322166, 10.421122383759013, 10.36104413505829, 10.465538336774845, 10.54049450659667, 11.132699469413465, 11.785588240555963, 11.730913664422445, 11.756653914481241, 11.864391756570065, 12.411438396487245, 12.468094405594405, 13.021349224674882, 13.086620034307566, 13.169765943701083, 13.291556525094428, 13.355625144170402, 13.419829065610667, 13.957597916500125, 14.455101037400489, 14.835194950071928, 14.932202087654927, 15.095449467041098, 15.543249944736798, 15.48287243509264, 15.691458420478796, 15.690430028306643, 15.89285802489906, 16.124697532092604, 16.16192020735517, 16.259685548395648, 16.483079904591534, 16.47129952157126, 16.605356436238246, 16.72077922077922, 16.89886195022234, 17.10201448781905, 17.30167424464784, 17.499584593107254, 17.695335710041594, 17.86042241818264, 17.994414357593868, 18.135256358775127, 18.23273220625956, 18.501518771742916, 18.782082033563142, 18.957061457061457, 19.230587476368726, 19.308041165184022, 19.387767671739795, 19.707327741926342, 19.781362881224183, 19.906842318755068, 19.70375682428056, 19.83164079682216, 20.10205514686158, 20.467145467145468, 20.831658004596605, 21.13978712890955, 21.33255491360587, 21.320900576561485, 21.757149177129527, 22.09509348306372, 21.943098196525597, 21.90598487228739, 21.8798590109504, 21.851357479614276, 21.826121190416796, 21.41812496042334, 21.40884883548237, 21.570142583373737, 21.532119592344554, 21.491128955783427, 21.447799041167126, 21.404624501553673, 21.367476568149744, 21.32631786866614, 21.30746417334186, 21.112263417163597, 20.9804345693785, 20.87099221623277, 20.76013776122802, 20.7243618450515, 20.675293266571156, 20.795821202798752, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.320460446639075, "threshold_area100": 16.314159791515394, "threshold_area25": 10.03159221309786}, "test": {"score": [7.8873978710439285, 7.938123344978387, 8.599244213648, 9.088503057398686, 9.027922701948954, 10.05412489533252, 10.672029498946197, 11.216999626695603, 11.310256448036416, 11.223232070048148, 11.331546624217909, 11.305551240003844, 11.518250148103853, 11.457553230522814, 11.840099454718063, 11.808417781290368, 11.874719301307927, 11.835200105608585, 11.705024196994009, 11.73718413474445, 11.745832503190433, 12.020828320916847, 12.036365386033806, 12.041951783256732, 12.201541455371279, 12.317761945024175, 12.400436389944128, 12.574045255706997, 12.724767639233047, 12.842057609436994, 12.946448950821404, 13.439228025950362, 13.462445616510513, 13.852422220725813, 14.034054669689286, 14.050182019467488, 14.080949869721355, 14.28208568457015, 14.490422473269055, 14.779566411774688, 14.898735341846875, 15.084285753875296, 15.41050122144708, 15.579394062088452, 15.737113127740129, 15.632929277781813, 15.906524151426337, 15.971078323356888, 16.279041775650132, 16.464587723287806, 16.539719453380165, 16.806770580324145, 16.881923455052185, 16.907387805444316, 17.170611105132824, 17.249237278465568, 17.287394911782865, 17.409618785700452, 17.521362089901626, 17.642329258398114, 17.807046712850617, 17.917373831211332, 17.95103182978675, 18.63310195556981, 18.755340603194046, 18.76449152629602, 19.425134169875733, 20.00880095055646, 20.231362781542973, 20.53029228106648, 21.26683250366985, 21.48012935602043, 21.274244957297668, 21.49800031464571, 21.650712520308517, 22.18264419238977, 22.502941612090805, 22.8248953805867, 23.043888595952684, 22.993625783632478, 22.95462328825931, 22.934444621976667, 23.266957262934227, 23.247649741125155, 23.163583724944285, 23.095547093750305, 23.015536532222946, 22.85831370272955, 22.826852789912692, 22.74807271671718, 22.692281330761475, 22.67006429302199, 22.582693368407654, 22.547031158142268, 22.502566823539443, 22.522074517379682, 22.47942936031285, 22.511020130029127, 22.481618160123222, 22.412865960495463, 22.361115727328066], "calls": [0.0, 0.4327563249001331, 1.1984021304926764, 1.6311584553928096, 2.130492676431425, 3.561917443408788, 4.027962716378163, 4.793608521970706, 6.125166444740346, 6.458055925432757, 7.256990679094541, 7.623169107856191, 8.455392809587217, 9.221038615179761, 10.019973368841544, 10.286284953395473, 11.8508655126498, 13.382157123834887, 14.547270306258323, 15.346205059920107, 15.745672436750999, 17.11051930758988, 18.442077230359523, 18.708388814913448, 20.47270306258322, 21.371504660452732, 22.503328894806923, 24.00133155792277, 24.73368841544607, 25.59920106524634, 26.830892143808256, 28.19573901464714, 29.360852197070574, 30.99201065246338, 32.35685752330226, 33.5219707057257, 34.42077230359521, 35.21970705725699, 36.81757656458056, 38.24900133155793, 38.81491344873502, 40.67909454061252, 42.143808255659124, 43.44207723035952, 44.50732356857524, 45.173102529960055, 46.97070572569907, 48.23568575233023, 50.06657789613848, 50.79893475366178, 51.76431424766977, 52.962716378162455, 53.59520639147804, 54.76031957390146, 56.22503328894807, 56.79094540612517, 57.689747003994675, 58.854860186418115, 60.05326231691078, 61.284953395472705, 62.18375499334221, 63.115845539280954, 63.78162450066578, 65.24633821571238, 65.77896138482025, 67.21038615179761, 68.37549933422103, 69.57390146471371, 70.1398135818908, 71.23834886817576, 72.50332889480693, 73.33555259653795, 73.73501997336885, 74.43408788282291, 75.0, 75.96537949400799, 76.79760319573901, 77.46338215712385, 78.82822902796272, 80.89214380825565, 82.25699067909454, 82.82290279627163, 83.48868175765645, 84.15446071904128, 85.18641810918774, 85.58588548601864, 86.28495339547271, 87.78295605858855, 88.41544607190413, 89.21438082556591, 90.31291611185085, 91.37816245006658, 91.94407456724367, 93.27563249001332, 94.47403462050599, 95.57256990679095, 96.33821571238349, 96.83754993342211, 98.1358189081225, 99.23435419440746, 100.0], "rescued": [0, 36, 101, 139, 181, 295, 332, 394, 494, 517, 576, 599, 656, 704, 751, 766, 847, 936, 992, 1034, 1054, 1123, 1192, 1205, 1287, 1331, 1388, 1460, 1493, 1536, 1588, 1657, 1713, 1784, 1845, 1894, 1936, 1974, 2049, 2115, 2143, 2227, 2297, 2344, 2385, 2412, 2473, 2511, 2562, 2586, 2612, 2644, 2662, 2689, 2731, 2747, 2770, 2802, 2833, 2865, 2889, 2909, 2924, 2966, 2980, 3013, 3045, 3075, 3092, 3119, 3155, 3175, 3184, 3201, 3213, 3238, 3259, 3278, 3303, 3305, 3305, 3305, 3306, 3306, 3307, 3307, 3307, 3307, 3307, 3307, 3307, 3309, 3309, 3310, 3310, 3311, 3311, 3312, 3314, 3314, 3314], "harmed": [0, 5, 23, 34, 47, 71, 78, 88, 109, 114, 130, 135, 147, 163, 176, 180, 210, 238, 261, 272, 276, 297, 317, 321, 342, 357, 381, 401, 415, 430, 455, 478, 498, 524, 546, 570, 588, 597, 625, 643, 655, 692, 718, 737, 755, 767, 793, 817, 853, 863, 885, 906, 914, 935, 960, 971, 986, 1016, 1043, 1075, 1092, 1116, 1130, 1163, 1174, 1213, 1236, 1264, 1281, 1299, 1324, 1350, 1358, 1375, 1383, 1403, 1421, 1437, 1463, 1499, 1519, 1530, 1544, 1554, 1579, 1588, 1605, 1641, 1660, 1679, 1703, 1730, 1745, 1770, 1803, 1827, 1843, 1855, 1891, 1924, 1945], "exact_score": [7.8873978710439285, 8.605616114853026, 9.029473847807077, 10.149860322130793, 10.667461647204822, 11.19016172744321, 11.344325529700285, 11.353009507379554, 11.293881318909923, 11.49756017080554, 11.631118360147736, 11.959624353290451, 11.85696289392525, 11.846411893259361, 11.817305551331184, 11.694136121078843, 11.91627125858658, 12.020466416796136, 12.005126117537129, 12.039535475853377, 12.039752712789968, 12.345217124530066, 12.428714547574065, 12.427047241910632, 12.570131556336037, 12.719076517563874, 12.847866814247777, 13.061375383327473, 13.3142687163661, 13.46894273621152, 13.584720671928185, 13.852422220725813, 14.012967535070016, 14.007841684256434, 14.088494219165751, 14.253756995717534, 14.414658705716374, 14.592500031885228, 14.740809731929122, 14.933371857867675, 15.014593067273175, 15.198765487131919, 15.387244474490663, 15.529826157891847, 15.644116773924422, 15.590006222478817, 15.745539425052717, 15.906524151426337, 15.955265196940912, 16.01813987353869, 16.27600159496675, 16.436119530049222, 16.58852617710013, 16.814480604668134, 16.920185226171906, 16.94139351334307, 17.163173160218022, 17.28429264245799, 17.333178312534965, 17.392823997516604, 17.508237314581613, 17.65036184224966, 17.725896874048175, 17.88163160400949, 17.9536399292574, 18.512242468740265, 18.517137408195794, 18.712461097829156, 18.990376213566435, 19.855192500126844, 20.213643965269373, 20.49158965743563, 20.741364510764186, 21.38625789611747, 21.348721012625386, 21.650712520308517, 22.134090117691706, 22.595308233419203, 22.825716629621372, 23.04160272492153, 22.97374756672434, 22.989687133561333, 22.962356055137523, 22.926836349767452, 23.25415478805968, 23.14714014976884, 23.026315873687246, 22.937608368661547, 22.84586166274585, 22.757358424147583, 22.721563458690028, 22.67468150759923, 22.535074320788606, 22.55732611345393, 22.52226818374528, 22.485734580269582, 22.510962017422507, 22.50680251833556, 22.44932440651126, 22.442871847682976, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.790349574366523, "threshold_area100": 16.776272970217587, "threshold_area25": 11.008500342413056}, "selected_budget": 77, "selected": {"val": {"score": 22.09509348306372, "calls": 76.8901569186876, "rescued": 783, "harmed": 346}, "test": {"score": 22.8248953805867, "calls": 77.46338215712385, "rescued": 3278, "harmed": 1437}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.9382147789001465, "operator": "ge"}, {"budget": 0.02, "threshold": 1.8028173446655273, "operator": "ge"}, {"budget": 0.03, "threshold": 1.710150957107544, "operator": "ge"}, {"budget": 0.04, "threshold": 1.6501473188400269, "operator": "ge"}, {"budget": 0.05, "threshold": 1.6193727254867554, "operator": "ge"}, {"budget": 0.06, "threshold": 1.5810956954956055, "operator": "ge"}, {"budget": 0.07, "threshold": 1.5479881763458252, "operator": "ge"}, {"budget": 0.08, "threshold": 1.5231451988220215, "operator": "ge"}, {"budget": 0.09, "threshold": 1.4885270595550537, "operator": "ge"}, {"budget": 0.1, "threshold": 1.4449381828308105, "operator": "ge"}, {"budget": 0.11, "threshold": 1.413855791091919, "operator": "ge"}, {"budget": 0.12, "threshold": 1.3887404203414917, "operator": "ge"}, {"budget": 0.13, "threshold": 1.3696919679641724, "operator": "ge"}, {"budget": 0.14, "threshold": 1.3500869274139404, "operator": "ge"}, {"budget": 0.15, "threshold": 1.3382192850112915, "operator": "ge"}, {"budget": 0.16, "threshold": 1.317222237586975, "operator": "ge"}, {"budget": 0.17, "threshold": 1.3042309284210205, "operator": "ge"}, {"budget": 0.18, "threshold": 1.2866365909576416, "operator": "ge"}, {"budget": 0.19, "threshold": 1.2621557712554932, "operator": "ge"}, {"budget": 0.2, "threshold": 1.2449349164962769, "operator": "ge"}, {"budget": 0.21, "threshold": 1.2206860780715942, "operator": "ge"}, {"budget": 0.22, "threshold": 1.2136338949203491, "operator": "ge"}, {"budget": 0.23, "threshold": 1.1985304355621338, "operator": "ge"}, {"budget": 0.24, "threshold": 1.1814137697219849, "operator": "ge"}, {"budget": 0.25, "threshold": 1.1606147289276123, "operator": "ge"}, {"budget": 0.26, "threshold": 1.1383564472198486, "operator": "ge"}, {"budget": 0.27, "threshold": 1.1106244325637817, "operator": "ge"}, {"budget": 0.28, "threshold": 1.094504952430725, "operator": "ge"}, {"budget": 0.29, "threshold": 1.0857057571411133, "operator": "ge"}, {"budget": 0.3, "threshold": 1.069509506225586, "operator": "ge"}, {"budget": 0.31, "threshold": 1.0570018291473389, "operator": "ge"}, {"budget": 0.32, "threshold": 1.0369269847869873, "operator": "ge"}, {"budget": 0.33, "threshold": 1.0166759490966797, "operator": "ge"}, {"budget": 0.34, "threshold": 1.0019415616989136, "operator": "ge"}, {"budget": 0.35, "threshold": 0.9827817678451538, "operator": "ge"}, {"budget": 0.36, "threshold": 0.9455773830413818, "operator": "ge"}, {"budget": 0.37, "threshold": 0.9110447764396667, "operator": "ge"}, {"budget": 0.38, "threshold": 0.8855770826339722, "operator": "ge"}, {"budget": 0.39, "threshold": 0.8517418503761292, "operator": "ge"}, {"budget": 0.4, "threshold": 0.8366825580596924, "operator": "ge"}, {"budget": 0.41, "threshold": 0.811823308467865, "operator": "ge"}, {"budget": 0.42, "threshold": 0.7894854545593262, "operator": "ge"}, {"budget": 0.43, "threshold": 0.7690551280975342, "operator": "ge"}, {"budget": 0.44, "threshold": 0.7573388814926147, "operator": "ge"}, {"budget": 0.45, "threshold": 0.7244023680686951, "operator": "ge"}, {"budget": 0.46, "threshold": 0.6975974440574646, "operator": "ge"}, {"budget": 0.47, "threshold": 0.6769745945930481, "operator": "ge"}, {"budget": 0.48, "threshold": 0.6419513821601868, "operator": "ge"}, {"budget": 0.49, "threshold": 0.6227978467941284, "operator": "ge"}, {"budget": 0.5, "threshold": 0.6023613214492798, "operator": "ge"}, {"budget": 0.51, "threshold": 0.5746973752975464, "operator": "ge"}, {"budget": 0.52, "threshold": 0.5541977286338806, "operator": "ge"}, {"budget": 0.53, "threshold": 0.5437400341033936, "operator": "ge"}, {"budget": 0.54, "threshold": 0.5225262641906738, "operator": "ge"}, {"budget": 0.55, "threshold": 0.49915486574172974, "operator": "ge"}, {"budget": 0.56, "threshold": 0.48610737919807434, "operator": "ge"}, {"budget": 0.57, "threshold": 0.468121200799942, "operator": "ge"}, {"budget": 0.58, "threshold": 0.4559691548347473, "operator": "ge"}, {"budget": 0.59, "threshold": 0.44448965787887573, "operator": "ge"}, {"budget": 0.6, "threshold": 0.43383288383483887, "operator": "ge"}, {"budget": 0.61, "threshold": 0.42048290371894836, "operator": "ge"}, {"budget": 0.62, "threshold": 0.40442168712615967, "operator": "ge"}, {"budget": 0.63, "threshold": 0.3894663453102112, "operator": "ge"}, {"budget": 0.64, "threshold": 0.37468427419662476, "operator": "ge"}, {"budget": 0.65, "threshold": 0.34603995084762573, "operator": "ge"}, {"budget": 0.66, "threshold": 0.32948750257492065, "operator": "ge"}, {"budget": 0.67, "threshold": 0.30002304911613464, "operator": "ge"}, {"budget": 0.68, "threshold": 0.2725852131843567, "operator": "ge"}, {"budget": 0.69, "threshold": 0.24521607160568237, "operator": "ge"}, {"budget": 0.7, "threshold": 0.2268752157688141, "operator": "ge"}, {"budget": 0.71, "threshold": 0.1901794672012329, "operator": "ge"}, {"budget": 0.72, "threshold": 0.17468130588531494, "operator": "ge"}, {"budget": 0.73, "threshold": 0.13488662242889404, "operator": "ge"}, {"budget": 0.74, "threshold": 0.1173647865653038, "operator": "ge"}, {"budget": 0.75, "threshold": 0.06766912341117859, "operator": "ge"}, {"budget": 0.76, "threshold": 0.01663830876350403, "operator": "ge"}, {"budget": 0.77, "threshold": -0.0527576208114624, "operator": "ge"}, {"budget": 0.78, "threshold": -2.2806484699249268, "operator": "ge"}, {"budget": 0.79, "threshold": -2.4127557277679443, "operator": "ge"}, {"budget": 0.8, "threshold": -2.4642016887664795, "operator": "ge"}, {"budget": 0.81, "threshold": -2.5016815662384033, "operator": "ge"}, {"budget": 0.82, "threshold": -2.58329176902771, "operator": "ge"}, {"budget": 0.83, "threshold": -2.613555431365967, "operator": "ge"}, {"budget": 0.84, "threshold": -2.619657039642334, "operator": "ge"}, {"budget": 0.85, "threshold": -2.6516408920288086, "operator": "ge"}, {"budget": 0.86, "threshold": -2.7331957817077637, "operator": "ge"}, {"budget": 0.87, "threshold": -2.762573719024658, "operator": "ge"}, {"budget": 0.88, "threshold": -2.788424253463745, "operator": "ge"}, {"budget": 0.89, "threshold": -2.8135745525360107, "operator": "ge"}, {"budget": 0.9, "threshold": -2.8411507606506348, "operator": "ge"}, {"budget": 0.91, "threshold": -2.879235029220581, "operator": "ge"}, {"budget": 0.92, "threshold": -2.913012742996216, "operator": "ge"}, {"budget": 0.93, "threshold": -2.9610795974731445, "operator": "ge"}, {"budget": 0.94, "threshold": -3.010089635848999, "operator": "ge"}, {"budget": 0.95, "threshold": -3.037154197692871, "operator": "ge"}, {"budget": 0.96, "threshold": -3.0716378688812256, "operator": "ge"}, {"budget": 0.97, "threshold": -3.116454601287842, "operator": "ge"}, {"budget": 0.98, "threshold": -3.1629786491394043, "operator": "ge"}, {"budget": 0.99, "threshold": -3.2498269081115723, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.740411126941578, 10.475223863972328, 10.417350974123211, 10.014413797256296, 9.651361785510563, 9.602889952968512, 9.426421410215244, 10.839849043497209, 10.634523147465384, 10.498736678820386, 10.283473034932557, 10.287875657796821, 10.297325222779271, 10.272752458006117, 11.13565642716143, 11.134334144304185, 11.185687950393833, 11.157878879803484, 11.193063287517917, 11.17390593789148, 11.218441991436121, 11.289414414414416, 11.367627876870417, 12.037858645895193, 12.108713613134075, 12.153260846815932, 12.285659248038836, 12.36262723965156, 12.342669151246788, 12.853060767112762, 13.299867357634017, 13.266659017746674, 13.805285733186622, 13.93261165339499, 13.863863444307272, 14.358840100571335, 14.428642325409264, 14.433470217738636, 14.611353559693146, 15.09162706349017, 15.206381079875056, 15.687325300256422, 15.842990997073139, 15.75178789464504, 15.952814985073049, 16.07117306437407, 16.015777164891162, 16.120112912307366, 16.268706622321407, 16.43527955956609, 16.61328456458181, 16.662722435420708, 16.836259772496827, 17.06415532255078, 17.160728748285152, 17.40786259268448, 17.574912750593093, 17.908719043172823, 18.072930791341278, 18.349859583146753, 18.408867494856274, 18.77186249526675, 19.05161141577026, 19.11894543557742, 19.12549249112291, 19.46409644939057, 19.52307946177951, 19.618855802739017, 19.7276765018481, 20.0487423003556, 20.488468256448346, 20.84642450380182, 21.143029218566028, 20.98445920812464, 21.33134295362623, 21.757149177129527, 22.09509348306372, 21.824479532135985, 21.609681613206167, 21.557850449416712, 21.32664858568473, 21.47800487900619, 21.429319959074338, 21.390717777522664, 21.347165027381937, 21.29948620558117, 21.266259198669957, 21.236683867447763, 20.97074894098949, 20.845915600794353, 20.81029581158248, 20.77777681073648, 20.741189677413313, 20.64796620427459, 20.62435261234647, 20.589627092442992, 20.736418565822838, 20.71423678900564, 20.69289935888572, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 16, 27, 40, 51, 67, 83, 101, 118, 133, 148, 163, 181, 195, 210, 223, 237, 248, 264, 278, 288, 301, 315, 326, 339, 349, 360, 373, 383, 396, 411, 423, 434, 448, 460, 470, 480, 490, 499, 512, 523, 533, 542, 554, 563, 572, 580, 587, 595, 603, 613, 621, 629, 637, 644, 651, 660, 665, 673, 678, 685, 690, 698, 704, 710, 717, 724, 729, 735, 741, 748, 754, 760, 766, 770, 775, 780, 783, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786], "harmed": [0, 2, 7, 9, 13, 18, 21, 26, 28, 33, 37, 43, 47, 50, 55, 56, 61, 63, 68, 72, 75, 79, 83, 85, 87, 89, 93, 94, 96, 102, 107, 111, 117, 121, 125, 130, 131, 135, 142, 146, 149, 153, 155, 161, 168, 171, 175, 180, 184, 188, 194, 198, 204, 208, 211, 218, 222, 226, 231, 235, 240, 246, 251, 254, 260, 267, 273, 283, 289, 295, 301, 307, 312, 319, 328, 335, 343, 346, 355, 364, 370, 375, 380, 386, 391, 397, 403, 408, 412, 419, 428, 434, 439, 445, 449, 453, 459, 462, 465, 470, 475], "exact_score": [7.641657905799261, 7.740411126941578, 10.475223863972328, 10.417350974123211, 10.014413797256296, 9.651361785510563, 9.602889952968512, 9.426421410215244, 10.839849043497209, 10.634523147465384, 10.498736678820386, 10.283473034932557, 10.287875657796821, 10.297325222779271, 10.272752458006117, 11.13565642716143, 11.134334144304185, 11.185687950393833, 11.157878879803484, 11.193063287517917, 11.17390593789148, 11.218441991436121, 11.289414414414416, 11.367627876870417, 12.037858645895193, 12.108713613134075, 12.153260846815932, 12.285659248038836, 12.36262723965156, 12.342669151246788, 12.853060767112762, 13.299867357634017, 13.266659017746674, 13.805285733186622, 13.93261165339499, 13.863863444307272, 14.358840100571335, 14.428642325409264, 14.433470217738636, 14.611353559693146, 15.09162706349017, 15.206381079875056, 15.687325300256422, 15.842990997073139, 15.75178789464504, 15.952814985073049, 16.07117306437407, 16.015777164891162, 16.120112912307366, 16.268706622321407, 16.43527955956609, 16.61328456458181, 16.662722435420708, 16.836259772496827, 17.06415532255078, 17.160728748285152, 17.40786259268448, 17.574912750593093, 17.908719043172823, 18.072930791341278, 18.349859583146753, 18.408867494856274, 18.77186249526675, 19.05161141577026, 19.11894543557742, 19.12549249112291, 19.46409644939057, 19.52307946177951, 19.618855802739017, 19.7276765018481, 20.0487423003556, 20.488468256448346, 20.84642450380182, 21.143029218566028, 20.98445920812464, 21.33134295362623, 21.757149177129527, 22.09509348306372, 21.824479532135985, 21.609681613206167, 21.557850449416712, 21.32664858568473, 21.47800487900619, 21.429319959074338, 21.390717777522664, 21.347165027381937, 21.29948620558117, 21.266259198669957, 21.236683867447763, 20.97074894098949, 20.845915600794353, 20.81029581158248, 20.77777681073648, 20.741189677413313, 20.64796620427459, 20.62435261234647, 20.589627092442992, 20.736418565822838, 20.71423678900564, 20.69289935888572, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.217526516266897, "threshold_area100": 16.211152328274054, "threshold_area25": 10.466969317770108}, "test": {"score": [7.8873978710439285, 8.708295766734794, 9.416782471041563, 9.765275591888493, 9.622609221966103, 9.57054183247174, 9.522107081096113, 10.672213573529742, 10.57421207320792, 10.465046240216411, 10.402663278451492, 10.657824659456532, 10.608710473626068, 10.604787596803913, 10.605967370791365, 10.82462780262369, 11.02936043081724, 11.22285318420844, 11.20438077737854, 11.185898401731038, 11.143427794904612, 11.446510429080483, 11.432178496700466, 11.595149547693568, 11.59529456410105, 11.623472867929893, 11.738924678480194, 12.183676383885567, 12.487435019688434, 12.526195684033178, 12.588772347585314, 12.921281281632382, 13.152737862432154, 13.212732726060702, 13.212378278419425, 13.361405260777811, 13.614804090541021, 13.778208377932119, 13.863938224979963, 14.060321078030105, 14.301630319556152, 14.594056857949116, 14.814037452966513, 14.804121544729455, 14.996702908743623, 15.111081870645707, 15.128109161335567, 15.128626112517217, 15.442657421713623, 15.562054946152537, 15.533068569747623, 15.658413077853355, 15.996494304241299, 15.988432091297666, 16.29433603501984, 16.57309158678725, 16.72662950514053, 17.06856378982142, 17.061795428167258, 17.147419148565476, 17.302575659647378, 17.31464420255894, 17.62798890707979, 17.806168987229583, 17.901024432941565, 18.766298989600095, 19.381772936981818, 19.744742362827708, 20.34392509263294, 20.582707854079114, 20.785626151031376, 21.096695301879617, 21.283666904577807, 21.496358638871463, 21.623672463607836, 22.1501770946136, 22.488612507112215, 22.90312918868105, 22.985139180318292, 22.903081091085095, 22.733440287546713, 22.703174335668464, 22.683168375123607, 22.643057581320804, 22.637662308848167, 22.542148522931143, 22.508113197297988, 22.46917379932081, 22.437027286225604, 22.416900516726063, 22.447173908270113, 22.355075154006823, 22.32715648612563, 22.243953931446423, 22.20369452159329, 22.212801428493144, 22.13854912180358, 22.107247790752428, 22.084349143202846, 22.373472785874466, 22.361115727328066], "calls": [0.0, 0.6657789613848202, 1.3315579227696404, 2.4966711051930757, 3.162450066577896, 3.7949400798934754, 4.327563249001331, 5.126498002663116, 5.625832223701731, 6.458055925432757, 7.456724367509987, 8.322237017310254, 9.187749667110518, 9.587217043941411, 10.419440745672437, 10.952063914780293, 11.651131824234353, 12.383488681757656, 13.015978695073235, 14.014647137150465, 14.913448735019974, 16.511318242343542, 16.711051930758988, 17.310252996005325, 18.042609853528628, 19.274300932090547, 20.838881491344875, 22.736351531291614, 23.93475366178429, 24.60053262316911, 25.732356857523303, 26.764314247669773, 28.262316910785618, 29.527296937416775, 30.22636484687084, 31.790945406125164, 34.22103861517976, 35.95206391478029, 37.383488681757655, 39.280958721704394, 40.41278295605859, 41.54460719041278, 42.676431424766974, 43.70838881491345, 44.40745672436751, 45.70572569906791, 46.80426098535286, 48.06924101198402, 49.23435419440746, 50.16644474034621, 51.364846870838875, 52.99600532623169, 54.12782956058588, 54.826897470039945, 55.992010652463385, 57.35685752330226, 58.1225033288948, 59.35419440745673, 60.01997336884154, 60.5525965379494, 61.55126498002663, 62.61651131824234, 63.84820239680427, 64.74700399467378, 65.47936085219706, 67.24367509986683, 68.37549933422103, 69.4074567243675, 70.63914780292943, 71.5046604527297, 72.13715046604527, 73.30226364846871, 73.7017310252996, 74.36750998668442, 74.83355525965379, 76.03195739014647, 76.76431424766977, 77.56324900133156, 78.99467376830891, 80.22636484687084, 81.12516644474034, 81.89081225033289, 83.32223701731026, 83.75499334221038, 83.88814913448735, 84.72037283621837, 86.55126498002663, 86.98402130492676, 87.68308921438083, 88.38215712383489, 89.01464713715048, 89.98002663115847, 90.77896138482025, 91.87749667110519, 92.97603195739015, 93.64181091877496, 94.54061251664447, 95.50599201065246, 96.33821571238349, 98.06924101198402, 100.0], "rescued": [0, 49, 91, 160, 205, 243, 281, 335, 362, 411, 467, 515, 566, 588, 625, 655, 697, 741, 782, 840, 884, 969, 979, 1011, 1048, 1111, 1192, 1290, 1352, 1387, 1437, 1485, 1561, 1620, 1653, 1729, 1852, 1932, 1985, 2071, 2123, 2176, 2227, 2262, 2292, 2350, 2387, 2428, 2474, 2505, 2542, 2589, 2631, 2650, 2689, 2731, 2753, 2788, 2805, 2820, 2849, 2879, 2910, 2934, 2952, 3009, 3042, 3069, 3103, 3125, 3143, 3172, 3183, 3199, 3208, 3239, 3258, 3281, 3304, 3306, 3306, 3306, 3307, 3307, 3307, 3307, 3308, 3308, 3308, 3309, 3310, 3310, 3310, 3310, 3310, 3311, 3311, 3312, 3312, 3313, 3314], "harmed": [0, 3, 7, 23, 34, 41, 50, 59, 66, 77, 91, 102, 114, 118, 126, 136, 148, 158, 167, 180, 197, 219, 222, 232, 240, 261, 288, 315, 332, 343, 363, 387, 413, 441, 456, 485, 530, 561, 587, 624, 638, 660, 680, 701, 710, 737, 759, 779, 799, 817, 845, 874, 898, 911, 936, 960, 973, 1001, 1015, 1028, 1056, 1085, 1119, 1139, 1155, 1196, 1223, 1255, 1284, 1301, 1314, 1347, 1356, 1368, 1379, 1404, 1420, 1440, 1476, 1503, 1532, 1551, 1577, 1589, 1592, 1610, 1654, 1667, 1685, 1699, 1715, 1733, 1749, 1780, 1805, 1821, 1842, 1864, 1880, 1911, 1945], "exact_score": [7.8873978710439285, 9.447321325033979, 9.314239830067127, 9.675863102991107, 9.53854875719217, 10.336693755862658, 10.499246301166334, 10.382966911222438, 10.679932337338181, 10.636033416503885, 10.609722013319375, 10.827793274893532, 11.24360833416665, 11.211380276817989, 11.183628215699146, 11.15179846600383, 11.139467056450234, 11.413953418868603, 11.597009870590929, 11.578824056652373, 11.76379817368025, 11.741973570426287, 12.081071833238632, 12.184230812303694, 12.484946213651858, 12.51220955660931, 12.70062446784912, 13.038892873053307, 13.148269947559507, 13.266741163016826, 13.174957492980749, 13.240384685513373, 13.350771812740895, 13.68816895757973, 13.623796171439661, 13.75955438732402, 13.786108839087992, 13.743811226606226, 13.913430480341567, 14.04816491013721, 14.137994370007107, 14.51282921556644, 14.669788269056012, 14.812047023254566, 14.829763956248462, 15.036149162861493, 15.132828433843853, 14.97055528294243, 15.143738657066628, 15.43057745885025, 15.543589418362528, 15.484590176057031, 15.542353059058014, 15.658413077853355, 15.975430169424277, 16.002771301134448, 16.29433603501984, 16.55560159431486, 16.7082413614639, 16.956419044161084, 17.044788625446174, 17.193056653693763, 17.398897598303943, 17.50227501675626, 17.672669828523585, 17.8614808593239, 18.03601622813166, 18.388120793741603, 19.03227482692173, 19.621943571087396, 19.95215203793224, 20.471969825585703, 20.719785288549442, 20.952463528099255, 21.28500841384407, 21.621425610481403, 22.136562868942626, 22.595308233419203, 22.87446743113663, 22.985139180318292, 22.92080692108836, 22.738998103735238, 22.700973050117703, 22.716597086075023, 22.635549682384, 22.533700950083247, 22.49179060640449, 22.46917379932081, 22.476641353386757, 22.449104816097158, 22.355075154006823, 22.317289486419654, 22.241185820089257, 22.20369452159329, 22.199619711860638, 22.083515101228627, 22.094004807858415, 22.421264676552223, 22.392503969545235, 22.349075756071567, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.325949231941987, "threshold_area100": 16.320139301850023, "threshold_area25": 10.339339215587689}, "selected_budget": 77, "selected": {"val": {"score": 22.09509348306372, "calls": 76.8901569186876, "rescued": 783, "harmed": 346}, "test": {"score": 22.90312918868105, "calls": 77.56324900133156, "rescued": 3281, "harmed": 1440}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.3064268827438354, "operator": "ge"}, {"budget": 0.02, "threshold": 1.2675728797912598, "operator": "ge"}, {"budget": 0.03, "threshold": 1.229678750038147, "operator": "ge"}, {"budget": 0.04, "threshold": 1.1969668865203857, "operator": "ge"}, {"budget": 0.05, "threshold": 1.1758183240890503, "operator": "ge"}, {"budget": 0.06, "threshold": 1.1576347351074219, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1331112384796143, "operator": "ge"}, {"budget": 0.08, "threshold": 1.1058218479156494, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0697097778320312, "operator": "ge"}, {"budget": 0.1, "threshold": 1.052140712738037, "operator": "ge"}, {"budget": 0.11, "threshold": 1.0296776294708252, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9951806664466858, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9526827335357666, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9258108735084534, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9014692902565002, "operator": "ge"}, {"budget": 0.16, "threshold": 0.8898300528526306, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8606322407722473, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8427361845970154, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8255335092544556, "operator": "ge"}, {"budget": 0.2, "threshold": 0.8118213415145874, "operator": "ge"}, {"budget": 0.21, "threshold": 0.8066665530204773, "operator": "ge"}, {"budget": 0.22, "threshold": 0.7999432682991028, "operator": "ge"}, {"budget": 0.23, "threshold": 0.7901776432991028, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7836178541183472, "operator": "ge"}, {"budget": 0.25, "threshold": 0.7738182544708252, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7676891684532166, "operator": "ge"}, {"budget": 0.27, "threshold": 0.7627406120300293, "operator": "ge"}, {"budget": 0.28, "threshold": 0.7572602033615112, "operator": "ge"}, {"budget": 0.29, "threshold": 0.751436710357666, "operator": "ge"}, {"budget": 0.3, "threshold": 0.7403678894042969, "operator": "ge"}, {"budget": 0.31, "threshold": 0.7374698519706726, "operator": "ge"}, {"budget": 0.32, "threshold": 0.7307642102241516, "operator": "ge"}, {"budget": 0.33, "threshold": 0.7178520560264587, "operator": "ge"}, {"budget": 0.34, "threshold": 0.7096744775772095, "operator": "ge"}, {"budget": 0.35, "threshold": 0.7047967910766602, "operator": "ge"}, {"budget": 0.36, "threshold": 0.689633846282959, "operator": "ge"}, {"budget": 0.37, "threshold": 0.6812238693237305, "operator": "ge"}, {"budget": 0.38, "threshold": 0.6738739013671875, "operator": "ge"}, {"budget": 0.39, "threshold": 0.6653323173522949, "operator": "ge"}, {"budget": 0.4, "threshold": 0.6453975439071655, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6274236440658569, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6014487147331238, "operator": "ge"}, {"budget": 0.43, "threshold": 0.5775206089019775, "operator": "ge"}, {"budget": 0.44, "threshold": 0.561477780342102, "operator": "ge"}, {"budget": 0.45, "threshold": 0.5414766073226929, "operator": "ge"}, {"budget": 0.46, "threshold": 0.5276497602462769, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5009504556655884, "operator": "ge"}, {"budget": 0.48, "threshold": 0.4965083599090576, "operator": "ge"}, {"budget": 0.49, "threshold": 0.48163604736328125, "operator": "ge"}, {"budget": 0.5, "threshold": 0.4745507836341858, "operator": "ge"}, {"budget": 0.51, "threshold": 0.46525174379348755, "operator": "ge"}, {"budget": 0.52, "threshold": 0.44226565957069397, "operator": "ge"}, {"budget": 0.53, "threshold": 0.43238478899002075, "operator": "ge"}, {"budget": 0.54, "threshold": 0.4206588864326477, "operator": "ge"}, {"budget": 0.55, "threshold": 0.41489893198013306, "operator": "ge"}, {"budget": 0.56, "threshold": 0.40896427631378174, "operator": "ge"}, {"budget": 0.57, "threshold": 0.3950413465499878, "operator": "ge"}, {"budget": 0.58, "threshold": 0.38141995668411255, "operator": "ge"}, {"budget": 0.59, "threshold": 0.37019282579421997, "operator": "ge"}, {"budget": 0.6, "threshold": 0.3575761318206787, "operator": "ge"}, {"budget": 0.61, "threshold": 0.33976906538009644, "operator": "ge"}, {"budget": 0.62, "threshold": 0.3330279588699341, "operator": "ge"}, {"budget": 0.63, "threshold": 0.3197653591632843, "operator": "ge"}, {"budget": 0.64, "threshold": 0.31286126375198364, "operator": "ge"}, {"budget": 0.65, "threshold": 0.28854453563690186, "operator": "ge"}, {"budget": 0.66, "threshold": 0.2683340907096863, "operator": "ge"}, {"budget": 0.67, "threshold": 0.2499505877494812, "operator": "ge"}, {"budget": 0.68, "threshold": 0.23180240392684937, "operator": "ge"}, {"budget": 0.69, "threshold": 0.20602083206176758, "operator": "ge"}, {"budget": 0.7, "threshold": 0.18668034672737122, "operator": "ge"}, {"budget": 0.71, "threshold": 0.16646695137023926, "operator": "ge"}, {"budget": 0.72, "threshold": 0.14598989486694336, "operator": "ge"}, {"budget": 0.73, "threshold": 0.11002707481384277, "operator": "ge"}, {"budget": 0.74, "threshold": 0.0957939624786377, "operator": "ge"}, {"budget": 0.75, "threshold": 0.05555042624473572, "operator": "ge"}, {"budget": 0.76, "threshold": 0.01431351900100708, "operator": "ge"}, {"budget": 0.77, "threshold": -0.04306858777999878, "operator": "ge"}, {"budget": 0.78, "threshold": -2.1666371822357178, "operator": "ge"}, {"budget": 0.79, "threshold": -2.259000301361084, "operator": "ge"}, {"budget": 0.8, "threshold": -2.354750156402588, "operator": "ge"}, {"budget": 0.81, "threshold": -2.386850595474243, "operator": "ge"}, {"budget": 0.82, "threshold": -2.467715263366699, "operator": "ge"}, {"budget": 0.83, "threshold": -2.494630813598633, "operator": "ge"}, {"budget": 0.84, "threshold": -2.527651786804199, "operator": "ge"}, {"budget": 0.85, "threshold": -2.5601446628570557, "operator": "ge"}, {"budget": 0.86, "threshold": -2.604567289352417, "operator": "ge"}, {"budget": 0.87, "threshold": -2.6386725902557373, "operator": "ge"}, {"budget": 0.88, "threshold": -2.667802095413208, "operator": "ge"}, {"budget": 0.89, "threshold": -2.6805543899536133, "operator": "ge"}, {"budget": 0.9, "threshold": -2.7112960815429688, "operator": "ge"}, {"budget": 0.91, "threshold": -2.7439985275268555, "operator": "ge"}, {"budget": 0.92, "threshold": -2.8005940914154053, "operator": "ge"}, {"budget": 0.93, "threshold": -2.826570510864258, "operator": "ge"}, {"budget": 0.94, "threshold": -2.875244140625, "operator": "ge"}, {"budget": 0.95, "threshold": -2.9011118412017822, "operator": "ge"}, {"budget": 0.96, "threshold": -2.9340596199035645, "operator": "ge"}, {"budget": 0.97, "threshold": -2.9728798866271973, "operator": "ge"}, {"budget": 0.98, "threshold": -3.041565418243408, "operator": "ge"}, {"budget": 0.99, "threshold": -3.1189231872558594, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.751224860383048, 7.859016190694443, 7.988148910564388, 9.29335995551654, 9.233639151633435, 9.143010406075879, 9.168326313446478, 9.213839466954285, 9.327027168971053, 9.386931765964022, 9.437567151294104, 9.533320643208064, 9.565655773891908, 9.66156658198957, 10.543340621761475, 10.603280148872289, 10.564510789078007, 10.669133733649861, 10.749671819390086, 10.844195558316787, 11.60314642912266, 11.582666365413461, 12.276207849885838, 12.877780675877034, 12.777537907437978, 13.369078581914676, 13.3122265239037, 13.389620752213533, 13.481397769232494, 13.556720203359681, 13.619090365947423, 13.462951551529228, 13.473050365114759, 13.453671399898473, 13.830978583895462, 14.35397167942174, 14.45091613424396, 14.896243509613388, 15.052557608409217, 15.567441332192884, 15.73249289517252, 15.905402765430393, 15.983688633330795, 16.11645244074871, 16.318122844418397, 16.484605623455554, 16.673177662132804, 16.924207867604093, 17.105590237774145, 17.096964969910577, 17.23695868451185, 17.42063492063492, 17.51139160195621, 17.559516886328716, 17.76297378358011, 17.977609933435367, 18.103523460666317, 18.37090866123124, 18.60321982544205, 18.821449630769692, 18.895628928523667, 18.98840903221379, 19.293493563958165, 19.61594405901551, 19.781672238971627, 19.972211201771753, 19.961355824875877, 19.925490943865064, 19.991219036964242, 20.31052240506764, 20.432489451476794, 20.909255985982327, 21.20859949670925, 20.98112936152809, 21.33134295362623, 21.757149177129527, 22.09509348306372, 21.824479532135985, 21.689059911282133, 21.551356942923206, 21.320228452145955, 21.28192644763364, 21.429319959074338, 21.384778124398405, 21.344465445606854, 21.137090990095555, 21.103439483165282, 21.070565222349842, 21.027857081271808, 21.001697747302273, 20.82029606158873, 20.77777681073648, 20.745925647103107, 20.717285587975244, 20.799965196539866, 20.7723439919005, 20.736418565822838, 20.71423678900564, 20.69289935888572, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 20, 39, 57, 74, 88, 105, 123, 141, 155, 169, 186, 201, 217, 232, 248, 263, 271, 282, 292, 303, 316, 325, 339, 352, 364, 375, 386, 399, 412, 421, 431, 442, 451, 463, 474, 486, 495, 507, 518, 529, 537, 546, 555, 566, 576, 585, 594, 603, 610, 617, 624, 631, 639, 645, 653, 659, 665, 672, 678, 684, 691, 696, 705, 713, 720, 727, 732, 737, 743, 749, 755, 761, 767, 770, 775, 780, 783, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786, 786, 786], "harmed": [0, 6, 11, 18, 25, 30, 35, 40, 45, 47, 52, 58, 59, 62, 66, 68, 72, 75, 78, 79, 82, 83, 85, 90, 93, 98, 100, 106, 109, 112, 117, 121, 127, 131, 136, 139, 141, 144, 148, 151, 154, 161, 163, 167, 171, 175, 179, 185, 189, 194, 201, 204, 206, 212, 217, 222, 227, 233, 237, 238, 241, 247, 252, 256, 263, 268, 273, 283, 288, 294, 300, 308, 313, 320, 328, 335, 343, 346, 355, 363, 371, 376, 381, 386, 392, 397, 405, 410, 415, 421, 425, 432, 439, 444, 449, 452, 456, 462, 465, 470, 475], "exact_score": [7.641657905799261, 7.751224860383048, 7.859016190694443, 7.988148910564388, 9.29335995551654, 9.233639151633435, 9.143010406075879, 9.168326313446478, 9.213839466954285, 9.327027168971053, 9.386931765964022, 9.437567151294104, 9.533320643208064, 9.565655773891908, 9.66156658198957, 10.543340621761475, 10.603280148872289, 10.564510789078007, 10.669133733649861, 10.749671819390086, 10.844195558316787, 11.60314642912266, 11.582666365413461, 12.276207849885838, 12.877780675877034, 12.777537907437978, 13.369078581914676, 13.3122265239037, 13.389620752213533, 13.481397769232494, 13.556720203359681, 13.619090365947423, 13.462951551529228, 13.473050365114759, 13.453671399898473, 13.830978583895462, 14.35397167942174, 14.45091613424396, 14.896243509613388, 15.052557608409217, 15.567441332192884, 15.73249289517252, 15.905402765430393, 15.983688633330795, 16.11645244074871, 16.318122844418397, 16.484605623455554, 16.673177662132804, 16.924207867604093, 17.105590237774145, 17.096964969910577, 17.23695868451185, 17.42063492063492, 17.51139160195621, 17.559516886328716, 17.76297378358011, 17.977609933435367, 18.103523460666317, 18.37090866123124, 18.60321982544205, 18.821449630769692, 18.895628928523667, 18.98840903221379, 19.293493563958165, 19.61594405901551, 19.781672238971627, 19.972211201771753, 19.961355824875877, 19.925490943865064, 19.991219036964242, 20.31052240506764, 20.432489451476794, 20.909255985982327, 21.20859949670925, 20.98112936152809, 21.33134295362623, 21.757149177129527, 22.09509348306372, 21.824479532135985, 21.689059911282133, 21.551356942923206, 21.320228452145955, 21.28192644763364, 21.429319959074338, 21.384778124398405, 21.344465445606854, 21.137090990095555, 21.103439483165282, 21.070565222349842, 21.027857081271808, 21.001697747302273, 20.82029606158873, 20.77777681073648, 20.745925647103107, 20.717285587975244, 20.799965196539866, 20.7723439919005, 20.736418565822838, 20.71423678900564, 20.69289935888572, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.288085128048202, "threshold_area100": 16.28181173807219, "threshold_area25": 10.04354100085887}, "test": {"score": [7.8873978710439285, 7.954531991538148, 8.632771671228632, 8.60950050552747, 8.618736773327447, 9.469231813864237, 10.927103443808182, 11.233132363040086, 11.199181925577184, 11.377100217240429, 11.582193251062838, 11.849364261366793, 11.742691802364817, 11.760268712778716, 11.946938975575081, 11.944360768099651, 11.996315692958287, 11.997703455696287, 11.98189336146652, 11.995147598653528, 12.046217573568295, 11.99268969588416, 12.020039069316347, 12.016651032368083, 12.083468831388673, 12.1748649481518, 12.386086555624635, 12.425114338877405, 12.804223900940235, 12.965837415037182, 13.204038480331926, 13.215227304224614, 13.408227074337379, 13.703737500202006, 13.74321019798951, 13.796153513345258, 14.171935754910947, 14.199645731038284, 14.391013232476645, 14.425936748328269, 14.638961928477903, 14.834843918110105, 15.21766770471201, 15.524082691056485, 15.561736460610371, 15.713022838194904, 15.905762098022594, 16.31504526527526, 16.38721001359649, 16.67146885250284, 16.71707404477401, 16.855869031769043, 17.03354946888734, 17.091757202735593, 17.260048861572884, 17.321477051687342, 17.41179549849774, 17.647649224095403, 17.811915723946495, 18.00312034396994, 18.2548444909601, 18.58331088260907, 18.73312882560466, 18.69009281208662, 18.826022975483685, 19.747921273421987, 19.94455974123257, 20.080600477279486, 20.38521379040949, 21.00111684221265, 21.104439007835143, 21.078207605128714, 21.289935278728592, 21.593837391786597, 21.63518978830768, 22.104040283589978, 22.488612507112215, 22.90312918868105, 22.979432010306404, 22.819180666734695, 22.735100279299175, 22.71573740660864, 22.61724677963903, 22.5984392411406, 22.551145993619603, 22.526382031996686, 22.422804091238696, 22.38269029624541, 22.351478775499622, 22.33705820349018, 22.357574769464954, 22.371738792969957, 22.294225445453247, 22.279791479793932, 22.217122602057863, 22.127013249215448, 22.108149788517657, 22.079241454679007, 22.422667372664883, 22.38385690228309, 22.361115727328066], "calls": [0.0, 0.5659121171770972, 0.9653794940079893, 1.6644474034620507, 2.2303595206391478, 3.029294274300932, 3.695073235685752, 4.394141145139813, 5.559254327563249, 6.657789613848203, 7.023968042609853, 7.55659121171771, 8.189081225033288, 9.354194407456724, 10.119840213049267, 10.752330226364846, 11.318242343541945, 12.916111850865514, 14.08122503328895, 15.11318242343542, 16.577896138482025, 17.0439414114514, 17.842876165113182, 19.00798934753662, 19.873501997336884, 21.205059920106525, 22.33688415446072, 23.33555259653795, 24.201065246338217, 25.233022636484687, 26.897470039946736, 27.363515312916114, 28.462050599201067, 30.09320905459387, 31.158455392809586, 31.924101198402134, 33.85486018641811, 34.88681757656458, 35.75233022636485, 36.451398135818906, 38.58189081225033, 39.913448735019976, 42.01065246338216, 43.97470039946737, 44.90679094540613, 46.304926764314246, 47.33688415446072, 49.40079893475366, 49.70039946737683, 50.699067909454065, 51.19840213049268, 51.86418109187749, 53.82822902796271, 54.627163781624496, 55.59254327563249, 56.19174434087883, 56.724367509986685, 58.022636484687084, 59.28761651131824, 60.81890812250332, 61.51797603195739, 63.182423435419445, 63.74833555259654, 64.94673768308921, 65.64580559254327, 67.37683089214381, 68.6750998668442, 69.77363515312916, 70.60585885486019, 71.5046604527297, 72.50332889480693, 73.26897470039947, 73.76830892143809, 74.56724367509987, 74.96671105193076, 75.99866844207723, 76.76431424766977, 77.56324900133156, 79.16111850865512, 79.9600532623169, 81.35818908122504, 81.8242343541944, 83.42210386151797, 83.85486018641811, 84.55392809587217, 85.15312916111851, 86.1185086551265, 87.05059920106525, 87.71637816245007, 88.04926764314249, 88.8814913448735, 89.74700399467378, 91.11185086551265, 91.51131824234355, 92.84287616511318, 93.50865512649801, 94.20772303595206, 95.07323568575234, 96.80426098535287, 98.20239680426099, 100.0], "rescued": [0, 48, 81, 137, 178, 242, 298, 351, 438, 515, 540, 573, 614, 679, 730, 766, 797, 883, 938, 992, 1071, 1096, 1130, 1175, 1216, 1276, 1329, 1377, 1418, 1469, 1552, 1574, 1625, 1697, 1743, 1780, 1875, 1922, 1960, 1985, 2081, 2135, 2222, 2308, 2343, 2391, 2426, 2501, 2512, 2548, 2564, 2586, 2647, 2670, 2701, 2718, 2735, 2772, 2809, 2850, 2867, 2911, 2927, 2960, 2980, 3029, 3058, 3083, 3105, 3128, 3153, 3172, 3185, 3205, 3212, 3238, 3258, 3281, 3304, 3304, 3306, 3306, 3306, 3307, 3307, 3307, 3307, 3308, 3308, 3308, 3309, 3310, 3310, 3310, 3310, 3310, 3311, 3311, 3313, 3313, 3314], "harmed": [0, 10, 15, 31, 42, 54, 66, 76, 99, 117, 124, 127, 139, 166, 178, 190, 200, 220, 239, 254, 271, 283, 290, 305, 322, 343, 356, 370, 384, 399, 429, 439, 455, 485, 502, 518, 548, 575, 588, 596, 634, 657, 696, 736, 756, 781, 797, 834, 840, 856, 867, 878, 913, 927, 948, 966, 976, 1003, 1030, 1062, 1076, 1108, 1121, 1149, 1165, 1207, 1239, 1269, 1289, 1303, 1328, 1350, 1356, 1378, 1383, 1404, 1420, 1440, 1479, 1497, 1531, 1544, 1579, 1589, 1605, 1619, 1643, 1665, 1682, 1690, 1707, 1732, 1759, 1768, 1798, 1816, 1833, 1853, 1889, 1915, 1945], "exact_score": [7.8873978710439285, 8.629032323079315, 8.602962227629634, 9.46546139035628, 11.27476530871143, 11.404153100184105, 11.151096291569843, 11.615846910823672, 11.739907245673162, 11.605671141874128, 11.95881463226338, 11.933875509788319, 12.015679565128117, 12.005670745322657, 11.980140767006557, 12.007135428499813, 12.018086470031394, 11.992632104644256, 12.007483131437493, 12.012038187941338, 12.07693960220422, 12.173421693568748, 12.231793929006201, 12.407695599860528, 12.695466448506398, 12.942814468576627, 13.052528479918243, 13.22074071260579, 13.259611810226795, 13.588191777844353, 13.723030760506166, 13.724544536965924, 13.790958496143201, 14.005008698933446, 14.197616364905985, 14.29112966249933, 14.386840978751362, 14.590505682629255, 14.59012504652831, 14.677935215018357, 14.836725628380401, 15.008904399616629, 15.215086154008567, 15.384380696621887, 15.524082691056485, 15.572823913045363, 15.680662372774998, 15.873979624471524, 16.01172071159401, 16.21333474357837, 16.445654543292562, 16.680117309800508, 16.887951895653234, 16.878328854197456, 17.060976326475103, 17.17757526585618, 17.27955066376807, 17.533815385745804, 17.647649224095403, 17.73834710888171, 17.82251585225118, 18.212820768069886, 18.39251619591786, 18.55740211390912, 18.73743527985163, 18.699384544402005, 18.89856475847784, 19.644547247590396, 19.82199116977951, 20.015891504828776, 20.132811772439954, 20.515570467396024, 21.005592590947156, 21.281555552741608, 21.31947388732595, 21.63518978830768, 22.104040283589978, 22.595308233419203, 22.87446743113663, 22.987399546706293, 22.816961869425242, 22.749641442399625, 22.691490676785673, 22.65742958883136, 22.576135948305385, 22.533202117884297, 22.42790290547962, 22.384708999908685, 22.340099213684802, 22.354157254415004, 22.365050235250145, 22.29607722618046, 22.26410151899231, 22.210375266597676, 22.065534396173632, 22.082245743523625, 22.449883342138428, 22.41729881291691, 22.392999734717737, 22.349075756071567, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.806939046811763, "threshold_area100": 16.75163083300586, "threshold_area25": 11.014626963238577}, "selected_budget": 77, "selected": {"val": {"score": 22.09509348306372, "calls": 76.8901569186876, "rescued": 783, "harmed": 346}, "test": {"score": 22.90312918868105, "calls": 77.56324900133156, "rescued": 3281, "harmed": 1440}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 21.91270620579996, "calls": 78.88730385164051}, "test": {"score": 22.5906692554865, "calls": 79.39414114513981}}, "old_normalized_interior": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405}, "test": {"score": 22.770354810705033, "calls": 78.52862849533955}}, "old_normalized_endpoints": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405}, "test": {"score": 22.770354810705033, "calls": 78.52862849533955}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 7.751224860383048, 7.64545723771259, 7.767960768223374, 7.888724772052258, 7.989258714466528, 8.102055883641974, 8.200662027159066, 8.325324607074949, 9.323842073689802, 10.320122448825382, 10.275068540788622, 10.306993632031023, 10.252931374772345, 10.356223593333057, 11.131537862318694, 11.820357922799854, 11.82177032656924, 11.896084790684698, 11.946202169679623, 11.962311381361484, 12.074499323462527, 12.73877019687005, 12.846865983380845, 12.836461774103954, 12.906556006220743, 12.987718276471611, 13.009281554331794, 12.92953512218318, 12.889996547915874, 12.95838682340438, 13.066925140698787, 13.197377530894014, 13.313304583617589, 13.283004653682504, 13.79575652118131, 14.609076924088885, 14.755575295423052, 15.265838477398635, 15.171535453793519, 15.320600302125175, 15.70072324452989, 15.890278945566246, 16.162991463428494, 16.347860636380933, 16.36675824175824, 16.578602490772703, 16.622480377580086, 16.655659155659155, 16.690510522375803, 16.579134862418996, 16.645021645021647, 16.748923795013184, 16.944222444222447, 17.17370119169525, 17.32675997780559, 17.12054485917865, 16.90594547359006, 16.85556039111295, 16.98491661798116, 17.1264659800798, 17.273576097105508, 17.472372449838232, 17.391325083671592, 17.6793895759413, 17.90034441961066, 18.12506522580754, 18.06409038116355, 18.381333350416142, 18.611557268395078, 18.84036680911681, 19.252491632347592, 19.405937200054847, 19.8110866362797, 20.242146828791142, 20.568666330460296, 21.198634550485068, 21.48844313784536, 21.936136832098168, 21.91270620579996, 21.8766100048668, 21.636901218310197, 21.810705944518677, 21.756684759274926, 21.714046781415853, 21.698779498166, 21.60368757259001, 21.75683189203868, 21.721491569188654, 21.672121916024356, 21.63796181207959, 21.6045342749073, 21.48066250515171, 21.43532052203639, 21.391448174467044, 21.209089789097167, 21.162883621477075, 21.12178582002093, 20.89773431194038, 20.78078099917753, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 7.751224860383048, 7.64545723771259, 7.767960768223374, 7.888724772052258, 7.989258714466528, 8.102055883641974, 8.200662027159066, 8.325324607074949, 9.323842073689802, 10.320122448825382, 10.275068540788622, 10.306993632031023, 10.252931374772345, 10.356223593333057, 11.131537862318694, 11.820357922799854, 11.82177032656924, 11.896084790684698, 11.946202169679623, 11.962311381361484, 12.074499323462527, 12.73877019687005, 12.846865983380845, 12.836461774103954, 12.906556006220743, 12.987718276471611, 13.009281554331794, 12.92953512218318, 12.889996547915874, 12.95838682340438, 13.066925140698787, 13.197377530894014, 13.313304583617589, 13.283004653682504, 13.79575652118131, 14.609076924088885, 14.755575295423052, 15.265838477398635, 15.171535453793519, 15.320600302125175, 15.70072324452989, 15.890278945566246, 16.162991463428494, 16.347860636380933, 16.36675824175824, 16.578602490772703, 16.622480377580086, 16.655659155659155, 16.690510522375803, 16.579134862418996, 16.645021645021647, 16.748923795013184, 16.944222444222447, 17.17370119169525, 17.32675997780559, 17.12054485917865, 16.90594547359006, 16.85556039111295, 16.98491661798116, 17.1264659800798, 17.273576097105508, 17.472372449838232, 17.391325083671592, 17.6793895759413, 17.90034441961066, 18.12506522580754, 18.06409038116355, 18.381333350416142, 18.611557268395078, 18.84036680911681, 19.252491632347592, 19.405937200054847, 19.8110866362797, 20.242146828791142, 20.568666330460296, 21.198634550485068, 21.48844313784536, 21.936136832098168, 21.91270620579996, 21.8766100048668, 21.636901218310197, 21.810705944518677, 21.756684759274926, 21.714046781415853, 21.698779498166, 21.60368757259001, 21.75683189203868, 21.721491569188654, 21.672121916024356, 21.63796181207959, 21.6045342749073, 21.48066250515171, 21.43532052203639, 21.391448174467044, 21.209089789097167, 21.162883621477075, 21.12178582002093, 20.89773431194038, 20.78078099917753, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 10.335467472596418, "exact_area100": 16.088099997436746}, "test": {"score": [7.8873978710439285, 7.921309110880968, 8.600746414726746, 10.553283403832003, 10.434348745179664, 10.689527090611897, 10.879299261020087, 10.706788070719117, 10.494473394126318, 10.914288994828732, 11.133451594666756, 11.096919077304515, 11.241611423257496, 11.17757808256136, 11.059969860462472, 11.296295370348815, 11.624536090551373, 11.77029593344544, 12.218845106419831, 12.368443444052044, 12.438240963505526, 12.446805211808016, 12.583260870125432, 12.627196684343309, 12.812259364638443, 12.828333346265145, 12.873627957836241, 13.03553508906787, 13.37004230182634, 13.419210571537388, 13.5092073982699, 13.578271481330878, 13.57527488628771, 13.700919360300778, 13.806552032027414, 13.94335205238441, 14.059038894322788, 14.263601555755068, 14.290551808168303, 14.219659827510627, 14.305109767008206, 14.434493964173328, 14.534658785029544, 14.651581434070613, 14.60646909907575, 15.000076684850704, 15.20001607887796, 15.275236373867854, 15.415089924708012, 15.456625126975974, 15.59680914972897, 15.968362527733074, 16.023540124664446, 16.024178859417756, 16.17519746973429, 16.21322862878492, 16.111105731849406, 16.123293087385473, 16.169657606360396, 16.36602004958377, 16.48662500893124, 16.760151834698583, 17.335138293654353, 17.49549184423393, 17.789617565464603, 18.0869436724996, 18.486201926275655, 18.660550268722016, 18.83285354430166, 19.217355367983533, 19.41907076811133, 19.874938872493235, 20.135084828313634, 20.349655403686487, 20.919682459858237, 21.62074945743378, 21.809827584285628, 22.67762637000411, 22.770354810705033, 22.92941118194236, 22.87655158208186, 22.83817097195747, 22.678139861059197, 22.69729120036025, 23.041981431649738, 22.765996978477755, 22.74920367491431, 22.72778034727234, 22.683106413531657, 22.627172843873907, 22.5995751749543, 22.59450223712854, 22.635896885666646, 22.65728260932075, 22.558545763802467, 22.529918699970043, 22.51860679446344, 22.435784632201372, 22.391137329469654, 22.408381361441645, 22.361115727328066], "calls": [0.0, 0.2996005326231691, 1.1651131824234355, 1.8641810918774968, 2.3635153129161117, 2.896138482023968, 3.695073235685752, 4.360852197070573, 5.89214380825566, 6.990679094540613, 7.523302263648468, 8.222370173102531, 8.85486018641811, 9.454061251664447, 10.652463382157123, 11.684420772303596, 12.716378162450068, 13.681757656458057, 14.680426098535287, 15.446071904127828, 17.276964047936087, 19.14114513981358, 20.03994673768309, 22.07057256990679, 23.06924101198402, 23.601864181091877, 24.533954727030626, 25.96537949400799, 27.463382157123835, 29.027962716378163, 30.25965379494008, 31.05858854860186, 31.757656458055926, 32.52330226364847, 34.05459387483356, 34.62050599201065, 35.38615179760319, 36.651131824234355, 37.08388814913449, 37.81624500665779, 38.715046604527295, 39.71371504660453, 40.279627163781626, 41.51131824234354, 42.743009320905465, 44.6404793608522, 45.50599201065246, 46.97070572569907, 48.035952063914785, 48.568575233022635, 49.40079893475366, 49.93342210386152, 50.99866844207723, 51.73102529960053, 53.46205059920106, 54.260985352862846, 55.45938748335553, 56.757656458055926, 57.39014647137151, 58.788282290279625, 59.553928095872166, 61.51797603195739, 62.849533954727036, 63.84820239680427, 64.54727030625833, 65.47936085219706, 66.94407456724367, 68.07589880159787, 68.6750998668442, 70.0399467376831, 70.80559254327564, 72.17043941411451, 73.16910785619174, 73.73501997336885, 74.90013315579228, 76.19840213049268, 76.49800266311586, 78.09587217043942, 78.52862849533955, 79.9600532623169, 80.6924101198402, 81.3249001331558, 82.85619174434088, 83.65512649800266, 84.12117177097204, 86.6844207723036, 87.15046604527296, 87.71637816245007, 88.8814913448735, 89.68042609853528, 90.31291611185085, 91.07856191744341, 91.34487350199734, 92.21038615179761, 93.24234354194407, 93.97470039946738, 95.67243675099867, 97.20372836218375, 98.00266311584554, 98.96804260985353, 100.0], "exact_score": [7.8873978710439285, 8.618330710811698, 10.522158270445942, 10.659925191397617, 10.819025010295135, 10.57969526067769, 10.487255901163897, 10.914288994828732, 11.122876515790276, 11.228360394540601, 11.106707244348252, 11.073615391747513, 11.478934316873417, 11.7848804933332, 11.921335552494064, 12.369451192374216, 12.525341232289772, 12.437235622345128, 12.435181650004164, 12.440367197507959, 12.58817878956891, 12.569370980544663, 12.618447532685378, 12.809833961583914, 12.81110445459262, 12.871599283001656, 13.035427143354891, 13.171309053898039, 13.509373148112402, 13.412659930698451, 13.474446922644704, 13.572393979186515, 13.60889529868323, 13.746334580481273, 13.79628994963819, 13.996250708357191, 14.194460416203258, 14.272728798905426, 14.246671232376954, 14.29707686905617, 14.481261407693893, 14.583428981359187, 14.55933444299139, 14.617774283930856, 14.928167795962807, 15.017805298963765, 15.254151284056686, 15.275236373867854, 15.410168133079729, 15.505098725314587, 16.027024128570343, 16.023540124664446, 16.049682463278792, 16.156710266671475, 16.239452459047403, 16.136941059400257, 16.096099282548483, 16.16341442005852, 16.184932494170425, 16.379051372116436, 16.53638207391665, 16.715066053743556, 17.11929009005682, 17.317632042001733, 17.460452535487573, 17.944331321572303, 18.224026873872617, 18.49371919840152, 18.63183401935251, 18.897120715414196, 19.2080588405611, 19.46272739995443, 19.815921573087387, 20.102077219202847, 20.42898498360715, 20.999417398462043, 21.510468198623855, 22.00775618154351, 22.648181552300038, 22.837278323351462, 22.927142361641657, 22.86385558220873, 22.813629888949666, 22.718560038131102, 23.04785807366119, 22.886111813720497, 22.835722444305556, 22.754089148184427, 22.71752195978246, 22.720475846768203, 22.61457190730851, 22.542268575303872, 22.614557707128917, 22.589822782937492, 22.529918699970043, 22.566786000230053, 22.545129565971834, 22.442417403252293, 22.392913663623485, 22.408381361441645, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 11.183076125557506, "exact_area100": 16.49424791238865}}, "old_normalized_endpoint_budget": 78}, {"seed": 2028, "selection": {"best_epoch": 6, "epochs": 14, "alpha": 0.25, "validation_area100": 0.16286456221776519, "training_seconds": 4.484756913967431, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5176533942126778, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5036372543489394, "operator": "ge"}, {"budget": 0.03, "threshold": 0.49888806876332653, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4918254936745694, "operator": "ge"}, {"budget": 0.05, "threshold": 0.48421705754201533, "operator": "ge"}, {"budget": 0.06, "threshold": 0.47063865027568835, "operator": "ge"}, {"budget": 0.07, "threshold": 0.46356602379898354, "operator": "ge"}, {"budget": 0.08, "threshold": 0.45080833468586223, "operator": "ge"}, {"budget": 0.09, "threshold": 0.43411744556985515, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4027964563142054, "operator": "ge"}, {"budget": 0.11, "threshold": 0.3887136143483392, "operator": "ge"}, {"budget": 0.12, "threshold": 0.36380280967749123, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3559723085192186, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3341991460668418, "operator": "ge"}, {"budget": 0.15, "threshold": 0.32638236105266893, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3123878340715194, "operator": "ge"}, {"budget": 0.17, "threshold": 0.29661438404269364, "operator": "ge"}, {"budget": 0.18, "threshold": 0.28926747447599305, "operator": "ge"}, {"budget": 0.19, "threshold": 0.27682379959732817, "operator": "ge"}, {"budget": 0.2, "threshold": 0.27329705809454863, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2700394117067391, "operator": "ge"}, {"budget": 0.22, "threshold": 0.26397882425886854, "operator": "ge"}, {"budget": 0.23, "threshold": 0.25919575990144306, "operator": "ge"}, {"budget": 0.24, "threshold": 0.2543838438111116, "operator": "ge"}, {"budget": 0.25, "threshold": 0.24930219622932476, "operator": "ge"}, {"budget": 0.26, "threshold": 0.2448388884021059, "operator": "ge"}, {"budget": 0.27, "threshold": 0.24293733252157565, "operator": "ge"}, {"budget": 0.28, "threshold": 0.2402063666784441, "operator": "ge"}, {"budget": 0.29, "threshold": 0.23787800250199176, "operator": "ge"}, {"budget": 0.3, "threshold": 0.23541759388621186, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2322985985851939, "operator": "ge"}, {"budget": 0.32, "threshold": 0.2260267405141371, "operator": "ge"}, {"budget": 0.33, "threshold": 0.22320924585907959, "operator": "ge"}, {"budget": 0.34, "threshold": 0.220065664183352, "operator": "ge"}, {"budget": 0.35, "threshold": 0.2160950394522737, "operator": "ge"}, {"budget": 0.36, "threshold": 0.21118643453939603, "operator": "ge"}, {"budget": 0.37, "threshold": 0.20721070529402105, "operator": "ge"}, {"budget": 0.38, "threshold": 0.20309094323329538, "operator": "ge"}, {"budget": 0.39, "threshold": 0.19959533347154146, "operator": "ge"}, {"budget": 0.4, "threshold": 0.19447106207392953, "operator": "ge"}, {"budget": 0.41, "threshold": 0.1877568308785728, "operator": "ge"}, {"budget": 0.42, "threshold": 0.17941165507023446, "operator": "ge"}, {"budget": 0.43, "threshold": 0.16934996577562475, "operator": "ge"}, {"budget": 0.44, "threshold": 0.16157503817272353, "operator": "ge"}, {"budget": 0.45, "threshold": 0.1495891934017579, "operator": "ge"}, {"budget": 0.46, "threshold": 0.13352044638747324, "operator": "ge"}, {"budget": 0.47, "threshold": 0.12239362550124719, "operator": "ge"}, {"budget": 0.48, "threshold": 0.11498515066008198, "operator": "ge"}, {"budget": 0.49, "threshold": 0.10563202730755313, "operator": "ge"}, {"budget": 0.5, "threshold": 0.10031466722669025, "operator": "ge"}, {"budget": 0.51, "threshold": 0.09316745463881262, "operator": "ge"}, {"budget": 0.52, "threshold": 0.08664733339335406, "operator": "ge"}, {"budget": 0.53, "threshold": 0.0824290653244978, "operator": "ge"}, {"budget": 0.54, "threshold": 0.07936112010902933, "operator": "ge"}, {"budget": 0.55, "threshold": 0.07395367286307263, "operator": "ge"}, {"budget": 0.56, "threshold": 0.0681567795050162, "operator": "ge"}, {"budget": 0.57, "threshold": 0.06363508057151801, "operator": "ge"}, {"budget": 0.58, "threshold": 0.06020667039901413, "operator": "ge"}, {"budget": 0.59, "threshold": 0.05771055240012194, "operator": "ge"}, {"budget": 0.6, "threshold": 0.05450875388427428, "operator": "ge"}, {"budget": 0.61, "threshold": 0.04993289904518409, "operator": "ge"}, {"budget": 0.62, "threshold": 0.04651263762127668, "operator": "ge"}, {"budget": 0.63, "threshold": 0.04319118881364312, "operator": "ge"}, {"budget": 0.64, "threshold": 0.03916298960845627, "operator": "ge"}, {"budget": 0.65, "threshold": 0.03625506303204126, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03271199507359373, "operator": "ge"}, {"budget": 0.67, "threshold": 0.030123614436404177, "operator": "ge"}, {"budget": 0.68, "threshold": 0.02692333045086973, "operator": "ge"}, {"budget": 0.69, "threshold": 0.02034684326597247, "operator": "ge"}, {"budget": 0.7, "threshold": 0.011770959962170998, "operator": "ge"}, {"budget": 0.71, "threshold": 0.0075599723944417855, "operator": "ge"}, {"budget": 0.72, "threshold": 0.0008977383657071253, "operator": "ge"}, {"budget": 0.73, "threshold": -0.004428942995332129, "operator": "ge"}, {"budget": 0.74, "threshold": -0.008006557148760508, "operator": "ge"}, {"budget": 0.75, "threshold": -0.018755675567439966, "operator": "ge"}, {"budget": 0.76, "threshold": -0.038806108906636515, "operator": "ge"}, {"budget": 0.77, "threshold": -0.07969550463462632, "operator": "ge"}, {"budget": 0.78, "threshold": -0.2036621480054694, "operator": "ge"}, {"budget": 0.79, "threshold": -0.2168814669666289, "operator": "ge"}, {"budget": 0.8, "threshold": -0.23052138920916468, "operator": "ge"}, {"budget": 0.81, "threshold": -0.23617057884136763, "operator": "ge"}, {"budget": 0.82, "threshold": -0.24301180460690128, "operator": "ge"}, {"budget": 0.83, "threshold": -0.24642147274102288, "operator": "ge"}, {"budget": 0.84, "threshold": -0.25715020176179376, "operator": "ge"}, {"budget": 0.85, "threshold": -0.26777645774109965, "operator": "ge"}, {"budget": 0.86, "threshold": -0.2785790436505985, "operator": "ge"}, {"budget": 0.87, "threshold": -0.2848926548136218, "operator": "ge"}, {"budget": 0.88, "threshold": -0.2926465915396901, "operator": "ge"}, {"budget": 0.89, "threshold": -0.2992032605024812, "operator": "ge"}, {"budget": 0.9, "threshold": -0.3080615764714195, "operator": "ge"}, {"budget": 0.91, "threshold": -0.3121223655611783, "operator": "ge"}, {"budget": 0.92, "threshold": -0.3177171897290691, "operator": "ge"}, {"budget": 0.93, "threshold": -0.32053146400389837, "operator": "ge"}, {"budget": 0.94, "threshold": -0.3268808830505559, "operator": "ge"}, {"budget": 0.95, "threshold": -0.3309722524509837, "operator": "ge"}, {"budget": 0.96, "threshold": -0.3377083823760348, "operator": "ge"}, {"budget": 0.97, "threshold": -0.34069254980135094, "operator": "ge"}, {"budget": 0.98, "threshold": -0.34872391035849204, "operator": "ge"}, {"budget": 0.99, "threshold": -0.36080865388423056, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.737353307510221, 7.850490860690032, 7.9794837885934236, 9.26246112877068, 9.305048188150739, 9.245608677950328, 9.26709708272292, 9.311849723564494, 9.290766119279034, 9.385588562414043, 9.474783576482237, 9.569314481921392, 9.544397120620147, 9.663079355692785, 9.768340929283696, 9.87248615225274, 9.985144258684409, 10.12254150052544, 10.250740990828007, 11.027716527803957, 11.142348168863435, 11.213144285433444, 11.194195868303066, 11.854432204280538, 12.390584560167724, 12.731646447611794, 13.322841621015424, 13.369324134928085, 13.484463536277008, 13.486816342039509, 13.40156633906634, 13.400833962221181, 13.526158773780002, 13.607763778027401, 13.717497771525005, 13.835108164239365, 14.23367917404487, 14.399510794857282, 14.521103279550465, 15.02004744082209, 15.559842963994575, 15.64653464613649, 15.80226997842866, 16.31407745784247, 16.475800732141536, 16.667716126544647, 16.690519896738742, 16.881012506012503, 17.05432819955205, 17.233098175984725, 17.422706922706922, 17.668964035318268, 17.897601709675413, 18.03925187646118, 18.187231624302193, 18.52731375332096, 18.4467466235422, 18.61812551848184, 18.907165362083393, 18.737522236196565, 19.031316894058854, 19.080799617712373, 19.276008384614098, 19.504334014643295, 19.697597524030645, 19.994106429030115, 20.00455665170752, 20.009756001661223, 20.32643109198834, 20.62303985078694, 20.620207018147205, 20.636904574433075, 20.650174096027186, 21.157470817217344, 21.562524710365302, 21.762801505973826, 21.88429108891366, 21.936136832098168, 21.90983701177802, 21.672161403669815, 21.64005455206768, 21.606201887555606, 21.79667076417321, 21.760683911289927, 21.729451995252163, 21.683648265924266, 21.45972076700165, 21.434431578037145, 21.395504851748225, 21.34479896053917, 21.221834942142905, 21.190666427999115, 21.16396511061139, 20.97229478209827, 20.855008200248754, 20.755909967713457, 20.642771521878917, 20.597252294865026, 20.7253775942232, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 18, 38, 59, 76, 95, 114, 131, 148, 162, 178, 193, 208, 220, 234, 249, 261, 273, 286, 298, 311, 324, 334, 344, 356, 368, 381, 393, 404, 416, 425, 435, 446, 458, 468, 479, 489, 500, 512, 521, 533, 546, 558, 568, 578, 585, 593, 601, 608, 615, 621, 628, 635, 641, 649, 656, 664, 670, 676, 684, 688, 695, 700, 706, 713, 720, 726, 731, 737, 744, 750, 754, 758, 763, 769, 774, 779, 783, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786], "harmed": [0, 6, 12, 18, 24, 26, 32, 38, 41, 46, 50, 54, 56, 60, 62, 67, 70, 73, 77, 80, 85, 89, 93, 97, 101, 103, 110, 112, 115, 118, 123, 128, 134, 139, 143, 147, 150, 152, 155, 157, 161, 164, 169, 172, 174, 175, 179, 184, 189, 194, 194, 200, 202, 206, 211, 217, 222, 229, 235, 238, 247, 251, 256, 262, 270, 275, 279, 285, 291, 296, 299, 306, 312, 319, 324, 328, 337, 347, 350, 353, 359, 364, 373, 374, 378, 382, 388, 394, 398, 404, 411, 418, 423, 428, 435, 443, 448, 455, 462, 467, 475], "exact_score": [7.641657905799261, 7.737353307510221, 7.850490860690032, 7.9794837885934236, 9.26246112877068, 9.305048188150739, 9.245608677950328, 9.26709708272292, 9.311849723564494, 9.290766119279034, 9.385588562414043, 9.474783576482237, 9.569314481921392, 9.544397120620147, 9.663079355692785, 9.768340929283696, 9.87248615225274, 9.985144258684409, 10.12254150052544, 10.250740990828007, 11.027716527803957, 11.142348168863435, 11.213144285433444, 11.194195868303066, 11.854432204280538, 12.390584560167724, 12.731646447611794, 13.322841621015424, 13.369324134928085, 13.484463536277008, 13.486816342039509, 13.40156633906634, 13.400833962221181, 13.526158773780002, 13.607763778027401, 13.717497771525005, 13.835108164239365, 14.23367917404487, 14.399510794857282, 14.521103279550465, 15.02004744082209, 15.559842963994575, 15.64653464613649, 15.80226997842866, 16.31407745784247, 16.475800732141536, 16.667716126544647, 16.690519896738742, 16.881012506012503, 17.05432819955205, 17.233098175984725, 17.422706922706922, 17.668964035318268, 17.897601709675413, 18.03925187646118, 18.187231624302193, 18.52731375332096, 18.4467466235422, 18.61812551848184, 18.907165362083393, 18.737522236196565, 19.031316894058854, 19.080799617712373, 19.276008384614098, 19.504334014643295, 19.697597524030645, 19.994106429030115, 20.00455665170752, 20.009756001661223, 20.32643109198834, 20.62303985078694, 20.620207018147205, 20.636904574433075, 20.650174096027186, 21.157470817217344, 21.562524710365302, 21.762801505973826, 21.88429108891366, 21.936136832098168, 21.90983701177802, 21.672161403669815, 21.64005455206768, 21.606201887555606, 21.79667076417321, 21.760683911289927, 21.729451995252163, 21.683648265924266, 21.45972076700165, 21.434431578037145, 21.395504851748225, 21.34479896053917, 21.221834942142905, 21.190666427999115, 21.16396511061139, 20.97229478209827, 20.855008200248754, 20.755909967713457, 20.642771521878917, 20.597252294865026, 20.7253775942232, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.28645622177652, "threshold_area100": 16.280157306052015, "threshold_area25": 9.900563088127186}, "test": {"score": [7.8873978710439285, 8.014781095210576, 8.603053112149095, 8.60762316934087, 8.616465158001866, 9.487065400596888, 10.190513348645204, 10.1123013084933, 10.896064218124902, 11.63652152957578, 11.413122845516593, 11.395678397264371, 11.372225669412174, 11.58674000719647, 11.740965856500912, 12.0225759908905, 12.15772276054919, 12.120946086001073, 12.148538684617497, 12.202374527277803, 12.192069129124054, 12.502431494043902, 12.4459889477613, 12.88884965022884, 12.8593976180477, 12.961670155412428, 13.311474680777616, 13.326904151280505, 13.425216628309562, 13.544446478712091, 13.681482743016508, 13.847123464638361, 14.03737820029357, 14.091807343596496, 14.134934647412248, 14.239159079431227, 14.522768808663933, 14.714748127170457, 14.794946406136718, 14.958145308047344, 15.043635336653017, 15.239325337213844, 15.367723577610965, 15.6513140568019, 15.652959223186263, 15.766182813104363, 16.046298342492445, 16.127908226482017, 16.225255976230127, 16.437614399604282, 16.54292490982621, 16.811701281841067, 17.01447245124389, 17.27974715632827, 17.40642687263187, 17.75929051468112, 18.05633654956588, 18.528555288576293, 18.594866598900264, 18.736530993083296, 18.661551481480267, 18.58228817775358, 18.7976006567505, 18.933596450311097, 19.167117088967565, 19.314900152867857, 19.369117117399888, 19.553228916470264, 19.80156155869795, 20.153492594124415, 21.266792300275362, 21.500524217003623, 21.881463944933007, 22.13704663375495, 22.341937491249627, 22.44272875466738, 22.590292073043237, 22.984203488341187, 22.993133826371604, 23.34556410432601, 23.341410546668257, 23.2177979572178, 23.078132475644697, 23.06251240161437, 22.918886099556378, 22.8005678487065, 22.77396123437623, 22.76053435945165, 22.727448319297856, 22.596975883821642, 22.489637486415837, 22.465161649944257, 22.493091987647762, 22.50905203029322, 22.465932246561383, 22.47737589267644, 22.43861037696343, 22.441420561249544, 22.426272661543763, 22.39679757422306, 22.361115727328066], "calls": [0.0, 1.0985352862849533, 1.8308921438082555, 1.9640479360852199, 2.463382157123835, 2.9960053262316912, 3.7283621837549936, 4.194407456724368, 4.993342210386151, 5.792276964047936, 7.1904127829560585, 7.523302263648468, 8.621837549933423, 9.087882822902795, 10.352862849533954, 10.985352862849535, 12.21704394141145, 13.581890812250332, 14.780292942743008, 16.411451398135817, 16.711051930758988, 17.276964047936087, 18.40878828229028, 19.640479360852197, 20.77230359520639, 22.00399467376831, 23.66844207723036, 24.167776298268976, 25.099866844207725, 25.898801597869507, 26.79760319573902, 27.762982689747005, 30.426098535286282, 31.125166444740348, 32.123834886817576, 34.02130492676431, 35.352862849533956, 36.284953395472705, 37.25033288948069, 38.24900133155793, 39.38082556591212, 40.37949400798935, 41.6444740346205, 43.00932090545938, 43.9081225033289, 45.206391478029296, 46.870838881491345, 47.969374167776294, 48.63515312916112, 49.93342210386152, 50.53262316910786, 51.53129161118508, 52.629826897470046, 53.26231691078562, 53.99467376830892, 55.09320905459387, 56.59121171770972, 57.623169107856185, 58.48868175765646, 59.08788282290279, 59.886817576564574, 61.55126498002663, 62.849533954727036, 63.881491344873496, 65.21304926764314, 65.9121171770972, 66.81091877496671, 67.41011984021304, 68.40878828229027, 69.94007989347537, 72.27030625832224, 73.06924101198402, 74.16777629826898, 74.90013315579228, 75.43275632490013, 76.23169107856191, 77.39680426098535, 78.12916111850865, 79.49400798934754, 80.22636484687084, 81.72436750998668, 82.25699067909454, 83.22237017310253, 83.65512649800266, 85.05326231691079, 85.98535286284952, 86.75099866844208, 87.11717709720372, 87.94940079893475, 89.18109187749667, 90.64580559254327, 91.41145139813582, 92.07723035952064, 92.7762982689747, 94.10785619174435, 95.1398135818908, 96.33821571238349, 96.90412782956058, 98.10252996005326, 99.2676431424767, 100.0], "rescued": [0, 93, 154, 165, 204, 251, 310, 345, 408, 470, 569, 592, 659, 686, 758, 797, 868, 940, 1002, 1087, 1104, 1135, 1193, 1255, 1317, 1375, 1452, 1474, 1517, 1555, 1599, 1646, 1770, 1805, 1848, 1941, 2000, 2042, 2080, 2126, 2177, 2223, 2276, 2334, 2370, 2412, 2467, 2500, 2519, 2557, 2574, 2604, 2636, 2652, 2672, 2703, 2743, 2772, 2794, 2812, 2828, 2870, 2906, 2933, 2969, 2988, 3010, 3026, 3050, 3088, 3147, 3171, 3199, 3218, 3231, 3248, 3277, 3296, 3304, 3305, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3307, 3307, 3308, 3309, 3309, 3310, 3310, 3311, 3312, 3314, 3314], "harmed": [0, 23, 35, 37, 48, 54, 66, 76, 92, 104, 130, 134, 147, 157, 183, 192, 216, 238, 253, 281, 289, 301, 326, 347, 372, 397, 423, 427, 443, 458, 475, 487, 538, 548, 563, 602, 623, 640, 651, 664, 680, 696, 719, 738, 748, 766, 788, 805, 815, 837, 846, 859, 878, 893, 905, 926, 960, 982, 1001, 1011, 1032, 1070, 1095, 1120, 1152, 1168, 1189, 1203, 1218, 1256, 1301, 1321, 1346, 1363, 1378, 1397, 1434, 1454, 1472, 1484, 1513, 1524, 1544, 1552, 1581, 1602, 1618, 1626, 1643, 1674, 1712, 1728, 1745, 1761, 1792, 1816, 1841, 1852, 1886, 1923, 1945], "exact_score": [7.8873978710439285, 8.002237216377223, 8.606465614631349, 9.487065400596888, 10.13806900721194, 10.896064218124902, 11.59209247096216, 11.508833638957308, 11.361210486956425, 11.357447529229587, 11.76102762365901, 12.0225759908905, 11.958993961200962, 12.086612634349802, 12.114675631915684, 12.142791684735165, 12.17150050751337, 12.349027962480662, 12.459528923574615, 12.732420245679451, 12.87663584893745, 12.8648144606189, 12.964387757250426, 13.099214620155832, 13.3091784254016, 13.425719857284161, 13.555187691270232, 13.702576328972507, 13.854693840574917, 13.974848666648334, 13.985895534565302, 14.083088964732271, 14.119693247776926, 14.134796424064225, 14.399115504481038, 14.489412609864996, 14.703096798122232, 14.761621482564639, 14.92150862942486, 15.052781960234487, 15.041138125370015, 15.329838693152166, 15.466413382016942, 15.647364886154508, 15.669413437176097, 15.726889222987644, 15.927802581576698, 16.03119479343061, 16.127908226482017, 16.290370577924527, 16.46020374146374, 16.73156009566445, 16.808480515922845, 17.263562341272415, 17.40642687263187, 17.728657152778567, 17.958759507117133, 18.474471266779794, 18.598237074253536, 18.720260898686867, 18.669082414879444, 18.522984772979633, 18.688763740813975, 18.810651618797895, 18.959149586594357, 19.18716101009441, 19.335209685474087, 19.408449642351286, 19.738841582457077, 19.93570216851799, 20.16379367280893, 20.75496603880107, 21.153056050217124, 21.472056901333367, 21.746696512886697, 22.16378329859392, 22.414456191684636, 22.85397251724585, 22.902197429126424, 23.05591532024877, 22.979969618697115, 23.32590919712267, 23.279934925833, 23.08629653972926, 23.05469493855394, 22.922547555548313, 22.8005678487065, 22.76547537654123, 22.726057604587606, 22.607162851704334, 22.515966043033274, 22.480591785978273, 22.496320042099814, 22.50334920274895, 22.47097768368256, 22.48672867980701, 22.451128638131657, 22.43781453354476, 22.447760903589813, 22.406384392875747, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 17.059374557137215, "threshold_area100": 16.959192695553764, "threshold_area25": 11.107873475871244}, "selected_budget": 78, "selected": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405, "rescued": 784, "harmed": 350}, "test": {"score": 22.993133826371604, "calls": 79.49400798934754, "rescued": 3304, "harmed": 1472}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5134528279304504, "operator": "ge"}, {"budget": 0.02, "threshold": 0.49995332956314087, "operator": "ge"}, {"budget": 0.03, "threshold": 0.4926009178161621, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4883396327495575, "operator": "ge"}, {"budget": 0.05, "threshold": 0.4778917729854584, "operator": "ge"}, {"budget": 0.06, "threshold": 0.46820640563964844, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4600778818130493, "operator": "ge"}, {"budget": 0.08, "threshold": 0.44683367013931274, "operator": "ge"}, {"budget": 0.09, "threshold": 0.43200182914733887, "operator": "ge"}, {"budget": 0.1, "threshold": 0.3769170045852661, "operator": "ge"}, {"budget": 0.11, "threshold": 0.3653019666671753, "operator": "ge"}, {"budget": 0.12, "threshold": 0.35370105504989624, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3316904306411743, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3145137429237366, "operator": "ge"}, {"budget": 0.15, "threshold": 0.30100423097610474, "operator": "ge"}, {"budget": 0.16, "threshold": 0.28752976655960083, "operator": "ge"}, {"budget": 0.17, "threshold": 0.27815115451812744, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2695232629776001, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2597832977771759, "operator": "ge"}, {"budget": 0.2, "threshold": 0.25548622012138367, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2506538927555084, "operator": "ge"}, {"budget": 0.22, "threshold": 0.24424807727336884, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2412412017583847, "operator": "ge"}, {"budget": 0.24, "threshold": 0.23626087605953217, "operator": "ge"}, {"budget": 0.25, "threshold": 0.23322883248329163, "operator": "ge"}, {"budget": 0.26, "threshold": 0.23126721382141113, "operator": "ge"}, {"budget": 0.27, "threshold": 0.2283003330230713, "operator": "ge"}, {"budget": 0.28, "threshold": 0.2240438461303711, "operator": "ge"}, {"budget": 0.29, "threshold": 0.2222115695476532, "operator": "ge"}, {"budget": 0.3, "threshold": 0.21910037100315094, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2171158492565155, "operator": "ge"}, {"budget": 0.32, "threshold": 0.21291278302669525, "operator": "ge"}, {"budget": 0.33, "threshold": 0.20818960666656494, "operator": "ge"}, {"budget": 0.34, "threshold": 0.20704565942287445, "operator": "ge"}, {"budget": 0.35, "threshold": 0.20173677802085876, "operator": "ge"}, {"budget": 0.36, "threshold": 0.19813132286071777, "operator": "ge"}, {"budget": 0.37, "threshold": 0.1941879838705063, "operator": "ge"}, {"budget": 0.38, "threshold": 0.19044363498687744, "operator": "ge"}, {"budget": 0.39, "threshold": 0.18687108159065247, "operator": "ge"}, {"budget": 0.4, "threshold": 0.18318910896778107, "operator": "ge"}, {"budget": 0.41, "threshold": 0.1729607880115509, "operator": "ge"}, {"budget": 0.42, "threshold": 0.16562259197235107, "operator": "ge"}, {"budget": 0.43, "threshold": 0.1538163423538208, "operator": "ge"}, {"budget": 0.44, "threshold": 0.14384666085243225, "operator": "ge"}, {"budget": 0.45, "threshold": 0.1255021095275879, "operator": "ge"}, {"budget": 0.46, "threshold": 0.11138768494129181, "operator": "ge"}, {"budget": 0.47, "threshold": 0.10348840057849884, "operator": "ge"}, {"budget": 0.48, "threshold": 0.09430482983589172, "operator": "ge"}, {"budget": 0.49, "threshold": 0.08841314911842346, "operator": "ge"}, {"budget": 0.5, "threshold": 0.08215965330600739, "operator": "ge"}, {"budget": 0.51, "threshold": 0.07823342084884644, "operator": "ge"}, {"budget": 0.52, "threshold": 0.07324710488319397, "operator": "ge"}, {"budget": 0.53, "threshold": 0.06807535886764526, "operator": "ge"}, {"budget": 0.54, "threshold": 0.06514842808246613, "operator": "ge"}, {"budget": 0.55, "threshold": 0.06185637414455414, "operator": "ge"}, {"budget": 0.56, "threshold": 0.0579926073551178, "operator": "ge"}, {"budget": 0.57, "threshold": 0.05309939384460449, "operator": "ge"}, {"budget": 0.58, "threshold": 0.05154041945934296, "operator": "ge"}, {"budget": 0.59, "threshold": 0.047289520502090454, "operator": "ge"}, {"budget": 0.6, "threshold": 0.04481780529022217, "operator": "ge"}, {"budget": 0.61, "threshold": 0.0418439656496048, "operator": "ge"}, {"budget": 0.62, "threshold": 0.038207828998565674, "operator": "ge"}, {"budget": 0.63, "threshold": 0.03589865565299988, "operator": "ge"}, {"budget": 0.64, "threshold": 0.032449349761009216, "operator": "ge"}, {"budget": 0.65, "threshold": 0.029467344284057617, "operator": "ge"}, {"budget": 0.66, "threshold": 0.02722930908203125, "operator": "ge"}, {"budget": 0.67, "threshold": 0.02520693838596344, "operator": "ge"}, {"budget": 0.68, "threshold": 0.0225222110748291, "operator": "ge"}, {"budget": 0.69, "threshold": 0.017204180359840393, "operator": "ge"}, {"budget": 0.7, "threshold": 0.00996243953704834, "operator": "ge"}, {"budget": 0.71, "threshold": 0.006506606936454773, "operator": "ge"}, {"budget": 0.72, "threshold": 0.000814780592918396, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0040642619132995605, "operator": "ge"}, {"budget": 0.74, "threshold": -0.006924659013748169, "operator": "ge"}, {"budget": 0.75, "threshold": -0.01586003601551056, "operator": "ge"}, {"budget": 0.76, "threshold": -0.031513214111328125, "operator": "ge"}, {"budget": 0.77, "threshold": -0.06450513005256653, "operator": "ge"}, {"budget": 0.78, "threshold": -0.12286390364170074, "operator": "ge"}, {"budget": 0.79, "threshold": -0.13375627994537354, "operator": "ge"}, {"budget": 0.8, "threshold": -0.14565031230449677, "operator": "ge"}, {"budget": 0.81, "threshold": -0.14894865453243256, "operator": "ge"}, {"budget": 0.82, "threshold": -0.15463662147521973, "operator": "ge"}, {"budget": 0.83, "threshold": -0.15940441191196442, "operator": "ge"}, {"budget": 0.84, "threshold": -0.1671046018600464, "operator": "ge"}, {"budget": 0.85, "threshold": -0.17699795961380005, "operator": "ge"}, {"budget": 0.86, "threshold": -0.18610712885856628, "operator": "ge"}, {"budget": 0.87, "threshold": -0.19331224262714386, "operator": "ge"}, {"budget": 0.88, "threshold": -0.1986115574836731, "operator": "ge"}, {"budget": 0.89, "threshold": -0.20333941280841827, "operator": "ge"}, {"budget": 0.9, "threshold": -0.2116299420595169, "operator": "ge"}, {"budget": 0.91, "threshold": -0.21520277857780457, "operator": "ge"}, {"budget": 0.92, "threshold": -0.2216687798500061, "operator": "ge"}, {"budget": 0.93, "threshold": -0.2246985137462616, "operator": "ge"}, {"budget": 0.94, "threshold": -0.23010101914405823, "operator": "ge"}, {"budget": 0.95, "threshold": -0.23582816123962402, "operator": "ge"}, {"budget": 0.96, "threshold": -0.2405387908220291, "operator": "ge"}, {"budget": 0.97, "threshold": -0.24477963149547577, "operator": "ge"}, {"budget": 0.98, "threshold": -0.2493499219417572, "operator": "ge"}, {"budget": 0.99, "threshold": -0.2621338367462158, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.737353307510221, 7.850490860690032, 7.9794837885934236, 9.253677415833996, 9.27130999651781, 9.245608677950328, 9.246688919457613, 9.297571774707453, 9.290766119279034, 9.373459811821268, 9.48167416055598, 9.48462326156339, 9.57992650479917, 9.65281217991059, 9.776914501248116, 9.880536817774423, 9.967970837536056, 10.123573003285543, 10.961811579565166, 11.033363055189612, 11.130282531797798, 11.213144285433444, 11.083636772435856, 12.23129283018623, 12.32807756685688, 12.92933108188174, 13.034418022528161, 12.848180015901534, 12.983765908275787, 13.528551478964001, 13.490737213729295, 13.551100182102205, 13.517997221041961, 13.634509599737822, 13.600390634319753, 13.712706137266368, 14.255568764386567, 14.386390562543601, 14.543650793650794, 15.031681124115396, 15.429889503424189, 15.948080221321733, 16.070584977980047, 16.327051965280145, 16.48904735204895, 16.472110611992456, 16.70589040921622, 16.883880485139066, 17.083697301634917, 17.223986103296447, 17.42963517637699, 17.554281159907596, 17.794255327116804, 18.04315247642717, 18.155915036817227, 18.405815887830276, 18.456595929166156, 18.638246242827915, 18.857181604877457, 18.895901165495992, 18.809358070276325, 19.080799617712373, 19.270830415566646, 19.49938828971087, 19.742724498881124, 20.013261172977895, 20.00455665170752, 20.009756001661223, 20.32643109198834, 20.62303985078694, 20.620207018147205, 20.636904574433075, 20.650174096027186, 21.157470817217344, 21.562524710365302, 21.762801505973826, 21.88429108891366, 21.936136832098168, 21.90983701177802, 21.675102249488752, 21.64005455206768, 21.606201887555606, 21.79667076417321, 21.766559351947976, 21.733558024769835, 21.683648265924266, 21.64444983104106, 21.434431578037145, 21.395504851748225, 21.34479896053917, 21.221834942142905, 21.190666427999115, 21.014203125272207, 20.97229478209827, 20.855008200248754, 20.838608328643453, 20.795098610927397, 20.597252294865026, 20.7253775942232, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 18, 38, 59, 76, 95, 114, 134, 149, 162, 179, 195, 209, 222, 235, 250, 262, 273, 284, 300, 311, 323, 334, 347, 360, 370, 383, 396, 406, 418, 430, 440, 450, 459, 469, 479, 489, 502, 513, 524, 535, 548, 562, 569, 579, 588, 593, 602, 609, 616, 621, 628, 636, 643, 650, 657, 664, 671, 677, 684, 689, 693, 700, 707, 713, 720, 727, 731, 737, 744, 750, 754, 758, 763, 769, 774, 779, 783, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786], "harmed": [0, 6, 12, 18, 24, 27, 32, 38, 42, 46, 49, 52, 57, 58, 63, 66, 70, 74, 76, 81, 85, 89, 93, 100, 103, 105, 108, 111, 118, 120, 123, 130, 134, 139, 142, 148, 151, 153, 156, 159, 162, 167, 171, 173, 175, 177, 181, 183, 189, 192, 196, 200, 204, 207, 211, 216, 221, 229, 234, 239, 243, 252, 256, 263, 270, 276, 279, 285, 291, 296, 299, 306, 312, 319, 324, 328, 337, 347, 350, 353, 359, 364, 373, 374, 377, 382, 388, 393, 398, 404, 411, 418, 423, 429, 435, 443, 446, 453, 462, 467, 475], "exact_score": [7.641657905799261, 7.737353307510221, 7.850490860690032, 7.9794837885934236, 9.253677415833996, 9.27130999651781, 9.245608677950328, 9.246688919457613, 9.297571774707453, 9.290766119279034, 9.373459811821268, 9.48167416055598, 9.48462326156339, 9.57992650479917, 9.65281217991059, 9.776914501248116, 9.880536817774423, 9.967970837536056, 10.123573003285543, 10.961811579565166, 11.033363055189612, 11.130282531797798, 11.213144285433444, 11.083636772435856, 12.23129283018623, 12.32807756685688, 12.92933108188174, 13.034418022528161, 12.848180015901534, 12.983765908275787, 13.528551478964001, 13.490737213729295, 13.551100182102205, 13.517997221041961, 13.634509599737822, 13.600390634319753, 13.712706137266368, 14.255568764386567, 14.386390562543601, 14.543650793650794, 15.031681124115396, 15.429889503424189, 15.948080221321733, 16.070584977980047, 16.327051965280145, 16.48904735204895, 16.472110611992456, 16.70589040921622, 16.883880485139066, 17.083697301634917, 17.223986103296447, 17.42963517637699, 17.554281159907596, 17.794255327116804, 18.04315247642717, 18.155915036817227, 18.405815887830276, 18.456595929166156, 18.638246242827915, 18.857181604877457, 18.895901165495992, 18.809358070276325, 19.080799617712373, 19.270830415566646, 19.49938828971087, 19.742724498881124, 20.013261172977895, 20.00455665170752, 20.009756001661223, 20.32643109198834, 20.62303985078694, 20.620207018147205, 20.636904574433075, 20.650174096027186, 21.157470817217344, 21.562524710365302, 21.762801505973826, 21.88429108891366, 21.936136832098168, 21.90983701177802, 21.675102249488752, 21.64005455206768, 21.606201887555606, 21.79667076417321, 21.766559351947976, 21.733558024769835, 21.683648265924266, 21.64444983104106, 21.434431578037145, 21.395504851748225, 21.34479896053917, 21.221834942142905, 21.190666427999115, 21.014203125272207, 20.97229478209827, 20.855008200248754, 20.838608328643453, 20.795098610927397, 20.597252294865026, 20.7253775942232, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.286058319793632, "threshold_area100": 16.27975883563773, "threshold_area25": 9.887983020220977}, "test": {"score": [7.8873978710439285, 8.014781095210576, 8.597989602730841, 8.602628994356184, 8.614856765032544, 9.463114174155908, 10.199937939140344, 10.101013677525378, 10.883849680344301, 11.630926050602932, 11.345173572399943, 11.364196866063953, 11.33622427509975, 11.309360777134101, 11.736158229375434, 11.70998151471331, 12.13541981324607, 12.122226960441617, 12.265491489149639, 12.26948577375768, 12.285122318628952, 12.450338361211198, 12.824625926423206, 12.783109285324803, 12.92858942836729, 13.147924992452884, 13.204689887010343, 13.35209039797257, 13.592152347916645, 13.609863288597396, 13.837216764693835, 13.960346494281605, 13.842275115147922, 13.938852561241452, 13.958782692706318, 14.415958994193817, 14.618385041288063, 14.743005878607162, 14.799744405951461, 14.95579563026905, 14.979165172541158, 15.078916493702776, 15.421820602879183, 15.451098634290211, 15.579857542741468, 15.926690534827149, 15.947242219681035, 16.091629503468887, 16.31259620243903, 16.486945494657412, 16.6459950558253, 16.664867970816616, 16.84147019536054, 17.088484475551354, 17.540382724404612, 17.65294783240547, 18.02466099156635, 18.476879595283517, 18.45467519900872, 18.650058052430403, 18.697948606807486, 18.41986590956449, 18.690685778512474, 18.875756285438538, 19.208943190348684, 19.371276818789244, 19.505536503828615, 19.623686443607323, 19.803324501637366, 20.16379367280893, 21.186913890298655, 21.486767110218334, 21.867824204780728, 22.13704663375495, 22.35570632539153, 22.44272875466738, 22.5582845867967, 22.93404153501415, 22.993133826371604, 23.34556410432601, 23.28156889589867, 23.27444730100573, 23.194874599550122, 23.061541719514803, 22.942929126436617, 22.80180917554481, 22.774599622356977, 22.75484134762367, 22.729538697981567, 22.679068244776936, 22.497461031475137, 22.474837843607748, 22.485998499360697, 22.465065261742474, 22.46228949818059, 22.463870025362816, 22.402075778499757, 22.423415478386428, 22.451275280586618, 22.398518638824452, 22.361115727328066], "calls": [0.0, 1.0985352862849533, 1.7310252996005324, 2.130492676431425, 2.4966711051930757, 3.162450066577896, 3.695073235685752, 4.194407456724368, 5.026631158455393, 5.758988015978695, 7.589880159786951, 7.989347536617843, 8.255659121171771, 9.054593874833555, 10.086551264980026, 11.051930758988016, 12.48335552596538, 13.249001331557922, 14.447403462050598, 16.145139813581892, 16.844207723035954, 17.743009320905458, 19.174434087882823, 19.973368841544605, 21.13848202396804, 22.303595206391478, 23.135818908122506, 24.234354194407455, 25.532623169107854, 26.331557922769637, 27.5965379494008, 28.29560585885486, 30.09320905459387, 31.924101198402134, 32.323568575233026, 34.354194407456724, 35.25299600532623, 36.38482023968043, 37.35019973368841, 38.41544607190413, 39.21438082556591, 40.612516644474034, 41.67776298268975, 43.075898801597866, 43.9081225033289, 45.772303595206395, 47.20372836218375, 48.06924101198402, 49.00133155792277, 49.73368841544607, 50.7323568575233, 51.33155792276965, 52.36351531291611, 53.42876165113183, 54.02796271637816, 55.026631158455395, 56.15845539280959, 57.822902796271634, 58.15579227696405, 59.52063914780293, 60.05326231691078, 61.61784287616512, 63.04926764314248, 64.0479360852197, 65.5126498002663, 66.24500665778962, 66.94407456724367, 67.54327563249002, 68.44207723035952, 70.0399467376831, 72.30359520639148, 73.00266311584554, 74.13448735019973, 74.90013315579228, 75.46604527296937, 76.19840213049268, 77.39680426098535, 78.09587217043942, 79.36085219707057, 80.32623169107856, 81.89081225033289, 82.15712383488682, 82.92276964047936, 83.75499334221038, 84.88681757656458, 85.98535286284952, 86.75099866844208, 87.38348868175765, 87.94940079893475, 88.81491344873503, 90.41278295605859, 91.17842876165113, 92.27696404793608, 92.9427430093209, 94.20772303595206, 95.57256990679095, 96.5046604527297, 97.30359520639148, 97.86950732356857, 99.23435419440746, 100.0], "rescued": [0, 93, 146, 178, 207, 263, 307, 346, 411, 468, 598, 624, 642, 689, 748, 802, 885, 927, 990, 1079, 1115, 1162, 1236, 1277, 1339, 1393, 1434, 1486, 1548, 1589, 1647, 1677, 1756, 1845, 1865, 1962, 2002, 2049, 2092, 2139, 2175, 2234, 2283, 2342, 2373, 2439, 2480, 2505, 2532, 2555, 2581, 2599, 2627, 2656, 2672, 2700, 2731, 2777, 2783, 2819, 2833, 2869, 2910, 2937, 2976, 2997, 3015, 3029, 3051, 3089, 3148, 3169, 3198, 3218, 3232, 3248, 3277, 3295, 3304, 3305, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3307, 3307, 3308, 3308, 3309, 3310, 3310, 3311, 3312, 3314, 3314], "harmed": [0, 23, 34, 42, 49, 58, 65, 77, 92, 103, 136, 140, 145, 159, 178, 194, 219, 233, 251, 279, 292, 310, 343, 360, 384, 406, 422, 439, 456, 471, 490, 501, 535, 569, 576, 608, 622, 638, 655, 671, 682, 704, 723, 741, 752, 776, 798, 807, 818, 833, 847, 860, 872, 892, 902, 926, 950, 987, 993, 1023, 1034, 1073, 1105, 1126, 1158, 1175, 1191, 1205, 1218, 1257, 1302, 1320, 1345, 1363, 1379, 1397, 1434, 1453, 1472, 1484, 1517, 1521, 1536, 1553, 1577, 1602, 1618, 1630, 1642, 1663, 1707, 1722, 1749, 1764, 1791, 1824, 1844, 1864, 1878, 1922, 1945], "exact_score": [7.8873978710439285, 8.004847255632214, 8.599042442848619, 9.476734822084493, 10.138577892413748, 10.894634859568859, 11.59225928441229, 11.388460172930023, 11.364196866063953, 11.312752066477803, 11.735343497405347, 11.700752948367393, 12.122414317013538, 12.121822899893111, 12.286053872763087, 12.2710952406132, 12.257965607854974, 12.289625082867998, 12.602196263024412, 12.826586666200548, 12.783109285324803, 12.924827959616245, 12.99833646306367, 13.198447922880533, 13.226047863206947, 13.46955602592444, 13.61354149145286, 13.677216915471554, 13.96108101159822, 13.964150590418944, 13.85693285476436, 13.93516825277435, 13.915592394292, 14.18857584188212, 14.200782112517253, 14.57701604818843, 14.771297741243455, 14.74725515549953, 14.922680983120737, 14.98620089215182, 15.0179971924362, 15.283559237671465, 15.449521004288336, 15.617691412135242, 15.588246113086127, 15.783913498354977, 15.896546716024556, 15.900725159732234, 16.07117740350751, 16.309480889020804, 16.541650604262603, 16.63527864886688, 16.781267709721305, 16.98823097632223, 17.536330853415887, 17.647288878509, 17.932860661601723, 18.11499635313611, 18.467355038735665, 18.591965180735883, 18.690321893644285, 18.72208024518483, 18.446967114902087, 18.68401372424584, 18.9076872109063, 19.058834260649064, 19.289035066812847, 19.46432969333558, 19.68201896548489, 19.93628242962791, 20.153492594124415, 20.466395552150253, 21.16450754548606, 21.472056901333367, 21.746696512886697, 22.16378329859392, 22.43348214758423, 22.708217376179505, 22.902197429126424, 23.053637998519264, 22.978255683501907, 23.326454061710205, 23.279934925833, 23.192670512757342, 23.05192547460477, 22.939263836722414, 22.80180917554481, 22.76468504012412, 22.729538697981567, 22.671630662321967, 22.56358509065232, 22.482579433119373, 22.497606973822936, 22.463144114176995, 22.47097768368256, 22.484851943826225, 22.451779521476023, 22.437198467855175, 22.431531612277915, 22.390211005449515, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 17.03937417881067, "threshold_area100": 16.950811831055137, "threshold_area25": 11.064210602329306}, "selected_budget": 78, "selected": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405, "rescued": 784, "harmed": 350}, "test": {"score": 22.993133826371604, "calls": 79.36085219707057, "rescued": 3304, "harmed": 1472}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.017225503921509, "operator": "ge"}, {"budget": 0.02, "threshold": 1.739428997039795, "operator": "ge"}, {"budget": 0.03, "threshold": 1.660774827003479, "operator": "ge"}, {"budget": 0.04, "threshold": 1.5848822593688965, "operator": "ge"}, {"budget": 0.05, "threshold": 1.511816143989563, "operator": "ge"}, {"budget": 0.06, "threshold": 1.4787976741790771, "operator": "ge"}, {"budget": 0.07, "threshold": 1.4534382820129395, "operator": "ge"}, {"budget": 0.08, "threshold": 1.4268102645874023, "operator": "ge"}, {"budget": 0.09, "threshold": 1.3841451406478882, "operator": "ge"}, {"budget": 0.1, "threshold": 1.3621500730514526, "operator": "ge"}, {"budget": 0.11, "threshold": 1.340090036392212, "operator": "ge"}, {"budget": 0.12, "threshold": 1.3201147317886353, "operator": "ge"}, {"budget": 0.13, "threshold": 1.284066081047058, "operator": "ge"}, {"budget": 0.14, "threshold": 1.2598905563354492, "operator": "ge"}, {"budget": 0.15, "threshold": 1.2379817962646484, "operator": "ge"}, {"budget": 0.16, "threshold": 1.2215219736099243, "operator": "ge"}, {"budget": 0.17, "threshold": 1.1863775253295898, "operator": "ge"}, {"budget": 0.18, "threshold": 1.166208028793335, "operator": "ge"}, {"budget": 0.19, "threshold": 1.1459276676177979, "operator": "ge"}, {"budget": 0.2, "threshold": 1.1219168901443481, "operator": "ge"}, {"budget": 0.21, "threshold": 1.099726915359497, "operator": "ge"}, {"budget": 0.22, "threshold": 1.084705114364624, "operator": "ge"}, {"budget": 0.23, "threshold": 1.0556081533432007, "operator": "ge"}, {"budget": 0.24, "threshold": 1.0462440252304077, "operator": "ge"}, {"budget": 0.25, "threshold": 1.0270652770996094, "operator": "ge"}, {"budget": 0.26, "threshold": 1.0092732906341553, "operator": "ge"}, {"budget": 0.27, "threshold": 0.9899866580963135, "operator": "ge"}, {"budget": 0.28, "threshold": 0.9692094326019287, "operator": "ge"}, {"budget": 0.29, "threshold": 0.9579432010650635, "operator": "ge"}, {"budget": 0.3, "threshold": 0.927458643913269, "operator": "ge"}, {"budget": 0.31, "threshold": 0.9069013595581055, "operator": "ge"}, {"budget": 0.32, "threshold": 0.8747014403343201, "operator": "ge"}, {"budget": 0.33, "threshold": 0.8661364316940308, "operator": "ge"}, {"budget": 0.34, "threshold": 0.8395349979400635, "operator": "ge"}, {"budget": 0.35, "threshold": 0.8327801823616028, "operator": "ge"}, {"budget": 0.36, "threshold": 0.8203648328781128, "operator": "ge"}, {"budget": 0.37, "threshold": 0.7950592637062073, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7737091183662415, "operator": "ge"}, {"budget": 0.39, "threshold": 0.7440610527992249, "operator": "ge"}, {"budget": 0.4, "threshold": 0.7138677835464478, "operator": "ge"}, {"budget": 0.41, "threshold": 0.6936405897140503, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6741797924041748, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6344009637832642, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6076069474220276, "operator": "ge"}, {"budget": 0.45, "threshold": 0.586773693561554, "operator": "ge"}, {"budget": 0.46, "threshold": 0.5523014664649963, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5292733311653137, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5128113031387329, "operator": "ge"}, {"budget": 0.49, "threshold": 0.4790758490562439, "operator": "ge"}, {"budget": 0.5, "threshold": 0.45123207569122314, "operator": "ge"}, {"budget": 0.51, "threshold": 0.419179230928421, "operator": "ge"}, {"budget": 0.52, "threshold": 0.4022417366504669, "operator": "ge"}, {"budget": 0.53, "threshold": 0.36768388748168945, "operator": "ge"}, {"budget": 0.54, "threshold": 0.3343338370323181, "operator": "ge"}, {"budget": 0.55, "threshold": 0.3119998872280121, "operator": "ge"}, {"budget": 0.56, "threshold": 0.28967416286468506, "operator": "ge"}, {"budget": 0.57, "threshold": 0.2676923871040344, "operator": "ge"}, {"budget": 0.58, "threshold": 0.25875020027160645, "operator": "ge"}, {"budget": 0.59, "threshold": 0.24650195240974426, "operator": "ge"}, {"budget": 0.6, "threshold": 0.22767865657806396, "operator": "ge"}, {"budget": 0.61, "threshold": 0.2111758589744568, "operator": "ge"}, {"budget": 0.62, "threshold": 0.1937672197818756, "operator": "ge"}, {"budget": 0.63, "threshold": 0.17621606588363647, "operator": "ge"}, {"budget": 0.64, "threshold": 0.16469810903072357, "operator": "ge"}, {"budget": 0.65, "threshold": 0.14700035750865936, "operator": "ge"}, {"budget": 0.66, "threshold": 0.13381201028823853, "operator": "ge"}, {"budget": 0.67, "threshold": 0.11982786655426025, "operator": "ge"}, {"budget": 0.68, "threshold": 0.10338819026947021, "operator": "ge"}, {"budget": 0.69, "threshold": 0.07362636923789978, "operator": "ge"}, {"budget": 0.7, "threshold": 0.05776900053024292, "operator": "ge"}, {"budget": 0.71, "threshold": 0.02549433708190918, "operator": "ge"}, {"budget": 0.72, "threshold": 0.004012346267700195, "operator": "ge"}, {"budget": 0.73, "threshold": -0.018461227416992188, "operator": "ge"}, {"budget": 0.74, "threshold": -0.031051352620124817, "operator": "ge"}, {"budget": 0.75, "threshold": -0.06808307766914368, "operator": "ge"}, {"budget": 0.76, "threshold": -0.15225881338119507, "operator": "ge"}, {"budget": 0.77, "threshold": -0.34906309843063354, "operator": "ge"}, {"budget": 0.78, "threshold": -2.8626151084899902, "operator": "ge"}, {"budget": 0.79, "threshold": -3.0851824283599854, "operator": "ge"}, {"budget": 0.8, "threshold": -3.2288928031921387, "operator": "ge"}, {"budget": 0.81, "threshold": -3.2981224060058594, "operator": "ge"}, {"budget": 0.82, "threshold": -3.335921287536621, "operator": "ge"}, {"budget": 0.83, "threshold": -3.395745277404785, "operator": "ge"}, {"budget": 0.84, "threshold": -3.4446473121643066, "operator": "ge"}, {"budget": 0.85, "threshold": -3.478989601135254, "operator": "ge"}, {"budget": 0.86, "threshold": -3.517972946166992, "operator": "ge"}, {"budget": 0.87, "threshold": -3.545579195022583, "operator": "ge"}, {"budget": 0.88, "threshold": -3.5730233192443848, "operator": "ge"}, {"budget": 0.89, "threshold": -3.5994739532470703, "operator": "ge"}, {"budget": 0.9, "threshold": -3.6213362216949463, "operator": "ge"}, {"budget": 0.91, "threshold": -3.677164077758789, "operator": "ge"}, {"budget": 0.92, "threshold": -3.7229955196380615, "operator": "ge"}, {"budget": 0.93, "threshold": -3.7604117393493652, "operator": "ge"}, {"budget": 0.94, "threshold": -3.778796672821045, "operator": "ge"}, {"budget": 0.95, "threshold": -3.8333075046539307, "operator": "ge"}, {"budget": 0.96, "threshold": -3.850973129272461, "operator": "ge"}, {"budget": 0.97, "threshold": -3.909665584564209, "operator": "ge"}, {"budget": 0.98, "threshold": -3.9459738731384277, "operator": "ge"}, {"budget": 0.99, "threshold": -4.043704986572266, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.740411126941578, 7.6312696206647495, 10.268534084323559, 9.930534916811974, 9.668975401213045, 9.479457753420695, 10.836136057245465, 10.602369370455225, 10.416938366568896, 10.346705391633959, 10.338420393866674, 10.347071858149711, 10.379804406437444, 10.372830446081553, 10.390678942088913, 10.436230011891169, 10.523836880743898, 10.56720820675384, 10.614527690270739, 11.327479963913595, 11.410049728662232, 11.512631960056819, 11.504156706621496, 11.63610259754284, 11.735740580456435, 11.83331455669277, 11.935300340223575, 12.525389042515817, 12.520323244007454, 12.64234989156113, 12.754007543611504, 13.771298220523533, 14.355898289821997, 14.424425884233601, 14.438933616999078, 14.436374375243874, 14.555124901699878, 14.698937751721521, 15.102018872064955, 15.105333095450332, 14.991785367552241, 15.180082968045442, 15.167954733501993, 15.341495608581882, 15.50593971697558, 15.548597496935807, 16.429200952756013, 16.673000809452216, 16.85598955248469, 17.037900979339504, 17.220114375299712, 17.38553915666285, 17.572663542915258, 17.704880357828877, 17.934281958158362, 18.152763574096305, 18.129175440820028, 17.993786163654715, 18.15294437090588, 18.435416709298643, 18.471737984804545, 18.67984536551114, 18.915742436500853, 19.15707269814704, 19.416433209622763, 19.68198134753542, 19.825243391032863, 19.972571721418632, 20.32643109198834, 20.62303985078694, 20.693609548968062, 20.636904574433075, 20.931990458598598, 21.157470817217344, 21.49728997289973, 21.762801505973826, 21.88429108891366, 21.821690914277564, 21.686351930840342, 21.562822115792248, 21.34908270788326, 21.137846768120934, 21.09099005895115, 21.05154070198473, 21.031296992990008, 21.191341545773906, 21.142485087720402, 21.09235884381239, 21.051746235595743, 21.009489347956766, 20.988782263546543, 20.877506445361842, 21.01501624085381, 20.919457247448882, 20.881700525146027, 20.847781626276202, 20.819199850027015, 20.72462780014751, 20.689793532200838, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 16, 32, 42, 59, 76, 94, 112, 121, 135, 149, 167, 185, 197, 210, 224, 239, 254, 268, 280, 293, 305, 313, 324, 338, 348, 358, 371, 384, 394, 406, 417, 431, 443, 457, 464, 477, 489, 500, 510, 520, 530, 544, 554, 564, 574, 582, 594, 606, 612, 620, 626, 633, 640, 646, 654, 661, 667, 672, 678, 685, 692, 698, 704, 711, 718, 723, 731, 737, 744, 750, 755, 758, 764, 769, 775, 779, 783, 784, 784, 784, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786, 786, 786, 786, 786], "harmed": [0, 1, 6, 10, 14, 18, 23, 26, 30, 35, 39, 42, 46, 48, 51, 55, 59, 61, 65, 69, 73, 75, 75, 82, 85, 88, 91, 95, 99, 102, 105, 109, 113, 114, 120, 126, 131, 135, 137, 142, 149, 154, 160, 165, 168, 173, 177, 179, 185, 190, 195, 198, 201, 205, 211, 215, 219, 225, 233, 239, 244, 250, 255, 260, 266, 272, 277, 283, 290, 296, 299, 305, 312, 318, 324, 330, 337, 347, 355, 363, 370, 377, 381, 387, 392, 395, 398, 405, 412, 418, 424, 427, 433, 438, 442, 448, 454, 459, 464, 470, 475], "exact_score": [7.641657905799261, 7.740411126941578, 7.6312696206647495, 10.268534084323559, 9.930534916811974, 9.668975401213045, 9.479457753420695, 10.836136057245465, 10.602369370455225, 10.416938366568896, 10.346705391633959, 10.338420393866674, 10.347071858149711, 10.379804406437444, 10.372830446081553, 10.390678942088913, 10.436230011891169, 10.523836880743898, 10.56720820675384, 10.614527690270739, 11.327479963913595, 11.410049728662232, 11.512631960056819, 11.504156706621496, 11.63610259754284, 11.735740580456435, 11.83331455669277, 11.935300340223575, 12.525389042515817, 12.520323244007454, 12.64234989156113, 12.754007543611504, 13.771298220523533, 14.355898289821997, 14.424425884233601, 14.438933616999078, 14.436374375243874, 14.555124901699878, 14.698937751721521, 15.102018872064955, 15.105333095450332, 14.991785367552241, 15.180082968045442, 15.167954733501993, 15.341495608581882, 15.50593971697558, 15.548597496935807, 16.429200952756013, 16.673000809452216, 16.85598955248469, 17.037900979339504, 17.220114375299712, 17.38553915666285, 17.572663542915258, 17.704880357828877, 17.934281958158362, 18.152763574096305, 18.129175440820028, 17.993786163654715, 18.15294437090588, 18.435416709298643, 18.471737984804545, 18.67984536551114, 18.915742436500853, 19.15707269814704, 19.416433209622763, 19.68198134753542, 19.825243391032863, 19.972571721418632, 20.32643109198834, 20.62303985078694, 20.693609548968062, 20.636904574433075, 20.931990458598598, 21.157470817217344, 21.49728997289973, 21.762801505973826, 21.88429108891366, 21.821690914277564, 21.686351930840342, 21.562822115792248, 21.34908270788326, 21.137846768120934, 21.09099005895115, 21.05154070198473, 21.031296992990008, 21.191341545773906, 21.142485087720402, 21.09235884381239, 21.051746235595743, 21.009489347956766, 20.988782263546543, 20.877506445361842, 21.01501624085381, 20.919457247448882, 20.881700525146027, 20.847781626276202, 20.819199850027015, 20.72462780014751, 20.689793532200838, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.24934500612199, "threshold_area100": 16.243018491562292, "threshold_area25": 10.284507788395473}, "test": {"score": [7.8873978710439285, 8.694324324043851, 9.325609455353403, 9.29710065003662, 9.68354883554228, 9.597476251719353, 9.953102348906098, 10.207420333391717, 10.209293517914976, 10.474405160925917, 10.734132677461227, 10.703372529217562, 10.626245290651362, 10.805088564299771, 11.110317716527828, 11.578785363399465, 11.555550374997809, 11.524947329490708, 11.755136240282512, 11.668588190881064, 11.648099492978686, 11.631543120654927, 11.850190495066395, 12.151868207596687, 12.167193749457894, 12.458293752546865, 12.656496523883181, 12.959553389331774, 12.928072745024027, 12.951002291993014, 12.983521442869522, 13.197477084430705, 13.446057726596136, 13.65805823486328, 13.903099284767748, 13.900651410820842, 13.916122067145057, 14.142881551198938, 14.481762170765359, 14.79788866169223, 15.002420649667169, 14.996218514457697, 15.08808706231813, 15.16652955663397, 15.306399918386088, 15.653732061282803, 15.7228994686579, 15.828412880203729, 15.859579418580234, 16.028796080086074, 16.104829010657472, 16.14765671432244, 16.29526572613377, 16.61290253840322, 17.189693034404257, 17.483185808586978, 17.657039566295335, 18.06893037128524, 18.11379545374851, 18.189538437394404, 18.083209266098503, 18.20963209743321, 18.35051334469692, 18.432882848386782, 18.648588563093217, 19.017990918701585, 19.2875917144468, 19.361729010851818, 19.650646316446043, 20.170307436852845, 20.66154380066421, 21.551879090940883, 21.852488908940522, 22.14786003905444, 22.34271030524642, 22.473808314831796, 22.63862963653765, 23.050282754852525, 22.935294906336086, 22.805215685295156, 22.76296214522765, 22.73391832681734, 22.699433249720137, 22.641743930034668, 22.56275432152516, 22.88591140620399, 22.81461194862808, 22.774437845593972, 22.7390432430794, 22.717916907273597, 22.751451929700888, 22.69469638055301, 22.63598708226696, 22.540058499592554, 22.495058109274442, 22.491675702964546, 22.478438248772886, 22.46938397255875, 22.441667656839652, 22.37452182574511, 22.361115727328066], "calls": [0.0, 0.4327563249001331, 1.6644474034620507, 2.263648468708389, 3.129161118508655, 3.9280958721704398, 4.560585885486019, 5.093209054593874, 5.625832223701731, 6.624500665778961, 7.157123834886818, 7.723035952063914, 8.355525965379494, 9.454061251664447, 10.153129161118509, 10.952063914780293, 11.484687083888149, 12.749667110519308, 13.648468708388814, 14.214380825565911, 15.14647137150466, 16.145139813581892, 17.07723035952064, 18.808255659121173, 19.174434087882823, 20.10652463382157, 21.304926764314246, 22.270306258322236, 23.66844207723036, 24.467376830892142, 25.798934753661783, 27.330226364846872, 29.360852197070574, 30.09320905459387, 31.857523302263647, 32.2237017310253, 33.32223701731025, 35.05326231691079, 36.58455392809587, 38.1491344873502, 39.74700399467377, 40.94540612516644, 42.110519307589875, 44.241011984021306, 45.43941411451398, 46.40479360852197, 47.470039946737685, 48.1358189081225, 48.53528628495339, 49.93342210386152, 50.56591211717709, 51.2982689747004, 51.96404793608522, 53.12916111850865, 54.32756324900133, 55.45938748335553, 56.55792276964048, 57.822902796271634, 58.22237017310253, 58.95472703062583, 60.21970705725699, 61.18508655126498, 62.31691078561917, 63.38215712383488, 64.31424766977364, 65.5126498002663, 66.57789613848203, 67.24367509986683, 68.40878828229027, 70.50599201065246, 71.47137150466045, 73.20239680426099, 74.10119840213049, 74.96671105193076, 75.3994673768309, 76.1318242343542, 77.43009320905459, 78.26231691078563, 79.06125166444741, 80.25965379494008, 81.3249001331558, 82.15712383488682, 82.52330226364847, 83.62183754993342, 84.4207723035952, 85.15312916111851, 85.88548601864181, 86.81757656458056, 87.54993342210386, 88.11584553928095, 88.61517976031958, 89.84687083888149, 91.54460719041279, 92.80958721704394, 93.20905459387482, 94.34087882822902, 94.74034620505992, 96.271637816245, 97.03728362183755, 98.7017310252996, 100.0], "rescued": [0, 34, 122, 163, 226, 274, 316, 353, 383, 446, 480, 509, 549, 615, 653, 699, 729, 791, 841, 867, 915, 967, 1013, 1096, 1113, 1159, 1217, 1267, 1337, 1373, 1435, 1510, 1608, 1648, 1734, 1753, 1809, 1897, 1969, 2046, 2129, 2175, 2228, 2326, 2371, 2415, 2453, 2477, 2489, 2534, 2554, 2576, 2596, 2630, 2667, 2701, 2731, 2768, 2778, 2798, 2830, 2855, 2883, 2910, 2934, 2969, 2999, 3015, 3045, 3098, 3122, 3174, 3197, 3220, 3230, 3247, 3278, 3300, 3304, 3304, 3305, 3306, 3306, 3306, 3306, 3307, 3308, 3308, 3309, 3309, 3310, 3310, 3311, 3311, 3311, 3312, 3312, 3313, 3313, 3313, 3314], "harmed": [0, 2, 13, 19, 35, 47, 55, 65, 69, 82, 91, 100, 111, 131, 136, 144, 150, 167, 182, 196, 209, 224, 234, 258, 263, 277, 295, 309, 334, 346, 365, 395, 434, 452, 490, 498, 524, 564, 586, 618, 645, 669, 692, 728, 745, 762, 780, 791, 796, 817, 828, 844, 859, 878, 902, 924, 951, 971, 985, 1004, 1031, 1052, 1078, 1109, 1124, 1149, 1179, 1194, 1214, 1262, 1284, 1323, 1345, 1366, 1377, 1396, 1435, 1457, 1476, 1504, 1527, 1547, 1556, 1579, 1596, 1613, 1632, 1654, 1672, 1688, 1698, 1722, 1759, 1790, 1798, 1825, 1834, 1866, 1884, 1920, 1945], "exact_score": [7.8873978710439285, 8.724624301005322, 9.311911633340204, 9.737927841493537, 9.98948807043538, 10.196248640178732, 10.175498393866828, 10.750054907905918, 10.663289604848792, 10.855761147526382, 11.091424808750324, 11.58198477959402, 11.483823105913018, 11.495843178105563, 11.676579686491603, 11.64231582829521, 11.627553893635119, 11.858354124288468, 12.041952833356161, 12.148930545777324, 12.463135731494745, 12.54264396832942, 12.9764214401319, 12.97632527585096, 12.93381195407251, 12.987693830552102, 12.983476335270637, 13.099833201859573, 13.258543332666148, 13.44067349296068, 13.660756100235336, 13.81122342477646, 13.892655364261847, 13.951219706100796, 13.921941719879138, 14.067874888871176, 14.221321748174665, 14.5966141546075, 14.782065612697725, 15.10093696483293, 15.013217396380396, 14.999051126977792, 15.061867057399267, 15.208206678336722, 15.106546650658698, 15.247326745973064, 15.335904965481358, 15.70665305544269, 15.826829192451356, 15.906347784230784, 16.05163560899867, 16.115882595908317, 16.30228281772097, 16.386586961489275, 17.088548520781764, 17.403854954464066, 17.567226013350375, 17.945945989630914, 18.073532645176513, 18.197936153215704, 18.08392500511813, 18.162146157091765, 18.263800169158376, 18.335140473055684, 18.62473478660325, 18.900002385480587, 19.151774139206992, 19.316484618208758, 19.527623747090136, 19.74131394303744, 20.019966880795902, 20.254060906134754, 21.00124992466107, 21.4211653831404, 21.746696512886697, 22.14533283444006, 22.443538652213068, 22.93492795329148, 22.933926714005644, 22.98860887255659, 22.81576656289203, 22.78144959775442, 22.757040012236352, 22.66572185899095, 22.58208599078623, 22.8931182221524, 22.809442843741625, 22.748946590860484, 22.722979790391957, 22.725025346757278, 22.687527140015966, 22.704921712690805, 22.620140825274248, 22.518902195784747, 22.52377867728087, 22.471002187030233, 22.478417783615246, 22.44337131878356, 22.394211209817545, 22.349785802043197, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.77287421451739, "threshold_area100": 16.673529253214987, "threshold_area25": 10.746267919470826}, "selected_budget": 77, "selected": {"val": {"score": 21.88429108891366, "calls": 76.8901569186876, "rescued": 783, "harmed": 347}, "test": {"score": 23.050282754852525, "calls": 78.26231691078563, "rescued": 3300, "harmed": 1457}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.215555191040039, "operator": "ge"}, {"budget": 0.02, "threshold": 1.1718699932098389, "operator": "ge"}, {"budget": 0.03, "threshold": 1.141616940498352, "operator": "ge"}, {"budget": 0.04, "threshold": 1.131468415260315, "operator": "ge"}, {"budget": 0.05, "threshold": 1.1047797203063965, "operator": "ge"}, {"budget": 0.06, "threshold": 1.091101884841919, "operator": "ge"}, {"budget": 0.07, "threshold": 1.0583710670471191, "operator": "ge"}, {"budget": 0.08, "threshold": 1.04119873046875, "operator": "ge"}, {"budget": 0.09, "threshold": 1.0171430110931396, "operator": "ge"}, {"budget": 0.1, "threshold": 0.988044261932373, "operator": "ge"}, {"budget": 0.11, "threshold": 0.9660726189613342, "operator": "ge"}, {"budget": 0.12, "threshold": 0.9388444423675537, "operator": "ge"}, {"budget": 0.13, "threshold": 0.8895111680030823, "operator": "ge"}, {"budget": 0.14, "threshold": 0.8589663505554199, "operator": "ge"}, {"budget": 0.15, "threshold": 0.8287370800971985, "operator": "ge"}, {"budget": 0.16, "threshold": 0.7961788177490234, "operator": "ge"}, {"budget": 0.17, "threshold": 0.7733252644538879, "operator": "ge"}, {"budget": 0.18, "threshold": 0.7636764049530029, "operator": "ge"}, {"budget": 0.19, "threshold": 0.7374541759490967, "operator": "ge"}, {"budget": 0.2, "threshold": 0.7083833813667297, "operator": "ge"}, {"budget": 0.21, "threshold": 0.6963494420051575, "operator": "ge"}, {"budget": 0.22, "threshold": 0.6888731718063354, "operator": "ge"}, {"budget": 0.23, "threshold": 0.6639180183410645, "operator": "ge"}, {"budget": 0.24, "threshold": 0.6544275283813477, "operator": "ge"}, {"budget": 0.25, "threshold": 0.6449592113494873, "operator": "ge"}, {"budget": 0.26, "threshold": 0.6255093216896057, "operator": "ge"}, {"budget": 0.27, "threshold": 0.6157380938529968, "operator": "ge"}, {"budget": 0.28, "threshold": 0.6050132513046265, "operator": "ge"}, {"budget": 0.29, "threshold": 0.5940439701080322, "operator": "ge"}, {"budget": 0.3, "threshold": 0.5886443853378296, "operator": "ge"}, {"budget": 0.31, "threshold": 0.57962566614151, "operator": "ge"}, {"budget": 0.32, "threshold": 0.5753405690193176, "operator": "ge"}, {"budget": 0.33, "threshold": 0.5650397539138794, "operator": "ge"}, {"budget": 0.34, "threshold": 0.5576829314231873, "operator": "ge"}, {"budget": 0.35, "threshold": 0.5448269844055176, "operator": "ge"}, {"budget": 0.36, "threshold": 0.5392045974731445, "operator": "ge"}, {"budget": 0.37, "threshold": 0.5319589376449585, "operator": "ge"}, {"budget": 0.38, "threshold": 0.5236600637435913, "operator": "ge"}, {"budget": 0.39, "threshold": 0.51051265001297, "operator": "ge"}, {"budget": 0.4, "threshold": 0.4982953667640686, "operator": "ge"}, {"budget": 0.41, "threshold": 0.4832002520561218, "operator": "ge"}, {"budget": 0.42, "threshold": 0.4737164080142975, "operator": "ge"}, {"budget": 0.43, "threshold": 0.45959436893463135, "operator": "ge"}, {"budget": 0.44, "threshold": 0.448243111371994, "operator": "ge"}, {"budget": 0.45, "threshold": 0.43301334977149963, "operator": "ge"}, {"budget": 0.46, "threshold": 0.42579275369644165, "operator": "ge"}, {"budget": 0.47, "threshold": 0.4102064371109009, "operator": "ge"}, {"budget": 0.48, "threshold": 0.39467376470565796, "operator": "ge"}, {"budget": 0.49, "threshold": 0.3802533745765686, "operator": "ge"}, {"budget": 0.5, "threshold": 0.36007124185562134, "operator": "ge"}, {"budget": 0.51, "threshold": 0.31660598516464233, "operator": "ge"}, {"budget": 0.52, "threshold": 0.3083880543708801, "operator": "ge"}, {"budget": 0.53, "threshold": 0.2853519916534424, "operator": "ge"}, {"budget": 0.54, "threshold": 0.25953489542007446, "operator": "ge"}, {"budget": 0.55, "threshold": 0.24535194039344788, "operator": "ge"}, {"budget": 0.56, "threshold": 0.2290966510772705, "operator": "ge"}, {"budget": 0.57, "threshold": 0.22073185443878174, "operator": "ge"}, {"budget": 0.58, "threshold": 0.2080896496772766, "operator": "ge"}, {"budget": 0.59, "threshold": 0.1947249174118042, "operator": "ge"}, {"budget": 0.6, "threshold": 0.18464145064353943, "operator": "ge"}, {"budget": 0.61, "threshold": 0.17003858089447021, "operator": "ge"}, {"budget": 0.62, "threshold": 0.15865981578826904, "operator": "ge"}, {"budget": 0.63, "threshold": 0.14736801385879517, "operator": "ge"}, {"budget": 0.64, "threshold": 0.13777205348014832, "operator": "ge"}, {"budget": 0.65, "threshold": 0.12819582223892212, "operator": "ge"}, {"budget": 0.66, "threshold": 0.11221301555633545, "operator": "ge"}, {"budget": 0.67, "threshold": 0.10186952352523804, "operator": "ge"}, {"budget": 0.68, "threshold": 0.08483052253723145, "operator": "ge"}, {"budget": 0.69, "threshold": 0.06583690643310547, "operator": "ge"}, {"budget": 0.7, "threshold": 0.03883609175682068, "operator": "ge"}, {"budget": 0.71, "threshold": 0.02438807487487793, "operator": "ge"}, {"budget": 0.72, "threshold": 0.002401590347290039, "operator": "ge"}, {"budget": 0.73, "threshold": -0.011462748050689697, "operator": "ge"}, {"budget": 0.74, "threshold": -0.024753808975219727, "operator": "ge"}, {"budget": 0.75, "threshold": -0.0611497163772583, "operator": "ge"}, {"budget": 0.76, "threshold": -0.1451825499534607, "operator": "ge"}, {"budget": 0.77, "threshold": -0.30289018154144287, "operator": "ge"}, {"budget": 0.78, "threshold": -2.64520263671875, "operator": "ge"}, {"budget": 0.79, "threshold": -2.8595190048217773, "operator": "ge"}, {"budget": 0.8, "threshold": -3.040472984313965, "operator": "ge"}, {"budget": 0.81, "threshold": -3.1186418533325195, "operator": "ge"}, {"budget": 0.82, "threshold": -3.146883487701416, "operator": "ge"}, {"budget": 0.83, "threshold": -3.2182931900024414, "operator": "ge"}, {"budget": 0.84, "threshold": -3.2485036849975586, "operator": "ge"}, {"budget": 0.85, "threshold": -3.3258767127990723, "operator": "ge"}, {"budget": 0.86, "threshold": -3.376687526702881, "operator": "ge"}, {"budget": 0.87, "threshold": -3.3886802196502686, "operator": "ge"}, {"budget": 0.88, "threshold": -3.4054453372955322, "operator": "ge"}, {"budget": 0.89, "threshold": -3.4279870986938477, "operator": "ge"}, {"budget": 0.9, "threshold": -3.463850259780884, "operator": "ge"}, {"budget": 0.91, "threshold": -3.503444194793701, "operator": "ge"}, {"budget": 0.92, "threshold": -3.5420098304748535, "operator": "ge"}, {"budget": 0.93, "threshold": -3.58150053024292, "operator": "ge"}, {"budget": 0.94, "threshold": -3.617079734802246, "operator": "ge"}, {"budget": 0.95, "threshold": -3.6404736042022705, "operator": "ge"}, {"budget": 0.96, "threshold": -3.6956095695495605, "operator": "ge"}, {"budget": 0.97, "threshold": -3.7321372032165527, "operator": "ge"}, {"budget": 0.98, "threshold": -3.778514862060547, "operator": "ge"}, {"budget": 0.99, "threshold": -3.832164764404297, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.731515598133138, 7.614238378511091, 7.723851671220092, 7.837117559160654, 7.947960547829265, 9.357295311221332, 9.347644920726179, 9.385328491485707, 9.418843991192892, 9.488725081414163, 9.459315747109665, 9.529434918203819, 9.531802816228739, 9.650524126454743, 9.756085685313357, 9.842514136543398, 9.954571572218631, 9.98724128383532, 10.102182130881683, 10.232119082790591, 10.25850252388446, 10.992322308462981, 11.072262899694605, 11.76270226871083, 12.43437340345216, 12.396730840168585, 12.457326778412334, 13.07364629645164, 13.602233129229372, 13.697914404168365, 13.825459802689405, 13.7691027963233, 13.743787813310284, 13.778952772325114, 13.853799584349868, 13.944600426455263, 13.97452356498916, 14.120078737483304, 14.618071683119318, 14.811471274885909, 14.975681089776463, 15.825607396220157, 16.027805069470084, 16.034795186975597, 16.18094956573461, 16.46253288681544, 16.711467378575186, 16.883153556381266, 16.90760502968925, 17.418528386964542, 17.623719297993766, 17.83177003195645, 17.841887302426, 17.99537811248837, 18.071186655689246, 18.36790254694941, 18.629109901218726, 18.51815028069618, 18.587934978769947, 18.78829730975857, 18.96994031153399, 19.276229320034076, 19.276008384614098, 19.531181093681095, 19.66976505323478, 19.99090125466909, 20.1228946337642, 20.008433725145913, 20.32643109198834, 20.62303985078694, 20.677786764157936, 20.636904574433075, 20.650174096027186, 21.157470817217344, 21.43311735852734, 21.762801505973826, 21.88429108891366, 21.81489096867713, 21.686351930840342, 21.569242249331023, 21.17333235613326, 21.12915139089966, 21.09948164456177, 21.059771524169044, 21.02319207109198, 20.981618353279636, 21.125651960125296, 21.089669921910772, 21.041103814611997, 21.014722210332483, 20.97347594586392, 20.950920685475968, 21.00420853694909, 20.903892113349265, 20.869314976771488, 20.847781626276202, 20.82669437326004, 20.72462780014751, 20.69708854241385, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 15, 31, 49, 67, 85, 105, 123, 139, 159, 173, 188, 201, 213, 225, 238, 249, 262, 276, 290, 301, 314, 327, 337, 349, 362, 372, 382, 394, 408, 417, 428, 438, 447, 455, 467, 479, 491, 501, 513, 523, 535, 547, 557, 567, 575, 586, 597, 603, 610, 619, 627, 634, 640, 647, 653, 661, 669, 674, 680, 686, 693, 699, 706, 713, 719, 725, 731, 735, 744, 750, 754, 758, 763, 769, 774, 779, 783, 784, 784, 784, 784, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786, 786, 786, 786, 786], "harmed": [0, 5, 12, 18, 23, 27, 31, 35, 40, 45, 48, 52, 59, 62, 64, 67, 68, 71, 75, 80, 82, 87, 92, 94, 97, 99, 103, 109, 110, 114, 116, 120, 125, 130, 133, 139, 144, 148, 151, 154, 157, 163, 167, 169, 174, 179, 182, 184, 187, 192, 195, 200, 205, 209, 213, 219, 225, 231, 238, 243, 247, 251, 256, 262, 268, 273, 279, 285, 290, 296, 299, 306, 312, 319, 324, 329, 337, 347, 356, 363, 373, 377, 382, 386, 391, 396, 402, 407, 412, 419, 423, 429, 432, 439, 444, 450, 454, 458, 464, 469, 475], "exact_score": [7.641657905799261, 7.731515598133138, 7.614238378511091, 7.723851671220092, 7.837117559160654, 7.947960547829265, 9.357295311221332, 9.347644920726179, 9.385328491485707, 9.418843991192892, 9.488725081414163, 9.459315747109665, 9.529434918203819, 9.531802816228739, 9.650524126454743, 9.756085685313357, 9.842514136543398, 9.954571572218631, 9.98724128383532, 10.102182130881683, 10.232119082790591, 10.25850252388446, 10.992322308462981, 11.072262899694605, 11.76270226871083, 12.43437340345216, 12.396730840168585, 12.457326778412334, 13.07364629645164, 13.602233129229372, 13.697914404168365, 13.825459802689405, 13.7691027963233, 13.743787813310284, 13.778952772325114, 13.853799584349868, 13.944600426455263, 13.97452356498916, 14.120078737483304, 14.618071683119318, 14.811471274885909, 14.975681089776463, 15.825607396220157, 16.027805069470084, 16.034795186975597, 16.18094956573461, 16.46253288681544, 16.711467378575186, 16.883153556381266, 16.90760502968925, 17.418528386964542, 17.623719297993766, 17.83177003195645, 17.841887302426, 17.99537811248837, 18.071186655689246, 18.36790254694941, 18.629109901218726, 18.51815028069618, 18.587934978769947, 18.78829730975857, 18.96994031153399, 19.276229320034076, 19.276008384614098, 19.531181093681095, 19.66976505323478, 19.99090125466909, 20.1228946337642, 20.008433725145913, 20.32643109198834, 20.62303985078694, 20.677786764157936, 20.636904574433075, 20.650174096027186, 21.157470817217344, 21.43311735852734, 21.762801505973826, 21.88429108891366, 21.81489096867713, 21.686351930840342, 21.569242249331023, 21.17333235613326, 21.12915139089966, 21.09948164456177, 21.059771524169044, 21.02319207109198, 20.981618353279636, 21.125651960125296, 21.089669921910772, 21.041103814611997, 21.014722210332483, 20.97347594586392, 20.950920685475968, 21.00420853694909, 20.903892113349265, 20.869314976771488, 20.847781626276202, 20.82669437326004, 20.72462780014751, 20.69708854241385, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.16172720985646, "threshold_area100": 16.155270316294793, "threshold_area25": 9.492581210394619}, "test": {"score": [7.8873978710439285, 8.010278459056286, 8.0911302995417, 8.150834152993781, 8.667169859801954, 8.7071743832607, 9.157502106332233, 9.935680121092314, 10.256204787837984, 10.185288497985255, 10.440476611355091, 11.750035690405056, 11.861506131452765, 11.930948925974553, 12.11754731327417, 12.141493849503476, 12.293429396319333, 12.299102317208833, 12.303783176509839, 12.318840532612164, 12.27913866668296, 12.27505229701541, 12.26639644614968, 12.445844340554341, 12.437894555270837, 12.69671699224489, 12.760531530054248, 12.9109014866159, 13.198860034958065, 13.343057729089841, 13.32267941755908, 13.527012020398121, 13.680107876800085, 13.939210128752922, 14.046897337733402, 14.31778076694806, 14.327002764746592, 14.451100611763735, 14.616292696711161, 15.040082397722147, 15.192944986614707, 15.595474760297378, 15.657800528207918, 15.805889410204333, 15.70728692732013, 15.681919872957542, 15.840593319252658, 16.057458436711396, 16.0636860885768, 16.23362643435956, 16.42942602284784, 17.028150175623917, 17.125663689166352, 17.356300966762362, 17.94196892008255, 18.1108864141497, 18.374294726841907, 18.540436422082813, 18.58001510436655, 18.838859076409495, 18.754215216088372, 18.86909182675056, 18.941384123849588, 19.112609818212047, 19.2755622872226, 19.471131348992245, 19.50197502449172, 19.548036647217515, 19.93737359742609, 20.215150017397086, 21.32678516345971, 21.501513066877017, 21.881463944933007, 22.1212038454242, 22.341937491249627, 22.426327635689358, 22.66850617764843, 23.03209411593111, 22.935294906336086, 22.81354776558258, 22.758054711933255, 22.755393108415536, 23.062752303465473, 22.933299572144268, 22.87245295136966, 22.690256616508773, 22.703290563354315, 22.745758791580005, 22.73236583436665, 22.742344811770515, 22.715828538711193, 22.68335196984613, 22.691959140916563, 22.65444798883628, 22.578600249768392, 22.54040427190573, 22.490083043709262, 22.463853449781404, 22.430042100366137, 22.39039767332584, 22.361115727328066], "calls": [0.0, 1.0652463382157125, 1.7976031957390146, 2.330226364846871, 2.596537949400799, 3.129161118508655, 3.4953395472703064, 4.394141145139813, 4.8934753661784285, 5.459387483355526, 6.225033288948069, 7.290279627163781, 8.388814913448735, 9.953395472703063, 11.018641810918774, 11.684420772303596, 13.21571238348868, 14.214380825565911, 14.580559254327563, 15.81225033288948, 17.0439414114514, 17.643142476697736, 18.042609853528628, 19.607190412782955, 20.572569906790946, 21.40479360852197, 23.26897470039947, 24.201065246338217, 25.466045272969374, 26.864181091877498, 27.396804260985352, 28.328894806924097, 28.828229027962713, 30.525965379494007, 31.724367509986685, 33.655126498002666, 34.45406125166445, 35.519307589880164, 36.38482023968043, 37.81624500665779, 39.34753661784288, 40.812250332889484, 41.711051930758984, 42.676431424766974, 43.37549933422104, 44.806924101198405, 45.30625832223702, 46.37150466045273, 47.070572569906794, 47.969374167776294, 49.03462050599201, 51.364846870838875, 51.76431424766977, 53.02929427430093, 54.92676431424767, 55.692410119840204, 56.8242343541944, 57.456724367509985, 58.089214380825574, 59.35419440745673, 60.28628495339547, 61.58455392809587, 62.45006657789614, 63.51531291611185, 64.31424766977364, 65.27962716378163, 66.51131824234355, 67.17709720372837, 68.70838881491345, 70.10652463382158, 72.17043941411451, 73.06924101198402, 74.16777629826898, 74.83355525965379, 75.43275632490013, 76.26498002663115, 77.49667110519307, 78.22902796271637, 79.09454061251664, 80.12649800266311, 81.39147802929428, 82.02396804260985, 82.55659121171772, 83.78828229027962, 84.0878828229028, 85.78561917443409, 86.98402130492676, 87.38348868175765, 87.7496671105193, 88.3155792276964, 89.08122503328894, 90.14647137150466, 91.17842876165113, 92.27696404793608, 93.07589880159787, 93.70838881491345, 95.33954727030626, 96.23834886817576, 97.37017310252996, 98.03595206391478, 100.0], "rescued": [0, 84, 140, 181, 201, 243, 271, 342, 378, 416, 466, 543, 618, 713, 770, 804, 890, 943, 965, 1026, 1087, 1118, 1141, 1217, 1264, 1307, 1391, 1435, 1493, 1553, 1576, 1622, 1643, 1723, 1775, 1870, 1903, 1952, 1990, 2052, 2121, 2189, 2228, 2274, 2301, 2354, 2376, 2420, 2447, 2477, 2515, 2590, 2603, 2641, 2696, 2719, 2751, 2768, 2784, 2816, 2841, 2874, 2898, 2927, 2948, 2974, 3003, 3020, 3059, 3094, 3147, 3170, 3199, 3217, 3231, 3249, 3280, 3299, 3304, 3304, 3305, 3306, 3307, 3307, 3307, 3307, 3308, 3309, 3309, 3310, 3310, 3310, 3311, 3312, 3312, 3312, 3313, 3313, 3313, 3313, 3314], "harmed": [0, 18, 33, 42, 46, 52, 58, 72, 81, 93, 110, 125, 147, 171, 187, 194, 220, 234, 241, 263, 285, 298, 307, 335, 352, 369, 396, 415, 433, 449, 460, 478, 482, 517, 538, 572, 588, 601, 613, 634, 667, 682, 699, 720, 729, 753, 761, 775, 789, 803, 817, 862, 872, 895, 934, 952, 974, 983, 996, 1019, 1038, 1072, 1089, 1113, 1128, 1149, 1181, 1196, 1226, 1263, 1297, 1320, 1346, 1363, 1378, 1398, 1437, 1457, 1476, 1500, 1525, 1544, 1558, 1584, 1590, 1631, 1658, 1672, 1680, 1694, 1709, 1728, 1753, 1778, 1797, 1811, 1852, 1870, 1893, 1908, 1945], "exact_score": [7.8873978710439285, 8.00193007466512, 8.112606109771002, 8.695490313526813, 9.163185997681603, 10.24025411032109, 10.460583720516764, 11.511673702437491, 11.903605655983172, 12.071342487484877, 11.932558343365457, 12.114310946469335, 12.126663759022316, 12.289365055580864, 12.27732028484565, 12.305501262724304, 12.299782504636186, 12.283581950654945, 12.266704130697336, 12.427079281820976, 12.43888910745027, 12.538177438322357, 12.721415208697506, 12.745802622513708, 12.893553596318291, 13.165874194269486, 13.264353135159606, 13.351005031138758, 13.541817169098374, 13.695900602010935, 13.84871790907731, 13.92231706097578, 14.073590164914776, 14.156875599853846, 14.313050288206375, 14.393611563464937, 14.606406343211125, 14.735850946114354, 15.232082215320444, 15.168606988964179, 15.394627701993446, 15.629095560306174, 15.680461497650667, 15.85439972166266, 15.594311081553577, 15.786403137461914, 16.005710745757636, 16.0423585926779, 16.23362643435956, 16.40958747627431, 16.599671759137873, 16.953963839618364, 17.208800538925313, 17.348617444408056, 17.573924148919296, 17.95000853494412, 18.250763954741647, 18.427513492656697, 18.54754544573431, 18.813584257314684, 18.617146600826338, 18.764658340021438, 18.969577476058575, 19.065754244422628, 19.23619214660955, 19.45120941536256, 19.479460504057847, 19.55886460009387, 19.73098396612669, 19.94256714019744, 20.183834027373877, 20.741314690442156, 21.221405466984013, 21.47305611926974, 21.80596273478472, 22.14533283444006, 22.416175547041, 22.905625409807833, 22.933926714005644, 22.99086923894459, 22.81576656289203, 22.729424767001234, 22.7575864649822, 23.028506082128196, 22.924826571595876, 22.78808865095362, 22.682146754917078, 22.703290563354315, 22.723370774751384, 22.719661131076666, 22.68712079302411, 22.70038908200252, 22.66591480733764, 22.58313599984982, 22.534187135625427, 22.50536518759521, 22.470793175737935, 22.44087395884977, 22.39284144334148, 22.348852146461955, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 17.037472311791653, "threshold_area100": 16.900980675051652, "threshold_area25": 10.772068188489328}, "selected_budget": 77, "selected": {"val": {"score": 21.88429108891366, "calls": 76.8901569186876, "rescued": 783, "harmed": 347}, "test": {"score": 23.03209411593111, "calls": 78.22902796271637, "rescued": 3299, "harmed": 1457}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405}, "test": {"score": 23.706753735104392, "calls": 79.42743009320905}}, "old_normalized_interior": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405}, "test": {"score": 23.706753735104392, "calls": 79.42743009320905}}, "old_normalized_endpoints": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405}, "test": {"score": 23.706753735104392, "calls": 79.42743009320905}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 10.517864445954938, 9.925362151597527, 9.819697384814415, 9.613584901909707, 9.546594081870545, 9.450294127574493, 9.209605417186596, 9.253784895207037, 9.316811657030124, 9.409442731005052, 9.387093962141385, 10.283922909011668, 10.259320573535538, 10.324491040435312, 10.394020241450798, 10.44419238050903, 10.516669065549527, 11.17345308635828, 11.157247754209571, 11.725663183521954, 12.163741484821756, 12.295328411770917, 12.315329534588908, 12.393414837936007, 12.434801073366746, 12.457246666286215, 12.514878557570134, 12.602727220327944, 12.694250194250195, 12.790657839333852, 12.906539512563594, 12.925226016870223, 13.299077541015889, 13.398899182440239, 13.514708889850985, 13.639687843285971, 14.462112799325414, 14.6289601301822, 14.615305080487987, 14.762415783893, 14.985710410520891, 15.207532165294701, 15.470980188163793, 15.528411947669518, 16.1345690900337, 16.65416266614099, 16.76603125480136, 16.884540091900746, 17.08066581306018, 17.05224023819838, 17.2270990966165, 17.413634094594478, 17.404104735283102, 17.548591724784078, 17.70511339014878, 17.869593167325412, 17.902843787601487, 18.094500859128324, 17.925875835352105, 17.925096014538784, 18.159664655675297, 18.1113769271664, 17.964862281032666, 18.009820278655894, 18.268137393174342, 18.583333333333332, 18.828905177863152, 18.837198340874814, 19.253342056688645, 19.523632429529002, 19.618438216566283, 19.834143912332717, 20.365351226933146, 20.563706012999983, 20.720769420930075, 21.147486532981635, 21.548648807684952, 21.936136832098168, 21.70757217387469, 21.694359952569986, 21.66218860945647, 21.618278088382826, 21.590448774839224, 21.55425560947735, 21.71973571075935, 21.614652484870714, 21.563381450141033, 21.53019050878987, 21.316947317706628, 21.283265394095515, 21.233551752641024, 21.12150919299976, 21.018759971934315, 20.97229478209827, 20.863661713575162, 20.827196459974722, 20.95732588962293, 20.9091859125147, 20.784199047745844, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 10.517864445954938, 9.925362151597527, 9.819697384814415, 9.613584901909707, 9.546594081870545, 9.450294127574493, 9.209605417186596, 9.253784895207037, 9.316811657030124, 9.409442731005052, 9.387093962141385, 10.283922909011668, 10.259320573535538, 10.324491040435312, 10.394020241450798, 10.44419238050903, 10.516669065549527, 11.17345308635828, 11.157247754209571, 11.725663183521954, 12.163741484821756, 12.295328411770917, 12.315329534588908, 12.393414837936007, 12.434801073366746, 12.457246666286215, 12.514878557570134, 12.602727220327944, 12.694250194250195, 12.790657839333852, 12.906539512563594, 12.925226016870223, 13.299077541015889, 13.398899182440239, 13.514708889850985, 13.639687843285971, 14.462112799325414, 14.6289601301822, 14.615305080487987, 14.762415783893, 14.985710410520891, 15.207532165294701, 15.470980188163793, 15.528411947669518, 16.1345690900337, 16.65416266614099, 16.76603125480136, 16.884540091900746, 17.08066581306018, 17.05224023819838, 17.2270990966165, 17.413634094594478, 17.404104735283102, 17.548591724784078, 17.70511339014878, 17.869593167325412, 17.902843787601487, 18.094500859128324, 17.925875835352105, 17.925096014538784, 18.159664655675297, 18.1113769271664, 17.964862281032666, 18.009820278655894, 18.268137393174342, 18.583333333333332, 18.828905177863152, 18.837198340874814, 19.253342056688645, 19.523632429529002, 19.618438216566283, 19.834143912332717, 20.365351226933146, 20.563706012999983, 20.720769420930075, 21.147486532981635, 21.548648807684952, 21.936136832098168, 21.70757217387469, 21.694359952569986, 21.66218860945647, 21.618278088382826, 21.590448774839224, 21.55425560947735, 21.71973571075935, 21.614652484870714, 21.563381450141033, 21.53019050878987, 21.316947317706628, 21.283265394095515, 21.233551752641024, 21.12150919299976, 21.018759971934315, 20.97229478209827, 20.863661713575162, 20.827196459974722, 20.95732588962293, 20.9091859125147, 20.784199047745844, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 10.22278994548627, "exact_area100": 16.169482767797742}, "test": {"score": [7.8873978710439285, 7.931012081218859, 7.988331681775998, 8.101641257580239, 8.637722311488217, 9.729898626059388, 11.440085908400723, 11.392532977168056, 11.527408432416797, 11.43917957635547, 11.683773875666606, 11.588914376246, 11.549313421890876, 11.444513160180273, 11.716499955080742, 11.86941473166699, 12.023487671034818, 11.954632265674798, 11.962322727049113, 12.327601590587896, 12.292117909491319, 12.278794547759466, 12.371363555696357, 12.374224260615987, 12.385157773821238, 12.441327088998388, 12.52860953291219, 12.68654399633243, 12.77204405230092, 13.022769235880627, 13.162136752824887, 13.540568848659564, 13.66469411134415, 13.732559228022406, 13.963420984219056, 14.170586928004061, 14.278781784848391, 14.349882103571721, 14.523131150766933, 14.724198410366865, 15.01464391603543, 15.254892476443318, 15.314860330255339, 15.370803601972844, 15.469873894048902, 15.689713968997875, 16.051249987909987, 16.0735992637263, 16.192207957122704, 16.355985946120473, 16.53920698786677, 16.69081336237682, 16.767140485419944, 16.855540518813612, 17.138969320053874, 17.19368879776039, 17.656869160062318, 18.119280231814606, 18.29794489316271, 18.306198046163775, 18.466865907774253, 18.562346273706428, 18.740681693631313, 18.785315777093576, 18.88563694294579, 18.80691130665781, 19.19761776594301, 19.577591414057128, 19.470344313377005, 19.828194929699617, 20.09768327218453, 20.489736704371385, 20.85039400712384, 21.33749295354154, 21.671428100776108, 22.20166692958944, 22.741247934443166, 23.336907577790488, 23.706753735104392, 23.298551390816385, 23.227145381191036, 23.097584868056952, 23.07633912460817, 23.037710843122696, 22.76255857245299, 22.752393290085934, 22.775945003771838, 22.764501126052636, 22.764448876003097, 22.66256329999125, 22.55755448782451, 22.625194957790367, 22.585700391873676, 22.56333677641631, 22.4940609932423, 22.414129586260735, 22.40196528617369, 22.385195960580127, 22.379373542201307, 22.35287825853484, 22.361115727328066], "calls": [0.0, 0.5992010652463382, 1.1318242343541944, 2.0639147802929427, 2.862849533954727, 4.1278295605858855, 5.3928095872170445, 5.792276964047936, 6.4247669773635145, 7.057256990679095, 7.989347536617843, 8.655126498002662, 9.387483355525966, 10.053262316910786, 11.651131824234353, 12.350199733688415, 12.849533954727029, 14.280958721704396, 15.07989347536618, 16.844207723035954, 17.976031957390145, 19.174434087882823, 20.406125166444742, 20.872170439414113, 21.438082556591212, 22.37017310252996, 23.06924101198402, 24.33422103861518, 25.332889480692412, 26.398135818908123, 27.330226364846872, 28.462050599201067, 30.193075898801595, 31.19174434087883, 32.92276964047936, 34.12117177097203, 34.92010652463382, 36.38482023968043, 37.45006657789614, 38.61517976031957, 40.645805592543276, 41.87749667110519, 42.34354194407457, 43.275632490013315, 44.207723035952064, 45.43941411451398, 47.70306258322237, 48.06924101198402, 50.266311584553925, 51.364846870838875, 52.06391478029294, 53.06258322237017, 53.56191744340879, 53.92809587217043, 54.627163781624496, 55.72569906790945, 56.35818908122503, 57.39014647137151, 58.055925432756325, 58.55525965379494, 59.52063914780293, 59.92010652463382, 61.01864181091877, 62.31691078561917, 62.982689747003995, 64.14780292942743, 65.71238348868175, 67.54327563249002, 68.60852197070572, 69.90679094540613, 70.73901464713715, 72.03728362183755, 73.30226364846871, 74.43408788282291, 75.16644474034621, 75.99866844207723, 77.09720372836219, 78.42876165113182, 79.42743009320905, 80.05992010652463, 80.9254327563249, 81.85752330226364, 82.35685752330227, 83.45539280958721, 85.21970705725698, 85.51930758988016, 86.35153129161118, 86.88415446071905, 88.1491344873502, 88.68175765645806, 90.31291611185085, 90.74567243675101, 91.84420772303595, 92.60985352862849, 94.00798934753661, 95.80559254327564, 97.20372836218375, 97.53661784287615, 98.36884154460719, 98.93475366178428, 100.0], "exact_score": [7.8873978710439285, 7.974844828206979, 8.093010458084697, 9.036116029580567, 9.756985736213577, 10.938237658470227, 11.617024624134805, 11.44447616305532, 11.683773875666606, 11.588765655302458, 11.45903724784516, 11.768215648547093, 11.695599555616045, 12.028576359917903, 11.947658150331442, 11.960912066554044, 12.279203787144134, 12.295887076547455, 12.292117909491319, 12.276624077616564, 12.216479157592937, 12.381235790173816, 12.388789617288742, 12.400797967516388, 12.663061295915687, 12.637013641470945, 13.021793093322763, 13.042010486334643, 13.326337254675991, 13.563937906157259, 13.656617734745161, 13.717996555110364, 13.863057730094372, 13.973051146044174, 14.170962853094474, 14.290780158107708, 14.30444113537876, 14.487312793320006, 14.630320814357395, 14.767707464989844, 14.836448947416162, 15.150606729718302, 15.27756592353089, 15.37373751720005, 15.445817958666156, 15.629014005264871, 15.883114894223983, 16.058914760987207, 16.053997771708307, 16.15240608813483, 16.188278416023685, 16.31290677242381, 16.518020300998952, 16.688345937446456, 16.871194923311332, 17.168216469521056, 17.24521369959619, 18.040608588928126, 18.28426588683795, 18.407768236777912, 18.568095184803617, 18.732974707578073, 18.83996392120647, 18.88563694294579, 18.751279645029012, 18.874190594078517, 19.232129118709043, 19.450342356037623, 19.3253376554431, 19.55719867468304, 19.85005123739697, 20.153631260045827, 20.479034891246346, 20.84752966808687, 21.09648196112142, 21.64023179651016, 22.20166692958944, 22.728196434687284, 23.24101942404284, 23.41320310178056, 23.281910597162835, 23.224918362312767, 23.091550736086838, 23.053179083129326, 22.902244709397003, 22.769755313526005, 22.74141605973617, 22.761626554212548, 22.76992830731607, 22.607448285181878, 22.589019943499757, 22.61834718112649, 22.582398773417882, 22.533017626882902, 22.495310790150306, 22.446057013734368, 22.41109035976242, 22.425287121917112, 22.39980797561553, 22.35287825853484, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 11.147913524581094, "exact_area100": 16.844491196498627}}, "old_normalized_endpoint_budget": 78}, {"seed": 2029, "selection": {"best_epoch": 7, "epochs": 15, "alpha": 0.0, "validation_area100": 0.16518105006270187, "training_seconds": 4.839659303077497, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.562401533126831, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5476144552230835, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5225101709365845, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5162160992622375, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5081911087036133, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5007472038269043, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4886986017227173, "operator": "ge"}, {"budget": 0.08, "threshold": 0.47367915511131287, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4594859480857849, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4294031858444214, "operator": "ge"}, {"budget": 0.11, "threshold": 0.3874030113220215, "operator": "ge"}, {"budget": 0.12, "threshold": 0.36742639541625977, "operator": "ge"}, {"budget": 0.13, "threshold": 0.34740594029426575, "operator": "ge"}, {"budget": 0.14, "threshold": 0.33319902420043945, "operator": "ge"}, {"budget": 0.15, "threshold": 0.32046765089035034, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3076246976852417, "operator": "ge"}, {"budget": 0.17, "threshold": 0.2963269352912903, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2908954918384552, "operator": "ge"}, {"budget": 0.19, "threshold": 0.28546977043151855, "operator": "ge"}, {"budget": 0.2, "threshold": 0.28068965673446655, "operator": "ge"}, {"budget": 0.21, "threshold": 0.27192598581314087, "operator": "ge"}, {"budget": 0.22, "threshold": 0.26402825117111206, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2613295018672943, "operator": "ge"}, {"budget": 0.24, "threshold": 0.2582085132598877, "operator": "ge"}, {"budget": 0.25, "threshold": 0.25220006704330444, "operator": "ge"}, {"budget": 0.26, "threshold": 0.24887080490589142, "operator": "ge"}, {"budget": 0.27, "threshold": 0.24481351673603058, "operator": "ge"}, {"budget": 0.28, "threshold": 0.2414548397064209, "operator": "ge"}, {"budget": 0.29, "threshold": 0.2382417470216751, "operator": "ge"}, {"budget": 0.3, "threshold": 0.23540565371513367, "operator": "ge"}, {"budget": 0.31, "threshold": 0.23301905393600464, "operator": "ge"}, {"budget": 0.32, "threshold": 0.2300785630941391, "operator": "ge"}, {"budget": 0.33, "threshold": 0.22698502242565155, "operator": "ge"}, {"budget": 0.34, "threshold": 0.22217971086502075, "operator": "ge"}, {"budget": 0.35, "threshold": 0.21922563016414642, "operator": "ge"}, {"budget": 0.36, "threshold": 0.21631769835948944, "operator": "ge"}, {"budget": 0.37, "threshold": 0.21346096694469452, "operator": "ge"}, {"budget": 0.38, "threshold": 0.21189725399017334, "operator": "ge"}, {"budget": 0.39, "threshold": 0.20544126629829407, "operator": "ge"}, {"budget": 0.4, "threshold": 0.19613584876060486, "operator": "ge"}, {"budget": 0.41, "threshold": 0.18755994737148285, "operator": "ge"}, {"budget": 0.42, "threshold": 0.17437893152236938, "operator": "ge"}, {"budget": 0.43, "threshold": 0.1570960432291031, "operator": "ge"}, {"budget": 0.44, "threshold": 0.14416588842868805, "operator": "ge"}, {"budget": 0.45, "threshold": 0.13097108900547028, "operator": "ge"}, {"budget": 0.46, "threshold": 0.12036074697971344, "operator": "ge"}, {"budget": 0.47, "threshold": 0.11317489296197891, "operator": "ge"}, {"budget": 0.48, "threshold": 0.1046387106180191, "operator": "ge"}, {"budget": 0.49, "threshold": 0.09976215660572052, "operator": "ge"}, {"budget": 0.5, "threshold": 0.09640084207057953, "operator": "ge"}, {"budget": 0.51, "threshold": 0.09012529253959656, "operator": "ge"}, {"budget": 0.52, "threshold": 0.08595035970211029, "operator": "ge"}, {"budget": 0.53, "threshold": 0.08438055217266083, "operator": "ge"}, {"budget": 0.54, "threshold": 0.07876846194267273, "operator": "ge"}, {"budget": 0.55, "threshold": 0.07254940271377563, "operator": "ge"}, {"budget": 0.56, "threshold": 0.07015599310398102, "operator": "ge"}, {"budget": 0.57, "threshold": 0.06705030798912048, "operator": "ge"}, {"budget": 0.58, "threshold": 0.06544587016105652, "operator": "ge"}, {"budget": 0.59, "threshold": 0.06082373857498169, "operator": "ge"}, {"budget": 0.6, "threshold": 0.05757074058055878, "operator": "ge"}, {"budget": 0.61, "threshold": 0.05466780066490173, "operator": "ge"}, {"budget": 0.62, "threshold": 0.051687926054000854, "operator": "ge"}, {"budget": 0.63, "threshold": 0.04578307271003723, "operator": "ge"}, {"budget": 0.64, "threshold": 0.04174025356769562, "operator": "ge"}, {"budget": 0.65, "threshold": 0.038356050848960876, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03220337629318237, "operator": "ge"}, {"budget": 0.67, "threshold": 0.028309263288974762, "operator": "ge"}, {"budget": 0.68, "threshold": 0.023308873176574707, "operator": "ge"}, {"budget": 0.69, "threshold": 0.018583059310913086, "operator": "ge"}, {"budget": 0.7, "threshold": 0.012941941618919373, "operator": "ge"}, {"budget": 0.71, "threshold": 0.008696600794792175, "operator": "ge"}, {"budget": 0.72, "threshold": 0.0034252703189849854, "operator": "ge"}, {"budget": 0.73, "threshold": -0.00031051039695739746, "operator": "ge"}, {"budget": 0.74, "threshold": -0.005284950137138367, "operator": "ge"}, {"budget": 0.75, "threshold": -0.010452404618263245, "operator": "ge"}, {"budget": 0.76, "threshold": -0.02237015962600708, "operator": "ge"}, {"budget": 0.77, "threshold": -0.04619631543755531, "operator": "ge"}, {"budget": 0.78, "threshold": -0.07148108631372452, "operator": "ge"}, {"budget": 0.79, "threshold": -0.08277718722820282, "operator": "ge"}, {"budget": 0.8, "threshold": -0.09244003891944885, "operator": "ge"}, {"budget": 0.81, "threshold": -0.09918755292892456, "operator": "ge"}, {"budget": 0.82, "threshold": -0.10529748350381851, "operator": "ge"}, {"budget": 0.83, "threshold": -0.11200641095638275, "operator": "ge"}, {"budget": 0.84, "threshold": -0.1160203218460083, "operator": "ge"}, {"budget": 0.85, "threshold": -0.12322317808866501, "operator": "ge"}, {"budget": 0.86, "threshold": -0.12778690457344055, "operator": "ge"}, {"budget": 0.87, "threshold": -0.12906843423843384, "operator": "ge"}, {"budget": 0.88, "threshold": -0.13212484121322632, "operator": "ge"}, {"budget": 0.89, "threshold": -0.14142785966396332, "operator": "ge"}, {"budget": 0.9, "threshold": -0.14800646901130676, "operator": "ge"}, {"budget": 0.91, "threshold": -0.15205591917037964, "operator": "ge"}, {"budget": 0.92, "threshold": -0.1571233868598938, "operator": "ge"}, {"budget": 0.93, "threshold": -0.16460110247135162, "operator": "ge"}, {"budget": 0.94, "threshold": -0.1697249710559845, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1742798089981079, "operator": "ge"}, {"budget": 0.96, "threshold": -0.17611780762672424, "operator": "ge"}, {"budget": 0.97, "threshold": -0.18029256165027618, "operator": "ge"}, {"budget": 0.98, "threshold": -0.19128714501857758, "operator": "ge"}, {"budget": 0.99, "threshold": -0.1985744684934616, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.751224860383048, 7.8647554833266256, 7.9911652288462935, 9.577805334988993, 9.262409433216886, 9.29972123206288, 9.28835558612428, 9.311849723564494, 9.377778616605433, 9.278099205529992, 9.39363439730512, 9.479117740278683, 9.573867545244191, 9.687804714986205, 9.778586607904547, 9.872512800466046, 11.333587160929284, 11.446139361945868, 11.405916184369028, 12.127586948468522, 12.138878707168395, 12.187292277660415, 12.278893171055277, 12.223722012809205, 12.71682606274546, 12.76225880444769, 12.817712416329977, 12.949638359943702, 12.924574226387698, 13.479698816830876, 13.5728732180524, 13.715311014962584, 13.823834408389304, 14.204049295774649, 14.285401155119882, 14.40149150665076, 15.232279282813291, 15.344942114886543, 15.232511610527796, 15.27030854210941, 15.446756280089613, 15.967915815965203, 16.20206231239208, 16.37467010601339, 16.483942494525603, 16.45151844279751, 16.66703199192118, 16.873411282941067, 17.01904265312638, 17.236342417065305, 17.473634791403992, 17.69666390823557, 17.970942545948667, 18.154546730647496, 18.428005610209, 18.527961168001326, 18.539392501186523, 18.95590432347296, 18.98447097122594, 19.301065603530628, 19.776565083700277, 19.857531848682292, 20.086248087693885, 20.479566202564115, 20.79592509484593, 20.90470815961128, 20.835281761294127, 21.047021534722752, 21.29129678405359, 20.90524471512084, 20.92767547431795, 20.958811642884374, 21.29253535915357, 21.69301411375175, 21.695164782935986, 21.664808006327934, 21.88429108891366, 21.936136832098168, 21.886727169206157, 21.841661913929773, 21.816685004651358, 21.59357789952178, 21.3845179085125, 21.345765798372078, 21.305031236608095, 21.197984374216016, 21.175821253648643, 21.15137036752496, 21.312974247333784, 21.2673681777618, 21.235240789488056, 21.195356639527656, 21.084582447005342, 21.05343617189555, 20.9271618043598, 20.827196459974722, 20.716736018920333, 20.667983503163832, 20.628109214552293, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 20, 40, 60, 79, 95, 111, 131, 148, 164, 179, 196, 208, 222, 234, 247, 258, 273, 285, 297, 310, 321, 333, 346, 359, 369, 382, 394, 406, 416, 428, 441, 453, 464, 474, 481, 493, 504, 515, 525, 537, 548, 561, 572, 582, 588, 593, 600, 608, 613, 621, 628, 635, 642, 650, 657, 664, 670, 677, 683, 691, 698, 704, 709, 716, 722, 727, 733, 740, 745, 748, 755, 760, 765, 770, 775, 779, 783, 784, 784, 784, 784, 784, 784, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786], "harmed": [0, 5, 11, 17, 20, 27, 31, 37, 41, 46, 50, 52, 54, 58, 61, 66, 70, 73, 75, 80, 82, 86, 90, 94, 101, 104, 109, 113, 115, 119, 121, 125, 128, 133, 137, 143, 147, 150, 154, 158, 164, 167, 169, 172, 176, 178, 181, 183, 186, 189, 195, 197, 202, 209, 214, 218, 226, 229, 235, 243, 249, 254, 262, 268, 275, 276, 281, 286, 290, 295, 301, 309, 314, 318, 324, 331, 337, 347, 350, 356, 362, 365, 370, 374, 379, 384, 389, 393, 401, 405, 411, 416, 422, 427, 433, 442, 447, 454, 461, 467, 475], "exact_score": [7.641657905799261, 7.751224860383048, 7.8647554833266256, 7.9911652288462935, 9.577805334988993, 9.262409433216886, 9.29972123206288, 9.28835558612428, 9.311849723564494, 9.377778616605433, 9.278099205529992, 9.39363439730512, 9.479117740278683, 9.573867545244191, 9.687804714986205, 9.778586607904547, 9.872512800466046, 11.333587160929284, 11.446139361945868, 11.405916184369028, 12.127586948468522, 12.138878707168395, 12.187292277660415, 12.278893171055277, 12.223722012809205, 12.71682606274546, 12.76225880444769, 12.817712416329977, 12.949638359943702, 12.924574226387698, 13.479698816830876, 13.5728732180524, 13.715311014962584, 13.823834408389304, 14.204049295774649, 14.285401155119882, 14.40149150665076, 15.232279282813291, 15.344942114886543, 15.232511610527796, 15.27030854210941, 15.446756280089613, 15.967915815965203, 16.20206231239208, 16.37467010601339, 16.483942494525603, 16.45151844279751, 16.66703199192118, 16.873411282941067, 17.01904265312638, 17.236342417065305, 17.473634791403992, 17.69666390823557, 17.970942545948667, 18.154546730647496, 18.428005610209, 18.527961168001326, 18.539392501186523, 18.95590432347296, 18.98447097122594, 19.301065603530628, 19.776565083700277, 19.857531848682292, 20.086248087693885, 20.479566202564115, 20.79592509484593, 20.90470815961128, 20.835281761294127, 21.047021534722752, 21.29129678405359, 20.90524471512084, 20.92767547431795, 20.958811642884374, 21.29253535915357, 21.69301411375175, 21.695164782935986, 21.664808006327934, 21.88429108891366, 21.936136832098168, 21.886727169206157, 21.841661913929773, 21.816685004651358, 21.59357789952178, 21.3845179085125, 21.345765798372078, 21.305031236608095, 21.197984374216016, 21.175821253648643, 21.15137036752496, 21.312974247333784, 21.2673681777618, 21.235240789488056, 21.195356639527656, 21.084582447005342, 21.05343617189555, 20.9271618043598, 20.827196459974722, 20.716736018920333, 20.667983503163832, 20.628109214552293, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.518105006270186, "threshold_area100": 16.51220649479473, "threshold_area25": 10.125184835878462}, "test": {"score": [7.8873978710439285, 7.939639427944812, 8.616751542507087, 9.568062882112246, 9.52061840828356, 9.49386570248796, 9.820450057785092, 10.723886860304797, 11.466861996120391, 11.322245983649767, 11.436769048766424, 11.288066677676422, 11.271940067925636, 11.26576985259875, 11.27796672507794, 11.642518333359677, 11.576312484995857, 12.190035536203224, 12.200829610402756, 12.226483856382602, 12.09070572314501, 12.117893666404521, 12.270228051136693, 12.272347656646044, 12.25428539822696, 12.38072070840513, 12.615278282334474, 12.642171871696542, 12.814651985340438, 12.885723618579423, 12.8857153607486, 13.04298916112967, 13.093592256928464, 13.381337311299976, 13.587344795111944, 13.861039854964874, 13.96872294693306, 14.208903573097654, 14.243815505637059, 14.393893836578167, 14.798548491980076, 15.09247317022116, 15.474694616812473, 15.475271487377304, 15.57553467569651, 15.703708522014658, 15.928167113875805, 16.066024585234377, 16.30745999628492, 16.47185487020651, 16.54818599047496, 16.75175920318677, 17.042036778171518, 17.189022469033894, 17.39386898704253, 17.735539264462766, 18.209210431957924, 18.67226240721224, 18.690518517553027, 19.10838942136915, 19.28978785935233, 19.52706561704815, 19.63396964068353, 19.830384432903145, 19.990088910659964, 20.215732662626635, 20.56579098009815, 20.791372646400994, 20.93100639819463, 21.12619287624727, 21.597272492810887, 21.74751000140118, 21.964135416300678, 22.15495041060595, 22.307505955090846, 22.406420747226196, 22.333753514234683, 22.873898438477497, 23.02798282770131, 22.89358493523716, 22.8661642005517, 22.978238097921505, 22.871523168579643, 22.8073370271752, 22.791826412841086, 22.715227972788046, 22.684796533950372, 22.680667804851087, 22.668584748372023, 22.66344070922748, 22.52324816394023, 22.502289021075512, 22.46489421529157, 22.415516482674928, 22.36421172506622, 22.353384812240886, 22.326494622470392, 22.35072965742292, 22.349995174930694, 22.39453064308806, 22.361115727328066], "calls": [0.0, 0.4327563249001331, 0.8655126498002662, 2.097203728362184, 2.5299600532623168, 2.96271637816245, 3.362183754993342, 4.360852197070573, 5.25965379494008, 5.9254327563249, 6.824234354194407, 7.92276964047936, 8.355525965379494, 8.988015978695072, 9.753661784287617, 10.5525965379494, 12.15046604527297, 13.581890812250332, 14.414114513981357, 15.07989347536618, 15.679094540612518, 17.41011984021305, 19.10785619174434, 19.740346205059918, 20.406125166444742, 21.904127829560586, 22.836218375499335, 24.06790945406125, 25.0, 26.098535286284953, 27.230359520639148, 28.19573901464714, 28.99467376830892, 30.059920106524636, 31.125166444740348, 32.323568575233026, 33.15579227696405, 34.05459387483356, 34.58721704394141, 36.78428761651132, 38.74833555259654, 40.37949400798935, 42.24367509986685, 43.47536617842876, 44.374167776298265, 45.40612516644474, 46.23834886817576, 47.13715046604528, 48.50199733688416, 49.30093209054594, 49.93342210386152, 51.231691078561916, 52.26364846870839, 52.563249001331556, 53.82822902796271, 55.625832223701735, 56.42476697736352, 57.290279627163784, 57.822902796271634, 59.38748335552596, 60.21970705725699, 61.55126498002663, 62.150466045272964, 63.715046604527295, 64.64713715046605, 65.5792276964048, 67.27696404793608, 68.50865512649801, 69.47403462050599, 70.50599201065246, 71.73768308921437, 72.47003994673769, 73.43541944074568, 74.13448735019973, 75.03328894806924, 75.33288948069242, 76.49800266311586, 77.89613848202397, 79.12782956058588, 80.32623169107856, 81.19174434087883, 82.42343541944075, 83.38881491344874, 84.72037283621837, 85.18641810918774, 86.25166444740347, 86.95073235685751, 87.05059920106525, 87.51664447403462, 88.74833555259653, 90.31291611185085, 90.87882822902796, 91.81091877496671, 93.70838881491345, 94.70705725699068, 95.63914780292943, 95.97203728362183, 96.80426098535287, 98.60186418109188, 99.23435419440746, 100.0], "rescued": [0, 38, 76, 175, 212, 246, 277, 359, 433, 480, 546, 621, 648, 679, 724, 770, 863, 940, 984, 1022, 1050, 1142, 1219, 1248, 1279, 1352, 1399, 1459, 1503, 1554, 1607, 1651, 1690, 1740, 1791, 1853, 1891, 1933, 1958, 2056, 2155, 2222, 2305, 2352, 2387, 2421, 2448, 2477, 2522, 2545, 2563, 2598, 2629, 2639, 2676, 2727, 2753, 2777, 2790, 2834, 2856, 2893, 2907, 2946, 2969, 2993, 3037, 3067, 3090, 3115, 3145, 3160, 3184, 3202, 3222, 3230, 3258, 3289, 3302, 3303, 3305, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3307, 3307, 3308, 3308, 3309, 3309, 3310, 3312, 3314, 3314], "harmed": [0, 9, 16, 38, 46, 53, 59, 78, 94, 107, 123, 140, 146, 156, 167, 183, 216, 236, 250, 261, 272, 302, 327, 340, 353, 381, 399, 423, 437, 460, 481, 493, 509, 530, 550, 569, 587, 597, 607, 644, 678, 704, 730, 744, 758, 773, 781, 796, 822, 838, 848, 864, 886, 892, 922, 969, 984, 1006, 1016, 1050, 1067, 1100, 1113, 1142, 1161, 1175, 1211, 1244, 1267, 1293, 1315, 1332, 1352, 1368, 1387, 1392, 1416, 1447, 1461, 1480, 1500, 1524, 1546, 1573, 1586, 1607, 1624, 1626, 1634, 1663, 1700, 1713, 1736, 1780, 1804, 1827, 1834, 1852, 1904, 1924, 1945], "exact_score": [7.8873978710439285, 9.198904871589873, 9.558365331042522, 9.487193883377222, 10.767902983717786, 11.507998324430412, 11.316392913580977, 11.345744506351716, 11.296881829093133, 11.26576985259875, 11.688659130600302, 11.57697904397365, 11.588967627809925, 12.193893061256555, 12.173284539306014, 12.223217220894174, 12.092835600899733, 12.128381631802847, 12.296745550559821, 12.29172633415358, 12.279479034840543, 12.258985369048686, 12.383430829495525, 12.610555625759076, 12.639183301544444, 12.814651985340438, 12.878974237748405, 12.863091372022625, 12.915217339635857, 13.093592256928464, 13.375387785679022, 13.601094396944182, 13.822622260142317, 13.873842548897366, 14.197120639833269, 14.222119050498923, 14.326537423226352, 14.506278114719162, 14.680936696534323, 14.789802306564702, 15.050549990508372, 15.169709566312223, 15.400357073476354, 15.57846160645793, 15.559667419857274, 15.659992584352986, 15.8789158662557, 16.03119532796636, 16.22180748018425, 16.340293906806956, 16.557438049813957, 16.492394511880914, 16.97426757689498, 17.274478991146538, 17.372021116470496, 17.5996583382293, 17.816094003000728, 18.332139421450243, 18.74482402842641, 18.947094238248116, 19.233108184260804, 19.38827362051418, 19.570253938905662, 19.705571887143943, 19.862708012033888, 20.020934451577382, 20.29435093798476, 20.5568606935498, 20.719800166130426, 20.92916417138577, 21.069716937667426, 21.371102799832247, 21.72077109881988, 21.816758291790624, 22.09888330541147, 22.292216206473224, 22.15444445583054, 22.532542899139138, 22.92071156308659, 23.011454552111445, 23.003308185020124, 22.88428576265977, 22.786821433001347, 22.90769732179668, 22.83044635872889, 22.79697745372237, 22.772261639208597, 22.682730347366707, 22.650531227322727, 22.61176806884516, 22.535798555227515, 22.497060450084113, 22.459777499535704, 22.388930830707437, 22.405540673639184, 22.377011942024396, 22.326494622470392, 22.34237924096282, 22.325925284096115, 22.36620033845359, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 17.05663555091456, "threshold_area100": 16.922007323760454, "threshold_area25": 10.95958361949672}, "selected_budget": 78, "selected": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405, "rescued": 784, "harmed": 350}, "test": {"score": 23.02798282770131, "calls": 79.12782956058588, "rescued": 3302, "harmed": 1461}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.562401533126831, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5476144552230835, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5225101709365845, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5162160992622375, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5081911087036133, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5007472038269043, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4886986017227173, "operator": "ge"}, {"budget": 0.08, "threshold": 0.47367915511131287, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4594859480857849, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4294031858444214, "operator": "ge"}, {"budget": 0.11, "threshold": 0.3874030113220215, "operator": "ge"}, {"budget": 0.12, "threshold": 0.36742639541625977, "operator": "ge"}, {"budget": 0.13, "threshold": 0.34740594029426575, "operator": "ge"}, {"budget": 0.14, "threshold": 0.33319902420043945, "operator": "ge"}, {"budget": 0.15, "threshold": 0.32046765089035034, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3076246976852417, "operator": "ge"}, {"budget": 0.17, "threshold": 0.2963269352912903, "operator": "ge"}, {"budget": 0.18, "threshold": 0.2908954918384552, "operator": "ge"}, {"budget": 0.19, "threshold": 0.28546977043151855, "operator": "ge"}, {"budget": 0.2, "threshold": 0.28068965673446655, "operator": "ge"}, {"budget": 0.21, "threshold": 0.27192598581314087, "operator": "ge"}, {"budget": 0.22, "threshold": 0.26402825117111206, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2613295018672943, "operator": "ge"}, {"budget": 0.24, "threshold": 0.2582085132598877, "operator": "ge"}, {"budget": 0.25, "threshold": 0.25220006704330444, "operator": "ge"}, {"budget": 0.26, "threshold": 0.24887080490589142, "operator": "ge"}, {"budget": 0.27, "threshold": 0.24481351673603058, "operator": "ge"}, {"budget": 0.28, "threshold": 0.2414548397064209, "operator": "ge"}, {"budget": 0.29, "threshold": 0.2382417470216751, "operator": "ge"}, {"budget": 0.3, "threshold": 0.23540565371513367, "operator": "ge"}, {"budget": 0.31, "threshold": 0.23301905393600464, "operator": "ge"}, {"budget": 0.32, "threshold": 0.2300785630941391, "operator": "ge"}, {"budget": 0.33, "threshold": 0.22698502242565155, "operator": "ge"}, {"budget": 0.34, "threshold": 0.22217971086502075, "operator": "ge"}, {"budget": 0.35, "threshold": 0.21922563016414642, "operator": "ge"}, {"budget": 0.36, "threshold": 0.21631769835948944, "operator": "ge"}, {"budget": 0.37, "threshold": 0.21346096694469452, "operator": "ge"}, {"budget": 0.38, "threshold": 0.21189725399017334, "operator": "ge"}, {"budget": 0.39, "threshold": 0.20544126629829407, "operator": "ge"}, {"budget": 0.4, "threshold": 0.19613584876060486, "operator": "ge"}, {"budget": 0.41, "threshold": 0.18755994737148285, "operator": "ge"}, {"budget": 0.42, "threshold": 0.17437893152236938, "operator": "ge"}, {"budget": 0.43, "threshold": 0.1570960432291031, "operator": "ge"}, {"budget": 0.44, "threshold": 0.14416588842868805, "operator": "ge"}, {"budget": 0.45, "threshold": 0.13097108900547028, "operator": "ge"}, {"budget": 0.46, "threshold": 0.12036074697971344, "operator": "ge"}, {"budget": 0.47, "threshold": 0.11317489296197891, "operator": "ge"}, {"budget": 0.48, "threshold": 0.1046387106180191, "operator": "ge"}, {"budget": 0.49, "threshold": 0.09976215660572052, "operator": "ge"}, {"budget": 0.5, "threshold": 0.09640084207057953, "operator": "ge"}, {"budget": 0.51, "threshold": 0.09012529253959656, "operator": "ge"}, {"budget": 0.52, "threshold": 0.08595035970211029, "operator": "ge"}, {"budget": 0.53, "threshold": 0.08438055217266083, "operator": "ge"}, {"budget": 0.54, "threshold": 0.07876846194267273, "operator": "ge"}, {"budget": 0.55, "threshold": 0.07254940271377563, "operator": "ge"}, {"budget": 0.56, "threshold": 0.07015599310398102, "operator": "ge"}, {"budget": 0.57, "threshold": 0.06705030798912048, "operator": "ge"}, {"budget": 0.58, "threshold": 0.06544587016105652, "operator": "ge"}, {"budget": 0.59, "threshold": 0.06082373857498169, "operator": "ge"}, {"budget": 0.6, "threshold": 0.05757074058055878, "operator": "ge"}, {"budget": 0.61, "threshold": 0.05466780066490173, "operator": "ge"}, {"budget": 0.62, "threshold": 0.051687926054000854, "operator": "ge"}, {"budget": 0.63, "threshold": 0.04578307271003723, "operator": "ge"}, {"budget": 0.64, "threshold": 0.04174025356769562, "operator": "ge"}, {"budget": 0.65, "threshold": 0.038356050848960876, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03220337629318237, "operator": "ge"}, {"budget": 0.67, "threshold": 0.028309263288974762, "operator": "ge"}, {"budget": 0.68, "threshold": 0.023308873176574707, "operator": "ge"}, {"budget": 0.69, "threshold": 0.018583059310913086, "operator": "ge"}, {"budget": 0.7, "threshold": 0.012941941618919373, "operator": "ge"}, {"budget": 0.71, "threshold": 0.008696600794792175, "operator": "ge"}, {"budget": 0.72, "threshold": 0.0034252703189849854, "operator": "ge"}, {"budget": 0.73, "threshold": -0.00031051039695739746, "operator": "ge"}, {"budget": 0.74, "threshold": -0.005284950137138367, "operator": "ge"}, {"budget": 0.75, "threshold": -0.010452404618263245, "operator": "ge"}, {"budget": 0.76, "threshold": -0.02237015962600708, "operator": "ge"}, {"budget": 0.77, "threshold": -0.04619631543755531, "operator": "ge"}, {"budget": 0.78, "threshold": -0.07148108631372452, "operator": "ge"}, {"budget": 0.79, "threshold": -0.08277718722820282, "operator": "ge"}, {"budget": 0.8, "threshold": -0.09244003891944885, "operator": "ge"}, {"budget": 0.81, "threshold": -0.09918755292892456, "operator": "ge"}, {"budget": 0.82, "threshold": -0.10529748350381851, "operator": "ge"}, {"budget": 0.83, "threshold": -0.11200641095638275, "operator": "ge"}, {"budget": 0.84, "threshold": -0.1160203218460083, "operator": "ge"}, {"budget": 0.85, "threshold": -0.12322317808866501, "operator": "ge"}, {"budget": 0.86, "threshold": -0.12778690457344055, "operator": "ge"}, {"budget": 0.87, "threshold": -0.12906843423843384, "operator": "ge"}, {"budget": 0.88, "threshold": -0.13212484121322632, "operator": "ge"}, {"budget": 0.89, "threshold": -0.14142785966396332, "operator": "ge"}, {"budget": 0.9, "threshold": -0.14800646901130676, "operator": "ge"}, {"budget": 0.91, "threshold": -0.15205591917037964, "operator": "ge"}, {"budget": 0.92, "threshold": -0.1571233868598938, "operator": "ge"}, {"budget": 0.93, "threshold": -0.16460110247135162, "operator": "ge"}, {"budget": 0.94, "threshold": -0.1697249710559845, "operator": "ge"}, {"budget": 0.95, "threshold": -0.1742798089981079, "operator": "ge"}, {"budget": 0.96, "threshold": -0.17611780762672424, "operator": "ge"}, {"budget": 0.97, "threshold": -0.18029256165027618, "operator": "ge"}, {"budget": 0.98, "threshold": -0.19128714501857758, "operator": "ge"}, {"budget": 0.99, "threshold": -0.1985744684934616, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.751224860383048, 7.8647554833266256, 7.9911652288462935, 9.577805334988993, 9.262409433216886, 9.29972123206288, 9.28835558612428, 9.311849723564494, 9.377778616605433, 9.278099205529992, 9.39363439730512, 9.479117740278683, 9.573867545244191, 9.687804714986205, 9.778586607904547, 9.872512800466046, 11.333587160929284, 11.446139361945868, 11.405916184369028, 12.127586948468522, 12.138878707168395, 12.187292277660415, 12.278893171055277, 12.223722012809205, 12.71682606274546, 12.76225880444769, 12.817712416329977, 12.949638359943702, 12.924574226387698, 13.479698816830876, 13.5728732180524, 13.715311014962584, 13.823834408389304, 14.204049295774649, 14.285401155119882, 14.40149150665076, 15.232279282813291, 15.344942114886543, 15.232511610527796, 15.27030854210941, 15.446756280089613, 15.967915815965203, 16.20206231239208, 16.37467010601339, 16.483942494525603, 16.45151844279751, 16.66703199192118, 16.873411282941067, 17.01904265312638, 17.236342417065305, 17.473634791403992, 17.69666390823557, 17.970942545948667, 18.154546730647496, 18.428005610209, 18.527961168001326, 18.539392501186523, 18.95590432347296, 18.98447097122594, 19.301065603530628, 19.776565083700277, 19.857531848682292, 20.086248087693885, 20.479566202564115, 20.79592509484593, 20.90470815961128, 20.835281761294127, 21.047021534722752, 21.29129678405359, 20.90524471512084, 20.92767547431795, 20.958811642884374, 21.29253535915357, 21.69301411375175, 21.695164782935986, 21.664808006327934, 21.88429108891366, 21.936136832098168, 21.886727169206157, 21.841661913929773, 21.816685004651358, 21.59357789952178, 21.3845179085125, 21.345765798372078, 21.305031236608095, 21.197984374216016, 21.175821253648643, 21.15137036752496, 21.312974247333784, 21.2673681777618, 21.235240789488056, 21.195356639527656, 21.084582447005342, 21.05343617189555, 20.9271618043598, 20.827196459974722, 20.716736018920333, 20.667983503163832, 20.628109214552293, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 20, 40, 60, 79, 95, 111, 131, 148, 164, 179, 196, 208, 222, 234, 247, 258, 273, 285, 297, 310, 321, 333, 346, 359, 369, 382, 394, 406, 416, 428, 441, 453, 464, 474, 481, 493, 504, 515, 525, 537, 548, 561, 572, 582, 588, 593, 600, 608, 613, 621, 628, 635, 642, 650, 657, 664, 670, 677, 683, 691, 698, 704, 709, 716, 722, 727, 733, 740, 745, 748, 755, 760, 765, 770, 775, 779, 783, 784, 784, 784, 784, 784, 784, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786], "harmed": [0, 5, 11, 17, 20, 27, 31, 37, 41, 46, 50, 52, 54, 58, 61, 66, 70, 73, 75, 80, 82, 86, 90, 94, 101, 104, 109, 113, 115, 119, 121, 125, 128, 133, 137, 143, 147, 150, 154, 158, 164, 167, 169, 172, 176, 178, 181, 183, 186, 189, 195, 197, 202, 209, 214, 218, 226, 229, 235, 243, 249, 254, 262, 268, 275, 276, 281, 286, 290, 295, 301, 309, 314, 318, 324, 331, 337, 347, 350, 356, 362, 365, 370, 374, 379, 384, 389, 393, 401, 405, 411, 416, 422, 427, 433, 442, 447, 454, 461, 467, 475], "exact_score": [7.641657905799261, 7.751224860383048, 7.8647554833266256, 7.9911652288462935, 9.577805334988993, 9.262409433216886, 9.29972123206288, 9.28835558612428, 9.311849723564494, 9.377778616605433, 9.278099205529992, 9.39363439730512, 9.479117740278683, 9.573867545244191, 9.687804714986205, 9.778586607904547, 9.872512800466046, 11.333587160929284, 11.446139361945868, 11.405916184369028, 12.127586948468522, 12.138878707168395, 12.187292277660415, 12.278893171055277, 12.223722012809205, 12.71682606274546, 12.76225880444769, 12.817712416329977, 12.949638359943702, 12.924574226387698, 13.479698816830876, 13.5728732180524, 13.715311014962584, 13.823834408389304, 14.204049295774649, 14.285401155119882, 14.40149150665076, 15.232279282813291, 15.344942114886543, 15.232511610527796, 15.27030854210941, 15.446756280089613, 15.967915815965203, 16.20206231239208, 16.37467010601339, 16.483942494525603, 16.45151844279751, 16.66703199192118, 16.873411282941067, 17.01904265312638, 17.236342417065305, 17.473634791403992, 17.69666390823557, 17.970942545948667, 18.154546730647496, 18.428005610209, 18.527961168001326, 18.539392501186523, 18.95590432347296, 18.98447097122594, 19.301065603530628, 19.776565083700277, 19.857531848682292, 20.086248087693885, 20.479566202564115, 20.79592509484593, 20.90470815961128, 20.835281761294127, 21.047021534722752, 21.29129678405359, 20.90524471512084, 20.92767547431795, 20.958811642884374, 21.29253535915357, 21.69301411375175, 21.695164782935986, 21.664808006327934, 21.88429108891366, 21.936136832098168, 21.886727169206157, 21.841661913929773, 21.816685004651358, 21.59357789952178, 21.3845179085125, 21.345765798372078, 21.305031236608095, 21.197984374216016, 21.175821253648643, 21.15137036752496, 21.312974247333784, 21.2673681777618, 21.235240789488056, 21.195356639527656, 21.084582447005342, 21.05343617189555, 20.9271618043598, 20.827196459974722, 20.716736018920333, 20.667983503163832, 20.628109214552293, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.518105006270186, "threshold_area100": 16.51220649479473, "threshold_area25": 10.125184835878462}, "test": {"score": [7.8873978710439285, 7.939639427944812, 8.616751542507087, 9.568062882112246, 9.52061840828356, 9.49386570248796, 9.820450057785092, 10.723886860304797, 11.466861996120391, 11.322245983649767, 11.436769048766424, 11.288066677676422, 11.271940067925636, 11.26576985259875, 11.27796672507794, 11.642518333359677, 11.576312484995857, 12.190035536203224, 12.200829610402756, 12.226483856382602, 12.09070572314501, 12.117893666404521, 12.270228051136693, 12.272347656646044, 12.25428539822696, 12.38072070840513, 12.615278282334474, 12.642171871696542, 12.814651985340438, 12.885723618579423, 12.8857153607486, 13.04298916112967, 13.093592256928464, 13.381337311299976, 13.587344795111944, 13.861039854964874, 13.96872294693306, 14.208903573097654, 14.243815505637059, 14.393893836578167, 14.798548491980076, 15.09247317022116, 15.474694616812473, 15.475271487377304, 15.57553467569651, 15.703708522014658, 15.928167113875805, 16.066024585234377, 16.30745999628492, 16.47185487020651, 16.54818599047496, 16.75175920318677, 17.042036778171518, 17.189022469033894, 17.39386898704253, 17.735539264462766, 18.209210431957924, 18.67226240721224, 18.690518517553027, 19.10838942136915, 19.28978785935233, 19.52706561704815, 19.63396964068353, 19.830384432903145, 19.990088910659964, 20.215732662626635, 20.56579098009815, 20.791372646400994, 20.93100639819463, 21.12619287624727, 21.597272492810887, 21.74751000140118, 21.964135416300678, 22.15495041060595, 22.307505955090846, 22.406420747226196, 22.333753514234683, 22.873898438477497, 23.02798282770131, 22.89358493523716, 22.8661642005517, 22.978238097921505, 22.871523168579643, 22.8073370271752, 22.791826412841086, 22.715227972788046, 22.684796533950372, 22.680667804851087, 22.668584748372023, 22.66344070922748, 22.52324816394023, 22.502289021075512, 22.46489421529157, 22.415516482674928, 22.36421172506622, 22.353384812240886, 22.326494622470392, 22.35072965742292, 22.349995174930694, 22.39453064308806, 22.361115727328066], "calls": [0.0, 0.4327563249001331, 0.8655126498002662, 2.097203728362184, 2.5299600532623168, 2.96271637816245, 3.362183754993342, 4.360852197070573, 5.25965379494008, 5.9254327563249, 6.824234354194407, 7.92276964047936, 8.355525965379494, 8.988015978695072, 9.753661784287617, 10.5525965379494, 12.15046604527297, 13.581890812250332, 14.414114513981357, 15.07989347536618, 15.679094540612518, 17.41011984021305, 19.10785619174434, 19.740346205059918, 20.406125166444742, 21.904127829560586, 22.836218375499335, 24.06790945406125, 25.0, 26.098535286284953, 27.230359520639148, 28.19573901464714, 28.99467376830892, 30.059920106524636, 31.125166444740348, 32.323568575233026, 33.15579227696405, 34.05459387483356, 34.58721704394141, 36.78428761651132, 38.74833555259654, 40.37949400798935, 42.24367509986685, 43.47536617842876, 44.374167776298265, 45.40612516644474, 46.23834886817576, 47.13715046604528, 48.50199733688416, 49.30093209054594, 49.93342210386152, 51.231691078561916, 52.26364846870839, 52.563249001331556, 53.82822902796271, 55.625832223701735, 56.42476697736352, 57.290279627163784, 57.822902796271634, 59.38748335552596, 60.21970705725699, 61.55126498002663, 62.150466045272964, 63.715046604527295, 64.64713715046605, 65.5792276964048, 67.27696404793608, 68.50865512649801, 69.47403462050599, 70.50599201065246, 71.73768308921437, 72.47003994673769, 73.43541944074568, 74.13448735019973, 75.03328894806924, 75.33288948069242, 76.49800266311586, 77.89613848202397, 79.12782956058588, 80.32623169107856, 81.19174434087883, 82.42343541944075, 83.38881491344874, 84.72037283621837, 85.18641810918774, 86.25166444740347, 86.95073235685751, 87.05059920106525, 87.51664447403462, 88.74833555259653, 90.31291611185085, 90.87882822902796, 91.81091877496671, 93.70838881491345, 94.70705725699068, 95.63914780292943, 95.97203728362183, 96.80426098535287, 98.60186418109188, 99.23435419440746, 100.0], "rescued": [0, 38, 76, 175, 212, 246, 277, 359, 433, 480, 546, 621, 648, 679, 724, 770, 863, 940, 984, 1022, 1050, 1142, 1219, 1248, 1279, 1352, 1399, 1459, 1503, 1554, 1607, 1651, 1690, 1740, 1791, 1853, 1891, 1933, 1958, 2056, 2155, 2222, 2305, 2352, 2387, 2421, 2448, 2477, 2522, 2545, 2563, 2598, 2629, 2639, 2676, 2727, 2753, 2777, 2790, 2834, 2856, 2893, 2907, 2946, 2969, 2993, 3037, 3067, 3090, 3115, 3145, 3160, 3184, 3202, 3222, 3230, 3258, 3289, 3302, 3303, 3305, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3307, 3307, 3308, 3308, 3309, 3309, 3310, 3312, 3314, 3314], "harmed": [0, 9, 16, 38, 46, 53, 59, 78, 94, 107, 123, 140, 146, 156, 167, 183, 216, 236, 250, 261, 272, 302, 327, 340, 353, 381, 399, 423, 437, 460, 481, 493, 509, 530, 550, 569, 587, 597, 607, 644, 678, 704, 730, 744, 758, 773, 781, 796, 822, 838, 848, 864, 886, 892, 922, 969, 984, 1006, 1016, 1050, 1067, 1100, 1113, 1142, 1161, 1175, 1211, 1244, 1267, 1293, 1315, 1332, 1352, 1368, 1387, 1392, 1416, 1447, 1461, 1480, 1500, 1524, 1546, 1573, 1586, 1607, 1624, 1626, 1634, 1663, 1700, 1713, 1736, 1780, 1804, 1827, 1834, 1852, 1904, 1924, 1945], "exact_score": [7.8873978710439285, 9.198904871589873, 9.558365331042522, 9.487193883377222, 10.767902983717786, 11.507998324430412, 11.316392913580977, 11.345744506351716, 11.296881829093133, 11.26576985259875, 11.688659130600302, 11.57697904397365, 11.588967627809925, 12.193893061256555, 12.173284539306014, 12.223217220894174, 12.092835600899733, 12.128381631802847, 12.296745550559821, 12.29172633415358, 12.279479034840543, 12.258985369048686, 12.383430829495525, 12.610555625759076, 12.639183301544444, 12.814651985340438, 12.878974237748405, 12.863091372022625, 12.915217339635857, 13.093592256928464, 13.375387785679022, 13.601094396944182, 13.822622260142317, 13.873842548897366, 14.197120639833269, 14.222119050498923, 14.326537423226352, 14.506278114719162, 14.680936696534323, 14.789802306564702, 15.050549990508372, 15.169709566312223, 15.400357073476354, 15.57846160645793, 15.559667419857274, 15.659992584352986, 15.8789158662557, 16.03119532796636, 16.22180748018425, 16.340293906806956, 16.557438049813957, 16.492394511880914, 16.97426757689498, 17.274478991146538, 17.372021116470496, 17.5996583382293, 17.816094003000728, 18.332139421450243, 18.74482402842641, 18.947094238248116, 19.233108184260804, 19.38827362051418, 19.570253938905662, 19.705571887143943, 19.862708012033888, 20.020934451577382, 20.29435093798476, 20.5568606935498, 20.719800166130426, 20.92916417138577, 21.069716937667426, 21.371102799832247, 21.72077109881988, 21.816758291790624, 22.09888330541147, 22.292216206473224, 22.15444445583054, 22.532542899139138, 22.92071156308659, 23.011454552111445, 23.003308185020124, 22.88428576265977, 22.786821433001347, 22.90769732179668, 22.83044635872889, 22.79697745372237, 22.772261639208597, 22.682730347366707, 22.650531227322727, 22.61176806884516, 22.535798555227515, 22.497060450084113, 22.459777499535704, 22.388930830707437, 22.405540673639184, 22.377011942024396, 22.326494622470392, 22.34237924096282, 22.325925284096115, 22.36620033845359, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 17.05663555091456, "threshold_area100": 16.922007323760454, "threshold_area25": 10.95958361949672}, "selected_budget": 78, "selected": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405, "rescued": 784, "harmed": 350}, "test": {"score": 23.02798282770131, "calls": 79.12782956058588, "rescued": 3302, "harmed": 1461}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.9836065769195557, "operator": "ge"}, {"budget": 0.02, "threshold": 1.88120436668396, "operator": "ge"}, {"budget": 0.03, "threshold": 1.8182530403137207, "operator": "ge"}, {"budget": 0.04, "threshold": 1.761995792388916, "operator": "ge"}, {"budget": 0.05, "threshold": 1.685920000076294, "operator": "ge"}, {"budget": 0.06, "threshold": 1.6214509010314941, "operator": "ge"}, {"budget": 0.07, "threshold": 1.5676859617233276, "operator": "ge"}, {"budget": 0.08, "threshold": 1.551149606704712, "operator": "ge"}, {"budget": 0.09, "threshold": 1.5203171968460083, "operator": "ge"}, {"budget": 0.1, "threshold": 1.4643487930297852, "operator": "ge"}, {"budget": 0.11, "threshold": 1.4326410293579102, "operator": "ge"}, {"budget": 0.12, "threshold": 1.4044915437698364, "operator": "ge"}, {"budget": 0.13, "threshold": 1.3911895751953125, "operator": "ge"}, {"budget": 0.14, "threshold": 1.3665902614593506, "operator": "ge"}, {"budget": 0.15, "threshold": 1.3429641723632812, "operator": "ge"}, {"budget": 0.16, "threshold": 1.302901268005371, "operator": "ge"}, {"budget": 0.17, "threshold": 1.2767951488494873, "operator": "ge"}, {"budget": 0.18, "threshold": 1.2526791095733643, "operator": "ge"}, {"budget": 0.19, "threshold": 1.2346742153167725, "operator": "ge"}, {"budget": 0.2, "threshold": 1.2115905284881592, "operator": "ge"}, {"budget": 0.21, "threshold": 1.1918752193450928, "operator": "ge"}, {"budget": 0.22, "threshold": 1.156391978263855, "operator": "ge"}, {"budget": 0.23, "threshold": 1.1320593357086182, "operator": "ge"}, {"budget": 0.24, "threshold": 1.1156964302062988, "operator": "ge"}, {"budget": 0.25, "threshold": 1.0937244892120361, "operator": "ge"}, {"budget": 0.26, "threshold": 1.0763158798217773, "operator": "ge"}, {"budget": 0.27, "threshold": 1.0508761405944824, "operator": "ge"}, {"budget": 0.28, "threshold": 1.0122778415679932, "operator": "ge"}, {"budget": 0.29, "threshold": 0.9740498661994934, "operator": "ge"}, {"budget": 0.3, "threshold": 0.9575134515762329, "operator": "ge"}, {"budget": 0.31, "threshold": 0.9317517280578613, "operator": "ge"}, {"budget": 0.32, "threshold": 0.9157034754753113, "operator": "ge"}, {"budget": 0.33, "threshold": 0.9073486924171448, "operator": "ge"}, {"budget": 0.34, "threshold": 0.8785534501075745, "operator": "ge"}, {"budget": 0.35, "threshold": 0.8591265678405762, "operator": "ge"}, {"budget": 0.36, "threshold": 0.8466217517852783, "operator": "ge"}, {"budget": 0.37, "threshold": 0.8130110502243042, "operator": "ge"}, {"budget": 0.38, "threshold": 0.7956295013427734, "operator": "ge"}, {"budget": 0.39, "threshold": 0.7504947781562805, "operator": "ge"}, {"budget": 0.4, "threshold": 0.7261941432952881, "operator": "ge"}, {"budget": 0.41, "threshold": 0.7100965976715088, "operator": "ge"}, {"budget": 0.42, "threshold": 0.6805095076560974, "operator": "ge"}, {"budget": 0.43, "threshold": 0.6678675413131714, "operator": "ge"}, {"budget": 0.44, "threshold": 0.6607424020767212, "operator": "ge"}, {"budget": 0.45, "threshold": 0.6459220051765442, "operator": "ge"}, {"budget": 0.46, "threshold": 0.620721161365509, "operator": "ge"}, {"budget": 0.47, "threshold": 0.5777188539505005, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5555380582809448, "operator": "ge"}, {"budget": 0.49, "threshold": 0.5348286628723145, "operator": "ge"}, {"budget": 0.5, "threshold": 0.4968504309654236, "operator": "ge"}, {"budget": 0.51, "threshold": 0.48128455877304077, "operator": "ge"}, {"budget": 0.52, "threshold": 0.4541483521461487, "operator": "ge"}, {"budget": 0.53, "threshold": 0.4267812669277191, "operator": "ge"}, {"budget": 0.54, "threshold": 0.3883640468120575, "operator": "ge"}, {"budget": 0.55, "threshold": 0.37528008222579956, "operator": "ge"}, {"budget": 0.56, "threshold": 0.3515145778656006, "operator": "ge"}, {"budget": 0.57, "threshold": 0.3306446969509125, "operator": "ge"}, {"budget": 0.58, "threshold": 0.31221142411231995, "operator": "ge"}, {"budget": 0.59, "threshold": 0.30317914485931396, "operator": "ge"}, {"budget": 0.6, "threshold": 0.2860971689224243, "operator": "ge"}, {"budget": 0.61, "threshold": 0.27768611907958984, "operator": "ge"}, {"budget": 0.62, "threshold": 0.25837212800979614, "operator": "ge"}, {"budget": 0.63, "threshold": 0.24501419067382812, "operator": "ge"}, {"budget": 0.64, "threshold": 0.23308700323104858, "operator": "ge"}, {"budget": 0.65, "threshold": 0.20083856582641602, "operator": "ge"}, {"budget": 0.66, "threshold": 0.17357295751571655, "operator": "ge"}, {"budget": 0.67, "threshold": 0.1495928317308426, "operator": "ge"}, {"budget": 0.68, "threshold": 0.12849190831184387, "operator": "ge"}, {"budget": 0.69, "threshold": 0.08761619031429291, "operator": "ge"}, {"budget": 0.7, "threshold": 0.0617215633392334, "operator": "ge"}, {"budget": 0.71, "threshold": 0.037034809589385986, "operator": "ge"}, {"budget": 0.72, "threshold": 0.02195630967617035, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0014001727104187012, "operator": "ge"}, {"budget": 0.74, "threshold": -0.026258736848831177, "operator": "ge"}, {"budget": 0.75, "threshold": -0.04835638403892517, "operator": "ge"}, {"budget": 0.76, "threshold": -0.10706144571304321, "operator": "ge"}, {"budget": 0.77, "threshold": -0.19780509173870087, "operator": "ge"}, {"budget": 0.78, "threshold": -3.374541759490967, "operator": "ge"}, {"budget": 0.79, "threshold": -3.6506149768829346, "operator": "ge"}, {"budget": 0.8, "threshold": -3.771613597869873, "operator": "ge"}, {"budget": 0.81, "threshold": -3.8124423027038574, "operator": "ge"}, {"budget": 0.82, "threshold": -3.935145616531372, "operator": "ge"}, {"budget": 0.83, "threshold": -3.993921995162964, "operator": "ge"}, {"budget": 0.84, "threshold": -4.0642828941345215, "operator": "ge"}, {"budget": 0.85, "threshold": -4.120278358459473, "operator": "ge"}, {"budget": 0.86, "threshold": -4.138845443725586, "operator": "ge"}, {"budget": 0.87, "threshold": -4.194132328033447, "operator": "ge"}, {"budget": 0.88, "threshold": -4.217778205871582, "operator": "ge"}, {"budget": 0.89, "threshold": -4.269472122192383, "operator": "ge"}, {"budget": 0.9, "threshold": -4.295374393463135, "operator": "ge"}, {"budget": 0.91, "threshold": -4.337586879730225, "operator": "ge"}, {"budget": 0.92, "threshold": -4.397620677947998, "operator": "ge"}, {"budget": 0.93, "threshold": -4.425322532653809, "operator": "ge"}, {"budget": 0.94, "threshold": -4.456562042236328, "operator": "ge"}, {"budget": 0.95, "threshold": -4.490599632263184, "operator": "ge"}, {"budget": 0.96, "threshold": -4.531587600708008, "operator": "ge"}, {"budget": 0.97, "threshold": -4.5916619300842285, "operator": "ge"}, {"budget": 0.98, "threshold": -4.668234348297119, "operator": "ge"}, {"budget": 0.99, "threshold": -4.7300591468811035, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.530318894796231, 7.62839671788378, 9.917895298563694, 9.751616274478698, 9.67967569172439, 9.486508467365544, 9.381073913571882, 9.403976325304189, 9.397029055207803, 9.38875050455287, 9.350172187417218, 10.44718414342308, 10.38399951660171, 10.362214619549805, 10.382333977512555, 10.384248412997211, 10.400254671284664, 10.48064597905531, 10.575284116184568, 10.681592742905966, 11.424618373834878, 11.483950387594962, 11.527644851578403, 12.201742986086725, 12.291489968976746, 12.910701439287031, 12.918928297507712, 13.015477241828046, 12.865305285966958, 12.882720424272136, 13.394230000430527, 13.901375339566515, 14.022718370544455, 14.114580271198097, 14.084135466006972, 14.213881815004628, 14.384248613925399, 14.501096491228072, 14.694222067818735, 15.267667578421849, 15.252559191220378, 15.377110326500443, 15.569649780969586, 15.565480282663888, 15.658642407660498, 15.684509991226554, 16.194626463143884, 16.38363323956986, 16.960329954987287, 16.968393928452738, 17.195986472140046, 17.363072666633755, 17.71335872478488, 17.744639081848383, 17.75009583864954, 18.044733044733047, 18.48080310695563, 18.662738045783954, 19.014543308981334, 19.25738857995367, 19.47611676249447, 19.738188221954054, 19.844232226399498, 19.91621813946797, 20.366835661205123, 20.456473600502182, 20.649903499012108, 20.88303056263978, 21.24156670112976, 20.801673553096705, 20.787986051600953, 20.958811642884374, 21.35373035036712, 21.491875576984707, 21.98418212397645, 21.781037930250164, 21.874374031575037, 21.81489096867713, 21.411268914793467, 21.361850838628918, 21.320228452145955, 21.277766590617944, 21.063886762048313, 21.02319207109198, 21.162435963342528, 21.100868130197444, 21.06542880358718, 21.041712763638827, 21.01755901408878, 20.971083940475392, 20.9532585555992, 20.85755237602752, 20.840191066028634, 20.72893051744024, 20.69808210005916, 20.766271448846584, 20.73336237815821, 20.717234042753752, 20.685604348672708, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 14, 31, 51, 66, 74, 88, 103, 120, 135, 151, 163, 179, 192, 207, 219, 231, 246, 260, 274, 291, 304, 313, 324, 339, 347, 360, 371, 381, 393, 402, 414, 428, 441, 454, 467, 478, 487, 499, 510, 523, 536, 544, 555, 563, 570, 578, 589, 598, 609, 616, 624, 629, 637, 643, 649, 657, 666, 672, 681, 688, 695, 702, 707, 714, 721, 725, 731, 737, 744, 748, 754, 760, 766, 770, 776, 778, 783, 784, 784, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786, 786], "harmed": [0, 3, 6, 11, 14, 18, 22, 26, 29, 33, 37, 40, 43, 47, 52, 55, 58, 64, 67, 70, 75, 77, 79, 82, 85, 85, 89, 91, 95, 101, 106, 111, 116, 120, 124, 129, 132, 135, 140, 143, 146, 152, 155, 160, 165, 168, 172, 177, 183, 186, 192, 199, 203, 206, 214, 220, 224, 230, 233, 237, 245, 251, 255, 262, 269, 273, 279, 283, 289, 294, 301, 308, 314, 319, 324, 330, 339, 348, 356, 365, 371, 376, 381, 386, 396, 402, 410, 415, 418, 422, 429, 432, 436, 439, 446, 451, 456, 462, 465, 471, 475], "exact_score": [7.641657905799261, 7.530318894796231, 7.62839671788378, 9.917895298563694, 9.751616274478698, 9.67967569172439, 9.486508467365544, 9.381073913571882, 9.403976325304189, 9.397029055207803, 9.38875050455287, 9.350172187417218, 10.44718414342308, 10.38399951660171, 10.362214619549805, 10.382333977512555, 10.384248412997211, 10.400254671284664, 10.48064597905531, 10.575284116184568, 10.681592742905966, 11.424618373834878, 11.483950387594962, 11.527644851578403, 12.201742986086725, 12.291489968976746, 12.910701439287031, 12.918928297507712, 13.015477241828046, 12.865305285966958, 12.882720424272136, 13.394230000430527, 13.901375339566515, 14.022718370544455, 14.114580271198097, 14.084135466006972, 14.213881815004628, 14.384248613925399, 14.501096491228072, 14.694222067818735, 15.267667578421849, 15.252559191220378, 15.377110326500443, 15.569649780969586, 15.565480282663888, 15.658642407660498, 15.684509991226554, 16.194626463143884, 16.38363323956986, 16.960329954987287, 16.968393928452738, 17.195986472140046, 17.363072666633755, 17.71335872478488, 17.744639081848383, 17.75009583864954, 18.044733044733047, 18.48080310695563, 18.662738045783954, 19.014543308981334, 19.25738857995367, 19.47611676249447, 19.738188221954054, 19.844232226399498, 19.91621813946797, 20.366835661205123, 20.456473600502182, 20.649903499012108, 20.88303056263978, 21.24156670112976, 20.801673553096705, 20.787986051600953, 20.958811642884374, 21.35373035036712, 21.491875576984707, 21.98418212397645, 21.781037930250164, 21.874374031575037, 21.81489096867713, 21.411268914793467, 21.361850838628918, 21.320228452145955, 21.277766590617944, 21.063886762048313, 21.02319207109198, 21.162435963342528, 21.100868130197444, 21.06542880358718, 21.041712763638827, 21.01755901408878, 20.971083940475392, 20.9532585555992, 20.85755237602752, 20.840191066028634, 20.72893051744024, 20.69808210005916, 20.766271448846584, 20.73336237815821, 20.717234042753752, 20.685604348672708, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.329498370334683, "threshold_area100": 16.323289352854943, "threshold_area25": 10.019785370816757}, "test": {"score": [7.8873978710439285, 8.722222609411451, 9.356210638858627, 9.31899495102009, 9.260868413479194, 9.177726038442001, 9.643808769818799, 10.000430995651952, 10.322768767447082, 10.625462628858214, 10.525856028657302, 10.489197087059166, 10.746123159228828, 11.005894888024716, 11.261988466245526, 11.436105603150887, 11.331528796174743, 11.497827258970911, 11.417219549201171, 11.396842172559156, 11.446499724693934, 11.538607619972867, 11.712482389678964, 11.74839094887213, 11.91100401645361, 11.938967767466192, 11.955332758609911, 11.893204334914204, 12.087702986202794, 12.383644967070998, 12.538784883817067, 12.681245677496552, 12.826165434408898, 12.7758097544014, 12.92332441922876, 13.09500907194107, 13.131044740907294, 13.608595232870686, 13.73621595585362, 14.034366713633679, 14.227498827885798, 14.474687226021654, 14.631310958311877, 14.69167227665355, 14.751085848590028, 15.012250850834619, 15.108892944661715, 15.404400487089736, 15.54924892542574, 15.60334289333677, 15.971752149429333, 16.072295313760083, 16.56636193381315, 16.820185920665544, 17.09245658617052, 17.277316771567577, 17.640243384591255, 17.88858154133578, 18.00340743014404, 18.157807339596904, 18.629436305180775, 18.734042242386597, 19.00751322676492, 19.18339932105525, 19.353706460072036, 19.572782680171148, 19.94369718053885, 20.34454150329407, 20.746712147185004, 21.286071868595624, 21.578197930210774, 21.754964468959255, 21.900826797434274, 22.15495041060595, 22.307505955090846, 22.43275195915528, 22.343640119352447, 22.873898438477497, 22.968526124749264, 22.745131680808527, 22.61617822458154, 22.614513835499935, 22.534525380275912, 22.457267556030967, 22.38158458591297, 22.309900207801856, 22.290799973816398, 22.36577423243351, 22.388990793083487, 22.352962962121985, 22.339497079531377, 22.615604916842422, 22.617864196327545, 22.59549766439607, 22.553067200665666, 22.532622704025208, 22.498954027855405, 22.42848437970085, 22.3932386565971, 22.380498159848667, 22.361115727328066], "calls": [0.0, 1.0652463382157125, 1.6311584553928096, 2.130492676431425, 2.463382157123835, 3.462050599201065, 4.227696404793608, 5.159786950732357, 5.692410119840213, 6.324900133155792, 7.356857523302264, 8.255659121171771, 9.021304926764314, 9.187749667110518, 9.85352862849534, 10.585885486018643, 11.917443408788282, 12.849533954727029, 13.681757656458057, 14.480692410119842, 15.579227696404793, 16.311584553928096, 17.7762982689747, 18.575233022636482, 19.274300932090547, 19.973368841544605, 20.605858854860188, 21.97070572569907, 23.26897470039947, 25.133155792276963, 25.665778961384824, 27.097203728362185, 28.22902796271638, 28.66178428761651, 30.559254327563252, 31.491344873501998, 32.39014647137151, 34.387483355525966, 35.18641810918775, 38.24900133155793, 39.4474034620506, 40.37949400798935, 42.01065246338216, 42.64314247669774, 43.275632490013315, 44.14114513981358, 45.372836218375504, 47.270306258322236, 48.1358189081225, 48.801597869507326, 50.19973368841545, 50.76564580559254, 51.76431424766977, 52.92942743009321, 54.627163781624496, 55.426098535286286, 56.45805592543276, 57.55659121171771, 58.62183754993342, 59.32090545938748, 60.31957390146472, 60.81890812250332, 61.98402130492676, 62.916111850865505, 63.78162450066578, 65.31291611185085, 66.74434087882824, 67.87616511318242, 69.00798934753661, 70.93874833555259, 71.63781624500666, 72.7363515312916, 73.16910785619174, 74.13448735019973, 75.0, 75.3994673768309, 76.53129161118508, 77.86284953395473, 79.16111850865512, 80.45938748335553, 81.45805592543276, 81.6577896138482, 82.78961384820239, 83.65512649800266, 84.88681757656458, 85.88548601864181, 86.35153129161118, 87.1837549933422, 87.78295605858855, 88.61517976031958, 89.08122503328894, 90.07989347536618, 91.44474034620505, 92.01065246338216, 92.67643142476697, 93.24234354194407, 94.04127829560586, 95.17310252996005, 96.33821571238349, 97.60319573901465, 100.0], "rescued": [0, 79, 118, 148, 170, 233, 284, 342, 373, 406, 459, 514, 560, 572, 613, 654, 734, 785, 830, 871, 926, 964, 1046, 1077, 1115, 1152, 1184, 1246, 1312, 1397, 1420, 1491, 1549, 1566, 1661, 1712, 1753, 1843, 1884, 2029, 2078, 2121, 2193, 2218, 2248, 2297, 2349, 2435, 2467, 2488, 2540, 2558, 2592, 2628, 2676, 2700, 2730, 2761, 2791, 2812, 2839, 2853, 2884, 2910, 2935, 2974, 3012, 3046, 3077, 3123, 3142, 3166, 3178, 3202, 3222, 3232, 3259, 3289, 3304, 3305, 3305, 3305, 3306, 3306, 3306, 3306, 3306, 3308, 3309, 3309, 3309, 3311, 3312, 3312, 3312, 3312, 3312, 3312, 3312, 3313, 3314], "harmed": [0, 6, 11, 16, 22, 38, 46, 59, 67, 72, 83, 96, 106, 110, 117, 131, 153, 167, 180, 190, 201, 213, 238, 248, 259, 269, 278, 303, 325, 360, 364, 394, 418, 429, 461, 476, 492, 528, 543, 602, 619, 634, 671, 687, 702, 722, 742, 782, 797, 807, 837, 848, 867, 887, 928, 942, 963, 987, 1011, 1026, 1048, 1059, 1089, 1109, 1130, 1161, 1194, 1220, 1249, 1297, 1313, 1338, 1348, 1368, 1387, 1394, 1419, 1447, 1476, 1508, 1533, 1534, 1561, 1581, 1613, 1634, 1649, 1666, 1681, 1701, 1709, 1732, 1761, 1775, 1792, 1806, 1828, 1852, 1876, 1900, 1945], "exact_score": [7.8873978710439285, 8.715046950192141, 9.359074864431992, 9.235159026677273, 9.645267281385495, 9.599203293680548, 10.305517083179975, 10.566487846481802, 10.509186336201134, 10.74301991313363, 11.51170482344724, 11.376160495236102, 11.327667251547306, 11.486692812448805, 11.406210173630079, 11.42817085466777, 11.424363750610318, 11.712956631298024, 11.721180286821783, 11.737161741586759, 11.938967767466192, 11.925776557430709, 11.893204334914204, 12.051338370790965, 12.347379430884939, 12.373385977602453, 12.446162928086316, 12.663379413610787, 12.834118239864697, 12.812537042575944, 12.832394737337777, 12.930298007754432, 13.10784491460021, 13.369159759481647, 13.534168487080938, 13.612725375406063, 13.855000265659775, 13.950130086596282, 14.000974161353295, 14.167630112616408, 14.4328897803918, 14.455744353061172, 14.628235034862994, 14.582083873378968, 14.990918517651385, 14.973464730754246, 15.201213775227215, 15.386388481742951, 15.525607827947443, 15.610198942915316, 15.917829284372742, 16.36190017685225, 16.685311479925087, 16.827396993288467, 16.84424884419078, 17.407029964014185, 17.585798015868715, 17.707452676927318, 17.874067739204417, 18.059515204736744, 18.538700259785625, 18.840952930188337, 19.00751322676492, 19.196909374422784, 19.362932544741287, 19.53677100234333, 19.78623975375737, 20.02093870850625, 20.358076348584245, 20.736082823795957, 20.922832743493277, 21.331003495493587, 21.639653677226903, 21.81936796990985, 22.09888330541147, 22.307505955090846, 22.594219162306388, 22.49727186807724, 22.93677930397971, 22.99086923894459, 22.814747165837765, 22.674635123991862, 22.645461702078997, 22.522653643679345, 22.437676996836938, 22.37745846412361, 22.304349858502555, 22.31474241011509, 22.383872892076393, 22.34259749764491, 22.620139117671535, 22.629900073423254, 22.597298384684187, 22.54341195215139, 22.502024085875018, 22.432792914722317, 22.40503781357057, 22.412926360763386, 22.371345770168357, 22.343165346158607, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.650407046111866, "threshold_area100": 16.52702656187264, "threshold_area25": 10.499874042839837}, "selected_budget": 75, "selected": {"val": {"score": 21.98418212397645, "calls": 74.89300998573466, "rescued": 776, "harmed": 330}, "test": {"score": 22.43275195915528, "calls": 75.3994673768309, "rescued": 3232, "harmed": 1394}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.3841099739074707, "operator": "ge"}, {"budget": 0.02, "threshold": 1.3280714750289917, "operator": "ge"}, {"budget": 0.03, "threshold": 1.28305184841156, "operator": "ge"}, {"budget": 0.04, "threshold": 1.2211428880691528, "operator": "ge"}, {"budget": 0.05, "threshold": 1.2018858194351196, "operator": "ge"}, {"budget": 0.06, "threshold": 1.1788320541381836, "operator": "ge"}, {"budget": 0.07, "threshold": 1.1669158935546875, "operator": "ge"}, {"budget": 0.08, "threshold": 1.136621117591858, "operator": "ge"}, {"budget": 0.09, "threshold": 1.1156083345413208, "operator": "ge"}, {"budget": 0.1, "threshold": 1.0696393251419067, "operator": "ge"}, {"budget": 0.11, "threshold": 1.0432310104370117, "operator": "ge"}, {"budget": 0.12, "threshold": 1.0151877403259277, "operator": "ge"}, {"budget": 0.13, "threshold": 0.9948302507400513, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9737638235092163, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9500482082366943, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9173214435577393, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8941655158996582, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8449310064315796, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8196066617965698, "operator": "ge"}, {"budget": 0.2, "threshold": 0.786255955696106, "operator": "ge"}, {"budget": 0.21, "threshold": 0.7696319222450256, "operator": "ge"}, {"budget": 0.22, "threshold": 0.7436544299125671, "operator": "ge"}, {"budget": 0.23, "threshold": 0.7381226420402527, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7268004417419434, "operator": "ge"}, {"budget": 0.25, "threshold": 0.7178480625152588, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7005915641784668, "operator": "ge"}, {"budget": 0.27, "threshold": 0.6934118866920471, "operator": "ge"}, {"budget": 0.28, "threshold": 0.6802068948745728, "operator": "ge"}, {"budget": 0.29, "threshold": 0.6641900539398193, "operator": "ge"}, {"budget": 0.3, "threshold": 0.6447098255157471, "operator": "ge"}, {"budget": 0.31, "threshold": 0.642487645149231, "operator": "ge"}, {"budget": 0.32, "threshold": 0.6334774494171143, "operator": "ge"}, {"budget": 0.33, "threshold": 0.6305820345878601, "operator": "ge"}, {"budget": 0.34, "threshold": 0.62615966796875, "operator": "ge"}, {"budget": 0.35, "threshold": 0.6141831874847412, "operator": "ge"}, {"budget": 0.36, "threshold": 0.6069166660308838, "operator": "ge"}, {"budget": 0.37, "threshold": 0.5942542552947998, "operator": "ge"}, {"budget": 0.38, "threshold": 0.5821768045425415, "operator": "ge"}, {"budget": 0.39, "threshold": 0.579082190990448, "operator": "ge"}, {"budget": 0.4, "threshold": 0.5649487972259521, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5552624464035034, "operator": "ge"}, {"budget": 0.42, "threshold": 0.5465511083602905, "operator": "ge"}, {"budget": 0.43, "threshold": 0.5331722497940063, "operator": "ge"}, {"budget": 0.44, "threshold": 0.5150928497314453, "operator": "ge"}, {"budget": 0.45, "threshold": 0.5053344368934631, "operator": "ge"}, {"budget": 0.46, "threshold": 0.4803967773914337, "operator": "ge"}, {"budget": 0.47, "threshold": 0.46560823917388916, "operator": "ge"}, {"budget": 0.48, "threshold": 0.44705864787101746, "operator": "ge"}, {"budget": 0.49, "threshold": 0.4219782054424286, "operator": "ge"}, {"budget": 0.5, "threshold": 0.39833036065101624, "operator": "ge"}, {"budget": 0.51, "threshold": 0.3778242766857147, "operator": "ge"}, {"budget": 0.52, "threshold": 0.36664241552352905, "operator": "ge"}, {"budget": 0.53, "threshold": 0.35388126969337463, "operator": "ge"}, {"budget": 0.54, "threshold": 0.32426875829696655, "operator": "ge"}, {"budget": 0.55, "threshold": 0.3123324513435364, "operator": "ge"}, {"budget": 0.56, "threshold": 0.2982025444507599, "operator": "ge"}, {"budget": 0.57, "threshold": 0.2792671322822571, "operator": "ge"}, {"budget": 0.58, "threshold": 0.26689407229423523, "operator": "ge"}, {"budget": 0.59, "threshold": 0.25557032227516174, "operator": "ge"}, {"budget": 0.6, "threshold": 0.2403818815946579, "operator": "ge"}, {"budget": 0.61, "threshold": 0.22519099712371826, "operator": "ge"}, {"budget": 0.62, "threshold": 0.21236690878868103, "operator": "ge"}, {"budget": 0.63, "threshold": 0.19616949558258057, "operator": "ge"}, {"budget": 0.64, "threshold": 0.18040108680725098, "operator": "ge"}, {"budget": 0.65, "threshold": 0.16122540831565857, "operator": "ge"}, {"budget": 0.66, "threshold": 0.13335707783699036, "operator": "ge"}, {"budget": 0.67, "threshold": 0.11338472366333008, "operator": "ge"}, {"budget": 0.68, "threshold": 0.1016765832901001, "operator": "ge"}, {"budget": 0.69, "threshold": 0.07680389285087585, "operator": "ge"}, {"budget": 0.7, "threshold": 0.05416730046272278, "operator": "ge"}, {"budget": 0.71, "threshold": 0.034271836280822754, "operator": "ge"}, {"budget": 0.72, "threshold": 0.016619205474853516, "operator": "ge"}, {"budget": 0.73, "threshold": -0.0013638138771057129, "operator": "ge"}, {"budget": 0.74, "threshold": -0.023326724767684937, "operator": "ge"}, {"budget": 0.75, "threshold": -0.0404604971408844, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0926232784986496, "operator": "ge"}, {"budget": 0.77, "threshold": -0.17123916745185852, "operator": "ge"}, {"budget": 0.78, "threshold": -3.0454840660095215, "operator": "ge"}, {"budget": 0.79, "threshold": -3.308070182800293, "operator": "ge"}, {"budget": 0.8, "threshold": -3.440373659133911, "operator": "ge"}, {"budget": 0.81, "threshold": -3.522728204727173, "operator": "ge"}, {"budget": 0.82, "threshold": -3.5921401977539062, "operator": "ge"}, {"budget": 0.83, "threshold": -3.6883649826049805, "operator": "ge"}, {"budget": 0.84, "threshold": -3.7570557594299316, "operator": "ge"}, {"budget": 0.85, "threshold": -3.8251118659973145, "operator": "ge"}, {"budget": 0.86, "threshold": -3.8914785385131836, "operator": "ge"}, {"budget": 0.87, "threshold": -3.918229579925537, "operator": "ge"}, {"budget": 0.88, "threshold": -3.9414191246032715, "operator": "ge"}, {"budget": 0.89, "threshold": -3.9815497398376465, "operator": "ge"}, {"budget": 0.9, "threshold": -4.017856597900391, "operator": "ge"}, {"budget": 0.91, "threshold": -4.0542449951171875, "operator": "ge"}, {"budget": 0.92, "threshold": -4.088169574737549, "operator": "ge"}, {"budget": 0.93, "threshold": -4.14713716506958, "operator": "ge"}, {"budget": 0.94, "threshold": -4.174187183380127, "operator": "ge"}, {"budget": 0.95, "threshold": -4.211908340454102, "operator": "ge"}, {"budget": 0.96, "threshold": -4.2439069747924805, "operator": "ge"}, {"budget": 0.97, "threshold": -4.2823567390441895, "operator": "ge"}, {"budget": 0.98, "threshold": -4.334841251373291, "operator": "ge"}, {"budget": 0.99, "threshold": -4.508983612060547, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.742172034620776, 7.843963096312895, 7.744109685989099, 7.842662080466958, 7.9597917930100115, 9.422900981230727, 9.43722961285429, 9.419944462140146, 9.436462354221488, 9.495084330066264, 9.565611422674506, 9.566009923328783, 9.54514190391733, 9.567961215360752, 9.672768076851018, 9.6564744750109, 9.774656856755884, 9.841946322520846, 10.640491856844955, 10.734526649968949, 10.815748616431252, 10.81952841987999, 11.482218974546338, 12.183496241002535, 12.230686132283244, 12.275557653400648, 13.407337071200336, 13.50920124714541, 13.636265355173215, 13.733106821515973, 13.916445287792536, 14.030978079891124, 14.11917391652251, 14.233693701481648, 14.23950412799679, 14.374610580328179, 14.363544024461808, 14.362059862916784, 15.317920105070392, 15.343126065200485, 15.76130618803151, 15.824887837517021, 15.867356843858992, 16.0287699497436, 16.177032203561108, 16.345024464269915, 16.47033300590613, 16.6742154715381, 16.845723069209306, 17.12606145578243, 17.675677337061092, 17.96637196542062, 17.97322753288932, 18.25345717852033, 18.34137497652514, 18.54710212268352, 19.050878386486946, 19.352143836047244, 19.686734421007284, 19.734941174844124, 19.940855845957312, 20.185840040046042, 20.330728498235974, 20.467522446689113, 20.92717745954965, 21.090958634410033, 21.250369254401512, 21.093862121514753, 21.228382363757582, 20.908492429229756, 21.049766156312995, 20.958811642884374, 21.35373035036712, 21.491875576984707, 21.695164782935986, 21.85600621805136, 21.874374031575037, 21.73202614379085, 21.497215357368617, 21.359437751004016, 21.139786389077404, 21.084742651303294, 21.063886762048313, 21.0315893913479, 20.981842395276058, 20.927805315149254, 21.06303544276124, 20.96044949330227, 20.93946839357991, 20.900564692613756, 20.88034831210798, 20.706452404065136, 20.671881639296352, 20.640697123027866, 20.624117603017737, 20.593886062685232, 20.57334362166725, 20.712927410540143, 20.685604348672708, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 17, 34, 51, 66, 84, 103, 118, 137, 155, 171, 188, 197, 209, 225, 239, 248, 262, 273, 289, 303, 314, 322, 336, 346, 358, 370, 380, 392, 402, 413, 424, 436, 447, 458, 467, 477, 486, 498, 511, 520, 531, 542, 552, 562, 572, 581, 591, 599, 606, 614, 623, 632, 638, 645, 652, 659, 669, 677, 684, 690, 696, 703, 708, 713, 722, 729, 735, 737, 742, 748, 755, 760, 766, 770, 775, 779, 783, 784, 784, 784, 784, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786], "harmed": [0, 4, 8, 14, 17, 22, 28, 33, 39, 44, 47, 53, 56, 59, 62, 66, 69, 72, 76, 79, 84, 87, 91, 95, 95, 99, 104, 107, 110, 112, 114, 116, 119, 123, 126, 131, 133, 138, 143, 146, 153, 156, 162, 167, 172, 177, 180, 186, 191, 195, 201, 203, 207, 214, 217, 225, 229, 236, 242, 248, 255, 260, 265, 269, 273, 277, 281, 285, 289, 294, 301, 307, 314, 319, 324, 331, 337, 348, 357, 363, 371, 380, 387, 390, 394, 401, 409, 415, 420, 423, 429, 432, 438, 444, 449, 452, 457, 461, 466, 471, 475], "exact_score": [7.641657905799261, 7.742172034620776, 7.843963096312895, 7.744109685989099, 7.842662080466958, 7.9597917930100115, 9.422900981230727, 9.43722961285429, 9.419944462140146, 9.436462354221488, 9.495084330066264, 9.565611422674506, 9.566009923328783, 9.54514190391733, 9.567961215360752, 9.672768076851018, 9.6564744750109, 9.774656856755884, 9.841946322520846, 10.640491856844955, 10.734526649968949, 10.815748616431252, 10.81952841987999, 11.482218974546338, 12.183496241002535, 12.230686132283244, 12.275557653400648, 13.407337071200336, 13.50920124714541, 13.636265355173215, 13.733106821515973, 13.916445287792536, 14.030978079891124, 14.11917391652251, 14.233693701481648, 14.23950412799679, 14.374610580328179, 14.363544024461808, 14.362059862916784, 15.317920105070392, 15.343126065200485, 15.76130618803151, 15.824887837517021, 15.867356843858992, 16.0287699497436, 16.177032203561108, 16.345024464269915, 16.47033300590613, 16.6742154715381, 16.845723069209306, 17.12606145578243, 17.675677337061092, 17.96637196542062, 17.97322753288932, 18.25345717852033, 18.34137497652514, 18.54710212268352, 19.050878386486946, 19.352143836047244, 19.686734421007284, 19.734941174844124, 19.940855845957312, 20.185840040046042, 20.330728498235974, 20.467522446689113, 20.92717745954965, 21.090958634410033, 21.250369254401512, 21.093862121514753, 21.228382363757582, 20.908492429229756, 21.049766156312995, 20.958811642884374, 21.35373035036712, 21.491875576984707, 21.695164782935986, 21.85600621805136, 21.874374031575037, 21.73202614379085, 21.497215357368617, 21.359437751004016, 21.139786389077404, 21.084742651303294, 21.063886762048313, 21.0315893913479, 20.981842395276058, 20.927805315149254, 21.06303544276124, 20.96044949330227, 20.93946839357991, 20.900564692613756, 20.88034831210798, 20.706452404065136, 20.671881639296352, 20.640697123027866, 20.624117603017737, 20.593886062685232, 20.57334362166725, 20.712927410540143, 20.685604348672708, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.362517794679658, "threshold_area100": 16.35635594780613, "threshold_area25": 9.5596685737875}, "test": {"score": [7.8873978710439285, 7.977590326264251, 8.045603547937551, 8.66648810998744, 10.120880260884496, 10.074760221220252, 10.012398783580348, 9.968822259431004, 9.923863183542696, 10.250822949742236, 11.237745908543596, 11.750016790518627, 11.880146902241323, 11.82449723991083, 11.989909955059517, 11.902835484669456, 12.068246620047201, 12.104520174776212, 12.240400610334902, 12.263107044035749, 12.21632181890609, 12.224673451567508, 12.211631940574827, 12.20595239540814, 12.35185470265831, 12.332895011828485, 12.516357144847865, 12.55395457417929, 12.616642313728832, 12.753087886467906, 13.11707777472973, 13.13762548711721, 13.00997016960872, 13.031502902712887, 13.087230609994558, 13.346789966931091, 13.386854183527827, 13.547866180634152, 13.747809099631732, 13.85562369096028, 14.269011494717033, 14.157662926455497, 14.141775514177581, 14.489523371088357, 14.731169165322733, 14.95373767086216, 15.52582628949716, 15.690863689425546, 16.171413820308274, 16.440376931147025, 16.71393401650107, 16.916934335971426, 17.02269886082678, 17.282014294741316, 18.240896904384183, 18.075134113938134, 18.319995713912814, 18.81098417693608, 19.109508375185573, 19.222357188458883, 19.471133545025037, 19.686422368006156, 19.80276846007319, 20.012330067398022, 20.35000135125783, 20.41039322749031, 20.93746410741351, 21.094056040397653, 21.22432052890532, 21.28213192515058, 21.593960152564488, 21.813480448505846, 21.94129856493714, 22.15495041060595, 22.307505955090846, 22.39085866736224, 22.333753514234683, 22.858577678174857, 22.79083919862158, 22.713577961211033, 22.63072632499173, 22.854096260841175, 22.82344063511782, 22.783843993665005, 22.717905917804522, 22.652633824582225, 22.664545161161325, 22.728161496553724, 22.775561725445925, 22.750935987275923, 22.677455799590255, 22.612242817880638, 22.637966130539873, 22.590480377116677, 22.566345757420272, 22.562049244476555, 22.52085675128784, 22.47445609034009, 22.455696586458764, 22.36242091483517, 22.361115727328066], "calls": [0.0, 0.7989347536617843, 1.3981358189081226, 1.7643142476697737, 3.2290279627163785, 3.4287616511318246, 3.9280958721704398, 4.1278295605858855, 4.627163781624501, 5.326231691078561, 6.990679094540613, 7.822902796271637, 8.821571238348868, 9.720372836218376, 10.352862849533954, 11.018641810918774, 12.316910785619175, 13.548601864181093, 15.179760319573901, 15.945406125166445, 17.243675099866845, 17.942743009320907, 19.174434087882823, 19.37416777629827, 20.10652463382157, 20.73901464713715, 21.904127829560586, 22.303595206391478, 23.435419440745672, 24.833555259653796, 26.864181091877498, 27.130492676431423, 27.996005326231693, 28.428761651131822, 28.86151797603196, 30.22636484687084, 30.792276964047936, 32.290279627163784, 33.95472703062583, 34.12117177097203, 35.885486018641814, 36.8508655126498, 37.88282290279627, 39.513981358189085, 41.24500665778961, 42.27696404793608, 44.6404793608522, 45.63914780292942, 47.17043941411451, 48.63515312916112, 49.96671105193076, 51.231691078561916, 51.69773635153129, 52.529960053262315, 54.12782956058588, 55.026631158455395, 55.758988015978694, 57.39014647137151, 58.18908122503329, 58.988015978695074, 60.05326231691078, 61.085219707057256, 62.017310252996005, 62.949400798934754, 64.0479360852197, 65.14647137150466, 67.27696404793608, 68.47536617842877, 69.20772303595206, 70.47270306258322, 71.53794940079894, 72.60319573901465, 73.26897470039947, 74.13448735019973, 75.03328894806924, 75.29960053262317, 76.53129161118508, 77.82956058588549, 79.39414114513981, 80.6924101198402, 81.52463382157123, 82.15712383488682, 82.55659121171772, 83.92143808255659, 84.95339547270306, 86.18508655126497, 87.48335552596538, 88.04926764314249, 88.38215712383489, 88.94806924101198, 89.61384820239681, 90.21304926764314, 90.94540612516646, 92.14380825565912, 92.64314247669773, 93.57523302263648, 94.20772303595206, 95.23968042609854, 95.93874833555259, 98.30226364846871, 100.0], "rescued": [0, 60, 104, 131, 242, 255, 289, 305, 343, 394, 518, 572, 637, 685, 724, 762, 833, 904, 988, 1022, 1088, 1124, 1189, 1198, 1231, 1262, 1319, 1338, 1389, 1452, 1543, 1556, 1597, 1617, 1639, 1693, 1719, 1787, 1861, 1869, 1959, 2003, 2046, 2121, 2194, 2240, 2343, 2385, 2443, 2493, 2539, 2577, 2593, 2619, 2676, 2701, 2726, 2777, 2803, 2824, 2853, 2883, 2905, 2928, 2960, 2985, 3041, 3071, 3088, 3116, 3140, 3164, 3181, 3202, 3222, 3229, 3259, 3288, 3304, 3305, 3305, 3306, 3306, 3307, 3307, 3307, 3308, 3310, 3311, 3311, 3311, 3311, 3312, 3312, 3312, 3313, 3313, 3313, 3313, 3313, 3314], "harmed": [0, 10, 21, 25, 44, 48, 59, 64, 75, 86, 118, 129, 145, 158, 169, 179, 202, 219, 247, 260, 283, 296, 318, 322, 333, 345, 364, 368, 384, 411, 446, 452, 472, 476, 480, 502, 512, 534, 569, 570, 605, 627, 645, 673, 701, 717, 759, 777, 806, 829, 855, 886, 892, 905, 935, 959, 974, 1008, 1024, 1040, 1065, 1089, 1110, 1126, 1144, 1169, 1212, 1246, 1259, 1294, 1313, 1334, 1349, 1368, 1387, 1392, 1417, 1446, 1483, 1515, 1533, 1547, 1559, 1588, 1614, 1646, 1674, 1686, 1693, 1707, 1723, 1734, 1749, 1778, 1793, 1810, 1826, 1857, 1871, 1917, 1945], "exact_score": [7.8873978710439285, 8.00134517069259, 9.74331841730725, 9.669567329684673, 9.993807665190655, 10.28560102433297, 10.74059450816603, 11.237745908543596, 11.738942622408597, 11.880983544142223, 11.81841734842225, 11.899593213433516, 11.855724948619052, 12.10922740452563, 12.105825221821592, 12.268804903861133, 12.269172782109024, 12.250946685325165, 12.221032249114085, 12.243015178325747, 12.378982783426624, 12.348340564436183, 12.52112038173233, 12.578858257944253, 12.735740712498526, 12.879916119712686, 12.938649121876708, 13.12899276211515, 13.00997016960872, 13.078696002515686, 13.317826459987423, 13.371501712287731, 13.53962384615449, 13.646240087809069, 13.749062505947721, 14.073129506206033, 14.259097537030216, 14.176953951636223, 14.132690488836271, 14.444043686133167, 14.551256560315256, 14.698122874242644, 14.910913101188228, 15.089224886638622, 15.371480598521092, 15.551145496231571, 15.827303306162154, 16.079473064098455, 16.34284225107677, 16.482045130268734, 16.721095601414458, 16.85392393950293, 17.171791619264376, 17.677814390474992, 18.215387758034552, 18.06805136217641, 18.346660407990754, 18.75726068326891, 18.996347886216714, 19.222357188458883, 19.45929771724433, 19.661661289714186, 19.843055864016467, 20.0232177201342, 20.34359182978087, 20.44200300337484, 20.729022996870537, 20.923515141300328, 21.094560691790328, 21.073300854730544, 21.211838300237048, 21.476904693904856, 21.68152899053849, 21.887548751376755, 22.09888330541147, 22.294693834945548, 22.612277465670665, 22.484551144076036, 22.93677930397971, 22.993133826371604, 22.703780274706855, 22.65537249669406, 22.519230782941875, 22.791032990757902, 22.78014275245711, 22.715862553063918, 22.660696730062284, 22.68445873805727, 22.748084027785378, 22.750935987275923, 22.6171126926577, 22.63649733201706, 22.598356954638536, 22.555473354844395, 22.547198654742793, 22.487877011192296, 22.453979435555546, 22.408303897316994, 22.366500121387904, 22.344098842915987, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 17.081859319571592, "threshold_area100": 16.869760993015987, "threshold_area25": 11.108361974955121}, "selected_budget": 77, "selected": {"val": {"score": 21.874374031575037, "calls": 76.8901569186876, "rescued": 783, "harmed": 348}, "test": {"score": 22.858577678174857, "calls": 77.82956058588549, "rescued": 3288, "harmed": 1446}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 21.708418052382868, "calls": 84.87874465049929}, "test": {"score": 23.03142048126501, "calls": 85.38615179760319}}, "old_normalized_interior": {"val": {"score": 21.708418052382868, "calls": 84.87874465049929}, "test": {"score": 23.03142048126501, "calls": 85.35286284953395}}, "old_normalized_endpoints": {"val": {"score": 21.708418052382868, "calls": 84.87874465049929}, "test": {"score": 23.03142048126501, "calls": 85.35286284953395}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 7.737353307510221, 7.756224406001378, 9.944316315466638, 9.778788716961259, 9.595010105491472, 11.000227624510606, 10.763130009996152, 10.611558425986619, 10.223604781049698, 10.290188894642354, 10.242877941150654, 11.154408303601276, 11.097481198192098, 11.097298291065957, 11.04392498240787, 11.130063310590693, 11.11811425879846, 11.809979374691567, 11.861077575294114, 11.920119981370163, 11.984556935783527, 12.08272713073187, 12.12340755511244, 12.606983277102266, 12.586918581759779, 12.676687449173219, 12.770428729844753, 13.31431985683636, 13.337671297179588, 13.409981130985708, 13.488216588903088, 13.592359829423858, 13.65539424494687, 13.648026094689051, 14.07104753111933, 14.16580170357207, 14.140593895584821, 14.299272047527692, 14.27765931381352, 14.397946201381314, 14.556415202308411, 14.70090329747309, 14.847727819513205, 15.046619666292283, 15.11299091526602, 15.316443989469676, 15.87679745897686, 15.828170265214005, 16.26954550141176, 16.43426080113669, 16.55094373845748, 16.535774160965502, 17.077300152275896, 17.223239723239725, 17.454200202441772, 17.401569861121516, 17.37898124181254, 17.421621936884797, 17.590392529744822, 17.570617184076585, 17.784148391446045, 18.010907308913744, 18.22635531926916, 18.317946302402262, 18.601256467110126, 18.842439598404926, 18.515796582886733, 18.86731397162082, 18.890608194844393, 18.87496033421538, 19.21395139212444, 19.445181646614767, 19.5551265442854, 19.888080881670625, 20.442279942279942, 20.74892292532863, 21.33873735187056, 21.66212002377447, 21.70757217387469, 21.694359952569986, 21.655840612249587, 21.612206811593314, 21.585034006642733, 21.554995115726154, 21.708418052382868, 21.69321001891231, 21.588728582987113, 21.539454056728733, 21.428165584415584, 21.382206902008882, 21.351006080585204, 21.25134632591645, 21.207317230398317, 21.173363715418727, 21.127758846818757, 21.01192552401413, 21.07119830524086, 20.941404217289232, 20.90250929513203, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 7.737353307510221, 7.756224406001378, 9.944316315466638, 9.778788716961259, 9.595010105491472, 11.000227624510606, 10.763130009996152, 10.611558425986619, 10.223604781049698, 10.290188894642354, 10.242877941150654, 11.154408303601276, 11.097481198192098, 11.097298291065957, 11.04392498240787, 11.130063310590693, 11.11811425879846, 11.809979374691567, 11.861077575294114, 11.920119981370163, 11.984556935783527, 12.08272713073187, 12.12340755511244, 12.606983277102266, 12.586918581759779, 12.676687449173219, 12.770428729844753, 13.31431985683636, 13.337671297179588, 13.409981130985708, 13.488216588903088, 13.592359829423858, 13.65539424494687, 13.648026094689051, 14.07104753111933, 14.16580170357207, 14.140593895584821, 14.299272047527692, 14.27765931381352, 14.397946201381314, 14.556415202308411, 14.70090329747309, 14.847727819513205, 15.046619666292283, 15.11299091526602, 15.316443989469676, 15.87679745897686, 15.828170265214005, 16.26954550141176, 16.43426080113669, 16.55094373845748, 16.535774160965502, 17.077300152275896, 17.223239723239725, 17.454200202441772, 17.401569861121516, 17.37898124181254, 17.421621936884797, 17.590392529744822, 17.570617184076585, 17.784148391446045, 18.010907308913744, 18.22635531926916, 18.317946302402262, 18.601256467110126, 18.842439598404926, 18.515796582886733, 18.86731397162082, 18.890608194844393, 18.87496033421538, 19.21395139212444, 19.445181646614767, 19.5551265442854, 19.888080881670625, 20.442279942279942, 20.74892292532863, 21.33873735187056, 21.66212002377447, 21.70757217387469, 21.694359952569986, 21.655840612249587, 21.612206811593314, 21.585034006642733, 21.554995115726154, 21.708418052382868, 21.69321001891231, 21.588728582987113, 21.539454056728733, 21.428165584415584, 21.382206902008882, 21.351006080585204, 21.25134632591645, 21.207317230398317, 21.173363715418727, 21.127758846818757, 21.01192552401413, 21.07119830524086, 20.941404217289232, 20.90250929513203, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 10.592706441538276, "exact_area100": 16.142313904194708}, "test": {"score": [7.8873978710439285, 7.919770837804603, 7.9835065470219835, 8.040857950952391, 8.57634232586485, 9.034700586585554, 9.362249371879692, 9.691778907835287, 9.91536927322557, 10.155017041085545, 10.137557253024736, 10.341814225321112, 10.71971640126264, 10.93637751492443, 11.339546089704763, 11.486677666803976, 11.486603925702193, 11.488337452417788, 11.53734372013113, 11.96506438514031, 12.174367095897658, 12.196217330926359, 12.350010033666878, 12.359161003371582, 12.534103815746326, 12.565273130106064, 12.609860748804646, 12.615425136944955, 12.766467479736065, 12.800804123134368, 12.82342703692608, 13.23162084538405, 13.233348830054492, 13.274176244887128, 13.37816987179453, 13.613904302804183, 13.818278825117005, 13.96523442601929, 14.073510196492386, 14.17042565363866, 14.462427627496172, 14.675264966708323, 15.003903405701177, 15.302932303671174, 15.53756087494522, 15.89323510211123, 16.07892604523862, 16.456471197981017, 16.59459151179672, 16.74813370743175, 17.006080901477212, 17.094856136291337, 17.17226353692794, 17.1819289554308, 17.424660340573357, 17.54786164954468, 17.938582767479737, 17.98519160248288, 18.237913787342183, 18.28357154210935, 18.427312821486026, 18.5777005856292, 18.230197408709387, 18.288451391987802, 18.377239731466812, 18.50824792032102, 18.529351385694433, 18.85275355823716, 19.133942178191376, 19.308572120708746, 19.722513761490696, 20.35157025673237, 20.752659813175054, 21.05753253604506, 21.487122456855605, 21.72048932527819, 22.176084400474725, 23.029296395727123, 23.221786909101656, 23.62233070712321, 23.228897486545364, 23.197482939476668, 23.117537211282855, 23.08950580894242, 23.04702230807231, 23.03142048126501, 22.821570978477347, 22.708556124329995, 22.68093498022217, 22.684514761990027, 22.62891988844286, 22.630343784445106, 22.544400478643553, 22.500066964351554, 22.434791580681747, 22.399574169812286, 22.4140721037617, 22.354956306872605, 22.325609833727672, 22.375446585202017, 22.361115727328066], "calls": [0.0, 0.4660452729693742, 1.3315579227696404, 1.7976031957390146, 2.9960053262316912, 4.061251664447404, 5.0599201065246335, 5.89214380825566, 7.057256990679095, 7.723035952063914, 8.488681757656458, 8.988015978695072, 10.186418109187748, 11.917443408788282, 13.781624500665778, 14.347536617842877, 15.14647137150466, 15.645805592543274, 16.977363515312916, 18.541944074567244, 19.507323568575234, 21.13848202396804, 21.271637816245008, 21.471371504660453, 22.270306258322236, 23.468708388814914, 24.400798934753663, 25.532623169107854, 25.898801597869507, 26.564580559254324, 27.06391478029294, 27.929427430093206, 28.062583222370176, 28.828229027962713, 30.326231691078565, 31.391478029294273, 33.15579227696405, 34.0878828229028, 35.319573901464715, 36.31824234354195, 37.48335552596538, 39.41411451398136, 40.67909454061252, 42.50998668442077, 44.14114513981358, 45.23968042609853, 46.63781624500666, 48.33555259653795, 48.968042609853526, 50.0, 51.73102529960053, 52.529960053262315, 53.12916111850865, 53.99467376830892, 54.72703062583223, 55.39280958721704, 56.724367509986685, 57.35685752330226, 58.62183754993342, 59.15446071904128, 60.38615179760319, 61.21837549933422, 62.18375499334221, 63.01597869507324, 63.948069241011986, 64.84687083888149, 64.94673768308921, 66.81091877496671, 68.00932090545939, 68.70838881491345, 70.30625832223701, 72.30359520639148, 73.96804260985353, 74.6005326231691, 75.69906790945407, 76.16511318242344, 77.1970705725699, 78.69507323568575, 79.02796271637817, 80.09320905459387, 81.05858854860186, 81.85752330226364, 82.82290279627163, 83.6884154460719, 84.62050599201065, 85.35286284953395, 86.55126498002663, 87.51664447403462, 88.3155792276964, 89.74700399467378, 90.64580559254327, 91.51131824234355, 93.07589880159787, 94.14114513981359, 94.9733688415446, 95.80559254327564, 96.6711051930759, 97.56990679094541, 98.20239680426099, 99.16777629826898, 100.0], "exact_score": [7.8873978710439285, 7.949565575047178, 8.044332402387862, 8.57634232586485, 8.616711236879567, 9.354403276430766, 9.69575188216895, 9.918107267860908, 10.159625060690871, 10.341814225321112, 10.748586143516516, 10.888878543648504, 10.93552511864119, 10.932416497890587, 11.352760439981536, 11.468090752451058, 11.491803343599734, 11.53734372013113, 11.817103378580175, 12.000030416218193, 12.174737133170233, 12.19945777220974, 12.537337448193298, 12.532224989931851, 12.614988923034412, 12.614622422040853, 12.77338634730719, 12.822673588838585, 13.23502303789593, 13.280686388538582, 13.36125958213692, 13.51299458125355, 13.620957673649354, 13.831100893057185, 13.96248864326516, 14.092899009386297, 14.123472370009704, 14.316508358970168, 14.508523287871919, 14.615849722299796, 14.866289558489337, 15.117122550478202, 15.154927786841677, 15.461239576276942, 15.508016807178052, 15.883815909553729, 15.96542238479754, 16.20759961318833, 16.385594146524443, 16.59459151179672, 16.74813370743175, 16.956988454041984, 17.063061220924837, 17.15728972659586, 17.1819289554308, 17.490941574018837, 17.62327235278405, 17.946045399730263, 18.113199960811535, 18.294681353150548, 18.362102485118147, 18.544962304877423, 18.46100168777977, 18.280415447303945, 18.44341610265027, 18.538099687067984, 18.67735084464036, 18.891013319042322, 19.122081251976958, 19.29658277697009, 19.5645324544947, 19.983375003653514, 20.248563458111445, 20.635518046366037, 20.752659813175054, 21.231791026173457, 21.64126092438877, 22.091738207582974, 22.72128445656449, 23.221786909101656, 23.624026372645154, 23.59789795848618, 23.195870189585317, 23.110623870956374, 23.078446325231088, 23.04251864351157, 22.889911146287055, 22.79129883447203, 22.744572896169327, 22.65842929029408, 22.67487715245297, 22.62152272518186, 22.619018585676336, 22.54820925762519, 22.506950072470943, 22.434791580681747, 22.390298044871315, 22.386566697842472, 22.336724690726356, 22.331640307273496, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 10.611927620577385, "exact_area100": 16.639008252693614}}, "old_normalized_endpoint_budget": 85}, {"seed": 2030, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.0, "validation_area100": 0.1642143647341603, "training_seconds": 4.194330655038357, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.576108455657959, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5528685450553894, "operator": "ge"}, {"budget": 0.03, "threshold": 0.539177417755127, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5201751589775085, "operator": "ge"}, {"budget": 0.05, "threshold": 0.510645866394043, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5004016757011414, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4834264814853668, "operator": "ge"}, {"budget": 0.08, "threshold": 0.46981096267700195, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4583054184913635, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4280409812927246, "operator": "ge"}, {"budget": 0.11, "threshold": 0.4100438952445984, "operator": "ge"}, {"budget": 0.12, "threshold": 0.39387038350105286, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3819783926010132, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3571593463420868, "operator": "ge"}, {"budget": 0.15, "threshold": 0.33426782488822937, "operator": "ge"}, {"budget": 0.16, "threshold": 0.32629823684692383, "operator": "ge"}, {"budget": 0.17, "threshold": 0.30695950984954834, "operator": "ge"}, {"budget": 0.18, "threshold": 0.29336801171302795, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2891826033592224, "operator": "ge"}, {"budget": 0.2, "threshold": 0.2797831594944, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2752266824245453, "operator": "ge"}, {"budget": 0.22, "threshold": 0.26880329847335815, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2641579806804657, "operator": "ge"}, {"budget": 0.24, "threshold": 0.25728997588157654, "operator": "ge"}, {"budget": 0.25, "threshold": 0.25426700711250305, "operator": "ge"}, {"budget": 0.26, "threshold": 0.25123634934425354, "operator": "ge"}, {"budget": 0.27, "threshold": 0.24920129776000977, "operator": "ge"}, {"budget": 0.28, "threshold": 0.24501262605190277, "operator": "ge"}, {"budget": 0.29, "threshold": 0.2406313121318817, "operator": "ge"}, {"budget": 0.3, "threshold": 0.23822909593582153, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2356654703617096, "operator": "ge"}, {"budget": 0.32, "threshold": 0.2340231090784073, "operator": "ge"}, {"budget": 0.33, "threshold": 0.23157188296318054, "operator": "ge"}, {"budget": 0.34, "threshold": 0.2283622920513153, "operator": "ge"}, {"budget": 0.35, "threshold": 0.22618381679058075, "operator": "ge"}, {"budget": 0.36, "threshold": 0.222722589969635, "operator": "ge"}, {"budget": 0.37, "threshold": 0.21798686683177948, "operator": "ge"}, {"budget": 0.38, "threshold": 0.2153925746679306, "operator": "ge"}, {"budget": 0.39, "threshold": 0.21063289046287537, "operator": "ge"}, {"budget": 0.4, "threshold": 0.20717036724090576, "operator": "ge"}, {"budget": 0.41, "threshold": 0.20286978781223297, "operator": "ge"}, {"budget": 0.42, "threshold": 0.1930568665266037, "operator": "ge"}, {"budget": 0.43, "threshold": 0.1820988953113556, "operator": "ge"}, {"budget": 0.44, "threshold": 0.1743784099817276, "operator": "ge"}, {"budget": 0.45, "threshold": 0.16468122601509094, "operator": "ge"}, {"budget": 0.46, "threshold": 0.14870384335517883, "operator": "ge"}, {"budget": 0.47, "threshold": 0.1427670121192932, "operator": "ge"}, {"budget": 0.48, "threshold": 0.13779586553573608, "operator": "ge"}, {"budget": 0.49, "threshold": 0.12866023182868958, "operator": "ge"}, {"budget": 0.5, "threshold": 0.12540848553180695, "operator": "ge"}, {"budget": 0.51, "threshold": 0.11182671785354614, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1032901406288147, "operator": "ge"}, {"budget": 0.53, "threshold": 0.09803740680217743, "operator": "ge"}, {"budget": 0.54, "threshold": 0.09156489372253418, "operator": "ge"}, {"budget": 0.55, "threshold": 0.08725692331790924, "operator": "ge"}, {"budget": 0.56, "threshold": 0.08025092631578445, "operator": "ge"}, {"budget": 0.57, "threshold": 0.07360437512397766, "operator": "ge"}, {"budget": 0.58, "threshold": 0.06775113940238953, "operator": "ge"}, {"budget": 0.59, "threshold": 0.06454424560070038, "operator": "ge"}, {"budget": 0.6, "threshold": 0.06216925382614136, "operator": "ge"}, {"budget": 0.61, "threshold": 0.05872151255607605, "operator": "ge"}, {"budget": 0.62, "threshold": 0.0556211918592453, "operator": "ge"}, {"budget": 0.63, "threshold": 0.05247463285923004, "operator": "ge"}, {"budget": 0.64, "threshold": 0.04676137864589691, "operator": "ge"}, {"budget": 0.65, "threshold": 0.040239766240119934, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03528057038784027, "operator": "ge"}, {"budget": 0.67, "threshold": 0.030192211270332336, "operator": "ge"}, {"budget": 0.68, "threshold": 0.02709519863128662, "operator": "ge"}, {"budget": 0.69, "threshold": 0.023363560438156128, "operator": "ge"}, {"budget": 0.7, "threshold": 0.019998878240585327, "operator": "ge"}, {"budget": 0.71, "threshold": 0.018134072422981262, "operator": "ge"}, {"budget": 0.72, "threshold": 0.01392248272895813, "operator": "ge"}, {"budget": 0.73, "threshold": 0.0048241764307022095, "operator": "ge"}, {"budget": 0.74, "threshold": -0.000997781753540039, "operator": "ge"}, {"budget": 0.75, "threshold": -0.012595951557159424, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0217428058385849, "operator": "ge"}, {"budget": 0.77, "threshold": -0.06022007763385773, "operator": "ge"}, {"budget": 0.78, "threshold": -0.0899617075920105, "operator": "ge"}, {"budget": 0.79, "threshold": -0.09787754714488983, "operator": "ge"}, {"budget": 0.8, "threshold": -0.10156916081905365, "operator": "ge"}, {"budget": 0.81, "threshold": -0.10831496864557266, "operator": "ge"}, {"budget": 0.82, "threshold": -0.11247636377811432, "operator": "ge"}, {"budget": 0.83, "threshold": -0.1164432018995285, "operator": "ge"}, {"budget": 0.84, "threshold": -0.12411130219697952, "operator": "ge"}, {"budget": 0.85, "threshold": -0.12993937730789185, "operator": "ge"}, {"budget": 0.86, "threshold": -0.13692553341388702, "operator": "ge"}, {"budget": 0.87, "threshold": -0.1413743942975998, "operator": "ge"}, {"budget": 0.88, "threshold": -0.14720480144023895, "operator": "ge"}, {"budget": 0.89, "threshold": -0.15211032330989838, "operator": "ge"}, {"budget": 0.9, "threshold": -0.1550770103931427, "operator": "ge"}, {"budget": 0.91, "threshold": -0.15806519985198975, "operator": "ge"}, {"budget": 0.92, "threshold": -0.1626899689435959, "operator": "ge"}, {"budget": 0.93, "threshold": -0.1656728982925415, "operator": "ge"}, {"budget": 0.94, "threshold": -0.16759361326694489, "operator": "ge"}, {"budget": 0.95, "threshold": -0.17026422917842865, "operator": "ge"}, {"budget": 0.96, "threshold": -0.17283079028129578, "operator": "ge"}, {"budget": 0.97, "threshold": -0.17714357376098633, "operator": "ge"}, {"budget": 0.98, "threshold": -0.17988504469394684, "operator": "ge"}, {"budget": 0.99, "threshold": -0.1859617680311203, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.762403481598755, 7.6425767215240885, 7.761904566672986, 7.852887515359125, 7.980358151165605, 8.08777610662656, 9.249971914989553, 9.328133881625691, 9.372610146090429, 9.359700290028968, 9.44493508609137, 9.447807157470175, 9.547012293340323, 9.664319478043929, 9.775746779149069, 10.503332002347376, 10.573291660953679, 10.660715692528957, 10.760607819149492, 11.535203799646816, 12.200385555714654, 12.680415840777407, 12.77131661858339, 12.686245194820644, 12.73817908198597, 12.86320908371725, 12.885200324863899, 12.960527851193914, 13.002958006000592, 13.122642004748819, 13.239488567146454, 13.771452936787124, 13.901071900945208, 14.71329752265151, 14.710135181514241, 14.74879240556266, 15.269908048454717, 15.364290278676826, 15.629162107719422, 15.736552202069443, 15.870587388854293, 16.137840239306993, 16.25597165280802, 16.27088046308566, 16.312753107107948, 16.452590960136806, 16.61007574991981, 16.891819672844356, 17.099048663998275, 17.344665138782787, 17.504930063652274, 17.581378030294438, 17.81338238978377, 17.799108975938243, 17.958858258519648, 18.045031873393874, 18.159066563020463, 18.336101882613512, 18.589154411764707, 18.906238008626115, 19.167566191570657, 19.609647212543553, 19.754143836850304, 20.05439705019537, 19.915526652169515, 19.871845169020702, 20.273561832398364, 20.3200120800311, 20.570047533290406, 20.721722027020885, 20.642518375667915, 20.981448793049502, 21.35677123026902, 21.690966124583046, 21.422371282936794, 21.651679180490078, 21.888546869331478, 21.919506151400395, 21.68552681398791, 21.66273100004854, 21.829538336232428, 21.788459130702776, 21.77063911288276, 21.749121546786384, 21.71813433687568, 21.676847870016758, 21.63371099942594, 21.58656799857909, 21.382552702628864, 21.34479896053917, 21.23154179518115, 21.03872114899431, 21.00791212937117, 20.968685699122396, 20.923660623943796, 20.895212823325718, 20.79158631832153, 20.750143565972355, 20.87419239066582, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 36, 56, 74, 94, 112, 131, 144, 165, 181, 197, 210, 224, 237, 248, 262, 272, 283, 297, 312, 326, 339, 350, 362, 375, 388, 396, 404, 417, 430, 441, 453, 466, 479, 490, 501, 513, 524, 537, 546, 555, 565, 573, 580, 588, 595, 603, 611, 619, 626, 631, 638, 645, 651, 657, 662, 668, 673, 679, 685, 690, 696, 701, 708, 712, 716, 722, 729, 736, 743, 748, 755, 762, 769, 772, 779, 783, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786], "harmed": [0, 7, 13, 18, 22, 29, 34, 36, 40, 46, 52, 56, 58, 62, 63, 67, 70, 74, 77, 83, 85, 89, 94, 96, 103, 108, 111, 116, 117, 124, 127, 132, 136, 141, 145, 149, 153, 155, 160, 163, 166, 169, 174, 176, 179, 183, 187, 191, 194, 199, 205, 207, 210, 214, 217, 223, 227, 230, 235, 239, 243, 247, 251, 254, 261, 269, 274, 278, 285, 291, 295, 303, 309, 315, 322, 332, 339, 346, 352, 357, 360, 364, 370, 373, 380, 384, 390, 395, 401, 405, 411, 416, 423, 428, 435, 442, 447, 453, 459, 465, 475], "exact_score": [7.641657905799261, 7.762403481598755, 7.6425767215240885, 7.761904566672986, 7.852887515359125, 7.980358151165605, 8.08777610662656, 9.249971914989553, 9.328133881625691, 9.372610146090429, 9.359700290028968, 9.44493508609137, 9.447807157470175, 9.547012293340323, 9.664319478043929, 9.775746779149069, 10.503332002347376, 10.573291660953679, 10.660715692528957, 10.760607819149492, 11.535203799646816, 12.200385555714654, 12.680415840777407, 12.77131661858339, 12.686245194820644, 12.73817908198597, 12.86320908371725, 12.885200324863899, 12.960527851193914, 13.002958006000592, 13.122642004748819, 13.239488567146454, 13.771452936787124, 13.901071900945208, 14.71329752265151, 14.710135181514241, 14.74879240556266, 15.269908048454717, 15.364290278676826, 15.629162107719422, 15.736552202069443, 15.870587388854293, 16.137840239306993, 16.25597165280802, 16.27088046308566, 16.312753107107948, 16.452590960136806, 16.61007574991981, 16.891819672844356, 17.099048663998275, 17.344665138782787, 17.504930063652274, 17.581378030294438, 17.81338238978377, 17.799108975938243, 17.958858258519648, 18.045031873393874, 18.159066563020463, 18.336101882613512, 18.589154411764707, 18.906238008626115, 19.167566191570657, 19.609647212543553, 19.754143836850304, 20.05439705019537, 19.915526652169515, 19.871845169020702, 20.273561832398364, 20.3200120800311, 20.570047533290406, 20.721722027020885, 20.642518375667915, 20.981448793049502, 21.35677123026902, 21.690966124583046, 21.422371282936794, 21.651679180490078, 21.888546869331478, 21.919506151400395, 21.68552681398791, 21.66273100004854, 21.829538336232428, 21.788459130702776, 21.77063911288276, 21.749121546786384, 21.71813433687568, 21.676847870016758, 21.63371099942594, 21.58656799857909, 21.382552702628864, 21.34479896053917, 21.23154179518115, 21.03872114899431, 21.00791212937117, 20.968685699122396, 20.923660623943796, 20.895212823325718, 20.79158631832153, 20.750143565972355, 20.87419239066582, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.42143647341603, "threshold_area100": 16.415224090339276, "threshold_area25": 9.768185502776614}, "test": {"score": [7.8873978710439285, 7.9674789436895335, 8.01620106522597, 8.618067784533892, 8.625079837069949, 8.627597603938234, 8.637952342331998, 9.361748552957192, 10.170380473076063, 10.687030919035138, 11.035271549897569, 11.26353574412672, 11.224987423543096, 11.424457645147502, 11.60931411930997, 11.518916393977996, 11.693191351581204, 11.736409290229558, 11.91168869524431, 11.949869614841756, 11.985332951135348, 12.114890663474682, 12.34640436600841, 12.40501790327925, 12.634033999952287, 12.63970434862329, 12.65109400600474, 13.050578721779202, 13.423015058167348, 13.512972539653004, 13.670992886010252, 13.871630360516788, 13.999726360678094, 14.081341571156461, 14.29336393371458, 14.458989316144363, 14.52136793285716, 14.663907742060953, 14.809596788665088, 14.986480392615622, 15.121011152007387, 15.255231930306717, 15.614666062591315, 15.624200654815985, 15.773441990070591, 15.861050345182234, 16.097219629652685, 16.220570302258842, 16.342094155852497, 16.59873460857322, 16.784121147068305, 16.96808680755657, 17.10492821233932, 17.147608400909895, 17.08255362527783, 17.42279641167602, 17.857162228163048, 17.991321968103836, 18.137030662776766, 18.59619135160869, 18.67111357813719, 18.768587452466072, 18.76779838249633, 18.729613511150138, 18.66511416881047, 19.139054981710995, 19.476439010043332, 19.76680147414496, 20.047062772845443, 20.231700499476215, 20.42132885759581, 20.58143463660705, 21.17106326042561, 21.613103750401912, 21.92582387021028, 22.32140869182026, 22.416983781133876, 22.908806045820917, 22.998639928840948, 23.305518769048163, 23.289254498090013, 23.255112505203705, 23.092548706579407, 22.968803933516487, 22.831627985646847, 22.807750237867292, 22.76676589998253, 22.68343753545973, 22.645769820159998, 22.616302510549815, 22.591403167830006, 22.484517933444128, 22.488759345530106, 22.426442916495553, 22.396900401374964, 22.391297780029365, 22.38595153670027, 22.382816839982635, 22.355127903766068, 22.286554709594395, 22.361115727328066], "calls": [0.0, 0.6990679094540613, 1.0985352862849533, 1.3648468708388815, 1.9973368841544608, 2.2969374167776295, 2.762982689747004, 3.9280958721704398, 5.126498002663116, 5.692410119840213, 6.790945406125166, 7.523302263648468, 8.288948069241012, 8.921438082556591, 9.986684420772303, 11.418109187749668, 11.950732356857523, 13.548601864181093, 15.246338215712383, 15.81225033288948, 17.14380825565912, 18.209054593874836, 19.37416777629827, 20.372836218375497, 22.37017310252996, 22.902796271637815, 23.801597869507322, 24.93342210386152, 26.364846870838882, 27.796271637816243, 28.56191744340879, 29.793608521970704, 30.62583222370173, 31.158455392809586, 32.42343541944074, 33.18908122503329, 34.187749667110516, 35.61917443408788, 36.684420772303596, 38.01597869507324, 38.78162450066578, 39.78029294274301, 41.54460719041278, 43.142476697736356, 44.241011984021306, 44.87350199733689, 46.57123834886818, 47.20372836218375, 47.86950732356858, 49.06790945406125, 49.63382157123835, 51.564580559254324, 52.762982689747005, 53.59520639147804, 54.593874833555255, 55.32623169107856, 56.757656458055926, 57.75632490013316, 59.32090545938748, 59.986684420772306, 60.719041278295606, 61.51797603195739, 62.18375499334221, 62.949400798934754, 64.41411451398136, 65.87882822902796, 67.57656458055925, 68.87483355525966, 70.07323568575234, 70.77230359520638, 71.30492676431425, 71.63781624500666, 72.76964047936086, 74.06790945406125, 74.80026631158455, 76.19840213049268, 76.8641810918775, 78.06258322237018, 79.26098535286285, 80.39280958721704, 81.15845539280959, 82.32356857523303, 83.18908122503329, 83.95472703062583, 85.41944074567243, 86.18508655126497, 87.01731025299601, 87.48335552596538, 88.18242343541944, 89.14780292942743, 89.81358189081226, 90.5126498002663, 91.97736351531292, 93.20905459387482, 93.87483355525966, 95.0399467376831, 95.87217043941412, 97.47003994673769, 98.00266311584554, 98.8681757656458, 100.0], "rescued": [0, 58, 92, 117, 168, 192, 229, 326, 419, 461, 545, 593, 642, 678, 739, 824, 855, 941, 1035, 1065, 1132, 1186, 1248, 1302, 1405, 1433, 1478, 1531, 1603, 1672, 1709, 1767, 1809, 1833, 1892, 1931, 1975, 2035, 2080, 2137, 2171, 2214, 2290, 2352, 2396, 2418, 2470, 2489, 2505, 2538, 2555, 2607, 2642, 2660, 2686, 2703, 2741, 2765, 2803, 2821, 2837, 2857, 2870, 2889, 2925, 2967, 3011, 3046, 3079, 3098, 3112, 3123, 3154, 3191, 3212, 3248, 3262, 3294, 3301, 3303, 3304, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3307, 3307, 3308, 3308, 3308, 3309, 3310, 3311, 3311, 3311, 3314], "harmed": [0, 12, 20, 24, 36, 43, 53, 81, 104, 112, 130, 139, 153, 162, 178, 208, 218, 243, 274, 282, 307, 332, 353, 373, 413, 425, 442, 459, 484, 513, 527, 552, 568, 580, 601, 612, 629, 649, 661, 682, 695, 705, 731, 751, 766, 775, 795, 807, 817, 842, 853, 894, 914, 933, 945, 955, 975, 996, 1022, 1034, 1046, 1062, 1076, 1095, 1129, 1155, 1199, 1228, 1254, 1266, 1281, 1287, 1314, 1342, 1361, 1402, 1424, 1451, 1468, 1486, 1503, 1530, 1545, 1563, 1594, 1608, 1623, 1634, 1652, 1671, 1686, 1703, 1735, 1764, 1783, 1812, 1831, 1872, 1889, 1916, 1945], "exact_score": [7.8873978710439285, 8.004251529441989, 8.625079837069949, 9.064936860691834, 9.364048252509479, 10.163101355112408, 10.652920718544273, 11.048840548631906, 11.232796898357062, 11.642280916131496, 11.60931411930997, 11.520208698996372, 11.695165295521196, 11.74970258174281, 11.918122929039376, 11.893289696654339, 11.967525058102355, 11.971742378719123, 12.109032913516682, 12.303932611928074, 12.377863211418093, 12.437559505956129, 12.5921844312102, 12.641645325668001, 12.67874208795629, 13.055785723608329, 13.315609564132815, 13.486348594819395, 13.623191511810399, 13.761002220548727, 13.871547504835238, 14.01206062784905, 14.154570702521166, 14.439176262228642, 14.50141866848319, 14.644538813589158, 14.71295064760374, 14.845104935929953, 14.980929497071141, 15.150116203989702, 15.262388130024679, 15.481691757721281, 15.623620519443666, 15.589860393004637, 15.73721155438183, 15.840450028772413, 15.965168684012243, 16.182882550028197, 16.387579358754596, 16.56657595834847, 16.806786571008413, 16.914396119285463, 16.974442517737383, 17.160075746964406, 17.227221087032568, 17.42889980452718, 17.77688210817734, 17.845579260198566, 18.02939921964829, 18.131076036396507, 18.59619135160869, 18.694882605681943, 18.784145110463477, 18.736167320274276, 18.91639373673328, 18.902761256881874, 19.164602555163995, 19.32125703122826, 19.549365110320956, 19.7938730605101, 20.01718887017843, 20.27064745579872, 20.698930885908833, 21.215484755819404, 21.575559323753374, 21.98425946059008, 22.29771366240339, 22.47593257568422, 22.87784966712277, 22.985030619450253, 22.93882113574609, 23.276513102125655, 23.24931965391462, 23.097920839312902, 22.968803933516487, 22.86525232731217, 22.814264089933754, 22.76676589998253, 22.610215319205114, 22.621051491437537, 22.566689832584792, 22.468813124999336, 22.488759345530106, 22.452647409074423, 22.39503318920572, 22.39250935493135, 22.384114470448313, 22.39238702722077, 22.35805983753285, 22.28368392969821, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.92323722025531, "threshold_area100": 16.799984432292216, "threshold_area25": 10.686133921756552}, "selected_budget": 78, "selected": {"val": {"score": 21.919506151400395, "calls": 77.88873038516405, "rescued": 784, "harmed": 352}, "test": {"score": 22.998639928840948, "calls": 79.26098535286285, "rescued": 3301, "harmed": 1468}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.576108455657959, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5528685450553894, "operator": "ge"}, {"budget": 0.03, "threshold": 0.539177417755127, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5201751589775085, "operator": "ge"}, {"budget": 0.05, "threshold": 0.510645866394043, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5004016757011414, "operator": "ge"}, {"budget": 0.07, "threshold": 0.4834264814853668, "operator": "ge"}, {"budget": 0.08, "threshold": 0.46981096267700195, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4583054184913635, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4280409812927246, "operator": "ge"}, {"budget": 0.11, "threshold": 0.4100438952445984, "operator": "ge"}, {"budget": 0.12, "threshold": 0.39387038350105286, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3819783926010132, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3571593463420868, "operator": "ge"}, {"budget": 0.15, "threshold": 0.33426782488822937, "operator": "ge"}, {"budget": 0.16, "threshold": 0.32629823684692383, "operator": "ge"}, {"budget": 0.17, "threshold": 0.30695950984954834, "operator": "ge"}, {"budget": 0.18, "threshold": 0.29336801171302795, "operator": "ge"}, {"budget": 0.19, "threshold": 0.2891826033592224, "operator": "ge"}, {"budget": 0.2, "threshold": 0.2797831594944, "operator": "ge"}, {"budget": 0.21, "threshold": 0.2752266824245453, "operator": "ge"}, {"budget": 0.22, "threshold": 0.26880329847335815, "operator": "ge"}, {"budget": 0.23, "threshold": 0.2641579806804657, "operator": "ge"}, {"budget": 0.24, "threshold": 0.25728997588157654, "operator": "ge"}, {"budget": 0.25, "threshold": 0.25426700711250305, "operator": "ge"}, {"budget": 0.26, "threshold": 0.25123634934425354, "operator": "ge"}, {"budget": 0.27, "threshold": 0.24920129776000977, "operator": "ge"}, {"budget": 0.28, "threshold": 0.24501262605190277, "operator": "ge"}, {"budget": 0.29, "threshold": 0.2406313121318817, "operator": "ge"}, {"budget": 0.3, "threshold": 0.23822909593582153, "operator": "ge"}, {"budget": 0.31, "threshold": 0.2356654703617096, "operator": "ge"}, {"budget": 0.32, "threshold": 0.2340231090784073, "operator": "ge"}, {"budget": 0.33, "threshold": 0.23157188296318054, "operator": "ge"}, {"budget": 0.34, "threshold": 0.2283622920513153, "operator": "ge"}, {"budget": 0.35, "threshold": 0.22618381679058075, "operator": "ge"}, {"budget": 0.36, "threshold": 0.222722589969635, "operator": "ge"}, {"budget": 0.37, "threshold": 0.21798686683177948, "operator": "ge"}, {"budget": 0.38, "threshold": 0.2153925746679306, "operator": "ge"}, {"budget": 0.39, "threshold": 0.21063289046287537, "operator": "ge"}, {"budget": 0.4, "threshold": 0.20717036724090576, "operator": "ge"}, {"budget": 0.41, "threshold": 0.20286978781223297, "operator": "ge"}, {"budget": 0.42, "threshold": 0.1930568665266037, "operator": "ge"}, {"budget": 0.43, "threshold": 0.1820988953113556, "operator": "ge"}, {"budget": 0.44, "threshold": 0.1743784099817276, "operator": "ge"}, {"budget": 0.45, "threshold": 0.16468122601509094, "operator": "ge"}, {"budget": 0.46, "threshold": 0.14870384335517883, "operator": "ge"}, {"budget": 0.47, "threshold": 0.1427670121192932, "operator": "ge"}, {"budget": 0.48, "threshold": 0.13779586553573608, "operator": "ge"}, {"budget": 0.49, "threshold": 0.12866023182868958, "operator": "ge"}, {"budget": 0.5, "threshold": 0.12540848553180695, "operator": "ge"}, {"budget": 0.51, "threshold": 0.11182671785354614, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1032901406288147, "operator": "ge"}, {"budget": 0.53, "threshold": 0.09803740680217743, "operator": "ge"}, {"budget": 0.54, "threshold": 0.09156489372253418, "operator": "ge"}, {"budget": 0.55, "threshold": 0.08725692331790924, "operator": "ge"}, {"budget": 0.56, "threshold": 0.08025092631578445, "operator": "ge"}, {"budget": 0.57, "threshold": 0.07360437512397766, "operator": "ge"}, {"budget": 0.58, "threshold": 0.06775113940238953, "operator": "ge"}, {"budget": 0.59, "threshold": 0.06454424560070038, "operator": "ge"}, {"budget": 0.6, "threshold": 0.06216925382614136, "operator": "ge"}, {"budget": 0.61, "threshold": 0.05872151255607605, "operator": "ge"}, {"budget": 0.62, "threshold": 0.0556211918592453, "operator": "ge"}, {"budget": 0.63, "threshold": 0.05247463285923004, "operator": "ge"}, {"budget": 0.64, "threshold": 0.04676137864589691, "operator": "ge"}, {"budget": 0.65, "threshold": 0.040239766240119934, "operator": "ge"}, {"budget": 0.66, "threshold": 0.03528057038784027, "operator": "ge"}, {"budget": 0.67, "threshold": 0.030192211270332336, "operator": "ge"}, {"budget": 0.68, "threshold": 0.02709519863128662, "operator": "ge"}, {"budget": 0.69, "threshold": 0.023363560438156128, "operator": "ge"}, {"budget": 0.7, "threshold": 0.019998878240585327, "operator": "ge"}, {"budget": 0.71, "threshold": 0.018134072422981262, "operator": "ge"}, {"budget": 0.72, "threshold": 0.01392248272895813, "operator": "ge"}, {"budget": 0.73, "threshold": 0.0048241764307022095, "operator": "ge"}, {"budget": 0.74, "threshold": -0.000997781753540039, "operator": "ge"}, {"budget": 0.75, "threshold": -0.012595951557159424, "operator": "ge"}, {"budget": 0.76, "threshold": -0.0217428058385849, "operator": "ge"}, {"budget": 0.77, "threshold": -0.06022007763385773, "operator": "ge"}, {"budget": 0.78, "threshold": -0.0899617075920105, "operator": "ge"}, {"budget": 0.79, "threshold": -0.09787754714488983, "operator": "ge"}, {"budget": 0.8, "threshold": -0.10156916081905365, "operator": "ge"}, {"budget": 0.81, "threshold": -0.10831496864557266, "operator": "ge"}, {"budget": 0.82, "threshold": -0.11247636377811432, "operator": "ge"}, {"budget": 0.83, "threshold": -0.1164432018995285, "operator": "ge"}, {"budget": 0.84, "threshold": -0.12411130219697952, "operator": "ge"}, {"budget": 0.85, "threshold": -0.12993937730789185, "operator": "ge"}, {"budget": 0.86, "threshold": -0.13692553341388702, "operator": "ge"}, {"budget": 0.87, "threshold": -0.1413743942975998, "operator": "ge"}, {"budget": 0.88, "threshold": -0.14720480144023895, "operator": "ge"}, {"budget": 0.89, "threshold": -0.15211032330989838, "operator": "ge"}, {"budget": 0.9, "threshold": -0.1550770103931427, "operator": "ge"}, {"budget": 0.91, "threshold": -0.15806519985198975, "operator": "ge"}, {"budget": 0.92, "threshold": -0.1626899689435959, "operator": "ge"}, {"budget": 0.93, "threshold": -0.1656728982925415, "operator": "ge"}, {"budget": 0.94, "threshold": -0.16759361326694489, "operator": "ge"}, {"budget": 0.95, "threshold": -0.17026422917842865, "operator": "ge"}, {"budget": 0.96, "threshold": -0.17283079028129578, "operator": "ge"}, {"budget": 0.97, "threshold": -0.17714357376098633, "operator": "ge"}, {"budget": 0.98, "threshold": -0.17988504469394684, "operator": "ge"}, {"budget": 0.99, "threshold": -0.1859617680311203, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.762403481598755, 7.6425767215240885, 7.761904566672986, 7.852887515359125, 7.980358151165605, 8.08777610662656, 9.249971914989553, 9.328133881625691, 9.372610146090429, 9.359700290028968, 9.44493508609137, 9.447807157470175, 9.547012293340323, 9.664319478043929, 9.775746779149069, 10.503332002347376, 10.573291660953679, 10.660715692528957, 10.760607819149492, 11.535203799646816, 12.200385555714654, 12.680415840777407, 12.77131661858339, 12.686245194820644, 12.73817908198597, 12.86320908371725, 12.885200324863899, 12.960527851193914, 13.002958006000592, 13.122642004748819, 13.239488567146454, 13.771452936787124, 13.901071900945208, 14.71329752265151, 14.710135181514241, 14.74879240556266, 15.269908048454717, 15.364290278676826, 15.629162107719422, 15.736552202069443, 15.870587388854293, 16.137840239306993, 16.25597165280802, 16.27088046308566, 16.312753107107948, 16.452590960136806, 16.61007574991981, 16.891819672844356, 17.099048663998275, 17.344665138782787, 17.504930063652274, 17.581378030294438, 17.81338238978377, 17.799108975938243, 17.958858258519648, 18.045031873393874, 18.159066563020463, 18.336101882613512, 18.589154411764707, 18.906238008626115, 19.167566191570657, 19.609647212543553, 19.754143836850304, 20.05439705019537, 19.915526652169515, 19.871845169020702, 20.273561832398364, 20.3200120800311, 20.570047533290406, 20.721722027020885, 20.642518375667915, 20.981448793049502, 21.35677123026902, 21.690966124583046, 21.422371282936794, 21.651679180490078, 21.888546869331478, 21.919506151400395, 21.68552681398791, 21.66273100004854, 21.829538336232428, 21.788459130702776, 21.77063911288276, 21.749121546786384, 21.71813433687568, 21.676847870016758, 21.63371099942594, 21.58656799857909, 21.382552702628864, 21.34479896053917, 21.23154179518115, 21.03872114899431, 21.00791212937117, 20.968685699122396, 20.923660623943796, 20.895212823325718, 20.79158631832153, 20.750143565972355, 20.87419239066582, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 36, 56, 74, 94, 112, 131, 144, 165, 181, 197, 210, 224, 237, 248, 262, 272, 283, 297, 312, 326, 339, 350, 362, 375, 388, 396, 404, 417, 430, 441, 453, 466, 479, 490, 501, 513, 524, 537, 546, 555, 565, 573, 580, 588, 595, 603, 611, 619, 626, 631, 638, 645, 651, 657, 662, 668, 673, 679, 685, 690, 696, 701, 708, 712, 716, 722, 729, 736, 743, 748, 755, 762, 769, 772, 779, 783, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786], "harmed": [0, 7, 13, 18, 22, 29, 34, 36, 40, 46, 52, 56, 58, 62, 63, 67, 70, 74, 77, 83, 85, 89, 94, 96, 103, 108, 111, 116, 117, 124, 127, 132, 136, 141, 145, 149, 153, 155, 160, 163, 166, 169, 174, 176, 179, 183, 187, 191, 194, 199, 205, 207, 210, 214, 217, 223, 227, 230, 235, 239, 243, 247, 251, 254, 261, 269, 274, 278, 285, 291, 295, 303, 309, 315, 322, 332, 339, 346, 352, 357, 360, 364, 370, 373, 380, 384, 390, 395, 401, 405, 411, 416, 423, 428, 435, 442, 447, 453, 459, 465, 475], "exact_score": [7.641657905799261, 7.762403481598755, 7.6425767215240885, 7.761904566672986, 7.852887515359125, 7.980358151165605, 8.08777610662656, 9.249971914989553, 9.328133881625691, 9.372610146090429, 9.359700290028968, 9.44493508609137, 9.447807157470175, 9.547012293340323, 9.664319478043929, 9.775746779149069, 10.503332002347376, 10.573291660953679, 10.660715692528957, 10.760607819149492, 11.535203799646816, 12.200385555714654, 12.680415840777407, 12.77131661858339, 12.686245194820644, 12.73817908198597, 12.86320908371725, 12.885200324863899, 12.960527851193914, 13.002958006000592, 13.122642004748819, 13.239488567146454, 13.771452936787124, 13.901071900945208, 14.71329752265151, 14.710135181514241, 14.74879240556266, 15.269908048454717, 15.364290278676826, 15.629162107719422, 15.736552202069443, 15.870587388854293, 16.137840239306993, 16.25597165280802, 16.27088046308566, 16.312753107107948, 16.452590960136806, 16.61007574991981, 16.891819672844356, 17.099048663998275, 17.344665138782787, 17.504930063652274, 17.581378030294438, 17.81338238978377, 17.799108975938243, 17.958858258519648, 18.045031873393874, 18.159066563020463, 18.336101882613512, 18.589154411764707, 18.906238008626115, 19.167566191570657, 19.609647212543553, 19.754143836850304, 20.05439705019537, 19.915526652169515, 19.871845169020702, 20.273561832398364, 20.3200120800311, 20.570047533290406, 20.721722027020885, 20.642518375667915, 20.981448793049502, 21.35677123026902, 21.690966124583046, 21.422371282936794, 21.651679180490078, 21.888546869331478, 21.919506151400395, 21.68552681398791, 21.66273100004854, 21.829538336232428, 21.788459130702776, 21.77063911288276, 21.749121546786384, 21.71813433687568, 21.676847870016758, 21.63371099942594, 21.58656799857909, 21.382552702628864, 21.34479896053917, 21.23154179518115, 21.03872114899431, 21.00791212937117, 20.968685699122396, 20.923660623943796, 20.895212823325718, 20.79158631832153, 20.750143565972355, 20.87419239066582, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.42143647341603, "threshold_area100": 16.415224090339276, "threshold_area25": 9.768185502776614}, "test": {"score": [7.8873978710439285, 7.9674789436895335, 8.01620106522597, 8.618067784533892, 8.625079837069949, 8.627597603938234, 8.637952342331998, 9.361748552957192, 10.170380473076063, 10.687030919035138, 11.035271549897569, 11.26353574412672, 11.224987423543096, 11.424457645147502, 11.60931411930997, 11.518916393977996, 11.693191351581204, 11.736409290229558, 11.91168869524431, 11.949869614841756, 11.985332951135348, 12.114890663474682, 12.34640436600841, 12.40501790327925, 12.634033999952287, 12.63970434862329, 12.65109400600474, 13.050578721779202, 13.423015058167348, 13.512972539653004, 13.670992886010252, 13.871630360516788, 13.999726360678094, 14.081341571156461, 14.29336393371458, 14.458989316144363, 14.52136793285716, 14.663907742060953, 14.809596788665088, 14.986480392615622, 15.121011152007387, 15.255231930306717, 15.614666062591315, 15.624200654815985, 15.773441990070591, 15.861050345182234, 16.097219629652685, 16.220570302258842, 16.342094155852497, 16.59873460857322, 16.784121147068305, 16.96808680755657, 17.10492821233932, 17.147608400909895, 17.08255362527783, 17.42279641167602, 17.857162228163048, 17.991321968103836, 18.137030662776766, 18.59619135160869, 18.67111357813719, 18.768587452466072, 18.76779838249633, 18.729613511150138, 18.66511416881047, 19.139054981710995, 19.476439010043332, 19.76680147414496, 20.047062772845443, 20.231700499476215, 20.42132885759581, 20.58143463660705, 21.17106326042561, 21.613103750401912, 21.92582387021028, 22.32140869182026, 22.416983781133876, 22.908806045820917, 22.998639928840948, 23.305518769048163, 23.289254498090013, 23.255112505203705, 23.092548706579407, 22.968803933516487, 22.831627985646847, 22.807750237867292, 22.76676589998253, 22.68343753545973, 22.645769820159998, 22.616302510549815, 22.591403167830006, 22.484517933444128, 22.488759345530106, 22.426442916495553, 22.396900401374964, 22.391297780029365, 22.38595153670027, 22.382816839982635, 22.355127903766068, 22.286554709594395, 22.361115727328066], "calls": [0.0, 0.6990679094540613, 1.0985352862849533, 1.3648468708388815, 1.9973368841544608, 2.2969374167776295, 2.762982689747004, 3.9280958721704398, 5.126498002663116, 5.692410119840213, 6.790945406125166, 7.523302263648468, 8.288948069241012, 8.921438082556591, 9.986684420772303, 11.418109187749668, 11.950732356857523, 13.548601864181093, 15.246338215712383, 15.81225033288948, 17.14380825565912, 18.209054593874836, 19.37416777629827, 20.372836218375497, 22.37017310252996, 22.902796271637815, 23.801597869507322, 24.93342210386152, 26.364846870838882, 27.796271637816243, 28.56191744340879, 29.793608521970704, 30.62583222370173, 31.158455392809586, 32.42343541944074, 33.18908122503329, 34.187749667110516, 35.61917443408788, 36.684420772303596, 38.01597869507324, 38.78162450066578, 39.78029294274301, 41.54460719041278, 43.142476697736356, 44.241011984021306, 44.87350199733689, 46.57123834886818, 47.20372836218375, 47.86950732356858, 49.06790945406125, 49.63382157123835, 51.564580559254324, 52.762982689747005, 53.59520639147804, 54.593874833555255, 55.32623169107856, 56.757656458055926, 57.75632490013316, 59.32090545938748, 59.986684420772306, 60.719041278295606, 61.51797603195739, 62.18375499334221, 62.949400798934754, 64.41411451398136, 65.87882822902796, 67.57656458055925, 68.87483355525966, 70.07323568575234, 70.77230359520638, 71.30492676431425, 71.63781624500666, 72.76964047936086, 74.06790945406125, 74.80026631158455, 76.19840213049268, 76.8641810918775, 78.06258322237018, 79.26098535286285, 80.39280958721704, 81.15845539280959, 82.32356857523303, 83.18908122503329, 83.95472703062583, 85.41944074567243, 86.18508655126497, 87.01731025299601, 87.48335552596538, 88.18242343541944, 89.14780292942743, 89.81358189081226, 90.5126498002663, 91.97736351531292, 93.20905459387482, 93.87483355525966, 95.0399467376831, 95.87217043941412, 97.47003994673769, 98.00266311584554, 98.8681757656458, 100.0], "rescued": [0, 58, 92, 117, 168, 192, 229, 326, 419, 461, 545, 593, 642, 678, 739, 824, 855, 941, 1035, 1065, 1132, 1186, 1248, 1302, 1405, 1433, 1478, 1531, 1603, 1672, 1709, 1767, 1809, 1833, 1892, 1931, 1975, 2035, 2080, 2137, 2171, 2214, 2290, 2352, 2396, 2418, 2470, 2489, 2505, 2538, 2555, 2607, 2642, 2660, 2686, 2703, 2741, 2765, 2803, 2821, 2837, 2857, 2870, 2889, 2925, 2967, 3011, 3046, 3079, 3098, 3112, 3123, 3154, 3191, 3212, 3248, 3262, 3294, 3301, 3303, 3304, 3306, 3306, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3307, 3307, 3308, 3308, 3308, 3309, 3310, 3311, 3311, 3311, 3314], "harmed": [0, 12, 20, 24, 36, 43, 53, 81, 104, 112, 130, 139, 153, 162, 178, 208, 218, 243, 274, 282, 307, 332, 353, 373, 413, 425, 442, 459, 484, 513, 527, 552, 568, 580, 601, 612, 629, 649, 661, 682, 695, 705, 731, 751, 766, 775, 795, 807, 817, 842, 853, 894, 914, 933, 945, 955, 975, 996, 1022, 1034, 1046, 1062, 1076, 1095, 1129, 1155, 1199, 1228, 1254, 1266, 1281, 1287, 1314, 1342, 1361, 1402, 1424, 1451, 1468, 1486, 1503, 1530, 1545, 1563, 1594, 1608, 1623, 1634, 1652, 1671, 1686, 1703, 1735, 1764, 1783, 1812, 1831, 1872, 1889, 1916, 1945], "exact_score": [7.8873978710439285, 8.004251529441989, 8.625079837069949, 9.064936860691834, 9.364048252509479, 10.163101355112408, 10.652920718544273, 11.048840548631906, 11.232796898357062, 11.642280916131496, 11.60931411930997, 11.520208698996372, 11.695165295521196, 11.74970258174281, 11.918122929039376, 11.893289696654339, 11.967525058102355, 11.971742378719123, 12.109032913516682, 12.303932611928074, 12.377863211418093, 12.437559505956129, 12.5921844312102, 12.641645325668001, 12.67874208795629, 13.055785723608329, 13.315609564132815, 13.486348594819395, 13.623191511810399, 13.761002220548727, 13.871547504835238, 14.01206062784905, 14.154570702521166, 14.439176262228642, 14.50141866848319, 14.644538813589158, 14.71295064760374, 14.845104935929953, 14.980929497071141, 15.150116203989702, 15.262388130024679, 15.481691757721281, 15.623620519443666, 15.589860393004637, 15.73721155438183, 15.840450028772413, 15.965168684012243, 16.182882550028197, 16.387579358754596, 16.56657595834847, 16.806786571008413, 16.914396119285463, 16.974442517737383, 17.160075746964406, 17.227221087032568, 17.42889980452718, 17.77688210817734, 17.845579260198566, 18.02939921964829, 18.131076036396507, 18.59619135160869, 18.694882605681943, 18.784145110463477, 18.736167320274276, 18.91639373673328, 18.902761256881874, 19.164602555163995, 19.32125703122826, 19.549365110320956, 19.7938730605101, 20.01718887017843, 20.27064745579872, 20.698930885908833, 21.215484755819404, 21.575559323753374, 21.98425946059008, 22.29771366240339, 22.47593257568422, 22.87784966712277, 22.985030619450253, 22.93882113574609, 23.276513102125655, 23.24931965391462, 23.097920839312902, 22.968803933516487, 22.86525232731217, 22.814264089933754, 22.76676589998253, 22.610215319205114, 22.621051491437537, 22.566689832584792, 22.468813124999336, 22.488759345530106, 22.452647409074423, 22.39503318920572, 22.39250935493135, 22.384114470448313, 22.39238702722077, 22.35805983753285, 22.28368392969821, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.92323722025531, "threshold_area100": 16.799984432292216, "threshold_area25": 10.686133921756552}, "selected_budget": 78, "selected": {"val": {"score": 21.919506151400395, "calls": 77.88873038516405, "rescued": 784, "harmed": 352}, "test": {"score": 22.998639928840948, "calls": 79.26098535286285, "rescued": 3301, "harmed": 1468}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.0177531242370605, "operator": "ge"}, {"budget": 0.02, "threshold": 1.7818398475646973, "operator": "ge"}, {"budget": 0.03, "threshold": 1.7090797424316406, "operator": "ge"}, {"budget": 0.04, "threshold": 1.651679515838623, "operator": "ge"}, {"budget": 0.05, "threshold": 1.5958640575408936, "operator": "ge"}, {"budget": 0.06, "threshold": 1.5462476015090942, "operator": "ge"}, {"budget": 0.07, "threshold": 1.5136183500289917, "operator": "ge"}, {"budget": 0.08, "threshold": 1.4914343357086182, "operator": "ge"}, {"budget": 0.09, "threshold": 1.451158046722412, "operator": "ge"}, {"budget": 0.1, "threshold": 1.4183272123336792, "operator": "ge"}, {"budget": 0.11, "threshold": 1.4025622606277466, "operator": "ge"}, {"budget": 0.12, "threshold": 1.3801525831222534, "operator": "ge"}, {"budget": 0.13, "threshold": 1.3537348508834839, "operator": "ge"}, {"budget": 0.14, "threshold": 1.3410954475402832, "operator": "ge"}, {"budget": 0.15, "threshold": 1.3186010122299194, "operator": "ge"}, {"budget": 0.16, "threshold": 1.2957077026367188, "operator": "ge"}, {"budget": 0.17, "threshold": 1.259680151939392, "operator": "ge"}, {"budget": 0.18, "threshold": 1.2252209186553955, "operator": "ge"}, {"budget": 0.19, "threshold": 1.1979598999023438, "operator": "ge"}, {"budget": 0.2, "threshold": 1.1668040752410889, "operator": "ge"}, {"budget": 0.21, "threshold": 1.1357886791229248, "operator": "ge"}, {"budget": 0.22, "threshold": 1.1182494163513184, "operator": "ge"}, {"budget": 0.23, "threshold": 1.0801498889923096, "operator": "ge"}, {"budget": 0.24, "threshold": 1.0633691549301147, "operator": "ge"}, {"budget": 0.25, "threshold": 1.0470556020736694, "operator": "ge"}, {"budget": 0.26, "threshold": 1.0260632038116455, "operator": "ge"}, {"budget": 0.27, "threshold": 1.0145905017852783, "operator": "ge"}, {"budget": 0.28, "threshold": 1.0043225288391113, "operator": "ge"}, {"budget": 0.29, "threshold": 0.9963728189468384, "operator": "ge"}, {"budget": 0.3, "threshold": 0.9855434894561768, "operator": "ge"}, {"budget": 0.31, "threshold": 0.9778604507446289, "operator": "ge"}, {"budget": 0.32, "threshold": 0.957152247428894, "operator": "ge"}, {"budget": 0.33, "threshold": 0.9411914944648743, "operator": "ge"}, {"budget": 0.34, "threshold": 0.9288604855537415, "operator": "ge"}, {"budget": 0.35, "threshold": 0.9205183386802673, "operator": "ge"}, {"budget": 0.36, "threshold": 0.8999106884002686, "operator": "ge"}, {"budget": 0.37, "threshold": 0.8818106651306152, "operator": "ge"}, {"budget": 0.38, "threshold": 0.8635603189468384, "operator": "ge"}, {"budget": 0.39, "threshold": 0.8427860736846924, "operator": "ge"}, {"budget": 0.4, "threshold": 0.8220574855804443, "operator": "ge"}, {"budget": 0.41, "threshold": 0.8013790249824524, "operator": "ge"}, {"budget": 0.42, "threshold": 0.783990740776062, "operator": "ge"}, {"budget": 0.43, "threshold": 0.7602241039276123, "operator": "ge"}, {"budget": 0.44, "threshold": 0.7493870854377747, "operator": "ge"}, {"budget": 0.45, "threshold": 0.7342157363891602, "operator": "ge"}, {"budget": 0.46, "threshold": 0.7204660177230835, "operator": "ge"}, {"budget": 0.47, "threshold": 0.7040280103683472, "operator": "ge"}, {"budget": 0.48, "threshold": 0.6971588134765625, "operator": "ge"}, {"budget": 0.49, "threshold": 0.6751060485839844, "operator": "ge"}, {"budget": 0.5, "threshold": 0.6490070819854736, "operator": "ge"}, {"budget": 0.51, "threshold": 0.6138527989387512, "operator": "ge"}, {"budget": 0.52, "threshold": 0.5692905783653259, "operator": "ge"}, {"budget": 0.53, "threshold": 0.5463988780975342, "operator": "ge"}, {"budget": 0.54, "threshold": 0.51432204246521, "operator": "ge"}, {"budget": 0.55, "threshold": 0.49586689472198486, "operator": "ge"}, {"budget": 0.56, "threshold": 0.46022430062294006, "operator": "ge"}, {"budget": 0.57, "threshold": 0.4299973249435425, "operator": "ge"}, {"budget": 0.58, "threshold": 0.3859340250492096, "operator": "ge"}, {"budget": 0.59, "threshold": 0.36874741315841675, "operator": "ge"}, {"budget": 0.6, "threshold": 0.3416663408279419, "operator": "ge"}, {"budget": 0.61, "threshold": 0.3235204815864563, "operator": "ge"}, {"budget": 0.62, "threshold": 0.3064591586589813, "operator": "ge"}, {"budget": 0.63, "threshold": 0.2659238874912262, "operator": "ge"}, {"budget": 0.64, "threshold": 0.23819315433502197, "operator": "ge"}, {"budget": 0.65, "threshold": 0.2128915786743164, "operator": "ge"}, {"budget": 0.66, "threshold": 0.171768918633461, "operator": "ge"}, {"budget": 0.67, "threshold": 0.14114981889724731, "operator": "ge"}, {"budget": 0.68, "threshold": 0.12416636943817139, "operator": "ge"}, {"budget": 0.69, "threshold": 0.10115212202072144, "operator": "ge"}, {"budget": 0.7, "threshold": 0.08601897954940796, "operator": "ge"}, {"budget": 0.71, "threshold": 0.07784801721572876, "operator": "ge"}, {"budget": 0.72, "threshold": 0.058483660221099854, "operator": "ge"}, {"budget": 0.73, "threshold": 0.019886910915374756, "operator": "ge"}, {"budget": 0.74, "threshold": -0.005008101463317871, "operator": "ge"}, {"budget": 0.75, "threshold": -0.05806398391723633, "operator": "ge"}, {"budget": 0.76, "threshold": -0.10177299380302429, "operator": "ge"}, {"budget": 0.77, "threshold": -0.2277953028678894, "operator": "ge"}, {"budget": 0.78, "threshold": -2.36606764793396, "operator": "ge"}, {"budget": 0.79, "threshold": -2.4513354301452637, "operator": "ge"}, {"budget": 0.8, "threshold": -2.5622658729553223, "operator": "ge"}, {"budget": 0.81, "threshold": -2.647669553756714, "operator": "ge"}, {"budget": 0.82, "threshold": -2.6866726875305176, "operator": "ge"}, {"budget": 0.83, "threshold": -2.7524545192718506, "operator": "ge"}, {"budget": 0.84, "threshold": -2.767226219177246, "operator": "ge"}, {"budget": 0.85, "threshold": -2.7869935035705566, "operator": "ge"}, {"budget": 0.86, "threshold": -2.8203415870666504, "operator": "ge"}, {"budget": 0.87, "threshold": -2.8372573852539062, "operator": "ge"}, {"budget": 0.88, "threshold": -2.883244514465332, "operator": "ge"}, {"budget": 0.89, "threshold": -2.906503438949585, "operator": "ge"}, {"budget": 0.9, "threshold": -2.948946237564087, "operator": "ge"}, {"budget": 0.91, "threshold": -2.992619037628174, "operator": "ge"}, {"budget": 0.92, "threshold": -3.020932197570801, "operator": "ge"}, {"budget": 0.93, "threshold": -3.04618501663208, "operator": "ge"}, {"budget": 0.94, "threshold": -3.0875420570373535, "operator": "ge"}, {"budget": 0.95, "threshold": -3.1216485500335693, "operator": "ge"}, {"budget": 0.96, "threshold": -3.163073778152466, "operator": "ge"}, {"budget": 0.97, "threshold": -3.2283058166503906, "operator": "ge"}, {"budget": 0.98, "threshold": -3.268101692199707, "operator": "ge"}, {"budget": 0.99, "threshold": -3.3372230529785156, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.521874087636671, 7.6312696206647495, 7.645795514003866, 7.758046453452012, 7.764263428696541, 7.8770545343192495, 7.976285964145864, 9.434458538345709, 9.324888943639852, 9.359029185307294, 9.4049522550767, 9.46185419205198, 9.508876789737206, 9.596778152637789, 10.473519416461691, 10.546076819424776, 10.612740418598346, 10.628929430015342, 11.296098819506396, 11.366042096501404, 11.323300096294226, 11.430797208397221, 12.149296174482245, 12.147235163541223, 12.218445047097127, 12.318339908907861, 12.333402421161198, 12.443694989774205, 12.415812198095448, 12.509587512412878, 12.634101729939545, 12.713955418201442, 13.272863088449819, 13.344618736983529, 14.23469750434416, 14.818113266220085, 14.934271137395976, 14.909564179341055, 15.07260358717937, 15.037304625719258, 15.024903572807766, 15.146431825791911, 15.673035884303491, 16.195660724604604, 16.304125089094796, 16.30506577922808, 16.295071667022636, 16.543609280993273, 16.789775819668545, 16.878609582427618, 17.02107850439446, 17.31322564154323, 17.62125907509639, 17.720446971768887, 17.918453577943072, 17.582321443629308, 17.65861282419532, 17.877575963152502, 17.991368486984193, 18.51902963857423, 18.713424117128447, 19.01675651814739, 19.018787455613953, 19.386492562755112, 19.519493823367913, 19.876356520192136, 19.889257060898853, 20.130429403693455, 20.252099183554655, 20.47414559550482, 20.69286189616069, 20.981448793049502, 21.35677123026902, 21.690966124583046, 21.480615970580885, 21.855002427291588, 21.888546869331478, 21.82742733824579, 21.69191919191919, 21.560263537041617, 21.503666514290344, 21.45120816189639, 21.408333749337107, 21.202947858535726, 21.173903740612314, 21.15099693337227, 21.111700532318523, 21.076166260015768, 21.047139353064416, 21.01279722463981, 20.90639881085975, 20.957770488973285, 20.944843337186875, 20.91929912236864, 20.878486540243397, 20.846308488941922, 20.814770759762155, 20.786891907581563, 20.69289935888572, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 14, 29, 47, 62, 77, 95, 109, 127, 141, 157, 174, 191, 207, 222, 241, 255, 265, 274, 286, 294, 306, 317, 327, 339, 351, 362, 370, 381, 392, 401, 414, 423, 433, 446, 459, 473, 483, 495, 508, 522, 532, 542, 554, 566, 575, 581, 589, 598, 609, 616, 622, 633, 642, 649, 656, 659, 664, 669, 675, 682, 689, 696, 700, 705, 710, 716, 722, 729, 736, 742, 748, 755, 762, 769, 773, 779, 783, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786, 786, 786, 786, 786, 786], "harmed": [0, 3, 6, 11, 17, 22, 27, 30, 32, 37, 40, 43, 47, 53, 58, 62, 65, 68, 71, 73, 74, 77, 79, 82, 88, 91, 93, 101, 102, 106, 108, 112, 115, 117, 124, 127, 130, 131, 138, 142, 151, 155, 160, 166, 170, 175, 180, 188, 192, 195, 201, 205, 209, 211, 216, 218, 223, 228, 230, 234, 240, 245, 249, 257, 263, 269, 274, 280, 285, 291, 296, 302, 309, 315, 322, 333, 339, 346, 355, 363, 370, 377, 384, 389, 393, 397, 400, 405, 415, 420, 425, 431, 438, 440, 444, 450, 455, 460, 465, 470, 475], "exact_score": [7.641657905799261, 7.521874087636671, 7.6312696206647495, 7.645795514003866, 7.758046453452012, 7.764263428696541, 7.8770545343192495, 7.976285964145864, 9.434458538345709, 9.324888943639852, 9.359029185307294, 9.4049522550767, 9.46185419205198, 9.508876789737206, 9.596778152637789, 10.473519416461691, 10.546076819424776, 10.612740418598346, 10.628929430015342, 11.296098819506396, 11.366042096501404, 11.323300096294226, 11.430797208397221, 12.149296174482245, 12.147235163541223, 12.218445047097127, 12.318339908907861, 12.333402421161198, 12.443694989774205, 12.415812198095448, 12.509587512412878, 12.634101729939545, 12.713955418201442, 13.272863088449819, 13.344618736983529, 14.23469750434416, 14.818113266220085, 14.934271137395976, 14.909564179341055, 15.07260358717937, 15.037304625719258, 15.024903572807766, 15.146431825791911, 15.673035884303491, 16.195660724604604, 16.304125089094796, 16.30506577922808, 16.295071667022636, 16.543609280993273, 16.789775819668545, 16.878609582427618, 17.02107850439446, 17.31322564154323, 17.62125907509639, 17.720446971768887, 17.918453577943072, 17.582321443629308, 17.65861282419532, 17.877575963152502, 17.991368486984193, 18.51902963857423, 18.713424117128447, 19.01675651814739, 19.018787455613953, 19.386492562755112, 19.519493823367913, 19.876356520192136, 19.889257060898853, 20.130429403693455, 20.252099183554655, 20.47414559550482, 20.69286189616069, 20.981448793049502, 21.35677123026902, 21.690966124583046, 21.480615970580885, 21.855002427291588, 21.888546869331478, 21.82742733824579, 21.69191919191919, 21.560263537041617, 21.503666514290344, 21.45120816189639, 21.408333749337107, 21.202947858535726, 21.173903740612314, 21.15099693337227, 21.111700532318523, 21.076166260015768, 21.047139353064416, 21.01279722463981, 20.90639881085975, 20.957770488973285, 20.944843337186875, 20.91929912236864, 20.878486540243397, 20.846308488941922, 20.814770759762155, 20.786891907581563, 20.69289935888572, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.118255325522775, "threshold_area100": 16.111739321543155, "threshold_area25": 9.778581120683025}, "test": {"score": [7.8873978710439285, 7.9355988729966604, 9.259633391313892, 9.234503883167552, 9.19220657031736, 9.182416726777788, 9.506070709497841, 9.937601161601988, 9.894094536838239, 9.877481543812202, 9.883954805503055, 9.839572204251207, 10.458083880104507, 10.689674358910683, 10.885120757194773, 11.072565404684799, 11.02857889369197, 11.219765533669435, 11.185986592506971, 11.173599999818112, 11.577904037865501, 11.571623313913932, 11.621924423136692, 11.911017108586671, 12.011328895212392, 11.967621113867848, 11.977052556020329, 12.296996847552007, 12.321894656552294, 12.410263903632964, 12.441049225536368, 12.44894121007025, 12.645816670082521, 13.03050867530443, 13.077138585730605, 13.17029171111394, 13.418004037535733, 13.546959199745787, 13.679818437812107, 14.042427994160859, 14.531376044576543, 14.673868755133274, 14.776962874509278, 14.968162639023813, 15.124913255882639, 15.450836274643864, 15.570577814792873, 15.721967280783925, 15.56652565723362, 15.916440726594583, 15.995641985666134, 16.30879294692341, 16.581318597415084, 16.662653620686164, 16.644053626442666, 16.725787792034655, 16.918846470438716, 17.28522593888137, 17.574505781160713, 17.645473806847214, 17.826812961294202, 18.232957434975404, 18.31593426587677, 18.574673711532196, 18.530009815653326, 18.694153839801913, 19.292195410310946, 19.678364774235142, 19.945120839838697, 20.243695091005854, 20.445732379630964, 20.640558840816233, 20.95629983937161, 21.60141793224717, 21.92582387021028, 22.31897444511938, 22.453509560014034, 22.896786232798668, 22.90846095668512, 22.83969837799898, 22.82444459190435, 22.78223348973588, 22.705855299359552, 22.585898643916348, 22.57584005886031, 22.49433211751809, 22.530453807455487, 22.469578454891533, 22.376439879768864, 22.400316279274286, 22.294333254345112, 22.255977975703182, 22.219126284769285, 22.258260107264412, 22.540081743603537, 22.47442025118522, 22.408810813629092, 22.361942211645033, 22.38967116175764, 22.43481316148357, 22.361115727328066], "calls": [0.0, 0.4327563249001331, 1.5645805592543276, 1.9640479360852199, 2.596537949400799, 3.162450066577896, 3.9280958721704398, 4.593874833555259, 4.8934753661784285, 5.89214380825566, 6.358189081225034, 6.890812250332889, 7.490013315579228, 8.089214380825565, 8.754993342210387, 9.52063914780293, 10.119840213049267, 11.018641810918774, 12.51664447403462, 13.448735019973368, 14.513981358189081, 16.11185086551265, 17.11051930758988, 19.241011984021306, 20.539280958721704, 21.40479360852197, 22.736351531291614, 23.635153129161118, 24.36750998668442, 25.133155792276963, 25.898801597869507, 26.531291611185086, 28.462050599201067, 29.96005326231691, 30.85885486018642, 31.491344873501998, 33.48868175765646, 34.753661784287615, 36.21837549933422, 37.91611185086551, 38.98135818908123, 40.67909454061252, 41.81091877496671, 43.57523302263648, 44.241011984021306, 45.372836218375504, 46.07190412782956, 47.070572569906794, 47.50332889480692, 48.801597869507326, 49.70039946737683, 50.99866844207723, 53.06258322237017, 53.76165113182424, 54.793608521970704, 55.22636484687084, 56.45805592543276, 57.35685752330226, 58.75499334221038, 59.42077230359521, 60.5525965379494, 61.085219707057256, 61.55126498002663, 63.01597869507324, 64.41411451398136, 65.04660452729694, 67.21038615179761, 68.74167776298269, 69.87350199733689, 70.87217043941412, 71.5046604527297, 72.00399467376832, 72.7363515312916, 74.03462050599201, 74.80026631158455, 76.23169107856191, 76.83089214380826, 77.86284953395473, 79.36085219707057, 79.8601864181092, 80.89214380825565, 82.0572569906791, 82.72303595206392, 83.78828229027962, 84.05459387483354, 84.72037283621837, 85.38615179760319, 85.75233022636485, 87.1837549933422, 87.61651131824235, 89.08122503328894, 90.1797603195739, 90.97869507323568, 91.37816245006658, 92.60985352862849, 93.1757656458056, 94.50732356857523, 96.1051930758988, 96.90412782956058, 98.10252996005326, 100.0], "rescued": [0, 31, 123, 154, 200, 243, 295, 337, 351, 414, 447, 481, 514, 548, 586, 634, 667, 719, 798, 858, 914, 999, 1050, 1159, 1219, 1255, 1313, 1355, 1390, 1423, 1462, 1488, 1583, 1656, 1699, 1729, 1815, 1873, 1937, 2015, 2067, 2148, 2200, 2271, 2304, 2349, 2377, 2418, 2433, 2486, 2514, 2553, 2619, 2638, 2665, 2681, 2718, 2742, 2779, 2793, 2823, 2838, 2851, 2887, 2923, 2938, 2999, 3041, 3071, 3100, 3117, 3132, 3151, 3190, 3212, 3248, 3263, 3290, 3304, 3304, 3305, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3307, 3308, 3308, 3308, 3309, 3310, 3311, 3311, 3311, 3311, 3312, 3314, 3314], "harmed": [0, 0, 19, 24, 35, 44, 59, 63, 66, 80, 87, 97, 104, 114, 123, 137, 147, 164, 189, 204, 215, 242, 254, 280, 299, 312, 332, 347, 359, 370, 386, 393, 429, 455, 475, 486, 523, 543, 571, 600, 621, 658, 679, 717, 729, 744, 756, 775, 783, 808, 826, 851, 894, 905, 932, 940, 968, 984, 1006, 1017, 1033, 1042, 1052, 1090, 1123, 1134, 1181, 1226, 1245, 1267, 1283, 1294, 1313, 1341, 1361, 1403, 1425, 1446, 1481, 1495, 1516, 1547, 1566, 1587, 1592, 1611, 1625, 1634, 1668, 1679, 1715, 1742, 1758, 1765, 1790, 1803, 1835, 1866, 1881, 1905, 1945], "exact_score": [7.8873978710439285, 9.335263407998454, 9.226329546334586, 9.173474045379566, 9.51382376014278, 9.892089055261707, 9.877149609623986, 9.84183462070274, 10.397441064081752, 10.871851775911864, 11.036291164166244, 10.984130155930469, 11.210001610632906, 11.165948790683037, 11.548010941973025, 11.565165179155198, 11.571856978583273, 11.606076010873464, 11.869799284000528, 11.903979846834236, 11.970647797574307, 11.98366952538254, 11.94463927441571, 11.979890696501789, 12.290961994409216, 12.396537282927225, 12.427629845322441, 12.468294448936525, 12.621014152691751, 12.760145297210599, 13.030570602523836, 13.06422268942406, 13.374563600156876, 13.384260407521298, 13.475499784720146, 13.556523403817522, 13.691762387309042, 13.874868951748043, 14.051593651207522, 14.531376044576543, 14.715661717521133, 14.72677406866693, 14.796544723504459, 14.88814957687393, 14.991344669974332, 15.304416682435479, 15.560829776735602, 15.705537395892295, 15.751531070390703, 15.889244583625771, 16.03426294250131, 16.30879294692341, 16.34401172791403, 16.554069386464423, 16.368980961183993, 16.680748377877393, 16.917691180209687, 17.21855560810404, 17.39637753760564, 17.55321340894899, 17.6882380809455, 18.199088138773376, 18.391348350095242, 18.575663215434126, 18.333603438577565, 18.632074021550356, 18.862693292509363, 19.18172109214967, 19.483600743449106, 19.741159934442802, 19.97189716994003, 20.26316283356001, 20.629586293614466, 21.09418982518537, 21.577347201939272, 21.99677358577359, 22.29771366240339, 22.508694774547067, 22.892714784854, 22.940356178719306, 22.83525667829717, 22.818354706172038, 22.785471638537338, 22.648742476885854, 22.579515665192947, 22.542950601205074, 22.455254746433788, 22.38191809499386, 22.341625221213107, 22.29876058290911, 22.261190738073033, 22.219126284769285, 22.562485379960943, 22.482062837183367, 22.428071662052325, 22.391781165065307, 22.36616339127742, 22.38674870869007, 22.436534226084962, 22.40765939087431, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.494081721258436, "threshold_area100": 16.360972912864366, "threshold_area25": 10.328870093457406}, "selected_budget": 77, "selected": {"val": {"score": 21.888546869331478, "calls": 76.8901569186876, "rescued": 783, "harmed": 346}, "test": {"score": 22.896786232798668, "calls": 77.86284953395473, "rescued": 3290, "harmed": 1446}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.39252769947052, "operator": "ge"}, {"budget": 0.02, "threshold": 1.3685581684112549, "operator": "ge"}, {"budget": 0.03, "threshold": 1.3152694702148438, "operator": "ge"}, {"budget": 0.04, "threshold": 1.2488205432891846, "operator": "ge"}, {"budget": 0.05, "threshold": 1.2110638618469238, "operator": "ge"}, {"budget": 0.06, "threshold": 1.1797151565551758, "operator": "ge"}, {"budget": 0.07, "threshold": 1.159388780593872, "operator": "ge"}, {"budget": 0.08, "threshold": 1.1454582214355469, "operator": "ge"}, {"budget": 0.09, "threshold": 1.1263006925582886, "operator": "ge"}, {"budget": 0.1, "threshold": 1.0949095487594604, "operator": "ge"}, {"budget": 0.11, "threshold": 1.06037175655365, "operator": "ge"}, {"budget": 0.12, "threshold": 1.033896803855896, "operator": "ge"}, {"budget": 0.13, "threshold": 1.0019007921218872, "operator": "ge"}, {"budget": 0.14, "threshold": 0.9491471648216248, "operator": "ge"}, {"budget": 0.15, "threshold": 0.9205570220947266, "operator": "ge"}, {"budget": 0.16, "threshold": 0.9006993770599365, "operator": "ge"}, {"budget": 0.17, "threshold": 0.8698439598083496, "operator": "ge"}, {"budget": 0.18, "threshold": 0.8354647159576416, "operator": "ge"}, {"budget": 0.19, "threshold": 0.8252371549606323, "operator": "ge"}, {"budget": 0.2, "threshold": 0.7878643274307251, "operator": "ge"}, {"budget": 0.21, "threshold": 0.7682279944419861, "operator": "ge"}, {"budget": 0.22, "threshold": 0.7548360824584961, "operator": "ge"}, {"budget": 0.23, "threshold": 0.7329598069190979, "operator": "ge"}, {"budget": 0.24, "threshold": 0.7241089344024658, "operator": "ge"}, {"budget": 0.25, "threshold": 0.7133130431175232, "operator": "ge"}, {"budget": 0.26, "threshold": 0.7057516574859619, "operator": "ge"}, {"budget": 0.27, "threshold": 0.6971549987792969, "operator": "ge"}, {"budget": 0.28, "threshold": 0.6843782067298889, "operator": "ge"}, {"budget": 0.29, "threshold": 0.6779982447624207, "operator": "ge"}, {"budget": 0.3, "threshold": 0.6696403622627258, "operator": "ge"}, {"budget": 0.31, "threshold": 0.658576488494873, "operator": "ge"}, {"budget": 0.32, "threshold": 0.6507406234741211, "operator": "ge"}, {"budget": 0.33, "threshold": 0.6412254571914673, "operator": "ge"}, {"budget": 0.34, "threshold": 0.6355416178703308, "operator": "ge"}, {"budget": 0.35, "threshold": 0.6291868686676025, "operator": "ge"}, {"budget": 0.36, "threshold": 0.6195762157440186, "operator": "ge"}, {"budget": 0.37, "threshold": 0.6129157543182373, "operator": "ge"}, {"budget": 0.38, "threshold": 0.6048840880393982, "operator": "ge"}, {"budget": 0.39, "threshold": 0.5965372323989868, "operator": "ge"}, {"budget": 0.4, "threshold": 0.5911188125610352, "operator": "ge"}, {"budget": 0.41, "threshold": 0.5863189697265625, "operator": "ge"}, {"budget": 0.42, "threshold": 0.5781311392784119, "operator": "ge"}, {"budget": 0.43, "threshold": 0.5743218660354614, "operator": "ge"}, {"budget": 0.44, "threshold": 0.5663020014762878, "operator": "ge"}, {"budget": 0.45, "threshold": 0.5603017807006836, "operator": "ge"}, {"budget": 0.46, "threshold": 0.5508615970611572, "operator": "ge"}, {"budget": 0.47, "threshold": 0.538245677947998, "operator": "ge"}, {"budget": 0.48, "threshold": 0.5153958201408386, "operator": "ge"}, {"budget": 0.49, "threshold": 0.4850517213344574, "operator": "ge"}, {"budget": 0.5, "threshold": 0.4686098098754883, "operator": "ge"}, {"budget": 0.51, "threshold": 0.4573659300804138, "operator": "ge"}, {"budget": 0.52, "threshold": 0.4346396327018738, "operator": "ge"}, {"budget": 0.53, "threshold": 0.4124407172203064, "operator": "ge"}, {"budget": 0.54, "threshold": 0.38532406091690063, "operator": "ge"}, {"budget": 0.55, "threshold": 0.3667523264884949, "operator": "ge"}, {"budget": 0.56, "threshold": 0.3433496057987213, "operator": "ge"}, {"budget": 0.57, "threshold": 0.3152170479297638, "operator": "ge"}, {"budget": 0.58, "threshold": 0.2963707447052002, "operator": "ge"}, {"budget": 0.59, "threshold": 0.2927529513835907, "operator": "ge"}, {"budget": 0.6, "threshold": 0.26734524965286255, "operator": "ge"}, {"budget": 0.61, "threshold": 0.2556893527507782, "operator": "ge"}, {"budget": 0.62, "threshold": 0.2367388904094696, "operator": "ge"}, {"budget": 0.63, "threshold": 0.21629220247268677, "operator": "ge"}, {"budget": 0.64, "threshold": 0.20033395290374756, "operator": "ge"}, {"budget": 0.65, "threshold": 0.16787582635879517, "operator": "ge"}, {"budget": 0.66, "threshold": 0.1500493884086609, "operator": "ge"}, {"budget": 0.67, "threshold": 0.13381993770599365, "operator": "ge"}, {"budget": 0.68, "threshold": 0.11149340867996216, "operator": "ge"}, {"budget": 0.69, "threshold": 0.09598737955093384, "operator": "ge"}, {"budget": 0.7, "threshold": 0.07934266328811646, "operator": "ge"}, {"budget": 0.71, "threshold": 0.0720953643321991, "operator": "ge"}, {"budget": 0.72, "threshold": 0.054718732833862305, "operator": "ge"}, {"budget": 0.73, "threshold": 0.018829286098480225, "operator": "ge"}, {"budget": 0.74, "threshold": -0.004641294479370117, "operator": "ge"}, {"budget": 0.75, "threshold": -0.05403167009353638, "operator": "ge"}, {"budget": 0.76, "threshold": -0.09445798397064209, "operator": "ge"}, {"budget": 0.77, "threshold": -0.21494990587234497, "operator": "ge"}, {"budget": 0.78, "threshold": -2.2746119499206543, "operator": "ge"}, {"budget": 0.79, "threshold": -2.3086485862731934, "operator": "ge"}, {"budget": 0.8, "threshold": -2.417872905731201, "operator": "ge"}, {"budget": 0.81, "threshold": -2.5125513076782227, "operator": "ge"}, {"budget": 0.82, "threshold": -2.5536794662475586, "operator": "ge"}, {"budget": 0.83, "threshold": -2.6106531620025635, "operator": "ge"}, {"budget": 0.84, "threshold": -2.65037202835083, "operator": "ge"}, {"budget": 0.85, "threshold": -2.6630592346191406, "operator": "ge"}, {"budget": 0.86, "threshold": -2.680818557739258, "operator": "ge"}, {"budget": 0.87, "threshold": -2.7139523029327393, "operator": "ge"}, {"budget": 0.88, "threshold": -2.7429399490356445, "operator": "ge"}, {"budget": 0.89, "threshold": -2.7830376625061035, "operator": "ge"}, {"budget": 0.9, "threshold": -2.814761161804199, "operator": "ge"}, {"budget": 0.91, "threshold": -2.8606529235839844, "operator": "ge"}, {"budget": 0.92, "threshold": -2.87618350982666, "operator": "ge"}, {"budget": 0.93, "threshold": -2.908107280731201, "operator": "ge"}, {"budget": 0.94, "threshold": -2.946946620941162, "operator": "ge"}, {"budget": 0.95, "threshold": -2.98972749710083, "operator": "ge"}, {"budget": 0.96, "threshold": -3.034846305847168, "operator": "ge"}, {"budget": 0.97, "threshold": -3.082512617111206, "operator": "ge"}, {"budget": 0.98, "threshold": -3.119762420654297, "operator": "ge"}, {"budget": 0.99, "threshold": -3.194037914276123, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.734150644246444, 7.6370483679447005, 7.741646934622576, 7.837117559160654, 7.954016749379653, 8.07838098882875, 8.187981868506558, 8.316994868928923, 8.426959709381887, 9.492649191011182, 9.556801078846107, 9.650625806332242, 9.633386400068844, 9.735894982203222, 9.738657736267019, 9.864062707324596, 9.943246370820102, 10.065928259784636, 10.229906210634939, 11.010215137120731, 11.039186815560338, 12.364071576401086, 12.328731145232545, 12.378006946237154, 13.011696326692942, 13.088684725859794, 13.15072613570736, 13.200986542560692, 13.13094109270464, 13.279479441084801, 13.344578124931687, 13.421645556523435, 13.52513273195283, 13.677242817541549, 13.723956955766067, 14.316635089199666, 14.453438974624893, 14.613762444182916, 14.578771366370102, 14.735028542741556, 14.86048710781794, 15.535036265173089, 15.720270442066061, 16.200160892015266, 16.2915646158461, 16.807144400631042, 16.982337431898838, 17.013995070813078, 17.18059705045372, 17.445301958864505, 17.469568395626748, 17.766049376196413, 18.045878718010066, 17.953042007912384, 18.330739264754136, 18.470161779694074, 18.645320197044335, 18.75781019813679, 18.92742931016048, 19.242077292690652, 19.253026692929936, 19.46633913199263, 19.51939475122196, 19.522936506865104, 19.731358230147578, 19.876356520192136, 19.889257060898853, 19.986093224034526, 20.252099183554655, 20.47900358041894, 20.7420132610006, 20.981448793049502, 21.35677123026902, 21.639608549663556, 21.480615970580885, 21.847787940602316, 21.888546869331478, 21.90598487228739, 21.689059911282133, 21.54493680938443, 21.497389444426556, 21.261155142472017, 21.408333749337107, 21.194173619700944, 21.171012292673574, 21.15099693337227, 21.122838792925325, 21.08150785501604, 21.055339435065235, 21.0208184005442, 20.903428084355095, 20.974234521394763, 20.944843337186875, 20.91929912236864, 20.875331564986734, 20.846308488941922, 20.817877586097282, 20.786891907581563, 20.689793532200838, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 16, 33, 49, 66, 86, 105, 121, 138, 151, 167, 183, 202, 217, 233, 244, 258, 270, 282, 295, 307, 316, 331, 340, 351, 363, 375, 386, 396, 407, 417, 428, 436, 447, 460, 470, 482, 491, 501, 508, 519, 530, 538, 548, 559, 569, 580, 591, 601, 609, 619, 626, 635, 642, 647, 656, 662, 669, 676, 681, 687, 691, 695, 701, 706, 711, 716, 722, 729, 736, 742, 748, 755, 762, 768, 773, 779, 783, 784, 784, 784, 784, 784, 785, 785, 785, 785, 785, 785, 785, 785, 785, 786, 786, 786, 786, 786, 786, 786, 786, 786], "harmed": [0, 5, 12, 17, 22, 26, 32, 36, 41, 44, 48, 52, 57, 62, 65, 67, 71, 75, 77, 79, 83, 90, 93, 99, 102, 105, 108, 111, 116, 122, 127, 131, 134, 138, 143, 146, 151, 157, 160, 163, 167, 172, 175, 179, 182, 188, 191, 194, 201, 202, 205, 208, 212, 215, 220, 222, 224, 228, 234, 237, 241, 247, 248, 258, 264, 268, 274, 280, 286, 291, 296, 302, 309, 315, 324, 333, 340, 346, 354, 363, 372, 378, 384, 389, 394, 397, 400, 408, 414, 419, 424, 431, 436, 440, 444, 450, 455, 460, 465, 470, 475], "exact_score": [7.641657905799261, 7.734150644246444, 7.6370483679447005, 7.741646934622576, 7.837117559160654, 7.954016749379653, 8.07838098882875, 8.187981868506558, 8.316994868928923, 8.426959709381887, 9.492649191011182, 9.556801078846107, 9.650625806332242, 9.633386400068844, 9.735894982203222, 9.738657736267019, 9.864062707324596, 9.943246370820102, 10.065928259784636, 10.229906210634939, 11.010215137120731, 11.039186815560338, 12.364071576401086, 12.328731145232545, 12.378006946237154, 13.011696326692942, 13.088684725859794, 13.15072613570736, 13.200986542560692, 13.13094109270464, 13.279479441084801, 13.344578124931687, 13.421645556523435, 13.52513273195283, 13.677242817541549, 13.723956955766067, 14.316635089199666, 14.453438974624893, 14.613762444182916, 14.578771366370102, 14.735028542741556, 14.86048710781794, 15.535036265173089, 15.720270442066061, 16.200160892015266, 16.2915646158461, 16.807144400631042, 16.982337431898838, 17.013995070813078, 17.18059705045372, 17.445301958864505, 17.469568395626748, 17.766049376196413, 18.045878718010066, 17.953042007912384, 18.330739264754136, 18.470161779694074, 18.645320197044335, 18.75781019813679, 18.92742931016048, 19.242077292690652, 19.253026692929936, 19.46633913199263, 19.51939475122196, 19.522936506865104, 19.731358230147578, 19.876356520192136, 19.889257060898853, 19.986093224034526, 20.252099183554655, 20.47900358041894, 20.7420132610006, 20.981448793049502, 21.35677123026902, 21.639608549663556, 21.480615970580885, 21.847787940602316, 21.888546869331478, 21.90598487228739, 21.689059911282133, 21.54493680938443, 21.497389444426556, 21.261155142472017, 21.408333749337107, 21.194173619700944, 21.171012292673574, 21.15099693337227, 21.122838792925325, 21.08150785501604, 21.055339435065235, 21.0208184005442, 20.903428084355095, 20.974234521394763, 20.944843337186875, 20.91929912236864, 20.875331564986734, 20.846308488941922, 20.817877586097282, 20.786891907581563, 20.689793532200838, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 16.234062316814647, "threshold_area100": 16.227713969841652, "threshold_area25": 9.704443186004937}, "test": {"score": [7.8873978710439285, 7.961381389337456, 7.9852381419668745, 8.02763022655158, 8.663109208015324, 8.688214680862062, 8.70863649187091, 8.732854014815919, 9.16229909194832, 9.903409349259906, 10.144533349196616, 10.627697479198522, 11.424892016471064, 11.608880794325268, 11.526967367129979, 11.593361451988603, 11.629196345863013, 11.837366996034266, 11.985744023414183, 12.016312133396601, 12.096507837285229, 12.179547978719683, 12.361373172697306, 12.534738077118213, 12.560446110132736, 12.661331484596461, 12.721375683902195, 12.761953339181712, 12.813938574943196, 12.859443435105241, 12.952092900241349, 13.118531479979396, 13.145879887730723, 13.668715715099239, 13.795471460753305, 13.987902444802131, 14.216048685163942, 14.397785684924946, 14.655244610138919, 14.913545223788113, 15.239564490835269, 15.487223721245801, 15.37534235074329, 15.62480888302747, 15.637429840680744, 15.852734223695226, 15.977402389254205, 15.979344102315702, 16.37902565500832, 16.85528575214218, 17.184408010495588, 17.51881570868239, 17.72818575281343, 17.55781020067672, 17.837915673248503, 17.958999147551115, 18.103049695553707, 18.270239852012377, 18.525957206007483, 18.60038322032151, 18.66904034501747, 18.73361102304471, 18.74649703731555, 18.9485099553183, 18.666291602544867, 19.163611509284976, 19.393720382526354, 19.551561318141513, 20.022581226781288, 20.232414696417468, 20.456322839817318, 20.60738158016351, 20.95629983937161, 21.60141793224717, 21.92582387021028, 22.32140869182026, 22.496760189204952, 22.899142265601892, 22.89829162070629, 22.843639332072456, 22.80301319512589, 22.785471638537338, 22.718775992348586, 22.547812533252827, 22.514544955133974, 22.504784059488912, 22.489718644530686, 22.470633228772133, 22.350800862901632, 22.34315491474993, 22.305143965687964, 22.218625258318216, 22.200621526700505, 22.527502463073574, 22.505227872995118, 22.462844330557335, 22.39133535693716, 22.366631726232974, 22.39259896347986, 22.429666115275353, 22.361115727328066], "calls": [0.0, 0.6657789613848202, 0.8655126498002662, 1.2316910785619175, 1.9973368841544608, 2.6631158455392807, 3.262316910785619, 3.7283621837549936, 3.9280958721704398, 4.593874833555259, 5.725699067909454, 7.1904127829560585, 8.355525965379494, 9.320905459387484, 10.818908122503329, 12.017310252996005, 12.88282290279627, 14.08122503328895, 15.246338215712383, 15.645805592543274, 17.876165113182424, 18.941411451398135, 19.77363515312916, 21.13848202396804, 21.70439414114514, 22.403462050599202, 22.869507323568573, 23.535286284953397, 24.500665778961384, 25.299600532623167, 26.2982689747004, 27.762982689747005, 28.761651131824234, 29.926764314247674, 30.792276964047936, 31.857523302263647, 33.18908122503329, 34.387483355525966, 35.65246338215712, 37.217043941411454, 37.949400798934754, 38.64846870838882, 40.11318242343542, 40.51264980026631, 41.6444740346205, 42.41011984021305, 43.20905459387483, 44.44074567243675, 46.53794940079894, 49.03462050599201, 50.266311584553925, 50.86551264980027, 52.09720372836218, 53.06258322237017, 54.260985352862846, 54.89347536617842, 56.12516644474035, 57.689747003994675, 58.88814913448736, 59.05459387483355, 60.31957390146472, 60.852197070572565, 62.18375499334221, 63.51531291611185, 64.18109187749667, 66.34487350199734, 67.44340878828228, 68.37549933422103, 70.07323568575234, 70.77230359520638, 71.53794940079894, 71.90412782956058, 72.7363515312916, 74.03462050599201, 74.80026631158455, 76.19840213049268, 76.93075898801598, 77.86284953395473, 79.6271637816245, 79.76031957390146, 80.55925432756325, 81.95739014647138, 82.49001331557922, 83.58854860186418, 84.25432756324899, 84.58721704394141, 85.25299600532622, 85.91877496671106, 86.65113182423435, 88.04926764314249, 88.8814913448735, 89.94673768308921, 90.54593874833556, 91.44474034620505, 92.31025299600533, 93.50865512649801, 94.94007989347537, 95.97203728362183, 96.87083888149135, 98.30226364846871, 100.0], "rescued": [0, 52, 69, 97, 148, 198, 243, 274, 288, 341, 427, 526, 608, 670, 766, 835, 884, 947, 1015, 1039, 1154, 1201, 1243, 1309, 1338, 1370, 1391, 1419, 1464, 1498, 1547, 1612, 1650, 1705, 1746, 1796, 1851, 1904, 1955, 2020, 2059, 2091, 2155, 2169, 2210, 2244, 2280, 2330, 2417, 2512, 2556, 2582, 2622, 2648, 2682, 2700, 2733, 2772, 2799, 2803, 2832, 2843, 2876, 2907, 2922, 2976, 3005, 3030, 3077, 3097, 3118, 3129, 3151, 3190, 3212, 3248, 3266, 3290, 3304, 3304, 3304, 3306, 3306, 3306, 3306, 3306, 3307, 3307, 3307, 3308, 3308, 3308, 3308, 3310, 3311, 3311, 3311, 3311, 3312, 3314, 3314], "harmed": [0, 11, 15, 21, 31, 42, 55, 64, 66, 79, 104, 130, 155, 170, 195, 210, 224, 246, 273, 281, 319, 334, 349, 367, 380, 395, 400, 414, 433, 446, 463, 489, 504, 520, 542, 559, 582, 602, 625, 655, 668, 677, 705, 715, 735, 746, 761, 778, 811, 848, 867, 878, 894, 912, 940, 952, 965, 997, 1014, 1016, 1044, 1057, 1087, 1105, 1118, 1162, 1193, 1216, 1250, 1265, 1284, 1292, 1313, 1341, 1361, 1402, 1428, 1445, 1490, 1493, 1505, 1545, 1555, 1583, 1600, 1606, 1623, 1633, 1657, 1686, 1708, 1734, 1749, 1767, 1783, 1811, 1845, 1863, 1879, 1908, 1945], "exact_score": [7.8873978710439285, 8.001287791645614, 8.663109208015324, 8.701673966541629, 9.161918246783344, 9.858020578567398, 10.115491887600763, 10.308507452153853, 11.236624628493427, 11.618232667004136, 11.514658136324632, 11.529808014090772, 11.588910008816981, 11.643501565656901, 11.828925459266097, 12.002056491520117, 12.028718479847802, 12.032435858602552, 12.096891542125654, 12.187067805839401, 12.389623095408, 12.511265676546179, 12.52909536947158, 12.721472729815325, 12.766347367874253, 12.82741500150822, 12.930372375316812, 12.996903261846606, 13.121918082364207, 13.171837774454739, 13.674665143115728, 13.81766483160849, 13.99847308880395, 14.21198487035507, 14.38360407617266, 14.55132735988033, 14.668137799027054, 14.712501787194586, 15.251235462793813, 15.529237765879525, 15.263560724280447, 15.668872614567922, 15.69097392879582, 15.935755524313599, 15.852392631156482, 16.100320955914636, 16.30045434601094, 16.5086594602337, 16.752617702372604, 16.84143020376184, 17.097983154545123, 17.535344391476887, 17.723967643810006, 17.527597145045682, 17.81549539356876, 17.96754211947991, 18.06487828699085, 18.241955753070297, 18.22875011626717, 18.56598705048057, 18.611399932007988, 18.670470390418302, 18.759991359816617, 18.95925091389093, 18.911412901191905, 18.828148455280683, 19.09420997336133, 19.289788806179754, 19.518079594987714, 19.754182019648514, 19.993736355603392, 20.280941948165722, 20.683024712855623, 21.19402472550155, 21.577347201939272, 21.98425946059008, 22.29771366240339, 22.494382034652883, 22.892714784854, 22.9380873584186, 22.819180666734695, 22.822766905486898, 22.785471638537338, 22.631779192113473, 22.527630692469128, 22.440854974676387, 22.468589864031525, 22.336766408077953, 22.346652292548214, 22.30365409679643, 22.217190596898313, 22.505658983517744, 22.513483865705425, 22.482530212589452, 22.427227437417656, 22.39133535693716, 22.366631726232974, 22.391367800276864, 22.437744332897054, 22.406469804850246, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 16.902585668038604, "threshold_area100": 16.699110158751555, "threshold_area25": 10.559396399430542}, "selected_budget": 78, "selected": {"val": {"score": 21.90598487228739, "calls": 77.88873038516405, "rescued": 784, "harmed": 354}, "test": {"score": 22.89829162070629, "calls": 79.6271637816245, "rescued": 3304, "harmed": 1490}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 21.936136832098168, "calls": 77.88873038516405}, "test": {"score": 23.057666691849263, "calls": 78.99467376830891}}, "old_normalized_interior": {"val": {"score": 22.09509348306372, "calls": 76.8901569186876}, "test": {"score": 22.26857516273408, "calls": 77.06391478029295}}, "old_normalized_endpoints": {"val": {"score": 22.09509348306372, "calls": 76.8901569186876}, "test": {"score": 22.26857516273408, "calls": 77.06391478029295}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 10.517864445954938, 9.936704710211867, 9.691983463997044, 9.487322275647081, 9.432589073171403, 9.398958484892574, 9.36551986275052, 9.160204405012182, 9.253822551174027, 9.344598710851368, 9.454070594251016, 10.146916843780554, 10.226801064693376, 10.305914668235564, 11.084217872421565, 11.08569369389199, 11.00448932375943, 10.981302480045203, 11.575425087906059, 11.602008839053271, 12.214301399084007, 12.177138120036206, 12.106050728932086, 12.165709767249973, 13.589517963312467, 13.620539618713732, 13.666579248896074, 13.747689977792465, 13.85100121690802, 13.924709649741985, 14.036817314437236, 14.039304722267392, 14.108145930872157, 14.22259823475629, 14.258039821302768, 14.724375701594674, 14.830926226774652, 15.184165190843188, 15.357290778037413, 15.359270620464649, 15.539691291669754, 15.589942812925027, 15.762873752162598, 15.933418731287802, 16.195203891434552, 16.452178832762115, 16.50130309639512, 16.30148997762777, 16.55183200928987, 16.662017657056527, 16.812080469833806, 16.714467907495333, 16.66185206061331, 16.83643887406278, 17.047935154488663, 17.223573901839238, 17.428529707911018, 17.66801953100703, 17.872725113594868, 18.12291144527987, 18.292962144065086, 18.155661681575733, 18.292805650916584, 18.439947366691083, 18.48792443806791, 18.813588497571608, 18.84824439322502, 19.060327931250004, 19.419697492365728, 19.78690934988364, 19.72367158713181, 20.060791649572735, 20.312279999962705, 20.707794342504105, 21.31284092584402, 21.695740153524415, 22.09509348306372, 21.936136832098168, 21.90598487228739, 21.880083092400632, 21.64633162193147, 21.618278088382826, 21.585034006642733, 21.749121546786384, 21.727929295736853, 21.696202708726435, 21.65665771261792, 21.446902766978926, 21.41307897595687, 21.371529781175365, 21.264767944894018, 21.220199124715172, 21.10841599315059, 21.060875878748632, 20.95055754185829, 20.830760267114652, 20.788165676105148, 20.754257099497654, 20.799342091264624, 20.666016863629594], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 10.517864445954938, 9.936704710211867, 9.691983463997044, 9.487322275647081, 9.432589073171403, 9.398958484892574, 9.36551986275052, 9.160204405012182, 9.253822551174027, 9.344598710851368, 9.454070594251016, 10.146916843780554, 10.226801064693376, 10.305914668235564, 11.084217872421565, 11.08569369389199, 11.00448932375943, 10.981302480045203, 11.575425087906059, 11.602008839053271, 12.214301399084007, 12.177138120036206, 12.106050728932086, 12.165709767249973, 13.589517963312467, 13.620539618713732, 13.666579248896074, 13.747689977792465, 13.85100121690802, 13.924709649741985, 14.036817314437236, 14.039304722267392, 14.108145930872157, 14.22259823475629, 14.258039821302768, 14.724375701594674, 14.830926226774652, 15.184165190843188, 15.357290778037413, 15.359270620464649, 15.539691291669754, 15.589942812925027, 15.762873752162598, 15.933418731287802, 16.195203891434552, 16.452178832762115, 16.50130309639512, 16.30148997762777, 16.55183200928987, 16.662017657056527, 16.812080469833806, 16.714467907495333, 16.66185206061331, 16.83643887406278, 17.047935154488663, 17.223573901839238, 17.428529707911018, 17.66801953100703, 17.872725113594868, 18.12291144527987, 18.292962144065086, 18.155661681575733, 18.292805650916584, 18.439947366691083, 18.48792443806791, 18.813588497571608, 18.84824439322502, 19.060327931250004, 19.419697492365728, 19.78690934988364, 19.72367158713181, 20.060791649572735, 20.312279999962705, 20.707794342504105, 21.31284092584402, 21.695740153524415, 22.09509348306372, 21.936136832098168, 21.90598487228739, 21.880083092400632, 21.64633162193147, 21.618278088382826, 21.585034006642733, 21.749121546786384, 21.727929295736853, 21.696202708726435, 21.65665771261792, 21.446902766978926, 21.41307897595687, 21.371529781175365, 21.264767944894018, 21.220199124715172, 21.10841599315059, 21.060875878748632, 20.95055754185829, 20.830760267114652, 20.788165676105148, 20.754257099497654, 20.799342091264624, 20.666016863629594], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 10.415800486010696, "exact_area100": 16.328190110601486}, "test": {"score": [7.8873978710439285, 7.9570062877259655, 9.044398063962626, 9.010852189209274, 9.411123271632661, 10.403612316421919, 11.093090221026086, 10.935978476834064, 10.77224481197172, 10.767370603539765, 11.006165734014687, 10.99065974172338, 10.97729700477892, 11.155393862701676, 11.418780543846733, 11.890433844610598, 12.607529745722307, 13.146538382660305, 13.0494409588614, 12.908386156126692, 12.886313180497705, 12.940872918858027, 12.840532816513587, 12.862260221855093, 13.203376833858524, 13.204318855303363, 13.409846277604492, 13.59130477693346, 13.631119574053285, 13.716170105116667, 13.697709935594778, 13.705388668295381, 13.780041460475934, 14.141214958940079, 14.181875400618777, 14.408562992081212, 14.435553017642045, 14.669267228596194, 14.77699928199323, 14.872570004892061, 14.920760628891806, 15.154105900142644, 15.245086242918743, 15.570917707621392, 15.724798062206313, 15.94443574181614, 15.918522681298192, 15.786057314481353, 15.58730724250329, 15.555901284840932, 15.590687081737423, 15.733015193771754, 15.630963805236592, 15.705512278717782, 16.083701584773998, 16.188392646949104, 16.295142445333923, 16.58319093903826, 16.865001034903194, 16.956671053176258, 17.091540192564143, 17.346020771440525, 17.694464829998015, 17.95829957477612, 18.346348324484264, 18.616386850490866, 18.71164525422637, 19.094761378585044, 19.71699097653073, 20.00188881691746, 20.249427115592606, 20.692007965286304, 21.250799093509944, 21.40963037891039, 21.670847135630726, 21.931872918697977, 22.11534580028245, 22.26857516273408, 23.057666691849263, 22.964141883885507, 22.97725708095551, 23.316513096887746, 23.209844353196306, 23.163304980539724, 23.075769042462717, 23.08690654075491, 22.833487501771206, 22.77874231574493, 22.75402203783876, 22.716369976817646, 22.548713983112727, 22.56449173689948, 22.492014814080036, 22.569023631418233, 22.52391323315118, 22.4932976701906, 22.517809629194133, 22.44384290095144, 22.421028688925126, 22.418638182974398, 22.361115727328066], "calls": [0.0, 0.6324900133155792, 1.4980026631158456, 1.7643142476697737, 2.263648468708389, 3.262316910785619, 4.194407456724368, 4.960053262316911, 5.858854860186418, 6.191744340878828, 6.92410119840213, 7.689747003994674, 8.754993342210387, 9.720372836218376, 11.218375499334222, 12.549933422103862, 13.781624500665778, 14.880159786950733, 15.545938748335553, 16.17842876165113, 16.444740346205062, 17.07723035952064, 17.976031957390145, 19.44074567243675, 20.539280958721704, 21.837549933422103, 23.135818908122506, 24.866844207723034, 26.43142476697736, 27.62982689747004, 28.362183754993342, 28.695073235685754, 29.560585885486017, 31.158455392809586, 31.89081225033289, 33.88814913448735, 35.58588548601864, 37.18375499334221, 38.21571238348868, 39.18109187749667, 39.84687083888149, 40.9121171770972, 41.84420772303595, 43.77496671105193, 44.77363515312916, 46.20505992010652, 46.6711051930759, 47.636484687083886, 49.60053262316911, 50.8988015978695, 51.73102529960053, 52.330226364846865, 53.52862849533955, 55.32623169107856, 57.02396804260985, 57.78961384820239, 58.655126498002666, 60.35286284953395, 60.81890812250332, 61.784287616511314, 62.68308921438083, 64.0479360852197, 64.94673768308921, 65.87882822902796, 67.0439414114514, 68.14247669773636, 68.57523302263648, 69.00798934753661, 70.30625832223701, 70.87217043941412, 72.20372836218375, 73.16910785619174, 74.30093209054593, 74.56724367509987, 75.06657789613848, 75.7323568575233, 76.79760319573901, 77.06391478029295, 78.96138482023967, 80.42609853528629, 81.15845539280959, 82.29027962716378, 83.38881491344874, 84.22103861517975, 85.18641810918774, 85.68575233022636, 86.65113182423435, 87.7496671105193, 88.44873501997337, 89.41411451398136, 90.81225033288948, 91.4780292942743, 92.24367509986683, 93.3422103861518, 94.37416777629826, 95.1398135818908, 95.93874833555259, 96.97070572569906, 97.43675099866844, 99.03462050599201, 100.0], "exact_score": [7.8873978710439285, 9.133631734380927, 9.467255566426342, 10.456149906391481, 11.18549986950045, 10.925509858131992, 10.778356935827436, 10.999384032616808, 10.99672805877772, 10.967195797558215, 11.091148977863833, 11.417181113656971, 11.784789897060563, 11.876005686352345, 12.4932518609336, 13.13585814334405, 12.937962427606875, 12.949996557339139, 12.840532816513587, 12.91740811882135, 13.1106373141095, 13.180819135008667, 13.20126061875351, 13.42341939954407, 13.592111923818456, 13.605840100723315, 13.564123603109532, 13.666140374653224, 13.715204040270976, 13.73532540565241, 13.94925381558486, 14.143685916702244, 14.197353883774491, 14.305205385620717, 14.411495671747135, 14.37967523654965, 14.495726783689843, 14.640926609749053, 14.74161817803148, 14.833655174054043, 14.95225641608223, 15.169485411501126, 15.26012499341491, 15.428409730764866, 15.583107664112184, 15.7332453148958, 15.887163089574685, 15.947682199860354, 15.758123417784391, 15.716678550810952, 15.66110819925953, 15.562528206192141, 15.622028487472845, 15.71238810117145, 15.707994614860986, 15.662327970479625, 15.803950874653072, 16.07779166136563, 16.21287277946139, 16.361534488301952, 16.57575143538534, 16.84733055928345, 16.98576707067895, 17.139212832876872, 17.35465368806321, 17.709399119125855, 17.993944479423867, 18.322912309584236, 18.573838922848886, 19.082099139589502, 19.594227813716707, 20.080711728110312, 20.473376850826753, 20.614478756544564, 21.06941773339635, 21.6220548483955, 22.097259240912933, 22.236829136454304, 22.775213472002793, 23.057666691849263, 22.977713461557506, 22.985669316031135, 23.324771415559134, 23.281864744935106, 23.16856084086798, 23.082398536899305, 23.0564085712045, 22.803279007924132, 22.769774302628996, 22.736387171833147, 22.601137114680096, 22.541477980605023, 22.550008178614398, 22.579345029120613, 22.549019607843135, 22.497569560309234, 22.51594523505443, 22.44384290095144, 22.38178473227027, 22.422099282380266, 22.361115727328066], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 11.346476687743712, "exact_area100": 16.718725980305244}}, "old_normalized_endpoint_budget": 77}]}, {"expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "task": "construction", "small": 7.8873978710439285, "large": 27.001009067886244, "runs": [{"seed": 2026, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.0, "validation_area100": 0.19453288685362638, "training_seconds": 4.0791098199551925, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5747206807136536, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5526160001754761, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5423272252082825, "operator": "ge"}, {"budget": 0.04, "threshold": 0.532315731048584, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5237470865249634, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5145057439804077, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5092116594314575, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5022686719894409, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4929208755493164, "operator": "ge"}, {"budget": 0.1, "threshold": 0.48475557565689087, "operator": "ge"}, {"budget": 0.11, "threshold": 0.4698891341686249, "operator": "ge"}, {"budget": 0.12, "threshold": 0.44665706157684326, "operator": "ge"}, {"budget": 0.13, "threshold": 0.4126831889152527, "operator": "ge"}, {"budget": 0.14, "threshold": 0.40410301089286804, "operator": "ge"}, {"budget": 0.15, "threshold": 0.39591196179389954, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3912280201911926, "operator": "ge"}, {"budget": 0.17, "threshold": 0.3884928524494171, "operator": "ge"}, {"budget": 0.18, "threshold": 0.38206052780151367, "operator": "ge"}, {"budget": 0.19, "threshold": 0.37744832038879395, "operator": "ge"}, {"budget": 0.2, "threshold": 0.36932796239852905, "operator": "ge"}, {"budget": 0.21, "threshold": 0.3651222288608551, "operator": "ge"}, {"budget": 0.22, "threshold": 0.3596850335597992, "operator": "ge"}, {"budget": 0.23, "threshold": 0.35650914907455444, "operator": "ge"}, {"budget": 0.24, "threshold": 0.3516850769519806, "operator": "ge"}, {"budget": 0.25, "threshold": 0.34820082783699036, "operator": "ge"}, {"budget": 0.26, "threshold": 0.3425414264202118, "operator": "ge"}, {"budget": 0.27, "threshold": 0.3406660556793213, "operator": "ge"}, {"budget": 0.28, "threshold": 0.33415722846984863, "operator": "ge"}, {"budget": 0.29, "threshold": 0.33045241236686707, "operator": "ge"}, {"budget": 0.3, "threshold": 0.32840344309806824, "operator": "ge"}, {"budget": 0.31, "threshold": 0.3237122595310211, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3206685483455658, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3148639500141144, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3115175664424896, "operator": "ge"}, {"budget": 0.35, "threshold": 0.30551832914352417, "operator": "ge"}, {"budget": 0.36, "threshold": 0.29977625608444214, "operator": "ge"}, {"budget": 0.37, "threshold": 0.29268866777420044, "operator": "ge"}, {"budget": 0.38, "threshold": 0.2855967879295349, "operator": "ge"}, {"budget": 0.39, "threshold": 0.27774155139923096, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2697201371192932, "operator": "ge"}, {"budget": 0.41, "threshold": 0.26697343587875366, "operator": "ge"}, {"budget": 0.42, "threshold": 0.25702357292175293, "operator": "ge"}, {"budget": 0.43, "threshold": 0.24722811579704285, "operator": "ge"}, {"budget": 0.44, "threshold": 0.24166254699230194, "operator": "ge"}, {"budget": 0.45, "threshold": 0.2319689840078354, "operator": "ge"}, {"budget": 0.46, "threshold": 0.22196589410305023, "operator": "ge"}, {"budget": 0.47, "threshold": 0.2100147306919098, "operator": "ge"}, {"budget": 0.48, "threshold": 0.20013198256492615, "operator": "ge"}, {"budget": 0.49, "threshold": 0.1885652244091034, "operator": "ge"}, {"budget": 0.5, "threshold": 0.17361947894096375, "operator": "ge"}, {"budget": 0.51, "threshold": 0.16849064826965332, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1634911596775055, "operator": "ge"}, {"budget": 0.53, "threshold": 0.15918420255184174, "operator": "ge"}, {"budget": 0.54, "threshold": 0.15627962350845337, "operator": "ge"}, {"budget": 0.55, "threshold": 0.15486936271190643, "operator": "ge"}, {"budget": 0.56, "threshold": 0.15182510018348694, "operator": "ge"}, {"budget": 0.57, "threshold": 0.1483381986618042, "operator": "ge"}, {"budget": 0.58, "threshold": 0.1471657156944275, "operator": "ge"}, {"budget": 0.59, "threshold": 0.1447974145412445, "operator": "ge"}, {"budget": 0.6, "threshold": 0.14139559864997864, "operator": "ge"}, {"budget": 0.61, "threshold": 0.13793663680553436, "operator": "ge"}, {"budget": 0.62, "threshold": 0.1344996988773346, "operator": "ge"}, {"budget": 0.63, "threshold": 0.129487544298172, "operator": "ge"}, {"budget": 0.64, "threshold": 0.12624944746494293, "operator": "ge"}, {"budget": 0.65, "threshold": 0.12324493378400803, "operator": "ge"}, {"budget": 0.66, "threshold": 0.11896435171365738, "operator": "ge"}, {"budget": 0.67, "threshold": 0.11523118615150452, "operator": "ge"}, {"budget": 0.68, "threshold": 0.1124737411737442, "operator": "ge"}, {"budget": 0.69, "threshold": 0.11036255210638046, "operator": "ge"}, {"budget": 0.7, "threshold": 0.10665874928236008, "operator": "ge"}, {"budget": 0.71, "threshold": 0.10320481657981873, "operator": "ge"}, {"budget": 0.72, "threshold": 0.09453701227903366, "operator": "ge"}, {"budget": 0.73, "threshold": 0.09257092326879501, "operator": "ge"}, {"budget": 0.74, "threshold": 0.08732516318559647, "operator": "ge"}, {"budget": 0.75, "threshold": 0.0838906541466713, "operator": "ge"}, {"budget": 0.76, "threshold": 0.0687154084444046, "operator": "ge"}, {"budget": 0.77, "threshold": 0.03901508450508118, "operator": "ge"}, {"budget": 0.78, "threshold": -0.014959589578211308, "operator": "ge"}, {"budget": 0.79, "threshold": -0.019287047907710075, "operator": "ge"}, {"budget": 0.8, "threshold": -0.023396914824843407, "operator": "ge"}, {"budget": 0.81, "threshold": -0.02937357686460018, "operator": "ge"}, {"budget": 0.82, "threshold": -0.032248012721538544, "operator": "ge"}, {"budget": 0.83, "threshold": -0.03421695530414581, "operator": "ge"}, {"budget": 0.84, "threshold": -0.03664484620094299, "operator": "ge"}, {"budget": 0.85, "threshold": -0.037857044488191605, "operator": "ge"}, {"budget": 0.86, "threshold": -0.041384633630514145, "operator": "ge"}, {"budget": 0.87, "threshold": -0.0437280610203743, "operator": "ge"}, {"budget": 0.88, "threshold": -0.045368485152721405, "operator": "ge"}, {"budget": 0.89, "threshold": -0.04761158302426338, "operator": "ge"}, {"budget": 0.9, "threshold": -0.050944387912750244, "operator": "ge"}, {"budget": 0.91, "threshold": -0.05446704104542732, "operator": "ge"}, {"budget": 0.92, "threshold": -0.05701490491628647, "operator": "ge"}, {"budget": 0.93, "threshold": -0.058684006333351135, "operator": "ge"}, {"budget": 0.94, "threshold": -0.060278553515672684, "operator": "ge"}, {"budget": 0.95, "threshold": -0.06395154446363449, "operator": "ge"}, {"budget": 0.96, "threshold": -0.07168066501617432, "operator": "ge"}, {"budget": 0.97, "threshold": -0.07613411545753479, "operator": "ge"}, {"budget": 0.98, "threshold": -0.08845474570989609, "operator": "ge"}, {"budget": 0.99, "threshold": -0.09889992326498032, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.759670120931815, 7.878079426546922, 7.685999953441813, 9.72432728266982, 9.64512746063057, 9.595789996516437, 9.709959951583663, 9.659701554232806, 9.730422409379646, 9.713996696732565, 9.879155502049809, 9.90808091624093, 9.890335797645978, 12.254360369384976, 12.359247389081176, 12.210776383545262, 12.345409259205336, 12.483889634130302, 12.473805526609562, 12.47282117947967, 12.504129303538814, 12.630294178916573, 12.787275635707692, 12.817154589237854, 12.833715093948769, 12.948930741854175, 13.045237169027327, 14.628440863605427, 14.662224688800935, 14.844847973969769, 16.468361629484697, 16.49328450907099, 16.64365014785244, 16.617520849625077, 16.770981539753745, 16.94129625164108, 17.08938035323132, 17.872374846254107, 18.13406752880715, 18.247289433730113, 18.934801486761806, 19.235027344271042, 19.498091798301655, 19.80743324377849, 20.070603337612322, 19.35158196783206, 19.64051128394366, 19.707110319667596, 20.290259747267097, 20.53140518895852, 20.827829232157193, 21.016877263173885, 21.247119882030365, 21.427540512805457, 21.560110608463944, 21.69222214567607, 21.926299589603282, 22.17730060059112, 22.23721203838462, 22.174328154207956, 22.396683923438726, 22.65272329628592, 23.62507555198585, 23.832685852495786, 23.976054865770276, 24.243375449546555, 24.194797893425292, 24.432315841921696, 24.633233142541798, 24.760284798773657, 24.795039562909864, 24.783347841981062, 25.08643325390898, 25.44248784482535, 25.698729268996022, 26.111921586505588, 26.739258752325757, 26.93115545451994, 26.790379957892128, 26.790379957892128, 26.901466207008912, 26.655773420479306, 26.655773420479306, 27.026143790849677, 26.80122129758052, 27.13340601085888, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.77666495463106, 26.77666495463106, 26.54822798684691, 26.464888866074638, 26.359924400320235, 26.2168885125301, 26.030273685704703, 25.953468770190106, 25.854030401246824, 25.665150070600163, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 20, 39, 55, 76, 96, 115, 133, 151, 170, 188, 209, 226, 240, 254, 266, 276, 290, 304, 317, 328, 341, 353, 366, 378, 387, 397, 407, 420, 431, 443, 457, 470, 483, 495, 507, 519, 532, 544, 555, 565, 576, 587, 596, 608, 620, 627, 636, 641, 650, 656, 663, 669, 676, 683, 690, 695, 701, 708, 714, 716, 720, 727, 734, 739, 745, 750, 755, 762, 767, 772, 778, 780, 787, 791, 798, 804, 810, 812, 812, 812, 813, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 5, 7, 11, 14, 17, 20, 20, 24, 26, 29, 29, 30, 32, 32, 33, 37, 37, 37, 39, 41, 43, 43, 43, 45, 47, 47, 48, 50, 52, 53, 53, 56, 57, 61, 62, 63, 65, 66, 66, 68, 70, 70, 70, 71, 72, 74, 74, 75, 78, 78, 78, 78, 78, 79, 81, 82, 82, 83, 86, 87, 90, 91, 92, 93, 96, 100, 104, 107, 109, 112, 116, 123, 126, 129, 133, 138, 141, 141, 143, 143, 144, 148, 148, 149, 153, 154, 160, 160, 160, 161, 161, 164, 166, 168, 171, 174, 176, 178, 181, 185], "exact_score": [7.641657905799261, 7.759670120931815, 7.878079426546922, 7.685999953441813, 9.72432728266982, 9.64512746063057, 9.595789996516437, 9.709959951583663, 9.659701554232806, 9.730422409379646, 9.713996696732565, 9.879155502049809, 9.90808091624093, 9.890335797645978, 12.254360369384976, 12.359247389081176, 12.210776383545262, 12.345409259205336, 12.483889634130302, 12.473805526609562, 12.47282117947967, 12.504129303538814, 12.630294178916573, 12.787275635707692, 12.817154589237854, 12.833715093948769, 12.948930741854175, 13.045237169027327, 14.628440863605427, 14.662224688800935, 14.844847973969769, 16.468361629484697, 16.49328450907099, 16.64365014785244, 16.617520849625077, 16.770981539753745, 16.94129625164108, 17.08938035323132, 17.872374846254107, 18.13406752880715, 18.247289433730113, 18.934801486761806, 19.235027344271042, 19.498091798301655, 19.80743324377849, 20.070603337612322, 19.35158196783206, 19.64051128394366, 19.707110319667596, 20.290259747267097, 20.53140518895852, 20.827829232157193, 21.016877263173885, 21.247119882030365, 21.427540512805457, 21.560110608463944, 21.69222214567607, 21.926299589603282, 22.17730060059112, 22.23721203838462, 22.174328154207956, 22.396683923438726, 22.65272329628592, 23.62507555198585, 23.832685852495786, 23.976054865770276, 24.243375449546555, 24.194797893425292, 24.432315841921696, 24.633233142541798, 24.760284798773657, 24.795039562909864, 24.783347841981062, 25.08643325390898, 25.44248784482535, 25.698729268996022, 26.111921586505588, 26.739258752325757, 26.93115545451994, 26.790379957892128, 26.790379957892128, 26.901466207008912, 26.655773420479306, 26.655773420479306, 27.026143790849677, 26.80122129758052, 27.13340601085888, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.77666495463106, 26.77666495463106, 26.54822798684691, 26.464888866074638, 26.359924400320235, 26.2168885125301, 26.030273685704703, 25.953468770190106, 25.854030401246824, 25.665150070600163, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.453288685362637, "threshold_area100": 19.44456925756093, "threshold_area25": 10.8857758451596}, "test": {"score": [7.8873978710439285, 8.71247248244506, 9.882249609515107, 10.47273124647672, 11.34722610901188, 11.68178200975752, 11.529767857250384, 11.478192129466379, 11.423505497373402, 11.936079341099527, 12.189413103412425, 12.117756626837382, 12.27694478302973, 12.338366534254272, 12.365455000278438, 12.332337411854533, 12.332497940418111, 12.350886140405452, 12.380353508850103, 13.251651370504087, 13.08969371993414, 13.096401850413294, 13.085434845258625, 13.12616917259437, 13.311001078663478, 13.228262623841836, 13.653874331437782, 13.642989782924312, 13.965512376132946, 14.190630303598217, 14.252058308357899, 14.832734924515242, 14.852167546583756, 14.960415556783854, 15.066971604256368, 15.090687338363065, 15.046672506572628, 15.383517738472232, 15.805041415374935, 16.395142182155237, 16.62441493120192, 16.808525772637974, 17.30960050261279, 17.72330253507476, 17.864855319111673, 18.013359042330197, 18.30662908096728, 18.667608876164632, 18.76488958113399, 19.088749303360693, 19.01981184303481, 18.86698205480651, 19.16858702079777, 19.616483646800162, 19.79712710855201, 19.88864883867098, 20.335879420561756, 20.566885802878616, 21.2374196480804, 21.307662603039333, 21.692619818982827, 22.311362693136093, 22.859449708070386, 23.445195719919433, 23.74052084588511, 23.80836350123975, 24.12823421379571, 24.35760539327493, 24.4816788138369, 24.67502463397586, 25.001286501602095, 25.143486066737264, 25.500819201731773, 25.525536958812996, 25.666466525096315, 25.886187066721455, 26.593054215418675, 26.883621913251776, 27.03438042302786, 27.03438042302786, 27.00121975110067, 26.89588640574994, 26.754058765965365, 26.746692213889506, 26.768573843785525, 26.744937461747835, 26.843442031752186, 26.860170567756985, 26.762550486003, 26.762183165355836, 27.22314787989189, 27.152982429484236, 27.115260581505986, 27.08494855291168, 27.042284421171715, 27.056871798503256, 27.354783482264068, 27.301798917235292, 27.075857191353887, 26.942254201054638, 27.001009067886244], "calls": [0.0, 0.6990679094540613, 1.7976031957390146, 2.62982689747004, 3.3288948069241013, 4.094540612516644, 4.660452729693742, 5.126498002663116, 5.525965379494008, 6.191744340878828, 6.724367509986684, 7.157123834886818, 8.422103861517977, 10.685752330226366, 11.584553928095872, 12.51664447403462, 13.249001331557922, 13.615179760319574, 14.547270306258323, 15.179760319573901, 16.81091877496671, 17.609853528628495, 18.575233022636482, 19.241011984021306, 20.406125166444742, 21.371504660452732, 22.836218375499335, 23.33555259653795, 25.233022636484687, 26.331557922769637, 26.99733688415446, 28.462050599201067, 28.99467376830892, 30.9254327563249, 31.69107856191744, 33.12250332889481, 34.28761651131824, 35.58588548601864, 37.616511318242345, 38.8814913448735, 40.279627163781626, 40.77896138482024, 42.80958721704394, 44.34087882822903, 44.940079893475364, 45.938748335552596, 46.870838881491345, 47.969374167776294, 48.368841544607186, 49.4340878828229, 51.16511318242344, 51.53129161118508, 52.99600532623169, 55.09320905459387, 56.39147802929428, 56.8242343541944, 58.48868175765646, 59.720372836218374, 60.31957390146472, 61.18508655126498, 62.45006657789614, 63.68175765645806, 64.88015978695073, 66.67776298268974, 67.60985352862849, 68.37549933422103, 69.57390146471371, 70.5392809587217, 71.07190412782957, 71.57123834886818, 72.13715046604527, 72.7363515312916, 74.33422103861518, 74.56724367509987, 75.06657789613848, 75.53262316910786, 77.16378162450066, 78.12916111850865, 78.92809587217043, 79.36085219707057, 80.12649800266311, 81.49134487350199, 82.65645805592543, 83.22237017310253, 84.0878828229028, 84.65379494007989, 86.38482023968042, 87.38348868175765, 88.04926764314249, 88.8814913448735, 90.44607190412783, 91.81091877496671, 92.54327563249002, 92.97603195739015, 93.44207723035952, 94.60719041278296, 95.97203728362183, 96.53794940079894, 98.30226364846871, 98.90146471371504, 100.0], "rescued": [0, 59, 145, 213, 271, 336, 382, 422, 454, 507, 546, 580, 675, 811, 865, 916, 956, 974, 1021, 1054, 1141, 1182, 1234, 1266, 1324, 1372, 1451, 1476, 1571, 1626, 1661, 1736, 1763, 1862, 1899, 1972, 2024, 2089, 2187, 2255, 2324, 2349, 2447, 2515, 2542, 2577, 2611, 2652, 2664, 2694, 2738, 2748, 2789, 2846, 2878, 2889, 2934, 2966, 2985, 3003, 3035, 3070, 3103, 3151, 3176, 3191, 3224, 3247, 3261, 3275, 3287, 3302, 3338, 3341, 3353, 3367, 3412, 3435, 3447, 3447, 3447, 3447, 3447, 3447, 3448, 3448, 3450, 3451, 3451, 3452, 3455, 3455, 3455, 3455, 3455, 3456, 3459, 3459, 3460, 3460, 3461], "harmed": [0, 3, 17, 20, 28, 35, 42, 46, 50, 61, 67, 72, 85, 92, 95, 101, 103, 105, 108, 109, 122, 124, 131, 135, 145, 158, 169, 174, 188, 195, 198, 211, 215, 229, 231, 244, 257, 264, 276, 287, 298, 305, 321, 332, 335, 340, 344, 348, 348, 352, 357, 358, 361, 370, 378, 378, 383, 387, 389, 401, 412, 416, 428, 442, 446, 457, 467, 474, 481, 483, 486, 495, 503, 506, 514, 522, 534, 546, 553, 553, 555, 563, 572, 573, 574, 576, 582, 585, 593, 597, 605, 611, 615, 618, 620, 622, 629, 634, 648, 657, 676], "exact_score": [7.8873978710439285, 8.725017182939334, 10.459171924649446, 11.492442784296774, 11.70492870486157, 11.543799257740618, 11.693794909936273, 12.126074175188762, 12.31721762832923, 12.307935388312242, 12.37586369395657, 12.322425397315495, 12.361652907500737, 12.31014156514691, 12.362285748379092, 12.985457568484, 13.222727995161678, 13.112572411459666, 13.142696291278305, 13.109817042446476, 13.32154660992739, 13.242717772519995, 13.63697870362525, 13.66432625093028, 13.621264349133904, 13.748968048508106, 14.150919550787568, 14.252058308357899, 14.63446159269052, 14.852167546583756, 14.852091804466832, 14.96951801533433, 15.111348951932568, 15.090516383452885, 15.036325829034391, 15.278132106654146, 15.583818125296792, 15.710169852288086, 16.000496499178833, 16.407767501159423, 16.56217518234476, 16.83648509858725, 17.10184117320608, 17.363071481386676, 17.50573454352782, 17.871305998163642, 18.019369577158514, 18.3337799169003, 18.667608876164632, 18.98480807842244, 19.010156685405818, 18.976127824105067, 18.9609177699338, 19.16858702079777, 19.443398455133263, 19.60365399359601, 19.734642898639184, 19.920217480922037, 20.250517976688386, 20.40366725062647, 20.911361802938867, 21.27752291310563, 21.514097762167204, 21.917074917258304, 22.355021252589864, 22.880324976097192, 23.112031814617602, 23.52311245773146, 23.78681350620912, 23.981078659228675, 24.22543666687116, 24.439802936981422, 24.76244827599196, 25.219174321474547, 25.61777836038831, 25.635429064230507, 26.10307125090548, 26.528190404828567, 26.84232776058164, 27.03438042302786, 27.017781846870687, 26.93475128341411, 26.843445560271572, 26.754058765965365, 26.768573843785525, 26.834339647884313, 26.778188534295293, 26.79937634073466, 26.762550486003, 26.762183165355836, 27.24536587011597, 27.21579725508878, 27.152982429484236, 27.08494855291168, 27.019255428225343, 27.27232519802179, 27.354783482264068, 27.15283575694635, 27.153557269053962, 26.935410554505108, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.347757573225554, "threshold_area100": 19.401558530957868, "threshold_area25": 11.9702112984803}, "selected_budget": 86, "selected": {"val": {"score": 27.13340601085888, "calls": 85.87731811697576, "rescued": 815, "harmed": 154}, "test": {"score": 26.843442031752186, "calls": 86.38482023968042, "rescued": 3450, "harmed": 582}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5747206807136536, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5526160001754761, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5423272252082825, "operator": "ge"}, {"budget": 0.04, "threshold": 0.532315731048584, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5237470865249634, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5145057439804077, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5092116594314575, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5022686719894409, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4929208755493164, "operator": "ge"}, {"budget": 0.1, "threshold": 0.48475557565689087, "operator": "ge"}, {"budget": 0.11, "threshold": 0.4698891341686249, "operator": "ge"}, {"budget": 0.12, "threshold": 0.44665706157684326, "operator": "ge"}, {"budget": 0.13, "threshold": 0.4126831889152527, "operator": "ge"}, {"budget": 0.14, "threshold": 0.40410301089286804, "operator": "ge"}, {"budget": 0.15, "threshold": 0.39591196179389954, "operator": "ge"}, {"budget": 0.16, "threshold": 0.3912280201911926, "operator": "ge"}, {"budget": 0.17, "threshold": 0.3884928524494171, "operator": "ge"}, {"budget": 0.18, "threshold": 0.38206052780151367, "operator": "ge"}, {"budget": 0.19, "threshold": 0.37744832038879395, "operator": "ge"}, {"budget": 0.2, "threshold": 0.36932796239852905, "operator": "ge"}, {"budget": 0.21, "threshold": 0.3651222288608551, "operator": "ge"}, {"budget": 0.22, "threshold": 0.3596850335597992, "operator": "ge"}, {"budget": 0.23, "threshold": 0.35650914907455444, "operator": "ge"}, {"budget": 0.24, "threshold": 0.3516850769519806, "operator": "ge"}, {"budget": 0.25, "threshold": 0.34820082783699036, "operator": "ge"}, {"budget": 0.26, "threshold": 0.3425414264202118, "operator": "ge"}, {"budget": 0.27, "threshold": 0.3406660556793213, "operator": "ge"}, {"budget": 0.28, "threshold": 0.33415722846984863, "operator": "ge"}, {"budget": 0.29, "threshold": 0.33045241236686707, "operator": "ge"}, {"budget": 0.3, "threshold": 0.32840344309806824, "operator": "ge"}, {"budget": 0.31, "threshold": 0.3237122595310211, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3206685483455658, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3148639500141144, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3115175664424896, "operator": "ge"}, {"budget": 0.35, "threshold": 0.30551832914352417, "operator": "ge"}, {"budget": 0.36, "threshold": 0.29977625608444214, "operator": "ge"}, {"budget": 0.37, "threshold": 0.29268866777420044, "operator": "ge"}, {"budget": 0.38, "threshold": 0.2855967879295349, "operator": "ge"}, {"budget": 0.39, "threshold": 0.27774155139923096, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2697201371192932, "operator": "ge"}, {"budget": 0.41, "threshold": 0.26697343587875366, "operator": "ge"}, {"budget": 0.42, "threshold": 0.25702357292175293, "operator": "ge"}, {"budget": 0.43, "threshold": 0.24722811579704285, "operator": "ge"}, {"budget": 0.44, "threshold": 0.24166254699230194, "operator": "ge"}, {"budget": 0.45, "threshold": 0.2319689840078354, "operator": "ge"}, {"budget": 0.46, "threshold": 0.22196589410305023, "operator": "ge"}, {"budget": 0.47, "threshold": 0.2100147306919098, "operator": "ge"}, {"budget": 0.48, "threshold": 0.20013198256492615, "operator": "ge"}, {"budget": 0.49, "threshold": 0.1885652244091034, "operator": "ge"}, {"budget": 0.5, "threshold": 0.17361947894096375, "operator": "ge"}, {"budget": 0.51, "threshold": 0.16849064826965332, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1634911596775055, "operator": "ge"}, {"budget": 0.53, "threshold": 0.15918420255184174, "operator": "ge"}, {"budget": 0.54, "threshold": 0.15627962350845337, "operator": "ge"}, {"budget": 0.55, "threshold": 0.15486936271190643, "operator": "ge"}, {"budget": 0.56, "threshold": 0.15182510018348694, "operator": "ge"}, {"budget": 0.57, "threshold": 0.1483381986618042, "operator": "ge"}, {"budget": 0.58, "threshold": 0.1471657156944275, "operator": "ge"}, {"budget": 0.59, "threshold": 0.1447974145412445, "operator": "ge"}, {"budget": 0.6, "threshold": 0.14139559864997864, "operator": "ge"}, {"budget": 0.61, "threshold": 0.13793663680553436, "operator": "ge"}, {"budget": 0.62, "threshold": 0.1344996988773346, "operator": "ge"}, {"budget": 0.63, "threshold": 0.129487544298172, "operator": "ge"}, {"budget": 0.64, "threshold": 0.12624944746494293, "operator": "ge"}, {"budget": 0.65, "threshold": 0.12324493378400803, "operator": "ge"}, {"budget": 0.66, "threshold": 0.11896435171365738, "operator": "ge"}, {"budget": 0.67, "threshold": 0.11523118615150452, "operator": "ge"}, {"budget": 0.68, "threshold": 0.1124737411737442, "operator": "ge"}, {"budget": 0.69, "threshold": 0.11036255210638046, "operator": "ge"}, {"budget": 0.7, "threshold": 0.10665874928236008, "operator": "ge"}, {"budget": 0.71, "threshold": 0.10320481657981873, "operator": "ge"}, {"budget": 0.72, "threshold": 0.09453701227903366, "operator": "ge"}, {"budget": 0.73, "threshold": 0.09257092326879501, "operator": "ge"}, {"budget": 0.74, "threshold": 0.08732516318559647, "operator": "ge"}, {"budget": 0.75, "threshold": 0.0838906541466713, "operator": "ge"}, {"budget": 0.76, "threshold": 0.0687154084444046, "operator": "ge"}, {"budget": 0.77, "threshold": 0.03901508450508118, "operator": "ge"}, {"budget": 0.78, "threshold": -0.014959589578211308, "operator": "ge"}, {"budget": 0.79, "threshold": -0.019287047907710075, "operator": "ge"}, {"budget": 0.8, "threshold": -0.023396914824843407, "operator": "ge"}, {"budget": 0.81, "threshold": -0.02937357686460018, "operator": "ge"}, {"budget": 0.82, "threshold": -0.032248012721538544, "operator": "ge"}, {"budget": 0.83, "threshold": -0.03421695530414581, "operator": "ge"}, {"budget": 0.84, "threshold": -0.03664484620094299, "operator": "ge"}, {"budget": 0.85, "threshold": -0.037857044488191605, "operator": "ge"}, {"budget": 0.86, "threshold": -0.041384633630514145, "operator": "ge"}, {"budget": 0.87, "threshold": -0.0437280610203743, "operator": "ge"}, {"budget": 0.88, "threshold": -0.045368485152721405, "operator": "ge"}, {"budget": 0.89, "threshold": -0.04761158302426338, "operator": "ge"}, {"budget": 0.9, "threshold": -0.050944387912750244, "operator": "ge"}, {"budget": 0.91, "threshold": -0.05446704104542732, "operator": "ge"}, {"budget": 0.92, "threshold": -0.05701490491628647, "operator": "ge"}, {"budget": 0.93, "threshold": -0.058684006333351135, "operator": "ge"}, {"budget": 0.94, "threshold": -0.060278553515672684, "operator": "ge"}, {"budget": 0.95, "threshold": -0.06395154446363449, "operator": "ge"}, {"budget": 0.96, "threshold": -0.07168066501617432, "operator": "ge"}, {"budget": 0.97, "threshold": -0.07613411545753479, "operator": "ge"}, {"budget": 0.98, "threshold": -0.08845474570989609, "operator": "ge"}, {"budget": 0.99, "threshold": -0.09889992326498032, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.759670120931815, 7.878079426546922, 7.685999953441813, 9.72432728266982, 9.64512746063057, 9.595789996516437, 9.709959951583663, 9.659701554232806, 9.730422409379646, 9.713996696732565, 9.879155502049809, 9.90808091624093, 9.890335797645978, 12.254360369384976, 12.359247389081176, 12.210776383545262, 12.345409259205336, 12.483889634130302, 12.473805526609562, 12.47282117947967, 12.504129303538814, 12.630294178916573, 12.787275635707692, 12.817154589237854, 12.833715093948769, 12.948930741854175, 13.045237169027327, 14.628440863605427, 14.662224688800935, 14.844847973969769, 16.468361629484697, 16.49328450907099, 16.64365014785244, 16.617520849625077, 16.770981539753745, 16.94129625164108, 17.08938035323132, 17.872374846254107, 18.13406752880715, 18.247289433730113, 18.934801486761806, 19.235027344271042, 19.498091798301655, 19.80743324377849, 20.070603337612322, 19.35158196783206, 19.64051128394366, 19.707110319667596, 20.290259747267097, 20.53140518895852, 20.827829232157193, 21.016877263173885, 21.247119882030365, 21.427540512805457, 21.560110608463944, 21.69222214567607, 21.926299589603282, 22.17730060059112, 22.23721203838462, 22.174328154207956, 22.396683923438726, 22.65272329628592, 23.62507555198585, 23.832685852495786, 23.976054865770276, 24.243375449546555, 24.194797893425292, 24.432315841921696, 24.633233142541798, 24.760284798773657, 24.795039562909864, 24.783347841981062, 25.08643325390898, 25.44248784482535, 25.698729268996022, 26.111921586505588, 26.739258752325757, 26.93115545451994, 26.790379957892128, 26.790379957892128, 26.901466207008912, 26.655773420479306, 26.655773420479306, 27.026143790849677, 26.80122129758052, 27.13340601085888, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.77666495463106, 26.77666495463106, 26.54822798684691, 26.464888866074638, 26.359924400320235, 26.2168885125301, 26.030273685704703, 25.953468770190106, 25.854030401246824, 25.665150070600163, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 20, 39, 55, 76, 96, 115, 133, 151, 170, 188, 209, 226, 240, 254, 266, 276, 290, 304, 317, 328, 341, 353, 366, 378, 387, 397, 407, 420, 431, 443, 457, 470, 483, 495, 507, 519, 532, 544, 555, 565, 576, 587, 596, 608, 620, 627, 636, 641, 650, 656, 663, 669, 676, 683, 690, 695, 701, 708, 714, 716, 720, 727, 734, 739, 745, 750, 755, 762, 767, 772, 778, 780, 787, 791, 798, 804, 810, 812, 812, 812, 813, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 5, 7, 11, 14, 17, 20, 20, 24, 26, 29, 29, 30, 32, 32, 33, 37, 37, 37, 39, 41, 43, 43, 43, 45, 47, 47, 48, 50, 52, 53, 53, 56, 57, 61, 62, 63, 65, 66, 66, 68, 70, 70, 70, 71, 72, 74, 74, 75, 78, 78, 78, 78, 78, 79, 81, 82, 82, 83, 86, 87, 90, 91, 92, 93, 96, 100, 104, 107, 109, 112, 116, 123, 126, 129, 133, 138, 141, 141, 143, 143, 144, 148, 148, 149, 153, 154, 160, 160, 160, 161, 161, 164, 166, 168, 171, 174, 176, 178, 181, 185], "exact_score": [7.641657905799261, 7.759670120931815, 7.878079426546922, 7.685999953441813, 9.72432728266982, 9.64512746063057, 9.595789996516437, 9.709959951583663, 9.659701554232806, 9.730422409379646, 9.713996696732565, 9.879155502049809, 9.90808091624093, 9.890335797645978, 12.254360369384976, 12.359247389081176, 12.210776383545262, 12.345409259205336, 12.483889634130302, 12.473805526609562, 12.47282117947967, 12.504129303538814, 12.630294178916573, 12.787275635707692, 12.817154589237854, 12.833715093948769, 12.948930741854175, 13.045237169027327, 14.628440863605427, 14.662224688800935, 14.844847973969769, 16.468361629484697, 16.49328450907099, 16.64365014785244, 16.617520849625077, 16.770981539753745, 16.94129625164108, 17.08938035323132, 17.872374846254107, 18.13406752880715, 18.247289433730113, 18.934801486761806, 19.235027344271042, 19.498091798301655, 19.80743324377849, 20.070603337612322, 19.35158196783206, 19.64051128394366, 19.707110319667596, 20.290259747267097, 20.53140518895852, 20.827829232157193, 21.016877263173885, 21.247119882030365, 21.427540512805457, 21.560110608463944, 21.69222214567607, 21.926299589603282, 22.17730060059112, 22.23721203838462, 22.174328154207956, 22.396683923438726, 22.65272329628592, 23.62507555198585, 23.832685852495786, 23.976054865770276, 24.243375449546555, 24.194797893425292, 24.432315841921696, 24.633233142541798, 24.760284798773657, 24.795039562909864, 24.783347841981062, 25.08643325390898, 25.44248784482535, 25.698729268996022, 26.111921586505588, 26.739258752325757, 26.93115545451994, 26.790379957892128, 26.790379957892128, 26.901466207008912, 26.655773420479306, 26.655773420479306, 27.026143790849677, 26.80122129758052, 27.13340601085888, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.77666495463106, 26.77666495463106, 26.54822798684691, 26.464888866074638, 26.359924400320235, 26.2168885125301, 26.030273685704703, 25.953468770190106, 25.854030401246824, 25.665150070600163, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.453288685362637, "threshold_area100": 19.44456925756093, "threshold_area25": 10.8857758451596}, "test": {"score": [7.8873978710439285, 8.71247248244506, 9.882249609515107, 10.47273124647672, 11.34722610901188, 11.68178200975752, 11.529767857250384, 11.478192129466379, 11.423505497373402, 11.936079341099527, 12.189413103412425, 12.117756626837382, 12.27694478302973, 12.338366534254272, 12.365455000278438, 12.332337411854533, 12.332497940418111, 12.350886140405452, 12.380353508850103, 13.251651370504087, 13.08969371993414, 13.096401850413294, 13.085434845258625, 13.12616917259437, 13.311001078663478, 13.228262623841836, 13.653874331437782, 13.642989782924312, 13.965512376132946, 14.190630303598217, 14.252058308357899, 14.832734924515242, 14.852167546583756, 14.960415556783854, 15.066971604256368, 15.090687338363065, 15.046672506572628, 15.383517738472232, 15.805041415374935, 16.395142182155237, 16.62441493120192, 16.808525772637974, 17.30960050261279, 17.72330253507476, 17.864855319111673, 18.013359042330197, 18.30662908096728, 18.667608876164632, 18.76488958113399, 19.088749303360693, 19.01981184303481, 18.86698205480651, 19.16858702079777, 19.616483646800162, 19.79712710855201, 19.88864883867098, 20.335879420561756, 20.566885802878616, 21.2374196480804, 21.307662603039333, 21.692619818982827, 22.311362693136093, 22.859449708070386, 23.445195719919433, 23.74052084588511, 23.80836350123975, 24.12823421379571, 24.35760539327493, 24.4816788138369, 24.67502463397586, 25.001286501602095, 25.143486066737264, 25.500819201731773, 25.525536958812996, 25.666466525096315, 25.886187066721455, 26.593054215418675, 26.883621913251776, 27.03438042302786, 27.03438042302786, 27.00121975110067, 26.89588640574994, 26.754058765965365, 26.746692213889506, 26.768573843785525, 26.744937461747835, 26.843442031752186, 26.860170567756985, 26.762550486003, 26.762183165355836, 27.22314787989189, 27.152982429484236, 27.115260581505986, 27.08494855291168, 27.042284421171715, 27.056871798503256, 27.354783482264068, 27.301798917235292, 27.075857191353887, 26.942254201054638, 27.001009067886244], "calls": [0.0, 0.6990679094540613, 1.7976031957390146, 2.62982689747004, 3.3288948069241013, 4.094540612516644, 4.660452729693742, 5.126498002663116, 5.525965379494008, 6.191744340878828, 6.724367509986684, 7.157123834886818, 8.422103861517977, 10.685752330226366, 11.584553928095872, 12.51664447403462, 13.249001331557922, 13.615179760319574, 14.547270306258323, 15.179760319573901, 16.81091877496671, 17.609853528628495, 18.575233022636482, 19.241011984021306, 20.406125166444742, 21.371504660452732, 22.836218375499335, 23.33555259653795, 25.233022636484687, 26.331557922769637, 26.99733688415446, 28.462050599201067, 28.99467376830892, 30.9254327563249, 31.69107856191744, 33.12250332889481, 34.28761651131824, 35.58588548601864, 37.616511318242345, 38.8814913448735, 40.279627163781626, 40.77896138482024, 42.80958721704394, 44.34087882822903, 44.940079893475364, 45.938748335552596, 46.870838881491345, 47.969374167776294, 48.368841544607186, 49.4340878828229, 51.16511318242344, 51.53129161118508, 52.99600532623169, 55.09320905459387, 56.39147802929428, 56.8242343541944, 58.48868175765646, 59.720372836218374, 60.31957390146472, 61.18508655126498, 62.45006657789614, 63.68175765645806, 64.88015978695073, 66.67776298268974, 67.60985352862849, 68.37549933422103, 69.57390146471371, 70.5392809587217, 71.07190412782957, 71.57123834886818, 72.13715046604527, 72.7363515312916, 74.33422103861518, 74.56724367509987, 75.06657789613848, 75.53262316910786, 77.16378162450066, 78.12916111850865, 78.92809587217043, 79.36085219707057, 80.12649800266311, 81.49134487350199, 82.65645805592543, 83.22237017310253, 84.0878828229028, 84.65379494007989, 86.38482023968042, 87.38348868175765, 88.04926764314249, 88.8814913448735, 90.44607190412783, 91.81091877496671, 92.54327563249002, 92.97603195739015, 93.44207723035952, 94.60719041278296, 95.97203728362183, 96.53794940079894, 98.30226364846871, 98.90146471371504, 100.0], "rescued": [0, 59, 145, 213, 271, 336, 382, 422, 454, 507, 546, 580, 675, 811, 865, 916, 956, 974, 1021, 1054, 1141, 1182, 1234, 1266, 1324, 1372, 1451, 1476, 1571, 1626, 1661, 1736, 1763, 1862, 1899, 1972, 2024, 2089, 2187, 2255, 2324, 2349, 2447, 2515, 2542, 2577, 2611, 2652, 2664, 2694, 2738, 2748, 2789, 2846, 2878, 2889, 2934, 2966, 2985, 3003, 3035, 3070, 3103, 3151, 3176, 3191, 3224, 3247, 3261, 3275, 3287, 3302, 3338, 3341, 3353, 3367, 3412, 3435, 3447, 3447, 3447, 3447, 3447, 3447, 3448, 3448, 3450, 3451, 3451, 3452, 3455, 3455, 3455, 3455, 3455, 3456, 3459, 3459, 3460, 3460, 3461], "harmed": [0, 3, 17, 20, 28, 35, 42, 46, 50, 61, 67, 72, 85, 92, 95, 101, 103, 105, 108, 109, 122, 124, 131, 135, 145, 158, 169, 174, 188, 195, 198, 211, 215, 229, 231, 244, 257, 264, 276, 287, 298, 305, 321, 332, 335, 340, 344, 348, 348, 352, 357, 358, 361, 370, 378, 378, 383, 387, 389, 401, 412, 416, 428, 442, 446, 457, 467, 474, 481, 483, 486, 495, 503, 506, 514, 522, 534, 546, 553, 553, 555, 563, 572, 573, 574, 576, 582, 585, 593, 597, 605, 611, 615, 618, 620, 622, 629, 634, 648, 657, 676], "exact_score": [7.8873978710439285, 8.725017182939334, 10.459171924649446, 11.492442784296774, 11.70492870486157, 11.543799257740618, 11.693794909936273, 12.126074175188762, 12.31721762832923, 12.307935388312242, 12.37586369395657, 12.322425397315495, 12.361652907500737, 12.31014156514691, 12.362285748379092, 12.985457568484, 13.222727995161678, 13.112572411459666, 13.142696291278305, 13.109817042446476, 13.32154660992739, 13.242717772519995, 13.63697870362525, 13.66432625093028, 13.621264349133904, 13.748968048508106, 14.150919550787568, 14.252058308357899, 14.63446159269052, 14.852167546583756, 14.852091804466832, 14.96951801533433, 15.111348951932568, 15.090516383452885, 15.036325829034391, 15.278132106654146, 15.583818125296792, 15.710169852288086, 16.000496499178833, 16.407767501159423, 16.56217518234476, 16.83648509858725, 17.10184117320608, 17.363071481386676, 17.50573454352782, 17.871305998163642, 18.019369577158514, 18.3337799169003, 18.667608876164632, 18.98480807842244, 19.010156685405818, 18.976127824105067, 18.9609177699338, 19.16858702079777, 19.443398455133263, 19.60365399359601, 19.734642898639184, 19.920217480922037, 20.250517976688386, 20.40366725062647, 20.911361802938867, 21.27752291310563, 21.514097762167204, 21.917074917258304, 22.355021252589864, 22.880324976097192, 23.112031814617602, 23.52311245773146, 23.78681350620912, 23.981078659228675, 24.22543666687116, 24.439802936981422, 24.76244827599196, 25.219174321474547, 25.61777836038831, 25.635429064230507, 26.10307125090548, 26.528190404828567, 26.84232776058164, 27.03438042302786, 27.017781846870687, 26.93475128341411, 26.843445560271572, 26.754058765965365, 26.768573843785525, 26.834339647884313, 26.778188534295293, 26.79937634073466, 26.762550486003, 26.762183165355836, 27.24536587011597, 27.21579725508878, 27.152982429484236, 27.08494855291168, 27.019255428225343, 27.27232519802179, 27.354783482264068, 27.15283575694635, 27.153557269053962, 26.935410554505108, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.347757573225554, "threshold_area100": 19.401558530957868, "threshold_area25": 11.9702112984803}, "selected_budget": 86, "selected": {"val": {"score": 27.13340601085888, "calls": 85.87731811697576, "rescued": 815, "harmed": 154}, "test": {"score": 26.843442031752186, "calls": 86.38482023968042, "rescued": 3450, "harmed": 582}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.4645307064056396, "operator": "ge"}, {"budget": 0.02, "threshold": 2.3383610248565674, "operator": "ge"}, {"budget": 0.03, "threshold": 2.30389666557312, "operator": "ge"}, {"budget": 0.04, "threshold": 2.2464301586151123, "operator": "ge"}, {"budget": 0.05, "threshold": 2.1984424591064453, "operator": "ge"}, {"budget": 0.06, "threshold": 2.158447504043579, "operator": "ge"}, {"budget": 0.07, "threshold": 2.114521026611328, "operator": "ge"}, {"budget": 0.08, "threshold": 2.072789192199707, "operator": "ge"}, {"budget": 0.09, "threshold": 2.0485754013061523, "operator": "ge"}, {"budget": 0.1, "threshold": 2.025750160217285, "operator": "ge"}, {"budget": 0.11, "threshold": 2.0055172443389893, "operator": "ge"}, {"budget": 0.12, "threshold": 1.986574411392212, "operator": "ge"}, {"budget": 0.13, "threshold": 1.9686182737350464, "operator": "ge"}, {"budget": 0.14, "threshold": 1.961216926574707, "operator": "ge"}, {"budget": 0.15, "threshold": 1.9341344833374023, "operator": "ge"}, {"budget": 0.16, "threshold": 1.921105146408081, "operator": "ge"}, {"budget": 0.17, "threshold": 1.9104406833648682, "operator": "ge"}, {"budget": 0.18, "threshold": 1.8941197395324707, "operator": "ge"}, {"budget": 0.19, "threshold": 1.8777962923049927, "operator": "ge"}, {"budget": 0.2, "threshold": 1.8571431636810303, "operator": "ge"}, {"budget": 0.21, "threshold": 1.8383020162582397, "operator": "ge"}, {"budget": 0.22, "threshold": 1.8227789402008057, "operator": "ge"}, {"budget": 0.23, "threshold": 1.794368028640747, "operator": "ge"}, {"budget": 0.24, "threshold": 1.7698792219161987, "operator": "ge"}, {"budget": 0.25, "threshold": 1.7550842761993408, "operator": "ge"}, {"budget": 0.26, "threshold": 1.7428348064422607, "operator": "ge"}, {"budget": 0.27, "threshold": 1.728750467300415, "operator": "ge"}, {"budget": 0.28, "threshold": 1.71979558467865, "operator": "ge"}, {"budget": 0.29, "threshold": 1.6841715574264526, "operator": "ge"}, {"budget": 0.3, "threshold": 1.6689238548278809, "operator": "ge"}, {"budget": 0.31, "threshold": 1.6583809852600098, "operator": "ge"}, {"budget": 0.32, "threshold": 1.6460034847259521, "operator": "ge"}, {"budget": 0.33, "threshold": 1.625478982925415, "operator": "ge"}, {"budget": 0.34, "threshold": 1.5985236167907715, "operator": "ge"}, {"budget": 0.35, "threshold": 1.5645133256912231, "operator": "ge"}, {"budget": 0.36, "threshold": 1.5466158390045166, "operator": "ge"}, {"budget": 0.37, "threshold": 1.5249484777450562, "operator": "ge"}, {"budget": 0.38, "threshold": 1.5171126127243042, "operator": "ge"}, {"budget": 0.39, "threshold": 1.4964498281478882, "operator": "ge"}, {"budget": 0.4, "threshold": 1.4826816320419312, "operator": "ge"}, {"budget": 0.41, "threshold": 1.460130214691162, "operator": "ge"}, {"budget": 0.42, "threshold": 1.4417979717254639, "operator": "ge"}, {"budget": 0.43, "threshold": 1.424590826034546, "operator": "ge"}, {"budget": 0.44, "threshold": 1.398454189300537, "operator": "ge"}, {"budget": 0.45, "threshold": 1.3872817754745483, "operator": "ge"}, {"budget": 0.46, "threshold": 1.3724782466888428, "operator": "ge"}, {"budget": 0.47, "threshold": 1.3571035861968994, "operator": "ge"}, {"budget": 0.48, "threshold": 1.339128851890564, "operator": "ge"}, {"budget": 0.49, "threshold": 1.3238866329193115, "operator": "ge"}, {"budget": 0.5, "threshold": 1.3022453784942627, "operator": "ge"}, {"budget": 0.51, "threshold": 1.2807934284210205, "operator": "ge"}, {"budget": 0.52, "threshold": 1.2647273540496826, "operator": "ge"}, {"budget": 0.53, "threshold": 1.2539564371109009, "operator": "ge"}, {"budget": 0.54, "threshold": 1.2427868843078613, "operator": "ge"}, {"budget": 0.55, "threshold": 1.2329230308532715, "operator": "ge"}, {"budget": 0.56, "threshold": 1.2121829986572266, "operator": "ge"}, {"budget": 0.57, "threshold": 1.1851001977920532, "operator": "ge"}, {"budget": 0.58, "threshold": 1.169776201248169, "operator": "ge"}, {"budget": 0.59, "threshold": 1.1595046520233154, "operator": "ge"}, {"budget": 0.6, "threshold": 1.1439977884292603, "operator": "ge"}, {"budget": 0.61, "threshold": 1.1232980489730835, "operator": "ge"}, {"budget": 0.62, "threshold": 1.1000477075576782, "operator": "ge"}, {"budget": 0.63, "threshold": 1.0857481956481934, "operator": "ge"}, {"budget": 0.64, "threshold": 1.0671815872192383, "operator": "ge"}, {"budget": 0.65, "threshold": 1.0277223587036133, "operator": "ge"}, {"budget": 0.66, "threshold": 1.0103071928024292, "operator": "ge"}, {"budget": 0.67, "threshold": 0.9747191667556763, "operator": "ge"}, {"budget": 0.68, "threshold": 0.9412158727645874, "operator": "ge"}, {"budget": 0.69, "threshold": 0.9019222259521484, "operator": "ge"}, {"budget": 0.7, "threshold": 0.8541272878646851, "operator": "ge"}, {"budget": 0.71, "threshold": 0.831379234790802, "operator": "ge"}, {"budget": 0.72, "threshold": 0.7504067420959473, "operator": "ge"}, {"budget": 0.73, "threshold": 0.7085189819335938, "operator": "ge"}, {"budget": 0.74, "threshold": 0.6724963784217834, "operator": "ge"}, {"budget": 0.75, "threshold": 0.6084538698196411, "operator": "ge"}, {"budget": 0.76, "threshold": 0.49122142791748047, "operator": "ge"}, {"budget": 0.77, "threshold": 0.2682567834854126, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7581188678741455, "operator": "ge"}, {"budget": 0.79, "threshold": -0.9213058948516846, "operator": "ge"}, {"budget": 0.8, "threshold": -1.1816070079803467, "operator": "ge"}, {"budget": 0.81, "threshold": -1.3314324617385864, "operator": "ge"}, {"budget": 0.82, "threshold": -1.4804710149765015, "operator": "ge"}, {"budget": 0.83, "threshold": -1.5449525117874146, "operator": "ge"}, {"budget": 0.84, "threshold": -1.5757609605789185, "operator": "ge"}, {"budget": 0.85, "threshold": -1.6610156297683716, "operator": "ge"}, {"budget": 0.86, "threshold": -1.6924488544464111, "operator": "ge"}, {"budget": 0.87, "threshold": -1.8005579710006714, "operator": "ge"}, {"budget": 0.88, "threshold": -1.8720356225967407, "operator": "ge"}, {"budget": 0.89, "threshold": -1.982909083366394, "operator": "ge"}, {"budget": 0.9, "threshold": -2.0545177459716797, "operator": "ge"}, {"budget": 0.91, "threshold": -2.1001996994018555, "operator": "ge"}, {"budget": 0.92, "threshold": -2.15311598777771, "operator": "ge"}, {"budget": 0.93, "threshold": -2.2111165523529053, "operator": "ge"}, {"budget": 0.94, "threshold": -2.233748197555542, "operator": "ge"}, {"budget": 0.95, "threshold": -2.305276393890381, "operator": "ge"}, {"budget": 0.96, "threshold": -2.348853826522827, "operator": "ge"}, {"budget": 0.97, "threshold": -2.406412124633789, "operator": "ge"}, {"budget": 0.98, "threshold": -2.476968765258789, "operator": "ge"}, {"budget": 0.99, "threshold": -2.6282663345336914, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 11.07686911145863, 10.77014622019873, 10.420605385653142, 10.430465858708864, 10.31933660203021, 12.32989438619598, 12.384370053205162, 12.446949647931497, 12.53496069927135, 12.442849548112706, 12.570135909585709, 12.421116890093522, 12.228227665588674, 12.06311911590474, 11.954111346024302, 12.031097800328569, 12.110511666646746, 13.715157977178235, 13.826804915514593, 13.671120096606057, 13.650860830566538, 13.662707968557067, 14.845067539678316, 14.587631436362203, 14.580483385227911, 14.477399618591068, 14.592754995439561, 14.4144792478305, 14.235572493315004, 15.219495414166339, 15.187953424547434, 14.832707988212684, 14.945749506465198, 14.988293618057682, 14.966254052551553, 15.07806668905807, 15.21195573892613, 15.329510475623076, 15.447443767243177, 15.630017708096009, 15.69111001109186, 16.54171009698178, 16.517430327385682, 16.687256842536346, 16.823872558172216, 17.00845042540819, 17.237831174786205, 17.374498501949486, 18.192866161616163, 18.46150173320983, 18.748075892500825, 18.82040401842382, 19.542186001917543, 19.534081104400343, 19.788271619487592, 20.03921780608215, 20.93620467010272, 21.1648281089668, 21.25487044325466, 22.048382399661122, 22.375655158263854, 23.11788980842313, 23.357634400967395, 23.479383296659044, 23.70971276458013, 23.969585786480714, 24.411627766526582, 24.40420068222759, 24.46275946275946, 24.77435996114868, 25.036711514302024, 25.137167698436098, 25.173084769167996, 25.391520990930022, 25.828380132540822, 26.331624233851958, 26.739258752325757, 26.790379957892128, 26.790379957892128, 26.790379957892128, 26.59397518699238, 26.733149659978928, 26.975049024103697, 27.08682939232138, 27.08682939232138, 26.929892425582082, 26.885683760683765, 26.733935998641883, 26.604681367017584, 26.562664560294895, 26.359924400320235, 26.280601290854342, 26.20379637533975, 26.140083597015508, 26.140083597015508, 26.102588283929645, 26.039473747655396, 25.727093126457586, 25.691314951144523, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 16, 30, 40, 55, 69, 87, 96, 109, 121, 135, 152, 169, 186, 201, 216, 227, 237, 249, 259, 275, 287, 301, 318, 329, 343, 354, 364, 377, 391, 406, 417, 427, 441, 455, 465, 473, 486, 498, 509, 518, 526, 537, 545, 553, 562, 573, 582, 591, 601, 611, 621, 628, 640, 649, 659, 666, 676, 684, 691, 698, 708, 717, 725, 728, 735, 744, 752, 758, 763, 770, 775, 779, 783, 790, 797, 804, 810, 812, 812, 812, 812, 813, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 1, 3, 6, 7, 8, 10, 10, 10, 10, 11, 11, 13, 15, 17, 19, 19, 19, 19, 19, 21, 22, 23, 25, 28, 30, 32, 33, 36, 40, 42, 44, 46, 47, 50, 52, 53, 54, 55, 57, 57, 58, 59, 61, 61, 62, 66, 66, 68, 69, 69, 69, 70, 73, 77, 78, 78, 78, 79, 81, 83, 85, 88, 90, 94, 96, 99, 99, 103, 105, 111, 116, 119, 127, 132, 134, 135, 141, 143, 143, 143, 147, 151, 154, 155, 155, 158, 159, 162, 164, 165, 168, 170, 172, 173, 173, 174, 175, 180, 181, 185], "exact_score": [7.641657905799261, 11.07686911145863, 10.77014622019873, 10.420605385653142, 10.430465858708864, 10.31933660203021, 12.32989438619598, 12.384370053205162, 12.446949647931497, 12.53496069927135, 12.442849548112706, 12.570135909585709, 12.421116890093522, 12.228227665588674, 12.06311911590474, 11.954111346024302, 12.031097800328569, 12.110511666646746, 13.715157977178235, 13.826804915514593, 13.671120096606057, 13.650860830566538, 13.662707968557067, 14.845067539678316, 14.587631436362203, 14.580483385227911, 14.477399618591068, 14.592754995439561, 14.4144792478305, 14.235572493315004, 15.219495414166339, 15.187953424547434, 14.832707988212684, 14.945749506465198, 14.988293618057682, 14.966254052551553, 15.07806668905807, 15.21195573892613, 15.329510475623076, 15.447443767243177, 15.630017708096009, 15.69111001109186, 16.54171009698178, 16.517430327385682, 16.687256842536346, 16.823872558172216, 17.00845042540819, 17.237831174786205, 17.374498501949486, 18.192866161616163, 18.46150173320983, 18.748075892500825, 18.82040401842382, 19.542186001917543, 19.534081104400343, 19.788271619487592, 20.03921780608215, 20.93620467010272, 21.1648281089668, 21.25487044325466, 22.048382399661122, 22.375655158263854, 23.11788980842313, 23.357634400967395, 23.479383296659044, 23.70971276458013, 23.969585786480714, 24.411627766526582, 24.40420068222759, 24.46275946275946, 24.77435996114868, 25.036711514302024, 25.137167698436098, 25.173084769167996, 25.391520990930022, 25.828380132540822, 26.331624233851958, 26.739258752325757, 26.790379957892128, 26.790379957892128, 26.790379957892128, 26.59397518699238, 26.733149659978928, 26.975049024103697, 27.08682939232138, 27.08682939232138, 26.929892425582082, 26.885683760683765, 26.733935998641883, 26.604681367017584, 26.562664560294895, 26.359924400320235, 26.280601290854342, 26.20379637533975, 26.140083597015508, 26.140083597015508, 26.102588283929645, 26.039473747655396, 25.727093126457586, 25.691314951144523, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.284377378347127, "threshold_area100": 19.27539795947786, "threshold_area25": 11.899697647657371}, "test": {"score": [7.8873978710439285, 7.920737928310809, 8.754972819008287, 8.780470143420178, 9.507053744475995, 9.497764622790292, 10.138428387207023, 10.195599193599861, 10.117903005302322, 10.74124135177337, 10.707683156677565, 11.239935666543241, 11.188266340805407, 11.180130171383523, 11.19427123383432, 11.210453903545554, 11.622322892142373, 11.65512749541096, 12.021759026927668, 11.997291927185698, 12.35509105733997, 12.66739253135302, 12.705468619715484, 13.04292507668876, 13.043868495690175, 13.0268551212804, 13.050870409393506, 13.022193355848271, 12.966142950194321, 13.227849285340119, 13.238458450043955, 13.147061451888526, 13.112149989391932, 13.189280423766894, 13.225249559710406, 13.84686795270142, 13.919846356766326, 14.23229554459811, 14.545789878787316, 14.871400232372482, 14.91132066483963, 15.220468338653385, 15.550200288040747, 15.962576324089097, 16.37801450885563, 16.464228145595587, 16.964981401359676, 17.345905688184395, 17.815398500479148, 18.516643918247798, 18.52378457380713, 18.67264014983565, 18.818871031094904, 18.903513967232456, 19.071893990662954, 19.34253769318302, 19.638107969951278, 20.173957169441834, 20.42341467018643, 20.61552508964072, 21.042303221166396, 21.664046110974507, 21.91052158293381, 22.058793941531757, 22.30010560852215, 22.741755458320807, 23.168688699582997, 23.42106850178733, 23.709638615501394, 24.44914521395087, 24.77820662329874, 25.01141394806868, 25.621757379875476, 25.471423988271685, 25.61456100905799, 26.030309195752633, 26.551928531660497, 26.84232776058164, 27.03438042302786, 27.03438042302786, 27.010270001268005, 26.903958001148542, 26.925677374281666, 26.82386729024196, 26.807257098228142, 26.734122843277774, 26.703222494580814, 26.710330435637275, 27.258279989789862, 27.153949252819366, 27.187972536682857, 27.10188329626142, 27.060616254139408, 27.063341940391123, 27.05622155622156, 26.97720949163423, 26.95433615759619, 26.899131561932688, 26.89740039703381, 26.9840565003097, 27.001009067886244], "calls": [0.0, 1.2982689747003995, 2.2969374167776295, 2.862849533954727, 3.4287616511318246, 4.027962716378163, 4.826897470039946, 5.958721704394141, 6.990679094540613, 7.723035952063914, 8.5219707057257, 9.121171770972039, 9.920106524633821, 10.952063914780293, 11.384820239680426, 12.749667110519308, 13.448735019973368, 14.08122503328895, 14.81358189081225, 15.512649800266312, 16.67776298268975, 18.10918774966711, 18.908122503328894, 20.272969374167776, 21.770972037283624, 22.63648468708389, 23.40213049267643, 24.30093209054594, 24.93342210386152, 27.19707057256991, 27.862849533954726, 28.428761651131822, 29.22769640479361, 30.29294274300932, 31.69107856191744, 34.05459387483356, 35.28628495339547, 36.8508655126498, 37.64980026631159, 38.98135818908123, 39.74700399467377, 41.37816245006658, 42.310252996005325, 43.70838881491345, 45.80559254327563, 46.43808255659121, 47.53661784287617, 48.568575233022635, 49.60053262316911, 50.93209054593875, 52.23035952063915, 53.76165113182424, 54.89347536617842, 55.426098535286286, 56.29161118508655, 56.89081225033289, 58.28894806924101, 59.886817576564574, 60.75233022636485, 61.18508655126498, 61.817576564580555, 62.982689747003995, 64.08122503328894, 64.61384820239681, 65.41278295605859, 66.51131824234355, 67.31025299600533, 68.37549933422103, 69.47403462050599, 70.67243675099867, 71.90412782956058, 72.47003994673769, 74.33422103861518, 74.93342210386152, 75.13315579227697, 75.79893475366178, 76.99733688415446, 78.06258322237018, 78.8948069241012, 79.26098535286285, 79.99334221038616, 80.95872170439414, 81.99067909454061, 82.55659121171772, 82.98934753661784, 84.05459387483354, 84.75366178428762, 86.21837549933421, 87.7496671105193, 89.64713715046605, 90.41278295605859, 91.37816245006658, 92.21038615179761, 93.50865512649801, 93.80825565912117, 94.84021304926765, 95.40612516644474, 95.83888149134488, 96.83754993342211, 98.26897470039947, 100.0], "rescued": [0, 70, 129, 161, 192, 226, 273, 338, 399, 440, 485, 518, 565, 626, 650, 730, 770, 804, 850, 888, 952, 1029, 1069, 1145, 1227, 1264, 1303, 1355, 1386, 1493, 1519, 1544, 1583, 1635, 1702, 1802, 1849, 1916, 1950, 2012, 2045, 2110, 2150, 2212, 2304, 2327, 2367, 2408, 2457, 2513, 2561, 2614, 2651, 2672, 2710, 2730, 2784, 2850, 2886, 2901, 2929, 2975, 3012, 3030, 3055, 3091, 3122, 3157, 3191, 3235, 3270, 3288, 3333, 3345, 3351, 3372, 3405, 3431, 3447, 3447, 3447, 3447, 3448, 3448, 3448, 3449, 3449, 3450, 3454, 3455, 3456, 3456, 3456, 3457, 3457, 3458, 3458, 3458, 3459, 3460, 3461], "harmed": [0, 5, 7, 9, 10, 12, 14, 19, 25, 26, 30, 32, 36, 40, 41, 46, 49, 50, 56, 60, 66, 74, 76, 83, 90, 94, 97, 104, 109, 127, 130, 139, 149, 153, 164, 176, 182, 194, 196, 204, 210, 220, 223, 226, 244, 247, 250, 260, 267, 274, 286, 300, 309, 313, 321, 326, 338, 349, 354, 359, 364, 377, 387, 390, 398, 403, 415, 430, 442, 461, 478, 482, 495, 507, 510, 516, 525, 543, 553, 553, 555, 562, 566, 572, 573, 582, 585, 588, 596, 606, 612, 620, 623, 629, 630, 636, 638, 642, 645, 658, 676], "exact_score": [7.8873978710439285, 7.8934667274551975, 8.726101236914378, 8.787289966680586, 9.51627070881575, 10.15086571954358, 10.1990362125281, 10.117903005302322, 10.713955698532668, 11.230259675648716, 11.16892974473725, 11.182217592278812, 11.179897428618418, 11.235730206446577, 11.645057102209035, 11.984532477683162, 12.399586422431438, 12.308977973682804, 12.652469310682992, 12.710706618405974, 13.031129644232529, 13.032746147139147, 13.041610128729333, 12.992583981327252, 12.995631724307454, 12.974852424500764, 12.91603664974888, 13.206962402059002, 13.230454515204148, 13.100311594452904, 13.167097530863694, 13.214313674569967, 13.24950699023755, 13.477761925825169, 13.834913230668644, 13.882921635391684, 14.203009346629036, 14.475269252425468, 14.558547475421541, 14.871400232372482, 14.88805199378973, 15.180055549666694, 15.323132975396318, 15.65012307919347, 16.0940474901248, 16.335013918933676, 16.397196393458568, 16.684492321942358, 17.01702674400667, 17.540502489849636, 18.045855448203437, 18.52728803898273, 18.516112022555646, 18.55852758382155, 18.716774161961567, 18.824607924152268, 19.013218647801924, 19.368292423523293, 19.55523539530744, 19.78532133762408, 20.212620609893303, 20.465667631609918, 21.076841449932765, 21.664046110974507, 21.870890576348806, 22.231770201303483, 22.565484785865017, 22.895312132517546, 23.352222652664683, 23.585789702747732, 24.05225947651447, 24.54282098783031, 24.813424052054913, 25.158303219822255, 25.520241963275964, 25.58014157401352, 26.105388803356817, 26.551928531660497, 26.80801562487592, 27.03438042302786, 27.010270001268005, 26.903958001148542, 26.925677374281666, 26.807257098228142, 26.74159093585225, 26.703222494580814, 26.710330435637275, 27.22845330104472, 27.22327640733713, 27.16123347980298, 27.240677372553733, 27.10188329626142, 27.060616254139408, 27.104115838119498, 27.01419463715865, 26.96140499160024, 26.93647269290642, 26.89045801557094, 27.02694595778352, 26.870333214620505, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 18.58732482210149, "threshold_area100": 18.68536518016434, "threshold_area25": 10.84562384730311}, "selected_budget": 84, "selected": {"val": {"score": 27.08682939232138, "calls": 83.88017118402283, "rescued": 815, "harmed": 155}, "test": {"score": 26.807257098228142, "calls": 82.98934753661784, "rescued": 3448, "harmed": 573}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.6738648414611816, "operator": "ge"}, {"budget": 0.02, "threshold": 1.5823087692260742, "operator": "ge"}, {"budget": 0.03, "threshold": 1.5556724071502686, "operator": "ge"}, {"budget": 0.04, "threshold": 1.5077645778656006, "operator": "ge"}, {"budget": 0.05, "threshold": 1.4729232788085938, "operator": "ge"}, {"budget": 0.06, "threshold": 1.4471030235290527, "operator": "ge"}, {"budget": 0.07, "threshold": 1.4296705722808838, "operator": "ge"}, {"budget": 0.08, "threshold": 1.4117727279663086, "operator": "ge"}, {"budget": 0.09, "threshold": 1.3944817781448364, "operator": "ge"}, {"budget": 0.1, "threshold": 1.38448965549469, "operator": "ge"}, {"budget": 0.11, "threshold": 1.366002082824707, "operator": "ge"}, {"budget": 0.12, "threshold": 1.3527588844299316, "operator": "ge"}, {"budget": 0.13, "threshold": 1.347733974456787, "operator": "ge"}, {"budget": 0.14, "threshold": 1.3377554416656494, "operator": "ge"}, {"budget": 0.15, "threshold": 1.3237606287002563, "operator": "ge"}, {"budget": 0.16, "threshold": 1.3165199756622314, "operator": "ge"}, {"budget": 0.17, "threshold": 1.3078945875167847, "operator": "ge"}, {"budget": 0.18, "threshold": 1.300981044769287, "operator": "ge"}, {"budget": 0.19, "threshold": 1.2865748405456543, "operator": "ge"}, {"budget": 0.2, "threshold": 1.275539755821228, "operator": "ge"}, {"budget": 0.21, "threshold": 1.2693428993225098, "operator": "ge"}, {"budget": 0.22, "threshold": 1.2583284378051758, "operator": "ge"}, {"budget": 0.23, "threshold": 1.2446117401123047, "operator": "ge"}, {"budget": 0.24, "threshold": 1.2371907234191895, "operator": "ge"}, {"budget": 0.25, "threshold": 1.2129031419754028, "operator": "ge"}, {"budget": 0.26, "threshold": 1.1992043256759644, "operator": "ge"}, {"budget": 0.27, "threshold": 1.1875585317611694, "operator": "ge"}, {"budget": 0.28, "threshold": 1.1691029071807861, "operator": "ge"}, {"budget": 0.29, "threshold": 1.1577820777893066, "operator": "ge"}, {"budget": 0.3, "threshold": 1.1482840776443481, "operator": "ge"}, {"budget": 0.31, "threshold": 1.1452045440673828, "operator": "ge"}, {"budget": 0.32, "threshold": 1.1212825775146484, "operator": "ge"}, {"budget": 0.33, "threshold": 1.0920121669769287, "operator": "ge"}, {"budget": 0.34, "threshold": 1.0821503400802612, "operator": "ge"}, {"budget": 0.35, "threshold": 1.0688427686691284, "operator": "ge"}, {"budget": 0.36, "threshold": 1.0387451648712158, "operator": "ge"}, {"budget": 0.37, "threshold": 1.0288465023040771, "operator": "ge"}, {"budget": 0.38, "threshold": 1.0201475620269775, "operator": "ge"}, {"budget": 0.39, "threshold": 1.0077862739562988, "operator": "ge"}, {"budget": 0.4, "threshold": 1.0011361837387085, "operator": "ge"}, {"budget": 0.41, "threshold": 0.9930708408355713, "operator": "ge"}, {"budget": 0.42, "threshold": 0.9866324663162231, "operator": "ge"}, {"budget": 0.43, "threshold": 0.9802389144897461, "operator": "ge"}, {"budget": 0.44, "threshold": 0.9593970775604248, "operator": "ge"}, {"budget": 0.45, "threshold": 0.9492758512496948, "operator": "ge"}, {"budget": 0.46, "threshold": 0.9233018159866333, "operator": "ge"}, {"budget": 0.47, "threshold": 0.9132933616638184, "operator": "ge"}, {"budget": 0.48, "threshold": 0.9061889052391052, "operator": "ge"}, {"budget": 0.49, "threshold": 0.8913512229919434, "operator": "ge"}, {"budget": 0.5, "threshold": 0.8766651153564453, "operator": "ge"}, {"budget": 0.51, "threshold": 0.8672325611114502, "operator": "ge"}, {"budget": 0.52, "threshold": 0.8616023063659668, "operator": "ge"}, {"budget": 0.53, "threshold": 0.8547050952911377, "operator": "ge"}, {"budget": 0.54, "threshold": 0.8451035022735596, "operator": "ge"}, {"budget": 0.55, "threshold": 0.8291616439819336, "operator": "ge"}, {"budget": 0.56, "threshold": 0.8229683637619019, "operator": "ge"}, {"budget": 0.57, "threshold": 0.8184551000595093, "operator": "ge"}, {"budget": 0.58, "threshold": 0.8039280772209167, "operator": "ge"}, {"budget": 0.59, "threshold": 0.7909321784973145, "operator": "ge"}, {"budget": 0.6, "threshold": 0.7858525514602661, "operator": "ge"}, {"budget": 0.61, "threshold": 0.7721346616744995, "operator": "ge"}, {"budget": 0.62, "threshold": 0.7585951089859009, "operator": "ge"}, {"budget": 0.63, "threshold": 0.7454633712768555, "operator": "ge"}, {"budget": 0.64, "threshold": 0.7382775545120239, "operator": "ge"}, {"budget": 0.65, "threshold": 0.728458046913147, "operator": "ge"}, {"budget": 0.66, "threshold": 0.7107888460159302, "operator": "ge"}, {"budget": 0.67, "threshold": 0.6871039271354675, "operator": "ge"}, {"budget": 0.68, "threshold": 0.6664149761199951, "operator": "ge"}, {"budget": 0.69, "threshold": 0.6564322710037231, "operator": "ge"}, {"budget": 0.7, "threshold": 0.6260879635810852, "operator": "ge"}, {"budget": 0.71, "threshold": 0.5914916396141052, "operator": "ge"}, {"budget": 0.72, "threshold": 0.5675815343856812, "operator": "ge"}, {"budget": 0.73, "threshold": 0.5377151966094971, "operator": "ge"}, {"budget": 0.74, "threshold": 0.5171186327934265, "operator": "ge"}, {"budget": 0.75, "threshold": 0.4912448525428772, "operator": "ge"}, {"budget": 0.76, "threshold": 0.415090411901474, "operator": "ge"}, {"budget": 0.77, "threshold": 0.2491077184677124, "operator": "ge"}, {"budget": 0.78, "threshold": -0.6662198305130005, "operator": "ge"}, {"budget": 0.79, "threshold": -0.7804535627365112, "operator": "ge"}, {"budget": 0.8, "threshold": -1.0571091175079346, "operator": "ge"}, {"budget": 0.81, "threshold": -1.2418465614318848, "operator": "ge"}, {"budget": 0.82, "threshold": -1.329927682876587, "operator": "ge"}, {"budget": 0.83, "threshold": -1.3878604173660278, "operator": "ge"}, {"budget": 0.84, "threshold": -1.429650068283081, "operator": "ge"}, {"budget": 0.85, "threshold": -1.5053260326385498, "operator": "ge"}, {"budget": 0.86, "threshold": -1.54570472240448, "operator": "ge"}, {"budget": 0.87, "threshold": -1.6284925937652588, "operator": "ge"}, {"budget": 0.88, "threshold": -1.7021102905273438, "operator": "ge"}, {"budget": 0.89, "threshold": -1.8081893920898438, "operator": "ge"}, {"budget": 0.9, "threshold": -1.878281593322754, "operator": "ge"}, {"budget": 0.91, "threshold": -1.940794587135315, "operator": "ge"}, {"budget": 0.92, "threshold": -1.983496904373169, "operator": "ge"}, {"budget": 0.93, "threshold": -2.02614688873291, "operator": "ge"}, {"budget": 0.94, "threshold": -2.0531883239746094, "operator": "ge"}, {"budget": 0.95, "threshold": -2.123533248901367, "operator": "ge"}, {"budget": 0.96, "threshold": -2.164841413497925, "operator": "ge"}, {"budget": 0.97, "threshold": -2.229206085205078, "operator": "ge"}, {"budget": 0.98, "threshold": -2.2818028926849365, "operator": "ge"}, {"budget": 0.99, "threshold": -2.434922695159912, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.753350655836484, 7.669895080292037, 7.78290195041082, 7.792774722572863, 7.863860110797427, 7.964152055549012, 8.04736407792834, 8.163203958066452, 8.285385431858879, 8.306868733329276, 8.329199588139353, 8.449833815567832, 10.25867671697576, 10.30641887584384, 10.353525435025828, 11.80047129091715, 13.110408302898863, 13.115540358074892, 13.04466216446222, 12.833233908911563, 12.901628222523746, 12.93863609103734, 14.081196060962387, 14.077438556067587, 14.190208875646388, 14.213908725478284, 14.226396062703628, 14.321446845718691, 14.393499967234181, 14.5513556696612, 14.531885351766565, 14.50004960712402, 14.642366224840453, 14.798918826884606, 14.916219540747871, 15.056770849848434, 15.21852370262457, 15.305668384382972, 15.487955935062198, 15.586339272779954, 15.692588381161626, 16.44618456214469, 16.68980577336015, 17.48747670083877, 17.522414942631677, 18.370426561083157, 18.578190746905765, 18.751125001125, 19.04365912073363, 19.172438176050147, 19.97964448920817, 20.193723212103947, 20.39892375696675, 20.636829415310427, 20.753949599122084, 20.86657849894453, 20.929636082075106, 21.125541125541126, 21.331422200987422, 21.56653706258623, 21.80891265597148, 22.615604210031453, 22.732346211153683, 23.12523186595644, 23.082199539350306, 23.817320940735726, 24.2325759439242, 24.726021493262873, 24.79012474980217, 24.94080408414115, 25.16200227308675, 25.19202610550977, 25.38176189586614, 25.80252363609609, 26.062471104946834, 26.400586353944565, 26.739258752325757, 26.790379957892128, 26.790379957892128, 26.790379957892128, 26.59397518699238, 26.733149659978928, 26.975049024103697, 27.08682939232138, 27.020636527215476, 26.885683760683765, 26.885683760683765, 26.733935998641883, 26.669000933706815, 26.521342246245304, 26.4407133407827, 26.24189161343499, 26.20379637533975, 26.140083597015508, 26.140083597015508, 26.076969060741256, 26.002564298836496, 25.727093126457586, 25.691314951144523, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 18, 34, 51, 69, 82, 96, 110, 127, 143, 158, 174, 190, 208, 221, 238, 252, 265, 278, 292, 307, 321, 333, 346, 361, 376, 391, 403, 414, 424, 434, 445, 454, 465, 475, 484, 492, 503, 513, 522, 529, 536, 548, 558, 568, 574, 586, 594, 601, 611, 618, 626, 634, 644, 652, 661, 668, 674, 682, 688, 697, 706, 713, 721, 730, 738, 744, 751, 760, 767, 773, 778, 783, 788, 794, 798, 805, 810, 812, 812, 812, 812, 813, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 2, 6, 7, 10, 10, 10, 10, 13, 13, 16, 17, 19, 20, 21, 23, 24, 25, 26, 29, 33, 36, 37, 37, 40, 41, 43, 45, 46, 47, 47, 50, 52, 53, 54, 55, 55, 56, 58, 58, 60, 65, 69, 69, 70, 71, 72, 72, 73, 73, 78, 78, 79, 81, 82, 84, 87, 88, 90, 91, 94, 95, 96, 99, 100, 102, 102, 105, 105, 109, 112, 117, 123, 126, 127, 133, 135, 141, 143, 143, 143, 147, 151, 154, 155, 156, 159, 159, 162, 163, 166, 168, 171, 172, 173, 173, 174, 176, 180, 181, 185], "exact_score": [7.641657905799261, 7.753350655836484, 7.669895080292037, 7.78290195041082, 7.792774722572863, 7.863860110797427, 7.964152055549012, 8.04736407792834, 8.163203958066452, 8.285385431858879, 8.306868733329276, 8.329199588139353, 8.449833815567832, 10.25867671697576, 10.30641887584384, 10.353525435025828, 11.80047129091715, 13.110408302898863, 13.115540358074892, 13.04466216446222, 12.833233908911563, 12.901628222523746, 12.93863609103734, 14.081196060962387, 14.077438556067587, 14.190208875646388, 14.213908725478284, 14.226396062703628, 14.321446845718691, 14.393499967234181, 14.5513556696612, 14.531885351766565, 14.50004960712402, 14.642366224840453, 14.798918826884606, 14.916219540747871, 15.056770849848434, 15.21852370262457, 15.305668384382972, 15.487955935062198, 15.586339272779954, 15.692588381161626, 16.44618456214469, 16.68980577336015, 17.48747670083877, 17.522414942631677, 18.370426561083157, 18.578190746905765, 18.751125001125, 19.04365912073363, 19.172438176050147, 19.97964448920817, 20.193723212103947, 20.39892375696675, 20.636829415310427, 20.753949599122084, 20.86657849894453, 20.929636082075106, 21.125541125541126, 21.331422200987422, 21.56653706258623, 21.80891265597148, 22.615604210031453, 22.732346211153683, 23.12523186595644, 23.082199539350306, 23.817320940735726, 24.2325759439242, 24.726021493262873, 24.79012474980217, 24.94080408414115, 25.16200227308675, 25.19202610550977, 25.38176189586614, 25.80252363609609, 26.062471104946834, 26.400586353944565, 26.739258752325757, 26.790379957892128, 26.790379957892128, 26.790379957892128, 26.59397518699238, 26.733149659978928, 26.975049024103697, 27.08682939232138, 27.020636527215476, 26.885683760683765, 26.885683760683765, 26.733935998641883, 26.669000933706815, 26.521342246245304, 26.4407133407827, 26.24189161343499, 26.20379637533975, 26.140083597015508, 26.140083597015508, 26.076969060741256, 26.002564298836496, 25.727093126457586, 25.691314951144523, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 18.839685549118194, "threshold_area100": 18.830070856207172, "threshold_area25": 10.054684315757385}, "test": {"score": [7.8873978710439285, 7.966457162178009, 8.84486981316467, 9.544519490513, 9.555854393235402, 10.789832527080641, 11.13976261042854, 11.014416757658276, 11.433300835489838, 11.4517229059536, 11.398234619386926, 11.388784350815085, 11.356570809755834, 11.704253096061654, 11.757555001298952, 12.147360978975364, 12.467827534374248, 12.748978571404468, 12.772244544692876, 12.62376569783619, 12.552800593797581, 12.57024237096029, 12.480731135832945, 13.136051228128343, 13.128893059271338, 13.356711334340133, 13.40217684821368, 13.46810471871797, 13.812121042920342, 13.834655307829907, 13.872621815557235, 13.912544024265314, 13.956026719160471, 14.536703224801723, 14.539823096455898, 14.879378586472408, 15.135349685410388, 15.439545349164513, 15.615566730908945, 15.943034121954097, 15.886860220955842, 15.964415963934563, 16.097660229336654, 16.322508490276054, 17.07706774027439, 17.362053240333513, 18.20669439206556, 18.554120939160534, 18.693843041185207, 18.91067021694024, 19.23905962133188, 19.41414591949828, 19.647337418626687, 19.73034989602918, 19.97407807420143, 20.20102085562013, 20.466840527830975, 20.593659128181287, 20.9299693854642, 21.24119099355856, 21.393994770304957, 21.92323558249184, 22.412935992640904, 22.56928091327167, 22.706292168609572, 22.878704023982475, 23.27501791474316, 23.8302514828289, 24.35110584829614, 24.512856236026455, 24.7712331684905, 25.370039166996015, 25.63689163149514, 25.947728086579303, 26.158916548444644, 26.345543149823126, 26.528005833596925, 26.7913068814466, 27.03438042302786, 27.03438042302786, 27.002787556235198, 26.887612283249386, 26.908677510280583, 26.868014749011536, 26.791053285009614, 26.757824702086015, 26.68710011930351, 26.710330435637275, 27.2435173443238, 27.153949252819366, 27.211219399262294, 27.067725451409473, 27.053533145666663, 26.996859251282817, 27.040372647629855, 27.009940904409753, 26.96140499160024, 26.943490534544743, 26.929480140787515, 26.743815431922773, 27.001009067886244], "calls": [0.0, 0.7323568575233023, 1.8308921438082555, 2.097203728362184, 2.862849533954727, 4.061251664447404, 4.593874833555259, 5.3928095872170445, 5.9920106524633825, 6.7576564580559255, 7.523302263648468, 8.85486018641811, 9.85352862849534, 10.252996005326231, 11.118508655126497, 12.083888149134488, 12.649800266311583, 13.448735019973368, 14.247669773635153, 15.412782956058587, 16.311584553928096, 16.611185086551263, 17.343541944074566, 18.242343541944077, 18.841544607190414, 21.205059920106525, 21.97070572569907, 22.769640479360852, 24.201065246338217, 24.766977363515313, 25.499334221038616, 25.732356857523303, 27.529960053262315, 30.22636484687084, 31.05858854860186, 32.689747003994675, 35.15312916111851, 36.18508655126498, 37.050599201065246, 38.18242343541944, 38.64846870838882, 39.34753661784288, 40.279627163781626, 40.812250332889484, 43.20905459387483, 44.207723035952064, 46.80426098535286, 48.202396804260985, 49.06790945406125, 50.59920106524633, 52.06391478029294, 53.02929427430093, 53.495339547270305, 54.16111850865513, 55.29294274300932, 57.25699067909454, 57.922769640479366, 58.48868175765646, 60.153129161118514, 61.55126498002663, 62.217043941411454, 63.715046604527295, 64.98002663115847, 65.84553928095872, 66.3115845539281, 67.0439414114514, 68.20905459387482, 69.50732356857523, 70.80559254327564, 71.07190412782957, 71.90412782956058, 73.16910785619174, 74.03462050599201, 74.80026631158455, 75.16644474034621, 75.7323568575233, 76.96404793608522, 77.96271637816245, 78.8948069241012, 79.16111850865512, 80.0266311584554, 81.19174434087883, 81.99067909454061, 82.49001331557922, 83.08921438082557, 84.05459387483354, 84.95339547270306, 85.9520639147803, 87.68308921438083, 89.54727030625833, 90.27962716378163, 91.6444740346205, 92.37683089214381, 93.24234354194407, 93.70838881491345, 94.6404793608522, 95.27296937416777, 95.90545938748336, 96.47137150466045, 98.20239680426099, 100.0], "rescued": [0, 55, 139, 154, 205, 289, 321, 373, 414, 458, 508, 594, 660, 680, 731, 792, 824, 864, 911, 977, 1031, 1049, 1088, 1140, 1177, 1304, 1345, 1390, 1456, 1482, 1517, 1529, 1611, 1736, 1768, 1847, 1948, 1994, 2029, 2074, 2096, 2123, 2161, 2181, 2271, 2307, 2410, 2463, 2490, 2541, 2594, 2635, 2650, 2673, 2719, 2788, 2809, 2830, 2888, 2931, 2954, 3004, 3053, 3080, 3097, 3122, 3165, 3209, 3249, 3257, 3282, 3311, 3332, 3351, 3360, 3375, 3404, 3428, 3447, 3447, 3447, 3447, 3448, 3448, 3448, 3449, 3449, 3450, 3454, 3455, 3456, 3456, 3456, 3456, 3457, 3458, 3458, 3459, 3459, 3459, 3461], "harmed": [0, 3, 6, 7, 12, 16, 23, 30, 34, 36, 40, 46, 52, 54, 55, 60, 64, 69, 71, 81, 89, 90, 99, 104, 108, 126, 130, 133, 140, 143, 148, 150, 161, 177, 183, 194, 208, 214, 229, 233, 239, 243, 246, 249, 265, 270, 287, 298, 308, 318, 330, 337, 338, 345, 353, 372, 375, 377, 383, 388, 392, 403, 414, 423, 428, 436, 446, 461, 467, 470, 477, 488, 496, 503, 508, 516, 527, 542, 553, 553, 556, 563, 567, 570, 574, 580, 586, 588, 598, 606, 610, 622, 624, 628, 631, 634, 637, 641, 643, 657, 676], "exact_score": [7.8873978710439285, 7.999057226023007, 9.534841416822056, 9.552539444898818, 10.814988572115963, 11.088283617259458, 11.433300835489838, 11.478704555717236, 11.374586291876383, 11.351160478001123, 11.708862007270104, 11.741585958962979, 12.134801909043475, 12.436655686450623, 12.780690219270783, 12.653731671186122, 12.589498750884642, 12.501819200970779, 13.143353611768418, 13.142811949857498, 13.164117806629577, 13.084748627974108, 13.40217684821368, 13.476020183565273, 13.7969113041542, 13.8727592155625, 13.95227564146484, 13.94661372610062, 13.96177265790734, 14.21456168635771, 14.508156665312832, 14.530729433590569, 14.709076659003035, 14.914323864926146, 14.993720422539633, 15.127389841904455, 15.404196485750493, 15.61668417715012, 15.934051474403315, 15.91478408017224, 16.07961043459769, 16.351711825456437, 16.611820027219824, 17.052375643793006, 17.37411064098765, 17.454344419560286, 17.77733131264807, 18.226347097923384, 18.511259413536685, 18.679625317450192, 18.79448731286984, 18.947818448420513, 19.234889443910372, 19.41093832391193, 19.70264902552419, 19.887903847692524, 20.106567230175354, 20.12222406141403, 20.608789802148245, 20.700599196059212, 20.870632340542887, 21.13392462438203, 21.33104932024602, 21.79085212015872, 22.026387412882343, 22.412935992640904, 22.6113977699414, 22.856472340989853, 23.183809423646704, 23.567255059035283, 24.026469689548723, 24.487612810735413, 24.7812239667366, 25.10010310443322, 25.6304062425893, 26.116434469810834, 26.463225414543118, 26.544447105374136, 26.80801562487592, 27.03438042302786, 27.002787556235198, 26.903958001148542, 26.908677510280583, 26.791053285009614, 26.757824702086015, 26.68710011930351, 26.710330435637275, 27.2769019637191, 27.2435173443238, 27.19967843705799, 27.233271428048944, 27.150330080131834, 27.067725451409473, 27.003890611147817, 27.02455753202022, 27.002846365554777, 26.943490534544743, 26.89045801557094, 26.784079662829026, 26.895712636984854, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.15613481880783, "threshold_area100": 19.14077951841334, "threshold_area25": 11.502056664386508}, "selected_budget": 84, "selected": {"val": {"score": 27.08682939232138, "calls": 83.88017118402283, "rescued": 815, "harmed": 155}, "test": {"score": 26.791053285009614, "calls": 83.08921438082557, "rescued": 3448, "harmed": 574}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 26.99586861016554, "calls": 79.88587731811697}, "test": {"score": 26.926889477331827, "calls": 80.99201065246339}}, "old_normalized_interior": {"val": {"score": 26.99586861016554, "calls": 79.88587731811697}, "test": {"score": 26.926889477331827, "calls": 80.99201065246339}}, "old_normalized_endpoints": {"val": {"score": 26.99586861016554, "calls": 79.88587731811697}, "test": {"score": 26.926889477331827, "calls": 80.99201065246339}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 10.887403481598755, 10.317646245239498, 9.986331472884679, 9.817820433091502, 9.544345915520971, 9.568103465162288, 9.652677038667166, 9.745334053638032, 9.805627693618518, 11.070818436697047, 11.21005135882561, 11.181115209584165, 11.21127546000104, 11.275317593480343, 12.403313360905699, 12.5287711944864, 12.636210070648469, 12.686629717212423, 12.784128299495478, 12.775577787895815, 12.93933937890199, 12.930354202059856, 13.930695774079105, 14.007069989137907, 14.024012697653026, 14.104537855229292, 15.102385645824254, 14.975889789535907, 15.088932557377873, 15.164400462425892, 15.308466700410623, 15.289004374092752, 15.416495821618245, 15.552557835659353, 15.704709857174976, 15.895892725161016, 16.622942032625044, 16.72309023590278, 17.48687078994949, 18.33306822860137, 18.447444368497003, 18.65153590474103, 18.767042207152464, 19.04797047640389, 19.384975671282888, 19.797700851842663, 19.96662481535518, 20.19578287808688, 20.430546867613465, 20.54209187536615, 20.743519532676157, 21.45866341066119, 21.646683067076275, 21.84569377990431, 22.13149350649351, 22.312541473125417, 22.4400183395253, 22.660918441072717, 22.816331732446137, 22.993615386292294, 23.734169559144554, 22.93873607937456, 23.25073372485952, 23.247481926097162, 23.509493897670303, 23.712526525826554, 23.901613742620466, 24.051782682512734, 24.35745297447425, 24.409426884076808, 24.55209113849917, 24.76606939171802, 25.970493701377517, 26.418996238497872, 26.63969497533991, 26.832549543987938, 26.836625528947593, 26.88788788788789, 26.88788788788789, 26.99586861016554, 26.925675023642714, 26.677402318254988, 26.635700900406782, 26.594750859456745, 26.48638285833408, 26.84665854755714, 26.715068415375008, 26.606280193236714, 26.606280193236714, 26.458333333333332, 26.458333333333332, 26.458333333333332, 26.669000933706815, 26.521342246245304, 26.416984568855895, 26.092798366499753, 26.01599345098516, 25.800339784578807, 25.580499348307644, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 10.887403481598755, 10.317646245239498, 9.986331472884679, 9.817820433091502, 9.544345915520971, 9.568103465162288, 9.652677038667166, 9.745334053638032, 9.805627693618518, 11.070818436697047, 11.21005135882561, 11.181115209584165, 11.21127546000104, 11.275317593480343, 12.403313360905699, 12.5287711944864, 12.636210070648469, 12.686629717212423, 12.784128299495478, 12.775577787895815, 12.93933937890199, 12.930354202059856, 13.930695774079105, 14.007069989137907, 14.024012697653026, 14.104537855229292, 15.102385645824254, 14.975889789535907, 15.088932557377873, 15.164400462425892, 15.308466700410623, 15.289004374092752, 15.416495821618245, 15.552557835659353, 15.704709857174976, 15.895892725161016, 16.622942032625044, 16.72309023590278, 17.48687078994949, 18.33306822860137, 18.447444368497003, 18.65153590474103, 18.767042207152464, 19.04797047640389, 19.384975671282888, 19.797700851842663, 19.96662481535518, 20.19578287808688, 20.430546867613465, 20.54209187536615, 20.743519532676157, 21.45866341066119, 21.646683067076275, 21.84569377990431, 22.13149350649351, 22.312541473125417, 22.4400183395253, 22.660918441072717, 22.816331732446137, 22.993615386292294, 23.734169559144554, 22.93873607937456, 23.25073372485952, 23.247481926097162, 23.509493897670303, 23.712526525826554, 23.901613742620466, 24.051782682512734, 24.35745297447425, 24.409426884076808, 24.55209113849917, 24.76606939171802, 25.970493701377517, 26.418996238497872, 26.63969497533991, 26.832549543987938, 26.836625528947593, 26.88788788788789, 26.88788788788789, 26.99586861016554, 26.925675023642714, 26.677402318254988, 26.635700900406782, 26.594750859456745, 26.48638285833408, 26.84665854755714, 26.715068415375008, 26.606280193236714, 26.606280193236714, 26.458333333333332, 26.458333333333332, 26.458333333333332, 26.669000933706815, 26.521342246245304, 26.416984568855895, 26.092798366499753, 26.01599345098516, 25.800339784578807, 25.580499348307644, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 11.325378160549135, "exact_area100": 19.61825622310397}, "test": {"score": [7.8873978710439285, 9.370749710879897, 9.78914173403534, 10.297694028987676, 10.604302701255522, 10.619561373324922, 10.676832599583024, 11.36606233237387, 12.11834918403586, 11.940346719678136, 11.891262708431844, 11.922108133349393, 12.213528688910104, 12.169149838955631, 12.111586724604296, 12.408446091978956, 12.412996222506244, 13.025377627783891, 12.988206378962802, 13.220769855272287, 13.450690837974353, 13.70417605992029, 14.194891614271118, 14.54959306097288, 14.479512253475828, 14.489821949455678, 14.411516860994212, 14.443305355600627, 14.650879262660247, 14.626869462010767, 14.654645712327035, 15.240611240429322, 15.337933852486549, 15.564757601953433, 15.848609239987976, 15.913197855243869, 16.169288904805647, 16.343836078718194, 16.7331593954639, 16.83336436659044, 16.898618601656885, 16.971409539196824, 17.238765137878616, 17.41558621794839, 17.495416305054444, 17.607631534488554, 17.786607511641915, 17.88531533535313, 18.577866213998007, 18.875706483942825, 19.07141815829195, 20.025817870786035, 20.14305258370368, 20.305729193678452, 20.627757503075582, 20.9632568450826, 21.388158546418854, 21.48573105328974, 21.846790857443573, 22.113744840782708, 22.486247818148016, 22.680531226861632, 22.752791291835948, 22.541553328064627, 23.563861730044955, 23.80783908867547, 23.86734881507873, 24.278553273425395, 24.578767791938017, 24.922396088277328, 25.37608107022423, 25.706209063633217, 25.93683932985975, 26.267989272173004, 26.581731616403403, 26.75327204600152, 26.997705765098228, 26.987681219315835, 27.067687499687622, 27.017781846870687, 26.926889477331827, 26.867287089925078, 26.834259527614478, 26.801532324380933, 26.8085945233431, 26.78480761001929, 26.923876998692585, 26.877169768348818, 26.87795639273417, 26.80854146463941, 26.773088882783057, 27.073606240868248, 27.38815461043905, 27.320839220872156, 27.423245556609572, 27.377527940787093, 27.199113854911555, 27.14275698877743, 27.175668162411792, 27.131043160431865, 27.001009067886244], "calls": [0.0, 0.6324900133155792, 1.9640479360852199, 2.2303595206391478, 3.029294274300932, 3.162450066577896, 3.5952063914780292, 4.693741677762983, 5.093209054593874, 5.858854860186418, 6.957390146471372, 8.322237017310254, 9.021304926764314, 9.653794940079893, 10.51930758988016, 11.950732356857523, 13.881491344873503, 15.046604527296935, 15.81225033288948, 17.310252996005325, 18.10918774966711, 18.64181091877497, 19.241011984021306, 20.77230359520639, 21.571238348868178, 23.03595206391478, 24.4340878828229, 25.76564580559254, 27.097203728362185, 28.362183754993342, 29.161118508655125, 31.657789613848202, 33.022636484687084, 33.988015978695074, 35.61917443408788, 36.18508655126498, 37.050599201065246, 38.04926764314248, 38.68175765645805, 39.314247669773636, 39.88015978695073, 41.24500665778961, 42.110519307589875, 42.80958721704394, 44.241011984021306, 44.84021304926764, 45.50599201065246, 46.17177097203729, 48.46870838881491, 49.2676431424767, 50.23302263648469, 52.197070572569906, 52.82956058588548, 53.02929427430093, 54.49400798934754, 55.65912117177098, 56.524633821571236, 57.12383488681758, 58.22237017310253, 59.254327563249, 60.719041278295606, 61.55126498002663, 62.050599201065246, 63.04926764314248, 65.64580559254327, 66.711051930759, 67.0439414114514, 68.30892143808256, 69.77363515312916, 70.97203728362183, 71.73768308921437, 72.96937416777631, 74.03462050599201, 75.16644474034621, 76.19840213049268, 76.66444740346205, 77.49667110519307, 78.29560585885486, 78.66178428761651, 79.72703062583223, 80.99201065246339, 81.49134487350199, 81.92410119840213, 82.72303595206392, 83.88814913448735, 85.08655126498003, 85.91877496671106, 87.05059920106525, 88.48202396804261, 89.01464713715048, 89.34753661784288, 90.37949400798935, 91.74434087882824, 93.04260985352862, 94.54061251664447, 95.47270306258322, 96.40479360852197, 96.93741677762982, 97.86950732356857, 99.16777629826898, 100.0], "exact_score": [7.8873978710439285, 9.305712265257315, 10.32090623973514, 10.62271519097406, 10.61731601929559, 11.719154110149754, 11.910304203127408, 11.893421442593253, 11.907095637313148, 12.208149949689707, 12.18644435174324, 12.398591579374486, 12.413433453761991, 12.360734685824951, 12.403808978206063, 13.016278931281375, 12.92983259275721, 13.21955438089885, 13.456337511606892, 13.721058184680428, 14.11288761291902, 14.560921448563668, 14.457952909445446, 14.49799744136212, 14.450474929280643, 14.394135108527642, 14.464973218054586, 14.630984105211228, 14.595096719830252, 14.67087898194667, 14.89044601604311, 15.183044258476919, 15.279116092073592, 15.330728115935239, 15.564757601953433, 15.724960485714497, 15.885193261265067, 16.176555796943354, 16.329311944898162, 16.824115544120254, 16.915882691828045, 16.895287711150416, 17.210447628292812, 17.29517178875331, 17.43425691007896, 17.647751630467063, 17.848772308410467, 18.166797886339474, 18.46736227747306, 18.664124602754857, 19.011474144660372, 19.43657332315603, 19.78854774727098, 20.305729193678452, 20.575901347579826, 20.71384390902133, 21.140001599493672, 21.466357067940073, 21.789126360814983, 22.045591783073373, 22.30630846607065, 22.543628452653692, 22.732885064385968, 22.52036209832216, 22.77396039067759, 23.2427789582205, 23.650470240751282, 23.88395106035182, 24.177270969327676, 24.447241024996128, 24.625464210299597, 24.922396088277328, 25.45314715114681, 25.706209063633217, 25.907262249953412, 26.20470721884281, 26.5317526695241, 26.850022031793557, 27.137886290249014, 27.051015600234997, 27.017781846870687, 26.926889477331827, 26.834259527614478, 26.786712893447078, 26.8085945233431, 26.78480761001929, 26.923876998692585, 26.877169768348818, 26.854129655976706, 26.80854146463941, 26.84440929920105, 27.057460224722234, 27.364813854405323, 27.320839220872156, 27.33101186786365, 27.40727148156005, 27.278516871270163, 27.136113654795967, 27.175668162411792, 27.184073463462173, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 11.911714184391977, "exact_area100": 19.64551134016833}}, "old_normalized_endpoint_budget": 80}, {"seed": 2027, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 0.25, "validation_area100": 0.19740499032650846, "training_seconds": 3.5687192529439926, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6447188532478986, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6222574764759835, "operator": "ge"}, {"budget": 0.03, "threshold": 0.611571360680604, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6054489600055508, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5982033747952524, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5914743864602778, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5832045924858413, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5751520137603336, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5593063619635759, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5515323503460094, "operator": "ge"}, {"budget": 0.11, "threshold": 0.540549594529373, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5298957656600494, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5203594518018777, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5095164040433046, "operator": "ge"}, {"budget": 0.15, "threshold": 0.5041070536797242, "operator": "ge"}, {"budget": 0.16, "threshold": 0.5010447490523864, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4972460952406648, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4962367096933519, "operator": "ge"}, {"budget": 0.19, "threshold": 0.490894225265067, "operator": "ge"}, {"budget": 0.2, "threshold": 0.48734441721611554, "operator": "ge"}, {"budget": 0.21, "threshold": 0.481713833846885, "operator": "ge"}, {"budget": 0.22, "threshold": 0.4792677425028145, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4774530893221202, "operator": "ge"}, {"budget": 0.24, "threshold": 0.475636149963135, "operator": "ge"}, {"budget": 0.25, "threshold": 0.47205255732055595, "operator": "ge"}, {"budget": 0.26, "threshold": 0.46902116102541697, "operator": "ge"}, {"budget": 0.27, "threshold": 0.4661364604631399, "operator": "ge"}, {"budget": 0.28, "threshold": 0.4635614252225131, "operator": "ge"}, {"budget": 0.29, "threshold": 0.4588671211807007, "operator": "ge"}, {"budget": 0.3, "threshold": 0.45588671411096293, "operator": "ge"}, {"budget": 0.31, "threshold": 0.45347797030855524, "operator": "ge"}, {"budget": 0.32, "threshold": 0.44710344763811793, "operator": "ge"}, {"budget": 0.33, "threshold": 0.4435064730669271, "operator": "ge"}, {"budget": 0.34, "threshold": 0.4373807359477603, "operator": "ge"}, {"budget": 0.35, "threshold": 0.4339411827256688, "operator": "ge"}, {"budget": 0.36, "threshold": 0.4291671465211149, "operator": "ge"}, {"budget": 0.37, "threshold": 0.42646217238111866, "operator": "ge"}, {"budget": 0.38, "threshold": 0.41975250704430733, "operator": "ge"}, {"budget": 0.39, "threshold": 0.4176283700913335, "operator": "ge"}, {"budget": 0.4, "threshold": 0.4108088764010087, "operator": "ge"}, {"budget": 0.41, "threshold": 0.4031278688702644, "operator": "ge"}, {"budget": 0.42, "threshold": 0.39142348131359883, "operator": "ge"}, {"budget": 0.43, "threshold": 0.3791347092617527, "operator": "ge"}, {"budget": 0.44, "threshold": 0.36527081539269024, "operator": "ge"}, {"budget": 0.45, "threshold": 0.35484616562825405, "operator": "ge"}, {"budget": 0.46, "threshold": 0.34282542772682983, "operator": "ge"}, {"budget": 0.47, "threshold": 0.32693755041278305, "operator": "ge"}, {"budget": 0.48, "threshold": 0.32113422037195616, "operator": "ge"}, {"budget": 0.49, "threshold": 0.31424419231418727, "operator": "ge"}, {"budget": 0.5, "threshold": 0.30942269014400586, "operator": "ge"}, {"budget": 0.51, "threshold": 0.3032130167857535, "operator": "ge"}, {"budget": 0.52, "threshold": 0.2996598645206416, "operator": "ge"}, {"budget": 0.53, "threshold": 0.28953996340921223, "operator": "ge"}, {"budget": 0.54, "threshold": 0.28510863979658996, "operator": "ge"}, {"budget": 0.55, "threshold": 0.28057008664643146, "operator": "ge"}, {"budget": 0.56, "threshold": 0.27467258133666084, "operator": "ge"}, {"budget": 0.57, "threshold": 0.26953097080007665, "operator": "ge"}, {"budget": 0.58, "threshold": 0.26718626385059757, "operator": "ge"}, {"budget": 0.59, "threshold": 0.26198698743540555, "operator": "ge"}, {"budget": 0.6, "threshold": 0.25979020511255185, "operator": "ge"}, {"budget": 0.61, "threshold": 0.2544122875429865, "operator": "ge"}, {"budget": 0.62, "threshold": 0.2502796857995943, "operator": "ge"}, {"budget": 0.63, "threshold": 0.24443259369273415, "operator": "ge"}, {"budget": 0.64, "threshold": 0.24133149207529064, "operator": "ge"}, {"budget": 0.65, "threshold": 0.2326884809858958, "operator": "ge"}, {"budget": 0.66, "threshold": 0.22679801336714608, "operator": "ge"}, {"budget": 0.67, "threshold": 0.21768883166914638, "operator": "ge"}, {"budget": 0.68, "threshold": 0.2123640188841028, "operator": "ge"}, {"budget": 0.69, "threshold": 0.20220753794449273, "operator": "ge"}, {"budget": 0.7, "threshold": 0.182728211965129, "operator": "ge"}, {"budget": 0.71, "threshold": 0.17743280089548155, "operator": "ge"}, {"budget": 0.72, "threshold": 0.16087158581768765, "operator": "ge"}, {"budget": 0.73, "threshold": 0.13603577849549542, "operator": "ge"}, {"budget": 0.74, "threshold": 0.12419346782223274, "operator": "ge"}, {"budget": 0.75, "threshold": 0.11286336380681536, "operator": "ge"}, {"budget": 0.76, "threshold": 0.08792111295024924, "operator": "ge"}, {"budget": 0.77, "threshold": 0.06636564090435133, "operator": "ge"}, {"budget": 0.78, "threshold": 0.04814509072917199, "operator": "ge"}, {"budget": 0.79, "threshold": 0.02986882481107933, "operator": "ge"}, {"budget": 0.8, "threshold": 0.006030116631128583, "operator": "ge"}, {"budget": 0.81, "threshold": -0.008446112863132179, "operator": "ge"}, {"budget": 0.82, "threshold": -0.013656230013165556, "operator": "ge"}, {"budget": 0.83, "threshold": -0.020434263374848032, "operator": "ge"}, {"budget": 0.84, "threshold": -0.02782955124437577, "operator": "ge"}, {"budget": 0.85, "threshold": -0.03258310632552786, "operator": "ge"}, {"budget": 0.86, "threshold": -0.0370510884879595, "operator": "ge"}, {"budget": 0.87, "threshold": -0.04239023823700249, "operator": "ge"}, {"budget": 0.88, "threshold": -0.04443770882191759, "operator": "ge"}, {"budget": 0.89, "threshold": -0.04890828644219704, "operator": "ge"}, {"budget": 0.9, "threshold": -0.050845470734309624, "operator": "ge"}, {"budget": 0.91, "threshold": -0.05379129520592954, "operator": "ge"}, {"budget": 0.92, "threshold": -0.05611164572651193, "operator": "ge"}, {"budget": 0.93, "threshold": -0.0594403208969967, "operator": "ge"}, {"budget": 0.94, "threshold": -0.06351703133831833, "operator": "ge"}, {"budget": 0.95, "threshold": -0.06754866270297478, "operator": "ge"}, {"budget": 0.96, "threshold": -0.06971497045995774, "operator": "ge"}, {"budget": 0.97, "threshold": -0.0736493522265013, "operator": "ge"}, {"budget": 0.98, "threshold": -0.07563612130396057, "operator": "ge"}, {"budget": 0.99, "threshold": -0.09344833051515242, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.762403481598755, 7.876181366312142, 7.997813588777568, 8.12044437533475, 9.809603042258388, 9.74837623508065, 9.825747383690203, 9.649647043845324, 9.720291804853161, 9.831286632694122, 9.851889872854104, 9.951375399545581, 11.061902010551439, 11.084686684189823, 12.151523135977389, 13.96412710888095, 13.88429775204179, 13.856845790617275, 13.895220542316775, 14.832163476914273, 14.940118984813655, 14.979672971317948, 15.877691698401335, 15.93458221452111, 16.06027716032231, 16.177459556813428, 16.00796209295693, 16.0727852192862, 16.25145975134135, 16.35490936392706, 16.419073720102244, 16.366205466854783, 16.507684440671763, 17.909640847023628, 17.96156880024186, 18.03433570826321, 18.302793936188966, 18.59057971014493, 18.71973365617433, 18.738894113750153, 19.048127796525037, 19.25793305502801, 19.398082740988464, 19.538989363411595, 19.76998491704374, 20.471102948243534, 20.865363615087666, 21.170243342033952, 20.52989870647054, 20.627273888174095, 20.62912844301148, 20.642457249727386, 20.724806594371813, 21.601675671443115, 21.664873168331486, 21.70635350355422, 21.84626586228417, 21.95389134483983, 21.948252569694695, 22.04762324012312, 22.308216382663602, 22.36815944407168, 22.225375402491178, 22.49364691813244, 22.729487046550144, 22.9821816744615, 23.186376539514974, 23.45690794048116, 23.687262436591954, 23.937135380919695, 24.214732104439683, 24.49268760261557, 24.938850003763076, 25.241531417064223, 25.647021290562865, 25.973554589936278, 26.51280947696344, 26.739258752325757, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.453926046943245, 27.197394026479483, 27.194364490228757, 27.146956146360235, 26.942147858109777, 26.809119478722177, 26.573440698573254, 26.46566176639076, 26.359220032077342, 26.15024169423576, 26.15024169423576, 26.15024169423576, 26.069612788773156, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.616277523620706, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 41, 60, 78, 99, 117, 135, 152, 172, 191, 207, 223, 237, 251, 264, 277, 289, 304, 315, 328, 338, 350, 362, 375, 386, 396, 407, 418, 430, 442, 452, 465, 478, 492, 504, 516, 529, 542, 554, 566, 578, 586, 595, 604, 616, 625, 635, 641, 649, 655, 660, 666, 673, 678, 682, 687, 694, 700, 706, 711, 716, 721, 725, 732, 738, 743, 747, 752, 758, 764, 770, 776, 783, 790, 796, 802, 808, 810, 812, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 5, 8, 12, 17, 18, 21, 22, 25, 27, 28, 29, 31, 34, 36, 37, 39, 41, 43, 44, 44, 44, 45, 45, 46, 46, 46, 50, 51, 51, 52, 53, 57, 58, 59, 61, 63, 63, 63, 65, 68, 68, 68, 69, 70, 72, 74, 74, 77, 78, 80, 83, 86, 88, 88, 90, 93, 95, 97, 100, 102, 105, 110, 114, 115, 116, 116, 120, 120, 122, 124, 129, 131, 131, 134, 139, 141, 141, 141, 144, 144, 144, 146, 149, 152, 153, 157, 159, 163, 165, 167, 171, 171, 171, 173, 177, 177, 177, 177, 181, 185], "exact_score": [7.641657905799261, 7.762403481598755, 7.876181366312142, 7.997813588777568, 8.12044437533475, 9.809603042258388, 9.74837623508065, 9.825747383690203, 9.649647043845324, 9.720291804853161, 9.831286632694122, 9.851889872854104, 9.951375399545581, 11.061902010551439, 11.084686684189823, 12.151523135977389, 13.96412710888095, 13.88429775204179, 13.856845790617275, 13.895220542316775, 14.832163476914273, 14.940118984813655, 14.979672971317948, 15.877691698401335, 15.93458221452111, 16.06027716032231, 16.177459556813428, 16.00796209295693, 16.0727852192862, 16.25145975134135, 16.35490936392706, 16.419073720102244, 16.366205466854783, 16.507684440671763, 17.909640847023628, 17.96156880024186, 18.03433570826321, 18.302793936188966, 18.59057971014493, 18.71973365617433, 18.738894113750153, 19.048127796525037, 19.25793305502801, 19.398082740988464, 19.538989363411595, 19.76998491704374, 20.471102948243534, 20.865363615087666, 21.170243342033952, 20.52989870647054, 20.627273888174095, 20.62912844301148, 20.642457249727386, 20.724806594371813, 21.601675671443115, 21.664873168331486, 21.70635350355422, 21.84626586228417, 21.95389134483983, 21.948252569694695, 22.04762324012312, 22.308216382663602, 22.36815944407168, 22.225375402491178, 22.49364691813244, 22.729487046550144, 22.9821816744615, 23.186376539514974, 23.45690794048116, 23.687262436591954, 23.937135380919695, 24.214732104439683, 24.49268760261557, 24.938850003763076, 25.241531417064223, 25.647021290562865, 25.973554589936278, 26.51280947696344, 26.739258752325757, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.453926046943245, 27.197394026479483, 27.194364490228757, 27.146956146360235, 26.942147858109777, 26.809119478722177, 26.573440698573254, 26.46566176639076, 26.359220032077342, 26.15024169423576, 26.15024169423576, 26.15024169423576, 26.069612788773156, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.616277523620706, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.740499032650845, "threshold_area100": 19.73222481430739, "threshold_area25": 11.695108764180992}, "test": {"score": [7.8873978710439285, 8.61089476875382, 11.082283925239185, 10.833702622771089, 10.772400941854542, 11.055887990083779, 11.407685622369598, 11.346764394484628, 11.665480285757742, 11.833960262763226, 12.112530180513236, 12.043513854770849, 12.325099339143643, 12.290721496782837, 12.436017754888745, 12.575299553207397, 12.528478473637053, 12.665946701495667, 12.631684071001425, 12.85399965887771, 13.34973670858435, 13.321137256447305, 13.27964993752917, 13.30438554699011, 13.552616876562942, 13.620988280312027, 13.995848346068385, 14.331659218889905, 14.294594549496754, 14.530920309097391, 14.763905716994968, 15.016896817585303, 15.132146243024014, 15.42856486788714, 15.604270423260388, 15.583040371687714, 15.828646548830058, 15.920799100335087, 16.179404757756863, 16.26438835124719, 16.234578343432716, 16.57842117532149, 16.85341242966531, 17.624406858974062, 17.902580332192496, 18.151567681640035, 18.37657829659774, 18.837370412268122, 18.968041370771704, 19.069251238736033, 19.19776224555202, 19.268709572800148, 19.382649292750198, 19.469425913665624, 19.327052861785678, 19.457129604223013, 19.734234661239647, 19.95505010984887, 20.079028322744257, 20.50261715909281, 20.573762549023375, 20.782090101124417, 21.100941397866087, 21.490080338380327, 21.521552945891003, 22.31964853650796, 22.502784427300032, 22.93924027154322, 23.077729209205817, 23.509841502583438, 24.262673656244207, 24.3292443654275, 24.894705200846992, 25.662375357600375, 25.89218221119286, 26.083307293282342, 26.464977638699416, 26.78335460185299, 26.950650474147196, 26.961660151404732, 26.90406061051509, 26.796951062450784, 26.776779485961384, 26.703750784498276, 26.62793940103645, 26.64270183272822, 26.65145996193697, 26.63524905091324, 26.63524905091324, 26.608608781831343, 26.535925154563916, 26.775316336147686, 26.804995867815247, 26.97164408523885, 27.135629892982838, 27.03745839566735, 27.084304805150005, 27.08931981999447, 27.08931981999447, 26.885906728327356, 27.001009067886244], "calls": [0.0, 0.6324900133155792, 1.5978695073235687, 2.396804260985353, 2.862849533954727, 3.561917443408788, 4.427430093209055, 5.026631158455393, 5.625832223701731, 6.624500665778961, 7.090545938748336, 7.689747003994674, 8.488681757656458, 9.054593874833555, 10.818908122503329, 12.050599201065246, 12.716378162450068, 13.781624500665778, 14.280958721704396, 15.81225033288948, 17.11051930758988, 19.174434087882823, 19.906790945406126, 20.672436750998667, 21.438082556591212, 22.802929427430094, 23.93475366178429, 25.19973368841545, 26.264980026631157, 27.996005326231693, 29.127829560585884, 29.860186418109187, 31.62450066577896, 32.72303595206392, 34.753661784287615, 35.25299600532623, 36.451398135818906, 36.98402130492676, 38.315579227696404, 38.81491344873502, 39.81358189081225, 40.812250332889484, 42.37683089214381, 43.87483355525965, 44.940079893475364, 45.83888149134487, 46.737683089214386, 47.90279627163782, 48.60186418109188, 49.533954727030626, 50.665778961384824, 51.398135818908116, 52.09720372836218, 53.628495339547264, 54.56058588548602, 55.35952063914781, 56.29161118508655, 57.490013315579226, 58.022636484687084, 59.35419440745673, 59.92010652463382, 61.35153129161118, 62.35019973368841, 63.74833555259654, 64.2809587217044, 66.14513981358189, 66.81091877496671, 68.10918774966711, 68.50865512649801, 69.94007989347537, 71.57123834886818, 71.87083888149135, 73.33555259653795, 75.33288948069242, 76.03195739014647, 76.53129161118508, 77.49667110519307, 78.29560585885486, 78.728362183755, 79.19440745672436, 80.25965379494008, 81.35818908122504, 81.99067909454061, 82.88948069241012, 84.05459387483354, 85.41944074567243, 86.81757656458056, 88.11584553928095, 88.64846870838882, 89.94673768308921, 90.54593874833556, 91.41145139813582, 92.21038615179761, 93.20905459387482, 94.3075898801598, 95.30625832223701, 96.20505992010652, 97.03728362183755, 97.33688415446072, 99.03462050599201, 100.0], "rescued": [0, 51, 130, 194, 233, 292, 360, 408, 462, 541, 576, 622, 672, 705, 808, 878, 916, 972, 996, 1082, 1153, 1257, 1296, 1336, 1374, 1441, 1504, 1573, 1626, 1717, 1767, 1802, 1884, 1940, 2043, 2067, 2130, 2158, 2223, 2253, 2302, 2356, 2430, 2494, 2534, 2564, 2608, 2648, 2670, 2696, 2726, 2745, 2761, 2800, 2820, 2838, 2865, 2898, 2912, 2946, 2960, 2994, 3023, 3054, 3064, 3111, 3129, 3157, 3169, 3209, 3252, 3258, 3302, 3357, 3375, 3389, 3414, 3434, 3444, 3445, 3447, 3447, 3448, 3448, 3448, 3450, 3451, 3452, 3452, 3453, 3453, 3454, 3455, 3456, 3457, 3458, 3459, 3460, 3460, 3460, 3461], "harmed": [0, 7, 14, 25, 29, 39, 45, 50, 56, 69, 72, 78, 86, 90, 109, 121, 127, 138, 142, 152, 158, 174, 180, 185, 188, 195, 202, 205, 216, 227, 233, 235, 248, 252, 269, 272, 281, 284, 290, 295, 309, 313, 322, 330, 336, 342, 347, 351, 354, 363, 372, 376, 379, 386, 393, 395, 400, 406, 409, 413, 416, 426, 432, 439, 444, 453, 455, 463, 464, 469, 483, 485, 501, 517, 526, 532, 539, 549, 553, 554, 560, 568, 572, 577, 581, 590, 596, 601, 601, 606, 611, 613, 617, 621, 629, 639, 643, 645, 645, 660, 676], "exact_score": [7.8873978710439285, 8.615146717967741, 10.91406983595813, 10.683624398281745, 11.015236297416852, 11.367861224961802, 11.974192855374884, 12.100529842221768, 12.375404724003301, 12.28234650567086, 12.5047681087261, 12.706164510283058, 12.568566729717872, 12.501216754635506, 12.643033159334227, 12.587419919467186, 13.315661907609549, 13.32900883194698, 13.282477161341621, 13.323262110602476, 13.257396039022773, 13.508567347276667, 13.543809366484119, 13.742382500468835, 14.001240930682343, 14.311575761867946, 14.267195950781453, 14.338301879990675, 14.530920309097391, 14.75072280493006, 15.013999937466831, 15.086995547301049, 15.204130704603823, 15.470095785443297, 15.573664944520635, 15.539743962306826, 15.796616158141196, 15.920799100335087, 16.109596425504797, 16.289402893963935, 16.259460749055588, 16.596580732426546, 16.7375490475394, 17.21568310416392, 17.626412891447252, 17.89624167977882, 18.16395585669426, 18.413004902963014, 18.859166719219015, 18.97730926494237, 19.10183484467941, 19.2620399287474, 19.375516632374488, 19.542848069875056, 19.26820229814973, 19.395095346138888, 19.665073012131092, 19.84071657148505, 20.070673754884282, 20.463435482512804, 20.5718707876356, 20.700199441961196, 21.022364898687076, 21.367943104388658, 21.498715942790394, 21.870383015594193, 22.289214075559634, 22.537469316542122, 22.925132423527998, 23.2421706832316, 23.522945205479452, 23.927639299767993, 24.371735086735924, 24.797426382458823, 25.082288982885437, 25.544132490982722, 25.872100191998676, 26.274830648536927, 26.680663728459947, 26.961660151404732, 26.90406061051509, 26.84502720046788, 26.776779485961384, 26.703750784498276, 26.62793940103645, 26.666586969349108, 26.619222945558825, 26.65145996193697, 26.651119043930443, 26.63524905091324, 26.608608781831343, 26.799562088842993, 26.819560590669372, 26.988525607645236, 27.16575733072369, 27.00692340106585, 27.105434749393165, 27.114699242358814, 27.050731153379644, 26.901501166014214, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.2450821032384, "threshold_area100": 19.23285396638542, "threshold_area25": 11.96952950161335}, "selected_budget": 82, "selected": {"val": {"score": 27.453926046943245, "calls": 81.8830242510699, "rescued": 814, "harmed": 146}, "test": {"score": 26.776779485961384, "calls": 81.99067909454061, "rescued": 3448, "harmed": 572}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6145386099815369, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5999903082847595, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5907056927680969, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5826537013053894, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5754802823066711, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5631184577941895, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5540719628334045, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5464040040969849, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5326576232910156, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5167074799537659, "operator": "ge"}, {"budget": 0.11, "threshold": 0.5059541463851929, "operator": "ge"}, {"budget": 0.12, "threshold": 0.4938555061817169, "operator": "ge"}, {"budget": 0.13, "threshold": 0.48084303736686707, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4684670567512512, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4636971056461334, "operator": "ge"}, {"budget": 0.16, "threshold": 0.4590004086494446, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4564071297645569, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4538458585739136, "operator": "ge"}, {"budget": 0.19, "threshold": 0.45094263553619385, "operator": "ge"}, {"budget": 0.2, "threshold": 0.4458504915237427, "operator": "ge"}, {"budget": 0.21, "threshold": 0.44251900911331177, "operator": "ge"}, {"budget": 0.22, "threshold": 0.4364975690841675, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4320533573627472, "operator": "ge"}, {"budget": 0.24, "threshold": 0.4304584264755249, "operator": "ge"}, {"budget": 0.25, "threshold": 0.42747196555137634, "operator": "ge"}, {"budget": 0.26, "threshold": 0.42460763454437256, "operator": "ge"}, {"budget": 0.27, "threshold": 0.4223823547363281, "operator": "ge"}, {"budget": 0.28, "threshold": 0.4208255708217621, "operator": "ge"}, {"budget": 0.29, "threshold": 0.41712966561317444, "operator": "ge"}, {"budget": 0.3, "threshold": 0.4133371114730835, "operator": "ge"}, {"budget": 0.31, "threshold": 0.40656325221061707, "operator": "ge"}, {"budget": 0.32, "threshold": 0.40248599648475647, "operator": "ge"}, {"budget": 0.33, "threshold": 0.39530953764915466, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3909146189689636, "operator": "ge"}, {"budget": 0.35, "threshold": 0.3871475160121918, "operator": "ge"}, {"budget": 0.36, "threshold": 0.38280510902404785, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3773021399974823, "operator": "ge"}, {"budget": 0.38, "threshold": 0.37135452032089233, "operator": "ge"}, {"budget": 0.39, "threshold": 0.36566978693008423, "operator": "ge"}, {"budget": 0.4, "threshold": 0.35986095666885376, "operator": "ge"}, {"budget": 0.41, "threshold": 0.35586100816726685, "operator": "ge"}, {"budget": 0.42, "threshold": 0.34748512506484985, "operator": "ge"}, {"budget": 0.43, "threshold": 0.3303814232349396, "operator": "ge"}, {"budget": 0.44, "threshold": 0.3246249556541443, "operator": "ge"}, {"budget": 0.45, "threshold": 0.31471195816993713, "operator": "ge"}, {"budget": 0.46, "threshold": 0.2972179055213928, "operator": "ge"}, {"budget": 0.47, "threshold": 0.29220500588417053, "operator": "ge"}, {"budget": 0.48, "threshold": 0.27486634254455566, "operator": "ge"}, {"budget": 0.49, "threshold": 0.2657380700111389, "operator": "ge"}, {"budget": 0.5, "threshold": 0.2629396617412567, "operator": "ge"}, {"budget": 0.51, "threshold": 0.2509850561618805, "operator": "ge"}, {"budget": 0.52, "threshold": 0.24706125259399414, "operator": "ge"}, {"budget": 0.53, "threshold": 0.24012219905853271, "operator": "ge"}, {"budget": 0.54, "threshold": 0.23248089849948883, "operator": "ge"}, {"budget": 0.55, "threshold": 0.22379598021507263, "operator": "ge"}, {"budget": 0.56, "threshold": 0.21492044627666473, "operator": "ge"}, {"budget": 0.57, "threshold": 0.20669430494308472, "operator": "ge"}, {"budget": 0.58, "threshold": 0.20466549694538116, "operator": "ge"}, {"budget": 0.59, "threshold": 0.2017977386713028, "operator": "ge"}, {"budget": 0.6, "threshold": 0.1976190060377121, "operator": "ge"}, {"budget": 0.61, "threshold": 0.1921219527721405, "operator": "ge"}, {"budget": 0.62, "threshold": 0.1897401362657547, "operator": "ge"}, {"budget": 0.63, "threshold": 0.18559660017490387, "operator": "ge"}, {"budget": 0.64, "threshold": 0.1831677258014679, "operator": "ge"}, {"budget": 0.65, "threshold": 0.17741113901138306, "operator": "ge"}, {"budget": 0.66, "threshold": 0.17385756969451904, "operator": "ge"}, {"budget": 0.67, "threshold": 0.16634298861026764, "operator": "ge"}, {"budget": 0.68, "threshold": 0.15898476541042328, "operator": "ge"}, {"budget": 0.69, "threshold": 0.1476544439792633, "operator": "ge"}, {"budget": 0.7, "threshold": 0.13999629020690918, "operator": "ge"}, {"budget": 0.71, "threshold": 0.13110467791557312, "operator": "ge"}, {"budget": 0.72, "threshold": 0.12087943404912949, "operator": "ge"}, {"budget": 0.73, "threshold": 0.10528623312711716, "operator": "ge"}, {"budget": 0.74, "threshold": 0.09397172182798386, "operator": "ge"}, {"budget": 0.75, "threshold": 0.08572517335414886, "operator": "ge"}, {"budget": 0.76, "threshold": 0.061245694756507874, "operator": "ge"}, {"budget": 0.77, "threshold": 0.043366044759750366, "operator": "ge"}, {"budget": 0.78, "threshold": 0.030608994886279106, "operator": "ge"}, {"budget": 0.79, "threshold": 0.01643163338303566, "operator": "ge"}, {"budget": 0.8, "threshold": 0.0033054500818252563, "operator": "ge"}, {"budget": 0.81, "threshold": -0.004744388163089752, "operator": "ge"}, {"budget": 0.82, "threshold": -0.007923062890768051, "operator": "ge"}, {"budget": 0.83, "threshold": -0.011649731546640396, "operator": "ge"}, {"budget": 0.84, "threshold": -0.015167243778705597, "operator": "ge"}, {"budget": 0.85, "threshold": -0.019073929637670517, "operator": "ge"}, {"budget": 0.86, "threshold": -0.020732466131448746, "operator": "ge"}, {"budget": 0.87, "threshold": -0.023698166012763977, "operator": "ge"}, {"budget": 0.88, "threshold": -0.025599384680390358, "operator": "ge"}, {"budget": 0.89, "threshold": -0.027702882885932922, "operator": "ge"}, {"budget": 0.9, "threshold": -0.028576510027050972, "operator": "ge"}, {"budget": 0.91, "threshold": -0.030579786747694016, "operator": "ge"}, {"budget": 0.92, "threshold": -0.032069358974695206, "operator": "ge"}, {"budget": 0.93, "threshold": -0.033977117389440536, "operator": "ge"}, {"budget": 0.94, "threshold": -0.03574046492576599, "operator": "ge"}, {"budget": 0.95, "threshold": -0.038626596331596375, "operator": "ge"}, {"budget": 0.96, "threshold": -0.03944460675120354, "operator": "ge"}, {"budget": 0.97, "threshold": -0.0418933667242527, "operator": "ge"}, {"budget": 0.98, "threshold": -0.04698425903916359, "operator": "ge"}, {"budget": 0.99, "threshold": -0.05603707954287529, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.751224860383048, 7.872350829270848, 7.993809745087625, 9.909371512454658, 9.774613699540037, 9.844715348760804, 9.789862072031205, 9.61708804894564, 9.720291804853161, 9.844876234823024, 9.86194300121185, 9.957814105395112, 11.220792664892043, 13.204445227088454, 13.189886030172381, 13.146731784545429, 13.988252014764932, 13.749032280739751, 13.799468603685472, 14.697186690531439, 14.721360372251368, 14.834739011935156, 15.637254901960784, 15.692107713082235, 15.842190485088375, 15.85442762555958, 15.924909623826714, 15.973274073994418, 16.132803274758707, 16.23806951497792, 17.082872303960993, 17.022993015217306, 17.88742064442999, 18.028742190471725, 17.893417834830622, 17.962482106594756, 18.064956121162208, 18.34863185815999, 18.482767094568864, 18.779871126597687, 18.931758203447412, 19.226558191239317, 19.508606471576435, 19.459759749642185, 19.669261421555, 20.50915469862405, 20.901408403387695, 20.250596610197054, 20.378743731281684, 20.491633443258895, 20.5017873639764, 20.598404078283885, 20.765455304928988, 20.75139452768987, 20.824951485848416, 21.613434640518204, 21.798376914304335, 21.94294079496697, 21.644577131163622, 21.85873362143952, 21.803273286208935, 22.018220733114838, 22.278657584929366, 22.44940187857843, 22.67913016270338, 22.878048812968924, 23.08225579558883, 23.410487857296367, 23.6428110896196, 23.937135380919695, 24.166197839248454, 24.49268760261557, 24.81357539269157, 25.287318962851767, 25.555539125805883, 26.042516710028877, 26.623625079800316, 26.93115545451994, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.38229314922339, 27.197394026479483, 27.194364490228757, 27.146956146360235, 26.920289855072465, 26.682584428406635, 26.617649363471575, 26.46566176639076, 26.401948988066515, 26.15024169423576, 26.15024169423576, 26.15024169423576, 26.069612788773156, 26.030273685704703, 25.99156400828535, 25.915973457104247, 25.915973457104247, 25.616277523620706, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 20, 40, 59, 80, 100, 118, 136, 153, 172, 192, 207, 223, 238, 252, 262, 275, 289, 302, 315, 325, 337, 348, 361, 373, 386, 396, 408, 418, 429, 441, 453, 466, 480, 493, 505, 517, 529, 542, 553, 565, 574, 585, 595, 604, 614, 626, 637, 644, 651, 654, 659, 665, 671, 675, 681, 686, 693, 699, 703, 710, 715, 721, 726, 731, 738, 743, 748, 754, 759, 764, 770, 776, 783, 790, 796, 803, 809, 812, 812, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 4, 10, 13, 14, 19, 20, 23, 26, 27, 28, 29, 30, 32, 35, 36, 38, 40, 44, 45, 45, 46, 46, 47, 48, 48, 50, 51, 52, 52, 53, 53, 57, 57, 58, 62, 64, 66, 66, 67, 67, 68, 68, 68, 71, 73, 74, 74, 76, 78, 82, 85, 86, 87, 90, 93, 94, 96, 101, 106, 107, 110, 111, 114, 115, 117, 118, 119, 123, 124, 124, 130, 131, 133, 133, 140, 141, 141, 141, 144, 144, 144, 146, 149, 152, 153, 157, 161, 162, 165, 166, 171, 171, 171, 173, 174, 175, 177, 177, 181, 185], "exact_score": [7.641657905799261, 7.751224860383048, 7.872350829270848, 7.993809745087625, 9.909371512454658, 9.774613699540037, 9.844715348760804, 9.789862072031205, 9.61708804894564, 9.720291804853161, 9.844876234823024, 9.86194300121185, 9.957814105395112, 11.220792664892043, 13.204445227088454, 13.189886030172381, 13.146731784545429, 13.988252014764932, 13.749032280739751, 13.799468603685472, 14.697186690531439, 14.721360372251368, 14.834739011935156, 15.637254901960784, 15.692107713082235, 15.842190485088375, 15.85442762555958, 15.924909623826714, 15.973274073994418, 16.132803274758707, 16.23806951497792, 17.082872303960993, 17.022993015217306, 17.88742064442999, 18.028742190471725, 17.893417834830622, 17.962482106594756, 18.064956121162208, 18.34863185815999, 18.482767094568864, 18.779871126597687, 18.931758203447412, 19.226558191239317, 19.508606471576435, 19.459759749642185, 19.669261421555, 20.50915469862405, 20.901408403387695, 20.250596610197054, 20.378743731281684, 20.491633443258895, 20.5017873639764, 20.598404078283885, 20.765455304928988, 20.75139452768987, 20.824951485848416, 21.613434640518204, 21.798376914304335, 21.94294079496697, 21.644577131163622, 21.85873362143952, 21.803273286208935, 22.018220733114838, 22.278657584929366, 22.44940187857843, 22.67913016270338, 22.878048812968924, 23.08225579558883, 23.410487857296367, 23.6428110896196, 23.937135380919695, 24.166197839248454, 24.49268760261557, 24.81357539269157, 25.287318962851767, 25.555539125805883, 26.042516710028877, 26.623625079800316, 26.93115545451994, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.38229314922339, 27.197394026479483, 27.194364490228757, 27.146956146360235, 26.920289855072465, 26.682584428406635, 26.617649363471575, 26.46566176639076, 26.401948988066515, 26.15024169423576, 26.15024169423576, 26.15024169423576, 26.069612788773156, 26.030273685704703, 25.99156400828535, 25.915973457104247, 25.915973457104247, 25.616277523620706, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.735053281032364, "threshold_area100": 19.726771283043732, "threshold_area25": 11.849697370102142}, "test": {"score": [7.8873978710439285, 9.275064902381116, 10.440304807408088, 10.812327067293518, 11.156357986623714, 10.98339536880867, 11.397389532934831, 11.698508294390763, 11.5989221637827, 11.511796220180335, 11.790359091511498, 12.003788817127212, 12.280687999168189, 12.505968823319034, 12.2713547310335, 12.656608224736745, 12.536599003316267, 12.53006367074807, 12.732328997698783, 13.18654193039964, 13.137406041583175, 13.326126665535561, 13.360271376772143, 13.894775120538009, 13.939760592245328, 14.13695218740137, 14.154016922111282, 14.221503377598193, 14.26841206773978, 14.283106671778542, 14.389463046588697, 14.851090730663113, 15.207173886446231, 15.247895076593496, 15.396108751228985, 15.579837596295535, 15.69143942266222, 15.777255132682747, 15.971412463236447, 16.23575278470656, 16.361256633756955, 16.508388851470578, 16.758092657228328, 17.764020273960632, 17.878920589051642, 18.107013309980402, 18.342113190292743, 18.57622301899977, 18.984022911747555, 18.893445042967528, 18.966924931814695, 19.388770602691572, 19.59603077460248, 19.012728398931102, 19.369329037049145, 19.456054043658888, 19.78175239733497, 20.21331635955555, 20.400783271182192, 20.55722154679895, 20.69823855523478, 20.960834991891492, 21.040620030445762, 21.384707026015203, 21.524876663078352, 22.057710854247105, 22.343943678439253, 22.65989397081322, 22.868924102954303, 23.87780995048261, 24.15379736300201, 24.380633243880045, 24.877355071519204, 25.432366302358062, 25.78425081825319, 25.98863434148333, 26.50365282929555, 26.870078288997245, 26.978258727561904, 26.989568494530463, 26.90406061051509, 26.796951062450784, 26.776779485961384, 26.703750784498276, 26.644167036404433, 26.739588061171577, 26.660175776439885, 26.63524905091324, 26.63524905091324, 26.55923904936755, 26.589687614031792, 26.782491946074526, 26.995556634511125, 26.90899578458634, 26.864446460774044, 27.105434749393165, 27.09836591538911, 27.114699242358814, 27.08235242094074, 26.901501166014214, 27.001009067886244], "calls": [0.0, 0.9320905459387484, 1.6311584553928096, 2.1637816245006656, 2.6964047936085223, 3.2956058588548602, 4.360852197070573, 5.0599201065246335, 5.692410119840213, 6.5912117177097205, 7.323568575233022, 7.756324900133156, 8.288948069241012, 9.387483355525966, 11.384820239680426, 12.283621837549934, 13.515312916111851, 14.014647137150465, 14.647137150466044, 15.712383488681759, 17.876165113182424, 18.608521970705727, 20.03994673768309, 21.40479360852197, 21.80426098535286, 23.10252996005326, 23.834886817576564, 24.60053262316911, 25.133155792276963, 26.531291611185086, 27.862849533954726, 30.359520639147803, 31.19174434087883, 33.15579227696405, 34.42077230359521, 35.5525965379494, 36.38482023968043, 37.28362183754994, 38.348868175765645, 39.34753661784288, 40.279627163781626, 40.97869507323569, 42.27696404793608, 44.00798934753662, 44.67376830892144, 45.50599201065246, 46.77097203728362, 47.17043941411451, 48.568575233022635, 49.66711051930759, 50.09986684420772, 52.197070572569906, 52.729693741677764, 53.495339547270305, 54.29427430093209, 55.1930758988016, 56.55792276964048, 57.72303595206392, 58.28894806924101, 59.08788282290279, 60.11984021304927, 61.75099866844208, 62.28362183754993, 63.61517976031957, 64.24766977363515, 65.84553928095872, 66.27829560585884, 67.64314247669773, 68.87483355525966, 70.43941411451398, 71.30492676431425, 72.4034620505992, 73.50199733688414, 75.09986684420772, 75.89880159786951, 76.39813581890812, 77.52996005326231, 78.36218375499334, 78.728362183755, 79.22769640479362, 80.22636484687084, 81.29161118508655, 82.02396804260985, 82.9560585885486, 84.02130492676432, 85.78561917443409, 86.71770972037284, 88.11584553928095, 88.98135818908123, 89.94673768308921, 90.47936085219706, 91.6444740346205, 92.31025299600533, 93.30892143808256, 94.3075898801598, 95.70572569906791, 96.1051930758988, 96.97070572569906, 97.93608521970705, 99.06790945406125, 100.0], "rescued": [0, 74, 134, 180, 223, 271, 353, 414, 468, 540, 598, 627, 660, 727, 841, 892, 959, 985, 1020, 1073, 1192, 1234, 1303, 1378, 1401, 1466, 1502, 1543, 1568, 1639, 1701, 1832, 1872, 1961, 2026, 2081, 2128, 2175, 2229, 2278, 2324, 2360, 2421, 2498, 2523, 2556, 2605, 2624, 2670, 2699, 2711, 2766, 2779, 2795, 2815, 2834, 2869, 2902, 2919, 2940, 2964, 3005, 3018, 3045, 3060, 3101, 3114, 3148, 3178, 3221, 3246, 3274, 3306, 3349, 3371, 3385, 3418, 3439, 3445, 3446, 3447, 3447, 3448, 3448, 3448, 3451, 3451, 3452, 3452, 3452, 3453, 3454, 3455, 3455, 3456, 3459, 3459, 3460, 3460, 3460, 3461], "harmed": [0, 10, 16, 24, 32, 40, 45, 52, 59, 68, 75, 80, 87, 96, 120, 131, 144, 147, 152, 158, 176, 182, 191, 200, 201, 209, 215, 218, 220, 229, 236, 247, 254, 265, 274, 278, 285, 294, 301, 308, 315, 319, 325, 331, 334, 339, 348, 349, 353, 367, 370, 380, 381, 385, 390, 395, 400, 402, 407, 413, 421, 431, 435, 442, 446, 451, 454, 460, 471, 474, 481, 500, 506, 519, 530, 533, 542, 551, 553, 554, 560, 568, 572, 577, 580, 590, 596, 601, 601, 605, 606, 612, 615, 619, 628, 640, 641, 644, 646, 663, 676], "exact_score": [7.8873978710439285, 9.26523708792631, 10.920626344445058, 11.129045229120543, 10.931404685149705, 11.717123559572693, 11.590210324729867, 11.794169861910767, 12.344622655453616, 12.263254856352619, 12.382453729440053, 12.28651191025103, 12.457988844378786, 12.657831713597133, 12.525749801440234, 12.95382879182114, 13.164111979977694, 13.103903975428954, 13.115997901031962, 13.336862032206486, 13.356701791832492, 13.711344413819418, 13.949296178013226, 14.121004941922088, 14.174015387118565, 14.263479648198665, 14.331159940967156, 14.323974729674784, 14.407908328414502, 14.493584534941906, 14.829745304751722, 14.874233567275978, 15.305421612262293, 15.22128998095763, 15.32405544911247, 15.499781761818154, 15.63658156529836, 15.71379498163533, 15.988416356358451, 16.187907519014036, 16.381150626917236, 16.508388851470578, 16.686380538459595, 17.037999862802565, 17.759593928351567, 18.071888691891537, 18.20664961020699, 18.398630456284163, 18.854030899093992, 18.82140646937052, 18.963848223553992, 19.13480731888152, 19.3591723720521, 19.171757339421557, 19.093327786598493, 19.44174206518541, 19.688575354258973, 20.064106449223893, 20.25865925996935, 20.531324395485594, 20.690328121951644, 20.871401267491784, 20.995669639927765, 21.13541204260444, 21.485731516095907, 21.735632548148892, 22.09243552497579, 22.53280668752006, 22.68389690968384, 22.891980488329526, 23.532095039388544, 24.054560367987964, 24.346660192381172, 24.702945527852297, 25.072564669722492, 25.38266886092973, 25.840260475235006, 26.268253172896472, 26.72969279507672, 26.961660151404732, 26.90406061051509, 26.84502720046788, 26.776779485961384, 26.703750784498276, 26.660429987627403, 26.6696789649697, 26.72337195928745, 26.64406463289894, 26.63524905091324, 26.63524905091324, 26.55923904936755, 26.55068413691642, 26.761045290706587, 26.935703759587675, 26.890345145120882, 27.013903577365163, 27.09836591538911, 27.114699242358814, 27.08235242094074, 26.908369297882345, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.219318497039982, "threshold_area100": 19.22673636067003, "threshold_area25": 11.915988751172547}, "selected_budget": 82, "selected": {"val": {"score": 27.38229314922339, "calls": 81.8830242510699, "rescued": 814, "harmed": 146}, "test": {"score": 26.776779485961384, "calls": 82.02396804260985, "rescued": 3448, "harmed": 572}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.9822890758514404, "operator": "ge"}, {"budget": 0.02, "threshold": 2.9094560146331787, "operator": "ge"}, {"budget": 0.03, "threshold": 2.857914924621582, "operator": "ge"}, {"budget": 0.04, "threshold": 2.799546241760254, "operator": "ge"}, {"budget": 0.05, "threshold": 2.7574949264526367, "operator": "ge"}, {"budget": 0.06, "threshold": 2.725229263305664, "operator": "ge"}, {"budget": 0.07, "threshold": 2.705787420272827, "operator": "ge"}, {"budget": 0.08, "threshold": 2.6762962341308594, "operator": "ge"}, {"budget": 0.09, "threshold": 2.6477651596069336, "operator": "ge"}, {"budget": 0.1, "threshold": 2.624539852142334, "operator": "ge"}, {"budget": 0.11, "threshold": 2.6028518676757812, "operator": "ge"}, {"budget": 0.12, "threshold": 2.571110248565674, "operator": "ge"}, {"budget": 0.13, "threshold": 2.553180694580078, "operator": "ge"}, {"budget": 0.14, "threshold": 2.5286991596221924, "operator": "ge"}, {"budget": 0.15, "threshold": 2.5128965377807617, "operator": "ge"}, {"budget": 0.16, "threshold": 2.497929334640503, "operator": "ge"}, {"budget": 0.17, "threshold": 2.4852612018585205, "operator": "ge"}, {"budget": 0.18, "threshold": 2.4690499305725098, "operator": "ge"}, {"budget": 0.19, "threshold": 2.4471957683563232, "operator": "ge"}, {"budget": 0.2, "threshold": 2.433234214782715, "operator": "ge"}, {"budget": 0.21, "threshold": 2.4069297313690186, "operator": "ge"}, {"budget": 0.22, "threshold": 2.396886110305786, "operator": "ge"}, {"budget": 0.23, "threshold": 2.3888518810272217, "operator": "ge"}, {"budget": 0.24, "threshold": 2.350504159927368, "operator": "ge"}, {"budget": 0.25, "threshold": 2.3455164432525635, "operator": "ge"}, {"budget": 0.26, "threshold": 2.329429864883423, "operator": "ge"}, {"budget": 0.27, "threshold": 2.318472385406494, "operator": "ge"}, {"budget": 0.28, "threshold": 2.3098995685577393, "operator": "ge"}, {"budget": 0.29, "threshold": 2.2817606925964355, "operator": "ge"}, {"budget": 0.3, "threshold": 2.2649521827697754, "operator": "ge"}, {"budget": 0.31, "threshold": 2.2471237182617188, "operator": "ge"}, {"budget": 0.32, "threshold": 2.2364213466644287, "operator": "ge"}, {"budget": 0.33, "threshold": 2.2261338233947754, "operator": "ge"}, {"budget": 0.34, "threshold": 2.216979503631592, "operator": "ge"}, {"budget": 0.35, "threshold": 2.1912989616394043, "operator": "ge"}, {"budget": 0.36, "threshold": 2.175827741622925, "operator": "ge"}, {"budget": 0.37, "threshold": 2.165064811706543, "operator": "ge"}, {"budget": 0.38, "threshold": 2.1451892852783203, "operator": "ge"}, {"budget": 0.39, "threshold": 2.13615083694458, "operator": "ge"}, {"budget": 0.4, "threshold": 2.124873399734497, "operator": "ge"}, {"budget": 0.41, "threshold": 2.1166036128997803, "operator": "ge"}, {"budget": 0.42, "threshold": 2.103729248046875, "operator": "ge"}, {"budget": 0.43, "threshold": 2.082124710083008, "operator": "ge"}, {"budget": 0.44, "threshold": 2.060529947280884, "operator": "ge"}, {"budget": 0.45, "threshold": 2.034158706665039, "operator": "ge"}, {"budget": 0.46, "threshold": 2.01223087310791, "operator": "ge"}, {"budget": 0.47, "threshold": 1.9975303411483765, "operator": "ge"}, {"budget": 0.48, "threshold": 1.9778300523757935, "operator": "ge"}, {"budget": 0.49, "threshold": 1.971695899963379, "operator": "ge"}, {"budget": 0.5, "threshold": 1.949526071548462, "operator": "ge"}, {"budget": 0.51, "threshold": 1.9225610494613647, "operator": "ge"}, {"budget": 0.52, "threshold": 1.9085170030593872, "operator": "ge"}, {"budget": 0.53, "threshold": 1.8813201189041138, "operator": "ge"}, {"budget": 0.54, "threshold": 1.8709098100662231, "operator": "ge"}, {"budget": 0.55, "threshold": 1.8466954231262207, "operator": "ge"}, {"budget": 0.56, "threshold": 1.8248082399368286, "operator": "ge"}, {"budget": 0.57, "threshold": 1.8032914400100708, "operator": "ge"}, {"budget": 0.58, "threshold": 1.7724220752716064, "operator": "ge"}, {"budget": 0.59, "threshold": 1.7508792877197266, "operator": "ge"}, {"budget": 0.6, "threshold": 1.720198392868042, "operator": "ge"}, {"budget": 0.61, "threshold": 1.685234785079956, "operator": "ge"}, {"budget": 0.62, "threshold": 1.6618915796279907, "operator": "ge"}, {"budget": 0.63, "threshold": 1.5945184230804443, "operator": "ge"}, {"budget": 0.64, "threshold": 1.550830364227295, "operator": "ge"}, {"budget": 0.65, "threshold": 1.4909640550613403, "operator": "ge"}, {"budget": 0.66, "threshold": 1.437738299369812, "operator": "ge"}, {"budget": 0.67, "threshold": 1.3790465593338013, "operator": "ge"}, {"budget": 0.68, "threshold": 1.270151138305664, "operator": "ge"}, {"budget": 0.69, "threshold": 1.205403447151184, "operator": "ge"}, {"budget": 0.7, "threshold": 1.1071404218673706, "operator": "ge"}, {"budget": 0.71, "threshold": 1.0221716165542603, "operator": "ge"}, {"budget": 0.72, "threshold": 0.9543750882148743, "operator": "ge"}, {"budget": 0.73, "threshold": 0.8845636248588562, "operator": "ge"}, {"budget": 0.74, "threshold": 0.8078128695487976, "operator": "ge"}, {"budget": 0.75, "threshold": 0.7634724378585815, "operator": "ge"}, {"budget": 0.76, "threshold": 0.6332863569259644, "operator": "ge"}, {"budget": 0.77, "threshold": 0.5868613719940186, "operator": "ge"}, {"budget": 0.78, "threshold": 0.475969135761261, "operator": "ge"}, {"budget": 0.79, "threshold": 0.32729774713516235, "operator": "ge"}, {"budget": 0.8, "threshold": 0.08383667469024658, "operator": "ge"}, {"budget": 0.81, "threshold": -0.10949939489364624, "operator": "ge"}, {"budget": 0.82, "threshold": -0.17494702339172363, "operator": "ge"}, {"budget": 0.83, "threshold": -0.2502095699310303, "operator": "ge"}, {"budget": 0.84, "threshold": -0.37211525440216064, "operator": "ge"}, {"budget": 0.85, "threshold": -0.42996662855148315, "operator": "ge"}, {"budget": 0.86, "threshold": -0.46754878759384155, "operator": "ge"}, {"budget": 0.87, "threshold": -0.5143622159957886, "operator": "ge"}, {"budget": 0.88, "threshold": -0.5462891459465027, "operator": "ge"}, {"budget": 0.89, "threshold": -0.6071330308914185, "operator": "ge"}, {"budget": 0.9, "threshold": -0.6482386589050293, "operator": "ge"}, {"budget": 0.91, "threshold": -0.6862614750862122, "operator": "ge"}, {"budget": 0.92, "threshold": -0.7044823169708252, "operator": "ge"}, {"budget": 0.93, "threshold": -0.7578762173652649, "operator": "ge"}, {"budget": 0.94, "threshold": -0.7857555747032166, "operator": "ge"}, {"budget": 0.95, "threshold": -0.8357611894607544, "operator": "ge"}, {"budget": 0.96, "threshold": -0.8710709810256958, "operator": "ge"}, {"budget": 0.97, "threshold": -0.905582845211029, "operator": "ge"}, {"budget": 0.98, "threshold": -0.948163628578186, "operator": "ge"}, {"budget": 0.99, "threshold": -1.0220024585723877, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.645323168850962, 7.639448370709796, 7.736135009598074, 10.325471899882062, 10.413521228434533, 12.876755519280298, 14.764879116958104, 14.294842276626074, 14.402524820960636, 14.254540684446626, 14.332114285551178, 14.437601933403798, 14.518899328695714, 14.37386475681664, 14.087088504301871, 14.011453770712254, 13.719138381297904, 15.080958881794453, 15.225810603961865, 15.000239736466444, 14.733483045196378, 14.556622702341704, 14.658762290341237, 14.550941350889035, 15.811812491007318, 15.775339981949136, 16.164114676581516, 17.198394508496996, 16.90948967412254, 16.710517717151937, 16.742127733820677, 16.746578930256167, 16.79697349700313, 16.58437204313163, 16.718949512170983, 16.712563266252335, 16.83448954154291, 16.9771729065253, 16.96329033285555, 17.09313606597138, 17.364950602737245, 17.2896551310466, 17.35563969908483, 17.414180460567614, 17.586141319259816, 17.61052494968437, 17.8819702811635, 18.65966219223651, 18.88442202958332, 18.458740116559635, 18.46020484369999, 18.825998959139138, 19.480954644616887, 19.671501962851135, 19.800469483568076, 20.67703109591708, 20.840399019257276, 20.90653280916458, 21.240369799691834, 21.54524900164435, 21.591296562935813, 22.420834963527764, 22.64485382787681, 22.760334240227202, 23.085318487009815, 23.09121729334495, 23.43131724660959, 23.700519377832155, 23.836994579186456, 24.052616578900086, 24.184243895505418, 24.501030293851738, 24.674688057040996, 25.02967404206105, 25.30557677616501, 25.426044150984005, 25.88089770925592, 26.098385898978112, 26.975104991137712, 27.163354321263473, 27.163354321263473, 27.453926046943245, 27.081518226019508, 27.146956146360235, 27.054612937433724, 26.897939193211464, 26.76491081382386, 26.655892007771154, 26.590956942836087, 26.526637376146855, 26.172121475965643, 26.069612788773156, 25.99156400828535, 25.953468770190106, 25.879064008285347, 25.879064008285347, 25.879064008285347, 25.842726798983023, 25.545267404678757, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 14, 27, 44, 54, 68, 84, 95, 110, 124, 138, 148, 162, 175, 186, 202, 214, 230, 245, 260, 273, 287, 297, 307, 320, 334, 344, 355, 368, 381, 389, 400, 408, 419, 429, 439, 453, 460, 468, 480, 488, 504, 513, 523, 531, 538, 547, 560, 568, 577, 582, 592, 606, 618, 628, 639, 654, 667, 674, 685, 695, 700, 707, 715, 720, 727, 733, 741, 749, 756, 761, 767, 773, 778, 784, 790, 793, 799, 805, 811, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 3, 4, 7, 8, 8, 8, 9, 11, 11, 12, 12, 12, 12, 13, 15, 16, 19, 20, 20, 22, 24, 26, 27, 29, 30, 32, 33, 34, 37, 41, 42, 43, 45, 49, 49, 51, 51, 53, 55, 56, 57, 60, 62, 64, 67, 69, 69, 73, 73, 74, 80, 80, 83, 84, 87, 89, 93, 95, 96, 97, 100, 103, 105, 107, 107, 110, 111, 113, 117, 122, 126, 127, 130, 131, 134, 139, 139, 143, 144, 144, 144, 146, 151, 153, 155, 158, 160, 162, 163, 164, 171, 173, 175, 176, 178, 178, 178, 179, 183, 185], "exact_score": [7.641657905799261, 7.645323168850962, 7.639448370709796, 7.736135009598074, 10.325471899882062, 10.413521228434533, 12.876755519280298, 14.764879116958104, 14.294842276626074, 14.402524820960636, 14.254540684446626, 14.332114285551178, 14.437601933403798, 14.518899328695714, 14.37386475681664, 14.087088504301871, 14.011453770712254, 13.719138381297904, 15.080958881794453, 15.225810603961865, 15.000239736466444, 14.733483045196378, 14.556622702341704, 14.658762290341237, 14.550941350889035, 15.811812491007318, 15.775339981949136, 16.164114676581516, 17.198394508496996, 16.90948967412254, 16.710517717151937, 16.742127733820677, 16.746578930256167, 16.79697349700313, 16.58437204313163, 16.718949512170983, 16.712563266252335, 16.83448954154291, 16.9771729065253, 16.96329033285555, 17.09313606597138, 17.364950602737245, 17.2896551310466, 17.35563969908483, 17.414180460567614, 17.586141319259816, 17.61052494968437, 17.8819702811635, 18.65966219223651, 18.88442202958332, 18.458740116559635, 18.46020484369999, 18.825998959139138, 19.480954644616887, 19.671501962851135, 19.800469483568076, 20.67703109591708, 20.840399019257276, 20.90653280916458, 21.240369799691834, 21.54524900164435, 21.591296562935813, 22.420834963527764, 22.64485382787681, 22.760334240227202, 23.085318487009815, 23.09121729334495, 23.43131724660959, 23.700519377832155, 23.836994579186456, 24.052616578900086, 24.184243895505418, 24.501030293851738, 24.674688057040996, 25.02967404206105, 25.30557677616501, 25.426044150984005, 25.88089770925592, 26.098385898978112, 26.975104991137712, 27.163354321263473, 27.163354321263473, 27.453926046943245, 27.081518226019508, 27.146956146360235, 27.054612937433724, 26.897939193211464, 26.76491081382386, 26.655892007771154, 26.590956942836087, 26.526637376146855, 26.172121475965643, 26.069612788773156, 25.99156400828535, 25.953468770190106, 25.879064008285347, 25.879064008285347, 25.879064008285347, 25.842726798983023, 25.545267404678757, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.70110423101025, "threshold_area100": 19.69282445589226, "threshold_area25": 13.096425070410552}, "test": {"score": [7.8873978710439285, 7.966153895881754, 9.37075423216149, 9.348258134729287, 9.946150244728527, 9.941238116101813, 9.943708373608768, 9.958004586443693, 10.047769173269693, 10.598080910258858, 10.56843390498953, 10.618290201048126, 10.626835404009938, 11.149330565204465, 11.153502363310563, 11.180527090951916, 11.645590763029103, 11.65220880747893, 12.033128377337288, 11.992929886413998, 12.737560993277356, 12.845891250067215, 12.843259800428491, 13.1684124282535, 13.002897613086514, 13.01893775558018, 13.043545899572301, 13.077394101302428, 13.127367071242235, 13.146662377531202, 13.202875685497334, 13.712913785263398, 13.79637189753896, 13.787716650053545, 13.763214594912498, 13.754875152878506, 14.30353054722876, 14.55993785189321, 14.625913379510012, 14.751775388553979, 15.005575547599518, 15.241728591404296, 15.332290000729861, 15.66881347016403, 15.966091118086052, 16.12522954980669, 16.519436019551065, 16.66590971992574, 16.88100757991691, 16.955198271850545, 17.240418902348374, 17.636212232573005, 18.268643809911772, 18.416813533002838, 18.528105615677603, 18.915593051748658, 19.072433785237465, 19.217309833306658, 19.414494755007812, 19.806835986655308, 20.243326836078072, 20.77109850187174, 21.082402426619502, 21.519792005366487, 21.858906871542118, 22.253340129210827, 22.39434699614199, 22.72632603911377, 23.557555227996257, 23.891723072079678, 24.188919988242112, 24.47440040572507, 25.004785735772693, 25.189462201553027, 25.46170053309681, 25.64633587275097, 26.103270209477735, 26.189599480122833, 26.469746953511457, 26.811672118892755, 26.90406061051509, 26.83523828734, 26.771076898120782, 26.72008471416853, 26.724154491067786, 26.610497437178527, 26.563619318473684, 26.60151765212793, 26.688875159145176, 26.776401309473357, 26.81610357318769, 27.002144372610694, 26.969556789144416, 26.872878640009834, 26.814830506522203, 26.737237686258702, 27.025454900810043, 27.018624966182276, 27.176029032998162, 27.118324800194436, 27.001009067886244], "calls": [0.0, 0.9653794940079893, 1.5645805592543276, 2.1970705725699067, 3.5952063914780292, 4.760319573901465, 5.659121171770972, 6.291611185086551, 7.55659121171771, 8.422103861517977, 9.454061251664447, 10.652463382157123, 12.21704394141145, 13.315579227696405, 14.580559254327563, 15.612516644474034, 16.74434087882823, 17.576564580559253, 18.508655126498002, 20.03994673768309, 20.77230359520639, 22.37017310252996, 22.969374167776298, 23.302263648468706, 25.432756324900136, 25.798934753661783, 26.564580559254324, 27.163781624500665, 27.66311584553928, 29.460719041278296, 30.9254327563249, 32.456724367509985, 33.15579227696405, 33.85486018641811, 34.22103861517976, 36.051930758988014, 37.18375499334221, 37.616511318242345, 38.715046604527295, 39.54727030625832, 40.31291611185087, 40.77896138482024, 41.4780292942743, 42.50998668442077, 44.14114513981358, 45.60585885486019, 47.4034620505992, 48.368841544607186, 49.46737683089214, 50.0, 51.1318242343542, 52.729693741677764, 53.56191744340879, 54.76031957390146, 55.426098535286286, 56.3249001331558, 57.12383488681758, 57.822902796271634, 58.588548601864176, 59.38748335552596, 60.21970705725699, 61.35153129161118, 62.050599201065246, 63.64846870838882, 64.94673768308921, 66.24500665778962, 66.97736351531292, 67.90945406125167, 70.0399467376831, 70.83888149134488, 71.97070572569906, 72.76964047936086, 73.30226364846871, 74.13448735019973, 74.93342210386152, 75.46604527296937, 76.76431424766977, 77.13049267643143, 78.09587217043942, 78.99467376830891, 80.22636484687084, 81.35818908122504, 81.95739014647138, 82.78961384820239, 84.45406125166444, 85.68575233022636, 86.51797603195739, 87.21704394141145, 87.84953395472704, 89.88015978695073, 90.47936085219706, 91.54460719041279, 92.07723035952064, 93.00932090545939, 93.57523302263648, 94.67376830892144, 95.47270306258322, 96.37150466045273, 97.237017310253, 98.63515312916113, 100.0], "rescued": [0, 53, 90, 125, 204, 269, 322, 358, 423, 467, 517, 582, 659, 714, 779, 828, 892, 940, 988, 1067, 1111, 1187, 1217, 1233, 1336, 1349, 1389, 1414, 1437, 1517, 1576, 1647, 1675, 1704, 1724, 1803, 1853, 1869, 1914, 1950, 1982, 2002, 2031, 2078, 2144, 2201, 2283, 2327, 2373, 2398, 2459, 2533, 2578, 2633, 2663, 2707, 2744, 2771, 2804, 2839, 2870, 2914, 2942, 2997, 3041, 3090, 3111, 3137, 3203, 3227, 3258, 3279, 3297, 3315, 3336, 3351, 3384, 3392, 3414, 3435, 3447, 3448, 3448, 3448, 3450, 3450, 3450, 3452, 3453, 3454, 3455, 3456, 3456, 3456, 3456, 3456, 3459, 3459, 3460, 3461, 3461], "harmed": [0, 3, 8, 11, 17, 22, 26, 28, 29, 30, 36, 41, 47, 52, 57, 60, 66, 71, 76, 86, 89, 93, 96, 98, 116, 118, 123, 126, 130, 144, 154, 166, 167, 173, 176, 193, 198, 199, 207, 208, 212, 216, 217, 224, 234, 246, 262, 267, 280, 285, 295, 308, 316, 329, 335, 348, 357, 363, 368, 378, 384, 395, 400, 410, 421, 435, 443, 450, 463, 470, 481, 483, 490, 499, 506, 511, 523, 527, 539, 549, 560, 568, 573, 576, 584, 592, 596, 600, 600, 613, 617, 621, 623, 630, 637, 644, 645, 646, 652, 663, 676], "exact_score": [7.8873978710439285, 7.969705695593077, 9.333897161082064, 9.334350270995055, 9.93365952000209, 9.92017209051238, 9.950830679200049, 10.017249308969852, 10.5622525793018, 10.558839985521708, 10.57644972687232, 10.603769419637098, 10.604239498064311, 11.115689250070531, 11.165104428819998, 11.165942737937876, 11.656813190008872, 11.65277402686863, 12.032049261851986, 12.060339135065021, 11.98778389087607, 12.76049022500603, 12.863668479109883, 12.843259800428491, 13.173702354281112, 13.044355692187343, 13.038346504809823, 13.057283385803414, 13.074935042367727, 13.091541558333954, 13.125339423919982, 13.174994188461678, 13.681143719576433, 13.764781052405539, 13.72722284781396, 13.837691542723388, 13.789665532683811, 14.30409730381994, 14.571561828258556, 14.661256979495105, 14.96723969004679, 15.278119426350703, 15.60064610217043, 15.724055874597612, 16.00952746690074, 16.12035379869348, 16.436369103547126, 16.4942724470453, 16.61077868304861, 16.71397028331224, 16.955198271850545, 17.23050508549277, 17.540572943597855, 17.824548504491773, 18.18007390107386, 18.45287585702586, 18.8554139477384, 19.029239511346393, 19.27177713258996, 19.69439412148948, 20.078845991559362, 20.68426881156379, 21.086899451009025, 21.341588419789808, 21.55566082282142, 21.870345610738863, 22.150164043719148, 22.39434699614199, 22.750940090783587, 23.04541978099252, 23.530700698158462, 23.937676546451772, 24.188919988242112, 24.542748577258024, 25.141445333424894, 25.498250943808003, 25.826860435980958, 26.12109543190192, 26.431189005132673, 26.811672118892755, 26.90096406670415, 26.84502720046788, 26.771076898120782, 26.693062299786916, 26.62065801077633, 26.673854773000976, 26.587123168313415, 26.578164117855707, 26.672786637552992, 26.559147199083117, 26.776401309473357, 26.81610357318769, 26.969556789144416, 26.872878640009834, 26.764805786028973, 26.737237686258702, 27.018624966182276, 27.21616650514454, 27.13179496534493, 27.10269739329472, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 18.338173101002482, "threshold_area100": 18.469197168492936, "threshold_area25": 10.976185583726535}, "selected_budget": 82, "selected": {"val": {"score": 27.453926046943245, "calls": 81.8830242510699, "rescued": 814, "harmed": 146}, "test": {"score": 26.771076898120782, "calls": 81.95739014647138, "rescued": 3448, "harmed": 573}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.9369823932647705, "operator": "ge"}, {"budget": 0.02, "threshold": 1.8458102941513062, "operator": "ge"}, {"budget": 0.03, "threshold": 1.8166577816009521, "operator": "ge"}, {"budget": 0.04, "threshold": 1.8004554510116577, "operator": "ge"}, {"budget": 0.05, "threshold": 1.7688839435577393, "operator": "ge"}, {"budget": 0.06, "threshold": 1.7393121719360352, "operator": "ge"}, {"budget": 0.07, "threshold": 1.7116020917892456, "operator": "ge"}, {"budget": 0.08, "threshold": 1.6990783214569092, "operator": "ge"}, {"budget": 0.09, "threshold": 1.6864352226257324, "operator": "ge"}, {"budget": 0.1, "threshold": 1.6635990142822266, "operator": "ge"}, {"budget": 0.11, "threshold": 1.6487404108047485, "operator": "ge"}, {"budget": 0.12, "threshold": 1.637993335723877, "operator": "ge"}, {"budget": 0.13, "threshold": 1.6292048692703247, "operator": "ge"}, {"budget": 0.14, "threshold": 1.608793020248413, "operator": "ge"}, {"budget": 0.15, "threshold": 1.6009142398834229, "operator": "ge"}, {"budget": 0.16, "threshold": 1.5891046524047852, "operator": "ge"}, {"budget": 0.17, "threshold": 1.582749366760254, "operator": "ge"}, {"budget": 0.18, "threshold": 1.5774989128112793, "operator": "ge"}, {"budget": 0.19, "threshold": 1.569885015487671, "operator": "ge"}, {"budget": 0.2, "threshold": 1.56167733669281, "operator": "ge"}, {"budget": 0.21, "threshold": 1.555220127105713, "operator": "ge"}, {"budget": 0.22, "threshold": 1.5457279682159424, "operator": "ge"}, {"budget": 0.23, "threshold": 1.5366716384887695, "operator": "ge"}, {"budget": 0.24, "threshold": 1.5310966968536377, "operator": "ge"}, {"budget": 0.25, "threshold": 1.5168287754058838, "operator": "ge"}, {"budget": 0.26, "threshold": 1.5072417259216309, "operator": "ge"}, {"budget": 0.27, "threshold": 1.4956196546554565, "operator": "ge"}, {"budget": 0.28, "threshold": 1.4898334741592407, "operator": "ge"}, {"budget": 0.29, "threshold": 1.482229471206665, "operator": "ge"}, {"budget": 0.3, "threshold": 1.4729728698730469, "operator": "ge"}, {"budget": 0.31, "threshold": 1.4664664268493652, "operator": "ge"}, {"budget": 0.32, "threshold": 1.4574058055877686, "operator": "ge"}, {"budget": 0.33, "threshold": 1.443196177482605, "operator": "ge"}, {"budget": 0.34, "threshold": 1.437636137008667, "operator": "ge"}, {"budget": 0.35, "threshold": 1.432793378829956, "operator": "ge"}, {"budget": 0.36, "threshold": 1.4260931015014648, "operator": "ge"}, {"budget": 0.37, "threshold": 1.4173833131790161, "operator": "ge"}, {"budget": 0.38, "threshold": 1.4105939865112305, "operator": "ge"}, {"budget": 0.39, "threshold": 1.4048287868499756, "operator": "ge"}, {"budget": 0.4, "threshold": 1.397286057472229, "operator": "ge"}, {"budget": 0.41, "threshold": 1.3893134593963623, "operator": "ge"}, {"budget": 0.42, "threshold": 1.3694615364074707, "operator": "ge"}, {"budget": 0.43, "threshold": 1.365034818649292, "operator": "ge"}, {"budget": 0.44, "threshold": 1.3532075881958008, "operator": "ge"}, {"budget": 0.45, "threshold": 1.34586501121521, "operator": "ge"}, {"budget": 0.46, "threshold": 1.3353794813156128, "operator": "ge"}, {"budget": 0.47, "threshold": 1.3220828771591187, "operator": "ge"}, {"budget": 0.48, "threshold": 1.3079962730407715, "operator": "ge"}, {"budget": 0.49, "threshold": 1.2977209091186523, "operator": "ge"}, {"budget": 0.5, "threshold": 1.2909507751464844, "operator": "ge"}, {"budget": 0.51, "threshold": 1.2786352634429932, "operator": "ge"}, {"budget": 0.52, "threshold": 1.2622976303100586, "operator": "ge"}, {"budget": 0.53, "threshold": 1.2502222061157227, "operator": "ge"}, {"budget": 0.54, "threshold": 1.229986548423767, "operator": "ge"}, {"budget": 0.55, "threshold": 1.2150185108184814, "operator": "ge"}, {"budget": 0.56, "threshold": 1.1928801536560059, "operator": "ge"}, {"budget": 0.57, "threshold": 1.175705075263977, "operator": "ge"}, {"budget": 0.58, "threshold": 1.165358066558838, "operator": "ge"}, {"budget": 0.59, "threshold": 1.1586546897888184, "operator": "ge"}, {"budget": 0.6, "threshold": 1.1322550773620605, "operator": "ge"}, {"budget": 0.61, "threshold": 1.0999186038970947, "operator": "ge"}, {"budget": 0.62, "threshold": 1.0913771390914917, "operator": "ge"}, {"budget": 0.63, "threshold": 1.0752742290496826, "operator": "ge"}, {"budget": 0.64, "threshold": 1.0232523679733276, "operator": "ge"}, {"budget": 0.65, "threshold": 0.9988008737564087, "operator": "ge"}, {"budget": 0.66, "threshold": 0.9910268783569336, "operator": "ge"}, {"budget": 0.67, "threshold": 0.9570596814155579, "operator": "ge"}, {"budget": 0.68, "threshold": 0.9380878806114197, "operator": "ge"}, {"budget": 0.69, "threshold": 0.9073655605316162, "operator": "ge"}, {"budget": 0.7, "threshold": 0.8694701194763184, "operator": "ge"}, {"budget": 0.71, "threshold": 0.8287351727485657, "operator": "ge"}, {"budget": 0.72, "threshold": 0.7811539173126221, "operator": "ge"}, {"budget": 0.73, "threshold": 0.7586427927017212, "operator": "ge"}, {"budget": 0.74, "threshold": 0.6933316588401794, "operator": "ge"}, {"budget": 0.75, "threshold": 0.6372925639152527, "operator": "ge"}, {"budget": 0.76, "threshold": 0.542654275894165, "operator": "ge"}, {"budget": 0.77, "threshold": 0.49337029457092285, "operator": "ge"}, {"budget": 0.78, "threshold": 0.4034581780433655, "operator": "ge"}, {"budget": 0.79, "threshold": 0.2797319293022156, "operator": "ge"}, {"budget": 0.8, "threshold": 0.07325482368469238, "operator": "ge"}, {"budget": 0.81, "threshold": -0.09537762403488159, "operator": "ge"}, {"budget": 0.82, "threshold": -0.1544288992881775, "operator": "ge"}, {"budget": 0.83, "threshold": -0.2214563488960266, "operator": "ge"}, {"budget": 0.84, "threshold": -0.32276833057403564, "operator": "ge"}, {"budget": 0.85, "threshold": -0.3794342279434204, "operator": "ge"}, {"budget": 0.86, "threshold": -0.4092063903808594, "operator": "ge"}, {"budget": 0.87, "threshold": -0.4518660604953766, "operator": "ge"}, {"budget": 0.88, "threshold": -0.49181196093559265, "operator": "ge"}, {"budget": 0.89, "threshold": -0.5340012907981873, "operator": "ge"}, {"budget": 0.9, "threshold": -0.5802474021911621, "operator": "ge"}, {"budget": 0.91, "threshold": -0.6036821603775024, "operator": "ge"}, {"budget": 0.92, "threshold": -0.629340410232544, "operator": "ge"}, {"budget": 0.93, "threshold": -0.6736653447151184, "operator": "ge"}, {"budget": 0.94, "threshold": -0.697336733341217, "operator": "ge"}, {"budget": 0.95, "threshold": -0.748879611492157, "operator": "ge"}, {"budget": 0.96, "threshold": -0.7890173196792603, "operator": "ge"}, {"budget": 0.97, "threshold": -0.8216665983200073, "operator": "ge"}, {"budget": 0.98, "threshold": -0.8609185218811035, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9151333570480347, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.756956858505074, 7.759848632008815, 7.864718358557379, 7.978398406960629, 8.071254623115973, 8.191247728649302, 8.273002383550326, 8.304004528640725, 8.199370709382151, 8.325254394079556, 8.436159015464336, 8.527551553757778, 10.184525360972625, 10.191193949123873, 11.324172439581751, 11.29029078835701, 11.308288621757875, 11.413694210211508, 11.448933326158828, 11.585185197410828, 11.614605868137069, 11.766426632814564, 11.8917877152455, 13.92148845519516, 14.084874122268646, 14.210165108157238, 14.17749487981357, 15.070007678785247, 15.170144731879992, 15.243452427714447, 15.223225431876635, 15.271223270943315, 15.435344997479664, 16.08012768925371, 16.13495602006466, 16.370058626388044, 16.481120476505964, 17.24531953626521, 17.439382065165766, 17.609059094340093, 17.802255612972363, 17.809573206135422, 18.646566270266874, 19.45972254180158, 19.6086541304624, 19.6734318175363, 19.798319154037586, 19.994190985322334, 20.052156136427175, 20.277142177401725, 20.28250416477359, 20.48214285714286, 21.165172039102163, 21.269184402095796, 22.047720797720796, 22.256910644942216, 22.578163736882942, 22.77264265069143, 22.823230917088086, 23.04875599407497, 23.378268619700844, 23.56324347920987, 23.76436296854936, 23.938897801004405, 24.044326328769213, 24.011287977712026, 24.221065380652895, 23.59363100078741, 23.679332910532786, 23.94399612781388, 24.243981052016768, 24.45052524334669, 24.660031168993676, 24.964011080661493, 25.22440794499618, 25.457365112537524, 25.950549450549453, 26.13632415120208, 26.975104991137712, 27.163354321263473, 27.163354321263473, 27.453926046943245, 27.081518226019508, 27.146956146360235, 27.054612937433724, 26.897939193211464, 26.76491081382386, 26.699351373264417, 26.526637376146855, 26.526637376146855, 26.172121475965643, 26.069612788773156, 25.99156400828535, 25.953468770190106, 25.879064008285347, 25.879064008285347, 25.879064008285347, 25.80694862366996, 25.656083007515633, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 19, 32, 46, 62, 76, 94, 108, 123, 138, 155, 170, 183, 202, 217, 231, 246, 259, 271, 285, 297, 308, 323, 339, 352, 366, 382, 393, 402, 410, 419, 424, 433, 443, 456, 469, 481, 493, 505, 518, 530, 542, 551, 560, 571, 580, 589, 596, 604, 613, 623, 631, 638, 647, 654, 662, 668, 678, 687, 692, 700, 706, 711, 717, 725, 731, 737, 744, 750, 755, 760, 768, 773, 777, 783, 788, 794, 800, 805, 811, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 3, 5, 7, 9, 9, 12, 12, 14, 16, 19, 21, 22, 23, 26, 30, 34, 35, 35, 37, 37, 39, 39, 40, 40, 40, 41, 43, 44, 45, 49, 50, 51, 51, 51, 54, 54, 57, 60, 61, 63, 64, 68, 69, 70, 72, 73, 77, 78, 80, 81, 85, 85, 87, 89, 89, 89, 89, 91, 93, 95, 98, 103, 104, 107, 110, 114, 115, 116, 119, 123, 126, 128, 129, 131, 133, 136, 139, 143, 144, 144, 144, 146, 151, 153, 155, 158, 160, 161, 164, 164, 171, 173, 175, 176, 178, 178, 178, 180, 182, 185], "exact_score": [7.641657905799261, 7.756956858505074, 7.759848632008815, 7.864718358557379, 7.978398406960629, 8.071254623115973, 8.191247728649302, 8.273002383550326, 8.304004528640725, 8.199370709382151, 8.325254394079556, 8.436159015464336, 8.527551553757778, 10.184525360972625, 10.191193949123873, 11.324172439581751, 11.29029078835701, 11.308288621757875, 11.413694210211508, 11.448933326158828, 11.585185197410828, 11.614605868137069, 11.766426632814564, 11.8917877152455, 13.92148845519516, 14.084874122268646, 14.210165108157238, 14.17749487981357, 15.070007678785247, 15.170144731879992, 15.243452427714447, 15.223225431876635, 15.271223270943315, 15.435344997479664, 16.08012768925371, 16.13495602006466, 16.370058626388044, 16.481120476505964, 17.24531953626521, 17.439382065165766, 17.609059094340093, 17.802255612972363, 17.809573206135422, 18.646566270266874, 19.45972254180158, 19.6086541304624, 19.6734318175363, 19.798319154037586, 19.994190985322334, 20.052156136427175, 20.277142177401725, 20.28250416477359, 20.48214285714286, 21.165172039102163, 21.269184402095796, 22.047720797720796, 22.256910644942216, 22.578163736882942, 22.77264265069143, 22.823230917088086, 23.04875599407497, 23.378268619700844, 23.56324347920987, 23.76436296854936, 23.938897801004405, 24.044326328769213, 24.011287977712026, 24.221065380652895, 23.59363100078741, 23.679332910532786, 23.94399612781388, 24.243981052016768, 24.45052524334669, 24.660031168993676, 24.964011080661493, 25.22440794499618, 25.457365112537524, 25.950549450549453, 26.13632415120208, 26.975104991137712, 27.163354321263473, 27.163354321263473, 27.453926046943245, 27.081518226019508, 27.146956146360235, 27.054612937433724, 26.897939193211464, 26.76491081382386, 26.699351373264417, 26.526637376146855, 26.526637376146855, 26.172121475965643, 26.069612788773156, 25.99156400828535, 25.953468770190106, 25.879064008285347, 25.879064008285347, 25.879064008285347, 25.80694862366996, 25.656083007515633, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.12153488369932, "threshold_area100": 19.11234799836886, "threshold_area25": 10.033826533644413}, "test": {"score": [7.8873978710439285, 7.981255799014164, 8.756011942279173, 9.441706003735481, 10.05867948209075, 10.472488957498099, 11.053295587210526, 11.037794022736152, 11.026430119222525, 10.946883261840805, 11.662302077372381, 11.626213696290408, 11.61635344305026, 11.6084205524741, 12.3802485882261, 12.473222432791852, 12.422318275068621, 12.44448755178678, 12.790229530667766, 12.758322231148433, 13.089386943794764, 13.28634854818401, 13.252398287027736, 13.317547845298638, 13.371058024118105, 13.426828317773401, 13.43196689799368, 13.653162061874685, 13.601155642420027, 13.663595543388995, 14.141927332405086, 14.344188548620377, 14.601027035353026, 14.835688312414838, 14.82522591972433, 14.821861861221446, 14.900049988207009, 14.900875460143626, 15.001738138039839, 15.227352226718239, 15.501893813656785, 15.791957187210548, 16.3011876530118, 16.46799207201734, 16.78396226933065, 17.015720939650187, 17.31243595297782, 17.594039318036796, 18.12310244926441, 18.1117537981747, 18.154229298839393, 18.48084566089882, 18.973068121840786, 19.11437061286508, 19.490781006676254, 19.82793596342267, 20.219008095778683, 20.43051673868174, 20.579393706534717, 20.662012148549866, 21.28525479315024, 22.17942089291542, 22.312800116066583, 22.547395989890344, 22.854932343073816, 23.423405270983228, 23.459997980251753, 23.76764881242652, 23.76825680304101, 24.01722355131719, 24.336857703472432, 24.519553864024328, 25.02262975768416, 25.046567016245504, 25.35787304763671, 25.648669822446468, 25.987547589546782, 26.17367993123715, 26.471290206229963, 26.811672118892755, 26.90406061051509, 26.83523828734, 26.771076898120782, 26.72008471416853, 26.724154491067786, 26.610497437178527, 26.57088940207965, 26.60151765212793, 26.631735617520203, 26.776401309473357, 26.81610357318769, 27.053686569393378, 26.994929365395688, 26.887175879621356, 26.835847740654195, 26.744092322277535, 26.79625154434055, 27.002864631342703, 27.18695048218107, 27.10269739329472, 27.001009067886244], "calls": [0.0, 0.9653794940079893, 2.263648468708389, 2.896138482023968, 3.3288948069241013, 4.227696404793608, 5.193075898801598, 6.391478029294275, 7.090545938748336, 7.689747003994674, 8.921438082556591, 9.953395472703063, 10.619174434087883, 11.284953395472703, 13.448735019973368, 14.280958721704396, 15.179760319573901, 15.81225033288948, 16.4780292942743, 17.14380825565912, 18.275632490013315, 19.174434087882823, 20.672436750998667, 21.6711051930759, 22.503328894806923, 23.96804260985353, 25.03328894806924, 26.764314247669773, 27.5965379494008, 28.59520639147803, 29.394141145139812, 30.15978695073236, 31.19174434087883, 32.689747003994675, 33.38881491344873, 34.05459387483356, 34.753661784287615, 35.5525965379494, 36.55126498002663, 37.416777629826896, 38.41544607190413, 39.21438082556591, 41.51131824234354, 41.977363515312916, 43.54194407456725, 44.041278295605856, 45.03994673768309, 46.27163781624501, 47.93608521970706, 48.46870838881491, 48.83488681757657, 49.900133155792275, 51.1318242343542, 52.13049267643143, 53.79494007989347, 55.05992010652463, 56.3249001331558, 57.65645805592543, 58.72170439414115, 59.15446071904128, 61.085219707057256, 63.215712383488686, 63.74833555259654, 64.74700399467378, 67.44340878828228, 68.37549933422103, 68.54194407456724, 69.77363515312916, 70.30625832223701, 70.97203728362183, 72.00399467376832, 72.56990679094541, 73.33555259653795, 73.50199733688414, 74.53395472703063, 75.46604527296937, 76.63115845539281, 77.13049267643143, 78.12916111850865, 78.99467376830891, 80.22636484687084, 81.35818908122504, 81.99067909454061, 82.78961384820239, 84.25432756324899, 85.68575233022636, 86.38482023968042, 87.15046604527296, 88.1491344873502, 89.64713715046605, 90.54593874833556, 91.3115845539281, 91.94407456724367, 92.87616511318242, 93.47536617842877, 94.70705725699068, 95.57256990679095, 96.47137150466045, 97.33688415446072, 98.43541944074568, 100.0], "rescued": [0, 64, 146, 185, 214, 273, 326, 401, 443, 485, 557, 620, 661, 698, 821, 863, 915, 944, 984, 1022, 1084, 1131, 1208, 1259, 1303, 1377, 1434, 1511, 1549, 1595, 1639, 1667, 1713, 1786, 1814, 1850, 1881, 1915, 1958, 1996, 2048, 2087, 2186, 2205, 2273, 2294, 2338, 2390, 2461, 2484, 2494, 2532, 2587, 2627, 2690, 2740, 2786, 2832, 2861, 2875, 2939, 3014, 3031, 3061, 3137, 3165, 3168, 3202, 3217, 3236, 3261, 3276, 3297, 3302, 3327, 3352, 3380, 3392, 3414, 3435, 3447, 3448, 3448, 3448, 3450, 3450, 3450, 3452, 3453, 3454, 3455, 3456, 3456, 3456, 3456, 3456, 3458, 3459, 3460, 3461, 3461], "harmed": [0, 2, 9, 12, 14, 23, 26, 31, 34, 40, 52, 59, 63, 68, 78, 80, 86, 88, 91, 96, 101, 110, 119, 122, 125, 134, 144, 159, 166, 174, 181, 186, 192, 207, 213, 222, 224, 232, 238, 245, 253, 256, 267, 270, 283, 286, 293, 304, 321, 325, 327, 335, 344, 354, 367, 372, 378, 391, 398, 402, 418, 425, 429, 435, 451, 455, 455, 465, 471, 472, 481, 484, 489, 493, 501, 513, 525, 528, 540, 549, 560, 568, 573, 576, 584, 592, 595, 600, 604, 613, 617, 619, 622, 628, 634, 643, 645, 647, 652, 664, 676], "exact_score": [7.8873978710439285, 7.985404183956481, 8.757210932177584, 10.050927375809671, 10.529739679519443, 11.034721556374299, 11.02574103812451, 11.019223697895535, 11.344480397842192, 11.644630883452711, 11.630056758226196, 11.595413384923084, 11.61990678651026, 11.985720487907958, 12.41782036089506, 12.44295264903172, 12.44074803008701, 12.737397079904127, 13.120329133964685, 13.352533183140668, 13.292889530607042, 13.273584237995815, 13.326890937381028, 13.364735444785836, 13.426828317773401, 13.43079008128288, 13.460170272907437, 13.681710464530067, 13.63140706843977, 13.696927230466358, 14.353270240009925, 14.623044154337162, 14.632459999040643, 14.841451692615312, 14.840121345103407, 14.90057377910172, 14.930584673613653, 15.162196485261322, 15.46766899352067, 15.754007159172613, 15.894038562623619, 16.222521333442895, 16.46799207201734, 16.727745141841577, 17.006795301488044, 17.309101787438365, 17.54554392885326, 17.71794124255329, 18.121573878103028, 18.326935891777, 18.50620416759814, 18.933835386400453, 19.0830698577139, 19.3741584459387, 19.55623863098156, 19.797696747916685, 20.13502685983957, 20.353589642521015, 20.459243080411994, 20.640750696656582, 20.919134181244896, 21.277392242291615, 21.540440675771816, 21.918030201059185, 22.368093061701057, 22.594497507413074, 22.930112272035778, 22.927528637226928, 23.20783240070094, 23.56716704061837, 23.838964312963203, 24.01722355131719, 24.325465586817156, 24.942395042464153, 25.178149388851445, 25.51157558367297, 25.823898478168317, 26.115796128156056, 26.431189005132673, 26.811672118892755, 26.90096406670415, 26.84502720046788, 26.771076898120782, 26.693062299786916, 26.709743793503733, 26.69155312772334, 26.59442105338632, 26.578164117855707, 26.672786637552992, 26.559147199083117, 26.76045036951545, 26.81610357318769, 26.994929365395688, 26.88001407048016, 26.757876236449697, 26.721489833178243, 27.025454900810043, 26.996059264825107, 27.164685339444727, 27.10269739329472, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 18.992085317797, "threshold_area100": 18.981893902979234, "threshold_area25": 11.670902701173153}, "selected_budget": 82, "selected": {"val": {"score": 27.453926046943245, "calls": 81.8830242510699, "rescued": 814, "harmed": 146}, "test": {"score": 26.771076898120782, "calls": 81.99067909454061, "rescued": 3448, "harmed": 573}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 27.38229314922339, "calls": 81.8830242510699}, "test": {"score": 26.776779485961384, "calls": 82.02396804260985}}, "old_normalized_interior": {"val": {"score": 27.453926046943245, "calls": 81.8830242510699}, "test": {"score": 26.776779485961384, "calls": 81.99067909454061}}, "old_normalized_endpoints": {"val": {"score": 27.453926046943245, "calls": 81.8830242510699}, "test": {"score": 26.776779485961384, "calls": 81.99067909454061}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 7.762403481598755, 7.876181366312142, 7.997813588777568, 8.12044437533475, 9.809603042258388, 9.74837623508065, 9.825747383690203, 9.649647043845324, 9.720291804853161, 9.831286632694122, 9.851889872854104, 9.951375399545581, 11.061902010551439, 11.084686684189823, 12.151523135977389, 13.96412710888095, 13.88429775204179, 13.856845790617275, 13.895220542316775, 14.832163476914273, 14.940118984813655, 14.979672971317948, 15.877691698401335, 15.93458221452111, 16.06027716032231, 16.177459556813428, 16.00796209295693, 16.0727852192862, 16.25145975134135, 16.35490936392706, 16.419073720102244, 16.366205466854783, 16.507684440671763, 17.909640847023628, 17.96156880024186, 18.03433570826321, 18.302793936188966, 18.59057971014493, 18.71973365617433, 18.738894113750153, 19.048127796525037, 19.25793305502801, 19.398082740988464, 19.538989363411595, 19.76998491704374, 20.471102948243534, 20.865363615087666, 21.170243342033952, 20.52989870647054, 20.627273888174095, 20.62912844301148, 20.642457249727386, 20.724806594371813, 21.601675671443115, 21.664873168331486, 21.70635350355422, 21.84626586228417, 21.95389134483983, 21.948252569694695, 22.04762324012312, 22.308216382663602, 22.36815944407168, 22.225375402491178, 22.49364691813244, 22.729487046550144, 22.9821816744615, 23.186376539514974, 23.45690794048116, 23.687262436591954, 23.937135380919695, 24.214732104439683, 24.49268760261557, 24.938850003763076, 25.241531417064223, 25.647021290562865, 25.973554589936278, 26.51280947696344, 26.739258752325757, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.453926046943245, 27.197394026479483, 27.194364490228757, 27.146956146360235, 26.942147858109777, 26.809119478722177, 26.573440698573254, 26.46566176639076, 26.359220032077342, 26.15024169423576, 26.15024169423576, 26.15024169423576, 26.069612788773156, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.616277523620706, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 7.762403481598755, 7.876181366312142, 7.997813588777568, 8.12044437533475, 9.809603042258388, 9.74837623508065, 9.825747383690203, 9.649647043845324, 9.720291804853161, 9.831286632694122, 9.851889872854104, 9.951375399545581, 11.061902010551439, 11.084686684189823, 12.151523135977389, 13.96412710888095, 13.88429775204179, 13.856845790617275, 13.895220542316775, 14.832163476914273, 14.940118984813655, 14.979672971317948, 15.877691698401335, 15.93458221452111, 16.06027716032231, 16.177459556813428, 16.00796209295693, 16.0727852192862, 16.25145975134135, 16.35490936392706, 16.419073720102244, 16.366205466854783, 16.507684440671763, 17.909640847023628, 17.96156880024186, 18.03433570826321, 18.302793936188966, 18.59057971014493, 18.71973365617433, 18.738894113750153, 19.048127796525037, 19.25793305502801, 19.398082740988464, 19.538989363411595, 19.76998491704374, 20.471102948243534, 20.865363615087666, 21.170243342033952, 20.52989870647054, 20.627273888174095, 20.62912844301148, 20.642457249727386, 20.724806594371813, 21.601675671443115, 21.664873168331486, 21.70635350355422, 21.84626586228417, 21.95389134483983, 21.948252569694695, 22.04762324012312, 22.308216382663602, 22.36815944407168, 22.225375402491178, 22.49364691813244, 22.729487046550144, 22.9821816744615, 23.186376539514974, 23.45690794048116, 23.687262436591954, 23.937135380919695, 24.214732104439683, 24.49268760261557, 24.938850003763076, 25.241531417064223, 25.647021290562865, 25.973554589936278, 26.51280947696344, 26.739258752325757, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.453926046943245, 27.197394026479483, 27.194364490228757, 27.146956146360235, 26.942147858109777, 26.809119478722177, 26.573440698573254, 26.46566176639076, 26.359220032077342, 26.15024169423576, 26.15024169423576, 26.15024169423576, 26.069612788773156, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.616277523620706, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 11.695108764180992, "exact_area100": 19.740499032650845}, "test": {"score": [7.8873978710439285, 8.61089476875382, 11.082283925239185, 10.833702622771089, 10.772400941854542, 11.055887990083779, 11.407685622369598, 11.346764394484628, 11.665480285757742, 11.833960262763226, 12.112530180513236, 12.043513854770849, 12.325099339143643, 12.290721496782837, 12.436017754888745, 12.575299553207397, 12.528478473637053, 12.665946701495667, 12.631684071001425, 12.85399965887771, 13.34973670858435, 13.321137256447305, 13.27964993752917, 13.30438554699011, 13.552616876562942, 13.620988280312027, 13.995848346068385, 14.331659218889905, 14.294594549496754, 14.530920309097391, 14.763905716994968, 15.016896817585303, 15.132146243024014, 15.42856486788714, 15.604270423260388, 15.583040371687714, 15.828646548830058, 15.920799100335087, 16.179404757756863, 16.26438835124719, 16.234578343432716, 16.57842117532149, 16.85341242966531, 17.624406858974062, 17.902580332192496, 18.151567681640035, 18.37657829659774, 18.837370412268122, 18.968041370771704, 19.069251238736033, 19.19776224555202, 19.268709572800148, 19.382649292750198, 19.469425913665624, 19.327052861785678, 19.457129604223013, 19.734234661239647, 19.95505010984887, 20.079028322744257, 20.50261715909281, 20.573762549023375, 20.782090101124417, 21.100941397866087, 21.490080338380327, 21.521552945891003, 22.31964853650796, 22.502784427300032, 22.93924027154322, 23.077729209205817, 23.509841502583438, 24.262673656244207, 24.3292443654275, 24.894705200846992, 25.662375357600375, 25.89218221119286, 26.083307293282342, 26.464977638699416, 26.78335460185299, 26.950650474147196, 26.961660151404732, 26.90406061051509, 26.796951062450784, 26.776779485961384, 26.703750784498276, 26.62793940103645, 26.64270183272822, 26.65145996193697, 26.63524905091324, 26.63524905091324, 26.608608781831343, 26.535925154563916, 26.775316336147686, 26.804995867815247, 26.97164408523885, 27.135629892982838, 27.03745839566735, 27.084304805150005, 27.08931981999447, 27.08931981999447, 26.885906728327356, 27.001009067886244], "calls": [0.0, 0.6324900133155792, 1.5978695073235687, 2.396804260985353, 2.862849533954727, 3.561917443408788, 4.427430093209055, 5.026631158455393, 5.625832223701731, 6.624500665778961, 7.090545938748336, 7.689747003994674, 8.488681757656458, 9.054593874833555, 10.818908122503329, 12.050599201065246, 12.716378162450068, 13.781624500665778, 14.280958721704396, 15.81225033288948, 17.11051930758988, 19.174434087882823, 19.906790945406126, 20.672436750998667, 21.438082556591212, 22.802929427430094, 23.93475366178429, 25.19973368841545, 26.264980026631157, 27.996005326231693, 29.127829560585884, 29.860186418109187, 31.62450066577896, 32.72303595206392, 34.753661784287615, 35.25299600532623, 36.451398135818906, 36.98402130492676, 38.315579227696404, 38.81491344873502, 39.81358189081225, 40.812250332889484, 42.37683089214381, 43.87483355525965, 44.940079893475364, 45.83888149134487, 46.737683089214386, 47.90279627163782, 48.60186418109188, 49.533954727030626, 50.665778961384824, 51.398135818908116, 52.09720372836218, 53.628495339547264, 54.56058588548602, 55.35952063914781, 56.29161118508655, 57.490013315579226, 58.022636484687084, 59.35419440745673, 59.92010652463382, 61.35153129161118, 62.35019973368841, 63.74833555259654, 64.2809587217044, 66.14513981358189, 66.81091877496671, 68.10918774966711, 68.50865512649801, 69.94007989347537, 71.57123834886818, 71.87083888149135, 73.33555259653795, 75.33288948069242, 76.03195739014647, 76.53129161118508, 77.49667110519307, 78.29560585885486, 78.728362183755, 79.19440745672436, 80.25965379494008, 81.35818908122504, 81.99067909454061, 82.88948069241012, 84.05459387483354, 85.41944074567243, 86.81757656458056, 88.11584553928095, 88.64846870838882, 89.94673768308921, 90.54593874833556, 91.41145139813582, 92.21038615179761, 93.20905459387482, 94.3075898801598, 95.30625832223701, 96.20505992010652, 97.03728362183755, 97.33688415446072, 99.03462050599201, 100.0], "exact_score": [7.8873978710439285, 8.615146717967741, 10.91406983595813, 10.683624398281745, 11.015236297416852, 11.367861224961802, 11.974192855374884, 12.100529842221768, 12.375404724003301, 12.28234650567086, 12.5047681087261, 12.706164510283058, 12.568566729717872, 12.501216754635506, 12.643033159334227, 12.587419919467186, 13.315661907609549, 13.32900883194698, 13.282477161341621, 13.323262110602476, 13.257396039022773, 13.508567347276667, 13.543809366484119, 13.742382500468835, 14.001240930682343, 14.311575761867946, 14.267195950781453, 14.338301879990675, 14.530920309097391, 14.75072280493006, 15.013999937466831, 15.086995547301049, 15.204130704603823, 15.470095785443297, 15.573664944520635, 15.539743962306826, 15.796616158141196, 15.920799100335087, 16.109596425504797, 16.289402893963935, 16.259460749055588, 16.596580732426546, 16.7375490475394, 17.21568310416392, 17.626412891447252, 17.89624167977882, 18.16395585669426, 18.413004902963014, 18.859166719219015, 18.97730926494237, 19.10183484467941, 19.2620399287474, 19.375516632374488, 19.542848069875056, 19.26820229814973, 19.395095346138888, 19.665073012131092, 19.84071657148505, 20.070673754884282, 20.463435482512804, 20.5718707876356, 20.700199441961196, 21.022364898687076, 21.367943104388658, 21.498715942790394, 21.870383015594193, 22.289214075559634, 22.537469316542122, 22.925132423527998, 23.2421706832316, 23.522945205479452, 23.927639299767993, 24.371735086735924, 24.797426382458823, 25.082288982885437, 25.544132490982722, 25.872100191998676, 26.274830648536927, 26.680663728459947, 26.961660151404732, 26.90406061051509, 26.84502720046788, 26.776779485961384, 26.703750784498276, 26.62793940103645, 26.666586969349108, 26.619222945558825, 26.65145996193697, 26.651119043930443, 26.63524905091324, 26.608608781831343, 26.799562088842993, 26.819560590669372, 26.988525607645236, 27.16575733072369, 27.00692340106585, 27.105434749393165, 27.114699242358814, 27.050731153379644, 26.901501166014214, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 11.96952950161335, "exact_area100": 19.2450821032384}}, "old_normalized_endpoint_budget": 82}, {"seed": 2028, "selection": {"best_epoch": 2, "epochs": 10, "alpha": 0.25, "validation_area100": 0.1944900324880725, "training_seconds": 4.0175768920453265, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6591981809056362, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6362040634123252, "operator": "ge"}, {"budget": 0.03, "threshold": 0.624787929426612, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6086708362036842, "operator": "ge"}, {"budget": 0.05, "threshold": 0.600394238693181, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5922417505568063, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5870832373791045, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5817690572400974, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5587367488361271, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5473252038528486, "operator": "ge"}, {"budget": 0.11, "threshold": 0.5319717471359823, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5164817615272161, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5038272148233733, "operator": "ge"}, {"budget": 0.14, "threshold": 0.5002239142153593, "operator": "ge"}, {"budget": 0.15, "threshold": 0.49353501282668644, "operator": "ge"}, {"budget": 0.16, "threshold": 0.48607437519661684, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4782697251503901, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4720403514028779, "operator": "ge"}, {"budget": 0.19, "threshold": 0.46792966447380574, "operator": "ge"}, {"budget": 0.2, "threshold": 0.46236257493419636, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4570603248705436, "operator": "ge"}, {"budget": 0.22, "threshold": 0.45415454890164314, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4505289931876068, "operator": "ge"}, {"budget": 0.24, "threshold": 0.4463318716982839, "operator": "ge"}, {"budget": 0.25, "threshold": 0.4446674016958638, "operator": "ge"}, {"budget": 0.26, "threshold": 0.4364730594709487, "operator": "ge"}, {"budget": 0.27, "threshold": 0.4302026500845095, "operator": "ge"}, {"budget": 0.28, "threshold": 0.4225614482399268, "operator": "ge"}, {"budget": 0.29, "threshold": 0.41947972867394584, "operator": "ge"}, {"budget": 0.3, "threshold": 0.41571271891024697, "operator": "ge"}, {"budget": 0.31, "threshold": 0.40944952032769805, "operator": "ge"}, {"budget": 0.32, "threshold": 0.4035376567028955, "operator": "ge"}, {"budget": 0.33, "threshold": 0.3981052801008177, "operator": "ge"}, {"budget": 0.34, "threshold": 0.39377130044179715, "operator": "ge"}, {"budget": 0.35, "threshold": 0.3898100200244686, "operator": "ge"}, {"budget": 0.36, "threshold": 0.38484973604427886, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3800010990168502, "operator": "ge"}, {"budget": 0.38, "threshold": 0.3736334515927168, "operator": "ge"}, {"budget": 0.39, "threshold": 0.3694327159299617, "operator": "ge"}, {"budget": 0.4, "threshold": 0.3542284720090504, "operator": "ge"}, {"budget": 0.41, "threshold": 0.3386006458601797, "operator": "ge"}, {"budget": 0.42, "threshold": 0.3243429714214009, "operator": "ge"}, {"budget": 0.43, "threshold": 0.3159602841050513, "operator": "ge"}, {"budget": 0.44, "threshold": 0.3029918515462874, "operator": "ge"}, {"budget": 0.45, "threshold": 0.28997751427112395, "operator": "ge"}, {"budget": 0.46, "threshold": 0.2814512194035924, "operator": "ge"}, {"budget": 0.47, "threshold": 0.273788579539153, "operator": "ge"}, {"budget": 0.48, "threshold": 0.2648934007294365, "operator": "ge"}, {"budget": 0.49, "threshold": 0.2606028965907747, "operator": "ge"}, {"budget": 0.5, "threshold": 0.2538747247618918, "operator": "ge"}, {"budget": 0.51, "threshold": 0.2511646641351334, "operator": "ge"}, {"budget": 0.52, "threshold": 0.2486985039773629, "operator": "ge"}, {"budget": 0.53, "threshold": 0.24660724579599058, "operator": "ge"}, {"budget": 0.54, "threshold": 0.24002639393043926, "operator": "ge"}, {"budget": 0.55, "threshold": 0.23678397392059086, "operator": "ge"}, {"budget": 0.56, "threshold": 0.2350212881645494, "operator": "ge"}, {"budget": 0.57, "threshold": 0.2327948511955998, "operator": "ge"}, {"budget": 0.58, "threshold": 0.22809447015439266, "operator": "ge"}, {"budget": 0.59, "threshold": 0.2258583698188997, "operator": "ge"}, {"budget": 0.6, "threshold": 0.22050122546861153, "operator": "ge"}, {"budget": 0.61, "threshold": 0.21665436894778634, "operator": "ge"}, {"budget": 0.62, "threshold": 0.21459273085187103, "operator": "ge"}, {"budget": 0.63, "threshold": 0.21059920874521226, "operator": "ge"}, {"budget": 0.64, "threshold": 0.20351326855373625, "operator": "ge"}, {"budget": 0.65, "threshold": 0.20218212767628368, "operator": "ge"}, {"budget": 0.66, "threshold": 0.19544244616757384, "operator": "ge"}, {"budget": 0.67, "threshold": 0.1922347684830292, "operator": "ge"}, {"budget": 0.68, "threshold": 0.1886132103994334, "operator": "ge"}, {"budget": 0.69, "threshold": 0.1818173272032251, "operator": "ge"}, {"budget": 0.7, "threshold": 0.17808365460026704, "operator": "ge"}, {"budget": 0.71, "threshold": 0.17422155571861753, "operator": "ge"}, {"budget": 0.72, "threshold": 0.1678692146762942, "operator": "ge"}, {"budget": 0.73, "threshold": 0.16476381030062318, "operator": "ge"}, {"budget": 0.74, "threshold": 0.15335267750841344, "operator": "ge"}, {"budget": 0.75, "threshold": 0.14693833319677727, "operator": "ge"}, {"budget": 0.76, "threshold": 0.1200050055763653, "operator": "ge"}, {"budget": 0.77, "threshold": 0.05116334080362187, "operator": "ge"}, {"budget": 0.78, "threshold": -0.00019826898850444164, "operator": "ge"}, {"budget": 0.79, "threshold": -0.02202703072868514, "operator": "ge"}, {"budget": 0.8, "threshold": -0.03970965271512957, "operator": "ge"}, {"budget": 0.81, "threshold": -0.0513172756540462, "operator": "ge"}, {"budget": 0.82, "threshold": -0.05729726619883513, "operator": "ge"}, {"budget": 0.83, "threshold": -0.0643820390578486, "operator": "ge"}, {"budget": 0.84, "threshold": -0.06992062743327966, "operator": "ge"}, {"budget": 0.85, "threshold": -0.075476986549853, "operator": "ge"}, {"budget": 0.86, "threshold": -0.08037727392819086, "operator": "ge"}, {"budget": 0.87, "threshold": -0.08387402210888566, "operator": "ge"}, {"budget": 0.88, "threshold": -0.0882394444385098, "operator": "ge"}, {"budget": 0.89, "threshold": -0.08994414247726831, "operator": "ge"}, {"budget": 0.9, "threshold": -0.09323436923426771, "operator": "ge"}, {"budget": 0.91, "threshold": -0.10148638026197315, "operator": "ge"}, {"budget": 0.92, "threshold": -0.10897039042756813, "operator": "ge"}, {"budget": 0.93, "threshold": -0.11082123171908878, "operator": "ge"}, {"budget": 0.94, "threshold": -0.11296128834887892, "operator": "ge"}, {"budget": 0.95, "threshold": -0.11644336306134045, "operator": "ge"}, {"budget": 0.96, "threshold": -0.12242208035463557, "operator": "ge"}, {"budget": 0.97, "threshold": -0.13073761750703128, "operator": "ge"}, {"budget": 0.98, "threshold": -0.14678996906863906, "operator": "ge"}, {"budget": 0.99, "threshold": -0.16482031324021992, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.762403481598755, 7.880938202990603, 7.901836902535109, 7.799334165626301, 7.902368116608821, 8.009115148194095, 8.147092861683133, 8.262734908758482, 9.760044633922528, 9.746047978783846, 9.90413714818734, 10.002411439467856, 10.038217864650987, 10.000776962114, 11.249184729926416, 12.296862086186996, 12.294417406949556, 12.386505495701805, 12.47202774901497, 12.634665046331458, 12.731138100326298, 12.819542120000834, 12.850199849030453, 12.890611355758134, 13.026644682864546, 13.178569808839203, 13.246760271696715, 13.306197647589201, 13.398812079576356, 13.536464083223384, 14.459849842201894, 16.04010416939544, 16.158170474862178, 16.230392790839314, 17.038728808770756, 17.086818103816253, 17.254947288341405, 17.374839569245648, 18.041306923988486, 18.936863477093365, 19.178489166106118, 19.30744092508798, 19.42790968924873, 19.72730411754802, 19.735157856745044, 19.987398594191795, 20.79727037720818, 21.039673864957642, 21.22889910752084, 21.55893576086966, 21.762207449654806, 21.806137100290997, 22.06274884535754, 22.209388349721614, 23.044539785611214, 22.292841478696744, 22.31965884179759, 22.469762361466323, 22.624280690537084, 22.84468134969206, 23.036175710594318, 23.35852444276597, 23.580664958131962, 23.708614300467772, 23.950227809007202, 24.15163068420687, 24.45585904322941, 24.741399904443384, 24.807868508089747, 24.816150236318304, 25.026256448274793, 25.161520027971783, 25.500166581668594, 25.555627037389012, 25.862163047036997, 26.424491013498397, 26.906991010527566, 26.722903574626272, 27.14259897095718, 27.095877937997614, 27.095877937997614, 27.163354321263473, 26.90614287104511, 26.92914643064568, 27.194364490228757, 27.079468578280668, 26.89986358035557, 26.854096989966557, 26.617649363471575, 26.401948988066515, 26.401948988066515, 26.359220032077342, 26.29610549580309, 26.172121475965643, 26.030273685704703, 25.99156400828535, 25.953468770190106, 25.915973457104247, 25.727093126457586, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 40, 58, 75, 93, 111, 131, 149, 169, 188, 207, 223, 239, 253, 268, 281, 294, 307, 320, 333, 345, 356, 367, 380, 391, 403, 415, 426, 437, 449, 460, 470, 482, 494, 506, 517, 528, 540, 552, 566, 579, 589, 600, 610, 618, 626, 635, 642, 647, 655, 660, 664, 671, 677, 684, 689, 695, 700, 706, 713, 718, 725, 730, 737, 743, 749, 755, 761, 766, 772, 776, 781, 787, 792, 797, 804, 810, 811, 812, 812, 812, 813, 813, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 5, 8, 13, 15, 17, 20, 21, 24, 25, 28, 28, 30, 31, 34, 34, 36, 39, 40, 41, 41, 42, 43, 45, 47, 47, 48, 50, 52, 54, 55, 55, 55, 56, 58, 59, 61, 62, 64, 67, 67, 68, 70, 72, 72, 74, 74, 74, 74, 74, 74, 74, 76, 77, 78, 79, 84, 86, 87, 89, 90, 94, 95, 95, 99, 100, 102, 103, 108, 116, 120, 121, 124, 125, 133, 135, 136, 139, 143, 143, 144, 144, 144, 147, 150, 152, 154, 157, 158, 162, 166, 166, 167, 168, 171, 174, 175, 176, 177, 180, 185], "exact_score": [7.641657905799261, 7.762403481598755, 7.880938202990603, 7.901836902535109, 7.799334165626301, 7.902368116608821, 8.009115148194095, 8.147092861683133, 8.262734908758482, 9.760044633922528, 9.746047978783846, 9.90413714818734, 10.002411439467856, 10.038217864650987, 10.000776962114, 11.249184729926416, 12.296862086186996, 12.294417406949556, 12.386505495701805, 12.47202774901497, 12.634665046331458, 12.731138100326298, 12.819542120000834, 12.850199849030453, 12.890611355758134, 13.026644682864546, 13.178569808839203, 13.246760271696715, 13.306197647589201, 13.398812079576356, 13.536464083223384, 14.459849842201894, 16.04010416939544, 16.158170474862178, 16.230392790839314, 17.038728808770756, 17.086818103816253, 17.254947288341405, 17.374839569245648, 18.041306923988486, 18.936863477093365, 19.178489166106118, 19.30744092508798, 19.42790968924873, 19.72730411754802, 19.735157856745044, 19.987398594191795, 20.79727037720818, 21.039673864957642, 21.22889910752084, 21.55893576086966, 21.762207449654806, 21.806137100290997, 22.06274884535754, 22.209388349721614, 23.044539785611214, 22.292841478696744, 22.31965884179759, 22.469762361466323, 22.624280690537084, 22.84468134969206, 23.036175710594318, 23.35852444276597, 23.580664958131962, 23.708614300467772, 23.950227809007202, 24.15163068420687, 24.45585904322941, 24.741399904443384, 24.807868508089747, 24.816150236318304, 25.026256448274793, 25.161520027971783, 25.500166581668594, 25.555627037389012, 25.862163047036997, 26.424491013498397, 26.906991010527566, 26.722903574626272, 27.14259897095718, 27.095877937997614, 27.095877937997614, 27.163354321263473, 26.90614287104511, 26.92914643064568, 27.194364490228757, 27.079468578280668, 26.89986358035557, 26.854096989966557, 26.617649363471575, 26.401948988066515, 26.401948988066515, 26.359220032077342, 26.29610549580309, 26.172121475965643, 26.030273685704703, 25.99156400828535, 25.953468770190106, 25.915973457104247, 25.727093126457586, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.44900324880725, "threshold_area100": 19.440233453913418, "threshold_area25": 10.373283433196487}, "test": {"score": [7.8873978710439285, 7.968557092616793, 9.861983037240899, 9.744659509203162, 9.741098045845566, 10.242816301467341, 11.113960446360938, 11.053934995428321, 10.972739969393082, 11.70776402079631, 11.956750290669035, 12.165884923254525, 12.204248513471764, 12.497957009378261, 12.494580511661473, 12.484165788614735, 12.44509202940827, 12.339624654308183, 12.720482008782911, 12.836722809340786, 12.904850507156073, 13.32829777993493, 13.350447562331466, 13.30832160527434, 13.360161482922509, 13.37875012937074, 13.48200644255804, 13.558848562640591, 14.267465828422008, 14.32569649269137, 14.66081525544205, 14.70438749459786, 14.83288665770934, 15.07321585941212, 15.25230891341952, 15.30774758512804, 15.745468268005874, 15.6810734200547, 16.078632836093053, 16.149282364373846, 16.536940042742323, 16.735288351120765, 17.122517663550077, 17.389509089899903, 17.507900522576794, 17.816882004648576, 18.08436154301386, 18.240057319777453, 18.609396307852062, 18.735918572498548, 19.016496315286922, 19.366571208615422, 19.422121891808512, 19.53529904680983, 19.960507194242396, 20.25423708648043, 20.677140190434717, 20.993146619178674, 21.208144531713355, 21.392564441333167, 21.73067350167065, 21.662635118416727, 22.018244602364227, 22.284000485890253, 22.703817610162943, 22.774547664193456, 23.590275173871035, 23.740643715769618, 24.116394575451324, 24.606294249320225, 24.737286209375753, 24.806679239312913, 25.341030425050437, 25.407313159849075, 25.74716049868399, 25.91478905509732, 26.469947303499996, 26.872116589355986, 26.95834144311357, 26.99907962201799, 26.931691355375253, 26.903783012249523, 26.846831963531244, 26.755201760497588, 26.742282063717997, 26.671311424348264, 26.724320779016914, 26.657242498079466, 26.707197037253216, 26.690992521048702, 26.644037142965825, 26.866839611016104, 26.76911494024109, 26.76911494024109, 26.722890367356513, 27.14792366854223, 27.118801136142885, 27.135752904716824, 27.08235242094074, 26.915262046710573, 27.001009067886244], "calls": [0.0, 0.7656458055925432, 1.7976031957390146, 2.330226364846871, 3.1957390146471374, 3.761651131824234, 4.427430093209055, 4.727030625832223, 5.026631158455393, 6.324900133155792, 6.824234354194407, 7.623169107856191, 8.621837549933423, 9.620505992010653, 10.086551264980026, 11.018641810918774, 12.316910785619175, 13.681757656458057, 14.647137150466044, 15.579227696404793, 16.77762982689747, 17.909454061251665, 18.342210386151798, 19.10785619174434, 19.840213049267643, 20.07323568575233, 22.569906790945407, 23.96804260985353, 25.5659121171771, 25.96537949400799, 26.93075898801598, 28.79494007989348, 30.326231691078565, 31.82423435419441, 32.49001331557923, 32.822902796271634, 33.75499334221039, 34.88681757656458, 36.1185086551265, 36.78428761651132, 38.715046604527295, 40.74567243675099, 42.34354194407457, 43.17576564580559, 44.14114513981358, 45.372836218375504, 46.37150466045273, 47.10386151797603, 48.46870838881491, 49.46737683089214, 51.09853528628495, 51.664447403462056, 52.197070572569906, 52.92942743009321, 55.29294274300932, 56.55792276964048, 57.490013315579226, 58.1225033288948, 59.78695073235686, 60.58588548601864, 62.28362183754993, 63.64846870838882, 64.4474034620506, 65.24633821571238, 67.11051930758988, 67.34354194407457, 68.9081225033289, 69.44074567243675, 70.20639147802929, 71.5046604527297, 72.03728362183755, 72.33688415446072, 73.43541944074568, 73.66844207723035, 74.80026631158455, 75.33288948069242, 76.59786950732357, 77.762982689747, 78.56191744340879, 79.29427430093209, 80.32623169107856, 81.02529960053262, 81.75765645805593, 82.88948069241012, 84.0878828229028, 85.41944074567243, 86.48468708388815, 87.21704394141145, 88.38215712383489, 89.0479360852197, 89.78029294274302, 92.07723035952064, 93.77496671105193, 94.14114513981359, 94.74034620505992, 95.57256990679095, 96.17177097203728, 96.93741677762982, 98.06924101198402, 98.83488681757656, 100.0], "rescued": [0, 62, 146, 191, 262, 309, 363, 389, 412, 517, 550, 606, 670, 737, 766, 821, 900, 977, 1032, 1083, 1148, 1207, 1224, 1263, 1301, 1313, 1442, 1510, 1593, 1611, 1662, 1756, 1838, 1912, 1945, 1962, 2008, 2062, 2126, 2160, 2255, 2350, 2427, 2462, 2503, 2554, 2591, 2619, 2665, 2690, 2732, 2752, 2766, 2784, 2843, 2877, 2902, 2920, 2961, 2985, 3026, 3056, 3076, 3096, 3143, 3149, 3194, 3208, 3230, 3265, 3279, 3287, 3318, 3322, 3348, 3361, 3393, 3424, 3439, 3444, 3446, 3446, 3446, 3446, 3447, 3448, 3450, 3450, 3452, 3452, 3452, 3454, 3454, 3454, 3454, 3457, 3458, 3460, 3460, 3460, 3461], "harmed": [0, 4, 18, 27, 33, 36, 41, 45, 50, 59, 63, 72, 75, 81, 83, 87, 96, 108, 109, 113, 118, 118, 119, 125, 128, 129, 144, 160, 171, 171, 177, 192, 202, 216, 225, 225, 231, 244, 256, 261, 276, 297, 311, 318, 328, 335, 342, 345, 351, 355, 360, 363, 367, 374, 380, 381, 384, 386, 395, 399, 406, 420, 423, 428, 445, 445, 456, 461, 466, 478, 485, 488, 493, 493, 506, 514, 522, 536, 547, 551, 558, 559, 563, 569, 578, 585, 587, 591, 596, 597, 601, 611, 618, 618, 622, 630, 634, 641, 646, 661, 676], "exact_score": [7.8873978710439285, 8.721661187564903, 9.801366465438086, 9.736130885754202, 10.712623795131288, 10.970829708470967, 11.738243499181, 12.224276765004667, 12.186092203036104, 12.203579865687443, 12.485791333347814, 12.481508196058561, 12.42892466472401, 12.45764380254041, 12.637569899654322, 12.723887919637445, 12.84824795385454, 12.933153972103895, 13.335950421034237, 13.355654797649022, 13.36081980713894, 13.386598630926272, 13.448697185618286, 13.343978829348933, 13.558848562640591, 13.875677659849018, 14.330956595280325, 14.663475918561167, 14.701463934133411, 14.70702880784159, 14.805773970292883, 14.867820918054441, 15.062397027435036, 15.513950247298288, 15.751582542033033, 15.858066422457382, 16.072620865941957, 16.35624507209984, 16.43950401466322, 16.548534883092458, 16.671061028548856, 16.766083759364665, 16.901373367804513, 17.328127321687937, 17.375350445203495, 17.76677827141416, 18.019310808926043, 18.224867609318473, 18.42630696942797, 18.69385954371003, 18.838994556280092, 18.993098635755384, 19.394402608286, 19.553010960436968, 19.749292737222248, 19.902903258837497, 20.112582380006085, 20.359721766021572, 20.988825156041223, 21.056496743796302, 21.38097823548643, 21.50117375013412, 21.655592394163527, 21.58065753044805, 21.757797960247153, 22.238312201553235, 22.42529183851076, 22.68150412689499, 22.965404055317272, 23.61928893624703, 24.02299484018767, 24.475240078557455, 24.740308288549034, 25.16237974018494, 25.22065305094148, 25.81000257713177, 26.23642710769665, 26.636356506943002, 26.896768476910047, 26.990661766301027, 26.94825345114527, 26.903783012249523, 26.830414593234458, 26.755201760497588, 26.758332397564942, 26.687361758195205, 26.671311424348264, 26.664540383152367, 26.714564353681254, 26.690992521048702, 26.628344468306352, 26.604960382676012, 26.88283582274358, 26.817099502369597, 26.76911494024109, 26.7462094980229, 27.114458261624623, 27.135752904716824, 27.08235242094074, 26.885906728327356, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.297518281006052, "threshold_area100": 19.26637104254289, "threshold_area25": 11.644331377622905}, "selected_budget": 85, "selected": {"val": {"score": 27.194364490228757, "calls": 84.87874465049929, "rescued": 815, "harmed": 152}, "test": {"score": 26.671311424348264, "calls": 85.41944074567243, "rescued": 3448, "harmed": 585}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.630348265171051, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6173833012580872, "operator": "ge"}, {"budget": 0.03, "threshold": 0.6020373702049255, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5882610082626343, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5796006917953491, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5729618072509766, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5668425559997559, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5531046390533447, "operator": "ge"}, {"budget": 0.09, "threshold": 0.5377300977706909, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5259867310523987, "operator": "ge"}, {"budget": 0.11, "threshold": 0.49934616684913635, "operator": "ge"}, {"budget": 0.12, "threshold": 0.4758751392364502, "operator": "ge"}, {"budget": 0.13, "threshold": 0.4653400182723999, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4534664452075958, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4488106667995453, "operator": "ge"}, {"budget": 0.16, "threshold": 0.4396958649158478, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4287140965461731, "operator": "ge"}, {"budget": 0.18, "threshold": 0.42468520998954773, "operator": "ge"}, {"budget": 0.19, "threshold": 0.41801661252975464, "operator": "ge"}, {"budget": 0.2, "threshold": 0.4153231382369995, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4106563627719879, "operator": "ge"}, {"budget": 0.22, "threshold": 0.40583035349845886, "operator": "ge"}, {"budget": 0.23, "threshold": 0.40414875745773315, "operator": "ge"}, {"budget": 0.24, "threshold": 0.40201807022094727, "operator": "ge"}, {"budget": 0.25, "threshold": 0.3980847895145416, "operator": "ge"}, {"budget": 0.26, "threshold": 0.396129310131073, "operator": "ge"}, {"budget": 0.27, "threshold": 0.3862791061401367, "operator": "ge"}, {"budget": 0.28, "threshold": 0.383270263671875, "operator": "ge"}, {"budget": 0.29, "threshold": 0.37917762994766235, "operator": "ge"}, {"budget": 0.3, "threshold": 0.3760327398777008, "operator": "ge"}, {"budget": 0.31, "threshold": 0.3728179633617401, "operator": "ge"}, {"budget": 0.32, "threshold": 0.36551880836486816, "operator": "ge"}, {"budget": 0.33, "threshold": 0.36082032322883606, "operator": "ge"}, {"budget": 0.34, "threshold": 0.35369551181793213, "operator": "ge"}, {"budget": 0.35, "threshold": 0.34825626015663147, "operator": "ge"}, {"budget": 0.36, "threshold": 0.34390002489089966, "operator": "ge"}, {"budget": 0.37, "threshold": 0.34114620089530945, "operator": "ge"}, {"budget": 0.38, "threshold": 0.33316245675086975, "operator": "ge"}, {"budget": 0.39, "threshold": 0.3289385735988617, "operator": "ge"}, {"budget": 0.4, "threshold": 0.3192194998264313, "operator": "ge"}, {"budget": 0.41, "threshold": 0.3001529574394226, "operator": "ge"}, {"budget": 0.42, "threshold": 0.28559383749961853, "operator": "ge"}, {"budget": 0.43, "threshold": 0.2745145857334137, "operator": "ge"}, {"budget": 0.44, "threshold": 0.2592984735965729, "operator": "ge"}, {"budget": 0.45, "threshold": 0.24877206981182098, "operator": "ge"}, {"budget": 0.46, "threshold": 0.23392431437969208, "operator": "ge"}, {"budget": 0.47, "threshold": 0.2250843346118927, "operator": "ge"}, {"budget": 0.48, "threshold": 0.21102219820022583, "operator": "ge"}, {"budget": 0.49, "threshold": 0.20431220531463623, "operator": "ge"}, {"budget": 0.5, "threshold": 0.20060943067073822, "operator": "ge"}, {"budget": 0.51, "threshold": 0.19589969515800476, "operator": "ge"}, {"budget": 0.52, "threshold": 0.19175086915493011, "operator": "ge"}, {"budget": 0.53, "threshold": 0.18585161864757538, "operator": "ge"}, {"budget": 0.54, "threshold": 0.18280786275863647, "operator": "ge"}, {"budget": 0.55, "threshold": 0.18059800565242767, "operator": "ge"}, {"budget": 0.56, "threshold": 0.1783580332994461, "operator": "ge"}, {"budget": 0.57, "threshold": 0.1742631047964096, "operator": "ge"}, {"budget": 0.58, "threshold": 0.17219331860542297, "operator": "ge"}, {"budget": 0.59, "threshold": 0.1694350242614746, "operator": "ge"}, {"budget": 0.6, "threshold": 0.16762810945510864, "operator": "ge"}, {"budget": 0.61, "threshold": 0.16423964500427246, "operator": "ge"}, {"budget": 0.62, "threshold": 0.1617070734500885, "operator": "ge"}, {"budget": 0.63, "threshold": 0.1587727665901184, "operator": "ge"}, {"budget": 0.64, "threshold": 0.15629681944847107, "operator": "ge"}, {"budget": 0.65, "threshold": 0.15171800553798676, "operator": "ge"}, {"budget": 0.66, "threshold": 0.14945141971111298, "operator": "ge"}, {"budget": 0.67, "threshold": 0.1451193392276764, "operator": "ge"}, {"budget": 0.68, "threshold": 0.14198291301727295, "operator": "ge"}, {"budget": 0.69, "threshold": 0.13997335731983185, "operator": "ge"}, {"budget": 0.7, "threshold": 0.13590314984321594, "operator": "ge"}, {"budget": 0.71, "threshold": 0.1313748061656952, "operator": "ge"}, {"budget": 0.72, "threshold": 0.1273280829191208, "operator": "ge"}, {"budget": 0.73, "threshold": 0.12362774461507797, "operator": "ge"}, {"budget": 0.74, "threshold": 0.1213897317647934, "operator": "ge"}, {"budget": 0.75, "threshold": 0.10967583954334259, "operator": "ge"}, {"budget": 0.76, "threshold": 0.0959620326757431, "operator": "ge"}, {"budget": 0.77, "threshold": 0.03898455202579498, "operator": "ge"}, {"budget": 0.78, "threshold": -9.875744581222534e-05, "operator": "ge"}, {"budget": 0.79, "threshold": -0.010287376120686531, "operator": "ge"}, {"budget": 0.8, "threshold": -0.019464435055851936, "operator": "ge"}, {"budget": 0.81, "threshold": -0.02597229927778244, "operator": "ge"}, {"budget": 0.82, "threshold": -0.02946220338344574, "operator": "ge"}, {"budget": 0.83, "threshold": -0.03268517553806305, "operator": "ge"}, {"budget": 0.84, "threshold": -0.03611276298761368, "operator": "ge"}, {"budget": 0.85, "threshold": -0.038048408925533295, "operator": "ge"}, {"budget": 0.86, "threshold": -0.04153572395443916, "operator": "ge"}, {"budget": 0.87, "threshold": -0.04314465820789337, "operator": "ge"}, {"budget": 0.88, "threshold": -0.04533335939049721, "operator": "ge"}, {"budget": 0.89, "threshold": -0.04745742678642273, "operator": "ge"}, {"budget": 0.9, "threshold": -0.04954168573021889, "operator": "ge"}, {"budget": 0.91, "threshold": -0.053270123898983, "operator": "ge"}, {"budget": 0.92, "threshold": -0.05816449970006943, "operator": "ge"}, {"budget": 0.93, "threshold": -0.05953340604901314, "operator": "ge"}, {"budget": 0.94, "threshold": -0.06164593994617462, "operator": "ge"}, {"budget": 0.95, "threshold": -0.06470978260040283, "operator": "ge"}, {"budget": 0.96, "threshold": -0.06756851822137833, "operator": "ge"}, {"budget": 0.97, "threshold": -0.07253406941890717, "operator": "ge"}, {"budget": 0.98, "threshold": -0.08560249209403992, "operator": "ge"}, {"budget": 0.99, "threshold": -0.09754504263401031, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.762403481598755, 7.880938202990603, 7.901836902535109, 7.806837222572864, 7.914361872695205, 8.021339105691212, 8.14481336640586, 9.65162379764737, 9.72765360139131, 9.84980288933188, 9.875603524782463, 9.992090819340262, 10.036211191059854, 9.990790171190534, 12.23295664749605, 12.211491707489182, 12.220628509521521, 12.243531821330683, 12.414877923320738, 12.5466188401225, 12.715229732524833, 12.764955364116053, 12.68991247880455, 12.870312789436078, 12.952782194453686, 13.110066529939884, 13.272261411482871, 13.33889084028265, 13.439711408401383, 14.931651204242561, 15.055059523809522, 15.238101315493196, 16.1091107306417, 16.855624978633177, 16.97733093114879, 17.126734429303692, 17.31682949276132, 17.2389478726436, 18.06595365418895, 18.936863477093365, 19.1707731167234, 19.325591218055067, 19.449977934533212, 19.60684486665399, 19.802771515908, 20.576133810395522, 20.854648399254938, 21.16414728543397, 21.4792081961527, 21.75115890279843, 20.93166477215288, 20.94218473524939, 21.07157070016167, 21.23200329082682, 22.10397378608499, 22.218713953250045, 22.272597819537545, 22.49298489016274, 22.57687236693878, 22.76877682274426, 23.020470004344332, 23.142911610401022, 23.355883240695483, 23.67748788590533, 23.963335085762193, 24.224687376199856, 24.22435763500081, 24.5366380377822, 24.737441896514067, 24.914914914914913, 25.013597949720534, 25.207455413148434, 25.456930531495566, 25.47754876702245, 25.862163047036997, 26.424491013498397, 26.906991010527566, 26.722903574626272, 27.14259897095718, 27.14259897095718, 27.095877937997614, 27.095877937997614, 26.842040306942543, 26.86817082088958, 27.13338888047266, 27.018492968524573, 26.951657454242877, 26.809119478722177, 26.617649363471575, 26.44540835355978, 26.401948988066515, 26.359220032077342, 26.359220032077342, 26.15024169423576, 26.069612788773156, 25.99156400828535, 25.953468770190106, 25.879064008285347, 25.727093126457586, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 40, 58, 75, 94, 112, 131, 150, 170, 190, 208, 223, 240, 253, 268, 282, 294, 306, 320, 331, 344, 354, 365, 377, 390, 402, 415, 427, 438, 449, 458, 469, 483, 497, 508, 522, 531, 539, 553, 566, 579, 590, 601, 610, 620, 628, 635, 643, 651, 658, 663, 669, 673, 680, 685, 690, 694, 700, 706, 712, 717, 724, 729, 735, 742, 749, 754, 761, 767, 772, 777, 782, 787, 792, 797, 804, 810, 811, 812, 812, 812, 812, 812, 813, 814, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 5, 9, 13, 17, 18, 19, 22, 24, 26, 27, 29, 30, 31, 34, 35, 38, 40, 42, 42, 42, 42, 43, 48, 48, 49, 49, 50, 52, 53, 56, 56, 56, 57, 59, 60, 62, 62, 66, 67, 67, 68, 70, 72, 73, 74, 74, 74, 74, 74, 75, 76, 79, 79, 81, 81, 85, 87, 87, 90, 91, 91, 94, 98, 98, 99, 101, 106, 108, 115, 121, 123, 125, 127, 131, 135, 136, 139, 143, 143, 143, 144, 144, 147, 150, 152, 154, 155, 159, 162, 165, 166, 167, 167, 171, 173, 175, 176, 178, 180, 185], "exact_score": [7.641657905799261, 7.762403481598755, 7.880938202990603, 7.901836902535109, 7.806837222572864, 7.914361872695205, 8.021339105691212, 8.14481336640586, 9.65162379764737, 9.72765360139131, 9.84980288933188, 9.875603524782463, 9.992090819340262, 10.036211191059854, 9.990790171190534, 12.23295664749605, 12.211491707489182, 12.220628509521521, 12.243531821330683, 12.414877923320738, 12.5466188401225, 12.715229732524833, 12.764955364116053, 12.68991247880455, 12.870312789436078, 12.952782194453686, 13.110066529939884, 13.272261411482871, 13.33889084028265, 13.439711408401383, 14.931651204242561, 15.055059523809522, 15.238101315493196, 16.1091107306417, 16.855624978633177, 16.97733093114879, 17.126734429303692, 17.31682949276132, 17.2389478726436, 18.06595365418895, 18.936863477093365, 19.1707731167234, 19.325591218055067, 19.449977934533212, 19.60684486665399, 19.802771515908, 20.576133810395522, 20.854648399254938, 21.16414728543397, 21.4792081961527, 21.75115890279843, 20.93166477215288, 20.94218473524939, 21.07157070016167, 21.23200329082682, 22.10397378608499, 22.218713953250045, 22.272597819537545, 22.49298489016274, 22.57687236693878, 22.76877682274426, 23.020470004344332, 23.142911610401022, 23.355883240695483, 23.67748788590533, 23.963335085762193, 24.224687376199856, 24.22435763500081, 24.5366380377822, 24.737441896514067, 24.914914914914913, 25.013597949720534, 25.207455413148434, 25.456930531495566, 25.47754876702245, 25.862163047036997, 26.424491013498397, 26.906991010527566, 26.722903574626272, 27.14259897095718, 27.14259897095718, 27.095877937997614, 27.095877937997614, 26.842040306942543, 26.86817082088958, 27.13338888047266, 27.018492968524573, 26.951657454242877, 26.809119478722177, 26.617649363471575, 26.44540835355978, 26.401948988066515, 26.359220032077342, 26.359220032077342, 26.15024169423576, 26.069612788773156, 25.99156400828535, 25.953468770190106, 25.879064008285347, 25.727093126457586, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.435738807025118, "threshold_area100": 19.42695006292874, "threshold_area25": 10.568192059954423}, "test": {"score": [7.8873978710439285, 8.70561916963328, 9.33532923283876, 9.74035946107897, 9.727813224445855, 9.737650125015945, 10.635289609257134, 10.998170045287505, 11.696704822548945, 11.709442314369948, 11.971020888381556, 12.142074350112992, 12.464214885435213, 12.399776498181621, 12.40199377836227, 12.39714200728059, 12.326045786189152, 12.691383281642896, 12.618804016635094, 13.001343132630213, 13.045070629564309, 13.192874891330716, 13.27204709217332, 13.276151238596656, 13.286005804054035, 13.253028200916308, 13.476674980103622, 13.971482001700132, 14.208606058418274, 14.501093061177588, 14.523377552994923, 14.585912771461606, 14.670068040356279, 15.114235813102928, 15.434559680513281, 15.847105827648544, 15.80644114469083, 16.18035461429173, 16.271315577574086, 16.318374748847877, 16.5014748360614, 16.69389871435104, 16.8624567235031, 17.295644721564575, 17.845058856757007, 17.96088523364303, 18.167494946501055, 18.38703761793827, 18.97834314112618, 19.133792446035496, 19.22674859574374, 19.422821039782185, 19.568617727011457, 19.95768472783515, 19.867013188044204, 19.833759673437715, 19.955154750453914, 20.363580137739266, 20.59594850913273, 21.185289992725085, 21.364018285831033, 21.598738974499092, 21.783817470536736, 22.24129149011762, 22.707320004071395, 23.064882220706686, 23.30309791164621, 23.797398456041556, 24.07969122662327, 24.377157287796738, 24.640353360404372, 24.756324038413055, 25.01044246728027, 25.337058013469573, 25.65881153546107, 26.042653388803654, 26.404824404749526, 26.877731084010627, 26.95834144311357, 26.982659260770042, 26.922593008964117, 26.887257276733667, 26.846831963531244, 26.852667647437155, 26.718561295187243, 26.6716805469749, 26.670566923221628, 26.670566923221628, 26.76887018522576, 26.691504362593744, 26.636272554852376, 26.921686142259937, 26.768380814162807, 26.768380814162807, 26.67557860761279, 27.14792366854223, 27.125806834110566, 27.093575865549646, 27.05958260374856, 26.930889453610295, 27.001009067886244], "calls": [0.0, 0.8655126498002662, 1.4980026631158456, 2.2969374167776295, 3.095872170439414, 3.661784287616511, 4.2942743009320905, 4.826897470039946, 5.492676431424767, 5.958721704394141, 6.324900133155792, 7.589880159786951, 8.821571238348868, 9.687083888149134, 10.752330226364846, 11.218375499334222, 12.416777629826898, 13.948069241011984, 14.513981358189081, 16.145139813581892, 16.611185086551263, 17.643142476697736, 18.575233022636482, 19.241011984021306, 20.00665778961385, 20.938748335552596, 21.50466045272969, 24.134487350199734, 24.73368841544607, 25.865512649800266, 26.66444740346205, 27.762982689747005, 29.860186418109187, 31.22503328894807, 32.7896138482024, 33.85486018641811, 34.8202396804261, 35.519307589880164, 36.61784287616511, 37.050599201065246, 38.61517976031957, 40.94540612516644, 42.24367509986685, 43.54194407456725, 44.54061251664447, 45.572569906790946, 46.60452729693741, 47.237017310252995, 48.968042609853526, 49.83355525965379, 50.36617842876166, 51.2982689747004, 52.29693741677763, 54.260985352862846, 55.09320905459387, 56.19174434087883, 57.05725699067909, 58.88814913448736, 59.886817576564574, 61.118508655126504, 61.91744340878829, 63.215712383488686, 64.11451398135819, 65.34620505992011, 66.41145139813582, 67.9427430093209, 68.70838881491345, 69.6404793608522, 70.40612516644474, 70.80559254327564, 71.93741677762982, 72.66977363515313, 73.23568575233023, 73.96804260985353, 74.30093209054593, 75.36617842876166, 76.43142476697736, 77.79627163781625, 78.56191744340879, 79.29427430093209, 80.32623169107856, 81.19174434087883, 81.95739014647138, 82.9560585885486, 84.32090545938749, 85.41944074567243, 86.61784287616511, 87.15046604527296, 88.01597869507324, 89.14780292942743, 90.0133155792277, 91.711051930759, 93.64181091877496, 94.0745672436751, 94.87350199733689, 95.70572569906791, 96.13848202396804, 96.80426098535287, 98.26897470039947, 98.80159786950732, 100.0], "rescued": [0, 72, 125, 188, 253, 301, 351, 398, 453, 492, 521, 606, 696, 752, 810, 839, 907, 995, 1026, 1116, 1142, 1197, 1245, 1278, 1316, 1361, 1387, 1519, 1549, 1608, 1648, 1708, 1817, 1886, 1965, 2019, 2064, 2101, 2157, 2178, 2254, 2361, 2425, 2486, 2528, 2567, 2607, 2629, 2684, 2707, 2721, 2747, 2770, 2818, 2840, 2868, 2888, 2936, 2959, 2992, 3012, 3041, 3067, 3097, 3125, 3165, 3187, 3214, 3234, 3247, 3275, 3295, 3311, 3330, 3340, 3363, 3389, 3425, 3439, 3443, 3445, 3446, 3446, 3447, 3447, 3447, 3448, 3448, 3451, 3451, 3451, 3453, 3453, 3453, 3454, 3457, 3458, 3459, 3460, 3460, 3461], "harmed": [0, 5, 13, 27, 33, 37, 42, 48, 55, 57, 62, 73, 79, 86, 90, 93, 103, 108, 113, 119, 120, 123, 126, 131, 136, 143, 148, 165, 168, 176, 184, 193, 208, 218, 228, 237, 247, 253, 263, 266, 278, 305, 318, 326, 334, 340, 344, 348, 353, 355, 357, 360, 366, 373, 377, 380, 384, 390, 401, 408, 410, 413, 422, 438, 444, 455, 461, 467, 471, 476, 487, 494, 495, 502, 507, 514, 522, 537, 547, 551, 557, 560, 563, 568, 580, 584, 587, 587, 592, 596, 600, 606, 617, 617, 626, 630, 633, 639, 648, 660, 676], "exact_score": [7.8873978710439285, 8.711905945584173, 9.759061886916673, 9.739513117911134, 10.628136053498794, 11.400562817590485, 11.71310413203949, 12.169965396089557, 12.162435264180536, 12.416990090638611, 12.42146427354488, 12.376203348287346, 12.37003977428649, 12.32986792138694, 12.693710007076001, 12.644392206518019, 12.738177107369273, 13.07384394259847, 13.213614172045274, 13.289872444747889, 13.296350887509217, 13.258111326175213, 13.478764712915888, 13.610664357499672, 13.802099454254785, 14.215095691338952, 14.491412076627148, 14.551195444932496, 14.604015065201176, 14.563784167387208, 14.67497777171594, 15.086659527304679, 15.354194110289084, 15.46402079534327, 15.807706282041433, 15.965458312115688, 16.20063804724139, 16.318596262522224, 16.390990349383657, 16.51901965374084, 16.581402593234067, 16.703216593070742, 16.844954653859766, 17.28482840102715, 17.53246241188994, 17.90990724557391, 18.060319996388124, 18.22688244223881, 18.647878948412068, 18.97834314112618, 19.171126907398513, 19.37661242357089, 19.52866293540529, 19.68693044792821, 19.91645867504763, 20.085963362592196, 19.785304491319806, 19.965179813110556, 20.140039215713156, 20.37016655147289, 20.613999519469854, 20.942191841420424, 21.383356071028587, 21.558406412871808, 21.738716784784856, 21.996752423301945, 22.571446425574866, 22.81995822833164, 23.07697096173684, 23.604158888296734, 23.916980025314103, 24.402118660122465, 24.6593716179316, 24.882020958804, 25.337058013469573, 25.904808281172787, 26.339091687585732, 26.639020619218922, 26.91755042057317, 26.990661766301027, 26.930276163284155, 26.903783012249523, 26.846831963531244, 26.8450827535119, 26.7494616438842, 26.711179042123607, 26.655987872315436, 26.670566923221628, 26.76887018522576, 26.718240825500423, 26.636272554852376, 26.621676561274615, 26.898235658810098, 26.809069453669782, 26.768380814162807, 26.67557860761279, 27.15197114598916, 27.067938784075, 27.08235242094074, 26.885906728327356, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.287230550209568, "threshold_area100": 19.25997655223746, "threshold_area25": 11.544219337244504}, "selected_budget": 79, "selected": {"val": {"score": 27.14259897095718, "calls": 78.88730385164051, "rescued": 812, "harmed": 143}, "test": {"score": 26.982659260770042, "calls": 79.29427430093209, "rescued": 3443, "harmed": 551}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.1012866497039795, "operator": "ge"}, {"budget": 0.02, "threshold": 3.006488561630249, "operator": "ge"}, {"budget": 0.03, "threshold": 2.926449775695801, "operator": "ge"}, {"budget": 0.04, "threshold": 2.8527021408081055, "operator": "ge"}, {"budget": 0.05, "threshold": 2.798243284225464, "operator": "ge"}, {"budget": 0.06, "threshold": 2.755965232849121, "operator": "ge"}, {"budget": 0.07, "threshold": 2.7320022583007812, "operator": "ge"}, {"budget": 0.08, "threshold": 2.689767360687256, "operator": "ge"}, {"budget": 0.09, "threshold": 2.6192402839660645, "operator": "ge"}, {"budget": 0.1, "threshold": 2.5946290493011475, "operator": "ge"}, {"budget": 0.11, "threshold": 2.536170482635498, "operator": "ge"}, {"budget": 0.12, "threshold": 2.510686159133911, "operator": "ge"}, {"budget": 0.13, "threshold": 2.4746928215026855, "operator": "ge"}, {"budget": 0.14, "threshold": 2.443772315979004, "operator": "ge"}, {"budget": 0.15, "threshold": 2.409365653991699, "operator": "ge"}, {"budget": 0.16, "threshold": 2.3875691890716553, "operator": "ge"}, {"budget": 0.17, "threshold": 2.351377010345459, "operator": "ge"}, {"budget": 0.18, "threshold": 2.321491241455078, "operator": "ge"}, {"budget": 0.19, "threshold": 2.286742925643921, "operator": "ge"}, {"budget": 0.2, "threshold": 2.2771759033203125, "operator": "ge"}, {"budget": 0.21, "threshold": 2.258881092071533, "operator": "ge"}, {"budget": 0.22, "threshold": 2.2346200942993164, "operator": "ge"}, {"budget": 0.23, "threshold": 2.224958896636963, "operator": "ge"}, {"budget": 0.24, "threshold": 2.2093305587768555, "operator": "ge"}, {"budget": 0.25, "threshold": 2.185969352722168, "operator": "ge"}, {"budget": 0.26, "threshold": 2.1663119792938232, "operator": "ge"}, {"budget": 0.27, "threshold": 2.1480712890625, "operator": "ge"}, {"budget": 0.28, "threshold": 2.1184945106506348, "operator": "ge"}, {"budget": 0.29, "threshold": 2.0962185859680176, "operator": "ge"}, {"budget": 0.3, "threshold": 2.0791845321655273, "operator": "ge"}, {"budget": 0.31, "threshold": 2.0555038452148438, "operator": "ge"}, {"budget": 0.32, "threshold": 2.037748098373413, "operator": "ge"}, {"budget": 0.33, "threshold": 2.024320602416992, "operator": "ge"}, {"budget": 0.34, "threshold": 2.0064873695373535, "operator": "ge"}, {"budget": 0.35, "threshold": 1.993760108947754, "operator": "ge"}, {"budget": 0.36, "threshold": 1.985084891319275, "operator": "ge"}, {"budget": 0.37, "threshold": 1.9718196392059326, "operator": "ge"}, {"budget": 0.38, "threshold": 1.9482219219207764, "operator": "ge"}, {"budget": 0.39, "threshold": 1.9297785758972168, "operator": "ge"}, {"budget": 0.4, "threshold": 1.9134490489959717, "operator": "ge"}, {"budget": 0.41, "threshold": 1.881414771080017, "operator": "ge"}, {"budget": 0.42, "threshold": 1.856350302696228, "operator": "ge"}, {"budget": 0.43, "threshold": 1.839491605758667, "operator": "ge"}, {"budget": 0.44, "threshold": 1.8165278434753418, "operator": "ge"}, {"budget": 0.45, "threshold": 1.8062350749969482, "operator": "ge"}, {"budget": 0.46, "threshold": 1.7889426946640015, "operator": "ge"}, {"budget": 0.47, "threshold": 1.767806053161621, "operator": "ge"}, {"budget": 0.48, "threshold": 1.752956509590149, "operator": "ge"}, {"budget": 0.49, "threshold": 1.7278624773025513, "operator": "ge"}, {"budget": 0.5, "threshold": 1.7038170099258423, "operator": "ge"}, {"budget": 0.51, "threshold": 1.6843247413635254, "operator": "ge"}, {"budget": 0.52, "threshold": 1.6676996946334839, "operator": "ge"}, {"budget": 0.53, "threshold": 1.6553272008895874, "operator": "ge"}, {"budget": 0.54, "threshold": 1.6224583387374878, "operator": "ge"}, {"budget": 0.55, "threshold": 1.6008970737457275, "operator": "ge"}, {"budget": 0.56, "threshold": 1.5822358131408691, "operator": "ge"}, {"budget": 0.57, "threshold": 1.5541307926177979, "operator": "ge"}, {"budget": 0.58, "threshold": 1.5333796739578247, "operator": "ge"}, {"budget": 0.59, "threshold": 1.4961411952972412, "operator": "ge"}, {"budget": 0.6, "threshold": 1.4610595703125, "operator": "ge"}, {"budget": 0.61, "threshold": 1.4271783828735352, "operator": "ge"}, {"budget": 0.62, "threshold": 1.4060057401657104, "operator": "ge"}, {"budget": 0.63, "threshold": 1.3755881786346436, "operator": "ge"}, {"budget": 0.64, "threshold": 1.3523285388946533, "operator": "ge"}, {"budget": 0.65, "threshold": 1.3261723518371582, "operator": "ge"}, {"budget": 0.66, "threshold": 1.2779412269592285, "operator": "ge"}, {"budget": 0.67, "threshold": 1.2466065883636475, "operator": "ge"}, {"budget": 0.68, "threshold": 1.1944358348846436, "operator": "ge"}, {"budget": 0.69, "threshold": 1.1731926202774048, "operator": "ge"}, {"budget": 0.7, "threshold": 1.1502788066864014, "operator": "ge"}, {"budget": 0.71, "threshold": 1.1077477931976318, "operator": "ge"}, {"budget": 0.72, "threshold": 1.0725600719451904, "operator": "ge"}, {"budget": 0.73, "threshold": 1.0092194080352783, "operator": "ge"}, {"budget": 0.74, "threshold": 0.8991430401802063, "operator": "ge"}, {"budget": 0.75, "threshold": 0.7613353133201599, "operator": "ge"}, {"budget": 0.76, "threshold": 0.6067081093788147, "operator": "ge"}, {"budget": 0.77, "threshold": 0.3740063011646271, "operator": "ge"}, {"budget": 0.78, "threshold": -0.003682531416416168, "operator": "ge"}, {"budget": 0.79, "threshold": -0.40612924098968506, "operator": "ge"}, {"budget": 0.8, "threshold": -0.6936975717544556, "operator": "ge"}, {"budget": 0.81, "threshold": -0.9239163398742676, "operator": "ge"}, {"budget": 0.82, "threshold": -1.1095831394195557, "operator": "ge"}, {"budget": 0.83, "threshold": -1.2025704383850098, "operator": "ge"}, {"budget": 0.84, "threshold": -1.2888436317443848, "operator": "ge"}, {"budget": 0.85, "threshold": -1.3883113861083984, "operator": "ge"}, {"budget": 0.86, "threshold": -1.4538121223449707, "operator": "ge"}, {"budget": 0.87, "threshold": -1.500826120376587, "operator": "ge"}, {"budget": 0.88, "threshold": -1.575981616973877, "operator": "ge"}, {"budget": 0.89, "threshold": -1.6152225732803345, "operator": "ge"}, {"budget": 0.9, "threshold": -1.6902927160263062, "operator": "ge"}, {"budget": 0.91, "threshold": -1.7378578186035156, "operator": "ge"}, {"budget": 0.92, "threshold": -1.7617508172988892, "operator": "ge"}, {"budget": 0.93, "threshold": -1.8377251625061035, "operator": "ge"}, {"budget": 0.94, "threshold": -1.8889565467834473, "operator": "ge"}, {"budget": 0.95, "threshold": -1.9368078708648682, "operator": "ge"}, {"budget": 0.96, "threshold": -1.969069480895996, "operator": "ge"}, {"budget": 0.97, "threshold": -2.0276331901550293, "operator": "ge"}, {"budget": 0.98, "threshold": -2.1073763370513916, "operator": "ge"}, {"budget": 0.99, "threshold": -2.2030978202819824, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.719935860289169, 10.844680292672994, 10.744624990818503, 10.83049540298611, 10.719661215365116, 10.63999930864096, 12.84478654311178, 12.686087750224706, 12.673031522841502, 12.54725872741882, 12.653636698333317, 12.548506590331598, 12.28731231977727, 12.114177611135016, 12.231249934643957, 12.06803345245191, 13.423159662354264, 13.408015697403597, 13.431897835520065, 13.576445437158599, 14.849524241272801, 14.919713245357535, 14.90916693796275, 14.95028208987019, 14.795698708742187, 14.804355962312668, 14.942774206925149, 14.94865342773071, 15.029255155213534, 15.176332211264002, 15.111745474064314, 14.811497629003584, 14.884356042484223, 14.659367957139608, 14.783566366869094, 16.308416647291327, 15.964765767915374, 16.013311591986124, 17.016770503302283, 16.883098553638956, 16.84520040042529, 16.9767113311701, 17.129964618130817, 17.142516789759796, 17.352956183601346, 17.345115430850132, 17.543340380549683, 17.70965315172632, 17.668940945876052, 17.903079248721607, 18.081952139198293, 18.3714658458904, 18.53864311942647, 18.742183742183744, 18.697108048941136, 18.88894175866007, 19.083283102270446, 19.92024544645272, 20.721054645915704, 20.889592543792755, 21.124502432551967, 21.45250294666553, 21.733922506633753, 21.96346707484236, 22.772789541082226, 23.13941539228676, 23.237423147224995, 24.12996985114, 24.414553529368277, 24.545693558595843, 24.868672500251446, 25.190892394300384, 25.162758194789784, 25.65984375135032, 25.954627217171073, 26.452617024655716, 26.842888446425, 26.722903574626272, 27.21007535422304, 27.163354321263473, 27.163354321263473, 27.042581374403568, 27.33315310008334, 27.081518226019508, 27.10037952782274, 26.96605644546148, 26.788537549407117, 26.743560038162734, 26.529981333079988, 26.29610549580309, 26.2540886890804, 26.21276637503081, 26.069612788773156, 25.99156400828535, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.800339784578807, 25.554334467763283, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 15, 28, 41, 54, 67, 80, 95, 107, 117, 131, 146, 157, 171, 186, 199, 211, 225, 236, 254, 268, 279, 285, 299, 310, 322, 335, 349, 361, 376, 387, 397, 406, 419, 430, 445, 457, 467, 476, 488, 503, 512, 518, 527, 538, 550, 558, 567, 575, 585, 594, 604, 613, 622, 629, 638, 646, 653, 661, 672, 678, 686, 696, 705, 713, 724, 734, 741, 750, 756, 764, 770, 778, 783, 790, 797, 804, 809, 811, 813, 813, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 2, 3, 4, 4, 5, 6, 8, 9, 10, 11, 11, 12, 14, 16, 16, 18, 20, 21, 22, 22, 23, 23, 24, 25, 28, 30, 30, 31, 32, 33, 35, 37, 38, 42, 43, 43, 48, 50, 50, 55, 57, 61, 61, 65, 67, 69, 69, 69, 73, 73, 75, 79, 81, 81, 84, 85, 86, 86, 92, 93, 94, 95, 96, 98, 100, 105, 109, 110, 115, 120, 121, 124, 128, 128, 133, 135, 139, 143, 143, 144, 144, 145, 147, 151, 154, 156, 159, 160, 164, 168, 169, 170, 173, 175, 177, 177, 177, 178, 182, 185], "exact_score": [7.641657905799261, 7.719935860289169, 10.844680292672994, 10.744624990818503, 10.83049540298611, 10.719661215365116, 10.63999930864096, 12.84478654311178, 12.686087750224706, 12.673031522841502, 12.54725872741882, 12.653636698333317, 12.548506590331598, 12.28731231977727, 12.114177611135016, 12.231249934643957, 12.06803345245191, 13.423159662354264, 13.408015697403597, 13.431897835520065, 13.576445437158599, 14.849524241272801, 14.919713245357535, 14.90916693796275, 14.95028208987019, 14.795698708742187, 14.804355962312668, 14.942774206925149, 14.94865342773071, 15.029255155213534, 15.176332211264002, 15.111745474064314, 14.811497629003584, 14.884356042484223, 14.659367957139608, 14.783566366869094, 16.308416647291327, 15.964765767915374, 16.013311591986124, 17.016770503302283, 16.883098553638956, 16.84520040042529, 16.9767113311701, 17.129964618130817, 17.142516789759796, 17.352956183601346, 17.345115430850132, 17.543340380549683, 17.70965315172632, 17.668940945876052, 17.903079248721607, 18.081952139198293, 18.3714658458904, 18.53864311942647, 18.742183742183744, 18.697108048941136, 18.88894175866007, 19.083283102270446, 19.92024544645272, 20.721054645915704, 20.889592543792755, 21.124502432551967, 21.45250294666553, 21.733922506633753, 21.96346707484236, 22.772789541082226, 23.13941539228676, 23.237423147224995, 24.12996985114, 24.414553529368277, 24.545693558595843, 24.868672500251446, 25.190892394300384, 25.162758194789784, 25.65984375135032, 25.954627217171073, 26.452617024655716, 26.842888446425, 26.722903574626272, 27.21007535422304, 27.163354321263473, 27.163354321263473, 27.042581374403568, 27.33315310008334, 27.081518226019508, 27.10037952782274, 26.96605644546148, 26.788537549407117, 26.743560038162734, 26.529981333079988, 26.29610549580309, 26.2540886890804, 26.21276637503081, 26.069612788773156, 25.99156400828535, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.800339784578807, 25.554334467763283, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.21170007206319, "threshold_area100": 19.202714671673082, "threshold_area25": 12.058658724371444}, "test": {"score": [7.8873978710439285, 7.917438745931732, 8.712182635990997, 8.747043229787172, 8.863747829338573, 8.87112928981803, 8.923419024842595, 8.97753841171231, 9.028285675520424, 10.285324699905804, 10.263443135920214, 10.742833581792745, 10.801929777230793, 10.74422867541073, 10.719628121998639, 10.833720242020387, 10.879409011930678, 10.935413885001044, 11.395331091352624, 11.529745159639464, 11.923188842320991, 11.985750940421442, 12.038299372911455, 12.047107955036344, 12.11289615655792, 12.374312320016855, 12.416926267378505, 13.09146898057999, 13.15447917312999, 13.191414622369676, 13.196396910782864, 13.209643093382418, 13.214716995597641, 13.473671316872785, 13.623189624248946, 13.671627172034448, 13.718191172868528, 14.055063866450206, 14.168456399186615, 14.249736309740179, 14.60947141995462, 14.787723183329124, 15.029549636096473, 15.155048940346003, 15.667883617015304, 15.656088578172648, 15.740888829280946, 16.064158978125747, 16.15847124427731, 16.693353051769716, 16.971238300135177, 17.144712446666666, 17.22684070850064, 17.357628362515314, 17.8952327953289, 18.01562357949856, 18.462413591221384, 18.974642630179638, 19.127209649031236, 19.907140938928265, 20.43983334552391, 20.662778999463967, 20.743117637296326, 21.10114428351029, 21.38197779683324, 21.525564939663976, 22.117670417320856, 22.575206294992974, 23.327214849033794, 23.622745678664543, 23.975328782378256, 24.001620071694855, 24.329107217806943, 24.753170406998265, 25.345092412009013, 25.964304640775232, 26.246461884198485, 26.80127441909481, 26.95834144311357, 27.017781846870687, 26.956006534759357, 26.88757111564729, 26.826192287139918, 26.684349792344204, 26.602895852284885, 26.61209026434864, 26.53297254151181, 26.681925681684938, 26.651119043930443, 26.624664017475418, 26.744473174729176, 26.71496057243753, 26.714876828582103, 26.886328833842644, 27.095325309611024, 27.05616592756272, 27.030528846088075, 27.06850030511602, 26.99849124557412, 27.15218029549246, 27.001009067886244], "calls": [0.0, 0.7989347536617843, 1.4980026631158456, 1.9973368841544608, 3.162450066577896, 4.027962716378163, 4.660452729693742, 5.292942743009322, 6.358189081225034, 7.756324900133156, 8.455392809587217, 10.286284953395473, 11.118508655126497, 12.48335552596538, 13.482023968042608, 15.013315579227696, 15.679094540612518, 17.010652463382154, 18.275632490013315, 19.573901464713714, 20.139813581890813, 20.938748335552596, 22.13715046604527, 22.569906790945407, 23.40213049267643, 24.500665778961384, 25.26631158455393, 26.498002663115845, 27.82956058588549, 29.26098535286285, 30.126498002663116, 31.358189081225035, 32.090545938748335, 32.889480692410125, 34.02130492676431, 34.78695073235686, 35.319573901464715, 35.918774966711055, 37.11717709720373, 38.24900133155793, 39.21438082556591, 40.71238348868176, 42.177097203728366, 43.04260985352863, 44.30758988015979, 44.607190412782955, 45.372836218375504, 46.504660452729695, 47.13715046604528, 48.76830892143808, 50.13315579227696, 50.96537949400799, 51.597869507323566, 52.23035952063915, 53.8948069241012, 54.826897470039945, 55.59254327563249, 56.724367509986685, 57.52330226364847, 59.254327563249, 61.085219707057256, 62.18375499334221, 62.81624500665779, 63.948069241011986, 64.84687083888149, 65.67909454061251, 67.17709720372837, 67.87616511318242, 69.67376830892144, 70.10652463382158, 70.70572569906791, 71.6711051930759, 72.37017310252996, 73.5352862849534, 74.73368841544608, 75.7323568575233, 76.59786950732357, 77.762982689747, 78.56191744340879, 79.32756324900133, 80.05992010652463, 80.89214380825565, 82.2237017310253, 83.25565912117176, 83.82157123834887, 85.58588548601864, 86.58455392809587, 87.38348868175765, 88.68175765645806, 89.14780292942743, 91.01198402130493, 91.87749667110519, 92.21038615179761, 93.40878828229027, 94.2410119840213, 95.23968042609854, 95.67243675099867, 96.63781624500666, 97.7363515312916, 98.5352862849534, 100.0], "rescued": [0, 43, 85, 112, 176, 221, 255, 287, 342, 410, 451, 554, 598, 681, 731, 806, 837, 907, 975, 1034, 1063, 1102, 1160, 1180, 1224, 1279, 1315, 1366, 1429, 1493, 1538, 1589, 1619, 1659, 1711, 1743, 1771, 1802, 1860, 1905, 1946, 2012, 2071, 2104, 2167, 2180, 2208, 2258, 2283, 2350, 2397, 2434, 2461, 2492, 2550, 2583, 2615, 2658, 2689, 2755, 2820, 2866, 2890, 2933, 2972, 3000, 3063, 3092, 3159, 3176, 3200, 3226, 3249, 3290, 3336, 3365, 3388, 3419, 3439, 3447, 3447, 3447, 3448, 3448, 3448, 3449, 3449, 3452, 3452, 3452, 3453, 3454, 3455, 3456, 3457, 3458, 3458, 3459, 3460, 3461, 3461], "harmed": [0, 4, 7, 8, 11, 15, 15, 15, 18, 21, 26, 37, 38, 46, 52, 58, 59, 64, 71, 76, 79, 81, 86, 88, 91, 97, 100, 107, 113, 121, 129, 138, 144, 151, 157, 161, 165, 165, 172, 175, 179, 195, 204, 207, 215, 220, 224, 230, 232, 251, 261, 264, 268, 275, 288, 294, 299, 308, 313, 325, 340, 352, 360, 371, 383, 394, 410, 418, 440, 442, 447, 458, 466, 483, 502, 509, 521, 535, 547, 554, 557, 561, 568, 577, 583, 589, 595, 597, 600, 601, 611, 618, 622, 629, 632, 639, 640, 644, 653, 658, 676], "exact_score": [7.8873978710439285, 7.935176994021936, 8.747043229787172, 8.856123781529874, 8.868370830982117, 8.94702528057608, 9.00429102059107, 10.24077075102292, 10.307797675798742, 10.264855411755283, 10.75235841120734, 10.78630211661387, 10.723855839034236, 10.725406684776885, 10.750121524209662, 10.831277169414774, 10.875330806692265, 10.932018211483442, 11.395797609759786, 11.493120087412228, 11.523199482559363, 11.986571715890102, 12.03322958518523, 12.107462542165065, 12.17383475662957, 12.396568852141336, 12.757887973251671, 13.136234418557057, 13.161929417646936, 13.18477247393464, 13.203181586397324, 13.162999149678766, 13.2089409736585, 13.469384498312357, 13.61733129860162, 13.68214558243813, 14.06607687384116, 14.125922883474084, 14.197202104569135, 14.577837501796179, 14.680898517059704, 14.782861042320894, 15.016456908466314, 15.146421881073854, 15.40636466220899, 15.704178129375387, 15.842145650113698, 16.120275826329156, 16.603861666830568, 16.6994774852209, 16.9880327778999, 17.144712446666666, 17.295506655136524, 17.607007809967193, 17.93294173164403, 18.17480165816315, 18.702532392663677, 19.024031357872982, 19.17098690637754, 19.819090776098975, 20.05426205221103, 20.432407471107492, 20.590924276769325, 20.909512723767023, 21.116092285973835, 21.405280630033797, 21.628594609474312, 21.933120482981344, 22.614387079534747, 22.99860294361696, 23.545585111085657, 24.04416857959848, 24.22530728167576, 24.489966757947716, 25.020588603786504, 25.462438742276206, 26.033196329802767, 26.515811711890148, 26.835593860804046, 27.03401573623391, 26.989568494530463, 26.88757111564729, 26.87024181233183, 26.700648174833457, 26.5866682169169, 26.642983833873203, 26.604764418030246, 26.620940553143946, 26.6745583652569, 26.624664017475418, 26.577972176659987, 26.744473174729176, 26.691771087339962, 26.925738418154143, 27.102382472631493, 27.070050270982033, 27.01469064775977, 27.066524985211437, 26.99849124557412, 27.11616755844521, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 18.13213958521161, "threshold_area100": 18.135892095042784, "threshold_area25": 10.404467321122004}, "selected_budget": 83, "selected": {"val": {"score": 27.33315310008334, "calls": 82.88159771754636, "rescued": 814, "harmed": 147}, "test": {"score": 26.684349792344204, "calls": 83.25565912117176, "rescued": 3448, "harmed": 577}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.921313762664795, "operator": "ge"}, {"budget": 0.02, "threshold": 1.821012020111084, "operator": "ge"}, {"budget": 0.03, "threshold": 1.7843191623687744, "operator": "ge"}, {"budget": 0.04, "threshold": 1.734431266784668, "operator": "ge"}, {"budget": 0.05, "threshold": 1.6972341537475586, "operator": "ge"}, {"budget": 0.06, "threshold": 1.687016487121582, "operator": "ge"}, {"budget": 0.07, "threshold": 1.6653512716293335, "operator": "ge"}, {"budget": 0.08, "threshold": 1.6434216499328613, "operator": "ge"}, {"budget": 0.09, "threshold": 1.6337242126464844, "operator": "ge"}, {"budget": 0.1, "threshold": 1.6097078323364258, "operator": "ge"}, {"budget": 0.11, "threshold": 1.5998048782348633, "operator": "ge"}, {"budget": 0.12, "threshold": 1.5887441635131836, "operator": "ge"}, {"budget": 0.13, "threshold": 1.5798265933990479, "operator": "ge"}, {"budget": 0.14, "threshold": 1.5693352222442627, "operator": "ge"}, {"budget": 0.15, "threshold": 1.5623760223388672, "operator": "ge"}, {"budget": 0.16, "threshold": 1.550950288772583, "operator": "ge"}, {"budget": 0.17, "threshold": 1.5419731140136719, "operator": "ge"}, {"budget": 0.18, "threshold": 1.5273232460021973, "operator": "ge"}, {"budget": 0.19, "threshold": 1.5140503644943237, "operator": "ge"}, {"budget": 0.2, "threshold": 1.504842758178711, "operator": "ge"}, {"budget": 0.21, "threshold": 1.496803641319275, "operator": "ge"}, {"budget": 0.22, "threshold": 1.4880483150482178, "operator": "ge"}, {"budget": 0.23, "threshold": 1.4821799993515015, "operator": "ge"}, {"budget": 0.24, "threshold": 1.4733388423919678, "operator": "ge"}, {"budget": 0.25, "threshold": 1.459087610244751, "operator": "ge"}, {"budget": 0.26, "threshold": 1.4401018619537354, "operator": "ge"}, {"budget": 0.27, "threshold": 1.4247366189956665, "operator": "ge"}, {"budget": 0.28, "threshold": 1.4092597961425781, "operator": "ge"}, {"budget": 0.29, "threshold": 1.3986049890518188, "operator": "ge"}, {"budget": 0.3, "threshold": 1.382079005241394, "operator": "ge"}, {"budget": 0.31, "threshold": 1.3581633567810059, "operator": "ge"}, {"budget": 0.32, "threshold": 1.3435659408569336, "operator": "ge"}, {"budget": 0.33, "threshold": 1.3255019187927246, "operator": "ge"}, {"budget": 0.34, "threshold": 1.3133962154388428, "operator": "ge"}, {"budget": 0.35, "threshold": 1.2971101999282837, "operator": "ge"}, {"budget": 0.36, "threshold": 1.2860963344573975, "operator": "ge"}, {"budget": 0.37, "threshold": 1.2753827571868896, "operator": "ge"}, {"budget": 0.38, "threshold": 1.26053786277771, "operator": "ge"}, {"budget": 0.39, "threshold": 1.2488386631011963, "operator": "ge"}, {"budget": 0.4, "threshold": 1.2377065420150757, "operator": "ge"}, {"budget": 0.41, "threshold": 1.2232041358947754, "operator": "ge"}, {"budget": 0.42, "threshold": 1.2036936283111572, "operator": "ge"}, {"budget": 0.43, "threshold": 1.1888089179992676, "operator": "ge"}, {"budget": 0.44, "threshold": 1.1839816570281982, "operator": "ge"}, {"budget": 0.45, "threshold": 1.1721603870391846, "operator": "ge"}, {"budget": 0.46, "threshold": 1.1645594835281372, "operator": "ge"}, {"budget": 0.47, "threshold": 1.1562273502349854, "operator": "ge"}, {"budget": 0.48, "threshold": 1.1507320404052734, "operator": "ge"}, {"budget": 0.49, "threshold": 1.1354273557662964, "operator": "ge"}, {"budget": 0.5, "threshold": 1.1226893663406372, "operator": "ge"}, {"budget": 0.51, "threshold": 1.1021392345428467, "operator": "ge"}, {"budget": 0.52, "threshold": 1.0909559726715088, "operator": "ge"}, {"budget": 0.53, "threshold": 1.082592487335205, "operator": "ge"}, {"budget": 0.54, "threshold": 1.0769004821777344, "operator": "ge"}, {"budget": 0.55, "threshold": 1.0648318529129028, "operator": "ge"}, {"budget": 0.56, "threshold": 1.0522501468658447, "operator": "ge"}, {"budget": 0.57, "threshold": 1.0366871356964111, "operator": "ge"}, {"budget": 0.58, "threshold": 1.0214869976043701, "operator": "ge"}, {"budget": 0.59, "threshold": 1.0059236288070679, "operator": "ge"}, {"budget": 0.6, "threshold": 1.0003104209899902, "operator": "ge"}, {"budget": 0.61, "threshold": 0.9789069890975952, "operator": "ge"}, {"budget": 0.62, "threshold": 0.9677611589431763, "operator": "ge"}, {"budget": 0.63, "threshold": 0.9524420499801636, "operator": "ge"}, {"budget": 0.64, "threshold": 0.9367643594741821, "operator": "ge"}, {"budget": 0.65, "threshold": 0.9239281415939331, "operator": "ge"}, {"budget": 0.66, "threshold": 0.9062923789024353, "operator": "ge"}, {"budget": 0.67, "threshold": 0.8866369128227234, "operator": "ge"}, {"budget": 0.68, "threshold": 0.8642474412918091, "operator": "ge"}, {"budget": 0.69, "threshold": 0.8425493240356445, "operator": "ge"}, {"budget": 0.7, "threshold": 0.8197456002235413, "operator": "ge"}, {"budget": 0.71, "threshold": 0.7893372774124146, "operator": "ge"}, {"budget": 0.72, "threshold": 0.7693812847137451, "operator": "ge"}, {"budget": 0.73, "threshold": 0.7385375499725342, "operator": "ge"}, {"budget": 0.74, "threshold": 0.6902318000793457, "operator": "ge"}, {"budget": 0.75, "threshold": 0.6274529099464417, "operator": "ge"}, {"budget": 0.76, "threshold": 0.5255376696586609, "operator": "ge"}, {"budget": 0.77, "threshold": 0.32950109243392944, "operator": "ge"}, {"budget": 0.78, "threshold": -0.0032086819410324097, "operator": "ge"}, {"budget": 0.79, "threshold": -0.36463236808776855, "operator": "ge"}, {"budget": 0.8, "threshold": -0.6339066028594971, "operator": "ge"}, {"budget": 0.81, "threshold": -0.8480727076530457, "operator": "ge"}, {"budget": 0.82, "threshold": -1.03096604347229, "operator": "ge"}, {"budget": 0.83, "threshold": -1.1187233924865723, "operator": "ge"}, {"budget": 0.84, "threshold": -1.209024429321289, "operator": "ge"}, {"budget": 0.85, "threshold": -1.298789143562317, "operator": "ge"}, {"budget": 0.86, "threshold": -1.368635654449463, "operator": "ge"}, {"budget": 0.87, "threshold": -1.425076961517334, "operator": "ge"}, {"budget": 0.88, "threshold": -1.489737629890442, "operator": "ge"}, {"budget": 0.89, "threshold": -1.5196115970611572, "operator": "ge"}, {"budget": 0.9, "threshold": -1.601691484451294, "operator": "ge"}, {"budget": 0.91, "threshold": -1.6451489925384521, "operator": "ge"}, {"budget": 0.92, "threshold": -1.667928695678711, "operator": "ge"}, {"budget": 0.93, "threshold": -1.7464951276779175, "operator": "ge"}, {"budget": 0.94, "threshold": -1.785272479057312, "operator": "ge"}, {"budget": 0.95, "threshold": -1.8407938480377197, "operator": "ge"}, {"budget": 0.96, "threshold": -1.8690179586410522, "operator": "ge"}, {"budget": 0.97, "threshold": -1.9513771533966064, "operator": "ge"}, {"budget": 0.98, "threshold": -2.0044925212860107, "operator": "ge"}, {"budget": 0.99, "threshold": -2.115266799926758, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.756956858505074, 7.8696311021312635, 7.885242457733163, 7.992149176359703, 7.883115449377531, 7.966775398606806, 8.06949379219607, 8.188444486384828, 8.311665810289917, 8.322549742078113, 8.40811374159706, 10.039330904025947, 10.144407023109046, 11.550062947689325, 11.68080259574059, 11.783747907723509, 13.172860751930019, 13.231813100143846, 13.198152264579724, 13.310614117065727, 13.10551499605367, 13.074583677185657, 13.11583325850943, 13.074910678946974, 13.134593100494705, 14.170239898803441, 14.285518140939828, 14.425850159588036, 14.477782895121605, 14.51904443415479, 14.594214543993573, 14.699492981268602, 14.739885333035978, 14.798413280447964, 14.925205233110711, 15.104017024763413, 15.223203828075427, 15.379131036039851, 15.455001016958676, 15.639114453200076, 15.735634731957898, 15.73989696483682, 15.847800930307551, 16.63694635235973, 16.87229012167153, 17.078550700580458, 17.14626858648598, 17.278172211745222, 17.515886784085232, 18.448561716766466, 18.70750158533683, 20.222763300062713, 21.054221198605934, 21.30420547047136, 21.211190945349344, 21.392504556861216, 21.43597823744883, 21.770663783711992, 21.899889666330118, 22.34063410032402, 22.968172735789867, 23.036284658791057, 23.339993672561334, 24.11168537511104, 24.28942204228393, 24.671718560631362, 24.891057218643425, 24.221250771795635, 24.56615470509607, 24.9269663335048, 24.875399596826778, 24.98270210606048, 25.402937687597245, 25.886078765028415, 26.081856362028777, 26.452617024655716, 26.842888446425, 26.722903574626272, 27.21007535422304, 27.163354321263473, 27.163354321263473, 27.042581374403568, 26.996709814770536, 27.197394026479483, 27.079468578280668, 26.96605644546148, 26.788537549407117, 26.67862497322767, 26.509121131884022, 26.401948988066515, 26.2540886890804, 26.21276637503081, 26.069612788773156, 25.99156400828535, 25.953468770190106, 25.915973457104247, 25.915973457104247, 25.763430335759907, 25.616277523620706, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 19, 36, 52, 68, 82, 93, 109, 126, 143, 155, 166, 184, 196, 216, 232, 245, 259, 277, 292, 304, 319, 333, 342, 356, 371, 384, 396, 409, 417, 426, 438, 450, 457, 466, 476, 487, 494, 503, 510, 520, 529, 536, 541, 550, 557, 566, 572, 582, 591, 601, 612, 622, 633, 641, 650, 658, 666, 676, 684, 695, 701, 710, 717, 724, 730, 738, 745, 752, 761, 768, 772, 778, 785, 793, 799, 804, 809, 811, 813, 813, 813, 813, 813, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 4, 6, 8, 10, 14, 14, 14, 15, 17, 19, 21, 23, 23, 25, 25, 25, 25, 26, 28, 28, 32, 34, 35, 38, 40, 42, 43, 45, 47, 48, 50, 51, 53, 55, 56, 56, 56, 56, 57, 61, 63, 65, 66, 66, 69, 71, 72, 75, 75, 76, 78, 78, 79, 79, 83, 85, 88, 89, 92, 92, 94, 97, 101, 105, 107, 107, 110, 113, 119, 120, 123, 126, 127, 128, 134, 135, 139, 143, 143, 144, 144, 145, 146, 149, 154, 156, 159, 161, 164, 166, 169, 170, 173, 175, 176, 177, 177, 179, 181, 185], "exact_score": [7.641657905799261, 7.756956858505074, 7.8696311021312635, 7.885242457733163, 7.992149176359703, 7.883115449377531, 7.966775398606806, 8.06949379219607, 8.188444486384828, 8.311665810289917, 8.322549742078113, 8.40811374159706, 10.039330904025947, 10.144407023109046, 11.550062947689325, 11.68080259574059, 11.783747907723509, 13.172860751930019, 13.231813100143846, 13.198152264579724, 13.310614117065727, 13.10551499605367, 13.074583677185657, 13.11583325850943, 13.074910678946974, 13.134593100494705, 14.170239898803441, 14.285518140939828, 14.425850159588036, 14.477782895121605, 14.51904443415479, 14.594214543993573, 14.699492981268602, 14.739885333035978, 14.798413280447964, 14.925205233110711, 15.104017024763413, 15.223203828075427, 15.379131036039851, 15.455001016958676, 15.639114453200076, 15.735634731957898, 15.73989696483682, 15.847800930307551, 16.63694635235973, 16.87229012167153, 17.078550700580458, 17.14626858648598, 17.278172211745222, 17.515886784085232, 18.448561716766466, 18.70750158533683, 20.222763300062713, 21.054221198605934, 21.30420547047136, 21.211190945349344, 21.392504556861216, 21.43597823744883, 21.770663783711992, 21.899889666330118, 22.34063410032402, 22.968172735789867, 23.036284658791057, 23.339993672561334, 24.11168537511104, 24.28942204228393, 24.671718560631362, 24.891057218643425, 24.221250771795635, 24.56615470509607, 24.9269663335048, 24.875399596826778, 24.98270210606048, 25.402937687597245, 25.886078765028415, 26.081856362028777, 26.452617024655716, 26.842888446425, 26.722903574626272, 27.21007535422304, 27.163354321263473, 27.163354321263473, 27.042581374403568, 26.996709814770536, 27.197394026479483, 27.079468578280668, 26.96605644546148, 26.788537549407117, 26.67862497322767, 26.509121131884022, 26.401948988066515, 26.2540886890804, 26.21276637503081, 26.069612788773156, 25.99156400828535, 25.953468770190106, 25.915973457104247, 25.915973457104247, 25.763430335759907, 25.616277523620706, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 18.870044280417915, "threshold_area100": 18.86052655528556, "threshold_area25": 10.317041481481787}, "test": {"score": [7.8873978710439285, 8.00272136596867, 8.821755717225695, 9.428894460481482, 9.92446988628918, 10.406745993743325, 10.412670499572716, 10.45162451792768, 11.298199998840765, 11.294687448013233, 11.30150747117959, 11.277215418014158, 11.320542725418768, 11.386784180550855, 11.293066719977054, 11.329521239724047, 11.763601914376657, 11.765868209401253, 11.756733986009984, 11.752140949314892, 12.135056341346338, 12.107187437391035, 12.098337184168347, 12.099413382497321, 12.492098155239551, 12.561151428000962, 12.943785052669151, 12.944891403128194, 13.41751170588174, 13.502461873243602, 13.64667107681381, 13.76336960906977, 14.049893117665436, 14.355192131965605, 14.553258957033718, 14.699720396276316, 14.991270847969934, 15.060591154120138, 15.293948934508029, 15.608864738180845, 15.877646411559343, 15.977107038156419, 16.39249785152562, 16.706327199271488, 16.747056130101136, 17.263840603477643, 17.35668735329335, 17.60764654956804, 17.658034002544674, 17.907447292768392, 18.406524140002137, 18.84858192473709, 19.1057824732678, 19.483919096040932, 19.50802626696334, 19.716860707254824, 20.098556707597485, 20.559035878097518, 20.71997168862405, 20.92235127706296, 21.114863700910213, 21.288788700770098, 21.667120149466648, 22.266318634942152, 22.324155087564872, 22.647617449877718, 23.06342371582023, 23.284238324174307, 23.66734494342842, 23.962931946895793, 24.528240087589197, 24.97597057684938, 25.12235939918256, 25.313074529204304, 25.98352537600166, 25.905217237698057, 26.304021503425567, 26.824555266406332, 26.95834144311357, 27.017781846870687, 26.956006534759357, 26.88757111564729, 26.826192287139918, 26.684349792344204, 26.5866682169169, 26.63560158080957, 26.636990886990887, 26.65145996193697, 26.651119043930443, 26.6085988194501, 26.695706096682915, 26.628825825610917, 26.674889564933572, 26.886328833842644, 26.872136528099833, 27.070050270982033, 27.030528846088075, 27.04573684382399, 27.005383994402347, 27.129733596657267, 27.001009067886244], "calls": [0.0, 1.0652463382157125, 2.097203728362184, 2.6964047936085223, 3.761651131824234, 4.560585885486019, 4.826897470039946, 5.59254327563249, 6.524633821571238, 7.023968042609853, 8.222370173102531, 9.021304926764314, 9.986684420772303, 10.85219707057257, 11.717709720372836, 12.250332889480692, 13.182423435419441, 13.715046604527299, 14.713715046604529, 15.545938748335553, 16.577896138482025, 17.210386151797604, 18.07589880159787, 18.541944074567244, 19.10785619174434, 20.206391478029296, 21.870838881491345, 22.603195739014648, 23.635153129161118, 24.4340878828229, 25.63249001331558, 27.762982689747005, 28.828229027962713, 30.29294274300932, 31.324900133155793, 32.689747003994675, 33.58854860186418, 34.45406125166445, 35.918774966711055, 37.25033288948069, 38.315579227696404, 39.88015978695073, 41.777629826897474, 43.20905459387483, 43.57523302263648, 44.940079893475364, 45.80559254327563, 46.37150466045273, 46.904127829560586, 48.568575233022635, 49.73368841544607, 51.664447403462056, 52.82956058588548, 53.728362183754996, 54.19440745672437, 55.35952063914781, 56.35818908122503, 57.98934753661784, 59.12117177097203, 60.38615179760319, 60.81890812250332, 62.4167776298269, 63.182423435419445, 64.54727030625833, 65.47936085219706, 66.17842876165113, 67.27696404793608, 68.44207723035952, 69.90679094540613, 70.80559254327564, 71.77097203728363, 72.76964047936086, 73.33555259653795, 73.90146471371504, 74.96671105193076, 75.56591211717709, 76.69773635153129, 77.762982689747, 78.56191744340879, 79.32756324900133, 80.05992010652463, 80.89214380825565, 82.25699067909454, 83.15579227696405, 83.95472703062583, 85.35286284953395, 86.6844207723036, 87.61651131824235, 88.64846870838882, 89.11451398135819, 91.01198402130493, 91.6444740346205, 92.14380825565912, 93.3422103861518, 94.00798934753661, 94.90679094540613, 95.63914780292943, 96.80426098535287, 97.6364846870839, 98.56857523302264, 100.0], "rescued": [0, 75, 143, 182, 251, 303, 321, 370, 427, 460, 537, 582, 638, 689, 742, 771, 824, 858, 908, 963, 1021, 1055, 1099, 1123, 1158, 1214, 1298, 1335, 1391, 1427, 1482, 1570, 1618, 1681, 1728, 1784, 1816, 1847, 1907, 1962, 2007, 2076, 2157, 2219, 2231, 2283, 2312, 2334, 2356, 2426, 2471, 2545, 2588, 2618, 2636, 2678, 2720, 2782, 2819, 2862, 2876, 2924, 2956, 3005, 3030, 3058, 3100, 3136, 3185, 3213, 3248, 3284, 3300, 3317, 3351, 3362, 3391, 3419, 3439, 3447, 3447, 3447, 3448, 3448, 3448, 3449, 3450, 3451, 3452, 3452, 3453, 3453, 3454, 3456, 3456, 3458, 3458, 3459, 3460, 3461, 3461], "harmed": [0, 4, 7, 13, 26, 32, 33, 35, 41, 43, 48, 51, 53, 54, 63, 64, 66, 69, 73, 79, 84, 89, 95, 99, 102, 106, 113, 120, 129, 130, 132, 143, 151, 162, 172, 178, 183, 187, 206, 211, 219, 234, 244, 256, 258, 264, 269, 272, 279, 293, 301, 314, 321, 325, 331, 337, 343, 355, 363, 376, 379, 391, 394, 406, 411, 415, 423, 433, 451, 458, 470, 481, 488, 494, 504, 508, 521, 534, 547, 554, 557, 561, 568, 577, 584, 587, 594, 596, 600, 602, 614, 618, 621, 629, 631, 637, 640, 646, 652, 660, 676], "exact_score": [7.8873978710439285, 7.995944205420513, 8.81984209989325, 9.394508042195774, 10.460834099727435, 10.426921449212273, 10.47644308633641, 11.28987342786929, 11.279583833098686, 11.272140634806632, 11.320542725418768, 11.379733144930327, 11.321658963447003, 11.745972166479056, 11.77801588246529, 11.768585255570894, 12.108411575562256, 12.101830738642851, 12.085596500988114, 12.474218841686415, 12.542325940757266, 12.8862582598027, 12.939947135367458, 13.13265110518285, 13.452282887737072, 13.582163035171268, 13.650998651560437, 13.698840474371664, 13.803952539408911, 14.072850417188334, 14.091622513134801, 14.596747036108788, 14.62297994709305, 14.739678022718614, 15.004619989746795, 15.185509601127247, 15.30054046202756, 15.558827284479133, 15.863339246269145, 15.90937127879119, 15.994367888088604, 16.063097942175023, 16.42860115992912, 16.686901297770525, 16.776246493363487, 17.25980311172683, 17.519118960509005, 17.671942106933265, 17.84119072496354, 18.143280998143673, 18.466555922724382, 18.6364159910622, 18.969634395999886, 19.28211036080267, 19.513413588721527, 19.680177281680827, 19.846587762370017, 20.214347901262478, 20.559035878097518, 20.698170590944, 20.847413825909634, 21.14409246427547, 21.18562654624878, 21.60617560272147, 21.97124829108319, 22.21527281481154, 22.61145519638689, 22.971485876835835, 23.131638176271757, 23.430200612321396, 23.687646364148897, 24.017159833755247, 24.613806693802466, 25.023242450317696, 25.341141178736144, 25.98352537600166, 26.04997230497589, 26.51945593037823, 26.835593860804046, 27.03401573623391, 26.989568494530463, 26.88757111564729, 26.87024181233183, 26.684349792344204, 26.5866682169169, 26.642983833873203, 26.59746653295734, 26.620940553143946, 26.6745583652569, 26.624664017475418, 26.535797954461305, 26.695706096682915, 26.691771087339962, 26.90062607345417, 26.872136528099833, 27.070050270982033, 27.007786205244184, 27.08307797479772, 26.99849124557412, 27.100573120758355, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 18.90483761933629, "threshold_area100": 18.820053002518204, "threshold_area25": 11.07942113910315}, "selected_budget": 79, "selected": {"val": {"score": 27.21007535422304, "calls": 78.88730385164051, "rescued": 813, "harmed": 143}, "test": {"score": 27.017781846870687, "calls": 79.32756324900133, "rescued": 3447, "harmed": 554}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 26.99586861016554, "calls": 78.88730385164051}, "test": {"score": 27.017380559026776, "calls": 79.36085219707057}}, "old_normalized_interior": {"val": {"score": 26.99586861016554, "calls": 78.88730385164051}, "test": {"score": 27.017380559026776, "calls": 79.36085219707057}}, "old_normalized_endpoints": {"val": {"score": 26.99586861016554, "calls": 78.88730385164051}, "test": {"score": 27.017380559026776, "calls": 79.36085219707057}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 7.762403481598755, 7.683906910729702, 7.70158608247332, 10.078324513432971, 12.104340582790801, 12.10667020154142, 11.909411053960298, 11.6647423800519, 11.654906170619553, 11.657657497240015, 11.67936110839663, 11.701481590418874, 11.644557185207224, 11.759908142142486, 11.809783526172946, 11.835373049248545, 11.920794230171946, 11.896549874681497, 11.9218202964333, 12.052160676013564, 12.055607556160652, 12.122274650207132, 12.129778201206772, 12.057334196059045, 12.219584795482783, 12.390925803218467, 12.514766929256316, 12.689682068502767, 13.750821644303205, 13.908978633974488, 13.807424722878523, 13.840927082951746, 14.905679761786969, 15.009613586898812, 15.018337137902355, 15.07133010358817, 15.181132543733627, 15.27111959899901, 16.15491927520091, 16.366226366226368, 17.195565853299634, 17.49297313012514, 18.335246842709527, 19.05521185641668, 19.295603169717673, 19.476886579767175, 20.306370278892665, 20.500377342745978, 20.58024394321638, 20.84018065117289, 21.003177328478532, 21.338931554526475, 20.48564141103596, 20.711725769290446, 20.943202288514957, 21.183498947024066, 21.139577080940445, 21.360342555994734, 21.51649860151845, 21.65586312688241, 21.72386791952009, 21.91943068373274, 22.205745587388098, 22.513085275881092, 22.82037528595726, 23.08164472343577, 23.17826079940617, 23.34691717313668, 23.71074879227053, 23.93731471960045, 24.791676355319176, 24.929436746107932, 25.9227328701807, 26.242058594999772, 26.515897526285137, 26.838117386375526, 26.88073426645401, 26.93115545451994, 26.99586861016554, 26.99586861016554, 26.99586861016554, 26.788696616282827, 26.957994579945805, 26.912949534900754, 26.912949534900754, 26.80122129758052, 26.757763649034377, 26.715068415375008, 26.606280193236714, 26.606280193236714, 26.540087328130806, 26.312525555404115, 26.272000945913987, 26.116529521686395, 26.223960017103863, 26.183315118038696, 26.040279230248565, 25.87593033575991, 25.627581125884507, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 7.762403481598755, 7.683906910729702, 7.70158608247332, 10.078324513432971, 12.104340582790801, 12.10667020154142, 11.909411053960298, 11.6647423800519, 11.654906170619553, 11.657657497240015, 11.67936110839663, 11.701481590418874, 11.644557185207224, 11.759908142142486, 11.809783526172946, 11.835373049248545, 11.920794230171946, 11.896549874681497, 11.9218202964333, 12.052160676013564, 12.055607556160652, 12.122274650207132, 12.129778201206772, 12.057334196059045, 12.219584795482783, 12.390925803218467, 12.514766929256316, 12.689682068502767, 13.750821644303205, 13.908978633974488, 13.807424722878523, 13.840927082951746, 14.905679761786969, 15.009613586898812, 15.018337137902355, 15.07133010358817, 15.181132543733627, 15.27111959899901, 16.15491927520091, 16.366226366226368, 17.195565853299634, 17.49297313012514, 18.335246842709527, 19.05521185641668, 19.295603169717673, 19.476886579767175, 20.306370278892665, 20.500377342745978, 20.58024394321638, 20.84018065117289, 21.003177328478532, 21.338931554526475, 20.48564141103596, 20.711725769290446, 20.943202288514957, 21.183498947024066, 21.139577080940445, 21.360342555994734, 21.51649860151845, 21.65586312688241, 21.72386791952009, 21.91943068373274, 22.205745587388098, 22.513085275881092, 22.82037528595726, 23.08164472343577, 23.17826079940617, 23.34691717313668, 23.71074879227053, 23.93731471960045, 24.791676355319176, 24.929436746107932, 25.9227328701807, 26.242058594999772, 26.515897526285137, 26.838117386375526, 26.88073426645401, 26.93115545451994, 26.99586861016554, 26.99586861016554, 26.99586861016554, 26.788696616282827, 26.957994579945805, 26.912949534900754, 26.912949534900754, 26.80122129758052, 26.757763649034377, 26.715068415375008, 26.606280193236714, 26.606280193236714, 26.540087328130806, 26.312525555404115, 26.272000945913987, 26.116529521686395, 26.223960017103863, 26.183315118038696, 26.040279230248565, 25.87593033575991, 25.627581125884507, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 11.51091272657167, "exact_area100": 19.15614391309327}, "test": {"score": [7.8873978710439285, 7.927718209201699, 7.983573591156783, 9.301360193477104, 9.375880915424661, 9.323488805563292, 9.32197337966506, 9.85085750226733, 10.563553420281012, 10.537460063471949, 10.55680627997983, 10.571758515553562, 10.598489188961443, 11.082335742777639, 11.054737323966012, 11.081081437778318, 11.369990626598616, 11.636774319990094, 11.71341018402931, 11.797065072790737, 11.951838258271147, 11.992037525824564, 12.03696203643624, 12.047258503491477, 12.384216085925882, 13.007137478127037, 13.120906900592857, 13.610761738906088, 13.538791331534924, 13.641305940621118, 14.573210864114913, 14.502880041065133, 15.152980194362996, 15.285954765885473, 15.52839307914658, 15.778526825669637, 15.814396493786203, 16.106297430829045, 16.306977492043774, 16.65988954335853, 17.001693444619352, 17.10997741130266, 17.35437727647069, 17.367621189859918, 17.83020669585267, 17.93969057327422, 18.16092907640963, 18.427178167918804, 18.74787052364009, 18.772394776431536, 19.241527369718234, 19.385058010772124, 19.538894910071473, 19.70279285888332, 20.04888017871606, 20.392632414330457, 20.50303680453538, 20.851414027489458, 20.923826766023282, 21.14247208226334, 21.76899184832778, 22.03883220075866, 22.263852909247035, 22.530184458687806, 22.660683771794883, 23.15721987774108, 23.238324193855796, 23.30943800992884, 23.993282621105312, 24.170534144082637, 24.704902567969373, 24.997328803978263, 25.055506540912038, 25.714939983768453, 25.90488261615275, 26.23621769356601, 26.899543949000925, 27.211555516419327, 27.03401573623391, 27.017380559026776, 26.95080636130434, 27.12598169322541, 27.125617354641925, 27.09269879515809, 27.116230961941202, 27.07988825229719, 27.02745529914476, 26.917230581887942, 27.034328108302297, 27.017257965533826, 26.96365433622453, 27.021647820294035, 26.96636926345598, 26.906674408871872, 26.869742463836065, 27.077121710465406, 27.223971721691793, 27.053120973988403, 27.057271455347003, 26.98215433535589, 27.001009067886244], "calls": [0.0, 0.5659121171770972, 1.1318242343541944, 1.9973368841544608, 2.62982689747004, 3.3288948069241013, 4.027962716378163, 4.793608521970706, 6.058588548601864, 6.92410119840213, 7.356857523302264, 8.422103861517977, 9.154460719041278, 10.252996005326231, 10.785619174434089, 11.218375499334222, 12.083888149134488, 13.1491344873502, 14.280958721704396, 15.479360852197072, 17.543275632490012, 18.10918774966711, 18.841544607190414, 19.67376830892144, 21.937416777629824, 23.93475366178429, 25.06657789613848, 25.499334221038616, 26.564580559254324, 27.796271637816243, 28.86151797603196, 29.89347536617843, 32.290279627163784, 33.82157123834887, 34.753661784287615, 35.15312916111851, 35.98535286284953, 36.684420772303596, 37.982689747003995, 39.647137150466044, 40.3462050599201, 41.24500665778961, 42.07723035952064, 42.97603195739015, 44.44074567243675, 45.23968042609853, 46.07190412782956, 47.5699067909454, 49.10119840213049, 50.23302263648469, 51.664447403462056, 52.4300932090546, 53.22902796271638, 55.29294274300932, 56.491344873501994, 57.19041278295606, 57.55659121171771, 58.18908122503329, 58.72170439414115, 59.78695073235686, 60.78561917443409, 62.35019973368841, 63.54860186418109, 64.31424766977364, 64.78029294274302, 66.41145139813582, 66.84420772303595, 67.24367509986683, 68.77496671105193, 69.27430093209055, 71.13848202396804, 72.30359520639148, 72.47003994673769, 74.56724367509987, 75.23302263648469, 75.86551264980027, 77.3635153129161, 78.22902796271637, 79.06125166444741, 79.36085219707057, 80.1930758988016, 81.52463382157123, 82.45672436750998, 82.98934753661784, 83.85486018641811, 84.8202396804261, 85.68575233022636, 86.48468708388815, 88.41544607190413, 88.84820239680427, 89.84687083888149, 91.21171770972038, 92.70972037283622, 93.47536617842877, 94.57390146471371, 95.17310252996005, 96.30492676431425, 97.33688415446072, 98.1358189081225, 98.90146471371504, 100.0], "exact_score": [7.8873978710439285, 7.969506856765212, 9.301360193477104, 9.405074320981484, 9.320111750346651, 9.83122334449992, 10.599047341898531, 10.52823207362706, 10.573190353965723, 10.60262005811542, 11.065222940801648, 11.058903149027415, 11.360614563742068, 11.635158601421912, 11.69194008226996, 11.749508804374798, 11.847021767972402, 11.924478023788565, 11.972103383973733, 12.013196407352288, 12.035427395523378, 12.033728864197272, 12.36573223356884, 12.721722117923997, 13.014207570229763, 13.114422615430174, 13.650155535464082, 13.541235649467618, 13.884229374418716, 14.517358342310455, 14.503265101748559, 15.005978604909942, 15.128274693538504, 15.225758019140004, 15.246753409603176, 15.568599799055214, 15.814396493786203, 16.09716115287333, 16.306977492043774, 16.582183007515816, 16.792521819017324, 17.075763777707973, 17.339421594483586, 17.367621189859918, 17.759255352960647, 17.9130608782586, 18.14925774120888, 18.240079960337425, 18.531640075048816, 18.712067004407128, 18.719846197337738, 18.95032536130579, 19.301133557827736, 19.48412882824357, 19.514648773896393, 19.65325672700039, 19.812721979760283, 20.350696159199067, 20.799105650817395, 21.016065178531893, 21.381808095424084, 21.79604871869221, 21.946207184280595, 22.064243378710817, 22.331218824378386, 22.70966035024976, 23.05978486776571, 23.290040175762826, 23.69597039494867, 24.06115530556247, 24.34776989073284, 24.678548371195138, 24.908806167550036, 25.224381991259914, 25.538103711678843, 25.839058946867016, 26.267890718987495, 26.487708940981626, 27.149261082553107, 27.03401573623391, 26.96741655331816, 26.94329451570166, 27.118557533868685, 27.09269879515809, 27.116230961941202, 27.063459133231316, 26.966196965401174, 27.161610975215766, 27.119300918928165, 27.017257965533826, 26.96365433622453, 26.9417258554597, 26.99834924120904, 26.915174878884557, 26.895961074898228, 27.110547047393947, 27.25561442005408, 27.093323131285064, 27.057271455347003, 26.98215433535589, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 10.672096491235616, "exact_area100": 19.085949402864898}}, "old_normalized_endpoint_budget": 79}, {"seed": 2029, "selection": {"best_epoch": 4, "epochs": 12, "alpha": 0.0, "validation_area100": 0.1953904042359823, "training_seconds": 4.162612269981764, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6875173449516296, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6727772951126099, "operator": "ge"}, {"budget": 0.03, "threshold": 0.662344217300415, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6430286169052124, "operator": "ge"}, {"budget": 0.05, "threshold": 0.6233881115913391, "operator": "ge"}, {"budget": 0.06, "threshold": 0.6085754632949829, "operator": "ge"}, {"budget": 0.07, "threshold": 0.6052143573760986, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5948078632354736, "operator": "ge"}, {"budget": 0.09, "threshold": 0.580604076385498, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5654982328414917, "operator": "ge"}, {"budget": 0.11, "threshold": 0.550902247428894, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5381088256835938, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5066416263580322, "operator": "ge"}, {"budget": 0.14, "threshold": 0.49514758586883545, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4762077331542969, "operator": "ge"}, {"budget": 0.16, "threshold": 0.46762245893478394, "operator": "ge"}, {"budget": 0.17, "threshold": 0.46237099170684814, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4532196521759033, "operator": "ge"}, {"budget": 0.19, "threshold": 0.448012113571167, "operator": "ge"}, {"budget": 0.2, "threshold": 0.4415961802005768, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4367254972457886, "operator": "ge"}, {"budget": 0.22, "threshold": 0.4314587712287903, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4260784387588501, "operator": "ge"}, {"budget": 0.24, "threshold": 0.42220985889434814, "operator": "ge"}, {"budget": 0.25, "threshold": 0.417667031288147, "operator": "ge"}, {"budget": 0.26, "threshold": 0.4123449921607971, "operator": "ge"}, {"budget": 0.27, "threshold": 0.40861108899116516, "operator": "ge"}, {"budget": 0.28, "threshold": 0.4044460654258728, "operator": "ge"}, {"budget": 0.29, "threshold": 0.40078988671302795, "operator": "ge"}, {"budget": 0.3, "threshold": 0.3936903178691864, "operator": "ge"}, {"budget": 0.31, "threshold": 0.389843612909317, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3865933418273926, "operator": "ge"}, {"budget": 0.33, "threshold": 0.38157182931900024, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3762601613998413, "operator": "ge"}, {"budget": 0.35, "threshold": 0.3729773461818695, "operator": "ge"}, {"budget": 0.36, "threshold": 0.36961132287979126, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3624095916748047, "operator": "ge"}, {"budget": 0.38, "threshold": 0.35778114199638367, "operator": "ge"}, {"budget": 0.39, "threshold": 0.3550163805484772, "operator": "ge"}, {"budget": 0.4, "threshold": 0.35012751817703247, "operator": "ge"}, {"budget": 0.41, "threshold": 0.3414279818534851, "operator": "ge"}, {"budget": 0.42, "threshold": 0.3339560925960541, "operator": "ge"}, {"budget": 0.43, "threshold": 0.3281591832637787, "operator": "ge"}, {"budget": 0.44, "threshold": 0.32204729318618774, "operator": "ge"}, {"budget": 0.45, "threshold": 0.30906546115875244, "operator": "ge"}, {"budget": 0.46, "threshold": 0.29985785484313965, "operator": "ge"}, {"budget": 0.47, "threshold": 0.2822531461715698, "operator": "ge"}, {"budget": 0.48, "threshold": 0.2716369926929474, "operator": "ge"}, {"budget": 0.49, "threshold": 0.25238704681396484, "operator": "ge"}, {"budget": 0.5, "threshold": 0.22593383491039276, "operator": "ge"}, {"budget": 0.51, "threshold": 0.21857517957687378, "operator": "ge"}, {"budget": 0.52, "threshold": 0.21286816895008087, "operator": "ge"}, {"budget": 0.53, "threshold": 0.2094028890132904, "operator": "ge"}, {"budget": 0.54, "threshold": 0.20660758018493652, "operator": "ge"}, {"budget": 0.55, "threshold": 0.2042023241519928, "operator": "ge"}, {"budget": 0.56, "threshold": 0.2021006941795349, "operator": "ge"}, {"budget": 0.57, "threshold": 0.19915568828582764, "operator": "ge"}, {"budget": 0.58, "threshold": 0.1960657387971878, "operator": "ge"}, {"budget": 0.59, "threshold": 0.19429096579551697, "operator": "ge"}, {"budget": 0.6, "threshold": 0.19091668725013733, "operator": "ge"}, {"budget": 0.61, "threshold": 0.1881229430437088, "operator": "ge"}, {"budget": 0.62, "threshold": 0.18420562148094177, "operator": "ge"}, {"budget": 0.63, "threshold": 0.18223929405212402, "operator": "ge"}, {"budget": 0.64, "threshold": 0.17795152962207794, "operator": "ge"}, {"budget": 0.65, "threshold": 0.17567382752895355, "operator": "ge"}, {"budget": 0.66, "threshold": 0.1707758903503418, "operator": "ge"}, {"budget": 0.67, "threshold": 0.1639167070388794, "operator": "ge"}, {"budget": 0.68, "threshold": 0.1578557938337326, "operator": "ge"}, {"budget": 0.69, "threshold": 0.15342578291893005, "operator": "ge"}, {"budget": 0.7, "threshold": 0.14815057814121246, "operator": "ge"}, {"budget": 0.71, "threshold": 0.14299333095550537, "operator": "ge"}, {"budget": 0.72, "threshold": 0.1388607621192932, "operator": "ge"}, {"budget": 0.73, "threshold": 0.13488787412643433, "operator": "ge"}, {"budget": 0.74, "threshold": 0.12929901480674744, "operator": "ge"}, {"budget": 0.75, "threshold": 0.12357384711503983, "operator": "ge"}, {"budget": 0.76, "threshold": 0.11838802695274353, "operator": "ge"}, {"budget": 0.77, "threshold": 0.09310916066169739, "operator": "ge"}, {"budget": 0.78, "threshold": -0.004140518140047789, "operator": "ge"}, {"budget": 0.79, "threshold": -0.006415552459657192, "operator": "ge"}, {"budget": 0.8, "threshold": -0.007582559250295162, "operator": "ge"}, {"budget": 0.81, "threshold": -0.00946833472698927, "operator": "ge"}, {"budget": 0.82, "threshold": -0.01023726537823677, "operator": "ge"}, {"budget": 0.83, "threshold": -0.011371977627277374, "operator": "ge"}, {"budget": 0.84, "threshold": -0.012292206287384033, "operator": "ge"}, {"budget": 0.85, "threshold": -0.014902378432452679, "operator": "ge"}, {"budget": 0.86, "threshold": -0.01621578074991703, "operator": "ge"}, {"budget": 0.87, "threshold": -0.01805972307920456, "operator": "ge"}, {"budget": 0.88, "threshold": -0.019136948511004448, "operator": "ge"}, {"budget": 0.89, "threshold": -0.02064557559788227, "operator": "ge"}, {"budget": 0.9, "threshold": -0.022731292992830276, "operator": "ge"}, {"budget": 0.91, "threshold": -0.024263180792331696, "operator": "ge"}, {"budget": 0.92, "threshold": -0.0258526299148798, "operator": "ge"}, {"budget": 0.93, "threshold": -0.028291072696447372, "operator": "ge"}, {"budget": 0.94, "threshold": -0.03007497452199459, "operator": "ge"}, {"budget": 0.95, "threshold": -0.032629966735839844, "operator": "ge"}, {"budget": 0.96, "threshold": -0.03486819565296173, "operator": "ge"}, {"budget": 0.97, "threshold": -0.041700251400470734, "operator": "ge"}, {"budget": 0.98, "threshold": -0.05179660767316818, "operator": "ge"}, {"budget": 0.99, "threshold": -0.0680239349603653, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.762403481598755, 10.507760313680564, 10.089263311129425, 10.063882396409083, 9.93980001401545, 9.770449004471077, 9.809076694496765, 9.588861017411912, 9.707077440247145, 9.84980288933188, 9.838339848628303, 9.894405688180202, 9.863099999494244, 12.07105642389979, 12.11749188525183, 12.23018838455614, 12.22495614568321, 12.339380768795152, 12.45726530434912, 12.468900572814963, 12.543629865412424, 13.548972745917592, 13.614781899284292, 13.614706932248161, 14.586572624766688, 14.764138085070028, 15.626569939883945, 15.73760771891719, 15.776243298251247, 15.898808982535348, 15.992148938502094, 16.068964284849624, 16.04685203943894, 16.06949617403873, 16.166932868689802, 16.323613291839425, 16.538237224947885, 16.807450083312155, 17.68556436659171, 17.83806739069897, 18.14840654127365, 18.306241912436604, 18.986995738847018, 19.304830167447438, 19.51579652649989, 20.212728130708165, 20.109410578160574, 20.075276051683392, 20.290259747267097, 20.501389519761716, 20.60055650488447, 20.74203297677966, 20.992785453022556, 21.23091363863569, 21.480780537624124, 21.63814033373858, 21.743068868974067, 21.86423416400533, 22.120162446249402, 22.34263605107595, 22.42059494547708, 22.70816497869638, 22.64969208007183, 23.522337235911785, 23.759742738897, 23.989003674040287, 24.216923796166434, 24.49120254464266, 24.661222457668103, 24.89101738480753, 25.277741068675386, 25.490784194581266, 25.578416226328727, 25.731206271561945, 26.11899294735116, 26.51337774967813, 26.792555315919802, 26.93115545451994, 26.93115545451994, 26.88788788788789, 26.99586861016554, 26.788696616282827, 26.635700900406782, 26.503449125400348, 26.35479272615195, 26.582040035987408, 26.474527887571362, 26.86433298502264, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.626984126984127, 26.441891613434994, 26.193113079324306, 25.96833062984728, 25.891525714332687, 25.738396728721387, 25.554334467763283, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 42, 63, 84, 103, 120, 138, 154, 171, 190, 206, 225, 239, 254, 268, 280, 293, 304, 316, 329, 342, 356, 368, 380, 393, 406, 419, 431, 440, 454, 464, 473, 487, 497, 509, 522, 534, 546, 558, 567, 579, 589, 600, 610, 619, 628, 637, 643, 650, 656, 661, 667, 674, 681, 688, 694, 700, 706, 711, 716, 721, 727, 732, 738, 745, 750, 756, 761, 766, 770, 776, 782, 787, 792, 799, 805, 811, 812, 812, 812, 813, 813, 813, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 2, 5, 10, 12, 16, 21, 23, 27, 27, 27, 29, 30, 32, 34, 35, 35, 38, 38, 38, 41, 42, 43, 44, 46, 46, 46, 47, 48, 49, 51, 52, 53, 57, 60, 62, 64, 65, 65, 65, 66, 66, 68, 70, 70, 72, 73, 75, 77, 78, 79, 80, 81, 81, 81, 81, 82, 84, 86, 89, 89, 90, 90, 94, 95, 98, 99, 101, 105, 107, 111, 115, 118, 124, 128, 130, 132, 138, 141, 141, 142, 143, 146, 149, 151, 154, 157, 159, 159, 160, 160, 160, 160, 164, 166, 171, 175, 177, 179, 182, 185], "exact_score": [7.641657905799261, 7.762403481598755, 10.507760313680564, 10.089263311129425, 10.063882396409083, 9.93980001401545, 9.770449004471077, 9.809076694496765, 9.588861017411912, 9.707077440247145, 9.84980288933188, 9.838339848628303, 9.894405688180202, 9.863099999494244, 12.07105642389979, 12.11749188525183, 12.23018838455614, 12.22495614568321, 12.339380768795152, 12.45726530434912, 12.468900572814963, 12.543629865412424, 13.548972745917592, 13.614781899284292, 13.614706932248161, 14.586572624766688, 14.764138085070028, 15.626569939883945, 15.73760771891719, 15.776243298251247, 15.898808982535348, 15.992148938502094, 16.068964284849624, 16.04685203943894, 16.06949617403873, 16.166932868689802, 16.323613291839425, 16.538237224947885, 16.807450083312155, 17.68556436659171, 17.83806739069897, 18.14840654127365, 18.306241912436604, 18.986995738847018, 19.304830167447438, 19.51579652649989, 20.212728130708165, 20.109410578160574, 20.075276051683392, 20.290259747267097, 20.501389519761716, 20.60055650488447, 20.74203297677966, 20.992785453022556, 21.23091363863569, 21.480780537624124, 21.63814033373858, 21.743068868974067, 21.86423416400533, 22.120162446249402, 22.34263605107595, 22.42059494547708, 22.70816497869638, 22.64969208007183, 23.522337235911785, 23.759742738897, 23.989003674040287, 24.216923796166434, 24.49120254464266, 24.661222457668103, 24.89101738480753, 25.277741068675386, 25.490784194581266, 25.578416226328727, 25.731206271561945, 26.11899294735116, 26.51337774967813, 26.792555315919802, 26.93115545451994, 26.93115545451994, 26.88788788788789, 26.99586861016554, 26.788696616282827, 26.635700900406782, 26.503449125400348, 26.35479272615195, 26.582040035987408, 26.474527887571362, 26.86433298502264, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.626984126984127, 26.441891613434994, 26.193113079324306, 25.96833062984728, 25.891525714332687, 25.738396728721387, 25.554334467763283, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.53904042359823, "threshold_area100": 19.530522652281736, "threshold_area25": 11.09802212533942}, "test": {"score": [7.8873978710439285, 10.573657199289089, 11.45228537098348, 11.764441571520877, 11.379872350578399, 11.607409232828164, 11.616734495192365, 11.57424271723863, 11.523071219409568, 11.526758193796487, 11.759264019216573, 12.294819531586858, 12.298790461037926, 12.230808067250159, 12.17478391027617, 12.162171200722591, 12.141476188431795, 12.119479164813242, 12.662959428482543, 12.715703216669521, 13.55626936919384, 13.698912983819156, 13.764078350145285, 13.717814278347298, 13.68538949155123, 13.690426801651501, 13.737313556128408, 13.770156869623465, 13.803446385596283, 13.874258672106043, 14.590869641879491, 14.859426346154612, 14.989927804650666, 15.284334808879644, 15.383403069920156, 15.691767000775855, 15.947846936877855, 16.12654133322839, 16.308684693105675, 16.361662583721966, 16.570930540357434, 16.934011299459755, 17.17454822864621, 17.39519715761628, 17.765606505060795, 18.12633485736551, 18.312438732433538, 18.582291181862626, 18.927351634449668, 19.184020546811805, 19.10002917898827, 18.977501637676482, 19.410400509919846, 19.685886786214734, 20.12333802903539, 20.23221727103078, 20.597682517167875, 20.847214122956682, 21.031274936347433, 21.1469628813515, 21.299617814742714, 21.449650288740703, 22.076001846316732, 22.20945971397235, 22.76843355730479, 22.892004188891963, 23.564185124452045, 24.08428865870408, 24.584752163901154, 24.794035202894896, 25.104107499270267, 25.78244333180676, 25.940518787816618, 26.15385530492788, 26.40030237918513, 26.564055065228757, 26.67669322290979, 26.96509088414254, 27.067687499687622, 26.98421988709958, 26.952699860948236, 26.8624728799547, 26.855048720597974, 26.823866535697043, 26.968191982487728, 26.95297071726646, 26.936636787596207, 26.86692262454396, 26.771129558462192, 26.68381122308976, 26.851773174632093, 27.16551730338022, 27.093466129205773, 27.145758430722406, 27.12435199384081, 27.31125012130041, 27.293749303904857, 27.241950605276934, 27.02847558895322, 26.894657519464683, 27.001009067886244], "calls": [0.0, 1.0319573901464714, 1.4980026631158456, 1.8641810918774968, 2.796271637816245, 3.96138482023968, 4.727030625832223, 4.960053262316911, 5.559254327563249, 6.1584553928095875, 7.057256990679095, 8.189081225033288, 8.721704394141145, 9.953395472703063, 10.719041278295606, 12.48335552596538, 13.282290279627162, 13.914780292942744, 15.013315579227696, 15.446071904127828, 16.81091877496671, 17.942743009320907, 19.340878828229027, 21.171770972037287, 22.503328894806923, 23.635153129161118, 24.93342210386152, 25.798934753661783, 26.73102529960053, 27.330226364846872, 29.19440745672437, 30.193075898801595, 31.158455392809586, 32.39014647137151, 33.88814913448735, 34.8202396804261, 35.78561917443409, 37.18375499334221, 38.18242343541944, 38.68175765645805, 39.84687083888149, 41.61118508655127, 42.709720372836216, 43.37549933422104, 44.44074567243675, 45.50599201065246, 46.40479360852197, 47.66977363515313, 48.50199733688416, 49.66711051930759, 51.89747003994674, 53.22902796271638, 54.593874833555255, 55.72569906790945, 56.69107856191744, 57.490013315579226, 58.28894806924101, 59.35419440745673, 60.45272969374168, 60.9520639147803, 61.98402130492676, 62.849533954727036, 64.24766977363515, 64.74700399467378, 65.81225033288948, 66.37816245006658, 67.64314247669773, 69.37416777629826, 71.13848202396804, 71.97070572569906, 73.10252996005326, 74.13448735019973, 74.73368841544608, 75.43275632490013, 76.09853528628496, 76.63115845539281, 76.89747003994674, 78.22902796271637, 78.99467376830891, 80.09320905459387, 80.9254327563249, 82.39014647137151, 82.85619174434088, 83.62183754993342, 84.35419440745673, 85.88548601864181, 86.81757656458056, 87.8162450066578, 88.38215712383489, 89.21438082556591, 90.81225033288948, 91.711051930759, 92.50998668442078, 93.37549933422103, 93.94141145139814, 94.57390146471371, 95.17310252996005, 96.5046604527297, 98.1358189081225, 99.30093209054593, 100.0], "rescued": [0, 95, 134, 164, 240, 338, 398, 417, 465, 507, 577, 656, 693, 769, 817, 917, 962, 998, 1055, 1080, 1146, 1208, 1276, 1373, 1445, 1503, 1565, 1614, 1662, 1694, 1795, 1847, 1893, 1954, 2023, 2075, 2125, 2191, 2242, 2268, 2321, 2398, 2450, 2483, 2528, 2573, 2600, 2640, 2670, 2703, 2756, 2790, 2825, 2854, 2882, 2903, 2925, 2957, 2987, 3001, 3027, 3050, 3085, 3098, 3128, 3142, 3173, 3219, 3262, 3281, 3306, 3336, 3351, 3367, 3384, 3396, 3403, 3438, 3447, 3447, 3447, 3447, 3447, 3447, 3450, 3450, 3450, 3450, 3451, 3451, 3452, 3454, 3454, 3456, 3456, 3457, 3458, 3459, 3460, 3460, 3461], "harmed": [0, 11, 19, 25, 39, 50, 53, 56, 61, 65, 75, 83, 85, 93, 99, 108, 112, 117, 122, 122, 137, 149, 164, 179, 193, 199, 208, 216, 223, 225, 238, 243, 246, 253, 263, 269, 276, 286, 292, 298, 303, 316, 321, 327, 334, 339, 346, 347, 348, 352, 361, 373, 378, 384, 387, 392, 395, 399, 408, 410, 422, 433, 443, 444, 448, 450, 453, 461, 471, 477, 487, 495, 503, 511, 519, 523, 525, 545, 551, 556, 559, 565, 566, 569, 574, 576, 577, 584, 593, 597, 605, 608, 614, 616, 619, 622, 627, 633, 651, 664, 676], "exact_score": [7.8873978710439285, 10.602774921912797, 11.66103333161233, 11.34058612746552, 11.611948305190948, 11.578943479914122, 11.527252207290664, 11.752072233769951, 12.296161148595857, 12.25525995608922, 12.21056440818788, 12.178750420576268, 12.135104264052881, 12.142866607520732, 12.128867777539936, 12.659780032308444, 13.37977392274593, 13.554808821959307, 13.701165192592477, 13.843160803060762, 13.722285745762195, 13.69126199303925, 13.676989732472839, 13.743985256481789, 13.687988757987322, 13.734393835567813, 13.767543758872144, 13.852792220391407, 14.19604895973896, 14.402232774030526, 14.845602109227961, 14.982703160386187, 15.258884247681593, 15.332472019748703, 15.392684685760742, 15.702937293691562, 15.98858046525169, 16.11449752087004, 16.255386952806287, 16.431953385642757, 16.60675621606354, 16.726535785189185, 17.03858895222293, 17.24813856580085, 17.688117413446065, 17.863586013221838, 18.307695818737376, 18.437004238328488, 18.65687962162302, 19.053541977251804, 19.278360169453315, 18.940929124272362, 18.85874050520021, 18.961755067893176, 19.10133990051684, 19.45086411568949, 19.752772046745424, 20.167777702736696, 20.4147352827216, 20.76686638201753, 20.99234173327476, 21.156922386503314, 21.299617814742714, 21.481989449090012, 22.008557291871806, 22.44920402178467, 22.781954403190046, 23.212446037949398, 23.65949858324548, 23.973632343497076, 24.199643634467908, 24.54038707048618, 24.794035202894896, 25.05832289574887, 25.675434301383536, 26.038816187312268, 26.341600849107838, 26.703055448507047, 26.84611060196523, 27.067687499687622, 27.00121975110067, 26.952699860948236, 26.88634346698338, 26.855048720597974, 26.84091941897083, 26.968191982487728, 26.95297071726646, 26.929069978205984, 26.86692262454396, 26.68381122308976, 26.859002787124865, 26.844570639339672, 27.15822138075869, 27.124068458873218, 27.12435199384081, 27.316729076922066, 27.32424798861382, 27.243387716987215, 27.035393087328412, 26.957914681831397, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.38585973537182, "threshold_area100": 19.491768040242206, "threshold_area25": 11.974805231661776}, "selected_budget": 81, "selected": {"val": {"score": 26.99586861016554, "calls": 80.88445078459344, "rescued": 813, "harmed": 143}, "test": {"score": 26.8624728799547, "calls": 82.39014647137151, "rescued": 3447, "harmed": 565}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.6875173449516296, "operator": "ge"}, {"budget": 0.02, "threshold": 0.6727772951126099, "operator": "ge"}, {"budget": 0.03, "threshold": 0.662344217300415, "operator": "ge"}, {"budget": 0.04, "threshold": 0.6430286169052124, "operator": "ge"}, {"budget": 0.05, "threshold": 0.6233881115913391, "operator": "ge"}, {"budget": 0.06, "threshold": 0.6085754632949829, "operator": "ge"}, {"budget": 0.07, "threshold": 0.6052143573760986, "operator": "ge"}, {"budget": 0.08, "threshold": 0.5948078632354736, "operator": "ge"}, {"budget": 0.09, "threshold": 0.580604076385498, "operator": "ge"}, {"budget": 0.1, "threshold": 0.5654982328414917, "operator": "ge"}, {"budget": 0.11, "threshold": 0.550902247428894, "operator": "ge"}, {"budget": 0.12, "threshold": 0.5381088256835938, "operator": "ge"}, {"budget": 0.13, "threshold": 0.5066416263580322, "operator": "ge"}, {"budget": 0.14, "threshold": 0.49514758586883545, "operator": "ge"}, {"budget": 0.15, "threshold": 0.4762077331542969, "operator": "ge"}, {"budget": 0.16, "threshold": 0.46762245893478394, "operator": "ge"}, {"budget": 0.17, "threshold": 0.46237099170684814, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4532196521759033, "operator": "ge"}, {"budget": 0.19, "threshold": 0.448012113571167, "operator": "ge"}, {"budget": 0.2, "threshold": 0.4415961802005768, "operator": "ge"}, {"budget": 0.21, "threshold": 0.4367254972457886, "operator": "ge"}, {"budget": 0.22, "threshold": 0.4314587712287903, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4260784387588501, "operator": "ge"}, {"budget": 0.24, "threshold": 0.42220985889434814, "operator": "ge"}, {"budget": 0.25, "threshold": 0.417667031288147, "operator": "ge"}, {"budget": 0.26, "threshold": 0.4123449921607971, "operator": "ge"}, {"budget": 0.27, "threshold": 0.40861108899116516, "operator": "ge"}, {"budget": 0.28, "threshold": 0.4044460654258728, "operator": "ge"}, {"budget": 0.29, "threshold": 0.40078988671302795, "operator": "ge"}, {"budget": 0.3, "threshold": 0.3936903178691864, "operator": "ge"}, {"budget": 0.31, "threshold": 0.389843612909317, "operator": "ge"}, {"budget": 0.32, "threshold": 0.3865933418273926, "operator": "ge"}, {"budget": 0.33, "threshold": 0.38157182931900024, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3762601613998413, "operator": "ge"}, {"budget": 0.35, "threshold": 0.3729773461818695, "operator": "ge"}, {"budget": 0.36, "threshold": 0.36961132287979126, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3624095916748047, "operator": "ge"}, {"budget": 0.38, "threshold": 0.35778114199638367, "operator": "ge"}, {"budget": 0.39, "threshold": 0.3550163805484772, "operator": "ge"}, {"budget": 0.4, "threshold": 0.35012751817703247, "operator": "ge"}, {"budget": 0.41, "threshold": 0.3414279818534851, "operator": "ge"}, {"budget": 0.42, "threshold": 0.3339560925960541, "operator": "ge"}, {"budget": 0.43, "threshold": 0.3281591832637787, "operator": "ge"}, {"budget": 0.44, "threshold": 0.32204729318618774, "operator": "ge"}, {"budget": 0.45, "threshold": 0.30906546115875244, "operator": "ge"}, {"budget": 0.46, "threshold": 0.29985785484313965, "operator": "ge"}, {"budget": 0.47, "threshold": 0.2822531461715698, "operator": "ge"}, {"budget": 0.48, "threshold": 0.2716369926929474, "operator": "ge"}, {"budget": 0.49, "threshold": 0.25238704681396484, "operator": "ge"}, {"budget": 0.5, "threshold": 0.22593383491039276, "operator": "ge"}, {"budget": 0.51, "threshold": 0.21857517957687378, "operator": "ge"}, {"budget": 0.52, "threshold": 0.21286816895008087, "operator": "ge"}, {"budget": 0.53, "threshold": 0.2094028890132904, "operator": "ge"}, {"budget": 0.54, "threshold": 0.20660758018493652, "operator": "ge"}, {"budget": 0.55, "threshold": 0.2042023241519928, "operator": "ge"}, {"budget": 0.56, "threshold": 0.2021006941795349, "operator": "ge"}, {"budget": 0.57, "threshold": 0.19915568828582764, "operator": "ge"}, {"budget": 0.58, "threshold": 0.1960657387971878, "operator": "ge"}, {"budget": 0.59, "threshold": 0.19429096579551697, "operator": "ge"}, {"budget": 0.6, "threshold": 0.19091668725013733, "operator": "ge"}, {"budget": 0.61, "threshold": 0.1881229430437088, "operator": "ge"}, {"budget": 0.62, "threshold": 0.18420562148094177, "operator": "ge"}, {"budget": 0.63, "threshold": 0.18223929405212402, "operator": "ge"}, {"budget": 0.64, "threshold": 0.17795152962207794, "operator": "ge"}, {"budget": 0.65, "threshold": 0.17567382752895355, "operator": "ge"}, {"budget": 0.66, "threshold": 0.1707758903503418, "operator": "ge"}, {"budget": 0.67, "threshold": 0.1639167070388794, "operator": "ge"}, {"budget": 0.68, "threshold": 0.1578557938337326, "operator": "ge"}, {"budget": 0.69, "threshold": 0.15342578291893005, "operator": "ge"}, {"budget": 0.7, "threshold": 0.14815057814121246, "operator": "ge"}, {"budget": 0.71, "threshold": 0.14299333095550537, "operator": "ge"}, {"budget": 0.72, "threshold": 0.1388607621192932, "operator": "ge"}, {"budget": 0.73, "threshold": 0.13488787412643433, "operator": "ge"}, {"budget": 0.74, "threshold": 0.12929901480674744, "operator": "ge"}, {"budget": 0.75, "threshold": 0.12357384711503983, "operator": "ge"}, {"budget": 0.76, "threshold": 0.11838802695274353, "operator": "ge"}, {"budget": 0.77, "threshold": 0.09310916066169739, "operator": "ge"}, {"budget": 0.78, "threshold": -0.004140518140047789, "operator": "ge"}, {"budget": 0.79, "threshold": -0.006415552459657192, "operator": "ge"}, {"budget": 0.8, "threshold": -0.007582559250295162, "operator": "ge"}, {"budget": 0.81, "threshold": -0.00946833472698927, "operator": "ge"}, {"budget": 0.82, "threshold": -0.01023726537823677, "operator": "ge"}, {"budget": 0.83, "threshold": -0.011371977627277374, "operator": "ge"}, {"budget": 0.84, "threshold": -0.012292206287384033, "operator": "ge"}, {"budget": 0.85, "threshold": -0.014902378432452679, "operator": "ge"}, {"budget": 0.86, "threshold": -0.01621578074991703, "operator": "ge"}, {"budget": 0.87, "threshold": -0.01805972307920456, "operator": "ge"}, {"budget": 0.88, "threshold": -0.019136948511004448, "operator": "ge"}, {"budget": 0.89, "threshold": -0.02064557559788227, "operator": "ge"}, {"budget": 0.9, "threshold": -0.022731292992830276, "operator": "ge"}, {"budget": 0.91, "threshold": -0.024263180792331696, "operator": "ge"}, {"budget": 0.92, "threshold": -0.0258526299148798, "operator": "ge"}, {"budget": 0.93, "threshold": -0.028291072696447372, "operator": "ge"}, {"budget": 0.94, "threshold": -0.03007497452199459, "operator": "ge"}, {"budget": 0.95, "threshold": -0.032629966735839844, "operator": "ge"}, {"budget": 0.96, "threshold": -0.03486819565296173, "operator": "ge"}, {"budget": 0.97, "threshold": -0.041700251400470734, "operator": "ge"}, {"budget": 0.98, "threshold": -0.05179660767316818, "operator": "ge"}, {"budget": 0.99, "threshold": -0.0680239349603653, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.762403481598755, 10.507760313680564, 10.089263311129425, 10.063882396409083, 9.93980001401545, 9.770449004471077, 9.809076694496765, 9.588861017411912, 9.707077440247145, 9.84980288933188, 9.838339848628303, 9.894405688180202, 9.863099999494244, 12.07105642389979, 12.11749188525183, 12.23018838455614, 12.22495614568321, 12.339380768795152, 12.45726530434912, 12.468900572814963, 12.543629865412424, 13.548972745917592, 13.614781899284292, 13.614706932248161, 14.586572624766688, 14.764138085070028, 15.626569939883945, 15.73760771891719, 15.776243298251247, 15.898808982535348, 15.992148938502094, 16.068964284849624, 16.04685203943894, 16.06949617403873, 16.166932868689802, 16.323613291839425, 16.538237224947885, 16.807450083312155, 17.68556436659171, 17.83806739069897, 18.14840654127365, 18.306241912436604, 18.986995738847018, 19.304830167447438, 19.51579652649989, 20.212728130708165, 20.109410578160574, 20.075276051683392, 20.290259747267097, 20.501389519761716, 20.60055650488447, 20.74203297677966, 20.992785453022556, 21.23091363863569, 21.480780537624124, 21.63814033373858, 21.743068868974067, 21.86423416400533, 22.120162446249402, 22.34263605107595, 22.42059494547708, 22.70816497869638, 22.64969208007183, 23.522337235911785, 23.759742738897, 23.989003674040287, 24.216923796166434, 24.49120254464266, 24.661222457668103, 24.89101738480753, 25.277741068675386, 25.490784194581266, 25.578416226328727, 25.731206271561945, 26.11899294735116, 26.51337774967813, 26.792555315919802, 26.93115545451994, 26.93115545451994, 26.88788788788789, 26.99586861016554, 26.788696616282827, 26.635700900406782, 26.503449125400348, 26.35479272615195, 26.582040035987408, 26.474527887571362, 26.86433298502264, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.626984126984127, 26.441891613434994, 26.193113079324306, 25.96833062984728, 25.891525714332687, 25.738396728721387, 25.554334467763283, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 21, 42, 63, 84, 103, 120, 138, 154, 171, 190, 206, 225, 239, 254, 268, 280, 293, 304, 316, 329, 342, 356, 368, 380, 393, 406, 419, 431, 440, 454, 464, 473, 487, 497, 509, 522, 534, 546, 558, 567, 579, 589, 600, 610, 619, 628, 637, 643, 650, 656, 661, 667, 674, 681, 688, 694, 700, 706, 711, 716, 721, 727, 732, 738, 745, 750, 756, 761, 766, 770, 776, 782, 787, 792, 799, 805, 811, 812, 812, 812, 813, 813, 813, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 2, 5, 10, 12, 16, 21, 23, 27, 27, 27, 29, 30, 32, 34, 35, 35, 38, 38, 38, 41, 42, 43, 44, 46, 46, 46, 47, 48, 49, 51, 52, 53, 57, 60, 62, 64, 65, 65, 65, 66, 66, 68, 70, 70, 72, 73, 75, 77, 78, 79, 80, 81, 81, 81, 81, 82, 84, 86, 89, 89, 90, 90, 94, 95, 98, 99, 101, 105, 107, 111, 115, 118, 124, 128, 130, 132, 138, 141, 141, 142, 143, 146, 149, 151, 154, 157, 159, 159, 160, 160, 160, 160, 164, 166, 171, 175, 177, 179, 182, 185], "exact_score": [7.641657905799261, 7.762403481598755, 10.507760313680564, 10.089263311129425, 10.063882396409083, 9.93980001401545, 9.770449004471077, 9.809076694496765, 9.588861017411912, 9.707077440247145, 9.84980288933188, 9.838339848628303, 9.894405688180202, 9.863099999494244, 12.07105642389979, 12.11749188525183, 12.23018838455614, 12.22495614568321, 12.339380768795152, 12.45726530434912, 12.468900572814963, 12.543629865412424, 13.548972745917592, 13.614781899284292, 13.614706932248161, 14.586572624766688, 14.764138085070028, 15.626569939883945, 15.73760771891719, 15.776243298251247, 15.898808982535348, 15.992148938502094, 16.068964284849624, 16.04685203943894, 16.06949617403873, 16.166932868689802, 16.323613291839425, 16.538237224947885, 16.807450083312155, 17.68556436659171, 17.83806739069897, 18.14840654127365, 18.306241912436604, 18.986995738847018, 19.304830167447438, 19.51579652649989, 20.212728130708165, 20.109410578160574, 20.075276051683392, 20.290259747267097, 20.501389519761716, 20.60055650488447, 20.74203297677966, 20.992785453022556, 21.23091363863569, 21.480780537624124, 21.63814033373858, 21.743068868974067, 21.86423416400533, 22.120162446249402, 22.34263605107595, 22.42059494547708, 22.70816497869638, 22.64969208007183, 23.522337235911785, 23.759742738897, 23.989003674040287, 24.216923796166434, 24.49120254464266, 24.661222457668103, 24.89101738480753, 25.277741068675386, 25.490784194581266, 25.578416226328727, 25.731206271561945, 26.11899294735116, 26.51337774967813, 26.792555315919802, 26.93115545451994, 26.93115545451994, 26.88788788788789, 26.99586861016554, 26.788696616282827, 26.635700900406782, 26.503449125400348, 26.35479272615195, 26.582040035987408, 26.474527887571362, 26.86433298502264, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.626984126984127, 26.441891613434994, 26.193113079324306, 25.96833062984728, 25.891525714332687, 25.738396728721387, 25.554334467763283, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.53904042359823, "threshold_area100": 19.530522652281736, "threshold_area25": 11.09802212533942}, "test": {"score": [7.8873978710439285, 10.573657199289089, 11.45228537098348, 11.764441571520877, 11.379872350578399, 11.607409232828164, 11.616734495192365, 11.57424271723863, 11.523071219409568, 11.526758193796487, 11.759264019216573, 12.294819531586858, 12.298790461037926, 12.230808067250159, 12.17478391027617, 12.162171200722591, 12.141476188431795, 12.119479164813242, 12.662959428482543, 12.715703216669521, 13.55626936919384, 13.698912983819156, 13.764078350145285, 13.717814278347298, 13.68538949155123, 13.690426801651501, 13.737313556128408, 13.770156869623465, 13.803446385596283, 13.874258672106043, 14.590869641879491, 14.859426346154612, 14.989927804650666, 15.284334808879644, 15.383403069920156, 15.691767000775855, 15.947846936877855, 16.12654133322839, 16.308684693105675, 16.361662583721966, 16.570930540357434, 16.934011299459755, 17.17454822864621, 17.39519715761628, 17.765606505060795, 18.12633485736551, 18.312438732433538, 18.582291181862626, 18.927351634449668, 19.184020546811805, 19.10002917898827, 18.977501637676482, 19.410400509919846, 19.685886786214734, 20.12333802903539, 20.23221727103078, 20.597682517167875, 20.847214122956682, 21.031274936347433, 21.1469628813515, 21.299617814742714, 21.449650288740703, 22.076001846316732, 22.20945971397235, 22.76843355730479, 22.892004188891963, 23.564185124452045, 24.08428865870408, 24.584752163901154, 24.794035202894896, 25.104107499270267, 25.78244333180676, 25.940518787816618, 26.15385530492788, 26.40030237918513, 26.564055065228757, 26.67669322290979, 26.96509088414254, 27.067687499687622, 26.98421988709958, 26.952699860948236, 26.8624728799547, 26.855048720597974, 26.823866535697043, 26.968191982487728, 26.95297071726646, 26.936636787596207, 26.86692262454396, 26.771129558462192, 26.68381122308976, 26.851773174632093, 27.16551730338022, 27.093466129205773, 27.145758430722406, 27.12435199384081, 27.31125012130041, 27.293749303904857, 27.241950605276934, 27.02847558895322, 26.894657519464683, 27.001009067886244], "calls": [0.0, 1.0319573901464714, 1.4980026631158456, 1.8641810918774968, 2.796271637816245, 3.96138482023968, 4.727030625832223, 4.960053262316911, 5.559254327563249, 6.1584553928095875, 7.057256990679095, 8.189081225033288, 8.721704394141145, 9.953395472703063, 10.719041278295606, 12.48335552596538, 13.282290279627162, 13.914780292942744, 15.013315579227696, 15.446071904127828, 16.81091877496671, 17.942743009320907, 19.340878828229027, 21.171770972037287, 22.503328894806923, 23.635153129161118, 24.93342210386152, 25.798934753661783, 26.73102529960053, 27.330226364846872, 29.19440745672437, 30.193075898801595, 31.158455392809586, 32.39014647137151, 33.88814913448735, 34.8202396804261, 35.78561917443409, 37.18375499334221, 38.18242343541944, 38.68175765645805, 39.84687083888149, 41.61118508655127, 42.709720372836216, 43.37549933422104, 44.44074567243675, 45.50599201065246, 46.40479360852197, 47.66977363515313, 48.50199733688416, 49.66711051930759, 51.89747003994674, 53.22902796271638, 54.593874833555255, 55.72569906790945, 56.69107856191744, 57.490013315579226, 58.28894806924101, 59.35419440745673, 60.45272969374168, 60.9520639147803, 61.98402130492676, 62.849533954727036, 64.24766977363515, 64.74700399467378, 65.81225033288948, 66.37816245006658, 67.64314247669773, 69.37416777629826, 71.13848202396804, 71.97070572569906, 73.10252996005326, 74.13448735019973, 74.73368841544608, 75.43275632490013, 76.09853528628496, 76.63115845539281, 76.89747003994674, 78.22902796271637, 78.99467376830891, 80.09320905459387, 80.9254327563249, 82.39014647137151, 82.85619174434088, 83.62183754993342, 84.35419440745673, 85.88548601864181, 86.81757656458056, 87.8162450066578, 88.38215712383489, 89.21438082556591, 90.81225033288948, 91.711051930759, 92.50998668442078, 93.37549933422103, 93.94141145139814, 94.57390146471371, 95.17310252996005, 96.5046604527297, 98.1358189081225, 99.30093209054593, 100.0], "rescued": [0, 95, 134, 164, 240, 338, 398, 417, 465, 507, 577, 656, 693, 769, 817, 917, 962, 998, 1055, 1080, 1146, 1208, 1276, 1373, 1445, 1503, 1565, 1614, 1662, 1694, 1795, 1847, 1893, 1954, 2023, 2075, 2125, 2191, 2242, 2268, 2321, 2398, 2450, 2483, 2528, 2573, 2600, 2640, 2670, 2703, 2756, 2790, 2825, 2854, 2882, 2903, 2925, 2957, 2987, 3001, 3027, 3050, 3085, 3098, 3128, 3142, 3173, 3219, 3262, 3281, 3306, 3336, 3351, 3367, 3384, 3396, 3403, 3438, 3447, 3447, 3447, 3447, 3447, 3447, 3450, 3450, 3450, 3450, 3451, 3451, 3452, 3454, 3454, 3456, 3456, 3457, 3458, 3459, 3460, 3460, 3461], "harmed": [0, 11, 19, 25, 39, 50, 53, 56, 61, 65, 75, 83, 85, 93, 99, 108, 112, 117, 122, 122, 137, 149, 164, 179, 193, 199, 208, 216, 223, 225, 238, 243, 246, 253, 263, 269, 276, 286, 292, 298, 303, 316, 321, 327, 334, 339, 346, 347, 348, 352, 361, 373, 378, 384, 387, 392, 395, 399, 408, 410, 422, 433, 443, 444, 448, 450, 453, 461, 471, 477, 487, 495, 503, 511, 519, 523, 525, 545, 551, 556, 559, 565, 566, 569, 574, 576, 577, 584, 593, 597, 605, 608, 614, 616, 619, 622, 627, 633, 651, 664, 676], "exact_score": [7.8873978710439285, 10.602774921912797, 11.66103333161233, 11.34058612746552, 11.611948305190948, 11.578943479914122, 11.527252207290664, 11.752072233769951, 12.296161148595857, 12.25525995608922, 12.21056440818788, 12.178750420576268, 12.135104264052881, 12.142866607520732, 12.128867777539936, 12.659780032308444, 13.37977392274593, 13.554808821959307, 13.701165192592477, 13.843160803060762, 13.722285745762195, 13.69126199303925, 13.676989732472839, 13.743985256481789, 13.687988757987322, 13.734393835567813, 13.767543758872144, 13.852792220391407, 14.19604895973896, 14.402232774030526, 14.845602109227961, 14.982703160386187, 15.258884247681593, 15.332472019748703, 15.392684685760742, 15.702937293691562, 15.98858046525169, 16.11449752087004, 16.255386952806287, 16.431953385642757, 16.60675621606354, 16.726535785189185, 17.03858895222293, 17.24813856580085, 17.688117413446065, 17.863586013221838, 18.307695818737376, 18.437004238328488, 18.65687962162302, 19.053541977251804, 19.278360169453315, 18.940929124272362, 18.85874050520021, 18.961755067893176, 19.10133990051684, 19.45086411568949, 19.752772046745424, 20.167777702736696, 20.4147352827216, 20.76686638201753, 20.99234173327476, 21.156922386503314, 21.299617814742714, 21.481989449090012, 22.008557291871806, 22.44920402178467, 22.781954403190046, 23.212446037949398, 23.65949858324548, 23.973632343497076, 24.199643634467908, 24.54038707048618, 24.794035202894896, 25.05832289574887, 25.675434301383536, 26.038816187312268, 26.341600849107838, 26.703055448507047, 26.84611060196523, 27.067687499687622, 27.00121975110067, 26.952699860948236, 26.88634346698338, 26.855048720597974, 26.84091941897083, 26.968191982487728, 26.95297071726646, 26.929069978205984, 26.86692262454396, 26.68381122308976, 26.859002787124865, 26.844570639339672, 27.15822138075869, 27.124068458873218, 27.12435199384081, 27.316729076922066, 27.32424798861382, 27.243387716987215, 27.035393087328412, 26.957914681831397, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.38585973537182, "threshold_area100": 19.491768040242206, "threshold_area25": 11.974805231661776}, "selected_budget": 81, "selected": {"val": {"score": 26.99586861016554, "calls": 80.88445078459344, "rescued": 813, "harmed": 143}, "test": {"score": 26.8624728799547, "calls": 82.39014647137151, "rescued": 3447, "harmed": 565}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 3.560123920440674, "operator": "ge"}, {"budget": 0.02, "threshold": 3.432774305343628, "operator": "ge"}, {"budget": 0.03, "threshold": 3.3760743141174316, "operator": "ge"}, {"budget": 0.04, "threshold": 3.3017125129699707, "operator": "ge"}, {"budget": 0.05, "threshold": 3.1999053955078125, "operator": "ge"}, {"budget": 0.06, "threshold": 3.1682188510894775, "operator": "ge"}, {"budget": 0.07, "threshold": 3.140463352203369, "operator": "ge"}, {"budget": 0.08, "threshold": 3.0886011123657227, "operator": "ge"}, {"budget": 0.09, "threshold": 3.075334310531616, "operator": "ge"}, {"budget": 0.1, "threshold": 3.057097911834717, "operator": "ge"}, {"budget": 0.11, "threshold": 3.0244486331939697, "operator": "ge"}, {"budget": 0.12, "threshold": 2.9825539588928223, "operator": "ge"}, {"budget": 0.13, "threshold": 2.958198070526123, "operator": "ge"}, {"budget": 0.14, "threshold": 2.9067909717559814, "operator": "ge"}, {"budget": 0.15, "threshold": 2.87551212310791, "operator": "ge"}, {"budget": 0.16, "threshold": 2.843784809112549, "operator": "ge"}, {"budget": 0.17, "threshold": 2.8361549377441406, "operator": "ge"}, {"budget": 0.18, "threshold": 2.8149614334106445, "operator": "ge"}, {"budget": 0.19, "threshold": 2.7874417304992676, "operator": "ge"}, {"budget": 0.2, "threshold": 2.773059844970703, "operator": "ge"}, {"budget": 0.21, "threshold": 2.7412071228027344, "operator": "ge"}, {"budget": 0.22, "threshold": 2.716386318206787, "operator": "ge"}, {"budget": 0.23, "threshold": 2.7048449516296387, "operator": "ge"}, {"budget": 0.24, "threshold": 2.6874427795410156, "operator": "ge"}, {"budget": 0.25, "threshold": 2.6708428859710693, "operator": "ge"}, {"budget": 0.26, "threshold": 2.6384878158569336, "operator": "ge"}, {"budget": 0.27, "threshold": 2.6019349098205566, "operator": "ge"}, {"budget": 0.28, "threshold": 2.5767691135406494, "operator": "ge"}, {"budget": 0.29, "threshold": 2.564460039138794, "operator": "ge"}, {"budget": 0.3, "threshold": 2.5455617904663086, "operator": "ge"}, {"budget": 0.31, "threshold": 2.52414870262146, "operator": "ge"}, {"budget": 0.32, "threshold": 2.50584077835083, "operator": "ge"}, {"budget": 0.33, "threshold": 2.487102508544922, "operator": "ge"}, {"budget": 0.34, "threshold": 2.4459331035614014, "operator": "ge"}, {"budget": 0.35, "threshold": 2.421247959136963, "operator": "ge"}, {"budget": 0.36, "threshold": 2.405344009399414, "operator": "ge"}, {"budget": 0.37, "threshold": 2.3837478160858154, "operator": "ge"}, {"budget": 0.38, "threshold": 2.351618766784668, "operator": "ge"}, {"budget": 0.39, "threshold": 2.338353395462036, "operator": "ge"}, {"budget": 0.4, "threshold": 2.31821870803833, "operator": "ge"}, {"budget": 0.41, "threshold": 2.308098793029785, "operator": "ge"}, {"budget": 0.42, "threshold": 2.2946674823760986, "operator": "ge"}, {"budget": 0.43, "threshold": 2.285684585571289, "operator": "ge"}, {"budget": 0.44, "threshold": 2.26644229888916, "operator": "ge"}, {"budget": 0.45, "threshold": 2.2532386779785156, "operator": "ge"}, {"budget": 0.46, "threshold": 2.2236523628234863, "operator": "ge"}, {"budget": 0.47, "threshold": 2.201781749725342, "operator": "ge"}, {"budget": 0.48, "threshold": 2.1860790252685547, "operator": "ge"}, {"budget": 0.49, "threshold": 2.1727871894836426, "operator": "ge"}, {"budget": 0.5, "threshold": 2.1452431678771973, "operator": "ge"}, {"budget": 0.51, "threshold": 2.1271581649780273, "operator": "ge"}, {"budget": 0.52, "threshold": 2.09397029876709, "operator": "ge"}, {"budget": 0.53, "threshold": 2.0738234519958496, "operator": "ge"}, {"budget": 0.54, "threshold": 2.052212715148926, "operator": "ge"}, {"budget": 0.55, "threshold": 2.026111602783203, "operator": "ge"}, {"budget": 0.56, "threshold": 2.008469820022583, "operator": "ge"}, {"budget": 0.57, "threshold": 1.9774677753448486, "operator": "ge"}, {"budget": 0.58, "threshold": 1.9451014995574951, "operator": "ge"}, {"budget": 0.59, "threshold": 1.9323174953460693, "operator": "ge"}, {"budget": 0.6, "threshold": 1.8959956169128418, "operator": "ge"}, {"budget": 0.61, "threshold": 1.8707058429718018, "operator": "ge"}, {"budget": 0.62, "threshold": 1.8278372287750244, "operator": "ge"}, {"budget": 0.63, "threshold": 1.781150460243225, "operator": "ge"}, {"budget": 0.64, "threshold": 1.7479432821273804, "operator": "ge"}, {"budget": 0.65, "threshold": 1.7170169353485107, "operator": "ge"}, {"budget": 0.66, "threshold": 1.6882503032684326, "operator": "ge"}, {"budget": 0.67, "threshold": 1.636399507522583, "operator": "ge"}, {"budget": 0.68, "threshold": 1.5872700214385986, "operator": "ge"}, {"budget": 0.69, "threshold": 1.4829435348510742, "operator": "ge"}, {"budget": 0.7, "threshold": 1.4387286901474, "operator": "ge"}, {"budget": 0.71, "threshold": 1.3774077892303467, "operator": "ge"}, {"budget": 0.72, "threshold": 1.3139827251434326, "operator": "ge"}, {"budget": 0.73, "threshold": 1.2789617776870728, "operator": "ge"}, {"budget": 0.74, "threshold": 1.2006540298461914, "operator": "ge"}, {"budget": 0.75, "threshold": 1.1218888759613037, "operator": "ge"}, {"budget": 0.76, "threshold": 1.0441391468048096, "operator": "ge"}, {"budget": 0.77, "threshold": 0.7043609619140625, "operator": "ge"}, {"budget": 0.78, "threshold": -0.8464820384979248, "operator": "ge"}, {"budget": 0.79, "threshold": -1.0140461921691895, "operator": "ge"}, {"budget": 0.8, "threshold": -1.2407920360565186, "operator": "ge"}, {"budget": 0.81, "threshold": -1.3411140441894531, "operator": "ge"}, {"budget": 0.82, "threshold": -1.4155470132827759, "operator": "ge"}, {"budget": 0.83, "threshold": -1.5104761123657227, "operator": "ge"}, {"budget": 0.84, "threshold": -1.5674372911453247, "operator": "ge"}, {"budget": 0.85, "threshold": -1.6155641078948975, "operator": "ge"}, {"budget": 0.86, "threshold": -1.7329479455947876, "operator": "ge"}, {"budget": 0.87, "threshold": -1.7763829231262207, "operator": "ge"}, {"budget": 0.88, "threshold": -1.906416654586792, "operator": "ge"}, {"budget": 0.89, "threshold": -2.000868797302246, "operator": "ge"}, {"budget": 0.9, "threshold": -2.0642483234405518, "operator": "ge"}, {"budget": 0.91, "threshold": -2.1399645805358887, "operator": "ge"}, {"budget": 0.92, "threshold": -2.2538318634033203, "operator": "ge"}, {"budget": 0.93, "threshold": -2.2923402786254883, "operator": "ge"}, {"budget": 0.94, "threshold": -2.3409998416900635, "operator": "ge"}, {"budget": 0.95, "threshold": -2.4492437839508057, "operator": "ge"}, {"budget": 0.96, "threshold": -2.5184175968170166, "operator": "ge"}, {"budget": 0.97, "threshold": -2.5756492614746094, "operator": "ge"}, {"budget": 0.98, "threshold": -2.6976816654205322, "operator": "ge"}, {"budget": 0.99, "threshold": -2.7643980979919434, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.727747157015708, 10.939482507438132, 10.946793912019476, 10.848879946267482, 10.930632456126123, 10.497139796055022, 10.560166352244389, 10.372945934443258, 10.447872777756064, 10.318130087181965, 10.296693803806436, 10.068729040912139, 10.159426486825867, 10.263502870497367, 10.229883753426144, 12.104753587896088, 12.091000414853823, 12.17719462276091, 12.265363368348742, 12.37458744104385, 12.451904901919605, 12.239631336405528, 12.26280716819881, 12.357128605632196, 13.921817336208406, 13.910093211216035, 14.009427362547935, 15.32416163491497, 15.287585417027879, 15.359022985236576, 15.005052418845521, 15.0074545065329, 14.842698255838584, 15.917693851939237, 16.059283307765938, 15.94549866010612, 15.815514184397164, 15.958603762335152, 16.04843086151827, 16.013598189621632, 15.983292060195907, 16.010992519613207, 16.135365016665304, 16.08630721449535, 16.17078922124794, 16.341259420205862, 16.514085686530667, 16.72621137905032, 16.943605208818926, 17.01354853460484, 17.142536702029552, 17.359724971719366, 17.58434864572756, 17.762619732728428, 17.986330398364885, 18.212032998710463, 18.51658219354622, 19.197609408135722, 19.49805868438657, 19.541369784560448, 20.41766662456318, 21.308269279379935, 22.028460548151124, 22.825944679423877, 23.095519531689742, 23.467939060904065, 23.540101421164717, 23.8109699618396, 24.561163337714053, 24.84911374753516, 24.91077288101368, 25.204678362573098, 25.59433088928491, 25.819775189371295, 26.09479435338492, 26.501726655616075, 26.906991010527566, 26.93115545451994, 26.818236146594355, 26.734061062419272, 26.841499939467628, 27.00096064066652, 27.00096064066652, 26.71233546519588, 26.498857942823463, 26.458333333333332, 26.28800107613667, 26.172438672438673, 26.070861281606366, 26.070861281606366, 26.03420927574205, 26.2168885125301, 26.2168885125301, 26.140083597015508, 26.024449924490067, 25.923840075129956, 25.923840075129956, 25.861315394334905, 25.677253133376805, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 10, 26, 38, 49, 61, 72, 84, 95, 104, 116, 127, 137, 149, 162, 179, 194, 208, 219, 230, 242, 250, 265, 279, 289, 305, 318, 330, 347, 358, 374, 386, 394, 404, 417, 424, 434, 449, 458, 470, 480, 490, 503, 516, 530, 538, 550, 561, 572, 585, 597, 603, 612, 621, 630, 640, 650, 660, 668, 678, 685, 693, 702, 709, 717, 726, 735, 742, 749, 758, 765, 774, 781, 787, 791, 798, 804, 810, 812, 812, 812, 813, 814, 814, 814, 814, 814, 814, 814, 814, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 1, 2, 3, 4, 4, 7, 7, 9, 9, 11, 12, 14, 14, 14, 16, 16, 17, 17, 17, 17, 17, 21, 22, 23, 23, 24, 24, 25, 27, 28, 32, 33, 36, 37, 37, 39, 42, 42, 43, 45, 49, 51, 52, 56, 57, 58, 59, 60, 61, 66, 67, 68, 69, 74, 80, 82, 82, 84, 85, 87, 87, 87, 90, 92, 94, 95, 98, 100, 105, 107, 113, 116, 121, 127, 130, 132, 139, 141, 143, 145, 146, 150, 150, 155, 159, 160, 163, 166, 168, 168, 169, 171, 171, 173, 174, 176, 176, 177, 180, 185], "exact_score": [7.641657905799261, 7.727747157015708, 10.939482507438132, 10.946793912019476, 10.848879946267482, 10.930632456126123, 10.497139796055022, 10.560166352244389, 10.372945934443258, 10.447872777756064, 10.318130087181965, 10.296693803806436, 10.068729040912139, 10.159426486825867, 10.263502870497367, 10.229883753426144, 12.104753587896088, 12.091000414853823, 12.17719462276091, 12.265363368348742, 12.37458744104385, 12.451904901919605, 12.239631336405528, 12.26280716819881, 12.357128605632196, 13.921817336208406, 13.910093211216035, 14.009427362547935, 15.32416163491497, 15.287585417027879, 15.359022985236576, 15.005052418845521, 15.0074545065329, 14.842698255838584, 15.917693851939237, 16.059283307765938, 15.94549866010612, 15.815514184397164, 15.958603762335152, 16.04843086151827, 16.013598189621632, 15.983292060195907, 16.010992519613207, 16.135365016665304, 16.08630721449535, 16.17078922124794, 16.341259420205862, 16.514085686530667, 16.72621137905032, 16.943605208818926, 17.01354853460484, 17.142536702029552, 17.359724971719366, 17.58434864572756, 17.762619732728428, 17.986330398364885, 18.212032998710463, 18.51658219354622, 19.197609408135722, 19.49805868438657, 19.541369784560448, 20.41766662456318, 21.308269279379935, 22.028460548151124, 22.825944679423877, 23.095519531689742, 23.467939060904065, 23.540101421164717, 23.8109699618396, 24.561163337714053, 24.84911374753516, 24.91077288101368, 25.204678362573098, 25.59433088928491, 25.819775189371295, 26.09479435338492, 26.501726655616075, 26.906991010527566, 26.93115545451994, 26.818236146594355, 26.734061062419272, 26.841499939467628, 27.00096064066652, 27.00096064066652, 26.71233546519588, 26.498857942823463, 26.458333333333332, 26.28800107613667, 26.172438672438673, 26.070861281606366, 26.070861281606366, 26.03420927574205, 26.2168885125301, 26.2168885125301, 26.140083597015508, 26.024449924490067, 25.923840075129956, 25.923840075129956, 25.861315394334905, 25.677253133376805, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 18.641744372789677, "threshold_area100": 18.63185695092516, "threshold_area25": 10.926994271756383}, "test": {"score": [7.8873978710439285, 7.9661198286457955, 8.032824052284385, 8.080762500852165, 8.147464436826656, 8.315750235073876, 8.258846516415092, 8.313854083932421, 9.02546320895469, 9.04115748886625, 9.081946837592522, 9.79483330720191, 10.465868713777805, 10.478530905053642, 10.421704411942079, 11.921770697775372, 11.852323157902797, 11.78065106072768, 11.722501754682211, 11.795166984666746, 11.772449425856497, 12.06473400156211, 12.030094290132359, 12.030447138915799, 12.017895942869433, 12.057699043631569, 12.80911777171904, 13.193248830233257, 13.208833509950557, 13.500968788725693, 13.438140897249621, 13.399536579353574, 13.906509544756776, 14.25944361337863, 15.011664432780176, 14.960859113989223, 14.976868271200514, 14.978473405898715, 15.419537060665197, 15.423705432567733, 15.531925607113578, 15.572326861713382, 15.857587729596323, 15.851011921178287, 16.011103338615914, 16.164866106995053, 16.325446392081755, 16.705761484219508, 16.992778434577946, 17.192924386755642, 17.46678656935111, 17.5369036091878, 17.94523448077644, 18.096723224003092, 18.681030863670163, 18.94025913197867, 19.096909035658495, 19.253168720373374, 19.947685079652942, 20.02331211945804, 20.683766884900084, 20.95977899031282, 21.39059163481566, 22.015192580087163, 22.47372485398544, 22.91568642472574, 23.298814305053522, 23.76729689068138, 24.222785926699196, 24.850062253711126, 25.124676812253604, 25.34266356138883, 25.478937914541255, 25.58168626510123, 25.869434183359363, 26.25522738715108, 26.463190333842107, 26.837474721410164, 27.067687499687622, 27.002386268391287, 26.884090423706503, 26.788618533836612, 26.77230846635817, 26.770281976526572, 26.882760542805524, 26.875378289741885, 26.79552405076783, 27.154074529074528, 27.12178870785596, 27.325319552747917, 27.48037448453513, 27.421787355147924, 27.389807695144263, 27.249711724707094, 27.310471719220953, 27.178440072234526, 27.1172976899382, 27.084265246191467, 27.213522796866563, 27.137183933714397, 27.001009067886244], "calls": [0.0, 1.3981358189081226, 2.2303595206391478, 2.796271637816245, 3.561917443408788, 5.0599201065246335, 5.59254327563249, 6.191744340878828, 7.490013315579228, 7.723035952063914, 8.155792276964048, 9.021304926764314, 10.186418109187748, 10.918774966711052, 12.683089214380825, 13.914780292942744, 15.14647137150466, 15.346205059920107, 16.17842876165113, 17.343541944074566, 18.242343541944077, 19.44074567243675, 20.672436750998667, 21.40479360852197, 22.33688415446072, 23.10252996005326, 24.567243675099867, 26.73102529960053, 27.896138482023968, 28.59520639147803, 29.82689747003995, 31.19174434087883, 32.39014647137151, 33.32223701731025, 35.75233022636485, 37.08388814913449, 37.849533954727036, 38.64846870838882, 40.246338215712385, 40.84553928095872, 42.07723035952064, 42.47669773635153, 43.10918774966711, 43.641810918774965, 44.90679094540613, 45.60585885486019, 47.17043941411451, 48.202396804260985, 49.00133155792277, 49.66711051930759, 50.93209054593875, 51.69773635153129, 53.19573901464714, 54.061251664447404, 55.05992010652463, 56.42476697736352, 57.19041278295606, 58.18908122503329, 59.38748335552596, 59.720372836218374, 61.118508655126504, 62.017310252996005, 63.24900133155792, 64.51398135818907, 65.67909454061251, 66.61118508655126, 67.41011984021304, 68.64181091877496, 69.84021304926765, 71.73768308921437, 72.50332889480693, 73.20239680426099, 73.73501997336885, 74.13448735019973, 74.96671105193076, 75.99866844207723, 76.56458055925432, 77.92942743009321, 79.12782956058588, 79.79360852197071, 80.85885486018641, 81.75765645805593, 82.42343541944075, 83.18908122503329, 83.85486018641811, 84.55392809587217, 85.7190412782956, 86.38482023968042, 87.9161118508655, 89.18109187749667, 89.98002663115847, 90.9121171770972, 92.34354194407457, 93.10918774966711, 93.60852197070572, 95.40612516644474, 96.23834886817576, 96.77097203728363, 97.93608521970705, 98.5352862849534, 100.0], "rescued": [0, 68, 110, 138, 178, 252, 272, 301, 360, 369, 392, 438, 497, 532, 609, 679, 734, 742, 782, 832, 874, 928, 987, 1024, 1064, 1098, 1165, 1261, 1313, 1341, 1400, 1467, 1525, 1573, 1698, 1759, 1799, 1845, 1917, 1949, 2006, 2021, 2048, 2071, 2133, 2164, 2233, 2282, 2317, 2348, 2406, 2439, 2513, 2545, 2589, 2651, 2688, 2729, 2785, 2798, 2856, 2893, 2937, 2988, 3035, 3070, 3101, 3151, 3191, 3258, 3279, 3297, 3313, 3323, 3349, 3377, 3391, 3429, 3447, 3447, 3447, 3447, 3447, 3448, 3450, 3450, 3450, 3453, 3454, 3456, 3457, 3457, 3458, 3458, 3459, 3459, 3460, 3460, 3461, 3461, 3461], "harmed": [0, 3, 4, 4, 7, 11, 15, 18, 21, 21, 21, 21, 24, 25, 33, 38, 45, 47, 51, 52, 56, 67, 74, 77, 81, 83, 92, 103, 107, 111, 120, 130, 141, 143, 159, 170, 177, 184, 199, 205, 212, 213, 217, 222, 229, 238, 247, 250, 253, 261, 273, 281, 293, 296, 302, 311, 319, 329, 342, 344, 350, 355, 373, 382, 392, 398, 408, 420, 433, 455, 466, 474, 484, 489, 501, 513, 518, 540, 551, 556, 563, 571, 572, 575, 577, 578, 584, 586, 597, 605, 614, 618, 622, 631, 633, 642, 648, 650, 656, 663, 676], "exact_score": [7.8873978710439285, 7.92936332143105, 8.013803837577463, 8.098149439304999, 8.224365047232157, 8.308800587400125, 8.29460208676393, 8.336607668790197, 9.063792021484005, 9.791192397417133, 10.452309356727735, 10.484630943026547, 10.39337979970911, 10.975270623930951, 11.926213205488697, 11.839343516812207, 11.769754933199362, 11.789816022449802, 11.764702931020619, 12.195727012286522, 12.043724138601245, 11.984842580256153, 11.984156416986844, 12.0470266851411, 12.036693115203084, 12.786512570225414, 12.773134007358145, 13.188159014958497, 13.21320116137703, 13.470406621181152, 13.428298290732025, 13.416885245956076, 13.681786677622565, 14.240039088895411, 14.233232420470646, 14.47207296284273, 15.02774441163816, 14.954742143284966, 14.964376927622702, 14.99879456416893, 15.419268991224918, 15.432861822951379, 15.513608129945647, 15.83144308223422, 15.892589796726384, 16.008717995320545, 16.211735215260077, 16.30669767186493, 16.64746440488554, 16.986111021907067, 17.218864536308196, 17.47937897537953, 17.594628868217402, 17.91308088998731, 18.096723224003092, 18.67611994992658, 18.854309547009773, 19.056215118424575, 19.227742364729075, 19.738418468130792, 20.21434799500568, 20.692638178199495, 20.949349021385906, 21.315019906101494, 21.739147529028592, 22.10742512024269, 22.723198961304643, 23.03353424957794, 23.489748829581046, 23.848149734093465, 24.238965808735358, 24.58096132424391, 24.849147863906953, 25.35037698581983, 25.525676109587327, 25.869434183359363, 26.25522738715108, 26.34888568507805, 26.851736101914835, 27.067687499687622, 26.952699860948236, 26.867287089925078, 26.77230846635817, 26.770281976526572, 26.882760542805524, 26.851603959924798, 27.15150315588032, 27.11423301465674, 27.12178870785596, 27.332615475369447, 27.48037448453513, 27.421787355147924, 27.424957038276865, 27.256641274286363, 27.2852400472141, 27.228173136805655, 27.150624485819037, 27.058366561844636, 27.206727729769227, 27.077983173963883, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 18.39894964795643, "threshold_area100": 18.511497893282538, "threshold_area25": 10.212893130727203}, "selected_budget": 82, "selected": {"val": {"score": 27.00096064066652, "calls": 81.8830242510699, "rescued": 814, "harmed": 150}, "test": {"score": 26.77230846635817, "calls": 82.42343541944075, "rescued": 3447, "harmed": 572}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.271299362182617, "operator": "ge"}, {"budget": 0.02, "threshold": 2.1229212284088135, "operator": "ge"}, {"budget": 0.03, "threshold": 2.057863235473633, "operator": "ge"}, {"budget": 0.04, "threshold": 2.022291660308838, "operator": "ge"}, {"budget": 0.05, "threshold": 1.9845831394195557, "operator": "ge"}, {"budget": 0.06, "threshold": 1.9554331302642822, "operator": "ge"}, {"budget": 0.07, "threshold": 1.94295072555542, "operator": "ge"}, {"budget": 0.08, "threshold": 1.917008876800537, "operator": "ge"}, {"budget": 0.09, "threshold": 1.9018887281417847, "operator": "ge"}, {"budget": 0.1, "threshold": 1.8841869831085205, "operator": "ge"}, {"budget": 0.11, "threshold": 1.8640384674072266, "operator": "ge"}, {"budget": 0.12, "threshold": 1.8542747497558594, "operator": "ge"}, {"budget": 0.13, "threshold": 1.8402765989303589, "operator": "ge"}, {"budget": 0.14, "threshold": 1.8286662101745605, "operator": "ge"}, {"budget": 0.15, "threshold": 1.8145148754119873, "operator": "ge"}, {"budget": 0.16, "threshold": 1.8027170896530151, "operator": "ge"}, {"budget": 0.17, "threshold": 1.7896029949188232, "operator": "ge"}, {"budget": 0.18, "threshold": 1.7844021320343018, "operator": "ge"}, {"budget": 0.19, "threshold": 1.7703324556350708, "operator": "ge"}, {"budget": 0.2, "threshold": 1.7574671506881714, "operator": "ge"}, {"budget": 0.21, "threshold": 1.7397809028625488, "operator": "ge"}, {"budget": 0.22, "threshold": 1.7270326614379883, "operator": "ge"}, {"budget": 0.23, "threshold": 1.7162444591522217, "operator": "ge"}, {"budget": 0.24, "threshold": 1.7043817043304443, "operator": "ge"}, {"budget": 0.25, "threshold": 1.694588303565979, "operator": "ge"}, {"budget": 0.26, "threshold": 1.6793323755264282, "operator": "ge"}, {"budget": 0.27, "threshold": 1.6668208837509155, "operator": "ge"}, {"budget": 0.28, "threshold": 1.6564199924468994, "operator": "ge"}, {"budget": 0.29, "threshold": 1.6418366432189941, "operator": "ge"}, {"budget": 0.3, "threshold": 1.6340806484222412, "operator": "ge"}, {"budget": 0.31, "threshold": 1.6256791353225708, "operator": "ge"}, {"budget": 0.32, "threshold": 1.61275053024292, "operator": "ge"}, {"budget": 0.33, "threshold": 1.5983508825302124, "operator": "ge"}, {"budget": 0.34, "threshold": 1.5854706764221191, "operator": "ge"}, {"budget": 0.35, "threshold": 1.5737383365631104, "operator": "ge"}, {"budget": 0.36, "threshold": 1.5599135160446167, "operator": "ge"}, {"budget": 0.37, "threshold": 1.5434826612472534, "operator": "ge"}, {"budget": 0.38, "threshold": 1.5319795608520508, "operator": "ge"}, {"budget": 0.39, "threshold": 1.5276782512664795, "operator": "ge"}, {"budget": 0.4, "threshold": 1.5061389207839966, "operator": "ge"}, {"budget": 0.41, "threshold": 1.4908626079559326, "operator": "ge"}, {"budget": 0.42, "threshold": 1.4820438623428345, "operator": "ge"}, {"budget": 0.43, "threshold": 1.464035987854004, "operator": "ge"}, {"budget": 0.44, "threshold": 1.4405449628829956, "operator": "ge"}, {"budget": 0.45, "threshold": 1.4285719394683838, "operator": "ge"}, {"budget": 0.46, "threshold": 1.4136545658111572, "operator": "ge"}, {"budget": 0.47, "threshold": 1.3984110355377197, "operator": "ge"}, {"budget": 0.48, "threshold": 1.382968544960022, "operator": "ge"}, {"budget": 0.49, "threshold": 1.3779382705688477, "operator": "ge"}, {"budget": 0.5, "threshold": 1.3681718111038208, "operator": "ge"}, {"budget": 0.51, "threshold": 1.353074312210083, "operator": "ge"}, {"budget": 0.52, "threshold": 1.3311665058135986, "operator": "ge"}, {"budget": 0.53, "threshold": 1.3190654516220093, "operator": "ge"}, {"budget": 0.54, "threshold": 1.305945873260498, "operator": "ge"}, {"budget": 0.55, "threshold": 1.2949254512786865, "operator": "ge"}, {"budget": 0.56, "threshold": 1.280215859413147, "operator": "ge"}, {"budget": 0.57, "threshold": 1.2682994604110718, "operator": "ge"}, {"budget": 0.58, "threshold": 1.2570348978042603, "operator": "ge"}, {"budget": 0.59, "threshold": 1.2500907182693481, "operator": "ge"}, {"budget": 0.6, "threshold": 1.2263844013214111, "operator": "ge"}, {"budget": 0.61, "threshold": 1.2028179168701172, "operator": "ge"}, {"budget": 0.62, "threshold": 1.1910754442214966, "operator": "ge"}, {"budget": 0.63, "threshold": 1.1670467853546143, "operator": "ge"}, {"budget": 0.64, "threshold": 1.1558740139007568, "operator": "ge"}, {"budget": 0.65, "threshold": 1.142194151878357, "operator": "ge"}, {"budget": 0.66, "threshold": 1.1153006553649902, "operator": "ge"}, {"budget": 0.67, "threshold": 1.101897954940796, "operator": "ge"}, {"budget": 0.68, "threshold": 1.0921903848648071, "operator": "ge"}, {"budget": 0.69, "threshold": 1.067117691040039, "operator": "ge"}, {"budget": 0.7, "threshold": 1.04327392578125, "operator": "ge"}, {"budget": 0.71, "threshold": 0.9932005405426025, "operator": "ge"}, {"budget": 0.72, "threshold": 0.9710846543312073, "operator": "ge"}, {"budget": 0.73, "threshold": 0.9319713115692139, "operator": "ge"}, {"budget": 0.74, "threshold": 0.8960179686546326, "operator": "ge"}, {"budget": 0.75, "threshold": 0.8229583501815796, "operator": "ge"}, {"budget": 0.76, "threshold": 0.7693363428115845, "operator": "ge"}, {"budget": 0.77, "threshold": 0.6150144338607788, "operator": "ge"}, {"budget": 0.78, "threshold": -0.7641030550003052, "operator": "ge"}, {"budget": 0.79, "threshold": -0.8848724365234375, "operator": "ge"}, {"budget": 0.8, "threshold": -1.147250771522522, "operator": "ge"}, {"budget": 0.81, "threshold": -1.2149007320404053, "operator": "ge"}, {"budget": 0.82, "threshold": -1.2891387939453125, "operator": "ge"}, {"budget": 0.83, "threshold": -1.3782628774642944, "operator": "ge"}, {"budget": 0.84, "threshold": -1.4558337926864624, "operator": "ge"}, {"budget": 0.85, "threshold": -1.4898865222930908, "operator": "ge"}, {"budget": 0.86, "threshold": -1.6005887985229492, "operator": "ge"}, {"budget": 0.87, "threshold": -1.673797369003296, "operator": "ge"}, {"budget": 0.88, "threshold": -1.7708669900894165, "operator": "ge"}, {"budget": 0.89, "threshold": -1.826490044593811, "operator": "ge"}, {"budget": 0.9, "threshold": -1.8970619440078735, "operator": "ge"}, {"budget": 0.91, "threshold": -1.9443310499191284, "operator": "ge"}, {"budget": 0.92, "threshold": -2.0260186195373535, "operator": "ge"}, {"budget": 0.93, "threshold": -2.102940797805786, "operator": "ge"}, {"budget": 0.94, "threshold": -2.158886432647705, "operator": "ge"}, {"budget": 0.95, "threshold": -2.2097978591918945, "operator": "ge"}, {"budget": 0.96, "threshold": -2.3130054473876953, "operator": "ge"}, {"budget": 0.97, "threshold": -2.3513500690460205, "operator": "ge"}, {"budget": 0.98, "threshold": -2.4380064010620117, "operator": "ge"}, {"budget": 0.99, "threshold": -2.5317888259887695, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.749774885257923, 7.864864536212851, 7.967245385794952, 8.063232947847238, 8.175211580211931, 8.275189121846099, 8.285253964271487, 8.37614716347037, 8.234983133850447, 10.348449875559865, 10.333682544635893, 10.384228253257621, 10.432610255321757, 10.387757838389554, 10.358801887429458, 11.887140637140638, 13.240762012759347, 13.319953090482517, 13.492666005184178, 13.392055648517873, 13.287368880818459, 13.422595095572015, 13.299624379490368, 13.459051302619478, 13.60983706783877, 13.674021930418304, 13.627977816002415, 13.635525969480753, 13.616556504879151, 13.6631966344629, 13.663715083661296, 13.80191030506456, 13.722046057138407, 13.873750713816937, 14.045007754207923, 14.215909030968293, 14.426657029555972, 14.410117812481147, 14.478064117087719, 14.659764717847878, 15.518808922170265, 15.619142599399439, 15.85678701902019, 15.921165756085065, 16.095894434204553, 16.348656087745272, 18.61623916374551, 18.792153091409304, 19.05834382185336, 19.222706584080033, 19.376832467175145, 19.590836378811062, 19.80915783886081, 19.93807274541064, 20.150563180943195, 20.288106639231884, 20.599344097260435, 20.725100171420852, 20.831779833108584, 20.928205230790333, 21.2218920854203, 21.2621641445191, 22.04770612078531, 23.064119970387864, 23.223961274233016, 23.476718463463655, 22.855101433774895, 23.040384134134133, 23.95774122148055, 23.987634552826314, 24.358851857514143, 24.806567701304544, 25.234944034687203, 25.94099052397148, 26.256389358448047, 26.59048236943431, 26.792555315919802, 26.93115545451994, 26.818236146594355, 26.734061062419272, 26.70452153210774, 27.00096064066652, 26.868708865660086, 26.66819291819292, 26.540087328130806, 26.458333333333332, 26.275252525252522, 26.172438672438673, 26.108119105749438, 26.070861281606366, 26.4407133407827, 26.2168885125301, 26.140083597015508, 26.140083597015508, 26.140083597015508, 25.923840075129956, 25.923840075129956, 25.861315394334905, 25.554334467763283, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 17, 35, 48, 61, 76, 89, 100, 112, 123, 137, 143, 158, 174, 188, 200, 214, 225, 235, 254, 265, 278, 292, 310, 324, 336, 347, 357, 369, 383, 392, 405, 418, 428, 441, 451, 460, 473, 480, 489, 501, 511, 518, 530, 539, 546, 558, 573, 581, 594, 605, 613, 624, 634, 643, 651, 659, 668, 676, 682, 690, 698, 703, 712, 721, 730, 737, 744, 751, 761, 765, 771, 780, 787, 792, 799, 805, 811, 812, 812, 812, 813, 814, 814, 814, 814, 814, 814, 814, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 1, 4, 4, 5, 7, 8, 11, 12, 13, 14, 15, 16, 17, 19, 21, 23, 25, 25, 25, 27, 29, 29, 32, 32, 32, 34, 36, 38, 41, 42, 45, 46, 49, 50, 50, 50, 50, 58, 60, 61, 64, 66, 67, 69, 70, 71, 71, 72, 73, 75, 76, 78, 80, 86, 87, 88, 88, 89, 91, 93, 93, 94, 96, 97, 99, 101, 105, 107, 108, 113, 117, 119, 124, 125, 130, 132, 138, 141, 143, 145, 148, 150, 152, 155, 158, 160, 164, 166, 167, 168, 168, 171, 173, 173, 173, 176, 176, 177, 182, 185], "exact_score": [7.641657905799261, 7.749774885257923, 7.864864536212851, 7.967245385794952, 8.063232947847238, 8.175211580211931, 8.275189121846099, 8.285253964271487, 8.37614716347037, 8.234983133850447, 10.348449875559865, 10.333682544635893, 10.384228253257621, 10.432610255321757, 10.387757838389554, 10.358801887429458, 11.887140637140638, 13.240762012759347, 13.319953090482517, 13.492666005184178, 13.392055648517873, 13.287368880818459, 13.422595095572015, 13.299624379490368, 13.459051302619478, 13.60983706783877, 13.674021930418304, 13.627977816002415, 13.635525969480753, 13.616556504879151, 13.6631966344629, 13.663715083661296, 13.80191030506456, 13.722046057138407, 13.873750713816937, 14.045007754207923, 14.215909030968293, 14.426657029555972, 14.410117812481147, 14.478064117087719, 14.659764717847878, 15.518808922170265, 15.619142599399439, 15.85678701902019, 15.921165756085065, 16.095894434204553, 16.348656087745272, 18.61623916374551, 18.792153091409304, 19.05834382185336, 19.222706584080033, 19.376832467175145, 19.590836378811062, 19.80915783886081, 19.93807274541064, 20.150563180943195, 20.288106639231884, 20.599344097260435, 20.725100171420852, 20.831779833108584, 20.928205230790333, 21.2218920854203, 21.2621641445191, 22.04770612078531, 23.064119970387864, 23.223961274233016, 23.476718463463655, 22.855101433774895, 23.040384134134133, 23.95774122148055, 23.987634552826314, 24.358851857514143, 24.806567701304544, 25.234944034687203, 25.94099052397148, 26.256389358448047, 26.59048236943431, 26.792555315919802, 26.93115545451994, 26.818236146594355, 26.734061062419272, 26.70452153210774, 27.00096064066652, 26.868708865660086, 26.66819291819292, 26.540087328130806, 26.458333333333332, 26.275252525252522, 26.172438672438673, 26.108119105749438, 26.070861281606366, 26.4407133407827, 26.2168885125301, 26.140083597015508, 26.140083597015508, 26.140083597015508, 25.923840075129956, 25.923840075129956, 25.861315394334905, 25.554334467763283, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 18.557979918158605, "threshold_area100": 18.548060631834346, "threshold_area25": 10.580053295707629}, "test": {"score": [7.8873978710439285, 7.944939139486225, 9.402145502801881, 10.038077051755204, 11.12185463573268, 11.142884877903278, 11.07392877096581, 11.04044979964302, 10.98385852670146, 10.971552909133543, 11.393520498326136, 12.158159439446628, 12.60610449621854, 12.588958222633655, 12.918361414124538, 12.836414309589298, 12.828612296425385, 12.687743113702558, 12.724440294467096, 12.99426929654443, 12.933446183450448, 12.94341574923458, 13.277100891082357, 13.233148270782765, 13.225274669836098, 13.502579055125208, 13.539478742993586, 13.542578115897284, 13.901615755294102, 14.08826701053299, 14.167367830643423, 14.19856831800805, 14.426044101977848, 14.461135796606111, 14.748844392004576, 14.752147272399881, 14.780383992834356, 15.303250071755913, 15.796448124218582, 15.845022181286478, 16.02081189438009, 16.157784842027663, 16.527783888535357, 16.689020412104593, 17.30192887602998, 17.47389872589893, 17.590623897728356, 17.726377545292994, 17.894965600036556, 18.024500564613025, 18.314999936215195, 18.672502770984124, 18.889555889601738, 19.049635160312974, 19.45314583650696, 19.59451713634688, 19.778816081956414, 19.83558239936158, 19.894004592952687, 20.149068467154603, 20.937971443106022, 21.743783826897587, 21.91750687099719, 22.45552184528788, 22.84490990940437, 23.167128022752312, 23.941767956300374, 24.12734973918984, 24.352338728694146, 24.64699363322041, 24.933931557572862, 25.389244563297748, 25.64031612158686, 25.891759623677157, 26.058417818378356, 26.54257338460963, 26.66689819072, 27.020953130695023, 27.067687499687622, 26.98578769223411, 26.851291568075332, 26.80496425173577, 26.77230846635817, 26.770281976526572, 26.890171243857843, 26.85909164479075, 26.817221836550125, 27.10414333876586, 27.039800584078733, 27.35602719406025, 27.276338757666384, 27.21036919848956, 27.416587516930313, 27.391012084745725, 27.308129893367127, 27.269401848885792, 27.113786461727408, 27.084265246191467, 27.00333959758533, 27.168668870978017, 27.001009067886244], "calls": [0.0, 0.5326231691078562, 1.9307589880159786, 3.129161118508655, 3.96138482023968, 4.92676431424767, 6.058588548601864, 6.657789613848203, 7.623169107856191, 8.355525965379494, 9.387483355525966, 10.952063914780293, 11.684420772303596, 12.549933422103862, 13.348868175765647, 14.08122503328895, 14.580559254327563, 15.579227696404793, 15.978695073235686, 16.910785619174433, 17.976031957390145, 19.540612516644472, 20.63914780292943, 21.837549933422103, 22.63648468708389, 23.235685752330227, 24.33422103861518, 25.432756324900136, 26.364846870838882, 27.363515312916114, 27.896138482023968, 28.59520639147803, 29.82689747003995, 30.825565912117174, 31.82423435419441, 32.7896138482024, 33.82157123834887, 35.21970705725699, 36.21837549933422, 36.61784287616511, 38.348868175765645, 39.34753661784288, 40.21304926764314, 41.61118508655127, 43.54194407456725, 44.44074567243675, 45.67243675099867, 47.003994673768304, 48.33555259653795, 48.8681757656458, 50.03328894806924, 51.33155792276965, 53.495339547270305, 54.49400798934754, 55.692410119840204, 56.35818908122503, 57.32356857523302, 58.25565912117177, 59.18774966711052, 59.720372836218374, 61.48468708388815, 63.315579227696404, 64.18109187749667, 65.9121171770972, 66.61118508655126, 67.64314247669773, 69.44074567243675, 70.1398135818908, 70.5392809587217, 71.60452729693742, 72.30359520639148, 73.36884154460719, 73.90146471371504, 74.66711051930758, 75.49933422103862, 76.56458055925432, 76.8641810918775, 77.72969374167776, 79.09454061251664, 79.6271637816245, 81.0918774966711, 81.69107856191744, 82.39014647137151, 83.25565912117176, 84.28761651131825, 84.68708388814915, 85.78561917443409, 86.95073235685751, 88.24900133155792, 88.91478029294274, 89.88015978695073, 90.87882822902796, 92.07723035952064, 92.97603195739015, 94.17443408788282, 94.8069241011984, 96.23834886817576, 96.70439414114514, 97.50332889480693, 98.36884154460719, 100.0], "rescued": [0, 39, 129, 203, 250, 299, 354, 384, 433, 467, 522, 595, 637, 680, 721, 761, 782, 835, 851, 897, 949, 1022, 1082, 1140, 1176, 1208, 1268, 1318, 1366, 1413, 1439, 1466, 1514, 1559, 1607, 1662, 1724, 1794, 1848, 1869, 1943, 1988, 2025, 2086, 2177, 2220, 2270, 2324, 2386, 2409, 2460, 2510, 2593, 2633, 2686, 2712, 2744, 2778, 2813, 2830, 2901, 2971, 3000, 3063, 3085, 3124, 3187, 3207, 3225, 3253, 3276, 3305, 3325, 3348, 3365, 3393, 3401, 3426, 3447, 3447, 3447, 3447, 3447, 3448, 3450, 3450, 3451, 3453, 3453, 3456, 3456, 3456, 3458, 3459, 3459, 3459, 3460, 3460, 3460, 3461, 3461], "harmed": [0, 2, 11, 17, 21, 23, 28, 31, 38, 43, 47, 58, 59, 62, 66, 71, 75, 85, 86, 92, 100, 108, 114, 123, 129, 132, 139, 146, 149, 158, 158, 161, 170, 175, 179, 189, 198, 206, 211, 213, 228, 231, 239, 251, 261, 272, 281, 291, 303, 304, 315, 323, 340, 349, 361, 366, 372, 380, 386, 391, 404, 417, 422, 434, 439, 444, 455, 459, 459, 470, 473, 480, 486, 497, 509, 521, 525, 538, 551, 557, 565, 570, 572, 575, 576, 579, 585, 590, 595, 602, 609, 617, 624, 627, 633, 636, 647, 650, 655, 660, 676], "exact_score": [7.8873978710439285, 7.9924686782349985, 9.408079792481058, 9.981908012803869, 11.12464153453914, 11.117187508646795, 11.067623546636424, 11.00324728204128, 10.986741684159378, 11.401950592853094, 11.808474269319685, 12.161989651682978, 12.634800137272695, 12.56109404878282, 12.859204012888684, 12.810244041307978, 12.724440294467096, 13.000976320411164, 12.933446183450448, 12.90454398593213, 13.281211283266355, 13.288983245653416, 13.24948455158731, 13.200427130844233, 13.523896319083182, 13.52567418356892, 13.86710126379779, 14.106827558646163, 14.182655684555428, 14.362287230502723, 14.430454620910139, 14.488552561547014, 14.710404334787231, 14.757248037715145, 14.796971634697394, 15.091723664988502, 15.765635476779922, 16.04304107735794, 16.019588536316416, 16.111331704911986, 16.378750520207245, 16.66886467369542, 16.906747015019967, 17.176634780945616, 17.27579547163922, 17.53645206120759, 17.59528265769567, 17.719143352385245, 17.82397364890919, 18.021845799249743, 18.32393070577974, 18.635428193157992, 18.782808628018582, 18.94284598198258, 18.935573847270387, 19.279311904905178, 19.64404166981759, 19.701102238700287, 19.755017446708607, 19.890409687042673, 20.22884141317396, 20.85437153824212, 21.169627825742506, 21.691923023856692, 21.924279045551636, 22.088164189560107, 22.67368686843172, 22.952809791382805, 23.403696335290746, 23.815979463119817, 24.062276567441415, 24.481173652387884, 24.811585447174718, 25.281544155717828, 25.665321464554204, 25.93830728370866, 26.338163209892755, 26.748006362253303, 26.84252569921362, 27.067687499687622, 26.952699860948236, 26.851291568075332, 26.788618533836612, 26.786580359015826, 26.859525682791553, 26.85173767622161, 27.05574903528366, 27.08793882256134, 27.047140046758457, 27.35602719406025, 27.269123750451378, 27.44454678951093, 27.43250433723056, 27.391012084745725, 27.31517316196791, 27.242133087978292, 27.113786461727408, 27.061694655342283, 26.980850870726535, 27.104102576948954, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.12025966327376, "threshold_area100": 19.18446162248748, "threshold_area25": 11.800250866470746}, "selected_budget": 82, "selected": {"val": {"score": 27.00096064066652, "calls": 81.8830242510699, "rescued": 814, "harmed": 150}, "test": {"score": 26.77230846635817, "calls": 82.39014647137151, "rescued": 3447, "harmed": 572}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 26.99586861016554, "calls": 80.88445078459344}, "test": {"score": 26.8624728799547, "calls": 82.39014647137151}}, "old_normalized_interior": {"val": {"score": 26.99586861016554, "calls": 80.88445078459344}, "test": {"score": 26.8624728799547, "calls": 82.39014647137151}}, "old_normalized_endpoints": {"val": {"score": 26.99586861016554, "calls": 80.88445078459344}, "test": {"score": 26.8624728799547, "calls": 82.39014647137151}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 7.762403481598755, 10.507760313680564, 10.089263311129425, 10.063882396409083, 9.93980001401545, 9.770449004471077, 9.809076694496765, 9.588861017411912, 9.707077440247145, 9.84980288933188, 9.838339848628303, 9.894405688180202, 9.863099999494244, 12.07105642389979, 12.11749188525183, 12.23018838455614, 12.22495614568321, 12.339380768795152, 12.45726530434912, 12.468900572814963, 12.543629865412424, 13.548972745917592, 13.614781899284292, 13.614706932248161, 14.586572624766688, 14.764138085070028, 15.626569939883945, 15.73760771891719, 15.776243298251247, 15.898808982535348, 15.992148938502094, 16.068964284849624, 16.04685203943894, 16.06949617403873, 16.166932868689802, 16.323613291839425, 16.538237224947885, 16.807450083312155, 17.68556436659171, 17.83806739069897, 18.14840654127365, 18.306241912436604, 18.986995738847018, 19.304830167447438, 19.51579652649989, 20.212728130708165, 20.109410578160574, 20.075276051683392, 20.290259747267097, 20.501389519761716, 20.60055650488447, 20.74203297677966, 20.992785453022556, 21.23091363863569, 21.480780537624124, 21.63814033373858, 21.743068868974067, 21.86423416400533, 22.120162446249402, 22.34263605107595, 22.42059494547708, 22.70816497869638, 22.64969208007183, 23.522337235911785, 23.759742738897, 23.989003674040287, 24.216923796166434, 24.49120254464266, 24.661222457668103, 24.89101738480753, 25.277741068675386, 25.490784194581266, 25.578416226328727, 25.731206271561945, 26.11899294735116, 26.51337774967813, 26.792555315919802, 26.93115545451994, 26.93115545451994, 26.88788788788789, 26.99586861016554, 26.788696616282827, 26.635700900406782, 26.503449125400348, 26.35479272615195, 26.582040035987408, 26.474527887571362, 26.86433298502264, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.626984126984127, 26.441891613434994, 26.193113079324306, 25.96833062984728, 25.891525714332687, 25.738396728721387, 25.554334467763283, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 7.762403481598755, 10.507760313680564, 10.089263311129425, 10.063882396409083, 9.93980001401545, 9.770449004471077, 9.809076694496765, 9.588861017411912, 9.707077440247145, 9.84980288933188, 9.838339848628303, 9.894405688180202, 9.863099999494244, 12.07105642389979, 12.11749188525183, 12.23018838455614, 12.22495614568321, 12.339380768795152, 12.45726530434912, 12.468900572814963, 12.543629865412424, 13.548972745917592, 13.614781899284292, 13.614706932248161, 14.586572624766688, 14.764138085070028, 15.626569939883945, 15.73760771891719, 15.776243298251247, 15.898808982535348, 15.992148938502094, 16.068964284849624, 16.04685203943894, 16.06949617403873, 16.166932868689802, 16.323613291839425, 16.538237224947885, 16.807450083312155, 17.68556436659171, 17.83806739069897, 18.14840654127365, 18.306241912436604, 18.986995738847018, 19.304830167447438, 19.51579652649989, 20.212728130708165, 20.109410578160574, 20.075276051683392, 20.290259747267097, 20.501389519761716, 20.60055650488447, 20.74203297677966, 20.992785453022556, 21.23091363863569, 21.480780537624124, 21.63814033373858, 21.743068868974067, 21.86423416400533, 22.120162446249402, 22.34263605107595, 22.42059494547708, 22.70816497869638, 22.64969208007183, 23.522337235911785, 23.759742738897, 23.989003674040287, 24.216923796166434, 24.49120254464266, 24.661222457668103, 24.89101738480753, 25.277741068675386, 25.490784194581266, 25.578416226328727, 25.731206271561945, 26.11899294735116, 26.51337774967813, 26.792555315919802, 26.93115545451994, 26.93115545451994, 26.88788788788789, 26.99586861016554, 26.788696616282827, 26.635700900406782, 26.503449125400348, 26.35479272615195, 26.582040035987408, 26.474527887571362, 26.86433298502264, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.82012432012432, 26.626984126984127, 26.441891613434994, 26.193113079324306, 25.96833062984728, 25.891525714332687, 25.738396728721387, 25.554334467763283, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 11.09802212533942, "exact_area100": 19.53904042359823}, "test": {"score": [7.8873978710439285, 10.573657199289089, 11.45228537098348, 11.764441571520877, 11.379872350578399, 11.607409232828164, 11.616734495192365, 11.57424271723863, 11.523071219409568, 11.526758193796487, 11.759264019216573, 12.294819531586858, 12.298790461037926, 12.230808067250159, 12.17478391027617, 12.162171200722591, 12.141476188431795, 12.119479164813242, 12.662959428482543, 12.715703216669521, 13.55626936919384, 13.698912983819156, 13.764078350145285, 13.717814278347298, 13.68538949155123, 13.690426801651501, 13.737313556128408, 13.770156869623465, 13.803446385596283, 13.874258672106043, 14.590869641879491, 14.859426346154612, 14.989927804650666, 15.284334808879644, 15.383403069920156, 15.691767000775855, 15.947846936877855, 16.12654133322839, 16.308684693105675, 16.361662583721966, 16.570930540357434, 16.934011299459755, 17.17454822864621, 17.39519715761628, 17.765606505060795, 18.12633485736551, 18.312438732433538, 18.582291181862626, 18.927351634449668, 19.184020546811805, 19.10002917898827, 18.977501637676482, 19.410400509919846, 19.685886786214734, 20.12333802903539, 20.23221727103078, 20.597682517167875, 20.847214122956682, 21.031274936347433, 21.1469628813515, 21.299617814742714, 21.449650288740703, 22.076001846316732, 22.20945971397235, 22.76843355730479, 22.892004188891963, 23.564185124452045, 24.08428865870408, 24.584752163901154, 24.794035202894896, 25.104107499270267, 25.78244333180676, 25.940518787816618, 26.15385530492788, 26.40030237918513, 26.564055065228757, 26.67669322290979, 26.96509088414254, 27.067687499687622, 26.98421988709958, 26.952699860948236, 26.8624728799547, 26.855048720597974, 26.823866535697043, 26.968191982487728, 26.95297071726646, 26.936636787596207, 26.86692262454396, 26.771129558462192, 26.68381122308976, 26.851773174632093, 27.16551730338022, 27.093466129205773, 27.145758430722406, 27.12435199384081, 27.31125012130041, 27.293749303904857, 27.241950605276934, 27.02847558895322, 26.894657519464683, 27.001009067886244], "calls": [0.0, 1.0319573901464714, 1.4980026631158456, 1.8641810918774968, 2.796271637816245, 3.96138482023968, 4.727030625832223, 4.960053262316911, 5.559254327563249, 6.1584553928095875, 7.057256990679095, 8.189081225033288, 8.721704394141145, 9.953395472703063, 10.719041278295606, 12.48335552596538, 13.282290279627162, 13.914780292942744, 15.013315579227696, 15.446071904127828, 16.81091877496671, 17.942743009320907, 19.340878828229027, 21.171770972037287, 22.503328894806923, 23.635153129161118, 24.93342210386152, 25.798934753661783, 26.73102529960053, 27.330226364846872, 29.19440745672437, 30.193075898801595, 31.158455392809586, 32.39014647137151, 33.88814913448735, 34.8202396804261, 35.78561917443409, 37.18375499334221, 38.18242343541944, 38.68175765645805, 39.84687083888149, 41.61118508655127, 42.709720372836216, 43.37549933422104, 44.44074567243675, 45.50599201065246, 46.40479360852197, 47.66977363515313, 48.50199733688416, 49.66711051930759, 51.89747003994674, 53.22902796271638, 54.593874833555255, 55.72569906790945, 56.69107856191744, 57.490013315579226, 58.28894806924101, 59.35419440745673, 60.45272969374168, 60.9520639147803, 61.98402130492676, 62.849533954727036, 64.24766977363515, 64.74700399467378, 65.81225033288948, 66.37816245006658, 67.64314247669773, 69.37416777629826, 71.13848202396804, 71.97070572569906, 73.10252996005326, 74.13448735019973, 74.73368841544608, 75.43275632490013, 76.09853528628496, 76.63115845539281, 76.89747003994674, 78.22902796271637, 78.99467376830891, 80.09320905459387, 80.9254327563249, 82.39014647137151, 82.85619174434088, 83.62183754993342, 84.35419440745673, 85.88548601864181, 86.81757656458056, 87.8162450066578, 88.38215712383489, 89.21438082556591, 90.81225033288948, 91.711051930759, 92.50998668442078, 93.37549933422103, 93.94141145139814, 94.57390146471371, 95.17310252996005, 96.5046604527297, 98.1358189081225, 99.30093209054593, 100.0], "exact_score": [7.8873978710439285, 10.602774921912797, 11.66103333161233, 11.34058612746552, 11.611948305190948, 11.578943479914122, 11.527252207290664, 11.752072233769951, 12.296161148595857, 12.25525995608922, 12.21056440818788, 12.178750420576268, 12.135104264052881, 12.142866607520732, 12.128867777539936, 12.659780032308444, 13.37977392274593, 13.554808821959307, 13.701165192592477, 13.843160803060762, 13.722285745762195, 13.69126199303925, 13.676989732472839, 13.743985256481789, 13.687988757987322, 13.734393835567813, 13.767543758872144, 13.852792220391407, 14.19604895973896, 14.402232774030526, 14.845602109227961, 14.982703160386187, 15.258884247681593, 15.332472019748703, 15.392684685760742, 15.702937293691562, 15.98858046525169, 16.11449752087004, 16.255386952806287, 16.431953385642757, 16.60675621606354, 16.726535785189185, 17.03858895222293, 17.24813856580085, 17.688117413446065, 17.863586013221838, 18.307695818737376, 18.437004238328488, 18.65687962162302, 19.053541977251804, 19.278360169453315, 18.940929124272362, 18.85874050520021, 18.961755067893176, 19.10133990051684, 19.45086411568949, 19.752772046745424, 20.167777702736696, 20.4147352827216, 20.76686638201753, 20.99234173327476, 21.156922386503314, 21.299617814742714, 21.481989449090012, 22.008557291871806, 22.44920402178467, 22.781954403190046, 23.212446037949398, 23.65949858324548, 23.973632343497076, 24.199643634467908, 24.54038707048618, 24.794035202894896, 25.05832289574887, 25.675434301383536, 26.038816187312268, 26.341600849107838, 26.703055448507047, 26.84611060196523, 27.067687499687622, 27.00121975110067, 26.952699860948236, 26.88634346698338, 26.855048720597974, 26.84091941897083, 26.968191982487728, 26.95297071726646, 26.929069978205984, 26.86692262454396, 26.68381122308976, 26.859002787124865, 26.844570639339672, 27.15822138075869, 27.124068458873218, 27.12435199384081, 27.316729076922066, 27.32424798861382, 27.243387716987215, 27.035393087328412, 26.957914681831397, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 11.974805231661776, "exact_area100": 19.38585973537182}}, "old_normalized_endpoint_budget": 81}, {"seed": 2030, "selection": {"best_epoch": 1, "epochs": 9, "alpha": 0.5, "validation_area100": 0.1988759021457664, "training_seconds": 3.6972284601069987, "test_read_during_selection": false}, "methods": {"normalized": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.572329728396382, "operator": "ge"}, {"budget": 0.02, "threshold": 0.5465720693831128, "operator": "ge"}, {"budget": 0.03, "threshold": 0.5374322063609457, "operator": "ge"}, {"budget": 0.04, "threshold": 0.5209902541130818, "operator": "ge"}, {"budget": 0.05, "threshold": 0.5141333671592765, "operator": "ge"}, {"budget": 0.06, "threshold": 0.5078937097731431, "operator": "ge"}, {"budget": 0.07, "threshold": 0.5030643000015104, "operator": "ge"}, {"budget": 0.08, "threshold": 0.494250652070851, "operator": "ge"}, {"budget": 0.09, "threshold": 0.4853979543434777, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4772703941379862, "operator": "ge"}, {"budget": 0.11, "threshold": 0.47104510514841547, "operator": "ge"}, {"budget": 0.12, "threshold": 0.4656308823572231, "operator": "ge"}, {"budget": 0.13, "threshold": 0.4621623212302724, "operator": "ge"}, {"budget": 0.14, "threshold": 0.4594926608188635, "operator": "ge"}, {"budget": 0.15, "threshold": 0.45533931215134477, "operator": "ge"}, {"budget": 0.16, "threshold": 0.45055670915392015, "operator": "ge"}, {"budget": 0.17, "threshold": 0.4480251408774666, "operator": "ge"}, {"budget": 0.18, "threshold": 0.4439760720698484, "operator": "ge"}, {"budget": 0.19, "threshold": 0.4411679828038295, "operator": "ge"}, {"budget": 0.2, "threshold": 0.43931550395253344, "operator": "ge"}, {"budget": 0.21, "threshold": 0.43848576360320546, "operator": "ge"}, {"budget": 0.22, "threshold": 0.43692717973763257, "operator": "ge"}, {"budget": 0.23, "threshold": 0.4351664269223775, "operator": "ge"}, {"budget": 0.24, "threshold": 0.43254799788806797, "operator": "ge"}, {"budget": 0.25, "threshold": 0.43091737350077347, "operator": "ge"}, {"budget": 0.26, "threshold": 0.42868728713675563, "operator": "ge"}, {"budget": 0.27, "threshold": 0.4249349847167223, "operator": "ge"}, {"budget": 0.28, "threshold": 0.42322705713374004, "operator": "ge"}, {"budget": 0.29, "threshold": 0.4209029178308862, "operator": "ge"}, {"budget": 0.3, "threshold": 0.4193082788368544, "operator": "ge"}, {"budget": 0.31, "threshold": 0.4169096500524064, "operator": "ge"}, {"budget": 0.32, "threshold": 0.4150658191583358, "operator": "ge"}, {"budget": 0.33, "threshold": 0.41047006812093323, "operator": "ge"}, {"budget": 0.34, "threshold": 0.40714911457948366, "operator": "ge"}, {"budget": 0.35, "threshold": 0.40415740308444653, "operator": "ge"}, {"budget": 0.36, "threshold": 0.3987751418951768, "operator": "ge"}, {"budget": 0.37, "threshold": 0.3935822986432651, "operator": "ge"}, {"budget": 0.38, "threshold": 0.3913795068123737, "operator": "ge"}, {"budget": 0.39, "threshold": 0.3882825064250148, "operator": "ge"}, {"budget": 0.4, "threshold": 0.3821143929669363, "operator": "ge"}, {"budget": 0.41, "threshold": 0.3780211281573881, "operator": "ge"}, {"budget": 0.42, "threshold": 0.3717299689918499, "operator": "ge"}, {"budget": 0.43, "threshold": 0.3655278832125894, "operator": "ge"}, {"budget": 0.44, "threshold": 0.3578155231033102, "operator": "ge"}, {"budget": 0.45, "threshold": 0.3493800965997522, "operator": "ge"}, {"budget": 0.46, "threshold": 0.34346644209525873, "operator": "ge"}, {"budget": 0.47, "threshold": 0.32602063996500086, "operator": "ge"}, {"budget": 0.48, "threshold": 0.314775867374529, "operator": "ge"}, {"budget": 0.49, "threshold": 0.3100054556210375, "operator": "ge"}, {"budget": 0.5, "threshold": 0.30717510812182, "operator": "ge"}, {"budget": 0.51, "threshold": 0.30149342055199047, "operator": "ge"}, {"budget": 0.52, "threshold": 0.29844346244884706, "operator": "ge"}, {"budget": 0.53, "threshold": 0.2954815444963165, "operator": "ge"}, {"budget": 0.54, "threshold": 0.2935367495684913, "operator": "ge"}, {"budget": 0.55, "threshold": 0.29085014943474663, "operator": "ge"}, {"budget": 0.56, "threshold": 0.28642773536158983, "operator": "ge"}, {"budget": 0.57, "threshold": 0.2832406018993828, "operator": "ge"}, {"budget": 0.58, "threshold": 0.28125262167709925, "operator": "ge"}, {"budget": 0.59, "threshold": 0.27835149900654416, "operator": "ge"}, {"budget": 0.6, "threshold": 0.2720262485943632, "operator": "ge"}, {"budget": 0.61, "threshold": 0.26707371516176365, "operator": "ge"}, {"budget": 0.62, "threshold": 0.26255991117646466, "operator": "ge"}, {"budget": 0.63, "threshold": 0.25323121439349766, "operator": "ge"}, {"budget": 0.64, "threshold": 0.2456490430970277, "operator": "ge"}, {"budget": 0.65, "threshold": 0.23973986468692648, "operator": "ge"}, {"budget": 0.66, "threshold": 0.22986592381967047, "operator": "ge"}, {"budget": 0.67, "threshold": 0.2218585021275819, "operator": "ge"}, {"budget": 0.68, "threshold": 0.21081727693632238, "operator": "ge"}, {"budget": 0.69, "threshold": 0.2034789761846648, "operator": "ge"}, {"budget": 0.7, "threshold": 0.19296547891494437, "operator": "ge"}, {"budget": 0.71, "threshold": 0.17799370721689398, "operator": "ge"}, {"budget": 0.72, "threshold": 0.15596685833712717, "operator": "ge"}, {"budget": 0.73, "threshold": 0.1494524180044864, "operator": "ge"}, {"budget": 0.74, "threshold": 0.12077424276436807, "operator": "ge"}, {"budget": 0.75, "threshold": 0.09064535019472736, "operator": "ge"}, {"budget": 0.76, "threshold": 0.07490472797978144, "operator": "ge"}, {"budget": 0.77, "threshold": 0.04976960848514279, "operator": "ge"}, {"budget": 0.78, "threshold": 0.03063844997830165, "operator": "ge"}, {"budget": 0.79, "threshold": 0.014436275812085083, "operator": "ge"}, {"budget": 0.8, "threshold": -0.031948209575491965, "operator": "ge"}, {"budget": 0.81, "threshold": -0.04046202160706626, "operator": "ge"}, {"budget": 0.82, "threshold": -0.04945142256409497, "operator": "ge"}, {"budget": 0.83, "threshold": -0.05905465074322661, "operator": "ge"}, {"budget": 0.84, "threshold": -0.06915286305848478, "operator": "ge"}, {"budget": 0.85, "threshold": -0.07263612075046469, "operator": "ge"}, {"budget": 0.86, "threshold": -0.07602972801970001, "operator": "ge"}, {"budget": 0.87, "threshold": -0.08158683199837223, "operator": "ge"}, {"budget": 0.88, "threshold": -0.08308161909151161, "operator": "ge"}, {"budget": 0.89, "threshold": -0.08831556465803223, "operator": "ge"}, {"budget": 0.9, "threshold": -0.09174658304251561, "operator": "ge"}, {"budget": 0.91, "threshold": -0.09433648481903953, "operator": "ge"}, {"budget": 0.92, "threshold": -0.09873044324412725, "operator": "ge"}, {"budget": 0.93, "threshold": -0.10163432649295738, "operator": "ge"}, {"budget": 0.94, "threshold": -0.10433101207597419, "operator": "ge"}, {"budget": 0.95, "threshold": -0.10636921371006217, "operator": "ge"}, {"budget": 0.96, "threshold": -0.1103366495578229, "operator": "ge"}, {"budget": 0.97, "threshold": -0.11599801100697062, "operator": "ge"}, {"budget": 0.98, "threshold": -0.12691519977458496, "operator": "ge"}, {"budget": 0.99, "threshold": -0.13606217537158466, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.755221508351605, 7.678439532806242, 7.76745652721252, 7.883475751396031, 7.902368116608821, 7.908354181367202, 8.032329372754903, 8.163837923202498, 9.73378258726343, 9.7683744530542, 11.113117354339352, 12.384119164907343, 12.508174588395097, 12.53346039383334, 13.563314041370475, 13.66362483936227, 14.445601146729405, 15.362580064797221, 15.267307609150006, 15.377513896023157, 15.413036226261296, 15.42576899719757, 15.553907227573077, 16.402219116149848, 16.299198679576204, 16.34284869962099, 16.52118474297766, 17.42299932982817, 17.28703476592556, 17.272109139681298, 17.321062043228114, 17.29766926520493, 17.41772629013027, 17.47716730930326, 17.418626036065433, 17.530820293057754, 17.622854133908078, 17.817271522109245, 17.946602724084457, 18.10303419155535, 17.928986695209517, 18.179677740371382, 19.004465817751445, 19.743752160349676, 20.072976808983736, 20.332156945102128, 20.707113048662343, 20.783730158730158, 21.001607394012456, 21.750613464704116, 21.86195362665951, 22.01298701298701, 21.886642281379125, 22.087891800034786, 22.116772116772115, 22.23501343857381, 22.346781973755323, 22.448986668053607, 22.729933013638732, 22.044158160684468, 22.30924567056712, 22.54002985091888, 22.62188874389831, 22.60973512211902, 22.92795320279648, 23.108311838622434, 23.29786276627381, 23.50125313283208, 23.831521739130434, 24.030532878562134, 24.337403889642697, 24.607508105350178, 24.873694491720073, 25.27542561238038, 25.746911149447914, 25.965138380232723, 26.183292206656695, 26.336865218663103, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.367307022479437, 27.202291172879413, 27.31513743708866, 26.96605644546148, 26.854096989966557, 26.809119478722177, 26.721451448330598, 26.50621118012422, 26.400569299385403, 26.400569299385403, 26.29621162199599, 26.092798366499753, 25.953468770190106, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.879064008285347, 25.727093126457586, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 19, 38, 56, 76, 93, 108, 127, 147, 167, 183, 201, 214, 229, 243, 259, 271, 284, 297, 312, 324, 336, 348, 359, 372, 383, 395, 408, 419, 433, 446, 456, 466, 479, 492, 504, 515, 525, 534, 547, 557, 566, 576, 589, 599, 610, 620, 632, 638, 644, 649, 654, 660, 663, 668, 674, 683, 689, 694, 700, 707, 713, 718, 723, 726, 734, 741, 745, 751, 758, 764, 770, 777, 783, 789, 796, 802, 805, 808, 812, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 6, 9, 10, 14, 19, 21, 24, 24, 25, 27, 28, 28, 28, 29, 31, 31, 34, 35, 37, 37, 38, 40, 41, 42, 44, 45, 45, 45, 48, 50, 51, 53, 55, 57, 61, 62, 63, 63, 65, 66, 71, 71, 72, 73, 73, 74, 75, 78, 78, 82, 83, 84, 87, 87, 89, 92, 94, 96, 99, 101, 105, 105, 111, 112, 113, 116, 120, 122, 123, 126, 127, 130, 132, 136, 136, 140, 141, 143, 144, 144, 144, 147, 150, 151, 156, 158, 159, 161, 165, 167, 167, 169, 173, 176, 177, 177, 177, 178, 180, 185], "exact_score": [7.641657905799261, 7.755221508351605, 7.678439532806242, 7.76745652721252, 7.883475751396031, 7.902368116608821, 7.908354181367202, 8.032329372754903, 8.163837923202498, 9.73378258726343, 9.7683744530542, 11.113117354339352, 12.384119164907343, 12.508174588395097, 12.53346039383334, 13.563314041370475, 13.66362483936227, 14.445601146729405, 15.362580064797221, 15.267307609150006, 15.377513896023157, 15.413036226261296, 15.42576899719757, 15.553907227573077, 16.402219116149848, 16.299198679576204, 16.34284869962099, 16.52118474297766, 17.42299932982817, 17.28703476592556, 17.272109139681298, 17.321062043228114, 17.29766926520493, 17.41772629013027, 17.47716730930326, 17.418626036065433, 17.530820293057754, 17.622854133908078, 17.817271522109245, 17.946602724084457, 18.10303419155535, 17.928986695209517, 18.179677740371382, 19.004465817751445, 19.743752160349676, 20.072976808983736, 20.332156945102128, 20.707113048662343, 20.783730158730158, 21.001607394012456, 21.750613464704116, 21.86195362665951, 22.01298701298701, 21.886642281379125, 22.087891800034786, 22.116772116772115, 22.23501343857381, 22.346781973755323, 22.448986668053607, 22.729933013638732, 22.044158160684468, 22.30924567056712, 22.54002985091888, 22.62188874389831, 22.60973512211902, 22.92795320279648, 23.108311838622434, 23.29786276627381, 23.50125313283208, 23.831521739130434, 24.030532878562134, 24.337403889642697, 24.607508105350178, 24.873694491720073, 25.27542561238038, 25.746911149447914, 25.965138380232723, 26.183292206656695, 26.336865218663103, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.367307022479437, 27.202291172879413, 27.31513743708866, 26.96605644546148, 26.854096989966557, 26.809119478722177, 26.721451448330598, 26.50621118012422, 26.400569299385403, 26.400569299385403, 26.29621162199599, 26.092798366499753, 25.953468770190106, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.879064008285347, 25.727093126457586, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.88759021457664, "threshold_area100": 19.879446972491053, "threshold_area25": 11.716399759948878}, "test": {"score": [7.8873978710439285, 7.92025125665438, 10.022897001407612, 10.598405832861445, 10.935886124758946, 11.272390499708452, 11.282912581480646, 11.248693786339544, 11.17003033475677, 11.756164123660742, 11.64483700118651, 11.993138828864582, 11.915379266957459, 12.527241763917903, 12.50733038428111, 12.544501062033778, 12.307368205870524, 12.43749128551216, 13.03873186335737, 13.028282048156612, 13.299966157781205, 13.302967635470056, 13.527057792637518, 13.532391237558189, 13.547649409894852, 13.517398206374176, 13.707747213323518, 13.861386643001355, 13.925913619262145, 14.320019687096844, 14.567559904646984, 14.849487495804265, 15.116626615947137, 15.519675149262971, 15.672525831183895, 15.76853289670726, 16.11556163831169, 16.216461094265313, 16.327469039273225, 16.389935634243436, 16.545142620652896, 16.59901275738111, 16.919873258837534, 16.989631865387572, 17.20032381009755, 17.73991464150713, 17.83521823610966, 17.87596984854971, 18.155638642393004, 18.346538800999543, 18.476827734123262, 18.54132386859783, 18.711452243499462, 18.88716776122585, 19.27540334248122, 19.219051538362784, 19.600881981470618, 19.746281134767546, 19.838834755049056, 19.940336867119697, 20.3106077475395, 21.047283717293148, 21.44953968734262, 22.18601483624841, 22.412067778780248, 22.545960585322664, 23.02690106358149, 23.20854560052026, 23.419235164438625, 23.579607505708704, 24.015725653656688, 24.739155433607234, 25.243037991445632, 25.412845029840852, 25.827501374118523, 26.24933854152766, 26.358947128572453, 26.593668348635852, 26.76540338517288, 26.989568494530463, 26.937586210032027, 26.87696272623368, 26.76415220681962, 26.663932165812867, 26.674814954866104, 26.58674782919362, 26.693293741050077, 26.715062818028628, 26.70761098233252, 26.883550436364278, 26.861861445907437, 26.948143779905827, 27.17690106225915, 27.117738912078703, 27.039799102275826, 27.105679776846603, 27.083110821066615, 26.97763910098948, 26.980258366261644, 26.896670567331594, 27.001009067886244], "calls": [0.0, 0.3328894806924101, 1.4980026631158456, 1.9307589880159786, 2.929427430093209, 3.395472703062583, 3.7283621837549936, 4.194407456724368, 4.92676431424767, 5.59254327563249, 6.1584553928095875, 6.657789613848203, 7.589880159786951, 8.355525965379494, 9.021304926764314, 10.252996005326231, 12.649800266311583, 13.948069241011984, 15.745672436750999, 16.77762982689747, 17.7762982689747, 17.942743009320907, 18.608521970705727, 19.307589880159785, 20.70572569906791, 21.537949400798933, 23.03595206391478, 24.766977363515313, 25.798934753661783, 26.864181091877498, 27.363515312916114, 28.395472703062584, 29.294274300932088, 31.591211717709722, 32.75632490013315, 33.95472703062583, 35.65246338215712, 37.217043941411454, 37.91611185086551, 38.41544607190413, 39.88015978695073, 40.71238348868176, 41.84420772303595, 42.709720372836216, 44.10785619174435, 45.40612516644474, 46.17177097203729, 47.70306258322237, 48.7017310252996, 50.03328894806924, 50.832223701731024, 52.962716378162455, 54.260985352862846, 55.32623169107856, 56.15845539280959, 57.090545938748335, 58.3555259653795, 59.08788282290279, 59.58721704394141, 60.01997336884154, 61.55126498002663, 62.35019973368841, 62.916111850865505, 64.34753661784288, 65.31291611185085, 65.9121171770972, 67.21038615179761, 67.97603195739015, 69.37416777629826, 69.87350199733689, 71.03861517976033, 72.47003994673769, 73.93475366178428, 74.36750998668442, 75.7323568575233, 77.06391478029295, 77.52996005326231, 78.09587217043942, 78.59520639147803, 79.09454061251664, 80.45938748335553, 80.82556591211718, 81.6577896138482, 82.49001331557922, 84.25432756324899, 85.21970705725698, 86.15179760319573, 87.35019973368841, 87.78295605858855, 89.81358189081226, 90.54593874833556, 91.34487350199734, 92.31025299600533, 93.27563249001332, 94.2410119840213, 94.77363515312916, 95.70572569906791, 96.90412782956058, 98.16910785619174, 98.93475366178428, 100.0], "rescued": [0, 27, 122, 154, 239, 275, 303, 340, 396, 442, 481, 514, 571, 617, 650, 723, 855, 925, 1026, 1086, 1143, 1153, 1188, 1227, 1302, 1343, 1422, 1511, 1569, 1628, 1659, 1715, 1767, 1886, 1943, 2006, 2086, 2164, 2196, 2220, 2290, 2322, 2377, 2414, 2468, 2522, 2553, 2603, 2634, 2669, 2689, 2741, 2776, 2804, 2831, 2856, 2894, 2915, 2929, 2942, 2989, 3014, 3029, 3069, 3095, 3108, 3140, 3157, 3190, 3205, 3239, 3280, 3320, 3333, 3371, 3405, 3415, 3428, 3437, 3446, 3447, 3447, 3447, 3447, 3448, 3448, 3451, 3452, 3452, 3454, 3454, 3455, 3456, 3456, 3456, 3457, 3458, 3459, 3460, 3460, 3461], "harmed": [0, 3, 10, 14, 25, 31, 32, 35, 41, 50, 56, 60, 67, 73, 78, 82, 103, 119, 137, 143, 147, 148, 152, 156, 164, 170, 181, 190, 196, 201, 203, 208, 213, 227, 232, 244, 253, 263, 266, 270, 278, 287, 296, 307, 316, 321, 329, 338, 343, 347, 348, 361, 371, 380, 386, 391, 397, 401, 405, 407, 415, 422, 429, 437, 444, 449, 455, 458, 467, 471, 479, 491, 500, 503, 520, 537, 546, 550, 553, 554, 558, 561, 570, 576, 583, 589, 590, 594, 595, 606, 609, 609, 612, 619, 626, 627, 633, 640, 653, 661, 676], "exact_score": [7.8873978710439285, 9.385691199033147, 10.607014724065985, 10.886995681134353, 11.251480662221043, 11.132795060465142, 11.681096088731294, 11.950485050656404, 12.257252647244954, 12.506060671997915, 12.514939040453227, 12.465605022280354, 12.420055756946361, 12.29887380374021, 12.419722833745093, 12.575536441831794, 13.044082832538306, 13.007265914495877, 13.307566938348394, 13.54675677097075, 13.497074182497135, 13.555566828996898, 13.489687173219936, 13.716701233693632, 13.75804044864708, 13.887624658384269, 13.92250071040878, 14.330481760984783, 14.811091624801936, 15.11196029225728, 15.340141855596729, 15.470228260094428, 15.602397136523424, 15.70686053663336, 15.77343114114952, 15.878079775290255, 16.185375446708754, 16.290856126262142, 16.331182187887638, 16.50044815857474, 16.56092601480848, 16.77465374797612, 16.94284498489534, 16.999451677577518, 17.163645586166435, 17.629775924543, 17.811488078872795, 17.905962000293275, 17.8983029118888, 18.211534969415936, 18.346538800999543, 18.494315902646907, 18.413902519057483, 18.548729286284622, 18.669420736132167, 18.829851654407225, 19.133282926264325, 19.193379699569622, 19.540879014531914, 19.73800209959461, 19.923183286960267, 20.164179388371707, 20.751172953972016, 21.47132790658189, 21.93063742480102, 22.362366590401837, 22.721608381678863, 22.974930007524037, 23.20854560052026, 23.2383924026918, 23.619367202525236, 24.009757143686, 24.453429561409283, 24.902699748790198, 25.246829349184345, 25.60144526845346, 25.8921080715241, 26.21796012896782, 26.5317341035184, 26.961660151404732, 26.989568494530463, 26.869480281200865, 26.740175462729944, 26.640355050620595, 26.682225655918423, 26.602846423093073, 26.624796634848057, 26.757955225427448, 26.684087621965713, 26.906854172574672, 26.869063981199858, 26.948143779905827, 27.17690106225915, 27.13203615169023, 27.060893323128965, 27.091464853822067, 27.12052075905354, 26.97763910098948, 27.056400498241338, 26.88989959621687, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.189318810152546, "threshold_area100": 19.161997674054312, "threshold_area25": 11.892818551883801}, "selected_budget": 82, "selected": {"val": {"score": 27.367307022479437, "calls": 81.8830242510699, "rescued": 814, "harmed": 147}, "test": {"score": 26.76415220681962, "calls": 81.6577896138482, "rescued": 3447, "harmed": 570}}}, "difference": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 0.5294074416160583, "operator": "ge"}, {"budget": 0.02, "threshold": 0.494640588760376, "operator": "ge"}, {"budget": 0.03, "threshold": 0.47742119431495667, "operator": "ge"}, {"budget": 0.04, "threshold": 0.4612107276916504, "operator": "ge"}, {"budget": 0.05, "threshold": 0.45329347252845764, "operator": "ge"}, {"budget": 0.06, "threshold": 0.4491395056247711, "operator": "ge"}, {"budget": 0.07, "threshold": 0.44568443298339844, "operator": "ge"}, {"budget": 0.08, "threshold": 0.4372227191925049, "operator": "ge"}, {"budget": 0.09, "threshold": 0.42639732360839844, "operator": "ge"}, {"budget": 0.1, "threshold": 0.4168891906738281, "operator": "ge"}, {"budget": 0.11, "threshold": 0.4019535183906555, "operator": "ge"}, {"budget": 0.12, "threshold": 0.3922868072986603, "operator": "ge"}, {"budget": 0.13, "threshold": 0.3785994052886963, "operator": "ge"}, {"budget": 0.14, "threshold": 0.3742225766181946, "operator": "ge"}, {"budget": 0.15, "threshold": 0.36738845705986023, "operator": "ge"}, {"budget": 0.16, "threshold": 0.36418235301971436, "operator": "ge"}, {"budget": 0.17, "threshold": 0.36226019263267517, "operator": "ge"}, {"budget": 0.18, "threshold": 0.3604455888271332, "operator": "ge"}, {"budget": 0.19, "threshold": 0.358246773481369, "operator": "ge"}, {"budget": 0.2, "threshold": 0.35388338565826416, "operator": "ge"}, {"budget": 0.21, "threshold": 0.3500102162361145, "operator": "ge"}, {"budget": 0.22, "threshold": 0.34869658946990967, "operator": "ge"}, {"budget": 0.23, "threshold": 0.3452640473842621, "operator": "ge"}, {"budget": 0.24, "threshold": 0.3425993323326111, "operator": "ge"}, {"budget": 0.25, "threshold": 0.34005898237228394, "operator": "ge"}, {"budget": 0.26, "threshold": 0.33765482902526855, "operator": "ge"}, {"budget": 0.27, "threshold": 0.3353276252746582, "operator": "ge"}, {"budget": 0.28, "threshold": 0.33160531520843506, "operator": "ge"}, {"budget": 0.29, "threshold": 0.32694539427757263, "operator": "ge"}, {"budget": 0.3, "threshold": 0.32440873980522156, "operator": "ge"}, {"budget": 0.31, "threshold": 0.32169538736343384, "operator": "ge"}, {"budget": 0.32, "threshold": 0.31803667545318604, "operator": "ge"}, {"budget": 0.33, "threshold": 0.31494542956352234, "operator": "ge"}, {"budget": 0.34, "threshold": 0.3126857578754425, "operator": "ge"}, {"budget": 0.35, "threshold": 0.3109026849269867, "operator": "ge"}, {"budget": 0.36, "threshold": 0.30761486291885376, "operator": "ge"}, {"budget": 0.37, "threshold": 0.30497637391090393, "operator": "ge"}, {"budget": 0.38, "threshold": 0.3007478713989258, "operator": "ge"}, {"budget": 0.39, "threshold": 0.296500027179718, "operator": "ge"}, {"budget": 0.4, "threshold": 0.2938615083694458, "operator": "ge"}, {"budget": 0.41, "threshold": 0.2903341054916382, "operator": "ge"}, {"budget": 0.42, "threshold": 0.28405970335006714, "operator": "ge"}, {"budget": 0.43, "threshold": 0.2800697982311249, "operator": "ge"}, {"budget": 0.44, "threshold": 0.27229321002960205, "operator": "ge"}, {"budget": 0.45, "threshold": 0.2659602463245392, "operator": "ge"}, {"budget": 0.46, "threshold": 0.2515752613544464, "operator": "ge"}, {"budget": 0.47, "threshold": 0.24298372864723206, "operator": "ge"}, {"budget": 0.48, "threshold": 0.23436039686203003, "operator": "ge"}, {"budget": 0.49, "threshold": 0.21194443106651306, "operator": "ge"}, {"budget": 0.5, "threshold": 0.19784560799598694, "operator": "ge"}, {"budget": 0.51, "threshold": 0.1924704611301422, "operator": "ge"}, {"budget": 0.52, "threshold": 0.1872616410255432, "operator": "ge"}, {"budget": 0.53, "threshold": 0.1849394589662552, "operator": "ge"}, {"budget": 0.54, "threshold": 0.1807606816291809, "operator": "ge"}, {"budget": 0.55, "threshold": 0.17702814936637878, "operator": "ge"}, {"budget": 0.56, "threshold": 0.17233505845069885, "operator": "ge"}, {"budget": 0.57, "threshold": 0.17004229128360748, "operator": "ge"}, {"budget": 0.58, "threshold": 0.1656397581100464, "operator": "ge"}, {"budget": 0.59, "threshold": 0.16230988502502441, "operator": "ge"}, {"budget": 0.6, "threshold": 0.1576804518699646, "operator": "ge"}, {"budget": 0.61, "threshold": 0.1540406048297882, "operator": "ge"}, {"budget": 0.62, "threshold": 0.15068459510803223, "operator": "ge"}, {"budget": 0.63, "threshold": 0.1465812623500824, "operator": "ge"}, {"budget": 0.64, "threshold": 0.14213751256465912, "operator": "ge"}, {"budget": 0.65, "threshold": 0.1385810226202011, "operator": "ge"}, {"budget": 0.66, "threshold": 0.12826214730739594, "operator": "ge"}, {"budget": 0.67, "threshold": 0.12137158960103989, "operator": "ge"}, {"budget": 0.68, "threshold": 0.11455860733985901, "operator": "ge"}, {"budget": 0.69, "threshold": 0.10654232650995255, "operator": "ge"}, {"budget": 0.7, "threshold": 0.10056981444358826, "operator": "ge"}, {"budget": 0.71, "threshold": 0.09293831139802933, "operator": "ge"}, {"budget": 0.72, "threshold": 0.08128825575113297, "operator": "ge"}, {"budget": 0.73, "threshold": 0.07514631003141403, "operator": "ge"}, {"budget": 0.74, "threshold": 0.06552620977163315, "operator": "ge"}, {"budget": 0.75, "threshold": 0.0468968003988266, "operator": "ge"}, {"budget": 0.76, "threshold": 0.03131581097841263, "operator": "ge"}, {"budget": 0.77, "threshold": 0.01941778138279915, "operator": "ge"}, {"budget": 0.78, "threshold": 0.008787915110588074, "operator": "ge"}, {"budget": 0.79, "threshold": 0.003965241834521294, "operator": "ge"}, {"budget": 0.8, "threshold": -0.007380368188023567, "operator": "ge"}, {"budget": 0.81, "threshold": -0.01067301258444786, "operator": "ge"}, {"budget": 0.82, "threshold": -0.013408130034804344, "operator": "ge"}, {"budget": 0.83, "threshold": -0.015626760199666023, "operator": "ge"}, {"budget": 0.84, "threshold": -0.018017388880252838, "operator": "ge"}, {"budget": 0.85, "threshold": -0.019606532528996468, "operator": "ge"}, {"budget": 0.86, "threshold": -0.020753193646669388, "operator": "ge"}, {"budget": 0.87, "threshold": -0.021337540820240974, "operator": "ge"}, {"budget": 0.88, "threshold": -0.022338194772601128, "operator": "ge"}, {"budget": 0.89, "threshold": -0.023110508918762207, "operator": "ge"}, {"budget": 0.9, "threshold": -0.024006342515349388, "operator": "ge"}, {"budget": 0.91, "threshold": -0.02561001665890217, "operator": "ge"}, {"budget": 0.92, "threshold": -0.026932328939437866, "operator": "ge"}, {"budget": 0.93, "threshold": -0.02742921933531761, "operator": "ge"}, {"budget": 0.94, "threshold": -0.028195923194289207, "operator": "ge"}, {"budget": 0.95, "threshold": -0.02919921651482582, "operator": "ge"}, {"budget": 0.96, "threshold": -0.03163732960820198, "operator": "ge"}, {"budget": 0.97, "threshold": -0.033344581723213196, "operator": "ge"}, {"budget": 0.98, "threshold": -0.03688095510005951, "operator": "ge"}, {"budget": 0.99, "threshold": -0.04126995801925659, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.755221508351605, 7.880021491155563, 7.79077622309625, 7.874981018130656, 7.994923181344685, 8.018657277757294, 8.154522985159993, 8.174688911998853, 8.312854226717, 9.749401282906176, 9.890805560027646, 11.220362889246523, 13.505017581116835, 14.14682637650026, 15.176759087602013, 15.029016408640029, 15.029679053270323, 15.043777829493749, 15.156777454571571, 15.176609598889687, 16.02468961053202, 16.142778581974593, 16.041416326203635, 16.680666077217804, 16.695842288289402, 16.54500673656278, 16.51960150161589, 16.672245789490315, 16.67401632141401, 16.75825038527307, 16.910659364592806, 16.92706687558044, 17.14063421503339, 17.085943531381908, 17.203710032657398, 17.17383054432545, 17.395147248088424, 17.56192452533916, 17.715643274853797, 18.436243585620034, 18.519907693435396, 18.789090712119023, 19.06405031892275, 19.932436229117382, 20.102157681773683, 20.410569105691057, 20.780495443820296, 21.018084426753155, 20.4775426139744, 20.546433061852404, 20.62997080438941, 20.490197435875587, 20.67008845129292, 20.590088211482414, 21.17552598807105, 21.306818181818183, 21.340941040156437, 21.56132811078163, 21.59796942183034, 21.72740579536624, 22.020409467217977, 22.11624795744195, 22.32816517139809, 22.452629972752266, 22.683416442749795, 22.87428705033905, 23.016903731189444, 23.34013605442177, 23.625965537730245, 23.855772839540695, 24.214966898247198, 24.487642635079983, 24.827017867373545, 25.287005344382397, 25.747683145988233, 26.049313464407803, 26.375599898964385, 26.810382507660037, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.02691581790501, 27.318166973339387, 27.242627038491303, 27.079468578280668, 26.96605644546148, 26.89986358035557, 26.809119478722177, 26.721451448330598, 26.54822798684691, 26.317203225354653, 26.19156400828535, 26.15024169423576, 26.069612788773156, 26.030273685704703, 25.953468770190106, 25.915973457104247, 25.879064008285347, 25.691314951144523, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 19, 40, 58, 74, 94, 111, 131, 148, 168, 189, 207, 222, 240, 250, 263, 275, 287, 300, 312, 325, 340, 351, 362, 374, 384, 397, 409, 420, 433, 446, 456, 468, 480, 493, 503, 515, 526, 537, 547, 560, 571, 581, 591, 603, 611, 620, 629, 638, 649, 657, 662, 666, 671, 675, 679, 684, 688, 694, 700, 706, 711, 717, 722, 728, 735, 741, 745, 751, 758, 764, 770, 777, 783, 790, 796, 803, 807, 811, 812, 813, 813, 813, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 6, 7, 11, 15, 17, 22, 24, 25, 26, 28, 29, 29, 30, 34, 34, 36, 37, 38, 38, 39, 40, 40, 42, 45, 47, 51, 53, 53, 55, 56, 56, 58, 58, 62, 63, 66, 66, 67, 68, 70, 72, 72, 72, 72, 73, 73, 73, 75, 76, 79, 84, 89, 89, 92, 95, 96, 98, 98, 100, 102, 108, 111, 115, 116, 118, 120, 125, 125, 127, 129, 129, 132, 133, 133, 136, 140, 141, 141, 144, 144, 144, 146, 148, 151, 154, 156, 157, 159, 161, 164, 168, 170, 171, 173, 174, 176, 177, 178, 181, 185], "exact_score": [7.641657905799261, 7.755221508351605, 7.880021491155563, 7.79077622309625, 7.874981018130656, 7.994923181344685, 8.018657277757294, 8.154522985159993, 8.174688911998853, 8.312854226717, 9.749401282906176, 9.890805560027646, 11.220362889246523, 13.505017581116835, 14.14682637650026, 15.176759087602013, 15.029016408640029, 15.029679053270323, 15.043777829493749, 15.156777454571571, 15.176609598889687, 16.02468961053202, 16.142778581974593, 16.041416326203635, 16.680666077217804, 16.695842288289402, 16.54500673656278, 16.51960150161589, 16.672245789490315, 16.67401632141401, 16.75825038527307, 16.910659364592806, 16.92706687558044, 17.14063421503339, 17.085943531381908, 17.203710032657398, 17.17383054432545, 17.395147248088424, 17.56192452533916, 17.715643274853797, 18.436243585620034, 18.519907693435396, 18.789090712119023, 19.06405031892275, 19.932436229117382, 20.102157681773683, 20.410569105691057, 20.780495443820296, 21.018084426753155, 20.4775426139744, 20.546433061852404, 20.62997080438941, 20.490197435875587, 20.67008845129292, 20.590088211482414, 21.17552598807105, 21.306818181818183, 21.340941040156437, 21.56132811078163, 21.59796942183034, 21.72740579536624, 22.020409467217977, 22.11624795744195, 22.32816517139809, 22.452629972752266, 22.683416442749795, 22.87428705033905, 23.016903731189444, 23.34013605442177, 23.625965537730245, 23.855772839540695, 24.214966898247198, 24.487642635079983, 24.827017867373545, 25.287005344382397, 25.747683145988233, 26.049313464407803, 26.375599898964385, 26.810382507660037, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.02691581790501, 27.318166973339387, 27.242627038491303, 27.079468578280668, 26.96605644546148, 26.89986358035557, 26.809119478722177, 26.721451448330598, 26.54822798684691, 26.317203225354653, 26.19156400828535, 26.15024169423576, 26.069612788773156, 26.030273685704703, 25.953468770190106, 25.915973457104247, 25.879064008285347, 25.691314951144523, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.774890890402506, "threshold_area100": 19.76661220512189, "threshold_area25": 12.053288649557379}, "test": {"score": [7.8873978710439285, 7.902496843122984, 10.006345866084077, 10.404434663573523, 11.26743929525685, 11.205626750076433, 11.207137560994362, 11.172321938300476, 11.042590948010874, 10.938590239939355, 11.34263514284806, 11.829488037382118, 12.112811042879075, 12.72565150793701, 12.8080602729306, 12.770055379339471, 12.718809654692178, 12.68774511404093, 13.134737160019425, 13.012581188710943, 13.016600001024711, 13.41323715783807, 13.41047364293183, 13.413319261603299, 13.656290804450109, 13.680234690328058, 13.8302033769031, 13.853032221113873, 14.046511761185677, 14.243149741153205, 14.410731464732764, 14.670251870749418, 15.216966514183833, 15.575593489965739, 15.607858958049153, 15.679842286924359, 15.790147745309918, 15.954056967875124, 16.17578676834414, 16.29397893203091, 16.411178648232593, 16.555996011728958, 16.88915171817375, 17.287153778786923, 17.39078980211962, 17.561358971905566, 18.332775307622352, 18.54388786288819, 18.74232782288322, 18.88026423595136, 18.76506759152696, 18.91031420756463, 19.36037761508298, 19.53095210130287, 19.579529474712945, 19.714423239479085, 19.89297654306364, 20.038978012250592, 20.313830324281653, 20.45555250947324, 20.42702968877587, 20.59703199812237, 20.71327761985891, 21.04386370163555, 21.38084652513642, 21.7428066207934, 22.075867832310294, 22.87151525865644, 23.117758907626012, 23.813262524787437, 24.155233510947998, 24.626063276937437, 25.17471830990892, 25.4095138392739, 25.612713333557746, 26.185164554201812, 26.39698517133162, 26.682093795014616, 26.950650474147196, 26.989568494530463, 26.95414830580205, 26.88757111564729, 26.804732297512558, 26.663932165812867, 26.70586878293465, 26.714949265764194, 26.713480489152047, 26.70597149667569, 26.689767315089536, 26.689767315089536, 26.626775405876963, 26.831377797623198, 26.903028861707114, 27.139224486050466, 27.09952508171018, 27.042679492760442, 27.11382368421616, 27.284290007283502, 27.058095803630856, 26.910284926364405, 27.001009067886244], "calls": [0.0, 0.16644474034620504, 1.0985352862849533, 1.9307589880159786, 3.095872170439414, 3.6284953395472703, 3.894806924101198, 4.227696404793608, 4.860186418109188, 5.459387483355526, 6.058588548601864, 7.157123834886818, 8.055925432756325, 9.55392809587217, 10.652463382157123, 12.51664447403462, 13.448735019973368, 14.047936085219707, 14.880159786950733, 15.84553928095872, 17.0439414114514, 18.375499334221036, 18.74167776298269, 19.873501997336884, 20.805592543275633, 21.6711051930759, 22.603195739014648, 23.435419440745672, 25.19973368841545, 27.296937416777627, 28.62849533954727, 29.82689747003995, 31.458055925432753, 32.856191744340876, 34.02130492676431, 34.720372836218374, 36.01864181091877, 36.95073235685752, 38.48202396804261, 39.78029294274301, 40.71238348868176, 41.61118508655127, 42.743009320905465, 43.37549933422104, 44.50732356857524, 45.372836218375504, 47.53661784287617, 48.435419440745676, 49.167776298268976, 50.466045272969374, 51.33155792276965, 52.197070572569906, 52.962716378162455, 53.26231691078562, 54.36085219707058, 55.52596537949401, 56.89081225033289, 57.78961384820239, 59.12117177097203, 59.986684420772306, 60.918774966711055, 61.65113182423435, 62.51664447403462, 63.61517976031957, 64.51398135818907, 65.24633821571238, 66.94407456724367, 68.10918774966711, 69.0745672436751, 70.47270306258322, 71.40479360852197, 72.4034620505992, 73.90146471371504, 74.56724367509987, 75.36617842876166, 76.83089214380826, 77.46338215712385, 78.19573901464713, 78.76165113182424, 79.06125166444741, 80.25965379494008, 80.7589880159787, 81.85752330226364, 82.58988015978696, 83.88814913448735, 85.18641810918774, 86.55126498002663, 87.05059920106525, 88.11584553928095, 88.78162450066579, 89.71371504660452, 91.27829560585884, 92.97603195739015, 93.37549933422103, 94.00798934753661, 94.8069241011984, 96.43808255659121, 96.90412782956058, 98.16910785619174, 99.00133155792277, 100.0], "rescued": [0, 13, 92, 162, 255, 294, 316, 342, 395, 443, 490, 572, 635, 723, 787, 882, 928, 962, 1011, 1066, 1129, 1204, 1222, 1285, 1333, 1377, 1429, 1472, 1566, 1675, 1739, 1805, 1885, 1954, 2011, 2048, 2111, 2157, 2231, 2288, 2337, 2377, 2425, 2458, 2507, 2540, 2617, 2653, 2677, 2719, 2741, 2762, 2781, 2791, 2812, 2838, 2873, 2897, 2930, 2950, 2973, 2993, 3012, 3039, 3062, 3084, 3124, 3154, 3181, 3219, 3246, 3277, 3321, 3338, 3361, 3402, 3417, 3433, 3444, 3446, 3447, 3447, 3447, 3447, 3448, 3449, 3450, 3450, 3451, 3451, 3451, 3453, 3455, 3456, 3456, 3456, 3459, 3460, 3460, 3460, 3461], "harmed": [0, 2, 9, 20, 29, 34, 35, 37, 45, 53, 56, 70, 78, 101, 115, 135, 144, 150, 156, 166, 173, 183, 185, 194, 197, 202, 209, 215, 220, 229, 236, 240, 244, 248, 259, 264, 276, 279, 290, 302, 306, 310, 319, 324, 331, 334, 344, 349, 350, 356, 362, 364, 368, 369, 380, 386, 394, 399, 408, 415, 423, 425, 432, 436, 443, 447, 460, 467, 474, 480, 491, 497, 507, 510, 523, 537, 546, 552, 553, 554, 557, 561, 567, 576, 580, 586, 589, 590, 595, 595, 600, 608, 614, 616, 620, 624, 631, 636, 648, 663, 676], "exact_score": [7.8873978710439285, 9.34614471559377, 10.412031466827372, 10.826805512421572, 11.219688984504065, 11.012301445487296, 11.334871437086408, 11.859533252511065, 12.131526994276498, 12.182195939409397, 12.979805746363837, 12.669176230070962, 12.620005704488843, 12.752567279744142, 12.697989241385258, 13.127140631992479, 13.00900712353573, 13.00914268685442, 13.188798794253028, 13.439356153392925, 13.408362067233295, 13.656569218531923, 13.709536802495597, 13.851997264123241, 13.935221398442154, 14.016516693351317, 14.091408790514482, 14.194088799292814, 14.190991830121938, 14.542687864312976, 14.698471465743095, 15.148318933021912, 15.294532955813159, 15.55397025822232, 15.599122027649889, 15.69355726690523, 15.7806256339094, 15.962602892962671, 16.117339908090482, 16.201826437513844, 16.350611136325327, 16.43257150345843, 16.60991339541927, 17.061421173245645, 17.41511069699801, 17.48556803494466, 17.671767554413904, 17.854477161453687, 18.41005163409505, 18.71684780347736, 18.984401921177966, 18.733648938741307, 18.859689240009953, 19.367515373826734, 19.55644809205553, 19.64092099089235, 19.759689234353317, 19.91736886995124, 20.175024972731944, 20.278823752650876, 20.45555250947324, 20.43631966886877, 20.664866770947604, 20.80397381311118, 21.281280867634923, 21.512018653781585, 21.840595432957194, 22.08768816327956, 22.834723270394967, 23.136614415273076, 23.505526865419085, 23.964898684887633, 24.357278939320032, 24.79603755381875, 25.206350679208146, 25.550242192386573, 25.87992345785018, 26.224319946321074, 26.6041422483153, 26.989568494530463, 26.98205664892778, 26.88757111564729, 26.804732297512558, 26.631272688921076, 26.70586878293465, 26.714949265764194, 26.69907927274699, 26.70597149667569, 26.689767315089536, 26.689767315089536, 26.59611697568256, 26.87439642373229, 26.940820582807206, 26.903028861707114, 27.09952508171018, 27.0467032967033, 27.083462349169157, 27.284290007283502, 27.10649411599102, 26.917128572913935, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 19.179683420756408, "threshold_area100": 19.1633417504559, "threshold_area25": 11.823746710794934}, "selected_budget": 83, "selected": {"val": {"score": 27.318166973339387, "calls": 82.88159771754636, "rescued": 814, "harmed": 148}, "test": {"score": 26.663932165812867, "calls": 82.58988015978696, "rescued": 3447, "harmed": 576}}}, "rescue_harm_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 2.3957366943359375, "operator": "ge"}, {"budget": 0.02, "threshold": 2.2695231437683105, "operator": "ge"}, {"budget": 0.03, "threshold": 2.230997323989868, "operator": "ge"}, {"budget": 0.04, "threshold": 2.190727710723877, "operator": "ge"}, {"budget": 0.05, "threshold": 2.157111406326294, "operator": "ge"}, {"budget": 0.06, "threshold": 2.1332497596740723, "operator": "ge"}, {"budget": 0.07, "threshold": 2.11081600189209, "operator": "ge"}, {"budget": 0.08, "threshold": 2.0937349796295166, "operator": "ge"}, {"budget": 0.09, "threshold": 2.080495595932007, "operator": "ge"}, {"budget": 0.1, "threshold": 2.0570149421691895, "operator": "ge"}, {"budget": 0.11, "threshold": 2.0344552993774414, "operator": "ge"}, {"budget": 0.12, "threshold": 2.02536678314209, "operator": "ge"}, {"budget": 0.13, "threshold": 1.9969736337661743, "operator": "ge"}, {"budget": 0.14, "threshold": 1.9824767112731934, "operator": "ge"}, {"budget": 0.15, "threshold": 1.9695277214050293, "operator": "ge"}, {"budget": 0.16, "threshold": 1.9476834535598755, "operator": "ge"}, {"budget": 0.17, "threshold": 1.9344542026519775, "operator": "ge"}, {"budget": 0.18, "threshold": 1.9259600639343262, "operator": "ge"}, {"budget": 0.19, "threshold": 1.9092966318130493, "operator": "ge"}, {"budget": 0.2, "threshold": 1.8947861194610596, "operator": "ge"}, {"budget": 0.21, "threshold": 1.8885055780410767, "operator": "ge"}, {"budget": 0.22, "threshold": 1.8754878044128418, "operator": "ge"}, {"budget": 0.23, "threshold": 1.860732078552246, "operator": "ge"}, {"budget": 0.24, "threshold": 1.8455649614334106, "operator": "ge"}, {"budget": 0.25, "threshold": 1.8337652683258057, "operator": "ge"}, {"budget": 0.26, "threshold": 1.8217251300811768, "operator": "ge"}, {"budget": 0.27, "threshold": 1.8094710111618042, "operator": "ge"}, {"budget": 0.28, "threshold": 1.803640604019165, "operator": "ge"}, {"budget": 0.29, "threshold": 1.798583984375, "operator": "ge"}, {"budget": 0.3, "threshold": 1.7883992195129395, "operator": "ge"}, {"budget": 0.31, "threshold": 1.7747349739074707, "operator": "ge"}, {"budget": 0.32, "threshold": 1.766329288482666, "operator": "ge"}, {"budget": 0.33, "threshold": 1.752553939819336, "operator": "ge"}, {"budget": 0.34, "threshold": 1.7393290996551514, "operator": "ge"}, {"budget": 0.35, "threshold": 1.725762128829956, "operator": "ge"}, {"budget": 0.36, "threshold": 1.7197058200836182, "operator": "ge"}, {"budget": 0.37, "threshold": 1.7129018306732178, "operator": "ge"}, {"budget": 0.38, "threshold": 1.7062160968780518, "operator": "ge"}, {"budget": 0.39, "threshold": 1.6957290172576904, "operator": "ge"}, {"budget": 0.4, "threshold": 1.6879730224609375, "operator": "ge"}, {"budget": 0.41, "threshold": 1.6762678623199463, "operator": "ge"}, {"budget": 0.42, "threshold": 1.6619129180908203, "operator": "ge"}, {"budget": 0.43, "threshold": 1.6482305526733398, "operator": "ge"}, {"budget": 0.44, "threshold": 1.6339011192321777, "operator": "ge"}, {"budget": 0.45, "threshold": 1.626000165939331, "operator": "ge"}, {"budget": 0.46, "threshold": 1.6171929836273193, "operator": "ge"}, {"budget": 0.47, "threshold": 1.5986146926879883, "operator": "ge"}, {"budget": 0.48, "threshold": 1.586938500404358, "operator": "ge"}, {"budget": 0.49, "threshold": 1.5731332302093506, "operator": "ge"}, {"budget": 0.5, "threshold": 1.5553072690963745, "operator": "ge"}, {"budget": 0.51, "threshold": 1.5343669652938843, "operator": "ge"}, {"budget": 0.52, "threshold": 1.5217207670211792, "operator": "ge"}, {"budget": 0.53, "threshold": 1.5097702741622925, "operator": "ge"}, {"budget": 0.54, "threshold": 1.488913655281067, "operator": "ge"}, {"budget": 0.55, "threshold": 1.4669792652130127, "operator": "ge"}, {"budget": 0.56, "threshold": 1.4609363079071045, "operator": "ge"}, {"budget": 0.57, "threshold": 1.4337115287780762, "operator": "ge"}, {"budget": 0.58, "threshold": 1.4082690477371216, "operator": "ge"}, {"budget": 0.59, "threshold": 1.3845257759094238, "operator": "ge"}, {"budget": 0.6, "threshold": 1.3368269205093384, "operator": "ge"}, {"budget": 0.61, "threshold": 1.295599341392517, "operator": "ge"}, {"budget": 0.62, "threshold": 1.2613389492034912, "operator": "ge"}, {"budget": 0.63, "threshold": 1.224048137664795, "operator": "ge"}, {"budget": 0.64, "threshold": 1.188956618309021, "operator": "ge"}, {"budget": 0.65, "threshold": 1.158361792564392, "operator": "ge"}, {"budget": 0.66, "threshold": 1.1246742010116577, "operator": "ge"}, {"budget": 0.67, "threshold": 1.0900170803070068, "operator": "ge"}, {"budget": 0.68, "threshold": 1.0594978332519531, "operator": "ge"}, {"budget": 0.69, "threshold": 0.9974542856216431, "operator": "ge"}, {"budget": 0.7, "threshold": 0.9204879999160767, "operator": "ge"}, {"budget": 0.71, "threshold": 0.8226309418678284, "operator": "ge"}, {"budget": 0.72, "threshold": 0.769464910030365, "operator": "ge"}, {"budget": 0.73, "threshold": 0.690574049949646, "operator": "ge"}, {"budget": 0.74, "threshold": 0.6121153235435486, "operator": "ge"}, {"budget": 0.75, "threshold": 0.5276080965995789, "operator": "ge"}, {"budget": 0.76, "threshold": 0.4301804304122925, "operator": "ge"}, {"budget": 0.77, "threshold": 0.3688187599182129, "operator": "ge"}, {"budget": 0.78, "threshold": 0.2119354009628296, "operator": "ge"}, {"budget": 0.79, "threshold": 0.09403067827224731, "operator": "ge"}, {"budget": 0.8, "threshold": -0.23619961738586426, "operator": "ge"}, {"budget": 0.81, "threshold": -0.392531156539917, "operator": "ge"}, {"budget": 0.82, "threshold": -0.4248359203338623, "operator": "ge"}, {"budget": 0.83, "threshold": -0.523636519908905, "operator": "ge"}, {"budget": 0.84, "threshold": -0.5787290930747986, "operator": "ge"}, {"budget": 0.85, "threshold": -0.6203084588050842, "operator": "ge"}, {"budget": 0.86, "threshold": -0.6402263641357422, "operator": "ge"}, {"budget": 0.87, "threshold": -0.7023704648017883, "operator": "ge"}, {"budget": 0.88, "threshold": -0.7378014922142029, "operator": "ge"}, {"budget": 0.89, "threshold": -0.7848295569419861, "operator": "ge"}, {"budget": 0.9, "threshold": -0.811161458492279, "operator": "ge"}, {"budget": 0.91, "threshold": -0.829643964767456, "operator": "ge"}, {"budget": 0.92, "threshold": -0.8785548210144043, "operator": "ge"}, {"budget": 0.93, "threshold": -0.8990645408630371, "operator": "ge"}, {"budget": 0.94, "threshold": -0.920975387096405, "operator": "ge"}, {"budget": 0.95, "threshold": -0.9495043158531189, "operator": "ge"}, {"budget": 0.96, "threshold": -0.9697915315628052, "operator": "ge"}, {"budget": 0.97, "threshold": -0.9951140880584717, "operator": "ge"}, {"budget": 0.98, "threshold": -1.0260932445526123, "operator": "ge"}, {"budget": 0.99, "threshold": -1.1248996257781982, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 11.039144899837192, 10.929507062243015, 10.74250612563926, 10.448761048248906, 10.369775920764857, 10.007622471437411, 10.00483634246102, 9.843796177129512, 11.638549853587103, 11.727867742398098, 11.690653001048066, 13.231543104782538, 13.18616153489372, 13.154279214787307, 13.246961584225733, 13.360644532571584, 13.0643775651941, 13.017127753246568, 12.966527664187764, 12.905465152075127, 12.907239652337116, 12.877715700967626, 12.925449991667376, 12.827536811826178, 12.951488151324803, 12.994585692783708, 13.438883632807705, 14.502497069765072, 14.677947069154449, 14.796479441621482, 15.892834807248398, 16.033041130273627, 16.23323792742314, 16.281058224816643, 16.296983798099742, 16.31595029315994, 16.38659955743188, 17.430892482779274, 18.418075124243625, 18.178426543031005, 19.007608401449613, 18.97194679133628, 19.17733539021129, 19.35415311583536, 19.272193818266324, 19.21424218965331, 19.267156037743078, 19.493500902082992, 19.47714281022985, 19.665692174109743, 19.58769472701361, 19.78390028278835, 20.049967009851315, 20.84746695134289, 21.10126613267862, 21.900404594591205, 22.019459298871062, 22.284699344447773, 22.600176617305397, 21.92982730438153, 22.156100556722212, 22.229867296685494, 22.49690734399083, 22.659074671458573, 22.974307602168004, 23.201690821256037, 23.37634163507526, 23.731884057971016, 23.89183229146226, 24.18078700824939, 24.451704793417285, 24.719142645971917, 24.999724427092538, 25.348662741919348, 25.538461538461537, 25.849800380347855, 26.112168451322415, 26.336865218663103, 26.62633944440017, 27.163354321263473, 27.163354321263473, 27.202291172879413, 27.202291172879413, 27.22115247468264, 27.086199708150932, 26.831103678929768, 26.699351373264417, 26.613163051781978, 26.613163051781978, 26.359924400320235, 26.280601290854342, 26.24189161343499, 26.24189161343499, 26.129391613434986, 26.065678835110745, 26.065678835110745, 25.905251479778073, 25.80694862366996, 25.656083007515633, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 28, 39, 49, 60, 74, 88, 103, 116, 128, 142, 159, 173, 187, 200, 213, 225, 234, 243, 260, 271, 285, 299, 308, 319, 331, 343, 358, 372, 381, 394, 404, 418, 428, 442, 449, 458, 472, 482, 494, 505, 514, 530, 543, 554, 562, 569, 578, 584, 594, 602, 613, 623, 635, 648, 658, 670, 677, 686, 694, 702, 709, 717, 725, 733, 739, 744, 751, 757, 764, 771, 776, 782, 789, 793, 798, 804, 808, 811, 813, 813, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 1, 2, 4, 6, 7, 11, 12, 15, 16, 16, 17, 18, 19, 21, 21, 22, 25, 26, 27, 29, 30, 32, 34, 37, 37, 39, 39, 41, 41, 42, 43, 43, 43, 44, 46, 47, 48, 48, 48, 53, 54, 56, 57, 59, 62, 66, 67, 70, 73, 74, 80, 82, 83, 85, 87, 88, 91, 98, 99, 103, 105, 107, 108, 111, 112, 113, 114, 118, 121, 123, 126, 127, 129, 131, 136, 137, 141, 143, 144, 144, 144, 150, 150, 153, 156, 159, 161, 163, 163, 168, 170, 171, 171, 174, 175, 175, 178, 180, 182, 185], "exact_score": [7.641657905799261, 11.039144899837192, 10.929507062243015, 10.74250612563926, 10.448761048248906, 10.369775920764857, 10.007622471437411, 10.00483634246102, 9.843796177129512, 11.638549853587103, 11.727867742398098, 11.690653001048066, 13.231543104782538, 13.18616153489372, 13.154279214787307, 13.246961584225733, 13.360644532571584, 13.0643775651941, 13.017127753246568, 12.966527664187764, 12.905465152075127, 12.907239652337116, 12.877715700967626, 12.925449991667376, 12.827536811826178, 12.951488151324803, 12.994585692783708, 13.438883632807705, 14.502497069765072, 14.677947069154449, 14.796479441621482, 15.892834807248398, 16.033041130273627, 16.23323792742314, 16.281058224816643, 16.296983798099742, 16.31595029315994, 16.38659955743188, 17.430892482779274, 18.418075124243625, 18.178426543031005, 19.007608401449613, 18.97194679133628, 19.17733539021129, 19.35415311583536, 19.272193818266324, 19.21424218965331, 19.267156037743078, 19.493500902082992, 19.47714281022985, 19.665692174109743, 19.58769472701361, 19.78390028278835, 20.049967009851315, 20.84746695134289, 21.10126613267862, 21.900404594591205, 22.019459298871062, 22.284699344447773, 22.600176617305397, 21.92982730438153, 22.156100556722212, 22.229867296685494, 22.49690734399083, 22.659074671458573, 22.974307602168004, 23.201690821256037, 23.37634163507526, 23.731884057971016, 23.89183229146226, 24.18078700824939, 24.451704793417285, 24.719142645971917, 24.999724427092538, 25.348662741919348, 25.538461538461537, 25.849800380347855, 26.112168451322415, 26.336865218663103, 26.62633944440017, 27.163354321263473, 27.163354321263473, 27.202291172879413, 27.202291172879413, 27.22115247468264, 27.086199708150932, 26.831103678929768, 26.699351373264417, 26.613163051781978, 26.613163051781978, 26.359924400320235, 26.280601290854342, 26.24189161343499, 26.24189161343499, 26.129391613434986, 26.065678835110745, 26.065678835110745, 25.905251479778073, 25.80694862366996, 25.656083007515633, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.527850797805076, "threshold_area100": 19.519244363780487, "threshold_area25": 11.709328685605168}, "test": {"score": [7.8873978710439285, 7.8767238365262155, 7.998759411422404, 8.75988938738311, 8.80314865694507, 9.523254134569013, 9.550801028823766, 10.257702121134708, 10.313598797681077, 11.464325243665934, 11.388886373175811, 11.42244566477335, 11.437270429698065, 11.826918570324485, 12.233167699108526, 12.20087163393764, 12.235300162273692, 12.313278847281078, 12.348756249924431, 12.724357847937137, 12.655193790236074, 12.701295878127322, 12.6059326567422, 12.578516015949228, 12.98156654710329, 12.959772976420119, 13.616295848853987, 14.03655059395324, 14.039259863720927, 13.963306168570465, 13.943995446970595, 13.967100988233117, 14.166062338254463, 14.231512002456906, 14.546507792722643, 14.660939860443198, 14.671620270844443, 14.877167007281708, 15.093314738488282, 15.284312534636776, 15.482946939987155, 15.662643651306155, 15.734437783392782, 15.976703598241759, 16.01544335853867, 16.119261231456427, 16.363803847355157, 16.62067526894538, 16.581316597207536, 16.796905760481305, 17.25955755730341, 17.632022423412007, 17.968378422923877, 18.238544829682006, 18.54324554066289, 18.713984553426105, 18.795715340068494, 19.11816751756207, 19.613150168285593, 20.06132212818844, 20.69713194515514, 21.214050561648744, 21.508416015017037, 21.931500580249786, 22.082589306779965, 22.53481048328782, 22.866851151513465, 23.24778239743716, 23.58358412874519, 24.02329658279168, 24.333662067745927, 25.0184103677193, 25.292580535297827, 25.425134478960675, 25.619443121378104, 25.829589742621334, 26.20076879262902, 26.31049854318129, 26.722459884019607, 26.961660151404732, 26.94509805563471, 26.83760304111116, 26.81375439623221, 26.784164200848476, 26.73676620788535, 26.623037197138444, 26.653654261333156, 26.70101562847409, 26.887410896650948, 27.075903483983808, 26.986359517295575, 26.9528350865327, 26.965797241619004, 26.918754711429173, 26.885944946051453, 27.14310802769132, 27.153787042017697, 27.1129158412921, 27.1172976899382, 26.935230312096976, 27.001009067886244], "calls": [0.0, 0.9320905459387484, 2.3635153129161117, 2.96271637816245, 3.761651131824234, 4.394141145139813, 5.226364846870839, 6.191744340878828, 6.790945406125166, 7.456724367509987, 8.488681757656458, 9.487350199733688, 9.953395472703063, 11.617842876165113, 12.549933422103862, 13.21571238348868, 14.414114513981357, 15.545938748335553, 16.145139813581892, 17.44340878828229, 18.908122503328894, 19.340878828229027, 20.505992010652463, 21.97070572569907, 23.33555259653795, 24.33422103861518, 25.39946737683089, 26.564580559254324, 27.030625832223702, 27.62982689747004, 28.66178428761651, 30.126498002663116, 31.19174434087883, 32.52330226364847, 34.12117177097203, 35.61917443408788, 36.35153129161118, 36.98402130492676, 37.949400798934754, 39.01464713715047, 39.61384820239681, 40.77896138482024, 42.2103861517976, 43.641810918774965, 44.77363515312916, 45.572569906790946, 46.17177097203729, 47.70306258322237, 48.30226364846871, 49.06790945406125, 50.499334221038616, 51.83089214380826, 52.762982689747005, 53.42876165113183, 54.460719041278296, 55.426098535286286, 55.85885486018641, 57.290279627163784, 58.655126498002666, 59.42077230359521, 60.98535286284953, 62.54993342210386, 63.48202396804261, 64.68042609853528, 65.41278295605859, 66.6444740346205, 67.54327563249002, 68.40878828229027, 69.10785619174435, 70.5392809587217, 71.70439414114514, 73.16910785619174, 73.90146471371504, 74.53395472703063, 75.36617842876166, 76.03195739014647, 77.16378162450066, 77.56324900133156, 78.4953395472703, 79.09454061251664, 80.12649800266311, 81.29161118508655, 81.55792276964048, 82.78961384820239, 83.62183754993342, 84.88681757656458, 85.35286284953395, 87.01731025299601, 88.11584553928095, 89.58055925432757, 90.37949400798935, 91.01198402130493, 92.47669773635153, 93.30892143808256, 94.00798934753661, 94.60719041278296, 95.27296937416777, 95.97203728362183, 96.6711051930759, 98.60186418109188, 100.0], "rescued": [0, 45, 127, 158, 199, 234, 274, 330, 361, 393, 446, 492, 519, 592, 637, 674, 736, 793, 824, 894, 968, 989, 1047, 1117, 1186, 1229, 1278, 1341, 1362, 1387, 1432, 1509, 1560, 1619, 1695, 1777, 1813, 1842, 1892, 1940, 1970, 2016, 2081, 2144, 2192, 2227, 2254, 2327, 2347, 2382, 2441, 2505, 2550, 2585, 2630, 2672, 2687, 2755, 2806, 2840, 2907, 2965, 3004, 3048, 3065, 3107, 3137, 3163, 3185, 3228, 3257, 3295, 3317, 3332, 3354, 3372, 3400, 3410, 3435, 3445, 3447, 3447, 3447, 3449, 3449, 3449, 3450, 3453, 3454, 3455, 3455, 3455, 3456, 3456, 3456, 3458, 3459, 3459, 3460, 3460, 3461], "harmed": [0, 7, 8, 10, 12, 13, 15, 18, 18, 21, 27, 29, 30, 42, 47, 51, 58, 60, 61, 67, 74, 74, 82, 93, 98, 106, 110, 114, 116, 122, 129, 142, 151, 158, 167, 176, 183, 189, 194, 206, 207, 214, 227, 241, 253, 256, 262, 276, 279, 286, 300, 311, 317, 326, 334, 342, 344, 356, 365, 372, 388, 404, 413, 427, 434, 444, 447, 451, 455, 466, 476, 488, 494, 502, 512, 520, 534, 541, 552, 554, 557, 565, 567, 577, 581, 589, 590, 601, 607, 612, 619, 621, 630, 634, 636, 638, 639, 642, 648, 661, 676], "exact_score": [7.8873978710439285, 7.883365135935739, 7.972067790314169, 8.763127281916317, 8.819900529281036, 9.534610302351862, 10.23652719128938, 11.468375359189253, 11.471871769793784, 11.428097852705763, 11.441502685025123, 11.91067976673795, 11.820469924204122, 12.219108466895797, 12.250605725946656, 12.254370622482993, 12.336082370914033, 12.709459081118865, 12.663904395699241, 12.660514549397547, 12.643261646373603, 12.635654307468847, 12.578516015949228, 12.938015740038422, 12.98672969149236, 13.314433473728293, 13.650386010105805, 14.035930828426094, 13.92915855274017, 13.904496145095996, 13.983559842994303, 14.15191071072845, 14.195107322054302, 14.266474179381897, 14.529766059698574, 14.58398980085942, 14.644771803250903, 14.877167007281708, 15.098601450929966, 15.29358255962611, 15.498700936845268, 15.66040461112481, 15.699833704310512, 15.955822182509255, 16.008972064468136, 16.0398582583954, 16.35408269773825, 16.57799586291013, 16.65059111061916, 16.78617579226191, 17.164182878397927, 17.34463357248339, 17.66143385987732, 18.28003753003993, 18.472064312219906, 18.635700981485385, 18.84841786728383, 18.913283740887195, 19.372189642188154, 19.66403307187077, 20.362271400094187, 20.69713194515514, 20.917585925355002, 21.320585077183978, 21.623698851836934, 21.927552038199337, 22.222726633829094, 22.64507429455404, 23.07572232247794, 23.54465122845893, 23.90959385103152, 24.03740050320082, 24.423228455289443, 24.948466974459052, 25.306605712612694, 25.487510667125758, 25.812165583566276, 26.129854697747813, 26.466452044738297, 26.906738920012906, 26.989568494530463, 26.84502720046788, 26.75678565474376, 26.784164200848476, 26.72935550683303, 26.653654261333156, 26.642810848790493, 26.70101562847409, 26.910625678586513, 26.871459956693037, 27.02062463214879, 26.9528350865327, 26.993974570944896, 26.951862271479655, 26.885944946051453, 27.14310802769132, 27.1129158412921, 27.1172976899382, 26.967332896536433, 27.104949495361915, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 18.61289908847531, "threshold_area100": 18.66636768027327, "threshold_area25": 11.238358271130114}, "selected_budget": 84, "selected": {"val": {"score": 27.22115247468264, "calls": 83.88017118402283, "rescued": 815, "harmed": 153}, "test": {"score": 26.73676620788535, "calls": 83.62183754993342, "rescued": 3449, "harmed": 581}}}, "error_ratio": {"thresholds": [{"budget": 0.0, "threshold": null, "operator": "never"}, {"budget": 0.01, "threshold": 1.514380693435669, "operator": "ge"}, {"budget": 0.02, "threshold": 1.4867349863052368, "operator": "ge"}, {"budget": 0.03, "threshold": 1.447588562965393, "operator": "ge"}, {"budget": 0.04, "threshold": 1.4347405433654785, "operator": "ge"}, {"budget": 0.05, "threshold": 1.4185665845870972, "operator": "ge"}, {"budget": 0.06, "threshold": 1.4090664386749268, "operator": "ge"}, {"budget": 0.07, "threshold": 1.3952646255493164, "operator": "ge"}, {"budget": 0.08, "threshold": 1.375082015991211, "operator": "ge"}, {"budget": 0.09, "threshold": 1.3702161312103271, "operator": "ge"}, {"budget": 0.1, "threshold": 1.3465795516967773, "operator": "ge"}, {"budget": 0.11, "threshold": 1.334604263305664, "operator": "ge"}, {"budget": 0.12, "threshold": 1.3235678672790527, "operator": "ge"}, {"budget": 0.13, "threshold": 1.3133544921875, "operator": "ge"}, {"budget": 0.14, "threshold": 1.3062819242477417, "operator": "ge"}, {"budget": 0.15, "threshold": 1.2982091903686523, "operator": "ge"}, {"budget": 0.16, "threshold": 1.2897844314575195, "operator": "ge"}, {"budget": 0.17, "threshold": 1.2832529544830322, "operator": "ge"}, {"budget": 0.18, "threshold": 1.2776025533676147, "operator": "ge"}, {"budget": 0.19, "threshold": 1.2701297998428345, "operator": "ge"}, {"budget": 0.2, "threshold": 1.2635643482208252, "operator": "ge"}, {"budget": 0.21, "threshold": 1.2551167011260986, "operator": "ge"}, {"budget": 0.22, "threshold": 1.2508753538131714, "operator": "ge"}, {"budget": 0.23, "threshold": 1.2436556816101074, "operator": "ge"}, {"budget": 0.24, "threshold": 1.2393274307250977, "operator": "ge"}, {"budget": 0.25, "threshold": 1.2320356369018555, "operator": "ge"}, {"budget": 0.26, "threshold": 1.222440242767334, "operator": "ge"}, {"budget": 0.27, "threshold": 1.2183630466461182, "operator": "ge"}, {"budget": 0.28, "threshold": 1.2115458250045776, "operator": "ge"}, {"budget": 0.29, "threshold": 1.2059999704360962, "operator": "ge"}, {"budget": 0.3, "threshold": 1.1987354755401611, "operator": "ge"}, {"budget": 0.31, "threshold": 1.1917853355407715, "operator": "ge"}, {"budget": 0.32, "threshold": 1.187327265739441, "operator": "ge"}, {"budget": 0.33, "threshold": 1.1828827857971191, "operator": "ge"}, {"budget": 0.34, "threshold": 1.1770964860916138, "operator": "ge"}, {"budget": 0.35, "threshold": 1.170233964920044, "operator": "ge"}, {"budget": 0.36, "threshold": 1.1576820611953735, "operator": "ge"}, {"budget": 0.37, "threshold": 1.1513797044754028, "operator": "ge"}, {"budget": 0.38, "threshold": 1.1468095779418945, "operator": "ge"}, {"budget": 0.39, "threshold": 1.14365816116333, "operator": "ge"}, {"budget": 0.4, "threshold": 1.1392273902893066, "operator": "ge"}, {"budget": 0.41, "threshold": 1.1305205821990967, "operator": "ge"}, {"budget": 0.42, "threshold": 1.1249215602874756, "operator": "ge"}, {"budget": 0.43, "threshold": 1.1209518909454346, "operator": "ge"}, {"budget": 0.44, "threshold": 1.1115858554840088, "operator": "ge"}, {"budget": 0.45, "threshold": 1.1080081462860107, "operator": "ge"}, {"budget": 0.46, "threshold": 1.1019880771636963, "operator": "ge"}, {"budget": 0.47, "threshold": 1.083709478378296, "operator": "ge"}, {"budget": 0.48, "threshold": 1.0726128816604614, "operator": "ge"}, {"budget": 0.49, "threshold": 1.0604724884033203, "operator": "ge"}, {"budget": 0.5, "threshold": 1.053613543510437, "operator": "ge"}, {"budget": 0.51, "threshold": 1.0366973876953125, "operator": "ge"}, {"budget": 0.52, "threshold": 1.026667833328247, "operator": "ge"}, {"budget": 0.53, "threshold": 1.0160666704177856, "operator": "ge"}, {"budget": 0.54, "threshold": 1.011652946472168, "operator": "ge"}, {"budget": 0.55, "threshold": 0.9966540336608887, "operator": "ge"}, {"budget": 0.56, "threshold": 0.9794559478759766, "operator": "ge"}, {"budget": 0.57, "threshold": 0.9620364904403687, "operator": "ge"}, {"budget": 0.58, "threshold": 0.9507637619972229, "operator": "ge"}, {"budget": 0.59, "threshold": 0.9236412048339844, "operator": "ge"}, {"budget": 0.6, "threshold": 0.9077932238578796, "operator": "ge"}, {"budget": 0.61, "threshold": 0.8997998237609863, "operator": "ge"}, {"budget": 0.62, "threshold": 0.8855594396591187, "operator": "ge"}, {"budget": 0.63, "threshold": 0.8710206747055054, "operator": "ge"}, {"budget": 0.64, "threshold": 0.8395200967788696, "operator": "ge"}, {"budget": 0.65, "threshold": 0.8158515691757202, "operator": "ge"}, {"budget": 0.66, "threshold": 0.7903183102607727, "operator": "ge"}, {"budget": 0.67, "threshold": 0.7396942973136902, "operator": "ge"}, {"budget": 0.68, "threshold": 0.7299667596817017, "operator": "ge"}, {"budget": 0.69, "threshold": 0.7058785557746887, "operator": "ge"}, {"budget": 0.7, "threshold": 0.6807356476783752, "operator": "ge"}, {"budget": 0.71, "threshold": 0.6406855583190918, "operator": "ge"}, {"budget": 0.72, "threshold": 0.603944718837738, "operator": "ge"}, {"budget": 0.73, "threshold": 0.5372152328491211, "operator": "ge"}, {"budget": 0.74, "threshold": 0.4923349916934967, "operator": "ge"}, {"budget": 0.75, "threshold": 0.43077507615089417, "operator": "ge"}, {"budget": 0.76, "threshold": 0.3612598478794098, "operator": "ge"}, {"budget": 0.77, "threshold": 0.3075248897075653, "operator": "ge"}, {"budget": 0.78, "threshold": 0.18074765801429749, "operator": "ge"}, {"budget": 0.79, "threshold": 0.07920798659324646, "operator": "ge"}, {"budget": 0.8, "threshold": -0.20240157842636108, "operator": "ge"}, {"budget": 0.81, "threshold": -0.3323083519935608, "operator": "ge"}, {"budget": 0.82, "threshold": -0.36431193351745605, "operator": "ge"}, {"budget": 0.83, "threshold": -0.45405644178390503, "operator": "ge"}, {"budget": 0.84, "threshold": -0.49284401535987854, "operator": "ge"}, {"budget": 0.85, "threshold": -0.5343270897865295, "operator": "ge"}, {"budget": 0.86, "threshold": -0.5500978827476501, "operator": "ge"}, {"budget": 0.87, "threshold": -0.6201569437980652, "operator": "ge"}, {"budget": 0.88, "threshold": -0.6575062870979309, "operator": "ge"}, {"budget": 0.89, "threshold": -0.6770203113555908, "operator": "ge"}, {"budget": 0.9, "threshold": -0.7042049169540405, "operator": "ge"}, {"budget": 0.91, "threshold": -0.7214980721473694, "operator": "ge"}, {"budget": 0.92, "threshold": -0.7640591263771057, "operator": "ge"}, {"budget": 0.93, "threshold": -0.7849154472351074, "operator": "ge"}, {"budget": 0.94, "threshold": -0.8000345230102539, "operator": "ge"}, {"budget": 0.95, "threshold": -0.8272554874420166, "operator": "ge"}, {"budget": 0.96, "threshold": -0.8517398238182068, "operator": "ge"}, {"budget": 0.97, "threshold": -0.8782287240028381, "operator": "ge"}, {"budget": 0.98, "threshold": -0.9083074331283569, "operator": "ge"}, {"budget": 0.99, "threshold": -0.9958252310752869, "operator": "ge"}, {"budget": 1.0, "threshold": null, "operator": "always"}], "val": {"score": [7.641657905799261, 7.539609209663722, 7.645176818355988, 10.517778724812116, 10.22293096876917, 10.119945516920941, 9.946697809214488, 9.982693466300024, 9.99069438825962, 9.935389684176021, 9.923319400905607, 10.049736474012622, 10.019444269774397, 10.101495806470643, 10.19904089911928, 10.326317303876088, 10.469279174273026, 12.117374727668844, 11.954032809295967, 11.842639882347118, 11.830694184957533, 11.748015847255083, 11.805249834910853, 11.872946997103288, 12.001750786330478, 13.160106736555122, 13.311829747075768, 14.157238017501003, 14.180701100861674, 15.007189007189009, 15.096722996266376, 16.896379912547324, 17.063867327621445, 17.09287274091871, 17.131819765048604, 17.661804391126548, 17.675685234305927, 17.532280671608593, 17.72443358461567, 17.83427516259338, 17.779573367808663, 17.921813291244472, 18.050361994327513, 18.241956885078107, 18.375583220436518, 18.28438957196685, 18.5660000973094, 18.62959188045456, 18.851716029380885, 18.728720400783306, 18.940107050572166, 19.75481030713589, 20.03031864834211, 20.784410844341583, 20.893817525143156, 21.062763401473077, 21.848901098901095, 22.00732303052632, 22.08258212602376, 22.474517872527823, 22.887080814983953, 22.907360679646153, 23.180142477677084, 23.45952120860261, 23.57611849137273, 23.90868140868141, 24.089852267165043, 24.19694080652985, 24.455852503832755, 23.71756162517032, 24.12199026663768, 24.35892101795992, 24.59494611668525, 25.011286315747466, 25.348662741919348, 25.538461538461537, 25.849800380347855, 26.112168451322415, 26.336865218663103, 26.62633944440017, 27.163354321263473, 27.503066096083295, 27.202291172879413, 27.202291172879413, 27.22115247468264, 27.062920804969682, 26.951876625789673, 26.699351373264417, 26.655892007771154, 26.57114624505929, 26.319940393922796, 26.280601290854342, 26.24189161343499, 26.20379637533975, 26.129391613434986, 26.065678835110745, 26.065678835110745, 25.869473304465014, 25.80694862366996, 25.656083007515633, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "rescued": [0, 12, 29, 45, 64, 79, 91, 106, 119, 132, 146, 162, 175, 187, 198, 213, 228, 244, 258, 271, 285, 297, 308, 320, 331, 348, 362, 373, 380, 393, 401, 416, 427, 437, 446, 456, 467, 478, 488, 499, 510, 518, 528, 539, 546, 556, 568, 576, 584, 591, 604, 615, 624, 634, 640, 649, 658, 668, 673, 685, 695, 702, 711, 718, 724, 731, 737, 741, 749, 754, 763, 770, 775, 783, 789, 793, 798, 804, 808, 811, 813, 814, 814, 814, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815, 815], "harmed": [0, 2, 4, 5, 8, 10, 14, 15, 16, 18, 20, 20, 23, 23, 23, 23, 23, 23, 26, 29, 32, 36, 37, 38, 38, 39, 39, 43, 44, 46, 46, 46, 46, 49, 50, 50, 52, 56, 56, 58, 61, 61, 62, 66, 67, 70, 70, 71, 71, 79, 81, 82, 82, 84, 87, 92, 96, 99, 101, 102, 102, 105, 107, 108, 109, 113, 115, 117, 120, 123, 124, 127, 128, 130, 131, 136, 137, 141, 143, 144, 144, 145, 150, 150, 153, 156, 158, 161, 162, 164, 169, 170, 171, 172, 174, 175, 175, 179, 180, 182, 185], "exact_score": [7.641657905799261, 7.539609209663722, 7.645176818355988, 10.517778724812116, 10.22293096876917, 10.119945516920941, 9.946697809214488, 9.982693466300024, 9.99069438825962, 9.935389684176021, 9.923319400905607, 10.049736474012622, 10.019444269774397, 10.101495806470643, 10.19904089911928, 10.326317303876088, 10.469279174273026, 12.117374727668844, 11.954032809295967, 11.842639882347118, 11.830694184957533, 11.748015847255083, 11.805249834910853, 11.872946997103288, 12.001750786330478, 13.160106736555122, 13.311829747075768, 14.157238017501003, 14.180701100861674, 15.007189007189009, 15.096722996266376, 16.896379912547324, 17.063867327621445, 17.09287274091871, 17.131819765048604, 17.661804391126548, 17.675685234305927, 17.532280671608593, 17.72443358461567, 17.83427516259338, 17.779573367808663, 17.921813291244472, 18.050361994327513, 18.241956885078107, 18.375583220436518, 18.28438957196685, 18.5660000973094, 18.62959188045456, 18.851716029380885, 18.728720400783306, 18.940107050572166, 19.75481030713589, 20.03031864834211, 20.784410844341583, 20.893817525143156, 21.062763401473077, 21.848901098901095, 22.00732303052632, 22.08258212602376, 22.474517872527823, 22.887080814983953, 22.907360679646153, 23.180142477677084, 23.45952120860261, 23.57611849137273, 23.90868140868141, 24.089852267165043, 24.19694080652985, 24.455852503832755, 23.71756162517032, 24.12199026663768, 24.35892101795992, 24.59494611668525, 25.011286315747466, 25.348662741919348, 25.538461538461537, 25.849800380347855, 26.112168451322415, 26.336865218663103, 26.62633944440017, 27.163354321263473, 27.503066096083295, 27.202291172879413, 27.202291172879413, 27.22115247468264, 27.062920804969682, 26.951876625789673, 26.699351373264417, 26.655892007771154, 26.57114624505929, 26.319940393922796, 26.280601290854342, 26.24189161343499, 26.20379637533975, 26.129391613434986, 26.065678835110745, 26.065678835110745, 25.869473304465014, 25.80694862366996, 25.656083007515633, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_area100": 19.25594728812837, "threshold_area100": 19.246952420518532, "threshold_area25": 10.520231745567472}, "test": {"score": [7.8873978710439285, 7.916807829020112, 8.699909634347325, 9.434347419975154, 9.458160118520174, 9.522534263120997, 9.556746452131135, 9.564788594708402, 10.216520854174957, 10.191845243639563, 11.239862450096677, 11.240147047759521, 11.602233423665826, 11.628585876718526, 12.509661351300142, 12.409121562347366, 12.857628016846393, 12.797111435828088, 12.760902672819874, 13.547548307937415, 13.40131069982173, 14.05615013042904, 14.008893470151515, 14.023679651614513, 14.060144134009128, 14.07883791563003, 13.970502867576576, 13.860901806584197, 13.801951145132657, 13.860353396412965, 14.171387316861564, 14.448693486178698, 14.466386787708158, 14.527823798469269, 14.909706977603864, 15.196103970990304, 15.235609528551539, 15.287827639182927, 15.625459310859501, 15.65809922839091, 15.723043815296528, 15.762680472477756, 16.23992100756364, 16.299295321489833, 16.648322978589754, 16.736751386143872, 16.710338481011707, 17.016319993165023, 17.17822792040598, 17.55828912599318, 17.698203448274096, 17.99254176862912, 18.319391019586227, 18.41523000353824, 18.416328031607655, 18.884441605249503, 19.22706042661574, 19.64071099967969, 20.041226731312296, 20.542220175949005, 20.668047432625166, 20.912513534930056, 21.0987657853559, 21.124076212924162, 21.77438222890239, 22.571294998204305, 22.862034366793605, 23.622364343744305, 23.717740135474426, 24.357609120588805, 24.67095092839994, 25.05111732628642, 25.2567844498284, 25.422590458052134, 25.658640062619554, 25.853232298377332, 26.16330915085297, 26.33209786344205, 26.69760378773494, 26.961660151404732, 26.94509805563471, 26.83760304111116, 26.78053369215401, 26.784164200848476, 26.744205521683046, 26.653654261333156, 26.653654261333156, 26.66887314709765, 26.871459956693037, 27.110408267913506, 26.986359517295575, 26.986359517295575, 26.97983429945168, 26.92568426100845, 26.885944946051453, 26.897774777853776, 27.146957107389934, 27.070687962407707, 27.133058024777775, 26.92629299339533, 27.001009067886244], "calls": [0.0, 1.1318242343541944, 1.7976031957390146, 2.729693741677763, 2.9960053262316912, 3.6284953395472703, 3.96138482023968, 4.527296937416778, 5.459387483355526, 5.858854860186418, 7.623169107856191, 8.754993342210387, 9.55392809587217, 10.419440745672437, 11.051930758988016, 12.383488681757656, 13.348868175765647, 14.280958721704396, 15.07989347536618, 16.278295605858855, 17.41011984021305, 18.808255659121173, 19.40745672436751, 20.47270306258322, 21.50466045272969, 22.503328894806923, 24.400798934753663, 25.36617842876165, 26.864181091877498, 27.862849533954726, 29.19440745672437, 30.193075898801595, 30.825565912117174, 31.591211717709722, 32.52330226364847, 33.72170439414114, 35.685752330226364, 36.651131824234355, 37.416777629826896, 37.74966711051931, 38.68175765645805, 39.74700399467377, 40.645805592543276, 41.27829560585886, 42.676431424766974, 43.142476697736356, 43.74167776298269, 45.772303595206395, 47.20372836218375, 48.76830892143808, 49.70039946737683, 51.49800266311585, 52.49667110519307, 53.22902796271638, 53.52862849533955, 55.026631158455395, 56.757656458055926, 57.9560585885486, 58.62183754993342, 60.25299600532623, 61.15179760319573, 61.75099866844208, 62.51664447403462, 63.38215712383488, 65.04660452729694, 66.51131824234355, 67.41011984021304, 69.8069241011984, 70.07323568575234, 70.80559254327564, 71.83754993342211, 72.86950732356857, 73.7017310252996, 74.70039946737683, 75.53262316910786, 76.09853528628496, 77.03062583222369, 77.62982689747004, 78.46205059920106, 79.12782956058588, 80.12649800266311, 81.19174434087883, 81.62450066577897, 82.85619174434088, 83.38881491344874, 84.88681757656458, 85.18641810918774, 87.48335552596538, 88.48202396804261, 89.2809587217044, 90.27962716378163, 90.84553928095872, 92.34354194407457, 93.10918774966711, 93.87483355525966, 94.60719041278296, 95.40612516644474, 96.07190412782957, 96.77097203728363, 98.63515312916113, 100.0], "rescued": [0, 68, 107, 166, 182, 221, 241, 275, 323, 346, 441, 499, 552, 604, 641, 713, 767, 813, 858, 920, 979, 1057, 1088, 1145, 1200, 1248, 1339, 1390, 1457, 1501, 1569, 1615, 1642, 1679, 1724, 1783, 1869, 1914, 1951, 1964, 2007, 2052, 2090, 2113, 2172, 2195, 2218, 2302, 2361, 2427, 2465, 2524, 2570, 2594, 2604, 2663, 2735, 2784, 2809, 2868, 2900, 2920, 2945, 2970, 3030, 3081, 3113, 3192, 3200, 3221, 3252, 3286, 3314, 3336, 3359, 3374, 3397, 3412, 3434, 3445, 3447, 3447, 3447, 3449, 3449, 3450, 3450, 3453, 3454, 3455, 3455, 3455, 3456, 3456, 3456, 3457, 3459, 3459, 3460, 3460, 3461], "harmed": [0, 7, 9, 14, 14, 14, 14, 17, 19, 22, 31, 36, 42, 45, 46, 55, 58, 62, 66, 73, 82, 90, 94, 98, 104, 108, 126, 135, 147, 151, 160, 167, 171, 175, 181, 191, 209, 217, 224, 226, 235, 244, 254, 257, 264, 266, 271, 281, 293, 305, 310, 315, 324, 330, 332, 350, 367, 377, 382, 393, 399, 402, 407, 413, 428, 437, 442, 458, 459, 465, 474, 485, 495, 505, 516, 521, 532, 543, 552, 554, 557, 565, 569, 577, 580, 590, 590, 603, 608, 610, 619, 619, 628, 633, 636, 637, 640, 644, 647, 661, 676], "exact_score": [7.8873978710439285, 7.901949074690205, 8.705567442582783, 9.458160118520174, 9.561192104647255, 10.19794178763187, 10.18764511293078, 11.352519517667838, 11.273185675935324, 11.624416251915406, 11.601201462393803, 12.501294656670597, 12.461614305682476, 12.820477452959139, 12.801870937037965, 12.749515718759973, 13.13737256798383, 13.442105993669118, 13.7528842455326, 13.9900545372037, 14.046794475753726, 14.003266604551243, 14.038841612898, 14.030088174489821, 14.00134097301601, 13.87414874476726, 13.86313508511071, 13.814329253663265, 14.132104128449562, 14.15256593885445, 14.440985020184216, 14.490641471861773, 14.768097318542656, 14.945115613495808, 15.167070245558492, 15.21211640431305, 15.293614396718002, 15.272721178540422, 15.696654290262419, 15.686432123152946, 15.78560567331692, 16.256136721543736, 16.585837100536924, 16.7124374280733, 16.75017537417285, 16.926284847538614, 17.041895885059812, 17.119955586882213, 17.41889306320172, 17.62197050675045, 17.74852177766037, 17.956660137263402, 18.250299416267943, 18.385671551872296, 18.485665408771997, 18.884441605249503, 19.10283461642978, 19.404970824172622, 19.65521559441477, 20.385620393589846, 20.487347992655046, 20.62102889778362, 20.992875006961825, 21.20559944469904, 21.526647050581165, 21.745622002322236, 22.311889499389498, 22.70803967608455, 22.990309729168498, 23.352603050432265, 23.67972326023472, 24.37602990373825, 24.73088294802885, 25.07910576874362, 25.304894809086054, 25.49491879382666, 25.815367419262593, 26.14656344117713, 26.466452044738297, 26.906738920012906, 26.94509805563471, 26.84502720046788, 26.75678565474376, 26.784164200848476, 26.71316307240944, 26.653654261333156, 26.635512963717588, 26.724423050587802, 26.887410896650948, 26.871459956693037, 27.038607920024166, 26.986359517295575, 26.98689146247215, 26.939619231147798, 26.885944946051453, 27.16064167803653, 27.1129158412921, 27.11042955807007, 27.00648795220118, 26.865441491447267, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "exact_area100": 18.944995456128083, "threshold_area100": 18.87064456448432, "threshold_area25": 11.51118937374475}, "selected_budget": 81, "selected": {"val": {"score": 27.503066096083295, "calls": 80.88445078459344, "rescued": 814, "harmed": 145}, "test": {"score": 26.83760304111116, "calls": 81.19174434087883, "rescued": 3447, "harmed": 565}}}}, "baseline_policies": {"old_difference_interior": {"val": {"score": 27.318166973339387, "calls": 82.88159771754636}, "test": {"score": 26.663932165812867, "calls": 82.58988015978696}}, "old_normalized_interior": {"val": {"score": 27.367307022479437, "calls": 81.8830242510699}, "test": {"score": 26.76415220681962, "calls": 81.6577896138482}}, "old_normalized_endpoints": {"val": {"score": 27.367307022479437, "calls": 81.8830242510699}, "test": {"score": 26.76415220681962, "calls": 81.6577896138482}}}, "baseline_old_normalized": {"val": {"score": [7.641657905799261, 7.755221508351605, 7.678439532806242, 7.76745652721252, 7.883475751396031, 7.902368116608821, 7.908354181367202, 8.032329372754903, 8.163837923202498, 9.73378258726343, 9.7683744530542, 11.113117354339352, 12.384119164907343, 12.508174588395097, 12.53346039383334, 13.563314041370475, 13.66362483936227, 14.445601146729405, 15.362580064797221, 15.267307609150006, 15.377513896023157, 15.413036226261296, 15.42576899719757, 15.553907227573077, 16.402219116149848, 16.299198679576204, 16.34284869962099, 16.52118474297766, 17.42299932982817, 17.28703476592556, 17.272109139681298, 17.321062043228114, 17.29766926520493, 17.41772629013027, 17.47716730930326, 17.418626036065433, 17.530820293057754, 17.622854133908078, 17.817271522109245, 17.946602724084457, 18.10303419155535, 17.928986695209517, 18.179677740371382, 19.004465817751445, 19.743752160349676, 20.072976808983736, 20.332156945102128, 20.707113048662343, 20.783730158730158, 21.001607394012456, 21.750613464704116, 21.86195362665951, 22.01298701298701, 21.886642281379125, 22.087891800034786, 22.116772116772115, 22.23501343857381, 22.346781973755323, 22.448986668053607, 22.729933013638732, 22.044158160684468, 22.30924567056712, 22.54002985091888, 22.62188874389831, 22.60973512211902, 22.92795320279648, 23.108311838622434, 23.29786276627381, 23.50125313283208, 23.831521739130434, 24.030532878562134, 24.337403889642697, 24.607508105350178, 24.873694491720073, 25.27542561238038, 25.746911149447914, 25.965138380232723, 26.183292206656695, 26.336865218663103, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.367307022479437, 27.202291172879413, 27.31513743708866, 26.96605644546148, 26.854096989966557, 26.809119478722177, 26.721451448330598, 26.50621118012422, 26.400569299385403, 26.400569299385403, 26.29621162199599, 26.092798366499753, 25.953468770190106, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.879064008285347, 25.727093126457586, 25.448626222520154], "calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "exact_score": [7.641657905799261, 7.755221508351605, 7.678439532806242, 7.76745652721252, 7.883475751396031, 7.902368116608821, 7.908354181367202, 8.032329372754903, 8.163837923202498, 9.73378258726343, 9.7683744530542, 11.113117354339352, 12.384119164907343, 12.508174588395097, 12.53346039383334, 13.563314041370475, 13.66362483936227, 14.445601146729405, 15.362580064797221, 15.267307609150006, 15.377513896023157, 15.413036226261296, 15.42576899719757, 15.553907227573077, 16.402219116149848, 16.299198679576204, 16.34284869962099, 16.52118474297766, 17.42299932982817, 17.28703476592556, 17.272109139681298, 17.321062043228114, 17.29766926520493, 17.41772629013027, 17.47716730930326, 17.418626036065433, 17.530820293057754, 17.622854133908078, 17.817271522109245, 17.946602724084457, 18.10303419155535, 17.928986695209517, 18.179677740371382, 19.004465817751445, 19.743752160349676, 20.072976808983736, 20.332156945102128, 20.707113048662343, 20.783730158730158, 21.001607394012456, 21.750613464704116, 21.86195362665951, 22.01298701298701, 21.886642281379125, 22.087891800034786, 22.116772116772115, 22.23501343857381, 22.346781973755323, 22.448986668053607, 22.729933013638732, 22.044158160684468, 22.30924567056712, 22.54002985091888, 22.62188874389831, 22.60973512211902, 22.92795320279648, 23.108311838622434, 23.29786276627381, 23.50125313283208, 23.831521739130434, 24.030532878562134, 24.337403889642697, 24.607508105350178, 24.873694491720073, 25.27542561238038, 25.746911149447914, 25.965138380232723, 26.183292206656695, 26.336865218663103, 26.747112391260075, 27.163354321263473, 27.163354321263473, 27.367307022479437, 27.202291172879413, 27.31513743708866, 26.96605644546148, 26.854096989966557, 26.809119478722177, 26.721451448330598, 26.50621118012422, 26.400569299385403, 26.400569299385403, 26.29621162199599, 26.092798366499753, 25.953468770190106, 25.915973457104247, 25.915973457104247, 25.915973457104247, 25.879064008285347, 25.727093126457586, 25.448626222520154], "exact_calls": [0.0, 0.9985734664764622, 1.9971469329529243, 2.9957203994293864, 3.9942938659058487, 4.9928673323823105, 5.991440798858773, 6.990014265335236, 7.988587731811697, 8.98716119828816, 9.985734664764621, 10.984308131241084, 11.982881597717546, 12.981455064194009, 13.980028530670472, 14.978601997146935, 15.977175463623395, 16.975748930099858, 17.97432239657632, 18.972895863052784, 19.971469329529242, 20.970042796005707, 21.96861626248217, 22.96718972895863, 23.96576319543509, 24.964336661911553, 25.962910128388017, 26.961483594864475, 27.960057061340944, 28.9586305278174, 29.95720399429387, 30.955777460770328, 31.95435092724679, 32.95292439372325, 33.951497860199716, 34.950071326676174, 35.94864479315264, 36.947218259629096, 37.94579172610557, 38.944365192582026, 39.942938659058484, 40.94151212553495, 41.940085592011414, 42.93865905848788, 43.93723252496434, 44.935805991440795, 45.93437945791726, 46.932952924393724, 47.93152639087018, 48.93009985734665, 49.928673323823105, 50.92724679029958, 51.925820256776035, 52.92439372325249, 53.92296718972895, 54.92154065620542, 55.92011412268189, 56.918687589158345, 57.9172610556348, 58.91583452211127, 59.91440798858774, 60.9129814550642, 61.911554921540656, 62.910128388017114, 63.90870185449358, 64.90727532097004, 65.9058487874465, 66.90442225392297, 67.90299572039943, 68.90156918687589, 69.90014265335235, 70.89871611982882, 71.89728958630528, 72.89586305278173, 73.89443651925819, 74.89300998573466, 75.89158345221114, 76.8901569186876, 77.88873038516405, 78.88730385164051, 79.88587731811697, 80.88445078459344, 81.8830242510699, 82.88159771754636, 83.88017118402283, 84.87874465049929, 85.87731811697576, 86.87589158345222, 87.87446504992867, 88.87303851640513, 89.87161198288159, 90.87018544935806, 91.86875891583452, 92.86733238231099, 93.86590584878745, 94.8644793152639, 95.86305278174036, 96.86162624821684, 97.8601997146933, 98.85877318116975, 100.0], "threshold_area25": 11.716399759948878, "exact_area100": 19.88759021457664}, "test": {"score": [7.8873978710439285, 7.92025125665438, 10.022897001407612, 10.598405832861445, 10.935886124758946, 11.272390499708452, 11.282912581480646, 11.248693786339544, 11.17003033475677, 11.756164123660742, 11.64483700118651, 11.993138828864582, 11.915379266957459, 12.527241763917903, 12.50733038428111, 12.544501062033778, 12.307368205870524, 12.43749128551216, 13.03873186335737, 13.028282048156612, 13.299966157781205, 13.302967635470056, 13.527057792637518, 13.532391237558189, 13.547649409894852, 13.517398206374176, 13.707747213323518, 13.861386643001355, 13.925913619262145, 14.320019687096844, 14.567559904646984, 14.849487495804265, 15.116626615947137, 15.519675149262971, 15.672525831183895, 15.76853289670726, 16.11556163831169, 16.216461094265313, 16.327469039273225, 16.389935634243436, 16.545142620652896, 16.59901275738111, 16.919873258837534, 16.989631865387572, 17.20032381009755, 17.73991464150713, 17.83521823610966, 17.87596984854971, 18.155638642393004, 18.346538800999543, 18.476827734123262, 18.54132386859783, 18.711452243499462, 18.88716776122585, 19.27540334248122, 19.219051538362784, 19.600881981470618, 19.746281134767546, 19.838834755049056, 19.940336867119697, 20.3106077475395, 21.047283717293148, 21.44953968734262, 22.18601483624841, 22.412067778780248, 22.545960585322664, 23.02690106358149, 23.20854560052026, 23.419235164438625, 23.579607505708704, 24.015725653656688, 24.739155433607234, 25.243037991445632, 25.412845029840852, 25.827501374118523, 26.24933854152766, 26.358947128572453, 26.593668348635852, 26.76540338517288, 26.989568494530463, 26.937586210032027, 26.87696272623368, 26.76415220681962, 26.663932165812867, 26.674814954866104, 26.58674782919362, 26.693293741050077, 26.715062818028628, 26.70761098233252, 26.883550436364278, 26.861861445907437, 26.948143779905827, 27.17690106225915, 27.117738912078703, 27.039799102275826, 27.105679776846603, 27.083110821066615, 26.97763910098948, 26.980258366261644, 26.896670567331594, 27.001009067886244], "calls": [0.0, 0.3328894806924101, 1.4980026631158456, 1.9307589880159786, 2.929427430093209, 3.395472703062583, 3.7283621837549936, 4.194407456724368, 4.92676431424767, 5.59254327563249, 6.1584553928095875, 6.657789613848203, 7.589880159786951, 8.355525965379494, 9.021304926764314, 10.252996005326231, 12.649800266311583, 13.948069241011984, 15.745672436750999, 16.77762982689747, 17.7762982689747, 17.942743009320907, 18.608521970705727, 19.307589880159785, 20.70572569906791, 21.537949400798933, 23.03595206391478, 24.766977363515313, 25.798934753661783, 26.864181091877498, 27.363515312916114, 28.395472703062584, 29.294274300932088, 31.591211717709722, 32.75632490013315, 33.95472703062583, 35.65246338215712, 37.217043941411454, 37.91611185086551, 38.41544607190413, 39.88015978695073, 40.71238348868176, 41.84420772303595, 42.709720372836216, 44.10785619174435, 45.40612516644474, 46.17177097203729, 47.70306258322237, 48.7017310252996, 50.03328894806924, 50.832223701731024, 52.962716378162455, 54.260985352862846, 55.32623169107856, 56.15845539280959, 57.090545938748335, 58.3555259653795, 59.08788282290279, 59.58721704394141, 60.01997336884154, 61.55126498002663, 62.35019973368841, 62.916111850865505, 64.34753661784288, 65.31291611185085, 65.9121171770972, 67.21038615179761, 67.97603195739015, 69.37416777629826, 69.87350199733689, 71.03861517976033, 72.47003994673769, 73.93475366178428, 74.36750998668442, 75.7323568575233, 77.06391478029295, 77.52996005326231, 78.09587217043942, 78.59520639147803, 79.09454061251664, 80.45938748335553, 80.82556591211718, 81.6577896138482, 82.49001331557922, 84.25432756324899, 85.21970705725698, 86.15179760319573, 87.35019973368841, 87.78295605858855, 89.81358189081226, 90.54593874833556, 91.34487350199734, 92.31025299600533, 93.27563249001332, 94.2410119840213, 94.77363515312916, 95.70572569906791, 96.90412782956058, 98.16910785619174, 98.93475366178428, 100.0], "exact_score": [7.8873978710439285, 9.385691199033147, 10.607014724065985, 10.886995681134353, 11.251480662221043, 11.132795060465142, 11.681096088731294, 11.950485050656404, 12.257252647244954, 12.506060671997915, 12.514939040453227, 12.465605022280354, 12.420055756946361, 12.29887380374021, 12.419722833745093, 12.575536441831794, 13.044082832538306, 13.007265914495877, 13.307566938348394, 13.54675677097075, 13.497074182497135, 13.555566828996898, 13.489687173219936, 13.716701233693632, 13.75804044864708, 13.887624658384269, 13.92250071040878, 14.330481760984783, 14.811091624801936, 15.11196029225728, 15.340141855596729, 15.470228260094428, 15.602397136523424, 15.70686053663336, 15.77343114114952, 15.878079775290255, 16.185375446708754, 16.290856126262142, 16.331182187887638, 16.50044815857474, 16.56092601480848, 16.77465374797612, 16.94284498489534, 16.999451677577518, 17.163645586166435, 17.629775924543, 17.811488078872795, 17.905962000293275, 17.8983029118888, 18.211534969415936, 18.346538800999543, 18.494315902646907, 18.413902519057483, 18.548729286284622, 18.669420736132167, 18.829851654407225, 19.133282926264325, 19.193379699569622, 19.540879014531914, 19.73800209959461, 19.923183286960267, 20.164179388371707, 20.751172953972016, 21.47132790658189, 21.93063742480102, 22.362366590401837, 22.721608381678863, 22.974930007524037, 23.20854560052026, 23.2383924026918, 23.619367202525236, 24.009757143686, 24.453429561409283, 24.902699748790198, 25.246829349184345, 25.60144526845346, 25.8921080715241, 26.21796012896782, 26.5317341035184, 26.961660151404732, 26.989568494530463, 26.869480281200865, 26.740175462729944, 26.640355050620595, 26.682225655918423, 26.602846423093073, 26.624796634848057, 26.757955225427448, 26.684087621965713, 26.906854172574672, 26.869063981199858, 26.948143779905827, 27.17690106225915, 27.13203615169023, 27.060893323128965, 27.091464853822067, 27.12052075905354, 26.97763910098948, 27.056400498241338, 26.88989959621687, 27.001009067886244], "exact_calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "threshold_area25": 11.892818551883801, "exact_area100": 19.189318810152546}}, "old_normalized_endpoint_budget": 82}]}]} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/historical_results_v44.json b/releases/20260913_lr/paper_data/historical_results_v44.json new file mode 100644 index 0000000000000000000000000000000000000000..339f20611b3c7b28f1874b95c44ed7d52712f4bf --- /dev/null +++ b/releases/20260913_lr/paper_data/historical_results_v44.json @@ -0,0 +1,68573 @@ +{ + "protocol": "Five-seed probability-difference router, selected by task-balanced validation Score25 mean minus sample SD. Each seed fixes its own validation-selected threshold on test; frozen endpoint and encoder records.", + "source": "Five-seed router study: dse_results/objective_multiseed_20260912/results_five_seeds.json and runs/repeat_*/test_evaluation.json; original specialist/VLM outcomes are reused.", + "groups": [ + { + "key": [ + "cub", + "cub", + "MiniCPM" + ], + "expert_display": "ResNet", + "run": "five_seed_cub_MiniCPM_four_difference", + "validation_rows": 1200, + "test_rows": 5794, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/cub/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9058333333333333, + "validation_point": { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.985, + "rescued": 4, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 851, + "selected_both_correct": 1, + "selected_both_wrong": 2 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.113565757680359, + "metric": 0.799275112185019, + "rescued": 88, + "harmed": 175, + "missed_rescue": 144, + "avoided_harm": 3236, + "selected_both_correct": 56, + "selected_both_wrong": 339 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9858333333333333, + "rescued": 0, + "harmed": 0, + "missed_rescue": 6, + "avoided_harm": 856, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.985, + "rescued": 4, + "harmed": 5, + "missed_rescue": 2, + "avoided_harm": 851, + "selected_both_correct": 1, + "selected_both_wrong": 2 + }, + { + "budget": 0.02, + "threshold": -0.3627302646636963, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.98, + "rescued": 5, + "harmed": 12, + "missed_rescue": 1, + "avoided_harm": 844, + "selected_both_correct": 2, + "selected_both_wrong": 5 + }, + { + "budget": 0.03, + "threshold": -0.4222676455974579, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9741666666666666, + "rescued": 5, + "harmed": 19, + "missed_rescue": 1, + "avoided_harm": 837, + "selected_both_correct": 5, + "selected_both_wrong": 7 + }, + { + "budget": 0.04, + "threshold": -0.46618229150772095, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9683333333333334, + "rescued": 5, + "harmed": 26, + "missed_rescue": 1, + "avoided_harm": 830, + "selected_both_correct": 8, + "selected_both_wrong": 9 + }, + { + "budget": 0.05, + "threshold": -0.5066347122192383, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.9625, + "rescued": 5, + "harmed": 33, + "missed_rescue": 1, + "avoided_harm": 823, + "selected_both_correct": 13, + "selected_both_wrong": 9 + }, + { + "budget": 0.06, + "threshold": -0.5393391251564026, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.9566666666666667, + "rescued": 5, + "harmed": 40, + "missed_rescue": 1, + "avoided_harm": 816, + "selected_both_correct": 16, + "selected_both_wrong": 11 + }, + { + "budget": 0.07, + "threshold": -0.5679479837417603, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.95, + "rescued": 6, + "harmed": 49, + "missed_rescue": 0, + "avoided_harm": 807, + "selected_both_correct": 18, + "selected_both_wrong": 11 + }, + { + "budget": 0.08, + "threshold": -0.5907968282699585, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.9416666666666667, + "rescued": 6, + "harmed": 59, + "missed_rescue": 0, + "avoided_harm": 797, + "selected_both_correct": 20, + "selected_both_wrong": 11 + }, + { + "budget": 0.09, + "threshold": -0.6168767809867859, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9333333333333333, + "rescued": 6, + "harmed": 69, + "missed_rescue": 0, + "avoided_harm": 787, + "selected_both_correct": 22, + "selected_both_wrong": 11 + }, + { + "budget": 0.1, + "threshold": -0.6361178159713745, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9258333333333333, + "rescued": 6, + "harmed": 78, + "missed_rescue": 0, + "avoided_harm": 778, + "selected_both_correct": 25, + "selected_both_wrong": 11 + }, + { + "budget": 0.11, + "threshold": -0.6547113656997681, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9183333333333333, + "rescued": 6, + "harmed": 87, + "missed_rescue": 0, + "avoided_harm": 769, + "selected_both_correct": 28, + "selected_both_wrong": 11 + }, + { + "budget": 0.12, + "threshold": -0.6680066585540771, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9091666666666667, + "rescued": 6, + "harmed": 98, + "missed_rescue": 0, + "avoided_harm": 758, + "selected_both_correct": 29, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -0.6809748411178589, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9025, + "rescued": 6, + "harmed": 106, + "missed_rescue": 0, + "avoided_harm": 750, + "selected_both_correct": 33, + "selected_both_wrong": 11 + }, + { + "budget": 0.14, + "threshold": -0.6963143348693848, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.895, + "rescued": 6, + "harmed": 115, + "missed_rescue": 0, + "avoided_harm": 741, + "selected_both_correct": 36, + "selected_both_wrong": 11 + }, + { + "budget": 0.15, + "threshold": -0.7065573930740356, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.8866666666666667, + "rescued": 6, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 731, + "selected_both_correct": 38, + "selected_both_wrong": 11 + }, + { + "budget": 0.16, + "threshold": -0.7129085659980774, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.8791666666666667, + "rescued": 6, + "harmed": 134, + "missed_rescue": 0, + "avoided_harm": 722, + "selected_both_correct": 41, + "selected_both_wrong": 11 + }, + { + "budget": 0.17, + "threshold": -0.7240402102470398, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.8733333333333333, + "rescued": 6, + "harmed": 141, + "missed_rescue": 0, + "avoided_harm": 715, + "selected_both_correct": 46, + "selected_both_wrong": 11 + }, + { + "budget": 0.18, + "threshold": -0.7345248460769653, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.865, + "rescued": 6, + "harmed": 151, + "missed_rescue": 0, + "avoided_harm": 705, + "selected_both_correct": 48, + "selected_both_wrong": 11 + }, + { + "budget": 0.19, + "threshold": -0.7443157434463501, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.8591666666666666, + "rescued": 6, + "harmed": 158, + "missed_rescue": 0, + "avoided_harm": 698, + "selected_both_correct": 53, + "selected_both_wrong": 11 + }, + { + "budget": 0.2, + "threshold": -0.7533873319625854, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.8525, + "rescued": 6, + "harmed": 166, + "missed_rescue": 0, + "avoided_harm": 690, + "selected_both_correct": 57, + "selected_both_wrong": 11 + }, + { + "budget": 0.21, + "threshold": -0.7606735825538635, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.845, + "rescued": 6, + "harmed": 175, + "missed_rescue": 0, + "avoided_harm": 681, + "selected_both_correct": 60, + "selected_both_wrong": 11 + }, + { + "budget": 0.22, + "threshold": -0.7657199501991272, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.8383333333333334, + "rescued": 6, + "harmed": 183, + "missed_rescue": 0, + "avoided_harm": 673, + "selected_both_correct": 64, + "selected_both_wrong": 11 + }, + { + "budget": 0.23, + "threshold": -0.7712029218673706, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.8316666666666667, + "rescued": 6, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 665, + "selected_both_correct": 68, + "selected_both_wrong": 11 + }, + { + "budget": 0.24, + "threshold": -0.7754930257797241, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.825, + "rescued": 6, + "harmed": 199, + "missed_rescue": 0, + "avoided_harm": 657, + "selected_both_correct": 72, + "selected_both_wrong": 11 + }, + { + "budget": 0.25, + "threshold": -0.7811619639396667, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.8175, + "rescued": 6, + "harmed": 208, + "missed_rescue": 0, + "avoided_harm": 648, + "selected_both_correct": 75, + "selected_both_wrong": 11 + }, + { + "budget": 0.26, + "threshold": -0.7841500043869019, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.8108333333333333, + "rescued": 6, + "harmed": 216, + "missed_rescue": 0, + "avoided_harm": 640, + "selected_both_correct": 79, + "selected_both_wrong": 11 + }, + { + "budget": 0.27, + "threshold": -0.78773033618927, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.8033333333333333, + "rescued": 6, + "harmed": 225, + "missed_rescue": 0, + "avoided_harm": 631, + "selected_both_correct": 82, + "selected_both_wrong": 11 + }, + { + "budget": 0.28, + "threshold": -0.7923226356506348, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.7975, + "rescued": 6, + "harmed": 232, + "missed_rescue": 0, + "avoided_harm": 624, + "selected_both_correct": 87, + "selected_both_wrong": 11 + }, + { + "budget": 0.29, + "threshold": -0.7950519323348999, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.79, + "rescued": 6, + "harmed": 241, + "missed_rescue": 0, + "avoided_harm": 615, + "selected_both_correct": 90, + "selected_both_wrong": 11 + }, + { + "budget": 0.3, + "threshold": -0.800890326499939, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.7816666666666666, + "rescued": 6, + "harmed": 251, + "missed_rescue": 0, + "avoided_harm": 605, + "selected_both_correct": 92, + "selected_both_wrong": 11 + }, + { + "budget": 0.31, + "threshold": -0.8055772185325623, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.7733333333333333, + "rescued": 6, + "harmed": 261, + "missed_rescue": 0, + "avoided_harm": 595, + "selected_both_correct": 94, + "selected_both_wrong": 11 + }, + { + "budget": 0.32, + "threshold": -0.809782087802887, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.7675, + "rescued": 6, + "harmed": 268, + "missed_rescue": 0, + "avoided_harm": 588, + "selected_both_correct": 99, + "selected_both_wrong": 11 + }, + { + "budget": 0.33, + "threshold": -0.8134240508079529, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.7575, + "rescued": 6, + "harmed": 280, + "missed_rescue": 0, + "avoided_harm": 576, + "selected_both_correct": 99, + "selected_both_wrong": 11 + }, + { + "budget": 0.34, + "threshold": -0.8167235255241394, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.7516666666666667, + "rescued": 6, + "harmed": 287, + "missed_rescue": 0, + "avoided_harm": 569, + "selected_both_correct": 104, + "selected_both_wrong": 11 + }, + { + "budget": 0.35, + "threshold": -0.8208380937576294, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.7425, + "rescued": 6, + "harmed": 298, + "missed_rescue": 0, + "avoided_harm": 558, + "selected_both_correct": 105, + "selected_both_wrong": 11 + }, + { + "budget": 0.36, + "threshold": -0.8250406384468079, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.7375, + "rescued": 6, + "harmed": 304, + "missed_rescue": 0, + "avoided_harm": 552, + "selected_both_correct": 111, + "selected_both_wrong": 11 + }, + { + "budget": 0.37, + "threshold": -0.8295461535453796, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.7291666666666666, + "rescued": 6, + "harmed": 314, + "missed_rescue": 0, + "avoided_harm": 542, + "selected_both_correct": 113, + "selected_both_wrong": 11 + }, + { + "budget": 0.38, + "threshold": -0.8328037261962891, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.7191666666666666, + "rescued": 6, + "harmed": 326, + "missed_rescue": 0, + "avoided_harm": 530, + "selected_both_correct": 113, + "selected_both_wrong": 11 + }, + { + "budget": 0.39, + "threshold": -0.8359338641166687, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.7116666666666667, + "rescued": 6, + "harmed": 335, + "missed_rescue": 0, + "avoided_harm": 521, + "selected_both_correct": 116, + "selected_both_wrong": 11 + }, + { + "budget": 0.4, + "threshold": -0.8391940593719482, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.7033333333333334, + "rescued": 6, + "harmed": 345, + "missed_rescue": 0, + "avoided_harm": 511, + "selected_both_correct": 118, + "selected_both_wrong": 11 + }, + { + "budget": 0.41, + "threshold": -0.8424465656280518, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.6966666666666667, + "rescued": 6, + "harmed": 353, + "missed_rescue": 0, + "avoided_harm": 503, + "selected_both_correct": 121, + "selected_both_wrong": 11 + }, + { + "budget": 0.42, + "threshold": -0.8460794687271118, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.6883333333333334, + "rescued": 6, + "harmed": 363, + "missed_rescue": 0, + "avoided_harm": 493, + "selected_both_correct": 124, + "selected_both_wrong": 11 + }, + { + "budget": 0.43, + "threshold": -0.8500295281410217, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.68, + "rescued": 6, + "harmed": 373, + "missed_rescue": 0, + "avoided_harm": 483, + "selected_both_correct": 126, + "selected_both_wrong": 11 + }, + { + "budget": 0.44, + "threshold": -0.852453887462616, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.6741666666666667, + "rescued": 6, + "harmed": 380, + "missed_rescue": 0, + "avoided_harm": 476, + "selected_both_correct": 131, + "selected_both_wrong": 11 + }, + { + "budget": 0.45, + "threshold": -0.8539360165596008, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.6683333333333333, + "rescued": 6, + "harmed": 387, + "missed_rescue": 0, + "avoided_harm": 469, + "selected_both_correct": 136, + "selected_both_wrong": 11 + }, + { + "budget": 0.46, + "threshold": -0.8562509417533875, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.6608333333333334, + "rescued": 6, + "harmed": 396, + "missed_rescue": 0, + "avoided_harm": 460, + "selected_both_correct": 139, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.8592684864997864, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.6533333333333333, + "rescued": 6, + "harmed": 405, + "missed_rescue": 0, + "avoided_harm": 451, + "selected_both_correct": 142, + "selected_both_wrong": 11 + }, + { + "budget": 0.48, + "threshold": -0.8621411919593811, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.645, + "rescued": 6, + "harmed": 415, + "missed_rescue": 0, + "avoided_harm": 441, + "selected_both_correct": 144, + "selected_both_wrong": 11 + }, + { + "budget": 0.49, + "threshold": -0.863978385925293, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.6391666666666667, + "rescued": 6, + "harmed": 422, + "missed_rescue": 0, + "avoided_harm": 434, + "selected_both_correct": 149, + "selected_both_wrong": 11 + }, + { + "budget": 0.5, + "threshold": -0.8661247491836548, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.6341666666666667, + "rescued": 6, + "harmed": 428, + "missed_rescue": 0, + "avoided_harm": 428, + "selected_both_correct": 155, + "selected_both_wrong": 11 + }, + { + "budget": 0.51, + "threshold": -0.8688956499099731, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.6258333333333334, + "rescued": 6, + "harmed": 438, + "missed_rescue": 0, + "avoided_harm": 418, + "selected_both_correct": 157, + "selected_both_wrong": 11 + }, + { + "budget": 0.52, + "threshold": -0.8706188201904297, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.6191666666666666, + "rescued": 6, + "harmed": 446, + "missed_rescue": 0, + "avoided_harm": 410, + "selected_both_correct": 161, + "selected_both_wrong": 11 + }, + { + "budget": 0.53, + "threshold": -0.8727357983589172, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.6108333333333333, + "rescued": 6, + "harmed": 456, + "missed_rescue": 0, + "avoided_harm": 400, + "selected_both_correct": 163, + "selected_both_wrong": 11 + }, + { + "budget": 0.54, + "threshold": -0.874894380569458, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.6033333333333334, + "rescued": 6, + "harmed": 465, + "missed_rescue": 0, + "avoided_harm": 391, + "selected_both_correct": 166, + "selected_both_wrong": 11 + }, + { + "budget": 0.55, + "threshold": -0.8764559626579285, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.5966666666666667, + "rescued": 6, + "harmed": 473, + "missed_rescue": 0, + "avoided_harm": 383, + "selected_both_correct": 170, + "selected_both_wrong": 11 + }, + { + "budget": 0.56, + "threshold": -0.879269540309906, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.5891666666666666, + "rescued": 6, + "harmed": 482, + "missed_rescue": 0, + "avoided_harm": 374, + "selected_both_correct": 173, + "selected_both_wrong": 11 + }, + { + "budget": 0.57, + "threshold": -0.881222128868103, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.5816666666666667, + "rescued": 6, + "harmed": 491, + "missed_rescue": 0, + "avoided_harm": 365, + "selected_both_correct": 175, + "selected_both_wrong": 11 + }, + { + "budget": 0.58, + "threshold": -0.882451593875885, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.575, + "rescued": 6, + "harmed": 499, + "missed_rescue": 0, + "avoided_harm": 357, + "selected_both_correct": 180, + "selected_both_wrong": 11 + }, + { + "budget": 0.59, + "threshold": -0.8843295574188232, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.5675, + "rescued": 6, + "harmed": 508, + "missed_rescue": 0, + "avoided_harm": 348, + "selected_both_correct": 183, + "selected_both_wrong": 11 + }, + { + "budget": 0.6, + "threshold": -0.8863861560821533, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.5608333333333333, + "rescued": 6, + "harmed": 516, + "missed_rescue": 0, + "avoided_harm": 340, + "selected_both_correct": 187, + "selected_both_wrong": 11 + }, + { + "budget": 0.61, + "threshold": -0.8876380920410156, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.5525, + "rescued": 6, + "harmed": 526, + "missed_rescue": 0, + "avoided_harm": 330, + "selected_both_correct": 189, + "selected_both_wrong": 11 + }, + { + "budget": 0.62, + "threshold": -0.8888679146766663, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.5466666666666666, + "rescued": 6, + "harmed": 533, + "missed_rescue": 0, + "avoided_harm": 323, + "selected_both_correct": 194, + "selected_both_wrong": 11 + }, + { + "budget": 0.63, + "threshold": -0.8905391693115234, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.5408333333333334, + "rescued": 6, + "harmed": 540, + "missed_rescue": 0, + "avoided_harm": 316, + "selected_both_correct": 199, + "selected_both_wrong": 11 + }, + { + "budget": 0.64, + "threshold": -0.8919612169265747, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.5325, + "rescued": 6, + "harmed": 550, + "missed_rescue": 0, + "avoided_harm": 306, + "selected_both_correct": 201, + "selected_both_wrong": 11 + }, + { + "budget": 0.65, + "threshold": -0.8940081596374512, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.525, + "rescued": 6, + "harmed": 559, + "missed_rescue": 0, + "avoided_harm": 297, + "selected_both_correct": 204, + "selected_both_wrong": 11 + }, + { + "budget": 0.66, + "threshold": -0.895609974861145, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.5175, + "rescued": 6, + "harmed": 568, + "missed_rescue": 0, + "avoided_harm": 288, + "selected_both_correct": 207, + "selected_both_wrong": 11 + }, + { + "budget": 0.67, + "threshold": -0.8972456455230713, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.5091666666666667, + "rescued": 6, + "harmed": 578, + "missed_rescue": 0, + "avoided_harm": 278, + "selected_both_correct": 209, + "selected_both_wrong": 11 + }, + { + "budget": 0.68, + "threshold": -0.8986453413963318, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.5008333333333334, + "rescued": 6, + "harmed": 588, + "missed_rescue": 0, + "avoided_harm": 268, + "selected_both_correct": 211, + "selected_both_wrong": 11 + }, + { + "budget": 0.69, + "threshold": -0.8996874094009399, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.49333333333333335, + "rescued": 6, + "harmed": 597, + "missed_rescue": 0, + "avoided_harm": 259, + "selected_both_correct": 213, + "selected_both_wrong": 11 + }, + { + "budget": 0.7, + "threshold": -0.9008907079696655, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.48333333333333334, + "rescued": 6, + "harmed": 609, + "missed_rescue": 0, + "avoided_harm": 247, + "selected_both_correct": 214, + "selected_both_wrong": 11 + }, + { + "budget": 0.71, + "threshold": -0.9031620621681213, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.47583333333333333, + "rescued": 6, + "harmed": 618, + "missed_rescue": 0, + "avoided_harm": 238, + "selected_both_correct": 217, + "selected_both_wrong": 11 + }, + { + "budget": 0.72, + "threshold": -0.9056642055511475, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.4716666666666667, + "rescued": 6, + "harmed": 623, + "missed_rescue": 0, + "avoided_harm": 233, + "selected_both_correct": 224, + "selected_both_wrong": 11 + }, + { + "budget": 0.73, + "threshold": -0.9068957567214966, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.465, + "rescued": 6, + "harmed": 631, + "missed_rescue": 0, + "avoided_harm": 225, + "selected_both_correct": 228, + "selected_both_wrong": 11 + }, + { + "budget": 0.74, + "threshold": -0.9093236327171326, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.45916666666666667, + "rescued": 6, + "harmed": 638, + "missed_rescue": 0, + "avoided_harm": 218, + "selected_both_correct": 233, + "selected_both_wrong": 11 + }, + { + "budget": 0.75, + "threshold": -0.9112368822097778, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.45166666666666666, + "rescued": 6, + "harmed": 647, + "missed_rescue": 0, + "avoided_harm": 209, + "selected_both_correct": 236, + "selected_both_wrong": 11 + }, + { + "budget": 0.76, + "threshold": -0.9129221439361572, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.44416666666666665, + "rescued": 6, + "harmed": 656, + "missed_rescue": 0, + "avoided_harm": 200, + "selected_both_correct": 239, + "selected_both_wrong": 11 + }, + { + "budget": 0.77, + "threshold": -0.9144289493560791, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.4375, + "rescued": 6, + "harmed": 664, + "missed_rescue": 0, + "avoided_harm": 192, + "selected_both_correct": 243, + "selected_both_wrong": 11 + }, + { + "budget": 0.78, + "threshold": -0.9158422946929932, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.43416666666666665, + "rescued": 6, + "harmed": 668, + "missed_rescue": 0, + "avoided_harm": 188, + "selected_both_correct": 251, + "selected_both_wrong": 11 + }, + { + "budget": 0.79, + "threshold": -0.9167781472206116, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.4266666666666667, + "rescued": 6, + "harmed": 677, + "missed_rescue": 0, + "avoided_harm": 179, + "selected_both_correct": 254, + "selected_both_wrong": 11 + }, + { + "budget": 0.8, + "threshold": -0.9187189936637878, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.42, + "rescued": 6, + "harmed": 685, + "missed_rescue": 0, + "avoided_harm": 171, + "selected_both_correct": 258, + "selected_both_wrong": 11 + }, + { + "budget": 0.81, + "threshold": -0.9207327365875244, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.41333333333333333, + "rescued": 6, + "harmed": 693, + "missed_rescue": 0, + "avoided_harm": 163, + "selected_both_correct": 262, + "selected_both_wrong": 11 + }, + { + "budget": 0.82, + "threshold": -0.9226345419883728, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.4075, + "rescued": 6, + "harmed": 700, + "missed_rescue": 0, + "avoided_harm": 156, + "selected_both_correct": 266, + "selected_both_wrong": 11 + }, + { + "budget": 0.83, + "threshold": -0.9243993759155273, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.39916666666666667, + "rescued": 6, + "harmed": 710, + "missed_rescue": 0, + "avoided_harm": 146, + "selected_both_correct": 269, + "selected_both_wrong": 11 + }, + { + "budget": 0.84, + "threshold": -0.9264947175979614, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.39, + "rescued": 6, + "harmed": 721, + "missed_rescue": 0, + "avoided_harm": 135, + "selected_both_correct": 270, + "selected_both_wrong": 11 + }, + { + "budget": 0.85, + "threshold": -0.9274600148200989, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.38416666666666666, + "rescued": 6, + "harmed": 728, + "missed_rescue": 0, + "avoided_harm": 128, + "selected_both_correct": 275, + "selected_both_wrong": 11 + }, + { + "budget": 0.86, + "threshold": -0.930335283279419, + "operator": "ge", + "actual_fraction": 0.86, + "metric": 0.37833333333333335, + "rescued": 6, + "harmed": 735, + "missed_rescue": 0, + "avoided_harm": 121, + "selected_both_correct": 280, + "selected_both_wrong": 11 + }, + { + "budget": 0.87, + "threshold": -0.9319414496421814, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.37166666666666665, + "rescued": 6, + "harmed": 743, + "missed_rescue": 0, + "avoided_harm": 113, + "selected_both_correct": 284, + "selected_both_wrong": 11 + }, + { + "budget": 0.88, + "threshold": -0.9340114593505859, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.36583333333333334, + "rescued": 6, + "harmed": 750, + "missed_rescue": 0, + "avoided_harm": 106, + "selected_both_correct": 289, + "selected_both_wrong": 11 + }, + { + "budget": 0.89, + "threshold": -0.9363258481025696, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.3591666666666667, + "rescued": 6, + "harmed": 758, + "missed_rescue": 0, + "avoided_harm": 98, + "selected_both_correct": 293, + "selected_both_wrong": 11 + }, + { + "budget": 0.9, + "threshold": -0.9400071501731873, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.35, + "rescued": 6, + "harmed": 769, + "missed_rescue": 0, + "avoided_harm": 87, + "selected_both_correct": 294, + "selected_both_wrong": 11 + }, + { + "budget": 0.91, + "threshold": -0.9423198103904724, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.3425, + "rescued": 6, + "harmed": 778, + "missed_rescue": 0, + "avoided_harm": 78, + "selected_both_correct": 297, + "selected_both_wrong": 11 + }, + { + "budget": 0.92, + "threshold": -0.9443356394767761, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.335, + "rescued": 6, + "harmed": 787, + "missed_rescue": 0, + "avoided_harm": 69, + "selected_both_correct": 300, + "selected_both_wrong": 11 + }, + { + "budget": 0.93, + "threshold": -0.9468213319778442, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.3283333333333333, + "rescued": 6, + "harmed": 795, + "missed_rescue": 0, + "avoided_harm": 61, + "selected_both_correct": 304, + "selected_both_wrong": 11 + }, + { + "budget": 0.94, + "threshold": -0.9496435523033142, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.32083333333333336, + "rescued": 6, + "harmed": 804, + "missed_rescue": 0, + "avoided_harm": 52, + "selected_both_correct": 307, + "selected_both_wrong": 11 + }, + { + "budget": 0.95, + "threshold": -0.9529175758361816, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.3125, + "rescued": 6, + "harmed": 814, + "missed_rescue": 0, + "avoided_harm": 42, + "selected_both_correct": 309, + "selected_both_wrong": 11 + }, + { + "budget": 0.96, + "threshold": -0.9560379981994629, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.305, + "rescued": 6, + "harmed": 823, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 312, + "selected_both_wrong": 11 + }, + { + "budget": 0.97, + "threshold": -0.9591385126113892, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.2991666666666667, + "rescued": 6, + "harmed": 830, + "missed_rescue": 0, + "avoided_harm": 26, + "selected_both_correct": 317, + "selected_both_wrong": 11 + }, + { + "budget": 0.98, + "threshold": -0.9627243280410767, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.2925, + "rescued": 6, + "harmed": 838, + "missed_rescue": 0, + "avoided_harm": 18, + "selected_both_correct": 321, + "selected_both_wrong": 11 + }, + { + "budget": 0.99, + "threshold": -0.9702208638191223, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.28583333333333333, + "rescued": 6, + "harmed": 846, + "missed_rescue": 0, + "avoided_harm": 10, + "selected_both_correct": 325, + "selected_both_wrong": 11 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.2775, + "rescued": 6, + "harmed": 856, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 327, + "selected_both_wrong": 11 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.81429064549534, + "rescued": 0, + "harmed": 0, + "missed_rescue": 232, + "avoided_harm": 3411, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.113565757680359, + "metric": 0.799275112185019, + "rescued": 88, + "harmed": 175, + "missed_rescue": 144, + "avoided_harm": 3236, + "selected_both_correct": 56, + "selected_both_wrong": 339 + }, + { + "budget": 0.02, + "threshold": -0.3627302646636963, + "operator": "ge", + "actual_fraction": 0.15947531929582326, + "metric": 0.7839143942008975, + "rescued": 113, + "harmed": 289, + "missed_rescue": 119, + "avoided_harm": 3122, + "selected_both_correct": 88, + "selected_both_wrong": 434 + }, + { + "budget": 0.03, + "threshold": -0.4222676455974579, + "operator": "ge", + "actual_fraction": 0.2143596824301001, + "metric": 0.7611322057300656, + "rescued": 142, + "harmed": 450, + "missed_rescue": 90, + "avoided_harm": 2961, + "selected_both_correct": 128, + "selected_both_wrong": 522 + }, + { + "budget": 0.04, + "threshold": -0.46618229150772095, + "operator": "ge", + "actual_fraction": 0.2545736969278564, + "metric": 0.7430100103555403, + "rescued": 159, + "harmed": 572, + "missed_rescue": 73, + "avoided_harm": 2839, + "selected_both_correct": 159, + "selected_both_wrong": 585 + }, + { + "budget": 0.05, + "threshold": -0.5066347122192383, + "operator": "ge", + "actual_fraction": 0.29375215740421123, + "metric": 0.7229892992751122, + "rescued": 170, + "harmed": 699, + "missed_rescue": 62, + "avoided_harm": 2712, + "selected_both_correct": 209, + "selected_both_wrong": 624 + }, + { + "budget": 0.06, + "threshold": -0.5393391251564026, + "operator": "ge", + "actual_fraction": 0.3208491542975492, + "metric": 0.7102174663444943, + "rescued": 176, + "harmed": 779, + "missed_rescue": 56, + "avoided_harm": 2632, + "selected_both_correct": 243, + "selected_both_wrong": 661 + }, + { + "budget": 0.07, + "threshold": -0.5679479837417603, + "operator": "ge", + "actual_fraction": 0.350707628581291, + "metric": 0.6948567483603728, + "rescued": 190, + "harmed": 882, + "missed_rescue": 42, + "avoided_harm": 2529, + "selected_both_correct": 270, + "selected_both_wrong": 690 + }, + { + "budget": 0.08, + "threshold": -0.5907968282699585, + "operator": "ge", + "actual_fraction": 0.36882982395581637, + "metric": 0.684328615809458, + "rescued": 195, + "harmed": 948, + "missed_rescue": 37, + "avoided_harm": 2463, + "selected_both_correct": 289, + "selected_both_wrong": 705 + }, + { + "budget": 0.09, + "threshold": -0.6168767809867859, + "operator": "ge", + "actual_fraction": 0.391439420089748, + "metric": 0.6720745598895409, + "rescued": 201, + "harmed": 1025, + "missed_rescue": 31, + "avoided_harm": 2386, + "selected_both_correct": 319, + "selected_both_wrong": 723 + }, + { + "budget": 0.1, + "threshold": -0.6361178159713745, + "operator": "ge", + "actual_fraction": 0.4138764238867794, + "metric": 0.6601656886434243, + "rescued": 208, + "harmed": 1101, + "missed_rescue": 24, + "avoided_harm": 2310, + "selected_both_correct": 349, + "selected_both_wrong": 740 + }, + { + "budget": 0.11, + "threshold": -0.6547113656997681, + "operator": "ge", + "actual_fraction": 0.43372454263030724, + "metric": 0.6482568173973076, + "rescued": 212, + "harmed": 1174, + "missed_rescue": 20, + "avoided_harm": 2237, + "selected_both_correct": 373, + "selected_both_wrong": 754 + }, + { + "budget": 0.12, + "threshold": -0.6680066585540771, + "operator": "ge", + "actual_fraction": 0.4480497065930273, + "metric": 0.6384190541939938, + "rescued": 212, + "harmed": 1231, + "missed_rescue": 20, + "avoided_harm": 2180, + "selected_both_correct": 394, + "selected_both_wrong": 759 + }, + { + "budget": 0.13, + "threshold": -0.6809748411178589, + "operator": "ge", + "actual_fraction": 0.4630652399033483, + "metric": 0.6287538833275803, + "rescued": 214, + "harmed": 1289, + "missed_rescue": 18, + "avoided_harm": 2122, + "selected_both_correct": 412, + "selected_both_wrong": 768 + }, + { + "budget": 0.14, + "threshold": -0.6963143348693848, + "operator": "ge", + "actual_fraction": 0.479288919571971, + "metric": 0.617880566102865, + "rescued": 217, + "harmed": 1355, + "missed_rescue": 15, + "avoided_harm": 2056, + "selected_both_correct": 433, + "selected_both_wrong": 772 + }, + { + "budget": 0.15, + "threshold": -0.7065573930740356, + "operator": "ge", + "actual_fraction": 0.4932688988608906, + "metric": 0.6102865032792544, + "rescued": 220, + "harmed": 1402, + "missed_rescue": 12, + "avoided_harm": 2009, + "selected_both_correct": 457, + "selected_both_wrong": 779 + }, + { + "budget": 0.16, + "threshold": -0.7129085659980774, + "operator": "ge", + "actual_fraction": 0.5008629616845012, + "metric": 0.6054539178460476, + "rescued": 221, + "harmed": 1431, + "missed_rescue": 11, + "avoided_harm": 1980, + "selected_both_correct": 468, + "selected_both_wrong": 782 + }, + { + "budget": 0.17, + "threshold": -0.7240402102470398, + "operator": "ge", + "actual_fraction": 0.5155333103210218, + "metric": 0.5944080082844322, + "rescued": 221, + "harmed": 1495, + "missed_rescue": 11, + "avoided_harm": 1916, + "selected_both_correct": 487, + "selected_both_wrong": 784 + }, + { + "budget": 0.18, + "threshold": -0.7345248460769653, + "operator": "ge", + "actual_fraction": 0.5298584742837418, + "metric": 0.5845702450811184, + "rescued": 223, + "harmed": 1554, + "missed_rescue": 9, + "avoided_harm": 1857, + "selected_both_correct": 502, + "selected_both_wrong": 791 + }, + { + "budget": 0.19, + "threshold": -0.7443157434463501, + "operator": "ge", + "actual_fraction": 0.5455643769416638, + "metric": 0.5745598895409044, + "rescued": 224, + "harmed": 1613, + "missed_rescue": 8, + "avoided_harm": 1798, + "selected_both_correct": 529, + "selected_both_wrong": 795 + }, + { + "budget": 0.2, + "threshold": -0.7533873319625854, + "operator": "ge", + "actual_fraction": 0.5560925094925785, + "metric": 0.5674836037279944, + "rescued": 226, + "harmed": 1656, + "missed_rescue": 6, + "avoided_harm": 1755, + "selected_both_correct": 542, + "selected_both_wrong": 798 + }, + { + "budget": 0.21, + "threshold": -0.7606735825538635, + "operator": "ge", + "actual_fraction": 0.5676561960648947, + "metric": 0.5598895409043838, + "rescued": 226, + "harmed": 1700, + "missed_rescue": 6, + "avoided_harm": 1711, + "selected_both_correct": 563, + "selected_both_wrong": 800 + }, + { + "budget": 0.22, + "threshold": -0.7657199501991272, + "operator": "ge", + "actual_fraction": 0.5766309975837073, + "metric": 0.5541939937866759, + "rescued": 226, + "harmed": 1733, + "missed_rescue": 6, + "avoided_harm": 1678, + "selected_both_correct": 580, + "selected_both_wrong": 802 + }, + { + "budget": 0.23, + "threshold": -0.7712029218673706, + "operator": "ge", + "actual_fraction": 0.5861235761132205, + "metric": 0.5479806696582672, + "rescued": 226, + "harmed": 1769, + "missed_rescue": 6, + "avoided_harm": 1642, + "selected_both_correct": 597, + "selected_both_wrong": 804 + }, + { + "budget": 0.24, + "threshold": -0.7754930257797241, + "operator": "ge", + "actual_fraction": 0.5923369002416292, + "metric": 0.5434932688988608, + "rescued": 226, + "harmed": 1795, + "missed_rescue": 6, + "avoided_harm": 1616, + "selected_both_correct": 604, + "selected_both_wrong": 807 + }, + { + "budget": 0.25, + "threshold": -0.7811619639396667, + "operator": "ge", + "actual_fraction": 0.6013117017604418, + "metric": 0.5367621677597515, + "rescued": 226, + "harmed": 1834, + "missed_rescue": 6, + "avoided_harm": 1577, + "selected_both_correct": 616, + "selected_both_wrong": 808 + }, + { + "budget": 0.26, + "threshold": -0.7841500043869019, + "operator": "ge", + "actual_fraction": 0.6052813255091474, + "metric": 0.5345184673800484, + "rescued": 226, + "harmed": 1847, + "missed_rescue": 6, + "avoided_harm": 1564, + "selected_both_correct": 625, + "selected_both_wrong": 809 + }, + { + "budget": 0.27, + "threshold": -0.78773033618927, + "operator": "ge", + "actual_fraction": 0.6123576113220573, + "metric": 0.5291681049361409, + "rescued": 227, + "harmed": 1879, + "missed_rescue": 5, + "avoided_harm": 1532, + "selected_both_correct": 632, + "selected_both_wrong": 810 + }, + { + "budget": 0.28, + "threshold": -0.7923226356506348, + "operator": "ge", + "actual_fraction": 0.6206420434932689, + "metric": 0.5241629271660339, + "rescued": 227, + "harmed": 1908, + "missed_rescue": 5, + "avoided_harm": 1503, + "selected_both_correct": 649, + "selected_both_wrong": 812 + }, + { + "budget": 0.29, + "threshold": -0.7950519323348999, + "operator": "ge", + "actual_fraction": 0.6251294442526751, + "metric": 0.52140144977563, + "rescued": 227, + "harmed": 1924, + "missed_rescue": 5, + "avoided_harm": 1487, + "selected_both_correct": 658, + "selected_both_wrong": 813 + }, + { + "budget": 0.3, + "threshold": -0.800890326499939, + "operator": "ge", + "actual_fraction": 0.6373835001725924, + "metric": 0.5117362789092165, + "rescued": 227, + "harmed": 1980, + "missed_rescue": 5, + "avoided_harm": 1431, + "selected_both_correct": 672, + "selected_both_wrong": 814 + }, + { + "budget": 0.31, + "threshold": -0.8055772185325623, + "operator": "ge", + "actual_fraction": 0.6470486710390059, + "metric": 0.5055229547808078, + "rescued": 227, + "harmed": 2016, + "missed_rescue": 5, + "avoided_harm": 1395, + "selected_both_correct": 691, + "selected_both_wrong": 815 + }, + { + "budget": 0.32, + "threshold": -0.809782087802887, + "operator": "ge", + "actual_fraction": 0.6556782878840179, + "metric": 0.5005177770107008, + "rescued": 227, + "harmed": 2045, + "missed_rescue": 5, + "avoided_harm": 1366, + "selected_both_correct": 708, + "selected_both_wrong": 819 + }, + { + "budget": 0.33, + "threshold": -0.8134240508079529, + "operator": "ge", + "actual_fraction": 0.6636175353814291, + "metric": 0.4955125992405937, + "rescued": 227, + "harmed": 2074, + "missed_rescue": 5, + "avoided_harm": 1337, + "selected_both_correct": 724, + "selected_both_wrong": 820 + }, + { + "budget": 0.34, + "threshold": -0.8167235255241394, + "operator": "ge", + "actual_fraction": 0.6708664135312392, + "metric": 0.489989644459786, + "rescued": 227, + "harmed": 2106, + "missed_rescue": 5, + "avoided_harm": 1305, + "selected_both_correct": 733, + "selected_both_wrong": 821 + }, + { + "budget": 0.35, + "threshold": -0.8208380937576294, + "operator": "ge", + "actual_fraction": 0.6812219537452537, + "metric": 0.4830859509837763, + "rescued": 228, + "harmed": 2147, + "missed_rescue": 4, + "avoided_harm": 1264, + "selected_both_correct": 751, + "selected_both_wrong": 821 + }, + { + "budget": 0.36, + "threshold": -0.8250406384468079, + "operator": "ge", + "actual_fraction": 0.689333793579565, + "metric": 0.4777355885398688, + "rescued": 228, + "harmed": 2178, + "missed_rescue": 4, + "avoided_harm": 1233, + "selected_both_correct": 765, + "selected_both_wrong": 823 + }, + { + "budget": 0.37, + "threshold": -0.8295461535453796, + "operator": "ge", + "actual_fraction": 0.7017604418363824, + "metric": 0.468588194684156, + "rescued": 228, + "harmed": 2231, + "missed_rescue": 4, + "avoided_harm": 1180, + "selected_both_correct": 782, + "selected_both_wrong": 825 + }, + { + "budget": 0.38, + "threshold": -0.8328037261962891, + "operator": "ge", + "actual_fraction": 0.7088367276492924, + "metric": 0.463583016914049, + "rescued": 228, + "harmed": 2260, + "missed_rescue": 4, + "avoided_harm": 1151, + "selected_both_correct": 793, + "selected_both_wrong": 826 + }, + { + "budget": 0.39, + "threshold": -0.8359338641166687, + "operator": "ge", + "actual_fraction": 0.7166033828098033, + "metric": 0.4577148774594408, + "rescued": 228, + "harmed": 2294, + "missed_rescue": 4, + "avoided_harm": 1117, + "selected_both_correct": 804, + "selected_both_wrong": 826 + }, + { + "budget": 0.4, + "threshold": -0.8391940593719482, + "operator": "ge", + "actual_fraction": 0.7238522609596134, + "metric": 0.4527096996893338, + "rescued": 229, + "harmed": 2324, + "missed_rescue": 3, + "avoided_harm": 1087, + "selected_both_correct": 814, + "selected_both_wrong": 827 + }, + { + "budget": 0.41, + "threshold": -0.8424465656280518, + "operator": "ge", + "actual_fraction": 0.7302381774249224, + "metric": 0.44839489126682774, + "rescued": 229, + "harmed": 2349, + "missed_rescue": 3, + "avoided_harm": 1062, + "selected_both_correct": 825, + "selected_both_wrong": 828 + }, + { + "budget": 0.42, + "threshold": -0.8460794687271118, + "operator": "ge", + "actual_fraction": 0.7392129789437349, + "metric": 0.4418363824646186, + "rescued": 229, + "harmed": 2387, + "missed_rescue": 3, + "avoided_harm": 1024, + "selected_both_correct": 839, + "selected_both_wrong": 828 + }, + { + "budget": 0.43, + "threshold": -0.8500295281410217, + "operator": "ge", + "actual_fraction": 0.747152226441146, + "metric": 0.43596824301001036, + "rescued": 229, + "harmed": 2421, + "missed_rescue": 3, + "avoided_harm": 990, + "selected_both_correct": 851, + "selected_both_wrong": 828 + }, + { + "budget": 0.44, + "threshold": -0.852453887462616, + "operator": "ge", + "actual_fraction": 0.7523299965481532, + "metric": 0.4325163962720055, + "rescued": 229, + "harmed": 2441, + "missed_rescue": 3, + "avoided_harm": 970, + "selected_both_correct": 860, + "selected_both_wrong": 829 + }, + { + "budget": 0.45, + "threshold": -0.8539360165596008, + "operator": "ge", + "actual_fraction": 0.7562996202968588, + "metric": 0.42975491888160167, + "rescued": 229, + "harmed": 2457, + "missed_rescue": 3, + "avoided_harm": 954, + "selected_both_correct": 867, + "selected_both_wrong": 829 + }, + { + "budget": 0.46, + "threshold": -0.8562509417533875, + "operator": "ge", + "actual_fraction": 0.7613047980669658, + "metric": 0.42595788746979635, + "rescued": 229, + "harmed": 2479, + "missed_rescue": 3, + "avoided_harm": 932, + "selected_both_correct": 874, + "selected_both_wrong": 829 + }, + { + "budget": 0.47, + "threshold": -0.8592684864997864, + "operator": "ge", + "actual_fraction": 0.7668277528477736, + "metric": 0.4219882637210908, + "rescued": 229, + "harmed": 2502, + "missed_rescue": 3, + "avoided_harm": 909, + "selected_both_correct": 882, + "selected_both_wrong": 830 + }, + { + "budget": 0.48, + "threshold": -0.8621411919593811, + "operator": "ge", + "actual_fraction": 0.774249223334484, + "metric": 0.4161201242664826, + "rescued": 230, + "harmed": 2537, + "missed_rescue": 2, + "avoided_harm": 874, + "selected_both_correct": 888, + "selected_both_wrong": 831 + }, + { + "budget": 0.49, + "threshold": -0.863978385925293, + "operator": "ge", + "actual_fraction": 0.7790818087676907, + "metric": 0.41266827752847773, + "rescued": 230, + "harmed": 2557, + "missed_rescue": 2, + "avoided_harm": 854, + "selected_both_correct": 895, + "selected_both_wrong": 832 + }, + { + "budget": 0.5, + "threshold": -0.8661247491836548, + "operator": "ge", + "actual_fraction": 0.7849499482222989, + "metric": 0.40852606144287196, + "rescued": 230, + "harmed": 2581, + "missed_rescue": 2, + "avoided_harm": 830, + "selected_both_correct": 904, + "selected_both_wrong": 833 + }, + { + "budget": 0.51, + "threshold": -0.8688956499099731, + "operator": "ge", + "actual_fraction": 0.7928891957197101, + "metric": 0.402830514325164, + "rescued": 230, + "harmed": 2614, + "missed_rescue": 2, + "avoided_harm": 797, + "selected_both_correct": 916, + "selected_both_wrong": 834 + }, + { + "budget": 0.52, + "threshold": -0.8706188201904297, + "operator": "ge", + "actual_fraction": 0.7970314118053159, + "metric": 0.39972385226095963, + "rescued": 230, + "harmed": 2632, + "missed_rescue": 2, + "avoided_harm": 779, + "selected_both_correct": 922, + "selected_both_wrong": 834 + }, + { + "budget": 0.53, + "threshold": -0.8727357983589172, + "operator": "ge", + "actual_fraction": 0.8016914049016224, + "metric": 0.39748015188125646, + "rescued": 230, + "harmed": 2645, + "missed_rescue": 2, + "avoided_harm": 766, + "selected_both_correct": 936, + "selected_both_wrong": 834 + }, + { + "budget": 0.54, + "threshold": -0.874894380569458, + "operator": "ge", + "actual_fraction": 0.8070417673455299, + "metric": 0.39368312046945114, + "rescued": 230, + "harmed": 2667, + "missed_rescue": 2, + "avoided_harm": 744, + "selected_both_correct": 945, + "selected_both_wrong": 834 + }, + { + "budget": 0.55, + "threshold": -0.8764559626579285, + "operator": "ge", + "actual_fraction": 0.8120469451156369, + "metric": 0.39040386606834654, + "rescued": 230, + "harmed": 2686, + "missed_rescue": 2, + "avoided_harm": 725, + "selected_both_correct": 955, + "selected_both_wrong": 834 + }, + { + "budget": 0.56, + "threshold": -0.879269540309906, + "operator": "ge", + "actual_fraction": 0.8180876769071453, + "metric": 0.3860890576458405, + "rescued": 230, + "harmed": 2711, + "missed_rescue": 2, + "avoided_harm": 700, + "selected_both_correct": 964, + "selected_both_wrong": 835 + }, + { + "budget": 0.57, + "threshold": -0.881222128868103, + "operator": "ge", + "actual_fraction": 0.823610631687953, + "metric": 0.3833275802554367, + "rescued": 231, + "harmed": 2728, + "missed_rescue": 1, + "avoided_harm": 683, + "selected_both_correct": 976, + "selected_both_wrong": 837 + }, + { + "budget": 0.58, + "threshold": -0.882451593875885, + "operator": "ge", + "actual_fraction": 0.8272350707628582, + "metric": 0.38073869520193304, + "rescued": 231, + "harmed": 2743, + "missed_rescue": 1, + "avoided_harm": 668, + "selected_both_correct": 982, + "selected_both_wrong": 837 + }, + { + "budget": 0.59, + "threshold": -0.8843295574188232, + "operator": "ge", + "actual_fraction": 0.8313772868484639, + "metric": 0.37797721781152915, + "rescued": 232, + "harmed": 2760, + "missed_rescue": 0, + "avoided_harm": 651, + "selected_both_correct": 988, + "selected_both_wrong": 837 + }, + { + "budget": 0.6, + "threshold": -0.8863861560821533, + "operator": "ge", + "actual_fraction": 0.8355195029340697, + "metric": 0.3748705557473248, + "rescued": 232, + "harmed": 2778, + "missed_rescue": 0, + "avoided_harm": 633, + "selected_both_correct": 994, + "selected_both_wrong": 837 + }, + { + "budget": 0.61, + "threshold": -0.8876380920410156, + "operator": "ge", + "actual_fraction": 0.8391439420089748, + "metric": 0.3726268553676217, + "rescued": 232, + "harmed": 2791, + "missed_rescue": 0, + "avoided_harm": 620, + "selected_both_correct": 1002, + "selected_both_wrong": 837 + }, + { + "budget": 0.62, + "threshold": -0.8888679146766663, + "operator": "ge", + "actual_fraction": 0.8425957887469796, + "metric": 0.3702105626510183, + "rescued": 232, + "harmed": 2805, + "missed_rescue": 0, + "avoided_harm": 606, + "selected_both_correct": 1008, + "selected_both_wrong": 837 + }, + { + "budget": 0.63, + "threshold": -0.8905391693115234, + "operator": "ge", + "actual_fraction": 0.8469105971694857, + "metric": 0.3669313082499137, + "rescued": 232, + "harmed": 2824, + "missed_rescue": 0, + "avoided_harm": 587, + "selected_both_correct": 1014, + "selected_both_wrong": 837 + }, + { + "budget": 0.64, + "threshold": -0.8919612169265747, + "operator": "ge", + "actual_fraction": 0.8503624439074905, + "metric": 0.36468760787021054, + "rescued": 232, + "harmed": 2837, + "missed_rescue": 0, + "avoided_harm": 574, + "selected_both_correct": 1021, + "selected_both_wrong": 837 + }, + { + "budget": 0.65, + "threshold": -0.8940081596374512, + "operator": "ge", + "actual_fraction": 0.8560579910251985, + "metric": 0.360717984121505, + "rescued": 232, + "harmed": 2860, + "missed_rescue": 0, + "avoided_harm": 551, + "selected_both_correct": 1031, + "selected_both_wrong": 837 + }, + { + "budget": 0.66, + "threshold": -0.895609974861145, + "operator": "ge", + "actual_fraction": 0.8603727994477045, + "metric": 0.3574387297204004, + "rescued": 232, + "harmed": 2879, + "missed_rescue": 0, + "avoided_harm": 532, + "selected_both_correct": 1036, + "selected_both_wrong": 838 + }, + { + "budget": 0.67, + "threshold": -0.8972456455230713, + "operator": "ge", + "actual_fraction": 0.8643424231964101, + "metric": 0.3548498446668968, + "rescued": 232, + "harmed": 2894, + "missed_rescue": 0, + "avoided_harm": 517, + "selected_both_correct": 1043, + "selected_both_wrong": 839 + }, + { + "budget": 0.68, + "threshold": -0.8986453413963318, + "operator": "ge", + "actual_fraction": 0.8688298239558163, + "metric": 0.35174318260269244, + "rescued": 232, + "harmed": 2912, + "missed_rescue": 0, + "avoided_harm": 499, + "selected_both_correct": 1051, + "selected_both_wrong": 839 + }, + { + "budget": 0.69, + "threshold": -0.8996874094009399, + "operator": "ge", + "actual_fraction": 0.8712461166724197, + "metric": 0.3496720745598895, + "rescued": 232, + "harmed": 2924, + "missed_rescue": 0, + "avoided_harm": 487, + "selected_both_correct": 1053, + "selected_both_wrong": 839 + }, + { + "budget": 0.7, + "threshold": -0.9008907079696655, + "operator": "ge", + "actual_fraction": 0.8746979634104246, + "metric": 0.3470831895063859, + "rescued": 232, + "harmed": 2939, + "missed_rescue": 0, + "avoided_harm": 472, + "selected_both_correct": 1058, + "selected_both_wrong": 839 + }, + { + "budget": 0.71, + "threshold": -0.9031620621681213, + "operator": "ge", + "actual_fraction": 0.8817742492233345, + "metric": 0.34311356575768037, + "rescued": 232, + "harmed": 2962, + "missed_rescue": 0, + "avoided_harm": 449, + "selected_both_correct": 1076, + "selected_both_wrong": 839 + }, + { + "budget": 0.72, + "threshold": -0.9056642055511475, + "operator": "ge", + "actual_fraction": 0.8893683120469451, + "metric": 0.3384535726613738, + "rescued": 232, + "harmed": 2989, + "missed_rescue": 0, + "avoided_harm": 422, + "selected_both_correct": 1092, + "selected_both_wrong": 840 + }, + { + "budget": 0.73, + "threshold": -0.9068957567214966, + "operator": "ge", + "actual_fraction": 0.8933379357956507, + "metric": 0.3369002416292717, + "rescued": 232, + "harmed": 2998, + "missed_rescue": 0, + "avoided_harm": 413, + "selected_both_correct": 1106, + "selected_both_wrong": 840 + }, + { + "budget": 0.74, + "threshold": -0.9093236327171326, + "operator": "ge", + "actual_fraction": 0.8981705212288574, + "metric": 0.3336209872281671, + "rescued": 232, + "harmed": 3017, + "missed_rescue": 0, + "avoided_harm": 394, + "selected_both_correct": 1115, + "selected_both_wrong": 840 + }, + { + "budget": 0.75, + "threshold": -0.9112368822097778, + "operator": "ge", + "actual_fraction": 0.9023127373144633, + "metric": 0.3308595098377632, + "rescued": 232, + "harmed": 3033, + "missed_rescue": 0, + "avoided_harm": 378, + "selected_both_correct": 1123, + "selected_both_wrong": 840 + }, + { + "budget": 0.76, + "threshold": -0.9129221439361572, + "operator": "ge", + "actual_fraction": 0.9081808767690714, + "metric": 0.32688988608905767, + "rescued": 232, + "harmed": 3056, + "missed_rescue": 0, + "avoided_harm": 355, + "selected_both_correct": 1133, + "selected_both_wrong": 841 + }, + { + "budget": 0.77, + "threshold": -0.9144289493560791, + "operator": "ge", + "actual_fraction": 0.9123230928546773, + "metric": 0.32361063168795307, + "rescued": 232, + "harmed": 3075, + "missed_rescue": 0, + "avoided_harm": 336, + "selected_both_correct": 1138, + "selected_both_wrong": 841 + }, + { + "budget": 0.78, + "threshold": -0.9158422946929932, + "operator": "ge", + "actual_fraction": 0.9149119779081809, + "metric": 0.3217121159820504, + "rescued": 232, + "harmed": 3086, + "missed_rescue": 0, + "avoided_harm": 325, + "selected_both_correct": 1142, + "selected_both_wrong": 841 + }, + { + "budget": 0.79, + "threshold": -0.9167781472206116, + "operator": "ge", + "actual_fraction": 0.9168104936140835, + "metric": 0.32033137728684846, + "rescued": 232, + "harmed": 3094, + "missed_rescue": 0, + "avoided_harm": 317, + "selected_both_correct": 1145, + "selected_both_wrong": 841 + }, + { + "budget": 0.8, + "threshold": -0.9187189936637878, + "operator": "ge", + "actual_fraction": 0.922851225405592, + "metric": 0.3161891612012427, + "rescued": 232, + "harmed": 3118, + "missed_rescue": 0, + "avoided_harm": 293, + "selected_both_correct": 1156, + "selected_both_wrong": 841 + }, + { + "budget": 0.81, + "threshold": -0.9207327365875244, + "operator": "ge", + "actual_fraction": 0.9280289955125992, + "metric": 0.3127373144632378, + "rescued": 232, + "harmed": 3138, + "missed_rescue": 0, + "avoided_harm": 273, + "selected_both_correct": 1166, + "selected_both_wrong": 841 + }, + { + "budget": 0.82, + "threshold": -0.9226345419883728, + "operator": "ge", + "actual_fraction": 0.9342423196410079, + "metric": 0.3079047290300311, + "rescued": 232, + "harmed": 3166, + "missed_rescue": 0, + "avoided_harm": 245, + "selected_both_correct": 1174, + "selected_both_wrong": 841 + }, + { + "budget": 0.83, + "threshold": -0.9243993759155273, + "operator": "ge", + "actual_fraction": 0.9387297204004142, + "metric": 0.30479806696582673, + "rescued": 232, + "harmed": 3184, + "missed_rescue": 0, + "avoided_harm": 227, + "selected_both_correct": 1182, + "selected_both_wrong": 841 + }, + { + "budget": 0.84, + "threshold": -0.9264947175979614, + "operator": "ge", + "actual_fraction": 0.9440800828443218, + "metric": 0.3015188125647221, + "rescued": 232, + "harmed": 3203, + "missed_rescue": 0, + "avoided_harm": 208, + "selected_both_correct": 1194, + "selected_both_wrong": 841 + }, + { + "budget": 0.85, + "threshold": -0.9274600148200989, + "operator": "ge", + "actual_fraction": 0.9463237832240249, + "metric": 0.2999654815326199, + "rescued": 232, + "harmed": 3212, + "missed_rescue": 0, + "avoided_harm": 199, + "selected_both_correct": 1198, + "selected_both_wrong": 841 + }, + { + "budget": 0.86, + "threshold": -0.930335283279419, + "operator": "ge", + "actual_fraction": 0.9534000690369347, + "metric": 0.29530548843631343, + "rescued": 232, + "harmed": 3239, + "missed_rescue": 0, + "avoided_harm": 172, + "selected_both_correct": 1211, + "selected_both_wrong": 842 + }, + { + "budget": 0.87, + "threshold": -0.9319414496421814, + "operator": "ge", + "actual_fraction": 0.9558163617535381, + "metric": 0.2939247497411115, + "rescued": 232, + "harmed": 3247, + "missed_rescue": 0, + "avoided_harm": 164, + "selected_both_correct": 1217, + "selected_both_wrong": 842 + }, + { + "budget": 0.88, + "threshold": -0.9340114593505859, + "operator": "ge", + "actual_fraction": 0.958577839143942, + "metric": 0.29202623403520883, + "rescued": 232, + "harmed": 3258, + "missed_rescue": 0, + "avoided_harm": 153, + "selected_both_correct": 1222, + "selected_both_wrong": 842 + }, + { + "budget": 0.89, + "threshold": -0.9363258481025696, + "operator": "ge", + "actual_fraction": 0.9628926475664481, + "metric": 0.28874697963410423, + "rescued": 232, + "harmed": 3277, + "missed_rescue": 0, + "avoided_harm": 134, + "selected_both_correct": 1228, + "selected_both_wrong": 842 + }, + { + "budget": 0.9, + "threshold": -0.9400071501731873, + "operator": "ge", + "actual_fraction": 0.9694511563686572, + "metric": 0.2839143942008975, + "rescued": 232, + "harmed": 3305, + "missed_rescue": 0, + "avoided_harm": 106, + "selected_both_correct": 1238, + "selected_both_wrong": 842 + }, + { + "budget": 0.91, + "threshold": -0.9423198103904724, + "operator": "ge", + "actual_fraction": 0.9734207801173628, + "metric": 0.2823610631687953, + "rescued": 232, + "harmed": 3314, + "missed_rescue": 0, + "avoided_harm": 97, + "selected_both_correct": 1251, + "selected_both_wrong": 843 + }, + { + "budget": 0.92, + "threshold": -0.9443356394767761, + "operator": "ge", + "actual_fraction": 0.9761822575077667, + "metric": 0.2806351397997929, + "rescued": 232, + "harmed": 3324, + "missed_rescue": 0, + "avoided_harm": 87, + "selected_both_correct": 1256, + "selected_both_wrong": 844 + }, + { + "budget": 0.93, + "threshold": -0.9468213319778442, + "operator": "ge", + "actual_fraction": 0.9803244735933725, + "metric": 0.27839143942008976, + "rescued": 232, + "harmed": 3337, + "missed_rescue": 0, + "avoided_harm": 74, + "selected_both_correct": 1267, + "selected_both_wrong": 844 + }, + { + "budget": 0.94, + "threshold": -0.9496435523033142, + "operator": "ge", + "actual_fraction": 0.9837763203313773, + "metric": 0.27632033137728684, + "rescued": 232, + "harmed": 3349, + "missed_rescue": 0, + "avoided_harm": 62, + "selected_both_correct": 1275, + "selected_both_wrong": 844 + }, + { + "budget": 0.95, + "threshold": -0.9529175758361816, + "operator": "ge", + "actual_fraction": 0.9860200207110804, + "metric": 0.2752847773558854, + "rescued": 232, + "harmed": 3355, + "missed_rescue": 0, + "avoided_harm": 56, + "selected_both_correct": 1282, + "selected_both_wrong": 844 + }, + { + "budget": 0.96, + "threshold": -0.9560379981994629, + "operator": "ge", + "actual_fraction": 0.988608905764584, + "metric": 0.27338626164998275, + "rescued": 232, + "harmed": 3366, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 1286, + "selected_both_wrong": 844 + }, + { + "budget": 0.97, + "threshold": -0.9591385126113892, + "operator": "ge", + "actual_fraction": 0.9920607525025888, + "metric": 0.2711425612702796, + "rescued": 232, + "harmed": 3379, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 1293, + "selected_both_wrong": 844 + }, + { + "budget": 0.98, + "threshold": -0.9627243280410767, + "operator": "ge", + "actual_fraction": 0.9946496375560925, + "metric": 0.2697618225750777, + "rescued": 232, + "harmed": 3387, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 1300, + "selected_both_wrong": 844 + }, + { + "budget": 0.99, + "threshold": -0.9702208638191223, + "operator": "ge", + "actual_fraction": 0.9972385226095961, + "metric": 0.2676907145322748, + "rescued": 232, + "harmed": 3399, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 1303, + "selected_both_wrong": 844 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.26561960648947186, + "rescued": 232, + "harmed": 3411, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1307, + "selected_both_wrong": 844 + } + ] + }, + "learned": { + "validation_area_0_25": 0.9743333333333333, + "validation_point": { + "metric": 0.9865, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.01, + "actual_fraction_std": 0.0, + "per_seed": [ + { + "seed": 2026, + "budget": 0.01, + "threshold": -0.004982936196029186, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9866666666666667, + "rescue": 1, + "harm": 0, + "missed_rescue": 5 + }, + { + "seed": 2027, + "budget": 0.01, + "threshold": -0.0061280992813408375, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescue": 0, + "harm": 0, + "missed_rescue": 6 + }, + { + "seed": 2028, + "budget": 0.01, + "threshold": -0.000379358563804999, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9875, + "rescue": 2, + "harm": 0, + "missed_rescue": 4 + }, + { + "seed": 2029, + "budget": 0.01, + "threshold": -0.003421929432079196, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescue": 0, + "harm": 0, + "missed_rescue": 6 + }, + { + "seed": 2030, + "budget": 0.01, + "threshold": -0.004842729773372412, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9866666666666667, + "rescue": 1, + "harm": 0, + "missed_rescue": 5 + } + ], + "rescued": 0.8, + "harmed": 0.0, + "missed_rescue": 5.2, + "avoided_harm": 856.0 + }, + "test_point": { + "metric": 0.8139454608215395, + "metric_std": 0.000585289090708094, + "actual_fraction": 0.013841905419399378, + "actual_fraction_std": 0.0046301734514364885, + "per_seed": [ + { + "seed": 2026, + "budget": 0.01, + "threshold": -0.004982936196029186, + "operator": "ge", + "actual_fraction": 0.013634794615119088, + "metric": 0.8139454608215395, + "rescue": 6, + "harm": 8, + "missed_rescue": 226 + }, + { + "seed": 2027, + "budget": 0.01, + "threshold": -0.0061280992813408375, + "operator": "ge", + "actual_fraction": 0.006731101139109423, + "metric": 0.81429064549534, + "rescue": 0, + "harm": 0, + "missed_rescue": 232 + }, + { + "seed": 2028, + "budget": 0.01, + "threshold": -0.000379358563804999, + "operator": "ge", + "actual_fraction": 0.019330341732827064, + "metric": 0.8130824991370383, + "rescue": 10, + "harm": 17, + "missed_rescue": 222 + }, + { + "seed": 2029, + "budget": 0.01, + "threshold": -0.003421929432079196, + "operator": "ge", + "actual_fraction": 0.013462202278218847, + "metric": 0.8137728684846393, + "rescue": 3, + "harm": 6, + "missed_rescue": 229 + }, + { + "seed": 2030, + "budget": 0.01, + "threshold": -0.004842729773372412, + "operator": "ge", + "actual_fraction": 0.01605108733172247, + "metric": 0.8146358301691405, + "rescue": 17, + "harm": 15, + "missed_rescue": 215 + } + ], + "rescued": 7.2, + "harmed": 9.2, + "missed_rescue": 224.8, + "avoided_harm": 3401.8 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.9865, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.01, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.9865, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.02, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.9865, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.03, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.9865, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.04, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.9865, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.05, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.9865, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.06, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.9863333333333333, + "metric_std": 0.0007453559924999471, + "actual_fraction": 0.07, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.986, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.08, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.9855, + "metric_std": 0.0013944333775568248, + "actual_fraction": 0.09, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.9838333333333333, + "metric_std": 0.0020916500663351796, + "actual_fraction": 0.1, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.9823333333333333, + "metric_std": 0.002386303510546062, + "actual_fraction": 0.11000000000000001, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.12, + "metric": 0.9803333333333333, + "metric_std": 0.0022515426810176963, + "actual_fraction": 0.12, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.9781666666666666, + "metric_std": 0.002527625148017183, + "actual_fraction": 0.13, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.976, + "metric_std": 0.002236067977499802, + "actual_fraction": 0.14, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.9743333333333334, + "metric_std": 0.0019895560643855463, + "actual_fraction": 0.15, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.9720000000000001, + "metric_std": 0.0011180339887499207, + "actual_fraction": 0.16, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.9703333333333333, + "metric_std": 0.0007453559924999471, + "actual_fraction": 0.17, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.9686666666666668, + "metric_std": 0.0007453559924998913, + "actual_fraction": 0.18, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.9655000000000001, + "metric_std": 0.0015138251770487807, + "actual_fraction": 0.19, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.9631666666666667, + "metric_std": 0.001707825127659913, + "actual_fraction": 0.2, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.9598333333333334, + "metric_std": 0.002312406154251932, + "actual_fraction": 0.21000000000000002, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.9550000000000001, + "metric_std": 0.004208127057650874, + "actual_fraction": 0.22000000000000003, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.23, + "metric": 0.9508333333333333, + "metric_std": 0.004750730937913067, + "actual_fraction": 0.23000000000000004, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.24, + "metric": 0.9466666666666665, + "metric_std": 0.0058034951154933975, + "actual_fraction": 0.24, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.9418333333333333, + "metric_std": 0.005815448774122637, + "actual_fraction": 0.25, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.26, + "metric": 0.9365, + "metric_std": 0.00649251706026083, + "actual_fraction": 0.26, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.9315, + "metric_std": 0.005477225575051653, + "actual_fraction": 0.27, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.9258333333333333, + "metric_std": 0.0048591265790377675, + "actual_fraction": 0.28, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.9203333333333333, + "metric_std": 0.003660980439414788, + "actual_fraction": 0.29, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.9131666666666666, + "metric_std": 0.004385899122313575, + "actual_fraction": 0.3, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.9061666666666666, + "metric_std": 0.003890872509976244, + "actual_fraction": 0.31, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.8985000000000001, + "metric_std": 0.0037914377220257763, + "actual_fraction": 0.32, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.8911666666666666, + "metric_std": 0.00443314535942344, + "actual_fraction": 0.33, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.8826666666666666, + "metric_std": 0.0036036247184066074, + "actual_fraction": 0.34, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.8756666666666666, + "metric_std": 0.00285043856274782, + "actual_fraction": 0.35, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.8684999999999998, + "metric_std": 0.0016029486718059363, + "actual_fraction": 0.36, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.8611666666666666, + "metric_std": 0.0029814239699996986, + "actual_fraction": 0.37, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.8530000000000001, + "metric_std": 0.0030390970881350892, + "actual_fraction": 0.38, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.8453333333333333, + "metric_std": 0.0029814239699997003, + "actual_fraction": 0.39, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.8375, + "metric_std": 0.002357022603955154, + "actual_fraction": 0.4, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.8295, + "metric_std": 0.002400810048481318, + "actual_fraction": 0.4091666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.42, + "metric": 0.8201666666666666, + "metric_std": 0.0022360679774997834, + "actual_fraction": 0.42000000000000004, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.8116666666666668, + "metric_std": 0.002357022603955164, + "actual_fraction": 0.43, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.44, + "metric": 0.8028333333333334, + "metric_std": 0.002738612787525806, + "actual_fraction": 0.44000000000000006, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.45, + "metric": 0.7941666666666667, + "metric_std": 0.002357022603955164, + "actual_fraction": 0.45, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.7861666666666667, + "metric_std": 0.0028625940062196225, + "actual_fraction": 0.4600000000000001, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.47, + "metric": 0.7778333333333334, + "metric_std": 0.0027386127875258176, + "actual_fraction": 0.4699999999999999, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.48, + "metric": 0.7691666666666667, + "metric_std": 0.0029462782549439454, + "actual_fraction": 0.48, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.7601666666666667, + "metric_std": 0.0031402406135694607, + "actual_fraction": 0.49000000000000005, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.7518333333333335, + "metric_std": 0.003698723503528679, + "actual_fraction": 0.5, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.51, + "metric": 0.7423333333333334, + "metric_std": 0.0034054694569504063, + "actual_fraction": 0.51, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.7331666666666667, + "metric_std": 0.0030844592538869365, + "actual_fraction": 0.52, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.7243333333333333, + "metric_std": 0.0027888667551135963, + "actual_fraction": 0.53, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.7148333333333333, + "metric_std": 0.0025954019170662565, + "actual_fraction": 0.54, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.7056666666666667, + "metric_std": 0.0023124061542519002, + "actual_fraction": 0.55, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.6975, + "metric_std": 0.002282177322938204, + "actual_fraction": 0.56, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.6893333333333335, + "metric_std": 0.002074983266331461, + "actual_fraction": 0.5691666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.6791666666666667, + "metric_std": 0.00212459146399701, + "actual_fraction": 0.58, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.6696666666666667, + "metric_std": 0.001513825177048744, + "actual_fraction": 0.59, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.6603333333333333, + "metric_std": 0.001513825177048741, + "actual_fraction": 0.6, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.6511666666666667, + "metric_std": 0.0017280367794439658, + "actual_fraction": 0.61, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.6421666666666667, + "metric_std": 0.0019184774055368834, + "actual_fraction": 0.62, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.633, + "metric_std": 0.002327373340628165, + "actual_fraction": 0.63, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.6236666666666666, + "metric_std": 0.002608745973749767, + "actual_fraction": 0.64, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.6146666666666667, + "metric_std": 0.0017280367794439684, + "actual_fraction": 0.65, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.6048333333333333, + "metric_std": 0.0018066236157232266, + "actual_fraction": 0.66, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.5958333333333333, + "metric_std": 0.0014433756729740495, + "actual_fraction": 0.67, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.5863333333333333, + "metric_std": 0.001263812574008621, + "actual_fraction": 0.68, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.5778333333333332, + "metric_std": 0.001263812574008621, + "actual_fraction": 0.6891666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.5671666666666667, + "metric_std": 0.0013944333775568248, + "actual_fraction": 0.7, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.5573333333333335, + "metric_std": 0.0014907119849998508, + "actual_fraction": 0.71, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.5476666666666666, + "metric_std": 0.0018066236157232342, + "actual_fraction": 0.72, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.5393333333333332, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.73, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.5295, + "metric_std": 0.0007453559924999471, + "actual_fraction": 0.74, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.52, + "metric_std": 0.0005892556509888032, + "actual_fraction": 0.75, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.5101666666666667, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.76, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.5005, + "metric_std": 0.0007453559924999471, + "actual_fraction": 0.77, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.4911666666666667, + "metric_std": 0.0007453559924999191, + "actual_fraction": 0.78, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.48116666666666674, + "metric_std": 0.0007453559924999471, + "actual_fraction": 0.79, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.4715, + "metric_std": 0.0009128709291752828, + "actual_fraction": 0.8, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.4616666666666666, + "metric_std": 0.0005892556509888032, + "actual_fraction": 0.8100000000000002, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.82, + "metric": 0.4528333333333333, + "metric_std": 0.0004564354645876186, + "actual_fraction": 0.8191666666666666, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.83, + "metric": 0.442, + "metric_std": 0.000456435464587649, + "actual_fraction": 0.8299999999999998, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.43216666666666664, + "metric_std": 0.000456435464587649, + "actual_fraction": 0.8400000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.42299999999999993, + "metric_std": 0.0007453559924999471, + "actual_fraction": 0.85, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.4133333333333333, + "metric_std": 0.0005892556509888032, + "actual_fraction": 0.86, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.87, + "metric": 0.4035, + "metric_std": 0.0006972166887783926, + "actual_fraction": 0.8699999999999999, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.88, + "metric": 0.39383333333333337, + "metric_std": 0.0009501461875826174, + "actual_fraction": 0.8800000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.89, + "metric": 0.38466666666666666, + "metric_std": 0.0009501461875826004, + "actual_fraction": 0.89, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.3748333333333333, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.9, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.36500000000000005, + "metric_std": 0.0010206207261596358, + "actual_fraction": 0.9099999999999999, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.3556666666666667, + "metric_std": 0.001086533734200443, + "actual_fraction": 0.9200000000000002, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.93, + "metric": 0.3456666666666667, + "metric_std": 0.001086533734200443, + "actual_fraction": 0.93, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.3358333333333333, + "metric_std": 0.0010206207261596585, + "actual_fraction": 0.9399999999999998, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.95, + "metric": 0.32599999999999996, + "metric_std": 0.001086533734200441, + "actual_fraction": 0.95, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.3165, + "metric_std": 0.0006972166887783858, + "actual_fraction": 0.96, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.30683333333333335, + "metric_std": 0.0009128709291752675, + "actual_fraction": 0.97, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.98, + "metric": 0.29700000000000004, + "metric_std": 0.0007453559924999191, + "actual_fraction": 0.9800000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.28733333333333333, + "metric_std": 0.0003726779962499487, + "actual_fraction": 0.99, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.2775, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.8142906454953401, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.8139454608215395, + "metric_std": 0.000585289090708094, + "actual_fraction": 0.013841905419399378, + "actual_fraction_std": 0.0046301734514364885 + }, + { + "budget": 0.02, + "metric": 0.8139109423541594, + "metric_std": 0.0006503777591100172, + "actual_fraction": 0.01922678633068692, + "actual_fraction_std": 0.005734119031588948 + }, + { + "budget": 0.03, + "metric": 0.8139109423541594, + "metric_std": 0.0005375357784741999, + "actual_fraction": 0.025681739730755947, + "actual_fraction_std": 0.004526067445163992 + }, + { + "budget": 0.04, + "metric": 0.8139799792889196, + "metric_std": 0.0005375357784741965, + "actual_fraction": 0.03161891612012426, + "actual_fraction_std": 0.00558250173832892 + }, + { + "budget": 0.05, + "metric": 0.8140835346910598, + "metric_std": 0.0004120585566556126, + "actual_fraction": 0.03886779426993441, + "actual_fraction_std": 0.010207344854782873 + }, + { + "budget": 0.06, + "metric": 0.8139799792889196, + "metric_std": 0.00047892429842981074, + "actual_fraction": 0.04794615119088712, + "actual_fraction_std": 0.009101198311426808 + }, + { + "budget": 0.07, + "metric": 0.8138764238867795, + "metric_std": 0.0006968115028714815, + "actual_fraction": 0.062478425957887475, + "actual_fraction_std": 0.009552794855805766 + }, + { + "budget": 0.08, + "metric": 0.8138419054193994, + "metric_std": 0.0005402994767862798, + "actual_fraction": 0.07507766655160511, + "actual_fraction_std": 0.00856465017314429 + }, + { + "budget": 0.09, + "metric": 0.8138419054193994, + "metric_std": 0.0005539111679627642, + "actual_fraction": 0.08477735588539868, + "actual_fraction_std": 0.0065441861667963125 + }, + { + "budget": 0.1, + "metric": 0.813565757680359, + "metric_std": 0.0008918198607361813, + "actual_fraction": 0.09413186054539177, + "actual_fraction_std": 0.005572086863186391 + }, + { + "budget": 0.11, + "metric": 0.812875388332758, + "metric_std": 0.0012409857102000325, + "actual_fraction": 0.106420434932689, + "actual_fraction_std": 0.007135602606199722 + }, + { + "budget": 0.12, + "metric": 0.8125302036589576, + "metric_std": 0.0015552475424327151, + "actual_fraction": 0.12088367276492921, + "actual_fraction_std": 0.005672750688303549 + }, + { + "budget": 0.13, + "metric": 0.811460131170176, + "metric_std": 0.0018085173486268463, + "actual_fraction": 0.13648602002071106, + "actual_fraction_std": 0.007895415616122928 + }, + { + "budget": 0.14, + "metric": 0.8108387987573351, + "metric_std": 0.0023696117048232572, + "actual_fraction": 0.1462202278218847, + "actual_fraction_std": 0.007472869511396553 + }, + { + "budget": 0.15, + "metric": 0.809389023127373, + "metric_std": 0.0032578375170974195, + "actual_fraction": 0.15864687607870212, + "actual_fraction_std": 0.008458257220676963 + }, + { + "budget": 0.16, + "metric": 0.8084915429754919, + "metric_std": 0.0029759043192218236, + "actual_fraction": 0.16876078702105626, + "actual_fraction_std": 0.007785234556224363 + }, + { + "budget": 0.17, + "metric": 0.8066965826717294, + "metric_std": 0.0032678799152160267, + "actual_fraction": 0.18312046945115637, + "actual_fraction_std": 0.008539397222542624 + }, + { + "budget": 0.18, + "metric": 0.804970659302727, + "metric_std": 0.00295178311641132, + "actual_fraction": 0.19333793579565067, + "actual_fraction_std": 0.006771252929369274 + }, + { + "budget": 0.19, + "metric": 0.8023817742492234, + "metric_std": 0.004056837857626224, + "actual_fraction": 0.2050742147048671, + "actual_fraction_std": 0.00677235264076452 + }, + { + "budget": 0.2, + "metric": 0.7990680013807385, + "metric_std": 0.004404318372864773, + "actual_fraction": 0.21829478771142563, + "actual_fraction_std": 0.006371205849675289 + }, + { + "budget": 0.21, + "metric": 0.7965481532619952, + "metric_std": 0.004474770467794362, + "actual_fraction": 0.2317915084570245, + "actual_fraction_std": 0.008635655464026155 + }, + { + "budget": 0.22, + "metric": 0.7927511218501898, + "metric_std": 0.004441361231861015, + "actual_fraction": 0.24383845357266137, + "actual_fraction_std": 0.007862333988268112 + }, + { + "budget": 0.23, + "metric": 0.7884363134276837, + "metric_std": 0.0039003577023447485, + "actual_fraction": 0.2556437694166379, + "actual_fraction_std": 0.006877106208223859 + }, + { + "budget": 0.24, + "metric": 0.7840179496030375, + "metric_std": 0.0040744222110504665, + "actual_fraction": 0.2705557473248188, + "actual_fraction_std": 0.005383110442421251 + }, + { + "budget": 0.25, + "metric": 0.7779772178115293, + "metric_std": 0.004339933750721671, + "actual_fraction": 0.287124611667242, + "actual_fraction_std": 0.005909366034462896 + }, + { + "budget": 0.26, + "metric": 0.7714877459440801, + "metric_std": 0.004505618737385786, + "actual_fraction": 0.3032792544011046, + "actual_fraction_std": 0.006281744631723168 + }, + { + "budget": 0.27, + "metric": 0.7661719019675527, + "metric_std": 0.005300580141692667, + "actual_fraction": 0.3185018985157059, + "actual_fraction_std": 0.011842777575759293 + }, + { + "budget": 0.28, + "metric": 0.7576803589920609, + "metric_std": 0.0051676924133862295, + "actual_fraction": 0.3376941663790128, + "actual_fraction_std": 0.015296881311175799 + }, + { + "budget": 0.29, + "metric": 0.7507076285812909, + "metric_std": 0.004905388781783933, + "actual_fraction": 0.35326199516741463, + "actual_fraction_std": 0.01632955194937787 + }, + { + "budget": 0.3, + "metric": 0.7442871936486021, + "metric_std": 0.0045754876298597005, + "actual_fraction": 0.3684156023472558, + "actual_fraction_std": 0.017227533884897585 + }, + { + "budget": 0.31, + "metric": 0.7382464618570935, + "metric_std": 0.003879682253297783, + "actual_fraction": 0.379979288919572, + "actual_fraction_std": 0.01868291120505315 + }, + { + "budget": 0.32, + "metric": 0.7319986192613047, + "metric_std": 0.004560489197785538, + "actual_fraction": 0.39375215740421127, + "actual_fraction_std": 0.020980655571120353 + }, + { + "budget": 0.33, + "metric": 0.7240593717638937, + "metric_std": 0.005439260572572439, + "actual_fraction": 0.40731791508457016, + "actual_fraction_std": 0.02208440180107282 + }, + { + "budget": 0.34, + "metric": 0.7170176044183638, + "metric_std": 0.007647418237667047, + "actual_fraction": 0.4209181912323093, + "actual_fraction_std": 0.02605755639707729 + }, + { + "budget": 0.35, + "metric": 0.7073524335519503, + "metric_std": 0.00949304922421683, + "actual_fraction": 0.438557128063514, + "actual_fraction_std": 0.029715970013567017 + }, + { + "budget": 0.36, + "metric": 0.7002761477390403, + "metric_std": 0.008215467753920054, + "actual_fraction": 0.4517777010700724, + "actual_fraction_std": 0.0281850737229077 + }, + { + "budget": 0.37, + "metric": 0.6930617880566102, + "metric_std": 0.007981905777544684, + "actual_fraction": 0.4646185709354505, + "actual_fraction_std": 0.025539000777485547 + }, + { + "budget": 0.38, + "metric": 0.6841215050051777, + "metric_std": 0.008936050538590852, + "actual_fraction": 0.47897825336555055, + "actual_fraction_std": 0.028440944424955275 + }, + { + "budget": 0.39, + "metric": 0.6765964791163273, + "metric_std": 0.0095124842284435, + "actual_fraction": 0.4916810493614083, + "actual_fraction_std": 0.02920367403537206 + }, + { + "budget": 0.4, + "metric": 0.6686572316189161, + "metric_std": 0.008042136206240101, + "actual_fraction": 0.5043148084225061, + "actual_fraction_std": 0.028183858289840366 + }, + { + "budget": 0.41, + "metric": 0.661373835001726, + "metric_std": 0.008583758026101637, + "actual_fraction": 0.5161201242664826, + "actual_fraction_std": 0.029769451531783862 + }, + { + "budget": 0.42, + "metric": 0.6537107352433551, + "metric_std": 0.009838974303060239, + "actual_fraction": 0.5276492923714187, + "actual_fraction_std": 0.03130598538990098 + }, + { + "budget": 0.43, + "metric": 0.6469451156368657, + "metric_std": 0.011876685556473133, + "actual_fraction": 0.5375560925094925, + "actual_fraction_std": 0.03417820733476067 + }, + { + "budget": 0.44, + "metric": 0.6393510528132551, + "metric_std": 0.009955601148595852, + "actual_fraction": 0.5497411114946497, + "actual_fraction_std": 0.03154131449717005 + }, + { + "budget": 0.45, + "metric": 0.6316189161201243, + "metric_std": 0.010129126500170465, + "actual_fraction": 0.560752502588885, + "actual_fraction_std": 0.03187939781757573 + }, + { + "budget": 0.46, + "metric": 0.6227476700034519, + "metric_std": 0.01112143021794921, + "actual_fraction": 0.574249223334484, + "actual_fraction_std": 0.034218449386572244 + }, + { + "budget": 0.47, + "metric": 0.6167069382119433, + "metric_std": 0.012279189731571055, + "actual_fraction": 0.5837072833966171, + "actual_fraction_std": 0.037013512125062493 + }, + { + "budget": 0.48, + "metric": 0.6101484294097341, + "metric_std": 0.012505145582446338, + "actual_fraction": 0.5937521574042113, + "actual_fraction_std": 0.03625659343868569 + }, + { + "budget": 0.49, + "metric": 0.6016568864342424, + "metric_std": 0.013494800464067732, + "actual_fraction": 0.6062478425957888, + "actual_fraction_std": 0.03689735959603709 + }, + { + "budget": 0.5, + "metric": 0.5924749741111495, + "metric_std": 0.013508258754000633, + "actual_fraction": 0.620089748015188, + "actual_fraction_std": 0.03632574658716115 + }, + { + "budget": 0.51, + "metric": 0.5854332067656196, + "metric_std": 0.012427247521315432, + "actual_fraction": 0.6296858819468415, + "actual_fraction_std": 0.03533311231532198 + }, + { + "budget": 0.52, + "metric": 0.5760441836382464, + "metric_std": 0.013463087338032318, + "actual_fraction": 0.6416983085950984, + "actual_fraction_std": 0.036165611245960815 + }, + { + "budget": 0.53, + "metric": 0.5684501208146358, + "metric_std": 0.01200205671292983, + "actual_fraction": 0.6516396272005524, + "actual_fraction_std": 0.03475895687230668 + }, + { + "budget": 0.54, + "metric": 0.5610286503279254, + "metric_std": 0.011178335465593657, + "actual_fraction": 0.6617190196755264, + "actual_fraction_std": 0.03351981355120372 + }, + { + "budget": 0.55, + "metric": 0.553607179841215, + "metric_std": 0.013108271524155835, + "actual_fraction": 0.67134967207456, + "actual_fraction_std": 0.03306109717549356 + }, + { + "budget": 0.56, + "metric": 0.5462547462892647, + "metric_std": 0.013505832840390471, + "actual_fraction": 0.6801173627890922, + "actual_fraction_std": 0.03351905817161892 + }, + { + "budget": 0.57, + "metric": 0.5397997928891958, + "metric_std": 0.01354690434722983, + "actual_fraction": 0.6891957197100449, + "actual_fraction_std": 0.03389284625086914 + }, + { + "budget": 0.58, + "metric": 0.5346910597169485, + "metric_std": 0.011207211333403551, + "actual_fraction": 0.695650673110114, + "actual_fraction_std": 0.030304236110951843 + }, + { + "budget": 0.59, + "metric": 0.5282706247842597, + "metric_std": 0.011475378813132224, + "actual_fraction": 0.7034518467380049, + "actual_fraction_std": 0.029038987658191705 + }, + { + "budget": 0.6, + "metric": 0.5199516741456679, + "metric_std": 0.012303545894193654, + "actual_fraction": 0.7131515360717984, + "actual_fraction_std": 0.02956018535585732 + }, + { + "budget": 0.61, + "metric": 0.5124956851915774, + "metric_std": 0.012093539011901359, + "actual_fraction": 0.7222298929927511, + "actual_fraction_std": 0.029781206577303283 + }, + { + "budget": 0.62, + "metric": 0.5067311011391095, + "metric_std": 0.012435035331753566, + "actual_fraction": 0.7290300310666207, + "actual_fraction_std": 0.030116250067224324 + }, + { + "budget": 0.63, + "metric": 0.4982740766309976, + "metric_std": 0.012516812276406183, + "actual_fraction": 0.7399033482913359, + "actual_fraction_std": 0.028966475836887152 + }, + { + "budget": 0.64, + "metric": 0.492129789437349, + "metric_std": 0.014269744036028023, + "actual_fraction": 0.7474628926475664, + "actual_fraction_std": 0.030821483274999294 + }, + { + "budget": 0.65, + "metric": 0.48450120814635833, + "metric_std": 0.013725920348071921, + "actual_fraction": 0.7566793234380393, + "actual_fraction_std": 0.02845508037032804 + }, + { + "budget": 0.66, + "metric": 0.47787366240938906, + "metric_std": 0.014882223988333037, + "actual_fraction": 0.7644804970659302, + "actual_fraction_std": 0.029475062397104238 + }, + { + "budget": 0.67, + "metric": 0.4724197445633414, + "metric_std": 0.014007332917720884, + "actual_fraction": 0.7710390058681394, + "actual_fraction_std": 0.028169744859295406 + }, + { + "budget": 0.68, + "metric": 0.46489471867449084, + "metric_std": 0.015128483800766335, + "actual_fraction": 0.7794615119088713, + "actual_fraction_std": 0.029284398463229517 + }, + { + "budget": 0.69, + "metric": 0.4592336900241629, + "metric_std": 0.016001087586682106, + "actual_fraction": 0.7862271315153606, + "actual_fraction_std": 0.029682018436726767 + }, + { + "budget": 0.7, + "metric": 0.4529168104936141, + "metric_std": 0.015458050334492785, + "actual_fraction": 0.7936831204694511, + "actual_fraction_std": 0.027867009062399667 + }, + { + "budget": 0.71, + "metric": 0.44742837418018644, + "metric_std": 0.014888827771060742, + "actual_fraction": 0.8001035554021403, + "actual_fraction_std": 0.02648305544139038 + }, + { + "budget": 0.72, + "metric": 0.4394200897480152, + "metric_std": 0.012229722981054096, + "actual_fraction": 0.8095270969968933, + "actual_fraction_std": 0.02338693035611874 + }, + { + "budget": 0.73, + "metric": 0.43441491197790827, + "metric_std": 0.012716319473679186, + "actual_fraction": 0.8151536071798411, + "actual_fraction_std": 0.023034126985599448 + }, + { + "budget": 0.74, + "metric": 0.42878840179496025, + "metric_std": 0.010787780435818252, + "actual_fraction": 0.8218156713841905, + "actual_fraction_std": 0.02046351288578157 + }, + { + "budget": 0.75, + "metric": 0.42292026234035207, + "metric_std": 0.010907234879439185, + "actual_fraction": 0.8289264756644805, + "actual_fraction_std": 0.02003424607651972 + }, + { + "budget": 0.76, + "metric": 0.41794960303762513, + "metric_std": 0.011760746287958179, + "actual_fraction": 0.8347600966517087, + "actual_fraction_std": 0.020264574814506435 + }, + { + "budget": 0.77, + "metric": 0.41152916810493617, + "metric_std": 0.009566505368574662, + "actual_fraction": 0.8418018639972387, + "actual_fraction_std": 0.016932794836279505 + }, + { + "budget": 0.78, + "metric": 0.4046599930963065, + "metric_std": 0.010486032042739374, + "actual_fraction": 0.8493268898860891, + "actual_fraction_std": 0.017413446360950206 + }, + { + "budget": 0.79, + "metric": 0.39723852260959613, + "metric_std": 0.010262062273213579, + "actual_fraction": 0.8575077666551605, + "actual_fraction_std": 0.016776383539237574 + }, + { + "budget": 0.8, + "metric": 0.3895754228512254, + "metric_std": 0.011045505040918727, + "actual_fraction": 0.8659302726958924, + "actual_fraction_std": 0.016910702507203574 + }, + { + "budget": 0.81, + "metric": 0.3839489126682775, + "metric_std": 0.00909563252961251, + "actual_fraction": 0.8722471522264412, + "actual_fraction_std": 0.014367417959724387 + }, + { + "budget": 0.82, + "metric": 0.3781843286158095, + "metric_std": 0.009360966679768958, + "actual_fraction": 0.8784949948222298, + "actual_fraction_std": 0.014087172197039574 + }, + { + "budget": 0.83, + "metric": 0.3713496720745598, + "metric_std": 0.006792336279598953, + "actual_fraction": 0.8861926130479807, + "actual_fraction_std": 0.010070137986452189 + }, + { + "budget": 0.84, + "metric": 0.36303072143596826, + "metric_std": 0.005647752940901342, + "actual_fraction": 0.8954090438384537, + "actual_fraction_std": 0.009667482094584063 + }, + { + "budget": 0.85, + "metric": 0.35605799102519853, + "metric_std": 0.005882082564015182, + "actual_fraction": 0.9032447359337246, + "actual_fraction_std": 0.0095415625099279 + }, + { + "budget": 0.86, + "metric": 0.3491888160165688, + "metric_std": 0.005179783292894358, + "actual_fraction": 0.910700724887815, + "actual_fraction_std": 0.008545151005281621 + }, + { + "budget": 0.87, + "metric": 0.3426648256817397, + "metric_std": 0.005224447296274673, + "actual_fraction": 0.917707973765965, + "actual_fraction_std": 0.008396399894874581 + }, + { + "budget": 0.88, + "metric": 0.33672764929237137, + "metric_std": 0.005103891295347322, + "actual_fraction": 0.9242664825681739, + "actual_fraction_std": 0.008294673510965862 + }, + { + "budget": 0.89, + "metric": 0.3293406972730411, + "metric_std": 0.00531180781820402, + "actual_fraction": 0.9323092854677253, + "actual_fraction_std": 0.008113124994590572 + }, + { + "budget": 0.9, + "metric": 0.3239558163617535, + "metric_std": 0.006437023196055076, + "actual_fraction": 0.9382809803244736, + "actual_fraction_std": 0.008517217612986001 + }, + { + "budget": 0.91, + "metric": 0.31867449085260613, + "metric_std": 0.005830709679945157, + "actual_fraction": 0.9438039351052812, + "actual_fraction_std": 0.008266073894472977 + }, + { + "budget": 0.92, + "metric": 0.31284086986537796, + "metric_std": 0.007222736695559718, + "actual_fraction": 0.9502243700379702, + "actual_fraction_std": 0.009716350760360938 + }, + { + "budget": 0.93, + "metric": 0.30724887814981017, + "metric_std": 0.0057169501931568795, + "actual_fraction": 0.9563341387642389, + "actual_fraction_std": 0.007768572616449348 + }, + { + "budget": 0.94, + "metric": 0.30003451846738, + "metric_std": 0.0033234670077765807, + "actual_fraction": 0.9639627200552295, + "actual_fraction_std": 0.005289328632206555 + }, + { + "budget": 0.95, + "metric": 0.29382119433897136, + "metric_std": 0.0030719496667293124, + "actual_fraction": 0.9705557473248188, + "actual_fraction_std": 0.004272488352575995 + }, + { + "budget": 0.96, + "metric": 0.28933379357956507, + "metric_std": 0.0036786873384350924, + "actual_fraction": 0.9753883327580256, + "actual_fraction_std": 0.0045231048202415474 + }, + { + "budget": 0.97, + "metric": 0.28246461857093547, + "metric_std": 0.0038099538476119655, + "actual_fraction": 0.9826717293752157, + "actual_fraction_std": 0.0042251647510697585 + }, + { + "budget": 0.98, + "metric": 0.2777355885398688, + "metric_std": 0.002326478121816317, + "actual_fraction": 0.9876078702105626, + "actual_fraction_std": 0.0025112036743343757 + }, + { + "budget": 0.99, + "metric": 0.2715913013462202, + "metric_std": 0.0010956555228238839, + "actual_fraction": 0.9939247497411114, + "actual_fraction_std": 0.0010455734367180447 + }, + { + "budget": 1.0, + "metric": 0.26561960648947186, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 327, + "rescuable": 6, + "harmful": 856, + "both_wrong": 11, + "unit": "sample", + "total": 1200 + }, + "test": { + "both_correct": 1307, + "rescuable": 232, + "harmful": 3411, + "both_wrong": 844, + "unit": "sample", + "total": 5794 + } + }, + "small": { + "validation": 0.9858333333333333, + "test": 0.81429064549534 + }, + "large": { + "validation": 0.2775, + "test": 0.26561960648947186 + }, + "learned_minus_confidence_test": 0.014670348636520503, + "learned_minus_small_test": -0.00034518467380051376, + "learned_minus_large_test": 0.5483258543320677, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_cub_MiniCPM" + }, + { + "key": [ + "cub", + "cub", + "Qwen3.8" + ], + "expert_display": "ResNet", + "run": "five_seed_cub_Qwen3.8_four_difference", + "validation_rows": 1200, + "test_rows": 5794, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/cub/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9171666666666667, + "validation_point": { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9825, + "rescued": 1, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 519, + "selected_both_correct": 1, + "selected_both_wrong": 5 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.113565757680359, + "metric": 0.8158439765274422, + "rescued": 132, + "harmed": 123, + "missed_rescue": 233, + "avoided_harm": 1784, + "selected_both_correct": 108, + "selected_both_wrong": 295 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9858333333333333, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2, + "avoided_harm": 524, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9825, + "rescued": 1, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 519, + "selected_both_correct": 1, + "selected_both_wrong": 5 + }, + { + "budget": 0.02, + "threshold": -0.3627302646636963, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.9775, + "rescued": 1, + "harmed": 11, + "missed_rescue": 1, + "avoided_harm": 513, + "selected_both_correct": 3, + "selected_both_wrong": 9 + }, + { + "budget": 0.03, + "threshold": -0.4222676455974579, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9725, + "rescued": 2, + "harmed": 18, + "missed_rescue": 0, + "avoided_harm": 506, + "selected_both_correct": 6, + "selected_both_wrong": 10 + }, + { + "budget": 0.04, + "threshold": -0.46618229150772095, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9675, + "rescued": 2, + "harmed": 24, + "missed_rescue": 0, + "avoided_harm": 500, + "selected_both_correct": 10, + "selected_both_wrong": 12 + }, + { + "budget": 0.05, + "threshold": -0.5066347122192383, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.9608333333333333, + "rescued": 2, + "harmed": 32, + "missed_rescue": 0, + "avoided_harm": 492, + "selected_both_correct": 14, + "selected_both_wrong": 12 + }, + { + "budget": 0.06, + "threshold": -0.5393391251564026, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.955, + "rescued": 2, + "harmed": 39, + "missed_rescue": 0, + "avoided_harm": 485, + "selected_both_correct": 17, + "selected_both_wrong": 14 + }, + { + "budget": 0.07, + "threshold": -0.5679479837417603, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.9508333333333333, + "rescued": 2, + "harmed": 44, + "missed_rescue": 0, + "avoided_harm": 480, + "selected_both_correct": 23, + "selected_both_wrong": 15 + }, + { + "budget": 0.08, + "threshold": -0.5907968282699585, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.945, + "rescued": 2, + "harmed": 51, + "missed_rescue": 0, + "avoided_harm": 473, + "selected_both_correct": 28, + "selected_both_wrong": 15 + }, + { + "budget": 0.09, + "threshold": -0.6168767809867859, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9391666666666667, + "rescued": 2, + "harmed": 58, + "missed_rescue": 0, + "avoided_harm": 466, + "selected_both_correct": 33, + "selected_both_wrong": 15 + }, + { + "budget": 0.1, + "threshold": -0.6361178159713745, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9333333333333333, + "rescued": 2, + "harmed": 65, + "missed_rescue": 0, + "avoided_harm": 459, + "selected_both_correct": 38, + "selected_both_wrong": 15 + }, + { + "budget": 0.11, + "threshold": -0.6547113656997681, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9291666666666667, + "rescued": 2, + "harmed": 70, + "missed_rescue": 0, + "avoided_harm": 454, + "selected_both_correct": 45, + "selected_both_wrong": 15 + }, + { + "budget": 0.12, + "threshold": -0.6680066585540771, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9225, + "rescued": 2, + "harmed": 78, + "missed_rescue": 0, + "avoided_harm": 446, + "selected_both_correct": 49, + "selected_both_wrong": 15 + }, + { + "budget": 0.13, + "threshold": -0.6809748411178589, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9141666666666667, + "rescued": 2, + "harmed": 88, + "missed_rescue": 0, + "avoided_harm": 436, + "selected_both_correct": 51, + "selected_both_wrong": 15 + }, + { + "budget": 0.14, + "threshold": -0.6963143348693848, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.9083333333333333, + "rescued": 2, + "harmed": 95, + "missed_rescue": 0, + "avoided_harm": 429, + "selected_both_correct": 56, + "selected_both_wrong": 15 + }, + { + "budget": 0.15, + "threshold": -0.7065573930740356, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.9033333333333333, + "rescued": 2, + "harmed": 101, + "missed_rescue": 0, + "avoided_harm": 423, + "selected_both_correct": 62, + "selected_both_wrong": 15 + }, + { + "budget": 0.16, + "threshold": -0.7129085659980774, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.8966666666666666, + "rescued": 2, + "harmed": 109, + "missed_rescue": 0, + "avoided_harm": 415, + "selected_both_correct": 66, + "selected_both_wrong": 15 + }, + { + "budget": 0.17, + "threshold": -0.7240402102470398, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.8891666666666667, + "rescued": 2, + "harmed": 118, + "missed_rescue": 0, + "avoided_harm": 406, + "selected_both_correct": 69, + "selected_both_wrong": 15 + }, + { + "budget": 0.18, + "threshold": -0.7345248460769653, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.8833333333333333, + "rescued": 2, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 399, + "selected_both_correct": 74, + "selected_both_wrong": 15 + }, + { + "budget": 0.19, + "threshold": -0.7443157434463501, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.8775, + "rescued": 2, + "harmed": 132, + "missed_rescue": 0, + "avoided_harm": 392, + "selected_both_correct": 79, + "selected_both_wrong": 15 + }, + { + "budget": 0.2, + "threshold": -0.7533873319625854, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.8716666666666667, + "rescued": 2, + "harmed": 139, + "missed_rescue": 0, + "avoided_harm": 385, + "selected_both_correct": 84, + "selected_both_wrong": 15 + }, + { + "budget": 0.21, + "threshold": -0.7606735825538635, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.8666666666666667, + "rescued": 2, + "harmed": 145, + "missed_rescue": 0, + "avoided_harm": 379, + "selected_both_correct": 90, + "selected_both_wrong": 15 + }, + { + "budget": 0.22, + "threshold": -0.7657199501991272, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.8641666666666666, + "rescued": 2, + "harmed": 148, + "missed_rescue": 0, + "avoided_harm": 376, + "selected_both_correct": 99, + "selected_both_wrong": 15 + }, + { + "budget": 0.23, + "threshold": -0.7712029218673706, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.86, + "rescued": 2, + "harmed": 153, + "missed_rescue": 0, + "avoided_harm": 371, + "selected_both_correct": 106, + "selected_both_wrong": 15 + }, + { + "budget": 0.24, + "threshold": -0.7754930257797241, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.855, + "rescued": 2, + "harmed": 159, + "missed_rescue": 0, + "avoided_harm": 365, + "selected_both_correct": 112, + "selected_both_wrong": 15 + }, + { + "budget": 0.25, + "threshold": -0.7811619639396667, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.8475, + "rescued": 2, + "harmed": 168, + "missed_rescue": 0, + "avoided_harm": 356, + "selected_both_correct": 115, + "selected_both_wrong": 15 + }, + { + "budget": 0.26, + "threshold": -0.7841500043869019, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.8408333333333333, + "rescued": 2, + "harmed": 176, + "missed_rescue": 0, + "avoided_harm": 348, + "selected_both_correct": 119, + "selected_both_wrong": 15 + }, + { + "budget": 0.27, + "threshold": -0.78773033618927, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.8358333333333333, + "rescued": 2, + "harmed": 182, + "missed_rescue": 0, + "avoided_harm": 342, + "selected_both_correct": 125, + "selected_both_wrong": 15 + }, + { + "budget": 0.28, + "threshold": -0.7923226356506348, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.8316666666666667, + "rescued": 2, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 337, + "selected_both_correct": 132, + "selected_both_wrong": 15 + }, + { + "budget": 0.29, + "threshold": -0.7950519323348999, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.8291666666666667, + "rescued": 2, + "harmed": 190, + "missed_rescue": 0, + "avoided_harm": 334, + "selected_both_correct": 141, + "selected_both_wrong": 15 + }, + { + "budget": 0.3, + "threshold": -0.800890326499939, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.825, + "rescued": 2, + "harmed": 195, + "missed_rescue": 0, + "avoided_harm": 329, + "selected_both_correct": 148, + "selected_both_wrong": 15 + }, + { + "budget": 0.31, + "threshold": -0.8055772185325623, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.8208333333333333, + "rescued": 2, + "harmed": 200, + "missed_rescue": 0, + "avoided_harm": 324, + "selected_both_correct": 155, + "selected_both_wrong": 15 + }, + { + "budget": 0.32, + "threshold": -0.809782087802887, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.8183333333333334, + "rescued": 2, + "harmed": 203, + "missed_rescue": 0, + "avoided_harm": 321, + "selected_both_correct": 164, + "selected_both_wrong": 15 + }, + { + "budget": 0.33, + "threshold": -0.8134240508079529, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.8125, + "rescued": 2, + "harmed": 210, + "missed_rescue": 0, + "avoided_harm": 314, + "selected_both_correct": 169, + "selected_both_wrong": 15 + }, + { + "budget": 0.34, + "threshold": -0.8167235255241394, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.81, + "rescued": 2, + "harmed": 213, + "missed_rescue": 0, + "avoided_harm": 311, + "selected_both_correct": 178, + "selected_both_wrong": 15 + }, + { + "budget": 0.35, + "threshold": -0.8208380937576294, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.8041666666666667, + "rescued": 2, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 304, + "selected_both_correct": 183, + "selected_both_wrong": 15 + }, + { + "budget": 0.36, + "threshold": -0.8250406384468079, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.8016666666666666, + "rescued": 2, + "harmed": 223, + "missed_rescue": 0, + "avoided_harm": 301, + "selected_both_correct": 192, + "selected_both_wrong": 15 + }, + { + "budget": 0.37, + "threshold": -0.8295461535453796, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.7966666666666666, + "rescued": 2, + "harmed": 229, + "missed_rescue": 0, + "avoided_harm": 295, + "selected_both_correct": 198, + "selected_both_wrong": 15 + }, + { + "budget": 0.38, + "threshold": -0.8328037261962891, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.7916666666666666, + "rescued": 2, + "harmed": 235, + "missed_rescue": 0, + "avoided_harm": 289, + "selected_both_correct": 204, + "selected_both_wrong": 15 + }, + { + "budget": 0.39, + "threshold": -0.8359338641166687, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.7875, + "rescued": 2, + "harmed": 240, + "missed_rescue": 0, + "avoided_harm": 284, + "selected_both_correct": 211, + "selected_both_wrong": 15 + }, + { + "budget": 0.4, + "threshold": -0.8391940593719482, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.78, + "rescued": 2, + "harmed": 249, + "missed_rescue": 0, + "avoided_harm": 275, + "selected_both_correct": 214, + "selected_both_wrong": 15 + }, + { + "budget": 0.41, + "threshold": -0.8424465656280518, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.7775, + "rescued": 2, + "harmed": 252, + "missed_rescue": 0, + "avoided_harm": 272, + "selected_both_correct": 222, + "selected_both_wrong": 15 + }, + { + "budget": 0.42, + "threshold": -0.8460794687271118, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.7741666666666667, + "rescued": 2, + "harmed": 256, + "missed_rescue": 0, + "avoided_harm": 268, + "selected_both_correct": 231, + "selected_both_wrong": 15 + }, + { + "budget": 0.43, + "threshold": -0.8500295281410217, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.7716666666666666, + "rescued": 2, + "harmed": 259, + "missed_rescue": 0, + "avoided_harm": 265, + "selected_both_correct": 240, + "selected_both_wrong": 15 + }, + { + "budget": 0.44, + "threshold": -0.852453887462616, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.7666666666666667, + "rescued": 2, + "harmed": 265, + "missed_rescue": 0, + "avoided_harm": 259, + "selected_both_correct": 246, + "selected_both_wrong": 15 + }, + { + "budget": 0.45, + "threshold": -0.8539360165596008, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.7625, + "rescued": 2, + "harmed": 270, + "missed_rescue": 0, + "avoided_harm": 254, + "selected_both_correct": 253, + "selected_both_wrong": 15 + }, + { + "budget": 0.46, + "threshold": -0.8562509417533875, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.76, + "rescued": 2, + "harmed": 273, + "missed_rescue": 0, + "avoided_harm": 251, + "selected_both_correct": 262, + "selected_both_wrong": 15 + }, + { + "budget": 0.47, + "threshold": -0.8592684864997864, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.7558333333333334, + "rescued": 2, + "harmed": 278, + "missed_rescue": 0, + "avoided_harm": 246, + "selected_both_correct": 269, + "selected_both_wrong": 15 + }, + { + "budget": 0.48, + "threshold": -0.8621411919593811, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.7525, + "rescued": 2, + "harmed": 282, + "missed_rescue": 0, + "avoided_harm": 242, + "selected_both_correct": 277, + "selected_both_wrong": 15 + }, + { + "budget": 0.49, + "threshold": -0.863978385925293, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.7483333333333333, + "rescued": 2, + "harmed": 287, + "missed_rescue": 0, + "avoided_harm": 237, + "selected_both_correct": 284, + "selected_both_wrong": 15 + }, + { + "budget": 0.5, + "threshold": -0.8661247491836548, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.7425, + "rescued": 2, + "harmed": 294, + "missed_rescue": 0, + "avoided_harm": 230, + "selected_both_correct": 289, + "selected_both_wrong": 15 + }, + { + "budget": 0.51, + "threshold": -0.8688956499099731, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.7383333333333333, + "rescued": 2, + "harmed": 299, + "missed_rescue": 0, + "avoided_harm": 225, + "selected_both_correct": 296, + "selected_both_wrong": 15 + }, + { + "budget": 0.52, + "threshold": -0.8706188201904297, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.7358333333333333, + "rescued": 2, + "harmed": 302, + "missed_rescue": 0, + "avoided_harm": 222, + "selected_both_correct": 305, + "selected_both_wrong": 15 + }, + { + "budget": 0.53, + "threshold": -0.8727357983589172, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.73, + "rescued": 2, + "harmed": 309, + "missed_rescue": 0, + "avoided_harm": 215, + "selected_both_correct": 310, + "selected_both_wrong": 15 + }, + { + "budget": 0.54, + "threshold": -0.874894380569458, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.725, + "rescued": 2, + "harmed": 315, + "missed_rescue": 0, + "avoided_harm": 209, + "selected_both_correct": 316, + "selected_both_wrong": 15 + }, + { + "budget": 0.55, + "threshold": -0.8764559626579285, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.72, + "rescued": 2, + "harmed": 321, + "missed_rescue": 0, + "avoided_harm": 203, + "selected_both_correct": 322, + "selected_both_wrong": 15 + }, + { + "budget": 0.56, + "threshold": -0.879269540309906, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.7166666666666667, + "rescued": 2, + "harmed": 325, + "missed_rescue": 0, + "avoided_harm": 199, + "selected_both_correct": 330, + "selected_both_wrong": 15 + }, + { + "budget": 0.57, + "threshold": -0.881222128868103, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.7116666666666667, + "rescued": 2, + "harmed": 331, + "missed_rescue": 0, + "avoided_harm": 193, + "selected_both_correct": 335, + "selected_both_wrong": 15 + }, + { + "budget": 0.58, + "threshold": -0.882451593875885, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.7075, + "rescued": 2, + "harmed": 336, + "missed_rescue": 0, + "avoided_harm": 188, + "selected_both_correct": 343, + "selected_both_wrong": 15 + }, + { + "budget": 0.59, + "threshold": -0.8843295574188232, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.7025, + "rescued": 2, + "harmed": 342, + "missed_rescue": 0, + "avoided_harm": 182, + "selected_both_correct": 349, + "selected_both_wrong": 15 + }, + { + "budget": 0.6, + "threshold": -0.8863861560821533, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.6983333333333334, + "rescued": 2, + "harmed": 347, + "missed_rescue": 0, + "avoided_harm": 177, + "selected_both_correct": 356, + "selected_both_wrong": 15 + }, + { + "budget": 0.61, + "threshold": -0.8876380920410156, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.6933333333333334, + "rescued": 2, + "harmed": 353, + "missed_rescue": 0, + "avoided_harm": 171, + "selected_both_correct": 362, + "selected_both_wrong": 15 + }, + { + "budget": 0.62, + "threshold": -0.8888679146766663, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.69, + "rescued": 2, + "harmed": 357, + "missed_rescue": 0, + "avoided_harm": 167, + "selected_both_correct": 370, + "selected_both_wrong": 15 + }, + { + "budget": 0.63, + "threshold": -0.8905391693115234, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.6883333333333334, + "rescued": 2, + "harmed": 359, + "missed_rescue": 0, + "avoided_harm": 165, + "selected_both_correct": 380, + "selected_both_wrong": 15 + }, + { + "budget": 0.64, + "threshold": -0.8919612169265747, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.6875, + "rescued": 2, + "harmed": 360, + "missed_rescue": 0, + "avoided_harm": 164, + "selected_both_correct": 391, + "selected_both_wrong": 15 + }, + { + "budget": 0.65, + "threshold": -0.8940081596374512, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.6841666666666667, + "rescued": 2, + "harmed": 364, + "missed_rescue": 0, + "avoided_harm": 160, + "selected_both_correct": 399, + "selected_both_wrong": 15 + }, + { + "budget": 0.66, + "threshold": -0.895609974861145, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.68, + "rescued": 2, + "harmed": 369, + "missed_rescue": 0, + "avoided_harm": 155, + "selected_both_correct": 406, + "selected_both_wrong": 15 + }, + { + "budget": 0.67, + "threshold": -0.8972456455230713, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.6733333333333333, + "rescued": 2, + "harmed": 377, + "missed_rescue": 0, + "avoided_harm": 147, + "selected_both_correct": 410, + "selected_both_wrong": 15 + }, + { + "budget": 0.68, + "threshold": -0.8986453413963318, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.6716666666666666, + "rescued": 2, + "harmed": 379, + "missed_rescue": 0, + "avoided_harm": 145, + "selected_both_correct": 420, + "selected_both_wrong": 15 + }, + { + "budget": 0.69, + "threshold": -0.8996874094009399, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.67, + "rescued": 2, + "harmed": 381, + "missed_rescue": 0, + "avoided_harm": 143, + "selected_both_correct": 429, + "selected_both_wrong": 15 + }, + { + "budget": 0.7, + "threshold": -0.9008907079696655, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.665, + "rescued": 2, + "harmed": 387, + "missed_rescue": 0, + "avoided_harm": 137, + "selected_both_correct": 436, + "selected_both_wrong": 15 + }, + { + "budget": 0.71, + "threshold": -0.9031620621681213, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.6583333333333333, + "rescued": 2, + "harmed": 395, + "missed_rescue": 0, + "avoided_harm": 129, + "selected_both_correct": 440, + "selected_both_wrong": 15 + }, + { + "budget": 0.72, + "threshold": -0.9056642055511475, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.6541666666666667, + "rescued": 2, + "harmed": 400, + "missed_rescue": 0, + "avoided_harm": 124, + "selected_both_correct": 447, + "selected_both_wrong": 15 + }, + { + "budget": 0.73, + "threshold": -0.9068957567214966, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.6508333333333334, + "rescued": 2, + "harmed": 404, + "missed_rescue": 0, + "avoided_harm": 120, + "selected_both_correct": 455, + "selected_both_wrong": 15 + }, + { + "budget": 0.74, + "threshold": -0.9093236327171326, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.6491666666666667, + "rescued": 2, + "harmed": 406, + "missed_rescue": 0, + "avoided_harm": 118, + "selected_both_correct": 465, + "selected_both_wrong": 15 + }, + { + "budget": 0.75, + "threshold": -0.9112368822097778, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.645, + "rescued": 2, + "harmed": 411, + "missed_rescue": 0, + "avoided_harm": 113, + "selected_both_correct": 472, + "selected_both_wrong": 15 + }, + { + "budget": 0.76, + "threshold": -0.9129221439361572, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.6425, + "rescued": 2, + "harmed": 414, + "missed_rescue": 0, + "avoided_harm": 110, + "selected_both_correct": 481, + "selected_both_wrong": 15 + }, + { + "budget": 0.77, + "threshold": -0.9144289493560791, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.64, + "rescued": 2, + "harmed": 417, + "missed_rescue": 0, + "avoided_harm": 107, + "selected_both_correct": 490, + "selected_both_wrong": 15 + }, + { + "budget": 0.78, + "threshold": -0.9158422946929932, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.6383333333333333, + "rescued": 2, + "harmed": 419, + "missed_rescue": 0, + "avoided_harm": 105, + "selected_both_correct": 500, + "selected_both_wrong": 15 + }, + { + "budget": 0.79, + "threshold": -0.9167781472206116, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.6341666666666667, + "rescued": 2, + "harmed": 424, + "missed_rescue": 0, + "avoided_harm": 100, + "selected_both_correct": 507, + "selected_both_wrong": 15 + }, + { + "budget": 0.8, + "threshold": -0.9187189936637878, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.6283333333333333, + "rescued": 2, + "harmed": 431, + "missed_rescue": 0, + "avoided_harm": 93, + "selected_both_correct": 512, + "selected_both_wrong": 15 + }, + { + "budget": 0.81, + "threshold": -0.9207327365875244, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.625, + "rescued": 2, + "harmed": 435, + "missed_rescue": 0, + "avoided_harm": 89, + "selected_both_correct": 520, + "selected_both_wrong": 15 + }, + { + "budget": 0.82, + "threshold": -0.9226345419883728, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.6216666666666667, + "rescued": 2, + "harmed": 439, + "missed_rescue": 0, + "avoided_harm": 85, + "selected_both_correct": 527, + "selected_both_wrong": 15 + }, + { + "budget": 0.83, + "threshold": -0.9243993759155273, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.6191666666666666, + "rescued": 2, + "harmed": 442, + "missed_rescue": 0, + "avoided_harm": 82, + "selected_both_correct": 537, + "selected_both_wrong": 15 + }, + { + "budget": 0.84, + "threshold": -0.9264947175979614, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.6166666666666667, + "rescued": 2, + "harmed": 445, + "missed_rescue": 0, + "avoided_harm": 79, + "selected_both_correct": 546, + "selected_both_wrong": 15 + }, + { + "budget": 0.85, + "threshold": -0.9274600148200989, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.6141666666666666, + "rescued": 2, + "harmed": 448, + "missed_rescue": 0, + "avoided_harm": 76, + "selected_both_correct": 555, + "selected_both_wrong": 15 + }, + { + "budget": 0.86, + "threshold": -0.930335283279419, + "operator": "ge", + "actual_fraction": 0.86, + "metric": 0.6108333333333333, + "rescued": 2, + "harmed": 452, + "missed_rescue": 0, + "avoided_harm": 72, + "selected_both_correct": 563, + "selected_both_wrong": 15 + }, + { + "budget": 0.87, + "threshold": -0.9319414496421814, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.6066666666666667, + "rescued": 2, + "harmed": 457, + "missed_rescue": 0, + "avoided_harm": 67, + "selected_both_correct": 570, + "selected_both_wrong": 15 + }, + { + "budget": 0.88, + "threshold": -0.9340114593505859, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.6033333333333334, + "rescued": 2, + "harmed": 461, + "missed_rescue": 0, + "avoided_harm": 63, + "selected_both_correct": 578, + "selected_both_wrong": 15 + }, + { + "budget": 0.89, + "threshold": -0.9363258481025696, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.5983333333333334, + "rescued": 2, + "harmed": 467, + "missed_rescue": 0, + "avoided_harm": 57, + "selected_both_correct": 584, + "selected_both_wrong": 15 + }, + { + "budget": 0.9, + "threshold": -0.9400071501731873, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.595, + "rescued": 2, + "harmed": 471, + "missed_rescue": 0, + "avoided_harm": 53, + "selected_both_correct": 592, + "selected_both_wrong": 15 + }, + { + "budget": 0.91, + "threshold": -0.9423198103904724, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.5908333333333333, + "rescued": 2, + "harmed": 476, + "missed_rescue": 0, + "avoided_harm": 48, + "selected_both_correct": 599, + "selected_both_wrong": 15 + }, + { + "budget": 0.92, + "threshold": -0.9443356394767761, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.5866666666666667, + "rescued": 2, + "harmed": 481, + "missed_rescue": 0, + "avoided_harm": 43, + "selected_both_correct": 606, + "selected_both_wrong": 15 + }, + { + "budget": 0.93, + "threshold": -0.9468213319778442, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.5833333333333334, + "rescued": 2, + "harmed": 485, + "missed_rescue": 0, + "avoided_harm": 39, + "selected_both_correct": 614, + "selected_both_wrong": 15 + }, + { + "budget": 0.94, + "threshold": -0.9496435523033142, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.5791666666666667, + "rescued": 2, + "harmed": 490, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 621, + "selected_both_wrong": 15 + }, + { + "budget": 0.95, + "threshold": -0.9529175758361816, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.5766666666666667, + "rescued": 2, + "harmed": 493, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 630, + "selected_both_wrong": 15 + }, + { + "budget": 0.96, + "threshold": -0.9560379981994629, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.57, + "rescued": 2, + "harmed": 501, + "missed_rescue": 0, + "avoided_harm": 23, + "selected_both_correct": 634, + "selected_both_wrong": 15 + }, + { + "budget": 0.97, + "threshold": -0.9591385126113892, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.5666666666666667, + "rescued": 2, + "harmed": 505, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 642, + "selected_both_wrong": 15 + }, + { + "budget": 0.98, + "threshold": -0.9627243280410767, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.56, + "rescued": 2, + "harmed": 513, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 646, + "selected_both_wrong": 15 + }, + { + "budget": 0.99, + "threshold": -0.9702208638191223, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.5558333333333333, + "rescued": 2, + "harmed": 518, + "missed_rescue": 0, + "avoided_harm": 6, + "selected_both_correct": 653, + "selected_both_wrong": 15 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5508333333333333, + "rescued": 2, + "harmed": 524, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 659, + "selected_both_wrong": 15 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.81429064549534, + "rescued": 0, + "harmed": 0, + "missed_rescue": 365, + "avoided_harm": 1907, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.305810809135437, + "operator": "ge", + "actual_fraction": 0.113565757680359, + "metric": 0.8158439765274422, + "rescued": 132, + "harmed": 123, + "missed_rescue": 233, + "avoided_harm": 1784, + "selected_both_correct": 108, + "selected_both_wrong": 295 + }, + { + "budget": 0.02, + "threshold": -0.3627302646636963, + "operator": "ge", + "actual_fraction": 0.15947531929582326, + "metric": 0.8099758370728339, + "rescued": 182, + "harmed": 207, + "missed_rescue": 183, + "avoided_harm": 1700, + "selected_both_correct": 170, + "selected_both_wrong": 365 + }, + { + "budget": 0.03, + "threshold": -0.4222676455974579, + "operator": "ge", + "actual_fraction": 0.2143596824301001, + "metric": 0.798584742837418, + "rescued": 226, + "harmed": 317, + "missed_rescue": 139, + "avoided_harm": 1590, + "selected_both_correct": 261, + "selected_both_wrong": 438 + }, + { + "budget": 0.04, + "threshold": -0.46618229150772095, + "operator": "ge", + "actual_fraction": 0.2545736969278564, + "metric": 0.789264756644805, + "rescued": 257, + "harmed": 402, + "missed_rescue": 108, + "avoided_harm": 1505, + "selected_both_correct": 329, + "selected_both_wrong": 487 + }, + { + "budget": 0.05, + "threshold": -0.5066347122192383, + "operator": "ge", + "actual_fraction": 0.29375215740421123, + "metric": 0.7773558853986883, + "rescued": 277, + "harmed": 491, + "missed_rescue": 88, + "avoided_harm": 1416, + "selected_both_correct": 417, + "selected_both_wrong": 517 + }, + { + "budget": 0.06, + "threshold": -0.5393391251564026, + "operator": "ge", + "actual_fraction": 0.3208491542975492, + "metric": 0.7709699689333793, + "rescued": 291, + "harmed": 542, + "missed_rescue": 74, + "avoided_harm": 1365, + "selected_both_correct": 480, + "selected_both_wrong": 546 + }, + { + "budget": 0.07, + "threshold": -0.5679479837417603, + "operator": "ge", + "actual_fraction": 0.350707628581291, + "metric": 0.7604418363824647, + "rescued": 303, + "harmed": 615, + "missed_rescue": 62, + "avoided_harm": 1292, + "selected_both_correct": 537, + "selected_both_wrong": 577 + }, + { + "budget": 0.08, + "threshold": -0.5907968282699585, + "operator": "ge", + "actual_fraction": 0.36882982395581637, + "metric": 0.7540559199171557, + "rescued": 312, + "harmed": 661, + "missed_rescue": 53, + "avoided_harm": 1246, + "selected_both_correct": 576, + "selected_both_wrong": 588 + }, + { + "budget": 0.09, + "threshold": -0.6168767809867859, + "operator": "ge", + "actual_fraction": 0.391439420089748, + "metric": 0.7452537107352434, + "rescued": 317, + "harmed": 717, + "missed_rescue": 48, + "avoided_harm": 1190, + "selected_both_correct": 627, + "selected_both_wrong": 607 + }, + { + "budget": 0.1, + "threshold": -0.6361178159713745, + "operator": "ge", + "actual_fraction": 0.4138764238867794, + "metric": 0.7392129789437349, + "rescued": 327, + "harmed": 762, + "missed_rescue": 38, + "avoided_harm": 1145, + "selected_both_correct": 688, + "selected_both_wrong": 621 + }, + { + "budget": 0.11, + "threshold": -0.6547113656997681, + "operator": "ge", + "actual_fraction": 0.43372454263030724, + "metric": 0.732827062478426, + "rescued": 336, + "harmed": 808, + "missed_rescue": 29, + "avoided_harm": 1099, + "selected_both_correct": 739, + "selected_both_wrong": 630 + }, + { + "budget": 0.12, + "threshold": -0.6680066585540771, + "operator": "ge", + "actual_fraction": 0.4480497065930273, + "metric": 0.7267863306869174, + "rescued": 338, + "harmed": 845, + "missed_rescue": 27, + "avoided_harm": 1062, + "selected_both_correct": 780, + "selected_both_wrong": 633 + }, + { + "budget": 0.13, + "threshold": -0.6809748411178589, + "operator": "ge", + "actual_fraction": 0.4630652399033483, + "metric": 0.7216085605799103, + "rescued": 343, + "harmed": 880, + "missed_rescue": 22, + "avoided_harm": 1027, + "selected_both_correct": 821, + "selected_both_wrong": 639 + }, + { + "budget": 0.14, + "threshold": -0.6963143348693848, + "operator": "ge", + "actual_fraction": 0.479288919571971, + "metric": 0.7152226441146013, + "rescued": 346, + "harmed": 920, + "missed_rescue": 19, + "avoided_harm": 987, + "selected_both_correct": 868, + "selected_both_wrong": 643 + }, + { + "budget": 0.15, + "threshold": -0.7065573930740356, + "operator": "ge", + "actual_fraction": 0.4932688988608906, + "metric": 0.710735243355195, + "rescued": 348, + "harmed": 948, + "missed_rescue": 17, + "avoided_harm": 959, + "selected_both_correct": 911, + "selected_both_wrong": 651 + }, + { + "budget": 0.16, + "threshold": -0.7129085659980774, + "operator": "ge", + "actual_fraction": 0.5008629616845012, + "metric": 0.7086641353123921, + "rescued": 350, + "harmed": 962, + "missed_rescue": 15, + "avoided_harm": 945, + "selected_both_correct": 937, + "selected_both_wrong": 653 + }, + { + "budget": 0.17, + "threshold": -0.7240402102470398, + "operator": "ge", + "actual_fraction": 0.5155333103210218, + "metric": 0.701415257162582, + "rescued": 350, + "harmed": 1004, + "missed_rescue": 15, + "avoided_harm": 903, + "selected_both_correct": 978, + "selected_both_wrong": 655 + }, + { + "budget": 0.18, + "threshold": -0.7345248460769653, + "operator": "ge", + "actual_fraction": 0.5298584742837418, + "metric": 0.6967552640662754, + "rescued": 355, + "harmed": 1036, + "missed_rescue": 10, + "avoided_harm": 871, + "selected_both_correct": 1020, + "selected_both_wrong": 659 + }, + { + "budget": 0.19, + "threshold": -0.7443157434463501, + "operator": "ge", + "actual_fraction": 0.5455643769416638, + "metric": 0.6908871246116672, + "rescued": 355, + "harmed": 1070, + "missed_rescue": 10, + "avoided_harm": 837, + "selected_both_correct": 1072, + "selected_both_wrong": 664 + }, + { + "budget": 0.2, + "threshold": -0.7533873319625854, + "operator": "ge", + "actual_fraction": 0.5560925094925785, + "metric": 0.6872626855367622, + "rescued": 356, + "harmed": 1092, + "missed_rescue": 9, + "avoided_harm": 815, + "selected_both_correct": 1106, + "selected_both_wrong": 668 + }, + { + "budget": 0.21, + "threshold": -0.7606735825538635, + "operator": "ge", + "actual_fraction": 0.5676561960648947, + "metric": 0.6841560234725578, + "rescued": 356, + "harmed": 1110, + "missed_rescue": 9, + "avoided_harm": 797, + "selected_both_correct": 1153, + "selected_both_wrong": 670 + }, + { + "budget": 0.22, + "threshold": -0.7657199501991272, + "operator": "ge", + "actual_fraction": 0.5766309975837073, + "metric": 0.6808767690714532, + "rescued": 356, + "harmed": 1129, + "missed_rescue": 9, + "avoided_harm": 778, + "selected_both_correct": 1184, + "selected_both_wrong": 672 + }, + { + "budget": 0.23, + "threshold": -0.7712029218673706, + "operator": "ge", + "actual_fraction": 0.5861235761132205, + "metric": 0.6779426993441491, + "rescued": 357, + "harmed": 1147, + "missed_rescue": 8, + "avoided_harm": 760, + "selected_both_correct": 1219, + "selected_both_wrong": 673 + }, + { + "budget": 0.24, + "threshold": -0.7754930257797241, + "operator": "ge", + "actual_fraction": 0.5923369002416292, + "metric": 0.6755264066275457, + "rescued": 357, + "harmed": 1161, + "missed_rescue": 8, + "avoided_harm": 746, + "selected_both_correct": 1238, + "selected_both_wrong": 676 + }, + { + "budget": 0.25, + "threshold": -0.7811619639396667, + "operator": "ge", + "actual_fraction": 0.6013117017604418, + "metric": 0.6727649292371418, + "rescued": 358, + "harmed": 1178, + "missed_rescue": 7, + "avoided_harm": 729, + "selected_both_correct": 1272, + "selected_both_wrong": 676 + }, + { + "budget": 0.26, + "threshold": -0.7841500043869019, + "operator": "ge", + "actual_fraction": 0.6052813255091474, + "metric": 0.6712115982050397, + "rescued": 359, + "harmed": 1188, + "missed_rescue": 6, + "avoided_harm": 719, + "selected_both_correct": 1284, + "selected_both_wrong": 676 + }, + { + "budget": 0.27, + "threshold": -0.78773033618927, + "operator": "ge", + "actual_fraction": 0.6123576113220573, + "metric": 0.6681049361408353, + "rescued": 359, + "harmed": 1206, + "missed_rescue": 6, + "avoided_harm": 701, + "selected_both_correct": 1305, + "selected_both_wrong": 678 + }, + { + "budget": 0.28, + "threshold": -0.7923226356506348, + "operator": "ge", + "actual_fraction": 0.6206420434932689, + "metric": 0.664998274076631, + "rescued": 359, + "harmed": 1224, + "missed_rescue": 6, + "avoided_harm": 683, + "selected_both_correct": 1333, + "selected_both_wrong": 680 + }, + { + "budget": 0.29, + "threshold": -0.7950519323348999, + "operator": "ge", + "actual_fraction": 0.6251294442526751, + "metric": 0.6632723507076286, + "rescued": 359, + "harmed": 1234, + "missed_rescue": 6, + "avoided_harm": 673, + "selected_both_correct": 1348, + "selected_both_wrong": 681 + }, + { + "budget": 0.3, + "threshold": -0.800890326499939, + "operator": "ge", + "actual_fraction": 0.6373835001725924, + "metric": 0.6580945806006213, + "rescued": 359, + "harmed": 1264, + "missed_rescue": 6, + "avoided_harm": 643, + "selected_both_correct": 1388, + "selected_both_wrong": 682 + }, + { + "budget": 0.31, + "threshold": -0.8055772185325623, + "operator": "ge", + "actual_fraction": 0.6470486710390059, + "metric": 0.6546427338626165, + "rescued": 359, + "harmed": 1284, + "missed_rescue": 6, + "avoided_harm": 623, + "selected_both_correct": 1423, + "selected_both_wrong": 683 + }, + { + "budget": 0.32, + "threshold": -0.809782087802887, + "operator": "ge", + "actual_fraction": 0.6556782878840179, + "metric": 0.6527442181567138, + "rescued": 360, + "harmed": 1296, + "missed_rescue": 5, + "avoided_harm": 611, + "selected_both_correct": 1457, + "selected_both_wrong": 686 + }, + { + "budget": 0.33, + "threshold": -0.8134240508079529, + "operator": "ge", + "actual_fraction": 0.6636175353814291, + "metric": 0.6498101484294098, + "rescued": 360, + "harmed": 1313, + "missed_rescue": 5, + "avoided_harm": 594, + "selected_both_correct": 1485, + "selected_both_wrong": 687 + }, + { + "budget": 0.34, + "threshold": -0.8167235255241394, + "operator": "ge", + "actual_fraction": 0.6708664135312392, + "metric": 0.6472212633759061, + "rescued": 360, + "harmed": 1328, + "missed_rescue": 5, + "avoided_harm": 579, + "selected_both_correct": 1511, + "selected_both_wrong": 688 + }, + { + "budget": 0.35, + "threshold": -0.8208380937576294, + "operator": "ge", + "actual_fraction": 0.6812219537452537, + "metric": 0.6435968243010011, + "rescued": 360, + "harmed": 1349, + "missed_rescue": 5, + "avoided_harm": 558, + "selected_both_correct": 1549, + "selected_both_wrong": 689 + }, + { + "budget": 0.36, + "threshold": -0.8250406384468079, + "operator": "ge", + "actual_fraction": 0.689333793579565, + "metric": 0.6408353469105972, + "rescued": 360, + "harmed": 1365, + "missed_rescue": 5, + "avoided_harm": 542, + "selected_both_correct": 1578, + "selected_both_wrong": 691 + }, + { + "budget": 0.37, + "threshold": -0.8295461535453796, + "operator": "ge", + "actual_fraction": 0.7017604418363824, + "metric": 0.6347946151190887, + "rescued": 360, + "harmed": 1400, + "missed_rescue": 5, + "avoided_harm": 507, + "selected_both_correct": 1613, + "selected_both_wrong": 693 + }, + { + "budget": 0.38, + "threshold": -0.8328037261962891, + "operator": "ge", + "actual_fraction": 0.7088367276492924, + "metric": 0.6320331377286849, + "rescued": 360, + "harmed": 1416, + "missed_rescue": 5, + "avoided_harm": 491, + "selected_both_correct": 1637, + "selected_both_wrong": 694 + }, + { + "budget": 0.39, + "threshold": -0.8359338641166687, + "operator": "ge", + "actual_fraction": 0.7166033828098033, + "metric": 0.6290990680013807, + "rescued": 360, + "harmed": 1433, + "missed_rescue": 5, + "avoided_harm": 474, + "selected_both_correct": 1665, + "selected_both_wrong": 694 + }, + { + "budget": 0.4, + "threshold": -0.8391940593719482, + "operator": "ge", + "actual_fraction": 0.7238522609596134, + "metric": 0.6265101829478771, + "rescued": 361, + "harmed": 1449, + "missed_rescue": 4, + "avoided_harm": 458, + "selected_both_correct": 1689, + "selected_both_wrong": 695 + }, + { + "budget": 0.41, + "threshold": -0.8424465656280518, + "operator": "ge", + "actual_fraction": 0.7302381774249224, + "metric": 0.6244390749050742, + "rescued": 361, + "harmed": 1461, + "missed_rescue": 4, + "avoided_harm": 446, + "selected_both_correct": 1713, + "selected_both_wrong": 696 + }, + { + "budget": 0.42, + "threshold": -0.8460794687271118, + "operator": "ge", + "actual_fraction": 0.7392129789437349, + "metric": 0.6211598205039697, + "rescued": 361, + "harmed": 1480, + "missed_rescue": 4, + "avoided_harm": 427, + "selected_both_correct": 1746, + "selected_both_wrong": 696 + }, + { + "budget": 0.43, + "threshold": -0.8500295281410217, + "operator": "ge", + "actual_fraction": 0.747152226441146, + "metric": 0.618570935450466, + "rescued": 361, + "harmed": 1495, + "missed_rescue": 4, + "avoided_harm": 412, + "selected_both_correct": 1777, + "selected_both_wrong": 696 + }, + { + "budget": 0.44, + "threshold": -0.852453887462616, + "operator": "ge", + "actual_fraction": 0.7523299965481532, + "metric": 0.6175353814290645, + "rescued": 361, + "harmed": 1501, + "missed_rescue": 4, + "avoided_harm": 406, + "selected_both_correct": 1800, + "selected_both_wrong": 697 + }, + { + "budget": 0.45, + "threshold": -0.8539360165596008, + "operator": "ge", + "actual_fraction": 0.7562996202968588, + "metric": 0.6171901967552641, + "rescued": 361, + "harmed": 1503, + "missed_rescue": 4, + "avoided_harm": 404, + "selected_both_correct": 1821, + "selected_both_wrong": 697 + }, + { + "budget": 0.46, + "threshold": -0.8562509417533875, + "operator": "ge", + "actual_fraction": 0.7613047980669658, + "metric": 0.6159820503969624, + "rescued": 361, + "harmed": 1510, + "missed_rescue": 4, + "avoided_harm": 397, + "selected_both_correct": 1843, + "selected_both_wrong": 697 + }, + { + "budget": 0.47, + "threshold": -0.8592684864997864, + "operator": "ge", + "actual_fraction": 0.7668277528477736, + "metric": 0.6144287193648602, + "rescued": 361, + "harmed": 1519, + "missed_rescue": 4, + "avoided_harm": 388, + "selected_both_correct": 1865, + "selected_both_wrong": 698 + }, + { + "budget": 0.48, + "threshold": -0.8621411919593811, + "operator": "ge", + "actual_fraction": 0.774249223334484, + "metric": 0.6127027959958578, + "rescued": 362, + "harmed": 1530, + "missed_rescue": 3, + "avoided_harm": 377, + "selected_both_correct": 1895, + "selected_both_wrong": 699 + }, + { + "budget": 0.49, + "threshold": -0.863978385925293, + "operator": "ge", + "actual_fraction": 0.7790818087676907, + "metric": 0.6104590956161546, + "rescued": 362, + "harmed": 1543, + "missed_rescue": 3, + "avoided_harm": 364, + "selected_both_correct": 1909, + "selected_both_wrong": 700 + }, + { + "budget": 0.5, + "threshold": -0.8661247491836548, + "operator": "ge", + "actual_fraction": 0.7849499482222989, + "metric": 0.6080428028995513, + "rescued": 362, + "harmed": 1557, + "missed_rescue": 3, + "avoided_harm": 350, + "selected_both_correct": 1928, + "selected_both_wrong": 701 + }, + { + "budget": 0.51, + "threshold": -0.8688956499099731, + "operator": "ge", + "actual_fraction": 0.7928891957197101, + "metric": 0.6056265101829479, + "rescued": 363, + "harmed": 1572, + "missed_rescue": 2, + "avoided_harm": 335, + "selected_both_correct": 1958, + "selected_both_wrong": 701 + }, + { + "budget": 0.52, + "threshold": -0.8706188201904297, + "operator": "ge", + "actual_fraction": 0.7970314118053159, + "metric": 0.6044183638246462, + "rescued": 363, + "harmed": 1579, + "missed_rescue": 2, + "avoided_harm": 328, + "selected_both_correct": 1975, + "selected_both_wrong": 701 + }, + { + "budget": 0.53, + "threshold": -0.8727357983589172, + "operator": "ge", + "actual_fraction": 0.8016914049016224, + "metric": 0.6037279944770452, + "rescued": 363, + "harmed": 1583, + "missed_rescue": 2, + "avoided_harm": 324, + "selected_both_correct": 1998, + "selected_both_wrong": 701 + }, + { + "budget": 0.54, + "threshold": -0.874894380569458, + "operator": "ge", + "actual_fraction": 0.8070417673455299, + "metric": 0.6018294787711426, + "rescued": 363, + "harmed": 1594, + "missed_rescue": 2, + "avoided_harm": 313, + "selected_both_correct": 2018, + "selected_both_wrong": 701 + }, + { + "budget": 0.55, + "threshold": -0.8764559626579285, + "operator": "ge", + "actual_fraction": 0.8120469451156369, + "metric": 0.6007939247497411, + "rescued": 363, + "harmed": 1600, + "missed_rescue": 2, + "avoided_harm": 307, + "selected_both_correct": 2041, + "selected_both_wrong": 701 + }, + { + "budget": 0.56, + "threshold": -0.879269540309906, + "operator": "ge", + "actual_fraction": 0.8180876769071453, + "metric": 0.5990680013807387, + "rescued": 363, + "harmed": 1610, + "missed_rescue": 2, + "avoided_harm": 297, + "selected_both_correct": 2065, + "selected_both_wrong": 702 + }, + { + "budget": 0.57, + "threshold": -0.881222128868103, + "operator": "ge", + "actual_fraction": 0.823610631687953, + "metric": 0.5976872626855367, + "rescued": 363, + "harmed": 1618, + "missed_rescue": 2, + "avoided_harm": 289, + "selected_both_correct": 2086, + "selected_both_wrong": 705 + }, + { + "budget": 0.58, + "threshold": -0.882451593875885, + "operator": "ge", + "actual_fraction": 0.8272350707628582, + "metric": 0.5963065239903348, + "rescued": 363, + "harmed": 1626, + "missed_rescue": 2, + "avoided_harm": 281, + "selected_both_correct": 2099, + "selected_both_wrong": 705 + }, + { + "budget": 0.59, + "threshold": -0.8843295574188232, + "operator": "ge", + "actual_fraction": 0.8313772868484639, + "metric": 0.5957887469796341, + "rescued": 364, + "harmed": 1630, + "missed_rescue": 1, + "avoided_harm": 277, + "selected_both_correct": 2118, + "selected_both_wrong": 705 + }, + { + "budget": 0.6, + "threshold": -0.8863861560821533, + "operator": "ge", + "actual_fraction": 0.8355195029340697, + "metric": 0.5942354159475319, + "rescued": 364, + "harmed": 1639, + "missed_rescue": 1, + "avoided_harm": 268, + "selected_both_correct": 2133, + "selected_both_wrong": 705 + }, + { + "budget": 0.61, + "threshold": -0.8876380920410156, + "operator": "ge", + "actual_fraction": 0.8391439420089748, + "metric": 0.5931998619261305, + "rescued": 364, + "harmed": 1645, + "missed_rescue": 1, + "avoided_harm": 262, + "selected_both_correct": 2148, + "selected_both_wrong": 705 + }, + { + "budget": 0.62, + "threshold": -0.8888679146766663, + "operator": "ge", + "actual_fraction": 0.8425957887469796, + "metric": 0.5923369002416292, + "rescued": 364, + "harmed": 1650, + "missed_rescue": 1, + "avoided_harm": 257, + "selected_both_correct": 2163, + "selected_both_wrong": 705 + }, + { + "budget": 0.63, + "threshold": -0.8905391693115234, + "operator": "ge", + "actual_fraction": 0.8469105971694857, + "metric": 0.5913013462202278, + "rescued": 364, + "harmed": 1656, + "missed_rescue": 1, + "avoided_harm": 251, + "selected_both_correct": 2182, + "selected_both_wrong": 705 + }, + { + "budget": 0.64, + "threshold": -0.8919612169265747, + "operator": "ge", + "actual_fraction": 0.8503624439074905, + "metric": 0.5900931998619261, + "rescued": 364, + "harmed": 1663, + "missed_rescue": 1, + "avoided_harm": 244, + "selected_both_correct": 2195, + "selected_both_wrong": 705 + }, + { + "budget": 0.65, + "threshold": -0.8940081596374512, + "operator": "ge", + "actual_fraction": 0.8560579910251985, + "metric": 0.5881946841560235, + "rescued": 364, + "harmed": 1674, + "missed_rescue": 1, + "avoided_harm": 233, + "selected_both_correct": 2217, + "selected_both_wrong": 705 + }, + { + "budget": 0.66, + "threshold": -0.895609974861145, + "operator": "ge", + "actual_fraction": 0.8603727994477045, + "metric": 0.586468760787021, + "rescued": 365, + "harmed": 1685, + "missed_rescue": 0, + "avoided_harm": 222, + "selected_both_correct": 2230, + "selected_both_wrong": 705 + }, + { + "budget": 0.67, + "threshold": -0.8972456455230713, + "operator": "ge", + "actual_fraction": 0.8643424231964101, + "metric": 0.5861235761132205, + "rescued": 365, + "harmed": 1687, + "missed_rescue": 0, + "avoided_harm": 220, + "selected_both_correct": 2250, + "selected_both_wrong": 706 + }, + { + "budget": 0.68, + "threshold": -0.8986453413963318, + "operator": "ge", + "actual_fraction": 0.8688298239558163, + "metric": 0.5849154297549188, + "rescued": 365, + "harmed": 1694, + "missed_rescue": 0, + "avoided_harm": 213, + "selected_both_correct": 2269, + "selected_both_wrong": 706 + }, + { + "budget": 0.69, + "threshold": -0.8996874094009399, + "operator": "ge", + "actual_fraction": 0.8712461166724197, + "metric": 0.5840524680704177, + "rescued": 365, + "harmed": 1699, + "missed_rescue": 0, + "avoided_harm": 208, + "selected_both_correct": 2278, + "selected_both_wrong": 706 + }, + { + "budget": 0.7, + "threshold": -0.9008907079696655, + "operator": "ge", + "actual_fraction": 0.8746979634104246, + "metric": 0.5826717293752157, + "rescued": 365, + "harmed": 1707, + "missed_rescue": 0, + "avoided_harm": 200, + "selected_both_correct": 2290, + "selected_both_wrong": 706 + }, + { + "budget": 0.71, + "threshold": -0.9031620621681213, + "operator": "ge", + "actual_fraction": 0.8817742492233345, + "metric": 0.5806006213324129, + "rescued": 365, + "harmed": 1719, + "missed_rescue": 0, + "avoided_harm": 188, + "selected_both_correct": 2319, + "selected_both_wrong": 706 + }, + { + "budget": 0.72, + "threshold": -0.9056642055511475, + "operator": "ge", + "actual_fraction": 0.8893683120469451, + "metric": 0.5781843286158095, + "rescued": 365, + "harmed": 1733, + "missed_rescue": 0, + "avoided_harm": 174, + "selected_both_correct": 2348, + "selected_both_wrong": 707 + }, + { + "budget": 0.73, + "threshold": -0.9068957567214966, + "operator": "ge", + "actual_fraction": 0.8933379357956507, + "metric": 0.5773213669313082, + "rescued": 365, + "harmed": 1738, + "missed_rescue": 0, + "avoided_harm": 169, + "selected_both_correct": 2366, + "selected_both_wrong": 707 + }, + { + "budget": 0.74, + "threshold": -0.9093236327171326, + "operator": "ge", + "actual_fraction": 0.8981705212288574, + "metric": 0.5762858129099068, + "rescued": 365, + "harmed": 1744, + "missed_rescue": 0, + "avoided_harm": 163, + "selected_both_correct": 2388, + "selected_both_wrong": 707 + }, + { + "budget": 0.75, + "threshold": -0.9112368822097778, + "operator": "ge", + "actual_fraction": 0.9023127373144633, + "metric": 0.5754228512254056, + "rescued": 365, + "harmed": 1749, + "missed_rescue": 0, + "avoided_harm": 158, + "selected_both_correct": 2407, + "selected_both_wrong": 707 + }, + { + "budget": 0.76, + "threshold": -0.9129221439361572, + "operator": "ge", + "actual_fraction": 0.9081808767690714, + "metric": 0.5738695201933034, + "rescued": 365, + "harmed": 1758, + "missed_rescue": 0, + "avoided_harm": 149, + "selected_both_correct": 2431, + "selected_both_wrong": 708 + }, + { + "budget": 0.77, + "threshold": -0.9144289493560791, + "operator": "ge", + "actual_fraction": 0.9123230928546773, + "metric": 0.5723161891612012, + "rescued": 365, + "harmed": 1767, + "missed_rescue": 0, + "avoided_harm": 140, + "selected_both_correct": 2446, + "selected_both_wrong": 708 + }, + { + "budget": 0.78, + "threshold": -0.9158422946929932, + "operator": "ge", + "actual_fraction": 0.9149119779081809, + "metric": 0.5711080428028995, + "rescued": 365, + "harmed": 1774, + "missed_rescue": 0, + "avoided_harm": 133, + "selected_both_correct": 2454, + "selected_both_wrong": 708 + }, + { + "budget": 0.79, + "threshold": -0.9167781472206116, + "operator": "ge", + "actual_fraction": 0.9168104936140835, + "metric": 0.5705902657921988, + "rescued": 365, + "harmed": 1777, + "missed_rescue": 0, + "avoided_harm": 130, + "selected_both_correct": 2462, + "selected_both_wrong": 708 + }, + { + "budget": 0.8, + "threshold": -0.9187189936637878, + "operator": "ge", + "actual_fraction": 0.922851225405592, + "metric": 0.5683465654124957, + "rescued": 365, + "harmed": 1790, + "missed_rescue": 0, + "avoided_harm": 117, + "selected_both_correct": 2484, + "selected_both_wrong": 708 + }, + { + "budget": 0.81, + "threshold": -0.9207327365875244, + "operator": "ge", + "actual_fraction": 0.9280289955125992, + "metric": 0.5667932343803935, + "rescued": 365, + "harmed": 1799, + "missed_rescue": 0, + "avoided_harm": 108, + "selected_both_correct": 2505, + "selected_both_wrong": 708 + }, + { + "budget": 0.82, + "threshold": -0.9226345419883728, + "operator": "ge", + "actual_fraction": 0.9342423196410079, + "metric": 0.5655850880220918, + "rescued": 365, + "harmed": 1806, + "missed_rescue": 0, + "avoided_harm": 101, + "selected_both_correct": 2534, + "selected_both_wrong": 708 + }, + { + "budget": 0.83, + "threshold": -0.9243993759155273, + "operator": "ge", + "actual_fraction": 0.9387297204004142, + "metric": 0.5642043493268899, + "rescued": 365, + "harmed": 1814, + "missed_rescue": 0, + "avoided_harm": 93, + "selected_both_correct": 2552, + "selected_both_wrong": 708 + }, + { + "budget": 0.84, + "threshold": -0.9264947175979614, + "operator": "ge", + "actual_fraction": 0.9440800828443218, + "metric": 0.5626510182947877, + "rescued": 365, + "harmed": 1823, + "missed_rescue": 0, + "avoided_harm": 84, + "selected_both_correct": 2574, + "selected_both_wrong": 708 + }, + { + "budget": 0.85, + "threshold": -0.9274600148200989, + "operator": "ge", + "actual_fraction": 0.9463237832240249, + "metric": 0.5624784259578874, + "rescued": 365, + "harmed": 1824, + "missed_rescue": 0, + "avoided_harm": 83, + "selected_both_correct": 2586, + "selected_both_wrong": 708 + }, + { + "budget": 0.86, + "threshold": -0.930335283279419, + "operator": "ge", + "actual_fraction": 0.9534000690369347, + "metric": 0.5607525025888851, + "rescued": 365, + "harmed": 1834, + "missed_rescue": 0, + "avoided_harm": 73, + "selected_both_correct": 2616, + "selected_both_wrong": 709 + }, + { + "budget": 0.87, + "threshold": -0.9319414496421814, + "operator": "ge", + "actual_fraction": 0.9558163617535381, + "metric": 0.5600621332412841, + "rescued": 365, + "harmed": 1838, + "missed_rescue": 0, + "avoided_harm": 69, + "selected_both_correct": 2626, + "selected_both_wrong": 709 + }, + { + "budget": 0.88, + "threshold": -0.9340114593505859, + "operator": "ge", + "actual_fraction": 0.958577839143942, + "metric": 0.5591991715567829, + "rescued": 365, + "harmed": 1843, + "missed_rescue": 0, + "avoided_harm": 64, + "selected_both_correct": 2637, + "selected_both_wrong": 709 + }, + { + "budget": 0.89, + "threshold": -0.9363258481025696, + "operator": "ge", + "actual_fraction": 0.9628926475664481, + "metric": 0.5574732481877804, + "rescued": 365, + "harmed": 1853, + "missed_rescue": 0, + "avoided_harm": 54, + "selected_both_correct": 2652, + "selected_both_wrong": 709 + }, + { + "budget": 0.9, + "threshold": -0.9400071501731873, + "operator": "ge", + "actual_fraction": 0.9694511563686572, + "metric": 0.5562651018294787, + "rescued": 365, + "harmed": 1860, + "missed_rescue": 0, + "avoided_harm": 47, + "selected_both_correct": 2683, + "selected_both_wrong": 709 + }, + { + "budget": 0.91, + "threshold": -0.9423198103904724, + "operator": "ge", + "actual_fraction": 0.9734207801173628, + "metric": 0.5550569554711771, + "rescued": 365, + "harmed": 1867, + "missed_rescue": 0, + "avoided_harm": 40, + "selected_both_correct": 2698, + "selected_both_wrong": 710 + }, + { + "budget": 0.92, + "threshold": -0.9443356394767761, + "operator": "ge", + "actual_fraction": 0.9761822575077667, + "metric": 0.5545391784604764, + "rescued": 365, + "harmed": 1870, + "missed_rescue": 0, + "avoided_harm": 37, + "selected_both_correct": 2710, + "selected_both_wrong": 711 + }, + { + "budget": 0.93, + "threshold": -0.9468213319778442, + "operator": "ge", + "actual_fraction": 0.9803244735933725, + "metric": 0.5536762167759751, + "rescued": 365, + "harmed": 1875, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 2729, + "selected_both_wrong": 711 + }, + { + "budget": 0.94, + "threshold": -0.9496435523033142, + "operator": "ge", + "actual_fraction": 0.9837763203313773, + "metric": 0.5526406627545737, + "rescued": 365, + "harmed": 1881, + "missed_rescue": 0, + "avoided_harm": 26, + "selected_both_correct": 2743, + "selected_both_wrong": 711 + }, + { + "budget": 0.95, + "threshold": -0.9529175758361816, + "operator": "ge", + "actual_fraction": 0.9860200207110804, + "metric": 0.5524680704176734, + "rescued": 365, + "harmed": 1882, + "missed_rescue": 0, + "avoided_harm": 25, + "selected_both_correct": 2755, + "selected_both_wrong": 711 + }, + { + "budget": 0.96, + "threshold": -0.9560379981994629, + "operator": "ge", + "actual_fraction": 0.988608905764584, + "metric": 0.5519502934069728, + "rescued": 365, + "harmed": 1885, + "missed_rescue": 0, + "avoided_harm": 22, + "selected_both_correct": 2767, + "selected_both_wrong": 711 + }, + { + "budget": 0.97, + "threshold": -0.9591385126113892, + "operator": "ge", + "actual_fraction": 0.9920607525025888, + "metric": 0.5505695547117708, + "rescued": 365, + "harmed": 1893, + "missed_rescue": 0, + "avoided_harm": 14, + "selected_both_correct": 2779, + "selected_both_wrong": 711 + }, + { + "budget": 0.98, + "threshold": -0.9627243280410767, + "operator": "ge", + "actual_fraction": 0.9946496375560925, + "metric": 0.5498791853641698, + "rescued": 365, + "harmed": 1897, + "missed_rescue": 0, + "avoided_harm": 10, + "selected_both_correct": 2790, + "selected_both_wrong": 711 + }, + { + "budget": 0.99, + "threshold": -0.9702208638191223, + "operator": "ge", + "actual_fraction": 0.9972385226095961, + "metric": 0.5488436313427684, + "rescued": 365, + "harmed": 1903, + "missed_rescue": 0, + "avoided_harm": 4, + "selected_both_correct": 2799, + "selected_both_wrong": 711 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5481532619951675, + "rescued": 365, + "harmed": 1907, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2811, + "selected_both_wrong": 711 + } + ] + }, + "learned": { + "validation_area_0_25": 0.9834833333333333, + "validation_point": { + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.01, + "actual_fraction_std": 0.0, + "per_seed": [ + { + "seed": 2026, + "budget": 0.01, + "threshold": -0.001442005392163992, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + }, + { + "seed": 2027, + "budget": 0.01, + "threshold": -0.008533324114978313, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + }, + { + "seed": 2028, + "budget": 0.01, + "threshold": -0.0035147510934621096, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + }, + { + "seed": 2029, + "budget": 0.01, + "threshold": -0.014309233985841274, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + }, + { + "seed": 2030, + "budget": 0.01, + "threshold": -0.00484488345682621, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + } + ], + "rescued": 0.0, + "harmed": 0.0, + "missed_rescue": 2.0, + "avoided_harm": 524.0 + }, + "test_point": { + "metric": 0.8142906454953401, + "metric_std": 0.00012204121180300483, + "actual_fraction": 0.008146358301691404, + "actual_fraction_std": 0.0018118085567153201, + "per_seed": [ + { + "seed": 2026, + "budget": 0.01, + "threshold": -0.001442005392163992, + "operator": "ge", + "actual_fraction": 0.009492578529513289, + "metric": 0.8144632378322403, + "rescue": 2, + "harm": 1, + "missed_rescue": 363 + }, + { + "seed": 2027, + "budget": 0.01, + "threshold": -0.008533324114978313, + "operator": "ge", + "actual_fraction": 0.006731101139109423, + "metric": 0.8141180531584398, + "rescue": 0, + "harm": 1, + "missed_rescue": 365 + }, + { + "seed": 2028, + "budget": 0.01, + "threshold": -0.0035147510934621096, + "operator": "ge", + "actual_fraction": 0.008111839834311357, + "metric": 0.81429064549534, + "rescue": 0, + "harm": 0, + "missed_rescue": 365 + }, + { + "seed": 2029, + "budget": 0.01, + "threshold": -0.014309233985841274, + "operator": "ge", + "actual_fraction": 0.010355540214014497, + "metric": 0.81429064549534, + "rescue": 0, + "harm": 0, + "missed_rescue": 365 + }, + { + "seed": 2030, + "budget": 0.01, + "threshold": -0.00484488345682621, + "operator": "ge", + "actual_fraction": 0.0060407317915084574, + "metric": 0.81429064549534, + "rescue": 0, + "harm": 0, + "missed_rescue": 365 + } + ], + "rescued": 0.4, + "harmed": 0.4, + "missed_rescue": 364.6, + "avoided_harm": 1906.6 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.01, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.02, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.03, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.04, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.05, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.06, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.07, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.9858333333333335, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.08, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.9856666666666667, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.09, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.9853333333333334, + "metric_std": 0.00045643546458764894, + "actual_fraction": 0.1, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.9853333333333334, + "metric_std": 0.00045643546458764894, + "actual_fraction": 0.11000000000000001, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.12, + "metric": 0.985, + "metric_std": 0.0008333333333333526, + "actual_fraction": 0.12, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.9845, + "metric_std": 0.0009501461875826369, + "actual_fraction": 0.13, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.9845, + "metric_std": 0.0009501461875826369, + "actual_fraction": 0.14, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.9841666666666666, + "metric_std": 0.0005892556509888032, + "actual_fraction": 0.15, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.9838333333333333, + "metric_std": 0.000950146187582598, + "actual_fraction": 0.16, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.9831666666666667, + "metric_std": 0.0012360330811825905, + "actual_fraction": 0.17, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.9824999999999999, + "metric_std": 0.0010206207261596358, + "actual_fraction": 0.18, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.9813333333333333, + "metric_std": 0.0012638125740085772, + "actual_fraction": 0.19, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.9806666666666667, + "metric_std": 0.001236033081182639, + "actual_fraction": 0.2, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.9804999999999999, + "metric_std": 0.001263812574008621, + "actual_fraction": 0.21000000000000002, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.9791666666666666, + "metric_std": 0.0010206207261596358, + "actual_fraction": 0.22000000000000003, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.23, + "metric": 0.9778333333333332, + "metric_std": 0.001513825177048741, + "actual_fraction": 0.23000000000000004, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.24, + "metric": 0.9775, + "metric_std": 0.0016666666666666633, + "actual_fraction": 0.24, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.977, + "metric_std": 0.0009501461875826368, + "actual_fraction": 0.25, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.26, + "metric": 0.9755, + "metric_std": 0.0013944333775567817, + "actual_fraction": 0.26, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.9746666666666666, + "metric_std": 0.0020916500663351973, + "actual_fraction": 0.27, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.9738333333333333, + "metric_std": 0.002400810048481322, + "actual_fraction": 0.28, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.9731666666666665, + "metric_std": 0.002386303510546062, + "actual_fraction": 0.29, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.9719999999999999, + "metric_std": 0.002922612986125006, + "actual_fraction": 0.3, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.9709999999999999, + "metric_std": 0.0027259045389660076, + "actual_fraction": 0.31, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.9690000000000001, + "metric_std": 0.0021570298509241315, + "actual_fraction": 0.32, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.9671666666666667, + "metric_std": 0.001918477405536893, + "actual_fraction": 0.33, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.9641666666666666, + "metric_std": 0.0015590239111557857, + "actual_fraction": 0.34, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.9628333333333334, + "metric_std": 0.001513825177048732, + "actual_fraction": 0.35, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.9606666666666666, + "metric_std": 0.0019002923751652398, + "actual_fraction": 0.36, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.9574999999999999, + "metric_std": 0.0020412414523193283, + "actual_fraction": 0.37, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.9561666666666667, + "metric_std": 0.0022515426810177084, + "actual_fraction": 0.38, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.9536666666666667, + "metric_std": 0.0029814239699997276, + "actual_fraction": 0.39, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.9515, + "metric_std": 0.0025954019170662565, + "actual_fraction": 0.4, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.9490000000000001, + "metric_std": 0.002157029850924123, + "actual_fraction": 0.4091666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.42, + "metric": 0.9461666666666666, + "metric_std": 0.0020069324297987347, + "actual_fraction": 0.42000000000000004, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.9435, + "metric_std": 0.002074983266331461, + "actual_fraction": 0.43, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.44, + "metric": 0.9399999999999998, + "metric_std": 0.002763853991962847, + "actual_fraction": 0.44000000000000006, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.45, + "metric": 0.9363333333333334, + "metric_std": 0.0024008100484813276, + "actual_fraction": 0.45, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.933, + "metric_std": 0.0028012893856619258, + "actual_fraction": 0.4600000000000001, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.47, + "metric": 0.9296666666666666, + "metric_std": 0.0021730674684008818, + "actual_fraction": 0.4699999999999999, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.48, + "metric": 0.9251666666666667, + "metric_std": 0.0025954019170662495, + "actual_fraction": 0.48, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.9216666666666666, + "metric_std": 0.0020412414523193058, + "actual_fraction": 0.49000000000000005, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.9188333333333333, + "metric_std": 0.0015138251770487443, + "actual_fraction": 0.5, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.51, + "metric": 0.9143333333333334, + "metric_std": 0.0021570298509241506, + "actual_fraction": 0.51, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.9094999999999999, + "metric_std": 0.0026744677559801724, + "actual_fraction": 0.52, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.905, + "metric_std": 0.00358430219460108, + "actual_fraction": 0.53, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.8995, + "metric_std": 0.003800584750330459, + "actual_fraction": 0.54, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.8945000000000001, + "metric_std": 0.004191592113319783, + "actual_fraction": 0.55, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.8893333333333334, + "metric_std": 0.00350594732799378, + "actual_fraction": 0.56, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.8841666666666667, + "metric_std": 0.00263523138347364, + "actual_fraction": 0.5691666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.8786666666666667, + "metric_std": 0.002327373340628141, + "actual_fraction": 0.58, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.874, + "metric_std": 0.0019895560643855463, + "actual_fraction": 0.59, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.869, + "metric_std": 0.002157029850924131, + "actual_fraction": 0.6, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.8643333333333334, + "metric_std": 0.0018066236157232188, + "actual_fraction": 0.61, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.8576666666666666, + "metric_std": 0.002527625148017183, + "actual_fraction": 0.62, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.8521666666666666, + "metric_std": 0.0020069324297987347, + "actual_fraction": 0.63, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.8458333333333332, + "metric_std": 0.0023570226039551345, + "actual_fraction": 0.64, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.8413333333333333, + "metric_std": 0.0029814239699997185, + "actual_fraction": 0.65, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.8345, + "metric_std": 0.002674467755980205, + "actual_fraction": 0.66, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.8283333333333334, + "metric_std": 0.0019543398999264266, + "actual_fraction": 0.67, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.8218333333333334, + "metric_std": 0.0026614532371118773, + "actual_fraction": 0.68, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.8146666666666664, + "metric_std": 0.00254132511366282, + "actual_fraction": 0.6891666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.8068333333333333, + "metric_std": 0.002312406154251934, + "actual_fraction": 0.7, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.7996666666666666, + "metric_std": 0.0013944333775567782, + "actual_fraction": 0.71, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.7935, + "metric_std": 0.001236033081182639, + "actual_fraction": 0.72, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.7856666666666667, + "metric_std": 0.0009128709291752675, + "actual_fraction": 0.73, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.7779999999999999, + "metric_std": 0.0016244657241348648, + "actual_fraction": 0.74, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.7701666666666667, + "metric_std": 0.0018066236157232266, + "actual_fraction": 0.75, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.7626666666666667, + "metric_std": 0.0016029486718059332, + "actual_fraction": 0.76, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.7549999999999999, + "metric_std": 0.0013176156917368551, + "actual_fraction": 0.77, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.7471666666666666, + "metric_std": 0.001394433377556785, + "actual_fraction": 0.78, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.7390000000000001, + "metric_std": 0.0013693063937628997, + "actual_fraction": 0.79, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.7311666666666666, + "metric_std": 0.0015138251770487807, + "actual_fraction": 0.8, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.7223333333333334, + "metric_std": 0.0012360330811825979, + "actual_fraction": 0.8100000000000002, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.82, + "metric": 0.7146666666666667, + "metric_std": 0.0018257418583505502, + "actual_fraction": 0.8191666666666666, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.83, + "metric": 0.7058333333333333, + "metric_std": 0.0016666666666666635, + "actual_fraction": 0.8299999999999998, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.6975, + "metric_std": 0.00220479275922047, + "actual_fraction": 0.8400000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.6883333333333335, + "metric_std": 0.002568505834570394, + "actual_fraction": 0.85, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.6801666666666666, + "metric_std": 0.002312406154251932, + "actual_fraction": 0.86, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.87, + "metric": 0.6725000000000001, + "metric_std": 0.0013176156917368551, + "actual_fraction": 0.8699999999999999, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.88, + "metric": 0.6633333333333333, + "metric_std": 0.001020620726159681, + "actual_fraction": 0.8800000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.89, + "metric": 0.6548333333333334, + "metric_std": 0.000912870929175298, + "actual_fraction": 0.89, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.6463333333333334, + "metric_std": 0.0011180339887498626, + "actual_fraction": 0.9, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.6371666666666667, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.9099999999999999, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.6275000000000001, + "metric_std": 0.0008333333333332971, + "actual_fraction": 0.9200000000000002, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.93, + "metric": 0.6178333333333333, + "metric_std": 0.0007453559924998913, + "actual_fraction": 0.93, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.6086666666666666, + "metric_std": 0.00045643546458764894, + "actual_fraction": 0.9399999999999998, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.95, + "metric": 0.5986666666666667, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.95, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.5888333333333333, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.96, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.5790000000000001, + "metric_std": 0.0006972166887783726, + "actual_fraction": 0.97, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.98, + "metric": 0.5698333333333333, + "metric_std": 0.0003726779962499239, + "actual_fraction": 0.9800000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.5605, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.99, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.5508333333333333, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.8142906454953401, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.8142906454953401, + "metric_std": 0.00012204121180300483, + "actual_fraction": 0.008146358301691404, + "actual_fraction_std": 0.0018118085567153201 + }, + { + "budget": 0.02, + "metric": 0.8142906454953401, + "metric_std": 0.00012204121180300483, + "actual_fraction": 0.015464273386261652, + "actual_fraction_std": 0.002026031148198714 + }, + { + "budget": 0.03, + "metric": 0.8142906454953401, + "metric_std": 0.00012204121180300483, + "actual_fraction": 0.02323092854677252, + "actual_fraction_std": 0.0019434847442539518 + }, + { + "budget": 0.04, + "metric": 0.81425612702796, + "metric_std": 0.00014440110917054703, + "actual_fraction": 0.030928546772523297, + "actual_fraction_std": 0.0031551842975782924 + }, + { + "budget": 0.05, + "metric": 0.8142216085605799, + "metric_std": 0.0001543712790818079, + "actual_fraction": 0.03772868484639282, + "actual_fraction_std": 0.0037903572057459045 + }, + { + "budget": 0.06, + "metric": 0.8140490162236796, + "metric_std": 0.0001543712790818079, + "actual_fraction": 0.045253710735243355, + "actual_fraction_std": 0.003738526908267119 + }, + { + "budget": 0.07, + "metric": 0.8139109423541594, + "metric_std": 0.00014440110917054703, + "actual_fraction": 0.05129444252675182, + "actual_fraction_std": 0.00353161204673229 + }, + { + "budget": 0.08, + "metric": 0.8138419054193994, + "metric_std": 9.453271617280195e-05, + "actual_fraction": 0.059337245426303066, + "actual_fraction_std": 0.004552317150224355 + }, + { + "budget": 0.09, + "metric": 0.8136693130824991, + "metric_std": 0.00015437127908176444, + "actual_fraction": 0.0657921988263721, + "actual_fraction_std": 0.005265339343674645 + }, + { + "budget": 0.1, + "metric": 0.813634794615119, + "metric_std": 0.00018906543234557352, + "actual_fraction": 0.07324818778046256, + "actual_fraction_std": 0.005328884479658877 + }, + { + "budget": 0.11, + "metric": 0.8134622022782189, + "metric_std": 0.0001890654323455583, + "actual_fraction": 0.0806351397997929, + "actual_fraction_std": 0.005045780582084395 + }, + { + "budget": 0.12, + "metric": 0.8133241284086987, + "metric_std": 0.00035789508721654746, + "actual_fraction": 0.08843631342768381, + "actual_fraction_std": 0.006510186608371036 + }, + { + "budget": 0.13, + "metric": 0.8132550914739385, + "metric_std": 0.00040476490851081254, + "actual_fraction": 0.09754918881601657, + "actual_fraction_std": 0.008133293701826692 + }, + { + "budget": 0.14, + "metric": 0.8128408698653781, + "metric_std": 0.0001543712790818079, + "actual_fraction": 0.1049361408353469, + "actual_fraction_std": 0.008954864874748256 + }, + { + "budget": 0.15, + "metric": 0.812875388332758, + "metric_std": 0.00014440110917054703, + "actual_fraction": 0.11094235415947531, + "actual_fraction_std": 0.008804239445849345 + }, + { + "budget": 0.16, + "metric": 0.812806351397998, + "metric_std": 0.00028880221834109407, + "actual_fraction": 0.11781152916810493, + "actual_fraction_std": 0.00787766332566993 + }, + { + "budget": 0.17, + "metric": 0.8125302036589576, + "metric_std": 0.00033198798863774085, + "actual_fraction": 0.12509492578529513, + "actual_fraction_std": 0.00785949194101889 + }, + { + "budget": 0.18, + "metric": 0.8123921297894373, + "metric_std": 0.0004566364016335142, + "actual_fraction": 0.1322057300655851, + "actual_fraction_std": 0.00830687473324741 + }, + { + "budget": 0.19, + "metric": 0.812219537452537, + "metric_std": 0.00032289069613166695, + "actual_fraction": 0.14004142216085605, + "actual_fraction_std": 0.012360913276189269 + }, + { + "budget": 0.2, + "metric": 0.8120124266482568, + "metric_std": 0.00028359814851836534, + "actual_fraction": 0.1466344494304453, + "actual_fraction_std": 0.012976946834828909 + }, + { + "budget": 0.21, + "metric": 0.811460131170176, + "metric_std": 0.00035789508721659565, + "actual_fraction": 0.15736969278564034, + "actual_fraction_std": 0.012578772958961126 + }, + { + "budget": 0.22, + "metric": 0.8113565757680359, + "metric_std": 0.00042276315892015784, + "actual_fraction": 0.1646530894028305, + "actual_fraction_std": 0.01351189680776654 + }, + { + "budget": 0.23, + "metric": 0.8109768726268551, + "metric_std": 0.0004120585566556033, + "actual_fraction": 0.17152226441146012, + "actual_fraction_std": 0.013914517939333555 + }, + { + "budget": 0.24, + "metric": 0.8105281325509146, + "metric_std": 0.00044674418728150015, + "actual_fraction": 0.18049706593027268, + "actual_fraction_std": 0.015547877954031614 + }, + { + "budget": 0.25, + "metric": 0.8098722816706939, + "metric_std": 0.0004500657511358334, + "actual_fraction": 0.19036934760096652, + "actual_fraction_std": 0.014501351527524202 + }, + { + "budget": 0.26, + "metric": 0.8094925785295132, + "metric_std": 0.0004467441872814701, + "actual_fraction": 0.1982050396962375, + "actual_fraction_std": 0.01479298575126219 + }, + { + "budget": 0.27, + "metric": 0.8091473938557128, + "metric_std": 0.0007363040734450236, + "actual_fraction": 0.2077666551605109, + "actual_fraction_std": 0.0165964381938009 + }, + { + "budget": 0.28, + "metric": 0.8087676907145323, + "metric_std": 0.000800277743829482, + "actual_fraction": 0.21615464273386262, + "actual_fraction_std": 0.01668647491428839 + }, + { + "budget": 0.29, + "metric": 0.8081808767690714, + "metric_std": 0.0007483425401684159, + "actual_fraction": 0.22544011045909565, + "actual_fraction_std": 0.017784298300806543 + }, + { + "budget": 0.3, + "metric": 0.8075250258888506, + "metric_std": 0.0006150683355610761, + "actual_fraction": 0.234276838108388, + "actual_fraction_std": 0.018679323450797036 + }, + { + "budget": 0.31, + "metric": 0.8070072488781499, + "metric_std": 0.000916528688115404, + "actual_fraction": 0.24432171211598205, + "actual_fraction_std": 0.02025428248065419 + }, + { + "budget": 0.32, + "metric": 0.8062133241284087, + "metric_std": 0.0010239832222430686, + "actual_fraction": 0.2561615464273386, + "actual_fraction_std": 0.020825609581064584 + }, + { + "budget": 0.33, + "metric": 0.805626510182948, + "metric_std": 0.0012409857102000396, + "actual_fraction": 0.2655850880220918, + "actual_fraction_std": 0.021546893515383806 + }, + { + "budget": 0.34, + "metric": 0.8043838453572661, + "metric_std": 0.0014776565269592067, + "actual_fraction": 0.27566448049706593, + "actual_fraction_std": 0.02381317711244374 + }, + { + "budget": 0.35, + "metric": 0.803520883672765, + "metric_std": 0.0017710698698467516, + "actual_fraction": 0.2857783914394201, + "actual_fraction_std": 0.0267090196341251 + }, + { + "budget": 0.36, + "metric": 0.8026234035208837, + "metric_std": 0.0020001359483352854, + "actual_fraction": 0.29789437348981707, + "actual_fraction_std": 0.027540513284834225 + }, + { + "budget": 0.37, + "metric": 0.802174663444943, + "metric_std": 0.0019311840843215958, + "actual_fraction": 0.3106662064204349, + "actual_fraction_std": 0.026683244160227614 + }, + { + "budget": 0.38, + "metric": 0.8011045909561616, + "metric_std": 0.002080435789479345, + "actual_fraction": 0.32440455643769417, + "actual_fraction_std": 0.028583344098724126 + }, + { + "budget": 0.39, + "metric": 0.8008284432171212, + "metric_std": 0.0021383347762105607, + "actual_fraction": 0.3343458750431481, + "actual_fraction_std": 0.030248844350598124 + }, + { + "budget": 0.4, + "metric": 0.799378667587159, + "metric_std": 0.003013207158135341, + "actual_fraction": 0.34425267518122193, + "actual_fraction_std": 0.03331917302761277 + }, + { + "budget": 0.41, + "metric": 0.7984121505005177, + "metric_std": 0.0029794056785695778, + "actual_fraction": 0.3544356230583362, + "actual_fraction_std": 0.03387574750940587 + }, + { + "budget": 0.42, + "metric": 0.7979634104245771, + "metric_std": 0.0031861872997485264, + "actual_fraction": 0.3665516051087332, + "actual_fraction_std": 0.03152194802838754 + }, + { + "budget": 0.43, + "metric": 0.7968243010010355, + "metric_std": 0.003489183643928435, + "actual_fraction": 0.38298239558163616, + "actual_fraction_std": 0.033744339891144684 + }, + { + "budget": 0.44, + "metric": 0.7954435623058336, + "metric_std": 0.003827115888742187, + "actual_fraction": 0.39654815326199516, + "actual_fraction_std": 0.03522714875040597 + }, + { + "budget": 0.45, + "metric": 0.7942008974801519, + "metric_std": 0.004586866626864949, + "actual_fraction": 0.40945806006213326, + "actual_fraction_std": 0.040319542546584486 + }, + { + "budget": 0.46, + "metric": 0.7923369002416292, + "metric_std": 0.004714646458796176, + "actual_fraction": 0.42222989299275115, + "actual_fraction_std": 0.0432715871422603 + }, + { + "budget": 0.47, + "metric": 0.7908526061442872, + "metric_std": 0.004553952732912922, + "actual_fraction": 0.43438039351052815, + "actual_fraction_std": 0.04104113675983878 + }, + { + "budget": 0.48, + "metric": 0.7888505350362444, + "metric_std": 0.0045673424180925504, + "actual_fraction": 0.44787711425612703, + "actual_fraction_std": 0.04296076009746757 + }, + { + "budget": 0.49, + "metric": 0.7866413531239212, + "metric_std": 0.00437037060007359, + "actual_fraction": 0.4626855367621677, + "actual_fraction_std": 0.04223814428420171 + }, + { + "budget": 0.5, + "metric": 0.7853986882982397, + "metric_std": 0.004627277481101205, + "actual_fraction": 0.4737659647911633, + "actual_fraction_std": 0.041974193754515436 + }, + { + "budget": 0.51, + "metric": 0.7834656541249568, + "metric_std": 0.0036726092035265032, + "actual_fraction": 0.4879530548843631, + "actual_fraction_std": 0.04214565631837744 + }, + { + "budget": 0.52, + "metric": 0.7815326199516741, + "metric_std": 0.004046545030238835, + "actual_fraction": 0.5025888850535036, + "actual_fraction_std": 0.04397363500295294 + }, + { + "budget": 0.53, + "metric": 0.7785985502243701, + "metric_std": 0.004446388620443855, + "actual_fraction": 0.5151536071798413, + "actual_fraction_std": 0.04355766939019842 + }, + { + "budget": 0.54, + "metric": 0.7767345529858474, + "metric_std": 0.004426581286608026, + "actual_fraction": 0.5305143251639627, + "actual_fraction_std": 0.04327344577948368 + }, + { + "budget": 0.55, + "metric": 0.7741801863997239, + "metric_std": 0.004219873813462589, + "actual_fraction": 0.5445633413876425, + "actual_fraction_std": 0.04403073733577918 + }, + { + "budget": 0.56, + "metric": 0.7706247842595789, + "metric_std": 0.004512225220021179, + "actual_fraction": 0.5588885053503625, + "actual_fraction_std": 0.044049168920498594 + }, + { + "budget": 0.57, + "metric": 0.7686227131515361, + "metric_std": 0.004668933751826342, + "actual_fraction": 0.5703486365205385, + "actual_fraction_std": 0.044204971172463206 + }, + { + "budget": 0.58, + "metric": 0.7663099758370728, + "metric_std": 0.005521606238538647, + "actual_fraction": 0.5818432861580946, + "actual_fraction_std": 0.044170219851466554 + }, + { + "budget": 0.59, + "metric": 0.7636865723161892, + "metric_std": 0.006586156140624602, + "actual_fraction": 0.5946841560234726, + "actual_fraction_std": 0.04703016089009171 + }, + { + "budget": 0.6, + "metric": 0.7609596133931653, + "metric_std": 0.006243216171103195, + "actual_fraction": 0.6093890231273732, + "actual_fraction_std": 0.04116343517334494 + }, + { + "budget": 0.61, + "metric": 0.7583016914049017, + "metric_std": 0.006915118264998005, + "actual_fraction": 0.6240248532965136, + "actual_fraction_std": 0.040637084715821524 + }, + { + "budget": 0.62, + "metric": 0.7551950293406973, + "metric_std": 0.007216547722361081, + "actual_fraction": 0.6382809803244734, + "actual_fraction_std": 0.037345089401636965 + }, + { + "budget": 0.63, + "metric": 0.752157404211253, + "metric_std": 0.007268371311441335, + "actual_fraction": 0.652088367276493, + "actual_fraction_std": 0.03645528498633828 + }, + { + "budget": 0.64, + "metric": 0.7480151881256473, + "metric_std": 0.007374157815529268, + "actual_fraction": 0.6666206420434933, + "actual_fraction_std": 0.03380246364119087 + }, + { + "budget": 0.65, + "metric": 0.7434242319641008, + "metric_std": 0.00937225654762593, + "actual_fraction": 0.6833275802554367, + "actual_fraction_std": 0.03389196734855529 + }, + { + "budget": 0.66, + "metric": 0.7390058681394546, + "metric_std": 0.010059631377511563, + "actual_fraction": 0.6967897825336555, + "actual_fraction_std": 0.03172634944819162 + }, + { + "budget": 0.67, + "metric": 0.7346220227821885, + "metric_std": 0.010615901966341459, + "actual_fraction": 0.7094580600621333, + "actual_fraction_std": 0.030840613492851093 + }, + { + "budget": 0.68, + "metric": 0.7285122540559199, + "metric_std": 0.011757199774113058, + "actual_fraction": 0.7251294442526752, + "actual_fraction_std": 0.031115148306596504 + }, + { + "budget": 0.69, + "metric": 0.7230238177424922, + "metric_std": 0.013194794212810066, + "actual_fraction": 0.7374525371073524, + "actual_fraction_std": 0.03289896979009793 + }, + { + "budget": 0.7, + "metric": 0.7170866413531238, + "metric_std": 0.014456295400222237, + "actual_fraction": 0.750776665516051, + "actual_fraction_std": 0.03419615661073426 + }, + { + "budget": 0.71, + "metric": 0.710562651018295, + "metric_std": 0.0157087473289372, + "actual_fraction": 0.76420434932689, + "actual_fraction_std": 0.036217135568146534 + }, + { + "budget": 0.72, + "metric": 0.7040386606834657, + "metric_std": 0.014486142758734258, + "actual_fraction": 0.7778736624093889, + "actual_fraction_std": 0.03211864094565943 + }, + { + "budget": 0.73, + "metric": 0.6975491888160166, + "metric_std": 0.015381841173585481, + "actual_fraction": 0.7898170521228858, + "actual_fraction_std": 0.03227693586860327 + }, + { + "budget": 0.74, + "metric": 0.6911287538833275, + "metric_std": 0.014552410255339567, + "actual_fraction": 0.8014497756299621, + "actual_fraction_std": 0.029627021786071062 + }, + { + "budget": 0.75, + "metric": 0.6849154297549189, + "metric_std": 0.012892890163835918, + "actual_fraction": 0.8122885743872972, + "actual_fraction_std": 0.026972653987640524 + }, + { + "budget": 0.76, + "metric": 0.6796686227131516, + "metric_std": 0.01328624732752346, + "actual_fraction": 0.8219192267863307, + "actual_fraction_std": 0.026788365686784766 + }, + { + "budget": 0.77, + "metric": 0.6745253710735243, + "metric_std": 0.013436399335605097, + "actual_fraction": 0.8297894373489816, + "actual_fraction_std": 0.02570710570056535 + }, + { + "budget": 0.78, + "metric": 0.6676907145322748, + "metric_std": 0.011634695092214974, + "actual_fraction": 0.8401104590956162, + "actual_fraction_std": 0.02254745655122583 + }, + { + "budget": 0.79, + "metric": 0.6600276147739039, + "metric_std": 0.01114885045398072, + "actual_fraction": 0.8538142906454953, + "actual_fraction_std": 0.020026661656003135 + }, + { + "budget": 0.8, + "metric": 0.655022437003797, + "metric_std": 0.01110199452480238, + "actual_fraction": 0.8622713151536072, + "actual_fraction_std": 0.019075552831053257 + }, + { + "budget": 0.81, + "metric": 0.6502243700379703, + "metric_std": 0.010092299083337645, + "actual_fraction": 0.8701415257162581, + "actual_fraction_std": 0.01666503916320078 + }, + { + "budget": 0.82, + "metric": 0.6466344494304452, + "metric_std": 0.010464704787045169, + "actual_fraction": 0.8759406282361063, + "actual_fraction_std": 0.017060209050687444 + }, + { + "budget": 0.83, + "metric": 0.6418363824646185, + "metric_std": 0.011961533045530461, + "actual_fraction": 0.8834656541249568, + "actual_fraction_std": 0.01847076706396573 + }, + { + "budget": 0.84, + "metric": 0.6353469105971696, + "metric_std": 0.008833794441127321, + "actual_fraction": 0.8931308249913703, + "actual_fraction_std": 0.014048740582108369 + }, + { + "budget": 0.85, + "metric": 0.6291681049361408, + "metric_std": 0.009528285058468233, + "actual_fraction": 0.9023817742492234, + "actual_fraction_std": 0.014206980958963673 + }, + { + "budget": 0.86, + "metric": 0.623610631687953, + "metric_std": 0.008753341577771806, + "actual_fraction": 0.9098032447359337, + "actual_fraction_std": 0.013305626720805562 + }, + { + "budget": 0.87, + "metric": 0.6184328615809458, + "metric_std": 0.0083608473792742, + "actual_fraction": 0.9168450120814636, + "actual_fraction_std": 0.013364710760208886 + }, + { + "budget": 0.88, + "metric": 0.6134967207455989, + "metric_std": 0.007736488801815069, + "actual_fraction": 0.9238867794269934, + "actual_fraction_std": 0.012431441563941397 + }, + { + "budget": 0.89, + "metric": 0.6084570245081118, + "metric_std": 0.006711822848268334, + "actual_fraction": 0.9304798066965827, + "actual_fraction_std": 0.010340715426994251 + }, + { + "budget": 0.9, + "metric": 0.6013807386952019, + "metric_std": 0.006252275025556675, + "actual_fraction": 0.9395581636175354, + "actual_fraction_std": 0.009380040221492924 + }, + { + "budget": 0.91, + "metric": 0.5937521574042113, + "metric_std": 0.005723459600445173, + "actual_fraction": 0.9490162236796686, + "actual_fraction_std": 0.008335155670233185 + }, + { + "budget": 0.92, + "metric": 0.5881601656886435, + "metric_std": 0.0028053555779466726, + "actual_fraction": 0.9558853986882981, + "actual_fraction_std": 0.004314117997920028 + }, + { + "budget": 0.93, + "metric": 0.5833275802554366, + "metric_std": 0.0029300059168672586, + "actual_fraction": 0.9616499827407663, + "actual_fraction_std": 0.004223049171490669 + }, + { + "budget": 0.94, + "metric": 0.5772178115291681, + "metric_std": 0.0016760128942092144, + "actual_fraction": 0.9691059716948567, + "actual_fraction_std": 0.002045783498641628 + }, + { + "budget": 0.95, + "metric": 0.5730065585088022, + "metric_std": 0.001712930034628878, + "actual_fraction": 0.9740075940628236, + "actual_fraction_std": 0.0019319551700318605 + }, + { + "budget": 0.96, + "metric": 0.5667587159130134, + "metric_std": 0.001216745373283677, + "actual_fraction": 0.9807041767345531, + "actual_fraction_std": 0.0011667548347363006 + }, + { + "budget": 0.97, + "metric": 0.5616499827407664, + "metric_std": 0.0011920121953629566, + "actual_fraction": 0.9859509837763204, + "actual_fraction_std": 0.0011357047895314088 + }, + { + "budget": 0.98, + "metric": 0.5568519157749396, + "metric_std": 0.0011616374643054219, + "actual_fraction": 0.9909216430790473, + "actual_fraction_std": 0.0011158598464097187 + }, + { + "budget": 0.99, + "metric": 0.5524680704176734, + "metric_std": 0.0013087461933123314, + "actual_fraction": 0.9955471177079737, + "actual_fraction_std": 0.0013446704301541424 + }, + { + "budget": 1.0, + "metric": 0.5481532619951675, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 659, + "rescuable": 2, + "harmful": 524, + "both_wrong": 15, + "unit": "sample", + "total": 1200 + }, + "test": { + "both_correct": 2811, + "rescuable": 365, + "harmful": 1907, + "both_wrong": 711, + "unit": "sample", + "total": 5794 + } + }, + "small": { + "validation": 0.9858333333333333, + "test": 0.81429064549534 + }, + "large": { + "validation": 0.5508333333333333, + "test": 0.5481532619951675 + }, + "learned_minus_confidence_test": -0.00155333103210209, + "learned_minus_small_test": 1.1102230246251565e-16, + "learned_minus_large_test": 0.2661373835001727, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_cub_Qwen3.8" + }, + { + "key": [ + "cub", + "glsim", + "MiniCPM" + ], + "expert_display": "GLSim", + "run": "five_seed_glsim_MiniCPM_four_difference", + "validation_rows": 1200, + "test_rows": 5794, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/glsim/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9005, + "validation_point": { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.985, + "rescued": 1, + "harmed": 6, + "missed_rescue": 2, + "avoided_harm": 851, + "selected_both_correct": 0, + "selected_both_wrong": 5 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.08698653779772178, + "metric": 0.8769416637901277, + "rescued": 50, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 3598, + "selected_both_correct": 42, + "selected_both_wrong": 219 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9891666666666666, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3, + "avoided_harm": 857, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.985, + "rescued": 1, + "harmed": 6, + "missed_rescue": 2, + "avoided_harm": 851, + "selected_both_correct": 0, + "selected_both_wrong": 5 + }, + { + "budget": 0.02, + "threshold": -0.6014261841773987, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.9791666666666666, + "rescued": 2, + "harmed": 14, + "missed_rescue": 1, + "avoided_harm": 843, + "selected_both_correct": 2, + "selected_both_wrong": 6 + }, + { + "budget": 0.03, + "threshold": -0.6401472687721252, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9716666666666667, + "rescued": 2, + "harmed": 23, + "missed_rescue": 1, + "avoided_harm": 834, + "selected_both_correct": 3, + "selected_both_wrong": 8 + }, + { + "budget": 0.04, + "threshold": -0.6884650588035583, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9658333333333333, + "rescued": 3, + "harmed": 31, + "missed_rescue": 0, + "avoided_harm": 826, + "selected_both_correct": 5, + "selected_both_wrong": 9 + }, + { + "budget": 0.05, + "threshold": -0.7208121418952942, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.9591666666666666, + "rescued": 3, + "harmed": 39, + "missed_rescue": 0, + "avoided_harm": 818, + "selected_both_correct": 9, + "selected_both_wrong": 9 + }, + { + "budget": 0.06, + "threshold": -0.7578181624412537, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.9516666666666667, + "rescued": 3, + "harmed": 48, + "missed_rescue": 0, + "avoided_harm": 809, + "selected_both_correct": 12, + "selected_both_wrong": 9 + }, + { + "budget": 0.07, + "threshold": -0.7790470719337463, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.9441666666666667, + "rescued": 3, + "harmed": 57, + "missed_rescue": 0, + "avoided_harm": 800, + "selected_both_correct": 15, + "selected_both_wrong": 9 + }, + { + "budget": 0.08, + "threshold": -0.7927228808403015, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.9375, + "rescued": 3, + "harmed": 65, + "missed_rescue": 0, + "avoided_harm": 792, + "selected_both_correct": 18, + "selected_both_wrong": 10 + }, + { + "budget": 0.09, + "threshold": -0.8049252033233643, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.93, + "rescued": 3, + "harmed": 74, + "missed_rescue": 0, + "avoided_harm": 783, + "selected_both_correct": 21, + "selected_both_wrong": 10 + }, + { + "budget": 0.1, + "threshold": -0.80850750207901, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9216666666666666, + "rescued": 3, + "harmed": 84, + "missed_rescue": 0, + "avoided_harm": 773, + "selected_both_correct": 23, + "selected_both_wrong": 10 + }, + { + "budget": 0.11, + "threshold": -0.819871723651886, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9133333333333333, + "rescued": 3, + "harmed": 94, + "missed_rescue": 0, + "avoided_harm": 763, + "selected_both_correct": 25, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -0.8310170769691467, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.905, + "rescued": 3, + "harmed": 104, + "missed_rescue": 0, + "avoided_harm": 753, + "selected_both_correct": 27, + "selected_both_wrong": 10 + }, + { + "budget": 0.13, + "threshold": -0.8376906514167786, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.8966666666666666, + "rescued": 3, + "harmed": 114, + "missed_rescue": 0, + "avoided_harm": 743, + "selected_both_correct": 29, + "selected_both_wrong": 10 + }, + { + "budget": 0.14, + "threshold": -0.8419997692108154, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.8875, + "rescued": 3, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 732, + "selected_both_correct": 30, + "selected_both_wrong": 10 + }, + { + "budget": 0.15, + "threshold": -0.8447275161743164, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.8808333333333334, + "rescued": 3, + "harmed": 133, + "missed_rescue": 0, + "avoided_harm": 724, + "selected_both_correct": 34, + "selected_both_wrong": 10 + }, + { + "budget": 0.16, + "threshold": -0.8506776690483093, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.8716666666666667, + "rescued": 3, + "harmed": 144, + "missed_rescue": 0, + "avoided_harm": 713, + "selected_both_correct": 35, + "selected_both_wrong": 10 + }, + { + "budget": 0.17, + "threshold": -0.8545639514923096, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.8633333333333333, + "rescued": 3, + "harmed": 154, + "missed_rescue": 0, + "avoided_harm": 703, + "selected_both_correct": 37, + "selected_both_wrong": 10 + }, + { + "budget": 0.18, + "threshold": -0.8578683137893677, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.855, + "rescued": 3, + "harmed": 164, + "missed_rescue": 0, + "avoided_harm": 693, + "selected_both_correct": 39, + "selected_both_wrong": 10 + }, + { + "budget": 0.19, + "threshold": -0.86090087890625, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.8491666666666666, + "rescued": 3, + "harmed": 171, + "missed_rescue": 0, + "avoided_harm": 686, + "selected_both_correct": 44, + "selected_both_wrong": 10 + }, + { + "budget": 0.2, + "threshold": -0.8640955090522766, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.8425, + "rescued": 3, + "harmed": 179, + "missed_rescue": 0, + "avoided_harm": 678, + "selected_both_correct": 48, + "selected_both_wrong": 10 + }, + { + "budget": 0.21, + "threshold": -0.8664237260818481, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.8358333333333333, + "rescued": 3, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 670, + "selected_both_correct": 52, + "selected_both_wrong": 10 + }, + { + "budget": 0.22, + "threshold": -0.8686786890029907, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.8283333333333334, + "rescued": 3, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 661, + "selected_both_correct": 55, + "selected_both_wrong": 10 + }, + { + "budget": 0.23, + "threshold": -0.8709638118743896, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.8208333333333333, + "rescued": 3, + "harmed": 205, + "missed_rescue": 0, + "avoided_harm": 652, + "selected_both_correct": 58, + "selected_both_wrong": 10 + }, + { + "budget": 0.24, + "threshold": -0.8724074959754944, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.8158333333333333, + "rescued": 3, + "harmed": 211, + "missed_rescue": 0, + "avoided_harm": 646, + "selected_both_correct": 64, + "selected_both_wrong": 10 + }, + { + "budget": 0.25, + "threshold": -0.8736594319343567, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.8075, + "rescued": 3, + "harmed": 221, + "missed_rescue": 0, + "avoided_harm": 636, + "selected_both_correct": 66, + "selected_both_wrong": 10 + }, + { + "budget": 0.26, + "threshold": -0.8752554059028625, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.8016666666666666, + "rescued": 3, + "harmed": 228, + "missed_rescue": 0, + "avoided_harm": 629, + "selected_both_correct": 71, + "selected_both_wrong": 10 + }, + { + "budget": 0.27, + "threshold": -0.8766024112701416, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.7933333333333333, + "rescued": 3, + "harmed": 238, + "missed_rescue": 0, + "avoided_harm": 619, + "selected_both_correct": 73, + "selected_both_wrong": 10 + }, + { + "budget": 0.28, + "threshold": -0.878311276435852, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.7866666666666666, + "rescued": 3, + "harmed": 246, + "missed_rescue": 0, + "avoided_harm": 611, + "selected_both_correct": 77, + "selected_both_wrong": 10 + }, + { + "budget": 0.29, + "threshold": -0.880103588104248, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.78, + "rescued": 3, + "harmed": 254, + "missed_rescue": 0, + "avoided_harm": 603, + "selected_both_correct": 81, + "selected_both_wrong": 10 + }, + { + "budget": 0.3, + "threshold": -0.8824952244758606, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.7725, + "rescued": 3, + "harmed": 263, + "missed_rescue": 0, + "avoided_harm": 594, + "selected_both_correct": 84, + "selected_both_wrong": 10 + }, + { + "budget": 0.31, + "threshold": -0.8831692337989807, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.7658333333333334, + "rescued": 3, + "harmed": 271, + "missed_rescue": 0, + "avoided_harm": 586, + "selected_both_correct": 88, + "selected_both_wrong": 10 + }, + { + "budget": 0.32, + "threshold": -0.8845130205154419, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.76, + "rescued": 3, + "harmed": 278, + "missed_rescue": 0, + "avoided_harm": 579, + "selected_both_correct": 93, + "selected_both_wrong": 10 + }, + { + "budget": 0.33, + "threshold": -0.8855781555175781, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.7508333333333334, + "rescued": 3, + "harmed": 289, + "missed_rescue": 0, + "avoided_harm": 568, + "selected_both_correct": 94, + "selected_both_wrong": 10 + }, + { + "budget": 0.34, + "threshold": -0.8865919709205627, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.7433333333333333, + "rescued": 3, + "harmed": 298, + "missed_rescue": 0, + "avoided_harm": 559, + "selected_both_correct": 97, + "selected_both_wrong": 10 + }, + { + "budget": 0.35, + "threshold": -0.8877584338188171, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.735, + "rescued": 3, + "harmed": 308, + "missed_rescue": 0, + "avoided_harm": 549, + "selected_both_correct": 99, + "selected_both_wrong": 10 + }, + { + "budget": 0.36, + "threshold": -0.8890330195426941, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.7291666666666666, + "rescued": 3, + "harmed": 315, + "missed_rescue": 0, + "avoided_harm": 542, + "selected_both_correct": 104, + "selected_both_wrong": 10 + }, + { + "budget": 0.37, + "threshold": -0.890850305557251, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.7225, + "rescued": 3, + "harmed": 323, + "missed_rescue": 0, + "avoided_harm": 534, + "selected_both_correct": 108, + "selected_both_wrong": 10 + }, + { + "budget": 0.38, + "threshold": -0.8916605710983276, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.7183333333333334, + "rescued": 3, + "harmed": 328, + "missed_rescue": 0, + "avoided_harm": 529, + "selected_both_correct": 115, + "selected_both_wrong": 10 + }, + { + "budget": 0.39, + "threshold": -0.8928589224815369, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.7116666666666667, + "rescued": 3, + "harmed": 336, + "missed_rescue": 0, + "avoided_harm": 521, + "selected_both_correct": 119, + "selected_both_wrong": 10 + }, + { + "budget": 0.4, + "threshold": -0.8940943479537964, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.7033333333333334, + "rescued": 3, + "harmed": 346, + "missed_rescue": 0, + "avoided_harm": 511, + "selected_both_correct": 121, + "selected_both_wrong": 10 + }, + { + "budget": 0.41, + "threshold": -0.8949463963508606, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.6983333333333334, + "rescued": 3, + "harmed": 352, + "missed_rescue": 0, + "avoided_harm": 505, + "selected_both_correct": 126, + "selected_both_wrong": 10 + }, + { + "budget": 0.42, + "threshold": -0.8960229158401489, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.6916666666666667, + "rescued": 3, + "harmed": 360, + "missed_rescue": 0, + "avoided_harm": 497, + "selected_both_correct": 131, + "selected_both_wrong": 10 + }, + { + "budget": 0.43, + "threshold": -0.8966536521911621, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.6841666666666667, + "rescued": 3, + "harmed": 369, + "missed_rescue": 0, + "avoided_harm": 488, + "selected_both_correct": 134, + "selected_both_wrong": 10 + }, + { + "budget": 0.44, + "threshold": -0.8975363373756409, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.68, + "rescued": 3, + "harmed": 374, + "missed_rescue": 0, + "avoided_harm": 483, + "selected_both_correct": 141, + "selected_both_wrong": 10 + }, + { + "budget": 0.45, + "threshold": -0.8988122940063477, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.6741666666666667, + "rescued": 3, + "harmed": 381, + "missed_rescue": 0, + "avoided_harm": 476, + "selected_both_correct": 146, + "selected_both_wrong": 10 + }, + { + "budget": 0.46, + "threshold": -0.9000959396362305, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.665, + "rescued": 3, + "harmed": 392, + "missed_rescue": 0, + "avoided_harm": 465, + "selected_both_correct": 147, + "selected_both_wrong": 10 + }, + { + "budget": 0.47, + "threshold": -0.9013051986694336, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.6583333333333333, + "rescued": 3, + "harmed": 400, + "missed_rescue": 0, + "avoided_harm": 457, + "selected_both_correct": 151, + "selected_both_wrong": 10 + }, + { + "budget": 0.48, + "threshold": -0.9029324054718018, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.6525, + "rescued": 3, + "harmed": 407, + "missed_rescue": 0, + "avoided_harm": 450, + "selected_both_correct": 156, + "selected_both_wrong": 10 + }, + { + "budget": 0.49, + "threshold": -0.9036871790885925, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.6475, + "rescued": 3, + "harmed": 413, + "missed_rescue": 0, + "avoided_harm": 444, + "selected_both_correct": 162, + "selected_both_wrong": 10 + }, + { + "budget": 0.5, + "threshold": -0.9044621586799622, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.6425, + "rescued": 3, + "harmed": 419, + "missed_rescue": 0, + "avoided_harm": 438, + "selected_both_correct": 168, + "selected_both_wrong": 10 + }, + { + "budget": 0.51, + "threshold": -0.9054575562477112, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.6358333333333334, + "rescued": 3, + "harmed": 427, + "missed_rescue": 0, + "avoided_harm": 430, + "selected_both_correct": 172, + "selected_both_wrong": 10 + }, + { + "budget": 0.52, + "threshold": -0.906682550907135, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.6266666666666667, + "rescued": 3, + "harmed": 438, + "missed_rescue": 0, + "avoided_harm": 419, + "selected_both_correct": 173, + "selected_both_wrong": 10 + }, + { + "budget": 0.53, + "threshold": -0.9072114825248718, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.6225, + "rescued": 3, + "harmed": 443, + "missed_rescue": 0, + "avoided_harm": 414, + "selected_both_correct": 180, + "selected_both_wrong": 10 + }, + { + "budget": 0.54, + "threshold": -0.9083563089370728, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.6158333333333333, + "rescued": 3, + "harmed": 451, + "missed_rescue": 0, + "avoided_harm": 406, + "selected_both_correct": 184, + "selected_both_wrong": 10 + }, + { + "budget": 0.55, + "threshold": -0.9095192551612854, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.6091666666666666, + "rescued": 3, + "harmed": 459, + "missed_rescue": 0, + "avoided_harm": 398, + "selected_both_correct": 188, + "selected_both_wrong": 10 + }, + { + "budget": 0.56, + "threshold": -0.9106979370117188, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.6016666666666667, + "rescued": 3, + "harmed": 468, + "missed_rescue": 0, + "avoided_harm": 389, + "selected_both_correct": 191, + "selected_both_wrong": 10 + }, + { + "budget": 0.57, + "threshold": -0.9115362167358398, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.595, + "rescued": 3, + "harmed": 476, + "missed_rescue": 0, + "avoided_harm": 381, + "selected_both_correct": 194, + "selected_both_wrong": 10 + }, + { + "budget": 0.58, + "threshold": -0.9124338030815125, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.5866666666666667, + "rescued": 3, + "harmed": 486, + "missed_rescue": 0, + "avoided_harm": 371, + "selected_both_correct": 197, + "selected_both_wrong": 10 + }, + { + "budget": 0.59, + "threshold": -0.9134801626205444, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.5783333333333334, + "rescued": 3, + "harmed": 496, + "missed_rescue": 0, + "avoided_harm": 361, + "selected_both_correct": 199, + "selected_both_wrong": 10 + }, + { + "budget": 0.6, + "threshold": -0.9143747091293335, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.5733333333333334, + "rescued": 3, + "harmed": 502, + "missed_rescue": 0, + "avoided_harm": 355, + "selected_both_correct": 205, + "selected_both_wrong": 10 + }, + { + "budget": 0.61, + "threshold": -0.915804386138916, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.5641666666666667, + "rescued": 3, + "harmed": 513, + "missed_rescue": 0, + "avoided_harm": 344, + "selected_both_correct": 206, + "selected_both_wrong": 10 + }, + { + "budget": 0.62, + "threshold": -0.9165146350860596, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.555, + "rescued": 3, + "harmed": 524, + "missed_rescue": 0, + "avoided_harm": 333, + "selected_both_correct": 207, + "selected_both_wrong": 10 + }, + { + "budget": 0.63, + "threshold": -0.9172822833061218, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.55, + "rescued": 3, + "harmed": 530, + "missed_rescue": 0, + "avoided_harm": 327, + "selected_both_correct": 213, + "selected_both_wrong": 10 + }, + { + "budget": 0.64, + "threshold": -0.9181071519851685, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.5416666666666666, + "rescued": 3, + "harmed": 540, + "missed_rescue": 0, + "avoided_harm": 317, + "selected_both_correct": 215, + "selected_both_wrong": 10 + }, + { + "budget": 0.65, + "threshold": -0.9192029237747192, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.5383333333333333, + "rescued": 3, + "harmed": 544, + "missed_rescue": 0, + "avoided_harm": 313, + "selected_both_correct": 223, + "selected_both_wrong": 10 + }, + { + "budget": 0.66, + "threshold": -0.9200543761253357, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.5333333333333333, + "rescued": 3, + "harmed": 550, + "missed_rescue": 0, + "avoided_harm": 307, + "selected_both_correct": 229, + "selected_both_wrong": 10 + }, + { + "budget": 0.67, + "threshold": -0.9208071231842041, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.5266666666666666, + "rescued": 3, + "harmed": 558, + "missed_rescue": 0, + "avoided_harm": 299, + "selected_both_correct": 233, + "selected_both_wrong": 10 + }, + { + "budget": 0.68, + "threshold": -0.9215553998947144, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.5175, + "rescued": 3, + "harmed": 569, + "missed_rescue": 0, + "avoided_harm": 288, + "selected_both_correct": 234, + "selected_both_wrong": 10 + }, + { + "budget": 0.69, + "threshold": -0.9220835566520691, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.5116666666666667, + "rescued": 3, + "harmed": 576, + "missed_rescue": 0, + "avoided_harm": 281, + "selected_both_correct": 238, + "selected_both_wrong": 10 + }, + { + "budget": 0.7, + "threshold": -0.9232237935066223, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.5041666666666667, + "rescued": 3, + "harmed": 585, + "missed_rescue": 0, + "avoided_harm": 272, + "selected_both_correct": 242, + "selected_both_wrong": 10 + }, + { + "budget": 0.71, + "threshold": -0.9243364334106445, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.49833333333333335, + "rescued": 3, + "harmed": 592, + "missed_rescue": 0, + "avoided_harm": 265, + "selected_both_correct": 247, + "selected_both_wrong": 10 + }, + { + "budget": 0.72, + "threshold": -0.9252541065216064, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.49083333333333334, + "rescued": 3, + "harmed": 601, + "missed_rescue": 0, + "avoided_harm": 256, + "selected_both_correct": 250, + "selected_both_wrong": 10 + }, + { + "budget": 0.73, + "threshold": -0.925781786441803, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.485, + "rescued": 3, + "harmed": 608, + "missed_rescue": 0, + "avoided_harm": 249, + "selected_both_correct": 255, + "selected_both_wrong": 10 + }, + { + "budget": 0.74, + "threshold": -0.9274963140487671, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.47833333333333333, + "rescued": 3, + "harmed": 616, + "missed_rescue": 0, + "avoided_harm": 241, + "selected_both_correct": 259, + "selected_both_wrong": 10 + }, + { + "budget": 0.75, + "threshold": -0.9286423325538635, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.4725, + "rescued": 3, + "harmed": 623, + "missed_rescue": 0, + "avoided_harm": 234, + "selected_both_correct": 264, + "selected_both_wrong": 10 + }, + { + "budget": 0.76, + "threshold": -0.9301291704177856, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.4666666666666667, + "rescued": 3, + "harmed": 630, + "missed_rescue": 0, + "avoided_harm": 227, + "selected_both_correct": 269, + "selected_both_wrong": 10 + }, + { + "budget": 0.77, + "threshold": -0.9318777918815613, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.4583333333333333, + "rescued": 3, + "harmed": 640, + "missed_rescue": 0, + "avoided_harm": 217, + "selected_both_correct": 271, + "selected_both_wrong": 10 + }, + { + "budget": 0.78, + "threshold": -0.9332055449485779, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.45, + "rescued": 3, + "harmed": 650, + "missed_rescue": 0, + "avoided_harm": 207, + "selected_both_correct": 273, + "selected_both_wrong": 10 + }, + { + "budget": 0.79, + "threshold": -0.9344666600227356, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.4425, + "rescued": 3, + "harmed": 659, + "missed_rescue": 0, + "avoided_harm": 198, + "selected_both_correct": 276, + "selected_both_wrong": 10 + }, + { + "budget": 0.8, + "threshold": -0.9362465143203735, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.43583333333333335, + "rescued": 3, + "harmed": 667, + "missed_rescue": 0, + "avoided_harm": 190, + "selected_both_correct": 280, + "selected_both_wrong": 10 + }, + { + "budget": 0.81, + "threshold": -0.937842071056366, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.4275, + "rescued": 3, + "harmed": 677, + "missed_rescue": 0, + "avoided_harm": 180, + "selected_both_correct": 282, + "selected_both_wrong": 10 + }, + { + "budget": 0.82, + "threshold": -0.9384457468986511, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.4216666666666667, + "rescued": 3, + "harmed": 684, + "missed_rescue": 0, + "avoided_harm": 173, + "selected_both_correct": 286, + "selected_both_wrong": 10 + }, + { + "budget": 0.83, + "threshold": -0.9396296143531799, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.41333333333333333, + "rescued": 3, + "harmed": 694, + "missed_rescue": 0, + "avoided_harm": 163, + "selected_both_correct": 289, + "selected_both_wrong": 10 + }, + { + "budget": 0.84, + "threshold": -0.9407488703727722, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.4058333333333333, + "rescued": 3, + "harmed": 703, + "missed_rescue": 0, + "avoided_harm": 154, + "selected_both_correct": 292, + "selected_both_wrong": 10 + }, + { + "budget": 0.85, + "threshold": -0.942738950252533, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.4, + "rescued": 3, + "harmed": 710, + "missed_rescue": 0, + "avoided_harm": 147, + "selected_both_correct": 297, + "selected_both_wrong": 10 + }, + { + "budget": 0.86, + "threshold": -0.9438527226448059, + "operator": "gt", + "actual_fraction": 0.8591666666666666, + "metric": 0.39416666666666667, + "rescued": 3, + "harmed": 717, + "missed_rescue": 0, + "avoided_harm": 140, + "selected_both_correct": 301, + "selected_both_wrong": 10 + }, + { + "budget": 0.87, + "threshold": -0.9450756907463074, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.38916666666666666, + "rescued": 3, + "harmed": 723, + "missed_rescue": 0, + "avoided_harm": 134, + "selected_both_correct": 308, + "selected_both_wrong": 10 + }, + { + "budget": 0.88, + "threshold": -0.9481598138809204, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.38083333333333336, + "rescued": 3, + "harmed": 733, + "missed_rescue": 0, + "avoided_harm": 124, + "selected_both_correct": 310, + "selected_both_wrong": 10 + }, + { + "budget": 0.89, + "threshold": -0.9500836730003357, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.37333333333333335, + "rescued": 3, + "harmed": 742, + "missed_rescue": 0, + "avoided_harm": 115, + "selected_both_correct": 313, + "selected_both_wrong": 10 + }, + { + "budget": 0.9, + "threshold": -0.9519118666648865, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.365, + "rescued": 3, + "harmed": 752, + "missed_rescue": 0, + "avoided_harm": 105, + "selected_both_correct": 315, + "selected_both_wrong": 10 + }, + { + "budget": 0.91, + "threshold": -0.9543898701667786, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.3566666666666667, + "rescued": 3, + "harmed": 762, + "missed_rescue": 0, + "avoided_harm": 95, + "selected_both_correct": 317, + "selected_both_wrong": 10 + }, + { + "budget": 0.92, + "threshold": -0.9570822715759277, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.34833333333333333, + "rescued": 3, + "harmed": 772, + "missed_rescue": 0, + "avoided_harm": 85, + "selected_both_correct": 319, + "selected_both_wrong": 10 + }, + { + "budget": 0.93, + "threshold": -0.9593446254730225, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.3383333333333333, + "rescued": 3, + "harmed": 784, + "missed_rescue": 0, + "avoided_harm": 73, + "selected_both_correct": 319, + "selected_both_wrong": 10 + }, + { + "budget": 0.94, + "threshold": -0.9610995054244995, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.33, + "rescued": 3, + "harmed": 794, + "missed_rescue": 0, + "avoided_harm": 63, + "selected_both_correct": 321, + "selected_both_wrong": 10 + }, + { + "budget": 0.95, + "threshold": -0.9635111093521118, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.32, + "rescued": 3, + "harmed": 806, + "missed_rescue": 0, + "avoided_harm": 51, + "selected_both_correct": 321, + "selected_both_wrong": 10 + }, + { + "budget": 0.96, + "threshold": -0.9648061990737915, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.31333333333333335, + "rescued": 3, + "harmed": 814, + "missed_rescue": 0, + "avoided_harm": 43, + "selected_both_correct": 325, + "selected_both_wrong": 10 + }, + { + "budget": 0.97, + "threshold": -0.9674390554428101, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.30416666666666664, + "rescued": 3, + "harmed": 825, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 326, + "selected_both_wrong": 10 + }, + { + "budget": 0.98, + "threshold": -0.9724470973014832, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.29583333333333334, + "rescued": 3, + "harmed": 835, + "missed_rescue": 0, + "avoided_harm": 22, + "selected_both_correct": 328, + "selected_both_wrong": 10 + }, + { + "budget": 0.99, + "threshold": -0.9774226546287537, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.2866666666666667, + "rescued": 3, + "harmed": 846, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 329, + "selected_both_wrong": 10 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.2775, + "rescued": 3, + "harmed": 857, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 330, + "selected_both_wrong": 10 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9016223679668622, + "rescued": 0, + "harmed": 0, + "missed_rescue": 106, + "avoided_harm": 3791, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.08698653779772178, + "metric": 0.8769416637901277, + "rescued": 50, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 3598, + "selected_both_correct": 42, + "selected_both_wrong": 219 + }, + { + "budget": 0.02, + "threshold": -0.6014261841773987, + "operator": "ge", + "actual_fraction": 0.13910942354159475, + "metric": 0.8493268898860891, + "rescued": 66, + "harmed": 369, + "missed_rescue": 40, + "avoided_harm": 3422, + "selected_both_correct": 85, + "selected_both_wrong": 286 + }, + { + "budget": 0.03, + "threshold": -0.6401472687721252, + "operator": "ge", + "actual_fraction": 0.15947531929582326, + "metric": 0.8381083879875734, + "rescued": 74, + "harmed": 442, + "missed_rescue": 32, + "avoided_harm": 3349, + "selected_both_correct": 104, + "selected_both_wrong": 304 + }, + { + "budget": 0.04, + "threshold": -0.6884650588035583, + "operator": "ge", + "actual_fraction": 0.1886434242319641, + "metric": 0.8208491542975492, + "rescued": 81, + "harmed": 549, + "missed_rescue": 25, + "avoided_harm": 3242, + "selected_both_correct": 139, + "selected_both_wrong": 324 + }, + { + "budget": 0.05, + "threshold": -0.7208121418952942, + "operator": "ge", + "actual_fraction": 0.2143596824301001, + "metric": 0.8041076976182258, + "rescued": 84, + "harmed": 649, + "missed_rescue": 22, + "avoided_harm": 3142, + "selected_both_correct": 165, + "selected_both_wrong": 344 + }, + { + "budget": 0.06, + "threshold": -0.7578181624412537, + "operator": "ge", + "actual_fraction": 0.24525371073524335, + "metric": 0.7825336555056955, + "rescued": 88, + "harmed": 778, + "missed_rescue": 18, + "avoided_harm": 3013, + "selected_both_correct": 198, + "selected_both_wrong": 357 + }, + { + "budget": 0.07, + "threshold": -0.7790470719337463, + "operator": "ge", + "actual_fraction": 0.26734552985847426, + "metric": 0.767172937521574, + "rescued": 90, + "harmed": 869, + "missed_rescue": 16, + "avoided_harm": 2922, + "selected_both_correct": 221, + "selected_both_wrong": 369 + }, + { + "budget": 0.08, + "threshold": -0.7927228808403015, + "operator": "ge", + "actual_fraction": 0.2827062478425958, + "metric": 0.75716258198136, + "rescued": 92, + "harmed": 929, + "missed_rescue": 14, + "avoided_harm": 2862, + "selected_both_correct": 244, + "selected_both_wrong": 373 + }, + { + "budget": 0.09, + "threshold": -0.8049252033233643, + "operator": "ge", + "actual_fraction": 0.2999654815326199, + "metric": 0.7449085260614429, + "rescued": 93, + "harmed": 1001, + "missed_rescue": 13, + "avoided_harm": 2790, + "selected_both_correct": 264, + "selected_both_wrong": 380 + }, + { + "budget": 0.1, + "threshold": -0.80850750207901, + "operator": "ge", + "actual_fraction": 0.30531584397652745, + "metric": 0.7416292716603383, + "rescued": 93, + "harmed": 1020, + "missed_rescue": 13, + "avoided_harm": 2771, + "selected_both_correct": 274, + "selected_both_wrong": 382 + }, + { + "budget": 0.11, + "threshold": -0.819871723651886, + "operator": "ge", + "actual_fraction": 0.3258543320676562, + "metric": 0.7279944770452192, + "rescued": 96, + "harmed": 1102, + "missed_rescue": 10, + "avoided_harm": 2689, + "selected_both_correct": 302, + "selected_both_wrong": 388 + }, + { + "budget": 0.12, + "threshold": -0.8310170769691467, + "operator": "ge", + "actual_fraction": 0.35053503624439075, + "metric": 0.710735243355195, + "rescued": 99, + "harmed": 1205, + "missed_rescue": 7, + "avoided_harm": 2586, + "selected_both_correct": 332, + "selected_both_wrong": 395 + }, + { + "budget": 0.13, + "threshold": -0.8376906514167786, + "operator": "ge", + "actual_fraction": 0.36796686227131514, + "metric": 0.6976182257507767, + "rescued": 99, + "harmed": 1281, + "missed_rescue": 7, + "avoided_harm": 2510, + "selected_both_correct": 353, + "selected_both_wrong": 399 + }, + { + "budget": 0.14, + "threshold": -0.8419997692108154, + "operator": "ge", + "actual_fraction": 0.3798757335174318, + "metric": 0.6896789782533655, + "rescued": 99, + "harmed": 1327, + "missed_rescue": 7, + "avoided_harm": 2464, + "selected_both_correct": 374, + "selected_both_wrong": 401 + }, + { + "budget": 0.15, + "threshold": -0.8447275161743164, + "operator": "ge", + "actual_fraction": 0.3867794269934415, + "metric": 0.6855367621677597, + "rescued": 99, + "harmed": 1351, + "missed_rescue": 7, + "avoided_harm": 2440, + "selected_both_correct": 390, + "selected_both_wrong": 401 + }, + { + "budget": 0.16, + "threshold": -0.8506776690483093, + "operator": "ge", + "actual_fraction": 0.4050742147048671, + "metric": 0.6717293752157404, + "rescued": 100, + "harmed": 1432, + "missed_rescue": 6, + "avoided_harm": 2359, + "selected_both_correct": 411, + "selected_both_wrong": 404 + }, + { + "budget": 0.17, + "threshold": -0.8545639514923096, + "operator": "ge", + "actual_fraction": 0.4175008629616845, + "metric": 0.6632723507076286, + "rescued": 100, + "harmed": 1481, + "missed_rescue": 6, + "avoided_harm": 2310, + "selected_both_correct": 430, + "selected_both_wrong": 408 + }, + { + "budget": 0.18, + "threshold": -0.8578683137893677, + "operator": "ge", + "actual_fraction": 0.4290645495340007, + "metric": 0.6544701415257163, + "rescued": 101, + "harmed": 1533, + "missed_rescue": 5, + "avoided_harm": 2258, + "selected_both_correct": 443, + "selected_both_wrong": 409 + }, + { + "budget": 0.19, + "threshold": -0.86090087890625, + "operator": "ge", + "actual_fraction": 0.43993786675871593, + "metric": 0.6473938557128064, + "rescued": 101, + "harmed": 1574, + "missed_rescue": 5, + "avoided_harm": 2217, + "selected_both_correct": 462, + "selected_both_wrong": 412 + }, + { + "budget": 0.2, + "threshold": -0.8640955090522766, + "operator": "ge", + "actual_fraction": 0.4520193303417328, + "metric": 0.6389368312046945, + "rescued": 101, + "harmed": 1623, + "missed_rescue": 5, + "avoided_harm": 2168, + "selected_both_correct": 482, + "selected_both_wrong": 413 + }, + { + "budget": 0.21, + "threshold": -0.8664237260818481, + "operator": "ge", + "actual_fraction": 0.46047635484984467, + "metric": 0.6332412840869865, + "rescued": 101, + "harmed": 1656, + "missed_rescue": 5, + "avoided_harm": 2135, + "selected_both_correct": 496, + "selected_both_wrong": 415 + }, + { + "budget": 0.22, + "threshold": -0.8686786890029907, + "operator": "ge", + "actual_fraction": 0.46945115636865725, + "metric": 0.6272005522954781, + "rescued": 101, + "harmed": 1691, + "missed_rescue": 5, + "avoided_harm": 2100, + "selected_both_correct": 513, + "selected_both_wrong": 415 + }, + { + "budget": 0.23, + "threshold": -0.8709638118743896, + "operator": "ge", + "actual_fraction": 0.48066965826717295, + "metric": 0.6197790818087677, + "rescued": 101, + "harmed": 1734, + "missed_rescue": 5, + "avoided_harm": 2057, + "selected_both_correct": 534, + "selected_both_wrong": 416 + }, + { + "budget": 0.24, + "threshold": -0.8724074959754944, + "operator": "ge", + "actual_fraction": 0.48912668277528476, + "metric": 0.61425612702796, + "rescued": 101, + "harmed": 1766, + "missed_rescue": 5, + "avoided_harm": 2025, + "selected_both_correct": 550, + "selected_both_wrong": 417 + }, + { + "budget": 0.25, + "threshold": -0.8736594319343567, + "operator": "ge", + "actual_fraction": 0.49413186054539177, + "metric": 0.6113220573006558, + "rescued": 101, + "harmed": 1783, + "missed_rescue": 5, + "avoided_harm": 2008, + "selected_both_correct": 561, + "selected_both_wrong": 418 + }, + { + "budget": 0.26, + "threshold": -0.8752554059028625, + "operator": "ge", + "actual_fraction": 0.5005177770107008, + "metric": 0.6071798412150501, + "rescued": 101, + "harmed": 1807, + "missed_rescue": 5, + "avoided_harm": 1984, + "selected_both_correct": 574, + "selected_both_wrong": 418 + }, + { + "budget": 0.27, + "threshold": -0.8766024112701416, + "operator": "ge", + "actual_fraction": 0.5081118398343114, + "metric": 0.6020020711080428, + "rescued": 101, + "harmed": 1837, + "missed_rescue": 5, + "avoided_harm": 1954, + "selected_both_correct": 587, + "selected_both_wrong": 419 + }, + { + "budget": 0.28, + "threshold": -0.878311276435852, + "operator": "ge", + "actual_fraction": 0.5169140490162237, + "metric": 0.596479116327235, + "rescued": 102, + "harmed": 1870, + "missed_rescue": 4, + "avoided_harm": 1921, + "selected_both_correct": 603, + "selected_both_wrong": 420 + }, + { + "budget": 0.29, + "threshold": -0.880103588104248, + "operator": "ge", + "actual_fraction": 0.5277873662409389, + "metric": 0.5894028305143252, + "rescued": 102, + "harmed": 1911, + "missed_rescue": 4, + "avoided_harm": 1880, + "selected_both_correct": 624, + "selected_both_wrong": 421 + }, + { + "budget": 0.3, + "threshold": -0.8824952244758606, + "operator": "ge", + "actual_fraction": 0.5419399378667588, + "metric": 0.5788746979634104, + "rescued": 102, + "harmed": 1972, + "missed_rescue": 4, + "avoided_harm": 1819, + "selected_both_correct": 645, + "selected_both_wrong": 421 + }, + { + "budget": 0.31, + "threshold": -0.8831692337989807, + "operator": "ge", + "actual_fraction": 0.5453917846047636, + "metric": 0.5766309975837073, + "rescued": 102, + "harmed": 1985, + "missed_rescue": 4, + "avoided_harm": 1806, + "selected_both_correct": 651, + "selected_both_wrong": 422 + }, + { + "budget": 0.32, + "threshold": -0.8845130205154419, + "operator": "ge", + "actual_fraction": 0.5566102865032793, + "metric": 0.5704176734552986, + "rescued": 102, + "harmed": 2021, + "missed_rescue": 4, + "avoided_harm": 1770, + "selected_both_correct": 677, + "selected_both_wrong": 425 + }, + { + "budget": 0.33, + "threshold": -0.8855781555175781, + "operator": "ge", + "actual_fraction": 0.5633413876423887, + "metric": 0.5661028650327925, + "rescued": 102, + "harmed": 2046, + "missed_rescue": 4, + "avoided_harm": 1745, + "selected_both_correct": 691, + "selected_both_wrong": 425 + }, + { + "budget": 0.34, + "threshold": -0.8865919709205627, + "operator": "ge", + "actual_fraction": 0.5716258198136003, + "metric": 0.5607525025888851, + "rescued": 102, + "harmed": 2077, + "missed_rescue": 4, + "avoided_harm": 1714, + "selected_both_correct": 707, + "selected_both_wrong": 426 + }, + { + "budget": 0.35, + "threshold": -0.8877584338188171, + "operator": "ge", + "actual_fraction": 0.5819813600276148, + "metric": 0.5545391784604764, + "rescued": 103, + "harmed": 2114, + "missed_rescue": 3, + "avoided_harm": 1677, + "selected_both_correct": 728, + "selected_both_wrong": 427 + }, + { + "budget": 0.36, + "threshold": -0.8890330195426941, + "operator": "ge", + "actual_fraction": 0.5902657921988264, + "metric": 0.5493614083534691, + "rescued": 104, + "harmed": 2145, + "missed_rescue": 2, + "avoided_harm": 1646, + "selected_both_correct": 743, + "selected_both_wrong": 428 + }, + { + "budget": 0.37, + "threshold": -0.890850305557251, + "operator": "ge", + "actual_fraction": 0.6047635484984467, + "metric": 0.5383154987918537, + "rescued": 104, + "harmed": 2209, + "missed_rescue": 2, + "avoided_harm": 1582, + "selected_both_correct": 763, + "selected_both_wrong": 428 + }, + { + "budget": 0.38, + "threshold": -0.8916605710983276, + "operator": "ge", + "actual_fraction": 0.6113220573006558, + "metric": 0.5343458750431481, + "rescued": 104, + "harmed": 2232, + "missed_rescue": 2, + "avoided_harm": 1559, + "selected_both_correct": 778, + "selected_both_wrong": 428 + }, + { + "budget": 0.39, + "threshold": -0.8928589224815369, + "operator": "ge", + "actual_fraction": 0.6206420434932689, + "metric": 0.5279599585778392, + "rescued": 104, + "harmed": 2269, + "missed_rescue": 2, + "avoided_harm": 1522, + "selected_both_correct": 794, + "selected_both_wrong": 429 + }, + { + "budget": 0.4, + "threshold": -0.8940943479537964, + "operator": "ge", + "actual_fraction": 0.6289264756644805, + "metric": 0.5222644114601311, + "rescued": 104, + "harmed": 2302, + "missed_rescue": 2, + "avoided_harm": 1489, + "selected_both_correct": 809, + "selected_both_wrong": 429 + }, + { + "budget": 0.41, + "threshold": -0.8949463963508606, + "operator": "ge", + "actual_fraction": 0.6370383154987919, + "metric": 0.5169140490162237, + "rescued": 104, + "harmed": 2333, + "missed_rescue": 2, + "avoided_harm": 1458, + "selected_both_correct": 824, + "selected_both_wrong": 430 + }, + { + "budget": 0.42, + "threshold": -0.8960229158401489, + "operator": "ge", + "actual_fraction": 0.6435968243010011, + "metric": 0.5127718329306179, + "rescued": 104, + "harmed": 2357, + "missed_rescue": 2, + "avoided_harm": 1434, + "selected_both_correct": 836, + "selected_both_wrong": 432 + }, + { + "budget": 0.43, + "threshold": -0.8966536521911621, + "operator": "ge", + "actual_fraction": 0.6491197790818087, + "metric": 0.5082844321712116, + "rescued": 104, + "harmed": 2383, + "missed_rescue": 2, + "avoided_harm": 1408, + "selected_both_correct": 842, + "selected_both_wrong": 432 + }, + { + "budget": 0.44, + "threshold": -0.8975363373756409, + "operator": "ge", + "actual_fraction": 0.6558508802209182, + "metric": 0.5044874007594062, + "rescued": 104, + "harmed": 2405, + "missed_rescue": 2, + "avoided_harm": 1386, + "selected_both_correct": 859, + "selected_both_wrong": 432 + }, + { + "budget": 0.45, + "threshold": -0.8988122940063477, + "operator": "ge", + "actual_fraction": 0.6641353123921298, + "metric": 0.49896444597859857, + "rescued": 104, + "harmed": 2437, + "missed_rescue": 2, + "avoided_harm": 1354, + "selected_both_correct": 875, + "selected_both_wrong": 432 + }, + { + "budget": 0.46, + "threshold": -0.9000959396362305, + "operator": "ge", + "actual_fraction": 0.6739730755954436, + "metric": 0.4917155678287884, + "rescued": 104, + "harmed": 2479, + "missed_rescue": 2, + "avoided_harm": 1312, + "selected_both_correct": 890, + "selected_both_wrong": 432 + }, + { + "budget": 0.47, + "threshold": -0.9013051986694336, + "operator": "ge", + "actual_fraction": 0.6848463928201588, + "metric": 0.48412150500517775, + "rescued": 104, + "harmed": 2523, + "missed_rescue": 2, + "avoided_harm": 1268, + "selected_both_correct": 907, + "selected_both_wrong": 434 + }, + { + "budget": 0.48, + "threshold": -0.9029324054718018, + "operator": "ge", + "actual_fraction": 0.696410079392475, + "metric": 0.4775629962029686, + "rescued": 104, + "harmed": 2561, + "missed_rescue": 2, + "avoided_harm": 1230, + "selected_both_correct": 935, + "selected_both_wrong": 435 + }, + { + "budget": 0.49, + "threshold": -0.9036871790885925, + "operator": "ge", + "actual_fraction": 0.7026234035208837, + "metric": 0.47411114946496374, + "rescued": 104, + "harmed": 2581, + "missed_rescue": 2, + "avoided_harm": 1210, + "selected_both_correct": 949, + "selected_both_wrong": 437 + }, + { + "budget": 0.5, + "threshold": -0.9044621586799622, + "operator": "ge", + "actual_fraction": 0.7088367276492924, + "metric": 0.46996893337935797, + "rescued": 104, + "harmed": 2605, + "missed_rescue": 2, + "avoided_harm": 1186, + "selected_both_correct": 961, + "selected_both_wrong": 437 + }, + { + "budget": 0.51, + "threshold": -0.9054575562477112, + "operator": "ge", + "actual_fraction": 0.717121159820504, + "metric": 0.46496375560925096, + "rescued": 104, + "harmed": 2634, + "missed_rescue": 2, + "avoided_harm": 1157, + "selected_both_correct": 980, + "selected_both_wrong": 437 + }, + { + "budget": 0.52, + "threshold": -0.906682550907135, + "operator": "ge", + "actual_fraction": 0.7262685536762168, + "metric": 0.45875043148084227, + "rescued": 104, + "harmed": 2670, + "missed_rescue": 2, + "avoided_harm": 1121, + "selected_both_correct": 997, + "selected_both_wrong": 437 + }, + { + "budget": 0.53, + "threshold": -0.9072114825248718, + "operator": "ge", + "actual_fraction": 0.7300655850880221, + "metric": 0.4559889540904384, + "rescued": 104, + "harmed": 2686, + "missed_rescue": 2, + "avoided_harm": 1105, + "selected_both_correct": 1002, + "selected_both_wrong": 438 + }, + { + "budget": 0.54, + "threshold": -0.9083563089370728, + "operator": "ge", + "actual_fraction": 0.7381774249223334, + "metric": 0.4518467380048326, + "rescued": 105, + "harmed": 2711, + "missed_rescue": 1, + "avoided_harm": 1080, + "selected_both_correct": 1023, + "selected_both_wrong": 438 + }, + { + "budget": 0.55, + "threshold": -0.9095192551612854, + "operator": "ge", + "actual_fraction": 0.7487055574732482, + "metric": 0.4458060062133241, + "rescued": 105, + "harmed": 2746, + "missed_rescue": 1, + "avoided_harm": 1045, + "selected_both_correct": 1047, + "selected_both_wrong": 440 + }, + { + "budget": 0.56, + "threshold": -0.9106979370117188, + "operator": "ge", + "actual_fraction": 0.759233690024163, + "metric": 0.43924749741111496, + "rescued": 105, + "harmed": 2784, + "missed_rescue": 1, + "avoided_harm": 1007, + "selected_both_correct": 1069, + "selected_both_wrong": 441 + }, + { + "budget": 0.57, + "threshold": -0.9115362167358398, + "operator": "ge", + "actual_fraction": 0.767863306869175, + "metric": 0.4340697273041077, + "rescued": 105, + "harmed": 2814, + "missed_rescue": 1, + "avoided_harm": 977, + "selected_both_correct": 1089, + "selected_both_wrong": 441 + }, + { + "budget": 0.58, + "threshold": -0.9124338030815125, + "operator": "ge", + "actual_fraction": 0.7751121850189852, + "metric": 0.4301001035554021, + "rescued": 105, + "harmed": 2837, + "missed_rescue": 1, + "avoided_harm": 954, + "selected_both_correct": 1108, + "selected_both_wrong": 441 + }, + { + "budget": 0.59, + "threshold": -0.9134801626205444, + "operator": "ge", + "actual_fraction": 0.7830514325163963, + "metric": 0.42440455643769415, + "rescued": 105, + "harmed": 2870, + "missed_rescue": 1, + "avoided_harm": 921, + "selected_both_correct": 1121, + "selected_both_wrong": 441 + }, + { + "budget": 0.6, + "threshold": -0.9143747091293335, + "operator": "ge", + "actual_fraction": 0.7904729030031067, + "metric": 0.41939937866758714, + "rescued": 105, + "harmed": 2899, + "missed_rescue": 1, + "avoided_harm": 892, + "selected_both_correct": 1134, + "selected_both_wrong": 442 + }, + { + "budget": 0.61, + "threshold": -0.915804386138916, + "operator": "ge", + "actual_fraction": 0.8010010355540214, + "metric": 0.4133586468760787, + "rescued": 105, + "harmed": 2934, + "missed_rescue": 1, + "avoided_harm": 857, + "selected_both_correct": 1158, + "selected_both_wrong": 444 + }, + { + "budget": 0.62, + "threshold": -0.9165146350860596, + "operator": "ge", + "actual_fraction": 0.8063513979979289, + "metric": 0.41025198481187436, + "rescued": 105, + "harmed": 2952, + "missed_rescue": 1, + "avoided_harm": 839, + "selected_both_correct": 1171, + "selected_both_wrong": 444 + }, + { + "budget": 0.63, + "threshold": -0.9172822833061218, + "operator": "ge", + "actual_fraction": 0.8110113910942354, + "metric": 0.4062823610631688, + "rescued": 105, + "harmed": 2975, + "missed_rescue": 1, + "avoided_harm": 816, + "selected_both_correct": 1175, + "selected_both_wrong": 444 + }, + { + "budget": 0.64, + "threshold": -0.9181071519851685, + "operator": "ge", + "actual_fraction": 0.8172247152226441, + "metric": 0.40248532965136347, + "rescued": 105, + "harmed": 2997, + "missed_rescue": 1, + "avoided_harm": 794, + "selected_both_correct": 1188, + "selected_both_wrong": 445 + }, + { + "budget": 0.65, + "threshold": -0.9192029237747192, + "operator": "ge", + "actual_fraction": 0.824301001035554, + "metric": 0.3979979288919572, + "rescued": 105, + "harmed": 3023, + "missed_rescue": 1, + "avoided_harm": 768, + "selected_both_correct": 1202, + "selected_both_wrong": 446 + }, + { + "budget": 0.66, + "threshold": -0.9200543761253357, + "operator": "ge", + "actual_fraction": 0.829306178805661, + "metric": 0.39420089748015186, + "rescued": 105, + "harmed": 3045, + "missed_rescue": 1, + "avoided_harm": 746, + "selected_both_correct": 1209, + "selected_both_wrong": 446 + }, + { + "budget": 0.67, + "threshold": -0.9208071231842041, + "operator": "ge", + "actual_fraction": 0.8348291335864687, + "metric": 0.3898860890576458, + "rescued": 105, + "harmed": 3070, + "missed_rescue": 1, + "avoided_harm": 721, + "selected_both_correct": 1216, + "selected_both_wrong": 446 + }, + { + "budget": 0.68, + "threshold": -0.9215553998947144, + "operator": "ge", + "actual_fraction": 0.840006903693476, + "metric": 0.3867794269934415, + "rescued": 105, + "harmed": 3088, + "missed_rescue": 1, + "avoided_harm": 703, + "selected_both_correct": 1228, + "selected_both_wrong": 446 + }, + { + "budget": 0.69, + "threshold": -0.9220835566520691, + "operator": "ge", + "actual_fraction": 0.8443217121159821, + "metric": 0.38401794960303764, + "rescued": 105, + "harmed": 3104, + "missed_rescue": 1, + "avoided_harm": 687, + "selected_both_correct": 1236, + "selected_both_wrong": 447 + }, + { + "budget": 0.7, + "threshold": -0.9232237935066223, + "operator": "ge", + "actual_fraction": 0.852088367276493, + "metric": 0.37849499482222987, + "rescued": 105, + "harmed": 3136, + "missed_rescue": 1, + "avoided_harm": 655, + "selected_both_correct": 1247, + "selected_both_wrong": 449 + }, + { + "budget": 0.71, + "threshold": -0.9243364334106445, + "operator": "ge", + "actual_fraction": 0.8598550224370038, + "metric": 0.37279944770452195, + "rescued": 105, + "harmed": 3169, + "missed_rescue": 1, + "avoided_harm": 622, + "selected_both_correct": 1257, + "selected_both_wrong": 451 + }, + { + "budget": 0.72, + "threshold": -0.9252541065216064, + "operator": "ge", + "actual_fraction": 0.8641698308595098, + "metric": 0.37003797031411806, + "rescued": 105, + "harmed": 3185, + "missed_rescue": 1, + "avoided_harm": 606, + "selected_both_correct": 1266, + "selected_both_wrong": 451 + }, + { + "budget": 0.73, + "threshold": -0.925781786441803, + "operator": "ge", + "actual_fraction": 0.8681394546082154, + "metric": 0.3674490852606144, + "rescued": 105, + "harmed": 3200, + "missed_rescue": 1, + "avoided_harm": 591, + "selected_both_correct": 1274, + "selected_both_wrong": 451 + }, + { + "budget": 0.74, + "threshold": -0.9274963140487671, + "operator": "ge", + "actual_fraction": 0.877114256127028, + "metric": 0.36123576113220573, + "rescued": 105, + "harmed": 3236, + "missed_rescue": 1, + "avoided_harm": 555, + "selected_both_correct": 1290, + "selected_both_wrong": 451 + }, + { + "budget": 0.75, + "threshold": -0.9286423325538635, + "operator": "ge", + "actual_fraction": 0.8828098032447359, + "metric": 0.35726613738350016, + "rescued": 105, + "harmed": 3259, + "missed_rescue": 1, + "avoided_harm": 532, + "selected_both_correct": 1299, + "selected_both_wrong": 452 + }, + { + "budget": 0.76, + "threshold": -0.9301291704177856, + "operator": "ge", + "actual_fraction": 0.8900586813945461, + "metric": 0.3520883672764929, + "rescued": 105, + "harmed": 3289, + "missed_rescue": 1, + "avoided_harm": 502, + "selected_both_correct": 1310, + "selected_both_wrong": 453 + }, + { + "budget": 0.77, + "threshold": -0.9318777918815613, + "operator": "ge", + "actual_fraction": 0.8979979288919572, + "metric": 0.34673800483258543, + "rescued": 105, + "harmed": 3320, + "missed_rescue": 1, + "avoided_harm": 471, + "selected_both_correct": 1325, + "selected_both_wrong": 453 + }, + { + "budget": 0.78, + "threshold": -0.9332055449485779, + "operator": "ge", + "actual_fraction": 0.9052468070417673, + "metric": 0.34173282706247843, + "rescued": 105, + "harmed": 3349, + "missed_rescue": 1, + "avoided_harm": 442, + "selected_both_correct": 1338, + "selected_both_wrong": 453 + }, + { + "budget": 0.79, + "threshold": -0.9344666600227356, + "operator": "ge", + "actual_fraction": 0.9100793924749742, + "metric": 0.3379357956506731, + "rescued": 105, + "harmed": 3371, + "missed_rescue": 1, + "avoided_harm": 420, + "selected_both_correct": 1344, + "selected_both_wrong": 453 + }, + { + "budget": 0.8, + "threshold": -0.9362465143203735, + "operator": "ge", + "actual_fraction": 0.917155678287884, + "metric": 0.33224024853296513, + "rescued": 105, + "harmed": 3404, + "missed_rescue": 1, + "avoided_harm": 387, + "selected_both_correct": 1351, + "selected_both_wrong": 454 + }, + { + "budget": 0.81, + "threshold": -0.937842071056366, + "operator": "ge", + "actual_fraction": 0.9237141870900932, + "metric": 0.3274076630997584, + "rescued": 105, + "harmed": 3432, + "missed_rescue": 1, + "avoided_harm": 359, + "selected_both_correct": 1361, + "selected_both_wrong": 454 + }, + { + "budget": 0.82, + "threshold": -0.9384457468986511, + "operator": "ge", + "actual_fraction": 0.9249223334483949, + "metric": 0.3267172937521574, + "rescued": 105, + "harmed": 3436, + "missed_rescue": 1, + "avoided_harm": 355, + "selected_both_correct": 1363, + "selected_both_wrong": 455 + }, + { + "budget": 0.83, + "threshold": -0.9396296143531799, + "operator": "ge", + "actual_fraction": 0.927166033828098, + "metric": 0.324991370383155, + "rescued": 105, + "harmed": 3446, + "missed_rescue": 1, + "avoided_harm": 345, + "selected_both_correct": 1365, + "selected_both_wrong": 456 + }, + { + "budget": 0.84, + "threshold": -0.9407488703727722, + "operator": "ge", + "actual_fraction": 0.9316534345875043, + "metric": 0.3211943389713497, + "rescued": 105, + "harmed": 3468, + "missed_rescue": 1, + "avoided_harm": 323, + "selected_both_correct": 1369, + "selected_both_wrong": 456 + }, + { + "budget": 0.85, + "threshold": -0.942738950252533, + "operator": "ge", + "actual_fraction": 0.9397652744218157, + "metric": 0.31429064549534, + "rescued": 105, + "harmed": 3508, + "missed_rescue": 1, + "avoided_harm": 283, + "selected_both_correct": 1376, + "selected_both_wrong": 456 + }, + { + "budget": 0.86, + "threshold": -0.9438527226448059, + "operator": "gt", + "actual_fraction": 0.9432171211598205, + "metric": 0.31239212978943737, + "rescued": 105, + "harmed": 3519, + "missed_rescue": 1, + "avoided_harm": 272, + "selected_both_correct": 1384, + "selected_both_wrong": 457 + }, + { + "budget": 0.87, + "threshold": -0.9450756907463074, + "operator": "ge", + "actual_fraction": 0.9456334138764239, + "metric": 0.3106662064204349, + "rescued": 105, + "harmed": 3529, + "missed_rescue": 1, + "avoided_harm": 262, + "selected_both_correct": 1388, + "selected_both_wrong": 457 + }, + { + "budget": 0.88, + "threshold": -0.9481598138809204, + "operator": "ge", + "actual_fraction": 0.9539178460476355, + "metric": 0.3044528822920262, + "rescued": 105, + "harmed": 3565, + "missed_rescue": 1, + "avoided_harm": 226, + "selected_both_correct": 1398, + "selected_both_wrong": 459 + }, + { + "budget": 0.89, + "threshold": -0.9500836730003357, + "operator": "ge", + "actual_fraction": 0.9606489471867449, + "metric": 0.29858474283741804, + "rescued": 105, + "harmed": 3599, + "missed_rescue": 1, + "avoided_harm": 192, + "selected_both_correct": 1402, + "selected_both_wrong": 460 + }, + { + "budget": 0.9, + "threshold": -0.9519118666648865, + "operator": "ge", + "actual_fraction": 0.9647911632723507, + "metric": 0.2956506731101139, + "rescued": 105, + "harmed": 3616, + "missed_rescue": 1, + "avoided_harm": 175, + "selected_both_correct": 1408, + "selected_both_wrong": 461 + }, + { + "budget": 0.91, + "threshold": -0.9543898701667786, + "operator": "ge", + "actual_fraction": 0.9697963410424577, + "metric": 0.2911632723507076, + "rescued": 105, + "harmed": 3642, + "missed_rescue": 1, + "avoided_harm": 149, + "selected_both_correct": 1410, + "selected_both_wrong": 462 + }, + { + "budget": 0.92, + "threshold": -0.9570822715759277, + "operator": "ge", + "actual_fraction": 0.9760096651708664, + "metric": 0.28633068691750085, + "rescued": 106, + "harmed": 3671, + "missed_rescue": 0, + "avoided_harm": 120, + "selected_both_correct": 1416, + "selected_both_wrong": 462 + }, + { + "budget": 0.93, + "threshold": -0.9593446254730225, + "operator": "ge", + "actual_fraction": 0.9801518812564722, + "metric": 0.2827062478425958, + "rescued": 106, + "harmed": 3692, + "missed_rescue": 0, + "avoided_harm": 99, + "selected_both_correct": 1419, + "selected_both_wrong": 462 + }, + { + "budget": 0.94, + "threshold": -0.9610995054244995, + "operator": "ge", + "actual_fraction": 0.9817052122885744, + "metric": 0.28132550914739385, + "rescued": 106, + "harmed": 3700, + "missed_rescue": 0, + "avoided_harm": 91, + "selected_both_correct": 1420, + "selected_both_wrong": 462 + }, + { + "budget": 0.95, + "threshold": -0.9635111093521118, + "operator": "ge", + "actual_fraction": 0.9853296513634795, + "metric": 0.2787366240938902, + "rescued": 106, + "harmed": 3715, + "missed_rescue": 0, + "avoided_harm": 76, + "selected_both_correct": 1426, + "selected_both_wrong": 462 + }, + { + "budget": 0.96, + "threshold": -0.9648061990737915, + "operator": "ge", + "actual_fraction": 0.9875733517431826, + "metric": 0.27666551605108736, + "rescued": 106, + "harmed": 3727, + "missed_rescue": 0, + "avoided_harm": 64, + "selected_both_correct": 1427, + "selected_both_wrong": 462 + }, + { + "budget": 0.97, + "threshold": -0.9674390554428101, + "operator": "ge", + "actual_fraction": 0.9906800138073869, + "metric": 0.27390403866068347, + "rescued": 106, + "harmed": 3743, + "missed_rescue": 0, + "avoided_harm": 48, + "selected_both_correct": 1429, + "selected_both_wrong": 462 + }, + { + "budget": 0.98, + "threshold": -0.9724470973014832, + "operator": "ge", + "actual_fraction": 0.9937866758715913, + "metric": 0.2711425612702796, + "rescued": 106, + "harmed": 3759, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 1430, + "selected_both_wrong": 463 + }, + { + "budget": 0.99, + "threshold": -0.9774226546287537, + "operator": "ge", + "actual_fraction": 0.9965481532619952, + "metric": 0.26889886089057646, + "rescued": 106, + "harmed": 3772, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 1432, + "selected_both_wrong": 464 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.26561960648947186, + "rescued": 106, + "harmed": 3791, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1433, + "selected_both_wrong": 464 + } + ] + }, + "learned": { + "validation_area_0_25": 0.9787166666666666, + "validation_point": { + "metric": 0.9893333333333333, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.01, + "actual_fraction_std": 0.0, + "per_seed": [ + { + "seed": 2026, + "budget": 0.01, + "threshold": -0.0050849150866270065, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescue": 0, + "harm": 0, + "missed_rescue": 3 + }, + { + "seed": 2027, + "budget": 0.01, + "threshold": -0.005811857525259256, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescue": 0, + "harm": 0, + "missed_rescue": 3 + }, + { + "seed": 2028, + "budget": 0.01, + "threshold": -0.0011190498480573297, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescue": 0, + "harm": 0, + "missed_rescue": 3 + }, + { + "seed": 2029, + "budget": 0.01, + "threshold": -0.004320982377976179, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescue": 0, + "harm": 0, + "missed_rescue": 3 + }, + { + "seed": 2030, + "budget": 0.01, + "threshold": -0.002133682370185852, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.99, + "rescue": 1, + "harm": 0, + "missed_rescue": 2 + } + ], + "rescued": 0.2, + "harmed": 0.0, + "missed_rescue": 2.8, + "avoided_harm": 857.0 + }, + "test_point": { + "metric": 0.901484294097342, + "metric_std": 0.00014440110917050722, + "actual_fraction": 0.013289609941318605, + "actual_fraction_std": 0.00372375716437546, + "per_seed": [ + { + "seed": 2026, + "budget": 0.01, + "threshold": -0.0050849150866270065, + "operator": "ge", + "actual_fraction": 0.014152571625819814, + "metric": 0.9016223679668622, + "rescue": 0, + "harm": 0, + "missed_rescue": 106 + }, + { + "seed": 2027, + "budget": 0.01, + "threshold": -0.005811857525259256, + "operator": "ge", + "actual_fraction": 0.008802209181912324, + "metric": 0.9016223679668622, + "rescue": 0, + "harm": 0, + "missed_rescue": 106 + }, + { + "seed": 2028, + "budget": 0.01, + "threshold": -0.0011190498480573297, + "operator": "ge", + "actual_fraction": 0.01052813255091474, + "metric": 0.901449775629962, + "rescue": 0, + "harm": 1, + "missed_rescue": 106 + }, + { + "seed": 2029, + "budget": 0.01, + "threshold": -0.004320982377976179, + "operator": "ge", + "actual_fraction": 0.014670348636520539, + "metric": 0.901449775629962, + "rescue": 0, + "harm": 1, + "missed_rescue": 106 + }, + { + "seed": 2030, + "budget": 0.01, + "threshold": -0.002133682370185852, + "operator": "ge", + "actual_fraction": 0.018294787711425613, + "metric": 0.9012771832930618, + "rescue": 2, + "harm": 4, + "missed_rescue": 104 + } + ], + "rescued": 0.4, + "harmed": 1.2, + "missed_rescue": 105.6, + "avoided_harm": 3789.8 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.9891666666666665, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.9893333333333333, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.01, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.9893333333333333, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.02, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.9888333333333333, + "metric_std": 0.00045643546458764894, + "actual_fraction": 0.03, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.9886666666666667, + "metric_std": 0.000456435464587649, + "actual_fraction": 0.04, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.9884999999999999, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.05, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.9884999999999999, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.06, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.9884999999999999, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.07, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.9883333333333333, + "metric_std": 0.0008333333333332971, + "actual_fraction": 0.08, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.9880000000000001, + "metric_std": 0.0011180339887498626, + "actual_fraction": 0.09, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.986, + "metric_std": 0.001369306393762947, + "actual_fraction": 0.1, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.9846666666666668, + "metric_std": 0.001624465724134828, + "actual_fraction": 0.11000000000000001, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.12, + "metric": 0.9831666666666667, + "metric_std": 0.001369306393762886, + "actual_fraction": 0.12, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.9818333333333333, + "metric_std": 0.0009128709291752523, + "actual_fraction": 0.13, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.9804999999999999, + "metric_std": 0.0007453559924999471, + "actual_fraction": 0.14, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.9788333333333334, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.15, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.9758333333333333, + "metric_std": 0.0011785113019776065, + "actual_fraction": 0.16, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.9743333333333334, + "metric_std": 0.0010865337342004155, + "actual_fraction": 0.17, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.9738333333333333, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.18, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.9718333333333333, + "metric_std": 0.000912870929175298, + "actual_fraction": 0.19, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.97, + "metric_std": 0.001020620726159681, + "actual_fraction": 0.2, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.9680000000000002, + "metric_std": 0.0007453559924999036, + "actual_fraction": 0.21000000000000002, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.9650000000000001, + "metric_std": 0.00131761569173682, + "actual_fraction": 0.22000000000000003, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.23, + "metric": 0.96, + "metric_std": 0.001863389981249818, + "actual_fraction": 0.23000000000000004, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.24, + "metric": 0.9551666666666666, + "metric_std": 0.0016029486718059363, + "actual_fraction": 0.24, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.9513333333333334, + "metric_std": 0.0013944333775567817, + "actual_fraction": 0.25, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.26, + "metric": 0.9466666666666667, + "metric_std": 0.0014433756729740495, + "actual_fraction": 0.26, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.9408333333333332, + "metric_std": 0.0010206207261596358, + "actual_fraction": 0.27, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.9353333333333333, + "metric_std": 0.0009501461875825834, + "actual_fraction": 0.28, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.9275, + "metric_std": 0.0014433756729740977, + "actual_fraction": 0.29, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.921, + "metric_std": 0.0013693063937628895, + "actual_fraction": 0.3, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.9136666666666666, + "metric_std": 0.002006932429798723, + "actual_fraction": 0.31, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.9053333333333333, + "metric_std": 0.0024008100484813276, + "actual_fraction": 0.32, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.8991666666666667, + "metric_std": 0.0023570226039551835, + "actual_fraction": 0.33, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.8913333333333332, + "metric_std": 0.001728036779443979, + "actual_fraction": 0.34, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.8831666666666667, + "metric_std": 0.0019002923751652277, + "actual_fraction": 0.35, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.8756666666666666, + "metric_std": 0.0012360330811825979, + "actual_fraction": 0.36, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.867, + "metric_std": 0.000950146187582598, + "actual_fraction": 0.37, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.859, + "metric_std": 0.000912870929175298, + "actual_fraction": 0.38, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.852, + "metric_std": 0.0017280367794439816, + "actual_fraction": 0.39, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.8428333333333333, + "metric_std": 0.0019184774055368834, + "actual_fraction": 0.4, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.8355, + "metric_std": 0.001825741858350563, + "actual_fraction": 0.4091666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.42, + "metric": 0.8263333333333334, + "metric_std": 0.0009501461875826368, + "actual_fraction": 0.42000000000000004, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.8175000000000001, + "metric_std": 0.0014433756729740497, + "actual_fraction": 0.43, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.44, + "metric": 0.8086666666666668, + "metric_std": 0.0020916500663351973, + "actual_fraction": 0.44000000000000006, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.45, + "metric": 0.7995000000000001, + "metric_std": 0.0013944333775567817, + "actual_fraction": 0.45, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.7908333333333334, + "metric_std": 0.000589255650988764, + "actual_fraction": 0.4600000000000001, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.47, + "metric": 0.7818333333333334, + "metric_std": 0.0006972166887783725, + "actual_fraction": 0.4699999999999999, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.48, + "metric": 0.7726666666666666, + "metric_std": 0.0010865337342004667, + "actual_fraction": 0.48, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.7634999999999998, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.49000000000000005, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.754, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.5, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.51, + "metric": 0.7446666666666667, + "metric_std": 0.0007453559924999471, + "actual_fraction": 0.51, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.7356666666666667, + "metric_std": 0.000912870929175298, + "actual_fraction": 0.52, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.7265, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.53, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.7173333333333334, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.54, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.7075, + "metric_std": 0.0005892556509888032, + "actual_fraction": 0.55, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.6980000000000001, + "metric_std": 0.00045643546458764894, + "actual_fraction": 0.56, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.6895000000000001, + "metric_std": 0.0007453559924998974, + "actual_fraction": 0.5691666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.68, + "metric_std": 0.0008333333333332971, + "actual_fraction": 0.58, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.6708333333333334, + "metric_std": 0.0015590239111557857, + "actual_fraction": 0.59, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.6611666666666667, + "metric_std": 0.0011180339887498628, + "actual_fraction": 0.6, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.6515000000000001, + "metric_std": 0.0010865337342004112, + "actual_fraction": 0.61, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.642, + "metric_std": 0.0007453559924998913, + "actual_fraction": 0.62, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.6326666666666666, + "metric_std": 0.0006972166887783726, + "actual_fraction": 0.63, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.623, + "metric_std": 0.0009501461875825834, + "actual_fraction": 0.64, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.6140000000000001, + "metric_std": 0.0010865337342004283, + "actual_fraction": 0.65, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.6048333333333333, + "metric_std": 0.001369306393762886, + "actual_fraction": 0.66, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.5956666666666666, + "metric_std": 0.001989556064385567, + "actual_fraction": 0.67, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.5863333333333334, + "metric_std": 0.0013944333775568248, + "actual_fraction": 0.68, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.5774999999999999, + "metric_std": 0.0013176156917368551, + "actual_fraction": 0.6891666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.5675, + "metric_std": 0.0018633899812498305, + "actual_fraction": 0.7, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.5581666666666667, + "metric_std": 0.0019002923751652349, + "actual_fraction": 0.71, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.5491666666666666, + "metric_std": 0.0011785113019775672, + "actual_fraction": 0.72, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.5395, + "metric_std": 0.0009501461875826369, + "actual_fraction": 0.73, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.5298333333333333, + "metric_std": 0.0012360330811825979, + "actual_fraction": 0.74, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.5199999999999999, + "metric_std": 0.0010206207261596358, + "actual_fraction": 0.75, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.5103333333333334, + "metric_std": 0.0009501461875826369, + "actual_fraction": 0.76, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.501, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.77, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.4915, + "metric_std": 0.0003726779962499487, + "actual_fraction": 0.78, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.48183333333333334, + "metric_std": 0.0006972166887783858, + "actual_fraction": 0.79, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.472, + "metric_std": 0.0004564354645876186, + "actual_fraction": 0.8, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.4621666666666667, + "metric_std": 0.0007453559924999254, + "actual_fraction": 0.8100000000000002, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.82, + "metric": 0.4533333333333333, + "metric_std": 0.0005892556509887836, + "actual_fraction": 0.8191666666666666, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.83, + "metric": 0.44266666666666665, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.8299999999999998, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.4328333333333333, + "metric_std": 0.00045643546458764894, + "actual_fraction": 0.8400000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.42350000000000004, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.85, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.4138333333333333, + "metric_std": 0.0007453559924999254, + "actual_fraction": 0.86, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.87, + "metric": 0.4043333333333333, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.8699999999999999, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.88, + "metric": 0.39449999999999996, + "metric_std": 0.0007453559924999223, + "actual_fraction": 0.8800000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.89, + "metric": 0.38483333333333336, + "metric_std": 0.001086533734200443, + "actual_fraction": 0.89, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.37516666666666665, + "metric_std": 0.0010865337342004474, + "actual_fraction": 0.9, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.36533333333333334, + "metric_std": 0.0011180339887498752, + "actual_fraction": 0.9099999999999999, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.3556666666666667, + "metric_std": 0.0012360330811826015, + "actual_fraction": 0.9200000000000002, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.93, + "metric": 0.346, + "metric_std": 0.0010865337342004474, + "actual_fraction": 0.93, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.33599999999999997, + "metric_std": 0.0010865337342004409, + "actual_fraction": 0.9399999999999998, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.95, + "metric": 0.32599999999999996, + "metric_std": 0.0010865337342004409, + "actual_fraction": 0.95, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.317, + "metric_std": 0.000456435464587649, + "actual_fraction": 0.96, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.30733333333333335, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.97, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.98, + "metric": 0.29733333333333334, + "metric_std": 0.0003726779962499487, + "actual_fraction": 0.9800000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.2875, + "metric_std": 0.0, + "actual_fraction": 0.99, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.2775, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.9016223679668622, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.901484294097342, + "metric_std": 0.00014440110917050722, + "actual_fraction": 0.013289609941318605, + "actual_fraction_std": 0.00372375716437546 + }, + { + "budget": 0.02, + "metric": 0.9014497756299621, + "metric_std": 0.0002113815794600336, + "actual_fraction": 0.021263375906109768, + "actual_fraction_std": 0.002784037977415148 + }, + { + "budget": 0.03, + "metric": 0.901415257162582, + "metric_std": 0.00018906543234555831, + "actual_fraction": 0.02999654815326199, + "actual_fraction_std": 0.0016254863207416103 + }, + { + "budget": 0.04, + "metric": 0.9013117017604418, + "metric_std": 0.00025599580556074936, + "actual_fraction": 0.03727994477045219, + "actual_fraction_std": 0.002739818409801095 + }, + { + "budget": 0.05, + "metric": 0.9011045909561615, + "metric_std": 0.00032289069613168175, + "actual_fraction": 0.044218156713841904, + "actual_fraction_std": 0.002749044327356576 + }, + { + "budget": 0.06, + "metric": 0.9007939247497412, + "metric_std": 0.0003741712700842374, + "actual_fraction": 0.054228512254055915, + "actual_fraction_std": 0.0030544455375393765 + }, + { + "budget": 0.07, + "metric": 0.9005522954780808, + "metric_std": 0.00037417127008420155, + "actual_fraction": 0.0652053848809113, + "actual_fraction_std": 0.0014816828464849683 + }, + { + "budget": 0.08, + "metric": 0.9002761477390404, + "metric_std": 0.0003537090357597093, + "actual_fraction": 0.07994477045219192, + "actual_fraction_std": 0.0020362970250262544 + }, + { + "budget": 0.09, + "metric": 0.8996202968588195, + "metric_std": 0.00026174923866248607, + "actual_fraction": 0.09447704521919227, + "actual_fraction_std": 0.001939649167702628 + }, + { + "budget": 0.1, + "metric": 0.8991715567828787, + "metric_std": 0.0005375357784741999, + "actual_fraction": 0.10465999309630651, + "actual_fraction_std": 0.00239026366962569 + }, + { + "budget": 0.11, + "metric": 0.8981705212288574, + "metric_std": 0.0007322472708179898, + "actual_fraction": 0.11491197790818089, + "actual_fraction_std": 0.0034763541472047567 + }, + { + "budget": 0.12, + "metric": 0.897169485674836, + "metric_std": 0.0010019278752985829, + "actual_fraction": 0.12474974111149464, + "actual_fraction_std": 0.004920243984068183 + }, + { + "budget": 0.13, + "metric": 0.8957887469796342, + "metric_std": 0.0007463496137535215, + "actual_fraction": 0.13455298584742836, + "actual_fraction_std": 0.004360476309899295 + }, + { + "budget": 0.14, + "metric": 0.8943734898170522, + "metric_std": 0.0005457848912958931, + "actual_fraction": 0.14390749050742146, + "actual_fraction_std": 0.0033274978970782828 + }, + { + "budget": 0.15, + "metric": 0.8929927511218502, + "metric_std": 0.0009531723349315986, + "actual_fraction": 0.15377977217811528, + "actual_fraction_std": 0.003247305440494786 + }, + { + "budget": 0.16, + "metric": 0.8916465308940282, + "metric_std": 0.001440913459768144, + "actual_fraction": 0.16320331377286848, + "actual_fraction_std": 0.003802127311311384 + }, + { + "budget": 0.17, + "metric": 0.8890231273731446, + "metric_std": 0.001331312445680136, + "actual_fraction": 0.1745598895409044, + "actual_fraction_std": 0.003637974626958772 + }, + { + "budget": 0.18, + "metric": 0.886365205384881, + "metric_std": 0.0021124061140206114, + "actual_fraction": 0.18733172247152224, + "actual_fraction_std": 0.004947713886942637 + }, + { + "budget": 0.19, + "metric": 0.8844666896789782, + "metric_std": 0.0023808986198906898, + "actual_fraction": 0.19489126682775285, + "actual_fraction_std": 0.0042053779534121215 + }, + { + "budget": 0.2, + "metric": 0.8824301001035554, + "metric_std": 0.002414442804848248, + "actual_fraction": 0.20390058681394546, + "actual_fraction_std": 0.004143294647291995 + }, + { + "budget": 0.21, + "metric": 0.880048325854332, + "metric_std": 0.001380738695201958, + "actual_fraction": 0.21508457024508112, + "actual_fraction_std": 0.004212455336327983 + }, + { + "budget": 0.22, + "metric": 0.8764238867794271, + "metric_std": 0.0014440110917053443, + "actual_fraction": 0.22795995857783913, + "actual_fraction_std": 0.005402167715568055 + }, + { + "budget": 0.23, + "metric": 0.8714532274767001, + "metric_std": 0.001935036440203505, + "actual_fraction": 0.24093890231273735, + "actual_fraction_std": 0.005460577155525151 + }, + { + "budget": 0.24, + "metric": 0.8666896789782534, + "metric_std": 0.0022390488412605803, + "actual_fraction": 0.2526751812219537, + "actual_fraction_std": 0.006307299521788604 + }, + { + "budget": 0.25, + "metric": 0.8623748705557472, + "metric_std": 0.0027765381501125524, + "actual_fraction": 0.2626855367621677, + "actual_fraction_std": 0.008588094790904194 + }, + { + "budget": 0.26, + "metric": 0.8579910251984811, + "metric_std": 0.0028026997451275513, + "actual_fraction": 0.27235070762858127, + "actual_fraction_std": 0.009878555761998054 + }, + { + "budget": 0.27, + "metric": 0.853607179841215, + "metric_std": 0.0038193245209525175, + "actual_fraction": 0.28184328615809456, + "actual_fraction_std": 0.011143238145316994 + }, + { + "budget": 0.28, + "metric": 0.8477045219192268, + "metric_std": 0.0040043653896122515, + "actual_fraction": 0.2939247497411115, + "actual_fraction_std": 0.010836957249534433 + }, + { + "budget": 0.29, + "metric": 0.8427683810838799, + "metric_std": 0.0032747093477753482, + "actual_fraction": 0.3034173282706248, + "actual_fraction_std": 0.010032648691290036 + }, + { + "budget": 0.3, + "metric": 0.8360372799447704, + "metric_std": 0.004585892390606324, + "actual_fraction": 0.3145667932343804, + "actual_fraction_std": 0.01341709781991186 + }, + { + "budget": 0.31, + "metric": 0.830652399033483, + "metric_std": 0.005459758825902133, + "actual_fraction": 0.3241974456334139, + "actual_fraction_std": 0.0159312169705078 + }, + { + "budget": 0.32, + "metric": 0.824370037970314, + "metric_std": 0.005345627978646615, + "actual_fraction": 0.33462202278218844, + "actual_fraction_std": 0.016091123363148708 + }, + { + "budget": 0.33, + "metric": 0.8168450120814637, + "metric_std": 0.006168093722313238, + "actual_fraction": 0.3466689678978253, + "actual_fraction_std": 0.01749093693847181 + }, + { + "budget": 0.34, + "metric": 0.8086641353123921, + "metric_std": 0.0059017999445078705, + "actual_fraction": 0.35988954090438385, + "actual_fraction_std": 0.017243002378050053 + }, + { + "budget": 0.35, + "metric": 0.8025543665861236, + "metric_std": 0.004807866164026271, + "actual_fraction": 0.3690369347600967, + "actual_fraction_std": 0.013844229410388742 + }, + { + "budget": 0.36, + "metric": 0.7966862271315154, + "metric_std": 0.005287357156382614, + "actual_fraction": 0.3781152916810494, + "actual_fraction_std": 0.01562575566896309 + }, + { + "budget": 0.37, + "metric": 0.790680013807387, + "metric_std": 0.006424748292216629, + "actual_fraction": 0.38667587159130135, + "actual_fraction_std": 0.016459554856955032 + }, + { + "budget": 0.38, + "metric": 0.7824991370383156, + "metric_std": 0.005806135502220223, + "actual_fraction": 0.39851570590265795, + "actual_fraction_std": 0.015395101008488909 + }, + { + "budget": 0.39, + "metric": 0.7738004832585433, + "metric_std": 0.005749682578801256, + "actual_fraction": 0.41132205730065585, + "actual_fraction_std": 0.015202534195413071 + }, + { + "budget": 0.4, + "metric": 0.7641353123921297, + "metric_std": 0.008284072596039612, + "actual_fraction": 0.4248878149810148, + "actual_fraction_std": 0.01669486308251398 + }, + { + "budget": 0.41, + "metric": 0.7575422851225404, + "metric_std": 0.0074188610740415824, + "actual_fraction": 0.4339661719019675, + "actual_fraction_std": 0.01451397912139658 + }, + { + "budget": 0.42, + "metric": 0.7483603727994477, + "metric_std": 0.006370738289710635, + "actual_fraction": 0.4466689678978253, + "actual_fraction_std": 0.014376537630433455 + }, + { + "budget": 0.43, + "metric": 0.7389713496720746, + "metric_std": 0.006016767891800478, + "actual_fraction": 0.45930272695892305, + "actual_fraction_std": 0.013872709649110537 + }, + { + "budget": 0.44, + "metric": 0.732102174663445, + "metric_std": 0.006594066356227773, + "actual_fraction": 0.4687262685536762, + "actual_fraction_std": 0.01428372344983957 + }, + { + "budget": 0.45, + "metric": 0.7225405591991716, + "metric_std": 0.010094512517188386, + "actual_fraction": 0.4815671384190542, + "actual_fraction_std": 0.020289108295970096 + }, + { + "budget": 0.46, + "metric": 0.714981014842941, + "metric_std": 0.009813965123373646, + "actual_fraction": 0.49143942008974795, + "actual_fraction_std": 0.019367521200469195 + }, + { + "budget": 0.47, + "metric": 0.7047635484984467, + "metric_std": 0.0075128236929246, + "actual_fraction": 0.505005177770107, + "actual_fraction_std": 0.016167121355780873 + }, + { + "budget": 0.48, + "metric": 0.6965481532619952, + "metric_std": 0.0074907857910349, + "actual_fraction": 0.5148774594408009, + "actual_fraction_std": 0.016375093056384914 + }, + { + "budget": 0.49, + "metric": 0.688574387297204, + "metric_std": 0.007025165373774351, + "actual_fraction": 0.5257852951328961, + "actual_fraction_std": 0.016155878123225446 + }, + { + "budget": 0.5, + "metric": 0.6809803244735934, + "metric_std": 0.006746130870748459, + "actual_fraction": 0.5354159475319296, + "actual_fraction_std": 0.013694416164030934 + }, + { + "budget": 0.51, + "metric": 0.672178115291681, + "metric_std": 0.006509042603827034, + "actual_fraction": 0.5462547462892648, + "actual_fraction_std": 0.013778771694231005 + }, + { + "budget": 0.52, + "metric": 0.6618225750776665, + "metric_std": 0.007038932538019977, + "actual_fraction": 0.5588194684156023, + "actual_fraction_std": 0.013519830981561115 + }, + { + "budget": 0.53, + "metric": 0.6544011045909561, + "metric_std": 0.006962341498630764, + "actual_fraction": 0.5676907145322747, + "actual_fraction_std": 0.012222048071839492 + }, + { + "budget": 0.54, + "metric": 0.6447359337245426, + "metric_std": 0.008387703465928107, + "actual_fraction": 0.5788056610286504, + "actual_fraction_std": 0.014374776326753155 + }, + { + "budget": 0.55, + "metric": 0.6380048325854333, + "metric_std": 0.008734775230439596, + "actual_fraction": 0.5869520193303417, + "actual_fraction_std": 0.013539096047307676 + }, + { + "budget": 0.56, + "metric": 0.627856403175699, + "metric_std": 0.009460673781999693, + "actual_fraction": 0.5989989644459787, + "actual_fraction_std": 0.01469246439569468 + }, + { + "budget": 0.57, + "metric": 0.6194684156023473, + "metric_std": 0.009287488568011107, + "actual_fraction": 0.6087331722471523, + "actual_fraction_std": 0.013236830851787363 + }, + { + "budget": 0.58, + "metric": 0.6081118398343114, + "metric_std": 0.011150319877539757, + "actual_fraction": 0.6219537452537107, + "actual_fraction_std": 0.015661078484050975 + }, + { + "budget": 0.59, + "metric": 0.599309630652399, + "metric_std": 0.011925493544623175, + "actual_fraction": 0.6322057300655851, + "actual_fraction_std": 0.016549526311358705 + }, + { + "budget": 0.6, + "metric": 0.5920262340352089, + "metric_std": 0.011114732093889159, + "actual_fraction": 0.6408008284432172, + "actual_fraction_std": 0.016088809179090228 + }, + { + "budget": 0.61, + "metric": 0.5846047635484984, + "metric_std": 0.011444837185740206, + "actual_fraction": 0.6493959268208491, + "actual_fraction_std": 0.015821455834873852 + }, + { + "budget": 0.62, + "metric": 0.5758715913013462, + "metric_std": 0.008554732043391407, + "actual_fraction": 0.6595098377632033, + "actual_fraction_std": 0.013483979977750901 + }, + { + "budget": 0.63, + "metric": 0.5680013807386952, + "metric_std": 0.00888641053631755, + "actual_fraction": 0.6688643424231964, + "actual_fraction_std": 0.013642766487564201 + }, + { + "budget": 0.64, + "metric": 0.5594062823610632, + "metric_std": 0.011073113291277803, + "actual_fraction": 0.6789437348981705, + "actual_fraction_std": 0.0164368264701164 + }, + { + "budget": 0.65, + "metric": 0.5494994822229893, + "metric_std": 0.011390048173222127, + "actual_fraction": 0.6902657921988264, + "actual_fraction_std": 0.016513307933819173 + }, + { + "budget": 0.66, + "metric": 0.5409389023127373, + "metric_std": 0.010980589788378377, + "actual_fraction": 0.7004832585433206, + "actual_fraction_std": 0.015590543890179413 + }, + { + "budget": 0.67, + "metric": 0.5342078011736279, + "metric_std": 0.010210992080662598, + "actual_fraction": 0.7084570245081119, + "actual_fraction_std": 0.014773238656023112 + }, + { + "budget": 0.68, + "metric": 0.5274421815671385, + "metric_std": 0.008789509407325681, + "actual_fraction": 0.7161546427338626, + "actual_fraction_std": 0.01253678706410476 + }, + { + "budget": 0.69, + "metric": 0.5194338971349672, + "metric_std": 0.0067483383021320455, + "actual_fraction": 0.7252329996548152, + "actual_fraction_std": 0.010126479400556415 + }, + { + "budget": 0.7, + "metric": 0.5100793924749741, + "metric_std": 0.005542606133270794, + "actual_fraction": 0.7356575768035899, + "actual_fraction_std": 0.009654840625722375 + }, + { + "budget": 0.71, + "metric": 0.5011045909561614, + "metric_std": 0.008565171861834836, + "actual_fraction": 0.7460476354849845, + "actual_fraction_std": 0.01247545357815643 + }, + { + "budget": 0.72, + "metric": 0.49316534345875046, + "metric_std": 0.006021716706497786, + "actual_fraction": 0.7548843631342769, + "actual_fraction_std": 0.010186896258324652 + }, + { + "budget": 0.73, + "metric": 0.48757335174318256, + "metric_std": 0.005983754167538734, + "actual_fraction": 0.7612357611322057, + "actual_fraction_std": 0.01000187111431559 + }, + { + "budget": 0.74, + "metric": 0.4812219537452537, + "metric_std": 0.0056369301939049755, + "actual_fraction": 0.7686572316189161, + "actual_fraction_std": 0.00904719745161027 + }, + { + "budget": 0.75, + "metric": 0.4742147048671039, + "metric_std": 0.006123502172426892, + "actual_fraction": 0.7765274421815671, + "actual_fraction_std": 0.009731514498496997 + }, + { + "budget": 0.76, + "metric": 0.4664480497065931, + "metric_std": 0.007291490054696233, + "actual_fraction": 0.7853986882982396, + "actual_fraction_std": 0.010332213976927358 + }, + { + "budget": 0.77, + "metric": 0.45778391439420096, + "metric_std": 0.006669527102138495, + "actual_fraction": 0.7950293406972729, + "actual_fraction_std": 0.008141713073306784 + }, + { + "budget": 0.78, + "metric": 0.450742147048671, + "metric_std": 0.009233770674435435, + "actual_fraction": 0.8029340697273041, + "actual_fraction_std": 0.009532035859712506 + }, + { + "budget": 0.79, + "metric": 0.4413186054539178, + "metric_std": 0.0073153501734631405, + "actual_fraction": 0.813703831549879, + "actual_fraction_std": 0.006789046322055943 + }, + { + "budget": 0.8, + "metric": 0.4321021746634449, + "metric_std": 0.005267036289309399, + "actual_fraction": 0.8237487055574733, + "actual_fraction_std": 0.005276642006340269 + }, + { + "budget": 0.81, + "metric": 0.42405937176389363, + "metric_std": 0.005745017941695982, + "actual_fraction": 0.8324473593372455, + "actual_fraction_std": 0.005542337407366566 + }, + { + "budget": 0.82, + "metric": 0.41632723507076286, + "metric_std": 0.0035714521890700484, + "actual_fraction": 0.8408353469105971, + "actual_fraction_std": 0.003647787118488704 + }, + { + "budget": 0.83, + "metric": 0.4075595443562306, + "metric_std": 0.003485339753564446, + "actual_fraction": 0.8502588885053504, + "actual_fraction_std": 0.004531329551222694 + }, + { + "budget": 0.84, + "metric": 0.4011045909561616, + "metric_std": 0.004001388719147246, + "actual_fraction": 0.8573006558508801, + "actual_fraction_std": 0.0047914194183955846 + }, + { + "budget": 0.85, + "metric": 0.3928201587849499, + "metric_std": 0.004197579049711867, + "actual_fraction": 0.8662409389023127, + "actual_fraction_std": 0.005649598654930242 + }, + { + "budget": 0.86, + "metric": 0.3847428374180186, + "metric_std": 0.003271068758158683, + "actual_fraction": 0.8750086296168449, + "actual_fraction_std": 0.004389074368605423 + }, + { + "budget": 0.87, + "metric": 0.37714877459440804, + "metric_std": 0.004224812228041219, + "actual_fraction": 0.8831204694511563, + "actual_fraction_std": 0.004905085145945548 + }, + { + "budget": 0.88, + "metric": 0.3705557473248188, + "metric_std": 0.005159038722000384, + "actual_fraction": 0.8901967552640663, + "actual_fraction_std": 0.005332795998180358 + }, + { + "budget": 0.89, + "metric": 0.3624439074905074, + "metric_std": 0.004489059994352764, + "actual_fraction": 0.8988263721090783, + "actual_fraction_std": 0.004461436795519777 + }, + { + "budget": 0.9, + "metric": 0.35484984466689673, + "metric_std": 0.005012611574545251, + "actual_fraction": 0.9065585088022093, + "actual_fraction_std": 0.00507697308432502 + }, + { + "budget": 0.91, + "metric": 0.3477045219192268, + "metric_std": 0.0037409165070462764, + "actual_fraction": 0.9142216085605799, + "actual_fraction_std": 0.0037615627748884117 + }, + { + "budget": 0.92, + "metric": 0.3380738695201933, + "metric_std": 0.003376375500056809, + "actual_fraction": 0.924301001035554, + "actual_fraction_std": 0.0035404574124841353 + }, + { + "budget": 0.93, + "metric": 0.330686917500863, + "metric_std": 0.0031257801650457083, + "actual_fraction": 0.9320331377286848, + "actual_fraction_std": 0.0032914946964877467 + }, + { + "budget": 0.94, + "metric": 0.32267863306869177, + "metric_std": 0.0022122810071335095, + "actual_fraction": 0.9405246807041767, + "actual_fraction_std": 0.002636203173644803 + }, + { + "budget": 0.95, + "metric": 0.31532619951674146, + "metric_std": 0.002056675058303921, + "actual_fraction": 0.9482913358646877, + "actual_fraction_std": 0.0021018033729197027 + }, + { + "budget": 0.96, + "metric": 0.30593717638936835, + "metric_std": 0.0018851318496255695, + "actual_fraction": 0.9580255436658612, + "actual_fraction_std": 0.0020289695560491115 + }, + { + "budget": 0.97, + "metric": 0.2937521574042113, + "metric_std": 0.0037077236898413715, + "actual_fraction": 0.970693821194339, + "actual_fraction_std": 0.003682329403685058 + }, + { + "budget": 0.98, + "metric": 0.2848809112875388, + "metric_std": 0.0030985011515376457, + "actual_fraction": 0.9800138073869519, + "actual_fraction_std": 0.0033807838783543394 + }, + { + "budget": 0.99, + "metric": 0.27487055574732483, + "metric_std": 0.002255617432199455, + "actual_fraction": 0.9905074214704868, + "actual_fraction_std": 0.002315569186226903 + }, + { + "budget": 1.0, + "metric": 0.26561960648947186, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 330, + "rescuable": 3, + "harmful": 857, + "both_wrong": 10, + "unit": "sample", + "total": 1200 + }, + "test": { + "both_correct": 1433, + "rescuable": 106, + "harmful": 3791, + "both_wrong": 464, + "unit": "sample", + "total": 5794 + } + }, + "small": { + "validation": 0.9891666666666666, + "test": 0.9016223679668622 + }, + "large": { + "validation": 0.2775, + "test": 0.26561960648947186 + }, + "learned_minus_confidence_test": 0.024542630307214264, + "learned_minus_small_test": -0.00013807386952024991, + "learned_minus_large_test": 0.6358646876078702, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_glsim_MiniCPM" + }, + { + "key": [ + "cub", + "glsim", + "Qwen3.8" + ], + "expert_display": "GLSim", + "run": "five_seed_glsim_Qwen3.8_four_difference", + "validation_rows": 1200, + "test_rows": 5794, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/glsim/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 1, + "validation_area_0_25": 0.9213333333333333, + "validation_point": { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescued": 1, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 523, + "selected_both_correct": 1, + "selected_both_wrong": 5 + }, + "test_point": { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.08698653779772178, + "metric": 0.8878149810148429, + "rescued": 70, + "harmed": 150, + "missed_rescue": 74, + "avoided_harm": 2042, + "selected_both_correct": 85, + "selected_both_wrong": 199 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9891666666666666, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2, + "avoided_harm": 528, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9858333333333333, + "rescued": 1, + "harmed": 5, + "missed_rescue": 1, + "avoided_harm": 523, + "selected_both_correct": 1, + "selected_both_wrong": 5 + }, + { + "budget": 0.02, + "threshold": -0.6014261841773987, + "operator": "ge", + "actual_fraction": 0.02, + "metric": 0.98, + "rescued": 1, + "harmed": 12, + "missed_rescue": 1, + "avoided_harm": 516, + "selected_both_correct": 4, + "selected_both_wrong": 7 + }, + { + "budget": 0.03, + "threshold": -0.6401472687721252, + "operator": "ge", + "actual_fraction": 0.03, + "metric": 0.9741666666666666, + "rescued": 2, + "harmed": 20, + "missed_rescue": 0, + "avoided_harm": 508, + "selected_both_correct": 6, + "selected_both_wrong": 8 + }, + { + "budget": 0.04, + "threshold": -0.6884650588035583, + "operator": "ge", + "actual_fraction": 0.04, + "metric": 0.9675, + "rescued": 2, + "harmed": 28, + "missed_rescue": 0, + "avoided_harm": 500, + "selected_both_correct": 8, + "selected_both_wrong": 10 + }, + { + "budget": 0.05, + "threshold": -0.7208121418952942, + "operator": "ge", + "actual_fraction": 0.05, + "metric": 0.96, + "rescued": 2, + "harmed": 37, + "missed_rescue": 0, + "avoided_harm": 491, + "selected_both_correct": 11, + "selected_both_wrong": 10 + }, + { + "budget": 0.06, + "threshold": -0.7578181624412537, + "operator": "ge", + "actual_fraction": 0.06, + "metric": 0.9558333333333333, + "rescued": 2, + "harmed": 42, + "missed_rescue": 0, + "avoided_harm": 486, + "selected_both_correct": 18, + "selected_both_wrong": 10 + }, + { + "budget": 0.07, + "threshold": -0.7790470719337463, + "operator": "ge", + "actual_fraction": 0.07, + "metric": 0.95, + "rescued": 2, + "harmed": 49, + "missed_rescue": 0, + "avoided_harm": 479, + "selected_both_correct": 23, + "selected_both_wrong": 10 + }, + { + "budget": 0.08, + "threshold": -0.7927228808403015, + "operator": "ge", + "actual_fraction": 0.08, + "metric": 0.945, + "rescued": 2, + "harmed": 55, + "missed_rescue": 0, + "avoided_harm": 473, + "selected_both_correct": 28, + "selected_both_wrong": 11 + }, + { + "budget": 0.09, + "threshold": -0.8049252033233643, + "operator": "ge", + "actual_fraction": 0.09, + "metric": 0.9408333333333333, + "rescued": 2, + "harmed": 60, + "missed_rescue": 0, + "avoided_harm": 468, + "selected_both_correct": 35, + "selected_both_wrong": 11 + }, + { + "budget": 0.1, + "threshold": -0.80850750207901, + "operator": "ge", + "actual_fraction": 0.1, + "metric": 0.9333333333333333, + "rescued": 2, + "harmed": 69, + "missed_rescue": 0, + "avoided_harm": 459, + "selected_both_correct": 38, + "selected_both_wrong": 11 + }, + { + "budget": 0.11, + "threshold": -0.819871723651886, + "operator": "ge", + "actual_fraction": 0.11, + "metric": 0.9258333333333333, + "rescued": 2, + "harmed": 78, + "missed_rescue": 0, + "avoided_harm": 450, + "selected_both_correct": 41, + "selected_both_wrong": 11 + }, + { + "budget": 0.12, + "threshold": -0.8310170769691467, + "operator": "ge", + "actual_fraction": 0.12, + "metric": 0.9225, + "rescued": 2, + "harmed": 82, + "missed_rescue": 0, + "avoided_harm": 446, + "selected_both_correct": 49, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -0.8376906514167786, + "operator": "ge", + "actual_fraction": 0.13, + "metric": 0.9183333333333333, + "rescued": 2, + "harmed": 87, + "missed_rescue": 0, + "avoided_harm": 441, + "selected_both_correct": 56, + "selected_both_wrong": 11 + }, + { + "budget": 0.14, + "threshold": -0.8419997692108154, + "operator": "ge", + "actual_fraction": 0.14, + "metric": 0.9108333333333334, + "rescued": 2, + "harmed": 96, + "missed_rescue": 0, + "avoided_harm": 432, + "selected_both_correct": 59, + "selected_both_wrong": 11 + }, + { + "budget": 0.15, + "threshold": -0.8447275161743164, + "operator": "ge", + "actual_fraction": 0.15, + "metric": 0.9058333333333334, + "rescued": 2, + "harmed": 102, + "missed_rescue": 0, + "avoided_harm": 426, + "selected_both_correct": 65, + "selected_both_wrong": 11 + }, + { + "budget": 0.16, + "threshold": -0.8506776690483093, + "operator": "ge", + "actual_fraction": 0.16, + "metric": 0.9016666666666666, + "rescued": 2, + "harmed": 107, + "missed_rescue": 0, + "avoided_harm": 421, + "selected_both_correct": 72, + "selected_both_wrong": 11 + }, + { + "budget": 0.17, + "threshold": -0.8545639514923096, + "operator": "ge", + "actual_fraction": 0.17, + "metric": 0.8966666666666666, + "rescued": 2, + "harmed": 113, + "missed_rescue": 0, + "avoided_harm": 415, + "selected_both_correct": 78, + "selected_both_wrong": 11 + }, + { + "budget": 0.18, + "threshold": -0.8578683137893677, + "operator": "ge", + "actual_fraction": 0.18, + "metric": 0.8916666666666667, + "rescued": 2, + "harmed": 119, + "missed_rescue": 0, + "avoided_harm": 409, + "selected_both_correct": 84, + "selected_both_wrong": 11 + }, + { + "budget": 0.19, + "threshold": -0.86090087890625, + "operator": "ge", + "actual_fraction": 0.19, + "metric": 0.8866666666666667, + "rescued": 2, + "harmed": 125, + "missed_rescue": 0, + "avoided_harm": 403, + "selected_both_correct": 90, + "selected_both_wrong": 11 + }, + { + "budget": 0.2, + "threshold": -0.8640955090522766, + "operator": "ge", + "actual_fraction": 0.2, + "metric": 0.8841666666666667, + "rescued": 2, + "harmed": 128, + "missed_rescue": 0, + "avoided_harm": 400, + "selected_both_correct": 99, + "selected_both_wrong": 11 + }, + { + "budget": 0.21, + "threshold": -0.8664237260818481, + "operator": "ge", + "actual_fraction": 0.21, + "metric": 0.8791666666666667, + "rescued": 2, + "harmed": 134, + "missed_rescue": 0, + "avoided_harm": 394, + "selected_both_correct": 105, + "selected_both_wrong": 11 + }, + { + "budget": 0.22, + "threshold": -0.8686786890029907, + "operator": "ge", + "actual_fraction": 0.22, + "metric": 0.8741666666666666, + "rescued": 2, + "harmed": 140, + "missed_rescue": 0, + "avoided_harm": 388, + "selected_both_correct": 111, + "selected_both_wrong": 11 + }, + { + "budget": 0.23, + "threshold": -0.8709638118743896, + "operator": "ge", + "actual_fraction": 0.23, + "metric": 0.8683333333333333, + "rescued": 2, + "harmed": 147, + "missed_rescue": 0, + "avoided_harm": 381, + "selected_both_correct": 116, + "selected_both_wrong": 11 + }, + { + "budget": 0.24, + "threshold": -0.8724074959754944, + "operator": "ge", + "actual_fraction": 0.24, + "metric": 0.8625, + "rescued": 2, + "harmed": 154, + "missed_rescue": 0, + "avoided_harm": 374, + "selected_both_correct": 121, + "selected_both_wrong": 11 + }, + { + "budget": 0.25, + "threshold": -0.8736594319343567, + "operator": "ge", + "actual_fraction": 0.25, + "metric": 0.8575, + "rescued": 2, + "harmed": 160, + "missed_rescue": 0, + "avoided_harm": 368, + "selected_both_correct": 127, + "selected_both_wrong": 11 + }, + { + "budget": 0.26, + "threshold": -0.8752554059028625, + "operator": "ge", + "actual_fraction": 0.26, + "metric": 0.8533333333333334, + "rescued": 2, + "harmed": 165, + "missed_rescue": 0, + "avoided_harm": 363, + "selected_both_correct": 134, + "selected_both_wrong": 11 + }, + { + "budget": 0.27, + "threshold": -0.8766024112701416, + "operator": "ge", + "actual_fraction": 0.27, + "metric": 0.8508333333333333, + "rescued": 2, + "harmed": 168, + "missed_rescue": 0, + "avoided_harm": 360, + "selected_both_correct": 143, + "selected_both_wrong": 11 + }, + { + "budget": 0.28, + "threshold": -0.878311276435852, + "operator": "ge", + "actual_fraction": 0.28, + "metric": 0.8475, + "rescued": 2, + "harmed": 172, + "missed_rescue": 0, + "avoided_harm": 356, + "selected_both_correct": 151, + "selected_both_wrong": 11 + }, + { + "budget": 0.29, + "threshold": -0.880103588104248, + "operator": "ge", + "actual_fraction": 0.29, + "metric": 0.8425, + "rescued": 2, + "harmed": 178, + "missed_rescue": 0, + "avoided_harm": 350, + "selected_both_correct": 157, + "selected_both_wrong": 11 + }, + { + "budget": 0.3, + "threshold": -0.8824952244758606, + "operator": "ge", + "actual_fraction": 0.3, + "metric": 0.8375, + "rescued": 2, + "harmed": 184, + "missed_rescue": 0, + "avoided_harm": 344, + "selected_both_correct": 163, + "selected_both_wrong": 11 + }, + { + "budget": 0.31, + "threshold": -0.8831692337989807, + "operator": "ge", + "actual_fraction": 0.31, + "metric": 0.8358333333333333, + "rescued": 2, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 342, + "selected_both_correct": 173, + "selected_both_wrong": 11 + }, + { + "budget": 0.32, + "threshold": -0.8845130205154419, + "operator": "ge", + "actual_fraction": 0.32, + "metric": 0.8316666666666667, + "rescued": 2, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 337, + "selected_both_correct": 180, + "selected_both_wrong": 11 + }, + { + "budget": 0.33, + "threshold": -0.8855781555175781, + "operator": "ge", + "actual_fraction": 0.33, + "metric": 0.8291666666666667, + "rescued": 2, + "harmed": 194, + "missed_rescue": 0, + "avoided_harm": 334, + "selected_both_correct": 189, + "selected_both_wrong": 11 + }, + { + "budget": 0.34, + "threshold": -0.8865919709205627, + "operator": "ge", + "actual_fraction": 0.34, + "metric": 0.8266666666666667, + "rescued": 2, + "harmed": 197, + "missed_rescue": 0, + "avoided_harm": 331, + "selected_both_correct": 198, + "selected_both_wrong": 11 + }, + { + "budget": 0.35, + "threshold": -0.8877584338188171, + "operator": "ge", + "actual_fraction": 0.35, + "metric": 0.8208333333333333, + "rescued": 2, + "harmed": 204, + "missed_rescue": 0, + "avoided_harm": 324, + "selected_both_correct": 203, + "selected_both_wrong": 11 + }, + { + "budget": 0.36, + "threshold": -0.8890330195426941, + "operator": "ge", + "actual_fraction": 0.36, + "metric": 0.8166666666666667, + "rescued": 2, + "harmed": 209, + "missed_rescue": 0, + "avoided_harm": 319, + "selected_both_correct": 210, + "selected_both_wrong": 11 + }, + { + "budget": 0.37, + "threshold": -0.890850305557251, + "operator": "ge", + "actual_fraction": 0.37, + "metric": 0.815, + "rescued": 2, + "harmed": 211, + "missed_rescue": 0, + "avoided_harm": 317, + "selected_both_correct": 220, + "selected_both_wrong": 11 + }, + { + "budget": 0.38, + "threshold": -0.8916605710983276, + "operator": "ge", + "actual_fraction": 0.38, + "metric": 0.8125, + "rescued": 2, + "harmed": 214, + "missed_rescue": 0, + "avoided_harm": 314, + "selected_both_correct": 229, + "selected_both_wrong": 11 + }, + { + "budget": 0.39, + "threshold": -0.8928589224815369, + "operator": "ge", + "actual_fraction": 0.39, + "metric": 0.8066666666666666, + "rescued": 2, + "harmed": 221, + "missed_rescue": 0, + "avoided_harm": 307, + "selected_both_correct": 234, + "selected_both_wrong": 11 + }, + { + "budget": 0.4, + "threshold": -0.8940943479537964, + "operator": "ge", + "actual_fraction": 0.4, + "metric": 0.8, + "rescued": 2, + "harmed": 229, + "missed_rescue": 0, + "avoided_harm": 299, + "selected_both_correct": 238, + "selected_both_wrong": 11 + }, + { + "budget": 0.41, + "threshold": -0.8949463963508606, + "operator": "ge", + "actual_fraction": 0.4091666666666667, + "metric": 0.7991666666666667, + "rescued": 2, + "harmed": 230, + "missed_rescue": 0, + "avoided_harm": 298, + "selected_both_correct": 248, + "selected_both_wrong": 11 + }, + { + "budget": 0.42, + "threshold": -0.8960229158401489, + "operator": "ge", + "actual_fraction": 0.42, + "metric": 0.7941666666666667, + "rescued": 2, + "harmed": 236, + "missed_rescue": 0, + "avoided_harm": 292, + "selected_both_correct": 255, + "selected_both_wrong": 11 + }, + { + "budget": 0.43, + "threshold": -0.8966536521911621, + "operator": "ge", + "actual_fraction": 0.43, + "metric": 0.7916666666666666, + "rescued": 2, + "harmed": 239, + "missed_rescue": 0, + "avoided_harm": 289, + "selected_both_correct": 264, + "selected_both_wrong": 11 + }, + { + "budget": 0.44, + "threshold": -0.8975363373756409, + "operator": "ge", + "actual_fraction": 0.44, + "metric": 0.7891666666666667, + "rescued": 2, + "harmed": 242, + "missed_rescue": 0, + "avoided_harm": 286, + "selected_both_correct": 273, + "selected_both_wrong": 11 + }, + { + "budget": 0.45, + "threshold": -0.8988122940063477, + "operator": "ge", + "actual_fraction": 0.45, + "metric": 0.7875, + "rescued": 2, + "harmed": 244, + "missed_rescue": 0, + "avoided_harm": 284, + "selected_both_correct": 283, + "selected_both_wrong": 11 + }, + { + "budget": 0.46, + "threshold": -0.9000959396362305, + "operator": "ge", + "actual_fraction": 0.46, + "metric": 0.7816666666666666, + "rescued": 2, + "harmed": 251, + "missed_rescue": 0, + "avoided_harm": 277, + "selected_both_correct": 288, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.9013051986694336, + "operator": "ge", + "actual_fraction": 0.47, + "metric": 0.7791666666666667, + "rescued": 2, + "harmed": 254, + "missed_rescue": 0, + "avoided_harm": 274, + "selected_both_correct": 297, + "selected_both_wrong": 11 + }, + { + "budget": 0.48, + "threshold": -0.9029324054718018, + "operator": "ge", + "actual_fraction": 0.48, + "metric": 0.775, + "rescued": 2, + "harmed": 259, + "missed_rescue": 0, + "avoided_harm": 269, + "selected_both_correct": 304, + "selected_both_wrong": 11 + }, + { + "budget": 0.49, + "threshold": -0.9036871790885925, + "operator": "ge", + "actual_fraction": 0.49, + "metric": 0.7733333333333333, + "rescued": 2, + "harmed": 261, + "missed_rescue": 0, + "avoided_harm": 267, + "selected_both_correct": 314, + "selected_both_wrong": 11 + }, + { + "budget": 0.5, + "threshold": -0.9044621586799622, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.77, + "rescued": 2, + "harmed": 265, + "missed_rescue": 0, + "avoided_harm": 263, + "selected_both_correct": 322, + "selected_both_wrong": 11 + }, + { + "budget": 0.51, + "threshold": -0.9054575562477112, + "operator": "ge", + "actual_fraction": 0.51, + "metric": 0.7666666666666667, + "rescued": 2, + "harmed": 269, + "missed_rescue": 0, + "avoided_harm": 259, + "selected_both_correct": 330, + "selected_both_wrong": 11 + }, + { + "budget": 0.52, + "threshold": -0.906682550907135, + "operator": "ge", + "actual_fraction": 0.52, + "metric": 0.7616666666666667, + "rescued": 2, + "harmed": 275, + "missed_rescue": 0, + "avoided_harm": 253, + "selected_both_correct": 336, + "selected_both_wrong": 11 + }, + { + "budget": 0.53, + "threshold": -0.9072114825248718, + "operator": "ge", + "actual_fraction": 0.53, + "metric": 0.7575, + "rescued": 2, + "harmed": 280, + "missed_rescue": 0, + "avoided_harm": 248, + "selected_both_correct": 343, + "selected_both_wrong": 11 + }, + { + "budget": 0.54, + "threshold": -0.9083563089370728, + "operator": "ge", + "actual_fraction": 0.54, + "metric": 0.7533333333333333, + "rescued": 2, + "harmed": 285, + "missed_rescue": 0, + "avoided_harm": 243, + "selected_both_correct": 350, + "selected_both_wrong": 11 + }, + { + "budget": 0.55, + "threshold": -0.9095192551612854, + "operator": "ge", + "actual_fraction": 0.55, + "metric": 0.75, + "rescued": 2, + "harmed": 289, + "missed_rescue": 0, + "avoided_harm": 239, + "selected_both_correct": 358, + "selected_both_wrong": 11 + }, + { + "budget": 0.56, + "threshold": -0.9106979370117188, + "operator": "ge", + "actual_fraction": 0.56, + "metric": 0.7466666666666667, + "rescued": 2, + "harmed": 293, + "missed_rescue": 0, + "avoided_harm": 235, + "selected_both_correct": 366, + "selected_both_wrong": 11 + }, + { + "budget": 0.57, + "threshold": -0.9115362167358398, + "operator": "ge", + "actual_fraction": 0.5691666666666667, + "metric": 0.7425, + "rescued": 2, + "harmed": 298, + "missed_rescue": 0, + "avoided_harm": 230, + "selected_both_correct": 372, + "selected_both_wrong": 11 + }, + { + "budget": 0.58, + "threshold": -0.9124338030815125, + "operator": "ge", + "actual_fraction": 0.58, + "metric": 0.7375, + "rescued": 2, + "harmed": 304, + "missed_rescue": 0, + "avoided_harm": 224, + "selected_both_correct": 379, + "selected_both_wrong": 11 + }, + { + "budget": 0.59, + "threshold": -0.9134801626205444, + "operator": "ge", + "actual_fraction": 0.59, + "metric": 0.735, + "rescued": 2, + "harmed": 307, + "missed_rescue": 0, + "avoided_harm": 221, + "selected_both_correct": 388, + "selected_both_wrong": 11 + }, + { + "budget": 0.6, + "threshold": -0.9143747091293335, + "operator": "ge", + "actual_fraction": 0.6, + "metric": 0.7316666666666667, + "rescued": 2, + "harmed": 311, + "missed_rescue": 0, + "avoided_harm": 217, + "selected_both_correct": 396, + "selected_both_wrong": 11 + }, + { + "budget": 0.61, + "threshold": -0.915804386138916, + "operator": "ge", + "actual_fraction": 0.61, + "metric": 0.7283333333333334, + "rescued": 2, + "harmed": 315, + "missed_rescue": 0, + "avoided_harm": 213, + "selected_both_correct": 404, + "selected_both_wrong": 11 + }, + { + "budget": 0.62, + "threshold": -0.9165146350860596, + "operator": "ge", + "actual_fraction": 0.62, + "metric": 0.7266666666666667, + "rescued": 2, + "harmed": 317, + "missed_rescue": 0, + "avoided_harm": 211, + "selected_both_correct": 414, + "selected_both_wrong": 11 + }, + { + "budget": 0.63, + "threshold": -0.9172822833061218, + "operator": "ge", + "actual_fraction": 0.63, + "metric": 0.725, + "rescued": 2, + "harmed": 319, + "missed_rescue": 0, + "avoided_harm": 209, + "selected_both_correct": 424, + "selected_both_wrong": 11 + }, + { + "budget": 0.64, + "threshold": -0.9181071519851685, + "operator": "ge", + "actual_fraction": 0.64, + "metric": 0.7208333333333333, + "rescued": 2, + "harmed": 324, + "missed_rescue": 0, + "avoided_harm": 204, + "selected_both_correct": 431, + "selected_both_wrong": 11 + }, + { + "budget": 0.65, + "threshold": -0.9192029237747192, + "operator": "ge", + "actual_fraction": 0.65, + "metric": 0.7175, + "rescued": 2, + "harmed": 328, + "missed_rescue": 0, + "avoided_harm": 200, + "selected_both_correct": 439, + "selected_both_wrong": 11 + }, + { + "budget": 0.66, + "threshold": -0.9200543761253357, + "operator": "ge", + "actual_fraction": 0.66, + "metric": 0.7133333333333334, + "rescued": 2, + "harmed": 333, + "missed_rescue": 0, + "avoided_harm": 195, + "selected_both_correct": 446, + "selected_both_wrong": 11 + }, + { + "budget": 0.67, + "threshold": -0.9208071231842041, + "operator": "ge", + "actual_fraction": 0.67, + "metric": 0.71, + "rescued": 2, + "harmed": 337, + "missed_rescue": 0, + "avoided_harm": 191, + "selected_both_correct": 454, + "selected_both_wrong": 11 + }, + { + "budget": 0.68, + "threshold": -0.9215553998947144, + "operator": "ge", + "actual_fraction": 0.68, + "metric": 0.7058333333333333, + "rescued": 2, + "harmed": 342, + "missed_rescue": 0, + "avoided_harm": 186, + "selected_both_correct": 461, + "selected_both_wrong": 11 + }, + { + "budget": 0.69, + "threshold": -0.9220835566520691, + "operator": "ge", + "actual_fraction": 0.6891666666666667, + "metric": 0.7016666666666667, + "rescued": 2, + "harmed": 347, + "missed_rescue": 0, + "avoided_harm": 181, + "selected_both_correct": 467, + "selected_both_wrong": 11 + }, + { + "budget": 0.7, + "threshold": -0.9232237935066223, + "operator": "ge", + "actual_fraction": 0.7, + "metric": 0.6975, + "rescued": 2, + "harmed": 352, + "missed_rescue": 0, + "avoided_harm": 176, + "selected_both_correct": 475, + "selected_both_wrong": 11 + }, + { + "budget": 0.71, + "threshold": -0.9243364334106445, + "operator": "ge", + "actual_fraction": 0.71, + "metric": 0.6933333333333334, + "rescued": 2, + "harmed": 357, + "missed_rescue": 0, + "avoided_harm": 171, + "selected_both_correct": 482, + "selected_both_wrong": 11 + }, + { + "budget": 0.72, + "threshold": -0.9252541065216064, + "operator": "ge", + "actual_fraction": 0.72, + "metric": 0.69, + "rescued": 2, + "harmed": 361, + "missed_rescue": 0, + "avoided_harm": 167, + "selected_both_correct": 490, + "selected_both_wrong": 11 + }, + { + "budget": 0.73, + "threshold": -0.925781786441803, + "operator": "ge", + "actual_fraction": 0.73, + "metric": 0.6866666666666666, + "rescued": 2, + "harmed": 365, + "missed_rescue": 0, + "avoided_harm": 163, + "selected_both_correct": 498, + "selected_both_wrong": 11 + }, + { + "budget": 0.74, + "threshold": -0.9274963140487671, + "operator": "ge", + "actual_fraction": 0.74, + "metric": 0.6833333333333333, + "rescued": 2, + "harmed": 369, + "missed_rescue": 0, + "avoided_harm": 159, + "selected_both_correct": 506, + "selected_both_wrong": 11 + }, + { + "budget": 0.75, + "threshold": -0.9286423325538635, + "operator": "ge", + "actual_fraction": 0.75, + "metric": 0.6783333333333333, + "rescued": 2, + "harmed": 375, + "missed_rescue": 0, + "avoided_harm": 153, + "selected_both_correct": 512, + "selected_both_wrong": 11 + }, + { + "budget": 0.76, + "threshold": -0.9301291704177856, + "operator": "ge", + "actual_fraction": 0.76, + "metric": 0.6741666666666667, + "rescued": 2, + "harmed": 380, + "missed_rescue": 0, + "avoided_harm": 148, + "selected_both_correct": 519, + "selected_both_wrong": 11 + }, + { + "budget": 0.77, + "threshold": -0.9318777918815613, + "operator": "ge", + "actual_fraction": 0.77, + "metric": 0.67, + "rescued": 2, + "harmed": 385, + "missed_rescue": 0, + "avoided_harm": 143, + "selected_both_correct": 526, + "selected_both_wrong": 11 + }, + { + "budget": 0.78, + "threshold": -0.9332055449485779, + "operator": "ge", + "actual_fraction": 0.78, + "metric": 0.665, + "rescued": 2, + "harmed": 391, + "missed_rescue": 0, + "avoided_harm": 137, + "selected_both_correct": 532, + "selected_both_wrong": 11 + }, + { + "budget": 0.79, + "threshold": -0.9344666600227356, + "operator": "ge", + "actual_fraction": 0.79, + "metric": 0.6616666666666666, + "rescued": 2, + "harmed": 395, + "missed_rescue": 0, + "avoided_harm": 133, + "selected_both_correct": 540, + "selected_both_wrong": 11 + }, + { + "budget": 0.8, + "threshold": -0.9362465143203735, + "operator": "ge", + "actual_fraction": 0.8, + "metric": 0.6583333333333333, + "rescued": 2, + "harmed": 399, + "missed_rescue": 0, + "avoided_harm": 129, + "selected_both_correct": 548, + "selected_both_wrong": 11 + }, + { + "budget": 0.81, + "threshold": -0.937842071056366, + "operator": "ge", + "actual_fraction": 0.81, + "metric": 0.6533333333333333, + "rescued": 2, + "harmed": 405, + "missed_rescue": 0, + "avoided_harm": 123, + "selected_both_correct": 554, + "selected_both_wrong": 11 + }, + { + "budget": 0.82, + "threshold": -0.9384457468986511, + "operator": "ge", + "actual_fraction": 0.8191666666666667, + "metric": 0.65, + "rescued": 2, + "harmed": 409, + "missed_rescue": 0, + "avoided_harm": 119, + "selected_both_correct": 561, + "selected_both_wrong": 11 + }, + { + "budget": 0.83, + "threshold": -0.9396296143531799, + "operator": "ge", + "actual_fraction": 0.83, + "metric": 0.6458333333333334, + "rescued": 2, + "harmed": 414, + "missed_rescue": 0, + "avoided_harm": 114, + "selected_both_correct": 569, + "selected_both_wrong": 11 + }, + { + "budget": 0.84, + "threshold": -0.9407488703727722, + "operator": "ge", + "actual_fraction": 0.84, + "metric": 0.6383333333333333, + "rescued": 2, + "harmed": 423, + "missed_rescue": 0, + "avoided_harm": 105, + "selected_both_correct": 572, + "selected_both_wrong": 11 + }, + { + "budget": 0.85, + "threshold": -0.942738950252533, + "operator": "ge", + "actual_fraction": 0.85, + "metric": 0.6358333333333334, + "rescued": 2, + "harmed": 426, + "missed_rescue": 0, + "avoided_harm": 102, + "selected_both_correct": 581, + "selected_both_wrong": 11 + }, + { + "budget": 0.86, + "threshold": -0.9438527226448059, + "operator": "gt", + "actual_fraction": 0.8591666666666666, + "metric": 0.6325, + "rescued": 2, + "harmed": 430, + "missed_rescue": 0, + "avoided_harm": 98, + "selected_both_correct": 588, + "selected_both_wrong": 11 + }, + { + "budget": 0.87, + "threshold": -0.9450756907463074, + "operator": "ge", + "actual_fraction": 0.87, + "metric": 0.6308333333333334, + "rescued": 2, + "harmed": 432, + "missed_rescue": 0, + "avoided_harm": 96, + "selected_both_correct": 599, + "selected_both_wrong": 11 + }, + { + "budget": 0.88, + "threshold": -0.9481598138809204, + "operator": "ge", + "actual_fraction": 0.88, + "metric": 0.625, + "rescued": 2, + "harmed": 439, + "missed_rescue": 0, + "avoided_harm": 89, + "selected_both_correct": 604, + "selected_both_wrong": 11 + }, + { + "budget": 0.89, + "threshold": -0.9500836730003357, + "operator": "ge", + "actual_fraction": 0.89, + "metric": 0.62, + "rescued": 2, + "harmed": 445, + "missed_rescue": 0, + "avoided_harm": 83, + "selected_both_correct": 610, + "selected_both_wrong": 11 + }, + { + "budget": 0.9, + "threshold": -0.9519118666648865, + "operator": "ge", + "actual_fraction": 0.9, + "metric": 0.6133333333333333, + "rescued": 2, + "harmed": 453, + "missed_rescue": 0, + "avoided_harm": 75, + "selected_both_correct": 614, + "selected_both_wrong": 11 + }, + { + "budget": 0.91, + "threshold": -0.9543898701667786, + "operator": "ge", + "actual_fraction": 0.91, + "metric": 0.6066666666666667, + "rescued": 2, + "harmed": 461, + "missed_rescue": 0, + "avoided_harm": 67, + "selected_both_correct": 618, + "selected_both_wrong": 11 + }, + { + "budget": 0.92, + "threshold": -0.9570822715759277, + "operator": "ge", + "actual_fraction": 0.92, + "metric": 0.6008333333333333, + "rescued": 2, + "harmed": 468, + "missed_rescue": 0, + "avoided_harm": 60, + "selected_both_correct": 623, + "selected_both_wrong": 11 + }, + { + "budget": 0.93, + "threshold": -0.9593446254730225, + "operator": "ge", + "actual_fraction": 0.93, + "metric": 0.5941666666666666, + "rescued": 2, + "harmed": 476, + "missed_rescue": 0, + "avoided_harm": 52, + "selected_both_correct": 627, + "selected_both_wrong": 11 + }, + { + "budget": 0.94, + "threshold": -0.9610995054244995, + "operator": "ge", + "actual_fraction": 0.94, + "metric": 0.5883333333333334, + "rescued": 2, + "harmed": 483, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 632, + "selected_both_wrong": 11 + }, + { + "budget": 0.95, + "threshold": -0.9635111093521118, + "operator": "ge", + "actual_fraction": 0.95, + "metric": 0.5825, + "rescued": 2, + "harmed": 490, + "missed_rescue": 0, + "avoided_harm": 38, + "selected_both_correct": 637, + "selected_both_wrong": 11 + }, + { + "budget": 0.96, + "threshold": -0.9648061990737915, + "operator": "ge", + "actual_fraction": 0.96, + "metric": 0.5775, + "rescued": 2, + "harmed": 496, + "missed_rescue": 0, + "avoided_harm": 32, + "selected_both_correct": 643, + "selected_both_wrong": 11 + }, + { + "budget": 0.97, + "threshold": -0.9674390554428101, + "operator": "ge", + "actual_fraction": 0.97, + "metric": 0.57, + "rescued": 2, + "harmed": 505, + "missed_rescue": 0, + "avoided_harm": 23, + "selected_both_correct": 646, + "selected_both_wrong": 11 + }, + { + "budget": 0.98, + "threshold": -0.9724470973014832, + "operator": "ge", + "actual_fraction": 0.98, + "metric": 0.5641666666666667, + "rescued": 2, + "harmed": 512, + "missed_rescue": 0, + "avoided_harm": 16, + "selected_both_correct": 651, + "selected_both_wrong": 11 + }, + { + "budget": 0.99, + "threshold": -0.9774226546287537, + "operator": "ge", + "actual_fraction": 0.99, + "metric": 0.5575, + "rescued": 2, + "harmed": 520, + "missed_rescue": 0, + "avoided_harm": 8, + "selected_both_correct": 655, + "selected_both_wrong": 11 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5508333333333333, + "rescued": 2, + "harmed": 528, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 659, + "selected_both_wrong": 11 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.9016223679668622, + "rescued": 0, + "harmed": 0, + "missed_rescue": 144, + "avoided_harm": 2192, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.48843976855278015, + "operator": "ge", + "actual_fraction": 0.08698653779772178, + "metric": 0.8878149810148429, + "rescued": 70, + "harmed": 150, + "missed_rescue": 74, + "avoided_harm": 2042, + "selected_both_correct": 85, + "selected_both_wrong": 199 + }, + { + "budget": 0.02, + "threshold": -0.6014261841773987, + "operator": "ge", + "actual_fraction": 0.13910942354159475, + "metric": 0.8683120469451157, + "rescued": 93, + "harmed": 286, + "missed_rescue": 51, + "avoided_harm": 1906, + "selected_both_correct": 168, + "selected_both_wrong": 259 + }, + { + "budget": 0.03, + "threshold": -0.6401472687721252, + "operator": "ge", + "actual_fraction": 0.15947531929582326, + "metric": 0.8596824301001036, + "rescued": 100, + "harmed": 343, + "missed_rescue": 44, + "avoided_harm": 1849, + "selected_both_correct": 203, + "selected_both_wrong": 278 + }, + { + "budget": 0.04, + "threshold": -0.6884650588035583, + "operator": "ge", + "actual_fraction": 0.1886434242319641, + "metric": 0.845702450811184, + "rescued": 106, + "harmed": 430, + "missed_rescue": 38, + "avoided_harm": 1762, + "selected_both_correct": 258, + "selected_both_wrong": 299 + }, + { + "budget": 0.05, + "threshold": -0.7208121418952942, + "operator": "ge", + "actual_fraction": 0.2143596824301001, + "metric": 0.8329306178805661, + "rescued": 111, + "harmed": 509, + "missed_rescue": 33, + "avoided_harm": 1683, + "selected_both_correct": 305, + "selected_both_wrong": 317 + }, + { + "budget": 0.06, + "threshold": -0.7578181624412537, + "operator": "ge", + "actual_fraction": 0.24525371073524335, + "metric": 0.819986192613048, + "rescued": 119, + "harmed": 592, + "missed_rescue": 25, + "avoided_harm": 1600, + "selected_both_correct": 384, + "selected_both_wrong": 326 + }, + { + "budget": 0.07, + "threshold": -0.7790470719337463, + "operator": "ge", + "actual_fraction": 0.26734552985847426, + "metric": 0.8087676907145323, + "rescued": 122, + "harmed": 660, + "missed_rescue": 22, + "avoided_harm": 1532, + "selected_both_correct": 430, + "selected_both_wrong": 337 + }, + { + "budget": 0.08, + "threshold": -0.7927228808403015, + "operator": "ge", + "actual_fraction": 0.2827062478425958, + "metric": 0.8015188125647221, + "rescued": 125, + "harmed": 705, + "missed_rescue": 19, + "avoided_harm": 1487, + "selected_both_correct": 468, + "selected_both_wrong": 340 + }, + { + "budget": 0.09, + "threshold": -0.8049252033233643, + "operator": "ge", + "actual_fraction": 0.2999654815326199, + "metric": 0.7951328960994132, + "rescued": 127, + "harmed": 744, + "missed_rescue": 17, + "avoided_harm": 1448, + "selected_both_correct": 521, + "selected_both_wrong": 346 + }, + { + "budget": 0.1, + "threshold": -0.80850750207901, + "operator": "ge", + "actual_fraction": 0.30531584397652745, + "metric": 0.793579565067311, + "rescued": 127, + "harmed": 753, + "missed_rescue": 17, + "avoided_harm": 1439, + "selected_both_correct": 541, + "selected_both_wrong": 348 + }, + { + "budget": 0.11, + "threshold": -0.819871723651886, + "operator": "ge", + "actual_fraction": 0.3258543320676562, + "metric": 0.7839143942008975, + "rescued": 129, + "harmed": 811, + "missed_rescue": 15, + "avoided_harm": 1381, + "selected_both_correct": 593, + "selected_both_wrong": 355 + }, + { + "budget": 0.12, + "threshold": -0.8310170769691467, + "operator": "ge", + "actual_fraction": 0.35053503624439075, + "metric": 0.7733862616499827, + "rescued": 132, + "harmed": 875, + "missed_rescue": 12, + "avoided_harm": 1317, + "selected_both_correct": 662, + "selected_both_wrong": 362 + }, + { + "budget": 0.13, + "threshold": -0.8376906514167786, + "operator": "ge", + "actual_fraction": 0.36796686227131514, + "metric": 0.7654470141525717, + "rescued": 133, + "harmed": 922, + "missed_rescue": 11, + "avoided_harm": 1270, + "selected_both_correct": 712, + "selected_both_wrong": 365 + }, + { + "budget": 0.14, + "threshold": -0.8419997692108154, + "operator": "ge", + "actual_fraction": 0.3798757335174318, + "metric": 0.7609596133931653, + "rescued": 133, + "harmed": 948, + "missed_rescue": 11, + "avoided_harm": 1244, + "selected_both_correct": 753, + "selected_both_wrong": 367 + }, + { + "budget": 0.15, + "threshold": -0.8447275161743164, + "operator": "ge", + "actual_fraction": 0.3867794269934415, + "metric": 0.7581981360027614, + "rescued": 133, + "harmed": 964, + "missed_rescue": 11, + "avoided_harm": 1228, + "selected_both_correct": 777, + "selected_both_wrong": 367 + }, + { + "budget": 0.16, + "threshold": -0.8506776690483093, + "operator": "ge", + "actual_fraction": 0.4050742147048671, + "metric": 0.7509492578529513, + "rescued": 135, + "harmed": 1008, + "missed_rescue": 9, + "avoided_harm": 1184, + "selected_both_correct": 835, + "selected_both_wrong": 369 + }, + { + "budget": 0.17, + "threshold": -0.8545639514923096, + "operator": "ge", + "actual_fraction": 0.4175008629616845, + "metric": 0.7462892647566448, + "rescued": 135, + "harmed": 1035, + "missed_rescue": 9, + "avoided_harm": 1157, + "selected_both_correct": 876, + "selected_both_wrong": 373 + }, + { + "budget": 0.18, + "threshold": -0.8578683137893677, + "operator": "ge", + "actual_fraction": 0.4290645495340007, + "metric": 0.74283741801864, + "rescued": 136, + "harmed": 1056, + "missed_rescue": 8, + "avoided_harm": 1136, + "selected_both_correct": 920, + "selected_both_wrong": 374 + }, + { + "budget": 0.19, + "threshold": -0.86090087890625, + "operator": "ge", + "actual_fraction": 0.43993786675871593, + "metric": 0.7399033482913359, + "rescued": 137, + "harmed": 1074, + "missed_rescue": 7, + "avoided_harm": 1118, + "selected_both_correct": 962, + "selected_both_wrong": 376 + }, + { + "budget": 0.2, + "threshold": -0.8640955090522766, + "operator": "ge", + "actual_fraction": 0.4520193303417328, + "metric": 0.7347255781843286, + "rescued": 138, + "harmed": 1105, + "missed_rescue": 6, + "avoided_harm": 1087, + "selected_both_correct": 1000, + "selected_both_wrong": 376 + }, + { + "budget": 0.21, + "threshold": -0.8664237260818481, + "operator": "ge", + "actual_fraction": 0.46047635484984467, + "metric": 0.731446323783224, + "rescued": 138, + "harmed": 1124, + "missed_rescue": 6, + "avoided_harm": 1068, + "selected_both_correct": 1028, + "selected_both_wrong": 378 + }, + { + "budget": 0.22, + "threshold": -0.8686786890029907, + "operator": "ge", + "actual_fraction": 0.46945115636865725, + "metric": 0.7279944770452192, + "rescued": 138, + "harmed": 1144, + "missed_rescue": 6, + "avoided_harm": 1048, + "selected_both_correct": 1060, + "selected_both_wrong": 378 + }, + { + "budget": 0.23, + "threshold": -0.8709638118743896, + "operator": "ge", + "actual_fraction": 0.48066965826717295, + "metric": 0.7241974456334139, + "rescued": 138, + "harmed": 1166, + "missed_rescue": 6, + "avoided_harm": 1026, + "selected_both_correct": 1102, + "selected_both_wrong": 379 + }, + { + "budget": 0.24, + "threshold": -0.8724074959754944, + "operator": "ge", + "actual_fraction": 0.48912668277528476, + "metric": 0.7210907835692095, + "rescued": 138, + "harmed": 1184, + "missed_rescue": 6, + "avoided_harm": 1008, + "selected_both_correct": 1132, + "selected_both_wrong": 380 + }, + { + "budget": 0.25, + "threshold": -0.8736594319343567, + "operator": "ge", + "actual_fraction": 0.49413186054539177, + "metric": 0.7200552295478081, + "rescued": 138, + "harmed": 1190, + "missed_rescue": 6, + "avoided_harm": 1002, + "selected_both_correct": 1154, + "selected_both_wrong": 381 + }, + { + "budget": 0.26, + "threshold": -0.8752554059028625, + "operator": "ge", + "actual_fraction": 0.5005177770107008, + "metric": 0.717811529168105, + "rescued": 138, + "harmed": 1203, + "missed_rescue": 6, + "avoided_harm": 989, + "selected_both_correct": 1178, + "selected_both_wrong": 381 + }, + { + "budget": 0.27, + "threshold": -0.8766024112701416, + "operator": "ge", + "actual_fraction": 0.5081118398343114, + "metric": 0.7153952364515016, + "rescued": 138, + "harmed": 1217, + "missed_rescue": 6, + "avoided_harm": 975, + "selected_both_correct": 1207, + "selected_both_wrong": 382 + }, + { + "budget": 0.28, + "threshold": -0.878311276435852, + "operator": "ge", + "actual_fraction": 0.5169140490162237, + "metric": 0.7131515360717984, + "rescued": 138, + "harmed": 1230, + "missed_rescue": 6, + "avoided_harm": 962, + "selected_both_correct": 1243, + "selected_both_wrong": 384 + }, + { + "budget": 0.29, + "threshold": -0.880103588104248, + "operator": "ge", + "actual_fraction": 0.5277873662409389, + "metric": 0.7105626510182947, + "rescued": 138, + "harmed": 1245, + "missed_rescue": 6, + "avoided_harm": 947, + "selected_both_correct": 1290, + "selected_both_wrong": 385 + }, + { + "budget": 0.3, + "threshold": -0.8824952244758606, + "operator": "ge", + "actual_fraction": 0.5419399378667588, + "metric": 0.704349326889886, + "rescued": 138, + "harmed": 1281, + "missed_rescue": 6, + "avoided_harm": 911, + "selected_both_correct": 1336, + "selected_both_wrong": 385 + }, + { + "budget": 0.31, + "threshold": -0.8831692337989807, + "operator": "ge", + "actual_fraction": 0.5453917846047636, + "metric": 0.7034863652053849, + "rescued": 138, + "harmed": 1286, + "missed_rescue": 6, + "avoided_harm": 906, + "selected_both_correct": 1350, + "selected_both_wrong": 386 + }, + { + "budget": 0.32, + "threshold": -0.8845130205154419, + "operator": "ge", + "actual_fraction": 0.5566102865032793, + "metric": 0.7003797031411805, + "rescued": 138, + "harmed": 1304, + "missed_rescue": 6, + "avoided_harm": 888, + "selected_both_correct": 1394, + "selected_both_wrong": 389 + }, + { + "budget": 0.33, + "threshold": -0.8855781555175781, + "operator": "ge", + "actual_fraction": 0.5633413876423887, + "metric": 0.6976182257507767, + "rescued": 138, + "harmed": 1320, + "missed_rescue": 6, + "avoided_harm": 872, + "selected_both_correct": 1417, + "selected_both_wrong": 389 + }, + { + "budget": 0.34, + "threshold": -0.8865919709205627, + "operator": "ge", + "actual_fraction": 0.5716258198136003, + "metric": 0.6958923023817742, + "rescued": 138, + "harmed": 1330, + "missed_rescue": 6, + "avoided_harm": 862, + "selected_both_correct": 1454, + "selected_both_wrong": 390 + }, + { + "budget": 0.35, + "threshold": -0.8877584338188171, + "operator": "ge", + "actual_fraction": 0.5819813600276148, + "metric": 0.6929582326544701, + "rescued": 139, + "harmed": 1348, + "missed_rescue": 5, + "avoided_harm": 844, + "selected_both_correct": 1494, + "selected_both_wrong": 391 + }, + { + "budget": 0.36, + "threshold": -0.8890330195426941, + "operator": "ge", + "actual_fraction": 0.5902657921988264, + "metric": 0.6910597169485675, + "rescued": 139, + "harmed": 1359, + "missed_rescue": 5, + "avoided_harm": 833, + "selected_both_correct": 1529, + "selected_both_wrong": 393 + }, + { + "budget": 0.37, + "threshold": -0.890850305557251, + "operator": "ge", + "actual_fraction": 0.6047635484984467, + "metric": 0.6853641698308595, + "rescued": 139, + "harmed": 1392, + "missed_rescue": 5, + "avoided_harm": 800, + "selected_both_correct": 1580, + "selected_both_wrong": 393 + }, + { + "budget": 0.38, + "threshold": -0.8916605710983276, + "operator": "ge", + "actual_fraction": 0.6113220573006558, + "metric": 0.6834656541249569, + "rescued": 139, + "harmed": 1403, + "missed_rescue": 5, + "avoided_harm": 789, + "selected_both_correct": 1607, + "selected_both_wrong": 393 + }, + { + "budget": 0.39, + "threshold": -0.8928589224815369, + "operator": "ge", + "actual_fraction": 0.6206420434932689, + "metric": 0.681394546082154, + "rescued": 140, + "harmed": 1416, + "missed_rescue": 4, + "avoided_harm": 776, + "selected_both_correct": 1647, + "selected_both_wrong": 393 + }, + { + "budget": 0.4, + "threshold": -0.8940943479537964, + "operator": "ge", + "actual_fraction": 0.6289264756644805, + "metric": 0.6789782533655506, + "rescued": 140, + "harmed": 1430, + "missed_rescue": 4, + "avoided_harm": 762, + "selected_both_correct": 1681, + "selected_both_wrong": 393 + }, + { + "budget": 0.41, + "threshold": -0.8949463963508606, + "operator": "ge", + "actual_fraction": 0.6370383154987919, + "metric": 0.6762167759751467, + "rescued": 140, + "harmed": 1446, + "missed_rescue": 4, + "avoided_harm": 746, + "selected_both_correct": 1711, + "selected_both_wrong": 394 + }, + { + "budget": 0.42, + "threshold": -0.8960229158401489, + "operator": "ge", + "actual_fraction": 0.6435968243010011, + "metric": 0.6739730755954436, + "rescued": 140, + "harmed": 1459, + "missed_rescue": 4, + "avoided_harm": 733, + "selected_both_correct": 1734, + "selected_both_wrong": 396 + }, + { + "budget": 0.43, + "threshold": -0.8966536521911621, + "operator": "ge", + "actual_fraction": 0.6491197790818087, + "metric": 0.6724197445633414, + "rescued": 140, + "harmed": 1468, + "missed_rescue": 4, + "avoided_harm": 724, + "selected_both_correct": 1757, + "selected_both_wrong": 396 + }, + { + "budget": 0.44, + "threshold": -0.8975363373756409, + "operator": "ge", + "actual_fraction": 0.6558508802209182, + "metric": 0.670003451846738, + "rescued": 140, + "harmed": 1482, + "missed_rescue": 4, + "avoided_harm": 710, + "selected_both_correct": 1782, + "selected_both_wrong": 396 + }, + { + "budget": 0.45, + "threshold": -0.8988122940063477, + "operator": "ge", + "actual_fraction": 0.6641353123921298, + "metric": 0.6681049361408353, + "rescued": 140, + "harmed": 1493, + "missed_rescue": 4, + "avoided_harm": 699, + "selected_both_correct": 1819, + "selected_both_wrong": 396 + }, + { + "budget": 0.46, + "threshold": -0.9000959396362305, + "operator": "ge", + "actual_fraction": 0.6739730755954436, + "metric": 0.6644804970659303, + "rescued": 140, + "harmed": 1514, + "missed_rescue": 4, + "avoided_harm": 678, + "selected_both_correct": 1855, + "selected_both_wrong": 396 + }, + { + "budget": 0.47, + "threshold": -0.9013051986694336, + "operator": "ge", + "actual_fraction": 0.6848463928201588, + "metric": 0.6596479116327235, + "rescued": 141, + "harmed": 1543, + "missed_rescue": 3, + "avoided_harm": 649, + "selected_both_correct": 1887, + "selected_both_wrong": 397 + }, + { + "budget": 0.48, + "threshold": -0.9029324054718018, + "operator": "ge", + "actual_fraction": 0.696410079392475, + "metric": 0.6575768035899207, + "rescued": 141, + "harmed": 1555, + "missed_rescue": 3, + "avoided_harm": 637, + "selected_both_correct": 1941, + "selected_both_wrong": 398 + }, + { + "budget": 0.49, + "threshold": -0.9036871790885925, + "operator": "ge", + "actual_fraction": 0.7026234035208837, + "metric": 0.6561960648947187, + "rescued": 141, + "harmed": 1563, + "missed_rescue": 3, + "avoided_harm": 629, + "selected_both_correct": 1967, + "selected_both_wrong": 400 + }, + { + "budget": 0.5, + "threshold": -0.9044621586799622, + "operator": "ge", + "actual_fraction": 0.7088367276492924, + "metric": 0.654297549188816, + "rescued": 141, + "harmed": 1574, + "missed_rescue": 3, + "avoided_harm": 618, + "selected_both_correct": 1992, + "selected_both_wrong": 400 + }, + { + "budget": 0.51, + "threshold": -0.9054575562477112, + "operator": "ge", + "actual_fraction": 0.717121159820504, + "metric": 0.6525716258198136, + "rescued": 141, + "harmed": 1584, + "missed_rescue": 3, + "avoided_harm": 608, + "selected_both_correct": 2030, + "selected_both_wrong": 400 + }, + { + "budget": 0.52, + "threshold": -0.906682550907135, + "operator": "ge", + "actual_fraction": 0.7262685536762168, + "metric": 0.64998274076631, + "rescued": 141, + "harmed": 1599, + "missed_rescue": 3, + "avoided_harm": 593, + "selected_both_correct": 2068, + "selected_both_wrong": 400 + }, + { + "budget": 0.53, + "threshold": -0.9072114825248718, + "operator": "ge", + "actual_fraction": 0.7300655850880221, + "metric": 0.6484294097342078, + "rescued": 141, + "harmed": 1608, + "missed_rescue": 3, + "avoided_harm": 584, + "selected_both_correct": 2080, + "selected_both_wrong": 401 + }, + { + "budget": 0.54, + "threshold": -0.9083563089370728, + "operator": "ge", + "actual_fraction": 0.7381774249223334, + "metric": 0.6460131170176044, + "rescued": 141, + "harmed": 1622, + "missed_rescue": 3, + "avoided_harm": 570, + "selected_both_correct": 2112, + "selected_both_wrong": 402 + }, + { + "budget": 0.55, + "threshold": -0.9095192551612854, + "operator": "ge", + "actual_fraction": 0.7487055574732482, + "metric": 0.6435968243010011, + "rescued": 141, + "harmed": 1636, + "missed_rescue": 3, + "avoided_harm": 556, + "selected_both_correct": 2157, + "selected_both_wrong": 404 + }, + { + "budget": 0.56, + "threshold": -0.9106979370117188, + "operator": "ge", + "actual_fraction": 0.759233690024163, + "metric": 0.6394546082153952, + "rescued": 141, + "harmed": 1660, + "missed_rescue": 3, + "avoided_harm": 532, + "selected_both_correct": 2193, + "selected_both_wrong": 405 + }, + { + "budget": 0.57, + "threshold": -0.9115362167358398, + "operator": "ge", + "actual_fraction": 0.767863306869175, + "metric": 0.6372109078356921, + "rescued": 141, + "harmed": 1673, + "missed_rescue": 3, + "avoided_harm": 519, + "selected_both_correct": 2230, + "selected_both_wrong": 405 + }, + { + "budget": 0.58, + "threshold": -0.9124338030815125, + "operator": "ge", + "actual_fraction": 0.7751121850189852, + "metric": 0.6339316534345875, + "rescued": 141, + "harmed": 1692, + "missed_rescue": 3, + "avoided_harm": 500, + "selected_both_correct": 2253, + "selected_both_wrong": 405 + }, + { + "budget": 0.59, + "threshold": -0.9134801626205444, + "operator": "ge", + "actual_fraction": 0.7830514325163963, + "metric": 0.6316879530548843, + "rescued": 141, + "harmed": 1705, + "missed_rescue": 3, + "avoided_harm": 487, + "selected_both_correct": 2286, + "selected_both_wrong": 405 + }, + { + "budget": 0.6, + "threshold": -0.9143747091293335, + "operator": "ge", + "actual_fraction": 0.7904729030031067, + "metric": 0.629271660338281, + "rescued": 141, + "harmed": 1719, + "missed_rescue": 3, + "avoided_harm": 473, + "selected_both_correct": 2314, + "selected_both_wrong": 406 + }, + { + "budget": 0.61, + "threshold": -0.915804386138916, + "operator": "ge", + "actual_fraction": 0.8010010355540214, + "metric": 0.6272005522954781, + "rescued": 141, + "harmed": 1731, + "missed_rescue": 3, + "avoided_harm": 461, + "selected_both_correct": 2361, + "selected_both_wrong": 408 + }, + { + "budget": 0.62, + "threshold": -0.9165146350860596, + "operator": "ge", + "actual_fraction": 0.8063513979979289, + "metric": 0.6253020365895754, + "rescued": 141, + "harmed": 1742, + "missed_rescue": 3, + "avoided_harm": 450, + "selected_both_correct": 2381, + "selected_both_wrong": 408 + }, + { + "budget": 0.63, + "threshold": -0.9172822833061218, + "operator": "ge", + "actual_fraction": 0.8110113910942354, + "metric": 0.6240938902312737, + "rescued": 141, + "harmed": 1749, + "missed_rescue": 3, + "avoided_harm": 443, + "selected_both_correct": 2401, + "selected_both_wrong": 408 + }, + { + "budget": 0.64, + "threshold": -0.9181071519851685, + "operator": "ge", + "actual_fraction": 0.8172247152226441, + "metric": 0.6220227821884708, + "rescued": 141, + "harmed": 1761, + "missed_rescue": 3, + "avoided_harm": 431, + "selected_both_correct": 2424, + "selected_both_wrong": 409 + }, + { + "budget": 0.65, + "threshold": -0.9192029237747192, + "operator": "ge", + "actual_fraction": 0.824301001035554, + "metric": 0.619951674145668, + "rescued": 141, + "harmed": 1773, + "missed_rescue": 3, + "avoided_harm": 419, + "selected_both_correct": 2452, + "selected_both_wrong": 410 + }, + { + "budget": 0.66, + "threshold": -0.9200543761253357, + "operator": "ge", + "actual_fraction": 0.829306178805661, + "metric": 0.618570935450466, + "rescued": 141, + "harmed": 1781, + "missed_rescue": 3, + "avoided_harm": 411, + "selected_both_correct": 2473, + "selected_both_wrong": 410 + }, + { + "budget": 0.67, + "threshold": -0.9208071231842041, + "operator": "ge", + "actual_fraction": 0.8348291335864687, + "metric": 0.6164998274076631, + "rescued": 141, + "harmed": 1793, + "missed_rescue": 3, + "avoided_harm": 399, + "selected_both_correct": 2493, + "selected_both_wrong": 410 + }, + { + "budget": 0.68, + "threshold": -0.9215553998947144, + "operator": "ge", + "actual_fraction": 0.840006903693476, + "metric": 0.6151190887124611, + "rescued": 141, + "harmed": 1801, + "missed_rescue": 3, + "avoided_harm": 391, + "selected_both_correct": 2515, + "selected_both_wrong": 410 + }, + { + "budget": 0.69, + "threshold": -0.9220835566520691, + "operator": "ge", + "actual_fraction": 0.8443217121159821, + "metric": 0.613565757680359, + "rescued": 141, + "harmed": 1810, + "missed_rescue": 3, + "avoided_harm": 382, + "selected_both_correct": 2530, + "selected_both_wrong": 411 + }, + { + "budget": 0.7, + "threshold": -0.9232237935066223, + "operator": "ge", + "actual_fraction": 0.852088367276493, + "metric": 0.6113220573006558, + "rescued": 141, + "harmed": 1823, + "missed_rescue": 3, + "avoided_harm": 369, + "selected_both_correct": 2560, + "selected_both_wrong": 413 + }, + { + "budget": 0.71, + "threshold": -0.9243364334106445, + "operator": "ge", + "actual_fraction": 0.8598550224370038, + "metric": 0.6082153952364515, + "rescued": 141, + "harmed": 1841, + "missed_rescue": 3, + "avoided_harm": 351, + "selected_both_correct": 2585, + "selected_both_wrong": 415 + }, + { + "budget": 0.72, + "threshold": -0.9252541065216064, + "operator": "ge", + "actual_fraction": 0.8641698308595098, + "metric": 0.6071798412150501, + "rescued": 141, + "harmed": 1847, + "missed_rescue": 3, + "avoided_harm": 345, + "selected_both_correct": 2604, + "selected_both_wrong": 415 + }, + { + "budget": 0.73, + "threshold": -0.925781786441803, + "operator": "ge", + "actual_fraction": 0.8681394546082154, + "metric": 0.6057991025198481, + "rescued": 141, + "harmed": 1855, + "missed_rescue": 3, + "avoided_harm": 337, + "selected_both_correct": 2619, + "selected_both_wrong": 415 + }, + { + "budget": 0.74, + "threshold": -0.9274963140487671, + "operator": "ge", + "actual_fraction": 0.877114256127028, + "metric": 0.6018294787711426, + "rescued": 141, + "harmed": 1878, + "missed_rescue": 3, + "avoided_harm": 314, + "selected_both_correct": 2648, + "selected_both_wrong": 415 + }, + { + "budget": 0.75, + "threshold": -0.9286423325538635, + "operator": "ge", + "actual_fraction": 0.8828098032447359, + "metric": 0.5995857783914395, + "rescued": 141, + "harmed": 1891, + "missed_rescue": 3, + "avoided_harm": 301, + "selected_both_correct": 2667, + "selected_both_wrong": 416 + }, + { + "budget": 0.76, + "threshold": -0.9301291704177856, + "operator": "ge", + "actual_fraction": 0.8900586813945461, + "metric": 0.5969968933379358, + "rescued": 141, + "harmed": 1906, + "missed_rescue": 3, + "avoided_harm": 286, + "selected_both_correct": 2693, + "selected_both_wrong": 417 + }, + { + "budget": 0.77, + "threshold": -0.9318777918815613, + "operator": "ge", + "actual_fraction": 0.8979979288919572, + "metric": 0.5945806006213324, + "rescued": 141, + "harmed": 1920, + "missed_rescue": 3, + "avoided_harm": 272, + "selected_both_correct": 2725, + "selected_both_wrong": 417 + }, + { + "budget": 0.78, + "threshold": -0.9332055449485779, + "operator": "ge", + "actual_fraction": 0.9052468070417673, + "metric": 0.5916465308940283, + "rescued": 141, + "harmed": 1937, + "missed_rescue": 3, + "avoided_harm": 255, + "selected_both_correct": 2750, + "selected_both_wrong": 417 + }, + { + "budget": 0.79, + "threshold": -0.9344666600227356, + "operator": "ge", + "actual_fraction": 0.9100793924749742, + "metric": 0.5888850535036244, + "rescued": 141, + "harmed": 1953, + "missed_rescue": 3, + "avoided_harm": 239, + "selected_both_correct": 2762, + "selected_both_wrong": 417 + }, + { + "budget": 0.8, + "threshold": -0.9362465143203735, + "operator": "ge", + "actual_fraction": 0.917155678287884, + "metric": 0.5862961684501208, + "rescued": 141, + "harmed": 1968, + "missed_rescue": 3, + "avoided_harm": 224, + "selected_both_correct": 2787, + "selected_both_wrong": 418 + }, + { + "budget": 0.81, + "threshold": -0.937842071056366, + "operator": "ge", + "actual_fraction": 0.9237141870900932, + "metric": 0.5830169140490162, + "rescued": 141, + "harmed": 1987, + "missed_rescue": 3, + "avoided_harm": 205, + "selected_both_correct": 2806, + "selected_both_wrong": 418 + }, + { + "budget": 0.82, + "threshold": -0.9384457468986511, + "operator": "ge", + "actual_fraction": 0.9249223334483949, + "metric": 0.5823265447014152, + "rescued": 141, + "harmed": 1991, + "missed_rescue": 3, + "avoided_harm": 201, + "selected_both_correct": 2808, + "selected_both_wrong": 419 + }, + { + "budget": 0.83, + "threshold": -0.9396296143531799, + "operator": "ge", + "actual_fraction": 0.927166033828098, + "metric": 0.5819813600276148, + "rescued": 142, + "harmed": 1994, + "missed_rescue": 2, + "avoided_harm": 198, + "selected_both_correct": 2817, + "selected_both_wrong": 419 + }, + { + "budget": 0.84, + "threshold": -0.9407488703727722, + "operator": "ge", + "actual_fraction": 0.9316534345875043, + "metric": 0.5802554366586123, + "rescued": 142, + "harmed": 2004, + "missed_rescue": 2, + "avoided_harm": 188, + "selected_both_correct": 2833, + "selected_both_wrong": 419 + }, + { + "budget": 0.85, + "threshold": -0.942738950252533, + "operator": "ge", + "actual_fraction": 0.9397652744218157, + "metric": 0.576458405246807, + "rescued": 142, + "harmed": 2026, + "missed_rescue": 2, + "avoided_harm": 166, + "selected_both_correct": 2858, + "selected_both_wrong": 419 + }, + { + "budget": 0.86, + "threshold": -0.9438527226448059, + "operator": "gt", + "actual_fraction": 0.9432171211598205, + "metric": 0.5749050742147048, + "rescued": 142, + "harmed": 2035, + "missed_rescue": 2, + "avoided_harm": 157, + "selected_both_correct": 2868, + "selected_both_wrong": 420 + }, + { + "budget": 0.87, + "threshold": -0.9450756907463074, + "operator": "ge", + "actual_fraction": 0.9456334138764239, + "metric": 0.5742147048671039, + "rescued": 142, + "harmed": 2039, + "missed_rescue": 2, + "avoided_harm": 153, + "selected_both_correct": 2878, + "selected_both_wrong": 420 + }, + { + "budget": 0.88, + "threshold": -0.9481598138809204, + "operator": "ge", + "actual_fraction": 0.9539178460476355, + "metric": 0.5704176734552986, + "rescued": 142, + "harmed": 2061, + "missed_rescue": 2, + "avoided_harm": 131, + "selected_both_correct": 2902, + "selected_both_wrong": 422 + }, + { + "budget": 0.89, + "threshold": -0.9500836730003357, + "operator": "ge", + "actual_fraction": 0.9606489471867449, + "metric": 0.5676561960648947, + "rescued": 142, + "harmed": 2077, + "missed_rescue": 2, + "avoided_harm": 115, + "selected_both_correct": 2924, + "selected_both_wrong": 423 + }, + { + "budget": 0.9, + "threshold": -0.9519118666648865, + "operator": "ge", + "actual_fraction": 0.9647911632723507, + "metric": 0.5662754573696928, + "rescued": 143, + "harmed": 2086, + "missed_rescue": 1, + "avoided_harm": 106, + "selected_both_correct": 2938, + "selected_both_wrong": 423 + }, + { + "budget": 0.91, + "threshold": -0.9543898701667786, + "operator": "ge", + "actual_fraction": 0.9697963410424577, + "metric": 0.5636865723161891, + "rescued": 143, + "harmed": 2101, + "missed_rescue": 1, + "avoided_harm": 91, + "selected_both_correct": 2951, + "selected_both_wrong": 424 + }, + { + "budget": 0.92, + "threshold": -0.9570822715759277, + "operator": "ge", + "actual_fraction": 0.9760096651708664, + "metric": 0.5605799102519848, + "rescued": 143, + "harmed": 2119, + "missed_rescue": 1, + "avoided_harm": 73, + "selected_both_correct": 2968, + "selected_both_wrong": 425 + }, + { + "budget": 0.93, + "threshold": -0.9593446254730225, + "operator": "ge", + "actual_fraction": 0.9801518812564722, + "metric": 0.5586813945460821, + "rescued": 143, + "harmed": 2130, + "missed_rescue": 1, + "avoided_harm": 62, + "selected_both_correct": 2981, + "selected_both_wrong": 425 + }, + { + "budget": 0.94, + "threshold": -0.9610995054244995, + "operator": "ge", + "actual_fraction": 0.9817052122885744, + "metric": 0.5579910251984812, + "rescued": 143, + "harmed": 2134, + "missed_rescue": 1, + "avoided_harm": 58, + "selected_both_correct": 2986, + "selected_both_wrong": 425 + }, + { + "budget": 0.95, + "threshold": -0.9635111093521118, + "operator": "ge", + "actual_fraction": 0.9853296513634795, + "metric": 0.556437694166379, + "rescued": 143, + "harmed": 2143, + "missed_rescue": 1, + "avoided_harm": 49, + "selected_both_correct": 2998, + "selected_both_wrong": 425 + }, + { + "budget": 0.96, + "threshold": -0.9648061990737915, + "operator": "ge", + "actual_fraction": 0.9875733517431826, + "metric": 0.5548843631342768, + "rescued": 143, + "harmed": 2152, + "missed_rescue": 1, + "avoided_harm": 40, + "selected_both_correct": 3002, + "selected_both_wrong": 425 + }, + { + "budget": 0.97, + "threshold": -0.9674390554428101, + "operator": "ge", + "actual_fraction": 0.9906800138073869, + "metric": 0.5533310321021747, + "rescued": 143, + "harmed": 2161, + "missed_rescue": 1, + "avoided_harm": 31, + "selected_both_correct": 3011, + "selected_both_wrong": 425 + }, + { + "budget": 0.98, + "threshold": -0.9724470973014832, + "operator": "ge", + "actual_fraction": 0.9937866758715913, + "metric": 0.5516051087331723, + "rescued": 143, + "harmed": 2171, + "missed_rescue": 1, + "avoided_harm": 21, + "selected_both_correct": 3018, + "selected_both_wrong": 426 + }, + { + "budget": 0.99, + "threshold": -0.9774226546287537, + "operator": "ge", + "actual_fraction": 0.9965481532619952, + "metric": 0.5502243700379703, + "rescued": 144, + "harmed": 2180, + "missed_rescue": 0, + "avoided_harm": 12, + "selected_both_correct": 3024, + "selected_both_wrong": 426 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5481532619951675, + "rescued": 144, + "harmed": 2192, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 3032, + "selected_both_wrong": 426 + } + ] + }, + "learned": { + "validation_area_0_25": 0.9862166666666667, + "validation_point": { + "metric": 0.9890000000000001, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.01, + "actual_fraction_std": 0.0, + "per_seed": [ + { + "seed": 2026, + "budget": 0.01, + "threshold": -0.001153669785708189, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9883333333333333, + "rescue": 0, + "harm": 1, + "missed_rescue": 2 + }, + { + "seed": 2027, + "budget": 0.01, + "threshold": -0.005822875536978245, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + }, + { + "seed": 2028, + "budget": 0.01, + "threshold": -0.01126081869006157, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + }, + { + "seed": 2029, + "budget": 0.01, + "threshold": -0.009776376187801361, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + }, + { + "seed": 2030, + "budget": 0.01, + "threshold": -0.00619181152433157, + "operator": "ge", + "actual_fraction": 0.01, + "metric": 0.9891666666666666, + "rescue": 0, + "harm": 0, + "missed_rescue": 2 + } + ], + "rescued": 0.0, + "harmed": 0.2, + "missed_rescue": 2.0, + "avoided_harm": 527.8 + }, + "test_point": { + "metric": 0.9014152571625818, + "metric_std": 0.00037417127008418876, + "actual_fraction": 0.008215395236451501, + "actual_fraction_std": 0.002613506210765189, + "per_seed": [ + { + "seed": 2026, + "budget": 0.01, + "threshold": -0.001153669785708189, + "operator": "ge", + "actual_fraction": 0.012426648256817397, + "metric": 0.9007594062823611, + "rescue": 0, + "harm": 5, + "missed_rescue": 144 + }, + { + "seed": 2027, + "budget": 0.01, + "threshold": -0.005822875536978245, + "operator": "ge", + "actual_fraction": 0.0075940628236106315, + "metric": 0.9016223679668622, + "rescue": 0, + "harm": 0, + "missed_rescue": 144 + }, + { + "seed": 2028, + "budget": 0.01, + "threshold": -0.01126081869006157, + "operator": "ge", + "actual_fraction": 0.005350362443907491, + "metric": 0.9016223679668622, + "rescue": 0, + "harm": 0, + "missed_rescue": 144 + }, + { + "seed": 2029, + "budget": 0.01, + "threshold": -0.009776376187801361, + "operator": "ge", + "actual_fraction": 0.007248878149810148, + "metric": 0.9016223679668622, + "rescue": 0, + "harm": 0, + "missed_rescue": 144 + }, + { + "seed": 2030, + "budget": 0.01, + "threshold": -0.00619181152433157, + "operator": "ge", + "actual_fraction": 0.00845702450811184, + "metric": 0.901449775629962, + "rescue": 0, + "harm": 1, + "missed_rescue": 144 + } + ], + "rescued": 0.0, + "harmed": 1.2, + "missed_rescue": 144.0, + "avoided_harm": 2190.8 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.9891666666666665, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.9890000000000001, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.01, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.9890000000000001, + "metric_std": 0.00037267799624997355, + "actual_fraction": 0.02, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.9886666666666667, + "metric_std": 0.0007453559924999036, + "actual_fraction": 0.03, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.9884999999999999, + "metric_std": 0.0006972166887783726, + "actual_fraction": 0.04, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.9883333333333333, + "metric_std": 0.000589255650988764, + "actual_fraction": 0.05, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.9883333333333333, + "metric_std": 0.000589255650988764, + "actual_fraction": 0.06, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.9883333333333333, + "metric_std": 0.000589255650988764, + "actual_fraction": 0.07, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.9883333333333333, + "metric_std": 0.000589255650988764, + "actual_fraction": 0.08, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.9879999999999999, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.09, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.9878333333333333, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.1, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.9878333333333333, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.11000000000000001, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.12, + "metric": 0.9878333333333333, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.12, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.9871666666666666, + "metric_std": 0.000456435464587649, + "actual_fraction": 0.13, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.9870000000000001, + "metric_std": 0.00045643546458764894, + "actual_fraction": 0.14, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.9870000000000001, + "metric_std": 0.00045643546458764894, + "actual_fraction": 0.15, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.9866666666666667, + "metric_std": 0.0, + "actual_fraction": 0.16, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.9863333333333333, + "metric_std": 0.000456435464587649, + "actual_fraction": 0.17, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.9856666666666666, + "metric_std": 0.0006972166887784124, + "actual_fraction": 0.18, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.9846666666666668, + "metric_std": 0.0011180339887499207, + "actual_fraction": 0.19, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.9836666666666666, + "metric_std": 0.001263812574008577, + "actual_fraction": 0.2, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.9828333333333333, + "metric_std": 0.001118033988749871, + "actual_fraction": 0.21000000000000002, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.9813333333333334, + "metric_std": 0.0011180339887499207, + "actual_fraction": 0.22000000000000003, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.23, + "metric": 0.9803333333333333, + "metric_std": 0.001263812574008621, + "actual_fraction": 0.23000000000000004, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.24, + "metric": 0.9796666666666667, + "metric_std": 0.0019184774055368834, + "actual_fraction": 0.24, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.9793333333333333, + "metric_std": 0.002312406154251934, + "actual_fraction": 0.25, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.26, + "metric": 0.9780000000000001, + "metric_std": 0.0024008100484813216, + "actual_fraction": 0.26, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.9764999999999999, + "metric_std": 0.0025276251480171943, + "actual_fraction": 0.27, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.9756666666666666, + "metric_std": 0.0027259045389660076, + "actual_fraction": 0.28, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.9738333333333333, + "metric_std": 0.002472066162365224, + "actual_fraction": 0.29, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.9716666666666667, + "metric_std": 0.00212459146399701, + "actual_fraction": 0.3, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.9700000000000001, + "metric_std": 0.0015590239111557857, + "actual_fraction": 0.3098333333333333, + "actual_fraction_std": 0.00037267799624997355 + }, + { + "budget": 0.32, + "metric": 0.9673333333333332, + "metric_std": 0.0014907119849998601, + "actual_fraction": 0.32, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.9661666666666667, + "metric_std": 0.0020916500663351973, + "actual_fraction": 0.33, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.9646666666666667, + "metric_std": 0.002400810048481322, + "actual_fraction": 0.34, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.9631666666666666, + "metric_std": 0.0014907119849998445, + "actual_fraction": 0.35, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.9613333333333334, + "metric_std": 0.0011180339887499207, + "actual_fraction": 0.36, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.9596666666666666, + "metric_std": 0.0013944333775567817, + "actual_fraction": 0.37, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.958, + "metric_std": 0.0009501461875826028, + "actual_fraction": 0.38, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.9561666666666667, + "metric_std": 0.0009501461875826369, + "actual_fraction": 0.39, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.9543333333333333, + "metric_std": 0.0013693063937628895, + "actual_fraction": 0.4, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.9521666666666668, + "metric_std": 0.000456435464587649, + "actual_fraction": 0.4091666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.42, + "metric": 0.9491666666666667, + "metric_std": 0.0013176156917368024, + "actual_fraction": 0.42000000000000004, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.946, + "metric_std": 0.002157029850924123, + "actual_fraction": 0.43, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.44, + "metric": 0.9423333333333332, + "metric_std": 0.002457980200625455, + "actual_fraction": 0.44000000000000006, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.45, + "metric": 0.9388333333333332, + "metric_std": 0.0037080992435478185, + "actual_fraction": 0.45, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.9355, + "metric_std": 0.00267446775598021, + "actual_fraction": 0.4600000000000001, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.47, + "metric": 0.932, + "metric_std": 0.002006932429798723, + "actual_fraction": 0.4699999999999999, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.48, + "metric": 0.9288333333333334, + "metric_std": 0.001918477405536893, + "actual_fraction": 0.48, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.9243333333333332, + "metric_std": 0.0019002923751652277, + "actual_fraction": 0.49000000000000005, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.9195, + "metric_std": 0.0017280367794439684, + "actual_fraction": 0.5, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.51, + "metric": 0.916, + "metric_std": 0.002157029850924123, + "actual_fraction": 0.51, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.9123333333333334, + "metric_std": 0.0029697549468676, + "actual_fraction": 0.52, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.9083333333333334, + "metric_std": 0.0038188130791298822, + "actual_fraction": 0.53, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.9038333333333333, + "metric_std": 0.003935239651039198, + "actual_fraction": 0.54, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.8983333333333334, + "metric_std": 0.004082482904638651, + "actual_fraction": 0.55, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.8936666666666666, + "metric_std": 0.003205897343611893, + "actual_fraction": 0.56, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.8879999999999999, + "metric_std": 0.0033644382063644242, + "actual_fraction": 0.5691666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.8818333333333331, + "metric_std": 0.0018066236157232185, + "actual_fraction": 0.58, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.8756666666666668, + "metric_std": 0.0016029486718059506, + "actual_fraction": 0.59, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.8703333333333333, + "metric_std": 0.0031512784001981797, + "actual_fraction": 0.6, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.8636666666666667, + "metric_std": 0.002674467755980205, + "actual_fraction": 0.61, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.8564999999999999, + "metric_std": 0.0029697549468675364, + "actual_fraction": 0.62, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.8515, + "metric_std": 0.002074983266331461, + "actual_fraction": 0.63, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.8448333333333334, + "metric_std": 0.0025276251480171982, + "actual_fraction": 0.64, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.8393333333333335, + "metric_std": 0.002312406154251934, + "actual_fraction": 0.65, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.8333333333333334, + "metric_std": 0.002568505834570412, + "actual_fraction": 0.66, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.8261666666666667, + "metric_std": 0.002922612986125008, + "actual_fraction": 0.67, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.8195, + "metric_std": 0.0026087459737497635, + "actual_fraction": 0.68, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.8141666666666667, + "metric_std": 0.0011785113019775672, + "actual_fraction": 0.6891666666666667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.8063333333333332, + "metric_std": 0.0022515426810177145, + "actual_fraction": 0.7, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.8006666666666666, + "metric_std": 0.002074983266331461, + "actual_fraction": 0.71, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.7929999999999999, + "metric_std": 0.0020916500663351796, + "actual_fraction": 0.72, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.786, + "metric_std": 0.0018066236157232188, + "actual_fraction": 0.73, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.7789999999999999, + "metric_std": 0.0019895560643855554, + "actual_fraction": 0.74, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.7708333333333334, + "metric_std": 0.0022821773229381942, + "actual_fraction": 0.75, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.762, + "metric_std": 0.0020916500663351843, + "actual_fraction": 0.76, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.7543333333333333, + "metric_std": 0.0019895560643855554, + "actual_fraction": 0.77, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.7475, + "metric_std": 0.0024295632895188837, + "actual_fraction": 0.78, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.7398333333333333, + "metric_std": 0.00231240615425195, + "actual_fraction": 0.79, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.7311666666666666, + "metric_std": 0.0024008100484813103, + "actual_fraction": 0.8, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.7225000000000001, + "metric_std": 0.0018633899812498305, + "actual_fraction": 0.8100000000000002, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.82, + "metric": 0.7151666666666666, + "metric_std": 0.0017078251276599291, + "actual_fraction": 0.8191666666666666, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.83, + "metric": 0.7060000000000001, + "metric_std": 0.002157029850924123, + "actual_fraction": 0.8299999999999998, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.6978333333333333, + "metric_std": 0.0017280367794439819, + "actual_fraction": 0.8400000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.6895, + "metric_std": 0.001513825177048732, + "actual_fraction": 0.85, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.6808333333333334, + "metric_std": 0.0014433756729740495, + "actual_fraction": 0.86, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.87, + "metric": 0.6719999999999999, + "metric_std": 0.0009501461875826369, + "actual_fraction": 0.8699999999999999, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.88, + "metric": 0.6628333333333334, + "metric_std": 0.0015138251770487443, + "actual_fraction": 0.8800000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.89, + "metric": 0.6535, + "metric_std": 0.0014907119849998942, + "actual_fraction": 0.89, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.6445, + "metric_std": 0.0012638125740086208, + "actual_fraction": 0.9, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.6359999999999999, + "metric_std": 0.0010865337342004283, + "actual_fraction": 0.9099999999999999, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.6265000000000001, + "metric_std": 0.0013693063937628895, + "actual_fraction": 0.9200000000000002, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.93, + "metric": 0.6175, + "metric_std": 0.000589255650988764, + "actual_fraction": 0.93, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.6081666666666666, + "metric_std": 0.0006972166887783593, + "actual_fraction": 0.9399999999999998, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.95, + "metric": 0.5983333333333334, + "metric_std": 0.0008333333333332971, + "actual_fraction": 0.95, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.589, + "metric_std": 0.0006972166887783593, + "actual_fraction": 0.96, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.5793333333333333, + "metric_std": 0.00037267799624992394, + "actual_fraction": 0.97, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.98, + "metric": 0.5698333333333333, + "metric_std": 0.0006972166887783593, + "actual_fraction": 0.9800000000000001, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.5603333333333333, + "metric_std": 0.00045643546458758817, + "actual_fraction": 0.99, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.5508333333333333, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.9016223679668622, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.9014152571625818, + "metric_std": 0.00037417127008418876, + "actual_fraction": 0.008215395236451501, + "actual_fraction_std": 0.002613506210765189 + }, + { + "budget": 0.02, + "metric": 0.9014152571625818, + "metric_std": 0.00037417127008418876, + "actual_fraction": 0.01995167414566793, + "actual_fraction_std": 0.002816482728458603 + }, + { + "budget": 0.03, + "metric": 0.9013117017604418, + "metric_std": 0.0003319879886377322, + "actual_fraction": 0.028753883327580256, + "actual_fraction_std": 0.0028921449866280155 + }, + { + "budget": 0.04, + "metric": 0.9012081463583017, + "metric_std": 0.0003135295499583091, + "actual_fraction": 0.036624093890231274, + "actual_fraction_std": 0.003199714837586805 + }, + { + "budget": 0.05, + "metric": 0.9011045909561617, + "metric_std": 0.00021138157946007892, + "actual_fraction": 0.046668967897825334, + "actual_fraction_std": 0.004633710512820112 + }, + { + "budget": 0.06, + "metric": 0.9009665170866414, + "metric_std": 0.000255995805560798, + "actual_fraction": 0.05491888160165689, + "actual_fraction_std": 0.0039041744736879385 + }, + { + "budget": 0.07, + "metric": 0.9007594062823611, + "metric_std": 0.0002989386965082897, + "actual_fraction": 0.06133931653434588, + "actual_fraction_std": 0.005799975689149572 + }, + { + "budget": 0.08, + "metric": 0.9004142216085607, + "metric_std": 0.0006102060590149771, + "actual_fraction": 0.07217811529168106, + "actual_fraction_std": 0.007436907466732381 + }, + { + "budget": 0.09, + "metric": 0.9001035554021402, + "metric_std": 0.0006150683355611243, + "actual_fraction": 0.08101484294097341, + "actual_fraction_std": 0.006619090510344644 + }, + { + "budget": 0.1, + "metric": 0.8997583707283396, + "metric_std": 0.0005776044366820986, + "actual_fraction": 0.08978253365550569, + "actual_fraction_std": 0.006614363472258347 + }, + { + "budget": 0.11, + "metric": 0.8994477045219191, + "metric_std": 0.0006411520752849909, + "actual_fraction": 0.09920607525025889, + "actual_fraction_std": 0.007277995992053029 + }, + { + "budget": 0.12, + "metric": 0.8989299275112185, + "metric_std": 0.0007582286032616198, + "actual_fraction": 0.11104590956161546, + "actual_fraction_std": 0.00650217434972065 + }, + { + "budget": 0.13, + "metric": 0.8984466689678978, + "metric_std": 0.0006639759772755134, + "actual_fraction": 0.12147048671039007, + "actual_fraction_std": 0.007468882279487798 + }, + { + "budget": 0.14, + "metric": 0.897894373489817, + "metric_std": 0.0006750986267037721, + "actual_fraction": 0.1315843976527442, + "actual_fraction_std": 0.007122022311653187 + }, + { + "budget": 0.15, + "metric": 0.897204004142216, + "metric_std": 0.0004332033275116013, + "actual_fraction": 0.14138764238867796, + "actual_fraction_std": 0.007628698434211337 + }, + { + "budget": 0.16, + "metric": 0.8966862271315152, + "metric_std": 0.0003781308646911647, + "actual_fraction": 0.14987918536416983, + "actual_fraction_std": 0.00575874189025859 + }, + { + "budget": 0.17, + "metric": 0.8959613393165343, + "metric_std": 0.0007053095181809402, + "actual_fraction": 0.16230583362098722, + "actual_fraction_std": 0.006553510803850919 + }, + { + "budget": 0.18, + "metric": 0.8953400069036934, + "metric_std": 0.0007965468249371029, + "actual_fraction": 0.17138419054193993, + "actual_fraction_std": 0.006343795633237521 + }, + { + "budget": 0.19, + "metric": 0.8944080082844323, + "metric_std": 0.0008749597415005943, + "actual_fraction": 0.18060062133241286, + "actual_fraction_std": 0.006228407649442422 + }, + { + "budget": 0.2, + "metric": 0.8937521574042112, + "metric_std": 0.0008150306491624303, + "actual_fraction": 0.18867794269934418, + "actual_fraction_std": 0.005818947503107896 + }, + { + "budget": 0.21, + "metric": 0.8926820849154298, + "metric_std": 0.0006946707558680845, + "actual_fraction": 0.19868829823955816, + "actual_fraction_std": 0.005382833754058268 + }, + { + "budget": 0.22, + "metric": 0.8918881601656887, + "metric_std": 0.00041565738967184535, + "actual_fraction": 0.20856057991025198, + "actual_fraction_std": 0.003546762024967106 + }, + { + "budget": 0.23, + "metric": 0.8906800138073869, + "metric_std": 0.0006526638039761655, + "actual_fraction": 0.2208146358301691, + "actual_fraction_std": 0.0037229571306001217 + }, + { + "budget": 0.24, + "metric": 0.8898515705902659, + "metric_std": 0.0007363040734450067, + "actual_fraction": 0.2280635139799793, + "actual_fraction_std": 0.0036112652409477407 + }, + { + "budget": 0.25, + "metric": 0.8891266827752847, + "metric_std": 0.0007776237712163767, + "actual_fraction": 0.23589920607525022, + "actual_fraction_std": 0.004355007768716635 + }, + { + "budget": 0.26, + "metric": 0.8880220918191233, + "metric_std": 0.000940588649874966, + "actual_fraction": 0.2449430445288229, + "actual_fraction_std": 0.006079318545579006 + }, + { + "budget": 0.27, + "metric": 0.887090093199862, + "metric_std": 0.0009640483288760213, + "actual_fraction": 0.2556782878840179, + "actual_fraction_std": 0.007550396944621299 + }, + { + "budget": 0.28, + "metric": 0.8856748360372799, + "metric_std": 0.0007383241158511308, + "actual_fraction": 0.26599930963065244, + "actual_fraction_std": 0.0051451625420648435 + }, + { + "budget": 0.29, + "metric": 0.8847083189506385, + "metric_std": 0.0012682894767603755, + "actual_fraction": 0.27366240938902314, + "actual_fraction_std": 0.006273678030311238 + }, + { + "budget": 0.3, + "metric": 0.8835692095270969, + "metric_std": 0.0006294298422095061, + "actual_fraction": 0.28412150500517774, + "actual_fraction_std": 0.005648807708478197 + }, + { + "budget": 0.31, + "metric": 0.882222989299275, + "metric_std": 0.0012600422677559026, + "actual_fraction": 0.29523645150155337, + "actual_fraction_std": 0.008229053486056313 + }, + { + "budget": 0.32, + "metric": 0.8806006213324128, + "metric_std": 0.0012529300250522632, + "actual_fraction": 0.30497065930272693, + "actual_fraction_std": 0.0074744638086063206 + }, + { + "budget": 0.33, + "metric": 0.8789092164307905, + "metric_std": 0.0018289902370367704, + "actual_fraction": 0.31656886434242326, + "actual_fraction_std": 0.00984185006766093 + }, + { + "budget": 0.34, + "metric": 0.877114256127028, + "metric_std": 0.002626013816828228, + "actual_fraction": 0.3273731446323783, + "actual_fraction_std": 0.010683591124392393 + }, + { + "budget": 0.35, + "metric": 0.8759751467034864, + "metric_std": 0.0023683542827634676, + "actual_fraction": 0.3372454263030722, + "actual_fraction_std": 0.008488665949159776 + }, + { + "budget": 0.36, + "metric": 0.8743872972040041, + "metric_std": 0.002438992930885693, + "actual_fraction": 0.3464963755609251, + "actual_fraction_std": 0.008354788406018637 + }, + { + "budget": 0.37, + "metric": 0.8724197445633415, + "metric_std": 0.0025052656105159147, + "actual_fraction": 0.35640317569899893, + "actual_fraction_std": 0.007154779885458797 + }, + { + "budget": 0.38, + "metric": 0.8702105626510181, + "metric_std": 0.0022503287556792178, + "actual_fraction": 0.3667587159130135, + "actual_fraction_std": 0.003777367730106878 + }, + { + "budget": 0.39, + "metric": 0.8687262685536762, + "metric_std": 0.002925427382382944, + "actual_fraction": 0.3773558853986883, + "actual_fraction_std": 0.006815321514066764 + }, + { + "budget": 0.4, + "metric": 0.8667241974456334, + "metric_std": 0.0033586841260168036, + "actual_fraction": 0.38709009319986193, + "actual_fraction_std": 0.007933054269728132 + }, + { + "budget": 0.41, + "metric": 0.8643079047290302, + "metric_std": 0.00397037407925736, + "actual_fraction": 0.398550224370038, + "actual_fraction_std": 0.007942436063339442 + }, + { + "budget": 0.42, + "metric": 0.8624093890231274, + "metric_std": 0.003921303971563751, + "actual_fraction": 0.40831895063859164, + "actual_fraction_std": 0.008399060266886109 + }, + { + "budget": 0.43, + "metric": 0.8609596133931653, + "metric_std": 0.0038565795334743794, + "actual_fraction": 0.415740421125302, + "actual_fraction_std": 0.010492279814524883 + }, + { + "budget": 0.44, + "metric": 0.8584397652744217, + "metric_std": 0.005143714953891715, + "actual_fraction": 0.42778736624093894, + "actual_fraction_std": 0.017681493172406335 + }, + { + "budget": 0.45, + "metric": 0.8555747324818779, + "metric_std": 0.00456864662783306, + "actual_fraction": 0.4394546082153952, + "actual_fraction_std": 0.01830756485193509 + }, + { + "budget": 0.46, + "metric": 0.8532619951674146, + "metric_std": 0.004329281904069918, + "actual_fraction": 0.45025888850535034, + "actual_fraction_std": 0.018893698177162244 + }, + { + "budget": 0.47, + "metric": 0.850742147048671, + "metric_std": 0.004233616497641031, + "actual_fraction": 0.4583362098722817, + "actual_fraction_std": 0.01904804909094023 + }, + { + "budget": 0.48, + "metric": 0.8476700034518467, + "metric_std": 0.0039801155043046104, + "actual_fraction": 0.47027959958577836, + "actual_fraction_std": 0.017137297949476285 + }, + { + "budget": 0.49, + "metric": 0.8454608215395236, + "metric_std": 0.002879241625287089, + "actual_fraction": 0.48142906454953405, + "actual_fraction_std": 0.015203611836959535 + }, + { + "budget": 0.5, + "metric": 0.843631342768381, + "metric_std": 0.0036551291999564026, + "actual_fraction": 0.4900931998619262, + "actual_fraction_std": 0.017323233084308682 + }, + { + "budget": 0.51, + "metric": 0.8411114946496376, + "metric_std": 0.004676902037228779, + "actual_fraction": 0.4993096306523991, + "actual_fraction_std": 0.019773312829136372 + }, + { + "budget": 0.52, + "metric": 0.8381429064549535, + "metric_std": 0.004575162099808409, + "actual_fraction": 0.5090783569209527, + "actual_fraction_std": 0.018529618105076976 + }, + { + "budget": 0.53, + "metric": 0.8346565412495686, + "metric_std": 0.004359451480951004, + "actual_fraction": 0.519986192613048, + "actual_fraction_std": 0.018869244798142993 + }, + { + "budget": 0.54, + "metric": 0.8317915084570245, + "metric_std": 0.004716541540185303, + "actual_fraction": 0.5301001035554022, + "actual_fraction_std": 0.01829177523865659 + }, + { + "budget": 0.55, + "metric": 0.8279944770452191, + "metric_std": 0.0048365903815007815, + "actual_fraction": 0.5444597859855023, + "actual_fraction_std": 0.016347874723462975 + }, + { + "budget": 0.56, + "metric": 0.824301001035554, + "metric_std": 0.004778345944048032, + "actual_fraction": 0.5575077666551607, + "actual_fraction_std": 0.014311329100839983 + }, + { + "budget": 0.57, + "metric": 0.8218501898515707, + "metric_std": 0.0047225376142844165, + "actual_fraction": 0.5676561960648947, + "actual_fraction_std": 0.014072891705681196 + }, + { + "budget": 0.58, + "metric": 0.819226786330687, + "metric_std": 0.0053581512672277445, + "actual_fraction": 0.577079737659648, + "actual_fraction_std": 0.01565537130173109 + }, + { + "budget": 0.59, + "metric": 0.8160510873317224, + "metric_std": 0.006188584583139514, + "actual_fraction": 0.5873662409389023, + "actual_fraction_std": 0.017435414211166244 + }, + { + "budget": 0.6, + "metric": 0.8130824991370382, + "metric_std": 0.006328516436761851, + "actual_fraction": 0.5954780807732136, + "actual_fraction_std": 0.019139626402918716 + }, + { + "budget": 0.61, + "metric": 0.8097342078011736, + "metric_std": 0.005999414893179516, + "actual_fraction": 0.6056610286503279, + "actual_fraction_std": 0.018162176256746328 + }, + { + "budget": 0.62, + "metric": 0.8052813255091473, + "metric_std": 0.005425552045880466, + "actual_fraction": 0.6181221953745253, + "actual_fraction_std": 0.01899676426800904 + }, + { + "budget": 0.63, + "metric": 0.8003451846738006, + "metric_std": 0.0048470492166470544, + "actual_fraction": 0.6297204004142216, + "actual_fraction_std": 0.017268982403824948 + }, + { + "budget": 0.64, + "metric": 0.795650673110114, + "metric_std": 0.0051935669738802474, + "actual_fraction": 0.6410424577148774, + "actual_fraction_std": 0.019763669000719568 + }, + { + "budget": 0.65, + "metric": 0.7902657921988263, + "metric_std": 0.006485890506275319, + "actual_fraction": 0.6530548843631342, + "actual_fraction_std": 0.019077270498233476 + }, + { + "budget": 0.66, + "metric": 0.7855367621677598, + "metric_std": 0.006066074361474203, + "actual_fraction": 0.6633413876423887, + "actual_fraction_std": 0.0185833944910152 + }, + { + "budget": 0.67, + "metric": 0.7803935105281326, + "metric_std": 0.005324690378361411, + "actual_fraction": 0.6744908526061443, + "actual_fraction_std": 0.015159169215047966 + }, + { + "budget": 0.68, + "metric": 0.7762167759751466, + "metric_std": 0.006569400327235919, + "actual_fraction": 0.6828098032447361, + "actual_fraction_std": 0.016666737165055824 + }, + { + "budget": 0.69, + "metric": 0.7704176734552985, + "metric_std": 0.008305440225662878, + "actual_fraction": 0.6936831204694511, + "actual_fraction_std": 0.018179389340431912 + }, + { + "budget": 0.7, + "metric": 0.7646530894028305, + "metric_std": 0.008434628294915786, + "actual_fraction": 0.7039351052813255, + "actual_fraction_std": 0.01790166123539544 + }, + { + "budget": 0.71, + "metric": 0.7595788746979635, + "metric_std": 0.009933434961357715, + "actual_fraction": 0.7131860545391785, + "actual_fraction_std": 0.018779045613927917 + }, + { + "budget": 0.72, + "metric": 0.7533310321021747, + "metric_std": 0.009045386382139283, + "actual_fraction": 0.7246461857093545, + "actual_fraction_std": 0.017357160914912774 + }, + { + "budget": 0.73, + "metric": 0.7480842250604074, + "metric_std": 0.008739889168094256, + "actual_fraction": 0.7341732827062478, + "actual_fraction_std": 0.015530625319279394 + }, + { + "budget": 0.74, + "metric": 0.7421125302036591, + "metric_std": 0.008523161118806319, + "actual_fraction": 0.7435277873662409, + "actual_fraction_std": 0.015112920627625046 + }, + { + "budget": 0.75, + "metric": 0.7357611322057301, + "metric_std": 0.007954241368175996, + "actual_fraction": 0.7554021401449775, + "actual_fraction_std": 0.015551230410328 + }, + { + "budget": 0.76, + "metric": 0.7294442526751812, + "metric_std": 0.008570387002174268, + "actual_fraction": 0.7644804970659302, + "actual_fraction_std": 0.016819829436670243 + }, + { + "budget": 0.77, + "metric": 0.7237832240248533, + "metric_std": 0.006813135785036478, + "actual_fraction": 0.7737314463237832, + "actual_fraction_std": 0.014343347397558735 + }, + { + "budget": 0.78, + "metric": 0.7173627890921643, + "metric_std": 0.005269863317739756, + "actual_fraction": 0.784363134276838, + "actual_fraction_std": 0.010256981217573957 + }, + { + "budget": 0.79, + "metric": 0.7109768726268555, + "metric_std": 0.005743202892992647, + "actual_fraction": 0.7941318605453919, + "actual_fraction_std": 0.010165673992111721 + }, + { + "budget": 0.8, + "metric": 0.7040731791508457, + "metric_std": 0.005032775971331379, + "actual_fraction": 0.8036244390749051, + "actual_fraction_std": 0.009937482480053344 + }, + { + "budget": 0.81, + "metric": 0.6962374870555748, + "metric_std": 0.005323851161611066, + "actual_fraction": 0.8149810148429409, + "actual_fraction_std": 0.009635847326343004 + }, + { + "budget": 0.82, + "metric": 0.6909906800138075, + "metric_std": 0.005499442963834028, + "actual_fraction": 0.822851225405592, + "actual_fraction_std": 0.010034578432202665 + }, + { + "budget": 0.83, + "metric": 0.6820503969623749, + "metric_std": 0.005228436947330567, + "actual_fraction": 0.8353814290645495, + "actual_fraction_std": 0.010786261623249737 + }, + { + "budget": 0.84, + "metric": 0.6753883327580256, + "metric_std": 0.007004782839404359, + "actual_fraction": 0.8446668967897825, + "actual_fraction_std": 0.011167271037673803 + }, + { + "budget": 0.85, + "metric": 0.6691750086296169, + "metric_std": 0.007571867964524642, + "actual_fraction": 0.852916810493614, + "actual_fraction_std": 0.011140832004688246 + }, + { + "budget": 0.86, + "metric": 0.6630652399033483, + "metric_std": 0.008713263916602473, + "actual_fraction": 0.8619261304798067, + "actual_fraction_std": 0.012164393123121293 + }, + { + "budget": 0.87, + "metric": 0.6562651018294787, + "metric_std": 0.009226509326481957, + "actual_fraction": 0.8711770797376597, + "actual_fraction_std": 0.013185873812979854 + }, + { + "budget": 0.88, + "metric": 0.6490852606144287, + "metric_std": 0.00893938339359951, + "actual_fraction": 0.8805315843976528, + "actual_fraction_std": 0.012472946201268811 + }, + { + "budget": 0.89, + "metric": 0.6410079392474973, + "metric_std": 0.008147564896045659, + "actual_fraction": 0.8911632723507077, + "actual_fraction_std": 0.011082659138322607 + }, + { + "budget": 0.9, + "metric": 0.6311356575768036, + "metric_std": 0.005840663405757733, + "actual_fraction": 0.9045564376941663, + "actual_fraction_std": 0.010114706131758855 + }, + { + "budget": 0.91, + "metric": 0.6218501898515706, + "metric_std": 0.005213602853639368, + "actual_fraction": 0.9163272350707627, + "actual_fraction_std": 0.008416774590919694 + }, + { + "budget": 0.92, + "metric": 0.6123230928546772, + "metric_std": 0.006962127572235393, + "actual_fraction": 0.9274767000345184, + "actual_fraction_std": 0.009799384957288418 + }, + { + "budget": 0.93, + "metric": 0.6052813255091474, + "metric_std": 0.0032610361801331035, + "actual_fraction": 0.9355195029340697, + "actual_fraction_std": 0.005643795806841778 + }, + { + "budget": 0.94, + "metric": 0.5947531929582326, + "metric_std": 0.001854865831912728, + "actual_fraction": 0.9480842250604071, + "actual_fraction_std": 0.003331970943723418 + }, + { + "budget": 0.95, + "metric": 0.5881946841560234, + "metric_std": 0.0029918770837693695, + "actual_fraction": 0.9561960648947186, + "actual_fraction_std": 0.004395856007953678 + }, + { + "budget": 0.96, + "metric": 0.5801518812564723, + "metric_std": 0.0043843209796375245, + "actual_fraction": 0.965136347946151, + "actual_fraction_std": 0.00533363380751206 + }, + { + "budget": 0.97, + "metric": 0.5723161891612013, + "metric_std": 0.004211748133211066, + "actual_fraction": 0.9738695201933034, + "actual_fraction_std": 0.0048288856096965135 + }, + { + "budget": 0.98, + "metric": 0.565723161891612, + "metric_std": 0.0019156971937192649, + "actual_fraction": 0.9809458060062134, + "actual_fraction_std": 0.0022556174321994413 + }, + { + "budget": 0.99, + "metric": 0.5581290990680013, + "metric_std": 0.002170824671526452, + "actual_fraction": 0.9892302381774251, + "actual_fraction_std": 0.0023933772096581003 + }, + { + "budget": 1.0, + "metric": 0.5481532619951675, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 659, + "rescuable": 2, + "harmful": 528, + "both_wrong": 11, + "unit": "sample", + "total": 1200 + }, + "test": { + "both_correct": 3032, + "rescuable": 144, + "harmful": 2192, + "both_wrong": 426, + "unit": "sample", + "total": 5794 + } + }, + "small": { + "validation": 0.9891666666666666, + "test": 0.9016223679668622 + }, + "large": { + "validation": 0.5508333333333333, + "test": 0.5481532619951675 + }, + "learned_minus_confidence_test": 0.013600276147738843, + "learned_minus_small_test": -0.0002071108042804859, + "learned_minus_large_test": 0.3532619951674143, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_glsim_Qwen3.8" + }, + { + "key": [ + "grefcoco", + "groundingdino", + "MiniCPM" + ], + "expert_display": "GroundingDINO", + "run": "five_seed_groundingdino_MiniCPM_four_difference", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "max_score", + "mean_score", + "query_count", + "detection_count_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/groundingdino/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 99, + "validation_area_0_25": 0.5566729917773561, + "validation_point": { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6372197624569541, + "rescued": 2529, + "harmed": 1438, + "missed_rescue": 60, + "avoided_harm": 3, + "selected_both_correct": 6534, + "selected_both_wrong": 3585 + }, + "test_point": { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9873805405098829, + "metric": 0.4968380455434875, + "rescued": 10590, + "harmed": 2481, + "missed_rescue": 133, + "avoided_harm": 19, + "selected_both_correct": 6845, + "selected_both_wrong": 14902 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.5605453650994447, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2589, + "avoided_harm": 1441, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.1584138125181198, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.5602642490688031, + "rescued": 0, + "harmed": 4, + "missed_rescue": 2589, + "avoided_harm": 1437, + "selected_both_correct": 138, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.21403978765010834, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.5599128540305011, + "rescued": 0, + "harmed": 9, + "missed_rescue": 2589, + "avoided_harm": 1432, + "selected_both_correct": 275, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.24253316223621368, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.559561458992199, + "rescued": 0, + "harmed": 14, + "missed_rescue": 2589, + "avoided_harm": 1427, + "selected_both_correct": 409, + "selected_both_wrong": 3 + }, + { + "budget": 0.04, + "threshold": -0.2599165141582489, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.5589289479232553, + "rescued": 0, + "harmed": 23, + "missed_rescue": 2589, + "avoided_harm": 1418, + "selected_both_correct": 542, + "selected_both_wrong": 4 + }, + { + "budget": 0.05, + "threshold": -0.2725968658924103, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.5583667158619721, + "rescued": 1, + "harmed": 32, + "missed_rescue": 2588, + "avoided_harm": 1409, + "selected_both_correct": 666, + "selected_both_wrong": 12 + }, + { + "budget": 0.06, + "threshold": -0.2842792272567749, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.5578747628083491, + "rescued": 2, + "harmed": 40, + "missed_rescue": 2587, + "avoided_harm": 1401, + "selected_both_correct": 794, + "selected_both_wrong": 17 + }, + { + "budget": 0.07, + "threshold": -0.2935555577278137, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.5569611357087638, + "rescued": 3, + "harmed": 54, + "missed_rescue": 2586, + "avoided_harm": 1387, + "selected_both_correct": 911, + "selected_both_wrong": 28 + }, + { + "budget": 0.08, + "threshold": -0.30114513635635376, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.5566800196781222, + "rescued": 8, + "harmed": 63, + "missed_rescue": 2581, + "avoided_harm": 1378, + "selected_both_correct": 1032, + "selected_both_wrong": 35 + }, + { + "budget": 0.09, + "threshold": -0.308601438999176, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.5562583456321597, + "rescued": 14, + "harmed": 75, + "missed_rescue": 2575, + "avoided_harm": 1366, + "selected_both_correct": 1140, + "selected_both_wrong": 51 + }, + { + "budget": 0.1, + "threshold": -0.3146189749240875, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.555625834563216, + "rescued": 18, + "harmed": 88, + "missed_rescue": 2571, + "avoided_harm": 1353, + "selected_both_correct": 1249, + "selected_both_wrong": 67 + }, + { + "budget": 0.11, + "threshold": -0.32043084502220154, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.555625834563216, + "rescued": 24, + "harmed": 94, + "missed_rescue": 2565, + "avoided_harm": 1347, + "selected_both_correct": 1360, + "selected_both_wrong": 87 + }, + { + "budget": 0.12, + "threshold": -0.32644838094711304, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.5553447185325743, + "rescued": 35, + "harmed": 109, + "missed_rescue": 2554, + "avoided_harm": 1332, + "selected_both_correct": 1461, + "selected_both_wrong": 102 + }, + { + "budget": 0.13, + "threshold": -0.33121857047080994, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.5553447185325743, + "rescued": 46, + "harmed": 120, + "missed_rescue": 2543, + "avoided_harm": 1321, + "selected_both_correct": 1562, + "selected_both_wrong": 121 + }, + { + "budget": 0.14, + "threshold": -0.33570271730422974, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.5554149975402347, + "rescued": 57, + "harmed": 130, + "missed_rescue": 2532, + "avoided_harm": 1311, + "selected_both_correct": 1662, + "selected_both_wrong": 143 + }, + { + "budget": 0.15, + "threshold": -0.3407592475414276, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.5549933234942722, + "rescued": 67, + "harmed": 146, + "missed_rescue": 2522, + "avoided_harm": 1295, + "selected_both_correct": 1754, + "selected_both_wrong": 167 + }, + { + "budget": 0.16, + "threshold": -0.34456855058670044, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.5546419284559702, + "rescued": 81, + "harmed": 165, + "missed_rescue": 2508, + "avoided_harm": 1276, + "selected_both_correct": 1844, + "selected_both_wrong": 186 + }, + { + "budget": 0.17, + "threshold": -0.3487326204776764, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.5551338815095931, + "rescued": 99, + "harmed": 176, + "missed_rescue": 2490, + "avoided_harm": 1265, + "selected_both_correct": 1931, + "selected_both_wrong": 212 + }, + { + "budget": 0.18, + "threshold": -0.35280776023864746, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.5553447185325743, + "rescued": 114, + "harmed": 188, + "missed_rescue": 2475, + "avoided_harm": 1253, + "selected_both_correct": 2017, + "selected_both_wrong": 242 + }, + { + "budget": 0.19, + "threshold": -0.3564819097518921, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.555274439524914, + "rescued": 126, + "harmed": 201, + "missed_rescue": 2463, + "avoided_harm": 1240, + "selected_both_correct": 2103, + "selected_both_wrong": 273 + }, + { + "budget": 0.2, + "threshold": -0.35992002487182617, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.5554852765478951, + "rescued": 142, + "harmed": 214, + "missed_rescue": 2447, + "avoided_harm": 1227, + "selected_both_correct": 2186, + "selected_both_wrong": 303 + }, + { + "budget": 0.21, + "threshold": -0.36340460181236267, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.5566097406704618, + "rescued": 166, + "harmed": 222, + "missed_rescue": 2423, + "avoided_harm": 1219, + "selected_both_correct": 2269, + "selected_both_wrong": 331 + }, + { + "budget": 0.22, + "threshold": -0.3674520254135132, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.5566800196781222, + "rescued": 182, + "harmed": 237, + "missed_rescue": 2407, + "avoided_harm": 1204, + "selected_both_correct": 2349, + "selected_both_wrong": 362 + }, + { + "budget": 0.23, + "threshold": -0.37108901143074036, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.556820577693443, + "rescued": 197, + "harmed": 250, + "missed_rescue": 2392, + "avoided_harm": 1191, + "selected_both_correct": 2436, + "selected_both_wrong": 389 + }, + { + "budget": 0.24, + "threshold": -0.3745429813861847, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.5575233677700471, + "rescued": 219, + "harmed": 262, + "missed_rescue": 2370, + "avoided_harm": 1179, + "selected_both_correct": 2514, + "selected_both_wrong": 419 + }, + { + "budget": 0.25, + "threshold": -0.3773327171802521, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.5572422517394054, + "rescued": 233, + "harmed": 280, + "missed_rescue": 2356, + "avoided_harm": 1161, + "selected_both_correct": 2590, + "selected_both_wrong": 454 + }, + { + "budget": 0.26, + "threshold": -0.3803459703922272, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.5584369948696324, + "rescued": 260, + "harmed": 290, + "missed_rescue": 2329, + "avoided_harm": 1151, + "selected_both_correct": 2663, + "selected_both_wrong": 486 + }, + { + "budget": 0.27, + "threshold": -0.38361650705337524, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.5585775528849533, + "rescued": 278, + "harmed": 306, + "missed_rescue": 2311, + "avoided_harm": 1135, + "selected_both_correct": 2737, + "selected_both_wrong": 520 + }, + { + "budget": 0.28, + "threshold": -0.38652583956718445, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.5589992269309157, + "rescued": 298, + "harmed": 320, + "missed_rescue": 2291, + "avoided_harm": 1121, + "selected_both_correct": 2802, + "selected_both_wrong": 564 + }, + { + "budget": 0.29, + "threshold": -0.38952407240867615, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.5599831330381615, + "rescued": 323, + "harmed": 331, + "missed_rescue": 2266, + "avoided_harm": 1110, + "selected_both_correct": 2873, + "selected_both_wrong": 599 + }, + { + "budget": 0.3, + "threshold": -0.39262744784355164, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.5608264811300865, + "rescued": 348, + "harmed": 344, + "missed_rescue": 2241, + "avoided_harm": 1097, + "selected_both_correct": 2941, + "selected_both_wrong": 635 + }, + { + "budget": 0.31, + "threshold": -0.3954668939113617, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.5613184341837093, + "rescued": 370, + "harmed": 359, + "missed_rescue": 2219, + "avoided_harm": 1082, + "selected_both_correct": 3011, + "selected_both_wrong": 670 + }, + { + "budget": 0.32, + "threshold": -0.3986983895301819, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.5613887131913697, + "rescued": 394, + "harmed": 382, + "missed_rescue": 2195, + "avoided_harm": 1059, + "selected_both_correct": 3076, + "selected_both_wrong": 701 + }, + { + "budget": 0.33, + "threshold": -0.4014112949371338, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.5616698292220114, + "rescued": 415, + "harmed": 399, + "missed_rescue": 2174, + "avoided_harm": 1042, + "selected_both_correct": 3145, + "selected_both_wrong": 736 + }, + { + "budget": 0.34, + "threshold": -0.4042409658432007, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.562302340290955, + "rescued": 437, + "harmed": 412, + "missed_rescue": 2152, + "avoided_harm": 1029, + "selected_both_correct": 3220, + "selected_both_wrong": 768 + }, + { + "budget": 0.35, + "threshold": -0.40724706649780273, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.5628645723522384, + "rescued": 461, + "harmed": 428, + "missed_rescue": 2128, + "avoided_harm": 1013, + "selected_both_correct": 3288, + "selected_both_wrong": 803 + }, + { + "budget": 0.36, + "threshold": -0.40968337655067444, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.5624428983062759, + "rescued": 478, + "harmed": 451, + "missed_rescue": 2111, + "avoided_harm": 990, + "selected_both_correct": 3361, + "selected_both_wrong": 832 + }, + { + "budget": 0.37, + "threshold": -0.4123799502849579, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.5635673624288425, + "rescued": 508, + "harmed": 465, + "missed_rescue": 2081, + "avoided_harm": 976, + "selected_both_correct": 3425, + "selected_both_wrong": 866 + }, + { + "budget": 0.38, + "threshold": -0.41514647006988525, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.5650432215897112, + "rescued": 538, + "harmed": 474, + "missed_rescue": 2051, + "avoided_harm": 967, + "selected_both_correct": 3487, + "selected_both_wrong": 908 + }, + { + "budget": 0.39, + "threshold": -0.4179389476776123, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.565183779605032, + "rescued": 557, + "harmed": 491, + "missed_rescue": 2032, + "avoided_harm": 950, + "selected_both_correct": 3556, + "selected_both_wrong": 945 + }, + { + "budget": 0.4, + "threshold": -0.42099592089653015, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.5658865696816361, + "rescued": 580, + "harmed": 504, + "missed_rescue": 2009, + "avoided_harm": 937, + "selected_both_correct": 3625, + "selected_both_wrong": 982 + }, + { + "budget": 0.41, + "threshold": -0.4243226647377014, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.5672218708271839, + "rescued": 614, + "harmed": 519, + "missed_rescue": 1975, + "avoided_harm": 922, + "selected_both_correct": 3695, + "selected_both_wrong": 1005 + }, + { + "budget": 0.42, + "threshold": -0.4275209903717041, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.5686977299880526, + "rescued": 645, + "harmed": 529, + "missed_rescue": 1944, + "avoided_harm": 912, + "selected_both_correct": 3759, + "selected_both_wrong": 1043 + }, + { + "budget": 0.43, + "threshold": -0.4307159185409546, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.5695410780799776, + "rescued": 672, + "harmed": 544, + "missed_rescue": 1917, + "avoided_harm": 897, + "selected_both_correct": 3828, + "selected_both_wrong": 1074 + }, + { + "budget": 0.44, + "threshold": -0.4336964190006256, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.5705952631948837, + "rescued": 702, + "harmed": 559, + "missed_rescue": 1887, + "avoided_harm": 882, + "selected_both_correct": 3892, + "selected_both_wrong": 1107 + }, + { + "budget": 0.45, + "threshold": -0.4373021125793457, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.5709466582331858, + "rescued": 725, + "harmed": 577, + "missed_rescue": 1864, + "avoided_harm": 864, + "selected_both_correct": 3955, + "selected_both_wrong": 1146 + }, + { + "budget": 0.46, + "threshold": -0.4404309391975403, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.5722116803710732, + "rescued": 759, + "harmed": 593, + "missed_rescue": 1830, + "avoided_harm": 848, + "selected_both_correct": 4019, + "selected_both_wrong": 1174 + }, + { + "budget": 0.47, + "threshold": -0.44344601035118103, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.5731955864783189, + "rescued": 784, + "harmed": 604, + "missed_rescue": 1805, + "avoided_harm": 837, + "selected_both_correct": 4091, + "selected_both_wrong": 1208 + }, + { + "budget": 0.48, + "threshold": -0.4465356171131134, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.5732658654859794, + "rescued": 805, + "harmed": 624, + "missed_rescue": 1784, + "avoided_harm": 817, + "selected_both_correct": 4159, + "selected_both_wrong": 1241 + }, + { + "budget": 0.49, + "threshold": -0.44965147972106934, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.5741794925855647, + "rescued": 836, + "harmed": 642, + "missed_rescue": 1753, + "avoided_harm": 799, + "selected_both_correct": 4222, + "selected_both_wrong": 1272 + }, + { + "budget": 0.5, + "threshold": -0.453143447637558, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.574741724646848, + "rescued": 860, + "harmed": 658, + "missed_rescue": 1729, + "avoided_harm": 783, + "selected_both_correct": 4278, + "selected_both_wrong": 1318 + }, + { + "budget": 0.51, + "threshold": -0.4565357565879822, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.5757256307540938, + "rescued": 892, + "harmed": 676, + "missed_rescue": 1697, + "avoided_harm": 765, + "selected_both_correct": 4336, + "selected_both_wrong": 1352 + }, + { + "budget": 0.52, + "threshold": -0.45961302518844604, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.5763581418230375, + "rescued": 916, + "harmed": 691, + "missed_rescue": 1673, + "avoided_harm": 750, + "selected_both_correct": 4403, + "selected_both_wrong": 1389 + }, + { + "budget": 0.53, + "threshold": -0.46368208527565, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.5774826059456041, + "rescued": 948, + "harmed": 707, + "missed_rescue": 1641, + "avoided_harm": 734, + "selected_both_correct": 4468, + "selected_both_wrong": 1418 + }, + { + "budget": 0.54, + "threshold": -0.46793216466903687, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.5792395811371144, + "rescued": 986, + "harmed": 720, + "missed_rescue": 1603, + "avoided_harm": 721, + "selected_both_correct": 4528, + "selected_both_wrong": 1449 + }, + { + "budget": 0.55, + "threshold": -0.4718930721282959, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.5795909761754164, + "rescued": 1010, + "harmed": 739, + "missed_rescue": 1579, + "avoided_harm": 702, + "selected_both_correct": 4579, + "selected_both_wrong": 1497 + }, + { + "budget": 0.56, + "threshold": -0.47629430890083313, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.5794504181600956, + "rescued": 1031, + "harmed": 762, + "missed_rescue": 1558, + "avoided_harm": 679, + "selected_both_correct": 4644, + "selected_both_wrong": 1531 + }, + { + "budget": 0.57, + "threshold": -0.4799485504627228, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.5802234872443601, + "rescued": 1060, + "harmed": 780, + "missed_rescue": 1529, + "avoided_harm": 661, + "selected_both_correct": 4700, + "selected_both_wrong": 1570 + }, + { + "budget": 0.58, + "threshold": -0.4838995635509491, + "operator": "ge", + "actual_fraction": 0.5799423712137185, + "metric": 0.5812776723592663, + "rescued": 1093, + "harmed": 798, + "missed_rescue": 1496, + "avoided_harm": 643, + "selected_both_correct": 4756, + "selected_both_wrong": 1605 + }, + { + "budget": 0.59, + "threshold": -0.4886912405490875, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.5816290673975684, + "rescued": 1120, + "harmed": 820, + "missed_rescue": 1469, + "avoided_harm": 621, + "selected_both_correct": 4815, + "selected_both_wrong": 1640 + }, + { + "budget": 0.6, + "threshold": -0.49332454800605774, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.5819804624358704, + "rescued": 1142, + "harmed": 837, + "missed_rescue": 1447, + "avoided_harm": 604, + "selected_both_correct": 4882, + "selected_both_wrong": 1676 + }, + { + "budget": 0.61, + "threshold": -0.49750789999961853, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.582261578466512, + "rescued": 1166, + "harmed": 857, + "missed_rescue": 1423, + "avoided_harm": 584, + "selected_both_correct": 4941, + "selected_both_wrong": 1715 + }, + { + "budget": 0.62, + "threshold": -0.5024909377098083, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.5823318574741725, + "rescued": 1183, + "harmed": 873, + "missed_rescue": 1406, + "avoided_harm": 568, + "selected_both_correct": 5010, + "selected_both_wrong": 1755 + }, + { + "budget": 0.63, + "threshold": -0.5067122578620911, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.5824021364818329, + "rescued": 1205, + "harmed": 894, + "missed_rescue": 1384, + "avoided_harm": 547, + "selected_both_correct": 5076, + "selected_both_wrong": 1789 + }, + { + "budget": 0.64, + "threshold": -0.512323796749115, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.5825426944971537, + "rescued": 1223, + "harmed": 910, + "missed_rescue": 1366, + "avoided_harm": 531, + "selected_both_correct": 5140, + "selected_both_wrong": 1833 + }, + { + "budget": 0.65, + "threshold": -0.5170255303382874, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.5833157635814182, + "rescued": 1250, + "harmed": 926, + "missed_rescue": 1339, + "avoided_harm": 515, + "selected_both_correct": 5200, + "selected_both_wrong": 1872 + }, + { + "budget": 0.66, + "threshold": -0.5215396881103516, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.5833157635814182, + "rescued": 1269, + "harmed": 945, + "missed_rescue": 1320, + "avoided_harm": 496, + "selected_both_correct": 5264, + "selected_both_wrong": 1913 + }, + { + "budget": 0.67, + "threshold": -0.5264327526092529, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.5826129735048141, + "rescued": 1288, + "harmed": 974, + "missed_rescue": 1301, + "avoided_harm": 467, + "selected_both_correct": 5321, + "selected_both_wrong": 1950 + }, + { + "budget": 0.68, + "threshold": -0.5312389135360718, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.583104926558437, + "rescued": 1316, + "harmed": 995, + "missed_rescue": 1273, + "avoided_harm": 446, + "selected_both_correct": 5379, + "selected_both_wrong": 1985 + }, + { + "budget": 0.69, + "threshold": -0.5368650555610657, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.5835266006043994, + "rescued": 1336, + "harmed": 1009, + "missed_rescue": 1253, + "avoided_harm": 432, + "selected_both_correct": 5445, + "selected_both_wrong": 2028 + }, + { + "budget": 0.7, + "threshold": -0.5431069135665894, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.5843699486963244, + "rescued": 1363, + "harmed": 1024, + "missed_rescue": 1226, + "avoided_harm": 417, + "selected_both_correct": 5492, + "selected_both_wrong": 2081 + }, + { + "budget": 0.71, + "threshold": -0.5487136244773865, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.5852835757959097, + "rescued": 1393, + "harmed": 1041, + "missed_rescue": 1196, + "avoided_harm": 400, + "selected_both_correct": 5550, + "selected_both_wrong": 2118 + }, + { + "budget": 0.72, + "threshold": -0.5549414753913879, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.5849321807576077, + "rescued": 1415, + "harmed": 1068, + "missed_rescue": 1174, + "avoided_harm": 373, + "selected_both_correct": 5611, + "selected_both_wrong": 2150 + }, + { + "budget": 0.73, + "threshold": -0.5604677200317383, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.5845105067116453, + "rescued": 1432, + "harmed": 1091, + "missed_rescue": 1157, + "avoided_harm": 350, + "selected_both_correct": 5683, + "selected_both_wrong": 2181 + }, + { + "budget": 0.74, + "threshold": -0.5666090250015259, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.584651064726966, + "rescued": 1452, + "harmed": 1109, + "missed_rescue": 1137, + "avoided_harm": 332, + "selected_both_correct": 5738, + "selected_both_wrong": 2230 + }, + { + "budget": 0.75, + "threshold": -0.574927568435669, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.5847213437346265, + "rescued": 1473, + "harmed": 1129, + "missed_rescue": 1116, + "avoided_harm": 312, + "selected_both_correct": 5796, + "selected_both_wrong": 2273 + }, + { + "budget": 0.76, + "threshold": -0.5812664031982422, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.5850727387729285, + "rescued": 1499, + "harmed": 1150, + "missed_rescue": 1090, + "avoided_harm": 291, + "selected_both_correct": 5856, + "selected_both_wrong": 2309 + }, + { + "budget": 0.77, + "threshold": -0.5883752107620239, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.5853538548035702, + "rescued": 1522, + "harmed": 1169, + "missed_rescue": 1067, + "avoided_harm": 272, + "selected_both_correct": 5923, + "selected_both_wrong": 2342 + }, + { + "budget": 0.78, + "threshold": -0.5972744226455688, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.5852132967882494, + "rescued": 1540, + "harmed": 1189, + "missed_rescue": 1049, + "avoided_harm": 252, + "selected_both_correct": 5976, + "selected_both_wrong": 2393 + }, + { + "budget": 0.79, + "threshold": -0.6058458089828491, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.5857755288495327, + "rescued": 1569, + "harmed": 1210, + "missed_rescue": 1020, + "avoided_harm": 231, + "selected_both_correct": 6037, + "selected_both_wrong": 2424 + }, + { + "budget": 0.8, + "threshold": -0.6140543818473816, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.5853538548035702, + "rescued": 1596, + "harmed": 1243, + "missed_rescue": 993, + "avoided_harm": 198, + "selected_both_correct": 6083, + "selected_both_wrong": 2461 + }, + { + "budget": 0.81, + "threshold": -0.6230268478393555, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.5852132967882494, + "rescued": 1619, + "harmed": 1268, + "missed_rescue": 970, + "avoided_harm": 173, + "selected_both_correct": 6141, + "selected_both_wrong": 2497 + }, + { + "budget": 0.82, + "threshold": -0.6330366134643555, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.5852835757959097, + "rescued": 1639, + "harmed": 1287, + "missed_rescue": 950, + "avoided_harm": 154, + "selected_both_correct": 6203, + "selected_both_wrong": 2538 + }, + { + "budget": 0.83, + "threshold": -0.6429333686828613, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.584299669688664, + "rescued": 1655, + "harmed": 1317, + "missed_rescue": 934, + "avoided_harm": 124, + "selected_both_correct": 6256, + "selected_both_wrong": 2582 + }, + { + "budget": 0.84, + "threshold": -0.6513019800186157, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.5847213437346265, + "rescued": 1679, + "harmed": 1335, + "missed_rescue": 910, + "avoided_harm": 106, + "selected_both_correct": 6315, + "selected_both_wrong": 2623 + }, + { + "budget": 0.85, + "threshold": -0.6624053120613098, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.5849321807576077, + "rescued": 1702, + "harmed": 1355, + "missed_rescue": 887, + "avoided_harm": 86, + "selected_both_correct": 6367, + "selected_both_wrong": 2670 + }, + { + "budget": 0.86, + "threshold": -0.6759263277053833, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.584651064726966, + "rescued": 1724, + "harmed": 1381, + "missed_rescue": 865, + "avoided_harm": 60, + "selected_both_correct": 6420, + "selected_both_wrong": 2711 + }, + { + "budget": 0.87, + "threshold": -0.6904275417327881, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.5847213437346265, + "rescued": 1741, + "harmed": 1397, + "missed_rescue": 848, + "avoided_harm": 44, + "selected_both_correct": 6489, + "selected_both_wrong": 2752 + }, + { + "budget": 0.88, + "threshold": -0.7045450210571289, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.5859160868648534, + "rescued": 1778, + "harmed": 1417, + "missed_rescue": 811, + "avoided_harm": 24, + "selected_both_correct": 6525, + "selected_both_wrong": 2801 + }, + { + "budget": 0.89, + "threshold": -0.7205488681793213, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.5913275704547052, + "rescued": 1856, + "harmed": 1418, + "missed_rescue": 733, + "avoided_harm": 23, + "selected_both_correct": 6525, + "selected_both_wrong": 2864 + }, + { + "budget": 0.9, + "threshold": -0.7357243299484253, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.5963876590062548, + "rescued": 1928, + "harmed": 1418, + "missed_rescue": 661, + "avoided_harm": 23, + "selected_both_correct": 6525, + "selected_both_wrong": 2935 + }, + { + "budget": 0.91, + "threshold": -0.7513164281845093, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.6009557945041816, + "rescued": 1993, + "harmed": 1418, + "missed_rescue": 596, + "avoided_harm": 23, + "selected_both_correct": 6525, + "selected_both_wrong": 3012 + }, + { + "budget": 0.92, + "threshold": -0.7671180963516235, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.606156441071052, + "rescued": 2069, + "harmed": 1420, + "missed_rescue": 520, + "avoided_harm": 21, + "selected_both_correct": 6525, + "selected_both_wrong": 3076 + }, + { + "budget": 0.93, + "threshold": -0.7833515405654907, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.6112165296226018, + "rescued": 2142, + "harmed": 1421, + "missed_rescue": 447, + "avoided_harm": 20, + "selected_both_correct": 6526, + "selected_both_wrong": 3143 + }, + { + "budget": 0.94, + "threshold": -0.8013419508934021, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.6155738280975472, + "rescued": 2206, + "harmed": 1423, + "missed_rescue": 383, + "avoided_harm": 18, + "selected_both_correct": 6527, + "selected_both_wrong": 3219 + }, + { + "budget": 0.95, + "threshold": -0.8188987374305725, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.6207744746644177, + "rescued": 2280, + "harmed": 1423, + "missed_rescue": 309, + "avoided_harm": 18, + "selected_both_correct": 6528, + "selected_both_wrong": 3286 + }, + { + "budget": 0.96, + "threshold": -0.8395025134086609, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.6249912151240424, + "rescued": 2344, + "harmed": 1427, + "missed_rescue": 245, + "avoided_harm": 14, + "selected_both_correct": 6529, + "selected_both_wrong": 3359 + }, + { + "budget": 0.97, + "threshold": -0.8628664612770081, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.6291376765760067, + "rescued": 2405, + "harmed": 1429, + "missed_rescue": 184, + "avoided_harm": 12, + "selected_both_correct": 6531, + "selected_both_wrong": 3437 + }, + { + "budget": 0.98, + "threshold": -0.8857419490814209, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.6332138590203107, + "rescued": 2467, + "harmed": 1433, + "missed_rescue": 122, + "avoided_harm": 8, + "selected_both_correct": 6533, + "selected_both_wrong": 3511 + }, + { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6372197624569541, + "rescued": 2529, + "harmed": 1438, + "missed_rescue": 60, + "avoided_harm": 3, + "selected_both_correct": 6534, + "selected_both_wrong": 3585 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.6412256658935975, + "rescued": 2589, + "harmed": 1441, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6535, + "selected_both_wrong": 3664 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.26688029946402747, + "rescued": 0, + "harmed": 0, + "missed_rescue": 10723, + "avoided_harm": 2500, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.1584138125181198, + "operator": "ge", + "actual_fraction": 0.029095652667101495, + "metric": 0.2671071661514902, + "rescued": 14, + "harmed": 6, + "missed_rescue": 10709, + "avoided_harm": 2494, + "selected_both_correct": 996, + "selected_both_wrong": 10 + }, + { + "budget": 0.02, + "threshold": -0.21403978765010834, + "operator": "ge", + "actual_fraction": 0.04622408757054136, + "metric": 0.2674474661826844, + "rescued": 34, + "harmed": 14, + "missed_rescue": 10689, + "avoided_harm": 2486, + "selected_both_correct": 1522, + "selected_both_wrong": 60 + }, + { + "budget": 0.03, + "threshold": -0.24253316223621368, + "operator": "ge", + "actual_fraction": 0.056149505147037974, + "metric": 0.26801463290134137, + "rescued": 58, + "harmed": 18, + "missed_rescue": 10665, + "avoided_harm": 2482, + "selected_both_correct": 1796, + "selected_both_wrong": 108 + }, + { + "budget": 0.04, + "threshold": -0.2599165141582489, + "operator": "ge", + "actual_fraction": 0.06372118084110824, + "metric": 0.2684683662762669, + "rescued": 81, + "harmed": 25, + "missed_rescue": 10642, + "avoided_harm": 2475, + "selected_both_correct": 1988, + "selected_both_wrong": 153 + }, + { + "budget": 0.05, + "threshold": -0.2725968658924103, + "operator": "ge", + "actual_fraction": 0.06882568130902078, + "metric": 0.26883702464339393, + "rescued": 101, + "harmed": 32, + "missed_rescue": 10622, + "avoided_harm": 2468, + "selected_both_correct": 2095, + "selected_both_wrong": 199 + }, + { + "budget": 0.06, + "threshold": -0.2842792272567749, + "operator": "ge", + "actual_fraction": 0.07475257351898591, + "metric": 0.26920568301052095, + "rescued": 119, + "harmed": 37, + "missed_rescue": 10604, + "avoided_harm": 2463, + "selected_both_correct": 2209, + "selected_both_wrong": 271 + }, + { + "budget": 0.07, + "threshold": -0.2935555577278137, + "operator": "ge", + "actual_fraction": 0.08033916569775686, + "metric": 0.2694609080339166, + "rescued": 140, + "harmed": 49, + "missed_rescue": 10583, + "avoided_harm": 2451, + "selected_both_correct": 2306, + "selected_both_wrong": 338 + }, + { + "budget": 0.08, + "threshold": -0.30114513635635376, + "operator": "ge", + "actual_fraction": 0.08632277457958767, + "metric": 0.269942999744775, + "rescued": 170, + "harmed": 62, + "missed_rescue": 10553, + "avoided_harm": 2438, + "selected_both_correct": 2387, + "selected_both_wrong": 425 + }, + { + "budget": 0.09, + "threshold": -0.308601438999176, + "operator": "ge", + "actual_fraction": 0.09227802512548564, + "metric": 0.27028329977596915, + "rescued": 200, + "harmed": 80, + "missed_rescue": 10523, + "avoided_harm": 2420, + "selected_both_correct": 2473, + "selected_both_wrong": 501 + }, + { + "budget": 0.1, + "threshold": -0.3146189749240875, + "operator": "ge", + "actual_fraction": 0.09783625896832374, + "metric": 0.27076539148682754, + "rescued": 226, + "harmed": 89, + "missed_rescue": 10497, + "avoided_harm": 2411, + "selected_both_correct": 2547, + "selected_both_wrong": 588 + }, + { + "budget": 0.11, + "threshold": -0.32043084502220154, + "operator": "ge", + "actual_fraction": 0.10362135949862462, + "metric": 0.2713325582054845, + "rescued": 258, + "harmed": 101, + "missed_rescue": 10465, + "avoided_harm": 2399, + "selected_both_correct": 2611, + "selected_both_wrong": 684 + }, + { + "budget": 0.12, + "threshold": -0.32644838094711304, + "operator": "ge", + "actual_fraction": 0.11031392677877662, + "metric": 0.27221166661940277, + "rescued": 305, + "harmed": 117, + "missed_rescue": 10418, + "avoided_harm": 2383, + "selected_both_correct": 2671, + "selected_both_wrong": 797 + }, + { + "budget": 0.13, + "threshold": -0.33121857047080994, + "operator": "ge", + "actual_fraction": 0.11658111901993591, + "metric": 0.2733176417207838, + "rescued": 353, + "harmed": 126, + "missed_rescue": 10370, + "avoided_harm": 2374, + "selected_both_correct": 2743, + "selected_both_wrong": 889 + }, + { + "budget": 0.14, + "threshold": -0.33570271730422974, + "operator": "ge", + "actual_fraction": 0.12262144457363242, + "metric": 0.2742534668065678, + "rescued": 399, + "harmed": 139, + "missed_rescue": 10324, + "avoided_harm": 2361, + "selected_both_correct": 2797, + "selected_both_wrong": 989 + }, + { + "budget": 0.15, + "threshold": -0.3407592475414276, + "operator": "ge", + "actual_fraction": 0.13024983693956838, + "metric": 0.27535944190794887, + "rescued": 455, + "harmed": 156, + "missed_rescue": 10268, + "avoided_harm": 2344, + "selected_both_correct": 2866, + "selected_both_wrong": 1116 + }, + { + "budget": 0.16, + "threshold": -0.34456855058670044, + "operator": "ge", + "actual_fraction": 0.13714091257125033, + "metric": 0.2768907920483226, + "rescued": 526, + "harmed": 173, + "missed_rescue": 10197, + "avoided_harm": 2327, + "selected_both_correct": 2922, + "selected_both_wrong": 1215 + }, + { + "budget": 0.17, + "threshold": -0.3487326204776764, + "operator": "ge", + "actual_fraction": 0.143521538156141, + "metric": 0.27833706718089785, + "rescued": 589, + "harmed": 185, + "missed_rescue": 10134, + "avoided_harm": 2315, + "selected_both_correct": 2969, + "selected_both_wrong": 1318 + }, + { + "budget": 0.18, + "threshold": -0.35280776023864746, + "operator": "ge", + "actual_fraction": 0.15095142217054702, + "metric": 0.2800669256728015, + "rescued": 665, + "harmed": 200, + "missed_rescue": 10058, + "avoided_harm": 2300, + "selected_both_correct": 3012, + "selected_both_wrong": 1446 + }, + { + "budget": 0.19, + "threshold": -0.3564819097518921, + "operator": "ge", + "actual_fraction": 0.15823951450528884, + "metric": 0.28097439242265265, + "rescued": 720, + "harmed": 223, + "missed_rescue": 10003, + "avoided_harm": 2277, + "selected_both_correct": 3066, + "selected_both_wrong": 1571 + }, + { + "budget": 0.2, + "threshold": -0.35992002487182617, + "operator": "ge", + "actual_fraction": 0.16496044012137367, + "metric": 0.2826191759067578, + "rescued": 797, + "harmed": 242, + "missed_rescue": 9926, + "avoided_harm": 2258, + "selected_both_correct": 3110, + "selected_both_wrong": 1668 + }, + { + "budget": 0.21, + "threshold": -0.36340460181236267, + "operator": "ge", + "actual_fraction": 0.17290077418257097, + "metric": 0.28457590108612424, + "rescued": 880, + "harmed": 256, + "missed_rescue": 9843, + "avoided_harm": 2244, + "selected_both_correct": 3166, + "selected_both_wrong": 1795 + }, + { + "budget": 0.22, + "threshold": -0.3674520254135132, + "operator": "ge", + "actual_fraction": 0.1829679834387318, + "metric": 0.28667441794515497, + "rescued": 979, + "harmed": 281, + "missed_rescue": 9744, + "avoided_harm": 2219, + "selected_both_correct": 3230, + "selected_both_wrong": 1962 + }, + { + "budget": 0.23, + "threshold": -0.37108901143074036, + "operator": "ge", + "actual_fraction": 0.19122025919519042, + "metric": 0.28814905141366304, + "rescued": 1052, + "harmed": 302, + "missed_rescue": 9671, + "avoided_harm": 2198, + "selected_both_correct": 3292, + "selected_both_wrong": 2097 + }, + { + "budget": 0.24, + "threshold": -0.3745429813861847, + "operator": "ge", + "actual_fraction": 0.1999262683265746, + "metric": 0.28993562657743244, + "rescued": 1139, + "harmed": 326, + "missed_rescue": 9584, + "avoided_harm": 2174, + "selected_both_correct": 3343, + "selected_both_wrong": 2242 + }, + { + "budget": 0.25, + "threshold": -0.3773327171802521, + "operator": "ge", + "actual_fraction": 0.2080651107393018, + "metric": 0.2917789184130675, + "rescued": 1224, + "harmed": 346, + "missed_rescue": 9499, + "avoided_harm": 2154, + "selected_both_correct": 3394, + "selected_both_wrong": 2373 + }, + { + "budget": 0.26, + "threshold": -0.3803459703922272, + "operator": "ge", + "actual_fraction": 0.21603380313643195, + "metric": 0.29379236026429967, + "rescued": 1315, + "harmed": 366, + "missed_rescue": 9408, + "avoided_harm": 2134, + "selected_both_correct": 3435, + "selected_both_wrong": 2502 + }, + { + "budget": 0.27, + "threshold": -0.38361650705337524, + "operator": "ge", + "actual_fraction": 0.2243144372288234, + "metric": 0.29623117715452457, + "rescued": 1420, + "harmed": 385, + "missed_rescue": 9303, + "avoided_harm": 2115, + "selected_both_correct": 3485, + "selected_both_wrong": 2620 + }, + { + "budget": 0.28, + "threshold": -0.38652583956718445, + "operator": "ge", + "actual_fraction": 0.2324249213056178, + "metric": 0.29813118566202534, + "rescued": 1512, + "harmed": 410, + "missed_rescue": 9211, + "avoided_harm": 2090, + "selected_both_correct": 3520, + "selected_both_wrong": 2754 + }, + { + "budget": 0.29, + "threshold": -0.38952407240867615, + "operator": "ge", + "actual_fraction": 0.2419816805149874, + "metric": 0.3005983608881831, + "rescued": 1618, + "harmed": 429, + "missed_rescue": 9105, + "avoided_harm": 2071, + "selected_both_correct": 3587, + "selected_both_wrong": 2899 + }, + { + "budget": 0.3, + "threshold": -0.39262744784355164, + "operator": "ge", + "actual_fraction": 0.2512832147009613, + "metric": 0.3034625528174007, + "rescued": 1743, + "harmed": 453, + "missed_rescue": 8980, + "avoided_harm": 2047, + "selected_both_correct": 3641, + "selected_both_wrong": 3024 + }, + { + "budget": 0.31, + "threshold": -0.3954668939113617, + "operator": "ge", + "actual_fraction": 0.2596772821370842, + "metric": 0.30561778634829706, + "rescued": 1837, + "harmed": 471, + "missed_rescue": 8886, + "avoided_harm": 2029, + "selected_both_correct": 3675, + "selected_both_wrong": 3174 + }, + { + "budget": 0.32, + "threshold": -0.3986983895301819, + "operator": "ge", + "actual_fraction": 0.26985792473697645, + "metric": 0.308793919972776, + "rescued": 1971, + "harmed": 493, + "missed_rescue": 8752, + "avoided_harm": 2007, + "selected_both_correct": 3725, + "selected_both_wrong": 3327 + }, + { + "budget": 0.33, + "threshold": -0.4014112949371338, + "operator": "ge", + "actual_fraction": 0.27757139211071097, + "metric": 0.31109094518333663, + "rescued": 2067, + "harmed": 508, + "missed_rescue": 8656, + "avoided_harm": 1992, + "selected_both_correct": 3757, + "selected_both_wrong": 3456 + }, + { + "budget": 0.34, + "threshold": -0.4042409658432007, + "operator": "ge", + "actual_fraction": 0.28729830133567763, + "metric": 0.3132178203783002, + "rescued": 2174, + "harmed": 540, + "missed_rescue": 8549, + "avoided_harm": 1960, + "selected_both_correct": 3810, + "selected_both_wrong": 3607 + }, + { + "budget": 0.35, + "threshold": -0.40724706649780273, + "operator": "ge", + "actual_fraction": 0.29793267731049544, + "metric": 0.31659246235430905, + "rescued": 2314, + "harmed": 561, + "missed_rescue": 8409, + "avoided_harm": 1939, + "selected_both_correct": 3868, + "selected_both_wrong": 3763 + }, + { + "budget": 0.36, + "threshold": -0.40968337655067444, + "operator": "ge", + "actual_fraction": 0.3067237614496781, + "metric": 0.31928650426792954, + "rescued": 2427, + "harmed": 579, + "missed_rescue": 8296, + "avoided_harm": 1921, + "selected_both_correct": 3912, + "selected_both_wrong": 3898 + }, + { + "budget": 0.37, + "threshold": -0.4123799502849579, + "operator": "ge", + "actual_fraction": 0.3161387289793835, + "metric": 0.32192382950968434, + "rescued": 2547, + "harmed": 606, + "missed_rescue": 8176, + "avoided_harm": 1894, + "selected_both_correct": 3952, + "selected_both_wrong": 4043 + }, + { + "budget": 0.38, + "threshold": -0.41514647006988525, + "operator": "ge", + "actual_fraction": 0.32538354649349177, + "metric": 0.3242492130561779, + "rescued": 2654, + "harmed": 631, + "missed_rescue": 8069, + "avoided_harm": 1869, + "selected_both_correct": 4003, + "selected_both_wrong": 4186 + }, + { + "budget": 0.39, + "threshold": -0.4179389476776123, + "operator": "ge", + "actual_fraction": 0.3354507557496526, + "metric": 0.3270283299775969, + "rescued": 2782, + "harmed": 661, + "missed_rescue": 7941, + "avoided_harm": 1839, + "selected_both_correct": 4053, + "selected_both_wrong": 4333 + }, + { + "budget": 0.4, + "threshold": -0.42099592089653015, + "operator": "ge", + "actual_fraction": 0.34653886509939597, + "metric": 0.33009103025834446, + "rescued": 2921, + "harmed": 692, + "missed_rescue": 7802, + "avoided_harm": 1808, + "selected_both_correct": 4098, + "selected_both_wrong": 4509 + }, + { + "budget": 0.41, + "threshold": -0.4243226647377014, + "operator": "ge", + "actual_fraction": 0.3577404077928707, + "metric": 0.33306865553129344, + "rescued": 3055, + "harmed": 721, + "missed_rescue": 7668, + "avoided_harm": 1779, + "selected_both_correct": 4157, + "selected_both_wrong": 4682 + }, + { + "budget": 0.42, + "threshold": -0.4275209903717041, + "operator": "ge", + "actual_fraction": 0.3693673255253382, + "metric": 0.33661344752289935, + "rescued": 3203, + "harmed": 744, + "missed_rescue": 7520, + "avoided_harm": 1756, + "selected_both_correct": 4207, + "selected_both_wrong": 4871 + }, + { + "budget": 0.43, + "threshold": -0.4307159185409546, + "operator": "ge", + "actual_fraction": 0.38039871820321586, + "metric": 0.33953435612398264, + "rescued": 3331, + "harmed": 769, + "missed_rescue": 7392, + "avoided_harm": 1731, + "selected_both_correct": 4259, + "selected_both_wrong": 5055 + }, + { + "budget": 0.44, + "threshold": -0.4336964190006256, + "operator": "ge", + "actual_fraction": 0.39049428579530954, + "metric": 0.3427388480843944, + "rescued": 3465, + "harmed": 790, + "missed_rescue": 7258, + "avoided_harm": 1710, + "selected_both_correct": 4309, + "selected_both_wrong": 5206 + }, + { + "budget": 0.45, + "threshold": -0.4373021125793457, + "operator": "ge", + "actual_fraction": 0.4023764285511726, + "metric": 0.34597169838073905, + "rescued": 3606, + "harmed": 817, + "missed_rescue": 7117, + "avoided_harm": 1683, + "selected_both_correct": 4365, + "selected_both_wrong": 5401 + }, + { + "budget": 0.46, + "threshold": -0.4404309391975403, + "operator": "ge", + "actual_fraction": 0.41196154609647506, + "metric": 0.3488358903099566, + "rescued": 3729, + "harmed": 839, + "missed_rescue": 6994, + "avoided_harm": 1661, + "selected_both_correct": 4405, + "selected_both_wrong": 5554 + }, + { + "budget": 0.47, + "threshold": -0.44344601035118103, + "operator": "ge", + "actual_fraction": 0.42157502197771035, + "metric": 0.35172844057510705, + "rescued": 3850, + "harmed": 858, + "missed_rescue": 6873, + "avoided_harm": 1642, + "selected_both_correct": 4436, + "selected_both_wrong": 5722 + }, + { + "budget": 0.48, + "threshold": -0.4465356171131134, + "operator": "ge", + "actual_fraction": 0.43184073958540115, + "metric": 0.35459263250432466, + "rescued": 3979, + "harmed": 886, + "missed_rescue": 6744, + "avoided_harm": 1614, + "selected_both_correct": 4479, + "selected_both_wrong": 5884 + }, + { + "budget": 0.49, + "threshold": -0.44965147972106934, + "operator": "ge", + "actual_fraction": 0.44165272381816634, + "metric": 0.35751354110540795, + "rescued": 4106, + "harmed": 910, + "missed_rescue": 6617, + "avoided_harm": 1590, + "selected_both_correct": 4530, + "selected_both_wrong": 6028 + }, + { + "budget": 0.5, + "threshold": -0.453143447637558, + "operator": "ge", + "actual_fraction": 0.452315458128917, + "metric": 0.3607463914017525, + "rescued": 4242, + "harmed": 932, + "missed_rescue": 6481, + "avoided_harm": 1568, + "selected_both_correct": 4574, + "selected_both_wrong": 6202 + }, + { + "budget": 0.51, + "threshold": -0.4565357565879822, + "operator": "ge", + "actual_fraction": 0.4631483424552647, + "metric": 0.36440461673708985, + "rescued": 4392, + "harmed": 953, + "missed_rescue": 6331, + "avoided_harm": 1547, + "selected_both_correct": 4610, + "selected_both_wrong": 6377 + }, + { + "budget": 0.52, + "threshold": -0.45961302518844604, + "operator": "ge", + "actual_fraction": 0.47176927657885037, + "metric": 0.36641805858832205, + "rescued": 4484, + "harmed": 974, + "missed_rescue": 6239, + "avoided_harm": 1526, + "selected_both_correct": 4654, + "selected_both_wrong": 6524 + }, + { + "budget": 0.53, + "threshold": -0.46368208527565, + "operator": "ge", + "actual_fraction": 0.4840484360377733, + "metric": 0.36970762555653236, + "rescued": 4626, + "harmed": 1000, + "missed_rescue": 6097, + "avoided_harm": 1500, + "selected_both_correct": 4715, + "selected_both_wrong": 6728 + }, + { + "budget": 0.54, + "threshold": -0.46793216466903687, + "operator": "ge", + "actual_fraction": 0.495079828715651, + "metric": 0.3731389842044069, + "rescued": 4772, + "harmed": 1025, + "missed_rescue": 5951, + "avoided_harm": 1475, + "selected_both_correct": 4766, + "selected_both_wrong": 6895 + }, + { + "budget": 0.55, + "threshold": -0.4718930721282959, + "operator": "ge", + "actual_fraction": 0.5065365964325214, + "metric": 0.37634347616481867, + "rescued": 4911, + "harmed": 1051, + "missed_rescue": 5812, + "avoided_harm": 1449, + "selected_both_correct": 4826, + "selected_both_wrong": 7074 + }, + { + "budget": 0.56, + "threshold": -0.47629430890083313, + "operator": "ge", + "actual_fraction": 0.5183620225165188, + "metric": 0.379434534781499, + "rescued": 5056, + "harmed": 1087, + "missed_rescue": 5667, + "avoided_harm": 1413, + "selected_both_correct": 4873, + "selected_both_wrong": 7263 + }, + { + "budget": 0.57, + "threshold": -0.4799485504627228, + "operator": "ge", + "actual_fraction": 0.5273232566712985, + "metric": 0.3818733516717239, + "rescued": 5170, + "harmed": 1115, + "missed_rescue": 5553, + "avoided_harm": 1385, + "selected_both_correct": 4915, + "selected_both_wrong": 7395 + }, + { + "budget": 0.58, + "threshold": -0.4838995635509491, + "operator": "ge", + "actual_fraction": 0.5381561409976463, + "metric": 0.3852763519836656, + "rescued": 5317, + "harmed": 1142, + "missed_rescue": 5406, + "avoided_harm": 1358, + "selected_both_correct": 4957, + "selected_both_wrong": 7561 + }, + { + "budget": 0.59, + "threshold": -0.4886912405490875, + "operator": "ge", + "actual_fraction": 0.5503502254487707, + "metric": 0.3888211439752715, + "rescued": 5470, + "harmed": 1170, + "missed_rescue": 5253, + "avoided_harm": 1330, + "selected_both_correct": 4996, + "selected_both_wrong": 7771 + }, + { + "budget": 0.6, + "threshold": -0.49332454800605774, + "operator": "ge", + "actual_fraction": 0.5619487848453053, + "metric": 0.39256444431840737, + "rescued": 5631, + "harmed": 1199, + "missed_rescue": 5092, + "avoided_harm": 1301, + "selected_both_correct": 5037, + "selected_both_wrong": 7949 + }, + { + "budget": 0.61, + "threshold": -0.49750789999961853, + "operator": "ge", + "actual_fraction": 0.5722712191248618, + "metric": 0.3955987862632221, + "rescued": 5771, + "harmed": 1232, + "missed_rescue": 4952, + "avoided_harm": 1268, + "selected_both_correct": 5078, + "selected_both_wrong": 8099 + }, + { + "budget": 0.62, + "threshold": -0.5024909377098083, + "operator": "ge", + "actual_fraction": 0.5818563366701642, + "metric": 0.39840626152057396, + "rescued": 5893, + "harmed": 1255, + "missed_rescue": 4830, + "avoided_harm": 1245, + "selected_both_correct": 5118, + "selected_both_wrong": 8252 + }, + { + "budget": 0.63, + "threshold": -0.5067122578620911, + "operator": "ge", + "actual_fraction": 0.590817570824944, + "metric": 0.4011003034341945, + "rescued": 6008, + "harmed": 1275, + "missed_rescue": 4715, + "avoided_harm": 1225, + "selected_both_correct": 5160, + "selected_both_wrong": 8391 + }, + { + "budget": 0.64, + "threshold": -0.512323796749115, + "operator": "ge", + "actual_fraction": 0.604202705385248, + "metric": 0.40532569548818875, + "rescued": 6184, + "harmed": 1302, + "missed_rescue": 4539, + "avoided_harm": 1198, + "selected_both_correct": 5218, + "selected_both_wrong": 8602 + }, + { + "budget": 0.65, + "threshold": -0.5170255303382874, + "operator": "ge", + "actual_fraction": 0.6130505061962964, + "metric": 0.4078495873862122, + "rescued": 6296, + "harmed": 1325, + "missed_rescue": 4427, + "avoided_harm": 1175, + "selected_both_correct": 5258, + "selected_both_wrong": 8739 + }, + { + "budget": 0.66, + "threshold": -0.5215396881103516, + "operator": "ge", + "actual_fraction": 0.6222102487026061, + "metric": 0.4103167626123699, + "rescued": 6411, + "harmed": 1353, + "missed_rescue": 4312, + "avoided_harm": 1147, + "selected_both_correct": 5298, + "selected_both_wrong": 8879 + }, + { + "budget": 0.67, + "threshold": -0.5264327526092529, + "operator": "ge", + "actual_fraction": 0.6320505912713043, + "metric": 0.41216005444800496, + "rescued": 6514, + "harmed": 1391, + "missed_rescue": 4209, + "avoided_harm": 1109, + "selected_both_correct": 5333, + "selected_both_wrong": 9050 + }, + { + "budget": 0.68, + "threshold": -0.5312389135360718, + "operator": "ge", + "actual_fraction": 0.6424297422227264, + "metric": 0.4155630547599467, + "rescued": 6659, + "harmed": 1416, + "missed_rescue": 4064, + "avoided_harm": 1084, + "selected_both_correct": 5378, + "selected_both_wrong": 9201 + }, + { + "budget": 0.69, + "threshold": -0.5368650555610657, + "operator": "ge", + "actual_fraction": 0.6538865099395967, + "metric": 0.4177182882908431, + "rescued": 6784, + "harmed": 1465, + "missed_rescue": 3939, + "avoided_harm": 1035, + "selected_both_correct": 5424, + "selected_both_wrong": 9385 + }, + { + "budget": 0.7, + "threshold": -0.5431069135665894, + "operator": "ge", + "actual_fraction": 0.6647761109378102, + "metric": 0.4212347219465162, + "rescued": 6935, + "harmed": 1492, + "missed_rescue": 3788, + "avoided_harm": 1008, + "selected_both_correct": 5463, + "selected_both_wrong": 9552 + }, + { + "budget": 0.71, + "threshold": -0.5487136244773865, + "operator": "ge", + "actual_fraction": 0.6740492867878513, + "metric": 0.42384368885233814, + "rescued": 7054, + "harmed": 1519, + "missed_rescue": 3669, + "avoided_harm": 981, + "selected_both_correct": 5498, + "selected_both_wrong": 9698 + }, + { + "budget": 0.72, + "threshold": -0.5549414753913879, + "operator": "ge", + "actual_fraction": 0.6839179876924822, + "metric": 0.42628250574256304, + "rescued": 7167, + "harmed": 1546, + "missed_rescue": 3556, + "avoided_harm": 954, + "selected_both_correct": 5541, + "selected_both_wrong": 9863 + }, + { + "budget": 0.73, + "threshold": -0.5604677200317383, + "operator": "ge", + "actual_fraction": 0.6935314635737175, + "metric": 0.429260131015512, + "rescued": 7298, + "harmed": 1572, + "missed_rescue": 3425, + "avoided_harm": 928, + "selected_both_correct": 5579, + "selected_both_wrong": 10007 + }, + { + "budget": 0.74, + "threshold": -0.5666090250015259, + "operator": "ge", + "actual_fraction": 0.7039673311970054, + "metric": 0.4321243229447296, + "rescued": 7424, + "harmed": 1597, + "missed_rescue": 3299, + "avoided_harm": 903, + "selected_both_correct": 5614, + "selected_both_wrong": 10189 + }, + { + "budget": 0.75, + "threshold": -0.574927568435669, + "operator": "ge", + "actual_fraction": 0.7186853075461532, + "metric": 0.4360094149675297, + "rescued": 7607, + "harmed": 1643, + "missed_rescue": 3116, + "avoided_harm": 857, + "selected_both_correct": 5687, + "selected_both_wrong": 10406 + }, + { + "budget": 0.76, + "threshold": -0.5812664031982422, + "operator": "ge", + "actual_fraction": 0.7283555000992542, + "metric": 0.438363156849956, + "rescued": 7724, + "harmed": 1677, + "missed_rescue": 2999, + "avoided_harm": 823, + "selected_both_correct": 5729, + "selected_both_wrong": 10554 + }, + { + "budget": 0.77, + "threshold": -0.5883752107620239, + "operator": "ge", + "actual_fraction": 0.7390465927459376, + "metric": 0.44173779882596487, + "rescued": 7869, + "harmed": 1703, + "missed_rescue": 2854, + "avoided_harm": 797, + "selected_both_correct": 5774, + "selected_both_wrong": 10715 + }, + { + "budget": 0.78, + "threshold": -0.5972744226455688, + "operator": "ge", + "actual_fraction": 0.752176502282846, + "metric": 0.44596319087995917, + "rescued": 8052, + "harmed": 1737, + "missed_rescue": 2671, + "avoided_harm": 763, + "selected_both_correct": 5823, + "selected_both_wrong": 10912 + }, + { + "budget": 0.79, + "threshold": -0.6058458089828491, + "operator": "ge", + "actual_fraction": 0.7635765533278507, + "metric": 0.44888409948104246, + "rescued": 8191, + "harmed": 1773, + "missed_rescue": 2532, + "avoided_harm": 727, + "selected_both_correct": 5871, + "selected_both_wrong": 11091 + }, + { + "budget": 0.8, + "threshold": -0.6140543818473816, + "operator": "ge", + "actual_fraction": 0.7745228709979298, + "metric": 0.4523721748007827, + "rescued": 8338, + "harmed": 1797, + "missed_rescue": 2385, + "avoided_harm": 703, + "selected_both_correct": 5916, + "selected_both_wrong": 11261 + }, + { + "budget": 0.81, + "threshold": -0.6230268478393555, + "operator": "ge", + "actual_fraction": 0.7864050137537929, + "metric": 0.4548109916910076, + "rescued": 8468, + "harmed": 1841, + "missed_rescue": 2255, + "avoided_harm": 659, + "selected_both_correct": 5956, + "selected_both_wrong": 11466 + }, + { + "budget": 0.82, + "threshold": -0.6330366134643555, + "operator": "ge", + "actual_fraction": 0.798287156509656, + "metric": 0.45827070867481495, + "rescued": 8625, + "harmed": 1876, + "missed_rescue": 2098, + "avoided_harm": 624, + "selected_both_correct": 5991, + "selected_both_wrong": 11658 + }, + { + "budget": 0.83, + "threshold": -0.6429333686828613, + "operator": "ge", + "actual_fraction": 0.8092334741797351, + "metric": 0.4612766922836968, + "rescued": 8762, + "harmed": 1907, + "missed_rescue": 1961, + "avoided_harm": 593, + "selected_both_correct": 6039, + "selected_both_wrong": 11828 + }, + { + "budget": 0.84, + "threshold": -0.6513019800186157, + "operator": "ge", + "actual_fraction": 0.8177126166236566, + "metric": 0.46340356747866035, + "rescued": 8858, + "harmed": 1928, + "missed_rescue": 1865, + "avoided_harm": 572, + "selected_both_correct": 6071, + "selected_both_wrong": 11978 + }, + { + "budget": 0.85, + "threshold": -0.6624053120613098, + "operator": "ge", + "actual_fraction": 0.8288858009811985, + "metric": 0.4659274593766838, + "rescued": 8984, + "harmed": 1965, + "missed_rescue": 1739, + "avoided_harm": 535, + "selected_both_correct": 6119, + "selected_both_wrong": 12161 + }, + { + "budget": 0.86, + "threshold": -0.6759263277053833, + "operator": "ge", + "actual_fraction": 0.8421291438618382, + "metric": 0.4693021013526926, + "rescued": 9143, + "harmed": 2005, + "missed_rescue": 1580, + "avoided_harm": 495, + "selected_both_correct": 6163, + "selected_both_wrong": 12385 + }, + { + "budget": 0.87, + "threshold": -0.6904275417327881, + "operator": "ge", + "actual_fraction": 0.8543799449848283, + "metric": 0.47202450160224596, + "rescued": 9273, + "harmed": 2039, + "missed_rescue": 1450, + "avoided_harm": 461, + "selected_both_correct": 6209, + "selected_both_wrong": 12607 + }, + { + "budget": 0.88, + "threshold": -0.7045450210571289, + "operator": "ge", + "actual_fraction": 0.8657516376939001, + "metric": 0.47395286844567963, + "rescued": 9384, + "harmed": 2082, + "missed_rescue": 1339, + "avoided_harm": 418, + "selected_both_correct": 6264, + "selected_both_wrong": 12799 + }, + { + "budget": 0.89, + "threshold": -0.7205488681793213, + "operator": "ge", + "actual_fraction": 0.8767830303717777, + "metric": 0.4759663102969118, + "rescued": 9501, + "harmed": 2128, + "missed_rescue": 1222, + "avoided_harm": 372, + "selected_both_correct": 6312, + "selected_both_wrong": 12977 + }, + { + "budget": 0.9, + "threshold": -0.7357243299484253, + "operator": "ge", + "actual_fraction": 0.8868785979638715, + "metric": 0.4782066188356067, + "rescued": 9610, + "harmed": 2158, + "missed_rescue": 1113, + "avoided_harm": 342, + "selected_both_correct": 6365, + "selected_both_wrong": 13141 + }, + { + "budget": 0.91, + "threshold": -0.7513164281845093, + "operator": "ge", + "actual_fraction": 0.8974562572668235, + "metric": 0.4802200606868389, + "rescued": 9710, + "harmed": 2187, + "missed_rescue": 1013, + "avoided_harm": 313, + "selected_both_correct": 6424, + "selected_both_wrong": 13326 + }, + { + "budget": 0.92, + "threshold": -0.7671180963516235, + "operator": "ge", + "actual_fraction": 0.9084025749369027, + "metric": 0.4826021609051981, + "rescued": 9825, + "harmed": 2218, + "missed_rescue": 898, + "avoided_harm": 282, + "selected_both_correct": 6474, + "selected_both_wrong": 13516 + }, + { + "budget": 0.93, + "threshold": -0.7833515405654907, + "operator": "ge", + "actual_fraction": 0.9181011258259365, + "metric": 0.4839917193659076, + "rescued": 9916, + "harmed": 2260, + "missed_rescue": 807, + "avoided_harm": 240, + "selected_both_correct": 6514, + "selected_both_wrong": 13685 + }, + { + "budget": 0.94, + "threshold": -0.8013419508934021, + "operator": "ge", + "actual_fraction": 0.9289623684882171, + "metric": 0.48634546124833394, + "rescued": 10031, + "harmed": 2292, + "missed_rescue": 692, + "avoided_harm": 208, + "selected_both_correct": 6566, + "selected_both_wrong": 13869 + }, + { + "budget": 0.95, + "threshold": -0.8188987374305725, + "operator": "ge", + "actual_fraction": 0.9394549527833707, + "metric": 0.48770666137311064, + "rescued": 10126, + "harmed": 2339, + "missed_rescue": 597, + "avoided_harm": 161, + "selected_both_correct": 6606, + "selected_both_wrong": 14057 + }, + { + "budget": 0.96, + "threshold": -0.8395025134086609, + "operator": "ge", + "actual_fraction": 0.9509684371721068, + "metric": 0.4894648782009472, + "rescued": 10222, + "harmed": 2373, + "missed_rescue": 501, + "avoided_harm": 127, + "selected_both_correct": 6654, + "selected_both_wrong": 14285 + }, + { + "budget": 0.97, + "threshold": -0.8628664612770081, + "operator": "ge", + "actual_fraction": 0.9640699883730822, + "metric": 0.49218727845050053, + "rescued": 10358, + "harmed": 2413, + "missed_rescue": 365, + "avoided_harm": 87, + "selected_both_correct": 6713, + "selected_both_wrong": 14512 + }, + { + "budget": 0.98, + "threshold": -0.8857419490814209, + "operator": "ge", + "actual_fraction": 0.9760372061367439, + "metric": 0.49439922865326263, + "rescued": 10476, + "harmed": 2453, + "missed_rescue": 247, + "avoided_harm": 47, + "selected_both_correct": 6784, + "selected_both_wrong": 14705 + }, + { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9873805405098829, + "metric": 0.4968380455434875, + "rescued": 10590, + "harmed": 2481, + "missed_rescue": 133, + "avoided_harm": 19, + "selected_both_correct": 6845, + "selected_both_wrong": 14902 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5000708958398321, + "rescued": 10723, + "harmed": 2500, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6911, + "selected_both_wrong": 15129 + } + ] + }, + "learned": { + "validation_area_0_25": 0.6137648464403682, + "validation_point": { + "metric": 0.6562513177313936, + "metric_std": 0.0022540834511329062, + "actual_fraction": 0.4439665471923536, + "actual_fraction_std": 0.03360550791504779, + "per_seed": [ + { + "seed": 2026, + "budget": 0.45, + "threshold": 0.2908308804035187, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.6554923044486611, + "rescue": 2118, + "harm": 767, + "missed_rescue": 471 + }, + { + "seed": 2027, + "budget": 0.4, + "threshold": 0.3213382363319397, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.6602712769695692, + "rescue": 2044, + "harm": 625, + "missed_rescue": 545 + }, + { + "seed": 2028, + "budget": 0.47, + "threshold": 0.24761390686035156, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.6552814674256799, + "rescue": 2109, + "harm": 761, + "missed_rescue": 480 + }, + { + "seed": 2029, + "budget": 0.42, + "threshold": 0.31439468264579773, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.6550003513950383, + "rescue": 2037, + "harm": 693, + "missed_rescue": 552 + }, + { + "seed": 2030, + "budget": 0.48, + "threshold": 0.16629476845264435, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.6552111884180195, + "rescue": 2163, + "harm": 816, + "missed_rescue": 426 + } + ], + "rescued": 2094.2, + "harmed": 732.4, + "missed_rescue": 494.8, + "avoided_harm": 708.6 + }, + "test_point": { + "metric": 0.49913507075404817, + "metric_std": 0.005887753308252541, + "actual_fraction": 0.5624138615546039, + "actual_fraction_std": 0.03987669902841215, + "per_seed": [ + { + "seed": 2026, + "budget": 0.45, + "threshold": 0.2908308804035187, + "operator": "ge", + "actual_fraction": 0.5800697615063948, + "metric": 0.5033037461361767, + "rescue": 9544, + "harm": 1207, + "missed_rescue": 1179 + }, + { + "seed": 2027, + "budget": 0.4, + "threshold": 0.3213382363319397, + "operator": "ge", + "actual_fraction": 0.5114993052207697, + "metric": 0.49249922014576186, + "rescue": 8938, + "harm": 982, + "missed_rescue": 1785 + }, + { + "seed": 2028, + "budget": 0.47, + "threshold": 0.24761390686035156, + "operator": "ge", + "actual_fraction": 0.5808070782406488, + "metric": 0.5009783625896832, + "rescue": 9465, + "harm": 1210, + "missed_rescue": 1258 + }, + { + "seed": 2029, + "budget": 0.42, + "threshold": 0.31439468264579773, + "operator": "ge", + "actual_fraction": 0.5310665570144344, + "metric": 0.4933783285596801, + "rescue": 9054, + "harm": 1067, + "missed_rescue": 1669 + }, + { + "seed": 2030, + "budget": 0.48, + "threshold": 0.16629476845264435, + "operator": "ge", + "actual_fraction": 0.6086266057907722, + "metric": 0.5055156963389388, + "rescue": 9688, + "harm": 1273, + "missed_rescue": 1035 + } + ], + "rescued": 9337.8, + "harmed": 1147.8, + "missed_rescue": 1385.2, + "avoided_harm": 1352.2 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.5605453650994447, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.567924660903788, + "metric_std": 0.0007520182219071431, + "actual_fraction": 0.009979619087778481, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.5744465528146743, + "metric_std": 0.0011920529630144716, + "actual_fraction": 0.019959238175556962, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.5802234872443601, + "metric_std": 0.0009122745790812356, + "actual_fraction": 0.029938857263335443, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.5857614730480005, + "metric_std": 0.0010534820904098856, + "actual_fraction": 0.03998875535877434, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.5906810035842295, + "metric_std": 0.0016243423314844835, + "actual_fraction": 0.04996837444655281, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.595403752899009, + "metric_std": 0.0014117161730691284, + "actual_fraction": 0.05994799353433129, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.5998734977862112, + "metric_std": 0.0015378100382719862, + "actual_fraction": 0.06999789162977019, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.6039215686274509, + "metric_std": 0.00214056982459815, + "actual_fraction": 0.07997751071754867, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.6077728582472416, + "metric_std": 0.002074123910798177, + "actual_fraction": 0.08995712980532715, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.6113992550425189, + "metric_std": 0.002392378351738703, + "actual_fraction": 0.09993674889310562, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.6151099866469885, + "metric_std": 0.0019831884227652415, + "actual_fraction": 0.10998664698854452, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.12, + "metric": 0.618244430388643, + "metric_std": 0.0017414451744133604, + "actual_fraction": 0.11996626607632302, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.13, + "metric": 0.6209853116873989, + "metric_std": 0.0017632889578036712, + "actual_fraction": 0.1299458851641015, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.6237683603907513, + "metric_std": 0.0024211092638632722, + "actual_fraction": 0.13998172745800827, + "actual_fraction_std": 3.142972770397948e-05 + }, + { + "budget": 0.15, + "metric": 0.6262702930634619, + "metric_std": 0.0021760405511914824, + "actual_fraction": 0.14997540234731885, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.6292922903928597, + "metric_std": 0.0023023214125117734, + "actual_fraction": 0.15995502143509735, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.6313725490196079, + "metric_std": 0.0021497795899398774, + "actual_fraction": 0.16993464052287582, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.6335793098601448, + "metric_std": 0.0019516823341299654, + "actual_fraction": 0.1799845386183147, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.6357579590976176, + "metric_std": 0.002093913469384757, + "actual_fraction": 0.18996415770609318, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.6374727668845316, + "metric_std": 0.0027012200804758285, + "actual_fraction": 0.19994377679387168, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.6393703000913628, + "metric_std": 0.0031592739553501135, + "actual_fraction": 0.20997961908777846, + "actual_fraction_std": 3.142972770397948e-05 + }, + { + "budget": 0.22, + "metric": 0.6409023824583596, + "metric_std": 0.0032532425167694926, + "actual_fraction": 0.219959238175557, + "actual_fraction_std": 3.1429727703979486e-05 + }, + { + "budget": 0.23, + "metric": 0.6423782416192283, + "metric_std": 0.003525737487129288, + "actual_fraction": 0.22993885726333546, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.24, + "metric": 0.6438400449785648, + "metric_std": 0.0037384865113447956, + "actual_fraction": 0.23993253215264604, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.25, + "metric": 0.6454564621547543, + "metric_std": 0.003302515186662492, + "actual_fraction": 0.24998243024808492, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.6468760981094948, + "metric_std": 0.002728509499836441, + "actual_fraction": 0.2599479935343313, + "actual_fraction_std": 3.142972770396707e-05 + }, + { + "budget": 0.27, + "metric": 0.6481692318504463, + "metric_std": 0.002975218440444267, + "actual_fraction": 0.26994166842364187, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.6490828589500317, + "metric_std": 0.00289699583172133, + "actual_fraction": 0.27999156651908075, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.6500667650572773, + "metric_std": 0.0028553522745014396, + "actual_fraction": 0.2899571298053271, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.3, + "metric": 0.651036615362991, + "metric_std": 0.0026952709548764445, + "actual_fraction": 0.2999508046946377, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.6515707358212102, + "metric_std": 0.002883752450503664, + "actual_fraction": 0.3099304237824162, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.6515145126150819, + "metric_std": 0.003196188515019942, + "actual_fraction": 0.3199803218778551, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.6521470236840257, + "metric_std": 0.003574777492634835, + "actual_fraction": 0.3299599409656336, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.652385972310071, + "metric_std": 0.0031210526707429443, + "actual_fraction": 0.339939560053412, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.6532293204019959, + "metric_std": 0.003307372213873363, + "actual_fraction": 0.3499613465457868, + "actual_fraction_std": 6.285945540798379e-05 + }, + { + "budget": 0.36, + "metric": 0.6540445568908566, + "metric_std": 0.0031211317959104795, + "actual_fraction": 0.3599690772366294, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.6541991707077096, + "metric_std": 0.0029486210738044836, + "actual_fraction": 0.3699346405228758, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.38, + "metric": 0.6545083983414155, + "metric_std": 0.0026305386565764, + "actual_fraction": 0.3799985944198468, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.6548597933797174, + "metric_std": 0.002752120914920419, + "actual_fraction": 0.3899782135076253, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.6552955232272121, + "metric_std": 0.002883324231911321, + "actual_fraction": 0.39995783259540374, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.6554501370440649, + "metric_std": 0.0024700846756891934, + "actual_fraction": 0.4099374516831823, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.42, + "metric": 0.6555766392578537, + "metric_std": 0.0024169235813326423, + "actual_fraction": 0.4199873497786212, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.65561880666245, + "metric_std": 0.0022715453830718275, + "actual_fraction": 0.42995291306486755, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.44, + "metric": 0.6554782486471291, + "metric_std": 0.0022704579468550176, + "actual_fraction": 0.439918476351114, + "actual_fraction_std": 3.849339781466781e-05 + }, + { + "budget": 0.45, + "metric": 0.6555063602501933, + "metric_std": 0.002060625567048607, + "actual_fraction": 0.44999648604961695, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.46, + "metric": 0.6554501370440649, + "metric_std": 0.0015232883717637062, + "actual_fraction": 0.4599761051373955, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.47, + "metric": 0.6555204160517254, + "metric_std": 0.001543420454711628, + "actual_fraction": 0.4699557242251739, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.48, + "metric": 0.6554782486471291, + "metric_std": 0.001171152798508496, + "actual_fraction": 0.4799353433129524, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.6553236348302762, + "metric_std": 0.0012498998518596283, + "actual_fraction": 0.4899852414083913, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.6549441281889099, + "metric_std": 0.0013095300806917963, + "actual_fraction": 0.49996486049616984, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.51, + "metric": 0.6546208447536721, + "metric_std": 0.00124296526639589, + "actual_fraction": 0.5099444795839483, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.6540164452877925, + "metric_std": 0.001314799840738613, + "actual_fraction": 0.5199943776793872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.6535385480357018, + "metric_std": 0.0015905429744783204, + "actual_fraction": 0.5299739967671656, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.6533277110127205, + "metric_std": 0.0013558567421302187, + "actual_fraction": 0.5399536158549442, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.6533698784173168, + "metric_std": 0.001451733645059054, + "actual_fraction": 0.5499191791411906, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.56, + "metric": 0.6531590413943356, + "metric_std": 0.0017607661680815315, + "actual_fraction": 0.5599831330381615, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.6526108651345843, + "metric_std": 0.0018034475255261978, + "actual_fraction": 0.5699627521259399, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.6524000281116031, + "metric_std": 0.0018567495696989085, + "actual_fraction": 0.5799423712137185, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.652034577271769, + "metric_std": 0.0021121151177186975, + "actual_fraction": 0.5899922693091574, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.6516410148288706, + "metric_std": 0.0023033938059801983, + "actual_fraction": 0.5999718883969358, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.6510647269660552, + "metric_std": 0.002200080939278716, + "actual_fraction": 0.6099515074847143, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.6506289971185607, + "metric_std": 0.002355343039218861, + "actual_fraction": 0.6199311265724928, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.6501089324618736, + "metric_std": 0.0025786746062547274, + "actual_fraction": 0.6299810246679317, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.6495045329959941, + "metric_std": 0.0027710790814046175, + "actual_fraction": 0.6399606437557102, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.6486752407056012, + "metric_std": 0.002770455175265588, + "actual_fraction": 0.6499402628434886, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.647944339025933, + "metric_std": 0.003086920911520195, + "actual_fraction": 0.6599901609389276, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.6471572141401364, + "metric_std": 0.0031235046188850947, + "actual_fraction": 0.669969780026706, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.6468058191018342, + "metric_std": 0.003102083872724563, + "actual_fraction": 0.6799493991144845, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.6464403682620002, + "metric_std": 0.0033382158747132518, + "actual_fraction": 0.6899992972099234, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.6462295312390189, + "metric_std": 0.0035431355304588593, + "actual_fraction": 0.6999789162977019, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.6459905826129735, + "metric_std": 0.003663080492946213, + "actual_fraction": 0.7099585353854804, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.6457797455899923, + "metric_std": 0.0038366159310811243, + "actual_fraction": 0.7199381544732588, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.6456391875746714, + "metric_std": 0.003991513548383514, + "actual_fraction": 0.7299880525686977, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.6455126853608827, + "metric_std": 0.004123467681975119, + "actual_fraction": 0.7399676716564763, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.6454142947501582, + "metric_std": 0.0040202418186590615, + "actual_fraction": 0.7499472907442547, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.6453721273455619, + "metric_std": 0.003952217963033942, + "actual_fraction": 0.7599971888396936, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.6453440157424978, + "metric_std": 0.003957400874354936, + "actual_fraction": 0.769976807927472, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.645386183147094, + "metric_std": 0.003991204183886103, + "actual_fraction": 0.7799564270152506, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.6453159041394335, + "metric_std": 0.003974773409409647, + "actual_fraction": 0.7899219903014969, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.8, + "metric": 0.6451753461241128, + "metric_std": 0.003961704398220247, + "actual_fraction": 0.7999859441984679, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.6450769555133882, + "metric_std": 0.003941706393620681, + "actual_fraction": 0.8099655632862464, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.6449082858950032, + "metric_std": 0.0038095500891801508, + "actual_fraction": 0.8199451823740249, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.6447677278796823, + "metric_std": 0.0038415048169174897, + "actual_fraction": 0.8299950804694639, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.644542835055169, + "metric_std": 0.0037444270116757036, + "actual_fraction": 0.8399746995572424, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.644374165436784, + "metric_std": 0.0037464050852143095, + "actual_fraction": 0.8499543186450207, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.644219551619931, + "metric_std": 0.0036886772548663863, + "actual_fraction": 0.8599339377327994, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.87, + "metric": 0.6440930494061424, + "metric_std": 0.003739477250082384, + "actual_fraction": 0.8699838358282381, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.88, + "metric": 0.6437557101693724, + "metric_std": 0.0034683242112202935, + "actual_fraction": 0.8799634549160166, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.89, + "metric": 0.6435308173448591, + "metric_std": 0.0034670423106857263, + "actual_fraction": 0.889943074003795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.6431231991004287, + "metric_std": 0.0034400838430024222, + "actual_fraction": 0.8999929720992339, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.91, + "metric": 0.642673413451402, + "metric_std": 0.0031969610824129882, + "actual_fraction": 0.9099725911870123, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.6420409023824583, + "metric_std": 0.0030070875495376128, + "actual_fraction": 0.919952210274791, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.93, + "metric": 0.6418581769625413, + "metric_std": 0.002865108877734164, + "actual_fraction": 0.9299318293625694, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.6417176189472205, + "metric_std": 0.0023430435136930906, + "actual_fraction": 0.9399817274580083, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.95, + "metric": 0.6412678332981938, + "metric_std": 0.0020601461287994578, + "actual_fraction": 0.9499613465457868, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.6410569962752126, + "metric_std": 0.0016229734393472285, + "actual_fraction": 0.9599409656335652, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.640958605664488, + "metric_std": 0.001134739404380051, + "actual_fraction": 0.9699908637290042, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.98, + "metric": 0.64097266146602, + "metric_std": 0.0009514957507105915, + "actual_fraction": 0.9799704828167826, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.6410007730690842, + "metric_std": 0.0006964362438309095, + "actual_fraction": 0.9899501019045612, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 1.0, + "metric": 0.6412256658935975, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.26688029946402747, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.28129200578510055, + "metric_std": 0.003548555980248032, + "actual_fraction": 0.017338286589342937, + "actual_fraction_std": 0.004421381977446603 + }, + { + "budget": 0.02, + "metric": 0.2940022119502027, + "metric_std": 0.0029263137097559535, + "actual_fraction": 0.03389388310693928, + "actual_fraction_std": 0.004102923008830257 + }, + { + "budget": 0.03, + "metric": 0.30544196466551343, + "metric_std": 0.00398882954103208, + "actual_fraction": 0.04907126449819924, + "actual_fraction_std": 0.005747500855567202 + }, + { + "budget": 0.04, + "metric": 0.3162408189887417, + "metric_std": 0.0043548100722564445, + "actual_fraction": 0.06443013923942943, + "actual_fraction_std": 0.0064234291383080885 + }, + { + "budget": 0.05, + "metric": 0.3259053398746562, + "metric_std": 0.00554420335469047, + "actual_fraction": 0.07889289056518164, + "actual_fraction_std": 0.008380188160589906 + }, + { + "budget": 0.06, + "metric": 0.33519553072625696, + "metric_std": 0.0045000935383466625, + "actual_fraction": 0.0934180302299861, + "actual_fraction_std": 0.00707773163476333 + }, + { + "budget": 0.07, + "metric": 0.3445651249184698, + "metric_std": 0.004448626670463828, + "actual_fraction": 0.10852167994782067, + "actual_fraction_std": 0.006979417999627261 + }, + { + "budget": 0.08, + "metric": 0.35349800073731674, + "metric_std": 0.004169620552842928, + "actual_fraction": 0.12368204633752092, + "actual_fraction_std": 0.0068099755381643735 + }, + { + "budget": 0.09, + "metric": 0.36146102146726033, + "metric_std": 0.004693899998499233, + "actual_fraction": 0.1379236026429969, + "actual_fraction_std": 0.008091836368646682 + }, + { + "budget": 0.1, + "metric": 0.36878881547230813, + "metric_std": 0.004872751064783166, + "actual_fraction": 0.15194963559538327, + "actual_fraction_std": 0.009144286024868133 + }, + { + "budget": 0.11, + "metric": 0.3759294444601991, + "metric_std": 0.004390718505850493, + "actual_fraction": 0.16571477185718742, + "actual_fraction_std": 0.0091510157464788 + }, + { + "budget": 0.12, + "metric": 0.3829055950996796, + "metric_std": 0.004923606592346998, + "actual_fraction": 0.17929841476902136, + "actual_fraction_std": 0.010449048237687776 + }, + { + "budget": 0.13, + "metric": 0.3889515923205626, + "metric_std": 0.004873411177099707, + "actual_fraction": 0.1917647392451011, + "actual_fraction_std": 0.010708480896318593 + }, + { + "budget": 0.14, + "metric": 0.3952811729007742, + "metric_std": 0.004944745707866216, + "actual_fraction": 0.20520091881008423, + "actual_fraction_std": 0.010485779920440199 + }, + { + "budget": 0.15, + "metric": 0.4010265717607691, + "metric_std": 0.005133665621263919, + "actual_fraction": 0.21804724498766412, + "actual_fraction_std": 0.011091318236736328 + }, + { + "budget": 0.16, + "metric": 0.4063352522473981, + "metric_std": 0.005212945600345818, + "actual_fraction": 0.23079148115588577, + "actual_fraction_std": 0.011475119253206765 + }, + { + "budget": 0.17, + "metric": 0.4119161727589825, + "metric_std": 0.0051075559518554935, + "actual_fraction": 0.243541388991294, + "actual_fraction_std": 0.011760233025472371 + }, + { + "budget": 0.18, + "metric": 0.4170547032300145, + "metric_std": 0.005194090699944122, + "actual_fraction": 0.2558886084564557, + "actual_fraction_std": 0.011583253823742867 + }, + { + "budget": 0.19, + "metric": 0.4220230836854494, + "metric_std": 0.004925353962289805, + "actual_fraction": 0.26822448458724446, + "actual_fraction_std": 0.01142861344654542 + }, + { + "budget": 0.2, + "metric": 0.4270028074752574, + "metric_std": 0.005221384989484401, + "actual_fraction": 0.2809290190851601, + "actual_fraction_std": 0.012172212429584344 + }, + { + "budget": 0.21, + "metric": 0.4319031279244534, + "metric_std": 0.004541071964887381, + "actual_fraction": 0.29411564529393414, + "actual_fraction_std": 0.011267855679390511 + }, + { + "budget": 0.22, + "metric": 0.43669001502991805, + "metric_std": 0.004613483792922728, + "actual_fraction": 0.3069676431387006, + "actual_fraction_std": 0.011265232499293218 + }, + { + "budget": 0.23, + "metric": 0.44134645379009163, + "metric_std": 0.004708483033885649, + "actual_fraction": 0.32002382100218363, + "actual_fraction_std": 0.010505701406982997 + }, + { + "budget": 0.24, + "metric": 0.44564557751751127, + "metric_std": 0.004507521591286498, + "actual_fraction": 0.3330629838641069, + "actual_fraction_std": 0.011062866356309714 + }, + { + "budget": 0.25, + "metric": 0.44961574454810993, + "metric_std": 0.004278369802910235, + "actual_fraction": 0.345523636673, + "actual_fraction_std": 0.010201759114329977 + }, + { + "budget": 0.26, + "metric": 0.4530868048662905, + "metric_std": 0.00419465707646832, + "actual_fraction": 0.35654368601650455, + "actual_fraction_std": 0.010336728202840642 + }, + { + "budget": 0.27, + "metric": 0.45674503020162777, + "metric_std": 0.003721560272809581, + "actual_fraction": 0.36826702209114365, + "actual_fraction_std": 0.010302439427074862 + }, + { + "budget": 0.28, + "metric": 0.46059609222130843, + "metric_std": 0.0033334617755784255, + "actual_fraction": 0.3807106598984772, + "actual_fraction_std": 0.009614667861282237 + }, + { + "budget": 0.29, + "metric": 0.4640898392082352, + "metric_std": 0.0029658806653807307, + "actual_fraction": 0.393228029379236, + "actual_fraction_std": 0.008318776584710358 + }, + { + "budget": 0.3, + "metric": 0.4672432861639678, + "metric_std": 0.002777423586506072, + "actual_fraction": 0.4050477837960469, + "actual_fraction_std": 0.008598456024503409 + }, + { + "budget": 0.31, + "metric": 0.47027195644159603, + "metric_std": 0.002712664202126277, + "actual_fraction": 0.41658395485352917, + "actual_fraction_std": 0.00863889590279397 + }, + { + "budget": 0.32, + "metric": 0.4729773416895897, + "metric_std": 0.002932079147467668, + "actual_fraction": 0.4274111675126903, + "actual_fraction_std": 0.009505086442754515 + }, + { + "budget": 0.33, + "metric": 0.4760457136375237, + "metric_std": 0.002847888621642436, + "actual_fraction": 0.4393443552732326, + "actual_fraction_std": 0.0093601541101824 + }, + { + "budget": 0.34, + "metric": 0.4787227405495845, + "metric_std": 0.0021877679658286555, + "actual_fraction": 0.4505175396307745, + "actual_fraction_std": 0.008107697341955144 + }, + { + "budget": 0.35, + "metric": 0.4815585741428693, + "metric_std": 0.002456264105522625, + "actual_fraction": 0.46214445736324195, + "actual_fraction_std": 0.008270226257885576 + }, + { + "budget": 0.36, + "metric": 0.4839066443581091, + "metric_std": 0.0027698415783285043, + "actual_fraction": 0.47280719167399254, + "actual_fraction_std": 0.00862155443986684 + }, + { + "budget": 0.37, + "metric": 0.486266057907722, + "metric_std": 0.002730761979295407, + "actual_fraction": 0.4838896293565494, + "actual_fraction_std": 0.00908116986221341 + }, + { + "budget": 0.38, + "metric": 0.4883759181011258, + "metric_std": 0.0027468528870937645, + "actual_fraction": 0.494144003629867, + "actual_fraction_std": 0.009078167305721392 + }, + { + "budget": 0.39, + "metric": 0.49050846496327594, + "metric_std": 0.0026346862912963696, + "actual_fraction": 0.5054986813373792, + "actual_fraction_std": 0.008151125050517082 + }, + { + "budget": 0.4, + "metric": 0.4924084734707767, + "metric_std": 0.002466474317144588, + "actual_fraction": 0.5157530556106968, + "actual_fraction_std": 0.007599002266776645 + }, + { + "budget": 0.41, + "metric": 0.4942857953095313, + "metric_std": 0.0024936799142084003, + "actual_fraction": 0.5268638516291864, + "actual_fraction_std": 0.007799300285683034 + }, + { + "budget": 0.42, + "metric": 0.4960780421404872, + "metric_std": 0.002519379206921931, + "actual_fraction": 0.5379292743101833, + "actual_fraction_std": 0.008362512168426373 + }, + { + "budget": 0.43, + "metric": 0.4977681989620849, + "metric_std": 0.0024910502067383568, + "actual_fraction": 0.548592008620934, + "actual_fraction_std": 0.008005166953581262 + }, + { + "budget": 0.44, + "metric": 0.49929954910245866, + "metric_std": 0.0026118473857080528, + "actual_fraction": 0.5593228029379237, + "actual_fraction_std": 0.008112853535674218 + }, + { + "budget": 0.45, + "metric": 0.5007628392365936, + "metric_std": 0.0023770815830268656, + "actual_fraction": 0.5699458355783683, + "actual_fraction_std": 0.008960740562554317 + }, + { + "budget": 0.46, + "metric": 0.5021297110285567, + "metric_std": 0.002260997159366677, + "actual_fraction": 0.5797521481439469, + "actual_fraction_std": 0.00937356471957802 + }, + { + "budget": 0.47, + "metric": 0.5035419561580127, + "metric_std": 0.002404663150616728, + "actual_fraction": 0.5912713041998696, + "actual_fraction_std": 0.00891279986456672 + }, + { + "budget": 0.48, + "metric": 0.5048067379406177, + "metric_std": 0.0025408024526723865, + "actual_fraction": 0.6016277684825455, + "actual_fraction_std": 0.00866388701377314 + }, + { + "budget": 0.49, + "metric": 0.5060771913904093, + "metric_std": 0.00256884890391671, + "actual_fraction": 0.61167229106996, + "actual_fraction_std": 0.008748594894879228 + }, + { + "budget": 0.5, + "metric": 0.50720018149335, + "metric_std": 0.0027150496526747348, + "actual_fraction": 0.6229135354337407, + "actual_fraction_std": 0.009059886971852844 + }, + { + "budget": 0.51, + "metric": 0.5080849615744548, + "metric_std": 0.002766035512128331, + "actual_fraction": 0.6336102997476109, + "actual_fraction_std": 0.009166631956773844 + }, + { + "budget": 0.52, + "metric": 0.5088392933102684, + "metric_std": 0.00244453132482935, + "actual_fraction": 0.6439497490287269, + "actual_fraction_std": 0.009354331430651652 + }, + { + "budget": 0.53, + "metric": 0.5095539233757762, + "metric_std": 0.002583302763563939, + "actual_fraction": 0.6547032300144627, + "actual_fraction_std": 0.00905966505819418 + }, + { + "budget": 0.54, + "metric": 0.5100757167569407, + "metric_std": 0.0024319153899715422, + "actual_fraction": 0.665082380965885, + "actual_fraction_std": 0.010151143529375961 + }, + { + "budget": 0.55, + "metric": 0.5107279584833962, + "metric_std": 0.002232253212078163, + "actual_fraction": 0.6750645152142474, + "actual_fraction_std": 0.00997793197968431 + }, + { + "budget": 0.56, + "metric": 0.5112100501942546, + "metric_std": 0.002432659314917841, + "actual_fraction": 0.6858747128718488, + "actual_fraction_std": 0.009819705972299518 + }, + { + "budget": 0.57, + "metric": 0.5117885602472847, + "metric_std": 0.0023429031759463342, + "actual_fraction": 0.6967586422028755, + "actual_fraction_std": 0.010222686475901856 + }, + { + "budget": 0.58, + "metric": 0.5120948302753594, + "metric_std": 0.0024435112825046223, + "actual_fraction": 0.707160479823044, + "actual_fraction_std": 0.012024479518544422 + }, + { + "budget": 0.59, + "metric": 0.512548563650285, + "metric_std": 0.002335649428682973, + "actual_fraction": 0.7188044125570711, + "actual_fraction_std": 0.01172385534858387 + }, + { + "budget": 0.6, + "metric": 0.5130476703627032, + "metric_std": 0.0022850555144590763, + "actual_fraction": 0.729807446899016, + "actual_fraction_std": 0.012687216513646293 + }, + { + "budget": 0.61, + "metric": 0.5131951337095539, + "metric_std": 0.002275957431712845, + "actual_fraction": 0.7401412245129456, + "actual_fraction_std": 0.01319859835933791 + }, + { + "budget": 0.62, + "metric": 0.5133822987267107, + "metric_std": 0.0020884908468353077, + "actual_fraction": 0.7513824688767263, + "actual_fraction_std": 0.012474718839543745 + }, + { + "budget": 0.63, + "metric": 0.5134220003970167, + "metric_std": 0.002069517447749362, + "actual_fraction": 0.7633156566372685, + "actual_fraction_std": 0.01389337653264658 + }, + { + "budget": 0.64, + "metric": 0.5133142387204719, + "metric_std": 0.0020554414542574995, + "actual_fraction": 0.7744945126619969, + "actual_fraction_std": 0.014043504932618843 + }, + { + "budget": 0.65, + "metric": 0.5132688653829793, + "metric_std": 0.0019537020647679604, + "actual_fraction": 0.7852082919774267, + "actual_fraction_std": 0.01339856198621453 + }, + { + "budget": 0.66, + "metric": 0.5133028953860987, + "metric_std": 0.0020010820181621137, + "actual_fraction": 0.7944474378243485, + "actual_fraction_std": 0.01285135832710495 + }, + { + "budget": 0.67, + "metric": 0.5134106570626435, + "metric_std": 0.0018694575613043762, + "actual_fraction": 0.8030059836088818, + "actual_fraction_std": 0.011861644806916706 + }, + { + "budget": 0.68, + "metric": 0.5133085670532853, + "metric_std": 0.0017348952239883528, + "actual_fraction": 0.811581544394975, + "actual_fraction_std": 0.009710574998377682 + }, + { + "budget": 0.69, + "metric": 0.5132178203783002, + "metric_std": 0.001640695782119038, + "actual_fraction": 0.8250630972974505, + "actual_fraction_std": 0.015478319030096268 + }, + { + "budget": 0.7, + "metric": 0.513155432039248, + "metric_std": 0.0016140103983680366, + "actual_fraction": 0.8346765731786858, + "actual_fraction_std": 0.017293544769020435 + }, + { + "budget": 0.71, + "metric": 0.513076028698636, + "metric_std": 0.001592946065572261, + "actual_fraction": 0.8429798939398235, + "actual_fraction_std": 0.018505666630767515 + }, + { + "budget": 0.72, + "metric": 0.5129796103564643, + "metric_std": 0.0015840612095842943, + "actual_fraction": 0.8524799364773274, + "actual_fraction_std": 0.018310582458434897 + }, + { + "budget": 0.73, + "metric": 0.5129852820236509, + "metric_std": 0.0015508665495926372, + "actual_fraction": 0.8610554972634205, + "actual_fraction_std": 0.019275582528158065 + }, + { + "budget": 0.74, + "metric": 0.5129399086861584, + "metric_std": 0.0016058682392929337, + "actual_fraction": 0.8676403028670278, + "actual_fraction_std": 0.01861897638941408 + }, + { + "budget": 0.75, + "metric": 0.5128605053455464, + "metric_std": 0.0016257267852492739, + "actual_fraction": 0.8735445084082466, + "actual_fraction_std": 0.017500498697181694 + }, + { + "budget": 0.76, + "metric": 0.5128151320080538, + "metric_std": 0.0016626064534286096, + "actual_fraction": 0.8786660238777187, + "actual_fraction_std": 0.016354594330330232 + }, + { + "budget": 0.77, + "metric": 0.5127697586705613, + "metric_std": 0.0016555537255466067, + "actual_fraction": 0.8836797776706463, + "actual_fraction_std": 0.014291340884092426 + }, + { + "budget": 0.78, + "metric": 0.5127414003346284, + "metric_std": 0.0016385867601680663, + "actual_fraction": 0.8889600998213425, + "actual_fraction_std": 0.01147051045962194 + }, + { + "budget": 0.79, + "metric": 0.5126733403283895, + "metric_std": 0.0015920623368961065, + "actual_fraction": 0.8940759436236281, + "actual_fraction_std": 0.009570852251631979 + }, + { + "budget": 0.8, + "metric": 0.5126563253268298, + "metric_std": 0.0016124899922843844, + "actual_fraction": 0.8985282023650851, + "actual_fraction_std": 0.007747469350629403 + }, + { + "budget": 0.81, + "metric": 0.5126052803221507, + "metric_std": 0.0015670386385612262, + "actual_fraction": 0.902997476108102, + "actual_fraction_std": 0.0064087204087449135 + }, + { + "budget": 0.82, + "metric": 0.5124748319768595, + "metric_std": 0.0014610186050031048, + "actual_fraction": 0.9075177948557979, + "actual_fraction_std": 0.005628376029085234 + }, + { + "budget": 0.83, + "metric": 0.5123840853018744, + "metric_std": 0.001408584551628672, + "actual_fraction": 0.9109037801661799, + "actual_fraction_std": 0.005433578529951563 + }, + { + "budget": 0.84, + "metric": 0.5123954286362475, + "metric_std": 0.0014257786185905173, + "actual_fraction": 0.91419334713439, + "actual_fraction_std": 0.0054737612328589836 + }, + { + "budget": 0.85, + "metric": 0.5122649802909565, + "metric_std": 0.0014005113830208337, + "actual_fraction": 0.9169951507245555, + "actual_fraction_std": 0.005205319139566468 + }, + { + "budget": 0.86, + "metric": 0.5122422936222102, + "metric_std": 0.0013733166505316876, + "actual_fraction": 0.9200862093412358, + "actual_fraction_std": 0.005121290921801161 + }, + { + "budget": 0.87, + "metric": 0.5121572186144117, + "metric_std": 0.0014101537195567683, + "actual_fraction": 0.923143237954797, + "actual_fraction_std": 0.005134950000974088 + }, + { + "budget": 0.88, + "metric": 0.5121742336159714, + "metric_std": 0.0014322223080571114, + "actual_fraction": 0.9261378782293054, + "actual_fraction_std": 0.004806532986622673 + }, + { + "budget": 0.89, + "metric": 0.5119246802597623, + "metric_std": 0.0013964860904909077, + "actual_fraction": 0.9294671468678217, + "actual_fraction_std": 0.004795469232863655 + }, + { + "budget": 0.9, + "metric": 0.5117999035816578, + "metric_std": 0.0013791892351982644, + "actual_fraction": 0.9328928338485097, + "actual_fraction_std": 0.004849383238163038 + }, + { + "budget": 0.91, + "metric": 0.5115957235629414, + "metric_std": 0.0012175274301942267, + "actual_fraction": 0.9367892692056831, + "actual_fraction_std": 0.0053162993519596915 + }, + { + "budget": 0.92, + "metric": 0.5114709468848369, + "metric_std": 0.0011678677527138265, + "actual_fraction": 0.9406289878909906, + "actual_fraction_std": 0.005891153362512259 + }, + { + "budget": 0.93, + "metric": 0.5112894535348665, + "metric_std": 0.0009225398625665758, + "actual_fraction": 0.9448940816152908, + "actual_fraction_std": 0.006219028976010971 + }, + { + "budget": 0.94, + "metric": 0.5111476618552023, + "metric_std": 0.0007488426568582936, + "actual_fraction": 0.9494767887020391, + "actual_fraction_std": 0.0061744570376664676 + }, + { + "budget": 0.95, + "metric": 0.5108130334911947, + "metric_std": 0.0005459992370850681, + "actual_fraction": 0.95486487252928, + "actual_fraction_std": 0.005544457189069127 + }, + { + "budget": 0.96, + "metric": 0.5103706434506423, + "metric_std": 0.00043629463420444297, + "actual_fraction": 0.9607237047330063, + "actual_fraction_std": 0.004637996955445372 + }, + { + "budget": 0.97, + "metric": 0.5097013867226271, + "metric_std": 0.00026798252343651787, + "actual_fraction": 0.9677792587130988, + "actual_fraction_std": 0.004074699153206785 + }, + { + "budget": 0.98, + "metric": 0.5085330232821937, + "metric_std": 0.0002923798070222552, + "actual_fraction": 0.975668547769617, + "actual_fraction_std": 0.0034207959016614897 + }, + { + "budget": 0.99, + "metric": 0.5066273431075065, + "metric_std": 0.0002409624341235268, + "actual_fraction": 0.9843802285681876, + "actual_fraction_std": 0.0016210702360476125 + }, + { + "budget": 1.0, + "metric": 0.5000708958398321, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 6535, + "rescuable": 2589, + "harmful": 1441, + "both_wrong": 3664, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 6911, + "rescuable": 10723, + "harmful": 2500, + "both_wrong": 15129, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.5605453650994447, + "test": 0.26688029946402747 + }, + "large": { + "validation": 0.6412256658935975, + "test": 0.5000708958398321 + }, + "learned_minus_confidence_test": 0.002297025210560688, + "learned_minus_small_test": 0.2322547712900207, + "learned_minus_large_test": -0.0009358250857839367, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_groundingdino_MiniCPM" + }, + { + "key": [ + "grefcoco", + "groundingdino", + "Qwen3.8" + ], + "expert_display": "GroundingDINO", + "run": "five_seed_groundingdino_Qwen3.8_four_difference", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "max_score", + "mean_score", + "query_count", + "detection_count_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/groundingdino/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 99, + "validation_area_0_25": 0.5648886077728582, + "validation_point": { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.7994237121371847, + "rescued": 4531, + "harmed": 1132, + "missed_rescue": 95, + "avoided_harm": 1, + "selected_both_correct": 6840, + "selected_both_wrong": 1583 + }, + "test_point": { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9873805405098829, + "metric": 0.7387062927147435, + "rescued": 18604, + "harmed": 1966, + "missed_rescue": 216, + "avoided_harm": 4, + "selected_both_correct": 7360, + "selected_both_wrong": 6888 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.5605453650994447, + "rescued": 0, + "harmed": 0, + "missed_rescue": 4626, + "avoided_harm": 1133, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.1584138125181198, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.5602642490688031, + "rescued": 0, + "harmed": 4, + "missed_rescue": 4626, + "avoided_harm": 1129, + "selected_both_correct": 138, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.21403978765010834, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.5601939700611427, + "rescued": 0, + "harmed": 5, + "missed_rescue": 4626, + "avoided_harm": 1128, + "selected_both_correct": 279, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.24253316223621368, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.5596317379998594, + "rescued": 0, + "harmed": 13, + "missed_rescue": 4626, + "avoided_harm": 1120, + "selected_both_correct": 410, + "selected_both_wrong": 3 + }, + { + "budget": 0.04, + "threshold": -0.2599165141582489, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.5594209009768782, + "rescued": 0, + "harmed": 16, + "missed_rescue": 4626, + "avoided_harm": 1117, + "selected_both_correct": 549, + "selected_both_wrong": 4 + }, + { + "budget": 0.05, + "threshold": -0.2725968658924103, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.5591397849462365, + "rescued": 3, + "harmed": 23, + "missed_rescue": 4623, + "avoided_harm": 1110, + "selected_both_correct": 675, + "selected_both_wrong": 10 + }, + { + "budget": 0.06, + "threshold": -0.2842792272567749, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.5588586689155949, + "rescued": 5, + "harmed": 29, + "missed_rescue": 4621, + "avoided_harm": 1104, + "selected_both_correct": 805, + "selected_both_wrong": 14 + }, + { + "budget": 0.07, + "threshold": -0.2935555577278137, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.5586478318926137, + "rescued": 12, + "harmed": 39, + "missed_rescue": 4614, + "avoided_harm": 1094, + "selected_both_correct": 926, + "selected_both_wrong": 19 + }, + { + "budget": 0.08, + "threshold": -0.30114513635635376, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.5589289479232553, + "rescued": 19, + "harmed": 42, + "missed_rescue": 4607, + "avoided_harm": 1091, + "selected_both_correct": 1053, + "selected_both_wrong": 24 + }, + { + "budget": 0.09, + "threshold": -0.308601438999176, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.5594209009768782, + "rescued": 34, + "harmed": 50, + "missed_rescue": 4592, + "avoided_harm": 1083, + "selected_both_correct": 1165, + "selected_both_wrong": 31 + }, + { + "budget": 0.1, + "threshold": -0.3146189749240875, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.559561458992199, + "rescued": 48, + "harmed": 62, + "missed_rescue": 4578, + "avoided_harm": 1071, + "selected_both_correct": 1275, + "selected_both_wrong": 37 + }, + { + "budget": 0.11, + "threshold": -0.32043084502220154, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.5599831330381615, + "rescued": 64, + "harmed": 72, + "missed_rescue": 4562, + "avoided_harm": 1061, + "selected_both_correct": 1382, + "selected_both_wrong": 47 + }, + { + "budget": 0.12, + "threshold": -0.32644838094711304, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.5604750860917844, + "rescued": 84, + "harmed": 85, + "missed_rescue": 4542, + "avoided_harm": 1048, + "selected_both_correct": 1485, + "selected_both_wrong": 53 + }, + { + "budget": 0.13, + "threshold": -0.33121857047080994, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.5616698292220114, + "rescued": 106, + "harmed": 90, + "missed_rescue": 4520, + "avoided_harm": 1043, + "selected_both_correct": 1592, + "selected_both_wrong": 61 + }, + { + "budget": 0.14, + "threshold": -0.33570271730422974, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.562794293344578, + "rescued": 126, + "harmed": 94, + "missed_rescue": 4500, + "avoided_harm": 1039, + "selected_both_correct": 1698, + "selected_both_wrong": 74 + }, + { + "budget": 0.15, + "threshold": -0.3407592475414276, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.5637781994518237, + "rescued": 151, + "harmed": 105, + "missed_rescue": 4475, + "avoided_harm": 1028, + "selected_both_correct": 1795, + "selected_both_wrong": 83 + }, + { + "budget": 0.16, + "threshold": -0.34456855058670044, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.5646918265514091, + "rescued": 180, + "harmed": 121, + "missed_rescue": 4446, + "avoided_harm": 1012, + "selected_both_correct": 1888, + "selected_both_wrong": 87 + }, + { + "budget": 0.17, + "threshold": -0.3487326204776764, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.5665190807505798, + "rescued": 211, + "harmed": 126, + "missed_rescue": 4415, + "avoided_harm": 1007, + "selected_both_correct": 1981, + "selected_both_wrong": 100 + }, + { + "budget": 0.18, + "threshold": -0.35280776023864746, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.5680652189191089, + "rescued": 242, + "harmed": 135, + "missed_rescue": 4384, + "avoided_harm": 998, + "selected_both_correct": 2070, + "selected_both_wrong": 114 + }, + { + "budget": 0.19, + "threshold": -0.3564819097518921, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.5697519151029588, + "rescued": 276, + "harmed": 145, + "missed_rescue": 4350, + "avoided_harm": 988, + "selected_both_correct": 2159, + "selected_both_wrong": 123 + }, + { + "budget": 0.2, + "threshold": -0.35992002487182617, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.5714386112868086, + "rescued": 309, + "harmed": 154, + "missed_rescue": 4317, + "avoided_harm": 979, + "selected_both_correct": 2246, + "selected_both_wrong": 136 + }, + { + "budget": 0.21, + "threshold": -0.36340460181236267, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.5737578185396022, + "rescued": 351, + "harmed": 163, + "missed_rescue": 4275, + "avoided_harm": 970, + "selected_both_correct": 2328, + "selected_both_wrong": 146 + }, + { + "budget": 0.22, + "threshold": -0.3674520254135132, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.5751633986928104, + "rescued": 384, + "harmed": 176, + "missed_rescue": 4242, + "avoided_harm": 957, + "selected_both_correct": 2410, + "selected_both_wrong": 160 + }, + { + "budget": 0.23, + "threshold": -0.37108901143074036, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.5764986998383583, + "rescued": 414, + "harmed": 187, + "missed_rescue": 4212, + "avoided_harm": 946, + "selected_both_correct": 2499, + "selected_both_wrong": 172 + }, + { + "budget": 0.24, + "threshold": -0.3745429813861847, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.578325954037529, + "rescued": 452, + "harmed": 199, + "missed_rescue": 4174, + "avoided_harm": 934, + "selected_both_correct": 2577, + "selected_both_wrong": 186 + }, + { + "budget": 0.25, + "threshold": -0.3773327171802521, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.5805046032750018, + "rescued": 490, + "harmed": 206, + "missed_rescue": 4136, + "avoided_harm": 927, + "selected_both_correct": 2664, + "selected_both_wrong": 197 + }, + { + "budget": 0.26, + "threshold": -0.3803459703922272, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.5829643685431162, + "rescued": 536, + "harmed": 217, + "missed_rescue": 4090, + "avoided_harm": 916, + "selected_both_correct": 2736, + "selected_both_wrong": 210 + }, + { + "budget": 0.27, + "threshold": -0.38361650705337524, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.584651064726966, + "rescued": 575, + "harmed": 232, + "missed_rescue": 4051, + "avoided_harm": 901, + "selected_both_correct": 2811, + "selected_both_wrong": 223 + }, + { + "budget": 0.28, + "threshold": -0.38652583956718445, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.5876730620563637, + "rescued": 626, + "harmed": 240, + "missed_rescue": 4000, + "avoided_harm": 893, + "selected_both_correct": 2882, + "selected_both_wrong": 236 + }, + { + "budget": 0.29, + "threshold": -0.38952407240867615, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.5904842223627802, + "rescued": 676, + "harmed": 250, + "missed_rescue": 3950, + "avoided_harm": 883, + "selected_both_correct": 2954, + "selected_both_wrong": 246 + }, + { + "budget": 0.3, + "threshold": -0.39262744784355164, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.5935764986998383, + "rescued": 726, + "harmed": 256, + "missed_rescue": 3900, + "avoided_harm": 877, + "selected_both_correct": 3029, + "selected_both_wrong": 257 + }, + { + "budget": 0.31, + "threshold": -0.3954668939113617, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.5956145899219903, + "rescued": 765, + "harmed": 266, + "missed_rescue": 3861, + "avoided_harm": 867, + "selected_both_correct": 3104, + "selected_both_wrong": 275 + }, + { + "budget": 0.32, + "threshold": -0.3986983895301819, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.5973715651135006, + "rescued": 804, + "harmed": 280, + "missed_rescue": 3822, + "avoided_harm": 853, + "selected_both_correct": 3178, + "selected_both_wrong": 291 + }, + { + "budget": 0.33, + "threshold": -0.4014112949371338, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.5994096563356526, + "rescued": 849, + "harmed": 296, + "missed_rescue": 3777, + "avoided_harm": 837, + "selected_both_correct": 3248, + "selected_both_wrong": 302 + }, + { + "budget": 0.34, + "threshold": -0.4042409658432007, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.6013774685501441, + "rescued": 890, + "harmed": 309, + "missed_rescue": 3736, + "avoided_harm": 824, + "selected_both_correct": 3323, + "selected_both_wrong": 315 + }, + { + "budget": 0.35, + "threshold": -0.40724706649780273, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.6035561177876169, + "rescued": 932, + "harmed": 320, + "missed_rescue": 3694, + "avoided_harm": 813, + "selected_both_correct": 3396, + "selected_both_wrong": 332 + }, + { + "budget": 0.36, + "threshold": -0.40968337655067444, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.6046805819101835, + "rescued": 965, + "harmed": 337, + "missed_rescue": 3661, + "avoided_harm": 796, + "selected_both_correct": 3475, + "selected_both_wrong": 345 + }, + { + "budget": 0.37, + "threshold": -0.4123799502849579, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.6079836952702228, + "rescued": 1019, + "harmed": 344, + "missed_rescue": 3607, + "avoided_harm": 789, + "selected_both_correct": 3546, + "selected_both_wrong": 355 + }, + { + "budget": 0.38, + "threshold": -0.41514647006988525, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.611778761683885, + "rescued": 1079, + "harmed": 350, + "missed_rescue": 3547, + "avoided_harm": 783, + "selected_both_correct": 3611, + "selected_both_wrong": 367 + }, + { + "budget": 0.39, + "threshold": -0.4179389476776123, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.6131843418370932, + "rescued": 1121, + "harmed": 372, + "missed_rescue": 3505, + "avoided_harm": 761, + "selected_both_correct": 3675, + "selected_both_wrong": 381 + }, + { + "budget": 0.4, + "threshold": -0.42099592089653015, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.6156441071052077, + "rescued": 1166, + "harmed": 382, + "missed_rescue": 3460, + "avoided_harm": 751, + "selected_both_correct": 3747, + "selected_both_wrong": 396 + }, + { + "budget": 0.41, + "threshold": -0.4243226647377014, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.617401082296718, + "rescued": 1208, + "harmed": 399, + "missed_rescue": 3418, + "avoided_harm": 734, + "selected_both_correct": 3815, + "selected_both_wrong": 411 + }, + { + "budget": 0.42, + "threshold": -0.4275209903717041, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.6203528006184552, + "rescued": 1261, + "harmed": 410, + "missed_rescue": 3365, + "avoided_harm": 723, + "selected_both_correct": 3878, + "selected_both_wrong": 427 + }, + { + "budget": 0.43, + "threshold": -0.4307159185409546, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.6228828448942301, + "rescued": 1310, + "harmed": 423, + "missed_rescue": 3316, + "avoided_harm": 710, + "selected_both_correct": 3949, + "selected_both_wrong": 436 + }, + { + "budget": 0.44, + "threshold": -0.4336964190006256, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.625764284208307, + "rescued": 1364, + "harmed": 436, + "missed_rescue": 3262, + "avoided_harm": 697, + "selected_both_correct": 4015, + "selected_both_wrong": 445 + }, + { + "budget": 0.45, + "threshold": -0.4373021125793457, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.627661817415138, + "rescued": 1410, + "harmed": 455, + "missed_rescue": 3216, + "avoided_harm": 678, + "selected_both_correct": 4077, + "selected_both_wrong": 461 + }, + { + "budget": 0.46, + "threshold": -0.4404309391975403, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.6299107456602713, + "rescued": 1459, + "harmed": 472, + "missed_rescue": 3167, + "avoided_harm": 661, + "selected_both_correct": 4140, + "selected_both_wrong": 474 + }, + { + "budget": 0.47, + "threshold": -0.44344601035118103, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.6319488368824232, + "rescued": 1501, + "harmed": 485, + "missed_rescue": 3125, + "avoided_harm": 648, + "selected_both_correct": 4210, + "selected_both_wrong": 491 + }, + { + "budget": 0.48, + "threshold": -0.4465356171131134, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.6338463700892544, + "rescued": 1545, + "harmed": 502, + "missed_rescue": 3081, + "avoided_harm": 631, + "selected_both_correct": 4281, + "selected_both_wrong": 501 + }, + { + "budget": 0.49, + "threshold": -0.44965147972106934, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.6358844613114063, + "rescued": 1592, + "harmed": 520, + "missed_rescue": 3034, + "avoided_harm": 613, + "selected_both_correct": 4344, + "selected_both_wrong": 516 + }, + { + "budget": 0.5, + "threshold": -0.453143447637558, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.6390470166561248, + "rescued": 1648, + "harmed": 531, + "missed_rescue": 2978, + "avoided_harm": 602, + "selected_both_correct": 4405, + "selected_both_wrong": 530 + }, + { + "budget": 0.51, + "threshold": -0.4565357565879822, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.6419987349778621, + "rescued": 1702, + "harmed": 543, + "missed_rescue": 2924, + "avoided_harm": 590, + "selected_both_correct": 4469, + "selected_both_wrong": 542 + }, + { + "budget": 0.52, + "threshold": -0.45961302518844604, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.6445990582612974, + "rescued": 1747, + "harmed": 551, + "missed_rescue": 2879, + "avoided_harm": 582, + "selected_both_correct": 4543, + "selected_both_wrong": 558 + }, + { + "budget": 0.53, + "threshold": -0.46368208527565, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.6468479865064305, + "rescued": 1795, + "harmed": 567, + "missed_rescue": 2831, + "avoided_harm": 566, + "selected_both_correct": 4608, + "selected_both_wrong": 571 + }, + { + "budget": 0.54, + "threshold": -0.46793216466903687, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.6500808208588095, + "rescued": 1850, + "harmed": 576, + "missed_rescue": 2776, + "avoided_harm": 557, + "selected_both_correct": 4672, + "selected_both_wrong": 585 + }, + { + "budget": 0.55, + "threshold": -0.4718930721282959, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.653243376203528, + "rescued": 1906, + "harmed": 587, + "missed_rescue": 2720, + "avoided_harm": 546, + "selected_both_correct": 4731, + "selected_both_wrong": 601 + }, + { + "budget": 0.56, + "threshold": -0.47629430890083313, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.654438119333755, + "rescued": 1942, + "harmed": 606, + "missed_rescue": 2684, + "avoided_harm": 527, + "selected_both_correct": 4800, + "selected_both_wrong": 620 + }, + { + "budget": 0.57, + "threshold": -0.4799485504627228, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.6574601166631527, + "rescued": 1998, + "harmed": 619, + "missed_rescue": 2628, + "avoided_harm": 514, + "selected_both_correct": 4861, + "selected_both_wrong": 632 + }, + { + "budget": 0.58, + "threshold": -0.4838995635509491, + "operator": "ge", + "actual_fraction": 0.5799423712137185, + "metric": 0.6599901609389276, + "rescued": 2048, + "harmed": 633, + "missed_rescue": 2578, + "avoided_harm": 500, + "selected_both_correct": 4921, + "selected_both_wrong": 650 + }, + { + "budget": 0.59, + "threshold": -0.4886912405490875, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.6618876941457587, + "rescued": 2098, + "harmed": 656, + "missed_rescue": 2528, + "avoided_harm": 477, + "selected_both_correct": 4979, + "selected_both_wrong": 662 + }, + { + "budget": 0.6, + "threshold": -0.49332454800605774, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.6642771804062126, + "rescued": 2146, + "harmed": 670, + "missed_rescue": 2480, + "avoided_harm": 463, + "selected_both_correct": 5049, + "selected_both_wrong": 672 + }, + { + "budget": 0.61, + "threshold": -0.49750789999961853, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.6670883407126291, + "rescued": 2199, + "harmed": 683, + "missed_rescue": 2427, + "avoided_harm": 450, + "selected_both_correct": 5115, + "selected_both_wrong": 682 + }, + { + "budget": 0.62, + "threshold": -0.5024909377098083, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.668775036896479, + "rescued": 2238, + "harmed": 698, + "missed_rescue": 2388, + "avoided_harm": 435, + "selected_both_correct": 5185, + "selected_both_wrong": 700 + }, + { + "budget": 0.63, + "threshold": -0.5067122578620911, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.6707428491109706, + "rescued": 2280, + "harmed": 712, + "missed_rescue": 2346, + "avoided_harm": 421, + "selected_both_correct": 5258, + "selected_both_wrong": 714 + }, + { + "budget": 0.64, + "threshold": -0.512323796749115, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.6732728933867453, + "rescued": 2329, + "harmed": 725, + "missed_rescue": 2297, + "avoided_harm": 408, + "selected_both_correct": 5325, + "selected_both_wrong": 727 + }, + { + "budget": 0.65, + "threshold": -0.5170255303382874, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.675592100639539, + "rescued": 2377, + "harmed": 740, + "missed_rescue": 2249, + "avoided_harm": 393, + "selected_both_correct": 5386, + "selected_both_wrong": 745 + }, + { + "budget": 0.66, + "threshold": -0.5215396881103516, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.6772787968233889, + "rescued": 2417, + "harmed": 756, + "missed_rescue": 2209, + "avoided_harm": 377, + "selected_both_correct": 5453, + "selected_both_wrong": 765 + }, + { + "budget": 0.67, + "threshold": -0.5264327526092529, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.6788952139995783, + "rescued": 2457, + "harmed": 773, + "missed_rescue": 2169, + "avoided_harm": 360, + "selected_both_correct": 5522, + "selected_both_wrong": 781 + }, + { + "budget": 0.68, + "threshold": -0.5312389135360718, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.6810738632370511, + "rescued": 2504, + "harmed": 789, + "missed_rescue": 2122, + "avoided_harm": 344, + "selected_both_correct": 5585, + "selected_both_wrong": 797 + }, + { + "budget": 0.69, + "threshold": -0.5368650555610657, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.6833930704898447, + "rescued": 2550, + "harmed": 802, + "missed_rescue": 2076, + "avoided_harm": 331, + "selected_both_correct": 5652, + "selected_both_wrong": 814 + }, + { + "budget": 0.7, + "threshold": -0.5431069135665894, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.6867664628575445, + "rescued": 2614, + "harmed": 818, + "missed_rescue": 2012, + "avoided_harm": 315, + "selected_both_correct": 5698, + "selected_both_wrong": 830 + }, + { + "budget": 0.71, + "threshold": -0.5487136244773865, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.6895073441563006, + "rescued": 2664, + "harmed": 829, + "missed_rescue": 1962, + "avoided_harm": 304, + "selected_both_correct": 5762, + "selected_both_wrong": 847 + }, + { + "budget": 0.72, + "threshold": -0.5549414753913879, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.6909129243095088, + "rescued": 2704, + "harmed": 849, + "missed_rescue": 1922, + "avoided_harm": 284, + "selected_both_correct": 5830, + "selected_both_wrong": 861 + }, + { + "budget": 0.73, + "threshold": -0.5604677200317383, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.6918265514090941, + "rescued": 2739, + "harmed": 871, + "missed_rescue": 1887, + "avoided_harm": 262, + "selected_both_correct": 5903, + "selected_both_wrong": 874 + }, + { + "budget": 0.74, + "threshold": -0.5666090250015259, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.6940754796542273, + "rescued": 2787, + "harmed": 887, + "missed_rescue": 1839, + "avoided_harm": 246, + "selected_both_correct": 5960, + "selected_both_wrong": 895 + }, + { + "budget": 0.75, + "threshold": -0.574927568435669, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.6962541288917, + "rescued": 2831, + "harmed": 900, + "missed_rescue": 1795, + "avoided_harm": 233, + "selected_both_correct": 6025, + "selected_both_wrong": 915 + }, + { + "budget": 0.76, + "threshold": -0.5812664031982422, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.6987841731674749, + "rescued": 2879, + "harmed": 912, + "missed_rescue": 1747, + "avoided_harm": 221, + "selected_both_correct": 6094, + "selected_both_wrong": 929 + }, + { + "budget": 0.77, + "threshold": -0.5883752107620239, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.7008925433972872, + "rescued": 2923, + "harmed": 926, + "missed_rescue": 1703, + "avoided_harm": 207, + "selected_both_correct": 6166, + "selected_both_wrong": 941 + }, + { + "budget": 0.78, + "threshold": -0.5972744226455688, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.7033523086654017, + "rescued": 2977, + "harmed": 945, + "missed_rescue": 1649, + "avoided_harm": 188, + "selected_both_correct": 6220, + "selected_both_wrong": 956 + }, + { + "budget": 0.79, + "threshold": -0.6058458089828491, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.7058120739335161, + "rescued": 3026, + "harmed": 959, + "missed_rescue": 1600, + "avoided_harm": 174, + "selected_both_correct": 6288, + "selected_both_wrong": 967 + }, + { + "budget": 0.8, + "threshold": -0.6140543818473816, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.7068662590484223, + "rescued": 3067, + "harmed": 985, + "missed_rescue": 1559, + "avoided_harm": 148, + "selected_both_correct": 6341, + "selected_both_wrong": 990 + }, + { + "budget": 0.81, + "threshold": -0.6230268478393555, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.7082718392016305, + "rescued": 3109, + "harmed": 1007, + "missed_rescue": 1517, + "avoided_harm": 126, + "selected_both_correct": 6402, + "selected_both_wrong": 1007 + }, + { + "budget": 0.82, + "threshold": -0.6330366134643555, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.7100990934008011, + "rescued": 3152, + "harmed": 1024, + "missed_rescue": 1474, + "avoided_harm": 109, + "selected_both_correct": 6466, + "selected_both_wrong": 1025 + }, + { + "budget": 0.83, + "threshold": -0.6429333686828613, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.711434394546349, + "rescued": 3192, + "harmed": 1045, + "missed_rescue": 1434, + "avoided_harm": 88, + "selected_both_correct": 6528, + "selected_both_wrong": 1045 + }, + { + "budget": 0.84, + "threshold": -0.6513019800186157, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.7131913697378593, + "rescued": 3238, + "harmed": 1066, + "missed_rescue": 1388, + "avoided_harm": 67, + "selected_both_correct": 6584, + "selected_both_wrong": 1064 + }, + { + "budget": 0.85, + "threshold": -0.6624053120613098, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.7150889029446904, + "rescued": 3282, + "harmed": 1083, + "missed_rescue": 1344, + "avoided_harm": 50, + "selected_both_correct": 6639, + "selected_both_wrong": 1090 + }, + { + "budget": 0.86, + "threshold": -0.6759263277053833, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.7167755991285403, + "rescued": 3326, + "harmed": 1103, + "missed_rescue": 1300, + "avoided_harm": 30, + "selected_both_correct": 6698, + "selected_both_wrong": 1109 + }, + { + "budget": 0.87, + "threshold": -0.6904275417327881, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.7191650853889943, + "rescued": 3371, + "harmed": 1114, + "missed_rescue": 1255, + "avoided_harm": 19, + "selected_both_correct": 6772, + "selected_both_wrong": 1122 + }, + { + "budget": 0.88, + "threshold": -0.7045450210571289, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.7228195937873357, + "rescued": 3437, + "harmed": 1128, + "missed_rescue": 1189, + "avoided_harm": 5, + "selected_both_correct": 6814, + "selected_both_wrong": 1142 + }, + { + "budget": 0.89, + "threshold": -0.7205488681793213, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.7301286105840186, + "rescued": 3541, + "harmed": 1128, + "missed_rescue": 1085, + "avoided_harm": 5, + "selected_both_correct": 6815, + "selected_both_wrong": 1179 + }, + { + "budget": 0.9, + "threshold": -0.7357243299484253, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.7374376273807014, + "rescued": 3645, + "harmed": 1128, + "missed_rescue": 981, + "avoided_harm": 5, + "selected_both_correct": 6815, + "selected_both_wrong": 1218 + }, + { + "budget": 0.91, + "threshold": -0.7513164281845093, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.7445358071544029, + "rescued": 3746, + "harmed": 1128, + "missed_rescue": 880, + "avoided_harm": 5, + "selected_both_correct": 6815, + "selected_both_wrong": 1259 + }, + { + "budget": 0.92, + "threshold": -0.7671180963516235, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.7521962189893878, + "rescued": 3855, + "harmed": 1128, + "missed_rescue": 771, + "avoided_harm": 5, + "selected_both_correct": 6817, + "selected_both_wrong": 1290 + }, + { + "budget": 0.93, + "threshold": -0.7833515405654907, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.758872724717127, + "rescued": 3951, + "harmed": 1129, + "missed_rescue": 675, + "avoided_harm": 4, + "selected_both_correct": 6818, + "selected_both_wrong": 1334 + }, + { + "budget": 0.94, + "threshold": -0.8013419508934021, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.7654789514372057, + "rescued": 4045, + "harmed": 1129, + "missed_rescue": 581, + "avoided_harm": 4, + "selected_both_correct": 6821, + "selected_both_wrong": 1380 + }, + { + "budget": 0.95, + "threshold": -0.8188987374305725, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.7727879682338885, + "rescued": 4149, + "harmed": 1129, + "missed_rescue": 477, + "avoided_harm": 4, + "selected_both_correct": 6822, + "selected_both_wrong": 1417 + }, + { + "budget": 0.96, + "threshold": -0.8395025134086609, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.7791130789233256, + "rescued": 4240, + "harmed": 1130, + "missed_rescue": 386, + "avoided_harm": 3, + "selected_both_correct": 6826, + "selected_both_wrong": 1463 + }, + { + "budget": 0.97, + "threshold": -0.8628664612770081, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.7857193056434043, + "rescued": 4335, + "harmed": 1131, + "missed_rescue": 291, + "avoided_harm": 2, + "selected_both_correct": 6829, + "selected_both_wrong": 1507 + }, + { + "budget": 0.98, + "threshold": -0.8857419490814209, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.7927472064094455, + "rescued": 4435, + "harmed": 1131, + "missed_rescue": 191, + "avoided_harm": 2, + "selected_both_correct": 6835, + "selected_both_wrong": 1543 + }, + { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.7994237121371847, + "rescued": 4531, + "harmed": 1132, + "missed_rescue": 95, + "avoided_harm": 1, + "selected_both_correct": 6840, + "selected_both_wrong": 1583 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 4626, + "harmed": 1133, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6843, + "selected_both_wrong": 1627 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.26688029946402747, + "rescued": 0, + "harmed": 0, + "missed_rescue": 18820, + "avoided_harm": 1970, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.1584138125181198, + "operator": "ge", + "actual_fraction": 0.029095652667101495, + "metric": 0.2670788078155574, + "rescued": 15, + "harmed": 8, + "missed_rescue": 18805, + "avoided_harm": 1962, + "selected_both_correct": 994, + "selected_both_wrong": 9 + }, + { + "budget": 0.02, + "threshold": -0.21403978765010834, + "operator": "ge", + "actual_fraction": 0.04622408757054136, + "metric": 0.26747582451861723, + "rescued": 38, + "harmed": 17, + "missed_rescue": 18782, + "avoided_harm": 1953, + "selected_both_correct": 1519, + "selected_both_wrong": 56 + }, + { + "budget": 0.03, + "threshold": -0.24253316223621368, + "operator": "ge", + "actual_fraction": 0.056149505147037974, + "metric": 0.26821314125287127, + "rescued": 68, + "harmed": 21, + "missed_rescue": 18752, + "avoided_harm": 1949, + "selected_both_correct": 1793, + "selected_both_wrong": 98 + }, + { + "budget": 0.04, + "threshold": -0.2599165141582489, + "operator": "ge", + "actual_fraction": 0.06372118084110824, + "metric": 0.26883702464339393, + "rescued": 98, + "harmed": 29, + "missed_rescue": 18722, + "avoided_harm": 1941, + "selected_both_correct": 1984, + "selected_both_wrong": 136 + }, + { + "budget": 0.05, + "threshold": -0.2725968658924103, + "operator": "ge", + "actual_fraction": 0.06882568130902078, + "metric": 0.2692907580183195, + "rescued": 125, + "harmed": 40, + "missed_rescue": 18695, + "avoided_harm": 1930, + "selected_both_correct": 2087, + "selected_both_wrong": 175 + }, + { + "budget": 0.06, + "threshold": -0.2842792272567749, + "operator": "ge", + "actual_fraction": 0.07475257351898591, + "metric": 0.2701415080963049, + "rescued": 161, + "harmed": 46, + "missed_rescue": 18659, + "avoided_harm": 1924, + "selected_both_correct": 2200, + "selected_both_wrong": 229 + }, + { + "budget": 0.07, + "threshold": -0.2935555577278137, + "operator": "ge", + "actual_fraction": 0.08033916569775686, + "metric": 0.27119076652582025, + "rescued": 209, + "harmed": 57, + "missed_rescue": 18611, + "avoided_harm": 1913, + "selected_both_correct": 2298, + "selected_both_wrong": 269 + }, + { + "budget": 0.08, + "threshold": -0.30114513635635376, + "operator": "ge", + "actual_fraction": 0.08632277457958767, + "metric": 0.27260868332246263, + "rescued": 270, + "harmed": 68, + "missed_rescue": 18550, + "avoided_harm": 1902, + "selected_both_correct": 2381, + "selected_both_wrong": 325 + }, + { + "budget": 0.09, + "threshold": -0.308601438999176, + "operator": "ge", + "actual_fraction": 0.09227802512548564, + "metric": 0.27419675013470207, + "rescued": 338, + "harmed": 80, + "missed_rescue": 18482, + "avoided_harm": 1890, + "selected_both_correct": 2473, + "selected_both_wrong": 363 + }, + { + "budget": 0.1, + "threshold": -0.3146189749240875, + "operator": "ge", + "actual_fraction": 0.09783625896832374, + "metric": 0.275699741939143, + "rescued": 405, + "harmed": 94, + "missed_rescue": 18415, + "avoided_harm": 1876, + "selected_both_correct": 2542, + "selected_both_wrong": 409 + }, + { + "budget": 0.11, + "threshold": -0.32043084502220154, + "operator": "ge", + "actual_fraction": 0.10362135949862462, + "metric": 0.2776564671185095, + "rescued": 482, + "harmed": 102, + "missed_rescue": 18338, + "avoided_harm": 1868, + "selected_both_correct": 2610, + "selected_both_wrong": 460 + }, + { + "budget": 0.12, + "threshold": -0.32644838094711304, + "operator": "ge", + "actual_fraction": 0.11031392677877662, + "metric": 0.28032215069619715, + "rescued": 589, + "harmed": 115, + "missed_rescue": 18231, + "avoided_harm": 1855, + "selected_both_correct": 2673, + "selected_both_wrong": 513 + }, + { + "budget": 0.13, + "threshold": -0.33121857047080994, + "operator": "ge", + "actual_fraction": 0.11658111901993591, + "metric": 0.28267589257862347, + "rescued": 680, + "harmed": 123, + "missed_rescue": 18140, + "avoided_harm": 1847, + "selected_both_correct": 2746, + "selected_both_wrong": 562 + }, + { + "budget": 0.14, + "threshold": -0.33570271730422974, + "operator": "ge", + "actual_fraction": 0.12262144457363242, + "metric": 0.2851997844766469, + "rescued": 777, + "harmed": 131, + "missed_rescue": 18043, + "avoided_harm": 1839, + "selected_both_correct": 2805, + "selected_both_wrong": 611 + }, + { + "budget": 0.15, + "threshold": -0.3407592475414276, + "operator": "ge", + "actual_fraction": 0.13024983693956838, + "metric": 0.2880072597339988, + "rescued": 897, + "harmed": 152, + "missed_rescue": 17923, + "avoided_harm": 1818, + "selected_both_correct": 2870, + "selected_both_wrong": 674 + }, + { + "budget": 0.16, + "threshold": -0.34456855058670044, + "operator": "ge", + "actual_fraction": 0.13714091257125033, + "metric": 0.29106996001474633, + "rescued": 1018, + "harmed": 165, + "missed_rescue": 17802, + "avoided_harm": 1805, + "selected_both_correct": 2930, + "selected_both_wrong": 723 + }, + { + "budget": 0.17, + "threshold": -0.3487326204776764, + "operator": "ge", + "actual_fraction": 0.143521538156141, + "metric": 0.2940475852876953, + "rescued": 1137, + "harmed": 179, + "missed_rescue": 17683, + "avoided_harm": 1791, + "selected_both_correct": 2975, + "selected_both_wrong": 770 + }, + { + "budget": 0.18, + "threshold": -0.35280776023864746, + "operator": "ge", + "actual_fraction": 0.15095142217054702, + "metric": 0.29747894393556984, + "rescued": 1269, + "harmed": 190, + "missed_rescue": 17551, + "avoided_harm": 1780, + "selected_both_correct": 3022, + "selected_both_wrong": 842 + }, + { + "budget": 0.19, + "threshold": -0.3564819097518921, + "operator": "ge", + "actual_fraction": 0.15823951450528884, + "metric": 0.3003714942007203, + "rescued": 1395, + "harmed": 214, + "missed_rescue": 17425, + "avoided_harm": 1756, + "selected_both_correct": 3075, + "selected_both_wrong": 896 + }, + { + "budget": 0.2, + "threshold": -0.35992002487182617, + "operator": "ge", + "actual_fraction": 0.16496044012137367, + "metric": 0.30334911947366927, + "rescued": 1511, + "harmed": 225, + "missed_rescue": 17309, + "avoided_harm": 1745, + "selected_both_correct": 3127, + "selected_both_wrong": 954 + }, + { + "budget": 0.21, + "threshold": -0.36340460181236267, + "operator": "ge", + "actual_fraction": 0.17290077418257097, + "metric": 0.30697898647307376, + "rescued": 1651, + "harmed": 237, + "missed_rescue": 17169, + "avoided_harm": 1733, + "selected_both_correct": 3185, + "selected_both_wrong": 1024 + }, + { + "budget": 0.22, + "threshold": -0.3674520254135132, + "operator": "ge", + "actual_fraction": 0.1829679834387318, + "metric": 0.3114879618863965, + "rescued": 1832, + "harmed": 259, + "missed_rescue": 16988, + "avoided_harm": 1711, + "selected_both_correct": 3252, + "selected_both_wrong": 1109 + }, + { + "budget": 0.23, + "threshold": -0.37108901143074036, + "operator": "ge", + "actual_fraction": 0.19122025919519042, + "metric": 0.31480588719053965, + "rescued": 1970, + "harmed": 280, + "missed_rescue": 16850, + "avoided_harm": 1690, + "selected_both_correct": 3314, + "selected_both_wrong": 1179 + }, + { + "budget": 0.24, + "threshold": -0.3745429813861847, + "operator": "ge", + "actual_fraction": 0.1999262683265746, + "metric": 0.3192581459319967, + "rescued": 2144, + "harmed": 297, + "missed_rescue": 16676, + "avoided_harm": 1673, + "selected_both_correct": 3372, + "selected_both_wrong": 1237 + }, + { + "budget": 0.25, + "threshold": -0.3773327171802521, + "operator": "ge", + "actual_fraction": 0.2080651107393018, + "metric": 0.32294472960326687, + "rescued": 2294, + "harmed": 317, + "missed_rescue": 16526, + "avoided_harm": 1653, + "selected_both_correct": 3423, + "selected_both_wrong": 1303 + }, + { + "budget": 0.26, + "threshold": -0.3803459703922272, + "operator": "ge", + "actual_fraction": 0.21603380313643195, + "metric": 0.3270283299775969, + "rescued": 2457, + "harmed": 336, + "missed_rescue": 16363, + "avoided_harm": 1634, + "selected_both_correct": 3465, + "selected_both_wrong": 1360 + }, + { + "budget": 0.27, + "threshold": -0.38361650705337524, + "operator": "ge", + "actual_fraction": 0.2243144372288234, + "metric": 0.33170745540651675, + "rescued": 2634, + "harmed": 348, + "missed_rescue": 16186, + "avoided_harm": 1622, + "selected_both_correct": 3522, + "selected_both_wrong": 1406 + }, + { + "budget": 0.28, + "threshold": -0.38652583956718445, + "operator": "ge", + "actual_fraction": 0.2324249213056178, + "metric": 0.33542239741371976, + "rescued": 2793, + "harmed": 376, + "missed_rescue": 16027, + "avoided_harm": 1594, + "selected_both_correct": 3554, + "selected_both_wrong": 1473 + }, + { + "budget": 0.29, + "threshold": -0.38952407240867615, + "operator": "ge", + "actual_fraction": 0.2419816805149874, + "metric": 0.3401015228426396, + "rescued": 2975, + "harmed": 393, + "missed_rescue": 15845, + "avoided_harm": 1577, + "selected_both_correct": 3623, + "selected_both_wrong": 1542 + }, + { + "budget": 0.3, + "threshold": -0.39262744784355164, + "operator": "ge", + "actual_fraction": 0.2512832147009613, + "metric": 0.3448373649434251, + "rescued": 3169, + "harmed": 420, + "missed_rescue": 15651, + "avoided_harm": 1550, + "selected_both_correct": 3674, + "selected_both_wrong": 1598 + }, + { + "budget": 0.31, + "threshold": -0.3954668939113617, + "operator": "ge", + "actual_fraction": 0.2596772821370842, + "metric": 0.3491194736692851, + "rescued": 3340, + "harmed": 440, + "missed_rescue": 15480, + "avoided_harm": 1530, + "selected_both_correct": 3706, + "selected_both_wrong": 1671 + }, + { + "budget": 0.32, + "threshold": -0.3986983895301819, + "operator": "ge", + "actual_fraction": 0.26985792473697645, + "metric": 0.35459263250432466, + "rescued": 3553, + "harmed": 460, + "missed_rescue": 15267, + "avoided_harm": 1510, + "selected_both_correct": 3758, + "selected_both_wrong": 1745 + }, + { + "budget": 0.33, + "threshold": -0.4014112949371338, + "operator": "ge", + "actual_fraction": 0.27757139211071097, + "metric": 0.35878966622238606, + "rescued": 3718, + "harmed": 477, + "missed_rescue": 15102, + "avoided_harm": 1493, + "selected_both_correct": 3788, + "selected_both_wrong": 1805 + }, + { + "budget": 0.34, + "threshold": -0.4042409658432007, + "operator": "ge", + "actual_fraction": 0.28729830133567763, + "metric": 0.3635255083231716, + "rescued": 3913, + "harmed": 505, + "missed_rescue": 14907, + "avoided_harm": 1465, + "selected_both_correct": 3845, + "selected_both_wrong": 1868 + }, + { + "budget": 0.35, + "threshold": -0.40724706649780273, + "operator": "ge", + "actual_fraction": 0.29793267731049544, + "metric": 0.36928225051753966, + "rescued": 4137, + "harmed": 526, + "missed_rescue": 14683, + "avoided_harm": 1444, + "selected_both_correct": 3903, + "selected_both_wrong": 1940 + }, + { + "budget": 0.36, + "threshold": -0.40968337655067444, + "operator": "ge", + "actual_fraction": 0.3067237614496781, + "metric": 0.3743583926495193, + "rescued": 4332, + "harmed": 542, + "missed_rescue": 14488, + "avoided_harm": 1428, + "selected_both_correct": 3949, + "selected_both_wrong": 1993 + }, + { + "budget": 0.37, + "threshold": -0.4123799502849579, + "operator": "ge", + "actual_fraction": 0.3161387289793835, + "metric": 0.3794912514533647, + "rescued": 4536, + "harmed": 565, + "missed_rescue": 14284, + "avoided_harm": 1405, + "selected_both_correct": 3993, + "selected_both_wrong": 2054 + }, + { + "budget": 0.38, + "threshold": -0.41514647006988525, + "operator": "ge", + "actual_fraction": 0.32538354649349177, + "metric": 0.3843121685619488, + "rescued": 4725, + "harmed": 584, + "missed_rescue": 14095, + "avoided_harm": 1386, + "selected_both_correct": 4050, + "selected_both_wrong": 2115 + }, + { + "budget": 0.39, + "threshold": -0.4179389476776123, + "operator": "ge", + "actual_fraction": 0.3354507557496526, + "metric": 0.389671894053257, + "rescued": 4940, + "harmed": 610, + "missed_rescue": 13880, + "avoided_harm": 1360, + "selected_both_correct": 4104, + "selected_both_wrong": 2175 + }, + { + "budget": 0.4, + "threshold": -0.42099592089653015, + "operator": "ge", + "actual_fraction": 0.34653886509939597, + "metric": 0.395428636247625, + "rescued": 5169, + "harmed": 636, + "missed_rescue": 13651, + "avoided_harm": 1334, + "selected_both_correct": 4154, + "selected_both_wrong": 2261 + }, + { + "budget": 0.41, + "threshold": -0.4243226647377014, + "operator": "ge", + "actual_fraction": 0.3577404077928707, + "metric": 0.4010719450982616, + "rescued": 5399, + "harmed": 667, + "missed_rescue": 13421, + "avoided_harm": 1303, + "selected_both_correct": 4211, + "selected_both_wrong": 2338 + }, + { + "budget": 0.42, + "threshold": -0.4275209903717041, + "operator": "ge", + "actual_fraction": 0.3693673255253382, + "metric": 0.40776451237841366, + "rescued": 5657, + "harmed": 689, + "missed_rescue": 13163, + "avoided_harm": 1281, + "selected_both_correct": 4262, + "selected_both_wrong": 2417 + }, + { + "budget": 0.43, + "threshold": -0.4307159185409546, + "operator": "ge", + "actual_fraction": 0.38039871820321586, + "metric": 0.41423021297110285, + "rescued": 5901, + "harmed": 705, + "missed_rescue": 12919, + "avoided_harm": 1265, + "selected_both_correct": 4323, + "selected_both_wrong": 2485 + }, + { + "budget": 0.44, + "threshold": -0.4336964190006256, + "operator": "ge", + "actual_fraction": 0.39049428579530954, + "metric": 0.42007203017326944, + "rescued": 6128, + "harmed": 726, + "missed_rescue": 12692, + "avoided_harm": 1244, + "selected_both_correct": 4373, + "selected_both_wrong": 2543 + }, + { + "budget": 0.45, + "threshold": -0.4373021125793457, + "operator": "ge", + "actual_fraction": 0.4023764285511726, + "metric": 0.42665116410969006, + "rescued": 6384, + "harmed": 750, + "missed_rescue": 12436, + "avoided_harm": 1220, + "selected_both_correct": 4432, + "selected_both_wrong": 2623 + }, + { + "budget": 0.46, + "threshold": -0.4404309391975403, + "operator": "ge", + "actual_fraction": 0.41196154609647506, + "metric": 0.4321810396165953, + "rescued": 6599, + "harmed": 770, + "missed_rescue": 12221, + "avoided_harm": 1200, + "selected_both_correct": 4474, + "selected_both_wrong": 2684 + }, + { + "budget": 0.47, + "threshold": -0.44344601035118103, + "operator": "ge", + "actual_fraction": 0.42157502197771035, + "metric": 0.43782434846723195, + "rescued": 6813, + "harmed": 785, + "missed_rescue": 12007, + "avoided_harm": 1185, + "selected_both_correct": 4509, + "selected_both_wrong": 2759 + }, + { + "budget": 0.48, + "threshold": -0.4465356171131134, + "operator": "ge", + "actual_fraction": 0.43184073958540115, + "metric": 0.4434960156538014, + "rescued": 7036, + "harmed": 808, + "missed_rescue": 11784, + "avoided_harm": 1162, + "selected_both_correct": 4557, + "selected_both_wrong": 2827 + }, + { + "budget": 0.49, + "threshold": -0.44965147972106934, + "operator": "ge", + "actual_fraction": 0.44165272381816634, + "metric": 0.4488557411451096, + "rescued": 7253, + "harmed": 836, + "missed_rescue": 11567, + "avoided_harm": 1134, + "selected_both_correct": 4604, + "selected_both_wrong": 2881 + }, + { + "budget": 0.5, + "threshold": -0.453143447637558, + "operator": "ge", + "actual_fraction": 0.452315458128917, + "metric": 0.4548393500269404, + "rescued": 7483, + "harmed": 855, + "missed_rescue": 11337, + "avoided_harm": 1115, + "selected_both_correct": 4651, + "selected_both_wrong": 2961 + }, + { + "budget": 0.51, + "threshold": -0.4565357565879822, + "operator": "ge", + "actual_fraction": 0.4631483424552647, + "metric": 0.4610781839321669, + "rescued": 7721, + "harmed": 873, + "missed_rescue": 11099, + "avoided_harm": 1097, + "selected_both_correct": 4690, + "selected_both_wrong": 3048 + }, + { + "budget": 0.52, + "threshold": -0.45961302518844604, + "operator": "ge", + "actual_fraction": 0.47176927657885037, + "metric": 0.46538865099395965, + "rescued": 7891, + "harmed": 891, + "missed_rescue": 10929, + "avoided_harm": 1079, + "selected_both_correct": 4737, + "selected_both_wrong": 3117 + }, + { + "budget": 0.53, + "threshold": -0.46368208527565, + "operator": "ge", + "actual_fraction": 0.4840484360377733, + "metric": 0.4715707682273204, + "rescued": 8142, + "harmed": 924, + "missed_rescue": 10678, + "avoided_harm": 1046, + "selected_both_correct": 4791, + "selected_both_wrong": 3212 + }, + { + "budget": 0.54, + "threshold": -0.46793216466903687, + "operator": "ge", + "actual_fraction": 0.495079828715651, + "metric": 0.47809318549187535, + "rescued": 8393, + "harmed": 945, + "missed_rescue": 10427, + "avoided_harm": 1025, + "selected_both_correct": 4846, + "selected_both_wrong": 3274 + }, + { + "budget": 0.55, + "threshold": -0.4718930721282959, + "operator": "ge", + "actual_fraction": 0.5065365964325214, + "metric": 0.48393500269404194, + "rescued": 8631, + "harmed": 977, + "missed_rescue": 10189, + "avoided_harm": 993, + "selected_both_correct": 4900, + "selected_both_wrong": 3354 + }, + { + "budget": 0.56, + "threshold": -0.47629430890083313, + "operator": "ge", + "actual_fraction": 0.5183620225165188, + "metric": 0.4900887615914698, + "rescued": 8881, + "harmed": 1010, + "missed_rescue": 9939, + "avoided_harm": 960, + "selected_both_correct": 4950, + "selected_both_wrong": 3438 + }, + { + "budget": 0.57, + "threshold": -0.4799485504627228, + "operator": "ge", + "actual_fraction": 0.5273232566712985, + "metric": 0.4943708703173298, + "rescued": 9063, + "harmed": 1041, + "missed_rescue": 9757, + "avoided_harm": 929, + "selected_both_correct": 4989, + "selected_both_wrong": 3502 + }, + { + "budget": 0.58, + "threshold": -0.4838995635509491, + "operator": "ge", + "actual_fraction": 0.5381561409976463, + "metric": 0.5008082125740861, + "rescued": 9310, + "harmed": 1061, + "missed_rescue": 9510, + "avoided_harm": 909, + "selected_both_correct": 5038, + "selected_both_wrong": 3568 + }, + { + "budget": 0.59, + "threshold": -0.4886912405490875, + "operator": "ge", + "actual_fraction": 0.5503502254487707, + "metric": 0.5080395882369623, + "rescued": 9590, + "harmed": 1086, + "missed_rescue": 9230, + "avoided_harm": 884, + "selected_both_correct": 5080, + "selected_both_wrong": 3651 + }, + { + "budget": 0.6, + "threshold": -0.49332454800605774, + "operator": "ge", + "actual_fraction": 0.5619487848453053, + "metric": 0.5147321555171143, + "rescued": 9849, + "harmed": 1109, + "missed_rescue": 8971, + "avoided_harm": 861, + "selected_both_correct": 5127, + "selected_both_wrong": 3731 + }, + { + "budget": 0.61, + "threshold": -0.49750789999961853, + "operator": "ge", + "actual_fraction": 0.5722712191248618, + "metric": 0.5204038227036838, + "rescued": 10085, + "harmed": 1145, + "missed_rescue": 8735, + "avoided_harm": 825, + "selected_both_correct": 5165, + "selected_both_wrong": 3785 + }, + { + "budget": 0.62, + "threshold": -0.5024909377098083, + "operator": "ge", + "actual_fraction": 0.5818563366701642, + "metric": 0.5257635481949919, + "rescued": 10296, + "harmed": 1167, + "missed_rescue": 8524, + "avoided_harm": 803, + "selected_both_correct": 5206, + "selected_both_wrong": 3849 + }, + { + "budget": 0.63, + "threshold": -0.5067122578620911, + "operator": "ge", + "actual_fraction": 0.590817570824944, + "metric": 0.5306128236395088, + "rescued": 10487, + "harmed": 1187, + "missed_rescue": 8333, + "avoided_harm": 783, + "selected_both_correct": 5248, + "selected_both_wrong": 3912 + }, + { + "budget": 0.64, + "threshold": -0.512323796749115, + "operator": "ge", + "actual_fraction": 0.604202705385248, + "metric": 0.5381277826617135, + "rescued": 10781, + "harmed": 1216, + "missed_rescue": 8039, + "avoided_harm": 754, + "selected_both_correct": 5304, + "selected_both_wrong": 4005 + }, + { + "budget": 0.65, + "threshold": -0.5170255303382874, + "operator": "ge", + "actual_fraction": 0.6130505061962964, + "metric": 0.5431472081218274, + "rescued": 10982, + "harmed": 1240, + "missed_rescue": 7838, + "avoided_harm": 730, + "selected_both_correct": 5343, + "selected_both_wrong": 4053 + }, + { + "budget": 0.66, + "threshold": -0.5215396881103516, + "operator": "ge", + "actual_fraction": 0.6222102487026061, + "metric": 0.548421858605337, + "rescued": 11191, + "harmed": 1263, + "missed_rescue": 7629, + "avoided_harm": 707, + "selected_both_correct": 5388, + "selected_both_wrong": 4099 + }, + { + "budget": 0.67, + "threshold": -0.5264327526092529, + "operator": "ge", + "actual_fraction": 0.6320505912713043, + "metric": 0.552902475682727, + "rescued": 11382, + "harmed": 1296, + "missed_rescue": 7438, + "avoided_harm": 674, + "selected_both_correct": 5428, + "selected_both_wrong": 4182 + }, + { + "budget": 0.68, + "threshold": -0.5312389135360718, + "operator": "ge", + "actual_fraction": 0.6424297422227264, + "metric": 0.5588010095567592, + "rescued": 11615, + "harmed": 1321, + "missed_rescue": 7205, + "avoided_harm": 649, + "selected_both_correct": 5473, + "selected_both_wrong": 4245 + }, + { + "budget": 0.69, + "threshold": -0.5368650555610657, + "operator": "ge", + "actual_fraction": 0.6538865099395967, + "metric": 0.5642174517199331, + "rescued": 11846, + "harmed": 1361, + "missed_rescue": 6974, + "avoided_harm": 609, + "selected_both_correct": 5528, + "selected_both_wrong": 4323 + }, + { + "budget": 0.7, + "threshold": -0.5431069135665894, + "operator": "ge", + "actual_fraction": 0.6647761109378102, + "metric": 0.5704562856251595, + "rescued": 12087, + "harmed": 1382, + "missed_rescue": 6733, + "avoided_harm": 588, + "selected_both_correct": 5573, + "selected_both_wrong": 4400 + }, + { + "budget": 0.71, + "threshold": -0.5487136244773865, + "operator": "ge", + "actual_fraction": 0.6740492867878513, + "metric": 0.5753339194056093, + "rescued": 12285, + "harmed": 1408, + "missed_rescue": 6535, + "avoided_harm": 562, + "selected_both_correct": 5609, + "selected_both_wrong": 4467 + }, + { + "budget": 0.72, + "threshold": -0.5549414753913879, + "operator": "ge", + "actual_fraction": 0.6839179876924822, + "metric": 0.5810906615999774, + "rescued": 12503, + "harmed": 1423, + "missed_rescue": 6317, + "avoided_harm": 547, + "selected_both_correct": 5664, + "selected_both_wrong": 4527 + }, + { + "budget": 0.73, + "threshold": -0.5604677200317383, + "operator": "ge", + "actual_fraction": 0.6935314635737175, + "metric": 0.5866205371068826, + "rescued": 12724, + "harmed": 1449, + "missed_rescue": 6096, + "avoided_harm": 521, + "selected_both_correct": 5702, + "selected_both_wrong": 4581 + }, + { + "budget": 0.74, + "threshold": -0.5666090250015259, + "operator": "ge", + "actual_fraction": 0.7039673311970054, + "metric": 0.5924623543090491, + "rescued": 12953, + "harmed": 1472, + "missed_rescue": 5867, + "avoided_harm": 498, + "selected_both_correct": 5739, + "selected_both_wrong": 4660 + }, + { + "budget": 0.75, + "threshold": -0.574927568435669, + "operator": "ge", + "actual_fraction": 0.7186853075461532, + "metric": 0.6001758216827836, + "rescued": 13270, + "harmed": 1517, + "missed_rescue": 5550, + "avoided_harm": 453, + "selected_both_correct": 5813, + "selected_both_wrong": 4743 + }, + { + "budget": 0.76, + "threshold": -0.5812664031982422, + "operator": "ge", + "actual_fraction": 0.7283555000992542, + "metric": 0.6051668888069648, + "rescued": 13473, + "harmed": 1544, + "missed_rescue": 5347, + "avoided_harm": 426, + "selected_both_correct": 5862, + "selected_both_wrong": 4805 + }, + { + "budget": 0.77, + "threshold": -0.5883752107620239, + "operator": "ge", + "actual_fraction": 0.7390465927459376, + "metric": 0.6112072143606613, + "rescued": 13710, + "harmed": 1568, + "missed_rescue": 5110, + "avoided_harm": 402, + "selected_both_correct": 5909, + "selected_both_wrong": 4874 + }, + { + "budget": 0.78, + "threshold": -0.5972744226455688, + "operator": "ge", + "actual_fraction": 0.752176502282846, + "metric": 0.618722173382866, + "rescued": 14008, + "harmed": 1601, + "missed_rescue": 4812, + "avoided_harm": 369, + "selected_both_correct": 5959, + "selected_both_wrong": 4956 + }, + { + "budget": 0.79, + "threshold": -0.6058458089828491, + "operator": "ge", + "actual_fraction": 0.7635765533278507, + "metric": 0.6246490655928311, + "rescued": 14250, + "harmed": 1634, + "missed_rescue": 4570, + "avoided_harm": 336, + "selected_both_correct": 6010, + "selected_both_wrong": 5032 + }, + { + "budget": 0.8, + "threshold": -0.6140543818473816, + "operator": "ge", + "actual_fraction": 0.7745228709979298, + "metric": 0.6308878994980575, + "rescued": 14487, + "harmed": 1651, + "missed_rescue": 4333, + "avoided_harm": 319, + "selected_both_correct": 6062, + "selected_both_wrong": 5112 + }, + { + "budget": 0.81, + "threshold": -0.6230268478393555, + "operator": "ge", + "actual_fraction": 0.7864050137537929, + "metric": 0.6366730000283584, + "rescued": 14732, + "harmed": 1692, + "missed_rescue": 4088, + "avoided_harm": 278, + "selected_both_correct": 6105, + "selected_both_wrong": 5202 + }, + { + "budget": 0.82, + "threshold": -0.6330366134643555, + "operator": "ge", + "actual_fraction": 0.798287156509656, + "metric": 0.6438476590193687, + "rescued": 15012, + "harmed": 1719, + "missed_rescue": 3808, + "avoided_harm": 251, + "selected_both_correct": 6148, + "selected_both_wrong": 5271 + }, + { + "budget": 0.83, + "threshold": -0.6429333686828613, + "operator": "ge", + "actual_fraction": 0.8092334741797351, + "metric": 0.6499730595808638, + "rescued": 15248, + "harmed": 1739, + "missed_rescue": 3572, + "avoided_harm": 231, + "selected_both_correct": 6207, + "selected_both_wrong": 5342 + }, + { + "budget": 0.84, + "threshold": -0.6513019800186157, + "operator": "ge", + "actual_fraction": 0.8177126166236566, + "metric": 0.6548223350253807, + "rescued": 15437, + "harmed": 1757, + "missed_rescue": 3383, + "avoided_harm": 213, + "selected_both_correct": 6242, + "selected_both_wrong": 5399 + }, + { + "budget": 0.85, + "threshold": -0.6624053120613098, + "operator": "ge", + "actual_fraction": 0.8288858009811985, + "metric": 0.6607492272353458, + "rescued": 15678, + "harmed": 1789, + "missed_rescue": 3142, + "avoided_harm": 181, + "selected_both_correct": 6295, + "selected_both_wrong": 5467 + }, + { + "budget": 0.86, + "threshold": -0.6759263277053833, + "operator": "ge", + "actual_fraction": 0.8421291438618382, + "metric": 0.6682925445934833, + "rescued": 15980, + "harmed": 1825, + "missed_rescue": 2840, + "avoided_harm": 145, + "selected_both_correct": 6343, + "selected_both_wrong": 5548 + }, + { + "budget": 0.87, + "threshold": -0.6904275417327881, + "operator": "ge", + "actual_fraction": 0.8543799449848283, + "metric": 0.6742194368034484, + "rescued": 16226, + "harmed": 1862, + "missed_rescue": 2594, + "avoided_harm": 108, + "selected_both_correct": 6386, + "selected_both_wrong": 5654 + }, + { + "budget": 0.88, + "threshold": -0.7045450210571289, + "operator": "ge", + "actual_fraction": 0.8657516376939001, + "metric": 0.6793806539432267, + "rescued": 16441, + "harmed": 1895, + "missed_rescue": 2379, + "avoided_harm": 75, + "selected_both_correct": 6451, + "selected_both_wrong": 5742 + }, + { + "budget": 0.89, + "threshold": -0.7205488681793213, + "operator": "ge", + "actual_fraction": 0.8767830303717777, + "metric": 0.6845985877548706, + "rescued": 16636, + "harmed": 1906, + "missed_rescue": 2184, + "avoided_harm": 64, + "selected_both_correct": 6534, + "selected_both_wrong": 5842 + }, + { + "budget": 0.9, + "threshold": -0.7357243299484253, + "operator": "ge", + "actual_fraction": 0.8868785979638715, + "metric": 0.6893911465275218, + "rescued": 16811, + "harmed": 1912, + "missed_rescue": 2009, + "avoided_harm": 58, + "selected_both_correct": 6611, + "selected_both_wrong": 5940 + }, + { + "budget": 0.91, + "threshold": -0.7513164281845093, + "operator": "ge", + "actual_fraction": 0.8974562572668235, + "metric": 0.6947508720188299, + "rescued": 17004, + "harmed": 1916, + "missed_rescue": 1816, + "avoided_harm": 54, + "selected_both_correct": 6695, + "selected_both_wrong": 6032 + }, + { + "budget": 0.92, + "threshold": -0.7671180963516235, + "operator": "ge", + "actual_fraction": 0.9084025749369027, + "metric": 0.7000255225023395, + "rescued": 17196, + "harmed": 1922, + "missed_rescue": 1624, + "avoided_harm": 48, + "selected_both_correct": 6770, + "selected_both_wrong": 6145 + }, + { + "budget": 0.93, + "threshold": -0.7833515405654907, + "operator": "ge", + "actual_fraction": 0.9181011258259365, + "metric": 0.704959872954655, + "rescued": 17376, + "harmed": 1928, + "missed_rescue": 1444, + "avoided_harm": 42, + "selected_both_correct": 6846, + "selected_both_wrong": 6225 + }, + { + "budget": 0.94, + "threshold": -0.8013419508934021, + "operator": "ge", + "actual_fraction": 0.9289623684882171, + "metric": 0.710546465133426, + "rescued": 17577, + "harmed": 1932, + "missed_rescue": 1243, + "avoided_harm": 38, + "selected_both_correct": 6926, + "selected_both_wrong": 6323 + }, + { + "budget": 0.95, + "threshold": -0.8188987374305725, + "operator": "ge", + "actual_fraction": 0.9394549527833707, + "metric": 0.7157360406091371, + "rescued": 17763, + "harmed": 1935, + "missed_rescue": 1057, + "avoided_harm": 35, + "selected_both_correct": 7010, + "selected_both_wrong": 6420 + }, + { + "budget": 0.96, + "threshold": -0.8395025134086609, + "operator": "ge", + "actual_fraction": 0.9509684371721068, + "metric": 0.7211808411082438, + "rescued": 17963, + "harmed": 1943, + "missed_rescue": 857, + "avoided_harm": 27, + "selected_both_correct": 7084, + "selected_both_wrong": 6544 + }, + { + "budget": 0.97, + "threshold": -0.8628664612770081, + "operator": "ge", + "actual_fraction": 0.9640699883730822, + "metric": 0.727476391685336, + "rescued": 18191, + "harmed": 1949, + "missed_rescue": 629, + "avoided_harm": 21, + "selected_both_correct": 7177, + "selected_both_wrong": 6679 + }, + { + "budget": 0.98, + "threshold": -0.8857419490814209, + "operator": "ge", + "actual_fraction": 0.9760372061367439, + "metric": 0.7329779088563083, + "rescued": 18395, + "harmed": 1959, + "missed_rescue": 425, + "avoided_harm": 11, + "selected_both_correct": 7278, + "selected_both_wrong": 6786 + }, + { + "budget": 0.99, + "threshold": -0.9162307977676392, + "operator": "ge", + "actual_fraction": 0.9873805405098829, + "metric": 0.7387062927147435, + "rescued": 18604, + "harmed": 1966, + "missed_rescue": 216, + "avoided_harm": 4, + "selected_both_correct": 7360, + "selected_both_wrong": 6888 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 18820, + "harmed": 1970, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7441, + "selected_both_wrong": 7032 + } + ] + }, + "learned": { + "validation_area_0_25": 0.6484278585986366, + "validation_point": { + "metric": 0.8127205003865345, + "metric_std": 0.0014669638122516435, + "actual_fraction": 0.7859723100709818, + "actual_fraction_std": 0.05939858081016824, + "per_seed": [ + { + "seed": 2026, + "budget": 0.89, + "threshold": -0.005598927848041058, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.8122144915313796, + "rescue": 4618, + "harm": 1037, + "missed_rescue": 8 + }, + { + "seed": 2027, + "budget": 0.76, + "threshold": 0.007067253813147545, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8137606296999086, + "rescue": 4596, + "harm": 993, + "missed_rescue": 30 + }, + { + "seed": 2028, + "budget": 0.74, + "threshold": 0.04995465278625488, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8108791903858318, + "rescue": 4592, + "harm": 1030, + "missed_rescue": 34 + }, + { + "seed": 2029, + "budget": 0.77, + "threshold": 0.0001585048739798367, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8146039777918336, + "rescue": 4605, + "harm": 990, + "missed_rescue": 21 + }, + { + "seed": 2030, + "budget": 0.77, + "threshold": 0.0009468437638133764, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8121442125237192, + "rescue": 4604, + "harm": 1024, + "missed_rescue": 22 + } + ], + "rescued": 4603.0, + "harmed": 1014.8, + "missed_rescue": 23.0, + "avoided_harm": 118.20000000000005 + }, + "test_point": { + "metric": 0.7511102288517709, + "metric_std": 0.0012934323753779572, + "actual_fraction": 0.8917222017412019, + "actual_fraction_std": 0.03061690111015076, + "per_seed": [ + { + "seed": 2026, + "budget": 0.89, + "threshold": -0.005598927848041058, + "operator": "ge", + "actual_fraction": 0.9439639281966934, + "metric": 0.7510988855173978, + "rescue": 18744, + "harm": 1669, + "missed_rescue": 76 + }, + { + "seed": 2027, + "budget": 0.76, + "threshold": 0.007067253813147545, + "operator": "ge", + "actual_fraction": 0.8772651220826362, + "metric": 0.7514108272126592, + "rescue": 18710, + "harm": 1624, + "missed_rescue": 110 + }, + { + "seed": 2028, + "budget": 0.74, + "threshold": 0.04995465278625488, + "operator": "ge", + "actual_fraction": 0.8646456625925191, + "metric": 0.7490003686583672, + "rescue": 18734, + "harm": 1733, + "missed_rescue": 86 + }, + { + "seed": 2029, + "budget": 0.77, + "threshold": 0.0001585048739798367, + "operator": "ge", + "actual_fraction": 0.8896009982134249, + "metric": 0.7525168023140402, + "rescue": 18714, + "harm": 1589, + "missed_rescue": 106 + }, + { + "seed": 2030, + "budget": 0.77, + "threshold": 0.0009468437638133764, + "operator": "ge", + "actual_fraction": 0.8831352976207356, + "metric": 0.7515242605563905, + "rescue": 18745, + "harm": 1655, + "missed_rescue": 75 + } + ], + "rescued": 18729.4, + "harmed": 1654.0, + "missed_rescue": 90.6, + "avoided_harm": 316.0 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.5605453650994447, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.5700751985381967, + "metric_std": 0.0003604155289889192, + "actual_fraction": 0.009979619087778481, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.579169302129454, + "metric_std": 0.0006536325545624493, + "actual_fraction": 0.019959238175556962, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.5876449504532997, + "metric_std": 0.0011140940759317099, + "actual_fraction": 0.029924801461803357, + "actual_fraction_std": 3.142972770398258e-05 + }, + { + "budget": 0.04, + "metric": 0.5961627661817415, + "metric_std": 0.0013531218706396646, + "actual_fraction": 0.03998875535877434, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.6044135216810738, + "metric_std": 0.0016044560461172244, + "actual_fraction": 0.04995431864502073, + "actual_fraction_std": 3.1429727703979486e-05 + }, + { + "budget": 0.06, + "metric": 0.6123831611497645, + "metric_std": 0.0019492766785173226, + "actual_fraction": 0.0599339377327992, + "actual_fraction_std": 3.1429727703979486e-05 + }, + { + "budget": 0.07, + "metric": 0.6200716845878136, + "metric_std": 0.0018752778219307356, + "actual_fraction": 0.06999789162977019, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.626959027338534, + "metric_std": 0.002637570314821201, + "actual_fraction": 0.07997751071754867, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.6339166490969148, + "metric_std": 0.0029704834078864316, + "actual_fraction": 0.08994307400379506, + "actual_fraction_std": 3.1429727703979486e-05 + }, + { + "budget": 0.1, + "metric": 0.6404104294047368, + "metric_std": 0.0030381347582781695, + "actual_fraction": 0.09993674889310562, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.646580926277321, + "metric_std": 0.002838436788551885, + "actual_fraction": 0.10998664698854452, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.12, + "metric": 0.6524281397146672, + "metric_std": 0.0029665733976450294, + "actual_fraction": 0.11996626607632302, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.13, + "metric": 0.6582191299458853, + "metric_std": 0.003058307925717985, + "actual_fraction": 0.1299458851641015, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.663996064375571, + "metric_std": 0.0029108577452994187, + "actual_fraction": 0.13999578325954037, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.6689155949117997, + "metric_std": 0.0031882201763198444, + "actual_fraction": 0.14997540234731885, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.6738070138449644, + "metric_std": 0.0032596885669826437, + "actual_fraction": 0.15995502143509735, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.6787265443811933, + "metric_std": 0.0030845199459928283, + "actual_fraction": 0.16992058472134375, + "actual_fraction_std": 3.1429727703979486e-05 + }, + { + "budget": 0.18, + "metric": 0.6834352378944409, + "metric_std": 0.003419418599696085, + "actual_fraction": 0.1799845386183147, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.6876800899571298, + "metric_std": 0.0032881263800097365, + "actual_fraction": 0.18996415770609318, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.6920795558366717, + "metric_std": 0.0032471639566819875, + "actual_fraction": 0.19994377679387168, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.6962681846932322, + "metric_std": 0.0034620526298224847, + "actual_fraction": 0.2099936748893106, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.7005130367559209, + "metric_std": 0.003773073031705017, + "actual_fraction": 0.21995923817555693, + "actual_fraction_std": 3.142972770397948e-05 + }, + { + "budget": 0.23, + "metric": 0.7045751633986927, + "metric_std": 0.004481519896327001, + "actual_fraction": 0.22995291306486756, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.24, + "metric": 0.7082577834000985, + "metric_std": 0.004572296282732033, + "actual_fraction": 0.23993253215264604, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.25, + "metric": 0.7120669056152927, + "metric_std": 0.004743182240887488, + "actual_fraction": 0.24998243024808492, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.7153981305783963, + "metric_std": 0.004966891489280682, + "actual_fraction": 0.25996204933586337, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.7188277461522243, + "metric_std": 0.005079913296846257, + "actual_fraction": 0.26994166842364187, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.7224119755429053, + "metric_std": 0.005086714759874377, + "actual_fraction": 0.27999156651908075, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.7258978143228617, + "metric_std": 0.005432299079742833, + "actual_fraction": 0.28997118560685925, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.7296366575303957, + "metric_std": 0.005245093304391109, + "actual_fraction": 0.2999508046946377, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.7327429896689858, + "metric_std": 0.005207859409109553, + "actual_fraction": 0.3099304237824162, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.735989879822897, + "metric_std": 0.005192472582895046, + "actual_fraction": 0.3199803218778551, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.7388853749385058, + "metric_std": 0.005509981769435921, + "actual_fraction": 0.3299599409656336, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.741794925855647, + "metric_std": 0.005750223081303747, + "actual_fraction": 0.339939560053412, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.7445217513528709, + "metric_std": 0.005604150483006645, + "actual_fraction": 0.34998945814885096, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.7472204652470307, + "metric_std": 0.005136477232194287, + "actual_fraction": 0.3599690772366294, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.7497083421182094, + "metric_std": 0.004947961579015667, + "actual_fraction": 0.3699346405228758, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.38, + "metric": 0.7525054466230937, + "metric_std": 0.004862372895140892, + "actual_fraction": 0.3799985944198468, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.7551198257080611, + "metric_std": 0.004879610731872325, + "actual_fraction": 0.3899782135076253, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.7579590976175417, + "metric_std": 0.004741359592359375, + "actual_fraction": 0.39995783259540374, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.7603766954810599, + "metric_std": 0.0043121769673192135, + "actual_fraction": 0.4099374516831823, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.42, + "metric": 0.7630051303675592, + "metric_std": 0.0038162860121742715, + "actual_fraction": 0.4199873497786212, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.7658303464755077, + "metric_std": 0.0037247874060018196, + "actual_fraction": 0.4299669688663997, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.44, + "metric": 0.7681073863237052, + "metric_std": 0.003399136312396265, + "actual_fraction": 0.43994658795417807, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.45, + "metric": 0.770468760981095, + "metric_std": 0.0033487781223471864, + "actual_fraction": 0.44999648604961695, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.46, + "metric": 0.7729425820507415, + "metric_std": 0.003214217925751195, + "actual_fraction": 0.4599761051373955, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.47, + "metric": 0.7751915102958746, + "metric_std": 0.003829206406694692, + "actual_fraction": 0.4699416684236418, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.48, + "metric": 0.7771733783118983, + "metric_std": 0.0035847805562064334, + "actual_fraction": 0.4799353433129524, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.7792676927401786, + "metric_std": 0.0033877835088940377, + "actual_fraction": 0.4899852414083913, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.781558788389908, + "metric_std": 0.0035295178393520052, + "actual_fraction": 0.49996486049616984, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.51, + "metric": 0.7833579309860146, + "metric_std": 0.0036418497684262627, + "actual_fraction": 0.5099444795839483, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.7853538548035701, + "metric_std": 0.0039704218521116694, + "actual_fraction": 0.5199943776793872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.787026495185888, + "metric_std": 0.004076969977795329, + "actual_fraction": 0.5299599409656336, + "actual_fraction_std": 3.1429727703942244e-05 + }, + { + "budget": 0.54, + "metric": 0.7886569681636096, + "metric_std": 0.0037019763309658915, + "actual_fraction": 0.5399536158549442, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.7901468831260102, + "metric_std": 0.004039667682395088, + "actual_fraction": 0.5499332349427226, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.7914119052638977, + "metric_std": 0.004386389820117005, + "actual_fraction": 0.5599831330381615, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.7931548246538759, + "metric_std": 0.0043023727083999815, + "actual_fraction": 0.5699627521259399, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.7950945252653032, + "metric_std": 0.00447131373437345, + "actual_fraction": 0.5799423712137185, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.7966547192353645, + "metric_std": 0.004381207091200159, + "actual_fraction": 0.5899922693091574, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.7981586899992973, + "metric_std": 0.0044124919958794745, + "actual_fraction": 0.5999718883969358, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.7993112657249279, + "metric_std": 0.004501314206119455, + "actual_fraction": 0.6099515074847143, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.8007590132827325, + "metric_std": 0.004517305882781325, + "actual_fraction": 0.6199311265724928, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.8017991425961066, + "metric_std": 0.004128674876140917, + "actual_fraction": 0.6299810246679317, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.8032750017569752, + "metric_std": 0.004167373304916436, + "actual_fraction": 0.6399606437557102, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.8043854100780097, + "metric_std": 0.004039545414573714, + "actual_fraction": 0.6499402628434886, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.8052287581699347, + "metric_std": 0.003446753666687173, + "actual_fraction": 0.6599901609389276, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.8061423852695201, + "metric_std": 0.0030519220453772114, + "actual_fraction": 0.669969780026706, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.8070841239721694, + "metric_std": 0.0027908819525848355, + "actual_fraction": 0.6799493991144845, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.8079977510717548, + "metric_std": 0.0024218231695991056, + "actual_fraction": 0.6899992972099234, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.8090097687820649, + "metric_std": 0.0023856591908533197, + "actual_fraction": 0.6999789162977019, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.8098531168739898, + "metric_std": 0.0021919841101545397, + "actual_fraction": 0.7099585353854804, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.8108229671797034, + "metric_std": 0.0017001129652634598, + "actual_fraction": 0.7199381544732588, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.8113430318363906, + "metric_std": 0.001794249354626569, + "actual_fraction": 0.7299880525686977, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.811778761683885, + "metric_std": 0.001553627107479691, + "actual_fraction": 0.7399676716564763, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.8119755429053341, + "metric_std": 0.0016471396442045466, + "actual_fraction": 0.7499472907442547, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.8123831611497645, + "metric_std": 0.0016768577118720955, + "actual_fraction": 0.7599971888396936, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.8125237191650854, + "metric_std": 0.0015386127788592845, + "actual_fraction": 0.769976807927472, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.8124956075620213, + "metric_std": 0.001496629379369356, + "actual_fraction": 0.7799564270152506, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.8124112727528287, + "metric_std": 0.001501900377431282, + "actual_fraction": 0.789936046103029, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.8123831611497645, + "metric_std": 0.0014738498213924679, + "actual_fraction": 0.7999859441984679, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.8123409937451683, + "metric_std": 0.0015247467621138145, + "actual_fraction": 0.8099515074847143, + "actual_fraction_std": 3.1429727703942244e-05 + }, + { + "budget": 0.82, + "metric": 0.8123409937451683, + "metric_std": 0.0014961342712191163, + "actual_fraction": 0.8199451823740249, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.8123269379436362, + "metric_std": 0.001470494820862418, + "actual_fraction": 0.8299950804694639, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.8122988263405719, + "metric_std": 0.0014661218424800461, + "actual_fraction": 0.8399746995572424, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.8122847705390399, + "metric_std": 0.0014716699410620595, + "actual_fraction": 0.8499543186450207, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.8122285473329118, + "metric_std": 0.0014686463037006617, + "actual_fraction": 0.8599339377327994, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.87, + "metric": 0.8122144915313797, + "metric_std": 0.0014437162544222568, + "actual_fraction": 0.8699838358282381, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.88, + "metric": 0.8121723241267833, + "metric_std": 0.0014322096189225287, + "actual_fraction": 0.8799634549160166, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.89, + "metric": 0.8121582683252513, + "metric_std": 0.0014686463037006962, + "actual_fraction": 0.889943074003795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.8120458219129946, + "metric_std": 0.0015450196888518821, + "actual_fraction": 0.8999929720992339, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.91, + "metric": 0.8118209290884814, + "metric_std": 0.0014148614785048346, + "actual_fraction": 0.9099725911870123, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.8116100920655, + "metric_std": 0.0013040496652256984, + "actual_fraction": 0.919952210274791, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.93, + "metric": 0.811455478248647, + "metric_std": 0.0012838187788185894, + "actual_fraction": 0.9299318293625694, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.8113008644317942, + "metric_std": 0.00132229163836016, + "actual_fraction": 0.9399817274580083, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.95, + "metric": 0.8110619158057488, + "metric_std": 0.001361491401304081, + "actual_fraction": 0.9499613465457868, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.8107386323705109, + "metric_std": 0.001248318200037361, + "actual_fraction": 0.9599409656335652, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.8101623445076955, + "metric_std": 0.0009981103745110326, + "actual_fraction": 0.9699908637290042, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.98, + "metric": 0.8093611638203668, + "metric_std": 0.0007724299199681942, + "actual_fraction": 0.9799704828167826, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.8080118068732869, + "metric_std": 0.0004979405322627838, + "actual_fraction": 0.9899501019045612, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 1.0, + "metric": 0.8060299388572634, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.26688029946402747, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.27997050733062984, + "metric_std": 0.0018893548445667674, + "actual_fraction": 0.013589314579020503, + "actual_fraction_std": 0.0018965558116069757 + }, + { + "budget": 0.02, + "metric": 0.2948472903610016, + "metric_std": 0.0026112315074237214, + "actual_fraction": 0.02948132603578822, + "actual_fraction_std": 0.0027710462264887385 + }, + { + "budget": 0.03, + "metric": 0.3090321299946119, + "metric_std": 0.003970956934234381, + "actual_fraction": 0.044964977455122936, + "actual_fraction_std": 0.004119559542327317 + }, + { + "budget": 0.04, + "metric": 0.3236366730000283, + "metric_std": 0.0043447339949398735, + "actual_fraction": 0.061322065621189345, + "actual_fraction_std": 0.004887006109251926 + }, + { + "budget": 0.05, + "metric": 0.3373450925899668, + "metric_std": 0.0030178919248727188, + "actual_fraction": 0.0768907920483226, + "actual_fraction_std": 0.0034611462276553594 + }, + { + "budget": 0.06, + "metric": 0.3514051555454726, + "metric_std": 0.003263340975211296, + "actual_fraction": 0.09320817854408303, + "actual_fraction_std": 0.003765705126306685 + }, + { + "budget": 0.07, + "metric": 0.36448402007770186, + "metric_std": 0.003530322395000869, + "actual_fraction": 0.10875988996965658, + "actual_fraction_std": 0.004029898143715629 + }, + { + "budget": 0.08, + "metric": 0.3781470663301477, + "metric_std": 0.0038396286245434403, + "actual_fraction": 0.1255423531747157, + "actual_fraction_std": 0.003791754486696723 + }, + { + "budget": 0.09, + "metric": 0.3909366758358619, + "metric_std": 0.004722092972083808, + "actual_fraction": 0.14134361795649833, + "actual_fraction_std": 0.004290898301277685 + }, + { + "budget": 0.1, + "metric": 0.40404389870402396, + "metric_std": 0.004711556378543797, + "actual_fraction": 0.15792190114284094, + "actual_fraction_std": 0.004618727666751605 + }, + { + "budget": 0.11, + "metric": 0.4161075348098574, + "metric_std": 0.004798444966188633, + "actual_fraction": 0.17372316592462356, + "actual_fraction_std": 0.005403360745761528 + }, + { + "budget": 0.12, + "metric": 0.4282392309219295, + "metric_std": 0.004791803609536213, + "actual_fraction": 0.1892692056830105, + "actual_fraction_std": 0.005826047404364911 + }, + { + "budget": 0.13, + "metric": 0.43928196693418026, + "metric_std": 0.005162486876106892, + "actual_fraction": 0.20417434704931514, + "actual_fraction_std": 0.007000414763975217 + }, + { + "budget": 0.14, + "metric": 0.44913932450443805, + "metric_std": 0.005614091105421223, + "actual_fraction": 0.21746873493463403, + "actual_fraction_std": 0.007499388612138117 + }, + { + "budget": 0.15, + "metric": 0.4591554887559198, + "metric_std": 0.00493689226913374, + "actual_fraction": 0.23150043955420693, + "actual_fraction_std": 0.006719507549687359 + }, + { + "budget": 0.16, + "metric": 0.46992031307602866, + "metric_std": 0.004431138584696091, + "actual_fraction": 0.24654737260017584, + "actual_fraction_std": 0.005919356856912024 + }, + { + "budget": 0.17, + "metric": 0.47930125060261464, + "metric_std": 0.005096640733247387, + "actual_fraction": 0.2597793721464424, + "actual_fraction_std": 0.007161549498195312 + }, + { + "budget": 0.18, + "metric": 0.4885800981198424, + "metric_std": 0.0054768089373147455, + "actual_fraction": 0.27313047670362706, + "actual_fraction_std": 0.007593486571323966 + }, + { + "budget": 0.19, + "metric": 0.49750730227150275, + "metric_std": 0.005083369155133303, + "actual_fraction": 0.28643620792331903, + "actual_fraction_std": 0.007711475777591609 + }, + { + "budget": 0.2, + "metric": 0.5059977880497973, + "metric_std": 0.005255825682252722, + "actual_fraction": 0.299322235771205, + "actual_fraction_std": 0.007710573655126433 + }, + { + "budget": 0.21, + "metric": 0.5137623004282108, + "metric_std": 0.005493010402209459, + "actual_fraction": 0.3114766185520234, + "actual_fraction_std": 0.007667891623850209 + }, + { + "budget": 0.22, + "metric": 0.5213906927941468, + "metric_std": 0.005325118626045544, + "actual_fraction": 0.3235402546578567, + "actual_fraction_std": 0.007811859429107187 + }, + { + "budget": 0.23, + "metric": 0.5286390834585827, + "metric_std": 0.005188552615252091, + "actual_fraction": 0.33463970734197324, + "actual_fraction_std": 0.008074823902276757 + }, + { + "budget": 0.24, + "metric": 0.536290162493265, + "metric_std": 0.005211086328918176, + "actual_fraction": 0.3468451351274707, + "actual_fraction_std": 0.008331426204055732 + }, + { + "budget": 0.25, + "metric": 0.543113178118708, + "metric_std": 0.005023805472747157, + "actual_fraction": 0.3576383177835124, + "actual_fraction_std": 0.00789830821267428 + }, + { + "budget": 0.26, + "metric": 0.5500666420894422, + "metric_std": 0.004879035008572042, + "actual_fraction": 0.3689532938207186, + "actual_fraction_std": 0.007842313094023825 + }, + { + "budget": 0.27, + "metric": 0.557275331083572, + "metric_std": 0.005137376914752037, + "actual_fraction": 0.3810112582593653, + "actual_fraction_std": 0.007968455233140255 + }, + { + "budget": 0.28, + "metric": 0.5641437200465076, + "metric_std": 0.005007699931998013, + "actual_fraction": 0.3924283243059298, + "actual_fraction_std": 0.007319154105570272 + }, + { + "budget": 0.29, + "metric": 0.5698834472393159, + "metric_std": 0.0044481204749812955, + "actual_fraction": 0.4027110569151803, + "actual_fraction_std": 0.006887614578734751 + }, + { + "budget": 0.3, + "metric": 0.5760315344695572, + "metric_std": 0.004805026934116809, + "actual_fraction": 0.41368573292119215, + "actual_fraction_std": 0.007861767512944533 + }, + { + "budget": 0.31, + "metric": 0.5825539517341122, + "metric_std": 0.004787505304149561, + "actual_fraction": 0.4248419022771744, + "actual_fraction_std": 0.008381085374666367 + }, + { + "budget": 0.32, + "metric": 0.588497858945637, + "metric_std": 0.004660328203433668, + "actual_fraction": 0.4354422482488728, + "actual_fraction_std": 0.008122423468842472 + }, + { + "budget": 0.33, + "metric": 0.5944814678274679, + "metric_std": 0.005159323639797239, + "actual_fraction": 0.44603692255338456, + "actual_fraction_std": 0.00882442429335068 + }, + { + "budget": 0.34, + "metric": 0.5999829849984402, + "metric_std": 0.005751200571334529, + "actual_fraction": 0.45615517681422457, + "actual_fraction_std": 0.009988227014272441 + }, + { + "budget": 0.35, + "metric": 0.6056716671865694, + "metric_std": 0.0056956289903017975, + "actual_fraction": 0.4666420894421915, + "actual_fraction_std": 0.010136258661362823 + }, + { + "budget": 0.36, + "metric": 0.6120863227745795, + "metric_std": 0.005485890584371898, + "actual_fraction": 0.47875109888551737, + "actual_fraction_std": 0.009790741662444803 + }, + { + "budget": 0.37, + "metric": 0.6178770949720669, + "metric_std": 0.005331269107035659, + "actual_fraction": 0.4892947281853501, + "actual_fraction_std": 0.009639602825181707 + }, + { + "budget": 0.38, + "metric": 0.6231971187930692, + "metric_std": 0.005304957149323496, + "actual_fraction": 0.5001786575163769, + "actual_fraction_std": 0.009812709556456458 + }, + { + "budget": 0.39, + "metric": 0.6281881859172503, + "metric_std": 0.005196250113860674, + "actual_fraction": 0.5105975101381052, + "actual_fraction_std": 0.009605973470539368 + }, + { + "budget": 0.4, + "metric": 0.6332019397101779, + "metric_std": 0.004837968679958111, + "actual_fraction": 0.5205399427161614, + "actual_fraction_std": 0.009248065973589478 + }, + { + "budget": 0.41, + "metric": 0.6385786802030456, + "metric_std": 0.004866467269317707, + "actual_fraction": 0.5309814820066359, + "actual_fraction_std": 0.009835417446438018 + }, + { + "budget": 0.42, + "metric": 0.6440631823724584, + "metric_std": 0.005046772034912898, + "actual_fraction": 0.5419448146782747, + "actual_fraction_std": 0.009684083150977855 + }, + { + "budget": 0.43, + "metric": 0.6487933528060573, + "metric_std": 0.00507410594361071, + "actual_fraction": 0.5524033689703087, + "actual_fraction_std": 0.00937438401607063 + }, + { + "budget": 0.44, + "metric": 0.653580239911522, + "metric_std": 0.0049702949217102655, + "actual_fraction": 0.5626747582451862, + "actual_fraction_std": 0.009224430967198993 + }, + { + "budget": 0.45, + "metric": 0.658480560360718, + "metric_std": 0.004813697007662453, + "actual_fraction": 0.5731843575418994, + "actual_fraction_std": 0.009031793873073115 + }, + { + "budget": 0.46, + "metric": 0.6633184924708618, + "metric_std": 0.004606182971605393, + "actual_fraction": 0.5839605251963814, + "actual_fraction_std": 0.009112129520385247 + }, + { + "budget": 0.47, + "metric": 0.6681961262513115, + "metric_std": 0.004148205191905422, + "actual_fraction": 0.594821767858662, + "actual_fraction_std": 0.008239582704584062 + }, + { + "budget": 0.48, + "metric": 0.6728582366786716, + "metric_std": 0.0038183317279186276, + "actual_fraction": 0.6053200238210023, + "actual_fraction_std": 0.007217059994752356 + }, + { + "budget": 0.49, + "metric": 0.677214077077957, + "metric_std": 0.0038272513555842225, + "actual_fraction": 0.6156991747724244, + "actual_fraction_std": 0.006624831395936569 + }, + { + "budget": 0.5, + "metric": 0.6820349941865411, + "metric_std": 0.004044698067588128, + "actual_fraction": 0.6265320590987721, + "actual_fraction_std": 0.007519628670875167 + }, + { + "budget": 0.51, + "metric": 0.6869069562998045, + "metric_std": 0.0037987787814327653, + "actual_fraction": 0.6380001701500155, + "actual_fraction_std": 0.007342821464339821 + }, + { + "budget": 0.52, + "metric": 0.6914669767178061, + "metric_std": 0.003579300593908503, + "actual_fraction": 0.6490145478263336, + "actual_fraction_std": 0.006822947263359786 + }, + { + "budget": 0.53, + "metric": 0.6959929671326888, + "metric_std": 0.0035859225116209315, + "actual_fraction": 0.6599268354932932, + "actual_fraction_std": 0.006826882861092371 + }, + { + "budget": 0.54, + "metric": 0.700757167569407, + "metric_std": 0.003169886114098323, + "actual_fraction": 0.6715707682273204, + "actual_fraction_std": 0.007009461322659707 + }, + { + "budget": 0.55, + "metric": 0.7047443496015655, + "metric_std": 0.0028760022712401197, + "actual_fraction": 0.6821143975271531, + "actual_fraction_std": 0.00675239907549953 + }, + { + "budget": 0.56, + "metric": 0.7093610866914329, + "metric_std": 0.003097711763037068, + "actual_fraction": 0.6936051952471429, + "actual_fraction_std": 0.006988232816758843 + }, + { + "budget": 0.57, + "metric": 0.7135411054079346, + "metric_std": 0.0029103998660589847, + "actual_fraction": 0.7043700195672519, + "actual_fraction_std": 0.006742530574412777 + }, + { + "budget": 0.58, + "metric": 0.7173808240932422, + "metric_std": 0.002669903818946732, + "actual_fraction": 0.7147605138530471, + "actual_fraction_std": 0.006527920159272148 + }, + { + "budget": 0.59, + "metric": 0.721231886112923, + "metric_std": 0.0025120267929092757, + "actual_fraction": 0.7254969798372233, + "actual_fraction_std": 0.006117761366303903 + }, + { + "budget": 0.6, + "metric": 0.7249014547826333, + "metric_std": 0.0027867314098864015, + "actual_fraction": 0.7360235941354962, + "actual_fraction_std": 0.006650939720299822 + }, + { + "budget": 0.61, + "metric": 0.7286277401242096, + "metric_std": 0.0028447948477932144, + "actual_fraction": 0.7463857300853587, + "actual_fraction_std": 0.0070208333098637195 + }, + { + "budget": 0.62, + "metric": 0.7315373053909197, + "metric_std": 0.002553069343331215, + "actual_fraction": 0.75622040098687, + "actual_fraction_std": 0.007153937570542101 + }, + { + "budget": 0.63, + "metric": 0.7343220939795254, + "metric_std": 0.0024608598581945494, + "actual_fraction": 0.7653744718259932, + "actual_fraction_std": 0.006768963114296589 + }, + { + "budget": 0.64, + "metric": 0.7371295692368772, + "metric_std": 0.0021908168140915697, + "actual_fraction": 0.7758840711227066, + "actual_fraction_std": 0.006052262869048993 + }, + { + "budget": 0.65, + "metric": 0.7395967444630349, + "metric_std": 0.002242784038319898, + "actual_fraction": 0.7852082919774268, + "actual_fraction_std": 0.006133089247531163 + }, + { + "budget": 0.66, + "metric": 0.7417746646626775, + "metric_std": 0.00233051351342232, + "actual_fraction": 0.794611916172759, + "actual_fraction_std": 0.006018437997060289 + }, + { + "budget": 0.67, + "metric": 0.7436066131639396, + "metric_std": 0.001964150018307604, + "actual_fraction": 0.8036525536681507, + "actual_fraction_std": 0.00567370153232735 + }, + { + "budget": 0.68, + "metric": 0.7455576666761196, + "metric_std": 0.0018411746516056706, + "actual_fraction": 0.8130731928650426, + "actual_fraction_std": 0.005586109922293983 + }, + { + "budget": 0.69, + "metric": 0.746867821796217, + "metric_std": 0.0016741508802586727, + "actual_fraction": 0.8214218869636729, + "actual_fraction_std": 0.00538619542816535 + }, + { + "budget": 0.7, + "metric": 0.7482233502538072, + "metric_std": 0.0016777736789107972, + "actual_fraction": 0.8303434194481468, + "actual_fraction_std": 0.005473364539310233 + }, + { + "budget": 0.71, + "metric": 0.7491591753395911, + "metric_std": 0.0014061846248857436, + "actual_fraction": 0.8385956952046053, + "actual_fraction_std": 0.004539752425138923 + }, + { + "budget": 0.72, + "metric": 0.7499191787425914, + "metric_std": 0.0012208255384960306, + "actual_fraction": 0.8466267759407877, + "actual_fraction_std": 0.004144811151495381 + }, + { + "budget": 0.73, + "metric": 0.7504863454612483, + "metric_std": 0.0012670496878410186, + "actual_fraction": 0.8548223350253806, + "actual_fraction_std": 0.0031793982193398206 + }, + { + "budget": 0.74, + "metric": 0.7508663471627485, + "metric_std": 0.0012373142019611798, + "actual_fraction": 0.8626152057397272, + "actual_fraction_std": 0.002911988259823556 + }, + { + "budget": 0.75, + "metric": 0.7510194821767859, + "metric_std": 0.0013587460982066987, + "actual_fraction": 0.870152851430678, + "actual_fraction_std": 0.002816155530205638 + }, + { + "budget": 0.76, + "metric": 0.7510932138502112, + "metric_std": 0.0012502456730779149, + "actual_fraction": 0.8779513938122111, + "actual_fraction_std": 0.002714368309292558 + }, + { + "budget": 0.77, + "metric": 0.7511782888580097, + "metric_std": 0.0012087106562392318, + "actual_fraction": 0.8862093412358563, + "actual_fraction_std": 0.0031052442591459942 + }, + { + "budget": 0.78, + "metric": 0.75116127385645, + "metric_std": 0.0012785490281379025, + "actual_fraction": 0.8943084819782776, + "actual_fraction_std": 0.002529859212197652 + }, + { + "budget": 0.79, + "metric": 0.7511556021892635, + "metric_std": 0.0012962892831737689, + "actual_fraction": 0.9013867226271163, + "actual_fraction_std": 0.0037226513717839894 + }, + { + "budget": 0.8, + "metric": 0.7511442588548903, + "metric_std": 0.0012840722149355403, + "actual_fraction": 0.9075858548620367, + "actual_fraction_std": 0.004471382127027707 + }, + { + "budget": 0.81, + "metric": 0.7511442588548903, + "metric_std": 0.0012777940165894749, + "actual_fraction": 0.9125655786518447, + "actual_fraction_std": 0.005283158745382697 + }, + { + "budget": 0.82, + "metric": 0.7511102288517709, + "metric_std": 0.0012710103664679304, + "actual_fraction": 0.9171653007401526, + "actual_fraction_std": 0.006020388560947863 + }, + { + "budget": 0.83, + "metric": 0.7510875421830246, + "metric_std": 0.001281250832669991, + "actual_fraction": 0.9215097978050647, + "actual_fraction_std": 0.006296553258222334 + }, + { + "budget": 0.84, + "metric": 0.7510932138502112, + "metric_std": 0.0012481534257215042, + "actual_fraction": 0.9252417548138275, + "actual_fraction_std": 0.006172594246083751 + }, + { + "budget": 0.85, + "metric": 0.7511102288517709, + "metric_std": 0.0012736965929971004, + "actual_fraction": 0.9290928168335082, + "actual_fraction_std": 0.006061709562114552 + }, + { + "budget": 0.86, + "metric": 0.7510932138502112, + "metric_std": 0.001277605193986676, + "actual_fraction": 0.932836117176644, + "actual_fraction_std": 0.005848098087996701 + }, + { + "budget": 0.87, + "metric": 0.7510648555142784, + "metric_std": 0.0012890415653181453, + "actual_fraction": 0.9363865808354366, + "actual_fraction_std": 0.00613600606301309 + }, + { + "budget": 0.88, + "metric": 0.7509967955080395, + "metric_std": 0.001291565770053767, + "actual_fraction": 0.9404871962113261, + "actual_fraction_std": 0.005453751126965734 + }, + { + "budget": 0.89, + "metric": 0.7509741088392933, + "metric_std": 0.001313330842234637, + "actual_fraction": 0.9441454215466637, + "actual_fraction_std": 0.005611426100190744 + }, + { + "budget": 0.9, + "metric": 0.7509230638346142, + "metric_std": 0.0013036200874147863, + "actual_fraction": 0.947883050222613, + "actual_fraction_std": 0.005015514651863455 + }, + { + "budget": 0.91, + "metric": 0.7508833621643081, + "metric_std": 0.0013756862301717763, + "actual_fraction": 0.9521311289453536, + "actual_fraction_std": 0.004397919867526741 + }, + { + "budget": 0.92, + "metric": 0.7508776904971216, + "metric_std": 0.0013693582391761615, + "actual_fraction": 0.9563735360009075, + "actual_fraction_std": 0.003992567688838401 + }, + { + "budget": 0.93, + "metric": 0.7508493321611888, + "metric_std": 0.0013734630392682435, + "actual_fraction": 0.9604401213736778, + "actual_fraction_std": 0.004006311300448832 + }, + { + "budget": 0.94, + "metric": 0.7507756004877634, + "metric_std": 0.0013150136785819323, + "actual_fraction": 0.9648413351104559, + "actual_fraction_std": 0.0035053472806209087 + }, + { + "budget": 0.95, + "metric": 0.7506791821455916, + "metric_std": 0.001193478348416188, + "actual_fraction": 0.9692595638487933, + "actual_fraction_std": 0.003561597008851539 + }, + { + "budget": 0.96, + "metric": 0.7504523154581288, + "metric_std": 0.0012010349539118474, + "actual_fraction": 0.9739330176105266, + "actual_fraction_std": 0.0023064934855112915 + }, + { + "budget": 0.97, + "metric": 0.7501857471003601, + "metric_std": 0.0011167208795469685, + "actual_fraction": 0.9787936363894166, + "actual_fraction_std": 0.0023682521575132333 + }, + { + "budget": 0.98, + "metric": 0.7497887303973003, + "metric_std": 0.0009994855168578743, + "actual_fraction": 0.9834727618183365, + "actual_fraction_std": 0.0019104975348987734 + }, + { + "budget": 0.99, + "metric": 0.7483935002694042, + "metric_std": 0.0006256854094329568, + "actual_fraction": 0.9899157757422794, + "actual_fraction_std": 0.0014699359539496264 + }, + { + "budget": 1.0, + "metric": 0.7447182599325072, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 6843, + "rescuable": 4626, + "harmful": 1133, + "both_wrong": 1627, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 7441, + "rescuable": 18820, + "harmful": 1970, + "both_wrong": 7032, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.5605453650994447, + "test": 0.26688029946402747 + }, + "large": { + "validation": 0.8060299388572634, + "test": 0.7447182599325072 + }, + "learned_minus_confidence_test": 0.012403936137027394, + "learned_minus_small_test": 0.48422992938774345, + "learned_minus_large_test": 0.006391968919263746, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_groundingdino_Qwen3.8" + }, + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "expert_display": "InstanceVG", + "run": "five_seed_instancevg_MiniCPM_four_difference", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detections_above_07", + "max_score", + "mean_score", + "query_count", + "detections_above_07_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 65, + "validation_area_0_25": 0.7465176751704266, + "validation_point": { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.7716635041113219, + "rescued": 1117, + "harmed": 813, + "missed_rescue": 252, + "avoided_harm": 2108, + "selected_both_correct": 5984, + "selected_both_wrong": 1334 + }, + "test_point": { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.6187788900547316, + "rescued": 1563, + "harmed": 3352, + "missed_rescue": 1064, + "avoided_harm": 5250, + "selected_both_correct": 8439, + "selected_both_wrong": 5016 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1369, + "avoided_harm": 2921, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7498770117365943, + "rescued": 0, + "harmed": 6, + "missed_rescue": 1369, + "avoided_harm": 2915, + "selected_both_correct": 136, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7497364537212735, + "rescued": 0, + "harmed": 8, + "missed_rescue": 1369, + "avoided_harm": 2913, + "selected_both_correct": 276, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7494553376906318, + "rescued": 0, + "harmed": 12, + "missed_rescue": 1369, + "avoided_harm": 2909, + "selected_both_correct": 414, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.749314779675311, + "rescued": 0, + "harmed": 14, + "missed_rescue": 1369, + "avoided_harm": 2907, + "selected_both_correct": 555, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7492445006676506, + "rescued": 0, + "harmed": 15, + "missed_rescue": 1369, + "avoided_harm": 2906, + "selected_both_correct": 696, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7491742216599901, + "rescued": 0, + "harmed": 16, + "missed_rescue": 1369, + "avoided_harm": 2905, + "selected_both_correct": 837, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7491039426523297, + "rescued": 0, + "harmed": 17, + "missed_rescue": 1369, + "avoided_harm": 2904, + "selected_both_correct": 979, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7488931056293485, + "rescued": 0, + "harmed": 20, + "missed_rescue": 1369, + "avoided_harm": 2901, + "selected_both_correct": 1118, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.7485417105910465, + "rescued": 0, + "harmed": 25, + "missed_rescue": 1369, + "avoided_harm": 2896, + "selected_both_correct": 1255, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.748120036545084, + "rescued": 0, + "harmed": 31, + "missed_rescue": 1369, + "avoided_harm": 2890, + "selected_both_correct": 1391, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7477686415067819, + "rescued": 0, + "harmed": 36, + "missed_rescue": 1369, + "avoided_harm": 2885, + "selected_both_correct": 1529, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7472766884531591, + "rescued": 0, + "harmed": 43, + "missed_rescue": 1369, + "avoided_harm": 2878, + "selected_both_correct": 1664, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.746573898376555, + "rescued": 0, + "harmed": 53, + "missed_rescue": 1369, + "avoided_harm": 2868, + "selected_both_correct": 1796, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7463630613535737, + "rescued": 0, + "harmed": 56, + "missed_rescue": 1369, + "avoided_harm": 2865, + "selected_both_correct": 1936, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7461522243305925, + "rescued": 0, + "harmed": 59, + "missed_rescue": 1369, + "avoided_harm": 2862, + "selected_both_correct": 2075, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.74573055028463, + "rescued": 0, + "harmed": 65, + "missed_rescue": 1369, + "avoided_harm": 2856, + "selected_both_correct": 2211, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7454494342539884, + "rescued": 0, + "harmed": 69, + "missed_rescue": 1369, + "avoided_harm": 2852, + "selected_both_correct": 2349, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7451683182233467, + "rescued": 0, + "harmed": 73, + "missed_rescue": 1369, + "avoided_harm": 2848, + "selected_both_correct": 2488, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7444655281467426, + "rescued": 0, + "harmed": 83, + "missed_rescue": 1369, + "avoided_harm": 2838, + "selected_both_correct": 2620, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7439032960854592, + "rescued": 0, + "harmed": 91, + "missed_rescue": 1369, + "avoided_harm": 2830, + "selected_both_correct": 2754, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7430599479935344, + "rescued": 0, + "harmed": 103, + "missed_rescue": 1369, + "avoided_harm": 2818, + "selected_both_correct": 2885, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7422868789092698, + "rescued": 0, + "harmed": 114, + "missed_rescue": 1369, + "avoided_harm": 2807, + "selected_both_correct": 3016, + "selected_both_wrong": 0 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7413732518096845, + "rescued": 0, + "harmed": 127, + "missed_rescue": 1369, + "avoided_harm": 2794, + "selected_both_correct": 3144, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7406704617330804, + "rescued": 0, + "harmed": 137, + "missed_rescue": 1369, + "avoided_harm": 2784, + "selected_both_correct": 3276, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7400379506641366, + "rescued": 0, + "harmed": 146, + "missed_rescue": 1369, + "avoided_harm": 2775, + "selected_both_correct": 3410, + "selected_both_wrong": 1 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7391946025722117, + "rescued": 0, + "harmed": 158, + "missed_rescue": 1369, + "avoided_harm": 2763, + "selected_both_correct": 3540, + "selected_both_wrong": 1 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.738210696464966, + "rescued": 0, + "harmed": 172, + "missed_rescue": 1369, + "avoided_harm": 2749, + "selected_both_correct": 3668, + "selected_both_wrong": 1 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7374376273807014, + "rescued": 0, + "harmed": 183, + "missed_rescue": 1369, + "avoided_harm": 2738, + "selected_both_correct": 3799, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7362428842504743, + "rescued": 0, + "harmed": 200, + "missed_rescue": 1369, + "avoided_harm": 2721, + "selected_both_correct": 3924, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7352589781432286, + "rescued": 1, + "harmed": 215, + "missed_rescue": 1368, + "avoided_harm": 2706, + "selected_both_correct": 4050, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7342047930283224, + "rescued": 1, + "harmed": 230, + "missed_rescue": 1368, + "avoided_harm": 2691, + "selected_both_correct": 4177, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7327289338674537, + "rescued": 1, + "harmed": 251, + "missed_rescue": 1368, + "avoided_harm": 2670, + "selected_both_correct": 4299, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7313233537142455, + "rescued": 1, + "harmed": 271, + "missed_rescue": 1368, + "avoided_harm": 2650, + "selected_both_correct": 4421, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.7298474945533769, + "rescued": 1, + "harmed": 292, + "missed_rescue": 1368, + "avoided_harm": 2629, + "selected_both_correct": 4541, + "selected_both_wrong": 3 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7284419144001687, + "rescued": 1, + "harmed": 312, + "missed_rescue": 1368, + "avoided_harm": 2609, + "selected_both_correct": 4664, + "selected_both_wrong": 3 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7268957762316396, + "rescued": 1, + "harmed": 334, + "missed_rescue": 1368, + "avoided_harm": 2587, + "selected_both_correct": 4784, + "selected_both_wrong": 3 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.7252793590554502, + "rescued": 1, + "harmed": 357, + "missed_rescue": 1368, + "avoided_harm": 2564, + "selected_both_correct": 4903, + "selected_both_wrong": 3 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7234521048562794, + "rescued": 1, + "harmed": 383, + "missed_rescue": 1368, + "avoided_harm": 2538, + "selected_both_correct": 5019, + "selected_both_wrong": 4 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7216248506571087, + "rescued": 1, + "harmed": 409, + "missed_rescue": 1368, + "avoided_harm": 2512, + "selected_both_correct": 5134, + "selected_both_wrong": 5 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.719446201419636, + "rescued": 1, + "harmed": 440, + "missed_rescue": 1368, + "avoided_harm": 2481, + "selected_both_correct": 5242, + "selected_both_wrong": 8 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7169161571438611, + "rescued": 1, + "harmed": 476, + "missed_rescue": 1368, + "avoided_harm": 2445, + "selected_both_correct": 5347, + "selected_both_wrong": 9 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.7148077869140488, + "rescued": 3, + "harmed": 508, + "missed_rescue": 1366, + "avoided_harm": 2413, + "selected_both_correct": 5450, + "selected_both_wrong": 15 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7126994166842364, + "rescued": 5, + "harmed": 540, + "missed_rescue": 1364, + "avoided_harm": 2381, + "selected_both_correct": 5552, + "selected_both_wrong": 21 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7100288143931408, + "rescued": 5, + "harmed": 578, + "missed_rescue": 1364, + "avoided_harm": 2343, + "selected_both_correct": 5648, + "selected_both_wrong": 29 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7070770960714035, + "rescued": 5, + "harmed": 620, + "missed_rescue": 1364, + "avoided_harm": 2301, + "selected_both_correct": 5742, + "selected_both_wrong": 36 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7045470517956286, + "rescued": 6, + "harmed": 657, + "missed_rescue": 1363, + "avoided_harm": 2264, + "selected_both_correct": 5841, + "selected_both_wrong": 41 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7017358914892122, + "rescued": 7, + "harmed": 698, + "missed_rescue": 1362, + "avoided_harm": 2223, + "selected_both_correct": 5930, + "selected_both_wrong": 52 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7059526319488368, + "rescued": 76, + "harmed": 707, + "missed_rescue": 1293, + "avoided_harm": 2214, + "selected_both_correct": 5945, + "selected_both_wrong": 101 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7123480216459344, + "rescued": 167, + "harmed": 707, + "missed_rescue": 1202, + "avoided_harm": 2214, + "selected_both_correct": 5945, + "selected_both_wrong": 153 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7179000632511069, + "rescued": 247, + "harmed": 708, + "missed_rescue": 1122, + "avoided_harm": 2213, + "selected_both_correct": 5945, + "selected_both_wrong": 214 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.723381825848619, + "rescued": 326, + "harmed": 709, + "missed_rescue": 1043, + "avoided_harm": 2212, + "selected_both_correct": 5945, + "selected_both_wrong": 276 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.729004146461452, + "rescued": 407, + "harmed": 710, + "missed_rescue": 962, + "avoided_harm": 2211, + "selected_both_correct": 5945, + "selected_both_wrong": 337 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.7339939560053412, + "rescued": 478, + "harmed": 710, + "missed_rescue": 891, + "avoided_harm": 2211, + "selected_both_correct": 5946, + "selected_both_wrong": 407 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7390540445568908, + "rescued": 552, + "harmed": 712, + "missed_rescue": 817, + "avoided_harm": 2209, + "selected_both_correct": 5946, + "selected_both_wrong": 473 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.743692459062478, + "rescued": 620, + "harmed": 714, + "missed_rescue": 749, + "avoided_harm": 2207, + "selected_both_correct": 5947, + "selected_both_wrong": 544 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7482605945604048, + "rescued": 690, + "harmed": 719, + "missed_rescue": 679, + "avoided_harm": 2202, + "selected_both_correct": 5948, + "selected_both_wrong": 611 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7525476140276899, + "rescued": 755, + "harmed": 723, + "missed_rescue": 614, + "avoided_harm": 2198, + "selected_both_correct": 5948, + "selected_both_wrong": 684 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.7554993323494272, + "rescued": 802, + "harmed": 728, + "missed_rescue": 567, + "avoided_harm": 2193, + "selected_both_correct": 5949, + "selected_both_wrong": 772 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7591538407477686, + "rescued": 859, + "harmed": 733, + "missed_rescue": 510, + "avoided_harm": 2188, + "selected_both_correct": 5953, + "selected_both_wrong": 850 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7622461170848268, + "rescued": 911, + "harmed": 741, + "missed_rescue": 458, + "avoided_harm": 2180, + "selected_both_correct": 5956, + "selected_both_wrong": 929 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7658303464755077, + "rescued": 970, + "harmed": 749, + "missed_rescue": 399, + "avoided_harm": 2172, + "selected_both_correct": 5959, + "selected_both_wrong": 1001 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7682198327359617, + "rescued": 1014, + "harmed": 759, + "missed_rescue": 355, + "avoided_harm": 2162, + "selected_both_correct": 5961, + "selected_both_wrong": 1087 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.769976807927472, + "rescued": 1049, + "harmed": 769, + "missed_rescue": 320, + "avoided_harm": 2152, + "selected_both_correct": 5971, + "selected_both_wrong": 1175 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.7713121090730198, + "rescued": 1089, + "harmed": 790, + "missed_rescue": 280, + "avoided_harm": 2131, + "selected_both_correct": 5977, + "selected_both_wrong": 1250 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.7716635041113219, + "rescued": 1117, + "harmed": 813, + "missed_rescue": 252, + "avoided_harm": 2108, + "selected_both_correct": 5984, + "selected_both_wrong": 1334 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.7709607140347178, + "rescued": 1141, + "harmed": 847, + "missed_rescue": 228, + "avoided_harm": 2074, + "selected_both_correct": 5993, + "selected_both_wrong": 1410 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.7709607140347178, + "rescued": 1164, + "harmed": 870, + "missed_rescue": 205, + "avoided_harm": 2051, + "selected_both_correct": 6013, + "selected_both_wrong": 1486 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.7703282029657741, + "rescued": 1191, + "harmed": 906, + "missed_rescue": 178, + "avoided_harm": 2015, + "selected_both_correct": 6033, + "selected_both_wrong": 1545 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.7687117857895847, + "rescued": 1208, + "harmed": 946, + "missed_rescue": 161, + "avoided_harm": 1975, + "selected_both_correct": 6063, + "selected_both_wrong": 1601 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.6999086372900415, + "metric": 0.7656195094525265, + "rescued": 1224, + "harmed": 1006, + "missed_rescue": 145, + "avoided_harm": 1915, + "selected_both_correct": 6082, + "selected_both_wrong": 1647 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.7630191861690913, + "rescued": 1234, + "harmed": 1053, + "missed_rescue": 135, + "avoided_harm": 1868, + "selected_both_correct": 6118, + "selected_both_wrong": 1697 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.7605594209009768, + "rescued": 1247, + "harmed": 1101, + "missed_rescue": 122, + "avoided_harm": 1820, + "selected_both_correct": 6158, + "selected_both_wrong": 1738 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.7559210063953897, + "rescued": 1253, + "harmed": 1173, + "missed_rescue": 116, + "avoided_harm": 1748, + "selected_both_correct": 6185, + "selected_both_wrong": 1776 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.7533206831119544, + "rescued": 1265, + "harmed": 1222, + "missed_rescue": 104, + "avoided_harm": 1699, + "selected_both_correct": 6233, + "selected_both_wrong": 1809 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.7491742216599901, + "rescued": 1272, + "harmed": 1288, + "missed_rescue": 97, + "avoided_harm": 1633, + "selected_both_correct": 6272, + "selected_both_wrong": 1839 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.7455899922693091, + "rescued": 1280, + "harmed": 1347, + "missed_rescue": 89, + "avoided_harm": 1574, + "selected_both_correct": 6315, + "selected_both_wrong": 1872 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.7415840888326657, + "rescued": 1288, + "harmed": 1412, + "missed_rescue": 81, + "avoided_harm": 1509, + "selected_both_correct": 6353, + "selected_both_wrong": 1903 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.7378593014266639, + "rescued": 1295, + "harmed": 1472, + "missed_rescue": 74, + "avoided_harm": 1449, + "selected_both_correct": 6399, + "selected_both_wrong": 1932 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.7342750720359829, + "rescued": 1306, + "harmed": 1534, + "missed_rescue": 63, + "avoided_harm": 1387, + "selected_both_correct": 6446, + "selected_both_wrong": 1954 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.7297069365380561, + "rescued": 1316, + "harmed": 1609, + "missed_rescue": 53, + "avoided_harm": 1312, + "selected_both_correct": 6491, + "selected_both_wrong": 1967 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.7263335441703563, + "rescued": 1324, + "harmed": 1665, + "missed_rescue": 45, + "avoided_harm": 1256, + "selected_both_correct": 6553, + "selected_both_wrong": 1983 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.7221168037107316, + "rescued": 1329, + "harmed": 1730, + "missed_rescue": 40, + "avoided_harm": 1191, + "selected_both_correct": 6599, + "selected_both_wrong": 2009 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.718251458289409, + "rescued": 1333, + "harmed": 1789, + "missed_rescue": 36, + "avoided_harm": 1132, + "selected_both_correct": 6654, + "selected_both_wrong": 2034 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.7131913697378593, + "rescued": 1337, + "harmed": 1865, + "missed_rescue": 32, + "avoided_harm": 1056, + "selected_both_correct": 6699, + "selected_both_wrong": 2051 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.7093260243165367, + "rescued": 1340, + "harmed": 1923, + "missed_rescue": 29, + "avoided_harm": 998, + "selected_both_correct": 6763, + "selected_both_wrong": 2068 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.7037037037037037, + "rescued": 1343, + "harmed": 2006, + "missed_rescue": 26, + "avoided_harm": 915, + "selected_both_correct": 6810, + "selected_both_wrong": 2077 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.6985030571368332, + "rescued": 1347, + "harmed": 2084, + "missed_rescue": 22, + "avoided_harm": 837, + "selected_both_correct": 6856, + "selected_both_wrong": 2092 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.6937943636235856, + "rescued": 1353, + "harmed": 2157, + "missed_rescue": 16, + "avoided_harm": 764, + "selected_both_correct": 6910, + "selected_both_wrong": 2101 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.6895776231639609, + "rescued": 1356, + "harmed": 2220, + "missed_rescue": 13, + "avoided_harm": 701, + "selected_both_correct": 6973, + "selected_both_wrong": 2114 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.6846580926277321, + "rescued": 1359, + "harmed": 2293, + "missed_rescue": 10, + "avoided_harm": 628, + "selected_both_correct": 7029, + "selected_both_wrong": 2125 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.6794574460608617, + "rescued": 1359, + "harmed": 2367, + "missed_rescue": 10, + "avoided_harm": 554, + "selected_both_correct": 7084, + "selected_both_wrong": 2138 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.6744676365169724, + "rescued": 1361, + "harmed": 2440, + "missed_rescue": 8, + "avoided_harm": 481, + "selected_both_correct": 7144, + "selected_both_wrong": 2145 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.6701806170496872, + "rescued": 1364, + "harmed": 2504, + "missed_rescue": 5, + "avoided_harm": 417, + "selected_both_correct": 7216, + "selected_both_wrong": 2148 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.6651908075057981, + "rescued": 1367, + "harmed": 2578, + "missed_rescue": 2, + "avoided_harm": 343, + "selected_both_correct": 7278, + "selected_both_wrong": 2152 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.6606226720078713, + "rescued": 1367, + "harmed": 2643, + "missed_rescue": 2, + "avoided_harm": 278, + "selected_both_correct": 7349, + "selected_both_wrong": 2158 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.6557031414716424, + "rescued": 1368, + "harmed": 2714, + "missed_rescue": 1, + "avoided_harm": 207, + "selected_both_correct": 7414, + "selected_both_wrong": 2163 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.6521891910886218, + "rescued": 1368, + "harmed": 2764, + "missed_rescue": 1, + "avoided_harm": 157, + "selected_both_correct": 7498, + "selected_both_wrong": 2172 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.6481832876519783, + "rescued": 1369, + "harmed": 2822, + "missed_rescue": 0, + "avoided_harm": 99, + "selected_both_correct": 7574, + "selected_both_wrong": 2179 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6447396162766181, + "rescued": 1369, + "harmed": 2871, + "missed_rescue": 0, + "avoided_harm": 50, + "selected_both_correct": 7664, + "selected_both_wrong": 2182 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.6412256658935975, + "rescued": 1369, + "harmed": 2921, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7755, + "selected_both_wrong": 2184 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2627, + "avoided_harm": 8602, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.002467175226157729, + "metric": 0.6686044862887446, + "rescued": 0, + "harmed": 32, + "missed_rescue": 2627, + "avoided_harm": 8570, + "selected_both_correct": 55, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.004849275444516916, + "metric": 0.6678388112185577, + "rescued": 0, + "harmed": 59, + "missed_rescue": 2627, + "avoided_harm": 8543, + "selected_both_correct": 112, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.007600034030003119, + "metric": 0.667214927828035, + "rescued": 0, + "harmed": 81, + "missed_rescue": 2627, + "avoided_harm": 8521, + "selected_both_correct": 186, + "selected_both_wrong": 1 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.009811984232765221, + "metric": 0.6666761194453109, + "rescued": 0, + "harmed": 100, + "missed_rescue": 2627, + "avoided_harm": 8502, + "selected_both_correct": 245, + "selected_both_wrong": 1 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.012534384482318578, + "metric": 0.6662791027422511, + "rescued": 0, + "harmed": 114, + "missed_rescue": 2627, + "avoided_harm": 8488, + "selected_both_correct": 326, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.01607917647392451, + "metric": 0.6658537277032583, + "rescued": 0, + "harmed": 129, + "missed_rescue": 2627, + "avoided_harm": 8473, + "selected_both_correct": 436, + "selected_both_wrong": 2 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.019709043473328986, + "metric": 0.6654850693361314, + "rescued": 0, + "harmed": 142, + "missed_rescue": 2627, + "avoided_harm": 8460, + "selected_both_correct": 550, + "selected_both_wrong": 3 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.02353741882426339, + "metric": 0.6652865609846014, + "rescued": 0, + "harmed": 149, + "missed_rescue": 2627, + "avoided_harm": 8453, + "selected_both_correct": 678, + "selected_both_wrong": 3 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.028868785979638715, + "metric": 0.6648611859456087, + "rescued": 0, + "harmed": 164, + "missed_rescue": 2627, + "avoided_harm": 8438, + "selected_both_correct": 850, + "selected_both_wrong": 4 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.0345971698380739, + "metric": 0.6645208859144145, + "rescued": 0, + "harmed": 176, + "missed_rescue": 2627, + "avoided_harm": 8426, + "selected_both_correct": 1036, + "selected_both_wrong": 8 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.03981510364971783, + "metric": 0.6642089442191532, + "rescued": 0, + "harmed": 187, + "missed_rescue": 2627, + "avoided_harm": 8415, + "selected_both_correct": 1207, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.045855429203414344, + "metric": 0.6640387942035562, + "rescued": 0, + "harmed": 193, + "missed_rescue": 2627, + "avoided_harm": 8409, + "selected_both_correct": 1413, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.05317187987408899, + "metric": 0.6637268525082948, + "rescued": 1, + "harmed": 205, + "missed_rescue": 2626, + "avoided_harm": 8397, + "selected_both_correct": 1657, + "selected_both_wrong": 12 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.05960922213084536, + "metric": 0.6633581941411678, + "rescued": 1, + "harmed": 218, + "missed_rescue": 2626, + "avoided_harm": 8384, + "selected_both_correct": 1869, + "selected_both_wrong": 14 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.0675211978561098, + "metric": 0.6630178941099736, + "rescued": 1, + "harmed": 230, + "missed_rescue": 2626, + "avoided_harm": 8372, + "selected_both_correct": 2122, + "selected_both_wrong": 28 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.07551824858917279, + "metric": 0.6626492357428466, + "rescued": 1, + "harmed": 243, + "missed_rescue": 2626, + "avoided_harm": 8359, + "selected_both_correct": 2388, + "selected_both_wrong": 31 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.08212574086152624, + "metric": 0.6621671440319882, + "rescued": 1, + "harmed": 260, + "missed_rescue": 2626, + "avoided_harm": 8342, + "selected_both_correct": 2598, + "selected_both_wrong": 37 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.08856308311828262, + "metric": 0.6619969940163911, + "rescued": 1, + "harmed": 266, + "missed_rescue": 2626, + "avoided_harm": 8336, + "selected_both_correct": 2814, + "selected_both_wrong": 42 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.09480191702350906, + "metric": 0.6615716189773984, + "rescued": 1, + "harmed": 281, + "missed_rescue": 2626, + "avoided_harm": 8321, + "selected_both_correct": 3009, + "selected_both_wrong": 52 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.10007656750701868, + "metric": 0.6612029606102714, + "rescued": 1, + "harmed": 294, + "missed_rescue": 2626, + "avoided_harm": 8308, + "selected_both_correct": 3174, + "selected_both_wrong": 60 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.10554972634205825, + "metric": 0.6608910189150101, + "rescued": 1, + "harmed": 305, + "missed_rescue": 2626, + "avoided_harm": 8297, + "selected_both_correct": 3344, + "selected_both_wrong": 72 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.11005870175538099, + "metric": 0.6605790772197487, + "rescued": 1, + "harmed": 316, + "missed_rescue": 2626, + "avoided_harm": 8286, + "selected_both_correct": 3488, + "selected_both_wrong": 76 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6602671355244875, + "rescued": 3, + "harmed": 329, + "missed_rescue": 2624, + "avoided_harm": 8273, + "selected_both_correct": 3638, + "selected_both_wrong": 95 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.12029606102713893, + "metric": 0.6599835521651589, + "rescued": 4, + "harmed": 340, + "missed_rescue": 2623, + "avoided_harm": 8262, + "selected_both_correct": 3778, + "selected_both_wrong": 120 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.12477667810452883, + "metric": 0.6596999688058305, + "rescued": 4, + "harmed": 350, + "missed_rescue": 2623, + "avoided_harm": 8252, + "selected_both_correct": 3906, + "selected_both_wrong": 140 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.12914386183818732, + "metric": 0.659217877094972, + "rescued": 4, + "harmed": 367, + "missed_rescue": 2623, + "avoided_harm": 8235, + "selected_both_correct": 4023, + "selected_both_wrong": 160 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.1322916371267334, + "metric": 0.6587925020559794, + "rescued": 4, + "harmed": 382, + "missed_rescue": 2623, + "avoided_harm": 8220, + "selected_both_correct": 4102, + "selected_both_wrong": 177 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.13626180415733205, + "metric": 0.6582253353373224, + "rescued": 5, + "harmed": 403, + "missed_rescue": 2622, + "avoided_harm": 8199, + "selected_both_correct": 4197, + "selected_both_wrong": 200 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.14023197118793068, + "metric": 0.6578283186342625, + "rescued": 6, + "harmed": 418, + "missed_rescue": 2621, + "avoided_harm": 8184, + "selected_both_correct": 4295, + "selected_both_wrong": 226 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.14326631313274538, + "metric": 0.6573178685874713, + "rescued": 7, + "harmed": 437, + "missed_rescue": 2620, + "avoided_harm": 8165, + "selected_both_correct": 4366, + "selected_both_wrong": 242 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.14766185520233674, + "metric": 0.6569208518844114, + "rescued": 8, + "harmed": 452, + "missed_rescue": 2619, + "avoided_harm": 8150, + "selected_both_correct": 4472, + "selected_both_wrong": 275 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.15066783881121856, + "metric": 0.6565805518532173, + "rescued": 9, + "harmed": 465, + "missed_rescue": 2618, + "avoided_harm": 8137, + "selected_both_correct": 4550, + "selected_both_wrong": 289 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.15452457249808582, + "metric": 0.6561268184782917, + "rescued": 12, + "harmed": 484, + "missed_rescue": 2615, + "avoided_harm": 8118, + "selected_both_correct": 4633, + "selected_both_wrong": 320 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.15806936448969175, + "metric": 0.6558715934548961, + "rescued": 16, + "harmed": 497, + "missed_rescue": 2611, + "avoided_harm": 8105, + "selected_both_correct": 4709, + "selected_both_wrong": 352 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.16201117318435754, + "metric": 0.6555596517596347, + "rescued": 17, + "harmed": 509, + "missed_rescue": 2610, + "avoided_harm": 8093, + "selected_both_correct": 4781, + "selected_both_wrong": 406 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.1657544735274934, + "metric": 0.6551626350565749, + "rescued": 19, + "harmed": 525, + "missed_rescue": 2608, + "avoided_harm": 8077, + "selected_both_correct": 4844, + "selected_both_wrong": 457 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.17012165726115192, + "metric": 0.6545954683379179, + "rescued": 23, + "harmed": 549, + "missed_rescue": 2604, + "avoided_harm": 8053, + "selected_both_correct": 4930, + "selected_both_wrong": 497 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.17536794940872868, + "metric": 0.6540850182911266, + "rescued": 33, + "harmed": 577, + "missed_rescue": 2594, + "avoided_harm": 8025, + "selected_both_correct": 5012, + "selected_both_wrong": 562 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.17965005813458867, + "metric": 0.6534044182287383, + "rescued": 39, + "harmed": 607, + "missed_rescue": 2588, + "avoided_harm": 7995, + "selected_both_correct": 5077, + "selected_both_wrong": 612 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.18401724186824717, + "metric": 0.653092476533477, + "rescued": 44, + "harmed": 623, + "missed_rescue": 2583, + "avoided_harm": 7979, + "selected_both_correct": 5133, + "selected_both_wrong": 689 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.18858293395343562, + "metric": 0.6527521765022829, + "rescued": 46, + "harmed": 637, + "missed_rescue": 2581, + "avoided_harm": 7965, + "selected_both_correct": 5199, + "selected_both_wrong": 768 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.19238295096843716, + "metric": 0.6523835181351558, + "rescued": 52, + "harmed": 656, + "missed_rescue": 2575, + "avoided_harm": 7946, + "selected_both_correct": 5260, + "selected_both_wrong": 816 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.19689192638175992, + "metric": 0.6517596347446332, + "rescued": 63, + "harmed": 689, + "missed_rescue": 2564, + "avoided_harm": 7913, + "selected_both_correct": 5317, + "selected_both_wrong": 874 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.20182627683407536, + "metric": 0.6511641096900433, + "rescued": 77, + "harmed": 724, + "missed_rescue": 2550, + "avoided_harm": 7878, + "selected_both_correct": 5395, + "selected_both_wrong": 921 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.20718600232538353, + "metric": 0.6505402262995208, + "rescued": 97, + "harmed": 766, + "missed_rescue": 2530, + "avoided_harm": 7836, + "selected_both_correct": 5459, + "selected_both_wrong": 984 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.21189348609023623, + "metric": 0.6500864929245952, + "rescued": 122, + "harmed": 807, + "missed_rescue": 2505, + "avoided_harm": 7795, + "selected_both_correct": 5501, + "selected_both_wrong": 1042 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.217139778237813, + "metric": 0.6500581345886624, + "rescued": 151, + "harmed": 837, + "missed_rescue": 2476, + "avoided_harm": 7765, + "selected_both_correct": 5555, + "selected_both_wrong": 1114 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.22198905368232993, + "metric": 0.6507670929869835, + "rescued": 203, + "harmed": 864, + "missed_rescue": 2424, + "avoided_harm": 7738, + "selected_both_correct": 5574, + "selected_both_wrong": 1187 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.22763236253296656, + "metric": 0.6515611263931033, + "rescued": 267, + "harmed": 900, + "missed_rescue": 2360, + "avoided_harm": 7702, + "selected_both_correct": 5592, + "selected_both_wrong": 1268 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.2345234381646485, + "metric": 0.6527521765022829, + "rescued": 344, + "harmed": 935, + "missed_rescue": 2283, + "avoided_harm": 7667, + "selected_both_correct": 5617, + "selected_both_wrong": 1374 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.24382497235062248, + "metric": 0.6537447182599325, + "rescued": 430, + "harmed": 986, + "missed_rescue": 2197, + "avoided_harm": 7616, + "selected_both_correct": 5653, + "selected_both_wrong": 1529 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.252644414825738, + "metric": 0.6545387516660522, + "rescued": 501, + "harmed": 1029, + "missed_rescue": 2126, + "avoided_harm": 7573, + "selected_both_correct": 5683, + "selected_both_wrong": 1696 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.26401610753480986, + "metric": 0.6538581516036639, + "rescued": 571, + "harmed": 1123, + "missed_rescue": 2056, + "avoided_harm": 7479, + "selected_both_correct": 5730, + "selected_both_wrong": 1886 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.2760116836344043, + "metric": 0.6537163599239997, + "rescued": 647, + "harmed": 1204, + "missed_rescue": 1980, + "avoided_harm": 7398, + "selected_both_correct": 5804, + "selected_both_wrong": 2078 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.2893401015228426, + "metric": 0.6533477015568726, + "rescued": 728, + "harmed": 1298, + "missed_rescue": 1899, + "avoided_harm": 7304, + "selected_both_correct": 5894, + "selected_both_wrong": 2283 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.3036043444970649, + "metric": 0.6525536681507529, + "rescued": 811, + "harmed": 1409, + "missed_rescue": 1816, + "avoided_harm": 7193, + "selected_both_correct": 5983, + "selected_both_wrong": 2503 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.6509088846666478, + "rescued": 876, + "harmed": 1532, + "missed_rescue": 1751, + "avoided_harm": 7070, + "selected_both_correct": 6081, + "selected_both_wrong": 2710 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.33462836400760004, + "metric": 0.6498312679011996, + "rescued": 971, + "harmed": 1665, + "missed_rescue": 1656, + "avoided_harm": 6937, + "selected_both_correct": 6236, + "selected_both_wrong": 2928 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.35637920766809406, + "metric": 0.6465984176048549, + "rescued": 1048, + "harmed": 1856, + "missed_rescue": 1579, + "avoided_harm": 6746, + "selected_both_correct": 6453, + "selected_both_wrong": 3210 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.37968975980489467, + "metric": 0.6442730340583614, + "rescued": 1153, + "harmed": 2043, + "missed_rescue": 1474, + "avoided_harm": 6559, + "selected_both_correct": 6702, + "selected_both_wrong": 3491 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.4046450954258004, + "metric": 0.6405580920511584, + "rescued": 1230, + "harmed": 2251, + "missed_rescue": 1397, + "avoided_harm": 6351, + "selected_both_correct": 6991, + "selected_both_wrong": 3797 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.43141536454640844, + "metric": 0.6357371749425744, + "rescued": 1303, + "harmed": 2494, + "missed_rescue": 1324, + "avoided_harm": 6108, + "selected_both_correct": 7308, + "selected_both_wrong": 4108 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.4610214672603012, + "metric": 0.630604316138729, + "rescued": 1384, + "harmed": 2756, + "missed_rescue": 1243, + "avoided_harm": 5846, + "selected_both_correct": 7664, + "selected_both_wrong": 4453 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.4916484700677764, + "metric": 0.6244221989053682, + "rescued": 1476, + "harmed": 3066, + "missed_rescue": 1151, + "avoided_harm": 5536, + "selected_both_correct": 8035, + "selected_both_wrong": 4760 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.6187788900547316, + "rescued": 1563, + "harmed": 3352, + "missed_rescue": 1064, + "avoided_harm": 5250, + "selected_both_correct": 8439, + "selected_both_wrong": 5016 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.5529308340186598, + "metric": 0.6118310977511839, + "rescued": 1647, + "harmed": 3681, + "missed_rescue": 980, + "avoided_harm": 4921, + "selected_both_correct": 8872, + "selected_both_wrong": 5298 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.5888041289737118, + "metric": 0.6045146470805093, + "rescued": 1741, + "harmed": 4033, + "missed_rescue": 886, + "avoided_harm": 4569, + "selected_both_correct": 9334, + "selected_both_wrong": 5655 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6175027649377535, + "metric": 0.5983608881830814, + "rescued": 1805, + "harmed": 4314, + "missed_rescue": 822, + "avoided_harm": 4288, + "selected_both_correct": 9696, + "selected_both_wrong": 5960 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6421461588633979, + "metric": 0.5925474293168477, + "rescued": 1860, + "harmed": 4574, + "missed_rescue": 767, + "avoided_harm": 4028, + "selected_both_correct": 10008, + "selected_both_wrong": 6202 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.665910444375124, + "metric": 0.5877548705441965, + "rescued": 1925, + "harmed": 4808, + "missed_rescue": 702, + "avoided_harm": 3794, + "selected_both_correct": 10303, + "selected_both_wrong": 6446 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.5829339534356124, + "rescued": 1961, + "harmed": 5014, + "missed_rescue": 666, + "avoided_harm": 3588, + "selected_both_correct": 10525, + "selected_both_wrong": 6615 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7029464311034228, + "metric": 0.5779996029832969, + "rescued": 2000, + "harmed": 5227, + "missed_rescue": 627, + "avoided_harm": 3375, + "selected_both_correct": 10747, + "selected_both_wrong": 6814 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7180614241556306, + "metric": 0.5745966026713553, + "rescued": 2042, + "harmed": 5389, + "missed_rescue": 585, + "avoided_harm": 3213, + "selected_both_correct": 10923, + "selected_both_wrong": 6967 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7352465757309361, + "metric": 0.5710234523438165, + "rescued": 2087, + "harmed": 5560, + "missed_rescue": 540, + "avoided_harm": 3042, + "selected_both_correct": 11139, + "selected_both_wrong": 7141 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7478093185491875, + "metric": 0.567762243711539, + "rescued": 2117, + "harmed": 5705, + "missed_rescue": 510, + "avoided_harm": 2897, + "selected_both_correct": 11290, + "selected_both_wrong": 7258 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.759804894648782, + "metric": 0.565153276805717, + "rescued": 2149, + "harmed": 5829, + "missed_rescue": 478, + "avoided_harm": 2773, + "selected_both_correct": 11421, + "selected_both_wrong": 7394 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.7729064458497575, + "metric": 0.5620622181890367, + "rescued": 2183, + "harmed": 5972, + "missed_rescue": 444, + "avoided_harm": 2630, + "selected_both_correct": 11577, + "selected_both_wrong": 7523 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7855259053398747, + "metric": 0.5588860845645578, + "rescued": 2221, + "harmed": 6122, + "missed_rescue": 406, + "avoided_harm": 2480, + "selected_both_correct": 11724, + "selected_both_wrong": 7633 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.7964722230099538, + "metric": 0.5562771176587358, + "rescued": 2244, + "harmed": 6237, + "missed_rescue": 383, + "avoided_harm": 2365, + "selected_both_correct": 11855, + "selected_both_wrong": 7750 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.8082125740861527, + "metric": 0.5536114340810481, + "rescued": 2274, + "harmed": 6361, + "missed_rescue": 353, + "avoided_harm": 2241, + "selected_both_correct": 12000, + "selected_both_wrong": 7865 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8197544168108215, + "metric": 0.5504920171284349, + "rescued": 2300, + "harmed": 6497, + "missed_rescue": 327, + "avoided_harm": 2105, + "selected_both_correct": 12140, + "selected_both_wrong": 7970 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8333380597226555, + "metric": 0.5469188668008961, + "rescued": 2329, + "harmed": 6652, + "missed_rescue": 298, + "avoided_harm": 1950, + "selected_both_correct": 12321, + "selected_both_wrong": 8084 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8426112355726966, + "metric": 0.5450188582933954, + "rescued": 2356, + "harmed": 6746, + "missed_rescue": 271, + "avoided_harm": 1856, + "selected_both_correct": 12437, + "selected_both_wrong": 8174 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8529903865241187, + "metric": 0.5423531747157077, + "rescued": 2381, + "harmed": 6865, + "missed_rescue": 246, + "avoided_harm": 1737, + "selected_both_correct": 12586, + "selected_both_wrong": 8247 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8635396874911381, + "metric": 0.5395456994583557, + "rescued": 2414, + "harmed": 6997, + "missed_rescue": 213, + "avoided_harm": 1605, + "selected_both_correct": 12707, + "selected_both_wrong": 8333 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8710262881774098, + "metric": 0.5378725576383178, + "rescued": 2435, + "harmed": 7077, + "missed_rescue": 192, + "avoided_harm": 1525, + "selected_both_correct": 12808, + "selected_both_wrong": 8395 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8809517057539064, + "metric": 0.535774040779287, + "rescued": 2462, + "harmed": 7178, + "missed_rescue": 165, + "avoided_harm": 1424, + "selected_both_correct": 12942, + "selected_both_wrong": 8483 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8913024983693957, + "metric": 0.5330516405297338, + "rescued": 2480, + "harmed": 7292, + "missed_rescue": 147, + "avoided_harm": 1310, + "selected_both_correct": 13082, + "selected_both_wrong": 8576 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.900915974250631, + "metric": 0.5304710319598446, + "rescued": 2498, + "harmed": 7401, + "missed_rescue": 129, + "avoided_harm": 1201, + "selected_both_correct": 13224, + "selected_both_wrong": 8646 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.910954825170859, + "metric": 0.5277486317102913, + "rescued": 2518, + "harmed": 7517, + "missed_rescue": 109, + "avoided_harm": 1085, + "selected_both_correct": 13379, + "selected_both_wrong": 8709 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.921901142840938, + "metric": 0.5245724980858123, + "rescued": 2542, + "harmed": 7653, + "missed_rescue": 85, + "avoided_harm": 949, + "selected_both_correct": 13541, + "selected_both_wrong": 8773 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9303235686129938, + "metric": 0.5217366644925275, + "rescued": 2553, + "harmed": 7764, + "missed_rescue": 74, + "avoided_harm": 838, + "selected_both_correct": 13673, + "selected_both_wrong": 8816 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9393698777755721, + "metric": 0.5187306808836457, + "rescued": 2567, + "harmed": 7884, + "missed_rescue": 60, + "avoided_harm": 718, + "selected_both_correct": 13819, + "selected_both_wrong": 8855 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9490117119927403, + "metric": 0.515526188923234, + "rescued": 2578, + "harmed": 8008, + "missed_rescue": 49, + "avoided_harm": 594, + "selected_both_correct": 13977, + "selected_both_wrong": 8902 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9587953378895726, + "metric": 0.5122366219550237, + "rescued": 2588, + "harmed": 8134, + "missed_rescue": 39, + "avoided_harm": 468, + "selected_both_correct": 14140, + "selected_both_wrong": 8948 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9673595553412926, + "metric": 0.5097694467288659, + "rescued": 2604, + "harmed": 8237, + "missed_rescue": 23, + "avoided_harm": 365, + "selected_both_correct": 14295, + "selected_both_wrong": 8976 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9746192893401016, + "metric": 0.5075574965261038, + "rescued": 2615, + "harmed": 8326, + "missed_rescue": 12, + "avoided_harm": 276, + "selected_both_correct": 14432, + "selected_both_wrong": 8995 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9830984317840229, + "metric": 0.5046365879250205, + "rescued": 2623, + "harmed": 8437, + "missed_rescue": 4, + "avoided_harm": 165, + "selected_both_correct": 14600, + "selected_both_wrong": 9007 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9908118991577575, + "metric": 0.5019425460114001, + "rescued": 2626, + "harmed": 8535, + "missed_rescue": 1, + "avoided_harm": 67, + "selected_both_correct": 14761, + "selected_both_wrong": 9017 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5000708958398321, + "rescued": 2627, + "harmed": 8602, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 15007, + "selected_both_wrong": 9027 + } + ] + }, + "learned": { + "validation_area_0_25": 0.75516761543327, + "validation_point": { + "metric": 0.7565394616628013, + "metric_std": 0.0005053251250895244, + "actual_fraction": 0.08195937873357229, + "actual_fraction_std": 0.04968689558802639, + "per_seed": [ + { + "seed": 2026, + "budget": 0.07, + "threshold": -0.03360140696167946, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7559210063953897, + "rescue": 97, + "harm": 17, + "missed_rescue": 1272 + }, + { + "seed": 2027, + "budget": 0.17, + "threshold": -0.018624423071742058, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7562021224260314, + "rescue": 157, + "harm": 73, + "missed_rescue": 1212 + }, + { + "seed": 2028, + "budget": 0.06, + "threshold": -0.0012479540891945362, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.756483238456673, + "rescue": 106, + "harm": 18, + "missed_rescue": 1263 + }, + { + "seed": 2029, + "budget": 0.05, + "threshold": -0.0008232841501012444, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7571157495256167, + "rescue": 116, + "harm": 19, + "missed_rescue": 1253 + }, + { + "seed": 2030, + "budget": 0.06, + "threshold": -0.0011288531823083758, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7569751915102959, + "rescue": 123, + "harm": 28, + "missed_rescue": 1246 + } + ], + "rescued": 119.8, + "harmed": 31.0, + "missed_rescue": 1249.2, + "avoided_harm": 2890.0 + }, + "test_point": { + "metric": 0.6802540906899583, + "metric_std": 0.0012240819123290802, + "actual_fraction": 0.1512350055298755, + "actual_fraction_std": 0.050698419588036114, + "per_seed": [ + { + "seed": 2026, + "budget": 0.07, + "threshold": -0.03360140696167946, + "operator": "ge", + "actual_fraction": 0.13972152114113945, + "metric": 0.6787851288886368, + "rescue": 400, + "harm": 73, + "missed_rescue": 2227 + }, + { + "seed": 2027, + "budget": 0.17, + "threshold": -0.018624423071742058, + "operator": "ge", + "actual_fraction": 0.2406771970620764, + "metric": 0.6794090122791595, + "rescue": 709, + "harm": 360, + "missed_rescue": 1918 + }, + { + "seed": 2028, + "budget": 0.06, + "threshold": -0.0012479540891945362, + "operator": "ge", + "actual_fraction": 0.1307319286504268, + "metric": 0.681167229106996, + "rescue": 509, + "harm": 98, + "missed_rescue": 2118 + }, + { + "seed": 2029, + "budget": 0.05, + "threshold": -0.0008232841501012444, + "operator": "ge", + "actual_fraction": 0.1162124606528089, + "metric": 0.6801463290134135, + "rescue": 490, + "harm": 115, + "missed_rescue": 2137 + }, + { + "seed": 2030, + "budget": 0.06, + "threshold": -0.0011288531823083758, + "operator": "ge", + "actual_fraction": 0.12883192014292602, + "metric": 0.6817627541615858, + "rescue": 602, + "harm": 170, + "missed_rescue": 2025 + } + ], + "rescued": 542.0, + "harmed": 163.2, + "missed_rescue": 2085.0, + "avoided_harm": 8438.8 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.7502986857825568, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.7526741162414787, + "metric_std": 0.0002998204934668702, + "actual_fraction": 0.009979619087778481, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.754079696394687, + "metric_std": 0.0007972873877794269, + "actual_fraction": 0.019959238175556962, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.7551760489141893, + "metric_std": 0.0011839457862809614, + "actual_fraction": 0.029938857263335443, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.7559631737999859, + "metric_std": 0.001043116629901192, + "actual_fraction": 0.03998875535877434, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.756314568838288, + "metric_std": 0.0006967907545404636, + "actual_fraction": 0.04996837444655281, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.7563989036474805, + "metric_std": 0.000639114065325138, + "actual_fraction": 0.05994799353433129, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.7563989036474805, + "metric_std": 0.0006094450316586567, + "actual_fraction": 0.06999789162977019, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.7562724014336919, + "metric_std": 0.0007062952857629153, + "actual_fraction": 0.07997751071754867, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.7562442898306274, + "metric_std": 0.0006932374893897312, + "actual_fraction": 0.08995712980532715, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.7561740108229673, + "metric_std": 0.0006218798736560268, + "actual_fraction": 0.09993674889310562, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.7561599550214352, + "metric_std": 0.0005743992337255639, + "actual_fraction": 0.10998664698854452, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.12, + "metric": 0.7560896760137747, + "metric_std": 0.0006317297084614319, + "actual_fraction": 0.11996626607632302, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.13, + "metric": 0.7560896760137747, + "metric_std": 0.0005657350986716735, + "actual_fraction": 0.1299458851641015, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.7560896760137747, + "metric_std": 0.000563548252069119, + "actual_fraction": 0.13999578325954037, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.7560053412045822, + "metric_std": 0.000522149852301827, + "actual_fraction": 0.14997540234731885, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.7559631737999859, + "metric_std": 0.0005082489207045987, + "actual_fraction": 0.15995502143509735, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.7558647831892614, + "metric_std": 0.0005384491196676729, + "actual_fraction": 0.16993464052287582, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.7555977229601518, + "metric_std": 0.0005343053709676994, + "actual_fraction": 0.1799845386183147, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.7554290533417669, + "metric_std": 0.0005774009672202942, + "actual_fraction": 0.18996415770609318, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.7552322721203177, + "metric_std": 0.0006729925993323155, + "actual_fraction": 0.19994377679387168, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.7551057699065289, + "metric_std": 0.0007756204743170253, + "actual_fraction": 0.2099936748893106, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.7547824864712911, + "metric_std": 0.0007863715359167649, + "actual_fraction": 0.21997329397708903, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.23, + "metric": 0.7544029798299249, + "metric_std": 0.0009117330090845152, + "actual_fraction": 0.22995291306486756, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.24, + "metric": 0.7541218637992831, + "metric_std": 0.0009527925970950756, + "actual_fraction": 0.2399184763511139, + "actual_fraction_std": 3.142972770397948e-05 + }, + { + "budget": 0.25, + "metric": 0.7539250825778341, + "metric_std": 0.0011251227953493355, + "actual_fraction": 0.24998243024808492, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.7536017991425961, + "metric_std": 0.0011675625641883601, + "actual_fraction": 0.25996204933586337, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.753250404104294, + "metric_std": 0.0012483182000373625, + "actual_fraction": 0.26994166842364187, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.7527865626537353, + "metric_std": 0.001442518362462185, + "actual_fraction": 0.27999156651908075, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.752393000210837, + "metric_std": 0.0015827606449093142, + "actual_fraction": 0.28997118560685925, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.7518167123480216, + "metric_std": 0.0015656610887642903, + "actual_fraction": 0.2999508046946377, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.7511560896760138, + "metric_std": 0.0015788550952444478, + "actual_fraction": 0.3099163679808841, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.32, + "metric": 0.7507203598285193, + "metric_std": 0.0015373281927210272, + "actual_fraction": 0.3199803218778551, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.7502002951718321, + "metric_std": 0.0017880449159481387, + "actual_fraction": 0.3299599409656336, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.7495115608967602, + "metric_std": 0.001748803824864237, + "actual_fraction": 0.339939560053412, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.749132054255394, + "metric_std": 0.0015561683107440615, + "actual_fraction": 0.34998945814885096, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.7483870967741935, + "metric_std": 0.0015223153353362696, + "actual_fraction": 0.3599690772366294, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.7478670321175065, + "metric_std": 0.0015999861565215569, + "actual_fraction": 0.3699486963244079, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.7471080188347741, + "metric_std": 0.001788044915948184, + "actual_fraction": 0.3799985944198468, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.7466301215826834, + "metric_std": 0.0018483512624486132, + "actual_fraction": 0.3899782135076253, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.7458429966968866, + "metric_std": 0.0017672061389894655, + "actual_fraction": 0.39995783259540374, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.7454072668493921, + "metric_std": 0.0018004324124242241, + "actual_fraction": 0.4099374516831823, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.42, + "metric": 0.7445779745589992, + "metric_std": 0.0020875349738663294, + "actual_fraction": 0.4199873497786212, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.7441422447115047, + "metric_std": 0.002023864596378755, + "actual_fraction": 0.42995291306486755, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.44, + "metric": 0.7435800126502213, + "metric_std": 0.0019518088654493831, + "actual_fraction": 0.43994658795417807, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.45, + "metric": 0.7428631667720852, + "metric_std": 0.0021144523045770504, + "actual_fraction": 0.44999648604961695, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.46, + "metric": 0.7421322650924169, + "metric_std": 0.0019023425256831145, + "actual_fraction": 0.4599761051373955, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.47, + "metric": 0.7410499683744466, + "metric_std": 0.0022808759330676946, + "actual_fraction": 0.4699557242251739, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.48, + "metric": 0.7402768992901819, + "metric_std": 0.002351250346223846, + "actual_fraction": 0.4799353433129524, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.7397427788319628, + "metric_std": 0.002555006831217186, + "actual_fraction": 0.4899852414083913, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.7388994307400379, + "metric_std": 0.0028492916030263718, + "actual_fraction": 0.49996486049616984, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.51, + "metric": 0.7377749666174713, + "metric_std": 0.002858809746585389, + "actual_fraction": 0.5099444795839483, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.7367629489071615, + "metric_std": 0.0027555286712770367, + "actual_fraction": 0.5199803218778551, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.53, + "metric": 0.7362147726474102, + "metric_std": 0.002914927216960097, + "actual_fraction": 0.5299599409656336, + "actual_fraction_std": 3.1429727703942244e-05 + }, + { + "budget": 0.54, + "metric": 0.7354838709677419, + "metric_std": 0.0026461702975562554, + "actual_fraction": 0.5399536158549442, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.7344577974558999, + "metric_std": 0.0028083477391225636, + "actual_fraction": 0.5499332349427226, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.7335722819593787, + "metric_std": 0.0028968253349532687, + "actual_fraction": 0.5599831330381615, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.7326586548597934, + "metric_std": 0.002717808354440857, + "actual_fraction": 0.5699627521259399, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.7315904139433551, + "metric_std": 0.0028126533388477784, + "actual_fraction": 0.5799283154121864, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.59, + "metric": 0.7303816150115959, + "metric_std": 0.0031483112902165565, + "actual_fraction": 0.5899922693091574, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.7291447044767728, + "metric_std": 0.0034483295875598, + "actual_fraction": 0.5999578325954037, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.61, + "metric": 0.7284278585986366, + "metric_std": 0.003493651398393919, + "actual_fraction": 0.6099515074847143, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.7276126221097758, + "metric_std": 0.0034925909272520597, + "actual_fraction": 0.6199311265724928, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.7263897673764846, + "metric_std": 0.0033282137627859785, + "actual_fraction": 0.6299810246679317, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.7254761402768992, + "metric_std": 0.0036691430808927098, + "actual_fraction": 0.6399606437557102, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.724070560123691, + "metric_std": 0.0027549908847179865, + "actual_fraction": 0.6499402628434886, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.7231288214210415, + "metric_std": 0.0029959801660216015, + "actual_fraction": 0.6599901609389276, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.7214280694356596, + "metric_std": 0.0030904389534852283, + "actual_fraction": 0.669969780026706, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.7205425539391385, + "metric_std": 0.0029183141084957688, + "actual_fraction": 0.6799493991144845, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.7188839693583526, + "metric_std": 0.0034195630404674866, + "actual_fraction": 0.6899992972099234, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.7173799985944198, + "metric_std": 0.0037160253748051734, + "actual_fraction": 0.6999789162977019, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.7159463068381473, + "metric_std": 0.004073273309323374, + "actual_fraction": 0.7099585353854804, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.714512615081875, + "metric_std": 0.0034975370416262157, + "actual_fraction": 0.7199381544732588, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.7127275282873006, + "metric_std": 0.0034474700836611984, + "actual_fraction": 0.7299880525686977, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.7110267763019186, + "metric_std": 0.0033348851705954636, + "actual_fraction": 0.7399676716564763, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.7096071403471783, + "metric_std": 0.003018317699275064, + "actual_fraction": 0.7499472907442547, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.7077798861480076, + "metric_std": 0.0030035541002682124, + "actual_fraction": 0.7599971888396936, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.705826129735048, + "metric_std": 0.003007416031438144, + "actual_fraction": 0.769976807927472, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.7038442617190246, + "metric_std": 0.0028434785753241845, + "actual_fraction": 0.7799564270152506, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.702045119122918, + "metric_std": 0.002788491778396698, + "actual_fraction": 0.7899219903014969, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.8, + "metric": 0.7002740881298756, + "metric_std": 0.0026938045396672863, + "actual_fraction": 0.7999859441984679, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.698109494693935, + "metric_std": 0.002947364505538372, + "actual_fraction": 0.8099655632862464, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.6962541288917001, + "metric_std": 0.0025158510534357575, + "actual_fraction": 0.8199451823740249, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.6939068100358423, + "metric_std": 0.00263738304723494, + "actual_fraction": 0.8299950804694639, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.6915454353784525, + "metric_std": 0.002365073957476131, + "actual_fraction": 0.8399746995572424, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.6897744043854102, + "metric_std": 0.0024694847279795123, + "actual_fraction": 0.8499543186450207, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.6878628153770469, + "metric_std": 0.0027702768905602643, + "actual_fraction": 0.8599339377327994, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.87, + "metric": 0.6854030501089324, + "metric_std": 0.0029402338083401878, + "actual_fraction": 0.869969780026706, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.88, + "metric": 0.6829573406423501, + "metric_std": 0.0028012158072167627, + "actual_fraction": 0.8799493991144844, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.89, + "metric": 0.6801461803359338, + "metric_std": 0.002367370044997327, + "actual_fraction": 0.889943074003795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.6781924239229742, + "metric_std": 0.0017442790984595953, + "actual_fraction": 0.899978916297702, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.91, + "metric": 0.6755218216318786, + "metric_std": 0.0019840599077227124, + "actual_fraction": 0.9099725911870123, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.6724154894932883, + "metric_std": 0.0016522294271167843, + "actual_fraction": 0.919952210274791, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.93, + "metric": 0.669815166209853, + "metric_std": 0.002390209611339768, + "actual_fraction": 0.9299177735610373, + "actual_fraction_std": 3.1429727703942244e-05 + }, + { + "budget": 0.94, + "metric": 0.6666666666666667, + "metric_std": 0.0021264481369607574, + "actual_fraction": 0.9399676716564762, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.95, + "metric": 0.6637290041464614, + "metric_std": 0.0024311864475704374, + "actual_fraction": 0.9499613465457868, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.6605523930002108, + "metric_std": 0.0025315079983593823, + "actual_fraction": 0.9599409656335652, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.6567432707850165, + "metric_std": 0.001781680269416085, + "actual_fraction": 0.9699908637290042, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.98, + "metric": 0.652203246890154, + "metric_std": 0.0011972209962767403, + "actual_fraction": 0.9799704828167826, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.6472837163539251, + "metric_std": 0.000696436243830878, + "actual_fraction": 0.9899501019045612, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 1.0, + "metric": 0.6412256658935975, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.6695119530385957, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.674355556815926, + "metric_std": 0.0007538731722042529, + "actual_fraction": 0.01988486515611264, + "actual_fraction_std": 0.0036456954327566773 + }, + { + "budget": 0.02, + "metric": 0.677083628732666, + "metric_std": 0.0008277525746151176, + "actual_fraction": 0.044380795734906275, + "actual_fraction_std": 0.006013037248424046 + }, + { + "budget": 0.03, + "metric": 0.6788305022261294, + "metric_std": 0.0010592891890852782, + "actual_fraction": 0.0693077730198792, + "actual_fraction_std": 0.008699563768545507 + }, + { + "budget": 0.04, + "metric": 0.6798797606556447, + "metric_std": 0.0014442719804607256, + "actual_fraction": 0.09778521396364462, + "actual_fraction_std": 0.006737608807395872 + }, + { + "budget": 0.05, + "metric": 0.6802143890196524, + "metric_std": 0.0013264306598878798, + "actual_fraction": 0.11746589910104074, + "actual_fraction_std": 0.0030091521868304224 + }, + { + "budget": 0.06, + "metric": 0.6802881206930778, + "metric_std": 0.0012152796039597306, + "actual_fraction": 0.13140685704562854, + "actual_fraction_std": 0.004663433283700637 + }, + { + "budget": 0.07, + "metric": 0.6803221506961972, + "metric_std": 0.0011763412386233013, + "actual_fraction": 0.1422397413719763, + "actual_fraction_std": 0.0055957758980413365 + }, + { + "budget": 0.08, + "metric": 0.6803391656977569, + "metric_std": 0.0011194181466024139, + "actual_fraction": 0.15099112384085303, + "actual_fraction_std": 0.007129474029792084 + }, + { + "budget": 0.09, + "metric": 0.6803051356946375, + "metric_std": 0.001154292265991228, + "actual_fraction": 0.15834727618183364, + "actual_fraction_std": 0.008187223971605222 + }, + { + "budget": 0.1, + "metric": 0.6803391656977569, + "metric_std": 0.0010994845383243818, + "actual_fraction": 0.1653631284916201, + "actual_fraction_std": 0.009536086596367807 + }, + { + "budget": 0.11, + "metric": 0.6803051356946375, + "metric_std": 0.0011308897618267826, + "actual_fraction": 0.17206703910614524, + "actual_fraction_std": 0.011065722830507543 + }, + { + "budget": 0.12, + "metric": 0.6802484190227718, + "metric_std": 0.0011088073085827447, + "actual_fraction": 0.17780109463176702, + "actual_fraction_std": 0.012403026484438001 + }, + { + "budget": 0.13, + "metric": 0.6802370756883986, + "metric_std": 0.0011069563008359582, + "actual_fraction": 0.183716643507359, + "actual_fraction_std": 0.014412265999519596 + }, + { + "budget": 0.14, + "metric": 0.6802654340243315, + "metric_std": 0.0010743281506441784, + "actual_fraction": 0.18935995235799563, + "actual_fraction_std": 0.015926792311943862 + }, + { + "budget": 0.15, + "metric": 0.6802881206930778, + "metric_std": 0.0010384728142019113, + "actual_fraction": 0.19479340952272922, + "actual_fraction_std": 0.01714850638537999 + }, + { + "budget": 0.16, + "metric": 0.6802484190227718, + "metric_std": 0.0011089886136362995, + "actual_fraction": 0.1999432833281343, + "actual_fraction_std": 0.01891439994924246 + }, + { + "budget": 0.17, + "metric": 0.6801917023509061, + "metric_std": 0.0010786610402635068, + "actual_fraction": 0.20534838215693502, + "actual_fraction_std": 0.020649789779907116 + }, + { + "budget": 0.18, + "metric": 0.6800555823384283, + "metric_std": 0.0010426076321689867, + "actual_fraction": 0.21027706094206394, + "actual_fraction_std": 0.021861527433657865 + }, + { + "budget": 0.19, + "metric": 0.6798684173212716, + "metric_std": 0.0011622421029173072, + "actual_fraction": 0.2154836514193347, + "actual_fraction_std": 0.023240660669042255 + }, + { + "budget": 0.2, + "metric": 0.6797549839775401, + "metric_std": 0.00120970824422957, + "actual_fraction": 0.22028188185917247, + "actual_fraction_std": 0.024156961534962593 + }, + { + "budget": 0.21, + "metric": 0.679584833961943, + "metric_std": 0.0011613768629310526, + "actual_fraction": 0.22551683067237613, + "actual_fraction_std": 0.02587853615038678 + }, + { + "budget": 0.22, + "metric": 0.679454385616652, + "metric_std": 0.0012993256015320668, + "actual_fraction": 0.23028670277628108, + "actual_fraction_std": 0.026888462833130886 + }, + { + "budget": 0.23, + "metric": 0.6792445339307489, + "metric_std": 0.001366124386655156, + "actual_fraction": 0.23611150497688796, + "actual_fraction_std": 0.028601480115681595 + }, + { + "budget": 0.24, + "metric": 0.6789949805745399, + "metric_std": 0.001609644727803233, + "actual_fraction": 0.2415506338088081, + "actual_fraction_std": 0.0295631768954572 + }, + { + "budget": 0.25, + "metric": 0.6788078155573831, + "metric_std": 0.0016346557579893933, + "actual_fraction": 0.24712021098601938, + "actual_fraction_std": 0.030439430139812203 + }, + { + "budget": 0.26, + "metric": 0.6785015455293083, + "metric_std": 0.00173913145296339, + "actual_fraction": 0.25279754983977537, + "actual_fraction_std": 0.03152078215963538 + }, + { + "budget": 0.27, + "metric": 0.6782519921730993, + "metric_std": 0.0017094455708145922, + "actual_fraction": 0.2581005586592179, + "actual_fraction_std": 0.03175704169215061 + }, + { + "budget": 0.28, + "metric": 0.6780307971528231, + "metric_std": 0.0017007437655551807, + "actual_fraction": 0.2642883475597652, + "actual_fraction_std": 0.03294786262042393 + }, + { + "budget": 0.29, + "metric": 0.677832288801293, + "metric_std": 0.0016832768660069142, + "actual_fraction": 0.2703003147775288, + "actual_fraction_std": 0.03342615232925088 + }, + { + "budget": 0.3, + "metric": 0.6775146754388451, + "metric_std": 0.0017255994985924217, + "actual_fraction": 0.27680571704052404, + "actual_fraction_std": 0.03408282088692543 + }, + { + "budget": 0.31, + "metric": 0.6770155687264271, + "metric_std": 0.001973157445183299, + "actual_fraction": 0.2840711227065196, + "actual_fraction_std": 0.035185675447351995 + }, + { + "budget": 0.32, + "metric": 0.6765958653546209, + "metric_std": 0.0020850995615312516, + "actual_fraction": 0.29096787000538804, + "actual_fraction_std": 0.035604072026188247 + }, + { + "budget": 0.33, + "metric": 0.67606840030627, + "metric_std": 0.002267195308687241, + "actual_fraction": 0.29832969401355525, + "actual_fraction_std": 0.03597170798389997 + }, + { + "budget": 0.34, + "metric": 0.6756600402688371, + "metric_std": 0.0022035175200024386, + "actual_fraction": 0.30655361143408105, + "actual_fraction_std": 0.036385264561378125 + }, + { + "budget": 0.35, + "metric": 0.6751325752204862, + "metric_std": 0.0023441042336844772, + "actual_fraction": 0.3140572271219125, + "actual_fraction_std": 0.03658387238067934 + }, + { + "budget": 0.36, + "metric": 0.6746277968408814, + "metric_std": 0.002417504477686349, + "actual_fraction": 0.32198054618155003, + "actual_fraction_std": 0.03661382959297894 + }, + { + "budget": 0.37, + "metric": 0.6740946601253437, + "metric_std": 0.0025604766483161945, + "actual_fraction": 0.33028386694268785, + "actual_fraction_std": 0.03680710125647762 + }, + { + "budget": 0.38, + "metric": 0.6734764484020077, + "metric_std": 0.0028127266637735533, + "actual_fraction": 0.3393018177693333, + "actual_fraction_std": 0.036270674172948054 + }, + { + "budget": 0.39, + "metric": 0.6725803249865299, + "metric_std": 0.0030500309851385814, + "actual_fraction": 0.3478887218897995, + "actual_fraction_std": 0.03590572201123908 + }, + { + "budget": 0.4, + "metric": 0.6718316649179027, + "metric_std": 0.0030657446630015686, + "actual_fraction": 0.35660040268837023, + "actual_fraction_std": 0.03554938488735922 + }, + { + "budget": 0.41, + "metric": 0.6711113631852083, + "metric_std": 0.0031465359641322183, + "actual_fraction": 0.3657317868587472, + "actual_fraction_std": 0.034900047668308384 + }, + { + "budget": 0.42, + "metric": 0.6701471797634915, + "metric_std": 0.003285812235229236, + "actual_fraction": 0.3744661543260641, + "actual_fraction_std": 0.03455301423038343 + }, + { + "budget": 0.43, + "metric": 0.6692567280152001, + "metric_std": 0.003329116456574453, + "actual_fraction": 0.38352947849020225, + "actual_fraction_std": 0.033372114546538954 + }, + { + "budget": 0.44, + "metric": 0.6683606045997221, + "metric_std": 0.003343554840049297, + "actual_fraction": 0.3925247426481014, + "actual_fraction_std": 0.033643509156036026 + }, + { + "budget": 0.45, + "metric": 0.6672716444999007, + "metric_std": 0.003200725365887715, + "actual_fraction": 0.40237075688398605, + "actual_fraction_std": 0.03290054279897225 + }, + { + "budget": 0.46, + "metric": 0.6661146243938405, + "metric_std": 0.003476493730172364, + "actual_fraction": 0.4113546777075121, + "actual_fraction_std": 0.033234206949167754 + }, + { + "budget": 0.47, + "metric": 0.6650653659643252, + "metric_std": 0.003540217676430841, + "actual_fraction": 0.4206732268950458, + "actual_fraction_std": 0.032631052534423624 + }, + { + "budget": 0.48, + "metric": 0.6638799875223321, + "metric_std": 0.003909369151174779, + "actual_fraction": 0.43054759946686333, + "actual_fraction_std": 0.03243481247622407 + }, + { + "budget": 0.49, + "metric": 0.6625358023991152, + "metric_std": 0.0039418086946658056, + "actual_fraction": 0.4394351019482176, + "actual_fraction_std": 0.03204247517290313 + }, + { + "budget": 0.5, + "metric": 0.6612993789524431, + "metric_std": 0.004102001679832678, + "actual_fraction": 0.4491449961716246, + "actual_fraction_std": 0.031248566235310894 + }, + { + "budget": 0.51, + "metric": 0.6598757904886141, + "metric_std": 0.00429941740200273, + "actual_fraction": 0.4592292204293452, + "actual_fraction_std": 0.031672455355214464 + }, + { + "budget": 0.52, + "metric": 0.6586280237075688, + "metric_std": 0.004444186457386014, + "actual_fraction": 0.46937583302611807, + "actual_fraction_std": 0.031759447320887056 + }, + { + "budget": 0.53, + "metric": 0.6570059268922099, + "metric_std": 0.004362070715726049, + "actual_fraction": 0.47919348892606983, + "actual_fraction_std": 0.031341615681442746 + }, + { + "budget": 0.54, + "metric": 0.655293083401866, + "metric_std": 0.0045312414789695716, + "actual_fraction": 0.4892720415166038, + "actual_fraction_std": 0.03093808282360971 + }, + { + "budget": 0.55, + "metric": 0.6537957632646116, + "metric_std": 0.004705364954493196, + "actual_fraction": 0.49921447409466013, + "actual_fraction_std": 0.030792591911341517 + }, + { + "budget": 0.56, + "metric": 0.6524742648101409, + "metric_std": 0.004721318021541213, + "actual_fraction": 0.5092022800102091, + "actual_fraction_std": 0.030614672326524203 + }, + { + "budget": 0.57, + "metric": 0.6505345546323342, + "metric_std": 0.004578857088916336, + "actual_fraction": 0.5194850126194595, + "actual_fraction_std": 0.02989951570544599 + }, + { + "budget": 0.58, + "metric": 0.6484587244420498, + "metric_std": 0.004306015115969245, + "actual_fraction": 0.53054476363327, + "actual_fraction_std": 0.028176861705339614 + }, + { + "budget": 0.59, + "metric": 0.6464055809205116, + "metric_std": 0.0042221743464271495, + "actual_fraction": 0.5408842129143862, + "actual_fraction_std": 0.027415965947455163 + }, + { + "budget": 0.6, + "metric": 0.6445339307489437, + "metric_std": 0.004279121607548603, + "actual_fraction": 0.5514618722173383, + "actual_fraction_std": 0.026762558087465298 + }, + { + "budget": 0.61, + "metric": 0.6428267589257862, + "metric_std": 0.0043445211287076535, + "actual_fraction": 0.5614326631313274, + "actual_fraction_std": 0.025721052900268063 + }, + { + "budget": 0.62, + "metric": 0.640813317074554, + "metric_std": 0.004438691092917886, + "actual_fraction": 0.5718174857499362, + "actual_fraction_std": 0.025228880044587666 + }, + { + "budget": 0.63, + "metric": 0.6389473385701727, + "metric_std": 0.0044069802957014525, + "actual_fraction": 0.5819017100076567, + "actual_fraction_std": 0.02420726257866112 + }, + { + "budget": 0.64, + "metric": 0.6365368800158807, + "metric_std": 0.004220459773262121, + "actual_fraction": 0.5923262342965714, + "actual_fraction_std": 0.023370954413372294 + }, + { + "budget": 0.65, + "metric": 0.6344610498255963, + "metric_std": 0.003932105954795623, + "actual_fraction": 0.6026373252417547, + "actual_fraction_std": 0.02145833699651952 + }, + { + "budget": 0.66, + "metric": 0.6321299946119161, + "metric_std": 0.004134310822690512, + "actual_fraction": 0.6139069279414684, + "actual_fraction_std": 0.02083016198099768 + }, + { + "budget": 0.67, + "metric": 0.6298556560701019, + "metric_std": 0.003916459706178751, + "actual_fraction": 0.6240535405382412, + "actual_fraction_std": 0.019654024954561186 + }, + { + "budget": 0.68, + "metric": 0.6276720642032726, + "metric_std": 0.003981524503718168, + "actual_fraction": 0.6351700082239174, + "actual_fraction_std": 0.018304948890778536 + }, + { + "budget": 0.69, + "metric": 0.6251254856365028, + "metric_std": 0.004099785722382148, + "actual_fraction": 0.6465246859314295, + "actual_fraction_std": 0.017412856522390312 + }, + { + "budget": 0.7, + "metric": 0.6230666704477781, + "metric_std": 0.004059275641701459, + "actual_fraction": 0.6564671185094859, + "actual_fraction_std": 0.016454959552190642 + }, + { + "budget": 0.71, + "metric": 0.6200890451748291, + "metric_std": 0.004191924191285504, + "actual_fraction": 0.6685307546153191, + "actual_fraction_std": 0.015971189721643162 + }, + { + "budget": 0.72, + "metric": 0.6171227632362533, + "metric_std": 0.0038890022634119493, + "actual_fraction": 0.680191702350906, + "actual_fraction_std": 0.014523048349964044 + }, + { + "budget": 0.73, + "metric": 0.614332302980461, + "metric_std": 0.0039223484592232, + "actual_fraction": 0.6918583217536795, + "actual_fraction_std": 0.014342324771198522 + }, + { + "budget": 0.74, + "metric": 0.6114851260528031, + "metric_std": 0.003689985102414692, + "actual_fraction": 0.7036610611689307, + "actual_fraction_std": 0.01302465058884312 + }, + { + "budget": 0.75, + "metric": 0.6083146640955108, + "metric_std": 0.0040298282980880265, + "actual_fraction": 0.7157927572810028, + "actual_fraction_std": 0.012197760588232388 + }, + { + "budget": 0.76, + "metric": 0.6053710688256813, + "metric_std": 0.0034793262909278188, + "actual_fraction": 0.7270963899838357, + "actual_fraction_std": 0.011055216386120085 + }, + { + "budget": 0.77, + "metric": 0.6022119502027621, + "metric_std": 0.0034598564482223398, + "actual_fraction": 0.7380994243257806, + "actual_fraction_std": 0.011061888589487569 + }, + { + "budget": 0.78, + "metric": 0.5989053682329921, + "metric_std": 0.0031223416462126756, + "actual_fraction": 0.7492442503473896, + "actual_fraction_std": 0.009928832143037301 + }, + { + "budget": 0.79, + "metric": 0.5951280378867368, + "metric_std": 0.0033320139655529614, + "actual_fraction": 0.76089952641579, + "actual_fraction_std": 0.009375842272464604 + }, + { + "budget": 0.8, + "metric": 0.5918044409154071, + "metric_std": 0.003070724630598477, + "actual_fraction": 0.7725434591498171, + "actual_fraction_std": 0.007882734967263542 + }, + { + "budget": 0.81, + "metric": 0.588458157275331, + "metric_std": 0.0029656772967492527, + "actual_fraction": 0.7839718685307546, + "actual_fraction_std": 0.00807765682364763 + }, + { + "budget": 0.82, + "metric": 0.5852536653149193, + "metric_std": 0.003075631167353602, + "actual_fraction": 0.7946686328446246, + "actual_fraction_std": 0.006376988379364331 + }, + { + "budget": 0.83, + "metric": 0.5820151433513882, + "metric_std": 0.0032228596901171065, + "actual_fraction": 0.8049683804554348, + "actual_fraction_std": 0.006274713593988744 + }, + { + "budget": 0.84, + "metric": 0.5784930380285285, + "metric_std": 0.0028375770949825635, + "actual_fraction": 0.8165045515129172, + "actual_fraction_std": 0.004977417122101177 + }, + { + "budget": 0.85, + "metric": 0.5752148143946914, + "metric_std": 0.002804064456192344, + "actual_fraction": 0.8270481808127499, + "actual_fraction_std": 0.004217266906179628 + }, + { + "budget": 0.86, + "metric": 0.5717891274140033, + "metric_std": 0.0032437151724607795, + "actual_fraction": 0.8379831551484559, + "actual_fraction_std": 0.0042208884818482955 + }, + { + "budget": 0.87, + "metric": 0.5680571704052406, + "metric_std": 0.003913481182889722, + "actual_fraction": 0.8493605195247144, + "actual_fraction_std": 0.004704792370234814 + }, + { + "budget": 0.88, + "metric": 0.5641664067152539, + "metric_std": 0.0036851327477977467, + "actual_fraction": 0.8604656438760173, + "actual_fraction_std": 0.004978467208409149 + }, + { + "budget": 0.89, + "metric": 0.5600147463346851, + "metric_std": 0.003244855422980753, + "actual_fraction": 0.8719337549272609, + "actual_fraction_std": 0.004617796051234398 + }, + { + "budget": 0.9, + "metric": 0.5561523409806312, + "metric_std": 0.0032020818503248095, + "actual_fraction": 0.8831920142926014, + "actual_fraction_std": 0.0042671051080321095 + }, + { + "budget": 0.91, + "metric": 0.5513370955392338, + "metric_std": 0.0032283944693626125, + "actual_fraction": 0.895079828715651, + "actual_fraction_std": 0.00406852688181964 + }, + { + "budget": 0.92, + "metric": 0.5468224484587244, + "metric_std": 0.0034111789533246505, + "actual_fraction": 0.9058106230326406, + "actual_fraction_std": 0.0043601160524001014 + }, + { + "budget": 0.93, + "metric": 0.5420922780251256, + "metric_std": 0.0033446850978015277, + "actual_fraction": 0.9169611207214361, + "actual_fraction_std": 0.004471427090260003 + }, + { + "budget": 0.94, + "metric": 0.537078524232198, + "metric_std": 0.002716263797437144, + "actual_fraction": 0.9286107251226496, + "actual_fraction_std": 0.0036535289061868804 + }, + { + "budget": 0.95, + "metric": 0.531860590420554, + "metric_std": 0.0028426314462505716, + "actual_fraction": 0.9400448061707738, + "actual_fraction_std": 0.0037682669469528674 + }, + { + "budget": 0.96, + "metric": 0.5263250432464622, + "metric_std": 0.002628421550979396, + "actual_fraction": 0.951977993931316, + "actual_fraction_std": 0.00249952631530437 + }, + { + "budget": 0.97, + "metric": 0.520795167739557, + "metric_std": 0.0027886065496349113, + "actual_fraction": 0.9633723733091342, + "actual_fraction_std": 0.0031409081463861285 + }, + { + "budget": 0.98, + "metric": 0.5143464821484275, + "metric_std": 0.002058237003719859, + "actual_fraction": 0.9754530244165271, + "actual_fraction_std": 0.00250250063185031 + }, + { + "budget": 0.99, + "metric": 0.507041374812126, + "metric_std": 0.0012271985913663679, + "actual_fraction": 0.9881575589144429, + "actual_fraction_std": 0.0017727476042924111 + }, + { + "budget": 1.0, + "metric": 0.5000708958398321, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 7755, + "rescuable": 1369, + "harmful": 2921, + "both_wrong": 2184, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 15007, + "rescuable": 2627, + "harmful": 8602, + "both_wrong": 9027, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.7502986857825568, + "test": 0.6695119530385957 + }, + "large": { + "validation": 0.6412256658935975, + "test": 0.5000708958398321 + }, + "learned_minus_confidence_test": 0.06147520063522671, + "learned_minus_small_test": 0.010742137651362671, + "learned_minus_large_test": 0.18018319485012624, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_instancevg_MiniCPM" + }, + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "expert_display": "InstanceVG", + "run": "five_seed_instancevg_Qwen3.8_four_difference", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detections_above_07", + "max_score", + "mean_score", + "query_count", + "detections_above_07_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 71, + "validation_area_0_25": 0.7477616136060159, + "validation_point": { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8201560193970061, + "rescued": 1660, + "harmed": 666, + "missed_rescue": 304, + "avoided_harm": 505, + "selected_both_correct": 6505, + "selected_both_wrong": 1271 + }, + "test_point": { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.7243286163967898, + "rescued": 3794, + "harmed": 1861, + "missed_rescue": 1629, + "avoided_harm": 910, + "selected_both_correct": 13678, + "selected_both_wrong": 4782 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1964, + "avoided_harm": 1171, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7502284067748963, + "rescued": 0, + "harmed": 1, + "missed_rescue": 1964, + "avoided_harm": 1170, + "selected_both_correct": 141, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7501581277672359, + "rescued": 0, + "harmed": 2, + "missed_rescue": 1964, + "avoided_harm": 1169, + "selected_both_correct": 282, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7498770117365943, + "rescued": 0, + "harmed": 6, + "missed_rescue": 1964, + "avoided_harm": 1165, + "selected_both_correct": 420, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7495958957059526, + "rescued": 0, + "harmed": 10, + "missed_rescue": 1964, + "avoided_harm": 1161, + "selected_both_correct": 559, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7494553376906318, + "rescued": 0, + "harmed": 12, + "missed_rescue": 1964, + "avoided_harm": 1159, + "selected_both_correct": 699, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7493850586829714, + "rescued": 0, + "harmed": 13, + "missed_rescue": 1964, + "avoided_harm": 1158, + "selected_both_correct": 840, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.749314779675311, + "rescued": 0, + "harmed": 14, + "missed_rescue": 1964, + "avoided_harm": 1157, + "selected_both_correct": 982, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7491742216599901, + "rescued": 0, + "harmed": 16, + "missed_rescue": 1964, + "avoided_harm": 1155, + "selected_both_correct": 1122, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.748963384637009, + "rescued": 0, + "harmed": 19, + "missed_rescue": 1964, + "avoided_harm": 1152, + "selected_both_correct": 1261, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.7486119895987069, + "rescued": 0, + "harmed": 24, + "missed_rescue": 1964, + "avoided_harm": 1147, + "selected_both_correct": 1398, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7483308735680653, + "rescued": 0, + "harmed": 28, + "missed_rescue": 1964, + "avoided_harm": 1143, + "selected_both_correct": 1537, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7481903155527444, + "rescued": 0, + "harmed": 30, + "missed_rescue": 1964, + "avoided_harm": 1141, + "selected_both_correct": 1677, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.7477686415067819, + "rescued": 0, + "harmed": 36, + "missed_rescue": 1964, + "avoided_harm": 1135, + "selected_both_correct": 1813, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7476280834914611, + "rescued": 0, + "harmed": 38, + "missed_rescue": 1964, + "avoided_harm": 1133, + "selected_both_correct": 1954, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7474875254761403, + "rescued": 0, + "harmed": 40, + "missed_rescue": 1964, + "avoided_harm": 1131, + "selected_both_correct": 2094, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.7472766884531591, + "rescued": 0, + "harmed": 43, + "missed_rescue": 1964, + "avoided_harm": 1128, + "selected_both_correct": 2233, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7471361304378382, + "rescued": 0, + "harmed": 45, + "missed_rescue": 1964, + "avoided_harm": 1126, + "selected_both_correct": 2373, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7470658514301778, + "rescued": 0, + "harmed": 46, + "missed_rescue": 1964, + "avoided_harm": 1125, + "selected_both_correct": 2515, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7465036193688945, + "rescued": 0, + "harmed": 54, + "missed_rescue": 1964, + "avoided_harm": 1117, + "selected_both_correct": 2649, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7462225033382529, + "rescued": 0, + "harmed": 58, + "missed_rescue": 1964, + "avoided_harm": 1113, + "selected_both_correct": 2787, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7456602712769695, + "rescued": 0, + "harmed": 66, + "missed_rescue": 1964, + "avoided_harm": 1105, + "selected_both_correct": 2922, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7453088762386675, + "rescued": 0, + "harmed": 71, + "missed_rescue": 1964, + "avoided_harm": 1100, + "selected_both_correct": 3059, + "selected_both_wrong": 0 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7446763651697238, + "rescued": 1, + "harmed": 81, + "missed_rescue": 1963, + "avoided_harm": 1090, + "selected_both_correct": 3190, + "selected_both_wrong": 0 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7442546911237613, + "rescued": 1, + "harmed": 87, + "missed_rescue": 1963, + "avoided_harm": 1084, + "selected_both_correct": 3326, + "selected_both_wrong": 0 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7437627380701385, + "rescued": 1, + "harmed": 94, + "missed_rescue": 1963, + "avoided_harm": 1077, + "selected_both_correct": 3462, + "selected_both_wrong": 0 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7431302270011948, + "rescued": 1, + "harmed": 103, + "missed_rescue": 1963, + "avoided_harm": 1068, + "selected_both_correct": 3595, + "selected_both_wrong": 0 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7422868789092698, + "rescued": 1, + "harmed": 115, + "missed_rescue": 1963, + "avoided_harm": 1056, + "selected_both_correct": 3725, + "selected_both_wrong": 0 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7419354838709677, + "rescued": 1, + "harmed": 120, + "missed_rescue": 1963, + "avoided_harm": 1051, + "selected_both_correct": 3862, + "selected_both_wrong": 1 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7411624147867032, + "rescued": 1, + "harmed": 131, + "missed_rescue": 1963, + "avoided_harm": 1040, + "selected_both_correct": 3993, + "selected_both_wrong": 1 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7404596247100991, + "rescued": 2, + "harmed": 142, + "missed_rescue": 1962, + "avoided_harm": 1029, + "selected_both_correct": 4123, + "selected_both_wrong": 1 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7394757186028533, + "rescued": 2, + "harmed": 156, + "missed_rescue": 1962, + "avoided_harm": 1015, + "selected_both_correct": 4251, + "selected_both_wrong": 1 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7388432075339096, + "rescued": 2, + "harmed": 165, + "missed_rescue": 1962, + "avoided_harm": 1006, + "selected_both_correct": 4385, + "selected_both_wrong": 1 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7377890224190035, + "rescued": 2, + "harmed": 180, + "missed_rescue": 1962, + "avoided_harm": 991, + "selected_both_correct": 4512, + "selected_both_wrong": 1 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.7361023262351536, + "rescued": 2, + "harmed": 204, + "missed_rescue": 1962, + "avoided_harm": 967, + "selected_both_correct": 4629, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7348373040972661, + "rescued": 2, + "harmed": 222, + "missed_rescue": 1962, + "avoided_harm": 949, + "selected_both_correct": 4754, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7337128399746996, + "rescued": 2, + "harmed": 238, + "missed_rescue": 1962, + "avoided_harm": 933, + "selected_both_correct": 4880, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.732588375852133, + "rescued": 2, + "harmed": 254, + "missed_rescue": 1962, + "avoided_harm": 917, + "selected_both_correct": 5006, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7311827956989247, + "rescued": 2, + "harmed": 274, + "missed_rescue": 1962, + "avoided_harm": 897, + "selected_both_correct": 5128, + "selected_both_wrong": 3 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7297069365380561, + "rescued": 2, + "harmed": 295, + "missed_rescue": 1962, + "avoided_harm": 876, + "selected_both_correct": 5248, + "selected_both_wrong": 4 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.7282310773771874, + "rescued": 3, + "harmed": 317, + "missed_rescue": 1961, + "avoided_harm": 854, + "selected_both_correct": 5365, + "selected_both_wrong": 6 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7266849392086584, + "rescued": 3, + "harmed": 339, + "missed_rescue": 1961, + "avoided_harm": 832, + "selected_both_correct": 5484, + "selected_both_wrong": 7 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.7249279640171481, + "rescued": 7, + "harmed": 368, + "missed_rescue": 1957, + "avoided_harm": 803, + "selected_both_correct": 5590, + "selected_both_wrong": 11 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7233115468409586, + "rescued": 12, + "harmed": 396, + "missed_rescue": 1952, + "avoided_harm": 775, + "selected_both_correct": 5696, + "selected_both_wrong": 14 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7212734556188066, + "rescued": 15, + "harmed": 428, + "missed_rescue": 1949, + "avoided_harm": 743, + "selected_both_correct": 5798, + "selected_both_wrong": 19 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7190245273736735, + "rescued": 16, + "harmed": 461, + "missed_rescue": 1948, + "avoided_harm": 710, + "selected_both_correct": 5901, + "selected_both_wrong": 25 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7173378311898236, + "rescued": 21, + "harmed": 490, + "missed_rescue": 1943, + "avoided_harm": 681, + "selected_both_correct": 6008, + "selected_both_wrong": 26 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7155808559983133, + "rescued": 27, + "harmed": 521, + "missed_rescue": 1937, + "avoided_harm": 650, + "selected_both_correct": 6107, + "selected_both_wrong": 32 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7206409445498629, + "rescued": 105, + "harmed": 527, + "missed_rescue": 1859, + "avoided_harm": 644, + "selected_both_correct": 6125, + "selected_both_wrong": 72 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7280202403542062, + "rescued": 210, + "harmed": 527, + "missed_rescue": 1754, + "avoided_harm": 644, + "selected_both_correct": 6125, + "selected_both_wrong": 110 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7346264670742849, + "rescued": 305, + "harmed": 528, + "missed_rescue": 1659, + "avoided_harm": 643, + "selected_both_correct": 6125, + "selected_both_wrong": 156 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.7414435308173448, + "rescued": 403, + "harmed": 529, + "missed_rescue": 1561, + "avoided_harm": 642, + "selected_both_correct": 6125, + "selected_both_wrong": 199 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.7478389205144423, + "rescued": 495, + "harmed": 530, + "missed_rescue": 1469, + "avoided_harm": 641, + "selected_both_correct": 6125, + "selected_both_wrong": 249 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.754093752196219, + "rescued": 585, + "harmed": 531, + "missed_rescue": 1379, + "avoided_harm": 640, + "selected_both_correct": 6125, + "selected_both_wrong": 300 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7602783048703352, + "rescued": 673, + "harmed": 531, + "missed_rescue": 1291, + "avoided_harm": 640, + "selected_both_correct": 6127, + "selected_both_wrong": 352 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.7662520205214702, + "rescued": 759, + "harmed": 532, + "missed_rescue": 1205, + "avoided_harm": 639, + "selected_both_correct": 6129, + "selected_both_wrong": 405 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7718743411343032, + "rescued": 844, + "harmed": 537, + "missed_rescue": 1120, + "avoided_harm": 634, + "selected_both_correct": 6130, + "selected_both_wrong": 457 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7774263827394757, + "rescued": 924, + "harmed": 538, + "missed_rescue": 1040, + "avoided_harm": 633, + "selected_both_correct": 6133, + "selected_both_wrong": 515 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.7818539602220816, + "rescued": 991, + "harmed": 542, + "missed_rescue": 973, + "avoided_harm": 629, + "selected_both_correct": 6135, + "selected_both_wrong": 583 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7864220957200084, + "rescued": 1061, + "harmed": 547, + "missed_rescue": 903, + "avoided_harm": 624, + "selected_both_correct": 6139, + "selected_both_wrong": 648 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7909902312179352, + "rescued": 1130, + "harmed": 551, + "missed_rescue": 834, + "avoided_harm": 620, + "selected_both_correct": 6146, + "selected_both_wrong": 710 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7954880877082016, + "rescued": 1200, + "harmed": 557, + "missed_rescue": 764, + "avoided_harm": 614, + "selected_both_correct": 6151, + "selected_both_wrong": 771 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7996345491601659, + "rescued": 1263, + "harmed": 561, + "missed_rescue": 701, + "avoided_harm": 610, + "selected_both_correct": 6159, + "selected_both_wrong": 838 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.8028673835125448, + "rescued": 1317, + "harmed": 569, + "missed_rescue": 647, + "avoided_harm": 602, + "selected_both_correct": 6171, + "selected_both_wrong": 907 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.8064516129032258, + "rescued": 1380, + "harmed": 581, + "missed_rescue": 584, + "avoided_harm": 590, + "selected_both_correct": 6186, + "selected_both_wrong": 959 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.809895284278586, + "rescued": 1434, + "harmed": 586, + "missed_rescue": 530, + "avoided_harm": 585, + "selected_both_correct": 6211, + "selected_both_wrong": 1017 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.8123550495467003, + "rescued": 1479, + "harmed": 596, + "missed_rescue": 485, + "avoided_harm": 575, + "selected_both_correct": 6244, + "selected_both_wrong": 1072 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.8149553728301356, + "rescued": 1525, + "harmed": 605, + "missed_rescue": 439, + "avoided_harm": 566, + "selected_both_correct": 6278, + "selected_both_wrong": 1125 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.8169231850446271, + "rescued": 1569, + "harmed": 621, + "missed_rescue": 395, + "avoided_harm": 550, + "selected_both_correct": 6318, + "selected_both_wrong": 1167 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.818609881228477, + "rescued": 1605, + "harmed": 633, + "missed_rescue": 359, + "avoided_harm": 538, + "selected_both_correct": 6376, + "selected_both_wrong": 1204 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.6999086372900415, + "metric": 0.8196640663433832, + "rescued": 1636, + "harmed": 649, + "missed_rescue": 328, + "avoided_harm": 522, + "selected_both_correct": 6439, + "selected_both_wrong": 1235 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8201560193970061, + "rescued": 1660, + "harmed": 666, + "missed_rescue": 304, + "avoided_harm": 505, + "selected_both_correct": 6505, + "selected_both_wrong": 1271 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.8200154613816852, + "rescued": 1686, + "harmed": 694, + "missed_rescue": 278, + "avoided_harm": 477, + "selected_both_correct": 6565, + "selected_both_wrong": 1299 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.8199451823740249, + "rescued": 1703, + "harmed": 712, + "missed_rescue": 261, + "avoided_harm": 459, + "selected_both_correct": 6646, + "selected_both_wrong": 1326 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8200154613816852, + "rescued": 1722, + "harmed": 730, + "missed_rescue": 242, + "avoided_harm": 441, + "selected_both_correct": 6725, + "selected_both_wrong": 1352 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.8198749033663645, + "rescued": 1742, + "harmed": 752, + "missed_rescue": 222, + "avoided_harm": 419, + "selected_both_correct": 6808, + "selected_both_wrong": 1369 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8200154613816852, + "rescued": 1763, + "harmed": 771, + "missed_rescue": 201, + "avoided_harm": 400, + "selected_both_correct": 6891, + "selected_both_wrong": 1389 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8195235083280624, + "rescued": 1784, + "harmed": 799, + "missed_rescue": 180, + "avoided_harm": 372, + "selected_both_correct": 6966, + "selected_both_wrong": 1407 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.8191018342820999, + "rescued": 1800, + "harmed": 821, + "missed_rescue": 164, + "avoided_harm": 350, + "selected_both_correct": 7050, + "selected_both_wrong": 1427 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.8192423922974208, + "rescued": 1818, + "harmed": 837, + "missed_rescue": 146, + "avoided_harm": 334, + "selected_both_correct": 7143, + "selected_both_wrong": 1442 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.8190315552744395, + "rescued": 1834, + "harmed": 856, + "missed_rescue": 130, + "avoided_harm": 315, + "selected_both_correct": 7244, + "selected_both_wrong": 1449 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.8190315552744395, + "rescued": 1849, + "harmed": 871, + "missed_rescue": 115, + "avoided_harm": 300, + "selected_both_correct": 7347, + "selected_both_wrong": 1458 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.8187504392437979, + "rescued": 1859, + "harmed": 885, + "missed_rescue": 105, + "avoided_harm": 286, + "selected_both_correct": 7444, + "selected_both_wrong": 1479 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.818609881228477, + "rescued": 1874, + "harmed": 902, + "missed_rescue": 90, + "avoided_harm": 269, + "selected_both_correct": 7541, + "selected_both_wrong": 1493 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.8176962541288917, + "rescued": 1880, + "harmed": 921, + "missed_rescue": 84, + "avoided_harm": 250, + "selected_both_correct": 7643, + "selected_both_wrong": 1508 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.8175556961135709, + "rescued": 1889, + "harmed": 932, + "missed_rescue": 75, + "avoided_harm": 239, + "selected_both_correct": 7754, + "selected_both_wrong": 1519 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.8165717900063251, + "rescued": 1895, + "harmed": 952, + "missed_rescue": 69, + "avoided_harm": 219, + "selected_both_correct": 7864, + "selected_both_wrong": 1525 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.8160798369527023, + "rescued": 1907, + "harmed": 971, + "missed_rescue": 57, + "avoided_harm": 200, + "selected_both_correct": 7969, + "selected_both_wrong": 1532 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.8155878838990793, + "rescued": 1914, + "harmed": 985, + "missed_rescue": 50, + "avoided_harm": 186, + "selected_both_correct": 8082, + "selected_both_wrong": 1540 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.8148148148148148, + "rescued": 1919, + "harmed": 1001, + "missed_rescue": 45, + "avoided_harm": 170, + "selected_both_correct": 8192, + "selected_both_wrong": 1551 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.8144634197765127, + "rescued": 1927, + "harmed": 1014, + "missed_rescue": 37, + "avoided_harm": 157, + "selected_both_correct": 8308, + "selected_both_wrong": 1557 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.8129172816079837, + "rescued": 1932, + "harmed": 1041, + "missed_rescue": 32, + "avoided_harm": 130, + "selected_both_correct": 8410, + "selected_both_wrong": 1565 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.8123550495467003, + "rescued": 1937, + "harmed": 1054, + "missed_rescue": 27, + "avoided_harm": 117, + "selected_both_correct": 8530, + "selected_both_wrong": 1569 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.8113711434394546, + "rescued": 1942, + "harmed": 1073, + "missed_rescue": 22, + "avoided_harm": 98, + "selected_both_correct": 8647, + "selected_both_wrong": 1570 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.8102466793168881, + "rescued": 1945, + "harmed": 1092, + "missed_rescue": 19, + "avoided_harm": 79, + "selected_both_correct": 8764, + "selected_both_wrong": 1574 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.8087708201560194, + "rescued": 1948, + "harmed": 1116, + "missed_rescue": 16, + "avoided_harm": 55, + "selected_both_correct": 8876, + "selected_both_wrong": 1577 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8079977510717549, + "rescued": 1951, + "harmed": 1130, + "missed_rescue": 13, + "avoided_harm": 41, + "selected_both_correct": 8998, + "selected_both_wrong": 1580 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8075057980181319, + "rescued": 1956, + "harmed": 1142, + "missed_rescue": 8, + "avoided_harm": 29, + "selected_both_correct": 9120, + "selected_both_wrong": 1584 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8070138449645091, + "rescued": 1961, + "harmed": 1154, + "missed_rescue": 3, + "avoided_harm": 17, + "selected_both_correct": 9242, + "selected_both_wrong": 1587 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.806311054887905, + "rescued": 1962, + "harmed": 1165, + "missed_rescue": 2, + "avoided_harm": 6, + "selected_both_correct": 9370, + "selected_both_wrong": 1589 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 1964, + "harmed": 1171, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9505, + "selected_both_wrong": 1589 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 5423, + "avoided_harm": 2771, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.002467175226157729, + "metric": 0.6686895612965431, + "rescued": 0, + "harmed": 29, + "missed_rescue": 5423, + "avoided_harm": 2742, + "selected_both_correct": 58, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.004849275444516916, + "metric": 0.6680373195700876, + "rescued": 0, + "harmed": 52, + "missed_rescue": 5423, + "avoided_harm": 2719, + "selected_both_correct": 119, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.007600034030003119, + "metric": 0.6675835861951621, + "rescued": 0, + "harmed": 68, + "missed_rescue": 5423, + "avoided_harm": 2703, + "selected_both_correct": 199, + "selected_both_wrong": 1 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.009811984232765221, + "metric": 0.6671014944843037, + "rescued": 0, + "harmed": 85, + "missed_rescue": 5423, + "avoided_harm": 2686, + "selected_both_correct": 260, + "selected_both_wrong": 1 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.012534384482318578, + "metric": 0.6666194027734452, + "rescued": 0, + "harmed": 102, + "missed_rescue": 5423, + "avoided_harm": 2669, + "selected_both_correct": 338, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.01607917647392451, + "metric": 0.6662223860703854, + "rescued": 0, + "harmed": 116, + "missed_rescue": 5423, + "avoided_harm": 2655, + "selected_both_correct": 449, + "selected_both_wrong": 2 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.019709043473328986, + "metric": 0.6658253693673255, + "rescued": 0, + "harmed": 130, + "missed_rescue": 5423, + "avoided_harm": 2641, + "selected_both_correct": 562, + "selected_both_wrong": 3 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.02353741882426339, + "metric": 0.6655985026798628, + "rescued": 0, + "harmed": 138, + "missed_rescue": 5423, + "avoided_harm": 2633, + "selected_both_correct": 689, + "selected_both_wrong": 3 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.028868785979638715, + "metric": 0.6651164109690043, + "rescued": 0, + "harmed": 155, + "missed_rescue": 5423, + "avoided_harm": 2616, + "selected_both_correct": 859, + "selected_both_wrong": 4 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.0345971698380739, + "metric": 0.6647761109378102, + "rescued": 0, + "harmed": 167, + "missed_rescue": 5423, + "avoided_harm": 2604, + "selected_both_correct": 1045, + "selected_both_wrong": 8 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.03981510364971783, + "metric": 0.664435810906616, + "rescued": 0, + "harmed": 179, + "missed_rescue": 5423, + "avoided_harm": 2592, + "selected_both_correct": 1215, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.045855429203414344, + "metric": 0.664067152539489, + "rescued": 0, + "harmed": 192, + "missed_rescue": 5423, + "avoided_harm": 2579, + "selected_both_correct": 1414, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.05317187987408899, + "metric": 0.6637552108442276, + "rescued": 1, + "harmed": 204, + "missed_rescue": 5422, + "avoided_harm": 2567, + "selected_both_correct": 1658, + "selected_both_wrong": 12 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.05960922213084536, + "metric": 0.6636134191645634, + "rescued": 1, + "harmed": 209, + "missed_rescue": 5422, + "avoided_harm": 2562, + "selected_both_correct": 1878, + "selected_both_wrong": 14 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.0675211978561098, + "metric": 0.6632164024615036, + "rescued": 2, + "harmed": 224, + "missed_rescue": 5421, + "avoided_harm": 2547, + "selected_both_correct": 2128, + "selected_both_wrong": 27 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.07551824858917279, + "metric": 0.6626208774069138, + "rescued": 2, + "harmed": 245, + "missed_rescue": 5421, + "avoided_harm": 2526, + "selected_both_correct": 2386, + "selected_both_wrong": 30 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.08212574086152624, + "metric": 0.6622522190397867, + "rescued": 3, + "harmed": 259, + "missed_rescue": 5420, + "avoided_harm": 2512, + "selected_both_correct": 2599, + "selected_both_wrong": 35 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.08856308311828262, + "metric": 0.6619969940163911, + "rescued": 3, + "harmed": 268, + "missed_rescue": 5420, + "avoided_harm": 2503, + "selected_both_correct": 2812, + "selected_both_wrong": 40 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.09480191702350906, + "metric": 0.6615149023055327, + "rescued": 3, + "harmed": 285, + "missed_rescue": 5420, + "avoided_harm": 2486, + "selected_both_correct": 3005, + "selected_both_wrong": 50 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.10007656750701868, + "metric": 0.6612596772821371, + "rescued": 3, + "harmed": 294, + "missed_rescue": 5420, + "avoided_harm": 2477, + "selected_both_correct": 3174, + "selected_both_wrong": 58 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.10554972634205825, + "metric": 0.6608343022431443, + "rescued": 3, + "harmed": 309, + "missed_rescue": 5420, + "avoided_harm": 2462, + "selected_both_correct": 3340, + "selected_both_wrong": 70 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.11005870175538099, + "metric": 0.6605507188838159, + "rescued": 3, + "harmed": 319, + "missed_rescue": 5420, + "avoided_harm": 2452, + "selected_both_correct": 3485, + "selected_both_wrong": 74 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6600969855088903, + "rescued": 5, + "harmed": 337, + "missed_rescue": 5418, + "avoided_harm": 2434, + "selected_both_correct": 3630, + "selected_both_wrong": 93 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.12029606102713893, + "metric": 0.6598417604854947, + "rescued": 8, + "harmed": 349, + "missed_rescue": 5415, + "avoided_harm": 2422, + "selected_both_correct": 3769, + "selected_both_wrong": 116 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.12477667810452883, + "metric": 0.6594731021183677, + "rescued": 8, + "harmed": 362, + "missed_rescue": 5415, + "avoided_harm": 2409, + "selected_both_correct": 3894, + "selected_both_wrong": 136 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.12914386183818732, + "metric": 0.6590193687434421, + "rescued": 8, + "harmed": 378, + "missed_rescue": 5415, + "avoided_harm": 2393, + "selected_both_correct": 4012, + "selected_both_wrong": 156 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.1322916371267334, + "metric": 0.658679068712248, + "rescued": 10, + "harmed": 392, + "missed_rescue": 5413, + "avoided_harm": 2379, + "selected_both_correct": 4092, + "selected_both_wrong": 171 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.13626180415733205, + "metric": 0.6582820520091881, + "rescued": 12, + "harmed": 408, + "missed_rescue": 5411, + "avoided_harm": 2363, + "selected_both_correct": 4192, + "selected_both_wrong": 193 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.14023197118793068, + "metric": 0.657743243626464, + "rescued": 13, + "harmed": 428, + "missed_rescue": 5410, + "avoided_harm": 2343, + "selected_both_correct": 4285, + "selected_both_wrong": 219 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.14326631313274538, + "metric": 0.657374585259337, + "rescued": 16, + "harmed": 444, + "missed_rescue": 5407, + "avoided_harm": 2327, + "selected_both_correct": 4359, + "selected_both_wrong": 233 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.14766185520233674, + "metric": 0.6569775685562771, + "rescued": 18, + "harmed": 460, + "missed_rescue": 5405, + "avoided_harm": 2311, + "selected_both_correct": 4464, + "selected_both_wrong": 265 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.15066783881121856, + "metric": 0.6567507018688143, + "rescued": 20, + "harmed": 470, + "missed_rescue": 5403, + "avoided_harm": 2301, + "selected_both_correct": 4545, + "selected_both_wrong": 278 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.15452457249808582, + "metric": 0.6565238351813516, + "rescued": 27, + "harmed": 485, + "missed_rescue": 5396, + "avoided_harm": 2286, + "selected_both_correct": 4632, + "selected_both_wrong": 305 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.15806936448969175, + "metric": 0.6564104018376201, + "rescued": 39, + "harmed": 501, + "missed_rescue": 5384, + "avoided_harm": 2270, + "selected_both_correct": 4705, + "selected_both_wrong": 329 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.16201117318435754, + "metric": 0.6562686101579559, + "rescued": 46, + "harmed": 513, + "missed_rescue": 5377, + "avoided_harm": 2258, + "selected_both_correct": 4777, + "selected_both_wrong": 377 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.1657544735274934, + "metric": 0.6558999517908289, + "rescued": 48, + "harmed": 528, + "missed_rescue": 5375, + "avoided_harm": 2243, + "selected_both_correct": 4841, + "selected_both_wrong": 428 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.17012165726115192, + "metric": 0.6558715934548961, + "rescued": 61, + "harmed": 542, + "missed_rescue": 5362, + "avoided_harm": 2229, + "selected_both_correct": 4937, + "selected_both_wrong": 459 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.17536794940872868, + "metric": 0.6555596517596347, + "rescued": 79, + "harmed": 571, + "missed_rescue": 5344, + "avoided_harm": 2200, + "selected_both_correct": 5018, + "selected_both_wrong": 516 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.17965005813458867, + "metric": 0.6553044267362391, + "rescued": 92, + "harmed": 593, + "missed_rescue": 5331, + "avoided_harm": 2178, + "selected_both_correct": 5091, + "selected_both_wrong": 559 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.18401724186824717, + "metric": 0.6552193517284406, + "rescued": 107, + "harmed": 611, + "missed_rescue": 5316, + "avoided_harm": 2160, + "selected_both_correct": 5145, + "selected_both_wrong": 626 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.18858293395343562, + "metric": 0.6552760684003063, + "rescued": 127, + "harmed": 629, + "missed_rescue": 5296, + "avoided_harm": 2142, + "selected_both_correct": 5207, + "selected_both_wrong": 687 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.19238295096843716, + "metric": 0.6554745767518362, + "rescued": 144, + "harmed": 639, + "missed_rescue": 5279, + "avoided_harm": 2132, + "selected_both_correct": 5277, + "selected_both_wrong": 724 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.19689192638175992, + "metric": 0.655502935087769, + "rescued": 168, + "harmed": 662, + "missed_rescue": 5255, + "avoided_harm": 2109, + "selected_both_correct": 5344, + "selected_both_wrong": 769 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.20182627683407536, + "metric": 0.6555880100955676, + "rescued": 199, + "harmed": 690, + "missed_rescue": 5224, + "avoided_harm": 2081, + "selected_both_correct": 5429, + "selected_both_wrong": 799 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.20718600232538353, + "metric": 0.6559283101267618, + "rescued": 240, + "harmed": 719, + "missed_rescue": 5183, + "avoided_harm": 2052, + "selected_both_correct": 5506, + "selected_both_wrong": 841 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.21189348609023623, + "metric": 0.656070101806426, + "rescued": 280, + "harmed": 754, + "missed_rescue": 5143, + "avoided_harm": 2017, + "selected_both_correct": 5554, + "selected_both_wrong": 884 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.217139778237813, + "metric": 0.6567790602047472, + "rescued": 329, + "harmed": 778, + "missed_rescue": 5094, + "avoided_harm": 1993, + "selected_both_correct": 5614, + "selected_both_wrong": 936 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.22198905368232993, + "metric": 0.6584238436888523, + "rescued": 399, + "harmed": 790, + "missed_rescue": 5024, + "avoided_harm": 1981, + "selected_both_correct": 5648, + "selected_both_wrong": 991 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.22763236253296656, + "metric": 0.6604372855400845, + "rescued": 483, + "harmed": 803, + "missed_rescue": 4940, + "avoided_harm": 1968, + "selected_both_correct": 5689, + "selected_both_wrong": 1052 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.2345234381646485, + "metric": 0.662762669086578, + "rescued": 579, + "harmed": 817, + "missed_rescue": 4844, + "avoided_harm": 1954, + "selected_both_correct": 5735, + "selected_both_wrong": 1139 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.24382497235062248, + "metric": 0.6654567110001985, + "rescued": 702, + "harmed": 845, + "missed_rescue": 4721, + "avoided_harm": 1926, + "selected_both_correct": 5794, + "selected_both_wrong": 1257 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.252644414825738, + "metric": 0.6678955278904234, + "rescued": 813, + "harmed": 870, + "missed_rescue": 4610, + "avoided_harm": 1901, + "selected_both_correct": 5842, + "selected_both_wrong": 1384 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.26401610753480986, + "metric": 0.6703627031165811, + "rescued": 937, + "harmed": 907, + "missed_rescue": 4486, + "avoided_harm": 1864, + "selected_both_correct": 5946, + "selected_both_wrong": 1520 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.2760116836344043, + "metric": 0.6731134617020673, + "rescued": 1067, + "harmed": 940, + "missed_rescue": 4356, + "avoided_harm": 1831, + "selected_both_correct": 6068, + "selected_both_wrong": 1658 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.2893401015228426, + "metric": 0.6758358619516207, + "rescued": 1197, + "harmed": 974, + "missed_rescue": 4226, + "avoided_harm": 1797, + "selected_both_correct": 6218, + "selected_both_wrong": 1814 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.3036043444970649, + "metric": 0.679295578935428, + "rescued": 1352, + "harmed": 1007, + "missed_rescue": 4071, + "avoided_harm": 1764, + "selected_both_correct": 6385, + "selected_both_wrong": 1962 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.6822448458724442, + "rescued": 1489, + "harmed": 1040, + "missed_rescue": 3934, + "avoided_harm": 1731, + "selected_both_correct": 6573, + "selected_both_wrong": 2097 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.33462836400760004, + "metric": 0.6853926211609903, + "rescued": 1645, + "harmed": 1085, + "missed_rescue": 3778, + "avoided_harm": 1686, + "selected_both_correct": 6816, + "selected_both_wrong": 2254 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.35637920766809406, + "metric": 0.6889090548166633, + "rescued": 1818, + "harmed": 1134, + "missed_rescue": 3605, + "avoided_harm": 1637, + "selected_both_correct": 7175, + "selected_both_wrong": 2440 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.37968975980489467, + "metric": 0.6922553384567394, + "rescued": 1978, + "harmed": 1176, + "missed_rescue": 3445, + "avoided_harm": 1595, + "selected_both_correct": 7569, + "selected_both_wrong": 2666 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.4046450954258004, + "metric": 0.6949493803703599, + "rescued": 2126, + "harmed": 1229, + "missed_rescue": 3297, + "avoided_harm": 1542, + "selected_both_correct": 8013, + "selected_both_wrong": 2901 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.43141536454640844, + "metric": 0.6983807390182344, + "rescued": 2296, + "harmed": 1278, + "missed_rescue": 3127, + "avoided_harm": 1493, + "selected_both_correct": 8524, + "selected_both_wrong": 3115 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.4610214672603012, + "metric": 0.7015568726427134, + "rescued": 2479, + "harmed": 1349, + "missed_rescue": 2944, + "avoided_harm": 1422, + "selected_both_correct": 9071, + "selected_both_wrong": 3358 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.4916484700677764, + "metric": 0.7053285313217821, + "rescued": 2674, + "harmed": 1411, + "missed_rescue": 2749, + "avoided_harm": 1360, + "selected_both_correct": 9690, + "selected_both_wrong": 3562 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.7086180982899923, + "rescued": 2849, + "harmed": 1470, + "missed_rescue": 2574, + "avoided_harm": 1301, + "selected_both_correct": 10321, + "selected_both_wrong": 3730 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.5529308340186598, + "metric": 0.71148229021921, + "rescued": 3014, + "harmed": 1534, + "missed_rescue": 2409, + "avoided_harm": 1237, + "selected_both_correct": 11019, + "selected_both_wrong": 3931 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.5888041289737118, + "metric": 0.7155091739216742, + "rescued": 3227, + "harmed": 1605, + "missed_rescue": 2196, + "avoided_harm": 1166, + "selected_both_correct": 11762, + "selected_both_wrong": 4169 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6175027649377535, + "metric": 0.7182882908430933, + "rescued": 3404, + "harmed": 1684, + "missed_rescue": 2019, + "avoided_harm": 1087, + "selected_both_correct": 12326, + "selected_both_wrong": 4361 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6421461588633979, + "metric": 0.7211241244363781, + "rescued": 3553, + "harmed": 1733, + "missed_rescue": 1870, + "avoided_harm": 1038, + "selected_both_correct": 12849, + "selected_both_wrong": 4509 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.665910444375124, + "metric": 0.7233077163032073, + "rescued": 3694, + "harmed": 1797, + "missed_rescue": 1729, + "avoided_harm": 974, + "selected_both_correct": 13314, + "selected_both_wrong": 4677 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.7243286163967898, + "rescued": 3794, + "harmed": 1861, + "missed_rescue": 1629, + "avoided_harm": 910, + "selected_both_correct": 13678, + "selected_both_wrong": 4782 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7029464311034228, + "metric": 0.7260017582168279, + "rescued": 3908, + "harmed": 1916, + "missed_rescue": 1515, + "avoided_harm": 855, + "selected_both_correct": 14058, + "selected_both_wrong": 4906 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7180614241556306, + "metric": 0.7278450500524629, + "rescued": 4015, + "harmed": 1958, + "missed_rescue": 1408, + "avoided_harm": 813, + "selected_both_correct": 14354, + "selected_both_wrong": 4994 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7352465757309361, + "metric": 0.7297450585599637, + "rescued": 4134, + "harmed": 2010, + "missed_rescue": 1289, + "avoided_harm": 761, + "selected_both_correct": 14689, + "selected_both_wrong": 5094 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7478093185491875, + "metric": 0.7311629753566061, + "rescued": 4208, + "harmed": 2034, + "missed_rescue": 1215, + "avoided_harm": 737, + "selected_both_correct": 14961, + "selected_both_wrong": 5167 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.759804894648782, + "metric": 0.7328928338485098, + "rescued": 4299, + "harmed": 2064, + "missed_rescue": 1124, + "avoided_harm": 707, + "selected_both_correct": 15186, + "selected_both_wrong": 5244 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.7729064458497575, + "metric": 0.7342540339732865, + "rescued": 4385, + "harmed": 2102, + "missed_rescue": 1038, + "avoided_harm": 669, + "selected_both_correct": 15447, + "selected_both_wrong": 5321 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7855259053398747, + "metric": 0.735274934066869, + "rescued": 4462, + "harmed": 2143, + "missed_rescue": 961, + "avoided_harm": 628, + "selected_both_correct": 15703, + "selected_both_wrong": 5392 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.7964722230099538, + "metric": 0.7362958341604514, + "rescued": 4527, + "harmed": 2172, + "missed_rescue": 896, + "avoided_harm": 599, + "selected_both_correct": 15920, + "selected_both_wrong": 5467 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.8082125740861527, + "metric": 0.7377704676289595, + "rescued": 4609, + "harmed": 2202, + "missed_rescue": 814, + "avoided_harm": 569, + "selected_both_correct": 16159, + "selected_both_wrong": 5530 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8197544168108215, + "metric": 0.738422709355415, + "rescued": 4678, + "harmed": 2248, + "missed_rescue": 745, + "avoided_harm": 523, + "selected_both_correct": 16389, + "selected_both_wrong": 5592 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8333380597226555, + "metric": 0.7389048010662734, + "rescued": 4746, + "harmed": 2299, + "missed_rescue": 677, + "avoided_harm": 472, + "selected_both_correct": 16674, + "selected_both_wrong": 5667 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8426112355726966, + "metric": 0.7398689844879902, + "rescued": 4807, + "harmed": 2326, + "missed_rescue": 616, + "avoided_harm": 445, + "selected_both_correct": 16857, + "selected_both_wrong": 5723 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8529903865241187, + "metric": 0.7403227178629158, + "rescued": 4859, + "harmed": 2362, + "missed_rescue": 564, + "avoided_harm": 409, + "selected_both_correct": 17089, + "selected_both_wrong": 5769 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8635396874911381, + "metric": 0.7406913762300428, + "rescued": 4914, + "harmed": 2404, + "missed_rescue": 509, + "avoided_harm": 367, + "selected_both_correct": 17300, + "selected_both_wrong": 5833 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8710262881774098, + "metric": 0.7413152596205654, + "rescued": 4962, + "harmed": 2430, + "missed_rescue": 461, + "avoided_harm": 341, + "selected_both_correct": 17455, + "selected_both_wrong": 5868 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8809517057539064, + "metric": 0.7422227263704165, + "rescued": 5018, + "harmed": 2454, + "missed_rescue": 405, + "avoided_harm": 317, + "selected_both_correct": 17666, + "selected_both_wrong": 5927 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8913024983693957, + "metric": 0.7430167597765364, + "rescued": 5074, + "harmed": 2482, + "missed_rescue": 349, + "avoided_harm": 289, + "selected_both_correct": 17892, + "selected_both_wrong": 5982 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.900915974250631, + "metric": 0.743442134815529, + "rescued": 5120, + "harmed": 2513, + "missed_rescue": 303, + "avoided_harm": 258, + "selected_both_correct": 18112, + "selected_both_wrong": 6024 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.910954825170859, + "metric": 0.7443212432294473, + "rescued": 5172, + "harmed": 2534, + "missed_rescue": 251, + "avoided_harm": 237, + "selected_both_correct": 18362, + "selected_both_wrong": 6055 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.921901142840938, + "metric": 0.7448033349403057, + "rescued": 5223, + "harmed": 2568, + "missed_rescue": 200, + "avoided_harm": 203, + "selected_both_correct": 18626, + "selected_both_wrong": 6092 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9303235686129938, + "metric": 0.7450018432918356, + "rescued": 5253, + "harmed": 2591, + "missed_rescue": 170, + "avoided_harm": 180, + "selected_both_correct": 18846, + "selected_both_wrong": 6116 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9393698777755721, + "metric": 0.7448884099481042, + "rescued": 5284, + "harmed": 2626, + "missed_rescue": 139, + "avoided_harm": 145, + "selected_both_correct": 19077, + "selected_both_wrong": 6138 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9490117119927403, + "metric": 0.7448884099481042, + "rescued": 5314, + "harmed": 2656, + "missed_rescue": 109, + "avoided_harm": 115, + "selected_both_correct": 19329, + "selected_both_wrong": 6166 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9587953378895726, + "metric": 0.7449734849559028, + "rescued": 5347, + "harmed": 2686, + "missed_rescue": 76, + "avoided_harm": 85, + "selected_both_correct": 19588, + "selected_both_wrong": 6189 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9673595553412926, + "metric": 0.7452570683152313, + "rescued": 5375, + "harmed": 2704, + "missed_rescue": 48, + "avoided_harm": 67, + "selected_both_correct": 19828, + "selected_both_wrong": 6205 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9746192893401016, + "metric": 0.745313784987097, + "rescued": 5398, + "harmed": 2725, + "missed_rescue": 25, + "avoided_harm": 46, + "selected_both_correct": 20033, + "selected_both_wrong": 6212 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9830984317840229, + "metric": 0.7450585599637013, + "rescued": 5411, + "harmed": 2747, + "missed_rescue": 12, + "avoided_harm": 24, + "selected_both_correct": 20290, + "selected_both_wrong": 6219 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9908118991577575, + "metric": 0.7448884099481042, + "rescued": 5419, + "harmed": 2761, + "missed_rescue": 4, + "avoided_harm": 10, + "selected_both_correct": 20535, + "selected_both_wrong": 6224 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 5423, + "harmed": 2771, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 20838, + "selected_both_wrong": 6231 + } + ] + }, + "learned": { + "validation_area_0_25": 0.7676983624991215, + "validation_point": { + "metric": 0.8083069787054606, + "metric_std": 0.0006490828527103093, + "actual_fraction": 0.9679527725068523, + "actual_fraction_std": 0.008378985030784162, + "per_seed": [ + { + "seed": 2026, + "budget": 0.97, + "threshold": -0.7546408772468567, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8080680300794153, + "rescue": 1884, + "harm": 1062, + "missed_rescue": 80 + }, + { + "seed": 2027, + "budget": 0.96, + "threshold": -0.6507218480110168, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8084194251177174, + "rescue": 1853, + "harm": 1026, + "missed_rescue": 111 + }, + { + "seed": 2028, + "budget": 0.97, + "threshold": -0.7309830784797668, + "operator": "gt", + "actual_fraction": 0.9699205847213437, + "metric": 0.8073652400028112, + "rescue": 1875, + "harm": 1063, + "missed_rescue": 89 + }, + { + "seed": 2029, + "budget": 0.98, + "threshold": -0.6767090559005737, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8085599831330381, + "rescue": 1917, + "harm": 1088, + "missed_rescue": 47 + }, + { + "seed": 2030, + "budget": 0.96, + "threshold": -0.5991365909576416, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8091222151943215, + "rescue": 1864, + "harm": 1027, + "missed_rescue": 100 + } + ], + "rescued": 1878.6, + "harmed": 1053.2, + "missed_rescue": 85.4, + "avoided_harm": 117.79999999999995 + }, + "test_point": { + "metric": 0.7475030485211128, + "metric_std": 0.0016083702167531654, + "actual_fraction": 0.9685052321129796, + "actual_fraction_std": 0.00781786913048959, + "per_seed": [ + { + "seed": 2026, + "budget": 0.97, + "threshold": -0.7546408772468567, + "operator": "ge", + "actual_fraction": 0.9670192553100984, + "metric": 0.7457108016901568, + "rescue": 5184, + "harm": 2497, + "missed_rescue": 239 + }, + { + "seed": 2027, + "budget": 0.96, + "threshold": -0.6507218480110168, + "operator": "ge", + "actual_fraction": 0.9593057879363639, + "metric": 0.7500212687519496, + "rescue": 5193, + "harm": 2354, + "missed_rescue": 230 + }, + { + "seed": 2028, + "budget": 0.97, + "threshold": -0.7309830784797668, + "operator": "gt", + "actual_fraction": 0.9723789808014066, + "metric": 0.7466749851118737, + "rescue": 5233, + "harm": 2512, + "missed_rescue": 190 + }, + { + "seed": 2029, + "budget": 0.98, + "threshold": -0.6767090559005737, + "operator": "ge", + "actual_fraction": 0.9796387148002155, + "metric": 0.7477809602132547, + "rescue": 5314, + "harm": 2554, + "missed_rescue": 109 + }, + { + "seed": 2030, + "budget": 0.96, + "threshold": -0.5991365909576416, + "operator": "ge", + "actual_fraction": 0.9641834217168137, + "metric": 0.7473272268383291, + "rescue": 5208, + "harm": 2464, + "missed_rescue": 215 + } + ], + "rescued": 5226.4, + "harmed": 2476.2, + "missed_rescue": 196.6, + "avoided_harm": 294.8000000000002 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.7502986857825568, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.7547122074636305, + "metric_std": 0.0007259518734200321, + "actual_fraction": 0.009979619087778481, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.7588305573125308, + "metric_std": 0.0010883039614336476, + "actual_fraction": 0.019959238175556962, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.7620774474664418, + "metric_std": 0.0017168803412301464, + "actual_fraction": 0.029938857263335443, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.7640030922763371, + "metric_std": 0.001825225794254048, + "actual_fraction": 0.03998875535877434, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.7649307751774544, + "metric_std": 0.001029773337019879, + "actual_fraction": 0.04995431864502073, + "actual_fraction_std": 3.1429727703979486e-05 + }, + { + "budget": 0.06, + "metric": 0.7657319558647833, + "metric_std": 0.0005591489010026251, + "actual_fraction": 0.0599339377327992, + "actual_fraction_std": 3.1429727703979486e-05 + }, + { + "budget": 0.07, + "metric": 0.7660974067046173, + "metric_std": 0.0005718137776880627, + "actual_fraction": 0.06999789162977019, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.766729917773561, + "metric_std": 0.0004904447627343644, + "actual_fraction": 0.07997751071754867, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.7671375360179914, + "metric_std": 0.0008558471250753402, + "actual_fraction": 0.08995712980532715, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.7677841028884671, + "metric_std": 0.0010869415941629492, + "actual_fraction": 0.09993674889310562, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.7682901117436222, + "metric_std": 0.001423041445731718, + "actual_fraction": 0.10998664698854452, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.12, + "metric": 0.768922622812566, + "metric_std": 0.0018170895317798513, + "actual_fraction": 0.11996626607632302, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.13, + "metric": 0.7694286316677209, + "metric_std": 0.0018603372381519757, + "actual_fraction": 0.1299458851641015, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.7699908637290042, + "metric_std": 0.0022131751093655183, + "actual_fraction": 0.13999578325954037, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.7704547051795629, + "metric_std": 0.0024549414061066344, + "actual_fraction": 0.14997540234731885, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.7708904350270575, + "metric_std": 0.0028115995159038007, + "actual_fraction": 0.15995502143509735, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.7713823880806803, + "metric_std": 0.003061778186912915, + "actual_fraction": 0.16993464052287582, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.7717478389205145, + "metric_std": 0.002972311858158178, + "actual_fraction": 0.1799845386183147, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.772183568768009, + "metric_std": 0.0031541888849572534, + "actual_fraction": 0.18996415770609318, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.772703633424696, + "metric_std": 0.003409582272165642, + "actual_fraction": 0.19994377679387168, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.7731815306767867, + "metric_std": 0.0033316252625137173, + "actual_fraction": 0.2099936748893106, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.7736875395319418, + "metric_std": 0.003692959569185956, + "actual_fraction": 0.21997329397708903, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.23, + "metric": 0.7743903296085459, + "metric_std": 0.0037671121483741775, + "actual_fraction": 0.22995291306486756, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.24, + "metric": 0.7745871108299951, + "metric_std": 0.003818614909469239, + "actual_fraction": 0.23993253215264604, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.25, + "metric": 0.7749385058682972, + "metric_std": 0.003963324803755545, + "actual_fraction": 0.2499683744465528, + "actual_fraction_std": 3.142972770397948e-05 + }, + { + "budget": 0.26, + "metric": 0.7755007379295804, + "metric_std": 0.004047790203529965, + "actual_fraction": 0.25996204933586337, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.7758661887694146, + "metric_std": 0.004199839628248356, + "actual_fraction": 0.26994166842364187, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.7761473048000562, + "metric_std": 0.004213928361639644, + "actual_fraction": 0.27997751071754867, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.29, + "metric": 0.7765549230444866, + "metric_std": 0.004222768432520152, + "actual_fraction": 0.28997118560685925, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.7771452667088341, + "metric_std": 0.004163994136727271, + "actual_fraction": 0.2999508046946377, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.7776231639609248, + "metric_std": 0.00434918616677178, + "actual_fraction": 0.3099163679808841, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.32, + "metric": 0.7780026706022911, + "metric_std": 0.004538413501954933, + "actual_fraction": 0.319966266076323, + "actual_fraction_std": 3.142972770396707e-05 + }, + { + "budget": 0.33, + "metric": 0.7784524562513176, + "metric_std": 0.004725603536982599, + "actual_fraction": 0.3299599409656336, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.7789584651064727, + "metric_std": 0.005085743679196581, + "actual_fraction": 0.339939560053412, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.7790427999156652, + "metric_std": 0.005095349224641933, + "actual_fraction": 0.34998945814885096, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.7794644739616275, + "metric_std": 0.004965747783242506, + "actual_fraction": 0.3599690772366294, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.7797737015953334, + "metric_std": 0.0051993642917594835, + "actual_fraction": 0.3699486963244079, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.7801391524351675, + "metric_std": 0.00548775106241972, + "actual_fraction": 0.3799985944198468, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.7805608264811301, + "metric_std": 0.005585213834457296, + "actual_fraction": 0.3899782135076253, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.7810949469393492, + "metric_std": 0.0058385319275064814, + "actual_fraction": 0.39995783259540374, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.7815447325883759, + "metric_std": 0.005897284024144326, + "actual_fraction": 0.4099374516831823, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.42, + "metric": 0.7819945182374024, + "metric_std": 0.005940135388042866, + "actual_fraction": 0.4199873497786212, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.782598917703282, + "metric_std": 0.0062280446098776435, + "actual_fraction": 0.4299669688663997, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.44, + "metric": 0.7830346475507766, + "metric_std": 0.006102023152947558, + "actual_fraction": 0.43994658795417807, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.45, + "metric": 0.7835125448028674, + "metric_std": 0.006430164354199029, + "actual_fraction": 0.44999648604961695, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.46, + "metric": 0.7837093260243165, + "metric_std": 0.006356385615168822, + "actual_fraction": 0.4599761051373955, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.47, + "metric": 0.7842715580855998, + "metric_std": 0.006150838664933861, + "actual_fraction": 0.4699416684236419, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.48, + "metric": 0.7847916227422869, + "metric_std": 0.006023407071416089, + "actual_fraction": 0.4799212875114203, + "actual_fraction_std": 3.142972770396707e-05 + }, + { + "budget": 0.49, + "metric": 0.7852414083913135, + "metric_std": 0.006116777348375956, + "actual_fraction": 0.4899852414083913, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.7854381896127627, + "metric_std": 0.006354909075733296, + "actual_fraction": 0.49996486049616984, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.51, + "metric": 0.7857614730480005, + "metric_std": 0.006328350502828371, + "actual_fraction": 0.5099444795839483, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.7862393703000914, + "metric_std": 0.0061712812581221285, + "actual_fraction": 0.5199943776793872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.7869140487736312, + "metric_std": 0.006326867420671999, + "actual_fraction": 0.5299599409656336, + "actual_fraction_std": 3.1429727703942244e-05 + }, + { + "budget": 0.54, + "metric": 0.7871811090027409, + "metric_std": 0.00633747551769024, + "actual_fraction": 0.5399536158549442, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.7874200576287862, + "metric_std": 0.00644144582420692, + "actual_fraction": 0.5499332349427226, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.7876730620563637, + "metric_std": 0.006456763110328782, + "actual_fraction": 0.5599831330381615, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.7880947361023262, + "metric_std": 0.006586459211308146, + "actual_fraction": 0.5699486963244078, + "actual_fraction_std": 3.1429727703942244e-05 + }, + { + "budget": 0.58, + "metric": 0.788586689155949, + "metric_std": 0.006695601706686274, + "actual_fraction": 0.5799423712137185, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.7890364748049757, + "metric_std": 0.0070165415262990775, + "actual_fraction": 0.5899922693091574, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.7896689858739194, + "metric_std": 0.007287258737123402, + "actual_fraction": 0.5999718883969358, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.7901468831260102, + "metric_std": 0.00714482122129181, + "actual_fraction": 0.6099515074847143, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.7910745660271277, + "metric_std": 0.0071754144065847856, + "actual_fraction": 0.6199311265724928, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.7912572914470448, + "metric_std": 0.007018828918932621, + "actual_fraction": 0.6299669688663996, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.64, + "metric": 0.7920022489282451, + "metric_std": 0.007105934062292708, + "actual_fraction": 0.6399606437557102, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.7924098671726756, + "metric_std": 0.006520134698969428, + "actual_fraction": 0.6499402628434886, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.7924520345772718, + "metric_std": 0.006706657616287852, + "actual_fraction": 0.6599901609389276, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.7926488157987209, + "metric_std": 0.0066175439089052645, + "actual_fraction": 0.669969780026706, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.7932251036615362, + "metric_std": 0.00660704908281306, + "actual_fraction": 0.6799493991144845, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.7937732799212875, + "metric_std": 0.006301366478893631, + "actual_fraction": 0.6899992972099234, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.7944339025932954, + "metric_std": 0.006221776375580072, + "actual_fraction": 0.6999789162977019, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.794883688242322, + "metric_std": 0.006594927556976371, + "actual_fraction": 0.7099585353854804, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.7954318645020733, + "metric_std": 0.0065456511035688275, + "actual_fraction": 0.7199381544732588, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.7956567573265866, + "metric_std": 0.006260475146015711, + "actual_fraction": 0.7299880525686977, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.7959940965633565, + "metric_std": 0.006024022033190305, + "actual_fraction": 0.7399676716564763, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.7963033241970623, + "metric_std": 0.005618278116597946, + "actual_fraction": 0.7499472907442547, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.7963595474031907, + "metric_std": 0.005386692242359227, + "actual_fraction": 0.7599971888396936, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.7969077236629419, + "metric_std": 0.005083800961369111, + "actual_fraction": 0.769976807927472, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.7977791833579311, + "metric_std": 0.004973102683713694, + "actual_fraction": 0.7799423712137185, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.79, + "metric": 0.797624569541078, + "metric_std": 0.004761266617677199, + "actual_fraction": 0.789921990301497, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.8, + "metric": 0.7983976386253426, + "metric_std": 0.004472970369670437, + "actual_fraction": 0.7999859441984679, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.7990582612973505, + "metric_std": 0.0038533153795772745, + "actual_fraction": 0.8099655632862464, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.79966266076323, + "metric_std": 0.0034717402986026657, + "actual_fraction": 0.8199451823740249, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.8002951718321738, + "metric_std": 0.0028979333846415583, + "actual_fraction": 0.8299950804694639, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.8008995712980532, + "metric_std": 0.002574073592214846, + "actual_fraction": 0.8399606437557102, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.85, + "metric": 0.8015461381685289, + "metric_std": 0.0021946863722574197, + "actual_fraction": 0.8499543186450207, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.8021786492374728, + "metric_std": 0.0019737018175032092, + "actual_fraction": 0.8599339377327994, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.87, + "metric": 0.8029938857263336, + "metric_std": 0.002009045625059515, + "actual_fraction": 0.8699838358282381, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.88, + "metric": 0.8034858387799565, + "metric_std": 0.0017996092321270903, + "actual_fraction": 0.8799493991144844, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.89, + "metric": 0.8039075128259189, + "metric_std": 0.0016462398136844144, + "actual_fraction": 0.8899290182022629, + "actual_fraction_std": 3.1429727703942244e-05 + }, + { + "budget": 0.9, + "metric": 0.8041745730550286, + "metric_std": 0.0020709066425763484, + "actual_fraction": 0.8999929720992339, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.91, + "metric": 0.8043854100780097, + "metric_std": 0.002078643481993026, + "actual_fraction": 0.9099725911870123, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.8050882001546139, + "metric_std": 0.0015410184916624494, + "actual_fraction": 0.919952210274791, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.93, + "metric": 0.8056644880174293, + "metric_std": 0.0018768574469380137, + "actual_fraction": 0.9299318293625694, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.8064094454986297, + "metric_std": 0.0017418705569481625, + "actual_fraction": 0.9399536158549442, + "actual_fraction_std": 3.8493397814698215e-05 + }, + { + "budget": 0.95, + "metric": 0.806971677559913, + "metric_std": 0.0014679735385239663, + "actual_fraction": 0.9499613465457868, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.8076041886288566, + "metric_std": 0.001243958288257919, + "actual_fraction": 0.9599409656335652, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.8080539742778832, + "metric_std": 0.0004979405322627838, + "actual_fraction": 0.9699768079274721, + "actual_fraction_std": 3.1429727703991894e-05 + }, + { + "budget": 0.98, + "metric": 0.8076182444303888, + "metric_std": 0.0008558471250753119, + "actual_fraction": 0.9799704828167826, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.8073933516058753, + "metric_std": 0.0006733594527797161, + "actual_fraction": 0.9899501019045612, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 1.0, + "metric": 0.8060299388572634, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.6695119530385957, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.6820293225193546, + "metric_std": 0.0023891456235174244, + "actual_fraction": 0.022618608740039133, + "actual_fraction_std": 0.005373683923630361 + }, + { + "budget": 0.02, + "metric": 0.6924878768113887, + "metric_std": 0.0037742910345190993, + "actual_fraction": 0.04697274763916853, + "actual_fraction_std": 0.009129371563563294 + }, + { + "budget": 0.03, + "metric": 0.7009159742506309, + "metric_std": 0.003413889034750999, + "actual_fraction": 0.06999971641664067, + "actual_fraction_std": 0.009726983634309882 + }, + { + "budget": 0.04, + "metric": 0.7082607832572384, + "metric_std": 0.003196677616190534, + "actual_fraction": 0.09352579190653092, + "actual_fraction_std": 0.01002813071205432 + }, + { + "budget": 0.05, + "metric": 0.7125655786518447, + "metric_std": 0.0019112340291323213, + "actual_fraction": 0.11378498709695714, + "actual_fraction_std": 0.006376042492196417 + }, + { + "budget": 0.06, + "metric": 0.7150327538780025, + "metric_std": 0.003548329346963179, + "actual_fraction": 0.1395570427927289, + "actual_fraction_std": 0.01487862193804636 + }, + { + "budget": 0.07, + "metric": 0.7161784306496894, + "metric_std": 0.004805194296594225, + "actual_fraction": 0.16066698806114058, + "actual_fraction_std": 0.016240571649879125 + }, + { + "budget": 0.08, + "metric": 0.7164563423418314, + "metric_std": 0.004877452415627197, + "actual_fraction": 0.1810736465984176, + "actual_fraction_std": 0.01856396323083792 + }, + { + "budget": 0.09, + "metric": 0.7167852990386525, + "metric_std": 0.004937649672926311, + "actual_fraction": 0.19908118991577572, + "actual_fraction_std": 0.020690718805558013 + }, + { + "budget": 0.1, + "metric": 0.7171709724073392, + "metric_std": 0.005184210963019706, + "actual_fraction": 0.2153758897427899, + "actual_fraction_std": 0.02141659380482878 + }, + { + "budget": 0.11, + "metric": 0.7176757507869439, + "metric_std": 0.005401299023506229, + "actual_fraction": 0.2307120778152738, + "actual_fraction_std": 0.022922172920091213 + }, + { + "budget": 0.12, + "metric": 0.7181011258259365, + "metric_std": 0.005534222076845124, + "actual_fraction": 0.244397810736466, + "actual_fraction_std": 0.021973587939986574 + }, + { + "budget": 0.13, + "metric": 0.7185038141961829, + "metric_std": 0.0057056873234601286, + "actual_fraction": 0.2576298102827326, + "actual_fraction_std": 0.02247967811487851 + }, + { + "budget": 0.14, + "metric": 0.7189065025664294, + "metric_std": 0.0058450651183511995, + "actual_fraction": 0.27104897484615603, + "actual_fraction_std": 0.023650784162295248 + }, + { + "budget": 0.15, + "metric": 0.7192468025976235, + "metric_std": 0.006076642857476141, + "actual_fraction": 0.28379321101437766, + "actual_fraction_std": 0.025177552133844268 + }, + { + "budget": 0.16, + "metric": 0.7198253126506537, + "metric_std": 0.006161640603796259, + "actual_fraction": 0.29630490882794996, + "actual_fraction_std": 0.026547731573919525 + }, + { + "budget": 0.17, + "metric": 0.7200918810084225, + "metric_std": 0.006353298952844667, + "actual_fraction": 0.3085840682868729, + "actual_fraction_std": 0.02920552448228976 + }, + { + "budget": 0.18, + "metric": 0.7205115843802286, + "metric_std": 0.006389838543718528, + "actual_fraction": 0.3212659161160423, + "actual_fraction_std": 0.0307893271737821 + }, + { + "budget": 0.19, + "metric": 0.720834869409863, + "metric_std": 0.0064758729844944605, + "actual_fraction": 0.3311743186909792, + "actual_fraction_std": 0.03166062093967711 + }, + { + "budget": 0.2, + "metric": 0.7211751694410571, + "metric_std": 0.006633499115932629, + "actual_fraction": 0.34315855145620056, + "actual_fraction_std": 0.03219322179920129 + }, + { + "budget": 0.21, + "metric": 0.7215268128066245, + "metric_std": 0.006587513866180276, + "actual_fraction": 0.3541105407934662, + "actual_fraction_std": 0.03271342704990506 + }, + { + "budget": 0.22, + "metric": 0.7219181578424978, + "metric_std": 0.006494120846090928, + "actual_fraction": 0.3631341632872983, + "actual_fraction_std": 0.03314726420304771 + }, + { + "budget": 0.23, + "metric": 0.7221847262002665, + "metric_std": 0.006371247838988163, + "actual_fraction": 0.3724413691404588, + "actual_fraction_std": 0.034593616728042904 + }, + { + "budget": 0.24, + "metric": 0.7224172645549158, + "metric_std": 0.006547294334238303, + "actual_fraction": 0.38183932166860446, + "actual_fraction_std": 0.036219814632164705 + }, + { + "budget": 0.25, + "metric": 0.7227802512548565, + "metric_std": 0.006655925575639704, + "actual_fraction": 0.3922184726200267, + "actual_fraction_std": 0.036465387519189194 + }, + { + "budget": 0.26, + "metric": 0.7230921929501177, + "metric_std": 0.006812395968694786, + "actual_fraction": 0.4016220968153589, + "actual_fraction_std": 0.037671007873911075 + }, + { + "budget": 0.27, + "metric": 0.7234268213141253, + "metric_std": 0.00693189924517408, + "actual_fraction": 0.4109973626747583, + "actual_fraction_std": 0.039249489027500604 + }, + { + "budget": 0.28, + "metric": 0.7237217480078268, + "metric_std": 0.00700309091507378, + "actual_fraction": 0.4200833735076426, + "actual_fraction_std": 0.040396659062990316 + }, + { + "budget": 0.29, + "metric": 0.7240053313671554, + "metric_std": 0.0069895848581172535, + "actual_fraction": 0.4284717692765788, + "actual_fraction_std": 0.04151124946857941 + }, + { + "budget": 0.3, + "metric": 0.7244193630717749, + "metric_std": 0.007057107362624609, + "actual_fraction": 0.43784136346879166, + "actual_fraction_std": 0.04210843161121919 + }, + { + "budget": 0.31, + "metric": 0.724657573093611, + "metric_std": 0.0070608725828643715, + "actual_fraction": 0.4468309559595043, + "actual_fraction_std": 0.04319495240881481 + }, + { + "budget": 0.32, + "metric": 0.7250432464622977, + "metric_std": 0.007170689967913223, + "actual_fraction": 0.4549300967019255, + "actual_fraction_std": 0.043292514959209424 + }, + { + "budget": 0.33, + "metric": 0.7253438448231858, + "metric_std": 0.007242721217948674, + "actual_fraction": 0.46269460908033916, + "actual_fraction_std": 0.04394282687582479 + }, + { + "budget": 0.34, + "metric": 0.7255763831778352, + "metric_std": 0.007212473202827574, + "actual_fraction": 0.47102628817740977, + "actual_fraction_std": 0.044076679260183056 + }, + { + "budget": 0.35, + "metric": 0.725763548194992, + "metric_std": 0.007291639022343828, + "actual_fraction": 0.4794543856166521, + "actual_fraction_std": 0.04497514960715799 + }, + { + "budget": 0.36, + "metric": 0.7259790715480816, + "metric_std": 0.007310496294673, + "actual_fraction": 0.48683322462637896, + "actual_fraction_std": 0.045244844717461645 + }, + { + "budget": 0.37, + "metric": 0.7262739982417832, + "metric_std": 0.007294671368135503, + "actual_fraction": 0.49447863199387465, + "actual_fraction_std": 0.04515074305701586 + }, + { + "budget": 0.38, + "metric": 0.7267220599495221, + "metric_std": 0.007394313062334996, + "actual_fraction": 0.502668519411281, + "actual_fraction_std": 0.045031580233620666 + }, + { + "budget": 0.39, + "metric": 0.7270169866432237, + "metric_std": 0.007381408468435777, + "actual_fraction": 0.5101097467600602, + "actual_fraction_std": 0.045655480895039076 + }, + { + "budget": 0.4, + "metric": 0.7273459433400448, + "metric_std": 0.007526309836165254, + "actual_fraction": 0.5173297790885631, + "actual_fraction_std": 0.04645738810764011 + }, + { + "budget": 0.41, + "metric": 0.7276125116978135, + "metric_std": 0.00767944051223215, + "actual_fraction": 0.5249808581232454, + "actual_fraction_std": 0.046103278899981864 + }, + { + "budget": 0.42, + "metric": 0.7279868417321271, + "metric_std": 0.007773676304017261, + "actual_fraction": 0.5325298471485693, + "actual_fraction_std": 0.045926999632235795 + }, + { + "budget": 0.43, + "metric": 0.7282817684258287, + "metric_std": 0.007913490057783404, + "actual_fraction": 0.5394606244505573, + "actual_fraction_std": 0.04577753545989478 + }, + { + "budget": 0.44, + "metric": 0.7286787851288887, + "metric_std": 0.008021364732432888, + "actual_fraction": 0.5469188668008961, + "actual_fraction_std": 0.045946263043180925 + }, + { + "budget": 0.45, + "metric": 0.7290361001616426, + "metric_std": 0.008050971027183386, + "actual_fraction": 0.5541388991293991, + "actual_fraction_std": 0.04566562832049603 + }, + { + "budget": 0.46, + "metric": 0.729319683520971, + "metric_std": 0.008123388751865317, + "actual_fraction": 0.5611320647704392, + "actual_fraction_std": 0.045443303824226934 + }, + { + "budget": 0.47, + "metric": 0.7296769985537248, + "metric_std": 0.008081046061921072, + "actual_fraction": 0.5688795621472932, + "actual_fraction_std": 0.04489006163089001 + }, + { + "budget": 0.48, + "metric": 0.7301590902645833, + "metric_std": 0.008269764355896709, + "actual_fraction": 0.5762584011570201, + "actual_fraction_std": 0.04528542389861715 + }, + { + "budget": 0.49, + "metric": 0.7306752119785611, + "metric_std": 0.008394603928036582, + "actual_fraction": 0.5837166435073591, + "actual_fraction_std": 0.04498153530983198 + }, + { + "budget": 0.5, + "metric": 0.7310495420128748, + "metric_std": 0.00850773709603825, + "actual_fraction": 0.5911692141905112, + "actual_fraction_std": 0.044484687617334394 + }, + { + "budget": 0.51, + "metric": 0.7313331253722032, + "metric_std": 0.008632237398455657, + "actual_fraction": 0.5982928281768426, + "actual_fraction_std": 0.043731664671172385 + }, + { + "budget": 0.52, + "metric": 0.731679097070584, + "metric_std": 0.008760256719998166, + "actual_fraction": 0.605229277146017, + "actual_fraction_std": 0.043082301840624164 + }, + { + "budget": 0.53, + "metric": 0.7321952187845617, + "metric_std": 0.008729868566103199, + "actual_fraction": 0.6125967728213709, + "actual_fraction_std": 0.04227663916601819 + }, + { + "budget": 0.54, + "metric": 0.732546862150129, + "metric_std": 0.00867779907306517, + "actual_fraction": 0.6197997901483141, + "actual_fraction_std": 0.041206784539613864 + }, + { + "budget": 0.55, + "metric": 0.7330232821938008, + "metric_std": 0.008659922627593481, + "actual_fraction": 0.6277628108782576, + "actual_fraction_std": 0.04089442345313724 + }, + { + "budget": 0.56, + "metric": 0.7335337322405922, + "metric_std": 0.00867914735635373, + "actual_fraction": 0.6354592632504324, + "actual_fraction_std": 0.04035178659591545 + }, + { + "budget": 0.57, + "metric": 0.7340498539545701, + "metric_std": 0.008961018772424884, + "actual_fraction": 0.6434790006522417, + "actual_fraction_std": 0.03941488375266616 + }, + { + "budget": 0.58, + "metric": 0.7345943340044807, + "metric_std": 0.008974170039444425, + "actual_fraction": 0.6511924680259762, + "actual_fraction_std": 0.03836039135134453 + }, + { + "budget": 0.59, + "metric": 0.7348892606981823, + "metric_std": 0.00896492625977152, + "actual_fraction": 0.6585145903638375, + "actual_fraction_std": 0.03754349373273932 + }, + { + "budget": 0.6, + "metric": 0.7354337407480929, + "metric_std": 0.008773745586324991, + "actual_fraction": 0.6665173127640871, + "actual_fraction_std": 0.036258495238136096 + }, + { + "budget": 0.61, + "metric": 0.7360576241386155, + "metric_std": 0.008766501512535697, + "actual_fraction": 0.6734254033973287, + "actual_fraction_std": 0.03522870709271844 + }, + { + "budget": 0.62, + "metric": 0.7365907608541532, + "metric_std": 0.008584256122490596, + "actual_fraction": 0.6808779740804809, + "actual_fraction_std": 0.03487122649277465 + }, + { + "budget": 0.63, + "metric": 0.7369197175509742, + "metric_std": 0.008823203028224222, + "actual_fraction": 0.6882000964183421, + "actual_fraction_std": 0.034364962149672204 + }, + { + "budget": 0.64, + "metric": 0.7372486742477952, + "metric_std": 0.008451797524666066, + "actual_fraction": 0.6960213254686215, + "actual_fraction_std": 0.03314047641363662 + }, + { + "budget": 0.65, + "metric": 0.7377137509570939, + "metric_std": 0.008536353723595541, + "actual_fraction": 0.7038482261860874, + "actual_fraction_std": 0.03179769006943267 + }, + { + "budget": 0.66, + "metric": 0.7382752460085642, + "metric_std": 0.008475902904764555, + "actual_fraction": 0.711771545245725, + "actual_fraction_std": 0.030697858696738835 + }, + { + "budget": 0.67, + "metric": 0.7386949493803703, + "metric_std": 0.008537663118287825, + "actual_fraction": 0.7192014292601311, + "actual_fraction_std": 0.030536696726414254 + }, + { + "budget": 0.68, + "metric": 0.7390465927459375, + "metric_std": 0.008523225124707295, + "actual_fraction": 0.7269999716416641, + "actual_fraction_std": 0.02888396679065441 + }, + { + "budget": 0.69, + "metric": 0.7395400277911691, + "metric_std": 0.008408870655608847, + "actual_fraction": 0.7342653773076595, + "actual_fraction_std": 0.027372071961316035 + }, + { + "budget": 0.7, + "metric": 0.7400618211723338, + "metric_std": 0.008035583562678618, + "actual_fraction": 0.7417633213283045, + "actual_fraction_std": 0.026241624848910915 + }, + { + "budget": 0.71, + "metric": 0.7406063012222444, + "metric_std": 0.007911848670767876, + "actual_fraction": 0.7497206703910615, + "actual_fraction_std": 0.024870766660361737 + }, + { + "budget": 0.72, + "metric": 0.7412925729518192, + "metric_std": 0.007723818519052638, + "actual_fraction": 0.757564586110087, + "actual_fraction_std": 0.023717819383436766 + }, + { + "budget": 0.73, + "metric": 0.7415818279783342, + "metric_std": 0.007724448412500718, + "actual_fraction": 0.7648129767745229, + "actual_fraction_std": 0.022389823041914395 + }, + { + "budget": 0.74, + "metric": 0.7422057113688568, + "metric_std": 0.007808657169939039, + "actual_fraction": 0.772532115815444, + "actual_fraction_std": 0.021694269706581545 + }, + { + "budget": 0.75, + "metric": 0.7427785497547003, + "metric_std": 0.007686040306800451, + "actual_fraction": 0.7799563281626634, + "actual_fraction_std": 0.0205288166025934 + }, + { + "budget": 0.76, + "metric": 0.7432436264639991, + "metric_std": 0.007344234152977777, + "actual_fraction": 0.7873975555114426, + "actual_fraction_std": 0.020018705526676703 + }, + { + "budget": 0.77, + "metric": 0.7434818364858351, + "metric_std": 0.007040598712598621, + "actual_fraction": 0.7952357995632817, + "actual_fraction_std": 0.018819453651705776 + }, + { + "budget": 0.78, + "metric": 0.7435895981623799, + "metric_std": 0.006893683404955216, + "actual_fraction": 0.8029435952698296, + "actual_fraction_std": 0.017502015070681824 + }, + { + "budget": 0.79, + "metric": 0.7439412415279472, + "metric_std": 0.006748586872888711, + "actual_fraction": 0.8108328843263477, + "actual_fraction_std": 0.016922818850421295 + }, + { + "budget": 0.8, + "metric": 0.7444176615716189, + "metric_std": 0.00664588962159712, + "actual_fraction": 0.8184556050250972, + "actual_fraction_std": 0.01621183588435816 + }, + { + "budget": 0.81, + "metric": 0.7447693049371862, + "metric_std": 0.006572328082629666, + "actual_fraction": 0.8269120608002721, + "actual_fraction_std": 0.014942590881412605 + }, + { + "budget": 0.82, + "metric": 0.745047216629328, + "metric_std": 0.006315784053547189, + "actual_fraction": 0.834358959816238, + "actual_fraction_std": 0.014191852647896628 + }, + { + "budget": 0.83, + "metric": 0.7453591583245895, + "metric_std": 0.006167321974223526, + "actual_fraction": 0.8422312338711965, + "actual_fraction_std": 0.012979812029320635 + }, + { + "budget": 0.84, + "metric": 0.7457108016901568, + "metric_std": 0.005981760600547317, + "actual_fraction": 0.8502112696026998, + "actual_fraction_std": 0.012002244823753635 + }, + { + "budget": 0.85, + "metric": 0.7463630434166124, + "metric_std": 0.005472130197695882, + "actual_fraction": 0.858690412046621, + "actual_fraction_std": 0.01143704380091484 + }, + { + "budget": 0.86, + "metric": 0.7465728951025155, + "metric_std": 0.005390120764769276, + "actual_fraction": 0.8668973144655873, + "actual_fraction_std": 0.010714922445171362 + }, + { + "budget": 0.87, + "metric": 0.7467260301165528, + "metric_std": 0.005238637476966841, + "actual_fraction": 0.875988996965658, + "actual_fraction_std": 0.009926896151025443 + }, + { + "budget": 0.88, + "metric": 0.746986926807135, + "metric_std": 0.004984254863292578, + "actual_fraction": 0.8849275444516916, + "actual_fraction_std": 0.009095301315439994 + }, + { + "budget": 0.89, + "metric": 0.7471514051555455, + "metric_std": 0.004881605627827634, + "actual_fraction": 0.892828176842583, + "actual_fraction_std": 0.008189679244150608 + }, + { + "budget": 0.9, + "metric": 0.7471627484899186, + "metric_std": 0.004867871711747116, + "actual_fraction": 0.901982247681706, + "actual_fraction_std": 0.00766416754088231 + }, + { + "budget": 0.91, + "metric": 0.7475484218586054, + "metric_std": 0.004644252216813771, + "actual_fraction": 0.9107506451521425, + "actual_fraction_std": 0.0067505170695253336 + }, + { + "budget": 0.92, + "metric": 0.7478887218897995, + "metric_std": 0.004128909437621375, + "actual_fraction": 0.9197969543147207, + "actual_fraction_std": 0.006761319555960693 + }, + { + "budget": 0.93, + "metric": 0.7477752885460681, + "metric_std": 0.0036122887938997393, + "actual_fraction": 0.9289169951507246, + "actual_fraction_std": 0.006646282880780948 + }, + { + "budget": 0.94, + "metric": 0.7478263335507472, + "metric_std": 0.0032550132324391224, + "actual_fraction": 0.939398236111505, + "actual_fraction_std": 0.005735574220538935 + }, + { + "budget": 0.95, + "metric": 0.7477412585429487, + "metric_std": 0.002725618200426499, + "actual_fraction": 0.949448430366106, + "actual_fraction_std": 0.0046501445512638585 + }, + { + "budget": 0.96, + "metric": 0.7473896151773813, + "metric_std": 0.0019316377889654211, + "actual_fraction": 0.959804894648782, + "actual_fraction_std": 0.00394132922601292 + }, + { + "budget": 0.97, + "metric": 0.7472534951649038, + "metric_std": 0.0011022604654357547, + "actual_fraction": 0.9704222556220401, + "actual_fraction_std": 0.0024590945352570316 + }, + { + "budget": 0.98, + "metric": 0.746867821796217, + "metric_std": 0.000739603668542789, + "actual_fraction": 0.9806369282250518, + "actual_fraction_std": 0.0017494358338850417 + }, + { + "budget": 0.99, + "metric": 0.7459263250432464, + "metric_std": 0.00047983444081557583, + "actual_fraction": 0.9906871224796528, + "actual_fraction_std": 0.0011294666318829858 + }, + { + "budget": 1.0, + "metric": 0.7447182599325072, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 9505, + "rescuable": 1964, + "harmful": 1171, + "both_wrong": 1589, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 20838, + "rescuable": 5423, + "harmful": 2771, + "both_wrong": 6231, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.7502986857825568, + "test": 0.6695119530385957 + }, + "large": { + "validation": 0.8060299388572634, + "test": 0.7447182599325072 + }, + "learned_minus_confidence_test": 0.02317443212432302, + "learned_minus_small_test": 0.07799109548251715, + "learned_minus_large_test": 0.0027847885886056467, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_instancevg_Qwen3.8" + }, + { + "key": [ + "nlvr2", + "nlvr2", + "MiniCPM" + ], + "expert_display": "BEiT3", + "run": "five_seed_nlvr2_MiniCPM_four_difference", + "validation_rows": 6982, + "test_rows": 13937, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/nlvr2/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 86, + "validation_area_0_25": 0.8284302492122602, + "validation_point": { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8587797192781438, + "metric": 0.9004583213978803, + "rescued": 1073, + "harmed": 359, + "missed_rescue": 14, + "avoided_harm": 24, + "selected_both_correct": 4254, + "selected_both_wrong": 310 + }, + "test_point": { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8580038745784602, + "metric": 0.9077276314845376, + "rescued": 2191, + "harmed": 690, + "missed_rescue": 34, + "avoided_harm": 39, + "selected_both_correct": 8533, + "selected_both_wrong": 544 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7981953594958464, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1087, + "avoided_harm": 383, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5245773792266846, + "operator": "gt", + "actual_fraction": 0.0097393297049556, + "metric": 0.8009166427957606, + "rescued": 26, + "harmed": 7, + "missed_rescue": 1061, + "avoided_harm": 376, + "selected_both_correct": 33, + "selected_both_wrong": 2 + }, + { + "budget": 0.02, + "threshold": -0.5456489324569702, + "operator": "gt", + "actual_fraction": 0.019765110283586365, + "metric": 0.8037811515325122, + "rescued": 51, + "harmed": 12, + "missed_rescue": 1036, + "avoided_harm": 371, + "selected_both_correct": 63, + "selected_both_wrong": 12 + }, + { + "budget": 0.03, + "threshold": -0.5652987360954285, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.8066456602692639, + "rescued": 80, + "harmed": 21, + "missed_rescue": 1007, + "avoided_harm": 362, + "selected_both_correct": 86, + "selected_both_wrong": 22 + }, + { + "budget": 0.04, + "threshold": -0.5851011276245117, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.8083643655113149, + "rescued": 101, + "harmed": 30, + "missed_rescue": 986, + "avoided_harm": 353, + "selected_both_correct": 118, + "selected_both_wrong": 30 + }, + { + "budget": 0.05, + "threshold": -0.6055658459663391, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.809653394442853, + "rescued": 121, + "harmed": 41, + "missed_rescue": 966, + "avoided_harm": 342, + "selected_both_correct": 151, + "selected_both_wrong": 36 + }, + { + "budget": 0.06, + "threshold": -0.6293190717697144, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.8125179031796047, + "rescued": 145, + "harmed": 45, + "missed_rescue": 942, + "avoided_harm": 338, + "selected_both_correct": 183, + "selected_both_wrong": 45 + }, + { + "budget": 0.07, + "threshold": -0.6477982401847839, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.8168146662847322, + "rescued": 177, + "harmed": 47, + "missed_rescue": 910, + "avoided_harm": 336, + "selected_both_correct": 212, + "selected_both_wrong": 52 + }, + { + "budget": 0.08, + "threshold": -0.6693127155303955, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.8188198224004584, + "rescued": 201, + "harmed": 57, + "missed_rescue": 886, + "avoided_harm": 326, + "selected_both_correct": 239, + "selected_both_wrong": 61 + }, + { + "budget": 0.09, + "threshold": -0.6888871788978577, + "operator": "gt", + "actual_fraction": 0.08951589802348897, + "metric": 0.8205385276425093, + "rescued": 222, + "harmed": 66, + "missed_rescue": 865, + "avoided_harm": 317, + "selected_both_correct": 267, + "selected_both_wrong": 70 + }, + { + "budget": 0.1, + "threshold": -0.7061904072761536, + "operator": "gt", + "actual_fraction": 0.09925522772844457, + "metric": 0.8222572328845603, + "rescued": 240, + "harmed": 72, + "missed_rescue": 847, + "avoided_harm": 311, + "selected_both_correct": 305, + "selected_both_wrong": 76 + }, + { + "budget": 0.11, + "threshold": -0.7233095169067383, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.8254081924949871, + "rescued": 270, + "harmed": 80, + "missed_rescue": 817, + "avoided_harm": 303, + "selected_both_correct": 339, + "selected_both_wrong": 79 + }, + { + "budget": 0.12, + "threshold": -0.7424216866493225, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.8282727012317388, + "rescued": 296, + "harmed": 86, + "missed_rescue": 791, + "avoided_harm": 297, + "selected_both_correct": 369, + "selected_both_wrong": 86 + }, + { + "budget": 0.13, + "threshold": -0.7627776861190796, + "operator": "gt", + "actual_fraction": 0.1297622457748496, + "metric": 0.8308507590948152, + "rescued": 320, + "harmed": 92, + "missed_rescue": 767, + "avoided_harm": 291, + "selected_both_correct": 404, + "selected_both_wrong": 90 + }, + { + "budget": 0.14, + "threshold": -0.7786493301391602, + "operator": "gt", + "actual_fraction": 0.1396448009166428, + "metric": 0.8337152678315669, + "rescued": 346, + "harmed": 98, + "missed_rescue": 741, + "avoided_harm": 285, + "selected_both_correct": 436, + "selected_both_wrong": 95 + }, + { + "budget": 0.15, + "threshold": -0.7950222492218018, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8362933256946433, + "rescued": 367, + "harmed": 101, + "missed_rescue": 720, + "avoided_harm": 282, + "selected_both_correct": 474, + "selected_both_wrong": 105 + }, + { + "budget": 0.16, + "threshold": -0.8140527606010437, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8394442853050702, + "rescued": 392, + "harmed": 104, + "missed_rescue": 695, + "avoided_harm": 279, + "selected_both_correct": 513, + "selected_both_wrong": 108 + }, + { + "budget": 0.17, + "threshold": -0.8327813148498535, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.840876539673446, + "rescued": 409, + "harmed": 111, + "missed_rescue": 678, + "avoided_harm": 272, + "selected_both_correct": 552, + "selected_both_wrong": 114 + }, + { + "budget": 0.18, + "threshold": -0.8464508652687073, + "operator": "gt", + "actual_fraction": 0.17960469779432828, + "metric": 0.8428816957891722, + "rescued": 426, + "harmed": 114, + "missed_rescue": 661, + "avoided_harm": 269, + "selected_both_correct": 596, + "selected_both_wrong": 118 + }, + { + "budget": 0.19, + "threshold": -0.858244001865387, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8451733027785735, + "rescued": 449, + "harmed": 121, + "missed_rescue": 638, + "avoided_harm": 262, + "selected_both_correct": 631, + "selected_both_wrong": 125 + }, + { + "budget": 0.2, + "threshold": -0.8701565861701965, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8467487825837868, + "rescued": 467, + "harmed": 128, + "missed_rescue": 620, + "avoided_harm": 255, + "selected_both_correct": 672, + "selected_both_wrong": 129 + }, + { + "budget": 0.21, + "threshold": -0.8840392827987671, + "operator": "gt", + "actual_fraction": 0.2093955886565454, + "metric": 0.8481810369521627, + "rescued": 483, + "harmed": 134, + "missed_rescue": 604, + "avoided_harm": 249, + "selected_both_correct": 712, + "selected_both_wrong": 133 + }, + { + "budget": 0.22, + "threshold": -0.8933094143867493, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8497565167573761, + "rescued": 500, + "harmed": 140, + "missed_rescue": 587, + "avoided_harm": 243, + "selected_both_correct": 759, + "selected_both_wrong": 137 + }, + { + "budget": 0.23, + "threshold": -0.9046505093574524, + "operator": "gt", + "actual_fraction": 0.2297336006874821, + "metric": 0.851475221999427, + "rescued": 516, + "harmed": 144, + "missed_rescue": 571, + "avoided_harm": 239, + "selected_both_correct": 797, + "selected_both_wrong": 147 + }, + { + "budget": 0.24, + "threshold": -0.9136765599250793, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8543397307361787, + "rescued": 540, + "harmed": 148, + "missed_rescue": 547, + "avoided_harm": 235, + "selected_both_correct": 834, + "selected_both_wrong": 153 + }, + { + "budget": 0.25, + "threshold": -0.9230391979217529, + "operator": "gt", + "actual_fraction": 0.24906903466055572, + "metric": 0.8562016614150673, + "rescued": 555, + "harmed": 150, + "missed_rescue": 532, + "avoided_harm": 233, + "selected_both_correct": 874, + "selected_both_wrong": 160 + }, + { + "budget": 0.26, + "threshold": -0.9314625263214111, + "operator": "gt", + "actual_fraction": 0.2595244915496992, + "metric": 0.8589229447149814, + "rescued": 581, + "harmed": 157, + "missed_rescue": 506, + "avoided_harm": 226, + "selected_both_correct": 913, + "selected_both_wrong": 161 + }, + { + "budget": 0.27, + "threshold": -0.9376689791679382, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8607848753938699, + "rescued": 601, + "harmed": 164, + "missed_rescue": 486, + "avoided_harm": 219, + "selected_both_correct": 957, + "selected_both_wrong": 163 + }, + { + "budget": 0.28, + "threshold": -0.9449946880340576, + "operator": "gt", + "actual_fraction": 0.2794328272701232, + "metric": 0.8625035806359209, + "rescued": 616, + "harmed": 167, + "missed_rescue": 471, + "avoided_harm": 216, + "selected_both_correct": 1000, + "selected_both_wrong": 168 + }, + { + "budget": 0.29, + "threshold": -0.9511421918869019, + "operator": "gt", + "actual_fraction": 0.2896018332855915, + "metric": 0.8642222858779719, + "rescued": 633, + "harmed": 172, + "missed_rescue": 454, + "avoided_harm": 211, + "selected_both_correct": 1041, + "selected_both_wrong": 176 + }, + { + "budget": 0.3, + "threshold": -0.9559813141822815, + "operator": "gt", + "actual_fraction": 0.29977083930105985, + "metric": 0.8660842165568605, + "rescued": 652, + "harmed": 178, + "missed_rescue": 435, + "avoided_harm": 205, + "selected_both_correct": 1082, + "selected_both_wrong": 181 + }, + { + "budget": 0.31, + "threshold": -0.9597620368003845, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8672300200515611, + "rescued": 664, + "harmed": 182, + "missed_rescue": 423, + "avoided_harm": 201, + "selected_both_correct": 1131, + "selected_both_wrong": 187 + }, + { + "budget": 0.32, + "threshold": -0.9643214344978333, + "operator": "gt", + "actual_fraction": 0.3196791750214838, + "metric": 0.8692351761672873, + "rescued": 682, + "harmed": 186, + "missed_rescue": 405, + "avoided_harm": 197, + "selected_both_correct": 1172, + "selected_both_wrong": 192 + }, + { + "budget": 0.33, + "threshold": -0.9686195850372314, + "operator": "gt", + "actual_fraction": 0.3297049556001146, + "metric": 0.8695216270409625, + "rescued": 693, + "harmed": 195, + "missed_rescue": 394, + "avoided_harm": 188, + "selected_both_correct": 1217, + "selected_both_wrong": 197 + }, + { + "budget": 0.34, + "threshold": -0.9719924926757812, + "operator": "gt", + "actual_fraction": 0.339157834431395, + "metric": 0.8712403322830135, + "rescued": 709, + "harmed": 199, + "missed_rescue": 378, + "avoided_harm": 184, + "selected_both_correct": 1257, + "selected_both_wrong": 203 + }, + { + "budget": 0.35, + "threshold": -0.974821150302887, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.872099684904039, + "rescued": 722, + "harmed": 206, + "missed_rescue": 365, + "avoided_harm": 177, + "selected_both_correct": 1305, + "selected_both_wrong": 210 + }, + { + "budget": 0.36, + "threshold": -0.9777138829231262, + "operator": "gt", + "actual_fraction": 0.3587797192781438, + "metric": 0.8739616155829275, + "rescued": 739, + "harmed": 210, + "missed_rescue": 348, + "avoided_harm": 173, + "selected_both_correct": 1340, + "selected_both_wrong": 216 + }, + { + "budget": 0.37, + "threshold": -0.9802808165550232, + "operator": "gt", + "actual_fraction": 0.3685190489830994, + "metric": 0.8746777427671154, + "rescued": 749, + "harmed": 215, + "missed_rescue": 338, + "avoided_harm": 168, + "selected_both_correct": 1389, + "selected_both_wrong": 220 + }, + { + "budget": 0.38, + "threshold": -0.982557475566864, + "operator": "gt", + "actual_fraction": 0.3795474076195932, + "metric": 0.8771125751933544, + "rescued": 769, + "harmed": 218, + "missed_rescue": 318, + "avoided_harm": 165, + "selected_both_correct": 1439, + "selected_both_wrong": 224 + }, + { + "budget": 0.39, + "threshold": -0.9843363761901855, + "operator": "gt", + "actual_fraction": 0.3892867373245488, + "metric": 0.8781151532512174, + "rescued": 780, + "harmed": 222, + "missed_rescue": 307, + "avoided_harm": 161, + "selected_both_correct": 1488, + "selected_both_wrong": 228 + }, + { + "budget": 0.4, + "threshold": -0.9863631129264832, + "operator": "gt", + "actual_fraction": 0.3993125179031796, + "metric": 0.8795474076195933, + "rescued": 796, + "harmed": 228, + "missed_rescue": 291, + "avoided_harm": 155, + "selected_both_correct": 1535, + "selected_both_wrong": 229 + }, + { + "budget": 0.41, + "threshold": -0.9879462122917175, + "operator": "gt", + "actual_fraction": 0.40962474935548554, + "metric": 0.8811228874248066, + "rescued": 812, + "harmed": 233, + "missed_rescue": 275, + "avoided_harm": 150, + "selected_both_correct": 1583, + "selected_both_wrong": 232 + }, + { + "budget": 0.42, + "threshold": -0.9890130758285522, + "operator": "gt", + "actual_fraction": 0.41850472643941566, + "metric": 0.8825551417931825, + "rescued": 826, + "harmed": 237, + "missed_rescue": 261, + "avoided_harm": 146, + "selected_both_correct": 1624, + "selected_both_wrong": 235 + }, + { + "budget": 0.43, + "threshold": -0.9899863600730896, + "operator": "gt", + "actual_fraction": 0.4270982526496706, + "metric": 0.8832712689773704, + "rescued": 834, + "harmed": 240, + "missed_rescue": 253, + "avoided_harm": 143, + "selected_both_correct": 1671, + "selected_both_wrong": 237 + }, + { + "budget": 0.44, + "threshold": -0.9911527037620544, + "operator": "gt", + "actual_fraction": 0.4395588656545402, + "metric": 0.8864222285877972, + "rescued": 859, + "harmed": 243, + "missed_rescue": 228, + "avoided_harm": 140, + "selected_both_correct": 1727, + "selected_both_wrong": 240 + }, + { + "budget": 0.45, + "threshold": -0.9919379353523254, + "operator": "gt", + "actual_fraction": 0.4497278716700086, + "metric": 0.8865654540246348, + "rescued": 866, + "harmed": 249, + "missed_rescue": 221, + "avoided_harm": 134, + "selected_both_correct": 1781, + "selected_both_wrong": 244 + }, + { + "budget": 0.46, + "threshold": -0.9926542043685913, + "operator": "gt", + "actual_fraction": 0.4586078487539387, + "metric": 0.8877112575193354, + "rescued": 876, + "harmed": 251, + "missed_rescue": 211, + "avoided_harm": 132, + "selected_both_correct": 1828, + "selected_both_wrong": 247 + }, + { + "budget": 0.47, + "threshold": -0.9933071732521057, + "operator": "gt", + "actual_fraction": 0.46863362933256947, + "metric": 0.888570610140361, + "rescued": 886, + "harmed": 255, + "missed_rescue": 201, + "avoided_harm": 128, + "selected_both_correct": 1882, + "selected_both_wrong": 249 + }, + { + "budget": 0.48, + "threshold": -0.9939024448394775, + "operator": "gt", + "actual_fraction": 0.4798052134059009, + "metric": 0.8882841592666858, + "rescued": 893, + "harmed": 264, + "missed_rescue": 194, + "avoided_harm": 119, + "selected_both_correct": 1942, + "selected_both_wrong": 251 + }, + { + "budget": 0.49, + "threshold": -0.9944451451301575, + "operator": "gt", + "actual_fraction": 0.4891148668003437, + "metric": 0.8897164136350616, + "rescued": 908, + "harmed": 269, + "missed_rescue": 179, + "avoided_harm": 114, + "selected_both_correct": 1986, + "selected_both_wrong": 252 + }, + { + "budget": 0.5, + "threshold": -0.9949396848678589, + "operator": "gt", + "actual_fraction": 0.4992838728158121, + "metric": 0.890289315382412, + "rescued": 915, + "harmed": 272, + "missed_rescue": 172, + "avoided_harm": 111, + "selected_both_correct": 2043, + "selected_both_wrong": 256 + }, + { + "budget": 0.51, + "threshold": -0.9952448010444641, + "operator": "gt", + "actual_fraction": 0.5065883700945288, + "metric": 0.890289315382412, + "rescued": 918, + "harmed": 275, + "missed_rescue": 169, + "avoided_harm": 108, + "selected_both_correct": 2085, + "selected_both_wrong": 259 + }, + { + "budget": 0.52, + "threshold": -0.9956685304641724, + "operator": "gt", + "actual_fraction": 0.5179031796046978, + "metric": 0.890575766256087, + "rescued": 923, + "harmed": 278, + "missed_rescue": 164, + "avoided_harm": 105, + "selected_both_correct": 2153, + "selected_both_wrong": 262 + }, + { + "budget": 0.53, + "threshold": -0.9961755275726318, + "operator": "gt", + "actual_fraction": 0.528788312804354, + "metric": 0.8911486680034374, + "rescued": 931, + "harmed": 282, + "missed_rescue": 156, + "avoided_harm": 101, + "selected_both_correct": 2215, + "selected_both_wrong": 264 + }, + { + "budget": 0.54, + "threshold": -0.9965165853500366, + "operator": "gt", + "actual_fraction": 0.5380979661987969, + "metric": 0.8921512460613005, + "rescued": 939, + "harmed": 283, + "missed_rescue": 148, + "avoided_harm": 100, + "selected_both_correct": 2270, + "selected_both_wrong": 265 + }, + { + "budget": 0.55, + "threshold": -0.9968273043632507, + "operator": "gt", + "actual_fraction": 0.5489830993984531, + "metric": 0.8920080206244629, + "rescued": 943, + "harmed": 288, + "missed_rescue": 144, + "avoided_harm": 95, + "selected_both_correct": 2337, + "selected_both_wrong": 265 + }, + { + "budget": 0.56, + "threshold": -0.9971103668212891, + "operator": "gt", + "actual_fraction": 0.5592953308507591, + "metric": 0.8922944714981381, + "rescued": 950, + "harmed": 293, + "missed_rescue": 137, + "avoided_harm": 90, + "selected_both_correct": 2393, + "selected_both_wrong": 269 + }, + { + "budget": 0.57, + "threshold": -0.997285008430481, + "operator": "gt", + "actual_fraction": 0.5690346605557147, + "metric": 0.8925809223718132, + "rescued": 957, + "harmed": 298, + "missed_rescue": 130, + "avoided_harm": 85, + "selected_both_correct": 2445, + "selected_both_wrong": 273 + }, + { + "budget": 0.58, + "threshold": -0.9974491000175476, + "operator": "gt", + "actual_fraction": 0.5783443139501575, + "metric": 0.8928673732454884, + "rescued": 961, + "harmed": 300, + "missed_rescue": 126, + "avoided_harm": 83, + "selected_both_correct": 2499, + "selected_both_wrong": 278 + }, + { + "budget": 0.59, + "threshold": -0.9976769089698792, + "operator": "gt", + "actual_fraction": 0.5893726725866514, + "metric": 0.8927241478086508, + "rescued": 965, + "harmed": 305, + "missed_rescue": 122, + "avoided_harm": 78, + "selected_both_correct": 2565, + "selected_both_wrong": 280 + }, + { + "budget": 0.6, + "threshold": -0.9978172779083252, + "operator": "gt", + "actual_fraction": 0.5976797479232312, + "metric": 0.8935835004296763, + "rescued": 972, + "harmed": 306, + "missed_rescue": 115, + "avoided_harm": 77, + "selected_both_correct": 2614, + "selected_both_wrong": 281 + }, + { + "budget": 0.61, + "threshold": -0.9980121850967407, + "operator": "gt", + "actual_fraction": 0.6084216556860499, + "metric": 0.8944428530507018, + "rescued": 980, + "harmed": 308, + "missed_rescue": 107, + "avoided_harm": 75, + "selected_both_correct": 2678, + "selected_both_wrong": 282 + }, + { + "budget": 0.62, + "threshold": -0.9981897473335266, + "operator": "gt", + "actual_fraction": 0.6187338871383558, + "metric": 0.8941564021770266, + "rescued": 982, + "harmed": 312, + "missed_rescue": 105, + "avoided_harm": 71, + "selected_both_correct": 2743, + "selected_both_wrong": 283 + }, + { + "budget": 0.63, + "threshold": -0.9982993006706238, + "operator": "gt", + "actual_fraction": 0.6280435405327986, + "metric": 0.8954454311085649, + "rescued": 992, + "harmed": 313, + "missed_rescue": 95, + "avoided_harm": 70, + "selected_both_correct": 2796, + "selected_both_wrong": 284 + }, + { + "budget": 0.64, + "threshold": -0.9984021782875061, + "operator": "gt", + "actual_fraction": 0.6360641649957033, + "metric": 0.8958751074190776, + "rescued": 997, + "harmed": 315, + "missed_rescue": 90, + "avoided_harm": 68, + "selected_both_correct": 2843, + "selected_both_wrong": 286 + }, + { + "budget": 0.65, + "threshold": -0.9985449314117432, + "operator": "gt", + "actual_fraction": 0.6465196218848468, + "metric": 0.8961615582927528, + "rescued": 1000, + "harmed": 316, + "missed_rescue": 87, + "avoided_harm": 67, + "selected_both_correct": 2912, + "selected_both_wrong": 286 + }, + { + "budget": 0.66, + "threshold": -0.9986749291419983, + "operator": "gt", + "actual_fraction": 0.6586937840160413, + "metric": 0.8971641363506159, + "rescued": 1011, + "harmed": 320, + "missed_rescue": 76, + "avoided_harm": 63, + "selected_both_correct": 2982, + "selected_both_wrong": 286 + }, + { + "budget": 0.67, + "threshold": -0.9987550973892212, + "operator": "gt", + "actual_fraction": 0.6672873102262962, + "metric": 0.8973073617874534, + "rescued": 1015, + "harmed": 323, + "missed_rescue": 72, + "avoided_harm": 60, + "selected_both_correct": 3034, + "selected_both_wrong": 287 + }, + { + "budget": 0.68, + "threshold": -0.9988664388656616, + "operator": "gt", + "actual_fraction": 0.6793182469206531, + "metric": 0.8977370380979662, + "rescued": 1023, + "harmed": 328, + "missed_rescue": 64, + "avoided_harm": 55, + "selected_both_correct": 3102, + "selected_both_wrong": 290 + }, + { + "budget": 0.69, + "threshold": -0.9989677667617798, + "operator": "gt", + "actual_fraction": 0.6894872529361215, + "metric": 0.8984531652821541, + "rescued": 1028, + "harmed": 328, + "missed_rescue": 59, + "avoided_harm": 55, + "selected_both_correct": 3166, + "selected_both_wrong": 292 + }, + { + "budget": 0.7, + "threshold": -0.9990302324295044, + "operator": "gt", + "actual_fraction": 0.6969349756516757, + "metric": 0.8975938126611286, + "rescued": 1028, + "harmed": 334, + "missed_rescue": 59, + "avoided_harm": 49, + "selected_both_correct": 3212, + "selected_both_wrong": 292 + }, + { + "budget": 0.71, + "threshold": -0.9991169571876526, + "operator": "gt", + "actual_fraction": 0.7098252649670581, + "metric": 0.8977370380979662, + "rescued": 1031, + "harmed": 336, + "missed_rescue": 56, + "avoided_harm": 47, + "selected_both_correct": 3292, + "selected_both_wrong": 297 + }, + { + "budget": 0.72, + "threshold": -0.9991704225540161, + "operator": "gt", + "actual_fraction": 0.7189916929246635, + "metric": 0.8978802635348038, + "rescued": 1034, + "harmed": 338, + "missed_rescue": 53, + "avoided_harm": 45, + "selected_both_correct": 3351, + "selected_both_wrong": 297 + }, + { + "budget": 0.73, + "threshold": -0.9992445707321167, + "operator": "gt", + "actual_fraction": 0.7295903752506445, + "metric": 0.898166714408479, + "rescued": 1038, + "harmed": 340, + "missed_rescue": 49, + "avoided_harm": 43, + "selected_both_correct": 3419, + "selected_both_wrong": 297 + }, + { + "budget": 0.74, + "threshold": -0.9992903470993042, + "operator": "gt", + "actual_fraction": 0.7384703523345746, + "metric": 0.8985963907189917, + "rescued": 1044, + "harmed": 343, + "missed_rescue": 43, + "avoided_harm": 40, + "selected_both_correct": 3471, + "selected_both_wrong": 298 + }, + { + "budget": 0.75, + "threshold": -0.9993333220481873, + "operator": "gt", + "actual_fraction": 0.7484961329132054, + "metric": 0.8988828415926668, + "rescued": 1049, + "harmed": 346, + "missed_rescue": 38, + "avoided_harm": 37, + "selected_both_correct": 3533, + "selected_both_wrong": 298 + }, + { + "budget": 0.76, + "threshold": -0.999373733997345, + "operator": "gt", + "actual_fraction": 0.7553709538814093, + "metric": 0.8987396161558293, + "rescued": 1050, + "harmed": 348, + "missed_rescue": 37, + "avoided_harm": 35, + "selected_both_correct": 3578, + "selected_both_wrong": 298 + }, + { + "budget": 0.77, + "threshold": -0.9994296431541443, + "operator": "gt", + "actual_fraction": 0.7694070466914924, + "metric": 0.8994557433400172, + "rescued": 1056, + "harmed": 349, + "missed_rescue": 31, + "avoided_harm": 34, + "selected_both_correct": 3667, + "selected_both_wrong": 300 + }, + { + "budget": 0.78, + "threshold": -0.999464213848114, + "operator": "gt", + "actual_fraction": 0.7782870237754225, + "metric": 0.8995989687768547, + "rescued": 1058, + "harmed": 350, + "missed_rescue": 29, + "avoided_harm": 33, + "selected_both_correct": 3723, + "selected_both_wrong": 303 + }, + { + "budget": 0.79, + "threshold": -0.9994966983795166, + "operator": "gt", + "actual_fraction": 0.7885992552277284, + "metric": 0.8998854196505299, + "rescued": 1060, + "harmed": 350, + "missed_rescue": 27, + "avoided_harm": 33, + "selected_both_correct": 3792, + "selected_both_wrong": 304 + }, + { + "budget": 0.8, + "threshold": -0.9995417594909668, + "operator": "gt", + "actual_fraction": 0.7977656831853337, + "metric": 0.8997421942136924, + "rescued": 1061, + "harmed": 352, + "missed_rescue": 26, + "avoided_harm": 31, + "selected_both_correct": 3852, + "selected_both_wrong": 305 + }, + { + "budget": 0.81, + "threshold": -0.9995827078819275, + "operator": "gt", + "actual_fraction": 0.8090804926955028, + "metric": 0.9001718705242051, + "rescued": 1064, + "harmed": 352, + "missed_rescue": 23, + "avoided_harm": 31, + "selected_both_correct": 3928, + "selected_both_wrong": 305 + }, + { + "budget": 0.82, + "threshold": -0.9996140599250793, + "operator": "gt", + "actual_fraction": 0.8198224004583214, + "metric": 0.9003150959610426, + "rescued": 1066, + "harmed": 353, + "missed_rescue": 21, + "avoided_harm": 30, + "selected_both_correct": 3998, + "selected_both_wrong": 307 + }, + { + "budget": 0.83, + "threshold": -0.9996429681777954, + "operator": "gt", + "actual_fraction": 0.8281294757949011, + "metric": 0.9001718705242051, + "rescued": 1066, + "harmed": 354, + "missed_rescue": 21, + "avoided_harm": 29, + "selected_both_correct": 4054, + "selected_both_wrong": 308 + }, + { + "budget": 0.84, + "threshold": -0.9996750354766846, + "operator": "gt", + "actual_fraction": 0.8395875107419077, + "metric": 0.9003150959610426, + "rescued": 1070, + "harmed": 357, + "missed_rescue": 17, + "avoided_harm": 26, + "selected_both_correct": 4127, + "selected_both_wrong": 308 + }, + { + "budget": 0.85, + "threshold": -0.9997040629386902, + "operator": "gt", + "actual_fraction": 0.8458894299627614, + "metric": 0.9003150959610426, + "rescued": 1071, + "harmed": 358, + "missed_rescue": 16, + "avoided_harm": 25, + "selected_both_correct": 4167, + "selected_both_wrong": 310 + }, + { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8587797192781438, + "metric": 0.9004583213978803, + "rescued": 1073, + "harmed": 359, + "missed_rescue": 14, + "avoided_harm": 24, + "selected_both_correct": 4254, + "selected_both_wrong": 310 + }, + { + "budget": 0.87, + "threshold": -0.9997695088386536, + "operator": "gt", + "actual_fraction": 0.8693784016041249, + "metric": 0.9000286450873675, + "rescued": 1073, + "harmed": 362, + "missed_rescue": 14, + "avoided_harm": 21, + "selected_both_correct": 4323, + "selected_both_wrong": 312 + }, + { + "budget": 0.88, + "threshold": -0.999783456325531, + "operator": "gt", + "actual_fraction": 0.8749641936407906, + "metric": 0.8998854196505299, + "rescued": 1073, + "harmed": 363, + "missed_rescue": 14, + "avoided_harm": 20, + "selected_both_correct": 4361, + "selected_both_wrong": 312 + }, + { + "budget": 0.89, + "threshold": -0.9998205304145813, + "operator": "gt", + "actual_fraction": 0.8894299627613864, + "metric": 0.8994557433400172, + "rescued": 1074, + "harmed": 367, + "missed_rescue": 13, + "avoided_harm": 16, + "selected_both_correct": 4456, + "selected_both_wrong": 313 + }, + { + "budget": 0.9, + "threshold": -0.999841570854187, + "operator": "gt", + "actual_fraction": 0.8988828415926668, + "metric": 0.8993125179031796, + "rescued": 1075, + "harmed": 369, + "missed_rescue": 12, + "avoided_harm": 14, + "selected_both_correct": 4517, + "selected_both_wrong": 315 + }, + { + "budget": 0.91, + "threshold": -0.9998601675033569, + "operator": "gt", + "actual_fraction": 0.9093382984818104, + "metric": 0.8994557433400172, + "rescued": 1077, + "harmed": 370, + "missed_rescue": 10, + "avoided_harm": 13, + "selected_both_correct": 4586, + "selected_both_wrong": 316 + }, + { + "budget": 0.92, + "threshold": -0.9998766183853149, + "operator": "gt", + "actual_fraction": 0.9185047264394156, + "metric": 0.8993125179031796, + "rescued": 1077, + "harmed": 371, + "missed_rescue": 10, + "avoided_harm": 12, + "selected_both_correct": 4646, + "selected_both_wrong": 319 + }, + { + "budget": 0.93, + "threshold": -0.9998977184295654, + "operator": "gt", + "actual_fraction": 0.9276711543970209, + "metric": 0.899169292466342, + "rescued": 1078, + "harmed": 373, + "missed_rescue": 9, + "avoided_harm": 10, + "selected_both_correct": 4707, + "selected_both_wrong": 319 + }, + { + "budget": 0.94, + "threshold": -0.9999151229858398, + "operator": "gt", + "actual_fraction": 0.9394156402177026, + "metric": 0.8993125179031796, + "rescued": 1081, + "harmed": 375, + "missed_rescue": 6, + "avoided_harm": 8, + "selected_both_correct": 4784, + "selected_both_wrong": 319 + }, + { + "budget": 0.95, + "threshold": -0.999931812286377, + "operator": "gt", + "actual_fraction": 0.9497278716700086, + "metric": 0.8993125179031796, + "rescued": 1082, + "harmed": 376, + "missed_rescue": 5, + "avoided_harm": 7, + "selected_both_correct": 4854, + "selected_both_wrong": 319 + }, + { + "budget": 0.96, + "threshold": -0.9999452829360962, + "operator": "gt", + "actual_fraction": 0.9590375250644515, + "metric": 0.8995989687768547, + "rescued": 1085, + "harmed": 377, + "missed_rescue": 2, + "avoided_harm": 6, + "selected_both_correct": 4914, + "selected_both_wrong": 320 + }, + { + "budget": 0.97, + "threshold": -0.9999599456787109, + "operator": "gt", + "actual_fraction": 0.969492981953595, + "metric": 0.8994557433400172, + "rescued": 1085, + "harmed": 378, + "missed_rescue": 2, + "avoided_harm": 5, + "selected_both_correct": 4986, + "selected_both_wrong": 320 + }, + { + "budget": 0.98, + "threshold": -0.9999687671661377, + "operator": "gt", + "actual_fraction": 0.9783729590375251, + "metric": 0.899169292466342, + "rescued": 1085, + "harmed": 380, + "missed_rescue": 2, + "avoided_harm": 3, + "selected_both_correct": 5046, + "selected_both_wrong": 320 + }, + { + "budget": 0.99, + "threshold": -0.9999842643737793, + "operator": "gt", + "actual_fraction": 0.9898309939845317, + "metric": 0.899169292466342, + "rescued": 1087, + "harmed": 382, + "missed_rescue": 0, + "avoided_harm": 1, + "selected_both_correct": 5122, + "selected_both_wrong": 320 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8990260670295045, + "rescued": 1087, + "harmed": 383, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 5190, + "selected_both_wrong": 322 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.8000287005811868, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2225, + "avoided_harm": 729, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5245773792266846, + "operator": "gt", + "actual_fraction": 0.011623735380641457, + "metric": 0.8034727703235991, + "rescued": 65, + "harmed": 17, + "missed_rescue": 2160, + "avoided_harm": 712, + "selected_both_correct": 65, + "selected_both_wrong": 15 + }, + { + "budget": 0.02, + "threshold": -0.5456489324569702, + "operator": "gt", + "actual_fraction": 0.021453684437109852, + "metric": 0.8064145798952429, + "rescued": 115, + "harmed": 26, + "missed_rescue": 2110, + "avoided_harm": 703, + "selected_both_correct": 126, + "selected_both_wrong": 32 + }, + { + "budget": 0.03, + "threshold": -0.5652987360954285, + "operator": "ge", + "actual_fraction": 0.02970510152830595, + "metric": 0.8094281409198536, + "rescued": 165, + "harmed": 34, + "missed_rescue": 2060, + "avoided_harm": 695, + "selected_both_correct": 174, + "selected_both_wrong": 41 + }, + { + "budget": 0.04, + "threshold": -0.5851011276245117, + "operator": "ge", + "actual_fraction": 0.03795651861950204, + "metric": 0.8113654301499605, + "rescued": 204, + "harmed": 46, + "missed_rescue": 2021, + "avoided_harm": 683, + "selected_both_correct": 229, + "selected_both_wrong": 50 + }, + { + "budget": 0.05, + "threshold": -0.6055658459663391, + "operator": "ge", + "actual_fraction": 0.047212456052235056, + "metric": 0.8140202339097367, + "rescued": 253, + "harmed": 58, + "missed_rescue": 1972, + "avoided_harm": 671, + "selected_both_correct": 289, + "selected_both_wrong": 58 + }, + { + "budget": 0.06, + "threshold": -0.6293190717697144, + "operator": "ge", + "actual_fraction": 0.057759919638372674, + "metric": 0.817464303652149, + "rescued": 312, + "harmed": 69, + "missed_rescue": 1913, + "avoided_harm": 660, + "selected_both_correct": 351, + "selected_both_wrong": 73 + }, + { + "budget": 0.07, + "threshold": -0.6477982401847839, + "operator": "ge", + "actual_fraction": 0.06601133672956877, + "metric": 0.8196885986941236, + "rescued": 356, + "harmed": 82, + "missed_rescue": 1869, + "avoided_harm": 647, + "selected_both_correct": 401, + "selected_both_wrong": 81 + }, + { + "budget": 0.08, + "threshold": -0.6693127155303955, + "operator": "ge", + "actual_fraction": 0.07483676544450026, + "metric": 0.8226304082657674, + "rescued": 406, + "harmed": 91, + "missed_rescue": 1819, + "avoided_harm": 638, + "selected_both_correct": 453, + "selected_both_wrong": 93 + }, + { + "budget": 0.09, + "threshold": -0.6888871788978577, + "operator": "gt", + "actual_fraction": 0.08488196885986941, + "metric": 0.8254287149314774, + "rescued": 455, + "harmed": 101, + "missed_rescue": 1770, + "avoided_harm": 628, + "selected_both_correct": 523, + "selected_both_wrong": 104 + }, + { + "budget": 0.1, + "threshold": -0.7061904072761536, + "operator": "gt", + "actual_fraction": 0.09356389466886704, + "metric": 0.8282987730501543, + "rescued": 503, + "harmed": 109, + "missed_rescue": 1722, + "avoided_harm": 620, + "selected_both_correct": 576, + "selected_both_wrong": 116 + }, + { + "budget": 0.11, + "threshold": -0.7233095169067383, + "operator": "ge", + "actual_fraction": 0.1026045777426993, + "metric": 0.8310970797158642, + "rescued": 551, + "harmed": 118, + "missed_rescue": 1674, + "avoided_harm": 611, + "selected_both_correct": 638, + "selected_both_wrong": 123 + }, + { + "budget": 0.12, + "threshold": -0.7424216866493225, + "operator": "ge", + "actual_fraction": 0.11322379278180383, + "metric": 0.8338953863815742, + "rescued": 603, + "harmed": 131, + "missed_rescue": 1622, + "avoided_harm": 598, + "selected_both_correct": 708, + "selected_both_wrong": 136 + }, + { + "budget": 0.13, + "threshold": -0.7627776861190796, + "operator": "gt", + "actual_fraction": 0.12255148166750376, + "metric": 0.8363349357824496, + "rescued": 646, + "harmed": 140, + "missed_rescue": 1579, + "avoided_harm": 589, + "selected_both_correct": 776, + "selected_both_wrong": 146 + }, + { + "budget": 0.14, + "threshold": -0.7786493301391602, + "operator": "gt", + "actual_fraction": 0.13137691038243524, + "metric": 0.8390614909951927, + "rescued": 693, + "harmed": 149, + "missed_rescue": 1532, + "avoided_harm": 580, + "selected_both_correct": 832, + "selected_both_wrong": 157 + }, + { + "budget": 0.15, + "threshold": -0.7950222492218018, + "operator": "ge", + "actual_fraction": 0.14056109636220132, + "metric": 0.8419315491138696, + "rescued": 743, + "harmed": 159, + "missed_rescue": 1482, + "avoided_harm": 570, + "selected_both_correct": 891, + "selected_both_wrong": 166 + }, + { + "budget": 0.16, + "threshold": -0.8140527606010437, + "operator": "ge", + "actual_fraction": 0.15096505704240512, + "metric": 0.8448016072325465, + "rescued": 793, + "harmed": 169, + "missed_rescue": 1432, + "avoided_harm": 560, + "selected_both_correct": 965, + "selected_both_wrong": 177 + }, + { + "budget": 0.17, + "threshold": -0.8327813148498535, + "operator": "ge", + "actual_fraction": 0.1613690177226089, + "metric": 0.8465236421037526, + "rescued": 831, + "harmed": 183, + "missed_rescue": 1394, + "avoided_harm": 546, + "selected_both_correct": 1041, + "selected_both_wrong": 194 + }, + { + "budget": 0.18, + "threshold": -0.8464508652687073, + "operator": "gt", + "actual_fraction": 0.1702661978905073, + "metric": 0.8483891798808926, + "rescued": 871, + "harmed": 197, + "missed_rescue": 1354, + "avoided_harm": 532, + "selected_both_correct": 1104, + "selected_both_wrong": 201 + }, + { + "budget": 0.19, + "threshold": -0.858244001865387, + "operator": "ge", + "actual_fraction": 0.17916337805840568, + "metric": 0.8509722321877018, + "rescued": 915, + "harmed": 205, + "missed_rescue": 1310, + "avoided_harm": 524, + "selected_both_correct": 1164, + "selected_both_wrong": 213 + }, + { + "budget": 0.2, + "threshold": -0.8701565861701965, + "operator": "ge", + "actual_fraction": 0.18798880677333715, + "metric": 0.8528377699648418, + "rescued": 954, + "harmed": 218, + "missed_rescue": 1271, + "avoided_harm": 511, + "selected_both_correct": 1230, + "selected_both_wrong": 218 + }, + { + "budget": 0.21, + "threshold": -0.8840392827987671, + "operator": "gt", + "actual_fraction": 0.19659898112936788, + "metric": 0.8549903135538495, + "rescued": 992, + "harmed": 226, + "missed_rescue": 1233, + "avoided_harm": 503, + "selected_both_correct": 1299, + "selected_both_wrong": 223 + }, + { + "budget": 0.22, + "threshold": -0.8933094143867493, + "operator": "ge", + "actual_fraction": 0.20642893018583627, + "metric": 0.8574298629547248, + "rescued": 1035, + "harmed": 235, + "missed_rescue": 1190, + "avoided_harm": 494, + "selected_both_correct": 1377, + "selected_both_wrong": 230 + }, + { + "budget": 0.23, + "threshold": -0.9046505093574524, + "operator": "gt", + "actual_fraction": 0.21711989667790774, + "metric": 0.8599411638085671, + "rescued": 1079, + "harmed": 244, + "missed_rescue": 1146, + "avoided_harm": 485, + "selected_both_correct": 1461, + "selected_both_wrong": 242 + }, + { + "budget": 0.24, + "threshold": -0.9136765599250793, + "operator": "ge", + "actual_fraction": 0.22766736026404535, + "metric": 0.8620937073975748, + "rescued": 1120, + "harmed": 255, + "missed_rescue": 1105, + "avoided_harm": 474, + "selected_both_correct": 1539, + "selected_both_wrong": 259 + }, + { + "budget": 0.25, + "threshold": -0.9230391979217529, + "operator": "gt", + "actual_fraction": 0.23785606658534836, + "metric": 0.8648202626103179, + "rescued": 1170, + "harmed": 267, + "missed_rescue": 1055, + "avoided_harm": 462, + "selected_both_correct": 1612, + "selected_both_wrong": 266 + }, + { + "budget": 0.26, + "threshold": -0.9314625263214111, + "operator": "gt", + "actual_fraction": 0.24804477290665136, + "metric": 0.8671163091052594, + "rescued": 1211, + "harmed": 276, + "missed_rescue": 1014, + "avoided_harm": 453, + "selected_both_correct": 1693, + "selected_both_wrong": 277 + }, + { + "budget": 0.27, + "threshold": -0.9376689791679382, + "operator": "ge", + "actual_fraction": 0.25751596469828514, + "metric": 0.8688383439764655, + "rescued": 1245, + "harmed": 286, + "missed_rescue": 980, + "avoided_harm": 443, + "selected_both_correct": 1771, + "selected_both_wrong": 287 + }, + { + "budget": 0.28, + "threshold": -0.9449946880340576, + "operator": "gt", + "actual_fraction": 0.26863743990815814, + "metric": 0.8704886273947048, + "rescued": 1278, + "harmed": 296, + "missed_rescue": 947, + "avoided_harm": 433, + "selected_both_correct": 1869, + "selected_both_wrong": 301 + }, + { + "budget": 0.29, + "threshold": -0.9511421918869019, + "operator": "gt", + "actual_fraction": 0.2793284064002296, + "metric": 0.8725694195307455, + "rescued": 1322, + "harmed": 311, + "missed_rescue": 903, + "avoided_harm": 418, + "selected_both_correct": 1951, + "selected_both_wrong": 309 + }, + { + "budget": 0.3, + "threshold": -0.9559813141822815, + "operator": "gt", + "actual_fraction": 0.2897323670804334, + "metric": 0.8739326971371171, + "rescued": 1350, + "harmed": 320, + "missed_rescue": 875, + "avoided_harm": 409, + "selected_both_correct": 2050, + "selected_both_wrong": 318 + }, + { + "budget": 0.31, + "threshold": -0.9597620368003845, + "operator": "ge", + "actual_fraction": 0.2999928248547033, + "metric": 0.8755829805553562, + "rescued": 1377, + "harmed": 324, + "missed_rescue": 848, + "avoided_harm": 405, + "selected_both_correct": 2158, + "selected_both_wrong": 322 + }, + { + "budget": 0.32, + "threshold": -0.9643214344978333, + "operator": "gt", + "actual_fraction": 0.3091770108344694, + "metric": 0.8770897610676616, + "rescued": 1408, + "harmed": 334, + "missed_rescue": 817, + "avoided_harm": 395, + "selected_both_correct": 2234, + "selected_both_wrong": 333 + }, + { + "budget": 0.33, + "threshold": -0.9686195850372314, + "operator": "gt", + "actual_fraction": 0.3218770180096147, + "metric": 0.878596541579967, + "rescued": 1442, + "harmed": 347, + "missed_rescue": 783, + "avoided_harm": 382, + "selected_both_correct": 2358, + "selected_both_wrong": 339 + }, + { + "budget": 0.34, + "threshold": -0.9719924926757812, + "operator": "gt", + "actual_fraction": 0.33127645834828157, + "metric": 0.8806055822630409, + "rescued": 1476, + "harmed": 353, + "missed_rescue": 749, + "avoided_harm": 376, + "selected_both_correct": 2445, + "selected_both_wrong": 343 + }, + { + "budget": 0.35, + "threshold": -0.974821150302887, + "operator": "ge", + "actual_fraction": 0.3431871995407907, + "metric": 0.8816101026045777, + "rescued": 1507, + "harmed": 370, + "missed_rescue": 718, + "avoided_harm": 359, + "selected_both_correct": 2556, + "selected_both_wrong": 350 + }, + { + "budget": 0.36, + "threshold": -0.9777138829231262, + "operator": "gt", + "actual_fraction": 0.35280189423835834, + "metric": 0.8823993685872139, + "rescued": 1526, + "harmed": 378, + "missed_rescue": 699, + "avoided_harm": 351, + "selected_both_correct": 2652, + "selected_both_wrong": 361 + }, + { + "budget": 0.37, + "threshold": -0.9802808165550232, + "operator": "gt", + "actual_fraction": 0.36421037526009903, + "metric": 0.8837626461935855, + "rescued": 1555, + "harmed": 388, + "missed_rescue": 670, + "avoided_harm": 341, + "selected_both_correct": 2768, + "selected_both_wrong": 365 + }, + { + "budget": 0.38, + "threshold": -0.982557475566864, + "operator": "gt", + "actual_fraction": 0.3761928679055751, + "metric": 0.8851259237999569, + "rescued": 1587, + "harmed": 401, + "missed_rescue": 638, + "avoided_harm": 328, + "selected_both_correct": 2882, + "selected_both_wrong": 373 + }, + { + "budget": 0.39, + "threshold": -0.9843363761901855, + "operator": "gt", + "actual_fraction": 0.38803185764511733, + "metric": 0.8864892014063285, + "rescued": 1614, + "harmed": 409, + "missed_rescue": 611, + "avoided_harm": 320, + "selected_both_correct": 3004, + "selected_both_wrong": 381 + }, + { + "budget": 0.4, + "threshold": -0.9863631129264832, + "operator": "gt", + "actual_fraction": 0.4008036162732295, + "metric": 0.8881394848245677, + "rescued": 1647, + "harmed": 419, + "missed_rescue": 578, + "avoided_harm": 310, + "selected_both_correct": 3133, + "selected_both_wrong": 387 + }, + { + "budget": 0.41, + "threshold": -0.9879462122917175, + "operator": "gt", + "actual_fraction": 0.41156633421826794, + "metric": 0.8890722537131377, + "rescued": 1670, + "harmed": 429, + "missed_rescue": 555, + "avoided_harm": 300, + "selected_both_correct": 3241, + "selected_both_wrong": 396 + }, + { + "budget": 0.42, + "threshold": -0.9890130758285522, + "operator": "gt", + "actual_fraction": 0.4204635143861663, + "metric": 0.8895745138839062, + "rescued": 1684, + "harmed": 436, + "missed_rescue": 541, + "avoided_harm": 293, + "selected_both_correct": 3336, + "selected_both_wrong": 404 + }, + { + "budget": 0.43, + "threshold": -0.9899863600730896, + "operator": "gt", + "actual_fraction": 0.4293606945540647, + "metric": 0.890579034225443, + "rescued": 1706, + "harmed": 444, + "missed_rescue": 519, + "avoided_harm": 285, + "selected_both_correct": 3428, + "selected_both_wrong": 406 + }, + { + "budget": 0.44, + "threshold": -0.9911527037620544, + "operator": "gt", + "actual_fraction": 0.44119968429360695, + "metric": 0.89251632345555, + "rescued": 1742, + "harmed": 453, + "missed_rescue": 483, + "avoided_harm": 276, + "selected_both_correct": 3542, + "selected_both_wrong": 412 + }, + { + "budget": 0.45, + "threshold": -0.9919379353523254, + "operator": "gt", + "actual_fraction": 0.4516753964267776, + "metric": 0.8933773408911531, + "rescued": 1765, + "harmed": 464, + "missed_rescue": 460, + "avoided_harm": 265, + "selected_both_correct": 3649, + "selected_both_wrong": 417 + }, + { + "budget": 0.46, + "threshold": -0.9926542043685913, + "operator": "gt", + "actual_fraction": 0.4611465882184114, + "metric": 0.8938796010619215, + "rescued": 1781, + "harmed": 473, + "missed_rescue": 444, + "avoided_harm": 256, + "selected_both_correct": 3750, + "selected_both_wrong": 423 + }, + { + "budget": 0.47, + "threshold": -0.9933071732521057, + "operator": "gt", + "actual_fraction": 0.4706895314630121, + "metric": 0.8947406184975246, + "rescued": 1797, + "harmed": 477, + "missed_rescue": 428, + "avoided_harm": 252, + "selected_both_correct": 3860, + "selected_both_wrong": 426 + }, + { + "budget": 0.48, + "threshold": -0.9939024448394775, + "operator": "gt", + "actual_fraction": 0.481021740690249, + "metric": 0.8955298844801607, + "rescued": 1817, + "harmed": 486, + "missed_rescue": 408, + "avoided_harm": 243, + "selected_both_correct": 3969, + "selected_both_wrong": 432 + }, + { + "budget": 0.49, + "threshold": -0.9944451451301575, + "operator": "gt", + "actual_fraction": 0.4912821984645189, + "metric": 0.8963909019157638, + "rescued": 1835, + "harmed": 492, + "missed_rescue": 390, + "avoided_harm": 237, + "selected_both_correct": 4079, + "selected_both_wrong": 441 + }, + { + "budget": 0.5, + "threshold": -0.9949396848678589, + "operator": "gt", + "actual_fraction": 0.4992466097438473, + "metric": 0.8968931620865322, + "rescued": 1850, + "harmed": 500, + "missed_rescue": 375, + "avoided_harm": 229, + "selected_both_correct": 4163, + "selected_both_wrong": 445 + }, + { + "budget": 0.51, + "threshold": -0.9952448010444641, + "operator": "gt", + "actual_fraction": 0.5059912463227381, + "metric": 0.897108416445433, + "rescued": 1858, + "harmed": 505, + "missed_rescue": 367, + "avoided_harm": 224, + "selected_both_correct": 4239, + "selected_both_wrong": 450 + }, + { + "budget": 0.52, + "threshold": -0.9956685304641724, + "operator": "gt", + "actual_fraction": 0.5140991605080003, + "metric": 0.8974671737102676, + "rescued": 1869, + "harmed": 511, + "missed_rescue": 356, + "avoided_harm": 218, + "selected_both_correct": 4331, + "selected_both_wrong": 454 + }, + { + "budget": 0.53, + "threshold": -0.9961755275726318, + "operator": "gt", + "actual_fraction": 0.529166965631054, + "metric": 0.8992609600344407, + "rescued": 1903, + "harmed": 520, + "missed_rescue": 322, + "avoided_harm": 209, + "selected_both_correct": 4495, + "selected_both_wrong": 457 + }, + { + "budget": 0.54, + "threshold": -0.9965165853500366, + "operator": "gt", + "actual_fraction": 0.5406471981057617, + "metric": 0.899834971658176, + "rescued": 1920, + "harmed": 529, + "missed_rescue": 305, + "avoided_harm": 200, + "selected_both_correct": 4619, + "selected_both_wrong": 467 + }, + { + "budget": 0.55, + "threshold": -0.9968273043632507, + "operator": "gt", + "actual_fraction": 0.5504771471622301, + "metric": 0.89997847456411, + "rescued": 1927, + "harmed": 534, + "missed_rescue": 298, + "avoided_harm": 195, + "selected_both_correct": 4743, + "selected_both_wrong": 468 + }, + { + "budget": 0.56, + "threshold": -0.9971103668212891, + "operator": "gt", + "actual_fraction": 0.5621726339958384, + "metric": 0.9006959890937791, + "rescued": 1946, + "harmed": 543, + "missed_rescue": 279, + "avoided_harm": 186, + "selected_both_correct": 4874, + "selected_both_wrong": 472 + }, + { + "budget": 0.57, + "threshold": -0.997285008430481, + "operator": "gt", + "actual_fraction": 0.5677692473272584, + "metric": 0.9011264978115807, + "rescued": 1956, + "harmed": 547, + "missed_rescue": 269, + "avoided_harm": 182, + "selected_both_correct": 4935, + "selected_both_wrong": 475 + }, + { + "budget": 0.58, + "threshold": -0.9974491000175476, + "operator": "gt", + "actual_fraction": 0.5759489129654876, + "metric": 0.9014135036234484, + "rescued": 1968, + "harmed": 555, + "missed_rescue": 257, + "avoided_harm": 174, + "selected_both_correct": 5025, + "selected_both_wrong": 479 + }, + { + "budget": 0.59, + "threshold": -0.9976769089698792, + "operator": "gt", + "actual_fraction": 0.5864963765516251, + "metric": 0.9019875152471838, + "rescued": 1985, + "harmed": 564, + "missed_rescue": 240, + "avoided_harm": 165, + "selected_both_correct": 5144, + "selected_both_wrong": 481 + }, + { + "budget": 0.6, + "threshold": -0.9978172779083252, + "operator": "gt", + "actual_fraction": 0.5942455334720528, + "metric": 0.9022745210590515, + "rescued": 1994, + "harmed": 569, + "missed_rescue": 231, + "avoided_harm": 160, + "selected_both_correct": 5235, + "selected_both_wrong": 484 + }, + { + "budget": 0.61, + "threshold": -0.9980121850967407, + "operator": "gt", + "actual_fraction": 0.6054387601348927, + "metric": 0.9023462725120184, + "rescued": 2002, + "harmed": 576, + "missed_rescue": 223, + "avoided_harm": 153, + "selected_both_correct": 5372, + "selected_both_wrong": 488 + }, + { + "budget": 0.62, + "threshold": -0.9981897473335266, + "operator": "gt", + "actual_fraction": 0.6165602353447658, + "metric": 0.9033507928535552, + "rescued": 2018, + "harmed": 578, + "missed_rescue": 207, + "avoided_harm": 151, + "selected_both_correct": 5504, + "selected_both_wrong": 493 + }, + { + "budget": 0.63, + "threshold": -0.9982993006706238, + "operator": "gt", + "actual_fraction": 0.6242376408122264, + "metric": 0.9037095501183899, + "rescued": 2027, + "harmed": 582, + "missed_rescue": 198, + "avoided_harm": 147, + "selected_both_correct": 5596, + "selected_both_wrong": 495 + }, + { + "budget": 0.64, + "threshold": -0.9984021782875061, + "operator": "gt", + "actual_fraction": 0.6322020520915549, + "metric": 0.9040683073832245, + "rescued": 2037, + "harmed": 587, + "missed_rescue": 188, + "avoided_harm": 142, + "selected_both_correct": 5691, + "selected_both_wrong": 496 + }, + { + "budget": 0.65, + "threshold": -0.9985449314117432, + "operator": "gt", + "actual_fraction": 0.6454043194374686, + "metric": 0.9046423190069599, + "rescued": 2051, + "harmed": 593, + "missed_rescue": 174, + "avoided_harm": 136, + "selected_both_correct": 5853, + "selected_both_wrong": 498 + }, + { + "budget": 0.66, + "threshold": -0.9986749291419983, + "operator": "gt", + "actual_fraction": 0.6566692975532754, + "metric": 0.9050010762717945, + "rescued": 2062, + "harmed": 599, + "missed_rescue": 163, + "avoided_harm": 130, + "selected_both_correct": 5989, + "selected_both_wrong": 502 + }, + { + "budget": 0.67, + "threshold": -0.9987550973892212, + "operator": "gt", + "actual_fraction": 0.6646337088326039, + "metric": 0.9047858219128937, + "rescued": 2066, + "harmed": 606, + "missed_rescue": 159, + "avoided_harm": 123, + "selected_both_correct": 6087, + "selected_both_wrong": 504 + }, + { + "budget": 0.68, + "threshold": -0.9988664388656616, + "operator": "gt", + "actual_fraction": 0.6776207218196169, + "metric": 0.9051445791777284, + "rescued": 2077, + "harmed": 612, + "missed_rescue": 148, + "avoided_harm": 117, + "selected_both_correct": 6248, + "selected_both_wrong": 507 + }, + { + "budget": 0.69, + "threshold": -0.9989677667617798, + "operator": "gt", + "actual_fraction": 0.690464231900696, + "metric": 0.9052163306306953, + "rescued": 2084, + "harmed": 618, + "missed_rescue": 141, + "avoided_harm": 111, + "selected_both_correct": 6409, + "selected_both_wrong": 512 + }, + { + "budget": 0.7, + "threshold": -0.9990302324295044, + "operator": "gt", + "actual_fraction": 0.7000789265982637, + "metric": 0.9053598335366291, + "rescued": 2094, + "harmed": 626, + "missed_rescue": 131, + "avoided_harm": 103, + "selected_both_correct": 6523, + "selected_both_wrong": 514 + }, + { + "budget": 0.71, + "threshold": -0.9991169571876526, + "operator": "gt", + "actual_fraction": 0.7121331706967066, + "metric": 0.9056468393484968, + "rescued": 2103, + "harmed": 631, + "missed_rescue": 122, + "avoided_harm": 98, + "selected_both_correct": 6676, + "selected_both_wrong": 515 + }, + { + "budget": 0.72, + "threshold": -0.9991704225540161, + "operator": "gt", + "actual_fraction": 0.7201693334290019, + "metric": 0.9060055966133315, + "rescued": 2110, + "harmed": 633, + "missed_rescue": 115, + "avoided_harm": 96, + "selected_both_correct": 6779, + "selected_both_wrong": 515 + }, + { + "budget": 0.73, + "threshold": -0.9992445707321167, + "operator": "gt", + "actual_fraction": 0.7310755542799742, + "metric": 0.9062208509722322, + "rescued": 2117, + "harmed": 637, + "missed_rescue": 108, + "avoided_harm": 92, + "selected_both_correct": 6917, + "selected_both_wrong": 518 + }, + { + "budget": 0.74, + "threshold": -0.9992903470993042, + "operator": "gt", + "actual_fraction": 0.7394704742771041, + "metric": 0.9062208509722322, + "rescued": 2121, + "harmed": 641, + "missed_rescue": 104, + "avoided_harm": 88, + "selected_both_correct": 7024, + "selected_both_wrong": 520 + }, + { + "budget": 0.75, + "threshold": -0.9993333220481873, + "operator": "gt", + "actual_fraction": 0.7467173710267633, + "metric": 0.9065796082370668, + "rescued": 2127, + "harmed": 642, + "missed_rescue": 98, + "avoided_harm": 87, + "selected_both_correct": 7118, + "selected_both_wrong": 520 + }, + { + "budget": 0.76, + "threshold": -0.999373733997345, + "operator": "gt", + "actual_fraction": 0.7550405395709263, + "metric": 0.9065796082370668, + "rescued": 2129, + "harmed": 644, + "missed_rescue": 96, + "avoided_harm": 85, + "selected_both_correct": 7227, + "selected_both_wrong": 523 + }, + { + "budget": 0.77, + "threshold": -0.9994296431541443, + "operator": "gt", + "actual_fraction": 0.7659467604218986, + "metric": 0.9066513596900337, + "rescued": 2136, + "harmed": 650, + "missed_rescue": 89, + "avoided_harm": 79, + "selected_both_correct": 7362, + "selected_both_wrong": 527 + }, + { + "budget": 0.78, + "threshold": -0.999464213848114, + "operator": "gt", + "actual_fraction": 0.7743416804190285, + "metric": 0.9071536198608022, + "rescued": 2145, + "harmed": 652, + "missed_rescue": 80, + "avoided_harm": 77, + "selected_both_correct": 7467, + "selected_both_wrong": 528 + }, + { + "budget": 0.79, + "threshold": -0.9994966983795166, + "operator": "gt", + "actual_fraction": 0.7820908373394562, + "metric": 0.9071536198608022, + "rescued": 2150, + "harmed": 657, + "missed_rescue": 75, + "avoided_harm": 72, + "selected_both_correct": 7563, + "selected_both_wrong": 530 + }, + { + "budget": 0.8, + "threshold": -0.9995417594909668, + "operator": "gt", + "actual_fraction": 0.7935710698141637, + "metric": 0.9070101169548683, + "rescued": 2157, + "harmed": 666, + "missed_rescue": 68, + "avoided_harm": 63, + "selected_both_correct": 7706, + "selected_both_wrong": 531 + }, + { + "budget": 0.81, + "threshold": -0.9995827078819275, + "operator": "gt", + "actual_fraction": 0.8052665566477721, + "metric": 0.907297122766736, + "rescued": 2166, + "harmed": 671, + "missed_rescue": 59, + "avoided_harm": 58, + "selected_both_correct": 7854, + "selected_both_wrong": 532 + }, + { + "budget": 0.82, + "threshold": -0.9996140599250793, + "operator": "gt", + "actual_fraction": 0.8185405754466528, + "metric": 0.907297122766736, + "rescued": 2172, + "harmed": 677, + "missed_rescue": 53, + "avoided_harm": 52, + "selected_both_correct": 8022, + "selected_both_wrong": 537 + }, + { + "budget": 0.83, + "threshold": -0.9996429681777954, + "operator": "gt", + "actual_fraction": 0.8264332352730143, + "metric": 0.9075123771256368, + "rescued": 2176, + "harmed": 678, + "missed_rescue": 49, + "avoided_harm": 51, + "selected_both_correct": 8126, + "selected_both_wrong": 538 + }, + { + "budget": 0.84, + "threshold": -0.9996750354766846, + "operator": "gt", + "actual_fraction": 0.836837195953218, + "metric": 0.9075841285786037, + "rescued": 2179, + "harmed": 680, + "missed_rescue": 46, + "avoided_harm": 49, + "selected_both_correct": 8265, + "selected_both_wrong": 539 + }, + { + "budget": 0.85, + "threshold": -0.9997040629386902, + "operator": "gt", + "actual_fraction": 0.844299347061778, + "metric": 0.9075123771256368, + "rescued": 2183, + "harmed": 685, + "missed_rescue": 42, + "avoided_harm": 44, + "selected_both_correct": 8359, + "selected_both_wrong": 540 + }, + { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8580038745784602, + "metric": 0.9077276314845376, + "rescued": 2191, + "harmed": 690, + "missed_rescue": 34, + "avoided_harm": 39, + "selected_both_correct": 8533, + "selected_both_wrong": 544 + }, + { + "budget": 0.87, + "threshold": -0.9997695088386536, + "operator": "gt", + "actual_fraction": 0.870273373035804, + "metric": 0.9081581402023391, + "rescued": 2199, + "harmed": 692, + "missed_rescue": 26, + "avoided_harm": 37, + "selected_both_correct": 8692, + "selected_both_wrong": 546 + }, + { + "budget": 0.88, + "threshold": -0.999783456325531, + "operator": "gt", + "actual_fraction": 0.8750807203845878, + "metric": 0.9080863887493722, + "rescued": 2199, + "harmed": 693, + "missed_rescue": 26, + "avoided_harm": 36, + "selected_both_correct": 8757, + "selected_both_wrong": 547 + }, + { + "budget": 0.89, + "threshold": -0.9998205304145813, + "operator": "gt", + "actual_fraction": 0.8908660400373107, + "metric": 0.9080146372964053, + "rescued": 2202, + "harmed": 697, + "missed_rescue": 23, + "avoided_harm": 32, + "selected_both_correct": 8966, + "selected_both_wrong": 551 + }, + { + "budget": 0.9, + "threshold": -0.999841570854187, + "operator": "gt", + "actual_fraction": 0.8992609600344407, + "metric": 0.9077276314845376, + "rescued": 2204, + "harmed": 703, + "missed_rescue": 21, + "avoided_harm": 26, + "selected_both_correct": 9075, + "selected_both_wrong": 551 + }, + { + "budget": 0.91, + "threshold": -0.9998601675033569, + "operator": "gt", + "actual_fraction": 0.9091626605438761, + "metric": 0.9076558800315706, + "rescued": 2206, + "harmed": 706, + "missed_rescue": 19, + "avoided_harm": 23, + "selected_both_correct": 9206, + "selected_both_wrong": 553 + }, + { + "budget": 0.92, + "threshold": -0.9998766183853149, + "operator": "gt", + "actual_fraction": 0.9171270718232044, + "metric": 0.9077276314845376, + "rescued": 2209, + "harmed": 708, + "missed_rescue": 16, + "avoided_harm": 21, + "selected_both_correct": 9311, + "selected_both_wrong": 554 + }, + { + "budget": 0.93, + "threshold": -0.9998977184295654, + "operator": "gt", + "actual_fraction": 0.9273875295974744, + "metric": 0.9075123771256368, + "rescued": 2210, + "harmed": 712, + "missed_rescue": 15, + "avoided_harm": 17, + "selected_both_correct": 9448, + "selected_both_wrong": 555 + }, + { + "budget": 0.94, + "threshold": -0.9999151229858398, + "operator": "gt", + "actual_fraction": 0.9375044844658105, + "metric": 0.9073688742197029, + "rescued": 2213, + "harmed": 717, + "missed_rescue": 12, + "avoided_harm": 12, + "selected_both_correct": 9580, + "selected_both_wrong": 556 + }, + { + "budget": 0.95, + "threshold": -0.999931812286377, + "operator": "gt", + "actual_fraction": 0.948984716940518, + "metric": 0.9073688742197029, + "rescued": 2217, + "harmed": 721, + "missed_rescue": 8, + "avoided_harm": 8, + "selected_both_correct": 9731, + "selected_both_wrong": 557 + }, + { + "budget": 0.96, + "threshold": -0.9999452829360962, + "operator": "gt", + "actual_fraction": 0.9576666427495156, + "metric": 0.9073688742197029, + "rescued": 2217, + "harmed": 721, + "missed_rescue": 8, + "avoided_harm": 8, + "selected_both_correct": 9850, + "selected_both_wrong": 559 + }, + { + "budget": 0.97, + "threshold": -0.9999599456787109, + "operator": "gt", + "actual_fraction": 0.9709406615483963, + "metric": 0.9072253713137691, + "rescued": 2217, + "harmed": 723, + "missed_rescue": 8, + "avoided_harm": 6, + "selected_both_correct": 10033, + "selected_both_wrong": 559 + }, + { + "budget": 0.98, + "threshold": -0.9999687671661377, + "operator": "gt", + "actual_fraction": 0.9784745641099232, + "metric": 0.907297122766736, + "rescued": 2218, + "harmed": 723, + "missed_rescue": 7, + "avoided_harm": 6, + "selected_both_correct": 10136, + "selected_both_wrong": 560 + }, + { + "budget": 0.99, + "threshold": -0.9999842643737793, + "operator": "gt", + "actual_fraction": 0.98973954222573, + "metric": 0.9072253713137691, + "rescued": 2222, + "harmed": 728, + "missed_rescue": 3, + "avoided_harm": 1, + "selected_both_correct": 10282, + "selected_both_wrong": 562 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9073688742197029, + "rescued": 2225, + "harmed": 729, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 10421, + "selected_both_wrong": 562 + } + ] + }, + "learned": { + "validation_area_0_25": 0.8364508736751646, + "validation_point": { + "metric": 0.8987109710684618, + "metric_std": 0.0005104133394788471, + "actual_fraction": 0.9899742194213692, + "actual_fraction_std": 0.0, + "per_seed": [ + { + "seed": 2026, + "budget": 0.99, + "threshold": -0.10858634114265442, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8994557433400172, + "rescue": 1079, + "harm": 372, + "missed_rescue": 8 + }, + { + "seed": 2027, + "budget": 0.99, + "threshold": -0.18117938935756683, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8983099398453165, + "rescue": 1073, + "harm": 374, + "missed_rescue": 14 + }, + { + "seed": 2028, + "budget": 0.99, + "threshold": -0.3423033654689789, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8984531652821541, + "rescue": 1075, + "harm": 375, + "missed_rescue": 12 + }, + { + "seed": 2029, + "budget": 0.99, + "threshold": -0.20271095633506775, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8990260670295045, + "rescue": 1070, + "harm": 366, + "missed_rescue": 17 + }, + { + "seed": 2030, + "budget": 0.99, + "threshold": -0.19141565263271332, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8983099398453165, + "rescue": 1070, + "harm": 371, + "missed_rescue": 17 + } + ], + "rescued": 1073.4, + "harmed": 371.6, + "missed_rescue": 13.6, + "avoided_harm": 11.399999999999977 + }, + "test_point": { + "metric": 0.9070962186984287, + "metric_std": 0.0007583283398502093, + "actual_fraction": 0.9904427064648059, + "actual_fraction_std": 0.0008793578933689391, + "per_seed": [ + { + "seed": 2026, + "budget": 0.99, + "threshold": -0.10858634114265442, + "operator": "ge", + "actual_fraction": 0.9901700509435316, + "metric": 0.9071536198608022, + "rescue": 2205, + "harm": 712, + "missed_rescue": 20 + }, + { + "seed": 2027, + "budget": 0.99, + "threshold": -0.18117938935756683, + "operator": "ge", + "actual_fraction": 0.9910310683791347, + "metric": 0.9067948625959675, + "rescue": 2201, + "harm": 713, + "missed_rescue": 24 + }, + { + "seed": 2028, + "budget": 0.99, + "threshold": -0.3423033654689789, + "operator": "ge", + "actual_fraction": 0.9903853053024324, + "metric": 0.9080146372964053, + "rescue": 2206, + "harm": 701, + "missed_rescue": 19 + }, + { + "seed": 2029, + "budget": 0.99, + "threshold": -0.20271095633506775, + "operator": "ge", + "actual_fraction": 0.9891655306019946, + "metric": 0.9060055966133315, + "rescue": 2189, + "harm": 712, + "missed_rescue": 36 + }, + { + "seed": 2030, + "budget": 0.99, + "threshold": -0.19141565263271332, + "operator": "ge", + "actual_fraction": 0.9914615770969362, + "metric": 0.9075123771256368, + "rescue": 2204, + "harm": 706, + "missed_rescue": 21 + } + ], + "rescued": 2201.0, + "harmed": 708.8, + "missed_rescue": 24.0, + "avoided_harm": 20.200000000000045 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.7981953594958464, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.8012890289315383, + "metric_std": 0.0003869738208566602, + "actual_fraction": 0.009882555141793183, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.8056144371240332, + "metric_std": 0.0009024339158993529, + "actual_fraction": 0.01990833572042395, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.809767974792323, + "metric_std": 0.0009193243503799067, + "actual_fraction": 0.029934116299054715, + "actual_fraction_std": 3.878959614448864e-18 + }, + { + "budget": 0.04, + "metric": 0.8138928673732455, + "metric_std": 0.00031379121025787367, + "actual_fraction": 0.039959896877685475, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.8179031796046976, + "metric_std": 0.00032972971736527034, + "actual_fraction": 0.04998567745631624, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.821025494127757, + "metric_std": 0.0008388169844941975, + "actual_fraction": 0.05986823259810943, + "actual_fraction_std": 7.757919228897728e-18 + }, + { + "budget": 0.07, + "metric": 0.8238040676024061, + "metric_std": 0.0007805457802027524, + "actual_fraction": 0.06989401317674018, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.8266112861644228, + "metric_std": 0.0005223312096479592, + "actual_fraction": 0.07991979375537095, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.8293612145517043, + "metric_std": 0.0007686283509022277, + "actual_fraction": 0.08994557433400172, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.832225723288456, + "metric_std": 0.0005416118705582688, + "actual_fraction": 0.09997135491263248, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.8350902320252075, + "metric_std": 0.0006453096720749825, + "actual_fraction": 0.10999713549126325, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.12, + "metric": 0.8375250644514466, + "metric_std": 0.0002172121295918536, + "actual_fraction": 0.11987969063305644, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.8406187338871384, + "metric_std": 0.0009839910648706625, + "actual_fraction": 0.1299054712116872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.8430535663133772, + "metric_std": 0.0007118174528667612, + "actual_fraction": 0.13993125179031796, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.8452592380406762, + "metric_std": 0.0010056424910651319, + "actual_fraction": 0.14995703236894872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.8476081352048123, + "metric_std": 0.0012025820412307944, + "actual_fraction": 0.1599828129475795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.8500429676310512, + "metric_std": 0.0007712925819442129, + "actual_fraction": 0.16986536808937266, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.8521627040962475, + "metric_std": 0.0008557644942016358, + "actual_fraction": 0.17989114866800343, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.853623603551991, + "metric_std": 0.0006484807460711239, + "actual_fraction": 0.1899169292466342, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.8560011458034946, + "metric_std": 0.00016330212333128536, + "actual_fraction": 0.19994270982526496, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.8575766256087082, + "metric_std": 0.00042487530719538107, + "actual_fraction": 0.20996849040389573, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.22, + "metric": 0.8595531366370667, + "metric_std": 0.0006042687799979497, + "actual_fraction": 0.2199942709825265, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.23, + "metric": 0.8607562303065025, + "metric_std": 0.0009413736307737349, + "actual_fraction": 0.22987682612431967, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.24, + "metric": 0.8621884846748783, + "metric_std": 0.0007331127981706802, + "actual_fraction": 0.23990260670295044, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.8635348037811514, + "metric_std": 0.0010097139348046124, + "actual_fraction": 0.24992838728158123, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.8643655113148097, + "metric_std": 0.0008593526210255072, + "actual_fraction": 0.25995416786021197, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.865110283586365, + "metric_std": 0.0013827004739211959, + "actual_fraction": 0.26997994843884277, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.8655686049842451, + "metric_std": 0.0015245269654236885, + "actual_fraction": 0.2798625035806359, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.8660842165568605, + "metric_std": 0.002065626625874518, + "actual_fraction": 0.2898882841592667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.8665425379547408, + "metric_std": 0.002245033023938108, + "actual_fraction": 0.29991406473789745, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.8669722142652534, + "metric_std": 0.002622238325143688, + "actual_fraction": 0.30993984531652824, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.867172729876826, + "metric_std": 0.0028993866343328795, + "actual_fraction": 0.319965625895159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.8679175021483816, + "metric_std": 0.0034750989357616863, + "actual_fraction": 0.3299914064737898, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.8683185333715266, + "metric_std": 0.0036383207900343396, + "actual_fraction": 0.3398739616155829, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.8686049842452019, + "metric_std": 0.003954422926901229, + "actual_fraction": 0.3498997421942137, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.8690919507304496, + "metric_std": 0.0038604502927366124, + "actual_fraction": 0.35992552277284445, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.8694070466914925, + "metric_std": 0.004008265631558109, + "actual_fraction": 0.36995130335147525, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.8696648524778, + "metric_std": 0.004153537668289863, + "actual_fraction": 0.379977083930106, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.8702091091377829, + "metric_std": 0.004103102556484496, + "actual_fraction": 0.3898596390718992, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.8703809796619879, + "metric_std": 0.004142410445138811, + "actual_fraction": 0.3998854196505299, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.8706101403609281, + "metric_std": 0.0044198450853726, + "actual_fraction": 0.40991120022916067, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.42, + "metric": 0.8710684617588083, + "metric_std": 0.0043717790369701606, + "actual_fraction": 0.41993698080779146, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.43, + "metric": 0.8712116871956459, + "metric_std": 0.0043576795193712835, + "actual_fraction": 0.42996276138642225, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.44, + "metric": 0.8715840733314237, + "metric_std": 0.004457970811219807, + "actual_fraction": 0.439988541965053, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.45, + "metric": 0.8716986536808937, + "metric_std": 0.004712748066409505, + "actual_fraction": 0.4498710971068462, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.8718991692924662, + "metric_std": 0.004463719028801138, + "actual_fraction": 0.45989687768547693, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.47, + "metric": 0.8724147808650816, + "metric_std": 0.0047344619118776, + "actual_fraction": 0.46992265826410773, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.48, + "metric": 0.8726439415640218, + "metric_std": 0.004517167704524849, + "actual_fraction": 0.4799484388427385, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.49, + "metric": 0.8729590375250644, + "metric_std": 0.004334788298602592, + "actual_fraction": 0.4899742194213693, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.8733027785734745, + "metric_std": 0.004398909961214991, + "actual_fraction": 0.5, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.51, + "metric": 0.8733314236608422, + "metric_std": 0.004374359018922057, + "actual_fraction": 0.5098825551417931, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.8736751647092523, + "metric_std": 0.0043947109689390675, + "actual_fraction": 0.5199083357204239, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.8740189057576625, + "metric_std": 0.004347310810169398, + "actual_fraction": 0.5299341162990547, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.8740475508450303, + "metric_std": 0.004389573412513698, + "actual_fraction": 0.5399598968776855, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.8743340017187051, + "metric_std": 0.004347310810169382, + "actual_fraction": 0.5499856774563162, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.8747063878544828, + "metric_std": 0.0042948530080759075, + "actual_fraction": 0.5598682325981095, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.8753652248639359, + "metric_std": 0.0037009297680663956, + "actual_fraction": 0.5698940131767402, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.8753938699513034, + "metric_std": 0.0036725526797359346, + "actual_fraction": 0.579919793755371, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.8759094815239186, + "metric_std": 0.003665004320037097, + "actual_fraction": 0.5899455743340017, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.8763678029217988, + "metric_std": 0.003282332056289007, + "actual_fraction": 0.5999713549126325, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.8768261243196791, + "metric_std": 0.0032801439388710565, + "actual_fraction": 0.6099971354912632, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.8771125751933543, + "metric_std": 0.0032138077445311984, + "actual_fraction": 0.6198796906330565, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.8774849613291321, + "metric_std": 0.003445755172430944, + "actual_fraction": 0.6299054712116872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.8776568318533373, + "metric_std": 0.0034824698746328633, + "actual_fraction": 0.639931251790318, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.8780292179891148, + "metric_std": 0.0035179269590753146, + "actual_fraction": 0.6499570323689488, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.8786594099112003, + "metric_std": 0.003639448252345257, + "actual_fraction": 0.6599828129475795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.879203666571183, + "metric_std": 0.003677576312165234, + "actual_fraction": 0.6698653680893727, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.8796906330564308, + "metric_std": 0.003745831231916956, + "actual_fraction": 0.6798911486680035, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.8804354053279863, + "metric_std": 0.003542333087916835, + "actual_fraction": 0.6899169292466342, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.8805786307648239, + "metric_std": 0.003394473069253129, + "actual_fraction": 0.699942709825265, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.881094242337439, + "metric_std": 0.0030507354254497906, + "actual_fraction": 0.7099684904038958, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.881695789172157, + "metric_std": 0.003083507611230425, + "actual_fraction": 0.7199942709825266, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.8817244342595245, + "metric_std": 0.003052415993761843, + "actual_fraction": 0.7298768261243197, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.8823832712689773, + "metric_std": 0.0028472698114669816, + "actual_fraction": 0.7399026067029505, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.8826697221426526, + "metric_std": 0.002528649042187419, + "actual_fraction": 0.7499283872815812, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.883357204239473, + "metric_std": 0.0024861064560230176, + "actual_fraction": 0.759954167860212, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.8836436551131481, + "metric_std": 0.0029744719966963047, + "actual_fraction": 0.7699799484388428, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.8845602979089087, + "metric_std": 0.002552066697394057, + "actual_fraction": 0.7798625035806359, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.885247780005729, + "metric_std": 0.003154863340534737, + "actual_fraction": 0.7898882841592667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.8854482956173015, + "metric_std": 0.0033056852800776933, + "actual_fraction": 0.7999140647378975, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.8862790031509595, + "metric_std": 0.003071843395807397, + "actual_fraction": 0.8099398453165282, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.8867659696362074, + "metric_std": 0.0031027403432947066, + "actual_fraction": 0.819965625895159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.8869951303351475, + "metric_std": 0.003175279186106486, + "actual_fraction": 0.8299914064737898, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.84, + "metric": 0.8875680320824978, + "metric_std": 0.0027827784628652627, + "actual_fraction": 0.8398739616155829, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.85, + "metric": 0.8884846748782584, + "metric_std": 0.0023107727613569918, + "actual_fraction": 0.8498997421942137, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.8893153824119162, + "metric_std": 0.0023368141895746797, + "actual_fraction": 0.8599255227728445, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.87, + "metric": 0.8897450587224291, + "metric_std": 0.002276788957912832, + "actual_fraction": 0.8699513033514752, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.88, + "metric": 0.8908335720423946, + "metric_std": 0.0018519885276705567, + "actual_fraction": 0.879977083930106, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.89, + "metric": 0.8919793755370954, + "metric_std": 0.0020466709456197223, + "actual_fraction": 0.8898596390718991, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.8924376969349757, + "metric_std": 0.0019794149185169076, + "actual_fraction": 0.8998854196505299, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.8931824692065311, + "metric_std": 0.0018492173327504578, + "actual_fraction": 0.9099112002291607, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.92, + "metric": 0.8935835004296763, + "metric_std": 0.001685563204803752, + "actual_fraction": 0.9199369808077915, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.93, + "metric": 0.8944714981380694, + "metric_std": 0.002180133922026416, + "actual_fraction": 0.9299627613864223, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.94, + "metric": 0.8957605270696076, + "metric_std": 0.001884925107575145, + "actual_fraction": 0.9399885419650531, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.95, + "metric": 0.896276138642223, + "metric_std": 0.0015001112544000244, + "actual_fraction": 0.9498710971068463, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.96, + "metric": 0.8969349756516758, + "metric_std": 0.001322801476828286, + "actual_fraction": 0.959896877685477, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.97, + "metric": 0.8976224577484961, + "metric_std": 0.0007739476417133224, + "actual_fraction": 0.9699226582641078, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.98, + "metric": 0.8977370380979661, + "metric_std": 0.0006943110652272031, + "actual_fraction": 0.9799484388427386, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.8987109710684618, + "metric_std": 0.0005104133394788471, + "actual_fraction": 0.9899742194213692, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.8990260670295045, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.8000287005811868, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.8058118676903208, + "metric_std": 0.001014211292765124, + "actual_fraction": 0.010289158355456698, + "actual_fraction_std": 0.0018360522467173086 + }, + { + "budget": 0.02, + "metric": 0.8107770682356318, + "metric_std": 0.0008012424065519822, + "actual_fraction": 0.020693119035660474, + "actual_fraction_std": 0.0010829478909220591 + }, + { + "budget": 0.03, + "metric": 0.8156848676185693, + "metric_std": 0.0006926895823289052, + "actual_fraction": 0.03201549831384086, + "actual_fraction_std": 0.0013711850970040905 + }, + { + "budget": 0.04, + "metric": 0.8193872425916625, + "metric_std": 0.0012021313033542121, + "actual_fraction": 0.041816746789122484, + "actual_fraction_std": 0.001577226853874904 + }, + { + "budget": 0.05, + "metric": 0.8233048719236564, + "metric_std": 0.0017506472311705666, + "actual_fraction": 0.05342613187917056, + "actual_fraction_std": 0.0018085067232719405 + }, + { + "budget": 0.06, + "metric": 0.82709334864031, + "metric_std": 0.0017275568587923763, + "actual_fraction": 0.06374399081581403, + "actual_fraction_std": 0.002892660516022521 + }, + { + "budget": 0.07, + "metric": 0.830523068092129, + "metric_std": 0.0011580693399992244, + "actual_fraction": 0.0735882901628758, + "actual_fraction_std": 0.0027629434020709667 + }, + { + "budget": 0.08, + "metric": 0.8332926741766521, + "metric_std": 0.0007654235690949152, + "actual_fraction": 0.08347564038171773, + "actual_fraction_std": 0.0015502299520191068 + }, + { + "budget": 0.09, + "metric": 0.8362057831671091, + "metric_std": 0.0009470104586006392, + "actual_fraction": 0.09310468536987873, + "actual_fraction_std": 0.0013638439475742602 + }, + { + "budget": 0.1, + "metric": 0.8386166319867978, + "metric_std": 0.0015105342460432763, + "actual_fraction": 0.10271938006744637, + "actual_fraction_std": 0.0020823997372533373 + }, + { + "budget": 0.11, + "metric": 0.8416732438831886, + "metric_std": 0.0018604253801821225, + "actual_fraction": 0.11330989452536414, + "actual_fraction_std": 0.0033248108964797764 + }, + { + "budget": 0.12, + "metric": 0.8443567482241516, + "metric_std": 0.0020144148541097005, + "actual_fraction": 0.12349860084666715, + "actual_fraction_std": 0.003532037697157448 + }, + { + "budget": 0.13, + "metric": 0.8473129080863888, + "metric_std": 0.0018102139357885916, + "actual_fraction": 0.13360120542440984, + "actual_fraction_std": 0.0040103863421057945 + }, + { + "budget": 0.14, + "metric": 0.8492358470259023, + "metric_std": 0.0019040515854927052, + "actual_fraction": 0.14249838559230824, + "actual_fraction_std": 0.0030857296089900487 + }, + { + "budget": 0.15, + "metric": 0.8520915548539859, + "metric_std": 0.0015352128403290997, + "actual_fraction": 0.15314630121259956, + "actual_fraction_std": 0.00311916696270227 + }, + { + "budget": 0.16, + "metric": 0.854488053383081, + "metric_std": 0.0012714389859129812, + "actual_fraction": 0.1635933127645835, + "actual_fraction_std": 0.001790195156758928 + }, + { + "budget": 0.17, + "metric": 0.8565831958097151, + "metric_std": 0.0014398641369426414, + "actual_fraction": 0.17388247112004018, + "actual_fraction_std": 0.0026179941149883216 + }, + { + "budget": 0.18, + "metric": 0.8588218411422831, + "metric_std": 0.001434670257217751, + "actual_fraction": 0.184774341680419, + "actual_fraction_std": 0.003957858534664398 + }, + { + "budget": 0.19, + "metric": 0.8609743847312907, + "metric_std": 0.0015149584855952386, + "actual_fraction": 0.19522135323240294, + "actual_fraction_std": 0.002879191895100496 + }, + { + "budget": 0.2, + "metric": 0.8630982277391117, + "metric_std": 0.0011424039124697323, + "actual_fraction": 0.20549616129726625, + "actual_fraction_std": 0.001987139617439372 + }, + { + "budget": 0.21, + "metric": 0.8648489631915046, + "metric_std": 0.0011829201404061661, + "actual_fraction": 0.21544091267848176, + "actual_fraction_std": 0.002220008970937328 + }, + { + "budget": 0.22, + "metric": 0.8666283992250843, + "metric_std": 0.0015215707246093513, + "actual_fraction": 0.22654803759776138, + "actual_fraction_std": 0.0019894699663483865 + }, + { + "budget": 0.23, + "metric": 0.8679055750878956, + "metric_std": 0.0018391340486440806, + "actual_fraction": 0.23498600846667145, + "actual_fraction_std": 0.0029180884562908416 + }, + { + "budget": 0.24, + "metric": 0.869225801822487, + "metric_std": 0.0019089123122416002, + "actual_fraction": 0.24465810432661259, + "actual_fraction_std": 0.0038161386486608175 + }, + { + "budget": 0.25, + "metric": 0.8710052378560666, + "metric_std": 0.0016505953515976743, + "actual_fraction": 0.2550333644256296, + "actual_fraction_std": 0.0028779399558379417 + }, + { + "budget": 0.26, + "metric": 0.8723254645906579, + "metric_std": 0.001882433247494229, + "actual_fraction": 0.2648920140632848, + "actual_fraction_std": 0.0036805068486197782 + }, + { + "budget": 0.27, + "metric": 0.8736026404534692, + "metric_std": 0.002384912955217611, + "actual_fraction": 0.2743488555643252, + "actual_fraction_std": 0.00411980822881887 + }, + { + "budget": 0.28, + "metric": 0.8742197029489847, + "metric_std": 0.0027076074139503743, + "actual_fraction": 0.2831168831168831, + "actual_fraction_std": 0.005143483275215333 + }, + { + "budget": 0.29, + "metric": 0.8748080648633134, + "metric_std": 0.0030430568599440306, + "actual_fraction": 0.29198536270359476, + "actual_fraction_std": 0.004651575364415669 + }, + { + "budget": 0.3, + "metric": 0.8752098729999283, + "metric_std": 0.0033839007616316284, + "actual_fraction": 0.301399153332855, + "actual_fraction_std": 0.005106112889449927 + }, + { + "budget": 0.31, + "metric": 0.8757551840424769, + "metric_std": 0.0039640973224737116, + "actual_fraction": 0.31094209657745564, + "actual_fraction_std": 0.004918665461663474 + }, + { + "budget": 0.32, + "metric": 0.8763435459568056, + "metric_std": 0.004181823399251847, + "actual_fraction": 0.32080074621511084, + "actual_fraction_std": 0.003653692202475144 + }, + { + "budget": 0.33, + "metric": 0.8767597043840137, + "metric_std": 0.004377015065425256, + "actual_fraction": 0.33022888713496446, + "actual_fraction_std": 0.003943262977529103 + }, + { + "budget": 0.34, + "metric": 0.8772763148453755, + "metric_std": 0.004695911965376874, + "actual_fraction": 0.3400588361914329, + "actual_fraction_std": 0.00400608352349642 + }, + { + "budget": 0.35, + "metric": 0.8775059194948698, + "metric_std": 0.004864674430922108, + "actual_fraction": 0.34931477362416585, + "actual_fraction_std": 0.00296560134782027 + }, + { + "budget": 0.36, + "metric": 0.877893377340891, + "metric_std": 0.005065317186216055, + "actual_fraction": 0.35905862093707397, + "actual_fraction_std": 0.004375779874931288 + }, + { + "budget": 0.37, + "metric": 0.8781516825715722, + "metric_std": 0.0051327119903476, + "actual_fraction": 0.36722393628470973, + "actual_fraction_std": 0.003906337831331347 + }, + { + "budget": 0.38, + "metric": 0.8782951854775061, + "metric_std": 0.005174667828962441, + "actual_fraction": 0.3769390830164311, + "actual_fraction_std": 0.004367005870835402 + }, + { + "budget": 0.39, + "metric": 0.8784386883834397, + "metric_std": 0.005171931133491935, + "actual_fraction": 0.3859223649278898, + "actual_fraction_std": 0.0038521242855801626 + }, + { + "budget": 0.4, + "metric": 0.8788691971012413, + "metric_std": 0.005068365399514231, + "actual_fraction": 0.39688598694123556, + "actual_fraction_std": 0.003851455989694489 + }, + { + "budget": 0.41, + "metric": 0.8792566549472627, + "metric_std": 0.0050198846677708625, + "actual_fraction": 0.4077204563392408, + "actual_fraction_std": 0.0030926457716850815 + }, + { + "budget": 0.42, + "metric": 0.8796154122120974, + "metric_std": 0.005041120443347842, + "actual_fraction": 0.41744995336155555, + "actual_fraction_std": 0.002294925104645117 + }, + { + "budget": 0.43, + "metric": 0.8799885197675253, + "metric_std": 0.004931470628862282, + "actual_fraction": 0.427394704742771, + "actual_fraction_std": 0.003367200206344796 + }, + { + "budget": 0.44, + "metric": 0.8802898758699863, + "metric_std": 0.0049722773290572205, + "actual_fraction": 0.4383726770467102, + "actual_fraction_std": 0.0040249943095380936 + }, + { + "budget": 0.45, + "metric": 0.8805481811006673, + "metric_std": 0.004902625095912767, + "actual_fraction": 0.4482313266843653, + "actual_fraction_std": 0.0028660192576237657 + }, + { + "budget": 0.46, + "metric": 0.8810073903996555, + "metric_std": 0.0048937436642232535, + "actual_fraction": 0.45761641673243886, + "actual_fraction_std": 0.0033251979840081877 + }, + { + "budget": 0.47, + "metric": 0.8811939441773695, + "metric_std": 0.004825358563325134, + "actual_fraction": 0.4674320154983138, + "actual_fraction_std": 0.003852792465544878 + }, + { + "budget": 0.48, + "metric": 0.8814522494080504, + "metric_std": 0.004776836478140763, + "actual_fraction": 0.4774341680419029, + "actual_fraction_std": 0.003667404803806572 + }, + { + "budget": 0.49, + "metric": 0.8817249049293248, + "metric_std": 0.005027877793400059, + "actual_fraction": 0.4874937217478655, + "actual_fraction_std": 0.0037031674915307534 + }, + { + "budget": 0.5, + "metric": 0.8819401592882257, + "metric_std": 0.004848030693939284, + "actual_fraction": 0.49618999784745643, + "actual_fraction_std": 0.002790292141986562 + }, + { + "budget": 0.51, + "metric": 0.8821841142283132, + "metric_std": 0.004956825936138045, + "actual_fraction": 0.5051158785965415, + "actual_fraction_std": 0.0020258829293627874 + }, + { + "budget": 0.52, + "metric": 0.8824567697495874, + "metric_std": 0.005098747305286384, + "actual_fraction": 0.5146444715505489, + "actual_fraction_std": 0.0017149948174663685 + }, + { + "budget": 0.53, + "metric": 0.882930329339169, + "metric_std": 0.004884529896374357, + "actual_fraction": 0.5252062854272799, + "actual_fraction_std": 0.0023362784331575227 + }, + { + "budget": 0.54, + "metric": 0.8831168831168832, + "metric_std": 0.004787763218900264, + "actual_fraction": 0.5351940876802755, + "actual_fraction_std": 0.002693309087913251 + }, + { + "budget": 0.55, + "metric": 0.8833464877663773, + "metric_std": 0.0047155500331957645, + "actual_fraction": 0.5455693477792926, + "actual_fraction_std": 0.0022442267091589115 + }, + { + "budget": 0.56, + "metric": 0.8836334935782449, + "metric_std": 0.004694486519520725, + "actual_fraction": 0.5559446078783096, + "actual_fraction_std": 0.0017731466666492239 + }, + { + "budget": 0.57, + "metric": 0.8838774485183325, + "metric_std": 0.004597303225594276, + "actual_fraction": 0.566822128148095, + "actual_fraction_std": 0.0013777400040487094 + }, + { + "budget": 0.58, + "metric": 0.8841644543302001, + "metric_std": 0.004522789534754125, + "actual_fraction": 0.5766520772045634, + "actual_fraction_std": 0.0011629491787918075 + }, + { + "budget": 0.59, + "metric": 0.8845519121762215, + "metric_std": 0.004523472460498136, + "actual_fraction": 0.5875152471837555, + "actual_fraction_std": 0.0011417277337112091 + }, + { + "budget": 0.6, + "metric": 0.8850111214752099, + "metric_std": 0.004217679551151039, + "actual_fraction": 0.5974886991461578, + "actual_fraction_std": 0.0020328593492908607 + }, + { + "budget": 0.61, + "metric": 0.8853842290306378, + "metric_std": 0.004054306310325571, + "actual_fraction": 0.6083518691253499, + "actual_fraction_std": 0.001376992451088613 + }, + { + "budget": 0.62, + "metric": 0.8859008394919996, + "metric_std": 0.004111927969652147, + "actual_fraction": 0.6168902920284135, + "actual_fraction_std": 0.0015527186836776074 + }, + { + "budget": 0.63, + "metric": 0.8861447944320873, + "metric_std": 0.004107543515361389, + "actual_fraction": 0.6253713137691038, + "actual_fraction_std": 0.001399061340788551 + }, + { + "budget": 0.64, + "metric": 0.8867905575087895, + "metric_std": 0.004049478089621405, + "actual_fraction": 0.6351582119537921, + "actual_fraction_std": 0.0014699419512015268 + }, + { + "budget": 0.65, + "metric": 0.8873215182607448, + "metric_std": 0.003940977538814655, + "actual_fraction": 0.6448016072325464, + "actual_fraction_std": 0.0016961290742159413 + }, + { + "budget": 0.66, + "metric": 0.8879672813374471, + "metric_std": 0.004145967586024287, + "actual_fraction": 0.6547463586137618, + "actual_fraction_std": 0.0015019894651612162 + }, + { + "budget": 0.67, + "metric": 0.8881825356963479, + "metric_std": 0.003965071246339405, + "actual_fraction": 0.665049867259812, + "actual_fraction_std": 0.0017607636973555054 + }, + { + "budget": 0.68, + "metric": 0.8886417449953361, + "metric_std": 0.0037814445269574127, + "actual_fraction": 0.6760852407261246, + "actual_fraction_std": 0.0012309086709187812 + }, + { + "budget": 0.69, + "metric": 0.8890866040037311, + "metric_std": 0.00378566268844358, + "actual_fraction": 0.6852263758341107, + "actual_fraction_std": 0.0018243785800518028 + }, + { + "budget": 0.7, + "metric": 0.8895027624309393, + "metric_std": 0.003788585429416403, + "actual_fraction": 0.6960034440697425, + "actual_fraction_std": 0.0017482930230372441 + }, + { + "budget": 0.71, + "metric": 0.8898184688239936, + "metric_std": 0.0037515787700155043, + "actual_fraction": 0.7068235631771544, + "actual_fraction_std": 0.0009327688885699768 + }, + { + "budget": 0.72, + "metric": 0.8903781301571356, + "metric_std": 0.0033217125723546716, + "actual_fraction": 0.7165100093276889, + "actual_fraction_std": 0.0021674402621788386 + }, + { + "budget": 0.73, + "metric": 0.890837339456124, + "metric_std": 0.0033629929733933026, + "actual_fraction": 0.726440410418311, + "actual_fraction_std": 0.0022738532424448317 + }, + { + "budget": 0.74, + "metric": 0.891397000789266, + "metric_std": 0.0031658575439170627, + "actual_fraction": 0.7364282126713066, + "actual_fraction_std": 0.002053144977137637 + }, + { + "budget": 0.75, + "metric": 0.8919997129941881, + "metric_std": 0.0031322611720862163, + "actual_fraction": 0.7473918346846523, + "actual_fraction_std": 0.0013255583326574729 + }, + { + "budget": 0.76, + "metric": 0.8926741766520774, + "metric_std": 0.0031021199458683714, + "actual_fraction": 0.7582693549544378, + "actual_fraction_std": 0.002383509398274108 + }, + { + "budget": 0.77, + "metric": 0.893075984788692, + "metric_std": 0.00280308601447976, + "actual_fraction": 0.7678122981990386, + "actual_fraction_std": 0.0025413590799203555 + }, + { + "budget": 0.78, + "metric": 0.8937647987371744, + "metric_std": 0.0028242195850779633, + "actual_fraction": 0.7769390830164311, + "actual_fraction_std": 0.0028508902876508996 + }, + { + "budget": 0.79, + "metric": 0.8941522565831959, + "metric_std": 0.0026379761019713834, + "actual_fraction": 0.7874578460213819, + "actual_fraction_std": 0.002490922763285172 + }, + { + "budget": 0.8, + "metric": 0.8945971155915908, + "metric_std": 0.0026774927397640006, + "actual_fraction": 0.7963119753175001, + "actual_fraction_std": 0.00193954675679476 + }, + { + "budget": 0.81, + "metric": 0.8951424266341392, + "metric_std": 0.0024208008762722933, + "actual_fraction": 0.8059553705962547, + "actual_fraction_std": 0.003033398220382486 + }, + { + "budget": 0.82, + "metric": 0.8954724833177872, + "metric_std": 0.0024062959911286277, + "actual_fraction": 0.8159001219774702, + "actual_fraction_std": 0.0016039298675850624 + }, + { + "budget": 0.83, + "metric": 0.8964196024969505, + "metric_std": 0.002444082413631673, + "actual_fraction": 0.8265193370165745, + "actual_fraction_std": 0.001679198626806534 + }, + { + "budget": 0.84, + "metric": 0.8966492071464447, + "metric_std": 0.002251669887799065, + "actual_fraction": 0.8367797947908444, + "actual_fraction_std": 0.0018380139912633037 + }, + { + "budget": 0.85, + "metric": 0.8974241228384875, + "metric_std": 0.0020348843622191057, + "actual_fraction": 0.8465236421037525, + "actual_fraction_std": 0.0015397333525059387 + }, + { + "budget": 0.86, + "metric": 0.8978976824280693, + "metric_std": 0.0019051328187345282, + "actual_fraction": 0.8573724617923514, + "actual_fraction_std": 0.001814049289581468 + }, + { + "budget": 0.87, + "metric": 0.8982133888211237, + "metric_std": 0.0014480644942358911, + "actual_fraction": 0.8671593599770396, + "actual_fraction_std": 0.0022054673979695 + }, + { + "budget": 0.88, + "metric": 0.8988017507354524, + "metric_std": 0.0014220551500997499, + "actual_fraction": 0.8773050154265624, + "actual_fraction_std": 0.0007744504966966858 + }, + { + "budget": 0.89, + "metric": 0.8995192652651216, + "metric_std": 0.001313855116540832, + "actual_fraction": 0.8870058118676901, + "actual_fraction_std": 0.0015877998331347567 + }, + { + "budget": 0.9, + "metric": 0.9001219774700436, + "metric_std": 0.0009464666684561163, + "actual_fraction": 0.897366721676114, + "actual_fraction_std": 0.0025264256152420952 + }, + { + "budget": 0.91, + "metric": 0.9006672885125923, + "metric_std": 0.0007999563023415816, + "actual_fraction": 0.9073545239291096, + "actual_fraction_std": 0.003035858156984288 + }, + { + "budget": 0.92, + "metric": 0.9014852550764152, + "metric_std": 0.0009613088187539865, + "actual_fraction": 0.9183181459424553, + "actual_fraction_std": 0.002327004821828448 + }, + { + "budget": 0.93, + "metric": 0.9019014135036233, + "metric_std": 0.001460278688944841, + "actual_fraction": 0.9280189423835832, + "actual_fraction_std": 0.002146196185408728 + }, + { + "budget": 0.94, + "metric": 0.9028772332639736, + "metric_std": 0.00123508408545684, + "actual_fraction": 0.9379349931836121, + "actual_fraction_std": 0.0029514190717088583 + }, + { + "budget": 0.95, + "metric": 0.9034799454688958, + "metric_std": 0.0008139916941030037, + "actual_fraction": 0.9483102532826291, + "actual_fraction_std": 0.002992388302891557 + }, + { + "budget": 0.96, + "metric": 0.9044701155198392, + "metric_std": 0.0011686899918541373, + "actual_fraction": 0.958570711056899, + "actual_fraction_std": 0.002265914974388229 + }, + { + "budget": 0.97, + "metric": 0.9051015283059481, + "metric_std": 0.0009445609375770337, + "actual_fraction": 0.9689890220276961, + "actual_fraction_std": 0.0018408128568883331 + }, + { + "budget": 0.98, + "metric": 0.9061778001004519, + "metric_std": 0.001134036197180473, + "actual_fraction": 0.9799526440410418, + "actual_fraction_std": 0.0009714309418713672 + }, + { + "budget": 0.99, + "metric": 0.9070962186984287, + "metric_std": 0.0007583283398502093, + "actual_fraction": 0.9904427064648059, + "actual_fraction_std": 0.0008793578933689391 + }, + { + "budget": 1.0, + "metric": 0.907368874219703, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 5190, + "rescuable": 1087, + "harmful": 383, + "both_wrong": 322, + "unit": "sample", + "total": 6982 + }, + "test": { + "both_correct": 10421, + "rescuable": 2225, + "harmful": 729, + "both_wrong": 562, + "unit": "sample", + "total": 13937 + } + }, + "small": { + "validation": 0.7981953594958464, + "test": 0.8000287005811868 + }, + "large": { + "validation": 0.8990260670295045, + "test": 0.9073688742197029 + }, + "learned_minus_confidence_test": -0.0006314127861088981, + "learned_minus_small_test": 0.10706751811724191, + "learned_minus_large_test": -0.000272655521274201, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_nlvr2_MiniCPM" + }, + { + "key": [ + "nlvr2", + "nlvr2", + "Qwen3.8" + ], + "expert_display": "BEiT3", + "run": "five_seed_nlvr2_Qwen3.8_four_difference", + "validation_rows": 6982, + "test_rows": 13937, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/nlvr2/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 86, + "validation_area_0_25": 0.8312804354053279, + "validation_point": { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8587797192781438, + "metric": 0.9057576625608708, + "rescued": 1134, + "harmed": 383, + "missed_rescue": 20, + "avoided_harm": 25, + "selected_both_correct": 4230, + "selected_both_wrong": 249 + }, + "test_point": { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8580038745784602, + "metric": 0.9024180239649853, + "rescued": 2213, + "harmed": 786, + "missed_rescue": 34, + "avoided_harm": 59, + "selected_both_correct": 8437, + "selected_both_wrong": 522 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7981953594958464, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1154, + "avoided_harm": 408, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5245773792266846, + "operator": "gt", + "actual_fraction": 0.0097393297049556, + "metric": 0.8004869664852478, + "rescued": 26, + "harmed": 10, + "missed_rescue": 1128, + "avoided_harm": 398, + "selected_both_correct": 30, + "selected_both_wrong": 2 + }, + { + "budget": 0.02, + "threshold": -0.5456489324569702, + "operator": "gt", + "actual_fraction": 0.019765110283586365, + "metric": 0.8033514752219995, + "rescued": 51, + "harmed": 15, + "missed_rescue": 1103, + "avoided_harm": 393, + "selected_both_correct": 60, + "selected_both_wrong": 12 + }, + { + "budget": 0.03, + "threshold": -0.5652987360954285, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.8067888857061014, + "rescued": 83, + "harmed": 23, + "missed_rescue": 1071, + "avoided_harm": 385, + "selected_both_correct": 84, + "selected_both_wrong": 19 + }, + { + "budget": 0.04, + "threshold": -0.5851011276245117, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.8087940418218276, + "rescued": 105, + "harmed": 31, + "missed_rescue": 1049, + "avoided_harm": 377, + "selected_both_correct": 117, + "selected_both_wrong": 26 + }, + { + "budget": 0.05, + "threshold": -0.6055658459663391, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.8110856488112289, + "rescued": 125, + "harmed": 35, + "missed_rescue": 1029, + "avoided_harm": 373, + "selected_both_correct": 157, + "selected_both_wrong": 32 + }, + { + "budget": 0.06, + "threshold": -0.6293190717697144, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.8140933829848181, + "rescued": 152, + "harmed": 41, + "missed_rescue": 1002, + "avoided_harm": 367, + "selected_both_correct": 187, + "selected_both_wrong": 38 + }, + { + "budget": 0.07, + "threshold": -0.6477982401847839, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.8179604697794328, + "rescued": 183, + "harmed": 45, + "missed_rescue": 971, + "avoided_harm": 363, + "selected_both_correct": 214, + "selected_both_wrong": 46 + }, + { + "budget": 0.08, + "threshold": -0.6693127155303955, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.8212546548266972, + "rescued": 211, + "harmed": 50, + "missed_rescue": 943, + "avoided_harm": 358, + "selected_both_correct": 246, + "selected_both_wrong": 51 + }, + { + "budget": 0.09, + "threshold": -0.6888871788978577, + "operator": "gt", + "actual_fraction": 0.08951589802348897, + "metric": 0.8239759381266113, + "rescued": 238, + "harmed": 58, + "missed_rescue": 916, + "avoided_harm": 350, + "selected_both_correct": 275, + "selected_both_wrong": 54 + }, + { + "budget": 0.1, + "threshold": -0.7061904072761536, + "operator": "gt", + "actual_fraction": 0.09925522772844457, + "metric": 0.8252649670581496, + "rescued": 255, + "harmed": 66, + "missed_rescue": 899, + "avoided_harm": 342, + "selected_both_correct": 311, + "selected_both_wrong": 61 + }, + { + "budget": 0.11, + "threshold": -0.7233095169067383, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.8289888284159267, + "rescued": 284, + "harmed": 69, + "missed_rescue": 870, + "avoided_harm": 339, + "selected_both_correct": 350, + "selected_both_wrong": 65 + }, + { + "budget": 0.12, + "threshold": -0.7424216866493225, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.8319965625895159, + "rescued": 314, + "harmed": 78, + "missed_rescue": 840, + "avoided_harm": 330, + "selected_both_correct": 377, + "selected_both_wrong": 68 + }, + { + "budget": 0.13, + "threshold": -0.7627776861190796, + "operator": "gt", + "actual_fraction": 0.1297622457748496, + "metric": 0.8344313950157548, + "rescued": 338, + "harmed": 85, + "missed_rescue": 816, + "avoided_harm": 323, + "selected_both_correct": 411, + "selected_both_wrong": 72 + }, + { + "budget": 0.14, + "threshold": -0.7786493301391602, + "operator": "gt", + "actual_fraction": 0.1396448009166428, + "metric": 0.8371526783156689, + "rescued": 366, + "harmed": 94, + "missed_rescue": 788, + "avoided_harm": 314, + "selected_both_correct": 440, + "selected_both_wrong": 75 + }, + { + "budget": 0.15, + "threshold": -0.7950222492218018, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8400171870524205, + "rescued": 390, + "harmed": 98, + "missed_rescue": 764, + "avoided_harm": 310, + "selected_both_correct": 477, + "selected_both_wrong": 82 + }, + { + "budget": 0.16, + "threshold": -0.8140527606010437, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8424520194786594, + "rescued": 412, + "harmed": 103, + "missed_rescue": 742, + "avoided_harm": 305, + "selected_both_correct": 514, + "selected_both_wrong": 88 + }, + { + "budget": 0.17, + "threshold": -0.8327813148498535, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.844313950157548, + "rescued": 429, + "harmed": 107, + "missed_rescue": 725, + "avoided_harm": 301, + "selected_both_correct": 556, + "selected_both_wrong": 94 + }, + { + "budget": 0.18, + "threshold": -0.8464508652687073, + "operator": "gt", + "actual_fraction": 0.17960469779432828, + "metric": 0.8463191062732741, + "rescued": 446, + "harmed": 110, + "missed_rescue": 708, + "avoided_harm": 298, + "selected_both_correct": 600, + "selected_both_wrong": 98 + }, + { + "budget": 0.19, + "threshold": -0.858244001865387, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8490403895731882, + "rescued": 470, + "harmed": 115, + "missed_rescue": 684, + "avoided_harm": 293, + "selected_both_correct": 637, + "selected_both_wrong": 104 + }, + { + "budget": 0.2, + "threshold": -0.8701565861701965, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8509023202520768, + "rescued": 488, + "harmed": 120, + "missed_rescue": 666, + "avoided_harm": 288, + "selected_both_correct": 680, + "selected_both_wrong": 108 + }, + { + "budget": 0.21, + "threshold": -0.8840392827987671, + "operator": "gt", + "actual_fraction": 0.2093955886565454, + "metric": 0.852191349183615, + "rescued": 505, + "harmed": 128, + "missed_rescue": 649, + "avoided_harm": 280, + "selected_both_correct": 718, + "selected_both_wrong": 111 + }, + { + "budget": 0.22, + "threshold": -0.8933094143867493, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8533371526783157, + "rescued": 522, + "harmed": 137, + "missed_rescue": 632, + "avoided_harm": 271, + "selected_both_correct": 762, + "selected_both_wrong": 115 + }, + { + "budget": 0.23, + "threshold": -0.9046505093574524, + "operator": "gt", + "actual_fraction": 0.2297336006874821, + "metric": 0.855628759667717, + "rescued": 543, + "harmed": 142, + "missed_rescue": 611, + "avoided_harm": 266, + "selected_both_correct": 799, + "selected_both_wrong": 120 + }, + { + "budget": 0.24, + "threshold": -0.9136765599250793, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8583500429676311, + "rescued": 566, + "harmed": 146, + "missed_rescue": 588, + "avoided_harm": 262, + "selected_both_correct": 836, + "selected_both_wrong": 127 + }, + { + "budget": 0.25, + "threshold": -0.9230391979217529, + "operator": "gt", + "actual_fraction": 0.24906903466055572, + "metric": 0.860068748209682, + "rescued": 585, + "harmed": 153, + "missed_rescue": 569, + "avoided_harm": 255, + "selected_both_correct": 871, + "selected_both_wrong": 130 + }, + { + "budget": 0.26, + "threshold": -0.9314625263214111, + "operator": "gt", + "actual_fraction": 0.2595244915496992, + "metric": 0.8632197078201088, + "rescued": 610, + "harmed": 156, + "missed_rescue": 544, + "avoided_harm": 252, + "selected_both_correct": 914, + "selected_both_wrong": 132 + }, + { + "budget": 0.27, + "threshold": -0.9376689791679382, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8656545402463478, + "rescued": 630, + "harmed": 159, + "missed_rescue": 524, + "avoided_harm": 249, + "selected_both_correct": 962, + "selected_both_wrong": 134 + }, + { + "budget": 0.28, + "threshold": -0.9449946880340576, + "operator": "gt", + "actual_fraction": 0.2794328272701232, + "metric": 0.8673732454883988, + "rescued": 645, + "harmed": 162, + "missed_rescue": 509, + "avoided_harm": 246, + "selected_both_correct": 1005, + "selected_both_wrong": 139 + }, + { + "budget": 0.29, + "threshold": -0.9511421918869019, + "operator": "gt", + "actual_fraction": 0.2896018332855915, + "metric": 0.8689487252936121, + "rescued": 664, + "harmed": 170, + "missed_rescue": 490, + "avoided_harm": 238, + "selected_both_correct": 1043, + "selected_both_wrong": 145 + }, + { + "budget": 0.3, + "threshold": -0.9559813141822815, + "operator": "gt", + "actual_fraction": 0.29977083930105985, + "metric": 0.870380979661988, + "rescued": 681, + "harmed": 177, + "missed_rescue": 473, + "avoided_harm": 231, + "selected_both_correct": 1083, + "selected_both_wrong": 152 + }, + { + "budget": 0.31, + "threshold": -0.9597620368003845, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8716700085935262, + "rescued": 697, + "harmed": 184, + "missed_rescue": 457, + "avoided_harm": 224, + "selected_both_correct": 1129, + "selected_both_wrong": 154 + }, + { + "budget": 0.32, + "threshold": -0.9643214344978333, + "operator": "gt", + "actual_fraction": 0.3196791750214838, + "metric": 0.8743912918934402, + "rescued": 719, + "harmed": 187, + "missed_rescue": 435, + "avoided_harm": 221, + "selected_both_correct": 1171, + "selected_both_wrong": 155 + }, + { + "budget": 0.33, + "threshold": -0.9686195850372314, + "operator": "gt", + "actual_fraction": 0.3297049556001146, + "metric": 0.8746777427671154, + "rescued": 731, + "harmed": 197, + "missed_rescue": 423, + "avoided_harm": 211, + "selected_both_correct": 1215, + "selected_both_wrong": 159 + }, + { + "budget": 0.34, + "threshold": -0.9719924926757812, + "operator": "gt", + "actual_fraction": 0.339157834431395, + "metric": 0.8771125751933544, + "rescued": 751, + "harmed": 200, + "missed_rescue": 403, + "avoided_harm": 208, + "selected_both_correct": 1256, + "selected_both_wrong": 161 + }, + { + "budget": 0.35, + "threshold": -0.974821150302887, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8786880549985677, + "rescued": 767, + "harmed": 205, + "missed_rescue": 387, + "avoided_harm": 203, + "selected_both_correct": 1306, + "selected_both_wrong": 165 + }, + { + "budget": 0.36, + "threshold": -0.9777138829231262, + "operator": "gt", + "actual_fraction": 0.3587797192781438, + "metric": 0.8814093382984818, + "rescued": 786, + "harmed": 205, + "missed_rescue": 368, + "avoided_harm": 203, + "selected_both_correct": 1345, + "selected_both_wrong": 169 + }, + { + "budget": 0.37, + "threshold": -0.9802808165550232, + "operator": "gt", + "actual_fraction": 0.3685190489830994, + "metric": 0.881695789172157, + "rescued": 797, + "harmed": 214, + "missed_rescue": 357, + "avoided_harm": 194, + "selected_both_correct": 1390, + "selected_both_wrong": 172 + }, + { + "budget": 0.38, + "threshold": -0.982557475566864, + "operator": "gt", + "actual_fraction": 0.3795474076195932, + "metric": 0.8839873961615583, + "rescued": 817, + "harmed": 218, + "missed_rescue": 337, + "avoided_harm": 190, + "selected_both_correct": 1439, + "selected_both_wrong": 176 + }, + { + "budget": 0.39, + "threshold": -0.9843363761901855, + "operator": "gt", + "actual_fraction": 0.3892867373245488, + "metric": 0.8847035233457462, + "rescued": 827, + "harmed": 223, + "missed_rescue": 327, + "avoided_harm": 185, + "selected_both_correct": 1487, + "selected_both_wrong": 181 + }, + { + "budget": 0.4, + "threshold": -0.9863631129264832, + "operator": "gt", + "actual_fraction": 0.3993125179031796, + "metric": 0.8864222285877972, + "rescued": 843, + "harmed": 227, + "missed_rescue": 311, + "avoided_harm": 181, + "selected_both_correct": 1536, + "selected_both_wrong": 182 + }, + { + "budget": 0.41, + "threshold": -0.9879462122917175, + "operator": "gt", + "actual_fraction": 0.40962474935548554, + "metric": 0.8879977083930106, + "rescued": 857, + "harmed": 230, + "missed_rescue": 297, + "avoided_harm": 178, + "selected_both_correct": 1586, + "selected_both_wrong": 187 + }, + { + "budget": 0.42, + "threshold": -0.9890130758285522, + "operator": "gt", + "actual_fraction": 0.41850472643941566, + "metric": 0.8897164136350616, + "rescued": 872, + "harmed": 233, + "missed_rescue": 282, + "avoided_harm": 175, + "selected_both_correct": 1628, + "selected_both_wrong": 189 + }, + { + "budget": 0.43, + "threshold": -0.9899863600730896, + "operator": "gt", + "actual_fraction": 0.4270982526496706, + "metric": 0.890575766256087, + "rescued": 882, + "harmed": 237, + "missed_rescue": 272, + "avoided_harm": 171, + "selected_both_correct": 1674, + "selected_both_wrong": 189 + }, + { + "budget": 0.44, + "threshold": -0.9911527037620544, + "operator": "gt", + "actual_fraction": 0.4395588656545402, + "metric": 0.8928673732454884, + "rescued": 903, + "harmed": 242, + "missed_rescue": 251, + "avoided_harm": 166, + "selected_both_correct": 1728, + "selected_both_wrong": 196 + }, + { + "budget": 0.45, + "threshold": -0.9919379353523254, + "operator": "gt", + "actual_fraction": 0.4497278716700086, + "metric": 0.8937267258665139, + "rescued": 913, + "harmed": 246, + "missed_rescue": 241, + "avoided_harm": 162, + "selected_both_correct": 1784, + "selected_both_wrong": 197 + }, + { + "budget": 0.46, + "threshold": -0.9926542043685913, + "operator": "gt", + "actual_fraction": 0.4586078487539387, + "metric": 0.8953022056717272, + "rescued": 925, + "harmed": 247, + "missed_rescue": 229, + "avoided_harm": 161, + "selected_both_correct": 1832, + "selected_both_wrong": 198 + }, + { + "budget": 0.47, + "threshold": -0.9933071732521057, + "operator": "gt", + "actual_fraction": 0.46863362933256947, + "metric": 0.8961615582927528, + "rescued": 935, + "harmed": 251, + "missed_rescue": 219, + "avoided_harm": 157, + "selected_both_correct": 1886, + "selected_both_wrong": 200 + }, + { + "budget": 0.48, + "threshold": -0.9939024448394775, + "operator": "gt", + "actual_fraction": 0.4798052134059009, + "metric": 0.8960183328559153, + "rescued": 941, + "harmed": 258, + "missed_rescue": 213, + "avoided_harm": 150, + "selected_both_correct": 1948, + "selected_both_wrong": 203 + }, + { + "budget": 0.49, + "threshold": -0.9944451451301575, + "operator": "gt", + "actual_fraction": 0.4891148668003437, + "metric": 0.8970209109137783, + "rescued": 953, + "harmed": 263, + "missed_rescue": 201, + "avoided_harm": 145, + "selected_both_correct": 1992, + "selected_both_wrong": 207 + }, + { + "budget": 0.5, + "threshold": -0.9949396848678589, + "operator": "gt", + "actual_fraction": 0.4992838728158121, + "metric": 0.897450587224291, + "rescued": 960, + "harmed": 267, + "missed_rescue": 194, + "avoided_harm": 141, + "selected_both_correct": 2048, + "selected_both_wrong": 211 + }, + { + "budget": 0.51, + "threshold": -0.9952448010444641, + "operator": "gt", + "actual_fraction": 0.5065883700945288, + "metric": 0.8978802635348038, + "rescued": 965, + "harmed": 269, + "missed_rescue": 189, + "avoided_harm": 139, + "selected_both_correct": 2091, + "selected_both_wrong": 212 + }, + { + "budget": 0.52, + "threshold": -0.9956685304641724, + "operator": "gt", + "actual_fraction": 0.5179031796046978, + "metric": 0.8977370380979662, + "rescued": 972, + "harmed": 277, + "missed_rescue": 182, + "avoided_harm": 131, + "selected_both_correct": 2154, + "selected_both_wrong": 213 + }, + { + "budget": 0.53, + "threshold": -0.9961755275726318, + "operator": "gt", + "actual_fraction": 0.528788312804354, + "metric": 0.8980234889716414, + "rescued": 980, + "harmed": 283, + "missed_rescue": 174, + "avoided_harm": 125, + "selected_both_correct": 2214, + "selected_both_wrong": 215 + }, + { + "budget": 0.54, + "threshold": -0.9965165853500366, + "operator": "gt", + "actual_fraction": 0.5380979661987969, + "metric": 0.8988828415926668, + "rescued": 989, + "harmed": 286, + "missed_rescue": 165, + "avoided_harm": 122, + "selected_both_correct": 2267, + "selected_both_wrong": 215 + }, + { + "budget": 0.55, + "threshold": -0.9968273043632507, + "operator": "gt", + "actual_fraction": 0.5489830993984531, + "metric": 0.898166714408479, + "rescued": 993, + "harmed": 295, + "missed_rescue": 161, + "avoided_harm": 113, + "selected_both_correct": 2330, + "selected_both_wrong": 215 + }, + { + "budget": 0.56, + "threshold": -0.9971103668212891, + "operator": "gt", + "actual_fraction": 0.5592953308507591, + "metric": 0.8987396161558293, + "rescued": 1001, + "harmed": 299, + "missed_rescue": 153, + "avoided_harm": 109, + "selected_both_correct": 2387, + "selected_both_wrong": 218 + }, + { + "budget": 0.57, + "threshold": -0.997285008430481, + "operator": "gt", + "actual_fraction": 0.5690346605557147, + "metric": 0.899169292466342, + "rescued": 1008, + "harmed": 303, + "missed_rescue": 146, + "avoided_harm": 105, + "selected_both_correct": 2440, + "selected_both_wrong": 222 + }, + { + "budget": 0.58, + "threshold": -0.9974491000175476, + "operator": "gt", + "actual_fraction": 0.5783443139501575, + "metric": 0.899169292466342, + "rescued": 1012, + "harmed": 307, + "missed_rescue": 142, + "avoided_harm": 101, + "selected_both_correct": 2492, + "selected_both_wrong": 227 + }, + { + "budget": 0.59, + "threshold": -0.9976769089698792, + "operator": "gt", + "actual_fraction": 0.5893726725866514, + "metric": 0.899169292466342, + "rescued": 1017, + "harmed": 312, + "missed_rescue": 137, + "avoided_harm": 96, + "selected_both_correct": 2558, + "selected_both_wrong": 228 + }, + { + "budget": 0.6, + "threshold": -0.9978172779083252, + "operator": "gt", + "actual_fraction": 0.5976797479232312, + "metric": 0.8998854196505299, + "rescued": 1024, + "harmed": 314, + "missed_rescue": 130, + "avoided_harm": 94, + "selected_both_correct": 2606, + "selected_both_wrong": 229 + }, + { + "budget": 0.61, + "threshold": -0.9980121850967407, + "operator": "gt", + "actual_fraction": 0.6084216556860499, + "metric": 0.9007447722715555, + "rescued": 1033, + "harmed": 317, + "missed_rescue": 121, + "avoided_harm": 91, + "selected_both_correct": 2669, + "selected_both_wrong": 229 + }, + { + "budget": 0.62, + "threshold": -0.9981897473335266, + "operator": "gt", + "actual_fraction": 0.6187338871383558, + "metric": 0.9004583213978803, + "rescued": 1036, + "harmed": 322, + "missed_rescue": 118, + "avoided_harm": 86, + "selected_both_correct": 2733, + "selected_both_wrong": 229 + }, + { + "budget": 0.63, + "threshold": -0.9982993006706238, + "operator": "gt", + "actual_fraction": 0.6280435405327986, + "metric": 0.9014608994557434, + "rescued": 1047, + "harmed": 326, + "missed_rescue": 107, + "avoided_harm": 82, + "selected_both_correct": 2783, + "selected_both_wrong": 229 + }, + { + "budget": 0.64, + "threshold": -0.9984021782875061, + "operator": "gt", + "actual_fraction": 0.6360641649957033, + "metric": 0.9018905757662561, + "rescued": 1054, + "harmed": 330, + "missed_rescue": 100, + "avoided_harm": 78, + "selected_both_correct": 2828, + "selected_both_wrong": 229 + }, + { + "budget": 0.65, + "threshold": -0.9985449314117432, + "operator": "gt", + "actual_fraction": 0.6465196218848468, + "metric": 0.9021770266399313, + "rescued": 1057, + "harmed": 331, + "missed_rescue": 97, + "avoided_harm": 77, + "selected_both_correct": 2897, + "selected_both_wrong": 229 + }, + { + "budget": 0.66, + "threshold": -0.9986749291419983, + "operator": "gt", + "actual_fraction": 0.6586937840160413, + "metric": 0.9024634775136064, + "rescued": 1065, + "harmed": 337, + "missed_rescue": 89, + "avoided_harm": 71, + "selected_both_correct": 2965, + "selected_both_wrong": 232 + }, + { + "budget": 0.67, + "threshold": -0.9987550973892212, + "operator": "gt", + "actual_fraction": 0.6672873102262962, + "metric": 0.902606702950444, + "rescued": 1069, + "harmed": 340, + "missed_rescue": 85, + "avoided_harm": 68, + "selected_both_correct": 3017, + "selected_both_wrong": 233 + }, + { + "budget": 0.68, + "threshold": -0.9988664388656616, + "operator": "gt", + "actual_fraction": 0.6793182469206531, + "metric": 0.9034660555714695, + "rescued": 1079, + "harmed": 344, + "missed_rescue": 75, + "avoided_harm": 64, + "selected_both_correct": 3086, + "selected_both_wrong": 234 + }, + { + "budget": 0.69, + "threshold": -0.9989677667617798, + "operator": "gt", + "actual_fraction": 0.6894872529361215, + "metric": 0.9038957318819822, + "rescued": 1085, + "harmed": 347, + "missed_rescue": 69, + "avoided_harm": 61, + "selected_both_correct": 3147, + "selected_both_wrong": 235 + }, + { + "budget": 0.7, + "threshold": -0.9990302324295044, + "operator": "gt", + "actual_fraction": 0.6969349756516757, + "metric": 0.9033228301346319, + "rescued": 1085, + "harmed": 351, + "missed_rescue": 69, + "avoided_harm": 57, + "selected_both_correct": 3195, + "selected_both_wrong": 235 + }, + { + "budget": 0.71, + "threshold": -0.9991169571876526, + "operator": "gt", + "actual_fraction": 0.7098252649670581, + "metric": 0.9037525064451447, + "rescued": 1091, + "harmed": 354, + "missed_rescue": 63, + "avoided_harm": 54, + "selected_both_correct": 3274, + "selected_both_wrong": 237 + }, + { + "budget": 0.72, + "threshold": -0.9991704225540161, + "operator": "gt", + "actual_fraction": 0.7189916929246635, + "metric": 0.9040389573188198, + "rescued": 1094, + "harmed": 355, + "missed_rescue": 60, + "avoided_harm": 53, + "selected_both_correct": 3334, + "selected_both_wrong": 237 + }, + { + "budget": 0.73, + "threshold": -0.9992445707321167, + "operator": "gt", + "actual_fraction": 0.7295903752506445, + "metric": 0.9038957318819822, + "rescued": 1098, + "harmed": 360, + "missed_rescue": 56, + "avoided_harm": 48, + "selected_both_correct": 3399, + "selected_both_wrong": 237 + }, + { + "budget": 0.74, + "threshold": -0.9992903470993042, + "operator": "gt", + "actual_fraction": 0.7384703523345746, + "metric": 0.9040389573188198, + "rescued": 1103, + "harmed": 364, + "missed_rescue": 51, + "avoided_harm": 44, + "selected_both_correct": 3450, + "selected_both_wrong": 239 + }, + { + "budget": 0.75, + "threshold": -0.9993333220481873, + "operator": "gt", + "actual_fraction": 0.7484961329132054, + "metric": 0.9044686336293326, + "rescued": 1108, + "harmed": 366, + "missed_rescue": 46, + "avoided_harm": 42, + "selected_both_correct": 3513, + "selected_both_wrong": 239 + }, + { + "budget": 0.76, + "threshold": -0.999373733997345, + "operator": "gt", + "actual_fraction": 0.7553709538814093, + "metric": 0.9041821827556574, + "rescued": 1109, + "harmed": 369, + "missed_rescue": 45, + "avoided_harm": 39, + "selected_both_correct": 3557, + "selected_both_wrong": 239 + }, + { + "budget": 0.77, + "threshold": -0.9994296431541443, + "operator": "gt", + "actual_fraction": 0.7694070466914924, + "metric": 0.9047550845030078, + "rescued": 1115, + "harmed": 371, + "missed_rescue": 39, + "avoided_harm": 37, + "selected_both_correct": 3645, + "selected_both_wrong": 241 + }, + { + "budget": 0.78, + "threshold": -0.999464213848114, + "operator": "gt", + "actual_fraction": 0.7782870237754225, + "metric": 0.9050415353766829, + "rescued": 1117, + "harmed": 371, + "missed_rescue": 37, + "avoided_harm": 37, + "selected_both_correct": 3702, + "selected_both_wrong": 244 + }, + { + "budget": 0.79, + "threshold": -0.9994966983795166, + "operator": "gt", + "actual_fraction": 0.7885992552277284, + "metric": 0.9051847608135205, + "rescued": 1119, + "harmed": 372, + "missed_rescue": 35, + "avoided_harm": 36, + "selected_both_correct": 3770, + "selected_both_wrong": 245 + }, + { + "budget": 0.8, + "threshold": -0.9995417594909668, + "operator": "gt", + "actual_fraction": 0.7977656831853337, + "metric": 0.9048983099398453, + "rescued": 1120, + "harmed": 375, + "missed_rescue": 34, + "avoided_harm": 33, + "selected_both_correct": 3829, + "selected_both_wrong": 246 + }, + { + "budget": 0.81, + "threshold": -0.9995827078819275, + "operator": "gt", + "actual_fraction": 0.8090804926955028, + "metric": 0.9050415353766829, + "rescued": 1123, + "harmed": 377, + "missed_rescue": 31, + "avoided_harm": 31, + "selected_both_correct": 3903, + "selected_both_wrong": 246 + }, + { + "budget": 0.82, + "threshold": -0.9996140599250793, + "operator": "gt", + "actual_fraction": 0.8198224004583214, + "metric": 0.905327986250358, + "rescued": 1127, + "harmed": 379, + "missed_rescue": 27, + "avoided_harm": 29, + "selected_both_correct": 3972, + "selected_both_wrong": 246 + }, + { + "budget": 0.83, + "threshold": -0.9996429681777954, + "operator": "gt", + "actual_fraction": 0.8281294757949011, + "metric": 0.9051847608135205, + "rescued": 1128, + "harmed": 381, + "missed_rescue": 26, + "avoided_harm": 27, + "selected_both_correct": 4027, + "selected_both_wrong": 246 + }, + { + "budget": 0.84, + "threshold": -0.9996750354766846, + "operator": "gt", + "actual_fraction": 0.8395875107419077, + "metric": 0.9054712116871957, + "rescued": 1131, + "harmed": 382, + "missed_rescue": 23, + "avoided_harm": 26, + "selected_both_correct": 4102, + "selected_both_wrong": 247 + }, + { + "budget": 0.85, + "threshold": -0.9997040629386902, + "operator": "gt", + "actual_fraction": 0.8458894299627614, + "metric": 0.9056144371240332, + "rescued": 1132, + "harmed": 382, + "missed_rescue": 22, + "avoided_harm": 26, + "selected_both_correct": 4143, + "selected_both_wrong": 249 + }, + { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8587797192781438, + "metric": 0.9057576625608708, + "rescued": 1134, + "harmed": 383, + "missed_rescue": 20, + "avoided_harm": 25, + "selected_both_correct": 4230, + "selected_both_wrong": 249 + }, + { + "budget": 0.87, + "threshold": -0.9997695088386536, + "operator": "gt", + "actual_fraction": 0.8693784016041249, + "metric": 0.9057576625608708, + "rescued": 1136, + "harmed": 385, + "missed_rescue": 18, + "avoided_harm": 23, + "selected_both_correct": 4300, + "selected_both_wrong": 249 + }, + { + "budget": 0.88, + "threshold": -0.999783456325531, + "operator": "gt", + "actual_fraction": 0.8749641936407906, + "metric": 0.9057576625608708, + "rescued": 1136, + "harmed": 385, + "missed_rescue": 18, + "avoided_harm": 23, + "selected_both_correct": 4339, + "selected_both_wrong": 249 + }, + { + "budget": 0.89, + "threshold": -0.9998205304145813, + "operator": "gt", + "actual_fraction": 0.8894299627613864, + "metric": 0.905327986250358, + "rescued": 1137, + "harmed": 389, + "missed_rescue": 17, + "avoided_harm": 19, + "selected_both_correct": 4434, + "selected_both_wrong": 250 + }, + { + "budget": 0.9, + "threshold": -0.999841570854187, + "operator": "gt", + "actual_fraction": 0.8988828415926668, + "metric": 0.905327986250358, + "rescued": 1140, + "harmed": 392, + "missed_rescue": 14, + "avoided_harm": 16, + "selected_both_correct": 4494, + "selected_both_wrong": 250 + }, + { + "budget": 0.91, + "threshold": -0.9998601675033569, + "operator": "gt", + "actual_fraction": 0.9093382984818104, + "metric": 0.9054712116871957, + "rescued": 1143, + "harmed": 394, + "missed_rescue": 11, + "avoided_harm": 14, + "selected_both_correct": 4562, + "selected_both_wrong": 250 + }, + { + "budget": 0.92, + "threshold": -0.9998766183853149, + "operator": "gt", + "actual_fraction": 0.9185047264394156, + "metric": 0.9051847608135205, + "rescued": 1144, + "harmed": 397, + "missed_rescue": 10, + "avoided_harm": 11, + "selected_both_correct": 4620, + "selected_both_wrong": 252 + }, + { + "budget": 0.93, + "threshold": -0.9998977184295654, + "operator": "gt", + "actual_fraction": 0.9276711543970209, + "metric": 0.905327986250358, + "rescued": 1145, + "harmed": 397, + "missed_rescue": 9, + "avoided_harm": 11, + "selected_both_correct": 4683, + "selected_both_wrong": 252 + }, + { + "budget": 0.94, + "threshold": -0.9999151229858398, + "operator": "gt", + "actual_fraction": 0.9394156402177026, + "metric": 0.905327986250358, + "rescued": 1147, + "harmed": 399, + "missed_rescue": 7, + "avoided_harm": 9, + "selected_both_correct": 4760, + "selected_both_wrong": 253 + }, + { + "budget": 0.95, + "threshold": -0.999931812286377, + "operator": "gt", + "actual_fraction": 0.9497278716700086, + "metric": 0.9051847608135205, + "rescued": 1148, + "harmed": 401, + "missed_rescue": 6, + "avoided_harm": 7, + "selected_both_correct": 4829, + "selected_both_wrong": 253 + }, + { + "budget": 0.96, + "threshold": -0.9999452829360962, + "operator": "gt", + "actual_fraction": 0.9590375250644515, + "metric": 0.9054712116871957, + "rescued": 1151, + "harmed": 402, + "missed_rescue": 3, + "avoided_harm": 6, + "selected_both_correct": 4889, + "selected_both_wrong": 254 + }, + { + "budget": 0.97, + "threshold": -0.9999599456787109, + "operator": "gt", + "actual_fraction": 0.969492981953595, + "metric": 0.9050415353766829, + "rescued": 1151, + "harmed": 405, + "missed_rescue": 3, + "avoided_harm": 3, + "selected_both_correct": 4959, + "selected_both_wrong": 254 + }, + { + "budget": 0.98, + "threshold": -0.9999687671661377, + "operator": "gt", + "actual_fraction": 0.9783729590375251, + "metric": 0.9047550845030078, + "rescued": 1151, + "harmed": 407, + "missed_rescue": 3, + "avoided_harm": 1, + "selected_both_correct": 5019, + "selected_both_wrong": 254 + }, + { + "budget": 0.99, + "threshold": -0.9999842643737793, + "operator": "gt", + "actual_fraction": 0.9898309939845317, + "metric": 0.9048983099398453, + "rescued": 1153, + "harmed": 408, + "missed_rescue": 1, + "avoided_harm": 0, + "selected_both_correct": 5096, + "selected_both_wrong": 254 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9050415353766829, + "rescued": 1154, + "harmed": 408, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 5165, + "selected_both_wrong": 255 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.8000287005811868, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2247, + "avoided_harm": 845, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5245773792266846, + "operator": "gt", + "actual_fraction": 0.011623735380641457, + "metric": 0.8039032790414006, + "rescued": 70, + "harmed": 16, + "missed_rescue": 2177, + "avoided_harm": 829, + "selected_both_correct": 66, + "selected_both_wrong": 10 + }, + { + "budget": 0.02, + "threshold": -0.5456489324569702, + "operator": "gt", + "actual_fraction": 0.021453684437109852, + "metric": 0.8067733371600775, + "rescued": 122, + "harmed": 28, + "missed_rescue": 2125, + "avoided_harm": 817, + "selected_both_correct": 124, + "selected_both_wrong": 25 + }, + { + "budget": 0.03, + "threshold": -0.5652987360954285, + "operator": "ge", + "actual_fraction": 0.02970510152830595, + "metric": 0.8093563894668867, + "rescued": 169, + "harmed": 39, + "missed_rescue": 2078, + "avoided_harm": 806, + "selected_both_correct": 169, + "selected_both_wrong": 37 + }, + { + "budget": 0.04, + "threshold": -0.5851011276245117, + "operator": "ge", + "actual_fraction": 0.03795651861950204, + "metric": 0.8113654301499605, + "rescued": 209, + "harmed": 51, + "missed_rescue": 2038, + "avoided_harm": 794, + "selected_both_correct": 224, + "selected_both_wrong": 45 + }, + { + "budget": 0.05, + "threshold": -0.6055658459663391, + "operator": "ge", + "actual_fraction": 0.047212456052235056, + "metric": 0.8140202339097367, + "rescued": 259, + "harmed": 64, + "missed_rescue": 1988, + "avoided_harm": 781, + "selected_both_correct": 283, + "selected_both_wrong": 52 + }, + { + "budget": 0.06, + "threshold": -0.6293190717697144, + "operator": "ge", + "actual_fraction": 0.057759919638372674, + "metric": 0.817464303652149, + "rescued": 318, + "harmed": 75, + "missed_rescue": 1929, + "avoided_harm": 770, + "selected_both_correct": 345, + "selected_both_wrong": 67 + }, + { + "budget": 0.07, + "threshold": -0.6477982401847839, + "operator": "ge", + "actual_fraction": 0.06601133672956877, + "metric": 0.8201908588648921, + "rescued": 365, + "harmed": 84, + "missed_rescue": 1882, + "avoided_harm": 761, + "selected_both_correct": 399, + "selected_both_wrong": 72 + }, + { + "budget": 0.08, + "threshold": -0.6693127155303955, + "operator": "ge", + "actual_fraction": 0.07483676544450026, + "metric": 0.8232761713424697, + "rescued": 417, + "harmed": 93, + "missed_rescue": 1830, + "avoided_harm": 752, + "selected_both_correct": 451, + "selected_both_wrong": 82 + }, + { + "budget": 0.09, + "threshold": -0.6888871788978577, + "operator": "gt", + "actual_fraction": 0.08488196885986941, + "metric": 0.8261462294611466, + "rescued": 464, + "harmed": 100, + "missed_rescue": 1783, + "avoided_harm": 745, + "selected_both_correct": 524, + "selected_both_wrong": 95 + }, + { + "budget": 0.1, + "threshold": -0.7061904072761536, + "operator": "gt", + "actual_fraction": 0.09356389466886704, + "metric": 0.8290162875798235, + "rescued": 511, + "harmed": 107, + "missed_rescue": 1736, + "avoided_harm": 738, + "selected_both_correct": 578, + "selected_both_wrong": 108 + }, + { + "budget": 0.11, + "threshold": -0.7233095169067383, + "operator": "ge", + "actual_fraction": 0.1026045777426993, + "metric": 0.8319580971514673, + "rescued": 558, + "harmed": 113, + "missed_rescue": 1689, + "avoided_harm": 732, + "selected_both_correct": 643, + "selected_both_wrong": 116 + }, + { + "budget": 0.12, + "threshold": -0.7424216866493225, + "operator": "ge", + "actual_fraction": 0.11322379278180383, + "metric": 0.8348999067231111, + "rescued": 617, + "harmed": 131, + "missed_rescue": 1630, + "avoided_harm": 714, + "selected_both_correct": 708, + "selected_both_wrong": 122 + }, + { + "budget": 0.13, + "threshold": -0.7627776861190796, + "operator": "gt", + "actual_fraction": 0.12255148166750376, + "metric": 0.8375547104828873, + "rescued": 664, + "harmed": 141, + "missed_rescue": 1583, + "avoided_harm": 704, + "selected_both_correct": 775, + "selected_both_wrong": 128 + }, + { + "budget": 0.14, + "threshold": -0.7786493301391602, + "operator": "gt", + "actual_fraction": 0.13137691038243524, + "metric": 0.839635502618928, + "rescued": 705, + "harmed": 153, + "missed_rescue": 1542, + "avoided_harm": 692, + "selected_both_correct": 828, + "selected_both_wrong": 145 + }, + { + "budget": 0.15, + "threshold": -0.7950222492218018, + "operator": "ge", + "actual_fraction": 0.14056109636220132, + "metric": 0.8421468034727704, + "rescued": 753, + "harmed": 166, + "missed_rescue": 1494, + "avoided_harm": 679, + "selected_both_correct": 884, + "selected_both_wrong": 156 + }, + { + "budget": 0.16, + "threshold": -0.8140527606010437, + "operator": "ge", + "actual_fraction": 0.15096505704240512, + "metric": 0.8448733586855134, + "rescued": 804, + "harmed": 179, + "missed_rescue": 1443, + "avoided_harm": 666, + "selected_both_correct": 955, + "selected_both_wrong": 166 + }, + { + "budget": 0.17, + "threshold": -0.8327813148498535, + "operator": "ge", + "actual_fraction": 0.1613690177226089, + "metric": 0.8468823993685872, + "rescued": 844, + "harmed": 191, + "missed_rescue": 1403, + "avoided_harm": 654, + "selected_both_correct": 1033, + "selected_both_wrong": 181 + }, + { + "budget": 0.18, + "threshold": -0.8464508652687073, + "operator": "gt", + "actual_fraction": 0.1702661978905073, + "metric": 0.8490349429575949, + "rescued": 882, + "harmed": 199, + "missed_rescue": 1365, + "avoided_harm": 646, + "selected_both_correct": 1102, + "selected_both_wrong": 190 + }, + { + "budget": 0.19, + "threshold": -0.858244001865387, + "operator": "ge", + "actual_fraction": 0.17916337805840568, + "metric": 0.851761498170338, + "rescued": 932, + "harmed": 211, + "missed_rescue": 1315, + "avoided_harm": 634, + "selected_both_correct": 1158, + "selected_both_wrong": 196 + }, + { + "budget": 0.2, + "threshold": -0.8701565861701965, + "operator": "ge", + "actual_fraction": 0.18798880677333715, + "metric": 0.8539857932123125, + "rescued": 972, + "harmed": 220, + "missed_rescue": 1275, + "avoided_harm": 625, + "selected_both_correct": 1228, + "selected_both_wrong": 200 + }, + { + "budget": 0.21, + "threshold": -0.8840392827987671, + "operator": "gt", + "actual_fraction": 0.19659898112936788, + "metric": 0.855420822271651, + "rescued": 1004, + "harmed": 232, + "missed_rescue": 1243, + "avoided_harm": 613, + "selected_both_correct": 1293, + "selected_both_wrong": 211 + }, + { + "budget": 0.22, + "threshold": -0.8933094143867493, + "operator": "ge", + "actual_fraction": 0.20642893018583627, + "metric": 0.8580038745784602, + "rescued": 1047, + "harmed": 239, + "missed_rescue": 1200, + "avoided_harm": 606, + "selected_both_correct": 1373, + "selected_both_wrong": 218 + }, + { + "budget": 0.23, + "threshold": -0.9046505093574524, + "operator": "gt", + "actual_fraction": 0.21711989667790774, + "metric": 0.860012915261534, + "rescued": 1088, + "harmed": 252, + "missed_rescue": 1159, + "avoided_harm": 593, + "selected_both_correct": 1453, + "selected_both_wrong": 233 + }, + { + "budget": 0.24, + "threshold": -0.9136765599250793, + "operator": "ge", + "actual_fraction": 0.22766736026404535, + "metric": 0.8627394704742771, + "rescued": 1134, + "harmed": 260, + "missed_rescue": 1113, + "avoided_harm": 585, + "selected_both_correct": 1534, + "selected_both_wrong": 245 + }, + { + "budget": 0.25, + "threshold": -0.9230391979217529, + "operator": "gt", + "actual_fraction": 0.23785606658534836, + "metric": 0.865609528592954, + "rescued": 1183, + "harmed": 269, + "missed_rescue": 1064, + "avoided_harm": 576, + "selected_both_correct": 1610, + "selected_both_wrong": 253 + }, + { + "budget": 0.26, + "threshold": -0.9314625263214111, + "operator": "gt", + "actual_fraction": 0.24804477290665136, + "metric": 0.8672598120111932, + "rescued": 1221, + "harmed": 284, + "missed_rescue": 1026, + "avoided_harm": 561, + "selected_both_correct": 1685, + "selected_both_wrong": 267 + }, + { + "budget": 0.27, + "threshold": -0.9376689791679382, + "operator": "ge", + "actual_fraction": 0.25751596469828514, + "metric": 0.8686230896175647, + "rescued": 1253, + "harmed": 297, + "missed_rescue": 994, + "avoided_harm": 548, + "selected_both_correct": 1760, + "selected_both_wrong": 279 + }, + { + "budget": 0.28, + "threshold": -0.9449946880340576, + "operator": "gt", + "actual_fraction": 0.26863743990815814, + "metric": 0.8702016215828371, + "rescued": 1285, + "harmed": 307, + "missed_rescue": 962, + "avoided_harm": 538, + "selected_both_correct": 1858, + "selected_both_wrong": 294 + }, + { + "budget": 0.29, + "threshold": -0.9511421918869019, + "operator": "gt", + "actual_fraction": 0.2793284064002296, + "metric": 0.872067159359977, + "rescued": 1328, + "harmed": 324, + "missed_rescue": 919, + "avoided_harm": 521, + "selected_both_correct": 1938, + "selected_both_wrong": 303 + }, + { + "budget": 0.3, + "threshold": -0.9559813141822815, + "operator": "gt", + "actual_fraction": 0.2897323670804334, + "metric": 0.8733586855133817, + "rescued": 1356, + "harmed": 334, + "missed_rescue": 891, + "avoided_harm": 511, + "selected_both_correct": 2036, + "selected_both_wrong": 312 + }, + { + "budget": 0.31, + "threshold": -0.9597620368003845, + "operator": "ge", + "actual_fraction": 0.2999928248547033, + "metric": 0.8741479514960178, + "rescued": 1382, + "harmed": 349, + "missed_rescue": 865, + "avoided_harm": 496, + "selected_both_correct": 2133, + "selected_both_wrong": 317 + }, + { + "budget": 0.32, + "threshold": -0.9643214344978333, + "operator": "gt", + "actual_fraction": 0.3091770108344694, + "metric": 0.8758699863672239, + "rescued": 1416, + "harmed": 359, + "missed_rescue": 831, + "avoided_harm": 486, + "selected_both_correct": 2209, + "selected_both_wrong": 325 + }, + { + "budget": 0.33, + "threshold": -0.9686195850372314, + "operator": "gt", + "actual_fraction": 0.3218770180096147, + "metric": 0.8774485183324963, + "rescued": 1451, + "harmed": 372, + "missed_rescue": 796, + "avoided_harm": 473, + "selected_both_correct": 2333, + "selected_both_wrong": 330 + }, + { + "budget": 0.34, + "threshold": -0.9719924926757812, + "operator": "gt", + "actual_fraction": 0.33127645834828157, + "metric": 0.8790988017507354, + "rescued": 1484, + "harmed": 382, + "missed_rescue": 763, + "avoided_harm": 463, + "selected_both_correct": 2416, + "selected_both_wrong": 335 + }, + { + "budget": 0.35, + "threshold": -0.974821150302887, + "operator": "ge", + "actual_fraction": 0.3431871995407907, + "metric": 0.8801750735452393, + "rescued": 1514, + "harmed": 397, + "missed_rescue": 733, + "avoided_harm": 448, + "selected_both_correct": 2529, + "selected_both_wrong": 343 + }, + { + "budget": 0.36, + "threshold": -0.9777138829231262, + "operator": "gt", + "actual_fraction": 0.35280189423835834, + "metric": 0.8812513453397431, + "rescued": 1539, + "harmed": 407, + "missed_rescue": 708, + "avoided_harm": 438, + "selected_both_correct": 2623, + "selected_both_wrong": 348 + }, + { + "budget": 0.37, + "threshold": -0.9802808165550232, + "operator": "gt", + "actual_fraction": 0.36421037526009903, + "metric": 0.8827581258520485, + "rescued": 1569, + "harmed": 416, + "missed_rescue": 678, + "avoided_harm": 429, + "selected_both_correct": 2740, + "selected_both_wrong": 351 + }, + { + "budget": 0.38, + "threshold": -0.982557475566864, + "operator": "gt", + "actual_fraction": 0.3761928679055751, + "metric": 0.8843366578173208, + "rescued": 1602, + "harmed": 427, + "missed_rescue": 645, + "avoided_harm": 418, + "selected_both_correct": 2856, + "selected_both_wrong": 358 + }, + { + "budget": 0.39, + "threshold": -0.9843363761901855, + "operator": "gt", + "actual_fraction": 0.38803185764511733, + "metric": 0.8857716868766593, + "rescued": 1630, + "harmed": 435, + "missed_rescue": 617, + "avoided_harm": 410, + "selected_both_correct": 2978, + "selected_both_wrong": 365 + }, + { + "budget": 0.4, + "threshold": -0.9863631129264832, + "operator": "gt", + "actual_fraction": 0.4008036162732295, + "metric": 0.8869197101241301, + "rescued": 1658, + "harmed": 447, + "missed_rescue": 589, + "avoided_harm": 398, + "selected_both_correct": 3105, + "selected_both_wrong": 376 + }, + { + "budget": 0.41, + "threshold": -0.9879462122917175, + "operator": "gt", + "actual_fraction": 0.41156633421826794, + "metric": 0.8880677333716007, + "rescued": 1685, + "harmed": 458, + "missed_rescue": 562, + "avoided_harm": 387, + "selected_both_correct": 3212, + "selected_both_wrong": 381 + }, + { + "budget": 0.42, + "threshold": -0.9890130758285522, + "operator": "gt", + "actual_fraction": 0.4204635143861663, + "metric": 0.8884264906364354, + "rescued": 1701, + "harmed": 469, + "missed_rescue": 546, + "avoided_harm": 376, + "selected_both_correct": 3303, + "selected_both_wrong": 387 + }, + { + "budget": 0.43, + "threshold": -0.9899863600730896, + "operator": "gt", + "actual_fraction": 0.4293606945540647, + "metric": 0.8889287508072038, + "rescued": 1720, + "harmed": 481, + "missed_rescue": 527, + "avoided_harm": 364, + "selected_both_correct": 3391, + "selected_both_wrong": 392 + }, + { + "budget": 0.44, + "threshold": -0.9911527037620544, + "operator": "gt", + "actual_fraction": 0.44119968429360695, + "metric": 0.890579034225443, + "rescued": 1755, + "harmed": 493, + "missed_rescue": 492, + "avoided_harm": 352, + "selected_both_correct": 3502, + "selected_both_wrong": 399 + }, + { + "budget": 0.45, + "threshold": -0.9919379353523254, + "operator": "gt", + "actual_fraction": 0.4516753964267776, + "metric": 0.8913683002080792, + "rescued": 1778, + "harmed": 505, + "missed_rescue": 469, + "avoided_harm": 340, + "selected_both_correct": 3608, + "selected_both_wrong": 404 + }, + { + "budget": 0.46, + "threshold": -0.9926542043685913, + "operator": "gt", + "actual_fraction": 0.4611465882184114, + "metric": 0.89251632345555, + "rescued": 1797, + "harmed": 508, + "missed_rescue": 450, + "avoided_harm": 337, + "selected_both_correct": 3715, + "selected_both_wrong": 407 + }, + { + "budget": 0.47, + "threshold": -0.9933071732521057, + "operator": "gt", + "actual_fraction": 0.4706895314630121, + "metric": 0.8933055894381862, + "rescued": 1814, + "harmed": 514, + "missed_rescue": 433, + "avoided_harm": 331, + "selected_both_correct": 3823, + "selected_both_wrong": 409 + }, + { + "budget": 0.48, + "threshold": -0.9939024448394775, + "operator": "gt", + "actual_fraction": 0.481021740690249, + "metric": 0.8941666068737892, + "rescued": 1835, + "harmed": 523, + "missed_rescue": 412, + "avoided_harm": 322, + "selected_both_correct": 3932, + "selected_both_wrong": 414 + }, + { + "budget": 0.49, + "threshold": -0.9944451451301575, + "operator": "gt", + "actual_fraction": 0.4912821984645189, + "metric": 0.8946688670445576, + "rescued": 1853, + "harmed": 534, + "missed_rescue": 394, + "avoided_harm": 311, + "selected_both_correct": 4037, + "selected_both_wrong": 423 + }, + { + "budget": 0.5, + "threshold": -0.9949396848678589, + "operator": "gt", + "actual_fraction": 0.4992466097438473, + "metric": 0.89531463012126, + "rescued": 1869, + "harmed": 541, + "missed_rescue": 378, + "avoided_harm": 304, + "selected_both_correct": 4122, + "selected_both_wrong": 426 + }, + { + "budget": 0.51, + "threshold": -0.9952448010444641, + "operator": "gt", + "actual_fraction": 0.5059912463227381, + "metric": 0.8957451388390615, + "rescued": 1882, + "harmed": 548, + "missed_rescue": 365, + "avoided_harm": 297, + "selected_both_correct": 4196, + "selected_both_wrong": 426 + }, + { + "budget": 0.52, + "threshold": -0.9956685304641724, + "operator": "gt", + "actual_fraction": 0.5140991605080003, + "metric": 0.8960321446509292, + "rescued": 1894, + "harmed": 556, + "missed_rescue": 353, + "avoided_harm": 289, + "selected_both_correct": 4286, + "selected_both_wrong": 429 + }, + { + "budget": 0.53, + "threshold": -0.9961755275726318, + "operator": "gt", + "actual_fraction": 0.529166965631054, + "metric": 0.8973954222573007, + "rescued": 1921, + "harmed": 564, + "missed_rescue": 326, + "avoided_harm": 281, + "selected_both_correct": 4451, + "selected_both_wrong": 439 + }, + { + "budget": 0.54, + "threshold": -0.9965165853500366, + "operator": "gt", + "actual_fraction": 0.5406471981057617, + "metric": 0.8981129367869699, + "rescued": 1940, + "harmed": 573, + "missed_rescue": 307, + "avoided_harm": 272, + "selected_both_correct": 4575, + "selected_both_wrong": 447 + }, + { + "budget": 0.55, + "threshold": -0.9968273043632507, + "operator": "gt", + "actual_fraction": 0.5504771471622301, + "metric": 0.8979694338810361, + "rescued": 1947, + "harmed": 582, + "missed_rescue": 300, + "avoided_harm": 263, + "selected_both_correct": 4695, + "selected_both_wrong": 448 + }, + { + "budget": 0.56, + "threshold": -0.9971103668212891, + "operator": "gt", + "actual_fraction": 0.5621726339958384, + "metric": 0.8986151969577384, + "rescued": 1966, + "harmed": 592, + "missed_rescue": 281, + "avoided_harm": 253, + "selected_both_correct": 4825, + "selected_both_wrong": 452 + }, + { + "budget": 0.57, + "threshold": -0.997285008430481, + "operator": "gt", + "actual_fraction": 0.5677692473272584, + "metric": 0.898973954222573, + "rescued": 1977, + "harmed": 598, + "missed_rescue": 270, + "avoided_harm": 247, + "selected_both_correct": 4884, + "selected_both_wrong": 454 + }, + { + "budget": 0.58, + "threshold": -0.9974491000175476, + "operator": "gt", + "actual_fraction": 0.5759489129654876, + "metric": 0.8992609600344407, + "rescued": 1990, + "harmed": 607, + "missed_rescue": 257, + "avoided_harm": 238, + "selected_both_correct": 4973, + "selected_both_wrong": 457 + }, + { + "budget": 0.59, + "threshold": -0.9976769089698792, + "operator": "gt", + "actual_fraction": 0.5864963765516251, + "metric": 0.899834971658176, + "rescued": 2008, + "harmed": 617, + "missed_rescue": 239, + "avoided_harm": 228, + "selected_both_correct": 5091, + "selected_both_wrong": 458 + }, + { + "budget": 0.6, + "threshold": -0.9978172779083252, + "operator": "gt", + "actual_fraction": 0.5942455334720528, + "metric": 0.899834971658176, + "rescued": 2016, + "harmed": 625, + "missed_rescue": 231, + "avoided_harm": 220, + "selected_both_correct": 5179, + "selected_both_wrong": 462 + }, + { + "budget": 0.61, + "threshold": -0.9980121850967407, + "operator": "gt", + "actual_fraction": 0.6054387601348927, + "metric": 0.8994044629403746, + "rescued": 2025, + "harmed": 640, + "missed_rescue": 222, + "avoided_harm": 205, + "selected_both_correct": 5308, + "selected_both_wrong": 465 + }, + { + "budget": 0.62, + "threshold": -0.9981897473335266, + "operator": "gt", + "actual_fraction": 0.6165602353447658, + "metric": 0.9001219774700437, + "rescued": 2039, + "harmed": 644, + "missed_rescue": 208, + "avoided_harm": 201, + "selected_both_correct": 5438, + "selected_both_wrong": 472 + }, + { + "budget": 0.63, + "threshold": -0.9982993006706238, + "operator": "gt", + "actual_fraction": 0.6242376408122264, + "metric": 0.9004089832819114, + "rescued": 2049, + "harmed": 650, + "missed_rescue": 198, + "avoided_harm": 195, + "selected_both_correct": 5528, + "selected_both_wrong": 473 + }, + { + "budget": 0.64, + "threshold": -0.9984021782875061, + "operator": "gt", + "actual_fraction": 0.6322020520915549, + "metric": 0.900767740546746, + "rescued": 2058, + "harmed": 654, + "missed_rescue": 189, + "avoided_harm": 191, + "selected_both_correct": 5624, + "selected_both_wrong": 475 + }, + { + "budget": 0.65, + "threshold": -0.9985449314117432, + "operator": "gt", + "actual_fraction": 0.6454043194374686, + "metric": 0.9011982492645476, + "rescued": 2072, + "harmed": 662, + "missed_rescue": 175, + "avoided_harm": 183, + "selected_both_correct": 5784, + "selected_both_wrong": 477 + }, + { + "budget": 0.66, + "threshold": -0.9986749291419983, + "operator": "gt", + "actual_fraction": 0.6566692975532754, + "metric": 0.9017005094353161, + "rescued": 2085, + "harmed": 668, + "missed_rescue": 162, + "avoided_harm": 177, + "selected_both_correct": 5920, + "selected_both_wrong": 479 + }, + { + "budget": 0.67, + "threshold": -0.9987550973892212, + "operator": "gt", + "actual_fraction": 0.6646337088326039, + "metric": 0.9017005094353161, + "rescued": 2089, + "harmed": 672, + "missed_rescue": 158, + "avoided_harm": 173, + "selected_both_correct": 6021, + "selected_both_wrong": 481 + }, + { + "budget": 0.68, + "threshold": -0.9988664388656616, + "operator": "gt", + "actual_fraction": 0.6776207218196169, + "metric": 0.9021310181531176, + "rescued": 2102, + "harmed": 679, + "missed_rescue": 145, + "avoided_harm": 166, + "selected_both_correct": 6181, + "selected_both_wrong": 482 + }, + { + "budget": 0.69, + "threshold": -0.9989677667617798, + "operator": "gt", + "actual_fraction": 0.690464231900696, + "metric": 0.9020592667001507, + "rescued": 2113, + "harmed": 691, + "missed_rescue": 134, + "avoided_harm": 154, + "selected_both_correct": 6336, + "selected_both_wrong": 483 + }, + { + "budget": 0.7, + "threshold": -0.9990302324295044, + "operator": "gt", + "actual_fraction": 0.7000789265982637, + "metric": 0.9024180239649853, + "rescued": 2123, + "harmed": 696, + "missed_rescue": 124, + "avoided_harm": 149, + "selected_both_correct": 6453, + "selected_both_wrong": 485 + }, + { + "budget": 0.71, + "threshold": -0.9991169571876526, + "operator": "gt", + "actual_fraction": 0.7121331706967066, + "metric": 0.901772260888283, + "rescued": 2130, + "harmed": 712, + "missed_rescue": 117, + "avoided_harm": 133, + "selected_both_correct": 6595, + "selected_both_wrong": 488 + }, + { + "budget": 0.72, + "threshold": -0.9991704225540161, + "operator": "gt", + "actual_fraction": 0.7201693334290019, + "metric": 0.9017005094353161, + "rescued": 2136, + "harmed": 719, + "missed_rescue": 111, + "avoided_harm": 126, + "selected_both_correct": 6693, + "selected_both_wrong": 489 + }, + { + "budget": 0.73, + "threshold": -0.9992445707321167, + "operator": "gt", + "actual_fraction": 0.7310755542799742, + "metric": 0.9020592667001507, + "rescued": 2143, + "harmed": 721, + "missed_rescue": 104, + "avoided_harm": 124, + "selected_both_correct": 6833, + "selected_both_wrong": 492 + }, + { + "budget": 0.74, + "threshold": -0.9992903470993042, + "operator": "gt", + "actual_fraction": 0.7394704742771041, + "metric": 0.9018440123412499, + "rescued": 2147, + "harmed": 728, + "missed_rescue": 100, + "avoided_harm": 117, + "selected_both_correct": 6937, + "selected_both_wrong": 494 + }, + { + "budget": 0.75, + "threshold": -0.9993333220481873, + "operator": "gt", + "actual_fraction": 0.7467173710267633, + "metric": 0.9018440123412499, + "rescued": 2151, + "harmed": 732, + "missed_rescue": 96, + "avoided_harm": 113, + "selected_both_correct": 7028, + "selected_both_wrong": 496 + }, + { + "budget": 0.76, + "threshold": -0.999373733997345, + "operator": "gt", + "actual_fraction": 0.7550405395709263, + "metric": 0.9016287579823491, + "rescued": 2154, + "harmed": 738, + "missed_rescue": 93, + "avoided_harm": 107, + "selected_both_correct": 7133, + "selected_both_wrong": 498 + }, + { + "budget": 0.77, + "threshold": -0.9994296431541443, + "operator": "gt", + "actual_fraction": 0.7659467604218986, + "metric": 0.9018440123412499, + "rescued": 2164, + "harmed": 745, + "missed_rescue": 83, + "avoided_harm": 100, + "selected_both_correct": 7267, + "selected_both_wrong": 499 + }, + { + "budget": 0.78, + "threshold": -0.999464213848114, + "operator": "gt", + "actual_fraction": 0.7743416804190285, + "metric": 0.9022027696060845, + "rescued": 2172, + "harmed": 748, + "missed_rescue": 75, + "avoided_harm": 97, + "selected_both_correct": 7371, + "selected_both_wrong": 501 + }, + { + "budget": 0.79, + "threshold": -0.9994966983795166, + "operator": "gt", + "actual_fraction": 0.7820908373394562, + "metric": 0.9022745210590515, + "rescued": 2178, + "harmed": 753, + "missed_rescue": 69, + "avoided_harm": 92, + "selected_both_correct": 7467, + "selected_both_wrong": 502 + }, + { + "budget": 0.8, + "threshold": -0.9995417594909668, + "operator": "gt", + "actual_fraction": 0.7935710698141637, + "metric": 0.9023462725120184, + "rescued": 2182, + "harmed": 756, + "missed_rescue": 65, + "avoided_harm": 89, + "selected_both_correct": 7616, + "selected_both_wrong": 506 + }, + { + "budget": 0.81, + "threshold": -0.9995827078819275, + "operator": "gt", + "actual_fraction": 0.8052665566477721, + "metric": 0.9025615268709192, + "rescued": 2189, + "harmed": 760, + "missed_rescue": 58, + "avoided_harm": 85, + "selected_both_correct": 7765, + "selected_both_wrong": 509 + }, + { + "budget": 0.82, + "threshold": -0.9996140599250793, + "operator": "gt", + "actual_fraction": 0.8185405754466528, + "metric": 0.9026332783238861, + "rescued": 2196, + "harmed": 766, + "missed_rescue": 51, + "avoided_harm": 79, + "selected_both_correct": 7933, + "selected_both_wrong": 513 + }, + { + "budget": 0.83, + "threshold": -0.9996429681777954, + "operator": "gt", + "actual_fraction": 0.8264332352730143, + "metric": 0.9026332783238861, + "rescued": 2200, + "harmed": 770, + "missed_rescue": 47, + "avoided_harm": 75, + "selected_both_correct": 8034, + "selected_both_wrong": 514 + }, + { + "budget": 0.84, + "threshold": -0.9996750354766846, + "operator": "gt", + "actual_fraction": 0.836837195953218, + "metric": 0.9025615268709192, + "rescued": 2202, + "harmed": 773, + "missed_rescue": 45, + "avoided_harm": 72, + "selected_both_correct": 8172, + "selected_both_wrong": 516 + }, + { + "budget": 0.85, + "threshold": -0.9997040629386902, + "operator": "gt", + "actual_fraction": 0.844299347061778, + "metric": 0.9025615268709192, + "rescued": 2206, + "harmed": 777, + "missed_rescue": 41, + "avoided_harm": 68, + "selected_both_correct": 8267, + "selected_both_wrong": 517 + }, + { + "budget": 0.86, + "threshold": -0.9997387528419495, + "operator": "gt", + "actual_fraction": 0.8580038745784602, + "metric": 0.9024180239649853, + "rescued": 2213, + "harmed": 786, + "missed_rescue": 34, + "avoided_harm": 59, + "selected_both_correct": 8437, + "selected_both_wrong": 522 + }, + { + "budget": 0.87, + "threshold": -0.9997695088386536, + "operator": "gt", + "actual_fraction": 0.870273373035804, + "metric": 0.9026332783238861, + "rescued": 2220, + "harmed": 790, + "missed_rescue": 27, + "avoided_harm": 55, + "selected_both_correct": 8594, + "selected_both_wrong": 525 + }, + { + "budget": 0.88, + "threshold": -0.999783456325531, + "operator": "gt", + "actual_fraction": 0.8750807203845878, + "metric": 0.9024180239649853, + "rescued": 2220, + "harmed": 793, + "missed_rescue": 27, + "avoided_harm": 52, + "selected_both_correct": 8657, + "selected_both_wrong": 526 + }, + { + "budget": 0.89, + "threshold": -0.9998205304145813, + "operator": "gt", + "actual_fraction": 0.8908660400373107, + "metric": 0.9022745210590515, + "rescued": 2223, + "harmed": 798, + "missed_rescue": 24, + "avoided_harm": 47, + "selected_both_correct": 8865, + "selected_both_wrong": 530 + }, + { + "budget": 0.9, + "threshold": -0.999841570854187, + "operator": "gt", + "actual_fraction": 0.8992609600344407, + "metric": 0.9023462725120184, + "rescued": 2225, + "harmed": 799, + "missed_rescue": 22, + "avoided_harm": 46, + "selected_both_correct": 8979, + "selected_both_wrong": 530 + }, + { + "budget": 0.91, + "threshold": -0.9998601675033569, + "operator": "gt", + "actual_fraction": 0.9091626605438761, + "metric": 0.9023462725120184, + "rescued": 2227, + "harmed": 801, + "missed_rescue": 20, + "avoided_harm": 44, + "selected_both_correct": 9111, + "selected_both_wrong": 532 + }, + { + "budget": 0.92, + "threshold": -0.9998766183853149, + "operator": "gt", + "actual_fraction": 0.9171270718232044, + "metric": 0.9023462725120184, + "rescued": 2230, + "harmed": 804, + "missed_rescue": 17, + "avoided_harm": 41, + "selected_both_correct": 9215, + "selected_both_wrong": 533 + }, + { + "budget": 0.93, + "threshold": -0.9998977184295654, + "operator": "gt", + "actual_fraction": 0.9273875295974744, + "metric": 0.9020592667001507, + "rescued": 2232, + "harmed": 810, + "missed_rescue": 15, + "avoided_harm": 35, + "selected_both_correct": 9350, + "selected_both_wrong": 533 + }, + { + "budget": 0.94, + "threshold": -0.9999151229858398, + "operator": "gt", + "actual_fraction": 0.9375044844658105, + "metric": 0.9018440123412499, + "rescued": 2235, + "harmed": 816, + "missed_rescue": 12, + "avoided_harm": 29, + "selected_both_correct": 9481, + "selected_both_wrong": 534 + }, + { + "budget": 0.95, + "threshold": -0.999931812286377, + "operator": "gt", + "actual_fraction": 0.948984716940518, + "metric": 0.9016287579823491, + "rescued": 2239, + "harmed": 823, + "missed_rescue": 8, + "avoided_harm": 22, + "selected_both_correct": 9629, + "selected_both_wrong": 535 + }, + { + "budget": 0.96, + "threshold": -0.9999452829360962, + "operator": "gt", + "actual_fraction": 0.9576666427495156, + "metric": 0.9015570065293822, + "rescued": 2241, + "harmed": 826, + "missed_rescue": 6, + "avoided_harm": 19, + "selected_both_correct": 9745, + "selected_both_wrong": 535 + }, + { + "budget": 0.97, + "threshold": -0.9999599456787109, + "operator": "gt", + "actual_fraction": 0.9709406615483963, + "metric": 0.9009829949056468, + "rescued": 2241, + "harmed": 834, + "missed_rescue": 6, + "avoided_harm": 11, + "selected_both_correct": 9922, + "selected_both_wrong": 535 + }, + { + "budget": 0.98, + "threshold": -0.9999687671661377, + "operator": "gt", + "actual_fraction": 0.9784745641099232, + "metric": 0.900839491999713, + "rescued": 2241, + "harmed": 836, + "missed_rescue": 6, + "avoided_harm": 9, + "selected_both_correct": 10023, + "selected_both_wrong": 537 + }, + { + "budget": 0.99, + "threshold": -0.9999842643737793, + "operator": "gt", + "actual_fraction": 0.98973954222573, + "metric": 0.900839491999713, + "rescued": 2245, + "harmed": 840, + "missed_rescue": 2, + "avoided_harm": 5, + "selected_both_correct": 10170, + "selected_both_wrong": 539 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9006242376408122, + "rescued": 2247, + "harmed": 845, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 10305, + "selected_both_wrong": 540 + } + ] + }, + "learned": { + "validation_area_0_25": 0.8407676883414494, + "validation_point": { + "metric": 0.9091950730449728, + "metric_std": 0.0006243052053194811, + "actual_fraction": 0.9699226582641078, + "actual_fraction_std": 0.010025780578630794, + "per_seed": [ + { + "seed": 2026, + "budget": 0.97, + "threshold": -0.28458863496780396, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.9099112002291607, + "rescue": 1125, + "harm": 345, + "missed_rescue": 29 + }, + { + "seed": 2027, + "budget": 0.98, + "threshold": -0.25909605622291565, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.9084789458607849, + "rescue": 1129, + "harm": 359, + "missed_rescue": 25 + }, + { + "seed": 2028, + "budget": 0.96, + "threshold": -0.23020051419734955, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.9086221712976225, + "rescue": 1107, + "harm": 336, + "missed_rescue": 47 + }, + { + "seed": 2029, + "budget": 0.98, + "threshold": -0.3751336932182312, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.9093382984818104, + "rescue": 1134, + "harm": 358, + "missed_rescue": 20 + }, + { + "seed": 2030, + "budget": 0.96, + "threshold": -0.29492709040641785, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.9096247493554855, + "rescue": 1110, + "harm": 332, + "missed_rescue": 44 + } + ], + "rescued": 1121.0, + "harmed": 346.0, + "missed_rescue": 33.0, + "avoided_harm": 62.0 + }, + "test_point": { + "metric": 0.9061060486474851, + "metric_std": 0.0013050088305247716, + "actual_fraction": 0.9687737676687954, + "actual_fraction_std": 0.00929609232811199, + "per_seed": [ + { + "seed": 2026, + "budget": 0.97, + "threshold": -0.28458863496780396, + "operator": "ge", + "actual_fraction": 0.9679988519767525, + "metric": 0.9067948625959675, + "rescue": 2172, + "harm": 684, + "missed_rescue": 75 + }, + { + "seed": 2027, + "budget": 0.98, + "threshold": -0.25909605622291565, + "operator": "ge", + "actual_fraction": 0.9771830379565186, + "metric": 0.9039248044772906, + "rescue": 2180, + "harm": 732, + "missed_rescue": 67 + }, + { + "seed": 2028, + "budget": 0.96, + "threshold": -0.23020051419734955, + "operator": "ge", + "actual_fraction": 0.9595321805266557, + "metric": 0.9059338451603645, + "rescue": 2154, + "harm": 678, + "missed_rescue": 93 + }, + { + "seed": 2029, + "budget": 0.98, + "threshold": -0.3751336932182312, + "operator": "ge", + "actual_fraction": 0.9792638300925594, + "metric": 0.9066513596900337, + "rescue": 2205, + "harm": 719, + "missed_rescue": 42 + }, + { + "seed": 2030, + "budget": 0.96, + "threshold": -0.29492709040641785, + "operator": "ge", + "actual_fraction": 0.9598909377914903, + "metric": 0.9072253713137691, + "rescue": 2159, + "harm": 665, + "missed_rescue": 88 + } + ], + "rescued": 2174.0, + "harmed": 695.6, + "missed_rescue": 73.0, + "avoided_harm": 149.39999999999998 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.7981953594958464, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.8033514752219995, + "metric_std": 0.00033589342307532714, + "actual_fraction": 0.009882555141793183, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.807648238327127, + "metric_std": 0.0007578777745816562, + "actual_fraction": 0.01990833572042395, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.8122314523059295, + "metric_std": 0.0009819041249500414, + "actual_fraction": 0.029934116299054715, + "actual_fraction_std": 3.878959614448864e-18 + }, + { + "budget": 0.04, + "metric": 0.8165282154110571, + "metric_std": 0.0007303092972776932, + "actual_fraction": 0.039959896877685475, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.8201088513319965, + "metric_std": 0.0011413189236973132, + "actual_fraction": 0.04998567745631624, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.8238327126897736, + "metric_std": 0.0009001579124875594, + "actual_fraction": 0.05986823259810943, + "actual_fraction_std": 7.757919228897728e-18 + }, + { + "budget": 0.07, + "metric": 0.8272128329991407, + "metric_std": 0.0009902254951271137, + "actual_fraction": 0.06989401317674018, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.8303351475222, + "metric_std": 0.0007345105359471839, + "actual_fraction": 0.07991979375537095, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.8334001718705242, + "metric_std": 0.0011784590335562417, + "actual_fraction": 0.08994557433400172, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.836665711830421, + "metric_std": 0.0009259942638352719, + "actual_fraction": 0.09997135491263248, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.8391864795187626, + "metric_std": 0.0010879478095006054, + "actual_fraction": 0.10999713549126325, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.12, + "metric": 0.8418218275565741, + "metric_std": 0.0011705996414610627, + "actual_fraction": 0.11987969063305644, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.8449154969922658, + "metric_std": 0.001083223741116515, + "actual_fraction": 0.1299054712116872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.8477800057290175, + "metric_std": 0.0008852212626962134, + "actual_fraction": 0.13993125179031796, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.8507304497278717, + "metric_std": 0.00097351162807027, + "actual_fraction": 0.14995703236894872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.8536236035519907, + "metric_std": 0.0002679502568586331, + "actual_fraction": 0.1599828129475795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.855743340017187, + "metric_std": 0.0007739476417133645, + "actual_fraction": 0.16986536808937266, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.858178172443426, + "metric_std": 0.0004793239911395728, + "actual_fraction": 0.17989114866800343, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.8603838441707248, + "metric_std": 0.0006357015474411929, + "actual_fraction": 0.1899169292466342, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.8621025494127756, + "metric_std": 0.0005397148003843104, + "actual_fraction": 0.19994270982526496, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.8641363506158694, + "metric_std": 0.0005957214077079861, + "actual_fraction": 0.20996849040389573, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.22, + "metric": 0.8661415067315955, + "metric_std": 0.00038697382085667455, + "actual_fraction": 0.2199942709825265, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.23, + "metric": 0.867831566886279, + "metric_std": 0.0003266042466626267, + "actual_fraction": 0.22987682612431967, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.24, + "metric": 0.8695502721283301, + "metric_std": 0.0008252569630987148, + "actual_fraction": 0.23990260670295044, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.8702663993125179, + "metric_std": 0.0009137288976073639, + "actual_fraction": 0.24992838728158123, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.8709825264967058, + "metric_std": 0.001278643031834173, + "actual_fraction": 0.25995416786021197, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.8721856201661415, + "metric_std": 0.0014344011410545971, + "actual_fraction": 0.26997994843884277, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.8728731022629619, + "metric_std": 0.002042155671456632, + "actual_fraction": 0.2798625035806359, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.8734746490976798, + "metric_std": 0.0022432048208466658, + "actual_fraction": 0.2898882841592667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.8739043254081924, + "metric_std": 0.0024591436300584095, + "actual_fraction": 0.29991406473789745, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.8743053566313378, + "metric_std": 0.0022817389941022924, + "actual_fraction": 0.30993984531652824, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.8746777427671155, + "metric_std": 0.002117121956329802, + "actual_fraction": 0.319965625895159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.8751933543397307, + "metric_std": 0.002140250055859988, + "actual_fraction": 0.3299914064737898, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.875708965912346, + "metric_std": 0.002281289434646837, + "actual_fraction": 0.3398739616155829, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.8763678029217988, + "metric_std": 0.002270021513166331, + "actual_fraction": 0.3498997421942137, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.8768261243196791, + "metric_std": 0.002403030898820197, + "actual_fraction": 0.35992552277284445, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.8774563162417646, + "metric_std": 0.0022317446211151805, + "actual_fraction": 0.36995130335147525, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.8779146376396447, + "metric_std": 0.0021876484084929855, + "actual_fraction": 0.379977083930106, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.87820108851332, + "metric_std": 0.002152197508594904, + "actual_fraction": 0.3898596390718992, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.8786021197364653, + "metric_std": 0.0020270321184407826, + "actual_fraction": 0.3998854196505299, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.8791750214838154, + "metric_std": 0.002140250055860007, + "actual_fraction": 0.40991120022916067, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.42, + "metric": 0.8795760527069607, + "metric_std": 0.0021777803275563807, + "actual_fraction": 0.41993698080779146, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.43, + "metric": 0.8802921798911487, + "metric_std": 0.002258697380315596, + "actual_fraction": 0.42996276138642225, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.44, + "metric": 0.8808364365511314, + "metric_std": 0.002497220821277967, + "actual_fraction": 0.439988541965053, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.45, + "metric": 0.8812088226869091, + "metric_std": 0.002425547193356216, + "actual_fraction": 0.4498710971068462, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.8816957891721569, + "metric_std": 0.002289367998158968, + "actual_fraction": 0.45989687768547693, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.47, + "metric": 0.8822114007447721, + "metric_std": 0.002510738231123149, + "actual_fraction": 0.46992265826410773, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.48, + "metric": 0.8828129475794901, + "metric_std": 0.0025021446922855795, + "actual_fraction": 0.4799484388427385, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.49, + "metric": 0.883586364938413, + "metric_std": 0.002485693858483722, + "actual_fraction": 0.4899742194213693, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.8840160412489257, + "metric_std": 0.002524589561255098, + "actual_fraction": 0.5, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.51, + "metric": 0.8847321684331136, + "metric_std": 0.0022266834475388447, + "actual_fraction": 0.5098825551417931, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.8849040389573188, + "metric_std": 0.0023460134186930673, + "actual_fraction": 0.5199083357204239, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.8853337152678316, + "metric_std": 0.0019809688229183915, + "actual_fraction": 0.5299341162990547, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.8856774563162417, + "metric_std": 0.0021754241867307103, + "actual_fraction": 0.5399598968776855, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.8865940991120024, + "metric_std": 0.0021083835115107367, + "actual_fraction": 0.5499856774563162, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.8866800343741048, + "metric_std": 0.0020789901099010715, + "actual_fraction": 0.5598682325981095, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.8869664852477801, + "metric_std": 0.0018184554853169417, + "actual_fraction": 0.5698940131767402, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.8874820968203954, + "metric_std": 0.001793466152105217, + "actual_fraction": 0.579919793755371, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.8881695789172156, + "metric_std": 0.0018984800434980551, + "actual_fraction": 0.5899455743340017, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.8889716413635063, + "metric_std": 0.0020541743470990645, + "actual_fraction": 0.5999713549126325, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.8896591234603264, + "metric_std": 0.0018990202284694545, + "actual_fraction": 0.6099971354912632, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.8902033801203093, + "metric_std": 0.0017145227443879226, + "actual_fraction": 0.6198796906330565, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.8908622171297622, + "metric_std": 0.0020431599280560167, + "actual_fraction": 0.6299054712116872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.8914637639644802, + "metric_std": 0.0018767451538584392, + "actual_fraction": 0.639931251790318, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.8919220853623603, + "metric_std": 0.0021593342503276017, + "actual_fraction": 0.6499570323689488, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.8928387281581209, + "metric_std": 0.0022632338327273458, + "actual_fraction": 0.6599828129475795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.8932684044686336, + "metric_std": 0.0020315810056140957, + "actual_fraction": 0.6698653680893727, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.8939845316528215, + "metric_std": 0.0020239938472574825, + "actual_fraction": 0.6798911486680035, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.8947293039243771, + "metric_std": 0.002058164976110006, + "actual_fraction": 0.6899169292466342, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.8953308507590947, + "metric_std": 0.0020616504400397004, + "actual_fraction": 0.699942709825265, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.8956745918075051, + "metric_std": 0.0019887201326508138, + "actual_fraction": 0.7099684904038958, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.8964193640790604, + "metric_std": 0.001835298594058023, + "actual_fraction": 0.7199942709825266, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.8967631051274706, + "metric_std": 0.0017406457865848464, + "actual_fraction": 0.7298768261243197, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.8971068461758808, + "metric_std": 0.0018854691760256815, + "actual_fraction": 0.7399026067029505, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.8977083930105987, + "metric_std": 0.0017814158511299078, + "actual_fraction": 0.7499283872815812, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.8984818103695217, + "metric_std": 0.001662279134053771, + "actual_fraction": 0.759954167860212, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.8993125179031797, + "metric_std": 0.0018088414408930166, + "actual_fraction": 0.7699799484388428, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.8997421942136924, + "metric_std": 0.0019611950641118757, + "actual_fraction": 0.7798625035806359, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.900171870524205, + "metric_std": 0.0017453534240839117, + "actual_fraction": 0.7898882841592667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.9009739329704957, + "metric_std": 0.0017353344292020665, + "actual_fraction": 0.7999140647378975, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.901489544543111, + "metric_std": 0.0019092547046099305, + "actual_fraction": 0.8099398453165282, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.9021770266399314, + "metric_std": 0.0015123683816917208, + "actual_fraction": 0.819965625895159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.9026926382125465, + "metric_std": 0.001412786627975682, + "actual_fraction": 0.8299914064737898, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.84, + "metric": 0.9028072185620166, + "metric_std": 0.0011075688873778875, + "actual_fraction": 0.8398739616155829, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.85, + "metric": 0.9030650243483244, + "metric_std": 0.0009995042292963918, + "actual_fraction": 0.8498997421942137, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.9032368948725293, + "metric_std": 0.0011923033737191208, + "actual_fraction": 0.8599255227728445, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.87, + "metric": 0.9035806359209395, + "metric_std": 0.0009995042292963495, + "actual_fraction": 0.8699513033514752, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.88, + "metric": 0.904325408192495, + "metric_std": 0.0010670054730558096, + "actual_fraction": 0.879977083930106, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.89, + "metric": 0.9047550845030077, + "metric_std": 0.0011723507264211172, + "actual_fraction": 0.8898596390718991, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.9054712116871956, + "metric_std": 0.0006324664040624282, + "actual_fraction": 0.8998854196505299, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.905986823259811, + "metric_std": 0.0010157905028070884, + "actual_fraction": 0.9099112002291607, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.92, + "metric": 0.907103981667144, + "metric_std": 0.0008863791719717129, + "actual_fraction": 0.9199369808077915, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.93, + "metric": 0.9075050128902893, + "metric_std": 0.0006898650575074244, + "actual_fraction": 0.9299627613864223, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.94, + "metric": 0.9077341735892294, + "metric_std": 0.0005675053782915904, + "actual_fraction": 0.9399885419650531, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.95, + "metric": 0.9082211400744772, + "metric_std": 0.0004574253630097671, + "actual_fraction": 0.9498710971068463, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.96, + "metric": 0.9088799770839302, + "metric_std": 0.000740075089244558, + "actual_fraction": 0.959896877685477, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.97, + "metric": 0.9089086221712975, + "metric_std": 0.0007779132405614892, + "actual_fraction": 0.9699226582641078, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.98, + "metric": 0.9086221712976223, + "metric_std": 0.0010621882679884844, + "actual_fraction": 0.9799484388427386, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.907705528501862, + "metric_std": 0.0007619270168015237, + "actual_fraction": 0.9899742194213692, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.9050415353766829, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.8000287005811868, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.8066872354165172, + "metric_std": 0.0006544739471043368, + "actual_fraction": 0.011437181602927459, + "actual_fraction_std": 0.001264943725108073 + }, + { + "budget": 0.02, + "metric": 0.8118533400301355, + "metric_std": 0.0005159124061777307, + "actual_fraction": 0.02176939083016431, + "actual_fraction_std": 0.00046666032981298913 + }, + { + "budget": 0.03, + "metric": 0.815971873430437, + "metric_std": 0.00013801667548018947, + "actual_fraction": 0.031455836980698856, + "actual_fraction_std": 0.0010167461912272853 + }, + { + "budget": 0.04, + "metric": 0.8199612542153979, + "metric_std": 0.000751508701888538, + "actual_fraction": 0.04239075841285785, + "actual_fraction_std": 0.0011383408151432474 + }, + { + "budget": 0.05, + "metric": 0.823548826863744, + "metric_std": 0.0007324273605610887, + "actual_fraction": 0.0530099734519624, + "actual_fraction_std": 0.0020548994808869516 + }, + { + "budget": 0.06, + "metric": 0.8269354954437826, + "metric_std": 0.0006673270875274131, + "actual_fraction": 0.06220850972232187, + "actual_fraction_std": 0.0017428367369579014 + }, + { + "budget": 0.07, + "metric": 0.8302504125708545, + "metric_std": 0.0007092151133329717, + "actual_fraction": 0.07139269570208796, + "actual_fraction_std": 0.000738726421825861 + }, + { + "budget": 0.08, + "metric": 0.833881036090981, + "metric_std": 0.0012698182854633997, + "actual_fraction": 0.08196885986941235, + "actual_fraction_std": 0.0014301776356236532 + }, + { + "budget": 0.09, + "metric": 0.8372533543804263, + "metric_std": 0.0005881868847505108, + "actual_fraction": 0.09251632345554997, + "actual_fraction_std": 0.0008825720382684862 + }, + { + "budget": 0.1, + "metric": 0.8400516610461362, + "metric_std": 0.0007110275750498766, + "actual_fraction": 0.10139915333285499, + "actual_fraction_std": 0.002343978415911078 + }, + { + "budget": 0.11, + "metric": 0.8428786682930329, + "metric_std": 0.0009645167547624988, + "actual_fraction": 0.11124345267991678, + "actual_fraction_std": 0.002692161941234709 + }, + { + "budget": 0.12, + "metric": 0.8454904211810288, + "metric_std": 0.0009084425362999923, + "actual_fraction": 0.12115950347994549, + "actual_fraction_std": 0.0031228784415545593 + }, + { + "budget": 0.13, + "metric": 0.8480447729066514, + "metric_std": 0.0011097145686550065, + "actual_fraction": 0.13126210805768818, + "actual_fraction_std": 0.0038266469935764246 + }, + { + "budget": 0.14, + "metric": 0.8504699720169334, + "metric_std": 0.0008290320734944002, + "actual_fraction": 0.14097725478940948, + "actual_fraction_std": 0.0032827367946876183 + }, + { + "budget": 0.15, + "metric": 0.8528377699648418, + "metric_std": 0.001421512000872404, + "actual_fraction": 0.15240008610174358, + "actual_fraction_std": 0.004215970307935014 + }, + { + "budget": 0.16, + "metric": 0.8550620650068164, + "metric_std": 0.0013044169450167154, + "actual_fraction": 0.16220133457702518, + "actual_fraction_std": 0.0036220628095878717 + }, + { + "budget": 0.17, + "metric": 0.8572576594676041, + "metric_std": 0.0011686899918540979, + "actual_fraction": 0.17244744206070176, + "actual_fraction_std": 0.004116995582063549 + }, + { + "budget": 0.18, + "metric": 0.8591949486977111, + "metric_std": 0.0008206057806124997, + "actual_fraction": 0.18207648704886273, + "actual_fraction_std": 0.002465578715065174 + }, + { + "budget": 0.19, + "metric": 0.8615627466456196, + "metric_std": 0.0011861797777246402, + "actual_fraction": 0.19288225586568128, + "actual_fraction_std": 0.0028286821653646507 + }, + { + "budget": 0.2, + "metric": 0.8638731434311545, + "metric_std": 0.0008510944307790103, + "actual_fraction": 0.2040037310755543, + "actual_fraction_std": 0.002669019030779595 + }, + { + "budget": 0.21, + "metric": 0.8660113367295688, + "metric_std": 0.0010116700427359867, + "actual_fraction": 0.21403458420033009, + "actual_fraction_std": 0.001730236858977904 + }, + { + "budget": 0.22, + "metric": 0.8678338236349286, + "metric_std": 0.0014287370131118712, + "actual_fraction": 0.22436679342756688, + "actual_fraction_std": 0.002679414845352355 + }, + { + "budget": 0.23, + "metric": 0.8696276099591017, + "metric_std": 0.0018009471762949342, + "actual_fraction": 0.2333787759202124, + "actual_fraction_std": 0.001488388730316313 + }, + { + "budget": 0.24, + "metric": 0.8709478366936931, + "metric_std": 0.0018325439081147588, + "actual_fraction": 0.24340962904498817, + "actual_fraction_std": 0.0017884688350876296 + }, + { + "budget": 0.25, + "metric": 0.8722967640094712, + "metric_std": 0.002102574685988546, + "actual_fraction": 0.2530099734519624, + "actual_fraction_std": 0.002481085979591423 + }, + { + "budget": 0.26, + "metric": 0.8733299849321948, + "metric_std": 0.002173133449866248, + "actual_fraction": 0.26271076989309033, + "actual_fraction_std": 0.003644027359624018 + }, + { + "budget": 0.27, + "metric": 0.8740762000430509, + "metric_std": 0.0023038809439624506, + "actual_fraction": 0.2714787974456483, + "actual_fraction_std": 0.0033221775051825433 + }, + { + "budget": 0.28, + "metric": 0.8747506637009399, + "metric_std": 0.0026902489430955553, + "actual_fraction": 0.2810647915620291, + "actual_fraction_std": 0.0031367778915113737 + }, + { + "budget": 0.29, + "metric": 0.8755399296835762, + "metric_std": 0.003026601823067729, + "actual_fraction": 0.2908516897467174, + "actual_fraction_std": 0.0028137183846655817 + }, + { + "budget": 0.3, + "metric": 0.8761713424696851, + "metric_std": 0.0029283911981617962, + "actual_fraction": 0.30068163880318577, + "actual_fraction_std": 0.0029923883028915715 + }, + { + "budget": 0.31, + "metric": 0.87665925234986, + "metric_std": 0.002998146289126213, + "actual_fraction": 0.3110425486116094, + "actual_fraction_std": 0.00325501792988842 + }, + { + "budget": 0.32, + "metric": 0.8770897610676617, + "metric_std": 0.0030165491869954724, + "actual_fraction": 0.32140345842003304, + "actual_fraction_std": 0.0036791777543129975 + }, + { + "budget": 0.33, + "metric": 0.8774198177513094, + "metric_std": 0.0032441655955200567, + "actual_fraction": 0.3312047068953146, + "actual_fraction_std": 0.003804180567356401 + }, + { + "budget": 0.34, + "metric": 0.877893377340891, + "metric_std": 0.0035393181552623694, + "actual_fraction": 0.34116380856712347, + "actual_fraction_std": 0.0034557841722747808 + }, + { + "budget": 0.35, + "metric": 0.8782377843151326, + "metric_std": 0.003589007390514387, + "actual_fraction": 0.3513525148884265, + "actual_fraction_std": 0.003196927836984535 + }, + { + "budget": 0.36, + "metric": 0.8784817392552199, + "metric_std": 0.003596745108566625, + "actual_fraction": 0.36215828370524505, + "actual_fraction_std": 0.005020192331009097 + }, + { + "budget": 0.37, + "metric": 0.8789122479730216, + "metric_std": 0.003515674760793009, + "actual_fraction": 0.3721030350864605, + "actual_fraction_std": 0.0056319177594002815 + }, + { + "budget": 0.38, + "metric": 0.8792423046566693, + "metric_std": 0.0035227698267705286, + "actual_fraction": 0.38177513094640164, + "actual_fraction_std": 0.0041925203467580155 + }, + { + "budget": 0.39, + "metric": 0.8796297625026905, + "metric_std": 0.0034707239147621968, + "actual_fraction": 0.3914472268063428, + "actual_fraction_std": 0.004697830151791292 + }, + { + "budget": 0.4, + "metric": 0.8800315706393054, + "metric_std": 0.0035081251164044264, + "actual_fraction": 0.4012771758628112, + "actual_fraction_std": 0.003433440103214144 + }, + { + "budget": 0.41, + "metric": 0.8807777857501614, + "metric_std": 0.0033085901277418147, + "actual_fraction": 0.4120829446796298, + "actual_fraction_std": 0.003346034212307083 + }, + { + "budget": 0.42, + "metric": 0.8813230967927101, + "metric_std": 0.003238765531546932, + "actual_fraction": 0.42274521059051445, + "actual_fraction_std": 0.004509337643159343 + }, + { + "budget": 0.43, + "metric": 0.8818684078352585, + "metric_std": 0.003010826395168567, + "actual_fraction": 0.4345267991676831, + "actual_fraction_std": 0.006381645380946795 + }, + { + "budget": 0.44, + "metric": 0.88225586568128, + "metric_std": 0.0029479283323777254, + "actual_fraction": 0.44465810432661257, + "actual_fraction_std": 0.004804508580026014 + }, + { + "budget": 0.45, + "metric": 0.8827294252708617, + "metric_std": 0.0027498691694346814, + "actual_fraction": 0.4536844371098514, + "actual_fraction_std": 0.004308971962293422 + }, + { + "budget": 0.46, + "metric": 0.8832029848604435, + "metric_std": 0.002817740857605202, + "actual_fraction": 0.46258161727774993, + "actual_fraction_std": 0.004651686041108044 + }, + { + "budget": 0.47, + "metric": 0.8836191432876517, + "metric_std": 0.002598452889751279, + "actual_fraction": 0.47206715935997706, + "actual_fraction_std": 0.004605917967300741 + }, + { + "budget": 0.48, + "metric": 0.8840353017148598, + "metric_std": 0.0026379761019713287, + "actual_fraction": 0.4814953002798307, + "actual_fraction_std": 0.0040944248517499615 + }, + { + "budget": 0.49, + "metric": 0.8846667145009686, + "metric_std": 0.0025406499529434527, + "actual_fraction": 0.4909664920714644, + "actual_fraction_std": 0.0037443672926278037 + }, + { + "budget": 0.5, + "metric": 0.8852694267058908, + "metric_std": 0.0025655534446123547, + "actual_fraction": 0.5012413001363277, + "actual_fraction_std": 0.003832965035175069 + }, + { + "budget": 0.51, + "metric": 0.8859725909449667, + "metric_std": 0.0027361680672732037, + "actual_fraction": 0.5104828872784674, + "actual_fraction_std": 0.003784370525697571 + }, + { + "budget": 0.52, + "metric": 0.886776207218196, + "metric_std": 0.002760333509603395, + "actual_fraction": 0.5190787113439047, + "actual_fraction_std": 0.003203443239704703 + }, + { + "budget": 0.53, + "metric": 0.8872497668077779, + "metric_std": 0.0026763388138061717, + "actual_fraction": 0.5281624452895171, + "actual_fraction_std": 0.0039348983520689634 + }, + { + "budget": 0.54, + "metric": 0.8878811795938868, + "metric_std": 0.0025638471905434198, + "actual_fraction": 0.5376766879529311, + "actual_fraction_std": 0.003034246699664114 + }, + { + "budget": 0.55, + "metric": 0.8884982420894023, + "metric_std": 0.0025286661672822373, + "actual_fraction": 0.547119179163378, + "actual_fraction_std": 0.003705946917366941 + }, + { + "budget": 0.56, + "metric": 0.8889431010977973, + "metric_std": 0.002513554887945626, + "actual_fraction": 0.5564325177584846, + "actual_fraction_std": 0.0036719642912890977 + }, + { + "budget": 0.57, + "metric": 0.889459711559159, + "metric_std": 0.002553283292337022, + "actual_fraction": 0.5649278897897683, + "actual_fraction_std": 0.0028008811745886503 + }, + { + "budget": 0.58, + "metric": 0.890277678122982, + "metric_std": 0.0023080112665808277, + "actual_fraction": 0.5753892516323456, + "actual_fraction_std": 0.002382645259970286 + }, + { + "budget": 0.59, + "metric": 0.8910095429432445, + "metric_std": 0.0023851288137467924, + "actual_fraction": 0.585592308244242, + "actual_fraction_std": 0.00360568006949228 + }, + { + "budget": 0.6, + "metric": 0.8913826504986726, + "metric_std": 0.00249402105557689, + "actual_fraction": 0.595264404104183, + "actual_fraction_std": 0.0043936854438433376 + }, + { + "budget": 0.61, + "metric": 0.8921862667719023, + "metric_std": 0.0023539507387333055, + "actual_fraction": 0.6042620363062353, + "actual_fraction_std": 0.004013466122164546 + }, + { + "budget": 0.62, + "metric": 0.8926885269426705, + "metric_std": 0.002309795064560622, + "actual_fraction": 0.6140776350721102, + "actual_fraction_std": 0.005219295580579511 + }, + { + "budget": 0.63, + "metric": 0.893822199899548, + "metric_std": 0.0020816579217323612, + "actual_fraction": 0.624596398077061, + "actual_fraction_std": 0.005237856066585319 + }, + { + "budget": 0.64, + "metric": 0.8942240080361629, + "metric_std": 0.002351653186861771, + "actual_fraction": 0.6336944823132667, + "actual_fraction_std": 0.004682957397937968 + }, + { + "budget": 0.65, + "metric": 0.894683217335151, + "metric_std": 0.0024111050612602228, + "actual_fraction": 0.6440266915405038, + "actual_fraction_std": 0.003861734922803163 + }, + { + "budget": 0.66, + "metric": 0.8954437827366004, + "metric_std": 0.0022822151023945856, + "actual_fraction": 0.6536413862380713, + "actual_fraction_std": 0.0031453827090204417 + }, + { + "budget": 0.67, + "metric": 0.8957881897108416, + "metric_std": 0.0022002089046255307, + "actual_fraction": 0.6636435387816604, + "actual_fraction_std": 0.0031787594274059494 + }, + { + "budget": 0.68, + "metric": 0.8966779077276316, + "metric_std": 0.0024626536990357186, + "actual_fraction": 0.6739900982994905, + "actual_fraction_std": 0.0043814236174423045 + }, + { + "budget": 0.69, + "metric": 0.8973810719667072, + "metric_std": 0.002618878888000752, + "actual_fraction": 0.6838343976465524, + "actual_fraction_std": 0.005174966290006192 + }, + { + "budget": 0.7, + "metric": 0.8978115806845087, + "metric_std": 0.00238156464616349, + "actual_fraction": 0.6935208437970869, + "actual_fraction_std": 0.005144033757716118 + }, + { + "budget": 0.71, + "metric": 0.8982420894023104, + "metric_std": 0.0020505104067132644, + "actual_fraction": 0.7032503408194015, + "actual_fraction_std": 0.0054999723573424305 + }, + { + "budget": 0.72, + "metric": 0.8989309033507927, + "metric_std": 0.0018534943323925681, + "actual_fraction": 0.7143574657386811, + "actual_fraction_std": 0.004916152786778245 + }, + { + "budget": 0.73, + "metric": 0.8993040109062207, + "metric_std": 0.0018810652993101501, + "actual_fraction": 0.7239578101456553, + "actual_fraction_std": 0.0045298414692036635 + }, + { + "budget": 0.74, + "metric": 0.8996771184616488, + "metric_std": 0.0017541726193314399, + "actual_fraction": 0.7337734089115304, + "actual_fraction_std": 0.003809116980006407 + }, + { + "budget": 0.75, + "metric": 0.9004233335725049, + "metric_std": 0.0016906566951307234, + "actual_fraction": 0.7443208724976681, + "actual_fraction_std": 0.005847105102169993 + }, + { + "budget": 0.76, + "metric": 0.9007964411279328, + "metric_std": 0.0015560308006743353, + "actual_fraction": 0.7538781660328622, + "actual_fraction_std": 0.006167996289709345 + }, + { + "budget": 0.77, + "metric": 0.9015283059481953, + "metric_std": 0.0009924032662421214, + "actual_fraction": 0.7649135394991748, + "actual_fraction_std": 0.0045466309018659095 + }, + { + "budget": 0.78, + "metric": 0.9018583626318433, + "metric_std": 0.0011484716064987896, + "actual_fraction": 0.7738681208294469, + "actual_fraction_std": 0.00416733278896262 + }, + { + "budget": 0.79, + "metric": 0.902403673674392, + "metric_std": 0.001065937807910118, + "actual_fraction": 0.7842003300566837, + "actual_fraction_std": 0.004183977281372374 + }, + { + "budget": 0.8, + "metric": 0.9028198321016001, + "metric_std": 0.0011796514960251275, + "actual_fraction": 0.7942168328908661, + "actual_fraction_std": 0.003527151382974944 + }, + { + "budget": 0.81, + "metric": 0.903522996340676, + "metric_std": 0.0011260635630396412, + "actual_fraction": 0.8041472339814881, + "actual_fraction_std": 0.0027145386875857694 + }, + { + "budget": 0.82, + "metric": 0.9037669512807633, + "metric_std": 0.0017133429679527894, + "actual_fraction": 0.8140776350721103, + "actual_fraction_std": 0.0034854519198318345 + }, + { + "budget": 0.83, + "metric": 0.9041544091267848, + "metric_std": 0.0016891334434518735, + "actual_fraction": 0.8233909736672167, + "actual_fraction_std": 0.004286391204150996 + }, + { + "budget": 0.84, + "metric": 0.9044701155198392, + "metric_std": 0.0017899075521050022, + "actual_fraction": 0.8336370811508933, + "actual_fraction_std": 0.005093999444026073 + }, + { + "budget": 0.85, + "metric": 0.9047284207505204, + "metric_std": 0.0013916822738246173, + "actual_fraction": 0.8440410418310972, + "actual_fraction_std": 0.0041796684071849916 + }, + { + "budget": 0.86, + "metric": 0.9053311329554423, + "metric_std": 0.001634295871709295, + "actual_fraction": 0.8543445504771471, + "actual_fraction_std": 0.0059043642414655624 + }, + { + "budget": 0.87, + "metric": 0.9055320370237497, + "metric_std": 0.0013732485821399182, + "actual_fraction": 0.8658391332424481, + "actual_fraction_std": 0.005566035430532028 + }, + { + "budget": 0.88, + "metric": 0.9058620937073976, + "metric_std": 0.0011833552774966392, + "actual_fraction": 0.8769462581617278, + "actual_fraction_std": 0.0048225837738125645 + }, + { + "budget": 0.89, + "metric": 0.9063787041687595, + "metric_std": 0.001359874606068116, + "actual_fraction": 0.8868910095429431, + "actual_fraction_std": 0.004274965778338865 + }, + { + "budget": 0.9, + "metric": 0.906694410561814, + "metric_std": 0.0016208524586654276, + "actual_fraction": 0.8976967783597619, + "actual_fraction_std": 0.002899682086346399 + }, + { + "budget": 0.91, + "metric": 0.9070244672454617, + "metric_std": 0.001580487611653376, + "actual_fraction": 0.9078711343904715, + "actual_fraction_std": 0.002843024001744828 + }, + { + "budget": 0.92, + "metric": 0.9071966707325823, + "metric_std": 0.001760909885549338, + "actual_fraction": 0.9177871851905002, + "actual_fraction_std": 0.002106855300838142 + }, + { + "budget": 0.93, + "metric": 0.9071536198608022, + "metric_std": 0.0017450508010354285, + "actual_fraction": 0.9276458348281553, + "actual_fraction_std": 0.0018562698570566535 + }, + { + "budget": 0.94, + "metric": 0.9067087608524071, + "metric_std": 0.0018246607513596942, + "actual_fraction": 0.9376479873717443, + "actual_fraction_std": 0.0020590291816433327 + }, + { + "budget": 0.95, + "metric": 0.9064074047499462, + "metric_std": 0.0013245870140448346, + "actual_fraction": 0.9488986151969577, + "actual_fraction_std": 0.0014576321356509156 + }, + { + "budget": 0.96, + "metric": 0.9062065006816388, + "metric_std": 0.0010598833581225283, + "actual_fraction": 0.9587859654157997, + "actual_fraction_std": 0.0012423572942832195 + }, + { + "budget": 0.97, + "metric": 0.9059768960321447, + "metric_std": 0.0009924032662420948, + "actual_fraction": 0.9692616775489704, + "actual_fraction_std": 0.001078183468824685 + }, + { + "budget": 0.98, + "metric": 0.9056037884767166, + "metric_std": 0.0010528165780024498, + "actual_fraction": 0.9787759202123842, + "actual_fraction_std": 0.0015300061519492266 + }, + { + "budget": 0.99, + "metric": 0.9048719236564541, + "metric_std": 0.000440556258391596, + "actual_fraction": 0.9893377340891153, + "actual_fraction_std": 0.0009140921090157555 + }, + { + "budget": 1.0, + "metric": 0.9006242376408122, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 5165, + "rescuable": 1154, + "harmful": 408, + "both_wrong": 255, + "unit": "sample", + "total": 6982 + }, + "test": { + "both_correct": 10305, + "rescuable": 2247, + "harmful": 845, + "both_wrong": 540, + "unit": "sample", + "total": 13937 + } + }, + "small": { + "validation": 0.7981953594958464, + "test": 0.8000287005811868 + }, + "large": { + "validation": 0.9050415353766829, + "test": 0.9006242376408122 + }, + "learned_minus_confidence_test": 0.0036880246824998464, + "learned_minus_small_test": 0.10607734806629832, + "learned_minus_large_test": 0.0054818110066728876, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_nlvr2_Qwen3.8" + }, + { + "key": [ + "nlvr2", + "vilt", + "MiniCPM" + ], + "expert_display": "ViLT", + "run": "five_seed_vilt_MiniCPM_four_difference", + "validation_rows": 6982, + "test_rows": 13937, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/vilt/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 96, + "validation_area_0_25": 0.7926095674591808, + "validation_point": { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.8997421942136924, + "rescued": 1385, + "harmed": 379, + "missed_rescue": 4, + "avoided_harm": 9, + "selected_both_correct": 4621, + "selected_both_wrong": 317 + }, + "test_point": { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9593169261677549, + "metric": 0.906436105331133, + "rescued": 2682, + "harmed": 700, + "missed_rescue": 24, + "avoided_harm": 11, + "selected_both_correct": 9413, + "selected_both_wrong": 575 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7556574047550845, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1389, + "avoided_harm": 388, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5127503275871277, + "operator": "ge", + "actual_fraction": 0.009882555141793183, + "metric": 0.7590948152391864, + "rescued": 31, + "harmed": 7, + "missed_rescue": 1358, + "avoided_harm": 381, + "selected_both_correct": 27, + "selected_both_wrong": 4 + }, + { + "budget": 0.02, + "threshold": -0.5296345353126526, + "operator": "ge", + "actual_fraction": 0.01990833572042395, + "metric": 0.7619593239759381, + "rescued": 55, + "harmed": 11, + "missed_rescue": 1334, + "avoided_harm": 377, + "selected_both_correct": 64, + "selected_both_wrong": 9 + }, + { + "budget": 0.03, + "threshold": -0.5434877276420593, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.7639644800916643, + "rescued": 80, + "harmed": 22, + "missed_rescue": 1309, + "avoided_harm": 366, + "selected_both_correct": 93, + "selected_both_wrong": 14 + }, + { + "budget": 0.04, + "threshold": -0.56125408411026, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.7672586651389287, + "rescued": 109, + "harmed": 28, + "missed_rescue": 1280, + "avoided_harm": 360, + "selected_both_correct": 124, + "selected_both_wrong": 18 + }, + { + "budget": 0.05, + "threshold": -0.5779195427894592, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.771268977370381, + "rescued": 137, + "harmed": 28, + "missed_rescue": 1252, + "avoided_harm": 360, + "selected_both_correct": 158, + "selected_both_wrong": 26 + }, + { + "budget": 0.06, + "threshold": -0.5911363363265991, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.7734173589229447, + "rescued": 160, + "harmed": 36, + "missed_rescue": 1229, + "avoided_harm": 352, + "selected_both_correct": 189, + "selected_both_wrong": 33 + }, + { + "budget": 0.07, + "threshold": -0.6087464094161987, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.7769979948438843, + "rescued": 192, + "harmed": 43, + "missed_rescue": 1197, + "avoided_harm": 345, + "selected_both_correct": 215, + "selected_both_wrong": 38 + }, + { + "budget": 0.08, + "threshold": -0.6274068355560303, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.7805786307648238, + "rescued": 223, + "harmed": 49, + "missed_rescue": 1166, + "avoided_harm": 339, + "selected_both_correct": 245, + "selected_both_wrong": 41 + }, + { + "budget": 0.09, + "threshold": -0.6465112566947937, + "operator": "ge", + "actual_fraction": 0.08994557433400172, + "metric": 0.783586364938413, + "rescued": 249, + "harmed": 54, + "missed_rescue": 1140, + "avoided_harm": 334, + "selected_both_correct": 279, + "selected_both_wrong": 46 + }, + { + "budget": 0.1, + "threshold": -0.6620025038719177, + "operator": "ge", + "actual_fraction": 0.09997135491263248, + "metric": 0.7877399026067029, + "rescued": 281, + "harmed": 57, + "missed_rescue": 1108, + "avoided_harm": 331, + "selected_both_correct": 307, + "selected_both_wrong": 53 + }, + { + "budget": 0.11, + "threshold": -0.6800539493560791, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.7901747350329419, + "rescued": 303, + "harmed": 62, + "missed_rescue": 1086, + "avoided_harm": 326, + "selected_both_correct": 345, + "selected_both_wrong": 58 + }, + { + "budget": 0.12, + "threshold": -0.6957284212112427, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.7926095674591808, + "rescued": 329, + "harmed": 71, + "missed_rescue": 1060, + "avoided_harm": 317, + "selected_both_correct": 372, + "selected_both_wrong": 65 + }, + { + "budget": 0.13, + "threshold": -0.7128273248672485, + "operator": "ge", + "actual_fraction": 0.1299054712116872, + "metric": 0.7954740761959324, + "rescued": 355, + "harmed": 77, + "missed_rescue": 1034, + "avoided_harm": 311, + "selected_both_correct": 405, + "selected_both_wrong": 70 + }, + { + "budget": 0.14, + "threshold": -0.7332167625427246, + "operator": "ge", + "actual_fraction": 0.13993125179031796, + "metric": 0.7977656831853337, + "rescued": 379, + "harmed": 85, + "missed_rescue": 1010, + "avoided_harm": 303, + "selected_both_correct": 439, + "selected_both_wrong": 74 + }, + { + "budget": 0.15, + "threshold": -0.7503995895385742, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8010598682325981, + "rescued": 408, + "harmed": 91, + "missed_rescue": 981, + "avoided_harm": 297, + "selected_both_correct": 468, + "selected_both_wrong": 80 + }, + { + "budget": 0.16, + "threshold": -0.7655540704727173, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8026353480378116, + "rescued": 428, + "harmed": 100, + "missed_rescue": 961, + "avoided_harm": 288, + "selected_both_correct": 507, + "selected_both_wrong": 82 + }, + { + "budget": 0.17, + "threshold": -0.7812193036079407, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.8059295330850759, + "rescued": 453, + "harmed": 102, + "missed_rescue": 936, + "avoided_harm": 286, + "selected_both_correct": 543, + "selected_both_wrong": 88 + }, + { + "budget": 0.18, + "threshold": -0.7954917550086975, + "operator": "ge", + "actual_fraction": 0.17989114866800343, + "metric": 0.806932111142939, + "rescued": 470, + "harmed": 112, + "missed_rescue": 919, + "avoided_harm": 276, + "selected_both_correct": 584, + "selected_both_wrong": 90 + }, + { + "budget": 0.19, + "threshold": -0.8097342252731323, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8093669435691779, + "rescued": 494, + "harmed": 119, + "missed_rescue": 895, + "avoided_harm": 269, + "selected_both_correct": 619, + "selected_both_wrong": 94 + }, + { + "budget": 0.2, + "threshold": -0.8232117891311646, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8122314523059295, + "rescued": 517, + "harmed": 122, + "missed_rescue": 872, + "avoided_harm": 266, + "selected_both_correct": 658, + "selected_both_wrong": 99 + }, + { + "budget": 0.21, + "threshold": -0.8359466791152954, + "operator": "ge", + "actual_fraction": 0.20996849040389573, + "metric": 0.8146662847321684, + "rescued": 539, + "harmed": 127, + "missed_rescue": 850, + "avoided_harm": 261, + "selected_both_correct": 697, + "selected_both_wrong": 103 + }, + { + "budget": 0.22, + "threshold": -0.8473842144012451, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8168146662847322, + "rescued": 560, + "harmed": 133, + "missed_rescue": 829, + "avoided_harm": 255, + "selected_both_correct": 734, + "selected_both_wrong": 109 + }, + { + "budget": 0.23, + "threshold": -0.8595863580703735, + "operator": "ge", + "actual_fraction": 0.22987682612431967, + "metric": 0.8203953022056717, + "rescued": 588, + "harmed": 136, + "missed_rescue": 801, + "avoided_harm": 252, + "selected_both_correct": 767, + "selected_both_wrong": 114 + }, + { + "budget": 0.24, + "threshold": -0.8704345226287842, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8225436837582355, + "rescued": 610, + "harmed": 143, + "missed_rescue": 779, + "avoided_harm": 245, + "selected_both_correct": 805, + "selected_both_wrong": 117 + }, + { + "budget": 0.25, + "threshold": -0.8811548352241516, + "operator": "ge", + "actual_fraction": 0.2499283872815812, + "metric": 0.8258378688054998, + "rescued": 637, + "harmed": 147, + "missed_rescue": 752, + "avoided_harm": 241, + "selected_both_correct": 836, + "selected_both_wrong": 125 + }, + { + "budget": 0.26, + "threshold": -0.8929013609886169, + "operator": "ge", + "actual_fraction": 0.25995416786021197, + "metric": 0.8279862503580636, + "rescued": 656, + "harmed": 151, + "missed_rescue": 733, + "avoided_harm": 237, + "selected_both_correct": 880, + "selected_both_wrong": 128 + }, + { + "budget": 0.27, + "threshold": -0.903549313545227, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.829561730163277, + "rescued": 675, + "harmed": 159, + "missed_rescue": 714, + "avoided_harm": 229, + "selected_both_correct": 915, + "selected_both_wrong": 136 + }, + { + "budget": 0.28, + "threshold": -0.9115740656852722, + "operator": "ge", + "actual_fraction": 0.2798625035806359, + "metric": 0.8321397880263535, + "rescued": 697, + "harmed": 163, + "missed_rescue": 692, + "avoided_harm": 225, + "selected_both_correct": 953, + "selected_both_wrong": 141 + }, + { + "budget": 0.29, + "threshold": -0.9182595014572144, + "operator": "ge", + "actual_fraction": 0.2898882841592667, + "metric": 0.83471784588943, + "rescued": 718, + "harmed": 166, + "missed_rescue": 671, + "avoided_harm": 222, + "selected_both_correct": 997, + "selected_both_wrong": 143 + }, + { + "budget": 0.3, + "threshold": -0.9258768558502197, + "operator": "ge", + "actual_fraction": 0.29991406473789745, + "metric": 0.8361501002578058, + "rescued": 735, + "harmed": 173, + "missed_rescue": 654, + "avoided_harm": 215, + "selected_both_correct": 1041, + "selected_both_wrong": 145 + }, + { + "budget": 0.31, + "threshold": -0.9331877827644348, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8381552563735319, + "rescued": 754, + "harmed": 178, + "missed_rescue": 635, + "avoided_harm": 210, + "selected_both_correct": 1080, + "selected_both_wrong": 152 + }, + { + "budget": 0.32, + "threshold": -0.9399217367172241, + "operator": "ge", + "actual_fraction": 0.319965625895159, + "metric": 0.8405900887997708, + "rescued": 774, + "harmed": 181, + "missed_rescue": 615, + "avoided_harm": 207, + "selected_both_correct": 1123, + "selected_both_wrong": 156 + }, + { + "budget": 0.33, + "threshold": -0.9453923106193542, + "operator": "ge", + "actual_fraction": 0.3299914064737898, + "metric": 0.8423087940418218, + "rescued": 792, + "harmed": 187, + "missed_rescue": 597, + "avoided_harm": 201, + "selected_both_correct": 1162, + "selected_both_wrong": 163 + }, + { + "budget": 0.34, + "threshold": -0.9515207409858704, + "operator": "ge", + "actual_fraction": 0.3398739616155829, + "metric": 0.8438842738470352, + "rescued": 807, + "harmed": 191, + "missed_rescue": 582, + "avoided_harm": 197, + "selected_both_correct": 1206, + "selected_both_wrong": 169 + }, + { + "budget": 0.35, + "threshold": -0.9567410349845886, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8466055571469493, + "rescued": 829, + "harmed": 194, + "missed_rescue": 560, + "avoided_harm": 194, + "selected_both_correct": 1245, + "selected_both_wrong": 175 + }, + { + "budget": 0.36, + "threshold": -0.9606627225875854, + "operator": "ge", + "actual_fraction": 0.35992552277284445, + "metric": 0.8498997421942137, + "rescued": 854, + "harmed": 196, + "missed_rescue": 535, + "avoided_harm": 192, + "selected_both_correct": 1287, + "selected_both_wrong": 176 + }, + { + "budget": 0.37, + "threshold": -0.9650874733924866, + "operator": "ge", + "actual_fraction": 0.36995130335147525, + "metric": 0.8519048983099399, + "rescued": 872, + "harmed": 200, + "missed_rescue": 517, + "avoided_harm": 188, + "selected_both_correct": 1329, + "selected_both_wrong": 182 + }, + { + "budget": 0.38, + "threshold": -0.9686882495880127, + "operator": "ge", + "actual_fraction": 0.379977083930106, + "metric": 0.8547694070466915, + "rescued": 893, + "harmed": 201, + "missed_rescue": 496, + "avoided_harm": 187, + "selected_both_correct": 1372, + "selected_both_wrong": 187 + }, + { + "budget": 0.39, + "threshold": -0.971785306930542, + "operator": "ge", + "actual_fraction": 0.3898596390718992, + "metric": 0.8557719851045545, + "rescued": 903, + "harmed": 204, + "missed_rescue": 486, + "avoided_harm": 184, + "selected_both_correct": 1423, + "selected_both_wrong": 192 + }, + { + "budget": 0.4, + "threshold": -0.9748144745826721, + "operator": "ge", + "actual_fraction": 0.3998854196505299, + "metric": 0.8584932684044686, + "rescued": 923, + "harmed": 205, + "missed_rescue": 466, + "avoided_harm": 183, + "selected_both_correct": 1468, + "selected_both_wrong": 196 + }, + { + "budget": 0.41, + "threshold": -0.977255642414093, + "operator": "ge", + "actual_fraction": 0.4099112002291607, + "metric": 0.8596390718991693, + "rescued": 935, + "harmed": 209, + "missed_rescue": 454, + "avoided_harm": 179, + "selected_both_correct": 1520, + "selected_both_wrong": 198 + }, + { + "budget": 0.42, + "threshold": -0.9794421792030334, + "operator": "ge", + "actual_fraction": 0.41993698080779146, + "metric": 0.861787453451733, + "rescued": 954, + "harmed": 213, + "missed_rescue": 435, + "avoided_harm": 175, + "selected_both_correct": 1560, + "selected_both_wrong": 205 + }, + { + "budget": 0.43, + "threshold": -0.981472373008728, + "operator": "ge", + "actual_fraction": 0.42996276138642225, + "metric": 0.8632197078201088, + "rescued": 969, + "harmed": 218, + "missed_rescue": 420, + "avoided_harm": 170, + "selected_both_correct": 1606, + "selected_both_wrong": 209 + }, + { + "budget": 0.44, + "threshold": -0.9835578799247742, + "operator": "ge", + "actual_fraction": 0.439988541965053, + "metric": 0.8649384130621598, + "rescued": 986, + "harmed": 223, + "missed_rescue": 403, + "avoided_harm": 165, + "selected_both_correct": 1653, + "selected_both_wrong": 210 + }, + { + "budget": 0.45, + "threshold": -0.9851564168930054, + "operator": "ge", + "actual_fraction": 0.4498710971068462, + "metric": 0.8663706674305357, + "rescued": 1001, + "harmed": 228, + "missed_rescue": 388, + "avoided_harm": 160, + "selected_both_correct": 1699, + "selected_both_wrong": 213 + }, + { + "budget": 0.46, + "threshold": -0.986689031124115, + "operator": "ge", + "actual_fraction": 0.45989687768547693, + "metric": 0.8673732454883988, + "rescued": 1016, + "harmed": 236, + "missed_rescue": 373, + "avoided_harm": 152, + "selected_both_correct": 1742, + "selected_both_wrong": 217 + }, + { + "budget": 0.47, + "threshold": -0.9879001379013062, + "operator": "ge", + "actual_fraction": 0.46992265826410773, + "metric": 0.8680893726725867, + "rescued": 1024, + "harmed": 239, + "missed_rescue": 365, + "avoided_harm": 149, + "selected_both_correct": 1797, + "selected_both_wrong": 221 + }, + { + "budget": 0.48, + "threshold": -0.9891784191131592, + "operator": "ge", + "actual_fraction": 0.47994843884273847, + "metric": 0.8700945287883128, + "rescued": 1043, + "harmed": 244, + "missed_rescue": 346, + "avoided_harm": 144, + "selected_both_correct": 1839, + "selected_both_wrong": 225 + }, + { + "budget": 0.49, + "threshold": -0.990114688873291, + "operator": "ge", + "actual_fraction": 0.48997421942136926, + "metric": 0.8715267831566886, + "rescued": 1057, + "harmed": 248, + "missed_rescue": 332, + "avoided_harm": 140, + "selected_both_correct": 1889, + "selected_both_wrong": 227 + }, + { + "budget": 0.5, + "threshold": -0.9910277724266052, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.8722429103408765, + "rescued": 1066, + "harmed": 252, + "missed_rescue": 323, + "avoided_harm": 136, + "selected_both_correct": 1942, + "selected_both_wrong": 231 + }, + { + "budget": 0.51, + "threshold": -0.9918718934059143, + "operator": "ge", + "actual_fraction": 0.5098825551417931, + "metric": 0.8725293612145517, + "rescued": 1074, + "harmed": 258, + "missed_rescue": 315, + "avoided_harm": 130, + "selected_both_correct": 1996, + "selected_both_wrong": 232 + }, + { + "budget": 0.52, + "threshold": -0.9926809668540955, + "operator": "ge", + "actual_fraction": 0.5199083357204239, + "metric": 0.8729590375250644, + "rescued": 1083, + "harmed": 264, + "missed_rescue": 306, + "avoided_harm": 124, + "selected_both_correct": 2050, + "selected_both_wrong": 233 + }, + { + "budget": 0.53, + "threshold": -0.9932646751403809, + "operator": "ge", + "actual_fraction": 0.5299341162990547, + "metric": 0.87381839014609, + "rescued": 1090, + "harmed": 265, + "missed_rescue": 299, + "avoided_harm": 123, + "selected_both_correct": 2108, + "selected_both_wrong": 237 + }, + { + "budget": 0.54, + "threshold": -0.9938417673110962, + "operator": "ge", + "actual_fraction": 0.5399598968776855, + "metric": 0.8743912918934402, + "rescued": 1097, + "harmed": 268, + "missed_rescue": 292, + "avoided_harm": 120, + "selected_both_correct": 2166, + "selected_both_wrong": 239 + }, + { + "budget": 0.55, + "threshold": -0.9944412112236023, + "operator": "ge", + "actual_fraction": 0.5499856774563162, + "metric": 0.8751074190776282, + "rescued": 1107, + "harmed": 273, + "missed_rescue": 282, + "avoided_harm": 115, + "selected_both_correct": 2220, + "selected_both_wrong": 240 + }, + { + "budget": 0.56, + "threshold": -0.9949513673782349, + "operator": "ge", + "actual_fraction": 0.5598682325981095, + "metric": 0.8761099971354913, + "rescued": 1118, + "harmed": 277, + "missed_rescue": 271, + "avoided_harm": 111, + "selected_both_correct": 2272, + "selected_both_wrong": 242 + }, + { + "budget": 0.57, + "threshold": -0.9954326152801514, + "operator": "ge", + "actual_fraction": 0.5698940131767402, + "metric": 0.8776854769407046, + "rescued": 1131, + "harmed": 279, + "missed_rescue": 258, + "avoided_harm": 109, + "selected_both_correct": 2323, + "selected_both_wrong": 246 + }, + { + "budget": 0.58, + "threshold": -0.9958608746528625, + "operator": "ge", + "actual_fraction": 0.579919793755371, + "metric": 0.878258378688055, + "rescued": 1142, + "harmed": 286, + "missed_rescue": 247, + "avoided_harm": 102, + "selected_both_correct": 2371, + "selected_both_wrong": 250 + }, + { + "budget": 0.59, + "threshold": -0.9962128400802612, + "operator": "ge", + "actual_fraction": 0.5899455743340017, + "metric": 0.8792609567459181, + "rescued": 1154, + "harmed": 291, + "missed_rescue": 235, + "avoided_harm": 97, + "selected_both_correct": 2421, + "selected_both_wrong": 253 + }, + { + "budget": 0.6, + "threshold": -0.9965764880180359, + "operator": "ge", + "actual_fraction": 0.5999713549126325, + "metric": 0.8796906330564308, + "rescued": 1161, + "harmed": 295, + "missed_rescue": 228, + "avoided_harm": 93, + "selected_both_correct": 2479, + "selected_both_wrong": 254 + }, + { + "budget": 0.61, + "threshold": -0.9968799352645874, + "operator": "ge", + "actual_fraction": 0.6099971354912632, + "metric": 0.8808364365511315, + "rescued": 1174, + "harmed": 300, + "missed_rescue": 215, + "avoided_harm": 88, + "selected_both_correct": 2530, + "selected_both_wrong": 255 + }, + { + "budget": 0.62, + "threshold": -0.9971383810043335, + "operator": "ge", + "actual_fraction": 0.6198796906330565, + "metric": 0.8811228874248066, + "rescued": 1178, + "harmed": 302, + "missed_rescue": 211, + "avoided_harm": 86, + "selected_both_correct": 2591, + "selected_both_wrong": 257 + }, + { + "budget": 0.63, + "threshold": -0.997377872467041, + "operator": "ge", + "actual_fraction": 0.6299054712116872, + "metric": 0.8822686909195073, + "rescued": 1188, + "harmed": 304, + "missed_rescue": 201, + "avoided_harm": 84, + "selected_both_correct": 2645, + "selected_both_wrong": 261 + }, + { + "budget": 0.64, + "threshold": -0.9976021647453308, + "operator": "ge", + "actual_fraction": 0.639931251790318, + "metric": 0.8832712689773704, + "rescued": 1200, + "harmed": 309, + "missed_rescue": 189, + "avoided_harm": 79, + "selected_both_correct": 2695, + "selected_both_wrong": 264 + }, + { + "budget": 0.65, + "threshold": -0.9978214502334595, + "operator": "ge", + "actual_fraction": 0.6499570323689488, + "metric": 0.8834144944142079, + "rescued": 1205, + "harmed": 313, + "missed_rescue": 184, + "avoided_harm": 75, + "selected_both_correct": 2753, + "selected_both_wrong": 267 + }, + { + "budget": 0.66, + "threshold": -0.9980514049530029, + "operator": "ge", + "actual_fraction": 0.6599828129475795, + "metric": 0.8847035233457462, + "rescued": 1215, + "harmed": 314, + "missed_rescue": 174, + "avoided_harm": 74, + "selected_both_correct": 2810, + "selected_both_wrong": 269 + }, + { + "budget": 0.67, + "threshold": -0.9982016086578369, + "operator": "ge", + "actual_fraction": 0.6698653680893727, + "metric": 0.8855628759667717, + "rescued": 1223, + "harmed": 316, + "missed_rescue": 166, + "avoided_harm": 72, + "selected_both_correct": 2867, + "selected_both_wrong": 271 + }, + { + "budget": 0.68, + "threshold": -0.9983693957328796, + "operator": "ge", + "actual_fraction": 0.6798911486680035, + "metric": 0.8862790031509596, + "rescued": 1231, + "harmed": 319, + "missed_rescue": 158, + "avoided_harm": 69, + "selected_both_correct": 2925, + "selected_both_wrong": 272 + }, + { + "budget": 0.69, + "threshold": -0.9984915256500244, + "operator": "ge", + "actual_fraction": 0.6899169292466342, + "metric": 0.887854482956173, + "rescued": 1243, + "harmed": 320, + "missed_rescue": 146, + "avoided_harm": 68, + "selected_both_correct": 2980, + "selected_both_wrong": 274 + }, + { + "budget": 0.7, + "threshold": -0.9986109733581543, + "operator": "gt", + "actual_fraction": 0.6997994843884274, + "metric": 0.8881409338298482, + "rescued": 1248, + "harmed": 323, + "missed_rescue": 141, + "avoided_harm": 65, + "selected_both_correct": 3039, + "selected_both_wrong": 276 + }, + { + "budget": 0.71, + "threshold": -0.9986966252326965, + "operator": "ge", + "actual_fraction": 0.7099684904038958, + "metric": 0.888857061014036, + "rescued": 1253, + "harmed": 323, + "missed_rescue": 136, + "avoided_harm": 65, + "selected_both_correct": 3105, + "selected_both_wrong": 276 + }, + { + "budget": 0.72, + "threshold": -0.9987963438034058, + "operator": "ge", + "actual_fraction": 0.7199942709825266, + "metric": 0.8890002864508737, + "rescued": 1258, + "harmed": 327, + "missed_rescue": 131, + "avoided_harm": 61, + "selected_both_correct": 3164, + "selected_both_wrong": 278 + }, + { + "budget": 0.73, + "threshold": -0.998875081539154, + "operator": "ge", + "actual_fraction": 0.7298768261243197, + "metric": 0.8897164136350616, + "rescued": 1264, + "harmed": 328, + "missed_rescue": 125, + "avoided_harm": 60, + "selected_both_correct": 3223, + "selected_both_wrong": 281 + }, + { + "budget": 0.74, + "threshold": -0.9989551305770874, + "operator": "ge", + "actual_fraction": 0.7399026067029505, + "metric": 0.8901460899455743, + "rescued": 1272, + "harmed": 333, + "missed_rescue": 117, + "avoided_harm": 55, + "selected_both_correct": 3277, + "selected_both_wrong": 284 + }, + { + "budget": 0.75, + "threshold": -0.9990480542182922, + "operator": "ge", + "actual_fraction": 0.7499283872815812, + "metric": 0.8911486680034374, + "rescued": 1280, + "harmed": 334, + "missed_rescue": 109, + "avoided_harm": 54, + "selected_both_correct": 3335, + "selected_both_wrong": 287 + }, + { + "budget": 0.76, + "threshold": -0.999126136302948, + "operator": "ge", + "actual_fraction": 0.759954167860212, + "metric": 0.8915783443139501, + "rescued": 1286, + "harmed": 337, + "missed_rescue": 103, + "avoided_harm": 51, + "selected_both_correct": 3393, + "selected_both_wrong": 290 + }, + { + "budget": 0.77, + "threshold": -0.999184787273407, + "operator": "gt", + "actual_fraction": 0.7698367230020051, + "metric": 0.8920080206244629, + "rescued": 1291, + "harmed": 339, + "missed_rescue": 98, + "avoided_harm": 49, + "selected_both_correct": 3454, + "selected_both_wrong": 291 + }, + { + "budget": 0.78, + "threshold": -0.9992268085479736, + "operator": "ge", + "actual_fraction": 0.7798625035806359, + "metric": 0.893010598682326, + "rescued": 1301, + "harmed": 342, + "missed_rescue": 88, + "avoided_harm": 46, + "selected_both_correct": 3510, + "selected_both_wrong": 292 + }, + { + "budget": 0.79, + "threshold": -0.9992831349372864, + "operator": "ge", + "actual_fraction": 0.7898882841592667, + "metric": 0.8941564021770266, + "rescued": 1312, + "harmed": 345, + "missed_rescue": 77, + "avoided_harm": 43, + "selected_both_correct": 3564, + "selected_both_wrong": 294 + }, + { + "budget": 0.8, + "threshold": -0.9993318915367126, + "operator": "ge", + "actual_fraction": 0.7999140647378975, + "metric": 0.8942996276138642, + "rescued": 1314, + "harmed": 346, + "missed_rescue": 75, + "avoided_harm": 42, + "selected_both_correct": 3631, + "selected_both_wrong": 294 + }, + { + "budget": 0.81, + "threshold": -0.9993773102760315, + "operator": "ge", + "actual_fraction": 0.8099398453165282, + "metric": 0.8944428530507018, + "rescued": 1318, + "harmed": 349, + "missed_rescue": 71, + "avoided_harm": 39, + "selected_both_correct": 3693, + "selected_both_wrong": 295 + }, + { + "budget": 0.82, + "threshold": -0.9994175434112549, + "operator": "ge", + "actual_fraction": 0.819965625895159, + "metric": 0.8945860784875393, + "rescued": 1323, + "harmed": 353, + "missed_rescue": 66, + "avoided_harm": 35, + "selected_both_correct": 3752, + "selected_both_wrong": 297 + }, + { + "budget": 0.83, + "threshold": -0.999452531337738, + "operator": "ge", + "actual_fraction": 0.8299914064737898, + "metric": 0.894729303924377, + "rescued": 1327, + "harmed": 356, + "missed_rescue": 62, + "avoided_harm": 32, + "selected_both_correct": 3815, + "selected_both_wrong": 297 + }, + { + "budget": 0.84, + "threshold": -0.9994945526123047, + "operator": "ge", + "actual_fraction": 0.8398739616155829, + "metric": 0.8951589802348897, + "rescued": 1332, + "harmed": 358, + "missed_rescue": 57, + "avoided_harm": 30, + "selected_both_correct": 3876, + "selected_both_wrong": 298 + }, + { + "budget": 0.85, + "threshold": -0.9995274543762207, + "operator": "gt", + "actual_fraction": 0.8497565167573761, + "metric": 0.8954454311085649, + "rescued": 1335, + "harmed": 359, + "missed_rescue": 54, + "avoided_harm": 29, + "selected_both_correct": 3938, + "selected_both_wrong": 301 + }, + { + "budget": 0.86, + "threshold": -0.9995617270469666, + "operator": "ge", + "actual_fraction": 0.8599255227728445, + "metric": 0.8958751074190776, + "rescued": 1340, + "harmed": 361, + "missed_rescue": 49, + "avoided_harm": 27, + "selected_both_correct": 3999, + "selected_both_wrong": 304 + }, + { + "budget": 0.87, + "threshold": -0.9995949864387512, + "operator": "ge", + "actual_fraction": 0.8699513033514752, + "metric": 0.896448009166428, + "rescued": 1347, + "harmed": 364, + "missed_rescue": 42, + "avoided_harm": 24, + "selected_both_correct": 4059, + "selected_both_wrong": 304 + }, + { + "budget": 0.88, + "threshold": -0.9996248483657837, + "operator": "ge", + "actual_fraction": 0.879977083930106, + "metric": 0.8968776854769407, + "rescued": 1351, + "harmed": 365, + "missed_rescue": 38, + "avoided_harm": 23, + "selected_both_correct": 4123, + "selected_both_wrong": 305 + }, + { + "budget": 0.89, + "threshold": -0.9996511936187744, + "operator": "ge", + "actual_fraction": 0.8898596390718991, + "metric": 0.897450587224291, + "rescued": 1358, + "harmed": 368, + "missed_rescue": 31, + "avoided_harm": 20, + "selected_both_correct": 4180, + "selected_both_wrong": 307 + }, + { + "budget": 0.9, + "threshold": -0.9996830224990845, + "operator": "ge", + "actual_fraction": 0.8998854196505299, + "metric": 0.8977370380979662, + "rescued": 1361, + "harmed": 369, + "missed_rescue": 28, + "avoided_harm": 19, + "selected_both_correct": 4244, + "selected_both_wrong": 309 + }, + { + "budget": 0.91, + "threshold": -0.9997097849845886, + "operator": "gt", + "actual_fraction": 0.9097679747923231, + "metric": 0.8984531652821541, + "rescued": 1369, + "harmed": 372, + "missed_rescue": 20, + "avoided_harm": 16, + "selected_both_correct": 4302, + "selected_both_wrong": 309 + }, + { + "budget": 0.92, + "threshold": -0.9997391104698181, + "operator": "ge", + "actual_fraction": 0.9199369808077915, + "metric": 0.8985963907189917, + "rescued": 1372, + "harmed": 374, + "missed_rescue": 17, + "avoided_harm": 14, + "selected_both_correct": 4367, + "selected_both_wrong": 310 + }, + { + "budget": 0.93, + "threshold": -0.9997641444206238, + "operator": "ge", + "actual_fraction": 0.9299627613864222, + "metric": 0.8987396161558293, + "rescued": 1374, + "harmed": 375, + "missed_rescue": 15, + "avoided_harm": 13, + "selected_both_correct": 4429, + "selected_both_wrong": 315 + }, + { + "budget": 0.94, + "threshold": -0.9997876286506653, + "operator": "ge", + "actual_fraction": 0.939988541965053, + "metric": 0.8990260670295045, + "rescued": 1377, + "harmed": 376, + "missed_rescue": 12, + "avoided_harm": 12, + "selected_both_correct": 4494, + "selected_both_wrong": 316 + }, + { + "budget": 0.95, + "threshold": -0.9998131394386292, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.899169292466342, + "rescued": 1380, + "harmed": 378, + "missed_rescue": 9, + "avoided_harm": 10, + "selected_both_correct": 4557, + "selected_both_wrong": 317 + }, + { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.8997421942136924, + "rescued": 1385, + "harmed": 379, + "missed_rescue": 4, + "avoided_harm": 9, + "selected_both_correct": 4621, + "selected_both_wrong": 317 + }, + { + "budget": 0.97, + "threshold": -0.999868631362915, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.8993125179031796, + "rescued": 1385, + "harmed": 382, + "missed_rescue": 4, + "avoided_harm": 6, + "selected_both_correct": 4688, + "selected_both_wrong": 317 + }, + { + "budget": 0.98, + "threshold": -0.9998925924301147, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.899169292466342, + "rescued": 1385, + "harmed": 383, + "missed_rescue": 4, + "avoided_harm": 5, + "selected_both_correct": 4757, + "selected_both_wrong": 317 + }, + { + "budget": 0.99, + "threshold": -0.99992835521698, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8987396161558293, + "rescued": 1385, + "harmed": 386, + "missed_rescue": 4, + "avoided_harm": 2, + "selected_both_correct": 4824, + "selected_both_wrong": 317 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8990260670295045, + "rescued": 1389, + "harmed": 388, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 4888, + "selected_both_wrong": 317 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7642247255506924, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2706, + "avoided_harm": 711, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5127503275871277, + "operator": "ge", + "actual_fraction": 0.007749156920427639, + "metric": 0.7658032575159647, + "rescued": 33, + "harmed": 11, + "missed_rescue": 2673, + "avoided_harm": 700, + "selected_both_correct": 57, + "selected_both_wrong": 7 + }, + { + "budget": 0.02, + "threshold": -0.5296345353126526, + "operator": "ge", + "actual_fraction": 0.017650857429862953, + "metric": 0.7694625816172778, + "rescued": 96, + "harmed": 23, + "missed_rescue": 2610, + "avoided_harm": 688, + "selected_both_correct": 117, + "selected_both_wrong": 10 + }, + { + "budget": 0.03, + "threshold": -0.5434877276420593, + "operator": "ge", + "actual_fraction": 0.025113008538422903, + "metric": 0.7724043911889216, + "rescued": 142, + "harmed": 28, + "missed_rescue": 2564, + "avoided_harm": 683, + "selected_both_correct": 160, + "selected_both_wrong": 20 + }, + { + "budget": 0.04, + "threshold": -0.56125408411026, + "operator": "ge", + "actual_fraction": 0.0349429575948913, + "metric": 0.7759919638372677, + "rescued": 199, + "harmed": 35, + "missed_rescue": 2507, + "avoided_harm": 676, + "selected_both_correct": 224, + "selected_both_wrong": 29 + }, + { + "budget": 0.05, + "threshold": -0.5779195427894592, + "operator": "ge", + "actual_fraction": 0.043840137762789694, + "metric": 0.7782880103322092, + "rescued": 238, + "harmed": 42, + "missed_rescue": 2468, + "avoided_harm": 669, + "selected_both_correct": 291, + "selected_both_wrong": 40 + }, + { + "budget": 0.06, + "threshold": -0.5911363363265991, + "operator": "ge", + "actual_fraction": 0.05187630049508503, + "metric": 0.7806558082801177, + "rescued": 281, + "harmed": 52, + "missed_rescue": 2425, + "avoided_harm": 659, + "selected_both_correct": 346, + "selected_both_wrong": 44 + }, + { + "budget": 0.07, + "threshold": -0.6087464094161987, + "operator": "ge", + "actual_fraction": 0.06177800100452034, + "metric": 0.7828801033220922, + "rescued": 326, + "harmed": 66, + "missed_rescue": 2380, + "avoided_harm": 645, + "selected_both_correct": 409, + "selected_both_wrong": 60 + }, + { + "budget": 0.08, + "threshold": -0.6274068355560303, + "operator": "ge", + "actual_fraction": 0.07132094424912104, + "metric": 0.7859654157996699, + "rescued": 376, + "harmed": 73, + "missed_rescue": 2330, + "avoided_harm": 638, + "selected_both_correct": 477, + "selected_both_wrong": 68 + }, + { + "budget": 0.09, + "threshold": -0.6465112566947937, + "operator": "ge", + "actual_fraction": 0.08251417091196096, + "metric": 0.788691971012413, + "rescued": 423, + "harmed": 82, + "missed_rescue": 2283, + "avoided_harm": 629, + "selected_both_correct": 560, + "selected_both_wrong": 85 + }, + { + "budget": 0.1, + "threshold": -0.6620025038719177, + "operator": "ge", + "actual_fraction": 0.09090909090909091, + "metric": 0.7908445146014207, + "rescued": 462, + "harmed": 91, + "missed_rescue": 2244, + "avoided_harm": 620, + "selected_both_correct": 619, + "selected_both_wrong": 95 + }, + { + "budget": 0.11, + "threshold": -0.6800539493560791, + "operator": "ge", + "actual_fraction": 0.10030853124775777, + "metric": 0.7934275669082299, + "rescued": 508, + "harmed": 101, + "missed_rescue": 2198, + "avoided_harm": 610, + "selected_both_correct": 682, + "selected_both_wrong": 107 + }, + { + "budget": 0.12, + "threshold": -0.6957284212112427, + "operator": "ge", + "actual_fraction": 0.11078424338092846, + "metric": 0.7968716366506422, + "rescued": 564, + "harmed": 109, + "missed_rescue": 2142, + "avoided_harm": 602, + "selected_both_correct": 747, + "selected_both_wrong": 124 + }, + { + "budget": 0.13, + "threshold": -0.7128273248672485, + "operator": "ge", + "actual_fraction": 0.12075769534333071, + "metric": 0.7998851976752529, + "rescued": 613, + "harmed": 116, + "missed_rescue": 2093, + "avoided_harm": 595, + "selected_both_correct": 821, + "selected_both_wrong": 133 + }, + { + "budget": 0.14, + "threshold": -0.7332167625427246, + "operator": "ge", + "actual_fraction": 0.13431871995407907, + "metric": 0.8045490421181029, + "rescued": 690, + "harmed": 128, + "missed_rescue": 2016, + "avoided_harm": 583, + "selected_both_correct": 902, + "selected_both_wrong": 152 + }, + { + "budget": 0.15, + "threshold": -0.7503995895385742, + "operator": "ge", + "actual_fraction": 0.144579177728349, + "metric": 0.8069168400660114, + "rescued": 738, + "harmed": 143, + "missed_rescue": 1968, + "avoided_harm": 568, + "selected_both_correct": 974, + "selected_both_wrong": 160 + }, + { + "budget": 0.16, + "threshold": -0.7655540704727173, + "operator": "ge", + "actual_fraction": 0.1534763578962474, + "metric": 0.8095716438257875, + "rescued": 788, + "harmed": 156, + "missed_rescue": 1918, + "avoided_harm": 555, + "selected_both_correct": 1026, + "selected_both_wrong": 169 + }, + { + "budget": 0.17, + "threshold": -0.7812193036079407, + "operator": "ge", + "actual_fraction": 0.16280404678194735, + "metric": 0.8120111932266628, + "rescued": 836, + "harmed": 170, + "missed_rescue": 1870, + "avoided_harm": 541, + "selected_both_correct": 1083, + "selected_both_wrong": 180 + }, + { + "budget": 0.18, + "threshold": -0.7954917550086975, + "operator": "ge", + "actual_fraction": 0.17306450455621727, + "metric": 0.8153117600631413, + "rescued": 891, + "harmed": 179, + "missed_rescue": 1815, + "avoided_harm": 532, + "selected_both_correct": 1151, + "selected_both_wrong": 191 + }, + { + "budget": 0.19, + "threshold": -0.8097342252731323, + "operator": "ge", + "actual_fraction": 0.18196168472411567, + "metric": 0.8172490492932482, + "rescued": 928, + "harmed": 189, + "missed_rescue": 1778, + "avoided_harm": 522, + "selected_both_correct": 1219, + "selected_both_wrong": 200 + }, + { + "budget": 0.2, + "threshold": -0.8232117891311646, + "operator": "ge", + "actual_fraction": 0.1928679055750879, + "metric": 0.8200473559589582, + "rescued": 976, + "harmed": 198, + "missed_rescue": 1730, + "avoided_harm": 513, + "selected_both_correct": 1308, + "selected_both_wrong": 206 + }, + { + "budget": 0.21, + "threshold": -0.8359466791152954, + "operator": "ge", + "actual_fraction": 0.20104757121331707, + "metric": 0.8223434024538997, + "rescued": 1015, + "harmed": 205, + "missed_rescue": 1691, + "avoided_harm": 506, + "selected_both_correct": 1369, + "selected_both_wrong": 213 + }, + { + "budget": 0.22, + "threshold": -0.8473842144012451, + "operator": "ge", + "actual_fraction": 0.21051876300495084, + "metric": 0.8249264547607089, + "rescued": 1060, + "harmed": 214, + "missed_rescue": 1646, + "avoided_harm": 497, + "selected_both_correct": 1434, + "selected_both_wrong": 226 + }, + { + "budget": 0.23, + "threshold": -0.8595863580703735, + "operator": "ge", + "actual_fraction": 0.21998995479658462, + "metric": 0.8267919925378489, + "rescued": 1097, + "harmed": 225, + "missed_rescue": 1609, + "avoided_harm": 486, + "selected_both_correct": 1510, + "selected_both_wrong": 234 + }, + { + "budget": 0.24, + "threshold": -0.8704345226287842, + "operator": "ge", + "actual_fraction": 0.22838487479371458, + "metric": 0.8283705245031212, + "rescued": 1128, + "harmed": 234, + "missed_rescue": 1578, + "avoided_harm": 477, + "selected_both_correct": 1582, + "selected_both_wrong": 239 + }, + { + "budget": 0.25, + "threshold": -0.8811548352241516, + "operator": "ge", + "actual_fraction": 0.23900408983281912, + "metric": 0.8309535768099304, + "rescued": 1174, + "harmed": 244, + "missed_rescue": 1532, + "avoided_harm": 467, + "selected_both_correct": 1665, + "selected_both_wrong": 248 + }, + { + "budget": 0.26, + "threshold": -0.8929013609886169, + "operator": "ge", + "actual_fraction": 0.2506995766664275, + "metric": 0.8342541436464088, + "rescued": 1227, + "harmed": 251, + "missed_rescue": 1479, + "avoided_harm": 460, + "selected_both_correct": 1757, + "selected_both_wrong": 259 + }, + { + "budget": 0.27, + "threshold": -0.903549313545227, + "operator": "ge", + "actual_fraction": 0.2646911099949774, + "metric": 0.8369806988591519, + "rescued": 1280, + "harmed": 266, + "missed_rescue": 1426, + "avoided_harm": 445, + "selected_both_correct": 1872, + "selected_both_wrong": 271 + }, + { + "budget": 0.28, + "threshold": -0.9115740656852722, + "operator": "ge", + "actual_fraction": 0.27437755614551196, + "metric": 0.8399225084307957, + "rescued": 1326, + "harmed": 271, + "missed_rescue": 1380, + "avoided_harm": 440, + "selected_both_correct": 1952, + "selected_both_wrong": 275 + }, + { + "budget": 0.29, + "threshold": -0.9182595014572144, + "operator": "ge", + "actual_fraction": 0.2819832101600057, + "metric": 0.8419315491138696, + "rescued": 1362, + "harmed": 279, + "missed_rescue": 1344, + "avoided_harm": 432, + "selected_both_correct": 2007, + "selected_both_wrong": 282 + }, + { + "budget": 0.3, + "threshold": -0.9258768558502197, + "operator": "ge", + "actual_fraction": 0.2907368874219703, + "metric": 0.8444428499677118, + "rescued": 1406, + "harmed": 288, + "missed_rescue": 1300, + "avoided_harm": 423, + "selected_both_correct": 2069, + "selected_both_wrong": 289 + }, + { + "budget": 0.31, + "threshold": -0.9331877827644348, + "operator": "ge", + "actual_fraction": 0.302145368443711, + "metric": 0.8466671450096864, + "rescued": 1448, + "harmed": 299, + "missed_rescue": 1258, + "avoided_harm": 412, + "selected_both_correct": 2169, + "selected_both_wrong": 295 + }, + { + "budget": 0.32, + "threshold": -0.9399217367172241, + "operator": "ge", + "actual_fraction": 0.31412786108918705, + "metric": 0.8487479371457272, + "rescued": 1492, + "harmed": 314, + "missed_rescue": 1214, + "avoided_harm": 397, + "selected_both_correct": 2272, + "selected_both_wrong": 300 + }, + { + "budget": 0.33, + "threshold": -0.9453923106193542, + "operator": "ge", + "actual_fraction": 0.3238143072397216, + "metric": 0.8506852263758341, + "rescued": 1527, + "harmed": 322, + "missed_rescue": 1179, + "avoided_harm": 389, + "selected_both_correct": 2358, + "selected_both_wrong": 306 + }, + { + "budget": 0.34, + "threshold": -0.9515207409858704, + "operator": "ge", + "actual_fraction": 0.33529453971442924, + "metric": 0.8536987874004449, + "rescued": 1578, + "harmed": 331, + "missed_rescue": 1128, + "avoided_harm": 380, + "selected_both_correct": 2453, + "selected_both_wrong": 311 + }, + { + "budget": 0.35, + "threshold": -0.9567410349845886, + "operator": "ge", + "actual_fraction": 0.3452679916768315, + "metric": 0.8557078280835186, + "rescued": 1615, + "harmed": 340, + "missed_rescue": 1091, + "avoided_harm": 371, + "selected_both_correct": 2540, + "selected_both_wrong": 317 + }, + { + "budget": 0.36, + "threshold": -0.9606627225875854, + "operator": "ge", + "actual_fraction": 0.35595895816890294, + "metric": 0.8581473774843941, + "rescued": 1655, + "harmed": 346, + "missed_rescue": 1051, + "avoided_harm": 365, + "selected_both_correct": 2634, + "selected_both_wrong": 326 + }, + { + "budget": 0.37, + "threshold": -0.9650874733924866, + "operator": "ge", + "actual_fraction": 0.36837195953218055, + "metric": 0.8605151754323025, + "rescued": 1702, + "harmed": 360, + "missed_rescue": 1004, + "avoided_harm": 351, + "selected_both_correct": 2738, + "selected_both_wrong": 334 + }, + { + "budget": 0.38, + "threshold": -0.9686882495880127, + "operator": "ge", + "actual_fraction": 0.3797086891009543, + "metric": 0.8622372103035086, + "rescued": 1741, + "harmed": 375, + "missed_rescue": 965, + "avoided_harm": 336, + "selected_both_correct": 2832, + "selected_both_wrong": 344 + }, + { + "budget": 0.39, + "threshold": -0.971785306930542, + "operator": "ge", + "actual_fraction": 0.391117170122695, + "metric": 0.8652507713281193, + "rescued": 1786, + "harmed": 378, + "missed_rescue": 920, + "avoided_harm": 333, + "selected_both_correct": 2936, + "selected_both_wrong": 351 + }, + { + "budget": 0.4, + "threshold": -0.9748144745826721, + "operator": "ge", + "actual_fraction": 0.40231039678553493, + "metric": 0.8673315634641602, + "rescued": 1820, + "harmed": 383, + "missed_rescue": 886, + "avoided_harm": 328, + "selected_both_correct": 3042, + "selected_both_wrong": 362 + }, + { + "budget": 0.41, + "threshold": -0.977255642414093, + "operator": "ge", + "actual_fraction": 0.41235560020090406, + "metric": 0.8697711128650355, + "rescued": 1863, + "harmed": 392, + "missed_rescue": 843, + "avoided_harm": 319, + "selected_both_correct": 3125, + "selected_both_wrong": 367 + }, + { + "budget": 0.42, + "threshold": -0.9794421792030334, + "operator": "ge", + "actual_fraction": 0.4231183181459425, + "metric": 0.8717801535481093, + "rescued": 1903, + "harmed": 404, + "missed_rescue": 803, + "avoided_harm": 307, + "selected_both_correct": 3218, + "selected_both_wrong": 372 + }, + { + "budget": 0.43, + "threshold": -0.981472373008728, + "operator": "ge", + "actual_fraction": 0.43172849250197315, + "metric": 0.8732151826074478, + "rescued": 1925, + "harmed": 406, + "missed_rescue": 781, + "avoided_harm": 305, + "selected_both_correct": 3309, + "selected_both_wrong": 377 + }, + { + "budget": 0.44, + "threshold": -0.9835578799247742, + "operator": "ge", + "actual_fraction": 0.4432804764296477, + "metric": 0.8749372174786539, + "rescued": 1956, + "harmed": 413, + "missed_rescue": 750, + "avoided_harm": 298, + "selected_both_correct": 3425, + "selected_both_wrong": 384 + }, + { + "budget": 0.45, + "threshold": -0.9851564168930054, + "operator": "ge", + "actual_fraction": 0.4547607089043553, + "metric": 0.8768027552557939, + "rescued": 1994, + "harmed": 425, + "missed_rescue": 712, + "avoided_harm": 286, + "selected_both_correct": 3527, + "selected_both_wrong": 392 + }, + { + "budget": 0.46, + "threshold": -0.986689031124115, + "operator": "ge", + "actual_fraction": 0.4649494152256583, + "metric": 0.8787400444859008, + "rescued": 2026, + "harmed": 430, + "missed_rescue": 680, + "avoided_harm": 281, + "selected_both_correct": 3626, + "selected_both_wrong": 398 + }, + { + "budget": 0.47, + "threshold": -0.9879001379013062, + "operator": "ge", + "actual_fraction": 0.47399009829949057, + "metric": 0.8798880677333716, + "rescued": 2047, + "harmed": 435, + "missed_rescue": 659, + "avoided_harm": 276, + "selected_both_correct": 3721, + "selected_both_wrong": 403 + }, + { + "budget": 0.48, + "threshold": -0.9891784191131592, + "operator": "ge", + "actual_fraction": 0.48446581043266124, + "metric": 0.8814665996986439, + "rescued": 2076, + "harmed": 442, + "missed_rescue": 630, + "avoided_harm": 269, + "selected_both_correct": 3824, + "selected_both_wrong": 410 + }, + { + "budget": 0.49, + "threshold": -0.990114688873291, + "operator": "ge", + "actual_fraction": 0.491210447011552, + "metric": 0.8825428714931477, + "rescued": 2095, + "harmed": 446, + "missed_rescue": 611, + "avoided_harm": 265, + "selected_both_correct": 3892, + "selected_both_wrong": 413 + }, + { + "budget": 0.5, + "threshold": -0.9910277724266052, + "operator": "ge", + "actual_fraction": 0.5008251417091196, + "metric": 0.8833321374757839, + "rescued": 2116, + "harmed": 456, + "missed_rescue": 590, + "avoided_harm": 255, + "selected_both_correct": 3984, + "selected_both_wrong": 424 + }, + { + "budget": 0.51, + "threshold": -0.9918718934059143, + "operator": "ge", + "actual_fraction": 0.5107268422185549, + "metric": 0.8839061490995193, + "rescued": 2133, + "harmed": 465, + "missed_rescue": 573, + "avoided_harm": 246, + "selected_both_correct": 4093, + "selected_both_wrong": 427 + }, + { + "budget": 0.52, + "threshold": -0.9926809668540955, + "operator": "ge", + "actual_fraction": 0.5217048145224941, + "metric": 0.88505417234699, + "rescued": 2158, + "harmed": 474, + "missed_rescue": 548, + "avoided_harm": 237, + "selected_both_correct": 4209, + "selected_both_wrong": 430 + }, + { + "budget": 0.53, + "threshold": -0.9932646751403809, + "operator": "ge", + "actual_fraction": 0.5306737461433594, + "metric": 0.8867044557652293, + "rescued": 2184, + "harmed": 477, + "missed_rescue": 522, + "avoided_harm": 234, + "selected_both_correct": 4302, + "selected_both_wrong": 433 + }, + { + "budget": 0.54, + "threshold": -0.9938417673110962, + "operator": "ge", + "actual_fraction": 0.5398579321231255, + "metric": 0.8876372246537992, + "rescued": 2203, + "harmed": 483, + "missed_rescue": 503, + "avoided_harm": 228, + "selected_both_correct": 4401, + "selected_both_wrong": 437 + }, + { + "budget": 0.55, + "threshold": -0.9944412112236023, + "operator": "ge", + "actual_fraction": 0.5504771471622301, + "metric": 0.8882112362775346, + "rescued": 2220, + "harmed": 492, + "missed_rescue": 486, + "avoided_harm": 219, + "selected_both_correct": 4514, + "selected_both_wrong": 446 + }, + { + "budget": 0.56, + "threshold": -0.9949513673782349, + "operator": "ge", + "actual_fraction": 0.5604505991246322, + "metric": 0.8893592595250054, + "rescued": 2242, + "harmed": 498, + "missed_rescue": 464, + "avoided_harm": 213, + "selected_both_correct": 4617, + "selected_both_wrong": 454 + }, + { + "budget": 0.57, + "threshold": -0.9954326152801514, + "operator": "ge", + "actual_fraction": 0.570926311257803, + "metric": 0.8905072827724761, + "rescued": 2262, + "harmed": 502, + "missed_rescue": 444, + "avoided_harm": 209, + "selected_both_correct": 4734, + "selected_both_wrong": 459 + }, + { + "budget": 0.58, + "threshold": -0.9958608746528625, + "operator": "ge", + "actual_fraction": 0.5816172777498745, + "metric": 0.8916553060199469, + "rescued": 2287, + "harmed": 511, + "missed_rescue": 419, + "avoided_harm": 200, + "selected_both_correct": 4841, + "selected_both_wrong": 467 + }, + { + "budget": 0.59, + "threshold": -0.9962128400802612, + "operator": "ge", + "actual_fraction": 0.5920929898830452, + "metric": 0.8924445720025831, + "rescued": 2308, + "harmed": 521, + "missed_rescue": 398, + "avoided_harm": 190, + "selected_both_correct": 4951, + "selected_both_wrong": 472 + }, + { + "budget": 0.6, + "threshold": -0.9965764880180359, + "operator": "ge", + "actual_fraction": 0.6034297194518189, + "metric": 0.8928750807203846, + "rescued": 2326, + "harmed": 533, + "missed_rescue": 380, + "avoided_harm": 178, + "selected_both_correct": 5075, + "selected_both_wrong": 476 + }, + { + "budget": 0.61, + "threshold": -0.9968799352645874, + "operator": "ge", + "actual_fraction": 0.6144794432087249, + "metric": 0.8936643467030208, + "rescued": 2346, + "harmed": 542, + "missed_rescue": 360, + "avoided_harm": 169, + "selected_both_correct": 5197, + "selected_both_wrong": 479 + }, + { + "budget": 0.62, + "threshold": -0.9971383810043335, + "operator": "ge", + "actual_fraction": 0.6232331204706896, + "metric": 0.8946688670445576, + "rescued": 2363, + "harmed": 545, + "missed_rescue": 343, + "avoided_harm": 166, + "selected_both_correct": 5293, + "selected_both_wrong": 485 + }, + { + "budget": 0.63, + "threshold": -0.997377872467041, + "operator": "ge", + "actual_fraction": 0.6337805840568271, + "metric": 0.8953863815742269, + "rescued": 2381, + "harmed": 553, + "missed_rescue": 325, + "avoided_harm": 158, + "selected_both_correct": 5411, + "selected_both_wrong": 488 + }, + { + "budget": 0.64, + "threshold": -0.9976021647453308, + "operator": "ge", + "actual_fraction": 0.6445433020018655, + "metric": 0.8958886417449954, + "rescued": 2398, + "harmed": 563, + "missed_rescue": 308, + "avoided_harm": 148, + "selected_both_correct": 5525, + "selected_both_wrong": 497 + }, + { + "budget": 0.65, + "threshold": -0.9978214502334595, + "operator": "ge", + "actual_fraction": 0.6565257946473416, + "metric": 0.8963909019157638, + "rescued": 2411, + "harmed": 569, + "missed_rescue": 295, + "avoided_harm": 142, + "selected_both_correct": 5668, + "selected_both_wrong": 502 + }, + { + "budget": 0.66, + "threshold": -0.9980514049530029, + "operator": "ge", + "actual_fraction": 0.6692258018224869, + "metric": 0.8972519193513668, + "rescued": 2430, + "harmed": 576, + "missed_rescue": 276, + "avoided_harm": 135, + "selected_both_correct": 5815, + "selected_both_wrong": 506 + }, + { + "budget": 0.67, + "threshold": -0.9982016086578369, + "operator": "ge", + "actual_fraction": 0.6777642247255506, + "metric": 0.8978259309751022, + "rescued": 2443, + "harmed": 581, + "missed_rescue": 263, + "avoided_harm": 130, + "selected_both_correct": 5912, + "selected_both_wrong": 510 + }, + { + "budget": 0.68, + "threshold": -0.9983693957328796, + "operator": "ge", + "actual_fraction": 0.6885269426705891, + "metric": 0.8986869484107053, + "rescued": 2462, + "harmed": 588, + "missed_rescue": 244, + "avoided_harm": 123, + "selected_both_correct": 6035, + "selected_both_wrong": 511 + }, + { + "budget": 0.69, + "threshold": -0.9984915256500244, + "operator": "ge", + "actual_fraction": 0.6977828801033221, + "metric": 0.898973954222573, + "rescued": 2473, + "harmed": 595, + "missed_rescue": 233, + "avoided_harm": 116, + "selected_both_correct": 6142, + "selected_both_wrong": 515 + }, + { + "budget": 0.7, + "threshold": -0.9986109733581543, + "operator": "gt", + "actual_fraction": 0.7064648059123197, + "metric": 0.8988304513166392, + "rescued": 2481, + "harmed": 605, + "missed_rescue": 225, + "avoided_harm": 106, + "selected_both_correct": 6242, + "selected_both_wrong": 518 + }, + { + "budget": 0.71, + "threshold": -0.9986966252326965, + "operator": "ge", + "actual_fraction": 0.7137117026619789, + "metric": 0.8996197172992753, + "rescued": 2497, + "harmed": 610, + "missed_rescue": 209, + "avoided_harm": 101, + "selected_both_correct": 6320, + "selected_both_wrong": 520 + }, + { + "budget": 0.72, + "threshold": -0.9987963438034058, + "operator": "ge", + "actual_fraction": 0.7256941953074549, + "metric": 0.9004089832819114, + "rescued": 2512, + "harmed": 614, + "missed_rescue": 194, + "avoided_harm": 97, + "selected_both_correct": 6462, + "selected_both_wrong": 526 + }, + { + "budget": 0.73, + "threshold": -0.998875081539154, + "operator": "ge", + "actual_fraction": 0.7345196240223865, + "metric": 0.9009829949056468, + "rescued": 2521, + "harmed": 615, + "missed_rescue": 185, + "avoided_harm": 96, + "selected_both_correct": 6569, + "selected_both_wrong": 532 + }, + { + "budget": 0.74, + "threshold": -0.9989551305770874, + "operator": "ge", + "actual_fraction": 0.745067087608524, + "metric": 0.901772260888283, + "rescued": 2535, + "harmed": 618, + "missed_rescue": 171, + "avoided_harm": 93, + "selected_both_correct": 6698, + "selected_both_wrong": 533 + }, + { + "budget": 0.75, + "threshold": -0.9990480542182922, + "operator": "ge", + "actual_fraction": 0.7568343258950994, + "metric": 0.9023462725120184, + "rescued": 2548, + "harmed": 623, + "missed_rescue": 158, + "avoided_harm": 88, + "selected_both_correct": 6839, + "selected_both_wrong": 538 + }, + { + "budget": 0.76, + "threshold": -0.999126136302948, + "operator": "ge", + "actual_fraction": 0.7663772691397001, + "metric": 0.9028485326827869, + "rescued": 2561, + "harmed": 629, + "missed_rescue": 145, + "avoided_harm": 82, + "selected_both_correct": 6951, + "selected_both_wrong": 540 + }, + { + "budget": 0.77, + "threshold": -0.999184787273407, + "operator": "gt", + "actual_fraction": 0.7746286862308962, + "metric": 0.9030637870416875, + "rescued": 2567, + "harmed": 632, + "missed_rescue": 139, + "avoided_harm": 79, + "selected_both_correct": 7054, + "selected_both_wrong": 543 + }, + { + "budget": 0.78, + "threshold": -0.9992268085479736, + "operator": "ge", + "actual_fraction": 0.7817320800746215, + "metric": 0.9028485326827869, + "rescued": 2570, + "harmed": 638, + "missed_rescue": 136, + "avoided_harm": 73, + "selected_both_correct": 7143, + "selected_both_wrong": 544 + }, + { + "budget": 0.79, + "threshold": -0.9992831349372864, + "operator": "ge", + "actual_fraction": 0.7928535552844945, + "metric": 0.9028485326827869, + "rescued": 2579, + "harmed": 647, + "missed_rescue": 127, + "avoided_harm": 64, + "selected_both_correct": 7279, + "selected_both_wrong": 545 + }, + { + "budget": 0.8, + "threshold": -0.9993318915367126, + "operator": "ge", + "actual_fraction": 0.8023247470761283, + "metric": 0.9028485326827869, + "rescued": 2581, + "harmed": 649, + "missed_rescue": 125, + "avoided_harm": 62, + "selected_both_correct": 7406, + "selected_both_wrong": 546 + }, + { + "budget": 0.81, + "threshold": -0.9993773102760315, + "operator": "ge", + "actual_fraction": 0.811795938867762, + "metric": 0.9027767812298199, + "rescued": 2588, + "harmed": 657, + "missed_rescue": 118, + "avoided_harm": 54, + "selected_both_correct": 7521, + "selected_both_wrong": 548 + }, + { + "budget": 0.82, + "threshold": -0.9994175434112549, + "operator": "ge", + "actual_fraction": 0.8214106335653297, + "metric": 0.9034225443065222, + "rescued": 2601, + "harmed": 661, + "missed_rescue": 105, + "avoided_harm": 50, + "selected_both_correct": 7637, + "selected_both_wrong": 549 + }, + { + "budget": 0.83, + "threshold": -0.999452531337738, + "operator": "ge", + "actual_fraction": 0.8298773050154266, + "metric": 0.9037095501183899, + "rescued": 2607, + "harmed": 663, + "missed_rescue": 99, + "avoided_harm": 48, + "selected_both_correct": 7745, + "selected_both_wrong": 551 + }, + { + "budget": 0.84, + "threshold": -0.9994945526123047, + "operator": "ge", + "actual_fraction": 0.8420750520198034, + "metric": 0.9037095501183899, + "rescued": 2612, + "harmed": 668, + "missed_rescue": 94, + "avoided_harm": 43, + "selected_both_correct": 7903, + "selected_both_wrong": 553 + }, + { + "budget": 0.85, + "threshold": -0.9995274543762207, + "operator": "gt", + "actual_fraction": 0.8502547176580326, + "metric": 0.9039248044772906, + "rescued": 2616, + "harmed": 669, + "missed_rescue": 90, + "avoided_harm": 42, + "selected_both_correct": 8008, + "selected_both_wrong": 557 + }, + { + "budget": 0.86, + "threshold": -0.9995617270469666, + "operator": "ge", + "actual_fraction": 0.8604434239793356, + "metric": 0.9042835617421253, + "rescued": 2623, + "harmed": 671, + "missed_rescue": 83, + "avoided_harm": 40, + "selected_both_correct": 8140, + "selected_both_wrong": 558 + }, + { + "budget": 0.87, + "threshold": -0.9995949864387512, + "operator": "ge", + "actual_fraction": 0.8690535983353663, + "metric": 0.904570567553993, + "rescued": 2629, + "harmed": 673, + "missed_rescue": 77, + "avoided_harm": 38, + "selected_both_correct": 8251, + "selected_both_wrong": 559 + }, + { + "budget": 0.88, + "threshold": -0.9996248483657837, + "operator": "ge", + "actual_fraction": 0.8778790270502977, + "metric": 0.904498816101026, + "rescued": 2634, + "harmed": 679, + "missed_rescue": 72, + "avoided_harm": 32, + "selected_both_correct": 8361, + "selected_both_wrong": 561 + }, + { + "budget": 0.89, + "threshold": -0.9996511936187744, + "operator": "ge", + "actual_fraction": 0.886058692688527, + "metric": 0.9050010762717945, + "rescued": 2642, + "harmed": 680, + "missed_rescue": 64, + "avoided_harm": 31, + "selected_both_correct": 8464, + "selected_both_wrong": 563 + }, + { + "budget": 0.9, + "threshold": -0.9996830224990845, + "operator": "ge", + "actual_fraction": 0.8961756475568631, + "metric": 0.9052163306306953, + "rescued": 2648, + "harmed": 683, + "missed_rescue": 58, + "avoided_harm": 28, + "selected_both_correct": 8590, + "selected_both_wrong": 569 + }, + { + "budget": 0.91, + "threshold": -0.9997097849845886, + "operator": "gt", + "actual_fraction": 0.9053598335366291, + "metric": 0.9055750878955299, + "rescued": 2654, + "harmed": 684, + "missed_rescue": 52, + "avoided_harm": 27, + "selected_both_correct": 8711, + "selected_both_wrong": 569 + }, + { + "budget": 0.92, + "threshold": -0.9997391104698181, + "operator": "ge", + "actual_fraction": 0.9164813087465021, + "metric": 0.9057185908014638, + "rescued": 2659, + "harmed": 687, + "missed_rescue": 47, + "avoided_harm": 24, + "selected_both_correct": 8855, + "selected_both_wrong": 572 + }, + { + "budget": 0.93, + "threshold": -0.9997641444206238, + "operator": "ge", + "actual_fraction": 0.9265265121618713, + "metric": 0.9060773480662984, + "rescued": 2667, + "harmed": 690, + "missed_rescue": 39, + "avoided_harm": 21, + "selected_both_correct": 8984, + "selected_both_wrong": 572 + }, + { + "budget": 0.94, + "threshold": -0.9997876286506653, + "operator": "ge", + "actual_fraction": 0.9362847097653727, + "metric": 0.906364353878166, + "rescued": 2674, + "harmed": 693, + "missed_rescue": 32, + "avoided_harm": 18, + "selected_both_correct": 9109, + "selected_both_wrong": 573 + }, + { + "budget": 0.95, + "threshold": -0.9998131394386292, + "operator": "ge", + "actual_fraction": 0.9475496878811795, + "metric": 0.9065078567840998, + "rescued": 2680, + "harmed": 697, + "missed_rescue": 26, + "avoided_harm": 14, + "selected_both_correct": 9256, + "selected_both_wrong": 573 + }, + { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9593169261677549, + "metric": 0.906436105331133, + "rescued": 2682, + "harmed": 700, + "missed_rescue": 24, + "avoided_harm": 11, + "selected_both_correct": 9413, + "selected_both_wrong": 575 + }, + { + "budget": 0.97, + "threshold": -0.999868631362915, + "operator": "ge", + "actual_fraction": 0.9700796441127932, + "metric": 0.9070101169548683, + "rescued": 2690, + "harmed": 700, + "missed_rescue": 16, + "avoided_harm": 11, + "selected_both_correct": 9554, + "selected_both_wrong": 576 + }, + { + "budget": 0.98, + "threshold": -0.9998925924301147, + "operator": "ge", + "actual_fraction": 0.9793355815455262, + "metric": 0.9071536198608022, + "rescued": 2696, + "harmed": 704, + "missed_rescue": 10, + "avoided_harm": 7, + "selected_both_correct": 9672, + "selected_both_wrong": 577 + }, + { + "budget": 0.99, + "threshold": -0.99992835521698, + "operator": "ge", + "actual_fraction": 0.9909593169261678, + "metric": 0.9071536198608022, + "rescued": 2702, + "harmed": 710, + "missed_rescue": 4, + "avoided_harm": 1, + "selected_both_correct": 9821, + "selected_both_wrong": 578 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9073688742197029, + "rescued": 2706, + "harmed": 711, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9940, + "selected_both_wrong": 580 + } + ] + }, + "learned": { + "validation_area_0_25": 0.7937295903752506, + "validation_point": { + "metric": 0.8975938126611286, + "metric_std": 0.00045291859927933527, + "actual_fraction": 0.9899742194213692, + "actual_fraction_std": 0.0, + "per_seed": [ + { + "seed": 2026, + "budget": 0.99, + "threshold": -0.15563204884529114, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.897450587224291, + "rescue": 1373, + "harm": 383, + "missed_rescue": 16 + }, + { + "seed": 2027, + "budget": 0.99, + "threshold": -0.1142226904630661, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8983099398453165, + "rescue": 1376, + "harm": 380, + "missed_rescue": 13 + }, + { + "seed": 2028, + "budget": 0.99, + "threshold": -0.2333981692790985, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8977370380979662, + "rescue": 1373, + "harm": 381, + "missed_rescue": 16 + }, + { + "seed": 2029, + "budget": 0.99, + "threshold": -0.18295258283615112, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8973073617874534, + "rescue": 1368, + "harm": 379, + "missed_rescue": 21 + }, + { + "seed": 2030, + "budget": 0.99, + "threshold": -0.15542080998420715, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.8971641363506159, + "rescue": 1371, + "harm": 383, + "missed_rescue": 18 + } + ], + "rescued": 1372.2, + "harmed": 381.2, + "missed_rescue": 16.8, + "avoided_harm": 6.800000000000011 + }, + "test_point": { + "metric": 0.9061778001004519, + "metric_std": 0.0005525327777334696, + "actual_fraction": 0.9900839491999711, + "actual_fraction_std": 0.001108554143190012, + "per_seed": [ + { + "seed": 2026, + "budget": 0.99, + "threshold": -0.15563204884529114, + "operator": "ge", + "actual_fraction": 0.9898830451316639, + "metric": 0.9056468393484968, + "rescue": 2669, + "harm": 698, + "missed_rescue": 37 + }, + { + "seed": 2027, + "budget": 0.99, + "threshold": -0.1142226904630661, + "operator": "ge", + "actual_fraction": 0.9882327617134247, + "metric": 0.9057185908014638, + "rescue": 2668, + "harm": 696, + "missed_rescue": 38 + }, + { + "seed": 2028, + "budget": 0.99, + "threshold": -0.2333981692790985, + "operator": "ge", + "actual_fraction": 0.9908875654732008, + "metric": 0.906364353878166, + "rescue": 2677, + "harm": 696, + "missed_rescue": 29 + }, + { + "seed": 2029, + "budget": 0.99, + "threshold": -0.18295258283615112, + "operator": "ge", + "actual_fraction": 0.9906005596613331, + "metric": 0.9061490995192653, + "rescue": 2672, + "harm": 694, + "missed_rescue": 34 + }, + { + "seed": 2030, + "budget": 0.99, + "threshold": -0.15542080998420715, + "operator": "ge", + "actual_fraction": 0.9908158140202339, + "metric": 0.9070101169548683, + "rescue": 2686, + "harm": 696, + "missed_rescue": 20 + } + ], + "rescued": 2674.4, + "harmed": 696.0, + "missed_rescue": 31.6, + "avoided_harm": 15.0 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.7556574047550845, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.7593526210254942, + "metric_std": 0.0006516363887755376, + "actual_fraction": 0.009882555141793183, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.7630478372959038, + "metric_std": 0.0013343815180580903, + "actual_fraction": 0.01990833572042395, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.7664852477800057, + "metric_std": 0.0011167911022410852, + "actual_fraction": 0.029934116299054715, + "actual_fraction_std": 3.878959614448864e-18 + }, + { + "budget": 0.04, + "metric": 0.7693211114293897, + "metric_std": 0.0007619270168014893, + "actual_fraction": 0.039959896877685475, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.7728444571755945, + "metric_std": 0.0005063783881312983, + "actual_fraction": 0.04998567745631624, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.7766542537954739, + "metric_std": 0.00027919777556027314, + "actual_fraction": 0.05986823259810943, + "actual_fraction_std": 7.757919228897728e-18 + }, + { + "budget": 0.07, + "metric": 0.7793468920080207, + "metric_std": 0.0006453096720749505, + "actual_fraction": 0.06989401317674018, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.7820395302205672, + "metric_std": 0.0011158723105575614, + "actual_fraction": 0.07991979375537095, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.7853337152678315, + "metric_std": 0.0007400750892445687, + "actual_fraction": 0.08994557433400172, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.7885133199656258, + "metric_std": 0.001179329066183829, + "actual_fraction": 0.09997135491263248, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.7913491836150099, + "metric_std": 0.0012169948889285507, + "actual_fraction": 0.10999713549126325, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.12, + "metric": 0.794070466914924, + "metric_std": 0.0011066424389620134, + "actual_fraction": 0.11987969063305644, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.7964193640790604, + "metric_std": 0.0008264988854914441, + "actual_fraction": 0.1299054712116872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.799398453165282, + "metric_std": 0.0009798127399878193, + "actual_fraction": 0.13993125179031796, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.802492122600974, + "metric_std": 0.000828977148581359, + "actual_fraction": 0.14995703236894872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.8046977943282727, + "metric_std": 0.0012262305810944705, + "actual_fraction": 0.1599828129475795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.8073331423660843, + "metric_std": 0.0016806881021719843, + "actual_fraction": 0.16986536808937266, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.8093096533944429, + "metric_std": 0.0013111191625718498, + "actual_fraction": 0.17989114866800343, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.812202807218562, + "metric_std": 0.0008794088474940523, + "actual_fraction": 0.1899169292466342, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.8145517043826984, + "metric_std": 0.0009193243503799034, + "actual_fraction": 0.19994270982526496, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.8167573761099971, + "metric_std": 0.0008139943992198661, + "actual_fraction": 0.20996849040389573, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.22, + "metric": 0.819306788885706, + "metric_std": 0.00032972971736528725, + "actual_fraction": 0.2199942709825265, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.23, + "metric": 0.8213405900887997, + "metric_std": 0.0006210106957513035, + "actual_fraction": 0.22987682612431967, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.24, + "metric": 0.8233457462045258, + "metric_std": 0.0006913502369605849, + "actual_fraction": 0.23990260670295044, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.8248352907476368, + "metric_std": 0.000723253003320118, + "actual_fraction": 0.24992838728158123, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.8259524491549699, + "metric_std": 0.0012042866232929315, + "actual_fraction": 0.25995416786021197, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.8271555428244055, + "metric_std": 0.0018767451538584598, + "actual_fraction": 0.26997994843884277, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.827671154397021, + "metric_std": 0.0019933564506109075, + "actual_fraction": 0.2798625035806359, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.8284732168433114, + "metric_std": 0.00272052937738956, + "actual_fraction": 0.2898882841592667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.8290174735032941, + "metric_std": 0.0027908754415152696, + "actual_fraction": 0.29991406473789745, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.8292752792896019, + "metric_std": 0.002698574001921617, + "actual_fraction": 0.30993984531652824, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.8297622457748496, + "metric_std": 0.0024653919935565738, + "actual_fraction": 0.319965625895159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.8305070180464049, + "metric_std": 0.002484042783005282, + "actual_fraction": 0.3299914064737898, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.8308221140074477, + "metric_std": 0.0024335683442503267, + "actual_fraction": 0.3398739616155829, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.8314236608421656, + "metric_std": 0.002322726258984097, + "actual_fraction": 0.3498997421942137, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.8321111429389859, + "metric_std": 0.0021445588065591194, + "actual_fraction": 0.35992552277284445, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.8327413348610714, + "metric_std": 0.001710929608595596, + "actual_fraction": 0.36995130335147525, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.8333142366084216, + "metric_std": 0.0015339170135193795, + "actual_fraction": 0.379977083930106, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.8337725580063019, + "metric_std": 0.0015312400184364571, + "actual_fraction": 0.3898596390718992, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.8342881695789173, + "metric_std": 0.0015325791004749604, + "actual_fraction": 0.3998854196505299, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.8350042967631051, + "metric_std": 0.001803162175882852, + "actual_fraction": 0.40991120022916067, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.42, + "metric": 0.8354053279862503, + "metric_std": 0.0019984953002069854, + "actual_fraction": 0.41993698080779146, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.43, + "metric": 0.8360068748209682, + "metric_std": 0.002541999333901291, + "actual_fraction": 0.42996276138642225, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.44, + "metric": 0.836608421655686, + "metric_std": 0.0022880235517291113, + "actual_fraction": 0.439988541965053, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.45, + "metric": 0.8371240332283014, + "metric_std": 0.002242747537217205, + "actual_fraction": 0.4498710971068462, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.8376682898882841, + "metric_std": 0.0018745578044510123, + "actual_fraction": 0.45989687768547693, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.47, + "metric": 0.8386995130335148, + "metric_std": 0.0015832728278929178, + "actual_fraction": 0.46992265826410773, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.48, + "metric": 0.8395015754798052, + "metric_std": 0.0016442878990049984, + "actual_fraction": 0.4799484388427385, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.49, + "metric": 0.8401317674018906, + "metric_std": 0.0015339170135193916, + "actual_fraction": 0.4899742194213693, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.8411343454597537, + "metric_std": 0.0019199689004225223, + "actual_fraction": 0.5, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.51, + "metric": 0.841764537381839, + "metric_std": 0.0018657827636186946, + "actual_fraction": 0.5098825551417931, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.8426238900028645, + "metric_std": 0.0018903587455624672, + "actual_fraction": 0.5199083357204239, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.8434545975365225, + "metric_std": 0.0019428043492195173, + "actual_fraction": 0.5299341162990547, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.8443998854196504, + "metric_std": 0.0019391052759755254, + "actual_fraction": 0.5399598968776855, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.8449154969922658, + "metric_std": 0.0018492173327504342, + "actual_fraction": 0.5499856774563162, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.8456029790890861, + "metric_std": 0.002092758406789272, + "actual_fraction": 0.5598682325981095, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.8467774276711545, + "metric_std": 0.0020341037695384705, + "actual_fraction": 0.5698940131767402, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.8477513606416499, + "metric_std": 0.001969024217254005, + "actual_fraction": 0.579919793755371, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.8490690346605557, + "metric_std": 0.0018630320872539644, + "actual_fraction": 0.5899455743340017, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.8501861930678889, + "metric_std": 0.0019215708772556078, + "actual_fraction": 0.5999713549126325, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.8512747063878544, + "metric_std": 0.002142644875962272, + "actual_fraction": 0.6099971354912632, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.8523345746204527, + "metric_std": 0.002262327269970114, + "actual_fraction": 0.6198796906330565, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.8532225723288456, + "metric_std": 0.0022358770599158603, + "actual_fraction": 0.6299054712116872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.8543397307361786, + "metric_std": 0.00235125396152707, + "actual_fraction": 0.639931251790318, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.8556574047550847, + "metric_std": 0.0024102760203910233, + "actual_fraction": 0.6499570323689488, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.8568032082497853, + "metric_std": 0.002737067702226561, + "actual_fraction": 0.6599828129475795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.8576339157834431, + "metric_std": 0.0025919451548057698, + "actual_fraction": 0.6698653680893727, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.8586651389286738, + "metric_std": 0.0025367485153381165, + "actual_fraction": 0.6798911486680035, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.8599541678602121, + "metric_std": 0.0023996138606197886, + "actual_fraction": 0.6899169292466342, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.8610999713549127, + "metric_std": 0.0025266201170094484, + "actual_fraction": 0.699942709825265, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.8620739043254082, + "metric_std": 0.002221149014504528, + "actual_fraction": 0.7099684904038958, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.8631337725580064, + "metric_std": 0.0023828856308318975, + "actual_fraction": 0.7199942709825266, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.864451446576912, + "metric_std": 0.0021185748595932935, + "actual_fraction": 0.7298768261243197, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.8653394442853051, + "metric_std": 0.0021612334013106785, + "actual_fraction": 0.7399026067029505, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.8664279576052707, + "metric_std": 0.002101561822918917, + "actual_fraction": 0.7499283872815812, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.8676883414494414, + "metric_std": 0.0017582344359122577, + "actual_fraction": 0.759954167860212, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.8685190489830994, + "metric_std": 0.0016916373261268711, + "actual_fraction": 0.7699799484388428, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.8695502721283299, + "metric_std": 0.0018575185146804604, + "actual_fraction": 0.7798625035806359, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.8708106559725006, + "metric_std": 0.0016108882668579168, + "actual_fraction": 0.7898882841592667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.8720710398166714, + "metric_std": 0.001195739413620256, + "actual_fraction": 0.7999140647378975, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.873761099971355, + "metric_std": 0.0013908362516539604, + "actual_fraction": 0.8099398453165282, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.8750787739902606, + "metric_std": 0.0016025895996218192, + "actual_fraction": 0.819965625895159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.8763391578344315, + "metric_std": 0.0019990084585927046, + "actual_fraction": 0.8299914064737898, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.84, + "metric": 0.877542251503867, + "metric_std": 0.0023359361830850734, + "actual_fraction": 0.8398739616155829, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.85, + "metric": 0.8789458607848755, + "metric_std": 0.0020366234085194396, + "actual_fraction": 0.8498997421942137, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.8799770839301061, + "metric_std": 0.002211894186691592, + "actual_fraction": 0.8599255227728445, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.87, + "metric": 0.8809796619879691, + "metric_std": 0.0018811122223665988, + "actual_fraction": 0.8699513033514752, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.88, + "metric": 0.8819822400458321, + "metric_std": 0.0016424154885677012, + "actual_fraction": 0.879977083930106, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.89, + "metric": 0.8828988828415927, + "metric_std": 0.0014308213971632167, + "actual_fraction": 0.8898596390718991, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.8843311372099685, + "metric_std": 0.0015278871788134956, + "actual_fraction": 0.8998854196505299, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.8861071326267546, + "metric_std": 0.001890358745562487, + "actual_fraction": 0.9099112002291607, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.92, + "metric": 0.8873388713835577, + "metric_std": 0.0013382192640272702, + "actual_fraction": 0.9199369808077915, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.93, + "metric": 0.888857061014036, + "metric_std": 0.0014641254869851709, + "actual_fraction": 0.9299627613864223, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.94, + "metric": 0.8900601546834718, + "metric_std": 0.0008449086911345, + "actual_fraction": 0.9399885419650531, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.95, + "metric": 0.8917502148381553, + "metric_std": 0.000831448024846748, + "actual_fraction": 0.9498710971068463, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.96, + "metric": 0.893125179031796, + "metric_std": 0.000867667214601008, + "actual_fraction": 0.959896877685477, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.97, + "metric": 0.8944142079633343, + "metric_std": 0.0009839910648706794, + "actual_fraction": 0.9699226582641078, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.98, + "metric": 0.8955313663706674, + "metric_std": 0.0010258381311641248, + "actual_fraction": 0.9799484388427386, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.8975938126611286, + "metric_std": 0.00045291859927933527, + "actual_fraction": 0.9899742194213692, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.8990260670295045, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.7642247255506924, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.7677548970366649, + "metric_std": 0.00045209109570275654, + "actual_fraction": 0.010188706321303006, + "actual_fraction_std": 0.0009132469011521748 + }, + { + "budget": 0.02, + "metric": 0.77104111358255, + "metric_std": 0.0008259212479888747, + "actual_fraction": 0.019301140848102173, + "actual_fraction_std": 0.0016187865642072873 + }, + { + "budget": 0.03, + "metric": 0.7738107196670733, + "metric_std": 0.0006930610972831063, + "actual_fraction": 0.02860012915261534, + "actual_fraction_std": 0.0015493994864578075 + }, + { + "budget": 0.04, + "metric": 0.7769821338882112, + "metric_std": 0.0005306697879629957, + "actual_fraction": 0.03896103896103896, + "actual_fraction_std": 0.0014233216922492175 + }, + { + "budget": 0.05, + "metric": 0.7800387457846021, + "metric_std": 0.0005708638668556056, + "actual_fraction": 0.048303078137332284, + "actual_fraction_std": 0.0013398515211391344 + }, + { + "budget": 0.06, + "metric": 0.7828083518691253, + "metric_std": 0.0004892796762928036, + "actual_fraction": 0.057659467604218986, + "actual_fraction_std": 0.0010116700427360008 + }, + { + "budget": 0.07, + "metric": 0.7860228169620435, + "metric_std": 0.000528238855400927, + "actual_fraction": 0.06680060271220492, + "actual_fraction_std": 0.001129943011266676 + }, + { + "budget": 0.08, + "metric": 0.7888211236277535, + "metric_std": 0.0004226641737826951, + "actual_fraction": 0.07564038171772978, + "actual_fraction_std": 0.0012064063214811949 + }, + { + "budget": 0.09, + "metric": 0.7917629331993973, + "metric_std": 0.0008218595716884098, + "actual_fraction": 0.08535552844945112, + "actual_fraction_std": 0.0015928176136054066 + }, + { + "budget": 0.1, + "metric": 0.7951065509076558, + "metric_std": 0.00027133056470098754, + "actual_fraction": 0.0956446868049078, + "actual_fraction_std": 0.0012038431323500437 + }, + { + "budget": 0.11, + "metric": 0.7980053096075196, + "metric_std": 0.0006385476203282448, + "actual_fraction": 0.10518763004950851, + "actual_fraction_std": 0.0020609035827430667 + }, + { + "budget": 0.12, + "metric": 0.8006457630767023, + "metric_std": 0.0007552671230216494, + "actual_fraction": 0.11444356748224152, + "actual_fraction_std": 0.0032704025758996426 + }, + { + "budget": 0.13, + "metric": 0.8032575159646983, + "metric_std": 0.001055746499932735, + "actual_fraction": 0.1241013130515893, + "actual_fraction_std": 0.0029798888948651607 + }, + { + "budget": 0.14, + "metric": 0.8055679127502332, + "metric_std": 0.001219141434997201, + "actual_fraction": 0.13311329554423476, + "actual_fraction_std": 0.0032257844551348623 + }, + { + "budget": 0.15, + "metric": 0.8077778575016143, + "metric_std": 0.0011356240074150998, + "actual_fraction": 0.14165171844729854, + "actual_fraction_std": 0.0031698391940744616 + }, + { + "budget": 0.16, + "metric": 0.8105474635861375, + "metric_std": 0.0015158078204635383, + "actual_fraction": 0.1519982779651288, + "actual_fraction_std": 0.004952825625281874 + }, + { + "budget": 0.17, + "metric": 0.8126569563033652, + "metric_std": 0.0014332341504942488, + "actual_fraction": 0.16144076917557584, + "actual_fraction_std": 0.004615576389775532 + }, + { + "budget": 0.18, + "metric": 0.815340460644328, + "metric_std": 0.001193750984750565, + "actual_fraction": 0.17094066154839632, + "actual_fraction_std": 0.0029068638077834837 + }, + { + "budget": 0.19, + "metric": 0.81776565975461, + "metric_std": 0.0008495808346482388, + "actual_fraction": 0.18067015857071106, + "actual_fraction_std": 0.0025489444885314853 + }, + { + "budget": 0.2, + "metric": 0.8203343617708259, + "metric_std": 0.0008459371545204535, + "actual_fraction": 0.1905431584989596, + "actual_fraction_std": 0.0028169185463401488 + }, + { + "budget": 0.21, + "metric": 0.822730860299921, + "metric_std": 0.0006037363555417293, + "actual_fraction": 0.20117672382865753, + "actual_fraction_std": 0.002082894134158992 + }, + { + "budget": 0.22, + "metric": 0.8254430652220707, + "metric_std": 0.0005859946039777688, + "actual_fraction": 0.21093492143215903, + "actual_fraction_std": 0.0034684239724107387 + }, + { + "budget": 0.23, + "metric": 0.827839563751166, + "metric_std": 0.0007167966560230906, + "actual_fraction": 0.22113797804405538, + "actual_fraction_std": 0.002914116164332676 + }, + { + "budget": 0.24, + "metric": 0.8299777570495802, + "metric_std": 0.0011180340779670236, + "actual_fraction": 0.23108272942527086, + "actual_fraction_std": 0.003264730533632193 + }, + { + "budget": 0.25, + "metric": 0.8316280404678194, + "metric_std": 0.001345794092277, + "actual_fraction": 0.2403243165674105, + "actual_fraction_std": 0.0031100758126100074 + }, + { + "budget": 0.26, + "metric": 0.8331348209801248, + "metric_std": 0.0017703862252468846, + "actual_fraction": 0.24956590370955012, + "actual_fraction_std": 0.0028072154217453084 + }, + { + "budget": 0.27, + "metric": 0.8342684939370022, + "metric_std": 0.0019395467567947702, + "actual_fraction": 0.25989811293678694, + "actual_fraction_std": 0.0029397963420122533 + }, + { + "budget": 0.28, + "metric": 0.8353591160220996, + "metric_std": 0.002370297058477299, + "actual_fraction": 0.26908229891655305, + "actual_fraction_std": 0.003136777891511388 + }, + { + "budget": 0.29, + "metric": 0.8362057831671091, + "metric_std": 0.0033003328560556374, + "actual_fraction": 0.27770682356317716, + "actual_fraction_std": 0.0032026395861862182 + }, + { + "budget": 0.3, + "metric": 0.8369232976967785, + "metric_std": 0.0034460871611594213, + "actual_fraction": 0.2871349644830308, + "actual_fraction_std": 0.004151179675801584 + }, + { + "budget": 0.31, + "metric": 0.837511659611107, + "metric_std": 0.003427061516639971, + "actual_fraction": 0.29709406615483963, + "actual_fraction_std": 0.004016992133957721 + }, + { + "budget": 0.32, + "metric": 0.8379708689100955, + "metric_std": 0.003242101927229976, + "actual_fraction": 0.30633565329697926, + "actual_fraction_std": 0.004590018455847701 + }, + { + "budget": 0.33, + "metric": 0.8386453325679846, + "metric_std": 0.003152739574597068, + "actual_fraction": 0.3153045849178445, + "actual_fraction_std": 0.004876882444660048 + }, + { + "budget": 0.34, + "metric": 0.8391045418669728, + "metric_std": 0.0030755354515210926, + "actual_fraction": 0.3244457200258305, + "actual_fraction_std": 0.0033858019791592016 + }, + { + "budget": 0.35, + "metric": 0.8397072540718948, + "metric_std": 0.00312469133786564, + "actual_fraction": 0.3342756690822989, + "actual_fraction_std": 0.0037501375839460844 + }, + { + "budget": 0.36, + "metric": 0.8402669154050371, + "metric_std": 0.0030708448342348857, + "actual_fraction": 0.3454401951639521, + "actual_fraction_std": 0.003311700728840422 + }, + { + "budget": 0.37, + "metric": 0.8407978761569922, + "metric_std": 0.0029572568289828907, + "actual_fraction": 0.3549687881179594, + "actual_fraction_std": 0.0031244441866725204 + }, + { + "budget": 0.38, + "metric": 0.8411853340030137, + "metric_std": 0.0027796627711107304, + "actual_fraction": 0.3638946688670446, + "actual_fraction_std": 0.0028146330878478527 + }, + { + "budget": 0.39, + "metric": 0.8417162947549688, + "metric_std": 0.002612285056094935, + "actual_fraction": 0.37294970223147017, + "actual_fraction_std": 0.0011695706928018892 + }, + { + "budget": 0.4, + "metric": 0.8422616057975174, + "metric_std": 0.0024382828699943387, + "actual_fraction": 0.38247829518547755, + "actual_fraction_std": 0.0026964612248918905 + }, + { + "budget": 0.41, + "metric": 0.8429360694554064, + "metric_std": 0.0023975079926244485, + "actual_fraction": 0.39404462940374546, + "actual_fraction_std": 0.0020009523629831074 + }, + { + "budget": 0.42, + "metric": 0.8436966348568559, + "metric_std": 0.0023854525649163647, + "actual_fraction": 0.4036449738107197, + "actual_fraction_std": 0.0027135902445647726 + }, + { + "budget": 0.43, + "metric": 0.8443280476429648, + "metric_std": 0.0022430794125708952, + "actual_fraction": 0.4141493865250771, + "actual_fraction_std": 0.002005193171052903 + }, + { + "budget": 0.44, + "metric": 0.8446007031642392, + "metric_std": 0.002214087543194187, + "actual_fraction": 0.42363492860730434, + "actual_fraction_std": 0.0023950372755745684 + }, + { + "budget": 0.45, + "metric": 0.8456626246681495, + "metric_std": 0.002155531159618258, + "actual_fraction": 0.4341249910310684, + "actual_fraction_std": 0.0029152642713017856 + }, + { + "budget": 0.46, + "metric": 0.8462653368730717, + "metric_std": 0.0021790480366302135, + "actual_fraction": 0.4446150534548325, + "actual_fraction_std": 0.002629277144510727 + }, + { + "budget": 0.47, + "metric": 0.8469398005309607, + "metric_std": 0.002138385911089073, + "actual_fraction": 0.4542871493147736, + "actual_fraction_std": 0.0031397307713058775 + }, + { + "budget": 0.48, + "metric": 0.8480734734878382, + "metric_std": 0.002395144751052014, + "actual_fraction": 0.46507856784099866, + "actual_fraction_std": 0.002991441902691609 + }, + { + "budget": 0.49, + "metric": 0.8488053383081008, + "metric_std": 0.002268639795464173, + "actual_fraction": 0.4761856927602784, + "actual_fraction_std": 0.004403400172376863 + }, + { + "budget": 0.5, + "metric": 0.8496663557437039, + "metric_std": 0.0021830608024861887, + "actual_fraction": 0.4878094281409198, + "actual_fraction_std": 0.005134767792896755 + }, + { + "budget": 0.51, + "metric": 0.8506708760852408, + "metric_std": 0.0022509838562324858, + "actual_fraction": 0.498414292889431, + "actual_fraction_std": 0.00516052092754379 + }, + { + "budget": 0.52, + "metric": 0.85143144148669, + "metric_std": 0.0019575132457871364, + "actual_fraction": 0.5074262753820765, + "actual_fraction_std": 0.004253051865256826 + }, + { + "budget": 0.53, + "metric": 0.8522781086316998, + "metric_std": 0.0019289000631888978, + "actual_fraction": 0.5180167898399943, + "actual_fraction_std": 0.004042224945360116 + }, + { + "budget": 0.54, + "metric": 0.8530099734519624, + "metric_std": 0.0016499714259085717, + "actual_fraction": 0.5282485470330773, + "actual_fraction_std": 0.004256500359835602 + }, + { + "budget": 0.55, + "metric": 0.8537705388534118, + "metric_std": 0.0017406198565986994, + "actual_fraction": 0.5390973667216761, + "actual_fraction_std": 0.003689657573932102 + }, + { + "budget": 0.56, + "metric": 0.8546889574513884, + "metric_std": 0.0017940733070281585, + "actual_fraction": 0.5482672024108488, + "actual_fraction_std": 0.0032952592006156374 + }, + { + "budget": 0.57, + "metric": 0.8558656812800459, + "metric_std": 0.001840113539590414, + "actual_fraction": 0.5591160220994474, + "actual_fraction_std": 0.0034172830020057785 + }, + { + "budget": 0.58, + "metric": 0.8568271507498026, + "metric_std": 0.0017521170049423311, + "actual_fraction": 0.5695343330702447, + "actual_fraction_std": 0.003173572523655903 + }, + { + "budget": 0.59, + "metric": 0.8580469254502404, + "metric_std": 0.0014961511394392777, + "actual_fraction": 0.5808136614766449, + "actual_fraction_std": 0.0040415243919576065 + }, + { + "budget": 0.6, + "metric": 0.8589509937576235, + "metric_std": 0.0013243926648377152, + "actual_fraction": 0.5910597689603214, + "actual_fraction_std": 0.0038810837790726367 + }, + { + "budget": 0.61, + "metric": 0.8600272655521275, + "metric_std": 0.0013253641258829114, + "actual_fraction": 0.6013345770251848, + "actual_fraction_std": 0.0034464606276307552 + }, + { + "budget": 0.62, + "metric": 0.8605008251417091, + "metric_std": 0.0013953766850866175, + "actual_fraction": 0.6113510798593671, + "actual_fraction_std": 0.0024425020855916224 + }, + { + "budget": 0.63, + "metric": 0.8613044414149387, + "metric_std": 0.001702491284739157, + "actual_fraction": 0.6209514242663414, + "actual_fraction_std": 0.002405761056593176 + }, + { + "budget": 0.64, + "metric": 0.8624237640812227, + "metric_std": 0.0016008776671644737, + "actual_fraction": 0.6315419387242592, + "actual_fraction_std": 0.002501749999137093 + }, + { + "budget": 0.65, + "metric": 0.8633565329697926, + "metric_std": 0.0016048925143863082, + "actual_fraction": 0.6413718877807275, + "actual_fraction_std": 0.0026174041001556614 + }, + { + "budget": 0.66, + "metric": 0.8643467030207361, + "metric_std": 0.0019243573816628795, + "actual_fraction": 0.6514027409055033, + "actual_fraction_std": 0.0015806504729667886 + }, + { + "budget": 0.67, + "metric": 0.8652077204563392, + "metric_std": 0.00194498059962172, + "actual_fraction": 0.6616918992609601, + "actual_fraction_std": 0.0016060148730429333 + }, + { + "budget": 0.68, + "metric": 0.866169189926096, + "metric_std": 0.0016544895563959095, + "actual_fraction": 0.6717945038387028, + "actual_fraction_std": 0.0009236170771346043 + }, + { + "budget": 0.69, + "metric": 0.8676903207289948, + "metric_std": 0.001620375947569254, + "actual_fraction": 0.6827007246896749, + "actual_fraction_std": 0.0006283887219864056 + }, + { + "budget": 0.7, + "metric": 0.8690392480447731, + "metric_std": 0.0015755939443845186, + "actual_fraction": 0.6937791490277678, + "actual_fraction_std": 0.0011303985416263645 + }, + { + "budget": 0.71, + "metric": 0.8700724689674966, + "metric_std": 0.0016653448859750136, + "actual_fraction": 0.7034942957594891, + "actual_fraction_std": 0.0018028043388311407 + }, + { + "budget": 0.72, + "metric": 0.8709334864030996, + "metric_std": 0.0017133429679528369, + "actual_fraction": 0.712893736098156, + "actual_fraction_std": 0.0015593358825643844 + }, + { + "budget": 0.73, + "metric": 0.8720384587787903, + "metric_std": 0.0015642804120941684, + "actual_fraction": 0.7244170194446438, + "actual_fraction_std": 0.004029404703145846 + }, + { + "budget": 0.74, + "metric": 0.8732151826074478, + "metric_std": 0.0017024912847391665, + "actual_fraction": 0.7344765731506062, + "actual_fraction_std": 0.003894656705762376 + }, + { + "budget": 0.75, + "metric": 0.8743058046925449, + "metric_std": 0.001574776854693956, + "actual_fraction": 0.7451101384803043, + "actual_fraction_std": 0.005257868924876471 + }, + { + "budget": 0.76, + "metric": 0.8757408337518836, + "metric_std": 0.001640427155711966, + "actual_fraction": 0.7549831384085527, + "actual_fraction_std": 0.00516396159102071 + }, + { + "budget": 0.77, + "metric": 0.876774054674607, + "metric_std": 0.0015871512220647106, + "actual_fraction": 0.7644399799095931, + "actual_fraction_std": 0.004265743060666494 + }, + { + "budget": 0.78, + "metric": 0.8777211738537705, + "metric_std": 0.0017223338096462447, + "actual_fraction": 0.7743990815814021, + "actual_fraction_std": 0.0038127644604392195 + }, + { + "budget": 0.79, + "metric": 0.8788117959388677, + "metric_std": 0.001420606290681161, + "actual_fraction": 0.7841142283131235, + "actual_fraction_std": 0.003597389167146175 + }, + { + "budget": 0.8, + "metric": 0.8802037741264261, + "metric_std": 0.0017247234518771692, + "actual_fraction": 0.7939441773695918, + "actual_fraction_std": 0.004311778779754869 + }, + { + "budget": 0.81, + "metric": 0.8816388031857645, + "metric_std": 0.0018776410686714392, + "actual_fraction": 0.8035875726483461, + "actual_fraction_std": 0.004560763020907057 + }, + { + "budget": 0.82, + "metric": 0.8831312334074765, + "metric_std": 0.002254982764796485, + "actual_fraction": 0.814292889431011, + "actual_fraction_std": 0.00500139023888536 + }, + { + "budget": 0.83, + "metric": 0.8845088613044414, + "metric_std": 0.0020675128569559976, + "actual_fraction": 0.8249121044701155, + "actual_fraction_std": 0.005270533734846584 + }, + { + "budget": 0.84, + "metric": 0.8856855851330989, + "metric_std": 0.0018048022228245426, + "actual_fraction": 0.8344980985864965, + "actual_fraction_std": 0.004453390212448819 + }, + { + "budget": 0.85, + "metric": 0.8871206141924374, + "metric_std": 0.002240093691454006, + "actual_fraction": 0.8456769749587428, + "actual_fraction_std": 0.004873397561292773 + }, + { + "budget": 0.86, + "metric": 0.8883690894740619, + "metric_std": 0.002273173905704326, + "actual_fraction": 0.8561096362201335, + "actual_fraction_std": 0.005067400330355552 + }, + { + "budget": 0.87, + "metric": 0.8901341752170483, + "metric_std": 0.002217572650881197, + "actual_fraction": 0.8667001506780512, + "actual_fraction_std": 0.003926188105118881 + }, + { + "budget": 0.88, + "metric": 0.8911817464303653, + "metric_std": 0.002378428089715449, + "actual_fraction": 0.8770754107770683, + "actual_fraction_std": 0.004712273600880473 + }, + { + "budget": 0.89, + "metric": 0.8923728205496161, + "metric_std": 0.0017844342563601632, + "actual_fraction": 0.8868766592523498, + "actual_fraction_std": 0.0034969541193879124 + }, + { + "budget": 0.9, + "metric": 0.8938939513525149, + "metric_std": 0.0015425728046105222, + "actual_fraction": 0.8979550835904426, + "actual_fraction_std": 0.0032129912877167954 + }, + { + "budget": 0.91, + "metric": 0.8953289804118534, + "metric_std": 0.0017335067880951377, + "actual_fraction": 0.9084451460142067, + "actual_fraction_std": 0.0021798747981579087 + }, + { + "budget": 0.92, + "metric": 0.8966492071464447, + "metric_std": 0.00130500883052476, + "actual_fraction": 0.9191791633780586, + "actual_fraction_std": 0.0017499118813539879 + }, + { + "budget": 0.93, + "metric": 0.8978689818468825, + "metric_std": 0.000949995740412622, + "actual_fraction": 0.9293822199899548, + "actual_fraction_std": 0.002763781772671289 + }, + { + "budget": 0.94, + "metric": 0.8991748582908803, + "metric_std": 0.000711027575049815, + "actual_fraction": 0.9402597402597402, + "actual_fraction_std": 0.002873464291477129 + }, + { + "budget": 0.95, + "metric": 0.9003228815383512, + "metric_std": 0.000998609079297229, + "actual_fraction": 0.9495443782736601, + "actual_fraction_std": 0.00208042097473281 + }, + { + "budget": 0.96, + "metric": 0.9019444643754035, + "metric_std": 0.000713196449151997, + "actual_fraction": 0.959058620937074, + "actual_fraction_std": 0.0021288548479926057 + }, + { + "budget": 0.97, + "metric": 0.9032216402382149, + "metric_std": 0.0007751149760095566, + "actual_fraction": 0.9693477792925307, + "actual_fraction_std": 0.0029655145467680833 + }, + { + "budget": 0.98, + "metric": 0.9047140704599268, + "metric_std": 0.0005073593895290028, + "actual_fraction": 0.9798234914257014, + "actual_fraction_std": 0.0017702408196185054 + }, + { + "budget": 0.99, + "metric": 0.9061778001004519, + "metric_std": 0.0005525327777334696, + "actual_fraction": 0.9900839491999711, + "actual_fraction_std": 0.001108554143190012 + }, + { + "budget": 1.0, + "metric": 0.907368874219703, + "metric_std": 1.2412670766236366e-16, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 4888, + "rescuable": 1389, + "harmful": 388, + "both_wrong": 317, + "unit": "sample", + "total": 6982 + }, + "test": { + "both_correct": 9940, + "rescuable": 2706, + "harmful": 711, + "both_wrong": 580, + "unit": "sample", + "total": 13937 + } + }, + "small": { + "validation": 0.7556574047550845, + "test": 0.7642247255506924 + }, + "large": { + "validation": 0.8990260670295045, + "test": 0.9073688742197029 + }, + "learned_minus_confidence_test": -0.00025830523068104405, + "learned_minus_small_test": 0.14195307454975947, + "learned_minus_large_test": -0.0011910741192510121, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_vilt_MiniCPM" + }, + { + "key": [ + "nlvr2", + "vilt", + "Qwen3.8" + ], + "expert_display": "ViLT", + "run": "five_seed_vilt_Qwen3.8_four_difference", + "validation_rows": 6982, + "test_rows": 13937, + "confidence_rule": { + "column": "max_probability", + "columns": [ + "entropy", + "max_probability", + "top_two_margin", + "entropy_valid", + "max_probability_valid", + "top_two_margin_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/vilt/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 96, + "validation_area_0_25": 0.7940561443712404, + "validation_point": { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.9059008879977084, + "rescued": 1446, + "harmed": 397, + "missed_rescue": 3, + "avoided_harm": 9, + "selected_both_correct": 4603, + "selected_both_wrong": 256 + }, + "test_point": { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9593169261677549, + "metric": 0.8996197172992753, + "rescued": 2770, + "harmed": 883, + "missed_rescue": 21, + "avoided_harm": 7, + "selected_both_correct": 9230, + "selected_both_wrong": 487 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7556574047550845, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1449, + "avoided_harm": 406, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5127503275871277, + "operator": "ge", + "actual_fraction": 0.009882555141793183, + "metric": 0.7588083643655114, + "rescued": 28, + "harmed": 6, + "missed_rescue": 1421, + "avoided_harm": 400, + "selected_both_correct": 28, + "selected_both_wrong": 7 + }, + { + "budget": 0.02, + "threshold": -0.5296345353126526, + "operator": "ge", + "actual_fraction": 0.01990833572042395, + "metric": 0.7610999713549126, + "rescued": 51, + "harmed": 13, + "missed_rescue": 1398, + "avoided_harm": 393, + "selected_both_correct": 62, + "selected_both_wrong": 13 + }, + { + "budget": 0.03, + "threshold": -0.5434877276420593, + "operator": "ge", + "actual_fraction": 0.029934116299054712, + "metric": 0.7632483529074764, + "rescued": 77, + "harmed": 24, + "missed_rescue": 1372, + "avoided_harm": 382, + "selected_both_correct": 91, + "selected_both_wrong": 17 + }, + { + "budget": 0.04, + "threshold": -0.56125408411026, + "operator": "ge", + "actual_fraction": 0.039959896877685475, + "metric": 0.7656831853337153, + "rescued": 102, + "harmed": 32, + "missed_rescue": 1347, + "avoided_harm": 374, + "selected_both_correct": 120, + "selected_both_wrong": 25 + }, + { + "budget": 0.05, + "threshold": -0.5779195427894592, + "operator": "ge", + "actual_fraction": 0.04998567745631624, + "metric": 0.7698367230020051, + "rescued": 132, + "harmed": 33, + "missed_rescue": 1317, + "avoided_harm": 373, + "selected_both_correct": 153, + "selected_both_wrong": 31 + }, + { + "budget": 0.06, + "threshold": -0.5911363363265991, + "operator": "ge", + "actual_fraction": 0.059868232598109423, + "metric": 0.7727012317387568, + "rescued": 156, + "harmed": 37, + "missed_rescue": 1293, + "avoided_harm": 369, + "selected_both_correct": 188, + "selected_both_wrong": 37 + }, + { + "budget": 0.07, + "threshold": -0.6087464094161987, + "operator": "ge", + "actual_fraction": 0.06989401317674018, + "metric": 0.7772844457175594, + "rescued": 191, + "harmed": 40, + "missed_rescue": 1258, + "avoided_harm": 366, + "selected_both_correct": 218, + "selected_both_wrong": 39 + }, + { + "budget": 0.08, + "threshold": -0.6274068355560303, + "operator": "ge", + "actual_fraction": 0.07991979375537095, + "metric": 0.780865081638499, + "rescued": 221, + "harmed": 45, + "missed_rescue": 1228, + "avoided_harm": 361, + "selected_both_correct": 249, + "selected_both_wrong": 43 + }, + { + "budget": 0.09, + "threshold": -0.6465112566947937, + "operator": "ge", + "actual_fraction": 0.08994557433400172, + "metric": 0.784302492122601, + "rescued": 250, + "harmed": 50, + "missed_rescue": 1199, + "avoided_harm": 356, + "selected_both_correct": 283, + "selected_both_wrong": 45 + }, + { + "budget": 0.1, + "threshold": -0.6620025038719177, + "operator": "ge", + "actual_fraction": 0.09997135491263248, + "metric": 0.7884560297908909, + "rescued": 284, + "harmed": 55, + "missed_rescue": 1165, + "avoided_harm": 351, + "selected_both_correct": 309, + "selected_both_wrong": 50 + }, + { + "budget": 0.11, + "threshold": -0.6800539493560791, + "operator": "ge", + "actual_fraction": 0.10999713549126325, + "metric": 0.7910340876539673, + "rescued": 306, + "harmed": 59, + "missed_rescue": 1143, + "avoided_harm": 347, + "selected_both_correct": 348, + "selected_both_wrong": 55 + }, + { + "budget": 0.12, + "threshold": -0.6957284212112427, + "operator": "ge", + "actual_fraction": 0.11987969063305644, + "metric": 0.7944714981380693, + "rescued": 334, + "harmed": 63, + "missed_rescue": 1115, + "avoided_harm": 343, + "selected_both_correct": 380, + "selected_both_wrong": 60 + }, + { + "budget": 0.13, + "threshold": -0.7128273248672485, + "operator": "ge", + "actual_fraction": 0.1299054712116872, + "metric": 0.7974792323116585, + "rescued": 362, + "harmed": 70, + "missed_rescue": 1087, + "avoided_harm": 336, + "selected_both_correct": 412, + "selected_both_wrong": 63 + }, + { + "budget": 0.14, + "threshold": -0.7332167625427246, + "operator": "ge", + "actual_fraction": 0.13993125179031796, + "metric": 0.8006301919220854, + "rescued": 386, + "harmed": 72, + "missed_rescue": 1063, + "avoided_harm": 334, + "selected_both_correct": 452, + "selected_both_wrong": 67 + }, + { + "budget": 0.15, + "threshold": -0.7503995895385742, + "operator": "ge", + "actual_fraction": 0.14995703236894872, + "metric": 0.8039243769693497, + "rescued": 417, + "harmed": 80, + "missed_rescue": 1032, + "avoided_harm": 326, + "selected_both_correct": 479, + "selected_both_wrong": 71 + }, + { + "budget": 0.16, + "threshold": -0.7655540704727173, + "operator": "ge", + "actual_fraction": 0.1599828129475795, + "metric": 0.8054998567745631, + "rescued": 438, + "harmed": 90, + "missed_rescue": 1011, + "avoided_harm": 316, + "selected_both_correct": 517, + "selected_both_wrong": 72 + }, + { + "budget": 0.17, + "threshold": -0.7812193036079407, + "operator": "ge", + "actual_fraction": 0.16986536808937266, + "metric": 0.8086508163849899, + "rescued": 464, + "harmed": 94, + "missed_rescue": 985, + "avoided_harm": 312, + "selected_both_correct": 551, + "selected_both_wrong": 77 + }, + { + "budget": 0.18, + "threshold": -0.7954917550086975, + "operator": "ge", + "actual_fraction": 0.17989114866800343, + "metric": 0.810369521627041, + "rescued": 481, + "harmed": 99, + "missed_rescue": 968, + "avoided_harm": 307, + "selected_both_correct": 597, + "selected_both_wrong": 79 + }, + { + "budget": 0.19, + "threshold": -0.8097342252731323, + "operator": "ge", + "actual_fraction": 0.1899169292466342, + "metric": 0.8132340303637926, + "rescued": 505, + "harmed": 103, + "missed_rescue": 944, + "avoided_harm": 303, + "selected_both_correct": 635, + "selected_both_wrong": 83 + }, + { + "budget": 0.2, + "threshold": -0.8232117891311646, + "operator": "ge", + "actual_fraction": 0.19994270982526496, + "metric": 0.8158120882268691, + "rescued": 527, + "harmed": 107, + "missed_rescue": 922, + "avoided_harm": 299, + "selected_both_correct": 673, + "selected_both_wrong": 89 + }, + { + "budget": 0.21, + "threshold": -0.8359466791152954, + "operator": "ge", + "actual_fraction": 0.20996849040389573, + "metric": 0.8181036952162704, + "rescued": 550, + "harmed": 114, + "missed_rescue": 899, + "avoided_harm": 292, + "selected_both_correct": 710, + "selected_both_wrong": 92 + }, + { + "budget": 0.22, + "threshold": -0.8473842144012451, + "operator": "ge", + "actual_fraction": 0.2199942709825265, + "metric": 0.8203953022056717, + "rescued": 574, + "harmed": 122, + "missed_rescue": 875, + "avoided_harm": 284, + "selected_both_correct": 745, + "selected_both_wrong": 95 + }, + { + "budget": 0.23, + "threshold": -0.8595863580703735, + "operator": "ge", + "actual_fraction": 0.22987682612431967, + "metric": 0.8234030363792609, + "rescued": 597, + "harmed": 124, + "missed_rescue": 852, + "avoided_harm": 282, + "selected_both_correct": 779, + "selected_both_wrong": 105 + }, + { + "budget": 0.24, + "threshold": -0.8704345226287842, + "operator": "ge", + "actual_fraction": 0.23990260670295044, + "metric": 0.8256946433686623, + "rescued": 618, + "harmed": 129, + "missed_rescue": 831, + "avoided_harm": 277, + "selected_both_correct": 819, + "selected_both_wrong": 109 + }, + { + "budget": 0.25, + "threshold": -0.8811548352241516, + "operator": "ge", + "actual_fraction": 0.2499283872815812, + "metric": 0.828845602979089, + "rescued": 647, + "harmed": 136, + "missed_rescue": 802, + "avoided_harm": 270, + "selected_both_correct": 847, + "selected_both_wrong": 115 + }, + { + "budget": 0.26, + "threshold": -0.8929013609886169, + "operator": "ge", + "actual_fraction": 0.25995416786021197, + "metric": 0.8304210827843025, + "rescued": 667, + "harmed": 145, + "missed_rescue": 782, + "avoided_harm": 261, + "selected_both_correct": 886, + "selected_both_wrong": 117 + }, + { + "budget": 0.27, + "threshold": -0.903549313545227, + "operator": "ge", + "actual_fraction": 0.26997994843884277, + "metric": 0.8327126897737038, + "rescued": 693, + "harmed": 155, + "missed_rescue": 756, + "avoided_harm": 251, + "selected_both_correct": 919, + "selected_both_wrong": 118 + }, + { + "budget": 0.28, + "threshold": -0.9115740656852722, + "operator": "ge", + "actual_fraction": 0.2798625035806359, + "metric": 0.8350042967631052, + "rescued": 716, + "harmed": 162, + "missed_rescue": 733, + "avoided_harm": 244, + "selected_both_correct": 954, + "selected_both_wrong": 122 + }, + { + "budget": 0.29, + "threshold": -0.9182595014572144, + "operator": "ge", + "actual_fraction": 0.2898882841592667, + "metric": 0.8371526783156689, + "rescued": 738, + "harmed": 169, + "missed_rescue": 711, + "avoided_harm": 237, + "selected_both_correct": 994, + "selected_both_wrong": 123 + }, + { + "budget": 0.3, + "threshold": -0.9258768558502197, + "operator": "ge", + "actual_fraction": 0.29991406473789745, + "metric": 0.8377255800630192, + "rescued": 751, + "harmed": 178, + "missed_rescue": 698, + "avoided_harm": 228, + "selected_both_correct": 1036, + "selected_both_wrong": 129 + }, + { + "budget": 0.31, + "threshold": -0.9331877827644348, + "operator": "ge", + "actual_fraction": 0.30993984531652824, + "metric": 0.8401604124892581, + "rescued": 771, + "harmed": 181, + "missed_rescue": 678, + "avoided_harm": 225, + "selected_both_correct": 1077, + "selected_both_wrong": 135 + }, + { + "budget": 0.32, + "threshold": -0.9399217367172241, + "operator": "ge", + "actual_fraction": 0.319965625895159, + "metric": 0.842595244915497, + "rescued": 793, + "harmed": 186, + "missed_rescue": 656, + "avoided_harm": 220, + "selected_both_correct": 1118, + "selected_both_wrong": 137 + }, + { + "budget": 0.33, + "threshold": -0.9453923106193542, + "operator": "ge", + "actual_fraction": 0.3299914064737898, + "metric": 0.8447436264680608, + "rescued": 814, + "harmed": 192, + "missed_rescue": 635, + "avoided_harm": 214, + "selected_both_correct": 1157, + "selected_both_wrong": 141 + }, + { + "budget": 0.34, + "threshold": -0.9515207409858704, + "operator": "ge", + "actual_fraction": 0.3398739616155829, + "metric": 0.8466055571469493, + "rescued": 830, + "harmed": 195, + "missed_rescue": 619, + "avoided_harm": 211, + "selected_both_correct": 1202, + "selected_both_wrong": 146 + }, + { + "budget": 0.35, + "threshold": -0.9567410349845886, + "operator": "ge", + "actual_fraction": 0.3498997421942137, + "metric": 0.8491836150100258, + "rescued": 854, + "harmed": 201, + "missed_rescue": 595, + "avoided_harm": 205, + "selected_both_correct": 1238, + "selected_both_wrong": 150 + }, + { + "budget": 0.36, + "threshold": -0.9606627225875854, + "operator": "ge", + "actual_fraction": 0.35992552277284445, + "metric": 0.8519048983099399, + "rescued": 876, + "harmed": 204, + "missed_rescue": 573, + "avoided_harm": 202, + "selected_both_correct": 1279, + "selected_both_wrong": 154 + }, + { + "budget": 0.37, + "threshold": -0.9650874733924866, + "operator": "ge", + "actual_fraction": 0.36995130335147525, + "metric": 0.8534803781151532, + "rescued": 894, + "harmed": 211, + "missed_rescue": 555, + "avoided_harm": 195, + "selected_both_correct": 1318, + "selected_both_wrong": 160 + }, + { + "budget": 0.38, + "threshold": -0.9686882495880127, + "operator": "ge", + "actual_fraction": 0.379977083930106, + "metric": 0.8562016614150673, + "rescued": 916, + "harmed": 214, + "missed_rescue": 533, + "avoided_harm": 192, + "selected_both_correct": 1359, + "selected_both_wrong": 164 + }, + { + "budget": 0.39, + "threshold": -0.971785306930542, + "operator": "ge", + "actual_fraction": 0.3898596390718992, + "metric": 0.8567745631624176, + "rescued": 927, + "harmed": 221, + "missed_rescue": 522, + "avoided_harm": 185, + "selected_both_correct": 1406, + "selected_both_wrong": 168 + }, + { + "budget": 0.4, + "threshold": -0.9748144745826721, + "operator": "ge", + "actual_fraction": 0.3998854196505299, + "metric": 0.859066170151819, + "rescued": 948, + "harmed": 226, + "missed_rescue": 501, + "avoided_harm": 180, + "selected_both_correct": 1447, + "selected_both_wrong": 171 + }, + { + "budget": 0.41, + "threshold": -0.977255642414093, + "operator": "ge", + "actual_fraction": 0.4099112002291607, + "metric": 0.860068748209682, + "rescued": 960, + "harmed": 231, + "missed_rescue": 489, + "avoided_harm": 175, + "selected_both_correct": 1498, + "selected_both_wrong": 173 + }, + { + "budget": 0.42, + "threshold": -0.9794421792030334, + "operator": "ge", + "actual_fraction": 0.41993698080779146, + "metric": 0.8627900315095961, + "rescued": 982, + "harmed": 234, + "missed_rescue": 467, + "avoided_harm": 172, + "selected_both_correct": 1539, + "selected_both_wrong": 177 + }, + { + "budget": 0.43, + "threshold": -0.981472373008728, + "operator": "ge", + "actual_fraction": 0.42996276138642225, + "metric": 0.8642222858779719, + "rescued": 998, + "harmed": 240, + "missed_rescue": 451, + "avoided_harm": 166, + "selected_both_correct": 1584, + "selected_both_wrong": 180 + }, + { + "budget": 0.44, + "threshold": -0.9835578799247742, + "operator": "ge", + "actual_fraction": 0.439988541965053, + "metric": 0.8653680893726726, + "rescued": 1014, + "harmed": 248, + "missed_rescue": 435, + "avoided_harm": 158, + "selected_both_correct": 1628, + "selected_both_wrong": 182 + }, + { + "budget": 0.45, + "threshold": -0.9851564168930054, + "operator": "ge", + "actual_fraction": 0.4498710971068462, + "metric": 0.8668003437410484, + "rescued": 1030, + "harmed": 254, + "missed_rescue": 419, + "avoided_harm": 152, + "selected_both_correct": 1673, + "selected_both_wrong": 184 + }, + { + "budget": 0.46, + "threshold": -0.986689031124115, + "operator": "ge", + "actual_fraction": 0.45989687768547693, + "metric": 0.8686622744199369, + "rescued": 1048, + "harmed": 259, + "missed_rescue": 401, + "avoided_harm": 147, + "selected_both_correct": 1719, + "selected_both_wrong": 185 + }, + { + "budget": 0.47, + "threshold": -0.9879001379013062, + "operator": "ge", + "actual_fraction": 0.46992265826410773, + "metric": 0.8696648524778, + "rescued": 1057, + "harmed": 261, + "missed_rescue": 392, + "avoided_harm": 145, + "selected_both_correct": 1775, + "selected_both_wrong": 188 + }, + { + "budget": 0.48, + "threshold": -0.9891784191131592, + "operator": "ge", + "actual_fraction": 0.47994843884273847, + "metric": 0.8723861357777141, + "rescued": 1077, + "harmed": 262, + "missed_rescue": 372, + "avoided_harm": 144, + "selected_both_correct": 1821, + "selected_both_wrong": 191 + }, + { + "budget": 0.49, + "threshold": -0.990114688873291, + "operator": "ge", + "actual_fraction": 0.48997421942136926, + "metric": 0.8741048410197652, + "rescued": 1092, + "harmed": 265, + "missed_rescue": 357, + "avoided_harm": 141, + "selected_both_correct": 1872, + "selected_both_wrong": 192 + }, + { + "budget": 0.5, + "threshold": -0.9910277724266052, + "operator": "ge", + "actual_fraction": 0.5, + "metric": 0.8746777427671154, + "rescued": 1103, + "harmed": 272, + "missed_rescue": 346, + "avoided_harm": 134, + "selected_both_correct": 1922, + "selected_both_wrong": 194 + }, + { + "budget": 0.51, + "threshold": -0.9918718934059143, + "operator": "ge", + "actual_fraction": 0.5098825551417931, + "metric": 0.8749641936407906, + "rescued": 1110, + "harmed": 277, + "missed_rescue": 339, + "avoided_harm": 129, + "selected_both_correct": 1977, + "selected_both_wrong": 196 + }, + { + "budget": 0.52, + "threshold": -0.9926809668540955, + "operator": "ge", + "actual_fraction": 0.5199083357204239, + "metric": 0.8758235462618161, + "rescued": 1120, + "harmed": 281, + "missed_rescue": 329, + "avoided_harm": 125, + "selected_both_correct": 2033, + "selected_both_wrong": 196 + }, + { + "budget": 0.53, + "threshold": -0.9932646751403809, + "operator": "ge", + "actual_fraction": 0.5299341162990547, + "metric": 0.8762532225723288, + "rescued": 1129, + "harmed": 287, + "missed_rescue": 320, + "avoided_harm": 119, + "selected_both_correct": 2086, + "selected_both_wrong": 198 + }, + { + "budget": 0.54, + "threshold": -0.9938417673110962, + "operator": "ge", + "actual_fraction": 0.5399598968776855, + "metric": 0.8768261243196792, + "rescued": 1136, + "harmed": 290, + "missed_rescue": 313, + "avoided_harm": 116, + "selected_both_correct": 2144, + "selected_both_wrong": 200 + }, + { + "budget": 0.55, + "threshold": -0.9944412112236023, + "operator": "ge", + "actual_fraction": 0.5499856774563162, + "metric": 0.8778287023775423, + "rescued": 1146, + "harmed": 293, + "missed_rescue": 303, + "avoided_harm": 113, + "selected_both_correct": 2200, + "selected_both_wrong": 201 + }, + { + "budget": 0.56, + "threshold": -0.9949513673782349, + "operator": "ge", + "actual_fraction": 0.5598682325981095, + "metric": 0.878258378688055, + "rescued": 1156, + "harmed": 300, + "missed_rescue": 293, + "avoided_harm": 106, + "selected_both_correct": 2249, + "selected_both_wrong": 204 + }, + { + "budget": 0.57, + "threshold": -0.9954326152801514, + "operator": "ge", + "actual_fraction": 0.5698940131767402, + "metric": 0.8792609567459181, + "rescued": 1170, + "harmed": 307, + "missed_rescue": 279, + "avoided_harm": 99, + "selected_both_correct": 2295, + "selected_both_wrong": 207 + }, + { + "budget": 0.58, + "threshold": -0.9958608746528625, + "operator": "ge", + "actual_fraction": 0.579919793755371, + "metric": 0.8805499856774563, + "rescued": 1182, + "harmed": 310, + "missed_rescue": 267, + "avoided_harm": 96, + "selected_both_correct": 2347, + "selected_both_wrong": 210 + }, + { + "budget": 0.59, + "threshold": -0.9962128400802612, + "operator": "ge", + "actual_fraction": 0.5899455743340017, + "metric": 0.8819822400458321, + "rescued": 1195, + "harmed": 313, + "missed_rescue": 254, + "avoided_harm": 93, + "selected_both_correct": 2399, + "selected_both_wrong": 212 + }, + { + "budget": 0.6, + "threshold": -0.9965764880180359, + "operator": "ge", + "actual_fraction": 0.5999713549126325, + "metric": 0.8819822400458321, + "rescued": 1202, + "harmed": 320, + "missed_rescue": 247, + "avoided_harm": 86, + "selected_both_correct": 2454, + "selected_both_wrong": 213 + }, + { + "budget": 0.61, + "threshold": -0.9968799352645874, + "operator": "ge", + "actual_fraction": 0.6099971354912632, + "metric": 0.8832712689773704, + "rescued": 1214, + "harmed": 323, + "missed_rescue": 235, + "avoided_harm": 83, + "selected_both_correct": 2507, + "selected_both_wrong": 215 + }, + { + "budget": 0.62, + "threshold": -0.9971383810043335, + "operator": "ge", + "actual_fraction": 0.6198796906330565, + "metric": 0.8832712689773704, + "rescued": 1218, + "harmed": 327, + "missed_rescue": 231, + "avoided_harm": 79, + "selected_both_correct": 2566, + "selected_both_wrong": 217 + }, + { + "budget": 0.63, + "threshold": -0.997377872467041, + "operator": "ge", + "actual_fraction": 0.6299054712116872, + "metric": 0.8842738470352335, + "rescued": 1228, + "harmed": 330, + "missed_rescue": 221, + "avoided_harm": 76, + "selected_both_correct": 2619, + "selected_both_wrong": 221 + }, + { + "budget": 0.64, + "threshold": -0.9976021647453308, + "operator": "ge", + "actual_fraction": 0.639931251790318, + "metric": 0.886135777714122, + "rescued": 1243, + "harmed": 332, + "missed_rescue": 206, + "avoided_harm": 74, + "selected_both_correct": 2672, + "selected_both_wrong": 221 + }, + { + "budget": 0.65, + "threshold": -0.9978214502334595, + "operator": "ge", + "actual_fraction": 0.6499570323689488, + "metric": 0.8867086794614724, + "rescued": 1249, + "harmed": 334, + "missed_rescue": 200, + "avoided_harm": 72, + "selected_both_correct": 2732, + "selected_both_wrong": 223 + }, + { + "budget": 0.66, + "threshold": -0.9980514049530029, + "operator": "ge", + "actual_fraction": 0.6599828129475795, + "metric": 0.8882841592666858, + "rescued": 1261, + "harmed": 335, + "missed_rescue": 188, + "avoided_harm": 71, + "selected_both_correct": 2789, + "selected_both_wrong": 223 + }, + { + "budget": 0.67, + "threshold": -0.9982016086578369, + "operator": "ge", + "actual_fraction": 0.6698653680893727, + "metric": 0.889573188198224, + "rescued": 1271, + "harmed": 336, + "missed_rescue": 178, + "avoided_harm": 70, + "selected_both_correct": 2847, + "selected_both_wrong": 223 + }, + { + "budget": 0.68, + "threshold": -0.9983693957328796, + "operator": "ge", + "actual_fraction": 0.6798911486680035, + "metric": 0.8907189916929247, + "rescued": 1280, + "harmed": 337, + "missed_rescue": 169, + "avoided_harm": 69, + "selected_both_correct": 2907, + "selected_both_wrong": 223 + }, + { + "budget": 0.69, + "threshold": -0.9984915256500244, + "operator": "ge", + "actual_fraction": 0.6899169292466342, + "metric": 0.8921512460613005, + "rescued": 1293, + "harmed": 340, + "missed_rescue": 156, + "avoided_harm": 66, + "selected_both_correct": 2960, + "selected_both_wrong": 224 + }, + { + "budget": 0.7, + "threshold": -0.9986109733581543, + "operator": "gt", + "actual_fraction": 0.6997994843884274, + "metric": 0.8928673732454884, + "rescued": 1299, + "harmed": 341, + "missed_rescue": 150, + "avoided_harm": 65, + "selected_both_correct": 3021, + "selected_both_wrong": 225 + }, + { + "budget": 0.71, + "threshold": -0.9986966252326965, + "operator": "ge", + "actual_fraction": 0.7099684904038958, + "metric": 0.8931538241191636, + "rescued": 1304, + "harmed": 344, + "missed_rescue": 145, + "avoided_harm": 62, + "selected_both_correct": 3084, + "selected_both_wrong": 225 + }, + { + "budget": 0.72, + "threshold": -0.9987963438034058, + "operator": "ge", + "actual_fraction": 0.7199942709825266, + "metric": 0.8935835004296763, + "rescued": 1310, + "harmed": 347, + "missed_rescue": 139, + "avoided_harm": 59, + "selected_both_correct": 3144, + "selected_both_wrong": 226 + }, + { + "budget": 0.73, + "threshold": -0.998875081539154, + "operator": "ge", + "actual_fraction": 0.7298768261243197, + "metric": 0.8940131767401891, + "rescued": 1318, + "harmed": 352, + "missed_rescue": 131, + "avoided_harm": 54, + "selected_both_correct": 3199, + "selected_both_wrong": 227 + }, + { + "budget": 0.74, + "threshold": -0.9989551305770874, + "operator": "ge", + "actual_fraction": 0.7399026067029505, + "metric": 0.8951589802348897, + "rescued": 1327, + "harmed": 353, + "missed_rescue": 122, + "avoided_harm": 53, + "selected_both_correct": 3257, + "selected_both_wrong": 229 + }, + { + "budget": 0.75, + "threshold": -0.9990480542182922, + "operator": "ge", + "actual_fraction": 0.7499283872815812, + "metric": 0.8961615582927528, + "rescued": 1336, + "harmed": 355, + "missed_rescue": 113, + "avoided_harm": 51, + "selected_both_correct": 3314, + "selected_both_wrong": 231 + }, + { + "budget": 0.76, + "threshold": -0.999126136302948, + "operator": "ge", + "actual_fraction": 0.759954167860212, + "metric": 0.8973073617874534, + "rescued": 1345, + "harmed": 356, + "missed_rescue": 104, + "avoided_harm": 50, + "selected_both_correct": 3374, + "selected_both_wrong": 231 + }, + { + "budget": 0.77, + "threshold": -0.999184787273407, + "operator": "gt", + "actual_fraction": 0.7698367230020051, + "metric": 0.8977370380979662, + "rescued": 1350, + "harmed": 358, + "missed_rescue": 99, + "avoided_harm": 48, + "selected_both_correct": 3435, + "selected_both_wrong": 232 + }, + { + "budget": 0.78, + "threshold": -0.9992268085479736, + "operator": "ge", + "actual_fraction": 0.7798625035806359, + "metric": 0.8987396161558293, + "rescued": 1359, + "harmed": 360, + "missed_rescue": 90, + "avoided_harm": 46, + "selected_both_correct": 3492, + "selected_both_wrong": 234 + }, + { + "budget": 0.79, + "threshold": -0.9992831349372864, + "operator": "ge", + "actual_fraction": 0.7898882841592667, + "metric": 0.8997421942136924, + "rescued": 1370, + "harmed": 364, + "missed_rescue": 79, + "avoided_harm": 42, + "selected_both_correct": 3545, + "selected_both_wrong": 236 + }, + { + "budget": 0.8, + "threshold": -0.9993318915367126, + "operator": "ge", + "actual_fraction": 0.7999140647378975, + "metric": 0.8997421942136924, + "rescued": 1372, + "harmed": 366, + "missed_rescue": 77, + "avoided_harm": 40, + "selected_both_correct": 3611, + "selected_both_wrong": 236 + }, + { + "budget": 0.81, + "threshold": -0.9993773102760315, + "operator": "ge", + "actual_fraction": 0.8099398453165282, + "metric": 0.9000286450873675, + "rescued": 1376, + "harmed": 368, + "missed_rescue": 73, + "avoided_harm": 38, + "selected_both_correct": 3674, + "selected_both_wrong": 237 + }, + { + "budget": 0.82, + "threshold": -0.9994175434112549, + "operator": "ge", + "actual_fraction": 0.819965625895159, + "metric": 0.9003150959610426, + "rescued": 1381, + "harmed": 371, + "missed_rescue": 68, + "avoided_harm": 35, + "selected_both_correct": 3734, + "selected_both_wrong": 239 + }, + { + "budget": 0.83, + "threshold": -0.999452531337738, + "operator": "ge", + "actual_fraction": 0.8299914064737898, + "metric": 0.9003150959610426, + "rescued": 1383, + "harmed": 373, + "missed_rescue": 66, + "avoided_harm": 33, + "selected_both_correct": 3798, + "selected_both_wrong": 241 + }, + { + "budget": 0.84, + "threshold": -0.9994945526123047, + "operator": "ge", + "actual_fraction": 0.8398739616155829, + "metric": 0.9011744485820682, + "rescued": 1389, + "harmed": 373, + "missed_rescue": 60, + "avoided_harm": 33, + "selected_both_correct": 3861, + "selected_both_wrong": 241 + }, + { + "budget": 0.85, + "threshold": -0.9995274543762207, + "operator": "gt", + "actual_fraction": 0.8497565167573761, + "metric": 0.9016041248925809, + "rescued": 1394, + "harmed": 375, + "missed_rescue": 55, + "avoided_harm": 31, + "selected_both_correct": 3922, + "selected_both_wrong": 242 + }, + { + "budget": 0.86, + "threshold": -0.9995617270469666, + "operator": "ge", + "actual_fraction": 0.8599255227728445, + "metric": 0.902606702950444, + "rescued": 1401, + "harmed": 375, + "missed_rescue": 48, + "avoided_harm": 31, + "selected_both_correct": 3985, + "selected_both_wrong": 243 + }, + { + "budget": 0.87, + "threshold": -0.9995949864387512, + "operator": "ge", + "actual_fraction": 0.8699513033514752, + "metric": 0.9027499283872816, + "rescued": 1407, + "harmed": 380, + "missed_rescue": 42, + "avoided_harm": 26, + "selected_both_correct": 4043, + "selected_both_wrong": 244 + }, + { + "budget": 0.88, + "threshold": -0.9996248483657837, + "operator": "ge", + "actual_fraction": 0.879977083930106, + "metric": 0.9031796046977943, + "rescued": 1412, + "harmed": 382, + "missed_rescue": 37, + "avoided_harm": 24, + "selected_both_correct": 4106, + "selected_both_wrong": 244 + }, + { + "budget": 0.89, + "threshold": -0.9996511936187744, + "operator": "ge", + "actual_fraction": 0.8898596390718991, + "metric": 0.9034660555714695, + "rescued": 1419, + "harmed": 387, + "missed_rescue": 30, + "avoided_harm": 19, + "selected_both_correct": 4161, + "selected_both_wrong": 246 + }, + { + "budget": 0.9, + "threshold": -0.9996830224990845, + "operator": "ge", + "actual_fraction": 0.8998854196505299, + "metric": 0.9038957318819822, + "rescued": 1423, + "harmed": 388, + "missed_rescue": 26, + "avoided_harm": 18, + "selected_both_correct": 4225, + "selected_both_wrong": 247 + }, + { + "budget": 0.91, + "threshold": -0.9997097849845886, + "operator": "gt", + "actual_fraction": 0.9097679747923231, + "metric": 0.9046118590661701, + "rescued": 1431, + "harmed": 391, + "missed_rescue": 18, + "avoided_harm": 15, + "selected_both_correct": 4283, + "selected_both_wrong": 247 + }, + { + "budget": 0.92, + "threshold": -0.9997391104698181, + "operator": "ge", + "actual_fraction": 0.9199369808077915, + "metric": 0.9050415353766829, + "rescued": 1434, + "harmed": 391, + "missed_rescue": 15, + "avoided_harm": 15, + "selected_both_correct": 4350, + "selected_both_wrong": 248 + }, + { + "budget": 0.93, + "threshold": -0.9997641444206238, + "operator": "ge", + "actual_fraction": 0.9299627613864222, + "metric": 0.9051847608135205, + "rescued": 1437, + "harmed": 393, + "missed_rescue": 12, + "avoided_harm": 13, + "selected_both_correct": 4411, + "selected_both_wrong": 252 + }, + { + "budget": 0.94, + "threshold": -0.9997876286506653, + "operator": "ge", + "actual_fraction": 0.939988541965053, + "metric": 0.905327986250358, + "rescued": 1439, + "harmed": 394, + "missed_rescue": 10, + "avoided_harm": 12, + "selected_both_correct": 4476, + "selected_both_wrong": 254 + }, + { + "budget": 0.95, + "threshold": -0.9998131394386292, + "operator": "ge", + "actual_fraction": 0.9498710971068461, + "metric": 0.9056144371240332, + "rescued": 1442, + "harmed": 395, + "missed_rescue": 7, + "avoided_harm": 11, + "selected_both_correct": 4540, + "selected_both_wrong": 255 + }, + { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9598968776854769, + "metric": 0.9059008879977084, + "rescued": 1446, + "harmed": 397, + "missed_rescue": 3, + "avoided_harm": 9, + "selected_both_correct": 4603, + "selected_both_wrong": 256 + }, + { + "budget": 0.97, + "threshold": -0.999868631362915, + "operator": "ge", + "actual_fraction": 0.9699226582641077, + "metric": 0.9054712116871957, + "rescued": 1446, + "harmed": 400, + "missed_rescue": 3, + "avoided_harm": 6, + "selected_both_correct": 4670, + "selected_both_wrong": 256 + }, + { + "budget": 0.98, + "threshold": -0.9998925924301147, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.9051847608135205, + "rescued": 1446, + "harmed": 402, + "missed_rescue": 3, + "avoided_harm": 4, + "selected_both_correct": 4738, + "selected_both_wrong": 256 + }, + { + "budget": 0.99, + "threshold": -0.99992835521698, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.9047550845030078, + "rescued": 1446, + "harmed": 405, + "missed_rescue": 3, + "avoided_harm": 1, + "selected_both_correct": 4805, + "selected_both_wrong": 256 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9050415353766829, + "rescued": 1449, + "harmed": 406, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 4870, + "selected_both_wrong": 257 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7642247255506924, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2791, + "avoided_harm": 890, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.5127503275871277, + "operator": "ge", + "actual_fraction": 0.007749156920427639, + "metric": 0.7660185118748655, + "rescued": 36, + "harmed": 11, + "missed_rescue": 2755, + "avoided_harm": 879, + "selected_both_correct": 57, + "selected_both_wrong": 4 + }, + { + "budget": 0.02, + "threshold": -0.5296345353126526, + "operator": "ge", + "actual_fraction": 0.017650857429862953, + "metric": 0.7696778359761786, + "rescued": 96, + "harmed": 20, + "missed_rescue": 2695, + "avoided_harm": 870, + "selected_both_correct": 120, + "selected_both_wrong": 10 + }, + { + "budget": 0.03, + "threshold": -0.5434877276420593, + "operator": "ge", + "actual_fraction": 0.025113008538422903, + "metric": 0.7726196455478224, + "rescued": 144, + "harmed": 27, + "missed_rescue": 2647, + "avoided_harm": 863, + "selected_both_correct": 161, + "selected_both_wrong": 18 + }, + { + "budget": 0.04, + "threshold": -0.56125408411026, + "operator": "ge", + "actual_fraction": 0.0349429575948913, + "metric": 0.776565975461003, + "rescued": 205, + "harmed": 33, + "missed_rescue": 2586, + "avoided_harm": 857, + "selected_both_correct": 226, + "selected_both_wrong": 23 + }, + { + "budget": 0.05, + "threshold": -0.5779195427894592, + "operator": "ge", + "actual_fraction": 0.043840137762789694, + "metric": 0.7785750161440769, + "rescued": 242, + "harmed": 42, + "missed_rescue": 2549, + "avoided_harm": 848, + "selected_both_correct": 291, + "selected_both_wrong": 36 + }, + { + "budget": 0.06, + "threshold": -0.5911363363265991, + "operator": "ge", + "actual_fraction": 0.05187630049508503, + "metric": 0.7809428140919854, + "rescued": 286, + "harmed": 53, + "missed_rescue": 2505, + "avoided_harm": 837, + "selected_both_correct": 345, + "selected_both_wrong": 39 + }, + { + "budget": 0.07, + "threshold": -0.6087464094161987, + "operator": "ge", + "actual_fraction": 0.06177800100452034, + "metric": 0.7835976178517615, + "rescued": 336, + "harmed": 66, + "missed_rescue": 2455, + "avoided_harm": 824, + "selected_both_correct": 409, + "selected_both_wrong": 50 + }, + { + "budget": 0.08, + "threshold": -0.6274068355560303, + "operator": "ge", + "actual_fraction": 0.07132094424912104, + "metric": 0.7864676759704384, + "rescued": 387, + "harmed": 77, + "missed_rescue": 2404, + "avoided_harm": 813, + "selected_both_correct": 473, + "selected_both_wrong": 57 + }, + { + "budget": 0.09, + "threshold": -0.6465112566947937, + "operator": "ge", + "actual_fraction": 0.08251417091196096, + "metric": 0.7888354739183469, + "rescued": 439, + "harmed": 96, + "missed_rescue": 2352, + "avoided_harm": 794, + "selected_both_correct": 546, + "selected_both_wrong": 69 + }, + { + "budget": 0.1, + "threshold": -0.6620025038719177, + "operator": "ge", + "actual_fraction": 0.09090909090909091, + "metric": 0.7910597689603215, + "rescued": 483, + "harmed": 109, + "missed_rescue": 2308, + "avoided_harm": 781, + "selected_both_correct": 601, + "selected_both_wrong": 74 + }, + { + "budget": 0.11, + "threshold": -0.6800539493560791, + "operator": "ge", + "actual_fraction": 0.10030853124775777, + "metric": 0.7934275669082299, + "rescued": 529, + "harmed": 122, + "missed_rescue": 2262, + "avoided_harm": 768, + "selected_both_correct": 661, + "selected_both_wrong": 86 + }, + { + "budget": 0.12, + "threshold": -0.6957284212112427, + "operator": "ge", + "actual_fraction": 0.11078424338092846, + "metric": 0.7971586424625099, + "rescued": 592, + "harmed": 133, + "missed_rescue": 2199, + "avoided_harm": 757, + "selected_both_correct": 723, + "selected_both_wrong": 96 + }, + { + "budget": 0.13, + "threshold": -0.7128273248672485, + "operator": "ge", + "actual_fraction": 0.12075769534333071, + "metric": 0.7997416947693191, + "rescued": 640, + "harmed": 145, + "missed_rescue": 2151, + "avoided_harm": 745, + "selected_both_correct": 792, + "selected_both_wrong": 106 + }, + { + "budget": 0.14, + "threshold": -0.7332167625427246, + "operator": "ge", + "actual_fraction": 0.13431871995407907, + "metric": 0.8044055392121691, + "rescued": 720, + "harmed": 160, + "missed_rescue": 2071, + "avoided_harm": 730, + "selected_both_correct": 870, + "selected_both_wrong": 122 + }, + { + "budget": 0.15, + "threshold": -0.7503995895385742, + "operator": "ge", + "actual_fraction": 0.144579177728349, + "metric": 0.8072038458778791, + "rescued": 769, + "harmed": 170, + "missed_rescue": 2022, + "avoided_harm": 720, + "selected_both_correct": 947, + "selected_both_wrong": 129 + }, + { + "budget": 0.16, + "threshold": -0.7655540704727173, + "operator": "ge", + "actual_fraction": 0.1534763578962474, + "metric": 0.8099304010906221, + "rescued": 817, + "harmed": 180, + "missed_rescue": 1974, + "avoided_harm": 710, + "selected_both_correct": 1002, + "selected_both_wrong": 140 + }, + { + "budget": 0.17, + "threshold": -0.7812193036079407, + "operator": "ge", + "actual_fraction": 0.16280404678194735, + "metric": 0.8125852048503982, + "rescued": 868, + "harmed": 194, + "missed_rescue": 1923, + "avoided_harm": 696, + "selected_both_correct": 1059, + "selected_both_wrong": 148 + }, + { + "budget": 0.18, + "threshold": -0.7954917550086975, + "operator": "ge", + "actual_fraction": 0.17306450455621727, + "metric": 0.8159575231398436, + "rescued": 927, + "harmed": 206, + "missed_rescue": 1864, + "avoided_harm": 684, + "selected_both_correct": 1124, + "selected_both_wrong": 155 + }, + { + "budget": 0.19, + "threshold": -0.8097342252731323, + "operator": "ge", + "actual_fraction": 0.18196168472411567, + "metric": 0.8179665638229174, + "rescued": 966, + "harmed": 217, + "missed_rescue": 1825, + "avoided_harm": 673, + "selected_both_correct": 1191, + "selected_both_wrong": 162 + }, + { + "budget": 0.2, + "threshold": -0.8232117891311646, + "operator": "ge", + "actual_fraction": 0.1928679055750879, + "metric": 0.820262610317859, + "rescued": 1011, + "harmed": 230, + "missed_rescue": 1780, + "avoided_harm": 660, + "selected_both_correct": 1276, + "selected_both_wrong": 171 + }, + { + "budget": 0.21, + "threshold": -0.8359466791152954, + "operator": "ge", + "actual_fraction": 0.20104757121331707, + "metric": 0.8221998995479658, + "rescued": 1049, + "harmed": 241, + "missed_rescue": 1742, + "avoided_harm": 649, + "selected_both_correct": 1333, + "selected_both_wrong": 179 + }, + { + "budget": 0.22, + "threshold": -0.8473842144012451, + "operator": "ge", + "actual_fraction": 0.21051876300495084, + "metric": 0.8254287149314774, + "rescued": 1098, + "harmed": 245, + "missed_rescue": 1693, + "avoided_harm": 645, + "selected_both_correct": 1403, + "selected_both_wrong": 188 + }, + { + "budget": 0.23, + "threshold": -0.8595863580703735, + "operator": "ge", + "actual_fraction": 0.21998995479658462, + "metric": 0.8272225012556504, + "rescued": 1135, + "harmed": 257, + "missed_rescue": 1656, + "avoided_harm": 633, + "selected_both_correct": 1478, + "selected_both_wrong": 196 + }, + { + "budget": 0.24, + "threshold": -0.8704345226287842, + "operator": "ge", + "actual_fraction": 0.22838487479371458, + "metric": 0.8280117672382866, + "rescued": 1161, + "harmed": 272, + "missed_rescue": 1630, + "avoided_harm": 618, + "selected_both_correct": 1544, + "selected_both_wrong": 206 + }, + { + "budget": 0.25, + "threshold": -0.8811548352241516, + "operator": "ge", + "actual_fraction": 0.23900408983281912, + "metric": 0.8303078137332282, + "rescued": 1207, + "harmed": 286, + "missed_rescue": 1584, + "avoided_harm": 604, + "selected_both_correct": 1623, + "selected_both_wrong": 215 + }, + { + "budget": 0.26, + "threshold": -0.8929013609886169, + "operator": "ge", + "actual_fraction": 0.2506995766664275, + "metric": 0.8333931262108057, + "rescued": 1260, + "harmed": 296, + "missed_rescue": 1531, + "avoided_harm": 594, + "selected_both_correct": 1712, + "selected_both_wrong": 226 + }, + { + "budget": 0.27, + "threshold": -0.903549313545227, + "operator": "ge", + "actual_fraction": 0.2646911099949774, + "metric": 0.8365501901413503, + "rescued": 1318, + "harmed": 310, + "missed_rescue": 1473, + "avoided_harm": 580, + "selected_both_correct": 1828, + "selected_both_wrong": 233 + }, + { + "budget": 0.28, + "threshold": -0.9115740656852722, + "operator": "ge", + "actual_fraction": 0.27437755614551196, + "metric": 0.8389897395422258, + "rescued": 1366, + "harmed": 324, + "missed_rescue": 1425, + "avoided_harm": 566, + "selected_both_correct": 1899, + "selected_both_wrong": 235 + }, + { + "budget": 0.29, + "threshold": -0.9182595014572144, + "operator": "ge", + "actual_fraction": 0.2819832101600057, + "metric": 0.8409987802252996, + "rescued": 1401, + "harmed": 331, + "missed_rescue": 1390, + "avoided_harm": 559, + "selected_both_correct": 1955, + "selected_both_wrong": 243 + }, + { + "budget": 0.3, + "threshold": -0.9258768558502197, + "operator": "ge", + "actual_fraction": 0.2907368874219703, + "metric": 0.8434383296261749, + "rescued": 1446, + "harmed": 342, + "missed_rescue": 1345, + "avoided_harm": 548, + "selected_both_correct": 2015, + "selected_both_wrong": 249 + }, + { + "budget": 0.31, + "threshold": -0.9331877827644348, + "operator": "ge", + "actual_fraction": 0.302145368443711, + "metric": 0.8450168615914472, + "rescued": 1488, + "harmed": 362, + "missed_rescue": 1303, + "avoided_harm": 528, + "selected_both_correct": 2106, + "selected_both_wrong": 255 + }, + { + "budget": 0.32, + "threshold": -0.9399217367172241, + "operator": "ge", + "actual_fraction": 0.31412786108918705, + "metric": 0.847025902274521, + "rescued": 1531, + "harmed": 377, + "missed_rescue": 1260, + "avoided_harm": 513, + "selected_both_correct": 2209, + "selected_both_wrong": 261 + }, + { + "budget": 0.33, + "threshold": -0.9453923106193542, + "operator": "ge", + "actual_fraction": 0.3238143072397216, + "metric": 0.8486761856927603, + "rescued": 1564, + "harmed": 387, + "missed_rescue": 1227, + "avoided_harm": 503, + "selected_both_correct": 2293, + "selected_both_wrong": 269 + }, + { + "budget": 0.34, + "threshold": -0.9515207409858704, + "operator": "ge", + "actual_fraction": 0.33529453971442924, + "metric": 0.8514744923584703, + "rescued": 1615, + "harmed": 399, + "missed_rescue": 1176, + "avoided_harm": 491, + "selected_both_correct": 2385, + "selected_both_wrong": 274 + }, + { + "budget": 0.35, + "threshold": -0.9567410349845886, + "operator": "ge", + "actual_fraction": 0.3452679916768315, + "metric": 0.8531247757767095, + "rescued": 1654, + "harmed": 415, + "missed_rescue": 1137, + "avoided_harm": 475, + "selected_both_correct": 2465, + "selected_both_wrong": 278 + }, + { + "budget": 0.36, + "threshold": -0.9606627225875854, + "operator": "ge", + "actual_fraction": 0.35595895816890294, + "metric": 0.855420822271651, + "rescued": 1697, + "harmed": 426, + "missed_rescue": 1094, + "avoided_harm": 464, + "selected_both_correct": 2554, + "selected_both_wrong": 284 + }, + { + "budget": 0.37, + "threshold": -0.9650874733924866, + "operator": "ge", + "actual_fraction": 0.36837195953218055, + "metric": 0.8573581115017579, + "rescued": 1745, + "harmed": 447, + "missed_rescue": 1046, + "avoided_harm": 443, + "selected_both_correct": 2651, + "selected_both_wrong": 291 + }, + { + "budget": 0.38, + "threshold": -0.9686882495880127, + "operator": "ge", + "actual_fraction": 0.3797086891009543, + "metric": 0.8592954007318648, + "rescued": 1785, + "harmed": 460, + "missed_rescue": 1006, + "avoided_harm": 430, + "selected_both_correct": 2747, + "selected_both_wrong": 300 + }, + { + "budget": 0.39, + "threshold": -0.971785306930542, + "operator": "ge", + "actual_fraction": 0.391117170122695, + "metric": 0.8620219559446078, + "rescued": 1832, + "harmed": 469, + "missed_rescue": 959, + "avoided_harm": 421, + "selected_both_correct": 2845, + "selected_both_wrong": 305 + }, + { + "budget": 0.4, + "threshold": -0.9748144745826721, + "operator": "ge", + "actual_fraction": 0.40231039678553493, + "metric": 0.8641744995336156, + "rescued": 1873, + "harmed": 480, + "missed_rescue": 918, + "avoided_harm": 410, + "selected_both_correct": 2945, + "selected_both_wrong": 309 + }, + { + "budget": 0.41, + "threshold": -0.977255642414093, + "operator": "ge", + "actual_fraction": 0.41235560020090406, + "metric": 0.8662552916696563, + "rescued": 1912, + "harmed": 490, + "missed_rescue": 879, + "avoided_harm": 400, + "selected_both_correct": 3027, + "selected_both_wrong": 318 + }, + { + "budget": 0.42, + "threshold": -0.9794421792030334, + "operator": "ge", + "actual_fraction": 0.4231183181459425, + "metric": 0.8680490779938294, + "rescued": 1950, + "harmed": 503, + "missed_rescue": 841, + "avoided_harm": 387, + "selected_both_correct": 3119, + "selected_both_wrong": 325 + }, + { + "budget": 0.43, + "threshold": -0.981472373008728, + "operator": "ge", + "actual_fraction": 0.43172849250197315, + "metric": 0.869268852694267, + "rescued": 1972, + "harmed": 508, + "missed_rescue": 819, + "avoided_harm": 382, + "selected_both_correct": 3207, + "selected_both_wrong": 330 + }, + { + "budget": 0.44, + "threshold": -0.9835578799247742, + "operator": "ge", + "actual_fraction": 0.4432804764296477, + "metric": 0.8709191361125063, + "rescued": 2005, + "harmed": 518, + "missed_rescue": 786, + "avoided_harm": 372, + "selected_both_correct": 3320, + "selected_both_wrong": 335 + }, + { + "budget": 0.45, + "threshold": -0.9851564168930054, + "operator": "ge", + "actual_fraction": 0.4547607089043553, + "metric": 0.8727129224366793, + "rescued": 2042, + "harmed": 530, + "missed_rescue": 749, + "avoided_harm": 360, + "selected_both_correct": 3422, + "selected_both_wrong": 344 + }, + { + "budget": 0.46, + "threshold": -0.986689031124115, + "operator": "ge", + "actual_fraction": 0.4649494152256583, + "metric": 0.8740762000430509, + "rescued": 2074, + "harmed": 543, + "missed_rescue": 717, + "avoided_harm": 347, + "selected_both_correct": 3513, + "selected_both_wrong": 350 + }, + { + "budget": 0.47, + "threshold": -0.9879001379013062, + "operator": "ge", + "actual_fraction": 0.47399009829949057, + "metric": 0.8749372174786539, + "rescued": 2096, + "harmed": 553, + "missed_rescue": 695, + "avoided_harm": 337, + "selected_both_correct": 3603, + "selected_both_wrong": 354 + }, + { + "budget": 0.48, + "threshold": -0.9891784191131592, + "operator": "ge", + "actual_fraction": 0.48446581043266124, + "metric": 0.8764439979909593, + "rescued": 2126, + "harmed": 562, + "missed_rescue": 665, + "avoided_harm": 328, + "selected_both_correct": 3704, + "selected_both_wrong": 360 + }, + { + "budget": 0.49, + "threshold": -0.990114688873291, + "operator": "ge", + "actual_fraction": 0.491210447011552, + "metric": 0.8771615125206286, + "rescued": 2145, + "harmed": 571, + "missed_rescue": 646, + "avoided_harm": 319, + "selected_both_correct": 3767, + "selected_both_wrong": 363 + }, + { + "budget": 0.5, + "threshold": -0.9910277724266052, + "operator": "ge", + "actual_fraction": 0.5008251417091196, + "metric": 0.8780942814091985, + "rescued": 2168, + "harmed": 581, + "missed_rescue": 623, + "avoided_harm": 309, + "selected_both_correct": 3859, + "selected_both_wrong": 372 + }, + { + "budget": 0.51, + "threshold": -0.9918718934059143, + "operator": "ge", + "actual_fraction": 0.5107268422185549, + "metric": 0.878596541579967, + "rescued": 2183, + "harmed": 589, + "missed_rescue": 608, + "avoided_harm": 301, + "selected_both_correct": 3969, + "selected_both_wrong": 377 + }, + { + "budget": 0.52, + "threshold": -0.9926809668540955, + "operator": "ge", + "actual_fraction": 0.5217048145224941, + "metric": 0.8796010619215039, + "rescued": 2208, + "harmed": 600, + "missed_rescue": 583, + "avoided_harm": 290, + "selected_both_correct": 4083, + "selected_both_wrong": 380 + }, + { + "budget": 0.53, + "threshold": -0.9932646751403809, + "operator": "ge", + "actual_fraction": 0.5306737461433594, + "metric": 0.8807490851689747, + "rescued": 2231, + "harmed": 607, + "missed_rescue": 560, + "avoided_harm": 283, + "selected_both_correct": 4172, + "selected_both_wrong": 386 + }, + { + "budget": 0.54, + "threshold": -0.9938417673110962, + "operator": "ge", + "actual_fraction": 0.5398579321231255, + "metric": 0.8816101026045777, + "rescued": 2253, + "harmed": 617, + "missed_rescue": 538, + "avoided_harm": 273, + "selected_both_correct": 4267, + "selected_both_wrong": 387 + }, + { + "budget": 0.55, + "threshold": -0.9944412112236023, + "operator": "ge", + "actual_fraction": 0.5504771471622301, + "metric": 0.882327617134247, + "rescued": 2273, + "harmed": 627, + "missed_rescue": 518, + "avoided_harm": 263, + "selected_both_correct": 4379, + "selected_both_wrong": 393 + }, + { + "budget": 0.56, + "threshold": -0.9949513673782349, + "operator": "ge", + "actual_fraction": 0.5604505991246322, + "metric": 0.8834756403817178, + "rescued": 2298, + "harmed": 636, + "missed_rescue": 493, + "avoided_harm": 254, + "selected_both_correct": 4479, + "selected_both_wrong": 398 + }, + { + "budget": 0.57, + "threshold": -0.9954326152801514, + "operator": "ge", + "actual_fraction": 0.570926311257803, + "metric": 0.8845519121762215, + "rescued": 2321, + "harmed": 644, + "missed_rescue": 470, + "avoided_harm": 246, + "selected_both_correct": 4592, + "selected_both_wrong": 400 + }, + { + "budget": 0.58, + "threshold": -0.9958608746528625, + "operator": "ge", + "actual_fraction": 0.5816172777498745, + "metric": 0.8858434383296262, + "rescued": 2349, + "harmed": 654, + "missed_rescue": 442, + "avoided_harm": 236, + "selected_both_correct": 4698, + "selected_both_wrong": 405 + }, + { + "budget": 0.59, + "threshold": -0.9962128400802612, + "operator": "ge", + "actual_fraction": 0.5920929898830452, + "metric": 0.8866327043122624, + "rescued": 2374, + "harmed": 668, + "missed_rescue": 417, + "avoided_harm": 222, + "selected_both_correct": 4804, + "selected_both_wrong": 406 + }, + { + "budget": 0.6, + "threshold": -0.9965764880180359, + "operator": "ge", + "actual_fraction": 0.6034297194518189, + "metric": 0.8874219702948984, + "rescued": 2395, + "harmed": 678, + "missed_rescue": 396, + "avoided_harm": 212, + "selected_both_correct": 4930, + "selected_both_wrong": 407 + }, + { + "budget": 0.61, + "threshold": -0.9968799352645874, + "operator": "ge", + "actual_fraction": 0.6144794432087249, + "metric": 0.8879242304656669, + "rescued": 2415, + "harmed": 691, + "missed_rescue": 376, + "avoided_harm": 199, + "selected_both_correct": 5048, + "selected_both_wrong": 410 + }, + { + "budget": 0.62, + "threshold": -0.9971383810043335, + "operator": "ge", + "actual_fraction": 0.6232331204706896, + "metric": 0.8888569993542369, + "rescued": 2435, + "harmed": 698, + "missed_rescue": 356, + "avoided_harm": 192, + "selected_both_correct": 5140, + "selected_both_wrong": 413 + }, + { + "budget": 0.63, + "threshold": -0.997377872467041, + "operator": "ge", + "actual_fraction": 0.6337805840568271, + "metric": 0.8894310109779723, + "rescued": 2451, + "harmed": 706, + "missed_rescue": 340, + "avoided_harm": 184, + "selected_both_correct": 5258, + "selected_both_wrong": 418 + }, + { + "budget": 0.64, + "threshold": -0.9976021647453308, + "operator": "ge", + "actual_fraction": 0.6445433020018655, + "metric": 0.8904355313195093, + "rescued": 2472, + "harmed": 713, + "missed_rescue": 319, + "avoided_harm": 177, + "selected_both_correct": 5375, + "selected_both_wrong": 423 + }, + { + "budget": 0.65, + "threshold": -0.9978214502334595, + "operator": "ge", + "actual_fraction": 0.6565257946473416, + "metric": 0.8904355313195093, + "rescued": 2488, + "harmed": 729, + "missed_rescue": 303, + "avoided_harm": 161, + "selected_both_correct": 5508, + "selected_both_wrong": 425 + }, + { + "budget": 0.66, + "threshold": -0.9980514049530029, + "operator": "ge", + "actual_fraction": 0.6692258018224869, + "metric": 0.8909377914902776, + "rescued": 2507, + "harmed": 741, + "missed_rescue": 284, + "avoided_harm": 149, + "selected_both_correct": 5650, + "selected_both_wrong": 429 + }, + { + "budget": 0.67, + "threshold": -0.9982016086578369, + "operator": "ge", + "actual_fraction": 0.6777642247255506, + "metric": 0.8917988089258807, + "rescued": 2523, + "harmed": 745, + "missed_rescue": 268, + "avoided_harm": 145, + "selected_both_correct": 5748, + "selected_both_wrong": 430 + }, + { + "budget": 0.68, + "threshold": -0.9983693957328796, + "operator": "ge", + "actual_fraction": 0.6885269426705891, + "metric": 0.8920858147377484, + "rescued": 2541, + "harmed": 759, + "missed_rescue": 250, + "avoided_harm": 131, + "selected_both_correct": 5864, + "selected_both_wrong": 432 + }, + { + "budget": 0.69, + "threshold": -0.9984915256500244, + "operator": "ge", + "actual_fraction": 0.6977828801033221, + "metric": 0.89251632345555, + "rescued": 2552, + "harmed": 764, + "missed_rescue": 239, + "avoided_harm": 126, + "selected_both_correct": 5973, + "selected_both_wrong": 436 + }, + { + "budget": 0.7, + "threshold": -0.9986109733581543, + "operator": "gt", + "actual_fraction": 0.7064648059123197, + "metric": 0.8928750807203846, + "rescued": 2563, + "harmed": 770, + "missed_rescue": 228, + "avoided_harm": 120, + "selected_both_correct": 6077, + "selected_both_wrong": 436 + }, + { + "budget": 0.71, + "threshold": -0.9986966252326965, + "operator": "ge", + "actual_fraction": 0.7137117026619789, + "metric": 0.8933055894381862, + "rescued": 2575, + "harmed": 776, + "missed_rescue": 216, + "avoided_harm": 114, + "selected_both_correct": 6154, + "selected_both_wrong": 442 + }, + { + "budget": 0.72, + "threshold": -0.9987963438034058, + "operator": "ge", + "actual_fraction": 0.7256941953074549, + "metric": 0.8941666068737892, + "rescued": 2592, + "harmed": 781, + "missed_rescue": 199, + "avoided_harm": 109, + "selected_both_correct": 6295, + "selected_both_wrong": 446 + }, + { + "budget": 0.73, + "threshold": -0.998875081539154, + "operator": "ge", + "actual_fraction": 0.7345196240223865, + "metric": 0.8950993757623592, + "rescued": 2605, + "harmed": 781, + "missed_rescue": 186, + "avoided_harm": 109, + "selected_both_correct": 6403, + "selected_both_wrong": 448 + }, + { + "budget": 0.74, + "threshold": -0.9989551305770874, + "operator": "ge", + "actual_fraction": 0.745067087608524, + "metric": 0.8956733873860946, + "rescued": 2618, + "harmed": 786, + "missed_rescue": 173, + "avoided_harm": 104, + "selected_both_correct": 6530, + "selected_both_wrong": 450 + }, + { + "budget": 0.75, + "threshold": -0.9990480542182922, + "operator": "ge", + "actual_fraction": 0.7568343258950994, + "metric": 0.8963191504627969, + "rescued": 2632, + "harmed": 791, + "missed_rescue": 159, + "avoided_harm": 99, + "selected_both_correct": 6671, + "selected_both_wrong": 454 + }, + { + "budget": 0.76, + "threshold": -0.999126136302948, + "operator": "ge", + "actual_fraction": 0.7663772691397001, + "metric": 0.8967496591805985, + "rescued": 2644, + "harmed": 797, + "missed_rescue": 147, + "avoided_harm": 93, + "selected_both_correct": 6783, + "selected_both_wrong": 457 + }, + { + "budget": 0.77, + "threshold": -0.999184787273407, + "operator": "gt", + "actual_fraction": 0.7746286862308962, + "metric": 0.8969649135394991, + "rescued": 2653, + "harmed": 803, + "missed_rescue": 138, + "avoided_harm": 87, + "selected_both_correct": 6883, + "selected_both_wrong": 457 + }, + { + "budget": 0.78, + "threshold": -0.9992268085479736, + "operator": "ge", + "actual_fraction": 0.7817320800746215, + "metric": 0.8966061562746646, + "rescued": 2657, + "harmed": 812, + "missed_rescue": 134, + "avoided_harm": 78, + "selected_both_correct": 6969, + "selected_both_wrong": 457 + }, + { + "budget": 0.79, + "threshold": -0.9992831349372864, + "operator": "ge", + "actual_fraction": 0.7928535552844945, + "metric": 0.8967496591805985, + "rescued": 2667, + "harmed": 820, + "missed_rescue": 124, + "avoided_harm": 70, + "selected_both_correct": 7106, + "selected_both_wrong": 457 + }, + { + "budget": 0.8, + "threshold": -0.9993318915367126, + "operator": "ge", + "actual_fraction": 0.8023247470761283, + "metric": 0.8966061562746646, + "rescued": 2669, + "harmed": 824, + "missed_rescue": 122, + "avoided_harm": 66, + "selected_both_correct": 7231, + "selected_both_wrong": 458 + }, + { + "budget": 0.81, + "threshold": -0.9993773102760315, + "operator": "ge", + "actual_fraction": 0.811795938867762, + "metric": 0.8968214106335654, + "rescued": 2677, + "harmed": 829, + "missed_rescue": 114, + "avoided_harm": 61, + "selected_both_correct": 7349, + "selected_both_wrong": 459 + }, + { + "budget": 0.82, + "threshold": -0.9994175434112549, + "operator": "ge", + "actual_fraction": 0.8214106335653297, + "metric": 0.8973954222573007, + "rescued": 2689, + "harmed": 833, + "missed_rescue": 102, + "avoided_harm": 57, + "selected_both_correct": 7465, + "selected_both_wrong": 461 + }, + { + "budget": 0.83, + "threshold": -0.999452531337738, + "operator": "ge", + "actual_fraction": 0.8298773050154266, + "metric": 0.8976824280691684, + "rescued": 2696, + "harmed": 836, + "missed_rescue": 95, + "avoided_harm": 54, + "selected_both_correct": 7572, + "selected_both_wrong": 462 + }, + { + "budget": 0.84, + "threshold": -0.9994945526123047, + "operator": "ge", + "actual_fraction": 0.8420750520198034, + "metric": 0.8973954222573007, + "rescued": 2702, + "harmed": 846, + "missed_rescue": 89, + "avoided_harm": 44, + "selected_both_correct": 7725, + "selected_both_wrong": 463 + }, + { + "budget": 0.85, + "threshold": -0.9995274543762207, + "operator": "gt", + "actual_fraction": 0.8502547176580326, + "metric": 0.8976106766162015, + "rescued": 2707, + "harmed": 848, + "missed_rescue": 84, + "avoided_harm": 42, + "selected_both_correct": 7829, + "selected_both_wrong": 466 + }, + { + "budget": 0.86, + "threshold": -0.9995617270469666, + "operator": "ge", + "actual_fraction": 0.8604434239793356, + "metric": 0.8976824280691684, + "rescued": 2712, + "harmed": 852, + "missed_rescue": 79, + "avoided_harm": 38, + "selected_both_correct": 7959, + "selected_both_wrong": 469 + }, + { + "budget": 0.87, + "threshold": -0.9995949864387512, + "operator": "ge", + "actual_fraction": 0.8690535983353663, + "metric": 0.8978259309751022, + "rescued": 2717, + "harmed": 855, + "missed_rescue": 74, + "avoided_harm": 35, + "selected_both_correct": 8069, + "selected_both_wrong": 471 + }, + { + "budget": 0.88, + "threshold": -0.9996248483657837, + "operator": "ge", + "actual_fraction": 0.8778790270502977, + "metric": 0.8978976824280692, + "rescued": 2723, + "harmed": 860, + "missed_rescue": 68, + "avoided_harm": 30, + "selected_both_correct": 8180, + "selected_both_wrong": 472 + }, + { + "budget": 0.89, + "threshold": -0.9996511936187744, + "operator": "ge", + "actual_fraction": 0.886058692688527, + "metric": 0.8981129367869699, + "rescued": 2731, + "harmed": 865, + "missed_rescue": 60, + "avoided_harm": 25, + "selected_both_correct": 8279, + "selected_both_wrong": 474 + }, + { + "budget": 0.9, + "threshold": -0.9996830224990845, + "operator": "ge", + "actual_fraction": 0.8961756475568631, + "metric": 0.8986869484107053, + "rescued": 2740, + "harmed": 866, + "missed_rescue": 51, + "avoided_harm": 24, + "selected_both_correct": 8407, + "selected_both_wrong": 477 + }, + { + "budget": 0.91, + "threshold": -0.9997097849845886, + "operator": "gt", + "actual_fraction": 0.9053598335366291, + "metric": 0.8990457056755399, + "rescued": 2746, + "harmed": 867, + "missed_rescue": 45, + "avoided_harm": 23, + "selected_both_correct": 8528, + "selected_both_wrong": 477 + }, + { + "budget": 0.92, + "threshold": -0.9997391104698181, + "operator": "ge", + "actual_fraction": 0.9164813087465021, + "metric": 0.8992609600344407, + "rescued": 2752, + "harmed": 870, + "missed_rescue": 39, + "avoided_harm": 20, + "selected_both_correct": 8672, + "selected_both_wrong": 479 + }, + { + "budget": 0.93, + "threshold": -0.9997641444206238, + "operator": "ge", + "actual_fraction": 0.9265265121618713, + "metric": 0.8995479658463084, + "rescued": 2759, + "harmed": 873, + "missed_rescue": 32, + "avoided_harm": 17, + "selected_both_correct": 8801, + "selected_both_wrong": 480 + }, + { + "budget": 0.94, + "threshold": -0.9997876286506653, + "operator": "ge", + "actual_fraction": 0.9362847097653727, + "metric": 0.8997632202052092, + "rescued": 2765, + "harmed": 876, + "missed_rescue": 26, + "avoided_harm": 14, + "selected_both_correct": 8926, + "selected_both_wrong": 482 + }, + { + "budget": 0.95, + "threshold": -0.9998131394386292, + "operator": "ge", + "actual_fraction": 0.9475496878811795, + "metric": 0.899906723111143, + "rescued": 2769, + "harmed": 878, + "missed_rescue": 22, + "avoided_harm": 12, + "selected_both_correct": 9075, + "selected_both_wrong": 484 + }, + { + "budget": 0.96, + "threshold": -0.9998401403427124, + "operator": "ge", + "actual_fraction": 0.9593169261677549, + "metric": 0.8996197172992753, + "rescued": 2770, + "harmed": 883, + "missed_rescue": 21, + "avoided_harm": 7, + "selected_both_correct": 9230, + "selected_both_wrong": 487 + }, + { + "budget": 0.97, + "threshold": -0.999868631362915, + "operator": "ge", + "actual_fraction": 0.9700796441127932, + "metric": 0.89997847456411, + "rescued": 2775, + "harmed": 883, + "missed_rescue": 16, + "avoided_harm": 7, + "selected_both_correct": 9371, + "selected_both_wrong": 491 + }, + { + "budget": 0.98, + "threshold": -0.9998925924301147, + "operator": "ge", + "actual_fraction": 0.9793355815455262, + "metric": 0.9001937289230106, + "rescued": 2780, + "harmed": 885, + "missed_rescue": 11, + "avoided_harm": 5, + "selected_both_correct": 9491, + "selected_both_wrong": 493 + }, + { + "budget": 0.99, + "threshold": -0.99992835521698, + "operator": "ge", + "actual_fraction": 0.9909593169261678, + "metric": 0.9004089832819114, + "rescued": 2786, + "harmed": 888, + "missed_rescue": 5, + "avoided_harm": 2, + "selected_both_correct": 9643, + "selected_both_wrong": 494 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.9006242376408122, + "rescued": 2791, + "harmed": 890, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9761, + "selected_both_wrong": 495 + } + ] + }, + "learned": { + "validation_area_0_25": 0.7964737897450587, + "validation_point": { + "metric": 0.9066170151818964, + "metric_std": 0.0007089297433838111, + "actual_fraction": 0.987969063305643, + "actual_fraction_std": 0.004483665380263076, + "per_seed": [ + { + "seed": 2026, + "budget": 0.99, + "threshold": -0.7896829843521118, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.9054712116871957, + "rescue": 1430, + "harm": 384, + "missed_rescue": 19 + }, + { + "seed": 2027, + "budget": 0.99, + "threshold": -0.6514462232589722, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.9067602406187338, + "rescue": 1436, + "harm": 381, + "missed_rescue": 13 + }, + { + "seed": 2028, + "budget": 0.99, + "threshold": -0.6827153563499451, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.9071899169292467, + "rescue": 1437, + "harm": 379, + "missed_rescue": 12 + }, + { + "seed": 2029, + "budget": 0.98, + "threshold": -0.4852448105812073, + "operator": "ge", + "actual_fraction": 0.9799484388427385, + "metric": 0.9064737897450588, + "rescue": 1416, + "harm": 363, + "missed_rescue": 33 + }, + { + "seed": 2030, + "budget": 0.99, + "threshold": -0.5994459390640259, + "operator": "ge", + "actual_fraction": 0.9899742194213692, + "metric": 0.9071899169292467, + "rescue": 1434, + "harm": 376, + "missed_rescue": 15 + } + ], + "rescued": 1430.6, + "harmed": 376.6, + "missed_rescue": 18.4, + "avoided_harm": 29.399999999999977 + }, + "test_point": { + "metric": 0.9030637870416875, + "metric_std": 0.0007973778708076688, + "actual_fraction": 0.9871995407907009, + "actual_fraction_std": 0.00543220578901596, + "per_seed": [ + { + "seed": 2026, + "budget": 0.99, + "threshold": -0.7896829843521118, + "operator": "ge", + "actual_fraction": 0.9898830451316639, + "metric": 0.9028485326827869, + "rescue": 2761, + "harm": 829, + "missed_rescue": 30 + }, + { + "seed": 2027, + "budget": 0.99, + "threshold": -0.6514462232589722, + "operator": "ge", + "actual_fraction": 0.9890937791490277, + "metric": 0.9034942957594891, + "rescue": 2765, + "harm": 824, + "missed_rescue": 26 + }, + { + "seed": 2028, + "budget": 0.99, + "threshold": -0.6827153563499451, + "operator": "ge", + "actual_fraction": 0.9881610102604578, + "metric": 0.9034225443065222, + "rescue": 2759, + "harm": 819, + "missed_rescue": 32 + }, + { + "seed": 2029, + "budget": 0.98, + "threshold": -0.4852448105812073, + "operator": "ge", + "actual_fraction": 0.9776852981272871, + "metric": 0.9037813015713568, + "rescue": 2713, + "harm": 768, + "missed_rescue": 78 + }, + { + "seed": 2030, + "budget": 0.99, + "threshold": -0.5994459390640259, + "operator": "ge", + "actual_fraction": 0.9911745712850685, + "metric": 0.901772260888283, + "rescue": 2759, + "harm": 842, + "missed_rescue": 32 + } + ], + "rescued": 2751.4, + "harmed": 816.4, + "missed_rescue": 39.6, + "avoided_harm": 73.60000000000002 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.7556574047550845, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.7594958464623317, + "metric_std": 0.0004460730880091119, + "actual_fraction": 0.009882555141793183, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.7635920939558867, + "metric_std": 0.0007345105359471925, + "actual_fraction": 0.01990833572042395, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.7672873102262961, + "metric_std": 0.00021243765359770363, + "actual_fraction": 0.029934116299054715, + "actual_fraction_std": 3.878959614448864e-18 + }, + { + "budget": 0.04, + "metric": 0.770552850186193, + "metric_std": 0.0004857010873048619, + "actual_fraction": 0.039959896877685475, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.7746777427671154, + "metric_std": 0.00034194604372138907, + "actual_fraction": 0.04998567745631624, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.7778000572901747, + "metric_std": 0.0004706868153868366, + "actual_fraction": 0.05986823259810943, + "actual_fraction_std": 7.757919228897728e-18 + }, + { + "budget": 0.07, + "metric": 0.7806359209395588, + "metric_std": 0.0006292146272268397, + "actual_fraction": 0.06989401317674018, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.784073331423661, + "metric_std": 0.0005223312096479592, + "actual_fraction": 0.07991979375537095, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.787911773130908, + "metric_std": 0.000558395551120532, + "actual_fraction": 0.08994557433400172, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.791406473789745, + "metric_std": 0.0003594929933546508, + "actual_fraction": 0.09997135491263248, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.7941850472643941, + "metric_std": 0.0006243052053195129, + "actual_fraction": 0.10999713549126325, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.12, + "metric": 0.7968490403895732, + "metric_std": 0.0007414597024087245, + "actual_fraction": 0.11987969063305644, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.7999713549126325, + "metric_std": 0.0007132569234024202, + "actual_fraction": 0.1299054712116872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.8028072185620166, + "metric_std": 0.00046850264204513655, + "actual_fraction": 0.13993125179031796, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.8064164995703237, + "metric_std": 0.0010935897547304943, + "actual_fraction": 0.14995703236894872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.8091377828702377, + "metric_std": 0.0011121895536131458, + "actual_fraction": 0.1599828129475795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.8118017759954167, + "metric_std": 0.0005991549889244395, + "actual_fraction": 0.16986536808937266, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.8142079633342881, + "metric_std": 0.0007805457802027362, + "actual_fraction": 0.17989114866800343, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.8162131194500143, + "metric_std": 0.0008190191037236265, + "actual_fraction": 0.1899169292466342, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.819306788885706, + "metric_std": 0.0010157905028070273, + "actual_fraction": 0.19994270982526496, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.8213405900887997, + "metric_std": 0.0016442878990050105, + "actual_fraction": 0.20996849040389573, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.22, + "metric": 0.822400458321398, + "metric_std": 0.0022555164348357928, + "actual_fraction": 0.2199942709825265, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.23, + "metric": 0.8238327126897736, + "metric_std": 0.0028770136675165562, + "actual_fraction": 0.22987682612431967, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.24, + "metric": 0.8246634202234316, + "metric_std": 0.003090816873446567, + "actual_fraction": 0.23990260670295044, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.8254654826697221, + "metric_std": 0.0029744719966963034, + "actual_fraction": 0.24992838728158123, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.8262102549412775, + "metric_std": 0.003147051037977019, + "actual_fraction": 0.25995416786021197, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.8270123173875682, + "metric_std": 0.0031597362209662113, + "actual_fraction": 0.26997994843884277, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.8278143798338586, + "metric_std": 0.0031073648923112835, + "actual_fraction": 0.2798625035806359, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.8287023775422515, + "metric_std": 0.0029159657793615588, + "actual_fraction": 0.2898882841592667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.8293898596390719, + "metric_std": 0.002800048132523113, + "actual_fraction": 0.29991406473789745, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.8301059868232599, + "metric_std": 0.002556884952213671, + "actual_fraction": 0.30993984531652824, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.8312804354053279, + "metric_std": 0.0025318919406564803, + "actual_fraction": 0.319965625895159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.8319679175021484, + "metric_std": 0.002506240528538749, + "actual_fraction": 0.3299914064737898, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.8326553995989687, + "metric_std": 0.002484042783005289, + "actual_fraction": 0.3398739616155829, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.8334288169578917, + "metric_std": 0.0026545048686408496, + "actual_fraction": 0.3498997421942137, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.8337152678315668, + "metric_std": 0.0023944791603123013, + "actual_fraction": 0.35992552277284445, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.8346605557146949, + "metric_std": 0.0022248401662232787, + "actual_fraction": 0.36995130335147525, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.8357777141220281, + "metric_std": 0.002304105133450598, + "actual_fraction": 0.379977083930106, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.8368948725293613, + "metric_std": 0.002367339273720336, + "actual_fraction": 0.3898596390718992, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.8378115153251218, + "metric_std": 0.0020596594686047016, + "actual_fraction": 0.3998854196505299, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.83887138355772, + "metric_std": 0.0022713766151752496, + "actual_fraction": 0.40991120022916067, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.42, + "metric": 0.839702091091378, + "metric_std": 0.002316978569320312, + "actual_fraction": 0.41993698080779146, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.43, + "metric": 0.840761959323976, + "metric_std": 0.002288023551729117, + "actual_fraction": 0.42996276138642225, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.44, + "metric": 0.8415067315955314, + "metric_std": 0.0019151549298707272, + "actual_fraction": 0.439988541965053, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.45, + "metric": 0.8426811801775995, + "metric_std": 0.0014521674999297095, + "actual_fraction": 0.4498710971068462, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.8435691778859926, + "metric_std": 0.001815633109744567, + "actual_fraction": 0.45989687768547693, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.47, + "metric": 0.8442853050701805, + "metric_std": 0.0014655258942751631, + "actual_fraction": 0.46992265826410773, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.48, + "metric": 0.8454597536522487, + "metric_std": 0.0016361586449149, + "actual_fraction": 0.4799484388427385, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.49, + "metric": 0.8467774276711545, + "metric_std": 0.0012212015896472354, + "actual_fraction": 0.4899742194213693, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.5, + "metric": 0.8478372959037526, + "metric_std": 0.001148485831952404, + "actual_fraction": 0.5, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.51, + "metric": 0.8491836150100258, + "metric_std": 0.001090772429943275, + "actual_fraction": 0.5098825551417931, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.8500716127184187, + "metric_std": 0.0009787653717141125, + "actual_fraction": 0.5199083357204239, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.851188771125752, + "metric_std": 0.0012770377041053612, + "actual_fraction": 0.5299341162990547, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.8523059295330851, + "metric_std": 0.001212773596678121, + "actual_fraction": 0.5399598968776855, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.8533371526783157, + "metric_std": 0.0012810472515037466, + "actual_fraction": 0.5499856774563162, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.8543110856488113, + "metric_std": 0.0013412815546769697, + "actual_fraction": 0.5598682325981095, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.8553423087940418, + "metric_std": 0.0016424154885677133, + "actual_fraction": 0.5698940131767402, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.8564594672013749, + "metric_std": 0.0014655258942751527, + "actual_fraction": 0.579919793755371, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.8574906903466054, + "metric_std": 0.0013126828115026612, + "actual_fraction": 0.5899455743340017, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.8587797192781437, + "metric_std": 0.0010076802692258631, + "actual_fraction": 0.5999713549126325, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.8598109424233744, + "metric_std": 0.0011609214625700093, + "actual_fraction": 0.6099971354912632, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.8608135204812376, + "metric_std": 0.0015069330779217847, + "actual_fraction": 0.6198796906330565, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.8618733887138355, + "metric_std": 0.0017085299866452716, + "actual_fraction": 0.6299054712116872, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.8633629332569465, + "metric_std": 0.002150767318487725, + "actual_fraction": 0.639931251790318, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.8648811228874248, + "metric_std": 0.0019231715196632922, + "actual_fraction": 0.6499570323689488, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.8661415067315955, + "metric_std": 0.0023306611981447228, + "actual_fraction": 0.6599828129475795, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.8676310512747063, + "metric_std": 0.0024440823841537125, + "actual_fraction": 0.6698653680893727, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.8693211114293898, + "metric_std": 0.0024191959198383217, + "actual_fraction": 0.6798911486680035, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.8711543970209108, + "metric_std": 0.0026401697014275375, + "actual_fraction": 0.6899169292466342, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.8724434259524492, + "metric_std": 0.002614796002542739, + "actual_fraction": 0.699942709825265, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.8739043254081924, + "metric_std": 0.0023936223060496333, + "actual_fraction": 0.7099684904038958, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.8751074190776281, + "metric_std": 0.0025217440363304025, + "actual_fraction": 0.7199942709825266, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.8764823832712691, + "metric_std": 0.0024213148621030386, + "actual_fraction": 0.7298768261243197, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.877542251503867, + "metric_std": 0.002445341031074727, + "actual_fraction": 0.7399026067029505, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.8793755370953882, + "metric_std": 0.00248156411006171, + "actual_fraction": 0.7499283872815812, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.8807218562016613, + "metric_std": 0.0026378377389220838, + "actual_fraction": 0.759954167860212, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.8819822400458321, + "metric_std": 0.0027679960025906304, + "actual_fraction": 0.7699799484388428, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.8836723002005156, + "metric_std": 0.002631998793359483, + "actual_fraction": 0.7798625035806359, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.8849326840446864, + "metric_std": 0.0028386110895073182, + "actual_fraction": 0.7898882841592667, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.8861644228014895, + "metric_std": 0.002666840711415206, + "actual_fraction": 0.7999140647378975, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.8875680320824978, + "metric_std": 0.0033328777651733216, + "actual_fraction": 0.8099398453165282, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.8893153824119164, + "metric_std": 0.003087496626625646, + "actual_fraction": 0.819965625895159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.8904325408192495, + "metric_std": 0.002568092391440997, + "actual_fraction": 0.8299914064737898, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.84, + "metric": 0.8914637639644802, + "metric_std": 0.0026456029536565577, + "actual_fraction": 0.8398739616155829, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.85, + "metric": 0.8931538241191636, + "metric_std": 0.0023534340769342777, + "actual_fraction": 0.8498997421942137, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.86, + "metric": 0.8946147235749068, + "metric_std": 0.0021824849783787584, + "actual_fraction": 0.8599255227728445, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.87, + "metric": 0.8957891721569752, + "metric_std": 0.0015871549902257344, + "actual_fraction": 0.8699513033514752, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.88, + "metric": 0.8972214265253509, + "metric_std": 0.001517784221339311, + "actual_fraction": 0.879977083930106, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.89, + "metric": 0.8982526496705816, + "metric_std": 0.001596819104230152, + "actual_fraction": 0.8898596390718991, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.8992838728158121, + "metric_std": 0.0014011225553017257, + "actual_fraction": 0.8998854196505299, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.900057290174735, + "metric_std": 0.0011158723105575792, + "actual_fraction": 0.9099112002291607, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.92, + "metric": 0.9013463191062732, + "metric_std": 0.0010495600664887483, + "actual_fraction": 0.9199369808077915, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.93, + "metric": 0.9022343168146663, + "metric_std": 0.000661012933784832, + "actual_fraction": 0.9299627613864223, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.94, + "metric": 0.9032082497851619, + "metric_std": 0.0008497506143908286, + "actual_fraction": 0.9399885419650531, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.95, + "metric": 0.9042108278430249, + "metric_std": 0.0005852998207305202, + "actual_fraction": 0.9498710971068463, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.96, + "metric": 0.9048983099398452, + "metric_std": 0.0006160358970955928, + "actual_fraction": 0.959896877685477, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.97, + "metric": 0.9055857920366656, + "metric_std": 0.0003843141754511402, + "actual_fraction": 0.9699226582641078, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.98, + "metric": 0.9060154683471785, + "metric_std": 0.0010097139348046187, + "actual_fraction": 0.9799484388427386, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.99, + "metric": 0.9066170151818964, + "metric_std": 0.0007089297433838111, + "actual_fraction": 0.9899742194213692, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.9050415353766829, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.7642247255506924, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.7688168185405754, + "metric_std": 0.0006497370408364168, + "actual_fraction": 0.011523283346487768, + "actual_fraction_std": 0.0009949937255334044 + }, + { + "budget": 0.02, + "metric": 0.7723469900265479, + "metric_std": 0.000737680313552383, + "actual_fraction": 0.022515605941020304, + "actual_fraction_std": 0.0013214739967822436 + }, + { + "budget": 0.03, + "metric": 0.7755327545382794, + "metric_std": 0.0012151230899055394, + "actual_fraction": 0.03244600703164239, + "actual_fraction_std": 0.001480411755532703 + }, + { + "budget": 0.04, + "metric": 0.7789194231183181, + "metric_std": 0.0010874547573843916, + "actual_fraction": 0.041357537490134176, + "actual_fraction_std": 0.0019627662090640095 + }, + { + "budget": 0.05, + "metric": 0.7815455262969075, + "metric_std": 0.001277095240656485, + "actual_fraction": 0.05065652579464734, + "actual_fraction_std": 0.0024432396992152407 + }, + { + "budget": 0.06, + "metric": 0.7844299347061778, + "metric_std": 0.0009845909962209608, + "actual_fraction": 0.0597833106120399, + "actual_fraction_std": 0.002285596330633785 + }, + { + "budget": 0.07, + "metric": 0.7874434957307885, + "metric_std": 0.0009225015971658286, + "actual_fraction": 0.06878094281409199, + "actual_fraction_std": 0.002615042709613922 + }, + { + "budget": 0.08, + "metric": 0.7901413503623448, + "metric_std": 0.0011269775758224684, + "actual_fraction": 0.07763507211021023, + "actual_fraction_std": 0.0025695636908186796 + }, + { + "budget": 0.09, + "metric": 0.793169261677549, + "metric_std": 0.000914092109015787, + "actual_fraction": 0.0866614048934491, + "actual_fraction_std": 0.0021795205101909623 + }, + { + "budget": 0.1, + "metric": 0.7958097151467317, + "metric_std": 0.0009605051606906409, + "actual_fraction": 0.09531463012125996, + "actual_fraction_std": 0.002305332981644303 + }, + { + "budget": 0.11, + "metric": 0.7986510726842218, + "metric_std": 0.0007650871930869588, + "actual_fraction": 0.10373825069957668, + "actual_fraction_std": 0.002029057002893398 + }, + { + "budget": 0.12, + "metric": 0.801564181674679, + "metric_std": 0.0011134198028999653, + "actual_fraction": 0.11325249336299059, + "actual_fraction_std": 0.0017565189472410186 + }, + { + "budget": 0.13, + "metric": 0.8045346918275096, + "metric_std": 0.0011816137676825154, + "actual_fraction": 0.1232833464877664, + "actual_fraction_std": 0.00291561744402963 + }, + { + "budget": 0.14, + "metric": 0.8074191002367799, + "metric_std": 0.0012302811365058539, + "actual_fraction": 0.132338379852192, + "actual_fraction_std": 0.003106349037160654 + }, + { + "budget": 0.15, + "metric": 0.8098442993470616, + "metric_std": 0.000975133659673909, + "actual_fraction": 0.14176652077204563, + "actual_fraction_std": 0.0035097390257046025 + }, + { + "budget": 0.16, + "metric": 0.8127861089187055, + "metric_std": 0.0011281190504145104, + "actual_fraction": 0.15221353232402954, + "actual_fraction_std": 0.00371572781235144 + }, + { + "budget": 0.17, + "metric": 0.8148669010547464, + "metric_std": 0.0012191414349971908, + "actual_fraction": 0.1611537633637081, + "actual_fraction_std": 0.004080380916441969 + }, + { + "budget": 0.18, + "metric": 0.8178087106263903, + "metric_std": 0.0011618419242526423, + "actual_fraction": 0.17142857142857143, + "actual_fraction_std": 0.003429614374859736 + }, + { + "budget": 0.19, + "metric": 0.8201047571213318, + "metric_std": 0.0007915456910681879, + "actual_fraction": 0.1810289158355457, + "actual_fraction_std": 0.003442200707845595 + }, + { + "budget": 0.2, + "metric": 0.8230322164023821, + "metric_std": 0.0014102400818323522, + "actual_fraction": 0.19252349860084667, + "actual_fraction_std": 0.0037017769960283283 + }, + { + "budget": 0.21, + "metric": 0.8249838559230824, + "metric_std": 0.0007055762226556517, + "actual_fraction": 0.20325751596469832, + "actual_fraction_std": 0.003990890377041435 + }, + { + "budget": 0.22, + "metric": 0.826347133529454, + "metric_std": 0.0011462280575570629, + "actual_fraction": 0.2130587644399799, + "actual_fraction_std": 0.003543025428399365 + }, + { + "budget": 0.23, + "metric": 0.8278539140417592, + "metric_std": 0.0021056331560048375, + "actual_fraction": 0.2231613690177226, + "actual_fraction_std": 0.0048580555710419256 + }, + { + "budget": 0.24, + "metric": 0.8289445361268566, + "metric_std": 0.002378644536736847, + "actual_fraction": 0.2336370811508933, + "actual_fraction_std": 0.003302671914339815 + }, + { + "budget": 0.25, + "metric": 0.8300351582119537, + "metric_std": 0.002933660620366436, + "actual_fraction": 0.24352443136973526, + "actual_fraction_std": 0.003293930955338459 + }, + { + "budget": 0.26, + "metric": 0.831082729425271, + "metric_std": 0.003187654698732837, + "actual_fraction": 0.2534548324603573, + "actual_fraction_std": 0.002546216344678375 + }, + { + "budget": 0.27, + "metric": 0.8320441988950277, + "metric_std": 0.0028765085009093633, + "actual_fraction": 0.2634713352945397, + "actual_fraction_std": 0.0017937863241730622 + }, + { + "budget": 0.28, + "metric": 0.8327043122623232, + "metric_std": 0.0028050138277159837, + "actual_fraction": 0.2734878381287221, + "actual_fraction_std": 0.0023319774141971963 + }, + { + "budget": 0.29, + "metric": 0.833507928535553, + "metric_std": 0.0025668574618521534, + "actual_fraction": 0.28303078137332277, + "actual_fraction_std": 0.0028658396205771372 + }, + { + "budget": 0.3, + "metric": 0.8342541436464087, + "metric_std": 0.0023508868372541817, + "actual_fraction": 0.29232976967783597, + "actual_fraction_std": 0.0032972115304828278 + }, + { + "budget": 0.31, + "metric": 0.8351008107914184, + "metric_std": 0.0021854178111650716, + "actual_fraction": 0.30244672454617205, + "actual_fraction_std": 0.004237711518283428 + }, + { + "budget": 0.32, + "metric": 0.8358757264834613, + "metric_std": 0.0020668902433794113, + "actual_fraction": 0.3124488770897611, + "actual_fraction_std": 0.005359934391136216 + }, + { + "budget": 0.33, + "metric": 0.8366649924660974, + "metric_std": 0.0018437470930666056, + "actual_fraction": 0.32288153835115163, + "actual_fraction_std": 0.004886215997659544 + }, + { + "budget": 0.34, + "metric": 0.8374112075769535, + "metric_std": 0.001740619856598758, + "actual_fraction": 0.3314773624165889, + "actual_fraction_std": 0.005740205925969319 + }, + { + "budget": 0.35, + "metric": 0.8382865753031499, + "metric_std": 0.0017065687522981596, + "actual_fraction": 0.34235488268637443, + "actual_fraction_std": 0.005914513657965768 + }, + { + "budget": 0.36, + "metric": 0.8394058979694339, + "metric_std": 0.0012647402107112745, + "actual_fraction": 0.35351940876802757, + "actual_fraction_std": 0.00494720934948359 + }, + { + "budget": 0.37, + "metric": 0.8403673674391907, + "metric_std": 0.0011315365652122987, + "actual_fraction": 0.36395207002941815, + "actual_fraction_std": 0.005563537521264897 + }, + { + "budget": 0.38, + "metric": 0.8412714357465738, + "metric_std": 0.0012096026698937337, + "actual_fraction": 0.3739685728636005, + "actual_fraction_std": 0.0058220457128796 + }, + { + "budget": 0.39, + "metric": 0.8422042046351439, + "metric_std": 0.0012939156580440406, + "actual_fraction": 0.38385592308244243, + "actual_fraction_std": 0.00584199607430453 + }, + { + "budget": 0.4, + "metric": 0.8430795723613403, + "metric_std": 0.0012955062126187493, + "actual_fraction": 0.39397287795077846, + "actual_fraction_std": 0.005650945102322946 + }, + { + "budget": 0.41, + "metric": 0.8437540360192294, + "metric_std": 0.0012639258254300075, + "actual_fraction": 0.40387457846021385, + "actual_fraction_std": 0.005933155369302802 + }, + { + "budget": 0.42, + "metric": 0.844787256941953, + "metric_std": 0.0012667738833717914, + "actual_fraction": 0.4139771830379565, + "actual_fraction_std": 0.005752122050270619 + }, + { + "budget": 0.43, + "metric": 0.8457774269928965, + "metric_std": 0.0011598462023253635, + "actual_fraction": 0.4240797876156992, + "actual_fraction_std": 0.005538031676088957 + }, + { + "budget": 0.44, + "metric": 0.8468967496591807, + "metric_std": 0.0009537816763841775, + "actual_fraction": 0.43363708115089333, + "actual_fraction_std": 0.006461336861944982 + }, + { + "budget": 0.45, + "metric": 0.8479012700007175, + "metric_std": 0.0010415088575252368, + "actual_fraction": 0.4441558441558442, + "actual_fraction_std": 0.005954376366625386 + }, + { + "budget": 0.46, + "metric": 0.8487479371457273, + "metric_std": 0.0010654547180108944, + "actual_fraction": 0.4535839850756978, + "actual_fraction_std": 0.005220035322098998 + }, + { + "budget": 0.47, + "metric": 0.8494654516753964, + "metric_std": 0.0008305831170833361, + "actual_fraction": 0.4628973236708044, + "actual_fraction_std": 0.0065842661292600645 + }, + { + "budget": 0.48, + "metric": 0.8503695199827795, + "metric_std": 0.0006449653456050695, + "actual_fraction": 0.4740762000430509, + "actual_fraction_std": 0.007264883332859921 + }, + { + "budget": 0.49, + "metric": 0.8516323455549974, + "metric_std": 0.0010024682038644284, + "actual_fraction": 0.48357609241587135, + "actual_fraction_std": 0.006982894754285689 + }, + { + "budget": 0.5, + "metric": 0.8526081653153478, + "metric_std": 0.0006852169842183273, + "actual_fraction": 0.4932338379852192, + "actual_fraction_std": 0.006291542836198783 + }, + { + "budget": 0.51, + "metric": 0.8534117815885771, + "metric_std": 0.000475945295308231, + "actual_fraction": 0.501958814665997, + "actual_fraction_std": 0.007109624066033609 + }, + { + "budget": 0.52, + "metric": 0.8543876013489273, + "metric_std": 0.0003272369735521849, + "actual_fraction": 0.5122479730214538, + "actual_fraction_std": 0.007152040002905562 + }, + { + "budget": 0.53, + "metric": 0.8556360766305516, + "metric_std": 0.0005048162186794044, + "actual_fraction": 0.5212599555140992, + "actual_fraction_std": 0.006509126352458833 + }, + { + "budget": 0.54, + "metric": 0.8567123484250555, + "metric_std": 0.000875838101150446, + "actual_fraction": 0.5318074191002367, + "actual_fraction_std": 0.006335779246748976 + }, + { + "budget": 0.55, + "metric": 0.8578747219631196, + "metric_std": 0.0009095752581069364, + "actual_fraction": 0.5416804190284853, + "actual_fraction_std": 0.0059379692188843885 + }, + { + "budget": 0.56, + "metric": 0.85885054172347, + "metric_std": 0.0010228043154885617, + "actual_fraction": 0.5506780512305374, + "actual_fraction_std": 0.005881299889220559 + }, + { + "budget": 0.57, + "metric": 0.860199469039248, + "metric_std": 0.001304022205426776, + "actual_fraction": 0.5601348927315779, + "actual_fraction_std": 0.00613854542634215 + }, + { + "budget": 0.58, + "metric": 0.861089187056038, + "metric_std": 0.0009613088187540072, + "actual_fraction": 0.569806988591519, + "actual_fraction_std": 0.005388433798605616 + }, + { + "budget": 0.59, + "metric": 0.8622946114658822, + "metric_std": 0.000554393162720294, + "actual_fraction": 0.5795221353232403, + "actual_fraction_std": 0.00738541717129427 + }, + { + "budget": 0.6, + "metric": 0.863256080935639, + "metric_std": 0.0007532193999862715, + "actual_fraction": 0.5893951352514889, + "actual_fraction_std": 0.007758220206709129 + }, + { + "budget": 0.61, + "metric": 0.864303652148956, + "metric_std": 0.0009537816763841274, + "actual_fraction": 0.600114802324747, + "actual_fraction_std": 0.007313968793306693 + }, + { + "budget": 0.62, + "metric": 0.8654947262682068, + "metric_std": 0.0009027576085270169, + "actual_fraction": 0.6096720958599411, + "actual_fraction_std": 0.006420452158527138 + }, + { + "budget": 0.63, + "metric": 0.8664992466097438, + "metric_std": 0.0012423572942832381, + "actual_fraction": 0.6190141350362345, + "actual_fraction_std": 0.006354643441230039 + }, + { + "budget": 0.64, + "metric": 0.8674463657889072, + "metric_std": 0.0015609857993126627, + "actual_fraction": 0.6279974169476932, + "actual_fraction_std": 0.0062979222198570415 + }, + { + "budget": 0.65, + "metric": 0.8684508861304441, + "metric_std": 0.0012161818363727469, + "actual_fraction": 0.6386166319867976, + "actual_fraction_std": 0.005998138231928163 + }, + { + "budget": 0.66, + "metric": 0.8695128076343547, + "metric_std": 0.0013807261684927726, + "actual_fraction": 0.648475281624453, + "actual_fraction_std": 0.005997022323179289 + }, + { + "budget": 0.67, + "metric": 0.8705173279758915, + "metric_std": 0.001437359089945696, + "actual_fraction": 0.6580756260314271, + "actual_fraction_std": 0.006520426858953032 + }, + { + "budget": 0.68, + "metric": 0.8716940518045491, + "metric_std": 0.0017519700830515854, + "actual_fraction": 0.6683934849680706, + "actual_fraction_std": 0.00479302928178852 + }, + { + "budget": 0.69, + "metric": 0.8725694195307454, + "metric_std": 0.0018377338702541015, + "actual_fraction": 0.6791418526225156, + "actual_fraction_std": 0.004346266899732534 + }, + { + "budget": 0.7, + "metric": 0.8737030924876228, + "metric_std": 0.0014690660993981108, + "actual_fraction": 0.689344909234412, + "actual_fraction_std": 0.0031133847568511966 + }, + { + "budget": 0.71, + "metric": 0.8749085168974672, + "metric_std": 0.001779812110325832, + "actual_fraction": 0.6984429934706178, + "actual_fraction_std": 0.002968551073436857 + }, + { + "budget": 0.72, + "metric": 0.8759417378201908, + "metric_std": 0.0019037811812329262, + "actual_fraction": 0.7076558800315706, + "actual_fraction_std": 0.002296382805246997 + }, + { + "budget": 0.73, + "metric": 0.8772476142641888, + "metric_std": 0.0017349910822176701, + "actual_fraction": 0.7176436822845664, + "actual_fraction_std": 0.0017497647743063979 + }, + { + "budget": 0.74, + "metric": 0.8784817392552199, + "metric_std": 0.0016405840668181318, + "actual_fraction": 0.7280906938365501, + "actual_fraction_std": 0.002819658651277425 + }, + { + "budget": 0.75, + "metric": 0.8795867116309106, + "metric_std": 0.0014637999833967306, + "actual_fraction": 0.7391260673028628, + "actual_fraction_std": 0.002611989420536774 + }, + { + "budget": 0.76, + "metric": 0.8802037741264261, + "metric_std": 0.0014935681522920668, + "actual_fraction": 0.7493291239147593, + "actual_fraction_std": 0.002686897919035457 + }, + { + "budget": 0.77, + "metric": 0.881696204348138, + "metric_std": 0.0015492333399250997, + "actual_fraction": 0.7602353447657315, + "actual_fraction_std": 0.002836497474576105 + }, + { + "budget": 0.78, + "metric": 0.8831168831168832, + "metric_std": 0.0018482093524275598, + "actual_fraction": 0.7706967066083088, + "actual_fraction_std": 0.0019415365122236218 + }, + { + "budget": 0.79, + "metric": 0.8840927028772333, + "metric_std": 0.0020215581916120753, + "actual_fraction": 0.7815455262969075, + "actual_fraction_std": 0.003300800800315304 + }, + { + "budget": 0.8, + "metric": 0.8852263758341106, + "metric_std": 0.0019323666728593694, + "actual_fraction": 0.7912750233192222, + "actual_fraction_std": 0.003896771140694414 + }, + { + "budget": 0.81, + "metric": 0.8867331563464159, + "metric_std": 0.0020055782544039567, + "actual_fraction": 0.8012915261534046, + "actual_fraction_std": 0.004015710306069469 + }, + { + "budget": 0.82, + "metric": 0.8877950778503264, + "metric_std": 0.0019593533848787977, + "actual_fraction": 0.8110784243380926, + "actual_fraction_std": 0.003561287072893288 + }, + { + "budget": 0.83, + "metric": 0.8889718016789839, + "metric_std": 0.0020030096336593954, + "actual_fraction": 0.8210375260099017, + "actual_fraction_std": 0.002575767233542538 + }, + { + "budget": 0.84, + "metric": 0.8898184688239936, + "metric_std": 0.0021463161214889306, + "actual_fraction": 0.8319580971514673, + "actual_fraction_std": 0.002337159715132702 + }, + { + "budget": 0.85, + "metric": 0.8909664920714644, + "metric_std": 0.002412812642962402, + "actual_fraction": 0.8427351653870991, + "actual_fraction_std": 0.002893906086997263 + }, + { + "budget": 0.86, + "metric": 0.892028413575375, + "metric_std": 0.0022285728699020196, + "actual_fraction": 0.8521633063069526, + "actual_fraction_std": 0.002978160721609057 + }, + { + "budget": 0.87, + "metric": 0.8931477362416589, + "metric_std": 0.002126919305055328, + "actual_fraction": 0.863256080935639, + "actual_fraction_std": 0.0024893721710800994 + }, + { + "budget": 0.88, + "metric": 0.8940374542584488, + "metric_std": 0.002031592691214561, + "actual_fraction": 0.8745497596326326, + "actual_fraction_std": 0.0027606132589590394 + }, + { + "budget": 0.89, + "metric": 0.8949128219846452, + "metric_std": 0.0022453734195240524, + "actual_fraction": 0.8853842290306378, + "actual_fraction_std": 0.0030044073540920397 + }, + { + "budget": 0.9, + "metric": 0.8961038961038961, + "metric_std": 0.0015580146911536499, + "actual_fraction": 0.8964052522063571, + "actual_fraction_std": 0.0029914419026915626 + }, + { + "budget": 0.91, + "metric": 0.8974528234196744, + "metric_std": 0.0013067828785418736, + "actual_fraction": 0.9055750878955298, + "actual_fraction_std": 0.0028794600969823257 + }, + { + "budget": 0.92, + "metric": 0.8985864963765516, + "metric_std": 0.0013197196976134937, + "actual_fraction": 0.9168974671737102, + "actual_fraction_std": 0.002310240798849717 + }, + { + "budget": 0.93, + "metric": 0.8998923728205496, + "metric_std": 0.0009631814131907696, + "actual_fraction": 0.9273157781445075, + "actual_fraction_std": 0.002655288522632335 + }, + { + "budget": 0.94, + "metric": 0.9011121475209872, + "metric_std": 0.0012565726069664605, + "actual_fraction": 0.938236349286073, + "actual_fraction_std": 0.002377237278826911 + }, + { + "budget": 0.95, + "metric": 0.9019014135036235, + "metric_std": 0.0011201043002319118, + "actual_fraction": 0.9487981631628042, + "actual_fraction_std": 0.0015310152800035164 + }, + { + "budget": 0.96, + "metric": 0.9025615268709191, + "metric_std": 0.001451083333296757, + "actual_fraction": 0.9593886776207219, + "actual_fraction_std": 0.0013509485817437602 + }, + { + "budget": 0.97, + "metric": 0.903078137332281, + "metric_std": 0.001380539722449317, + "actual_fraction": 0.9693047284207505, + "actual_fraction_std": 0.0016990101680996873 + }, + { + "budget": 0.98, + "metric": 0.9031211882040611, + "metric_std": 0.001142854476049632, + "actual_fraction": 0.9793068809643396, + "actual_fraction_std": 0.002303098698330436 + }, + { + "budget": 0.99, + "metric": 0.9029202841357536, + "metric_std": 0.0006938035303800604, + "actual_fraction": 0.9889502762430938, + "actual_fraction_std": 0.0017865967708959802 + }, + { + "budget": 1.0, + "metric": 0.9006242376408122, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 4870, + "rescuable": 1449, + "harmful": 406, + "both_wrong": 257, + "unit": "sample", + "total": 6982 + }, + "test": { + "both_correct": 9761, + "rescuable": 2791, + "harmful": 890, + "both_wrong": 495, + "unit": "sample", + "total": 13937 + } + }, + "small": { + "validation": 0.7556574047550845, + "test": 0.7642247255506924 + }, + "large": { + "validation": 0.9050415353766829, + "test": 0.9006242376408122 + }, + "learned_minus_confidence_test": 0.0034440697424122924, + "learned_minus_small_test": 0.13883906149099512, + "learned_minus_large_test": 0.0024395494008753182, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_vilt_Qwen3.8" + }, + { + "key": [ + "construction", + "yolo26x", + "MiniCPM" + ], + "expert_display": "YOLO26x", + "run": "five_seed_yolo26x_MiniCPM_four_difference", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 94, + "validation_area_0_25": 0.1745164052515915, + "validation_point": { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.22641590025257546, + "rescued": 272, + "harmed": 478, + "missed_rescue": 14, + "avoided_harm": 40, + "selected_both_correct": 1782, + "selected_both_wrong": 100 + }, + "test_point": { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.2523047077304935, + "rescued": 1113, + "harmed": 1932, + "missed_rescue": 85, + "avoided_harm": 177, + "selected_both_correct": 7597, + "selected_both_wrong": 478 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 286, + "avoided_harm": 518, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.1583943405474877, + "rescued": 0, + "harmed": 88, + "missed_rescue": 286, + "avoided_harm": 430, + "selected_both_correct": 444, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.15744253541872588, + "rescued": 0, + "harmed": 94, + "missed_rescue": 286, + "avoided_harm": 424, + "selected_both_correct": 466, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.1562524865065214, + "rescued": 0, + "harmed": 100, + "missed_rescue": 286, + "avoided_harm": 418, + "selected_both_correct": 488, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.15524778086857327, + "rescued": 0, + "harmed": 106, + "missed_rescue": 286, + "avoided_harm": 412, + "selected_both_correct": 509, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.1542743955978643, + "rescued": 0, + "harmed": 112, + "missed_rescue": 286, + "avoided_harm": 406, + "selected_both_correct": 531, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.15300917812583928, + "rescued": 0, + "harmed": 118, + "missed_rescue": 286, + "avoided_harm": 400, + "selected_both_correct": 553, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24821683309557774, + "metric": 0.15249392985085833, + "rescued": 0, + "harmed": 122, + "missed_rescue": 286, + "avoided_harm": 396, + "selected_both_correct": 572, + "selected_both_wrong": 2 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.15123140165156973, + "rescued": 0, + "harmed": 131, + "missed_rescue": 286, + "avoided_harm": 387, + "selected_both_correct": 595, + "selected_both_wrong": 2 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.268188302425107, + "metric": 0.15063802192285164, + "rescued": 0, + "harmed": 136, + "missed_rescue": 286, + "avoided_harm": 382, + "selected_both_correct": 614, + "selected_both_wrong": 2 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.14987516228902426, + "rescued": 0, + "harmed": 143, + "missed_rescue": 286, + "avoided_harm": 375, + "selected_both_correct": 639, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.14942913195896182, + "rescued": 0, + "harmed": 147, + "missed_rescue": 286, + "avoided_harm": 371, + "selected_both_correct": 663, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.14874977232993136, + "rescued": 0, + "harmed": 152, + "missed_rescue": 286, + "avoided_harm": 366, + "selected_both_correct": 686, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30813124108416545, + "metric": 0.1481550075018755, + "rescued": 0, + "harmed": 156, + "missed_rescue": 286, + "avoided_harm": 362, + "selected_both_correct": 706, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3181169757489301, + "metric": 0.14737954730376035, + "rescued": 0, + "harmed": 162, + "missed_rescue": 286, + "avoided_harm": 356, + "selected_both_correct": 728, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.14624628747548335, + "rescued": 0, + "harmed": 170, + "missed_rescue": 286, + "avoided_harm": 348, + "selected_both_correct": 752, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.14552266133559724, + "rescued": 0, + "harmed": 176, + "missed_rescue": 286, + "avoided_harm": 342, + "selected_both_correct": 774, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34807417974322397, + "metric": 0.14490257033424028, + "rescued": 0, + "harmed": 181, + "missed_rescue": 286, + "avoided_harm": 337, + "selected_both_correct": 793, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.1443034349839422, + "rescued": 0, + "harmed": 188, + "missed_rescue": 286, + "avoided_harm": 330, + "selected_both_correct": 818, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3666191155492154, + "metric": 0.14397079647073363, + "rescued": 0, + "harmed": 192, + "missed_rescue": 286, + "avoided_harm": 326, + "selected_both_correct": 834, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37660485021398005, + "metric": 0.14328749409772273, + "rescued": 0, + "harmed": 198, + "missed_rescue": 286, + "avoided_harm": 320, + "selected_both_correct": 856, + "selected_both_wrong": 2 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.14535108686369555, + "rescued": 1, + "harmed": 209, + "missed_rescue": 285, + "avoided_harm": 309, + "selected_both_correct": 880, + "selected_both_wrong": 2 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.14474378206514643, + "rescued": 1, + "harmed": 214, + "missed_rescue": 285, + "avoided_harm": 304, + "selected_both_correct": 903, + "selected_both_wrong": 2 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.14399177962039386, + "rescued": 1, + "harmed": 221, + "missed_rescue": 285, + "avoided_harm": 297, + "selected_both_correct": 924, + "selected_both_wrong": 2 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.14353714231992915, + "rescued": 1, + "harmed": 227, + "missed_rescue": 285, + "avoided_harm": 291, + "selected_both_correct": 946, + "selected_both_wrong": 2 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.1430174966224799, + "rescued": 1, + "harmed": 234, + "missed_rescue": 285, + "avoided_harm": 284, + "selected_both_correct": 967, + "selected_both_wrong": 2 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.14225189964081136, + "rescued": 1, + "harmed": 242, + "missed_rescue": 285, + "avoided_harm": 276, + "selected_both_correct": 987, + "selected_both_wrong": 2 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.14143790301195314, + "rescued": 1, + "harmed": 248, + "missed_rescue": 285, + "avoided_harm": 270, + "selected_both_correct": 1009, + "selected_both_wrong": 2 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.14328380481065045, + "rescued": 2, + "harmed": 254, + "missed_rescue": 284, + "avoided_harm": 264, + "selected_both_correct": 1030, + "selected_both_wrong": 2 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.46790299572039945, + "metric": 0.145146248248267, + "rescued": 3, + "harmed": 259, + "missed_rescue": 283, + "avoided_harm": 259, + "selected_both_correct": 1048, + "selected_both_wrong": 2 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.14408640672685616, + "rescued": 3, + "harmed": 268, + "missed_rescue": 283, + "avoided_harm": 250, + "selected_both_correct": 1071, + "selected_both_wrong": 2 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.14363247863247863, + "rescued": 5, + "harmed": 274, + "missed_rescue": 281, + "avoided_harm": 244, + "selected_both_correct": 1090, + "selected_both_wrong": 3 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.14422472705491574, + "rescued": 8, + "harmed": 281, + "missed_rescue": 278, + "avoided_harm": 237, + "selected_both_correct": 1104, + "selected_both_wrong": 7 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.14584304986884322, + "rescued": 15, + "harmed": 285, + "missed_rescue": 271, + "avoided_harm": 233, + "selected_both_correct": 1120, + "selected_both_wrong": 8 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5178316690442225, + "metric": 0.14475873599710132, + "rescued": 18, + "harmed": 291, + "missed_rescue": 268, + "avoided_harm": 227, + "selected_both_correct": 1133, + "selected_both_wrong": 10 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.14599834427394986, + "rescued": 24, + "harmed": 297, + "missed_rescue": 262, + "avoided_harm": 221, + "selected_both_correct": 1151, + "selected_both_wrong": 12 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5378031383737518, + "metric": 0.14694159872855866, + "rescued": 29, + "harmed": 301, + "missed_rescue": 257, + "avoided_harm": 217, + "selected_both_correct": 1166, + "selected_both_wrong": 12 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.14684487529484538, + "rescued": 35, + "harmed": 305, + "missed_rescue": 251, + "avoided_harm": 213, + "selected_both_correct": 1187, + "selected_both_wrong": 13 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.14849467509528708, + "rescued": 42, + "harmed": 308, + "missed_rescue": 244, + "avoided_harm": 210, + "selected_both_correct": 1203, + "selected_both_wrong": 15 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.1496557181286853, + "rescued": 48, + "harmed": 311, + "missed_rescue": 238, + "avoided_harm": 207, + "selected_both_correct": 1221, + "selected_both_wrong": 16 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.15076631577130084, + "rescued": 53, + "harmed": 315, + "missed_rescue": 233, + "avoided_harm": 203, + "selected_both_correct": 1237, + "selected_both_wrong": 19 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5877318116975749, + "metric": 0.15169442750270445, + "rescued": 57, + "harmed": 320, + "missed_rescue": 229, + "avoided_harm": 198, + "selected_both_correct": 1250, + "selected_both_wrong": 21 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.5977175463623395, + "metric": 0.1523976411340631, + "rescued": 62, + "harmed": 322, + "missed_rescue": 224, + "avoided_harm": 196, + "selected_both_correct": 1268, + "selected_both_wrong": 24 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.15402113497563988, + "rescued": 68, + "harmed": 328, + "missed_rescue": 218, + "avoided_harm": 190, + "selected_both_correct": 1286, + "selected_both_wrong": 26 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6176890156918687, + "metric": 0.1533732028512746, + "rescued": 71, + "harmed": 331, + "missed_rescue": 215, + "avoided_harm": 187, + "selected_both_correct": 1302, + "selected_both_wrong": 28 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.15514650916956785, + "rescued": 79, + "harmed": 337, + "missed_rescue": 207, + "avoided_harm": 181, + "selected_both_correct": 1317, + "selected_both_wrong": 31 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.1583928135738368, + "rescued": 85, + "harmed": 341, + "missed_rescue": 201, + "avoided_harm": 177, + "selected_both_correct": 1332, + "selected_both_wrong": 34 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6476462196861626, + "metric": 0.15975557497296627, + "rescued": 92, + "harmed": 346, + "missed_rescue": 194, + "avoided_harm": 172, + "selected_both_correct": 1344, + "selected_both_wrong": 34 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6562054208273894, + "metric": 0.16074742775562448, + "rescued": 95, + "harmed": 347, + "missed_rescue": 191, + "avoided_harm": 171, + "selected_both_correct": 1361, + "selected_both_wrong": 37 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.16246869830774383, + "rescued": 102, + "harmed": 352, + "missed_rescue": 184, + "avoided_harm": 166, + "selected_both_correct": 1381, + "selected_both_wrong": 41 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.16459532061840704, + "rescued": 109, + "harmed": 357, + "missed_rescue": 177, + "avoided_harm": 161, + "selected_both_correct": 1396, + "selected_both_wrong": 42 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.16603896761133602, + "rescued": 114, + "harmed": 362, + "missed_rescue": 172, + "avoided_harm": 156, + "selected_both_correct": 1411, + "selected_both_wrong": 45 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.6975748930099858, + "metric": 0.16707723655265538, + "rescued": 119, + "harmed": 366, + "missed_rescue": 167, + "avoided_harm": 152, + "selected_both_correct": 1423, + "selected_both_wrong": 48 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.16940425786435637, + "rescued": 126, + "harmed": 371, + "missed_rescue": 160, + "avoided_harm": 147, + "selected_both_correct": 1441, + "selected_both_wrong": 50 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.1708064509820028, + "rescued": 134, + "harmed": 379, + "missed_rescue": 152, + "avoided_harm": 139, + "selected_both_correct": 1453, + "selected_both_wrong": 50 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.17255814544390938, + "rescued": 139, + "harmed": 381, + "missed_rescue": 147, + "avoided_harm": 137, + "selected_both_correct": 1470, + "selected_both_wrong": 54 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.17530145923526386, + "rescued": 146, + "harmed": 384, + "missed_rescue": 140, + "avoided_harm": 134, + "selected_both_correct": 1487, + "selected_both_wrong": 55 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.17900218396288498, + "rescued": 155, + "harmed": 389, + "missed_rescue": 131, + "avoided_harm": 129, + "selected_both_correct": 1499, + "selected_both_wrong": 57 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.18402223394204145, + "rescued": 163, + "harmed": 392, + "missed_rescue": 123, + "avoided_harm": 126, + "selected_both_correct": 1514, + "selected_both_wrong": 59 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.18675324648763036, + "rescued": 169, + "harmed": 395, + "missed_rescue": 117, + "avoided_harm": 123, + "selected_both_correct": 1531, + "selected_both_wrong": 61 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.18924798946215277, + "rescued": 175, + "harmed": 399, + "missed_rescue": 111, + "avoided_harm": 119, + "selected_both_correct": 1547, + "selected_both_wrong": 63 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.19150730156013804, + "rescued": 181, + "harmed": 403, + "missed_rescue": 105, + "avoided_harm": 115, + "selected_both_correct": 1562, + "selected_both_wrong": 66 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.19267082077365763, + "rescued": 185, + "harmed": 410, + "missed_rescue": 101, + "avoided_harm": 108, + "selected_both_correct": 1575, + "selected_both_wrong": 70 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.1985395820585592, + "rescued": 193, + "harmed": 413, + "missed_rescue": 93, + "avoided_harm": 105, + "selected_both_correct": 1589, + "selected_both_wrong": 73 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.20050380794511347, + "rescued": 198, + "harmed": 417, + "missed_rescue": 88, + "avoided_harm": 101, + "selected_both_correct": 1605, + "selected_both_wrong": 76 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.20479272888770159, + "rescued": 205, + "harmed": 422, + "missed_rescue": 81, + "avoided_harm": 96, + "selected_both_correct": 1619, + "selected_both_wrong": 78 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.20996516586206454, + "rescued": 211, + "harmed": 428, + "missed_rescue": 75, + "avoided_harm": 90, + "selected_both_correct": 1631, + "selected_both_wrong": 82 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2153593554624366, + "rescued": 218, + "harmed": 430, + "missed_rescue": 68, + "avoided_harm": 88, + "selected_both_correct": 1647, + "selected_both_wrong": 85 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.20675118175118173, + "rescued": 225, + "harmed": 439, + "missed_rescue": 61, + "avoided_harm": 79, + "selected_both_correct": 1658, + "selected_both_wrong": 86 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.2103979829447119, + "rescued": 232, + "harmed": 441, + "missed_rescue": 54, + "avoided_harm": 77, + "selected_both_correct": 1675, + "selected_both_wrong": 88 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.2127029758032997, + "rescued": 236, + "harmed": 447, + "missed_rescue": 50, + "avoided_harm": 71, + "selected_both_correct": 1691, + "selected_both_wrong": 90 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.21408261584117613, + "rescued": 244, + "harmed": 453, + "missed_rescue": 42, + "avoided_harm": 65, + "selected_both_correct": 1705, + "selected_both_wrong": 90 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.21447015425888666, + "rescued": 251, + "harmed": 460, + "missed_rescue": 35, + "avoided_harm": 58, + "selected_both_correct": 1716, + "selected_both_wrong": 93 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.9072753209700428, + "metric": 0.21778616982331747, + "rescued": 255, + "harmed": 462, + "missed_rescue": 31, + "avoided_harm": 56, + "selected_both_correct": 1730, + "selected_both_wrong": 97 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.22341697071831912, + "rescued": 265, + "harmed": 468, + "missed_rescue": 21, + "avoided_harm": 50, + "selected_both_correct": 1746, + "selected_both_wrong": 97 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2260324221654167, + "rescued": 269, + "harmed": 475, + "missed_rescue": 17, + "avoided_harm": 43, + "selected_both_correct": 1762, + "selected_both_wrong": 98 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.22641590025257546, + "rescued": 272, + "harmed": 478, + "missed_rescue": 14, + "avoided_harm": 40, + "selected_both_correct": 1782, + "selected_both_wrong": 100 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.2213594019034288, + "rescued": 273, + "harmed": 486, + "missed_rescue": 13, + "avoided_harm": 32, + "selected_both_correct": 1799, + "selected_both_wrong": 102 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.21682778770996153, + "rescued": 278, + "harmed": 495, + "missed_rescue": 8, + "avoided_harm": 23, + "selected_both_correct": 1813, + "selected_both_wrong": 102 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.21297215749387616, + "rescued": 280, + "harmed": 500, + "missed_rescue": 6, + "avoided_harm": 18, + "selected_both_correct": 1831, + "selected_both_wrong": 105 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.21025141881293422, + "rescued": 282, + "harmed": 504, + "missed_rescue": 4, + "avoided_harm": 14, + "selected_both_correct": 1851, + "selected_both_wrong": 107 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.20718702865761687, + "rescued": 284, + "harmed": 512, + "missed_rescue": 2, + "avoided_harm": 6, + "selected_both_correct": 1869, + "selected_both_wrong": 107 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.20666016863629594, + "rescued": 286, + "harmed": 518, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1891, + "selected_both_wrong": 109 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1198, + "avoided_harm": 2109, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.17942743009320905, + "metric": 0.16785638110940382, + "rescued": 3, + "harmed": 361, + "missed_rescue": 1195, + "avoided_harm": 1748, + "selected_both_correct": 1790, + "selected_both_wrong": 2 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19340878828229027, + "metric": 0.1667569975208824, + "rescued": 3, + "harmed": 392, + "missed_rescue": 1195, + "avoided_harm": 1717, + "selected_both_correct": 1927, + "selected_both_wrong": 2 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20306258322237017, + "metric": 0.16670205561344506, + "rescued": 4, + "harmed": 415, + "missed_rescue": 1194, + "avoided_harm": 1694, + "selected_both_correct": 2018, + "selected_both_wrong": 3 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.16636966208367632, + "rescued": 4, + "harmed": 426, + "missed_rescue": 1194, + "avoided_harm": 1683, + "selected_both_correct": 2071, + "selected_both_wrong": 3 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.21537949400798934, + "metric": 0.1674861536267502, + "rescued": 5, + "harmed": 440, + "missed_rescue": 1193, + "avoided_harm": 1669, + "selected_both_correct": 2140, + "selected_both_wrong": 3 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.1659144502174876, + "rescued": 5, + "harmed": 482, + "missed_rescue": 1193, + "avoided_harm": 1627, + "selected_both_correct": 2289, + "selected_both_wrong": 4 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24800266311584554, + "metric": 0.16716244725738397, + "rescued": 7, + "harmed": 526, + "missed_rescue": 1191, + "avoided_harm": 1583, + "selected_both_correct": 2441, + "selected_both_wrong": 6 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.2589880159786951, + "metric": 0.1678664772207598, + "rescued": 8, + "harmed": 550, + "missed_rescue": 1190, + "avoided_harm": 1559, + "selected_both_correct": 2548, + "selected_both_wrong": 6 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.27430093209054596, + "metric": 0.16649682507763983, + "rescued": 8, + "harmed": 594, + "missed_rescue": 1190, + "avoided_harm": 1515, + "selected_both_correct": 2688, + "selected_both_wrong": 6 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.281624500665779, + "metric": 0.16581256695877195, + "rescued": 8, + "harmed": 621, + "missed_rescue": 1190, + "avoided_harm": 1488, + "selected_both_correct": 2749, + "selected_both_wrong": 6 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.28761651131824234, + "metric": 0.1651274175371034, + "rescued": 8, + "harmed": 636, + "missed_rescue": 1190, + "avoided_harm": 1473, + "selected_both_correct": 2806, + "selected_both_wrong": 6 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2969374167776298, + "metric": 0.16471286313286057, + "rescued": 8, + "harmed": 657, + "missed_rescue": 1190, + "avoided_harm": 1452, + "selected_both_correct": 2897, + "selected_both_wrong": 6 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30459387483355527, + "metric": 0.16408718171666042, + "rescued": 8, + "harmed": 676, + "missed_rescue": 1190, + "avoided_harm": 1433, + "selected_both_correct": 2969, + "selected_both_wrong": 7 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3129161118508655, + "metric": 0.1636354375797962, + "rescued": 9, + "harmed": 701, + "missed_rescue": 1189, + "avoided_harm": 1408, + "selected_both_correct": 3043, + "selected_both_wrong": 7 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.31990679094540614, + "metric": 0.16348828966450218, + "rescued": 10, + "harmed": 720, + "missed_rescue": 1188, + "avoided_harm": 1389, + "selected_both_correct": 3107, + "selected_both_wrong": 7 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.3322237017310253, + "metric": 0.16484645578128018, + "rescued": 12, + "harmed": 749, + "missed_rescue": 1186, + "avoided_harm": 1360, + "selected_both_correct": 3224, + "selected_both_wrong": 7 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34054593874833555, + "metric": 0.16526596702267776, + "rescued": 13, + "harmed": 767, + "missed_rescue": 1185, + "avoided_harm": 1342, + "selected_both_correct": 3304, + "selected_both_wrong": 8 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3488681757656458, + "metric": 0.16520260386826902, + "rescued": 14, + "harmed": 792, + "missed_rescue": 1184, + "avoided_harm": 1317, + "selected_both_correct": 3378, + "selected_both_wrong": 8 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3581890812250333, + "metric": 0.16665740832732326, + "rescued": 17, + "harmed": 812, + "missed_rescue": 1181, + "avoided_harm": 1297, + "selected_both_correct": 3467, + "selected_both_wrong": 8 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37017310252996005, + "metric": 0.1659164559036785, + "rescued": 17, + "harmed": 844, + "missed_rescue": 1181, + "avoided_harm": 1265, + "selected_both_correct": 3579, + "selected_both_wrong": 8 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.16604558576074954, + "rescued": 18, + "harmed": 868, + "missed_rescue": 1180, + "avoided_harm": 1241, + "selected_both_correct": 3633, + "selected_both_wrong": 9 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.38348868175765644, + "metric": 0.16627533009733186, + "rescued": 19, + "harmed": 882, + "missed_rescue": 1179, + "avoided_harm": 1227, + "selected_both_correct": 3698, + "selected_both_wrong": 9 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4021304926764314, + "metric": 0.16596184167496, + "rescued": 21, + "harmed": 925, + "missed_rescue": 1177, + "avoided_harm": 1184, + "selected_both_correct": 3876, + "selected_both_wrong": 10 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.41311584553928093, + "metric": 0.16543665040595484, + "rescued": 22, + "harmed": 952, + "missed_rescue": 1176, + "avoided_harm": 1157, + "selected_both_correct": 3980, + "selected_both_wrong": 10 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42676431424766975, + "metric": 0.16433504278528627, + "rescued": 22, + "harmed": 996, + "missed_rescue": 1176, + "avoided_harm": 1113, + "selected_both_correct": 4100, + "selected_both_wrong": 10 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.4324234354194407, + "metric": 0.1640338494529538, + "rescued": 22, + "harmed": 1010, + "missed_rescue": 1176, + "avoided_harm": 1099, + "selected_both_correct": 4154, + "selected_both_wrong": 10 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44107856191744343, + "metric": 0.16355335117560724, + "rescued": 22, + "harmed": 1030, + "missed_rescue": 1176, + "avoided_harm": 1079, + "selected_both_correct": 4238, + "selected_both_wrong": 10 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.45772303595206393, + "metric": 0.16328618395901012, + "rescued": 25, + "harmed": 1072, + "missed_rescue": 1173, + "avoided_harm": 1037, + "selected_both_correct": 4393, + "selected_both_wrong": 10 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.4717043941411451, + "metric": 0.16343963675474008, + "rescued": 27, + "harmed": 1106, + "missed_rescue": 1171, + "avoided_harm": 1003, + "selected_both_correct": 4524, + "selected_both_wrong": 11 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4826897470039947, + "metric": 0.16288089231855363, + "rescued": 27, + "harmed": 1135, + "missed_rescue": 1171, + "avoided_harm": 974, + "selected_both_correct": 4626, + "selected_both_wrong": 12 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4896804260985353, + "metric": 0.16253980428982567, + "rescued": 28, + "harmed": 1150, + "missed_rescue": 1170, + "avoided_harm": 959, + "selected_both_correct": 4691, + "selected_both_wrong": 15 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.5013315579227696, + "metric": 0.16396334310527208, + "rescued": 56, + "harmed": 1172, + "missed_rescue": 1142, + "avoided_harm": 937, + "selected_both_correct": 4775, + "selected_both_wrong": 21 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5116511318242344, + "metric": 0.16505546455344486, + "rescued": 76, + "harmed": 1183, + "missed_rescue": 1122, + "avoided_harm": 926, + "selected_both_correct": 4858, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5183089214380826, + "metric": 0.16558890552522137, + "rescued": 91, + "harmed": 1197, + "missed_rescue": 1107, + "avoided_harm": 912, + "selected_both_correct": 4904, + "selected_both_wrong": 36 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5252996005326231, + "metric": 0.1665272377685939, + "rescued": 107, + "harmed": 1214, + "missed_rescue": 1091, + "avoided_harm": 895, + "selected_both_correct": 4950, + "selected_both_wrong": 41 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5396138482023968, + "metric": 0.16808344009752285, + "rescued": 144, + "harmed": 1241, + "missed_rescue": 1054, + "avoided_harm": 868, + "selected_both_correct": 5049, + "selected_both_wrong": 50 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5452729693741678, + "metric": 0.16859171218698246, + "rescued": 156, + "harmed": 1251, + "missed_rescue": 1042, + "avoided_harm": 858, + "selected_both_correct": 5088, + "selected_both_wrong": 57 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.559254327563249, + "metric": 0.1696257903027952, + "rescued": 182, + "harmed": 1277, + "missed_rescue": 1016, + "avoided_harm": 832, + "selected_both_correct": 5186, + "selected_both_wrong": 75 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.572237017310253, + "metric": 0.17132213898552048, + "rescued": 211, + "harmed": 1300, + "missed_rescue": 987, + "avoided_harm": 809, + "selected_both_correct": 5276, + "selected_both_wrong": 89 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.5862183754993342, + "metric": 0.17290558782298915, + "rescued": 244, + "harmed": 1328, + "missed_rescue": 954, + "avoided_harm": 781, + "selected_both_correct": 5376, + "selected_both_wrong": 96 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5998668442077231, + "metric": 0.1757330216164513, + "rescued": 281, + "harmed": 1358, + "missed_rescue": 917, + "avoided_harm": 751, + "selected_both_correct": 5460, + "selected_both_wrong": 109 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.6191744340878829, + "metric": 0.17918696387117494, + "rescued": 329, + "harmed": 1394, + "missed_rescue": 869, + "avoided_harm": 715, + "selected_both_correct": 5591, + "selected_both_wrong": 126 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.6264980026631158, + "metric": 0.18029020519999098, + "rescued": 347, + "harmed": 1407, + "missed_rescue": 851, + "avoided_harm": 702, + "selected_both_correct": 5643, + "selected_both_wrong": 131 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6384820239680427, + "metric": 0.18360411382005598, + "rescued": 376, + "harmed": 1424, + "missed_rescue": 822, + "avoided_harm": 685, + "selected_both_correct": 5729, + "selected_both_wrong": 143 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6461384820239681, + "metric": 0.18430872796712833, + "rescued": 393, + "harmed": 1439, + "missed_rescue": 805, + "avoided_harm": 670, + "selected_both_correct": 5780, + "selected_both_wrong": 152 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.653462050599201, + "metric": 0.18413764925256976, + "rescued": 406, + "harmed": 1452, + "missed_rescue": 792, + "avoided_harm": 657, + "selected_both_correct": 5831, + "selected_both_wrong": 163 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6597869507323568, + "metric": 0.18503793034066174, + "rescued": 422, + "harmed": 1464, + "missed_rescue": 776, + "avoided_harm": 645, + "selected_both_correct": 5873, + "selected_both_wrong": 169 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6661118508655126, + "metric": 0.186106537681911, + "rescued": 441, + "harmed": 1475, + "missed_rescue": 757, + "avoided_harm": 634, + "selected_both_correct": 5915, + "selected_both_wrong": 173 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6734354194407457, + "metric": 0.1880916072476674, + "rescued": 461, + "harmed": 1484, + "missed_rescue": 737, + "avoided_harm": 625, + "selected_both_correct": 5966, + "selected_both_wrong": 181 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6820905459387483, + "metric": 0.18987442927689613, + "rescued": 484, + "harmed": 1494, + "missed_rescue": 714, + "avoided_harm": 615, + "selected_both_correct": 6030, + "selected_both_wrong": 188 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6927430093209055, + "metric": 0.19160618326349385, + "rescued": 511, + "harmed": 1512, + "missed_rescue": 687, + "avoided_harm": 597, + "selected_both_correct": 6100, + "selected_both_wrong": 201 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.7090545938748336, + "metric": 0.19565888484893704, + "rescued": 558, + "harmed": 1543, + "missed_rescue": 640, + "avoided_harm": 566, + "selected_both_correct": 6200, + "selected_both_wrong": 219 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7170439414114514, + "metric": 0.1973992538335802, + "rescued": 581, + "harmed": 1558, + "missed_rescue": 617, + "avoided_harm": 551, + "selected_both_correct": 6250, + "selected_both_wrong": 227 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7243675099866844, + "metric": 0.1992894526144666, + "rescued": 603, + "harmed": 1570, + "missed_rescue": 595, + "avoided_harm": 539, + "selected_both_correct": 6292, + "selected_both_wrong": 239 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7330226364846871, + "metric": 0.2014755547124751, + "rescued": 624, + "harmed": 1582, + "missed_rescue": 574, + "avoided_harm": 527, + "selected_both_correct": 6355, + "selected_both_wrong": 247 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.7363515312916112, + "metric": 0.20209035743657205, + "rescued": 634, + "harmed": 1592, + "missed_rescue": 564, + "avoided_harm": 517, + "selected_both_correct": 6374, + "selected_both_wrong": 248 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7486684420772304, + "metric": 0.20546211362830763, + "rescued": 665, + "harmed": 1612, + "missed_rescue": 533, + "avoided_harm": 497, + "selected_both_correct": 6458, + "selected_both_wrong": 261 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.757323568575233, + "metric": 0.20773020664342898, + "rescued": 689, + "harmed": 1626, + "missed_rescue": 509, + "avoided_harm": 483, + "selected_both_correct": 6514, + "selected_both_wrong": 271 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7639813581890812, + "metric": 0.2112193086129685, + "rescued": 707, + "harmed": 1633, + "missed_rescue": 491, + "avoided_harm": 476, + "selected_both_correct": 6558, + "selected_both_wrong": 282 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7733022636484687, + "metric": 0.2156222025883268, + "rescued": 738, + "harmed": 1654, + "missed_rescue": 460, + "avoided_harm": 455, + "selected_both_correct": 6610, + "selected_both_wrong": 290 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.784953395472703, + "metric": 0.21617077858796782, + "rescued": 763, + "harmed": 1672, + "missed_rescue": 435, + "avoided_harm": 437, + "selected_both_correct": 6690, + "selected_both_wrong": 307 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7939414114513982, + "metric": 0.21811777255990603, + "rescued": 788, + "harmed": 1693, + "missed_rescue": 410, + "avoided_harm": 416, + "selected_both_correct": 6742, + "selected_both_wrong": 317 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8039280958721704, + "metric": 0.22157947433765235, + "rescued": 814, + "harmed": 1705, + "missed_rescue": 384, + "avoided_harm": 404, + "selected_both_correct": 6809, + "selected_both_wrong": 332 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.8122503328894807, + "metric": 0.22355438821966017, + "rescued": 835, + "harmed": 1722, + "missed_rescue": 363, + "avoided_harm": 387, + "selected_both_correct": 6857, + "selected_both_wrong": 346 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8215712383488681, + "metric": 0.22762279758152631, + "rescued": 863, + "harmed": 1740, + "missed_rescue": 335, + "avoided_harm": 369, + "selected_both_correct": 6911, + "selected_both_wrong": 358 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388814913448736, + "metric": 0.23330777488772814, + "rescued": 909, + "harmed": 1767, + "missed_rescue": 289, + "avoided_harm": 342, + "selected_both_correct": 7021, + "selected_both_wrong": 383 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8535286284953395, + "metric": 0.23874788475407085, + "rescued": 947, + "harmed": 1794, + "missed_rescue": 251, + "avoided_harm": 315, + "selected_both_correct": 7110, + "selected_both_wrong": 405 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8638482023968043, + "metric": 0.2419731256831217, + "rescued": 974, + "harmed": 1808, + "missed_rescue": 224, + "avoided_harm": 301, + "selected_both_correct": 7179, + "selected_both_wrong": 419 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8745006657789614, + "metric": 0.2400850101807912, + "rescued": 996, + "harmed": 1832, + "missed_rescue": 202, + "avoided_harm": 277, + "selected_both_correct": 7252, + "selected_both_wrong": 428 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8764980026631158, + "metric": 0.24058030940047917, + "rescued": 1000, + "harmed": 1834, + "missed_rescue": 198, + "avoided_harm": 275, + "selected_both_correct": 7268, + "selected_both_wrong": 430 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8841544607190412, + "metric": 0.24227931512045134, + "rescued": 1017, + "harmed": 1852, + "missed_rescue": 181, + "avoided_harm": 257, + "selected_both_correct": 7321, + "selected_both_wrong": 434 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8891478029294274, + "metric": 0.2414044779527529, + "rescued": 1030, + "harmed": 1861, + "missed_rescue": 168, + "avoided_harm": 248, + "selected_both_correct": 7354, + "selected_both_wrong": 439 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.8981358189081226, + "metric": 0.24325574507940786, + "rescued": 1055, + "harmed": 1885, + "missed_rescue": 143, + "avoided_harm": 224, + "selected_both_correct": 7403, + "selected_both_wrong": 449 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9054593874833555, + "metric": 0.24437346394419443, + "rescued": 1069, + "harmed": 1898, + "missed_rescue": 129, + "avoided_harm": 211, + "selected_both_correct": 7455, + "selected_both_wrong": 458 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.24877865873563798, + "rescued": 1105, + "harmed": 1927, + "missed_rescue": 93, + "avoided_harm": 182, + "selected_both_correct": 7575, + "selected_both_wrong": 473 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.2523047077304935, + "rescued": 1113, + "harmed": 1932, + "missed_rescue": 85, + "avoided_harm": 177, + "selected_both_correct": 7597, + "selected_both_wrong": 478 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.9364181091877497, + "metric": 0.2527194116218445, + "rescued": 1129, + "harmed": 1959, + "missed_rescue": 69, + "avoided_harm": 150, + "selected_both_correct": 7676, + "selected_both_wrong": 488 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.953728362183755, + "metric": 0.24014411561115867, + "rescued": 1148, + "harmed": 2005, + "missed_rescue": 50, + "avoided_harm": 104, + "selected_both_correct": 7804, + "selected_both_wrong": 503 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.96271637816245, + "metric": 0.2331497088327334, + "rescued": 1153, + "harmed": 2028, + "missed_rescue": 45, + "avoided_harm": 81, + "selected_both_correct": 7880, + "selected_both_wrong": 507 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.9700399467376831, + "metric": 0.23063278452496924, + "rescued": 1156, + "harmed": 2044, + "missed_rescue": 42, + "avoided_harm": 65, + "selected_both_correct": 7943, + "selected_both_wrong": 513 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.987017310252996, + "metric": 0.2242415410777232, + "rescued": 1174, + "harmed": 2087, + "missed_rescue": 24, + "avoided_harm": 22, + "selected_both_correct": 8072, + "selected_both_wrong": 527 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.22361115727328065, + "rescued": 1198, + "harmed": 2109, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 8176, + "selected_both_wrong": 533 + } + ] + }, + "learned": { + "validation_area_0_25": 0.2305961827838297, + "validation_point": { + "metric": 0.2840018248312731, + "metric_std": 0.0024167487582089655, + "actual_fraction": 0.46733238231098434, + "actual_fraction_std": 0.028594792423091146, + "per_seed": [ + { + "seed": 2026, + "budget": 0.47, + "threshold": -0.10248661041259766, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.2828447999106682, + "rescue": 271, + "harm": 210, + "missed_rescue": 15 + }, + { + "seed": 2027, + "budget": 0.43, + "threshold": -0.06325958669185638, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.2848598219459613, + "rescue": 262, + "harm": 189, + "missed_rescue": 24 + }, + { + "seed": 2028, + "budget": 0.47, + "threshold": -0.08103815466165543, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.28728969883242916, + "rescue": 274, + "harm": 211, + "missed_rescue": 12 + }, + { + "seed": 2029, + "budget": 0.46, + "threshold": -0.10648070275783539, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.2807737471705083, + "rescue": 270, + "harm": 203, + "missed_rescue": 16 + }, + { + "seed": 2030, + "budget": 0.51, + "threshold": -0.1446637064218521, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.28424105629679847, + "rescue": 273, + "harm": 224, + "missed_rescue": 13 + } + ], + "rescued": 270.0, + "harmed": 207.4, + "missed_rescue": 16.0, + "avoided_harm": 310.6 + }, + "test_point": { + "metric": 0.25938275238549513, + "metric_std": 0.004678859568347879, + "actual_fraction": 0.4645139813581891, + "actual_fraction_std": 0.028448517663268665, + "per_seed": [ + { + "seed": 2026, + "budget": 0.47, + "threshold": -0.10248661041259766, + "operator": "ge", + "actual_fraction": 0.4663781624500666, + "metric": 0.26215437873972275, + "rescue": 1128, + "harm": 801, + "missed_rescue": 70 + }, + { + "seed": 2027, + "budget": 0.43, + "threshold": -0.06325958669185638, + "operator": "ge", + "actual_fraction": 0.42809587217043943, + "metric": 0.2635851498163025, + "rescue": 1102, + "harm": 748, + "missed_rescue": 96 + }, + { + "seed": 2028, + "budget": 0.47, + "threshold": -0.08103815466165543, + "operator": "ge", + "actual_fraction": 0.46238348868175766, + "metric": 0.26237361710647344, + "rescue": 1133, + "harm": 799, + "missed_rescue": 65 + }, + { + "seed": 2029, + "budget": 0.46, + "threshold": -0.10648070275783539, + "operator": "ge", + "actual_fraction": 0.4580559254327563, + "metric": 0.255747541850064, + "rescue": 1125, + "harm": 797, + "missed_rescue": 73 + }, + { + "seed": 2030, + "budget": 0.51, + "threshold": -0.1446637064218521, + "operator": "ge", + "actual_fraction": 0.5076564580559254, + "metric": 0.25305307441491287, + "rescue": 1141, + "harm": 894, + "missed_rescue": 57 + } + ], + "rescued": 1125.8, + "harmed": 807.8, + "missed_rescue": 72.2, + "avoided_harm": 1301.2 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.18254072168965788, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.1855451431216153, + "metric_std": 0.0005064299326898716, + "actual_fraction": 0.009985734664764621, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.18801222189991562, + "metric_std": 0.0005533628656158161, + "actual_fraction": 0.019971469329529243, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.19516253031361294, + "metric_std": 0.0033201216286841617, + "actual_fraction": 0.029957203994293864, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.20206588499854156, + "metric_std": 0.0011026134230121276, + "actual_fraction": 0.039942938659058486, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.20567151125193198, + "metric_std": 0.0007596946664713425, + "actual_fraction": 0.04992867332382311, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.20970336373997722, + "metric_std": 0.0006431965432207254, + "actual_fraction": 0.05991440798858773, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.2143700600506107, + "metric_std": 0.0004492779826626765, + "actual_fraction": 0.06990014265335236, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.21910800744037492, + "metric_std": 0.0009199895882873678, + "actual_fraction": 0.07988587731811697, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.22338284356491092, + "metric_std": 0.0005434491713586444, + "actual_fraction": 0.0898716119828816, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.22640049009641433, + "metric_std": 0.0011590677684995068, + "actual_fraction": 0.09985734664764621, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.22849542698040767, + "metric_std": 0.005529697905662961, + "actual_fraction": 0.10984308131241086, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.12, + "metric": 0.23036768069162825, + "metric_std": 0.004658212065576742, + "actual_fraction": 0.11982881597717546, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.23647598977509574, + "metric_std": 0.004565367170458658, + "actual_fraction": 0.12981455064194009, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.23958043514788882, + "metric_std": 0.0054475877649030014, + "actual_fraction": 0.1398002853067047, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.24255731462435276, + "metric_std": 0.004824141861360469, + "actual_fraction": 0.14978601997146934, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.24321490837895748, + "metric_std": 0.006614120310287118, + "actual_fraction": 0.15977175463623394, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.24573027983280887, + "metric_std": 0.007589324821924352, + "actual_fraction": 0.16975748930099857, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.24858985377141352, + "metric_std": 0.006895478909596356, + "actual_fraction": 0.1797432239657632, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.2530611516185174, + "metric_std": 0.008465480365368601, + "actual_fraction": 0.18972895863052783, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.25587734278912533, + "metric_std": 0.010043650581855891, + "actual_fraction": 0.19971469329529243, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.2587352208591439, + "metric_std": 0.010909398071691806, + "actual_fraction": 0.20970042796005703, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.25990936249433566, + "metric_std": 0.011521782207675654, + "actual_fraction": 0.2196861626248217, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.23, + "metric": 0.2621239707381077, + "metric_std": 0.011726587990825979, + "actual_fraction": 0.2296718972895863, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.24, + "metric": 0.26204830634332066, + "metric_std": 0.011997482392718016, + "actual_fraction": 0.2396576319543509, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.26240778862499053, + "metric_std": 0.007013822626378713, + "actual_fraction": 0.24964336661911551, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.26088200766058955, + "metric_std": 0.012627541599374936, + "actual_fraction": 0.25962910128388017, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.2593066971530801, + "metric_std": 0.010722310432288096, + "actual_fraction": 0.26961483594864477, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.26029247737135996, + "metric_std": 0.009910159266796233, + "actual_fraction": 0.2796005706134094, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.2614740494511477, + "metric_std": 0.009987249705975344, + "actual_fraction": 0.289586305278174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.26222081690220034, + "metric_std": 0.009848331814908632, + "actual_fraction": 0.2995720399429387, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.26376958641510806, + "metric_std": 0.009448079438830798, + "actual_fraction": 0.3095577746077033, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.26436076281169, + "metric_std": 0.01064397395873195, + "actual_fraction": 0.3195435092724679, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.2659878097681742, + "metric_std": 0.011372712000497524, + "actual_fraction": 0.32952924393723254, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.26308538513821356, + "metric_std": 0.0076325926609994875, + "actual_fraction": 0.33951497860199714, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.2653439023014602, + "metric_std": 0.004368654348912257, + "actual_fraction": 0.34950071326676174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.2668342889907559, + "metric_std": 0.0040625334840454835, + "actual_fraction": 0.3594864479315264, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.2677035772284784, + "metric_std": 0.004151621523524204, + "actual_fraction": 0.369472182596291, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.2697877027747386, + "metric_std": 0.003652183292739056, + "actual_fraction": 0.37945791726105566, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.27085115597893566, + "metric_std": 0.005988349910976425, + "actual_fraction": 0.38944365192582026, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.2719349519922599, + "metric_std": 0.005612719472161027, + "actual_fraction": 0.39942938659058486, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.27394131696053786, + "metric_std": 0.0049511023785165295, + "actual_fraction": 0.4094151212553495, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.42, + "metric": 0.2760569513604888, + "metric_std": 0.004738734181255325, + "actual_fraction": 0.41940085592011406, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.279944515994212, + "metric_std": 0.004508399753318105, + "actual_fraction": 0.4293865905848787, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.44, + "metric": 0.28022090909719255, + "metric_std": 0.003721560899703926, + "actual_fraction": 0.4393723252496434, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.45, + "metric": 0.2816052359123282, + "metric_std": 0.0013859294182433294, + "actual_fraction": 0.44935805991440797, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.2824950668078987, + "metric_std": 0.0017434607003036374, + "actual_fraction": 0.4593437945791726, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.47, + "metric": 0.28213557939734535, + "metric_std": 0.0038599779514378197, + "actual_fraction": 0.4693295292439372, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.48, + "metric": 0.28132830943340503, + "metric_std": 0.004128759032982044, + "actual_fraction": 0.4793152639087018, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.28051732449564226, + "metric_std": 0.0041758312339796435, + "actual_fraction": 0.4893009985734665, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.5, + "metric": 0.28130158301602987, + "metric_std": 0.0035025807499149994, + "actual_fraction": 0.49928673323823103, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.51, + "metric": 0.2820814793750177, + "metric_std": 0.0030745964742282905, + "actual_fraction": 0.5092724679029957, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.28086215990297425, + "metric_std": 0.002624166122625179, + "actual_fraction": 0.5192582025677603, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.27992794991173986, + "metric_std": 0.0032401154619943247, + "actual_fraction": 0.5292439372325249, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.27811303093893824, + "metric_std": 0.0034994492978040284, + "actual_fraction": 0.5392296718972895, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.27584032802947955, + "metric_std": 0.005172145017599763, + "actual_fraction": 0.5492154065620543, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.27434019720185976, + "metric_std": 0.003228690630618303, + "actual_fraction": 0.5592011412268189, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.27284786481450396, + "metric_std": 0.0039670040461793465, + "actual_fraction": 0.5691868758915835, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.27047599332516425, + "metric_std": 0.005787464867160034, + "actual_fraction": 0.579172610556348, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.26752953076839076, + "metric_std": 0.006254814464251354, + "actual_fraction": 0.5891583452211127, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.2652753699845374, + "metric_std": 0.0065581167185016324, + "actual_fraction": 0.5991440798858774, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.2641329385474167, + "metric_std": 0.0060242045102393005, + "actual_fraction": 0.609129814550642, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.2626471813118446, + "metric_std": 0.00808573630857423, + "actual_fraction": 0.6191155492154066, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.26131352099091976, + "metric_std": 0.008009197530810365, + "actual_fraction": 0.6291012838801712, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.2582831646505966, + "metric_std": 0.006899984205360198, + "actual_fraction": 0.6390870185449358, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.2550814096394999, + "metric_std": 0.00607426119853444, + "actual_fraction": 0.6490727532097005, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.25381910260380575, + "metric_std": 0.0065602101714663024, + "actual_fraction": 0.6590584878744651, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.2534233406638672, + "metric_std": 0.007711578349123215, + "actual_fraction": 0.6690442225392297, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.25103447291448966, + "metric_std": 0.0060439332993141295, + "actual_fraction": 0.6790299572039943, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.24885779119628673, + "metric_std": 0.006682259479392259, + "actual_fraction": 0.6890156918687589, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.24670458456834582, + "metric_std": 0.005827867405121002, + "actual_fraction": 0.6990014265335235, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.2449081896051415, + "metric_std": 0.006639179163854115, + "actual_fraction": 0.7089871611982882, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.24316957276278547, + "metric_std": 0.005393059355168042, + "actual_fraction": 0.7189728958630528, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.24197361153689884, + "metric_std": 0.005633651199720119, + "actual_fraction": 0.7289586305278174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.24029209702056248, + "metric_std": 0.004485485839472714, + "actual_fraction": 0.738944365192582, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.2393902190344504, + "metric_std": 0.005044872341098788, + "actual_fraction": 0.7489300998573466, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.23727678820547132, + "metric_std": 0.006018006984847701, + "actual_fraction": 0.7589158345221113, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.2356337491626403, + "metric_std": 0.005809899290374404, + "actual_fraction": 0.7689015691868759, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.23381895542288703, + "metric_std": 0.005619970655326034, + "actual_fraction": 0.7788873038516405, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.23292275998121642, + "metric_std": 0.0055500405851094435, + "actual_fraction": 0.7888730385164051, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.2317529800721124, + "metric_std": 0.005485294875624933, + "actual_fraction": 0.7988587731811697, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.22952722150985, + "metric_std": 0.004067746405165031, + "actual_fraction": 0.8088445078459344, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.22714573044879197, + "metric_std": 0.004110307429098409, + "actual_fraction": 0.818830242510699, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.22694850225504165, + "metric_std": 0.004086588586033567, + "actual_fraction": 0.8288159771754635, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.22519036832906486, + "metric_std": 0.0030275488269503113, + "actual_fraction": 0.8388017118402281, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.2221526062281122, + "metric_std": 0.001993724500894231, + "actual_fraction": 0.8487874465049929, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.86, + "metric": 0.22011619567088386, + "metric_std": 0.0013725438697271622, + "actual_fraction": 0.8587731811697574, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.87, + "metric": 0.21888361893168456, + "metric_std": 0.0012281486279031612, + "actual_fraction": 0.8687589158345222, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.88, + "metric": 0.2186361061944599, + "metric_std": 0.0012803754124929177, + "actual_fraction": 0.8787446504992869, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.89, + "metric": 0.21817178951693544, + "metric_std": 0.0015523259726284658, + "actual_fraction": 0.8887303851640513, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.21785284562735593, + "metric_std": 0.0018182092430143275, + "actual_fraction": 0.8987161198288159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.21631769129028502, + "metric_std": 0.002933300095447149, + "actual_fraction": 0.9087018544935808, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.21579140885963147, + "metric_std": 0.003198569598920243, + "actual_fraction": 0.9186875891583453, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.93, + "metric": 0.21503776526907403, + "metric_std": 0.0026365986800850273, + "actual_fraction": 0.9286733238231099, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.21388133850192642, + "metric_std": 0.0019727606142934625, + "actual_fraction": 0.9386590584878745, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.95, + "metric": 0.21274781339506893, + "metric_std": 0.0026133318030421918, + "actual_fraction": 0.948644793152639, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.2122871428548939, + "metric_std": 0.0025253652335714646, + "actual_fraction": 0.9586305278174037, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.21187268628221587, + "metric_std": 0.00259347285399371, + "actual_fraction": 0.9686162624821684, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.98, + "metric": 0.2115632764863949, + "metric_std": 0.0022609707753641502, + "actual_fraction": 0.978601997146933, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.99, + "metric": 0.20979044994622004, + "metric_std": 0.0011437459866507713, + "actual_fraction": 0.9885877318116976, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.20666016863629594, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.18129624718318377, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.18250748039721212, + "metric_std": 0.0009644411455225839, + "actual_fraction": 0.008388814913448735, + "actual_fraction_std": 0.0028771358770082424 + }, + { + "budget": 0.02, + "metric": 0.18501247628370038, + "metric_std": 0.0014806518958539506, + "actual_fraction": 0.016378162450066576, + "actual_fraction_std": 0.003063668539239292 + }, + { + "budget": 0.03, + "metric": 0.1882323366456725, + "metric_std": 0.001617953276474261, + "actual_fraction": 0.024034620505992012, + "actual_fraction_std": 0.004219956848281148 + }, + { + "budget": 0.04, + "metric": 0.19120784928153392, + "metric_std": 0.0023298154432717924, + "actual_fraction": 0.0336884154460719, + "actual_fraction_std": 0.0051753341646857565 + }, + { + "budget": 0.05, + "metric": 0.1944391684680682, + "metric_std": 0.0019566445916691494, + "actual_fraction": 0.042276964047936086, + "actual_fraction_std": 0.0014510316057059513 + }, + { + "budget": 0.06, + "metric": 0.1993907619905855, + "metric_std": 0.0030552659361038357, + "actual_fraction": 0.05233022636484687, + "actual_fraction_std": 0.003256537177468868 + }, + { + "budget": 0.07, + "metric": 0.2019380207383441, + "metric_std": 0.0018309699221953964, + "actual_fraction": 0.06438082556591211, + "actual_fraction_std": 0.0036814024769505323 + }, + { + "budget": 0.08, + "metric": 0.20489117078350408, + "metric_std": 0.00075685702674752, + "actual_fraction": 0.07496671105193076, + "actual_fraction_std": 0.005839340762075573 + }, + { + "budget": 0.09, + "metric": 0.20849070638996783, + "metric_std": 0.001094817770884552, + "actual_fraction": 0.09161118508655126, + "actual_fraction_std": 0.004462457677346931 + }, + { + "budget": 0.1, + "metric": 0.21184627522436061, + "metric_std": 0.0005383777275185728, + "actual_fraction": 0.10752330226364845, + "actual_fraction_std": 0.0031668271621904847 + }, + { + "budget": 0.11, + "metric": 0.2136717885818597, + "metric_std": 0.00036791439353963334, + "actual_fraction": 0.11750998668442077, + "actual_fraction_std": 0.0015613900665191163 + }, + { + "budget": 0.12, + "metric": 0.21597840837444648, + "metric_std": 0.0006980874983323455, + "actual_fraction": 0.12822902796271637, + "actual_fraction_std": 0.0018742594221383847 + }, + { + "budget": 0.13, + "metric": 0.21851163980820365, + "metric_std": 0.0018902352254614532, + "actual_fraction": 0.13675099866844204, + "actual_fraction_std": 0.004443794033546001 + }, + { + "budget": 0.14, + "metric": 0.2210697203188019, + "metric_std": 0.0037064726093448894, + "actual_fraction": 0.14793608521970708, + "actual_fraction_std": 0.004531456145402299 + }, + { + "budget": 0.15, + "metric": 0.22711021858273547, + "metric_std": 0.005742184150001859, + "actual_fraction": 0.16205059920106524, + "actual_fraction_std": 0.012355985809970656 + }, + { + "budget": 0.16, + "metric": 0.22955402732177807, + "metric_std": 0.00395187485663966, + "actual_fraction": 0.16897470039946738, + "actual_fraction_std": 0.013018064408340989 + }, + { + "budget": 0.17, + "metric": 0.23127797344342277, + "metric_std": 0.004120801278735245, + "actual_fraction": 0.17802929427430092, + "actual_fraction_std": 0.01132656152316096 + }, + { + "budget": 0.18, + "metric": 0.23420191141691343, + "metric_std": 0.005094909965738097, + "actual_fraction": 0.18874833555259654, + "actual_fraction_std": 0.011342693100799484 + }, + { + "budget": 0.19, + "metric": 0.2348743878525282, + "metric_std": 0.007161312105913388, + "actual_fraction": 0.19713715046604527, + "actual_fraction_std": 0.01037686515069928 + }, + { + "budget": 0.2, + "metric": 0.23669913014821703, + "metric_std": 0.006910809465815879, + "actual_fraction": 0.20792276964047937, + "actual_fraction_std": 0.009310794401770166 + }, + { + "budget": 0.21, + "metric": 0.23964001791657594, + "metric_std": 0.005149530722092215, + "actual_fraction": 0.21544607190412784, + "actual_fraction_std": 0.007833166058442167 + }, + { + "budget": 0.22, + "metric": 0.24263914748951762, + "metric_std": 0.005505778418539659, + "actual_fraction": 0.22683089214380825, + "actual_fraction_std": 0.006977191986182025 + }, + { + "budget": 0.23, + "metric": 0.24445121031189934, + "metric_std": 0.006611114623952759, + "actual_fraction": 0.23828229027962716, + "actual_fraction_std": 0.005743670122150181 + }, + { + "budget": 0.24, + "metric": 0.2464389266765472, + "metric_std": 0.007888606508237939, + "actual_fraction": 0.2492010652463382, + "actual_fraction_std": 0.00762600322944146 + }, + { + "budget": 0.25, + "metric": 0.24758684851801935, + "metric_std": 0.007604153434298802, + "actual_fraction": 0.2579227696404794, + "actual_fraction_std": 0.007661522090720253 + }, + { + "budget": 0.26, + "metric": 0.2485503354878864, + "metric_std": 0.008310137451870517, + "actual_fraction": 0.2673768308921438, + "actual_fraction_std": 0.010201854226033525 + }, + { + "budget": 0.27, + "metric": 0.2492116774048331, + "metric_std": 0.009920882232440297, + "actual_fraction": 0.2770306258322237, + "actual_fraction_std": 0.01318469294106342 + }, + { + "budget": 0.28, + "metric": 0.253119779222599, + "metric_std": 0.009051119363592854, + "actual_fraction": 0.290612516644474, + "actual_fraction_std": 0.00878853447566183 + }, + { + "budget": 0.29, + "metric": 0.2566652984221664, + "metric_std": 0.008181605531479926, + "actual_fraction": 0.29973368841544606, + "actual_fraction_std": 0.007268891645249618 + }, + { + "budget": 0.3, + "metric": 0.2561831329441431, + "metric_std": 0.01007457873751603, + "actual_fraction": 0.30692410119840213, + "actual_fraction_std": 0.006982748656259331 + }, + { + "budget": 0.31, + "metric": 0.2575248138245454, + "metric_std": 0.009841045792394617, + "actual_fraction": 0.31984021304926763, + "actual_fraction_std": 0.005359417256324394 + }, + { + "budget": 0.32, + "metric": 0.25998551902538974, + "metric_std": 0.009692228154943807, + "actual_fraction": 0.3312916111850865, + "actual_fraction_std": 0.0058865930356544575 + }, + { + "budget": 0.33, + "metric": 0.2619203249576794, + "metric_std": 0.009778740679196497, + "actual_fraction": 0.34227696404793606, + "actual_fraction_std": 0.007596155744150676 + }, + { + "budget": 0.34, + "metric": 0.2620725755005845, + "metric_std": 0.009772432577507787, + "actual_fraction": 0.34946737683089213, + "actual_fraction_std": 0.0065419347241130995 + }, + { + "budget": 0.35, + "metric": 0.2615842244218646, + "metric_std": 0.009746346704759375, + "actual_fraction": 0.3567243675099866, + "actual_fraction_std": 0.005261339041313018 + }, + { + "budget": 0.36, + "metric": 0.2630331505743267, + "metric_std": 0.005915582311496638, + "actual_fraction": 0.3672436750998668, + "actual_fraction_std": 0.003036419241276001 + }, + { + "budget": 0.37, + "metric": 0.2637458033382704, + "metric_std": 0.005645100066291145, + "actual_fraction": 0.375699067909454, + "actual_fraction_std": 0.0031457615592350146 + }, + { + "budget": 0.38, + "metric": 0.26476053758156337, + "metric_std": 0.004635228494718565, + "actual_fraction": 0.3847536617842876, + "actual_fraction_std": 0.0025935487590921986 + }, + { + "budget": 0.39, + "metric": 0.2645981573598934, + "metric_std": 0.005518595642648873, + "actual_fraction": 0.3931424766977364, + "actual_fraction_std": 0.0024462281053094243 + }, + { + "budget": 0.4, + "metric": 0.2650598521942218, + "metric_std": 0.0048209725761537954, + "actual_fraction": 0.40139813581890804, + "actual_fraction_std": 0.004140427920302559 + }, + { + "budget": 0.41, + "metric": 0.26497038088953256, + "metric_std": 0.005613650634693715, + "actual_fraction": 0.41331557922769646, + "actual_fraction_std": 0.004188326494687509 + }, + { + "budget": 0.42, + "metric": 0.2635465854026759, + "metric_std": 0.005498016169685908, + "actual_fraction": 0.42010652463382153, + "actual_fraction_std": 0.006263278203138077 + }, + { + "budget": 0.43, + "metric": 0.26273287794373085, + "metric_std": 0.0036670013238243836, + "actual_fraction": 0.4287616511318242, + "actual_fraction_std": 0.007402581876809969 + }, + { + "budget": 0.44, + "metric": 0.25991097541489017, + "metric_std": 0.0038592498700754105, + "actual_fraction": 0.44194407456724366, + "actual_fraction_std": 0.004675882769485585 + }, + { + "budget": 0.45, + "metric": 0.2597735349048862, + "metric_std": 0.003780257582789684, + "actual_fraction": 0.45086551264980035, + "actual_fraction_std": 0.004952115651568334 + }, + { + "budget": 0.46, + "metric": 0.25914813652422636, + "metric_std": 0.003350500822533708, + "actual_fraction": 0.45905459387483355, + "actual_fraction_std": 0.0018683375766181598 + }, + { + "budget": 0.47, + "metric": 0.2586488316104785, + "metric_std": 0.0033475344622284865, + "actual_fraction": 0.4647137150466046, + "actual_fraction_std": 0.0022330905234684913 + }, + { + "budget": 0.48, + "metric": 0.25828865005372903, + "metric_std": 0.003391733718603162, + "actual_fraction": 0.4743675099866844, + "actual_fraction_std": 0.004459973704052689 + }, + { + "budget": 0.49, + "metric": 0.25728002079322737, + "metric_std": 0.00375989024981149, + "actual_fraction": 0.4824900133155793, + "actual_fraction_std": 0.0038361811693443143 + }, + { + "budget": 0.5, + "metric": 0.2550532766959338, + "metric_std": 0.0033245516023756192, + "actual_fraction": 0.49567243675099865, + "actual_fraction_std": 0.00372181753911415 + }, + { + "budget": 0.51, + "metric": 0.2548733441061287, + "metric_std": 0.0031213474368316993, + "actual_fraction": 0.5047270306258322, + "actual_fraction_std": 0.004487221783556476 + }, + { + "budget": 0.52, + "metric": 0.2537392273895381, + "metric_std": 0.003095384994962551, + "actual_fraction": 0.5173768308921438, + "actual_fraction_std": 0.0039416138789922055 + }, + { + "budget": 0.53, + "metric": 0.25275049434671304, + "metric_std": 0.0026682494642292142, + "actual_fraction": 0.5289613848202397, + "actual_fraction_std": 0.009570210726674477 + }, + { + "budget": 0.54, + "metric": 0.2539344252844016, + "metric_std": 0.004713784015715127, + "actual_fraction": 0.5410119840213049, + "actual_fraction_std": 0.007955988946093036 + }, + { + "budget": 0.55, + "metric": 0.25437867023905797, + "metric_std": 0.004568865691680407, + "actual_fraction": 0.5505992010652463, + "actual_fraction_std": 0.007406323389243317 + }, + { + "budget": 0.56, + "metric": 0.2531028065401387, + "metric_std": 0.00451195439387375, + "actual_fraction": 0.5608521970705725, + "actual_fraction_std": 0.005527368952065346 + }, + { + "budget": 0.57, + "metric": 0.252556481674515, + "metric_std": 0.003944572579249581, + "actual_fraction": 0.5723035952063915, + "actual_fraction_std": 0.007430224454644112 + }, + { + "budget": 0.58, + "metric": 0.2528470436956109, + "metric_std": 0.0062700669599500694, + "actual_fraction": 0.5806924101198402, + "actual_fraction_std": 0.007490383183647692 + }, + { + "budget": 0.59, + "metric": 0.25229411999523227, + "metric_std": 0.006393468034935942, + "actual_fraction": 0.5906790945406125, + "actual_fraction_std": 0.011443847571124044 + }, + { + "budget": 0.6, + "metric": 0.2544025851237041, + "metric_std": 0.0037243192294439367, + "actual_fraction": 0.6027962716378162, + "actual_fraction_std": 0.009223501908670178 + }, + { + "budget": 0.61, + "metric": 0.2534627445891557, + "metric_std": 0.0033641438083470013, + "actual_fraction": 0.6151131824234355, + "actual_fraction_std": 0.008455654923225865 + }, + { + "budget": 0.62, + "metric": 0.2531978465414907, + "metric_std": 0.0036902830790983125, + "actual_fraction": 0.6233688415446071, + "actual_fraction_std": 0.009322688625324026 + }, + { + "budget": 0.63, + "metric": 0.2515075361125158, + "metric_std": 0.003145484208562908, + "actual_fraction": 0.6361517976031956, + "actual_fraction_std": 0.009872292028187015 + }, + { + "budget": 0.64, + "metric": 0.24923987723491395, + "metric_std": 0.004627758739827187, + "actual_fraction": 0.648335552596538, + "actual_fraction_std": 0.010586199528040354 + }, + { + "budget": 0.65, + "metric": 0.247788144831785, + "metric_std": 0.004111326255914469, + "actual_fraction": 0.6586551264980026, + "actual_fraction_std": 0.011117212902003378 + }, + { + "budget": 0.66, + "metric": 0.24652263236267508, + "metric_std": 0.0036829484438922182, + "actual_fraction": 0.6719041278295607, + "actual_fraction_std": 0.01125835964026759 + }, + { + "budget": 0.67, + "metric": 0.24603750489868803, + "metric_std": 0.004405208384112475, + "actual_fraction": 0.6835552596537949, + "actual_fraction_std": 0.008890704660120825 + }, + { + "budget": 0.68, + "metric": 0.24429169560793446, + "metric_std": 0.005311909998290173, + "actual_fraction": 0.6934753661784286, + "actual_fraction_std": 0.011306977240294057 + }, + { + "budget": 0.69, + "metric": 0.242735695456975, + "metric_std": 0.0057443886875106764, + "actual_fraction": 0.70525965379494, + "actual_fraction_std": 0.011089767264172564 + }, + { + "budget": 0.7, + "metric": 0.24259543405492226, + "metric_std": 0.005562567772603744, + "actual_fraction": 0.7155126498002662, + "actual_fraction_std": 0.0110422004878165 + }, + { + "budget": 0.71, + "metric": 0.24271829599268221, + "metric_std": 0.005481081981203334, + "actual_fraction": 0.724167776298269, + "actual_fraction_std": 0.009649776472056924 + }, + { + "budget": 0.72, + "metric": 0.2409707434590942, + "metric_std": 0.004785597642973466, + "actual_fraction": 0.7316245006657789, + "actual_fraction_std": 0.012900057973679751 + }, + { + "budget": 0.73, + "metric": 0.2418878466805881, + "metric_std": 0.006307915354441106, + "actual_fraction": 0.7390812250332889, + "actual_fraction_std": 0.013034653084163818 + }, + { + "budget": 0.74, + "metric": 0.2414600492992251, + "metric_std": 0.00614499194282718, + "actual_fraction": 0.7464713715046605, + "actual_fraction_std": 0.0130052894900747 + }, + { + "budget": 0.75, + "metric": 0.24072278939146236, + "metric_std": 0.006596308486248245, + "actual_fraction": 0.754993342210386, + "actual_fraction_std": 0.011757645148102113 + }, + { + "budget": 0.76, + "metric": 0.24211089041657274, + "metric_std": 0.005691392476683869, + "actual_fraction": 0.7641145139813582, + "actual_fraction_std": 0.010275452261369097 + }, + { + "budget": 0.77, + "metric": 0.2407105394795269, + "metric_std": 0.005398138534240622, + "actual_fraction": 0.7731025299600534, + "actual_fraction_std": 0.008818744365293948 + }, + { + "budget": 0.78, + "metric": 0.24018916308468358, + "metric_std": 0.004443601434805938, + "actual_fraction": 0.7835552596537949, + "actual_fraction_std": 0.008209624238996205 + }, + { + "budget": 0.79, + "metric": 0.23906059907666938, + "metric_std": 0.0038630669744227583, + "actual_fraction": 0.7911451398135819, + "actual_fraction_std": 0.005472970339581199 + }, + { + "budget": 0.8, + "metric": 0.23823416165162473, + "metric_std": 0.003512431825003482, + "actual_fraction": 0.7990013315579227, + "actual_fraction_std": 0.005045225350754511 + }, + { + "budget": 0.81, + "metric": 0.2385185448873608, + "metric_std": 0.0037092350002102, + "actual_fraction": 0.8111850865512649, + "actual_fraction_std": 0.006605151435222698 + }, + { + "budget": 0.82, + "metric": 0.23716108575577222, + "metric_std": 0.0036970460433007027, + "actual_fraction": 0.8234354194407457, + "actual_fraction_std": 0.007107015000470379 + }, + { + "budget": 0.83, + "metric": 0.23765761667096746, + "metric_std": 0.005207868102867836, + "actual_fraction": 0.8342210386151796, + "actual_fraction_std": 0.00790919065946519 + }, + { + "budget": 0.84, + "metric": 0.23754052814111098, + "metric_std": 0.005916205775718, + "actual_fraction": 0.844607190412783, + "actual_fraction_std": 0.00810433781051858 + }, + { + "budget": 0.85, + "metric": 0.23709287295129106, + "metric_std": 0.005841698834635251, + "actual_fraction": 0.851797603195739, + "actual_fraction_std": 0.007647044522812915 + }, + { + "budget": 0.86, + "metric": 0.23692620470116438, + "metric_std": 0.00554476863125223, + "actual_fraction": 0.8612516644474034, + "actual_fraction_std": 0.008148655808678709 + }, + { + "budget": 0.87, + "metric": 0.2371993581177791, + "metric_std": 0.004982465249971163, + "actual_fraction": 0.8729693741677762, + "actual_fraction_std": 0.007287923179223928 + }, + { + "budget": 0.88, + "metric": 0.23690711027120787, + "metric_std": 0.004778700488710886, + "actual_fraction": 0.8822237017310253, + "actual_fraction_std": 0.009709308223903585 + }, + { + "budget": 0.89, + "metric": 0.23691663126286935, + "metric_std": 0.005282357369731843, + "actual_fraction": 0.8902796271637815, + "actual_fraction_std": 0.00991597247169331 + }, + { + "budget": 0.9, + "metric": 0.2360542543664029, + "metric_std": 0.0056362390968784214, + "actual_fraction": 0.900399467376831, + "actual_fraction_std": 0.00954818489603146 + }, + { + "budget": 0.91, + "metric": 0.2364046976979693, + "metric_std": 0.005365029935947709, + "actual_fraction": 0.9099201065246338, + "actual_fraction_std": 0.008815602332071624 + }, + { + "budget": 0.92, + "metric": 0.2357643406737131, + "metric_std": 0.005312295738185327, + "actual_fraction": 0.9206391478029294, + "actual_fraction_std": 0.007956685343218611 + }, + { + "budget": 0.93, + "metric": 0.235694415020906, + "metric_std": 0.0036834826538414515, + "actual_fraction": 0.9296937416777631, + "actual_fraction_std": 0.011309427130306212 + }, + { + "budget": 0.94, + "metric": 0.23421299715996763, + "metric_std": 0.004284002434132839, + "actual_fraction": 0.9422103861517975, + "actual_fraction_std": 0.011094762430077983 + }, + { + "budget": 0.95, + "metric": 0.23354222880631886, + "metric_std": 0.0041714418254096015, + "actual_fraction": 0.9484021304926765, + "actual_fraction_std": 0.010354414790276809 + }, + { + "budget": 0.96, + "metric": 0.23261580532880638, + "metric_std": 0.004161531374134907, + "actual_fraction": 0.9593209054593874, + "actual_fraction_std": 0.007762109019820165 + }, + { + "budget": 0.97, + "metric": 0.23066530863616283, + "metric_std": 0.004261254035068348, + "actual_fraction": 0.9671770972037284, + "actual_fraction_std": 0.008019113436545895 + }, + { + "budget": 0.98, + "metric": 0.22937868479748325, + "metric_std": 0.004667337964491216, + "actual_fraction": 0.9764314247669773, + "actual_fraction_std": 0.0071412358877771046 + }, + { + "budget": 0.99, + "metric": 0.22681745033532125, + "metric_std": 0.0018862677895069944, + "actual_fraction": 0.9876165113182424, + "actual_fraction_std": 0.0032565371774688546 + }, + { + "budget": 1.0, + "metric": 0.22361115727328068, + "metric_std": 3.1031676915590914e-17, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 1891, + "rescuable": 286, + "harmful": 518, + "both_wrong": 109, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 8176, + "rescuable": 1198, + "harmful": 2109, + "both_wrong": 533, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.18254072168965788, + "test": 0.18129624718318377 + }, + "large": { + "validation": 0.20666016863629594, + "test": 0.22361115727328065 + }, + "learned_minus_confidence_test": 0.007078044655001614, + "learned_minus_small_test": 0.07808650520231136, + "learned_minus_large_test": 0.03577159511221448, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_yolo26x_MiniCPM" + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "expert_display": "YOLO26x", + "run": "five_seed_yolo26x_Qwen3.8_four_difference", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 93, + "validation_area_0_25": 0.17942553980644182, + "validation_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + "test_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 498, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 526, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.17237009889855048, + "rescued": 0, + "harmed": 36, + "missed_rescue": 307, + "avoided_harm": 184, + "selected_both_correct": 552, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.17209204025345443, + "rescued": 0, + "harmed": 37, + "missed_rescue": 307, + "avoided_harm": 183, + "selected_both_correct": 578, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.1716297194027841, + "rescued": 0, + "harmed": 39, + "missed_rescue": 307, + "avoided_harm": 181, + "selected_both_correct": 604, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 629, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24821683309557774, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 652, + "selected_both_wrong": 2 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 681, + "selected_both_wrong": 2 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.268188302425107, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 705, + "selected_both_wrong": 2 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 734, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 762, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.16880965944961934, + "rescued": 0, + "harmed": 51, + "missed_rescue": 307, + "avoided_harm": 169, + "selected_both_correct": 787, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30813124108416545, + "metric": 0.16830460894456886, + "rescued": 0, + "harmed": 56, + "missed_rescue": 307, + "avoided_harm": 164, + "selected_both_correct": 806, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3181169757489301, + "metric": 0.16811053970066264, + "rescued": 0, + "harmed": 57, + "missed_rescue": 307, + "avoided_harm": 163, + "selected_both_correct": 833, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.1671811299174608, + "rescued": 0, + "harmed": 61, + "missed_rescue": 307, + "avoided_harm": 159, + "selected_both_correct": 861, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.16645634059427164, + "rescued": 0, + "harmed": 64, + "missed_rescue": 307, + "avoided_harm": 156, + "selected_both_correct": 886, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34807417974322397, + "metric": 0.16600354762332126, + "rescued": 0, + "harmed": 66, + "missed_rescue": 307, + "avoided_harm": 154, + "selected_both_correct": 908, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.16502549737843858, + "rescued": 0, + "harmed": 70, + "missed_rescue": 307, + "avoided_harm": 150, + "selected_both_correct": 936, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3666191155492154, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 72, + "missed_rescue": 307, + "avoided_harm": 148, + "selected_both_correct": 954, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37660485021398005, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 76, + "missed_rescue": 307, + "avoided_harm": 144, + "selected_both_correct": 978, + "selected_both_wrong": 2 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.16749463738508683, + "rescued": 1, + "harmed": 81, + "missed_rescue": 306, + "avoided_harm": 139, + "selected_both_correct": 1008, + "selected_both_wrong": 2 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.16607474321791724, + "rescued": 1, + "harmed": 87, + "missed_rescue": 306, + "avoided_harm": 133, + "selected_both_correct": 1030, + "selected_both_wrong": 2 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.16514206572141965, + "rescued": 1, + "harmed": 90, + "missed_rescue": 306, + "avoided_harm": 130, + "selected_both_correct": 1055, + "selected_both_wrong": 2 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.16444252141114554, + "rescued": 1, + "harmed": 93, + "missed_rescue": 306, + "avoided_harm": 127, + "selected_both_correct": 1080, + "selected_both_wrong": 2 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.1639139801849299, + "rescued": 1, + "harmed": 96, + "missed_rescue": 306, + "avoided_harm": 124, + "selected_both_correct": 1105, + "selected_both_wrong": 2 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.16237039262040495, + "rescued": 1, + "harmed": 103, + "missed_rescue": 306, + "avoided_harm": 117, + "selected_both_correct": 1126, + "selected_both_wrong": 2 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.16187473569847557, + "rescued": 1, + "harmed": 105, + "missed_rescue": 306, + "avoided_harm": 115, + "selected_both_correct": 1152, + "selected_both_wrong": 2 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.1640341119762699, + "rescued": 2, + "harmed": 110, + "missed_rescue": 305, + "avoided_harm": 110, + "selected_both_correct": 1174, + "selected_both_wrong": 2 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.46790299572039945, + "metric": 0.16656779979633995, + "rescued": 3, + "harmed": 113, + "missed_rescue": 304, + "avoided_harm": 107, + "selected_both_correct": 1194, + "selected_both_wrong": 2 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.16532395129425315, + "rescued": 3, + "harmed": 119, + "missed_rescue": 304, + "avoided_harm": 101, + "selected_both_correct": 1220, + "selected_both_wrong": 2 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.16467625831496033, + "rescued": 5, + "harmed": 123, + "missed_rescue": 302, + "avoided_harm": 97, + "selected_both_correct": 1241, + "selected_both_wrong": 3 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.16569589650413527, + "rescued": 11, + "harmed": 128, + "missed_rescue": 296, + "avoided_harm": 92, + "selected_both_correct": 1257, + "selected_both_wrong": 4 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.1671927745457157, + "rescued": 18, + "harmed": 129, + "missed_rescue": 289, + "avoided_harm": 91, + "selected_both_correct": 1276, + "selected_both_wrong": 5 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5178316690442225, + "metric": 0.16789087940773334, + "rescued": 23, + "harmed": 131, + "missed_rescue": 284, + "avoided_harm": 89, + "selected_both_correct": 1293, + "selected_both_wrong": 5 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.16946103210148153, + "rescued": 30, + "harmed": 132, + "missed_rescue": 277, + "avoided_harm": 88, + "selected_both_correct": 1316, + "selected_both_wrong": 6 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5378031383737518, + "metric": 0.17052688690619724, + "rescued": 35, + "harmed": 133, + "missed_rescue": 272, + "avoided_harm": 87, + "selected_both_correct": 1334, + "selected_both_wrong": 6 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.17103667850460152, + "rescued": 42, + "harmed": 134, + "missed_rescue": 265, + "avoided_harm": 86, + "selected_both_correct": 1358, + "selected_both_wrong": 6 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.17291017019862398, + "rescued": 49, + "harmed": 135, + "missed_rescue": 258, + "avoided_harm": 85, + "selected_both_correct": 1376, + "selected_both_wrong": 8 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.17468909977784217, + "rescued": 55, + "harmed": 135, + "missed_rescue": 252, + "avoided_harm": 85, + "selected_both_correct": 1397, + "selected_both_wrong": 9 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.1754433851238893, + "rescued": 60, + "harmed": 136, + "missed_rescue": 247, + "avoided_harm": 84, + "selected_both_correct": 1416, + "selected_both_wrong": 12 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5877318116975749, + "metric": 0.17567949038614306, + "rescued": 65, + "harmed": 140, + "missed_rescue": 242, + "avoided_harm": 80, + "selected_both_correct": 1430, + "selected_both_wrong": 13 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.5977175463623395, + "metric": 0.17726242688944494, + "rescued": 73, + "harmed": 142, + "missed_rescue": 234, + "avoided_harm": 78, + "selected_both_correct": 1448, + "selected_both_wrong": 13 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.17927753989081613, + "rescued": 81, + "harmed": 143, + "missed_rescue": 226, + "avoided_harm": 77, + "selected_both_correct": 1471, + "selected_both_wrong": 13 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6176890156918687, + "metric": 0.1803426895170312, + "rescued": 86, + "harmed": 144, + "missed_rescue": 221, + "avoided_harm": 76, + "selected_both_correct": 1489, + "selected_both_wrong": 13 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.18256297675338154, + "rescued": 94, + "harmed": 145, + "missed_rescue": 213, + "avoided_harm": 75, + "selected_both_correct": 1509, + "selected_both_wrong": 16 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.18741054744283442, + "rescued": 103, + "harmed": 149, + "missed_rescue": 204, + "avoided_harm": 71, + "selected_both_correct": 1524, + "selected_both_wrong": 16 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6476462196861626, + "metric": 0.18874453577558464, + "rescued": 109, + "harmed": 150, + "missed_rescue": 198, + "avoided_harm": 70, + "selected_both_correct": 1540, + "selected_both_wrong": 17 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6562054208273894, + "metric": 0.18973030032445837, + "rescued": 114, + "harmed": 152, + "missed_rescue": 193, + "avoided_harm": 68, + "selected_both_correct": 1556, + "selected_both_wrong": 18 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.1910322331722189, + "rescued": 122, + "harmed": 157, + "missed_rescue": 185, + "avoided_harm": 63, + "selected_both_correct": 1576, + "selected_both_wrong": 21 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.19273815380416676, + "rescued": 128, + "harmed": 158, + "missed_rescue": 179, + "avoided_harm": 62, + "selected_both_correct": 1595, + "selected_both_wrong": 23 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.19362404817095658, + "rescued": 133, + "harmed": 161, + "missed_rescue": 174, + "avoided_harm": 59, + "selected_both_correct": 1612, + "selected_both_wrong": 26 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.6975748930099858, + "metric": 0.19489832797986031, + "rescued": 138, + "harmed": 163, + "missed_rescue": 169, + "avoided_harm": 57, + "selected_both_correct": 1626, + "selected_both_wrong": 29 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.19709243659719292, + "rescued": 147, + "harmed": 166, + "missed_rescue": 160, + "avoided_harm": 54, + "selected_both_correct": 1646, + "selected_both_wrong": 29 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.20000323882160753, + "rescued": 155, + "harmed": 166, + "missed_rescue": 152, + "avoided_harm": 54, + "selected_both_correct": 1666, + "selected_both_wrong": 29 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2015690267335004, + "rescued": 160, + "harmed": 167, + "missed_rescue": 147, + "avoided_harm": 53, + "selected_both_correct": 1684, + "selected_both_wrong": 33 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2034984942593638, + "rescued": 166, + "harmed": 168, + "missed_rescue": 141, + "avoided_harm": 52, + "selected_both_correct": 1703, + "selected_both_wrong": 35 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.20679879821670866, + "rescued": 176, + "harmed": 170, + "missed_rescue": 131, + "avoided_harm": 50, + "selected_both_correct": 1718, + "selected_both_wrong": 36 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.20972162293572166, + "rescued": 184, + "harmed": 171, + "missed_rescue": 123, + "avoided_harm": 49, + "selected_both_correct": 1735, + "selected_both_wrong": 38 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.2126226551226551, + "rescued": 191, + "harmed": 174, + "missed_rescue": 116, + "avoided_harm": 46, + "selected_both_correct": 1752, + "selected_both_wrong": 39 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.21516614155218836, + "rescued": 198, + "harmed": 175, + "missed_rescue": 109, + "avoided_harm": 45, + "selected_both_correct": 1771, + "selected_both_wrong": 40 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2162880860385455, + "rescued": 204, + "harmed": 177, + "missed_rescue": 103, + "avoided_harm": 43, + "selected_both_correct": 1788, + "selected_both_wrong": 43 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.21779120256650406, + "rescued": 209, + "harmed": 179, + "missed_rescue": 98, + "avoided_harm": 41, + "selected_both_correct": 1806, + "selected_both_wrong": 46 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.2201843117420812, + "rescued": 216, + "harmed": 182, + "missed_rescue": 91, + "avoided_harm": 38, + "selected_both_correct": 1820, + "selected_both_wrong": 50 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.22188537527520574, + "rescued": 222, + "harmed": 184, + "missed_rescue": 85, + "avoided_harm": 36, + "selected_both_correct": 1838, + "selected_both_wrong": 52 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.2260242392475367, + "rescued": 231, + "harmed": 187, + "missed_rescue": 76, + "avoided_harm": 33, + "selected_both_correct": 1854, + "selected_both_wrong": 52 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2282665192342907, + "rescued": 238, + "harmed": 190, + "missed_rescue": 69, + "avoided_harm": 30, + "selected_both_correct": 1869, + "selected_both_wrong": 55 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.23641538775702553, + "rescued": 246, + "harmed": 193, + "missed_rescue": 61, + "avoided_harm": 27, + "selected_both_correct": 1884, + "selected_both_wrong": 57 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2387681003514125, + "rescued": 251, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 27, + "selected_both_correct": 1904, + "selected_both_wrong": 60 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.24246700995212897, + "rescued": 258, + "harmed": 193, + "missed_rescue": 49, + "avoided_harm": 27, + "selected_both_correct": 1923, + "selected_both_wrong": 62 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.2423696527366133, + "rescued": 259, + "harmed": 197, + "missed_rescue": 48, + "avoided_harm": 23, + "selected_both_correct": 1941, + "selected_both_wrong": 67 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.24478740599615373, + "rescued": 266, + "harmed": 199, + "missed_rescue": 41, + "avoided_harm": 21, + "selected_both_correct": 1959, + "selected_both_wrong": 68 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.25736315359477124, + "rescued": 273, + "harmed": 199, + "missed_rescue": 34, + "avoided_harm": 21, + "selected_both_correct": 1977, + "selected_both_wrong": 71 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.9072753209700428, + "metric": 0.2597551406926407, + "rescued": 278, + "harmed": 200, + "missed_rescue": 29, + "avoided_harm": 20, + "selected_both_correct": 1992, + "selected_both_wrong": 74 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2629241291812946, + "rescued": 284, + "harmed": 201, + "missed_rescue": 23, + "avoided_harm": 19, + "selected_both_correct": 2013, + "selected_both_wrong": 78 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2639590174546567, + "rescued": 290, + "harmed": 203, + "missed_rescue": 17, + "avoided_harm": 17, + "selected_both_correct": 2057, + "selected_both_wrong": 82 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26132796745180287, + "rescued": 293, + "harmed": 207, + "missed_rescue": 14, + "avoided_harm": 13, + "selected_both_correct": 2078, + "selected_both_wrong": 82 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.2584345323365104, + "rescued": 296, + "harmed": 212, + "missed_rescue": 11, + "avoided_harm": 8, + "selected_both_correct": 2096, + "selected_both_wrong": 84 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2503146324574896, + "rescued": 299, + "harmed": 215, + "missed_rescue": 8, + "avoided_harm": 5, + "selected_both_correct": 2116, + "selected_both_wrong": 86 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.2545437366865938, + "rescued": 302, + "harmed": 216, + "missed_rescue": 5, + "avoided_harm": 4, + "selected_both_correct": 2139, + "selected_both_wrong": 87 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.2541444987255433, + "rescued": 304, + "harmed": 218, + "missed_rescue": 3, + "avoided_harm": 2, + "selected_both_correct": 2163, + "selected_both_wrong": 87 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.17942743009320905, + "metric": 0.1784481207469488, + "rescued": 4, + "harmed": 94, + "missed_rescue": 1278, + "avoided_harm": 683, + "selected_both_correct": 2057, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19340878828229027, + "metric": 0.1780928651788194, + "rescued": 4, + "harmed": 99, + "missed_rescue": 1278, + "avoided_harm": 678, + "selected_both_correct": 2220, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20306258322237017, + "metric": 0.17829989216687975, + "rescued": 5, + "harmed": 108, + "missed_rescue": 1277, + "avoided_harm": 669, + "selected_both_correct": 2325, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.1779854683890517, + "rescued": 5, + "harmed": 112, + "missed_rescue": 1277, + "avoided_harm": 665, + "selected_both_correct": 2385, + "selected_both_wrong": 2 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.21537949400798934, + "metric": 0.17917657728013692, + "rescued": 6, + "harmed": 118, + "missed_rescue": 1276, + "avoided_harm": 659, + "selected_both_correct": 2462, + "selected_both_wrong": 2 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.17803687432776102, + "rescued": 6, + "harmed": 133, + "missed_rescue": 1276, + "avoided_harm": 644, + "selected_both_correct": 2638, + "selected_both_wrong": 3 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24800266311584554, + "metric": 0.18026802885144627, + "rescued": 10, + "harmed": 153, + "missed_rescue": 1272, + "avoided_harm": 624, + "selected_both_correct": 2814, + "selected_both_wrong": 3 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.2589880159786951, + "metric": 0.18119827871024874, + "rescued": 11, + "harmed": 163, + "missed_rescue": 1271, + "avoided_harm": 614, + "selected_both_correct": 2935, + "selected_both_wrong": 3 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.27430093209054596, + "metric": 0.17963665085316138, + "rescued": 11, + "harmed": 185, + "missed_rescue": 1271, + "avoided_harm": 592, + "selected_both_correct": 3097, + "selected_both_wrong": 3 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.281624500665779, + "metric": 0.17924095839307458, + "rescued": 11, + "harmed": 191, + "missed_rescue": 1271, + "avoided_harm": 586, + "selected_both_correct": 3179, + "selected_both_wrong": 3 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.28761651131824234, + "metric": 0.1789157174618718, + "rescued": 11, + "harmed": 196, + "missed_rescue": 1271, + "avoided_harm": 581, + "selected_both_correct": 3246, + "selected_both_wrong": 3 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2969374167776298, + "metric": 0.1787206845800119, + "rescued": 11, + "harmed": 199, + "missed_rescue": 1271, + "avoided_harm": 578, + "selected_both_correct": 3355, + "selected_both_wrong": 3 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30459387483355527, + "metric": 0.17813242189706796, + "rescued": 11, + "harmed": 207, + "missed_rescue": 1271, + "avoided_harm": 570, + "selected_both_correct": 3438, + "selected_both_wrong": 4 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3129161118508655, + "metric": 0.17772000842137498, + "rescued": 12, + "harmed": 218, + "missed_rescue": 1270, + "avoided_harm": 559, + "selected_both_correct": 3526, + "selected_both_wrong": 4 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.31990679094540614, + "metric": 0.17791259734473341, + "rescued": 13, + "harmed": 227, + "missed_rescue": 1269, + "avoided_harm": 550, + "selected_both_correct": 3600, + "selected_both_wrong": 4 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.3322237017310253, + "metric": 0.1789653934599337, + "rescued": 15, + "harmed": 238, + "missed_rescue": 1267, + "avoided_harm": 539, + "selected_both_correct": 3735, + "selected_both_wrong": 4 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34054593874833555, + "metric": 0.18042682200794885, + "rescued": 16, + "harmed": 249, + "missed_rescue": 1266, + "avoided_harm": 528, + "selected_both_correct": 3822, + "selected_both_wrong": 5 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3488681757656458, + "metric": 0.17962877325912951, + "rescued": 16, + "harmed": 260, + "missed_rescue": 1266, + "avoided_harm": 517, + "selected_both_correct": 3910, + "selected_both_wrong": 6 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3581890812250333, + "metric": 0.18158304685510065, + "rescued": 18, + "harmed": 271, + "missed_rescue": 1264, + "avoided_harm": 506, + "selected_both_correct": 4008, + "selected_both_wrong": 7 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37017310252996005, + "metric": 0.18069239723097547, + "rescued": 18, + "harmed": 285, + "missed_rescue": 1264, + "avoided_harm": 492, + "selected_both_correct": 4138, + "selected_both_wrong": 7 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.1812845803851709, + "rescued": 20, + "harmed": 292, + "missed_rescue": 1262, + "avoided_harm": 485, + "selected_both_correct": 4209, + "selected_both_wrong": 7 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.38348868175765644, + "metric": 0.1824781664689562, + "rescued": 21, + "harmed": 301, + "missed_rescue": 1261, + "avoided_harm": 476, + "selected_both_correct": 4279, + "selected_both_wrong": 7 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4021304926764314, + "metric": 0.18220594501399923, + "rescued": 23, + "harmed": 321, + "missed_rescue": 1259, + "avoided_harm": 456, + "selected_both_correct": 4480, + "selected_both_wrong": 8 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.41311584553928093, + "metric": 0.18162498930772586, + "rescued": 23, + "harmed": 329, + "missed_rescue": 1259, + "avoided_harm": 448, + "selected_both_correct": 4603, + "selected_both_wrong": 9 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42676431424766975, + "metric": 0.18066434967152634, + "rescued": 23, + "harmed": 351, + "missed_rescue": 1259, + "avoided_harm": 426, + "selected_both_correct": 4745, + "selected_both_wrong": 9 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.4324234354194407, + "metric": 0.1803913789289444, + "rescued": 23, + "harmed": 355, + "missed_rescue": 1259, + "avoided_harm": 422, + "selected_both_correct": 4809, + "selected_both_wrong": 9 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44107856191744343, + "metric": 0.17959468902865128, + "rescued": 23, + "harmed": 368, + "missed_rescue": 1259, + "avoided_harm": 409, + "selected_both_correct": 4900, + "selected_both_wrong": 9 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.45772303595206393, + "metric": 0.17863383514615866, + "rescued": 24, + "harmed": 387, + "missed_rescue": 1258, + "avoided_harm": 390, + "selected_both_correct": 5078, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.4717043941411451, + "metric": 0.17957308538379646, + "rescued": 26, + "harmed": 405, + "missed_rescue": 1256, + "avoided_harm": 372, + "selected_both_correct": 5225, + "selected_both_wrong": 12 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4826897470039947, + "metric": 0.17910245145856155, + "rescued": 27, + "harmed": 417, + "missed_rescue": 1255, + "avoided_harm": 360, + "selected_both_correct": 5344, + "selected_both_wrong": 12 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4896804260985353, + "metric": 0.17871833907402393, + "rescued": 29, + "harmed": 425, + "missed_rescue": 1253, + "avoided_harm": 352, + "selected_both_correct": 5416, + "selected_both_wrong": 14 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.5013315579227696, + "metric": 0.18015785955296285, + "rescued": 57, + "harmed": 431, + "missed_rescue": 1225, + "avoided_harm": 346, + "selected_both_correct": 5516, + "selected_both_wrong": 20 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5116511318242344, + "metric": 0.1810878167493375, + "rescued": 76, + "harmed": 435, + "missed_rescue": 1206, + "avoided_harm": 342, + "selected_both_correct": 5606, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5183089214380826, + "metric": 0.18177962418520882, + "rescued": 92, + "harmed": 439, + "missed_rescue": 1190, + "avoided_harm": 338, + "selected_both_correct": 5662, + "selected_both_wrong": 35 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5252996005326231, + "metric": 0.18246323539519374, + "rescued": 109, + "harmed": 445, + "missed_rescue": 1173, + "avoided_harm": 332, + "selected_both_correct": 5719, + "selected_both_wrong": 39 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5396138482023968, + "metric": 0.1839876166906015, + "rescued": 150, + "harmed": 461, + "missed_rescue": 1132, + "avoided_harm": 316, + "selected_both_correct": 5829, + "selected_both_wrong": 44 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5452729693741678, + "metric": 0.1846674213907964, + "rescued": 164, + "harmed": 464, + "missed_rescue": 1118, + "avoided_harm": 313, + "selected_both_correct": 5875, + "selected_both_wrong": 49 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.559254327563249, + "metric": 0.18676294233791663, + "rescued": 196, + "harmed": 473, + "missed_rescue": 1086, + "avoided_harm": 304, + "selected_both_correct": 5990, + "selected_both_wrong": 61 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.572237017310253, + "metric": 0.18938663221175195, + "rescued": 230, + "harmed": 483, + "missed_rescue": 1052, + "avoided_harm": 294, + "selected_both_correct": 6093, + "selected_both_wrong": 70 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.5862183754993342, + "metric": 0.1909571002676625, + "rescued": 267, + "harmed": 496, + "missed_rescue": 1015, + "avoided_harm": 281, + "selected_both_correct": 6208, + "selected_both_wrong": 73 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5998668442077231, + "metric": 0.1934300101790686, + "rescued": 299, + "harmed": 503, + "missed_rescue": 983, + "avoided_harm": 274, + "selected_both_correct": 6315, + "selected_both_wrong": 91 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.6191744340878829, + "metric": 0.1973701682130997, + "rescued": 350, + "harmed": 520, + "missed_rescue": 932, + "avoided_harm": 257, + "selected_both_correct": 6465, + "selected_both_wrong": 105 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.6264980026631158, + "metric": 0.1986727924998457, + "rescued": 370, + "harmed": 523, + "missed_rescue": 912, + "avoided_harm": 254, + "selected_both_correct": 6527, + "selected_both_wrong": 108 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6384820239680427, + "metric": 0.20186289191698192, + "rescued": 402, + "harmed": 532, + "missed_rescue": 880, + "avoided_harm": 245, + "selected_both_correct": 6621, + "selected_both_wrong": 117 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6461384820239681, + "metric": 0.20246031058520178, + "rescued": 425, + "harmed": 537, + "missed_rescue": 857, + "avoided_harm": 240, + "selected_both_correct": 6682, + "selected_both_wrong": 120 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.653462050599201, + "metric": 0.20349944137622245, + "rescued": 444, + "harmed": 539, + "missed_rescue": 838, + "avoided_harm": 238, + "selected_both_correct": 6744, + "selected_both_wrong": 125 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6597869507323568, + "metric": 0.20442990090722246, + "rescued": 461, + "harmed": 545, + "missed_rescue": 821, + "avoided_harm": 232, + "selected_both_correct": 6792, + "selected_both_wrong": 130 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6661118508655126, + "metric": 0.2044993942500819, + "rescued": 480, + "harmed": 556, + "missed_rescue": 802, + "avoided_harm": 221, + "selected_both_correct": 6834, + "selected_both_wrong": 134 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6734354194407457, + "metric": 0.2064379542733177, + "rescued": 500, + "harmed": 561, + "missed_rescue": 782, + "avoided_harm": 216, + "selected_both_correct": 6889, + "selected_both_wrong": 142 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6820905459387483, + "metric": 0.2065831206227621, + "rescued": 523, + "harmed": 566, + "missed_rescue": 759, + "avoided_harm": 211, + "selected_both_correct": 6958, + "selected_both_wrong": 149 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6927430093209055, + "metric": 0.20802802394105854, + "rescued": 553, + "harmed": 577, + "missed_rescue": 729, + "avoided_harm": 200, + "selected_both_correct": 7035, + "selected_both_wrong": 159 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.7090545938748336, + "metric": 0.21169680596278434, + "rescued": 602, + "harmed": 584, + "missed_rescue": 680, + "avoided_harm": 193, + "selected_both_correct": 7159, + "selected_both_wrong": 175 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7170439414114514, + "metric": 0.2139280527103987, + "rescued": 625, + "harmed": 588, + "missed_rescue": 657, + "avoided_harm": 189, + "selected_both_correct": 7220, + "selected_both_wrong": 183 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7243675099866844, + "metric": 0.2146900231025185, + "rescued": 647, + "harmed": 594, + "missed_rescue": 635, + "avoided_harm": 183, + "selected_both_correct": 7268, + "selected_both_wrong": 195 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7330226364846871, + "metric": 0.21653561577692484, + "rescued": 670, + "harmed": 599, + "missed_rescue": 612, + "avoided_harm": 178, + "selected_both_correct": 7338, + "selected_both_wrong": 201 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.7363515312916112, + "metric": 0.21721689648133474, + "rescued": 681, + "harmed": 601, + "missed_rescue": 601, + "avoided_harm": 176, + "selected_both_correct": 7365, + "selected_both_wrong": 201 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7486684420772304, + "metric": 0.22103445955201576, + "rescued": 718, + "harmed": 610, + "missed_rescue": 564, + "avoided_harm": 167, + "selected_both_correct": 7460, + "selected_both_wrong": 208 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.757323568575233, + "metric": 0.22340134248367294, + "rescued": 744, + "harmed": 618, + "missed_rescue": 538, + "avoided_harm": 159, + "selected_both_correct": 7522, + "selected_both_wrong": 216 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7639813581890812, + "metric": 0.2269322568854726, + "rescued": 767, + "harmed": 620, + "missed_rescue": 515, + "avoided_harm": 157, + "selected_both_correct": 7571, + "selected_both_wrong": 222 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7733022636484687, + "metric": 0.22757452798038888, + "rescued": 798, + "harmed": 628, + "missed_rescue": 484, + "avoided_harm": 149, + "selected_both_correct": 7636, + "selected_both_wrong": 230 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.784953395472703, + "metric": 0.227821050912831, + "rescued": 824, + "harmed": 634, + "missed_rescue": 458, + "avoided_harm": 143, + "selected_both_correct": 7728, + "selected_both_wrong": 246 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7939414114513982, + "metric": 0.22991865671924006, + "rescued": 852, + "harmed": 642, + "missed_rescue": 430, + "avoided_harm": 135, + "selected_both_correct": 7793, + "selected_both_wrong": 253 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8039280958721704, + "metric": 0.23486191981046528, + "rescued": 884, + "harmed": 646, + "missed_rescue": 398, + "avoided_harm": 131, + "selected_both_correct": 7868, + "selected_both_wrong": 262 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.8122503328894807, + "metric": 0.23803916047544685, + "rescued": 906, + "harmed": 655, + "missed_rescue": 376, + "avoided_harm": 122, + "selected_both_correct": 7924, + "selected_both_wrong": 275 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8215712383488681, + "metric": 0.24022947987632814, + "rescued": 933, + "harmed": 664, + "missed_rescue": 349, + "avoided_harm": 113, + "selected_both_correct": 7987, + "selected_both_wrong": 288 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388814913448736, + "metric": 0.24528627405740036, + "rescued": 988, + "harmed": 672, + "missed_rescue": 294, + "avoided_harm": 105, + "selected_both_correct": 8116, + "selected_both_wrong": 304 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8535286284953395, + "metric": 0.249537136326177, + "rescued": 1028, + "harmed": 680, + "missed_rescue": 254, + "avoided_harm": 97, + "selected_both_correct": 8224, + "selected_both_wrong": 324 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8638482023968043, + "metric": 0.25211074301800224, + "rescued": 1056, + "harmed": 688, + "missed_rescue": 226, + "avoided_harm": 89, + "selected_both_correct": 8299, + "selected_both_wrong": 337 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8745006657789614, + "metric": 0.2540042317315216, + "rescued": 1076, + "harmed": 695, + "missed_rescue": 206, + "avoided_harm": 82, + "selected_both_correct": 8389, + "selected_both_wrong": 348 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8764980026631158, + "metric": 0.2540996517958595, + "rescued": 1080, + "harmed": 698, + "missed_rescue": 202, + "avoided_harm": 79, + "selected_both_correct": 8404, + "selected_both_wrong": 350 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8841544607190412, + "metric": 0.25436194678023316, + "rescued": 1090, + "harmed": 705, + "missed_rescue": 192, + "avoided_harm": 72, + "selected_both_correct": 8468, + "selected_both_wrong": 361 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8891478029294274, + "metric": 0.2534339447803836, + "rescued": 1101, + "harmed": 709, + "missed_rescue": 181, + "avoided_harm": 68, + "selected_both_correct": 8506, + "selected_both_wrong": 368 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.8981358189081226, + "metric": 0.2563240880787248, + "rescued": 1126, + "harmed": 715, + "missed_rescue": 156, + "avoided_harm": 62, + "selected_both_correct": 8573, + "selected_both_wrong": 378 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9054593874833555, + "metric": 0.25551845731687656, + "rescued": 1143, + "harmed": 721, + "missed_rescue": 139, + "avoided_harm": 56, + "selected_both_correct": 8632, + "selected_both_wrong": 384 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.26306323674307786, + "rescued": 1189, + "harmed": 729, + "missed_rescue": 93, + "avoided_harm": 48, + "selected_both_correct": 8800, + "selected_both_wrong": 402 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.9364181091877497, + "metric": 0.26218185491449153, + "rescued": 1203, + "harmed": 738, + "missed_rescue": 79, + "avoided_harm": 39, + "selected_both_correct": 8897, + "selected_both_wrong": 414 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.953728362183755, + "metric": 0.26357052137555514, + "rescued": 1223, + "harmed": 754, + "missed_rescue": 59, + "avoided_harm": 23, + "selected_both_correct": 9055, + "selected_both_wrong": 428 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.96271637816245, + "metric": 0.26400290307863034, + "rescued": 1231, + "harmed": 758, + "missed_rescue": 51, + "avoided_harm": 19, + "selected_both_correct": 9150, + "selected_both_wrong": 429 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.9700399467376831, + "metric": 0.2647475724296899, + "rescued": 1237, + "harmed": 760, + "missed_rescue": 45, + "avoided_harm": 17, + "selected_both_correct": 9227, + "selected_both_wrong": 432 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.987017310252996, + "metric": 0.2649306125590882, + "rescued": 1257, + "harmed": 772, + "missed_rescue": 25, + "avoided_harm": 5, + "selected_both_correct": 9387, + "selected_both_wrong": 444 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + }, + "learned": { + "validation_area_0_25": 0.23894305491785256, + "validation_point": { + "metric": 0.32697453663947174, + "metric_std": 0.0035407217767539855, + "actual_fraction": 0.43138373751783166, + "actual_fraction_std": 0.020464666294355817, + "per_seed": [ + { + "seed": 2026, + "budget": 0.45, + "threshold": 0.009329468011856079, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.3281285807463818, + "rescue": 298, + "harm": 84, + "missed_rescue": 9 + }, + { + "seed": 2027, + "budget": 0.45, + "threshold": 0.013585388660430908, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.3270447636073488, + "rescue": 297, + "harm": 84, + "missed_rescue": 10 + }, + { + "seed": 2028, + "budget": 0.41, + "threshold": 0.039804454892873764, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.32187620296097463, + "rescue": 280, + "harm": 69, + "missed_rescue": 27 + }, + { + "seed": 2029, + "budget": 0.41, + "threshold": 0.0639844611287117, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.33167859760757346, + "rescue": 279, + "harm": 74, + "missed_rescue": 28 + }, + { + "seed": 2030, + "budget": 0.44, + "threshold": 0.0036868713796138763, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.32614453827508016, + "rescue": 289, + "harm": 81, + "missed_rescue": 18 + } + ], + "rescued": 288.6, + "harmed": 78.4, + "missed_rescue": 18.4, + "avoided_harm": 141.6 + }, + "test_point": { + "metric": 0.2987049451139474, + "metric_std": 0.002939193920775561, + "actual_fraction": 0.4330226364846871, + "actual_fraction_std": 0.015864550132763944, + "per_seed": [ + { + "seed": 2026, + "budget": 0.45, + "threshold": 0.009329468011856079, + "operator": "ge", + "actual_fraction": 0.4477363515312916, + "metric": 0.2992560160564012, + "rescue": 1233, + "harm": 316, + "missed_rescue": 49 + }, + { + "seed": 2027, + "budget": 0.45, + "threshold": 0.013585388660430908, + "operator": "ge", + "actual_fraction": 0.4444074567243675, + "metric": 0.2994603011846414, + "rescue": 1228, + "harm": 315, + "missed_rescue": 54 + }, + { + "seed": 2028, + "budget": 0.41, + "threshold": 0.039804454892873764, + "operator": "ge", + "actual_fraction": 0.41344873501997337, + "metric": 0.2952522823078242, + "rescue": 1172, + "harm": 284, + "missed_rescue": 110 + }, + { + "seed": 2029, + "budget": 0.41, + "threshold": 0.0639844611287117, + "operator": "ge", + "actual_fraction": 0.41844207723035953, + "metric": 0.2966589960709296, + "rescue": 1191, + "harm": 293, + "missed_rescue": 91 + }, + { + "seed": 2030, + "budget": 0.44, + "threshold": 0.0036868713796138763, + "operator": "ge", + "actual_fraction": 0.44107856191744343, + "metric": 0.3028971299499408, + "rescue": 1215, + "harm": 298, + "missed_rescue": 67 + } + ], + "rescued": 1207.8, + "harmed": 301.2, + "missed_rescue": 74.2, + "avoided_harm": 475.8 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.18254072168965788, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.18584072139306487, + "metric_std": 0.00040183979776355175, + "actual_fraction": 0.009985734664764621, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.19082139390495947, + "metric_std": 0.000776272761300659, + "actual_fraction": 0.019971469329529243, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.19533091906711636, + "metric_std": 0.0010811087495377488, + "actual_fraction": 0.029957203994293864, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.20148285520491865, + "metric_std": 0.00207510149474077, + "actual_fraction": 0.039942938659058486, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.20698341010997706, + "metric_std": 0.0018342613579979041, + "actual_fraction": 0.04992867332382311, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.2116372725268941, + "metric_std": 0.00154972369207631, + "actual_fraction": 0.05991440798858773, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.2199814781838306, + "metric_std": 0.00301589787258796, + "actual_fraction": 0.06990014265335236, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.22851724193786374, + "metric_std": 0.0012857899994321458, + "actual_fraction": 0.07988587731811697, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.23547342799412269, + "metric_std": 0.0004253646160305204, + "actual_fraction": 0.0898716119828816, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.23981447087825725, + "metric_std": 0.001812909800987374, + "actual_fraction": 0.09985734664764621, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.2440670335003016, + "metric_std": 0.0022915109001284957, + "actual_fraction": 0.10984308131241086, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.12, + "metric": 0.24830511412593825, + "metric_std": 0.004056136885706042, + "actual_fraction": 0.11982881597717546, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.25254044633752415, + "metric_std": 0.0040574610792162786, + "actual_fraction": 0.12981455064194009, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.25269357419618, + "metric_std": 0.005821005444317475, + "actual_fraction": 0.1398002853067047, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.2564395334371191, + "metric_std": 0.0069502943054834615, + "actual_fraction": 0.14978601997146934, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.25849925720288514, + "metric_std": 0.005789363098455391, + "actual_fraction": 0.15977175463623394, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.25949596780743583, + "metric_std": 0.006875934878651116, + "actual_fraction": 0.16975748930099857, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.2614581995892541, + "metric_std": 0.007162995732934064, + "actual_fraction": 0.1797432239657632, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.263289784186271, + "metric_std": 0.00821650454632698, + "actual_fraction": 0.18972895863052783, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.26603149992519576, + "metric_std": 0.008738877309225537, + "actual_fraction": 0.19971469329529243, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.26060122083120485, + "metric_std": 0.0033309348263470488, + "actual_fraction": 0.20970042796005703, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.26276817873770897, + "metric_std": 0.0029609057577578866, + "actual_fraction": 0.2196861626248217, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.23, + "metric": 0.26615284010941637, + "metric_std": 0.0030908246293656298, + "actual_fraction": 0.2296718972895863, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.24, + "metric": 0.26681400696152174, + "metric_std": 0.0020265476870299838, + "actual_fraction": 0.2396576319543509, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.26835199878776916, + "metric_std": 0.0024849580299274314, + "actual_fraction": 0.24964336661911551, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.2707705512635179, + "metric_std": 0.003680022746640243, + "actual_fraction": 0.25962910128388017, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.27294914216470667, + "metric_std": 0.004238919869871345, + "actual_fraction": 0.26961483594864477, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.2728250465518399, + "metric_std": 0.0029494555846525457, + "actual_fraction": 0.2796005706134094, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.2764554139556877, + "metric_std": 0.0031480479795532508, + "actual_fraction": 0.289586305278174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.2751690553914464, + "metric_std": 0.006862857515063747, + "actual_fraction": 0.2995720399429387, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.27373845022286064, + "metric_std": 0.00581867507067677, + "actual_fraction": 0.3095577746077033, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.2780754852027593, + "metric_std": 0.006275534265019767, + "actual_fraction": 0.3195435092724679, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.2822958500725304, + "metric_std": 0.008233323726544249, + "actual_fraction": 0.32952924393723254, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.28529304062893684, + "metric_std": 0.007884698713794772, + "actual_fraction": 0.33951497860199714, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.2878640622278018, + "metric_std": 0.006964944430266397, + "actual_fraction": 0.34950071326676174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.2929898110309905, + "metric_std": 0.008826843383621929, + "actual_fraction": 0.3594864479315264, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.29622253515410746, + "metric_std": 0.005048688419432268, + "actual_fraction": 0.369472182596291, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.30052289807333626, + "metric_std": 0.005277689151159374, + "actual_fraction": 0.37945791726105566, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.30239141184979507, + "metric_std": 0.008765831488534959, + "actual_fraction": 0.38944365192582026, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.30668373557655804, + "metric_std": 0.007498412626542974, + "actual_fraction": 0.39942938659058486, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.315967749019614, + "metric_std": 0.013851036269954192, + "actual_fraction": 0.4094151212553495, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.42, + "metric": 0.3175508682739074, + "metric_std": 0.010637753144487231, + "actual_fraction": 0.41940085592011406, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.31834553332730336, + "metric_std": 0.00968136289401714, + "actual_fraction": 0.4293865905848787, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.44, + "metric": 0.32102768552681393, + "metric_std": 0.00583390453069408, + "actual_fraction": 0.4393723252496434, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.45, + "metric": 0.32445943620661644, + "metric_std": 0.004555690143569261, + "actual_fraction": 0.44935805991440797, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.3239783147181333, + "metric_std": 0.004137662407291464, + "actual_fraction": 0.4593437945791726, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.47, + "metric": 0.3230159079805196, + "metric_std": 0.003485098489853575, + "actual_fraction": 0.4693295292439372, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.48, + "metric": 0.3225889030052027, + "metric_std": 0.0033576532170991238, + "actual_fraction": 0.4793152639087018, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.3202802945836932, + "metric_std": 0.0027595449158018606, + "actual_fraction": 0.4893009985734665, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.5, + "metric": 0.3192549747325758, + "metric_std": 0.0030747571406626877, + "actual_fraction": 0.49928673323823103, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.51, + "metric": 0.31870909509085876, + "metric_std": 0.0028844308741731613, + "actual_fraction": 0.5092724679029957, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.31727517065288646, + "metric_std": 0.0038882589313809896, + "actual_fraction": 0.5192582025677603, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.3164895358976282, + "metric_std": 0.0038673219429384044, + "actual_fraction": 0.5292439372325249, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.3156123632787352, + "metric_std": 0.004470344303961577, + "actual_fraction": 0.5392296718972895, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.3140381311209702, + "metric_std": 0.00447271768284129, + "actual_fraction": 0.5492154065620543, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.3127126991561658, + "metric_std": 0.0039208762143044425, + "actual_fraction": 0.5592011412268189, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.3111671251418948, + "metric_std": 0.004459436011985857, + "actual_fraction": 0.5691868758915835, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.30996100636476903, + "metric_std": 0.004940203557665433, + "actual_fraction": 0.579172610556348, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.3091302039188347, + "metric_std": 0.005395174803515626, + "actual_fraction": 0.5891583452211127, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.30884861556427545, + "metric_std": 0.006169999578965454, + "actual_fraction": 0.5991440798858774, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.30626801898638245, + "metric_std": 0.005578854374711873, + "actual_fraction": 0.609129814550642, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.3046110894978345, + "metric_std": 0.006147699164695165, + "actual_fraction": 0.6191155492154066, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.30309097080626446, + "metric_std": 0.005228252577723075, + "actual_fraction": 0.6291012838801712, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.30328468443610923, + "metric_std": 0.0021493866269632045, + "actual_fraction": 0.6390870185449358, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.3022039394743033, + "metric_std": 0.0023950802073987717, + "actual_fraction": 0.6490727532097005, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.30351438686480164, + "metric_std": 0.004444642863197173, + "actual_fraction": 0.6590584878744651, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.3028101847911976, + "metric_std": 0.004819985012415133, + "actual_fraction": 0.6690442225392297, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.30189421151235996, + "metric_std": 0.0044212334787356354, + "actual_fraction": 0.6790299572039943, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.3003340376290757, + "metric_std": 0.004430079911766776, + "actual_fraction": 0.6890156918687589, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.29881945281997896, + "metric_std": 0.005269748949995239, + "actual_fraction": 0.6990014265335235, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.29798463582573076, + "metric_std": 0.005225333658045543, + "actual_fraction": 0.7089871611982882, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.2967358291886303, + "metric_std": 0.004330383099317667, + "actual_fraction": 0.7189728958630528, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.29619424266840644, + "metric_std": 0.004900391337787624, + "actual_fraction": 0.7289586305278174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.2967235590098366, + "metric_std": 0.008453146840891162, + "actual_fraction": 0.738944365192582, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.29513958448890676, + "metric_std": 0.009384943281812554, + "actual_fraction": 0.7489300998573466, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.2926957652459374, + "metric_std": 0.010557234394713417, + "actual_fraction": 0.7589158345221113, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.29235464321699245, + "metric_std": 0.012412573542597493, + "actual_fraction": 0.7689015691868759, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.29028454850943686, + "metric_std": 0.012556190906271449, + "actual_fraction": 0.7788873038516405, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.2893397510978666, + "metric_std": 0.01166244349639195, + "actual_fraction": 0.7888730385164051, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.2891937559977261, + "metric_std": 0.008781863823843139, + "actual_fraction": 0.7988587731811697, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.2875236238888199, + "metric_std": 0.009773571972363202, + "actual_fraction": 0.8088445078459344, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.2849126598126858, + "metric_std": 0.009255799165253452, + "actual_fraction": 0.818830242510699, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.28491563554807214, + "metric_std": 0.008647836447804933, + "actual_fraction": 0.8288159771754635, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.28292270499112526, + "metric_std": 0.006275562835555672, + "actual_fraction": 0.8388017118402281, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.28092686483623736, + "metric_std": 0.0050801835216630255, + "actual_fraction": 0.8487874465049929, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.86, + "metric": 0.27777706526349355, + "metric_std": 0.004649182392890391, + "actual_fraction": 0.8587731811697574, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.87, + "metric": 0.2768274784678952, + "metric_std": 0.004358103426941459, + "actual_fraction": 0.8687589158345222, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.88, + "metric": 0.2756318328882888, + "metric_std": 0.0030423391233596795, + "actual_fraction": 0.8787446504992869, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.89, + "metric": 0.2735636753607668, + "metric_std": 0.00301034252671876, + "actual_fraction": 0.8887303851640513, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.27208106662386156, + "metric_std": 0.003761204326373389, + "actual_fraction": 0.8987161198288159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.2698429151247354, + "metric_std": 0.0032202742996738634, + "actual_fraction": 0.9087018544935808, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.26711420694994137, + "metric_std": 0.004845639714328782, + "actual_fraction": 0.9186875891583453, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.93, + "metric": 0.2657320125566297, + "metric_std": 0.004712319399696814, + "actual_fraction": 0.9286733238231099, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.2647632170249805, + "metric_std": 0.0039028850653929103, + "actual_fraction": 0.9386590584878745, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.95, + "metric": 0.2621779885858797, + "metric_std": 0.003619190640083323, + "actual_fraction": 0.948644793152639, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.2599418455566023, + "metric_std": 0.003626237533581548, + "actual_fraction": 0.9586305278174037, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.25962811904343563, + "metric_std": 0.004237112486180327, + "actual_fraction": 0.9686162624821684, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.98, + "metric": 0.25666431386537286, + "metric_std": 0.001600580642529223, + "actual_fraction": 0.978601997146933, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.99, + "metric": 0.25619928584193746, + "metric_std": 0.0016002217323731116, + "actual_fraction": 0.9885877318116976, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.25448626222520154, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.18129624718318377, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.18329085580101254, + "metric_std": 0.0009037325643508424, + "actual_fraction": 0.010319573901464712, + "actual_fraction_std": 0.003023618862281117 + }, + { + "budget": 0.02, + "metric": 0.18705179754534637, + "metric_std": 0.0041961682816421516, + "actual_fraction": 0.0244340878828229, + "actual_fraction_std": 0.00553838464529572 + }, + { + "budget": 0.03, + "metric": 0.1885792820457931, + "metric_std": 0.0035029275663606267, + "actual_fraction": 0.030559254327563245, + "actual_fraction_std": 0.0048492297562264045 + }, + { + "budget": 0.04, + "metric": 0.19071443977924615, + "metric_std": 0.0025578305400519286, + "actual_fraction": 0.038881491344873495, + "actual_fraction_std": 0.003892103036768047 + }, + { + "budget": 0.05, + "metric": 0.19402693980173558, + "metric_std": 0.0018244659000678832, + "actual_fraction": 0.050399467376830896, + "actual_fraction_std": 0.007363557450704493 + }, + { + "budget": 0.06, + "metric": 0.1980910599365008, + "metric_std": 0.001815898203866576, + "actual_fraction": 0.06165113182423435, + "actual_fraction_std": 0.005910077502309491 + }, + { + "budget": 0.07, + "metric": 0.2012273164434601, + "metric_std": 0.0010828189492063391, + "actual_fraction": 0.07363515312916112, + "actual_fraction_std": 0.007931576526019881 + }, + { + "budget": 0.08, + "metric": 0.20136804236084185, + "metric_std": 0.001645884457530363, + "actual_fraction": 0.08688415446071904, + "actual_fraction_std": 0.008157490515874066 + }, + { + "budget": 0.09, + "metric": 0.20338689284850514, + "metric_std": 0.0020775410833879975, + "actual_fraction": 0.09640479360852197, + "actual_fraction_std": 0.0031720717042800477 + }, + { + "budget": 0.1, + "metric": 0.20579620433613308, + "metric_std": 0.002525837126859493, + "actual_fraction": 0.10459387483355527, + "actual_fraction_std": 0.0036949232824265263 + }, + { + "budget": 0.11, + "metric": 0.20942439371696925, + "metric_std": 0.0018915178428864409, + "actual_fraction": 0.1163115845539281, + "actual_fraction_std": 0.0036359700188849887 + }, + { + "budget": 0.12, + "metric": 0.21128937085768912, + "metric_std": 0.0015628461358339328, + "actual_fraction": 0.12490013315579225, + "actual_fraction_std": 0.005159250004276794 + }, + { + "budget": 0.13, + "metric": 0.21631432851388094, + "metric_std": 0.005842838790406748, + "actual_fraction": 0.13641810918774966, + "actual_fraction_std": 0.008135045216567738 + }, + { + "budget": 0.14, + "metric": 0.21967808877813008, + "metric_std": 0.006384573214867751, + "actual_fraction": 0.14500665778961386, + "actual_fraction_std": 0.009313769381684983 + }, + { + "budget": 0.15, + "metric": 0.224629616057974, + "metric_std": 0.007336030963581105, + "actual_fraction": 0.1567243675099867, + "actual_fraction_std": 0.010185547772710664 + }, + { + "budget": 0.16, + "metric": 0.22781718347730529, + "metric_std": 0.00715541904151367, + "actual_fraction": 0.1659786950732357, + "actual_fraction_std": 0.01003538944416339 + }, + { + "budget": 0.17, + "metric": 0.22889277356496157, + "metric_std": 0.007335535575737338, + "actual_fraction": 0.17243675099866845, + "actual_fraction_std": 0.010487339548617424 + }, + { + "budget": 0.18, + "metric": 0.2313398388798349, + "metric_std": 0.005395867026591111, + "actual_fraction": 0.18249001331557924, + "actual_fraction_std": 0.006304719364077687 + }, + { + "budget": 0.19, + "metric": 0.23427073754042654, + "metric_std": 0.008377121094237258, + "actual_fraction": 0.19387483355525967, + "actual_fraction_std": 0.00927502021312626 + }, + { + "budget": 0.2, + "metric": 0.2365702263226365, + "metric_std": 0.00782282292980275, + "actual_fraction": 0.20539280958721706, + "actual_fraction_std": 0.008338864880076792 + }, + { + "budget": 0.21, + "metric": 0.23917356441574786, + "metric_std": 0.006156272140829932, + "actual_fraction": 0.21637816245006655, + "actual_fraction_std": 0.006552935980369354 + }, + { + "budget": 0.22, + "metric": 0.24115235052329936, + "metric_std": 0.006464726569114713, + "actual_fraction": 0.22423435419440746, + "actual_fraction_std": 0.007468158676595255 + }, + { + "budget": 0.23, + "metric": 0.24413041075983383, + "metric_std": 0.005158574810228732, + "actual_fraction": 0.23521970705725698, + "actual_fraction_std": 0.009399634220934866 + }, + { + "budget": 0.24, + "metric": 0.24679272153267928, + "metric_std": 0.005409184856483932, + "actual_fraction": 0.24527296937416782, + "actual_fraction_std": 0.010220845540926787 + }, + { + "budget": 0.25, + "metric": 0.24858546175199964, + "metric_std": 0.005470164687104912, + "actual_fraction": 0.2545938748335553, + "actual_fraction_std": 0.007279555434178511 + }, + { + "budget": 0.26, + "metric": 0.2499643430698945, + "metric_std": 0.004615346710200438, + "actual_fraction": 0.2631824234354194, + "actual_fraction_std": 0.004838935354285217 + }, + { + "budget": 0.27, + "metric": 0.2515383976374145, + "metric_std": 0.004647029608732975, + "actual_fraction": 0.2706391478029294, + "actual_fraction_std": 0.006227791922060487 + }, + { + "budget": 0.28, + "metric": 0.25423948786916145, + "metric_std": 0.004684899252803911, + "actual_fraction": 0.2818908122503329, + "actual_fraction_std": 0.009760533020484234 + }, + { + "budget": 0.29, + "metric": 0.25630182421664954, + "metric_std": 0.003298173762112391, + "actual_fraction": 0.2934753661784288, + "actual_fraction_std": 0.005962347051195243 + }, + { + "budget": 0.3, + "metric": 0.2603526011863125, + "metric_std": 0.003038055072491751, + "actual_fraction": 0.30332889480692415, + "actual_fraction_std": 0.004803307985466778 + }, + { + "budget": 0.31, + "metric": 0.26431280368730753, + "metric_std": 0.0024204445086333393, + "actual_fraction": 0.31431424766977367, + "actual_fraction_std": 0.004674697651542463 + }, + { + "budget": 0.32, + "metric": 0.26759916137338224, + "metric_std": 0.002369175927709364, + "actual_fraction": 0.3266311584553928, + "actual_fraction_std": 0.006335403711052745 + }, + { + "budget": 0.33, + "metric": 0.27219585570056337, + "metric_std": 0.003580227886535559, + "actual_fraction": 0.33888149134487355, + "actual_fraction_std": 0.005560350562080575 + }, + { + "budget": 0.34, + "metric": 0.2758051534037437, + "metric_std": 0.002985874321087061, + "actual_fraction": 0.3496005326231691, + "actual_fraction_std": 0.0047859737247205455 + }, + { + "budget": 0.35, + "metric": 0.2801067894245316, + "metric_std": 0.002889646107097193, + "actual_fraction": 0.36025299600532623, + "actual_fraction_std": 0.0047335900926374 + }, + { + "budget": 0.36, + "metric": 0.2838425181531855, + "metric_std": 0.0014709099567506495, + "actual_fraction": 0.37017310252996005, + "actual_fraction_std": 0.005480052474418872 + }, + { + "budget": 0.37, + "metric": 0.285120462510742, + "metric_std": 0.00153017226490547, + "actual_fraction": 0.3754993342210386, + "actual_fraction_std": 0.005173192499307616 + }, + { + "budget": 0.38, + "metric": 0.2885561791155984, + "metric_std": 0.001078780616540644, + "actual_fraction": 0.3858854860186418, + "actual_fraction_std": 0.007340194177957636 + }, + { + "budget": 0.39, + "metric": 0.29052594295913775, + "metric_std": 0.002982906889298439, + "actual_fraction": 0.3938082556591212, + "actual_fraction_std": 0.008268803161367657 + }, + { + "budget": 0.4, + "metric": 0.29383819219297747, + "metric_std": 0.002688393678029486, + "actual_fraction": 0.4033288948069241, + "actual_fraction_std": 0.0031192295006983992 + }, + { + "budget": 0.41, + "metric": 0.295919389424475, + "metric_std": 0.003350226645846243, + "actual_fraction": 0.41338215712383486, + "actual_fraction_std": 0.003920471510103143 + }, + { + "budget": 0.42, + "metric": 0.29773531943174897, + "metric_std": 0.001943621370529706, + "actual_fraction": 0.4219041278295606, + "actual_fraction_std": 0.0026169432765690594 + }, + { + "budget": 0.43, + "metric": 0.2995798831220195, + "metric_std": 0.0030565860267036116, + "actual_fraction": 0.4300932090545939, + "actual_fraction_std": 0.004077046842182391 + }, + { + "budget": 0.44, + "metric": 0.2994427650543422, + "metric_std": 0.003359011453835414, + "actual_fraction": 0.4369507323568575, + "actual_fraction_std": 0.003480247680075932 + }, + { + "budget": 0.45, + "metric": 0.3001333522027779, + "metric_std": 0.0025725481606782155, + "actual_fraction": 0.44773635153129165, + "actual_fraction_std": 0.002318306970237036 + }, + { + "budget": 0.46, + "metric": 0.3011461337530082, + "metric_std": 0.0023476712815172825, + "actual_fraction": 0.45665778961384823, + "actual_fraction_std": 0.003892103036768046 + }, + { + "budget": 0.47, + "metric": 0.30034534865024065, + "metric_std": 0.0023317022822784204, + "actual_fraction": 0.4613182423435419, + "actual_fraction_std": 0.005121523863498584 + }, + { + "budget": 0.48, + "metric": 0.29994665434158463, + "metric_std": 0.0024456831355890935, + "actual_fraction": 0.4721704394141145, + "actual_fraction_std": 0.00726507935587782 + }, + { + "budget": 0.49, + "metric": 0.29962264934810373, + "metric_std": 0.0021544918332044773, + "actual_fraction": 0.4818242343541944, + "actual_fraction_std": 0.007056157741689111 + }, + { + "budget": 0.5, + "metric": 0.29973972468716326, + "metric_std": 0.002762459449155375, + "actual_fraction": 0.49300932090545935, + "actual_fraction_std": 0.005263444840493293 + }, + { + "budget": 0.51, + "metric": 0.29946704915135125, + "metric_std": 0.0028576005913522216, + "actual_fraction": 0.5037283621837549, + "actual_fraction_std": 0.005083517754679377 + }, + { + "budget": 0.52, + "metric": 0.2984325005890893, + "metric_std": 0.002999857852368009, + "actual_fraction": 0.5171105193075899, + "actual_fraction_std": 0.0042734504756008545 + }, + { + "budget": 0.53, + "metric": 0.29775527179318995, + "metric_std": 0.004706095844533631, + "actual_fraction": 0.5312250332889481, + "actual_fraction_std": 0.007854357817046355 + }, + { + "budget": 0.54, + "metric": 0.2968907163611731, + "metric_std": 0.00452157798235272, + "actual_fraction": 0.5450066577896139, + "actual_fraction_std": 0.004554628861913979 + }, + { + "budget": 0.55, + "metric": 0.2973610396788611, + "metric_std": 0.0038045971538949582, + "actual_fraction": 0.5558588548601865, + "actual_fraction_std": 0.005045225350754487 + }, + { + "budget": 0.56, + "metric": 0.296968201145198, + "metric_std": 0.005407430410349543, + "actual_fraction": 0.5681757656458056, + "actual_fraction_std": 0.0052073534482720354 + }, + { + "budget": 0.57, + "metric": 0.29666207684349216, + "metric_std": 0.00476150448703151, + "actual_fraction": 0.5773635153129162, + "actual_fraction_std": 0.00479869163842676 + }, + { + "budget": 0.58, + "metric": 0.29577980111249136, + "metric_std": 0.00437036121486799, + "actual_fraction": 0.5872170439414115, + "actual_fraction_std": 0.004164446072968312 + }, + { + "budget": 0.59, + "metric": 0.2954162579172059, + "metric_std": 0.004220227814389221, + "actual_fraction": 0.5954061251664446, + "actual_fraction_std": 0.005171049946927724 + }, + { + "budget": 0.6, + "metric": 0.29495634930149345, + "metric_std": 0.003809069215225674, + "actual_fraction": 0.6072569906790946, + "actual_fraction_std": 0.004298014453150334 + }, + { + "budget": 0.61, + "metric": 0.2955712714018874, + "metric_std": 0.004439557024892787, + "actual_fraction": 0.6177762982689747, + "actual_fraction_std": 0.003239478195926179 + }, + { + "budget": 0.62, + "metric": 0.295480018395601, + "metric_std": 0.0029531655167969076, + "actual_fraction": 0.6294940079893476, + "actual_fraction_std": 0.004600624154821988 + }, + { + "budget": 0.63, + "metric": 0.29409550862108347, + "metric_std": 0.0036793520031143897, + "actual_fraction": 0.6394141145139813, + "actual_fraction_std": 0.004780181669198108 + }, + { + "budget": 0.64, + "metric": 0.2943274514943598, + "metric_std": 0.004597960021520727, + "actual_fraction": 0.6488681757656458, + "actual_fraction_std": 0.00427215372227846 + }, + { + "budget": 0.65, + "metric": 0.2946591229818803, + "metric_std": 0.004866757364500767, + "actual_fraction": 0.6578561917443408, + "actual_fraction_std": 0.0038563489900331795 + }, + { + "budget": 0.66, + "metric": 0.294128800673193, + "metric_std": 0.00510536073754036, + "actual_fraction": 0.6669773635153129, + "actual_fraction_std": 0.007265079355877813 + }, + { + "budget": 0.67, + "metric": 0.2933805456044235, + "metric_std": 0.00508426362287954, + "actual_fraction": 0.6752330226364845, + "actual_fraction_std": 0.005533380230570579 + }, + { + "budget": 0.68, + "metric": 0.29239958070817523, + "metric_std": 0.0045060714662551425, + "actual_fraction": 0.6820239680426099, + "actual_fraction_std": 0.005374902458363444 + }, + { + "budget": 0.69, + "metric": 0.2918624308598588, + "metric_std": 0.0033844388532709227, + "actual_fraction": 0.6911451398135818, + "actual_fraction_std": 0.005910077502309493 + }, + { + "budget": 0.7, + "metric": 0.29317482141250056, + "metric_std": 0.0023255054941606875, + "actual_fraction": 0.6992676431424767, + "actual_fraction_std": 0.007482982348737459 + }, + { + "budget": 0.71, + "metric": 0.2922382634139355, + "metric_std": 0.00244434880622609, + "actual_fraction": 0.709121171770972, + "actual_fraction_std": 0.008001128686108366 + }, + { + "budget": 0.72, + "metric": 0.2938274225849639, + "metric_std": 0.0021996654821087786, + "actual_fraction": 0.7209720372836218, + "actual_fraction_std": 0.008874486424406153 + }, + { + "budget": 0.73, + "metric": 0.29283239731792376, + "metric_std": 0.002337716269180936, + "actual_fraction": 0.7278961384820238, + "actual_fraction_std": 0.00862498592964397 + }, + { + "budget": 0.74, + "metric": 0.2920705139724137, + "metric_std": 0.001808449418915044, + "actual_fraction": 0.7380825565912117, + "actual_fraction_std": 0.008165637146249028 + }, + { + "budget": 0.75, + "metric": 0.29109797712605917, + "metric_std": 0.0017644260724988955, + "actual_fraction": 0.7532623169107857, + "actual_fraction_std": 0.008865116253464205 + }, + { + "budget": 0.76, + "metric": 0.28936047504801005, + "metric_std": 0.0013069647142368353, + "actual_fraction": 0.7611850865512649, + "actual_fraction_std": 0.01059404755308447 + }, + { + "budget": 0.77, + "metric": 0.28950468250251066, + "metric_std": 0.0017985371879600401, + "actual_fraction": 0.7703728362183755, + "actual_fraction_std": 0.007847300256256367 + }, + { + "budget": 0.78, + "metric": 0.28952481965125837, + "metric_std": 0.0016672986604908731, + "actual_fraction": 0.779427430093209, + "actual_fraction_std": 0.007830336157317147 + }, + { + "budget": 0.79, + "metric": 0.2889013402435306, + "metric_std": 0.0023032488869729243, + "actual_fraction": 0.7886817576564582, + "actual_fraction_std": 0.009965579118944598 + }, + { + "budget": 0.8, + "metric": 0.2895254683597344, + "metric_std": 0.003293017339282284, + "actual_fraction": 0.7952729693741678, + "actual_fraction_std": 0.009412593552329335 + }, + { + "budget": 0.81, + "metric": 0.28916423190213403, + "metric_std": 0.0041438944635216616, + "actual_fraction": 0.8049933422103862, + "actual_fraction_std": 0.005762931310087942 + }, + { + "budget": 0.82, + "metric": 0.2893681243077616, + "metric_std": 0.004662417806183233, + "actual_fraction": 0.816311584553928, + "actual_fraction_std": 0.004826323488532498 + }, + { + "budget": 0.83, + "metric": 0.2890581017616989, + "metric_std": 0.002643374475106413, + "actual_fraction": 0.8269640479360852, + "actual_fraction_std": 0.0035417938956778475 + }, + { + "budget": 0.84, + "metric": 0.28785619350057384, + "metric_std": 0.0029704858771600283, + "actual_fraction": 0.8370838881491345, + "actual_fraction_std": 0.0055881823083204505 + }, + { + "budget": 0.85, + "metric": 0.2879525132773096, + "metric_std": 0.0027434819019800582, + "actual_fraction": 0.8453395472703062, + "actual_fraction_std": 0.0073258379246368315 + }, + { + "budget": 0.86, + "metric": 0.28672901477442825, + "metric_std": 0.002111860220874941, + "actual_fraction": 0.8542609853528628, + "actual_fraction_std": 0.006634446427884461 + }, + { + "budget": 0.87, + "metric": 0.2861905743795651, + "metric_std": 0.0022822786610403605, + "actual_fraction": 0.8655792276964048, + "actual_fraction_std": 0.005522354554814322 + }, + { + "budget": 0.88, + "metric": 0.2847416906256778, + "metric_std": 0.0023216189718916155, + "actual_fraction": 0.8760319573901466, + "actual_fraction_std": 0.005133410526285265 + }, + { + "budget": 0.89, + "metric": 0.28296005854015394, + "metric_std": 0.0024331366203774597, + "actual_fraction": 0.8887483355525966, + "actual_fraction_std": 0.006130953093593262 + }, + { + "budget": 0.9, + "metric": 0.28137240071729097, + "metric_std": 0.0015132880766278378, + "actual_fraction": 0.8997336884154461, + "actual_fraction_std": 0.007719160835054529 + }, + { + "budget": 0.91, + "metric": 0.28046591492789263, + "metric_std": 0.0020953361640602616, + "actual_fraction": 0.9091877496671105, + "actual_fraction_std": 0.011277537048238591 + }, + { + "budget": 0.92, + "metric": 0.27955158971040195, + "metric_std": 0.0025092436931584824, + "actual_fraction": 0.9187083888149135, + "actual_fraction_std": 0.009166255565871128 + }, + { + "budget": 0.93, + "metric": 0.2777561260486451, + "metric_std": 0.002493526833366614, + "actual_fraction": 0.9291611185086552, + "actual_fraction_std": 0.00721456834226796 + }, + { + "budget": 0.94, + "metric": 0.27594617361191476, + "metric_std": 0.002661100755381945, + "actual_fraction": 0.9404793608521971, + "actual_fraction_std": 0.006144494197684124 + }, + { + "budget": 0.95, + "metric": 0.2751401821654004, + "metric_std": 0.0030329759884267617, + "actual_fraction": 0.9515978695073237, + "actual_fraction_std": 0.007752109059260947 + }, + { + "budget": 0.96, + "metric": 0.2746516824807885, + "metric_std": 0.002412840264296232, + "actual_fraction": 0.9601864181091877, + "actual_fraction_std": 0.0057107770128478836 + }, + { + "budget": 0.97, + "metric": 0.27312801399550707, + "metric_std": 0.0025500213264794845, + "actual_fraction": 0.9695073235685753, + "actual_fraction_std": 0.005691339321966356 + }, + { + "budget": 0.98, + "metric": 0.27414030903824826, + "metric_std": 0.0015883241321269707, + "actual_fraction": 0.9768974700399466, + "actual_fraction_std": 0.0048503722299744504 + }, + { + "budget": 0.99, + "metric": 0.272770584625888, + "metric_std": 0.0015765505046386335, + "actual_fraction": 0.9872170439414114, + "actual_fraction_std": 0.004208123327046744 + }, + { + "budget": 1.0, + "metric": 0.27001009067886245, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 2189, + "rescuable": 307, + "harmful": 220, + "both_wrong": 88, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 9508, + "rescuable": 1282, + "harmful": 777, + "both_wrong": 449, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.18254072168965788, + "test": 0.18129624718318377 + }, + "large": { + "validation": 0.25448626222520154, + "test": 0.27001009067886245 + }, + "learned_minus_confidence_test": 0.03895594463773688, + "learned_minus_small_test": 0.11740869793076364, + "learned_minus_large_test": 0.02869485443508496, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_yolo26x_Qwen3.8" + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "MiniCPM" + ], + "expert_display": "RT-DETR-X", + "run": "five_seed_rtdetr_x_fp32_MiniCPM_four_difference", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/rtdetr_x_fp32/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 98, + "validation_area_0_25": 0.08994758708890332, + "validation_point": { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.20572745422879232, + "rescued": 770, + "harmed": 465, + "missed_rescue": 16, + "avoided_harm": 10, + "selected_both_correct": 1366, + "selected_both_wrong": 143 + }, + "test_point": { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.9790279627163782, + "metric": 0.21609512872414097, + "rescued": 3244, + "harmed": 1926, + "missed_rescue": 70, + "avoided_harm": 19, + "selected_both_correct": 5928, + "selected_both_wrong": 666 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 0, + "missed_rescue": 786, + "avoided_harm": 475, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.021043073385953903, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.07628160544643427, + "rescued": 0, + "harmed": 5, + "missed_rescue": 786, + "avoided_harm": 470, + "selected_both_correct": 23, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.022327054291963577, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.07620479856453766, + "rescued": 0, + "harmed": 10, + "missed_rescue": 786, + "avoided_harm": 465, + "selected_both_correct": 46, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.024909960106015205, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.076179559423678, + "rescued": 0, + "harmed": 16, + "missed_rescue": 786, + "avoided_harm": 459, + "selected_both_correct": 68, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -0.027814723551273346, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.07608036783512472, + "rescued": 0, + "harmed": 21, + "missed_rescue": 786, + "avoided_harm": 454, + "selected_both_correct": 91, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -0.028914429247379303, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.09455032380508069, + "rescued": 1, + "harmed": 25, + "missed_rescue": 785, + "avoided_harm": 450, + "selected_both_correct": 114, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -0.030170045793056488, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.09386491892417406, + "rescued": 1, + "harmed": 27, + "missed_rescue": 785, + "avoided_harm": 448, + "selected_both_correct": 139, + "selected_both_wrong": 1 + }, + { + "budget": 0.07, + "threshold": -0.03123854100704193, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.09204233166673514, + "rescued": 1, + "harmed": 34, + "missed_rescue": 785, + "avoided_harm": 441, + "selected_both_correct": 160, + "selected_both_wrong": 1 + }, + { + "budget": 0.08, + "threshold": -0.03410686179995537, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.091007586947048, + "rescued": 1, + "harmed": 38, + "missed_rescue": 785, + "avoided_harm": 437, + "selected_both_correct": 184, + "selected_both_wrong": 1 + }, + { + "budget": 0.09, + "threshold": -0.03476816788315773, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.08934652523756015, + "rescued": 1, + "harmed": 43, + "missed_rescue": 785, + "avoided_harm": 432, + "selected_both_correct": 207, + "selected_both_wrong": 1 + }, + { + "budget": 0.1, + "threshold": -0.0365506075322628, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.08860787026161702, + "rescued": 1, + "harmed": 47, + "missed_rescue": 785, + "avoided_harm": 428, + "selected_both_correct": 231, + "selected_both_wrong": 1 + }, + { + "budget": 0.11, + "threshold": -0.03818900138139725, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.0875161669606114, + "rescued": 1, + "harmed": 54, + "missed_rescue": 785, + "avoided_harm": 421, + "selected_both_correct": 252, + "selected_both_wrong": 1 + }, + { + "budget": 0.12, + "threshold": -0.039673544466495514, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.0869090348188258, + "rescued": 1, + "harmed": 63, + "missed_rescue": 785, + "avoided_harm": 412, + "selected_both_correct": 271, + "selected_both_wrong": 1 + }, + { + "budget": 0.13, + "threshold": -0.04085410386323929, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.08635025131899243, + "rescued": 1, + "harmed": 68, + "missed_rescue": 785, + "avoided_harm": 407, + "selected_both_correct": 294, + "selected_both_wrong": 1 + }, + { + "budget": 0.14, + "threshold": -0.04204101860523224, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.0857792865956536, + "rescued": 1, + "harmed": 75, + "missed_rescue": 785, + "avoided_harm": 400, + "selected_both_correct": 315, + "selected_both_wrong": 1 + }, + { + "budget": 0.15, + "threshold": -0.04278091713786125, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.09418918796431942, + "rescued": 2, + "harmed": 81, + "missed_rescue": 784, + "avoided_harm": 394, + "selected_both_correct": 335, + "selected_both_wrong": 2 + }, + { + "budget": 0.16, + "threshold": -0.04322759434580803, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.09315792624073146, + "rescued": 2, + "harmed": 85, + "missed_rescue": 784, + "avoided_harm": 390, + "selected_both_correct": 359, + "selected_both_wrong": 2 + }, + { + "budget": 0.17, + "threshold": -0.04394488036632538, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.09222958631972636, + "rescued": 2, + "harmed": 89, + "missed_rescue": 784, + "avoided_harm": 386, + "selected_both_correct": 383, + "selected_both_wrong": 2 + }, + { + "budget": 0.18, + "threshold": -0.04530126228928566, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.0917587567603033, + "rescued": 2, + "harmed": 95, + "missed_rescue": 784, + "avoided_harm": 380, + "selected_both_correct": 405, + "selected_both_wrong": 2 + }, + { + "budget": 0.19, + "threshold": -0.04661082476377487, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.09090632738556746, + "rescued": 2, + "harmed": 101, + "missed_rescue": 784, + "avoided_harm": 374, + "selected_both_correct": 427, + "selected_both_wrong": 2 + }, + { + "budget": 0.2, + "threshold": -0.04730507358908653, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.09027150780105647, + "rescued": 2, + "harmed": 106, + "missed_rescue": 784, + "avoided_harm": 369, + "selected_both_correct": 450, + "selected_both_wrong": 2 + }, + { + "budget": 0.21, + "threshold": -0.048246145248413086, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.08904603886890428, + "rescued": 2, + "harmed": 113, + "missed_rescue": 784, + "avoided_harm": 362, + "selected_both_correct": 471, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.04900278151035309, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.08819855956614042, + "rescued": 2, + "harmed": 121, + "missed_rescue": 784, + "avoided_harm": 354, + "selected_both_correct": 491, + "selected_both_wrong": 2 + }, + { + "budget": 0.23, + "threshold": -0.050042688846588135, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.08769891005218598, + "rescued": 2, + "harmed": 126, + "missed_rescue": 784, + "avoided_harm": 349, + "selected_both_correct": 513, + "selected_both_wrong": 3 + }, + { + "budget": 0.24, + "threshold": -0.050901688635349274, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.08812163994607147, + "rescued": 8, + "harmed": 130, + "missed_rescue": 778, + "avoided_harm": 345, + "selected_both_correct": 530, + "selected_both_wrong": 4 + }, + { + "budget": 0.25, + "threshold": -0.05161922052502632, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.08782151216689346, + "rescued": 14, + "harmed": 135, + "missed_rescue": 772, + "avoided_harm": 340, + "selected_both_correct": 546, + "selected_both_wrong": 5 + }, + { + "budget": 0.26, + "threshold": -0.05245445668697357, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.08802770491567173, + "rescued": 21, + "harmed": 139, + "missed_rescue": 765, + "avoided_harm": 336, + "selected_both_correct": 563, + "selected_both_wrong": 5 + }, + { + "budget": 0.27, + "threshold": -0.053079016506671906, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.08800890547752473, + "rescued": 28, + "harmed": 144, + "missed_rescue": 758, + "avoided_harm": 331, + "selected_both_correct": 578, + "selected_both_wrong": 6 + }, + { + "budget": 0.28, + "threshold": -0.053604189306497574, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.08849150328196631, + "rescued": 36, + "harmed": 148, + "missed_rescue": 750, + "avoided_harm": 327, + "selected_both_correct": 591, + "selected_both_wrong": 9 + }, + { + "budget": 0.29, + "threshold": -0.05505518987774849, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.0887205564920429, + "rescued": 46, + "harmed": 153, + "missed_rescue": 740, + "avoided_harm": 322, + "selected_both_correct": 604, + "selected_both_wrong": 9 + }, + { + "budget": 0.3, + "threshold": -0.05605518817901611, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.08901323629975644, + "rescued": 53, + "harmed": 157, + "missed_rescue": 733, + "avoided_harm": 318, + "selected_both_correct": 621, + "selected_both_wrong": 9 + }, + { + "budget": 0.31, + "threshold": -0.056986160576343536, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.08941457984183532, + "rescued": 61, + "harmed": 162, + "missed_rescue": 725, + "avoided_harm": 313, + "selected_both_correct": 635, + "selected_both_wrong": 10 + }, + { + "budget": 0.32, + "threshold": -0.0582931749522686, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.08986547999267158, + "rescued": 69, + "harmed": 167, + "missed_rescue": 717, + "avoided_harm": 308, + "selected_both_correct": 650, + "selected_both_wrong": 10 + }, + { + "budget": 0.33, + "threshold": -0.0596650205552578, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.0900623716189768, + "rescued": 76, + "harmed": 173, + "missed_rescue": 710, + "avoided_harm": 302, + "selected_both_correct": 664, + "selected_both_wrong": 11 + }, + { + "budget": 0.34, + "threshold": -0.060549296438694, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.09018872466182963, + "rescued": 87, + "harmed": 180, + "missed_rescue": 699, + "avoided_harm": 295, + "selected_both_correct": 673, + "selected_both_wrong": 12 + }, + { + "budget": 0.35, + "threshold": -0.06178095191717148, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.09513465140887797, + "rescued": 96, + "harmed": 184, + "missed_rescue": 690, + "avoided_harm": 291, + "selected_both_correct": 686, + "selected_both_wrong": 14 + }, + { + "budget": 0.36, + "threshold": -0.06328777223825455, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.09560176179757868, + "rescued": 108, + "harmed": 190, + "missed_rescue": 678, + "avoided_harm": 285, + "selected_both_correct": 696, + "selected_both_wrong": 14 + }, + { + "budget": 0.37, + "threshold": -0.0647548958659172, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.09606601222778545, + "rescued": 118, + "harmed": 196, + "missed_rescue": 668, + "avoided_harm": 279, + "selected_both_correct": 707, + "selected_both_wrong": 15 + }, + { + "budget": 0.38, + "threshold": -0.06637481600046158, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.09683684024140088, + "rescued": 126, + "harmed": 199, + "missed_rescue": 660, + "avoided_harm": 276, + "selected_both_correct": 722, + "selected_both_wrong": 17 + }, + { + "budget": 0.39, + "threshold": -0.06774259358644485, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.09748138328334532, + "rescued": 135, + "harmed": 201, + "missed_rescue": 651, + "avoided_harm": 274, + "selected_both_correct": 736, + "selected_both_wrong": 20 + }, + { + "budget": 0.4, + "threshold": -0.06843380630016327, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.09725002075008596, + "rescued": 145, + "harmed": 205, + "missed_rescue": 641, + "avoided_harm": 270, + "selected_both_correct": 750, + "selected_both_wrong": 20 + }, + { + "budget": 0.41, + "threshold": -0.0694691464304924, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.09794654894022267, + "rescued": 155, + "harmed": 210, + "missed_rescue": 631, + "avoided_harm": 265, + "selected_both_correct": 763, + "selected_both_wrong": 20 + }, + { + "budget": 0.42, + "threshold": -0.07112707942724228, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.09874354292060185, + "rescued": 165, + "harmed": 216, + "missed_rescue": 621, + "avoided_harm": 259, + "selected_both_correct": 773, + "selected_both_wrong": 22 + }, + { + "budget": 0.43, + "threshold": -0.07192998379468918, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.09944895007868021, + "rescued": 174, + "harmed": 222, + "missed_rescue": 612, + "avoided_harm": 253, + "selected_both_correct": 785, + "selected_both_wrong": 23 + }, + { + "budget": 0.44, + "threshold": -0.07266208529472351, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.10024323502584373, + "rescued": 184, + "harmed": 226, + "missed_rescue": 602, + "avoided_harm": 249, + "selected_both_correct": 798, + "selected_both_wrong": 24 + }, + { + "budget": 0.45, + "threshold": -0.0744253620505333, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.10107630515550795, + "rescued": 194, + "harmed": 230, + "missed_rescue": 592, + "avoided_harm": 245, + "selected_both_correct": 810, + "selected_both_wrong": 26 + }, + { + "budget": 0.46, + "threshold": -0.07532940804958344, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.10181572307751467, + "rescued": 203, + "harmed": 235, + "missed_rescue": 583, + "avoided_harm": 240, + "selected_both_correct": 823, + "selected_both_wrong": 27 + }, + { + "budget": 0.47, + "threshold": -0.0770750343799591, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.10291838837461804, + "rescued": 218, + "harmed": 240, + "missed_rescue": 568, + "avoided_harm": 235, + "selected_both_correct": 830, + "selected_both_wrong": 28 + }, + { + "budget": 0.48, + "threshold": -0.07818037271499634, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.10398791390534362, + "rescued": 228, + "harmed": 245, + "missed_rescue": 558, + "avoided_harm": 230, + "selected_both_correct": 843, + "selected_both_wrong": 28 + }, + { + "budget": 0.49, + "threshold": -0.07998209446668625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.1035500155095013, + "rescued": 235, + "harmed": 252, + "missed_rescue": 551, + "avoided_harm": 223, + "selected_both_correct": 857, + "selected_both_wrong": 28 + }, + { + "budget": 0.5, + "threshold": -0.08124949783086777, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.10310317656730472, + "rescued": 242, + "harmed": 256, + "missed_rescue": 544, + "avoided_harm": 219, + "selected_both_correct": 871, + "selected_both_wrong": 31 + }, + { + "budget": 0.51, + "threshold": -0.08398465067148209, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.10437739691349379, + "rescued": 254, + "harmed": 261, + "missed_rescue": 532, + "avoided_harm": 214, + "selected_both_correct": 881, + "selected_both_wrong": 32 + }, + { + "budget": 0.52, + "threshold": -0.08537434041500092, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.10682502852650196, + "rescued": 262, + "harmed": 268, + "missed_rescue": 524, + "avoided_harm": 207, + "selected_both_correct": 890, + "selected_both_wrong": 36 + }, + { + "budget": 0.53, + "threshold": -0.08664409816265106, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.10806066671218575, + "rescued": 273, + "harmed": 275, + "missed_rescue": 513, + "avoided_harm": 200, + "selected_both_correct": 899, + "selected_both_wrong": 37 + }, + { + "budget": 0.54, + "threshold": -0.08777308464050293, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.10917273744598222, + "rescued": 284, + "harmed": 278, + "missed_rescue": 502, + "avoided_harm": 197, + "selected_both_correct": 911, + "selected_both_wrong": 39 + }, + { + "budget": 0.55, + "threshold": -0.08980843424797058, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.11036769675516778, + "rescued": 296, + "harmed": 284, + "missed_rescue": 490, + "avoided_harm": 191, + "selected_both_correct": 920, + "selected_both_wrong": 40 + }, + { + "budget": 0.56, + "threshold": -0.0906188115477562, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.11423875293101385, + "rescued": 307, + "harmed": 287, + "missed_rescue": 479, + "avoided_harm": 188, + "selected_both_correct": 931, + "selected_both_wrong": 43 + }, + { + "budget": 0.57, + "threshold": -0.09160211682319641, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.11535691235491538, + "rescued": 317, + "harmed": 294, + "missed_rescue": 469, + "avoided_harm": 181, + "selected_both_correct": 940, + "selected_both_wrong": 45 + }, + { + "budget": 0.58, + "threshold": -0.09283093363046646, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.11669725808975426, + "rescued": 328, + "harmed": 298, + "missed_rescue": 458, + "avoided_harm": 177, + "selected_both_correct": 952, + "selected_both_wrong": 46 + }, + { + "budget": 0.59, + "threshold": -0.0941123366355896, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.1177679103161983, + "rescued": 338, + "harmed": 303, + "missed_rescue": 448, + "avoided_harm": 172, + "selected_both_correct": 964, + "selected_both_wrong": 47 + }, + { + "budget": 0.6, + "threshold": -0.09614869952201843, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.11921675545489711, + "rescued": 351, + "harmed": 309, + "missed_rescue": 435, + "avoided_harm": 166, + "selected_both_correct": 972, + "selected_both_wrong": 48 + }, + { + "budget": 0.61, + "threshold": -0.09772664308547974, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.12100079401350544, + "rescued": 364, + "harmed": 313, + "missed_rescue": 422, + "avoided_harm": 162, + "selected_both_correct": 981, + "selected_both_wrong": 50 + }, + { + "budget": 0.62, + "threshold": -0.09911532700061798, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.12498749148623647, + "rescued": 375, + "harmed": 315, + "missed_rescue": 411, + "avoided_harm": 160, + "selected_both_correct": 993, + "selected_both_wrong": 53 + }, + { + "budget": 0.63, + "threshold": -0.10007988661527634, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.12628905170512741, + "rescued": 387, + "harmed": 319, + "missed_rescue": 399, + "avoided_harm": 156, + "selected_both_correct": 1004, + "selected_both_wrong": 54 + }, + { + "budget": 0.64, + "threshold": -0.10177135467529297, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.12942700411115388, + "rescued": 403, + "harmed": 324, + "missed_rescue": 383, + "avoided_harm": 151, + "selected_both_correct": 1010, + "selected_both_wrong": 55 + }, + { + "budget": 0.65, + "threshold": -0.10395076125860214, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.13068664709512867, + "rescued": 410, + "harmed": 328, + "missed_rescue": 376, + "avoided_harm": 147, + "selected_both_correct": 1024, + "selected_both_wrong": 58 + }, + { + "budget": 0.66, + "threshold": -0.10524322092533112, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.1324582559366836, + "rescued": 422, + "harmed": 332, + "missed_rescue": 364, + "avoided_harm": 143, + "selected_both_correct": 1033, + "selected_both_wrong": 61 + }, + { + "budget": 0.67, + "threshold": -0.10689426958560944, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.13402207594539173, + "rescued": 435, + "harmed": 338, + "missed_rescue": 351, + "avoided_harm": 137, + "selected_both_correct": 1041, + "selected_both_wrong": 62 + }, + { + "budget": 0.68, + "threshold": -0.11014843732118607, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.13570216904654514, + "rescued": 447, + "harmed": 341, + "missed_rescue": 339, + "avoided_harm": 134, + "selected_both_correct": 1052, + "selected_both_wrong": 64 + }, + { + "budget": 0.69, + "threshold": -0.11117545515298843, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.13971639727625254, + "rescued": 458, + "harmed": 343, + "missed_rescue": 328, + "avoided_harm": 132, + "selected_both_correct": 1062, + "selected_both_wrong": 69 + }, + { + "budget": 0.7, + "threshold": -0.11230025440454483, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.14039490967751408, + "rescued": 469, + "harmed": 346, + "missed_rescue": 317, + "avoided_harm": 129, + "selected_both_correct": 1074, + "selected_both_wrong": 71 + }, + { + "budget": 0.71, + "threshold": -0.11423606425523758, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.1389440908530641, + "rescued": 476, + "harmed": 351, + "missed_rescue": 310, + "avoided_harm": 124, + "selected_both_correct": 1084, + "selected_both_wrong": 77 + }, + { + "budget": 0.72, + "threshold": -0.11647249013185501, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.1412819215041655, + "rescued": 492, + "harmed": 358, + "missed_rescue": 294, + "avoided_harm": 117, + "selected_both_correct": 1089, + "selected_both_wrong": 77 + }, + { + "budget": 0.73, + "threshold": -0.11962417513132095, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.14288929438332787, + "rescued": 503, + "harmed": 361, + "missed_rescue": 283, + "avoided_harm": 114, + "selected_both_correct": 1099, + "selected_both_wrong": 81 + }, + { + "budget": 0.74, + "threshold": -0.12126146256923676, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.14459529608777652, + "rescued": 514, + "harmed": 363, + "missed_rescue": 272, + "avoided_harm": 112, + "selected_both_correct": 1111, + "selected_both_wrong": 84 + }, + { + "budget": 0.75, + "threshold": -0.1230982318520546, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.14606069842667782, + "rescued": 525, + "harmed": 367, + "missed_rescue": 261, + "avoided_harm": 108, + "selected_both_correct": 1121, + "selected_both_wrong": 87 + }, + { + "budget": 0.76, + "threshold": -0.12517565488815308, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.14665677647800887, + "rescued": 537, + "harmed": 373, + "missed_rescue": 249, + "avoided_harm": 102, + "selected_both_correct": 1130, + "selected_both_wrong": 88 + }, + { + "budget": 0.77, + "threshold": -0.12628257274627686, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.1471300269894615, + "rescued": 546, + "harmed": 381, + "missed_rescue": 240, + "avoided_harm": 94, + "selected_both_correct": 1139, + "selected_both_wrong": 90 + }, + { + "budget": 0.78, + "threshold": -0.12975578010082245, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.1503016824168161, + "rescued": 558, + "harmed": 384, + "missed_rescue": 228, + "avoided_harm": 91, + "selected_both_correct": 1151, + "selected_both_wrong": 91 + }, + { + "budget": 0.79, + "threshold": -0.13229140639305115, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.15155183161566646, + "rescued": 566, + "harmed": 386, + "missed_rescue": 220, + "avoided_harm": 89, + "selected_both_correct": 1163, + "selected_both_wrong": 97 + }, + { + "budget": 0.8, + "threshold": -0.1373685747385025, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.15468771641968415, + "rescued": 579, + "harmed": 392, + "missed_rescue": 207, + "avoided_harm": 83, + "selected_both_correct": 1170, + "selected_both_wrong": 99 + }, + { + "budget": 0.81, + "threshold": -0.1407174915075302, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.16068290931933413, + "rescued": 597, + "harmed": 396, + "missed_rescue": 189, + "avoided_harm": 79, + "selected_both_correct": 1175, + "selected_both_wrong": 100 + }, + { + "budget": 0.82, + "threshold": -0.1433146893978119, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.16451572912479911, + "rescued": 612, + "harmed": 400, + "missed_rescue": 174, + "avoided_harm": 75, + "selected_both_correct": 1183, + "selected_both_wrong": 101 + }, + { + "budget": 0.83, + "threshold": -0.1457100659608841, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.16497266103579622, + "rescued": 619, + "harmed": 406, + "missed_rescue": 167, + "avoided_harm": 69, + "selected_both_correct": 1196, + "selected_both_wrong": 103 + }, + { + "budget": 0.84, + "threshold": -0.148710235953331, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.1662001358662362, + "rescued": 628, + "harmed": 409, + "missed_rescue": 158, + "avoided_harm": 66, + "selected_both_correct": 1209, + "selected_both_wrong": 106 + }, + { + "budget": 0.85, + "threshold": -0.15291842818260193, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.1711356788768453, + "rescued": 639, + "harmed": 411, + "missed_rescue": 147, + "avoided_harm": 64, + "selected_both_correct": 1218, + "selected_both_wrong": 112 + }, + { + "budget": 0.86, + "threshold": -0.15889839828014374, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.17267738998377383, + "rescued": 647, + "harmed": 416, + "missed_rescue": 139, + "avoided_harm": 59, + "selected_both_correct": 1230, + "selected_both_wrong": 115 + }, + { + "budget": 0.87, + "threshold": -0.16629797220230103, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.17579084329823536, + "rescued": 656, + "harmed": 422, + "missed_rescue": 130, + "avoided_harm": 53, + "selected_both_correct": 1241, + "selected_both_wrong": 117 + }, + { + "budget": 0.88, + "threshold": -0.17295491695404053, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.17752359710692284, + "rescued": 666, + "harmed": 427, + "missed_rescue": 120, + "avoided_harm": 48, + "selected_both_correct": 1253, + "selected_both_wrong": 118 + }, + { + "budget": 0.89, + "threshold": -0.18053314089775085, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.18131944434726105, + "rescued": 676, + "harmed": 431, + "missed_rescue": 110, + "avoided_harm": 44, + "selected_both_correct": 1266, + "selected_both_wrong": 119 + }, + { + "budget": 0.9, + "threshold": -0.19442276656627655, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.18427430914771328, + "rescued": 688, + "harmed": 433, + "missed_rescue": 98, + "avoided_harm": 42, + "selected_both_correct": 1277, + "selected_both_wrong": 122 + }, + { + "budget": 0.91, + "threshold": -0.20900504291057587, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.18611646191425543, + "rescued": 697, + "harmed": 437, + "missed_rescue": 89, + "avoided_harm": 38, + "selected_both_correct": 1289, + "selected_both_wrong": 125 + }, + { + "budget": 0.92, + "threshold": -0.22383525967597961, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.1910397694023846, + "rescued": 707, + "harmed": 440, + "missed_rescue": 79, + "avoided_harm": 35, + "selected_both_correct": 1299, + "selected_both_wrong": 130 + }, + { + "budget": 0.93, + "threshold": -0.25952714681625366, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.19063222783395556, + "rescued": 717, + "harmed": 447, + "missed_rescue": 69, + "avoided_harm": 28, + "selected_both_correct": 1308, + "selected_both_wrong": 132 + }, + { + "budget": 0.94, + "threshold": -0.2804492115974426, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.19317109979249197, + "rescued": 726, + "harmed": 451, + "missed_rescue": 60, + "avoided_harm": 24, + "selected_both_correct": 1318, + "selected_both_wrong": 137 + }, + { + "budget": 0.95, + "threshold": -0.2984963357448578, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.19747037448756763, + "rescued": 738, + "harmed": 453, + "missed_rescue": 48, + "avoided_harm": 22, + "selected_both_correct": 1329, + "selected_both_wrong": 140 + }, + { + "budget": 0.96, + "threshold": -0.32223379611968994, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.2013335009829101, + "rescued": 748, + "harmed": 455, + "missed_rescue": 38, + "avoided_harm": 20, + "selected_both_correct": 1343, + "selected_both_wrong": 142 + }, + { + "budget": 0.97, + "threshold": -0.36503666639328003, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2032615598092342, + "rescued": 758, + "harmed": 460, + "missed_rescue": 28, + "avoided_harm": 15, + "selected_both_correct": 1355, + "selected_both_wrong": 143 + }, + { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.20572745422879232, + "rescued": 770, + "harmed": 465, + "missed_rescue": 16, + "avoided_harm": 10, + "selected_both_correct": 1366, + "selected_both_wrong": 143 + }, + { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.20340063714272894, + "rescued": 778, + "harmed": 472, + "missed_rescue": 8, + "avoided_harm": 3, + "selected_both_correct": 1374, + "selected_both_wrong": 148 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.20666016863629594, + "rescued": 786, + "harmed": 475, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1391, + "selected_both_wrong": 152 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07887397871043929, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3314, + "avoided_harm": 1945, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.021043073385953903, + "operator": "ge", + "actual_fraction": 0.007989347536617843, + "metric": 0.0788343605891772, + "rescued": 0, + "harmed": 21, + "missed_rescue": 3314, + "avoided_harm": 1924, + "selected_both_correct": 75, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.022327054291963577, + "operator": "ge", + "actual_fraction": 0.016644474034620507, + "metric": 0.07894526738759718, + "rescued": 1, + "harmed": 39, + "missed_rescue": 3313, + "avoided_harm": 1906, + "selected_both_correct": 160, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.024909960106015205, + "operator": "ge", + "actual_fraction": 0.0296271637816245, + "metric": 0.07886280908239668, + "rescued": 1, + "harmed": 69, + "missed_rescue": 3313, + "avoided_harm": 1876, + "selected_both_correct": 286, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -0.027814723551273346, + "operator": "ge", + "actual_fraction": 0.046937416777629824, + "metric": 0.07876261254060994, + "rescued": 1, + "harmed": 102, + "missed_rescue": 3313, + "avoided_harm": 1843, + "selected_both_correct": 461, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -0.028914429247379303, + "operator": "ge", + "actual_fraction": 0.05326231691078562, + "metric": 0.07922967968226176, + "rescued": 2, + "harmed": 110, + "missed_rescue": 3312, + "avoided_harm": 1835, + "selected_both_correct": 528, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -0.030170045793056488, + "operator": "ge", + "actual_fraction": 0.05992010652463382, + "metric": 0.07917808767949343, + "rescued": 2, + "harmed": 124, + "missed_rescue": 3312, + "avoided_harm": 1821, + "selected_both_correct": 594, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -0.03123854100704193, + "operator": "ge", + "actual_fraction": 0.06724367509986684, + "metric": 0.08652168148030992, + "rescued": 5, + "harmed": 139, + "missed_rescue": 3309, + "avoided_harm": 1806, + "selected_both_correct": 664, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -0.03410686179995537, + "operator": "ge", + "actual_fraction": 0.08721704394141146, + "metric": 0.08573997960186801, + "rescued": 6, + "harmed": 182, + "missed_rescue": 3308, + "avoided_harm": 1763, + "selected_both_correct": 860, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -0.03476816788315773, + "operator": "ge", + "actual_fraction": 0.09287616511318242, + "metric": 0.08533855094545621, + "rescued": 6, + "harmed": 200, + "missed_rescue": 3308, + "avoided_harm": 1745, + "selected_both_correct": 910, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -0.0365506075322628, + "operator": "ge", + "actual_fraction": 0.10785619174434088, + "metric": 0.08462726172642225, + "rescued": 6, + "harmed": 235, + "missed_rescue": 3308, + "avoided_harm": 1710, + "selected_both_correct": 1054, + "selected_both_wrong": 1 + }, + { + "budget": 0.11, + "threshold": -0.03818900138139725, + "operator": "ge", + "actual_fraction": 0.11784287616511319, + "metric": 0.08462444866867365, + "rescued": 7, + "harmed": 256, + "missed_rescue": 3307, + "avoided_harm": 1689, + "selected_both_correct": 1151, + "selected_both_wrong": 2 + }, + { + "budget": 0.12, + "threshold": -0.039673544466495514, + "operator": "ge", + "actual_fraction": 0.12716378162450068, + "metric": 0.08431158675236691, + "rescued": 7, + "harmed": 279, + "missed_rescue": 3307, + "avoided_harm": 1666, + "selected_both_correct": 1240, + "selected_both_wrong": 2 + }, + { + "budget": 0.13, + "threshold": -0.04085410386323929, + "operator": "ge", + "actual_fraction": 0.13648468708388814, + "metric": 0.0840276750632089, + "rescued": 7, + "harmed": 300, + "missed_rescue": 3307, + "avoided_harm": 1645, + "selected_both_correct": 1331, + "selected_both_wrong": 2 + }, + { + "budget": 0.14, + "threshold": -0.04204101860523224, + "operator": "ge", + "actual_fraction": 0.14613848202396804, + "metric": 0.08385470737270147, + "rescued": 7, + "harmed": 319, + "missed_rescue": 3307, + "avoided_harm": 1626, + "selected_both_correct": 1428, + "selected_both_wrong": 2 + }, + { + "budget": 0.15, + "threshold": -0.04278091713786125, + "operator": "ge", + "actual_fraction": 0.1514647137150466, + "metric": 0.08370670942943897, + "rescued": 7, + "harmed": 332, + "missed_rescue": 3307, + "avoided_harm": 1613, + "selected_both_correct": 1479, + "selected_both_wrong": 2 + }, + { + "budget": 0.16, + "threshold": -0.04322759434580803, + "operator": "ge", + "actual_fraction": 0.15579227696404793, + "metric": 0.08361544458700533, + "rescued": 7, + "harmed": 342, + "missed_rescue": 3307, + "avoided_harm": 1603, + "selected_both_correct": 1521, + "selected_both_wrong": 2 + }, + { + "budget": 0.17, + "threshold": -0.04394488036632538, + "operator": "ge", + "actual_fraction": 0.1617842876165113, + "metric": 0.08351220434393924, + "rescued": 7, + "harmed": 352, + "missed_rescue": 3307, + "avoided_harm": 1593, + "selected_both_correct": 1583, + "selected_both_wrong": 2 + }, + { + "budget": 0.18, + "threshold": -0.04530126228928566, + "operator": "ge", + "actual_fraction": 0.17177097203728361, + "metric": 0.08333120709231198, + "rescued": 7, + "harmed": 379, + "missed_rescue": 3307, + "avoided_harm": 1566, + "selected_both_correct": 1676, + "selected_both_wrong": 2 + }, + { + "budget": 0.19, + "threshold": -0.04661082476377487, + "operator": "ge", + "actual_fraction": 0.1810918774966711, + "metric": 0.08341398307559608, + "rescued": 8, + "harmed": 398, + "missed_rescue": 3306, + "avoided_harm": 1547, + "selected_both_correct": 1768, + "selected_both_wrong": 2 + }, + { + "budget": 0.2, + "threshold": -0.04730507358908653, + "operator": "ge", + "actual_fraction": 0.1870838881491345, + "metric": 0.08516887352267764, + "rescued": 9, + "harmed": 415, + "missed_rescue": 3305, + "avoided_harm": 1530, + "selected_both_correct": 1821, + "selected_both_wrong": 3 + }, + { + "budget": 0.21, + "threshold": -0.048246145248413086, + "operator": "ge", + "actual_fraction": 0.19640479360852198, + "metric": 0.08675280750372345, + "rescued": 10, + "harmed": 435, + "missed_rescue": 3304, + "avoided_harm": 1510, + "selected_both_correct": 1911, + "selected_both_wrong": 4 + }, + { + "budget": 0.22, + "threshold": -0.04900278151035309, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.08619173713061849, + "rescued": 10, + "harmed": 468, + "missed_rescue": 3304, + "avoided_harm": 1477, + "selected_both_correct": 2019, + "selected_both_wrong": 7 + }, + { + "budget": 0.23, + "threshold": -0.050042688846588135, + "operator": "ge", + "actual_fraction": 0.21471371504660453, + "metric": 0.08594774173844119, + "rescued": 10, + "harmed": 483, + "missed_rescue": 3304, + "avoided_harm": 1462, + "selected_both_correct": 2080, + "selected_both_wrong": 7 + }, + { + "budget": 0.24, + "threshold": -0.050901688635349274, + "operator": "ge", + "actual_fraction": 0.22536617842876164, + "metric": 0.08804054363419953, + "rescued": 41, + "harmed": 502, + "missed_rescue": 3273, + "avoided_harm": 1443, + "selected_both_correct": 2154, + "selected_both_wrong": 11 + }, + { + "budget": 0.25, + "threshold": -0.05161922052502632, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.08818772258585045, + "rescued": 60, + "harmed": 517, + "missed_rescue": 3254, + "avoided_harm": 1428, + "selected_both_correct": 2192, + "selected_both_wrong": 11 + }, + { + "budget": 0.26, + "threshold": -0.05245445668697357, + "operator": "ge", + "actual_fraction": 0.237017310252996, + "metric": 0.08841839409893854, + "rescued": 76, + "harmed": 525, + "missed_rescue": 3238, + "avoided_harm": 1420, + "selected_both_correct": 2235, + "selected_both_wrong": 12 + }, + { + "budget": 0.27, + "threshold": -0.053079016506671906, + "operator": "ge", + "actual_fraction": 0.24167776298268975, + "metric": 0.08850628284208313, + "rescued": 90, + "harmed": 536, + "missed_rescue": 3224, + "avoided_harm": 1409, + "selected_both_correct": 2265, + "selected_both_wrong": 13 + }, + { + "budget": 0.28, + "threshold": -0.053604189306497574, + "operator": "ge", + "actual_fraction": 0.2470039946737683, + "metric": 0.0886323434450378, + "rescued": 106, + "harmed": 548, + "missed_rescue": 3208, + "avoided_harm": 1397, + "selected_both_correct": 2301, + "selected_both_wrong": 13 + }, + { + "budget": 0.29, + "threshold": -0.05505518987774849, + "operator": "ge", + "actual_fraction": 0.2623169107856192, + "metric": 0.09067349863003854, + "rescued": 158, + "harmed": 582, + "missed_rescue": 3156, + "avoided_harm": 1363, + "selected_both_correct": 2396, + "selected_both_wrong": 16 + }, + { + "budget": 0.3, + "threshold": -0.05605518817901611, + "operator": "ge", + "actual_fraction": 0.27463382157123833, + "metric": 0.09250161598017743, + "rescued": 201, + "harmed": 607, + "missed_rescue": 3113, + "avoided_harm": 1338, + "selected_both_correct": 2469, + "selected_both_wrong": 23 + }, + { + "budget": 0.31, + "threshold": -0.056986160576343536, + "operator": "ge", + "actual_fraction": 0.28462050599201066, + "metric": 0.09322240603889413, + "rescued": 233, + "harmed": 625, + "missed_rescue": 3081, + "avoided_harm": 1320, + "selected_both_correct": 2537, + "selected_both_wrong": 25 + }, + { + "budget": 0.32, + "threshold": -0.0582931749522686, + "operator": "ge", + "actual_fraction": 0.29427430093209056, + "metric": 0.09484741449397432, + "rescued": 264, + "harmed": 642, + "missed_rescue": 3050, + "avoided_harm": 1303, + "selected_both_correct": 2598, + "selected_both_wrong": 32 + }, + { + "budget": 0.33, + "threshold": -0.0596650205552578, + "operator": "ge", + "actual_fraction": 0.3062583222370173, + "metric": 0.09509370405935859, + "rescued": 304, + "harmed": 671, + "missed_rescue": 3010, + "avoided_harm": 1274, + "selected_both_correct": 2671, + "selected_both_wrong": 34 + }, + { + "budget": 0.34, + "threshold": -0.060549296438694, + "operator": "ge", + "actual_fraction": 0.3132490013315579, + "metric": 0.09537687922458282, + "rescued": 331, + "harmed": 687, + "missed_rescue": 2983, + "avoided_harm": 1258, + "selected_both_correct": 2709, + "selected_both_wrong": 37 + }, + { + "budget": 0.35, + "threshold": -0.06178095191717148, + "operator": "ge", + "actual_fraction": 0.3189081225033289, + "metric": 0.09518164881276557, + "rescued": 349, + "harmed": 703, + "missed_rescue": 2965, + "avoided_harm": 1242, + "selected_both_correct": 2741, + "selected_both_wrong": 39 + }, + { + "budget": 0.36, + "threshold": -0.06328777223825455, + "operator": "ge", + "actual_fraction": 0.3318908122503329, + "metric": 0.09579948445467232, + "rescued": 389, + "harmed": 731, + "missed_rescue": 2925, + "avoided_harm": 1214, + "selected_both_correct": 2819, + "selected_both_wrong": 49 + }, + { + "budget": 0.37, + "threshold": -0.0647548958659172, + "operator": "ge", + "actual_fraction": 0.3418774966711052, + "metric": 0.09604832793526258, + "rescued": 424, + "harmed": 753, + "missed_rescue": 2890, + "avoided_harm": 1192, + "selected_both_correct": 2877, + "selected_both_wrong": 54 + }, + { + "budget": 0.38, + "threshold": -0.06637481600046158, + "operator": "ge", + "actual_fraction": 0.35352862849533956, + "metric": 0.09866030704730355, + "rescued": 472, + "harmed": 774, + "missed_rescue": 2842, + "avoided_harm": 1171, + "selected_both_correct": 2942, + "selected_both_wrong": 60 + }, + { + "budget": 0.39, + "threshold": -0.06774259358644485, + "operator": "ge", + "actual_fraction": 0.3641810918774967, + "metric": 0.10022118439970583, + "rescued": 514, + "harmed": 797, + "missed_rescue": 2800, + "avoided_harm": 1148, + "selected_both_correct": 3000, + "selected_both_wrong": 65 + }, + { + "budget": 0.4, + "threshold": -0.06843380630016327, + "operator": "ge", + "actual_fraction": 0.3698402130492676, + "metric": 0.10099642512611362, + "rescued": 537, + "harmed": 810, + "missed_rescue": 2777, + "avoided_harm": 1135, + "selected_both_correct": 3027, + "selected_both_wrong": 70 + }, + { + "budget": 0.41, + "threshold": -0.0694691464304924, + "operator": "ge", + "actual_fraction": 0.3801597869507324, + "metric": 0.10286568189357695, + "rescued": 582, + "harmed": 836, + "missed_rescue": 2732, + "avoided_harm": 1109, + "selected_both_correct": 3076, + "selected_both_wrong": 74 + }, + { + "budget": 0.42, + "threshold": -0.07112707942724228, + "operator": "ge", + "actual_fraction": 0.3938082556591212, + "metric": 0.1036415927515812, + "rescued": 630, + "harmed": 864, + "missed_rescue": 2684, + "avoided_harm": 1081, + "selected_both_correct": 3152, + "selected_both_wrong": 86 + }, + { + "budget": 0.43, + "threshold": -0.07192998379468918, + "operator": "ge", + "actual_fraction": 0.3988015978695073, + "metric": 0.10383106693904906, + "rescued": 650, + "harmed": 874, + "missed_rescue": 2664, + "avoided_harm": 1071, + "selected_both_correct": 3180, + "selected_both_wrong": 88 + }, + { + "budget": 0.44, + "threshold": -0.07266208529472351, + "operator": "ge", + "actual_fraction": 0.40279627163781623, + "metric": 0.10497587071399195, + "rescued": 665, + "harmed": 882, + "missed_rescue": 2649, + "avoided_harm": 1063, + "selected_both_correct": 3204, + "selected_both_wrong": 89 + }, + { + "budget": 0.45, + "threshold": -0.0744253620505333, + "operator": "ge", + "actual_fraction": 0.41711051930758986, + "metric": 0.10625030009652583, + "rescued": 721, + "harmed": 910, + "missed_rescue": 2593, + "avoided_harm": 1035, + "selected_both_correct": 3282, + "selected_both_wrong": 99 + }, + { + "budget": 0.46, + "threshold": -0.07532940804958344, + "operator": "ge", + "actual_fraction": 0.42243675099866845, + "metric": 0.10676350886210961, + "rescued": 745, + "harmed": 919, + "missed_rescue": 2569, + "avoided_harm": 1026, + "selected_both_correct": 3307, + "selected_both_wrong": 105 + }, + { + "budget": 0.47, + "threshold": -0.0770750343799591, + "operator": "ge", + "actual_fraction": 0.4380825565912117, + "metric": 0.10893787008842995, + "rescued": 817, + "harmed": 955, + "missed_rescue": 2497, + "avoided_harm": 990, + "selected_both_correct": 3377, + "selected_both_wrong": 115 + }, + { + "budget": 0.48, + "threshold": -0.07818037271499634, + "operator": "ge", + "actual_fraction": 0.44573901464713717, + "metric": 0.10946867779982243, + "rescued": 849, + "harmed": 973, + "missed_rescue": 2465, + "avoided_harm": 972, + "selected_both_correct": 3412, + "selected_both_wrong": 122 + }, + { + "budget": 0.49, + "threshold": -0.07998209446668625, + "operator": "ge", + "actual_fraction": 0.45905459387483355, + "metric": 0.11125477030640087, + "rescued": 900, + "harmed": 995, + "missed_rescue": 2414, + "avoided_harm": 950, + "selected_both_correct": 3484, + "selected_both_wrong": 137 + }, + { + "budget": 0.5, + "threshold": -0.08124949783086777, + "operator": "ge", + "actual_fraction": 0.46937416777629826, + "metric": 0.1119628778237788, + "rescued": 946, + "harmed": 1016, + "missed_rescue": 2368, + "avoided_harm": 929, + "selected_both_correct": 3537, + "selected_both_wrong": 141 + }, + { + "budget": 0.51, + "threshold": -0.08398465067148209, + "operator": "ge", + "actual_fraction": 0.49467376830892146, + "metric": 0.11739090876226205, + "rescued": 1064, + "harmed": 1070, + "missed_rescue": 2250, + "avoided_harm": 875, + "selected_both_correct": 3653, + "selected_both_wrong": 157 + }, + { + "budget": 0.52, + "threshold": -0.08537434041500092, + "operator": "ge", + "actual_fraction": 0.5056591211717709, + "metric": 0.11981696243623577, + "rescued": 1112, + "harmed": 1093, + "missed_rescue": 2202, + "avoided_harm": 852, + "selected_both_correct": 3704, + "selected_both_wrong": 167 + }, + { + "budget": 0.53, + "threshold": -0.08664409816265106, + "operator": "ge", + "actual_fraction": 0.5159786950732357, + "metric": 0.12050661028396976, + "rescued": 1152, + "harmed": 1116, + "missed_rescue": 2162, + "avoided_harm": 829, + "selected_both_correct": 3756, + "selected_both_wrong": 176 + }, + { + "budget": 0.54, + "threshold": -0.08777308464050293, + "operator": "ge", + "actual_fraction": 0.524300932090546, + "metric": 0.12109159247774551, + "rescued": 1190, + "harmed": 1137, + "missed_rescue": 2124, + "avoided_harm": 808, + "selected_both_correct": 3793, + "selected_both_wrong": 180 + }, + { + "budget": 0.55, + "threshold": -0.08980843424797058, + "operator": "ge", + "actual_fraction": 0.5386151797603196, + "metric": 0.12283117300011699, + "rescued": 1260, + "harmed": 1164, + "missed_rescue": 2054, + "avoided_harm": 781, + "selected_both_correct": 3860, + "selected_both_wrong": 188 + }, + { + "budget": 0.56, + "threshold": -0.0906188115477562, + "operator": "ge", + "actual_fraction": 0.546271637816245, + "metric": 0.12427034914228642, + "rescued": 1293, + "harmed": 1180, + "missed_rescue": 2021, + "avoided_harm": 765, + "selected_both_correct": 3897, + "selected_both_wrong": 194 + }, + { + "budget": 0.57, + "threshold": -0.09160211682319641, + "operator": "ge", + "actual_fraction": 0.5555925432756325, + "metric": 0.12600739832532146, + "rescued": 1339, + "harmed": 1194, + "missed_rescue": 1975, + "avoided_harm": 751, + "selected_both_correct": 3940, + "selected_both_wrong": 203 + }, + { + "budget": 0.58, + "threshold": -0.09283093363046646, + "operator": "ge", + "actual_fraction": 0.568242343541944, + "metric": 0.12948657153370394, + "rescued": 1401, + "harmed": 1224, + "missed_rescue": 1913, + "avoided_harm": 721, + "selected_both_correct": 3996, + "selected_both_wrong": 207 + }, + { + "budget": 0.59, + "threshold": -0.0941123366355896, + "operator": "ge", + "actual_fraction": 0.5778961384820239, + "metric": 0.1321361114270833, + "rescued": 1447, + "harmed": 1245, + "missed_rescue": 1867, + "avoided_harm": 700, + "selected_both_correct": 4036, + "selected_both_wrong": 216 + }, + { + "budget": 0.6, + "threshold": -0.09614869952201843, + "operator": "ge", + "actual_fraction": 0.5878828229027963, + "metric": 0.13359562560258884, + "rescued": 1495, + "harmed": 1262, + "missed_rescue": 1819, + "avoided_harm": 683, + "selected_both_correct": 4083, + "selected_both_wrong": 224 + }, + { + "budget": 0.61, + "threshold": -0.09772664308547974, + "operator": "ge", + "actual_fraction": 0.5975366178428761, + "metric": 0.135418307343696, + "rescued": 1537, + "harmed": 1279, + "missed_rescue": 1777, + "avoided_harm": 666, + "selected_both_correct": 4133, + "selected_both_wrong": 231 + }, + { + "budget": 0.62, + "threshold": -0.09911532700061798, + "operator": "ge", + "actual_fraction": 0.6081890812250332, + "metric": 0.13720391892557468, + "rescued": 1585, + "harmed": 1294, + "missed_rescue": 1729, + "avoided_harm": 651, + "selected_both_correct": 4189, + "selected_both_wrong": 240 + }, + { + "budget": 0.63, + "threshold": -0.10007988661527634, + "operator": "ge", + "actual_fraction": 0.6161784287616512, + "metric": 0.13839536227953134, + "rescued": 1625, + "harmed": 1312, + "missed_rescue": 1689, + "avoided_harm": 633, + "selected_both_correct": 4222, + "selected_both_wrong": 245 + }, + { + "budget": 0.64, + "threshold": -0.10177135467529297, + "operator": "ge", + "actual_fraction": 0.6278295605858855, + "metric": 0.14186473384550358, + "rescued": 1685, + "harmed": 1327, + "missed_rescue": 1629, + "avoided_harm": 618, + "selected_both_correct": 4280, + "selected_both_wrong": 252 + }, + { + "budget": 0.65, + "threshold": -0.10395076125860214, + "operator": "ge", + "actual_fraction": 0.6451398135818908, + "metric": 0.14514480336324412, + "rescued": 1771, + "harmed": 1357, + "missed_rescue": 1543, + "avoided_harm": 588, + "selected_both_correct": 4362, + "selected_both_wrong": 262 + }, + { + "budget": 0.66, + "threshold": -0.10524322092533112, + "operator": "ge", + "actual_fraction": 0.6491344873501997, + "metric": 0.14705419026423194, + "rescued": 1792, + "harmed": 1362, + "missed_rescue": 1522, + "avoided_harm": 583, + "selected_both_correct": 4381, + "selected_both_wrong": 265 + }, + { + "budget": 0.67, + "threshold": -0.10689426958560944, + "operator": "ge", + "actual_fraction": 0.6601198402130493, + "metric": 0.14778857092985256, + "rescued": 1842, + "harmed": 1385, + "missed_rescue": 1472, + "avoided_harm": 560, + "selected_both_correct": 4428, + "selected_both_wrong": 277 + }, + { + "budget": 0.68, + "threshold": -0.11014843732118607, + "operator": "ge", + "actual_fraction": 0.6754327563249002, + "metric": 0.15144863687072044, + "rescued": 1911, + "harmed": 1408, + "missed_rescue": 1403, + "avoided_harm": 537, + "selected_both_correct": 4509, + "selected_both_wrong": 288 + }, + { + "budget": 0.69, + "threshold": -0.11117545515298843, + "operator": "ge", + "actual_fraction": 0.6790945406125166, + "metric": 0.1527001767891735, + "rescued": 1931, + "harmed": 1414, + "missed_rescue": 1383, + "avoided_harm": 531, + "selected_both_correct": 4524, + "selected_both_wrong": 291 + }, + { + "budget": 0.7, + "threshold": -0.11230025440454483, + "operator": "ge", + "actual_fraction": 0.6870838881491345, + "metric": 0.15446609724321056, + "rescued": 1973, + "harmed": 1424, + "missed_rescue": 1341, + "avoided_harm": 521, + "selected_both_correct": 4561, + "selected_both_wrong": 298 + }, + { + "budget": 0.71, + "threshold": -0.11423606425523758, + "operator": "ge", + "actual_fraction": 0.6960719041278296, + "metric": 0.1566074758661257, + "rescued": 2016, + "harmed": 1440, + "missed_rescue": 1298, + "avoided_harm": 505, + "selected_both_correct": 4599, + "selected_both_wrong": 309 + }, + { + "budget": 0.72, + "threshold": -0.11647249013185501, + "operator": "ge", + "actual_fraction": 0.7053928095872171, + "metric": 0.15830843131436304, + "rescued": 2059, + "harmed": 1456, + "missed_rescue": 1255, + "avoided_harm": 489, + "selected_both_correct": 4643, + "selected_both_wrong": 318 + }, + { + "budget": 0.73, + "threshold": -0.11962417513132095, + "operator": "ge", + "actual_fraction": 0.7193741677762983, + "metric": 0.16212144588492078, + "rescued": 2121, + "harmed": 1481, + "missed_rescue": 1193, + "avoided_harm": 464, + "selected_both_correct": 4706, + "selected_both_wrong": 336 + }, + { + "budget": 0.74, + "threshold": -0.12126146256923676, + "operator": "ge", + "actual_fraction": 0.7250332889480693, + "metric": 0.16312718485088668, + "rescued": 2145, + "harmed": 1491, + "missed_rescue": 1169, + "avoided_harm": 454, + "selected_both_correct": 4732, + "selected_both_wrong": 344 + }, + { + "budget": 0.75, + "threshold": -0.1230982318520546, + "operator": "ge", + "actual_fraction": 0.7366844207723036, + "metric": 0.1660266588911306, + "rescued": 2198, + "harmed": 1515, + "missed_rescue": 1116, + "avoided_harm": 430, + "selected_both_correct": 4785, + "selected_both_wrong": 354 + }, + { + "budget": 0.76, + "threshold": -0.12517565488815308, + "operator": "ge", + "actual_fraction": 0.7446737683089214, + "metric": 0.16822905417177203, + "rescued": 2234, + "harmed": 1526, + "missed_rescue": 1080, + "avoided_harm": 419, + "selected_both_correct": 4828, + "selected_both_wrong": 360 + }, + { + "budget": 0.77, + "threshold": -0.12628257274627686, + "operator": "ge", + "actual_fraction": 0.7480026631158455, + "metric": 0.1691185459978786, + "rescued": 2251, + "harmed": 1531, + "missed_rescue": 1063, + "avoided_harm": 414, + "selected_both_correct": 4843, + "selected_both_wrong": 363 + }, + { + "budget": 0.78, + "threshold": -0.12975578010082245, + "operator": "ge", + "actual_fraction": 0.7603195739014648, + "metric": 0.17190478913940857, + "rescued": 2306, + "harmed": 1552, + "missed_rescue": 1008, + "avoided_harm": 393, + "selected_both_correct": 4897, + "selected_both_wrong": 381 + }, + { + "budget": 0.79, + "threshold": -0.13229140639305115, + "operator": "ge", + "actual_fraction": 0.7713049267643143, + "metric": 0.17440054880534744, + "rescued": 2355, + "harmed": 1568, + "missed_rescue": 959, + "avoided_harm": 377, + "selected_both_correct": 4952, + "selected_both_wrong": 393 + }, + { + "budget": 0.8, + "threshold": -0.1373685747385025, + "operator": "ge", + "actual_fraction": 0.7909454061251664, + "metric": 0.17894927236127425, + "rescued": 2455, + "harmed": 1608, + "missed_rescue": 859, + "avoided_harm": 337, + "selected_both_correct": 5030, + "selected_both_wrong": 411 + }, + { + "budget": 0.81, + "threshold": -0.1407174915075302, + "operator": "ge", + "actual_fraction": 0.8012649800266312, + "metric": 0.18049284683090172, + "rescued": 2502, + "harmed": 1631, + "missed_rescue": 812, + "avoided_harm": 314, + "selected_both_correct": 5069, + "selected_both_wrong": 426 + }, + { + "budget": 0.82, + "threshold": -0.1433146893978119, + "operator": "ge", + "actual_fraction": 0.8125832223701731, + "metric": 0.18188035357936716, + "rescued": 2556, + "harmed": 1658, + "missed_rescue": 758, + "avoided_harm": 287, + "selected_both_correct": 5115, + "selected_both_wrong": 435 + }, + { + "budget": 0.83, + "threshold": -0.1457100659608841, + "operator": "ge", + "actual_fraction": 0.8199067909454061, + "metric": 0.1842965007470308, + "rescued": 2593, + "harmed": 1670, + "missed_rescue": 721, + "avoided_harm": 275, + "selected_both_correct": 5145, + "selected_both_wrong": 444 + }, + { + "budget": 0.84, + "threshold": -0.148710235953331, + "operator": "ge", + "actual_fraction": 0.8242343541944075, + "metric": 0.18526487033383585, + "rescued": 2610, + "harmed": 1675, + "missed_rescue": 704, + "avoided_harm": 270, + "selected_both_correct": 5167, + "selected_both_wrong": 452 + }, + { + "budget": 0.85, + "threshold": -0.15291842818260193, + "operator": "ge", + "actual_fraction": 0.8338881491344874, + "metric": 0.18825160118942358, + "rescued": 2656, + "harmed": 1694, + "missed_rescue": 658, + "avoided_harm": 251, + "selected_both_correct": 5208, + "selected_both_wrong": 462 + }, + { + "budget": 0.86, + "threshold": -0.15889839828014374, + "operator": "ge", + "actual_fraction": 0.8458721704394141, + "metric": 0.19240881033397328, + "rescued": 2704, + "harmed": 1712, + "missed_rescue": 610, + "avoided_harm": 233, + "selected_both_correct": 5271, + "selected_both_wrong": 477 + }, + { + "budget": 0.87, + "threshold": -0.16629797220230103, + "operator": "ge", + "actual_fraction": 0.8585219707057257, + "metric": 0.1975333545503005, + "rescued": 2763, + "harmed": 1734, + "missed_rescue": 551, + "avoided_harm": 211, + "selected_both_correct": 5328, + "selected_both_wrong": 491 + }, + { + "budget": 0.88, + "threshold": -0.17295491695404053, + "operator": "ge", + "actual_fraction": 0.8655126498002663, + "metric": 0.19989071661116012, + "rescued": 2792, + "harmed": 1743, + "missed_rescue": 522, + "avoided_harm": 202, + "selected_both_correct": 5365, + "selected_both_wrong": 500 + }, + { + "budget": 0.89, + "threshold": -0.18053314089775085, + "operator": "ge", + "actual_fraction": 0.8738348868175766, + "metric": 0.20467943147275502, + "rescued": 2828, + "harmed": 1754, + "missed_rescue": 486, + "avoided_harm": 191, + "selected_both_correct": 5407, + "selected_both_wrong": 511 + }, + { + "budget": 0.9, + "threshold": -0.19442276656627655, + "operator": "ge", + "actual_fraction": 0.8878162450066578, + "metric": 0.20833827941063784, + "rescued": 2888, + "harmed": 1779, + "missed_rescue": 426, + "avoided_harm": 166, + "selected_both_correct": 5471, + "selected_both_wrong": 530 + }, + { + "budget": 0.91, + "threshold": -0.20900504291057587, + "operator": "ge", + "actual_fraction": 0.8961384820239681, + "metric": 0.2085245066088965, + "rescued": 2924, + "harmed": 1790, + "missed_rescue": 390, + "avoided_harm": 155, + "selected_both_correct": 5515, + "selected_both_wrong": 539 + }, + { + "budget": 0.92, + "threshold": -0.22383525967597961, + "operator": "ge", + "actual_fraction": 0.9027962716378163, + "metric": 0.2108748825898504, + "rescued": 2952, + "harmed": 1799, + "missed_rescue": 362, + "avoided_harm": 146, + "selected_both_correct": 5550, + "selected_both_wrong": 547 + }, + { + "budget": 0.93, + "threshold": -0.25952714681625366, + "operator": "ge", + "actual_fraction": 0.9184420772303595, + "metric": 0.21438099190536286, + "rescued": 3019, + "harmed": 1827, + "missed_rescue": 295, + "avoided_harm": 118, + "selected_both_correct": 5620, + "selected_both_wrong": 570 + }, + { + "budget": 0.94, + "threshold": -0.2804492115974426, + "operator": "ge", + "actual_fraction": 0.9257656458055925, + "metric": 0.21506366828770151, + "rescued": 3050, + "harmed": 1842, + "missed_rescue": 264, + "avoided_harm": 103, + "selected_both_correct": 5649, + "selected_both_wrong": 583 + }, + { + "budget": 0.95, + "threshold": -0.2984963357448578, + "operator": "ge", + "actual_fraction": 0.9320905459387483, + "metric": 0.21567531720794794, + "rescued": 3069, + "harmed": 1849, + "missed_rescue": 245, + "avoided_harm": 96, + "selected_both_correct": 5689, + "selected_both_wrong": 593 + }, + { + "budget": 0.96, + "threshold": -0.32223379611968994, + "operator": "ge", + "actual_fraction": 0.9410785619174434, + "metric": 0.21264209690310557, + "rescued": 3101, + "harmed": 1865, + "missed_rescue": 213, + "avoided_harm": 80, + "selected_both_correct": 5735, + "selected_both_wrong": 607 + }, + { + "budget": 0.97, + "threshold": -0.36503666639328003, + "operator": "ge", + "actual_fraction": 0.9573901464713716, + "metric": 0.21331564253317523, + "rescued": 3160, + "harmed": 1887, + "missed_rescue": 154, + "avoided_harm": 58, + "selected_both_correct": 5818, + "selected_both_wrong": 639 + }, + { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.9790279627163782, + "metric": 0.21609512872414097, + "rescued": 3244, + "harmed": 1926, + "missed_rescue": 70, + "avoided_harm": 19, + "selected_both_correct": 5928, + "selected_both_wrong": 666 + }, + { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9860186418109188, + "metric": 0.21817095656956434, + "rescued": 3268, + "harmed": 1932, + "missed_rescue": 46, + "avoided_harm": 13, + "selected_both_correct": 5967, + "selected_both_wrong": 681 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.22361115727328065, + "rescued": 3314, + "harmed": 1945, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 6060, + "selected_both_wrong": 697 + } + ] + }, + "learned": { + "validation_area_0_25": 0.10406870592389231, + "validation_point": { + "metric": 0.21913219001083611, + "metric_std": 0.001307931612591827, + "actual_fraction": 0.7908701854493579, + "actual_fraction_std": 0.034156467798975576, + "per_seed": [ + { + "seed": 2026, + "budget": 0.76, + "threshold": -0.01648801565170288, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.2207269708303888, + "rescue": 781, + "harm": 340, + "missed_rescue": 5 + }, + { + "seed": 2027, + "budget": 0.79, + "threshold": -0.08512895554304123, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2191270620579996, + "rescue": 784, + "harm": 353, + "missed_rescue": 2 + }, + { + "seed": 2028, + "budget": 0.78, + "threshold": -0.0833086222410202, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.2193613683209817, + "rescue": 784, + "harm": 350, + "missed_rescue": 2 + }, + { + "seed": 2029, + "budget": 0.85, + "threshold": -0.15434026718139648, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2170841805238287, + "rescue": 785, + "harm": 385, + "missed_rescue": 1 + }, + { + "seed": 2030, + "budget": 0.78, + "threshold": -0.17031188309192657, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.2193613683209817, + "rescue": 784, + "harm": 350, + "missed_rescue": 2 + } + ], + "rescued": 783.6, + "harmed": 355.6, + "missed_rescue": 2.4, + "avoided_harm": 119.39999999999998 + }, + "test_point": { + "metric": 0.22907685973193592, + "metric_std": 0.005805161470975003, + "actual_fraction": 0.8000665778961386, + "actual_fraction_std": 0.031921480323791725, + "per_seed": [ + { + "seed": 2026, + "budget": 0.76, + "threshold": -0.01648801565170288, + "operator": "ge", + "actual_fraction": 0.7683089214380826, + "metric": 0.22151919702262796, + "rescue": 3258, + "harm": 1424, + "missed_rescue": 56 + }, + { + "seed": 2027, + "budget": 0.79, + "threshold": -0.08512895554304123, + "operator": "ge", + "actual_fraction": 0.7939414114513982, + "metric": 0.22590669255486498, + "rescue": 3291, + "harm": 1474, + "missed_rescue": 23 + }, + { + "seed": 2028, + "budget": 0.78, + "threshold": -0.0833086222410202, + "operator": "ge", + "actual_fraction": 0.7942743009320905, + "metric": 0.23706753735104394, + "rescue": 3302, + "harm": 1468, + "missed_rescue": 12 + }, + { + "seed": 2029, + "budget": 0.85, + "threshold": -0.15434026718139648, + "operator": "ge", + "actual_fraction": 0.8538615179760319, + "metric": 0.23031420481265008, + "rescue": 3307, + "harm": 1586, + "missed_rescue": 7 + }, + { + "seed": 2030, + "budget": 0.78, + "threshold": -0.17031188309192657, + "operator": "ge", + "actual_fraction": 0.7899467376830892, + "metric": 0.23057666691849263, + "rescue": 3304, + "harm": 1469, + "missed_rescue": 10 + } + ], + "rescued": 3292.4, + "harmed": 1484.2, + "missed_rescue": 21.6, + "avoided_harm": 460.79999999999995 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.07641657905799261, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.09414101912073015, + "metric_std": 0.015285544931164205, + "actual_fraction": 0.009985734664764621, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.09099729460721984, + "metric_std": 0.012369708220388763, + "actual_fraction": 0.019971469329529243, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.09360247945693914, + "metric_std": 0.00894986485112822, + "actual_fraction": 0.029957203994293864, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.09257324431215734, + "metric_std": 0.007797806869091646, + "actual_fraction": 0.039942938659058486, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.0921819972692087, + "metric_std": 0.0069574162689952064, + "actual_fraction": 0.04992867332382311, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.09195762022749276, + "metric_std": 0.006255110661970155, + "actual_fraction": 0.05991440798858773, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.09376058168657833, + "metric_std": 0.008931725475845234, + "actual_fraction": 0.06990014265335236, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.09304861580390114, + "metric_std": 0.008018727831112528, + "actual_fraction": 0.07988587731811697, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.09470413672485042, + "metric_std": 0.004134625809739239, + "actual_fraction": 0.0898716119828816, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.09707586946494952, + "metric_std": 0.004561737695879227, + "actual_fraction": 0.09985734664764621, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.09736081505572086, + "metric_std": 0.004575044678369787, + "actual_fraction": 0.10984308131241086, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.12, + "metric": 0.10081679127356609, + "metric_std": 0.00335560943926372, + "actual_fraction": 0.11982881597717546, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.10080236540976997, + "metric_std": 0.004160378646411935, + "actual_fraction": 0.12981455064194009, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.10240883940353204, + "metric_std": 0.001781562048407857, + "actual_fraction": 0.1398002853067047, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.10861241960578012, + "metric_std": 0.0031592161290477337, + "actual_fraction": 0.14978601997146934, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.11023157919676534, + "metric_std": 0.005344448438972847, + "actual_fraction": 0.15977175463623394, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.11036200916490815, + "metric_std": 0.004984003227123095, + "actual_fraction": 0.16975748930099857, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.11563759795882507, + "metric_std": 0.004609346320432851, + "actual_fraction": 0.1797432239657632, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.1171065235739075, + "metric_std": 0.0035847406683710927, + "actual_fraction": 0.18972895863052783, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.11947110932297908, + "metric_std": 0.004044442185282369, + "actual_fraction": 0.19971469329529243, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.12192524896113058, + "metric_std": 0.0027107134294204215, + "actual_fraction": 0.20970042796005703, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.12340030611802784, + "metric_std": 0.003327474178708066, + "actual_fraction": 0.2196861626248217, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.23, + "metric": 0.12606618203213593, + "metric_std": 0.001852683285261849, + "actual_fraction": 0.2296718972895863, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.24, + "metric": 0.12638860851953035, + "metric_std": 0.0016686923142580114, + "actual_fraction": 0.2396576319543509, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.12958768885509553, + "metric_std": 0.004333413086289745, + "actual_fraction": 0.24964336661911551, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.1296550569750944, + "metric_std": 0.004231024034586545, + "actual_fraction": 0.25962910128388017, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.12994741217038264, + "metric_std": 0.004525191501283419, + "actual_fraction": 0.26961483594864477, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.1303893942385133, + "metric_std": 0.004262605097070228, + "actual_fraction": 0.2796005706134094, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.13095727807835086, + "metric_std": 0.004440448929857145, + "actual_fraction": 0.289586305278174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.1323689428345635, + "metric_std": 0.0037894381407460724, + "actual_fraction": 0.2995720399429387, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.13410664620385213, + "metric_std": 0.0043954116972769455, + "actual_fraction": 0.3095577746077033, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.13469018337605212, + "metric_std": 0.004353459221080619, + "actual_fraction": 0.3195435092724679, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.1379192063825932, + "metric_std": 0.004960088542264816, + "actual_fraction": 0.32952924393723254, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.1393500655007824, + "metric_std": 0.004656208947413315, + "actual_fraction": 0.33951497860199714, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.14184435984996058, + "metric_std": 0.005636027284561285, + "actual_fraction": 0.34950071326676174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.14348257165316555, + "metric_std": 0.00565799010334534, + "actual_fraction": 0.3594864479315264, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.1473097168380614, + "metric_std": 0.004142436276975065, + "actual_fraction": 0.369472182596291, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.14919078408265246, + "metric_std": 0.004037240858707113, + "actual_fraction": 0.37945791726105566, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.15003187557901917, + "metric_std": 0.004928108369170114, + "actual_fraction": 0.38944365192582026, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.15186332792493448, + "metric_std": 0.004808686741753248, + "actual_fraction": 0.39942938659058486, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.15362928902256845, + "metric_std": 0.00449670179750053, + "actual_fraction": 0.4094151212553495, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.42, + "metric": 0.1552210283162994, + "metric_std": 0.005056000584163085, + "actual_fraction": 0.41940085592011406, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.15627554236468594, + "metric_std": 0.004374282968918198, + "actual_fraction": 0.4293865905848787, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.44, + "metric": 0.1574134882230711, + "metric_std": 0.004903226130212338, + "actual_fraction": 0.4393723252496434, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.45, + "metric": 0.16113435627859163, + "metric_std": 0.003459218671098634, + "actual_fraction": 0.44935805991440797, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.16362436943464492, + "metric_std": 0.004442739654462731, + "actual_fraction": 0.4593437945791726, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.47, + "metric": 0.16420614329152441, + "metric_std": 0.004503998845122576, + "actual_fraction": 0.4693295292439372, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.48, + "metric": 0.16477003651850428, + "metric_std": 0.0026701737062594035, + "actual_fraction": 0.4793152639087018, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.1660743112331527, + "metric_std": 0.0028717444744918055, + "actual_fraction": 0.4893009985734665, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.5, + "metric": 0.16652844416012355, + "metric_std": 0.0033441872146407316, + "actual_fraction": 0.49928673323823103, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.51, + "metric": 0.1673211627397398, + "metric_std": 0.004062348243111666, + "actual_fraction": 0.5092724679029957, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.16876700210504056, + "metric_std": 0.00411569208218918, + "actual_fraction": 0.5192582025677603, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.169492657005102, + "metric_std": 0.0034370009957123853, + "actual_fraction": 0.5292439372325249, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.17115653136163675, + "metric_std": 0.003286224439548943, + "actual_fraction": 0.5392296718972895, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.1723947970619219, + "metric_std": 0.003316982395691029, + "actual_fraction": 0.5492154065620543, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.17290641719780783, + "metric_std": 0.003788788458687652, + "actual_fraction": 0.5592011412268189, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.17280755597775058, + "metric_std": 0.004341199662229731, + "actual_fraction": 0.5691868758915835, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.17416039702729444, + "metric_std": 0.004820317642684566, + "actual_fraction": 0.579172610556348, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.17540561863393814, + "metric_std": 0.003913059832033369, + "actual_fraction": 0.5891583452211127, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.17646930658784837, + "metric_std": 0.0037097444790474316, + "actual_fraction": 0.5991440798858774, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.17847416630966423, + "metric_std": 0.004012076376314852, + "actual_fraction": 0.609129814550642, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.17948058628877853, + "metric_std": 0.0027453016501652904, + "actual_fraction": 0.6191155492154066, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.1795913043722143, + "metric_std": 0.003644738547268133, + "actual_fraction": 0.6291012838801712, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.1808458196345418, + "metric_std": 0.0025564477103024873, + "actual_fraction": 0.6390870185449358, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.18360965186275374, + "metric_std": 0.0028634331507273202, + "actual_fraction": 0.6490727532097005, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.1859191709270269, + "metric_std": 0.002871996661161549, + "actual_fraction": 0.6590584878744651, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.18624461617566745, + "metric_std": 0.0034485117704404385, + "actual_fraction": 0.6690442225392297, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.1879634899028429, + "metric_std": 0.00296698973764637, + "actual_fraction": 0.6790299572039943, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.1911583021827145, + "metric_std": 0.0035329581904681865, + "actual_fraction": 0.6890156918687589, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.19391236925362237, + "metric_std": 0.005085987048346166, + "actual_fraction": 0.6990014265335235, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.19629928549076742, + "metric_std": 0.004998566971964412, + "actual_fraction": 0.7089871611982882, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.1989616537712589, + "metric_std": 0.0043253622788835605, + "actual_fraction": 0.7189728958630528, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.20214854386724088, + "metric_std": 0.006094179183624899, + "actual_fraction": 0.7289586305278174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.20529853570990103, + "metric_std": 0.005723733712065697, + "actual_fraction": 0.738944365192582, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.21007972396657565, + "metric_std": 0.004953332212885084, + "actual_fraction": 0.7489300998573466, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.2130622657078664, + "metric_std": 0.0059127445640678785, + "actual_fraction": 0.7589158345221113, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.21490645220198598, + "metric_std": 0.00484589043609993, + "actual_fraction": 0.7689015691868759, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.21721500748152162, + "metric_std": 0.0033814802075767753, + "actual_fraction": 0.7788873038516405, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.21775387471699456, + "metric_std": 0.001972792510722042, + "actual_fraction": 0.7888730385164051, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.2176289586982237, + "metric_std": 0.0010299212411249074, + "actual_fraction": 0.7988587731811697, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.21734266724919427, + "metric_std": 0.001045429848736021, + "actual_fraction": 0.8088445078459344, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.21693603363116365, + "metric_std": 0.001060039178352576, + "actual_fraction": 0.818830242510699, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.21661546726265907, + "metric_std": 0.0009972853804493156, + "actual_fraction": 0.8288159771754635, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.21664645674404523, + "metric_std": 0.0010150126529467197, + "actual_fraction": 0.8388017118402281, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.21713528994525316, + "metric_std": 0.00014384596498798205, + "actual_fraction": 0.8487874465049929, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.86, + "metric": 0.21675969764833222, + "metric_std": 0.00036259131156877187, + "actual_fraction": 0.8587731811697574, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.87, + "metric": 0.2160439959870398, + "metric_std": 0.00042825699923299286, + "actual_fraction": 0.8687589158345222, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.88, + "metric": 0.21534668696727408, + "metric_std": 0.0011758179557312856, + "actual_fraction": 0.8787446504992869, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.89, + "metric": 0.21428366971288576, + "metric_std": 0.0015275960380064357, + "actual_fraction": 0.8887303851640513, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.2138958608937696, + "metric_std": 0.0015545046667362961, + "actual_fraction": 0.8987161198288159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.21342514100325594, + "metric_std": 0.0016509081047086916, + "actual_fraction": 0.9087018544935808, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.21245029952231195, + "metric_std": 0.0019799501644197017, + "actual_fraction": 0.9186875891583453, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.93, + "metric": 0.21189059660206086, + "metric_std": 0.0020801976479249474, + "actual_fraction": 0.9286733238231099, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.21116873032081243, + "metric_std": 0.0017803536254012535, + "actual_fraction": 0.9386590584878745, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.95, + "metric": 0.21031606582622636, + "metric_std": 0.0016961499206498201, + "actual_fraction": 0.948644793152639, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.20934368235023415, + "metric_std": 0.0015289805833000642, + "actual_fraction": 0.9586305278174037, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.20994203536384987, + "metric_std": 0.0014426822840708892, + "actual_fraction": 0.9686162624821684, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.98, + "metric": 0.20902481705857986, + "metric_std": 0.0009355872909266658, + "actual_fraction": 0.978601997146933, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.99, + "metric": 0.20808550623221192, + "metric_std": 0.000600188911742931, + "actual_fraction": 0.9885877318116976, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.20666016863629594, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.07887397871043929, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.0808249873425464, + "metric_std": 0.003196408546866411, + "actual_fraction": 0.00492676431424767, + "actual_fraction_std": 0.0010367783956323194 + }, + { + "budget": 0.02, + "metric": 0.0844408324179301, + "metric_std": 0.004654988576944642, + "actual_fraction": 0.012583222370173102, + "actual_fraction_std": 0.0016375997172102317 + }, + { + "budget": 0.03, + "metric": 0.08949736688635994, + "metric_std": 0.008928855695973202, + "actual_fraction": 0.019241011984021304, + "actual_fraction_std": 0.0018293783840258443 + }, + { + "budget": 0.04, + "metric": 0.09460996238962568, + "metric_std": 0.008647457823444173, + "actual_fraction": 0.025499334221038616, + "actual_fraction_std": 0.0031014153658727337 + }, + { + "budget": 0.05, + "metric": 0.10030091233749505, + "metric_std": 0.006874592992960233, + "actual_fraction": 0.03568575233022636, + "actual_fraction_std": 0.0050121704500973125 + }, + { + "budget": 0.06, + "metric": 0.10670223926798542, + "metric_std": 0.008211509944458465, + "actual_fraction": 0.04387483355525966, + "actual_fraction_std": 0.0062650472409372686 + }, + { + "budget": 0.07, + "metric": 0.1064228330940539, + "metric_std": 0.00792226578460164, + "actual_fraction": 0.051531291611185095, + "actual_fraction_std": 0.004572840165511312 + }, + { + "budget": 0.08, + "metric": 0.10694232748713728, + "metric_std": 0.005923396110901602, + "actual_fraction": 0.061984021304926765, + "actual_fraction_std": 0.0032903898250736512 + }, + { + "budget": 0.09, + "metric": 0.10788410138489894, + "metric_std": 0.005969786975003347, + "actual_fraction": 0.06857523302263649, + "actual_fraction_std": 0.0037440818505998646 + }, + { + "budget": 0.1, + "metric": 0.11153342992020163, + "metric_std": 0.0033140142569648627, + "actual_fraction": 0.07836218375499335, + "actual_fraction_std": 0.006722389657840373 + }, + { + "budget": 0.11, + "metric": 0.1127009682199785, + "metric_std": 0.0026221562354102734, + "actual_fraction": 0.08448735019973368, + "actual_fraction_std": 0.006900534769931109 + }, + { + "budget": 0.12, + "metric": 0.1122270338248779, + "metric_std": 0.0024969554741084542, + "actual_fraction": 0.09167776298268974, + "actual_fraction_std": 0.0070561577416891124 + }, + { + "budget": 0.13, + "metric": 0.11165227047688112, + "metric_std": 0.0019737833417243743, + "actual_fraction": 0.10093209054593877, + "actual_fraction_std": 0.004797536857604234 + }, + { + "budget": 0.14, + "metric": 0.11355268572173945, + "metric_std": 0.002681390524424899, + "actual_fraction": 0.11351531291611186, + "actual_fraction_std": 0.004570416184336209 + }, + { + "budget": 0.15, + "metric": 0.11513254614491115, + "metric_std": 0.0047615378725324505, + "actual_fraction": 0.12283621837549932, + "actual_fraction_std": 0.006752817168808516 + }, + { + "budget": 0.16, + "metric": 0.11765708408237222, + "metric_std": 0.004747137185386672, + "actual_fraction": 0.13122503328894808, + "actual_fraction_std": 0.009307223170646627 + }, + { + "budget": 0.17, + "metric": 0.11874471355276506, + "metric_std": 0.006207332354314355, + "actual_fraction": 0.14420772303595206, + "actual_fraction_std": 0.009087334087411986 + }, + { + "budget": 0.18, + "metric": 0.1189154677291675, + "metric_std": 0.006291268804628001, + "actual_fraction": 0.15352862849533955, + "actual_fraction_std": 0.008764544465386959 + }, + { + "budget": 0.19, + "metric": 0.12034000788148043, + "metric_std": 0.0057280866941877085, + "actual_fraction": 0.16438082556591213, + "actual_fraction_std": 0.010689847848962078 + }, + { + "budget": 0.2, + "metric": 0.12135059533408779, + "metric_std": 0.005729488097514116, + "actual_fraction": 0.17350199733688415, + "actual_fraction_std": 0.008811830413337099 + }, + { + "budget": 0.21, + "metric": 0.12160166394273661, + "metric_std": 0.005491614783760345, + "actual_fraction": 0.1847536617842876, + "actual_fraction_std": 0.01116297112989828 + }, + { + "budget": 0.22, + "metric": 0.12386236536647144, + "metric_std": 0.0036958795530770776, + "actual_fraction": 0.19653794940079894, + "actual_fraction_std": 0.01116197838045171 + }, + { + "budget": 0.23, + "metric": 0.1258550175676895, + "metric_std": 0.0028996965542180428, + "actual_fraction": 0.20792276964047937, + "actual_fraction_std": 0.010358694800706707 + }, + { + "budget": 0.24, + "metric": 0.12648649804405415, + "metric_std": 0.002944595822083853, + "actual_fraction": 0.2182423435419441, + "actual_fraction_std": 0.010443926678978492 + }, + { + "budget": 0.25, + "metric": 0.12731271767567526, + "metric_std": 0.003163144094995851, + "actual_fraction": 0.22949400798934755, + "actual_fraction_std": 0.009929931952197277 + }, + { + "budget": 0.26, + "metric": 0.1286868421457982, + "metric_std": 0.003734559368381085, + "actual_fraction": 0.2396804260985353, + "actual_fraction_std": 0.007771383175718207 + }, + { + "budget": 0.27, + "metric": 0.12981802598363276, + "metric_std": 0.00401889481405094, + "actual_fraction": 0.2521970705725699, + "actual_fraction_std": 0.0080156579645724 + }, + { + "budget": 0.28, + "metric": 0.13092718581466845, + "metric_std": 0.004224734684663164, + "actual_fraction": 0.2629161118508655, + "actual_fraction_std": 0.0114239793445657 + }, + { + "budget": 0.29, + "metric": 0.1319198257852815, + "metric_std": 0.0036624806199764283, + "actual_fraction": 0.2734354194407457, + "actual_fraction_std": 0.011604911173466024 + }, + { + "budget": 0.3, + "metric": 0.13305819620286258, + "metric_std": 0.003635212203351078, + "actual_fraction": 0.2827563249001332, + "actual_fraction_std": 0.01293694354238888 + }, + { + "budget": 0.31, + "metric": 0.1346791690962181, + "metric_std": 0.0024059986512815416, + "actual_fraction": 0.29227696404793607, + "actual_fraction_std": 0.013605775843998535 + }, + { + "budget": 0.32, + "metric": 0.13547249313440743, + "metric_std": 0.0028518958560498054, + "actual_fraction": 0.3040612516644474, + "actual_fraction_std": 0.015400591359523328 + }, + { + "budget": 0.33, + "metric": 0.13647908099199996, + "metric_std": 0.002936835829923262, + "actual_fraction": 0.31398135818908124, + "actual_fraction_std": 0.0175745469527525 + }, + { + "budget": 0.34, + "metric": 0.1385820762806131, + "metric_std": 0.0026384877362658223, + "actual_fraction": 0.3278961384820239, + "actual_fraction_std": 0.01419569711066604 + }, + { + "budget": 0.35, + "metric": 0.13964160303820758, + "metric_std": 0.0027147702625854434, + "actual_fraction": 0.3386151797603196, + "actual_fraction_std": 0.014592942348740945 + }, + { + "budget": 0.36, + "metric": 0.14125097544502843, + "metric_std": 0.002050683038648574, + "actual_fraction": 0.3503328894806924, + "actual_fraction_std": 0.01287899453502313 + }, + { + "budget": 0.37, + "metric": 0.14244280975648127, + "metric_std": 0.0016378108910034828, + "actual_fraction": 0.3609853528628495, + "actual_fraction_std": 0.010852401295326204 + }, + { + "budget": 0.38, + "metric": 0.1441702691578275, + "metric_std": 0.0011836395595682125, + "actual_fraction": 0.37383488681757654, + "actual_fraction_std": 0.007410063012500686 + }, + { + "budget": 0.39, + "metric": 0.14542873325586433, + "metric_std": 0.002661612280316939, + "actual_fraction": 0.38455392809587213, + "actual_fraction_std": 0.008155452586746395 + }, + { + "budget": 0.4, + "metric": 0.14775274525695908, + "metric_std": 0.0028612118395689693, + "actual_fraction": 0.3983355525965379, + "actual_fraction_std": 0.005835544051165681 + }, + { + "budget": 0.41, + "metric": 0.1496557721230332, + "metric_std": 0.003258822480978681, + "actual_fraction": 0.4078561917443409, + "actual_fraction_std": 0.008083801403296223 + }, + { + "budget": 0.42, + "metric": 0.15080659580013825, + "metric_std": 0.0030558027228243475, + "actual_fraction": 0.4157123834886818, + "actual_fraction_std": 0.007012046477313704 + }, + { + "budget": 0.43, + "metric": 0.15219378739573583, + "metric_std": 0.003904754410616575, + "actual_fraction": 0.4266311584553928, + "actual_fraction_std": 0.009057408311713512 + }, + { + "budget": 0.44, + "metric": 0.1536921652911502, + "metric_std": 0.004290214892468822, + "actual_fraction": 0.43928095872170436, + "actual_fraction_std": 0.008360763023142807 + }, + { + "budget": 0.45, + "metric": 0.15618172790029944, + "metric_std": 0.003953646455304727, + "actual_fraction": 0.4515978695073236, + "actual_fraction_std": 0.006889284327281776 + }, + { + "budget": 0.46, + "metric": 0.15814698022060192, + "metric_std": 0.003545018305807059, + "actual_fraction": 0.46591211717709724, + "actual_fraction_std": 0.00737483568897364 + }, + { + "budget": 0.47, + "metric": 0.15812896773885335, + "metric_std": 0.003603532645056713, + "actual_fraction": 0.47383488681757663, + "actual_fraction_std": 0.006525822604276127 + }, + { + "budget": 0.48, + "metric": 0.1587683626651843, + "metric_std": 0.0047122628562237204, + "actual_fraction": 0.4934087882822903, + "actual_fraction_std": 0.00981431358803771 + }, + { + "budget": 0.49, + "metric": 0.15888393644403737, + "metric_std": 0.005568259617430122, + "actual_fraction": 0.5031957390146471, + "actual_fraction_std": 0.010494205583857077 + }, + { + "budget": 0.5, + "metric": 0.15947067430604075, + "metric_std": 0.007092198466558058, + "actual_fraction": 0.5108521970705726, + "actual_fraction_std": 0.01113215472976984 + }, + { + "budget": 0.51, + "metric": 0.16024476511198502, + "metric_std": 0.00798224072723504, + "actual_fraction": 0.5192410119840213, + "actual_fraction_std": 0.010940877567425145 + }, + { + "budget": 0.52, + "metric": 0.16146959244451872, + "metric_std": 0.007617021566452076, + "actual_fraction": 0.5290945406125166, + "actual_fraction_std": 0.010245752019286784 + }, + { + "budget": 0.53, + "metric": 0.162677945725956, + "metric_std": 0.007283389164476848, + "actual_fraction": 0.540612516644474, + "actual_fraction_std": 0.013317659624432418 + }, + { + "budget": 0.54, + "metric": 0.1647880031905607, + "metric_std": 0.007614536423399123, + "actual_fraction": 0.5503328894806925, + "actual_fraction_std": 0.013216173557342361 + }, + { + "budget": 0.55, + "metric": 0.16557254141228445, + "metric_std": 0.007800925909522179, + "actual_fraction": 0.5591877496671105, + "actual_fraction_std": 0.010304529216320127 + }, + { + "budget": 0.56, + "metric": 0.16727547682370997, + "metric_std": 0.008570308861989276, + "actual_fraction": 0.5691744340878828, + "actual_fraction_std": 0.012431984847330001 + }, + { + "budget": 0.57, + "metric": 0.16990323860036627, + "metric_std": 0.00969724885441064, + "actual_fraction": 0.5799600532623168, + "actual_fraction_std": 0.012733620953193346 + }, + { + "budget": 0.58, + "metric": 0.17152925645448444, + "metric_std": 0.010633629752712261, + "actual_fraction": 0.5876165113182423, + "actual_fraction_std": 0.012525229370263475 + }, + { + "budget": 0.59, + "metric": 0.17286076359605607, + "metric_std": 0.009567146104370459, + "actual_fraction": 0.5972037283621837, + "actual_fraction_std": 0.011986332814506541 + }, + { + "budget": 0.6, + "metric": 0.17455520932540983, + "metric_std": 0.009205748075428368, + "actual_fraction": 0.6060585885486018, + "actual_fraction_std": 0.014181248162776659 + }, + { + "budget": 0.61, + "metric": 0.1760110318333698, + "metric_std": 0.00910784989748052, + "actual_fraction": 0.6177762982689747, + "actual_fraction_std": 0.01522837486012527 + }, + { + "budget": 0.62, + "metric": 0.177362341436445, + "metric_std": 0.008609944259726863, + "actual_fraction": 0.6268308921438083, + "actual_fraction_std": 0.014435663187114896 + }, + { + "budget": 0.63, + "metric": 0.1787642962983092, + "metric_std": 0.00737809157182933, + "actual_fraction": 0.6383488681757656, + "actual_fraction_std": 0.01310714020956454 + }, + { + "budget": 0.64, + "metric": 0.18127172284714566, + "metric_std": 0.006198775004282207, + "actual_fraction": 0.6470039946737683, + "actual_fraction_std": 0.015076990517254125 + }, + { + "budget": 0.65, + "metric": 0.1830783552284434, + "metric_std": 0.004859931149308894, + "actual_fraction": 0.6577896138482024, + "actual_fraction_std": 0.015334610927138503 + }, + { + "budget": 0.66, + "metric": 0.1866378755580354, + "metric_std": 0.0031335043290851503, + "actual_fraction": 0.6670439414114514, + "actual_fraction_std": 0.013990074006429359 + }, + { + "budget": 0.67, + "metric": 0.18977055739043908, + "metric_std": 0.003725569318489589, + "actual_fraction": 0.6795605858854861, + "actual_fraction_std": 0.008264111261169885 + }, + { + "budget": 0.68, + "metric": 0.19253322760584102, + "metric_std": 0.003626061778464404, + "actual_fraction": 0.6898135818908122, + "actual_fraction_std": 0.008918083714647387 + }, + { + "budget": 0.69, + "metric": 0.19549519730467316, + "metric_std": 0.003479274349592312, + "actual_fraction": 0.6993342210386151, + "actual_fraction_std": 0.007708386399753608 + }, + { + "budget": 0.7, + "metric": 0.1985519805216049, + "metric_std": 0.003238815293563175, + "actual_fraction": 0.710652463382157, + "actual_fraction_std": 0.007071061543196652 + }, + { + "budget": 0.71, + "metric": 0.20299858794209463, + "metric_std": 0.00341586311489079, + "actual_fraction": 0.7230359520639148, + "actual_fraction_std": 0.005372840343398456 + }, + { + "budget": 0.72, + "metric": 0.2064620316828066, + "metric_std": 0.004360157137197088, + "actual_fraction": 0.7356191744340879, + "actual_fraction_std": 0.004468661569786908 + }, + { + "budget": 0.73, + "metric": 0.2090757439544892, + "metric_std": 0.004691816242000687, + "actual_fraction": 0.7421438082556591, + "actual_fraction_std": 0.004426303632115751 + }, + { + "budget": 0.74, + "metric": 0.21422402820383005, + "metric_std": 0.003092295249881844, + "actual_fraction": 0.7511318242343542, + "actual_fraction_std": 0.0018919137467886679 + }, + { + "budget": 0.75, + "metric": 0.21895361469247812, + "metric_std": 0.002231398738690623, + "actual_fraction": 0.7613848202396805, + "actual_fraction_std": 0.0027990443421542013 + }, + { + "budget": 0.76, + "metric": 0.22190165516928112, + "metric_std": 0.0032881256388441215, + "actual_fraction": 0.7697736351531292, + "actual_fraction_std": 0.003726281043663302 + }, + { + "budget": 0.77, + "metric": 0.2269148500296052, + "metric_std": 0.004147455991506267, + "actual_fraction": 0.7802929427430094, + "actual_fraction_std": 0.005130171440553402 + }, + { + "budget": 0.78, + "metric": 0.23140759097409985, + "metric_std": 0.0038873549529773893, + "actual_fraction": 0.79254327563249, + "actual_fraction_std": 0.0026568668361165184 + }, + { + "budget": 0.79, + "metric": 0.23020051115987986, + "metric_std": 0.002892715969557764, + "actual_fraction": 0.800133155792277, + "actual_fraction_std": 0.004101436173438445 + }, + { + "budget": 0.8, + "metric": 0.23074750969582683, + "metric_std": 0.003314855242484718, + "actual_fraction": 0.8085885486018641, + "actual_fraction_std": 0.003227483260596752 + }, + { + "budget": 0.81, + "metric": 0.23111738200368145, + "metric_std": 0.0032235443095482686, + "actual_fraction": 0.8171105193075899, + "actual_fraction_std": 0.004318591540034558 + }, + { + "budget": 0.82, + "metric": 0.23071553559934005, + "metric_std": 0.0030379553042898353, + "actual_fraction": 0.8249667110519308, + "actual_fraction_std": 0.005421092672655737 + }, + { + "budget": 0.83, + "metric": 0.23048519516038093, + "metric_std": 0.0032990764834630202, + "actual_fraction": 0.8358189081225034, + "actual_fraction_std": 0.0046568846908424935 + }, + { + "budget": 0.84, + "metric": 0.229355888612426, + "metric_std": 0.0013166973359941868, + "actual_fraction": 0.8492010652463382, + "actual_fraction_std": 0.004533900956017381 + }, + { + "budget": 0.85, + "metric": 0.22912127859231105, + "metric_std": 0.001498893895445914, + "actual_fraction": 0.8569241011984022, + "actual_fraction_std": 0.0025613028044835346 + }, + { + "budget": 0.86, + "metric": 0.22821252351440097, + "metric_std": 0.0008323271362470637, + "actual_fraction": 0.8666444740346204, + "actual_fraction_std": 0.0028982421917016074 + }, + { + "budget": 0.87, + "metric": 0.2276110739718097, + "metric_std": 0.0002511993793791489, + "actual_fraction": 0.8739014647137152, + "actual_fraction_std": 0.004511849969528592 + }, + { + "budget": 0.88, + "metric": 0.22707420277450813, + "metric_std": 0.0005547476766542725, + "actual_fraction": 0.8836218375499334, + "actual_fraction_std": 0.004728905687637744 + }, + { + "budget": 0.89, + "metric": 0.2266544697538749, + "metric_std": 0.00039009788818517466, + "actual_fraction": 0.8921438082556591, + "actual_fraction_std": 0.003585329432180066 + }, + { + "budget": 0.9, + "metric": 0.225764029397514, + "metric_std": 0.0004753666103727167, + "actual_fraction": 0.9057256990679095, + "actual_fraction_std": 0.006512223584330631 + }, + { + "budget": 0.91, + "metric": 0.22565933981785702, + "metric_std": 0.000666160511708541, + "actual_fraction": 0.9135153129161118, + "actual_fraction_std": 0.003948636179016948 + }, + { + "budget": 0.92, + "metric": 0.225401460262901, + "metric_std": 0.00037292418204203196, + "actual_fraction": 0.9240346205059919, + "actual_fraction_std": 0.005001103599654101 + }, + { + "budget": 0.93, + "metric": 0.22536517552633634, + "metric_std": 0.0005757677539477428, + "actual_fraction": 0.9330892143808256, + "actual_fraction_std": 0.007627456216637527 + }, + { + "budget": 0.94, + "metric": 0.22506315474151303, + "metric_std": 0.0006750079341938803, + "actual_fraction": 0.9430758988015977, + "actual_fraction_std": 0.005205224957094509 + }, + { + "budget": 0.95, + "metric": 0.2246540735559644, + "metric_std": 0.0006485039413728499, + "actual_fraction": 0.9542609853528627, + "actual_fraction_std": 0.005792659582997657 + }, + { + "budget": 0.96, + "metric": 0.22449186763410825, + "metric_std": 0.0006308581405572811, + "actual_fraction": 0.9647802929427429, + "actual_fraction_std": 0.005255016580737945 + }, + { + "budget": 0.97, + "metric": 0.2241007721558991, + "metric_std": 0.00043383590851483583, + "actual_fraction": 0.9723035952063915, + "actual_fraction_std": 0.0033899200048204793 + }, + { + "budget": 0.98, + "metric": 0.22382721519706097, + "metric_std": 0.000408649138887247, + "actual_fraction": 0.979960053262317, + "actual_fraction_std": 0.0042461354917226034 + }, + { + "budget": 0.99, + "metric": 0.2237912729695244, + "metric_std": 0.0003763404875270694, + "actual_fraction": 0.9905459387483356, + "actual_fraction_std": 0.0009875097852324387 + }, + { + "budget": 1.0, + "metric": 0.22361115727328068, + "metric_std": 3.1031676915590914e-17, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 1391, + "rescuable": 786, + "harmful": 475, + "both_wrong": 152, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 6060, + "rescuable": 3314, + "harmful": 1945, + "both_wrong": 697, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.07641657905799261, + "test": 0.07887397871043929 + }, + "large": { + "validation": 0.20666016863629594, + "test": 0.22361115727328065 + }, + "learned_minus_confidence_test": 0.012981731007794955, + "learned_minus_small_test": 0.15020288102149665, + "learned_minus_large_test": 0.005465702458655269, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_rtdetr_x_fp32_MiniCPM" + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "Qwen3.8" + ], + "expert_display": "RT-DETR-X", + "run": "five_seed_rtdetr_x_fp32_Qwen3.8_four_difference", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/rtdetr_x_fp32/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 99, + "validation_area_0_25": 0.10053453664183931, + "validation_point": { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.24862274646934207, + "rescued": 805, + "harmed": 184, + "missed_rescue": 10, + "avoided_harm": 1, + "selected_both_correct": 1662, + "selected_both_wrong": 121 + }, + "test_point": { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9860186418109188, + "metric": 0.26818431283513144, + "rescued": 3404, + "harmed": 671, + "missed_rescue": 57, + "avoided_harm": 5, + "selected_both_correct": 7228, + "selected_both_wrong": 545 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 0, + "missed_rescue": 815, + "avoided_harm": 185, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.021043073385953903, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 1, + "missed_rescue": 815, + "avoided_harm": 184, + "selected_both_correct": 27, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.022327054291963577, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 2, + "missed_rescue": 815, + "avoided_harm": 183, + "selected_both_correct": 54, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.024909960106015205, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 5, + "missed_rescue": 815, + "avoided_harm": 180, + "selected_both_correct": 79, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -0.027814723551273346, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.07641657905799261, + "rescued": 0, + "harmed": 5, + "missed_rescue": 815, + "avoided_harm": 180, + "selected_both_correct": 107, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -0.028914429247379303, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.10273236853167682, + "rescued": 1, + "harmed": 5, + "missed_rescue": 814, + "avoided_harm": 180, + "selected_both_correct": 134, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -0.030170045793056488, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.10349184930294494, + "rescued": 2, + "harmed": 6, + "missed_rescue": 813, + "avoided_harm": 179, + "selected_both_correct": 160, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -0.03123854100704193, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.10193024779448791, + "rescued": 2, + "harmed": 10, + "missed_rescue": 813, + "avoided_harm": 175, + "selected_both_correct": 184, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -0.03410686179995537, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.10070576146899149, + "rescued": 2, + "harmed": 12, + "missed_rescue": 813, + "avoided_harm": 173, + "selected_both_correct": 210, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -0.03476816788315773, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.09863536809425029, + "rescued": 2, + "harmed": 14, + "missed_rescue": 813, + "avoided_harm": 171, + "selected_both_correct": 236, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -0.0365506075322628, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.09772957099280101, + "rescued": 2, + "harmed": 15, + "missed_rescue": 813, + "avoided_harm": 170, + "selected_both_correct": 263, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -0.03818900138139725, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.09678681181213102, + "rescued": 2, + "harmed": 17, + "missed_rescue": 813, + "avoided_harm": 168, + "selected_both_correct": 289, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -0.039673544466495514, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.0951962932074597, + "rescued": 2, + "harmed": 20, + "missed_rescue": 813, + "avoided_harm": 165, + "selected_both_correct": 314, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -0.04085410386323929, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.09473838918283363, + "rescued": 2, + "harmed": 25, + "missed_rescue": 813, + "avoided_harm": 160, + "selected_both_correct": 337, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -0.04204101860523224, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.09404328215803626, + "rescued": 2, + "harmed": 27, + "missed_rescue": 813, + "avoided_harm": 158, + "selected_both_correct": 363, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -0.04278091713786125, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.10930560071560966, + "rescued": 3, + "harmed": 30, + "missed_rescue": 812, + "avoided_harm": 155, + "selected_both_correct": 386, + "selected_both_wrong": 1 + }, + { + "budget": 0.16, + "threshold": -0.04322759434580803, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.10819687589305238, + "rescued": 3, + "harmed": 32, + "missed_rescue": 812, + "avoided_harm": 153, + "selected_both_correct": 412, + "selected_both_wrong": 1 + }, + { + "budget": 0.17, + "threshold": -0.04394488036632538, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.10718881137692335, + "rescued": 3, + "harmed": 33, + "missed_rescue": 812, + "avoided_harm": 152, + "selected_both_correct": 439, + "selected_both_wrong": 1 + }, + { + "budget": 0.18, + "threshold": -0.04530126228928566, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.106004048568, + "rescued": 3, + "harmed": 37, + "missed_rescue": 812, + "avoided_harm": 148, + "selected_both_correct": 463, + "selected_both_wrong": 1 + }, + { + "budget": 0.19, + "threshold": -0.04661082476377487, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.106004048568, + "rescued": 3, + "harmed": 37, + "missed_rescue": 812, + "avoided_harm": 148, + "selected_both_correct": 491, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.04730507358908653, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.10416681691465626, + "rescued": 3, + "harmed": 40, + "missed_rescue": 812, + "avoided_harm": 145, + "selected_both_correct": 516, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.048246145248413086, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.10337316612100547, + "rescued": 3, + "harmed": 41, + "missed_rescue": 812, + "avoided_harm": 144, + "selected_both_correct": 543, + "selected_both_wrong": 1 + }, + { + "budget": 0.22, + "threshold": -0.04900278151035309, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.10251715413933904, + "rescued": 3, + "harmed": 43, + "missed_rescue": 812, + "avoided_harm": 142, + "selected_both_correct": 569, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.050042688846588135, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.10180591658599622, + "rescued": 3, + "harmed": 44, + "missed_rescue": 812, + "avoided_harm": 141, + "selected_both_correct": 595, + "selected_both_wrong": 2 + }, + { + "budget": 0.24, + "threshold": -0.050901688635349274, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.10179370893656609, + "rescued": 9, + "harmed": 48, + "missed_rescue": 806, + "avoided_harm": 137, + "selected_both_correct": 612, + "selected_both_wrong": 3 + }, + { + "budget": 0.25, + "threshold": -0.05161922052502632, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.10106007305091304, + "rescued": 16, + "harmed": 50, + "missed_rescue": 799, + "avoided_harm": 135, + "selected_both_correct": 631, + "selected_both_wrong": 3 + }, + { + "budget": 0.26, + "threshold": -0.05245445668697357, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.10034763622704679, + "rescued": 22, + "harmed": 52, + "missed_rescue": 793, + "avoided_harm": 133, + "selected_both_correct": 650, + "selected_both_wrong": 4 + }, + { + "budget": 0.27, + "threshold": -0.053079016506671906, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.10038369304556356, + "rescued": 30, + "harmed": 53, + "missed_rescue": 785, + "avoided_harm": 132, + "selected_both_correct": 669, + "selected_both_wrong": 4 + }, + { + "budget": 0.28, + "threshold": -0.053604189306497574, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.10087055535678022, + "rescued": 38, + "harmed": 53, + "missed_rescue": 777, + "avoided_harm": 132, + "selected_both_correct": 686, + "selected_both_wrong": 7 + }, + { + "budget": 0.29, + "threshold": -0.05505518987774849, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.10109647230706402, + "rescued": 48, + "harmed": 54, + "missed_rescue": 767, + "avoided_harm": 131, + "selected_both_correct": 703, + "selected_both_wrong": 7 + }, + { + "budget": 0.3, + "threshold": -0.05605518817901611, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.10072202785599546, + "rescued": 54, + "harmed": 57, + "missed_rescue": 761, + "avoided_harm": 128, + "selected_both_correct": 721, + "selected_both_wrong": 8 + }, + { + "budget": 0.31, + "threshold": -0.056986160576343536, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.10144063504785386, + "rescued": 62, + "harmed": 60, + "missed_rescue": 753, + "avoided_harm": 125, + "selected_both_correct": 737, + "selected_both_wrong": 9 + }, + { + "budget": 0.32, + "threshold": -0.0582931749522686, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.1016168805568569, + "rescued": 70, + "harmed": 61, + "missed_rescue": 745, + "avoided_harm": 124, + "selected_both_correct": 756, + "selected_both_wrong": 9 + }, + { + "budget": 0.33, + "threshold": -0.0596650205552578, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.10207649449543332, + "rescued": 76, + "harmed": 61, + "missed_rescue": 739, + "avoided_harm": 124, + "selected_both_correct": 776, + "selected_both_wrong": 11 + }, + { + "budget": 0.34, + "threshold": -0.060549296438694, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.10147122890331242, + "rescued": 87, + "harmed": 65, + "missed_rescue": 728, + "avoided_harm": 120, + "selected_both_correct": 788, + "selected_both_wrong": 12 + }, + { + "budget": 0.35, + "threshold": -0.06178095191717148, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.11031878913558303, + "rescued": 97, + "harmed": 67, + "missed_rescue": 718, + "avoided_harm": 118, + "selected_both_correct": 803, + "selected_both_wrong": 13 + }, + { + "budget": 0.36, + "threshold": -0.06328777223825455, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.1108012937861455, + "rescued": 109, + "harmed": 68, + "missed_rescue": 706, + "avoided_harm": 117, + "selected_both_correct": 818, + "selected_both_wrong": 13 + }, + { + "budget": 0.37, + "threshold": -0.0647548958659172, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.11116507183385692, + "rescued": 119, + "harmed": 69, + "missed_rescue": 696, + "avoided_harm": 116, + "selected_both_correct": 834, + "selected_both_wrong": 14 + }, + { + "budget": 0.38, + "threshold": -0.06637481600046158, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.11160958901594166, + "rescued": 128, + "harmed": 70, + "missed_rescue": 687, + "avoided_harm": 115, + "selected_both_correct": 851, + "selected_both_wrong": 15 + }, + { + "budget": 0.39, + "threshold": -0.06774259358644485, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.11234765239703698, + "rescued": 137, + "harmed": 70, + "missed_rescue": 678, + "avoided_harm": 115, + "selected_both_correct": 867, + "selected_both_wrong": 18 + }, + { + "budget": 0.4, + "threshold": -0.06843380630016327, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.11018425519727987, + "rescued": 146, + "harmed": 72, + "missed_rescue": 669, + "avoided_harm": 113, + "selected_both_correct": 883, + "selected_both_wrong": 19 + }, + { + "budget": 0.41, + "threshold": -0.0694691464304924, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.11067980295566503, + "rescued": 154, + "harmed": 76, + "missed_rescue": 661, + "avoided_harm": 109, + "selected_both_correct": 897, + "selected_both_wrong": 21 + }, + { + "budget": 0.42, + "threshold": -0.07112707942724228, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.11116873445502742, + "rescued": 166, + "harmed": 78, + "missed_rescue": 649, + "avoided_harm": 107, + "selected_both_correct": 911, + "selected_both_wrong": 21 + }, + { + "budget": 0.43, + "threshold": -0.07192998379468918, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.11166452557368921, + "rescued": 175, + "harmed": 82, + "missed_rescue": 640, + "avoided_harm": 103, + "selected_both_correct": 925, + "selected_both_wrong": 22 + }, + { + "budget": 0.44, + "threshold": -0.07266208529472351, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.11224389412088417, + "rescued": 185, + "harmed": 84, + "missed_rescue": 630, + "avoided_harm": 101, + "selected_both_correct": 940, + "selected_both_wrong": 23 + }, + { + "budget": 0.45, + "threshold": -0.0744253620505333, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.11275924414114559, + "rescued": 194, + "harmed": 85, + "missed_rescue": 621, + "avoided_harm": 100, + "selected_both_correct": 955, + "selected_both_wrong": 26 + }, + { + "budget": 0.46, + "threshold": -0.07532940804958344, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.11288099124724625, + "rescued": 203, + "harmed": 88, + "missed_rescue": 612, + "avoided_harm": 97, + "selected_both_correct": 970, + "selected_both_wrong": 27 + }, + { + "budget": 0.47, + "threshold": -0.0770750343799591, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.11312993997763046, + "rescued": 218, + "harmed": 91, + "missed_rescue": 597, + "avoided_harm": 94, + "selected_both_correct": 979, + "selected_both_wrong": 28 + }, + { + "budget": 0.48, + "threshold": -0.07818037271499634, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.11421587032830263, + "rescued": 227, + "harmed": 91, + "missed_rescue": 588, + "avoided_harm": 94, + "selected_both_correct": 997, + "selected_both_wrong": 29 + }, + { + "budget": 0.49, + "threshold": -0.07998209446668625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.11231395556672395, + "rescued": 233, + "harmed": 95, + "missed_rescue": 582, + "avoided_harm": 90, + "selected_both_correct": 1014, + "selected_both_wrong": 30 + }, + { + "budget": 0.5, + "threshold": -0.08124949783086777, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.11317276185775925, + "rescued": 242, + "harmed": 95, + "missed_rescue": 573, + "avoided_harm": 90, + "selected_both_correct": 1032, + "selected_both_wrong": 31 + }, + { + "budget": 0.51, + "threshold": -0.08398465067148209, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.11397997643625071, + "rescued": 255, + "harmed": 98, + "missed_rescue": 560, + "avoided_harm": 87, + "selected_both_correct": 1044, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.08537434041500092, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.12321191882582982, + "rescued": 266, + "harmed": 101, + "missed_rescue": 549, + "avoided_harm": 84, + "selected_both_correct": 1057, + "selected_both_wrong": 32 + }, + { + "budget": 0.53, + "threshold": -0.08664409816265106, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.12377158600708722, + "rescued": 276, + "harmed": 103, + "missed_rescue": 539, + "avoided_harm": 82, + "selected_both_correct": 1071, + "selected_both_wrong": 34 + }, + { + "budget": 0.54, + "threshold": -0.08777308464050293, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.12471535828861256, + "rescued": 288, + "harmed": 105, + "missed_rescue": 527, + "avoided_harm": 80, + "selected_both_correct": 1084, + "selected_both_wrong": 35 + }, + { + "budget": 0.55, + "threshold": -0.08980843424797058, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.12627445993419065, + "rescued": 301, + "harmed": 106, + "missed_rescue": 514, + "avoided_harm": 79, + "selected_both_correct": 1098, + "selected_both_wrong": 35 + }, + { + "budget": 0.56, + "threshold": -0.0906188115477562, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.1336597715556373, + "rescued": 314, + "harmed": 107, + "missed_rescue": 501, + "avoided_harm": 78, + "selected_both_correct": 1111, + "selected_both_wrong": 36 + }, + { + "budget": 0.57, + "threshold": -0.09160211682319641, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.13507916719864918, + "rescued": 325, + "harmed": 108, + "missed_rescue": 490, + "avoided_harm": 77, + "selected_both_correct": 1126, + "selected_both_wrong": 37 + }, + { + "budget": 0.58, + "threshold": -0.09283093363046646, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.13601889857898128, + "rescued": 336, + "harmed": 110, + "missed_rescue": 479, + "avoided_harm": 75, + "selected_both_correct": 1140, + "selected_both_wrong": 38 + }, + { + "budget": 0.59, + "threshold": -0.0941123366355896, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.13626938313219553, + "rescued": 346, + "harmed": 114, + "missed_rescue": 469, + "avoided_harm": 71, + "selected_both_correct": 1153, + "selected_both_wrong": 39 + }, + { + "budget": 0.6, + "threshold": -0.09614869952201843, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.13736913676837675, + "rescued": 359, + "harmed": 116, + "missed_rescue": 456, + "avoided_harm": 69, + "selected_both_correct": 1165, + "selected_both_wrong": 40 + }, + { + "budget": 0.61, + "threshold": -0.09772664308547974, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.13874937058804088, + "rescued": 371, + "harmed": 117, + "missed_rescue": 444, + "avoided_harm": 68, + "selected_both_correct": 1177, + "selected_both_wrong": 43 + }, + { + "budget": 0.62, + "threshold": -0.09911532700061798, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.1401946427657629, + "rescued": 383, + "harmed": 118, + "missed_rescue": 432, + "avoided_harm": 67, + "selected_both_correct": 1190, + "selected_both_wrong": 45 + }, + { + "budget": 0.63, + "threshold": -0.10007988661527634, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.14163850172491793, + "rescued": 395, + "harmed": 119, + "missed_rescue": 420, + "avoided_harm": 66, + "selected_both_correct": 1204, + "selected_both_wrong": 46 + }, + { + "budget": 0.64, + "threshold": -0.10177135467529297, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.14631539139000474, + "rescued": 409, + "harmed": 124, + "missed_rescue": 406, + "avoided_harm": 61, + "selected_both_correct": 1210, + "selected_both_wrong": 49 + }, + { + "budget": 0.65, + "threshold": -0.10395076125860214, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.14751840006124584, + "rescued": 418, + "harmed": 126, + "missed_rescue": 397, + "avoided_harm": 59, + "selected_both_correct": 1226, + "selected_both_wrong": 50 + }, + { + "budget": 0.66, + "threshold": -0.10524322092533112, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.14849238248708915, + "rescued": 430, + "harmed": 129, + "missed_rescue": 385, + "avoided_harm": 56, + "selected_both_correct": 1236, + "selected_both_wrong": 53 + }, + { + "budget": 0.67, + "threshold": -0.10689426958560944, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.15045288951822525, + "rescued": 444, + "harmed": 130, + "missed_rescue": 371, + "avoided_harm": 55, + "selected_both_correct": 1249, + "selected_both_wrong": 53 + }, + { + "budget": 0.68, + "threshold": -0.11014843732118607, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.15143717933254464, + "rescued": 455, + "harmed": 132, + "missed_rescue": 360, + "avoided_harm": 53, + "selected_both_correct": 1261, + "selected_both_wrong": 56 + }, + { + "budget": 0.69, + "threshold": -0.11117545515298843, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.158399005658965, + "rescued": 469, + "harmed": 136, + "missed_rescue": 346, + "avoided_harm": 49, + "selected_both_correct": 1269, + "selected_both_wrong": 58 + }, + { + "budget": 0.7, + "threshold": -0.11230025440454483, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.15918455527939407, + "rescued": 480, + "harmed": 137, + "missed_rescue": 335, + "avoided_harm": 48, + "selected_both_correct": 1283, + "selected_both_wrong": 60 + }, + { + "budget": 0.71, + "threshold": -0.11423606425523758, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.16079826154850815, + "rescued": 491, + "harmed": 138, + "missed_rescue": 324, + "avoided_harm": 47, + "selected_both_correct": 1297, + "selected_both_wrong": 62 + }, + { + "budget": 0.72, + "threshold": -0.11647249013185501, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.16232381124917994, + "rescued": 506, + "harmed": 142, + "missed_rescue": 309, + "avoided_harm": 43, + "selected_both_correct": 1305, + "selected_both_wrong": 63 + }, + { + "budget": 0.73, + "threshold": -0.11962417513132095, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.16281139401042977, + "rescued": 516, + "harmed": 145, + "missed_rescue": 299, + "avoided_harm": 40, + "selected_both_correct": 1315, + "selected_both_wrong": 68 + }, + { + "budget": 0.74, + "threshold": -0.12126146256923676, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.16441172693350847, + "rescued": 527, + "harmed": 146, + "missed_rescue": 288, + "avoided_harm": 39, + "selected_both_correct": 1328, + "selected_both_wrong": 71 + }, + { + "budget": 0.75, + "threshold": -0.1230982318520546, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.16630001788027923, + "rescued": 538, + "harmed": 146, + "missed_rescue": 277, + "avoided_harm": 39, + "selected_both_correct": 1342, + "selected_both_wrong": 74 + }, + { + "budget": 0.76, + "threshold": -0.12517565488815308, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.16644279851003654, + "rescued": 549, + "harmed": 149, + "missed_rescue": 266, + "avoided_harm": 36, + "selected_both_correct": 1354, + "selected_both_wrong": 76 + }, + { + "budget": 0.77, + "threshold": -0.12628257274627686, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.16746586450739254, + "rescued": 556, + "harmed": 155, + "missed_rescue": 259, + "avoided_harm": 30, + "selected_both_correct": 1365, + "selected_both_wrong": 80 + }, + { + "budget": 0.78, + "threshold": -0.12975578010082245, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.1703501656136907, + "rescued": 568, + "harmed": 156, + "missed_rescue": 247, + "avoided_harm": 29, + "selected_both_correct": 1379, + "selected_both_wrong": 81 + }, + { + "budget": 0.79, + "threshold": -0.13229140639305115, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.17215569220087368, + "rescued": 577, + "harmed": 157, + "missed_rescue": 238, + "avoided_harm": 28, + "selected_both_correct": 1392, + "selected_both_wrong": 86 + }, + { + "budget": 0.8, + "threshold": -0.1373685747385025, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.1747173129779689, + "rescued": 591, + "harmed": 160, + "missed_rescue": 224, + "avoided_harm": 25, + "selected_both_correct": 1402, + "selected_both_wrong": 87 + }, + { + "budget": 0.81, + "threshold": -0.1407174915075302, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.18313694998955, + "rescued": 609, + "harmed": 160, + "missed_rescue": 206, + "avoided_harm": 25, + "selected_both_correct": 1411, + "selected_both_wrong": 88 + }, + { + "budget": 0.82, + "threshold": -0.1433146893978119, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.18621937310674674, + "rescued": 623, + "harmed": 162, + "missed_rescue": 192, + "avoided_harm": 23, + "selected_both_correct": 1421, + "selected_both_wrong": 90 + }, + { + "budget": 0.83, + "threshold": -0.1457100659608841, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.1886261591198746, + "rescued": 632, + "harmed": 163, + "missed_rescue": 183, + "avoided_harm": 22, + "selected_both_correct": 1439, + "selected_both_wrong": 90 + }, + { + "budget": 0.84, + "threshold": -0.148710235953331, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.19060328592765763, + "rescued": 642, + "harmed": 167, + "missed_rescue": 173, + "avoided_harm": 18, + "selected_both_correct": 1451, + "selected_both_wrong": 92 + }, + { + "budget": 0.85, + "threshold": -0.15291842818260193, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.1979618629476584, + "rescued": 658, + "harmed": 169, + "missed_rescue": 157, + "avoided_harm": 16, + "selected_both_correct": 1460, + "selected_both_wrong": 93 + }, + { + "budget": 0.86, + "threshold": -0.15889839828014374, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2003153021893066, + "rescued": 667, + "harmed": 170, + "missed_rescue": 148, + "avoided_harm": 15, + "selected_both_correct": 1476, + "selected_both_wrong": 95 + }, + { + "budget": 0.87, + "threshold": -0.16629797220230103, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.20401478964488712, + "rescued": 678, + "harmed": 172, + "missed_rescue": 137, + "avoided_harm": 13, + "selected_both_correct": 1491, + "selected_both_wrong": 95 + }, + { + "budget": 0.88, + "threshold": -0.17295491695404053, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.20519541918721101, + "rescued": 687, + "harmed": 173, + "missed_rescue": 128, + "avoided_harm": 12, + "selected_both_correct": 1507, + "selected_both_wrong": 97 + }, + { + "budget": 0.89, + "threshold": -0.18053314089775085, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.20764101752605946, + "rescued": 695, + "harmed": 175, + "missed_rescue": 120, + "avoided_harm": 10, + "selected_both_correct": 1522, + "selected_both_wrong": 100 + }, + { + "budget": 0.9, + "threshold": -0.19442276656627655, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.2119684936431676, + "rescued": 709, + "harmed": 175, + "missed_rescue": 106, + "avoided_harm": 10, + "selected_both_correct": 1535, + "selected_both_wrong": 101 + }, + { + "budget": 0.91, + "threshold": -0.20900504291057587, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.21605348977753452, + "rescued": 719, + "harmed": 175, + "missed_rescue": 96, + "avoided_harm": 10, + "selected_both_correct": 1551, + "selected_both_wrong": 103 + }, + { + "budget": 0.92, + "threshold": -0.22383525967597961, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2203498969356328, + "rescued": 731, + "harmed": 175, + "missed_rescue": 84, + "avoided_harm": 10, + "selected_both_correct": 1564, + "selected_both_wrong": 106 + }, + { + "budget": 0.93, + "threshold": -0.25952714681625366, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.22073473473473473, + "rescued": 741, + "harmed": 178, + "missed_rescue": 74, + "avoided_harm": 7, + "selected_both_correct": 1577, + "selected_both_wrong": 108 + }, + { + "budget": 0.94, + "threshold": -0.2804492115974426, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.21785099794688836, + "rescued": 751, + "harmed": 179, + "missed_rescue": 64, + "avoided_harm": 6, + "selected_both_correct": 1590, + "selected_both_wrong": 112 + }, + { + "budget": 0.95, + "threshold": -0.2984963357448578, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.22594140701088306, + "rescued": 764, + "harmed": 179, + "missed_rescue": 51, + "avoided_harm": 6, + "selected_both_correct": 1603, + "selected_both_wrong": 114 + }, + { + "budget": 0.96, + "threshold": -0.32223379611968994, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.23296274186615876, + "rescued": 774, + "harmed": 179, + "missed_rescue": 41, + "avoided_harm": 6, + "selected_both_correct": 1619, + "selected_both_wrong": 116 + }, + { + "budget": 0.97, + "threshold": -0.36503666639328003, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.23681731712393103, + "rescued": 784, + "harmed": 180, + "missed_rescue": 31, + "avoided_harm": 5, + "selected_both_correct": 1635, + "selected_both_wrong": 117 + }, + { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.24211204189132882, + "rescued": 796, + "harmed": 182, + "missed_rescue": 19, + "avoided_harm": 3, + "selected_both_correct": 1649, + "selected_both_wrong": 117 + }, + { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.24862274646934207, + "rescued": 805, + "harmed": 184, + "missed_rescue": 10, + "avoided_harm": 1, + "selected_both_correct": 1662, + "selected_both_wrong": 121 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 815, + "harmed": 185, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 1681, + "selected_both_wrong": 123 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.07887397871043929, + "rescued": 0, + "harmed": 0, + "missed_rescue": 3461, + "avoided_harm": 676, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -0.021043073385953903, + "operator": "ge", + "actual_fraction": 0.007989347536617843, + "metric": 0.07887397871043929, + "rescued": 0, + "harmed": 5, + "missed_rescue": 3461, + "avoided_harm": 671, + "selected_both_correct": 91, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -0.022327054291963577, + "operator": "ge", + "actual_fraction": 0.016644474034620507, + "metric": 0.07904050790892239, + "rescued": 1, + "harmed": 8, + "missed_rescue": 3460, + "avoided_harm": 668, + "selected_both_correct": 191, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -0.024909960106015205, + "operator": "ge", + "actual_fraction": 0.0296271637816245, + "metric": 0.07904050790892239, + "rescued": 1, + "harmed": 11, + "missed_rescue": 3460, + "avoided_harm": 665, + "selected_both_correct": 344, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -0.027814723551273346, + "operator": "ge", + "actual_fraction": 0.046937416777629824, + "metric": 0.07894575740459331, + "rescued": 1, + "harmed": 19, + "missed_rescue": 3460, + "avoided_harm": 657, + "selected_both_correct": 544, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -0.028914429247379303, + "operator": "ge", + "actual_fraction": 0.05326231691078562, + "metric": 0.07941709585555166, + "rescued": 2, + "harmed": 22, + "missed_rescue": 3459, + "avoided_harm": 654, + "selected_both_correct": 616, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -0.030170045793056488, + "operator": "ge", + "actual_fraction": 0.05992010652463382, + "metric": 0.07940750674336808, + "rescued": 2, + "harmed": 23, + "missed_rescue": 3459, + "avoided_harm": 653, + "selected_both_correct": 695, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -0.03123854100704193, + "operator": "ge", + "actual_fraction": 0.06724367509986684, + "metric": 0.09221541101182128, + "rescued": 5, + "harmed": 28, + "missed_rescue": 3456, + "avoided_harm": 648, + "selected_both_correct": 775, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -0.03410686179995537, + "operator": "ge", + "actual_fraction": 0.08721704394141146, + "metric": 0.09167716576535306, + "rescued": 6, + "harmed": 39, + "missed_rescue": 3455, + "avoided_harm": 637, + "selected_both_correct": 1003, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -0.03476816788315773, + "operator": "ge", + "actual_fraction": 0.09287616511318242, + "metric": 0.09085537954855229, + "rescued": 6, + "harmed": 46, + "missed_rescue": 3455, + "avoided_harm": 630, + "selected_both_correct": 1064, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -0.0365506075322628, + "operator": "ge", + "actual_fraction": 0.10785619174434088, + "metric": 0.09006366890142659, + "rescued": 6, + "harmed": 56, + "missed_rescue": 3455, + "avoided_harm": 620, + "selected_both_correct": 1233, + "selected_both_wrong": 1 + }, + { + "budget": 0.11, + "threshold": -0.03818900138139725, + "operator": "ge", + "actual_fraction": 0.11784287616511319, + "metric": 0.09042296916775287, + "rescued": 8, + "harmed": 58, + "missed_rescue": 3453, + "avoided_harm": 618, + "selected_both_correct": 1349, + "selected_both_wrong": 1 + }, + { + "budget": 0.12, + "threshold": -0.039673544466495514, + "operator": "ge", + "actual_fraction": 0.12716378162450068, + "metric": 0.08993827377942153, + "rescued": 8, + "harmed": 65, + "missed_rescue": 3453, + "avoided_harm": 611, + "selected_both_correct": 1454, + "selected_both_wrong": 1 + }, + { + "budget": 0.13, + "threshold": -0.04085410386323929, + "operator": "ge", + "actual_fraction": 0.13648468708388814, + "metric": 0.08955740758160691, + "rescued": 8, + "harmed": 73, + "missed_rescue": 3453, + "avoided_harm": 603, + "selected_both_correct": 1558, + "selected_both_wrong": 1 + }, + { + "budget": 0.14, + "threshold": -0.04204101860523224, + "operator": "ge", + "actual_fraction": 0.14613848202396804, + "metric": 0.0893664971618984, + "rescued": 8, + "harmed": 76, + "missed_rescue": 3453, + "avoided_harm": 600, + "selected_both_correct": 1671, + "selected_both_wrong": 1 + }, + { + "budget": 0.15, + "threshold": -0.04278091713786125, + "operator": "ge", + "actual_fraction": 0.1514647137150466, + "metric": 0.08915747670369788, + "rescued": 8, + "harmed": 82, + "missed_rescue": 3453, + "avoided_harm": 594, + "selected_both_correct": 1729, + "selected_both_wrong": 1 + }, + { + "budget": 0.16, + "threshold": -0.04322759434580803, + "operator": "ge", + "actual_fraction": 0.15579227696404793, + "metric": 0.08910079841343506, + "rescued": 8, + "harmed": 85, + "missed_rescue": 3453, + "avoided_harm": 591, + "selected_both_correct": 1778, + "selected_both_wrong": 1 + }, + { + "budget": 0.17, + "threshold": -0.04394488036632538, + "operator": "ge", + "actual_fraction": 0.1617842876165113, + "metric": 0.08889771271713626, + "rescued": 8, + "harmed": 89, + "missed_rescue": 3453, + "avoided_harm": 587, + "selected_both_correct": 1846, + "selected_both_wrong": 1 + }, + { + "budget": 0.18, + "threshold": -0.04530126228928566, + "operator": "ge", + "actual_fraction": 0.17177097203728361, + "metric": 0.08882240770994018, + "rescued": 8, + "harmed": 94, + "missed_rescue": 3453, + "avoided_harm": 582, + "selected_both_correct": 1961, + "selected_both_wrong": 1 + }, + { + "budget": 0.19, + "threshold": -0.04661082476377487, + "operator": "ge", + "actual_fraction": 0.1810918774966711, + "metric": 0.08900834067514127, + "rescued": 9, + "harmed": 101, + "missed_rescue": 3452, + "avoided_harm": 575, + "selected_both_correct": 2065, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.04730507358908653, + "operator": "ge", + "actual_fraction": 0.1870838881491345, + "metric": 0.09716527158339765, + "rescued": 11, + "harmed": 107, + "missed_rescue": 3450, + "avoided_harm": 569, + "selected_both_correct": 2129, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.048246145248413086, + "operator": "ge", + "actual_fraction": 0.19640479360852198, + "metric": 0.09733135745692614, + "rescued": 12, + "harmed": 112, + "missed_rescue": 3449, + "avoided_harm": 564, + "selected_both_correct": 2234, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.04900278151035309, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.09686185392269288, + "rescued": 14, + "harmed": 124, + "missed_rescue": 3447, + "avoided_harm": 552, + "selected_both_correct": 2363, + "selected_both_wrong": 3 + }, + { + "budget": 0.23, + "threshold": -0.050042688846588135, + "operator": "ge", + "actual_fraction": 0.21471371504660453, + "metric": 0.09673072357913137, + "rescued": 14, + "harmed": 125, + "missed_rescue": 3447, + "avoided_harm": 551, + "selected_both_correct": 2438, + "selected_both_wrong": 3 + }, + { + "budget": 0.24, + "threshold": -0.050901688635349274, + "operator": "ge", + "actual_fraction": 0.22536617842876164, + "metric": 0.10080716337703104, + "rescued": 48, + "harmed": 130, + "missed_rescue": 3413, + "avoided_harm": 546, + "selected_both_correct": 2526, + "selected_both_wrong": 4 + }, + { + "budget": 0.25, + "threshold": -0.05161922052502632, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.10058232286929361, + "rescued": 65, + "harmed": 134, + "missed_rescue": 3396, + "avoided_harm": 542, + "selected_both_correct": 2575, + "selected_both_wrong": 6 + }, + { + "budget": 0.26, + "threshold": -0.05245445668697357, + "operator": "ge", + "actual_fraction": 0.237017310252996, + "metric": 0.10078195333663684, + "rescued": 81, + "harmed": 136, + "missed_rescue": 3380, + "avoided_harm": 540, + "selected_both_correct": 2624, + "selected_both_wrong": 7 + }, + { + "budget": 0.27, + "threshold": -0.053079016506671906, + "operator": "ge", + "actual_fraction": 0.24167776298268975, + "metric": 0.10061353044193348, + "rescued": 95, + "harmed": 141, + "missed_rescue": 3366, + "avoided_harm": 535, + "selected_both_correct": 2660, + "selected_both_wrong": 8 + }, + { + "budget": 0.28, + "threshold": -0.053604189306497574, + "operator": "ge", + "actual_fraction": 0.2470039946737683, + "metric": 0.10058305058987213, + "rescued": 111, + "harmed": 144, + "missed_rescue": 3350, + "avoided_harm": 532, + "selected_both_correct": 2705, + "selected_both_wrong": 8 + }, + { + "budget": 0.29, + "threshold": -0.05505518987774849, + "operator": "ge", + "actual_fraction": 0.2623169107856192, + "metric": 0.10443538242821891, + "rescued": 159, + "harmed": 148, + "missed_rescue": 3302, + "avoided_harm": 528, + "selected_both_correct": 2830, + "selected_both_wrong": 15 + }, + { + "budget": 0.3, + "threshold": -0.05605518817901611, + "operator": "ge", + "actual_fraction": 0.27463382157123833, + "metric": 0.10759036763747168, + "rescued": 204, + "harmed": 158, + "missed_rescue": 3257, + "avoided_harm": 518, + "selected_both_correct": 2918, + "selected_both_wrong": 20 + }, + { + "budget": 0.31, + "threshold": -0.056986160576343536, + "operator": "ge", + "actual_fraction": 0.28462050599201066, + "metric": 0.10779091241357944, + "rescued": 236, + "harmed": 165, + "missed_rescue": 3225, + "avoided_harm": 511, + "selected_both_correct": 2997, + "selected_both_wrong": 22 + }, + { + "budget": 0.32, + "threshold": -0.0582931749522686, + "operator": "ge", + "actual_fraction": 0.29427430093209056, + "metric": 0.11066132714400925, + "rescued": 268, + "harmed": 172, + "missed_rescue": 3193, + "avoided_harm": 504, + "selected_both_correct": 3068, + "selected_both_wrong": 28 + }, + { + "budget": 0.33, + "threshold": -0.0596650205552578, + "operator": "ge", + "actual_fraction": 0.3062583222370173, + "metric": 0.11055610302472227, + "rescued": 309, + "harmed": 182, + "missed_rescue": 3152, + "avoided_harm": 494, + "selected_both_correct": 3160, + "selected_both_wrong": 29 + }, + { + "budget": 0.34, + "threshold": -0.060549296438694, + "operator": "ge", + "actual_fraction": 0.3132490013315579, + "metric": 0.1105675542193679, + "rescued": 337, + "harmed": 186, + "missed_rescue": 3124, + "avoided_harm": 490, + "selected_both_correct": 3210, + "selected_both_wrong": 31 + }, + { + "budget": 0.35, + "threshold": -0.06178095191717148, + "operator": "ge", + "actual_fraction": 0.3189081225033289, + "metric": 0.11000647533107741, + "rescued": 355, + "harmed": 194, + "missed_rescue": 3106, + "avoided_harm": 482, + "selected_both_correct": 3250, + "selected_both_wrong": 33 + }, + { + "budget": 0.36, + "threshold": -0.06328777223825455, + "operator": "ge", + "actual_fraction": 0.3318908122503329, + "metric": 0.11032470041071389, + "rescued": 397, + "harmed": 200, + "missed_rescue": 3064, + "avoided_harm": 476, + "selected_both_correct": 3350, + "selected_both_wrong": 41 + }, + { + "budget": 0.37, + "threshold": -0.0647548958659172, + "operator": "ge", + "actual_fraction": 0.3418774966711052, + "metric": 0.11048463654687253, + "rescued": 437, + "harmed": 206, + "missed_rescue": 3024, + "avoided_harm": 470, + "selected_both_correct": 3424, + "selected_both_wrong": 41 + }, + { + "budget": 0.38, + "threshold": -0.06637481600046158, + "operator": "ge", + "actual_fraction": 0.35352862849533956, + "metric": 0.11263903545823285, + "rescued": 484, + "harmed": 220, + "missed_rescue": 2977, + "avoided_harm": 456, + "selected_both_correct": 3496, + "selected_both_wrong": 48 + }, + { + "budget": 0.39, + "threshold": -0.06774259358644485, + "operator": "ge", + "actual_fraction": 0.3641810918774967, + "metric": 0.11519364722073223, + "rescued": 527, + "harmed": 227, + "missed_rescue": 2934, + "avoided_harm": 449, + "selected_both_correct": 3570, + "selected_both_wrong": 52 + }, + { + "budget": 0.4, + "threshold": -0.06843380630016327, + "operator": "ge", + "actual_fraction": 0.3698402130492676, + "metric": 0.11587206094412077, + "rescued": 550, + "harmed": 228, + "missed_rescue": 2911, + "avoided_harm": 448, + "selected_both_correct": 3609, + "selected_both_wrong": 57 + }, + { + "budget": 0.41, + "threshold": -0.0694691464304924, + "operator": "ge", + "actual_fraction": 0.3801597869507324, + "metric": 0.11650140520518415, + "rescued": 593, + "harmed": 240, + "missed_rescue": 2868, + "avoided_harm": 436, + "selected_both_correct": 3672, + "selected_both_wrong": 63 + }, + { + "budget": 0.42, + "threshold": -0.07112707942724228, + "operator": "ge", + "actual_fraction": 0.3938082556591212, + "metric": 0.11662726768380335, + "rescued": 646, + "harmed": 251, + "missed_rescue": 2815, + "avoided_harm": 425, + "selected_both_correct": 3765, + "selected_both_wrong": 70 + }, + { + "budget": 0.43, + "threshold": -0.07192998379468918, + "operator": "ge", + "actual_fraction": 0.3988015978695073, + "metric": 0.1170616446996618, + "rescued": 667, + "harmed": 252, + "missed_rescue": 2794, + "avoided_harm": 424, + "selected_both_correct": 3802, + "selected_both_wrong": 71 + }, + { + "budget": 0.44, + "threshold": -0.07266208529472351, + "operator": "ge", + "actual_fraction": 0.40279627163781623, + "metric": 0.1191343028439446, + "rescued": 682, + "harmed": 256, + "missed_rescue": 2779, + "avoided_harm": 420, + "selected_both_correct": 3830, + "selected_both_wrong": 72 + }, + { + "budget": 0.45, + "threshold": -0.0744253620505333, + "operator": "ge", + "actual_fraction": 0.41711051930758986, + "metric": 0.12016591353047154, + "rescued": 738, + "harmed": 265, + "missed_rescue": 2723, + "avoided_harm": 411, + "selected_both_correct": 3927, + "selected_both_wrong": 82 + }, + { + "budget": 0.46, + "threshold": -0.07532940804958344, + "operator": "ge", + "actual_fraction": 0.42243675099866845, + "metric": 0.12056162004139302, + "rescued": 764, + "harmed": 271, + "missed_rescue": 2697, + "avoided_harm": 405, + "selected_both_correct": 3955, + "selected_both_wrong": 86 + }, + { + "budget": 0.47, + "threshold": -0.0770750343799591, + "operator": "ge", + "actual_fraction": 0.4380825565912117, + "metric": 0.121709486901851, + "rescued": 839, + "harmed": 283, + "missed_rescue": 2622, + "avoided_harm": 393, + "selected_both_correct": 4049, + "selected_both_wrong": 93 + }, + { + "budget": 0.48, + "threshold": -0.07818037271499634, + "operator": "ge", + "actual_fraction": 0.44573901464713717, + "metric": 0.1221443001236399, + "rescued": 873, + "harmed": 288, + "missed_rescue": 2588, + "avoided_harm": 388, + "selected_both_correct": 4097, + "selected_both_wrong": 98 + }, + { + "budget": 0.49, + "threshold": -0.07998209446668625, + "operator": "ge", + "actual_fraction": 0.45905459387483355, + "metric": 0.1251607808901142, + "rescued": 928, + "harmed": 294, + "missed_rescue": 2533, + "avoided_harm": 382, + "selected_both_correct": 4185, + "selected_both_wrong": 109 + }, + { + "budget": 0.5, + "threshold": -0.08124949783086777, + "operator": "ge", + "actual_fraction": 0.46937416777629826, + "metric": 0.12551652022923962, + "rescued": 974, + "harmed": 309, + "missed_rescue": 2487, + "avoided_harm": 367, + "selected_both_correct": 4244, + "selected_both_wrong": 113 + }, + { + "budget": 0.51, + "threshold": -0.08398465067148209, + "operator": "ge", + "actual_fraction": 0.49467376830892146, + "metric": 0.12932775258830043, + "rescued": 1093, + "harmed": 334, + "missed_rescue": 2368, + "avoided_harm": 342, + "selected_both_correct": 4389, + "selected_both_wrong": 128 + }, + { + "budget": 0.52, + "threshold": -0.08537434041500092, + "operator": "ge", + "actual_fraction": 0.5056591211717709, + "metric": 0.13382418312702152, + "rescued": 1145, + "harmed": 340, + "missed_rescue": 2316, + "avoided_harm": 336, + "selected_both_correct": 4457, + "selected_both_wrong": 134 + }, + { + "budget": 0.53, + "threshold": -0.08664409816265106, + "operator": "ge", + "actual_fraction": 0.5159786950732357, + "metric": 0.1342986185420429, + "rescued": 1190, + "harmed": 350, + "missed_rescue": 2271, + "avoided_harm": 326, + "selected_both_correct": 4522, + "selected_both_wrong": 138 + }, + { + "budget": 0.54, + "threshold": -0.08777308464050293, + "operator": "ge", + "actual_fraction": 0.524300932090546, + "metric": 0.13511006926922334, + "rescued": 1228, + "harmed": 356, + "missed_rescue": 2233, + "avoided_harm": 320, + "selected_both_correct": 4574, + "selected_both_wrong": 142 + }, + { + "budget": 0.55, + "threshold": -0.08980843424797058, + "operator": "ge", + "actual_fraction": 0.5386151797603196, + "metric": 0.13883610114027645, + "rescued": 1297, + "harmed": 367, + "missed_rescue": 2164, + "avoided_harm": 309, + "selected_both_correct": 4657, + "selected_both_wrong": 151 + }, + { + "budget": 0.56, + "threshold": -0.0906188115477562, + "operator": "ge", + "actual_fraction": 0.546271637816245, + "metric": 0.13992298734058117, + "rescued": 1332, + "harmed": 375, + "missed_rescue": 2129, + "avoided_harm": 301, + "selected_both_correct": 4702, + "selected_both_wrong": 155 + }, + { + "budget": 0.57, + "threshold": -0.09160211682319641, + "operator": "ge", + "actual_fraction": 0.5555925432756325, + "metric": 0.14072696062294485, + "rescued": 1379, + "harmed": 383, + "missed_rescue": 2082, + "avoided_harm": 293, + "selected_both_correct": 4751, + "selected_both_wrong": 163 + }, + { + "budget": 0.58, + "threshold": -0.09283093363046646, + "operator": "ge", + "actual_fraction": 0.568242343541944, + "metric": 0.1472561682256289, + "rescued": 1434, + "harmed": 393, + "missed_rescue": 2027, + "avoided_harm": 283, + "selected_both_correct": 4827, + "selected_both_wrong": 174 + }, + { + "budget": 0.59, + "threshold": -0.0941123366355896, + "operator": "ge", + "actual_fraction": 0.5778961384820239, + "metric": 0.15078547139034337, + "rescued": 1482, + "harmed": 399, + "missed_rescue": 1979, + "avoided_harm": 277, + "selected_both_correct": 4882, + "selected_both_wrong": 181 + }, + { + "budget": 0.6, + "threshold": -0.09614869952201843, + "operator": "ge", + "actual_fraction": 0.5878828229027963, + "metric": 0.15176768468059784, + "rescued": 1528, + "harmed": 406, + "missed_rescue": 1933, + "avoided_harm": 270, + "selected_both_correct": 4939, + "selected_both_wrong": 191 + }, + { + "budget": 0.61, + "threshold": -0.09772664308547974, + "operator": "ge", + "actual_fraction": 0.5975366178428761, + "metric": 0.1532082936739425, + "rescued": 1567, + "harmed": 414, + "missed_rescue": 1894, + "avoided_harm": 262, + "selected_both_correct": 4998, + "selected_both_wrong": 201 + }, + { + "budget": 0.62, + "threshold": -0.09911532700061798, + "operator": "ge", + "actual_fraction": 0.6081890812250332, + "metric": 0.15424732264646662, + "rescued": 1615, + "harmed": 422, + "missed_rescue": 1846, + "avoided_harm": 254, + "selected_both_correct": 5061, + "selected_both_wrong": 210 + }, + { + "budget": 0.63, + "threshold": -0.10007988661527634, + "operator": "ge", + "actual_fraction": 0.6161784287616512, + "metric": 0.1548871102093026, + "rescued": 1652, + "harmed": 430, + "missed_rescue": 1809, + "avoided_harm": 246, + "selected_both_correct": 5104, + "selected_both_wrong": 218 + }, + { + "budget": 0.64, + "threshold": -0.10177135467529297, + "operator": "ge", + "actual_fraction": 0.6278295605858855, + "metric": 0.15779981350208042, + "rescued": 1709, + "harmed": 438, + "missed_rescue": 1752, + "avoided_harm": 238, + "selected_both_correct": 5169, + "selected_both_wrong": 228 + }, + { + "budget": 0.65, + "threshold": -0.10395076125860214, + "operator": "ge", + "actual_fraction": 0.6451398135818908, + "metric": 0.16144984235200888, + "rescued": 1795, + "harmed": 448, + "missed_rescue": 1666, + "avoided_harm": 228, + "selected_both_correct": 5271, + "selected_both_wrong": 238 + }, + { + "budget": 0.66, + "threshold": -0.10524322092533112, + "operator": "ge", + "actual_fraction": 0.6491344873501997, + "metric": 0.16469010323619201, + "rescued": 1816, + "harmed": 448, + "missed_rescue": 1645, + "avoided_harm": 228, + "selected_both_correct": 5295, + "selected_both_wrong": 241 + }, + { + "budget": 0.67, + "threshold": -0.10689426958560944, + "operator": "ge", + "actual_fraction": 0.6601198402130493, + "metric": 0.16600232483815708, + "rescued": 1872, + "harmed": 456, + "missed_rescue": 1589, + "avoided_harm": 220, + "selected_both_correct": 5357, + "selected_both_wrong": 247 + }, + { + "budget": 0.68, + "threshold": -0.11014843732118607, + "operator": "ge", + "actual_fraction": 0.6754327563249002, + "metric": 0.16941529572817854, + "rescued": 1941, + "harmed": 465, + "missed_rescue": 1520, + "avoided_harm": 211, + "selected_both_correct": 5452, + "selected_both_wrong": 258 + }, + { + "budget": 0.69, + "threshold": -0.11117545515298843, + "operator": "ge", + "actual_fraction": 0.6790945406125166, + "metric": 0.17007927366959905, + "rescued": 1960, + "harmed": 467, + "missed_rescue": 1501, + "avoided_harm": 209, + "selected_both_correct": 5471, + "selected_both_wrong": 262 + }, + { + "budget": 0.7, + "threshold": -0.11230025440454483, + "operator": "ge", + "actual_fraction": 0.6870838881491345, + "metric": 0.17172535263617786, + "rescued": 2005, + "harmed": 471, + "missed_rescue": 1456, + "avoided_harm": 205, + "selected_both_correct": 5514, + "selected_both_wrong": 266 + }, + { + "budget": 0.71, + "threshold": -0.11423606425523758, + "operator": "ge", + "actual_fraction": 0.6960719041278296, + "metric": 0.17427123861822869, + "rescued": 2047, + "harmed": 477, + "missed_rescue": 1414, + "avoided_harm": 199, + "selected_both_correct": 5562, + "selected_both_wrong": 278 + }, + { + "budget": 0.72, + "threshold": -0.11647249013185501, + "operator": "ge", + "actual_fraction": 0.7053928095872171, + "metric": 0.17661848328954455, + "rescued": 2090, + "harmed": 486, + "missed_rescue": 1371, + "avoided_harm": 190, + "selected_both_correct": 5613, + "selected_both_wrong": 287 + }, + { + "budget": 0.73, + "threshold": -0.11962417513132095, + "operator": "ge", + "actual_fraction": 0.7193741677762983, + "metric": 0.18093016254845928, + "rescued": 2156, + "harmed": 500, + "missed_rescue": 1305, + "avoided_harm": 176, + "selected_both_correct": 5687, + "selected_both_wrong": 301 + }, + { + "budget": 0.74, + "threshold": -0.12126146256923676, + "operator": "ge", + "actual_fraction": 0.7250332889480693, + "metric": 0.18214911344128, + "rescued": 2186, + "harmed": 504, + "missed_rescue": 1275, + "avoided_harm": 172, + "selected_both_correct": 5719, + "selected_both_wrong": 303 + }, + { + "budget": 0.75, + "threshold": -0.1230982318520546, + "operator": "ge", + "actual_fraction": 0.7366844207723036, + "metric": 0.18474146474609127, + "rescued": 2239, + "harmed": 510, + "missed_rescue": 1222, + "avoided_harm": 166, + "selected_both_correct": 5790, + "selected_both_wrong": 313 + }, + { + "budget": 0.76, + "threshold": -0.12517565488815308, + "operator": "ge", + "actual_fraction": 0.7446737683089214, + "metric": 0.1868238104649864, + "rescued": 2278, + "harmed": 513, + "missed_rescue": 1183, + "avoided_harm": 163, + "selected_both_correct": 5841, + "selected_both_wrong": 316 + }, + { + "budget": 0.77, + "threshold": -0.12628257274627686, + "operator": "ge", + "actual_fraction": 0.7480026631158455, + "metric": 0.18780967278670027, + "rescued": 2296, + "harmed": 513, + "missed_rescue": 1165, + "avoided_harm": 163, + "selected_both_correct": 5861, + "selected_both_wrong": 318 + }, + { + "budget": 0.78, + "threshold": -0.12975578010082245, + "operator": "ge", + "actual_fraction": 0.7603195739014648, + "metric": 0.19033940523013337, + "rescued": 2354, + "harmed": 518, + "missed_rescue": 1107, + "avoided_harm": 158, + "selected_both_correct": 5931, + "selected_both_wrong": 333 + }, + { + "budget": 0.79, + "threshold": -0.13229140639305115, + "operator": "ge", + "actual_fraction": 0.7713049267643143, + "metric": 0.19289795000814608, + "rescued": 2407, + "harmed": 524, + "missed_rescue": 1054, + "avoided_harm": 152, + "selected_both_correct": 5996, + "selected_both_wrong": 341 + }, + { + "budget": 0.8, + "threshold": -0.1373685747385025, + "operator": "ge", + "actual_fraction": 0.7909454061251664, + "metric": 0.1970331406058424, + "rescued": 2510, + "harmed": 548, + "missed_rescue": 951, + "avoided_harm": 128, + "selected_both_correct": 6090, + "selected_both_wrong": 356 + }, + { + "budget": 0.81, + "threshold": -0.1407174915075302, + "operator": "ge", + "actual_fraction": 0.8012649800266312, + "metric": 0.1991542632866029, + "rescued": 2560, + "harmed": 556, + "missed_rescue": 901, + "avoided_harm": 120, + "selected_both_correct": 6144, + "selected_both_wrong": 368 + }, + { + "budget": 0.82, + "threshold": -0.1433146893978119, + "operator": "ge", + "actual_fraction": 0.8125832223701731, + "metric": 0.2006018605686543, + "rescued": 2613, + "harmed": 569, + "missed_rescue": 848, + "avoided_harm": 107, + "selected_both_correct": 6204, + "selected_both_wrong": 378 + }, + { + "budget": 0.83, + "threshold": -0.1457100659608841, + "operator": "ge", + "actual_fraction": 0.8199067909454061, + "metric": 0.20192079535680077, + "rescued": 2654, + "harmed": 577, + "missed_rescue": 807, + "avoided_harm": 99, + "selected_both_correct": 6238, + "selected_both_wrong": 383 + }, + { + "budget": 0.84, + "threshold": -0.148710235953331, + "operator": "ge", + "actual_fraction": 0.8242343541944075, + "metric": 0.20317936971986447, + "rescued": 2674, + "harmed": 578, + "missed_rescue": 787, + "avoided_harm": 98, + "selected_both_correct": 6264, + "selected_both_wrong": 388 + }, + { + "budget": 0.85, + "threshold": -0.15291842818260193, + "operator": "ge", + "actual_fraction": 0.8338881491344874, + "metric": 0.2060155880710891, + "rescued": 2727, + "harmed": 591, + "missed_rescue": 734, + "avoided_harm": 85, + "selected_both_correct": 6311, + "selected_both_wrong": 391 + }, + { + "budget": 0.86, + "threshold": -0.15889839828014374, + "operator": "ge", + "actual_fraction": 0.8458721704394141, + "metric": 0.20894280280516728, + "rescued": 2777, + "harmed": 600, + "missed_rescue": 684, + "avoided_harm": 76, + "selected_both_correct": 6383, + "selected_both_wrong": 404 + }, + { + "budget": 0.87, + "threshold": -0.16629797220230103, + "operator": "ge", + "actual_fraction": 0.8585219707057257, + "metric": 0.2146238532926011, + "rescued": 2839, + "harmed": 603, + "missed_rescue": 622, + "avoided_harm": 73, + "selected_both_correct": 6459, + "selected_both_wrong": 415 + }, + { + "budget": 0.88, + "threshold": -0.17295491695404053, + "operator": "ge", + "actual_fraction": 0.8655126498002663, + "metric": 0.21521435730631172, + "rescued": 2868, + "harmed": 611, + "missed_rescue": 593, + "avoided_harm": 65, + "selected_both_correct": 6497, + "selected_both_wrong": 424 + }, + { + "budget": 0.89, + "threshold": -0.18053314089775085, + "operator": "ge", + "actual_fraction": 0.8738348868175766, + "metric": 0.2202221543306381, + "rescued": 2899, + "harmed": 617, + "missed_rescue": 562, + "avoided_harm": 59, + "selected_both_correct": 6544, + "selected_both_wrong": 440 + }, + { + "budget": 0.9, + "threshold": -0.19442276656627655, + "operator": "ge", + "actual_fraction": 0.8878162450066578, + "metric": 0.22393167777841913, + "rescued": 2961, + "harmed": 626, + "missed_rescue": 500, + "avoided_harm": 50, + "selected_both_correct": 6624, + "selected_both_wrong": 457 + }, + { + "budget": 0.91, + "threshold": -0.20900504291057587, + "operator": "ge", + "actual_fraction": 0.8961384820239681, + "metric": 0.22795997864890608, + "rescued": 3002, + "harmed": 629, + "missed_rescue": 459, + "avoided_harm": 47, + "selected_both_correct": 6676, + "selected_both_wrong": 461 + }, + { + "budget": 0.92, + "threshold": -0.22383525967597961, + "operator": "ge", + "actual_fraction": 0.9027962716378163, + "metric": 0.23118004657749075, + "rescued": 3034, + "harmed": 634, + "missed_rescue": 427, + "avoided_harm": 42, + "selected_both_correct": 6715, + "selected_both_wrong": 465 + }, + { + "budget": 0.93, + "threshold": -0.25952714681625366, + "operator": "ge", + "actual_fraction": 0.9184420772303595, + "metric": 0.23697523764758885, + "rescued": 3106, + "harmed": 646, + "missed_rescue": 355, + "avoided_harm": 30, + "selected_both_correct": 6801, + "selected_both_wrong": 483 + }, + { + "budget": 0.94, + "threshold": -0.2804492115974426, + "operator": "ge", + "actual_fraction": 0.9257656458055925, + "metric": 0.24091409959663834, + "rescued": 3141, + "harmed": 647, + "missed_rescue": 320, + "avoided_harm": 29, + "selected_both_correct": 6844, + "selected_both_wrong": 492 + }, + { + "budget": 0.95, + "threshold": -0.2984963357448578, + "operator": "ge", + "actual_fraction": 0.9320905459387483, + "metric": 0.2426031328046549, + "rescued": 3163, + "harmed": 648, + "missed_rescue": 298, + "avoided_harm": 28, + "selected_both_correct": 6890, + "selected_both_wrong": 499 + }, + { + "budget": 0.96, + "threshold": -0.32223379611968994, + "operator": "ge", + "actual_fraction": 0.9410785619174434, + "metric": 0.24537310927695588, + "rescued": 3203, + "harmed": 656, + "missed_rescue": 258, + "avoided_harm": 20, + "selected_both_correct": 6944, + "selected_both_wrong": 505 + }, + { + "budget": 0.97, + "threshold": -0.36503666639328003, + "operator": "ge", + "actual_fraction": 0.9573901464713716, + "metric": 0.25207284548974734, + "rescued": 3277, + "harmed": 663, + "missed_rescue": 184, + "avoided_harm": 13, + "selected_both_correct": 7042, + "selected_both_wrong": 522 + }, + { + "budget": 0.98, + "threshold": -0.4221048653125763, + "operator": "ge", + "actual_fraction": 0.9790279627163782, + "metric": 0.2631165680871859, + "rescued": 3373, + "harmed": 668, + "missed_rescue": 88, + "avoided_harm": 8, + "selected_both_correct": 7186, + "selected_both_wrong": 537 + }, + { + "budget": 0.99, + "threshold": -0.44162338972091675, + "operator": "ge", + "actual_fraction": 0.9860186418109188, + "metric": 0.26818431283513144, + "rescued": 3404, + "harmed": 671, + "missed_rescue": 57, + "avoided_harm": 5, + "selected_both_correct": 7228, + "selected_both_wrong": 545 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 3461, + "harmed": 676, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7329, + "selected_both_wrong": 550 + } + ] + }, + "learned": { + "validation_area_0_25": 0.11567459806423948, + "validation_point": { + "metric": 0.2713761319061188, + "metric_std": 0.0019541144340920026, + "actual_fraction": 0.8088445078459342, + "actual_fraction_std": 0.01578883282537709, + "per_seed": [ + { + "seed": 2026, + "budget": 0.8, + "threshold": -0.0004259832203388214, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.2699586861016554, + "rescue": 813, + "harm": 143, + "missed_rescue": 2 + }, + { + "seed": 2027, + "budget": 0.82, + "threshold": -0.007923062890768051, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.2738229314922339, + "rescue": 814, + "harm": 146, + "missed_rescue": 1 + }, + { + "seed": 2028, + "budget": 0.79, + "threshold": -0.0023735486902296543, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2699586861016554, + "rescue": 813, + "harm": 143, + "missed_rescue": 2 + }, + { + "seed": 2029, + "budget": 0.81, + "threshold": -0.00946833472698927, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.2699586861016554, + "rescue": 813, + "harm": 143, + "missed_rescue": 2 + }, + { + "seed": 2030, + "budget": 0.83, + "threshold": -0.015626760199666023, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.27318166973339386, + "rescue": 814, + "harm": 148, + "missed_rescue": 1 + } + ], + "rescued": 813.4, + "harmed": 144.6, + "missed_rescue": 1.6, + "avoided_harm": 40.400000000000006 + }, + "test_point": { + "metric": 0.26849490913617513, + "metric_std": 0.0013607443068895128, + "actual_fraction": 0.8147137150466046, + "actual_fraction_std": 0.013308503446227726, + "per_seed": [ + { + "seed": 2026, + "budget": 0.8, + "threshold": -0.0004259832203388214, + "operator": "ge", + "actual_fraction": 0.8099201065246339, + "metric": 0.2692688947733183, + "rescue": 3447, + "harm": 560, + "missed_rescue": 14 + }, + { + "seed": 2027, + "budget": 0.82, + "threshold": -0.007923062890768051, + "operator": "ge", + "actual_fraction": 0.8202396804260985, + "metric": 0.26776779485961383, + "rescue": 3448, + "harm": 572, + "missed_rescue": 13 + }, + { + "seed": 2028, + "budget": 0.79, + "threshold": -0.0023735486902296543, + "operator": "ge", + "actual_fraction": 0.7936085219707057, + "metric": 0.27017380559026777, + "rescue": 3447, + "harm": 554, + "missed_rescue": 14 + }, + { + "seed": 2029, + "budget": 0.81, + "threshold": -0.00946833472698927, + "operator": "ge", + "actual_fraction": 0.8239014647137151, + "metric": 0.268624728799547, + "rescue": 3447, + "harm": 565, + "missed_rescue": 14 + }, + { + "seed": 2030, + "budget": 0.83, + "threshold": -0.015626760199666023, + "operator": "ge", + "actual_fraction": 0.8258988015978695, + "metric": 0.26663932165812865, + "rescue": 3447, + "harm": 576, + "missed_rescue": 14 + } + ], + "rescued": 3447.2, + "harmed": 565.4, + "missed_rescue": 13.8, + "avoided_harm": 110.60000000000002 + }, + "validation_curve": [ + { + "budget": 0.0, + "metric": 0.07641657905799261, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.08383731362706184, + "metric_std": 0.013996035030594175, + "actual_fraction": 0.009985734664764621, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.02, + "metric": 0.08852337158015235, + "metric_std": 0.014281593703726973, + "actual_fraction": 0.019971469329529243, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.03, + "metric": 0.08712353366934261, + "metric_std": 0.012151297338908098, + "actual_fraction": 0.029957203994293864, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.04, + "metric": 0.09548875974703774, + "metric_std": 0.009420403188872646, + "actual_fraction": 0.039942938659058486, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.05, + "metric": 0.0987160467864239, + "metric_std": 0.014686898253040547, + "actual_fraction": 0.04992867332382311, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.06, + "metric": 0.09861719059538576, + "metric_std": 0.014604375909154631, + "actual_fraction": 0.05991440798858773, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.07, + "metric": 0.09863109968863086, + "metric_std": 0.013378326202284744, + "actual_fraction": 0.06990014265335236, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.08, + "metric": 0.09758142882409267, + "metric_std": 0.012441025884924616, + "actual_fraction": 0.07988587731811697, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.09, + "metric": 0.09840151467211075, + "metric_std": 0.011894576937222302, + "actual_fraction": 0.0898716119828816, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.1, + "metric": 0.10434511268199628, + "metric_std": 0.008746288588808535, + "actual_fraction": 0.09985734664764621, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.11, + "metric": 0.10496100175418008, + "metric_std": 0.008819039443611133, + "actual_fraction": 0.10984308131241086, + "actual_fraction_std": 1.5515838457795457e-17 + }, + { + "budget": 0.12, + "metric": 0.10791035896564975, + "metric_std": 0.008160268351004222, + "actual_fraction": 0.11982881597717546, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.13, + "metric": 0.11488948578142275, + "metric_std": 0.013115965288903346, + "actual_fraction": 0.12981455064194009, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.14, + "metric": 0.12491510752622266, + "metric_std": 0.011661362915717526, + "actual_fraction": 0.1398002853067047, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.15, + "metric": 0.12939446778020974, + "metric_std": 0.01351617007411095, + "actual_fraction": 0.14978601997146934, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.16, + "metric": 0.12954016164295307, + "metric_std": 0.012548792454087439, + "actual_fraction": 0.15977175463623394, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.17, + "metric": 0.13159978302907777, + "metric_std": 0.013101521857600149, + "actual_fraction": 0.16975748930099857, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.18, + "metric": 0.13143074094184515, + "metric_std": 0.012637194183860835, + "actual_fraction": 0.1797432239657632, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.19, + "metric": 0.13223891991706987, + "metric_std": 0.012790459267750668, + "actual_fraction": 0.18972895863052783, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.2, + "metric": 0.13434087065229094, + "metric_std": 0.014059467826376662, + "actual_fraction": 0.19971469329529243, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.21, + "metric": 0.1365692535665169, + "metric_std": 0.016626713933332024, + "actual_fraction": 0.20970042796005703, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.22, + "metric": 0.13915823838418867, + "metric_std": 0.0159109092643646, + "actual_fraction": 0.2196861626248217, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.23, + "metric": 0.1427078542054692, + "metric_std": 0.015915626086108265, + "actual_fraction": 0.2296718972895863, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.24, + "metric": 0.14410376981549028, + "metric_std": 0.01811739768419971, + "actual_fraction": 0.2396576319543509, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.25, + "metric": 0.14673640578256056, + "metric_std": 0.017253112500041957, + "actual_fraction": 0.24964336661911551, + "actual_fraction_std": 3.1031676915590914e-17 + }, + { + "budget": 0.26, + "metric": 0.1473180722112803, + "metric_std": 0.01614172157027511, + "actual_fraction": 0.25962910128388017, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.27, + "metric": 0.15137646728081422, + "metric_std": 0.015531717188673987, + "actual_fraction": 0.26961483594864477, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.28, + "metric": 0.1520973988808812, + "metric_std": 0.015334544086925271, + "actual_fraction": 0.2796005706134094, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.29, + "metric": 0.15484563419221006, + "metric_std": 0.011272172139401371, + "actual_fraction": 0.289586305278174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.3, + "metric": 0.15593701595837345, + "metric_std": 0.011053353944604051, + "actual_fraction": 0.2995720399429387, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.31, + "metric": 0.15820314406069008, + "metric_std": 0.013347419817259287, + "actual_fraction": 0.3095577746077033, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.32, + "metric": 0.15829791126538376, + "metric_std": 0.013165275454516526, + "actual_fraction": 0.3195435092724679, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.33, + "metric": 0.16279416496461507, + "metric_std": 0.012260958600984695, + "actual_fraction": 0.32952924393723254, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.34, + "metric": 0.16349270663690105, + "metric_std": 0.012115091520567322, + "actual_fraction": 0.33951497860199714, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.35, + "metric": 0.1639742154625103, + "metric_std": 0.011535764320984899, + "actual_fraction": 0.34950071326676174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.36, + "metric": 0.16485429754301764, + "metric_std": 0.011212711087778606, + "actual_fraction": 0.3594864479315264, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.37, + "metric": 0.1676048303411144, + "metric_std": 0.01080174453222727, + "actual_fraction": 0.369472182596291, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.38, + "metric": 0.16942443260342616, + "metric_std": 0.011424518697144302, + "actual_fraction": 0.37945791726105566, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.39, + "metric": 0.17505152960232956, + "metric_std": 0.008449924682641203, + "actual_fraction": 0.38944365192582026, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.4, + "metric": 0.17950695339548886, + "metric_std": 0.009477243261699268, + "actual_fraction": 0.39942938659058486, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.41, + "metric": 0.1824861653199062, + "metric_std": 0.0065168468607519235, + "actual_fraction": 0.4094151212553495, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.42, + "metric": 0.18493279970132226, + "metric_std": 0.006492000644360687, + "actual_fraction": 0.41940085592011406, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.43, + "metric": 0.1893238831584164, + "metric_std": 0.004290284056137979, + "actual_fraction": 0.4293865905848787, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.44, + "metric": 0.19360041695805513, + "metric_std": 0.003643271818882578, + "actual_fraction": 0.4393723252496434, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.45, + "metric": 0.19593558894165825, + "metric_std": 0.003173070909363517, + "actual_fraction": 0.44935805991440797, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.46, + "metric": 0.20081407873326623, + "metric_std": 0.004343083990232476, + "actual_fraction": 0.4593437945791726, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.47, + "metric": 0.2041286190392328, + "metric_std": 0.004112124798442667, + "actual_fraction": 0.4693295292439372, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.48, + "metric": 0.2040802346189329, + "metric_std": 0.00374584349047438, + "actual_fraction": 0.4793152639087018, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.49, + "metric": 0.2043146738067061, + "metric_std": 0.001083716048689991, + "actual_fraction": 0.4893009985734665, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.5, + "metric": 0.20584345710282412, + "metric_std": 0.0014504444703166766, + "actual_fraction": 0.49928673323823103, + "actual_fraction_std": 6.206335383118183e-17 + }, + { + "budget": 0.51, + "metric": 0.20695802306880995, + "metric_std": 0.0019222622423085802, + "actual_fraction": 0.5092724679029957, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.52, + "metric": 0.20925645891225358, + "metric_std": 0.004430939458147504, + "actual_fraction": 0.5192582025677603, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.53, + "metric": 0.2091213073747134, + "metric_std": 0.004494617904900125, + "actual_fraction": 0.5292439372325249, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.54, + "metric": 0.21025963185400545, + "metric_std": 0.005191538861974136, + "actual_fraction": 0.5392296718972895, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.55, + "metric": 0.2131119076131041, + "metric_std": 0.00522471617840453, + "actual_fraction": 0.5492154065620543, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.56, + "metric": 0.21610886715244887, + "metric_std": 0.004382312826424027, + "actual_fraction": 0.5592011412268189, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.57, + "metric": 0.21692396448780116, + "metric_std": 0.00500466832685227, + "actual_fraction": 0.5691868758915835, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.58, + "metric": 0.21877952813364274, + "metric_std": 0.0049639157781070685, + "actual_fraction": 0.579172610556348, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.59, + "metric": 0.21939107866641588, + "metric_std": 0.005440645309086484, + "actual_fraction": 0.5891583452211127, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.6, + "metric": 0.2211565079621128, + "metric_std": 0.005591264265805283, + "actual_fraction": 0.5991440798858774, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.61, + "metric": 0.22340463035513727, + "metric_std": 0.008246265287100726, + "actual_fraction": 0.609129814550642, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.62, + "metric": 0.22340160086472097, + "metric_std": 0.004540142175847835, + "actual_fraction": 0.6191155492154066, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.63, + "metric": 0.22542598829729382, + "metric_std": 0.004307089723286586, + "actual_fraction": 0.6291012838801712, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.64, + "metric": 0.22836987257844146, + "metric_std": 0.005101656367898596, + "actual_fraction": 0.6390870185449358, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.65, + "metric": 0.2309043170559555, + "metric_std": 0.005077773215909408, + "actual_fraction": 0.6490727532097005, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.66, + "metric": 0.23307102157322115, + "metric_std": 0.00512739051737774, + "actual_fraction": 0.6590584878744651, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.67, + "metric": 0.23479191572994268, + "metric_std": 0.005441775245854065, + "actual_fraction": 0.6690442225392297, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.68, + "metric": 0.23728105262402047, + "metric_std": 0.005205261319610207, + "actual_fraction": 0.6790299572039943, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.69, + "metric": 0.23999640170352543, + "metric_std": 0.004785757719356924, + "actual_fraction": 0.6890156918687589, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.7, + "metric": 0.2420613344178904, + "metric_std": 0.004409564187196275, + "actual_fraction": 0.6990014265335235, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.71, + "metric": 0.24600534659997875, + "metric_std": 0.004568373384421225, + "actual_fraction": 0.7089871611982882, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.72, + "metric": 0.24833324114020555, + "metric_std": 0.004127782428357731, + "actual_fraction": 0.7189728958630528, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.73, + "metric": 0.2542244721159041, + "metric_std": 0.005701377080410563, + "actual_fraction": 0.7289586305278174, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.74, + "metric": 0.2579331708245875, + "metric_std": 0.005265659449132158, + "actual_fraction": 0.738944365192582, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.75, + "metric": 0.26115561544154064, + "metric_std": 0.0047009471165207935, + "actual_fraction": 0.7489300998573466, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.76, + "metric": 0.26455174970895656, + "metric_std": 0.0039606010533703205, + "actual_fraction": 0.7589158345221113, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.77, + "metric": 0.2670182801801722, + "metric_std": 0.002067303451806008, + "actual_fraction": 0.7689015691868759, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.78, + "metric": 0.2689834735182155, + "metric_std": 0.0005262206999478275, + "actual_fraction": 0.7788873038516405, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.79, + "metric": 0.2686182734701871, + "metric_std": 0.0011154779318245766, + "actual_fraction": 0.7888730385164051, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.8, + "metric": 0.27041266750149184, + "metric_std": 0.0011985179145270475, + "actual_fraction": 0.7988587731811697, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.81, + "metric": 0.27048824177300146, + "metric_std": 0.001084073405957334, + "actual_fraction": 0.8088445078459344, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.82, + "metric": 0.2693280090358981, + "metric_std": 0.0028180730322900304, + "actual_fraction": 0.818830242510699, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.83, + "metric": 0.2694899147611565, + "metric_std": 0.003139973727247616, + "actual_fraction": 0.8288159771754635, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.84, + "metric": 0.2688962820969558, + "metric_std": 0.0033693589703556845, + "actual_fraction": 0.8388017118402281, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.85, + "metric": 0.26796109968805537, + "metric_std": 0.003562753210994646, + "actual_fraction": 0.8487874465049929, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.86, + "metric": 0.268232532363318, + "metric_std": 0.0014918602888125496, + "actual_fraction": 0.8587731811697574, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.87, + "metric": 0.26705961592148586, + "metric_std": 0.0015368183021786644, + "actual_fraction": 0.8687589158345222, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.88, + "metric": 0.26722490087165623, + "metric_std": 0.001142097381550298, + "actual_fraction": 0.8787446504992869, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.89, + "metric": 0.26643959584263827, + "metric_std": 0.001338549476410818, + "actual_fraction": 0.8887303851640513, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.9, + "metric": 0.2656698296432156, + "metric_std": 0.0016203203620779275, + "actual_fraction": 0.8987161198288159, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.91, + "metric": 0.26457197980235775, + "metric_std": 0.002512011913363301, + "actual_fraction": 0.9087018544935808, + "actual_fraction_std": 1.2412670766236366e-16 + }, + { + "budget": 0.92, + "metric": 0.26386557782276576, + "metric_std": 0.00270407243083804, + "actual_fraction": 0.9186875891583453, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.93, + "metric": 0.2637369387901529, + "metric_std": 0.0025571733255439267, + "actual_fraction": 0.9286733238231099, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.94, + "metric": 0.26243797791782597, + "metric_std": 0.0021974424309861447, + "actual_fraction": 0.9386590584878745, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.95, + "metric": 0.2617892100733869, + "metric_std": 0.0016054301533237484, + "actual_fraction": 0.948644793152639, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.96, + "metric": 0.2603789537857224, + "metric_std": 0.0009781991240360827, + "actual_fraction": 0.9586305278174037, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.97, + "metric": 0.25955949061954986, + "metric_std": 0.0006720002162138521, + "actual_fraction": 0.9686162624821684, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.98, + "metric": 0.2584194086288994, + "metric_std": 0.0007153019403282045, + "actual_fraction": 0.978601997146933, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.99, + "metric": 0.25614001483344134, + "metric_std": 0.0005208047895928747, + "actual_fraction": 0.9885877318116976, + "actual_fraction_std": 0.0 + }, + { + "budget": 1.0, + "metric": 0.25448626222520154, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ], + "test_curve": [ + { + "budget": 0.0, + "metric": 0.07887397871043929, + "metric_std": 0.0, + "actual_fraction": 0.0, + "actual_fraction_std": 0.0 + }, + { + "budget": 0.01, + "metric": 0.09009937372974958, + "metric_std": 0.011228669434574885, + "actual_fraction": 0.006657789613848201, + "actual_fraction_std": 0.003411102119160985 + }, + { + "budget": 0.02, + "metric": 0.09934330273933553, + "metric_std": 0.012641118916272231, + "actual_fraction": 0.014647137150466043, + "actual_fraction_std": 0.0036161053565912835 + }, + { + "budget": 0.03, + "metric": 0.1051605150497054, + "metric_std": 0.008918898833250168, + "actual_fraction": 0.0203728362183755, + "actual_fraction_std": 0.0015507076829363675 + }, + { + "budget": 0.04, + "metric": 0.10756770649827829, + "metric_std": 0.008274676985466094, + "actual_fraction": 0.02849533954727031, + "actual_fraction_std": 0.0020466612714965904 + }, + { + "budget": 0.05, + "metric": 0.10747896306120297, + "metric_std": 0.008730902046249224, + "actual_fraction": 0.03475366178428761, + "actual_fraction_std": 0.0032068161328341655 + }, + { + "budget": 0.06, + "metric": 0.1084401351367393, + "metric_std": 0.00919203110754668, + "actual_fraction": 0.041211717709720375, + "actual_fraction_std": 0.0043619955606656975 + }, + { + "budget": 0.07, + "metric": 0.11132398556914214, + "metric_std": 0.0074401723904211, + "actual_fraction": 0.04747003994673769, + "actual_fraction_std": 0.0032326293971531865 + }, + { + "budget": 0.08, + "metric": 0.11369297387104002, + "metric_std": 0.0059021457146322814, + "actual_fraction": 0.05452729693741678, + "actual_fraction_std": 0.004785973724720545 + }, + { + "budget": 0.09, + "metric": 0.11290990287413254, + "metric_std": 0.005518149901918872, + "actual_fraction": 0.061984021304926765, + "actual_fraction_std": 0.005796484383403171 + }, + { + "budget": 0.1, + "metric": 0.1146806544839756, + "metric_std": 0.005508837796021111, + "actual_fraction": 0.06950732356857522, + "actual_fraction_std": 0.005270808512302889 + }, + { + "budget": 0.11, + "metric": 0.11724392606999827, + "metric_std": 0.006674861320916054, + "actual_fraction": 0.07969374167776298, + "actual_fraction_std": 0.005203095595190266 + }, + { + "budget": 0.12, + "metric": 0.11900861476191346, + "metric_std": 0.007316885682806908, + "actual_fraction": 0.08648468708388815, + "actual_fraction_std": 0.004692442993033602 + }, + { + "budget": 0.13, + "metric": 0.12142782796047895, + "metric_std": 0.0063337028028945435, + "actual_fraction": 0.09760319573901464, + "actual_fraction_std": 0.0034386049290590494 + }, + { + "budget": 0.14, + "metric": 0.12084104592562114, + "metric_std": 0.0063791641721212535, + "actual_fraction": 0.1081225033288948, + "actual_fraction_std": 0.003348808612505805 + }, + { + "budget": 0.15, + "metric": 0.12215672466911216, + "metric_std": 0.006762184586943782, + "actual_fraction": 0.12090545938748336, + "actual_fraction_std": 0.0053697456861329495 + }, + { + "budget": 0.16, + "metric": 0.12235974339109021, + "metric_std": 0.005276598261949232, + "actual_fraction": 0.13242343541944074, + "actual_fraction_std": 0.0068359971372275235 + }, + { + "budget": 0.17, + "metric": 0.12399887979475246, + "metric_std": 0.005366601115447669, + "actual_fraction": 0.1403462050599201, + "actual_fraction_std": 0.006751175944536173 + }, + { + "budget": 0.18, + "metric": 0.12646328429838574, + "metric_std": 0.005553000902589218, + "actual_fraction": 0.1492676431424767, + "actual_fraction_std": 0.00567671737610977 + }, + { + "budget": 0.19, + "metric": 0.12786532252768623, + "metric_std": 0.005881603122609034, + "actual_fraction": 0.1595872170439414, + "actual_fraction_std": 0.007734219828191247 + }, + { + "budget": 0.2, + "metric": 0.13022560901609445, + "metric_std": 0.006379286575025108, + "actual_fraction": 0.17476697736351532, + "actual_fraction_std": 0.005464865232816914 + }, + { + "budget": 0.21, + "metric": 0.1322689807858753, + "metric_std": 0.00710640818543305, + "actual_fraction": 0.1833555259653795, + "actual_fraction_std": 0.0030636685392392976 + }, + { + "budget": 0.22, + "metric": 0.13353335404111322, + "metric_std": 0.008082509436665813, + "actual_fraction": 0.19241011984021306, + "actual_fraction_std": 0.005140960456551433 + }, + { + "budget": 0.23, + "metric": 0.13524552044990593, + "metric_std": 0.009246476951805603, + "actual_fraction": 0.20579227696404795, + "actual_fraction_std": 0.00772633544468498 + }, + { + "budget": 0.24, + "metric": 0.13629033845529676, + "metric_std": 0.007702848156331887, + "actual_fraction": 0.21724367509986683, + "actual_fraction_std": 0.006176871733187804 + }, + { + "budget": 0.25, + "metric": 0.1380091462139273, + "metric_std": 0.005577561989058995, + "actual_fraction": 0.2307589880159787, + "actual_fraction_std": 0.00869791227358674 + }, + { + "budget": 0.26, + "metric": 0.1385079152334597, + "metric_std": 0.00487952669345694, + "actual_fraction": 0.24174434087882818, + "actual_fraction_std": 0.010029314251185611 + }, + { + "budget": 0.27, + "metric": 0.13979751912568447, + "metric_std": 0.0034258841044314636, + "actual_fraction": 0.25019973368841547, + "actual_fraction_std": 0.010098683996893662 + }, + { + "budget": 0.28, + "metric": 0.14061608161743383, + "metric_std": 0.004272648850050701, + "actual_fraction": 0.2614513981358189, + "actual_fraction_std": 0.009142650655804153 + }, + { + "budget": 0.29, + "metric": 0.14133738097533935, + "metric_std": 0.0038317079179665966, + "actual_fraction": 0.2746338215712384, + "actual_fraction_std": 0.006769207525853111 + }, + { + "budget": 0.3, + "metric": 0.1452378414592858, + "metric_std": 0.0011714444102404727, + "actual_fraction": 0.2874167776298269, + "actual_fraction_std": 0.00543232387772609 + }, + { + "budget": 0.31, + "metric": 0.1482485204581232, + "metric_std": 0.0027491214265008323, + "actual_fraction": 0.30386151797603195, + "actual_fraction_std": 0.0074339520545648454 + }, + { + "budget": 0.32, + "metric": 0.15180996450426054, + "metric_std": 0.0012639282991042074, + "actual_fraction": 0.3182423435419441, + "actual_fraction_std": 0.008109805402849427 + }, + { + "budget": 0.33, + "metric": 0.15391707148655556, + "metric_std": 0.0016367264874463075, + "actual_fraction": 0.33242343541944075, + "actual_fraction_std": 0.006660285817003028 + }, + { + "budget": 0.34, + "metric": 0.15552874619666573, + "metric_std": 0.0018998534985012006, + "actual_fraction": 0.3454061251664447, + "actual_fraction_std": 0.006925381424205504 + }, + { + "budget": 0.35, + "metric": 0.1572863431298185, + "metric_std": 0.0012493326955551286, + "actual_fraction": 0.3528628495339547, + "actual_fraction_std": 0.0059873854023318645 + }, + { + "budget": 0.36, + "metric": 0.1588262390068837, + "metric_std": 0.0018451909930928458, + "actual_fraction": 0.36245006657789614, + "actual_fraction_std": 0.004995560980640248 + }, + { + "budget": 0.37, + "metric": 0.16061597388666699, + "metric_std": 0.002111219721562653, + "actual_fraction": 0.37230359520639145, + "actual_fraction_std": 0.005128010913424297 + }, + { + "budget": 0.38, + "metric": 0.16299204162438788, + "metric_std": 0.0027890262665302423, + "actual_fraction": 0.3833555259653795, + "actual_fraction_std": 0.002690027143919697 + }, + { + "budget": 0.39, + "metric": 0.1647692964208168, + "metric_std": 0.0025760617805828828, + "actual_fraction": 0.3935419440745672, + "actual_fraction_std": 0.004246135491722606 + }, + { + "budget": 0.4, + "metric": 0.16648735573724643, + "metric_std": 0.0028816308024561675, + "actual_fraction": 0.4021304926764314, + "actual_fraction_std": 0.0035930481266516134 + }, + { + "budget": 0.41, + "metric": 0.16815956622631753, + "metric_std": 0.002677350337732343, + "actual_fraction": 0.4133821571238349, + "actual_fraction_std": 0.0027187106510113772 + }, + { + "budget": 0.42, + "metric": 0.17082987003679523, + "metric_std": 0.0024971554789433364, + "actual_fraction": 0.42383488681757653, + "actual_fraction_std": 0.0032223289055472995 + }, + { + "budget": 0.43, + "metric": 0.17445915723634428, + "metric_std": 0.0018440319000973275, + "actual_fraction": 0.43308921438082554, + "actual_fraction_std": 0.00463063506765764 + }, + { + "budget": 0.44, + "metric": 0.17672187979427834, + "metric_std": 0.002161414274306387, + "actual_fraction": 0.44460719041278296, + "actual_fraction_std": 0.001554276635143634 + }, + { + "budget": 0.45, + "metric": 0.17868405849402852, + "metric_std": 0.002696402388132837, + "actual_fraction": 0.45292942743009323, + "actual_fraction_std": 0.0027611772774071624 + }, + { + "budget": 0.46, + "metric": 0.18186972763680034, + "metric_std": 0.0023560520568902186, + "actual_fraction": 0.4645805592543275, + "actual_fraction_std": 0.0076150969886087634 + }, + { + "budget": 0.47, + "metric": 0.18402979113404502, + "metric_std": 0.002960502177619094, + "actual_fraction": 0.474034620505992, + "actual_fraction_std": 0.00826880316136766 + }, + { + "budget": 0.48, + "metric": 0.1879588782134371, + "metric_std": 0.0016231551218311795, + "actual_fraction": 0.48761651131824235, + "actual_fraction_std": 0.0034305387557891356 + }, + { + "budget": 0.49, + "metric": 0.1892116621722101, + "metric_std": 0.0015470404253178165, + "actual_fraction": 0.49860186418109187, + "actual_fraction_std": 0.004826323488532483 + }, + { + "budget": 0.5, + "metric": 0.19028993446068024, + "metric_std": 0.001771482129586688, + "actual_fraction": 0.5104527296937417, + "actual_fraction_std": 0.008284200695061769 + }, + { + "budget": 0.51, + "metric": 0.1933749246589817, + "metric_std": 0.004446230010308763, + "actual_fraction": 0.5245006657789613, + "actual_fraction_std": 0.004469901315009777 + }, + { + "budget": 0.52, + "metric": 0.19609751278676094, + "metric_std": 0.003128686274802688, + "actual_fraction": 0.5326897470039947, + "actual_fraction_std": 0.007639795450633948 + }, + { + "budget": 0.53, + "metric": 0.19647617867802097, + "metric_std": 0.004621457686055786, + "actual_fraction": 0.5416111850865513, + "actual_fraction_std": 0.012511065561229108 + }, + { + "budget": 0.54, + "metric": 0.19949766844517824, + "metric_std": 0.004933572489944255, + "actual_fraction": 0.5526631158455393, + "actual_fraction_std": 0.012136549679704938 + }, + { + "budget": 0.55, + "metric": 0.20151716762716365, + "metric_std": 0.005912466136203729, + "actual_fraction": 0.5621171770972037, + "actual_fraction_std": 0.010494205583857056 + }, + { + "budget": 0.56, + "metric": 0.2043272136170414, + "metric_std": 0.006442246735551964, + "actual_fraction": 0.5716378162450066, + "actual_fraction_std": 0.0075339001277804525 + }, + { + "budget": 0.57, + "metric": 0.20687330715108404, + "metric_std": 0.005777066673685734, + "actual_fraction": 0.5803595206391478, + "actual_fraction_std": 0.008294227165058558 + }, + { + "budget": 0.58, + "metric": 0.20903301231055624, + "metric_std": 0.006137938358809439, + "actual_fraction": 0.5896138482023968, + "actual_fraction_std": 0.00909038219875367 + }, + { + "budget": 0.59, + "metric": 0.21083190772133947, + "metric_std": 0.006595824927195377, + "actual_fraction": 0.5981358189081225, + "actual_fraction_std": 0.007359041315688661 + }, + { + "budget": 0.6, + "metric": 0.2133602514504583, + "metric_std": 0.008283782848480536, + "actual_fraction": 0.6090545938748335, + "actual_fraction_std": 0.006763475419014788 + }, + { + "budget": 0.61, + "metric": 0.21545376141274972, + "metric_std": 0.00834009046562481, + "actual_fraction": 0.6203062583222371, + "actual_fraction_std": 0.005533380230570608 + }, + { + "budget": 0.62, + "metric": 0.21769308739540877, + "metric_std": 0.008590782107442923, + "actual_fraction": 0.6292942743009322, + "actual_fraction_std": 0.009328630050071743 + }, + { + "budget": 0.63, + "metric": 0.21941953645675105, + "metric_std": 0.00688146806245296, + "actual_fraction": 0.6386817576564581, + "actual_fraction_std": 0.0066519614970184495 + }, + { + "budget": 0.64, + "metric": 0.2237974044947188, + "metric_std": 0.009166154939370825, + "actual_fraction": 0.6499999999999999, + "actual_fraction_std": 0.006941364324355976 + }, + { + "budget": 0.65, + "metric": 0.22731516126069806, + "metric_std": 0.008362133777463183, + "actual_fraction": 0.661185086551265, + "actual_fraction_std": 0.005786917642465749 + }, + { + "budget": 0.66, + "metric": 0.23017933928827222, + "metric_std": 0.007762440457825192, + "actual_fraction": 0.6695073235685752, + "actual_fraction_std": 0.004877711322388908 + }, + { + "budget": 0.67, + "metric": 0.23440737834305594, + "metric_std": 0.0071886429478643455, + "actual_fraction": 0.681358189081225, + "actual_fraction_std": 0.008066647792735538 + }, + { + "budget": 0.68, + "metric": 0.23828697117504963, + "metric_std": 0.008043492735662988, + "actual_fraction": 0.6952729693741678, + "actual_fraction_std": 0.009814878131947303 + }, + { + "budget": 0.69, + "metric": 0.2431560758210498, + "metric_std": 0.00515303043083075, + "actual_fraction": 0.7062583222370173, + "actual_fraction_std": 0.009763938451658549 + }, + { + "budget": 0.7, + "metric": 0.24698824402282776, + "metric_std": 0.005512516344108077, + "actual_fraction": 0.7173768308921439, + "actual_fraction_std": 0.00793716312754087 + }, + { + "budget": 0.71, + "metric": 0.2509853554404714, + "metric_std": 0.006296533825393029, + "actual_fraction": 0.7284287616511318, + "actual_fraction_std": 0.0076831872921113405 + }, + { + "budget": 0.72, + "metric": 0.2539698760800331, + "metric_std": 0.005056918424422834, + "actual_fraction": 0.7372836218375499, + "actual_fraction_std": 0.008320239440688137 + }, + { + "budget": 0.73, + "metric": 0.2579573294050026, + "metric_std": 0.003996891543539035, + "actual_fraction": 0.7496671105193076, + "actual_fraction_std": 0.0038534743351498453 + }, + { + "budget": 0.74, + "metric": 0.2605677615271045, + "metric_std": 0.004147655961081844, + "actual_fraction": 0.7575898801597869, + "actual_fraction_std": 0.004355639742688699 + }, + { + "budget": 0.75, + "metric": 0.26345468740096284, + "metric_std": 0.0030783417981880973, + "actual_fraction": 0.764780292942743, + "actual_fraction_std": 0.0037559021857705307 + }, + { + "budget": 0.76, + "metric": 0.26694916187527223, + "metric_std": 0.002545735100635062, + "actual_fraction": 0.7735019973368841, + "actual_fraction_std": 0.0026063354368925954 + }, + { + "budget": 0.77, + "metric": 0.26943299940777915, + "metric_std": 0.0019235101886531149, + "actual_fraction": 0.7826231691078562, + "actual_fraction_std": 0.0006657789613848109 + }, + { + "budget": 0.78, + "metric": 0.2701965998746365, + "metric_std": 0.000531407297494767, + "actual_fraction": 0.7884154460719042, + "actual_fraction_std": 0.0017551832658407583 + }, + { + "budget": 0.79, + "metric": 0.26999703856411594, + "metric_std": 0.00016465752191768785, + "actual_fraction": 0.7949400798934754, + "actual_fraction_std": 0.004151119811364803 + }, + { + "budget": 0.8, + "metric": 0.2693772092318031, + "metric_std": 0.00021884886538678296, + "actual_fraction": 0.8051930758988016, + "actual_fraction_std": 0.004025072969425271 + }, + { + "budget": 0.81, + "metric": 0.26908052768240653, + "metric_std": 0.0012648662993778596, + "actual_fraction": 0.8149134487350199, + "actual_fraction_std": 0.005884710229581791 + }, + { + "budget": 0.82, + "metric": 0.2687928747726566, + "metric_std": 0.0014085436199497406, + "actual_fraction": 0.8222370173102529, + "actual_fraction_std": 0.004236991365298884 + }, + { + "budget": 0.83, + "metric": 0.2681715612110944, + "metric_std": 0.0016774329253255475, + "actual_fraction": 0.8297603195739015, + "actual_fraction_std": 0.003971024090132242 + }, + { + "budget": 0.84, + "metric": 0.2684861065742222, + "metric_std": 0.0019345157571662689, + "actual_fraction": 0.8400133155792278, + "actual_fraction_std": 0.002073556791264638 + }, + { + "budget": 0.85, + "metric": 0.2685444078130374, + "metric_std": 0.0015655072598886844, + "actual_fraction": 0.8535286284953395, + "actual_fraction_std": 0.00461865410303316 + }, + { + "budget": 0.86, + "metric": 0.268523250702051, + "metric_std": 0.0015739910262644573, + "actual_fraction": 0.863382157123835, + "actual_fraction_std": 0.005051810376855917 + }, + { + "budget": 0.87, + "metric": 0.2680050870447393, + "metric_std": 0.001226374340422168, + "actual_fraction": 0.8730359520639148, + "actual_fraction_std": 0.006559696813718128 + }, + { + "budget": 0.88, + "metric": 0.26801686085100285, + "metric_std": 0.001589813428802072, + "actual_fraction": 0.8847536617842875, + "actual_fraction_std": 0.0031527990659258737 + }, + { + "budget": 0.89, + "metric": 0.2675172340354402, + "metric_std": 0.0017265409308145734, + "actual_fraction": 0.8916111850865513, + "actual_fraction_std": 0.004699522367930972 + }, + { + "budget": 0.9, + "metric": 0.26760995882709687, + "metric_std": 0.0015557865010918871, + "actual_fraction": 0.9003994673768307, + "actual_fraction_std": 0.0059427998456577395 + }, + { + "budget": 0.91, + "metric": 0.2697492822164804, + "metric_std": 0.0016269812141255678, + "actual_fraction": 0.9124500665778962, + "actual_fraction_std": 0.005310604571752274 + }, + { + "budget": 0.92, + "metric": 0.2706931509986381, + "metric_std": 0.0019100469573241517, + "actual_fraction": 0.9245006657789613, + "actual_fraction_std": 0.004650931876583994 + }, + { + "budget": 0.93, + "metric": 0.27084298466220647, + "metric_std": 0.0017679582817421609, + "actual_fraction": 0.933155792276964, + "actual_fraction_std": 0.001637599717210252 + }, + { + "budget": 0.94, + "metric": 0.2707626231135413, + "metric_std": 0.0022954713487111722, + "actual_fraction": 0.9427430093209056, + "actual_fraction_std": 0.0029305620608620266 + }, + { + "budget": 0.95, + "metric": 0.2718280280294131, + "metric_std": 0.0015100119533920692, + "actual_fraction": 0.9514647137150467, + "actual_fraction_std": 0.004644971433437189 + }, + { + "budget": 0.96, + "metric": 0.2718580489602269, + "metric_std": 0.0008079712301308884, + "actual_fraction": 0.9608521970705726, + "actual_fraction_std": 0.005261339041313002 + }, + { + "budget": 0.97, + "metric": 0.27167363563537017, + "metric_std": 0.0009442643109326534, + "actual_fraction": 0.9693075898801599, + "actual_fraction_std": 0.0029531631313204974 + }, + { + "budget": 0.98, + "metric": 0.27080372686256726, + "metric_std": 0.0005658706168957813, + "actual_fraction": 0.9804926764314248, + "actual_fraction_std": 0.0013644408476644046 + }, + { + "budget": 0.99, + "metric": 0.2696392822152621, + "metric_std": 0.0009979073675770906, + "actual_fraction": 0.990878828229028, + "actual_fraction_std": 0.001536348969269532 + }, + { + "budget": 1.0, + "metric": 0.27001009067886245, + "metric_std": 0.0, + "actual_fraction": 1.0, + "actual_fraction_std": 0.0 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 1681, + "rescuable": 815, + "harmful": 185, + "both_wrong": 123, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 7329, + "rescuable": 3461, + "harmful": 676, + "both_wrong": 550, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.07641657905799261, + "test": 0.07887397871043929 + }, + "large": { + "validation": 0.25448626222520154, + "test": 0.27001009067886245 + }, + "learned_minus_confidence_test": 0.0003105963010436863, + "learned_minus_small_test": 0.18962093042573586, + "learned_minus_large_test": -0.001515181542687316, + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "selected_method": "four_difference", + "historical_run": "shared_12_rtdetr_x_fp32_Qwen3.8" + } + ], + "summary": { + "learned_beats_confidence": 13, + "learned_beats_both": 8, + "learned_beats_large": 12 + }, + "multiseed_protocol": { + "selected_method": "four_difference", + "selection": "task-balanced validation Score25 mean minus sample standard deviation over five router seeds", + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "comparison": [ + { + "method": "error", + "validation_seed_scores": [ + 0.652854712535587, + 0.653201861091734, + 0.653549705081995, + 0.6529296233301471, + 0.6533734477104217 + ], + "validation_mean": 0.653181869949977, + "validation_std": 0.00029285862604605935, + "selection_value": 0.6528890113239308 + }, + { + "method": "four_difference", + "validation_seed_scores": [ + 0.6659279432360811, + 0.6666052399411169, + 0.6664902552160231, + 0.666460246201781, + 0.6671766800294149 + ], + "validation_mean": 0.6665320729248834, + "validation_std": 0.0004451831677229395, + "selection_value": 0.6660868897571605 + }, + { + "method": "four_ratio", + "validation_seed_scores": [ + 0.6669800355113474, + 0.6654694961331916, + 0.6669276041336858, + 0.6664081580190494, + 0.6645222674050018 + ], + "validation_mean": 0.6660615122404552, + "validation_std": 0.001052820164752406, + "selection_value": 0.6650086920757028 + }, + { + "method": "gain", + "validation_seed_scores": [ + 0.6656365114137299, + 0.6665939365548585, + 0.6663926476912081, + 0.6648075719596137, + 0.6656891621835576 + ], + "validation_mean": 0.6658239659605936, + "validation_std": 0.0007075700773489459, + "selection_value": 0.6651163958832447 + }, + { + "method": "route", + "validation_seed_scores": [ + 0.6522091985724042, + 0.6522614154026475, + 0.6528619080142624, + 0.6518457956309271, + 0.6514639634291153 + ], + "validation_mean": 0.6521284562098713, + "validation_std": 0.0005204797508024464, + "selection_value": 0.6516079764590688 + } + ], + "selector_reads_test_records": false, + "historical_pilot_selected_method": "four_ratio", + "rule_amendment": "User changed selection criterion after the seed-42 pilot and partial CUB test results had been reported. The final numerical criterion uses validation records only; this is not a prospectively untouched test set." + }, + "summary_comparison": "Five-seed mean metrics; differences within 1e-12 in raw fractions are treated as ties. CUB/ResNet/Q has exactly two rescues and two harms across five runs." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/joint_MiniCPM_comparison_20260910.json b/releases/20260913_lr/paper_data/joint_MiniCPM_comparison_20260910.json new file mode 100644 index 0000000000000000000000000000000000000000..d6562c709e03367006f3212a57bfdc7387de3957 --- /dev/null +++ b/releases/20260913_lr/paper_data/joint_MiniCPM_comparison_20260910.json @@ -0,0 +1,58 @@ +[ + { + "expert": "cub", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.7119433897134967, + "multi_task_test_fixed25": 0.7079737659647911, + "delta_multi_minus_single": -0.003969623748705575 + }, + { + "expert": "glsim", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.759233690024163, + "multi_task_test_fixed25": 0.7745944080082844, + "delta_multi_minus_single": 0.01536071798412142 + }, + { + "expert": "nlvr2", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.8346846523642104, + "multi_task_test_fixed25": 0.8309535768099304, + "delta_multi_minus_single": -0.0037310755542799834 + }, + { + "expert": "vilt", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.8046925450240368, + "multi_task_test_fixed25": 0.802611752887996, + "delta_multi_minus_single": -0.002080792136040732 + }, + { + "expert": "groundingdino", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.3598105663159686, + "multi_task_test_fixed25": 0.37004792558772653, + "delta_multi_minus_single": 0.010237359271757951 + }, + { + "expert": "instancevg", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.6394237586138446, + "multi_task_test_fixed25": 0.6243654822335025, + "delta_multi_minus_single": -0.015058276380342073 + }, + { + "expert": "yolo26x", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.18032734231886507, + "multi_task_test_fixed25": 0.1836164473038217, + "delta_multi_minus_single": 0.003289104984956631 + }, + { + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "single_task_test_fixed25": 0.10367040308754236, + "multi_task_test_fixed25": 0.11260360812088349, + "delta_multi_minus_single": 0.008933205033341135 + } +] \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/joint_Qwen3.8_comparison_20260910.json b/releases/20260913_lr/paper_data/joint_Qwen3.8_comparison_20260910.json new file mode 100644 index 0000000000000000000000000000000000000000..e5487d71fdc8487a4580f45e2ae7a63bffdc98f4 --- /dev/null +++ b/releases/20260913_lr/paper_data/joint_Qwen3.8_comparison_20260910.json @@ -0,0 +1,58 @@ +[ + { + "expert": "cub", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.7619951674145667, + "multi_task_test_fixed25": 0.7782188470831896, + "delta_multi_minus_single": 0.016223679668622815 + }, + { + "expert": "glsim", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.8301691404901622, + "multi_task_test_fixed25": 0.8448394891266828, + "delta_multi_minus_single": 0.014670348636520614 + }, + { + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.8372677046710196, + "multi_task_test_fixed25": 0.8337518834756403, + "delta_multi_minus_single": -0.0035158211953792984 + }, + { + "expert": "vilt", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.8045490421181029, + "multi_task_test_fixed25": 0.8067733371600775, + "delta_multi_minus_single": 0.0022242950419746332 + }, + { + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.4223974137197629, + "multi_task_test_fixed25": 0.45991549215892014, + "delta_multi_minus_single": 0.03751807843915722 + }, + { + "expert": "instancevg", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.7224853245611548, + "multi_task_test_fixed25": 0.7053001729858492, + "delta_multi_minus_single": -0.017185151575305535 + }, + { + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.19009376433100783, + "multi_task_test_fixed25": 0.19603467084956394, + "delta_multi_minus_single": 0.0059409065185561105 + }, + { + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "single_task_test_fixed25": 0.10483176081963008, + "multi_task_test_fixed25": 0.12038495915476813, + "delta_multi_minus_single": 0.015553198335138047 + } +] \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/layout_verification.json b/releases/20260913_lr/paper_data/layout_verification.json new file mode 100644 index 0000000000000000000000000000000000000000..5fcd538e4ee47c5ead19602b1a6b9f6a27ca963b --- /dev/null +++ b/releases/20260913_lr/paper_data/layout_verification.json @@ -0,0 +1,29 @@ +{ + "old_distribution": { + "width_pt": 523.1995239257812, + "height_pt": 309.7997741699219, + "height_at_7_16_in": 4.239618504261674 + }, + "old_budget": { + "width_pt": 526.7994995117188, + "height_pt": 237.7997589111328, + "height_at_7_16_in": 3.2320575007794505 + }, + "combined": { + "width_pt": 515.52001953125, + "height_pt": 324.0, + "height_at_7_16_in": 4.499999829510743 + }, + "reduction_percent": 39.77255134624947, + "old_pages": 16, + "new_pages": 14, + "new_figure_and_section_pages": [ + 6, + 7 + ], + "changed_sections": [ + "sections\\appendix.tex", + "sections\\experiments.tex" + ], + "figure1_identical": true +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/lr_control_curves.json b/releases/20260913_lr/paper_data/lr_control_curves.json new file mode 100644 index 0000000000000000000000000000000000000000..2215b83c9bb68887d71ae98492fe2afa278c119b --- /dev/null +++ b/releases/20260913_lr/paper_data/lr_control_curves.json @@ -0,0 +1 @@ +[{"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.25647221263375, 81.18743527787366, 81.11839834311357, 81.0666206420435, 81.01484294097342, 80.84225060407319, 80.70417673455299, 80.66965826717293, 80.60062133241284, 80.32447359337246, 79.96202968588196, 79.7894373489817, 79.54780807732136, 79.23714187090093, 78.90921643079048, 78.56403175698999, 78.32240248532966, 77.92544011045909, 77.51121850189851, 77.11425612702796, 76.9244045564377, 76.52744218156714, 75.99240593717639, 75.44011045909562, 74.99137038315499, 74.62892647566449, 74.0766309975837, 73.59337245426303, 73.09285467725233, 72.69589230238178, 72.143596824301, 71.57404211253021, 71.09078356920952, 70.59026579219882, 70.03797031411806, 69.48567483603728, 68.95063859164652, 68.32930617880567, 67.75975146703487, 67.19019675526407, 66.60338280980325, 65.94753192958233, 65.44701415257163, 64.87745944080083, 64.08353469105973, 63.375906109768735, 62.7890921643079, 62.115982050396966, 61.37383500172593, 60.68346565412496, 59.97583707283397, 59.285467725233, 58.62961684501209, 57.818432861580945, 57.05902657921988, 56.316879530548846, 55.591991715567836, 54.74628926475664, 53.96962374870555, 53.141180531584396, 52.26095961339317, 51.39799792889196, 50.62133241284087, 49.86192613047981, 48.9989644459786, 48.08422506040732, 47.204004142216085, 46.30652399033483, 45.3572661373835, 44.45978598550225, 43.56230583362099, 42.64756644804971, 41.69830859509838, 40.766309975837075, 39.81705212288574, 38.90231273731447, 37.93579565067311, 37.03831549879186, 36.10631687953055, 35.2088367276493, 34.29409734207801, 33.36209872281671, 32.41284086986538, 31.446323783224027, 30.479806696582678, 29.49603037625129, 28.546772523299968, 27.54573696927857, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.27373144632378, 81.10113910942354, 80.98032447359338, 81.03210217466345, 80.73869520193303, 80.4970659302727, 80.08284432171212, 79.82395581636176, 79.59958577839144, 79.27166033828098, 78.96099413186055, 78.78840179496031, 78.54677252329996, 78.18432861580946, 77.92544011045909, 77.83914394200899, 77.47670003451846, 77.14877459440801, 76.82084915429755, 76.47566448049706, 76.14773904038661, 75.75077666551606, 75.37107352433551, 75.16396272005524, 74.85329651363479, 74.47359337245426, 74.14566793234381, 73.81774249223335, 73.33448394891266, 72.98929927511219, 72.5750776665516, 72.09181912323092, 71.67759751467035, 71.24611667241975, 70.76285812909907, 70.43493268898861, 69.7790818087677, 69.10597169485675, 68.62271315153608, 68.0704176734553, 67.51812219537453, 66.84501208146358, 66.25819813600276, 65.61960648947188, 65.15360717984122, 64.70486710390058, 64.18709009319986, 63.66931308249914, 63.099758370728345, 62.44390749050742, 61.66724197445633, 61.04590956161546, 60.493614083534695, 59.803244735933724, 59.31998619261305, 58.83672764929238, 58.33620987228167, 57.73213669313083, 57.248878149810146, 56.748360372799446, 56.265101829478766, 55.902657921988265, 55.55747324818778, 55.195029340697275, 54.81532619951675, 54.47014152571625, 54.15947531929582, 53.79703141180532, 53.26199516741457, 52.67518122195375, 51.96755264066275, 51.31170176044184, 50.742147048671036, 49.94822229892993, 49.22333448394892, 48.51570590265793, 47.63548498446669, 46.910597169485676, 46.11667241974457, 45.270969968933386, 44.373489817052125, 43.51052813255092, 42.64756644804971, 41.750086296168455, 40.83534691059717, 39.86882982395582, 38.97134967207456, 38.004832585433206, 37.072833966171906, 36.10631687953055, 35.157059026579226, 34.22506040731791, 33.258543320676566, 32.257507766655166, 31.32550914739386, 30.4280289955126, 29.461511908871252, 28.512254055919918, 27.52847773558854, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.46358301691406, 81.23921297894374, 81.18743527787366, 81.01484294097342, 80.7904729030031, 80.63513979979288, 80.35899206075251, 80.01380738695202, 79.68588194684156, 79.39247497411115, 79.20262340352089, 79.04729030031066, 78.68484639282016, 78.42595788746979, 77.99447704521918, 77.52847773558854, 77.20055229547809, 76.95892302381775, 76.63099758370728, 76.33759061097687, 75.88885053503624, 75.66448049706594, 75.25025888850536, 74.97411114946496, 74.64618570935451, 74.18018639972385, 73.81774249223335, 73.40352088367277, 73.04107697618227, 72.50604073179151, 72.12633759061097, 71.69485674836038, 71.21159820503969, 70.72833966171902, 70.22782188470832, 69.76182257507767, 69.15774939592683, 68.60545391784605, 68.1394546082154, 67.72523299965482, 67.10390058681395, 66.46530894028305, 65.82671729375215, 65.27442181567137, 64.82568173973075, 64.15257162581982, 63.4967207455989, 62.85812909906801, 62.15050051777702, 61.52916810493614, 60.925094925785295, 60.2865032792544, 59.630652399033494, 58.92302381774249, 58.33620987228167, 57.818432861580945, 57.21435968243011, 56.748360372799446, 56.29962029685882, 55.91991715567829, 55.41939937866759, 55.02243700379703, 54.52191922678633, 54.073179150845704, 53.74525371073524, 53.348291335864694, 52.847773558853994, 52.46807041767345, 52.019330341732825, 51.656886434242324, 51.08733172247153, 50.569554711770806, 50.0, 49.395926820849155, 48.72281670693822, 47.980669658267175, 47.23852260959613, 46.51363479461512, 45.771487745944086, 44.89126682775285, 44.080082844321716, 43.251639627200554, 42.42319641007939, 41.542975491888164, 40.680013807386956, 39.74801518812565, 38.86779426993442, 37.953054884363134, 37.021056265101834, 36.10631687953055, 35.157059026579226, 34.24231964100794, 33.29306178805661, 32.30928546772523, 31.342768381083886, 30.35899206075251, 29.39247497411115, 28.460476354849852, 27.52847773558854, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.25647221263375, 81.18743527787366, 81.11839834311357, 81.0666206420435, 81.01484294097342, 80.84225060407319, 80.70417673455299, 80.66965826717293, 80.60062133241284, 80.32447359337246, 79.96202968588196, 79.7894373489817, 79.54780807732136, 79.23714187090093, 78.90921643079048, 78.56403175698999, 78.32240248532966, 77.92544011045909, 77.51121850189851, 77.11425612702796, 76.9244045564377, 76.52744218156714, 75.99240593717639, 75.44011045909562, 74.99137038315499, 74.62892647566449, 74.0766309975837, 73.59337245426303, 73.09285467725233, 72.69589230238178, 72.143596824301, 71.57404211253021, 71.09078356920952, 70.59026579219882, 70.03797031411806, 69.48567483603728, 68.95063859164652, 68.32930617880567, 67.75975146703487, 67.19019675526407, 66.60338280980325, 65.94753192958233, 65.44701415257163, 64.87745944080083, 64.08353469105973, 63.375906109768735, 62.7890921643079, 62.115982050396966, 61.37383500172593, 60.68346565412496, 59.97583707283397, 59.285467725233, 58.62961684501209, 57.818432861580945, 57.05902657921988, 56.316879530548846, 55.591991715567836, 54.74628926475664, 53.96962374870555, 53.141180531584396, 52.26095961339317, 51.39799792889196, 50.62133241284087, 49.86192613047981, 48.9989644459786, 48.08422506040732, 47.204004142216085, 46.30652399033483, 45.3572661373835, 44.45978598550225, 43.56230583362099, 42.64756644804971, 41.69830859509838, 40.766309975837075, 39.81705212288574, 38.90231273731447, 37.93579565067311, 37.03831549879186, 36.10631687953055, 35.2088367276493, 34.29409734207801, 33.36209872281671, 32.41284086986538, 31.446323783224027, 30.479806696582678, 29.49603037625129, 28.546772523299968, 27.54573696927857, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2026, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.44632378322403, 81.46358301691406, 81.46358301691406, 81.48084225060408, 81.46358301691406, 81.48084225060408, 81.46358301691406, 81.44632378322403, 81.429064549534, 81.429064549534, 81.48084225060408, 81.48084225060408, 81.37728684846392, 81.32550914739386, 81.25647221263375, 81.08387987573352, 81.01484294097342, 80.80773213669313, 80.63513979979288, 80.37625129444253, 80.20365895754229, 80.03106662064205, 79.77217811529168, 79.47877114256127, 79.20262340352089, 78.84017949603039, 78.49499482222988, 78.01173627890921, 77.59751467034863, 77.16603382809804, 76.94166379012772, 76.64825681739731, 76.14773904038661, 75.7852951328961, 75.57818432861582, 75.0604073179151, 74.59440800828443, 74.04211253020367, 73.558853986883, 73.02381774249224, 72.48878149810149, 71.95374525371074, 71.48774594408009, 70.93545046599931, 70.2795995857784, 69.72730410769762, 69.278564031757, 68.570935450466, 67.94960303762512, 67.39730755954436, 66.72419744563342, 66.08560579910252, 65.44701415257163, 64.998274076631, 64.4287193648602, 63.7901277183293, 63.16879530548844, 62.59924059371764, 61.8570935450466, 61.23576113220574, 60.56265101829479, 59.83776320331378, 59.16465308940283, 58.45702450811184, 57.76665516051087, 57.07628581290991, 56.3513979979289, 55.60925094925786, 54.8498446668968, 54.09043838453573, 53.24473593372454, 52.46807041767345, 51.67414566793235, 50.880220918191235, 49.98274076630998, 49.119779081808765, 48.20503969623749, 47.273041076976185, 46.358301691404904, 45.44356230583362, 44.63237832240249, 43.734898170521234, 42.7856403175699, 41.87090093199862, 40.88712461166724, 40.00690369347601, 39.04038660683466, 38.142906454953405, 37.17638936831205, 36.19261304798067, 35.260614428719364, 34.31135657576804, 33.31032102174664, 32.39558163617536, 31.446323783224027, 30.479806696582678, 29.530548843631344, 28.52951328960994, 27.52847773558854, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2026, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.48084225060408, 81.46358301691406, 81.48084225060408, 81.53261995167415, 81.53261995167415, 81.51536071798412, 81.48084225060408, 81.4981014842941, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.30824991370383, 81.18743527787366, 81.10113910942354, 81.01484294097342, 80.9975837072834, 80.80773213669313, 80.63513979979288, 80.41076976182258, 80.16914049016223, 79.87573351743183, 79.63410424577148, 79.46151190887124, 79.13358646876078, 78.80566102865033, 78.49499482222988, 78.27062478425958, 77.89092164307905, 77.45944080082845, 77.11425612702796, 76.51018294787711, 76.02692440455644, 75.59544356230583, 74.9223334483949, 74.45633413876425, 73.93855712806352, 73.38626164998274, 72.7821884708319, 72.24715222644114, 71.74663444943045, 71.09078356920952, 70.55574732481877, 70.003451846738, 69.38211943389715, 68.91612012426648, 68.24301001035555, 67.74249223334485, 67.10390058681395, 66.32723507076285, 65.65412495685192, 65.0673110113911, 64.41146013117019, 63.72109078356921, 63.01346220227821, 62.2885743872972, 61.68450120814636, 60.994131860545394, 60.25198481187435, 59.45806006213324, 58.664135312392126, 57.852951328961, 57.12806351397999, 56.24784259578875, 55.402140144977565, 54.53917846047636, 53.81429064549535, 52.951328960994125, 52.122885743872985, 51.32896099413187, 50.53503624439075, 49.75837072833966, 49.06800138073869, 48.15326199516742, 47.3593372454263, 46.47911632723507, 45.63341387642389, 44.787711425612706, 43.872972040041425, 43.13082499137039, 42.371418709009326, 41.473938557128065, 40.61097687262686, 39.67897825336556, 38.83327580255437, 37.953054884363134, 37.17638936831205, 36.26164998274077, 35.450465999309635, 34.70831895063859, 33.84535726613739, 33.01691404901623, 32.22298929927511, 31.360027614773912, 30.566102865032796, 29.875733517431826, 29.030031066620644, 28.14981014842941, 27.321366931308255, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.37728684846392, 81.23921297894374, 81.01484294097342, 80.91128753883328, 80.56610286503279, 80.4280289955126, 80.06558508802209, 79.75491888160165, 79.4960303762513, 78.9955125992406, 78.63306869175008, 78.32240248532966, 78.04625474628926, 77.52847773558854, 77.25232999654816, 76.80358992060754, 76.61373835001726, 76.16499827407664, 75.85433206765619, 75.49188816016569, 74.85329651363479, 74.50811183983431, 74.2147048671039, 73.73144632378322, 73.33448394891266, 72.85122540559199, 72.48878149810149, 71.90196755264067, 71.52226441146013, 71.12530203658957, 70.52122885743873, 70.03797031411806, 69.5719710044874, 69.17500862961685, 68.7090093199862, 68.15671384190543, 67.62167759751468, 66.98308595098378, 66.51708664135313, 65.96479116327235, 65.2916810493614, 64.68760787021056, 64.29064549534, 63.72109078356921, 63.117017604418365, 62.58198136002762, 62.08146358301691, 61.51190887124611, 60.8560579910252, 60.14842940973422, 59.527096996893334, 58.8539868829824, 58.301691404901625, 57.6458405246807, 57.093545046599935, 56.523990334829136, 56.00621332412841, 55.29858474283742, 54.57369692785641, 54.055919917155684, 53.31377286848464, 52.74421815671384, 52.157404211253024, 51.48429409734207, 50.880220918191235, 50.189851570590264, 49.53400069036935, 48.8608905764584, 48.17052122885744, 47.497411114946495, 46.87607870210563, 46.32378322402486, 45.68519157749396, 44.994822229892996, 44.40800828443217, 43.803935105281326, 43.096306523990336, 42.3541594753193, 41.59475319295824, 40.904383845357266, 40.283051432516395, 39.54090438384536, 38.781498101484296, 38.05661028650328, 37.40075940628236, 36.79668622713152, 36.02002071108043, 35.36416983085951, 34.60476354849845, 33.914394200897476, 33.137728684846394, 32.51639627200553, 31.82602692440456, 30.997583707283393, 30.220918191232315, 29.375215740421122, 28.633068691750086, 27.908180876769073, 27.26958923023818, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.429064549534, 81.429064549534, 81.37728684846392, 81.37728684846392, 81.39454608215395, 81.32550914739386, 81.27373144632378, 81.23921297894374, 81.04936140835348, 80.9285467725233, 80.82499137038316, 80.60062133241284, 80.44528822920263, 80.18639972385226, 80.06558508802209, 79.89299275112185, 79.70314118053159, 79.25440110459095, 78.84017949603039, 78.59855022437004, 78.28788401794961, 77.92544011045909, 77.47670003451846, 76.9244045564377, 76.57921988263722, 76.18225750776666, 75.83707283396618, 75.45736969278565, 75.12944425267519, 74.73248187780463, 74.37003797031412, 73.86952019330342, 73.38626164998274, 73.04107697618227, 72.47152226441146, 71.93648602002072, 71.45322747670004, 70.79737659647913, 70.19330341732827, 69.7100448740076, 69.08871246116672, 68.5018985157059, 67.89782533655506, 67.27649292371419, 66.68967897825337, 66.03382809803244, 65.23990334829134, 64.54953400069037, 63.91094235415948, 63.20331377286848, 62.58198136002762, 61.822575077666556, 61.32205730065585, 60.66620642043493, 59.95857783914394, 59.38902312737315, 58.64687607870211, 58.00828443217121, 57.21435968243011, 56.316879530548846, 55.45391784604764, 54.694511563686575, 53.952364515015525, 53.08940283051433, 52.295478080773215, 51.380738695201934, 50.517777010700726, 49.620296858819465, 48.80911287538833, 48.01518812564722, 47.08318950638592, 46.18570935450466, 45.32274767000345, 44.45978598550225, 43.51052813255092, 42.63030721435969, 41.73282706247843, 40.83534691059717, 39.86882982395582, 38.90231273731447, 37.953054884363134, 37.03831549879186, 36.10631687953055, 35.1397997928892, 34.22506040731791, 33.27580255436659, 32.29202623403521, 31.39454608215395, 30.445288229202628, 29.513289609941317, 28.564031756989994, 27.56299620296859, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.34276838108389, 81.25647221263375, 81.11839834311357, 80.84225060407319, 80.68691750086296, 80.4280289955126, 80.20365895754229, 79.9275112185019, 79.70314118053159, 79.44425267518123, 79.18536416983086, 78.97825336555057, 78.65032792544011, 78.39143942008975, 78.06351397997929, 77.82188470831896, 77.44218156713842, 77.14877459440801, 76.78633068691751, 76.47566448049706, 76.07870210562652, 75.69899896444599, 75.33655505695546, 74.88781498101484, 74.5598895409044, 74.33551950293406, 73.95581636175355, 73.54159475319297, 73.02381774249224, 72.67863306869175, 72.2816706938212, 71.79841215050052, 71.40144977562997, 70.7801173627891, 70.17604418363825, 69.65826717293753, 69.03693476009666, 68.58819468415602, 68.01863997238523, 67.5698998964446, 66.91404901622369, 66.39627200552296, 65.74042112530203, 65.0673110113911, 64.39420089748016, 63.738350017259236, 63.18605453917846, 62.66827752847773, 62.02968588194684, 61.23576113220574, 60.56265101829479, 59.87228167069383, 59.216430790472906, 58.61235761132206, 57.95650673110114, 57.38695201933035, 56.83465654124957, 56.23058336209872, 55.60925094925786, 55.07421470486711, 54.59095616154643, 54.17673455298585, 53.74525371073524, 53.192958232654476, 52.88229202623403, 52.46807041767345, 51.96755264066275, 51.48429409734207, 51.10459095616156, 50.673110113910944, 50.15533310321022, 49.63755609250949, 49.06800138073869, 48.4984466689679, 47.894373489817056, 47.23852260959613, 46.58267172937522, 45.926820849154296, 45.288229202623405, 44.54608215395236, 43.7866758715913, 42.95823265447014, 42.12978943734898, 41.30134622022783, 40.472903003106666, 39.558163617535385, 38.62616499827408, 37.76320331377287, 36.84846392820159, 35.91646530894029, 35.001725923369, 34.08698653779773, 33.15498791853642, 32.205730065585094, 31.29099068001381, 30.410769761822575, 29.461511908871252, 28.546772523299968, 27.54573696927857, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.41180531584398, 81.2909906800138, 81.20469451156369, 81.15291681049361, 80.9975837072834, 80.4970659302727, 80.30721435968243, 79.96202968588196, 79.58232654470142, 79.30617880566103, 79.03003106662064, 78.87469796341043, 78.56403175698999, 78.33966171901969, 78.06351397997929, 77.68381083879876, 77.37314463237833, 77.09699689333793, 76.78633068691751, 76.47566448049706, 76.06144287193649, 75.66448049706594, 75.45736969278565, 75.02588885053504, 74.73248187780463, 74.24922333448394, 73.73144632378322, 73.12737314463237, 72.60959613393166, 72.16085605799103, 71.7811529168105, 71.40144977562997, 71.02174663444943, 70.67656196064895, 70.33137728684846, 69.72730410769762, 69.278564031757, 68.7780462547463, 68.1394546082154, 67.60441836382465, 67.05212288574387, 66.43079047290301, 65.68864342423197, 65.01553331032103, 64.46323783224025, 63.7901277183293, 63.20331377286848, 62.4956851915775, 61.822575077666556, 61.23576113220574, 60.64894718674492, 60.06213324128409, 59.423541594753196, 58.83672764929238, 58.2844321712116, 57.6458405246807, 57.024508111839836, 56.48947186744908, 56.05799102519848, 55.5056955471177, 54.97065930272696, 54.504659993096304, 54.193993786675875, 53.88332758025545, 53.46910597169485, 53.08940283051433, 52.60614428719366, 52.071108042802905, 51.69140490162236, 51.20814635830169, 50.690369347600964, 50.20711080428028, 49.63755609250949, 49.05074214704867, 48.3603727994477, 47.618225750776666, 46.841560234725584, 46.18570935450466, 45.39178460476355, 44.63237832240249, 43.76941663790128, 42.95823265447014, 42.181567138419055, 41.266827752847774, 40.45564376941664, 39.54090438384536, 38.72972040041422, 37.866758715913015, 36.90024162927166, 36.089057645840526, 35.174318260269246, 34.22506040731791, 33.24128408698654, 32.274767000345186, 31.29099068001381, 30.376251294442525, 29.4269934414912, 28.460476354849852, 27.54573696927857, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.429064549534, 81.429064549534, 81.37728684846392, 81.37728684846392, 81.39454608215395, 81.32550914739386, 81.27373144632378, 81.23921297894374, 81.04936140835348, 80.9285467725233, 80.82499137038316, 80.60062133241284, 80.44528822920263, 80.18639972385226, 80.06558508802209, 79.89299275112185, 79.70314118053159, 79.25440110459095, 78.84017949603039, 78.59855022437004, 78.28788401794961, 77.92544011045909, 77.47670003451846, 76.9244045564377, 76.57921988263722, 76.18225750776666, 75.83707283396618, 75.45736969278565, 75.12944425267519, 74.73248187780463, 74.37003797031412, 73.86952019330342, 73.38626164998274, 73.04107697618227, 72.47152226441146, 71.93648602002072, 71.45322747670004, 70.79737659647913, 70.19330341732827, 69.7100448740076, 69.08871246116672, 68.5018985157059, 67.89782533655506, 67.27649292371419, 66.68967897825337, 66.03382809803244, 65.23990334829134, 64.54953400069037, 63.91094235415948, 63.20331377286848, 62.58198136002762, 61.822575077666556, 61.32205730065585, 60.66620642043493, 59.95857783914394, 59.38902312737315, 58.64687607870211, 58.00828443217121, 57.21435968243011, 56.316879530548846, 55.45391784604764, 54.694511563686575, 53.952364515015525, 53.08940283051433, 52.295478080773215, 51.380738695201934, 50.517777010700726, 49.620296858819465, 48.80911287538833, 48.01518812564722, 47.08318950638592, 46.18570935450466, 45.32274767000345, 44.45978598550225, 43.51052813255092, 42.63030721435969, 41.73282706247843, 40.83534691059717, 39.86882982395582, 38.90231273731447, 37.953054884363134, 37.03831549879186, 36.10631687953055, 35.1397997928892, 34.22506040731791, 33.27580255436659, 32.29202623403521, 31.39454608215395, 30.445288229202628, 29.513289609941317, 28.564031756989994, 27.56299620296859, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2027, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.44632378322403, 81.46358301691406, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.32550914739386, 81.32550914739386, 81.23921297894374, 81.17017604418363, 81.08387987573352, 80.98032447359338, 80.89402830514325, 80.75595443562305, 80.58336209872282, 80.35899206075251, 80.06558508802209, 79.9275112185019, 79.66862271315154, 79.47877114256127, 79.18536416983086, 78.94373489817053, 78.58129099068002, 78.27062478425958, 77.78736624093891, 77.40766309975837, 77.07973765964792, 76.61373835001726, 76.11322057300656, 75.71625819813602, 75.11218501898516, 74.68070417673457, 74.40455643769417, 73.88677942699344, 73.45529858474285, 72.98929927511219, 72.50604073179151, 72.05730065585088, 71.57404211253021, 70.98722816706939, 70.38315498791854, 69.65826717293753, 69.05419399378667, 68.39834311356576, 67.84604763548498, 67.17293752157406, 66.68967897825337, 66.1373835001726, 65.46427338626165, 64.86020020711081, 64.08353469105973, 63.2896099413186, 62.68553676216776, 61.9951674145668, 61.339316534345876, 60.75250258888505, 60.04487400759406, 59.40628236106317, 58.69865377977218, 58.04280289955126, 57.3524335519503, 56.71384190541941, 56.040731791508456, 55.33310321021747, 54.556437694166384, 53.74525371073524, 52.98584742837418, 52.157404211253024, 51.363479461511915, 50.58681394546083, 49.74111149464964, 48.84363134276838, 47.9979288919572, 47.04867103900587, 46.13393165343459, 45.270969968933386, 44.39074905074215, 43.54504659993096, 42.63030721435969, 41.750086296168455, 40.85260614428719, 39.86882982395582, 38.90231273731447, 38.02209181912323, 37.072833966171906, 36.0717984121505, 35.12254055919917, 34.20780117362789, 33.258543320676566, 32.361063168795305, 31.411805315843978, 30.445288229202628, 29.530548843631344, 28.564031756989994, 27.56299620296859, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2027, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.25647221263375, 81.20469451156369, 81.15291681049361, 81.15291681049361, 81.18743527787366, 81.10113910942354, 80.96306523990336, 80.91128753883328, 80.7904729030031, 80.72143596824301, 80.68691750086296, 80.58336209872282, 80.53158439765275, 80.44528822920263, 80.35899206075251, 80.30721435968243, 80.16914049016223, 80.01380738695202, 79.87573351743183, 79.75491888160165, 79.56506731101139, 79.37521574042113, 79.09906800138074, 78.90921643079048, 78.63306869175008, 78.39143942008975, 78.01173627890921, 77.70107007248879, 77.30410769761824, 76.80358992060754, 76.28581290990681, 75.90610976872627, 75.54366586123577, 75.0604073179151, 74.64618570935451, 74.37003797031412, 73.80048325854332, 73.2827062478426, 72.81670693821194, 72.12633759061097, 71.52226441146013, 70.96996893337936, 70.38315498791854, 69.67552640662755, 69.03693476009666, 68.5018985157059, 67.72523299965482, 67.03486365205384, 66.24093890231273, 65.44701415257163, 64.77390403866067, 64.10079392474974, 63.272350707628576, 62.51294442526751, 61.822575077666556, 61.08042802899551, 60.424577148774596, 59.59613393165344, 58.802209181912325, 57.99102519848118, 57.11080428028996, 56.265101829478766, 55.45391784604764, 54.694511563686575, 53.86606834656542, 53.141180531584396, 52.48532965136348, 51.622367966862285, 50.81118398343114, 50.0, 49.257852951328964, 48.41215050051778, 47.70452191922679, 46.858819468415604, 46.04763548498447, 45.20193303417329, 44.442526751812224, 43.64860200207111, 42.83741801863997, 42.07801173627891, 41.249568519157755, 40.45564376941664, 39.575422851225404, 38.7124611667242, 37.83224024853297, 37.12461166724197, 36.41698308595099, 35.62305833620987, 34.89817052122886, 34.08698653779773, 33.258543320676566, 32.37832240248533, 31.60165688643424, 30.790472903003106, 30.117362789092162, 29.461511908871252, 28.65032792544011, 27.787366240938905, 27.07973765964792, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.23921297894374, 81.18743527787366, 80.96306523990336, 80.70417673455299, 80.46254746289266, 80.25543665861235, 79.8584742837418, 79.4960303762513, 79.15084570245081, 78.90921643079048, 78.54677252329996, 78.13255091473938, 77.77010700724888, 77.47670003451846, 77.20055229547809, 76.88988608905764, 76.37210907835691, 76.04418363824647, 75.73351743182603, 75.33655505695546, 74.80151881256472, 74.35277873662409, 73.97307559544356, 73.558853986883, 73.09285467725233, 72.5750776665516, 72.09181912323092, 71.53952364515015, 71.17707973765965, 70.74559889540905, 70.26234035208837, 69.81360027614774, 69.31308249913704, 68.81256472212634, 68.17397307559546, 67.6389368312047, 67.12115982050398, 66.48256817397308, 65.94753192958233, 65.55056955471177, 64.91197790818089, 64.34242319641008, 63.824646185709355, 63.06523990334829, 62.56472212633759, 62.08146358301691, 61.615464273386266, 61.04590956161546, 60.44183638246462, 60.01035554021401, 59.49257852951328, 58.92302381774249, 58.38798757335175, 57.7839143942009, 57.248878149810146, 56.6448049706593, 56.07525025888851, 55.41939937866759, 54.81532619951675, 54.280289955125994, 53.503624439074905, 52.89955125992406, 52.347255781843295, 51.570590265792205, 50.81118398343114, 50.069036934760106, 49.48222298929928, 48.79185364169831, 48.20503969623749, 47.51467034863652, 46.858819468415604, 46.11667241974457, 45.52985847428375, 44.874007594062824, 44.21815671384191, 43.4242319641008, 42.64756644804971, 41.99171556782879, 41.35312392129789, 40.697273041076976, 39.95512599240594, 39.17846047635486, 38.505350362443906, 37.90127718329306, 37.29720400414222, 36.589575422851226, 35.950983776320335, 35.19157749395927, 34.535726613738355, 33.828098032447365, 33.12046945115637, 32.34380393510528, 31.67069382119434, 30.9285467725233, 30.203658957542288, 29.461511908871252, 28.667587159130136, 27.977217811529165, 27.304107697618228, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.34276838108389, 81.34276838108389, 81.27373144632378, 81.2909906800138, 81.2909906800138, 81.23921297894374, 81.15291681049361, 81.0666206420435, 80.9975837072834, 80.9285467725233, 80.84225060407319, 80.66965826717293, 80.39351052813255, 80.34173282706249, 80.16914049016223, 79.97928891957199, 79.63410424577148, 79.4269934414912, 79.11632723507076, 78.73662409389023, 78.42595788746979, 78.14981014842941, 77.75284777355886, 77.18329306178806, 76.87262685536763, 76.47566448049706, 76.09596133931653, 75.57818432861582, 75.14670348636521, 74.66344494304454, 74.12840869865379, 73.48981705212289, 72.86848463928202, 72.2816706938212, 71.71211598205039, 71.05626510182948, 70.55574732481877, 70.08974801518812, 69.45115636865722, 68.8643424231964, 68.19123230928547, 67.51812219537453, 66.94856748360372, 66.39627200552296, 65.72316189161201, 65.10182947877115, 64.37694166379013, 63.600276147739045, 62.97894373489817, 62.271315153607176, 61.64998274076632, 60.94235415947532, 60.26924404556438, 59.527096996893334, 58.7849499482223, 58.18087676907145, 57.47324818778047, 56.817397307559546, 56.07525025888851, 55.402140144977565, 54.67725232999655, 54.004142216085604, 53.29651363479462, 52.46807041767345, 51.81221953745254, 51.10459095616156, 50.4314808422506, 49.74111149464964, 48.9299275112185, 48.118743527787366, 47.37659647911633, 46.634449430445294, 45.82326544701416, 45.081118398343115, 44.25267518122196, 43.56230583362099, 42.751121850189854, 41.93993786675872, 41.1460131170176, 40.352088367276494, 39.506385916465305, 38.67794269934415, 37.76320331377287, 37.03831549879186, 36.27890921643079, 35.36416983085951, 34.5012081463583, 33.67276492923714, 32.82706247842596, 31.94684156023473, 31.04936140835347, 30.100103555402146, 29.27166033828098, 28.374180186399723, 27.511218501898515, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.23921297894374, 80.96306523990336, 80.72143596824301, 80.58336209872282, 80.27269589230238, 80.06558508802209, 79.80669658267173, 79.56506731101139, 79.25440110459095, 78.89195719710045, 78.70210562651019, 78.54677252329996, 78.09803244735933, 77.83914394200899, 77.51121850189851, 77.25232999654816, 76.8553676216776, 76.66551605108734, 76.23403520883673, 75.90610976872627, 75.50914739385571, 75.0776665516051, 74.66344494304454, 74.2147048671039, 73.88677942699344, 73.62789092164309, 73.29996548153262, 72.93752157404212, 72.48878149810149, 72.07455988954091, 71.57404211253021, 71.03900586813945, 70.43493268898861, 70.10700724887815, 69.6410079392475, 69.0024162927166, 68.60545391784605, 67.94960303762512, 67.3627890921643, 66.89678978253366, 66.24093890231273, 65.70590265792198, 65.17086641353124, 64.54953400069037, 64.03175698998965, 63.61753538142907, 63.01346220227821, 62.4956851915775, 61.80531584397653, 61.27027959958578, 60.57991025198481, 60.09665170866414, 59.45806006213324, 58.97480151881257, 58.49154297549188, 58.025543665861235, 57.55954435623059, 57.093545046599935, 56.48947186744908, 56.040731791508456, 55.591991715567836, 55.03969623748706, 54.36658612357612, 53.79703141180532, 53.21021746634449, 52.70969968933379, 52.122885743872985, 51.518812564722126, 50.93199861926131, 50.20711080428028, 49.603037625129446, 48.9299275112185, 48.25681739730756, 47.63548498446669, 46.996893337935795, 46.22022782188471, 45.44356230583362, 44.89126682775285, 44.26993441491198, 43.527787366240936, 42.7166033828098, 42.07801173627891, 41.37038315498792, 40.62823610631688, 39.782533655505695, 39.074905074214705, 38.33275802554367, 37.46979634104246, 36.62409389023127, 35.79565067311011, 34.93268898860891, 34.01794960303763, 33.10321021746635, 32.205730065585094, 31.342768381083886, 30.376251294442525, 29.4269934414912, 28.460476354849852, 27.49395926820849, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.2909906800138, 81.23921297894374, 81.10113910942354, 80.9285467725233, 80.60062133241284, 80.39351052813255, 80.16914049016223, 79.9275112185019, 79.68588194684156, 79.56506731101139, 79.21988263721092, 78.80566102865033, 78.58129099068002, 78.14981014842941, 77.78736624093891, 77.39040386606834, 77.07973765964792, 76.70003451846739, 76.47566448049706, 76.13047980669658, 75.64722126337591, 75.31929582326545, 74.99137038315499, 74.76700034518467, 74.45633413876425, 74.04211253020367, 73.54159475319297, 73.23092854677252, 72.88574387297204, 72.47152226441146, 72.19537452537108, 71.74663444943045, 71.36693130824992, 70.90093199861927, 70.33137728684846, 69.81360027614774, 69.38211943389715, 68.79530548843631, 68.24301001035555, 67.7079737659648, 67.138419054194, 66.58612357611322, 65.98205039696238, 65.48153261995168, 65.0673110113911, 64.63583016914049, 64.10079392474974, 63.686572316189164, 63.18605453917846, 62.58198136002762, 62.09872281670694, 61.59820503969624, 61.114946496375566, 60.63168795305489, 60.25198481187435, 59.803244735933724, 59.23369002416293, 58.8539868829824, 58.40524680704176, 57.95650673110114, 57.3524335519503, 56.869175008629625, 56.24784259578875, 55.57473248187781, 54.95340006903693, 54.34932688988609, 53.81429064549535, 53.10666206420436, 52.53710735243355, 51.846738004832595, 51.08733172247153, 50.46599930963065, 49.689333793579564, 48.9989644459786, 48.23955816361754, 47.46289264756645, 46.54815326199517, 45.771487745944086, 44.97756299620297, 44.20089748015189, 43.3551950293407, 42.52675181221954, 41.7846047635485, 40.93890231273732, 40.17949603037626, 39.36831204694512, 38.53986882982396, 37.659647911632725, 36.79668622713152, 35.89920607525026, 34.984466689678975, 34.10424577148775, 33.20676561960649, 32.274767000345186, 31.32550914739386, 30.376251294442525, 29.39247497411115, 28.477735588539865, 27.511218501898515, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.34276838108389, 81.34276838108389, 81.27373144632378, 81.2909906800138, 81.2909906800138, 81.23921297894374, 81.15291681049361, 81.0666206420435, 80.9975837072834, 80.9285467725233, 80.84225060407319, 80.66965826717293, 80.39351052813255, 80.34173282706249, 80.16914049016223, 79.97928891957199, 79.63410424577148, 79.4269934414912, 79.11632723507076, 78.73662409389023, 78.42595788746979, 78.14981014842941, 77.75284777355886, 77.18329306178806, 76.87262685536763, 76.47566448049706, 76.09596133931653, 75.57818432861582, 75.14670348636521, 74.66344494304454, 74.12840869865379, 73.48981705212289, 72.86848463928202, 72.2816706938212, 71.71211598205039, 71.05626510182948, 70.55574732481877, 70.08974801518812, 69.45115636865722, 68.8643424231964, 68.19123230928547, 67.51812219537453, 66.94856748360372, 66.39627200552296, 65.72316189161201, 65.10182947877115, 64.37694166379013, 63.600276147739045, 62.97894373489817, 62.271315153607176, 61.64998274076632, 60.94235415947532, 60.26924404556438, 59.527096996893334, 58.7849499482223, 58.18087676907145, 57.47324818778047, 56.817397307559546, 56.07525025888851, 55.402140144977565, 54.67725232999655, 54.004142216085604, 53.29651363479462, 52.46807041767345, 51.81221953745254, 51.10459095616156, 50.4314808422506, 49.74111149464964, 48.9299275112185, 48.118743527787366, 47.37659647911633, 46.634449430445294, 45.82326544701416, 45.081118398343115, 44.25267518122196, 43.56230583362099, 42.751121850189854, 41.93993786675872, 41.1460131170176, 40.352088367276494, 39.506385916465305, 38.67794269934415, 37.76320331377287, 37.03831549879186, 36.27890921643079, 35.36416983085951, 34.5012081463583, 33.67276492923714, 32.82706247842596, 31.94684156023473, 31.04936140835347, 30.100103555402146, 29.27166033828098, 28.374180186399723, 27.511218501898515, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2028, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.44632378322403, 81.48084225060408, 81.46358301691406, 81.46358301691406, 81.46358301691406, 81.48084225060408, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.20469451156369, 81.10113910942354, 81.08387987573352, 81.04936140835348, 80.89402830514325, 80.75595443562305, 80.66965826717293, 80.44528822920263, 80.16914049016223, 79.82395581636176, 79.46151190887124, 79.23714187090093, 78.9264756644805, 78.63306869175008, 78.39143942008975, 78.01173627890921, 77.70107007248879, 77.32136693130826, 76.97618225750776, 76.51018294787711, 75.90610976872627, 75.45736969278565, 74.87055574732481, 74.49085260614429, 73.93855712806352, 73.558853986883, 73.0583362098723, 72.36796686227132, 71.86744908526062, 71.36693130824992, 70.9181912323093, 70.15878494994823, 69.69278564031757, 69.01967552640663, 68.43286158094581, 67.74249223334485, 66.98308595098378, 66.32723507076285, 65.72316189161201, 65.25716258198136, 64.67034863652054, 64.10079392474974, 63.56575768035899, 62.823610631687956, 62.09872281670694, 61.30479806696583, 60.63168795305489, 59.993096306523995, 59.285467725233, 58.49154297549188, 57.749395926820846, 57.024508111839836, 56.403175698998965, 55.74732481877804, 54.97065930272696, 54.12495685191578, 53.36555056955472, 52.58888505350363, 51.846738004832595, 51.12185018985157, 50.362443907490515, 49.603037625129446, 48.774594408008284, 48.08422506040732, 47.186744908526066, 46.358301691404904, 45.564376941663795, 44.71867449085261, 43.89023127373145, 42.975491888160164, 42.06075250258889, 41.2150500517777, 40.40386606834657, 39.489126682775286, 38.62616499827408, 37.780462547462896, 36.93476009665171, 36.10631687953055, 35.1397997928892, 34.22506040731791, 33.24128408698654, 32.32654470141526, 31.377286848463925, 30.445288229202628, 29.49603037625129, 28.52951328960994, 27.52847773558854, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2028, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.4981014842941, 81.48084225060408, 81.48084225060408, 81.44632378322403, 81.429064549534, 81.37728684846392, 81.41180531584398, 81.36002761477391, 81.30824991370383, 81.2909906800138, 81.25647221263375, 81.17017604418363, 81.11839834311357, 81.08387987573352, 81.03210217466345, 80.96306523990336, 80.85950983776321, 80.70417673455299, 80.53158439765275, 80.35899206075251, 80.23817742492233, 79.96202968588196, 79.68588194684156, 79.44425267518123, 79.09906800138074, 78.82292026234036, 78.54677252329996, 78.33966171901969, 78.08077321366932, 77.90818087676908, 77.63203313772868, 77.21781152916812, 76.87262685536763, 76.37210907835691, 75.95788746979635, 75.37107352433551, 74.95685191577495, 74.43907490507422, 73.97307559544356, 73.576113220573, 73.04107697618227, 72.50604073179151, 72.02278218847084, 71.47048671039006, 70.79737659647913, 70.19330341732827, 69.58923023817742, 68.8643424231964, 68.24301001035555, 67.77701070072489, 67.138419054194, 66.43079047290301, 65.63686572316189, 64.9292371418709, 64.20434932688988, 63.4967207455989, 62.754573696927864, 61.94338971349672, 61.23576113220574, 60.54539178460476, 59.88954090438385, 59.06109768726269, 58.31895063859165, 57.43872972040042, 56.66206420434933, 55.83362098722817, 55.02243700379703, 54.21125302036589, 53.451846738004825, 52.623403520883684, 51.77770107007249, 50.94925785295133, 50.03451846738005, 49.20607525025889, 48.463928201587855, 47.53192958232655, 46.66896789782534, 45.771487745944086, 44.97756299620297, 44.14911977908181, 43.303417328270626, 42.44045564376942, 41.57749395926821, 40.680013807386956, 39.81705212288574, 38.919571971004494, 38.073869520193306, 37.2109078356921, 36.33068691750086, 35.39868829823956, 34.535726613738355, 33.65550569554712, 32.70624784259579, 31.79150845702451, 30.89402830514325, 29.927511218501902, 29.012771832930618, 28.098032447359344, 27.304107697618228, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.25647221263375, 81.08387987573352, 80.94580600621333, 80.68691750086296, 80.34173282706249, 79.9275112185019, 79.58232654470142, 79.40973420780118, 79.03003106662064, 78.68484639282016, 78.27062478425958, 77.87366240938903, 77.51121850189851, 77.07973765964792, 76.71729375215742, 76.54470141525717, 76.25129444252676, 75.94062823610632, 75.35381429064549, 74.85329651363479, 74.45633413876425, 74.0766309975837, 73.71418709009319, 73.31722471522265, 72.9202623403521, 72.45426303072145, 71.86744908526062, 71.45322747670004, 71.03900586813945, 70.4176734552986, 69.89989644459786, 69.43389713496721, 68.91612012426648, 68.38108387987573, 67.91508457024509, 67.46634449430445, 67.03486365205384, 66.5688643424232, 66.1373835001726, 65.63686572316189, 64.998274076631, 64.30790472903003, 63.72109078356921, 63.15153607179842, 62.4956851915775, 61.926130479806694, 61.391094235415956, 60.71798412150501, 60.027614773904034, 59.49257852951328, 58.871246116672424, 58.2844321712116, 57.69761822575078, 57.07628581290991, 56.472212633759064, 55.73006558508802, 55.17777010700725, 54.556437694166384, 53.917846047635486, 53.29651363479462, 52.726958923023815, 52.10562651018296, 51.43251639627201, 50.72488781498101, 50.086296168450126, 49.44770452191923, 48.84363134276838, 47.9979288919572, 47.41111494649638, 46.75526406627546, 45.96133931653435, 45.32274767000345, 44.666896789782534, 43.942008974801524, 43.251639627200554, 42.56127027959959, 41.83638246461857, 41.16327235070763, 40.576458405246804, 39.92060752502589, 39.212978943734896, 38.557128063513986, 37.93579565067311, 37.2799447704522, 36.53779772178115, 35.81290990680014, 35.0707628581291, 34.483948912668275, 33.81083879875734, 33.22402485329651, 32.550914739385576, 31.9986192613048, 31.342768381083886, 30.773213669313083, 29.944770452191925, 29.323438039351057, 28.633068691750086, 27.95995857783915, 27.200552295478087, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.37728684846392, 81.37728684846392, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.2909906800138, 81.30824991370383, 81.23921297894374, 81.22195374525371, 81.17017604418363, 81.11839834311357, 80.96306523990336, 80.87676907145323, 80.63513979979288, 80.47980669658268, 80.2899551259924, 80.08284432171212, 79.7894373489817, 79.47877114256127, 79.23714187090093, 78.9955125992406, 78.61580945806006, 78.32240248532966, 78.02899551259924, 77.68381083879876, 77.28684846392821, 76.78633068691751, 76.423886779427, 75.87159130134621, 75.49188816016569, 75.00862961684501, 74.43907490507422, 74.02485329651364, 73.45529858474285, 72.88574387297204, 72.47152226441146, 71.98826372109079, 71.43596824301002, 70.93545046599931, 70.38315498791854, 69.62374870555747, 69.12323092854677, 68.53641698308596, 67.89782533655506, 67.25923369002416, 66.55160510873317, 65.87849499482223, 65.25716258198136, 64.7911632723507, 64.04901622367967, 63.35864687607871, 62.823610631687956, 62.13324128408699, 61.52916810493614, 60.90783569209527, 60.14842940973422, 59.423541594753196, 58.715913013462206, 57.95650673110114, 57.21435968243011, 56.42043493268899, 55.66102865032793, 55.005177770107004, 54.12495685191578, 53.279254401104595, 52.43355195029341, 51.570590265792205, 50.70762858129099, 49.87918536416983, 49.06800138073869, 48.23955816361754, 47.39385571280636, 46.47911632723507, 45.61615464273386, 44.8567483603728, 44.011045909561616, 43.096306523990336, 42.2851225405592, 41.49119779081809, 40.52468070417674, 39.66171901967553, 38.72972040041422, 37.74594408008284, 36.90024162927166, 35.950983776320335, 35.053503624439074, 34.1387642388678, 33.20676561960649, 32.274767000345186, 31.32550914739386, 30.410769761822575, 29.47877114256128, 28.512254055919918, 27.54573696927857, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.37728684846392, 81.41180531584398, 81.34276838108389, 81.15291681049361, 80.9285467725233, 80.75595443562305, 80.46254746289266, 80.16914049016223, 79.94477045219193, 79.65136347946151, 79.3579565067311, 79.15084570245081, 78.89195719710045, 78.56403175698999, 78.28788401794961, 77.92544011045909, 77.49395926820849, 77.21781152916812, 76.90714532274767, 76.54470141525717, 76.14773904038661, 75.80255436658612, 75.49188816016569, 75.11218501898516, 74.73248187780463, 74.31826026924405, 73.95581636175355, 73.576113220573, 73.12737314463237, 72.69589230238178, 72.26441146013117, 71.74663444943045, 71.2806351397998, 70.76285812909907, 70.26234035208837, 69.72730410769762, 69.12323092854677, 68.58819468415602, 68.15671384190543, 67.5698998964446, 66.91404901622369, 66.37901277183293, 65.87849499482223, 65.27442181567137, 64.73938557128064, 64.15257162581982, 63.61753538142907, 63.18605453917846, 62.68553676216776, 62.202278218847084, 61.701760441836385, 61.16672419744563, 60.700724887814985, 60.18294787711426, 59.69968933379358, 59.16465308940283, 58.61235761132206, 58.249913703831545, 57.801173627890925, 57.26613738350017, 56.88643424231964, 56.472212633759064, 56.00621332412841, 55.47117707973766, 54.97065930272696, 54.47014152571625, 53.96962374870555, 53.503624439074905, 52.951328960994125, 52.32999654815327, 51.74318260269244, 51.018294787711426, 50.34518467380049, 49.82740766309976, 49.06800138073869, 48.30859509837763, 47.480151881256475, 46.72074559889541, 45.926820849154296, 45.06385916465309, 44.25267518122196, 43.441491197790825, 42.49223334483949, 41.6465308940283, 40.714532274767, 39.76527442181567, 38.90231273731447, 37.93579565067311, 37.05557473248188, 36.10631687953055, 35.157059026579226, 34.20780117362789, 33.29306178805661, 32.32654470141526, 31.360027614773912, 30.410769761822575, 29.444252675181225, 28.49499482222989, 27.511218501898515, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.18743527787366, 81.10113910942354, 80.87676907145323, 80.58336209872282, 80.37625129444253, 80.16914049016223, 79.91025198481188, 79.47877114256127, 79.20262340352089, 78.8574387297204, 78.56403175698999, 78.33966171901969, 78.01173627890921, 77.63203313772868, 77.28684846392821, 77.07973765964792, 76.68277528477736, 76.30307214359682, 75.95788746979635, 75.59544356230583, 75.19848118743528, 74.83603727994476, 74.4218156713842, 73.9212978943735, 73.35174318260269, 73.0755954435623, 72.69589230238178, 72.29892992751122, 71.76389368312047, 71.26337590610977, 71.0044874007594, 70.4176734552986, 69.79634104245773, 69.22678633068692, 68.7090093199862, 68.1394546082154, 67.62167759751468, 67.0693821194339, 66.6379012771833, 65.94753192958233, 65.41249568519159, 64.91197790818089, 64.27338626164997, 63.72109078356921, 63.16879530548844, 62.616499827407665, 62.2885743872972, 61.77079737659648, 61.339316534345876, 60.75250258888505, 60.44183638246462, 60.027614773904034, 59.45806006213324, 59.00931998619261, 58.526061442871935, 58.11183983431136, 57.66309975837073, 57.21435968243011, 56.73110113910942, 56.42043493268899, 55.95443562305834, 55.52295478080773, 54.901622367966866, 54.280289955125994, 53.67621677597515, 53.123921297894384, 52.450811183983426, 51.82947877114257, 51.12185018985157, 50.379703141180535, 49.79288919571972, 49.01622367966863, 48.274076630997584, 47.54918881601657, 46.65170866413531, 45.84052468070418, 44.943044528822924, 44.080082844321716, 43.19986192613048, 42.3541594753193, 41.473938557128065, 40.576458405246804, 39.7134967207456, 38.83327580255437, 37.90127718329306, 36.98653779772179, 36.05453917846047, 35.10528132550915, 34.173282706247846, 33.24128408698654, 32.32654470141526, 31.360027614773912, 30.410769761822575, 29.444252675181225, 28.477735588539865, 27.52847773558854, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.37728684846392, 81.37728684846392, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.2909906800138, 81.30824991370383, 81.23921297894374, 81.22195374525371, 81.17017604418363, 81.11839834311357, 80.96306523990336, 80.87676907145323, 80.63513979979288, 80.47980669658268, 80.2899551259924, 80.08284432171212, 79.7894373489817, 79.47877114256127, 79.23714187090093, 78.9955125992406, 78.61580945806006, 78.32240248532966, 78.02899551259924, 77.68381083879876, 77.28684846392821, 76.78633068691751, 76.423886779427, 75.87159130134621, 75.49188816016569, 75.00862961684501, 74.43907490507422, 74.02485329651364, 73.45529858474285, 72.88574387297204, 72.47152226441146, 71.98826372109079, 71.43596824301002, 70.93545046599931, 70.38315498791854, 69.62374870555747, 69.12323092854677, 68.53641698308596, 67.89782533655506, 67.25923369002416, 66.55160510873317, 65.87849499482223, 65.25716258198136, 64.7911632723507, 64.04901622367967, 63.35864687607871, 62.823610631687956, 62.13324128408699, 61.52916810493614, 60.90783569209527, 60.14842940973422, 59.423541594753196, 58.715913013462206, 57.95650673110114, 57.21435968243011, 56.42043493268899, 55.66102865032793, 55.005177770107004, 54.12495685191578, 53.279254401104595, 52.43355195029341, 51.570590265792205, 50.70762858129099, 49.87918536416983, 49.06800138073869, 48.23955816361754, 47.39385571280636, 46.47911632723507, 45.61615464273386, 44.8567483603728, 44.011045909561616, 43.096306523990336, 42.2851225405592, 41.49119779081809, 40.52468070417674, 39.66171901967553, 38.72972040041422, 37.74594408008284, 36.90024162927166, 35.950983776320335, 35.053503624439074, 34.1387642388678, 33.20676561960649, 32.274767000345186, 31.32550914739386, 30.410769761822575, 29.47877114256128, 28.512254055919918, 27.54573696927857, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2029, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.44632378322403, 81.46358301691406, 81.429064549534, 81.44632378322403, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.30824991370383, 81.25647221263375, 81.18743527787366, 81.20469451156369, 81.15291681049361, 81.04936140835348, 80.98032447359338, 80.66965826717293, 80.46254746289266, 80.23817742492233, 80.01380738695202, 79.72040041422161, 79.34069727304109, 78.94373489817053, 78.65032792544011, 78.25336555056955, 77.83914394200899, 77.45944080082845, 77.04521919226787, 76.63099758370728, 76.13047980669658, 75.64722126337591, 75.11218501898516, 74.45633413876425, 73.95581636175355, 73.50707628581291, 72.98929927511219, 72.45426303072145, 72.10907835692095, 71.55678287884018, 70.96996893337936, 70.40041422160857, 69.93441491197791, 69.36486020020712, 68.7090093199862, 68.26026924404557, 67.77701070072489, 67.19019675526407, 66.62064204349328, 66.03382809803244, 65.58508802209182, 65.01553331032103, 64.54953400069037, 63.89368312046946, 63.25509147393856, 62.72005522954781, 62.02968588194684, 61.44287193648602, 60.700724887814985, 60.01035554021401, 59.49257852951328, 58.73317224715223, 58.04280289955126, 57.23161891612012, 56.55850880220918, 55.74732481877804, 55.108733172247156, 54.418363824646185, 53.641698308595096, 52.795995857783915, 52.0020711080428, 51.173627890921644, 50.44874007594062, 49.58577839143942, 48.774594408008284, 47.9288919571971, 47.01415257162582, 46.11667241974457, 45.25371073524336, 44.373489817052125, 43.458750431480844, 42.56127027959959, 41.7155678287884, 40.7835692095271, 39.83431135657577, 38.850535036244395, 37.849499482222996, 36.93476009665171, 35.950983776320335, 35.036244390749054, 34.15602347255782, 33.258543320676566, 32.32654470141526, 31.411805315843978, 30.46254746289265, 29.49603037625129, 28.546772523299968, 27.54573696927857, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2029, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.46358301691406, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.429064549534, 81.429064549534, 81.39454608215395, 81.2909906800138, 81.25647221263375, 81.15291681049361, 81.08387987573352, 80.98032447359338, 80.89402830514325, 80.80773213669313, 80.68691750086296, 80.53158439765275, 80.25543665861235, 80.08284432171212, 79.84121505005179, 79.54780807732136, 79.25440110459095, 78.90921643079048, 78.61580945806006, 78.25336555056955, 77.87366240938903, 77.44218156713842, 77.13151536071798, 76.80358992060754, 76.44114601311702, 76.23403520883673, 75.80255436658612, 75.33655505695546, 74.93959268208492, 74.64618570935451, 74.12840869865379, 73.47255781843286, 73.0755954435623, 72.45426303072145, 71.9192267863307, 71.55678287884018, 70.96996893337936, 70.36589575422852, 69.86537797721782, 69.29582326544703, 68.79530548843631, 68.10493614083535, 67.39730755954436, 66.7069382119434, 66.03382809803244, 65.39523645150156, 64.61857093545046, 63.97997928891957, 63.35864687607871, 62.58198136002762, 61.839834311356576, 61.18398343113566, 60.510873317224714, 59.88954090438385, 59.00931998619261, 58.12909906800139, 57.43872972040042, 56.78287884017949, 55.93717638936832, 55.1432516396272, 54.34932688988609, 53.55540214014498, 52.67518122195375, 51.77770107007249, 50.98377632033137, 50.15533310321022, 49.20607525025889, 48.4294097342078, 47.51467034863652, 46.73800483258543, 45.94408008284432, 45.06385916465309, 44.166379012771834, 43.26889886089058, 42.371418709009326, 41.473938557128065, 40.6454953400069, 39.782533655505695, 38.97134967207456, 38.10838798757336, 37.228167069382124, 36.36520538488091, 35.450465999309635, 34.5012081463583, 33.60372799447705, 32.74076630997584, 31.843286158094582, 30.945806006213328, 30.031066620642044, 29.08180876769072, 28.23610631687953, 27.33862616499828, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.0666206420435, 81.03210217466345, 81.04936140835348, 80.82499137038316, 80.56610286503279, 80.46254746289266, 80.13462202278218, 79.87573351743183, 79.53054884363134, 79.13358646876078, 78.65032792544011, 78.30514325163963, 77.97721781152917, 77.64929237141871, 77.33862616499827, 77.06247842595789, 76.64825681739731, 76.23403520883673, 75.73351743182603, 75.33655505695546, 75.0604073179151, 74.68070417673457, 74.38729720400414, 73.95581636175355, 73.47255781843286, 73.1446323783224, 72.66137383500173, 72.24715222644114, 71.74663444943045, 71.19433897134968, 70.67656196064895, 70.1415257162582, 69.7100448740076, 69.24404556437695, 68.74352778736625, 68.34656541249569, 67.79426993441491, 67.3627890921643, 66.68967897825337, 66.1373835001726, 65.77493959268207, 65.18812564722126, 64.61857093545046, 64.16983085950984, 63.53123921297894, 62.9962029685882, 62.51294442526751, 61.87435277873663, 61.25302036589575, 60.75250258888505, 60.18294787711426, 59.64791163272351, 58.97480151881257, 58.40524680704176, 57.801173627890925, 57.23161891612012, 56.593027269589236, 55.91991715567829, 55.229547808077314, 54.65999309630652, 54.055919917155684, 53.40006903693476, 52.86503279254402, 52.26095961339317, 51.570590265792205, 51.0010355540214, 50.379703141180535, 49.72385226095962, 49.06800138073869, 48.394891266827756, 47.75629962029686, 47.04867103900587, 46.32378322402486, 45.650673110113914, 44.96030376251295, 44.23541594753193, 43.527787366240936, 42.82015878494995, 42.12978943734898, 41.37038315498792, 40.59371763893683, 39.95512599240594, 39.17846047635486, 38.48809112875389, 37.69416637901277, 36.98653779772179, 36.244390749050744, 35.5712806351398, 34.81187435277874, 34.173282706247846, 33.4311356575768, 32.654470141525714, 31.96410079392475, 31.187435277873664, 30.376251294442525, 29.599585778391447, 28.9264756644805, 28.132550914739383, 27.390403866068347, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.39454608215395, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.429064549534, 81.46358301691406, 81.41180531584398, 81.41180531584398, 81.36002761477391, 81.34276838108389, 81.27373144632378, 81.20469451156369, 81.17017604418363, 81.10113910942354, 81.03210217466345, 80.9285467725233, 80.82499137038316, 80.63513979979288, 80.37625129444253, 80.20365895754229, 79.87573351743183, 79.77217811529168, 79.65136347946151, 79.25440110459095, 78.90921643079048, 78.59855022437004, 78.14981014842941, 77.90818087676908, 77.56299620296859, 77.18329306178806, 76.73455298584743, 76.23403520883673, 75.7852951328961, 75.35381429064549, 75.00862961684501, 74.59440800828443, 74.11114946496376, 73.67966862271315, 73.12737314463237, 72.67863306869175, 72.17811529168105, 71.59130134622022, 71.09078356920952, 70.59026579219882, 69.96893337935795, 69.36486020020712, 68.91612012426648, 68.2084915429755, 67.60441836382465, 66.96582671729375, 66.39627200552296, 65.70590265792198, 65.11908871246118, 64.41146013117019, 63.772868484639275, 63.099758370728345, 62.40938902312738, 61.66724197445633, 61.04590956161546, 60.424577148774596, 59.95857783914394, 59.31998619261305, 58.526061442871935, 57.818432861580945, 57.11080428028996, 56.24784259578875, 55.5056955471177, 54.729030031066614, 53.81429064549535, 52.916810493614086, 52.19192267863306, 51.294442526751816, 50.396962374870554, 49.620296858819465, 48.757335174318264, 47.84259578874698, 46.996893337935795, 46.09941318605454, 45.288229202623405, 44.39074905074215, 43.441491197790825, 42.56127027959959, 41.681049361408355, 40.7835692095271, 39.83431135657577, 38.90231273731447, 37.97031411805316, 37.03831549879186, 36.089057645840526, 35.157059026579226, 34.173282706247846, 33.24128408698654, 32.29202623403521, 31.39454608215395, 30.46254746289265, 29.54780807732137, 28.546772523299968, 27.56299620296859, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.37728684846392, 81.23921297894374, 81.01484294097342, 80.89402830514325, 80.66965826717293, 80.47980669658268, 80.18639972385226, 80.11736278909216, 79.84121505005179, 79.39247497411115, 79.08180876769072, 78.9264756644805, 78.58129099068002, 78.25336555056955, 78.04625474628926, 77.68381083879876, 77.32136693130826, 76.99344149119779, 76.63099758370728, 76.21677597514672, 75.87159130134621, 75.50914739385571, 75.2157404211253, 74.93959268208492, 74.62892647566449, 74.11114946496376, 73.86952019330342, 73.47255781843286, 73.11011391094236, 72.67863306869175, 72.2816706938212, 71.71211598205039, 71.2806351397998, 70.74559889540905, 70.17604418363825, 69.74456334138765, 69.15774939592683, 68.60545391784605, 68.15671384190543, 67.5008629616845, 66.93130824991371, 66.39627200552296, 65.9302726958923, 65.25716258198136, 64.5667932343804, 63.9972385226096, 63.324128408698655, 62.68553676216776, 61.977908180876774, 61.391094235415956, 60.83879875733518, 60.200207110804286, 59.45806006213324, 58.76769071453227, 58.094580600621335, 57.455988954090444, 56.869175008629625, 56.33413876423887, 55.73006558508802, 55.195029340697275, 54.65999309630652, 54.21125302036589, 53.74525371073524, 53.38280980324473, 52.916810493614086, 52.519848118743525, 52.0020711080428, 51.70866413531239, 51.363479461511915, 50.862961684501215, 50.362443907490515, 49.896444597859855, 49.29237141870901, 48.56748360372799, 47.894373489817056, 47.273041076976185, 46.58267172937522, 45.857783914394204, 45.098377632033134, 44.33897134967207, 43.51052813255092, 42.7166033828098, 41.8536416983086, 40.99068001380739, 40.17949603037626, 39.36831204694512, 38.47083189506386, 37.62512944425268, 36.7103900586814, 35.81290990680014, 34.94994822229893, 34.10424577148775, 33.24128408698654, 32.30928546772523, 31.32550914739386, 30.410769761822575, 29.47877114256128, 28.49499482222989, 27.511218501898515, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.39454608215395, 81.36002761477391, 81.11839834311357, 80.94580600621333, 80.7904729030031, 80.61788056610285, 80.39351052813255, 80.16914049016223, 79.80669658267173, 79.54780807732136, 79.11632723507076, 78.87469796341043, 78.58129099068002, 78.27062478425958, 77.90818087676908, 77.58025543665862, 77.30410769761824, 77.07973765964792, 76.76907145322748, 76.40662754573697, 75.9233690024163, 75.45736969278565, 75.0776665516051, 74.68070417673457, 74.26648256817397, 73.83500172592336, 73.38626164998274, 72.95478080773215, 72.45426303072145, 71.93648602002072, 71.52226441146013, 71.17707973765965, 70.71108042802899, 70.38315498791854, 69.93441491197791, 69.46841560234725, 69.05419399378667, 68.38108387987573, 67.93234380393511, 67.31101139109424, 66.68967897825337, 66.10286503279255, 65.51605108733173, 64.96375560925095, 64.27338626164997, 63.61753538142907, 62.927166033828094, 62.3576113220573, 61.75353814290645, 61.06316879530549, 60.54539178460476, 59.83776320331378, 59.18191232309285, 58.49154297549188, 57.852951328961, 57.26613738350017, 56.73110113910942, 56.05799102519848, 55.41939937866759, 54.81532619951675, 54.332067656196074, 53.848809112875394, 53.40006903693476, 53.03762512944425, 52.6406627545737, 52.140144977563, 51.81221953745254, 51.518812564722126, 51.08733172247153, 50.72488781498101, 50.25888850535036, 49.689333793579564, 49.22333448394892, 48.619261304798066, 47.980669658267175, 47.39385571280636, 46.65170866413531, 45.90956161546427, 45.25371073524336, 44.442526751812224, 43.734898170521234, 42.85467725233, 42.06075250258889, 41.2840869865378, 40.45564376941664, 39.644459785985504, 38.74697963410424, 37.79772178115292, 36.86572316189161, 35.91646530894029, 35.01898515705903, 34.035208836727655, 33.0686917500863, 32.18847083189507, 31.32550914739386, 30.393510528132552, 29.444252675181225, 28.460476354849852, 27.511218501898515, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.39454608215395, 81.46358301691406, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.429064549534, 81.46358301691406, 81.41180531584398, 81.41180531584398, 81.36002761477391, 81.34276838108389, 81.27373144632378, 81.20469451156369, 81.17017604418363, 81.10113910942354, 81.03210217466345, 80.9285467725233, 80.82499137038316, 80.63513979979288, 80.37625129444253, 80.20365895754229, 79.87573351743183, 79.77217811529168, 79.65136347946151, 79.25440110459095, 78.90921643079048, 78.59855022437004, 78.14981014842941, 77.90818087676908, 77.56299620296859, 77.18329306178806, 76.73455298584743, 76.23403520883673, 75.7852951328961, 75.35381429064549, 75.00862961684501, 74.59440800828443, 74.11114946496376, 73.67966862271315, 73.12737314463237, 72.67863306869175, 72.17811529168105, 71.59130134622022, 71.09078356920952, 70.59026579219882, 69.96893337935795, 69.36486020020712, 68.91612012426648, 68.2084915429755, 67.60441836382465, 66.96582671729375, 66.39627200552296, 65.70590265792198, 65.11908871246118, 64.41146013117019, 63.772868484639275, 63.099758370728345, 62.40938902312738, 61.66724197445633, 61.04590956161546, 60.424577148774596, 59.95857783914394, 59.31998619261305, 58.526061442871935, 57.818432861580945, 57.11080428028996, 56.24784259578875, 55.5056955471177, 54.729030031066614, 53.81429064549535, 52.916810493614086, 52.19192267863306, 51.294442526751816, 50.396962374870554, 49.620296858819465, 48.757335174318264, 47.84259578874698, 46.996893337935795, 46.09941318605454, 45.288229202623405, 44.39074905074215, 43.441491197790825, 42.56127027959959, 41.681049361408355, 40.7835692095271, 39.83431135657577, 38.90231273731447, 37.97031411805316, 37.03831549879186, 36.089057645840526, 35.157059026579226, 34.173282706247846, 33.24128408698654, 32.29202623403521, 31.39454608215395, 30.46254746289265, 29.54780807732137, 28.546772523299968, 27.56299620296859, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2030, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.30824991370383, 81.36002761477391, 81.36002761477391, 81.36002761477391, 81.36002761477391, 81.30824991370383, 81.30824991370383, 81.27373144632378, 81.2909906800138, 81.2909906800138, 81.2909906800138, 81.2909906800138, 81.18743527787366, 81.15291681049361, 81.0666206420435, 81.03210217466345, 80.85950983776321, 80.7904729030031, 80.63513979979288, 80.37625129444253, 80.15188125647221, 79.87573351743183, 79.70314118053159, 79.44425267518123, 79.16810493614084, 78.87469796341043, 78.49499482222988, 77.99447704521918, 77.51121850189851, 77.23507076285813, 76.8553676216776, 76.423886779427, 76.00966517086641, 75.56092509492579, 75.11218501898516, 74.73248187780463, 74.23196410079392, 73.80048325854332, 73.17915084570245, 72.60959613393166, 72.12633759061097, 71.53952364515015, 71.09078356920952, 70.52122885743873, 70.05522954780807, 69.62374870555747, 69.01967552640663, 68.41560234725578, 67.65619606489473, 66.98308595098378, 66.39627200552296, 65.74042112530203, 65.20538488091128, 64.46323783224025, 63.63479461511909, 63.047980669658266, 62.392129789437355, 61.58094580600621, 60.71798412150501, 60.06213324128409, 59.31998619261305, 58.715913013462206, 58.04280289955126, 57.33517431826027, 56.66206420434933, 56.02347255781843, 55.28132550914739, 54.59095616154643, 53.848809112875394, 53.227476700034515, 52.53710735243355, 51.846738004832595, 51.03555402140145, 50.22437003797031, 49.37866758715913, 48.602002071108046, 47.79081808767691, 46.910597169485676, 46.04763548498447, 45.18467380048326, 44.304452882292026, 43.458750431480844, 42.57852951328962, 41.750086296168455, 40.80082844321712, 39.83431135657577, 38.95409043838454, 38.03935105281326, 37.072833966171906, 36.089057645840526, 35.174318260269246, 34.20780117362789, 33.20676561960649, 32.29202623403521, 31.360027614773912, 30.4280289955126, 29.530548843631344, 28.546772523299968, 27.56299620296859, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2030, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.4981014842941, 81.4981014842941, 81.46358301691406, 81.48084225060408, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.34276838108389, 81.34276838108389, 81.34276838108389, 81.32550914739386, 81.2909906800138, 81.22195374525371, 81.1356575768036, 81.04936140835348, 80.89402830514325, 80.70417673455299, 80.4970659302727, 80.41076976182258, 80.27269589230238, 80.10010355540214, 79.84121505005179, 79.51328960994132, 79.21988263721092, 78.90921643079048, 78.51225405591991, 78.14981014842941, 77.70107007248879, 77.25232999654816, 76.87262685536763, 76.423886779427, 75.97514670348636, 75.4228512254056, 75.00862961684501, 74.7152226441146, 74.31826026924405, 73.83500172592336, 73.36900241629272, 72.97204004142216, 72.45426303072145, 72.09181912323092, 71.69485674836038, 71.09078356920952, 70.48671039005868, 69.79634104245773, 69.1404901622368, 68.51915774939593, 67.74249223334485, 67.22471522264412, 66.48256817397308, 65.8612357611322, 65.0673110113911, 64.48049706593028, 63.80738695201933, 63.16879530548844, 62.46116672419745, 61.7880566102865, 61.14946496375562, 60.44183638246462, 59.66517086641353, 58.8539868829824, 58.04280289955126, 57.23161891612012, 56.523990334829136, 55.74732481877804, 54.918881601656885, 54.142216085605796, 53.417328270624786, 52.65792198826372, 51.881256472212634, 51.12185018985157, 50.53503624439075, 49.672074559889545, 48.895409043838455, 48.10148429409735, 47.32481877804626, 46.58267172937522, 45.771487745944086, 44.943044528822924, 44.06282361063169, 43.18260269244046, 42.44045564376942, 41.57749395926821, 40.766309975837075, 39.93786675871592, 39.04038660683466, 38.160165688643424, 37.33172247152227, 36.48602002071108, 35.62305833620987, 34.760096651708665, 33.84535726613739, 32.94787711425613, 32.01587849499482, 31.11839834311356, 30.27269589230238, 29.340697273041084, 28.443217121159826, 27.54573696927857, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "MiniCPM", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.23921297894374, 81.18743527787366, 80.9975837072834, 80.72143596824301, 80.39351052813255, 80.11736278909216, 79.73765964791163, 79.3579565067311, 79.06454953400069, 78.75388332758025, 78.30514325163963, 77.97721781152917, 77.56299620296859, 77.18329306178806, 76.68277528477736, 76.35484984466689, 76.04418363824647, 75.62996202968588, 75.37107352433551, 75.09492578529513, 74.50811183983431, 74.16292716603382, 73.71418709009319, 73.2827062478426, 72.93752157404212, 72.52329996548154, 72.09181912323092, 71.71211598205039, 71.24611667241975, 70.76285812909907, 70.38315498791854, 69.93441491197791, 69.38211943389715, 68.91612012426648, 68.51915774939593, 68.0013807386952, 67.43182602692441, 66.89678978253366, 66.39627200552296, 65.8612357611322, 65.27442181567137, 64.75664480497066, 64.18709009319986, 63.66931308249914, 63.099758370728345, 62.46116672419745, 61.891612012426656, 61.339316534345876, 60.71798412150501, 60.18294787711426, 59.50983776320331, 59.026579219882635, 58.439765274421816, 57.870210562651025, 57.19710044874008, 56.55850880220918, 55.93717638936832, 55.31584397652745, 54.62547462892648, 53.98688298239558, 53.31377286848464, 52.623403520883684, 51.98481187435278, 51.242664825681736, 50.60407317915085, 49.98274076630998, 49.24059371763894, 48.4984466689679, 47.894373489817056, 47.273041076976185, 46.59993096306524, 45.97859855022437, 45.288229202623405, 44.597859855022435, 43.89023127373145, 43.26889886089058, 42.664825681739735, 41.93993786675872, 41.266827752847774, 40.62823610631688, 39.83431135657577, 39.12668277528478, 38.52260959613393, 37.79772178115292, 37.090093199861926, 36.33068691750086, 35.5712806351398, 35.001725923369, 34.36313427683811, 33.70728339661719, 32.96513634794616, 32.361063168795305, 31.567138419054196, 30.85950983776321, 30.065585088022097, 29.409734207801176, 28.719364860200216, 28.063513979979295, 27.321366931308255, 26.561960648947192]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.27373144632378, 81.30824991370383, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.22195374525371, 81.18743527787366, 81.11839834311357, 81.03210217466345, 81.03210217466345, 81.01484294097342, 80.96306523990336, 80.89402830514325, 80.87676907145323, 80.66965826717293, 80.70417673455299, 80.61788056610285, 80.53158439765275, 80.35899206075251, 80.27269589230238, 80.11736278909216, 80.03106662064205, 79.94477045219193, 79.89299275112185, 79.70314118053159, 79.72040041422161, 79.53054884363134, 79.39247497411115, 79.28891957197101, 79.28891957197101, 79.15084570245081, 78.9955125992406, 78.87469796341043, 78.73662409389023, 78.56403175698999, 78.54677252329996, 78.37418018639973, 78.30514325163963, 78.13255091473938, 77.99447704521918, 77.82188470831896, 77.856403175699, 77.68381083879876, 77.58025543665862, 77.3558853986883, 77.16603382809804, 77.06247842595789, 76.78633068691751, 76.49292371418709, 76.30307214359682, 75.99240593717639, 75.71625819813602, 75.52640662754574, 75.37107352433551, 75.02588885053504, 74.80151881256472, 74.59440800828443, 74.2147048671039, 73.7832240248533, 73.31722471522265, 72.98929927511219, 72.50604073179151, 72.02278218847084, 71.43596824301002, 70.90093199861927, 70.48671039005868, 70.05522954780807, 69.38211943389715, 68.69175008629618, 68.12219537452538, 67.62167759751468, 67.01760441836383, 66.24093890231273, 65.68864342423197, 65.17086641353124, 64.54953400069037, 63.9972385226096, 63.22057300655851, 62.44390749050742, 61.71901967552641, 61.04590956161546, 60.200207110804286, 59.31998619261305, 58.439765274421816, 57.62858129099069, 56.748360372799446, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.44632378322403, 81.32550914739386, 81.1356575768036, 80.94580600621333, 80.91128753883328, 80.7904729030031, 80.65239903348291, 80.4970659302727, 80.4280289955126, 80.32447359337246, 80.18639972385226, 79.94477045219193, 79.66862271315154, 79.61684501208146, 79.4269934414912, 79.32343803935106, 79.13358646876078, 78.9264756644805, 78.8574387297204, 78.52951328960994, 78.39143942008975, 78.14981014842941, 77.83914394200899, 77.58025543665862, 77.37314463237833, 77.16603382809804, 77.07973765964792, 76.99344149119779, 76.88988608905764, 76.71729375215742, 76.63099758370728, 76.40662754573697, 76.23403520883673, 75.94062823610632, 75.68173973075596, 75.56092509492579, 75.25025888850536, 74.90507421470487, 74.76700034518467, 74.43907490507422, 74.19744563341388, 73.69692785640318, 73.50707628581291, 73.2827062478426, 72.9202623403521, 72.67863306869175, 72.2816706938212, 71.93648602002072, 71.60856057991025, 71.29789437348983, 70.93545046599931, 70.65930272695893, 70.33137728684846, 70.003451846738, 69.67552640662755, 69.45115636865722, 69.15774939592683, 68.88160165688643, 68.65723161891613, 68.53641698308596, 68.29478771142561, 68.0704176734553, 67.81152916810494, 67.5008629616845, 67.24197445633415, 67.08664135312392, 66.75871591301346, 66.55160510873317, 66.37901277183293, 66.08560579910252, 65.8612357611322, 65.65412495685192, 65.41249568519159, 65.13634794615119, 64.94649637556093, 64.67034863652054, 64.41146013117019, 64.23886779426994, 63.9972385226096, 63.61753538142907, 63.35864687607871, 63.03072143596824, 62.72005522954781, 62.3576113220573, 61.977908180876774, 61.51190887124611, 61.14946496375562, 60.890576458405256, 60.40731791508457, 60.079392474974114, 59.75146703486365, 59.423541594753196, 58.97480151881257, 58.45702450811184, 57.97376596479117, 57.455988954090444, 56.8519157749396, 56.24784259578875, 55.62651018294788, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.2909906800138, 81.17017604418363, 81.0666206420435, 80.98032447359338, 80.82499137038316, 80.68691750086296, 80.4280289955126, 80.22091819123231, 80.06558508802209, 79.94477045219193, 79.8584742837418, 79.72040041422161, 79.59958577839144, 79.47877114256127, 79.27166033828098, 79.20262340352089, 79.13358646876078, 79.09906800138074, 78.94373489817053, 78.73662409389023, 78.40869865377978, 78.21884708318952, 78.02899551259924, 77.856403175699, 77.54573696927856, 77.30410769761824, 77.13151536071798, 76.9244045564377, 76.75181221953746, 76.68277528477736, 76.44114601311702, 76.04418363824647, 75.73351743182603, 75.4228512254056, 75.26751812219537, 75.09492578529513, 74.76700034518467, 74.50811183983431, 74.35277873662409, 74.0766309975837, 73.7832240248533, 73.43803935105282, 73.19641007939248, 72.98929927511219, 72.69589230238178, 72.48878149810149, 72.31618916120124, 71.85018985157059, 71.47048671039006, 71.1425612702796, 70.84915429754919, 70.4176734552986, 70.03797031411806, 69.69278564031757, 69.39937866758716, 69.1404901622368, 68.79530548843631, 68.5018985157059, 68.15671384190543, 67.863306869175, 67.67345529858476, 67.24197445633415, 66.93130824991371, 66.68967897825337, 66.46530894028305, 66.22367966862271, 65.89575422851226, 65.61960648947188, 65.4297549188816, 65.18812564722126, 64.998274076631, 64.67034863652054, 64.37694166379013, 64.13531239212979, 63.89368312046946, 63.61753538142907, 63.46220227821885, 63.099758370728345, 62.702795995857784, 62.4956851915775, 62.254055919917164, 61.96064894718675, 61.64998274076632, 61.477390403866075, 61.13220573006559, 60.82153952364515, 60.40731791508457, 60.079392474974114, 59.75146703486365, 59.440800828443216, 59.09561615464274, 58.62961684501209, 58.2844321712116, 57.852951328961, 57.369692785640325, 56.98998964445978, 56.454953400069044, 55.95443562305834, 55.384880911287546, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.44632378322403, 81.429064549534, 81.41180531584398, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.39454608215395, 81.37728684846392, 81.27373144632378, 81.30824991370383, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.22195374525371, 81.18743527787366, 81.11839834311357, 81.03210217466345, 81.03210217466345, 81.01484294097342, 80.96306523990336, 80.89402830514325, 80.87676907145323, 80.66965826717293, 80.70417673455299, 80.61788056610285, 80.53158439765275, 80.35899206075251, 80.27269589230238, 80.11736278909216, 80.03106662064205, 79.94477045219193, 79.89299275112185, 79.70314118053159, 79.72040041422161, 79.53054884363134, 79.39247497411115, 79.28891957197101, 79.28891957197101, 79.15084570245081, 78.9955125992406, 78.87469796341043, 78.73662409389023, 78.56403175698999, 78.54677252329996, 78.37418018639973, 78.30514325163963, 78.13255091473938, 77.99447704521918, 77.82188470831896, 77.856403175699, 77.68381083879876, 77.58025543665862, 77.3558853986883, 77.16603382809804, 77.06247842595789, 76.78633068691751, 76.49292371418709, 76.30307214359682, 75.99240593717639, 75.71625819813602, 75.52640662754574, 75.37107352433551, 75.02588885053504, 74.80151881256472, 74.59440800828443, 74.2147048671039, 73.7832240248533, 73.31722471522265, 72.98929927511219, 72.50604073179151, 72.02278218847084, 71.43596824301002, 70.90093199861927, 70.48671039005868, 70.05522954780807, 69.38211943389715, 68.69175008629618, 68.12219537452538, 67.62167759751468, 67.01760441836383, 66.24093890231273, 65.68864342423197, 65.17086641353124, 64.54953400069037, 63.9972385226096, 63.22057300655851, 62.44390749050742, 61.71901967552641, 61.04590956161546, 60.200207110804286, 59.31998619261305, 58.439765274421816, 57.62858129099069, 56.748360372799446, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2026, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.39454608215395, 81.41180531584398, 81.41180531584398, 81.36002761477391, 81.36002761477391, 81.36002761477391, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.34276838108389, 81.20469451156369, 81.10113910942354, 81.10113910942354, 81.04936140835348, 81.04936140835348, 80.96306523990336, 80.89402830514325, 80.84225060407319, 80.80773213669313, 80.65239903348291, 80.4970659302727, 80.46254746289266, 80.44528822920263, 80.4280289955126, 80.4280289955126, 80.41076976182258, 80.35899206075251, 80.16914049016223, 80.13462202278218, 79.96202968588196, 79.73765964791163, 79.4960303762513, 79.39247497411115, 79.34069727304109, 79.20262340352089, 79.09906800138074, 78.84017949603039, 78.77114256127028, 78.63306869175008, 78.51225405591991, 78.39143942008975, 78.28788401794961, 78.13255091473938, 78.21884708318952, 78.02899551259924, 77.82188470831896, 77.63203313772868, 77.58025543665862, 77.33862616499827, 77.09699689333793, 76.90714532274767, 76.76907145322748, 76.68277528477736, 76.40662754573697, 76.21677597514672, 76.07870210562652, 75.69899896444599, 75.38833275802554, 75.02588885053504, 74.87055574732481, 74.50811183983431, 74.11114946496376, 73.85226095961339, 73.40352088367277, 73.00655850880221, 72.54055919917157, 72.10907835692095, 71.60856057991025, 71.03900586813945, 70.59026579219882, 70.03797031411806, 69.36486020020712, 68.81256472212634, 68.22575077666552, 67.7079737659648, 67.15567828788403, 66.44804970659303, 65.84397652744218, 65.39523645150156, 64.80842250604073, 64.13531239212979, 63.41042457714878, 62.68553676216776, 61.9951674145668, 61.25302036589575, 60.528132550914734, 59.855022437003804, 59.04383845357266, 58.33620987228167, 57.55954435623059, 56.71384190541941, 55.78184328615809, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2026, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.2909906800138, 81.2909906800138, 81.2909906800138, 81.23921297894374, 81.20469451156369, 81.1356575768036, 81.10113910942354, 81.04936140835348, 80.9975837072834, 80.96306523990336, 80.9285467725233, 80.89402830514325, 80.82499137038316, 80.80773213669313, 80.7904729030031, 80.75595443562305, 80.70417673455299, 80.72143596824301, 80.63513979979288, 80.60062133241284, 80.58336209872282, 80.53158439765275, 80.51432516396272, 80.44528822920263, 80.37625129444253, 80.35899206075251, 80.27269589230238, 80.18639972385226, 80.04832585433206, 79.97928891957199, 79.9275112185019, 79.84121505005179, 79.73765964791163, 79.54780807732136, 79.40973420780118, 79.28891957197101, 79.27166033828098, 79.13358646876078, 79.01277183293061, 78.84017949603039, 78.70210562651019, 78.47773558853987, 78.30514325163963, 78.11529168104936, 77.90818087676908, 77.71832930617882, 77.52847773558854, 77.39040386606834, 77.20055229547809, 76.97618225750776, 76.80358992060754, 76.68277528477736, 76.23403520883673, 75.95788746979635, 75.75077666551606, 75.45736969278565, 75.0776665516051, 74.66344494304454, 74.16292716603382, 73.71418709009319, 73.35174318260269, 72.97204004142216, 72.60959613393166, 72.19537452537108, 71.76389368312047, 71.38419054193994, 70.93545046599931, 70.67656196064895, 70.22782188470832, 69.79634104245773, 69.19226786330687, 68.55367621677597, 67.96686227131515, 67.51812219537453, 67.01760441836383, 66.39627200552296, 65.9302726958923, 65.46427338626165, 64.80842250604073, 64.23886779426994, 63.66931308249914, 63.01346220227821, 62.254055919917164, 61.51190887124611, 60.75250258888505, 60.25198481187435, 59.578874697963414, 58.81946841560235, 58.249913703831545, 57.54228512254056, 56.903693476009664, 56.040731791508456, 55.402140144977565, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.27373144632378, 81.10113910942354, 80.98032447359338, 80.85950983776321, 80.73869520193303, 80.70417673455299, 80.53158439765275, 80.22091819123231, 80.04832585433206, 79.996548153262, 79.87573351743183, 79.89299275112185, 79.80669658267173, 79.63410424577148, 79.4269934414912, 79.3579565067311, 79.3579565067311, 79.03003106662064, 78.9264756644805, 78.73662409389023, 78.61580945806006, 78.30514325163963, 77.95995857783915, 77.70107007248879, 77.32136693130826, 77.09699689333793, 76.80358992060754, 76.49292371418709, 76.23403520883673, 75.99240593717639, 75.71625819813602, 75.49188816016569, 75.2847773558854, 75.12944425267519, 74.93959268208492, 74.74974111149464, 74.5771487745944, 74.23196410079392, 74.0593717638937, 73.86952019330342, 73.67966862271315, 73.43803935105282, 72.95478080773215, 72.62685536762167, 72.4197445633414, 72.12633759061097, 71.83293061788056, 71.66033828098033, 71.33241284086986, 71.1425612702796, 70.98722816706939, 70.65930272695893, 70.31411805315844, 69.96893337935795, 69.6410079392475, 69.26130479806697, 68.96789782533655, 68.65723161891613, 68.34656541249569, 68.0704176734553, 67.82878840179495, 67.53538142906454, 67.15567828788403, 66.7932343803935, 66.46530894028305, 66.12012426648258, 65.82671729375215, 65.51605108733173, 65.27442181567137, 64.89471867449086, 64.58405246807042, 64.15257162581982, 63.89368312046946, 63.53123921297894, 63.117017604418365, 62.73731446323784, 62.46116672419745, 62.16775975146703, 61.839834311356576, 61.425612702795995, 61.13220573006559, 60.68346565412496, 60.39005868139456, 60.04487400759406, 59.75146703486365, 59.40628236106317, 59.026579219882635, 58.73317224715223, 58.45702450811184, 58.04280289955126, 57.83569209527097, 57.43872972040042, 56.97273041076977, 56.69658267172938, 56.38591646530895, 56.10976872626856, 55.83362098722817, 55.488436313427684, 55.12599240593718, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.18743527787366, 81.17017604418363, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.01484294097342, 80.9975837072834, 80.87676907145323, 80.82499137038316, 80.77321366931308, 80.70417673455299, 80.68691750086296, 80.68691750086296, 80.56610286503279, 80.41076976182258, 80.30721435968243, 80.23817742492233, 80.11736278909216, 80.10010355540214, 80.03106662064205, 80.08284432171212, 80.01380738695202, 79.996548153262, 79.996548153262, 79.89299275112185, 79.91025198481188, 79.8584742837418, 79.84121505005179, 79.77217811529168, 79.70314118053159, 79.58232654470142, 79.39247497411115, 79.27166033828098, 79.18536416983086, 79.08180876769072, 78.80566102865033, 78.77114256127028, 78.65032792544011, 78.47773558853987, 78.30514325163963, 78.11529168104936, 78.06351397997929, 77.89092164307905, 77.58025543665862, 77.25232999654816, 76.95892302381775, 76.71729375215742, 76.51018294787711, 76.33759061097687, 76.13047980669658, 76.04418363824647, 75.88885053503624, 75.71625819813602, 75.35381429064549, 75.0604073179151, 74.5771487745944, 74.18018639972385, 73.97307559544356, 73.43803935105282, 73.12737314463237, 72.71315153607179, 72.21263375906109, 71.7811529168105, 71.2806351397998, 70.84915429754919, 70.2795995857784, 69.74456334138765, 69.26130479806697, 68.65723161891613, 68.1394546082154, 67.53538142906454, 66.8622713151536, 66.10286503279255, 65.5678287884018, 64.94649637556093, 64.25612702795996, 63.53123921297894, 62.90990680013808, 62.2885743872972, 61.52916810493614, 60.87331722471523, 60.09665170866414, 59.31998619261305, 58.59509837763204, 57.680358992060754, 56.73110113910942, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.44632378322403, 81.51536071798412, 81.44632378322403, 81.48084225060408, 81.36002761477391, 81.15291681049361, 81.17017604418363, 80.9975837072834, 80.75595443562305, 80.63513979979288, 80.47980669658268, 80.47980669658268, 80.46254746289266, 80.34173282706249, 80.30721435968243, 80.10010355540214, 79.97928891957199, 79.72040041422161, 79.70314118053159, 79.39247497411115, 79.21988263721092, 78.9955125992406, 78.8574387297204, 78.71936486020022, 78.42595788746979, 78.23610631687953, 78.18432861580946, 77.77010700724888, 77.40766309975837, 77.26958923023818, 77.01070072488781, 76.76907145322748, 76.51018294787711, 76.33759061097687, 76.11322057300656, 75.88885053503624, 75.54366586123577, 75.19848118743528, 74.93959268208492, 74.54263030721437, 74.18018639972385, 73.81774249223335, 73.47255781843286, 73.16189161201243, 72.7821884708319, 72.33344839489126, 72.04004142216085, 71.59130134622022, 71.29789437348983, 70.98722816706939, 70.62478425957887, 70.26234035208837, 69.91715567828788, 69.55471177079737, 69.26130479806697, 68.84708318950639, 68.53641698308596, 68.24301001035555, 67.94960303762512, 67.5698998964446, 67.25923369002416, 66.93130824991371, 66.67241974456334, 66.30997583707283, 66.03382809803244, 65.68864342423197, 65.48153261995168, 65.27442181567137, 65.05005177770107, 64.77390403866067, 64.48049706593028, 64.23886779426994, 63.97997928891957, 63.7901277183293, 63.61753538142907, 63.23783224024854, 62.9962029685882, 62.66827752847773, 62.40938902312738, 62.09872281670694, 61.701760441836385, 61.391094235415956, 61.13220573006559, 60.890576458405256, 60.54539178460476, 60.18294787711426, 59.87228167069383, 59.47531929582327, 59.00931998619261, 58.68139454608215, 58.33620987228167, 58.077321366931315, 57.69761822575078, 57.2833966171902, 56.92095270969969, 56.62754573696929, 56.14428719364861, 55.83362098722817, 55.402140144977565, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.51536071798412, 81.41180531584398, 81.429064549534, 81.44632378322403, 81.30824991370383, 81.27373144632378, 81.22195374525371, 81.04936140835348, 80.9975837072834, 80.75595443562305, 80.68691750086296, 80.70417673455299, 80.60062133241284, 80.47980669658268, 80.25543665861235, 80.03106662064205, 80.03106662064205, 80.01380738695202, 79.8584742837418, 79.56506731101139, 79.40973420780118, 79.32343803935106, 79.09906800138074, 78.71936486020022, 78.59855022437004, 78.40869865377978, 78.14981014842941, 77.90818087676908, 77.75284777355886, 77.59751467034863, 77.20055229547809, 76.97618225750776, 76.75181221953746, 76.423886779427, 76.07870210562652, 75.7852951328961, 75.4228512254056, 75.12944425267519, 74.73248187780463, 74.23196410079392, 73.85226095961339, 73.47255781843286, 73.09285467725233, 72.86848463928202, 72.50604073179151, 72.22989299275112, 71.90196755264067, 71.55678287884018, 71.0735243355195, 70.83189506385916, 70.5039696237487, 70.17604418363825, 69.7790818087677, 69.52019330341733, 69.15774939592683, 68.89886089057646, 68.51915774939593, 68.12219537452538, 67.7079737659648, 67.43182602692441, 67.08664135312392, 66.7069382119434, 66.46530894028305, 66.24093890231273, 66.08560579910252, 65.8612357611322, 65.70590265792198, 65.53331032102174, 65.36071798412151, 65.17086641353124, 64.998274076631, 64.84294097342078, 64.65308940283052, 64.44597859855023, 64.29064549534, 64.0662754573697, 63.824646185709355, 63.56575768035899, 63.375906109768735, 63.117017604418365, 62.7890921643079, 62.40938902312738, 62.02968588194684, 61.73627890921644, 61.25302036589575, 60.95961339316535, 60.57991025198481, 60.18294787711426, 59.7859855022437, 59.38902312737315, 59.00931998619261, 58.577839143942015, 58.12909906800139, 57.6458405246807, 57.179841215050054, 56.66206420434933, 56.092509492578536, 55.52295478080773, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.30824991370383, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.18743527787366, 81.17017604418363, 81.15291681049361, 81.1356575768036, 81.10113910942354, 81.01484294097342, 80.9975837072834, 80.87676907145323, 80.82499137038316, 80.77321366931308, 80.70417673455299, 80.68691750086296, 80.68691750086296, 80.56610286503279, 80.41076976182258, 80.30721435968243, 80.23817742492233, 80.11736278909216, 80.10010355540214, 80.03106662064205, 80.08284432171212, 80.01380738695202, 79.996548153262, 79.996548153262, 79.89299275112185, 79.91025198481188, 79.8584742837418, 79.84121505005179, 79.77217811529168, 79.70314118053159, 79.58232654470142, 79.39247497411115, 79.27166033828098, 79.18536416983086, 79.08180876769072, 78.80566102865033, 78.77114256127028, 78.65032792544011, 78.47773558853987, 78.30514325163963, 78.11529168104936, 78.06351397997929, 77.89092164307905, 77.58025543665862, 77.25232999654816, 76.95892302381775, 76.71729375215742, 76.51018294787711, 76.33759061097687, 76.13047980669658, 76.04418363824647, 75.88885053503624, 75.71625819813602, 75.35381429064549, 75.0604073179151, 74.5771487745944, 74.18018639972385, 73.97307559544356, 73.43803935105282, 73.12737314463237, 72.71315153607179, 72.21263375906109, 71.7811529168105, 71.2806351397998, 70.84915429754919, 70.2795995857784, 69.74456334138765, 69.26130479806697, 68.65723161891613, 68.1394546082154, 67.53538142906454, 66.8622713151536, 66.10286503279255, 65.5678287884018, 64.94649637556093, 64.25612702795996, 63.53123921297894, 62.90990680013808, 62.2885743872972, 61.52916810493614, 60.87331722471523, 60.09665170866414, 59.31998619261305, 58.59509837763204, 57.680358992060754, 56.73110113910942, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2027, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.30824991370383, 81.27373144632378, 81.27373144632378, 81.27373144632378, 81.22195374525371, 81.18743527787366, 81.11839834311357, 81.0666206420435, 81.0666206420435, 80.9975837072834, 80.9975837072834, 80.94580600621333, 80.9285467725233, 80.80773213669313, 80.75595443562305, 80.70417673455299, 80.72143596824301, 80.66965826717293, 80.60062133241284, 80.53158439765275, 80.44528822920263, 80.41076976182258, 80.37625129444253, 80.35899206075251, 80.32447359337246, 80.32447359337246, 80.27269589230238, 80.13462202278218, 80.04832585433206, 80.04832585433206, 79.96202968588196, 79.91025198481188, 79.8584742837418, 79.75491888160165, 79.70314118053159, 79.58232654470142, 79.40973420780118, 79.21988263721092, 79.09906800138074, 78.94373489817053, 78.8574387297204, 78.65032792544011, 78.40869865377978, 78.14981014842941, 77.99447704521918, 77.87366240938903, 77.70107007248879, 77.63203313772868, 77.30410769761824, 77.02795995857784, 76.90714532274767, 76.66551605108734, 76.423886779427, 76.21677597514672, 75.94062823610632, 75.68173973075596, 75.30203658957542, 74.87055574732481, 74.76700034518467, 74.40455643769417, 74.14566793234381, 73.73144632378322, 73.31722471522265, 72.83396617190196, 72.40248532965137, 72.10907835692095, 71.66033828098033, 71.10804280289955, 70.6420434932689, 70.03797031411806, 69.52019330341733, 69.08871246116672, 68.65723161891613, 68.15671384190543, 67.62167759751468, 67.138419054194, 66.67241974456334, 66.17190196755264, 65.55056955471177, 64.80842250604073, 64.10079392474974, 63.4276838108388, 62.65101829478772, 61.8570935450466, 61.09768726268554, 60.33828098032448, 59.59613393165344, 58.81946841560235, 58.00828443217121, 57.38695201933035, 56.55850880220918, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2027, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.44632378322403, 81.46358301691406, 81.48084225060408, 81.46358301691406, 81.4981014842941, 81.4981014842941, 81.48084225060408, 81.429064549534, 81.39454608215395, 81.34276838108389, 81.32550914739386, 81.27373144632378, 81.2909906800138, 81.27373144632378, 81.23921297894374, 81.22195374525371, 81.22195374525371, 81.18743527787366, 81.10113910942354, 81.0666206420435, 81.04936140835348, 80.94580600621333, 80.89402830514325, 80.75595443562305, 80.68691750086296, 80.60062133241284, 80.60062133241284, 80.68691750086296, 80.61788056610285, 80.54884363134278, 80.56610286503279, 80.41076976182258, 80.34173282706249, 80.23817742492233, 80.23817742492233, 80.16914049016223, 80.08284432171212, 80.08284432171212, 79.91025198481188, 79.82395581636176, 79.77217811529168, 79.59958577839144, 79.58232654470142, 79.46151190887124, 79.28891957197101, 79.13358646876078, 79.01277183293061, 78.8574387297204, 78.71936486020022, 78.39143942008975, 78.16706938211944, 77.97721781152917, 77.68381083879876, 77.54573696927856, 77.45944080082845, 77.23507076285813, 77.06247842595789, 76.76907145322748, 76.35484984466689, 76.11322057300656, 75.88885053503624, 75.57818432861582, 75.0776665516051, 74.7842595788747, 74.45633413876425, 74.14566793234381, 73.80048325854332, 73.4207801173628, 72.98929927511219, 72.55781843286158, 72.17811529168105, 71.83293061788056, 71.29789437348983, 70.83189506385916, 70.43493268898861, 69.89989644459786, 69.45115636865722, 68.95063859164652, 68.41560234725578, 67.863306869175, 67.38004832585433, 66.77597514670349, 66.05108733172247, 65.44701415257163, 64.70486710390058, 64.08353469105973, 63.58301691404902, 62.90990680013808, 62.426648256817394, 61.839834311356576, 61.114946496375566, 60.3555402140145, 59.69968933379358, 59.00931998619261, 58.2844321712116, 57.6458405246807, 56.92095270969969, 56.24784259578875, 55.591991715567836, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.37728684846392, 81.20469451156369, 81.10113910942354, 80.9975837072834, 80.84225060407319, 80.66965826717293, 80.54884363134278, 80.25543665861235, 79.996548153262, 79.8584742837418, 79.75491888160165, 79.89299275112185, 79.87573351743183, 79.65136347946151, 79.53054884363134, 79.25440110459095, 79.11632723507076, 78.97825336555057, 78.87469796341043, 78.71936486020022, 78.42595788746979, 78.06351397997929, 77.82188470831896, 77.54573696927856, 77.21781152916812, 76.97618225750776, 76.78633068691751, 76.68277528477736, 76.423886779427, 76.11322057300656, 75.88885053503624, 75.62996202968588, 75.37107352433551, 75.0776665516051, 74.97411114946496, 74.5598895409044, 74.33551950293406, 74.0766309975837, 73.81774249223335, 73.558853986883, 73.29996548153262, 73.0755954435623, 72.86848463928202, 72.66137383500173, 72.35070762858129, 71.98826372109079, 71.71211598205039, 71.57404211253021, 71.33241284086986, 71.03900586813945, 70.7801173627891, 70.34863652053849, 70.17604418363825, 69.93441491197791, 69.69278564031757, 69.43389713496721, 69.17500862961685, 68.7780462547463, 68.43286158094581, 68.10493614083535, 67.75975146703487, 67.5008629616845, 67.22471522264412, 66.94856748360372, 66.67241974456334, 66.34449430445288, 66.01656886434243, 65.58508802209182, 65.23990334829134, 64.84294097342078, 64.37694166379013, 64.11805315843976, 63.75560925094926, 63.30686917500863, 63.047980669658266, 62.59924059371764, 62.2885743872972, 61.96064894718675, 61.63272350707629, 61.27027959958578, 60.90783569209527, 60.59716948567484, 60.33828098032448, 60.06213324128409, 59.69968933379358, 59.354504659993104, 58.97480151881257, 58.59509837763204, 58.23265447014153, 57.870210562651025, 57.6458405246807, 57.317915084570245, 57.07628581290991, 56.71384190541941, 56.316879530548846, 56.00621332412841, 55.74732481877804, 55.41939937866759, 55.108733172247156, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.32550914739386, 81.32550914739386, 81.2909906800138, 81.2909906800138, 81.27373144632378, 81.23921297894374, 81.20469451156369, 81.11839834311357, 81.0666206420435, 81.04936140835348, 80.9285467725233, 80.85950983776321, 80.84225060407319, 80.75595443562305, 80.68691750086296, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.39351052813255, 80.37625129444253, 80.30721435968243, 80.32447359337246, 80.18639972385226, 80.18639972385226, 80.13462202278218, 79.89299275112185, 79.82395581636176, 79.75491888160165, 79.59958577839144, 79.37521574042113, 79.23714187090093, 79.08180876769072, 78.94373489817053, 78.77114256127028, 78.68484639282016, 78.56403175698999, 78.49499482222988, 78.33966171901969, 78.30514325163963, 78.16706938211944, 77.94269934414912, 77.73558853986883, 77.56299620296859, 77.47670003451846, 77.28684846392821, 77.07973765964792, 76.82084915429755, 76.68277528477736, 76.61373835001726, 76.51018294787711, 76.21677597514672, 76.18225750776666, 75.99240593717639, 75.61270279599586, 75.44011045909562, 75.14670348636521, 74.90507421470487, 74.80151881256472, 74.5598895409044, 74.38729720400414, 74.12840869865379, 73.64515015533311, 73.36900241629272, 72.95478080773215, 72.48878149810149, 72.21263375906109, 71.72937521574042, 71.2806351397998, 70.72833966171902, 70.1415257162582, 69.52019330341733, 69.0024162927166, 68.570935450466, 67.93234380393511, 67.32827062478425, 66.87953054884363, 66.25819813600276, 65.70590265792198, 64.94649637556093, 64.32516396272005, 63.738350017259236, 62.97894373489817, 62.23679668622714, 61.615464273386266, 60.925094925785295, 60.11391094235417, 59.23369002416293, 58.47428374180187, 57.594062823610635, 56.748360372799446, 55.79910251984812, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.4981014842941, 81.2909906800138, 81.27373144632378, 81.08387987573352, 80.87676907145323, 80.80773213669313, 80.61788056610285, 80.44528822920263, 80.2899551259924, 80.15188125647221, 79.84121505005179, 79.61684501208146, 79.40973420780118, 79.28891957197101, 79.09906800138074, 78.8574387297204, 78.56403175698999, 78.30514325163963, 78.02899551259924, 77.92544011045909, 77.44218156713842, 77.26958923023818, 76.95892302381775, 76.70003451846739, 76.54470141525717, 76.32033137728685, 76.04418363824647, 75.88885053503624, 75.68173973075596, 75.49188816016569, 75.23299965481533, 75.11218501898516, 74.88781498101484, 74.66344494304454, 74.5598895409044, 74.38729720400414, 74.0766309975837, 73.7832240248533, 73.61063168795306, 73.2827062478426, 72.97204004142216, 72.52329996548154, 72.33344839489126, 72.05730065585088, 71.6430790472903, 71.33241284086986, 70.98722816706939, 70.6420434932689, 70.33137728684846, 70.05522954780807, 69.7790818087677, 69.5719710044874, 69.2095270969969, 68.84708318950639, 68.58819468415602, 68.29478771142561, 67.94960303762512, 67.65619606489473, 67.3627890921643, 67.08664135312392, 66.7932343803935, 66.46530894028305, 66.10286503279255, 65.80945806006213, 65.39523645150156, 64.998274076631, 64.72212633759061, 64.44597859855023, 64.15257162581982, 63.738350017259236, 63.393165343458755, 63.06523990334829, 62.616499827407665, 62.3576113220573, 62.01242664825681, 61.63272350707629, 61.339316534345876, 60.976872626855375, 60.64894718674492, 60.39005868139456, 60.06213324128409, 59.803244735933724, 59.440800828443216, 59.14739385571281, 58.92302381774249, 58.73317224715223, 58.49154297549188, 58.2844321712116, 57.88746979634105, 57.57680358992061, 57.30065585088022, 56.97273041076977, 56.76561960648947, 56.55850880220918, 56.33413876423887, 56.02347255781843, 55.74732481877804, 55.5056955471177, 55.12599240593718, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.41180531584398, 81.46358301691406, 81.51536071798412, 81.25647221263375, 81.11839834311357, 80.94580600621333, 80.96306523990336, 80.63513979979288, 80.56610286503279, 80.4280289955126, 80.25543665861235, 80.18639972385226, 79.96202968588196, 79.63410424577148, 79.4960303762513, 79.40973420780118, 79.21988263721092, 78.8574387297204, 78.65032792544011, 78.30514325163963, 78.13255091473938, 77.92544011045909, 77.92544011045909, 77.856403175699, 77.70107007248879, 77.44218156713842, 77.44218156713842, 77.14877459440801, 77.13151536071798, 76.90714532274767, 76.75181221953746, 76.51018294787711, 76.23403520883673, 75.87159130134621, 75.61270279599586, 75.25025888850536, 75.04314808422507, 74.7842595788747, 74.5598895409044, 74.16292716603382, 73.7832240248533, 73.36900241629272, 73.04107697618227, 72.6441146013117, 72.40248532965137, 72.10907835692095, 71.81567138419054, 71.40144977562997, 71.0735243355195, 70.72833966171902, 70.40041422160857, 70.15878494994823, 69.81360027614774, 69.52019330341733, 69.19226786330687, 68.93337935795651, 68.62271315153608, 68.2775284777356, 67.89782533655506, 67.65619606489473, 67.3627890921643, 67.17293752157406, 66.87953054884363, 66.65516051087333, 66.48256817397308, 66.25819813600276, 66.05108733172247, 65.84397652744218, 65.53331032102174, 65.30894028305143, 65.11908871246118, 64.80842250604073, 64.63583016914049, 64.39420089748016, 64.18709009319986, 63.91094235415948, 63.56575768035899, 63.30686917500863, 63.047980669658266, 62.72005522954781, 62.46116672419745, 62.18501898515706, 61.891612012426656, 61.64998274076632, 61.30479806696583, 61.04590956161546, 60.75250258888505, 60.2865032792544, 59.95857783914394, 59.61339316534347, 59.19917155678288, 58.8539868829824, 58.40524680704176, 58.04280289955126, 57.61132205730066, 57.04176734552986, 56.593027269589236, 56.092509492578536, 55.54021401449776, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.36002761477391, 81.32550914739386, 81.32550914739386, 81.2909906800138, 81.2909906800138, 81.27373144632378, 81.23921297894374, 81.20469451156369, 81.11839834311357, 81.0666206420435, 81.04936140835348, 80.9285467725233, 80.85950983776321, 80.84225060407319, 80.75595443562305, 80.68691750086296, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.39351052813255, 80.37625129444253, 80.30721435968243, 80.32447359337246, 80.18639972385226, 80.18639972385226, 80.13462202278218, 79.89299275112185, 79.82395581636176, 79.75491888160165, 79.59958577839144, 79.37521574042113, 79.23714187090093, 79.08180876769072, 78.94373489817053, 78.77114256127028, 78.68484639282016, 78.56403175698999, 78.49499482222988, 78.33966171901969, 78.30514325163963, 78.16706938211944, 77.94269934414912, 77.73558853986883, 77.56299620296859, 77.47670003451846, 77.28684846392821, 77.07973765964792, 76.82084915429755, 76.68277528477736, 76.61373835001726, 76.51018294787711, 76.21677597514672, 76.18225750776666, 75.99240593717639, 75.61270279599586, 75.44011045909562, 75.14670348636521, 74.90507421470487, 74.80151881256472, 74.5598895409044, 74.38729720400414, 74.12840869865379, 73.64515015533311, 73.36900241629272, 72.95478080773215, 72.48878149810149, 72.21263375906109, 71.72937521574042, 71.2806351397998, 70.72833966171902, 70.1415257162582, 69.52019330341733, 69.0024162927166, 68.570935450466, 67.93234380393511, 67.32827062478425, 66.87953054884363, 66.25819813600276, 65.70590265792198, 64.94649637556093, 64.32516396272005, 63.738350017259236, 62.97894373489817, 62.23679668622714, 61.615464273386266, 60.925094925785295, 60.11391094235417, 59.23369002416293, 58.47428374180187, 57.594062823610635, 56.748360372799446, 55.79910251984812, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2028, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.39454608215395, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.34276838108389, 81.32550914739386, 81.30824991370383, 81.30824991370383, 81.27373144632378, 81.20469451156369, 81.15291681049361, 81.11839834311357, 81.11839834311357, 81.10113910942354, 80.98032447359338, 80.96306523990336, 80.85950983776321, 80.73869520193303, 80.68691750086296, 80.61788056610285, 80.60062133241284, 80.53158439765275, 80.4970659302727, 80.53158439765275, 80.47980669658268, 80.4280289955126, 80.35899206075251, 80.22091819123231, 80.10010355540214, 79.97928891957199, 79.84121505005179, 79.68588194684156, 79.44425267518123, 79.37521574042113, 79.18536416983086, 79.11632723507076, 78.97825336555057, 78.84017949603039, 78.71936486020022, 78.51225405591991, 78.33966171901969, 78.23610631687953, 78.11529168104936, 77.92544011045909, 77.73558853986883, 77.52847773558854, 77.47670003451846, 77.28684846392821, 77.04521919226787, 76.95892302381775, 76.82084915429755, 76.64825681739731, 76.423886779427, 76.13047980669658, 76.07870210562652, 75.76803589920608, 75.71625819813602, 75.49188816016569, 75.4228512254056, 75.26751812219537, 75.04314808422507, 74.61166724197446, 74.2147048671039, 73.85226095961339, 73.576113220573, 73.11011391094236, 72.6441146013117, 72.31618916120124, 71.90196755264067, 71.41870900932, 70.98722816706939, 70.46945115636866, 69.91715567828788, 69.2095270969969, 68.74352778736625, 68.15671384190543, 67.67345529858476, 67.12115982050398, 66.62064204349328, 65.96479116327235, 65.44701415257163, 64.87745944080083, 64.25612702795996, 63.56575768035899, 62.87538833275803, 62.115982050396966, 61.23576113220574, 60.493614083534695, 59.83776320331378, 59.026579219882635, 58.301691404901625, 57.55954435623059, 56.748360372799446, 55.79910251984812, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2028, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.37728684846392, 81.36002761477391, 81.34276838108389, 81.37728684846392, 81.27373144632378, 81.25647221263375, 81.22195374525371, 81.17017604418363, 81.1356575768036, 81.10113910942354, 81.04936140835348, 81.03210217466345, 81.03210217466345, 81.01484294097342, 80.94580600621333, 80.84225060407319, 80.70417673455299, 80.66965826717293, 80.61788056610285, 80.56610286503279, 80.53158439765275, 80.39351052813255, 80.35899206075251, 80.35899206075251, 80.4280289955126, 80.35899206075251, 80.27269589230238, 80.20365895754229, 80.20365895754229, 80.15188125647221, 80.10010355540214, 79.97928891957199, 79.9275112185019, 79.82395581636176, 79.7894373489817, 79.68588194684156, 79.63410424577148, 79.61684501208146, 79.3579565067311, 79.18536416983086, 79.11632723507076, 78.9264756644805, 78.80566102865033, 78.61580945806006, 78.32240248532966, 78.21884708318952, 77.95995857783915, 77.82188470831896, 77.71832930617882, 77.4249223334484, 77.26958923023818, 77.18329306178806, 76.95892302381775, 76.63099758370728, 76.59647911632723, 76.33759061097687, 76.06144287193649, 75.87159130134621, 75.59544356230583, 75.44011045909562, 75.19848118743528, 74.88781498101484, 74.68070417673457, 74.30100103555402, 73.95581636175355, 73.59337245426303, 73.26544701415257, 72.79944770452192, 72.36796686227132, 72.10907835692095, 71.60856057991025, 71.17707973765965, 70.69382119433898, 70.1415257162582, 69.7100448740076, 69.15774939592683, 68.65723161891613, 68.08767690714532, 67.46634449430445, 66.87953054884363, 66.2927166033828, 65.74042112530203, 65.05005177770107, 64.44597859855023, 63.738350017259236, 63.099758370728345, 62.392129789437355, 61.64998274076632, 60.787021056265104, 60.01035554021401, 59.23369002416293, 58.47428374180187, 57.870210562651025, 57.12806351397999, 56.38591646530895, 55.678287884017955, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.30824991370383, 81.20469451156369, 80.9285467725233, 80.9285467725233, 80.85950983776321, 80.75595443562305, 80.65239903348291, 80.54884363134278, 80.46254746289266, 80.41076976182258, 80.46254746289266, 80.30721435968243, 80.10010355540214, 79.96202968588196, 79.82395581636176, 79.75491888160165, 79.63410424577148, 79.37521574042113, 79.25440110459095, 78.97825336555057, 78.82292026234036, 78.58129099068002, 78.49499482222988, 78.20158784949949, 77.92544011045909, 77.59751467034863, 77.33862616499827, 77.23507076285813, 77.04521919226787, 76.78633068691751, 76.61373835001726, 76.38936831204694, 75.94062823610632, 75.68173973075596, 75.44011045909562, 75.23299965481533, 74.97411114946496, 74.68070417673457, 74.40455643769417, 74.14566793234381, 73.88677942699344, 73.73144632378322, 73.558853986883, 73.21366931308249, 72.83396617190196, 72.4197445633414, 72.16085605799103, 72.00552295478082, 71.6430790472903, 71.38419054193994, 71.0735243355195, 70.88367276492924, 70.74559889540905, 70.38315498791854, 70.08974801518812, 69.84811874352779, 69.45115636865722, 69.05419399378667, 68.69175008629618, 68.3638246461857, 68.0013807386952, 67.7079737659648, 67.24197445633415, 67.0003451846738, 66.74145667932343, 66.51708664135313, 66.22367966862271, 65.84397652744218, 65.48153261995168, 65.15360717984122, 64.89471867449086, 64.5667932343804, 64.20434932688988, 63.9972385226096, 63.51397997928891, 63.272350707628576, 62.94442526751812, 62.59924059371764, 62.2885743872972, 61.87435277873663, 61.54642733862617, 61.14946496375562, 60.83879875733518, 60.45909561615465, 60.09665170866414, 59.76872626855367, 59.423541594753196, 59.11287538833277, 58.69865377977218, 58.2844321712116, 57.83569209527097, 57.50776665516051, 57.23161891612012, 56.8519157749396, 56.368657231618926, 56.092509492578536, 55.73006558508802, 55.31584397652745, 54.98791853641698, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.25647221263375, 81.23921297894374, 81.18743527787366, 81.1356575768036, 81.15291681049361, 81.0666206420435, 80.9975837072834, 80.9975837072834, 80.91128753883328, 80.87676907145323, 80.7904729030031, 80.77321366931308, 80.7904729030031, 80.72143596824301, 80.66965826717293, 80.53158439765275, 80.4970659302727, 80.46254746289266, 80.4280289955126, 80.35899206075251, 80.22091819123231, 80.27269589230238, 80.16914049016223, 80.10010355540214, 80.08284432171212, 80.03106662064205, 79.91025198481188, 79.87573351743183, 79.80669658267173, 79.58232654470142, 79.4960303762513, 79.30617880566103, 79.18536416983086, 78.94373489817053, 78.77114256127028, 78.70210562651019, 78.44321712115982, 78.20158784949949, 78.20158784949949, 77.97721781152917, 77.77010700724888, 77.63203313772868, 77.47670003451846, 77.37314463237833, 77.28684846392821, 77.14877459440801, 76.94166379012772, 76.73455298584743, 76.45840524680705, 76.23403520883673, 76.07870210562652, 75.76803589920608, 75.49188816016569, 75.18122195374526, 74.9223334483949, 74.7842595788747, 74.38729720400414, 74.2147048671039, 73.80048325854332, 73.4207801173628, 73.17915084570245, 72.79944770452192, 72.19537452537108, 71.88470831895064, 71.47048671039006, 70.95270969968934, 70.4176734552986, 69.96893337935795, 69.46841560234725, 68.96789782533655, 68.39834311356576, 67.82878840179495, 67.24197445633415, 66.6379012771833, 65.91301346220229, 65.27442181567137, 64.63583016914049, 63.9282015878495, 63.25509147393856, 62.53020365895754, 61.822575077666556, 61.06316879530549, 60.25198481187435, 59.38902312737315, 58.54332067656196, 57.66309975837073, 56.76561960648947, 55.78184328615809, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.46358301691406, 81.429064549534, 81.30824991370383, 81.11839834311357, 80.9285467725233, 80.66965826717293, 80.41076976182258, 80.25543665861235, 80.20365895754229, 80.10010355540214, 79.8584742837418, 79.87573351743183, 79.72040041422161, 79.51328960994132, 79.40973420780118, 79.37521574042113, 79.15084570245081, 79.06454953400069, 78.84017949603039, 78.87469796341043, 78.61580945806006, 78.44321712115982, 78.25336555056955, 78.02899551259924, 77.82188470831896, 77.51121850189851, 77.23507076285813, 76.97618225750776, 76.8553676216776, 76.66551605108734, 76.47566448049706, 76.23403520883673, 75.9233690024163, 75.59544356230583, 75.33655505695546, 74.97411114946496, 74.74974111149464, 74.47359337245426, 74.24922333448394, 73.9212978943735, 73.50707628581291, 73.12737314463237, 72.85122540559199, 72.60959613393166, 72.143596824301, 71.71211598205039, 71.36693130824992, 70.93545046599931, 70.60752502588885, 70.33137728684846, 69.98619261304798, 69.6410079392475, 69.43389713496721, 69.19226786330687, 68.82982395581637, 68.58819468415602, 68.26026924404557, 67.98412150500518, 67.62167759751468, 67.32827062478425, 66.91404901622369, 66.62064204349328, 66.34449430445288, 66.05108733172247, 65.75768035899206, 65.55056955471177, 65.23990334829134, 64.98101484294098, 64.58405246807042, 64.30790472903003, 64.13531239212979, 63.97997928891957, 63.70383154987918, 63.35864687607871, 63.047980669658266, 62.702795995857784, 62.478425957887474, 62.18501898515706, 62.01242664825681, 61.64998274076632, 61.40835346910597, 61.14946496375562, 60.94235415947532, 60.57991025198481, 60.303762512944424, 60.079392474974114, 59.76872626855367, 59.527096996893334, 59.19917155678288, 58.8539868829824, 58.50880220918191, 58.11183983431136, 57.83569209527097, 57.42147048671039, 56.955471177079744, 56.43769416637902, 55.88539868829825, 55.384880911287546, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.23921297894374, 81.10113910942354, 80.85950983776321, 80.68691750086296, 80.56610286503279, 80.25543665861235, 80.10010355540214, 79.8584742837418, 79.68588194684156, 79.4269934414912, 79.21988263721092, 79.01277183293061, 78.90921643079048, 78.66758715913014, 78.3569209527097, 78.25336555056955, 77.92544011045909, 77.77010700724888, 77.52847773558854, 77.25232999654816, 77.11425612702796, 76.97618225750776, 76.83810838798757, 76.66551605108734, 76.49292371418709, 76.30307214359682, 76.21677597514672, 75.99240593717639, 75.75077666551606, 75.75077666551606, 75.62996202968588, 75.56092509492579, 75.26751812219537, 75.09492578529513, 74.83603727994476, 74.50811183983431, 74.24922333448394, 74.0593717638937, 73.80048325854332, 73.50707628581291, 73.21366931308249, 72.86848463928202, 72.67863306869175, 72.4197445633414, 72.12633759061097, 71.81567138419054, 71.45322747670004, 71.17707973765965, 70.79737659647913, 70.36589575422852, 70.0724887814981, 69.81360027614774, 69.55471177079737, 69.33034173282707, 68.95063859164652, 68.51915774939593, 68.2775284777356, 68.0013807386952, 67.7079737659648, 67.22471522264412, 66.98308595098378, 66.60338280980325, 66.32723507076285, 66.03382809803244, 65.82671729375215, 65.5678287884018, 65.37797721781153, 65.17086641353124, 64.9292371418709, 64.5667932343804, 64.30790472903003, 63.91094235415948, 63.548498446668965, 63.30686917500863, 63.117017604418365, 62.7890921643079, 62.4956851915775, 62.16775975146703, 61.87435277873663, 61.66724197445633, 61.40835346910597, 61.114946496375566, 60.8560579910252, 60.54539178460476, 60.32102174663445, 59.97583707283397, 59.630652399033494, 59.33724542630308, 59.026579219882635, 58.7849499482223, 58.42250604073179, 58.06006213324129, 57.71487745944081, 57.248878149810146, 56.869175008629625, 56.55850880220918, 56.21332412840869, 55.74732481877804, 55.229547808077314, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.34276838108389, 81.32550914739386, 81.2909906800138, 81.27373144632378, 81.25647221263375, 81.25647221263375, 81.23921297894374, 81.18743527787366, 81.1356575768036, 81.15291681049361, 81.0666206420435, 80.9975837072834, 80.9975837072834, 80.91128753883328, 80.87676907145323, 80.7904729030031, 80.77321366931308, 80.7904729030031, 80.72143596824301, 80.66965826717293, 80.53158439765275, 80.4970659302727, 80.46254746289266, 80.4280289955126, 80.35899206075251, 80.22091819123231, 80.27269589230238, 80.16914049016223, 80.10010355540214, 80.08284432171212, 80.03106662064205, 79.91025198481188, 79.87573351743183, 79.80669658267173, 79.58232654470142, 79.4960303762513, 79.30617880566103, 79.18536416983086, 78.94373489817053, 78.77114256127028, 78.70210562651019, 78.44321712115982, 78.20158784949949, 78.20158784949949, 77.97721781152917, 77.77010700724888, 77.63203313772868, 77.47670003451846, 77.37314463237833, 77.28684846392821, 77.14877459440801, 76.94166379012772, 76.73455298584743, 76.45840524680705, 76.23403520883673, 76.07870210562652, 75.76803589920608, 75.49188816016569, 75.18122195374526, 74.9223334483949, 74.7842595788747, 74.38729720400414, 74.2147048671039, 73.80048325854332, 73.4207801173628, 73.17915084570245, 72.79944770452192, 72.19537452537108, 71.88470831895064, 71.47048671039006, 70.95270969968934, 70.4176734552986, 69.96893337935795, 69.46841560234725, 68.96789782533655, 68.39834311356576, 67.82878840179495, 67.24197445633415, 66.6379012771833, 65.91301346220229, 65.27442181567137, 64.63583016914049, 63.9282015878495, 63.25509147393856, 62.53020365895754, 61.822575077666556, 61.06316879530549, 60.25198481187435, 59.38902312737315, 58.54332067656196, 57.66309975837073, 56.76561960648947, 55.78184328615809, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2029, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.37728684846392, 81.36002761477391, 81.32550914739386, 81.30824991370383, 81.2909906800138, 81.2909906800138, 81.25647221263375, 81.23921297894374, 81.18743527787366, 81.10113910942354, 81.0666206420435, 80.96306523990336, 81.01484294097342, 80.9975837072834, 80.98032447359338, 80.91128753883328, 80.87676907145323, 80.85950983776321, 80.87676907145323, 80.84225060407319, 80.82499137038316, 80.77321366931308, 80.65239903348291, 80.61788056610285, 80.54884363134278, 80.51432516396272, 80.4970659302727, 80.37625129444253, 80.34173282706249, 80.27269589230238, 80.16914049016223, 80.06558508802209, 80.01380738695202, 79.91025198481188, 79.87573351743183, 79.87573351743183, 79.72040041422161, 79.53054884363134, 79.30617880566103, 79.13358646876078, 78.97825336555057, 78.94373489817053, 78.70210562651019, 78.49499482222988, 78.39143942008975, 78.14981014842941, 78.09803244735933, 77.83914394200899, 77.61477390403866, 77.39040386606834, 77.14877459440801, 77.02795995857784, 76.90714532274767, 76.66551605108734, 76.70003451846739, 76.52744218156714, 76.23403520883673, 76.09596133931653, 75.80255436658612, 75.40559199171557, 75.11218501898516, 74.85329651363479, 74.50811183983431, 74.14566793234381, 73.7832240248533, 73.33448394891266, 72.90300310666207, 72.54055919917157, 71.98826372109079, 71.45322747670004, 71.02174663444943, 70.59026579219882, 70.03797031411806, 69.46841560234725, 69.03693476009666, 68.48463928201588, 68.03589920607524, 67.48360372799448, 67.05212288574387, 66.37901277183293, 65.72316189161201, 65.05005177770107, 64.32516396272005, 63.65205384880912, 62.927166033828094, 62.16775975146703, 61.44287193648602, 60.700724887814985, 59.90680013807387, 59.18191232309285, 58.3534691059717, 57.61132205730066, 56.76561960648947, 55.79910251984812, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2029, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.41180531584398, 81.41180531584398, 81.39454608215395, 81.429064549534, 81.41180531584398, 81.41180531584398, 81.36002761477391, 81.34276838108389, 81.2909906800138, 81.2909906800138, 81.30824991370383, 81.27373144632378, 81.23921297894374, 81.22195374525371, 81.22195374525371, 81.10113910942354, 81.08387987573352, 81.03210217466345, 80.96306523990336, 80.94580600621333, 80.96306523990336, 80.91128753883328, 80.87676907145323, 80.80773213669313, 80.75595443562305, 80.72143596824301, 80.65239903348291, 80.60062133241284, 80.63513979979288, 80.56610286503279, 80.44528822920263, 80.39351052813255, 80.23817742492233, 80.20365895754229, 80.20365895754229, 80.10010355540214, 79.996548153262, 79.75491888160165, 79.63410424577148, 79.53054884363134, 79.40973420780118, 79.25440110459095, 79.09906800138074, 79.01277183293061, 78.8574387297204, 78.68484639282016, 78.58129099068002, 78.49499482222988, 78.33966171901969, 78.27062478425958, 77.95995857783915, 77.83914394200899, 77.64929237141871, 77.54573696927856, 77.40766309975837, 77.18329306178806, 77.02795995857784, 76.68277528477736, 76.52744218156714, 76.26855367621678, 76.02692440455644, 75.73351743182603, 75.4228512254056, 75.2157404211253, 74.88781498101484, 74.62892647566449, 74.40455643769417, 73.9212978943735, 73.59337245426303, 73.24818778046254, 72.83396617190196, 72.43700379703142, 71.95374525371074, 71.31515360717984, 70.88367276492924, 70.4176734552986, 69.89989644459786, 69.46841560234725, 68.95063859164652, 68.31204694511564, 67.62167759751468, 67.12115982050398, 66.5688643424232, 65.9993096306524, 65.32619951674145, 64.80842250604073, 64.11805315843976, 63.393165343458755, 62.72005522954781, 62.02968588194684, 61.27027959958578, 60.61442871936487, 59.88954090438385, 59.16465308940283, 58.577839143942015, 57.870210562651025, 56.98998964445978, 56.23058336209872, 55.54021401449776, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.34276838108389, 81.1356575768036, 81.08387987573352, 81.01484294097342, 80.91128753883328, 80.75595443562305, 80.56610286503279, 80.4970659302727, 80.44528822920263, 80.34173282706249, 80.06558508802209, 79.87573351743183, 79.66862271315154, 79.70314118053159, 79.4960303762513, 79.30617880566103, 79.13358646876078, 78.94373489817053, 78.63306869175008, 78.54677252329996, 78.3569209527097, 78.23610631687953, 78.01173627890921, 77.90818087676908, 77.63203313772868, 77.47670003451846, 77.28684846392821, 77.09699689333793, 76.9244045564377, 76.68277528477736, 76.423886779427, 76.21677597514672, 76.00966517086641, 75.73351743182603, 75.44011045909562, 75.19848118743528, 74.93959268208492, 74.69796341042458, 74.4218156713842, 74.23196410079392, 73.99033482913359, 73.64515015533311, 73.35174318260269, 73.11011391094236, 72.90300310666207, 72.59233690024163, 72.2816706938212, 72.05730065585088, 71.67759751467035, 71.2806351397998, 70.98722816706939, 70.6420434932689, 70.29685881946843, 70.10700724887815, 69.88263721090784, 69.46841560234725, 69.08871246116672, 68.84708318950639, 68.65723161891613, 68.38108387987573, 68.0013807386952, 67.65619606489473, 67.32827062478425, 67.05212288574387, 66.62064204349328, 66.30997583707283, 65.98205039696238, 65.74042112530203, 65.37797721781153, 65.10182947877115, 64.86020020711081, 64.4977562996203, 64.15257162581982, 63.70383154987918, 63.393165343458755, 62.96168450120815, 62.63375906109769, 62.271315153607176, 61.839834311356576, 61.477390403866075, 61.25302036589575, 60.83879875733518, 60.510873317224714, 60.200207110804286, 59.88954090438385, 59.527096996893334, 59.19917155678288, 58.88850535036245, 58.50880220918191, 58.11183983431136, 57.7839143942009, 57.47324818778047, 57.07628581290991, 56.73110113910942, 56.472212633759064, 56.12702795995859, 55.83362098722817, 55.488436313427684, 55.12599240593718, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.37728684846392, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.22195374525371, 81.18743527787366, 81.1356575768036, 81.10113910942354, 81.04936140835348, 81.01484294097342, 80.94580600621333, 80.91128753883328, 80.91128753883328, 80.85950983776321, 80.75595443562305, 80.65239903348291, 80.58336209872282, 80.53158439765275, 80.41076976182258, 80.35899206075251, 80.2899551259924, 80.15188125647221, 80.04832585433206, 80.01380738695202, 80.01380738695202, 79.91025198481188, 79.89299275112185, 79.84121505005179, 79.7894373489817, 79.65136347946151, 79.51328960994132, 79.30617880566103, 79.23714187090093, 79.09906800138074, 78.89195719710045, 78.80566102865033, 78.54677252329996, 78.33966171901969, 78.23610631687953, 78.14981014842941, 77.856403175699, 77.71832930617882, 77.47670003451846, 77.44218156713842, 77.28684846392821, 77.18329306178806, 77.01070072488781, 76.88988608905764, 76.70003451846739, 76.45840524680705, 76.28581290990681, 76.13047980669658, 75.87159130134621, 75.76803589920608, 75.68173973075596, 75.56092509492579, 75.19848118743528, 75.09492578529513, 74.81877804625475, 74.54263030721437, 74.18018639972385, 73.85226095961339, 73.36900241629272, 72.93752157404212, 72.54055919917157, 72.10907835692095, 71.67759751467035, 71.21159820503969, 70.65930272695893, 70.02071108042803, 69.58923023817742, 68.93337935795651, 68.32930617880567, 67.94960303762512, 67.34552985847428, 66.77597514670349, 66.30997583707283, 65.75768035899206, 64.94649637556093, 64.46323783224025, 63.66931308249914, 62.87538833275803, 62.16775975146703, 61.391094235415956, 60.63168795305489, 59.94131860545392, 59.18191232309285, 58.40524680704176, 57.61132205730066, 56.71384190541941, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.39454608215395, 81.41180531584398, 81.23921297894374, 81.22195374525371, 81.11839834311357, 80.96306523990336, 81.04936140835348, 80.96306523990336, 80.87676907145323, 80.87676907145323, 80.72143596824301, 80.53158439765275, 80.39351052813255, 80.35899206075251, 80.22091819123231, 79.96202968588196, 79.7894373489817, 79.54780807732136, 79.32343803935106, 79.3579565067311, 79.25440110459095, 79.11632723507076, 78.97825336555057, 78.82292026234036, 78.58129099068002, 78.3569209527097, 78.09803244735933, 78.01173627890921, 77.90818087676908, 77.83914394200899, 77.64929237141871, 77.47670003451846, 77.14877459440801, 76.88988608905764, 76.61373835001726, 76.40662754573697, 76.11322057300656, 76.04418363824647, 75.69899896444599, 75.35381429064549, 75.02588885053504, 74.7152226441146, 74.38729720400414, 73.93855712806352, 73.48981705212289, 73.1446323783224, 72.83396617190196, 72.54055919917157, 72.10907835692095, 71.76389368312047, 71.40144977562997, 70.93545046599931, 70.46945115636866, 70.12426648256817, 69.81360027614774, 69.46841560234725, 69.0714532274767, 68.58819468415602, 68.24301001035555, 67.98412150500518, 67.6389368312047, 67.44908526061442, 67.22471522264412, 67.08664135312392, 66.93130824991371, 66.7932343803935, 66.60338280980325, 66.37901277183293, 66.25819813600276, 66.01656886434243, 65.80945806006213, 65.65412495685192, 65.53331032102174, 65.20538488091128, 64.998274076631, 64.84294097342078, 64.63583016914049, 64.32516396272005, 64.0662754573697, 63.72109078356921, 63.44494304452882, 63.20331377286848, 62.892647566448055, 62.65101829478772, 62.392129789437355, 62.13324128408699, 61.80531584397653, 61.44287193648602, 60.994131860545394, 60.64894718674492, 60.200207110804286, 59.7859855022437, 59.371763893683124, 58.83672764929238, 58.18087676907145, 57.55954435623059, 56.955471177079744, 56.33413876423887, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.32550914739386, 81.25647221263375, 81.30824991370383, 81.27373144632378, 81.27373144632378, 81.10113910942354, 81.10113910942354, 81.04936140835348, 80.98032447359338, 80.77321366931308, 80.66965826717293, 80.56610286503279, 80.4970659302727, 80.32447359337246, 80.11736278909216, 79.94477045219193, 79.84121505005179, 79.75491888160165, 79.51328960994132, 79.37521574042113, 79.03003106662064, 78.8574387297204, 78.63306869175008, 78.49499482222988, 78.27062478425958, 78.01173627890921, 77.90818087676908, 77.70107007248879, 77.61477390403866, 77.4249223334484, 77.07973765964792, 76.97618225750776, 76.8553676216776, 76.47566448049706, 76.21677597514672, 76.00966517086641, 75.71625819813602, 75.37107352433551, 75.19848118743528, 74.9223334483949, 74.5598895409044, 74.09389023127373, 73.80048325854332, 73.62789092164309, 73.26544701415257, 72.85122540559199, 72.50604073179151, 72.07455988954091, 71.69485674836038, 71.2806351397998, 70.88367276492924, 70.62478425957887, 70.24508111839835, 69.84811874352779, 69.58923023817742, 69.17500862961685, 68.8643424231964, 68.38108387987573, 68.0013807386952, 67.6389368312047, 67.31101139109424, 67.01760441836383, 66.7932343803935, 66.55160510873317, 66.30997583707283, 66.10286503279255, 65.77493959268207, 65.5678287884018, 65.36071798412151, 65.15360717984122, 64.9292371418709, 64.70486710390058, 64.54953400069037, 64.29064549534, 64.18709009319986, 63.94546082153953, 63.72109078356921, 63.51397997928891, 63.324128408698655, 63.03072143596824, 62.7890921643079, 62.4956851915775, 62.18501898515706, 61.8570935450466, 61.615464273386266, 61.2875388332758, 61.011391094235414, 60.59716948567484, 60.37279944770453, 60.079392474974114, 59.76872626855367, 59.371763893683124, 58.95754228512254, 58.56057991025199, 58.06006213324129, 57.61132205730066, 57.024508111839836, 56.43769416637902, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.37728684846392, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.22195374525371, 81.18743527787366, 81.1356575768036, 81.10113910942354, 81.10113910942354, 81.04936140835348, 81.01484294097342, 80.96306523990336, 80.94580600621333, 80.84225060407319, 80.75595443562305, 80.73869520193303, 80.68691750086296, 80.58336209872282, 80.4970659302727, 80.39351052813255, 80.39351052813255, 80.30721435968243, 80.16914049016223, 80.06558508802209, 80.08284432171212, 80.06558508802209, 80.01380738695202, 79.87573351743183, 79.72040041422161, 79.72040041422161, 79.70314118053159, 79.59958577839144, 79.3579565067311, 79.37521574042113, 79.25440110459095, 79.13358646876078, 79.01277183293061, 78.78840179496031, 78.70210562651019, 78.46047635484985, 78.28788401794961, 78.16706938211944, 78.08077321366932, 77.856403175699, 77.71832930617882, 77.56299620296859, 77.45944080082845, 77.39040386606834, 77.06247842595789, 76.87262685536763, 76.78633068691751, 76.51018294787711, 76.40662754573697, 76.26855367621678, 76.04418363824647, 76.00966517086641, 75.75077666551606, 75.44011045909562, 75.14670348636521, 74.88781498101484, 74.59440800828443, 74.2147048671039, 73.95581636175355, 73.558853986883, 73.16189161201243, 72.5750776665516, 72.04004142216085, 71.52226441146013, 70.9181912323093, 70.26234035208837, 69.76182257507767, 69.19226786330687, 68.6399723852261, 68.0704176734553, 67.53538142906454, 66.89678978253366, 66.48256817397308, 65.89575422851226, 65.2916810493614, 64.54953400069037, 63.7901277183293, 62.96168450120815, 62.271315153607176, 61.615464273386266, 60.94235415947532, 60.14842940973422, 59.33724542630308, 58.54332067656196, 57.66309975837073, 56.71384190541941, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2030, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.429064549534, 81.429064549534, 81.429064549534, 81.39454608215395, 81.39454608215395, 81.39454608215395, 81.36002761477391, 81.32550914739386, 81.30824991370383, 81.30824991370383, 81.30824991370383, 81.2909906800138, 81.25647221263375, 81.22195374525371, 81.17017604418363, 81.1356575768036, 81.10113910942354, 80.98032447359338, 80.94580600621333, 80.9285467725233, 80.80773213669313, 80.75595443562305, 80.73869520193303, 80.65239903348291, 80.63513979979288, 80.63513979979288, 80.4970659302727, 80.35899206075251, 80.39351052813255, 80.32447359337246, 80.13462202278218, 80.06558508802209, 79.996548153262, 79.89299275112185, 79.77217811529168, 79.72040041422161, 79.61684501208146, 79.59958577839144, 79.58232654470142, 79.59958577839144, 79.39247497411115, 79.23714187090093, 78.97825336555057, 78.84017949603039, 78.70210562651019, 78.54677252329996, 78.32240248532966, 78.08077321366932, 77.80462547462893, 77.54573696927856, 77.44218156713842, 77.33862616499827, 77.25232999654816, 77.25232999654816, 77.02795995857784, 76.71729375215742, 76.49292371418709, 76.38936831204694, 76.13047980669658, 76.00966517086641, 75.81981360027615, 75.76803589920608, 75.66448049706594, 75.33655505695546, 74.97411114946496, 74.80151881256472, 74.5598895409044, 74.26648256817397, 73.69692785640318, 73.33448394891266, 73.04107697618227, 72.59233690024163, 72.10907835692095, 71.71211598205039, 71.21159820503969, 70.7801173627891, 70.40041422160857, 69.83085950983777, 69.31308249913704, 68.93337935795651, 68.41560234725578, 67.79426993441491, 67.17293752157406, 66.51708664135313, 66.10286503279255, 65.4987918536417, 64.84294097342078, 64.18709009319986, 63.47946151190887, 62.771832930617876, 62.06420434932689, 61.32205730065585, 60.61442871936487, 59.82050396962375, 59.06109768726269, 58.33620987228167, 57.49050742147048, 56.679323438039354, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2030, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.41180531584398, 81.39454608215395, 81.37728684846392, 81.41180531584398, 81.36002761477391, 81.37728684846392, 81.34276838108389, 81.34276838108389, 81.30824991370383, 81.2909906800138, 81.2909906800138, 81.2909906800138, 81.30824991370383, 81.27373144632378, 81.15291681049361, 81.15291681049361, 81.08387987573352, 81.08387987573352, 81.03210217466345, 81.04936140835348, 81.04936140835348, 80.94580600621333, 80.9285467725233, 80.80773213669313, 80.7904729030031, 80.7904729030031, 80.70417673455299, 80.60062133241284, 80.63513979979288, 80.54884363134278, 80.46254746289266, 80.4280289955126, 80.30721435968243, 80.18639972385226, 80.11736278909216, 80.03106662064205, 80.03106662064205, 80.08284432171212, 80.01380738695202, 79.996548153262, 79.97928891957199, 79.7894373489817, 79.68588194684156, 79.58232654470142, 79.4269934414912, 79.28891957197101, 79.30617880566103, 79.13358646876078, 78.96099413186055, 78.68484639282016, 78.52951328960994, 78.40869865377978, 78.16706938211944, 78.09803244735933, 77.97721781152917, 77.73558853986883, 77.54573696927856, 77.32136693130826, 77.07973765964792, 76.71729375215742, 76.45840524680705, 76.16499827407664, 75.88885053503624, 75.71625819813602, 75.47462892647566, 75.12944425267519, 74.88781498101484, 74.68070417673457, 74.16292716603382, 73.7832240248533, 73.40352088367277, 72.95478080773215, 72.45426303072145, 72.07455988954091, 71.74663444943045, 71.17707973765965, 70.52122885743873, 69.89989644459786, 69.43389713496721, 69.0024162927166, 68.34656541249569, 67.77701070072489, 67.10390058681395, 66.41353123921299, 65.84397652744218, 65.11908871246118, 64.54953400069037, 63.72109078356921, 63.08249913703832, 62.392129789437355, 61.615464273386266, 60.95961339316535, 60.303762512944424, 59.47531929582327, 58.715913013462206, 57.83569209527097, 57.024508111839836, 56.29962029685882, 55.55747324818778, 54.81532619951675]}, {"task": "cub", "expert": "cub", "endpoint": "Qwen3.8", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [81.429064549534, 81.39454608215395, 81.2909906800138, 81.08387987573352, 80.9975837072834, 80.9285467725233, 80.82499137038316, 80.56610286503279, 80.46254746289266, 80.22091819123231, 80.25543665861235, 80.04832585433206, 79.91025198481188, 79.59958577839144, 79.47877114256127, 79.40973420780118, 79.23714187090093, 79.09906800138074, 78.97825336555057, 78.8574387297204, 78.54677252329996, 78.33966171901969, 78.20158784949949, 77.97721781152917, 77.68381083879876, 77.30410769761824, 77.02795995857784, 76.73455298584743, 76.61373835001726, 76.44114601311702, 76.18225750776666, 76.00966517086641, 75.73351743182603, 75.47462892647566, 75.37107352433551, 75.12944425267519, 74.88781498101484, 74.59440800828443, 74.2147048671039, 73.88677942699344, 73.69692785640318, 73.50707628581291, 73.2827062478426, 73.09285467725233, 72.73041076976182, 72.50604073179151, 72.21263375906109, 71.93648602002072, 71.53952364515015, 71.12530203658957, 70.81463583016914, 70.71108042802899, 70.38315498791854, 70.08974801518812, 69.74456334138765, 69.5719710044874, 69.36486020020712, 69.03693476009666, 68.76078702105627, 68.48463928201588, 68.17397307559546, 67.77701070072489, 67.48360372799448, 67.24197445633415, 66.89678978253366, 66.5688643424232, 66.34449430445288, 65.96479116327235, 65.58508802209182, 65.2916810493614, 64.84294097342078, 64.51501553331033, 64.20434932688988, 63.7901277183293, 63.4276838108388, 63.13427683810839, 62.754573696927864, 62.3576113220573, 61.94338971349672, 61.563686572316186, 61.25302036589575, 60.90783569209527, 60.528132550914734, 60.217466344494305, 59.924059371763896, 59.47531929582327, 59.14739385571281, 58.802209181912325, 58.49154297549188, 58.1463583016914, 57.749395926820846, 57.3524335519503, 56.98998964445978, 56.69658267172938, 56.368657231618926, 56.12702795995859, 55.902657921988265, 55.678287884017955, 55.3503624439075, 55.05695547117708, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.04142216085606, 90.04142216085606, 89.98964445978598, 89.98964445978598, 89.95512599240594, 89.90334829133586, 89.79979288919571, 89.62720055229548, 89.5063859164653, 89.45460821539524, 89.29927511218501, 89.02312737314463, 88.83327580255437, 88.55712806351399, 88.33275802554367, 88.16016568864342, 87.95305488436314, 87.55609250949257, 87.14187090093199, 86.7103900586814, 86.27890921643079, 85.81290990680014, 85.39868829823956, 84.91542975491888, 84.29409734207802, 83.75906109768727, 83.22402485329651, 82.70624784259579, 82.05039696237488, 81.41180531584398, 80.7904729030031, 80.18639972385225, 79.54780807732136, 78.89195719710045, 78.30514325163962, 77.64929237141871, 76.88988608905764, 76.0787021056265, 75.45736969278563, 74.76700034518467, 74.0075940628236, 73.1446323783224, 72.35070762858129, 71.67759751467034, 70.91819123230928, 70.19330341732827, 69.55471177079737, 68.84708318950639, 68.01863997238522, 67.13841905419399, 66.34449430445288, 65.46427338626164, 64.53227476700035, 63.61753538142907, 62.771832930617876, 61.926130479806694, 61.02865032792544, 60.20020711080427, 59.33724542630308, 58.49154297549188, 57.6458405246807, 56.817397307559546, 55.937176389368304, 55.09147393855712, 54.21125302036589, 53.33103210217466, 52.48532965136348, 51.5015533310321, 50.690369347600964, 49.77562996202968, 48.8608905764584, 47.997928891957194, 47.16948567483603, 46.306523990334824, 45.512599240593715, 44.63237832240248, 43.66586123576113, 42.75112185018985, 41.81912323092855, 40.83534691059717, 39.920607525025886, 39.005868139454606, 38.022091819123226, 37.09009319986192, 36.123576113220565, 35.19157749395927, 34.25957887469796, 33.32758025543665, 32.34380393510528, 31.377286848463925, 30.393510528132552, 29.409734207801176, 28.46047635484984, 27.511218501898515, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 89.93786675871591, 89.66171901967553, 89.47186744908525, 89.14394200897479, 88.90231273731446, 88.69520193303417, 88.38453572661373, 87.98757335174318, 87.79772178115292, 87.40075940628236, 87.09009319986191, 86.79668622713152, 86.26164998274076, 85.74387297204004, 85.29513289609942, 84.76009665170866, 84.24231964100794, 83.84535726613737, 83.37935795650672, 82.861580945806, 82.32654470141524, 81.8605453917846, 81.39454608215395, 80.75595443562305, 80.27269589230238, 79.53054884363134, 79.06454953400069, 78.37418018639973, 77.80462547462892, 77.2868484639282, 76.61373835001726, 76.02692440455642, 75.52640662754573, 75.12944425267519, 74.7152226441146, 74.28374180186398, 73.85226095961339, 73.36900241629272, 72.98929927511219, 72.62685536762167, 72.28167069382118, 71.74663444943043, 71.33241284086986, 70.86641353123922, 70.34863652053849, 69.91715567828788, 69.38211943389715, 68.91612012426648, 68.4328615809458, 67.94960303762512, 67.55264066275457, 67.00034518467379, 66.55160510873317, 65.99930963065239, 65.49879185364169, 64.86020020711081, 64.23886779426992, 63.65205384880911, 63.01346220227821, 62.34035208836728, 61.52916810493614, 60.735243355195024, 59.97583707283396, 59.16465308940282, 58.474283741801855, 57.628581290990674, 56.86917500862961, 56.040731791508456, 55.229547808077314, 54.435623058336205, 53.60717984121505, 52.726958923023815, 51.84673800483258, 50.94925785295133, 50.15533310321021, 49.22333448394891, 48.25681739730756, 47.37659647911632, 46.41007939247498, 45.49534000690369, 44.615119088712454, 43.7176389368312, 42.78564031756989, 41.87090093199862, 40.921643079047286, 39.95512599240594, 39.04038660683465, 38.073869520193306, 37.14187090093199, 36.209872281670684, 35.277873662409384, 34.31135657576804, 33.34483948912668, 32.378322402485324, 31.429064549534004, 30.42802899551259, 29.444252675181215, 28.443217121159815, 27.511218501898515, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.12771832930618, 89.95512599240594, 89.69623748705557, 89.55816361753539, 89.45460821539524, 89.2129789437349, 88.88505350362443, 88.62616499827408, 88.26372109078356, 87.84949948222298, 87.5733517431826, 87.26268553676216, 86.88298239558164, 86.34794615119088, 86.0717984121505, 85.64031756989989, 85.13979979288919, 84.76009665170866, 84.1387642388678, 83.53469105971695, 83.08595098377631, 82.60269244045564, 82.17121159820505, 81.60165688643424, 80.91128753883328, 80.3589920607525, 79.9275112185019, 79.23714187090093, 78.7884017949603, 78.30514325163962, 77.7873662409389, 77.39040386606834, 76.7863306869175, 76.37210907835691, 75.92336900241628, 75.52640662754573, 75.12944425267519, 74.7152226441146, 74.37003797031412, 73.97307559544356, 73.55885398688298, 73.05833620987228, 72.60959613393166, 71.98826372109079, 71.47048671039006, 70.81463583016914, 70.27959958577839, 69.7100448740076, 69.08871246116672, 68.50189851570589, 68.03589920607524, 67.55264066275457, 67.0693821194339, 66.46530894028305, 65.7921988263721, 65.13634794615119, 64.44597859855021, 63.73835001725923, 62.96168450120814, 62.2885743872972, 61.54642733862617, 60.787021056265104, 60.01035554021401, 59.33724542630308, 58.64687607870211, 57.749395926820846, 56.92095270969968, 56.17880566102864, 55.350362443907485, 54.48740075940628, 53.69347600966516, 52.86503279254401, 52.0020711080428, 51.12185018985157, 50.25888850535036, 49.34414911977908, 48.412150500517775, 47.54918881601656, 46.61719019675526, 45.70245081118398, 44.804970659302725, 43.838453572661365, 42.94097342078012, 41.991715567828784, 41.02519848118743, 40.11045909561615, 39.16120124266482, 38.177424922333444, 37.19364860200207, 36.22713151536071, 35.22609596133931, 34.25957887469796, 33.25854332067656, 32.30928546772523, 31.32550914739385, 30.376251294442525, 29.426993441491188, 28.477735588539865, 27.545736969278558, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.17949603037624, 90.17949603037624, 90.17949603037624, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.04142216085606, 90.006903693476, 89.98964445978598, 89.83431135657577, 89.76527442181566, 89.69623748705557, 89.60994131860545, 89.43734898170521, 89.3683120469451, 89.28201587849499, 89.12668277528478, 88.88505350362443, 88.574387297204, 88.33275802554367, 88.05661028650327, 87.91853641698309, 87.69416637901277, 87.36624093890231, 86.98653779772178, 86.62409389023126, 86.10631687953054, 85.67483603727995, 85.10528132550914, 84.5012081463583, 84.05246807041767, 83.5001725923369, 82.9306178805661, 82.44735933724542, 81.82602692440454, 81.30824991370382, 80.60062133241284, 80.13462202278218, 79.4960303762513, 78.73662409389023, 77.97721781152916, 77.13151536071798, 76.45840524680703, 75.75077666551606, 75.23299965481533, 74.43907490507421, 73.88677942699344, 73.0755954435623, 72.41974456334138, 71.60856057991025, 70.78011736278908, 69.93441491197791, 69.10597169485675, 68.38108387987573, 67.62167759751468, 66.81049361408353, 65.98205039696236, 65.17086641353124, 64.35968243010011, 63.51397997928891, 62.720055229547796, 61.822575077666556, 60.92509492578528, 59.97583707283396, 59.026579219882635, 58.23265447014152, 57.35243355195029, 56.50673110113911, 55.67828788401794, 54.79806696582672, 53.935105281325505, 53.02036589575423, 52.12288574387297, 51.15636865723161, 50.20711080428028, 49.27511218501898, 48.46392820158785, 47.54918881601656, 46.65170866413531, 45.667932343803926, 44.77045219192268, 43.80393510528132, 42.85467725232999, 41.905419399378665, 41.00793924749741, 40.02416292716603, 39.074905074214705, 38.1429064549534, 37.15913013462202, 36.22713151536071, 35.22609596133931, 34.276838108387985, 33.31032102174664, 32.34380393510528, 31.39454608215395, 30.393510528132552, 29.426993441491188, 28.46047635484984, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2026, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.11045909561615, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.07594062823611, 90.02416292716603, 90.02416292716603, 89.97238522609597, 89.97238522609597, 89.85157059026578, 89.83431135657577, 89.7134967207456, 89.66171901967553, 89.60994131860545, 89.43734898170521, 89.31653434587504, 89.12668277528478, 88.91957197100449, 88.62616499827408, 88.43631342768381, 88.31549879185364, 87.97031411805315, 87.88401794960303, 87.62512944425266, 87.36624093890231, 86.79668622713152, 86.27890921643079, 85.84742837418018, 85.38142906454954, 84.86365205384881, 84.29409734207802, 83.84535726613737, 83.37935795650672, 82.7925440110459, 82.18847083189506, 81.53261995167415, 80.94580600621332, 80.44528822920262, 79.7204004142216, 79.11632723507076, 78.49499482222988, 77.87366240938903, 77.13151536071798, 76.40662754573697, 75.69899896444598, 74.95685191577493, 74.11114946496376, 73.35174318260269, 72.5060407317915, 71.78115291681048, 71.12530203658957, 70.33137728684846, 69.58923023817742, 68.89886089057646, 68.15671384190541, 67.41456679323437, 66.60338280980325, 65.72316189161201, 64.89471867449085, 63.910942354159474, 62.96168450120814, 62.046945115636866, 61.2185018985157, 60.372799447704516, 59.458060062133235, 58.64687607870211, 57.749395926820846, 56.86917500862961, 55.97169485674836, 55.10873317224715, 54.124956851915776, 53.21021746634449, 52.27821884708319, 51.27718329306179, 50.37970314118052, 49.482222989299274, 48.51570590265792, 47.58370728339661, 46.70348636520538, 45.7887469796341, 44.90852606144286, 43.97652744218156, 42.99275112185018, 42.00897480151881, 41.00793924749741, 40.05868139454608, 39.09216430790473, 38.177424922333444, 37.22816706938211, 36.261649982740764, 35.277873662409384, 34.34587504314808, 33.362098722816704, 32.39558163617535, 31.411805315843978, 30.445288229202617, 29.444252675181215, 28.46047635484984, 27.476700034518466, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2026, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.09319986192614, 90.11045909561615, 90.09319986192614, 90.05868139454608, 90.05868139454608, 90.04142216085606, 90.02416292716603, 89.98964445978598, 89.95512599240594, 89.92060752502589, 89.8688298239558, 89.74801518812563, 89.67897825336554, 89.54090438384536, 89.31653434587504, 89.16120124266482, 88.98860890576458, 88.85053503624438, 88.69520193303417, 88.53986882982396, 88.29823955816362, 88.0048325854332, 87.69416637901277, 87.31446323783224, 86.91750086296169, 86.46876078702105, 86.0027614773904, 85.67483603727995, 85.15705902657922, 84.69105971694857, 84.22506040731791, 83.77632033137729, 83.20676561960649, 82.56817397307559, 82.08491542975491, 81.39454608215395, 80.7904729030031, 80.25543665861235, 79.61684501208146, 78.96099413186055, 78.30514325163962, 77.7183293061788, 77.04521919226785, 76.42388677942698, 75.62996202968588, 74.88781498101484, 74.04211253020365, 73.33448394891266, 72.62685536762167, 71.81567138419054, 71.09078356920952, 70.41767345529858, 69.62374870555747, 68.7090093199862, 67.863306869175, 67.01760441836382, 66.15464273386262, 65.36071798412151, 64.48049706593027, 63.634794615119084, 62.720055229547796, 61.8570935450466, 60.89057645840524, 59.99309630652399, 59.14739385571281, 58.249913703831545, 57.404211253020364, 56.42043493268898, 55.54021401449776, 54.64273386261649, 53.69347600966516, 52.74421815671384, 51.794960303762515, 50.91473938557127, 49.98274076630997, 49.01622367966862, 48.066965826717286, 47.15222644114601, 46.220227821884706, 45.27096996893338, 44.37348981705212, 43.527787366240936, 42.57852951328961, 41.62927166033828, 40.66275457369692, 39.696237487055576, 38.76423886779426, 37.797721781152916, 36.882982395581635, 35.950983776320335, 35.001725923369, 34.03520883672765, 33.10321021746634, 32.153952364515014, 31.23921297894373, 30.307214359682423, 29.3579565067311, 28.46047635484984, 27.49395926820849, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 89.98964445978598, 89.78253365550569, 89.31653434587504, 88.95409043838454, 88.55712806351399, 88.0048325854332, 87.55609250949257, 87.0037970314118, 86.45150155333103, 85.9337245426303, 85.48498446668967, 85.07076285812909, 84.63928201587849, 84.17328270624785, 83.74180186399724, 83.10321021746634, 82.67172937521575, 82.25750776665515, 81.70521228857439, 81.20469451156367, 80.65239903348291, 80.13462202278218, 79.73765964791163, 79.2198826372109, 78.59855022437003, 78.13255091473938, 77.59751467034863, 76.95892302381773, 76.42388677942698, 75.92336900241628, 75.35381429064549, 74.76700034518467, 74.24922333448394, 73.83500172592336, 73.02381774249223, 72.41974456334138, 71.71211598205039, 70.90093199861926, 70.2105626510183, 69.55471177079737, 68.89886089057646, 68.2257507766655, 67.53538142906454, 66.81049361408353, 66.0683465654125, 65.32619951674145, 64.58405246807042, 63.824646185709355, 63.134276838108384, 62.46116672419744, 61.70176044183637, 60.97687262685536, 60.32102174663445, 59.647911632723506, 58.92302381774248, 58.2844321712116, 57.57680358992061, 56.851915774939584, 56.07525025888851, 55.350362443907485, 54.59095616154642, 53.91784604763548, 53.15843976527441, 52.34725578184328, 51.63962720055228, 51.018294787711426, 50.25888850535036, 49.534000690369346, 48.89540904383845, 48.11874352778736, 47.428374180186395, 46.720745598895405, 45.99585778391439, 45.25371073524335, 44.58060062133241, 43.838453572661365, 43.1480842250604, 42.42319641007939, 41.715567828788394, 40.95616154642733, 40.16223679668622, 39.47186744908525, 38.78149810148429, 38.03935105281325, 37.22816706938211, 36.55505695547118, 35.77839143942009, 35.01898515705902, 34.3631342768381, 33.586468760787014, 32.861580945806004, 32.18847083189505, 31.532619951674146, 30.945806006213317, 30.255436658612357, 29.49603037625129, 28.77114256127028, 28.063513979979284, 27.424922333448386, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.07594062823611, 90.07594062823611, 90.05868139454608, 90.05868139454608, 89.98964445978598, 89.92060752502589, 89.88608905764583, 89.81705212288574, 89.69623748705557, 89.48912668277528, 89.31653434587504, 89.2129789437349, 89.0058681394546, 88.86779426993441, 88.67794269934414, 88.5053503624439, 88.33275802554367, 88.03935105281326, 87.79772178115292, 87.43527787366241, 87.09009319986191, 86.74490852606144, 86.31342768381084, 85.86468760787021, 85.45046599930963, 85.03624439074905, 84.57024508111839, 84.05246807041767, 83.5001725923369, 82.861580945806, 82.46461857093544, 81.80876769071452, 81.20469451156367, 80.4970659302727, 79.77217811529168, 79.0818087676907, 78.40869865377978, 77.68381083879875, 77.11425612702796, 76.38936831204694, 75.52640662754573, 74.80151881256472, 73.99033482913359, 73.23092854677252, 72.43700379703141, 71.66033828098031, 70.96996893337936, 70.19330341732827, 69.29582326544701, 68.51915774939592, 67.7079737659648, 66.9313082499137, 66.15464273386262, 65.22264411460131, 64.30790472903003, 63.479461511908866, 62.616499827407665, 61.80531584397653, 60.92509492578528, 60.027614773904034, 59.11287538833275, 58.21539523645149, 57.38695201933034, 56.50673110113911, 55.60925094925785, 54.74628926475664, 53.84880911287539, 53.003106662064205, 52.07110804280289, 51.12185018985157, 50.22437003797031, 49.3614083534691, 48.39489126682775, 47.497411114946495, 46.582671729375214, 45.68519157749395, 44.7186744908526, 43.80393510528132, 42.87193648602002, 41.922678633068685, 41.059716948567484, 40.11045909561615, 39.12668277528477, 38.160165688643424, 37.21090783569208, 36.209872281670684, 35.24335519502934, 34.276838108387985, 33.29306178805661, 32.32654470141525, 31.411805315843978, 30.42802899551259, 29.46151190887124, 28.512254055919918, 27.56299620296858, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.11045909561615, 89.76527442181566, 89.52364515015533, 89.23023817742491, 89.0058681394546, 88.64342423196409, 88.31549879185364, 87.91853641698309, 87.64238867794269, 87.38350017259233, 87.05557473248187, 86.76216775975146, 86.38246461857094, 85.88194684156024, 85.46772523299964, 85.07076285812909, 84.63928201587849, 84.12150500517777, 83.60372799447704, 82.9996548153262, 82.39558163617535, 81.84328615809457, 81.39454608215395, 80.8595098377632, 80.2899551259924, 79.7894373489817, 79.18536416983085, 78.63306869175008, 78.14981014842941, 77.59751467034863, 77.09699689333793, 76.51018294787711, 76.09596133931653, 75.66448049706594, 75.16396272005524, 74.69796341042458, 74.05937176389368, 73.62789092164309, 73.19641007939246, 72.74767000345183, 72.35070762858129, 71.76389368312046, 71.36693130824992, 70.91819123230928, 70.52122885743873, 69.88263721090784, 69.31308249913704, 68.91612012426648, 68.3638246461857, 67.88056610286503, 67.31101139109424, 66.68967897825337, 65.94753192958233, 65.30894028305143, 64.75664480497065, 64.16983085950983, 63.58301691404902, 62.978943734898166, 62.30583362098723, 61.70176044183637, 61.149464963755605, 60.49361408353469, 59.85502243700379, 59.09561615464274, 58.37072833966172, 57.59406282361063, 56.817397307559546, 55.988954090438384, 55.212288574387294, 54.435623058336205, 53.60717984121505, 52.726958923023815, 51.829478771142554, 50.91473938557127, 50.03451846738005, 49.15429754918881, 48.2913358646876, 47.32481877804625, 46.427338626164996, 45.49534000690369, 44.597859855022435, 43.7176389368312, 42.73386261649982, 41.784604763548494, 40.88712461166723, 39.90334829133586, 39.023127373144625, 38.073869520193306, 37.14187090093199, 36.209872281670684, 35.22609596133931, 34.276838108387985, 33.29306178805661, 32.32654470141525, 31.342768381083875, 30.393510528132552, 29.444252675181215, 28.477735588539865, 27.545736969278558, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 90.02416292716603, 89.76527442181566, 89.60994131860545, 89.57542285122541, 89.17846047635484, 88.88505350362443, 88.60890576458405, 88.36727649292371, 88.03935105281326, 87.69416637901277, 87.24542630307214, 86.76216775975146, 86.36520538488091, 85.96824301001035, 85.51950293406972, 85.05350362443907, 84.51846738004832, 83.91439420089748, 83.27580255436658, 82.65447014152572, 82.15395236451502, 81.53261995167415, 80.82499137038315, 80.2209181912323, 79.61684501208146, 79.0818087676907, 78.47773558853987, 77.90818087676908, 77.40766309975837, 76.83810838798757, 76.18225750776665, 75.56092509492578, 74.99137038315499, 74.5081118398343, 73.99033482913359, 73.45529858474283, 72.98929927511219, 72.52329996548153, 71.95374525371074, 71.45322747670004, 70.95270969968934, 70.52122885743873, 70.15878494994821, 69.79634104245771, 69.31308249913704, 68.79530548843631, 68.17397307559544, 67.69071453227477, 67.08664135312392, 66.53434587504314, 66.01656886434242, 65.37797721781152, 64.82568173973075, 64.18709009319986, 63.44494304452881, 62.84086986537798, 62.150500517777004, 61.52916810493614, 60.80428028995512, 60.303762512944424, 59.69968933379357, 59.04383845357266, 58.33620987228167, 57.628581290990674, 56.98998964445978, 56.316879530548846, 55.626510182947875, 54.79806696582672, 54.03866068346566, 53.24473593372454, 52.46807041767345, 51.53607179841215, 50.70762858129099, 49.79288919571971, 48.981705212288574, 48.15326199516741, 47.25578184328615, 46.32378322402485, 45.40904383845357, 44.47704521919226, 43.527787366240936, 42.613047980669656, 41.715567828788394, 40.74905074214705, 39.782533655505695, 38.91957197100449, 37.98757335174317, 37.0037970314118, 36.0717984121505, 35.122540559199166, 34.173282706247846, 33.20676561960648, 32.24024853296513, 31.29099068001381, 30.341732827062472, 29.39247497411115, 28.46047635484984, 27.49395926820849, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.12771832930618, 90.05868139454608, 90.04142216085606, 90.04142216085606, 90.04142216085606, 89.90334829133586, 89.85157059026578, 89.78253365550569, 89.67897825336554, 89.52364515015533, 89.2129789437349, 89.12668277528478, 88.95409043838454, 88.76423886779426, 88.59164653089402, 88.40179496030376, 88.22920262340351, 88.0738695201933, 87.84949948222298, 87.53883327580255, 87.10735243355194, 86.69313082499137, 86.33068691750086, 85.88194684156024, 85.39868829823956, 84.94994822229893, 84.58750431480841, 84.15602347255782, 83.81083879875733, 83.20676561960649, 82.56817397307559, 81.96410079392474, 81.2909906800138, 80.58336209872282, 79.96202968588194, 79.23714187090093, 78.59855022437003, 77.89092164307905, 77.2868484639282, 76.47566448049706, 75.68173973075595, 74.9395926820849, 74.14566793234381, 73.36900241629272, 72.66137383500173, 71.97100448740076, 71.19433897134967, 70.45219192267864, 69.5719710044874, 68.76078702105626, 67.88056610286503, 67.17293752157404, 66.34449430445288, 65.51605108733172, 64.60131170176044, 63.7210907835692, 62.89264756644805, 62.01242664825681, 61.11494649637555, 60.2865032792544, 59.42354159475318, 58.49154297549188, 57.57680358992061, 56.67932343803934, 55.712806351397994, 54.780807732136694, 53.86606834656541, 53.02036589575423, 52.12288574387297, 51.19088712461166, 50.293406972730416, 49.39592682084915, 48.481187435277874, 47.497411114946495, 46.582671729375214, 45.68519157749395, 44.75319295823265, 43.838453572661365, 42.906454953400065, 41.95719710044873, 41.00793924749741, 40.12771832930617, 39.12668277528477, 38.160165688643424, 37.21090783569208, 36.22713151536071, 35.24335519502934, 34.31135657576804, 33.32758025543665, 32.34380393510528, 31.429064549534004, 30.445288229202617, 29.46151190887124, 28.52951328960994, 27.545736969278558, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2027, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.17949603037624, 90.17949603037624, 90.09319986192614, 90.09319986192614, 90.09319986192614, 90.07594062823611, 90.006903693476, 89.90334829133586, 89.83431135657577, 89.69623748705557, 89.60994131860545, 89.48912668277528, 89.31653434587504, 89.23023817742491, 89.10942354159475, 88.98860890576458, 88.83327580255437, 88.59164653089402, 88.26372109078356, 87.81498101484294, 87.40075940628236, 87.0728339661719, 86.74490852606144, 86.38246461857094, 85.89920607525026, 85.39868829823956, 84.94994822229893, 84.4321712115982, 83.75906109768727, 83.15498791853642, 82.4991370383155, 81.80876769071452, 81.20469451156367, 80.54884363134276, 79.80669658267172, 79.20262340352087, 78.42595788746979, 77.7183293061788, 76.97618225750776, 76.33759061097687, 75.61270279599586, 74.80151881256472, 74.0766309975837, 73.21366931308249, 72.54055919917155, 71.69485674836037, 70.98722816706939, 70.15878494994821, 69.33034173282707, 68.46738004832585, 67.74249223334483, 66.87953054884362, 66.0683465654125, 65.37797721781152, 64.58405246807042, 63.68657231618915, 62.823610631687956, 61.8570935450466, 61.011391094235414, 60.148429409734206, 59.38902312737314, 58.526061442871935, 57.6458405246807, 56.78287884017949, 55.91991715567828, 55.02243700379703, 54.280289955125994, 53.34829133586469, 52.519848118743525, 51.605108733172244, 50.828443217121155, 49.930963065239894, 48.981705212288574, 48.03244735933724, 47.15222644114601, 46.306523990334824, 45.512599240593715, 44.56334138764239, 43.59682430100103, 42.73386261649982, 41.836382464618566, 40.86986537797721, 39.95512599240594, 39.04038660683465, 38.09112875388332, 37.15913013462202, 36.22713151536071, 35.260614428719364, 34.31135657576804, 33.32758025543665, 32.3610631687953, 31.411805315843978, 30.42802899551259, 29.444252675181215, 28.49499482222989, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2027, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.11045909561615, 90.11045909561615, 90.09319986192614, 90.07594062823611, 90.04142216085606, 90.006903693476, 89.90334829133586, 89.88608905764583, 89.8688298239558, 89.83431135657577, 89.76527442181566, 89.57542285122541, 89.31653434587504, 89.14394200897479, 88.91957197100449, 88.86779426993441, 88.55712806351399, 88.31549879185364, 88.03935105281326, 87.74594408008284, 87.22816706938211, 86.74490852606144, 86.17535381429065, 85.81290990680014, 85.12254055919917, 84.57024508111839, 83.98343113565757, 83.53469105971695, 83.01691404901622, 82.44735933724542, 81.94684156023472, 81.23921297894373, 80.60062133241284, 79.94477045219192, 79.37521574042113, 78.70210562651017, 77.94269934414912, 77.16603382809804, 76.47566448049706, 75.69899896444598, 75.02588885053503, 74.30100103555401, 73.48981705212289, 72.74767000345183, 71.97100448740076, 71.33241284086986, 70.78011736278908, 69.96893337935795, 69.22678633068692, 68.53641698308594, 67.7252329996548, 67.00034518467379, 66.17190196755264, 65.39523645150155, 64.46323783224024, 63.66931308249914, 62.823610631687956, 61.96064894718673, 61.04590956161546, 60.079392474974114, 59.2164307904729, 58.26717293752157, 57.421470486710376, 56.48947186744908, 55.73006558508802, 54.86710390058681, 54.09043838453572, 53.227476700034515, 52.31273731446324, 51.46703486365205, 50.55229547808078, 49.65481532619951, 48.826372109078356, 47.911632723507076, 46.97963410424577, 46.06489471867449, 45.15015533310321, 44.25267518122195, 43.35519502934069, 42.45771487745944, 41.52571625819813, 40.61097687262685, 39.67897825336555, 38.76423886779426, 37.86675871591301, 36.900241629271655, 36.02002071108043, 35.053503624439074, 34.15602347255782, 33.15498791853641, 32.24024853296513, 31.273731446323783, 30.307214359682423, 29.3579565067311, 28.408698653779773, 27.442181567138412, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.05868139454608, 89.81705212288574, 89.52364515015533, 89.19571971004487, 88.91957197100449, 88.40179496030376, 88.14290645495339, 87.60787021056264, 87.0728339661719, 86.58957542285121, 86.08905764584051, 85.70935450465998, 85.17431826026925, 84.74283741801864, 84.20780117362789, 83.70728339661719, 83.32758025543666, 82.861580945806, 82.34380393510527, 81.73973075595443, 81.08387987573352, 80.56610286503279, 80.04832585433206, 79.4269934414912, 78.90921643079048, 78.27062478425958, 77.70107007248878, 77.0797376596479, 76.56196064894718, 76.11322057300656, 75.52640662754573, 74.99137038315499, 74.33551950293406, 73.80048325854331, 73.26544701415256, 72.55781843286158, 71.93648602002071, 71.21159820503969, 70.55574732481877, 69.89989644459786, 69.2095270969969, 68.45012081463582, 67.84604763548498, 67.08664135312392, 66.30997583707283, 65.61960648947186, 65.03279254401104, 64.44597859855021, 63.73835001725923, 63.117017604418365, 62.53020365895754, 61.80531584397653, 61.04590956161546, 60.3555402140145, 59.647911632723506, 58.88850535036244, 58.180876769071446, 57.4387297204004, 56.73110113910942, 55.988954090438384, 55.26406627545737, 54.53917846047636, 53.91784604763548, 53.12392129789437, 52.399033482913346, 51.62236796686227, 50.91473938557127, 50.20711080428028, 49.39592682084915, 48.67103900586814, 47.96341042457715, 47.23852260959613, 46.582671729375214, 45.8577839143942, 45.06385916465309, 44.35623058336209, 43.545046599930956, 42.78564031756989, 42.02623403520883, 41.215050051777695, 40.45564376941663, 39.799792889195714, 39.12668277528477, 38.45357266137383, 37.72868484639281, 36.96927856403175, 36.24439074905074, 35.51950293406973, 34.65654124956852, 33.94891266827752, 33.20676561960648, 32.378322402485324, 31.756989989644456, 30.980324473593367, 30.289955125992407, 29.58232654470141, 28.822920262340347, 28.132550914739383, 27.407663099758373, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.07594062823611, 90.05868139454608, 89.97238522609597, 89.8688298239558, 89.78253365550569, 89.66171901967553, 89.48912668277528, 89.38557128063513, 89.28201587849499, 89.07490507421471, 88.90231273731446, 88.72972040041421, 88.574387297204, 88.36727649292371, 88.05661028650327, 87.67690714532274, 87.31446323783224, 86.83120469451156, 86.41698308595099, 86.0027614773904, 85.5712806351398, 85.001725923369, 84.48394891266827, 83.96617190196754, 83.48291335864687, 82.861580945806, 82.18847083189506, 81.60165688643424, 80.9285467725233, 80.3589920607525, 79.70314118053157, 79.04729030031066, 78.30514325163962, 77.61477390403866, 76.90714532274767, 76.19951674145668, 75.50914739385571, 74.76700034518467, 74.02485329651363, 73.26544701415256, 72.55781843286158, 71.76389368312046, 71.10804280289955, 70.40041422160856, 69.58923023817742, 68.65723161891611, 67.96686227131515, 67.10390058681395, 66.2927166033828, 65.39523645150155, 64.51501553331032, 63.634794615119084, 62.7890921643079, 61.943389713496714, 61.08042802899551, 60.20020711080427, 59.33724542630308, 58.59509837763204, 57.749395926820846, 56.95547117707973, 56.127027959958575, 55.26406627545737, 54.41836382464618, 53.53814290645496, 52.70969968933379, 51.91577493959267, 50.98377632033137, 50.05177770107008, 49.17155678287884, 48.22229892992751, 47.27304107697618, 46.34104245771488, 45.391784604763544, 44.49430445288229, 43.59682430100103, 42.682084915429755, 41.715567828788394, 40.74905074214705, 39.799792889195714, 38.90231273731446, 38.0048325854332, 37.14187090093199, 36.175353814290645, 35.22609596133931, 34.25957887469796, 33.25854332067656, 32.25750776665516, 31.32550914739385, 30.42802899551259, 29.46151190887124, 28.52951328960994, 27.56299620296858, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 89.98964445978598, 89.76527442181566, 89.59268208491544, 89.24749741111494, 88.90231273731446, 88.55712806351399, 88.28098032447359, 87.90127718329306, 87.62512944425266, 87.34898170521228, 86.91750086296169, 86.52053848809112, 86.05453917846047, 85.64031756989989, 85.19157749395927, 84.62202278218847, 84.15602347255782, 83.55195029340697, 82.94787711425612, 82.30928546772522, 81.77424922333448, 81.18743527787366, 80.66965826717293, 80.01380738695202, 79.3579565067311, 78.63306869175008, 77.94269934414912, 77.3558853986883, 76.76907145322747, 76.06144287193648, 75.5781843286158, 74.90507421470487, 74.33551950293406, 73.81774249223334, 73.29996548153261, 72.76492923714186, 72.35070762858129, 71.97100448740076, 71.5050051777701, 71.12530203658957, 70.69382119433897, 70.24508111839833, 69.89989644459786, 69.46841560234725, 68.98515705902656, 68.46738004832585, 68.08767690714532, 67.5698998964446, 67.15567828788402, 66.7069382119434, 66.12012426648258, 65.58508802209182, 65.17086641353124, 64.61857093545046, 64.10079392474974, 63.66931308249914, 63.15153607179841, 62.58198136002762, 61.99516741456679, 61.339316534345876, 60.61442871936485, 59.92405937176388, 59.250949257852945, 58.45702450811183, 57.71487745944079, 56.92095270969968, 56.24784259578875, 55.40214014497755, 54.59095616154642, 53.79703141180532, 53.003106662064205, 52.19192267863306, 51.328960994131855, 50.4314808422506, 49.534000690369346, 48.72281670693821, 47.96341042457715, 47.04867103900586, 46.11667241974456, 45.2882292026234, 44.390749050742144, 43.44149119779082, 42.544011045909556, 41.73282706247842, 40.783569209527094, 39.86882982395581, 38.93683120469451, 38.022091819123226, 37.12461166724197, 36.19261304798067, 35.22609596133931, 34.276838108387985, 33.31032102174664, 32.3610631687953, 31.39454608215395, 30.445288229202617, 29.444252675181215, 28.477735588539865, 27.511218501898515, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.05868139454608, 89.81705212288574, 89.42008974801519, 89.14394200897479, 88.93683120469451, 88.60890576458405, 88.26372109078356, 87.86675871591301, 87.5733517431826, 87.19364860200207, 86.84846392820158, 86.27890921643079, 85.81290990680014, 85.46772523299964, 84.91542975491888, 84.36313427683811, 83.89713496720745, 83.3620987228167, 82.94787711425612, 82.3610631687953, 81.65343458750431, 81.11839834311357, 80.63513979979288, 80.11736278909216, 79.59958577839143, 78.89195719710045, 78.42595788746979, 77.97721781152916, 77.47670003451846, 76.95892302381773, 76.54470141525715, 75.97514670348636, 75.42285122540558, 74.92233344839488, 74.61166724197446, 74.14566793234381, 73.71418709009319, 73.23092854677252, 72.86848463928202, 72.45426303072144, 71.98826372109079, 71.48774594408009, 70.91819123230928, 70.33137728684846, 69.86537797721782, 69.29582326544701, 68.79530548843631, 68.2084915429755, 67.67345529858474, 67.05212288574387, 66.49982740766309, 65.89575422851225, 65.22264411460131, 64.65308940283052, 64.0662754573697, 63.56575768035899, 62.80635139799793, 62.133241284086985, 61.47739040386606, 60.80428028995512, 60.04487400759406, 59.2164307904729, 58.526061442871935, 57.749395926820846, 56.95547117707973, 56.265101829478766, 55.436658612357604, 54.659993096306515, 54.004142216085604, 53.14118053158439, 52.27821884708319, 51.432516396272, 50.517777010700726, 49.67207455988954, 48.75733517431826, 47.859855022436996, 46.99689333793579, 46.11667241974456, 45.20193303417328, 44.287193648602, 43.35519502934069, 42.57852951328961, 41.68104936140835, 40.69727304107697, 39.799792889195714, 38.81601656886434, 37.86675871591301, 36.96927856403175, 36.0717984121505, 35.13979979288919, 34.207801173627885, 33.24128408698653, 32.274767000345186, 31.360027614773898, 30.3589920607525, 29.426993441491188, 28.49499482222989, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.1449775629962, 90.12771832930618, 90.1449775629962, 90.11045909561615, 90.07594062823611, 90.02416292716603, 89.97238522609597, 89.8688298239558, 89.76527442181566, 89.69623748705557, 89.48912668277528, 89.29927511218501, 89.23023817742491, 89.05764584052469, 88.85053503624438, 88.47083189506385, 88.24646185709354, 88.03935105281326, 87.81498101484294, 87.5733517431826, 87.33172247152227, 86.93476009665171, 86.48602002071108, 86.03727994477045, 85.58853986882981, 85.08802209181911, 84.55298584742837, 84.06972730410769, 83.55195029340697, 82.94787711425612, 82.48187780462547, 81.98136002761477, 81.41180531584398, 80.89402830514325, 80.2899551259924, 79.70314118053157, 79.04729030031066, 78.32240248532965, 77.64929237141871, 76.90714532274767, 76.14773904038661, 75.47462892647566, 74.81877804625474, 74.04211253020365, 73.28270624784258, 72.54055919917155, 71.67759751467034, 70.93545046599931, 70.03797031411804, 69.12323092854677, 68.29478771142561, 67.58715913013462, 66.77597514670349, 65.96479116327234, 65.10182947877115, 64.29064549534, 63.46220227821884, 62.633759061097685, 61.839834311356576, 60.90783569209527, 60.01035554021401, 59.13013462202278, 58.249913703831545, 57.38695201933034, 56.541249568519156, 55.59199171556782, 54.71177079737659, 53.86606834656541, 53.02036589575423, 52.08836727649292, 51.17362789092164, 50.293406972730416, 49.34414911977908, 48.39489126682775, 47.44563341387642, 46.47911632723506, 45.47808077321366, 44.58060062133241, 43.683120469451154, 42.73386261649982, 41.85364169830859, 40.86986537797721, 39.93786675871591, 38.95409043838454, 38.073869520193306, 37.14187090093199, 36.175353814290645, 35.22609596133931, 34.24231964100793, 33.31032102174664, 32.30928546772523, 31.360027614773898, 30.410769761822575, 29.478771142561268, 28.52951328960994, 27.545736969278558, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2028, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.1449775629962, 90.1449775629962, 90.16223679668623, 90.16223679668623, 90.11045909561615, 90.11045909561615, 90.09319986192614, 90.07594062823611, 90.02416292716603, 89.90334829133586, 89.81705212288574, 89.73075595443561, 89.62720055229548, 89.47186744908525, 89.31653434587504, 89.16120124266482, 89.0058681394546, 88.72972040041421, 88.53986882982396, 88.24646185709354, 88.0738695201933, 87.83224024853297, 87.45253710735244, 87.0037970314118, 86.65861235761132, 86.20987228167068, 85.69209527096997, 85.05350362443907, 84.63928201587849, 84.10424577148774, 83.44839489126683, 82.82706247842594, 82.27476700034518, 81.60165688643424, 80.9285467725233, 80.37625129444253, 79.59958577839143, 78.9955125992406, 78.32240248532965, 77.54573696927855, 76.8553676216776, 76.09596133931653, 75.40559199171557, 74.73248187780463, 73.99033482913359, 73.19641007939246, 72.29892992751121, 71.5050051777701, 70.71108042802899, 69.96893337935795, 69.12323092854677, 68.3638246461857, 67.53538142906454, 66.74145667932343, 65.94753192958233, 65.13634794615119, 64.32516396272005, 63.44494304452881, 62.53020365895754, 61.73627890921642, 60.873317224715215, 60.04487400759406, 59.13013462202278, 58.2844321712116, 57.421470486710376, 56.57576803589921, 55.67828788401794, 54.86710390058681, 54.02140144977563, 53.175698998964435, 52.26095961339317, 51.48429409734207, 50.70762858129099, 49.79288919571971, 48.912668277528475, 48.066965826717286, 47.16948567483603, 46.27200552295478, 45.42630307214359, 44.58060062133241, 43.66586123576113, 42.76838108387987, 41.888160165688646, 40.921643079047286, 40.006903693476005, 39.074905074214705, 38.177424922333444, 37.176389368312044, 36.22713151536071, 35.29513289609941, 34.31135657576804, 33.362098722816704, 32.39558163617535, 31.429064549534004, 30.497065930272694, 29.513289609941317, 28.546772523299957, 27.56299620296858, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2028, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.17949603037624, 90.12771832930618, 90.07594062823611, 90.04142216085606, 90.04142216085606, 90.04142216085606, 90.04142216085606, 90.006903693476, 89.98964445978598, 89.90334829133586, 89.76527442181566, 89.73075595443561, 89.66171901967553, 89.54090438384536, 89.3683120469451, 89.2129789437349, 89.10942354159475, 89.04038660683466, 88.85053503624438, 88.59164653089402, 88.40179496030376, 88.17742492233344, 87.79772178115292, 87.33172247152227, 86.98653779772178, 86.58957542285121, 86.14083534691059, 85.60579910251984, 85.19157749395927, 84.69105971694857, 84.15602347255782, 83.67276492923715, 83.10321021746634, 82.41284086986538, 81.9295823265447, 81.3600276147739, 80.75595443562305, 80.18639972385225, 79.54780807732136, 79.01277183293061, 78.33966171901967, 77.68381083879875, 77.01070072488781, 76.16499827407664, 75.49188816016569, 74.74974111149464, 73.99033482913359, 73.11011391094236, 72.21263375906109, 71.47048671039006, 70.60752502588885, 69.79634104245771, 69.00241629271659, 68.10493614083535, 67.31101139109424, 66.49982740766309, 65.70590265792198, 64.80842250604073, 63.9282015878495, 63.03072143596824, 62.11598205039696, 61.20124266482567, 60.32102174663445, 59.49257852951328, 58.64687607870211, 57.749395926820846, 56.90369347600965, 55.937176389368304, 55.005177770107004, 54.10769761822575, 53.175698998964435, 52.24370037970314, 51.34622022782188, 50.414221608560574, 49.603037625129446, 48.67103900586814, 47.73904038660683, 46.8588194684156, 45.97859855022436, 45.02934069727304, 44.14911977908181, 43.23438039351053, 42.336900241629266, 41.45667932343803, 40.52468070417673, 39.60994131860544, 38.660683465654124, 37.745944080082836, 36.76216775975146, 35.830169140490156, 34.8636520538488, 33.94891266827752, 33.03417328270625, 32.10217466344494, 31.221953745253707, 30.324473593372446, 29.39247497411115, 28.443217121159815, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.006903693476, 89.8688298239558, 89.52364515015533, 89.0058681394546, 88.48809112875388, 88.02209181912323, 87.52157404211253, 87.0037970314118, 86.36520538488091, 85.86468760787021, 85.48498446668967, 84.98446668967897, 84.4321712115982, 83.91439420089748, 83.56920952709699, 83.06869175008629, 82.53365550569555, 82.15395236451502, 81.58439765274422, 81.03210217466345, 80.58336209872282, 79.97928891957197, 79.61684501208146, 79.0818087676907, 78.63306869175008, 78.02899551259924, 77.52847773558852, 76.99344149119779, 76.47566448049706, 75.90610976872627, 75.18122195374525, 74.62892647566449, 73.97307559544356, 73.33448394891266, 72.81670693821194, 72.12633759061097, 71.41870900931998, 70.71108042802899, 70.03797031411804, 69.38211943389715, 68.7090093199862, 67.91508457024507, 67.27649292371419, 66.5688643424232, 65.84397652744218, 65.0673110113911, 64.29064549534, 63.61753538142907, 62.87538833275802, 62.133241284086985, 61.39109423541594, 60.683465654124944, 59.99309630652399, 59.250949257852945, 58.49154297549188, 57.749395926820846, 56.9382119433897, 56.17880566102864, 55.45391784604763, 54.83258543320676, 54.09043838453572, 53.26199516741457, 52.502588885053505, 51.76044183638246, 51.104590956161545, 50.37970314118052, 49.74111149464964, 49.050742147048666, 48.274076630997584, 47.58370728339661, 46.87607870210562, 46.20296858819468, 45.512599240593715, 44.7877114256127, 44.011045909561616, 43.19986192613048, 42.47497411114946, 41.81912323092855, 41.11149464963756, 40.36934760096651, 39.66171901967552, 38.95409043838454, 38.298239558163615, 37.59061097687262, 36.900241629271655, 36.22713151536071, 35.51950293406973, 34.88091128753883, 34.173282706247846, 33.586468760787014, 32.79254401104591, 32.11943389713496, 31.549879185364173, 30.876769071453225, 30.13462202278219, 29.426993441491188, 28.753883327580255, 27.95995857783914, 27.37314463237832, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.12771832930618, 90.09319986192614, 90.07594062823611, 90.02416292716603, 90.04142216085606, 89.97238522609597, 89.97238522609597, 89.85157059026578, 89.7134967207456, 89.57542285122541, 89.48912668277528, 89.29927511218501, 89.14394200897479, 88.97134967207455, 88.72972040041421, 88.47083189506385, 88.14290645495339, 87.84949948222298, 87.59061097687263, 87.33172247152227, 86.91750086296169, 86.52053848809112, 86.08905764584051, 85.64031756989989, 85.20883672764928, 84.70831895063858, 84.12150500517777, 83.48291335864687, 82.74076630997584, 82.06765619606489, 81.39454608215395, 80.75595443562305, 80.06558508802209, 79.30617880566102, 78.63306869175008, 77.95995857783915, 77.14877459440801, 76.45840524680703, 75.7852951328961, 75.16396272005524, 74.42181567138418, 73.62789092164309, 73.04107697618225, 72.31618916120124, 71.53952364515015, 70.78011736278908, 69.93441491197791, 69.14049016223679, 68.45012081463582, 67.67345529858474, 66.87953054884362, 66.10286503279255, 65.22264411460131, 64.41146013117017, 63.49672074559889, 62.720055229547796, 61.839834311356576, 60.97687262685536, 60.16568864342423, 59.302726958923024, 58.40524680704176, 57.50776665516051, 56.78287884017949, 55.850880220918185, 54.98791853641698, 54.14221608560579, 53.41732827062478, 52.502588885053505, 51.674145667932336, 50.81118398343113, 49.99999999999999, 49.25785295132896, 48.343113565757676, 47.53192958232654, 46.63444943044529, 45.719710044874006, 44.83948912668277, 44.011045909561616, 43.1480842250604, 42.216085605799094, 41.45667932343803, 40.50742147048671, 39.59268208491543, 38.78149810148429, 37.83224024853297, 36.96927856403175, 36.037279944770454, 35.0707628581291, 34.10424577148774, 33.20676561960648, 32.274767000345186, 31.308249913703822, 30.3589920607525, 29.375215740421122, 28.42595788746979, 27.476700034518466, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 89.93786675871591, 89.67897825336554, 89.3683120469451, 89.12668277528478, 88.78149810148429, 88.41905419399379, 88.14290645495339, 87.84949948222298, 87.38350017259233, 86.93476009665171, 86.67587159130134, 86.15809458060062, 85.65757680358992, 85.31239212978944, 84.79461511908872, 84.19054193993787, 83.74180186399724, 83.27580255436658, 82.74076630997584, 82.24024853296513, 81.73973075595443, 81.1356575768036, 80.65239903348291, 79.996548153262, 79.46151190887124, 79.03003106662064, 78.47773558853987, 77.87366240938903, 77.3558853986883, 76.88988608905764, 76.40662754573697, 75.87159130134621, 75.45736969278563, 75.04314808422505, 74.64618570935451, 74.11114946496376, 73.64515015533311, 73.0755954435623, 72.52329996548153, 72.09181912323092, 71.62581981360027, 71.15982050396961, 70.6420434932689, 70.0724887814981, 69.55471177079737, 68.95063859164652, 68.45012081463582, 68.01863997238522, 67.62167759751468, 67.05212288574387, 66.58612357611322, 66.12012426648258, 65.5678287884018, 64.9292371418709, 64.39420089748015, 63.979979288919566, 63.37590610976872, 62.771832930617876, 62.150500517777004, 61.51190887124611, 60.76976182257508, 60.01035554021401, 59.250949257852945, 58.526061442871935, 57.697618225750766, 56.90369347600965, 56.17880566102864, 55.488436313427684, 54.729030031066614, 53.96962374870555, 53.279254401104595, 52.38177424922333, 51.57059026579219, 50.690369347600964, 49.77562996202968, 48.929927511218494, 47.997928891957194, 47.06593027269589, 46.13393165343459, 45.2191922678633, 44.321712115982045, 43.35519502934069, 42.42319641007939, 41.5774939592682, 40.628236106316876, 39.7134967207456, 38.86779426993441, 37.90127718329306, 36.93476009665171, 36.02002071108043, 35.08802209181911, 34.12150500517777, 33.18950638591647, 32.24024853296513, 31.29099068001381, 30.410769761822575, 29.46151190887124, 28.46047635484984, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.04142216085606, 89.67897825336554, 89.24749741111494, 88.91957197100449, 88.72972040041421, 88.35001725923368, 88.03935105281326, 87.76320331377286, 87.22816706938211, 86.81394546082154, 86.17535381429065, 85.77839143942009, 85.32965136347946, 84.82913358646876, 84.24231964100794, 83.81083879875733, 83.51743182602692, 83.01691404901622, 82.39558163617535, 81.91232309285468, 81.39454608215395, 81.01484294097342, 80.53158439765275, 79.87573351743183, 79.23714187090093, 78.7193648602002, 78.30514325163962, 77.75284777355886, 77.23507076285813, 76.70003451846738, 76.2858129099068, 75.76803589920607, 75.26751812219537, 74.7152226441146, 74.24922333448394, 73.73144632378322, 73.28270624784258, 72.78218847083188, 72.26441146013116, 71.78115291681048, 71.26337590610976, 70.72833966171902, 70.19330341732827, 69.65826717293751, 69.14049016223679, 68.69175008629617, 68.17397307559544, 67.74249223334483, 67.19019675526407, 66.81049361408353, 66.27545736969279, 65.70590265792198, 65.23990334829134, 64.63583016914049, 64.01449775629962, 63.49672074559889, 62.84086986537798, 62.16775975146703, 61.47739040386606, 60.873317224715215, 60.2174663444943, 59.647911632723506, 58.92302381774248, 58.19813600276147, 57.54228512254056, 56.69658267172937, 55.850880220918185, 55.02243700379703, 54.24577148774594, 53.503624439074905, 52.76147739040387, 51.88125647221262, 50.98377632033137, 50.12081463583016, 49.29237141870901, 48.481187435277874, 47.65274421815671, 46.77252329996548, 45.926820849154296, 45.04659993096306, 44.16637901277183, 43.30341732827062, 42.35415947531929, 41.47393855712806, 40.52468070417673, 39.60994131860544, 38.660683465654124, 37.7114256127028, 36.83120469451156, 35.950983776320335, 35.053503624439074, 34.13876423886779, 33.18950638591647, 32.222989299275106, 31.221953745253707, 30.289955125992407, 29.340697273041073, 28.408698653779773, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.07594062823611, 90.04142216085606, 90.006903693476, 89.97238522609597, 89.88608905764583, 89.83431135657577, 89.69623748705557, 89.60994131860545, 89.47186744908525, 89.33379357956507, 89.10942354159475, 88.95409043838454, 88.83327580255437, 88.64342423196409, 88.47083189506385, 88.22920262340351, 88.02209181912323, 87.59061097687263, 87.31446323783224, 87.0728339661719, 86.77942699344149, 86.43424231964102, 86.10631687953054, 85.60579910251984, 85.20883672764928, 84.79461511908872, 84.38039351052812, 83.75906109768727, 83.1377286848464, 82.51639627200552, 81.7915084570245, 81.08387987573352, 80.2899551259924, 79.65136347946151, 78.9955125992406, 78.44321712115982, 77.68381083879875, 77.0797376596479, 76.37210907835691, 75.73351743182603, 74.99137038315499, 74.40455643769417, 73.62789092164309, 72.85122540559199, 72.02278218847083, 71.33241284086986, 70.46945115636866, 69.77908180876769, 68.95063859164652, 68.26026924404556, 67.46634449430445, 66.74145667932343, 65.8612357611322, 65.01553331032102, 64.27338626164997, 63.479461511908866, 62.66827752847773, 61.8570935450466, 60.92509492578528, 60.027614773904034, 59.19917155678287, 58.301691404901625, 57.404211253020364, 56.50673110113911, 55.661028650327914, 54.76354849844667, 53.88332758025543, 53.003106662064205, 52.12288574387297, 51.294442526751816, 50.39696237487055, 49.603037625129446, 48.8608905764584, 48.04970659302727, 47.134967207455986, 46.220227821884706, 45.2882292026234, 44.35623058336209, 43.458750431480844, 42.57852951328961, 41.66379012771833, 40.74905074214705, 39.799792889195714, 38.83327580255437, 37.91853641698309, 36.96927856403175, 36.10631687953054, 35.15705902657922, 34.190541939937866, 33.24128408698653, 32.30928546772523, 31.360027614773898, 30.376251294442525, 29.39247497411115, 28.42595788746979, 27.49395926820849, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2029, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.11045909561615, 90.07594062823611, 90.04142216085606, 89.98964445978598, 89.93786675871591, 89.88608905764583, 89.79979288919571, 89.67897825336554, 89.54090438384536, 89.33379357956507, 89.24749741111494, 88.93683120469451, 88.79875733517432, 88.64342423196409, 88.2119433897135, 87.97031411805315, 87.60787021056264, 87.31446323783224, 86.86572316189161, 86.43424231964102, 86.0027614773904, 85.4332067656196, 84.79461511908872, 84.15602347255782, 83.51743182602692, 82.861580945806, 82.17121159820505, 81.68795305488436, 81.03210217466345, 80.3589920607525, 79.89299275112185, 79.20262340352087, 78.56403175698998, 77.82188470831895, 77.0797376596479, 76.441146013117, 75.64722126337591, 74.83603727994476, 74.11114946496376, 73.28270624784258, 72.52329996548153, 71.74663444943043, 71.10804280289955, 70.33137728684846, 69.60648947186745, 68.8643424231964, 68.13945460821539, 67.29375215740421, 66.49982740766309, 65.61960648947186, 64.84294097342078, 63.99723852260959, 63.18605453917845, 62.3576113220573, 61.58094580600621, 60.66620642043493, 59.7859855022437, 58.92302381774248, 58.04280289955126, 57.11080428028995, 56.21332412840869, 55.26406627545737, 54.349326889886086, 53.451846738004825, 52.64066275457368, 51.74318260269244, 50.94925785295133, 50.103555402140145, 49.25785295132896, 48.498446668967894, 47.58370728339661, 46.68622713151536, 45.97859855022436, 45.18467380048325, 44.35623058336209, 43.51052813255091, 42.6475664480497, 41.75008629616845, 40.83534691059717, 39.90334829133586, 38.98860890576458, 38.073869520193306, 37.14187090093199, 36.175353814290645, 35.24335519502934, 34.25957887469796, 33.29306178805661, 32.3610631687953, 31.429064549534004, 30.445288229202617, 29.478771142561268, 28.512254055919918, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2029, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.09319986192614, 90.09319986192614, 89.97238522609597, 89.95512599240594, 89.88608905764583, 89.85157059026578, 89.69623748705557, 89.57542285122541, 89.45460821539524, 89.33379357956507, 89.19571971004487, 88.95409043838454, 88.91957197100449, 88.67794269934414, 88.38453572661373, 88.03935105281326, 87.76320331377286, 87.31446323783224, 86.86572316189161, 86.39972385226096, 85.86468760787021, 85.31239212978944, 84.76009665170866, 84.41491197790818, 84.06972730410769, 83.55195029340697, 82.91335864687608, 82.37832240248532, 81.87780462547462, 81.18743527787366, 80.4970659302727, 79.96202968588194, 79.23714187090093, 78.66758715913014, 77.89092164307905, 77.0797376596479, 76.2858129099068, 75.66448049706594, 74.83603727994476, 74.09389023127373, 73.26544701415256, 72.55781843286158, 71.81567138419054, 71.22885743872972, 70.43493268898861, 69.60648947186745, 68.77804625474629, 67.9323438039351, 67.08664135312392, 66.17190196755264, 65.30894028305143, 64.5667932343804, 63.61753538142907, 62.75457369692785, 61.8570935450466, 61.02865032792544, 60.11391094235415, 59.19917155678287, 58.387987573351744, 57.50776665516051, 56.66206420434931, 55.73006558508802, 54.81532619951675, 53.91784604763548, 53.089402830514324, 52.20918191232309, 51.25992405937176, 50.34518467380048, 49.46496375560925, 48.636520538488085, 47.72178115291681, 46.75526406627545, 45.84052468070418, 44.87400759406282, 43.94200897480152, 43.02726958923023, 42.16430790472903, 41.215050051777695, 40.30031066620642, 39.38557128063513, 38.5053503624439, 37.62512944425267, 36.675871591301345, 35.709354504659984, 34.81187435277874, 33.914394200897476, 32.98239558163617, 32.084915429754915, 31.152916810493615, 30.23817742492233, 29.340697273041073, 28.356920952709697, 27.476700034518466, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 89.81705212288574, 89.6444597859855, 89.40283051432516, 89.05764584052469, 88.64342423196409, 88.16016568864342, 87.60787021056264, 87.09009319986191, 86.58957542285121, 86.10631687953054, 85.5712806351398, 85.24335519502934, 84.82913358646876, 84.29409734207802, 83.7935795650673, 83.2930617880566, 82.9306178805661, 82.4991370383155, 82.03313772868485, 81.51536071798412, 81.03210217466345, 80.58336209872282, 80.04832585433206, 79.54780807732136, 78.92647566448049, 78.20158784949948, 77.54573696927855, 77.02795995857782, 76.49292371418709, 75.95788746979633, 75.33655505695546, 74.74974111149464, 74.12840869865379, 73.576113220573, 72.98929927511219, 72.41974456334138, 71.81567138419054, 71.12530203658957, 70.5730065585088, 69.95167414566794, 69.24404556437693, 68.570935450466, 67.82878840179495, 67.2074559889541, 66.55160510873317, 65.7921988263721, 65.17086641353124, 64.41146013117017, 63.7210907835692, 63.06523990334829, 62.40938902312737, 61.684501208146344, 60.97687262685536, 60.42457714877458, 59.682430100103545, 58.94028305143251, 58.180876769071446, 57.404211253020364, 56.66206420434931, 55.90265792198825, 55.09147393855712, 54.36658612357611, 53.71073524335519, 52.98584742837418, 52.20918191232309, 51.432516396272, 50.742147048671036, 50.03451846738005, 49.22333448394891, 48.44666896789782, 47.65274421815671, 46.89333793579565, 46.15119088712461, 45.47808077321366, 44.77045219192268, 44.06282361063168, 43.30341732827062, 42.63030721435968, 41.888160165688646, 41.128753883327576, 40.283051432516395, 39.47186744908525, 38.76423886779426, 38.05661028650328, 37.245426303072136, 36.486020020711074, 35.830169140490156, 35.03624439074905, 34.24231964100793, 33.48291335864687, 32.80980324473594, 32.17121159820504, 31.463583016914043, 30.82499137038315, 30.117362789092162, 29.3579565067311, 28.667587159130125, 27.977217811529165, 27.286848463928195, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.07594062823611, 90.05868139454608, 90.02416292716603, 90.006903693476, 89.98964445978598, 89.92060752502589, 89.92060752502589, 89.81705212288574, 89.69623748705557, 89.59268208491544, 89.48912668277528, 89.35105281325508, 89.10942354159475, 88.93683120469451, 88.74697963410424, 88.574387297204, 88.2119433897135, 87.98757335174318, 87.65964791163272, 87.43527787366241, 86.98653779772178, 86.69313082499137, 86.29616845012082, 85.74387297204004, 85.26061442871936, 84.74283741801864, 84.15602347255782, 83.58646876078701, 82.9996548153262, 82.3610631687953, 81.8605453917846, 81.2909906800138, 80.70417673455297, 80.04832585433206, 79.37521574042113, 78.59855022437003, 77.94269934414912, 77.2178115291681, 76.51018294787711, 75.85433206765619, 75.06040731791508, 74.26648256817397, 73.64515015533311, 72.79944770452191, 71.98826372109079, 71.24611667241973, 70.53848809112876, 69.84811874352779, 68.98515705902656, 68.19123230928547, 67.38004832585432, 66.5688643424232, 65.74042112530203, 64.94649637556091, 64.2216085605799, 63.4276838108388, 62.58198136002762, 61.7190196755264, 60.89057645840524, 60.027614773904034, 59.19917155678287, 58.2844321712116, 57.421470486710376, 56.6448049706593, 55.78184328615809, 54.91888160165688, 54.055919917155684, 53.14118053158439, 52.295478080773215, 51.41525716258197, 50.55229547808078, 49.67207455988954, 48.8608905764584, 47.997928891957194, 47.15222644114601, 46.2892647566448, 45.3572661373835, 44.45978598550224, 43.545046599930956, 42.66482568173973, 41.715567828788394, 40.81808767690715, 39.93786675871591, 38.98860890576458, 38.03935105281325, 37.05557473248188, 36.10631687953054, 35.15705902657922, 34.207801173627885, 33.275802554366585, 32.3610631687953, 31.429064549534004, 30.42802899551259, 29.49603037625129, 28.52951328960994, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.04142216085606, 89.83431135657577, 89.59268208491544, 89.38557128063513, 89.23023817742491, 88.97134967207455, 88.64342423196409, 88.40179496030376, 88.09112875388332, 87.79772178115292, 87.55609250949257, 87.12461166724196, 86.69313082499137, 86.19261304798067, 85.72661373835001, 85.13979979288919, 84.60476354849844, 84.03520883672765, 83.6382464618571, 83.27580255436658, 82.7925440110459, 82.27476700034518, 81.70521228857439, 81.10113910942354, 80.51432516396272, 80.08284432171212, 79.51328960994132, 79.09906800138073, 78.66758715913014, 78.27062478425958, 77.80462547462892, 77.32136693130825, 76.97618225750776, 76.51018294787711, 76.09596133931653, 75.80255436658612, 75.31929582326543, 74.90507421470487, 74.59440800828443, 74.04211253020365, 73.48981705212289, 73.04107697618225, 72.66137383500173, 72.26441146013116, 71.79841215050051, 71.28063513979978, 70.69382119433897, 70.15878494994821, 69.55471177079737, 68.91612012426648, 68.45012081463582, 67.89782533655504, 67.27649292371419, 66.67241974456334, 65.94753192958233, 65.22264411460131, 64.53227476700035, 63.80738695201933, 63.099758370728345, 62.40938902312737, 61.7190196755264, 61.011391094235414, 60.234725578184324, 59.40628236106317, 58.62961684501208, 57.90472903003106, 57.07628581290991, 56.265101829478766, 55.47117707973766, 54.64273386261649, 53.79703141180532, 52.98584742837418, 52.14014497756298, 51.242664825681736, 50.27614773904039, 49.3614083534691, 48.44666896789782, 47.58370728339661, 46.66896789782533, 45.736969278564025, 44.77045219192268, 43.838453572661365, 42.94097342078012, 42.04349326889886, 41.1460131170176, 40.16223679668622, 39.230238177424916, 38.229202623403516, 37.26268553676216, 36.29616845012082, 35.346910597169476, 34.38039351052813, 33.39661719019675, 32.39558163617535, 31.429064549534004, 30.46254746289264, 29.478771142561268, 28.512254055919918, 27.511218501898515, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.07594062823611, 89.8688298239558, 89.59268208491544, 89.31653434587504, 89.07490507421471, 88.79875733517432, 88.52260959613393, 88.19468415602347, 87.93579565067311, 87.5043148084225, 87.05557473248187, 86.74490852606144, 86.33068691750086, 85.79565067311012, 85.20883672764928, 84.79461511908872, 84.24231964100794, 83.72454263030721, 83.31032102174663, 82.67172937521575, 82.17121159820505, 81.65343458750431, 80.96306523990334, 80.46254746289264, 79.91025198481188, 79.3579565067311, 78.73662409389023, 78.09803244735933, 77.56299620296858, 77.06247842595788, 76.56196064894718, 76.14773904038661, 75.716258198136, 75.26751812219537, 74.76700034518467, 74.35277873662409, 73.99033482913359, 73.61063168795306, 73.33448394891266, 72.95478080773213, 72.52329996548153, 72.12633759061097, 71.71211598205039, 71.31515360717984, 70.79737659647911, 70.34863652053849, 69.76182257507766, 69.27856403175699, 68.76078702105626, 68.13945460821539, 67.65619606489471, 67.2074559889541, 66.60338280980325, 66.01656886434242, 65.44701415257163, 64.7221263375906, 64.03175698998965, 63.324128408698655, 62.633759061097685, 61.90887124611667, 61.183983431135644, 60.42457714877458, 59.82050396962375, 59.13013462202278, 58.42250604073178, 57.628581290990674, 56.86917500862961, 55.95443562305833, 55.160510873317214, 54.38384535726614, 53.658957542285115, 52.795995857783915, 51.91577493959267, 51.104590956161545, 50.22437003797031, 49.46496375560925, 48.55022437003797, 47.60096651708664, 46.68622713151536, 45.719710044874006, 44.82222989299275, 43.85571280635139, 42.906454953400065, 41.95719710044873, 41.059716948567484, 40.12771832930617, 39.1439420089748, 38.177424922333444, 37.21090783569208, 36.261649982740764, 35.31239212978944, 34.34587504314808, 33.39661719019675, 32.430100103555404, 31.446323783224017, 30.46254746289264, 29.46151190887124, 28.477735588539865, 27.545736969278558, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.07594062823611, 90.07594062823611, 90.04142216085606, 90.006903693476, 89.98964445978598, 89.95512599240594, 89.92060752502589, 89.90334829133586, 89.85157059026578, 89.74801518812563, 89.67897825336554, 89.55816361753539, 89.45460821539524, 89.31653434587504, 89.24749741111494, 89.04038660683466, 88.81601656886434, 88.60890576458405, 88.41905419399379, 88.19468415602347, 87.95305488436314, 87.59061097687263, 87.27994477045219, 86.98653779772178, 86.72764929237141, 86.33068691750086, 85.98550224370037, 85.45046599930963, 85.01898515705902, 84.53572661373835, 83.98343113565757, 83.46565412495684, 82.84432171211597, 82.27476700034518, 81.7224715222644, 81.08387987573352, 80.4280289955126, 79.8584742837418, 79.27166033828098, 78.58129099068, 77.92544011045909, 77.20055229547808, 76.37210907835691, 75.69899896444598, 74.95685191577493, 74.33551950293406, 73.64515015533311, 72.81670693821194, 72.02278218847083, 71.31515360717984, 70.5039696237487, 69.79634104245771, 68.96789782533655, 68.17397307559544, 67.4318260269244, 66.65516051087332, 65.70590265792198, 64.86020020711081, 63.99723852260959, 63.15153607179841, 62.3576113220573, 61.58094580600621, 60.59716948567484, 59.682430100103545, 58.87124611667242, 58.00828443217121, 57.14532274767, 56.28236106316879, 55.36762167759751, 54.52191922678633, 53.676216775975135, 52.76147739040387, 51.84673800483258, 50.9319986192613, 50.03451846738005, 49.18881601656886, 48.274076630997584, 47.34207801173628, 46.41007939247498, 45.443562305833616, 44.54608215395236, 43.59682430100103, 42.7166033828098, 41.85364169830859, 40.86986537797721, 39.920607525025886, 38.97134967207455, 38.073869520193306, 37.12461166724197, 36.175353814290645, 35.32965136347946, 34.32861580945805, 33.37935795650673, 32.378322402485324, 31.429064549534004, 30.445288229202617, 29.478771142561268, 28.52951328960994, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2030, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.09319986192614, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.05868139454608, 90.02416292716603, 90.006903693476, 90.006903693476, 89.93786675871591, 89.8688298239558, 89.81705212288574, 89.73075595443561, 89.57542285122541, 89.40283051432516, 89.26475664480496, 89.16120124266482, 88.93683120469451, 88.7124611667242, 88.59164653089402, 88.31549879185364, 88.0738695201933, 87.84949948222298, 87.43527787366241, 87.12461166724196, 86.74490852606144, 86.5723161891612, 86.20987228167068, 85.55402140144977, 84.9326889886089, 84.4321712115982, 83.82809803244736, 83.18950638591647, 82.55091473938558, 81.91232309285468, 81.32550914739384, 80.77321366931308, 80.11736278909216, 79.32343803935105, 78.68484639282015, 78.04625474628925, 77.39040386606834, 76.64825681739731, 75.9406282361063, 75.18122195374525, 74.35277873662409, 73.69692785640318, 72.81670693821194, 72.16085605799101, 71.38419054193994, 70.65930272695891, 69.84811874352779, 68.95063859164652, 68.2257507766655, 67.29375215740421, 66.49982740766309, 65.70590265792198, 64.89471867449085, 63.96272005522954, 63.134276838108384, 62.3576113220573, 61.39109423541594, 60.459095616154634, 59.61339316534345, 58.7849499482223, 57.87021056265102, 57.00724887814981, 56.24784259578875, 55.28132550914739, 54.40110459095615, 53.46910597169485, 52.76147739040387, 51.898515705902646, 51.104590956161545, 50.20711080428028, 49.326889886089056, 48.53296513634795, 47.704521919226785, 46.7897825336555, 45.96133931653434, 45.2191922678633, 44.390749050742144, 43.476009665170864, 42.682084915429755, 41.76734552985847, 40.88712461166723, 39.95512599240594, 39.005868139454606, 38.03935105281325, 37.09009319986192, 36.15809458060062, 35.24335519502934, 34.32861580945805, 33.32758025543665, 32.378322402485324, 31.411805315843978, 30.445288229202617, 29.478771142561268, 28.512254055919918, 27.52847773558854, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2030, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.1449775629962, 90.09319986192614, 90.09319986192614, 90.02416292716603, 89.97238522609597, 89.90334829133586, 89.90334829133586, 89.85157059026578, 89.73075595443561, 89.69623748705557, 89.66171901967553, 89.55816361753539, 89.48912668277528, 89.3683120469451, 89.24749741111494, 89.10942354159475, 88.86779426993441, 88.76423886779426, 88.59164653089402, 88.36727649292371, 88.03935105281326, 87.65964791163272, 87.2109078356921, 86.76216775975146, 86.43424231964102, 85.98550224370037, 85.55402140144977, 85.01898515705902, 84.55298584742837, 84.10424577148774, 83.62098722816707, 82.9996548153262, 82.4991370383155, 81.7915084570245, 81.04936140835346, 80.44528822920262, 79.68588194684155, 79.0818087676907, 78.39143942008975, 77.70107007248878, 76.9416637901277, 76.25129444252676, 75.40559199171557, 74.73248187780463, 73.92129789437348, 73.09285467725233, 72.29892992751121, 71.40144977562996, 70.6420434932689, 69.76182257507766, 68.89886089057646, 68.17397307559544, 67.4318260269244, 66.62064204349328, 65.68864342423197, 64.91197790818087, 64.08353469105973, 63.2205730065585, 62.323092854677256, 61.49464963755609, 60.56265101829479, 59.82050396962375, 58.974801518812555, 58.16361753538142, 57.26613738350017, 56.38591646530894, 55.436658612357604, 54.67725232999654, 53.86606834656541, 52.96858819468415, 52.174663444943036, 51.25992405937176, 50.39696237487055, 49.430445288229194, 48.44666896789782, 47.56644804970659, 46.738004832585425, 45.8577839143942, 45.04659993096306, 44.114601311701755, 43.2171211598205, 42.319641007939246, 41.422160856057985, 40.45564376941663, 39.55816361753538, 38.6434242319641, 37.72868484639281, 36.79668622713152, 35.8646876078702, 34.91542975491888, 34.000690369347595, 33.06869175008629, 32.18847083189505, 31.273731446323783, 30.341732827062472, 29.409734207801176, 28.46047635484984, 27.49395926820849, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "MiniCPM", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.05868139454608, 89.73075595443561, 89.31653434587504, 88.98860890576458, 88.45357266137384, 87.97031411805315, 87.5043148084225, 87.10735243355194, 86.77942699344149, 86.24439074905074, 85.70935450465998, 85.24335519502934, 84.76009665170866, 84.25957887469797, 83.70728339661719, 83.25854332067657, 82.7925440110459, 82.2920262340352, 81.70521228857439, 81.17017604418363, 80.58336209872282, 80.08284432171212, 79.53054884363134, 78.9955125992406, 78.47773558853987, 77.83914394200897, 77.25232999654816, 76.66551605108734, 76.11322057300656, 75.54366586123575, 75.008629616845, 74.43907490507421, 73.85226095961339, 73.24818778046254, 72.69589230238176, 72.0055229547808, 71.40144977562996, 70.69382119433897, 69.98619261304798, 69.31308249913704, 68.60545391784605, 67.863306869175, 67.13841905419399, 66.41353123921297, 65.70590265792198, 65.15360717984122, 64.49775629962029, 63.84190541939938, 63.08249913703832, 62.37487055574732, 61.684501208146344, 60.959613393165334, 60.303762512944424, 59.54435623058336, 58.83672764929238, 58.12909906800138, 57.421470486710376, 56.67932343803934, 56.040731791508456, 55.28132550914739, 54.435623058336205, 53.69347600966516, 52.96858819468415, 52.27821884708319, 51.55333103210218, 50.828443217121155, 50.06903693476009, 49.27511218501898, 48.56748360372799, 47.911632723507076, 47.10044874007594, 46.2892647566448, 45.52985847428374, 44.83948912668277, 44.16637901277183, 43.49326889886089, 42.69934414911978, 41.95719710044873, 41.30134622022782, 40.490162236796685, 39.81705212288574, 39.09216430790473, 38.45357266137383, 37.76320331377286, 36.952019330341734, 36.15809458060062, 35.5022437003797, 34.72557818432861, 34.000690369347595, 33.37935795650673, 32.74076630997583, 32.084915429754915, 31.360027614773898, 30.652399033482915, 30.08284432171211, 29.426993441491188, 28.77114256127028, 28.2188470831895, 27.442181567138412, 26.56196064894718]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.04142216085606, 89.97238522609597, 89.93786675871591, 89.90334829133586, 89.85157059026578, 89.81705212288574, 89.78253365550569, 89.73075595443561, 89.69623748705557, 89.60994131860545, 89.55816361753539, 89.42008974801519, 89.31653434587504, 89.2129789437349, 89.16120124266482, 89.16120124266482, 89.05764584052469, 88.95409043838454, 88.88505350362443, 88.78149810148429, 88.69520193303417, 88.574387297204, 88.41905419399379, 88.28098032447359, 88.09112875388332, 87.97031411805315, 87.79772178115292, 87.64238867794269, 87.46979634104245, 87.31446323783224, 87.19364860200207, 87.09009319986191, 86.90024162927166, 86.65861235761132, 86.39972385226096, 86.19261304798067, 85.98550224370037, 85.70935450465998, 85.39868829823956, 85.12254055919917, 84.96720745598894, 84.74283741801864, 84.46668967897824, 84.1387642388678, 83.84535726613737, 83.58646876078701, 83.2930617880566, 82.9306178805661, 82.51639627200552, 82.30928546772522, 81.96410079392474, 81.70521228857439, 81.37728684846392, 80.96306523990334, 80.53158439765275, 80.30721435968242, 79.91025198481188, 79.40973420780118, 79.06454953400069, 78.54677252329995, 78.09803244735933, 77.56299620296858, 76.99344149119779, 76.54470141525715, 75.90610976872627, 75.40559199171557, 74.87055574732481, 74.19744563341388, 73.5070762858129, 72.88574387297204, 72.45426303072144, 71.81567138419054, 71.10804280289955, 70.46945115636866, 69.79634104245771, 69.00241629271659, 68.27752847773559, 67.53538142906454, 66.77597514670349, 65.96479116327234, 65.15360717984122, 64.4287193648602, 63.68657231618915, 62.96168450120814, 62.20227821884707, 61.44287193648601, 60.648947186744905, 59.82050396962375, 59.19917155678287, 58.33620987228167, 57.50776665516051, 56.61028650327925, 55.712806351397994, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.07594062823611, 89.92060752502589, 89.62720055229548, 89.57542285122541, 89.43734898170521, 89.28201587849499, 89.26475664480496, 89.04038660683466, 88.85053503624438, 88.79875733517432, 88.69520193303417, 88.43631342768381, 88.19468415602347, 87.81498101484294, 87.62512944425266, 87.43527787366241, 87.2109078356921, 86.95201933034173, 86.5723161891612, 86.33068691750086, 85.98550224370037, 85.67483603727995, 85.45046599930963, 85.10528132550914, 84.86365205384881, 84.65654124956852, 84.46668967897824, 84.19054193993787, 83.96617190196754, 83.65550569554712, 83.44839489126683, 83.22402485329651, 82.94787711425612, 82.75802554366585, 82.61995167414567, 82.34380393510527, 82.01587849499482, 81.67069382119433, 81.34276838108387, 81.03210217466345, 80.721435968243, 80.39351052813255, 80.13462202278218, 79.84121505005177, 79.54780807732136, 79.2198826372109, 78.82292026234035, 78.65032792544011, 78.33966171901967, 78.02899551259924, 77.63203313772868, 77.32136693130825, 77.01070072488781, 76.61373835001726, 76.25129444252676, 76.00966517086641, 75.66448049706594, 75.38833275802554, 75.09492578529513, 74.76700034518467, 74.35277873662409, 74.0766309975837, 73.78322402485328, 73.31722471522264, 73.02381774249223, 72.78218847083188, 72.38522609596134, 72.02278218847083, 71.69485674836037, 71.29789437348981, 70.90093199861926, 70.55574732481877, 70.27959958577839, 69.88263721090784, 69.52019330341732, 69.2095270969969, 68.74352778736623, 68.24301001035553, 67.81152916810494, 67.4318260269244, 66.94856748360372, 66.44804970659303, 66.0683465654125, 65.58508802209182, 65.13634794615119, 64.65308940283052, 64.10079392474974, 63.58301691404902, 63.01346220227821, 62.47842595788746, 61.77079737659648, 61.149464963755605, 60.59716948567484, 59.872281670693816, 59.16465308940282, 58.2844321712116, 57.45598895409043, 56.627545736969275, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.05868139454608, 89.98964445978598, 89.83431135657577, 89.67897825336554, 89.48912668277528, 89.38557128063513, 89.16120124266482, 88.90231273731446, 88.72972040041421, 88.55712806351399, 88.33275802554367, 88.09112875388332, 87.78046254746289, 87.60787021056264, 87.45253710735244, 87.24542630307214, 86.86572316189161, 86.64135312392129, 86.27890921643079, 85.98550224370037, 85.62305833620987, 85.34691059716948, 84.96720745598894, 84.62202278218847, 84.38039351052812, 83.98343113565757, 83.6382464618571, 83.2930617880566, 83.10321021746634, 82.7925440110459, 82.4991370383155, 82.11943389713497, 81.77424922333448, 81.4981014842941, 81.23921297894373, 80.87676907145323, 80.44528822920262, 80.04832585433206, 79.75491888160165, 79.54780807732136, 79.30617880566102, 78.94373489817052, 78.77114256127028, 78.58129099068, 78.33966171901967, 78.04625474628925, 77.83914394200897, 77.5802554366586, 77.16603382809804, 76.80358992060752, 76.40662754573697, 76.02692440455642, 75.66448049706594, 75.25025888850536, 74.92233344839488, 74.5081118398343, 74.12840869865379, 73.86952019330342, 73.59337245426303, 73.31722471522264, 73.11011391094236, 72.79944770452191, 72.55781843286158, 72.35070762858129, 72.09181912323092, 71.83293061788056, 71.59130134622022, 71.21159820503969, 70.88367276492924, 70.60752502588885, 70.27959958577839, 70.03797031411804, 69.74456334138763, 69.36486020020712, 68.9333793579565, 68.570935450466, 68.27752847773559, 67.88056610286503, 67.50086296168449, 67.0693821194339, 66.62064204349328, 66.17190196755264, 65.72316189161201, 65.20538488091128, 64.73938557128062, 64.25612702795995, 63.772868484639275, 63.25509147393855, 62.80635139799793, 62.271315153607176, 61.77079737659648, 61.16672419744563, 60.44183638246461, 59.83776320331378, 59.14739385571281, 58.26717293752157, 57.473248187780456, 56.59302726958922, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.07594062823611, 90.04142216085606, 89.97238522609597, 89.93786675871591, 89.90334829133586, 89.85157059026578, 89.81705212288574, 89.78253365550569, 89.73075595443561, 89.69623748705557, 89.60994131860545, 89.55816361753539, 89.42008974801519, 89.31653434587504, 89.2129789437349, 89.16120124266482, 89.16120124266482, 89.05764584052469, 88.95409043838454, 88.88505350362443, 88.78149810148429, 88.69520193303417, 88.574387297204, 88.41905419399379, 88.28098032447359, 88.09112875388332, 87.97031411805315, 87.79772178115292, 87.64238867794269, 87.46979634104245, 87.31446323783224, 87.19364860200207, 87.09009319986191, 86.90024162927166, 86.65861235761132, 86.39972385226096, 86.19261304798067, 85.98550224370037, 85.70935450465998, 85.39868829823956, 85.12254055919917, 84.96720745598894, 84.74283741801864, 84.46668967897824, 84.1387642388678, 83.84535726613737, 83.58646876078701, 83.2930617880566, 82.9306178805661, 82.51639627200552, 82.30928546772522, 81.96410079392474, 81.70521228857439, 81.37728684846392, 80.96306523990334, 80.53158439765275, 80.30721435968242, 79.91025198481188, 79.40973420780118, 79.06454953400069, 78.54677252329995, 78.09803244735933, 77.56299620296858, 76.99344149119779, 76.54470141525715, 75.90610976872627, 75.40559199171557, 74.87055574732481, 74.19744563341388, 73.5070762858129, 72.88574387297204, 72.45426303072144, 71.81567138419054, 71.10804280289955, 70.46945115636866, 69.79634104245771, 69.00241629271659, 68.27752847773559, 67.53538142906454, 66.77597514670349, 65.96479116327234, 65.15360717984122, 64.4287193648602, 63.68657231618915, 62.96168450120814, 62.20227821884707, 61.44287193648601, 60.648947186744905, 59.82050396962375, 59.19917155678287, 58.33620987228167, 57.50776665516051, 56.61028650327925, 55.712806351397994, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2026, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.05868139454608, 90.04142216085606, 90.02416292716603, 90.006903693476, 89.98964445978598, 89.95512599240594, 89.92060752502589, 89.85157059026578, 89.85157059026578, 89.81705212288574, 89.73075595443561, 89.69623748705557, 89.57542285122541, 89.43734898170521, 89.35105281325508, 89.24749741111494, 89.23023817742491, 89.10942354159475, 88.97134967207455, 88.83327580255437, 88.74697963410424, 88.64342423196409, 88.52260959613393, 88.33275802554367, 88.16016568864342, 87.91853641698309, 87.81498101484294, 87.69416637901277, 87.62512944425266, 87.43527787366241, 87.29720400414222, 87.03831549879185, 86.88298239558164, 86.69313082499137, 86.41698308595099, 86.15809458060062, 85.98550224370037, 85.72661373835001, 85.5022437003797, 85.26061442871936, 85.001725923369, 84.77735588539869, 84.62202278218847, 84.22506040731791, 83.89713496720745, 83.72454263030721, 83.4311356575768, 83.1377286848464, 82.861580945806, 82.4991370383155, 82.20573006558509, 81.84328615809457, 81.51536071798412, 81.08387987573352, 80.75595443562305, 80.41076976182258, 79.996548153262, 79.68588194684155, 79.27166033828098, 78.80566102865032, 78.42595788746979, 77.7873662409389, 77.33862616499827, 76.82084915429755, 76.37210907835691, 75.81981360027615, 75.11218501898516, 74.42181567138418, 73.88677942699344, 73.24818778046254, 72.78218847083188, 72.21263375906109, 71.53952364515015, 70.90093199861926, 70.19330341732827, 69.72730410769762, 69.12323092854677, 68.46738004832585, 67.91508457024507, 67.31101139109424, 66.67241974456334, 66.03382809803244, 65.27442181567137, 64.49775629962029, 63.85916465308941, 63.134276838108384, 62.47842595788746, 61.70176044183637, 60.90783569209527, 60.09665170866414, 59.26820849154297, 58.35346910597169, 57.473248187780456, 56.627545736969275, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2026, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.006903693476, 89.90334829133586, 89.88608905764583, 89.83431135657577, 89.79979288919571, 89.73075595443561, 89.66171901967553, 89.62720055229548, 89.59268208491544, 89.5063859164653, 89.45460821539524, 89.43734898170521, 89.40283051432516, 89.3683120469451, 89.28201587849499, 89.24749741111494, 89.16120124266482, 89.10942354159475, 89.02312737314463, 88.90231273731446, 88.76423886779426, 88.64342423196409, 88.574387297204, 88.5053503624439, 88.41905419399379, 88.31549879185364, 88.22920262340351, 88.17742492233344, 88.05661028650327, 87.91853641698309, 87.7114256127028, 87.53883327580255, 87.36624093890231, 87.14187090093199, 87.02105626510182, 86.79668622713152, 86.60683465654124, 86.41698308595099, 86.19261304798067, 86.08905764584051, 85.89920607525026, 85.72661373835001, 85.60579910251984, 85.41594753192958, 85.26061442871936, 85.03624439074905, 84.72557818432861, 84.38039351052812, 84.03520883672765, 83.84535726613737, 83.5001725923369, 83.24128408698654, 82.89609941318605, 82.41284086986538, 82.01587849499482, 81.68795305488436, 81.37728684846392, 81.06662064204349, 80.53158439765275, 80.18639972385225, 79.75491888160165, 79.34069727304107, 78.84017949603037, 78.25336555056955, 77.7183293061788, 77.26958923023818, 76.90714532274767, 76.52744218156712, 75.90610976872627, 75.33655505695546, 74.80151881256472, 74.26648256817397, 73.69692785640318, 73.0755954435623, 72.43700379703141, 71.88470831895064, 71.28063513979978, 70.69382119433897, 69.95167414566794, 69.26130479806696, 68.41560234725577, 67.74249223334483, 67.0693821194339, 66.3617535381429, 65.60234725578185, 64.94649637556091, 64.2216085605799, 63.479461511908866, 62.59924059371764, 61.822575077666556, 61.183983431135644, 60.372799447704516, 59.61339316534345, 58.92302381774248, 58.19813600276147, 57.45598895409043, 56.69658267172937, 55.988954090438384, 55.40214014497755, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 89.97238522609597, 89.90334829133586, 89.55816361753539, 89.42008974801519, 89.02312737314463, 88.7124611667242, 88.5053503624439, 88.22920262340351, 88.03935105281326, 87.74594408008284, 87.46979634104245, 87.15913013462202, 86.93476009665171, 86.58957542285121, 86.26164998274076, 85.95098377632033, 85.62305833620987, 85.26061442871936, 84.9326889886089, 84.65654124956852, 84.31135657576803, 84.05246807041767, 83.65550569554712, 83.31032102174663, 83.01691404901622, 82.58543320676561, 82.32654470141524, 81.98136002761477, 81.70521228857439, 81.23921297894373, 80.89402830514325, 80.47980669658267, 80.15188125647221, 79.66862271315154, 79.30617880566102, 78.97825336555057, 78.75388332758025, 78.56403175698998, 78.23610631687953, 77.89092164307905, 77.61477390403866, 77.39040386606834, 77.06247842595788, 76.7172937521574, 76.38936831204694, 76.14773904038661, 75.87159130134621, 75.59544356230583, 75.14670348636521, 74.85329651363479, 74.52537107352433, 74.31826026924404, 74.02485329651363, 73.73144632378322, 73.45529858474283, 73.02381774249223, 72.66137383500173, 72.33344839489126, 71.95374525371074, 71.52226441146013, 71.19433897134967, 70.83189506385916, 70.33137728684846, 69.98619261304798, 69.5719710044874, 69.12323092854677, 68.77804625474629, 68.45012081463582, 68.05315843976527, 67.69071453227477, 67.31101139109424, 66.7932343803935, 66.44804970659303, 66.05108733172247, 65.55056955471177, 65.08457024508112, 64.73938557128062, 64.2216085605799, 63.9282015878495, 63.49672074559889, 63.047980669658266, 62.73731446323782, 62.271315153607176, 61.90887124611667, 61.39109423541594, 60.97687262685536, 60.57991025198481, 60.11391094235415, 59.88954090438384, 59.40628236106317, 58.88850535036244, 58.474283741801855, 58.00828443217121, 57.525025888850536, 57.14532274767, 56.73110113910942, 56.19606489471867, 55.79910251984812, 55.350362443907485, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.09319986192614, 90.07594062823611, 90.05868139454608, 90.006903693476, 89.97238522609597, 89.95512599240594, 89.88608905764583, 89.85157059026578, 89.78253365550569, 89.74801518812563, 89.67897825336554, 89.62720055229548, 89.54090438384536, 89.42008974801519, 89.35105281325508, 89.24749741111494, 89.16120124266482, 89.05764584052469, 88.91957197100449, 88.85053503624438, 88.76423886779426, 88.66068346565412, 88.47083189506385, 88.33275802554367, 88.26372109078356, 88.09112875388332, 87.93579565067311, 87.76320331377286, 87.5733517431826, 87.34898170521228, 87.2109078356921, 87.02105626510182, 86.84846392820158, 86.72764929237141, 86.46876078702105, 86.24439074905074, 86.02002071108042, 85.77839143942009, 85.5712806351398, 85.26061442871936, 85.01898515705902, 84.72557818432861, 84.39765274421815, 84.24231964100794, 84.01794960303762, 83.72454263030721, 83.46565412495684, 83.12046945115637, 82.7925440110459, 82.56817397307559, 82.27476700034518, 82.01587849499482, 81.70521228857439, 81.51536071798412, 81.20469451156367, 80.70417673455297, 80.39351052813255, 79.9275112185019, 79.61684501208146, 79.2198826372109, 78.84017949603037, 78.47773558853987, 78.02899551259924, 77.39040386606834, 76.83810838798757, 76.51018294787711, 76.09596133931653, 75.45736969278563, 74.95685191577493, 74.33551950293406, 73.67966862271315, 73.02381774249223, 72.43700379703141, 71.86744908526062, 71.10804280289955, 70.43493268898861, 69.77908180876769, 69.19226786330687, 68.39834311356576, 67.7079737659648, 67.05212288574387, 66.3617535381429, 65.63686572316189, 64.9292371418709, 64.11805315843976, 63.410424577148774, 62.59924059371764, 61.73627890921642, 60.787021056265104, 59.99309630652399, 59.19917155678287, 58.387987573351744, 57.59406282361063, 56.6448049706593, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 90.02416292716603, 89.78253365550569, 89.67897825336554, 89.6444597859855, 89.5063859164653, 89.40283051432516, 89.17846047635484, 88.95409043838454, 88.81601656886434, 88.59164653089402, 88.47083189506385, 88.28098032447359, 87.91853641698309, 87.78046254746289, 87.65964791163272, 87.45253710735244, 87.24542630307214, 87.03831549879185, 86.7103900586814, 86.45150155333103, 86.20987228167068, 85.91646530894027, 85.55402140144977, 85.17431826026925, 84.77735588539869, 84.4321712115982, 83.91439420089748, 83.56920952709699, 83.25854332067657, 83.06869175008629, 82.70624784259579, 82.30928546772522, 81.91232309285468, 81.70521228857439, 81.44632378322402, 81.06662064204349, 80.68691750086295, 80.46254746289264, 80.18639972385225, 79.94477045219192, 79.65136347946151, 79.4269934414912, 79.11632723507076, 78.82292026234035, 78.52951328960994, 78.33966171901967, 78.09803244735933, 77.73558853986883, 77.39040386606834, 77.16603382809804, 76.88988608905764, 76.5792198826372, 76.37210907835691, 76.18225750776665, 75.85433206765619, 75.5781843286158, 75.2847773558854, 75.02588885053503, 74.88781498101484, 74.5771487745944, 74.24922333448394, 73.83500172592336, 73.576113220573, 73.28270624784258, 72.98929927511219, 72.69589230238176, 72.36796686227132, 72.0055229547808, 71.67759751467034, 71.47048671039006, 71.05626510182947, 70.90093199861926, 70.5730065585088, 70.19330341732827, 69.77908180876769, 69.43389713496721, 69.01967552640662, 68.67449085260614, 68.17397307559544, 67.74249223334483, 67.27649292371419, 66.84501208146358, 66.37901277183292, 65.82671729375215, 65.2916810493614, 64.82568173973075, 64.20434932688988, 63.548498446668965, 62.89264756644805, 62.16775975146703, 61.51190887124611, 60.75250258888505, 60.06213324128409, 59.285467725233, 58.42250604073178, 57.55954435623059, 56.66206420434931, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.12771832930618, 90.02416292716603, 89.81705212288574, 89.69623748705557, 89.55816361753539, 89.42008974801519, 89.35105281325508, 89.38557128063513, 89.09216430790472, 88.91957197100449, 88.62616499827408, 88.45357266137384, 88.26372109078356, 88.09112875388332, 87.91853641698309, 87.5733517431826, 87.34898170521228, 87.15913013462202, 86.91750086296169, 86.62409389023126, 86.22713151536071, 85.84742837418018, 85.51950293406972, 85.10528132550914, 84.74283741801864, 84.24231964100794, 83.81083879875733, 83.44839489126683, 82.91335864687608, 82.46461857093544, 82.08491542975491, 81.70521228857439, 81.18743527787366, 80.721435968243, 80.23817742492233, 79.77217811529168, 79.20262340352087, 78.7193648602002, 78.30514325163962, 77.95995857783915, 77.70107007248878, 77.37314463237833, 77.09699689333793, 76.88988608905764, 76.66551605108734, 76.49292371418709, 76.2167759751467, 75.95788746979633, 75.73351743182603, 75.59544356230583, 75.40559199171557, 75.18122195374525, 74.99137038315499, 74.73248187780463, 74.55988954090438, 74.19744563341388, 73.93855712806351, 73.64515015533311, 73.36900241629272, 73.0755954435623, 72.92026234035208, 72.78218847083188, 72.40248532965136, 72.10907835692095, 71.74663444943043, 71.5050051777701, 71.31515360717984, 71.02174663444943, 70.72833966171902, 70.46945115636866, 70.27959958577839, 69.93441491197791, 69.62374870555747, 69.22678633068692, 68.95063859164652, 68.60545391784605, 68.27752847773559, 67.96686227131515, 67.69071453227477, 67.34552985847428, 66.96582671729375, 66.46530894028305, 66.0683465654125, 65.65412495685192, 65.22264411460131, 64.68760787021056, 64.16983085950983, 63.634794615119084, 63.03072143596824, 62.40938902312737, 61.87435277873662, 61.25302036589575, 60.57991025198481, 59.85502243700379, 59.13013462202278, 58.31895063859165, 57.50776665516051, 56.627545736969275, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.09319986192614, 90.07594062823611, 90.05868139454608, 90.006903693476, 89.97238522609597, 89.95512599240594, 89.88608905764583, 89.85157059026578, 89.78253365550569, 89.74801518812563, 89.67897825336554, 89.62720055229548, 89.54090438384536, 89.42008974801519, 89.35105281325508, 89.24749741111494, 89.16120124266482, 89.05764584052469, 88.91957197100449, 88.85053503624438, 88.76423886779426, 88.66068346565412, 88.47083189506385, 88.33275802554367, 88.26372109078356, 88.09112875388332, 87.93579565067311, 87.76320331377286, 87.5733517431826, 87.34898170521228, 87.2109078356921, 87.02105626510182, 86.84846392820158, 86.72764929237141, 86.46876078702105, 86.24439074905074, 86.02002071108042, 85.77839143942009, 85.5712806351398, 85.26061442871936, 85.01898515705902, 84.72557818432861, 84.39765274421815, 84.24231964100794, 84.01794960303762, 83.72454263030721, 83.46565412495684, 83.12046945115637, 82.7925440110459, 82.56817397307559, 82.27476700034518, 82.01587849499482, 81.70521228857439, 81.51536071798412, 81.20469451156367, 80.70417673455297, 80.39351052813255, 79.9275112185019, 79.61684501208146, 79.2198826372109, 78.84017949603037, 78.47773558853987, 78.02899551259924, 77.39040386606834, 76.83810838798757, 76.51018294787711, 76.09596133931653, 75.45736969278563, 74.95685191577493, 74.33551950293406, 73.67966862271315, 73.02381774249223, 72.43700379703141, 71.86744908526062, 71.10804280289955, 70.43493268898861, 69.77908180876769, 69.19226786330687, 68.39834311356576, 67.7079737659648, 67.05212288574387, 66.3617535381429, 65.63686572316189, 64.9292371418709, 64.11805315843976, 63.410424577148774, 62.59924059371764, 61.73627890921642, 60.787021056265104, 59.99309630652399, 59.19917155678287, 58.387987573351744, 57.59406282361063, 56.6448049706593, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2027, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.02416292716603, 90.006903693476, 89.95512599240594, 89.90334829133586, 89.90334829133586, 89.83431135657577, 89.76527442181566, 89.73075595443561, 89.7134967207456, 89.66171901967553, 89.5063859164653, 89.42008974801519, 89.3683120469451, 89.31653434587504, 89.23023817742491, 89.14394200897479, 89.07490507421471, 88.95409043838454, 88.81601656886434, 88.59164653089402, 88.5053503624439, 88.36727649292371, 88.26372109078356, 88.0738695201933, 87.98757335174318, 87.76320331377286, 87.52157404211253, 87.24542630307214, 87.05557473248187, 86.91750086296169, 86.76216775975146, 86.5723161891612, 86.29616845012082, 86.15809458060062, 85.96824301001035, 85.81290990680014, 85.64031756989989, 85.36416983085951, 85.05350362443907, 84.82913358646876, 84.46668967897824, 84.19054193993787, 83.8626164998274, 83.60372799447704, 83.31032102174663, 82.9996548153262, 82.7925440110459, 82.51639627200552, 82.17121159820505, 81.98136002761477, 81.61891612012427, 81.2909906800138, 80.9285467725233, 80.53158439765275, 80.23817742492233, 79.89299275112185, 79.4960303762513, 79.09906800138073, 78.68484639282015, 78.3569209527097, 77.77010700724888, 77.25232999654816, 76.76907145322747, 76.38936831204694, 75.76803589920607, 75.14670348636521, 74.47359337245426, 73.85226095961339, 73.21366931308249, 72.59233690024163, 71.98826372109079, 71.28063513979978, 70.5730065585088, 69.93441491197791, 69.24404556437693, 68.4328615809458, 67.67345529858474, 66.9313082499137, 66.18916120124267, 65.32619951674145, 64.51501553331032, 63.75560925094925, 62.96168450120814, 62.25405591991715, 61.52916810493614, 60.787021056265104, 60.01035554021401, 59.26820849154297, 58.40524680704176, 57.49050742147048, 56.6448049706593, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2027, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.11045909561615, 90.006903693476, 89.97238522609597, 89.93786675871591, 89.92060752502589, 89.90334829133586, 89.85157059026578, 89.81705212288574, 89.74801518812563, 89.59268208491544, 89.54090438384536, 89.43734898170521, 89.35105281325508, 89.2129789437349, 89.14394200897479, 89.10942354159475, 89.02312737314463, 88.95409043838454, 88.85053503624438, 88.7124611667242, 88.60890576458405, 88.52260959613393, 88.43631342768381, 88.24646185709354, 88.09112875388332, 87.91853641698309, 87.76320331377286, 87.60787021056264, 87.48705557473248, 87.29720400414222, 87.17638936831204, 86.95201933034173, 86.74490852606144, 86.45150155333103, 86.26164998274076, 86.08905764584051, 85.95098377632033, 85.64031756989989, 85.46772523299964, 85.27787366240939, 85.001725923369, 84.74283741801864, 84.55298584742837, 84.41491197790818, 84.20780117362789, 83.96617190196754, 83.67276492923715, 83.32758025543666, 83.17224715222645, 82.94787711425612, 82.67172937521575, 82.41284086986538, 82.06765619606489, 81.77424922333448, 81.30824991370382, 80.91128753883328, 80.4970659302727, 80.15188125647221, 79.7894373489817, 79.39247497411115, 78.9955125992406, 78.56403175698998, 78.09803244735933, 77.7183293061788, 77.32136693130825, 76.76907145322747, 76.26855367621677, 75.83707283396618, 75.2157404211253, 74.61166724197446, 74.12840869865379, 73.54159475319295, 72.97204004142216, 72.52329996548153, 71.95374525371074, 71.45322747670004, 71.0044874007594, 70.48671039005868, 69.86537797721782, 69.2095270969969, 68.62271315153608, 67.98412150500516, 67.29375215740421, 66.65516051087332, 65.89575422851225, 65.15360717984122, 64.39420089748015, 63.61753538142907, 62.87538833275802, 62.20227821884707, 61.39109423541594, 60.54539178460476, 59.75146703486365, 58.974801518812555, 58.19813600276147, 57.317915084570245, 56.437694166379, 55.626510182947875, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.006903693476, 89.83431135657577, 89.62720055229548, 89.40283051432516, 89.23023817742491, 88.85053503624438, 88.48809112875388, 88.10838798757334, 87.76320331377286, 87.36624093890231, 87.05557473248187, 86.74490852606144, 86.39972385226096, 86.22713151536071, 85.95098377632033, 85.60579910251984, 85.17431826026925, 84.77735588539869, 84.63928201587849, 84.36313427683811, 84.08698653779771, 83.65550569554712, 83.27580255436658, 82.84432171211597, 82.58543320676561, 82.15395236451502, 81.84328615809457, 81.4981014842941, 81.08387987573352, 80.7904729030031, 80.56610286503279, 80.25543665861235, 79.87573351743183, 79.5823265447014, 79.23714187090093, 78.9955125992406, 78.7193648602002, 78.32240248532965, 78.02899551259924, 77.68381083879875, 77.26958923023818, 76.87262685536763, 76.54470141525715, 76.14773904038661, 75.88885053503624, 75.59544356230583, 75.33655505695546, 75.02588885053503, 74.62892647566449, 74.37003797031412, 74.0766309975837, 73.80048325854331, 73.36900241629272, 73.0755954435623, 72.76492923714186, 72.38522609596134, 72.05730065585088, 71.62581981360027, 71.41870900931998, 71.21159820503969, 70.86641353123922, 70.46945115636866, 70.0724887814981, 69.7100448740076, 69.39937866758716, 68.95063859164652, 68.62271315153608, 68.26026924404556, 67.94960303762512, 67.5698998964446, 67.15567828788402, 66.75871591301346, 66.30997583707283, 65.68864342423197, 65.39523645150155, 64.96375560925094, 64.60131170176044, 64.34242319641008, 63.979979288919566, 63.58301691404902, 63.099758370728345, 62.66827752847773, 62.25405591991715, 61.8570935450466, 61.49464963755609, 61.02865032792544, 60.56265101829479, 60.303762512944424, 59.872281670693816, 59.42354159475318, 58.957542285122535, 58.49154297549188, 58.025543665861235, 57.50776665516051, 57.11080428028995, 56.69658267172937, 56.29962029685882, 55.83362098722817, 55.29858474283742, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.04142216085606, 90.006903693476, 89.95512599240594, 89.92060752502589, 89.83431135657577, 89.78253365550569, 89.74801518812563, 89.73075595443561, 89.67897825336554, 89.57542285122541, 89.47186744908525, 89.45460821539524, 89.35105281325508, 89.28201587849499, 89.17846047635484, 89.0058681394546, 88.90231273731446, 88.7124611667242, 88.62616499827408, 88.53986882982396, 88.35001725923368, 88.26372109078356, 88.2119433897135, 88.0738695201933, 87.90127718329306, 87.74594408008284, 87.5733517431826, 87.45253710735244, 87.27994477045219, 87.02105626510182, 86.79668622713152, 86.58957542285121, 86.46876078702105, 86.19261304798067, 86.0027614773904, 85.79565067311012, 85.69209527096997, 85.45046599930963, 85.24335519502934, 85.08802209181911, 84.9326889886089, 84.81187435277874, 84.62202278218847, 84.25957887469797, 84.12150500517777, 83.8626164998274, 83.53469105971695, 83.17224715222645, 82.80980324473592, 82.51639627200552, 82.15395236451502, 81.87780462547462, 81.56713841905419, 81.3600276147739, 81.04936140835346, 80.68691750086295, 80.39351052813255, 79.996548153262, 79.59958577839143, 79.27166033828098, 78.87469796341043, 78.3569209527097, 77.80462547462892, 77.20055229547808, 76.61373835001726, 75.99240593717639, 75.30203658957542, 74.69796341042458, 73.95581636175353, 73.35174318260269, 72.71315153607179, 72.29892992751121, 71.59130134622022, 70.88367276492924, 70.14152571625819, 69.43389713496721, 68.69175008629617, 68.2084915429755, 67.50086296168449, 66.72419744563341, 65.87849499482222, 65.23990334829134, 64.49775629962029, 63.66931308249914, 62.87538833275802, 62.08146358301691, 61.235761132205724, 60.33828098032448, 59.458060062133235, 58.577839143942015, 57.66309975837072, 56.69658267172937, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.04142216085606, 89.85157059026578, 89.81705212288574, 89.7134967207456, 89.59268208491544, 89.3683120469451, 89.17846047635484, 88.91957197100449, 88.67794269934414, 88.47083189506385, 88.17742492233344, 87.91853641698309, 87.67690714532274, 87.45253710735244, 87.22816706938211, 87.05557473248187, 86.90024162927166, 86.64135312392129, 86.34794615119088, 86.24439074905074, 85.95098377632033, 85.69209527096997, 85.29513289609942, 84.96720745598894, 84.63928201587849, 84.53572661373835, 84.24231964100794, 83.96617190196754, 83.6382464618571, 83.34483948912667, 83.10321021746634, 82.84432171211597, 82.55091473938558, 82.32654470141524, 82.05039696237488, 81.70521228857439, 81.429064549534, 81.11839834311357, 80.9285467725233, 80.63513979979288, 80.4280289955126, 80.23817742492233, 80.04832585433206, 79.7894373489817, 79.39247497411115, 79.15084570245081, 78.77114256127028, 78.52951328960994, 78.25336555056955, 78.04625474628925, 77.63203313772868, 77.32136693130825, 76.99344149119779, 76.64825681739731, 76.42388677942698, 76.14773904038661, 75.81981360027615, 75.61270279599586, 75.31929582326543, 75.06040731791508, 74.88781498101484, 74.62892647566449, 74.42181567138418, 74.09389023127373, 73.90403866068347, 73.64515015533311, 73.43803935105281, 73.12737314463237, 72.73041076976182, 72.38522609596134, 72.04004142216085, 71.69485674836037, 71.38419054193994, 71.03900586813945, 70.59026579219882, 70.12426648256817, 69.69278564031757, 69.33034173282707, 68.79530548843631, 68.26026924404556, 67.62167759751468, 67.19019675526407, 66.74145667932343, 66.39627200552295, 65.7921988263721, 65.20538488091128, 64.67034863652053, 63.99723852260959, 63.46220227821884, 62.823610631687956, 62.20227821884707, 61.373835001725915, 60.63168795305488, 59.90680013807387, 59.11287538833275, 58.249913703831545, 57.45598895409043, 56.57576803589921, 55.712806351397994, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 89.95512599240594, 89.81705212288574, 89.79979288919571, 89.66171901967553, 89.52364515015533, 89.42008974801519, 89.31653434587504, 89.09216430790472, 88.72972040041421, 88.60890576458405, 88.29823955816362, 88.02209181912323, 87.76320331377286, 87.5043148084225, 87.17638936831204, 86.84846392820158, 86.67587159130134, 86.34794615119088, 86.10631687953054, 85.79565067311012, 85.5022437003797, 85.15705902657922, 84.77735588539869, 84.29409734207802, 84.05246807041767, 83.72454263030721, 83.24128408698654, 82.75802554366585, 82.30928546772522, 81.87780462547462, 81.41180531584398, 80.98032447359337, 80.46254746289264, 80.15188125647221, 79.80669658267172, 79.37521574042113, 78.97825336555057, 78.59855022437003, 78.20158784949948, 77.89092164307905, 77.51121850189851, 77.30410769761822, 77.09699689333793, 76.82084915429755, 76.59647911632723, 76.35484984466689, 76.09596133931653, 75.9406282361063, 75.716258198136, 75.56092509492578, 75.37107352433551, 75.06040731791508, 74.90507421470487, 74.80151881256472, 74.5081118398343, 74.31826026924404, 74.12840869865379, 73.86952019330342, 73.62789092164309, 73.36900241629272, 73.21366931308249, 72.92026234035208, 72.69589230238176, 72.41974456334138, 72.17811529168104, 71.98826372109079, 71.78115291681048, 71.5050051777701, 71.24611667241973, 71.02174663444943, 70.79737659647911, 70.55574732481877, 70.36589575422852, 70.05522954780807, 69.83085950983777, 69.33034173282707, 69.14049016223679, 68.91612012426648, 68.4328615809458, 68.01863997238522, 67.55264066275457, 67.19019675526407, 66.75871591301346, 66.25819813600276, 65.75768035899205, 65.27442181567137, 64.73938557128062, 64.23886779426992, 63.703831549879176, 63.047980669658266, 62.40938902312737, 61.59820503969624, 60.873317224715215, 60.09665170866414, 59.302726958923024, 58.49154297549188, 57.55954435623059, 56.59302726958922, 55.76458405246807, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.09319986192614, 90.04142216085606, 90.006903693476, 89.95512599240594, 89.92060752502589, 89.83431135657577, 89.78253365550569, 89.74801518812563, 89.73075595443561, 89.67897825336554, 89.57542285122541, 89.47186744908525, 89.45460821539524, 89.35105281325508, 89.28201587849499, 89.17846047635484, 89.0058681394546, 88.90231273731446, 88.7124611667242, 88.62616499827408, 88.53986882982396, 88.35001725923368, 88.26372109078356, 88.2119433897135, 88.0738695201933, 87.90127718329306, 87.74594408008284, 87.5733517431826, 87.45253710735244, 87.27994477045219, 87.02105626510182, 86.79668622713152, 86.58957542285121, 86.46876078702105, 86.19261304798067, 86.0027614773904, 85.79565067311012, 85.69209527096997, 85.45046599930963, 85.24335519502934, 85.08802209181911, 84.9326889886089, 84.81187435277874, 84.62202278218847, 84.25957887469797, 84.12150500517777, 83.8626164998274, 83.53469105971695, 83.17224715222645, 82.80980324473592, 82.51639627200552, 82.15395236451502, 81.87780462547462, 81.56713841905419, 81.3600276147739, 81.04936140835346, 80.68691750086295, 80.39351052813255, 79.996548153262, 79.59958577839143, 79.27166033828098, 78.87469796341043, 78.3569209527097, 77.80462547462892, 77.20055229547808, 76.61373835001726, 75.99240593717639, 75.30203658957542, 74.69796341042458, 73.95581636175353, 73.35174318260269, 72.71315153607179, 72.29892992751121, 71.59130134622022, 70.88367276492924, 70.14152571625819, 69.43389713496721, 68.69175008629617, 68.2084915429755, 67.50086296168449, 66.72419744563341, 65.87849499482222, 65.23990334829134, 64.49775629962029, 63.66931308249914, 62.87538833275802, 62.08146358301691, 61.235761132205724, 60.33828098032448, 59.458060062133235, 58.577839143942015, 57.66309975837072, 56.69658267172937, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2028, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.07594062823611, 90.07594062823611, 90.04142216085606, 89.95512599240594, 89.90334829133586, 89.88608905764583, 89.79979288919571, 89.78253365550569, 89.73075595443561, 89.66171901967553, 89.52364515015533, 89.47186744908525, 89.40283051432516, 89.33379357956507, 89.2129789437349, 89.12668277528478, 89.04038660683466, 88.91957197100449, 88.83327580255437, 88.7124611667242, 88.52260959613393, 88.38453572661373, 88.29823955816362, 88.10838798757334, 87.90127718329306, 87.78046254746289, 87.62512944425266, 87.55609250949257, 87.43527787366241, 87.27994477045219, 87.03831549879185, 86.84846392820158, 86.58957542285121, 86.52053848809112, 86.24439074905074, 86.05453917846047, 85.98550224370037, 85.84742837418018, 85.60579910251984, 85.38142906454954, 85.13979979288919, 84.88091128753882, 84.51846738004832, 84.31135657576803, 84.1387642388678, 83.9316534345875, 83.6382464618571, 83.32758025543666, 83.03417328270625, 82.70624784259579, 82.3610631687953, 82.05039696237488, 81.67069382119433, 81.20469451156367, 80.80773213669312, 80.53158439765275, 80.06558508802209, 79.65136347946151, 79.30617880566102, 78.89195719710045, 78.51225405591991, 78.14981014842941, 77.68381083879875, 77.2178115291681, 76.76907145322747, 76.2858129099068, 75.76803589920607, 75.26751812219537, 74.69796341042458, 74.04211253020365, 73.55885398688298, 72.85122540559199, 72.21263375906109, 71.62581981360027, 71.0735243355195, 70.31411805315844, 69.52019330341732, 68.77804625474629, 68.10493614083535, 67.38004832585432, 66.75871591301346, 65.98205039696236, 65.34345875043148, 64.70486710390058, 63.96272005522954, 63.203313772868476, 62.46116672419744, 61.63272350707628, 60.82153952364515, 59.97583707283396, 59.181912323092845, 58.301691404901625, 57.4387297204004, 56.57576803589921, 55.6437694166379, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2028, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.1449775629962, 90.05868139454608, 90.04142216085606, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.95512599240594, 89.90334829133586, 89.79979288919571, 89.69623748705557, 89.69623748705557, 89.6444597859855, 89.54090438384536, 89.47186744908525, 89.29927511218501, 89.19571971004487, 89.14394200897479, 89.02312737314463, 88.97134967207455, 88.86779426993441, 88.79875733517432, 88.574387297204, 88.47083189506385, 88.36727649292371, 88.22920262340351, 88.0048325854332, 87.88401794960303, 87.79772178115292, 87.69416637901277, 87.53883327580255, 87.38350017259233, 87.14187090093199, 86.95201933034173, 86.81394546082154, 86.62409389023126, 86.48602002071108, 86.29616845012082, 86.0027614773904, 85.9337245426303, 85.70935450465998, 85.5022437003797, 85.38142906454954, 85.12254055919917, 84.9326889886089, 84.70831895063858, 84.46668967897824, 84.15602347255782, 83.89713496720745, 83.53469105971695, 83.24128408698654, 82.96513634794614, 82.70624784259579, 82.41284086986538, 82.11943389713497, 81.8605453917846, 81.51536071798412, 81.17017604418363, 80.70417673455297, 80.20365895754227, 79.82395581636175, 79.32343803935105, 78.94373489817052, 78.39143942008975, 77.87366240938903, 77.37314463237833, 76.80358992060752, 76.2858129099068, 75.7852951328961, 75.30203658957542, 74.7152226441146, 74.16292716603382, 73.45529858474283, 72.81670693821194, 72.143596824301, 71.52226441146013, 70.74559889540903, 70.10700724887815, 69.34760096651709, 68.63997238522609, 67.82878840179495, 67.17293752157404, 66.55160510873317, 65.77493959268207, 65.08457024508112, 64.27338626164997, 63.46220227821884, 62.59924059371764, 61.7190196755264, 60.89057645840524, 60.027614773904034, 59.14739385571281, 58.301691404901625, 57.421470486710376, 56.61028650327925, 55.712806351397994, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.02416292716603, 89.74801518812563, 89.52364515015533, 89.35105281325508, 89.02312737314463, 88.69520193303417, 88.38453572661373, 88.0738695201933, 87.65964791163272, 87.45253710735244, 87.03831549879185, 86.76216775975146, 86.45150155333103, 86.22713151536071, 85.86468760787021, 85.51950293406972, 85.22609596133931, 84.89817052122885, 84.67380048325855, 84.31135657576803, 84.08698653779771, 83.74180186399724, 83.5001725923369, 83.17224715222645, 82.84432171211597, 82.58543320676561, 82.27476700034518, 81.9295823265447, 81.61891612012427, 81.41180531584398, 81.11839834311357, 80.80773213669312, 80.4280289955126, 80.15188125647221, 79.84121505005177, 79.56506731101139, 79.09906800138073, 78.84017949603037, 78.44321712115982, 78.02899551259924, 77.70107007248878, 77.40766309975837, 77.06247842595788, 76.75181221953746, 76.30307214359682, 75.83707283396618, 75.54366586123575, 75.2157404211253, 74.81877804625474, 74.43907490507421, 74.05937176389368, 73.71418709009319, 73.43803935105281, 73.1446323783224, 72.73041076976182, 72.22989299275112, 71.86744908526062, 71.5050051777701, 71.09078356920952, 70.62478425957887, 70.17604418363824, 69.79634104245771, 69.31308249913704, 68.98515705902656, 68.58819468415602, 68.2257507766655, 67.79426993441491, 67.46634449430445, 67.08664135312392, 66.60338280980325, 66.18916120124267, 65.72316189161201, 65.25716258198135, 64.9292371418709, 64.48049706593027, 64.11805315843976, 63.73835001725923, 63.37590610976872, 62.927166033828094, 62.56472212633759, 62.2885743872972, 61.87435277873662, 61.51190887124611, 61.06316879530549, 60.54539178460476, 60.16568864342423, 59.76872626855367, 59.37176389368312, 59.026579219882635, 58.66413531239212, 58.23265447014152, 57.87021056265102, 57.49050742147048, 57.05902657921988, 56.67932343803934, 56.33413876423887, 56.02347255781843, 55.60925094925785, 55.24680704176734, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.07594062823611, 89.98964445978598, 89.97238522609597, 89.93786675871591, 89.92060752502589, 89.83431135657577, 89.7134967207456, 89.66171901967553, 89.66171901967553, 89.59268208491544, 89.5063859164653, 89.3683120469451, 89.23023817742491, 89.17846047635484, 89.09216430790472, 88.93683120469451, 88.79875733517432, 88.7124611667242, 88.53986882982396, 88.43631342768381, 88.40179496030376, 88.24646185709354, 88.16016568864342, 88.02209181912323, 87.97031411805315, 87.79772178115292, 87.69416637901277, 87.53883327580255, 87.33172247152227, 87.19364860200207, 86.95201933034173, 86.76216775975146, 86.67587159130134, 86.41698308595099, 86.20987228167068, 85.91646530894027, 85.65757680358992, 85.51950293406972, 85.17431826026925, 84.86365205384881, 84.70831895063858, 84.51846738004832, 84.17328270624785, 83.9316534345875, 83.74180186399724, 83.48291335864687, 83.31032102174663, 83.12046945115637, 82.7925440110459, 82.68898860890576, 82.41284086986538, 82.08491542975491, 81.68795305488436, 81.44632378322402, 81.01484294097342, 80.54884363134276, 80.15188125647221, 79.70314118053157, 79.18536416983085, 78.70210562651017, 78.16706938211944, 77.70107007248878, 77.16603382809804, 76.63099758370728, 75.99240593717639, 75.45736969278563, 74.97411114946496, 74.37003797031412, 73.81774249223334, 73.04107697618225, 72.31618916120124, 71.69485674836037, 71.0735243355195, 70.33137728684846, 69.65826717293751, 68.95063859164652, 68.2084915429755, 67.46634449430445, 66.77597514670349, 65.98205039696236, 65.22264411460131, 64.44597859855021, 63.80738695201933, 62.96168450120814, 62.150500517777004, 61.2875388332758, 60.39005868139454, 59.44080082844321, 58.62961684501208, 57.66309975837072, 56.73110113910942, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 89.97238522609597, 89.90334829133586, 89.7134967207456, 89.57542285122541, 89.43734898170521, 89.31653434587504, 89.14394200897479, 89.02312737314463, 88.91957197100449, 88.79875733517432, 88.574387297204, 88.24646185709354, 88.0048325854332, 87.76320331377286, 87.38350017259233, 87.0037970314118, 86.62409389023126, 86.22713151536071, 85.88194684156024, 85.24335519502934, 84.81187435277874, 84.4321712115982, 84.1387642388678, 83.69002416292717, 83.2930617880566, 82.82706247842594, 82.3610631687953, 81.80876769071452, 81.25647221263375, 80.8595098377632, 80.39351052813255, 79.9275112185019, 79.3579565067311, 78.87469796341043, 78.42595788746979, 77.87366240938903, 77.5802554366586, 77.39040386606834, 76.97618225750776, 76.64825681739731, 76.35484984466689, 76.18225750776665, 75.9406282361063, 75.64722126337591, 75.52640662754573, 75.38833275802554, 75.12944425267519, 74.95685191577493, 74.7842595788747, 74.52537107352433, 74.26648256817397, 73.95581636175353, 73.81774249223334, 73.59337245426303, 73.38626164998274, 73.16189161201243, 72.95478080773213, 72.83396617190196, 72.59233690024163, 72.36796686227132, 72.07455988954091, 71.93648602002071, 71.67759751467034, 71.40144977562996, 71.21159820503969, 71.02174663444943, 70.76285812909906, 70.55574732481877, 70.2105626510183, 69.95167414566794, 69.65826717293751, 69.31308249913704, 69.00241629271659, 68.67449085260614, 68.32930617880567, 67.91508457024507, 67.53538142906454, 67.27649292371419, 67.0693821194339, 66.63790127718329, 66.2927166033828, 65.84397652744218, 65.46427338626164, 64.9292371418709, 64.4287193648602, 63.9282015878495, 63.30686917500863, 62.70279599585778, 62.23679668622712, 61.684501208146344, 61.011391094235414, 60.3555402140145, 59.7859855022437, 59.04383845357266, 58.26717293752157, 57.4387297204004, 56.59302726958922, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.07594062823611, 89.95512599240594, 89.81705212288574, 89.79979288919571, 89.66171901967553, 89.6444597859855, 89.33379357956507, 89.09216430790472, 88.83327580255437, 88.72972040041421, 88.59164653089402, 88.38453572661373, 88.0738695201933, 87.74594408008284, 87.34898170521228, 87.02105626510182, 86.60683465654124, 86.15809458060062, 85.74387297204004, 85.24335519502934, 84.77735588539869, 84.27683810838799, 83.75906109768727, 83.27580255436658, 82.87884017949602, 82.34380393510527, 81.75698998964445, 81.34276838108387, 80.80773213669312, 80.39351052813255, 80.01380738695202, 79.47877114256127, 79.01277183293061, 78.58129099068, 78.14981014842941, 77.73558853986883, 77.2868484639282, 76.9244045564377, 76.45840524680703, 76.09596133931653, 75.68173973075595, 75.40559199171557, 75.18122195374525, 74.90507421470487, 74.54263030721435, 74.42181567138418, 74.26648256817397, 73.99033482913359, 73.80048325854331, 73.55885398688298, 73.29996548153261, 73.0065585088022, 72.76492923714186, 72.52329996548153, 72.21263375906109, 71.91922678633068, 71.60856057991025, 71.38419054193994, 71.17707973765964, 71.0044874007594, 70.76285812909906, 70.41767345529858, 70.27959958577839, 70.08974801518812, 69.79634104245771, 69.5029340697273, 69.17500862961684, 68.96789782533655, 68.72626855367622, 68.55367621677597, 68.32930617880567, 67.98412150500516, 67.69071453227477, 67.38004832585432, 67.01760441836382, 66.72419744563341, 66.44804970659303, 66.05108733172247, 65.72316189161201, 65.36071798412151, 65.10182947877115, 64.67034863652053, 64.27338626164997, 63.84190541939938, 63.44494304452881, 62.96168450120814, 62.616499827407665, 62.16775975146703, 61.61546427338626, 61.183983431135644, 60.648947186744905, 60.148429409734206, 59.61339316534345, 59.00931998619261, 58.387987573351744, 57.68035899206075, 57.07628581290991, 56.265101829478766, 55.488436313427684, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.12771832930618, 90.12771832930618, 90.12771832930618, 90.11045909561615, 90.09319986192614, 90.07594062823611, 89.98964445978598, 89.97238522609597, 89.93786675871591, 89.92060752502589, 89.83431135657577, 89.7134967207456, 89.66171901967553, 89.66171901967553, 89.59268208491544, 89.5063859164653, 89.3683120469451, 89.23023817742491, 89.17846047635484, 89.09216430790472, 88.93683120469451, 88.79875733517432, 88.7124611667242, 88.53986882982396, 88.43631342768381, 88.40179496030376, 88.24646185709354, 88.16016568864342, 88.02209181912323, 87.97031411805315, 87.79772178115292, 87.69416637901277, 87.53883327580255, 87.33172247152227, 87.19364860200207, 86.95201933034173, 86.76216775975146, 86.67587159130134, 86.41698308595099, 86.20987228167068, 85.91646530894027, 85.65757680358992, 85.51950293406972, 85.17431826026925, 84.86365205384881, 84.70831895063858, 84.51846738004832, 84.17328270624785, 83.9316534345875, 83.74180186399724, 83.48291335864687, 83.31032102174663, 83.12046945115637, 82.7925440110459, 82.68898860890576, 82.41284086986538, 82.08491542975491, 81.68795305488436, 81.44632378322402, 81.01484294097342, 80.54884363134276, 80.15188125647221, 79.70314118053157, 79.18536416983085, 78.70210562651017, 78.16706938211944, 77.70107007248878, 77.16603382809804, 76.63099758370728, 75.99240593717639, 75.45736969278563, 74.97411114946496, 74.37003797031412, 73.81774249223334, 73.04107697618225, 72.31618916120124, 71.69485674836037, 71.0735243355195, 70.33137728684846, 69.65826717293751, 68.95063859164652, 68.2084915429755, 67.46634449430445, 66.77597514670349, 65.98205039696236, 65.22264411460131, 64.44597859855021, 63.80738695201933, 62.96168450120814, 62.150500517777004, 61.2875388332758, 60.39005868139454, 59.44080082844321, 58.62961684501208, 57.66309975837072, 56.73110113910942, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2029, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.11045909561615, 90.11045909561615, 90.11045909561615, 90.05868139454608, 90.02416292716603, 89.98964445978598, 89.97238522609597, 89.8688298239558, 89.76527442181566, 89.74801518812563, 89.67897825336554, 89.62720055229548, 89.52364515015533, 89.47186744908525, 89.40283051432516, 89.3683120469451, 89.24749741111494, 89.19571971004487, 89.05764584052469, 88.95409043838454, 88.79875733517432, 88.67794269934414, 88.52260959613393, 88.47083189506385, 88.29823955816362, 88.2119433897135, 88.09112875388332, 87.90127718329306, 87.7114256127028, 87.5043148084225, 87.40075940628236, 87.2109078356921, 87.02105626510182, 86.81394546082154, 86.58957542285121, 86.45150155333103, 86.24439074905074, 86.08905764584051, 85.79565067311012, 85.60579910251984, 85.32965136347946, 85.05350362443907, 84.79461511908872, 84.44943044528823, 84.31135657576803, 84.05246807041767, 83.8626164998274, 83.55195029340697, 83.1377286848464, 82.80980324473592, 82.48187780462547, 82.05039696237488, 81.60165688643424, 81.32550914739384, 80.98032447359337, 80.61788056610285, 80.18639972385225, 79.8584742837418, 79.54780807732136, 79.03003106662064, 78.56403175698998, 78.02899551259924, 77.59751467034863, 77.18329306178805, 76.8553676216776, 76.26855367621677, 75.7852951328961, 75.30203658957542, 74.62892647566449, 74.05937176389368, 73.42078011736278, 72.85122540559199, 72.12633759061097, 71.48774594408009, 70.91819123230928, 70.33137728684846, 69.76182257507766, 69.22678633068692, 68.50189851570589, 67.84604763548498, 67.0693821194339, 66.32723507076285, 65.61960648947186, 64.89471867449085, 64.16983085950983, 63.39316534345875, 62.547462892647566, 61.684501208146344, 60.94235415947531, 60.06213324128409, 59.19917155678287, 58.37072833966172, 57.54228512254056, 56.61028650327925, 55.69554711770797, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2029, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.1449775629962, 90.11045909561615, 90.09319986192614, 90.07594062823611, 90.05868139454608, 90.04142216085606, 89.98964445978598, 89.93786675871591, 89.90334829133586, 89.85157059026578, 89.85157059026578, 89.85157059026578, 89.81705212288574, 89.74801518812563, 89.67897825336554, 89.62720055229548, 89.5063859164653, 89.42008974801519, 89.33379357956507, 89.23023817742491, 89.10942354159475, 88.91957197100449, 88.81601656886434, 88.66068346565412, 88.574387297204, 88.48809112875388, 88.40179496030376, 88.29823955816362, 88.2119433897135, 88.16016568864342, 87.97031411805315, 87.84949948222298, 87.7114256127028, 87.45253710735244, 87.27994477045219, 87.0728339661719, 86.84846392820158, 86.7103900586814, 86.53779772178115, 86.34794615119088, 86.08905764584051, 85.89920607525026, 85.77839143942009, 85.4332067656196, 85.15705902657922, 84.88091128753882, 84.63928201587849, 84.38039351052812, 84.06972730410769, 83.84535726613737, 83.60372799447704, 83.15498791853642, 83.05143251639628, 82.68898860890576, 82.37832240248532, 82.136693130825, 81.75698998964445, 81.48084225060407, 81.17017604418363, 80.63513979979288, 80.13462202278218, 79.7894373489817, 79.39247497411115, 78.73662409389023, 78.27062478425958, 77.90818087676908, 77.44218156713842, 77.06247842595788, 76.47566448049706, 75.80255436658612, 75.14670348636521, 74.62892647566449, 74.0766309975837, 73.48981705212289, 72.85122540559199, 72.04004142216085, 71.38419054193994, 70.71108042802899, 70.10700724887815, 69.38211943389715, 68.7090093199862, 68.12219537452538, 67.3627890921643, 66.74145667932343, 65.99930963065239, 65.22264411460131, 64.4287193648602, 63.51397997928891, 62.68553676216775, 61.7880566102865, 61.011391094235414, 60.13117017604418, 59.38902312737314, 58.56057991025199, 57.7839143942009, 57.00724887814981, 56.24784259578875, 55.45391784604763, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 89.97238522609597, 89.83431135657577, 89.66171901967553, 89.38557128063513, 88.95409043838454, 88.52260959613393, 88.12564722126336, 87.90127718329306, 87.60787021056264, 87.36624093890231, 87.05557473248187, 86.93476009665171, 86.72764929237141, 86.52053848809112, 86.22713151536071, 85.83016914049016, 85.41594753192958, 85.05350362443907, 84.69105971694857, 84.36313427683811, 83.91439420089748, 83.51743182602692, 83.1377286848464, 82.87884017949602, 82.58543320676561, 82.20573006558509, 81.9295823265447, 81.58439765274422, 81.30824991370382, 80.91128753883328, 80.68691750086295, 80.34173282706247, 79.996548153262, 79.54780807732136, 79.288919571971, 79.04729030031066, 78.73662409389023, 78.47773558853987, 78.09803244735933, 77.70107007248878, 77.39040386606834, 77.11425612702796, 76.7863306869175, 76.47566448049706, 76.13047980669658, 75.90610976872627, 75.59544356230583, 75.2157404211253, 74.87055574732481, 74.55988954090438, 74.19744563341388, 73.93855712806351, 73.55885398688298, 73.24818778046254, 72.79944770452191, 72.36796686227132, 72.04004142216085, 71.78115291681048, 71.55678287884018, 71.28063513979978, 71.0044874007594, 70.53848809112876, 70.17604418363824, 69.81360027614774, 69.45115636865722, 69.03693476009664, 68.74352778736623, 68.32930617880567, 67.84604763548498, 67.50086296168449, 67.03486365205384, 66.55160510873317, 65.98205039696236, 65.55056955471177, 65.18812564722126, 64.7911632723507, 64.37694166379013, 63.9282015878495, 63.548498446668965, 63.047980669658266, 62.547462892647566, 62.06420434932689, 61.66724197445633, 61.183983431135644, 60.75250258888505, 60.33828098032448, 59.88954090438384, 59.42354159475318, 58.90576458405247, 58.526061442871935, 58.180876769071446, 57.818432861580945, 57.525025888850536, 57.04176734552986, 56.66206420434931, 56.28236106316879, 55.816361753538146, 55.350362443907485, 55.125992405937176, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.11045909561615, 90.11045909561615, 90.07594062823611, 90.07594062823611, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.95512599240594, 89.8688298239558, 89.81705212288574, 89.74801518812563, 89.67897825336554, 89.6444597859855, 89.55816361753539, 89.45460821539524, 89.40283051432516, 89.33379357956507, 89.2129789437349, 89.12668277528478, 89.05764584052469, 88.88505350362443, 88.83327580255437, 88.76423886779426, 88.60890576458405, 88.45357266137384, 88.35001725923368, 88.19468415602347, 88.0738695201933, 87.97031411805315, 87.84949948222298, 87.60787021056264, 87.38350017259233, 87.2109078356921, 86.91750086296169, 86.69313082499137, 86.5032792544011, 86.19261304798067, 86.0717984121505, 85.95098377632033, 85.67483603727995, 85.38142906454954, 85.13979979288919, 84.88091128753882, 84.69105971694857, 84.46668967897824, 84.24231964100794, 83.82809803244736, 83.58646876078701, 83.32758025543666, 83.20676561960649, 82.87884017949602, 82.60269244045564, 82.30928546772522, 81.94684156023472, 81.68795305488436, 81.34276838108387, 80.8595098377632, 80.47980669658267, 79.996548153262, 79.59958577839143, 79.16810493614084, 78.75388332758025, 78.2878840179496, 77.89092164307905, 77.47670003451846, 76.99344149119779, 76.45840524680703, 75.90610976872627, 75.38833275802554, 74.85329651363479, 74.19744563341388, 73.71418709009319, 73.02381774249223, 72.54055919917155, 71.85018985157059, 71.33241284086986, 70.55574732481877, 69.84811874352779, 69.2095270969969, 68.45012081463582, 67.863306869175, 67.17293752157404, 66.53434587504314, 65.8612357611322, 65.08457024508112, 64.35968243010011, 63.73835001725923, 62.87538833275802, 61.97790818087676, 61.149464963755605, 60.2865032792544, 59.458060062133235, 58.50880220918191, 57.59406282361063, 56.66206420434931, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 89.93786675871591, 89.74801518812563, 89.59268208491544, 89.57542285122541, 89.47186744908525, 89.31653434587504, 89.16120124266482, 88.91957197100449, 88.69520193303417, 88.47083189506385, 88.29823955816362, 88.03935105281326, 87.74594408008284, 87.64238867794269, 87.45253710735244, 87.24542630307214, 86.93476009665171, 86.62409389023126, 86.26164998274076, 85.91646530894027, 85.67483603727995, 85.26061442871936, 84.79461511908872, 84.48394891266827, 84.08698653779771, 83.70728339661719, 83.22402485329651, 82.84432171211597, 82.4991370383155, 82.15395236451502, 81.75698998964445, 81.34276838108387, 80.9285467725233, 80.65239903348291, 80.15188125647221, 79.8584742837418, 79.4960303762513, 79.11632723507076, 78.80566102865032, 78.54677252329995, 78.33966171901967, 78.09803244735933, 77.856403175699, 77.59751467034863, 77.37314463237833, 77.20055229547808, 77.0797376596479, 76.8553676216776, 76.54470141525715, 76.30307214359682, 75.99240593717639, 75.66448049706594, 75.4401104590956, 75.16396272005524, 74.92233344839488, 74.69796341042458, 74.40455643769417, 74.19744563341388, 73.83500172592336, 73.62789092164309, 73.26544701415256, 73.04107697618225, 72.76492923714186, 72.48878149810149, 72.17811529168104, 71.93648602002071, 71.71211598205039, 71.33241284086986, 71.03900586813945, 70.90093199861926, 70.60752502588885, 70.26234035208836, 69.96893337935795, 69.62374870555747, 69.26130479806696, 68.91612012426648, 68.51915774939592, 68.08767690714532, 67.82878840179495, 67.3627890921643, 67.01760441836382, 66.63790127718329, 66.17190196755264, 65.70590265792198, 65.23990334829134, 64.84294097342078, 64.29064549534, 63.7210907835692, 63.01346220227821, 62.37487055574732, 61.66724197445633, 60.89057645840524, 60.148429409734206, 59.31998619261305, 58.49154297549188, 57.59406282361063, 56.61028650327925, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.09319986192614, 89.98964445978598, 89.79979288919571, 89.67897825336554, 89.54090438384536, 89.48912668277528, 89.33379357956507, 89.10942354159475, 88.83327580255437, 88.72972040041421, 88.38453572661373, 88.24646185709354, 87.84949948222298, 87.62512944425266, 87.19364860200207, 86.93476009665171, 86.65861235761132, 86.34794615119088, 86.05453917846047, 85.58853986882981, 85.39868829823956, 85.07076285812909, 84.60476354849844, 84.1387642388678, 83.70728339661719, 83.2930617880566, 82.98239558163617, 82.46461857093544, 82.06765619606489, 81.68795305488436, 81.18743527787366, 80.77321366931308, 80.2899551259924, 79.82395581636175, 79.40973420780118, 79.11632723507076, 78.66758715913014, 78.33966171901967, 78.13255091473938, 77.856403175699, 77.66655160510874, 77.51121850189851, 77.25232999654816, 77.02795995857782, 76.82084915429755, 76.70003451846738, 76.49292371418709, 76.23403520883673, 76.06144287193648, 75.7852951328961, 75.62996202968588, 75.40559199171557, 75.19848118743528, 75.0776665516051, 74.87055574732481, 74.59440800828443, 74.40455643769417, 74.2147048671039, 73.85226095961339, 73.52433551950293, 73.24818778046254, 72.95478080773213, 72.6441146013117, 72.29892992751121, 72.04004142216085, 71.83293061788056, 71.57404211253021, 71.38419054193994, 71.1425612702796, 70.86641353123922, 70.65930272695891, 70.40041422160856, 70.12426648256817, 69.81360027614774, 69.39937866758716, 69.05419399378667, 68.82982395581637, 68.41560234725577, 68.08767690714532, 67.7252329996548, 67.27649292371419, 66.8622713151536, 66.34449430445288, 66.01656886434242, 65.51605108733172, 65.03279254401104, 64.4287193648602, 63.96272005522954, 63.4276838108388, 62.80635139799793, 62.133241284086985, 61.52916810493614, 60.75250258888505, 60.06213324128409, 59.250949257852945, 58.45702450811183, 57.57680358992061, 56.61028650327925, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.1449775629962, 90.1449775629962, 90.1449775629962, 90.11045909561615, 90.11045909561615, 90.07594062823611, 90.07594062823611, 90.02416292716603, 90.02416292716603, 89.98964445978598, 89.95512599240594, 89.8688298239558, 89.81705212288574, 89.74801518812563, 89.67897825336554, 89.6444597859855, 89.55816361753539, 89.45460821539524, 89.40283051432516, 89.33379357956507, 89.2129789437349, 89.12668277528478, 89.05764584052469, 88.88505350362443, 88.83327580255437, 88.76423886779426, 88.60890576458405, 88.45357266137384, 88.35001725923368, 88.19468415602347, 88.0738695201933, 87.97031411805315, 87.84949948222298, 87.60787021056264, 87.38350017259233, 87.2109078356921, 86.91750086296169, 86.69313082499137, 86.5032792544011, 86.19261304798067, 86.0717984121505, 85.95098377632033, 85.67483603727995, 85.38142906454954, 85.13979979288919, 84.88091128753882, 84.69105971694857, 84.46668967897824, 84.24231964100794, 83.82809803244736, 83.58646876078701, 83.32758025543666, 83.20676561960649, 82.87884017949602, 82.60269244045564, 82.30928546772522, 81.94684156023472, 81.68795305488436, 81.34276838108387, 80.8595098377632, 80.47980669658267, 79.996548153262, 79.59958577839143, 79.16810493614084, 78.75388332758025, 78.2878840179496, 77.89092164307905, 77.47670003451846, 76.99344149119779, 76.45840524680703, 75.90610976872627, 75.38833275802554, 74.85329651363479, 74.19744563341388, 73.71418709009319, 73.02381774249223, 72.54055919917155, 71.85018985157059, 71.33241284086986, 70.55574732481877, 69.84811874352779, 69.2095270969969, 68.45012081463582, 67.863306869175, 67.17293752157404, 66.53434587504314, 65.8612357611322, 65.08457024508112, 64.35968243010011, 63.73835001725923, 62.87538833275802, 61.97790818087676, 61.149464963755605, 60.2865032792544, 59.458060062133235, 58.50880220918191, 57.59406282361063, 56.66206420434931, 55.74732481877804, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2030, "method": "three", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.16223679668623, 90.11045909561615, 90.07594062823611, 90.05868139454608, 90.02416292716603, 90.006903693476, 89.98964445978598, 89.97238522609597, 89.92060752502589, 89.79979288919571, 89.74801518812563, 89.7134967207456, 89.66171901967553, 89.62720055229548, 89.5063859164653, 89.47186744908525, 89.3683120469451, 89.24749741111494, 89.12668277528478, 89.05764584052469, 88.90231273731446, 88.81601656886434, 88.74697963410424, 88.59164653089402, 88.40179496030376, 88.31549879185364, 88.12564722126336, 87.98757335174318, 87.81498101484294, 87.7114256127028, 87.53883327580255, 87.40075940628236, 87.27994477045219, 87.0728339661719, 86.84846392820158, 86.62409389023126, 86.39972385226096, 86.20987228167068, 85.95098377632033, 85.70935450465998, 85.41594753192958, 85.26061442871936, 85.08802209181911, 84.86365205384881, 84.60476354849844, 84.41491197790818, 84.27683810838799, 84.03520883672765, 83.72454263030721, 83.5001725923369, 83.22402485329651, 82.89609941318605, 82.48187780462547, 82.10217466344494, 81.80876769071452, 81.44632378322402, 81.1356575768036, 80.70417673455297, 80.37625129444253, 80.04832585433206, 79.65136347946151, 79.288919571971, 78.89195719710045, 78.40869865377978, 77.94269934414912, 77.49395926820849, 76.8553676216776, 76.32033137728685, 75.66448049706594, 75.0776665516051, 74.40455643769417, 73.83500172592336, 73.28270624784258, 72.6441146013117, 71.97100448740076, 71.31515360717984, 70.67656196064894, 70.02071108042801, 69.29582326544701, 68.50189851570589, 67.75975146703486, 67.15567828788402, 66.3617535381429, 65.5678287884018, 64.82568173973075, 64.11805315843976, 63.39316534345875, 62.616499827407665, 61.822575077666556, 60.99413186054539, 60.13117017604418, 59.181912323092845, 58.33620987228167, 57.49050742147048, 56.66206420434931, 55.73006558508802, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2030, "method": "gain", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.04142216085606, 89.97238522609597, 89.98964445978598, 89.98964445978598, 89.95512599240594, 89.92060752502589, 89.90334829133586, 89.8688298239558, 89.81705212288574, 89.74801518812563, 89.67897825336554, 89.62720055229548, 89.59268208491544, 89.57542285122541, 89.43734898170521, 89.3683120469451, 89.26475664480496, 89.14394200897479, 89.09216430790472, 89.0058681394546, 88.98860890576458, 88.90231273731446, 88.79875733517432, 88.69520193303417, 88.574387297204, 88.47083189506385, 88.33275802554367, 88.24646185709354, 88.17742492233344, 88.09112875388332, 87.98757335174318, 87.81498101484294, 87.7114256127028, 87.59061097687263, 87.43527787366241, 87.26268553676216, 87.03831549879185, 86.91750086296169, 86.7103900586814, 86.5723161891612, 86.39972385226096, 86.24439074905074, 86.0717984121505, 85.89920607525026, 85.58853986882981, 85.34691059716948, 85.10528132550914, 84.89817052122885, 84.70831895063858, 84.34587504314808, 84.03520883672765, 83.77632033137729, 83.37935795650672, 83.08595098377631, 82.72350707628581, 82.37832240248532, 82.10217466344494, 81.54987918536416, 81.17017604418363, 80.80773213669312, 80.46254746289264, 79.97928891957197, 79.77217811529168, 79.34069727304107, 78.89195719710045, 78.44321712115982, 77.856403175699, 77.33862616499827, 76.80358992060752, 76.33759061097687, 75.73351743182603, 75.14670348636521, 74.52537107352433, 74.0075940628236, 73.40352088367277, 72.79944770452191, 72.19537452537107, 71.45322747670004, 70.74559889540903, 70.10700724887815, 69.31308249913704, 68.67449085260614, 67.863306869175, 67.0693821194339, 66.3617535381429, 65.61960648947186, 64.9292371418709, 64.10079392474974, 63.2896099413186, 62.51294442526751, 61.80531584397653, 61.02865032792544, 60.26924404556438, 59.50983776320331, 58.802209181912325, 57.92198826372108, 57.14532274767, 56.351397997928885, 55.57473248187781, 54.81532619951675]}, {"task": "cub", "expert": "glsim", "endpoint": "Qwen3.8", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9837763203313773, 1.9848118743527785, 2.98584742837418, 3.9868829823955814, 4.987918536416983, 5.9889540904383844, 6.989989644459786, 7.9910251984811875, 8.992060752502589, 9.993096306523992, 10.994131860545393, 11.995167414566794, 12.996202968588195, 13.997238522609596, 14.998274076630997, 15.9993096306524, 16.983085950983774, 17.984121505005177, 18.98515705902658, 19.986192613047983, 20.987228167069382, 21.988263721090785, 22.989299275112185, 23.990334829133587, 24.991370383154987, 25.99240593717639, 26.99344149119779, 27.994477045219192, 28.99551259924059, 29.996548153261994, 30.9975837072834, 31.9986192613048, 32.9996548153262, 33.983431135657575, 34.984466689678975, 35.98550224370038, 36.98653779772178, 37.98757335174318, 38.98860890576458, 39.989644459785985, 40.990680013807385, 41.991715567828784, 42.99275112185019, 43.99378667587159, 44.994822229892996, 45.995857783914396, 46.996893337935795, 47.9979288919572, 48.9989644459786, 50.0, 50.98377632033137, 51.98481187435278, 52.98584742837418, 53.98688298239558, 54.98791853641698, 55.988954090438384, 56.98998964445978, 57.99102519848118, 58.99206075250258, 59.99309630652399, 60.99413186054539, 61.9951674145668, 62.9962029685882, 63.9972385226096, 64.998274076631, 65.9993096306524, 66.98308595098378, 67.98412150500518, 68.98515705902658, 69.98619261304798, 70.98722816706939, 71.98826372109079, 72.98929927511219, 73.99033482913359, 74.99137038315499, 75.99240593717639, 76.99344149119779, 77.99447704521918, 78.9955125992406, 79.996548153262, 80.9975837072834, 81.9986192613048, 82.9996548153262, 83.98343113565757, 84.98446668967898, 85.98550224370038, 86.98653779772178, 87.98757335174318, 88.9886089057646, 89.98964445978599, 90.99068001380739, 91.99171556782879, 92.99275112185019, 93.99378667587159, 94.99482222989299, 95.9958577839144, 96.9968933379358, 97.9979288919572, 98.9989644459786, 100.0], "score": [90.16223679668623, 90.04142216085606, 89.92060752502589, 89.78253365550569, 89.59268208491544, 89.43734898170521, 89.17846047635484, 88.72972040041421, 88.43631342768381, 88.16016568864342, 87.81498101484294, 87.60787021056264, 87.24542630307214, 87.0728339661719, 86.74490852606144, 86.5032792544011, 86.14083534691059, 85.69209527096997, 85.32965136347946, 85.001725923369, 84.74283741801864, 84.57024508111839, 84.20780117362789, 83.82809803244736, 83.6382464618571, 83.32758025543666, 83.08595098377631, 82.84432171211597, 82.4991370383155, 82.20573006558509, 81.77424922333448, 81.44632378322402, 81.11839834311357, 80.8595098377632, 80.53158439765275, 80.23817742492233, 79.91025198481188, 79.56506731101139, 79.2198826372109, 78.94373489817052, 78.56403175698998, 78.20158784949948, 77.89092164307905, 77.59751467034863, 77.13151536071798, 76.82084915429755, 76.54470141525715, 76.19951674145668, 75.88885053503624, 75.52640662754573, 75.16396272005524, 74.81877804625474, 74.5081118398343, 74.09389023127373, 73.74870555747324, 73.40352088367277, 73.12737314463237, 72.66137383500173, 72.33344839489126, 71.90196755264066, 71.38419054193994, 70.98722816706939, 70.52122885743873, 70.02071108042801, 69.67552640662754, 69.26130479806696, 68.8643424231964, 68.39834311356576, 68.08767690714532, 67.67345529858474, 67.31101139109424, 66.94856748360372, 66.58612357611322, 66.17190196755264, 65.77493959268207, 65.2916810493614, 64.77390403866067, 64.27338626164997, 63.824646185709355, 63.46220227821884, 63.01346220227821, 62.65101829478771, 62.23679668622712, 61.839834311356576, 61.44287193648601, 61.11494649637555, 60.717984121505, 60.3555402140145, 60.027614773904034, 59.63065239903348, 59.19917155678287, 58.681394546082146, 58.301691404901625, 57.818432861580945, 57.55954435623059, 57.16258198136002, 56.67932343803934, 56.265101829478766, 55.69554711770797, 55.350362443907485, 54.81532619951675]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.521765022828465, 28.276096758642204, 29.027592660862663, 29.77908856308312, 30.55043530045657, 31.26506536596433, 31.931486260386244, 32.59790715480816, 33.24731304767036, 33.84567393585345, 34.4865723279358, 35.087769049712165, 35.686129937895245, 36.20225165187307, 36.77509003771659, 37.25434591498171, 37.78748263051924, 38.26390267419108, 38.76300938660919, 39.27345943340045, 39.769730312225285, 40.23480702152398, 40.68854039644953, 41.13093043700196, 41.527947140061826, 41.964665513427676, 42.39571221960695, 42.792728922666825, 43.11601395230128, 43.47616481864844, 43.85049485296203, 44.21631738649576, 44.57930408643621, 44.899753282477384, 45.20885914414542, 45.48960666988061, 45.76751836202252, 46.04826588775771, 46.3091625783399, 46.56155176814225, 46.85080679465729, 47.10603181805292, 47.36692850863511, 47.62782519921731, 47.866035221053224, 48.09857357570258, 48.36514193347134, 48.643053625613256, 48.86424864588946, 49.079771998979105, 49.30096701925532, 49.46828120125911, 49.59589371295693, 49.75753622777416, 49.93902957774438, 50.09500042537505, 50.219777103479565, 50.33037461361768, 50.452315458128915, 50.61112213935286, 50.74157048464396, 50.87201882993506, 50.97978050647989, 51.0676913478717, 51.11873635255083, 51.172617190823246, 51.22933386268894, 51.27754303377478, 51.33425970564047, 51.38814054391288, 51.44485721577858, 51.46470805093158, 51.50440972123755, 51.50440972123755, 51.52142472279727, 51.52142472279727, 51.53276805717041, 51.52426055639056, 51.510081388424126, 51.510081388424126, 51.51858888920399, 51.4959022204577, 51.47321555171143, 51.433513881405446, 51.42784221421888, 51.41082721265916, 51.40799137906588, 51.40231971187931, 51.40231971187931, 51.39381221109945, 51.376797209539745, 51.34560304001361, 51.32575220486062, 51.2747072001815, 51.18963219238295, 51.113064685364265, 50.98261634007317, 50.74724215183053, 50.517539630774465, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.433854181436633, 28.054901738365995, 28.701471797634916, 29.336698522530703, 29.901029407594365, 30.493718628590877, 31.012676176161985, 31.52879789013981, 32.03641210333778, 32.521339647789475, 33.00626719224116, 33.428806397640585, 33.86836060459972, 34.285228142812585, 34.727618183365, 35.19553072625698, 35.720159941014664, 36.15404248078723, 36.57374585259337, 36.97927005643309, 37.413152596205656, 37.841363468791656, 38.28375350934407, 38.68360604599722, 39.0239060771914, 39.42092278025125, 39.80659614893798, 40.21495618637098, 40.57510705271815, 41.00048209171086, 41.41734962992372, 41.78317216345745, 42.15183053058447, 42.52048889771148, 42.85795309531237, 43.203924793693105, 43.527209823327574, 43.83631568499561, 44.15109321385021, 44.49990074582424, 44.783484105152716, 45.095425800414034, 45.38468082692908, 45.665428352664264, 45.99722088307858, 46.34035674786604, 46.595581771261664, 46.785582622011745, 47.012449309474526, 47.327226838329125, 47.56543686016504, 47.80931854918754, 48.0021552335309, 48.27723109207952, 48.48424694438931, 48.72529279981851, 48.89544281541559, 49.07693616538582, 49.27260868332246, 49.43992286532627, 49.59872954655021, 49.81141706604657, 49.93052207696452, 50.05246292147577, 50.22261293707285, 50.33604628080425, 50.50336046280805, 50.619629640132715, 50.73589881745739, 50.81246632447608, 50.877690497121634, 50.97978050647989, 51.042168845532146, 51.107393018177696, 51.16694552363668, 51.246348864248645, 51.311573036894195, 51.33993137282704, 51.37963304313303, 51.382468876726314, 51.447693049371864, 51.4420213821853, 51.46470805093158, 51.49873805405099, 51.481723052491276, 51.419334713439014, 51.382468876726314, 51.354110540793464, 51.33993137282704, 51.28605053455464, 51.181124691603095, 51.121572186144114, 51.06201968068513, 50.940078836173896, 50.823809658849214, 50.70186881433798, 50.582763803420015, 50.452315458128915, 50.25947877378556, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.52460085642175, 28.244902589116073, 28.999234324929812, 29.74789439355699, 30.49655446218416, 31.23103536284491, 31.90029209086011, 32.501488812636474, 33.17074554065167, 33.77194226242805, 34.34194481467828, 34.9516490372345, 35.52165158948473, 36.02642996908941, 36.54538751666052, 37.05867339704506, 37.56628761024303, 38.116439327340274, 38.55032186711284, 39.043756912344385, 39.4492811161841, 39.92002949266937, 40.38794203556135, 40.861526245639915, 41.303916286192326, 41.666902986132776, 42.05541218841279, 42.45242889147265, 42.792728922666825, 43.18974562572669, 43.49885148739472, 43.87885318889488, 44.170944049003204, 44.48572157785781, 44.84020077701841, 45.126619969940165, 45.42438249723506, 45.685279187817265, 46.00572838385843, 46.24393840569436, 46.54453676658254, 46.760060119672175, 47.06349431415365, 47.26483849927686, 47.54842185860534, 47.7781243796614, 48.01633440149732, 48.251708589739955, 48.43320193971018, 48.614695289680405, 48.83021864277004, 49.07693616538582, 49.23290701301648, 49.43141536454641, 49.58738621217707, 49.76037206136744, 49.89649207384511, 50.024104585542915, 50.16873209880044, 50.282165442531834, 50.42112128860279, 50.534554632334185, 50.63664464169243, 50.772764654170096, 50.874854663528346, 51.01948217678587, 51.09037801661799, 51.15560218926354, 51.20664719394266, 51.243513030655365, 51.317244704080764, 51.342767206420326, 51.373961375946465, 51.39948387828602, 51.43634971499873, 51.419334713439014, 51.45620055015172, 51.49306638686441, 51.484558886084564, 51.46187221733829, 51.46754388452486, 51.47321555171143, 51.459036383745, 51.447693049371864, 51.4420213821853, 51.42784221421888, 51.37112554235318, 51.3285880384539, 51.308737203300915, 51.27754303377478, 51.20097552675609, 51.11022885177098, 51.04500467912543, 50.95142217054703, 50.837988826815646, 50.73873465105068, 50.58843547060658, 50.44097212375578, 50.27932960893855, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.521765022828465, 28.276096758642204, 29.027592660862663, 29.77908856308312, 30.55043530045657, 31.26506536596433, 31.931486260386244, 32.59790715480816, 33.24731304767036, 33.84567393585345, 34.4865723279358, 35.087769049712165, 35.686129937895245, 36.20225165187307, 36.77509003771659, 37.25434591498171, 37.78748263051924, 38.26390267419108, 38.76300938660919, 39.27345943340045, 39.769730312225285, 40.23480702152398, 40.68854039644953, 41.13093043700196, 41.527947140061826, 41.964665513427676, 42.39571221960695, 42.792728922666825, 43.11601395230128, 43.47616481864844, 43.85049485296203, 44.21631738649576, 44.57930408643621, 44.899753282477384, 45.20885914414542, 45.48960666988061, 45.76751836202252, 46.04826588775771, 46.3091625783399, 46.56155176814225, 46.85080679465729, 47.10603181805292, 47.36692850863511, 47.62782519921731, 47.866035221053224, 48.09857357570258, 48.36514193347134, 48.643053625613256, 48.86424864588946, 49.079771998979105, 49.30096701925532, 49.46828120125911, 49.59589371295693, 49.75753622777416, 49.93902957774438, 50.09500042537505, 50.219777103479565, 50.33037461361768, 50.452315458128915, 50.61112213935286, 50.74157048464396, 50.87201882993506, 50.97978050647989, 51.0676913478717, 51.11873635255083, 51.172617190823246, 51.22933386268894, 51.27754303377478, 51.33425970564047, 51.38814054391288, 51.44485721577858, 51.46470805093158, 51.50440972123755, 51.50440972123755, 51.52142472279727, 51.52142472279727, 51.53276805717041, 51.52426055639056, 51.510081388424126, 51.510081388424126, 51.51858888920399, 51.4959022204577, 51.47321555171143, 51.433513881405446, 51.42784221421888, 51.41082721265916, 51.40799137906588, 51.40231971187931, 51.40231971187931, 51.39381221109945, 51.376797209539745, 51.34560304001361, 51.32575220486062, 51.2747072001815, 51.18963219238295, 51.113064685364265, 50.98261634007317, 50.74724215183053, 50.517539630774465, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2026, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.533108357201602, 28.281768425828773, 28.976547656183538, 29.685506054504724, 30.408643620792336, 31.15163202223294, 31.866262087740694, 32.527011314976036, 33.16223803987182, 33.8343306014803, 34.446870657629816, 34.988514873947196, 35.61239826446984, 36.19941581827978, 36.80061254005616, 37.31673425403398, 37.813005132858805, 38.29226101012393, 38.74032271786292, 39.245101097467604, 39.73002864191929, 40.16958284887843, 40.69988373082268, 41.13093043700196, 41.53645464084168, 41.92779967671497, 42.35601054930097, 42.67645974534214, 43.05362561324901, 43.464821484275305, 43.81362901624933, 44.159600714630066, 44.497064912230954, 44.82034994186541, 45.16915747383944, 45.48677083628733, 45.756175027649384, 46.04826588775771, 46.34319258145933, 46.57573093610867, 46.856478461843864, 47.13439015398577, 47.40095851175453, 47.65618353515016, 47.866035221053224, 48.050364404616744, 48.288574426452655, 48.45305277486317, 48.66574029435953, 48.86424864588946, 49.05141366304625, 49.2102203442702, 49.39454952783371, 49.578878711397216, 49.720670391061454, 49.904999574624966, 50.080821257408616, 50.219777103479565, 50.35873294955052, 50.46365879250205, 50.5685846354536, 50.679182145591696, 50.74724215183053, 50.82097382525593, 50.91455633383433, 50.940078836173896, 51.005303008819446, 51.050676346311995, 51.107393018177696, 51.15276635567025, 51.16694552363668, 51.21515469472251, 51.26052803221507, 51.311573036894195, 51.28888636814792, 51.28888636814792, 51.28888636814792, 51.29739386892778, 51.28605053455464, 51.27754303377478, 51.25769219862178, 51.22933386268894, 51.21515469472251, 51.20948302753594, 51.2179905283158, 51.20664719394266, 51.20381136034937, 51.19530385956952, 51.186796358789664, 51.20097552675609, 51.17545302441653, 51.14709468848368, 51.135751354110546, 51.06485551427842, 50.937243002580615, 50.880526330714915, 50.76142131979695, 50.62530130731929, 50.43246462297593, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2026, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.493406686895618, 28.298783427388486, 29.07863766554179, 29.81028273260925, 30.575957802796133, 31.222527862065057, 31.908799591639962, 32.53835464934917, 33.159402206278536, 33.76910642883476, 34.34761648186485, 34.9317982020815, 35.55851742619743, 36.15687831438051, 36.67300002835834, 37.19479340952273, 37.68822845475428, 38.16181266483283, 38.65524771006438, 39.12883192014293, 39.61092363100134, 40.0816720074866, 40.54958455037859, 40.94943708703174, 41.394662961177445, 41.80018716501716, 42.16600969855089, 42.52048889771148, 42.88631143124522, 43.23511896321924, 43.58392649519327, 43.966764030286704, 44.332586563820435, 44.61616992314891, 44.896917448884096, 45.1379633043133, 45.46692000113433, 45.7703541956158, 46.07662422369055, 46.39707341973173, 46.63528344156765, 46.90185179933641, 47.14289765476562, 47.34140600629555, 47.616481864844175, 47.82916938434053, 48.050364404616744, 48.2063352522474, 48.39066443581091, 48.59484445452741, 48.8387261435499, 48.980517823214136, 49.1364886708448, 49.34350452315458, 49.482460369225535, 49.59305787936364, 49.714998723874885, 49.86813373791226, 49.975895414457085, 50.029776252729484, 50.1403737628676, 50.22828460425942, 50.32753878002438, 50.4381362901625, 50.5487338003006, 50.61112213935286, 50.704704647931266, 50.775600487763384, 50.815302158069365, 50.88903383149477, 50.95425800414032, 51.02515384397244, 51.042168845532146, 51.050676346311995, 51.08187051583813, 51.06485551427842, 51.11022885177098, 51.1442588548904, 51.15843802285682, 51.15560218926354, 51.18963219238295, 51.18396052519638, 51.18396052519638, 51.16978135722996, 51.1442588548904, 51.15276635567025, 51.16127385645011, 51.141423021297115, 51.135751354110546, 51.113064685364265, 51.121572186144114, 51.09037801661799, 51.042168845532146, 51.03649717834558, 50.9570938377336, 50.85783966196864, 50.74157048464396, 50.579927969826734, 50.409777954229654, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.530272523608318, 28.298783427388486, 29.07863766554179, 29.83296940135553, 30.53625613249015, 31.28491620111732, 31.979695431472084, 32.631937157927574, 33.156566372685255, 33.74641976008848, 34.32776564671185, 34.86373819584267, 35.34299407310779, 35.912996625358026, 36.412103337776145, 36.90837421660097, 37.410316762612375, 37.847035135978224, 38.28375350934407, 38.780024388168904, 39.205399427161616, 39.66196863568046, 40.0816720074866, 40.51271871366588, 40.89272041516604, 41.3095879533789, 41.69809715565891, 42.063919689192645, 42.36168221648754, 42.69063891330857, 43.01675977653632, 43.34004480617077, 43.63780733346567, 43.932734027167285, 44.190794884156205, 44.48004991067124, 44.8033349403057, 45.061395797294615, 45.30527748631711, 45.54065167455974, 45.84408586904121, 46.09931089243683, 46.320505912713045, 46.558715934548964, 46.79976178997817, 47.00961347588124, 47.25065933131044, 47.486033519553075, 47.69588520545614, 47.84051271871367, 48.0021552335309, 48.163797748348124, 48.30842526160565, 48.50126194594901, 48.69977029747894, 48.827382809176754, 48.96066698806114, 49.074100331792536, 49.213056177863486, 49.35484785752772, 49.482460369225535, 49.618580381703204, 49.73768539262117, 49.788730397300284, 49.916342908998104, 50.021268751949634, 50.10067209256162, 50.236792105039285, 50.316195445651246, 50.35589711595724, 50.43246462297593, 50.50619629640133, 50.534554632334185, 50.56574880186031, 50.61679380653943, 50.68201797918499, 50.710376315117834, 50.66783881121856, 50.6593313104387, 50.60545047216629, 50.534554632334185, 50.49201712843491, 50.446643790942346, 50.42395712219607, 50.38992711907666, 50.39276295266995, 50.381419618296796, 50.384255451890084, 50.4182854550095, 50.452315458128915, 50.50619629640133, 50.5487338003006, 50.59410713779315, 50.60828630575958, 50.62813714091258, 50.613957972946146, 50.574256302640165, 50.52604713155432, 50.3445537815841, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.527436690015033, 28.33848509769447, 29.106996001474634, 29.88117857244137, 30.661032810594673, 31.358647874542722, 32.04775543771092, 32.65462382667385, 33.23596971329722, 33.797464764767604, 34.40716898732383, 34.994186541133764, 35.5273232566713, 36.03777330346256, 36.61344752289936, 37.21748007826901, 37.719422624280405, 38.207186002325386, 38.63539687491138, 39.106145251396654, 39.608087797408054, 40.041970337180615, 40.48436037773303, 40.9380937526586, 41.346453790091594, 41.78033632986416, 42.1830247001106, 42.56869806879733, 42.971386439043755, 43.32302980461107, 43.7257181748575, 44.07452570683152, 44.40631823724584, 44.72109576610045, 45.1209483027536, 45.47542750191419, 45.77886169639566, 46.08796755806369, 46.40841675410487, 46.714686782179626, 46.99543430791481, 47.262002665683575, 47.52857102345235, 47.752601877321844, 48.024841902277174, 48.23469358818024, 48.46156027564303, 48.66574029435953, 48.86708447948275, 49.06559283101268, 49.20171284349035, 49.40022119502028, 49.63843121685619, 49.80574539886, 49.950372912117516, 50.07514959022205, 50.20559793551315, 50.33604628080425, 50.47216629328191, 50.647987976065565, 50.758585486203664, 50.87201882993506, 50.9967955080396, 51.06485551427842, 51.1442588548904, 51.19246802597623, 51.19530385956952, 51.28605053455464, 51.373961375946465, 51.40799137906588, 51.433513881405446, 51.478887218897995, 51.46754388452486, 51.48739471967785, 51.50440972123755, 51.52142472279727, 51.50157388764427, 51.50157388764427, 51.50724555483084, 51.5157530556107, 51.49873805405099, 51.48739471967785, 51.47321555171143, 51.476051385304714, 51.46754388452486, 51.46470805093158, 51.45052888296515, 51.459036383745, 51.4420213821853, 51.43634971499873, 51.38814054391288, 51.37963304313303, 51.376797209539745, 51.3285880384539, 51.26903553299492, 51.16978135722996, 51.056348013498564, 50.86067549556192, 50.50619629640133, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.467884184556052, 28.11729007741826, 28.74968096872076, 29.37072852565012, 29.89819357400108, 30.465360292658026, 31.001332841788848, 31.52312622295324, 32.00521793381165, 32.464622975923774, 32.924028018035905, 33.41179139608088, 33.916569775685566, 34.37881065139098, 34.84672319428296, 35.286277401242096, 35.72583160820123, 36.145534980007376, 36.59076085415308, 37.03598672829879, 37.50106343759748, 37.86121430394465, 38.27524600856422, 38.680770212403935, 39.09480191702352, 39.52584862320279, 39.9512236621955, 40.39644953634121, 40.8246604089272, 41.26137878229306, 41.62436548223351, 42.02138218529336, 42.3872047188271, 42.77571392110711, 43.14153645464084, 43.524373989734286, 43.81646484984261, 44.08303320761138, 44.42049740521227, 44.71542409891388, 45.05288829651476, 45.387516660522365, 45.688115021410546, 45.949011711992746, 46.221251736948076, 46.49916342908998, 46.76856762045203, 47.0351359782208, 47.30170433598957, 47.54275019141877, 47.81215438278082, 48.02200606868389, 48.27439525848624, 48.47290361001617, 48.648725292799824, 48.85006947792304, 49.04574199585968, 49.213056177863486, 49.42857953095313, 49.62708788248305, 49.75753622777416, 49.89365624025183, 50.0099254175765, 50.1205229277146, 50.25380710659899, 50.42395712219607, 50.543062133114034, 50.639480475285715, 50.707540481524546, 50.80679465728951, 50.91455633383433, 51.002467175226165, 51.076198848651565, 51.178288858009815, 51.212318861129226, 51.26903553299492, 51.356946374386744, 51.373961375946465, 51.356946374386744, 51.41082721265916, 51.46470805093158, 51.48739471967785, 51.46754388452486, 51.433513881405446, 51.4420213821853, 51.42500638062559, 51.34560304001361, 51.28888636814792, 51.22649802909566, 51.11873635255083, 51.050676346311995, 50.95142217054703, 50.84366049400222, 50.718883815897684, 50.59127130419987, 50.46933045968862, 50.36156878314381, 50.25664294019228, 50.146045430054166, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.521765022828465, 28.32714176332133, 29.087145166321644, 29.835805234948815, 30.558942801236427, 31.24521453081133, 31.957008762725803, 32.57238465246859, 33.16790970705839, 33.7067180897825, 34.27672064203273, 34.889260698182234, 35.422397413719764, 35.94419079488416, 36.44613334089556, 36.976434222839806, 37.543600941496756, 38.054050988288004, 38.52479936477328, 38.987040240478684, 39.409579445878116, 39.86047698721039, 40.28868785979639, 40.75943623628165, 41.19331877605422, 41.59600714630066, 42.032725519666506, 42.40705555398009, 42.74168391798769, 43.133028953860986, 43.50735898817458, 43.822136517029186, 44.165272381816635, 44.533930748943654, 44.87423078013782, 45.20318747695885, 45.54632334174631, 45.89229504012705, 46.15319173070924, 46.45662592519071, 46.73453761733262, 47.04080764540737, 47.33857017270226, 47.58528769531804, 47.82916938434053, 48.06737940617645, 48.29141026004594, 48.50976944672887, 48.69693446388566, 48.85857697870289, 49.11096616850523, 49.30096701925532, 49.49663953719195, 49.66395371919576, 49.82843206760628, 50.00708958398321, 50.17156793239372, 50.29634461049825, 50.409777954229654, 50.5685846354536, 50.66500297762527, 50.832317159629085, 50.96843717210674, 51.073363015058284, 51.17545302441653, 51.246348864248645, 51.29739386892778, 51.356946374386744, 51.39948387828602, 51.46754388452486, 51.52709638998384, 51.549783058730114, 51.56679806028982, 51.6263505657488, 51.58664889544282, 51.5724697274764, 51.569633893883115, 51.569633893883115, 51.569633893883115, 51.54694722513683, 51.53276805717041, 51.49873805405099, 51.439185548592015, 51.39664804469274, 51.351274707200176, 51.39381221109945, 51.376797209539745, 51.303065536114346, 51.178288858009815, 51.107393018177696, 51.02231801037915, 50.937243002580615, 50.81246632447608, 50.68201797918499, 50.5685846354536, 50.46649462609534, 50.341717947990816, 50.25664294019228, 50.1573887644273, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.527436690015033, 28.33848509769447, 29.106996001474634, 29.88117857244137, 30.661032810594673, 31.358647874542722, 32.04775543771092, 32.65462382667385, 33.23596971329722, 33.797464764767604, 34.40716898732383, 34.994186541133764, 35.5273232566713, 36.03777330346256, 36.61344752289936, 37.21748007826901, 37.719422624280405, 38.207186002325386, 38.63539687491138, 39.106145251396654, 39.608087797408054, 40.041970337180615, 40.48436037773303, 40.9380937526586, 41.346453790091594, 41.78033632986416, 42.1830247001106, 42.56869806879733, 42.971386439043755, 43.32302980461107, 43.7257181748575, 44.07452570683152, 44.40631823724584, 44.72109576610045, 45.1209483027536, 45.47542750191419, 45.77886169639566, 46.08796755806369, 46.40841675410487, 46.714686782179626, 46.99543430791481, 47.262002665683575, 47.52857102345235, 47.752601877321844, 48.024841902277174, 48.23469358818024, 48.46156027564303, 48.66574029435953, 48.86708447948275, 49.06559283101268, 49.20171284349035, 49.40022119502028, 49.63843121685619, 49.80574539886, 49.950372912117516, 50.07514959022205, 50.20559793551315, 50.33604628080425, 50.47216629328191, 50.647987976065565, 50.758585486203664, 50.87201882993506, 50.9967955080396, 51.06485551427842, 51.1442588548904, 51.19246802597623, 51.19530385956952, 51.28605053455464, 51.373961375946465, 51.40799137906588, 51.433513881405446, 51.478887218897995, 51.46754388452486, 51.48739471967785, 51.50440972123755, 51.52142472279727, 51.50157388764427, 51.50157388764427, 51.50724555483084, 51.5157530556107, 51.49873805405099, 51.48739471967785, 51.47321555171143, 51.476051385304714, 51.46754388452486, 51.46470805093158, 51.45052888296515, 51.459036383745, 51.4420213821853, 51.43634971499873, 51.38814054391288, 51.37963304313303, 51.376797209539745, 51.3285880384539, 51.26903553299492, 51.16978135722996, 51.056348013498564, 50.86067549556192, 50.50619629640133, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2027, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.510421688455324, 28.312962595354907, 29.058786830388794, 29.77908856308312, 30.53625613249015, 31.222527862065057, 31.87760542211383, 32.586563820435025, 33.19910387658452, 33.81447976632732, 34.387318152170835, 34.94030570286136, 35.49329325355188, 36.07180330658197, 36.57091001900009, 37.115390068910756, 37.580466778209455, 38.10226015937385, 38.58435187023226, 39.038085245157816, 39.46346028415053, 39.908686158296234, 40.35958369962851, 40.770779570654796, 41.27839378385276, 41.71511215721861, 42.15750219777104, 42.61123557269659, 42.9458639367042, 43.325865638204355, 43.64347900065224, 44.031988202932254, 44.394974902872704, 44.718259932507166, 45.03870912854834, 45.37617332614923, 45.690950855003834, 46.00289255026515, 46.28647590959363, 46.51334259705641, 46.75438845248561, 46.99543430791481, 47.23364432975073, 47.4434960156538, 47.6675268695233, 47.953946062445056, 48.132603578822, 48.3736494342512, 48.54379944984829, 48.776337804497636, 48.96066698806114, 49.116637835691805, 49.33216118878144, 49.516490372344954, 49.67246121997562, 49.853954569945834, 49.9730595808638, 50.1205229277146, 50.2509712730057, 50.35589711595724, 50.4551512917222, 50.574256302640165, 50.65365964325213, 50.78127215494995, 50.874854663528346, 50.94858633695375, 50.98828800725974, 51.01664634319258, 51.10172135099113, 51.15560218926354, 51.22366219550237, 51.30590136970763, 51.30022970252106, 51.303065536114346, 51.311573036894195, 51.29455803533449, 51.29172220174121, 51.317244704080764, 51.32291637126733, 51.3285880384539, 51.348438873606895, 51.311573036894195, 51.28037886736807, 51.25769219862178, 51.252020531435214, 51.24067719706208, 51.24918469784193, 51.252020531435214, 51.24067719706208, 51.20664719394266, 51.17545302441653, 51.16410969004339, 51.16410969004339, 51.12440801973741, 51.07903468224485, 50.98545217366646, 50.89186966508805, 50.73873465105068, 50.50619629640133, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2027, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.552959192354592, 28.37251510081389, 29.11550350225449, 29.858491903695096, 30.629838641068545, 31.338797039389732, 31.99387459943851, 32.589399654028306, 33.250148881263655, 33.86836060459972, 34.46388565918952, 35.082097382525596, 35.618069931656414, 36.1597141479738, 36.69568669710461, 37.26001758216828, 37.753452627399824, 38.23838017185152, 38.76584522020248, 39.22525026231461, 39.70167030598645, 40.16391118169186, 40.58077871990472, 41.011825426084, 41.44287213226328, 41.85690383688285, 42.26242804072257, 42.58571307035703, 42.991237274196756, 43.41377647959618, 43.76825567875677, 44.14542154666364, 44.4516915747384, 44.75512576921986, 45.1379633043133, 45.427218330828346, 45.71647335734339, 46.04826588775771, 46.33184924708618, 46.623940107194514, 46.924538468082694, 47.22513682897088, 47.446331849247095, 47.69304937186286, 47.959617729631624, 48.13827524600857, 48.34245526472507, 48.552306950628136, 48.79051697246406, 48.99753282477384, 49.15066783881122, 49.352012023934435, 49.513654538751666, 49.68096872075547, 49.848282902759266, 50.00425375038993, 50.13470209568103, 50.30485211127811, 50.46365879250205, 50.59127130419987, 50.707540481524546, 50.75574965261038, 50.84366049400222, 50.89754133227462, 51.005303008819446, 51.070527181464996, 51.115900518957545, 51.212318861129226, 51.243513030655365, 51.26336386580835, 51.317244704080764, 51.33709553923376, 51.36545387516661, 51.342767206420326, 51.348438873606895, 51.356946374386744, 51.35978220798003, 51.37112554235318, 51.36261804157331, 51.354110540793464, 51.376797209539745, 51.356946374386744, 51.33709553923376, 51.348438873606895, 51.3285880384539, 51.28605053455464, 51.252020531435214, 51.24918469784193, 51.246348864248645, 51.23216969628223, 51.18963219238295, 51.178288858009815, 51.16127385645011, 51.11022885177098, 51.03082551115901, 51.002467175226165, 50.911720500241046, 50.750077985423815, 50.4551512917222, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.53878002438817, 28.318634262541476, 29.067294331168647, 29.807446899015964, 30.527748631710296, 31.21969202847177, 31.866262087740694, 32.51850381419619, 33.16790970705839, 33.763434761648185, 34.38448231857755, 34.926126534894934, 35.459263250432464, 35.95836996285058, 36.46031250886198, 36.976434222839806, 37.515242605563905, 37.943453478149905, 38.3716643507359, 38.81405439128832, 39.2082352607549, 39.61943113178119, 40.039134503587334, 40.42764370586735, 40.76227206987494, 41.136602104188526, 41.48257380256927, 41.82287383376344, 42.18586053370388, 42.56019056801747, 42.91750560077135, 43.25496979837224, 43.51303065536115, 43.77676317953663, 44.04900320449197, 44.34960156538015, 44.61049825596234, 44.894081615290816, 45.16632164024615, 45.387516660522365, 45.594532512832146, 45.83841420185464, 46.011400051045, 46.232595071321214, 46.47364092675042, 46.660805943907214, 46.890508464963276, 47.06349431415365, 47.219465161784306, 47.34140600629555, 47.53424269063892, 47.659019368743444, 47.82916938434053, 47.973796897598056, 48.10708107648243, 48.257380256926524, 48.40200777018405, 48.54379944984829, 48.66574029435953, 48.722456966225224, 48.82454697558347, 48.957831154467854, 49.11096616850523, 49.23290701301648, 49.36335535830757, 49.46260953407254, 49.53350537390466, 49.62425204888977, 49.70081955590846, 49.783058730113716, 49.86529790431897, 49.92201457618467, 49.95888041289737, 49.975895414457085, 50.0099254175765, 50.083657091001896, 50.12619459490117, 50.18007543317358, 50.225448770666134, 50.225448770666134, 50.148881263647446, 50.1035079261549, 50.09500042537505, 50.03261208632277, 49.99574624961007, 50.01276125116978, 50.00425375038993, 49.98723874883022, 49.98723874883022, 49.98723874883022, 49.984402915236934, 50.0269404191362, 50.05529875506905, 50.043955420695916, 50.07798542381533, 50.24813543941242, 50.404106287043085, 50.43246462297593, 50.33604628080425, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.53878002438817, 28.400873436746732, 29.14953350537391, 29.971925247426483, 30.70357031449395, 31.418200380001704, 32.06193460567734, 32.70850466494626, 33.394776394521166, 34.024331452230385, 34.64254317556646, 35.26075489890253, 35.85627995349233, 36.43195417292914, 36.92538921816068, 37.42449593057879, 37.96046847970961, 38.45106769134787, 38.88778606471372, 39.36420610838557, 39.81226781612455, 40.25182202308369, 40.68854039644953, 41.096900433882546, 41.56197714318124, 41.936307177494825, 42.33899554774126, 42.77571392110711, 43.16705895698041, 43.496015653801436, 43.84198735218218, 44.17661571618978, 44.488557411451104, 44.894081615290816, 45.22020247851856, 45.4725916683209, 45.795876697955364, 46.130505061962964, 46.419760088478014, 46.69483594702663, 46.93588180245584, 47.15424098913876, 47.39528684456796, 47.66469103593001, 47.87454272183309, 48.10991691007572, 48.322604429572074, 48.458724442049736, 48.67708362873267, 48.84439781073647, 49.02021949352012, 49.181862008337355, 49.349176190341154, 49.488132036412104, 49.66395371919576, 49.80574539886, 49.96738791367723, 50.09783625896833, 50.19709043473329, 50.333210447210966, 50.412613787822934, 50.52037546436775, 50.61112213935286, 50.65365964325213, 50.72171964949097, 50.79828715650966, 50.8464963275955, 50.90321299946119, 50.937243002580615, 51.005303008819446, 51.03933301193886, 51.06485551427842, 51.07903468224485, 51.09888551739784, 51.10455718458441, 51.107393018177696, 51.115900518957545, 51.141423021297115, 51.15276635567025, 51.12440801973741, 51.13007968692398, 51.138587187703834, 51.107393018177696, 51.10455718458441, 51.0676913478717, 51.06485551427842, 51.06201968068513, 51.06201968068513, 51.06485551427842, 51.06201968068513, 51.05918384709185, 51.05351217990528, 51.03366134475229, 51.008138842412734, 51.0138105095993, 50.97410883929331, 50.88619799790148, 50.79261548932309, 50.52321129796103, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.442361682216486, 28.097439242265267, 28.76953180387375, 29.399086861582962, 30.02297025210561, 30.59013697076256, 31.140288687859798, 31.6790970705839, 32.226412954087856, 32.70283299775969, 33.26716388282336, 33.72940475852877, 34.20015313501404, 34.64821484275303, 35.090604883305446, 35.57269659416386, 36.06329580580212, 36.4943425119814, 36.877180047074845, 37.339420922780256, 37.76196012817968, 38.19300683435896, 38.53614269914642, 38.92181606783314, 39.338683606046, 39.68465530442674, 40.013612001247765, 40.376598701188215, 40.75376456909508, 41.11675126903553, 41.51943963928197, 41.83705300172986, 42.168845532144175, 42.503473896151775, 42.795564756260106, 43.101834784334855, 43.41377647959618, 43.7257181748575, 44.05751070527182, 44.33542239741372, 44.55094575050336, 44.77497660437286, 45.05572413010805, 45.259904148824546, 45.48393500269404, 45.80438419873522, 46.07095255650398, 46.30349091115333, 46.493491761903414, 46.63528344156765, 46.85931429543715, 47.12021098601934, 47.31588350395599, 47.5370785242322, 47.73842270935542, 47.95678189603834, 48.087230241329436, 48.27439525848624, 48.419022771743755, 48.58917278734084, 48.76215863653121, 48.93514448572158, 49.05992116382611, 49.20738451067692, 49.36335535830757, 49.490967870005385, 49.615744548109916, 49.717834557468166, 49.81708873323314, 49.947537078524235, 50.05246292147577, 50.160224598020584, 50.19992626832658, 50.2878371097184, 50.39559878626323, 50.50336046280805, 50.57142046904688, 50.65082380965885, 50.73589881745739, 50.772764654170096, 50.81246632447608, 50.85500382837536, 50.84366049400222, 50.942914669767184, 50.99395967444631, 51.01664634319258, 51.06201968068513, 51.113064685364265, 51.12440801973741, 51.12440801973741, 51.135751354110546, 51.107393018177696, 51.03649717834558, 51.005303008819446, 50.96560133851346, 50.85500382837536, 50.72171964949097, 50.55440546748717, 50.32753878002438, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.530272523608318, 28.36400760003403, 29.118339335847775, 29.94640274508692, 30.655361143408104, 31.35014037376287, 31.985367098658653, 32.65462382667385, 33.30119388594277, 33.95627144599155, 34.54328899980149, 35.118963219238296, 35.66060743555568, 36.28449082607833, 36.80912004083601, 37.274196750134706, 37.744945126619974, 38.25823100700451, 38.73181521708307, 39.143011088109354, 39.633610299747616, 40.05898533874033, 40.4900320449196, 40.91257125031903, 41.31242378697218, 41.766157161897745, 42.13481552902476, 42.500638062558494, 42.900490599211636, 43.266313132745374, 43.59243399597312, 43.932734027167285, 44.233332388055466, 44.533930748943654, 44.85154411139155, 45.191844142585715, 45.523636673000034, 45.77319002920909, 46.01423588463829, 46.34035674786604, 46.6069251056348, 46.825284292317725, 47.05215097978051, 47.327226838329125, 47.53991435782549, 47.78946771403454, 48.047528571023456, 48.26021609051981, 48.419022771743755, 48.56932195218785, 48.73096446700508, 48.915293650568586, 49.06559283101268, 49.20171284349035, 49.30096701925532, 49.47395286844568, 49.64126705044948, 49.74619289340102, 49.89365624025183, 50.05529875506905, 50.160224598020584, 50.2311204378527, 50.30768794487139, 50.41544962141622, 50.50052462921476, 50.5685846354536, 50.65082380965885, 50.7160479823044, 50.79545132291637, 50.877690497121634, 50.93440716898733, 50.97410883929331, 51.01664634319258, 51.056348013498564, 51.073363015058284, 51.07903468224485, 51.05351217990528, 51.0676913478717, 51.09037801661799, 51.10172135099113, 51.10455718458441, 51.138587187703834, 51.14993052207697, 51.107393018177696, 51.113064685364265, 51.076198848651565, 51.05918384709185, 51.09321385021127, 51.09888551739784, 51.113064685364265, 51.0875421830247, 51.07903468224485, 51.06201968068513, 51.008138842412734, 50.89470549868133, 50.7897796557298, 50.6593313104387, 50.49768879562148, 50.267986274565416, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.530272523608318, 28.36400760003403, 29.118339335847775, 29.94640274508692, 30.655361143408104, 31.35014037376287, 31.985367098658653, 32.65462382667385, 33.30119388594277, 33.95627144599155, 34.54328899980149, 35.118963219238296, 35.66060743555568, 36.28449082607833, 36.80912004083601, 37.274196750134706, 37.744945126619974, 38.25823100700451, 38.73181521708307, 39.143011088109354, 39.633610299747616, 40.05898533874033, 40.4900320449196, 40.91257125031903, 41.31242378697218, 41.766157161897745, 42.13481552902476, 42.500638062558494, 42.900490599211636, 43.266313132745374, 43.59243399597312, 43.932734027167285, 44.233332388055466, 44.533930748943654, 44.85154411139155, 45.191844142585715, 45.523636673000034, 45.77319002920909, 46.01423588463829, 46.34035674786604, 46.6069251056348, 46.825284292317725, 47.05215097978051, 47.327226838329125, 47.53991435782549, 47.78946771403454, 48.047528571023456, 48.26021609051981, 48.419022771743755, 48.56932195218785, 48.73096446700508, 48.915293650568586, 49.06559283101268, 49.20171284349035, 49.30096701925532, 49.47395286844568, 49.64126705044948, 49.74619289340102, 49.89365624025183, 50.05529875506905, 50.160224598020584, 50.2311204378527, 50.30768794487139, 50.41544962141622, 50.50052462921476, 50.5685846354536, 50.65082380965885, 50.7160479823044, 50.79545132291637, 50.877690497121634, 50.93440716898733, 50.97410883929331, 51.01664634319258, 51.056348013498564, 51.073363015058284, 51.07903468224485, 51.05351217990528, 51.0676913478717, 51.09037801661799, 51.10172135099113, 51.10455718458441, 51.138587187703834, 51.14993052207697, 51.107393018177696, 51.113064685364265, 51.076198848651565, 51.05918384709185, 51.09321385021127, 51.09888551739784, 51.113064685364265, 51.0875421830247, 51.07903468224485, 51.06201968068513, 51.008138842412734, 50.89470549868133, 50.7897796557298, 50.6593313104387, 50.49768879562148, 50.267986274565416, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2028, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.510421688455324, 28.36400760003403, 29.158041006153763, 29.9322235771205, 30.692226980120807, 31.406857045628566, 32.095964608796756, 32.787908005558236, 33.457164733573435, 34.04418228738338, 34.66239401071945, 35.2664265660891, 35.859115787085614, 36.47449167682841, 37.02464339392565, 37.53792927431019, 38.017185151575305, 38.49360519524714, 38.92181606783314, 39.406743612284835, 39.89450699032981, 40.34540453166209, 40.83316790970706, 41.29824461900576, 41.70660465643876, 42.11496469387176, 42.46660805943907, 42.906162266398205, 43.24646229759238, 43.64631483424552, 43.95542069591357, 44.28721322632788, 44.599154921589204, 44.93094745200352, 45.197515809772284, 45.47542750191419, 45.730652525309814, 46.03975838697786, 46.326177579899614, 46.623940107194514, 46.882000964183426, 47.148569321952195, 47.375436009414976, 47.58528769531804, 47.83200521793381, 48.06737940617645, 48.280066925672806, 48.518276947508724, 48.69977029747894, 48.841561977143186, 49.014547826333555, 49.255593681762754, 49.35484785752772, 49.488132036412104, 49.62992371607634, 49.774551229333866, 49.90216374103168, 50.0269404191362, 50.154552930834015, 50.285001276125115, 50.384255451890084, 50.5487338003006, 50.67351047840513, 50.758585486203664, 50.84366049400222, 50.95425800414032, 51.0138105095993, 51.08187051583813, 51.121572186144114, 51.15560218926354, 51.17545302441653, 51.186796358789664, 51.20381136034937, 51.252020531435214, 51.26336386580835, 51.28321470096135, 51.33142387204719, 51.34560304001361, 51.35978220798003, 51.348438873606895, 51.29739386892778, 51.28888636814792, 51.27187136658821, 51.26336386580835, 51.25769219862178, 51.25769219862178, 51.2548563650285, 51.24918469784193, 51.252020531435214, 51.252020531435214, 51.24067719706208, 51.22366219550237, 51.178288858009815, 51.115900518957545, 51.03649717834558, 50.92306383461419, 50.85783966196864, 50.69052547996484, 50.4551512917222, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2028, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.501914187675467, 28.31579842894819, 29.106996001474634, 29.878342738848083, 30.629838641068545, 31.375662876102435, 32.11297961035647, 32.759549669625386, 33.3891047273346, 34.06686895612966, 34.66806567790602, 35.23239656296969, 35.79672744803335, 36.36956583387687, 36.851657544735275, 37.40180926183252, 37.87822930550435, 38.35748518276948, 38.84808439440774, 39.350026940419134, 39.81793948331112, 40.29719536057625, 40.799137906587646, 41.241527947140064, 41.63287298301336, 42.058248022006076, 42.47795139381221, 42.91750560077135, 43.28049230071179, 43.64631483424552, 44.04616737089868, 44.37795990131299, 44.70975243172731, 44.97348495590279, 45.291098318350684, 45.63707001673142, 45.91781754246661, 46.156027564302526, 46.50483509627655, 46.760060119672175, 47.00677764228796, 47.281853500836576, 47.48319768595979, 47.70722853982928, 47.87454272183309, 48.06454357258316, 48.257380256926524, 48.47290361001617, 48.63738195842669, 48.83589030995662, 49.03723449507983, 49.20454867708363, 49.32648952159487, 49.445594532512835, 49.59872954655021, 49.73768539262117, 49.848282902759266, 49.978731248050366, 50.086492924595184, 50.23395627144599, 50.30485211127811, 50.42112128860279, 50.534554632334185, 50.619629640132715, 50.704704647931266, 50.78127215494995, 50.8464963275955, 50.90888466664776, 50.96843717210674, 51.02515384397244, 51.04500467912543, 51.073363015058284, 51.0875421830247, 51.113064685364265, 51.14993052207697, 51.1442588548904, 51.10455718458441, 51.13007968692398, 51.13007968692398, 51.1272438533307, 51.10172135099113, 51.107393018177696, 51.09037801661799, 51.076198848651565, 51.050676346311995, 51.056348013498564, 51.010974676006015, 50.98828800725974, 50.9570938377336, 50.942914669767184, 50.94858633695375, 50.942914669767184, 50.945750503360465, 50.90321299946119, 50.877690497121634, 50.8294813260358, 50.76142131979695, 50.67634631199841, 50.412613787822934, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.518929189235173, 28.329977596914613, 29.075801831948507, 29.841476902135383, 30.58730113716927, 31.23670703003148, 31.957008762725803, 32.589399654028306, 33.20193971017781, 33.79462893117432, 34.387318152170835, 34.889260698182234, 35.41388991293991, 35.918668292544595, 36.44613334089556, 36.96225505487338, 37.42166009698551, 37.86405013753793, 38.2979326773105, 38.77718855457562, 39.18554859200862, 39.55987862632221, 39.973910330941784, 40.305702861356096, 40.674361228483114, 41.071377931542976, 41.43152879789014, 41.81436633298358, 42.168845532144175, 42.48362306099878, 42.730340583614556, 43.02243144372289, 43.334373138984205, 43.57258316082012, 43.824972350622474, 44.07452570683152, 44.31840739585401, 44.6558715934549, 44.85154411139155, 45.1209483027536, 45.322292487876815, 45.54632334174631, 45.78169752998894, 46.03408671979128, 46.26946090803392, 46.47647676034371, 46.6806567790602, 46.882000964183426, 47.07483764852679, 47.27334600005671, 47.426481014094094, 47.57678019453819, 47.74409437654199, 47.92842356010549, 48.06170773898988, 48.220514420213824, 48.419022771743755, 48.60902362249383, 48.77066613731106, 48.96633865524771, 49.07693616538582, 49.19887700989706, 49.3038028528486, 49.4342511981397, 49.5448487082778, 49.64126705044948, 49.717834557468166, 49.839775401979416, 49.92485040977795, 49.99291041601679, 50.12335876130789, 50.22828460425942, 50.33037461361768, 50.39276295266995, 50.47216629328191, 50.46082295890877, 50.49485296202819, 50.543062133114034, 50.55156963389389, 50.52037546436775, 50.50052462921476, 50.517539630774465, 50.46649462609534, 50.48634546124834, 50.49768879562148, 50.534554632334185, 50.577092136233446, 50.62530130731929, 50.642316308878996, 50.710376315117834, 50.78694382213652, 50.883362164308195, 50.88619799790148, 50.942914669767184, 50.92306383461419, 50.88903383149477, 50.76425715339023, 50.60261463857301, 50.39559878626323, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.521765022828465, 28.335649264101185, 29.106996001474634, 29.869835238068234, 30.578793636389417, 31.299095369083744, 32.01939710177807, 32.70283299775969, 33.31253722031591, 33.97895811473783, 34.585826503700765, 35.13314238720472, 35.72583160820123, 36.23911748858577, 36.828970875989, 37.3677792587131, 37.89808014065735, 38.38584351870232, 38.81972605847489, 39.2649519326206, 39.698834472393166, 40.12420951138588, 40.58645038709129, 40.98346709015115, 41.39182712758416, 41.75481382752461, 42.194368034483745, 42.57720556957718, 42.962878938263906, 43.28616396789837, 43.666165669398524, 43.9866148654397, 44.29004905992117, 44.599154921589204, 44.97348495590279, 45.31945665428353, 45.65408501829113, 45.89229504012705, 46.18155006664209, 46.47364092675042, 46.70901511499306, 46.97274763916853, 47.23080849615745, 47.52289935626578, 47.72707937498228, 47.93976689447863, 48.217678586620536, 48.43036610611689, 48.58917278734084, 48.80469614043048, 49.0060403255537, 49.15917533959108, 49.315146187221735, 49.482460369225535, 49.59589371295693, 49.77171539574058, 49.916342908998104, 50.046791254289204, 50.1403737628676, 50.25947877378556, 50.36724045033038, 50.446643790942346, 50.50052462921476, 50.59977880497972, 50.68768964637156, 50.78127215494995, 50.84366049400222, 50.883362164308195, 50.93157133539405, 50.9570938377336, 50.9967955080396, 51.01948217678587, 51.09321385021127, 51.10455718458441, 51.132915520517265, 51.1442588548904, 51.135751354110546, 51.121572186144114, 51.1272438533307, 51.141423021297115, 51.16978135722996, 51.15843802285682, 51.16978135722996, 51.16410969004339, 51.15843802285682, 51.15560218926354, 51.16127385645011, 51.15276635567025, 51.14993052207697, 51.141423021297115, 51.11873635255083, 51.08187051583813, 51.03933301193886, 51.0676913478717, 51.01664634319258, 50.95425800414032, 50.89470549868133, 50.707540481524546, 50.446643790942346, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.48489918611576, 28.2023650852168, 28.882965147605137, 29.512520205314352, 30.10804525990415, 30.672376144967817, 31.25372203159119, 31.74432124322945, 32.30014462751326, 32.77089300399852, 33.2785072171965, 33.7804497632079, 34.2625414740663, 34.70209568102544, 35.184187391883846, 35.654935768369114, 36.06613163939541, 36.480163344014976, 36.89703088222783, 37.31673425403398, 37.74210929302669, 38.212857669511955, 38.6694268780308, 39.0976377506168, 39.49181862008337, 39.86614865439696, 40.2319711879307, 40.609137055837564, 41.02600459405043, 41.403170461957295, 41.78317216345745, 42.14048719621133, 42.52899639849134, 42.8522814281258, 43.1869097921334, 43.54138899129399, 43.86183818733517, 44.2276607208689, 44.491393245044385, 44.794827439525854, 45.07273913166775, 45.37333749255594, 45.6796075206307, 45.96602671355245, 46.24961007288093, 46.487820094716845, 46.74871678529904, 46.9784193063551, 47.20245016022459, 47.4633468508068, 47.687377704676294, 47.885886056206225, 48.06454357258316, 48.251708589739955, 48.42753027252361, 48.620366956866974, 48.8387261435499, 49.00320449196042, 49.176190341150786, 49.29813118566203, 49.442758698919555, 49.62708788248305, 49.740521226214454, 49.848282902759266, 49.94470124493095, 50.00708958398321, 50.086492924595184, 50.18007543317358, 50.2311204378527, 50.29918044409154, 50.39276295266995, 50.4551512917222, 50.540226299520754, 50.62530130731929, 50.66783881121856, 50.718883815897684, 50.78694382213652, 50.778436321356665, 50.78694382213652, 50.849332161188784, 50.91739216742761, 50.942914669767184, 50.98261634007317, 51.02798967756572, 51.047840512718714, 51.0875421830247, 51.11873635255083, 51.16694552363668, 51.172617190823246, 51.18396052519638, 51.17545302441653, 51.181124691603095, 51.15276635567025, 51.08470634943142, 51.005303008819446, 50.90888466664776, 50.81246632447608, 50.70186881433798, 50.443807957349065, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.4962425204889, 28.28744009301534, 29.010577659302957, 29.74789439355699, 30.490882794997592, 31.18849785894564, 31.87760542211383, 32.57522048606188, 33.17925304143154, 33.786121430394466, 34.40149732013725, 34.96015653801435, 35.496129087145164, 36.03493746986927, 36.54538751666052, 37.081360065791344, 37.59464594617588, 38.10226015937385, 38.575844369452405, 38.99838357485183, 39.401071945098266, 39.82644698409098, 40.21495618637098, 40.66585372770326, 41.040183762016845, 41.371976292431164, 41.75481382752461, 42.072427189972494, 42.46660805943907, 42.81257975781981, 43.15287978901398, 43.41944814678275, 43.70870317329779, 44.04900320449197, 44.324079063040585, 44.60482658877577, 44.91393245044381, 45.18050080821258, 45.50662167144032, 45.76468252842923, 46.03408671979128, 46.283640076000346, 46.53602926580268, 46.75438845248561, 46.98692680713496, 47.20245016022459, 47.3697643422284, 47.556929359385194, 47.80081104840768, 47.979468564784625, 48.132603578822, 48.34812693191164, 48.53245611547514, 48.688426963105805, 48.827382809176754, 49.02589116070669, 49.20738451067692, 49.320817854408304, 49.49380370359867, 49.612908714516635, 49.720670391061454, 49.79156623089357, 49.941865411337666, 50.083657091001896, 50.18858293395344, 50.313359612057965, 50.41544962141622, 50.52321129796103, 50.57142046904688, 50.639480475285715, 50.72455548308425, 50.76425715339023, 50.837988826815646, 50.89186966508805, 50.90888466664776, 50.92589966820748, 50.97978050647989, 50.99112384085303, 51.056348013498564, 51.050676346311995, 51.13007968692398, 51.113064685364265, 51.12440801973741, 51.13007968692398, 51.115900518957545, 51.09604968380456, 51.1272438533307, 51.15560218926354, 51.15276635567025, 51.19530385956952, 51.1442588548904, 51.135751354110546, 51.10172135099113, 51.08187051583813, 51.02515384397244, 50.91739216742761, 50.75574965261038, 50.59410713779315, 50.35589711595724, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.521765022828465, 28.335649264101185, 29.106996001474634, 29.869835238068234, 30.578793636389417, 31.299095369083744, 32.01939710177807, 32.70283299775969, 33.31253722031591, 33.97895811473783, 34.585826503700765, 35.13314238720472, 35.72583160820123, 36.23911748858577, 36.828970875989, 37.3677792587131, 37.89808014065735, 38.38584351870232, 38.81972605847489, 39.2649519326206, 39.698834472393166, 40.12420951138588, 40.58645038709129, 40.98346709015115, 41.39182712758416, 41.75481382752461, 42.194368034483745, 42.57720556957718, 42.962878938263906, 43.28616396789837, 43.666165669398524, 43.9866148654397, 44.29004905992117, 44.599154921589204, 44.97348495590279, 45.31945665428353, 45.65408501829113, 45.89229504012705, 46.18155006664209, 46.47364092675042, 46.70901511499306, 46.97274763916853, 47.23080849615745, 47.52289935626578, 47.72707937498228, 47.93976689447863, 48.217678586620536, 48.43036610611689, 48.58917278734084, 48.80469614043048, 49.0060403255537, 49.15917533959108, 49.315146187221735, 49.482460369225535, 49.59589371295693, 49.77171539574058, 49.916342908998104, 50.046791254289204, 50.1403737628676, 50.25947877378556, 50.36724045033038, 50.446643790942346, 50.50052462921476, 50.59977880497972, 50.68768964637156, 50.78127215494995, 50.84366049400222, 50.883362164308195, 50.93157133539405, 50.9570938377336, 50.9967955080396, 51.01948217678587, 51.09321385021127, 51.10455718458441, 51.132915520517265, 51.1442588548904, 51.135751354110546, 51.121572186144114, 51.1272438533307, 51.141423021297115, 51.16978135722996, 51.15843802285682, 51.16978135722996, 51.16410969004339, 51.15843802285682, 51.15560218926354, 51.16127385645011, 51.15276635567025, 51.14993052207697, 51.141423021297115, 51.11873635255083, 51.08187051583813, 51.03933301193886, 51.0676913478717, 51.01664634319258, 50.95425800414032, 50.89470549868133, 50.707540481524546, 50.446643790942346, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2029, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.482063352522474, 28.290275926608626, 29.067294331168647, 29.8386410685421, 30.573121969202848, 31.29625953549046, 31.9768595978788, 32.631937157927574, 33.24164138048379, 33.83149476788702, 34.37881065139098, 34.94030570286136, 35.53299492385787, 36.07463914017526, 36.63329835805235, 37.13524090406375, 37.63434761648187, 38.09091682500071, 38.607038538978536, 39.08345858265037, 39.53435612398265, 39.99376116609478, 40.433315373053915, 40.8813770807929, 41.31809545415875, 41.78033632986416, 42.131979695431475, 42.52048889771148, 42.87496809687208, 43.2436264639991, 43.60661316393954, 43.95258486232028, 44.27019822476817, 44.60766242236905, 44.94229078637666, 45.27408331679097, 45.56901001049258, 45.821399200294934, 46.116325893996546, 46.40558092051158, 46.67214927828035, 46.918866800896126, 47.13439015398577, 47.34707767348212, 47.57678019453819, 47.7979752148144, 47.99081189915776, 48.19215608428098, 48.36514193347134, 48.53245611547514, 48.75081530215807, 48.906786149788736, 49.074100331792536, 49.2300711794232, 49.352012023934435, 49.510818705158385, 49.669625386382336, 49.788730397300284, 49.9730595808638, 50.151717097240734, 50.282165442531834, 50.39276295266995, 50.5288829651476, 50.619629640132715, 50.7699288205768, 50.90321299946119, 50.9967955080396, 51.09037801661799, 51.12440801973741, 51.16694552363668, 51.23216969628223, 51.27187136658821, 51.28888636814792, 51.28888636814792, 51.3285880384539, 51.348438873606895, 51.311573036894195, 51.342767206420326, 51.33993137282704, 51.34560304001361, 51.33993137282704, 51.33425970564047, 51.32008053767405, 51.311573036894195, 51.29455803533449, 51.27754303377478, 51.26052803221507, 51.25769219862178, 51.252020531435214, 51.246348864248645, 51.22366219550237, 51.20097552675609, 51.16694552363668, 51.13007968692398, 51.05918384709185, 50.96560133851346, 50.86351132915521, 50.752913819017095, 50.48067379406177, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2029, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.521765022828465, 28.341320931287754, 29.101324334288066, 29.923716076340646, 30.641181975441683, 31.406857045628566, 32.11865127754304, 32.77089300399852, 33.37776139296146, 34.01015228426396, 34.6170206732269, 35.23806823015626, 35.76553327850722, 36.34404333153731, 36.94240421972039, 37.481212602444494, 38.03136431954173, 38.49644102884043, 38.94450273657942, 39.466296117743816, 39.9512236621955, 40.29719536057625, 40.711227065195814, 41.12809460340867, 41.54779797521482, 41.936307177494825, 42.3503388821144, 42.667952244562294, 43.028103110909456, 43.35705980773048, 43.70870317329779, 44.0433315373054, 44.40915407083912, 44.68706576298103, 45.00467912542893, 45.31378498709696, 45.67393585344413, 45.9376683776196, 46.2354309049145, 46.555880100955676, 46.84797096106401, 47.12021098601934, 47.361256841448544, 47.5937951960979, 47.866035221053224, 48.06737940617645, 48.32544026316536, 48.523948614695286, 48.67708362873267, 48.88977114822902, 49.079771998979105, 49.247086180982905, 49.39171369424042, 49.52783370671809, 49.655446218415904, 49.80858123245328, 49.947537078524235, 50.06097042225562, 50.18858293395344, 50.2878371097184, 50.39276295266995, 50.48067379406177, 50.56291296826703, 50.6593313104387, 50.750077985423815, 50.809630490882796, 50.883362164308195, 50.91739216742761, 50.9570938377336, 50.99963134163288, 51.047840512718714, 51.056348013498564, 51.0875421830247, 51.107393018177696, 51.115900518957545, 51.10172135099113, 51.107393018177696, 51.10172135099113, 51.0875421830247, 51.11022885177098, 51.132915520517265, 51.132915520517265, 51.13007968692398, 51.11873635255083, 51.14709468848368, 51.15843802285682, 51.15560218926354, 51.135751354110546, 51.1272438533307, 51.11022885177098, 51.0875421830247, 51.073363015058284, 51.03649717834558, 51.008138842412734, 50.96276550492017, 50.93440716898733, 50.8464963275955, 50.707540481524546, 50.446643790942346, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.530272523608318, 28.312962595354907, 29.058786830388794, 29.776252729489837, 30.52491279811701, 31.21969202847177, 31.846411252587703, 32.51283214700962, 33.1111930351927, 33.70955392337578, 34.30224314437229, 34.86657402943595, 35.38269574341378, 35.847772452712476, 36.27598332529847, 36.780761704903156, 37.27986841732127, 37.747780960213255, 38.19300683435896, 38.63823270850467, 39.035249411564536, 39.42659444743782, 39.769730312225285, 40.08450784107988, 40.52122621444574, 40.909735416725745, 41.26988628307291, 41.58466381192752, 41.899441340782126, 42.22556220400987, 42.565862235204044, 42.88347559765193, 43.164223123387124, 43.4846723194283, 43.78527068031648, 44.05751070527182, 44.301392394294304, 44.59064742080935, 44.84587244420498, 45.126619969940165, 45.37333749255594, 45.58886084564558, 45.77319002920909, 45.949011711992746, 46.12766922836968, 46.33184924708618, 46.5700592689221, 46.78841845560503, 46.96991180557525, 47.128718486799194, 47.34424183988884, 47.52289935626578, 47.690213538269575, 47.84051271871367, 48.044692737430175, 48.2630519241131, 48.410515270963906, 48.59484445452741, 48.75932280293792, 48.9323086521283, 49.05424949663954, 49.18753367552392, 49.33499702237473, 49.45410203329269, 49.547684541871085, 49.68096872075547, 49.81141706604657, 49.89649207384511, 49.95888041289737, 50.083657091001896, 50.14320959646088, 50.25380710659899, 50.36156878314381, 50.42112128860279, 50.5288829651476, 50.55724130108046, 50.5855996370133, 50.63664464169243, 50.70186881433798, 50.76142131979695, 50.79828715650966, 50.778436321356665, 50.76425715339023, 50.76142131979695, 50.772764654170096, 50.80395882369623, 50.8266454924425, 50.837988826815646, 50.89186966508805, 50.92589966820748, 50.942914669767184, 50.9967955080396, 51.04500467912543, 51.08187051583813, 51.076198848651565, 51.01948217678587, 50.91739216742761, 50.744406318237246, 50.42679295578936, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.541615857981455, 28.329977596914613, 29.061622663982078, 29.81879023338911, 30.539091966083433, 31.25939369877776, 31.957008762725803, 32.657459660267136, 33.31253722031591, 33.97328644755126, 34.61985650682018, 35.23523239656297, 35.8052349488132, 36.33269999716417, 36.880015880668125, 37.35643592433996, 37.81584096645209, 38.32912684683663, 38.86226356237416, 39.31316110370644, 39.778237813005134, 40.25182202308369, 40.65167455973684, 41.02033292686386, 41.44570796585656, 41.87959050562913, 42.28511470946885, 42.65944474378244, 43.064968947622155, 43.38258231007005, 43.79377818109633, 44.17377988259649, 44.58497575362278, 44.84303661061169, 45.126619969940165, 45.430054164421634, 45.72214502452996, 46.03408671979128, 46.27513257522049, 46.57856676970196, 46.825284292317725, 47.06065848056036, 47.3328985055157, 47.61081019765761, 47.820661883560675, 48.03618523665032, 48.30558942801236, 48.51260528032215, 48.711113631852086, 48.90395031619544, 49.13365283725151, 49.349176190341154, 49.53917704109123, 49.669625386382336, 49.80858123245328, 49.99291041601679, 50.16306043161387, 50.27932960893855, 50.37574795111023, 50.48918129484162, 50.582763803420015, 50.72739131667754, 50.823809658849214, 50.883362164308195, 50.95142217054703, 51.03082551115901, 51.056348013498564, 51.08187051583813, 51.15560218926354, 51.181124691603095, 51.22933386268894, 51.2747072001815, 51.32008053767405, 51.39097637750617, 51.430678047812165, 51.439185548592015, 51.4420213821853, 51.39664804469274, 51.382468876726314, 51.36545387516661, 51.382468876726314, 51.356946374386744, 51.356946374386744, 51.36261804157331, 51.35978220798003, 51.35978220798003, 51.354110540793464, 51.348438873606895, 51.34560304001361, 51.31440887048748, 51.26903553299492, 51.23500552987551, 51.1981396931628, 51.16978135722996, 51.121572186144114, 51.03366134475229, 50.945750503360465, 50.72739131667754, 50.50619629640133, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.44519751580977, 28.12296174460483, 28.724158466381194, 29.365056858463547, 29.90386524118765, 30.48237529421774, 31.01551200975527, 31.545812891699516, 32.09029294161019, 32.58372798684173, 33.119700535972555, 33.58194141167796, 34.03851062019681, 34.47522899356266, 34.90911153333522, 35.39971074497349, 35.859115787085614, 36.35255083231716, 36.820463375209144, 37.25151008138843, 37.753452627399824, 38.144797663273124, 38.57017270226583, 38.95868190454584, 39.39256444431841, 39.78958114737827, 40.10152284263959, 40.46167370898676, 40.901227915945896, 41.25003544791992, 41.61585798145365, 42.01854635170008, 42.38436888523381, 42.71048974846156, 43.04795394606244, 43.41661231318946, 43.77392734594334, 44.10571987635766, 44.44318407395854, 44.80049910671242, 45.095425800414034, 45.37333749255594, 45.6796075206307, 46.04259422057114, 46.31767007911976, 46.57573093610867, 46.86215012903043, 47.10319598445963, 47.30454016958285, 47.53991435782549, 47.814990216374106, 48.00782690071747, 48.26872359129967, 48.458724442049736, 48.64021779201997, 48.79902447324391, 49.014547826333555, 49.19320534271049, 49.36335535830757, 49.53634120749795, 49.7688795621473, 49.95888041289737, 50.035447919916066, 50.19425460114001, 50.29634461049825, 50.42112128860279, 50.52037546436775, 50.622465473725995, 50.67067464481184, 50.758585486203664, 50.823809658849214, 50.95142217054703, 51.0676913478717, 51.15560218926354, 51.2179905283158, 51.246348864248645, 51.27187136658821, 51.28321470096135, 51.33709553923376, 51.36545387516661, 51.37112554235318, 51.373961375946465, 51.35978220798003, 51.35978220798003, 51.39097637750617, 51.376797209539745, 51.356946374386744, 51.36545387516661, 51.33142387204719, 51.237841363468796, 51.19530385956952, 51.11022885177098, 51.02798967756572, 50.94858633695375, 50.80679465728951, 50.67351047840513, 50.540226299520754, 50.4012704534498, 50.242463772225854, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.535944190794886, 28.304455094575054, 29.027592660862663, 29.76774522870998, 30.507897796557298, 31.23670703003148, 31.934322093979528, 32.55536965090889, 33.24164138048379, 33.893883106939285, 34.53761733261492, 35.124634886424865, 35.66911493633554, 36.21926665343278, 36.749567535377025, 37.237330913422, 37.7024076227207, 38.15897683123954, 38.66091937725095, 39.140175254516066, 39.585401128661765, 39.99943283328135, 40.473017043359896, 40.841675410486914, 41.241527947140064, 41.61302214786037, 41.99018801576724, 42.401383886793525, 42.787057255480256, 43.18974562572669, 43.57825482800669, 43.94124152794714, 44.31557156226072, 44.653035759861616, 44.964977455122934, 45.197515809772284, 45.44706916598134, 45.73916002608967, 46.00289255026515, 46.283640076000346, 46.50199926268327, 46.79692595638488, 47.01528514306781, 47.219465161784306, 47.47469018517994, 47.704392706236, 47.93409522729206, 48.149618580381706, 48.39350026940419, 48.60335195530726, 48.753651135751355, 48.94648782009472, 49.15633950599779, 49.349176190341154, 49.5448487082778, 49.68947622153532, 49.8426112355727, 50.0099254175765, 50.16306043161387, 50.26515044097213, 50.36440461673709, 50.49485296202819, 50.5685846354536, 50.642316308878996, 50.76709298698352, 50.877690497121634, 50.95992967132689, 51.03649717834558, 51.076198848651565, 51.10455718458441, 51.12440801973741, 51.181124691603095, 51.18963219238295, 51.23216969628223, 51.26903553299492, 51.317244704080764, 51.3853047103196, 51.376797209539745, 51.40515554547259, 51.37963304313303, 51.37963304313303, 51.36261804157331, 51.35978220798003, 51.34560304001361, 51.33993137282704, 51.342767206420326, 51.3285880384539, 51.28321470096135, 51.252020531435214, 51.21515469472251, 51.15560218926354, 51.05918384709185, 50.9967955080396, 50.880526330714915, 50.73022715027082, 50.61112213935286, 50.446643790942346, 50.338882114397535, 50.239627938632566, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.541615857981455, 28.329977596914613, 29.061622663982078, 29.81879023338911, 30.539091966083433, 31.25939369877776, 31.957008762725803, 32.657459660267136, 33.31253722031591, 33.97328644755126, 34.61985650682018, 35.23523239656297, 35.8052349488132, 36.33269999716417, 36.880015880668125, 37.35643592433996, 37.81584096645209, 38.32912684683663, 38.86226356237416, 39.31316110370644, 39.778237813005134, 40.25182202308369, 40.65167455973684, 41.02033292686386, 41.44570796585656, 41.87959050562913, 42.28511470946885, 42.65944474378244, 43.064968947622155, 43.38258231007005, 43.79377818109633, 44.17377988259649, 44.58497575362278, 44.84303661061169, 45.126619969940165, 45.430054164421634, 45.72214502452996, 46.03408671979128, 46.27513257522049, 46.57856676970196, 46.825284292317725, 47.06065848056036, 47.3328985055157, 47.61081019765761, 47.820661883560675, 48.03618523665032, 48.30558942801236, 48.51260528032215, 48.711113631852086, 48.90395031619544, 49.13365283725151, 49.349176190341154, 49.53917704109123, 49.669625386382336, 49.80858123245328, 49.99291041601679, 50.16306043161387, 50.27932960893855, 50.37574795111023, 50.48918129484162, 50.582763803420015, 50.72739131667754, 50.823809658849214, 50.883362164308195, 50.95142217054703, 51.03082551115901, 51.056348013498564, 51.08187051583813, 51.15560218926354, 51.181124691603095, 51.22933386268894, 51.2747072001815, 51.32008053767405, 51.39097637750617, 51.430678047812165, 51.439185548592015, 51.4420213821853, 51.39664804469274, 51.382468876726314, 51.36545387516661, 51.382468876726314, 51.356946374386744, 51.356946374386744, 51.36261804157331, 51.35978220798003, 51.35978220798003, 51.354110540793464, 51.348438873606895, 51.34560304001361, 51.31440887048748, 51.26903553299492, 51.23500552987551, 51.1981396931628, 51.16978135722996, 51.121572186144114, 51.03366134475229, 50.945750503360465, 50.72739131667754, 50.50619629640133, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2030, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.541615857981455, 28.366843433627313, 29.20057851005303, 29.935059410713784, 30.68655531293424, 31.395513711255425, 32.00521793381165, 32.631937157927574, 33.255820548450224, 33.86552477100644, 34.51493066386865, 35.10478405127188, 35.66911493633554, 36.22493832061935, 36.73822420100389, 37.237330913422, 37.69390012194085, 38.19017100076568, 38.686441879590504, 39.205399427161616, 39.62226696537448, 40.05047783796047, 40.45033037461362, 40.83033207611378, 41.20466211042736, 41.58466381192752, 41.95899384624111, 42.34750304852112, 42.72466891642799, 43.062133114028875, 43.459149817088736, 43.8193006834359, 44.094376541984516, 44.45452740833168, 44.80617077389899, 45.14363497149987, 45.46692000113433, 45.82990670107478, 46.11916172758983, 46.41125258769816, 46.663641777500494, 46.93021013526927, 47.14006182117234, 47.39812267816125, 47.59946686328447, 47.8348410515271, 48.04185690383689, 48.288574426452655, 48.51260528032215, 48.71962113263194, 48.93514448572158, 49.119473669285085, 49.249922014576185, 49.38320619346057, 49.547684541871085, 49.69798372231517, 49.82559623401299, 49.99574624961007, 50.151717097240734, 50.26231460737884, 50.412613787822934, 50.52037546436775, 50.63664464169243, 50.73873465105068, 50.823809658849214, 50.911720500241046, 50.95425800414032, 51.042168845532146, 51.09321385021127, 51.11022885177098, 51.16410969004339, 51.18963219238295, 51.2548563650285, 51.31440887048748, 51.29739386892778, 51.33425970564047, 51.308737203300915, 51.308737203300915, 51.303065536114346, 51.303065536114346, 51.29455803533449, 51.29455803533449, 51.2747072001815, 51.28037886736807, 51.27187136658821, 51.2747072001815, 51.26052803221507, 51.26336386580835, 51.26336386580835, 51.25769219862178, 51.24067719706208, 51.21515469472251, 51.178288858009815, 51.09604968380456, 51.02798967756572, 50.94858633695375, 50.815302158069365, 50.67351047840513, 50.49201712843491, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2030, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.516093355641893, 28.329977596914613, 29.05027932960894, 29.835805234948815, 30.598644471542414, 31.31043870345688, 31.98253126506537, 32.657459660267136, 33.28134305078978, 33.893883106939285, 34.461049825596234, 35.07075404815246, 35.6946374386751, 36.30434166123132, 36.83747837676885, 37.353600090746674, 37.86121430394465, 38.346141848396336, 38.80554689050847, 39.335847772452716, 39.83211865127755, 40.26032952386354, 40.70839123160253, 41.13093043700196, 41.55914130958796, 41.941978844681394, 42.35884638289426, 42.727504750021275, 43.11034228511471, 43.47049315146187, 43.771091512350054, 44.1170632107308, 44.44601990755183, 44.77214077077957, 45.07273913166775, 45.37617332614923, 45.662592519070984, 45.90931004168676, 46.17304256586224, 46.40841675410487, 46.660805943907214, 46.91319513370956, 47.13155432039248, 47.33006267192241, 47.517227689079206, 47.755437710915125, 47.93976689447863, 48.12976774522871, 48.34812693191164, 48.48708277798259, 48.685591129512524, 48.90111448260216, 49.02589116070669, 49.241414513796336, 49.40022119502028, 49.59872954655021, 49.720670391061454, 49.876641238692116, 49.99858208320336, 50.12619459490117, 50.27365794175198, 50.36440461673709, 50.4551512917222, 50.56291296826703, 50.642316308878996, 50.718883815897684, 50.7897796557298, 50.86918299634178, 50.96843717210674, 51.01664634319258, 51.047840512718714, 51.076198848651565, 51.09037801661799, 51.1272438533307, 51.132915520517265, 51.138587187703834, 51.15276635567025, 51.17545302441653, 51.20097552675609, 51.22933386268894, 51.23216969628223, 51.2548563650285, 51.252020531435214, 51.24918469784193, 51.237841363468796, 51.23500552987551, 51.22082636190909, 51.212318861129226, 51.181124691603095, 51.14709468848368, 51.09321385021127, 51.073363015058284, 51.050676346311995, 50.98261634007317, 50.93440716898733, 50.837988826815646, 50.8266454924425, 50.704704647931266, 50.452315458128915, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "MiniCPM", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.53878002438817, 28.32147009613476, 29.075801831948507, 29.815954399795825, 30.533420298896864, 31.208348694098632, 31.7726795791623, 32.40790630405808, 33.04880469614043, 33.68403142103622, 34.316422312338716, 34.8779173638091, 35.43657658168618, 35.983892465190145, 36.457476675268694, 36.950911720500244, 37.450018432918355, 37.87539347191107, 38.28658934293736, 38.62688937413153, 39.049428579530954, 39.48898278649009, 39.86331282080368, 40.28301619260982, 40.67152539488983, 41.040183762016845, 41.38331962680431, 41.74914216033804, 42.07526302356578, 42.42690638913309, 42.78422142188696, 43.07914811558858, 43.34004480617077, 43.64347900065224, 43.89019652326802, 44.199302384936054, 44.48572157785781, 44.74094660125344, 44.92527578481695, 45.17199330743272, 45.455576666761196, 45.66826418625755, 45.84408586904121, 46.065280889317414, 46.289311743186914, 46.46796925956385, 46.74304511811247, 46.93304596886255, 47.13155432039248, 47.324391004735844, 47.49737685392621, 47.67603437030316, 47.78946771403454, 47.922751892918924, 48.10991691007572, 48.23185775458696, 48.410515270963906, 48.54379944984829, 48.73947196778493, 48.88126364744917, 49.014547826333555, 49.20171284349035, 49.34634035674787, 49.485296202818816, 49.59872954655021, 49.714998723874885, 49.79723789808014, 49.93052207696452, 50.0099254175765, 50.046791254289204, 50.12335876130789, 50.211269602699716, 50.242463772225854, 50.341717947990816, 50.378583784703515, 50.4551512917222, 50.543062133114034, 50.5855996370133, 50.65082380965885, 50.68201797918499, 50.73873465105068, 50.75574965261038, 50.72171964949097, 50.69619714715141, 50.69336131355813, 50.679182145591696, 50.67351047840513, 50.69052547996484, 50.6990329807447, 50.7160479823044, 50.76142131979695, 50.815302158069365, 50.880526330714915, 50.937243002580615, 50.937243002580615, 50.90604883305447, 50.78694382213652, 50.65649547684542, 50.37574795111023, 50.00708958398321]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.635198366559855, 28.57102345234382, 29.523863539687493, 30.422822788758758, 31.327453705016595, 32.22074128690129, 33.1111930351927, 33.95059977880498, 34.784334855230696, 35.63224909962284, 36.443297507302276, 37.2004650767093, 38.02569265235516, 38.81972605847489, 39.61092363100134, 40.413464537900914, 41.15928877293481, 41.93347134390154, 42.71616141564814, 43.44780648271559, 44.16243654822335, 44.87423078013782, 45.628562515951565, 46.38005841817202, 47.07483764852679, 47.76961687888155, 48.47290361001617, 49.15633950599779, 49.82559623401299, 50.446643790942346, 51.07903468224485, 51.75112724385333, 52.45724980858124, 53.0414315287979, 53.68800158806681, 54.311884978589454, 54.904574199585966, 55.50577092136233, 56.07293764001928, 56.665626861015795, 57.21577857811304, 57.81413946629612, 58.32175367949408, 58.92578623486374, 59.45892295040127, 59.986387998752235, 60.550718883815904, 61.14057227121913, 61.616992314890965, 62.10475569293594, 62.63505657488019, 63.19087995916399, 63.65028500127612, 64.13237671213453, 64.64282675892579, 65.15611263931032, 65.69775685562772, 66.18835606726597, 66.63641777500496, 67.07030031477753, 67.51552618892323, 67.93806539432265, 68.36627626690866, 68.78881547230809, 69.20568301052094, 69.62255054873381, 70.00822391742052, 70.35703144939455, 70.77957065479397, 71.13688568754786, 71.48569321952188, 71.88554575617503, 72.23718912174235, 72.62569832402235, 72.92346085131724, 73.18719337549273, 73.50197090434733, 73.74868842696311, 73.99257011598559, 74.26764597453422, 74.44630349091115, 74.65048350962766, 74.7922751892919, 74.90854436661657, 75.00496270878824, 75.07302271502708, 75.08153021580695, 75.1070527181465, 75.11272438533308, 75.10988855173979, 75.1070527181465, 75.09570938377337, 75.10138105095993, 75.10421688455322, 75.10421688455322, 75.11272438533308, 75.08720188299351, 75.02197771034795, 74.82063352522475, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.635198366559855, 28.519978447664695, 29.427445197515812, 30.320732779400505, 31.177154524572497, 32.04208377052435, 32.892833848509774, 33.70955392337578, 34.53761733261492, 35.37135240904064, 36.20508748546635, 36.97076255565324, 37.76479596177297, 38.547486033519554, 39.32450443807958, 40.0646570059269, 40.78779457221451, 41.51093213850211, 42.17735303292403, 42.89481893202507, 43.632135666279105, 44.38930323568613, 45.10393330119389, 45.824235033888215, 46.53035759861611, 47.23364432975073, 47.86319938745994, 48.54379944984829, 49.2300711794232, 49.92768624337124, 50.579927969826734, 51.20664719394266, 51.836202251651876, 52.49127981170065, 53.13501403737628, 53.7844199302385, 54.388452485608155, 55.035022544877066, 55.61069676431387, 56.23458015483651, 56.92085188441143, 57.561750276493775, 58.11473782718431, 58.73011371692709, 59.291608768397474, 59.889969656580554, 60.440121373677805, 61.021467260301165, 61.59997731333126, 62.12177069449565, 62.680429912372745, 63.22490996228341, 63.743867509854525, 64.24013838867934, 64.69954343079148, 65.21282931117602, 65.66656268610159, 66.19686356804583, 66.69313444687066, 67.16104698976264, 67.6147803646882, 68.05433457164733, 68.4513512747072, 68.8795621472932, 69.36732552533817, 69.7558347276182, 70.19822476817062, 70.60374897201032, 71.0347956781896, 71.3892748773502, 71.70121657261153, 72.12091994441766, 72.45838414201855, 72.78734083883958, 73.09361086691433, 73.3970450613958, 73.64376258401157, 73.85077843632135, 74.02376428551173, 74.18540680032896, 74.31585514562006, 74.40376598701188, 74.56540850182911, 74.69018517993365, 74.80645435725832, 74.87451436349714, 74.99929104160168, 75.03899271190765, 75.0900377165868, 75.10988855173979, 75.11272438533308, 75.12123188611292, 75.09570938377337, 75.05317187987409, 75.01630604316138, 74.94824603692255, 74.883021864277, 74.7355585174262, 74.60227433854182, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.618183365000142, 28.559680117970682, 29.489833536568078, 30.408643620792336, 31.276408700337466, 32.14984544706917, 33.02328219380087, 33.822987267107166, 34.63119984119332, 35.479114085585465, 36.25046082295891, 37.08703173297791, 37.8328559680118, 38.55315770070612, 39.372713609165416, 40.1185378441993, 40.83600374330034, 41.60167881348723, 42.37869721804724, 43.12735728667442, 43.83347985140232, 44.531094915350366, 45.197515809772284, 45.9008025409069, 46.53602926580268, 47.256330998497006, 47.89722939057937, 48.580665286560986, 49.20171284349035, 49.86529790431897, 50.514703797181184, 51.056348013498564, 51.75396307744662, 52.35515979922298, 53.01590902645833, 53.60859824745484, 54.15874996455208, 54.808155857414285, 55.39233757763095, 55.91129512520205, 56.49264101182543, 57.028613560956245, 57.570257777273625, 58.089215324844744, 58.622352040382275, 59.186682925445936, 59.67444630349091, 60.184896350282166, 60.655644726767434, 61.174602274338554, 61.70490315628279, 62.27206987493974, 62.73431075064515, 63.22207412869013, 63.67580750361569, 64.12386921135467, 64.56058758472054, 64.96327595496696, 65.4198451634858, 65.89059353997108, 66.31596857896379, 66.76686612029606, 67.18373365850891, 67.58358619516207, 67.99478206618836, 68.3379179309758, 68.70657629810283, 69.0837421660097, 69.46374386750986, 69.85225306978987, 70.20389643535718, 70.53852479936478, 70.86464566259252, 71.2021098601934, 71.49136488670844, 71.80897824915634, 72.10674077645123, 72.38181663499986, 72.68525082948133, 72.9886850239628, 73.28361171766441, 73.56435924339961, 73.79689759804894, 73.99824178317216, 74.24212347219465, 74.4207809885716, 74.55973683464254, 74.76108101976577, 74.89720103224343, 75.03615687831437, 75.09854521736665, 75.1240677197062, 75.10988855173979, 75.06451521424722, 75.02481354394124, 74.93690270254942, 74.86884269631058, 74.76675268695233, 74.62496100728809, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.635198366559855, 28.57102345234382, 29.523863539687493, 30.422822788758758, 31.327453705016595, 32.22074128690129, 33.1111930351927, 33.95059977880498, 34.784334855230696, 35.63224909962284, 36.443297507302276, 37.2004650767093, 38.02569265235516, 38.81972605847489, 39.61092363100134, 40.413464537900914, 41.15928877293481, 41.93347134390154, 42.71616141564814, 43.44780648271559, 44.16243654822335, 44.87423078013782, 45.628562515951565, 46.38005841817202, 47.07483764852679, 47.76961687888155, 48.47290361001617, 49.15633950599779, 49.82559623401299, 50.446643790942346, 51.07903468224485, 51.75112724385333, 52.45724980858124, 53.0414315287979, 53.68800158806681, 54.311884978589454, 54.904574199585966, 55.50577092136233, 56.07293764001928, 56.665626861015795, 57.21577857811304, 57.81413946629612, 58.32175367949408, 58.92578623486374, 59.45892295040127, 59.986387998752235, 60.550718883815904, 61.14057227121913, 61.616992314890965, 62.10475569293594, 62.63505657488019, 63.19087995916399, 63.65028500127612, 64.13237671213453, 64.64282675892579, 65.15611263931032, 65.69775685562772, 66.18835606726597, 66.63641777500496, 67.07030031477753, 67.51552618892323, 67.93806539432265, 68.36627626690866, 68.78881547230809, 69.20568301052094, 69.62255054873381, 70.00822391742052, 70.35703144939455, 70.77957065479397, 71.13688568754786, 71.48569321952188, 71.88554575617503, 72.23718912174235, 72.62569832402235, 72.92346085131724, 73.18719337549273, 73.50197090434733, 73.74868842696311, 73.99257011598559, 74.26764597453422, 74.44630349091115, 74.65048350962766, 74.7922751892919, 74.90854436661657, 75.00496270878824, 75.07302271502708, 75.08153021580695, 75.1070527181465, 75.11272438533308, 75.10988855173979, 75.1070527181465, 75.09570938377337, 75.10138105095993, 75.10421688455322, 75.10421688455322, 75.11272438533308, 75.08720188299351, 75.02197771034795, 74.82063352522475, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2026, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.672064203272555, 28.619232623429657, 29.574908544366618, 30.493718628590877, 31.42954371437484, 32.32566712985282, 33.21895471173752, 34.123585627995354, 34.99985820832034, 35.853444119899045, 36.67867169554491, 37.50389927119077, 38.32912684683663, 39.19122025919519, 39.95689532938208, 40.72257039956896, 41.50242463772226, 42.32481637977484, 43.11601395230128, 43.81646484984261, 44.658707427048185, 45.393188327708934, 46.11916172758983, 46.84797096106401, 47.57678019453819, 48.251708589739955, 48.93514448572158, 49.578878711397216, 50.17440376598701, 50.823809658849214, 51.43634971499873, 52.091427275047494, 52.67844482885744, 53.361880724839075, 54.00277911692142, 54.572781669171654, 55.1768142245413, 55.763831778351246, 56.33950599778805, 56.963389388310695, 57.527720273374364, 58.09205115843803, 58.65638204350169, 59.1923545926325, 59.78504381362902, 60.32385219635312, 60.87683974704364, 61.41848396336103, 61.93176984374558, 62.47341406006296, 62.91580410061538, 63.4149108130335, 63.919689192638174, 64.38760173553015, 64.82999177608258, 65.27521765022829, 65.77999602983297, 66.23940107194511, 66.71865694921021, 67.14119615460964, 67.54955619204266, 68.01463290134136, 68.4513512747072, 68.80866630746108, 69.20284717692766, 69.65658055185322, 70.04225392053995, 70.50449479624537, 70.9440490032045, 71.36091654141735, 71.71255990698467, 72.03584493661911, 72.39599580296628, 72.70793749822761, 73.08793919972776, 73.42823923092193, 73.74301675977654, 74.04361512066473, 74.29316847687377, 74.51436349714999, 74.67884184556051, 74.863171029124, 74.97093270566883, 75.05317187987409, 75.09570938377337, 75.11839605251964, 75.14675438845248, 75.15526188923234, 75.16660522360547, 75.16660522360547, 75.16376939001219, 75.16660522360547, 75.16093355641891, 75.15526188923234, 75.14959022204577, 75.12973938689278, 75.13257522048606, 75.03615687831437, 74.82346935881803, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2026, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.589825029067296, 28.5369934492244, 29.464311034228512, 30.363270283299777, 31.299095369083744, 32.198054618155005, 33.07716303207328, 33.953435612398266, 34.80702152397698, 35.63508493321612, 36.42628250574256, 37.30255508606755, 38.1674843320194, 39.012562742818254, 39.85764115361711, 40.674361228483114, 41.45705130022971, 42.21705470323002, 42.97422227263704, 43.71437484048436, 44.46019907551825, 45.22870997929842, 45.957519212772596, 46.663641777500494, 47.361256841448544, 48.10424524288915, 48.81036780761705, 49.50231120437853, 50.18291126676687, 50.8266454924425, 51.46470805093158, 52.11127811020049, 52.72098233275672, 53.32217905453308, 53.93471911068259, 54.53875166605224, 55.122933386268905, 55.68442843773928, 56.27995349232907, 56.838612710206164, 57.35757025777728, 57.93891614440065, 58.43802285681876, 59.00518957547571, 59.56952046053938, 60.096985508890334, 60.57624138615546, 61.081019765760146, 61.59714147973797, 62.08490485778295, 62.584011570201056, 63.142670788078156, 63.67580750361569, 64.13237671213453, 64.59178175424667, 64.99163429089981, 65.43118849785895, 65.87641437200466, 66.32731191333693, 66.73850778436321, 67.17522615772906, 67.66298953577405, 68.13090207866604, 68.57329211921845, 69.03553299492387, 69.42687803079716, 69.90046224087571, 70.291807276749, 70.70583898136857, 71.10285568442845, 71.44315571562261, 71.8174857499362, 72.16912911550351, 72.47823497717154, 72.78166917165301, 73.06808836457476, 73.39137339420923, 73.638090916825, 73.89898760740721, 74.11451096049684, 74.24779513938122, 74.41227348779174, 74.56540850182911, 74.65048350962766, 74.7157076822732, 74.74973768539263, 74.78660352210534, 74.79794685647848, 74.81779769163145, 74.84332019397102, 74.84899186115759, 74.87167852990386, 74.883021864277, 74.87735019709044, 74.88869353146357, 74.90287269943, 74.87451436349714, 74.8461560275643, 74.76108101976577, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.672064203272555, 28.644755125769223, 29.61744604826589, 30.556106967643142, 31.412528712815135, 32.30014462751326, 33.17641720783825, 34.06686895612966, 34.86940986302924, 35.728667441794514, 36.57091001900009, 37.37061509230639, 38.18449933357911, 38.947338570172704, 39.73286447551258, 40.5098828800726, 41.272722116666195, 42.02421801888665, 42.75586308595412, 43.459149817088736, 44.199302384936054, 44.88557411451096, 45.51229333862689, 46.18438590023538, 46.830955959504294, 47.45200351643366, 48.07021523976973, 48.648725292799824, 49.24425034738962, 49.845447069165985, 50.43246462297593, 51.03366134475229, 51.5724697274764, 52.213368119558744, 52.77486317102913, 53.29665655219351, 53.75606159430565, 54.23248163797749, 54.697558347276185, 55.156963389388316, 55.64189093384, 56.12965431188498, 56.563536851657545, 57.00592689220997, 57.490854436661664, 57.93324447721409, 58.40115702010606, 58.82369622550549, 59.21220542778549, 59.64608796755806, 60.020418001871654, 60.4032555369651, 60.82295890877123, 61.248333947763946, 61.65669398519695, 62.07072568981653, 62.43654822335025, 62.79386325610413, 63.17670079119757, 63.568045827070875, 63.96506253013074, 64.33939256444432, 64.70521509797805, 65.11357513541105, 65.49074100331794, 65.85939937044493, 66.19969940163911, 66.50880526330715, 66.82925445934833, 67.15821115616936, 67.47582451861724, 67.8189603834047, 68.13940957944588, 68.47403794345348, 68.86538297932677, 69.21702634489408, 69.54314720812182, 69.79553639792417, 70.15568726427132, 70.47046479312594, 70.79091398916712, 71.08584068286874, 71.3524090406375, 71.68987323823839, 71.99897909990642, 72.29674162720133, 72.5973399880895, 72.90077418257097, 73.229730879392, 73.51615007231376, 73.77421092930267, 74.05495845503786, 74.37824348467232, 74.7327226838329, 74.80078269007176, 74.86884269631058, 74.91705186739642, 74.92272353458299, 74.77526018773219, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.66922836967927, 28.616396789836372, 29.53520687406063, 30.459688625471458, 31.384170376882288, 32.28312962595355, 33.15373053909197, 34.01298811785725, 34.8779173638091, 35.74284660976094, 36.619119190085925, 37.447182599325075, 38.29226101012393, 39.108981084989935, 39.939880327822365, 40.69137623004282, 41.44287213226328, 42.20571136885688, 42.98556560701018, 43.75124067719707, 44.431840739585404, 45.197515809772284, 45.92348920965318, 46.621104273601226, 47.33857017270226, 48.07305107336302, 48.8018603068372, 49.53350537390466, 50.20559793551315, 50.849332161188784, 51.50157388764427, 52.12262144457364, 52.73516150072314, 53.33635822249951, 53.963077446615436, 54.58979667073136, 55.17397839094802, 55.75816011116468, 56.37920766809403, 56.969061055497264, 57.51070527181466, 58.120409494370875, 58.684740379434544, 59.30295210277061, 59.850267986274574, 60.43161387289794, 60.99310892436832, 61.560275643025264, 62.09057652496952, 62.56132490145478, 63.03490911153333, 63.568045827070875, 63.999092533250156, 64.51521424722797, 64.9008876159147, 65.35462099084026, 65.77716019623969, 66.15999773133312, 66.55134276720642, 66.99656864135213, 67.38224201003885, 67.79627371465843, 68.22164875365114, 68.66687462779683, 69.08090633241642, 69.49210220344271, 69.92314890962199, 70.27479227518928, 70.65479397668946, 71.0744973484956, 71.48569321952188, 71.82882908430935, 72.22868162096248, 72.60868332246264, 72.95749085443666, 73.2637608825114, 73.52182173950033, 73.83376343476165, 74.086152624564, 74.30451181124693, 74.50018432918357, 74.70720018149335, 74.87451436349714, 74.97376853926212, 75.05317187987409, 75.1240677197062, 75.13257522048606, 75.14959022204577, 75.15809772282563, 75.15809772282563, 75.16093355641891, 75.14675438845248, 75.14108272126592, 75.11839605251964, 75.11556021892636, 75.10988855173979, 75.08720188299351, 75.03048521112781, 74.85466352834416, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.626690865779995, 28.548336783597538, 29.478490202194934, 30.374613617672917, 31.270737033150898, 32.14700961347589, 32.97507302271503, 33.83149476788702, 34.66522984431274, 35.479114085585465, 36.27598332529847, 37.08986756657119, 37.88673680628421, 38.672262711624086, 39.443609448997535, 40.14689618013215, 40.861526245639915, 41.570484643961095, 42.27093554150243, 42.996908941383325, 43.68601650455152, 44.40064657005927, 45.1209483027536, 45.78453336358223, 46.47931259393699, 47.18259932507161, 47.83200521793381, 48.50126194594901, 49.181862008337355, 49.82843206760628, 50.517539630774465, 51.1981396931628, 51.86456058758473, 52.52247398122678, 53.14919320534271, 53.75889742789893, 54.396959986388005, 55.02084337691065, 55.60786093072059, 56.2742818251425, 56.963389388310695, 57.49369027025495, 58.10906615999774, 58.70459121458753, 59.277429600431056, 59.83041715112157, 60.391912202591946, 60.947735586875766, 61.50355897115958, 62.01684485154411, 62.55565323426821, 63.1171482857386, 63.61909083174999, 64.12670504494797, 64.56909508550038, 65.0682017979185, 65.57865184470975, 66.09477355868758, 66.56268610157956, 67.0759719819641, 67.55806369282251, 68.00045373337494, 68.4683662762669, 68.8795621472932, 69.22836967926722, 69.57434137764797, 69.9401639111817, 70.32300144627513, 70.72285398292829, 71.07733318208888, 71.39494654453678, 71.72106740776452, 72.01315826787284, 72.30241329438789, 72.6171908232425, 72.92062501772396, 73.16167087315317, 73.40555256217564, 73.69764342228397, 73.96421178005275, 74.12301846127669, 74.2903326432805, 74.47182599325072, 74.63630434166124, 74.77809602132547, 74.88585769787028, 74.95958937129569, 75.03615687831437, 75.07585854862036, 75.1269035532995, 75.15242605563905, 75.1269035532995, 75.11272438533308, 75.07869438221365, 75.02481354394124, 74.96242520488899, 74.85749936193744, 74.75824518617247, 74.62496100728809, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.65788503530613, 28.596545954683382, 29.523863539687493, 30.43983779031847, 31.352976207356154, 32.260442957207275, 33.11686470237927, 33.96761478036469, 34.81552902475683, 35.654935768369114, 36.485835011201544, 37.32807758840712, 38.141961829679836, 38.930323568613, 39.75555114425886, 40.53256954881888, 41.281229617446044, 42.05257635481951, 42.78138558829368, 43.48750815302158, 44.22482488727562, 44.91960411763038, 45.63423418313813, 46.394237586138445, 47.077673482120076, 47.792303547627824, 48.45305277486317, 49.13081700365823, 49.7688795621473, 50.398434619856516, 51.02515384397244, 51.6263505657488, 52.253069789864725, 52.83725151008139, 53.35337322405922, 53.909196608343024, 54.476363326999966, 55.06054504721662, 55.630547599466865, 56.17502764937754, 56.72234353288149, 57.21010691092647, 57.72055695771772, 58.27354450840825, 58.758472052859936, 59.36250460822959, 59.83325298471486, 60.38907636899866, 60.902362249383216, 61.38161812664833, 61.8892323398463, 62.34296571477186, 62.8562515951564, 63.23341746306327, 63.71834500751496, 64.15506338088082, 64.57760258628025, 64.97178345574682, 65.37163599239997, 65.76298102827327, 66.15716189773984, 66.57119360235941, 67.0022403085387, 67.30851033661345, 67.6714970365539, 67.99478206618836, 68.3776196012818, 68.72926296684912, 69.03269716133057, 69.40419136205087, 69.7898647307376, 70.13867226271162, 70.56688313529762, 70.91852650086493, 71.26733403283896, 71.59629072965998, 71.91390409210787, 72.24002495533563, 72.58883248730965, 72.8554008450784, 73.209880044239, 73.5246575730936, 73.8054050988288, 74.05495845503786, 74.2874968096872, 74.55973683464254, 74.76958852054561, 74.91138020020985, 75.05884354706066, 75.12973938689278, 75.13541105407936, 75.14108272126592, 75.12973938689278, 75.11556021892636, 75.05600771346738, 74.9766043728554, 74.89152936505685, 74.78093185491875, 74.6448118424411, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.66922836967927, 28.616396789836372, 29.53520687406063, 30.459688625471458, 31.384170376882288, 32.28312962595355, 33.15373053909197, 34.01298811785725, 34.8779173638091, 35.74284660976094, 36.619119190085925, 37.447182599325075, 38.29226101012393, 39.108981084989935, 39.939880327822365, 40.69137623004282, 41.44287213226328, 42.20571136885688, 42.98556560701018, 43.75124067719707, 44.431840739585404, 45.197515809772284, 45.92348920965318, 46.621104273601226, 47.33857017270226, 48.07305107336302, 48.8018603068372, 49.53350537390466, 50.20559793551315, 50.849332161188784, 51.50157388764427, 52.12262144457364, 52.73516150072314, 53.33635822249951, 53.963077446615436, 54.58979667073136, 55.17397839094802, 55.75816011116468, 56.37920766809403, 56.969061055497264, 57.51070527181466, 58.120409494370875, 58.684740379434544, 59.30295210277061, 59.850267986274574, 60.43161387289794, 60.99310892436832, 61.560275643025264, 62.09057652496952, 62.56132490145478, 63.03490911153333, 63.568045827070875, 63.999092533250156, 64.51521424722797, 64.9008876159147, 65.35462099084026, 65.77716019623969, 66.15999773133312, 66.55134276720642, 66.99656864135213, 67.38224201003885, 67.79627371465843, 68.22164875365114, 68.66687462779683, 69.08090633241642, 69.49210220344271, 69.92314890962199, 70.27479227518928, 70.65479397668946, 71.0744973484956, 71.48569321952188, 71.82882908430935, 72.22868162096248, 72.60868332246264, 72.95749085443666, 73.2637608825114, 73.52182173950033, 73.83376343476165, 74.086152624564, 74.30451181124693, 74.50018432918357, 74.70720018149335, 74.87451436349714, 74.97376853926212, 75.05317187987409, 75.1240677197062, 75.13257522048606, 75.14959022204577, 75.15809772282563, 75.15809772282563, 75.16093355641891, 75.14675438845248, 75.14108272126592, 75.11839605251964, 75.11556021892636, 75.10988855173979, 75.08720188299351, 75.03048521112781, 74.85466352834416, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2027, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.65788503530613, 28.576695119530388, 29.5040127045345, 30.451181124691608, 31.37282704250915, 32.24626378924085, 33.1480588719054, 33.99030144911097, 34.87224569662252, 35.7315032753878, 36.5680741854068, 37.37345092589967, 38.19017100076568, 38.984204406885404, 39.8037603153447, 40.61480872302413, 41.38899129399087, 42.168845532144175, 42.96004310467062, 43.666165669398524, 44.40348240365255, 45.126619969940165, 45.85826503700763, 46.55020843376911, 47.25065933131044, 47.922751892918924, 48.59200862093412, 49.27544451691575, 49.89365624025183, 50.56007713467374, 51.18396052519638, 51.87873975555115, 52.573518985905906, 53.21725321158155, 53.843972435697474, 54.479199160593254, 55.02367921050393, 55.66174176899299, 56.31114766185521, 56.88115021410544, 57.490854436661664, 58.09205115843803, 58.64503870912855, 59.20936959419221, 59.714147973796905, 60.24161302214785, 60.825794742364515, 61.3334089555625, 61.886396506253014, 62.4280407225704, 62.89595326546238, 63.324164138048374, 63.792076680940355, 64.2287950543062, 64.75909593625046, 65.292232651788, 65.77716019623969, 66.22522190397868, 66.69880611405723, 67.19224115928877, 67.58925786234865, 68.0770212403936, 68.50239627938632, 68.92777131837904, 69.32195218784562, 69.75016306043162, 70.15568726427132, 70.52150979780507, 70.91569066727165, 71.30136403595837, 71.72673907495108, 72.13226327879082, 72.52927998185066, 72.81286334117914, 73.14182003800018, 73.4509258996682, 73.7855542636758, 74.04361512066473, 74.27331764172078, 74.52287099792984, 74.67317017837394, 74.80929019085161, 74.91138020020985, 74.9766043728554, 75.03332104472109, 75.10988855173979, 75.13257522048606, 75.1269035532995, 75.13541105407936, 75.13824688767264, 75.13824688767264, 75.13541105407936, 75.13824688767264, 75.1269035532995, 75.12123188611292, 75.09854521736665, 75.05600771346738, 74.99078354082182, 74.84048436037773, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2027, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.63236253296657, 28.59087428749681, 29.532371040467346, 30.451181124691608, 31.37282704250915, 32.2746221251737, 33.15373053909197, 34.01582395145053, 34.860902362249384, 35.71165244023481, 36.54255168306724, 37.39897342823924, 38.22420100388509, 39.052264413124234, 39.8037603153447, 40.572271219124865, 41.34361795649831, 42.14048719621133, 42.835266426566086, 43.53855315770071, 44.230496554462185, 44.9167682840371, 45.57468167767915, 46.31767007911976, 47.02946431103423, 47.74125854294871, 48.40767943737062, 49.079771998979105, 49.717834557468166, 50.37291211751695, 50.97410883929331, 51.63485806652866, 52.295607293764, 52.936505685846356, 53.46113490060404, 54.11337662705953, 54.76561835351501, 55.37815840966452, 55.95099679550803, 56.50114851260528, 57.0739868984488, 57.661004452258744, 58.2083203357627, 58.849218727845056, 59.3596687746363, 59.87011882142757, 60.42877803930465, 60.98460142358846, 61.41848396336103, 61.96012817967842, 62.4280407225704, 62.969684938887795, 63.42908998099992, 63.93670419419788, 64.4386467402093, 64.87536511357514, 65.34044182287384, 65.72327935796729, 66.20537106882568, 66.69880611405723, 67.13552448742308, 67.57507869438221, 68.06851373961376, 68.55344128406546, 68.97881632305817, 69.36165385815161, 69.77284972917789, 70.20389643535718, 70.59807730482375, 70.9809148399172, 71.39778237813006, 71.69270907183167, 72.01599410146613, 72.36196579984686, 72.73629583416046, 73.05390919660834, 73.34600005671668, 73.57003091058617, 73.81391259960866, 74.01809261832516, 74.24212347219465, 74.38958681904546, 74.53988599948956, 74.63630434166124, 74.74406601820604, 74.82063352522475, 74.87735019709044, 74.90003686583671, 74.92839520176956, 74.95675353770241, 74.942574369736, 74.9198877009897, 74.90003686583671, 74.85749936193744, 74.84899186115759, 74.8121260244449, 74.75540935257919, 74.71854351586649, 74.68451351274707, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.646541700932993, 28.58520262031024, 29.467146867821796, 30.357598616113208, 31.25939369877776, 32.14984544706917, 33.03178969458072, 33.8343306014803, 34.66239401071945, 35.49329325355188, 36.32986416357088, 37.15225590562346, 37.90375180784392, 38.62121770694496, 39.3954002779117, 40.16391118169186, 40.93525791906531, 41.70660465643876, 42.41839888835324, 43.093327283555006, 43.84198735218218, 44.562289084876504, 45.24288914726484, 45.87244420497405, 46.54170093298926, 47.21379349459774, 47.87737855542637, 48.47290361001617, 49.07693616538582, 49.7121628902816, 50.31903127924453, 50.92306383461419, 51.510081388424126, 52.04888977114823, 52.627399824178326, 53.21158154439498, 53.80427076539149, 54.33173581374244, 54.85920086209342, 55.41218841278395, 55.931145960355046, 56.4047301704336, 56.92935938519128, 57.40294359526983, 57.8538411366021, 58.31324617871423, 58.758472052859936, 59.23205626293849, 59.64608796755806, 60.184896350282166, 60.63012222442787, 61.046989762640735, 61.475200635226734, 61.94027734452543, 62.365652383518146, 62.76266908657799, 63.171029124011, 63.58222499503729, 63.96506253013074, 64.28267589257864, 64.61730425658622, 65.04551512917223, 65.39999432833281, 65.73178685874713, 66.07775855712788, 66.4435810906616, 66.80373195700878, 67.14119615460964, 67.5041828545501, 67.80478121543828, 68.14508124663246, 68.41732127158778, 68.74627796840882, 69.06672716444999, 69.3247880214389, 69.6026997135808, 69.91747724243542, 70.16703059864447, 70.4222556220401, 70.72001814933499, 70.97240733913735, 71.22479652893968, 71.46867821796216, 71.70688823979809, 71.97629243116015, 72.22017412018262, 72.44420497405213, 72.67957916229476, 72.89510251538441, 73.09077503332104, 73.3233133879704, 73.5274934066869, 73.68346425431756, 73.8791367722542, 74.03227178629159, 74.1598842979894, 74.28182514250064, 74.40943765419846, 74.50302016277686, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.66922836967927, 28.616396789836372, 29.555057709213628, 30.434166123131895, 31.324617871423303, 32.20939795252814, 33.11402886878598, 33.99597311629754, 34.826872359129965, 35.63792076680941, 36.46882000964183, 37.28554008450784, 38.09658849218728, 38.93315940220628, 39.72152114113944, 40.48152454413975, 41.233020446360214, 41.94765051186796, 42.727504750021275, 43.44497064912231, 44.19646655134277, 44.94229078637666, 45.688115021410546, 46.388565918951876, 47.10319598445963, 47.78946771403454, 48.47290361001617, 49.15633950599779, 49.8426112355727, 50.48067379406177, 51.14993052207697, 51.82202308368545, 52.41754813827525, 53.05844653035761, 53.69367325525338, 54.30054164421632, 54.913081700365815, 55.534129257295184, 56.135325979071546, 56.77338853756062, 57.292346085131726, 57.85100530300882, 58.4493661911919, 58.9513087372033, 59.54399795819981, 60.099821342483615, 60.60459972208832, 61.06967643138701, 61.58579814536483, 62.116099027309076, 62.57266823582792, 63.017894109973625, 63.491478320052174, 63.976405864503874, 64.46416924254885, 64.95193262059382, 65.38297932677311, 65.88208603919122, 66.35567024926978, 66.80940362419534, 67.27731616708732, 67.70836287326661, 68.17060374897201, 68.59314295437144, 69.04404049570371, 69.45523636673, 69.8664322377563, 70.27762810878258, 70.62076397357004, 71.0546465133426, 71.44599154921589, 71.79479908118992, 72.14927828035052, 72.45838414201855, 72.81286334117914, 73.09077503332104, 73.43107506451521, 73.69764342228397, 73.91316677537363, 74.15704846439611, 74.35272098233277, 74.50018432918357, 74.62779684088137, 74.7355585174262, 74.82914102600459, 74.87451436349714, 74.90570853302329, 74.90570853302329, 74.9198877009897, 74.90854436661657, 74.90287269943, 74.90570853302329, 74.90287269943, 74.90287269943, 74.88585769787028, 74.863171029124, 74.85749936193744, 74.83481269319117, 74.76958852054561, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.63236253296657, 28.559680117970682, 29.458639367041943, 30.363270283299777, 31.239542863624763, 32.093128775203475, 32.97790885630831, 33.822987267107166, 34.64254317556646, 35.4904574199586, 36.31001332841789, 37.10971840172419, 37.93494597737005, 38.68927771318379, 39.49749028726994, 40.271672858236684, 41.077049598729545, 41.80585883220373, 42.53183223208462, 43.266313132745374, 43.938405694353854, 44.60766242236905, 45.30811331991039, 46.04259422057114, 46.74020928451918, 47.3697643422284, 48.00782690071747, 48.66006862717296, 49.31798202081502, 49.981567081643654, 50.639480475285715, 51.29739386892778, 51.972322264129545, 52.601877321838764, 53.17187987408899, 53.71352409040637, 54.28919830984318, 54.88755919802626, 55.44905424949664, 56.03890763689987, 56.569208518844114, 57.17607690780705, 57.757422794430425, 58.35861951620679, 58.96832373876302, 59.578027961319236, 60.19907551824859, 60.76340640331226, 61.39863312820803, 61.991322349204545, 62.5584890678615, 63.05192411309304, 63.54252332473131, 64.08983920823526, 64.59461758783995, 65.07103763151179, 65.55596517596346, 66.02954938604204, 66.557014434393, 67.0391061452514, 67.58075036156879, 68.03448373649434, 68.43717210674079, 68.82851714261406, 69.24822051442021, 69.66225221903979, 70.06494058928622, 70.45061395797293, 70.82494399228653, 71.19927402660012, 71.53390239060772, 71.8174857499362, 72.18614411706321, 72.50659331310439, 72.77032583727987, 73.05107336301506, 73.35167172390325, 73.60122508011231, 73.83092760116836, 74.06063012222444, 74.23928763860137, 74.38958681904546, 74.52287099792984, 74.67600601196722, 74.72421518305306, 74.82063352522475, 74.86033519553072, 74.90287269943, 74.90287269943, 74.90570853302329, 74.89720103224343, 74.90854436661657, 74.87735019709044, 74.86033519553072, 74.81496185803817, 74.75540935257919, 74.69302101352693, 74.62779684088137, 74.56540850182911, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.65788503530613, 28.605053455463235, 29.5408785412472, 30.43133028953861, 31.31043870345688, 32.1810396165953, 33.06014803051357, 33.87970393897286, 34.758812352891134, 35.550009925417584, 36.34120749794402, 37.143748404843606, 37.9264384765902, 38.70062104755693, 39.52301278960951, 40.26316535745683, 41.003317925304145, 41.71511215721861, 42.45242889147265, 43.15287978901398, 43.88736068967473, 44.57079658565636, 45.2684116496044, 45.985877548705446, 46.70901511499306, 47.39245101097468, 48.08155857414287, 48.708277798258806, 49.40872869580013, 50.106343759748185, 50.74724215183053, 51.3853047103196, 52.00068060006239, 52.6132206562119, 53.237104046734544, 53.79576326461164, 54.37427331764173, 54.9754700394181, 55.58233842838103, 56.166520148597684, 56.73935853444121, 57.30936108669143, 57.84816946941553, 58.33309701386723, 58.854890395031624, 59.388027110569155, 59.89564132376712, 60.4032555369651, 60.90803391656978, 61.37594645946176, 61.855202336726876, 62.2975923772793, 62.73431075064515, 63.19938745994384, 63.61909083174999, 64.06998837308227, 64.53790091597425, 64.96611178856024, 65.4396959986388, 65.89626520715764, 66.39820775316905, 66.80373195700878, 67.24328616396791, 67.65164620140091, 68.05717040524061, 68.4712021098602, 68.88806964807306, 69.29926551909934, 69.72180472449877, 70.09329892521907, 70.48180812749908, 70.87031732977908, 71.18509485863369, 71.51688738904801, 71.83166491790263, 72.16062161472365, 72.48957831154468, 72.79584833961943, 73.09077503332104, 73.36301505827637, 73.56152340980631, 73.79689759804894, 74.01525678473187, 74.2506309729745, 74.40376598701188, 74.5512293338627, 74.67317017837394, 74.75257351898591, 74.83481269319117, 74.89152936505685, 74.89720103224343, 74.90287269943, 74.89436519865014, 74.86033519553072, 74.82063352522475, 74.77526018773219, 74.69869268071349, 74.62212517369481, 74.55973683464254, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.66922836967927, 28.616396789836372, 29.555057709213628, 30.434166123131895, 31.324617871423303, 32.20939795252814, 33.11402886878598, 33.99597311629754, 34.826872359129965, 35.63792076680941, 36.46882000964183, 37.28554008450784, 38.09658849218728, 38.93315940220628, 39.72152114113944, 40.48152454413975, 41.233020446360214, 41.94765051186796, 42.727504750021275, 43.44497064912231, 44.19646655134277, 44.94229078637666, 45.688115021410546, 46.388565918951876, 47.10319598445963, 47.78946771403454, 48.47290361001617, 49.15633950599779, 49.8426112355727, 50.48067379406177, 51.14993052207697, 51.82202308368545, 52.41754813827525, 53.05844653035761, 53.69367325525338, 54.30054164421632, 54.913081700365815, 55.534129257295184, 56.135325979071546, 56.77338853756062, 57.292346085131726, 57.85100530300882, 58.4493661911919, 58.9513087372033, 59.54399795819981, 60.099821342483615, 60.60459972208832, 61.06967643138701, 61.58579814536483, 62.116099027309076, 62.57266823582792, 63.017894109973625, 63.491478320052174, 63.976405864503874, 64.46416924254885, 64.95193262059382, 65.38297932677311, 65.88208603919122, 66.35567024926978, 66.80940362419534, 67.27731616708732, 67.70836287326661, 68.17060374897201, 68.59314295437144, 69.04404049570371, 69.45523636673, 69.8664322377563, 70.27762810878258, 70.62076397357004, 71.0546465133426, 71.44599154921589, 71.79479908118992, 72.14927828035052, 72.45838414201855, 72.81286334117914, 73.09077503332104, 73.43107506451521, 73.69764342228397, 73.91316677537363, 74.15704846439611, 74.35272098233277, 74.50018432918357, 74.62779684088137, 74.7355585174262, 74.82914102600459, 74.87451436349714, 74.90570853302329, 74.90570853302329, 74.9198877009897, 74.90854436661657, 74.90287269943, 74.90570853302329, 74.90287269943, 74.90287269943, 74.88585769787028, 74.863171029124, 74.85749936193744, 74.83481269319117, 74.76958852054561, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2028, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.63803420015314, 28.58520262031024, 29.5040127045345, 30.419986955165474, 31.32178203783002, 32.19238295096844, 33.03746136176729, 33.93074894365199, 34.75030485211128, 35.589711595723564, 36.40359583699629, 37.22031591186229, 38.02285681876187, 38.84241272722117, 39.61375946459462, 40.41062870430763, 41.19331877605422, 41.95048634546124, 42.65093724300259, 43.43929898193574, 44.193630717749485, 44.94229078637666, 45.690950855003834, 46.42543175566458, 47.157076822732044, 47.82633355074724, 48.5040977795423, 49.23290701301648, 49.9532087457108, 50.59410713779315, 51.22082636190909, 51.85605308680488, 52.48560814451408, 53.12650653659643, 53.72486742477951, 54.33740748092902, 54.95278337067181, 55.534129257295184, 56.16084848141112, 56.73652270084793, 57.2413010804526, 57.80279613192298, 58.2820520091881, 58.82369622550549, 59.3993704449423, 59.91549215892012, 60.51101721350991, 61.02713892748775, 61.54042480787227, 62.01968068513739, 62.581175736607776, 63.06326744746619, 63.55953832629102, 64.05297337152255, 64.53506508238095, 64.98596262371325, 65.4198451634858, 65.8849218727845, 66.34716274848992, 66.78955278904233, 67.24328616396791, 67.7452287099793, 68.22448458724442, 68.67538212857669, 69.10642883475597, 69.5488188753084, 69.99971641664067, 70.37688228454755, 70.81076482432012, 71.18225902504041, 71.57076822732043, 71.9479340952273, 72.28539829282819, 72.6342058248022, 72.92062501772396, 73.24107421376513, 73.54734424183988, 73.79973343164224, 74.05495845503786, 74.2874968096872, 74.44630349091115, 74.58809517057539, 74.6816776791538, 74.76958852054561, 74.85749936193744, 74.85466352834416, 74.88018603068372, 74.883021864277, 74.91421603380314, 74.91705186739642, 74.93406686895614, 74.9198877009897, 74.91138020020985, 74.90570853302329, 74.91138020020985, 74.90570853302329, 74.883021864277, 74.87735019709044, 74.80078269007176, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2028, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.63803420015314, 28.588038453903525, 29.478490202194934, 30.360434449706492, 31.282080367524035, 32.17820378300201, 33.094178033632986, 33.9846297819244, 34.849559027876246, 35.70598077304824, 36.53404418228738, 37.36494342511982, 38.244051839038086, 39.0239060771914, 39.8406261520574, 40.6204803902107, 41.40033462836401, 42.12063636105834, 42.89481893202507, 43.6888523381448, 44.42900490599212, 45.13512747072002, 45.88662337294048, 46.54737260017583, 47.21095766100446, 47.86319938745994, 48.49275444516916, 49.17902617474407, 49.89082040665854, 50.56291296826703, 51.20948302753594, 51.847545586025014, 52.52247398122678, 53.16337237330914, 53.73337492555936, 54.40830332076114, 55.097410883929335, 55.75248844397811, 56.322490996228346, 56.8896577148853, 57.41428692964297, 57.98145364829992, 58.53727703258373, 59.04772707937498, 59.578027961319236, 60.122508011229904, 60.6244505572413, 61.10087060091314, 61.6311714828574, 62.12744236168221, 62.63789240847347, 63.13416328729831, 63.621926665343274, 64.047301704336, 64.50103507926156, 64.96044012137368, 65.47089016816494, 65.93313104387035, 66.42089442191532, 66.86612029606103, 67.32836117176643, 67.7650795451323, 68.2273204208377, 68.74344213481554, 69.16598134021497, 69.64807305107337, 70.0706122564728, 70.49598729546551, 70.8674814961858, 71.23330402971955, 71.61614156481298, 71.98479993194, 72.31375662876103, 72.58599665371636, 72.88375918101127, 73.209880044239, 73.44525423248164, 73.64659841760486, 73.8819726058475, 74.09182429175057, 74.22794430422823, 74.40660182060516, 74.55406516745597, 74.65899101040752, 74.71854351586649, 74.78943935569862, 74.84048436037773, 74.86884269631058, 74.883021864277, 74.89152936505685, 74.88585769787028, 74.90570853302329, 74.89720103224343, 74.89152936505685, 74.88018603068372, 74.86884269631058, 74.84899186115759, 74.82063352522475, 74.76391685335905, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.65221336811956, 28.60788928905652, 29.557893542806912, 30.462524459064745, 31.361483708136006, 32.21506961971472, 33.08283469925985, 33.916569775685566, 34.747469018518, 35.53015909026458, 36.386580835436575, 37.146584238436894, 37.91793097581034, 38.68927771318379, 39.41808694665797, 40.141224512945584, 40.87854124719962, 41.635708816606645, 42.37869721804724, 43.11034228511471, 43.80512151546947, 44.47154240989139, 45.160649973059584, 45.81005586592179, 46.453790091597426, 47.10319598445963, 47.76394521169498, 48.39633610299748, 49.023055327113404, 49.646938717636054, 50.27365794175198, 50.840824660408934, 51.49873805405099, 52.029038935995246, 52.576354819499194, 53.12650653659643, 53.65964325213397, 54.229645804384205, 54.68621501290305, 55.28457590108613, 55.897115957235634, 56.40189433684031, 56.861299378952445, 57.32637608825114, 57.85667697019539, 58.33026118027394, 58.77265122082637, 59.217877094972074, 59.66310296911777, 60.122508011229904, 60.57907721974875, 60.9760939228086, 61.43833479851403, 61.880724839066445, 62.26923404134646, 62.711624081898876, 63.11431245214531, 63.5255083231716, 63.92819669341803, 64.28551172617192, 64.71655843235119, 65.02850012761252, 65.36880015880668, 65.74596602671356, 66.17701273289283, 66.53149193205343, 66.8717919632476, 67.1865694921022, 67.57791452797551, 67.88985622323682, 68.20179791849814, 68.49956044579304, 68.83986047698721, 69.17448884099483, 69.47792303547628, 69.80687973229732, 70.13867226271162, 70.47330062671922, 70.71434648214843, 70.94688483679778, 71.21912486175312, 71.48285738592858, 71.74658991010408, 72.01315826787284, 72.25703995689533, 72.49524997873125, 72.75047500212688, 72.9688341888098, 73.17301420752631, 73.38286589342937, 73.53600090746674, 73.7118225902504, 73.95003261208632, 74.13152596205654, 74.23645180500809, 74.33287014717976, 74.39525848623202, 74.4576468252843, 74.54272183308284, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.6635567024927, 28.62206845702294, 29.549386042027052, 30.48237529421774, 31.39834954484871, 32.294472960326694, 33.21044721095766, 34.07537645690951, 34.94881320364121, 35.816578283186345, 36.675835861951626, 37.48688426963106, 38.3347985140232, 39.1515185888892, 39.92853699344923, 40.75660040268837, 41.52227547287525, 42.29078637665542, 43.064968947622155, 43.79661401468962, 44.51407991379066, 45.22870997929842, 45.92348920965318, 46.660805943907214, 47.310211836769426, 48.02200606868389, 48.688426963105805, 49.41156452939342, 50.06664208944219, 50.704704647931266, 51.348438873606895, 51.95814309616311, 52.596205654652195, 53.25411904829424, 53.86382327085046, 54.54158749964553, 55.13994838782861, 55.7354734424184, 56.34801349856791, 56.971896889090544, 57.522048606187795, 58.0410061537589, 58.63085954116213, 59.16966792388622, 59.714147973796905, 60.292658026827, 60.86266057907722, 61.36460312508862, 61.90341150781272, 62.450727391316676, 62.98953577404078, 63.47162748489919, 63.92536085982474, 64.48402007770184, 65.01999262683266, 65.51342767206421, 65.99835521651589, 66.49746192893402, 66.96821030541929, 67.40776451237842, 67.85582622011741, 68.31523126222953, 68.77180047074837, 69.20851884411422, 69.61687888154724, 70.03658225335337, 70.38255395173411, 70.7370331508947, 71.11703485239487, 71.52823072342115, 71.85718742024218, 72.2059949522162, 72.51793664747753, 72.87525168023141, 73.20420837705244, 73.4736125684145, 73.73734509258996, 73.97555511442589, 74.26481014094092, 74.46899015965744, 74.65615517681424, 74.82630519241131, 74.92839520176956, 75.0701868814338, 75.15809772282563, 75.22332189547117, 75.2573518985906, 75.24884439781074, 75.25451606499732, 75.24884439781074, 75.25168023140402, 75.24600856421746, 75.24600856421746, 75.2375010634376, 75.23466522984432, 75.19779939313162, 75.17227689079205, 75.08436604940023, 74.83764852678445, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.64370586733971, 28.573859285937104, 29.489833536568078, 30.428494455945327, 31.338797039389732, 32.27178629158041, 33.187760542211386, 34.07254062331623, 34.886424864588946, 35.68896577148853, 36.46031250886198, 37.26285341576157, 38.017185151575305, 38.771516887389055, 39.537191957575935, 40.29152369338968, 41.008989592490714, 41.737798825964894, 42.41839888835324, 43.133028953860986, 43.81362901624933, 44.531094915350366, 45.21169497773871, 45.88095170575391, 46.57289510251539, 47.284689334429856, 47.922751892918924, 48.617531123273686, 49.306638686441886, 49.916342908998104, 50.56574880186031, 51.26052803221507, 51.92694892663698, 52.55650398434619, 53.18605904205542, 53.75889742789893, 54.41964665513428, 55.06338088080991, 55.68442843773928, 56.328162663414915, 56.94921022034427, 57.564586110087056, 58.13742449593058, 58.69891954740096, 59.23205626293849, 59.756685477696166, 60.357882199472535, 60.89101891501008, 61.44684229929388, 61.97714318123813, 62.53296656552194, 63.03490911153333, 63.61058333097014, 64.09267504182854, 64.61446842299294, 65.0852167994782, 65.5673085103366, 66.03805688682188, 66.61656693985198, 67.05895698040439, 67.51552618892323, 67.99478206618836, 68.45418710830049, 68.9164279840059, 69.33329552221876, 69.73881972605848, 70.10464225959221, 70.51300229702522, 70.87031732977908, 71.2786773672121, 71.63882823355925, 71.94226242804072, 72.23435328814905, 72.55763831778351, 72.91211751694411, 73.19570087627258, 73.45943340044808, 73.68346425431756, 73.91600260896692, 74.16555596517597, 74.35555681592605, 74.47182599325072, 74.63630434166124, 74.80361852366504, 74.94824603692255, 75.07869438221365, 75.1807843915719, 75.23182939625104, 75.26869523296374, 75.26585939937046, 75.27153106655702, 75.24884439781074, 75.19212772594504, 75.13541105407936, 75.07869438221365, 75.00212687519496, 74.88018603068372, 74.74123018461276, 74.6079460057284, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.660720868899414, 28.613560956243088, 29.56356520999348, 30.476703627031164, 31.387006210475572, 32.30014462751326, 33.193432209397955, 34.06686895612966, 34.92045486770836, 35.73717494257437, 36.56240251822023, 37.37912259308624, 38.181663499985824, 38.98987607407198, 39.76122281144543, 40.504211212886034, 41.306752119785614, 42.066755522785925, 42.85511726171908, 43.6151206647194, 44.34392989819358, 45.06990329807447, 45.76468252842923, 46.44528259081757, 47.09468848367978, 47.76678104528826, 48.40200777018405, 49.10529450131867, 49.75186456058759, 50.39559878626323, 51.02515384397244, 51.63769390012194, 52.23038312111845, 52.772027337435844, 53.370388225618925, 53.97725661458186, 54.6011400051045, 55.19099339250773, 55.77517511272438, 56.31114766185521, 56.835776876612876, 57.33204775543772, 57.85100530300882, 58.39832118651278, 58.85205456143834, 59.422057113688574, 59.94385049485296, 60.49116637835692, 60.98743725718174, 61.52340980631257, 62.053710688256814, 62.51878739755552, 62.9866999404475, 63.482970819272325, 63.9537191957576, 64.42163173864958, 64.83849927686244, 65.28656098460142, 65.74880186030684, 66.21955023679212, 66.65343277656469, 67.12701698664323, 67.52686952329637, 67.93522956072937, 68.3577687661288, 68.69239713013641, 69.08657799960298, 69.5289680401554, 69.91180557524885, 70.31732977908857, 70.67748064543574, 71.04330317896945, 71.38360321016364, 71.68987323823839, 71.97062076397357, 72.30241329438789, 72.63704165839549, 72.92062501772396, 73.19853670986586, 73.49629923716077, 73.81107676601538, 74.07764512378414, 74.29316847687377, 74.48600516121714, 74.67033434478066, 74.81496185803817, 74.99078354082182, 75.16093355641891, 75.2573518985906, 75.27153106655702, 75.2573518985906, 75.24884439781074, 75.21765022828461, 75.17794855797861, 75.10421688455322, 75.01630604316138, 74.89720103224343, 74.74123018461276, 74.62779684088137, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.6635567024927, 28.62206845702294, 29.549386042027052, 30.48237529421774, 31.39834954484871, 32.294472960326694, 33.21044721095766, 34.07537645690951, 34.94881320364121, 35.816578283186345, 36.675835861951626, 37.48688426963106, 38.3347985140232, 39.1515185888892, 39.92853699344923, 40.75660040268837, 41.52227547287525, 42.29078637665542, 43.064968947622155, 43.79661401468962, 44.51407991379066, 45.22870997929842, 45.92348920965318, 46.660805943907214, 47.310211836769426, 48.02200606868389, 48.688426963105805, 49.41156452939342, 50.06664208944219, 50.704704647931266, 51.348438873606895, 51.95814309616311, 52.596205654652195, 53.25411904829424, 53.86382327085046, 54.54158749964553, 55.13994838782861, 55.7354734424184, 56.34801349856791, 56.971896889090544, 57.522048606187795, 58.0410061537589, 58.63085954116213, 59.16966792388622, 59.714147973796905, 60.292658026827, 60.86266057907722, 61.36460312508862, 61.90341150781272, 62.450727391316676, 62.98953577404078, 63.47162748489919, 63.92536085982474, 64.48402007770184, 65.01999262683266, 65.51342767206421, 65.99835521651589, 66.49746192893402, 66.96821030541929, 67.40776451237842, 67.85582622011741, 68.31523126222953, 68.77180047074837, 69.20851884411422, 69.61687888154724, 70.03658225335337, 70.38255395173411, 70.7370331508947, 71.11703485239487, 71.52823072342115, 71.85718742024218, 72.2059949522162, 72.51793664747753, 72.87525168023141, 73.20420837705244, 73.4736125684145, 73.73734509258996, 73.97555511442589, 74.26481014094092, 74.46899015965744, 74.65615517681424, 74.82630519241131, 74.92839520176956, 75.0701868814338, 75.15809772282563, 75.22332189547117, 75.2573518985906, 75.24884439781074, 75.25451606499732, 75.24884439781074, 75.25168023140402, 75.24600856421746, 75.24600856421746, 75.2375010634376, 75.23466522984432, 75.19779939313162, 75.17227689079205, 75.08436604940023, 74.83764852678445, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2029, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.6635567024927, 28.616396789836372, 29.560729376400197, 30.488046961404308, 31.412528712815135, 32.30298046110654, 33.193432209397955, 34.08104812409608, 34.96015653801435, 35.81941411677963, 36.64464169242549, 37.469869268071356, 38.28091767575079, 39.04659274593767, 39.78958114737827, 40.54107704959873, 41.33511045571846, 42.11780052746505, 42.863624762498944, 43.60377733034626, 44.30706406148087, 45.05288829651476, 45.83274253466807, 46.57289510251539, 47.22797266256416, 47.93693106088534, 48.63738195842669, 49.32365368800159, 49.984402915236934, 50.6593313104387, 51.32575220486062, 51.99500893287582, 52.61889232339847, 53.29382071860023, 53.88083827241017, 54.533079998865674, 55.13427672064204, 55.721294274451985, 56.29980432748207, 56.89816521566515, 57.47383943510195, 58.04667782094547, 58.64503870912855, 59.18384709185265, 59.708476306610336, 60.19340385106202, 60.68116722910699, 61.214303944644534, 61.69639565550293, 62.14162152964864, 62.61520573972719, 63.1171482857386, 63.58789666222386, 64.06431670589569, 64.60028925502652, 65.06253013073193, 65.57298017752318, 66.02104188526218, 66.51447693049373, 67.0022403085387, 67.46731701783739, 67.94090122791594, 68.3776196012818, 68.86254714573349, 69.33896718940534, 69.76717806199133, 70.21523976973032, 70.58673397045061, 71.0347956781896, 71.43181238124947, 71.84584408586905, 72.21733828658934, 72.5803249865298, 72.90928168335083, 73.2467458809517, 73.49913507075405, 73.76570342852283, 73.99540594957887, 74.26481014094092, 74.46331849247086, 74.61078183932167, 74.75257351898591, 74.90287269943, 74.96242520488899, 75.03899271190765, 75.06451521424722, 75.07585854862036, 75.09287355018009, 75.09287355018009, 75.10138105095993, 75.12123188611292, 75.10138105095993, 75.08720188299351, 75.06735104784052, 75.02764937753452, 74.99078354082182, 74.95108187051585, 74.87167852990386, 74.71854351586649, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2029, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.64370586733971, 28.531321782037832, 29.427445197515812, 30.295210277060946, 31.174318690979213, 32.00805376740493, 32.83611717664407, 33.66701641947651, 34.54612483339478, 35.37702407622721, 36.20225165187307, 36.990613390806224, 37.813005132858805, 38.646740209284516, 39.52584862320279, 40.35107619884866, 41.139437937781814, 41.899441340782126, 42.633922241442875, 43.40243314522304, 44.17377988259649, 44.91393245044381, 45.66826418625755, 46.3091625783399, 47.020956810254376, 47.71290020701585, 48.416186938150474, 49.09395116694553, 49.743357059807735, 50.409777954229654, 51.070527181464996, 51.71709724073392, 52.36933896718941, 53.01874486005161, 53.62844908260783, 54.201287468451355, 54.864872529279985, 55.39233757763095, 55.97935513144089, 56.57771601962397, 57.10518106797493, 57.68652695459831, 58.23667867169555, 58.789666222386074, 59.322802937923605, 59.799222981595435, 60.33519553072626, 60.83146640955108, 61.37594645946176, 61.815500666420895, 62.33729404758529, 62.884609931089244, 63.38655247710064, 63.86013668717921, 64.3592433995973, 64.83566344326915, 65.3092476533477, 65.77148852905313, 66.23656523835182, 66.67895527890424, 67.17239032413578, 67.62328786546806, 68.04582707086749, 68.41164960440122, 68.83418880980065, 69.2312055128605, 69.64807305107337, 69.98553724867425, 70.34285228142812, 70.71718231574171, 71.13121402036128, 71.4800215523353, 71.83450075149591, 72.18614411706321, 72.41584663811928, 72.74196750134703, 72.9886850239628, 73.27226838329128, 73.51615007231376, 73.72883759181012, 73.91316677537363, 74.15137679720954, 74.30451181124693, 74.44913932450443, 74.59943850494854, 74.61361767291496, 74.66466267759408, 74.68451351274707, 74.69585684712021, 74.70720018149335, 74.71003601508663, 74.69585684712021, 74.69585684712021, 74.69018517993365, 74.67033434478066, 74.6448118424411, 74.58809517057539, 74.56824433542239, 74.54839350026941, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.65221336811956, 28.605053455463235, 29.56923687718005, 30.499390295777445, 31.375662876102435, 32.26895045798713, 33.165073873465104, 33.99030144911097, 34.84388736068967, 35.657771601962395, 36.485835011201544, 37.22031591186229, 37.97748348126932, 38.73465105067635, 39.443609448997535, 40.18659785043814, 40.94376541984517, 41.638544650199925, 42.41272722116666, 43.14153645464084, 43.84765901936875, 44.49990074582424, 45.20318747695885, 45.84125003544792, 46.54170093298926, 47.20528599381789, 47.86319938745994, 48.49842611235573, 49.09395116694553, 49.77738706292715, 50.406942120636366, 50.97410883929331, 51.51858888920399, 52.13112894535349, 52.743669001503, 53.31367155375323, 53.897853273969886, 54.45934832544026, 54.96129087145167, 55.50009925417576, 56.05308680486629, 56.628761024303095, 57.15055440546749, 57.60712361398633, 58.13458866233729, 58.63085954116213, 59.11295125202054, 59.578027961319236, 60.04310467061793, 60.46564387601736, 60.859824745483934, 61.32490145478264, 61.741768992995496, 62.10475569293594, 62.56983240223464, 62.9866999404475, 63.352522473981224, 63.76088251141423, 64.16073504806738, 64.58043841987353, 65.01715679323937, 65.4198451634858, 65.78283186342627, 66.15432606414655, 66.55985026798628, 66.92567280152001, 67.33403283895301, 67.66015370218076, 68.00328956696822, 68.35209709894224, 68.74060630122226, 69.02702549414401, 69.36165385815161, 69.69911805575248, 69.98270141508097, 70.26061310722287, 70.56688313529762, 70.83345149306639, 71.15390068910756, 71.42046904687633, 71.68987323823839, 71.9479340952273, 72.19748745143634, 72.43569747327227, 72.6909224966679, 72.90077418257097, 73.0822675325412, 73.29779088563083, 73.44525423248164, 73.59838924651902, 73.80824093242208, 73.93301761052662, 74.0492867878513, 74.14286929642968, 74.24212347219465, 74.31301931202677, 74.39809431982532, 74.40660182060516, 74.44063182372457, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.63803420015314, 28.582366786716957, 29.495505203754647, 30.425658622352042, 31.347304540169585, 32.21223378612143, 33.10268553441284, 34.01298811785725, 34.8949323653688, 35.74851827694751, 36.56240251822023, 37.353600090746674, 38.09375265859399, 38.87077106315402, 39.7101778067663, 40.51271871366588, 41.35779712446473, 42.14048719621133, 42.90899809999149, 43.70303150611122, 44.47437824348467, 45.17199330743272, 45.84692170263449, 46.61543260641466, 47.33573433910898, 48.053200238210025, 48.713949465445374, 49.380370359867285, 50.03261208632277, 50.67067464481184, 51.34560304001361, 51.99217309928253, 52.66426566089102, 53.302328219380094, 53.93188327708931, 54.58979667073136, 55.18248589172787, 55.73830927601169, 56.2941326602955, 56.88115021410544, 57.49652610384823, 58.07220032328504, 58.653546209908406, 59.1923545926325, 59.71131214020362, 60.28131469245385, 60.780421404871966, 61.319229787596065, 61.82967983438732, 62.320279046025576, 62.878938263902675, 63.380880809914075, 63.919689192638174, 64.36491506678388, 64.81864844170944, 65.25253098148201, 65.68924935484786, 66.09477355868758, 66.53999943283328, 66.95119530385958, 67.3567195076993, 67.78493038028527, 68.190454584125, 68.62433712389758, 69.02135382695744, 69.45523636673, 69.84090973541672, 70.271956441596, 70.74270481808126, 71.16240818988742, 71.49703655389501, 71.8742024218019, 72.25420412330205, 72.60584748886936, 72.94331168647024, 73.29779088563083, 73.59838924651902, 73.85361426991463, 74.12585429486998, 74.33003431358648, 74.54555766667612, 74.71854351586649, 74.87167852990386, 74.97944020644869, 75.05600771346738, 75.08436604940023, 75.14675438845248, 75.14675438845248, 75.15526188923234, 75.15242605563905, 75.15526188923234, 75.15526188923234, 75.15242605563905, 75.13824688767264, 75.12973938689278, 75.1269035532995, 75.09854521736665, 75.05884354706066, 74.863171029124, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.621019198593427, 28.551172617190822, 29.45013186626209, 30.380285284859486, 31.29625953549046, 32.17253211581544, 33.07999886566657, 33.927913110058704, 34.68508067946573, 35.5074724215183, 36.35255083231716, 37.180614241556306, 37.93778181096334, 38.76017355301591, 39.55420695913564, 40.30003119416953, 41.045855429203414, 41.79735133142387, 42.58287723676375, 43.28616396789837, 44.00930153418597, 44.76079743640643, 45.48109916910076, 46.1985650682018, 46.918866800896126, 47.56543686016504, 48.248872756146675, 48.920965317755154, 49.584550378583785, 50.2311204378527, 50.90604883305447, 51.54694722513683, 52.199188951592326, 52.874117346794094, 53.55188157558914, 54.204123302044636, 54.864872529279985, 55.51995008932876, 56.11547514391856, 56.69398519694864, 57.28100275075858, 57.90488614128123, 58.511754530244175, 59.12713041998696, 59.64608796755806, 60.19340385106202, 60.76624223690554, 61.32206562118936, 61.79848566486119, 62.38833905226442, 62.91580410061538, 63.380880809914075, 63.871480021552344, 64.30536256132491, 64.81014094092959, 65.3092476533477, 65.8111901993591, 66.36701358364292, 66.78388112185577, 67.27731616708732, 67.76791537872558, 68.19612625131157, 68.64418795905056, 69.06956299804328, 69.54031137452854, 69.91464140884214, 70.28329977596914, 70.72852565011485, 71.07733318208888, 71.38360321016364, 71.73808240932422, 72.01882993505941, 72.36196579984686, 72.7277883333806, 73.02838669426879, 73.27510421688456, 73.51898590590704, 73.72033009103026, 73.94152511130648, 74.17122763236253, 74.36122848311261, 74.52287099792984, 74.65899101040752, 74.82063352522475, 74.93973853614271, 75.03615687831437, 75.10988855173979, 75.13257522048606, 75.16660522360547, 75.16376939001219, 75.1269035532995, 75.09287355018009, 75.05600771346738, 74.99929104160168, 74.9198877009897, 74.82914102600459, 74.74690185179934, 74.65048350962766, 74.56540850182911, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.64370586733971, 28.57102345234382, 29.481326035788225, 30.397300286419192, 31.33028953860988, 32.17820378300201, 33.07999886566657, 33.95627144599155, 34.818364858350115, 35.66344326914897, 36.49150667838811, 37.26852508294813, 37.98315514845589, 38.76868105379577, 39.57122196069535, 40.407792870714346, 41.22451294558036, 42.02421801888665, 42.7586989195474, 43.49317982020815, 44.22198905368233, 44.928111618410234, 45.63707001673142, 46.31767007911976, 46.93304596886255, 47.69872103904943, 48.39066443581091, 49.05141366304625, 49.69231205512861, 50.30485211127811, 50.87201882993506, 51.478887218897995, 52.08575560786093, 52.68411649604401, 53.32217905453308, 53.98576411536171, 54.572781669171654, 55.17965005813459, 55.721294274451985, 56.2544309899895, 56.7592093695942, 57.33488358903101, 57.84816946941553, 58.44085869041204, 58.985338740322725, 59.52981879023339, 60.020418001871654, 60.50250971273006, 60.9590789212489, 61.50639480475286, 61.92893401015229, 62.39968238663756, 62.887445764682525, 63.34968664038794, 63.803420015313506, 64.27700422539205, 64.73924510109748, 65.15894847290362, 65.55880100955676, 65.95014604543006, 66.36984941723621, 66.78388112185577, 67.23477866318804, 67.6913478717069, 68.0770212403936, 68.45985877548706, 68.8823979808865, 69.25389218160677, 69.63105804951365, 69.92598474321528, 70.29747894393556, 70.67180897824916, 71.02061651022318, 71.38643904375692, 71.7976349147832, 72.12375577801096, 72.47823497717154, 72.79301250602614, 73.13898420440688, 73.43674673170179, 73.72600175821682, 73.99540594957887, 74.24212347219465, 74.43779599013129, 74.68451351274707, 74.83197685959789, 74.97093270566883, 75.07302271502708, 75.16093355641891, 75.15526188923234, 75.13541105407936, 75.10988855173979, 75.0900377165868, 75.03048521112781, 74.9766043728554, 74.86033519553072, 74.76675268695233, 74.67600601196722, 74.58809517057539, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.63803420015314, 28.582366786716957, 29.495505203754647, 30.425658622352042, 31.347304540169585, 32.21223378612143, 33.10268553441284, 34.01298811785725, 34.8949323653688, 35.74851827694751, 36.56240251822023, 37.353600090746674, 38.09375265859399, 38.87077106315402, 39.7101778067663, 40.51271871366588, 41.35779712446473, 42.14048719621133, 42.90899809999149, 43.70303150611122, 44.47437824348467, 45.17199330743272, 45.84692170263449, 46.61543260641466, 47.33573433910898, 48.053200238210025, 48.713949465445374, 49.380370359867285, 50.03261208632277, 50.67067464481184, 51.34560304001361, 51.99217309928253, 52.66426566089102, 53.302328219380094, 53.93188327708931, 54.58979667073136, 55.18248589172787, 55.73830927601169, 56.2941326602955, 56.88115021410544, 57.49652610384823, 58.07220032328504, 58.653546209908406, 59.1923545926325, 59.71131214020362, 60.28131469245385, 60.780421404871966, 61.319229787596065, 61.82967983438732, 62.320279046025576, 62.878938263902675, 63.380880809914075, 63.919689192638174, 64.36491506678388, 64.81864844170944, 65.25253098148201, 65.68924935484786, 66.09477355868758, 66.53999943283328, 66.95119530385958, 67.3567195076993, 67.78493038028527, 68.190454584125, 68.62433712389758, 69.02135382695744, 69.45523636673, 69.84090973541672, 70.271956441596, 70.74270481808126, 71.16240818988742, 71.49703655389501, 71.8742024218019, 72.25420412330205, 72.60584748886936, 72.94331168647024, 73.29779088563083, 73.59838924651902, 73.85361426991463, 74.12585429486998, 74.33003431358648, 74.54555766667612, 74.71854351586649, 74.87167852990386, 74.97944020644869, 75.05600771346738, 75.08436604940023, 75.14675438845248, 75.14675438845248, 75.15526188923234, 75.15242605563905, 75.15526188923234, 75.15526188923234, 75.15242605563905, 75.13824688767264, 75.12973938689278, 75.1269035532995, 75.09854521736665, 75.05884354706066, 74.863171029124, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2030, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.672064203272555, 28.63624762498937, 29.572072710773334, 30.485211127811024, 31.381334543289004, 32.27745795876698, 33.15373053909197, 34.018659785043816, 34.855230695062815, 35.69180160508182, 36.54538751666052, 37.376286759492956, 38.17032016561268, 38.97569690610555, 39.75271531066558, 40.51555454725917, 41.346453790091594, 42.137651362618044, 42.87780393046536, 43.64347900065224, 44.38079573490627, 45.095425800414034, 45.790205030768796, 46.453790091597426, 47.15991265632533, 47.84902021949352, 48.49559027876245, 49.17051867396422, 49.8596262371324, 50.54589796670732, 51.1442588548904, 51.81918725009217, 52.42038397186853, 53.04710319598447, 53.662479085727256, 54.29486997702975, 54.88472336443297, 55.48308425261605, 56.10980347673199, 56.69682103054192, 57.224286078892895, 57.765930295210275, 58.32175367949408, 58.91160706689731, 59.40220627853559, 59.91265632532683, 60.434449706491236, 60.93639225250263, 61.43833479851403, 61.95445651249185, 62.453563224909956, 62.96684910529451, 63.39505997788051, 63.83177835124635, 64.33088506366447, 64.79596177296317, 65.26671014944843, 65.74596602671356, 66.17701273289283, 66.65059694297139, 67.12418115304995, 67.56940702719565, 68.05149873805405, 68.49388877860648, 68.95045798712532, 69.41269886283072, 69.82389473385702, 70.26912060800272, 70.67464481184244, 71.0546465133426, 71.44031988202933, 71.90256075773473, 72.23718912174235, 72.63136999120891, 72.94331168647024, 73.25808921532484, 73.53316507387346, 73.81107676601538, 74.06346595581772, 74.31018347843349, 74.49734849559029, 74.65048350962766, 74.8121260244449, 74.883021864277, 74.98227604004197, 75.02481354394124, 75.03615687831437, 75.05317187987409, 75.04466437909424, 75.04466437909424, 75.04750021268752, 75.04182854550095, 75.04182854550095, 75.03048521112781, 75.01914187675467, 75.01063437597482, 74.9936193744151, 74.95391770410913, 74.80361852366504, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2030, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.59266086266058, 28.494455945325132, 29.393415194396393, 30.272523608314668, 31.17148285738593, 32.02506876896464, 32.91552051725605, 33.757763094461616, 34.60851317244704, 35.40821824575334, 36.188072483906645, 37.016135893145794, 37.841363468791656, 38.672262711624086, 39.50316195445651, 40.26600119105011, 41.05152709638998, 41.760485494711176, 42.55168306723762, 43.34571647335734, 44.07452570683152, 44.77214077077957, 45.46692000113433, 46.21274423616823, 46.984090973541676, 47.704392706236, 48.39917193659076, 49.07126449819925, 49.70649122309503, 50.37291211751695, 51.002467175226165, 51.66888806964808, 52.30127896095058, 52.92799818506651, 53.56606074355557, 54.15307829736551, 54.70039418086947, 55.32994923857868, 55.891444290049066, 56.48129767745229, 57.08533023282194, 57.695034455378156, 58.26787284122168, 58.809517057539075, 59.34265377307659, 59.90982049173355, 60.440121373677805, 60.94489975328248, 61.483708136006584, 61.945949011711996, 62.481921560842814, 63.03490911153333, 63.48864248645889, 64.0104358676233, 64.45282590817571, 64.92641011825427, 65.33760598928056, 65.78566769701955, 66.2223860703854, 66.68179111249752, 67.11850948586337, 67.53821285766952, 68.03164790290106, 68.47403794345348, 68.86254714573349, 69.31060885347249, 69.68777472137934, 70.1244930947452, 70.46479312593938, 70.84763066103281, 71.17658735785383, 71.51688738904801, 71.89405325695488, 72.27405495845505, 72.66539999432833, 72.91778918413068, 73.15032753878002, 73.39137339420923, 73.6040609137056, 73.8054050988288, 74.01525678473187, 74.1967501347021, 74.30451181124693, 74.38958681904546, 74.48600516121714, 74.58809517057539, 74.63346850806795, 74.6618268440008, 74.68451351274707, 74.72137934945977, 74.75540935257919, 74.76958852054561, 74.8121260244449, 74.83764852678445, 74.85182769475087, 74.84048436037773, 74.80645435725832, 74.78943935569862, 74.6816776791538, 74.47182599325072]}, {"task": "grefcoco", "expert": "groundingdino", "endpoint": "Qwen3.8", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [26.688029946402747, 27.666392536085983, 28.63624762498937, 29.580580211553187, 30.490882794997592, 31.43521538156141, 32.33984629781925, 33.230298046110654, 34.058361455349804, 34.8949323653688, 35.76553327850722, 36.551059183847094, 37.381958426679525, 38.204350168732105, 39.02957774437796, 39.78107364659842, 40.538241216005446, 41.301080452599045, 42.04974052122622, 42.7586989195474, 43.433627314749174, 44.18512321696963, 44.91109661685053, 45.557666676119446, 46.24961007288093, 46.94155346964241, 47.60513853047104, 48.27723109207952, 48.93514448572158, 49.553356209057654, 50.18574710036015, 50.79261548932309, 51.36545387516661, 51.94112809460341, 52.49978731248051, 53.109491535036724, 53.73337492555936, 54.3203924793693, 54.85636502850013, 55.41218841278395, 56.036071803306584, 56.61174602274339, 57.159061906247345, 57.655332785072176, 58.13742449593058, 58.67623287865469, 59.14698125513995, 59.61205796443865, 60.07713467373734, 60.55922638459575, 61.092363100133284, 61.61415648129768, 62.12744236168221, 62.541474066301795, 62.964013271701226, 63.440433315373056, 63.84595751921278, 64.24297422227264, 64.60879675580638, 65.06536596432522, 65.45671100019851, 65.81402603295238, 66.20253523523239, 66.53999943283328, 66.88029946402744, 67.26030116552761, 67.59209369594193, 68.00328956696822, 68.27552959192356, 68.598814621558, 68.92777131837904, 69.24538468082693, 69.58852054561439, 69.88628307290928, 70.18404560020419, 70.50449479624537, 70.81643649150668, 71.10285568442845, 71.3892748773502, 71.61614156481298, 71.8912174233616, 72.14927828035052, 72.43286163967899, 72.6909224966679, 72.90644584975755, 73.15316337237331, 73.3601792246831, 73.58704591214588, 73.73734509258996, 73.89898760740721, 74.11734679409012, 74.3640643167059, 74.53988599948956, 74.79794685647848, 74.90003686583671, 74.96809687207555, 74.96242520488899, 74.93973853614271, 74.87451436349714, 74.75540935257919, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.22910699600148, 67.42477951393812, 67.54388452485608, 67.6147803646882, 67.70552703967331, 67.75090037716586, 67.79627371465841, 67.81612454981142, 67.8388112185577, 67.8388112185577, 67.8388112185577, 67.8558262201174, 67.87000538808383, 67.87851288886368, 67.88702038964354, 67.88702038964354, 67.88702038964354, 67.8756770552704, 67.88702038964354, 67.88418455605026, 67.8955278904234, 67.86716955449053, 67.83597538496441, 67.79060204747185, 67.77642287950543, 67.73388537560616, 67.66582536936733, 67.6346311998412, 67.56089952641578, 67.46164535065083, 67.37940617644557, 67.31985367098659, 67.26880866630745, 67.14403198820294, 67.01358364291184, 66.92283696792673, 66.84343362731475, 66.72149278280351, 66.63358194141168, 66.53432776564671, 66.40104358676233, 66.25925190709809, 66.18835606726597, 66.02671355244874, 65.89342937356436, 65.73178685874713, 65.64104018376202, 65.45954683379179, 65.31208348694099, 65.11073930181777, 64.93208178544083, 64.77611093781017, 64.53222924878767, 64.37625840115702, 64.18058588322037, 64.03028670277628, 63.84312168561949, 63.678643337208975, 63.42341831381334, 63.21356662791027, 63.023565777160194, 62.756997419391425, 62.55565323426821, 62.33162238039872, 62.096248192156075, 61.86370983750673, 61.656693985196945, 61.45534980007373, 61.22848311261095, 60.993108924368315, 60.74639140175254, 60.44862887445765, 60.230269687774715, 59.926835493293254, 59.67161046989763, 59.42489294728185, 59.084592916087686, 58.81235289113235, 58.54578453336357, 58.27070867481496, 57.9672744803335, 57.67801945381845, 57.38592859371012, 57.068315231262225, 56.74503020162776, 56.45293934151944, 56.0133851345603, 55.66174176899299, 55.36397924169809, 54.90741003317925, 54.49054249496639, 54.048152454413966, 53.59441907948841, 53.157700706122554, 52.66142982729774, 52.14530811331991, 51.640529733715226, 51.09888551739784, 50.574256302640165, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.05328531321781, 67.17239032413578, 67.26313699912089, 67.39358534441199, 67.58358619516207, 67.73672120919944, 67.85299038652411, 67.93806539432265, 68.01463290134134, 68.06567790602047, 68.13940957944588, 68.21314125287128, 68.17627541615859, 68.210305419278, 68.20179791849814, 68.23866375521084, 68.23299208802428, 68.22448458724442, 68.18761875053173, 68.0997079091399, 68.00045373337493, 67.8926920568301, 67.81328871621814, 67.69134787170688, 67.56940702719564, 67.39358534441199, 67.28015200068059, 67.11000198508351, 66.93985196948643, 66.80656779060205, 66.7044777812438, 66.50596942971386, 66.25925190709809, 66.0153702180756, 65.84238436888523, 65.63536851657544, 65.38865099395967, 65.13059013697075, 64.96044012137368, 64.79312593936987, 64.54073674956753, 64.29401922695176, 64.07566004026883, 63.87148002155233, 63.64461333408955, 63.39222414428721, 63.22490996228341, 63.023565777160194, 62.83072909281683, 62.65774324362646, 62.43371238975697, 62.21251736948076, 62.01684485154411, 61.80982899923433, 61.59714147973797, 61.40146896180132, 61.20863227745795, 60.98176558999518, 60.786093072058534, 60.54788305022261, 60.30967302838669, 60.04310467061793, 59.796387148002154, 59.61772963162522, 59.325638771516886, 59.095936250460824, 58.888920398151036, 58.68757621302782, 58.48056036071802, 58.27638034200153, 58.08354365765817, 57.876527805348374, 57.66100445225874, 57.47667526869523, 57.2413010804526, 56.99174772424354, 56.793239372713614, 56.58055185321725, 56.35368516575448, 56.14950514703797, 55.90845929160877, 55.72413010804526, 55.51995008932875, 55.29308340186597, 55.06905254799648, 54.799648356634435, 54.53591583245895, 54.26367580750362, 54.05949578878712, 53.773076595865355, 53.45829906701074, 53.208745710801686, 52.92516235147321, 52.58202648668576, 52.22754728752517, 51.884411422737706, 51.50440972123755, 51.11873635255083, 50.574256302640165, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.2234353288149, 67.43045118112468, 67.5382128576695, 67.61761619828148, 67.70552703967331, 67.75373621075916, 67.81045288262484, 67.88985622323682, 67.97493123103536, 68.0032895669682, 68.07418540680032, 68.12523041147945, 68.1734395825653, 68.15075291381902, 68.08836457476674, 68.10537957632647, 68.03731957008763, 68.05717040524061, 68.04582707086749, 67.93522956072937, 67.81328871621814, 67.69134787170688, 67.5410486912628, 67.39358534441199, 67.20074866006863, 66.98522530697898, 66.76686612029606, 66.58253693673255, 66.36134191645634, 66.18268440007941, 65.98701188214275, 65.77148852905312, 65.57298017752318, 65.34894932365368, 65.0852167994782, 64.844170944049, 64.6768567620452, 64.415960071463, 64.16357088166066, 63.87998752233218, 63.607747497376856, 63.358194141167786, 63.11147661855202, 62.85057992796983, 62.56983240223464, 62.328786546805425, 62.096248192156075, 61.84953066954031, 61.62266398207753, 61.432663131327445, 61.19445310949153, 60.956243087655615, 60.70668973144656, 60.47131554320392, 60.261463857300846, 60.07429884014406, 59.819073816748435, 59.60071463006551, 59.37384794260272, 59.11295125202053, 58.94563707001673, 58.727277883333805, 58.579814536483, 58.355783682613506, 58.160111164676856, 58.00130448345291, 57.81413946629612, 57.564586110087056, 57.38025692652356, 57.18742024218019, 57.01443439298982, 56.83010520942631, 56.68547769616878, 56.52950684853813, 56.34234183138134, 56.16084848141111, 55.9027876244222, 55.72696594163854, 55.52562175651532, 55.31860590420554, 55.14845588860845, 54.969798372231516, 54.7684541871083, 54.51322916371267, 54.3402433145223, 54.15591413095879, 53.909196608343024, 53.69650908884667, 53.52068740606302, 53.26829821626067, 53.02441652723818, 52.73232566712986, 52.4629214757678, 52.19068145081247, 51.87023225477129, 51.575305561069676, 51.252020531435214, 50.92306383461418, 50.52037546436775, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.22910699600148, 67.43045118112468, 67.5410486912628, 67.61761619828148, 67.70552703967331, 67.75373621075916, 67.81045288262484, 67.8926920568301, 67.97209539744208, 68.02314040212121, 68.07418540680032, 68.07418540680032, 68.0770212403936, 68.08269290758018, 68.11955874429289, 68.13940957944588, 68.14791708022572, 68.15642458100558, 68.17911124975187, 68.18478291693843, 68.16493208178544, 68.170603748972, 68.14224541303916, 68.15075291381902, 68.1535887474123, 68.15926041459886, 68.15075291381902, 68.11388707710631, 68.08552874117346, 68.08836457476674, 67.98911039900179, 67.99194623259507, 67.93806539432265, 67.86149788730397, 67.83030371777784, 67.7991095482517, 67.74239287638602, 67.65731786858747, 67.57224286078892, 67.49000368658366, 67.41627201315826, 67.36239117488586, 67.26313699912089, 67.15537532257608, 67.05044947962453, 66.92567280152001, 66.8320902929416, 66.73283611717665, 66.61089527266539, 66.47477526018774, 66.37835691801605, 66.21387856960553, 66.06925105634801, 65.8877577063778, 65.68641352125456, 65.51059183847092, 65.32909848850069, 65.15044097212376, 64.91223095028784, 64.69103593001162, 64.48685591129512, 64.27133255820549, 64.05013753792927, 63.817599183279924, 63.56237415988429, 63.36670164194766, 63.12565578651844, 62.847744094376544, 62.581175736607776, 62.33445821399201, 62.028188185917244, 61.69639565550293, 61.39863312820803, 61.078183932166866, 60.70101806425998, 60.423106372118085, 60.122508011229904, 59.78504381362902, 59.46743045118112, 59.149817088733236, 58.81802455831892, 58.460709525565036, 58.08921532484474, 57.737571959277425, 57.36607775855712, 56.93503105237785, 56.5521935172844, 56.140997646258114, 55.74398094319826, 55.31009840342569, 54.9159175339591, 54.479199160593254, 53.99710744973485, 53.50934407168987, 52.987550690525474, 52.414712304681956, 51.83903808524516, 51.223662195502364, 50.60545047216629, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2026, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.15821115616936, 67.32268950457987, 67.43895868190455, 67.53254119048295, 67.61194453109492, 67.65731786858747, 67.69985537248674, 67.74806454357258, 67.83597538496441, 67.8926920568301, 67.95224456228908, 67.93806539432265, 67.93522956072937, 67.9295578935428, 67.93806539432265, 67.93522956072937, 67.9295578935428, 67.92105039276295, 67.91821455916967, 67.92672205994951, 67.91821455916967, 67.90687122479653, 67.87851288886368, 67.86433372089725, 67.85015455293083, 67.83030371777784, 67.82463205059128, 67.77925871309871, 67.73388537560616, 67.69418370530018, 67.68567620452032, 67.62328786546806, 67.54388452485608, 67.50701868814339, 67.4474661826844, 67.41343617956498, 67.31985367098659, 67.2262711624082, 67.1497036553895, 67.06179281399767, 66.96821030541928, 66.88313529762073, 66.7583586195162, 66.67328361171766, 66.54850693361314, 66.45208859144145, 66.27343107506451, 66.09193772509428, 65.95865354620992, 65.80551853217254, 65.62118934860902, 65.41700932989252, 65.29790431897456, 65.09656013385134, 64.94342511981397, 64.72790176672433, 64.53506508238095, 64.35357173241074, 64.16357088166066, 63.94521169497773, 63.74953917704109, 63.53401582395145, 63.29580580211554, 63.09162578339903, 62.85341576156311, 62.60386240535405, 62.388339052264406, 62.17281569917476, 61.94027734452542, 61.730425658622345, 61.495051470379714, 61.20296061027138, 60.98460142358846, 60.664152227547284, 60.3465388650994, 60.03176133624479, 59.663102969117766, 59.30011626917733, 58.93145790205031, 58.534441198990436, 58.233842838102255, 57.88787113972151, 57.516376939001226, 57.17324107421376, 56.85562771176586, 56.42174517199331, 56.002041800187165, 55.61069676431387, 55.21935172844057, 54.83084252616056, 54.42815415591413, 54.02262995207441, 53.62561324901455, 53.12650653659643, 52.689788163230574, 52.15381561409976, 51.63769390012194, 51.09888551739784, 50.56291296826703, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2026, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.16671865694921, 67.353883674106, 67.48716785299038, 67.5779145279755, 67.69701953889346, 67.79343788106513, 67.80478121543828, 67.84164705215098, 67.88418455605026, 67.9295578935428, 67.99478206618835, 68.01746873493462, 68.00896123415477, 67.98343873181521, 67.96358789666222, 67.95508039588238, 67.94090122791594, 67.9465728951025, 67.95224456228908, 67.93806539432265, 67.9295578935428, 67.91537872557637, 67.91537872557637, 67.88702038964354, 67.86716955449053, 67.8558262201174, 67.82463205059128, 67.782094546692, 67.7452287099793, 67.68284037092704, 67.62328786546806, 67.54672035844936, 67.45313784987097, 67.34254033973286, 67.24045033037461, 67.13836032101636, 67.08164364915066, 66.98522530697898, 66.85761279528117, 66.76686612029606, 66.67895527890423, 66.6307461078184, 66.53716359923999, 66.42940192269518, 66.3131327453705, 66.24507273913166, 66.06641522275473, 65.9047727079375, 65.75163769390012, 65.60417434704931, 65.37730765958653, 65.18730680883645, 65.01715679323937, 64.86118594560871, 64.64282675892579, 64.47834841051528, 64.30252672773162, 64.11819754416811, 63.89983835748518, 63.718345007514955, 63.56520999347758, 63.378044976320794, 63.15684995604458, 62.97252077248107, 62.77117658735784, 62.589683237387625, 62.3713240507047, 62.096248192156075, 61.83535150157389, 61.560275643025264, 61.30505061962964, 60.96758642202875, 60.689674729886846, 60.38624053540538, 60.14803051356946, 59.84176048549471, 59.52131128945353, 59.18951875903922, 58.86906956299804, 58.50324702946431, 58.26787284122168, 57.93040864362079, 57.54757110852735, 57.229957746079464, 56.83294104301959, 56.498312679011995, 56.16084848141111, 55.7553242775714, 55.36397924169809, 54.99532087457107, 54.59263250432465, 54.18710830048492, 53.750389927119066, 53.24844738110768, 52.73516150072314, 52.23321895471174, 51.714261407140626, 51.15843802285682, 50.63664464169242, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.16388282335592, 67.33403283895301, 67.42477951393812, 67.55239202563594, 67.63746703343448, 67.69701953889346, 67.73672120919944, 67.7650795451323, 67.76791537872558, 67.83597538496441, 67.90687122479653, 67.93239372713609, 67.94373706150922, 67.9465728951025, 67.94373706150922, 67.94090122791594, 67.96358789666222, 67.94373706150922, 67.86149788730397, 67.79627371465841, 67.69701953889346, 67.55239202563594, 67.39358534441199, 67.24612199756118, 67.15253948898278, 67.02492697728498, 66.82925445934832, 66.71298528202365, 66.55134276720642, 66.33865524771007, 66.09760939228086, 65.85939937044493, 65.68924935484786, 65.48790516972464, 65.2156651447693, 64.98312679011995, 64.75909593625046, 64.50387091285484, 64.31670589569804, 64.08700337464197, 63.848793352806055, 63.621926665343274, 63.38655247710064, 63.09446161699231, 62.80804242407056, 62.54430989989507, 62.20400986870091, 61.96012817967842, 61.685052321129795, 61.39579729461475, 61.11504976887956, 60.820123075177946, 60.57907721974875, 60.31818052916654, 60.057283838584354, 59.76235714488274, 59.53832629101324, 59.25474293168477, 58.98533874032271, 58.70459121458753, 58.421007855259056, 58.13175282874401, 57.84249780222896, 57.6127952811729, 57.31219692028472, 57.00876272580325, 56.705328531321776, 56.407566004026876, 56.1438334798514, 55.82338428381022, 55.56248759322803, 55.25054589796671, 54.96696253863823, 54.65502084337691, 54.32322831296259, 53.95740577942886, 53.6341207497944, 53.36188072483906, 53.08680486629045, 52.75217650228284, 52.46008564217451, 52.142472279726626, 51.79082891415931, 51.518588889203976, 51.33142387204719, 51.246348864248645, 51.18963219238295, 51.13575135411054, 51.09888551739784, 51.07336301505827, 51.05351217990528, 51.01948217678587, 50.95992967132689, 50.92306383461418, 50.86067549556192, 50.744406318237246, 50.577092136233446, 50.40127045344979, 50.225448770666134, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.15537532257608, 67.32268950457987, 67.42194368034482, 67.49851118736352, 67.60343703031506, 67.66582536936733, 67.75940787794572, 67.77358704591214, 67.84731871933755, 67.88702038964354, 67.92672205994951, 67.95508039588238, 67.95791622947566, 67.96358789666222, 67.9692595638488, 67.97776706462865, 67.98060289822193, 67.97776706462865, 67.97493123103536, 67.97209539744208, 67.9692595638488, 67.98343873181521, 67.98060289822193, 67.9465728951025, 67.89836372401668, 67.85015455293083, 67.8388112185577, 67.77925871309871, 67.69985537248674, 67.66015370218076, 67.58075036156879, 67.5580636928225, 67.5382128576695, 67.41343617956498, 67.36239117488586, 67.29716700224031, 67.20074866006863, 67.11850948586337, 67.03343447806482, 66.88029946402744, 66.79238862263563, 66.66761194453109, 66.517312764087, 66.4237302555086, 66.33865524771007, 66.17701273289283, 66.03805688682188, 65.8111901993591, 65.61551768142245, 65.47939766894478, 65.31491932053427, 65.1617843064969, 64.97178345574682, 64.80446927374301, 64.65984176048549, 64.51805008082125, 64.31387006210475, 64.1011825426084, 63.89133085670533, 63.70416583954853, 63.46311998411933, 63.27028329977596, 63.04341661231319, 62.816549924850406, 62.60102657176076, 62.351473215551714, 62.06505402262995, 61.80982899923433, 61.54042480787227, 61.27102061651022, 61.00728809233475, 60.729376400192834, 60.400419703371796, 60.15370218075603, 59.855939653461135, 59.561012959759516, 59.200862093412354, 58.8775770637779, 58.59966537163599, 58.26220117403511, 57.90772197487451, 57.6127952811729, 57.315032753878, 56.99458355783682, 56.61741768992995, 56.28562515951564, 55.98786263222074, 55.61353259790715, 55.22502339562714, 54.79114085585458, 54.38278081842157, 53.940390777869155, 53.52068740606302, 53.11516320222329, 52.58769815387233, 52.09426310864078, 51.63769390012194, 51.12724385333068, 50.582763803420015, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.03627031165811, 67.10433031789694, 67.23477866318804, 67.39925701159855, 67.56373536000908, 67.72254204123301, 67.87851288886368, 67.98343873181521, 68.00896123415477, 68.00045373337493, 68.00896123415477, 67.98343873181521, 68.04582707086749, 68.0259762357145, 68.04582707086749, 68.0259762357145, 67.99761789978163, 67.93239372713609, 67.8558262201174, 67.77925871309871, 67.71119870685988, 67.59776536312849, 67.45880951705755, 67.26880866630745, 67.15253948898278, 67.05044947962453, 66.97104613901256, 66.79806028982219, 66.63641777500496, 66.49746192893402, 66.30462524459064, 66.13163939540027, 65.98701188214275, 65.82536936732552, 65.6438760173553, 65.49074100331792, 65.35178515724697, 65.1787993080566, 64.96894762215352, 64.75909593625046, 64.57476675268695, 64.41312423786972, 64.25998922383235, 64.04163003714942, 63.82894251765306, 63.6361058333097, 63.43192581459319, 63.23908913024984, 63.063267447466174, 62.86192326234297, 62.660579077219744, 62.43654822335025, 62.221024870260614, 62.031024019510525, 61.815500666420895, 61.616992314890965, 61.43833479851401, 61.23982644698409, 61.021467260301165, 60.825794742364515, 60.584748886935316, 60.32952386353969, 60.060119672177635, 59.824745483935004, 59.61205796443865, 59.382355443382586, 59.16683209029294, 58.91444290049059, 58.68190454584125, 58.45503785837847, 58.24518617247539, 58.02399115219919, 57.77160196239684, 57.54473527493407, 57.28951025153843, 57.1108527351615, 56.901001049258426, 56.668462694609076, 56.41607350480674, 56.15517681422454, 55.939653461134895, 55.69010010492585, 55.46039758386978, 55.1597992229816, 54.88472336443297, 54.64651334259706, 54.374273317641716, 54.045316620820685, 53.69650908884667, 53.38456739358535, 53.12934237018971, 52.811729007741825, 52.53098148200663, 52.182173950032606, 51.82769475087201, 51.510081388424126, 51.16127385645011, 50.80112299010294, 50.378583784703515, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.12985282023651, 67.28015200068059, 67.4077645123784, 67.49851118736352, 67.56657119360236, 67.64030286702776, 67.7452287099793, 67.77925871309871, 67.84448288574426, 67.88702038964354, 67.92672205994951, 67.96075206306894, 67.97776706462865, 67.94373706150922, 67.84731871933755, 67.82746788418456, 67.76791537872558, 67.69418370530018, 67.6686612029606, 67.58075036156879, 67.47582451861724, 67.35671950769928, 67.2773161670873, 67.16671865694921, 67.11000198508351, 67.0192553100984, 66.82074695856846, 66.67611944531095, 66.49462609534072, 66.33014774693021, 66.12029606102713, 65.94163854465019, 65.7544735274934, 65.55880100955676, 65.39148682755295, 65.24118764710887, 65.07954513229164, 64.92924595184755, 64.73073760031761, 64.59178175424665, 64.38760173553015, 64.20043671837337, 63.95939086294417, 63.75804667782094, 63.53685165754474, 63.34117913960809, 63.145506621671444, 62.87610243030939, 62.72013158267873, 62.5017723959958, 62.29192071009273, 62.12744236168221, 62.00266568357768, 61.815500666420895, 61.63400731645067, 61.432663131327445, 61.262513115730364, 60.99027309077503, 60.76624223690553, 60.55355471740918, 60.27564302526728, 60.08280634092391, 59.87295465502084, 59.6574313019312, 59.456087116807986, 59.21504126137877, 58.959816237983155, 58.7102628817741, 58.48623202790459, 58.26220117403511, 58.02399115219919, 57.79145279754984, 57.570257777273625, 57.329211921844426, 57.11368856875478, 56.926523551597995, 56.665626861015795, 56.39622266965374, 56.135325979071546, 55.874429288489345, 55.62771176587358, 55.36114340810481, 55.11726171908232, 54.88755919802626, 54.65218500978363, 54.38561665201485, 54.11054079346624, 53.878002438816885, 53.59158324589512, 53.35904489124578, 53.06411819754416, 52.806057340555256, 52.52814564841335, 52.278592292204294, 51.955307262569825, 51.615007231375664, 51.186796358789664, 50.77560048776337, 50.40977795422964, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.13836032101636, 67.29716700224031, 67.41343617956498, 67.49851118736352, 67.56940702719564, 67.64313870062104, 67.73955704279273, 67.782094546692, 67.84731871933755, 67.88702038964354, 67.92672205994951, 67.96075206306894, 67.9862745654085, 67.96358789666222, 67.9125428919831, 67.8756770552704, 67.87851288886368, 67.85299038652411, 67.85015455293083, 67.83597538496441, 67.82463205059128, 67.82179621699798, 67.83030371777784, 67.8189603834047, 67.762243711539, 67.75373621075916, 67.7083628732666, 67.71687037404645, 67.68284037092704, 67.67433287014718, 67.60060119672178, 67.53537702407623, 67.47582451861724, 67.42194368034482, 67.3510478405127, 67.26030116552761, 67.19791282647533, 67.1128378186768, 67.0391061452514, 67.00507614213198, 66.87462779684088, 66.79522445622891, 66.74417945154978, 66.62507444063182, 66.50596942971386, 66.4265660891019, 66.29044607662422, 66.14014689618013, 66.01820605166888, 65.8849218727845, 65.75163769390012, 65.59283101267617, 65.42551683067236, 65.19014264242973, 64.99730595808639, 64.79029010577659, 64.57476675268695, 64.38193006834358, 64.16640671525394, 63.96222669653745, 63.752375010634374, 63.556702492697724, 63.3383433060148, 63.10296911777217, 62.85341576156311, 62.62938490769362, 62.3713240507047, 62.12177069449565, 61.86370983750673, 61.58579814536482, 61.31072228681621, 61.02430309389445, 60.75773473612568, 60.43161387289794, 60.11116467685675, 59.802058815188715, 59.498624620707254, 59.16683209029294, 58.82653205909877, 58.54578453336357, 58.231007004508974, 57.876527805348374, 57.51921277259451, 57.15339023906076, 56.793239372713614, 56.46711850948586, 56.115475143918545, 55.80353344865723, 55.403680912004084, 55.04353004565692, 54.66636417775005, 54.21546663641777, 53.71919575759294, 53.24844738110768, 52.777699004622406, 52.26724895783115, 51.69441057198764, 51.16978135722996, 50.60828630575958, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2027, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.1865694921022, 67.38224201003885, 67.52119785610981, 67.60627286390834, 67.73388537560616, 67.82179621699798, 67.92672205994951, 67.96075206306894, 67.96642373025551, 67.99761789978163, 67.99761789978163, 68.00045373337493, 67.99478206618835, 67.99478206618835, 67.99478206618835, 67.98911039900179, 67.98060289822193, 67.97493123103536, 67.96075206306894, 67.95791622947566, 67.95224456228908, 67.95791622947566, 67.93239372713609, 67.91537872557637, 67.90970705838981, 67.8926920568301, 67.8728412216771, 67.84448288574426, 67.79627371465841, 67.71403454045316, 67.67433287014718, 67.61194453109492, 67.52686952329637, 67.4474661826844, 67.39925701159855, 67.33970450613957, 67.24045033037461, 67.11850948586337, 67.03627031165811, 66.95970280463942, 66.85194112809461, 66.74985111873634, 66.65059694297139, 66.52865609846013, 66.40104358676233, 66.31880441255706, 66.19969940163911, 66.05223605478831, 65.8849218727845, 65.74313019312027, 65.58432351189633, 65.43686016504552, 65.29790431897456, 65.11357513541105, 64.8980517823214, 64.6967075971982, 64.48118424410856, 64.24581005586592, 64.0246150355897, 63.79774834812692, 63.613419164563425, 63.32699997164166, 63.16535745682443, 62.99520744122735, 62.73714658423843, 62.47624989365623, 62.209681535887476, 61.94027734452542, 61.67087315316338, 61.392961461021464, 61.151915605592265, 60.84848141111079, 60.6045997220883, 60.37773303462552, 60.099821342483615, 59.83892465190143, 59.54116212460653, 59.25474293168477, 58.98533874032271, 58.66488954428154, 58.330261180273936, 58.006976150639474, 57.629810282732606, 57.343391089810844, 57.022941893769676, 56.6457760258628, 56.245923489209645, 55.871593454896065, 55.45189008308993, 55.01517170972407, 54.58979667073136, 54.11621246065281, 53.676658253693674, 53.18038737486884, 52.68695232963729, 52.18784561721919, 51.640529733715226, 51.11022885177098, 50.57142046904688, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2027, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.11000198508351, 67.26597283271417, 67.41627201315826, 67.52970535688966, 67.6346311998412, 67.73955704279273, 67.83313955137112, 67.83597538496441, 67.86433372089725, 67.87000538808383, 67.8756770552704, 67.88702038964354, 67.88134872245696, 67.88418455605026, 67.90119955760996, 67.88985622323682, 67.87000538808383, 67.8728412216771, 67.87000538808383, 67.84448288574426, 67.8558262201174, 67.83597538496441, 67.81612454981142, 67.80194538184499, 67.77925871309871, 67.7650795451323, 67.73104954201288, 67.7083628732666, 67.66298953577405, 67.60343703031506, 67.54672035844936, 67.49567535377024, 67.45880951705755, 67.39925701159855, 67.29716700224031, 67.20074866006863, 67.13836032101636, 67.05328531321781, 66.98522530697898, 66.9086577999603, 66.79522445622891, 66.67044777812437, 66.55134276720642, 66.43790942347503, 66.31596857896378, 66.13731106258685, 66.00686271729575, 65.86507103763151, 65.74880186030684, 65.57581601111646, 65.44536766582537, 65.26103848226185, 65.0852167994782, 64.88670844794827, 64.74208093469075, 64.59178175424665, 64.39610923631001, 64.20894421915322, 63.984913365283724, 63.7665541786008, 63.50282165442531, 63.29013413492896, 63.10864078495874, 62.89311743186909, 62.64072824206676, 62.3713240507047, 62.110427360122515, 61.89206817343958, 61.63967898363724, 61.36743895868191, 61.10370643450642, 60.834302243144364, 60.54788305022261, 60.28131469245385, 60.099821342483615, 59.88146215580069, 59.53549045741996, 59.20653376059892, 58.894592065337605, 58.579814536483, 58.27638034200153, 57.91906530924765, 57.626974449139325, 57.27533108357201, 56.98891189065025, 56.57771601962397, 56.15801264781782, 55.82054845021693, 55.39517341122423, 54.96129087145166, 54.572781669171654, 54.15874996455208, 53.670986586507105, 53.18889487564869, 52.65859399370445, 52.19635311799904, 51.67455973683464, 51.13575135411054, 50.5855996370133, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.08164364915066, 67.21776366162834, 67.34537617332614, 67.46164535065083, 67.60627286390834, 67.64881036780761, 67.71119870685988, 67.71403454045316, 67.77075121231886, 67.8558262201174, 67.9465728951025, 67.9465728951025, 67.97493123103536, 67.9692595638488, 67.91821455916967, 67.85866205371069, 67.82463205059128, 67.8189603834047, 67.77642287950543, 67.70552703967331, 67.6516462014009, 67.58075036156879, 67.47866035221053, 67.30851033661345, 67.18373365850891, 66.95970280463942, 66.79522445622891, 66.59955193829225, 66.40387942035562, 66.16850523211298, 65.96999688058305, 65.70910019000084, 65.52477100643735, 65.22133681195588, 65.05685846354535, 64.82715594248928, 64.57760258628024, 64.38760173553015, 64.16924254884722, 63.99625669965686, 63.76088251141423, 63.46028415052605, 63.22490996228341, 63.03774494512662, 62.85341576156311, 62.62938490769362, 62.413861554603976, 62.19266653432777, 61.96012817967842, 61.74460482658878, 61.546096475058846, 61.28236395088336, 61.11221393528628, 60.893854748603346, 60.698182230666696, 60.44579304086436, 60.27280719167399, 60.0657913393642, 59.864447154240985, 59.63474463318492, 59.492952953520685, 59.28026543402433, 59.11295125202053, 58.97683123954286, 58.78115872160621, 58.537277032583724, 58.34444034824037, 58.14026032952386, 57.95593114596035, 57.79996029832969, 57.55607860930721, 57.35473442418398, 57.17040524062048, 56.94637438675099, 56.776224371153894, 56.543686016504545, 56.37070016731418, 56.20622181890367, 55.99069846581403, 55.78651844709752, 55.602189263534015, 55.398009244817516, 55.139948387828596, 54.85352919490685, 54.66636417775005, 54.433825823100705, 54.16725746533193, 53.90636077474974, 53.61994158182798, 53.339194056092786, 53.06411819754416, 52.777699004622406, 52.42889147264839, 52.10277060942063, 51.78799308056603, 51.527096389983825, 51.16978135722996, 50.83231715962907, 50.48067379406176, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.23761449678133, 67.5013470209568, 67.69418370530018, 67.81328871621814, 67.91821455916967, 67.98060289822193, 68.0429912372742, 68.10254374273318, 68.11955874429289, 68.12239457788617, 68.12239457788617, 68.11955874429289, 68.11672291069961, 68.11672291069961, 68.10821540991975, 68.11105124351303, 68.10254374273318, 68.0997079091399, 68.09120040836004, 68.07134957320704, 68.04866290446077, 68.01463290134134, 67.96642373025551, 67.9295578935428, 67.8955278904234, 67.82463205059128, 67.74806454357258, 67.65448203499419, 67.61761619828148, 67.5779145279755, 67.52686952329637, 67.44463034909111, 67.35671950769928, 67.25462949834103, 67.16388282335592, 67.06462864759096, 66.9823894733857, 66.88313529762073, 66.76686612029606, 66.59955193829225, 66.48328276096758, 66.36134191645634, 66.27910274225108, 66.17701273289283, 66.00686271729575, 65.86223520403823, 65.69492102203442, 65.58715934548961, 65.39715849473953, 65.21282931117602, 65.00864929245952, 64.77894677140345, 64.65417009329893, 64.5095425800414, 64.3620792331906, 64.16073504806738, 63.914017525451605, 63.76371834500751, 63.579389161444, 63.33550747242151, 63.10296911777217, 62.92714743498852, 62.725803249865294, 62.524459064742075, 62.29475654368601, 62.01117318435754, 61.727589825029064, 61.50072313756628, 61.26534894932365, 61.086691432946715, 60.84280974392422, 60.61027138927487, 60.33803136431955, 60.03459716983808, 59.68578963786405, 59.35683294104302, 59.08742874968097, 58.80384539035249, 58.45503785837847, 58.151603663897, 57.80846779910954, 57.46816776791538, 57.13070357031449, 56.798911039900176, 56.433088506366445, 56.09278847517227, 55.78935428069081, 55.372486742477946, 55.09457505033605, 54.66069251056348, 54.29770581062303, 53.90636077474974, 53.512179905283155, 53.04993902957774, 52.610384822618606, 52.14530811331991, 51.711425573547345, 51.15560218926354, 50.579927969826734, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.02209114369168, 67.09865865071038, 67.24045033037461, 67.4077645123784, 67.56089952641578, 67.73388537560616, 67.88702038964354, 68.0259762357145, 68.12239457788617, 68.11105124351303, 68.170603748972, 68.170603748972, 68.14224541303916, 68.12806624507274, 68.08269290758018, 68.06284207242719, 68.05717040524061, 67.95791622947566, 67.94940872869579, 67.83313955137112, 67.73104954201288, 67.62045203187476, 67.5382128576695, 67.30851033661345, 67.17806199132235, 67.04477781243796, 66.91149363355358, 66.77253778748263, 66.65343277656467, 66.48611859456088, 66.36701358364292, 66.27343107506451, 66.108952726654, 66.00119105010917, 65.81969770013896, 65.64671185094858, 65.4396959986388, 65.26671014944843, 65.07954513229164, 64.85551427842215, 64.68820009641834, 64.47834841051528, 64.30252672773162, 64.14655588010095, 63.96789836372402, 63.82043501687321, 63.621926665343274, 63.41491081303349, 63.2277457958767, 63.04625244590647, 62.87043076312282, 62.74281825142501, 62.581175736607776, 62.33445821399201, 62.17848736636134, 61.95162067889856, 61.750276493775345, 61.58012647817826, 61.392961461021464, 61.20296061027138, 61.061168930607145, 60.86833224626379, 60.67549556192042, 60.50818137991663, 60.301165527606834, 60.07997050733063, 59.89280549017384, 59.691461305050616, 59.50429628789382, 59.30295210277061, 59.10160791764739, 58.88041289737118, 58.662053710688255, 58.469217026344886, 58.239514505288824, 58.01264781782605, 57.74891529365057, 57.524884439781076, 57.272495249978725, 57.028613560956245, 56.78473187193376, 56.5153276805717, 56.18920681734396, 55.92831012676176, 55.62487593228029, 55.32427757139211, 54.99532087457107, 54.68621501290304, 54.3600941496753, 54.02830161926098, 53.70501658962652, 53.39023906077192, 53.013073192865036, 52.63023565777161, 52.199188951592326, 51.80784391571902, 51.39381221109945, 50.96560133851346, 50.489181294841615, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.2262711624082, 67.48716785299038, 67.67716870374046, 67.7991095482517, 67.91537872557637, 67.97776706462865, 68.05717040524061, 68.10254374273318, 68.12239457788617, 68.12523041147945, 68.09120040836004, 68.08836457476674, 68.07418540680032, 68.05717040524061, 68.05149873805405, 67.94940872869579, 67.89836372401668, 67.85015455293083, 67.74806454357258, 67.67433287014718, 67.55239202563594, 67.44179451549783, 67.32836117176643, 67.17806199132235, 67.0929869835238, 66.97955363979241, 66.84343362731475, 66.71582111561693, 66.54283526642656, 66.38970025238919, 66.1458185633667, 65.93596687746363, 65.7544735274934, 65.60984601423588, 65.42268099707908, 65.18163514164989, 64.99447012449309, 64.77327510421689, 64.60596092221309, 64.40745257068315, 64.26566089101891, 64.06148087230241, 63.83177835124635, 63.61909083174999, 63.38655247710064, 63.219238295096844, 63.012222442787056, 62.82505742563026, 62.63505657488019, 62.450727391316676, 62.23803987182032, 62.056546521850095, 61.82967983438732, 61.5914698125514, 61.43549896492073, 61.23415477979752, 61.052661429827296, 60.81728724158466, 60.63862972520773, 60.43728554008451, 60.25295635652099, 60.07146300655077, 59.90414882454698, 59.728327141763316, 59.50996795508039, 59.30011626917733, 59.11295125202053, 58.91160706689731, 58.70742704818082, 58.51459036383745, 58.32742534668065, 58.11757366077759, 57.90488614128123, 57.71204945693786, 57.505033604628075, 57.28667441794515, 57.08249439922866, 56.85562771176586, 56.671298528202364, 56.41890933840003, 56.177863482970814, 55.95099679550803, 55.72696594163854, 55.43487508153021, 55.139948387828596, 54.89606669880611, 54.609647505884354, 54.28352664265661, 53.99427161614157, 53.74188242633922, 53.41292572951819, 53.08964069988373, 52.72948983353657, 52.34665229844313, 52.020531435215375, 51.66888806964807, 51.303065536114346, 50.8663471627485, 50.42112128860279, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.2234353288149, 67.49283952017694, 67.69134787170688, 67.80478121543828, 67.91537872557637, 67.97776706462865, 68.05433457164733, 68.10254374273318, 68.12239457788617, 68.11388707710631, 68.11105124351303, 68.10537957632647, 68.08836457476674, 68.0968720755466, 68.09120040836004, 68.08269290758018, 68.08552874117346, 68.06567790602047, 68.0600062388339, 68.04866290446077, 68.03731957008763, 68.04015540368091, 68.02314040212121, 68.00896123415477, 68.00045373337493, 67.9465728951025, 67.90970705838981, 67.94090122791594, 67.88985622323682, 67.85299038652411, 67.7991095482517, 67.73104954201288, 67.6885120381136, 67.6147803646882, 67.53537702407623, 67.46731701783739, 67.44179451549783, 67.34254033973286, 67.25746533193431, 67.13552448742307, 67.05044947962453, 66.97388197260584, 66.88313529762073, 66.72149278280351, 66.62223860703854, 66.52865609846013, 66.39253608598247, 66.27343107506451, 66.14865439695998, 65.96999688058305, 65.79417519779939, 65.67790602047472, 65.5502935087769, 65.35745682443354, 65.18730680883645, 65.01715679323937, 64.84700677764229, 64.65417009329893, 64.45566174176899, 64.27416839179877, 64.03879420355614, 63.81476334968664, 63.59356832941043, 63.34968664038794, 63.0972974505856, 62.90162493264895, 62.646399909253326, 62.39684655304426, 62.12460652808893, 61.84102316876046, 61.60564898051783, 61.34475228993562, 61.05549726342058, 60.80027224002495, 60.522360547883046, 60.20474718543516, 59.9183279925134, 59.63474463318492, 59.34265377307659, 59.03638374500184, 58.727277883333805, 58.304738677934374, 57.90488614128123, 57.516376939001226, 57.11652440234807, 56.75920936959419, 56.34801349856791, 55.931145960355046, 55.50577092136233, 55.105918384709184, 54.70039418086946, 54.27785497547004, 53.812778266171335, 53.273969883447236, 52.77486317102912, 52.26441312423786, 51.76814224541304, 51.26052803221507, 50.63097297450585, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2028, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.15821115616936, 67.30851033661345, 67.43895868190455, 67.5580636928225, 67.66298953577405, 67.72254204123301, 67.77642287950543, 67.83313955137112, 67.86716955449053, 67.8926920568301, 67.90970705838981, 67.90687122479653, 67.89836372401668, 67.8955278904234, 67.8926920568301, 67.8926920568301, 67.90687122479653, 67.8955278904234, 67.8926920568301, 67.88985622323682, 67.86149788730397, 67.86716955449053, 67.85866205371069, 67.85015455293083, 67.81328871621814, 67.77075121231886, 67.74806454357258, 67.73388537560616, 67.71687037404645, 67.72254204123301, 67.64313870062104, 67.57224286078892, 67.50985452173667, 67.45880951705755, 67.38791367722541, 67.29433116864702, 67.18940532569549, 67.12418115304993, 67.04194197884468, 66.965374471826, 66.84059779372147, 66.74417945154978, 66.59671610469897, 66.50880526330715, 66.37835691801605, 66.26775940787795, 66.09477355868756, 65.97566854776962, 65.83671270169866, 65.72044352437399, 65.55312934237018, 65.39999432833281, 65.25253098148201, 65.0852167994782, 64.9746192893401, 64.82999177608258, 64.63148342455266, 64.3790942347503, 64.20327255196665, 63.979241698097155, 63.721180841108236, 63.47162748489919, 63.26461163258939, 62.99520744122735, 62.782519921730994, 62.50744406318237, 62.328786546805425, 62.07639735700309, 61.8693815046933, 61.62266398207753, 61.28236395088336, 61.092363100133284, 60.84280974392422, 60.55639055100246, 60.24728468933442, 59.929671326886535, 59.6375804667782, 59.260414598871336, 58.965487905169724, 58.6904120466211, 58.43235118963219, 58.07503615687831, 57.70921362334458, 57.31786858747129, 56.926523551597995, 56.51249184697842, 56.126818478291696, 55.72129427445197, 55.35547174091824, 54.88755919802626, 54.45367665825369, 54.048152454413966, 53.60292658026826, 53.14068570456285, 52.65575816011116, 52.122621444573625, 51.60366389700253, 51.09604968380456, 50.574256302640165, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2028, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.05044947962453, 67.18089782491563, 67.3170178373933, 67.4474661826844, 67.59492952953521, 67.70552703967331, 67.73955704279273, 67.80761704903156, 67.8388112185577, 67.84164705215098, 67.83597538496441, 67.83597538496441, 67.81328871621814, 67.82463205059128, 67.83030371777784, 67.83030371777784, 67.82463205059128, 67.81328871621814, 67.78493038028527, 67.74806454357258, 67.74239287638602, 67.73104954201288, 67.71119870685988, 67.72254204123301, 67.70552703967331, 67.68000453733374, 67.6317953662479, 67.61194453109492, 67.58075036156879, 67.50701868814339, 67.41627201315826, 67.35955534129258, 67.28582366786718, 67.21492782803506, 67.11567365227008, 66.99656864135213, 66.84626946090803, 66.78388112185577, 66.67895527890423, 66.54567110001986, 66.44641692425488, 66.29611774381078, 66.15149023055326, 66.00969855088903, 65.86223520403823, 65.70626435640756, 65.58432351189633, 65.44253183223209, 65.28088931741485, 65.13626180415733, 64.96327595496696, 64.78178260499674, 64.65417009329893, 64.415960071463, 64.2855117261719, 64.05580920511585, 63.854465019992624, 63.69282250517539, 63.49998582083203, 63.29864163570882, 63.09446161699231, 62.95550577092136, 62.73714658423843, 62.53580239911521, 62.30609987805915, 62.08206902418966, 61.84385900235374, 61.62266398207753, 61.37594645946176, 61.10370643450642, 60.896690582196634, 60.695346397073415, 60.482658877577066, 60.233105521368, 60.02041800187165, 59.668774636304335, 59.390862944162436, 59.138473754360085, 58.86623372940476, 58.65638204350169, 58.421007855259056, 58.17996199982984, 57.85100530300882, 57.49369027025494, 57.1845844085869, 56.78189603834048, 56.41890933840003, 55.98219096503417, 55.57950259478773, 55.17397839094802, 54.72024501602246, 54.29770581062303, 53.841136602104186, 53.32785072171965, 52.83157984289482, 52.309786461730425, 51.74261974307348, 51.178288858009815, 50.58843547060658, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 66.97955363979241, 67.07030031477753, 67.13552448742307, 67.26880866630745, 67.3510478405127, 67.47866035221053, 67.54672035844936, 67.6686612029606, 67.81045288262484, 67.8728412216771, 67.9295578935428, 67.92105039276295, 67.8955278904234, 67.8955278904234, 67.8955278904234, 67.8756770552704, 67.88134872245696, 67.86716955449053, 67.80761704903156, 67.69134787170688, 67.60343703031506, 67.4474661826844, 67.28582366786718, 67.15253948898278, 66.96821030541928, 66.75552278592292, 66.52865609846013, 66.32164024615035, 66.11462439384057, 65.87357853841137, 65.60417434704931, 65.37447182599325, 65.09656013385134, 64.81864844170944, 64.66267759407877, 64.435810906616, 64.21461588633979, 64.02177920199642, 63.79491251453364, 63.599239996597, 63.361029974761074, 63.13132745370501, 62.98102827326092, 62.686101579559306, 62.44789155772339, 62.2408757054136, 61.9998298499844, 61.73326149221563, 61.495051470379714, 61.248333947763946, 61.018631426707884, 60.7917647392451, 60.533703882256184, 60.32101636275983, 60.105493009670184, 59.844596319088, 59.58369962850579, 59.35116127385645, 59.12429458639367, 58.90877123330402, 58.71309871536739, 58.50891869665088, 58.29339534356124, 58.07220032328503, 57.9105578084678, 57.692198621784875, 57.522048606187795, 57.28951025153843, 57.096673567195076, 56.903836882851714, 56.708164364915056, 56.540850182911264, 56.359356832941046, 56.175027649377526, 55.98786263222074, 55.826220117403516, 55.61636843150044, 55.46039758386978, 55.25621756515328, 55.0662167144032, 54.870544196466554, 54.64651334259706, 54.442333323880554, 54.23531747157077, 53.999943283328136, 53.753225760712354, 53.51785157246973, 53.273969883447236, 53.04143152879789, 52.85426651164109, 52.60471315543204, 52.420383971868524, 52.11411394379377, 51.816351416498875, 51.50157388764427, 51.22933386268893, 50.8663471627485, 50.619629640132715, 50.33888211439752, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.21209199444176, 67.46164535065083, 67.58075036156879, 67.69701953889346, 67.78776621387857, 67.85866205371069, 67.93239372713609, 67.99194623259507, 68.00612540056149, 68.00896123415477, 68.01179706774806, 68.01463290134134, 68.01463290134134, 68.01746873493462, 68.00612540056149, 68.0203045685279, 68.00896123415477, 68.0032895669682, 67.99761789978163, 67.98343873181521, 67.96075206306894, 67.91537872557637, 67.88985622323682, 67.86433372089725, 67.80478121543828, 67.77075121231886, 67.7253778748263, 67.72821370841959, 67.71687037404645, 67.64881036780761, 67.5779145279755, 67.51269035532995, 67.43895868190455, 67.35671950769928, 67.28582366786718, 67.18373365850891, 67.13268865382979, 67.0192553100984, 66.93985196948643, 66.87462779684088, 66.77253778748263, 66.65626861015795, 66.53716359923999, 66.40387942035562, 66.23089357116524, 66.08059439072115, 65.98701188214275, 65.85939937044493, 65.680741854068, 65.50208433769106, 65.34044182287383, 65.14760513853048, 64.94342511981397, 64.80446927374301, 64.65133425970564, 64.46984090973541, 64.24297422227264, 63.99909253325015, 63.857300853585905, 63.64461333408955, 63.437597481779775, 63.28446246774239, 63.06893911465274, 62.86192326234297, 62.64356407566004, 62.419533221790545, 62.152964864021776, 61.93176984374556, 61.66803731957009, 61.39012562742818, 61.0725122649803, 60.79743640643167, 60.49116637835692, 60.1962396846553, 59.9353429940731, 59.6375804667782, 59.322802937923605, 59.04772707937498, 58.798173723165924, 58.52876953180387, 58.160111164676856, 57.870856138161805, 57.60145194679976, 57.329211921844426, 57.00876272580325, 56.628761024303095, 56.245923489209645, 55.899951790828915, 55.466069251056346, 55.06905254799648, 54.669200011343335, 54.181436633298354, 53.77024076227207, 53.30799988656665, 52.82307234211496, 52.35799563281627, 51.83053058446529, 51.18396052519638, 50.60545047216629, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.04477781243796, 67.12418115304993, 67.25179366474775, 67.40492867878513, 67.60060119672178, 67.75657204435244, 67.85866205371069, 67.96075206306894, 68.03731957008763, 68.11672291069961, 68.11388707710631, 68.18194708334515, 68.21314125287128, 68.24433542239741, 68.210305419278, 68.210305419278, 68.16776791537872, 68.09403624195332, 68.07985707398689, 67.99194623259507, 67.88134872245696, 67.782094546692, 67.71687037404645, 67.65731786858747, 67.54388452485608, 67.41343617956498, 67.2773161670873, 67.21209199444176, 67.1128378186768, 66.93134446870658, 66.77537362107591, 66.5740294359527, 66.36417775004962, 66.19402773445255, 65.9614893798032, 65.76865269545983, 65.6240251822023, 65.40283016192609, 65.22417264554916, 64.9916342908998, 64.78745427218331, 64.59461758783995, 64.3620792331906, 64.08700337464197, 63.811927516093355, 63.658792502055974, 63.40356747866035, 63.185208291977425, 62.99237160763407, 62.72296741627201, 62.530130731928644, 62.283413209312876, 62.02535235232396, 61.787142330488045, 61.60281314692454, 61.31072228681621, 61.0328105946743, 60.854153078297365, 60.6045997220883, 60.34086719791283, 60.0856421745172, 59.82190965034172, 59.592207129285654, 59.35399710744973, 59.129966253580236, 58.87190539659133, 58.64220287553527, 58.35294784902022, 58.060856988911894, 57.802796131922975, 57.54757110852735, 57.35473442418398, 57.14488273828091, 56.91234438363156, 56.71383603210164, 56.5153276805717, 56.2544309899895, 56.070101806425996, 55.8460709525565, 55.66457760258628, 55.41502424637722, 55.19950089328758, 54.983977540197934, 54.71457334883589, 54.49621416215297, 54.28636247624989, 54.050988288007254, 53.832629101324336, 53.60292658026826, 53.35620905765249, 53.103819867850156, 52.803221506961975, 52.511130646853644, 52.213368119558744, 51.907098091483995, 51.56679806028982, 51.23216969628221, 50.84082466040892, 50.452315458128915, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.19791282647533, 67.43895868190455, 67.56940702719564, 67.69701953889346, 67.79060204747185, 67.86433372089725, 67.9295578935428, 67.99478206618835, 68.02881206930778, 68.11105124351303, 68.16493208178544, 68.15926041459886, 68.12239457788617, 68.18478291693843, 68.14508124663244, 68.0997079091399, 68.03164790290106, 68.00896123415477, 67.96642373025551, 67.8756770552704, 67.80478121543828, 67.68284037092704, 67.60627286390834, 67.4276153475314, 67.3000028358336, 67.14686782179622, 66.9625386382327, 66.70164194765051, 66.57686526954598, 66.41522275472876, 66.2195502367921, 65.98701188214275, 65.76014519467998, 65.5701443439299, 65.37163599239997, 65.19014264242973, 64.98029095652666, 64.73073760031761, 64.49819924566826, 64.3053625613249, 64.0983467090151, 63.83177835124635, 63.655956668462686, 63.375209142727506, 63.0972974505856, 62.8562515951564, 62.66625074440631, 62.43087655616368, 62.17565153276805, 61.91475484218586, 61.67087315316338, 61.424155630547595, 61.23415477979752, 61.00445225874146, 60.7719139040921, 60.525196381476334, 60.24444885574114, 60.026089669058216, 59.753849644102885, 59.4901171199274, 59.21504126137877, 59.00518957547571, 58.74429288489351, 58.48623202790459, 58.26503700762839, 58.05234948813204, 57.81697529988941, 57.57876527805348, 57.334883589030994, 57.05980773048237, 56.76771687037404, 56.495476845418715, 56.243087655616364, 56.004877633780445, 55.79786178147066, 55.54547259166832, 55.24771006437342, 55.00666420894422, 54.76278251992173, 54.484870827779815, 54.16158579814536, 53.89218160678331, 53.63979241698097, 53.36188072483906, 53.143521538156136, 52.89680401554037, 52.66142982729774, 52.45441397498794, 52.23038312111845, 51.98082976490939, 51.76814224541304, 51.56963389388311, 51.419334713439014, 51.28037886736806, 51.13007968692397, 50.95142217054703, 50.77843632135666, 50.613957972946146, 50.35022544877066, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.20642032725519, 67.4474661826844, 67.56940702719564, 67.69701953889346, 67.79343788106513, 67.86433372089725, 67.9295578935428, 67.99478206618835, 68.02881206930778, 68.11955874429289, 68.1337379122593, 68.170603748972, 68.1734395825653, 68.1734395825653, 68.15926041459886, 68.1337379122593, 68.13940957944588, 68.15075291381902, 68.16776791537872, 68.14508124663244, 68.13090207866603, 68.11388707710631, 68.10254374273318, 68.0968720755466, 68.08269290758018, 68.09403624195332, 68.07134957320704, 68.03164790290106, 68.01746873493462, 67.97209539744208, 67.90970705838981, 67.85866205371069, 67.80478121543828, 67.75657204435244, 67.70552703967331, 67.6516462014009, 67.59492952953521, 67.5410486912628, 67.45597368346425, 67.37940617644557, 67.28015200068059, 67.18940532569549, 67.12985282023651, 67.06179281399767, 66.95403113745286, 66.87746363043416, 66.76970195388935, 66.62507444063182, 66.52298443127357, 66.43790942347503, 66.32164024615035, 66.18268440007941, 65.99551938292261, 65.81686186654568, 65.65805518532173, 65.49924850409778, 65.33760598928055, 65.15611263931032, 64.9377534526274, 64.76193176984374, 64.55775175112724, 64.3422283980376, 64.12954087854125, 63.8970025238919, 63.63043416612313, 63.40640331225364, 63.139834954484876, 62.9299832685818, 62.70028074752574, 62.47057822646966, 62.25505487338003, 61.93744151093213, 61.63684315004395, 61.33908062274905, 61.012959759521316, 60.74071973456597, 60.352210532285966, 60.02041800187165, 59.6574313019312, 59.291608768397474, 58.903099566117454, 58.58265037007628, 58.24802200606868, 57.84816946941553, 57.44831693276238, 57.08533023282194, 56.68547769616878, 56.28278932592236, 55.894280123642346, 55.41502424637722, 54.89039503161954, 54.40263165357456, 53.95740577942886, 53.46964240138389, 52.973371522559056, 52.43172730624167, 51.87590392195786, 51.2548563650285, 50.61679380653943, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2029, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.18089782491563, 67.3510478405127, 67.45313784987097, 67.51836202251653, 67.62895953265462, 67.72254204123301, 67.79343788106513, 67.8756770552704, 67.88985622323682, 67.93239372713609, 67.93806539432265, 67.93806539432265, 67.94090122791594, 67.95224456228908, 67.95791622947566, 67.96358789666222, 67.96075206306894, 67.95508039588238, 67.96642373025551, 67.97209539744208, 67.9465728951025, 67.94940872869579, 67.94090122791594, 67.91537872557637, 67.91537872557637, 67.90403539120324, 67.86716955449053, 67.80761704903156, 67.79343788106513, 67.74239287638602, 67.70269120608002, 67.64313870062104, 67.5382128576695, 67.43045118112468, 67.39358534441199, 67.3000028358336, 67.19791282647533, 67.12701698664321, 67.01074780931854, 66.905821966367, 66.81791112497518, 66.71014944843037, 66.57686526954598, 66.44074525706831, 66.33581941411678, 66.23089357116524, 66.06357938916145, 65.96716104698976, 65.88208603919122, 65.70626435640756, 65.57298017752318, 65.41133766270595, 65.24969514788872, 65.0852167994782, 64.91506678388112, 64.78745427218331, 64.56058758472054, 64.29969089413832, 64.05864503870913, 63.82610668405978, 63.661628335649254, 63.47162748489919, 63.28446246774239, 63.142670788078156, 62.85341576156311, 62.60102657176076, 62.38266738507784, 62.09908402574936, 61.80982899923433, 61.54042480787227, 61.24549811417066, 61.01012392592803, 60.75773473612568, 60.49683804554349, 60.20474718543516, 59.90414882454698, 59.58086379491251, 59.3057879363639, 59.027876244221986, 58.662053710688255, 58.395485352919486, 58.04384198735219, 57.71488529053115, 57.41712276323625, 57.01443439298982, 56.665626861015795, 56.27995349232907, 55.871593454896065, 55.48024841902277, 55.06054504721662, 54.62382667385077, 54.144570796585654, 53.67949408728696, 53.25695488188753, 52.718146499163424, 52.182173950032606, 51.63485806652866, 51.07336301505827, 50.5288829651476, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2029, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.04761364603125, 67.13268865382979, 67.26880866630745, 67.4276153475314, 67.53254119048295, 67.6714970365539, 67.70552703967331, 67.69418370530018, 67.71970620763973, 67.73388537560616, 67.73955704279273, 67.73955704279273, 67.73672120919944, 67.71687037404645, 67.7083628732666, 67.6885120381136, 67.6714970365539, 67.6686612029606, 67.64597453421433, 67.6317953662479, 67.62895953265462, 67.62045203187476, 67.60627286390834, 67.58358619516207, 67.58358619516207, 67.54672035844936, 67.5041828545501, 67.47866035221053, 67.38791367722541, 67.34254033973286, 67.26030116552761, 67.17806199132235, 67.0759719819641, 66.99656864135213, 66.91149363355358, 66.82358279216174, 66.69880611405723, 66.60805943907211, 66.50880526330715, 66.37268525082948, 66.29044607662422, 66.11178856024729, 65.97283271417633, 65.81969770013896, 65.73462269234042, 65.62118934860902, 65.4935768369112, 65.33760598928055, 65.19581430961631, 65.03417179479908, 64.88387261435498, 64.71372259875791, 64.56342341831382, 64.37342256756374, 64.2486458894592, 64.0983467090151, 63.86864418795904, 63.678643337208975, 63.50282165442531, 63.29580580211554, 63.07177494824604, 62.881774097495956, 62.68326574596602, 62.48759322802938, 62.27774154212631, 62.01117318435754, 61.76161982814848, 61.495051470379714, 61.2199756118311, 60.88251141423021, 60.64713722598758, 60.400419703371796, 60.156538014349316, 59.90131299095369, 59.6375804667782, 59.42489294728185, 59.203697927005635, 59.019368743442136, 58.76981538723307, 58.494739528684455, 58.18279783342313, 57.90488614128123, 57.61846694835947, 57.30936108669143, 56.97756855627711, 56.603238521963526, 56.23458015483651, 55.84323511896322, 55.463233417463066, 55.12009755267561, 54.65218500978363, 54.1899441340782, 53.67382242010039, 53.22576071236141, 52.720982332756705, 52.20202478518561, 51.654708901681644, 51.11022885177098, 50.52604713155432, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.03627031165811, 67.11850948586337, 67.19507699288205, 67.2773161670873, 67.38224201003885, 67.5013470209568, 67.59776536312849, 67.67716870374046, 67.7452287099793, 67.782094546692, 67.88985622323682, 67.90687122479653, 67.93522956072937, 67.90119955760996, 67.90119955760996, 67.81612454981142, 67.72254204123301, 67.64030286702776, 67.57224286078892, 67.53537702407623, 67.46448118424411, 67.33686867254629, 67.19791282647533, 67.04194197884468, 66.90298613277372, 66.77537362107591, 66.67895527890423, 66.48328276096758, 66.26492357428467, 66.09477355868756, 65.93313104387035, 65.74029435952698, 65.59566684626947, 65.41700932989252, 65.23835181351558, 65.01999262683266, 64.89238011513484, 64.69954343079148, 64.5095425800414, 64.29118339335848, 64.06431670589569, 63.79491251453364, 63.55386665910444, 63.412074979440206, 63.154014122451294, 62.944162436548226, 62.756997419391425, 62.5584890678615, 62.29192071009273, 62.12744236168221, 61.91475484218586, 61.682216487536515, 61.43833479851401, 61.188781442304965, 60.993108924368315, 60.74071973456597, 60.56489805178232, 60.352210532285966, 60.15086634716275, 59.93250716047982, 59.73967047613647, 59.54116212460653, 59.325638771516886, 59.090264583274255, 58.86056206221819, 58.653546209908406, 58.421007855259056, 58.20264866857613, 58.00130448345291, 57.77160196239684, 57.59010861242662, 57.394436094489976, 57.19309190936676, 56.9435385531577, 56.69682103054192, 56.44726767433287, 56.21189348609023, 55.939653461134895, 55.695771772112415, 55.49159175339591, 55.210844227660715, 54.99815670816436, 54.72875251680232, 54.47352749340668, 54.249496639537185, 54.03113745285426, 53.747554093525785, 53.472478234977174, 53.16337237330912, 52.92516235147321, 52.64725065933131, 52.4062048039021, 52.142472279726626, 51.85321725321158, 51.50440972123755, 51.26336386580835, 50.94007883617388, 50.65649547684542, 50.35873294955052, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.29716700224031, 67.50701868814339, 67.6516462014009, 67.72821370841959, 67.86716955449053, 67.9295578935428, 68.04582707086749, 68.10254374273318, 68.12806624507274, 68.15926041459886, 68.17627541615859, 68.17627541615859, 68.17627541615859, 68.17627541615859, 68.17911124975187, 68.1734395825653, 68.1734395825653, 68.16209624819216, 68.16493208178544, 68.16209624819216, 68.13090207866603, 68.0968720755466, 68.08552874117346, 68.05717040524061, 68.0032895669682, 67.95508039588238, 67.9125428919831, 67.87851288886368, 67.82179621699798, 67.75940787794572, 67.70269120608002, 67.64030286702776, 67.55522785922922, 67.5013470209568, 67.43045118112468, 67.36806284207243, 67.29149533505374, 67.21492782803506, 67.12134531945665, 66.99373280775885, 66.84910529450133, 66.69029861327736, 66.55985026798628, 66.4237302555086, 66.29044607662422, 66.16850523211298, 66.02954938604204, 65.8877577063778, 65.72044352437399, 65.49924850409778, 65.38581516036639, 65.24402348070215, 65.07103763151179, 64.89238011513484, 64.72506593313105, 64.55208008394068, 64.33372089725775, 64.09267504182854, 63.88282335592548, 63.67013583642912, 63.43476164818648, 63.258939965402824, 63.057595780279605, 62.82505742563026, 62.654907410033175, 62.456399058503244, 62.24654737260017, 61.99415818279783, 61.76445566174177, 61.48654396959986, 61.30221478603636, 61.04131809545416, 60.79743640643167, 60.4741513767972, 60.14803051356946, 59.83325298471486, 59.52131128945353, 59.257578765278055, 58.92862206845702, 58.633695374755405, 58.335932847460505, 58.04951365453876, 57.746079460057274, 57.394436094489976, 57.11368856875478, 56.74786603522105, 56.367864333720895, 56.03607180330658, 55.60786093072059, 55.19099339250773, 54.75994668632844, 54.31755664577602, 53.86382327085046, 53.39591072795849, 52.981879023338905, 52.420383971868524, 51.82769475087201, 51.26903553299492, 50.6394804752857, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.02776281087826, 67.09865865071038, 67.23761449678133, 67.39358534441199, 67.53254119048295, 67.73672120919944, 67.88985622323682, 68.04582707086749, 68.12806624507274, 68.17627541615859, 68.18478291693843, 68.16776791537872, 68.1337379122593, 68.11672291069961, 68.0429912372742, 67.94090122791594, 67.89836372401668, 67.86433372089725, 67.8189603834047, 67.68567620452032, 67.57224286078892, 67.41910784675154, 67.29716700224031, 67.15253948898278, 67.01358364291184, 66.83492612653488, 66.7583586195162, 66.61940277344526, 66.48895442815416, 66.34432691489664, 66.20820690241897, 66.02671355244874, 65.85656353685165, 65.70626435640756, 65.55312934237018, 65.44820349941865, 65.20148597680287, 65.03417179479908, 64.84700677764229, 64.67402092845191, 64.4726767433287, 64.28834755976519, 64.14088421291439, 63.94521169497773, 63.77789751297394, 63.599239996597, 63.38938831069393, 63.185208291977425, 63.00371494200721, 62.81087825766384, 62.6123699061339, 62.43938405694354, 62.25221903978675, 62.07072568981652, 61.89773984062615, 61.67938065394323, 61.47803646882001, 61.29087145166321, 61.15475143918555, 60.98176558999518, 60.80877974080481, 60.66698806114057, 60.4769872103905, 60.3068371947934, 60.116836344043335, 59.90982049173355, 59.73967047613647, 59.5468337917931, 59.33698210589002, 59.11578708561381, 58.88324873096447, 58.70459121458753, 58.54294869977029, 58.33309701386722, 58.103394492811155, 57.86518447097524, 57.62130278195276, 57.394436094489976, 57.167569407027194, 56.95204605393756, 56.671298528202364, 56.41607350480674, 56.12965431188498, 55.83756345177665, 55.51427842214218, 55.2874117346794, 55.00099254175764, 54.70890168164932, 54.39412415279471, 54.09068995831324, 53.750389927119066, 53.41292572951819, 53.06411819754416, 52.63874315855146, 52.29844312735729, 51.836202251651876, 51.43351388140543, 50.98828800725973, 50.509032129994615, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.2773161670873, 67.4843320193971, 67.62328786546806, 67.69985537248674, 67.8558262201174, 67.93522956072937, 68.04015540368091, 68.09403624195332, 68.12806624507274, 68.15926041459886, 68.17911124975187, 68.14224541303916, 68.11388707710631, 68.00045373337493, 67.95224456228908, 67.88985622323682, 67.76791537872558, 67.65731786858747, 67.58358619516207, 67.46731701783739, 67.30851033661345, 67.21209199444176, 67.0958228171171, 66.94268780307972, 66.7782094546692, 66.66477611093781, 66.51447693049371, 66.3868644187959, 66.1628335649264, 65.99268354932933, 65.8111901993591, 65.63820435016872, 65.45954683379179, 65.292232651788, 65.13059013697075, 64.94909678700054, 64.78461843859003, 64.6399909253325, 64.47834841051528, 64.32521339647789, 64.20327255196665, 64.06431670589569, 63.90267419107847, 63.724016674701524, 63.57371749425743, 63.375209142727506, 63.250432464622975, 63.09162578339903, 62.881774097495956, 62.68893741315259, 62.515951563962226, 62.35714488273828, 62.16430819839491, 61.9998298499844, 61.8325156679806, 61.64251481723052, 61.492215636786426, 61.31639395400278, 61.18594560871168, 61.015795593114596, 60.828630575957796, 60.6045997220883, 60.44579304086436, 60.28131469245385, 60.07713467373734, 59.88146215580069, 59.70564047301704, 59.51847545586024, 59.30295210277061, 59.12713041998695, 58.91160706689731, 58.70175538099425, 58.491903695091175, 58.24802200606868, 58.0211553186059, 57.802796131922975, 57.51921277259451, 57.23846524685931, 56.98324022346368, 56.69682103054192, 56.410401837620164, 56.15801264781782, 55.85174261974307, 55.534129257295184, 55.19382922610101, 54.89323086521283, 54.58696083713808, 54.309049144996166, 54.005614950514705, 53.71352409040637, 53.38173155999206, 53.08680486629045, 52.720982332756705, 52.38918980234239, 51.9978447664691, 51.68873890480107, 51.288886368147914, 50.84082466040892, 50.446643790942346, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.28582366786718, 67.49000368658366, 67.63746703343448, 67.70269120608002, 67.86149788730397, 67.93522956072937, 68.0429912372742, 68.0997079091399, 68.12806624507274, 68.15926041459886, 68.17911124975187, 68.14508124663244, 68.12806624507274, 68.04582707086749, 68.0203045685279, 67.9862745654085, 67.96075206306894, 67.95791622947566, 67.9295578935428, 67.92672205994951, 67.94090122791594, 67.90119955760996, 67.90119955760996, 67.87000538808383, 67.85015455293083, 67.80194538184499, 67.75090037716586, 67.71119870685988, 67.6346311998412, 67.60627286390834, 67.5382128576695, 67.45313784987097, 67.38507784363213, 67.32836117176643, 67.24612199756118, 67.17239032413578, 67.14119615460964, 67.07313614837082, 66.97671780619913, 66.91149363355358, 66.81791112497518, 66.73000028358335, 66.66761194453109, 66.55134276720642, 66.43507358988174, 66.31880441255706, 66.19969940163911, 66.09760939228086, 65.98417604854947, 65.87641437200465, 65.69775685562772, 65.58715934548961, 65.39432266114625, 65.24969514788872, 65.0880526330715, 64.8810367807617, 64.69954343079148, 64.48402007770184, 64.2855117261719, 64.11252587698154, 63.93670419419788, 63.752375010634374, 63.58506082863058, 63.372373309134225, 63.171029124011, 62.96117743810793, 62.72013158267873, 62.51027989677566, 62.215353203074045, 61.97430734764484, 61.690723988316364, 61.44684229929388, 61.191617275898246, 60.87967558063693, 60.62728639083459, 60.352210532285966, 60.02041800187165, 59.756685477696166, 59.41071377931543, 59.138473754360085, 58.758472052859936, 58.426679522445625, 58.038170320165605, 57.65816861866545, 57.30368941950486, 56.92085188441142, 56.58055185321725, 56.21189348609023, 55.79219011428409, 55.29024756827269, 54.87621586365312, 54.41681082154099, 53.974420780988574, 53.4299407310779, 52.981879023338905, 52.46859314295437, 51.92694892663698, 51.30022970252106, 50.65365964325213, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2030, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.1865694921022, 67.37657034285228, 67.53254119048295, 67.6516462014009, 67.78776621387857, 67.85015455293083, 67.90687122479653, 67.94373706150922, 67.97209539744208, 67.98060289822193, 67.98060289822193, 67.97776706462865, 67.9862745654085, 67.98343873181521, 67.9862745654085, 67.98060289822193, 67.97209539744208, 67.96075206306894, 67.95224456228908, 67.94940872869579, 67.92672205994951, 67.92672205994951, 67.86433372089725, 67.84448288574426, 67.80478121543828, 67.74806454357258, 67.72821370841959, 67.66015370218076, 67.60627286390834, 67.52970535688966, 67.4843320193971, 67.41627201315826, 67.35955534129258, 67.2773161670873, 67.23477866318804, 67.14119615460964, 67.03627031165811, 66.90298613277372, 66.8122394577886, 66.71014944843037, 66.61089527266539, 66.52865609846013, 66.41805858832204, 66.25358023991153, 66.09477355868756, 65.96432521339648, 65.75163769390012, 65.60133851345603, 65.45103933301193, 65.29506848538128, 65.18163514164989, 64.98596262371323, 64.82999177608258, 64.6967075971982, 64.52372174800783, 64.3592433995973, 64.18625755040695, 64.00192836684343, 63.811927516093355, 63.63043416612313, 63.40640331225364, 63.19087995916399, 63.017894109973625, 62.844908260783264, 62.61520573972719, 62.385503218671126, 62.17848736636134, 61.91475484218586, 61.59997731333126, 61.33908062274905, 61.183109775118396, 60.947735586875766, 60.65280889317415, 60.38907636899866, 60.125343844823185, 59.84743215268128, 59.60071463006551, 59.3057879363639, 58.95698040438987, 58.633695374755405, 58.32458951308737, 57.99563281626634, 57.67234778663188, 57.337719422624275, 56.9804043898704, 56.59756685477696, 56.28278932592236, 55.85741428692964, 55.44621841590336, 55.0463658792502, 54.58979667073136, 54.144570796585654, 53.676658253693674, 53.25411904829424, 52.743669001502994, 52.23321895471174, 51.69724640558092, 51.15276635567025, 50.58843547060658, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2030, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.07880781555738, 67.19791282647533, 67.31985367098659, 67.47015285143067, 67.59776536312849, 67.68567620452032, 67.75940787794572, 67.782094546692, 67.79343788106513, 67.80194538184499, 67.81612454981142, 67.82746788418456, 67.84448288574426, 67.85299038652411, 67.8756770552704, 67.8558262201174, 67.8388112185577, 67.82463205059128, 67.82746788418456, 67.83030371777784, 67.7991095482517, 67.75373621075916, 67.73104954201288, 67.73955704279273, 67.7083628732666, 67.67716870374046, 67.67716870374046, 67.63746703343448, 67.56089952641578, 67.52403368970309, 67.47015285143067, 67.38507784363213, 67.26880866630745, 67.17806199132235, 67.13268865382979, 67.03627031165811, 66.89447863199388, 66.74417945154978, 66.66194027734453, 66.5002977625273, 66.34716274848992, 66.24223690553838, 66.12596772821371, 66.00402688370247, 65.91328020871735, 65.76581686186654, 65.59850267986275, 65.47372600175821, 65.29506848538128, 65.10223180103792, 64.9008876159147, 64.73357343391089, 64.57476675268695, 64.44148257380256, 64.27700422539205, 64.09267504182854, 63.96506253013074, 63.786405013753786, 63.582224995037286, 63.43192581459319, 63.193715792757274, 63.00655077560049, 62.76550492017128, 62.5386382327085, 62.30042821087258, 62.12460652808893, 61.8892323398463, 61.685052321129795, 61.43833479851401, 61.22564727901767, 61.05549726342058, 60.828630575957796, 60.56773388537561, 60.3068371947934, 60.00623883390522, 59.753849644102885, 59.447579616028136, 59.16683209029294, 58.837875393471904, 58.6365312083487, 58.401157020106055, 58.103394492811155, 57.805631965516255, 57.41712276323625, 57.037121061736094, 56.668462694609076, 56.33667016419477, 55.91129512520205, 55.508606754955615, 55.08890338314948, 54.672035844936616, 54.249496639537185, 53.72203159118622, 53.237104046734544, 52.73516150072314, 52.247398122678156, 51.72276890792048, 51.16410969004339, 50.59410713779315, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "MiniCPM", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.09015114993052, 67.16671865694921, 67.22910699600148, 67.34254033973286, 67.46448118424411, 67.58358619516207, 67.60627286390834, 67.66015370218076, 67.73672120919944, 67.78776621387857, 67.88418455605026, 67.94090122791594, 67.93522956072937, 67.98343873181521, 67.9692595638488, 67.9692595638488, 67.89836372401668, 67.8926920568301, 67.73955704279273, 67.65731786858747, 67.60343703031506, 67.50701868814339, 67.36806284207243, 67.24895783115447, 67.10433031789694, 66.91149363355358, 66.82074695856846, 66.65059694297139, 66.45208859144145, 66.29611774381078, 66.07492272353458, 65.86790687122479, 65.71477185718742, 65.53611434081049, 65.32059098772083, 65.11357513541105, 64.8810367807617, 64.72223009953775, 64.52088591441454, 64.30252672773162, 64.09551087542182, 63.8970025238919, 63.61058333097014, 63.38938831069393, 63.16819329041772, 62.99520744122735, 62.725803249865294, 62.49042906162266, 62.24654737260017, 62.022516518730676, 61.76161982814848, 61.44684229929388, 61.18594560871168, 60.96758642202875, 60.74922723534583, 60.485494711170354, 60.25862802370756, 59.992059665938804, 59.728327141763316, 59.4901171199274, 59.16966792388622, 58.92578623486373, 58.61384453960241, 58.355783682613506, 58.08354365765817, 57.831154467855825, 57.46816776791538, 57.17324107421376, 56.88682188129201, 56.600402688370245, 56.32816266341491, 56.03890763689987, 55.704279272892265, 55.45472591668321, 55.17397839094802, 54.92442503473895, 54.65218500978363, 54.34307914811558, 54.01695828488784, 53.702180756033236, 53.42710489748461, 53.12934237018971, 52.82590817570825, 52.54799648356634, 52.27008479142443, 52.02336726880866, 51.75396307744662, 51.51575305561069, 51.33709553923376, 51.15843802285682, 51.008138842412734, 50.89754133227462, 50.84366049400221, 50.74724215183053, 50.66500297762527, 50.58843547060658, 50.46933045968862, 50.378583784703515, 50.2311204378527, 50.00708958398321]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.56373536000908, 68.11388707710631, 68.638516291864, 69.0837421660097, 69.4921022034427, 69.84374556901001, 70.20106060176388, 70.50165896265207, 70.81927232509996, 71.10001985083515, 71.33539403907778, 71.35808070782407, 71.38076737657035, 71.40345404531662, 71.4062898789099, 71.4233048804696, 71.43748404843603, 71.45449904999575, 71.45166321640247, 71.46867821796216, 71.49420072030172, 71.52539488982787, 71.52823072342115, 71.5537532257607, 71.57643989450699, 71.6133057312197, 71.6104698976264, 71.65017156793239, 71.65300740152568, 71.67002240308538, 71.6671865694921, 71.70972407339137, 71.7607690780705, 71.76927657885035, 71.79196324759663, 71.80614241556306, 71.84017241868247, 71.88270992258174, 71.89688909054817, 71.93091909366758, 71.964949096787, 71.99047159912656, 72.01032243427954, 72.04151660380569, 72.08688994129824, 72.15211411394378, 72.18614411706321, 72.22017412018262, 72.25420412330205, 72.23718912174233, 72.25420412330205, 72.31092079516773, 72.36480163344015, 72.39032413577972, 72.47539914357826, 72.49808581232453, 72.52360831466409, 72.6171908232425, 72.69942999744775, 72.75047500212686, 72.78166917165301, 72.80719167399256, 72.86674417945154, 72.94614752006352, 73.01987919348892, 73.03405836145535, 73.0822675325412, 73.10211836769417, 73.17868587471287, 73.24391004735841, 73.27226838329126, 73.3970450613958, 73.50197090434733, 73.57853841136603, 73.61540424807872, 73.657941751978, 73.79122593086238, 73.85361426991463, 73.91316677537361, 73.97555511442589, 74.01809261832514, 74.0634659558177, 74.16272013158267, 74.2336159714148, 74.3442134815529, 74.39242265263874, 74.36406431670589, 74.36973598389245, 74.16555596517595, 74.16839179876925, 74.27615347531406, 74.3271984799932, 74.38675098545218, 74.3810793182656, 74.42928848935144, 74.54839350026941, 74.56257266823583, 74.57958766979553, 74.54272183308284, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.19791282647533, 67.51552618892323, 67.90119955760996, 68.4144854379945, 68.99299549102459, 69.50344553781585, 70.01389558460708, 70.45344979156623, 70.80792899072682, 71.0915123500553, 71.43748404843603, 71.68136573745852, 71.93942659444743, 72.17763661628337, 72.46972747639168, 72.74196750134702, 73.00286419192922, 73.23256671298527, 73.4877917363809, 73.72033009103025, 73.86212177069449, 73.99257011598559, 74.16839179876925, 74.3810793182656, 74.51719933074327, 74.59093100416867, 74.69302101352693, 74.79511102288518, 74.92839520176956, 74.9567535377024, 75.03332104472109, 75.13541105407936, 75.20630689391147, 75.24033689703089, 75.30272523608315, 75.37645690950855, 75.43600941496753, 75.48421858605336, 75.52675608995264, 75.58630859541162, 75.62033859853103, 75.6657119360236, 75.68556277117658, 75.73093610866914, 75.75929444460199, 75.83586195162067, 75.86705612114682, 75.87839945551995, 75.90675779145279, 75.95213112894535, 75.9975044664379, 76.04854947111703, 76.10810197657601, 76.13362447891558, 76.15347531406856, 76.15631114766185, 76.15063948047528, 76.16198281484843, 76.19601281796784, 76.20735615234098, 76.23855032186711, 76.2413861554604, 76.26123699061338, 76.28675949295295, 76.24705782264697, 76.24422198905368, 76.23855032186711, 76.21302781952754, 76.19601281796784, 76.21869948671412, 76.2215353203074, 76.21586365312083, 76.19601281796784, 76.17049031562829, 76.11944531094915, 76.09108697501631, 76.04571363752375, 76.00884780081105, 75.98332529847148, 75.92944446019906, 75.93228029379236, 75.89257862348637, 75.79048861412812, 75.77347361256841, 75.74795111022885, 75.67138360321016, 75.59198026259818, 75.55511442588548, 75.5154127555795, 75.47571108527352, 75.4048152454414, 75.29705356889657, 75.19496355953832, 75.1269035532995, 75.01914187675467, 74.90287269942999, 74.79227518929189, 74.70436434790007, 74.58809517057539, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.57224286078892, 68.12806624507274, 68.63568045827071, 69.09224966678956, 69.50911720500241, 69.86359640416299, 70.18971726739075, 70.50165896265207, 70.81643649150668, 71.10569151802171, 71.43748404843603, 71.7239032413578, 71.97062076397357, 72.22017412018262, 72.47256330998498, 72.73629583416044, 73.00002835833594, 73.22689504579871, 73.42540339732864, 73.6749567535377, 73.82242010038851, 73.98122678161245, 74.15137679720954, 74.33003431358648, 74.50585599637013, 74.61361767291496, 74.73839435101948, 74.84332019397102, 74.95391770410912, 75.04466437909423, 75.13257522048606, 75.22615772906445, 75.28287440093015, 75.36511357513541, 75.42466608059439, 75.50406942120637, 75.54377109151234, 75.60048776337804, 75.63168193290417, 75.66004026883702, 75.69974193914301, 75.75929444460199, 75.77914527975497, 75.83019028443411, 75.88690695629981, 75.93228029379236, 75.96063862972521, 75.9975044664379, 76.04571363752375, 76.07974364064316, 76.11660947735587, 76.127952811729, 76.15347531406856, 76.19034115078126, 76.21586365312083, 76.19034115078126, 76.17332614922157, 76.1846694835947, 76.19884865156112, 76.24422198905368, 76.25556532342682, 76.27258032498652, 76.28108782576638, 76.2584011570201, 76.26123699061338, 76.2413861554604, 76.25272948983354, 76.27258032498652, 76.24989365624025, 76.24705782264697, 76.23287865468053, 76.23287865468053, 76.24705782264697, 76.21019198593426, 76.19884865156112, 76.22437115390068, 76.2413861554604, 76.17899781640813, 76.15631114766185, 76.11377364376258, 76.08257947423644, 76.09392280860959, 76.05705697189688, 76.02586280237075, 75.98616113206477, 75.9407877945722, 75.88690695629981, 75.83586195162067, 75.7564586110087, 75.6827269375833, 75.61466693134446, 75.51257692198622, 75.41048691262796, 75.34526273998242, 75.20630689391147, 75.05884354706066, 74.92272353458299, 74.79227518929189, 74.6589910104075, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.57224286078892, 68.12806624507274, 68.63568045827071, 69.09224966678956, 69.50911720500241, 69.86359640416299, 70.18971726739075, 70.50165896265207, 70.81643649150668, 71.10569151802171, 71.43748404843603, 71.7239032413578, 71.97062076397357, 72.22017412018262, 72.47256330998498, 72.73629583416044, 73.00002835833594, 73.22689504579871, 73.42540339732864, 73.6749567535377, 73.82242010038851, 73.98122678161245, 74.15137679720954, 74.33003431358648, 74.50585599637013, 74.61361767291496, 74.73839435101948, 74.84332019397102, 74.95391770410912, 75.04466437909423, 75.13257522048606, 75.22615772906445, 75.28287440093015, 75.36511357513541, 75.42466608059439, 75.50406942120637, 75.54377109151234, 75.60048776337804, 75.63168193290417, 75.66004026883702, 75.69974193914301, 75.75929444460199, 75.77914527975497, 75.83019028443411, 75.88690695629981, 75.93228029379236, 75.96063862972521, 75.9975044664379, 76.04571363752375, 76.07974364064316, 76.11660947735587, 76.127952811729, 76.15347531406856, 76.19034115078126, 76.21586365312083, 76.19034115078126, 76.17332614922157, 76.1846694835947, 76.19884865156112, 76.24422198905368, 76.25556532342682, 76.27258032498652, 76.28108782576638, 76.2584011570201, 76.26123699061338, 76.2413861554604, 76.25272948983354, 76.27258032498652, 76.24989365624025, 76.24705782264697, 76.23287865468053, 76.23287865468053, 76.24705782264697, 76.21019198593426, 76.19884865156112, 76.22437115390068, 76.24422198905368, 76.17899781640813, 76.15914698125515, 76.11377364376258, 76.08257947423644, 76.09392280860959, 76.05705697189688, 76.02586280237075, 75.98616113206477, 75.9407877945722, 75.88690695629981, 75.83586195162067, 75.7564586110087, 75.6827269375833, 75.61466693134446, 75.51257692198622, 75.41048691262796, 75.34526273998242, 75.20630689391147, 75.05884354706066, 74.92555936817628, 74.79227518929189, 74.6589910104075, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2026, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.4276153475314, 67.88985622323682, 68.32941043019596, 68.73493463403567, 69.1035930011627, 69.53180387374869, 69.91180557524885, 70.24926977284973, 70.59524147123047, 70.97240733913733, 71.26733403283895, 71.32972237189121, 71.3495732070442, 71.36091654141735, 71.37793154297705, 71.38360321016363, 71.40061821172333, 71.42897654765619, 71.45733488358903, 71.48285738592858, 71.51972322264129, 71.52539488982787, 71.52539488982787, 71.5537532257607, 71.59912656325326, 71.63315656637268, 71.65584323511897, 71.70688823979809, 71.7607690780705, 71.79763491478319, 71.82032158352948, 71.84867991946233, 71.89972492414145, 71.945098261634, 71.97345659756685, 72.03300910302583, 72.06987493973853, 72.10674077645123, 72.15211411394378, 72.18047244987665, 72.22017412018262, 72.25703995689533, 72.29390579360803, 72.31659246235431, 72.3194282959476, 72.35345829906701, 72.36763746703343, 72.38748830218643, 72.393159969373, 72.43853330686555, 72.4668916427984, 72.5377874826305, 72.58599665371635, 72.6710716615149, 72.73913166775372, 72.77316167087315, 72.81286334117914, 72.8752516802314, 72.96599835521651, 72.99719252474264, 73.09361086691433, 73.17017837393301, 73.27226838329126, 73.37435839264953, 73.46226923404134, 73.54734424183988, 73.60973258089216, 73.64376258401157, 73.81391259960866, 73.93585344411989, 74.02376428551173, 74.08048095737742, 74.17973513314239, 74.2903326432805, 74.41227348779174, 74.48600516121714, 74.57391600260897, 74.66466267759408, 74.69585684712021, 74.78660352210532, 74.84332019397102, 74.83764852678445, 74.90003686583671, 74.85182769475087, 74.89436519865014, 74.9397385361427, 74.93690270254942, 74.98227604004197, 74.95958937129569, 75.00212687519496, 75.02764937753452, 74.96809687207553, 74.84332019397102, 74.74973768539262, 74.72421518305306, 74.76108101976577, 74.73839435101948, 74.71287184867992, 74.60794600572838, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2026, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.2773161670873, 67.63746703343448, 68.02881206930778, 68.4882171114199, 68.90224881603947, 69.33613135581204, 69.85225306978987, 70.3088222783087, 70.65762981028273, 70.9979298414769, 71.14255735473442, 71.1652440234807, 71.18225902504041, 71.19643819300683, 71.21912486175312, 71.25882653205909, 71.29285653517852, 71.34673737345092, 71.40345404531662, 71.45166321640247, 71.47434988514874, 71.52255905623458, 71.55942489294728, 71.58211156169357, 71.6303207327794, 71.67852990386524, 71.70972407339137, 71.73524657573094, 71.77211241244363, 71.8345007514959, 71.87987408898846, 71.93091909366758, 71.9479340952273, 71.96778493038029, 72.01599410146613, 72.0755466069251, 72.10390494285795, 72.1322632787908, 72.15778578113036, 72.2059949522162, 72.22300995377591, 72.28539829282818, 72.31375662876103, 72.32226412954088, 72.36480163344015, 72.35912996625358, 72.40450330374614, 72.46405580920512, 72.5037574795111, 72.56614581856337, 72.59733998808949, 72.62569832402235, 72.67390749510818, 72.71928083260075, 72.75614666931345, 72.81002750758586, 72.8951025153844, 72.9489833536568, 73.01987919348892, 73.08510336613448, 73.15316337237331, 73.19286504267929, 73.25808921532484, 73.35167172390325, 73.42823923092193, 73.5246575730936, 73.64659841760486, 73.72033009103025, 73.81674843320194, 73.88480843944077, 73.95003261208632, 74.01525678473186, 74.11734679409012, 74.18824263392224, 74.26197430734764, 74.31301931202677, 74.41227348779174, 74.42645265575815, 74.47749766043728, 74.49451266199699, 74.5512293338627, 74.65331934322094, 74.65048350962765, 74.70720018149335, 74.74123018461276, 74.7894393556986, 74.89720103224342, 74.91421603380314, 74.98227604004197, 75.0134702095681, 75.01063437597482, 75.0134702095681, 74.96242520488897, 74.95391770410912, 74.90854436661657, 74.85749936193744, 74.83197685959789, 74.7327226838329, 74.57391600260897, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.353883674106, 67.73388537560616, 68.0968720755466, 68.4683662762669, 68.83702464339392, 69.25672801520007, 69.60837138076738, 69.98837308226753, 70.33718061424156, 70.70300314777529, 71.03763151178288, 71.37509570938377, 71.71255990698465, 72.04718827099225, 72.3194282959476, 72.6342058248022, 72.86674417945154, 73.12764087003374, 73.35167172390325, 73.56719507699287, 73.81674843320194, 73.98689844879902, 74.14570513002296, 74.33570598077304, 74.52287099792983, 74.67884184556051, 74.78093185491875, 74.90003686583671, 74.98794770722854, 75.1240677197062, 75.22899356265775, 75.35660607435555, 75.46720358449366, 75.48989025323993, 75.57496526103849, 75.64869693446389, 75.70541360632957, 75.76213027819527, 75.7933244477214, 75.83869778521397, 75.84720528599381, 75.87556362192667, 75.94645946175879, 76.01168363440433, 76.02019113518419, 76.03720613674389, 76.09108697501631, 76.1109378101693, 76.08825114142302, 76.14496781328872, 76.12228114454244, 76.12511697813572, 76.16481864844171, 76.1846694835947, 76.19601281796784, 76.2045203187477, 76.19601281796784, 76.2045203187477, 76.19884865156112, 76.22720698749397, 76.24422198905368, 76.23571448827381, 76.21586365312083, 76.17332614922157, 76.17049031562829, 76.17332614922157, 76.17332614922157, 76.147803646882, 76.09675864220287, 76.05705697189688, 76.07974364064316, 76.04004197033719, 76.05138530471032, 76.03720613674389, 76.02869863596405, 76.00884780081105, 75.97481779769163, 75.97765363128491, 75.9237727930125, 75.91526529223265, 75.84436945240053, 75.76780194538185, 75.73377194226242, 75.69123443836315, 75.62317443212432, 75.54944275869892, 75.48421858605336, 75.46720358449366, 75.41899441340783, 75.36511357513541, 75.28854606811673, 75.25168023140402, 75.18929189235175, 75.14675438845248, 75.08153021580694, 74.97376853926212, 74.87735019709044, 74.7724243541389, 74.61645350650824, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.51836202251653, 67.98911039900179, 68.41164960440122, 68.81150214105436, 69.24822051442021, 69.64807305107335, 70.02523891898024, 70.37121061736097, 70.72285398292829, 71.06031818052917, 71.39778237813005, 71.7040524062048, 71.94226242804072, 72.17480078269007, 72.22017412018262, 72.23151745455576, 72.24853245611548, 72.27689079204832, 72.28539829282818, 72.32509996313416, 72.33927913110058, 72.38181663499985, 72.42718997249241, 72.47823497717154, 72.53211581544394, 72.5945041544962, 72.64554915917533, 72.69659416385447, 72.7448033349403, 72.79584833961943, 72.83271417633213, 72.88092334741798, 72.92629668491053, 72.99152085755608, 73.03973002864191, 73.03122252786206, 73.0822675325412, 73.11346170206731, 73.15883503955988, 73.19853670986586, 73.23256671298527, 73.24107421376513, 73.25808921532484, 73.29779088563083, 73.33465672234352, 73.39988089498908, 73.42540339732864, 73.47644840200776, 73.51898590590704, 73.53883674106004, 73.59271757933244, 73.62674758245186, 73.68062842072428, 73.72600175821682, 73.76003176133624, 73.7883900972691, 73.81391259960866, 73.88764427303406, 73.9330176105266, 73.98122678161245, 74.02092845191845, 74.0833167909707, 74.14854096361626, 74.19958596829538, 74.27331764172078, 74.35555681592604, 74.4179451549783, 74.50869182996341, 74.58809517057539, 74.65331934322094, 74.76108101976577, 74.81779769163145, 74.88018603068372, 74.96809687207553, 74.97944020644869, 75.05884354706066, 75.1269035532995, 75.17794855797861, 75.23466522984431, 75.26585939937044, 75.35660607435555, 75.38212857669512, 75.37078524232199, 75.39063607747497, 75.38780024388168, 75.37645690950855, 75.37929274310183, 75.40765107903468, 75.41332274622124, 75.36794940872869, 75.35377024076227, 75.30556106967643, 75.27153106655702, 75.19496355953832, 75.11272438533307, 74.99078354082182, 74.84899186115759, 74.72137934945977, 74.64481184244109, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.18373365850891, 67.43328701471798, 67.8756770552704, 68.36627626690866, 68.94478631993874, 69.46090803391657, 69.94299974477497, 70.34001644783484, 70.72285398292829, 71.04046734537617, 71.32972237189121, 71.64166406715253, 71.92808326007429, 72.18047244987665, 72.45554830842525, 72.72211666619403, 72.89793834897768, 73.15316337237331, 73.37152255905623, 73.56719507699287, 73.7685392621161, 73.94436094489976, 74.09749595893713, 74.28182514250064, 74.45197515809771, 74.57675183620225, 74.67317017837392, 74.82063352522474, 74.9397385361427, 75.04466437909423, 75.10705271814649, 75.19496355953832, 75.26018773218387, 75.35377024076227, 75.42750191418767, 75.48989025323993, 75.51824858917279, 75.56645776025863, 75.60615943056462, 75.64018943368403, 75.71392110710944, 75.75362277741542, 75.78198111334827, 75.82735445084083, 75.84720528599381, 75.8500411195871, 75.88123528911323, 75.95496696253863, 75.99183279925134, 76.02302696877747, 76.04854947111703, 76.05705697189688, 76.05989280549016, 76.04287780393047, 76.02586280237075, 76.03153446955733, 76.01168363440433, 76.01168363440433, 75.96914613050507, 75.99466863284462, 75.98332529847148, 75.98332529847148, 76.02019113518419, 76.02302696877747, 76.01451946799762, 75.99466863284462, 75.98899696565805, 75.97198196409835, 75.96631029691177, 75.90108612426623, 75.8698919547401, 75.83869778521397, 75.81601111646769, 75.76213027819527, 75.76213027819527, 75.74511527663557, 75.719592774296, 75.67421943680344, 75.63735360009073, 75.5891444290049, 75.5721294274452, 75.57780109463177, 75.52959192354592, 75.52959192354592, 75.50123358761309, 75.42466608059439, 75.4019794118481, 75.33108357201598, 75.28287440093015, 75.1977993931316, 75.13824688767264, 75.07585854862036, 75.00212687519496, 74.90287269942999, 74.81779769163145, 74.76958852054561, 74.72421518305306, 74.62779684088137, 74.55690100104925, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.49567535377024, 67.95791622947566, 68.31239542863624, 68.69806879732297, 69.06105549726342, 69.47508720188299, 69.89195474009585, 70.24076227206987, 70.62359980716332, 70.91852650086493, 71.30419986955165, 71.6303207327794, 71.91673992570115, 72.17196494909679, 72.47823497717154, 72.68525082948132, 72.91778918413068, 73.13898420440688, 73.34883589030996, 73.5643592433996, 73.71749425743697, 73.84227093554149, 73.96137594645947, 74.1031676261237, 74.21660096985508, 74.34137764795962, 74.44346765731787, 74.54839350026941, 74.72421518305306, 74.82630519241131, 74.91421603380314, 74.98794770722854, 75.09570938377335, 75.15526188923234, 75.20914272750476, 75.2743669001503, 75.3849644102884, 75.45586025012052, 75.52675608995264, 75.58630859541162, 75.62033859853103, 75.67989110399002, 75.73093610866914, 75.80466778209455, 75.85571278677367, 75.87556362192667, 75.90392195785951, 75.94645946175879, 76.02586280237075, 76.03153446955733, 76.03437030315061, 76.07690780704988, 76.15631114766185, 76.19601281796784, 76.16765448203499, 76.2016844851544, 76.20735615234098, 76.2016844851544, 76.2215353203074, 76.19034115078126, 76.16765448203499, 76.2045203187477, 76.21869948671412, 76.21302781952754, 76.20735615234098, 76.19884865156112, 76.147803646882, 76.10810197657601, 76.10526614298273, 76.05989280549016, 76.0740719734566, 76.07690780704988, 76.05705697189688, 76.04854947111703, 76.03153446955733, 76.03720613674389, 76.04571363752375, 76.04571363752375, 76.02586280237075, 75.97198196409835, 75.94929529535207, 75.92944446019906, 75.8698919547401, 75.76213027819527, 75.76213027819527, 75.70541360632957, 75.64018943368403, 75.55511442588548, 75.47571108527352, 75.39347191106826, 75.31406857045629, 75.24600856421745, 75.16660522360547, 75.08436604940022, 74.96809687207553, 74.87735019709044, 74.77526018773219, 74.70720018149335, 74.58242350338881, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.49567535377024, 67.95791622947566, 68.31239542863624, 68.69806879732297, 69.06105549726342, 69.47508720188299, 69.89195474009585, 70.24076227206987, 70.62359980716332, 70.91852650086493, 71.30419986955165, 71.6303207327794, 71.91673992570115, 72.17196494909679, 72.47823497717154, 72.68525082948132, 72.91778918413068, 73.13898420440688, 73.34883589030996, 73.5643592433996, 73.71749425743697, 73.84227093554149, 73.96137594645947, 74.1031676261237, 74.21660096985508, 74.34137764795962, 74.44346765731787, 74.54839350026941, 74.72421518305306, 74.82630519241131, 74.91421603380314, 74.98794770722854, 75.09570938377335, 75.15526188923234, 75.20914272750476, 75.2743669001503, 75.3849644102884, 75.45586025012052, 75.52675608995264, 75.58630859541162, 75.62033859853103, 75.67989110399002, 75.73093610866914, 75.80466778209455, 75.85571278677367, 75.87556362192667, 75.90392195785951, 75.94645946175879, 76.02586280237075, 76.03153446955733, 76.03437030315061, 76.07690780704988, 76.15631114766185, 76.19601281796784, 76.16765448203499, 76.2016844851544, 76.20735615234098, 76.2016844851544, 76.2215353203074, 76.19034115078126, 76.16765448203499, 76.2045203187477, 76.21869948671412, 76.21302781952754, 76.20735615234098, 76.19884865156112, 76.147803646882, 76.10810197657601, 76.10526614298273, 76.05989280549016, 76.0740719734566, 76.07690780704988, 76.05705697189688, 76.04854947111703, 76.03153446955733, 76.03720613674389, 76.04571363752375, 76.04571363752375, 76.02586280237075, 75.97198196409835, 75.94929529535207, 75.92944446019906, 75.8698919547401, 75.76213027819527, 75.76213027819527, 75.70541360632957, 75.64018943368403, 75.55511442588548, 75.47571108527352, 75.39347191106826, 75.31690440404957, 75.24600856421745, 75.16660522360547, 75.08436604940022, 74.96809687207553, 74.87735019709044, 74.77526018773219, 74.70720018149335, 74.58242350338881, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2027, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.38507784363213, 67.79343788106513, 68.2074695856847, 68.63000879108414, 69.03269716133057, 69.42120636361058, 69.78135722995773, 70.18404560020419, 70.5527039673312, 70.9241981680515, 71.22479652893968, 71.55091739216742, 71.84300825227575, 72.11241244363781, 72.12091994441766, 72.13509911238408, 72.16345744831693, 72.17196494909679, 72.19748745143634, 72.22300995377591, 72.24569662252219, 72.30241329438788, 72.34211496469388, 72.37330913422, 72.40733913733942, 72.45271247483197, 72.50092164591781, 72.5406233162238, 72.57465331934321, 72.60301165527606, 72.6342058248022, 72.65689249354847, 72.69375833026118, 72.72778833338059, 72.75898250290673, 72.78734083883958, 72.80435584039928, 72.81569917477242, 72.85256501148513, 72.86107251226498, 72.8951025153844, 72.92629668491053, 72.97167002240307, 72.96316252162323, 72.98017752318293, 73.00286419192922, 73.01987919348892, 73.05390919660834, 73.0624166973882, 73.0822675325412, 73.09077503332104, 73.13047670362704, 73.17017837393301, 73.26943254969798, 73.34600005671668, 73.41689589654878, 73.46794090122792, 73.5246575730936, 73.6012250801123, 73.69764342228397, 73.74585259336982, 73.76003176133624, 73.84794260272807, 73.93585344411989, 73.98406261520574, 74.03227178629159, 74.0833167909707, 74.17406346595583, 74.22794430422822, 74.2874968096872, 74.35555681592604, 74.43212432294473, 74.474661826844, 74.52570683152312, 74.58809517057539, 74.5852593369821, 74.65331934322094, 74.69869268071349, 74.7894393556986, 74.82063352522474, 74.87451436349714, 74.92555936817628, 74.97376853926212, 74.9567535377024, 74.97376853926212, 74.98511187363526, 75.0701868814338, 75.0872018829935, 75.14108272126592, 75.1269035532995, 75.12123188611292, 75.14675438845248, 75.08153021580694, 75.06451521424722, 74.96809687207553, 74.90003686583671, 74.79511102288518, 74.71854351586649, 74.61928934010153, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2027, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.33970450613957, 67.75657204435244, 68.1734395825653, 68.65269545983041, 69.03836882851714, 69.44672886595015, 69.86076057056971, 70.22941893769674, 70.62359980716332, 70.96389983835748, 71.27300570002552, 71.61614156481298, 71.82315741712276, 71.95076992882058, 72.0216657686527, 72.07838244051838, 72.14360661316394, 72.20032328502964, 72.25703995689533, 72.29390579360803, 72.31375662876103, 72.36196579984686, 72.41584663811928, 72.42718997249241, 72.45838414201855, 72.5037574795111, 72.54913081700366, 72.59166832090293, 72.64271332558205, 72.65405665995519, 72.68808666307461, 72.71644499900745, 72.7249524997873, 72.75614666931345, 72.79017667243286, 72.86107251226498, 72.86958001304484, 72.87241584663812, 72.90361001616425, 72.90361001616425, 72.96032668802994, 73.00286419192922, 73.02838669426878, 73.04823752942177, 73.08793919972776, 73.13047670362704, 73.14465587159346, 73.18719337549273, 73.19853670986586, 73.23823838017185, 73.27794005047782, 73.3204775543771, 73.38286589342937, 73.42256756373536, 73.46226923404134, 73.49913507075405, 73.56719507699287, 73.61540424807872, 73.66928508635114, 73.74585259336982, 73.7855542636758, 73.81674843320194, 73.85077843632135, 73.9160026089669, 73.98406261520574, 74.03794345347815, 74.12585429486998, 74.22227263704166, 74.31018347843347, 74.37540765107903, 74.42645265575815, 74.48884099481042, 74.53421433230298, 74.63630434166123, 74.72988685023962, 74.76108101976577, 74.81496185803816, 74.883021864277, 74.87451436349714, 74.94824603692255, 74.95108187051584, 75.01914187675467, 75.08436604940022, 75.14108272126592, 75.18929189235175, 75.19212772594503, 75.16376939001219, 75.12123188611292, 75.10705271814649, 75.06451521424722, 75.03332104472109, 75.03048521112781, 74.94541020332926, 74.89152936505685, 74.863171029124, 74.82914102600459, 74.7894393556986, 74.72988685023962, 74.58809517057539, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.37089867566571, 67.81328871621814, 68.18761875053173, 68.53926211609902, 68.86538297932677, 69.19433967614779, 69.59986387998752, 69.9940447494541, 70.37121061736097, 70.74554065167456, 71.13688568754786, 71.51405155545473, 71.84017241868247, 72.15494994753708, 72.44987664123869, 72.76465417009331, 73.04540169582849, 73.26659671610469, 73.46794090122792, 73.70047925587726, 73.8819726058475, 74.07197345659756, 74.3243626463999, 74.49167682840371, 74.61928934010153, 74.73839435101948, 74.82063352522474, 74.93406686895612, 75.05884354706066, 75.16093355641891, 75.27153106655702, 75.30839690326971, 75.38212857669512, 75.46153191730708, 75.55511442588548, 75.63451776649745, 75.6628761024303, 75.70257777273629, 75.76496611178855, 75.7961602813147, 75.8131752828744, 75.8500411195871, 75.83869778521397, 75.81033944928112, 75.77347361256841, 75.84436945240053, 75.85571278677367, 75.93228029379236, 75.95496696253863, 75.98616113206477, 75.99183279925134, 75.97481779769163, 75.97481779769163, 75.99466863284462, 75.98899696565805, 76.00884780081105, 75.99183279925134, 75.98616113206477, 75.98616113206477, 75.9804894648782, 75.98899696565805, 75.97198196409835, 75.98899696565805, 75.97765363128491, 75.97481779769163, 75.96914613050507, 75.98616113206477, 75.98616113206477, 75.96063862972521, 75.90108612426623, 75.88407112270652, 75.91526529223265, 75.90959362504609, 75.90959362504609, 75.88974278989309, 75.85571278677367, 75.82735445084083, 75.81033944928112, 75.77630944616169, 75.77347361256841, 75.78765278053483, 75.73660777585572, 75.73377194226242, 75.73377194226242, 75.69123443836315, 75.65153276805717, 75.57780109463177, 75.48138275246008, 75.4048152454414, 75.35093440716899, 75.2772027337436, 75.20347106031818, 75.13824688767264, 75.01914187675467, 74.92839520176956, 74.82630519241131, 74.72988685023962, 74.64764767603437, 74.53705016589626, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.58075036156879, 68.12239457788617, 68.60732212233785, 68.99299549102459, 69.35881802455832, 69.71896889090547, 70.05076142131979, 70.39106145251397, 70.7540481524544, 70.96673567195076, 70.99509400788362, 71.01210900944332, 71.04613901256273, 71.06882568130902, 71.07733318208888, 71.0744973484956, 71.10001985083515, 71.12270651958143, 71.15106485551428, 71.1652440234807, 71.20494569378668, 71.24748319768595, 71.27584153361882, 71.30703570314493, 71.32405070470465, 71.35808070782407, 71.37509570938377, 71.39778237813005, 71.44031988202933, 71.49136488670844, 71.52823072342115, 71.56793239372713, 71.60196239684655, 71.65584323511897, 71.6870374046451, 71.7437540765108, 71.77494824603691, 71.78345574681677, 71.80330658196978, 71.85718742024217, 71.88838158976831, 71.90823242492131, 71.97062076397357, 71.9989790999064, 72.05285993817883, 72.06420327255196, 72.10957661004453, 72.1322632787908, 72.14360661316394, 72.18330828346993, 72.25703995689533, 72.26838329126846, 72.30808496157445, 72.37047330062671, 72.4101749709327, 72.43853330686555, 72.52360831466409, 72.61151915605592, 72.64838499276863, 72.70510166463431, 72.73062416697388, 72.76749000368659, 72.82987834273884, 72.90361001616425, 73.02271502708221, 73.06808836457476, 73.12764087003374, 73.24107421376513, 73.3204775543771, 73.38853756061594, 73.42256756373536, 73.55018007543318, 73.6012250801123, 73.66928508635114, 73.7515242605564, 73.80256926523552, 73.93018177693332, 74.00674928395202, 74.09749595893713, 74.10883929331027, 74.16839179876925, 74.15137679720954, 74.24779513938122, 74.29316847687377, 74.3243626463999, 74.37257181748575, 74.4207809885716, 74.44346765731787, 74.45764682528429, 74.53988599948956, 74.6022743385418, 74.66749851118736, 74.69302101352693, 74.66182684400079, 74.65331934322094, 74.6816776791538, 74.69018517993365, 74.65048350962765, 74.57675183620225, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.18940532569549, 67.46448118424411, 67.88134872245696, 68.44284377392734, 68.9901596574313, 69.50060970422255, 69.99120891586081, 70.37121061736097, 70.72852565011485, 71.04613901256273, 71.36658820860391, 71.66435073589882, 71.93091909366758, 72.22017412018262, 72.46405580920512, 72.76749000368659, 72.9688341888098, 73.17017837393301, 73.38853756061594, 73.5813742449593, 73.75436009414967, 73.96704761364603, 74.11734679409012, 74.25630264016108, 74.40376598701188, 74.52287099792983, 74.59376683776196, 74.68451351274707, 74.76958852054561, 74.84332019397102, 74.91705186739642, 74.95391770410912, 75.03048521112781, 75.11556021892635, 75.13257522048606, 75.18929189235175, 75.22048606187789, 75.26585939937044, 75.3282477384227, 75.3849644102884, 75.42466608059439, 75.49272608683322, 75.53809942432578, 75.59198026259818, 75.64018943368403, 75.69407027195643, 75.71108527351615, 75.76213027819527, 75.77347361256841, 75.83869778521397, 75.86138445396024, 75.88407112270652, 75.9407877945722, 75.95496696253863, 76.00317613362448, 76.02586280237075, 76.02869863596405, 76.0542211383036, 76.08541530782973, 76.10526614298273, 76.12228114454244, 76.08541530782973, 76.09959447579617, 76.11377364376258, 76.11377364376258, 76.10810197657601, 76.13078864532228, 76.14496781328872, 76.16765448203499, 76.14496781328872, 76.13078864532228, 76.09108697501631, 76.09108697501631, 76.0740719734566, 76.01451946799762, 75.9804894648782, 75.95213112894535, 75.90108612426623, 75.88974278989309, 75.87556362192667, 75.86138445396024, 75.80750361568784, 75.77914527975497, 75.69690610554971, 75.65153276805717, 75.62033859853103, 75.56645776025863, 75.5324277571392, 75.45018858293395, 75.39063607747497, 75.34242690638912, 75.25168023140402, 75.16944105719875, 75.11272438533307, 75.02764937753452, 74.94541020332926, 74.84332019397102, 74.7724243541389, 74.66182684400079, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.60627286390834, 68.1535887474123, 68.58463545359157, 69.01851799336414, 69.34747469018518, 69.70478972293905, 70.05926892209965, 70.38538978532739, 70.72001814933499, 71.05181067974931, 71.36658820860391, 71.69554490542495, 72.01315826787284, 72.2995774607946, 72.5434591498171, 72.75331083572016, 72.95749085443666, 73.21555171142558, 73.35734339108981, 73.5246575730936, 73.6749567535377, 73.8989876074072, 74.07197345659756, 74.17122763236253, 74.2704818081275, 74.40376598701188, 74.51436349714999, 74.60511017213508, 74.72137934945977, 74.81212602444488, 74.91705186739642, 75.01914187675467, 75.11272438533307, 75.22899356265775, 75.28854606811673, 75.31406857045629, 75.37078524232199, 75.41899441340783, 75.45586025012052, 75.51257692198622, 75.60615943056462, 75.6628761024303, 75.71392110710944, 75.73660777585572, 75.83302611802739, 75.86422028755354, 75.90392195785951, 75.92660862660578, 75.98899696565805, 76.02019113518419, 76.08825114142302, 76.11377364376258, 76.15063948047528, 76.16765448203499, 76.21019198593426, 76.23855032186711, 76.2584011570201, 76.2952669937328, 76.28959532654623, 76.26974449139324, 76.2782519921731, 76.27258032498652, 76.27541615857982, 76.2782519921731, 76.28675949295295, 76.30377449451267, 76.29810282732609, 76.30661032810595, 76.27541615857982, 76.26974449139324, 76.23855032186711, 76.24705782264697, 76.2413861554604, 76.24705782264697, 76.24705782264697, 76.2045203187477, 76.16481864844171, 76.09108697501631, 76.02869863596405, 75.9407877945722, 75.90108612426623, 75.84153361880725, 75.81601111646769, 75.76496611178855, 75.67989110399002, 75.61466693134446, 75.58347276181834, 75.49272608683322, 75.4586960837138, 75.38780024388168, 75.33959107279584, 75.29988940248987, 75.26869523296374, 75.16944105719875, 75.11556021892635, 75.04182854550095, 74.96242520488897, 74.87167852990386, 74.7355585174262, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.60627286390834, 68.1535887474123, 68.58463545359157, 69.01851799336414, 69.34747469018518, 69.70478972293905, 70.05926892209965, 70.38538978532739, 70.72001814933499, 71.05181067974931, 71.36658820860391, 71.69554490542495, 72.01315826787284, 72.2995774607946, 72.5434591498171, 72.75331083572016, 72.95749085443666, 73.21555171142558, 73.35734339108981, 73.5246575730936, 73.6749567535377, 73.8989876074072, 74.07197345659756, 74.17122763236253, 74.2704818081275, 74.40376598701188, 74.51436349714999, 74.60511017213508, 74.72137934945977, 74.81212602444488, 74.91705186739642, 75.01914187675467, 75.11272438533307, 75.22899356265775, 75.28854606811673, 75.31406857045629, 75.37078524232199, 75.41899441340783, 75.45586025012052, 75.51257692198622, 75.60615943056462, 75.6628761024303, 75.71392110710944, 75.73660777585572, 75.83302611802739, 75.86422028755354, 75.90392195785951, 75.92660862660578, 75.98899696565805, 76.02019113518419, 76.08825114142302, 76.11377364376258, 76.15063948047528, 76.16765448203499, 76.21019198593426, 76.23855032186711, 76.2584011570201, 76.2952669937328, 76.28959532654623, 76.26974449139324, 76.2782519921731, 76.27258032498652, 76.27541615857982, 76.2782519921731, 76.28675949295295, 76.30377449451267, 76.29810282732609, 76.30661032810595, 76.27541615857982, 76.26974449139324, 76.23855032186711, 76.24705782264697, 76.2413861554604, 76.24705782264697, 76.24705782264697, 76.2045203187477, 76.16481864844171, 76.09108697501631, 76.02869863596405, 75.9407877945722, 75.90108612426623, 75.84153361880725, 75.81601111646769, 75.76496611178855, 75.67989110399002, 75.61466693134446, 75.58347276181834, 75.49272608683322, 75.4586960837138, 75.38780024388168, 75.33959107279584, 75.29988940248987, 75.26869523296374, 75.16944105719875, 75.11556021892635, 75.04182854550095, 74.95958937129569, 74.87167852990386, 74.7355585174262, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2028, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.39925701159855, 67.8728412216771, 68.33224626378924, 68.79165130590137, 69.18583217536795, 69.58852054561439, 69.93449224399512, 70.291807276749, 70.60091313841703, 70.93837733601792, 71.23897569690611, 71.52539488982787, 71.69554490542495, 71.70972407339137, 71.73524657573094, 71.7409182429175, 71.76644074525707, 71.76644074525707, 71.7607690780705, 71.77494824603691, 71.78629158041005, 71.78345574681677, 71.80047074837648, 71.80047074837648, 71.85151575305561, 71.8713665882086, 71.88270992258174, 71.88270992258174, 71.90539659132801, 71.93659076085414, 71.95644159600715, 71.98479993193999, 72.01315826787284, 72.03584493661911, 72.10106910926467, 72.11808411082437, 72.14644244675722, 72.18897995065649, 72.21450245299606, 72.21733828658934, 72.26554745767518, 72.32226412954088, 72.34495079828716, 72.35912996625358, 72.41584663811928, 72.48107081076482, 72.52644414825737, 72.56898165215665, 72.6171908232425, 72.6342058248022, 72.66539999432833, 72.73629583416044, 72.8185350083657, 72.83271417633213, 72.88659501460454, 72.94614752006352, 72.99435669114936, 73.03689419504863, 73.03405836145535, 73.08510336613448, 73.12196920284717, 73.14465587159346, 73.17017837393301, 73.21838754501886, 73.27794005047782, 73.31480588719053, 73.36868672546295, 73.42823923092193, 73.48212006919434, 73.48495590278762, 73.57286674417945, 73.62958341604514, 73.6948075886907, 73.74301675977654, 73.80256926523552, 73.90465927459377, 73.96421178005275, 74.0095851175453, 74.07764512378414, 74.11734679409012, 74.1598842979894, 74.16272013158267, 74.29316847687377, 74.29600431046705, 74.35555681592604, 74.36973598389245, 74.38958681904546, 74.37540765107903, 74.38958681904546, 74.35555681592604, 74.31018347843347, 74.36122848311261, 74.4179451549783, 74.53421433230298, 74.61078183932166, 74.66182684400079, 74.68734934634035, 74.67033434478064, 74.63914017525451, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2028, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.35671950769928, 67.71119870685988, 68.11672291069961, 68.57329211921844, 68.9901596574313, 69.38150469330459, 69.75867056121146, 69.99688058304739, 70.10464225959221, 70.1613589314579, 70.21240393613702, 70.24076227206987, 70.25777727362959, 70.28329977596914, 70.30315061112213, 70.33150894705499, 70.35986728298784, 70.36837478376768, 70.40524062048038, 70.4222556220401, 70.45628562515951, 70.47613646031252, 70.48464396109236, 70.52434563139835, 70.58389813685733, 70.62927147434989, 70.67180897824915, 70.72852565011485, 70.77389898760741, 70.78524232198055, 70.83628732665967, 70.84195899384625, 70.87031732977908, 70.91285483367837, 70.95822817117092, 70.96389983835748, 71.00927317585004, 71.04046734537617, 71.10569151802171, 71.19076652582027, 71.19643819300683, 71.24748319768595, 71.30703570314493, 71.34106570626435, 71.39778237813005, 71.4233048804696, 71.48852905311516, 71.53106655701443, 71.58494739528685, 71.6303207327794, 71.68136573745852, 71.71255990698465, 71.7607690780705, 71.76644074525707, 71.81464991634292, 71.80897824915634, 71.80330658196978, 71.83733658508919, 71.85718742024217, 71.89688909054817, 71.87703825539518, 71.92808326007429, 71.97345659756685, 72.01882993505941, 72.06136743895868, 72.10106910926467, 72.16062161472364, 72.19181578424978, 72.22017412018262, 72.25703995689533, 72.24853245611548, 72.24002495533561, 72.29674162720131, 72.29390579360803, 72.3562941326603, 72.39599580296628, 72.40733913733942, 72.44987664123869, 72.47256330998498, 72.50659331310439, 72.55196665059694, 72.64554915917533, 72.7249524997873, 72.7986841732127, 72.9489833536568, 73.10211836769417, 73.29211921844426, 73.42823923092193, 73.53883674106004, 73.68630008791084, 73.84510676913479, 73.98122678161245, 74.04928678785129, 74.13152596205654, 74.2903326432805, 74.38391515185889, 74.47749766043728, 74.52287099792983, 74.48600516121714, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.32552533817315, 67.76791537872558, 68.0770212403936, 68.45418710830047, 68.7462779684088, 69.11210050194254, 69.46941553469642, 69.86076057056971, 70.26628477440944, 70.68598814621558, 71.06315401412246, 71.37509570938377, 71.73241074213766, 72.07271077333182, 72.41584663811928, 72.67957916229474, 72.90361001616425, 73.19286504267929, 73.4707767348212, 73.68913592150412, 73.88764427303406, 74.05779428863114, 74.2336159714148, 74.3810793182656, 74.52854266511642, 74.6022743385418, 74.7157076822732, 74.76391685335905, 74.8461560275643, 74.90570853302327, 74.98511187363526, 75.05600771346738, 75.12123188611292, 75.18645605875847, 75.25735189859058, 75.33108357201598, 75.39063607747497, 75.41899441340783, 75.41332274622124, 75.40765107903468, 75.42466608059439, 75.47287525168022, 75.50690525479965, 75.55795025947877, 75.60048776337804, 75.64869693446389, 75.63168193290417, 75.64869693446389, 75.6657119360236, 75.65720443524374, 75.67138360321016, 75.68839860476987, 75.719592774296, 75.71675694070272, 75.76213027819527, 75.76496611178855, 75.73660777585572, 75.71108527351615, 75.75078694382213, 75.76213027819527, 75.77914527975497, 75.7564586110087, 75.77347361256841, 75.78198111334827, 75.7961602813147, 75.79048861412812, 75.75078694382213, 75.69123443836315, 75.69407027195643, 75.69690610554971, 75.6827269375833, 75.65436860165046, 75.63735360009073, 75.6260102657176, 75.62317443212432, 75.61466693134446, 75.59765192978476, 75.58063692822505, 75.56362192666535, 75.52108442276607, 75.49272608683322, 75.46436775090038, 75.43884524856081, 75.37362107591527, 75.33959107279584, 75.30839690326971, 75.26302356577716, 75.23466522984431, 75.18645605875847, 75.1269035532995, 75.06735104784052, 74.98511187363526, 74.91705186739642, 74.82630519241131, 74.76391685335905, 74.66749851118736, 74.59376683776196, 74.52287099792983, 74.50018432918355, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.52119785610981, 68.07134957320704, 68.47687377704676, 68.94195048634546, 69.35598219096504, 69.71329722371891, 70.07061225647278, 70.38255395173411, 70.69165981340215, 70.99509400788362, 71.25031903127925, 71.51688738904801, 71.81181408274962, 71.98763576553327, 72.01882993505941, 72.01315826787284, 72.01315826787284, 72.01882993505941, 72.03300910302583, 72.05002410458555, 72.08121827411166, 72.08688994129824, 72.10106910926467, 72.12091994441766, 72.13509911238408, 72.14077077957064, 72.16629328191021, 72.2059949522162, 72.20883078580948, 72.22868162096248, 72.2825624592349, 72.2995774607946, 72.32793579672744, 72.33927913110058, 72.35912996625358, 72.39883163655956, 72.44136914045883, 72.46972747639168, 72.52077248107081, 72.54913081700366, 72.55480248419023, 72.61435498964921, 72.65405665995519, 72.66256416073504, 72.71077333182089, 72.75047500212686, 72.77599750446643, 72.81002750758586, 72.83838584351871, 72.90361001616425, 72.97450585599637, 73.05958086379492, 73.15316337237331, 73.23256671298527, 73.29495505203755, 73.34316422312338, 73.41122422936222, 73.51047840512719, 73.58988174573916, 73.63809091682499, 73.6948075886907, 73.71182259025039, 73.79406176445566, 73.88764427303406, 73.95286844567961, 74.03227178629159, 74.12301846127669, 74.1570484643961, 74.20242180188866, 74.29884014406034, 74.33570598077304, 74.37824348467231, 74.4009301534186, 74.46615432606414, 74.5512293338627, 74.6589910104075, 74.7157076822732, 74.74123018461276, 74.71003601508663, 74.75824518617247, 74.85182769475087, 74.863171029124, 74.91138020020985, 74.95391770410912, 75.01630604316138, 75.02764937753452, 75.01063437597482, 75.05884354706066, 75.05884354706066, 75.09003771658679, 75.0872018829935, 75.03899271190765, 75.00779854238154, 74.99078354082182, 74.9198877009897, 74.78376768851204, 74.75824518617247, 74.77809602132547, 74.62779684088137, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.20925616084847, 67.50985452173667, 67.86433372089725, 68.38045543487507, 68.94195048634546, 69.49493803703598, 69.99688058304739, 70.40524062048038, 70.77106315401413, 71.09434818364858, 71.32972237189121, 71.59912656325326, 71.79196324759663, 72.02450160224598, 72.26554745767518, 72.54629498341038, 72.82420667555228, 73.0425658622352, 73.26092504891812, 73.48212006919434, 73.71749425743697, 73.91316677537361, 74.1031676261237, 74.2336159714148, 74.33854181436634, 74.51719933074327, 74.60794600572838, 74.69585684712021, 74.8092901908516, 74.86033519553072, 74.87451436349714, 74.93690270254942, 74.96526103848225, 74.99078354082182, 75.0701868814338, 75.12123188611292, 75.12973938689278, 75.15809772282563, 75.20630689391147, 75.22048606187789, 75.28854606811673, 75.35660607435555, 75.35944190794885, 75.42750191418767, 75.37645690950855, 75.39914357825482, 75.37929274310183, 75.35944190794885, 75.35944190794885, 75.36794940872869, 75.37078524232199, 75.38780024388168, 75.4019794118481, 75.37078524232199, 75.41332274622124, 75.43884524856081, 75.47287525168022, 75.48138275246008, 75.48989025323993, 75.47287525168022, 75.47571108527352, 75.47287525168022, 75.4586960837138, 75.48989025323993, 75.45018858293395, 75.46436775090038, 75.47003941808694, 75.43884524856081, 75.46436775090038, 75.44735274934067, 75.42466608059439, 75.43317358137425, 75.41048691262796, 75.39914357825482, 75.3849644102884, 75.37362107591527, 75.34526273998242, 75.31406857045629, 75.29988940248987, 75.26018773218387, 75.22332189547117, 75.21197856109804, 75.17511272438533, 75.12123188611292, 75.08436604940022, 75.04750021268751, 74.9936193744151, 74.9567535377024, 74.91138020020985, 74.85749936193744, 74.81496185803816, 74.74973768539262, 74.71854351586649, 74.68734934634035, 74.67033434478064, 74.65615517681422, 74.61361767291496, 74.5852593369821, 74.55690100104925, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.40209284519185, 67.84731871933755, 68.27836542551682, 68.71791963247597, 69.12911550350225, 69.54314720812182, 69.90046224087571, 70.26061310722287, 70.60374897201032, 70.91001900008507, 71.2021098601934, 71.49136488670844, 71.77211241244363, 72.02450160224598, 72.2797266256416, 72.58032498652979, 72.8752516802314, 73.07376003176134, 73.3403283895301, 73.58704591214588, 73.79406176445566, 73.96137594645947, 74.1598842979894, 74.2903326432805, 74.45764682528429, 74.63914017525451, 74.77526018773219, 74.87735019709044, 74.96526103848225, 75.01914187675467, 75.03332104472109, 75.12123188611292, 75.16660522360547, 75.23750106343759, 75.29705356889657, 75.33959107279584, 75.33959107279584, 75.36511357513541, 75.37929274310183, 75.39630774466154, 75.42750191418767, 75.43884524856081, 75.48138275246008, 75.53809942432578, 75.59765192978476, 75.61183109775118, 75.60615943056462, 75.5891444290049, 75.6089952641579, 75.63168193290417, 75.61466693134446, 75.61466693134446, 75.64302526727731, 75.62033859853103, 75.63451776649745, 75.64302526727731, 75.62317443212432, 75.63168193290417, 75.67421943680344, 75.67705527039674, 75.68839860476987, 75.69690610554971, 75.67989110399002, 75.69974193914301, 75.69407027195643, 75.64869693446389, 75.65436860165046, 75.6657119360236, 75.65153276805717, 75.66854776961688, 75.65720443524374, 75.63451776649745, 75.60615943056462, 75.62033859853103, 75.61750276493775, 75.58347276181834, 75.5522785922922, 75.5154127555795, 75.46153191730708, 75.43884524856081, 75.42183024700111, 75.3480985735757, 75.3282477384227, 75.29988940248987, 75.26018773218387, 75.20630689391147, 75.10138105095993, 75.05033604628079, 75.00496270878824, 74.93406686895612, 74.8660068627173, 74.79794685647846, 74.76108101976577, 74.72137934945977, 74.68451351274707, 74.63346850806795, 74.61645350650824, 74.56540850182911, 74.53705016589626, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.40209284519185, 67.84731871933755, 68.27836542551682, 68.71791963247597, 69.12911550350225, 69.54314720812182, 69.90046224087571, 70.26061310722287, 70.60374897201032, 70.91001900008507, 71.2021098601934, 71.49136488670844, 71.77211241244363, 72.02450160224598, 72.2797266256416, 72.58032498652979, 72.8752516802314, 73.07376003176134, 73.3403283895301, 73.58704591214588, 73.79406176445566, 73.96137594645947, 74.1598842979894, 74.2903326432805, 74.45764682528429, 74.63914017525451, 74.77526018773219, 74.87735019709044, 74.96526103848225, 75.01914187675467, 75.03332104472109, 75.12123188611292, 75.16660522360547, 75.23750106343759, 75.29705356889657, 75.33959107279584, 75.33959107279584, 75.36511357513541, 75.37929274310183, 75.39630774466154, 75.42750191418767, 75.43884524856081, 75.48138275246008, 75.53809942432578, 75.59765192978476, 75.61183109775118, 75.60615943056462, 75.5891444290049, 75.6089952641579, 75.63168193290417, 75.61466693134446, 75.61466693134446, 75.64302526727731, 75.62033859853103, 75.63451776649745, 75.64302526727731, 75.62317443212432, 75.63168193290417, 75.67421943680344, 75.67705527039674, 75.68839860476987, 75.69690610554971, 75.67989110399002, 75.69974193914301, 75.69407027195643, 75.64869693446389, 75.65436860165046, 75.6657119360236, 75.65153276805717, 75.66854776961688, 75.65720443524374, 75.63451776649745, 75.60615943056462, 75.62033859853103, 75.61750276493775, 75.58347276181834, 75.5522785922922, 75.5154127555795, 75.46153191730708, 75.43884524856081, 75.42183024700111, 75.3480985735757, 75.3282477384227, 75.29988940248987, 75.26018773218387, 75.20630689391147, 75.10138105095993, 75.05033604628079, 75.00496270878824, 74.93406686895612, 74.8660068627173, 74.79794685647846, 74.76108101976577, 74.72137934945977, 74.68451351274707, 74.63346850806795, 74.61645350650824, 74.56540850182911, 74.53705016589626, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2029, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.32836117176643, 67.70269120608002, 68.05717040524061, 68.4144854379945, 68.77180047074837, 69.12627966990897, 69.4921022034427, 69.82105890026374, 70.18688143379747, 70.56404730170433, 70.9412131696112, 71.36658820860391, 71.72673907495108, 71.77211241244363, 71.78912741400335, 71.80614241556306, 71.83733658508919, 71.86853075461532, 71.89405325695488, 71.92241159288773, 71.95360576241386, 71.98763576553327, 72.00465076709298, 72.01882993505941, 72.03300910302583, 72.05853160536539, 72.08972577489152, 72.12659161160423, 72.16345744831693, 72.19748745143634, 72.21166661940276, 72.25420412330205, 72.29106996001474, 72.32509996313416, 72.35062246547372, 72.39599580296628, 72.42151830530584, 72.44420497405211, 72.43853330686555, 72.47539914357826, 72.53495164903723, 72.5604741513768, 72.58883248730965, 72.60301165527606, 72.62286249042906, 72.64838499276863, 72.6710716615149, 72.67390749510818, 72.71928083260075, 72.76749000368659, 72.78734083883958, 72.79017667243286, 72.84405751070527, 72.90644584975755, 72.93764001928366, 72.98017752318293, 73.01420752630236, 73.02271502708221, 73.05390919660834, 73.1162975356606, 73.18152170830615, 73.22405921220543, 73.25525338173155, 73.30913422000397, 73.3771942262428, 73.44241839888835, 73.46510506763462, 73.5076425715339, 73.53316507387346, 73.5643592433996, 73.6012250801123, 73.62391174885857, 73.66928508635114, 73.77704676289595, 73.80824093242208, 73.84794260272807, 73.83376343476165, 73.9160026089669, 73.9330176105266, 73.99824178317216, 74.04928678785129, 74.14570513002296, 74.16555596517595, 74.17406346595583, 74.22510847063494, 74.25630264016108, 74.31869097921334, 74.37824348467231, 74.474661826844, 74.56824433542239, 74.62212517369481, 74.64764767603437, 74.65331934322094, 74.65615517681422, 74.65048350962765, 74.62779684088137, 74.6589910104075, 74.61928934010153, 74.51436349714999, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2029, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.45880951705755, 67.95508039588238, 68.45418710830047, 68.89941298244618, 69.32195218784561, 69.6395655502935, 69.96568641352125, 70.26061310722287, 70.58673397045061, 70.90151149930522, 71.17091569066727, 71.38076737657035, 71.47718571874202, 71.536738224201, 71.59629072965998, 71.63599239996597, 71.69554490542495, 71.72673907495108, 71.76644074525707, 71.82315741712276, 71.84867991946233, 71.8912174233616, 71.93375492726086, 71.98479993193999, 72.01882993505941, 72.05285993817883, 72.0925616084848, 72.14360661316394, 72.18330828346993, 72.21733828658934, 72.28539829282818, 72.3194282959476, 72.37047330062671, 72.40166747015286, 72.44987664123869, 72.47539914357826, 72.5377874826305, 72.58883248730965, 72.62853415761563, 72.65689249354847, 72.71928083260075, 72.76181833650001, 72.78734083883958, 72.8185350083657, 72.87808751382468, 72.88943084819782, 72.93480418569038, 72.98301335677623, 73.01137169270908, 73.05958086379492, 73.05674503020163, 73.10779003488075, 73.13047670362704, 73.16450670674645, 73.20137254345916, 73.229730879392, 73.25525338173155, 73.25808921532484, 73.25808921532484, 73.26943254969798, 73.26943254969798, 73.26943254969798, 73.29495505203755, 73.36585089186967, 73.38286589342937, 73.3970450613958, 73.43958256529507, 73.4877917363809, 73.54734424183988, 73.6012250801123, 73.66077758557128, 73.72883759181012, 73.81958426679522, 73.87630093866092, 73.92451010974675, 73.9557042792729, 73.99540594957887, 74.06063012222442, 74.10883929331027, 74.1939143011088, 74.22510847063494, 74.26481014094092, 74.24779513938122, 74.30734764484019, 74.36690015029917, 74.40943765419846, 74.44913932450443, 74.49167682840371, 74.54272183308284, 74.59093100416867, 74.6022743385418, 74.59943850494852, 74.59943850494852, 74.56257266823583, 74.60794600572838, 74.56257266823583, 74.63063267447467, 74.62212517369481, 74.6022743385418, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.33686867254629, 67.78493038028527, 68.21314125287128, 68.60732212233785, 68.92493548478575, 69.21986217848738, 69.54314720812182, 69.9373280775884, 70.18971726739075, 70.46195729234609, 70.72001814933499, 70.98658650710377, 71.22763236253297, 71.65300740152568, 72.01315826787284, 72.36763746703343, 72.6682358279216, 72.93764001928366, 73.14749170518674, 73.40838839576892, 73.62391174885857, 73.80824093242208, 73.95003261208632, 74.10883929331027, 74.2336159714148, 74.40660182060516, 74.49451266199699, 74.62496100728809, 74.76391685335905, 74.89436519865014, 75.00779854238154, 75.09287355018007, 75.13824688767264, 75.21481439469132, 75.26869523296374, 75.35660607435555, 75.43600941496753, 75.44735274934067, 75.50690525479965, 75.53809942432578, 75.64018943368403, 75.69407027195643, 75.73660777585572, 75.74795111022885, 75.73093610866914, 75.77347361256841, 75.76780194538185, 75.7961602813147, 75.82168278365425, 75.83302611802739, 75.86422028755354, 75.90108612426623, 75.91810112582593, 75.93795196097892, 75.94645946175879, 75.93795196097892, 75.95213112894535, 75.98899696565805, 75.98899696565805, 75.99183279925134, 75.98332529847148, 75.97765363128491, 75.96347446331849, 75.9436236281655, 75.92944446019906, 75.94929529535207, 75.92944446019906, 75.92944446019906, 75.92660862660578, 75.90675779145279, 75.90392195785951, 75.90392195785951, 75.88690695629981, 75.87272778833338, 75.87272778833338, 75.85854862036695, 75.84720528599381, 75.83869778521397, 75.7933244477214, 75.77914527975497, 75.76213027819527, 75.71675694070272, 75.65153276805717, 75.6260102657176, 75.59198026259818, 75.52959192354592, 75.46153191730708, 75.39063607747497, 75.35377024076227, 75.3282477384227, 75.29421773530329, 75.25735189859058, 75.17511272438533, 75.12123188611292, 75.04182854550095, 74.98511187363526, 74.90003686583671, 74.79794685647846, 74.64764767603437, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.56657119360236, 68.12806624507274, 68.62433712389758, 69.03836882851714, 69.45240053313671, 69.81255139948388, 70.15285143067804, 70.41091228766696, 70.74837648526784, 71.04330317896945, 71.27584153361882, 71.34390153985764, 71.34673737345092, 71.34390153985764, 71.35240904063748, 71.36375237501063, 71.36091654141735, 71.36658820860391, 71.36942404219721, 71.38076737657035, 71.38643904375691, 71.38927487735019, 71.4233048804696, 71.43748404843603, 71.44882738280917, 71.47434988514874, 71.52255905623458, 71.56226072654056, 71.58211156169357, 71.60196239684655, 71.65867906871225, 71.67852990386524, 71.72673907495108, 71.74942574369736, 71.79763491478319, 71.84017241868247, 71.90256075773473, 71.92524742648101, 71.95360576241386, 71.99047159912656, 72.01599410146613, 72.06136743895868, 72.09823327567139, 72.14360661316394, 72.17196494909679, 72.18047244987665, 72.19748745143634, 72.24002495533561, 72.27689079204832, 72.2627116240819, 72.32226412954088, 72.35912996625358, 72.38465246859313, 72.393159969373, 72.40166747015286, 72.43569747327227, 72.46405580920512, 72.5037574795111, 72.54913081700366, 72.60017582168278, 72.63987749198877, 72.70510166463431, 72.73913166775372, 72.79301250602614, 72.85256501148513, 72.91495335053739, 72.9858491903695, 73.03689419504863, 73.08510336613448, 73.09361086691433, 73.14465587159346, 73.21838754501886, 73.29779088563083, 73.37152255905623, 73.39137339420923, 73.45659756685478, 73.55301590902646, 73.6210759152653, 73.64376258401157, 73.72600175821682, 73.74018092618326, 73.77704676289595, 73.83376343476165, 73.86779343788106, 73.92451010974675, 74.05495845503786, 74.086152624564, 74.15137679720954, 74.22794430422822, 74.3442134815529, 74.45764682528429, 74.55690100104925, 74.59660267135524, 74.66182684400079, 74.70152851430679, 74.74123018461276, 74.7327226838329, 74.72421518305306, 74.59093100416867, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.17239032413578, 67.46731701783739, 67.84164705215098, 68.36911210050194, 68.90508464963276, 69.42120636361058, 69.90046224087571, 70.34285228142812, 70.70300314777529, 71.00643734225676, 71.2786773672121, 71.49420072030172, 71.7409182429175, 72.01599410146613, 72.28823412642146, 72.57465331934321, 72.81569917477242, 73.10495420128747, 73.28361171766441, 73.52182173950033, 73.71749425743697, 73.84794260272807, 74.07480929019084, 74.20525763548194, 74.34988514873947, 74.50585599637013, 74.6022743385418, 74.6816776791538, 74.76958852054561, 74.883021864277, 74.96809687207553, 75.00496270878824, 75.03615687831437, 75.02197771034795, 75.05317187987409, 75.06167938065394, 75.08436604940022, 75.15526188923234, 75.19496355953832, 75.26018773218387, 75.30839690326971, 75.34242690638912, 75.33675523920256, 75.37929274310183, 75.39347191106826, 75.42466608059439, 75.4019794118481, 75.43884524856081, 75.47571108527352, 75.47571108527352, 75.45302441652724, 75.48705441964665, 75.50690525479965, 75.53809942432578, 75.54944275869892, 75.54944275869892, 75.56645776025863, 75.56929359385191, 75.58347276181834, 75.60048776337804, 75.59481609619148, 75.61466693134446, 75.64586110087059, 75.65436860165046, 75.6628761024303, 75.69407027195643, 75.68556277117658, 75.7224286078893, 75.76780194538185, 75.79048861412812, 75.77063777897513, 75.74227944304228, 75.76213027819527, 75.75362277741542, 75.76780194538185, 75.78765278053483, 75.77630944616169, 75.76780194538185, 75.75078694382213, 75.75929444460199, 75.7933244477214, 75.77347361256841, 75.73093610866914, 75.69974193914301, 75.67421943680344, 75.64018943368403, 75.63451776649745, 75.60615943056462, 75.5721294274452, 75.52959192354592, 75.45302441652724, 75.38780024388168, 75.31406857045629, 75.26585939937044, 75.20347106031818, 75.15809772282563, 75.07869438221365, 74.95391770410912, 74.74123018461276, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.49567535377024, 68.06567790602047, 68.55911295125202, 68.99583132461787, 69.39284802767773, 69.75583472761818, 70.09897059240564, 70.41941978844682, 70.72285398292829, 71.00076567507018, 71.27300570002552, 71.49703655389501, 71.78345574681677, 72.05853160536539, 72.25987579048862, 72.5604741513768, 72.8554008450784, 73.10495420128747, 73.36585089186967, 73.58421007855259, 73.82809176757507, 74.01809261832514, 74.16555596517595, 74.30167597765363, 74.46899015965744, 74.58242350338881, 74.67600601196722, 74.72705101664634, 74.7894393556986, 74.84332019397102, 74.82914102600459, 74.86884269631058, 74.90287269942999, 74.9397385361427, 75.01914187675467, 75.08436604940022, 75.1439185548592, 75.20063522672488, 75.19496355953832, 75.22899356265775, 75.24884439781073, 75.26018773218387, 75.2772027337436, 75.26018773218387, 75.26869523296374, 75.29138190171001, 75.3282477384227, 75.39347191106826, 75.39063607747497, 75.41615857981454, 75.4416810821541, 75.44735274934067, 75.48138275246008, 75.5324277571392, 75.54093525791906, 75.58063692822505, 75.58630859541162, 75.57496526103849, 75.59198026259818, 75.6089952641579, 75.65436860165046, 75.6628761024303, 75.68839860476987, 75.71392110710944, 75.70824943992285, 75.70824943992285, 75.71392110710944, 75.71675694070272, 75.70824943992285, 75.719592774296, 75.72810027507586, 75.76496611178855, 75.77347361256841, 75.76496611178855, 75.77347361256841, 75.77630944616169, 75.76213027819527, 75.75362277741542, 75.77063777897513, 75.78198111334827, 75.78198111334827, 75.77347361256841, 75.74795111022885, 75.74227944304228, 75.719592774296, 75.67989110399002, 75.67138360321016, 75.6628761024303, 75.62033859853103, 75.5891444290049, 75.56078609307205, 75.50974108839293, 75.46153191730708, 75.37645690950855, 75.31690440404957, 75.21765022828461, 75.13541105407936, 75.00496270878824, 74.76108101976577, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.49567535377024, 68.06567790602047, 68.55911295125202, 68.99583132461787, 69.39284802767773, 69.75583472761818, 70.09897059240564, 70.41941978844682, 70.72285398292829, 71.00076567507018, 71.27300570002552, 71.49703655389501, 71.78345574681677, 72.05853160536539, 72.25987579048862, 72.5604741513768, 72.8554008450784, 73.10495420128747, 73.36585089186967, 73.58421007855259, 73.82809176757507, 74.01809261832514, 74.16555596517595, 74.30167597765363, 74.46899015965744, 74.58242350338881, 74.67600601196722, 74.72705101664634, 74.7894393556986, 74.84332019397102, 74.82914102600459, 74.86884269631058, 74.90287269942999, 74.9397385361427, 75.01914187675467, 75.08436604940022, 75.1439185548592, 75.20063522672488, 75.19496355953832, 75.22899356265775, 75.24884439781073, 75.26018773218387, 75.2772027337436, 75.26018773218387, 75.26869523296374, 75.29138190171001, 75.3282477384227, 75.39347191106826, 75.39063607747497, 75.41615857981454, 75.4416810821541, 75.44735274934067, 75.48138275246008, 75.5324277571392, 75.54093525791906, 75.58063692822505, 75.58630859541162, 75.57496526103849, 75.59198026259818, 75.6089952641579, 75.65436860165046, 75.6628761024303, 75.68839860476987, 75.71392110710944, 75.70824943992285, 75.70824943992285, 75.71392110710944, 75.71675694070272, 75.70824943992285, 75.719592774296, 75.72810027507586, 75.76496611178855, 75.77347361256841, 75.76496611178855, 75.77347361256841, 75.77630944616169, 75.76213027819527, 75.75362277741542, 75.77063777897513, 75.78198111334827, 75.78198111334827, 75.77347361256841, 75.74795111022885, 75.74511527663557, 75.719592774296, 75.67989110399002, 75.67138360321016, 75.6628761024303, 75.62033859853103, 75.5891444290049, 75.56078609307205, 75.50974108839293, 75.46153191730708, 75.37645690950855, 75.31690440404957, 75.21765022828461, 75.13541105407936, 75.00496270878824, 74.76108101976577, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2030, "method": "three", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.52686952329637, 68.0203045685279, 68.48254544423334, 68.89374131525962, 69.33613135581204, 69.69911805575248, 70.00822391742052, 70.3456881150214, 70.64912230950287, 71.00927317585004, 71.39778237813005, 71.69554490542495, 71.93659076085414, 71.93375492726086, 71.93942659444743, 71.95360576241386, 71.964949096787, 71.97629243116013, 71.9989790999064, 72.01882993505941, 72.04718827099225, 72.05569577177211, 72.08121827411166, 72.08972577489152, 72.10390494285795, 72.1294274451975, 72.15778578113036, 72.18614411706321, 72.20315911862292, 72.2258457873692, 72.25420412330205, 72.2797266256416, 72.27689079204832, 72.2995774607946, 72.30241329438788, 72.32509996313416, 72.38181663499985, 72.38465246859313, 72.41868247171256, 72.44136914045883, 72.44987664123869, 72.49241414513796, 72.52927998185066, 72.52360831466409, 72.55763831778351, 72.55196665059694, 72.61435498964921, 72.66539999432833, 72.71077333182089, 72.76465417009331, 72.78166917165301, 72.81286334117914, 72.87241584663812, 72.94047585287696, 72.97734168958965, 73.02271502708221, 73.09077503332104, 73.11346170206731, 73.13331253722032, 73.14182003800018, 73.17585004111959, 73.20704421064572, 73.25525338173155, 73.28928338485098, 73.32331338797039, 73.38570172702265, 73.44525423248164, 73.50480673794063, 73.63809091682499, 73.71749425743697, 73.80824093242208, 73.91883844256019, 73.95286844567961, 74.00674928395202, 74.07480929019084, 74.14570513002296, 74.1939143011088, 74.22510847063494, 74.24495930578794, 74.31301931202677, 74.37824348467231, 74.39525848623202, 74.42645265575815, 74.46048265887757, 74.51719933074327, 74.53988599948956, 74.58242350338881, 74.6816776791538, 74.7355585174262, 74.74123018461276, 74.72705101664634, 74.76108101976577, 74.76675268695233, 74.76391685335905, 74.7327226838329, 74.74123018461276, 74.69585684712021, 74.63063267447467, 74.59093100416867, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2030, "method": "gain", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.31418200380001, 67.73388537560616, 68.16776791537872, 68.60732212233785, 69.06956299804328, 69.48359470266284, 69.92314890962199, 70.28613560956242, 70.59524147123047, 70.74270481808126, 70.81360065791338, 70.90718316649178, 70.97524317273061, 71.02345234381646, 71.0915123500553, 71.15390068910756, 71.21061736097326, 71.23046819612625, 71.29285653517852, 71.36658820860391, 71.40345404531662, 71.46584238436888, 71.50837988826815, 71.556589059354, 71.58494739528685, 71.65584323511897, 71.70121657261151, 71.7409182429175, 71.76360491166378, 71.79763491478319, 71.86569492102203, 71.88838158976831, 71.93659076085414, 71.96778493038029, 72.00465076709298, 72.05569577177211, 72.0755466069251, 72.07271077333182, 72.0925616084848, 72.11241244363781, 72.15494994753708, 72.15778578113036, 72.18330828346993, 72.18614411706321, 72.23718912174233, 72.24569662252219, 72.25136828970876, 72.30808496157445, 72.34211496469388, 72.42718997249241, 72.47823497717154, 72.5377874826305, 72.54913081700366, 72.58883248730965, 72.63987749198877, 72.63987749198877, 72.67957916229474, 72.77599750446643, 72.83838584351871, 72.8752516802314, 72.85256501148513, 72.92062501772396, 72.9886850239628, 73.03405836145535, 73.0794316989479, 73.14465587159346, 73.21555171142558, 73.25241754813827, 73.25808921532484, 73.31197005359725, 73.33465672234352, 73.3601792246831, 73.4707767348212, 73.55301590902646, 73.5813742449593, 73.59838924651902, 73.66644925275784, 73.68346425431756, 73.75436009414967, 73.88764427303406, 73.91316677537361, 74.00107761676544, 73.94436094489976, 73.91033094178033, 73.93585344411989, 74.00107761676544, 74.03227178629159, 74.09466012534385, 74.15137679720954, 74.21092930266852, 74.27615347531406, 74.3810793182656, 74.46615432606414, 74.45197515809771, 74.49451266199699, 74.5313784987097, 74.49167682840371, 74.52570683152312, 74.54555766667612, 74.47182599325072]}, {"task": "grefcoco", "expert": "instancevg", "endpoint": "Qwen3.8", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9982134248362307, 1.999262683265746, 2.9974761081019765, 3.998525366531492, 4.999574624961007, 5.997788049797237, 6.998837308226753, 7.999886566656269, 8.998099991492499, 9.999149249922015, 10.997362674758245, 11.99841193318776, 12.999461191617275, 13.997674616453507, 14.998723874883021, 15.999773133312537, 16.997986558148767, 17.99903581657828, 18.99724924141451, 19.99829849984403, 20.999347758273544, 21.997561183109777, 22.99861044153929, 23.999659699968806, 24.997873124805036, 25.99892238323455, 26.999971641664068, 27.998185066500298, 28.999234324929812, 29.997447749766042, 30.998497008195557, 31.999546266625075, 32.997759691461304, 33.998808949890815, 34.99985820832033, 35.99807163315656, 36.99912089158608, 37.99733431642231, 38.99838357485183, 39.99943283328135, 40.99764625811758, 41.99869551654709, 42.999744774976605, 43.997958199812835, 44.99900745824235, 45.99722088307858, 46.998270141508094, 47.99931939993761, 48.99753282477384, 49.99858208320336, 50.99963134163288, 51.9978447664691, 52.998894024898625, 53.999943283328136, 54.99815670816437, 55.99920596659388, 56.997419391430114, 57.998468649859625, 58.99951790828914, 59.99773133312537, 60.99878059155489, 61.9998298499844, 62.99804327482064, 63.99909253325015, 64.99730595808639, 65.99835521651589, 66.99940447494541, 67.99761789978163, 68.99866715821116, 69.99971641664067, 70.9979298414769, 71.9989790999064, 72.99719252474264, 73.99824178317216, 74.99929104160168, 75.9975044664379, 76.99855372486742, 77.99960298329694, 78.99781640813318, 79.9988656665627, 80.9999149249922, 81.99812834982843, 82.99917760825795, 83.99739103309417, 84.99844029152369, 85.99948954995321, 86.99770297478943, 87.99875223321895, 88.99980149164847, 89.9980149164847, 90.99906417491421, 91.99727759975045, 92.99832685817996, 93.99937611660948, 94.9975895414457, 95.99863879987522, 96.99968805830473, 97.99790148314096, 98.99895074157048, 100.0], "score": [66.95119530385956, 67.46731701783739, 67.88418455605026, 68.27836542551682, 68.70374046450955, 69.08657799960298, 69.44105719876357, 69.79553639792417, 70.20106060176388, 70.5328531321782, 70.85613816181267, 71.24181153049939, 71.5537532257607, 71.78345574681677, 72.13509911238408, 72.40166747015286, 72.67957916229474, 72.9319683520971, 73.13898420440688, 73.35450755749653, 73.58704591214588, 73.76003176133624, 73.89048010662734, 74.10600345971699, 74.28182514250064, 74.44630349091115, 74.54839350026941, 74.64764767603437, 74.70720018149335, 74.81779769163145, 74.9198877009897, 74.9766043728554, 75.11839605251963, 75.20063522672488, 75.30272523608315, 75.41615857981454, 75.47003941808694, 75.58347276181834, 75.63451776649745, 75.67138360321016, 75.72526444148258, 75.82168278365425, 75.86422028755354, 75.9237727930125, 76.01451946799762, 76.00884780081105, 76.05989280549016, 76.13929614610214, 76.17049031562829, 76.24705782264697, 76.26974449139324, 76.24422198905368, 76.24422198905368, 76.23004282108725, 76.25556532342682, 76.22437115390068, 76.27541615857982, 76.29810282732609, 76.27258032498652, 76.27258032498652, 76.21869948671412, 76.22720698749397, 76.22437115390068, 76.2413861554604, 76.2016844851544, 76.1846694835947, 76.21586365312083, 76.2016844851544, 76.21019198593426, 76.21302781952754, 76.17616198281485, 76.16765448203499, 76.16198281484843, 76.16765448203499, 76.17899781640813, 76.127952811729, 76.05138530471032, 76.02586280237075, 75.97765363128491, 75.96631029691177, 75.92944446019906, 75.84436945240053, 75.82451861724753, 75.76780194538185, 75.72810027507586, 75.6827269375833, 75.63451776649745, 75.60332359697132, 75.54660692510564, 75.47003941808694, 75.39630774466154, 75.29705356889657, 75.1807843915719, 75.0872018829935, 75.03615687831437, 74.9936193744151, 74.90003686583671, 74.82063352522474, 74.7327226838329, 74.61645350650824, 74.47182599325072]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.61275740833752, 81.05761641673244, 81.53117600631413, 81.90428356174213, 82.22716510009327, 82.64332352730143, 82.9518547750592, 83.31778718519051, 83.63349357824497, 83.89897395422258, 84.1572791849035, 84.47298557795796, 84.6954150821554, 84.96089545813304, 85.22637583411064, 85.4488053383081, 85.68558513309895, 85.90083949199972, 86.15914472268064, 86.30982277391116, 86.49637655162518, 86.58247829518548, 86.6542297481524, 86.82643323527302, 87.09908875654733, 87.29281767955801, 87.46502116667862, 87.60852407261247, 87.65875008968932, 87.81660328621655, 87.93858075626032, 88.0462079357107, 88.23276171342471, 88.31886345698501, 88.41931549113869, 88.45519121762216, 88.48389179880894, 88.49106694410563, 88.54846810647916, 88.56999354236925, 88.6058692688527, 88.62739470474278, 88.62021955944608, 88.65609528592955, 88.68479586711632, 88.71349644830309, 88.72784673889646, 88.73502188419316, 88.75654732008323, 88.81394848245677, 88.86417449953362, 88.89287508072039, 88.86417449953362, 88.86417449953362, 88.91440051661046, 88.90005022601709, 88.89287508072039, 88.90722537131377, 88.92157566190716, 88.92157566190716, 88.94310109779724, 88.99332711487408, 89.02202769606085, 89.065078567841, 89.1440051661046, 89.20858147377486, 89.23728205496163, 89.23728205496163, 89.2659826361484, 89.32338379852193, 89.3377340891153, 89.3377340891153, 89.344909234412, 89.40948554208224, 89.45253641386238, 89.52428786682931, 89.55298844801608, 89.624739900983, 89.65344048216977, 89.68214106335654, 89.72519193513669, 89.76824280691685, 89.83999425988375, 89.82564396929038, 89.86869484107054, 89.85434455047715, 89.8973954222573, 89.94044629403746, 89.98349716581761, 90.02654803759776, 90.07677405467462, 90.07677405467462, 90.07677405467462, 90.19875152471838, 90.23462725120184, 90.30637870416876, 90.40683073832245, 90.52880820836621, 90.70818684078354, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.59840711774415, 81.05044127143574, 81.50965057042406, 81.95450957881899, 82.25586568128006, 82.55004663844443, 82.90162875798235, 83.16710913395997, 83.45411494582765, 83.66936930472842, 83.95637511659612, 84.22903063787042, 84.5303867403315, 84.82456769749588, 85.0900480734735, 85.43445504771472, 85.61383368013202, 85.73581115017579, 85.98694123556002, 86.22372103035086, 86.36722393628472, 86.54660256870201, 86.66858003874579, 86.89100954294324, 87.0345124488771, 87.1134390471407, 87.32151826074478, 87.50089689316209, 87.62287436320587, 87.76637726913971, 87.90988017507355, 88.03185764511731, 88.16818540575447, 88.31886345698501, 88.49824208940231, 88.59151897825932, 88.64174499533615, 88.70632130300639, 88.79959819186338, 88.8856999354237, 88.94310109779724, 89.02920284135753, 89.10095429432447, 89.21575661907154, 89.344909234412, 89.438186123269, 89.55298844801608, 89.57451388390615, 89.64626533687307, 89.66779077276315, 89.78259309751023, 89.83999425988375, 89.8902202769606, 89.8902202769606, 89.96197172992754, 90.04089832819116, 90.07677405467462, 90.16287579823492, 90.1700509435316, 90.22745210590514, 90.28485326827868, 90.32790414005885, 90.34942957594892, 90.34942957594892, 90.38530530243239, 90.40683073832245, 90.41400588361914, 90.41400588361914, 90.44988161010261, 90.47140704599268, 90.53598335366291, 90.56468393484968, 90.56468393484968, 90.60055966133315, 90.61490995192652, 90.65078567840999, 90.71536198608023, 90.73688742197031, 90.744062567267, 90.744062567267, 90.73688742197031, 90.744062567267, 90.75841285786038, 90.8301643108273, 90.79428858434385, 90.80863887493722, 90.82298916553061, 90.8301643108273, 90.85168974671738, 90.85886489201407, 90.85886489201407, 90.88756547320084, 90.88039032790415, 90.87321518260745, 90.85886489201407, 90.84451460142068, 90.84451460142068, 90.78711343904715, 90.75841285786038, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54818110066729, 81.02891583554567, 81.4665996986439, 81.79665638229174, 82.17693908301644, 82.46394489488412, 82.73660041615844, 82.93032933916912, 83.2173351510368, 83.55456697998135, 83.84874793714573, 84.12140345842005, 84.4299347061778, 84.73846595393557, 84.96807060342972, 85.22637583411064, 85.48468106479157, 85.76451173136256, 85.92236492788979, 86.05869268852695, 86.2882973380211, 86.47485111573509, 86.61117887637225, 86.8336083805697, 87.09191361125063, 87.22824137188778, 87.32151826074478, 87.47219631197532, 87.665925234986, 87.76637726913971, 87.86682930329339, 88.02468249982063, 88.16818540575447, 88.31886345698501, 88.46954150821554, 88.57716868766593, 88.67762072181962, 88.72067159359977, 88.81394848245677, 88.8856999354237, 88.94310109779724, 89.03637798665423, 89.17988089258809, 89.23728205496163, 89.35925952500538, 89.48123699504916, 89.5386381574227, 89.58886417449953, 89.64626533687307, 89.73236708043338, 89.8041185334003, 89.811293678697, 89.8973954222573, 89.94762143933414, 89.98349716581761, 90.04807347348785, 90.0839491999713, 90.1700509435316, 90.23462725120184, 90.24180239649853, 90.29920355887207, 90.34225443065222, 90.3566047212456, 90.37813015713569, 90.39248044772907, 90.3566047212456, 90.38530530243239, 90.42118102891584, 90.43553131950922, 90.50728277247615, 90.52880820836621, 90.56468393484968, 90.57185908014638, 90.60773480662984, 90.65078567840999, 90.67231111430006, 90.68666140489346, 90.72253713137691, 90.7297122766736, 90.744062567267, 90.73688742197031, 90.73688742197031, 90.78711343904715, 90.82298916553061, 90.80863887493722, 90.81581402023392, 90.82298916553061, 90.8301643108273, 90.85886489201407, 90.85886489201407, 90.83733945612398, 90.88039032790415, 90.87321518260745, 90.86604003731075, 90.85168974671738, 90.85168974671738, 90.84451460142068, 90.78711343904715, 90.75123771256368, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54818110066729, 81.02891583554567, 81.4665996986439, 81.79665638229174, 82.17693908301644, 82.46394489488412, 82.73660041615844, 82.93032933916912, 83.2173351510368, 83.55456697998135, 83.84874793714573, 84.12140345842005, 84.4299347061778, 84.73846595393557, 84.96807060342972, 85.22637583411064, 85.48468106479157, 85.76451173136256, 85.92236492788979, 86.05869268852695, 86.2882973380211, 86.47485111573509, 86.61117887637225, 86.8336083805697, 87.09191361125063, 87.22824137188778, 87.32151826074478, 87.47219631197532, 87.665925234986, 87.76637726913971, 87.86682930329339, 88.02468249982063, 88.16818540575447, 88.31886345698501, 88.46954150821554, 88.57716868766593, 88.67762072181962, 88.72067159359977, 88.81394848245677, 88.8856999354237, 88.94310109779724, 89.03637798665423, 89.17988089258809, 89.23728205496163, 89.35925952500538, 89.48123699504916, 89.5386381574227, 89.58886417449953, 89.64626533687307, 89.73236708043338, 89.8041185334003, 89.811293678697, 89.8973954222573, 89.94762143933414, 89.98349716581761, 90.04807347348785, 90.0839491999713, 90.1700509435316, 90.23462725120184, 90.24180239649853, 90.29920355887207, 90.34225443065222, 90.3566047212456, 90.37813015713569, 90.39248044772907, 90.3566047212456, 90.38530530243239, 90.42118102891584, 90.43553131950922, 90.50728277247615, 90.52880820836621, 90.56468393484968, 90.57185908014638, 90.60773480662984, 90.65078567840999, 90.67231111430006, 90.68666140489346, 90.72253713137691, 90.7297122766736, 90.744062567267, 90.73688742197031, 90.73688742197031, 90.78711343904715, 90.82298916553061, 90.80863887493722, 90.81581402023392, 90.82298916553061, 90.8301643108273, 90.85886489201407, 90.85886489201407, 90.83733945612398, 90.88039032790415, 90.87321518260745, 90.86604003731075, 90.85168974671738, 90.85168974671738, 90.84451460142068, 90.78711343904715, 90.75123771256368, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2026, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.52665566477721, 80.9930401090622, 81.43072397216044, 81.91145870703882, 82.24151539068667, 82.69354954437827, 83.02360622802613, 83.37518834756405, 83.6837195953218, 83.89179880892588, 84.27208150965058, 84.45863528736457, 84.75281624452894, 84.96807060342972, 85.16179952644042, 85.34117815885772, 85.56360766305518, 85.74298629547249, 85.97259094496664, 86.24524646624094, 86.45332567984502, 86.5609528592954, 86.74033149171271, 86.89818468823994, 87.13496448303079, 87.26411709837124, 87.44349573078856, 87.57264834612901, 87.72332639735954, 87.82377843151325, 87.90270502977687, 87.93858075626032, 87.93858075626032, 87.98163162804047, 88.00315706393054, 87.99598191863386, 88.00315706393054, 88.0462079357107, 88.08208366219417, 88.08925880749085, 88.12513453397432, 88.16818540575447, 88.21841142283131, 88.23276171342471, 88.31886345698501, 88.35473918346847, 88.412140345842, 88.46236636291886, 88.46954150821554, 88.52694267058908, 88.54846810647916, 88.54129296118248, 88.53411781588576, 88.56281839707253, 88.56999354236925, 88.57716868766593, 88.59151897825932, 88.57716868766593, 88.64174499533615, 88.66327043122624, 88.67762072181962, 88.70632130300639, 88.74219702948984, 88.83547391834685, 88.90722537131377, 88.96462653368731, 89.03637798665423, 89.08660400373107, 89.12247973021454, 89.23728205496163, 89.28033292674176, 89.33055894381862, 89.37360981559877, 89.41666068737892, 89.438186123269, 89.52428786682931, 89.56016359331277, 89.6175647556863, 89.66779077276315, 89.68214106335654, 89.64626533687307, 89.68214106335654, 89.76824280691685, 89.84716940518047, 89.85434455047715, 89.92609600344407, 89.86151969577384, 89.8902202769606, 89.95479658463084, 90.01219774700438, 90.02654803759776, 90.0839491999713, 90.15570065293824, 90.26332783238861, 90.32790414005885, 90.39248044772907, 90.47140704599268, 90.50728277247615, 90.64361053311329, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2026, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.51948051948052, 81.03609098084237, 81.44507426275382, 81.86123268996197, 82.14823850182967, 82.56439692903783, 82.95902992035589, 83.28191145870704, 83.54739183468466, 83.76982133888211, 83.99225084307957, 84.17880462079358, 84.4299347061778, 84.68823993685872, 84.92501973164957, 85.10439836406688, 85.30530243237426, 85.51338164597834, 85.79321231254933, 85.93671521848317, 86.18067015857072, 86.30982277391116, 86.42462509865825, 86.58247829518548, 86.79055750878956, 86.97711128650356, 87.0416875941738, 87.09908875654733, 87.14213962832747, 87.2067159359977, 87.26411709837124, 87.3143431154481, 87.35021884193155, 87.41479514960179, 87.47219631197532, 87.51524718375548, 87.52959747434886, 87.54394776494225, 87.63004950850255, 87.60134892731578, 87.62287436320587, 87.64439979909594, 87.65157494439264, 87.6731003802827, 87.74485183324963, 87.83095357680993, 87.90988017507355, 87.9601061921504, 87.98163162804047, 88.06055822630408, 88.10360909808423, 88.18253569634786, 88.19688598694124, 88.29016287579825, 88.33321374757838, 88.38343976465524, 88.412140345842, 88.46236636291886, 88.505417234699, 88.54846810647916, 88.6058692688527, 88.64174499533615, 88.691971012413, 88.77089761067663, 88.84264906364353, 88.86417449953362, 88.87134964483032, 88.95745138839062, 89.00767740546746, 89.03637798665423, 89.11530458491785, 89.18705603788477, 89.251632345555, 89.2588074908517, 89.24445720025831, 89.32338379852193, 89.38078496089547, 89.41666068737892, 89.438186123269, 89.52428786682931, 89.54581330271938, 89.58168902920285, 89.61038961038962, 89.58168902920285, 89.66061562746646, 89.72519193513669, 89.76824280691685, 89.78976824280693, 89.85434455047715, 89.88304513166392, 89.94762143933414, 90.06242376408122, 90.16287579823492, 90.23462725120184, 90.29920355887207, 90.42835617421254, 90.53598335366291, 90.55750878955298, 90.68666140489346, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.49795508359044, 80.89976322020522, 81.24417019444644, 81.63880318576452, 81.95450957881899, 82.27739111717013, 82.55004663844443, 82.80835186912535, 83.08818253569635, 83.39671378345412, 83.6908947406185, 83.87027337303581, 84.0711774413432, 84.3366578173208, 84.56626246681496, 84.74564109923226, 84.96089545813304, 85.22637583411064, 85.47033077419817, 85.6497094066155, 85.85061347492287, 86.11609385090048, 86.36722393628472, 86.49637655162518, 86.71163091052594, 86.86948410705317, 87.06321303006386, 87.27129224366794, 87.45067087608524, 87.5869986367224, 87.78072755973308, 87.945755901557, 88.09643395278755, 88.20406113223794, 88.29016287579825, 88.412140345842, 88.54846810647916, 88.62021955944608, 88.77807275597331, 88.878524790127, 89.01485255076416, 89.17270574729139, 89.29468321733516, 89.40948554208224, 89.47406184975246, 89.58886417449953, 89.66779077276315, 89.77541795221353, 89.79694338810361, 89.82564396929038, 89.98349716581761, 90.02654803759776, 90.04807347348785, 90.06242376408122, 90.15570065293824, 90.18440123412499, 90.22027696060846, 90.24180239649853, 90.26332783238861, 90.33507928535553, 90.34225443065222, 90.34225443065222, 90.40683073832245, 90.37095501183899, 90.37813015713569, 90.38530530243239, 90.39248044772907, 90.43553131950922, 90.43553131950922, 90.46423190069599, 90.48575733658608, 90.50728277247615, 90.54315849895961, 90.58620937073975, 90.62208509722322, 90.67948625959676, 90.65078567840999, 90.67231111430006, 90.61490995192652, 90.60773480662984, 90.59338451603645, 90.60055966133315, 90.67231111430006, 90.6364353878166, 90.68666140489346, 90.70818684078354, 90.70818684078354, 90.75841285786038, 90.81581402023392, 90.80863887493722, 90.81581402023392, 90.81581402023392, 90.80863887493722, 90.80146372964053, 90.8301643108273, 90.79428858434385, 90.78711343904715, 90.76558800315708, 90.72253713137691, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.59123197244745, 81.08631699791921, 81.57422687809428, 81.98321016000575, 82.32044198895028, 82.69354954437827, 83.01643108272943, 83.28908660400374, 83.51869125349789, 83.7913467747722, 84.0568271507498, 84.3366578173208, 84.64518906507857, 84.9034942957595, 85.15462438114372, 85.38422903063787, 85.58513309894525, 85.75016144076918, 85.9080146372964, 86.10891870560378, 86.25959675683433, 86.47485111573509, 86.58965344048217, 86.7546817823061, 86.87665925234987, 86.97711128650356, 87.01298701298703, 87.0273373035804, 87.07756332065725, 87.1134390471407, 87.1134390471407, 87.15648991892087, 87.19954079070101, 87.17801535481094, 87.2138910812944, 87.24259166248117, 87.25694195307454, 87.29281767955801, 87.34304369663487, 87.38609456841502, 87.44349573078856, 87.47937145727201, 87.51524718375548, 87.55829805553563, 87.55112291023893, 87.60134892731578, 87.63004950850255, 87.65157494439264, 87.63722465379924, 87.6731003802827, 87.70180096146947, 87.7735524144364, 87.80225299562316, 87.80942814091985, 87.8596541579967, 87.86682930329339, 87.91705532037024, 87.96728133744709, 87.99598191863386, 88.08925880749085, 88.12513453397432, 88.17536055105116, 88.22558656812801, 88.24711200401808, 88.31168831168831, 88.30451316639162, 88.40496520054532, 88.46954150821554, 88.5197675252924, 88.54846810647916, 88.64174499533615, 88.66327043122624, 88.72067159359977, 88.81394848245677, 88.92157566190716, 88.95027624309392, 89.04355313195093, 89.10812943962115, 89.12965487551124, 89.17988089258809, 89.30185836263185, 89.33055894381862, 89.36643467030207, 89.4238358326756, 89.5386381574227, 89.56016359331277, 89.65344048216977, 89.74671737102676, 89.811293678697, 89.91892085814737, 89.96914687522423, 90.01219774700438, 89.997847456411, 90.14135036234484, 90.33507928535553, 90.3637798665423, 90.47858219128938, 90.56468393484968, 90.66513596900337, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.57688168185406, 81.08631699791921, 81.5598765875009, 81.99038530530244, 82.29891655306021, 82.60744780081797, 82.90162875798235, 83.23886058692689, 83.5976178517615, 83.7913467747722, 84.01377627896966, 84.30078209083734, 84.56626246681496, 84.75999138982566, 85.01112147520988, 85.21202554351726, 85.49903135538494, 85.70711056898904, 85.88648920140633, 86.03716725263688, 86.23807132094426, 86.43180024395495, 86.5753031498888, 86.74033149171271, 86.94841070531677, 87.10626390184402, 87.25694195307454, 87.42914544019517, 87.63004950850255, 87.78790270502978, 87.78072755973308, 87.9529310468537, 88.12513453397432, 88.22558656812801, 88.33321374757838, 88.49824208940231, 88.64892014063285, 88.70632130300639, 88.81394848245677, 88.92875080720385, 89.15118031140132, 89.19423118318146, 89.28033292674176, 89.40231039678554, 89.52428786682931, 89.6175647556863, 89.67496591805984, 89.73954222573008, 89.85434455047715, 89.91174571285069, 89.94762143933414, 90.01219774700438, 90.00502260170768, 90.06242376408122, 90.10547463586138, 90.14135036234484, 90.19157637942169, 90.19157637942169, 90.24180239649853, 90.24180239649853, 90.29202841357538, 90.39248044772907, 90.40683073832245, 90.42118102891584, 90.44270646480591, 90.47858219128938, 90.48575733658608, 90.49293248188276, 90.51445791777284, 90.50010762717945, 90.54315849895961, 90.57185908014638, 90.54315849895961, 90.5503336442563, 90.53598335366291, 90.58620937073975, 90.61490995192652, 90.62926024251992, 90.64361053311329, 90.71536198608023, 90.73688742197031, 90.73688742197031, 90.75123771256368, 90.75841285786038, 90.75123771256368, 90.75841285786038, 90.77993829375045, 90.80863887493722, 90.79428858434385, 90.80146372964053, 90.77993829375045, 90.77276314845376, 90.77276314845376, 90.77993829375045, 90.78711343904715, 90.78711343904715, 90.79428858434385, 90.744062567267, 90.73688742197031, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54100595537061, 81.02174069024898, 81.50247542512736, 81.90428356174213, 82.21998995479659, 82.58592236492788, 82.84422759560881, 83.1312334074765, 83.5043409629045, 83.80569706536558, 84.01377627896966, 84.32230752672743, 84.48733586855134, 84.6954150821554, 84.95372031283634, 85.20485039822057, 85.51338164597834, 85.73581115017579, 85.85778862021957, 86.04434239793356, 86.21654588505417, 86.43897538925164, 86.5609528592954, 86.74033149171271, 86.94123556002009, 87.1206141924374, 87.25694195307454, 87.52959747434886, 87.60852407261247, 87.73767668795294, 87.80225299562316, 87.88835473918347, 88.01033220922724, 88.21841142283131, 88.36191432876515, 88.51259237999571, 88.6058692688527, 88.77089761067663, 88.85699935423692, 89.01485255076416, 89.12965487551124, 89.19423118318146, 89.28750807203846, 89.37360981559877, 89.56016359331277, 89.66061562746646, 89.71084164454331, 89.76824280691685, 89.86151969577384, 89.91174571285069, 89.91174571285069, 89.92609600344407, 90.02654803759776, 90.01937289230108, 90.0839491999713, 90.15570065293824, 90.20592667001507, 90.24180239649853, 90.27050297768531, 90.34225443065222, 90.3637798665423, 90.41400588361914, 90.40683073832245, 90.42118102891584, 90.44270646480591, 90.47140704599268, 90.49293248188276, 90.48575733658608, 90.47858219128938, 90.50010762717945, 90.5503336442563, 90.5503336442563, 90.61490995192652, 90.58620937073975, 90.58620937073975, 90.58620937073975, 90.59338451603645, 90.62926024251992, 90.66513596900337, 90.70101169548683, 90.744062567267, 90.744062567267, 90.75841285786038, 90.77276314845376, 90.75841285786038, 90.78711343904715, 90.81581402023392, 90.82298916553061, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.80863887493722, 90.79428858434385, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.77993829375045, 90.75123771256368, 90.744062567267, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54100595537061, 81.02174069024898, 81.50247542512736, 81.90428356174213, 82.21998995479659, 82.58592236492788, 82.84422759560881, 83.1312334074765, 83.5043409629045, 83.80569706536558, 84.01377627896966, 84.32230752672743, 84.48733586855134, 84.6954150821554, 84.95372031283634, 85.20485039822057, 85.51338164597834, 85.73581115017579, 85.85778862021957, 86.04434239793356, 86.21654588505417, 86.43897538925164, 86.5609528592954, 86.74033149171271, 86.94123556002009, 87.1206141924374, 87.25694195307454, 87.52959747434886, 87.60852407261247, 87.73767668795294, 87.80225299562316, 87.88835473918347, 88.01033220922724, 88.21841142283131, 88.36191432876515, 88.51259237999571, 88.6058692688527, 88.77089761067663, 88.85699935423692, 89.01485255076416, 89.12965487551124, 89.19423118318146, 89.28750807203846, 89.37360981559877, 89.56016359331277, 89.66061562746646, 89.71084164454331, 89.76824280691685, 89.86151969577384, 89.91174571285069, 89.91174571285069, 89.92609600344407, 90.02654803759776, 90.01937289230108, 90.0839491999713, 90.15570065293824, 90.20592667001507, 90.24180239649853, 90.27050297768531, 90.34225443065222, 90.3637798665423, 90.41400588361914, 90.40683073832245, 90.42118102891584, 90.44270646480591, 90.47140704599268, 90.49293248188276, 90.48575733658608, 90.47858219128938, 90.50010762717945, 90.5503336442563, 90.5503336442563, 90.61490995192652, 90.58620937073975, 90.58620937073975, 90.58620937073975, 90.59338451603645, 90.62926024251992, 90.66513596900337, 90.70101169548683, 90.744062567267, 90.744062567267, 90.75841285786038, 90.77276314845376, 90.75841285786038, 90.78711343904715, 90.81581402023392, 90.82298916553061, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.80863887493722, 90.79428858434385, 90.76558800315708, 90.77276314845376, 90.77993829375045, 90.77993829375045, 90.75123771256368, 90.744062567267, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2027, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54818110066729, 81.14371816029275, 81.48094998923729, 81.8397072540719, 82.24151539068667, 82.43524431369735, 82.82987730501543, 83.18863456985004, 83.46129009112435, 83.74112075769534, 84.00660113367296, 84.25055607376049, 84.58778790270503, 84.83174284279256, 85.06134749228671, 85.3555284494511, 85.56360766305518, 85.77168687665926, 85.93671521848317, 86.13761928679055, 86.33852335509793, 86.49637655162518, 86.72598120111932, 86.95558585061349, 87.14213962832747, 87.22106622659109, 87.22824137188778, 87.2067159359977, 87.26411709837124, 87.28564253426131, 87.33586855133817, 87.37174427782163, 87.4004448590084, 87.41479514960179, 87.45067087608524, 87.4937217478654, 87.54394776494225, 87.5941737820191, 87.63722465379924, 87.68745067087609, 87.70180096146947, 87.73050154265624, 87.75920212384301, 87.79507785032648, 87.81660328621655, 87.86682930329339, 87.89552988448015, 87.93140561096362, 87.93858075626032, 87.96728133744709, 87.98880677333716, 88.00315706393054, 88.03185764511731, 88.0533830810074, 88.08925880749085, 88.17536055105116, 88.17536055105116, 88.23276171342471, 88.30451316639162, 88.33321374757838, 88.36191432876515, 88.37626461935855, 88.39061490995192, 88.43366578173209, 88.51259237999571, 88.59151897825932, 88.6058692688527, 88.64174499533615, 88.64892014063285, 88.67044557652292, 88.75654732008323, 88.82112362775347, 88.84982420894023, 88.90722537131377, 89.01485255076416, 89.065078567841, 89.17270574729139, 89.17988089258809, 89.21575661907154, 89.28750807203846, 89.41666068737892, 89.50276243093923, 89.57451388390615, 89.59603931979623, 89.68931620865322, 89.68214106335654, 89.73954222573008, 89.86151969577384, 89.94762143933414, 89.8973954222573, 89.98349716581761, 90.01219774700438, 90.06242376408122, 90.1700509435316, 90.29920355887207, 90.44270646480591, 90.52163306306953, 90.56468393484968, 90.69383655019014, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2027, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54100595537061, 81.08631699791921, 81.51682571572074, 81.88275812585205, 82.26304082657674, 82.61462294611465, 82.96620506565259, 83.25321087752027, 83.46846523642104, 83.82722250125565, 84.17880462079358, 84.40840927028772, 84.7097653727488, 84.8891440051661, 85.11157350936357, 85.31247757767095, 85.60665853483533, 85.85061347492287, 86.10891870560378, 86.33852335509793, 86.51072684221856, 86.58965344048217, 86.76185692760279, 86.8479586711631, 86.94841070531677, 86.99863672239363, 87.07038817536056, 87.09191361125063, 87.14213962832747, 87.17801535481094, 87.19236564540432, 87.26411709837124, 87.32869340604148, 87.36456913252493, 87.40762000430509, 87.45784602138194, 87.52242232905216, 87.55112291023893, 87.65157494439264, 87.69462581617277, 87.75202697854631, 87.82377843151325, 87.85247901270002, 87.86682930329339, 87.89552988448015, 87.90988017507355, 87.9529310468537, 87.99598191863386, 88.03903279041401, 88.08208366219417, 88.12513453397432, 88.22558656812801, 88.25428714931478, 88.33321374757838, 88.36908947406185, 88.40496520054532, 88.45519121762216, 88.505417234699, 88.54846810647916, 88.61304441414939, 88.65609528592955, 88.68479586711632, 88.65609528592955, 88.691971012413, 88.75654732008323, 88.82112362775347, 88.91440051661046, 88.90722537131377, 88.95027624309392, 88.95745138839062, 88.97897682428069, 89.01485255076416, 89.11530458491785, 89.12965487551124, 89.158355456698, 89.251632345555, 89.28750807203846, 89.32338379852193, 89.38796010619215, 89.41666068737892, 89.45971155915908, 89.50276243093923, 89.6175647556863, 89.63909019157639, 89.624739900983, 89.66061562746646, 89.69649135394992, 89.76824280691685, 89.86151969577384, 89.96914687522423, 90.01219774700438, 90.0839491999713, 90.1772260888283, 90.25615268709191, 90.37095501183899, 90.46423190069599, 90.51445791777284, 90.6364353878166, 90.67948625959676, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54100595537061, 81.0002152543589, 81.41637368156705, 81.72490492932482, 82.10518763004951, 82.42089402310397, 82.77965128793858, 83.08818253569635, 83.46846523642104, 83.72677046710196, 83.98507569778289, 84.22185549257372, 84.5303867403315, 84.7097653727488, 84.97524574872641, 85.1761498170338, 85.47750591949487, 85.76451173136256, 85.97976609026333, 86.16631986797732, 86.36722393628472, 86.55377771399871, 86.79055750878956, 86.89818468823994, 87.07038817536056, 87.22106622659109, 87.37174427782163, 87.51524718375548, 87.57264834612901, 87.76637726913971, 87.86682930329339, 87.99598191863386, 88.09643395278755, 88.21123627753462, 88.34756403817178, 88.46236636291886, 88.59869412355602, 88.70632130300639, 88.79959819186338, 88.82829877305016, 88.94310109779724, 89.05072827724761, 89.11530458491785, 89.19423118318146, 89.29468321733516, 89.3520843797087, 89.40948554208224, 89.438186123269, 89.48841214034584, 89.5386381574227, 89.61038961038962, 89.66061562746646, 89.72519193513669, 89.8041185334003, 89.83999425988375, 89.8973954222573, 89.97632202052091, 90.01219774700438, 90.05524861878453, 90.13417521704815, 90.15570065293824, 90.277678122982, 90.34225443065222, 90.32790414005885, 90.39248044772907, 90.39965559302576, 90.42118102891584, 90.42118102891584, 90.49293248188276, 90.47858219128938, 90.49293248188276, 90.52880820836621, 90.54315849895961, 90.55750878955298, 90.60773480662984, 90.6364353878166, 90.6364353878166, 90.67231111430006, 90.69383655019014, 90.70101169548683, 90.69383655019014, 90.7297122766736, 90.68666140489346, 90.67231111430006, 90.70101169548683, 90.69383655019014, 90.70101169548683, 90.7297122766736, 90.744062567267, 90.75123771256368, 90.80146372964053, 90.80146372964053, 90.77993829375045, 90.81581402023392, 90.80146372964053, 90.81581402023392, 90.79428858434385, 90.77276314845376, 90.75841285786038, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54100595537061, 80.98586496376552, 81.39484824567698, 81.76078065580829, 82.01191074119252, 82.34914257013705, 82.71507498026835, 83.05948195450958, 83.34648776637728, 83.63349357824497, 83.84874793714573, 84.17880462079358, 84.48733586855134, 84.67388964626534, 84.97524574872641, 85.1689746717371, 85.34835330415441, 85.53490708186841, 85.79321231254933, 86.02281696204348, 86.24524646624094, 86.41744995336155, 86.55377771399871, 86.69010547463587, 86.81208294467963, 86.99146157709694, 87.13496448303079, 87.27846738896463, 87.29999282485471, 87.27846738896463, 87.28564253426131, 87.29999282485471, 87.32151826074478, 87.35739398722825, 87.37891942311832, 87.4004448590084, 87.44349573078856, 87.42914544019517, 87.43632058549186, 87.50089689316209, 87.52959747434886, 87.5869986367224, 87.5941737820191, 87.62287436320587, 87.70897610676617, 87.70180096146947, 87.75920212384301, 87.7735524144364, 87.80942814091985, 87.83812872210662, 87.86682930329339, 87.88835473918347, 87.90988017507355, 87.9529310468537, 87.98880677333716, 88.02468249982063, 87.99598191863386, 88.01033220922724, 88.02468249982063, 88.08208366219417, 88.14665996986439, 88.21123627753462, 88.22558656812801, 88.31168831168831, 88.46236636291886, 88.48389179880894, 88.49824208940231, 88.54129296118248, 88.54129296118248, 88.57716868766593, 88.63456985003947, 88.75654732008323, 88.76372246537993, 88.80677333716008, 88.82112362775347, 88.92157566190716, 88.99332711487408, 89.07942885843438, 89.16553060199469, 89.20858147377486, 89.2659826361484, 89.35925952500538, 89.41666068737892, 89.48123699504916, 89.58168902920285, 89.69649135394992, 89.74671737102676, 89.78259309751023, 89.91892085814737, 90.03372318289445, 90.14135036234484, 90.22745210590514, 90.27050297768531, 90.33507928535553, 90.39248044772907, 90.51445791777284, 90.56468393484968, 90.64361053311329, 90.80146372964053, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.52665566477721, 80.92128865609529, 81.32309679271006, 81.61727774987443, 81.95450957881899, 82.28456626246683, 82.60027265552128, 82.85140274090551, 83.1312334074765, 83.37518834756405, 83.6837195953218, 84.00660113367296, 84.29360694554066, 84.50168615914473, 84.75999138982566, 84.98242089402311, 85.26225156059411, 85.43445504771472, 85.62818397072542, 85.84343832962618, 86.04434239793356, 86.22372103035086, 86.36004879098802, 86.61117887637225, 86.70445576522926, 86.86230896175648, 87.0273373035804, 87.32151826074478, 87.42197029489847, 87.51524718375548, 87.65875008968932, 87.80942814091985, 87.93140561096362, 88.03185764511731, 88.12513453397432, 88.23276171342471, 88.33321374757838, 88.46954150821554, 88.54129296118248, 88.62021955944608, 88.77089761067663, 88.90722537131377, 89.05072827724761, 89.20858147377486, 89.30903350792853, 89.35925952500538, 89.48841214034584, 89.624739900983, 89.63191504627969, 89.71084164454331, 89.78259309751023, 89.8902202769606, 89.997847456411, 90.01219774700438, 90.07677405467462, 90.11264978115807, 90.18440123412499, 90.24180239649853, 90.28485326827868, 90.32790414005885, 90.43553131950922, 90.43553131950922, 90.47140704599268, 90.5503336442563, 90.55750878955298, 90.60055966133315, 90.65796082370669, 90.67948625959676, 90.65078567840999, 90.68666140489346, 90.70818684078354, 90.71536198608023, 90.72253713137691, 90.75123771256368, 90.7297122766736, 90.744062567267, 90.71536198608023, 90.75123771256368, 90.76558800315708, 90.75123771256368, 90.72253713137691, 90.71536198608023, 90.70101169548683, 90.68666140489346, 90.72253713137691, 90.73688742197031, 90.73688742197031, 90.78711343904715, 90.85168974671738, 90.8301643108273, 90.84451460142068, 90.84451460142068, 90.86604003731075, 90.85168974671738, 90.84451460142068, 90.85168974671738, 90.82298916553061, 90.79428858434385, 90.76558800315708, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.49795508359044, 80.88541292961183, 81.20111932266629, 81.53117600631413, 81.88993327114873, 82.29174140776351, 82.62179809141136, 82.96620506565259, 83.16710913395997, 83.49716581760781, 83.74829590299204, 84.01377627896966, 84.23620578316712, 84.4371098514745, 84.66671450096864, 84.86044342397933, 85.06852263758341, 85.38422903063787, 85.61383368013202, 85.75733658606588, 85.96541579966994, 86.1878453038674, 86.40309966276818, 86.60400373107555, 86.76903207289948, 86.86948410705317, 87.0273373035804, 87.32151826074478, 87.43632058549186, 87.51524718375548, 87.6802755255794, 87.78072755973308, 87.9529310468537, 88.07490851689748, 88.14665996986439, 88.23993685872139, 88.36191432876515, 88.44801607232546, 88.57716868766593, 88.68479586711632, 88.82829877305016, 88.96462653368731, 89.09377914902777, 89.23010690966493, 89.39513525148884, 89.48123699504916, 89.56016359331277, 89.59603931979623, 89.70366649924661, 89.76106766162016, 89.8184688239937, 89.92609600344407, 89.97632202052091, 90.01219774700438, 90.05524861878453, 90.04807347348785, 90.15570065293824, 90.277678122982, 90.34225443065222, 90.42835617421254, 90.47140704599268, 90.49293248188276, 90.48575733658608, 90.53598335366291, 90.61490995192652, 90.62208509722322, 90.6364353878166, 90.66513596900337, 90.69383655019014, 90.67948625959676, 90.67231111430006, 90.7297122766736, 90.73688742197031, 90.77276314845376, 90.77993829375045, 90.75123771256368, 90.72253713137691, 90.68666140489346, 90.71536198608023, 90.72253713137691, 90.68666140489346, 90.69383655019014, 90.68666140489346, 90.66513596900337, 90.72253713137691, 90.72253713137691, 90.77276314845376, 90.80146372964053, 90.85168974671738, 90.82298916553061, 90.83733945612398, 90.85168974671738, 90.87321518260745, 90.85168974671738, 90.85168974671738, 90.83733945612398, 90.81581402023392, 90.80146372964053, 90.75841285786038, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.49795508359044, 80.88541292961183, 81.20111932266629, 81.53117600631413, 81.88993327114873, 82.29174140776351, 82.62179809141136, 82.96620506565259, 83.16710913395997, 83.49716581760781, 83.74829590299204, 84.01377627896966, 84.23620578316712, 84.4371098514745, 84.66671450096864, 84.86044342397933, 85.06852263758341, 85.38422903063787, 85.61383368013202, 85.75733658606588, 85.96541579966994, 86.1878453038674, 86.40309966276818, 86.60400373107555, 86.76903207289948, 86.86948410705317, 87.0273373035804, 87.32151826074478, 87.43632058549186, 87.51524718375548, 87.6802755255794, 87.78072755973308, 87.9529310468537, 88.07490851689748, 88.14665996986439, 88.23993685872139, 88.36191432876515, 88.44801607232546, 88.57716868766593, 88.68479586711632, 88.82829877305016, 88.96462653368731, 89.09377914902777, 89.23010690966493, 89.39513525148884, 89.48123699504916, 89.56016359331277, 89.59603931979623, 89.70366649924661, 89.76106766162016, 89.8184688239937, 89.92609600344407, 89.97632202052091, 90.01219774700438, 90.05524861878453, 90.04807347348785, 90.15570065293824, 90.277678122982, 90.34225443065222, 90.42835617421254, 90.47140704599268, 90.49293248188276, 90.48575733658608, 90.53598335366291, 90.61490995192652, 90.62208509722322, 90.6364353878166, 90.66513596900337, 90.69383655019014, 90.67948625959676, 90.67231111430006, 90.7297122766736, 90.73688742197031, 90.77276314845376, 90.77993829375045, 90.75123771256368, 90.72253713137691, 90.68666140489346, 90.71536198608023, 90.72253713137691, 90.68666140489346, 90.69383655019014, 90.68666140489346, 90.66513596900337, 90.72253713137691, 90.72253713137691, 90.77276314845376, 90.80146372964053, 90.85168974671738, 90.82298916553061, 90.83733945612398, 90.85168974671738, 90.87321518260745, 90.85168974671738, 90.85168974671738, 90.83733945612398, 90.81581402023392, 90.80146372964053, 90.75841285786038, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2028, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56253139126068, 81.02174069024898, 81.4594245533472, 81.8468823993686, 82.25586568128006, 82.60027265552128, 82.86575303149888, 83.2173351510368, 83.44693980053096, 83.75547104828873, 84.07835258663988, 84.37970868910097, 84.55908732151826, 84.8102174069025, 85.01112147520988, 85.29812728707756, 85.53490708186841, 85.74298629547249, 85.94389036377987, 86.17349501327402, 86.33134820980125, 86.34569850039465, 86.36722393628472, 86.39592451747149, 86.43897538925164, 86.44615053454832, 86.48202626103179, 86.51072684221856, 86.53942742340533, 86.61117887637225, 86.6542297481524, 86.69728061993256, 86.70445576522926, 86.74033149171271, 86.77620721819616, 86.81925808997633, 86.8407835258664, 86.88383439764655, 86.88383439764655, 86.94123556002009, 86.95558585061349, 86.96993614120686, 86.98428643180026, 87.0273373035804, 87.00581186769031, 87.0345124488771, 87.06321303006386, 87.09191361125063, 87.17084020951424, 87.17801535481094, 87.22824137188778, 87.22824137188778, 87.26411709837124, 87.3143431154481, 87.35021884193155, 87.35021884193155, 87.38609456841502, 87.42914544019517, 87.52242232905216, 87.5941737820191, 87.64439979909594, 87.6731003802827, 87.71615125206286, 87.80942814091985, 87.89552988448015, 88.02468249982063, 88.08208366219417, 88.18253569634786, 88.26146229461148, 88.30451316639162, 88.38343976465524, 88.46954150821554, 88.54846810647916, 88.63456985003947, 88.691971012413, 88.75654732008323, 88.77807275597331, 88.85699935423692, 88.90722537131377, 88.97897682428069, 89.02202769606085, 89.1440051661046, 89.20140632847816, 89.24445720025831, 89.344909234412, 89.38796010619215, 89.48841214034584, 89.55298844801608, 89.70366649924661, 89.85434455047715, 89.811293678697, 89.87586998636723, 89.99067231111431, 90.11982492645475, 90.30637870416876, 90.39248044772907, 90.52880820836621, 90.60773480662984, 90.70101169548683, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2028, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56253139126068, 81.02174069024898, 81.3733228097869, 81.71055463873144, 82.1195379206429, 82.49982062136758, 82.87292817679558, 83.19580971514674, 83.4110640740475, 83.66936930472842, 83.97790055248619, 84.23620578316712, 84.46581043266126, 84.71694051804549, 84.93219487694627, 85.18332496233049, 85.39857932123127, 85.62818397072542, 85.8219128937361, 85.96541579966994, 86.1017435603071, 86.19502044916409, 86.30264762861448, 86.37439908158142, 86.41027480806487, 86.45332567984502, 86.48920140632848, 86.5609528592954, 86.58247829518548, 86.59682858577887, 86.66140489344909, 86.71163091052594, 86.7546817823061, 86.80490779938295, 86.86948410705317, 86.92688526942672, 86.96276099591017, 87.07038817536056, 87.1206141924374, 87.1134390471407, 87.17801535481094, 87.22824137188778, 87.24976680777786, 87.3143431154481, 87.32869340604148, 87.34304369663487, 87.37174427782163, 87.38609456841502, 87.45067087608524, 87.47219631197532, 87.52242232905216, 87.5798234914257, 87.65157494439264, 87.65157494439264, 87.69462581617277, 87.78072755973308, 87.81660328621655, 87.8596541579967, 87.89552988448015, 87.945755901557, 87.98163162804047, 88.01033220922724, 88.03903279041401, 88.0462079357107, 88.11078424338093, 88.19688598694124, 88.21841142283131, 88.29733802109493, 88.3260386022817, 88.36908947406185, 88.39779005524862, 88.44084092702877, 88.48389179880894, 88.53411781588576, 88.58434383296262, 88.65609528592955, 88.74219702948984, 88.79959819186338, 88.86417449953362, 88.92875080720385, 89.00767740546746, 89.10095429432447, 89.19423118318146, 89.37360981559877, 89.52428786682931, 89.58168902920285, 89.67496591805984, 89.8184688239937, 89.86869484107054, 89.997847456411, 90.04089832819116, 90.05524861878453, 90.16287579823492, 90.25615268709191, 90.32072899476215, 90.37095501183899, 90.50728277247615, 90.64361053311329, 90.67948625959676, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.4549042118103, 80.93563894668867, 81.35179737389683, 81.68185405754467, 82.15541364712637, 82.4782951854775, 82.8585778862022, 83.19580971514674, 83.56174212527804, 83.88462366362918, 84.17880462079358, 84.4371098514745, 84.7025902274521, 84.99677118461649, 85.21920068881396, 85.48468106479157, 85.72146085958241, 85.95824065437326, 86.1878453038674, 86.35287364569133, 86.51790198751526, 86.66858003874579, 86.86230896175648, 87.0273373035804, 87.2067159359977, 87.37891942311832, 87.51524718375548, 87.63004950850255, 87.73767668795294, 87.80225299562316, 87.89552988448015, 88.02468249982063, 88.21123627753462, 88.29733802109493, 88.43366578173209, 88.59151897825932, 88.74937217478654, 88.7924230465667, 88.82829877305016, 88.99332711487408, 89.04355313195093, 89.18705603788477, 89.2659826361484, 89.38078496089547, 89.45253641386238, 89.47406184975246, 89.49558728564254, 89.58168902920285, 89.68214106335654, 89.74671737102676, 89.76106766162016, 89.8184688239937, 89.85434455047715, 89.91174571285069, 89.94044629403746, 89.96197172992754, 90.04807347348785, 90.11982492645475, 90.11982492645475, 90.12700007175145, 90.15570065293824, 90.20592667001507, 90.24180239649853, 90.27050297768531, 90.34225443065222, 90.34225443065222, 90.3566047212456, 90.3637798665423, 90.41400588361914, 90.47140704599268, 90.49293248188276, 90.52880820836621, 90.52880820836621, 90.50728277247615, 90.50728277247615, 90.54315849895961, 90.5503336442563, 90.62926024251992, 90.67948625959676, 90.71536198608023, 90.744062567267, 90.76558800315708, 90.82298916553061, 90.81581402023392, 90.85886489201407, 90.89474061849752, 90.87321518260745, 90.85168974671738, 90.8301643108273, 90.85886489201407, 90.86604003731075, 90.80863887493722, 90.81581402023392, 90.80863887493722, 90.82298916553061, 90.82298916553061, 90.84451460142068, 90.77993829375045, 90.75123771256368, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.51948051948052, 81.02891583554567, 81.50247542512736, 81.8325321087752, 82.22716510009327, 82.57157207433451, 83.00925593743274, 83.30343689459711, 83.53304154409128, 83.7913467747722, 84.09270287723328, 84.31513238143073, 84.5375618856282, 84.86044342397933, 85.04699720169334, 85.29095214178088, 85.51338164597834, 85.74298629547249, 85.98694123556002, 86.26677190213103, 86.44615053454832, 86.66858003874579, 86.86230896175648, 87.0416875941738, 87.17801535481094, 87.23541651718448, 87.33586855133817, 87.42914544019517, 87.4937217478654, 87.4937217478654, 87.55829805553563, 87.60852407261247, 87.665925234986, 87.6731003802827, 87.70180096146947, 87.70897610676617, 87.71615125206286, 87.73767668795294, 87.76637726913971, 87.79507785032648, 87.82377843151325, 87.8596541579967, 87.91705532037024, 87.9601061921504, 87.93858075626032, 87.96728133744709, 87.97445648274378, 88.01750735452394, 88.06773337160078, 88.07490851689748, 88.10360909808423, 88.1394848245677, 88.18971084164454, 88.23993685872139, 88.24711200401808, 88.30451316639162, 88.36191432876515, 88.38343976465524, 88.45519121762216, 88.49824208940231, 88.56281839707253, 88.65609528592955, 88.70632130300639, 88.72067159359977, 88.72784673889646, 88.80677333716008, 88.84264906364353, 88.96462653368731, 88.96462653368731, 89.05790342254431, 89.03637798665423, 89.10812943962115, 89.16553060199469, 89.29468321733516, 89.38078496089547, 89.40948554208224, 89.48123699504916, 89.48841214034584, 89.50276243093923, 89.54581330271938, 89.52428786682931, 89.64626533687307, 89.71801678984, 89.78976824280693, 89.86869484107054, 89.8902202769606, 89.86869484107054, 89.8973954222573, 89.96197172992754, 89.98349716581761, 90.05524861878453, 90.14135036234484, 90.24897754179523, 90.3566047212456, 90.37813015713569, 90.47858219128938, 90.50010762717945, 90.5503336442563, 90.60773480662984, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.51948051948052, 80.97151467317214, 81.50965057042406, 81.88275812585205, 82.17693908301644, 82.4854703307742, 82.81552701442205, 83.16710913395997, 83.4110640740475, 83.69806988591519, 83.91332424481595, 84.19315491138696, 84.48733586855134, 84.74564109923226, 84.96089545813304, 85.1689746717371, 85.34117815885772, 85.62100882542872, 85.80038745784603, 85.98694123556002, 86.24524646624094, 86.4676759704384, 86.66140489344909, 86.88383439764655, 87.07038817536056, 87.24259166248117, 87.46502116667862, 87.52959747434886, 87.63722465379924, 87.74485183324963, 87.8596541579967, 88.01033220922724, 88.14665996986439, 88.29733802109493, 88.46954150821554, 88.64174499533615, 88.70632130300639, 88.82112362775347, 88.87134964483032, 88.99332711487408, 89.12965487551124, 89.21575661907154, 89.28033292674176, 89.27315778144508, 89.38078496089547, 89.40948554208224, 89.52428786682931, 89.57451388390615, 89.60321446509292, 89.624739900983, 89.73954222573008, 89.83999425988375, 89.84716940518047, 89.8973954222573, 89.93327114874077, 89.96914687522423, 90.09112434526799, 90.1772260888283, 90.22745210590514, 90.26332783238861, 90.24180239649853, 90.29202841357538, 90.277678122982, 90.31355384946545, 90.28485326827868, 90.31355384946545, 90.34942957594892, 90.42835617421254, 90.49293248188276, 90.47858219128938, 90.54315849895961, 90.57185908014638, 90.55750878955298, 90.60055966133315, 90.58620937073975, 90.61490995192652, 90.6364353878166, 90.67231111430006, 90.68666140489346, 90.68666140489346, 90.72253713137691, 90.69383655019014, 90.67231111430006, 90.70818684078354, 90.73688742197031, 90.75123771256368, 90.77993829375045, 90.84451460142068, 90.86604003731075, 90.85886489201407, 90.85168974671738, 90.85168974671738, 90.84451460142068, 90.85886489201407, 90.81581402023392, 90.78711343904715, 90.77993829375045, 90.77276314845376, 90.73688742197031, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.52665566477721, 80.92846380139198, 81.3733228097869, 81.78230609169836, 82.05496161297266, 82.3993685872139, 82.70789983497167, 82.95902992035589, 83.26038602281697, 83.62631843294827, 83.8774485183325, 84.22185549257372, 84.49451101384804, 84.65953935567195, 84.79586711630911, 85.02547176580326, 85.26225156059411, 85.53490708186841, 85.73581115017579, 86.0156418167468, 86.17349501327402, 86.40309966276818, 86.63270431226233, 86.85513381645978, 87.04886273947048, 87.24259166248117, 87.47219631197532, 87.54394776494225, 87.63004950850255, 87.73767668795294, 87.84530386740332, 88.07490851689748, 88.17536055105116, 88.30451316639162, 88.48389179880894, 88.63456985003947, 88.74937217478654, 88.84264906364353, 88.92157566190716, 89.01485255076416, 89.11530458491785, 89.17270574729139, 89.2588074908517, 89.29468321733516, 89.37360981559877, 89.45971155915908, 89.56733873860946, 89.6175647556863, 89.624739900983, 89.70366649924661, 89.75389251632346, 89.8902202769606, 89.87586998636723, 89.904570567554, 90.04089832819116, 90.07677405467462, 90.14852550764154, 90.22027696060846, 90.27050297768531, 90.28485326827868, 90.31355384946545, 90.277678122982, 90.29920355887207, 90.32790414005885, 90.32072899476215, 90.32790414005885, 90.38530530243239, 90.43553131950922, 90.44270646480591, 90.48575733658608, 90.49293248188276, 90.51445791777284, 90.54315849895961, 90.59338451603645, 90.60773480662984, 90.60055966133315, 90.64361053311329, 90.65078567840999, 90.71536198608023, 90.70818684078354, 90.68666140489346, 90.72253713137691, 90.70818684078354, 90.744062567267, 90.77993829375045, 90.79428858434385, 90.80863887493722, 90.82298916553061, 90.85168974671738, 90.85168974671738, 90.86604003731075, 90.84451460142068, 90.85886489201407, 90.84451460142068, 90.81581402023392, 90.76558800315708, 90.77276314845376, 90.77276314845376, 90.744062567267, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.52665566477721, 80.92846380139198, 81.3733228097869, 81.78230609169836, 82.05496161297266, 82.3993685872139, 82.70789983497167, 82.95902992035589, 83.26038602281697, 83.62631843294827, 83.8774485183325, 84.22185549257372, 84.49451101384804, 84.65953935567195, 84.79586711630911, 85.02547176580326, 85.26225156059411, 85.53490708186841, 85.73581115017579, 86.0156418167468, 86.17349501327402, 86.40309966276818, 86.63270431226233, 86.85513381645978, 87.04886273947048, 87.24259166248117, 87.47219631197532, 87.54394776494225, 87.63004950850255, 87.73767668795294, 87.84530386740332, 88.07490851689748, 88.17536055105116, 88.30451316639162, 88.48389179880894, 88.63456985003947, 88.74937217478654, 88.84264906364353, 88.92157566190716, 89.01485255076416, 89.11530458491785, 89.17270574729139, 89.2588074908517, 89.29468321733516, 89.37360981559877, 89.45971155915908, 89.56733873860946, 89.6175647556863, 89.624739900983, 89.70366649924661, 89.75389251632346, 89.8902202769606, 89.87586998636723, 89.904570567554, 90.04089832819116, 90.07677405467462, 90.14852550764154, 90.22027696060846, 90.27050297768531, 90.28485326827868, 90.31355384946545, 90.277678122982, 90.29920355887207, 90.32790414005885, 90.32072899476215, 90.32790414005885, 90.38530530243239, 90.43553131950922, 90.44270646480591, 90.48575733658608, 90.49293248188276, 90.51445791777284, 90.54315849895961, 90.59338451603645, 90.60773480662984, 90.60055966133315, 90.64361053311329, 90.65078567840999, 90.71536198608023, 90.70818684078354, 90.68666140489346, 90.72253713137691, 90.70818684078354, 90.744062567267, 90.77993829375045, 90.79428858434385, 90.80863887493722, 90.82298916553061, 90.85168974671738, 90.85168974671738, 90.86604003731075, 90.84451460142068, 90.85886489201407, 90.84451460142068, 90.81581402023392, 90.76558800315708, 90.77276314845376, 90.77276314845376, 90.744062567267, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2029, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.48360479299707, 81.0002152543589, 81.44507426275382, 81.71772978402812, 82.1267130659396, 82.51417091196097, 82.89445361268565, 83.2173351510368, 83.58326756116811, 83.83439764655235, 84.20750520198035, 84.40840927028772, 84.65236421037527, 84.8963191504628, 85.13309894525365, 85.37705388534118, 85.6353591160221, 85.9080146372964, 86.05151754323025, 86.2954724833178, 86.44615053454832, 86.63987945755902, 86.81208294467963, 86.91253497883332, 87.05603788476716, 87.17084020951424, 87.2067159359977, 87.24259166248117, 87.29281767955801, 87.34304369663487, 87.37891942311832, 87.40762000430509, 87.42197029489847, 87.47219631197532, 87.53677261964555, 87.62287436320587, 87.665925234986, 87.68745067087609, 87.70180096146947, 87.70180096146947, 87.76637726913971, 87.81660328621655, 87.84530386740332, 87.88117959388678, 87.90270502977687, 87.9601061921504, 88.02468249982063, 88.07490851689748, 88.10360909808423, 88.14665996986439, 88.21123627753462, 88.20406113223794, 88.18971084164454, 88.21841142283131, 88.26146229461148, 88.31886345698501, 88.36908947406185, 88.44084092702877, 88.44084092702877, 88.53411781588576, 88.61304441414939, 88.67044557652292, 88.74937217478654, 88.83547391834685, 88.90722537131377, 88.93592595250054, 88.99332711487408, 89.04355313195093, 89.04355313195093, 89.12965487551124, 89.21575661907154, 89.24445720025831, 89.30185836263185, 89.3520843797087, 89.3377340891153, 89.45971155915908, 89.47406184975246, 89.48123699504916, 89.47406184975246, 89.55298844801608, 89.57451388390615, 89.64626533687307, 89.71801678984, 89.73954222573008, 89.76106766162016, 89.78976824280693, 89.83999425988375, 89.93327114874077, 90.01937289230108, 90.0982994905647, 90.14852550764154, 90.19875152471838, 90.23462725120184, 90.3637798665423, 90.42118102891584, 90.49293248188276, 90.52880820836621, 90.66513596900337, 90.70818684078354, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2029, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.57688168185406, 81.03609098084237, 81.43072397216044, 81.87558298055536, 82.26304082657674, 82.62179809141136, 83.01643108272943, 83.29626174930043, 83.55456697998135, 83.84874793714573, 84.16445433020019, 84.46581043266126, 84.7097653727488, 84.97524574872641, 85.11874865466027, 85.43445504771472, 85.58513309894525, 85.74298629547249, 85.90083949199972, 86.07304297912033, 86.24524646624094, 86.36004879098802, 86.51790198751526, 86.64705460285572, 86.76903207289948, 86.91971012413002, 87.0345124488771, 87.1277893377341, 87.22824137188778, 87.28564253426131, 87.37174427782163, 87.40762000430509, 87.50089689316209, 87.55112291023893, 87.60134892731578, 87.60134892731578, 87.665925234986, 87.73767668795294, 87.80942814091985, 87.82377843151325, 87.85247901270002, 87.88835473918347, 87.91705532037024, 87.9601061921504, 88.03185764511731, 88.08925880749085, 88.15383511516109, 88.18971084164454, 88.24711200401808, 88.30451316639162, 88.3260386022817, 88.36908947406185, 88.37626461935855, 88.40496520054532, 88.44084092702877, 88.46236636291886, 88.5197675252924, 88.54846810647916, 88.58434383296262, 88.61304441414939, 88.63456985003947, 88.67044557652292, 88.76372246537993, 88.7924230465667, 88.81394848245677, 88.84264906364353, 88.83547391834685, 88.90005022601709, 88.93592595250054, 89.01485255076416, 89.02202769606085, 89.04355313195093, 89.05790342254431, 89.1440051661046, 89.18705603788477, 89.2659826361484, 89.29468321733516, 89.344909234412, 89.37360981559877, 89.36643467030207, 89.438186123269, 89.50993757623593, 89.60321446509292, 89.72519193513669, 89.76106766162016, 89.76824280691685, 89.83281911458707, 89.84716940518047, 89.904570567554, 89.98349716581761, 89.997847456411, 90.03372318289445, 90.06959890937792, 90.0982994905647, 90.22027696060846, 90.25615268709191, 90.37813015713569, 90.52880820836621, 90.66513596900337, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54100595537061, 81.05044127143574, 81.43789911745714, 81.77513094640166, 82.21281480949989, 82.52134605725766, 82.80117672382866, 83.14558369806988, 83.44693980053096, 83.76264619358543, 84.0711774413432, 84.26490636435389, 84.51603644973811, 84.67388964626534, 84.9034942957595, 85.11157350936357, 85.39140417593457, 85.69993542369232, 85.79321231254933, 85.92236492788979, 86.22372103035086, 86.38874937217479, 86.61835402166894, 86.78338236349286, 87.0273373035804, 87.29999282485471, 87.3932697137117, 87.51524718375548, 87.61569921790917, 87.75920212384301, 87.91705532037024, 87.98880677333716, 88.18971084164454, 88.27581258520485, 88.35473918346847, 88.42649063643539, 88.56281839707253, 88.62739470474278, 88.76372246537993, 88.84982420894023, 88.96462653368731, 89.03637798665423, 89.10095429432447, 89.18705603788477, 89.2659826361484, 89.33055894381862, 89.45971155915908, 89.5386381574227, 89.56016359331277, 89.63191504627969, 89.71084164454331, 89.75389251632346, 89.82564396929038, 89.88304513166392, 89.91174571285069, 89.96197172992754, 90.06959890937792, 90.09112434526799, 90.10547463586138, 90.12700007175145, 90.1700509435316, 90.19157637942169, 90.21310181531177, 90.28485326827868, 90.34225443065222, 90.37813015713569, 90.39248044772907, 90.39248044772907, 90.43553131950922, 90.47140704599268, 90.50010762717945, 90.50010762717945, 90.46423190069599, 90.47140704599268, 90.46423190069599, 90.51445791777284, 90.51445791777284, 90.55750878955298, 90.60055966133315, 90.60055966133315, 90.6364353878166, 90.61490995192652, 90.65796082370669, 90.65078567840999, 90.72253713137691, 90.75841285786038, 90.78711343904715, 90.82298916553061, 90.81581402023392, 90.82298916553061, 90.80863887493722, 90.79428858434385, 90.78711343904715, 90.79428858434385, 90.80863887493722, 90.80863887493722, 90.76558800315708, 90.7297122766736, 90.73688742197031, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56253139126068, 81.06479156202914, 81.52400086101744, 81.86840783525867, 82.14106335653297, 82.64332352730143, 82.98773050154266, 83.28908660400374, 83.61914328765158, 83.83439764655235, 84.16445433020019, 84.46581043266126, 84.7097653727488, 85.02547176580326, 85.31965272296765, 85.52055679127503, 85.76451173136256, 85.93671521848317, 86.05151754323025, 86.25959675683433, 86.49637655162518, 86.66858003874579, 86.87665925234987, 87.05603788476716, 87.15648991892087, 87.38609456841502, 87.53677261964555, 87.65875008968932, 87.74485183324963, 87.89552988448015, 87.98163162804047, 88.08925880749085, 88.08208366219417, 88.08208366219417, 88.11795938867762, 88.15383511516109, 88.18971084164454, 88.22558656812801, 88.23276171342471, 88.26863743990816, 88.26146229461148, 88.33321374757838, 88.38343976465524, 88.39779005524862, 88.42649063643539, 88.5197675252924, 88.53411781588576, 88.53411781588576, 88.6058692688527, 88.62739470474278, 88.66327043122624, 88.66327043122624, 88.72784673889646, 88.73502188419316, 88.74937217478654, 88.77089761067663, 88.81394848245677, 88.8856999354237, 88.89287508072039, 88.94310109779724, 88.93592595250054, 88.98615196957739, 89.05072827724761, 89.10095429432447, 89.17270574729139, 89.23728205496163, 89.22293176436823, 89.24445720025831, 89.28033292674176, 89.28033292674176, 89.31620865322523, 89.40948554208224, 89.47406184975246, 89.4310109779723, 89.51711272153261, 89.56016359331277, 89.58886417449953, 89.64626533687307, 89.68931620865322, 89.77541795221353, 89.79694338810361, 89.83281911458707, 89.91174571285069, 89.87586998636723, 89.96914687522423, 90.03372318289445, 90.03372318289445, 90.11264978115807, 90.14135036234484, 90.14852550764154, 90.22745210590514, 90.28485326827868, 90.42118102891584, 90.41400588361914, 90.50728277247615, 90.64361053311329, 90.66513596900337, 90.79428858434385, 90.77993829375045, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.58405682715075, 81.07914185262251, 81.49530027983067, 81.87558298055536, 82.16976393771974, 82.47112004018081, 82.80117672382866, 83.0523068092129, 83.33931262108058, 83.61196814235488, 83.89897395422258, 84.17880462079358, 84.49451101384804, 84.78151682571571, 85.0828729281768, 85.24790127000072, 85.52773193657171, 85.85778862021957, 86.04434239793356, 86.19502044916409, 86.38874937217479, 86.59682858577887, 86.73315634641602, 86.87665925234987, 87.0416875941738, 87.17801535481094, 87.43632058549186, 87.60852407261247, 87.73050154265624, 87.86682930329339, 87.9601061921504, 88.10360909808423, 88.22558656812801, 88.412140345842, 88.46954150821554, 88.58434383296262, 88.74219702948984, 88.7924230465667, 88.83547391834685, 88.94310109779724, 89.065078567841, 89.09377914902777, 89.13683002080792, 89.21575661907154, 89.31620865322523, 89.37360981559877, 89.48841214034584, 89.57451388390615, 89.63909019157639, 89.75389251632346, 89.83281911458707, 89.83281911458707, 89.87586998636723, 89.94762143933414, 90.01937289230108, 90.04807347348785, 90.11264978115807, 90.11982492645475, 90.16287579823492, 90.277678122982, 90.28485326827868, 90.32790414005885, 90.32072899476215, 90.37095501183899, 90.45705675539931, 90.47858219128938, 90.51445791777284, 90.56468393484968, 90.59338451603645, 90.62926024251992, 90.65078567840999, 90.6364353878166, 90.60773480662984, 90.69383655019014, 90.70101169548683, 90.65796082370669, 90.67231111430006, 90.66513596900337, 90.70818684078354, 90.70101169548683, 90.71536198608023, 90.73688742197031, 90.7297122766736, 90.7297122766736, 90.75123771256368, 90.80146372964053, 90.8301643108273, 90.85886489201407, 90.84451460142068, 90.85168974671738, 90.86604003731075, 90.8301643108273, 90.85168974671738, 90.85886489201407, 90.85168974671738, 90.82298916553061, 90.79428858434385, 90.75123771256368, 90.73688742197031, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.50513022888714, 81.06479156202914, 81.4665996986439, 81.81100667288513, 82.1195379206429, 82.4854703307742, 82.70072468967497, 83.0451316639162, 83.33931262108058, 83.5976178517615, 83.92049939011265, 84.12857860371673, 84.41558441558442, 84.7097653727488, 84.94654516753964, 85.26225156059411, 85.49185621008826, 85.62818397072542, 85.87213891081295, 86.17349501327402, 86.35287364569133, 86.51072684221856, 86.68293032933917, 86.85513381645978, 87.0273373035804, 87.2138910812944, 87.33586855133817, 87.55829805553563, 87.71615125206286, 87.85247901270002, 87.9601061921504, 88.11078424338093, 88.24711200401808, 88.36191432876515, 88.44084092702877, 88.56999354236925, 88.74219702948984, 88.79959819186338, 88.85699935423692, 88.91440051661046, 88.97897682428069, 89.09377914902777, 89.15118031140132, 89.28750807203846, 89.38796010619215, 89.48841214034584, 89.55298844801608, 89.63191504627969, 89.66779077276315, 89.74671737102676, 89.85434455047715, 89.904570567554, 89.95479658463084, 90.00502260170768, 90.01219774700438, 90.02654803759776, 90.10547463586138, 90.14852550764154, 90.19875152471838, 90.23462725120184, 90.31355384946545, 90.40683073832245, 90.43553131950922, 90.43553131950922, 90.50010762717945, 90.52163306306953, 90.52880820836621, 90.5503336442563, 90.56468393484968, 90.58620937073975, 90.60773480662984, 90.61490995192652, 90.65078567840999, 90.64361053311329, 90.62208509722322, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.67948625959676, 90.67948625959676, 90.72253713137691, 90.7297122766736, 90.72253713137691, 90.73688742197031, 90.744062567267, 90.77993829375045, 90.78711343904715, 90.84451460142068, 90.84451460142068, 90.88039032790415, 90.85886489201407, 90.88039032790415, 90.85886489201407, 90.85886489201407, 90.8301643108273, 90.8301643108273, 90.79428858434385, 90.75123771256368, 90.7297122766736, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.50513022888714, 81.06479156202914, 81.4665996986439, 81.81100667288513, 82.1195379206429, 82.4854703307742, 82.70072468967497, 83.0451316639162, 83.33931262108058, 83.5976178517615, 83.92049939011265, 84.12857860371673, 84.41558441558442, 84.7097653727488, 84.94654516753964, 85.26225156059411, 85.49185621008826, 85.62818397072542, 85.87213891081295, 86.17349501327402, 86.35287364569133, 86.51072684221856, 86.68293032933917, 86.85513381645978, 87.0273373035804, 87.2138910812944, 87.33586855133817, 87.55829805553563, 87.71615125206286, 87.85247901270002, 87.9601061921504, 88.11078424338093, 88.24711200401808, 88.36191432876515, 88.44084092702877, 88.56999354236925, 88.74219702948984, 88.79959819186338, 88.85699935423692, 88.91440051661046, 88.97897682428069, 89.09377914902777, 89.15118031140132, 89.28750807203846, 89.38796010619215, 89.48841214034584, 89.55298844801608, 89.63191504627969, 89.66779077276315, 89.74671737102676, 89.85434455047715, 89.904570567554, 89.95479658463084, 90.00502260170768, 90.01219774700438, 90.02654803759776, 90.10547463586138, 90.14852550764154, 90.19875152471838, 90.23462725120184, 90.31355384946545, 90.40683073832245, 90.43553131950922, 90.43553131950922, 90.50010762717945, 90.52163306306953, 90.52880820836621, 90.5503336442563, 90.56468393484968, 90.58620937073975, 90.60773480662984, 90.61490995192652, 90.65078567840999, 90.64361053311329, 90.62208509722322, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.67948625959676, 90.67948625959676, 90.72253713137691, 90.7297122766736, 90.72253713137691, 90.73688742197031, 90.744062567267, 90.77993829375045, 90.78711343904715, 90.84451460142068, 90.84451460142068, 90.88039032790415, 90.85886489201407, 90.88039032790415, 90.85886489201407, 90.85886489201407, 90.8301643108273, 90.8301643108273, 90.79428858434385, 90.75123771256368, 90.7297122766736, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2030, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.60558226304083, 81.08631699791921, 81.4665996986439, 81.86123268996197, 82.14823850182967, 82.52852120255434, 83.00208079213604, 83.28908660400374, 83.62631843294827, 83.91332424481595, 84.17880462079358, 84.4299347061778, 84.68823993685872, 84.95372031283634, 85.1761498170338, 85.42010475712134, 85.60665853483533, 85.83626318432948, 86.04434239793356, 86.27394704742771, 86.4676759704384, 86.58247829518548, 86.81208294467963, 86.91253497883332, 87.1206141924374, 87.27129224366794, 87.4004448590084, 87.48654660256871, 87.52959747434886, 87.56547320083232, 87.61569921790917, 87.68745067087609, 87.68745067087609, 87.75202697854631, 87.76637726913971, 87.81660328621655, 87.81660328621655, 87.85247901270002, 87.89552988448015, 87.91705532037024, 87.9529310468537, 87.98163162804047, 87.99598191863386, 88.00315706393054, 88.03903279041401, 88.11795938867762, 88.20406113223794, 88.23276171342471, 88.25428714931478, 88.31886345698501, 88.412140345842, 88.46236636291886, 88.51259237999571, 88.54129296118248, 88.58434383296262, 88.66327043122624, 88.71349644830309, 88.77807275597331, 88.85699935423692, 88.84982420894023, 88.82112362775347, 88.82829877305016, 88.85699935423692, 88.91440051661046, 88.93592595250054, 88.97897682428069, 88.99332711487408, 89.0722537131377, 89.10812943962115, 89.12247973021454, 89.20858147377486, 89.20858147377486, 89.30903350792853, 89.35925952500538, 89.40948554208224, 89.39513525148884, 89.4238358326756, 89.49558728564254, 89.51711272153261, 89.531463012126, 89.6175647556863, 89.66779077276315, 89.74671737102676, 89.83999425988375, 89.94762143933414, 90.01937289230108, 90.10547463586138, 90.21310181531177, 90.22027696060846, 90.28485326827868, 90.3637798665423, 90.43553131950922, 90.5503336442563, 90.5503336442563, 90.60773480662984, 90.65078567840999, 90.70818684078354, 90.70101169548683, 90.73688742197031, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2030, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.52665566477721, 81.03609098084237, 81.50965057042406, 81.94015928822559, 82.3132668436536, 82.65049867259813, 82.82270215971874, 83.12405826217982, 83.31778718519051, 83.59044270646481, 83.79852192006888, 84.12857860371673, 84.39405897969434, 84.56626246681496, 84.79586711630911, 85.01829662050658, 85.25507641529741, 85.4488053383081, 85.59948338953865, 85.8290880390328, 85.98694123556002, 86.1878453038674, 86.37439908158142, 86.44615053454832, 86.50355169692186, 86.54660256870201, 86.55377771399871, 86.60400373107555, 86.64705460285572, 86.67575518404249, 86.69728061993256, 86.74033149171271, 86.79055750878956, 86.85513381645978, 86.92688526942672, 86.94123556002009, 87.00581186769031, 87.08473846595393, 87.14213962832747, 87.19954079070101, 87.27846738896463, 87.3071679701514, 87.32151826074478, 87.37174427782163, 87.42197029489847, 87.52959747434886, 87.62287436320587, 87.6731003802827, 87.7735524144364, 87.80942814091985, 87.88835473918347, 87.90988017507355, 87.945755901557, 87.96728133744709, 88.03903279041401, 88.08208366219417, 88.08925880749085, 88.16818540575447, 88.26863743990816, 88.27581258520485, 88.29016287579825, 88.40496520054532, 88.47671665351223, 88.51259237999571, 88.53411781588576, 88.59151897825932, 88.63456985003947, 88.74219702948984, 88.81394848245677, 88.84264906364353, 88.90722537131377, 88.95027624309392, 89.01485255076416, 89.05790342254431, 89.10812943962115, 89.11530458491785, 89.10095429432447, 89.158355456698, 89.12247973021454, 89.17270574729139, 89.20858147377486, 89.21575661907154, 89.23728205496163, 89.2659826361484, 89.33055894381862, 89.30903350792853, 89.30903350792853, 89.38078496089547, 89.438186123269, 89.5386381574227, 89.63191504627969, 89.68214106335654, 89.82564396929038, 89.94044629403746, 90.0982994905647, 90.20592667001507, 90.34942957594892, 90.50010762717945, 90.60773480662984, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "MiniCPM", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.57688168185406, 81.02174069024898, 81.4594245533472, 81.85405754466528, 82.27739111717013, 82.61462294611465, 82.90162875798235, 83.08818253569635, 83.44693980053096, 83.71242017650857, 84.03530171485973, 84.36535839850757, 84.51603644973811, 84.84609313338596, 85.06134749228671, 85.29812728707756, 85.59230824424195, 85.87213891081295, 86.05151754323025, 86.2882973380211, 86.5609528592954, 86.69728061993256, 86.91971012413002, 87.02016215828371, 87.16366506421755, 87.34304369663487, 87.4937217478654, 87.57264834612901, 87.70180096146947, 87.78790270502978, 87.93858075626032, 88.00315706393054, 88.20406113223794, 88.29016287579825, 88.37626461935855, 88.53411781588576, 88.62739470474278, 88.71349644830309, 88.78524790127, 88.91440051661046, 89.05790342254431, 89.1440051661046, 89.23010690966493, 89.30903350792853, 89.40231039678554, 89.438186123269, 89.48841214034584, 89.5386381574227, 89.63909019157639, 89.66061562746646, 89.75389251632346, 89.79694338810361, 89.83281911458707, 89.904570567554, 89.95479658463084, 90.06242376408122, 90.0982994905647, 90.15570065293824, 90.18440123412499, 90.20592667001507, 90.23462725120184, 90.24897754179523, 90.28485326827868, 90.29920355887207, 90.3566047212456, 90.40683073832245, 90.39248044772907, 90.44270646480591, 90.44988161010261, 90.49293248188276, 90.47858219128938, 90.49293248188276, 90.52880820836621, 90.55750878955298, 90.59338451603645, 90.59338451603645, 90.62926024251992, 90.61490995192652, 90.62926024251992, 90.69383655019014, 90.68666140489346, 90.67948625959676, 90.67948625959676, 90.70818684078354, 90.744062567267, 90.77276314845376, 90.76558800315708, 90.75841285786038, 90.76558800315708, 90.79428858434385, 90.78711343904715, 90.77276314845376, 90.73688742197031, 90.72253713137691, 90.73688742197031, 90.77276314845376, 90.81581402023392, 90.79428858434385, 90.75841285786038, 90.73688742197031]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.61993255363421, 81.12219272440268, 81.58857716868766, 81.91145870703882, 82.19128937360982, 82.56439692903783, 82.92315419387243, 83.25321087752027, 83.62631843294827, 83.89897395422258, 84.20750520198035, 84.54473703092488, 84.8173925521992, 85.04699720169334, 85.19050010762719, 85.42010475712134, 85.62100882542872, 85.85778862021957, 86.07304297912033, 86.19502044916409, 86.3815742268781, 86.53942742340533, 86.64705460285572, 86.76185692760279, 86.8407835258664, 86.92688526942672, 86.98428643180026, 87.04886273947048, 87.09908875654733, 87.15648991892087, 87.22106622659109, 87.25694195307454, 87.3071679701514, 87.29999282485471, 87.34304369663487, 87.38609456841502, 87.41479514960179, 87.46502116667862, 87.51524718375548, 87.55829805553563, 87.64439979909594, 87.72332639735954, 87.80225299562316, 87.83812872210662, 87.89552988448015, 87.98880677333716, 88.03903279041401, 88.10360909808423, 88.16818540575447, 88.24711200401808, 88.3260386022817, 88.39061490995192, 88.42649063643539, 88.54129296118248, 88.6058692688527, 88.62739470474278, 88.75654732008323, 88.87134964483032, 88.90722537131377, 89.00050226017078, 89.00767740546746, 89.16553060199469, 89.24445720025831, 89.2659826361484, 89.3377340891153, 89.38078496089547, 89.438186123269, 89.50276243093923, 89.55298844801608, 89.66061562746646, 89.72519193513669, 89.72519193513669, 89.79694338810361, 89.85434455047715, 89.95479658463084, 90.05524861878453, 90.0839491999713, 90.1772260888283, 90.19875152471838, 90.26332783238861, 90.26332783238861, 90.31355384946545, 90.37095501183899, 90.39965559302576, 90.48575733658608, 90.52880820836621, 90.51445791777284, 90.55750878955298, 90.62208509722322, 90.62208509722322, 90.66513596900337, 90.72253713137691, 90.65078567840999, 90.65078567840999, 90.64361053311329, 90.65078567840999, 90.66513596900337, 90.55750878955298, 90.52163306306953, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54818110066729, 81.09349214321591, 81.50247542512736, 81.80383152758843, 82.16258879242305, 82.55722178374111, 82.88010332209228, 83.12405826217982, 83.44693980053096, 83.74112075769534, 84.09270287723328, 84.37253354380427, 84.65953935567195, 84.8963191504628, 85.16179952644042, 85.3627035947478, 85.49903135538494, 85.67840998780225, 85.83626318432948, 86.09456841501041, 86.24524646624094, 86.46050082514172, 86.61835402166894, 86.86948410705317, 87.09191361125063, 87.2067159359977, 87.45784602138194, 87.5798234914257, 87.80225299562316, 87.90988017507355, 87.97445648274378, 88.11078424338093, 88.27581258520485, 88.46954150821554, 88.691971012413, 88.878524790127, 89.02202769606085, 89.12965487551124, 89.23728205496163, 89.24445720025831, 89.38796010619215, 89.46688670445577, 89.531463012126, 89.624739900983, 89.67496591805984, 89.811293678697, 89.84716940518047, 89.91892085814737, 90.01219774700438, 90.06959890937792, 90.11982492645475, 90.16287579823492, 90.22745210590514, 90.25615268709191, 90.31355384946545, 90.3637798665423, 90.43553131950922, 90.42835617421254, 90.42835617421254, 90.49293248188276, 90.52880820836621, 90.57185908014638, 90.57903422544307, 90.60055966133315, 90.62926024251992, 90.64361053311329, 90.65078567840999, 90.62926024251992, 90.62926024251992, 90.62926024251992, 90.70818684078354, 90.75123771256368, 90.78711343904715, 90.81581402023392, 90.79428858434385, 90.75841285786038, 90.71536198608023, 90.65796082370669, 90.69383655019014, 90.65078567840999, 90.68666140489346, 90.67231111430006, 90.62208509722322, 90.59338451603645, 90.57185908014638, 90.58620937073975, 90.55750878955298, 90.5503336442563, 90.53598335366291, 90.50010762717945, 90.44988161010261, 90.43553131950922, 90.41400588361914, 90.41400588361914, 90.3566047212456, 90.31355384946545, 90.22027696060846, 90.16287579823492, 90.11982492645475, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.55535624596398, 80.98586496376552, 81.40202339097367, 81.72490492932482, 82.09083733945613, 82.44959460429074, 82.72942527086174, 83.03795651861951, 83.46846523642104, 83.82722250125565, 84.11422831312333, 84.4227595608811, 84.65953935567195, 84.93219487694627, 85.14744923584703, 85.32682786826433, 85.52773193657171, 85.69993542369232, 85.90083949199972, 86.08021812441702, 86.23807132094426, 86.47485111573509, 86.61835402166894, 86.89100954294324, 87.0273373035804, 87.22824137188778, 87.36456913252493, 87.5798234914257, 87.70180096146947, 87.88117959388678, 88.06773337160078, 88.12513453397432, 88.29733802109493, 88.5197675252924, 88.67762072181962, 88.81394848245677, 88.91440051661046, 89.065078567841, 89.22293176436823, 89.3377340891153, 89.37360981559877, 89.50276243093923, 89.51711272153261, 89.58168902920285, 89.66779077276315, 89.79694338810361, 89.88304513166392, 89.96197172992754, 90.01937289230108, 90.11264978115807, 90.1772260888283, 90.20592667001507, 90.20592667001507, 90.26332783238861, 90.32072899476215, 90.39248044772907, 90.40683073832245, 90.48575733658608, 90.50728277247615, 90.50728277247615, 90.52880820836621, 90.59338451603645, 90.60055966133315, 90.61490995192652, 90.66513596900337, 90.65796082370669, 90.6364353878166, 90.65796082370669, 90.68666140489346, 90.67948625959676, 90.69383655019014, 90.744062567267, 90.77993829375045, 90.82298916553061, 90.79428858434385, 90.76558800315708, 90.73688742197031, 90.70818684078354, 90.70101169548683, 90.68666140489346, 90.69383655019014, 90.67231111430006, 90.62926024251992, 90.57903422544307, 90.60773480662984, 90.59338451603645, 90.55750878955298, 90.5503336442563, 90.54315849895961, 90.52163306306953, 90.45705675539931, 90.42118102891584, 90.42835617421254, 90.40683073832245, 90.3566047212456, 90.32790414005885, 90.21310181531177, 90.16287579823492, 90.11982492645475, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.55535624596398, 80.98586496376552, 81.40202339097367, 81.72490492932482, 82.09083733945613, 82.44959460429074, 82.72942527086174, 83.03795651861951, 83.46846523642104, 83.82722250125565, 84.11422831312333, 84.4227595608811, 84.65953935567195, 84.93219487694627, 85.14744923584703, 85.32682786826433, 85.52773193657171, 85.69993542369232, 85.90083949199972, 86.08021812441702, 86.23807132094426, 86.47485111573509, 86.61835402166894, 86.89100954294324, 87.0273373035804, 87.22824137188778, 87.36456913252493, 87.5798234914257, 87.70180096146947, 87.88117959388678, 88.06773337160078, 88.12513453397432, 88.29733802109493, 88.5197675252924, 88.67762072181962, 88.81394848245677, 88.91440051661046, 89.065078567841, 89.22293176436823, 89.3377340891153, 89.37360981559877, 89.50276243093923, 89.51711272153261, 89.58168902920285, 89.66779077276315, 89.79694338810361, 89.88304513166392, 89.96197172992754, 90.01937289230108, 90.11264978115807, 90.1772260888283, 90.20592667001507, 90.20592667001507, 90.26332783238861, 90.32072899476215, 90.39248044772907, 90.40683073832245, 90.48575733658608, 90.50728277247615, 90.50728277247615, 90.52880820836621, 90.59338451603645, 90.60055966133315, 90.61490995192652, 90.66513596900337, 90.65796082370669, 90.6364353878166, 90.65796082370669, 90.68666140489346, 90.67948625959676, 90.69383655019014, 90.744062567267, 90.77993829375045, 90.82298916553061, 90.79428858434385, 90.76558800315708, 90.73688742197031, 90.70818684078354, 90.70101169548683, 90.68666140489346, 90.69383655019014, 90.67231111430006, 90.62926024251992, 90.57903422544307, 90.60773480662984, 90.59338451603645, 90.55750878955298, 90.5503336442563, 90.54315849895961, 90.52163306306953, 90.45705675539931, 90.42118102891584, 90.42835617421254, 90.40683073832245, 90.3566047212456, 90.32790414005885, 90.21310181531177, 90.16287579823492, 90.11982492645475, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2026, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.59840711774415, 81.07914185262251, 81.48094998923729, 81.85405754466528, 82.29174140776351, 82.60744780081797, 82.93750448446582, 83.2245102963335, 83.61196814235488, 83.89897395422258, 84.18597976609027, 84.49451101384804, 84.73846595393557, 84.9034942957595, 85.21202554351726, 85.42727990241804, 85.61383368013202, 85.75733658606588, 86.0084666714501, 86.25959675683433, 86.49637655162518, 86.70445576522926, 86.76185692760279, 86.78338236349286, 86.81925808997633, 86.89100954294324, 86.94123556002009, 86.98428643180026, 87.0416875941738, 87.08473846595393, 87.1277893377341, 87.17084020951424, 87.24259166248117, 87.26411709837124, 87.29281767955801, 87.35021884193155, 87.44349573078856, 87.50807203845878, 87.54394776494225, 87.61569921790917, 87.6731003802827, 87.70180096146947, 87.75202697854631, 87.79507785032648, 87.88117959388678, 87.98163162804047, 88.06055822630408, 88.11078424338093, 88.20406113223794, 88.31168831168831, 88.37626461935855, 88.43366578173209, 88.48389179880894, 88.54129296118248, 88.62739470474278, 88.68479586711632, 88.70632130300639, 88.77807275597331, 88.82829877305016, 88.87134964483032, 88.94310109779724, 89.05790342254431, 89.10812943962115, 89.19423118318146, 89.24445720025831, 89.31620865322523, 89.39513525148884, 89.45971155915908, 89.49558728564254, 89.56733873860946, 89.6175647556863, 89.72519193513669, 89.78976824280693, 89.8184688239937, 89.91892085814737, 89.93327114874077, 89.95479658463084, 90.03372318289445, 90.0839491999713, 90.21310181531177, 90.27050297768531, 90.34942957594892, 90.44270646480591, 90.50010762717945, 90.53598335366291, 90.61490995192652, 90.65078567840999, 90.67231111430006, 90.68666140489346, 90.75841285786038, 90.79428858434385, 90.77993829375045, 90.75123771256368, 90.67231111430006, 90.68666140489346, 90.62926024251992, 90.62208509722322, 90.55750878955298, 90.51445791777284, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2026, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.59123197244745, 81.09349214321591, 81.51682571572074, 81.94015928822559, 82.29891655306021, 82.65767381789482, 82.93032933916912, 83.23886058692689, 83.52586639879458, 83.77699648417881, 84.07835258663988, 84.36535839850757, 84.6164884838918, 84.9034942957595, 85.16179952644042, 85.47033077419817, 85.65688455191219, 85.86496376551625, 86.02999210734018, 86.1878453038674, 86.32417306450456, 86.47485111573509, 86.63987945755902, 86.74750663700941, 86.8407835258664, 86.88383439764655, 86.90535983353664, 86.92688526942672, 87.00581186769031, 86.99863672239363, 87.08473846595393, 87.1206141924374, 87.14931477362417, 87.2067159359977, 87.22824137188778, 87.24976680777786, 87.3071679701514, 87.35021884193155, 87.37174427782163, 87.40762000430509, 87.4937217478654, 87.55829805553563, 87.5869986367224, 87.62287436320587, 87.75920212384301, 87.82377843151325, 87.85247901270002, 87.89552988448015, 87.945755901557, 87.98163162804047, 87.99598191863386, 88.06055822630408, 88.11078424338093, 88.11078424338093, 88.14665996986439, 88.23276171342471, 88.29733802109493, 88.41931549113869, 88.48389179880894, 88.55564325177585, 88.58434383296262, 88.65609528592955, 88.71349644830309, 88.77089761067663, 88.84982420894023, 88.98615196957739, 89.09377914902777, 89.17270574729139, 89.251632345555, 89.32338379852193, 89.39513525148884, 89.48841214034584, 89.60321446509292, 89.72519193513669, 89.82564396929038, 89.91892085814737, 89.94762143933414, 89.93327114874077, 90.05524861878453, 90.09112434526799, 90.1772260888283, 90.24180239649853, 90.30637870416876, 90.46423190069599, 90.53598335366291, 90.59338451603645, 90.61490995192652, 90.58620937073975, 90.61490995192652, 90.67948625959676, 90.77276314845376, 90.80146372964053, 90.83733945612398, 90.77993829375045, 90.80863887493722, 90.79428858434385, 90.72253713137691, 90.62208509722322, 90.41400588361914, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.48360479299707, 80.9141135107986, 81.33027193800675, 81.71055463873144, 82.03343617708259, 82.45676974958744, 82.80835186912535, 83.1312334074765, 83.38236349286073, 83.69806988591519, 84.02812656956303, 84.24338092846381, 84.58061275740833, 84.78151682571571, 85.0828729281768, 85.29095214178088, 85.55643251775848, 85.76451173136256, 85.99411638085671, 86.15914472268064, 86.37439908158142, 86.54660256870201, 86.73315634641602, 86.94841070531677, 87.16366506421755, 87.35021884193155, 87.54394776494225, 87.78790270502978, 87.9529310468537, 88.15383511516109, 88.31168831168831, 88.44084092702877, 88.62021955944608, 88.71349644830309, 88.82829877305016, 88.92157566190716, 89.03637798665423, 89.12247973021454, 89.27315778144508, 89.4238358326756, 89.5386381574227, 89.65344048216977, 89.76824280691685, 89.8902202769606, 89.8973954222573, 89.93327114874077, 89.96914687522423, 90.0982994905647, 90.11982492645475, 90.22027696060846, 90.28485326827868, 90.34942957594892, 90.37095501183899, 90.3637798665423, 90.40683073832245, 90.46423190069599, 90.44270646480591, 90.50728277247615, 90.53598335366291, 90.60055966133315, 90.62926024251992, 90.67231111430006, 90.67948625959676, 90.66513596900337, 90.70818684078354, 90.70101169548683, 90.69383655019014, 90.66513596900337, 90.67231111430006, 90.70101169548683, 90.67948625959676, 90.67948625959676, 90.67231111430006, 90.62926024251992, 90.62926024251992, 90.65078567840999, 90.62208509722322, 90.55750878955298, 90.5503336442563, 90.58620937073975, 90.62208509722322, 90.62208509722322, 90.6364353878166, 90.62926024251992, 90.61490995192652, 90.62208509722322, 90.60773480662984, 90.58620937073975, 90.55750878955298, 90.56468393484968, 90.52163306306953, 90.46423190069599, 90.42835617421254, 90.34225443065222, 90.31355384946545, 90.27050297768531, 90.23462725120184, 90.16287579823492, 90.11264978115807, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.58405682715075, 81.12936786969937, 81.5598765875009, 82.00473559589582, 82.25586568128006, 82.63614838200473, 82.98773050154266, 83.29626174930043, 83.59044270646481, 83.97072540718949, 84.29360694554066, 84.6164884838918, 84.79586711630911, 85.0828729281768, 85.29095214178088, 85.52773193657171, 85.70711056898904, 85.87213891081295, 86.09456841501041, 86.33134820980125, 86.53942742340533, 86.7546817823061, 86.98428643180026, 87.15648991892087, 87.34304369663487, 87.51524718375548, 87.63722465379924, 87.71615125206286, 87.78790270502978, 87.82377843151325, 87.87400444859009, 87.93858075626032, 87.98880677333716, 88.01033220922724, 88.0533830810074, 88.07490851689748, 88.08208366219417, 88.12513453397432, 88.16818540575447, 88.23276171342471, 88.25428714931478, 88.30451316639162, 88.3260386022817, 88.36908947406185, 88.44084092702877, 88.47671665351223, 88.49106694410563, 88.56999354236925, 88.6058692688527, 88.66327043122624, 88.71349644830309, 88.7924230465667, 88.81394848245677, 88.89287508072039, 88.92875080720385, 88.97897682428069, 89.05790342254431, 89.09377914902777, 89.13683002080792, 89.20140632847816, 89.30903350792853, 89.344909234412, 89.40948554208224, 89.438186123269, 89.50993757623593, 89.56016359331277, 89.58886417449953, 89.65344048216977, 89.71801678984, 89.78259309751023, 89.78976824280693, 89.8902202769606, 89.94044629403746, 89.97632202052091, 89.997847456411, 90.0982994905647, 90.13417521704815, 90.16287579823492, 90.25615268709191, 90.33507928535553, 90.31355384946545, 90.32072899476215, 90.34225443065222, 90.37095501183899, 90.38530530243239, 90.39248044772907, 90.44988161010261, 90.52163306306953, 90.52163306306953, 90.49293248188276, 90.49293248188276, 90.50010762717945, 90.52880820836621, 90.42835617421254, 90.44270646480591, 90.44988161010261, 90.44270646480591, 90.38530530243239, 90.46423190069599, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.61993255363421, 81.07196670732583, 81.43789911745714, 81.80383152758843, 82.24869053598336, 82.54287149314774, 82.90880390327905, 83.2173351510368, 83.5043409629045, 83.84874793714573, 84.0711774413432, 84.29360694554066, 84.60213819329842, 84.8030422616058, 85.00394632991318, 85.2766018511875, 85.57795795364856, 85.83626318432948, 86.0084666714501, 86.17349501327402, 86.41744995336155, 86.63270431226233, 86.86948410705317, 87.09191361125063, 87.24259166248117, 87.41479514960179, 87.63004950850255, 87.84530386740332, 87.98880677333716, 88.09643395278755, 88.19688598694124, 88.33321374757838, 88.54846810647916, 88.66327043122624, 88.83547391834685, 88.90005022601709, 89.00050226017078, 89.07942885843438, 89.2659826361484, 89.31620865322523, 89.48841214034584, 89.56733873860946, 89.60321446509292, 89.67496591805984, 89.78259309751023, 89.94044629403746, 89.98349716581761, 90.0982994905647, 90.14135036234484, 90.23462725120184, 90.30637870416876, 90.33507928535553, 90.33507928535553, 90.34942957594892, 90.37095501183899, 90.3566047212456, 90.42118102891584, 90.52163306306953, 90.51445791777284, 90.5503336442563, 90.56468393484968, 90.57903422544307, 90.55750878955298, 90.60773480662984, 90.60055966133315, 90.68666140489346, 90.70101169548683, 90.71536198608023, 90.744062567267, 90.77993829375045, 90.76558800315708, 90.75841285786038, 90.7297122766736, 90.70101169548683, 90.67231111430006, 90.67231111430006, 90.62208509722322, 90.62926024251992, 90.64361053311329, 90.67231111430006, 90.69383655019014, 90.67948625959676, 90.73688742197031, 90.69383655019014, 90.67948625959676, 90.67231111430006, 90.67231111430006, 90.61490995192652, 90.57903422544307, 90.58620937073975, 90.58620937073975, 90.55750878955298, 90.50010762717945, 90.47140704599268, 90.40683073832245, 90.33507928535553, 90.30637870416876, 90.28485326827868, 90.19157637942169, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.49077993829376, 80.95716438257875, 81.40202339097367, 81.75360551051159, 82.16258879242305, 82.56439692903783, 82.81552701442205, 83.14558369806988, 83.43258950993759, 83.74112075769534, 84.02812656956303, 84.41558441558442, 84.65953935567195, 84.96807060342972, 85.19767525292387, 85.43445504771472, 85.67840998780225, 85.8219128937361, 85.94389036377987, 86.19502044916409, 86.44615053454832, 86.61835402166894, 86.89818468823994, 87.05603788476716, 87.25694195307454, 87.4004448590084, 87.63004950850255, 87.83812872210662, 87.93858075626032, 88.06055822630408, 88.23276171342471, 88.3260386022817, 88.505417234699, 88.64892014063285, 88.7924230465667, 88.92157566190716, 88.96462653368731, 89.07942885843438, 89.19423118318146, 89.32338379852193, 89.44536126856569, 89.52428786682931, 89.56733873860946, 89.64626533687307, 89.76824280691685, 89.8902202769606, 89.997847456411, 90.09112434526799, 90.1700509435316, 90.1772260888283, 90.23462725120184, 90.27050297768531, 90.32072899476215, 90.38530530243239, 90.37095501183899, 90.37813015713569, 90.42835617421254, 90.47140704599268, 90.52880820836621, 90.56468393484968, 90.57185908014638, 90.58620937073975, 90.56468393484968, 90.58620937073975, 90.68666140489346, 90.66513596900337, 90.67231111430006, 90.67231111430006, 90.70818684078354, 90.7297122766736, 90.75841285786038, 90.77276314845376, 90.73688742197031, 90.73688742197031, 90.73688742197031, 90.71536198608023, 90.68666140489346, 90.65796082370669, 90.64361053311329, 90.64361053311329, 90.6364353878166, 90.6364353878166, 90.65078567840999, 90.65796082370669, 90.67231111430006, 90.65078567840999, 90.60773480662984, 90.65078567840999, 90.61490995192652, 90.57903422544307, 90.55750878955298, 90.5503336442563, 90.49293248188276, 90.46423190069599, 90.39248044772907, 90.32790414005885, 90.28485326827868, 90.29202841357538, 90.19157637942169, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.49077993829376, 80.95716438257875, 81.40202339097367, 81.75360551051159, 82.16258879242305, 82.56439692903783, 82.81552701442205, 83.14558369806988, 83.43258950993759, 83.74112075769534, 84.02812656956303, 84.41558441558442, 84.65953935567195, 84.96807060342972, 85.19767525292387, 85.43445504771472, 85.67840998780225, 85.8219128937361, 85.94389036377987, 86.19502044916409, 86.44615053454832, 86.61835402166894, 86.89818468823994, 87.05603788476716, 87.25694195307454, 87.4004448590084, 87.63004950850255, 87.83812872210662, 87.93858075626032, 88.06055822630408, 88.23276171342471, 88.3260386022817, 88.505417234699, 88.64892014063285, 88.7924230465667, 88.92157566190716, 88.96462653368731, 89.07942885843438, 89.19423118318146, 89.32338379852193, 89.44536126856569, 89.52428786682931, 89.56733873860946, 89.64626533687307, 89.76824280691685, 89.8902202769606, 89.997847456411, 90.09112434526799, 90.1700509435316, 90.1772260888283, 90.23462725120184, 90.27050297768531, 90.32072899476215, 90.38530530243239, 90.37095501183899, 90.37813015713569, 90.42835617421254, 90.47140704599268, 90.52880820836621, 90.56468393484968, 90.57185908014638, 90.58620937073975, 90.56468393484968, 90.58620937073975, 90.68666140489346, 90.66513596900337, 90.67231111430006, 90.67231111430006, 90.70818684078354, 90.7297122766736, 90.75841285786038, 90.77276314845376, 90.73688742197031, 90.73688742197031, 90.73688742197031, 90.71536198608023, 90.68666140489346, 90.65796082370669, 90.64361053311329, 90.64361053311329, 90.6364353878166, 90.6364353878166, 90.65078567840999, 90.65796082370669, 90.67231111430006, 90.65078567840999, 90.60773480662984, 90.65078567840999, 90.61490995192652, 90.57903422544307, 90.55750878955298, 90.5503336442563, 90.49293248188276, 90.46423190069599, 90.39248044772907, 90.32790414005885, 90.28485326827868, 90.29202841357538, 90.19157637942169, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2027, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.6342828442276, 81.1006672885126, 81.50247542512736, 81.88275812585205, 82.24869053598336, 82.6720241084882, 82.95902992035589, 83.27473631341034, 83.64066872354165, 83.98507569778289, 84.22903063787042, 84.51603644973811, 84.79586711630911, 85.00394632991318, 85.20485039822057, 85.37705388534118, 85.69276027839564, 85.9080146372964, 86.0156418167468, 86.26677190213103, 86.45332567984502, 86.63987945755902, 86.76903207289948, 86.81925808997633, 86.86948410705317, 86.95558585061349, 87.01298701298703, 87.0416875941738, 87.1277893377341, 87.17084020951424, 87.19236564540432, 87.22824137188778, 87.29999282485471, 87.35739398722825, 87.41479514960179, 87.4937217478654, 87.5869986367224, 87.61569921790917, 87.63004950850255, 87.6731003802827, 87.73767668795294, 87.78072755973308, 87.84530386740332, 87.90270502977687, 87.93858075626032, 88.03185764511731, 88.07490851689748, 88.12513453397432, 88.19688598694124, 88.22558656812801, 88.31886345698501, 88.39779005524862, 88.45519121762216, 88.49824208940231, 88.52694267058908, 88.63456985003947, 88.691971012413, 88.74937217478654, 88.81394848245677, 88.84264906364353, 88.94310109779724, 89.00050226017078, 89.05072827724761, 89.12247973021454, 89.15118031140132, 89.22293176436823, 89.2659826361484, 89.35925952500538, 89.39513525148884, 89.48123699504916, 89.50276243093923, 89.56733873860946, 89.61038961038962, 89.68931620865322, 89.72519193513669, 89.73236708043338, 89.73236708043338, 89.78976824280693, 89.83999425988375, 89.87586998636723, 89.99067231111431, 90.06242376408122, 90.13417521704815, 90.1772260888283, 90.24180239649853, 90.34225443065222, 90.40683073832245, 90.47140704599268, 90.51445791777284, 90.57903422544307, 90.60055966133315, 90.56468393484968, 90.60055966133315, 90.60773480662984, 90.60773480662984, 90.57903422544307, 90.50010762717945, 90.48575733658608, 90.43553131950922, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2027, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54818110066729, 81.02174069024898, 81.48812513453397, 81.9329841429289, 82.3132668436536, 82.63614838200473, 82.99490564683936, 83.31778718519051, 83.66219415943173, 83.99942598837626, 84.30078209083734, 84.44428499677119, 84.68823993685872, 85.03982205639664, 85.25507641529741, 85.46315562890149, 85.70711056898904, 85.9151897825931, 86.1017435603071, 86.22372103035086, 86.31699791920788, 86.37439908158142, 86.51790198751526, 86.61835402166894, 86.69010547463587, 86.76185692760279, 86.82643323527302, 86.9340604147234, 86.97711128650356, 87.02016215828371, 87.0416875941738, 87.09191361125063, 87.17084020951424, 87.2138910812944, 87.25694195307454, 87.32151826074478, 87.37174427782163, 87.37891942311832, 87.43632058549186, 87.50089689316209, 87.55112291023893, 87.55829805553563, 87.62287436320587, 87.73050154265624, 87.75920212384301, 87.89552988448015, 88.01033220922724, 88.12513453397432, 88.1394848245677, 88.23276171342471, 88.28298773050155, 88.33321374757838, 88.412140345842, 88.44084092702877, 88.49824208940231, 88.62021955944608, 88.67762072181962, 88.77089761067663, 88.82829877305016, 88.90722537131377, 88.95027624309392, 89.00767740546746, 89.02202769606085, 89.10095429432447, 89.15118031140132, 89.2588074908517, 89.23010690966493, 89.28033292674176, 89.35925952500538, 89.41666068737892, 89.50276243093923, 89.59603931979623, 89.64626533687307, 89.66779077276315, 89.8041185334003, 89.8902202769606, 89.93327114874077, 89.95479658463084, 90.01219774700438, 90.05524861878453, 90.14852550764154, 90.1772260888283, 90.20592667001507, 90.19157637942169, 90.16287579823492, 90.19875152471838, 90.24180239649853, 90.3637798665423, 90.3637798665423, 90.40683073832245, 90.41400588361914, 90.40683073832245, 90.44270646480591, 90.49293248188276, 90.48575733658608, 90.47858219128938, 90.51445791777284, 90.50010762717945, 90.47858219128938, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.57688168185406, 81.05761641673244, 81.44507426275382, 81.9186338523355, 82.26304082657674, 82.64332352730143, 82.98055535624597, 83.34648776637728, 83.63349357824497, 83.95637511659612, 84.1429288943101, 84.40840927028772, 84.7097653727488, 85.03264691109995, 85.19767525292387, 85.39140417593457, 85.57795795364856, 85.77886202195594, 86.03716725263688, 86.19502044916409, 86.35287364569133, 86.51072684221856, 86.76903207289948, 86.99863672239363, 87.10626390184402, 87.29999282485471, 87.48654660256871, 87.6802755255794, 87.85247901270002, 88.01750735452394, 88.16101026045777, 88.22558656812801, 88.34756403817178, 88.44801607232546, 88.58434383296262, 88.72784673889646, 88.80677333716008, 88.92157566190716, 89.02202769606085, 89.10812943962115, 89.17270574729139, 89.17270574729139, 89.251632345555, 89.38078496089547, 89.51711272153261, 89.59603931979623, 89.66061562746646, 89.73954222573008, 89.83999425988375, 89.86869484107054, 89.98349716581761, 90.01219774700438, 90.0982994905647, 90.11982492645475, 90.10547463586138, 90.14852550764154, 90.22745210590514, 90.3566047212456, 90.42835617421254, 90.46423190069599, 90.48575733658608, 90.52163306306953, 90.55750878955298, 90.57903422544307, 90.59338451603645, 90.62208509722322, 90.64361053311329, 90.67231111430006, 90.70818684078354, 90.76558800315708, 90.84451460142068, 90.82298916553061, 90.80146372964053, 90.82298916553061, 90.80146372964053, 90.77993829375045, 90.79428858434385, 90.77993829375045, 90.76558800315708, 90.76558800315708, 90.744062567267, 90.77276314845376, 90.744062567267, 90.66513596900337, 90.57185908014638, 90.54315849895961, 90.5503336442563, 90.55750878955298, 90.53598335366291, 90.52163306306953, 90.47140704599268, 90.43553131950922, 90.37813015713569, 90.31355384946545, 90.29920355887207, 90.29202841357538, 90.23462725120184, 90.21310181531177, 90.14852550764154, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.6342828442276, 81.12219272440268, 81.53117600631413, 81.87558298055536, 82.23434024538997, 82.72225012556504, 83.05948195450958, 83.44693980053096, 83.75547104828873, 84.02095142426634, 84.26490636435389, 84.46581043266126, 84.71694051804549, 85.01829662050658, 85.1689746717371, 85.3698787400445, 85.6353591160221, 85.83626318432948, 86.0156418167468, 86.30982277391116, 86.5609528592954, 86.70445576522926, 86.94123556002009, 87.07038817536056, 87.23541651718448, 87.3143431154481, 87.34304369663487, 87.42197029489847, 87.47937145727201, 87.52242232905216, 87.5798234914257, 87.65157494439264, 87.70180096146947, 87.73050154265624, 87.75202697854631, 87.75202697854631, 87.80225299562316, 87.83812872210662, 87.88835473918347, 87.92423046566694, 87.97445648274378, 88.06773337160078, 88.10360909808423, 88.17536055105116, 88.28298773050155, 88.33321374757838, 88.35473918346847, 88.40496520054532, 88.46954150821554, 88.51259237999571, 88.62739470474278, 88.72784673889646, 88.79959819186338, 88.83547391834685, 88.89287508072039, 88.92875080720385, 88.93592595250054, 89.01485255076416, 89.03637798665423, 89.065078567841, 89.13683002080792, 89.21575661907154, 89.30903350792853, 89.35925952500538, 89.39513525148884, 89.48841214034584, 89.56016359331277, 89.66061562746646, 89.71801678984, 89.76824280691685, 89.78259309751023, 89.85434455047715, 89.8902202769606, 89.98349716581761, 90.04807347348785, 90.09112434526799, 90.13417521704815, 90.21310181531177, 90.22745210590514, 90.24897754179523, 90.30637870416876, 90.32790414005885, 90.34942957594892, 90.44270646480591, 90.41400588361914, 90.38530530243239, 90.46423190069599, 90.50728277247615, 90.57185908014638, 90.62926024251992, 90.62926024251992, 90.60773480662984, 90.60773480662984, 90.64361053311329, 90.55750878955298, 90.58620937073975, 90.57903422544307, 90.52880820836621, 90.39248044772907, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.58405682715075, 81.07914185262251, 81.50965057042406, 81.90428356174213, 82.21998995479659, 82.63614838200473, 82.93750448446582, 83.3249623304872, 83.57609241587143, 83.8774485183325, 84.16445433020019, 84.40123412499103, 84.68106479156204, 84.93937002224295, 85.13309894525365, 85.37705388534118, 85.62818397072542, 85.88648920140633, 86.08739326971371, 86.23089617564756, 86.43897538925164, 86.72598120111932, 86.87665925234987, 87.06321303006386, 87.26411709837124, 87.41479514960179, 87.60852407261247, 87.82377843151325, 87.9529310468537, 88.06055822630408, 88.10360909808423, 88.31168831168831, 88.47671665351223, 88.64174499533615, 88.82829877305016, 88.99332711487408, 89.10095429432447, 89.2659826361484, 89.38796010619215, 89.47406184975246, 89.59603931979623, 89.63909019157639, 89.71084164454331, 89.71801678984, 89.86151969577384, 89.97632202052091, 90.01937289230108, 90.11982492645475, 90.20592667001507, 90.27050297768531, 90.31355384946545, 90.34942957594892, 90.39248044772907, 90.44270646480591, 90.47858219128938, 90.50010762717945, 90.49293248188276, 90.49293248188276, 90.43553131950922, 90.45705675539931, 90.47140704599268, 90.45705675539931, 90.48575733658608, 90.45705675539931, 90.50010762717945, 90.55750878955298, 90.57903422544307, 90.62926024251992, 90.64361053311329, 90.65078567840999, 90.65078567840999, 90.61490995192652, 90.65796082370669, 90.65078567840999, 90.66513596900337, 90.64361053311329, 90.6364353878166, 90.62208509722322, 90.67231111430006, 90.67948625959676, 90.70101169548683, 90.75123771256368, 90.73688742197031, 90.68666140489346, 90.65796082370669, 90.67231111430006, 90.62926024251992, 90.60773480662984, 90.57185908014638, 90.53598335366291, 90.51445791777284, 90.52880820836621, 90.51445791777284, 90.44988161010261, 90.42835617421254, 90.39965559302576, 90.34942957594892, 90.29920355887207, 90.19157637942169, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56970653655738, 80.98586496376552, 81.40202339097367, 81.86123268996197, 82.24869053598336, 82.60027265552128, 82.91597904857574, 83.27473631341034, 83.6047929970582, 83.94919997129942, 84.28643180024396, 84.5232115950348, 84.72411566334218, 84.91784458635287, 85.18332496233049, 85.4488053383081, 85.67123484250556, 85.9080146372964, 86.11609385090048, 86.23807132094426, 86.48920140632848, 86.71163091052594, 86.8479586711631, 87.05603788476716, 87.25694195307454, 87.41479514960179, 87.56547320083232, 87.82377843151325, 87.98880677333716, 88.0533830810074, 88.15383511516109, 88.31168831168831, 88.47671665351223, 88.66327043122624, 88.7924230465667, 89.00767740546746, 89.09377914902777, 89.18705603788477, 89.344909234412, 89.44536126856569, 89.5386381574227, 89.66061562746646, 89.76106766162016, 89.75389251632346, 89.88304513166392, 89.94044629403746, 90.04089832819116, 90.09112434526799, 90.23462725120184, 90.26332783238861, 90.31355384946545, 90.34942957594892, 90.41400588361914, 90.46423190069599, 90.53598335366291, 90.5503336442563, 90.55750878955298, 90.54315849895961, 90.52163306306953, 90.49293248188276, 90.46423190069599, 90.48575733658608, 90.50010762717945, 90.53598335366291, 90.53598335366291, 90.54315849895961, 90.54315849895961, 90.58620937073975, 90.60773480662984, 90.61490995192652, 90.65078567840999, 90.66513596900337, 90.67231111430006, 90.68666140489346, 90.68666140489346, 90.67231111430006, 90.72253713137691, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.69383655019014, 90.71536198608023, 90.71536198608023, 90.69383655019014, 90.66513596900337, 90.62926024251992, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.53598335366291, 90.50728277247615, 90.49293248188276, 90.48575733658608, 90.42118102891584, 90.42118102891584, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.19875152471838, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56970653655738, 80.98586496376552, 81.40202339097367, 81.86123268996197, 82.24869053598336, 82.60027265552128, 82.91597904857574, 83.27473631341034, 83.6047929970582, 83.94919997129942, 84.28643180024396, 84.5232115950348, 84.72411566334218, 84.91784458635287, 85.18332496233049, 85.4488053383081, 85.67123484250556, 85.9080146372964, 86.11609385090048, 86.23807132094426, 86.48920140632848, 86.71163091052594, 86.8479586711631, 87.05603788476716, 87.25694195307454, 87.41479514960179, 87.56547320083232, 87.82377843151325, 87.98880677333716, 88.0533830810074, 88.15383511516109, 88.31168831168831, 88.47671665351223, 88.66327043122624, 88.7924230465667, 89.00767740546746, 89.09377914902777, 89.18705603788477, 89.344909234412, 89.44536126856569, 89.5386381574227, 89.66061562746646, 89.76106766162016, 89.75389251632346, 89.88304513166392, 89.94044629403746, 90.04089832819116, 90.09112434526799, 90.23462725120184, 90.26332783238861, 90.31355384946545, 90.34942957594892, 90.41400588361914, 90.46423190069599, 90.53598335366291, 90.5503336442563, 90.55750878955298, 90.54315849895961, 90.52163306306953, 90.49293248188276, 90.46423190069599, 90.48575733658608, 90.50010762717945, 90.53598335366291, 90.53598335366291, 90.54315849895961, 90.54315849895961, 90.58620937073975, 90.60773480662984, 90.61490995192652, 90.65078567840999, 90.66513596900337, 90.67231111430006, 90.68666140489346, 90.68666140489346, 90.67231111430006, 90.72253713137691, 90.67948625959676, 90.69383655019014, 90.70818684078354, 90.69383655019014, 90.71536198608023, 90.71536198608023, 90.69383655019014, 90.66513596900337, 90.62926024251992, 90.62208509722322, 90.58620937073975, 90.5503336442563, 90.53598335366291, 90.50728277247615, 90.49293248188276, 90.48575733658608, 90.42118102891584, 90.42118102891584, 90.39248044772907, 90.3637798665423, 90.30637870416876, 90.19875152471838, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2028, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.49795508359044, 80.96433952787544, 81.27287077563321, 81.68902920284135, 82.07648704886275, 82.37784315132382, 82.75095070675181, 83.09535768099305, 83.45411494582765, 83.82722250125565, 84.13575374901342, 84.38688383439765, 84.7097653727488, 84.96807060342972, 85.22637583411064, 85.48468106479157, 85.59230824424195, 85.80756260314271, 86.08739326971371, 86.2882973380211, 86.47485111573509, 86.59682858577887, 86.81925808997633, 87.04886273947048, 87.17801535481094, 87.35739398722825, 87.52959747434886, 87.68745067087609, 87.82377843151325, 87.98163162804047, 88.10360909808423, 88.11078424338093, 88.14665996986439, 88.13230967927102, 88.14665996986439, 88.14665996986439, 88.18971084164454, 88.19688598694124, 88.24711200401808, 88.28298773050155, 88.29016287579825, 88.29733802109493, 88.33321374757838, 88.36908947406185, 88.40496520054532, 88.46236636291886, 88.5197675252924, 88.54846810647916, 88.6058692688527, 88.70632130300639, 88.75654732008323, 88.81394848245677, 88.90722537131377, 88.99332711487408, 89.00767740546746, 89.0722537131377, 89.1440051661046, 89.20858147377486, 89.31620865322523, 89.38796010619215, 89.46688670445577, 89.56016359331277, 89.624739900983, 89.67496591805984, 89.71084164454331, 89.79694338810361, 89.86151969577384, 89.94044629403746, 89.98349716581761, 90.02654803759776, 90.06242376408122, 90.13417521704815, 90.1700509435316, 90.29202841357538, 90.40683073832245, 90.42835617421254, 90.47140704599268, 90.47858219128938, 90.51445791777284, 90.58620937073975, 90.65796082370669, 90.69383655019014, 90.75123771256368, 90.77276314845376, 90.80863887493722, 90.82298916553061, 90.83733945612398, 90.90191576379422, 90.94496663557437, 90.93061634498099, 90.94496663557437, 90.89474061849752, 90.81581402023392, 90.85886489201407, 90.83733945612398, 90.72253713137691, 90.70818684078354, 90.60055966133315, 90.45705675539931, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2028, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.58405682715075, 81.11501757910598, 81.48094998923729, 81.9186338523355, 82.32044198895028, 82.61462294611465, 82.9518547750592, 83.29626174930043, 83.57609241587143, 83.8774485183325, 84.25055607376049, 84.48016072325466, 84.73846595393557, 84.91066944105619, 85.19050010762719, 85.39857932123127, 85.50620650068164, 85.6497094066155, 85.69993542369232, 85.8290880390328, 85.87931405610964, 85.94389036377987, 86.0156418167468, 86.1017435603071, 86.15196957738395, 86.25242161153763, 86.31699791920788, 86.39592451747149, 86.4676759704384, 86.49637655162518, 86.58965344048217, 86.66140489344909, 86.74750663700941, 86.86230896175648, 86.94841070531677, 87.0345124488771, 87.07756332065725, 87.1206141924374, 87.2067159359977, 87.26411709837124, 87.35021884193155, 87.40762000430509, 87.47219631197532, 87.56547320083232, 87.665925234986, 87.7735524144364, 87.88117959388678, 87.92423046566694, 88.00315706393054, 88.0533830810074, 88.15383511516109, 88.28298773050155, 88.31886345698501, 88.37626461935855, 88.42649063643539, 88.43366578173209, 88.53411781588576, 88.59151897825932, 88.62739470474278, 88.59869412355602, 88.67044557652292, 88.72784673889646, 88.84982420894023, 88.91440051661046, 89.01485255076416, 89.09377914902777, 89.17270574729139, 89.251632345555, 89.29468321733516, 89.31620865322523, 89.36643467030207, 89.4310109779723, 89.54581330271938, 89.66061562746646, 89.70366649924661, 89.72519193513669, 89.8041185334003, 89.87586998636723, 89.91174571285069, 90.01937289230108, 90.04089832819116, 90.07677405467462, 90.15570065293824, 90.1700509435316, 90.15570065293824, 90.25615268709191, 90.26332783238861, 90.28485326827868, 90.29202841357538, 90.28485326827868, 90.3566047212456, 90.34942957594892, 90.41400588361914, 90.47858219128938, 90.48575733658608, 90.43553131950922, 90.48575733658608, 90.49293248188276, 90.32790414005885, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.59840711774415, 80.9930401090622, 81.39484824567698, 81.74643036521489, 82.08366219415943, 82.49264547607089, 82.85140274090551, 83.14558369806988, 83.4110640740475, 83.74112075769534, 84.02095142426634, 84.29360694554066, 84.5303867403315, 84.82456769749588, 85.13309894525365, 85.40575446652795, 85.6497094066155, 85.80038745784603, 86.06586783382365, 86.26677190213103, 86.47485111573509, 86.67575518404249, 86.77620721819616, 86.99146157709694, 87.15648991892087, 87.43632058549186, 87.665925234986, 87.78790270502978, 87.98163162804047, 88.17536055105116, 88.3260386022817, 88.42649063643539, 88.52694267058908, 88.64892014063285, 88.72067159359977, 88.84982420894023, 88.96462653368731, 89.12247973021454, 89.22293176436823, 89.3520843797087, 89.50276243093923, 89.60321446509292, 89.63191504627969, 89.68214106335654, 89.79694338810361, 89.85434455047715, 89.92609600344407, 90.05524861878453, 90.11982492645475, 90.21310181531177, 90.25615268709191, 90.32072899476215, 90.32072899476215, 90.34942957594892, 90.40683073832245, 90.41400588361914, 90.43553131950922, 90.43553131950922, 90.44270646480591, 90.44988161010261, 90.47140704599268, 90.45705675539931, 90.49293248188276, 90.51445791777284, 90.54315849895961, 90.56468393484968, 90.59338451603645, 90.60773480662984, 90.59338451603645, 90.57185908014638, 90.52163306306953, 90.53598335366291, 90.51445791777284, 90.57185908014638, 90.58620937073975, 90.61490995192652, 90.61490995192652, 90.60055966133315, 90.60055966133315, 90.62926024251992, 90.6364353878166, 90.6364353878166, 90.61490995192652, 90.61490995192652, 90.58620937073975, 90.58620937073975, 90.50010762717945, 90.47858219128938, 90.44988161010261, 90.39965559302576, 90.38530530243239, 90.39248044772907, 90.3637798665423, 90.29202841357538, 90.23462725120184, 90.19875152471838, 90.14852550764154, 90.09112434526799, 90.0839491999713, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.51230537418384, 81.02891583554567, 81.54552629690751, 81.86123268996197, 82.1984645189065, 82.50699576666428, 82.90162875798235, 83.23886058692689, 83.53304154409128, 83.87027337303581, 84.22185549257372, 84.48016072325466, 84.65953935567195, 84.88196885986942, 85.22637583411064, 85.47750591949487, 85.72863600487909, 85.89366434670302, 86.20219559446079, 86.38874937217479, 86.5681280045921, 86.80490779938295, 87.01298701298703, 87.2138910812944, 87.34304369663487, 87.45784602138194, 87.55829805553563, 87.70897610676617, 87.88117959388678, 87.9601061921504, 88.0462079357107, 88.0533830810074, 88.13230967927102, 88.22558656812801, 88.28298773050155, 88.33321374757838, 88.38343976465524, 88.40496520054532, 88.43366578173209, 88.46954150821554, 88.5197675252924, 88.55564325177585, 88.6058692688527, 88.59869412355602, 88.61304441414939, 88.64892014063285, 88.68479586711632, 88.72067159359977, 88.77089761067663, 88.82829877305016, 88.92875080720385, 89.04355313195093, 89.07942885843438, 89.1440051661046, 89.23728205496163, 89.27315778144508, 89.37360981559877, 89.45971155915908, 89.52428786682931, 89.61038961038962, 89.6175647556863, 89.69649135394992, 89.76106766162016, 89.82564396929038, 89.91174571285069, 89.91892085814737, 89.97632202052091, 90.09112434526799, 90.16287579823492, 90.1700509435316, 90.18440123412499, 90.21310181531177, 90.23462725120184, 90.27050297768531, 90.29920355887207, 90.32790414005885, 90.32790414005885, 90.37095501183899, 90.39965559302576, 90.48575733658608, 90.59338451603645, 90.67948625959676, 90.71536198608023, 90.75123771256368, 90.71536198608023, 90.79428858434385, 90.76558800315708, 90.71536198608023, 90.77276314845376, 90.72253713137691, 90.81581402023392, 90.81581402023392, 90.80863887493722, 90.76558800315708, 90.69383655019014, 90.68666140489346, 90.64361053311329, 90.6364353878166, 90.52163306306953, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.53383081007391, 81.02891583554567, 81.4665996986439, 81.78948123699506, 82.07648704886275, 82.34914257013705, 82.62897323670805, 82.97338021094927, 83.31061203989381, 83.57609241587143, 83.85592308244242, 84.09987802252996, 84.37253354380427, 84.58778790270503, 84.76716653512234, 84.99677118461649, 85.1761498170338, 85.42727990241804, 85.61383368013202, 85.80038745784603, 85.95824065437326, 86.16631986797732, 86.31699791920788, 86.50355169692186, 86.67575518404249, 86.86948410705317, 87.09908875654733, 87.22106622659109, 87.42197029489847, 87.5941737820191, 87.70897610676617, 87.91705532037024, 88.11795938867762, 88.21841142283131, 88.37626461935855, 88.49106694410563, 88.63456985003947, 88.77807275597331, 88.85699935423692, 88.98615196957739, 89.12247973021454, 89.16553060199469, 89.28750807203846, 89.38078496089547, 89.44536126856569, 89.45971155915908, 89.56016359331277, 89.66779077276315, 89.76106766162016, 89.83999425988375, 89.8902202769606, 89.96914687522423, 90.04807347348785, 90.15570065293824, 90.23462725120184, 90.34225443065222, 90.41400588361914, 90.50010762717945, 90.62208509722322, 90.65796082370669, 90.67948625959676, 90.744062567267, 90.76558800315708, 90.76558800315708, 90.80146372964053, 90.92344119968429, 90.98801750735453, 90.95931692616776, 90.98801750735453, 90.96649207146446, 90.95931692616776, 90.88039032790415, 90.87321518260745, 90.92344119968429, 90.93061634498099, 90.90191576379422, 90.90191576379422, 90.88039032790415, 90.81581402023392, 90.77993829375045, 90.82298916553061, 90.75841285786038, 90.77993829375045, 90.744062567267, 90.71536198608023, 90.73688742197031, 90.73688742197031, 90.71536198608023, 90.71536198608023, 90.65796082370669, 90.6364353878166, 90.54315849895961, 90.47140704599268, 90.42118102891584, 90.38530530243239, 90.34942957594892, 90.25615268709191, 90.15570065293824, 90.12700007175145, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.36162732295328, 80.74191002367799, 81.15806845088613, 81.50965057042406, 81.81100667288513, 82.14106335653297, 82.44959460429074, 82.75812585204851, 82.98773050154266, 83.2245102963335, 83.48281552701442, 83.74112075769534, 84.0568271507498, 84.27925665494726, 84.55908732151826, 84.85326827868265, 85.09722321877018, 85.30530243237426, 85.52773193657171, 85.75016144076918, 85.9151897825931, 86.13044414149387, 86.30264762861448, 86.51072684221856, 86.68293032933917, 86.85513381645978, 87.0416875941738, 87.24259166248117, 87.40762000430509, 87.57264834612901, 87.73050154265624, 87.945755901557, 88.12513453397432, 88.21123627753462, 88.36908947406185, 88.49106694410563, 88.64174499533615, 88.81394848245677, 88.878524790127, 89.00050226017078, 89.10812943962115, 89.17988089258809, 89.251632345555, 89.344909234412, 89.45971155915908, 89.46688670445577, 89.6175647556863, 89.71084164454331, 89.79694338810361, 89.91174571285069, 89.94762143933414, 90.01937289230108, 90.04807347348785, 90.16287579823492, 90.23462725120184, 90.33507928535553, 90.43553131950922, 90.53598335366291, 90.55750878955298, 90.6364353878166, 90.71536198608023, 90.75123771256368, 90.75123771256368, 90.77993829375045, 90.82298916553061, 90.90191576379422, 90.97366721676114, 91.0095429432446, 90.98801750735453, 90.97366721676114, 90.93779149027769, 90.89474061849752, 90.88756547320084, 90.90191576379422, 90.90191576379422, 90.89474061849752, 90.89474061849752, 90.86604003731075, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.78711343904715, 90.75123771256368, 90.73688742197031, 90.70818684078354, 90.72253713137691, 90.75123771256368, 90.7297122766736, 90.70101169548683, 90.65078567840999, 90.62208509722322, 90.51445791777284, 90.48575733658608, 90.42118102891584, 90.38530530243239, 90.34225443065222, 90.25615268709191, 90.14852550764154, 90.12700007175145, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.36162732295328, 80.74191002367799, 81.15806845088613, 81.50965057042406, 81.81100667288513, 82.14106335653297, 82.44959460429074, 82.75812585204851, 82.98773050154266, 83.2245102963335, 83.48281552701442, 83.74112075769534, 84.0568271507498, 84.27925665494726, 84.55908732151826, 84.85326827868265, 85.09722321877018, 85.30530243237426, 85.52773193657171, 85.75016144076918, 85.9151897825931, 86.13044414149387, 86.30264762861448, 86.51072684221856, 86.68293032933917, 86.85513381645978, 87.0416875941738, 87.24259166248117, 87.40762000430509, 87.57264834612901, 87.73050154265624, 87.945755901557, 88.12513453397432, 88.21123627753462, 88.36908947406185, 88.49106694410563, 88.64174499533615, 88.81394848245677, 88.878524790127, 89.00050226017078, 89.10812943962115, 89.17988089258809, 89.251632345555, 89.344909234412, 89.45971155915908, 89.46688670445577, 89.6175647556863, 89.71084164454331, 89.79694338810361, 89.91174571285069, 89.94762143933414, 90.01937289230108, 90.04807347348785, 90.16287579823492, 90.23462725120184, 90.33507928535553, 90.43553131950922, 90.53598335366291, 90.55750878955298, 90.6364353878166, 90.71536198608023, 90.75123771256368, 90.75123771256368, 90.77993829375045, 90.82298916553061, 90.90191576379422, 90.97366721676114, 91.0095429432446, 90.98801750735453, 90.97366721676114, 90.93779149027769, 90.89474061849752, 90.88756547320084, 90.90191576379422, 90.90191576379422, 90.89474061849752, 90.89474061849752, 90.86604003731075, 90.80146372964053, 90.80146372964053, 90.81581402023392, 90.78711343904715, 90.75123771256368, 90.73688742197031, 90.70818684078354, 90.72253713137691, 90.75123771256368, 90.7297122766736, 90.70101169548683, 90.65078567840999, 90.62208509722322, 90.51445791777284, 90.48575733658608, 90.42118102891584, 90.38530530243239, 90.34225443065222, 90.25615268709191, 90.14852550764154, 90.12700007175145, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2029, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.57688168185406, 81.09349214321591, 81.52400086101744, 81.88993327114873, 82.22716510009327, 82.53569634785106, 82.96620506565259, 83.33213747578388, 83.6837195953218, 84.03530171485973, 84.37970868910097, 84.6164884838918, 84.86761856927603, 85.01829662050658, 85.22637583411064, 85.37705388534118, 85.53490708186841, 85.72146085958241, 85.93671521848317, 86.11609385090048, 86.30982277391116, 86.48202626103179, 86.63270431226233, 86.7546817823061, 86.91253497883332, 86.96993614120686, 87.06321303006386, 87.1277893377341, 87.2067159359977, 87.25694195307454, 87.27846738896463, 87.29999282485471, 87.33586855133817, 87.35021884193155, 87.36456913252493, 87.42197029489847, 87.50089689316209, 87.52959747434886, 87.61569921790917, 87.64439979909594, 87.6802755255794, 87.68745067087609, 87.74485183324963, 87.76637726913971, 87.80225299562316, 87.88835473918347, 87.945755901557, 88.03903279041401, 88.08208366219417, 88.11078424338093, 88.17536055105116, 88.23993685872139, 88.29016287579825, 88.35473918346847, 88.39779005524862, 88.48389179880894, 88.53411781588576, 88.58434383296262, 88.66327043122624, 88.7924230465667, 88.84264906364353, 88.87134964483032, 88.95745138839062, 89.05072827724761, 89.13683002080792, 89.20140632847816, 89.23728205496163, 89.29468321733516, 89.35925952500538, 89.35925952500538, 89.36643467030207, 89.4310109779723, 89.48841214034584, 89.56733873860946, 89.65344048216977, 89.72519193513669, 89.73236708043338, 89.8041185334003, 89.83999425988375, 89.8973954222573, 89.96914687522423, 90.02654803759776, 90.05524861878453, 90.0839491999713, 90.24897754179523, 90.3637798665423, 90.39965559302576, 90.55750878955298, 90.60055966133315, 90.62926024251992, 90.62926024251992, 90.65796082370669, 90.62208509722322, 90.60055966133315, 90.57185908014638, 90.5503336442563, 90.5503336442563, 90.57185908014638, 90.47140704599268, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2029, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56253139126068, 81.08631699791921, 81.45224940805052, 81.8325321087752, 82.18411422831313, 82.64332352730143, 82.92315419387243, 83.2173351510368, 83.61196814235488, 83.88462366362918, 84.19315491138696, 84.49451101384804, 84.73129080863887, 85.00394632991318, 85.23355097940734, 85.4416301930114, 85.58513309894525, 85.76451173136256, 85.95824065437326, 86.06586783382365, 86.20219559446079, 86.3815742268781, 86.52507713281194, 86.63270431226233, 86.70445576522926, 86.79055750878956, 86.91971012413002, 87.0273373035804, 87.1277893377341, 87.27846738896463, 87.34304369663487, 87.41479514960179, 87.47937145727201, 87.56547320083232, 87.61569921790917, 87.63004950850255, 87.6802755255794, 87.75202697854631, 87.86682930329339, 87.92423046566694, 88.03185764511731, 88.14665996986439, 88.29016287579825, 88.35473918346847, 88.36908947406185, 88.40496520054532, 88.412140345842, 88.54846810647916, 88.64892014063285, 88.74219702948984, 88.84264906364353, 88.93592595250054, 89.02202769606085, 89.11530458491785, 89.16553060199469, 89.251632345555, 89.3377340891153, 89.3520843797087, 89.4238358326756, 89.46688670445577, 89.55298844801608, 89.64626533687307, 89.70366649924661, 89.77541795221353, 89.83281911458707, 89.93327114874077, 89.997847456411, 90.03372318289445, 90.0982994905647, 90.1700509435316, 90.20592667001507, 90.22027696060846, 90.25615268709191, 90.26332783238861, 90.24897754179523, 90.34942957594892, 90.34225443065222, 90.39248044772907, 90.44988161010261, 90.51445791777284, 90.5503336442563, 90.52163306306953, 90.48575733658608, 90.52163306306953, 90.5503336442563, 90.56468393484968, 90.58620937073975, 90.59338451603645, 90.62208509722322, 90.57903422544307, 90.60055966133315, 90.60773480662984, 90.64361053311329, 90.65796082370669, 90.67948625959676, 90.68666140489346, 90.67231111430006, 90.65078567840999, 90.47140704599268, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.54100595537061, 81.03609098084237, 81.43789911745714, 81.76795580110497, 82.15541364712637, 82.43524431369735, 82.80835186912535, 83.06665709980628, 83.4110640740475, 83.66219415943173, 83.92767453540935, 84.20033005668365, 84.44428499677119, 84.63801391978187, 84.8030422616058, 85.04699720169334, 85.28377699648418, 85.5492573724618, 85.80038745784603, 86.02999210734018, 86.22372103035086, 86.43180024395495, 86.64705460285572, 86.89100954294324, 86.97711128650356, 87.18519050010764, 87.4004448590084, 87.55112291023893, 87.65157494439264, 87.86682930329339, 87.98880677333716, 88.17536055105116, 88.31886345698501, 88.46954150821554, 88.58434383296262, 88.67762072181962, 88.7924230465667, 88.89287508072039, 89.02202769606085, 89.10095429432447, 89.158355456698, 89.20140632847816, 89.36643467030207, 89.4310109779723, 89.50993757623593, 89.6175647556863, 89.73236708043338, 89.8041185334003, 89.8902202769606, 89.904570567554, 89.96197172992754, 90.06959890937792, 90.11982492645475, 90.13417521704815, 90.19875152471838, 90.24897754179523, 90.26332783238861, 90.34942957594892, 90.34225443065222, 90.37813015713569, 90.44270646480591, 90.50010762717945, 90.56468393484968, 90.59338451603645, 90.59338451603645, 90.67948625959676, 90.70818684078354, 90.68666140489346, 90.71536198608023, 90.73688742197031, 90.75841285786038, 90.77993829375045, 90.73688742197031, 90.744062567267, 90.78711343904715, 90.8301643108273, 90.82298916553061, 90.77276314845376, 90.76558800315708, 90.73688742197031, 90.68666140489346, 90.65078567840999, 90.62926024251992, 90.60055966133315, 90.62208509722322, 90.61490995192652, 90.59338451603645, 90.53598335366291, 90.53598335366291, 90.55750878955298, 90.49293248188276, 90.44988161010261, 90.37813015713569, 90.37095501183899, 90.32790414005885, 90.28485326827868, 90.26332783238861, 90.18440123412499, 90.0982994905647, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56253139126068, 81.05044127143574, 81.51682571572074, 81.96885986941236, 82.32044198895028, 82.6720241084882, 82.93750448446582, 83.36801320226735, 83.67654445002512, 84.01377627896966, 84.23620578316712, 84.5375618856282, 84.78869197101243, 85.04699720169334, 85.24790127000072, 85.47033077419817, 85.67840998780225, 85.89366434670302, 86.15196957738395, 86.33852335509793, 86.51790198751526, 86.69728061993256, 86.95558585061349, 87.08473846595393, 87.24259166248117, 87.3932697137117, 87.44349573078856, 87.47219631197532, 87.51524718375548, 87.50807203845878, 87.55829805553563, 87.5941737820191, 87.60134892731578, 87.65157494439264, 87.65875008968932, 87.665925234986, 87.68745067087609, 87.74485183324963, 87.78790270502978, 87.83095357680993, 87.88835473918347, 87.91705532037024, 87.99598191863386, 88.01033220922724, 88.07490851689748, 88.11795938867762, 88.18253569634786, 88.19688598694124, 88.26863743990816, 88.33321374757838, 88.412140345842, 88.49106694410563, 88.55564325177585, 88.62021955944608, 88.691971012413, 88.74937217478654, 88.81394848245677, 88.91440051661046, 88.98615196957739, 89.05790342254431, 89.12247973021454, 89.2588074908517, 89.28033292674176, 89.37360981559877, 89.4238358326756, 89.48841214034584, 89.56733873860946, 89.64626533687307, 89.69649135394992, 89.72519193513669, 89.83999425988375, 89.8902202769606, 89.8902202769606, 89.96914687522423, 90.04807347348785, 90.09112434526799, 90.14135036234484, 90.23462725120184, 90.22745210590514, 90.26332783238861, 90.34225443065222, 90.3637798665423, 90.37813015713569, 90.44988161010261, 90.55750878955298, 90.62926024251992, 90.69383655019014, 90.70101169548683, 90.83733945612398, 90.89474061849752, 90.93061634498099, 90.94496663557437, 90.95931692616776, 90.90909090909092, 90.8301643108273, 90.72253713137691, 90.62926024251992, 90.56468393484968, 90.51445791777284, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.53383081007391, 81.03609098084237, 81.42354882686375, 81.76795580110497, 82.16258879242305, 82.43524431369735, 82.80117672382866, 83.06665709980628, 83.46129009112435, 83.75547104828873, 84.1429288943101, 84.37970868910097, 84.6164884838918, 84.84609313338596, 85.07569778288011, 85.32682786826433, 85.57078280835188, 85.77168687665926, 85.97976609026333, 86.19502044916409, 86.45332567984502, 86.63270431226233, 86.8407835258664, 86.95558585061349, 87.14213962832747, 87.38609456841502, 87.51524718375548, 87.73050154265624, 87.8596541579967, 87.98163162804047, 88.13230967927102, 88.26146229461148, 88.37626461935855, 88.55564325177585, 88.69914615770969, 88.82829877305016, 88.95745138839062, 89.158355456698, 89.2659826361484, 89.3377340891153, 89.38078496089547, 89.51711272153261, 89.6175647556863, 89.70366649924661, 89.86869484107054, 89.91174571285069, 90.01937289230108, 90.06959890937792, 90.13417521704815, 90.13417521704815, 90.19157637942169, 90.24180239649853, 90.32790414005885, 90.42835617421254, 90.45705675539931, 90.50010762717945, 90.47140704599268, 90.52163306306953, 90.62926024251992, 90.70101169548683, 90.71536198608023, 90.75123771256368, 90.7297122766736, 90.65796082370669, 90.62208509722322, 90.64361053311329, 90.64361053311329, 90.60773480662984, 90.62926024251992, 90.61490995192652, 90.67231111430006, 90.62208509722322, 90.62208509722322, 90.61490995192652, 90.61490995192652, 90.55750878955298, 90.56468393484968, 90.54315849895961, 90.5503336442563, 90.55750878955298, 90.56468393484968, 90.54315849895961, 90.48575733658608, 90.50010762717945, 90.52880820836621, 90.49293248188276, 90.52163306306953, 90.47140704599268, 90.45705675539931, 90.42835617421254, 90.37095501183899, 90.3637798665423, 90.34225443065222, 90.31355384946545, 90.25615268709191, 90.24180239649853, 90.21310181531177, 90.18440123412499, 90.0982994905647, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56970653655738, 81.01456554495229, 81.38767310038028, 81.78230609169836, 82.1267130659396, 82.52852120255434, 82.90162875798235, 83.24603573222358, 83.56174212527804, 83.81287221066228, 84.12857860371673, 84.41558441558442, 84.63801391978187, 84.94654516753964, 85.14744923584703, 85.43445504771472, 85.65688455191219, 85.87213891081295, 86.11609385090048, 86.24524646624094, 86.47485111573509, 86.70445576522926, 86.82643323527302, 86.99146157709694, 87.14931477362417, 87.38609456841502, 87.53677261964555, 87.70897610676617, 87.91705532037024, 87.96728133744709, 88.08925880749085, 88.23276171342471, 88.40496520054532, 88.52694267058908, 88.64174499533615, 88.878524790127, 88.95027624309392, 89.10095429432447, 89.20858147377486, 89.33055894381862, 89.4310109779723, 89.47406184975246, 89.55298844801608, 89.68214106335654, 89.75389251632346, 89.86151969577384, 89.93327114874077, 90.00502260170768, 90.06242376408122, 90.12700007175145, 90.20592667001507, 90.28485326827868, 90.32790414005885, 90.39248044772907, 90.47140704599268, 90.54315849895961, 90.55750878955298, 90.59338451603645, 90.60773480662984, 90.60773480662984, 90.67231111430006, 90.744062567267, 90.77276314845376, 90.70101169548683, 90.744062567267, 90.67948625959676, 90.62926024251992, 90.65078567840999, 90.65796082370669, 90.66513596900337, 90.66513596900337, 90.67231111430006, 90.65078567840999, 90.62926024251992, 90.60773480662984, 90.58620937073975, 90.57185908014638, 90.53598335366291, 90.50728277247615, 90.50728277247615, 90.54315849895961, 90.5503336442563, 90.52163306306953, 90.54315849895961, 90.52163306306953, 90.52163306306953, 90.52880820836621, 90.50010762717945, 90.47140704599268, 90.39965559302576, 90.3637798665423, 90.38530530243239, 90.3566047212456, 90.30637870416876, 90.26332783238861, 90.24180239649853, 90.18440123412499, 90.1700509435316, 90.12700007175145, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56970653655738, 81.01456554495229, 81.38767310038028, 81.78230609169836, 82.1267130659396, 82.52852120255434, 82.90162875798235, 83.24603573222358, 83.56174212527804, 83.81287221066228, 84.12857860371673, 84.41558441558442, 84.63801391978187, 84.94654516753964, 85.14744923584703, 85.43445504771472, 85.65688455191219, 85.87213891081295, 86.11609385090048, 86.24524646624094, 86.47485111573509, 86.70445576522926, 86.82643323527302, 86.99146157709694, 87.14931477362417, 87.38609456841502, 87.53677261964555, 87.70897610676617, 87.91705532037024, 87.96728133744709, 88.08925880749085, 88.23276171342471, 88.40496520054532, 88.52694267058908, 88.64174499533615, 88.878524790127, 88.95027624309392, 89.10095429432447, 89.20858147377486, 89.33055894381862, 89.4310109779723, 89.47406184975246, 89.55298844801608, 89.68214106335654, 89.75389251632346, 89.86151969577384, 89.93327114874077, 90.00502260170768, 90.06242376408122, 90.12700007175145, 90.20592667001507, 90.28485326827868, 90.32790414005885, 90.39248044772907, 90.47140704599268, 90.54315849895961, 90.55750878955298, 90.59338451603645, 90.60773480662984, 90.60773480662984, 90.67231111430006, 90.744062567267, 90.77276314845376, 90.70101169548683, 90.744062567267, 90.67948625959676, 90.62926024251992, 90.65078567840999, 90.65796082370669, 90.66513596900337, 90.66513596900337, 90.67231111430006, 90.65078567840999, 90.62926024251992, 90.60773480662984, 90.58620937073975, 90.57185908014638, 90.53598335366291, 90.50728277247615, 90.50728277247615, 90.54315849895961, 90.5503336442563, 90.52163306306953, 90.54315849895961, 90.52163306306953, 90.52163306306953, 90.52880820836621, 90.50010762717945, 90.47140704599268, 90.39965559302576, 90.3637798665423, 90.38530530243239, 90.3566047212456, 90.30637870416876, 90.26332783238861, 90.24180239649853, 90.18440123412499, 90.1700509435316, 90.12700007175145, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2030, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.61993255363421, 81.16524359618282, 81.59575231398436, 81.96885986941236, 82.3060916983569, 82.5930975102246, 82.90880390327905, 83.26038602281697, 83.6047929970582, 83.83439764655235, 84.15010403960682, 84.38688383439765, 84.66671450096864, 84.99677118461649, 85.2694267058908, 85.4559804836048, 85.72863600487909, 85.94389036377987, 86.13761928679055, 86.3815742268781, 86.5681280045921, 86.73315634641602, 86.89100954294324, 87.08473846595393, 87.2138910812944, 87.3143431154481, 87.33586855133817, 87.3932697137117, 87.40762000430509, 87.45784602138194, 87.48654660256871, 87.50807203845878, 87.54394776494225, 87.60134892731578, 87.60134892731578, 87.63722465379924, 87.6802755255794, 87.72332639735954, 87.74485183324963, 87.82377843151325, 87.8596541579967, 87.87400444859009, 87.88117959388678, 87.9601061921504, 88.03185764511731, 88.06055822630408, 88.08208366219417, 88.1394848245677, 88.23276171342471, 88.29733802109493, 88.37626461935855, 88.42649063643539, 88.53411781588576, 88.64892014063285, 88.74219702948984, 88.77089761067663, 88.81394848245677, 88.90722537131377, 88.94310109779724, 88.99332711487408, 89.02202769606085, 89.05072827724761, 89.09377914902777, 89.1440051661046, 89.17988089258809, 89.24445720025831, 89.2588074908517, 89.37360981559877, 89.51711272153261, 89.55298844801608, 89.51711272153261, 89.61038961038962, 89.66779077276315, 89.71801678984, 89.78259309751023, 89.83999425988375, 89.93327114874077, 89.96197172992754, 90.03372318289445, 90.06959890937792, 90.11264978115807, 90.14135036234484, 90.19157637942169, 90.21310181531177, 90.29202841357538, 90.33507928535553, 90.40683073832245, 90.46423190069599, 90.5503336442563, 90.59338451603645, 90.60773480662984, 90.60773480662984, 90.67948625959676, 90.65796082370669, 90.65796082370669, 90.61490995192652, 90.51445791777284, 90.43553131950922, 90.39248044772907, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2030, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.59123197244745, 81.09349214321591, 81.51682571572074, 81.94733443352227, 82.27739111717013, 82.58592236492788, 82.91597904857574, 83.24603573222358, 83.6047929970582, 83.88462366362918, 84.19315491138696, 84.45863528736457, 84.73129080863887, 84.96807060342972, 85.21920068881396, 85.4488053383081, 85.6497094066155, 85.80038745784603, 85.87931405610964, 86.02281696204348, 86.13044414149387, 86.24524646624094, 86.41027480806487, 86.53942742340533, 86.70445576522926, 86.79055750878956, 86.89100954294324, 87.01298701298703, 87.14213962832747, 87.28564253426131, 87.41479514960179, 87.50089689316209, 87.5798234914257, 87.69462581617277, 87.80942814091985, 87.93858075626032, 87.99598191863386, 88.08208366219417, 88.13230967927102, 88.24711200401808, 88.40496520054532, 88.48389179880894, 88.56281839707253, 88.66327043122624, 88.72067159359977, 88.7924230465667, 88.82112362775347, 88.86417449953362, 88.94310109779724, 88.95027624309392, 89.02920284135753, 89.065078567841, 89.07942885843438, 89.11530458491785, 89.19423118318146, 89.28033292674176, 89.35925952500538, 89.41666068737892, 89.46688670445577, 89.60321446509292, 89.66779077276315, 89.70366649924661, 89.72519193513669, 89.73954222573008, 89.78259309751023, 89.82564396929038, 89.84716940518047, 89.92609600344407, 90.04089832819116, 90.03372318289445, 90.04807347348785, 90.11264978115807, 90.12700007175145, 90.18440123412499, 90.19875152471838, 90.19875152471838, 90.23462725120184, 90.28485326827868, 90.33507928535553, 90.3637798665423, 90.37095501183899, 90.34942957594892, 90.37813015713569, 90.37813015713569, 90.38530530243239, 90.42118102891584, 90.47140704599268, 90.55750878955298, 90.49293248188276, 90.53598335366291, 90.52163306306953, 90.53598335366291, 90.53598335366291, 90.60055966133315, 90.70818684078354, 90.60055966133315, 90.50010762717945, 90.54315849895961, 90.41400588361914, 90.06242376408122]}, {"task": "nlvr2", "expert": "nlvr2", "endpoint": "Qwen3.8", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [80.00287005811867, 80.56970653655738, 81.0073903996556, 81.42354882686375, 81.8325321087752, 82.16258879242305, 82.49264547607089, 82.75812585204851, 83.08818253569635, 83.43258950993759, 83.76264619358543, 84.04247686015643, 84.3366578173208, 84.63083877448518, 84.83891798808926, 85.14744923584703, 85.38422903063787, 85.67123484250556, 85.92236492788979, 86.10891870560378, 86.1878453038674, 86.35287364569133, 86.47485111573509, 86.69728061993256, 86.94123556002009, 87.18519050010764, 87.35739398722825, 87.48654660256871, 87.65875008968932, 87.79507785032648, 87.93140561096362, 88.1394848245677, 88.28298773050155, 88.41931549113869, 88.56999354236925, 88.63456985003947, 88.77807275597331, 88.87134964483032, 88.98615196957739, 89.12965487551124, 89.23728205496163, 89.344909234412, 89.41666068737892, 89.48841214034584, 89.58168902920285, 89.71801678984, 89.74671737102676, 89.83281911458707, 89.92609600344407, 89.93327114874077, 90.01937289230108, 90.10547463586138, 90.11982492645475, 90.0839491999713, 90.13417521704815, 90.20592667001507, 90.21310181531177, 90.22745210590514, 90.20592667001507, 90.24180239649853, 90.27050297768531, 90.32072899476215, 90.31355384946545, 90.37813015713569, 90.41400588361914, 90.44270646480591, 90.42835617421254, 90.43553131950922, 90.45705675539931, 90.47140704599268, 90.52163306306953, 90.57903422544307, 90.60773480662984, 90.65078567840999, 90.60773480662984, 90.61490995192652, 90.57185908014638, 90.57903422544307, 90.55750878955298, 90.57903422544307, 90.58620937073975, 90.64361053311329, 90.60773480662984, 90.62926024251992, 90.65796082370669, 90.58620937073975, 90.60055966133315, 90.57185908014638, 90.55750878955298, 90.54315849895961, 90.53598335366291, 90.50010762717945, 90.44270646480591, 90.39248044772907, 90.3637798665423, 90.29920355887207, 90.26332783238861, 90.21310181531177, 90.1700509435316, 90.07677405467462, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.7381789481237, 77.08258592236493, 77.42699289660617, 77.72834899906724, 78.04405539212169, 78.28083518691254, 78.74004448590084, 78.95529884480162, 79.28535552844946, 79.622587357394, 79.95981918633852, 80.2540001435029, 80.59840711774415, 80.86388749372175, 81.1006672885126, 81.2800459209299, 81.5598765875009, 81.78948123699506, 82.09083733945613, 82.29174140776351, 82.58592236492788, 82.77247614264189, 82.89445361268565, 83.19580971514672, 83.42541436464089, 83.61914328765158, 83.94202482600272, 84.17880462079357, 84.25773121905719, 84.30078209083734, 84.32948267202411, 84.38688383439765, 84.45146014206787, 84.50886130444142, 84.55191217622158, 84.55191217622158, 84.56626246681496, 84.56626246681496, 84.60213819329842, 84.63801391978187, 84.68106479156204, 84.68823993685872, 84.74564109923226, 84.8030422616058, 84.84609313338596, 84.93937002224295, 84.98242089402311, 85.07569778288011, 85.1689746717371, 85.25507641529741, 85.33400301356102, 85.3627035947478, 85.4416301930114, 85.5492573724618, 85.67123484250556, 85.79321231254933, 85.79321231254933, 85.8290880390328, 85.9151897825931, 86.02281696204348, 86.04434239793356, 86.14479443208725, 86.23089617564756, 86.33852335509793, 86.39592451747149, 86.43897538925164, 86.5609528592954, 86.64705460285572, 86.80490779938295, 86.85513381645978, 86.95558585061349, 87.05603788476716, 87.16366506421755, 87.3071679701514, 87.40762000430509, 87.56547320083232, 87.665925234986, 87.72332639735954, 87.85247901270002, 88.00315706393054, 88.11078424338093, 88.29733802109493, 88.38343976465524, 88.46954150821554, 88.56999354236923, 88.73502188419316, 88.7924230465667, 88.94310109779724, 89.12247973021454, 89.23728205496163, 89.39513525148884, 89.56016359331277, 89.68931620865322, 89.88304513166392, 89.92609600344407, 90.13417521704815, 90.28485326827868, 90.43553131950922, 90.56468393484968, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75252923871709, 77.06823563177154, 77.3911171701227, 77.7498744349573, 78.06558082801178, 78.41716294754968, 78.72569419530745, 78.96964913539499, 79.27100523785607, 79.57236134031714, 79.83784171629476, 80.23964985290952, 80.51230537418384, 80.75626031427137, 81.03609098084237, 81.33744708330343, 81.63162804046782, 81.87558298055536, 82.14823850182967, 82.3060916983569, 82.56439692903783, 82.81552701442205, 82.99490564683936, 83.17428427925665, 83.40388892875082, 83.66936930472842, 83.93484968070604, 84.19315491138696, 84.36535839850757, 84.6093133385951, 84.85326827868265, 85.06134749228671, 85.23355097940734, 85.3555284494511, 85.57078280835188, 85.8290880390328, 86.03716725263688, 86.30982277391116, 86.48920140632848, 86.67575518404249, 86.86948410705317, 87.06321303006386, 87.2067159359977, 87.34304369663487, 87.55112291023893, 87.74485183324963, 87.86682930329339, 87.9529310468537, 88.03903279041401, 88.15383511516109, 88.30451316639162, 88.40496520054532, 88.5125923799957, 88.55564325177585, 88.64174499533615, 88.74937217478654, 88.8856999354237, 88.93592595250054, 89.04355313195093, 89.07942885843438, 89.13683002080792, 89.23728205496163, 89.28750807203846, 89.32338379852193, 89.39513525148884, 89.47406184975246, 89.51711272153261, 89.60321446509292, 89.65344048216977, 89.75389251632346, 89.79694338810361, 89.86151969577384, 89.88304513166392, 89.98349716581761, 90.07677405467462, 90.13417521704815, 90.14852550764154, 90.19157637942169, 90.24180239649853, 90.22027696060846, 90.25615268709191, 90.26332783238861, 90.32072899476215, 90.39965559302576, 90.46423190069599, 90.49293248188276, 90.55750878955298, 90.56468393484968, 90.60773480662984, 90.64361053311329, 90.67231111430006, 90.67231111430006, 90.70101169548683, 90.7297122766736, 90.72253713137691, 90.70101169548683, 90.67231111430006, 90.68666140489346, 90.69383655019014, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.77405467460716, 77.07541077706824, 77.44851833249623, 77.78575016144077, 78.10863169979191, 78.35976178517615, 78.68981846882399, 78.99117457128507, 79.32123125493291, 79.5149601779436, 79.86654229748153, 80.1678983999426, 80.51230537418384, 80.76343545956806, 81.08631699791921, 81.2800459209299, 81.50965057042406, 81.82535696347851, 82.10518763004951, 82.3993685872139, 82.58592236492788, 82.82987730501543, 83.00208079213604, 83.17428427925665, 83.46129009112435, 83.6837195953218, 83.93484968070604, 84.18597976609027, 84.37970868910097, 84.6093133385951, 84.9034942957595, 85.0828729281768, 85.25507641529741, 85.3627035947478, 85.59948338953865, 85.80038745784603, 86.05869268852695, 86.2882973380211, 86.46050082514172, 86.6542297481524, 86.8479586711631, 87.02016215828371, 87.22824137188778, 87.37891942311832, 87.52242232905216, 87.73767668795294, 87.84530386740332, 87.96728133744709, 88.0462079357107, 88.15383511516109, 88.26146229461148, 88.36908947406185, 88.49824208940231, 88.56281839707253, 88.63456985003947, 88.76372246537993, 88.84264906364353, 88.95027624309392, 88.98615196957739, 89.07942885843438, 89.17270574729139, 89.21575661907154, 89.2659826361484, 89.30903350792853, 89.32338379852193, 89.45253641386238, 89.49558728564253, 89.56016359331277, 89.6175647556863, 89.66061562746646, 89.76106766162016, 89.82564396929038, 89.95479658463084, 89.997847456411, 90.04807347348785, 90.06959890937792, 90.14135036234484, 90.18440123412499, 90.24897754179523, 90.29920355887207, 90.30637870416876, 90.29920355887207, 90.32072899476215, 90.39248044772907, 90.43553131950922, 90.49293248188276, 90.58620937073975, 90.58620937073975, 90.6364353878166, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.72253713137691, 90.70818684078353, 90.71536198608023, 90.69383655019014, 90.67231111430006, 90.67948625959676, 90.744062567267, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.77405467460716, 77.07541077706824, 77.44851833249623, 77.78575016144077, 78.10863169979191, 78.35976178517615, 78.68981846882399, 78.99117457128507, 79.32123125493291, 79.5149601779436, 79.86654229748153, 80.1678983999426, 80.51230537418384, 80.76343545956806, 81.08631699791921, 81.2800459209299, 81.50965057042406, 81.82535696347851, 82.10518763004951, 82.3993685872139, 82.58592236492788, 82.82987730501543, 83.00208079213604, 83.17428427925665, 83.46129009112435, 83.6837195953218, 83.93484968070604, 84.18597976609027, 84.37970868910097, 84.6093133385951, 84.9034942957595, 85.0828729281768, 85.25507641529741, 85.3627035947478, 85.59948338953865, 85.80038745784603, 86.05869268852695, 86.2882973380211, 86.46050082514172, 86.6542297481524, 86.8479586711631, 87.02016215828371, 87.22824137188778, 87.37891942311832, 87.52242232905216, 87.73767668795294, 87.84530386740332, 87.96728133744709, 88.0462079357107, 88.15383511516109, 88.26146229461148, 88.36908947406185, 88.49824208940231, 88.56281839707253, 88.63456985003947, 88.76372246537993, 88.84264906364353, 88.95027624309392, 88.98615196957739, 89.07942885843438, 89.17270574729139, 89.21575661907154, 89.2659826361484, 89.30903350792853, 89.32338379852193, 89.45253641386238, 89.49558728564253, 89.56016359331277, 89.6175647556863, 89.66061562746646, 89.76106766162016, 89.82564396929038, 89.95479658463084, 89.997847456411, 90.04807347348785, 90.06959890937792, 90.14135036234484, 90.18440123412499, 90.24897754179523, 90.29920355887207, 90.30637870416876, 90.29920355887207, 90.32072899476215, 90.39248044772907, 90.43553131950922, 90.49293248188276, 90.58620937073975, 90.58620937073975, 90.6364353878166, 90.65796082370669, 90.67231111430006, 90.68666140489346, 90.72253713137691, 90.70818684078353, 90.71536198608023, 90.69383655019014, 90.67231111430006, 90.67948625959676, 90.744062567267, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2026, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.71665351223362, 77.08258592236493, 77.44851833249623, 77.72834899906724, 78.07275597330846, 78.31671091339601, 78.58936643467031, 78.89789768242808, 79.16337805840568, 79.47190930616345, 79.81631628040469, 80.05309607519553, 80.2611752887996, 80.61275740833752, 80.92128865609529, 81.25134533974312, 81.43789911745714, 81.78230609169836, 82.03343617708259, 82.24869053598336, 82.4854703307742, 82.75095070675181, 83.03078137332281, 83.24603573222358, 83.39671378345412, 83.43258950993759, 83.48281552701442, 83.53304154409128, 83.64784386883835, 83.71242017650857, 83.71959532180527, 83.77699648417881, 83.85592308244242, 83.84874793714573, 83.89897395422258, 83.92767453540935, 83.95637511659612, 84.0568271507498, 84.0640022960465, 84.11422831312333, 84.16445433020019, 84.25055607376049, 84.30795723613403, 84.35818325321088, 84.40840927028772, 84.46581043266126, 84.58061275740833, 84.68106479156204, 84.73846595393557, 84.87479371457273, 84.96089545813304, 85.02547176580326, 85.0900480734735, 85.1689746717371, 85.14744923584703, 85.2694267058908, 85.34835330415441, 85.51338164597834, 85.66405969720887, 85.69276027839564, 85.77886202195594, 85.9151897825931, 86.07304297912033, 86.22372103035086, 86.35287364569133, 86.38874937217479, 86.50355169692186, 86.63270431226233, 86.67575518404249, 86.81925808997633, 86.87665925234987, 86.94841070531677, 87.07756332065725, 87.18519050010764, 87.3071679701514, 87.45784602138194, 87.60134892731578, 87.64439979909594, 87.7735524144364, 87.88117959388678, 88.01750735452393, 88.12513453397432, 88.21123627753462, 88.35473918346847, 88.44084092702877, 88.59869412355602, 88.81394848245677, 88.971801678984, 89.11530458491785, 89.32338379852193, 89.48123699504916, 89.63909019157639, 89.74671737102676, 89.811293678697, 89.87586998636723, 90.01219774700438, 90.24180239649853, 90.39965559302576, 90.56468393484968, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2026, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.64490205926671, 77.03235990528809, 77.32654086245246, 77.6709478366937, 78.02970510152831, 78.36693693047285, 78.6037167252637, 78.92659826361485, 79.21360407548254, 79.50778503264692, 79.75891511803115, 80.04592092989883, 80.32575159646983, 80.6342828442276, 80.89258807490852, 81.17241874147952, 81.40919853627037, 81.72490492932482, 81.99756045059912, 82.19128937360982, 82.41371887780727, 82.62897323670805, 82.68637439908159, 82.72225012556504, 82.76530099734521, 82.83705245031211, 82.88010332209228, 82.9446796297625, 82.98055535624597, 83.03078137332281, 83.06665709980628, 83.16710913395997, 83.2316854416302, 83.37518834756405, 83.5043409629045, 83.59044270646481, 83.66936930472842, 83.69806988591519, 83.83439764655235, 83.92049939011265, 84.01377627896966, 84.08552773193657, 84.17162947549687, 84.26490636435389, 84.35100810791418, 84.40840927028772, 84.55908732151826, 84.6236636291885, 84.72411566334218, 84.82456769749588, 85.01112147520988, 85.04699720169334, 85.19767525292387, 85.34835330415441, 85.42727990241804, 85.52773193657171, 85.62818397072542, 85.77886202195594, 85.92236492788979, 86.07304297912033, 86.14479443208725, 86.17349501327402, 86.2811221927244, 86.42462509865825, 86.51790198751526, 86.61117887637225, 86.62552916696563, 86.74750663700941, 86.8407835258664, 86.88383439764655, 86.95558585061349, 87.0416875941738, 87.13496448303079, 87.28564253426131, 87.42914544019517, 87.52242232905216, 87.61569921790917, 87.73050154265624, 87.87400444859009, 88.00315706393054, 88.07490851689747, 88.19688598694124, 88.25428714931478, 88.39779005524862, 88.6058692688527, 88.69914615770969, 88.82112362775347, 88.91440051661046, 88.971801678984, 89.08660400373107, 89.20858147377486, 89.36643467030207, 89.56016359331277, 89.73236708043338, 89.83999425988375, 89.997847456411, 90.14852550764154, 90.30637870416876, 90.52163306306953, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.08258592236493, 77.37676687952931, 77.5848460931334, 77.8646767597044, 78.16603286216545, 78.56784099878023, 78.82614622946114, 79.19207863959245, 79.52931046853699, 79.77326540862452, 79.96699433163522, 80.22529956231614, 80.46207935710699, 80.80648633134821, 81.09349214321591, 81.2800459209299, 81.55270144220421, 81.77513094640166, 81.98321016000574, 82.21281480949989, 82.53569634785106, 82.74377556145512, 82.98055535624597, 83.17428427925665, 83.41823921934419, 83.75547104828873, 84.01377627896966, 84.20033005668365, 84.37970868910097, 84.59496304800173, 84.78151682571571, 85.03982205639664, 85.21920068881394, 85.4416301930114, 85.6353591160221, 85.77168687665926, 86.04434239793356, 86.2954724833178, 86.46050082514172, 86.59682858577887, 86.7546817823061, 86.87665925234987, 86.97711128650354, 87.04886273947048, 87.09908875654733, 87.17084020951424, 87.27129224366794, 87.35739398722825, 87.52242232905216, 87.60134892731578, 87.72332639735954, 87.76637726913971, 87.8596541579967, 87.96010619215039, 88.03903279041401, 88.16101026045777, 88.29016287579825, 88.35473918346847, 88.36191432876515, 88.48389179880894, 88.56999354236923, 88.64174499533615, 88.73502188419316, 88.82829877305016, 88.91440051661046, 89.00050226017078, 89.09377914902777, 89.1440051661046, 89.158355456698, 89.1440051661046, 89.2659826361484, 89.37360981559877, 89.39513525148884, 89.46688670445577, 89.58168902920285, 89.6175647556863, 89.624739900983, 89.67496591805984, 89.71801678984, 89.76106766162016, 89.8041185334003, 89.904570567554, 89.95479658463084, 89.997847456411, 90.06242376408122, 90.10547463586138, 90.13417521704815, 90.22745210590514, 90.29202841357538, 90.31355384946545, 90.39965559302576, 90.41400588361914, 90.46423190069599, 90.52880820836621, 90.56468393484968, 90.61490995192652, 90.67948625959676, 90.70101169548683, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.74535409342039, 77.03953505058477, 77.383942024826, 77.69964841788047, 78.05123053741838, 78.37411207576955, 78.61089187056038, 78.97682428069169, 79.29253067374614, 79.55083590442706, 79.90241802396498, 80.13919781875583, 80.37597761354668, 80.69885915189784, 80.9930401090622, 81.21546961325967, 81.45224940805052, 81.71772978402812, 81.96885986941236, 82.18411422831312, 82.47112004018081, 82.68637439908159, 82.98773050154266, 83.19580971514672, 83.31061203989381, 83.42541436464089, 83.48281552701442, 83.53304154409128, 83.63349357824497, 83.66936930472842, 83.76264619358543, 83.76264619358543, 83.82722250125565, 83.84157279184905, 83.89897395422258, 84.00660113367296, 84.03530171485973, 84.09987802252996, 84.17162947549687, 84.22903063787042, 84.32230752672743, 84.32948267202411, 84.35818325321088, 84.50168615914473, 84.58778790270503, 84.65953935567195, 84.76716653512234, 84.85326827868265, 84.92501973164957, 85.0900480734735, 85.20485039822057, 85.25507641529741, 85.31965272296765, 85.39140417593455, 85.51338164597834, 85.59948338953865, 85.76451173136256, 85.85778862021957, 85.9151897825931, 85.99411638085671, 86.11609385090048, 86.20219559446079, 86.2954724833178, 86.42462509865825, 86.58965344048217, 86.74033149171271, 86.81925808997633, 86.89818468823994, 86.98428643180026, 87.07756332065725, 87.1277893377341, 87.18519050010764, 87.29999282485471, 87.46502116667862, 87.63004950850255, 87.7735524144364, 87.88117959388678, 88.03185764511731, 88.12513453397432, 88.34756403817178, 88.505417234699, 88.61304441414939, 88.65609528592955, 88.69914615770969, 88.85699935423692, 89.02202769606085, 89.1440051661046, 89.21575661907154, 89.32338379852193, 89.46688670445577, 89.63909019157639, 89.72519193513668, 89.84716940518047, 90.01937289230108, 90.15570065293824, 90.3566047212456, 90.42118102891584, 90.50728277247615, 90.58620937073975, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.10411135825501, 77.3911171701227, 77.76422472555069, 78.09428140919854, 78.45303867403315, 78.76874506708761, 79.04140058836192, 79.32840640022961, 79.622587357394, 79.90959316926168, 80.12484752816245, 80.56253139126068, 80.79213604075483, 81.02174069024898, 81.25134533974312, 81.4594245533472, 81.72490492932482, 81.99756045059912, 82.17693908301644, 82.43524431369735, 82.68637439908159, 82.98773050154266, 83.26038602281697, 83.49716581760781, 83.73394561239866, 83.9635502618928, 84.22185549257372, 84.45146014206787, 84.59496304800173, 84.77434168041903, 84.93937002224295, 85.25507641529741, 85.42727990241804, 85.59948338953865, 85.86496376551625, 86.10891870560378, 86.25959675683433, 86.50355169692186, 86.73315634641602, 86.89818468823994, 87.01298701298703, 87.1277893377341, 87.29281767955801, 87.50807203845878, 87.5941737820191, 87.665925234986, 87.79507785032648, 87.89552988448015, 87.98880677333716, 88.11078424338093, 88.21841142283131, 88.35473918346847, 88.44084092702877, 88.5125923799957, 88.59151897825932, 88.6847958671163, 88.75654732008323, 88.84264906364353, 88.86417449953362, 88.96462653368731, 89.01485255076416, 89.05790342254431, 89.11530458491785, 89.15118031140132, 89.20140632847816, 89.27315778144508, 89.3520843797087, 89.41666068737892, 89.44536126856569, 89.54581330271938, 89.63191504627969, 89.67496591805984, 89.73236708043338, 89.76106766162016, 89.78976824280693, 89.86869484107054, 89.91174571285069, 89.92609600344407, 90.00502260170768, 90.05524861878453, 90.0839491999713, 90.09112434526799, 90.16287579823492, 90.20592667001507, 90.24897754179523, 90.30637870416876, 90.29202841357538, 90.34942957594892, 90.32072899476215, 90.34942957594892, 90.41400588361914, 90.44270646480592, 90.48575733658608, 90.51445791777284, 90.55750878955298, 90.58620937073975, 90.67231111430006, 90.71536198608023, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.10411135825501, 77.44134318719954, 77.77857501614407, 78.130157135682, 78.52479012700007, 78.79027050297769, 79.11315204132883, 79.35710698141638, 79.65846308387745, 79.90959316926168, 80.1607232546459, 80.49795508359044, 80.78496089545814, 81.03609098084237, 81.23699504914975, 81.51682571572074, 81.77513094640166, 81.97603501470905, 82.16976393771974, 82.46394489488412, 82.6791992537849, 82.98773050154266, 83.25321087752027, 83.47564038171772, 83.6908947406185, 83.94202482600272, 84.25055607376049, 84.41558441558442, 84.6236636291885, 84.75999138982566, 85.00394632991318, 85.24072612470403, 85.39140417593455, 85.58513309894525, 85.86496376551625, 86.04434239793356, 86.2954724833178, 86.54660256870201, 86.70445576522926, 86.88383439764655, 86.96276099591017, 87.10626390184402, 87.3071679701514, 87.4937217478654, 87.61569921790917, 87.6802755255794, 87.7735524144364, 87.85247901270002, 87.99598191863386, 88.132309679271, 88.22558656812801, 88.38343976465524, 88.44801607232546, 88.52694267058908, 88.55564325177585, 88.6847958671163, 88.72067159359977, 88.81394848245677, 88.85699935423692, 88.96462653368731, 89.04355313195093, 89.0722537131377, 89.11530458491785, 89.158355456698, 89.20140632847816, 89.28033292674176, 89.3520843797087, 89.40231039678554, 89.46688670445577, 89.531463012126, 89.6103896103896, 89.70366649924661, 89.74671737102676, 89.73954222573008, 89.78259309751022, 89.83281911458707, 89.93327114874077, 89.95479658463084, 89.99067231111431, 90.05524861878453, 90.07677405467462, 90.12700007175145, 90.19157637942169, 90.22745210590514, 90.28485326827868, 90.29202841357538, 90.31355384946545, 90.32790414005885, 90.34942957594892, 90.39248044772907, 90.46423190069599, 90.48575733658608, 90.49293248188276, 90.52163306306953, 90.54315849895961, 90.65078567840999, 90.67948625959676, 90.70818684078353, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.10411135825501, 77.44134318719954, 77.77857501614407, 78.130157135682, 78.52479012700007, 78.79027050297769, 79.11315204132883, 79.35710698141638, 79.65846308387745, 79.90959316926168, 80.1607232546459, 80.49795508359044, 80.78496089545814, 81.03609098084237, 81.23699504914975, 81.51682571572074, 81.77513094640166, 81.97603501470905, 82.16976393771974, 82.46394489488412, 82.6791992537849, 82.98773050154266, 83.25321087752027, 83.47564038171772, 83.6908947406185, 83.94202482600272, 84.25055607376049, 84.41558441558442, 84.6236636291885, 84.75999138982566, 85.00394632991318, 85.24072612470403, 85.39140417593455, 85.58513309894525, 85.86496376551625, 86.04434239793356, 86.2954724833178, 86.54660256870201, 86.70445576522926, 86.88383439764655, 86.96276099591017, 87.10626390184402, 87.3071679701514, 87.4937217478654, 87.61569921790917, 87.6802755255794, 87.7735524144364, 87.85247901270002, 87.99598191863386, 88.132309679271, 88.22558656812801, 88.38343976465524, 88.44801607232546, 88.52694267058908, 88.55564325177585, 88.6847958671163, 88.72067159359977, 88.81394848245677, 88.85699935423692, 88.96462653368731, 89.04355313195093, 89.0722537131377, 89.11530458491785, 89.158355456698, 89.20140632847816, 89.28033292674176, 89.3520843797087, 89.40231039678554, 89.46688670445577, 89.52428786682931, 89.6103896103896, 89.70366649924661, 89.74671737102676, 89.73954222573008, 89.78259309751022, 89.83281911458707, 89.93327114874077, 89.95479658463084, 89.99067231111431, 90.05524861878453, 90.07677405467462, 90.12700007175145, 90.19157637942169, 90.22745210590514, 90.28485326827868, 90.29202841357538, 90.31355384946545, 90.32790414005885, 90.34942957594892, 90.39248044772907, 90.46423190069599, 90.48575733658608, 90.49293248188276, 90.52163306306953, 90.54315849895961, 90.64361053311329, 90.67948625959676, 90.70818684078353, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2027, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.81710554638732, 77.197388247112, 77.52026978546316, 77.79292530673746, 78.1445074262754, 78.41716294754968, 78.69699361412069, 79.02705029776853, 79.39298270789983, 79.64411279328407, 79.98851976752529, 80.20377412642607, 80.42620363062353, 80.78496089545814, 81.03609098084237, 81.19394417736959, 81.40202339097367, 81.67467891224798, 81.9186338523355, 82.0262610317859, 82.04778646767598, 82.05496161297266, 82.08366219415943, 82.09801248475281, 82.13388821123628, 82.18411422831312, 82.24151539068666, 82.3060916983569, 82.3921934419172, 82.44241945899404, 82.4782951854775, 82.56439692903783, 82.6791992537849, 82.71507498026835, 82.79400157853198, 82.88010332209228, 82.9518547750592, 83.03795651861951, 83.1312334074765, 83.20298486044344, 83.23886058692689, 83.25321087752027, 83.31778718519051, 83.38953863815742, 83.43976465523427, 83.55456697998135, 83.66219415943173, 83.72677046710196, 83.79852192006888, 83.92049939011265, 84.01377627896966, 84.10705316782665, 84.22903063787042, 84.35100810791418, 84.4371098514745, 84.56626246681496, 84.6954150821554, 84.77434168041903, 84.85326827868265, 85.01112147520988, 85.1617995264404, 85.24072612470403, 85.41292961182464, 85.52055679127503, 85.57795795364856, 85.73581115017579, 85.83626318432948, 85.93671521848317, 86.02281696204348, 86.17349501327402, 86.25242161153763, 86.46050082514172, 86.58965344048217, 86.71163091052594, 86.7546817823061, 86.94123556002009, 86.99863672239363, 87.15648991892087, 87.22824137188778, 87.3143431154481, 87.42197029489847, 87.65875008968932, 87.84530386740332, 88.01750735452393, 88.22558656812801, 88.412140345842, 88.54846810647916, 88.69914615770969, 88.83547391834685, 89.02202769606085, 89.22293176436823, 89.4238358326756, 89.66779077276314, 89.82564396929038, 89.96197172992754, 90.12700007175145, 90.3566047212456, 90.57185908014638, 90.64361053311329, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2027, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.7381789481237, 77.03953505058477, 77.34089115304586, 77.65659754610031, 78.02252995623161, 78.30953576809931, 78.63241730645045, 78.98399942598839, 79.27818038315276, 79.57953648561383, 79.87371744277823, 80.1607232546459, 80.39750304943676, 80.61275740833752, 80.92846380139198, 81.22264475855636, 81.48812513453397, 81.71772978402812, 81.91145870703882, 82.1267130659396, 82.21998995479659, 82.23434024538997, 82.34196742484036, 82.4854703307742, 82.49264547607089, 82.5930975102246, 82.6791992537849, 82.77247614264189, 82.84422759560881, 82.88010332209228, 83.00925593743274, 83.12405826217982, 83.21016000574012, 83.26756116811366, 83.34648776637728, 83.43258950993759, 83.51151610820119, 83.56174212527804, 83.66219415943173, 83.74112075769534, 83.82722250125565, 83.89179880892588, 84.03530171485973, 84.11422831312333, 84.18597976609027, 84.29360694554065, 84.35818325321088, 84.4227595608811, 84.51603644973811, 84.63083877448518, 84.68106479156204, 84.75999138982566, 84.83174284279256, 84.9895960393198, 85.09722321877018, 85.1617995264404, 85.21202554351726, 85.31965272296765, 85.39857932123127, 85.50620650068164, 85.58513309894525, 85.67840998780225, 85.77168687665926, 85.8290880390328, 85.97259094496664, 86.12326899619718, 86.2811221927244, 86.34569850039465, 86.39592451747149, 86.46050082514172, 86.63270431226233, 86.68293032933917, 86.73315634641602, 86.86948410705317, 86.88383439764655, 86.96276099591017, 87.07038817536056, 87.2138910812944, 87.3143431154481, 87.44349573078856, 87.5941737820191, 87.7735524144364, 87.93140561096362, 88.02468249982063, 88.23276171342471, 88.36908947406185, 88.49106694410563, 88.63456985003947, 88.83547391834685, 89.02920284135753, 89.23010690966493, 89.39513525148884, 89.50993757623593, 89.66061562746646, 89.86151969577384, 90.02654803759776, 90.22745210590514, 90.44270646480592, 90.59338451603645, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.77405467460716, 77.1112865035517, 77.383942024826, 77.67812298199038, 77.92207792207793, 78.17320800746216, 78.55349070818684, 78.80462079357106, 79.12750233192223, 79.41450814378992, 79.78044055392121, 80.06027122049221, 80.38315275884337, 80.70603429719452, 80.92846380139198, 81.27287077563321, 81.52400086101744, 81.80383152758843, 82.04778646767598, 82.25586568128006, 82.44959460429074, 82.60744780081797, 82.89445361268565, 82.99490564683936, 83.26756116811366, 83.51869125349789, 83.71959532180527, 83.99942598837626, 84.26490636435389, 84.45863528736457, 84.7025902274521, 84.87479371457273, 85.01112147520988, 85.28377699648418, 85.49903135538494, 85.67840998780225, 85.85061347492287, 86.09456841501041, 86.2954724833178, 86.4676759704384, 86.63270431226233, 86.8479586711631, 87.06321303006386, 87.17801535481094, 87.24259166248117, 87.32869340604148, 87.42914544019517, 87.58699863672238, 87.6731003802827, 87.83095357680993, 87.90988017507355, 88.0533830810074, 88.10360909808423, 88.18253569634786, 88.29016287579825, 88.36191432876515, 88.41931549113869, 88.52694267058908, 88.57716868766593, 88.691971012413, 88.78524790127, 88.78524790127, 88.89287508072039, 88.95745138839062, 88.97897682428069, 89.05072827724761, 89.17988089258809, 89.23010690966493, 89.2588074908517, 89.29468321733516, 89.30903350792853, 89.344909234412, 89.40948554208224, 89.46688670445577, 89.55298844801607, 89.6175647556863, 89.71084164454331, 89.8041185334003, 89.85434455047715, 89.904570567554, 89.92609600344407, 89.98349716581761, 90.04089832819116, 90.06242376408122, 90.1700509435316, 90.1700509435316, 90.24897754179523, 90.28485326827868, 90.28485326827868, 90.33507928535553, 90.42118102891584, 90.42118102891584, 90.48575733658608, 90.52163306306953, 90.53598335366291, 90.60055966133315, 90.6364353878166, 90.67231111430006, 90.7297122766736, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78840496520056, 77.21173853770539, 77.44851833249623, 77.69964841788047, 77.98665422974815, 78.38128722106623, 78.72569419530745, 78.97682428069169, 79.26383009255937, 79.67998851976753, 79.95981918633852, 80.27552557939298, 80.56970653655738, 80.80648633134821, 81.05044127143574, 81.33027193800675, 81.57422687809428, 81.8468823993686, 82.06213675826936, 82.27021597187343, 82.55004663844443, 82.78682643323528, 82.88727846738897, 82.97338021094927, 83.0523068092129, 83.15993398866327, 83.2173351510368, 83.28191145870704, 83.3249623304872, 83.38236349286073, 83.44693980053096, 83.54021668938796, 83.6047929970582, 83.64784386883835, 83.71242017650857, 83.74829590299204, 83.84874793714573, 83.92767453540935, 83.95637511659612, 84.04965200545311, 84.04247686015643, 84.13575374901342, 84.22903063787042, 84.27208150965058, 84.3438329626175, 84.40123412499103, 84.48733586855134, 84.58061275740833, 84.65236421037527, 84.75999138982566, 84.86761856927603, 84.93219487694627, 85.01112147520988, 85.1617995264404, 85.22637583411064, 85.31247757767095, 85.3698787400445, 85.57795795364856, 85.69276027839564, 85.77886202195594, 85.8290880390328, 85.9151897825931, 86.0084666714501, 86.09456841501041, 86.14479443208725, 86.25959675683433, 86.38874937217479, 86.53225227810863, 86.63987945755902, 86.7546817823061, 86.89100954294324, 86.94123556002009, 87.07756332065725, 87.2067159359977, 87.3143431154481, 87.50089689316209, 87.56547320083232, 87.68745067087609, 87.8596541579967, 87.99598191863386, 88.17536055105116, 88.3260386022817, 88.49106694410563, 88.6847958671163, 88.84982420894023, 88.94310109779724, 89.10095429432447, 89.20858147377486, 89.33055894381862, 89.45253641386238, 89.6103896103896, 89.77541795221353, 89.8184688239937, 89.92609600344407, 89.997847456411, 90.14852550764154, 90.277678122982, 90.47140704599268, 90.62926024251992, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.79558011049724, 77.15433737533185, 77.477218913683, 77.71399870847385, 78.00817966563824, 78.38846236636292, 78.63959245174715, 78.95529884480162, 79.22795436607592, 79.54366075913038, 79.83784171629476, 80.1607232546459, 80.47642964770037, 80.75626031427137, 81.02891583554567, 81.32309679271006, 81.64597833106122, 81.8325321087752, 82.04061132237928, 82.27739111717013, 82.51417091196097, 82.78682643323528, 83.03795651861951, 83.18145942455335, 83.47564038171772, 83.70524503121189, 83.95637511659612, 84.09270287723328, 84.26490636435389, 84.49451101384804, 84.7025902274521, 85.02547176580326, 85.28377699648418, 85.41292961182464, 85.65688455191219, 85.85778862021957, 86.05151754323025, 86.25959675683433, 86.40309966276818, 86.66140489344909, 86.89818468823994, 86.99863672239363, 87.1134390471407, 87.26411709837124, 87.42197029489847, 87.51524718375548, 87.70180096146947, 87.78790270502978, 87.86682930329339, 87.98880677333716, 88.01750735452393, 88.14665996986439, 88.23993685872139, 88.36191432876515, 88.39779005524862, 88.45519121762216, 88.52694267058908, 88.59869412355602, 88.69914615770969, 88.79959819186338, 88.94310109779724, 89.02920284135753, 89.15118031140132, 89.20140632847816, 89.22293176436823, 89.30903350792853, 89.36643467030207, 89.45253641386238, 89.51711272153261, 89.59603931979623, 89.63909019157639, 89.69649135394992, 89.77541795221353, 89.83281911458707, 89.92609600344407, 89.99067231111431, 90.00502260170768, 90.11982492645475, 90.19875152471838, 90.24180239649853, 90.25615268709191, 90.25615268709191, 90.32790414005885, 90.38530530243239, 90.37813015713569, 90.41400588361914, 90.42118102891584, 90.44988161010261, 90.48575733658608, 90.50010762717945, 90.53598335366291, 90.5503336442563, 90.59338451603645, 90.60055966133315, 90.66513596900337, 90.70101169548683, 90.66513596900337, 90.68666140489346, 90.71536198608023, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80275525579393, 77.13281193944178, 77.46286862308962, 77.71399870847385, 78.036880246825, 78.35258663987946, 78.66111788763723, 78.94094855420822, 79.19925378488915, 79.53648561383369, 79.8880677333716, 80.20377412642607, 80.48360479299707, 80.75626031427137, 81.04326612613906, 81.31592164741336, 81.60292745928105, 81.81818181818183, 82.06213675826936, 82.2989165530602, 82.53569634785106, 82.78682643323528, 83.03078137332281, 83.21016000574012, 83.45411494582765, 83.6908947406185, 83.8774485183325, 84.0568271507498, 84.22903063787042, 84.48733586855134, 84.73846595393557, 84.99677118461649, 85.21920068881394, 85.49185621008826, 85.60665853483533, 85.8290880390328, 86.08021812441702, 86.27394704742771, 86.46050082514172, 86.66140489344909, 86.8407835258664, 86.99863672239363, 87.13496448303079, 87.26411709837124, 87.4004448590084, 87.54394776494225, 87.6802755255794, 87.79507785032648, 87.83812872210662, 87.98163162804047, 88.08925880749085, 88.11795938867762, 88.1394848245677, 88.24711200401808, 88.30451316639162, 88.40496520054532, 88.5125923799957, 88.62739470474277, 88.67044557652292, 88.76372246537993, 88.87134964483032, 88.92875080720385, 89.03637798665423, 89.1440051661046, 89.23010690966493, 89.29468321733516, 89.36643467030207, 89.4238358326756, 89.47406184975246, 89.55298844801607, 89.63909019157639, 89.69649135394992, 89.76824280691685, 89.8184688239937, 89.92609600344407, 89.96197172992754, 89.997847456411, 90.06242376408122, 90.09112434526799, 90.1700509435316, 90.22027696060846, 90.24897754179523, 90.26332783238861, 90.33507928535553, 90.37095501183899, 90.39248044772907, 90.44988161010261, 90.47140704599268, 90.47140704599268, 90.50010762717945, 90.51445791777284, 90.5503336442563, 90.58620937073975, 90.64361053311329, 90.62208509722322, 90.66513596900337, 90.67231111430006, 90.65796082370669, 90.71536198608023, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80275525579393, 77.13281193944178, 77.46286862308962, 77.71399870847385, 78.036880246825, 78.35258663987946, 78.66111788763723, 78.94094855420822, 79.19925378488915, 79.53648561383369, 79.8880677333716, 80.20377412642607, 80.48360479299707, 80.75626031427137, 81.04326612613906, 81.31592164741336, 81.60292745928105, 81.81818181818183, 82.06213675826936, 82.2989165530602, 82.53569634785106, 82.78682643323528, 83.03078137332281, 83.21016000574012, 83.45411494582765, 83.6908947406185, 83.8774485183325, 84.0568271507498, 84.22903063787042, 84.48733586855134, 84.73846595393557, 84.99677118461649, 85.21920068881394, 85.49185621008826, 85.60665853483533, 85.8290880390328, 86.08021812441702, 86.27394704742771, 86.46050082514172, 86.66140489344909, 86.8407835258664, 86.99863672239363, 87.13496448303079, 87.26411709837124, 87.4004448590084, 87.54394776494225, 87.6802755255794, 87.79507785032648, 87.83812872210662, 87.98163162804047, 88.08925880749085, 88.11795938867762, 88.1394848245677, 88.24711200401808, 88.30451316639162, 88.40496520054532, 88.5125923799957, 88.62739470474277, 88.67044557652292, 88.76372246537993, 88.87134964483032, 88.92875080720385, 89.03637798665423, 89.1440051661046, 89.23010690966493, 89.29468321733516, 89.36643467030207, 89.4238358326756, 89.47406184975246, 89.55298844801607, 89.63909019157639, 89.69649135394992, 89.76824280691685, 89.8184688239937, 89.92609600344407, 89.96197172992754, 89.997847456411, 90.06242376408122, 90.09112434526799, 90.1700509435316, 90.22027696060846, 90.24897754179523, 90.26332783238861, 90.33507928535553, 90.37095501183899, 90.39248044772907, 90.44988161010261, 90.47140704599268, 90.47140704599268, 90.50010762717945, 90.51445791777284, 90.5503336442563, 90.58620937073975, 90.64361053311329, 90.62208509722322, 90.66513596900337, 90.67231111430006, 90.65796082370669, 90.71536198608023, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2028, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.82428069168401, 77.14716223003516, 77.37676687952931, 77.77139987084739, 78.12298199038531, 78.42433809284638, 78.66111788763723, 78.89072253713138, 79.156202913109, 79.40015785319653, 79.66563822917414, 79.9741694769319, 80.27552557939298, 80.61275740833752, 80.96433952787544, 81.19394417736959, 81.4665996986439, 81.76795580110497, 81.99756045059912, 82.1984645189065, 82.4782951854775, 82.71507498026835, 82.9518547750592, 83.17428427925665, 83.24603573222358, 83.26756116811366, 83.29626174930043, 83.33213747578388, 83.33931262108058, 83.36083805697065, 83.39671378345412, 83.40388892875082, 83.45411494582765, 83.51151610820119, 83.54739183468466, 83.63349357824497, 83.6908947406185, 83.74112075769534, 83.80569706536558, 83.8774485183325, 83.97072540718949, 84.08552773193657, 84.1572791849035, 84.24338092846381, 84.25773121905719, 84.28643180024396, 84.37253354380427, 84.4227595608811, 84.5232115950348, 84.68106479156204, 84.73846595393557, 84.83891798808926, 84.87479371457273, 85.00394632991318, 85.11874865466027, 85.23355097940734, 85.33400301356102, 85.50620650068164, 85.61383368013202, 85.72863600487909, 85.78603716725264, 85.85778862021957, 85.97259094496664, 86.05151754323025, 86.11609385090048, 86.31699791920786, 86.43897538925164, 86.49637655162518, 86.5609528592954, 86.68293032933917, 86.78338236349286, 86.88383439764655, 87.02016215828371, 87.1134390471407, 87.26411709837124, 87.38609456841502, 87.42914544019517, 87.5798234914257, 87.75202697854631, 87.90988017507355, 88.00315706393054, 88.132309679271, 88.24711200401808, 88.3260386022817, 88.49824208940231, 88.65609528592955, 88.77089761067663, 88.90005022601709, 89.07942885843438, 89.29468321733516, 89.44536126856569, 89.60321446509292, 89.77541795221353, 89.904570567554, 90.06959890937792, 90.14852550764154, 90.29202841357538, 90.46423190069599, 90.60055966133315, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2028, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.8386309822774, 77.16868766592523, 77.42699289660617, 77.70682356317715, 78.05123053741838, 78.28801033220924, 78.58936643467031, 78.876372246538, 79.2351295113726, 79.52213532324029, 79.85936715218483, 80.13202267345913, 80.38315275884337, 80.68450886130445, 81.0073903996556, 81.28722106622659, 81.51682571572074, 81.81818181818183, 81.99756045059912, 82.2056396642032, 82.37784315132382, 82.56439692903783, 82.62897323670805, 82.73660041615844, 82.87292817679558, 82.98055535624597, 83.07383224510296, 83.18145942455335, 83.19580971514672, 83.29626174930043, 83.38953863815742, 83.49716581760781, 83.59044270646481, 83.66219415943173, 83.70524503121189, 83.75547104828873, 83.80569706536558, 83.88462366362918, 83.99942598837626, 84.02812656956303, 84.10705316782665, 84.13575374901342, 84.23620578316711, 84.32230752672743, 84.3366578173208, 84.37970868910097, 84.45863528736457, 84.55191217622158, 84.6236636291885, 84.75281624452894, 84.82456769749588, 84.9034942957595, 84.9895960393198, 85.10439836406688, 85.15462438114372, 85.25507641529741, 85.33400301356102, 85.37705388534118, 85.52773193657171, 85.59948338953865, 85.67840998780225, 85.75016144076918, 85.80756260314271, 85.9151897825931, 86.02281696204348, 86.1017435603071, 86.25242161153763, 86.35287364569133, 86.47485111573509, 86.63270431226233, 86.66140489344909, 86.77620721819616, 86.90535983353664, 87.00581186769031, 87.15648991892087, 87.24259166248117, 87.35739398722825, 87.45067087608524, 87.64439979909594, 87.73767668795294, 87.80225299562316, 87.89552988448015, 88.08925880749085, 88.19688598694124, 88.40496520054532, 88.55564325177585, 88.69914615770969, 88.85699935423692, 88.95027624309392, 89.17988089258809, 89.33055894381862, 89.45253641386238, 89.64626533687307, 89.8902202769606, 90.0839491999713, 90.19875152471838, 90.39248044772907, 90.52163306306953, 90.60773480662984, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75970438401379, 77.05388534117816, 77.35524144363923, 77.68529812728708, 77.95795364856139, 78.3238860586927, 78.58936643467031, 78.90507282772477, 79.2423046566693, 79.54366075913038, 79.78761569921791, 80.01004520341537, 80.30422616057974, 80.59123197244745, 80.8208366219416, 81.07914185262251, 81.35897251919351, 81.61010260457775, 81.87558298055536, 82.13388821123628, 82.41371887780727, 82.6791992537849, 82.91597904857574, 83.12405826217982, 83.36801320226735, 83.6837195953218, 83.92049939011265, 84.13575374901342, 84.36535839850757, 84.57343761211165, 84.8102174069025, 85.06852263758341, 85.24072612470403, 85.39140417593455, 85.57795795364856, 85.83626318432948, 86.00129152615341, 86.23807132094426, 86.50355169692186, 86.71880605582263, 86.90535983353664, 87.06321303006386, 87.2138910812944, 87.4004448590084, 87.51524718375548, 87.6731003802827, 87.81660328621655, 87.89552988448015, 87.96010619215039, 88.02468249982063, 88.11078424338093, 88.19688598694124, 88.26863743990816, 88.31886345698501, 88.38343976465524, 88.43366578173209, 88.49824208940231, 88.56999354236923, 88.71349644830309, 88.74219702948984, 88.82112362775347, 88.90722537131377, 88.971801678984, 89.05790342254431, 89.12247973021454, 89.158355456698, 89.2588074908517, 89.28750807203846, 89.28750807203846, 89.39513525148884, 89.41666068737892, 89.50993757623593, 89.48123699504916, 89.56016359331277, 89.56733873860946, 89.624739900983, 89.73236708043338, 89.78976824280693, 89.78259309751022, 89.811293678697, 89.87586998636723, 89.91174571285069, 89.96914687522423, 90.06242376408122, 90.0982994905647, 90.14135036234484, 90.20592667001507, 90.25615268709191, 90.32790414005885, 90.39248044772907, 90.46423190069599, 90.51445791777284, 90.57185908014638, 90.55750878955298, 90.58620937073975, 90.64361053311329, 90.65078567840999, 90.70818684078353, 90.71536198608023, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.81710554638732, 77.1184616488484, 77.44851833249623, 77.80010045203416, 78.130157135682, 78.40281265695631, 78.76874506708761, 79.13467747721891, 79.45755901557007, 79.64411279328407, 79.88089258807491, 80.11767238286576, 80.36880246825, 80.62710769893091, 80.87106263901845, 81.09349214321591, 81.41637368156705, 81.66032862165459, 81.99038530530244, 82.1984645189065, 82.44959460429074, 82.71507498026835, 83.01643108272943, 83.18145942455335, 83.38953863815742, 83.5043409629045, 83.54739183468466, 83.55456697998135, 83.61196814235488, 83.71959532180527, 83.74112075769534, 83.79852192006888, 83.85592308244242, 83.95637511659612, 83.98507569778289, 84.03530171485973, 84.08552773193657, 84.12140345842003, 84.21468034727704, 84.27925665494726, 84.38688383439765, 84.44428499677119, 84.58778790270503, 84.65953935567195, 84.73129080863887, 84.8030422616058, 84.91066944105619, 84.96807060342972, 84.99677118461649, 85.07569778288011, 85.1761498170338, 85.22637583411064, 85.32682786826433, 85.40575446652795, 85.49903135538494, 85.65688455191219, 85.72146085958241, 85.77168687665926, 85.89366434670302, 86.0156418167468, 86.02999210734018, 86.09456841501041, 86.20937073975749, 86.30264762861448, 86.39592451747149, 86.43897538925164, 86.5681280045921, 86.74033149171271, 86.87665925234987, 87.01298701298703, 87.07756332065725, 87.19236564540432, 87.29281767955801, 87.36456913252493, 87.45067087608524, 87.55112291023893, 87.65157494439264, 87.73767668795294, 87.83095357680993, 87.93140561096362, 88.09643395278755, 88.18253569634786, 88.28298773050155, 88.46236636291886, 88.54846810647916, 88.64892014063285, 88.82112362775347, 88.94310109779724, 89.07942885843438, 89.27315778144508, 89.3520843797087, 89.49558728564253, 89.66061562746646, 89.76824280691685, 89.96197172992754, 90.0982994905647, 90.25615268709191, 90.40683073832245, 90.60055966133315, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75970438401379, 77.1184616488484, 77.46286862308962, 77.80727559733084, 78.10145655449523, 78.38846236636292, 78.61806701585708, 78.96964913539499, 79.22077922077922, 79.622587357394, 79.88089258807491, 80.088971801679, 80.27552557939298, 80.46925450240367, 80.77778575016144, 81.09349214321591, 81.44507426275382, 81.74643036521489, 82.00473559589582, 82.29174140776351, 82.52134605725766, 82.72225012556504, 82.92315419387243, 83.18145942455335, 83.39671378345412, 83.59044270646481, 83.85592308244242, 84.00660113367296, 84.20750520198035, 84.4371098514745, 84.63801391978187, 84.85326827868265, 85.05417234699003, 85.2694267058908, 85.4416301930114, 85.72863600487909, 85.95824065437326, 86.06586783382365, 86.2811221927244, 86.44615053454832, 86.63987945755902, 86.74750663700941, 86.89818468823994, 87.04886273947048, 87.14213962832747, 87.29281767955801, 87.4004448590084, 87.52242232905216, 87.61569921790917, 87.73050154265624, 87.75920212384301, 87.89552988448015, 87.96010619215039, 88.07490851689747, 88.12513453397432, 88.25428714931478, 88.33321374757838, 88.412140345842, 88.505417234699, 88.54846810647916, 88.64892014063285, 88.691971012413, 88.75654732008323, 88.82829877305016, 88.90722537131377, 88.96462653368731, 89.02920284135753, 89.09377914902777, 89.1440051661046, 89.23010690966493, 89.2588074908517, 89.27315778144508, 89.3377340891153, 89.38078496089545, 89.50276243093923, 89.60321446509292, 89.66779077276314, 89.74671737102676, 89.8184688239937, 89.83999425988375, 89.86869484107054, 89.93327114874077, 89.95479658463084, 90.02654803759776, 90.07677405467462, 90.11982492645475, 90.15570065293824, 90.19157637942169, 90.26332783238861, 90.31355384946545, 90.3566047212456, 90.41400588361914, 90.43553131950922, 90.48575733658608, 90.52163306306953, 90.57185908014638, 90.61490995192652, 90.67948625959676, 90.69383655019014, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80993040109063, 77.24043911889216, 77.4700437683863, 77.663772691397, 78.00100452034154, 78.30236062280261, 78.56784099878023, 78.90507282772477, 79.22795436607592, 79.40015785319653, 79.66563822917414, 79.96699433163522, 80.29705101528306, 80.58405682715075, 80.77778575016144, 81.06479156202914, 81.39484824567698, 81.76078065580829, 82.10518763004951, 82.36349286073043, 82.62897323670805, 82.80117672382866, 82.91597904857574, 83.18863456985004, 83.38953863815742, 83.58326756116811, 83.84874793714573, 84.04965200545311, 84.22903063787042, 84.41558441558442, 84.68106479156204, 84.85326827868265, 85.06134749228671, 85.28377699648418, 85.52055679127503, 85.66405969720887, 85.87931405610964, 86.06586783382365, 86.21654588505417, 86.39592451747149, 86.5753031498888, 86.69728061993256, 86.85513381645978, 87.0416875941738, 87.14213962832747, 87.27846738896463, 87.34304369663487, 87.41479514960177, 87.57264834612901, 87.6731003802827, 87.74485183324963, 87.88835473918347, 87.92423046566694, 87.98880677333716, 88.0462079357107, 88.1394848245677, 88.26146229461148, 88.35473918346847, 88.43366578173209, 88.47671665351223, 88.54846810647916, 88.62739470474277, 88.72067159359977, 88.76372246537993, 88.90005022601709, 88.99332711487408, 89.065078567841, 89.158355456698, 89.17988089258809, 89.251632345555, 89.3377340891153, 89.38796010619215, 89.44536126856569, 89.46688670445577, 89.55298844801607, 89.63909019157639, 89.67496591805984, 89.73954222573008, 89.84716940518047, 89.8973954222573, 89.8902202769606, 89.93327114874077, 89.98349716581761, 89.96914687522423, 90.02654803759776, 90.06242376408122, 90.14852550764154, 90.24180239649853, 90.23462725120184, 90.31355384946545, 90.33507928535553, 90.3637798665423, 90.42118102891584, 90.44270646480592, 90.49293248188276, 90.54315849895961, 90.59338451603645, 90.67231111430006, 90.72253713137691, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80993040109063, 77.24043911889216, 77.4700437683863, 77.663772691397, 78.00100452034154, 78.30236062280261, 78.56784099878023, 78.90507282772477, 79.22795436607592, 79.40015785319653, 79.66563822917414, 79.96699433163522, 80.29705101528306, 80.58405682715075, 80.77778575016144, 81.06479156202914, 81.39484824567698, 81.76078065580829, 82.10518763004951, 82.36349286073043, 82.62897323670805, 82.80117672382866, 82.91597904857574, 83.18863456985004, 83.38953863815742, 83.58326756116811, 83.84874793714573, 84.04965200545311, 84.22903063787042, 84.41558441558442, 84.68106479156204, 84.85326827868265, 85.06134749228671, 85.28377699648418, 85.52055679127503, 85.66405969720887, 85.87931405610964, 86.06586783382365, 86.21654588505417, 86.39592451747149, 86.5753031498888, 86.69728061993256, 86.85513381645978, 87.0416875941738, 87.14213962832747, 87.27846738896463, 87.34304369663487, 87.41479514960177, 87.57264834612901, 87.6731003802827, 87.74485183324963, 87.88835473918347, 87.92423046566694, 87.98880677333716, 88.0462079357107, 88.1394848245677, 88.26146229461148, 88.35473918346847, 88.43366578173209, 88.47671665351223, 88.54846810647916, 88.62739470474277, 88.72067159359977, 88.76372246537993, 88.90005022601709, 88.99332711487408, 89.065078567841, 89.158355456698, 89.17988089258809, 89.251632345555, 89.3377340891153, 89.38796010619215, 89.44536126856569, 89.46688670445577, 89.55298844801607, 89.63909019157639, 89.67496591805984, 89.73954222573008, 89.84716940518047, 89.8973954222573, 89.8902202769606, 89.93327114874077, 89.98349716581761, 89.96914687522423, 90.02654803759776, 90.06242376408122, 90.14852550764154, 90.24180239649853, 90.24180239649853, 90.31355384946545, 90.33507928535553, 90.3637798665423, 90.42118102891584, 90.44270646480592, 90.49293248188276, 90.54315849895961, 90.59338451603645, 90.67231111430006, 90.72253713137691, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2029, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.7381789481237, 77.0180096146947, 77.32654086245246, 77.73552414436392, 78.02970510152831, 78.35258663987946, 78.62524216115376, 78.89072253713138, 79.22795436607592, 79.50778503264692, 79.83784171629476, 80.13202267345913, 80.39032790414005, 80.66298342541437, 80.87106263901845, 81.14371816029275, 81.35897251919351, 81.63880318576452, 81.9329841429289, 82.1195379206429, 82.3060916983569, 82.50699576666428, 82.72225012556504, 82.84422759560881, 82.88727846738897, 82.9446796297625, 82.99490564683936, 83.07383224510296, 83.12405826217982, 83.19580971514672, 83.26756116811366, 83.31061203989381, 83.36801320226735, 83.43258950993759, 83.5043409629045, 83.58326756116811, 83.61196814235488, 83.71242017650857, 83.7913467747722, 83.8774485183325, 83.95637511659612, 84.02095142426634, 84.07835258663988, 84.22903063787042, 84.29360694554065, 84.35818325321088, 84.40123412499103, 84.47298557795796, 84.55191217622158, 84.6164884838918, 84.73846595393557, 84.85326827868265, 84.96807060342972, 85.07569778288011, 85.21920068881394, 85.33400301356102, 85.46315562890149, 85.54208222716511, 85.6353591160221, 85.70711056898902, 85.81473774843941, 85.92236492788979, 85.98694123556002, 86.13761928679055, 86.14479443208725, 86.27394704742771, 86.36722393628472, 86.45332567984502, 86.5609528592954, 86.6542297481524, 86.78338236349286, 86.97711128650354, 87.08473846595393, 87.17801535481094, 87.2138910812944, 87.37174427782163, 87.50807203845878, 87.60134892731578, 87.69462581617277, 87.8596541579967, 88.00315706393054, 88.1394848245677, 88.24711200401808, 88.44084092702877, 88.5125923799957, 88.691971012413, 88.84982420894023, 89.00050226017078, 89.17270574729139, 89.35925952500538, 89.47406184975246, 89.63191504627969, 89.70366649924661, 89.83999425988375, 89.8902202769606, 90.10547463586138, 90.32790414005885, 90.47140704599268, 90.56468393484968, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2029, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.8458061275741, 77.1184616488484, 77.383942024826, 77.663772691397, 77.943603357968, 78.28083518691254, 78.64676759704385, 78.89789768242808, 79.22077922077922, 79.57953648561383, 79.86654229748153, 80.1678983999426, 80.40467819473345, 80.65580828011768, 80.9069383655019, 81.25852048503982, 81.43072397216044, 81.68902920284135, 81.85405754466528, 82.09083733945613, 82.1984645189065, 82.34196742484036, 82.42806916840067, 82.55722178374111, 82.6720241084882, 82.75812585204851, 82.85140274090551, 82.93032933916912, 83.01643108272943, 83.15993398866327, 83.27473631341034, 83.37518834756405, 83.51151610820119, 83.6837195953218, 83.74829590299204, 83.85592308244242, 83.92049939011265, 83.99942598837626, 84.17162947549687, 84.21468034727704, 84.32230752672743, 84.37970868910097, 84.48733586855134, 84.54473703092488, 84.65953935567195, 84.8030422616058, 84.87479371457273, 85.00394632991318, 85.0900480734735, 85.1689746717371, 85.24072612470403, 85.3555284494511, 85.4559804836048, 85.49903135538494, 85.52055679127503, 85.59230824424195, 85.68558513309895, 85.76451173136256, 85.81473774843941, 85.97976609026333, 86.08739326971371, 86.16631986797732, 86.31699791920786, 86.38874937217479, 86.46050082514172, 86.5609528592954, 86.67575518404249, 86.74750663700941, 86.89100954294324, 87.00581186769031, 87.17801535481094, 87.35021884193155, 87.43632058549186, 87.55829805553563, 87.65157494439264, 87.75202697854631, 87.85247901270002, 87.93858075626032, 88.0462079357107, 88.14665996986439, 88.25428714931478, 88.36908947406185, 88.45519121762216, 88.61304441414939, 88.70632130300639, 88.76372246537993, 88.90722537131377, 88.971801678984, 89.10095429432447, 89.21575661907154, 89.32338379852193, 89.49558728564253, 89.66061562746646, 89.77541795221353, 89.91892085814737, 90.01937289230108, 90.13417521704815, 90.30637870416876, 90.52163306306953, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.69512807634355, 77.00365932410132, 77.31219057185909, 77.55614551194661, 77.82880103322093, 78.08710626390184, 78.38128722106623, 78.66829303293392, 78.92659826361485, 79.2423046566693, 79.50778503264692, 79.83784171629476, 80.15354810934922, 80.41185334003013, 80.70603429719452, 81.0073903996556, 81.31592164741336, 81.53117600631413, 81.85405754466528, 82.15541364712635, 82.40654373251058, 82.69354954437827, 82.86575303149888, 83.09535768099305, 83.36801320226735, 83.51869125349789, 83.69806988591519, 83.98507569778289, 84.21468034727704, 84.4371098514745, 84.66671450096864, 84.83174284279256, 85.13309894525365, 85.40575446652795, 85.58513309894525, 85.77168687665926, 85.88648920140633, 86.05151754323025, 86.18067015857072, 86.34569850039465, 86.51072684221856, 86.64705460285572, 86.76903207289948, 86.9340604147234, 87.05603788476716, 87.19954079070101, 87.27846738896463, 87.40762000430509, 87.55829805553563, 87.63722465379924, 87.79507785032648, 87.90988017507355, 87.97445648274378, 88.03185764511731, 88.132309679271, 88.19688598694124, 88.26863743990816, 88.35473918346847, 88.41931549113869, 88.49824208940231, 88.59151897825932, 88.67762072181962, 88.73502188419316, 88.77089761067663, 88.84264906364353, 88.8856999354237, 88.971801678984, 89.07942885843438, 89.18705603788477, 89.2659826361484, 89.30185836263185, 89.39513525148884, 89.48123699504916, 89.50993757623593, 89.51711272153261, 89.60321446509292, 89.68214106335654, 89.74671737102676, 89.74671737102676, 89.79694338810361, 89.8973954222573, 89.96914687522423, 90.06242376408122, 90.13417521704815, 90.1772260888283, 90.24897754179523, 90.277678122982, 90.37813015713569, 90.39248044772907, 90.44270646480592, 90.45705675539931, 90.53598335366291, 90.54315849895961, 90.54315849895961, 90.60773480662984, 90.64361053311329, 90.67231111430006, 90.70101169548683, 90.73688742197031, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75970438401379, 77.08258592236493, 77.43416804190285, 77.72117385377054, 78.01535481093492, 78.30236062280261, 78.68981846882399, 79.03422544306522, 79.37145727200976, 79.7086891009543, 79.93829375044845, 80.2540001435029, 80.46207935710699, 80.68450886130445, 80.96433952787544, 81.30157135681998, 81.53835115161083, 81.80383152758843, 81.99756045059912, 82.21281480949989, 82.49264547607089, 82.80117672382866, 83.03078137332281, 83.18145942455335, 83.42541436464089, 83.51869125349789, 83.59044270646481, 83.64784386883835, 83.73394561239866, 83.76264619358543, 83.84157279184905, 83.89897395422258, 83.98507569778289, 84.0711774413432, 84.08552773193657, 84.1572791849035, 84.23620578316711, 84.32230752672743, 84.38688383439765, 84.48016072325466, 84.58778790270503, 84.6164884838918, 84.65236421037527, 84.73846595393557, 84.85326827868265, 84.91066944105619, 85.05417234699003, 85.11874865466027, 85.1761498170338, 85.24790127000072, 85.31247757767095, 85.39140417593455, 85.43445504771472, 85.4559804836048, 85.53490708186841, 85.6353591160221, 85.71428571428572, 85.87213891081295, 86.02281696204348, 86.09456841501041, 86.20219559446079, 86.2954724833178, 86.41027480806487, 86.48920140632848, 86.5681280045921, 86.64705460285572, 86.71880605582263, 86.8479586711631, 86.99863672239363, 87.14213962832747, 87.3071679701514, 87.4004448590084, 87.43632058549186, 87.58699863672238, 87.62287436320587, 87.79507785032648, 87.89552988448015, 87.96010619215039, 88.09643395278755, 88.24711200401808, 88.39779005524862, 88.59151897825932, 88.72784673889646, 88.84982420894023, 89.01485255076416, 89.158355456698, 89.3520843797087, 89.4238358326756, 89.48123699504916, 89.60321446509292, 89.72519193513668, 89.8184688239937, 89.8902202769606, 90.00502260170768, 90.11264978115807, 90.23462725120184, 90.37813015713569, 90.52880820836621, 90.70101169548683, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75252923871709, 77.08976106766163, 77.41264260601277, 77.74269928966062, 78.036880246825, 78.37411207576955, 78.64676759704385, 79.00552486187846, 79.29253067374614, 79.57953648561383, 79.95981918633852, 80.30422616057974, 80.58405682715075, 80.82801176723828, 81.07914185262251, 81.28722106622659, 81.55270144220421, 81.8397072540719, 82.04778646767598, 82.21998995479659, 82.4854703307742, 82.77247614264189, 83.01643108272943, 83.18145942455335, 83.41823921934419, 83.6837195953218, 83.95637511659612, 84.18597976609027, 84.50168615914473, 84.72411566334218, 84.86761856927603, 84.99677118461649, 85.24072612470403, 85.51338164597834, 85.77886202195594, 85.9080146372964, 86.13761928679055, 86.3815742268781, 86.51072684221856, 86.68293032933917, 86.89100954294324, 87.07038817536056, 87.24976680777786, 87.42914544019517, 87.665925234986, 87.78790270502978, 87.93858075626032, 88.132309679271, 88.30451316639162, 88.37626461935855, 88.42649063643539, 88.52694267058908, 88.64174499533615, 88.69914615770969, 88.82829877305016, 88.91440051661046, 89.02920284135753, 89.10812943962115, 89.20858147377486, 89.3377340891153, 89.38078496089545, 89.37360981559877, 89.40948554208224, 89.48123699504916, 89.531463012126, 89.58886417449953, 89.67496591805984, 89.73236708043338, 89.76106766162016, 89.78976824280693, 89.82564396929038, 89.85434455047715, 89.904570567554, 89.92609600344407, 89.96197172992754, 89.99067231111431, 90.03372318289445, 90.07677405467462, 90.13417521704815, 90.20592667001507, 90.23462725120184, 90.25615268709191, 90.28485326827868, 90.277678122982, 90.30637870416876, 90.33507928535553, 90.42118102891584, 90.44988161010261, 90.51445791777284, 90.58620937073975, 90.59338451603645, 90.60773480662984, 90.62208509722322, 90.66513596900337, 90.71536198608023, 90.7297122766736, 90.71536198608023, 90.71536198608023, 90.7297122766736, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.71665351223362, 77.08258592236493, 77.3982923154194, 77.73552414436392, 78.02970510152831, 78.409987802253, 78.67546817823062, 78.99834971658176, 79.29970581904284, 79.65128793858077, 79.9741694769319, 80.26835043409629, 80.61993255363421, 80.87823778431513, 81.13654301499605, 81.33027193800675, 81.61010260457775, 81.81818181818183, 82.06931190356606, 82.27021597187343, 82.49264547607089, 82.74377556145512, 83.02360622802611, 83.18145942455335, 83.41823921934419, 83.69806988591519, 83.97072540718949, 84.17880462079357, 84.50168615914473, 84.68823993685872, 84.8891440051661, 84.99677118461649, 85.19767525292387, 85.44880533830809, 85.71428571428572, 85.90083949199972, 86.16631986797732, 86.33134820980125, 86.5609528592954, 86.66858003874579, 86.89818468823994, 87.0416875941738, 87.2067159359977, 87.40762000430509, 87.5941737820191, 87.75920212384301, 87.945755901557, 88.15383511516109, 88.24711200401808, 88.36908947406185, 88.46954150821554, 88.5125923799957, 88.6058692688527, 88.74937217478654, 88.90005022601709, 88.90722537131377, 88.95745138839062, 89.02202769606085, 89.08660400373107, 89.16553060199469, 89.23728205496163, 89.27315778144508, 89.40231039678554, 89.4310109779723, 89.54581330271938, 89.56016359331277, 89.66779077276314, 89.74671737102676, 89.78976824280693, 89.83999425988375, 89.91174571285069, 89.96197172992754, 90.01937289230108, 90.0839491999713, 90.13417521704815, 90.16287579823492, 90.14135036234484, 90.16287579823492, 90.23462725120184, 90.24180239649853, 90.26332783238861, 90.30637870416876, 90.29202841357538, 90.37095501183899, 90.41400588361914, 90.44270646480592, 90.42835617421254, 90.46423190069599, 90.52163306306953, 90.58620937073975, 90.62926024251992, 90.65078567840999, 90.67231111430006, 90.65796082370669, 90.64361053311329, 90.65796082370669, 90.70818684078353, 90.72253713137691, 90.71536198608023, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.71665351223362, 77.08258592236493, 77.3982923154194, 77.73552414436392, 78.02970510152831, 78.409987802253, 78.67546817823062, 78.99834971658176, 79.29970581904284, 79.65128793858077, 79.9741694769319, 80.26835043409629, 80.61993255363421, 80.87823778431513, 81.13654301499605, 81.33027193800675, 81.61010260457775, 81.81818181818183, 82.06931190356606, 82.27021597187343, 82.49264547607089, 82.74377556145512, 83.02360622802611, 83.18145942455335, 83.41823921934419, 83.69806988591519, 83.97072540718949, 84.17880462079357, 84.50168615914473, 84.68823993685872, 84.8891440051661, 84.99677118461649, 85.19767525292387, 85.44880533830809, 85.71428571428572, 85.90083949199972, 86.16631986797732, 86.33134820980125, 86.5609528592954, 86.66858003874579, 86.89818468823994, 87.0416875941738, 87.2067159359977, 87.40762000430509, 87.5941737820191, 87.75920212384301, 87.945755901557, 88.15383511516109, 88.24711200401808, 88.36908947406185, 88.46954150821554, 88.5125923799957, 88.6058692688527, 88.74937217478654, 88.90005022601709, 88.90722537131377, 88.95745138839062, 89.02202769606085, 89.08660400373107, 89.16553060199469, 89.23728205496163, 89.27315778144508, 89.40231039678554, 89.4310109779723, 89.54581330271938, 89.56016359331277, 89.66779077276314, 89.74671737102676, 89.78976824280693, 89.83999425988375, 89.91174571285069, 89.96197172992754, 90.01937289230108, 90.0839491999713, 90.13417521704815, 90.16287579823492, 90.14135036234484, 90.16287579823492, 90.23462725120184, 90.24180239649853, 90.26332783238861, 90.30637870416876, 90.29202841357538, 90.37095501183899, 90.41400588361914, 90.44270646480592, 90.42835617421254, 90.46423190069599, 90.52163306306953, 90.58620937073975, 90.62926024251992, 90.65078567840999, 90.67231111430006, 90.65796082370669, 90.64361053311329, 90.65796082370669, 90.70818684078353, 90.72253713137691, 90.71536198608023, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2030, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.82428069168401, 77.04671019588147, 77.41981775130947, 77.70682356317715, 78.11580684508861, 78.35976178517615, 78.64676759704385, 78.91224797302145, 79.25665494726269, 79.65128793858077, 79.99569491282199, 80.21812441701944, 80.43337877592022, 80.74191002367799, 80.97868981846884, 81.30157135681998, 81.5670517327976, 81.81100667288513, 82.04778646767598, 82.33479227954366, 82.5787472196312, 82.79400157853198, 82.99490564683936, 83.27473631341034, 83.51869125349789, 83.71242017650857, 83.92049939011265, 84.04965200545311, 84.1572791849035, 84.22903063787042, 84.27925665494726, 84.35818325321088, 84.37970868910097, 84.4227595608811, 84.48016072325466, 84.5375618856282, 84.59496304800173, 84.71694051804549, 84.76716653512234, 84.82456769749588, 84.91066944105619, 85.00394632991318, 85.06134749228671, 85.0900480734735, 85.1617995264404, 85.23355097940734, 85.29095214178088, 85.34835330415441, 85.47750591949487, 85.57078280835188, 85.62100882542872, 85.66405969720887, 85.75016144076918, 85.85061347492287, 85.9151897825931, 86.0156418167468, 86.06586783382365, 86.13044414149387, 86.21654588505417, 86.2811221927244, 86.36722393628472, 86.43897538925164, 86.49637655162518, 86.55377771399871, 86.63270431226233, 86.77620721819616, 86.87665925234987, 86.89818468823994, 86.97711128650354, 87.04886273947048, 87.07756332065725, 87.22106622659109, 87.29281767955801, 87.3932697137117, 87.48654660256871, 87.68745067087609, 87.7735524144364, 87.80225299562316, 87.88835473918347, 88.01033220922724, 88.16101026045777, 88.28298773050155, 88.39779005524862, 88.54846810647916, 88.67762072181962, 88.79959819186338, 88.878524790127, 89.01485255076416, 89.17270574729139, 89.39513525148884, 89.46688670445577, 89.624739900983, 89.71801678984, 89.811293678697, 89.92609600344407, 90.07677405467462, 90.19157637942169, 90.32072899476215, 90.55750878955298, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2030, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.70230322164025, 77.02518475999139, 77.3982923154194, 77.77857501614407, 78.06558082801178, 78.43868838343977, 78.72569419530745, 79.04857573365861, 79.28535552844946, 79.56518619502046, 79.85936715218483, 80.11049723756906, 80.37597761354668, 80.69885915189784, 80.94281409198535, 81.20829446796299, 81.4594245533472, 81.74643036521489, 82.04778646767598, 82.3060916983569, 82.4854703307742, 82.60744780081797, 82.65049867259813, 82.75095070675181, 82.82270215971874, 82.90880390327904, 82.9518547750592, 83.00925593743274, 83.12405826217982, 83.16710913395997, 83.18863456985004, 83.24603573222358, 83.28191145870704, 83.33213747578388, 83.38953863815742, 83.42541436464089, 83.45411494582765, 83.59044270646481, 83.66219415943173, 83.70524503121189, 83.77699648417881, 83.88462366362918, 83.92049939011265, 84.04965200545311, 84.1572791849035, 84.22185549257372, 84.27208150965058, 84.38688383439765, 84.50168615914473, 84.64518906507857, 84.78869197101243, 84.9034942957595, 84.9895960393198, 85.06134749228671, 85.13309894525365, 85.23355097940734, 85.33400301356102, 85.40575446652795, 85.53490708186841, 85.62100882542872, 85.75733658606588, 85.81473774843941, 85.94389036377987, 86.08739326971371, 86.22372103035086, 86.36722393628472, 86.4676759704384, 86.59682858577887, 86.69010547463587, 86.8479586711631, 86.94123556002009, 87.07038817536056, 87.18519050010764, 87.32151826074478, 87.45067087608524, 87.5798234914257, 87.72332639735954, 87.80225299562316, 87.90988017507355, 88.03185764511731, 88.14665996986439, 88.23276171342471, 88.29733802109493, 88.40496520054532, 88.56281839707253, 88.6847958671163, 88.80677333716008, 88.98615196957739, 89.1440051661046, 89.22293176436823, 89.35925952500538, 89.50276243093923, 89.65344048216977, 89.78259309751022, 89.95479658463084, 90.11982492645475, 90.19157637942169, 90.32790414005885, 90.47858219128938, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "MiniCPM", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.70230322164025, 77.07541077706824, 77.34806629834254, 77.71399870847385, 78.05123053741838, 78.43151323814308, 78.76874506708761, 79.01270000717516, 79.32840640022961, 79.6154122120973, 79.83066657099806, 80.1750735452393, 80.4477290665136, 80.75626031427137, 80.96433952787544, 81.24417019444644, 81.4665996986439, 81.61010260457775, 81.91145870703882, 82.16258879242305, 82.42806916840067, 82.70072468967497, 82.93750448446582, 83.26038602281697, 83.48281552701442, 83.76264619358543, 83.99942598837626, 84.22903063787042, 84.45863528736457, 84.68106479156204, 84.85326827868265, 85.0900480734735, 85.29812728707756, 85.54208222716511, 85.64253426131879, 85.84343832962618, 86.05151754323025, 86.32417306450456, 86.52507713281194, 86.71880605582263, 86.92688526942672, 87.1206141924374, 87.29999282485471, 87.47937145727201, 87.5798234914257, 87.78790270502978, 87.88835473918347, 88.02468249982063, 88.16818540575447, 88.29733802109493, 88.34756403817178, 88.43366578173209, 88.55564325177585, 88.6058692688527, 88.70632130300639, 88.78524790127, 88.84264906364353, 88.94310109779724, 89.08660400373107, 89.11530458491785, 89.158355456698, 89.20140632847816, 89.29468321733516, 89.36643467030207, 89.38796010619215, 89.45971155915908, 89.52428786682931, 89.55298844801607, 89.6175647556863, 89.68214106335654, 89.71801678984, 89.74671737102676, 89.79694338810361, 89.83281911458707, 89.85434455047715, 89.904570567554, 89.94044629403746, 90.01219774700438, 90.09112434526799, 90.11982492645475, 90.14135036234484, 90.15570065293824, 90.21310181531177, 90.25615268709191, 90.277678122982, 90.32072899476215, 90.32072899476215, 90.30637870416876, 90.3637798665423, 90.3637798665423, 90.38530530243239, 90.42835617421254, 90.49293248188276, 90.47858219128938, 90.50010762717945, 90.54315849895961, 90.59338451603645, 90.67948625959676, 90.72253713137691, 90.73688742197031]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80275525579393, 77.07541077706824, 77.31936571715578, 77.7498744349573, 77.99382937504484, 78.35258663987946, 78.6826433235273, 78.95529884480162, 79.3355815455263, 79.65846308387745, 80.01004520341537, 80.34727703235991, 80.67015857071105, 80.92846380139198, 81.21546961325967, 81.44507426275382, 81.63880318576452, 81.91145870703882, 82.14106335653297, 82.40654373251058, 82.5787472196312, 82.79400157853198, 83.00925593743274, 83.07383224510296, 83.13840855277319, 83.21016000574012, 83.29626174930043, 83.34648776637728, 83.45411494582765, 83.52586639879458, 83.6047929970582, 83.69806988591519, 83.81287221066228, 83.84874793714573, 83.95637511659612, 84.04247686015643, 84.0711774413432, 84.12857860371673, 84.22185549257372, 84.29360694554065, 84.36535839850757, 84.46581043266126, 84.55908732151826, 84.68106479156204, 84.78869197101243, 84.82456769749588, 84.87479371457273, 84.98242089402311, 85.0900480734735, 85.20485039822057, 85.28377699648418, 85.3627035947478, 85.47033077419817, 85.55643251775848, 85.66405969720887, 85.74298629547249, 85.85778862021957, 85.97976609026333, 86.08739326971371, 86.21654588505417, 86.31699791920786, 86.41027480806487, 86.49637655162518, 86.5681280045921, 86.69728061993256, 86.78338236349286, 86.86230896175647, 86.98428643180026, 87.09908875654733, 87.25694195307454, 87.35739398722825, 87.50807203845878, 87.6802755255794, 87.83095357680993, 87.90988017507355, 88.0533830810074, 88.19688598694124, 88.31168831168831, 88.47671665351223, 88.62739470474277, 88.73502188419316, 88.85699935423692, 88.95745138839062, 88.971801678984, 89.03637798665423, 89.12965487551124, 89.27315778144508, 89.36643467030207, 89.4238358326756, 89.56016359331277, 89.67496591805984, 89.78259309751022, 89.8902202769606, 89.997847456411, 90.15570065293824, 90.1772260888283, 90.22745210590514, 90.23462725120184, 90.277678122982, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.07541077706824, 77.34806629834254, 77.76422472555069, 78.09428140919854, 78.409987802253, 78.71851905001077, 78.9624739900983, 79.3355815455263, 79.622587357394, 79.95264404104184, 80.1607232546459, 80.39032790414005, 80.66298342541437, 80.88541292961182, 81.22264475855636, 81.54552629690751, 81.88993327114873, 82.08366219415943, 82.33479227954366, 82.5787472196312, 82.76530099734521, 83.00208079213604, 83.20298486044344, 83.33213747578388, 83.57609241587143, 83.7841716294755, 83.99942598837626, 84.30078209083734, 84.5232115950348, 84.71694051804549, 84.8173925521992, 85.06134749228671, 85.27660185118748, 85.49185621008826, 85.64253426131879, 85.79321231254933, 85.98694123556002, 86.13761928679055, 86.31699791920786, 86.51072684221856, 86.58965344048217, 86.67575518404249, 86.78338236349286, 86.90535983353664, 86.96993614120686, 87.06321303006386, 87.19236564540432, 87.34304369663487, 87.42914544019517, 87.45067087608524, 87.55112291023893, 87.65875008968932, 87.69462581617277, 87.73767668795294, 87.86682930329339, 87.9529310468537, 88.01750735452393, 88.0462079357107, 88.12513453397432, 88.15383511516109, 88.18971084164454, 88.23276171342471, 88.33321374757838, 88.36908947406185, 88.37626461935855, 88.45519121762216, 88.48389179880894, 88.56281839707253, 88.62021955944608, 88.74937217478654, 88.85699935423692, 88.90005022601709, 88.92157566190716, 89.00050226017078, 89.09377914902777, 89.13683002080792, 89.18705603788477, 89.22293176436823, 89.29468321733516, 89.35925952500538, 89.4310109779723, 89.48841214034584, 89.50993757623593, 89.531463012126, 89.55298844801607, 89.54581330271938, 89.624739900983, 89.65344048216977, 89.68214106335654, 89.72519193513668, 89.75389251632346, 89.76106766162016, 89.83999425988375, 89.8184688239937, 89.8184688239937, 89.91174571285069, 89.997847456411, 90.03372318289445, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.1112865035517, 77.44134318719954, 77.80727559733084, 78.24495946042907, 78.53196527229677, 78.78309535768099, 79.01987515247184, 79.37145727200976, 79.57953648561383, 79.8019659898113, 80.10332209227236, 80.4549042118103, 80.75626031427137, 81.10784243380928, 81.35179737389683, 81.6531534763579, 81.96168472411567, 82.24869053598336, 82.37784315132382, 82.60027265552128, 82.80835186912535, 83.02360622802611, 83.20298486044344, 83.37518834756405, 83.51869125349789, 83.80569706536558, 84.02812656956303, 84.22903063787042, 84.48016072325466, 84.75281624452894, 84.83891798808926, 85.01829662050658, 85.31965272296765, 85.43445504771472, 85.60665853483533, 85.80756260314271, 85.95106550907656, 86.13044414149387, 86.25959675683433, 86.44615053454832, 86.63270431226233, 86.74033149171271, 86.8479586711631, 86.9340604147234, 87.0273373035804, 87.10626390184402, 87.18519050010764, 87.28564253426131, 87.42197029489847, 87.47937145727201, 87.58699863672238, 87.70897610676617, 87.81660328621655, 87.88835473918347, 87.92423046566694, 87.96010619215039, 88.0533830810074, 88.10360909808423, 88.23276171342471, 88.21841142283131, 88.22558656812801, 88.3260386022817, 88.33321374757838, 88.41931549113869, 88.44084092702877, 88.47671665351223, 88.54846810647916, 88.6058692688527, 88.65609528592955, 88.72067159359977, 88.80677333716008, 88.89287508072039, 88.92157566190716, 88.971801678984, 89.07942885843438, 89.17270574729139, 89.18705603788477, 89.2588074908517, 89.32338379852193, 89.36643467030207, 89.39513525148884, 89.44536126856569, 89.48123699504916, 89.55298844801607, 89.56733873860946, 89.58886417449953, 89.59603931979623, 89.66779077276314, 89.69649135394992, 89.76106766162016, 89.76106766162016, 89.77541795221353, 89.83281911458707, 89.82564396929038, 89.83281911458707, 89.91892085814737, 90.01219774700438, 90.03372318289445, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.1112865035517, 77.44134318719954, 77.80727559733084, 78.24495946042907, 78.53196527229677, 78.78309535768099, 79.01987515247184, 79.37145727200976, 79.57953648561383, 79.8019659898113, 80.10332209227236, 80.4549042118103, 80.75626031427137, 81.10784243380928, 81.35179737389683, 81.6531534763579, 81.96168472411567, 82.24869053598336, 82.37784315132382, 82.60027265552128, 82.80835186912535, 83.02360622802611, 83.20298486044344, 83.37518834756405, 83.51869125349789, 83.80569706536558, 84.02812656956303, 84.22903063787042, 84.48016072325466, 84.75281624452894, 84.83891798808926, 85.01829662050658, 85.31965272296765, 85.43445504771472, 85.60665853483533, 85.80756260314271, 85.95106550907656, 86.13044414149387, 86.25959675683433, 86.44615053454832, 86.63270431226233, 86.74033149171271, 86.8479586711631, 86.9340604147234, 87.0273373035804, 87.10626390184402, 87.18519050010764, 87.28564253426131, 87.42197029489847, 87.47937145727201, 87.58699863672238, 87.70897610676617, 87.81660328621655, 87.88835473918347, 87.92423046566694, 87.9529310468537, 88.0533830810074, 88.11078424338093, 88.21841142283131, 88.21841142283131, 88.21841142283131, 88.3260386022817, 88.31886345698501, 88.43366578173209, 88.43366578173209, 88.49824208940231, 88.56281839707253, 88.59151897825932, 88.63456985003947, 88.72067159359977, 88.78524790127, 88.90005022601709, 88.89287508072039, 88.93592595250054, 89.05072827724761, 89.16553060199469, 89.19423118318146, 89.24445720025831, 89.30185836263185, 89.35925952500538, 89.38796010619215, 89.45253641386238, 89.48123699504916, 89.52428786682931, 89.57451388390615, 89.58168902920285, 89.6103896103896, 89.66061562746646, 89.69649135394992, 89.73954222573008, 89.77541795221353, 89.8041185334003, 89.811293678697, 89.83999425988375, 89.87586998636723, 89.88304513166392, 89.91892085814737, 90.02654803759776, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2026, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.917557580541, 77.26913970007892, 77.49874434957307, 77.83597617851761, 78.15168257157208, 78.44586352873645, 78.77592021238429, 79.07727631484538, 79.38580756260315, 79.67998851976753, 79.95264404104184, 80.28270072468968, 80.59840711774415, 80.89258807490852, 81.1867690320729, 81.38767310038028, 81.6531534763579, 81.94015928822559, 82.21281480949989, 82.44241945899404, 82.73660041615844, 82.99490564683936, 83.2316854416302, 83.36083805697065, 83.45411494582765, 83.51151610820119, 83.63349357824497, 83.6908947406185, 83.76982133888211, 83.86309822773912, 83.92767453540935, 84.04247686015643, 84.1429288943101, 84.20033005668365, 84.29360694554065, 84.38688383439765, 84.45863528736457, 84.59496304800173, 84.6954150821554, 84.8102174069025, 84.82456769749588, 84.86761856927603, 85.01112147520988, 85.05417234699003, 85.09722321877018, 85.26225156059411, 85.39140417593455, 85.4559804836048, 85.57078280835188, 85.60665853483533, 85.67840998780225, 85.81473774843941, 85.9080146372964, 86.07304297912033, 86.15196957738395, 86.22372103035086, 86.2882973380211, 86.42462509865825, 86.47485111573509, 86.61117887637225, 86.70445576522926, 86.76903207289948, 86.80490779938295, 86.89818468823994, 86.96276099591017, 87.06321303006386, 87.15648991892087, 87.29999282485471, 87.4937217478654, 87.57264834612901, 87.6802755255794, 87.84530386740332, 88.03185764511731, 88.10360909808423, 88.17536055105116, 88.24711200401808, 88.37626461935855, 88.44084092702877, 88.49824208940231, 88.59869412355602, 88.66327043122624, 88.70632130300639, 88.74219702948984, 88.82829877305016, 89.01485255076416, 89.08660400373107, 89.23728205496163, 89.37360981559877, 89.4238358326756, 89.50993757623593, 89.58886417449953, 89.65344048216977, 89.8041185334003, 89.96197172992754, 90.11264978115807, 90.16287579823492, 90.23462725120184, 90.27050297768531, 90.26332783238861, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2026, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.04671019588147, 77.41264260601277, 77.74269928966062, 78.07275597330846, 78.36693693047285, 78.72569419530745, 79.00552486187846, 79.25665494726269, 79.50778503264692, 79.90241802396498, 80.12484752816245, 80.41902848532683, 80.66298342541437, 81.00021525435889, 81.35897251919351, 81.63880318576452, 81.99038530530244, 82.24869053598336, 82.3850182966205, 82.6648489631915, 82.86575303149888, 83.2245102963335, 83.41823921934419, 83.47564038171772, 83.62631843294827, 83.70524503121189, 83.76982133888211, 83.7913467747722, 83.86309822773912, 83.92767453540935, 84.01377627896966, 84.0568271507498, 84.1429288943101, 84.24338092846381, 84.30078209083734, 84.3438329626175, 84.38688383439765, 84.47298557795796, 84.55191217622158, 84.66671450096864, 84.74564109923226, 84.82456769749588, 84.9034942957595, 84.9895960393198, 85.07569778288011, 85.1617995264404, 85.26225156059411, 85.37705388534118, 85.47033077419817, 85.57795795364856, 85.65688455191219, 85.77168687665926, 85.87931405610964, 85.97259094496664, 86.06586783382365, 86.15196957738395, 86.2811221927244, 86.42462509865825, 86.48202626103179, 86.54660256870201, 86.67575518404249, 86.79773265408625, 86.94841070531677, 87.0416875941738, 87.15648991892087, 87.24259166248117, 87.36456913252493, 87.40762000430509, 87.50089689316209, 87.57264834612901, 87.73050154265624, 87.92423046566694, 88.01750735452393, 88.16818540575447, 88.31886345698501, 88.44801607232546, 88.54846810647916, 88.66327043122624, 88.87134964483032, 89.02920284135753, 89.10812943962115, 89.24445720025831, 89.3520843797087, 89.46688670445577, 89.55298844801607, 89.65344048216977, 89.75389251632346, 89.87586998636723, 89.997847456411, 90.0839491999713, 90.12700007175145, 90.25615268709191, 90.34225443065222, 90.3566047212456, 90.37095501183899, 90.29202841357538, 90.24180239649853, 90.24897754179523, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75252923871709, 77.10411135825501, 77.44134318719954, 77.80727559733084, 78.2306091698357, 78.56066585348354, 78.89789768242808, 79.22077922077922, 79.55083590442706, 79.86654229748153, 80.13202267345913, 80.48360479299707, 80.80648633134821, 81.05761641673243, 81.32309679271006, 81.63162804046782, 81.89710841644543, 82.15541364712635, 82.44241945899404, 82.65049867259813, 82.93032933916912, 83.1312334074765, 83.38953863815742, 83.63349357824497, 83.84157279184905, 84.04965200545311, 84.26490636435389, 84.45863528736457, 84.71694051804549, 84.88196885986942, 85.10439836406688, 85.24072612470403, 85.31247757767095, 85.4559804836048, 85.64253426131879, 85.84343832962618, 86.0156418167468, 86.20219559446079, 86.41027480806487, 86.61117887637225, 86.82643323527302, 86.99146157709694, 87.15648991892087, 87.29281767955801, 87.36456913252493, 87.50089689316209, 87.5941737820191, 87.6802755255794, 87.82377843151325, 87.96010619215039, 88.08925880749085, 88.11795938867762, 88.20406113223794, 88.21841142283131, 88.31168831168831, 88.34756403817178, 88.41931549113869, 88.48389179880894, 88.59151897825932, 88.67044557652292, 88.74219702948984, 88.75654732008323, 88.81394848245677, 88.86417449953362, 88.96462653368731, 89.00050226017078, 88.97897682428069, 89.05072827724761, 89.07942885843438, 89.09377914902777, 89.16553060199469, 89.27315778144508, 89.33055894381862, 89.35925952500538, 89.40231039678554, 89.44536126856569, 89.43818612326899, 89.46688670445577, 89.50993757623593, 89.48123699504916, 89.52428786682931, 89.57451388390615, 89.624739900983, 89.6103896103896, 89.66061562746646, 89.67496591805984, 89.76106766162016, 89.79694338810361, 89.8184688239937, 89.83281911458707, 89.85434455047715, 89.8902202769606, 89.93327114874077, 89.95479658463084, 89.96914687522423, 89.997847456411, 90.04089832819116, 90.04807347348785, 90.05524861878453, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.8386309822774, 77.16151252062855, 77.52026978546316, 77.83597617851761, 78.17320800746216, 78.47456410992322, 78.71851905001077, 79.02705029776853, 79.32123125493291, 79.622587357394, 79.95264404104184, 80.30422616057974, 80.61275740833752, 80.88541292961182, 81.1867690320729, 81.43072397216044, 81.71772978402812, 81.94015928822559, 82.16976393771974, 82.36349286073043, 82.60744780081797, 82.88727846738897, 83.08100739039965, 83.31061203989381, 83.53304154409128, 83.6837195953218, 83.71242017650857, 83.73394561239866, 83.7841716294755, 83.83439764655235, 83.86309822773912, 83.89179880892588, 83.94202482600272, 84.01377627896966, 84.0568271507498, 84.16445433020019, 84.27208150965058, 84.35818325321088, 84.45863528736457, 84.55908732151826, 84.6164884838918, 84.7025902274521, 84.8102174069025, 84.91066944105619, 84.97524574872641, 85.06852263758341, 85.14027409055034, 85.29812728707756, 85.3698787400445, 85.41292961182464, 85.4559804836048, 85.62100882542872, 85.72146085958241, 85.88648920140633, 86.00129152615341, 86.11609385090048, 86.15196957738395, 86.2954724833178, 86.39592451747149, 86.51790198751526, 86.64705460285572, 86.77620721819616, 86.91971012413, 87.00581186769031, 87.1277893377341, 87.22824137188778, 87.33586855133817, 87.35021884193155, 87.42197029489847, 87.52242232905216, 87.65157494439264, 87.76637726913971, 87.82377843151325, 87.88835473918347, 87.93858075626032, 88.0533830810074, 88.11795938867762, 88.25428714931478, 88.29016287579825, 88.38343976465524, 88.505417234699, 88.63456985003947, 88.72784673889646, 88.77807275597331, 88.89287508072039, 89.09377914902777, 89.17988089258809, 89.20858147377486, 89.27315778144508, 89.4310109779723, 89.68214106335654, 89.811293678697, 89.997847456411, 90.12700007175145, 90.10547463586138, 90.0839491999713, 90.20592667001507, 90.25615268709191, 90.34225443065222, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.79558011049724, 77.18303795651862, 77.49156920427639, 77.79292530673746, 78.16603286216545, 78.47456410992322, 78.84049652005454, 79.03422544306522, 79.27818038315276, 79.60106192150391, 79.90241802396498, 80.1607232546459, 80.47642964770037, 80.79931118605153, 81.15089330558945, 81.42354882686375, 81.70337949343475, 81.9258089976322, 82.2056396642032, 82.4854703307742, 82.62897323670805, 82.9446796297625, 83.12405826217982, 83.34648776637728, 83.48999067231112, 83.71242017650857, 83.8774485183325, 84.1429288943101, 84.4299347061778, 84.7097653727488, 84.82456769749588, 84.9895960393198, 85.23355097940734, 85.39140417593455, 85.52055679127503, 85.71428571428572, 85.9080146372964, 86.08739326971371, 86.23807132094426, 86.41744995336155, 86.60400373107555, 86.69010547463587, 86.85513381645978, 86.94841070531677, 87.0416875941738, 87.1134390471407, 87.19236564540432, 87.3143431154481, 87.50089689316209, 87.60852407261247, 87.61569921790917, 87.70897610676617, 87.7735524144364, 87.79507785032648, 87.87400444859009, 87.96728133744709, 88.06055822630408, 88.14665996986439, 88.19688598694124, 88.23993685872139, 88.28298773050155, 88.34038889287508, 88.412140345842, 88.46236636291886, 88.54846810647916, 88.61304441414939, 88.64892014063285, 88.70632130300639, 88.77089761067663, 88.82829877305016, 88.86417449953362, 88.91440051661046, 88.971801678984, 89.0722537131377, 89.13683002080792, 89.17270574729139, 89.2659826361484, 89.28750807203846, 89.30903350792853, 89.30903350792853, 89.39513525148884, 89.43818612326899, 89.45971155915908, 89.50276243093923, 89.54581330271938, 89.59603931979623, 89.66061562746646, 89.75389251632346, 89.85434455047715, 89.82564396929038, 89.83281911458707, 89.86869484107054, 89.88304513166392, 89.91174571285069, 89.87586998636723, 89.95479658463084, 89.95479658463084, 89.99067231111431, 90.04807347348785, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.81710554638732, 77.13281193944178, 77.50591949486977, 77.81445074262754, 78.18755829805554, 78.41716294754968, 78.69699361412069, 79.0557508789553, 79.32840640022961, 79.63693764798737, 79.94546889574514, 80.2540001435029, 80.52665566477721, 80.87106263901845, 81.16524359618282, 81.50247542512736, 81.79665638229174, 82.04778646767598, 82.3060916983569, 82.52852120255434, 82.80117672382866, 82.97338021094927, 83.18863456985004, 83.31061203989381, 83.49716581760781, 83.69806988591519, 83.85592308244242, 84.09270287723328, 84.40123412499103, 84.65953935567195, 84.84609313338596, 85.04699720169334, 85.23355097940734, 85.44880533830809, 85.59230824424195, 85.67840998780225, 85.85778862021957, 86.11609385090048, 86.21654588505417, 86.38874937217479, 86.53225227810863, 86.76185692760279, 86.86948410705317, 87.00581186769031, 87.10626390184402, 87.16366506421755, 87.24259166248117, 87.3071679701514, 87.4937217478654, 87.57264834612901, 87.65875008968932, 87.71615125206286, 87.80942814091985, 87.89552988448015, 87.96728133744709, 88.00315706393054, 88.08208366219417, 88.07490851689747, 88.20406113223794, 88.22558656812801, 88.29016287579825, 88.39061490995192, 88.45519121762216, 88.5125923799957, 88.54846810647916, 88.62739470474277, 88.691971012413, 88.71349644830309, 88.78524790127, 88.84264906364353, 88.87134964483032, 88.92875080720385, 88.97897682428069, 89.02920284135753, 89.158355456698, 89.15118031140132, 89.20858147377486, 89.29468321733516, 89.3377340891153, 89.3520843797087, 89.38796010619215, 89.45253641386238, 89.45971155915908, 89.51711272153261, 89.56733873860946, 89.6103896103896, 89.69649135394992, 89.75389251632346, 89.83999425988375, 89.83281911458707, 89.83999425988375, 89.84716940518047, 89.88304513166392, 89.8902202769606, 89.86869484107054, 89.93327114874077, 89.94044629403746, 89.997847456411, 90.04807347348785, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.81710554638732, 77.13281193944178, 77.50591949486977, 77.81445074262754, 78.18755829805554, 78.41716294754968, 78.69699361412069, 79.0557508789553, 79.32840640022961, 79.63693764798737, 79.94546889574514, 80.2540001435029, 80.52665566477721, 80.87106263901845, 81.16524359618282, 81.50247542512736, 81.79665638229174, 82.04778646767598, 82.3060916983569, 82.52852120255434, 82.80117672382866, 82.97338021094927, 83.18863456985004, 83.31061203989381, 83.49716581760781, 83.69806988591519, 83.85592308244242, 84.09270287723328, 84.40123412499103, 84.65953935567195, 84.84609313338596, 85.04699720169334, 85.23355097940734, 85.44880533830809, 85.59230824424195, 85.67840998780225, 85.85778862021957, 86.11609385090048, 86.21654588505417, 86.38874937217479, 86.53225227810863, 86.76185692760279, 86.86948410705317, 87.00581186769031, 87.10626390184402, 87.16366506421755, 87.24259166248117, 87.3071679701514, 87.4937217478654, 87.57264834612901, 87.65875008968932, 87.71615125206286, 87.80942814091985, 87.89552988448015, 87.96728133744709, 88.00315706393054, 88.08208366219417, 88.07490851689747, 88.20406113223794, 88.22558656812801, 88.29016287579825, 88.39061490995192, 88.45519121762216, 88.5125923799957, 88.54846810647916, 88.62739470474277, 88.691971012413, 88.71349644830309, 88.78524790127, 88.84264906364353, 88.87134964483032, 88.92875080720385, 88.97897682428069, 89.02920284135753, 89.158355456698, 89.15118031140132, 89.20858147377486, 89.29468321733516, 89.3377340891153, 89.3520843797087, 89.38796010619215, 89.45253641386238, 89.45971155915908, 89.51711272153261, 89.56733873860946, 89.6103896103896, 89.69649135394992, 89.76106766162016, 89.84716940518047, 89.82564396929038, 89.83999425988375, 89.83999425988375, 89.88304513166392, 89.88304513166392, 89.86151969577384, 89.93327114874077, 89.94044629403746, 90.00502260170768, 90.04089832819116, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2027, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80993040109063, 77.07541077706824, 77.43416804190285, 77.77139987084739, 78.12298199038531, 78.48173925521992, 78.74721963119752, 79.03422544306522, 79.34993183611968, 79.6082370668006, 79.8019659898113, 80.09614694697568, 80.34010188706321, 80.69885915189784, 80.94998923728205, 81.25852048503982, 81.53117600631413, 81.77513094640166, 82.00473559589582, 82.26304082657674, 82.54287149314774, 82.80835186912535, 83.09535768099305, 83.26756116811366, 83.3249623304872, 83.40388892875082, 83.47564038171772, 83.54021668938796, 83.61196814235488, 83.66936930472842, 83.71242017650857, 83.77699648417881, 83.85592308244242, 83.90614909951927, 83.97790055248619, 84.09270287723328, 84.16445433020019, 84.21468034727704, 84.3438329626175, 84.47298557795796, 84.51603644973811, 84.57343761211165, 84.68823993685872, 84.8173925521992, 84.9034942957595, 84.96807060342972, 85.10439836406688, 85.22637583411064, 85.32682786826433, 85.39857932123127, 85.50620650068164, 85.57078280835188, 85.65688455191219, 85.72863600487909, 85.9151897825931, 86.03716725263688, 86.16631986797732, 86.30264762861448, 86.36004879098802, 86.51072684221856, 86.58965344048217, 86.74750663700941, 86.88383439764655, 86.98428643180026, 87.09908875654733, 87.2067159359977, 87.28564253426131, 87.34304369663487, 87.40762000430509, 87.55829805553563, 87.65875008968932, 87.74485183324963, 87.85247901270002, 88.00315706393054, 88.132309679271, 88.29016287579825, 88.36908947406185, 88.44084092702877, 88.56999354236923, 88.67044557652292, 88.73502188419316, 88.83547391834685, 88.94310109779724, 89.0722537131377, 89.1440051661046, 89.29468321733516, 89.43818612326899, 89.6103896103896, 89.68214106335654, 89.811293678697, 89.8902202769606, 89.94762143933414, 89.997847456411, 90.02654803759776, 90.09112434526799, 90.22027696060846, 90.29920355887207, 90.277678122982, 90.29202841357538, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2027, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75970438401379, 77.02518475999139, 77.40546746071608, 77.69964841788047, 78.02252995623161, 78.33823634928608, 78.73286934060415, 78.95529884480162, 79.29970581904284, 79.63693764798737, 79.95264404104184, 80.21812441701944, 80.57688168185406, 80.9069383655019, 81.23699504914975, 81.4665996986439, 81.76078065580829, 82.04778646767598, 82.37066800602713, 82.62897323670805, 82.80835186912535, 82.99490564683936, 83.12405826217982, 83.33213747578388, 83.55456697998135, 83.67654445002512, 83.82004735595896, 83.99942598837626, 84.13575374901342, 84.20033005668365, 84.35818325321088, 84.39405897969434, 84.40840927028772, 84.48016072325466, 84.5232115950348, 84.55191217622158, 84.64518906507857, 84.68823993685872, 84.72411566334218, 84.79586711630911, 84.86761856927603, 84.97524574872641, 85.06852263758341, 85.1689746717371, 85.25507641529741, 85.39857932123127, 85.49185621008826, 85.59948338953865, 85.68558513309895, 85.78603716725264, 85.92954007318649, 86.03716725263688, 86.15914472268064, 86.2811221927244, 86.41027480806487, 86.44615053454832, 86.51072684221856, 86.66858003874579, 86.82643323527302, 86.91971012413, 87.02016215828371, 87.16366506421755, 87.24259166248117, 87.33586855133817, 87.43632058549186, 87.53677261964555, 87.68745067087609, 87.78790270502978, 87.9529310468537, 88.03185764511731, 88.11795938867762, 88.23276171342471, 88.28298773050155, 88.45519121762216, 88.53411781588576, 88.6058692688527, 88.65609528592955, 88.75654732008323, 88.85699935423692, 88.91440051661046, 89.08660400373107, 89.16553060199469, 89.22293176436823, 89.35925952500538, 89.40231039678554, 89.48841214034584, 89.48841214034584, 89.58886417449953, 89.67496591805984, 89.76824280691685, 89.83281911458707, 89.8973954222573, 89.97632202052091, 90.13417521704815, 90.30637870416876, 90.28485326827868, 90.18440123412499, 90.12700007175145, 90.21310181531177, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.8314558369807, 77.18303795651862, 77.54897036664993, 77.943603357968, 78.28083518691254, 78.6037167252637, 78.91224797302145, 79.22077922077922, 79.55801104972376, 79.88089258807491, 80.22529956231614, 80.40467819473345, 80.74191002367799, 81.02174069024898, 81.2800459209299, 81.53835115161083, 81.8468823993686, 82.16258879242305, 82.3993685872139, 82.63614838200473, 82.85140274090551, 83.03078137332281, 83.31778718519051, 83.51151610820119, 83.74112075769534, 83.9635502618928, 84.13575374901342, 84.32230752672743, 84.6164884838918, 84.71694051804549, 84.77434168041903, 85.01829662050658, 85.10439836406688, 85.29812728707756, 85.51338164597834, 85.62100882542872, 85.9080146372964, 86.05869268852695, 86.24524646624094, 86.39592451747149, 86.59682858577887, 86.63270431226233, 86.79055750878956, 86.90535983353664, 86.99146157709694, 87.1206141924374, 87.22106622659109, 87.37174427782163, 87.51524718375548, 87.64439979909594, 87.72332639735954, 87.73050154265624, 87.75920212384301, 87.83095357680993, 87.945755901557, 88.01033220922724, 88.06773337160078, 88.09643395278755, 88.20406113223794, 88.26146229461148, 88.37626461935855, 88.412140345842, 88.45519121762216, 88.5125923799957, 88.56999354236923, 88.67762072181962, 88.74219702948984, 88.80677333716008, 88.8856999354237, 88.90005022601709, 88.87134964483032, 88.90722537131377, 88.96462653368731, 89.00767740546746, 89.05790342254431, 89.12247973021454, 89.21575661907154, 89.27315778144508, 89.37360981559877, 89.38796010619215, 89.45253641386238, 89.47406184975246, 89.46688670445577, 89.531463012126, 89.56016359331277, 89.58168902920285, 89.63191504627969, 89.68931620865322, 89.75389251632346, 89.78259309751022, 89.79694338810361, 89.85434455047715, 89.86869484107054, 89.87586998636723, 89.88304513166392, 89.91892085814737, 89.94044629403746, 90.01937289230108, 90.05524861878453, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.8458061275741, 77.17586281122193, 77.51309464016647, 77.8575016144077, 78.1445074262754, 78.43868838343977, 78.77592021238429, 79.0557508789553, 79.43603357967999, 79.74456482743776, 80.13202267345913, 80.39750304943676, 80.81366147664491, 81.07196670732583, 81.33744708330343, 81.60292745928105, 81.86840783525867, 82.06931190356606, 82.33479227954366, 82.4782951854775, 82.53569634785106, 82.65767381789482, 82.72225012556504, 82.84422759560881, 82.9518547750592, 83.03795651861951, 83.08100739039965, 83.15993398866327, 83.28908660400374, 83.38236349286073, 83.53304154409128, 83.58326756116811, 83.6837195953218, 83.75547104828873, 83.84874793714573, 83.97790055248619, 84.07835258663988, 84.21468034727704, 84.32230752672743, 84.35100810791418, 84.4299347061778, 84.5232115950348, 84.65953935567195, 84.74564109923226, 84.83891798808926, 84.8891440051661, 84.99677118461649, 85.04699720169334, 85.20485039822057, 85.31247757767095, 85.44880533830809, 85.60665853483533, 85.72863600487909, 85.9080146372964, 86.04434239793356, 86.23089617564756, 86.24524646624094, 86.36722393628472, 86.53942742340533, 86.66140489344909, 86.74750663700941, 86.89818468823994, 86.91253497883332, 87.0345124488771, 87.1277893377341, 87.19236564540432, 87.37891942311832, 87.54394776494225, 87.63722465379924, 87.80225299562316, 87.93140561096362, 88.03185764511731, 88.17536055105116, 88.25428714931478, 88.29733802109493, 88.46236636291886, 88.65609528592955, 88.73502188419316, 88.85699935423692, 88.99332711487408, 89.10095429432447, 89.23728205496163, 89.31620865322523, 89.45971155915908, 89.56733873860946, 89.66061562746646, 89.72519193513668, 89.8041185334003, 89.8041185334003, 89.8973954222573, 90.02654803759776, 90.04807347348785, 90.19157637942169, 90.31355384946545, 90.37095501183899, 90.50728277247615, 90.54315849895961, 90.43553131950922, 90.32072899476215, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.13281193944178, 77.4843940589797, 77.79292530673746, 78.12298199038531, 78.4960895458133, 78.81179593886777, 79.04140058836192, 79.32840640022961, 79.71586424625099, 80.06027122049221, 80.37597761354668, 80.65580828011768, 80.97151467317214, 81.24417019444644, 81.63162804046782, 81.88275812585205, 82.10518763004951, 82.34196742484036, 82.58592236492788, 82.80835186912535, 83.02360622802611, 83.31061203989381, 83.58326756116811, 83.76982133888211, 84.01377627896966, 84.26490636435389, 84.44428499677119, 84.6236636291885, 84.84609313338596, 85.0900480734735, 85.28377699648418, 85.4416301930114, 85.60665853483533, 85.79321231254933, 85.92236492788979, 86.10891870560378, 86.31699791920786, 86.44615053454832, 86.55377771399871, 86.76185692760279, 86.91253497883332, 87.00581186769031, 87.07756332065725, 87.19954079070101, 87.35021884193155, 87.42914544019517, 87.51524718375548, 87.5941737820191, 87.71615125206286, 87.72332639735954, 87.85247901270002, 87.88835473918347, 87.93140561096362, 87.99598191863386, 88.03185764511731, 88.11795938867762, 88.21123627753462, 88.26146229461148, 88.31168831168831, 88.39779005524862, 88.53411781588576, 88.64892014063285, 88.691971012413, 88.78524790127, 88.84982420894023, 88.92157566190716, 89.00767740546746, 89.04355313195093, 89.10095429432447, 89.10812943962115, 89.1440051661046, 89.20140632847816, 89.27315778144508, 89.30903350792853, 89.33055894381862, 89.37360981559877, 89.41666068737892, 89.44536126856569, 89.47406184975246, 89.49558728564253, 89.56733873860946, 89.60321446509292, 89.6175647556863, 89.60321446509292, 89.63191504627969, 89.63191504627969, 89.64626533687307, 89.67496591805984, 89.69649135394992, 89.76824280691685, 89.811293678697, 89.83281911458707, 89.811293678697, 89.85434455047715, 89.8902202769606, 89.91892085814737, 89.97632202052091, 90.01219774700438, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80275525579393, 77.197388247112, 77.44851833249623, 77.82162588792423, 78.18755829805554, 78.46738896462654, 78.76874506708761, 79.09880175073546, 79.45038387027338, 79.79479084451461, 80.03874578460214, 80.34727703235991, 80.75626031427137, 81.06479156202914, 81.3733228097869, 81.61727774987443, 81.87558298055536, 82.08366219415943, 82.33479227954366, 82.56439692903783, 82.80835186912535, 83.06665709980628, 83.25321087752027, 83.51869125349789, 83.81287221066228, 84.01377627896966, 84.28643180024396, 84.47298557795796, 84.66671450096864, 84.8891440051661, 85.03264691109995, 85.25507641529741, 85.43445504771472, 85.57795795364856, 85.79321231254933, 85.97976609026333, 86.14479443208725, 86.26677190213103, 86.41027480806487, 86.52507713281194, 86.68293032933917, 86.89100954294324, 87.0273373035804, 87.18519050010764, 87.32869340604148, 87.40762000430509, 87.50089689316209, 87.5798234914257, 87.62287436320587, 87.71615125206286, 87.80942814091985, 87.83095357680993, 87.87400444859009, 87.91705532037024, 88.01750735452393, 88.07490851689747, 88.16818540575447, 88.22558656812801, 88.29733802109493, 88.39779005524862, 88.47671665351223, 88.58434383296262, 88.64892014063285, 88.72067159359977, 88.83547391834685, 88.89287508072039, 88.89287508072039, 89.01485255076416, 89.05790342254431, 89.09377914902777, 89.18705603788477, 89.22293176436823, 89.2659826361484, 89.23728205496163, 89.27315778144508, 89.3377340891153, 89.39513525148884, 89.4310109779723, 89.50993757623593, 89.52428786682931, 89.56016359331277, 89.6103896103896, 89.6103896103896, 89.58886417449953, 89.60321446509292, 89.6103896103896, 89.68214106335654, 89.75389251632346, 89.72519193513668, 89.77541795221353, 89.75389251632346, 89.78259309751022, 89.82564396929038, 89.83281911458707, 89.86151969577384, 89.86151969577384, 89.91892085814737, 89.96197172992754, 90.00502260170768, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80275525579393, 77.197388247112, 77.44851833249623, 77.82162588792423, 78.18755829805554, 78.46738896462654, 78.76874506708761, 79.09880175073546, 79.45038387027338, 79.79479084451461, 80.03874578460214, 80.34727703235991, 80.75626031427137, 81.06479156202914, 81.3733228097869, 81.61727774987443, 81.87558298055536, 82.08366219415943, 82.33479227954366, 82.56439692903783, 82.80835186912535, 83.06665709980628, 83.25321087752027, 83.51869125349789, 83.81287221066228, 84.01377627896966, 84.28643180024396, 84.47298557795796, 84.66671450096864, 84.8891440051661, 85.03264691109995, 85.25507641529741, 85.43445504771472, 85.57795795364856, 85.79321231254933, 85.97976609026333, 86.14479443208725, 86.26677190213103, 86.41027480806487, 86.52507713281194, 86.68293032933917, 86.89100954294324, 87.0273373035804, 87.18519050010764, 87.32869340604148, 87.40762000430509, 87.50089689316209, 87.5798234914257, 87.62287436320587, 87.71615125206286, 87.80942814091985, 87.83095357680993, 87.87400444859009, 87.91705532037024, 88.01750735452393, 88.07490851689747, 88.16818540575447, 88.22558656812801, 88.29733802109493, 88.39779005524862, 88.47671665351223, 88.58434383296262, 88.64892014063285, 88.72067159359977, 88.83547391834685, 88.89287508072039, 88.89287508072039, 89.01485255076416, 89.05790342254431, 89.09377914902777, 89.18705603788477, 89.22293176436823, 89.2659826361484, 89.23728205496163, 89.27315778144508, 89.3377340891153, 89.39513525148884, 89.4310109779723, 89.50993757623593, 89.52428786682931, 89.56016359331277, 89.6103896103896, 89.6103896103896, 89.58886417449953, 89.60321446509292, 89.6103896103896, 89.68214106335654, 89.75389251632346, 89.72519193513668, 89.78259309751022, 89.75389251632346, 89.78259309751022, 89.82564396929038, 89.83281911458707, 89.86151969577384, 89.86151969577384, 89.91892085814737, 89.96197172992754, 90.00502260170768, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2028, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.85298127287078, 77.23326397359547, 77.56332065724332, 77.92207792207793, 78.13733228097868, 78.41716294754968, 78.68981846882399, 78.91942311831815, 79.19207863959245, 79.6082370668006, 79.93829375044845, 80.27552557939298, 80.54100595537061, 80.83518691253498, 81.15089330558945, 81.3804979550836, 81.5598765875009, 81.78948123699506, 82.1123627753462, 82.27021597187343, 82.4854703307742, 82.85140274090551, 83.10253282628973, 83.33931262108058, 83.43258950993759, 83.51869125349789, 83.57609241587143, 83.6047929970582, 83.64066872354165, 83.71242017650857, 83.82722250125565, 83.87027337303581, 83.94202482600272, 84.04965200545311, 84.13575374901342, 84.22903063787042, 84.30078209083734, 84.39405897969434, 84.44428499677119, 84.5303867403315, 84.6164884838918, 84.6954150821554, 84.75281624452894, 84.8102174069025, 84.8891440051661, 84.9895960393198, 85.12592379995695, 85.24790127000072, 85.34117815885772, 85.44880533830809, 85.4559804836048, 85.5492573724618, 85.61383368013202, 85.74298629547249, 85.80756260314271, 85.94389036377987, 86.02999210734018, 86.12326899619718, 86.23807132094426, 86.31699791920786, 86.41744995336155, 86.48202626103179, 86.58965344048217, 86.72598120111932, 86.8407835258664, 86.89818468823994, 86.96993614120686, 87.16366506421755, 87.27129224366794, 87.37174427782163, 87.44349573078856, 87.54394776494225, 87.65157494439264, 87.78790270502978, 87.88835473918347, 88.02468249982063, 88.18253569634786, 88.31168831168831, 88.412140345842, 88.56999354236923, 88.691971012413, 88.80677333716008, 88.98615196957739, 89.09377914902777, 89.13683002080792, 89.24445720025831, 89.40948554208224, 89.48123699504916, 89.56733873860946, 89.73236708043338, 89.84716940518047, 89.8973954222573, 89.97632202052091, 90.04089832819116, 90.13417521704815, 90.20592667001507, 90.34942957594892, 90.3637798665423, 90.23462725120184, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2028, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.72382865753032, 77.04671019588147, 77.41981775130947, 77.80727559733084, 78.10863169979191, 78.46021381932985, 78.76156992179092, 79.14185262251561, 79.47190930616345, 79.73738968214107, 80.05309607519553, 80.3544521776566, 80.73473487838129, 81.05761641673243, 81.27287077563321, 81.61727774987443, 81.86123268996197, 82.01908588648921, 82.24151539068666, 82.43524431369735, 82.61462294611465, 82.70789983497167, 82.80835186912535, 82.9446796297625, 83.05948195450958, 83.18863456985004, 83.28191145870704, 83.36083805697065, 83.48999067231112, 83.59044270646481, 83.70524503121189, 83.75547104828873, 83.81287221066228, 83.92049939011265, 84.04247686015643, 84.07835258663988, 84.17162947549687, 84.22185549257372, 84.30795723613403, 84.4299347061778, 84.48733586855134, 84.55908732151826, 84.68823993685872, 84.73129080863887, 84.83891798808926, 84.91066944105619, 84.96089545813304, 85.04699720169334, 85.15462438114372, 85.18332496233049, 85.29812728707756, 85.34117815885772, 85.39140417593455, 85.48468106479157, 85.61383368013202, 85.71428571428572, 85.80756260314271, 85.92954007318649, 85.99411638085671, 86.02281696204348, 86.14479443208725, 86.23089617564756, 86.35287364569133, 86.42462509865825, 86.60400373107555, 86.69728061993256, 86.81208294467963, 86.95558585061349, 87.01298701298703, 87.15648991892087, 87.28564253426131, 87.44349573078856, 87.57264834612901, 87.74485183324963, 87.88835473918347, 88.01033220922724, 88.16101026045777, 88.28298773050155, 88.36908947406185, 88.45519121762216, 88.55564325177585, 88.64892014063285, 88.82829877305016, 88.97897682428069, 89.11530458491785, 89.2588074908517, 89.38078496089545, 89.5386381574227, 89.60321446509292, 89.73954222573008, 89.8902202769606, 90.01219774700438, 90.11982492645475, 90.18440123412499, 90.28485326827868, 90.29202841357538, 90.39965559302576, 90.37813015713569, 90.29920355887207, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.8458061275741, 77.16868766592523, 77.59919638372678, 77.92207792207793, 78.25213460572577, 78.51043983640669, 78.876372246538, 79.25665494726269, 79.63693764798737, 79.95264404104184, 80.1750735452393, 80.4549042118103, 80.70603429719452, 80.95716438257875, 81.22264475855636, 81.51682571572074, 81.85405754466528, 82.14106335653297, 82.37784315132382, 82.58592236492788, 82.84422759560881, 83.08100739039965, 83.3249623304872, 83.63349357824497, 83.84157279184905, 84.12140345842003, 84.26490636435389, 84.44428499677119, 84.67388964626534, 84.8030422616058, 84.91784458635287, 85.20485039822057, 85.4559804836048, 85.58513309894525, 85.76451173136256, 85.97976609026333, 86.11609385090048, 86.25242161153763, 86.3743990815814, 86.54660256870201, 86.67575518404249, 86.79055750878956, 86.9340604147234, 87.06321303006386, 87.17084020951424, 87.3071679701514, 87.48654660256871, 87.61569921790917, 87.69462581617277, 87.74485183324963, 87.79507785032648, 87.83095357680993, 87.90988017507355, 88.03903279041401, 88.12513453397432, 88.21123627753462, 88.29733802109493, 88.39779005524862, 88.44801607232546, 88.49824208940231, 88.56281839707253, 88.62021955944608, 88.64174499533615, 88.70632130300639, 88.78524790127, 88.87134964483032, 88.91440051661046, 88.96462653368731, 88.96462653368731, 88.971801678984, 89.00050226017078, 89.00767740546746, 89.03637798665423, 89.07942885843438, 89.13683002080792, 89.22293176436823, 89.27315778144508, 89.251632345555, 89.28750807203846, 89.30185836263185, 89.35925952500538, 89.40231039678554, 89.45253641386238, 89.51711272153261, 89.56733873860946, 89.69649135394992, 89.74671737102676, 89.8041185334003, 89.82564396929038, 89.88304513166392, 89.86869484107054, 89.87586998636723, 89.88304513166392, 89.92609600344407, 89.94762143933414, 90.01937289230108, 90.03372318289445, 90.06959890937792, 90.05524861878453, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.88885699935423, 77.197388247112, 77.54179522135324, 77.91490277678123, 78.25213460572577, 78.53914041759346, 78.86919710124131, 79.20642893018584, 79.53648561383369, 79.88089258807491, 80.23964985290952, 80.49795508359044, 80.82801176723828, 81.04326612613906, 81.28722106622659, 81.60292745928105, 81.9186338523355, 82.14106335653297, 82.45676974958744, 82.58592236492788, 82.63614838200473, 82.70789983497167, 82.75095070675181, 82.84422759560881, 82.89445361268565, 83.01643108272943, 83.11688311688312, 83.18145942455335, 83.30343689459711, 83.37518834756405, 83.5043409629045, 83.56891727057473, 83.66936930472842, 83.79852192006888, 83.8774485183325, 83.97790055248619, 84.11422831312333, 84.11422831312333, 84.18597976609027, 84.26490636435389, 84.3366578173208, 84.41558441558442, 84.54473703092488, 84.64518906507857, 84.7025902274521, 84.8102174069025, 84.8963191504628, 85.05417234699003, 85.1761498170338, 85.29095214178088, 85.40575446652795, 85.48468106479157, 85.57795795364856, 85.75016144076918, 85.87213891081295, 85.97976609026333, 86.11609385090048, 86.25242161153763, 86.33134820980125, 86.38874937217479, 86.53942742340533, 86.63987945755902, 86.74750663700941, 86.86948410705317, 87.01298701298703, 87.15648991892087, 87.28564253426131, 87.37174427782163, 87.44349573078856, 87.5941737820191, 87.6731003802827, 87.78072755973308, 87.87400444859009, 87.97445648274378, 87.96728133744709, 88.09643395278755, 88.23276171342471, 88.29016287579825, 88.37626461935855, 88.56281839707253, 88.72784673889646, 88.77089761067663, 88.91440051661046, 89.05790342254431, 89.1440051661046, 89.2588074908517, 89.38796010619215, 89.5386381574227, 89.65344048216977, 89.78259309751022, 89.88304513166392, 89.93327114874077, 90.07677405467462, 90.21310181531177, 90.29202841357538, 90.28485326827868, 90.29202841357538, 90.38530530243239, 90.29920355887207, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78122981990386, 77.13998708473846, 77.44134318719954, 77.88620219559446, 78.2306091698357, 78.47456410992322, 78.82614622946114, 79.18490349429577, 79.44320872497669, 79.71586424625099, 80.02439549400876, 80.32575159646983, 80.59123197244745, 80.94281409198535, 81.26569563033652, 81.54552629690751, 81.88275812585205, 82.15541364712635, 82.37066800602713, 82.6648489631915, 82.93750448446582, 83.20298486044344, 83.4110640740475, 83.67654445002512, 83.83439764655235, 83.99942598837626, 84.1572791849035, 84.27925665494726, 84.50886130444142, 84.79586711630911, 84.98242089402311, 85.21202554351726, 85.3555284494511, 85.61383368013202, 85.77168687665926, 85.9080146372964, 86.04434239793356, 86.20937073975749, 86.3743990815814, 86.51790198751526, 86.66140489344909, 86.78338236349286, 86.95558585061349, 87.05603788476716, 87.17084020951424, 87.33586855133817, 87.44349573078856, 87.52959747434885, 87.63722465379924, 87.75202697854631, 87.88117959388678, 87.93140561096362, 88.03903279041401, 88.1394848245677, 88.17536055105116, 88.25428714931478, 88.35473918346847, 88.40496520054532, 88.48389179880894, 88.505417234699, 88.56999354236923, 88.59869412355602, 88.64174499533615, 88.65609528592955, 88.75654732008323, 88.78524790127, 88.8856999354237, 88.971801678984, 88.98615196957739, 89.065078567841, 89.065078567841, 89.05790342254431, 89.12965487551124, 89.16553060199469, 89.2659826361484, 89.28750807203846, 89.36643467030207, 89.45253641386238, 89.59603931979623, 89.66061562746646, 89.71084164454331, 89.78259309751022, 89.79694338810361, 89.8184688239937, 89.8041185334003, 89.84716940518047, 89.86151969577384, 89.8902202769606, 89.88304513166392, 89.87586998636723, 89.92609600344407, 89.94762143933414, 89.94044629403746, 89.96914687522423, 90.00502260170768, 89.997847456411, 90.00502260170768, 90.02654803759776, 90.05524861878453, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.82428069168401, 77.24761426418885, 77.61354667432016, 77.8575016144077, 78.24495946042907, 78.53914041759346, 78.83332137475783, 79.12750233192223, 79.44320872497669, 79.82349142570138, 80.13202267345913, 80.42620363062353, 80.78496089545814, 81.07196670732583, 81.42354882686375, 81.63162804046782, 81.89710841644543, 82.1195379206429, 82.40654373251058, 82.65049867259813, 82.91597904857574, 83.17428427925665, 83.45411494582765, 83.64066872354165, 83.82004735595896, 84.01377627896966, 84.25055607376049, 84.35100810791418, 84.45863528736457, 84.7025902274521, 84.96089545813304, 85.21202554351726, 85.39857932123127, 85.57078280835188, 85.72863600487909, 85.99411638085671, 86.12326899619718, 86.19502044916409, 86.39592451747149, 86.51790198751526, 86.62552916696563, 86.74750663700941, 86.88383439764655, 87.05603788476716, 87.17801535481094, 87.3143431154481, 87.4937217478654, 87.55829805553563, 87.65157494439264, 87.74485183324963, 87.84530386740332, 87.99598191863386, 88.08925880749085, 88.11795938867762, 88.21841142283131, 88.23993685872139, 88.34038889287508, 88.44084092702877, 88.505417234699, 88.52694267058908, 88.56281839707253, 88.61304441414939, 88.65609528592955, 88.6847958671163, 88.77807275597331, 88.84982420894023, 88.96462653368731, 88.99332711487408, 89.03637798665423, 89.065078567841, 89.11530458491785, 89.1440051661046, 89.11530458491785, 89.158355456698, 89.21575661907154, 89.32338379852193, 89.38078496089545, 89.46688670445577, 89.54581330271938, 89.63909019157639, 89.69649135394992, 89.77541795221353, 89.82564396929038, 89.86869484107054, 89.8902202769606, 89.87586998636723, 89.91174571285069, 89.904570567554, 89.904570567554, 89.94044629403746, 89.93327114874077, 89.94762143933414, 89.95479658463084, 89.98349716581761, 90.01219774700438, 90.01219774700438, 90.00502260170768, 90.04807347348785, 90.03372318289445, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.82428069168401, 77.24761426418885, 77.61354667432016, 77.8575016144077, 78.24495946042907, 78.53914041759346, 78.83332137475783, 79.12750233192223, 79.44320872497669, 79.82349142570138, 80.13202267345913, 80.42620363062353, 80.78496089545814, 81.07196670732583, 81.42354882686375, 81.63162804046782, 81.89710841644543, 82.1195379206429, 82.40654373251058, 82.65049867259813, 82.91597904857574, 83.17428427925665, 83.45411494582765, 83.64066872354165, 83.82004735595896, 84.01377627896966, 84.25055607376049, 84.35100810791418, 84.45863528736457, 84.7025902274521, 84.96089545813304, 85.21202554351726, 85.39857932123127, 85.57078280835188, 85.72863600487909, 85.99411638085671, 86.12326899619718, 86.19502044916409, 86.39592451747149, 86.51790198751526, 86.62552916696563, 86.74750663700941, 86.88383439764655, 87.05603788476716, 87.17801535481094, 87.3143431154481, 87.4937217478654, 87.55829805553563, 87.65157494439264, 87.74485183324963, 87.84530386740332, 87.99598191863386, 88.08925880749085, 88.11795938867762, 88.21841142283131, 88.23993685872139, 88.34038889287508, 88.44084092702877, 88.505417234699, 88.52694267058908, 88.55564325177585, 88.61304441414939, 88.65609528592955, 88.6847958671163, 88.77807275597331, 88.84982420894023, 88.96462653368731, 88.99332711487408, 89.03637798665423, 89.065078567841, 89.11530458491785, 89.1440051661046, 89.11530458491785, 89.158355456698, 89.21575661907154, 89.32338379852193, 89.38078496089545, 89.46688670445577, 89.54581330271938, 89.63909019157639, 89.69649135394992, 89.77541795221353, 89.82564396929038, 89.86869484107054, 89.8902202769606, 89.87586998636723, 89.91174571285069, 89.904570567554, 89.904570567554, 89.94762143933414, 89.93327114874077, 89.94762143933414, 89.95479658463084, 89.99067231111431, 90.01219774700438, 90.01219774700438, 90.00502260170768, 90.04807347348785, 90.04089832819116, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2029, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.8458061275741, 77.16151252062855, 77.53462007605654, 77.80727559733084, 78.1445074262754, 78.50326469110999, 78.876372246538, 79.2423046566693, 79.53648561383369, 79.84501686159145, 80.13202267345913, 80.46925450240367, 80.69168400660114, 81.00021525435889, 81.33744708330343, 81.61010260457775, 81.86840783525867, 82.1195379206429, 82.44959460429074, 82.63614838200473, 82.90162875798235, 83.09535768099305, 83.15993398866327, 83.2173351510368, 83.27473631341034, 83.31061203989381, 83.38236349286073, 83.46846523642104, 83.54021668938796, 83.65501901413504, 83.73394561239866, 83.84874793714573, 83.95637511659612, 84.04965200545311, 84.13575374901342, 84.20033005668365, 84.23620578316711, 84.32948267202411, 84.40123412499103, 84.46581043266126, 84.5232115950348, 84.58061275740833, 84.65953935567195, 84.74564109923226, 84.84609313338596, 84.96089545813304, 85.11874865466027, 85.20485039822057, 85.27660185118748, 85.4559804836048, 85.56360766305518, 85.67840998780225, 85.74298629547249, 85.80756260314271, 85.87213891081295, 86.0084666714501, 86.07304297912033, 86.23089617564756, 86.33134820980125, 86.4676759704384, 86.58965344048217, 86.70445576522926, 86.8479586711631, 86.96993614120686, 87.07038817536056, 87.17084020951424, 87.29281767955801, 87.42197029489847, 87.45784602138194, 87.58699863672238, 87.70897610676617, 87.80225299562316, 87.89552988448015, 87.98163162804047, 88.03903279041401, 88.11795938867762, 88.21123627753462, 88.25428714931478, 88.39061490995192, 88.5125923799957, 88.64174499533615, 88.75654732008323, 88.84982420894023, 88.98615196957739, 89.08660400373107, 89.16553060199469, 89.30185836263185, 89.38078496089545, 89.48841214034584, 89.60321446509292, 89.68931620865322, 89.83999425988375, 90.04807347348785, 90.18440123412499, 90.27050297768531, 90.24180239649853, 90.34942957594892, 90.34225443065222, 90.29920355887207, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2029, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.76687952931047, 77.05388534117816, 77.43416804190285, 77.8575016144077, 78.2306091698357, 78.53914041759346, 78.81179593886777, 79.09880175073546, 79.36428212671306, 79.6943388103609, 80.06744636578891, 80.34727703235991, 80.69885915189784, 81.01456554495229, 81.27287077563321, 81.55270144220421, 81.81818181818183, 82.1267130659396, 82.29174140776351, 82.49264547607089, 82.65049867259813, 82.84422759560881, 82.95902992035589, 83.09535768099305, 83.24603573222358, 83.36083805697065, 83.48281552701442, 83.5976178517615, 83.66219415943173, 83.76982133888211, 83.90614909951927, 84.00660113367296, 84.0568271507498, 84.13575374901342, 84.23620578316711, 84.27925665494726, 84.35818325321088, 84.45146014206787, 84.50886130444142, 84.64518906507857, 84.75999138982566, 84.85326827868265, 84.88196885986942, 84.94654516753964, 85.02547176580326, 85.15462438114372, 85.19050010762719, 85.27660185118748, 85.31247757767095, 85.42010475712134, 85.47750591949487, 85.5492573724618, 85.59230824424195, 85.67123484250556, 85.72863600487909, 85.8290880390328, 85.92236492788979, 86.00129152615341, 86.12326899619718, 86.25959675683433, 86.38874937217479, 86.51072684221856, 86.63987945755902, 86.71163091052594, 86.8336083805697, 87.01298701298703, 87.07756332065725, 87.19236564540432, 87.4004448590084, 87.5798234914257, 87.70180096146947, 87.80225299562316, 87.99598191863386, 88.1394848245677, 88.21123627753462, 88.35473918346847, 88.49824208940231, 88.62021955944608, 88.71349644830309, 88.84982420894023, 88.92157566190716, 89.0722537131377, 89.20858147377486, 89.2588074908517, 89.3520843797087, 89.45971155915908, 89.64626533687307, 89.78259309751022, 89.8041185334003, 89.94044629403746, 90.01937289230108, 90.0839491999713, 90.11264978115807, 90.1772260888283, 90.29920355887207, 90.3566047212456, 90.42835617421254, 90.42835617421254, 90.41400588361914, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75252923871709, 77.1184616488484, 77.4843940589797, 77.90055248618785, 78.2306091698357, 78.56784099878023, 78.85484681064791, 79.2351295113726, 79.52931046853699, 79.9741694769319, 80.23247470761284, 80.50513022888714, 80.79931118605153, 81.12936786969937, 81.45224940805052, 81.61727774987443, 81.88275812585205, 82.10518763004951, 82.32761713424696, 82.6648489631915, 82.90880390327904, 83.02360622802611, 83.23886058692689, 83.40388892875082, 83.67654445002512, 83.90614909951927, 84.12140345842003, 84.40123412499103, 84.63083877448518, 84.72411566334218, 84.92501973164957, 85.12592379995695, 85.3555284494511, 85.57795795364856, 85.75016144076918, 85.93671521848317, 86.06586783382365, 86.23089617564756, 86.36004879098802, 86.51072684221856, 86.68293032933917, 86.74750663700941, 86.86948410705317, 87.06321303006386, 87.15648991892087, 87.33586855133817, 87.43632058549186, 87.54394776494225, 87.65875008968932, 87.74485183324963, 87.78790270502978, 87.81660328621655, 87.90270502977687, 87.97445648274378, 88.01033220922724, 88.0533830810074, 88.10360909808423, 88.16818540575447, 88.24711200401808, 88.26146229461148, 88.26146229461148, 88.31886345698501, 88.33321374757838, 88.37626461935855, 88.39061490995192, 88.44084092702877, 88.49106694410563, 88.59869412355602, 88.65609528592955, 88.67762072181962, 88.74219702948984, 88.77089761067663, 88.76372246537993, 88.878524790127, 88.95745138839062, 88.98615196957739, 89.00767740546746, 89.065078567841, 89.12965487551124, 89.18705603788477, 89.24445720025831, 89.28033292674176, 89.41666068737892, 89.44536126856569, 89.5386381574227, 89.56733873860946, 89.57451388390615, 89.6103896103896, 89.66061562746646, 89.76106766162016, 89.78259309751022, 89.84716940518047, 89.87586998636723, 89.94044629403746, 89.96914687522423, 89.94044629403746, 89.94762143933414, 90.00502260170768, 90.04089832819116, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.80993040109063, 77.09693621295831, 77.46286862308962, 77.81445074262754, 78.08710626390184, 78.47456410992322, 78.85484681064791, 79.156202913109, 79.47190930616345, 79.74456482743776, 79.99569491282199, 80.22529956231614, 80.46925450240367, 80.82801176723828, 81.07196670732583, 81.2800459209299, 81.57422687809428, 81.82535696347851, 82.09083733945613, 82.35631771543373, 82.51417091196097, 82.58592236492788, 82.6720241084882, 82.71507498026835, 82.83705245031211, 82.90880390327904, 83.02360622802611, 83.14558369806988, 83.2173351510368, 83.29626174930043, 83.33931262108058, 83.46129009112435, 83.54021668938796, 83.67654445002512, 83.81287221066228, 83.90614909951927, 83.99942598837626, 84.1429288943101, 84.19315491138696, 84.31513238143073, 84.4371098514745, 84.58061275740833, 84.7025902274521, 84.79586711630911, 84.91066944105619, 84.9895960393198, 85.03982205639664, 85.18332496233049, 85.27660185118748, 85.39857932123127, 85.52055679127503, 85.6497094066155, 85.71428571428572, 85.74298629547249, 85.87931405610964, 85.98694123556002, 86.13044414149387, 86.26677190213103, 86.32417306450456, 86.43897538925164, 86.5609528592954, 86.60400373107555, 86.69010547463587, 86.7546817823061, 86.92688526942672, 87.02016215828371, 87.13496448303079, 87.1206141924374, 87.25694195307454, 87.37891942311832, 87.50807203845878, 87.63004950850255, 87.78072755973308, 87.92423046566694, 88.03903279041401, 88.21841142283131, 88.29733802109493, 88.44801607232546, 88.54846810647916, 88.67762072181962, 88.84982420894023, 88.87134964483032, 88.95027624309392, 89.05790342254431, 89.20140632847816, 89.2659826361484, 89.29468321733516, 89.344909234412, 89.44536126856569, 89.59603931979623, 89.70366649924661, 89.79694338810361, 89.8902202769606, 89.997847456411, 90.07677405467462, 90.18440123412499, 90.23462725120184, 90.21310181531177, 90.1700509435316, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75970438401379, 77.08258592236493, 77.42699289660617, 77.79292530673746, 78.08710626390184, 78.42433809284638, 78.79744564827438, 79.04140058836192, 79.36428212671306, 79.70151395565762, 80.00287005811867, 80.29705101528306, 80.54100595537061, 80.79931118605153, 81.12219272440268, 81.35179737389683, 81.63880318576452, 81.8397072540719, 82.14823850182967, 82.33479227954366, 82.54287149314774, 82.85140274090551, 83.11688311688312, 83.31061203989381, 83.44693980053096, 83.66936930472842, 83.84874793714573, 84.10705316782665, 84.3438329626175, 84.57343761211165, 84.8173925521992, 85.00394632991318, 85.18332496233049, 85.3555284494511, 85.52773193657171, 85.72146085958241, 86.02999210734018, 86.14479443208725, 86.2882973380211, 86.46050082514172, 86.62552916696563, 86.70445576522926, 86.8407835258664, 86.95558585061349, 87.0273373035804, 87.18519050010764, 87.33586855133817, 87.42914544019517, 87.51524718375548, 87.60134892731578, 87.6802755255794, 87.78072755973308, 87.84530386740332, 87.93140561096362, 88.01750735452393, 88.11078424338093, 88.16818540575447, 88.23993685872139, 88.34038889287508, 88.35473918346847, 88.44084092702877, 88.54846810647916, 88.65609528592955, 88.69914615770969, 88.77089761067663, 88.86417449953362, 88.92875080720385, 89.01485255076416, 89.065078567841, 89.12247973021454, 89.19423118318146, 89.28750807203846, 89.31620865322523, 89.3520843797087, 89.32338379852193, 89.37360981559877, 89.41666068737892, 89.4238358326756, 89.4238358326756, 89.50276243093923, 89.54581330271938, 89.57451388390615, 89.67496591805984, 89.73954222573008, 89.73954222573008, 89.78976824280693, 89.84716940518047, 89.811293678697, 89.85434455047715, 89.8973954222573, 89.94762143933414, 89.95479658463084, 89.99067231111431, 89.98349716581761, 90.00502260170768, 90.00502260170768, 90.02654803759776, 90.06959890937792, 90.07677405467462, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.74535409342039, 77.10411135825501, 77.42699289660617, 77.81445074262754, 78.23778431513239, 78.54631556289016, 78.90507282772477, 79.17055320370238, 79.50778503264692, 79.75173997273444, 80.01722034871206, 80.28270072468968, 80.59840711774415, 80.84236205783168, 81.11501757910598, 81.42354882686375, 81.68902920284135, 81.88993327114873, 82.13388821123628, 82.36349286073043, 82.53569634785106, 82.81552701442205, 83.13840855277319, 83.29626174930043, 83.45411494582765, 83.5976178517615, 83.85592308244242, 84.13575374901342, 84.32230752672743, 84.57343761211165, 84.83174284279256, 85.05417234699003, 85.18332496233049, 85.37705388534118, 85.51338164597834, 85.62100882542872, 85.87213891081295, 86.12326899619718, 86.30264762861448, 86.39592451747149, 86.52507713281194, 86.69010547463587, 86.89100954294324, 86.95558585061349, 87.04886273947048, 87.1206141924374, 87.22824137188778, 87.43632058549186, 87.52959747434885, 87.665925234986, 87.7735524144364, 87.88117959388678, 87.98163162804047, 87.98880677333716, 88.0533830810074, 88.12513453397432, 88.18253569634786, 88.26146229461148, 88.31886345698501, 88.39061490995192, 88.48389179880894, 88.5125923799957, 88.62021955944608, 88.74219702948984, 88.79959819186338, 88.87134964483032, 88.94310109779724, 89.04355313195093, 89.11530458491785, 89.15118031140132, 89.20140632847816, 89.27315778144508, 89.31620865322523, 89.3377340891153, 89.39513525148884, 89.38078496089545, 89.39513525148884, 89.47406184975246, 89.48123699504916, 89.531463012126, 89.54581330271938, 89.6103896103896, 89.66779077276314, 89.71801678984, 89.76106766162016, 89.78976824280693, 89.811293678697, 89.83999425988375, 89.83999425988375, 89.8902202769606, 89.904570567554, 89.98349716581761, 90.00502260170768, 90.00502260170768, 90.01937289230108, 90.02654803759776, 90.03372318289445, 90.06959890937792, 90.0839491999713, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.74535409342039, 77.10411135825501, 77.42699289660617, 77.81445074262754, 78.23778431513239, 78.54631556289016, 78.90507282772477, 79.17055320370238, 79.50778503264692, 79.75173997273444, 80.01722034871206, 80.28270072468968, 80.59840711774415, 80.84236205783168, 81.11501757910598, 81.42354882686375, 81.68902920284135, 81.88993327114873, 82.13388821123628, 82.36349286073043, 82.53569634785106, 82.81552701442205, 83.13840855277319, 83.29626174930043, 83.45411494582765, 83.5976178517615, 83.85592308244242, 84.13575374901342, 84.32230752672743, 84.57343761211165, 84.83174284279256, 85.05417234699003, 85.18332496233049, 85.37705388534118, 85.51338164597834, 85.62100882542872, 85.87213891081295, 86.12326899619718, 86.30264762861448, 86.39592451747149, 86.52507713281194, 86.69010547463587, 86.89100954294324, 86.95558585061349, 87.04886273947048, 87.1206141924374, 87.22824137188778, 87.43632058549186, 87.52959747434885, 87.665925234986, 87.7735524144364, 87.88117959388678, 87.98163162804047, 87.98880677333716, 88.0533830810074, 88.12513453397432, 88.18253569634786, 88.26146229461148, 88.31886345698501, 88.39061490995192, 88.48389179880894, 88.5125923799957, 88.62021955944608, 88.74219702948984, 88.79959819186338, 88.87134964483032, 88.94310109779724, 89.04355313195093, 89.11530458491785, 89.15118031140132, 89.20140632847816, 89.2659826361484, 89.31620865322523, 89.3377340891153, 89.39513525148884, 89.38078496089545, 89.39513525148884, 89.47406184975246, 89.48123699504916, 89.531463012126, 89.54581330271938, 89.6103896103896, 89.66779077276314, 89.71084164454331, 89.75389251632346, 89.78976824280693, 89.811293678697, 89.83999425988375, 89.83999425988375, 89.8902202769606, 89.91174571285069, 89.99067231111431, 90.00502260170768, 90.01219774700438, 90.01937289230108, 90.03372318289445, 90.03372318289445, 90.06959890937792, 90.07677405467462, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2030, "method": "three", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.78840496520056, 77.19021310181532, 77.50591949486977, 77.74269928966062, 78.15168257157208, 78.4960895458133, 78.84049652005454, 79.1490277678123, 79.45755901557007, 79.68716366506422, 80.06027122049221, 80.3544521776566, 80.66298342541437, 80.96433952787544, 81.23699504914975, 81.54552629690751, 81.78230609169836, 81.96885986941236, 82.3060916983569, 82.60744780081797, 82.91597904857574, 83.15993398866327, 83.39671378345412, 83.58326756116811, 83.76264619358543, 83.8774485183325, 83.95637511659612, 84.01377627896966, 84.09270287723328, 84.17162947549687, 84.25055607376049, 84.3366578173208, 84.40123412499103, 84.47298557795796, 84.50886130444142, 84.5375618856282, 84.55908732151826, 84.64518906507857, 84.72411566334218, 84.8963191504628, 84.97524574872641, 85.13309894525365, 85.19767525292387, 85.29812728707756, 85.38422903063787, 85.49185621008826, 85.6497094066155, 85.77886202195594, 85.92236492788979, 86.02999210734018, 86.15914472268064, 86.23807132094426, 86.24524646624094, 86.36004879098802, 86.44615053454832, 86.54660256870201, 86.60400373107555, 86.66140489344909, 86.76903207289948, 86.86230896175647, 86.92688526942672, 87.04886273947048, 87.18519050010764, 87.29281767955801, 87.44349573078856, 87.48654660256871, 87.5941737820191, 87.70897610676617, 87.81660328621655, 87.945755901557, 88.03903279041401, 88.15383511516109, 88.18971084164454, 88.36191432876515, 88.412140345842, 88.49824208940231, 88.61304441414939, 88.72067159359977, 88.85699935423692, 88.89287508072039, 89.05790342254431, 89.158355456698, 89.21575661907154, 89.3520843797087, 89.47406184975246, 89.5386381574227, 89.63191504627969, 89.76824280691685, 89.87586998636723, 89.94044629403746, 89.997847456411, 90.01937289230108, 90.04807347348785, 90.06959890937792, 90.19157637942169, 90.23462725120184, 90.25615268709191, 90.32072899476215, 90.30637870416876, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2030, "method": "gain", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.69512807634355, 77.05388534117816, 77.3982923154194, 77.72834899906724, 78.02970510152831, 78.35258663987946, 78.69699361412069, 79.04857573365861, 79.342756690823, 79.63693764798737, 79.92394345985507, 80.29705101528306, 80.55535624596398, 80.79213604075483, 81.06479156202914, 81.3733228097869, 81.72490492932482, 82.08366219415943, 82.34914257013705, 82.4782951854775, 82.5787472196312, 82.71507498026835, 82.8585778862022, 82.95902992035589, 83.07383224510296, 83.19580971514672, 83.31061203989381, 83.46129009112435, 83.56891727057473, 83.5976178517615, 83.75547104828873, 83.88462366362918, 84.02095142426634, 84.09987802252996, 84.21468034727704, 84.30078209083734, 84.39405897969434, 84.46581043266126, 84.55908732151826, 84.68823993685872, 84.76716653512234, 84.8030422616058, 84.88196885986942, 85.03264691109995, 85.11157350936357, 85.29095214178088, 85.39140417593455, 85.47033077419817, 85.55643251775848, 85.65688455191219, 85.83626318432948, 85.96541579966994, 86.08021812441702, 86.16631986797732, 86.20219559446079, 86.26677190213103, 86.40309966276818, 86.53225227810863, 86.59682858577887, 86.74033149171271, 86.81208294467963, 86.91253497883332, 86.99863672239363, 87.10626390184402, 87.2067159359977, 87.28564253426131, 87.37174427782163, 87.48654660256871, 87.60852407261247, 87.75202697854631, 87.86682930329339, 88.0533830810074, 88.16818540575447, 88.29733802109493, 88.39061490995192, 88.5125923799957, 88.58434383296262, 88.6847958671163, 88.83547391834685, 88.99332711487408, 89.04355313195093, 89.12247973021454, 89.23010690966493, 89.28033292674176, 89.35925952500538, 89.40948554208224, 89.48123699504916, 89.65344048216977, 89.73954222573008, 89.87586998636723, 89.95479658463084, 89.94762143933414, 90.02654803759776, 90.14135036234484, 90.25615268709191, 90.33507928535553, 90.38530530243239, 90.33507928535553, 90.27050297768531, 90.06242376408122]}, {"task": "nlvr2", "expert": "vilt", "endpoint": "Qwen3.8", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9973451962402239, 1.9946903924804478, 2.999210734017364, 3.9965559302575877, 4.9939011264978115, 5.998421468034728, 6.995766664274952, 7.993111860515175, 8.997632202052092, 9.994977398292315, 10.99949773982923, 11.996842936069456, 12.994188132309677, 13.998708473846596, 14.99605367008682, 15.993398866327043, 16.99791920786396, 17.995264404104184, 18.999784745641097, 19.997129941881322, 20.994475138121548, 21.99899547965846, 22.99634067589869, 23.99368587213891, 24.99820621367583, 25.99555140991605, 26.992896606156275, 27.997416947693193, 28.994762143933418, 29.99928248547033, 30.996627681710553, 31.993972877950778, 32.9984932194877, 33.99583841572792, 34.99318361196814, 35.99770395350505, 36.995049149745284, 37.999569491282195, 38.99691468752242, 39.994259883762645, 40.99878022529956, 41.99612542153979, 42.99347061778001, 43.99799095931692, 44.99533615555715, 45.999856497094065, 46.99720169333429, 47.994546889574515, 48.99906723111143, 49.99641242735166, 50.993757623591875, 51.99827796512879, 52.99562316136902, 53.992968357609236, 54.99748869914616, 55.994833895386385, 56.999354236923296, 57.99669943316352, 58.99404462940374, 59.99856497094066, 60.99591016718089, 61.993255363421106, 62.99777570495802, 63.995120901198256, 64.99964124273517, 65.9969864389754, 66.99433163521562, 67.99885197675253, 68.99619717299275, 69.99354236923297, 70.99806271076989, 71.9954079070101, 72.99992824854704, 73.99727344478725, 74.99461864102747, 75.99913898256439, 76.99648417880462, 77.99382937504484, 78.99834971658176, 79.99569491282197, 80.9930401090622, 81.99756045059912, 82.99490564683934, 83.99942598837626, 84.99677118461649, 85.99411638085671, 86.99863672239363, 87.99598191863384, 88.99332711487408, 89.997847456411, 90.99519265265121, 91.99971299418813, 92.99705819042836, 93.99440338666858, 94.9989237282055, 95.99626892444572, 96.99361412068595, 97.99813446222286, 98.99547965846308, 100.0], "score": [76.42247255506925, 76.75252923871709, 77.1112865035517, 77.41264260601277, 77.84315132381431, 78.15885771686877, 78.57501614407693, 78.92659826361485, 79.17055320370238, 79.50060988735022, 79.85936715218483, 80.11049723756906, 80.34727703235991, 80.64863313482098, 80.96433952787544, 81.22981990385306, 81.61010260457775, 81.88275812585205, 82.15541364712635, 82.34196742484036, 82.63614838200473, 82.95902992035589, 83.18145942455335, 83.39671378345412, 83.59044270646481, 83.71959532180527, 83.89179880892588, 84.16445433020019, 84.44428499677119, 84.68106479156204, 84.78151682571571, 85.03982205639664, 85.19050010762719, 85.26225156059411, 85.47750591949487, 85.59948338953865, 85.85778862021957, 85.95106550907656, 86.09456841501041, 86.2882973380211, 86.43180024395494, 86.58965344048217, 86.7546817823061, 86.82643323527302, 86.96276099591017, 87.0273373035804, 87.17084020951424, 87.23541651718448, 87.29281767955801, 87.43632058549186, 87.55112291023893, 87.63722465379924, 87.75202697854631, 87.75920212384301, 87.84530386740332, 87.97445648274378, 88.03903279041401, 88.132309679271, 88.16818540575447, 88.31168831168831, 88.412140345842, 88.49106694410563, 88.5125923799957, 88.57716868766593, 88.59151897825932, 88.62739470474277, 88.71349644830309, 88.72067159359977, 88.7924230465667, 88.83547391834685, 88.90722537131377, 88.95027624309392, 89.00767740546746, 89.065078567841, 89.10095429432447, 89.12965487551124, 89.1440051661046, 89.21575661907154, 89.20140632847816, 89.28750807203846, 89.29468321733516, 89.3520843797087, 89.40231039678554, 89.4238358326756, 89.45971155915908, 89.49558728564253, 89.48841214034584, 89.57451388390615, 89.66061562746646, 89.69649135394992, 89.74671737102676, 89.78976824280693, 89.79694338810361, 89.75389251632346, 89.78259309751022, 89.83999425988375, 89.87586998636723, 89.91174571285069, 89.97632202052091, 90.05524861878453, 90.06242376408122]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.274543238296438, 18.640376826081688, 19.14908200115018, 19.407410015644576, 19.99596597736334, 20.242340387050298, 20.505283176091762, 20.67936898549722, 20.805076131594678, 21.031416452369463, 21.111241897195075, 21.336229789985477, 22.006047034725448, 22.269943163604506, 22.796298703868427, 23.09700837798222, 23.106596352069968, 23.289142368837958, 23.506067330570417, 23.508585378721857, 23.83767020685708, 24.15054688862322, 24.550755992044472, 24.776678113290036, 25.042414379449205, 25.631101069267594, 25.776241550751354, 25.671599117847304, 26.07929715531259, 26.375721512601473, 26.09004254622811, 26.30334593009172, 26.52702690088478, 26.61817914089781, 26.646322195424528, 26.341126662605692, 26.381425184202318, 26.377503481968077, 26.50304991679535, 26.513676177334823, 26.172505837057773, 26.350480143907383, 26.41605606579513, 26.522865569075172, 26.30144389471275, 26.22936476668173, 26.206099097188734, 26.183218634074322, 26.132157707312135, 25.493202147002165, 25.390936627076705, 25.279199891750437, 25.22447722253621, 25.1911185906513, 25.51337272395705, 25.458733113077326, 25.147023023385657, 24.962813306373725, 24.854439297570554, 25.153584290539293, 24.982838097095136, 25.310384954824926, 25.347414684933696, 25.07444047215699, 25.015468294029176, 24.96158857134175, 24.813722599178224, 24.74089834401518, 24.588488266837473, 24.35703407038527, 24.269799401378346, 24.281257637673068, 24.194784767789862, 24.44152554173763, 23.89975756581927, 23.78923879564398, 23.765172675715384, 23.701272545302572, 23.545072998610646, 23.440398726966542, 23.423932820758463, 23.32750711788649, 23.330274530319098, 23.179748589558432, 23.136040291138094, 23.420510966989625, 23.340311492990143, 23.207377504714163, 23.501219578521525, 23.787439858174796, 23.741120726041125, 23.79278642063988, 23.59425095144735, 23.542011057997943, 23.40204867567031, 23.292223188569817, 23.095501777789256, 22.848460869856137, 22.6539166405275, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.500391851755616, 18.61374223941995, 19.006562390789128, 19.271289506261535, 19.825781537480754, 20.09848521962594, 20.298336443202754, 20.67165097441828, 20.777398870882386, 21.006770743866046, 21.101270122766426, 21.362508539281215, 21.846811866531947, 22.484271528973473, 22.79585181465934, 22.976792865789417, 23.101812781427757, 23.260039984240567, 23.385208645436727, 23.402062096711543, 23.784670064085695, 23.916077696650433, 24.35557828096914, 24.646441780997357, 24.986259538685214, 25.586912600609722, 25.697699002668934, 25.81908537444267, 26.12373863435594, 25.95205873486676, 26.12629607241745, 26.468836464311384, 26.809498749441225, 26.370644529648523, 26.7030355141396, 27.01340600992237, 26.92630065208758, 26.484508374874615, 26.450794014261113, 26.4793931493022, 26.6522786442623, 26.49806556083562, 25.91055449277081, 26.118399500655514, 26.101943131301486, 26.465100489025932, 26.533705883599296, 26.353885949147482, 25.292539441607726, 24.747437087187254, 24.381707247930585, 24.103221377501306, 24.212597893722865, 24.037429494456276, 23.967394412716605, 24.395726624818206, 24.229841782132176, 24.15933812827402, 24.497500964895828, 24.444429317319933, 24.491731358580417, 24.3848797443402, 24.3143898086046, 24.200272679243774, 24.00403583616227, 23.809287853997738, 23.73224130749973, 23.64813010760985, 23.532238861436966, 23.34230511867862, 23.205593244851553, 23.106599029506853, 22.957164288413228, 22.83273142167889, 22.909389866236175, 23.2015585725222, 23.329834850629386, 23.261473727218586, 23.07481109530066, 23.000525033907298, 22.93869857930069, 22.8430429838889, 22.72132747347091, 22.672441316579146, 22.639074109354162, 22.62179547768647, 22.585302017546574, 22.471422010647718, 22.432246985629632, 22.40097734390786, 22.358256226753337, 22.17736975383163, 22.14439243650343, 22.11232461485543, 22.00259691457653, 22.272060336994784, 22.23921312228618, 22.167895065663753, 22.47217201036357, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.478076821330212, 18.87777995980878, 19.09556361529419, 19.36473265090801, 19.970210306695257, 20.267439157319856, 20.505117616889066, 20.609385790705513, 20.74190398817451, 20.904651658960077, 21.206182364210633, 21.639363387952883, 22.000388796471192, 22.525947757743875, 22.774898317368493, 23.047902226916097, 23.2141805521847, 23.247894633997472, 23.442809492531435, 23.569852308094173, 23.89113077811606, 24.14295021166187, 24.620364713068994, 24.84156431621526, 25.408619805490073, 25.659317416069555, 25.781019290307217, 25.660677028194005, 26.101353646240437, 26.392367371943966, 26.10244354685662, 26.398523647386877, 26.59030668622738, 26.608197486768148, 26.810105309142067, 26.339956561521984, 26.393064878882814, 26.510103428793723, 26.559216324659552, 26.689979830860366, 26.774118368517954, 26.224662044155632, 26.343186445406857, 26.51774100453592, 25.894787858200765, 26.464897989206943, 26.478468587501318, 25.68249195426952, 25.44951076766548, 24.74622025253678, 24.51818094850203, 24.132219533625168, 24.105070497951626, 23.90363628499558, 24.310575768130953, 24.167382920233123, 23.95858145846906, 24.29525661202774, 24.343681279612962, 24.31114625956945, 24.03661350728255, 23.994655527602067, 23.830237201222143, 23.738936703205457, 23.689336899696613, 23.520992720107365, 23.428048472629033, 23.255035332555014, 23.12698448374423, 23.058169254159548, 22.906135056819384, 22.785537411095657, 23.07134740832383, 23.004066075942124, 22.931707574120118, 22.9183693186632, 22.85378936601355, 22.762241535620703, 22.63082921816197, 22.566043587357974, 22.87063363184927, 22.841323763661702, 22.74839804361305, 22.65175827888038, 22.501673713509245, 22.776874213175404, 22.718488317223425, 22.629797605365816, 22.999250791859367, 22.86528170394688, 22.823099037263837, 22.74834824886085, 22.67662252126587, 22.603790783073006, 22.56355611206329, 22.524958899961835, 22.499822706104347, 22.473372312638094, 22.39805029178677, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.280139245770282, 18.849281493873654, 19.11848207282174, 19.370473845067465, 19.994339574129505, 20.27044554600355, 20.47752324422003, 20.647966037619096, 20.819867810632246, 20.88569376842354, 21.15087463080124, 21.604666030048342, 21.959945438244407, 22.319962887677473, 22.79260726145417, 23.050808564138116, 23.100730484461394, 23.28442692573127, 23.442809492531435, 23.58327677142426, 23.815694946606587, 24.101100739856722, 24.626129730940548, 24.810566812751947, 25.411963471422226, 25.659317416069555, 25.79135680456166, 25.671599117847304, 26.090348128476293, 26.375721512601473, 26.10244354685662, 26.329196103419804, 26.549428143997766, 26.622499601475567, 26.74376437793763, 26.339956561521984, 26.36083418503804, 26.44116319923087, 26.529626037581355, 26.70371486005801, 26.73924482968893, 26.359685890546814, 26.312345193073373, 26.51774100453592, 25.88080975195145, 26.44196729732227, 26.30343118690418, 26.160486635327857, 26.10666449618163, 26.053153386375627, 25.915236489729732, 25.898081184769733, 25.282143446927147, 25.248851439155352, 25.19858907597814, 25.1232284454188, 25.450189828748087, 25.28759899393079, 25.093138036369737, 24.82632608140383, 24.69132906230442, 24.609008900424552, 24.381276487153126, 24.228092800860903, 24.059738964859353, 24.50195326816422, 24.34504898077988, 24.20734004615934, 24.10047697800265, 23.900814951670814, 23.804290819984704, 24.168043720575074, 24.174051043209477, 24.146414456988964, 24.088306661527124, 23.933960992274834, 23.69873273171577, 23.634319353060047, 23.486636268497104, 23.31628797445934, 23.254454260955683, 23.182368806735195, 23.18083121665232, 23.00297768890359, 22.845628731594356, 22.70502675475007, 22.60728723886719, 22.543349477558237, 22.878368161033084, 22.626320171419557, 22.49417523917801, 22.80944699894553, 23.08932438202462, 22.885021747423036, 22.778643402808495, 22.72281724147902, 22.65001517954417, 22.595191025434502, 22.428251854708854, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2026, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.328403701656864, 18.61741325127462, 18.95519385753109, 19.497092188647798, 19.89091650813027, 20.06644520824849, 20.113977515888983, 20.68742494978279, 20.888121876870745, 21.11545797138639, 21.25883038320885, 21.54131059073736, 22.0679870675577, 22.44238981772363, 22.800933738891597, 23.309824015077727, 23.905354962497817, 23.693211184890075, 23.869148678146978, 24.04004948252736, 24.5117362081213, 24.646483595072347, 25.28959481724808, 25.333580728589688, 25.379778092140924, 25.471280126659213, 26.176479549380865, 25.1956175237959, 25.31460707857628, 25.44558073947164, 25.77585486144774, 25.826929189210347, 25.914287074996846, 26.00966042946446, 26.18614366629909, 26.062418339012734, 26.15815395820472, 26.301199982648537, 26.47889090594161, 26.5681307031615, 26.726916593449936, 26.784506712852384, 27.077815974874795, 26.493133508581156, 26.2742103149665, 26.18930296359637, 25.631261566533446, 25.54068600152512, 25.47773436619776, 25.324998627072798, 25.222265077326245, 25.197395601473367, 25.124387184959456, 25.033754893708, 25.054736716822745, 24.969723366336027, 25.392615457988942, 25.285076361975005, 25.14896608380539, 24.944138453988216, 25.17795706361119, 24.58828651224651, 25.388102081512763, 25.282114772145732, 25.573802059005317, 25.41621025831478, 25.335360349781, 25.29334579908068, 25.174519956734997, 25.049348877079968, 24.85642910683838, 25.27485408491394, 25.157322491705408, 25.104651346825353, 24.892647238028797, 24.892505219320267, 24.892312218141825, 24.767249470857625, 24.65916485547759, 24.545612578795918, 24.42264631903706, 24.288078969812563, 24.174416076223558, 24.171983767912657, 24.14031960554125, 24.021317795831703, 23.873810285304664, 23.848963421985474, 23.790636234838015, 23.651999026272787, 23.503227699734694, 23.414884342056713, 23.369516948749506, 23.672109271406082, 23.527324899867367, 23.330446960845062, 23.30052084267949, 22.996044144653023, 22.761728478846546, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2026, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.31238862853331, 18.636911961924415, 19.094984442536035, 19.33269062938392, 19.615475604106, 20.117290465226247, 20.430222471354714, 20.530594320157444, 20.866180880991156, 20.996683484726443, 21.17832788503911, 21.505783434756353, 22.238633478006143, 22.60400175554502, 22.823295271749565, 23.362562197970536, 23.584799864155237, 23.788665288641976, 24.000186497064043, 23.610373234835965, 23.929699525825765, 24.041988487825698, 24.048837130138832, 24.252312939422456, 24.553399578934908, 25.07869627296727, 25.278959266745964, 25.361716051668896, 25.698586798378564, 25.91334023418422, 26.01589728873997, 25.86360926465317, 26.067538607964135, 26.21307746525442, 26.446897689334904, 26.608342588148716, 26.45329123302768, 26.600352270818927, 26.204473562929735, 26.25310140455774, 25.771149370867253, 25.938282553405443, 25.79360752106505, 25.86935281637382, 25.862478038742125, 25.725595444444554, 25.58707887188231, 25.510102976909742, 25.5592765634952, 25.518024210771085, 25.31148712503105, 25.258853499348305, 25.26310946934801, 25.274537039884365, 24.63798401414384, 24.59117045318255, 24.487445769653178, 25.382648243234822, 25.775527182532727, 25.643070033309456, 25.65358707605031, 25.5987397701935, 25.402804008903857, 25.305628406810605, 25.1947494541627, 25.12509260421964, 24.90693953016617, 25.252353154443114, 25.07589921587278, 24.946243528883333, 24.897215882187883, 24.708501826784943, 24.54997503657128, 24.538164557708896, 24.304562622378374, 24.196794519991208, 24.169423616550947, 23.946767340763166, 23.83654759168979, 24.1394509196521, 24.076654422117095, 23.988253687670223, 23.800885761981654, 23.80632161755693, 23.60786265007717, 23.54561992605413, 23.52686440805232, 23.361871854802924, 23.367620818778594, 23.276592705575716, 23.570999244233697, 23.46548380053955, 23.787245476115302, 23.66501755577598, 23.527128227064498, 23.25556772217632, 23.061362948614033, 22.95974738203909, 22.71854602960962, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.24533609758545, 18.46380146457598, 18.9028756095445, 19.359014037015168, 19.623255562680164, 19.971700255536444, 20.257849312687675, 20.427553841178625, 20.687753400130575, 20.948927944163742, 21.00950385444605, 21.141295927446073, 21.21157817436764, 21.34852424286196, 21.62855978562917, 21.694970070712255, 21.851994669810637, 22.132075810380233, 22.325859662436773, 22.509295402205172, 22.645511007113004, 22.552728263243914, 22.71899886838054, 23.0710955216266, 23.191673428117657, 23.10714141466508, 23.30710402559984, 23.456907981668586, 23.60701902339585, 24.521373594834518, 24.786338464553538, 24.979454724035648, 24.647440226539217, 24.752733500098895, 24.38358344163302, 24.807659106010178, 25.128341125502825, 24.992419488174395, 25.286982599099982, 25.65134287406159, 25.690423256883903, 25.699104698938914, 25.877245496322683, 25.796221492889003, 25.798631799243793, 26.44133795379678, 26.279344758541313, 25.794332917237945, 25.031515691295247, 24.576220172016313, 24.267464266114526, 24.123948185086654, 24.141947488740026, 23.978343876753154, 23.960691261228277, 23.836844770841203, 23.67206497645347, 23.60566479094214, 23.56684708668977, 23.48638981876917, 23.354896728646697, 23.617245658990566, 23.444376156590625, 23.40163016625895, 23.445358095516426, 23.32606516973662, 23.224525394611106, 23.5209486089907, 23.410352066220042, 23.353094817097816, 23.147439573337728, 22.980034777235662, 22.91139847994188, 23.27888650497537, 23.18248992651285, 22.878374378748653, 22.81081396210587, 22.818961562409285, 22.744141292719092, 22.58752211696994, 22.951956465692323, 22.8430429838889, 22.70457245899305, 22.676875791533515, 22.581962817473176, 22.862622996425028, 22.692285342730422, 22.639238209192417, 22.55388386083725, 22.522785118197394, 22.495220346877815, 22.451292166600112, 22.42672684025237, 22.397693939629402, 22.312248708195863, 22.58904970354492, 22.492935656888765, 22.422406492650257, 22.437223776200238, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.251019987058363, 18.745095544272484, 18.940430030117206, 19.123672796961998, 19.399696105119236, 19.615880991696212, 19.98731120439293, 20.512475054387398, 20.7558439881183, 20.99364498742483, 21.19456726696391, 21.437627747561297, 21.63809237746043, 22.07972717712839, 22.18965193162359, 22.30417401931253, 22.52741270308192, 22.715671359514257, 22.534649264117974, 22.78759871046756, 23.183085715085006, 23.507091398887585, 23.26196506951125, 23.496941274226224, 23.549806935967638, 23.580275868741275, 24.191869783188412, 24.196004775001644, 24.978862312505818, 24.695618265364697, 24.8403472468891, 24.843926731927617, 24.95144394763672, 25.069378298765148, 25.024405005223223, 25.958366676689298, 25.885854101027366, 26.06114484855603, 26.227715597583067, 26.28787497394453, 26.326747716183508, 26.34591848764814, 26.361141265192973, 25.686076226382543, 25.668801683188935, 25.628734331326196, 25.589539425177342, 25.500803064983092, 25.34492963897405, 25.28066604741126, 25.18397481440946, 25.153670937128243, 25.055941406875604, 25.05482646957336, 25.10637256711034, 25.023936004559822, 25.024489037597675, 24.91186506429121, 24.85996979350649, 25.291039627622574, 25.16922306946428, 25.16290285989811, 25.087271263218007, 24.99809835413934, 24.946863103438783, 24.850589081973943, 24.690369811849106, 24.620317621324382, 24.42034346037359, 24.302598020257136, 24.66420356428676, 24.561304504158237, 24.358473205449815, 24.30717245963639, 24.266665351283923, 24.71776402609294, 24.543682107251392, 24.243110023977344, 24.028921764331788, 23.816105598290996, 24.091301246171724, 24.046352429534544, 23.99137586702535, 24.292314905055047, 24.11916358148408, 24.039210845400817, 23.829605663220075, 23.728539732246148, 23.572920520687184, 23.445522786312985, 23.467285422038415, 23.362164484479244, 23.21693392042396, 23.40928701427766, 23.25887864302447, 23.284685591610543, 23.588346066220687, 23.392156208617813, 22.750783673952025, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.30134767632201, 18.711697495914056, 18.890343154054936, 19.13842779383719, 19.38536460538497, 19.512825638242987, 19.94535222954012, 20.28195725016488, 20.554425003625887, 20.901013903216096, 21.086241013500835, 21.317900870482795, 21.401246062047523, 21.631739509811993, 22.0644641454873, 22.179632589232014, 22.3511373317563, 22.609964824182214, 22.43581593239434, 22.595387254303034, 23.00091075333889, 23.237539099572707, 23.601175535515363, 23.94157783455648, 23.49778034628898, 23.548585621573427, 23.9899378256146, 24.497270135821243, 24.38490739856637, 24.61446787534576, 24.801623828902073, 24.8534618396016, 24.98382074253309, 25.125141094673225, 25.77718768746861, 25.810654444844744, 26.144902105332378, 26.136781282561337, 26.22991053999285, 26.3849806264505, 26.402554666028944, 25.874760517408912, 25.952258925110954, 25.945330858676492, 25.968557851725176, 26.029737553440523, 26.463873481364658, 26.026086822918543, 25.661562846591636, 25.078885780063587, 24.309711114085218, 24.112143419530252, 24.081430313133414, 24.0294971980611, 23.976661990422876, 23.925535696294, 23.85890310632198, 23.819377441590696, 23.747407447943747, 23.661479983383863, 23.516529536196856, 23.477443808312742, 23.904521659195176, 24.368810622297545, 24.271974235518957, 24.32225177827981, 24.197267258850825, 24.02173243550507, 23.917278356030746, 23.84571157205094, 23.70517529546757, 23.581667978715124, 23.469198983078154, 23.370768778010394, 23.35867154864042, 23.28488550226532, 23.214869206154397, 23.115147121562167, 23.350274092814484, 23.167944843843554, 23.33631723453325, 22.843468854544604, 22.810700147866566, 22.67464693578146, 22.641398054200582, 22.59645117304836, 22.57007933180017, 22.497136089898746, 22.351105730913893, 22.257527135083134, 22.212817970523453, 22.186741360252093, 22.47980240570598, 22.439028373488394, 22.3880105627249, 22.34209413536326, 22.26532285088291, 22.21751898809314, 22.484351121976843, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.21331420640158, 18.652621736088033, 18.832031820264454, 19.08996684407024, 19.28380956657901, 19.649542303215792, 20.02743676430226, 20.437703855334142, 20.763008808059357, 21.077670390340128, 21.268596270878703, 21.437371642391813, 21.71582899811564, 22.09916758405261, 22.30971126280735, 22.512399381248265, 22.66334382857894, 22.43845468191545, 22.52830576900757, 22.8251801347506, 23.137293226807294, 23.43781347947088, 23.807890997044964, 23.423763336037428, 23.54018654930266, 23.613657467874695, 24.18355173717534, 24.215487994894115, 24.986871324219717, 24.68962076032679, 24.792750663201364, 24.834444325874074, 24.957248071087896, 25.091470300337694, 25.168081653719643, 25.758848301838142, 25.71196926779134, 26.06150680642959, 26.19282981968188, 26.33948793091162, 26.353313523542127, 26.363765341967948, 26.196274533377416, 25.7452217063995, 25.823969499016574, 26.070853419796475, 26.443650259549838, 26.1210313009595, 25.596574792831138, 24.48929094515105, 24.255993116887897, 24.092439568060172, 23.903714585852434, 23.920672641680248, 24.00792984725119, 23.917888414268084, 23.678257363983942, 23.72155047662825, 23.762451129907788, 23.58565683872634, 23.440699842235762, 23.248164983984644, 23.213438704446197, 23.683612231469276, 23.636638843245695, 23.628599886564654, 23.446512117071155, 23.380732101916724, 23.34613159011779, 23.728665598305795, 23.642354462501437, 23.483574568690486, 23.40729403541535, 23.252270724932924, 23.2186144765831, 23.50728026716691, 23.290715848280033, 23.142894728868093, 22.90228766961315, 22.873724481945366, 23.053097770700678, 22.8430429838889, 22.690602379260756, 22.682328100348734, 22.60620840002229, 22.528148737154392, 22.806577043157382, 22.729493053723438, 22.62023815495471, 22.525262153929727, 22.43351332536117, 22.370584452374416, 22.343392680779132, 22.313496222867514, 22.26172368408346, 22.165356940111995, 22.137924487942772, 22.090280762383312, 22.402599850931047, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.28578577315594, 18.704077928870753, 18.881680777548095, 19.136533234370184, 19.407234650405442, 19.59075209293343, 19.993811798026986, 20.455598200016883, 20.755587549705197, 20.985990192902584, 21.28039681912009, 21.47193576138749, 21.646116924631514, 22.096182494415224, 22.235349411066192, 22.473162993565122, 22.56840105455414, 22.39091381100727, 22.50352474359722, 22.814062209718582, 23.070585463890563, 23.479942874767616, 23.783032534011696, 23.458029743956637, 23.528522283983282, 23.586854360888843, 24.189925666644914, 24.196004775001644, 24.978862312505818, 24.695618265364697, 24.8403472468891, 24.843926731927617, 24.95312554975385, 25.077536790630745, 25.05703329375329, 25.808419283722685, 25.85210987567812, 26.059544427383695, 26.202184800066224, 26.298631927306417, 26.349602249261828, 26.345867632122626, 26.338780804035377, 25.765517334583592, 25.823969499016574, 25.890193548375752, 25.822770979020977, 25.696102476985722, 25.635691860715493, 25.50814164880066, 25.447036304393748, 25.238375468713016, 25.143670761303145, 25.038326575740832, 25.044251062851785, 25.066221453416578, 24.940828670687917, 24.73834920021802, 25.219500157000834, 25.05841392872189, 24.96863980812006, 25.379647622783764, 25.274923940302045, 25.16431617811023, 24.89826108128304, 24.823552597112933, 24.78366851449878, 24.702787438788295, 24.518245457386783, 24.438381130579963, 24.275460357882768, 24.190803559966955, 24.197121485509797, 24.099836504779024, 23.937237178043098, 24.323884145767302, 24.14161394905665, 24.493915503578055, 24.345267137206513, 24.17081991415366, 24.09433936336197, 23.928411580318564, 23.82625257933949, 23.772314608452376, 23.653717635371528, 23.909702363348348, 23.721188522216547, 23.374594917647688, 23.547866588183787, 23.366744483753994, 23.312344297503373, 22.86973329312548, 22.883704629907722, 22.76867251341726, 22.725619714052257, 23.051881677762967, 22.861333823078326, 22.748947002861094, 22.609489167553647, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2027, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.323454229427824, 18.60600431180022, 18.88014240660772, 19.437585759031332, 19.74072889668689, 20.163546447799696, 20.395899988770513, 20.523667198187486, 20.810898394014632, 21.045021901838545, 21.18843214784502, 21.340624852410457, 21.576106465192364, 21.95036977552969, 22.583861326957557, 22.795634405325675, 22.754992453268315, 22.936874574497573, 23.101541361460168, 23.423430199004226, 23.542181168949185, 23.812666278163956, 24.094568752670305, 24.45453044400694, 24.329196056763916, 24.544912152353213, 24.62199364818329, 24.996373800934492, 25.03935714779412, 25.069230303071144, 25.2131878280642, 25.361601616331487, 25.724820713951647, 25.75007536277602, 25.411545067116304, 25.719876821836852, 26.30634176347666, 26.41443707882664, 26.49909102542557, 26.57151340622263, 26.629633690989436, 26.7801900925609, 27.041637330788827, 27.041464228371247, 27.084153904310153, 27.014926868613028, 26.971252627735225, 26.867685660615038, 26.904195133342036, 26.871128617688182, 26.341407722454914, 25.743956335188788, 25.664269005196594, 25.546046179859182, 25.976932994926283, 25.91502222142091, 25.72780158384967, 25.53660950839527, 25.378913448682066, 25.412673519487317, 24.89890836781716, 24.94527863308624, 24.8835210052616, 24.70873572813335, 24.72643162658782, 24.64369046739117, 24.866362431155675, 24.75527800695177, 25.542796452753176, 25.427514260639718, 25.34812828851599, 25.239006432771365, 25.11558717915099, 25.022328178036634, 25.43858353317384, 25.10671788404807, 24.986352557418186, 24.89471933649782, 24.839258254838153, 25.038460980809813, 24.895916545616746, 25.19836983472089, 25.014326329253283, 24.98079888775116, 24.85715391688477, 24.603326418722613, 24.497113746031072, 24.405460654674755, 24.30640034249045, 24.23463342365138, 24.20640069659478, 24.053522695682858, 23.854978164553334, 23.68261635124135, 23.150084253807655, 23.154668576079708, 23.06090335819846, 22.993908202529422, 22.74580418685616, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2027, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.319504793369905, 18.836038338055314, 19.059199207314425, 19.36508524193622, 19.86733266235225, 20.008746882911694, 20.287134573227473, 20.403345507053025, 20.776723825781087, 20.979391798950925, 21.249087951212488, 21.499162900059616, 21.76079497438654, 22.151146538675423, 22.381043987194694, 22.616687776188833, 23.225731873307375, 23.512045472126488, 23.36067218038665, 23.43680321550697, 23.34072644728412, 23.41987005004806, 23.32023478718786, 22.7675690128328, 22.996742888002483, 23.4586793497408, 23.527247693026457, 23.593361456110255, 23.817113396669075, 23.94482743966399, 24.13788398446883, 24.45292157846655, 24.69258674246067, 24.811874175147707, 25.190453364480064, 25.37677334045837, 25.71194694210962, 26.1025124586733, 26.117704281954683, 26.346121779747715, 26.043593846716064, 25.77573822488319, 25.831989423484757, 26.041499535422602, 25.927656537026177, 25.79442492145117, 25.736627846475464, 25.582565947103337, 25.522924792669897, 24.95641473732226, 24.859152567928952, 24.81284505243451, 25.19730741423008, 25.583832280946655, 25.54150892702816, 25.436716799610526, 25.43793078835123, 25.345486107665383, 25.24272307749989, 25.692790721593074, 25.5849063679776, 25.878967857381312, 25.84797853302033, 25.66582149413324, 25.703238421639018, 25.567667766787395, 25.890472926520047, 25.705941266573788, 25.478424553088985, 25.427807504802015, 25.32290007894968, 25.13613241547042, 25.10000753575643, 24.997994585299114, 24.88586272707783, 24.80088647247664, 24.71711938803595, 24.55046383641842, 24.364336087679217, 24.17991927887165, 24.133544969873594, 23.928001572206654, 23.890981941613482, 23.773929377889743, 23.49420499870262, 23.43780344520152, 23.383503205070042, 23.213610150158033, 23.491556529120242, 23.327143623455836, 23.107658733693242, 23.048858803548175, 23.40100358461109, 23.278881970618272, 23.18423815159448, 23.11091714813399, 23.052657857337877, 23.05142732654336, 22.853812077535256, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.07742442808907, 18.47696742447116, 18.665505464837803, 19.179192492824267, 19.486677373686156, 19.9565521346937, 20.178291746432247, 20.32220896728335, 20.47997012748532, 20.805026521484944, 21.140025163448904, 21.57494257562579, 21.926222589387386, 22.30185796393701, 22.69103380746608, 23.177874489419715, 23.376029541111063, 23.483179781622777, 23.07004701249192, 23.779596274543184, 23.77190798356207, 23.88631653414639, 23.93755516328332, 24.125695727324818, 24.28913905923211, 24.4628015081329, 24.647081829855207, 24.71403929373385, 24.759331591814586, 24.807214367722853, 25.02336800348408, 25.61486331807632, 25.935639194965976, 25.78041104441331, 25.895747866898976, 26.216582086248692, 26.340629709967807, 26.643820937482303, 26.873966674488216, 26.927872121667367, 26.9127498921318, 26.394708843390756, 26.78673334704383, 26.373540959724274, 25.92334893321632, 25.938872251923563, 26.36042663742812, 25.615413547998696, 25.34664138482599, 24.851821220798946, 24.332363073640924, 24.176830747333106, 24.05101925847236, 24.00524187411621, 23.999243637633732, 23.739376499216903, 23.658499158720105, 23.568510678706737, 23.944363524162927, 23.761645492094548, 23.67540592684903, 23.70312513430895, 23.539740281597084, 23.942250905250475, 23.75260373774846, 23.689100011365262, 23.600067904439335, 23.90058529421445, 23.827736892048136, 23.640980500574727, 23.543213830666645, 23.24882625647771, 23.17255688404235, 23.168963756645937, 23.07807142805145, 22.911234697740174, 22.78204728771164, 22.657578897379445, 22.98352016457662, 22.93674921278282, 22.94513012348386, 22.841323763661702, 23.03861398908317, 22.96164444283982, 22.91343982875458, 22.84030505606784, 22.808431161528713, 22.7272505730636, 22.63005710178103, 22.584151053742225, 22.47304402026808, 22.404461193175955, 22.313023911184203, 22.60035752437646, 22.547290030877214, 22.51519638114086, 22.4666358432461, 22.42454315820695, 22.385759651089625, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.320594624578483, 18.6151912086114, 18.916960534730176, 19.490753273810686, 20.016595534311676, 20.323483962417853, 20.50222401974213, 20.681914174787554, 20.85359457947883, 21.079588466862763, 21.24242277982101, 21.42686496670629, 21.69204129144996, 21.90999203267529, 22.687870296060648, 23.062427519059085, 23.50372634631576, 23.52806406400906, 23.344721621734728, 24.084099133476222, 24.29222939774071, 24.553274833597662, 24.75551228672431, 25.08836782187075, 25.237883923455946, 25.334553591265692, 25.43868590133979, 25.633029701738046, 25.882963179082054, 26.199477862402205, 26.502750681959856, 26.793489706171382, 26.881318884715228, 27.112509361593208, 27.392362350345543, 27.72310059882791, 27.29960371878144, 27.309104315153377, 27.356259729884364, 27.34390243704609, 27.42157921593022, 27.552602552033566, 26.610302507592053, 26.484012830297438, 26.506902885355466, 26.298788651168625, 26.173082088702603, 26.06140350148527, 26.059183524609765, 26.002297441980378, 25.91241764001198, 25.73881782587597, 26.137022344878197, 26.588009225776133, 26.02626000362884, 25.946985778896586, 26.359710628920634, 26.310697945421417, 26.107185169333775, 25.84044682589836, 25.808181965950496, 25.63704237733741, 25.448467152106407, 25.32783457500884, 25.09576789592232, 25.13327366077548, 25.037771931703052, 25.31647124503048, 25.29470547233958, 25.0641897468313, 24.87304247388897, 24.914589089969517, 24.773040737841733, 25.14354378737314, 24.929291247506978, 24.841015476183998, 24.723683587988447, 24.490904354928617, 24.369608816990677, 24.34059703813845, 24.25521163244115, 24.080484778969268, 24.270887719839497, 24.21868008948546, 24.286187716740752, 24.54103561799806, 24.386600119996938, 24.362432665888853, 24.224675668603815, 24.08582935121056, 24.064909783135477, 23.977530327229548, 23.74083587078853, 23.53443469671662, 23.486589921081215, 23.361226103078266, 22.809091116255402, 22.555267223025933, 22.569423360350445, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.52441211738528, 18.577331444450408, 19.042363756036625, 19.149179423531624, 19.624598158215708, 19.988181103143837, 20.395267092817328, 20.645759862808053, 20.724430225253325, 21.025384351866954, 21.152176078239346, 21.33166527677559, 21.573361478958045, 22.074516300870325, 22.353215526988933, 22.61769824575812, 22.973802009401485, 23.089619242691846, 23.545821353734386, 23.525679134702163, 24.182985002286912, 24.13453795633009, 24.39210784806735, 24.41655356987862, 24.632426795852492, 25.183121894260502, 25.380679256873623, 25.353473020372437, 25.46548725052294, 25.695831108644203, 25.918004793288823, 26.06933957505138, 26.64738151622497, 26.712672699645175, 27.233997045691854, 27.594757433038687, 27.27412914487314, 27.36826364638803, 27.453067366860473, 26.97980680746429, 26.508901042506068, 26.49116201149409, 26.461694073252794, 26.66989121041058, 26.641831530224007, 26.4916166503111, 26.24671097269421, 26.136887603121995, 25.116234198486847, 24.665775049508976, 24.264030565083416, 24.17328344692951, 24.138724985542552, 24.06432738016252, 24.083161809583668, 24.01781515561636, 23.887557075730076, 23.777437658558632, 23.67402769691021, 23.46216702204485, 23.855945178928632, 24.466133973221062, 24.369552320943257, 24.328213755820176, 24.18847047807161, 24.128036120923575, 24.086845812817682, 24.029092766012404, 24.01660841807801, 23.84937978456273, 23.7437270316709, 23.770024999096254, 23.70219969603213, 23.52141566073172, 23.390854626329375, 23.31537141621839, 23.118480976806516, 23.11191243231321, 23.304712338525064, 23.175097557942273, 23.22537358945126, 22.892212029100143, 22.785569049626172, 22.758599145706324, 22.67491558021057, 22.63922783853754, 22.542785942610482, 22.517844535669333, 22.485847568885564, 22.38292556134618, 22.287399661812714, 22.234114840352877, 22.180321122709852, 22.14796879704193, 22.065560860485625, 22.005212804619767, 21.97381816114763, 21.950958629646617, 22.49640833897146, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.456305741513486, 18.621752709149998, 18.888525284100158, 19.42199202508682, 20.08391794175031, 20.349935268145032, 20.593918572863814, 20.708750639147734, 20.890084251281866, 21.055927694465733, 21.235136288059014, 21.55369049449188, 22.131125720432763, 22.47133546968657, 22.601142037230247, 22.99029480207976, 23.141162369107025, 23.44737329748332, 23.984185238035526, 24.297928319058105, 24.127226326900004, 24.26191028576414, 24.558127627774226, 24.848517803986862, 25.138097295092308, 25.314014007783648, 25.32617477575766, 25.375514305724735, 25.8259978136588, 26.07359768905153, 26.435106473232977, 26.832282497089555, 26.787701580282274, 27.070243938549837, 27.398935025568065, 27.634807145755385, 27.289455450686102, 27.319737454016995, 27.362733334487245, 27.459756532597, 26.94423027437623, 27.0911968424064, 26.62250744073979, 26.592814598294034, 26.45681093800789, 26.504369453402326, 26.249672529757902, 25.450986441684186, 25.19580289784053, 24.783116245404575, 24.268293170605578, 24.145117152759386, 24.16250563377585, 24.067287202493507, 23.958069578326917, 23.834347761418904, 23.76544703120031, 23.706449521957545, 23.644906713762165, 23.489894774228404, 23.47255619941018, 23.339832847440054, 23.7329796335493, 23.593918142511665, 24.047100639740886, 23.852527900608738, 23.926243098795265, 23.91320906163717, 23.89969239284308, 23.830996092340474, 23.727884732159822, 23.557754454861488, 23.445860180684665, 23.36291901167519, 23.27336996236721, 23.173310621859947, 22.893213426641445, 22.790543487946785, 22.482989513338218, 22.62219153041747, 22.983508186654795, 22.843902876586505, 22.78897292559261, 22.73219076566638, 22.65575754401757, 22.596329330264158, 22.554016727486218, 22.341352652809388, 22.26197126021404, 22.21325216679261, 22.099313894040513, 22.43906809894061, 22.410503514313916, 22.370267892977036, 22.320502691928123, 22.326216920890833, 22.204348082132892, 22.115386142516513, 22.383084599107534, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.456305741513486, 18.621752709149998, 18.888525284100158, 19.42199202508682, 20.08391794175031, 20.349935268145032, 20.593918572863814, 20.708750639147734, 20.890084251281866, 21.055927694465733, 21.235136288059014, 21.55369049449188, 22.131125720432763, 22.47133546968657, 22.601142037230247, 22.99029480207976, 23.141162369107025, 23.44737329748332, 23.984185238035526, 24.297928319058105, 24.127226326900004, 24.26191028576414, 24.558127627774226, 24.848517803986862, 25.138097295092308, 25.314014007783648, 25.32617477575766, 25.375514305724735, 25.8259978136588, 26.07359768905153, 26.435106473232977, 26.832282497089555, 26.787701580282274, 27.070243938549837, 27.398935025568065, 27.634807145755385, 27.289455450686102, 27.319737454016995, 27.362733334487245, 27.459756532597, 26.94423027437623, 27.0911968424064, 26.62250744073979, 26.592814598294034, 26.45681093800789, 26.504369453402326, 26.249672529757902, 25.450986441684186, 25.19580289784053, 24.783116245404575, 24.268293170605578, 24.145117152759386, 24.16250563377585, 24.067287202493507, 23.958069578326917, 23.834347761418904, 23.76544703120031, 23.706449521957545, 23.644906713762165, 23.489894774228404, 23.47255619941018, 23.339832847440054, 23.7329796335493, 23.593918142511665, 24.047100639740886, 23.852527900608738, 23.926243098795265, 23.91320906163717, 23.89969239284308, 23.830996092340474, 23.727884732159822, 23.557754454861488, 23.445860180684665, 23.36291901167519, 23.27336996236721, 23.173310621859947, 22.893213426641445, 22.790543487946785, 22.482989513338218, 22.62219153041747, 22.983508186654795, 22.843902876586505, 22.78897292559261, 22.73219076566638, 22.65575754401757, 22.596329330264158, 22.554016727486218, 22.341352652809388, 22.26049898112601, 22.21325216679261, 22.099313894040513, 22.43906809894061, 22.41183658357258, 22.370267892977036, 22.320502691928123, 22.326216920890833, 22.206834583980008, 22.115862595483108, 22.383084599107534, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2028, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.312171701133913, 18.538994500924275, 19.04426238095787, 19.53008441141537, 19.683640744774863, 19.89948786344796, 20.42223091433683, 20.641907146403057, 20.8386189290467, 21.03836625834406, 21.089077201193483, 21.29982892686549, 21.828844646398625, 22.189795512997776, 22.538298433453708, 22.91473393437877, 23.13357594985878, 23.438356505972163, 23.848607157996643, 23.873142607994858, 24.21989208933868, 24.019533833488914, 24.16102649157016, 24.221610897620486, 24.78505246322881, 24.5020124365184, 24.63261725817807, 25.127484970082094, 25.230081450191584, 25.859824987872077, 26.124662236518763, 26.21001793657501, 26.25551679307772, 26.551626782362263, 26.773262869344773, 26.94731937570812, 26.40076126507226, 26.06515064750406, 26.08104432859641, 26.226501245515543, 26.181877608107516, 26.096715104595592, 26.256126923036767, 26.45893319208783, 25.840034777420463, 25.673713755088894, 25.580530997953844, 25.528015744952427, 25.467727129069868, 25.41147573012591, 25.39961441760289, 25.35606726279887, 25.216629388492677, 25.607747245451527, 25.442570355243472, 25.241514780972587, 25.174834313998556, 25.0036557317131, 24.72615481453993, 24.600982691171943, 24.47607624626537, 24.759812729380585, 24.184581186601864, 23.969918102849466, 23.899076861588316, 23.800154465874915, 24.123655276154587, 24.01134528601931, 24.461884835910688, 24.34955868378198, 24.11007775789003, 24.320886898906764, 24.35967652851965, 24.26495093290775, 24.151238453277845, 24.079609417474835, 24.04767178857245, 24.06605851312129, 24.04698010621239, 23.88636200550441, 23.915495426267935, 23.871919912718344, 24.13018637121763, 24.00985623206461, 23.96967748759596, 23.85667593669114, 23.878696312280457, 24.21300707608135, 24.1543515732525, 24.100704213512596, 24.054232963523177, 23.9800948870047, 23.990497936858088, 23.89101009347032, 23.82616768495402, 23.517800159040497, 23.453141101222776, 23.043700089795056, 22.80287162801588, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2028, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.31494875016155, 18.64041855489351, 18.85413389491527, 19.293828449110546, 19.618010657045488, 19.895143789278976, 20.093699711583827, 20.439586218832233, 20.728049755157908, 20.935911920682322, 21.249415298848614, 21.77025681335342, 22.098915577176452, 22.473684897660775, 22.872875096094532, 23.22092431103361, 24.02392883810545, 24.27220943706316, 24.274688468822394, 24.609874618917964, 24.777854604898174, 24.76794886475168, 24.361637098000934, 24.820759590185677, 25.15492712571104, 25.321229276508127, 25.511323257427353, 25.567359254587252, 25.82752947825305, 25.984058982164083, 26.269769109855833, 26.47092714165742, 26.554211124949433, 26.662135988889084, 26.683479986122055, 26.734551791274875, 26.436474738972336, 26.11785023324401, 26.358704835201973, 26.503369025104284, 26.22322111082439, 26.28564033829454, 26.625574522619676, 26.533526085622917, 26.333454917560424, 25.71608881312318, 25.63973290260379, 25.593041316081028, 25.54519666167098, 25.493412064278115, 25.435136609502734, 25.406624858750153, 25.376301492732615, 25.345022971363818, 25.13698536638101, 24.951520106672227, 24.861763195729015, 24.19397277745882, 24.497669263615887, 24.348691915879765, 24.233348634558315, 24.082185499842545, 23.93513517731169, 24.409867849938582, 24.429530207675796, 24.28730318011523, 24.19669618705135, 24.165960993766497, 24.438165876447513, 24.457591105234393, 24.354988327738187, 24.20438270747338, 23.991282003170973, 23.934415756476046, 24.35807324110688, 24.19062464931745, 24.585015217289772, 24.88750440221825, 24.852617494156252, 24.692743231964908, 24.556912472097142, 24.447525583794963, 24.387161043481818, 24.331330167349112, 24.299157866537428, 24.14678963924159, 24.11435955707039, 24.09074804722378, 23.902432199144734, 23.63195058297891, 23.453689588314912, 23.317162347426525, 23.0974397282426, 22.986803483678486, 22.839465732219665, 22.751696100947704, 22.738503052588882, 22.658079330003936, 22.835908711240776, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.476111245964123, 18.724602209341132, 18.78534237748321, 19.50740685024917, 19.791193433813483, 20.12845621713724, 20.35426963045388, 20.72020972436817, 21.016567627304248, 21.379059979835276, 21.80652366500598, 22.139957984691428, 22.54220521174371, 22.78634639576813, 23.03835009286691, 23.729497401264272, 23.835859527078792, 23.982979825650418, 24.062370485326458, 24.23214916257409, 24.69769916128693, 24.790111108553255, 25.05049298044191, 25.094628504831846, 25.10398247864421, 25.187538480228316, 25.39193574438718, 25.63757994696003, 25.993607581895517, 26.2828124003796, 26.47241856555646, 26.88897902890306, 27.086349906208056, 27.20584867414258, 26.97288239251871, 27.041615559872533, 26.931521875099286, 27.24519152591653, 27.380214441430862, 27.512752608778985, 27.649766962358708, 26.993234676675048, 27.063361189609488, 26.453932976972776, 25.8983279318343, 26.3887650810264, 25.89610991436532, 25.547022517290056, 25.353445109664086, 24.80661518692875, 24.27088532087266, 24.182764662084494, 24.16958197011589, 23.99094418988695, 23.982626523589857, 23.868540248209726, 23.698333984288766, 23.652268510617397, 23.597677989915127, 23.445552299537017, 23.283385027456948, 23.057731721954468, 23.09683338915276, 23.52901364786968, 23.339559839629718, 23.677530842970963, 23.572128204677057, 23.405464373271975, 23.303290544105323, 23.148930552272645, 23.074764051123225, 23.00876348827647, 22.88021162612936, 23.224310924470988, 23.073028573261652, 22.858451095002614, 22.79869600441037, 22.738292597901193, 23.18078397830818, 23.108302661365936, 22.900783467928584, 22.8430429838889, 22.759346493031764, 22.961706668874008, 22.805662711264088, 22.706312660052227, 22.63101551276854, 22.657886084104053, 22.584974150299374, 22.457607876177875, 22.748972697378182, 22.70792982084713, 22.61003326805147, 22.57744266937544, 22.531129633678727, 22.48052912805733, 22.453241887362417, 22.428917471868765, 22.40618386010611, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.298906847637568, 18.63094998628884, 19.018345655299843, 19.21486795000681, 19.794177158450964, 20.258757847971744, 20.45460439593852, 20.579021475287774, 20.81549215858112, 21.047521327537446, 21.189843405038207, 21.39299321908018, 21.593178270403328, 21.709632025401532, 21.828732949389114, 21.56500246082955, 22.125039477525732, 23.002532637201536, 23.653652178657183, 24.127054305673102, 24.653670518823837, 24.74210399059597, 24.796707318380086, 24.852312118838302, 25.14737242357959, 25.42272862050699, 25.053963268409436, 25.2872634894309, 25.707657496698506, 26.127990816451952, 26.34886021001966, 26.350725557760146, 26.529167518496465, 26.70409469453559, 25.763152946254348, 25.727711672076886, 25.963054972220156, 25.99737860188104, 26.281221856355636, 26.24012394434412, 26.224335042152624, 25.61042320427206, 25.626178039506165, 25.634009548149393, 25.611738351415713, 25.590766929920765, 25.569839710796614, 25.421873323006245, 25.255611495452623, 25.110994912540026, 25.483360953884667, 25.25171266777954, 25.16915671929149, 25.094263574660015, 25.06758287441514, 24.829662322857406, 25.07534783042491, 25.040404364501583, 24.950140609959625, 25.228414826451445, 25.054331423075926, 24.92235309597076, 24.7622205737389, 24.765377445244212, 24.21580157744952, 24.11731253328292, 24.460026779285368, 24.328772246029917, 24.290761444997845, 24.16349485220711, 23.988553989951573, 23.939271057456565, 24.330407873515142, 24.28193339884932, 24.221304107629223, 24.190752717296395, 24.22305769989587, 24.136341233462698, 24.066720673114318, 23.95321765742491, 23.91886962947004, 23.857102064082156, 24.183833827138677, 24.04652336448267, 24.011445976974784, 23.926074789009476, 23.881795238565047, 24.12517580409471, 24.24163385216741, 24.13957223294995, 24.054124754173635, 23.993645476667545, 23.995388955090107, 23.81874341274799, 23.778580981024756, 23.56010330910167, 23.255149804766027, 23.119263439042495, 22.766943655930913, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.522523423667206, 18.743984979312827, 19.053874117336793, 19.462178831061447, 19.68424053895151, 20.000446597405926, 20.47228583369601, 20.643676515812082, 20.811649487820365, 20.925871603489675, 21.042214475418515, 21.2585450743243, 21.386050911592708, 21.4224438831268, 21.646018788151085, 22.13447983139397, 22.32190486750912, 22.63183166526365, 23.181590446397536, 23.826062268273322, 24.25122706700486, 24.625844379366885, 24.79122707974659, 24.935349962833442, 25.200206252902568, 25.317980537215284, 25.43369023743663, 25.036321084146536, 25.288690447939683, 25.67332131081586, 26.050484591900062, 26.22145098307157, 26.45067632491361, 26.693561252382647, 25.763152946254348, 25.822990563186504, 26.05322782630174, 26.16052061851401, 26.31388172525353, 26.2089162389931, 26.341869888270352, 25.849789943504426, 25.85884924769274, 25.883184170702062, 26.018920948836353, 26.518906606940853, 26.374496813893554, 25.738120877270692, 25.082775054402646, 24.753529910323643, 24.381707247930585, 24.694089078417434, 24.650274525491, 24.62040016670707, 24.483427643928284, 24.44263231971509, 24.358260280466386, 24.243800292514607, 24.27476564254764, 24.158239477368724, 24.08924714766577, 23.994392096503766, 23.953047101012526, 23.867361376717906, 23.76278867734072, 24.140097907680317, 24.157703255972056, 24.01659683871938, 24.016820424438663, 23.896172516621178, 23.73292775287637, 23.650324088774425, 23.590637807156497, 23.508513902030597, 23.34839560986916, 23.74148756522249, 23.586652296037443, 23.36259685221721, 23.52868912502661, 23.387538314884697, 23.054689622339417, 22.843468854544604, 22.774315450565354, 22.738392316999395, 22.64391470501837, 22.562398167219307, 22.46923742233477, 22.43275677875246, 22.40087357977331, 22.24496806241661, 22.167157467135137, 22.165412498720514, 22.50498100711731, 22.474019424681927, 22.398804439909263, 22.328810328731784, 22.284241436026527, 22.239169376477726, 22.44902931252603, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.48593746558699, 18.604974055232944, 18.894201714642016, 19.351037638038264, 20.0288956496461, 20.277839911277965, 20.447749072770968, 20.640364382370873, 20.75771245998314, 20.991566119606418, 21.24937073525792, 21.36590293780272, 21.432948317891995, 21.654709714260456, 22.23205123609507, 22.475136249161018, 23.031661592313895, 23.327811474501353, 23.789433714518065, 24.20041667395839, 24.484052042149145, 24.583861912633242, 24.980745451729387, 25.139772416589484, 25.18371067876018, 25.41818084024409, 25.48397187498707, 25.356555623998823, 25.803593150999355, 26.128772339190544, 26.390508447391603, 26.365220359205438, 26.540759327544766, 26.68868421469534, 25.763152946254348, 25.725266318143667, 25.915376561408372, 26.044731512330493, 26.140459098628472, 26.195014738191613, 26.212900548096147, 26.188736335499062, 26.20663574722142, 25.781087383571833, 25.93710817395028, 26.46750664390019, 25.950300806057967, 25.785562205252653, 25.20324396099879, 24.7014870667545, 24.378259139927728, 24.050780908693753, 23.882099464977426, 23.78762762094358, 23.69758921923456, 23.653830193655427, 24.0642736483656, 23.90108403802865, 23.800161024939207, 23.638335762907204, 23.36328372793119, 23.276098311368813, 23.02049925255917, 22.99982015362458, 23.308688068655233, 23.227162728998763, 23.07425178226393, 23.032787984972376, 23.002523246267803, 22.931446735155063, 22.862099840021422, 22.74024825665597, 22.68776622402568, 22.532915711287618, 22.51746048930055, 22.8282820957528, 22.82109970821368, 22.711362448017812, 22.64846659921085, 22.906721771421914, 22.862354483872487, 22.8430429838889, 22.771768693157306, 22.615926985225716, 22.515809198232688, 22.464635917694746, 22.78422273670129, 22.6596523289441, 22.54538850714983, 22.88367386008462, 22.852782966981803, 22.85920902827906, 22.827054435370815, 22.801665240978352, 22.72374136508214, 22.634686372100262, 22.515009540684204, 22.489801670176586, 22.447159588882567, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.28126641345032, 18.62883459112026, 18.89628833535866, 19.19785089539956, 20.045582004954365, 20.29771205473379, 20.414148535162365, 20.661720998813106, 20.784355733295246, 20.988423856844907, 21.2134636297776, 21.50379929285465, 21.55059121262177, 21.587243797508698, 22.221777497033735, 22.448592874573333, 22.89873072547624, 23.334280519369443, 23.60160158506329, 24.13621994685205, 24.478761558119363, 24.58446571349797, 24.883822098334736, 25.076181771836513, 25.223713646995467, 25.427586052057816, 25.4254930859915, 25.28957971597272, 25.791743806449684, 26.139860361704443, 26.390508447391603, 26.366381353973768, 26.525602456107872, 26.68868421469534, 25.763152946254348, 25.725266318143667, 25.934727372894017, 26.04358731552759, 26.15275961605193, 26.255105016348917, 26.273145631567, 26.192245476812182, 25.670174834807995, 25.761683646289757, 25.920681345209644, 26.472188179922117, 26.449386043820244, 25.71390350416468, 25.61682448417117, 25.313606877000655, 25.25585480081798, 24.987918892022865, 24.94529590374381, 25.195555096268883, 25.170184317743665, 25.105630396103695, 25.158307956475127, 25.040869260099193, 24.899090977524846, 24.661904228114178, 24.437408664340083, 24.73346429427331, 24.679677321894154, 24.530765243208243, 24.374114459610297, 24.226268903945638, 24.03450298656188, 24.353683267878388, 24.288360279918606, 24.221913898046846, 24.144757967535654, 23.97835044672191, 23.86837413719975, 23.85067356233797, 23.784865020876804, 23.759612685165, 23.720189940564094, 23.621838316740362, 24.016745591201584, 23.94068813410294, 23.801084749312576, 23.749143799259578, 23.641112955030884, 23.510059286845028, 23.697009062855305, 23.63883412233617, 23.506199108284083, 23.38335953044062, 23.695535731679858, 23.53833265228707, 23.484446888389567, 23.382136583523682, 23.36560436835817, 23.196846150135375, 23.111122321559588, 22.963274070624205, 22.795761488855515, 22.74374792009879, 22.568387097993163, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2029, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.287624160002775, 18.61610061156217, 19.179681987591994, 19.706223763182322, 20.037277046339877, 20.216445048823402, 20.53022093495399, 20.663531024124417, 20.83610143625087, 21.001716073494077, 21.338480392156864, 21.812607109714357, 22.22220685382225, 22.483005680354346, 22.67238229704726, 23.066735510520203, 23.31823911915842, 23.425177258742284, 23.81655202008466, 23.942132877645246, 24.117729677678007, 24.133226994989222, 24.877266422992072, 24.95115930701402, 25.59748948465184, 25.939729419828534, 26.16462704661999, 26.49401575579476, 26.15606645862992, 26.300263712777927, 26.40777520512345, 26.600441195224082, 26.591750728428753, 26.829124401239785, 26.839022677290973, 27.06931759875588, 27.14299694821995, 27.34701906833334, 27.10896172130701, 27.20709757249866, 26.703652772508285, 26.382447474332572, 26.30189522020729, 25.869088500415664, 25.792709746009994, 25.7505753112348, 25.74142083843557, 25.641482194733996, 25.607633282289854, 25.422893837030152, 25.29603547405491, 25.24577504936455, 25.09928535353823, 25.1309539337253, 25.094282943068368, 24.42694629899957, 24.26078888453845, 24.0729306678872, 23.8573752734522, 23.715549015241717, 24.57136437839115, 24.48965860025969, 24.505897449128287, 24.405690789186636, 24.22727095555809, 24.094342066861778, 24.3430984039363, 24.275644251676677, 24.242486145888183, 24.077640373443682, 23.95508943367067, 23.79326734604766, 24.10504925060638, 24.10286750302865, 24.021565069491196, 23.994527159077027, 23.99606382856593, 23.948640253147634, 23.781106165486094, 23.58422584882924, 23.593095934471933, 23.904345993079033, 23.840678493379293, 23.77244617858198, 24.00273093394336, 23.821217567838456, 23.71938071090637, 23.51309884016252, 23.563570472174725, 23.50091044966653, 23.347383957932408, 22.91880777843549, 22.77900189267868, 22.861798339490804, 22.837339474912284, 23.12111500129849, 23.046130006739112, 22.94518573353504, 22.883059800021517, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2029, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.31569007222758, 18.815630455187776, 19.051191021440797, 19.2290969380484, 19.50706013836541, 19.73874744431206, 20.11861399526188, 20.426319837560683, 20.657721723840126, 20.852121502991196, 21.117571592051952, 21.381211276368322, 21.485242267782187, 21.23824412872304, 21.38509931668801, 21.483748597642812, 21.864137937063703, 22.153568866041475, 22.303494417060364, 22.39648578730215, 22.762056900276, 23.00256140926264, 23.324704647651238, 23.78983961428097, 24.17330756658982, 24.363531519919647, 24.620418177170876, 25.242679290824523, 25.416617539780773, 25.04868687477318, 25.28608271015665, 24.781421383645245, 25.366194529869624, 25.594017611710836, 25.265350164199013, 25.210290555983722, 25.471170980632152, 25.53721164396137, 25.790088464507072, 26.053720606689428, 26.264584295071447, 25.836850808149947, 25.75141210141718, 25.586435237675637, 25.57069776956323, 25.652165250194624, 25.646633744902374, 25.57410365494426, 25.594664338594903, 25.518710685211477, 25.37830239439337, 25.378810522629376, 25.247648549768552, 25.096496294498316, 25.07266727844297, 25.069240558797297, 24.953485615683643, 24.887469028624164, 24.30088816095791, 24.73691127133035, 25.186497218002803, 25.577066943684756, 25.534040596652403, 25.478540009472162, 25.36136255083166, 25.158725542190886, 25.158152325318326, 25.096841014579557, 24.920123308273883, 24.77329233827564, 24.699400940742457, 24.581159369180654, 24.44051147805784, 24.2932265068045, 24.25296099817153, 24.563471684587295, 24.441030857284158, 24.190449108632194, 24.092007581278278, 23.75446562015441, 23.638976629919426, 23.383184922718502, 23.28356094845091, 23.21724180327828, 23.179941325293072, 22.979830417403612, 23.249502351793222, 22.941673211273375, 22.783140376328387, 22.57875976312837, 22.38639972267431, 22.23059936559701, 22.013122872684768, 22.04278647529222, 22.160693246758925, 22.500578974043385, 22.610591897645218, 22.63615216864864, 22.48364176791097, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.456925992862335, 18.636417922154468, 18.93200913250784, 19.073339719397346, 19.66753815023552, 20.03796777902278, 20.24311563597126, 20.42482423047144, 20.613915000942068, 21.013712677355308, 21.309438436925653, 21.747559722636918, 22.154520390451633, 22.473083986566404, 22.99127881535393, 23.341126318858393, 23.754085735371945, 23.969836032518884, 24.31004748313895, 24.405111701243495, 24.56740980258044, 24.838082488684897, 24.988515994778993, 25.01460507867269, 25.134729788646958, 25.859868650832425, 25.883801465807863, 26.21279113211625, 26.424552282675855, 26.12755290265284, 26.146814994224954, 25.91310579925381, 25.496116274504843, 25.561327975855512, 25.723096088453563, 26.05042413891488, 26.13628671249126, 26.11046562283838, 26.159665396262422, 26.32407192362204, 26.379466106411336, 25.64411492310481, 25.682114981365604, 25.892696267106214, 25.914711253083013, 26.3826775488749, 25.99128168781007, 25.591925850644405, 25.32088883178843, 24.897706233733985, 24.173526977262615, 24.084970881556966, 23.990416782022887, 23.831674177848498, 23.630771242863624, 23.465050805736915, 23.33021474870798, 23.266729202026177, 23.1209093228464, 22.99045133388286, 23.339087973714104, 23.241026745624016, 23.261098793514744, 23.14832598723623, 23.147358681339092, 23.070499127822654, 23.12148122193369, 22.967048605430097, 22.805382180366333, 22.628197001945065, 22.51788464560742, 22.419609020981156, 22.246741979525886, 22.16975727166411, 22.035325241207595, 21.919312430926137, 21.841008002610398, 22.58886361244659, 22.54568192247791, 22.461467370143847, 22.834824728050865, 22.8430429838889, 22.82682113474902, 22.65736145340424, 22.599193182407472, 22.546200732253734, 22.85551373767265, 22.821783529566783, 22.79555889646037, 22.76971591180572, 22.737684664770388, 22.64409787709266, 22.563919036099946, 22.512637024927713, 22.420990006892406, 22.30206937514849, 22.229712107412574, 22.46966228941141, 22.35794701053036, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.301936903738063, 18.569186157296826, 19.025824371766305, 19.59118337115855, 19.879631281693563, 19.982551232754805, 20.13499797965635, 20.583589041567585, 20.832132744331744, 20.974765280895046, 21.287170635347792, 21.38605605164589, 21.508244885420222, 21.851441118454257, 22.216141791096085, 22.407846124526635, 23.417077906522465, 23.2557172929376, 23.05694307340169, 23.07949251848774, 23.333167034071778, 23.70545105652944, 23.83355290505721, 23.908916448335763, 24.260336253427067, 24.370886566008515, 24.474770943595132, 24.095566872379354, 24.238253200989725, 24.475083162371806, 24.619567104286745, 24.654976777174433, 25.010217255556455, 25.26196717990807, 25.292718091629453, 25.822793208114625, 26.23724936390488, 26.39584286967577, 25.9680114090001, 26.110245787813867, 26.075056948577952, 26.303220657781672, 25.617588926884405, 25.851261088294503, 25.840077072486867, 25.756700267059262, 25.701954572758012, 25.642971989572384, 25.532166866729956, 25.511098185287683, 25.3123516134034, 25.212472036156296, 25.1666954141071, 25.10790737824873, 25.03340673442821, 25.38977947856732, 25.317697676725142, 25.195365669064174, 25.17824698615838, 25.23122589816886, 25.616987179487182, 25.591713304839192, 25.459542657192756, 25.335903581712277, 25.23594342423782, 24.98609355186094, 24.805777381379286, 24.212944795774987, 24.01149455397398, 23.821941112970563, 23.599920415544464, 23.495018249760765, 23.44482791779354, 23.30514572049459, 23.6087447056697, 23.520164489830904, 23.34659682222359, 23.680381838218807, 23.66977281646678, 23.442870825341192, 23.597462193681846, 23.43202062081073, 23.35167000031305, 23.22470926822863, 23.08480112200032, 23.089073046746304, 23.019910064361866, 23.19064285532434, 23.02941053167535, 22.953227605733574, 22.806698302645405, 22.821810803694955, 22.748930480594446, 23.05217483438253, 22.906705329299704, 22.758551405228104, 22.622443415612516, 22.530520622090673, 22.5163583134802, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.47019441487749, 18.947501056143246, 19.293255219208984, 19.451852368842626, 19.765438367433337, 19.885881608103833, 20.147081342027004, 20.4628200289411, 20.652967498759317, 20.98560036301108, 21.019391202486954, 21.21227241252229, 21.37909334704956, 21.6618996486218, 21.977668325751644, 22.2564879697914, 22.379328862413047, 23.556064289395646, 23.492390600723468, 23.927773964359485, 23.896185141580396, 23.475549911701393, 23.597497814483656, 23.803353430350754, 24.207255642739515, 24.234365162130068, 24.320678076294445, 24.520776010325726, 24.259923011038218, 24.428100378853117, 24.562269578679473, 24.660393504026953, 25.044269140132503, 25.28623832891716, 25.42846634246818, 26.13492544905403, 26.578011658353667, 26.00492385081331, 26.12185371705521, 26.27119928627966, 25.860508309423068, 25.792801878674947, 25.86881880496894, 25.893134958196434, 25.969072909379648, 26.507512872063817, 25.76250431142517, 25.577768241836946, 25.043067119887773, 24.70984898706286, 24.257736207969284, 24.101438707837115, 23.9546541688688, 23.82839176016023, 23.741994906559523, 23.651418878735765, 23.557576646942785, 23.966048573151568, 23.966726314304545, 23.888482362059857, 23.829729458872002, 23.81618332640846, 23.733120875560296, 23.61453515129282, 23.690110723584006, 23.528186667780417, 23.392438431676993, 23.165399558344635, 22.990469222492308, 22.72184520090448, 22.586014055288818, 22.44867200538518, 22.39552456239238, 22.29712260970241, 22.20690435427251, 22.04624507036752, 21.98961877551196, 22.30532693475592, 22.69041021088758, 22.51512943934229, 22.437238674047173, 22.843468854544604, 22.818741847274136, 22.772166493480935, 22.69670383213856, 22.656146185909947, 22.627150962861535, 22.600667762151016, 22.86263169394444, 22.82856857297717, 22.791528851004404, 22.728547031766887, 22.635547653087627, 22.557763408464275, 22.529755444221077, 22.36589339763433, 22.298608302929686, 22.187617465422907, 22.142614107696204, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.483194754954912, 18.608099118916517, 19.068484434163704, 19.565566786941528, 19.787087918302014, 20.016145857614827, 20.33037140578586, 20.555027578740255, 20.77885587866109, 21.034668369816682, 21.203149129830866, 21.531282631649564, 21.744254427969764, 21.910829211248057, 22.212501292802383, 22.648510584165532, 22.981123860050616, 23.196535193991156, 23.509499085374884, 23.606976130989835, 23.746102736285064, 23.421838381685685, 23.73283914201823, 24.207491854550675, 24.204733206968804, 24.296859145610647, 24.4666353933268, 24.599888840767477, 24.244003999711772, 24.440818550676426, 24.59862804930969, 24.654976777174433, 25.010217255556455, 25.332485823529804, 25.330109302367738, 25.879270033455114, 26.223025322814486, 26.401097355590387, 25.9828885899542, 25.936445078887104, 26.139716212168036, 26.260958448494975, 25.732454679823103, 25.727406992056217, 25.8983279318343, 26.50097092665313, 25.816777773340547, 25.54150722375685, 25.052131090914674, 24.72569962569234, 24.252684495294318, 24.024943660491036, 23.866169162221794, 23.74499803156047, 23.610618513200425, 23.442891840889043, 23.337281576456213, 23.170385253961864, 23.052338074826476, 22.96220509563476, 22.776610176082407, 22.838928482298147, 22.82152214607561, 23.181151541815222, 23.217647082790542, 23.113934263552945, 23.0159246910874, 22.92526709732362, 22.780516147276394, 22.57242501974163, 22.479700957343947, 22.33020652470577, 22.17839442138301, 22.176270187047532, 22.60772741882012, 22.481158305817484, 22.379404701789138, 22.20660806701885, 22.58675920642706, 22.51659972061462, 22.446922649800182, 22.843902876586505, 22.78723666392128, 22.740922640226326, 22.674561586967172, 22.641451943482895, 22.988147315232464, 22.95919834696467, 22.889500953299102, 22.740895092357892, 22.664193606494063, 22.591152239728125, 22.494333298636686, 22.446600933865206, 22.394796032637984, 22.370788412476987, 22.280316931232573, 22.099197634630926, 22.474852623387946, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.32043510007531, 18.583852392992465, 19.023425185030412, 19.623575268347064, 19.870757185381738, 20.007997820240423, 20.126981947592224, 20.583781083195575, 20.743830319217864, 21.000140038180696, 21.261441603243593, 21.47028995284713, 21.69703517545612, 21.879735301159865, 22.22004638080076, 22.64860801485431, 23.44206952050089, 23.207276733858958, 23.496925320844053, 23.122264405188023, 23.2855299175478, 23.541460915947475, 23.667331393696582, 24.16374961617757, 24.23816613708872, 24.296599389171185, 24.48270754701302, 24.104989447622252, 24.238253200989725, 24.475083162371806, 24.59862804930969, 24.654976777174433, 25.010217255556455, 25.333295831697413, 25.333540056110227, 25.766103865711, 26.27899989665628, 26.3482240667275, 25.925787492123053, 25.936445078887104, 26.32521136711287, 26.21652033415437, 25.691639956560042, 25.790424234310134, 25.944457398252048, 26.460004686535676, 26.178987875217462, 25.97201572569383, 25.458894862495423, 25.40010557881582, 25.218084451260115, 25.01991289433389, 24.84471672020048, 24.776536861952465, 24.641512362763713, 24.64728296398687, 24.532131561633946, 24.912063813510578, 24.800109520750695, 24.6930277774646, 24.619000523989165, 24.497667419847627, 24.497175067417714, 24.44705854995187, 24.470235860436027, 24.318327029645484, 24.267138350343384, 23.972677924485154, 23.797574814593744, 23.699880573282286, 23.50851177757335, 23.213343819450184, 23.11197497493478, 23.036258970918492, 22.964485031576118, 22.80961351482742, 22.745383981665896, 22.55768349764083, 22.45308085670704, 23.17222905008906, 22.951293376734377, 22.748345518856087, 22.915625036049438, 22.86618740029434, 22.67416279896552, 22.6515661754838, 22.525436221237676, 22.463210959442343, 22.41735655440065, 22.367026878756754, 22.494375679817463, 22.62423246064408, 22.547192701458886, 22.407305156305256, 22.592987337682494, 22.485529992048633, 22.43717780847561, 22.413031330169293, 22.368315272454957, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2030, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.400352740067795, 18.719325693283707, 19.22396751427928, 19.584081822067937, 19.800523738192876, 19.897753313073235, 20.1747311827957, 20.507307034808626, 20.842283531129134, 21.020244928763496, 21.351762158968118, 21.970839266352932, 22.49389808000052, 22.882996124536813, 23.3160751548177, 23.521458664361734, 23.61513919026665, 23.98476996338896, 24.296680800888375, 24.409175486386303, 24.407451060695774, 24.626506985587515, 24.883298243172906, 24.854550746604723, 24.906537517390767, 25.43379591329392, 26.045388801953706, 26.024410035286227, 26.07339665921048, 26.17743409172132, 26.111581065879403, 26.263494459587434, 26.405859652091817, 26.924285076517563, 26.113489560499076, 25.628836864527955, 25.742075580518353, 25.848310910071714, 25.939018976987743, 26.061241996035655, 26.31396490663345, 26.273711688650618, 25.77120295022256, 25.68686768088273, 25.523288080640043, 25.473984295861403, 25.45918030726928, 25.45135144880942, 25.38362996556375, 25.327691787576995, 25.296152906389413, 25.163573760907216, 25.086939450549767, 25.051841186780006, 24.931851008885193, 24.993741225143772, 24.871689784164044, 24.794745286526837, 24.825265170357195, 24.72635090713915, 25.082772034726368, 24.942627041716033, 24.864741952200113, 24.87738881136676, 24.726647065149336, 25.55165155032136, 25.52105936990464, 25.474147258404216, 25.51582458549956, 25.37800238038107, 25.272917693851475, 24.75728683795971, 25.16924760313335, 25.058576849646624, 25.205535629315396, 25.113182348399892, 24.86083400161957, 25.237474799977345, 25.471723607904355, 25.294698145506167, 25.18146285455681, 24.92672363128078, 24.868328029689703, 24.791620737519388, 24.631983283178137, 24.563580163816244, 24.46102537357087, 24.302643367919792, 24.244689544076394, 24.033521734285095, 23.91800423109755, 23.71906149350713, 23.65509794901258, 23.61496545814865, 23.49335727435137, 23.321659977363705, 23.055067642078214, 22.943110311198115, 22.780495950604397, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2030, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.322343830126144, 18.615122569303278, 19.0505620961508, 19.31138385501213, 19.489646154870872, 19.980347358919463, 20.278943604357018, 20.496752507508056, 20.701467437913518, 20.827281538168187, 21.022969056552313, 21.382254138655824, 21.699165498267565, 22.347480259566133, 22.72474334156781, 23.12080249991183, 23.285036487283676, 23.40267940382357, 23.70221738809678, 23.82870413644792, 24.223232531475944, 24.3002678475191, 24.259555290568727, 24.375543523316423, 24.616494144561628, 24.791557051310328, 25.352874944159936, 25.590041591480272, 25.71398712135483, 26.407396153008634, 26.417273643890262, 26.074514177473308, 25.681710848353166, 25.689235744801735, 25.92031358962718, 25.582527681665535, 25.78916099616594, 25.85117612352511, 26.108107290426286, 26.121976336913406, 26.205885170569886, 26.36951324672852, 26.370144188821815, 26.301767258891996, 26.158195253022843, 25.560341399727232, 25.5506097513783, 25.505469268521097, 25.458124211698458, 25.421800939021217, 25.38701971772269, 25.388786738356966, 25.304340387887937, 25.115222606243258, 25.1149675204889, 25.062373285360596, 25.074829725907698, 24.972625843552255, 24.847959999982983, 24.786925919203533, 25.179141245087667, 25.128173971434148, 25.527414799038752, 25.490209659276513, 25.384638631588142, 25.277041042280747, 25.26242836985169, 25.69968395635292, 25.56431594777404, 25.476674439115882, 25.256621708285344, 25.263983185957706, 25.106839785483526, 24.53818324299091, 24.440204087837724, 24.291382170961413, 24.19289848807428, 24.16450221120571, 24.105250734985198, 23.95196325707653, 23.957716212791357, 24.16072829957611, 24.085104520542444, 24.01000104508876, 23.966749976001424, 24.233110303679048, 24.504910686548044, 24.43833392094053, 24.219916954582672, 24.05734853118592, 24.302703466409916, 24.173073946964163, 24.158778454781984, 23.953877215988925, 23.72314837553992, 23.717195899623082, 23.60705464130047, 23.231964130003316, 22.882788444647204, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "MiniCPM", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.424227964102705, 18.725345049248237, 19.01975558987866, 19.105857161017582, 19.59576163395894, 19.881852545454482, 20.397198322283355, 20.598232342447847, 20.87965036270146, 21.059975834391118, 21.430019643247384, 21.674820274381464, 21.869213750276543, 22.1418342413156, 22.30613430712084, 22.587591597384524, 23.08049155179518, 23.30204162087836, 23.845672048552537, 24.0155406607517, 24.256349404543013, 24.472236496148547, 24.521107195117622, 24.580945246645516, 24.806486006121485, 25.266550694948393, 25.553115816498455, 25.667016392874114, 25.733070359186762, 26.004441425851514, 25.725820228148216, 26.04377186072205, 26.101525486951903, 25.725678907032073, 26.197758850805958, 26.345183138204682, 26.11734921689673, 26.19562021965192, 26.482398294629935, 26.212859177406855, 26.176358993518615, 26.23366288101875, 26.280924571393143, 26.224008645140977, 25.919459571976066, 25.979533936512066, 25.941875133676152, 25.471589761582347, 25.285103181299913, 24.83516758617311, 24.254288099966427, 24.103919951151237, 24.103389218467832, 23.992446553660088, 24.013936426221804, 23.891174439292335, 23.888370066893238, 23.659692342024872, 23.56948190289077, 23.484141516681596, 23.41019306140132, 23.355075892854742, 23.791864641928846, 23.665039529568606, 23.455566189830797, 23.34595771724197, 23.199340051393598, 23.011105683514195, 22.85236639664841, 22.763452419207805, 23.121575760683704, 23.932032058843298, 23.793533335113676, 23.60718221898854, 23.42021413844457, 23.381312278074315, 23.245800117401632, 23.18996940119191, 23.0182987262569, 22.97256123291122, 22.885168485384728, 22.841323763661702, 22.703969216132993, 23.023437558441344, 22.912500809356647, 22.87780776326713, 22.80520872611532, 22.73873827435607, 22.6838113162625, 22.57189161212563, 22.475330653926832, 22.39183460634986, 22.670673870519177, 22.645986270815737, 22.62168378794972, 22.515892230716737, 22.500413894106785, 22.439122889107562, 22.407927006427386, 22.361115727328066]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.338699267186577, 18.527187211232558, 18.871744804758922, 19.213766537443693, 19.33898121335364, 19.668170325979677, 19.84985133713403, 19.85904065145679, 20.088988630436454, 20.301169407706176, 20.620966897407307, 20.91645645760922, 21.313907622678673, 21.62647731236938, 22.25055850687484, 22.827112369574, 23.052030091229906, 23.290692743927995, 23.43367067599932, 23.64878300418357, 23.768597514131606, 23.944157487284663, 24.137381116736936, 24.3561250405254, 24.802045076435324, 25.169020466854526, 25.361673532635624, 25.53046149467334, 25.853053951037985, 25.883913649222567, 26.078135644546883, 26.68482839640353, 26.78006743992808, 26.8335345635652, 27.502266062198494, 28.08369434091388, 28.164590885782133, 28.62287153053442, 28.843576062245273, 29.554185319182142, 29.944337777944018, 29.995319700460833, 30.157454714782904, 30.382496604563197, 29.98263974088433, 29.902475496756175, 29.878623379880803, 29.7699363478052, 29.75089968511021, 29.668245066187175, 29.611349488398673, 29.499580994998574, 29.51854918048963, 29.397227954115333, 29.30421255104282, 29.14670433047391, 29.387196398146187, 29.31056856890147, 29.229770726659034, 29.20443709806341, 29.48466603248606, 29.358986932475982, 29.283346793189313, 28.940118417658216, 28.934728638121122, 28.880930409486893, 28.796754030396766, 28.67962691610503, 28.518994771009275, 28.67979619993037, 29.023509850251468, 29.027143607618694, 29.17203292901848, 29.179260138508273, 29.02784065651544, 29.06203347214583, 28.952195714954705, 28.977665463120143, 29.044005290137953, 29.50219199996783, 29.590717828630243, 29.32545757060418, 29.181605006435102, 28.995871741281988, 29.29262662193698, 29.046186002700765, 28.913942325819658, 28.91173778920495, 28.749643539159543, 28.570674212373337, 28.348220424637915, 28.255482309436125, 28.043746014740755, 27.922722696174684, 27.857426787850493, 27.733967263129838, 27.44719643069776, 27.51209921789381, 27.275179726887703, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.584052194177175, 18.674604662760547, 19.024519816336966, 19.427841605614002, 19.758652385603785, 19.94290075582252, 20.269741765615738, 20.62952378560638, 20.925289082545966, 21.072878080628477, 20.600359730229766, 20.757071874459488, 21.01483702946001, 21.32611661665049, 21.544517356291017, 21.805292393545717, 22.0781732631752, 22.34013327242702, 22.558706424962537, 22.913391225364258, 23.09979539190953, 23.329857117472503, 23.56110807182852, 23.808405884188897, 23.972108836729777, 24.5233072048998, 24.998773387182705, 25.19518527138142, 25.819082299450347, 26.051277541837447, 26.858747093306278, 27.074288445248364, 27.199916601249218, 27.384615384615387, 27.61765436500983, 28.151979843366636, 28.23692402306075, 28.47886717117448, 28.7294198050012, 29.010230744385673, 29.453728461081408, 29.95651604444134, 30.2920858850977, 30.130286380286382, 29.98263974088433, 29.895252215945423, 29.896668597519938, 29.877463532155158, 29.82539809757065, 29.65272926259509, 29.860608986835363, 29.735834674779007, 29.60584584101415, 29.511939626974076, 29.436424478134498, 29.379155147446024, 29.53353020523745, 29.47870198732498, 29.53599614545586, 29.769244406360617, 29.832667608585677, 29.747659082885125, 29.680934119873086, 29.558570848699194, 29.408269122536566, 29.35056606430165, 29.136842932502997, 29.28905098945544, 29.034875836810812, 29.080626522464875, 29.188252606415627, 28.985003758410073, 28.861125184654597, 28.935385904491334, 29.01831655880675, 28.91510921480569, 28.824688654618726, 28.54302107352394, 28.25891300873821, 28.09664198042659, 27.792026493367278, 27.84367961397069, 27.711781111087515, 27.725671019760128, 27.664942352030796, 27.629808458936715, 27.605954017448354, 27.570588509506187, 27.489917233315076, 27.416702679565724, 27.351276174805587, 27.30094327938783, 27.2486181099194, 27.352362838050766, 27.252342798963287, 27.193781206681905, 27.12647581176894, 27.127002945545094, 27.260838692188976, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.59039698993343, 18.931688849882853, 18.979762153598667, 19.09916504711257, 19.322544641061988, 19.662535331481436, 19.774676199017243, 20.1138461045091, 20.451704517434084, 20.690447484547043, 20.94527132532559, 21.14716692051248, 21.289464296322024, 21.66350438289804, 21.866646436188923, 22.0396453365502, 22.34146463374706, 22.633215237908836, 22.782737148774228, 22.66539513406147, 23.085885002386824, 23.265832031032726, 23.518627756900468, 23.716123614822298, 24.704086889629494, 25.023645508720133, 25.22587056779541, 25.423577994332707, 25.6783811786332, 25.90339796821815, 25.924020555265017, 26.542533289946846, 26.89684461789237, 26.83896156576412, 27.446216253685847, 27.61452886007982, 27.86187227700069, 28.197242041910688, 28.609848887536828, 29.09534321820111, 29.806421228785258, 30.126356420461082, 30.141458429687688, 30.349971849266527, 29.98263974088433, 29.935025466550396, 29.981144311615182, 29.979805326905527, 29.87907986767316, 30.09007620665647, 29.874882555597654, 29.67053883488953, 29.586506529207142, 29.446900729202536, 29.436424478134498, 29.361862424940462, 29.285954293386908, 29.549583890060386, 29.84284224162939, 29.753449992886615, 29.648084147542498, 29.690304162001674, 29.49763677204148, 29.337177537755966, 29.175922027470335, 29.008837618910334, 29.133832576997854, 28.888470035303616, 28.667988673828205, 28.741128982307533, 28.654879187082273, 28.82479491470049, 28.91974161539956, 28.756895096259193, 28.587149152774593, 28.37810184179122, 28.17769141970934, 28.196035893233752, 28.08314689561737, 27.92601775062986, 27.73521369154927, 27.795211566550094, 27.706173322822313, 27.645791326127405, 27.621903837781197, 27.563632228887236, 27.539927345719406, 27.49950925936382, 27.407799951202577, 27.39326973341827, 27.557379054960652, 27.506637277618427, 27.46447101007663, 27.415631452552034, 27.315333005841865, 27.192692809098578, 27.09571689380404, 26.97136831271906, 27.170826112727255, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.441092378980578, 18.58016011713009, 18.911756220564367, 18.96641705999187, 19.189441588075585, 19.517183764442812, 19.82335406378871, 20.06954509202395, 20.41330416741251, 21.012881027080493, 21.298214601057, 21.614659926293232, 21.844084587728656, 22.074295376824146, 21.85916013983491, 22.107595062933573, 22.402149641371246, 22.322186425804176, 22.592065599120954, 22.831055300515338, 23.18685581188112, 23.676207048662683, 24.054385406480975, 24.3171712561297, 24.686330564634172, 24.71375574054778, 25.243725073572023, 25.340280529910913, 25.58919952882801, 25.824533048259045, 26.087233151246547, 26.480658607492195, 26.917418330958377, 27.0500618059537, 27.23200467392409, 27.278202163454445, 28.003193416527527, 28.308293364243976, 28.767064832302157, 29.53994925891369, 29.853873693472295, 30.11939515163906, 30.2514432675565, 30.35811430318645, 29.98263974088433, 29.875993589336698, 29.836480195635488, 29.796953497804836, 30.225258485105535, 30.116092465102735, 30.069245261964717, 30.069245261964717, 29.95817675144782, 29.88525230762732, 29.75955946605, 29.61492730951353, 29.56339556431047, 29.499292403918737, 29.373784578410916, 29.664086468122253, 29.53948676595847, 29.467781869279573, 29.356845499375734, 29.13994605164969, 29.071873501825863, 29.08073914507866, 29.045507201449773, 28.895181046343836, 28.84903113438537, 28.748725937652296, 28.987450662196334, 28.907504257409744, 29.00713956937811, 29.002020508221683, 29.323940975767986, 29.31434571033817, 29.055733636418967, 28.87956673351401, 28.967722112384553, 29.08001341179015, 28.947535532960554, 28.825828765782802, 28.560897628849585, 29.29313710222587, 29.13026976906668, 29.146329466554178, 29.162702146818088, 28.925801375597615, 28.783461136993072, 28.53205715643349, 28.37835980599628, 28.141687978593854, 28.104127018876856, 27.85744706701676, 27.738287738287738, 27.638803281413317, 27.42849466196493, 27.29642977190519, 27.2612699463711, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2026, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.436440107756873, 18.29911275340626, 18.876936722498666, 19.1973572281709, 19.27728588661349, 19.241243322128497, 19.454614600818175, 19.74374378745049, 20.087425581986267, 20.384497249548385, 20.58766324403843, 20.81153762424457, 21.09451479409151, 21.34805975183354, 21.713207526567277, 22.403069164116136, 23.08875702889123, 23.34282687909691, 23.956918252265798, 24.172962057495475, 24.20362819454954, 24.438269890740717, 24.746921909259573, 25.04124245198398, 25.304722823307056, 25.5641192040532, 25.667458065821258, 25.916330720840726, 26.051866884425184, 26.55268512228189, 26.809489375107397, 27.28935188512561, 27.369184817099868, 27.485815370450243, 27.60129747092681, 27.902485460043984, 28.933604520574725, 29.178009090195616, 29.392866131832175, 29.511078278819504, 29.671469588113098, 29.971463911886826, 30.30208189770489, 30.347925661695086, 29.91172860782436, 29.92379385647298, 29.905437702818748, 29.905437702818748, 29.780614336999946, 29.70252652330042, 29.593233135024178, 29.552261789863653, 29.87930653477849, 29.789818433037617, 29.713270621765176, 29.9746983796986, 29.955993790659356, 29.85052704418297, 29.750780871704784, 29.576584403168205, 29.48990301228275, 29.522209948280093, 29.443431706711152, 29.01139073668581, 28.87705427587711, 28.85718672620824, 29.171508205148562, 29.04360316276014, 28.978341887217056, 29.026997541141014, 28.821683412095673, 28.80637675439757, 28.848417340897875, 28.744083300656325, 28.90834165990965, 29.037012505095884, 28.90548586718863, 28.69823444130264, 28.940509998951846, 28.803032012747646, 28.737821856027203, 28.98624138853494, 28.873081108947186, 28.94376442713315, 28.847614313004755, 28.654534509911823, 28.470674680288788, 28.40161088290633, 28.729866550863477, 28.499687589163496, 28.390947593235026, 28.132305653663327, 27.95522934223434, 27.864996618327787, 27.677454613144626, 27.604659030253465, 27.619889406335, 27.320744287316295, 27.179231442335205, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2026, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.486987401864123, 18.865637514577738, 19.01398973936423, 19.376434973350936, 19.58728834452437, 20.01915567665067, 19.98523060860176, 19.68007013365568, 20.027582466100522, 20.307505686076883, 20.704477888276326, 20.94172495676921, 21.19135344322007, 21.51237472213491, 21.709971524305285, 21.915978292178146, 22.198810409400792, 22.468215502321723, 22.818454027259666, 23.1017053802403, 23.977345748204463, 24.155099059071844, 24.361363636363638, 24.630154536875175, 24.869028695515595, 25.1260650534088, 25.39142718441298, 26.307243447314022, 26.622693861088603, 27.281872711549433, 27.60605450565034, 27.996337324695535, 28.10337604655764, 28.007795432372053, 28.092063986878905, 28.258193070814848, 28.35288910690422, 28.5836080762085, 28.793450632283964, 29.13684980489373, 29.3867692004338, 29.103485011009205, 29.807674168537414, 29.623461524526252, 29.65166991042244, 29.628226063106037, 29.666266315628565, 29.630349002967066, 29.56472630409534, 29.378625580782664, 29.250732858697067, 29.055562779091286, 29.679051321831974, 29.709203636414117, 29.969965803435763, 29.83432716334101, 29.760103873681267, 29.673045680332688, 29.59463377636479, 29.47892016103341, 29.437763884009755, 29.443695445510325, 29.26993473399402, 29.200591614764175, 29.115203964319925, 29.084219736393653, 29.026595685801652, 28.89865878579562, 28.722035726518612, 28.91783963825636, 29.080129665037514, 29.28855021483691, 29.175434904601577, 28.961056027516896, 28.784465427220013, 28.63270163467907, 28.51433174265661, 29.047559239642585, 29.381530488395292, 29.2701624980534, 29.199135122770613, 29.0349097409741, 29.3255431084189, 29.126159320579365, 29.01030072077152, 28.899136132225188, 28.66897875245523, 28.504980684069025, 28.615746207345722, 28.349748370248427, 28.20423790199352, 28.058855709581067, 28.15369822573743, 28.202798698875625, 28.04161197868817, 27.758567575450705, 27.541084931843706, 27.333633593840332, 27.17453355216812, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.46435998243804, 18.82894519130404, 18.76942098260395, 19.034245817723765, 19.133923535461324, 19.431782936753017, 19.64415004817649, 19.981511542982737, 20.076142574060864, 20.4106814048593, 20.698466925640055, 20.822947374752886, 21.03977672793283, 21.1677515137123, 21.377669807042135, 21.628432605960697, 21.670699661689827, 22.28609954433334, 22.39651375255218, 22.589893087989363, 22.859333651468855, 23.109801384015626, 23.53588816893972, 23.778317073117815, 24.323760689707182, 24.531760036668263, 24.76022124490124, 25.299401883136497, 25.642510844787157, 25.868662681128907, 26.566694762189424, 26.757368378917285, 26.976857260142484, 27.15048257232609, 27.29848191047082, 27.423222133816612, 27.684670642450527, 27.96919031510094, 28.307311262573954, 29.09399930204003, 29.509369387024677, 29.89517863292287, 29.9448024592855, 29.629119052975028, 29.491637129042193, 29.943867098603285, 29.956903924371968, 29.885994472990053, 29.89667144409824, 29.689271410846796, 29.89573543850872, 30.056256147013567, 29.805850963631077, 29.6307266405244, 29.693000701172657, 29.61121790164326, 29.773953241480804, 29.539315083765636, 29.36314754753469, 29.12630356274845, 29.055916720267362, 28.944007586783542, 28.969288332000133, 29.07327900801703, 28.810665135131092, 28.657722337596915, 28.93157441438681, 29.067627783433625, 28.985502768730353, 28.791944426634576, 28.71687363334724, 28.646448091275506, 28.663740051155674, 28.6882786773327, 28.631341456940056, 28.449880301182173, 28.13229807658072, 28.26127801612111, 28.037833856464246, 27.97063664854494, 27.907284228501172, 27.778735383524587, 27.83055929641823, 27.801771331145975, 27.665294991402, 27.542803938202848, 27.462612447940256, 27.377807473449913, 27.328522759609854, 27.224648676553652, 27.43762706708766, 27.3989316451435, 27.350501356527467, 27.30318852203082, 27.258385020717657, 27.20928107936099, 27.150654098864884, 27.075179889261324, 27.04906048627625, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.42409659258952, 18.472017014889783, 19.003401593159253, 19.250546620619616, 19.65579902868569, 20.043749011521996, 20.23730027147096, 19.651560306940794, 19.97924406689619, 20.130568129393794, 20.505609651267918, 21.115406269947332, 21.34834122476443, 21.567673986925755, 21.775276052449964, 22.038216571557147, 22.263215935117234, 22.880637021085562, 23.033924603536768, 23.006485763598146, 23.289398219779855, 23.523957821976627, 23.76885114520933, 24.00881456628453, 24.252318704989744, 24.39507465344544, 24.58825757445405, 25.147255221925207, 25.267590383668033, 26.133392461050224, 26.316606965807516, 26.615821578913195, 27.096394302384237, 27.72720927250688, 28.011615770710197, 28.244010983719647, 28.584973985321792, 28.883430734305705, 29.1176760827363, 29.578538732457226, 29.788498134461722, 29.965775210080324, 29.746153049017142, 29.955429758779882, 29.89268995504254, 29.812922035952184, 29.780419691508964, 29.735258401186382, 29.67182177223081, 29.626165119959534, 29.539201419701623, 29.494645585672103, 29.457150550246165, 29.360360638040316, 29.579908384715154, 29.52530686956304, 29.41285260212025, 29.417178461481168, 29.25125419408956, 29.131826341647766, 29.21464973216827, 29.09333924548896, 29.294232436441693, 29.274882591240452, 29.238453581123185, 29.577266948400972, 29.482136808817106, 29.40596079799954, 29.551720109204087, 29.50647073912313, 29.39439395672649, 29.40957385714836, 29.633551907348277, 29.5572895521522, 29.397346053843233, 28.98828412017861, 28.95025164829147, 28.868069719383847, 29.03348138620429, 28.93422233211708, 28.85560069406361, 29.26069332294279, 29.084609499533443, 28.68995272184537, 28.75933204338907, 28.813478213922405, 28.616008029541273, 28.40406978649251, 28.169390394995837, 27.966230612893494, 27.95323555629515, 27.78002747769625, 27.616774461596872, 27.42479126078983, 27.36650573909675, 27.17687869091866, 27.283632511330367, 27.39620679646127, 27.171256822419615, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.468660577784412, 18.832199922810265, 19.15485006992745, 19.355869737285015, 19.487966161467547, 19.737373936881866, 20.026245016724832, 20.620767913523704, 20.79185433011529, 20.88288186606471, 21.029767651147832, 21.28917225422854, 21.624186346530145, 21.306825289932384, 21.55993453700209, 21.84829241878208, 22.060494839308802, 22.091466671212938, 22.357577071365526, 22.971781305114636, 23.204229217881092, 23.354022811836057, 23.608438283733808, 23.835071161900434, 24.09418496948473, 24.296373427670545, 24.391476804943498, 24.64535439794197, 25.525874053591696, 25.83446421963374, 26.637188553645792, 26.800196495293616, 26.877213987875155, 27.063423824753453, 27.242288370129064, 27.482610520662636, 28.01344521263134, 28.787162942291545, 29.13765222473056, 29.460745117109173, 29.640860792129292, 29.50117311680856, 29.775001710949127, 29.915366193017924, 29.89268995504254, 29.935025466550396, 29.96550295332159, 30.272351549387306, 29.838946193310523, 30.075862777165064, 29.814786401751498, 29.76303438018816, 29.6378625771815, 29.516945040871228, 29.455849497559516, 29.685826234441954, 29.70199146470454, 29.665338918762124, 29.905858384082112, 29.826448799006442, 29.67111631680589, 29.67773309373905, 29.622229628709473, 29.502523226933302, 29.21479173285927, 29.103105970090414, 28.96240072293168, 28.710771437399035, 28.4909039955687, 28.62402887376318, 28.508166665955308, 28.563927229034576, 28.42325057885419, 28.65398028598974, 28.839944328361046, 28.58454674428077, 28.663153110444423, 28.440224168488253, 28.358035419407923, 28.083920625619278, 28.059960345972417, 27.795211566550094, 27.75170098556665, 27.765619133037312, 27.69306947682888, 27.657416712351424, 27.64105982979736, 27.598484382383454, 27.583628262180167, 27.498871169575008, 27.364682960766924, 27.346534483213823, 27.474440082719013, 27.38815673411783, 27.289856972889794, 27.180875934181515, 27.125107543267703, 27.238555919170558, 27.101930748554913, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.481457132927407, 18.698727953885903, 19.072652319209045, 19.38442814745878, 19.91093766328458, 20.168580968107975, 20.218810555164275, 20.4929587596196, 20.826295809976937, 20.994701494180216, 21.08278794861363, 21.08681406046149, 21.3972598473124, 21.596423367955506, 21.735917419058325, 21.754957398531293, 21.914631012141385, 22.17491298208562, 22.530208696087865, 22.51976010112525, 22.784791982019176, 22.79509099080675, 23.333572788300025, 23.58046476641347, 23.806353837602696, 24.250686120268774, 24.843166435066372, 25.439354736086727, 25.546239953422905, 25.706262564732423, 26.302268741874325, 26.618930974695733, 26.907694385824303, 27.24758771362565, 27.60361465434525, 27.943379462962923, 28.091517092560803, 28.562898430674156, 29.53632133654675, 29.466936999441263, 29.761916052515474, 29.673270349304286, 29.7531947488902, 29.90851033363563, 29.89268995504254, 29.935025466550396, 30.301037453842095, 30.229096615621426, 30.146106977884468, 30.1369453480138, 29.814786401751498, 29.68330479233634, 29.625231530067698, 29.52931260487337, 29.37014583794163, 29.314600088455776, 29.40283273980263, 30.067434107873208, 29.888953696279486, 29.679525216012948, 29.486886272013624, 29.37494608100127, 29.392481849558227, 29.228949215585175, 28.99166900164976, 28.85343067624379, 28.578876820432036, 28.713888589655458, 28.433172435698516, 28.360640487789173, 28.476875112089584, 28.26277043615925, 28.08460230028858, 27.958374760021908, 28.079592278999222, 28.03759928538012, 27.89837839242329, 27.83832594171642, 28.115671113262035, 27.929882828976055, 27.9558995151858, 27.795211566550094, 27.72348763322643, 27.662734260746923, 27.63512600733222, 27.629808458936715, 27.61345157638265, 27.58216286606194, 27.482584904908613, 27.43553388915709, 27.34287573435321, 27.271062271062274, 27.457877959927142, 27.386517747063788, 27.36344913315929, 27.220842282996983, 27.126426543316857, 27.27639592046372, 27.19086853055951, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.39442127210876, 18.850102585349827, 19.191582010357422, 19.247847166001055, 19.595631609739975, 20.121998922453557, 19.9642833075005, 19.799845325747267, 19.924141055381707, 20.487116715350833, 20.746381381519896, 20.861287005588476, 21.15899757031383, 21.28491809977401, 21.570879251987762, 21.78247915551765, 22.050715352622902, 22.210352746015925, 22.517325712963995, 22.76006409160813, 22.91323754663689, 23.19640919215645, 23.70213939408893, 23.91642647585128, 24.096987260921974, 24.656331106238564, 24.85656007369391, 25.35414099824805, 25.636122328704914, 26.143244323464305, 26.6714927522255, 26.91503975832371, 27.039677459678302, 27.337705644422318, 27.64322997375979, 28.010332223931243, 28.3255036263954, 28.950364322412913, 28.98386480137231, 29.514284315211352, 29.750358027816056, 29.97591752541635, 29.78754915330707, 29.955429758779882, 29.89268995504254, 29.853825162996333, 29.81948140077279, 30.555405956385783, 30.50688253843853, 30.523674300710056, 30.481828373287446, 30.45440116395098, 30.31960804906131, 30.237891065705085, 29.827258295854236, 29.761794663780282, 29.735779907632487, 29.67745535518796, 29.568290250496194, 29.4788578561398, 29.371863898029083, 29.24090331675445, 29.172628746052993, 29.14562149903757, 29.03464099536829, 28.9222844296126, 29.247373893042898, 29.104310747917623, 29.040350609051064, 29.18346060095493, 29.09471734410401, 28.961877867869678, 28.83934243188353, 28.774975038649032, 28.64234842478442, 28.89519143358662, 28.93131319424728, 28.803854072191243, 28.692165888934813, 28.780349992673948, 28.92900184135912, 28.807722344874048, 28.66226106968343, 28.418775104559195, 28.804165914501517, 28.685456360873076, 28.695704462477124, 28.429791218108925, 28.344940429626774, 28.12724368348263, 27.925312319782186, 27.762585540554667, 27.679912110634724, 27.40651310188339, 27.317358720622114, 27.217333749841487, 27.267771364342853, 27.382436921323027, 27.182424788731748, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2027, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.474663963818433, 18.09911122968391, 18.32448928465982, 18.889428893914694, 19.275795980038783, 19.594362850828677, 19.517354613274097, 19.74676934244412, 20.176064323517355, 20.35441407285591, 20.69378698828593, 21.0723068541126, 21.298818805120085, 21.343289914718483, 21.90262288407405, 22.218665022957826, 22.445695807715186, 22.653792967213594, 22.713495933747424, 22.892764215468272, 22.96562856934374, 23.14085287177968, 23.437774533615084, 23.649315106661177, 23.918903702117156, 24.15307254313074, 24.75617246597696, 24.935087016918057, 25.48140638721337, 25.63896755339402, 25.917032303279775, 26.422783559092412, 27.100005352448854, 27.847850101338118, 28.204968248387907, 28.520345838487266, 28.71332921433271, 28.840845626753868, 29.182649781596687, 29.686775815721166, 29.88170215425176, 30.03397673020733, 29.834610300024135, 29.986412240981412, 30.03901774945289, 29.839064022957707, 29.786774929823395, 29.710329794489155, 29.688861353098, 29.64370006277542, 29.86553752815376, 29.76168477299661, 29.68776111929986, 29.6331168450757, 29.62587900137977, 29.5376812778446, 29.475327671258412, 29.39939869467058, 28.97438416585284, 28.856345631317897, 29.055382076136894, 29.004067784383153, 28.981399875229467, 28.890159633625384, 28.829914598132333, 29.11458635462566, 29.09983211998614, 28.96051485195381, 29.25054798985527, 29.205482743111023, 29.224043016293006, 29.114408004229087, 29.055613582979966, 28.822751263541065, 29.176749547034554, 29.059035671962093, 28.976926129016768, 28.903778290070846, 29.140742038614864, 29.029447187057713, 29.166509463623214, 29.012466673474602, 28.79029842320942, 28.9004377527064, 28.95157119844881, 28.818539615824694, 28.919962212945904, 28.650680365172278, 28.510039583056248, 28.31642191569465, 28.066362381579772, 28.13489147500807, 27.9731278062753, 27.841843851181224, 27.636613138047483, 27.454311934317182, 27.31030568289669, 27.238239611800825, 27.25435244799591, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2027, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.292287555924027, 18.215028465372868, 18.691716647446597, 19.01955691065212, 19.282609264830487, 19.653860865385287, 19.69938041385956, 19.76260949804838, 20.017560945702645, 20.314598504202902, 20.66626235088879, 21.022432552150892, 21.304756524624878, 21.546797660989274, 21.758055860723605, 22.023697321304027, 22.136553935448163, 22.37412247536644, 22.676300738802134, 23.413708902104897, 23.664743378334794, 23.822345363266077, 24.130081883685772, 24.682570221045086, 24.83818911267074, 25.09884997620787, 25.43579000165047, 25.753215208113367, 25.986454871011638, 26.06145995098456, 26.413297100216603, 26.58935928959038, 26.626590831847402, 27.16076231469971, 27.584607062177657, 27.938648206661572, 28.30484365279395, 28.691174009983627, 28.775520207682014, 29.019428152492665, 29.659121037828896, 29.344389713823332, 29.47386691685912, 29.643740265320368, 30.08169681309216, 30.03540251077229, 30.058414620595396, 30.02313997312681, 29.989994886111194, 30.01630618432487, 29.976791604196084, 29.783989078413835, 29.673735032433303, 29.587624693996663, 29.854322388330058, 29.78573817391575, 29.710608609349904, 29.291699195279975, 29.54997878189693, 29.51636037334577, 29.82279116234035, 29.740427331779873, 29.604202622944054, 29.36851884621408, 29.31941063485361, 29.30754589291175, 29.27847033801152, 29.499845597170903, 29.458217402398123, 29.249028708070202, 29.43803037698779, 29.58093916998334, 29.525595472266737, 29.43944943944944, 29.35092480957663, 29.122031055293768, 29.10414600988041, 29.134286105617957, 29.015889580764593, 29.139423109647716, 29.054043475948106, 28.806570252715126, 28.737817586400823, 28.60292622904126, 28.52481809192732, 28.325997472525984, 28.496094920264355, 28.26010922324011, 28.41448192069207, 28.312560443028378, 28.088242154697852, 27.758355264206415, 28.140231183605806, 27.95512046460813, 27.797035007210035, 27.532433200299057, 27.29162586166384, 27.229178845455245, 27.051957243384756, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.261050263278182, 18.5975706278719, 19.20339059310663, 19.565078131060172, 19.82534384564911, 20.086275815727973, 20.316430649434455, 20.355684317519536, 20.597634473440838, 20.858771903344138, 20.715501880993717, 21.165256055463676, 21.403163649613145, 21.40697537272291, 21.71398814980571, 21.797318944233258, 22.00618556975758, 22.323873723749077, 22.822308732230603, 23.08418660313568, 23.378079595239964, 23.40549696428796, 23.939586658588084, 24.16981624282425, 24.402060591804887, 24.62962221496522, 25.131492568255204, 25.180985958218198, 25.32202095696869, 25.52070387191882, 25.960543171345662, 26.247998717174582, 26.510804644651685, 27.03643707843795, 27.41787133101054, 27.671424803326644, 28.41793671142413, 28.470248450163595, 28.806080684345726, 28.95510632265989, 28.970123203706212, 28.993647375668807, 29.425920375335835, 29.661493428525574, 29.887341049455408, 30.327579370698686, 30.179442519344995, 30.07336869444434, 30.078996276019893, 30.029264802927614, 29.826870086628666, 29.650717505307362, 29.516022360937633, 29.390489111070668, 29.234780454776843, 29.15666814569317, 29.046061887838437, 28.832766447392032, 28.765024977084707, 28.738056238056238, 28.85492951828743, 29.573807750759727, 29.408946201079324, 29.165551654651168, 29.045063037000485, 28.870307149198833, 28.766132646758123, 28.608800509615527, 28.909194343663874, 28.654870293491708, 28.357608856147497, 28.14797386750067, 28.050533074387463, 28.023722787227335, 27.85867466497075, 27.78052221828195, 27.59882444845394, 27.459046945424337, 27.333030500889834, 27.440241355524623, 27.727457614753288, 27.778735383524587, 27.67138220325633, 27.608339967036517, 27.554095686661757, 27.520506518733317, 27.39360648664519, 27.343262891113103, 27.534423850156927, 27.421959692764737, 27.346323026003162, 27.323096565918664, 27.257511469223385, 27.200309494427145, 27.18486092281671, 27.12529805018059, 27.247936511471543, 27.163149105883143, 27.08468262118158, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.380743021078917, 18.55580501283816, 18.919904675765416, 19.069010461825066, 19.3293421451202, 19.643764291336947, 20.00529908792029, 20.234822915336736, 20.450136932679843, 20.73474615511203, 21.09895869572698, 20.941552955066264, 21.254612022822776, 21.61130232231228, 22.147634983447816, 22.416604942153015, 22.462128180092204, 22.682284134425643, 22.83094623854813, 23.323686193080682, 23.447693026935664, 23.86310239824759, 24.116271380338038, 24.363128554182598, 24.636497499614332, 24.84533292119612, 25.16040967937591, 25.45931526521458, 25.59792765515192, 25.727906784518712, 25.856073638110733, 26.269365496859994, 26.804867716959812, 27.09538377308834, 27.38220456584348, 27.6214588661505, 28.399577653884023, 28.544830954115337, 28.763299242114503, 29.330225953821458, 29.494122921354343, 29.682785501013104, 29.456178985866377, 29.64534714283414, 29.66246663690274, 30.290059601751963, 30.282485253094016, 30.221229968205193, 30.195140342668054, 30.13165453658697, 30.13165453658697, 30.175307812601364, 30.4455436909859, 30.34914432958915, 30.290521536223245, 30.5301011514369, 30.58417881192491, 30.530370814222024, 30.285903098251847, 30.183889981592603, 30.041538509214856, 30.010937834135458, 29.864819885773464, 29.703685599480924, 29.63832612235674, 29.571073979167505, 29.59577063278048, 29.4733110308808, 29.41447395709784, 29.54014068343036, 29.516508842634337, 29.41393410646855, 29.245114478252628, 29.125602577146704, 29.101245935094976, 29.00669588518011, 29.081330471612855, 28.82378371725961, 29.02623709170592, 28.731926893648936, 28.561847724492367, 28.44130904687147, 28.666926730024777, 28.528561280829752, 28.602547798771283, 28.49830443610295, 28.41640187689835, 28.36273888042859, 28.18692061869675, 28.118753171833667, 27.996939946449068, 27.881506770167636, 27.646964282061546, 27.510128719355954, 27.294347666910912, 27.12374446639622, 27.198415407612714, 27.11947057866182, 27.165892073910126, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.424440255627644, 18.633419627028243, 18.923968366806978, 18.981805266965992, 19.28164290581808, 19.313011747631272, 19.53839062731985, 19.773550642268606, 19.951650570590672, 20.24917421550473, 20.33089560414907, 20.599026707487763, 20.917657702562025, 21.22798990758941, 22.18109509788578, 22.45421341841846, 22.69206452817229, 23.055982792297907, 23.310036323948573, 23.730108407843844, 23.931925780438547, 24.58905502979546, 24.83197002426492, 24.645856274948343, 25.020253347497967, 25.073712591251184, 25.69479922090326, 25.872103186861274, 26.08657781842572, 26.2402218349791, 26.4944543581409, 26.7474102508292, 27.002116811403486, 27.300032042628107, 27.4565631122118, 27.538267022636198, 28.164149588091707, 28.386182927207386, 28.45839522739066, 28.613649435652828, 29.02554562454046, 29.233978674523186, 29.380222862959094, 29.67084042639941, 29.9099971283872, 30.661013879094966, 30.614347892370063, 30.241730938950923, 30.199150018466874, 30.14312363821963, 29.882736955596812, 29.911659963936206, 29.780426024157343, 29.551334573237625, 29.336058121937043, 29.28321294393903, 29.28269948547035, 29.19139119497952, 29.05024333498793, 29.067585408759395, 29.192297661547496, 29.075600396322997, 28.915880538318657, 28.79323237016743, 28.714508166581243, 28.712581306504198, 28.565049986599213, 28.72096912766221, 28.555488396560712, 28.758595622259953, 29.002001263778567, 28.898211404082346, 28.73236208129461, 28.655267185795086, 28.857879246373102, 28.733329914262168, 29.133914795456516, 28.821653310450817, 28.63319772913615, 28.382249570678525, 28.125844192975595, 27.814841049220522, 27.792625292197204, 27.78509965251783, 27.71656841663971, 27.634956987303248, 27.618635470981733, 27.577561348086004, 27.526007255115537, 27.451404530337157, 27.352113050024425, 27.303044663551567, 27.22723346123712, 27.138174309602658, 27.337588609055192, 27.231587404802145, 27.181698950766748, 27.04053617631348, 26.97847024739291, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.329245935515996, 18.781368369071338, 18.754278959869676, 18.989142499008977, 19.229261090993262, 19.360152452717152, 19.6083643536206, 20.237063217482238, 20.286931294730344, 20.540919841180074, 20.809690075497084, 21.145938921056235, 21.60966037815205, 21.911739376563933, 22.472633596281625, 22.8089648641737, 22.704185407927305, 22.997552171684326, 23.285641017621412, 23.431105151802207, 23.76713551605219, 24.062913821795895, 24.33683236659128, 24.611950612207085, 25.009557329706546, 25.27895878743363, 25.6630732119021, 25.699021731740917, 25.55546045874701, 26.144179461224915, 26.44104954684118, 26.818358951725273, 26.874230613504967, 27.67379521030746, 27.87907006320393, 28.275658797562116, 28.411054023631078, 28.630400389468008, 28.854500666742705, 28.966922385960398, 28.686618939618704, 28.882824235692745, 29.351772440214486, 29.569059532974084, 30.111403109998612, 30.589064848527403, 30.20195056989542, 30.172643449940722, 30.17867957537627, 30.06967112476155, 29.987789045482945, 29.700558490085005, 29.717047825416753, 29.49166870211591, 29.375301933342456, 29.19719631072053, 29.152657602203558, 29.101454706119934, 28.960448407187904, 28.968486487705285, 28.76570412021765, 28.916962251923046, 28.79647341114573, 28.916970223418403, 28.86670641560723, 28.766399840568642, 28.630176141176573, 28.500210038159857, 28.703013659103366, 28.529554757375212, 28.557151674860094, 28.44108663696509, 28.57307267581133, 28.275284634086912, 28.489363113053667, 28.32792404658874, 28.339212941369013, 28.303568103957794, 28.15273210996348, 27.879774802268507, 28.298381909887947, 28.17025201849316, 28.012916016377847, 27.864245084786717, 27.702279936632245, 27.855744454055763, 27.771591558407827, 27.610628326969593, 27.516742657894767, 27.4151004273768, 27.572438721285643, 27.43762706708766, 27.3989316451435, 27.413587687314678, 27.345412516643314, 27.274778370460375, 27.274778370460375, 27.14297949351495, 27.086934723248778, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.338662718614795, 18.68930204646847, 19.018141812271086, 19.103111204126442, 19.236009975660842, 19.62443429875249, 19.854962754509035, 20.052735706746216, 20.30321771741218, 20.614727975607757, 20.886445929329103, 21.187403014053842, 21.317239028712937, 21.558144259176622, 22.30376947045362, 22.614513008143945, 22.836755789830328, 22.969500799022317, 23.195682534913008, 23.32212749330842, 23.52653618094096, 23.930921605332657, 24.178464074305626, 24.438151013020363, 24.69751375882492, 25.20792361621964, 25.51713804453673, 25.828719988931258, 25.784434607620483, 26.352314157691126, 26.56735808923208, 26.64722690860645, 26.927188891550625, 27.29283186399975, 27.612886210614345, 28.402985788031355, 28.456875898097362, 28.753541954286522, 28.827123539928472, 28.906723010074963, 29.41909143320155, 29.663282798413565, 29.444569644311702, 29.582213355320103, 30.149390823219974, 30.577986636395917, 30.47616577775716, 30.405138103317014, 30.399478628519855, 30.299257882425874, 30.262356131878658, 30.034006734607743, 29.941007873776393, 30.223438061728448, 30.28580097307699, 30.24143707380845, 30.176593333102453, 30.43412264623696, 30.133797800411028, 30.014277117822235, 29.888433537362115, 29.871865128952756, 29.737480828266214, 29.6883726064541, 29.559348823094783, 29.46086599332344, 29.499939704999896, 29.365402252271462, 29.22010701513807, 29.087947066289253, 29.342732745790784, 29.25540930797832, 29.171985073912786, 28.94316634089685, 28.83927227034706, 28.83333578090107, 28.653409813857127, 29.104930394284818, 28.801628908938316, 28.94432679225404, 28.69964060292337, 28.46987843440681, 28.67894555082307, 28.533383489000393, 28.45709981572895, 28.62170399801621, 28.510459534184974, 28.467486132679586, 28.291489309330693, 28.11888848476265, 27.978648370978238, 27.863409505370207, 27.668656940010873, 27.461050040409074, 27.25432390817678, 27.088736145356187, 27.249747079786914, 27.112271805466705, 27.205821724098623, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2028, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.268250548974102, 18.646555936406443, 19.026952744142484, 19.252333417635537, 19.477396924045184, 20.174619173920007, 20.40520294838724, 20.36055206565291, 20.393616913218413, 20.308539899269125, 20.64195757769655, 20.9039158396126, 21.13279333644586, 21.367539110605318, 21.64521511661967, 21.892330536591615, 22.4104803048233, 23.003390827440658, 23.20894611858848, 23.324536444236713, 23.517664440495462, 23.741974117223087, 24.017578442789002, 24.57432347365153, 24.839734574266338, 25.303149662565122, 25.513233482986884, 25.837020685729506, 26.06205614885279, 26.263422240224966, 26.53565906336809, 26.97058368124754, 27.218779150606338, 27.512269735242047, 27.76747832503228, 28.482193862638777, 28.552118203570835, 28.78412041422374, 29.158654905411353, 29.27615168615525, 29.456271995533783, 29.68230741118152, 30.357538292233748, 30.32226080874975, 30.39916337559759, 30.313716293103187, 30.27750968275655, 30.27750968275655, 30.24063043462081, 30.24063043462081, 30.179693882473813, 30.109692656915698, 30.374335774085743, 30.278155612531627, 30.166614499486954, 30.093094241893798, 29.890002087562344, 29.77555968303455, 29.70364639104151, 29.606746570814547, 29.50698847857663, 29.466881089976603, 29.38766034954783, 29.16424550838454, 29.11453262932224, 29.04600851969273, 29.259006236137854, 29.35001100814568, 29.258962964024004, 29.120938101281858, 28.925637292108043, 29.131653625232833, 29.025990874494887, 29.174765733931608, 28.96035893457844, 29.014728654292327, 28.81948431102812, 28.840929004971947, 28.59341627029483, 28.544513000810078, 28.50966463598985, 28.624908415686555, 28.508803126501448, 28.276833723718337, 28.328142467866602, 28.119163745709137, 27.968669250645995, 27.770020217456263, 27.622711871699146, 27.28519292697808, 27.476281513485873, 27.383196655923932, 27.48132162176418, 27.565139720460746, 27.44317353462361, 27.25944080701881, 27.387365669517962, 27.245568130363335, 27.063053735777665, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2028, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.289943066143096, 18.71861027091851, 19.035741930584475, 19.344640086469774, 19.682040147275476, 19.402624564181757, 19.712073551962916, 19.694083406931384, 20.038254722421232, 20.38105329252021, 20.731678672023673, 20.90357002360346, 21.500674079318145, 21.745025876427356, 22.009909032391175, 22.077489674564806, 22.224416056393544, 22.48346553482642, 22.91477848898893, 23.325408978806912, 23.66605500231055, 23.876678876678874, 24.156961926596296, 24.348378231064085, 24.586232456179605, 25.205672705672704, 25.381718303339575, 25.760924909277183, 26.024278621102255, 26.31553964382607, 26.790537893143856, 27.14109185847195, 27.82794345207241, 27.96541192208818, 28.13402112007698, 28.55067027448015, 29.312178583055175, 29.524262290189068, 29.74987349329612, 29.768574039333238, 29.81124624657061, 29.92961651684553, 29.9666086829531, 30.074289690141914, 29.559392514807005, 29.57229555562362, 29.49710362977993, 29.827686617408954, 29.63622200042584, 29.577197832267778, 29.555667320456752, 29.572410661568366, 29.534470615030394, 29.564427416436857, 29.428328045841095, 29.33687561107956, 29.200980903460337, 29.188269688255357, 29.372591503198365, 29.263549851750582, 29.54169539451384, 29.483119050805414, 29.371697730210716, 29.348685406889203, 29.224788759312105, 29.036611598035833, 28.908265097730734, 28.820649221713012, 28.685454647162995, 28.945692453378285, 28.96782357043636, 29.260571848624263, 29.152965191166636, 29.047603244641973, 28.89239889667385, 28.731824739442125, 28.928774584212526, 29.072629198064416, 28.84304116341518, 28.75616384888874, 28.847786230236405, 28.784752158568978, 28.664630929043746, 28.768067828024897, 28.54229155066303, 28.59404667444162, 28.77039118998765, 28.60271765009118, 28.402140276583353, 28.29515517345944, 28.176268524423094, 28.3003230541716, 28.08591001040215, 27.924784350814825, 27.697668293302335, 27.509521585967455, 27.56609036716084, 27.278009792409698, 27.267595009530492, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.004585596442464, 18.539097467200204, 18.8636826600607, 19.178085104139722, 19.307851281867887, 19.498013470882913, 19.389608613322075, 19.697972362637582, 20.00819249200816, 20.376132482507444, 20.628593930827694, 20.827955006453994, 21.058138071162748, 21.65609016004461, 21.95735132073579, 22.286163464340643, 22.233426895314462, 22.615751411996197, 22.936888804931318, 23.16866140395552, 23.22847447278193, 23.37479341116267, 23.697791139069665, 23.80121785070422, 23.975520002026805, 24.491517874097863, 24.716477108084614, 24.920355779932958, 25.29665720042958, 25.789070305631867, 26.121105235506896, 26.30494217528483, 26.382107950792616, 26.85654244958459, 27.34448094739147, 27.604139886895958, 28.066055737264, 28.062168179812996, 28.756377846001563, 28.991517806283053, 29.10595343477278, 29.358867880392157, 29.632033398389947, 29.53110879452621, 29.92414497843759, 29.903144457643783, 29.838557038648567, 29.869432003403297, 29.812990770078525, 30.056415701206284, 29.90884828181024, 29.663326768162936, 29.50922752428045, 29.626918956765635, 29.52735450936977, 29.456184136125707, 29.31579958308192, 29.33253788142183, 29.292769180290062, 29.19676858649418, 29.213852936893108, 29.446366209495935, 29.548166679403113, 29.686950778897632, 29.480176551974544, 29.24973564436098, 29.26252856548911, 29.10444193233713, 28.807513914656766, 28.65733333773789, 28.63460323214147, 28.722664837400224, 28.84594963273902, 28.675065352998985, 28.514709873381577, 28.765605858392888, 28.563916169725506, 28.17813502231311, 28.08967989443993, 28.217499869608304, 27.988121109923103, 27.814841049220522, 27.75170098556665, 27.735260620330337, 27.69306947682888, 27.64157130061875, 27.55484988808482, 27.730311019059762, 27.72286915215495, 27.652525165902357, 27.598387481010906, 27.547635989879826, 27.48109383950058, 27.386210249811892, 27.217614671073715, 27.171661737898013, 27.293366912961968, 27.197006943056774, 27.143006712220615, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.192410073930475, 18.288519263957014, 18.320525953391105, 18.67076102374577, 19.254739169974453, 19.41121616061463, 19.849850554457294, 20.215773970839106, 20.139340750940296, 20.312435870479682, 20.596897699142087, 20.92664314899305, 21.229087240431127, 22.07247406126482, 22.82014675024072, 23.243041365144236, 23.719805245224403, 23.932972492234587, 24.68758646266873, 24.929653931468813, 24.953990061602436, 24.961974082781506, 25.203374981309288, 25.275747083153817, 25.531881323144745, 25.575009330563304, 25.581679890382404, 25.867198462814134, 26.046052560982524, 26.140383139635997, 26.601901293185904, 27.113914043569082, 27.33368897585314, 27.56456901852264, 28.07404724089783, 28.261837968205995, 28.784186175530323, 28.94130683231402, 28.846958003852013, 29.144142137436756, 29.49046155177215, 29.613492013953913, 29.68619936302718, 29.78113106346244, 30.29287022535189, 30.308140949600027, 30.155595094759395, 30.195559959570822, 30.237492296990844, 30.20319075389975, 30.167429079244982, 30.08792396332156, 30.046348509146505, 29.979158476361732, 29.96627111922729, 29.867591489794602, 29.625967127223618, 29.527405083124425, 29.653216751386047, 29.841042921964146, 29.726801034126826, 29.989016464571755, 29.890229174003135, 29.938188489596612, 30.136510078872725, 30.07465944186124, 29.978476750628648, 29.751595810503524, 29.239264984600798, 29.046242890237323, 29.256203553348882, 29.056821955850726, 29.241595116570124, 29.137743429047774, 28.964648374105522, 28.887429524534358, 28.812828716752243, 28.63434920473026, 28.659327890264453, 28.50514016368455, 28.48191135283432, 28.378823875400837, 28.55057844447282, 28.77373387236068, 28.700863046615748, 28.546166750926382, 28.353258446603423, 28.335711136039386, 28.22138260131457, 28.315061107600997, 28.398353582665177, 28.20440641015801, 27.979699517033673, 27.764160482109318, 27.92607538523159, 27.779023798291536, 27.697094440587083, 27.558614890962634, 27.30182812037027, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.2425838376296, 18.712913601830863, 18.744727736879312, 19.06101975310768, 19.181640841990177, 19.317178533283062, 19.55830741292984, 19.773549967781037, 20.081623744795, 20.328140680862884, 20.5522422642007, 20.80671727104768, 20.879477546335046, 21.19716252930187, 21.31130287769031, 21.68666627677466, 21.871034715189815, 21.77620088663142, 22.40973613540826, 22.678533839970836, 22.966548284829518, 23.587704717127725, 23.799687502677077, 24.346488982129664, 24.89986546897116, 25.15644833349895, 25.687491797130324, 25.9495202178129, 26.163064833171884, 26.202516610059735, 26.39499186459031, 26.502317601715504, 26.846863849846592, 26.987576072036713, 27.30660738948203, 28.119596932790202, 28.78536942153772, 28.78758482284274, 28.962374120482252, 28.99615657737501, 29.15060831990238, 29.245915545521843, 29.447601681490653, 29.58424478493408, 30.215388161144062, 30.308140949600027, 30.243419372669894, 30.29062508277412, 30.24304817474431, 29.92203219743318, 29.847425071212424, 29.78407523980815, 29.775922008516574, 29.601781197525877, 29.908277791932257, 29.82960123649889, 30.11620060517982, 30.024533139874286, 29.749291326757536, 29.71885453308308, 29.53557035199857, 29.589961128742075, 29.488229059435824, 29.375744560383477, 29.137442245230083, 29.009610275240682, 28.81555712974371, 28.633361037866695, 28.54565717875569, 28.34155131687225, 28.062874484848194, 28.173494228449613, 28.07450824860036, 28.27215740174393, 28.940819868518602, 28.73378331012156, 28.50177990149245, 28.329781905137235, 28.218278510359706, 28.204875489025188, 27.8916927893879, 27.795211566550094, 27.809088965897814, 27.756716811974215, 27.756716811974215, 27.70458545609349, 27.617744782999786, 27.574720999157126, 27.77204319073888, 27.65661009721431, 27.607138182768097, 27.486620795107036, 27.35377931131651, 27.343032552854986, 27.269150436895195, 27.170207864578206, 27.03959761223812, 27.017002536070333, 27.14228696143028, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.02110181187292, 18.208021870238188, 18.510836030383953, 19.05965143047679, 19.317326139791575, 19.660821072447153, 19.87121084413419, 19.87863037428884, 19.85920338951322, 20.20479987354947, 20.310092698629216, 20.596874152583382, 20.826912996324413, 21.383527408852398, 21.717368255174165, 21.779699490385045, 22.13399591975107, 22.53246290480333, 22.900448428848886, 23.055064620239406, 23.713647135500906, 23.60021291571407, 24.03426392897283, 24.313853655712812, 25.033908209160977, 25.420846838470055, 25.670026339907952, 25.826782022721638, 26.099141348349065, 26.353139946483356, 26.638780168707065, 26.864877519480768, 26.896547118433023, 27.296486343207327, 27.99245376197213, 28.598869851539977, 29.062215451728495, 29.27987406717808, 29.328191363090262, 29.69620305259123, 29.689843047813003, 29.246837977356655, 29.479978407216283, 29.67671605296514, 30.19220940061727, 30.288745088508122, 30.324941804133243, 30.335735948639176, 29.890724372480054, 30.14312363821963, 29.847425071212424, 29.66853660967149, 29.542562582249253, 29.543526301162164, 29.367707424988698, 29.580771892684115, 29.589857369515176, 29.687484913946534, 29.527511868734614, 29.292970326361043, 29.15077679330016, 28.90658416274053, 28.92404863812661, 28.80951402218741, 28.705911202708954, 28.519495616269808, 28.75940056025025, 28.6289704497954, 28.415923746973636, 28.34506317463368, 28.07290563462516, 27.917087895658614, 28.12059088920486, 28.193622616068648, 28.013233262237247, 27.844845208168213, 28.02349201366952, 27.88806177754871, 27.80610623705576, 28.02740559150169, 27.928170909267223, 27.778735383524587, 27.77118150504717, 27.64708088713337, 27.543599970963957, 27.433862433862434, 27.598783606298603, 27.57534716154676, 27.67179586586856, 27.57447554069351, 27.467864468298085, 27.371734887621535, 27.357825107104933, 27.384637233080944, 27.309965253017076, 27.218020976450358, 27.162331048108353, 27.08468262118158, 27.014290510930472, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.192410073930475, 18.288519263957014, 18.320525953391105, 18.67076102374577, 19.254739169974453, 19.41121616061463, 19.849850554457294, 20.215773970839106, 20.139340750940296, 20.312435870479682, 20.596897699142087, 20.92664314899305, 21.229087240431127, 22.07247406126482, 22.82014675024072, 23.243041365144236, 23.719805245224403, 23.932972492234587, 24.68758646266873, 24.929653931468813, 24.953990061602436, 24.961974082781506, 25.203374981309288, 25.275747083153817, 25.531881323144745, 25.575009330563304, 25.581679890382404, 25.867198462814134, 26.046052560982524, 26.140383139635997, 26.601901293185904, 27.113914043569082, 27.33368897585314, 27.56456901852264, 28.07404724089783, 28.261837968205995, 28.784186175530323, 28.94130683231402, 28.846958003852013, 29.144142137436756, 29.49046155177215, 29.613492013953913, 29.68619936302718, 29.78113106346244, 30.29287022535189, 30.308140949600027, 30.155595094759395, 30.195559959570822, 30.237492296990844, 30.20319075389975, 30.167429079244982, 30.08792396332156, 30.046348509146505, 29.979158476361732, 29.96627111922729, 29.867591489794602, 29.625967127223618, 29.527405083124425, 29.653216751386047, 29.841042921964146, 29.726801034126826, 29.989016464571755, 29.890229174003135, 29.938188489596612, 30.136510078872725, 30.07465944186124, 29.978476750628648, 29.751595810503524, 29.239264984600798, 29.046242890237323, 29.256203553348882, 29.056821955850726, 29.241595116570124, 29.137743429047774, 28.964648374105522, 28.887429524534358, 28.812828716752243, 28.63434920473026, 28.659327890264453, 28.50514016368455, 28.48191135283432, 28.378823875400837, 28.55057844447282, 28.77373387236068, 28.700863046615748, 28.546166750926382, 28.353258446603423, 28.335711136039386, 28.22138260131457, 28.315061107600997, 28.398353582665177, 28.20440641015801, 27.979699517033673, 27.764160482109318, 27.92607538523159, 27.779023798291536, 27.697094440587083, 27.558614890962634, 27.30182812037027, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2029, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.405587021919224, 18.55348738922084, 18.61057795341338, 19.082186057831528, 19.503748645710754, 19.790639004244216, 20.16963347799163, 20.353811937741746, 20.337971800247775, 20.46125851121012, 20.82755234774615, 21.07136699655907, 21.55759002113711, 21.85451339189657, 22.133554600951395, 22.342155795472618, 22.412087684467345, 22.871309207443662, 23.120753941611294, 23.277172084246605, 23.533248932353995, 23.72155398106955, 24.166293395657647, 24.370681201130242, 24.891928447753404, 25.42900454985456, 25.69310977633976, 25.739400712551458, 25.997331644475395, 26.122015131104025, 26.463880816777817, 26.6003151312691, 26.941093233776158, 27.22000220119085, 27.519852323024054, 28.158281717353837, 28.4409100390681, 28.67366232277909, 28.73485315823275, 29.173033081602178, 29.309683973682443, 29.345500949954662, 29.499918736114754, 29.663089237176564, 29.576322090403075, 29.640555005248448, 30.04813797826809, 30.0847253340767, 30.00652158743169, 30.031011244108196, 30.02180235166902, 30.31348054375519, 30.143209140170207, 30.15021100986367, 30.270609410610376, 30.270115630337635, 30.189792787370244, 30.120608008539048, 30.32491716959505, 30.086484076900405, 30.073347471851203, 29.993036117346385, 29.87101716472299, 30.000742714616603, 29.953615856926735, 29.799412309656724, 29.740146960779306, 29.625259657930574, 29.70538943546415, 29.52971757526842, 29.609110112434383, 29.53387865643984, 29.522270719387944, 29.460157436347913, 29.675356422550813, 29.55076179362311, 29.346896792994254, 29.35936421962292, 29.31221480015055, 29.167100668286334, 29.04045052833436, 29.00858951858544, 29.13303257390919, 28.959127596140288, 28.842704995315387, 28.704889010592392, 28.47623148746537, 28.358470581066403, 28.371140423403794, 28.04078909267764, 27.84533949050078, 27.932020867049822, 27.67806899641577, 27.492653122326683, 27.37757433244149, 27.234410262639695, 27.1734543432031, 27.04299191610149, 26.842469485383596, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2029, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.278306796663287, 18.277843349581655, 18.3908724642688, 18.98681120573743, 19.13147342522595, 19.472218395984783, 19.4826091099676, 19.708523033910755, 20.00100144385664, 20.33228949647639, 20.452589583904704, 20.70871798725398, 21.342871719203337, 21.557982343801314, 21.804906423550495, 22.39997201271119, 22.735298456781123, 22.929010756715456, 23.143471662768295, 23.37809173786454, 23.69979276451951, 23.98056311100499, 24.304243576395475, 24.47987649551795, 24.701621717434037, 25.068286883210213, 25.488160060998265, 25.444589461315136, 25.8473981968514, 26.434788097686223, 26.671444562745283, 27.016520978586584, 27.18130782757568, 27.512479067874864, 27.72605367327302, 28.115638252328267, 28.269819931346298, 28.54889009613666, 28.550405362990865, 29.637747801866976, 29.920850166751805, 30.090179099711676, 30.139605110336813, 30.265668311683402, 29.92607767567432, 29.958421910729793, 29.7400141779222, 29.664352743578288, 29.67686290757915, 29.6820114405651, 29.47986924533585, 29.556119008928288, 29.61003152651219, 29.5585222143961, 29.56013427283588, 29.34377334817242, 29.325131444391467, 29.25868371395996, 29.16294967042592, 29.147954401742183, 29.133685818318195, 30.095601115161656, 29.990187523552446, 29.911301092946434, 30.137267243559464, 30.070027103998676, 30.01160983892098, 29.866835181185465, 30.109212284935765, 29.92955271428987, 29.78141267832161, 30.057767888558555, 30.066674505167523, 29.944342299521765, 29.680730234453723, 29.398208578242404, 29.361308835643563, 29.0690897264103, 28.88256394885593, 28.777382900474603, 28.61584930337987, 28.856781902245615, 28.728435055487118, 28.671942887258332, 28.459021545189582, 28.797489045345458, 28.69760873812454, 28.518929650322388, 28.350795334401564, 28.10258999760621, 28.29592475963643, 28.17225650139079, 28.03491712354966, 28.09729054810992, 27.791241791241795, 27.647384735508908, 27.429640809755142, 27.248009533248403, 27.240661074607054, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.301317070795598, 18.743883078592873, 19.105054355218666, 19.518915916969384, 19.884951661566657, 20.06669325921246, 20.573314712199416, 20.516743869930583, 20.654411174927482, 20.832064782134303, 21.087504509848507, 21.381767522901207, 21.584688778327322, 21.828550545146065, 22.109401824358184, 21.900974912325456, 22.177059853820587, 22.595606944354447, 22.813391519425014, 22.91350645453231, 22.865116215570787, 23.084177157952183, 23.465083770964803, 23.978680044968083, 24.158618517732418, 24.358401578231298, 24.57557605325672, 24.744174655285935, 24.95950971142552, 25.263177424931992, 25.459034427787998, 25.724301353884236, 25.96086201397318, 26.20927011108225, 27.028313197099052, 27.212478185643164, 27.7753137211452, 27.962935346212088, 28.355624248826278, 28.6233051705386, 29.062399195697385, 29.468256824170773, 29.755822566421863, 29.877916817997846, 29.785464675358185, 29.809242293296695, 29.71693728733523, 30.033380556801596, 29.671315667436172, 29.8315005828944, 29.826870086628666, 29.66362054612398, 29.42191279055758, 29.649462540051374, 29.637287215376052, 29.608077831140832, 29.491302579968202, 29.297097251323777, 29.255294562966196, 29.523850528910224, 29.445829421266417, 29.296327455467306, 29.334634766663324, 29.526822697401922, 29.403483070641478, 29.18249790192214, 29.313050531331307, 29.127144840717346, 28.860413955505983, 28.929406271454937, 28.78205431151765, 28.99672314767606, 28.83375205522185, 28.584549871570562, 28.298722301689004, 28.36929845039215, 28.366570533288886, 28.228313253536257, 27.998095567894943, 27.7919375526761, 27.89197793259953, 27.795211566550094, 27.66743240272546, 27.600535149667827, 27.502951539881547, 27.427139962544132, 27.31550600398125, 27.492860021981198, 27.478113743701783, 27.421959692764737, 27.36961032079046, 27.346323026003162, 27.308829810775997, 27.296887114258865, 27.225990026300728, 27.363889534253182, 27.249087174806025, 27.178876050182804, 27.156402102807764, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.33860367522075, 18.851511528806746, 19.131875018946456, 19.277738594253595, 19.531137658578483, 20.00323619769678, 19.846899744341055, 20.174829354590994, 20.347771934260138, 20.675154931784313, 20.854735645779776, 20.91539947752346, 21.155039279855785, 21.523560179723436, 21.813664108274615, 22.334740013938976, 22.47708067531922, 22.70623368941035, 22.880127008745042, 23.374277353924523, 23.48851084292608, 24.146505219468377, 24.334455945202848, 24.408432539532082, 24.649313817937774, 24.81217343222691, 24.986408580194215, 25.12694429906025, 25.39182390810494, 25.660445350166512, 25.894092439253345, 26.557599322103982, 26.845415270714597, 27.09704519331435, 27.275456125949677, 27.856891731712217, 28.42734473945654, 28.539129903871054, 28.83784778791181, 28.895299783288884, 28.98998441991902, 29.14933724417837, 29.79941747251885, 30.276261499504287, 30.239707515030094, 30.226480207643032, 29.930764483951965, 29.922400727603254, 30.230356106414284, 30.13349430413013, 30.108542577283565, 30.017524295370073, 29.928540368364438, 29.422767066670335, 29.46356586036788, 29.758331622001432, 29.664618400468612, 29.365992325742457, 29.281569587840885, 29.26978479701086, 29.255132419128227, 29.459268160505168, 29.37292340159207, 29.18536820322534, 29.24900290065265, 29.138445975662602, 29.132336689742267, 29.0813039844127, 29.018687835017477, 29.21495791148737, 29.087756583957635, 29.210710654514056, 29.1446957031897, 29.12700128577839, 28.933026111938386, 29.055450903613433, 28.96744809362548, 29.01384820570142, 28.899632843500957, 28.64831616021023, 28.77189807702514, 28.956371365923683, 28.828598113256987, 28.8036352459875, 28.645150910245494, 28.573200220259043, 28.398213690429806, 28.27896152136268, 28.156087016291707, 27.973773103065035, 27.871860534892345, 27.688896216792706, 27.436994012275196, 27.336112294063025, 27.35353434403861, 27.252609663218152, 27.14660851919877, 27.279846092770928, 27.16912611520569, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.438415052907605, 18.72570846319579, 18.83590712273324, 19.095887577609076, 19.48124824774575, 19.943062812125074, 20.24763023294206, 20.425338776243457, 20.588051497055677, 20.860207068173573, 21.11136902049293, 21.07082677642597, 21.14045422371087, 21.426805742353192, 21.552947531644882, 21.8663159790763, 22.17736517859967, 22.553082245062438, 22.423319353591044, 22.856515721942532, 22.988797179325815, 23.19247127348717, 23.438222592189422, 23.51931887706214, 23.809183628329055, 24.004514973410537, 24.25195842496455, 24.457629912758268, 25.093696147721573, 25.20067260738147, 25.49854293903585, 25.77117978087929, 26.01360444878355, 26.684026306871132, 26.942870178939167, 27.267821426903758, 27.531652406879264, 27.929789084952084, 28.39113260412448, 28.628973642383983, 29.610519569554, 29.694761655968556, 29.963661843107577, 30.142978679812643, 30.063369725092464, 30.003066574504743, 30.26948895311718, 30.28517890120031, 30.228225945873007, 30.027722319370397, 29.89573369032011, 29.66370739179326, 29.626935378277057, 29.59052929817913, 29.462934217984778, 29.327556462260873, 29.211036765885197, 29.14858353715246, 29.333792622435578, 29.16107369745965, 29.319640852220786, 29.553692556901655, 29.5782626716796, 29.490800433707005, 29.346771411108374, 29.281194485706543, 29.242625617171047, 29.085328759338264, 29.181686389320618, 29.295479125464, 29.058416304133505, 29.203799351158892, 29.02917701779843, 28.759287444377, 28.620088919845244, 28.74670338167029, 28.60130955033918, 28.491533409505397, 28.3383211870755, 28.4443083786213, 28.322157722426972, 28.135979265524124, 27.97155949983219, 27.8943851706002, 27.802962873056337, 27.704859975136564, 27.647022692157748, 27.48155239218576, 27.430463080781355, 27.362477444941675, 27.30646646085101, 27.50387359227625, 27.43922132152533, 27.331552399146364, 27.224775536828062, 27.092414785956937, 27.069844195107745, 27.204221427895824, 27.157132879979258, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.35031126697247, 18.53258140348169, 18.923461822416204, 19.35805347606152, 19.31342706888648, 19.47773332632165, 19.8721711143875, 20.060434479750853, 20.144318938337666, 20.377922195830685, 20.55188468029287, 20.779380532433102, 20.945752835746493, 21.16354618159792, 21.36227551881713, 21.566473740749142, 21.85644328881754, 22.170910562804828, 22.5761396112663, 22.702467386005498, 22.986095013857945, 23.321896980273944, 23.47573736239012, 23.88875640234941, 24.5276572496365, 24.692578424665026, 24.80846943043153, 25.0018587126398, 25.015475507833905, 25.27727194726449, 25.8098006381805, 26.180249365030456, 26.47316449401147, 26.958895058975006, 27.18418747011686, 27.34169175386948, 27.617492555241796, 27.56917343602848, 28.206827821594104, 28.815573410794926, 28.990567912611187, 29.528472095237934, 29.704938334736177, 30.18027208969688, 30.057429040735112, 30.067832463523942, 30.26948895311718, 30.175813815910473, 30.228225945873007, 30.125009771826512, 29.978992133716996, 29.68164883935146, 29.613686822443025, 29.565281895606415, 29.541911890986867, 29.27431063818094, 29.197103555024835, 29.368515653571357, 29.395646377782374, 29.219946059492923, 28.980758819826313, 28.848985750555467, 29.252886893120376, 29.45899410333884, 29.32232205584227, 29.245397770085614, 29.153767721032516, 29.143103608649483, 29.35498775954554, 29.435085250424738, 29.24403232743963, 29.00818150554368, 28.985529868682043, 28.999240726917137, 28.89385943056467, 28.750375817659712, 28.503353334569066, 28.416111912465748, 28.25297553687095, 28.32602238405783, 28.172654542863178, 28.026827833800986, 27.965588582537748, 27.859728617030626, 27.798719738937134, 27.640750153188563, 27.59853462729305, 27.74594090582565, 27.72369799252129, 27.69442239339275, 27.58950989503314, 27.508848179461932, 27.646691633589622, 27.552064333718395, 27.46529716253991, 27.43340533960331, 27.37613450379408, 27.32679054271408, 27.174615933176295, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.33860367522075, 18.851511528806746, 19.131875018946456, 19.277738594253595, 19.531137658578483, 20.00323619769678, 19.846899744341055, 20.174829354590994, 20.347771934260138, 20.675154931784313, 20.854735645779776, 20.91539947752346, 21.155039279855785, 21.523560179723436, 21.813664108274615, 22.334740013938976, 22.47708067531922, 22.70623368941035, 22.880127008745042, 23.374277353924523, 23.48851084292608, 24.146505219468377, 24.334455945202848, 24.408432539532082, 24.649313817937774, 24.81217343222691, 24.986408580194215, 25.12694429906025, 25.39182390810494, 25.660445350166512, 25.894092439253345, 26.557599322103982, 26.845415270714597, 27.09704519331435, 27.275456125949677, 27.856891731712217, 28.42734473945654, 28.539129903871054, 28.83784778791181, 28.895299783288884, 28.98998441991902, 29.14933724417837, 29.79941747251885, 30.276261499504287, 30.239707515030094, 30.226480207643032, 29.930764483951965, 29.922400727603254, 30.230356106414284, 30.13349430413013, 30.108542577283565, 30.017524295370073, 29.928540368364438, 29.422767066670335, 29.46356586036788, 29.758331622001432, 29.664618400468612, 29.365992325742457, 29.281569587840885, 29.26978479701086, 29.255132419128227, 29.459268160505168, 29.37292340159207, 29.18536820322534, 29.24900290065265, 29.138445975662602, 29.132336689742267, 29.0813039844127, 29.018687835017477, 29.21495791148737, 29.087756583957635, 29.210710654514056, 29.1446957031897, 29.12700128577839, 28.933026111938386, 29.055450903613433, 28.96744809362548, 29.01384820570142, 28.899632843500957, 28.64831616021023, 28.77189807702514, 28.956371365923683, 28.828598113256987, 28.8036352459875, 28.645150910245494, 28.573200220259043, 28.398213690429806, 28.27896152136268, 28.156087016291707, 27.973773103065035, 27.871860534892345, 27.688896216792706, 27.436994012275196, 27.336112294063025, 27.35353434403861, 27.252609663218152, 27.14660851919877, 27.279846092770928, 27.16912611520569, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2030, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.384892937650058, 18.757206279278645, 19.063709696680913, 19.27210175714784, 19.276642372784053, 19.306979187696786, 19.552610031679993, 19.76019558181892, 20.08937264411717, 20.307233296094683, 20.578462968465484, 20.924132397275407, 21.199807505752354, 21.42840898981084, 22.08049552856043, 22.247683153176226, 22.577000548140216, 22.804479176461776, 22.91871717818449, 23.145610395309316, 23.44749728946902, 24.090999820196632, 24.200876604223772, 24.464519623876253, 24.755960405855337, 25.117939913333664, 25.0900114068209, 25.24669417192188, 25.46846355277978, 25.608040642512176, 26.179594746819568, 26.399135549475005, 27.19901356797826, 28.03370141627014, 28.336631992592697, 28.521308661217248, 28.725083547817263, 28.900006104072606, 29.16414747502775, 29.383085093798083, 29.586860317554358, 29.643307998658248, 30.267077476021093, 30.406135246055904, 30.067110778330203, 29.926700302957727, 29.896934898921444, 29.846076400823996, 29.822588438404736, 29.759985075473473, 29.677422671025468, 29.611349488398673, 29.932365159474028, 29.811397813676905, 29.743088520564136, 29.713929889010437, 29.63965141238214, 29.956608754436743, 30.05230767916792, 29.91457222014256, 29.93347545687812, 29.841577167904692, 29.69138795005268, 29.65445240058638, 29.546725208165647, 29.400598599009818, 29.137506622293643, 29.392617079259143, 29.259255859738744, 29.112923544829254, 28.998041755031167, 28.958427241189717, 28.918717190842695, 29.188857204419982, 29.341687755145706, 29.13888999386266, 29.087010642943323, 28.981126459901585, 28.892430708402138, 28.783245805606718, 29.16726312854614, 29.36149584656656, 29.216014755654367, 29.110354179721554, 28.978562150597558, 29.04667884338341, 29.23515805624705, 28.96316089687537, 28.843859453799215, 28.750856992898104, 28.604780493138392, 28.49561382592163, 28.403830786998473, 28.145578475262283, 27.861494554527717, 27.672792733684886, 27.497621378566876, 27.32330784644311, 27.127141728528837, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2030, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.2411578934466, 18.576190817551, 19.14691945937652, 19.561426451918813, 19.516254107701116, 19.857157963594155, 19.987588147747154, 19.76092829476288, 19.956983260595198, 20.146117829453743, 20.850508724569004, 21.078513005681764, 21.31110334614961, 21.45344144574817, 21.629099215831992, 21.882308745481694, 22.07300944339852, 22.22602079315954, 22.96305537181199, 23.559071071342153, 23.8025189766126, 24.118937679346562, 24.443578577927987, 24.862514878250522, 25.047925905395417, 25.26263227150496, 25.538472884336837, 26.035540059365303, 26.292083167392683, 26.605130586795706, 26.838464243045372, 27.165013685437998, 27.25687669657917, 27.57510308031356, 28.117451227934836, 28.333182320180782, 28.48960179605341, 29.170763230782537, 29.435342924564207, 29.605339265850944, 29.84482914891688, 29.949349090150633, 30.090366049492207, 30.281538673503206, 29.894359163871364, 29.816842952776103, 29.767223013493528, 29.772398508331655, 29.719995301712753, 29.643683336865156, 29.658873907680842, 29.93128482533448, 29.87897685720372, 29.777373268439472, 29.532839535671496, 29.095728027171752, 29.02954986099422, 29.256416365351885, 29.142280661597024, 29.06596161699827, 28.93664277567627, 28.900380360115896, 28.795389088530616, 28.695998018408726, 28.616805706716224, 28.489973902872055, 28.82496483266854, 28.696258186319135, 28.61638127326017, 29.18965570797678, 29.160497469863316, 29.67332832621064, 29.574577770598797, 29.37076958635413, 29.351659650456146, 29.50210909753359, 29.705501798821892, 29.58529408446502, 29.38655776762222, 29.380672799829977, 29.252240562484616, 29.14331277926102, 28.98395009501663, 29.15456900178297, 29.074232228933482, 28.992622479186274, 28.848158004158687, 28.734714686520178, 28.631529369661763, 28.4759036312471, 28.445871422033697, 28.27318302374438, 28.411140759369424, 28.244984281187413, 28.090018384229047, 28.120785663937443, 27.900851121686404, 27.58798183277338, 27.314223672432675, 27.001009067886244]}, {"task": "construction", "expert": "yolo26x", "endpoint": "Qwen3.8", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [18.129624718318375, 18.28501397889031, 18.650672626833323, 19.090387413923366, 19.382947863919572, 19.710280624102257, 20.074997485051664, 20.09566939990949, 20.115767832011162, 20.390446078697742, 20.516866048833005, 20.7525814225743, 21.01713417248417, 21.554200271549146, 21.44124890719934, 21.611193752839288, 21.81811589257623, 22.07692705908518, 22.11004914950206, 22.35049404303762, 22.610774757794193, 22.843930390560818, 23.39609860123484, 23.483069546179305, 23.717933301266633, 24.052579743273444, 24.654831677541484, 24.94399353990571, 25.231761081166887, 25.37000258917958, 25.538806803822578, 25.75344225039978, 26.88874853939069, 26.9333851957154, 27.228104732116453, 27.50881551965454, 27.915423136726886, 28.07680228060136, 28.382163011467398, 28.503297926643544, 28.668257805748855, 28.813390309394133, 29.35987467265544, 29.656577702492342, 29.663371678690787, 29.75874683858742, 29.776893710929325, 30.125720786527605, 30.10330685425498, 30.158058050319887, 30.0962544968623, 29.847425071212424, 29.650717505307362, 29.555339696918043, 29.386140421164768, 29.33586593273757, 29.345006099519864, 29.309411404447754, 29.144315064867243, 29.458711330995378, 29.490358694634544, 29.3570244276686, 29.54453612510701, 29.49454681470982, 29.29694162401765, 29.463738014959052, 29.155861990493058, 28.92061232281749, 28.63481700204491, 28.548457425582814, 28.484522737750407, 28.41234792977631, 28.40369203103369, 28.18984086408982, 28.300159913571378, 28.03785784457256, 28.056679449285877, 28.076339611030303, 27.816112817665008, 27.742509545072153, 27.917466773001777, 27.742999315304473, 27.814841049220522, 27.727734980650965, 27.671130068567663, 27.573422217217193, 27.523333691582252, 27.483510455954864, 27.483510455954864, 27.5663630025703, 27.53415128584801, 27.44407215537007, 27.420827455431862, 27.37522192497184, 27.252115047475485, 27.202500040141793, 27.092216239926803, 26.97709942250142, 26.935563127177996, 27.066875861227114, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.613048712068878, 9.541242879608937, 10.15604085862786, 10.628218173975938, 10.998571380877701, 10.893674404366505, 10.756245210336905, 11.213335347925648, 11.298364887587041, 11.17660714931372, 11.028373350894578, 10.932839440738263, 11.088776205283777, 11.078969658106349, 11.014369381986064, 11.29116061464605, 11.228996423790067, 11.463328120546603, 11.868349346765266, 12.3796739204838, 12.519645804537035, 12.471982172378684, 12.719430193639711, 12.697000609704789, 12.811933864720304, 12.908258961253225, 13.116444028424251, 13.202688875991456, 13.353353252828063, 13.455254002346576, 13.514896801424708, 13.55812336160786, 13.73699983032969, 13.884845908854269, 14.018394023278494, 14.054454978717363, 14.17521879535838, 14.350271409122886, 14.487192882002844, 14.721589922068615, 15.00891361897925, 15.208699071026258, 15.40549229377858, 15.641805321239088, 15.65551412661453, 15.834678565425014, 15.767682062616359, 15.775310380499713, 15.614590943306297, 15.407245613011336, 15.228522451523855, 15.113822990899955, 15.287625540864191, 15.471849693094692, 15.619743409718428, 15.772616501344746, 15.978981750565898, 16.191338541505736, 16.308014367455858, 16.507928213763115, 16.78943399904086, 16.732562028394348, 16.696545386105697, 16.8941331818875, 17.246163211040802, 17.499546607088927, 18.36594352852505, 18.602318047681056, 18.92565246724082, 19.321298695186893, 19.72504867133783, 20.108346185773893, 20.046764730007258, 20.851285539493563, 21.53086718976187, 21.86178053447084, 22.260170006601125, 22.755023949481725, 23.053637998519264, 22.978255683501907, 22.942334005076958, 22.863179997204533, 23.154822016625495, 23.036384210871336, 22.934580838844713, 22.849682266652724, 22.719676573746916, 22.631647071798742, 22.617653704293932, 22.59940851904116, 22.53695367976944, 22.545836951050617, 22.503630772264312, 22.448201473092848, 22.43622195943248, 22.45146187645203, 22.416584230386537, 22.374987314619375, 22.382223856943504, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.357065141155102, 9.801668759352426, 9.673989035495188, 9.563021705696283, 9.505750396330964, 9.797673501363784, 10.678470245223155, 10.659238349270854, 10.542673068229012, 10.463716123205138, 10.912153405154273, 11.040528991678437, 11.142842024527019, 11.096866716031867, 11.236030266777924, 11.205074848086513, 11.143082997480892, 11.0984639561343, 11.062911823574781, 11.20841785905179, 11.326480533397817, 11.300305683415932, 11.33121058021351, 11.21863418874424, 11.233100006623339, 11.548353318113744, 11.8658168564853, 12.030366713327773, 12.070127732456944, 12.155305779632927, 12.25216684847455, 12.379262426479453, 12.426472962969177, 12.489925542081215, 12.518673559420632, 12.799511613386299, 12.925110768707967, 13.21083765511381, 13.425756682148032, 13.732333419460208, 13.72543133922471, 13.849059796506472, 14.134548098739971, 14.31408388023511, 14.53153264213212, 14.710554471138023, 14.746197848858586, 14.840513281607533, 14.789350366730353, 14.94304502850216, 15.070262297825437, 15.214381479731317, 15.364210243635473, 15.54421531305952, 15.69569512109887, 15.865347725765524, 15.963403688474626, 15.635980123153187, 15.890477680492024, 16.119610927194287, 16.287313190155466, 16.508732147078447, 17.071807820000075, 17.531122993840572, 17.80314081527751, 18.02721141640901, 18.431223563121684, 18.735516593406913, 19.071371827717623, 19.297368075524364, 19.65677998607469, 19.698543007798115, 20.103140620049007, 20.851285539493563, 21.517434182752314, 21.859331954647136, 22.271604145525355, 22.78875332699201, 22.988094613756125, 22.963412544136897, 22.862173416059065, 22.868534744362, 22.694406920251218, 22.63083481606978, 22.62287247306675, 22.58215917066236, 22.529489454586894, 22.54708122413463, 22.473182381184266, 22.435074678855337, 22.432895456234196, 22.329294520037767, 22.289266198449987, 22.256450429087625, 22.2026116724076, 22.53508132999751, 22.445827249148472, 22.376106560590955, 22.35073989123565, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.9963604876418195, 9.160703824951288, 9.514654469031171, 10.455282097435942, 10.52934581879192, 11.0413187396933, 11.237079335873272, 11.039556598251417, 11.22059625222308, 11.349307928826171, 11.186177424998556, 11.155495622906814, 11.03860935941667, 11.008057516231574, 11.57941931358895, 11.538840789683011, 11.502375989984886, 11.477873795467339, 11.64917444535867, 11.769667348614934, 11.787839996239386, 11.897610743453416, 12.019308032437982, 12.06578231160944, 12.204861436834921, 12.267042092134513, 12.409019832605443, 12.623718157571354, 12.844435087094485, 12.855621394661028, 13.007474929786369, 13.049134232325924, 13.34267165495811, 13.516997782790199, 13.759203027875044, 13.972993600294256, 14.157112396261038, 14.32246031286089, 14.568607818812932, 14.553138251835648, 14.636433904889227, 14.76846559456112, 14.782954641402867, 14.90242804460881, 14.888690272338364, 15.023959537801366, 15.179471183997572, 15.402732601805674, 15.571038311853632, 15.462638721614603, 15.379915085993472, 15.411593309273133, 15.443308398241376, 15.550291428926203, 15.708238817514022, 15.814962389025453, 15.980966619014023, 16.128721305057, 16.206072241464575, 16.15672799538468, 16.293292937466862, 16.56411073914661, 16.756889686623925, 17.48544443808088, 17.815346483693688, 18.086736217784605, 18.449885494869356, 18.752413888679044, 19.050577416369123, 19.317109312379934, 19.701828285404574, 19.783259842119243, 20.103140620049007, 20.851285539493563, 21.517434182752314, 21.859331954647136, 22.271604145525355, 22.78875332699201, 22.98635271541698, 22.909603268274758, 22.86605470495033, 22.741227107224535, 22.695605351798335, 22.606783671872975, 22.564985587006873, 22.56936525459928, 22.533427496067596, 22.54630248768724, 22.43955658992584, 22.383512416252536, 22.36612728441098, 22.284025812047584, 22.286712663641843, 22.589641843826836, 22.560391802516325, 22.529938002607576, 22.447814400428662, 22.373694120672198, 22.34575523792584, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.613048712068878, 9.541242879608937, 10.15604085862786, 10.628218173975938, 10.998571380877701, 10.893674404366505, 10.756245210336905, 11.213335347925648, 11.298364887587041, 11.17660714931372, 11.028373350894578, 10.932839440738263, 11.088776205283777, 11.078969658106349, 11.014369381986064, 11.29116061464605, 11.228996423790067, 11.463328120546603, 11.868349346765266, 12.3796739204838, 12.519645804537035, 12.471982172378684, 12.719430193639711, 12.697000609704789, 12.811933864720304, 12.908258961253225, 13.116444028424251, 13.202688875991456, 13.353353252828063, 13.455254002346576, 13.514896801424708, 13.55812336160786, 13.73699983032969, 13.884845908854269, 14.018394023278494, 14.054454978717363, 14.17521879535838, 14.350271409122886, 14.487192882002844, 14.721589922068615, 15.00891361897925, 15.208699071026258, 15.40549229377858, 15.641805321239088, 15.65551412661453, 15.834678565425014, 15.767682062616359, 15.775310380499713, 15.614590943306297, 15.407245613011336, 15.228522451523855, 15.113822990899955, 15.287625540864191, 15.471849693094692, 15.619743409718428, 15.772616501344746, 15.978981750565898, 16.191338541505736, 16.308014367455858, 16.507928213763115, 16.78943399904086, 16.732562028394348, 16.696545386105697, 16.8941331818875, 17.246163211040802, 17.499546607088927, 18.36594352852505, 18.602318047681056, 18.92565246724082, 19.321298695186893, 19.72504867133783, 20.108346185773893, 20.046764730007258, 20.851285539493563, 21.53086718976187, 21.86178053447084, 22.260170006601125, 22.755023949481725, 23.053637998519264, 22.978255683501907, 22.942334005076958, 22.863179997204533, 23.154822016625495, 23.036384210871336, 22.934580838844713, 22.849682266652724, 22.719676573746916, 22.631647071798742, 22.617653704293932, 22.59940851904116, 22.53695367976944, 22.545836951050617, 22.503630772264312, 22.448201473092848, 22.43622195943248, 22.45146187645203, 22.416584230386537, 22.374987314619375, 22.382223856943504, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2026, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.0058165155538, 9.632219632219632, 9.504226453129744, 10.140843586349376, 10.999618758950714, 10.956616533942483, 11.279266658703664, 11.176540802603125, 11.846210826606708, 11.741606339763969, 11.624257709121242, 12.156786794692028, 12.077651035907502, 12.02547845000693, 12.156370016988651, 12.13661626601505, 12.159411857018108, 12.229484540872306, 12.25222401673364, 12.280430645912018, 12.27435426559073, 12.641066716952393, 12.620830883809656, 12.521310458314277, 12.554252017914806, 12.84467384293508, 12.988505139875958, 13.125609909046329, 13.177850718219045, 13.542885778463468, 13.658517430361789, 13.68779896406323, 13.912503602107194, 13.897452951319373, 14.016372361200032, 14.117203832272502, 14.337633506836205, 14.521443440153584, 14.750304844026102, 14.9375030530938, 15.124691983089036, 15.319505052730035, 15.386941336872173, 15.475202348106803, 15.439952130306498, 15.694735677488788, 15.78789411310933, 16.10198260114965, 16.445781084638575, 16.542192650686328, 16.898707713925106, 17.046719128765737, 17.186117473377895, 17.323907087372813, 17.705914894840504, 17.84682458159586, 18.084997478419133, 18.302140053302917, 18.4025790599141, 18.456571607827755, 18.574897456821805, 18.941902371596527, 18.929225822906457, 19.14836241794919, 19.353588295731676, 19.515483088986542, 19.663222265177705, 19.967670693828282, 20.28556326318994, 20.43659749980853, 20.716693035948335, 20.956013982129313, 21.232580261256746, 21.576055039023224, 22.044521092206235, 22.52079385170909, 22.494842991437054, 22.886444088760165, 23.06277294540043, 23.292703731868116, 23.14172448646419, 22.99282588094513, 22.86326810979556, 22.69401636524754, 22.66012760577223, 22.624560469198272, 22.575218037267327, 22.680052359438587, 22.68557361973732, 22.654571055275248, 22.5927835157607, 22.553405585162086, 22.55815572809037, 22.524449512873332, 22.486454122953855, 22.41758424704124, 22.415336453729054, 22.430962886086302, 22.40755397570226, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2026, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.74418888107495, 10.522141594891142, 11.54090300224323, 11.48979593390074, 11.561075314722927, 11.40544262075266, 11.453351764153778, 11.270841344378681, 11.493615297133688, 11.288551139178526, 11.337212880331652, 11.56352038661219, 11.47073821866623, 11.42953856831086, 11.528162519755508, 11.51752706049063, 11.61184569305626, 11.820060597774539, 12.209722193710688, 12.46097859197327, 12.501751395664801, 12.536691867834179, 12.725178558024306, 12.867665913815612, 13.086337441238, 13.117610850998338, 13.157669357359236, 13.278727323797495, 13.341040540518911, 13.637454600005988, 13.94408621170319, 14.036440870570102, 14.1749271044067, 14.275455867955506, 14.438267881554944, 14.56632872138133, 14.612756817755528, 14.696187053574766, 14.799616178435118, 14.908296667019838, 15.019821353281475, 15.175507236825696, 15.315638878685323, 15.431010337473902, 15.669196983552133, 15.8182416055855, 15.925768882160218, 15.65111166087863, 15.806365198438604, 15.675307439377775, 15.902469177007386, 16.0958257776281, 16.2983489036786, 16.577103176032935, 17.040237981110813, 17.18208934779174, 17.260133241953223, 17.195628810568962, 17.75644474414966, 17.806984825425907, 17.69369533707501, 17.801944944647254, 18.773731410145665, 18.91065545511049, 18.974050292677475, 19.215728945152286, 19.428398616855368, 19.712428217842657, 20.029456027395153, 20.275491159509716, 20.49047560519821, 20.73150140605147, 21.42911752256942, 21.762853452337826, 22.05586419118947, 22.39202963945872, 22.597759522663885, 22.852055755993383, 23.04508490040218, 22.876564327890254, 22.79290666879218, 22.76872693642038, 22.6763121413378, 22.588323687366746, 22.570085189505328, 22.46166465043974, 22.334370038498097, 22.268334297945703, 22.216805605976333, 22.144713095271797, 22.22564637906828, 22.51836506024317, 22.466639873917384, 22.48568316730036, 22.41828605077671, 22.418280876778326, 22.379780738107456, 22.34824237270596, 22.384575113702237, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.963931333932589, 8.675616963528674, 8.695467092433637, 8.682294329816637, 8.772395336460088, 9.289603224331053, 9.657730310956117, 10.036610528543818, 9.911038488810362, 9.89413748624766, 9.892161407134461, 9.938596462257854, 10.228410167543677, 10.220048268236898, 10.512323516811536, 10.503192118026258, 10.50635154117847, 10.951334728934205, 11.373284458794702, 11.502540728773454, 11.553001516678533, 11.53338352052063, 11.726269771605972, 12.060736024391993, 12.240468335026069, 12.427213187704064, 12.734514010122611, 12.650886636919786, 12.83679698468701, 13.27845002787951, 13.283329318013445, 13.407394258681144, 13.476713038209715, 13.78084094997772, 13.799947864055154, 13.793061436607482, 13.837361247750179, 13.95740051771179, 14.093122067500923, 14.329272380009398, 14.40688593072626, 14.656918322383733, 15.00169070047565, 15.18375349825318, 15.466495764703176, 15.638120875358643, 15.903141070991234, 16.155220245871668, 16.37721366688636, 16.703523912402463, 16.856856008174674, 17.155844168495392, 17.301343179024382, 17.5798643459641, 17.71545891106629, 17.803053340597668, 17.901077139421755, 18.025955690514266, 18.19295224694471, 18.35531539700718, 18.238685579283292, 18.29338783162716, 18.581356687987395, 18.802431412664983, 19.09780178567128, 19.296198076665057, 19.628287696100777, 19.93854808459844, 20.218750556387203, 19.97254527828479, 20.337982990950394, 20.67574787379676, 21.0050147435186, 21.425889776420007, 21.909713724440106, 22.36848706208908, 22.731715665596884, 23.21720583644677, 23.035885111520223, 23.039386790651427, 22.988794507758545, 22.91245657931193, 22.926324104992453, 22.812281382613314, 22.648054705790834, 22.65142581426176, 22.568585318867566, 22.8917234551864, 22.854351060304655, 22.81555922802171, 22.718961161055294, 22.63113576284632, 22.59154331608929, 22.548153227089372, 22.51345604468286, 22.46164286889683, 22.397888243728126, 22.371225344842124, 22.349590206226623, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.608789672202711, 10.497221917152357, 10.617366593245531, 10.798732767966825, 10.5460835967482, 10.46757321756562, 11.134380008906398, 11.30356175324125, 11.167911825884019, 11.055896061378519, 11.248853160313592, 11.606309674420274, 11.749195213582393, 11.934991637588128, 12.094535106190813, 12.308790322734877, 12.395750164645046, 12.306568926514073, 12.301587211001133, 12.416517487915895, 12.57756804123722, 12.672975797297187, 12.719595745453793, 12.761094918201113, 12.8663866067734, 13.026000720929312, 13.321800920768448, 13.360706717122111, 13.333130424360288, 13.456742113696391, 13.548429816847158, 13.679350059976755, 13.736223624428053, 13.969901491009603, 13.892138824497973, 14.140437457290783, 14.258589086007737, 14.241282263701269, 14.355742732073617, 14.453102645688984, 14.55920335119822, 14.560106379268975, 14.591777817640317, 14.914725574713476, 15.037869398388427, 15.208430977442289, 15.265437881724129, 15.43963067569108, 15.586219767646242, 15.65243055013654, 15.913343876650673, 15.952713509756085, 15.982036239881166, 16.050171689107383, 16.009155202475565, 16.033956047161357, 16.134688947447145, 16.074128997618757, 16.373767814506195, 16.55147751709151, 16.628061695786155, 17.102322162036447, 17.289346710610754, 17.46349269600805, 17.925846327520397, 18.18285556352628, 18.49371919840152, 18.63183401935251, 18.897120715414196, 19.2080588405611, 19.49627161924821, 19.81856496102254, 20.129448175437616, 20.47677330441576, 21.053386132360966, 21.53726014128038, 21.99488164152873, 22.501486182280527, 22.633789512878273, 22.580899288011054, 22.692775280564025, 22.672524436681357, 22.550837740182732, 22.92355223018412, 22.839077717278123, 22.82051730513511, 22.804738637898957, 22.765461088583905, 22.657681583587713, 22.57622504843674, 22.56461450310854, 22.56587996568961, 22.58337371644003, 22.602781568989485, 22.54442318693033, 22.53145741713746, 22.45514192616842, 22.408927577832586, 22.407766680063894, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.306649840208667, 9.743182194421527, 10.004865035431983, 9.816216274970763, 10.748793993807801, 10.582076863225181, 10.53908016518077, 10.70287275295165, 11.064119793444933, 11.350726413227722, 11.365924451604679, 11.419393805381596, 11.91264025115066, 11.86747557916325, 11.926982742731546, 12.153643786493623, 12.093584118986364, 12.463309087855016, 12.435305178719256, 12.49868722266946, 12.645004667871069, 12.685917063429129, 12.888051957077607, 12.930298670284134, 12.932553308000466, 12.844922968948921, 13.086827757803004, 13.311402572889982, 13.395773306295922, 13.448365006091107, 13.463610415291209, 13.372449436724043, 13.392248564231183, 13.515078677137597, 13.68716108284111, 13.828339151511098, 14.067727937850744, 14.148224603517306, 14.234800363804665, 14.241490416411896, 14.410302242245468, 14.549088265556867, 14.774970776902533, 14.825130756663638, 14.980926620386361, 15.015754985754986, 15.107367894756706, 15.342765980209675, 15.21637220857554, 15.207889258691399, 15.354204082792435, 15.601417785532135, 15.551854938422007, 15.618330586478724, 15.799024505258954, 15.68118365671344, 15.791844736719298, 15.933007416773837, 16.241624503178397, 16.447309194782473, 16.663799268109784, 16.877261918825496, 17.360373450182344, 17.734303605477038, 17.94519000854401, 18.262521500663944, 18.486201926275655, 18.63183401935251, 18.897120715414196, 19.19259925524196, 19.461128511741794, 19.798181294141244, 20.130402955951322, 20.456276823210928, 20.932965769760063, 21.520538972917798, 21.952946146268808, 22.63348892943317, 23.006038519462482, 22.819595902474948, 22.76527536307498, 22.709045974814824, 22.57663702085609, 22.52440208713973, 22.50784136886886, 22.459429789501225, 22.509403802333097, 22.407871169666222, 22.69827269518934, 22.719994562893536, 22.68702708338047, 22.584921770007522, 22.55179361923231, 22.51366207110412, 22.464816121326454, 22.414724592712158, 22.377623083682106, 22.40201905381528, 22.39398722879363, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.620940750066689, 9.109616143233039, 10.843457937544304, 10.885984108742777, 10.767882408292213, 10.644269474504698, 10.599684919651683, 10.762812957796774, 10.919659610984564, 11.316302928410552, 11.515736195971057, 11.753077206367735, 11.896760401637925, 12.244774111226615, 12.192951537369966, 12.124395411606367, 12.363968408216103, 12.539149890078347, 12.7864958086511, 12.699816260829099, 12.725315689266106, 12.898443089038159, 12.95472648139229, 13.091654377314057, 12.880281991712957, 12.960917280652225, 13.085434189080642, 13.271944699710419, 13.418765865986224, 13.565448206645184, 13.791606010312723, 13.853031458629692, 13.943123872648933, 13.991492151066742, 13.968010112409019, 14.155152205553097, 14.239224579702892, 14.421145454420284, 14.311860645996614, 14.346285539833925, 14.432675053329458, 14.742457801578409, 14.874129060012613, 15.005057794892549, 15.056386456699267, 15.12646311925035, 15.379656745154913, 15.725523322022397, 15.808279828542005, 15.96631752303686, 16.035983516717607, 16.08151421862293, 16.320556834421712, 16.354165616455962, 16.490353843785453, 16.77751121637219, 16.667204832602106, 16.595730268425303, 16.64788550172694, 16.69324229272573, 16.800480206180378, 17.191701820365783, 17.344593901933123, 17.751771443551302, 17.974342112030623, 18.27327346304048, 18.50210666735417, 18.63183401935251, 18.897120715414196, 19.183350334668972, 19.461922616780505, 19.799031375989195, 20.116310344860654, 20.458774325708433, 20.917448416321875, 21.483017812433385, 21.952946146268808, 22.615543536043837, 23.007766351378994, 22.81790926243944, 22.74659666555601, 22.71064619042282, 22.57720883476316, 22.5410217865676, 22.50784136886886, 22.465043111593708, 22.499318193005223, 22.4084605180141, 22.3376585790508, 22.776338484334833, 22.68756240131955, 22.585741716252848, 22.55179361923231, 22.51366207110412, 22.48065554260709, 22.413959289397738, 22.379532745824974, 22.400826931360463, 22.395159284311113, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.618330710811698, 10.522158270445942, 10.659925191397617, 10.819025010295135, 10.57969526067769, 10.487255901163897, 10.914288994828732, 11.122876515790276, 11.228360394540601, 11.106707244348252, 11.073615391747513, 11.478934316873417, 11.7848804933332, 11.921335552494064, 12.369451192374216, 12.525341232289772, 12.437235622345128, 12.435181650004164, 12.440367197507959, 12.58817878956891, 12.569370980544663, 12.618447532685378, 12.809833961583914, 12.81110445459262, 12.871599283001656, 13.035427143354891, 13.171309053898039, 13.509373148112402, 13.412659930698451, 13.474446922644704, 13.572393979186515, 13.60889529868323, 13.746334580481273, 13.79628994963819, 13.996250708357191, 14.194460416203258, 14.272728798905426, 14.246671232376954, 14.29707686905617, 14.481261407693893, 14.583428981359187, 14.55933444299139, 14.617774283930856, 14.928167795962807, 15.017805298963765, 15.254151284056686, 15.275236373867854, 15.410168133079729, 15.505098725314587, 16.027024128570343, 16.023540124664446, 16.049682463278792, 16.156710266671475, 16.239452459047403, 16.136941059400257, 16.096099282548483, 16.16341442005852, 16.184932494170425, 16.379051372116436, 16.53638207391665, 16.715066053743556, 17.11929009005682, 17.317632042001733, 17.460452535487573, 17.944331321572303, 18.224026873872617, 18.49371919840152, 18.63183401935251, 18.897120715414196, 19.2080588405611, 19.46272739995443, 19.815921573087387, 20.102077219202847, 20.42898498360715, 20.999417398462043, 21.510468198623855, 22.00775618154351, 22.648181552300038, 22.837278323351462, 22.927142361641657, 22.86385558220873, 22.813629888949666, 22.718560038131102, 23.04785807366119, 22.886111813720497, 22.835722444305556, 22.754089148184427, 22.71752195978246, 22.720475846768203, 22.61457190730851, 22.542268575303872, 22.614557707128917, 22.589822782937492, 22.529918699970043, 22.566786000230053, 22.545129565971834, 22.442417403252293, 22.392913663623485, 22.408381361441645, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2027, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.957161625851372, 8.00894189874453, 8.939579153529317, 8.90836035727394, 8.85106539843796, 9.528728465658713, 10.084467184074192, 10.526796707506081, 11.132832167943599, 11.015723260075214, 11.162217826385742, 11.231788959659378, 11.206475088410677, 11.377493253556098, 11.746175163540734, 11.697819032152584, 11.864134932720846, 11.817735184722325, 12.012371390735261, 12.155383438282815, 12.360395249264029, 12.36018387580476, 12.491640684917872, 12.487175092890917, 12.637089662297324, 12.690718208637175, 13.065584019253334, 13.070081209191962, 13.137388085779339, 13.445346896177574, 13.524657646335442, 13.768612295312394, 13.825777403734376, 13.89870186641293, 14.054389613859325, 14.24746889975764, 14.352543478935337, 14.372919491420147, 14.510523029918728, 14.686532497887045, 14.745731503900405, 14.919908670638412, 14.979026161131229, 15.119935536697565, 15.159421957233318, 15.380947058195343, 15.528023226426852, 15.746100367656215, 15.768221947836919, 15.830671777519154, 16.218960985646877, 16.303523736693997, 16.546579374338556, 16.576086090957638, 16.838783798460664, 16.985025769063753, 17.108397629311035, 17.443367139293706, 17.656922206972208, 17.87816043874326, 18.09205234144208, 18.568954335440427, 18.803556141918666, 19.1661469604462, 19.120269152323488, 19.01674447674562, 19.2281981121843, 19.38584072855834, 19.581577462796613, 19.935998610922294, 20.310869296938282, 20.616595564668277, 20.838551628483177, 20.91720847373115, 21.2087680263953, 21.721964362001987, 22.131419285861483, 22.33086746403321, 22.37499341285975, 22.549351936066707, 22.56524186776949, 22.600004033610396, 22.694234255401213, 22.756012492027246, 22.705001822795907, 22.608916723877474, 22.603280677964253, 22.544017010196413, 22.62172942645575, 22.539060476259255, 22.531512083736512, 22.479861607085184, 22.523502376971948, 22.516566317873, 22.526903362358873, 22.486897502807434, 22.396370909377282, 22.334157457982066, 22.277190377716884, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2027, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.195229870042525, 10.079614872070977, 10.965314467181907, 11.003527107733955, 10.822092305471319, 10.977178732815622, 11.334219190203042, 11.664666584275677, 12.335746763855052, 12.075988279556624, 12.234041862611802, 12.105380690550295, 12.38670904499203, 12.55242432724946, 12.578845248833131, 13.025349583909385, 13.217722935590093, 13.161410251553205, 13.126195459391907, 13.168532944996342, 12.989491108461326, 12.987008080324843, 12.809538347690932, 12.791766938528168, 12.818353145225615, 13.127414266827234, 13.258806958743532, 13.37874622352906, 13.54796500735587, 13.663222968943527, 13.742017992346073, 13.785240453826148, 13.846595807239797, 13.825849694749023, 13.740064283140569, 13.714668868931646, 13.794006392106157, 13.808139647304404, 13.796710485483777, 13.902882870087055, 13.923755190920922, 13.85121548219643, 13.888881713058504, 13.873868592333205, 13.894234812133513, 14.266448570213125, 14.214443059859319, 14.276973849952709, 14.33514312790689, 14.453498708009816, 14.572771727398875, 14.728871453795852, 14.78613286911833, 14.928616000897236, 15.08466383608796, 15.239865181128328, 15.36695949303239, 15.51996684801695, 15.605693916795646, 15.808241766724574, 15.881761839954923, 16.234018119097843, 16.307624975058864, 16.59467125544982, 16.812367171232374, 16.96917526866156, 17.187023083612225, 17.34161365735417, 17.76836048610874, 18.116030834065423, 18.992647253411096, 19.323470307727085, 19.76849496603871, 20.23234189511943, 20.738737588664964, 21.20526589112828, 21.662056984125726, 22.473776089574375, 22.75606539336643, 22.871567726193327, 22.8830926380637, 22.83711751447368, 22.828852835631473, 22.83405089837234, 22.77209505470375, 22.732731980091856, 22.697838330518934, 22.641681885371295, 22.554957683995724, 22.518613065381192, 22.463150140808256, 22.475775489872756, 22.47582439015102, 22.373760856922964, 22.32652421439838, 22.27483102444097, 22.510728192929182, 22.41904877826465, 22.433081314105188, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.647457025448084, 10.269704824073, 10.491270226519461, 11.12057780436558, 11.252766273012684, 11.939379481715608, 11.728163272508612, 12.01231803071934, 11.865913857156588, 11.726643449439145, 11.828443796954572, 11.799886577585733, 11.794618822411529, 11.803091327361132, 11.787561063323187, 11.950854180028008, 12.259815105320275, 12.373351844906304, 12.547447692735748, 12.656036697502305, 12.901226664768512, 12.89364257475834, 12.935869360108073, 13.084091208641011, 13.258157129829184, 13.450371198555347, 13.872852273397623, 14.090715273437052, 14.203877543508034, 14.175218129291652, 14.183476490242079, 14.32062225347821, 14.385479588003653, 14.234390901983557, 14.333124087708867, 14.469913881352356, 14.544352778895616, 14.642250383655945, 14.789675422936336, 14.967058987100518, 15.136227957310625, 15.285995154386505, 15.534548451530386, 15.788327857587465, 15.949572204051343, 16.15871113517491, 16.10520049478616, 16.413452848349642, 16.548358813598853, 16.668448060358006, 16.850879103083173, 17.05914560292309, 17.01321721168474, 17.149287514986625, 17.188108523390646, 17.389517599562996, 17.775794358191696, 18.37207415991381, 18.630988051155757, 18.795219744976222, 19.01073572495798, 19.23553887860682, 19.742946291253475, 20.320425382383167, 20.827904381287933, 20.887278582930755, 20.6436889958503, 20.84606525225617, 21.09302918935128, 20.87185684705439, 21.112743100825725, 21.170371928607857, 21.496521417272973, 21.7933100633946, 22.174900507029776, 22.519363078856987, 22.899477133958356, 22.93677930397971, 23.081911721120214, 23.058384490063272, 22.891810659079393, 22.868534744362, 22.755702868042064, 22.824257994672216, 22.782810317390172, 22.736236233992773, 22.65098325530459, 22.612465856825146, 22.61141988994257, 22.554989538249988, 22.46706482989484, 22.382151204636276, 22.30040147741538, 22.30929132389892, 22.185697381470458, 22.0984968297378, 22.049248023698905, 22.370806728112687, 22.347414206976115, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.974844828206979, 8.093010458084697, 9.036116029580567, 9.756985736213577, 10.938237658470227, 11.617024624134805, 11.44447616305532, 11.683773875666606, 11.588765655302458, 11.45903724784516, 11.768215648547093, 11.695599555616045, 12.028576359917903, 11.947658150331442, 11.960912066554044, 12.279203787144134, 12.295887076547455, 12.292117909491319, 12.276624077616564, 12.216479157592937, 12.381235790173816, 12.388789617288742, 12.400797967516388, 12.663061295915687, 12.637013641470945, 13.021793093322763, 13.042010486334643, 13.326337254675991, 13.563937906157259, 13.656617734745161, 13.717996555110364, 13.863057730094372, 13.973051146044174, 14.170962853094474, 14.290780158107708, 14.30444113537876, 14.487312793320006, 14.630320814357395, 14.767707464989844, 14.836448947416162, 15.150606729718302, 15.27756592353089, 15.37373751720005, 15.445817958666156, 15.629014005264871, 15.883114894223983, 16.058914760987207, 16.053997771708307, 16.15240608813483, 16.188278416023685, 16.31290677242381, 16.518020300998952, 16.688345937446456, 16.871194923311332, 17.168216469521056, 17.24521369959619, 18.040608588928126, 18.28426588683795, 18.407768236777912, 18.568095184803617, 18.732974707578073, 18.83996392120647, 18.88563694294579, 18.751279645029012, 18.874190594078517, 19.232129118709043, 19.450342356037623, 19.3253376554431, 19.55719867468304, 19.85005123739697, 20.153631260045827, 20.479034891246346, 20.84752966808687, 21.09648196112142, 21.64023179651016, 22.20166692958944, 22.728196434687284, 23.24101942404284, 23.41320310178056, 23.281910597162835, 23.224918362312767, 23.091550736086838, 23.053179083129326, 22.902244709397003, 22.769755313526005, 22.74141605973617, 22.761626554212548, 22.76992830731607, 22.607448285181878, 22.589019943499757, 22.61834718112649, 22.582398773417882, 22.533017626882902, 22.495310790150306, 22.446057013734368, 22.41109035976242, 22.425287121917112, 22.39980797561553, 22.35287825853484, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.721235575005252, 8.68735727789199, 8.74992781984269, 8.787833905111386, 9.407331511244587, 9.86861698566389, 10.270248478052066, 10.223145116309523, 10.377042399109637, 10.340407907425512, 10.251766259228047, 10.178127320202284, 10.43497746446181, 10.603368224594835, 10.540683290137883, 10.495983078911818, 10.497744406047982, 10.465447049474623, 10.470864068292336, 10.500202461406719, 11.077108357224287, 11.112918117296068, 11.792507716840682, 11.808677038035968, 11.803971537294586, 12.048995255084826, 12.085098517124147, 12.124735374589008, 12.278543040410138, 12.398309178812415, 12.437795541219495, 12.577857745306877, 12.535374704610854, 12.864988199295016, 12.94425611460106, 13.235231859580626, 13.484781344927258, 13.607567326641021, 13.735570988121108, 13.899188329381825, 14.121884060719195, 14.302147537796337, 14.491001295530449, 14.706660564610747, 14.825037106086587, 14.91021535366409, 15.017823820257604, 15.27640705787884, 15.491897637641861, 15.587720777852713, 15.922805182153969, 16.09631105007066, 16.427959702548222, 16.633296090733708, 17.2945486949124, 17.480392791123457, 17.66600239684021, 17.801718273348325, 17.944255026830692, 18.250987927310977, 18.4544316518966, 18.39344049891644, 18.525295337521886, 18.789138380716334, 18.78764218940091, 18.747670240523817, 18.924107890432794, 19.24671606007419, 19.569634966467227, 19.79635527615996, 20.13583361862484, 20.45144882617058, 20.885668084244994, 21.149564359932462, 21.64023179651016, 22.215018648842836, 22.711017974518175, 23.39291499119791, 22.989841239092122, 22.822029467890413, 22.74016832551668, 22.68710283342792, 22.59461027870439, 22.552861078938154, 22.619464295471897, 22.955647517862765, 22.914387788689982, 22.89520671453217, 22.856347657699807, 22.826687548033295, 22.712749937260938, 22.674968429865537, 22.64152874292929, 22.53488460532947, 22.506477762094203, 22.47484615525653, 22.44464851919032, 22.372020874109534, 22.352443813793744, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.9814564424080086, 8.065378767727607, 8.176681589245508, 8.249090091195354, 8.783647272067935, 9.160510970065685, 10.202381710751771, 10.421431541878135, 11.176474869261838, 11.627147087273798, 11.753918472899318, 11.860044524732274, 12.001447652042922, 12.164222994908538, 12.065641668513159, 12.209673804033478, 12.419332675378172, 12.42341812849645, 12.61371494069203, 12.553305275453882, 12.521888074748059, 12.469253944584931, 12.438495833826881, 12.34098984227248, 12.603051577825623, 12.79659222020713, 12.876418463113165, 12.932814757616681, 12.887526222146542, 13.079611866086077, 13.252602464793245, 13.448289927457747, 13.568360659882142, 13.72158125660726, 13.855585127002593, 14.023322051970327, 14.23284958276913, 14.377011729741774, 14.512200967214428, 14.543794777429364, 14.634707381192985, 14.8140269898489, 15.023417865681225, 15.15477772573877, 15.377231332750327, 15.660358601454277, 16.0437923747001, 16.234668266231488, 16.331162638943482, 16.46577781416591, 16.903479907995017, 17.27231555864542, 17.60173662400673, 17.745592508007853, 17.934293457854704, 18.069400876169468, 18.320685664475754, 18.605788947328243, 18.944833741461128, 19.05874273452933, 19.102649702446783, 18.960766707993514, 18.96262668779433, 19.05956514824925, 19.265510598067458, 19.409268371765314, 19.380415992368697, 19.54863648290908, 19.787291237464792, 19.95874927811237, 20.186480000236337, 20.622422649537043, 21.002928254314728, 21.16228658574957, 21.64023179651016, 22.186410018239695, 22.711017974518175, 23.378858786098313, 22.99435776261973, 22.78125691998513, 22.653180421253573, 22.952784893731536, 22.923415676269656, 22.871543901903344, 22.879078322047214, 22.96201098332824, 22.9143155478467, 22.93029043047933, 22.796378538073654, 22.746414794186066, 22.71507656579087, 22.626729354950037, 22.569748498318752, 22.53763880012185, 22.506872750586602, 22.47445137289269, 22.394088521650655, 22.374120573345554, 22.349316419016986, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.974844828206979, 8.093010458084697, 9.036116029580567, 9.756985736213577, 10.938237658470227, 11.617024624134805, 11.44447616305532, 11.683773875666606, 11.588765655302458, 11.45903724784516, 11.768215648547093, 11.695599555616045, 12.028576359917903, 11.947658150331442, 11.960912066554044, 12.279203787144134, 12.295887076547455, 12.292117909491319, 12.276624077616564, 12.216479157592937, 12.381235790173816, 12.388789617288742, 12.400797967516388, 12.663061295915687, 12.637013641470945, 13.021793093322763, 13.042010486334643, 13.326337254675991, 13.563937906157259, 13.656617734745161, 13.717996555110364, 13.863057730094372, 13.973051146044174, 14.170962853094474, 14.290780158107708, 14.30444113537876, 14.487312793320006, 14.630320814357395, 14.767707464989844, 14.836448947416162, 15.150606729718302, 15.27756592353089, 15.37373751720005, 15.445817958666156, 15.629014005264871, 15.883114894223983, 16.058914760987207, 16.053997771708307, 16.15240608813483, 16.188278416023685, 16.31290677242381, 16.518020300998952, 16.688345937446456, 16.871194923311332, 17.168216469521056, 17.24521369959619, 18.040608588928126, 18.28426588683795, 18.407768236777912, 18.568095184803617, 18.732974707578073, 18.83996392120647, 18.88563694294579, 18.751279645029012, 18.874190594078517, 19.232129118709043, 19.450342356037623, 19.3253376554431, 19.55719867468304, 19.85005123739697, 20.153631260045827, 20.479034891246346, 20.84752966808687, 21.09648196112142, 21.64023179651016, 22.20166692958944, 22.728196434687284, 23.24101942404284, 23.41320310178056, 23.281910597162835, 23.224918362312767, 23.091550736086838, 23.053179083129326, 22.902244709397003, 22.769755313526005, 22.74141605973617, 22.761626554212548, 22.76992830731607, 22.607448285181878, 22.589019943499757, 22.61834718112649, 22.582398773417882, 22.533017626882902, 22.495310790150306, 22.446057013734368, 22.41109035976242, 22.425287121917112, 22.39980797561553, 22.35287825853484, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2028, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.699121749712788, 11.262109526330018, 10.844939657550539, 10.915136231196811, 10.741454542787608, 10.910076146054582, 11.047767302249255, 11.226256595513464, 11.360905105001788, 11.131720605068034, 11.067739527345779, 11.013127102757677, 10.976489221967373, 10.978777303922488, 11.138641101154278, 11.16975673047151, 11.653061202313541, 11.733365275750959, 11.787703045030003, 11.911017499398318, 12.362856634082988, 12.473803870621273, 12.488133846356021, 12.530579868309822, 12.674031307967711, 13.070553849090235, 13.265482454493124, 13.27811151651741, 13.48480772681701, 13.94102048548847, 13.961275499397313, 14.028365619728076, 14.12322210227637, 14.206716822373732, 14.386279301753788, 14.469107172471185, 14.539527310141773, 14.810156805615321, 14.831607804153357, 14.99025918035222, 15.122398265885142, 15.284598558423243, 15.336461829247462, 15.584334710422926, 15.720551043108392, 15.89489424942396, 15.82910307844006, 15.799850470189863, 15.839183127199128, 15.747482984925204, 15.854905852570733, 15.863047492597627, 15.98451077458315, 16.064031373361157, 16.17605501177151, 16.2849965996252, 16.14718599583784, 16.31492764624059, 16.586478288895915, 16.708903539949173, 16.93398769386183, 17.214901977916526, 17.401251290879156, 17.740346207583148, 17.850093541386876, 18.14339680482882, 18.36129741847428, 18.647439495322963, 19.215449633070364, 19.49896145356169, 19.8655514128006, 20.432486721341544, 20.981724092717073, 21.327946067211403, 21.684388710523763, 22.05081209778238, 22.345782957525582, 22.759335700554445, 23.052426796573627, 22.977713461557506, 22.94401169149441, 23.28560268083913, 23.109255688567924, 23.026768741959856, 22.912812678286297, 22.83393685642266, 22.715039810072906, 22.567583971547396, 22.571389079077885, 22.536496961051668, 22.49917938760013, 22.504087615618577, 22.44632163250333, 22.411165576703063, 22.37787468963475, 22.395617750048803, 22.360430212968808, 22.358625485665126, 22.26540383549397, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2028, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.643419322059142, 9.223877260309372, 10.055349885126095, 10.287402124038845, 11.089635977598801, 11.792659773830113, 11.473368960502633, 11.307558516637533, 11.488928336729472, 11.467997651203758, 11.42222279181505, 11.573177484626623, 11.851381979254853, 11.799521287813745, 12.136803305112597, 12.243656486420942, 12.116007520254946, 12.117456819012915, 12.571330744251979, 12.569712131504252, 12.580758749041058, 12.553300214990104, 12.56099594764769, 12.598695651280181, 12.925567724788442, 12.973357590479317, 13.0317955963832, 13.170129339587252, 13.23490701010128, 13.276068478848574, 13.619795885129907, 13.699266148592912, 13.79450440611763, 13.986550386507707, 14.025326575964236, 14.421349438929745, 14.52454087484213, 14.786315082529624, 14.889976560156034, 15.024680428033832, 15.132416957296272, 15.311666508900403, 15.39538154072177, 15.579312635743934, 15.675794258281837, 15.724368934471059, 15.831252567830944, 15.980396990731577, 15.929908556936287, 16.00105086064194, 16.166101909789553, 16.34571393331811, 16.555814831581745, 16.538544531260406, 16.628317096539753, 16.820118828903613, 16.91487642899581, 16.99177059646346, 17.26501254592486, 17.45717535137619, 17.713968469668, 17.651716080882736, 18.233563953788895, 18.60710972345001, 19.23023718489361, 19.775932466882658, 20.004222502071038, 20.359399148238495, 20.739759935463745, 21.02689251889589, 21.288232013841547, 21.732158520173982, 22.077043411628797, 22.345149970889658, 22.66745734103353, 23.016784927829935, 22.741832283359173, 22.96499865566159, 23.05470838694742, 22.97993514831856, 22.949002723657035, 22.86922432871325, 22.75718258122983, 23.05323643953757, 22.96179060230443, 22.858365494191762, 22.821031133178522, 22.826763729995147, 22.726007265098534, 22.689230891970976, 22.580257699229957, 22.592084386202032, 22.497837893911957, 22.593295644009647, 22.60719281134422, 22.614079124084192, 22.548677591411735, 22.479239004145043, 22.423311600132518, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.302391924591555, 9.161537780694976, 9.520541739166527, 10.184606897755597, 10.762316536831747, 11.332823202334035, 11.214940396787835, 11.109705850467497, 11.31271937047517, 11.706429465912887, 11.739214642049092, 11.654235712541452, 11.537611783601372, 11.965846477706354, 11.936002305103624, 11.886421040611863, 11.983373371705369, 12.122236979254268, 12.092945189112292, 12.20537757437071, 12.149973911878321, 12.095118279017743, 12.194106507216768, 12.245744024848229, 12.294052713701141, 12.446242573283849, 12.48666353328942, 12.615828268785808, 13.207215016793175, 13.321735177107758, 13.49879924127084, 13.534467590479732, 13.755766703217892, 14.085329031637523, 14.221986041075708, 14.390968663587717, 14.247576300046596, 14.357882010347597, 14.445828117533354, 14.470787787240248, 14.703396053778073, 14.843858065217516, 14.884686183187082, 15.039345646272434, 15.184138297520244, 15.232611969574418, 15.577241788910607, 15.679477071643527, 15.99306004513012, 16.148741511837883, 16.521389046142783, 16.587312386492776, 16.663890622360473, 16.81375988015358, 16.962613532789963, 17.153911111228695, 17.081562064848345, 17.2018916534864, 17.35361209767312, 17.33045460178407, 17.326644286630344, 17.428165157584885, 17.490894877485268, 17.48925288998941, 17.750825180470788, 18.02186515631894, 18.268424820342627, 18.63085897407639, 18.92471897108754, 19.17339587211629, 19.35838629791955, 19.78968368273154, 20.14261539754985, 20.62601295829144, 21.007774904833727, 21.53953183973992, 21.99408366712643, 22.575385574491534, 22.97413263026934, 22.818516278706248, 22.756468456363127, 22.678625940787676, 22.581295531310662, 22.860039294626507, 22.83998365753058, 22.842024698610857, 22.7411425796916, 22.638577886064716, 22.592098400079983, 22.589232700246015, 22.64977187139384, 22.587337055530398, 22.55386341429969, 22.51104627543513, 22.39409470453571, 22.360157632222847, 22.334470687588535, 22.397590909981787, 22.402659206482955, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.952150617151155, 8.065917395882142, 8.583402435337556, 9.01099167358381, 9.292066369344669, 9.906481078622727, 9.845458117110645, 10.868525464112048, 11.043901155411632, 10.998503749586447, 11.081598838159325, 11.21251050145035, 11.115440577029506, 11.502036364437195, 11.671632546552344, 11.618445938846213, 11.658096076840396, 11.842777623071207, 12.000248940059375, 12.056678269706273, 12.359415818756547, 12.449704564097228, 12.52909775460928, 12.599844648885139, 12.714674998544684, 12.712315079860344, 12.795628508504672, 13.079629751138233, 13.137942302607716, 13.388520623999216, 13.528933211776273, 13.626363717751245, 13.828474233235442, 14.026982133462026, 14.172767392565024, 14.267458069157787, 14.422637785469645, 14.495904068658593, 14.80542798930125, 14.916849553622505, 15.246611300597273, 15.439199795354181, 15.479761011183859, 15.603986209668838, 15.869860805999863, 16.034648564457697, 16.19579481479608, 16.38778275769073, 16.48789225619182, 16.48348430107461, 16.814575526484443, 16.976163668815595, 17.086498668486154, 17.1822520929445, 17.501108458353233, 17.560904389629297, 17.889651529726937, 18.079959149576304, 18.269661644538573, 18.392487437805364, 18.517366510000137, 18.346501664682677, 18.414962475289727, 18.40612522107499, 18.48602186837389, 18.658966766406806, 18.869188842725592, 19.122081251976958, 19.29355472110555, 19.61614131764728, 19.97288890217705, 20.248563458111445, 20.62352615108446, 20.71757580544671, 21.143860047922125, 21.586032376853684, 21.830294210453605, 22.346914341839213, 23.043046450721427, 23.426608109938478, 23.554511931044104, 23.52149113777309, 23.433961600822162, 23.48866534808349, 23.04102494691122, 22.98711789528185, 22.808786845824656, 22.745181961556725, 22.708644170400774, 22.630773065828762, 22.66589624093887, 22.62173957206171, 22.546229219291646, 22.510290899367526, 22.436200081059873, 22.390966990501592, 22.39960013782691, 22.34017378454685, 22.33047318932122, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.980610756703929, 8.7748042563933, 8.78772152390352, 8.772018168088481, 8.698547426125488, 9.192200804885415, 9.206855915533149, 9.146436523350504, 9.184291946920535, 9.092963096499613, 9.135415620294467, 9.142065028032421, 9.792689952157877, 10.33927762094938, 10.653702521906244, 10.857673709895778, 11.052503847882585, 10.939143954611092, 11.286397228437698, 11.300557318201804, 11.29468185947057, 11.322178428199072, 11.280206090525228, 11.228079385826671, 11.424304671083704, 11.613413790643166, 11.734952062419325, 11.905936880641654, 12.061835319442359, 12.109848317410188, 12.275376502001901, 12.726324887289623, 12.770909761505141, 12.828560379193076, 12.935858163866401, 12.995508152192567, 12.847486205506058, 13.302799969231327, 13.340456524195716, 13.332750016856892, 13.499133665589849, 13.603113643959025, 13.792034824375751, 14.137298042603726, 14.414044795806952, 14.593539143620909, 14.770814466858088, 15.228801904762488, 15.31039682192291, 15.642298366077762, 15.840634167218266, 15.998028314730991, 16.239107520079404, 16.553091920889244, 16.6560137187759, 16.900446489549783, 17.03685404971062, 17.488407992264236, 17.65589670835394, 17.705308000446838, 17.593623665052235, 17.567266753199522, 17.707794140914576, 17.916037077463287, 18.103875311707714, 18.492875476762876, 18.828303826796425, 19.022353595327466, 19.329550788514144, 19.599243157411756, 19.983375003653514, 20.301877973220357, 20.634437228052892, 20.820662277891763, 21.27815291942716, 21.69711739135594, 22.23606442108766, 23.071188884039696, 22.974265579982205, 22.962284802728803, 22.838979056702534, 22.72375916568957, 22.68034051993557, 22.70540706672692, 22.653957907076, 22.612162862712196, 22.53279957587605, 22.523467163436074, 22.46084247106299, 22.7231529984529, 22.671820793338956, 22.618507129665517, 22.580383430614052, 22.552025132659583, 22.467839335183076, 22.4736785389543, 22.445507291347173, 22.373254707948544, 22.34913872196928, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.974860658202086, 8.6972659608597, 8.646647349861672, 8.650877141267872, 8.674955148607694, 8.640469296869355, 8.678408960320906, 9.435566542728987, 9.418827998185346, 9.734242762398654, 9.682199886961534, 9.995758828389642, 10.259718917648582, 10.251328563612667, 10.587324567345146, 10.56653396915598, 10.863578667748506, 10.870693192184671, 10.91849637327212, 11.335716732213303, 11.483364592437209, 11.729630008072517, 12.068268942318626, 12.070292192178274, 12.112534667230813, 12.32751927864729, 12.515234740414915, 12.72048684331307, 12.90036004757608, 13.045838797784478, 13.146512604121385, 13.28789838812604, 13.481427392031135, 13.752989267923619, 13.761441196901728, 14.027707192116502, 14.070586486074069, 14.32259846238913, 14.43138073400749, 14.530076569769879, 14.632685538471046, 14.755795070918962, 14.965798341900207, 15.165691297262873, 15.367963146672139, 15.578106279416964, 16.007437270329667, 16.257309934005498, 16.50883054277562, 16.7760341153515, 16.87080454670506, 16.667135047145603, 16.907374958416277, 17.391811290662684, 17.609676482033905, 17.785225605790124, 17.906276293892024, 18.109528704953433, 18.27682300084989, 18.467776669846675, 18.39188164415723, 18.236474607691573, 18.46344917603894, 18.405194304023738, 18.615126538195785, 18.858861719876803, 18.948886613194666, 19.191641062525267, 19.322505965568944, 19.5645324544947, 19.983375003653514, 20.23630885815454, 20.619357798891336, 20.80269595365792, 21.24577524948933, 21.683797194462763, 22.208535765591666, 23.107868820673122, 22.949587459686114, 22.879167435964305, 23.127872999678683, 23.143334185849362, 23.065585932804986, 22.981971395872616, 22.944280255424026, 22.90287039438091, 22.973736915070017, 22.844910565711846, 22.78790620135175, 22.724716475626135, 22.67666550747846, 22.640255145440364, 22.58152672980987, 22.549048838675855, 22.467490191111317, 22.473545526743283, 22.4484566015126, 22.371170702976983, 22.34839298651286, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.949565575047178, 8.044332402387862, 8.57634232586485, 8.616711236879567, 9.354403276430766, 9.69575188216895, 9.918107267860908, 10.159625060690871, 10.341814225321112, 10.748586143516516, 10.888878543648504, 10.93552511864119, 10.932416497890587, 11.352760439981536, 11.468090752451058, 11.491803343599734, 11.53734372013113, 11.817103378580175, 12.000030416218193, 12.174737133170233, 12.19945777220974, 12.537337448193298, 12.532224989931851, 12.614988923034412, 12.614622422040853, 12.77338634730719, 12.822673588838585, 13.23502303789593, 13.280686388538582, 13.36125958213692, 13.51299458125355, 13.620957673649354, 13.831100893057185, 13.96248864326516, 14.092899009386297, 14.123472370009704, 14.316508358970168, 14.508523287871919, 14.615849722299796, 14.866289558489337, 15.117122550478202, 15.154927786841677, 15.461239576276942, 15.508016807178052, 15.883815909553729, 15.96542238479754, 16.20759961318833, 16.385594146524443, 16.59459151179672, 16.74813370743175, 16.956988454041984, 17.063061220924837, 17.15728972659586, 17.1819289554308, 17.490941574018837, 17.62327235278405, 17.946045399730263, 18.113199960811535, 18.294681353150548, 18.362102485118147, 18.544962304877423, 18.46100168777977, 18.280415447303945, 18.44341610265027, 18.538099687067984, 18.67735084464036, 18.891013319042322, 19.122081251976958, 19.29658277697009, 19.5645324544947, 19.983375003653514, 20.248563458111445, 20.635518046366037, 20.752659813175054, 21.231791026173457, 21.64126092438877, 22.091738207582974, 22.72128445656449, 23.221786909101656, 23.624026372645154, 23.59789795848618, 23.195870189585317, 23.110623870956374, 23.078446325231088, 23.04251864351157, 22.889911146287055, 22.79129883447203, 22.744572896169327, 22.65842929029408, 22.67487715245297, 22.62152272518186, 22.619018585676336, 22.54820925762519, 22.506950072470943, 22.434791580681747, 22.390298044871315, 22.386566697842472, 22.336724690726356, 22.331640307273496, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2029, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.137337868136134, 10.439797025065408, 11.347426152733792, 11.392180479703528, 11.183592928656457, 11.553377423422306, 11.340686690144347, 11.227555328225636, 11.304672412620235, 11.388354641469565, 11.319943393110233, 11.176056140462451, 11.265502310298602, 11.18450968803821, 11.495122050664566, 11.909407109955001, 11.995778687939868, 11.911722296225276, 12.16330835496814, 12.126674152197168, 12.602521969474484, 12.668550569737214, 13.003571767607008, 12.99357963500121, 13.097155498519195, 13.168783195110393, 13.210177282020727, 13.460544529122467, 13.548064940472457, 13.61999806133013, 13.686998663448968, 13.742958549382738, 13.777508617729591, 13.984836145900129, 14.073838322095709, 14.309832098380232, 14.486805827843224, 14.695136660808107, 14.87558300454116, 14.963557205584952, 15.177269435514795, 15.401062595023754, 15.709595892798244, 15.988423890036266, 15.950266145849627, 15.788127433797328, 15.8334918938574, 15.773034083345532, 15.599371863007166, 15.551057750655598, 15.467093068437121, 15.386482488189438, 15.275521537344, 15.25622892638696, 15.196534691190362, 15.323864456031238, 15.499477947233368, 15.691920779040863, 16.50543973871845, 16.796159360114054, 17.13301664130423, 17.6048387526313, 17.747645845975875, 17.973833985970106, 18.388751164022317, 18.72295845962366, 19.341708655015335, 19.669015963833917, 19.921162131674404, 20.60460640713857, 20.96941045185251, 21.259654174269563, 21.49459729204828, 21.859163147248477, 22.075153420713498, 22.34515846214578, 22.464091853382552, 22.886869083438867, 23.0323440487663, 23.001626613610977, 22.970930628493523, 22.933031110860178, 22.78287106397674, 22.713391643012304, 22.566129664815968, 22.891528014634073, 22.799270754133822, 22.698063875236844, 22.664350747403876, 22.578243109541884, 22.590677044498342, 22.534125920808165, 22.532043133648916, 22.55084126053347, 22.48530458404089, 22.435601070830955, 22.43195264641201, 22.39452374146439, 22.315019641737077, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2029, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.987098540934798, 9.434708338238105, 10.132436274094124, 10.563838592816149, 12.075351610829369, 12.785093427877722, 12.800747709425014, 12.60018419776579, 12.598598944022338, 12.43362566186295, 12.325418951457221, 12.622847095187709, 12.893752728987229, 12.75231984058503, 12.552015660957972, 12.637806089569779, 13.000033949383083, 13.034984368073163, 13.01522327800266, 12.959694177836854, 12.878862435725525, 12.946432426041229, 12.910276583781814, 12.898003892519569, 12.903748177622685, 12.663487272821762, 12.553638815544588, 12.597968556208421, 12.595720928740771, 12.70381611650621, 12.751243755653272, 12.759051049436549, 12.848893431412614, 12.848389134277097, 12.90833348140325, 12.918501406134425, 12.93941515240666, 13.00030575131013, 13.099378019843371, 13.144757616892678, 13.179902592971784, 13.39393097611815, 13.45711900246777, 13.568794465384624, 13.644197353406785, 13.588133616402075, 13.659491971426188, 13.84908677203124, 13.973598982408388, 14.100340096574884, 14.047197622373126, 14.197988956124034, 14.451945613426567, 14.537941472462917, 14.644371804881198, 15.107612734134884, 15.453358345874523, 15.782843125466458, 15.994311970712097, 16.1485601172662, 16.27062637529635, 16.659571288657474, 16.95384466227205, 17.12286288333589, 17.390552195011182, 17.59053051280631, 17.709076711398133, 17.966238142430473, 18.260480357287555, 18.690193030754948, 19.150302801555625, 19.32592963693489, 19.646872947794993, 19.835218115195307, 20.237306912824998, 20.589748289394613, 20.779763323493817, 21.15152029048793, 21.364761153751807, 21.77709202972323, 22.076961654061535, 22.69132326423525, 22.860857394929756, 22.95492173586933, 23.00725947202159, 23.006561974074526, 22.955510032207183, 22.912356841338006, 22.781356257823678, 22.742058374519488, 22.697006119686414, 22.59570847304114, 22.612783637270987, 22.582914514605264, 22.52630746659404, 22.48626813344201, 22.443243324326225, 22.43704240685142, 22.38769468889872, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.649204428412963, 9.892270084376069, 10.533983506598785, 11.07095620675068, 11.505735750781069, 11.66033603839504, 11.35813630291593, 11.257555138209794, 11.389538905938265, 11.31053893228491, 11.441539664263612, 11.755496153846607, 11.855700320982908, 12.17500324342325, 12.446686262389072, 12.413752405800757, 12.524652600024455, 12.618956874432644, 12.754501062344428, 12.856084689745689, 12.94866233852966, 12.951110288804813, 12.999667703500755, 12.95427768789447, 13.004778696004932, 13.130625771233372, 13.23528009713785, 13.31131548053699, 13.370802537267515, 13.48543391631429, 13.663193241033184, 13.775420054882586, 13.779266918018662, 13.915138200068878, 14.130537006860584, 14.276694036269399, 14.30537389554199, 14.429164431118918, 14.676285543944767, 14.831215672955791, 15.078800899903788, 15.231809152973335, 15.253041352802194, 15.410133617337921, 15.5779137014425, 15.706649463503524, 15.702767352978919, 15.694000117133964, 15.848718079307849, 15.908524358820983, 15.970615479816761, 16.049665413838692, 16.264326889445417, 16.4389725983277, 16.694817709680105, 16.605860377721932, 16.721068111063346, 16.92223301346343, 17.126169834117515, 17.365385490195727, 17.407607326582962, 17.664907679867515, 17.879558230198747, 18.00475532788971, 18.206535615973625, 18.788999906182568, 18.935618694818597, 19.172657904000616, 19.53059267565138, 20.141013111982176, 21.100694494620786, 21.397811918895034, 21.801569320717316, 22.023721003931943, 22.033309733703835, 22.471267406447303, 22.990715417231776, 23.343960915875545, 23.037612943436734, 22.988502056785176, 22.98856397000101, 22.931223911455554, 22.749914654909684, 22.60098298098941, 22.623660618679185, 22.668811294064863, 22.680086323647256, 22.64524644393077, 22.575426317966517, 22.532843484324722, 22.443547262147547, 22.390928239582866, 22.39230788998496, 22.31331335738934, 22.14858626608364, 22.125888213602746, 22.07297324331626, 22.08174083378525, 22.395418700365106, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.133631734380927, 9.45137296066799, 10.456149906391481, 11.163810944699014, 10.896495823329664, 10.779100053248731, 10.983494633887178, 10.99672805877772, 10.949525417507585, 11.235210601923068, 11.39245677020191, 11.680723990325781, 12.141051002466492, 12.679611852646103, 12.917319082894974, 12.738267888923275, 12.857514136491405, 12.931891797994947, 12.965550026665456, 13.0647023730066, 13.229228336906859, 13.231052148324885, 13.524979687608873, 13.521332919920034, 13.526849486485212, 13.657003938672666, 13.697712096993294, 13.814181430952935, 13.719808266247412, 13.675889387123494, 13.852049524794074, 13.95314130412352, 14.012931874581364, 14.155863697957702, 14.252823461067504, 14.390535749494484, 14.527989062884863, 14.592590122844443, 14.831960916100723, 15.000854856200835, 15.100444820298286, 15.195746068177977, 15.357149209526707, 15.539308900208137, 15.724158933724564, 15.899448008048628, 15.900413554244997, 15.777963042646778, 15.61773925808126, 15.623856138066063, 15.545682384829146, 15.530315675231424, 15.679684895854248, 15.503315591976264, 15.651660787067948, 15.84738789377472, 16.064549473910976, 16.219551086919093, 16.354033680389346, 16.51232223830232, 16.85135878361889, 16.973302700302284, 17.137028011306132, 17.347164260119126, 17.687930202335515, 18.00265453818807, 18.319794791627334, 18.580341082830312, 19.082099139589502, 19.594227813716707, 20.080711728110312, 20.473376850826753, 20.616273729109825, 21.06941773339635, 21.636858200613368, 21.782777945871416, 22.236829136454304, 22.775213472002793, 23.057666691849263, 22.977713461557506, 22.985669316031135, 23.325353642632983, 23.281864744935106, 23.16856084086798, 23.08603697380897, 23.07460251290768, 22.801864433338828, 22.771792474137374, 22.73582331801566, 22.646709237985995, 22.54023858073549, 22.503589823605402, 22.47224133854488, 22.546351130061012, 22.499003048598958, 22.50163774902695, 22.442026676452883, 22.38178473227027, 22.422099282380266, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.75601089818206, 9.252389544754504, 9.686430606010141, 10.329051354067362, 10.186793725717836, 10.6924781208096, 10.554447028475701, 10.48778313341464, 10.490956259812675, 10.950407028914643, 11.039040987166224, 11.001662077678546, 10.990223892524979, 11.164233651775655, 11.157663589890959, 11.263432297347835, 11.387415630097431, 11.689169939456487, 11.834294238789969, 11.811218407704779, 11.89153140226268, 11.943332748626974, 12.03311324299964, 12.125254134545022, 12.288777064235386, 12.555897655148204, 12.881110641922763, 12.940234734321365, 13.088328883924314, 13.10769845368282, 13.222085685825066, 13.579726548110063, 13.57625083859558, 13.714772983961195, 13.867706220096427, 13.994200341191195, 14.093330968323528, 14.183940862444855, 14.242732595362591, 14.176309364677781, 14.161538807783721, 14.294180635451484, 14.303239625918643, 14.539718507852289, 14.669682155792065, 14.80327314079702, 14.946089690345193, 15.053171688852457, 15.206746331194404, 15.268683678340771, 15.388936988797733, 15.2898431800858, 15.353779973003595, 15.549609334288055, 15.730095732480134, 15.927057375690243, 16.053774215133693, 16.19697762433406, 16.35334756484788, 16.540886655335868, 16.695093207051627, 17.021233454765174, 17.143785931837954, 17.358579261251496, 17.712213652575304, 17.984703662045167, 18.32286232159889, 18.57301864270259, 19.094761378585044, 19.594227813716707, 20.050535495274914, 20.44554482919706, 20.593912296302445, 21.06656554365149, 21.614412475483665, 22.125827865854575, 22.26193807861392, 22.861208880918504, 22.9835948802788, 23.00591072694483, 22.91518911165185, 22.7968362107572, 22.799192942611878, 22.735455534403517, 22.677216517378724, 22.623165006296155, 22.56433571196695, 22.522905917944268, 22.46828296574559, 22.411946076598284, 22.425838056045997, 22.392335226795286, 22.31075501101441, 22.279880659669622, 22.25459633865835, 22.156609467882863, 22.13376835499453, 22.37731666740305, 22.347893730958603, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.134224479460098, 8.999907057206743, 10.493944533968497, 10.903262284825594, 10.985649448116297, 10.830071688007877, 11.019335129866855, 11.00140051960723, 11.239363705748499, 11.241789327507597, 11.400206682223434, 11.578252838356134, 11.494859985856575, 11.95328876041645, 12.289597094395596, 12.342626769934, 12.730123034604299, 12.860000886088624, 13.15851754803524, 13.25201048870674, 13.177205057716174, 13.257932018055232, 13.243697670455331, 13.673361202602388, 13.622445843362382, 13.863255843073283, 13.859367896858425, 14.079844206672352, 14.166451956931716, 14.218771985468711, 14.266546937299909, 14.402322428866437, 14.471553682545407, 14.56660995404484, 14.669243442819683, 14.785740306225728, 14.853524347275638, 15.019319369290058, 15.138124797959208, 15.22985157006585, 15.349138064382359, 15.44877576422779, 15.677614596867867, 15.582487523460689, 15.657778594686409, 15.804193481594053, 15.913825238229768, 15.941117574733529, 15.91773109087643, 15.75262205463646, 15.58590566422153, 15.75101992155847, 15.849370503799886, 15.964857523962305, 15.987566309787308, 16.05240468781534, 16.21623732116617, 16.239255340374374, 16.37445412767944, 16.575048104393932, 16.68049551670241, 16.99156854524769, 17.154487696682065, 17.578233931277133, 17.718549025589255, 18.00754910573297, 18.345974590814293, 18.58337669910899, 19.094761378585044, 19.594227813716707, 20.050535495274914, 20.44554482919706, 20.58185357440048, 21.05233458923457, 21.568899222579773, 21.690798067699472, 22.23315812797417, 22.861208880918504, 22.9835948802788, 23.004781150625075, 22.915667026638996, 22.800619111357285, 22.784055135488423, 22.759447605281814, 22.64676096661427, 22.64162519486659, 22.520458640333413, 22.50257890246841, 22.468691388831786, 22.411771128241114, 22.430493816326987, 22.344916177885924, 22.28882472879301, 22.275497952032758, 22.247601161515025, 22.159117757181782, 22.494033365202952, 22.422040565385405, 22.350260293458955, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.133631734380927, 9.467255566426342, 10.456149906391481, 11.18549986950045, 10.925509858131992, 10.778356935827436, 10.999384032616808, 10.99672805877772, 10.967195797558215, 11.091148977863833, 11.417181113656971, 11.784789897060563, 11.876005686352345, 12.4932518609336, 13.13585814334405, 12.937962427606875, 12.949996557339139, 12.840532816513587, 12.91740811882135, 13.1106373141095, 13.180819135008667, 13.20126061875351, 13.42341939954407, 13.592111923818456, 13.605840100723315, 13.564123603109532, 13.666140374653224, 13.715204040270976, 13.73532540565241, 13.94925381558486, 14.143685916702244, 14.197353883774491, 14.305205385620717, 14.411495671747135, 14.37967523654965, 14.495726783689843, 14.640926609749053, 14.74161817803148, 14.833655174054043, 14.95225641608223, 15.169485411501126, 15.26012499341491, 15.428409730764866, 15.583107664112184, 15.7332453148958, 15.887163089574685, 15.947682199860354, 15.758123417784391, 15.716678550810952, 15.66110819925953, 15.562528206192141, 15.622028487472845, 15.71238810117145, 15.707994614860986, 15.662327970479625, 15.803950874653072, 16.07779166136563, 16.21287277946139, 16.361534488301952, 16.57575143538534, 16.84733055928345, 16.98576707067895, 17.139212832876872, 17.35465368806321, 17.709399119125855, 17.993944479423867, 18.322912309584236, 18.573838922848886, 19.082099139589502, 19.594227813716707, 20.080711728110312, 20.473376850826753, 20.614478756544564, 21.06941773339635, 21.6220548483955, 22.097259240912933, 22.236829136454304, 22.775213472002793, 23.057666691849263, 22.977713461557506, 22.985669316031135, 23.324771415559134, 23.281864744935106, 23.16856084086798, 23.082398536899305, 23.0564085712045, 22.803279007924132, 22.769774302628996, 22.736387171833147, 22.601137114680096, 22.541477980605023, 22.550008178614398, 22.579345029120613, 22.549019607843135, 22.497569560309234, 22.51594523505443, 22.44384290095144, 22.38178473227027, 22.422099282380266, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2030, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.004177873507984, 9.02739528463503, 10.52379552147, 10.652135556944984, 10.463515581965373, 10.64074215295658, 11.09870421851158, 11.263699127891135, 11.348374759729042, 11.597034653055339, 11.84992965109176, 11.828466025005815, 11.813809703164914, 12.02926049201085, 12.034577716093745, 12.346700695260866, 12.333481472439681, 12.804844718423118, 13.091547203193757, 13.130603772114114, 13.089328169022785, 13.058783490763338, 13.232858280233273, 13.434500796427464, 13.514827046826383, 13.716926182837783, 13.81505311928043, 14.037776150794379, 14.087116347858588, 14.271328179348004, 14.329986395743186, 14.317733675730828, 14.401803471592775, 14.408680757588261, 14.559183456106798, 14.719135790509434, 14.79077258015421, 14.972084548186585, 15.119186428913082, 15.221472792440535, 15.317185263999257, 15.365489500760459, 15.707772442858174, 15.78686269304829, 15.852078424446391, 15.912321798435464, 16.01483492406617, 16.271522141570372, 16.502648891486967, 16.6389795165061, 16.663506190341213, 17.007497996554203, 17.22296837348036, 17.553476482229748, 17.817229066051823, 18.024310983010626, 18.29201628386546, 18.486694512884853, 18.768148030957057, 18.689902081653575, 19.212627382104166, 19.310930362676636, 19.442890330605263, 19.730775111908482, 19.79935930606145, 19.910839132337475, 20.18359660863837, 20.43936139031599, 20.21161295738254, 20.514080256549658, 20.45590922806166, 20.824182822760395, 21.18363751668105, 21.508587432289215, 21.632374920735693, 22.05226745488225, 22.14238154520258, 22.589990755756407, 22.91086644225551, 22.83294164250907, 22.86090045662685, 22.767418738448082, 23.095024210809633, 23.004128916625653, 22.955768833151623, 22.80203937040513, 22.7609973451283, 22.67865244223492, 22.55495596093708, 22.562184036891882, 22.462108889297404, 22.43608812397873, 22.425349064922, 22.44659208875925, 22.460832103736184, 22.395098161938407, 22.41052090927244, 22.41533369357747, 22.335039601332827, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2030, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.98899762773124, 9.524974245662445, 10.949420288932203, 11.05026901171994, 11.098405366558902, 11.683941845534033, 12.232586977296647, 12.012028570381256, 12.01859385101702, 12.324412739467503, 12.191902842802847, 12.260500003183585, 12.158479806655313, 12.262151963760298, 12.331275040354859, 12.361128328889754, 12.455500219909071, 12.476208589740105, 12.604876935191612, 12.63720108581955, 12.838258794359353, 12.97748152914438, 12.975484457176107, 13.03222170651344, 13.297815957886813, 13.49842006084275, 13.435941364357722, 13.513079647120275, 13.491222159566027, 13.524218707344632, 13.630195575292186, 13.682863575743726, 13.734207366553738, 13.833973101186213, 13.81111176611222, 13.93497594861076, 14.049965831304695, 14.04735653554751, 14.191764961336458, 14.276345252619915, 14.363718647483328, 14.392065268878348, 14.501755207926283, 14.379228318847664, 14.472984259657704, 14.520603971739545, 14.844720617886937, 15.125664140010302, 15.25011344391045, 15.384918797460518, 15.769032118984205, 15.919197036107438, 15.946991850792289, 16.10007401170763, 16.233813149453866, 16.651586441012178, 16.67184554289965, 16.812906802144038, 17.016188555765265, 17.404053846469925, 17.56306294207014, 17.849719636851994, 18.025848400213768, 18.309878580994678, 18.888105993213866, 19.116369856336693, 19.232779498089698, 19.55896383828455, 19.824967901344944, 20.132927583099676, 20.451726715600756, 20.86732568691332, 21.145014301303082, 21.52018125032275, 22.013399451786967, 22.277488508318754, 22.64201551428287, 22.912418066347612, 22.95644812869885, 22.843872676846058, 22.7353301817433, 22.62886194701064, 22.483583677006266, 22.675039709138606, 22.602555064361646, 22.602517143895742, 22.52568239320546, 22.490052163566084, 22.45318895613491, 22.418141947514524, 22.4426182787964, 22.407248304504186, 22.441778177996344, 22.4107405137124, 22.469128938504383, 22.45133497946577, 22.41870230222383, 22.42373145055055, 22.39003632724271, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "MiniCPM", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.971314530720472, 8.025333146859317, 8.023243834612597, 8.098744842753828, 8.645513063654834, 8.619309748445284, 8.646704176811687, 9.443620186513305, 9.779111472475318, 9.75640936812068, 10.310312719223132, 10.300438351612783, 10.26690856647114, 10.80238789885741, 10.834332386933102, 10.940744821799559, 10.965738993738777, 10.976267469906428, 11.175585341833678, 11.450622055265729, 11.779191468985328, 11.700684145564411, 12.054861682323983, 12.288606616789725, 12.308379800045172, 12.369691149988489, 12.460910121470468, 12.642226642204847, 12.789983864143888, 12.811954147661964, 12.989102366692123, 12.981193457081236, 13.029727761807418, 13.105516038226606, 13.229656581673343, 13.506163001085634, 13.678999589332374, 14.000407449504015, 14.094748663288875, 14.092682368274891, 14.203969887269002, 14.396903088243379, 14.45412315380716, 14.626037505883554, 14.983143470865162, 15.109393472243305, 15.230022684539316, 15.440834539060116, 15.742797148282154, 16.006864246230503, 16.550269984294882, 16.902431567494304, 17.092298363102252, 17.58122827150762, 17.639213472284005, 17.71767818457903, 17.86367238847644, 18.034250240874943, 18.203617355645584, 18.189900179784185, 18.269019899209997, 18.45439599004875, 18.427489975304503, 18.724430300976177, 18.851060160449563, 19.128627499242267, 19.35344263451642, 19.596610707866798, 19.746935094817122, 20.11753709810019, 20.412821987802705, 20.476832609506417, 20.898451752088885, 21.262348618031528, 21.71607952237844, 22.13247412362906, 22.495329968291053, 23.122287233141943, 22.98635271541698, 22.980693064697807, 22.958044937395844, 22.912030606738245, 22.736568120025193, 22.649911044064925, 22.72251318686466, 22.704193592885574, 22.603566961683423, 22.57494416486506, 22.828119587464407, 22.76990192884942, 22.72152947260598, 22.674225181332428, 22.636608572579128, 22.523556247253378, 22.48621432608214, 22.420752771904326, 22.39492616672554, 22.370443021124313, 22.346939775313228, 22.361115727328066]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.305712265257315, 10.32090623973514, 10.62271519097406, 10.61731601929559, 11.719154110149754, 11.910304203127408, 11.893421442593253, 11.907095637313148, 12.208149949689707, 12.18644435174324, 12.398591579374486, 12.413433453761991, 12.360734685824951, 12.403808978206063, 13.016278931281375, 12.92983259275721, 13.21955438089885, 13.456337511606892, 13.721058184680428, 14.11288761291902, 14.560921448563668, 14.457952909445446, 14.49799744136212, 14.450474929280643, 14.394135108527642, 14.464973218054586, 14.630984105211228, 14.595096719830252, 14.67087898194667, 14.89044601604311, 15.183044258476919, 15.279116092073592, 15.330728115935239, 15.564757601953433, 15.724960485714497, 15.885193261265067, 16.176555796943354, 16.329311944898162, 16.824115544120254, 16.915882691828045, 16.895287711150416, 17.210447628292812, 17.29517178875331, 17.43425691007896, 17.647751630467063, 17.848772308410467, 18.166797886339474, 18.46736227747306, 18.664124602754857, 19.011474144660372, 19.43657332315603, 19.78854774727098, 20.305729193678452, 20.575901347579826, 20.71384390902133, 21.140001599493672, 21.466357067940073, 21.789126360814983, 22.045591783073373, 22.30630846607065, 22.543628452653692, 22.732885064385968, 22.52036209832216, 22.77396039067759, 23.2427789582205, 23.650470240751282, 23.88395106035182, 24.177270969327676, 24.447241024996128, 24.625464210299597, 24.922396088277328, 25.45314715114681, 25.706209063633217, 25.907262249953412, 26.20470721884281, 26.5317526695241, 26.850022031793557, 27.137886290249014, 27.051015600234997, 27.017781846870687, 26.926889477331827, 26.834259527614478, 26.786712893447078, 26.8085945233431, 26.78480761001929, 26.923876998692585, 26.877169768348818, 26.854129655976706, 26.80854146463941, 26.84440929920105, 27.057460224722234, 27.364813854405323, 27.320839220872156, 27.33101186786365, 27.40727148156005, 27.278516871270163, 27.136113654795967, 27.175668162411792, 27.184073463462173, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.965088481987706, 7.962053228597654, 8.046397178697761, 8.119315596245594, 8.149065172075984, 8.205610775409829, 8.252584890229851, 8.28233510326936, 8.349039519865178, 8.426092819664765, 8.516037384633194, 8.560046351919045, 8.596069159855682, 9.262395613643099, 9.897967737742034, 9.981194812927884, 10.013539671595838, 10.029944990602548, 10.071432529052865, 10.146401582532517, 10.682488205013394, 10.63633111184234, 11.150298891780508, 11.601547330218555, 11.995507462364197, 12.351581961840186, 12.787679582360264, 13.515282446770815, 13.512986520845477, 13.836660657210862, 14.070609078949465, 14.091851064448838, 14.033516564086021, 14.001184610264492, 14.400294671065236, 15.09630673242793, 15.159161411667089, 15.19975483803655, 15.253456648040261, 15.218614501267814, 15.290636008072529, 15.48471383459646, 15.815937602827395, 15.866084395232058, 15.92931844354752, 16.132442315138153, 16.18737755339209, 16.76707049504202, 16.870396030321942, 16.96774725269259, 17.05756696274335, 17.33890259687639, 17.64918645681619, 17.97602233460494, 18.306452276112147, 18.828252993026094, 19.117935410931285, 19.305383246595976, 19.623862404827214, 19.824861105152518, 20.33934413711438, 20.764600374244818, 21.42335133389546, 21.7679908340152, 22.23352626962854, 22.459269197796043, 22.75944400176386, 23.28162492094135, 23.655349655696696, 24.016982089798812, 24.500328481127625, 24.77922944031634, 25.055434901402606, 25.56906079214208, 25.904506534263426, 26.298932070406835, 26.646172474290957, 27.040370552679548, 27.024753822250453, 26.96741655331816, 26.926889477331827, 26.810453882923362, 26.751363354896696, 26.765895197015382, 27.007034465917794, 26.96197642691197, 26.870933258397628, 27.36353558337431, 27.34277234121344, 27.493379424053565, 27.386173540292653, 27.37564625612588, 27.41386306790745, 27.32235975015758, 27.31452713053375, 27.28381845381532, 27.22734192475554, 27.108983164033322, 26.967684709648115, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.96049799138685, 8.097256675914595, 8.112130284187304, 8.196406880846478, 8.265043018979904, 9.003030882866561, 9.01972686674714, 9.099010323340615, 9.15111702335185, 9.79129674997576, 9.848957048339544, 9.855865643100746, 9.88662692852782, 9.924965413868211, 10.445068762739716, 10.916738281358887, 11.048352098427607, 11.52727549992661, 11.830900908129651, 11.848756825150838, 11.890164662426162, 12.41066635104923, 13.10763190612951, 13.06821654926079, 13.152471012803149, 13.16582995704539, 13.732439714466945, 14.288816914954664, 14.311562960076033, 14.862158098323814, 14.893241461459187, 15.06246680800509, 15.070071894964066, 15.154069398769233, 15.243806815064294, 15.43997964393461, 15.469585432861399, 15.941622926589636, 16.240838441634185, 16.251774210111314, 16.591558905942904, 16.526446619956044, 16.62089126584187, 16.631459348140503, 16.968047940694618, 17.274124220932734, 17.56198241620339, 17.65308075612199, 17.75878753235742, 17.80582229538027, 18.121089473254305, 18.284387323795094, 18.46254744814633, 18.65303609328253, 18.871281769795107, 18.93892890118382, 19.4519339065136, 19.626886491355815, 19.97300977209513, 20.167704146380157, 20.42612719232267, 20.73209073804864, 21.119041420308328, 21.52735260767848, 21.8368057084522, 22.354685257906958, 22.733979611841203, 23.208164125569255, 23.64759522405701, 24.12611701614413, 24.261686994113447, 24.57463032433147, 25.065883377583454, 25.551216581830182, 25.875191780605274, 26.262977127126213, 26.55087617126575, 26.929185075897504, 27.239070553629052, 26.96741655331816, 26.926889477331827, 26.803058611346238, 26.767783716144645, 26.765895197015382, 27.007034465917794, 26.96197642691197, 26.87791469850761, 27.36353558337431, 27.33599755592202, 27.509433383837823, 27.46587001933013, 27.37564625612588, 27.39794624760721, 27.306510841565878, 27.33608668413216, 27.295497149456054, 27.211581589915966, 27.115596670703518, 26.967684709648115, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2026, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.305712265257315, 10.32090623973514, 10.62271519097406, 10.61731601929559, 11.719154110149754, 11.910304203127408, 11.893421442593253, 11.907095637313148, 12.208149949689707, 12.18644435174324, 12.398591579374486, 12.413433453761991, 12.360734685824951, 12.403808978206063, 13.016278931281375, 12.92983259275721, 13.21955438089885, 13.456337511606892, 13.721058184680428, 14.11288761291902, 14.560921448563668, 14.457952909445446, 14.49799744136212, 14.450474929280643, 14.394135108527642, 14.464973218054586, 14.630984105211228, 14.595096719830252, 14.67087898194667, 14.89044601604311, 15.183044258476919, 15.279116092073592, 15.330728115935239, 15.564757601953433, 15.724960485714497, 15.885193261265067, 16.176555796943354, 16.329311944898162, 16.824115544120254, 16.915882691828045, 16.895287711150416, 17.210447628292812, 17.29517178875331, 17.43425691007896, 17.647751630467063, 17.848772308410467, 18.166797886339474, 18.46736227747306, 18.664124602754857, 19.011474144660372, 19.43657332315603, 19.78854774727098, 20.305729193678452, 20.575901347579826, 20.71384390902133, 21.140001599493672, 21.466357067940073, 21.789126360814983, 22.045591783073373, 22.30630846607065, 22.543628452653692, 22.732885064385968, 22.52036209832216, 22.77396039067759, 23.2427789582205, 23.650470240751282, 23.88395106035182, 24.177270969327676, 24.447241024996128, 24.625464210299597, 24.922396088277328, 25.45314715114681, 25.706209063633217, 25.907262249953412, 26.20470721884281, 26.5317526695241, 26.850022031793557, 27.137886290249014, 27.051015600234997, 27.017781846870687, 26.926889477331827, 26.834259527614478, 26.786712893447078, 26.8085945233431, 26.78480761001929, 26.923876998692585, 26.877169768348818, 26.854129655976706, 26.80854146463941, 26.84440929920105, 27.057460224722234, 27.364813854405323, 27.320839220872156, 27.33101186786365, 27.40727148156005, 27.278516871270163, 27.136113654795967, 27.175668162411792, 27.184073463462173, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2026, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 10.680049038534749, 10.390888382602714, 11.414866542254554, 11.633117246110821, 11.458593382825272, 11.693218406751393, 12.161064882383759, 12.101961641637368, 12.343244872044615, 12.585870474821458, 12.81279852120331, 13.028896217353678, 12.98323284175713, 13.090417894027729, 13.777317313041305, 13.774855105566797, 13.712919162009568, 13.687099553102207, 14.051752385680464, 13.994644282168892, 14.139285792126962, 14.113126278535356, 14.13980199635992, 14.197768985605762, 14.333259649053783, 14.318796920884102, 14.424296986917359, 14.597500036571084, 14.815259487903843, 15.010055389385125, 15.053806413590062, 15.13215816381885, 15.363794492667878, 15.518264955899221, 15.678510079872055, 15.902603844940316, 16.053680615558097, 16.184233553109113, 16.450211758363693, 16.622650282512236, 16.935460409528417, 17.09972442103831, 17.430462465597316, 17.482358205146458, 17.850563977991936, 18.03786222720422, 18.192578714856698, 18.512031100307297, 18.697036191850312, 18.890517191626028, 19.29250841570501, 19.40797273937931, 19.546754151495684, 19.317550183420217, 19.572812456062614, 20.046759193715094, 20.07030403206585, 20.272240721031977, 20.48395611868714, 20.805699028068215, 21.015536502585146, 21.152643455834944, 21.498674670206707, 21.866435412592907, 22.23145317487276, 22.42009646723743, 22.882306129863878, 23.370274439379287, 23.580956423416637, 24.10223680188846, 24.409244495868776, 24.759691048831222, 25.101065955421227, 25.340577407521636, 25.88568678491313, 26.023396410120235, 26.428440899322013, 26.921846231056755, 27.000781982869597, 26.99327013726692, 26.895500442265302, 26.77029253219913, 26.681818163357924, 26.681818163357924, 26.799663695609244, 26.751296456352634, 26.75844206503268, 26.751675806260334, 27.077523807151326, 27.27389758981168, 27.251762813618864, 27.195699178204723, 27.116220845966794, 27.161159074933174, 27.097222923671442, 27.03567800180063, 26.93393719799187, 27.10413616874351, 26.9695660025683, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2026, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.678194012580882, 9.316335703413198, 9.285990190392328, 9.804296204923398, 9.79059424050392, 10.256381988924671, 10.67698192466696, 10.674716060762753, 10.958515050421761, 10.850201763288133, 10.831981954831745, 11.236711830093201, 11.224100603530381, 11.198798395274888, 11.245634270342736, 11.606280955329023, 11.568817870522539, 11.563775103365474, 12.076565256511776, 12.063714797628984, 12.281342165489473, 12.304408406116902, 12.355147461780454, 12.610727461996877, 12.637498501303526, 12.726908202047744, 12.970284743928975, 13.241343546943185, 13.256675932428957, 13.512133836850646, 13.844521359796271, 14.272364146680545, 14.4493314908535, 14.871280952071759, 14.919737790755466, 15.056465590133605, 15.306068257524567, 15.576545638791952, 15.993314453139101, 16.19259783111094, 16.33590447025138, 16.44117923500733, 16.588284095466378, 16.88668150912738, 17.123524961660568, 17.323652089907995, 17.43020819307989, 17.651847890927446, 17.79739217252416, 18.091527396304347, 18.33865852822595, 18.54748050508649, 18.712199876804814, 18.73049344593236, 18.883599549277648, 19.24570139397205, 19.284147300778283, 19.526457345101033, 19.684953996021953, 20.26954813927461, 20.570321807244476, 20.754401418200775, 21.327435813298393, 21.604802541747883, 22.15953078482053, 22.61120088659785, 23.069377514637118, 23.573041592345707, 23.89945596981192, 24.24065694493052, 24.533686481808697, 24.7186510358954, 25.052833238273784, 25.039079382804424, 25.50188926389836, 25.935495179666958, 26.363603854674057, 26.906219169646278, 27.069502279886393, 27.11196025821277, 27.006645283491657, 26.91905392980808, 26.950536675016412, 26.97454287895174, 26.809163738932373, 26.714448832656423, 26.92935239460339, 26.84668284918373, 26.72942199802748, 27.18061121425899, 27.114748974074825, 27.078723372574437, 26.994959964232002, 27.059690884793277, 27.052934258473314, 27.032808906279648, 27.056839650965927, 27.009817667411824, 26.95334100114024, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2026, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.972778587532685, 8.100041677090436, 8.14530220800982, 8.21533047587007, 8.293269026209211, 8.374229013812176, 8.368471784343642, 8.463742086038058, 9.190456275671334, 9.215959695509106, 9.207062154119543, 9.229786987085044, 9.32040819617399, 9.400283160946078, 9.487265846124965, 10.075414783950801, 10.54770803794978, 11.031370497944277, 11.132637225540943, 11.160889857111982, 11.255304570483112, 11.760234376019522, 12.18001284340769, 12.15229330764694, 12.456691513008952, 12.490100664435294, 12.52194948672804, 13.179776860501851, 13.228988589195193, 13.208501378333354, 13.249181166225707, 13.932048706212221, 14.082984710053639, 14.125150233864293, 14.358903807608128, 14.447540648449214, 14.469850532188769, 14.538309571181735, 14.906890868208365, 15.194995949431501, 15.275080695843915, 15.320347985706745, 15.407693025478109, 15.573539166305858, 15.658527720002743, 15.720050433797999, 15.986116271687631, 16.01999125696246, 16.695614739814904, 16.79245154096245, 16.935363711646893, 17.378394169199034, 17.672019638261354, 17.986729027518496, 18.26127852312873, 18.618095823864433, 19.257764280691237, 19.345272348641007, 19.679754743395396, 20.017216503432685, 20.335169269887952, 20.765445695415345, 21.099203927920023, 21.488907105785263, 22.0708651267154, 22.281095176984635, 22.575563687593302, 23.206095573593714, 23.615100949965022, 24.09521860408224, 24.505957789630198, 24.940281195699153, 25.319282686902778, 25.658429787395665, 25.960337367513116, 26.32012893990328, 26.72430232001539, 26.897712688117203, 27.03438042302786, 27.003439744101733, 26.929504819252813, 27.045624332351114, 26.94990899805046, 26.830431690271507, 26.756299370196242, 27.01154269749604, 27.261797063142733, 27.296144062582794, 27.210426280974552, 27.15712339257236, 27.37604268882948, 27.291764113307032, 27.18866127324046, 27.10975147356896, 27.06205169960273, 26.986661777184263, 27.04288570174198, 26.96201734379427, 27.020966368545295, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.26523708792631, 10.920626344445058, 11.129045229120543, 10.931404685149705, 11.717123559572693, 11.590210324729867, 11.794169861910767, 12.344622655453616, 12.263254856352619, 12.382453729440053, 12.28651191025103, 12.457988844378786, 12.657831713597133, 12.525749801440234, 12.95382879182114, 13.164111979977694, 13.103903975428954, 13.115997901031962, 13.336862032206486, 13.356701791832492, 13.711344413819418, 13.949296178013226, 14.121004941922088, 14.174015387118565, 14.263479648198665, 14.331159940967156, 14.323974729674784, 14.407908328414502, 14.493584534941906, 14.829745304751722, 14.874233567275978, 15.305421612262293, 15.22128998095763, 15.32405544911247, 15.499781761818154, 15.63658156529836, 15.71379498163533, 15.988416356358451, 16.187907519014036, 16.381150626917236, 16.508388851470578, 16.686380538459595, 17.037999862802565, 17.759593928351567, 18.071888691891537, 18.20664961020699, 18.398630456284163, 18.854030899093992, 18.82140646937052, 18.963848223553992, 19.13480731888152, 19.3591723720521, 19.171757339421557, 19.093327786598493, 19.44174206518541, 19.688575354258973, 20.064106449223893, 20.25865925996935, 20.531324395485594, 20.690328121951644, 20.871401267491784, 20.995669639927765, 21.13541204260444, 21.485731516095907, 21.735632548148892, 22.09243552497579, 22.53280668752006, 22.68389690968384, 22.891980488329526, 23.532095039388544, 24.054560367987964, 24.346660192381172, 24.702945527852297, 25.072564669722492, 25.38266886092973, 25.840260475235006, 26.268253172896472, 26.72969279507672, 26.961660151404732, 26.90406061051509, 26.84502720046788, 26.776779485961384, 26.703750784498276, 26.660429987627403, 26.6696789649697, 26.72337195928745, 26.64406463289894, 26.63524905091324, 26.63524905091324, 26.55923904936755, 26.55068413691642, 26.761045290706587, 26.935703759587675, 26.890345145120882, 27.013903577365163, 27.09836591538911, 27.114699242358814, 27.08235242094074, 26.908369297882345, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.969705695593077, 9.333897161082064, 9.334350270995055, 9.93365952000209, 9.92017209051238, 9.950830679200049, 10.017249308969852, 10.5622525793018, 10.558839985521708, 10.57644972687232, 10.603769419637098, 10.604239498064311, 11.115689250070531, 11.165104428819998, 11.165942737937876, 11.656813190008872, 11.65277402686863, 12.032049261851986, 12.060339135065021, 11.98778389087607, 12.76049022500603, 12.863668479109883, 12.843259800428491, 13.173702354281112, 13.044355692187343, 13.038346504809823, 13.057283385803414, 13.074935042367727, 13.091541558333954, 13.125339423919982, 13.174994188461678, 13.681143719576433, 13.764781052405539, 13.72722284781396, 13.837691542723388, 13.789665532683811, 14.30409730381994, 14.571561828258556, 14.661256979495105, 14.96723969004679, 15.278119426350703, 15.60064610217043, 15.724055874597612, 16.00952746690074, 16.12035379869348, 16.436369103547126, 16.4942724470453, 16.61077868304861, 16.71397028331224, 16.955198271850545, 17.23050508549277, 17.540572943597855, 17.824548504491773, 18.18007390107386, 18.45287585702586, 18.8554139477384, 19.029239511346393, 19.27177713258996, 19.69439412148948, 20.078845991559362, 20.68426881156379, 21.086899451009025, 21.341588419789808, 21.55566082282142, 21.870345610738863, 22.150164043719148, 22.39434699614199, 22.750940090783587, 23.04541978099252, 23.530700698158462, 23.937676546451772, 24.188919988242112, 24.542748577258024, 25.141445333424894, 25.498250943808003, 25.826860435980958, 26.12109543190192, 26.431189005132673, 26.811672118892755, 26.90096406670415, 26.84502720046788, 26.771076898120782, 26.693062299786916, 26.62065801077633, 26.673854773000976, 26.587123168313415, 26.578164117855707, 26.672786637552992, 26.559147199083117, 26.776401309473357, 26.81610357318769, 26.969556789144416, 26.872878640009834, 26.764805786028973, 26.737237686258702, 27.018624966182276, 27.21616650514454, 27.13179496534493, 27.10269739329472, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.985404183956481, 8.757210932177584, 10.050927375809671, 10.529739679519443, 11.034721556374299, 11.02574103812451, 11.019223697895535, 11.344480397842192, 11.644630883452711, 11.630056758226196, 11.595413384923084, 11.61990678651026, 11.985720487907958, 12.41782036089506, 12.44295264903172, 12.44074803008701, 12.737397079904127, 13.120329133964685, 13.352533183140668, 13.292889530607042, 13.273584237995815, 13.326890937381028, 13.364735444785836, 13.426828317773401, 13.43079008128288, 13.460170272907437, 13.681710464530067, 13.63140706843977, 13.696927230466358, 14.353270240009925, 14.623044154337162, 14.632459999040643, 14.841451692615312, 14.840121345103407, 14.90057377910172, 14.930584673613653, 15.162196485261322, 15.46766899352067, 15.754007159172613, 15.894038562623619, 16.222521333442895, 16.46799207201734, 16.727745141841577, 17.006795301488044, 17.309101787438365, 17.54554392885326, 17.71794124255329, 18.121573878103028, 18.326935891777, 18.50620416759814, 18.933835386400453, 19.0830698577139, 19.3741584459387, 19.55623863098156, 19.797696747916685, 20.13502685983957, 20.353589642521015, 20.459243080411994, 20.640750696656582, 20.919134181244896, 21.277392242291615, 21.540440675771816, 21.918030201059185, 22.368093061701057, 22.594497507413074, 22.930112272035778, 22.927528637226928, 23.20783240070094, 23.56716704061837, 23.838964312963203, 24.01722355131719, 24.325465586817156, 24.942395042464153, 25.178149388851445, 25.51157558367297, 25.823898478168317, 26.115796128156056, 26.431189005132673, 26.811672118892755, 26.90096406670415, 26.84502720046788, 26.771076898120782, 26.693062299786916, 26.709743793503733, 26.69155312772334, 26.59442105338632, 26.578164117855707, 26.672786637552992, 26.559147199083117, 26.76045036951545, 26.81610357318769, 26.994929365395688, 26.88001407048016, 26.757876236449697, 26.721489833178243, 27.025454900810043, 26.996059264825107, 27.164685339444727, 27.10269739329472, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2027, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.615146717967741, 10.91406983595813, 10.683624398281745, 11.015236297416852, 11.367861224961802, 11.974192855374884, 12.100529842221768, 12.375404724003301, 12.28234650567086, 12.5047681087261, 12.706164510283058, 12.568566729717872, 12.501216754635506, 12.643033159334227, 12.587419919467186, 13.315661907609549, 13.32900883194698, 13.282477161341621, 13.323262110602476, 13.257396039022773, 13.508567347276667, 13.543809366484119, 13.742382500468835, 14.001240930682343, 14.311575761867946, 14.267195950781453, 14.338301879990675, 14.530920309097391, 14.75072280493006, 15.013999937466831, 15.086995547301049, 15.204130704603823, 15.470095785443297, 15.573664944520635, 15.539743962306826, 15.796616158141196, 15.920799100335087, 16.109596425504797, 16.289402893963935, 16.259460749055588, 16.596580732426546, 16.7375490475394, 17.21568310416392, 17.626412891447252, 17.89624167977882, 18.16395585669426, 18.413004902963014, 18.859166719219015, 18.97730926494237, 19.10183484467941, 19.2620399287474, 19.375516632374488, 19.542848069875056, 19.26820229814973, 19.395095346138888, 19.665073012131092, 19.84071657148505, 20.070673754884282, 20.463435482512804, 20.5718707876356, 20.700199441961196, 21.022364898687076, 21.367943104388658, 21.498715942790394, 21.870383015594193, 22.289214075559634, 22.537469316542122, 22.925132423527998, 23.2421706832316, 23.522945205479452, 23.927639299767993, 24.371735086735924, 24.797426382458823, 25.082288982885437, 25.544132490982722, 25.872100191998676, 26.274830648536927, 26.680663728459947, 26.961660151404732, 26.90406061051509, 26.84502720046788, 26.776779485961384, 26.703750784498276, 26.62793940103645, 26.666586969349108, 26.619222945558825, 26.65145996193697, 26.651119043930443, 26.63524905091324, 26.608608781831343, 26.799562088842993, 26.819560590669372, 26.988525607645236, 27.16575733072369, 27.00692340106585, 27.105434749393165, 27.114699242358814, 27.050731153379644, 26.901501166014214, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2027, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.289263271047126, 9.84942122212314, 10.86414266415206, 11.113122035781597, 12.195635352727415, 12.373098282699816, 12.244530496957681, 12.386455330764226, 12.975099075086028, 13.142746439843348, 13.056732454813119, 12.939755345890399, 12.883985908509452, 13.09908340084597, 12.991998780195987, 13.20165983823365, 13.314002455290035, 13.282347874242163, 13.489641868869642, 13.692837516563403, 14.096418622688056, 13.983916004481653, 14.206892686875882, 14.344392492306653, 14.382806024093101, 14.379762199943338, 14.624745034741274, 14.881989169647692, 14.978404049226008, 15.005411447088482, 15.116878104147574, 15.231825025098999, 15.350723382005462, 15.498381007050199, 15.580397391936735, 15.67842699954941, 15.824147440601575, 16.127352200242672, 16.39340112612006, 16.56288085089476, 16.822106345473596, 16.870645345688338, 17.19575750851426, 17.417893124250966, 17.66588814156274, 18.1292052818935, 18.31567430612822, 18.664025414358854, 18.983511000733152, 19.224833168025572, 19.28549104995732, 18.983254289797756, 19.251749286653705, 19.354591750667684, 19.81666498870513, 19.932493951033994, 20.029644134592946, 20.176761184255607, 20.31868665426955, 20.56087403344567, 20.776179623846048, 20.969420803721416, 21.2020163683998, 21.895361582921513, 22.149158141310608, 22.339225814848273, 22.544843934496946, 22.958001178798522, 23.27753172126079, 23.618569353122712, 24.09342468365107, 24.543719723559704, 24.857479380996388, 25.106104070087294, 25.561868890319833, 25.948866447642914, 26.258413058179325, 26.648276053842153, 26.80846712943704, 26.77914914752275, 26.732077371564095, 26.706410416659278, 26.697176747886687, 26.658526466032185, 26.7003518767562, 26.633846676950125, 26.6219235082378, 26.898839449401247, 26.847204348333626, 26.87611136367713, 26.845751796643853, 26.971677527590792, 27.197424189725943, 27.18294724510194, 27.12390711478303, 27.283892971044455, 27.156197470820974, 27.05535794707769, 26.90346563442893, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2027, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.333189955487507, 10.306586999463269, 10.62012273881656, 11.465580751745385, 11.830473954855341, 11.714339137016504, 11.855405693751038, 12.076210131920192, 12.29024330932647, 12.262295655693638, 12.169942861538422, 12.158925955470025, 12.169630814238324, 12.084024398207832, 12.613054867642504, 12.6513580513897, 12.636382506577643, 12.827720172796978, 12.799025495092417, 13.016527889098601, 13.163884482316732, 13.394724933186472, 13.835988342919833, 13.900574823578316, 13.944406024778255, 14.137125187087838, 14.216755857297002, 14.317973937701309, 14.825658902238226, 15.070166664203102, 15.187795110231207, 15.309804174306741, 15.394720236385377, 15.5100024086604, 15.636553036815776, 15.775100663478655, 16.026307330857495, 16.177813407808685, 16.344951992900274, 16.537547774095373, 16.712092619233655, 16.948254212404006, 17.09462198370425, 17.369723378791903, 17.68599949451013, 18.137697223631776, 18.35224567671716, 18.538927176076687, 18.958452838781984, 19.26138418712154, 19.426732453101362, 19.378445458217723, 19.166219292412435, 19.410446478455412, 19.62029545623332, 19.737581865058644, 19.867468210284375, 19.998485581726335, 20.155903998004998, 20.427357123352852, 20.6535723746397, 21.099557325677317, 21.301345650627947, 21.53794379865909, 21.91779782981305, 22.177343160637385, 22.42556247680379, 22.721142586392023, 23.108894554972988, 23.43775584200834, 23.67399052154953, 23.902870363682414, 24.206593743866343, 24.558907015814864, 25.102666448661637, 25.946632556477667, 26.56163597678802, 26.94050227247919, 27.026868577425176, 26.933559027391972, 26.89484730886641, 26.821767585517193, 26.851743011467526, 26.898964974592417, 26.845503884184758, 27.103966319107816, 27.47318956476881, 27.442952111803475, 27.353649686502713, 27.27709529960334, 27.230045440268892, 27.188169112463235, 27.196169488884326, 27.135742878464864, 27.30417158299395, 27.274951247414837, 27.230099334711426, 27.190775058992116, 27.04243143561797, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2027, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.9867064135238985, 8.029010506805514, 8.149612278109839, 8.189015434068764, 8.22826979784072, 8.300052852709904, 8.383317413302715, 8.416734225233641, 9.87906325647523, 9.962514932695262, 9.99973159331994, 10.566796346258565, 11.159809632956856, 11.731404336242623, 12.325700058604857, 12.199552281041063, 12.651422744232924, 12.614734166768171, 12.876543556234282, 12.807717484002938, 12.793504337572003, 12.960843095021298, 12.951827945202812, 12.927290931432378, 12.911702415750053, 12.95972553974843, 12.951800228997548, 13.271380904394587, 13.248296898116116, 13.210292090502767, 13.20772052533864, 13.771643841107013, 13.806365538070944, 13.851490436752686, 13.836955282132221, 14.203566423025324, 14.702309138292517, 14.975666544308783, 15.247106380366626, 15.400500926971198, 15.460410213056166, 15.410674421897117, 15.721257498533653, 15.967751397167815, 16.231671029220472, 16.25603599770921, 16.445514155263076, 16.613803159648665, 16.8665716421772, 17.275679808041826, 17.5744512987933, 17.62809182691632, 17.793332623389162, 17.969197201953556, 18.326892902447273, 18.70997330287948, 19.025762055774628, 19.380130076546052, 19.929562458535223, 20.120554926498365, 20.504979937783187, 20.874960612236645, 21.135330505299436, 21.441447925821784, 22.01339833806229, 22.314748129803647, 22.76312208541124, 23.103085543857066, 23.57818912387678, 23.930596528812963, 24.343009738458818, 24.826535128619565, 25.24611644117264, 25.63035712694546, 26.117221341381086, 26.41051410113454, 26.834268447609123, 26.943250481933223, 27.03438042302786, 26.952699860948236, 26.91940703229902, 26.8033230854609, 26.749683246103817, 26.771347759719855, 26.85125965757733, 26.827609327609323, 27.063074881602056, 27.390470982735042, 27.284177007847056, 27.149940495842316, 27.40678272931545, 27.286465469515004, 27.137765500819867, 27.108066041790917, 27.049828783116762, 27.057101665158932, 27.24268748858913, 27.17984754397667, 27.11080202416666, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.969506856765212, 9.301360193477104, 9.405074320981484, 9.320111750346651, 9.83122334449992, 10.599047341898531, 10.52823207362706, 10.573190353965723, 10.60262005811542, 11.065222940801648, 11.058903149027415, 11.360614563742068, 11.635158601421912, 11.69194008226996, 11.749508804374798, 11.847021767972402, 11.924478023788565, 11.972103383973733, 12.013196407352288, 12.035427395523378, 12.033728864197272, 12.36573223356884, 12.721722117923997, 13.014207570229763, 13.114422615430174, 13.650155535464082, 13.541235649467618, 13.884229374418716, 14.517358342310455, 14.503265101748559, 15.005978604909942, 15.128274693538504, 15.225758019140004, 15.246753409603176, 15.568599799055214, 15.814396493786203, 16.09716115287333, 16.306977492043774, 16.582183007515816, 16.792521819017324, 17.075763777707973, 17.339421594483586, 17.367621189859918, 17.759255352960647, 17.9130608782586, 18.14925774120888, 18.240079960337425, 18.531640075048816, 18.712067004407128, 18.719846197337738, 18.95032536130579, 19.301133557827736, 19.48412882824357, 19.514648773896393, 19.65325672700039, 19.812721979760283, 20.350696159199067, 20.799105650817395, 21.016065178531893, 21.381808095424084, 21.79604871869221, 21.946207184280595, 22.064243378710817, 22.331218824378386, 22.70966035024976, 23.05978486776571, 23.290040175762826, 23.69597039494867, 24.06115530556247, 24.34776989073284, 24.678548371195138, 24.908806167550036, 25.224381991259914, 25.538103711678843, 25.839058946867016, 26.267890718987495, 26.487708940981626, 27.149261082553107, 27.03401573623391, 26.96741655331816, 26.94329451570166, 27.118557533868685, 27.09269879515809, 27.116230961941202, 27.063459133231316, 26.966196965401174, 27.161610975215766, 27.119300918928165, 27.017257965533826, 26.96365433622453, 26.9417258554597, 26.99834924120904, 26.915174878884557, 26.895961074898228, 27.110547047393947, 27.25561442005408, 27.093323131285064, 27.057271455347003, 26.98215433535589, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.974289800068456, 8.86416369031418, 8.925958636896787, 9.008599849549986, 9.040785013025566, 9.772546927006424, 9.796393354411967, 9.834552895413113, 9.877278920861803, 9.832166368658743, 9.883595221832628, 10.008870870284722, 10.54819328977528, 11.14438311105628, 11.190561882624134, 11.688452778861524, 11.707537017386215, 11.650572652536399, 11.74446475239139, 11.77349791893045, 11.792084424097155, 12.177081618339129, 12.891361874634846, 12.936678925149057, 12.935571559189905, 13.355635728040923, 13.413885341246049, 13.390007666775647, 13.381532022579586, 13.741574411433627, 13.70216485005931, 13.807637703594636, 13.870157787516725, 13.847453003960682, 14.123973296560294, 14.175131390535503, 14.25726679328946, 14.456507127943732, 14.549459220819447, 14.732990438093015, 14.947279928906632, 15.069859021960172, 15.186935844858748, 15.331241665529493, 15.46882219873083, 15.45946470185303, 15.826218249468019, 15.992883699058325, 16.13855277964857, 16.276392833487087, 16.415856066456463, 16.464239346851162, 16.606059059212605, 16.9755372463286, 17.689063679976087, 18.200433449340572, 18.397022982051812, 18.582665725943404, 19.2480529242652, 19.606771868900235, 19.784739085252408, 20.211044285680206, 20.559362426061494, 21.024888403393078, 21.277047830889625, 21.492987333491808, 21.844471535093778, 22.231800022716712, 22.684059642105147, 23.265536190195064, 23.708219994630202, 24.090437992877863, 24.564739688979042, 25.00744134781237, 25.331830063713166, 25.930718810325697, 26.46727038727893, 27.17660840066904, 27.067687499687622, 26.99327013726692, 26.937822484859613, 26.904625746600587, 26.902263460084836, 26.855433287217394, 27.03071177922793, 27.45343328175477, 27.353923645006876, 27.35727096671998, 27.37131039442151, 27.235166096755858, 27.283793973434356, 27.251409000028897, 27.534123110797225, 27.39017881629015, 27.351567053719773, 27.245883966812347, 27.19867873065031, 27.0734262926814, 27.066859483212692, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.04368910829699, 8.060741359117971, 8.169981287512925, 8.968306997611496, 9.022583039537496, 9.099809562542315, 9.160525642554571, 9.210078226825845, 9.227870157758334, 9.925489732057443, 9.901756504998161, 9.959484100153967, 10.011271776790217, 10.059493194537392, 10.02901872352578, 10.61598236601741, 10.628048895689854, 10.634662645198237, 10.649254118714536, 11.172312536917376, 11.63888712760586, 11.716900681145578, 11.82494539908512, 11.85366694818476, 11.9457353744473, 11.977116601976624, 12.397032704979788, 12.36503079419466, 12.76544037724956, 12.900869370950893, 12.969750522822615, 13.049788649133006, 13.104842194750777, 13.079181024699635, 13.229089131361638, 13.583146283778907, 13.869198718941073, 13.962213583007834, 14.280526358697154, 14.384672100257998, 14.968913645208614, 15.108570161335475, 15.46089113563102, 15.60135850996065, 15.764406951504156, 16.06693598550027, 16.26376900074194, 16.556917542412265, 16.704836874542345, 17.05756741298695, 17.410986153643186, 17.700341824423326, 17.864285918180073, 18.09001908582356, 18.3383063629644, 18.809544069228686, 18.806502914687613, 19.367836765494644, 19.616076408875468, 20.075408551488884, 20.599734540893465, 21.00830756528466, 21.18875193797938, 21.3973748647785, 21.639273545190296, 22.22179297207986, 22.592209016432633, 22.64811272080962, 23.128804290619946, 23.579881669488596, 24.13341050908318, 24.463994243742526, 24.829590679386122, 25.12126523432352, 25.542707348170374, 26.114230385814626, 26.39560427746216, 27.15701450002096, 27.067687499687622, 26.948494689156675, 26.909609132519392, 26.83943667687916, 26.842115947054758, 26.785875626357996, 26.88151491181533, 27.09812366298246, 27.287487220860147, 27.2667003766229, 27.15543614904488, 27.565112120972866, 27.515586552952563, 27.40320922794722, 27.403300612264875, 27.343385331879272, 27.250499454359485, 27.176077752938333, 27.151203218818875, 27.080016388708515, 27.018339489858477, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2028, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.969506856765212, 9.301360193477104, 9.405074320981484, 9.320111750346651, 9.83122334449992, 10.599047341898531, 10.52823207362706, 10.573190353965723, 10.60262005811542, 11.065222940801648, 11.058903149027415, 11.360614563742068, 11.635158601421912, 11.69194008226996, 11.749508804374798, 11.847021767972402, 11.924478023788565, 11.972103383973733, 12.013196407352288, 12.035427395523378, 12.033728864197272, 12.36573223356884, 12.721722117923997, 13.014207570229763, 13.114422615430174, 13.650155535464082, 13.541235649467618, 13.884229374418716, 14.517358342310455, 14.503265101748559, 15.005978604909942, 15.128274693538504, 15.225758019140004, 15.246753409603176, 15.568599799055214, 15.814396493786203, 16.09716115287333, 16.306977492043774, 16.582183007515816, 16.792521819017324, 17.075763777707973, 17.339421594483586, 17.367621189859918, 17.759255352960647, 17.9130608782586, 18.14925774120888, 18.240079960337425, 18.531640075048816, 18.712067004407128, 18.719846197337738, 18.95032536130579, 19.301133557827736, 19.48412882824357, 19.514648773896393, 19.65325672700039, 19.812721979760283, 20.350696159199067, 20.799105650817395, 21.016065178531893, 21.381808095424084, 21.79604871869221, 21.946207184280595, 22.064243378710817, 22.331218824378386, 22.70966035024976, 23.05978486776571, 23.290040175762826, 23.69597039494867, 24.06115530556247, 24.34776989073284, 24.678548371195138, 24.908806167550036, 25.224381991259914, 25.538103711678843, 25.839058946867016, 26.267890718987495, 26.487708940981626, 27.149261082553107, 27.03401573623391, 26.96741655331816, 26.94329451570166, 27.118557533868685, 27.09269879515809, 27.116230961941202, 27.063459133231316, 26.966196965401174, 27.161610975215766, 27.119300918928165, 27.017257965533826, 26.96365433622453, 26.9417258554597, 26.99834924120904, 26.915174878884557, 26.895961074898228, 27.110547047393947, 27.25561442005408, 27.093323131285064, 27.057271455347003, 26.98215433535589, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2028, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.253431667379438, 10.201705077759001, 10.10882658451777, 11.3658581221984, 11.554959174185548, 11.457732557766427, 11.981549588196799, 11.964502855800248, 12.228525841698202, 12.203332291130163, 12.23611848737278, 12.340054568601344, 12.301057836476057, 12.378828597968356, 12.446816040410004, 12.482491640266208, 12.522205817139028, 12.596060187185385, 12.919881829513963, 12.918205980270702, 13.04753132694471, 13.31251553783011, 13.397810410438096, 13.61872867997646, 13.875698535733333, 13.98732933585183, 14.172184813120777, 14.379694550094005, 14.617938679353932, 15.143799731242394, 15.37817751381792, 15.348793513661432, 15.426143168017123, 15.456152251833313, 15.593648698221577, 16.003991863157776, 16.277361439818733, 16.56924797907403, 16.953693789319157, 17.287484887912917, 17.394460227428958, 17.430665285697803, 17.538350729972336, 17.58481155552553, 17.770915982659755, 18.066618563865145, 18.426555234351344, 18.611795076111175, 18.941786805079047, 19.166979233135976, 19.362203171583353, 19.59779971413484, 19.990818837878262, 20.360296724183126, 20.533245992069126, 20.85230981742027, 21.144528107879225, 21.417997668850067, 21.56451678338264, 21.843370915383336, 22.098718619524284, 22.28649211277353, 22.500272214907785, 22.74472673945896, 23.085788649113052, 23.239737919426528, 23.29824538825405, 24.062035915947988, 24.33385183398839, 24.697839995539812, 24.745149227002173, 25.137366877604567, 25.38802731732417, 25.801975976451285, 26.20110465704452, 26.463435554344645, 26.81030506255851, 27.136315683331958, 27.051015600234997, 26.99327013726692, 26.960182305981046, 26.92723943781143, 26.87908307196852, 26.746692213889506, 26.723303880494136, 26.775901954011516, 26.73592991924697, 26.793777471091396, 26.756830813029243, 26.718463663746594, 26.691150685769482, 27.42272909333156, 27.337873451514504, 27.32006650005837, 27.287287724262, 27.180092697962287, 27.04130719895253, 27.20983843695138, 27.151501166014206, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2028, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.6530507020268, 9.730359231663547, 10.568416298319313, 10.873870801918429, 11.104479945727665, 10.963888736417928, 10.78571034579741, 11.049369109494018, 11.073906899605946, 12.167900020309903, 12.965694950685611, 12.848957014468136, 12.803241576571663, 12.803724209095598, 12.777210868397665, 12.98165798501844, 13.22194301338738, 13.770943672143973, 13.805900265214207, 13.849507296723832, 13.848691720385494, 13.855598515908568, 13.889724085644426, 14.084383960191026, 14.105001844844198, 14.159878660522345, 13.943038337793471, 14.19657251320435, 14.230439849983384, 14.666006365801415, 14.749101378735935, 14.861321659213289, 14.995999099458315, 15.30260213711386, 15.515426449256953, 15.600823633669824, 15.78686960321074, 15.88666744861209, 15.854749101856807, 15.99885233813805, 16.0152331548612, 16.105376069597078, 16.29439688237878, 16.48605826013039, 16.616682589704762, 16.79730081842096, 17.051390359503575, 17.263219986276948, 17.190144390823768, 17.725542509053277, 18.03583929615219, 17.949101752013572, 18.148725870216936, 18.249192882449275, 18.518443288929326, 18.976626669831294, 19.211064657277603, 19.47903453016033, 19.713781050024547, 20.187390985732378, 20.40477071679298, 20.640687233594548, 20.839079210660884, 21.21735770849717, 21.821849266020102, 22.243206569334543, 22.51453603787811, 22.670601710359517, 23.366409415945363, 23.74116179692798, 24.152424540785603, 24.46172876144438, 24.836224620741007, 25.062502648600958, 25.282104525327416, 25.61056894701366, 26.038841177200272, 26.42403734230584, 27.065796904818583, 27.19292081853759, 27.147064562842406, 27.134150979953304, 27.323703760928776, 27.46892285529347, 27.382344635911366, 27.352020598851606, 27.264896462787863, 27.197369905280656, 27.205297137474744, 27.175821947824, 27.10061039278887, 27.295631190448795, 27.323646139183204, 27.22841940446777, 27.17956875220926, 27.116386746157094, 27.09408922309922, 27.04572890654423, 27.014358170679877, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2028, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.957534902676441, 8.808258026530657, 8.813614181085635, 8.842702221062538, 8.8667497644312, 9.520590092486463, 9.54218788233994, 9.585152282051622, 10.946610890856856, 10.920279793871448, 11.259071448298894, 11.32129358746153, 11.582937308698884, 11.487917469029746, 11.530865781845653, 11.622540241396809, 11.694250403599762, 11.724045879433412, 11.8179125523626, 12.129723010308364, 12.489383118865646, 12.544268158776243, 12.578359166391945, 12.664549865872802, 12.727023066099477, 12.741347171481168, 12.808092048950853, 13.141285518715168, 13.252642127123949, 13.271769383047562, 13.280690371974089, 13.406588017043763, 13.53114688315012, 13.859688199556047, 14.05086191847445, 14.370740963412116, 14.998972145133106, 15.00711272797252, 15.11987420984853, 15.187270993267393, 15.228584765172096, 15.509215609163588, 15.570061459144238, 15.724166767576719, 15.933481084900377, 16.09966681906944, 16.476738145116617, 16.557054699608386, 16.698185456578557, 17.091805524561206, 17.229148023654066, 17.51951616529722, 18.107148730981592, 18.501331043882928, 19.015040194811444, 19.17559959322438, 19.442806942894297, 19.934365636862793, 20.319920322370024, 20.63157772406367, 20.835611086733994, 21.173408350114535, 21.332495783053126, 21.844942445391098, 22.090311943427814, 22.313285337688107, 22.571052666163155, 22.992138731277798, 23.515061661279027, 23.99753895634567, 24.29830646709762, 24.716757441772344, 25.0930998777659, 25.367148606945513, 25.93900439431838, 26.31413708859245, 26.446994662789358, 26.911616317182418, 27.067687499687622, 27.002787556235198, 26.879504920415553, 26.937602982265197, 26.944449631879586, 26.877584242444613, 27.440733518035103, 27.389775079992475, 27.363060901105605, 27.259838418100212, 27.273176246604187, 27.168778551430115, 27.111345598876902, 27.100563677531795, 27.145640777165358, 27.06114794013476, 27.094590846432435, 26.987175864636836, 27.056229836074934, 26.991940412543403, 26.889013189767148, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 10.602774921912797, 11.66103333161233, 11.34058612746552, 11.611948305190948, 11.578943479914122, 11.527252207290664, 11.752072233769951, 12.296161148595857, 12.25525995608922, 12.21056440818788, 12.178750420576268, 12.135104264052881, 12.142866607520732, 12.128867777539936, 12.659780032308444, 13.37977392274593, 13.554808821959307, 13.701165192592477, 13.843160803060762, 13.722285745762195, 13.69126199303925, 13.676989732472839, 13.743985256481789, 13.687988757987322, 13.734393835567813, 13.767543758872144, 13.852792220391407, 14.19604895973896, 14.402232774030526, 14.845602109227961, 14.982703160386187, 15.258884247681593, 15.332472019748703, 15.392684685760742, 15.702937293691562, 15.98858046525169, 16.11449752087004, 16.255386952806287, 16.431953385642757, 16.60675621606354, 16.726535785189185, 17.03858895222293, 17.24813856580085, 17.688117413446065, 17.863586013221838, 18.307695818737376, 18.437004238328488, 18.65687962162302, 19.053541977251804, 19.278360169453315, 18.940929124272362, 18.85874050520021, 18.961755067893176, 19.10133990051684, 19.45086411568949, 19.752772046745424, 20.167777702736696, 20.4147352827216, 20.76686638201753, 20.99234173327476, 21.156922386503314, 21.299617814742714, 21.481989449090012, 22.008557291871806, 22.44920402178467, 22.781954403190046, 23.212446037949398, 23.65949858324548, 23.973632343497076, 24.199643634467908, 24.54038707048618, 24.794035202894896, 25.05832289574887, 25.675434301383536, 26.038816187312268, 26.341600849107838, 26.703055448507047, 26.84611060196523, 27.067687499687622, 27.00121975110067, 26.952699860948236, 26.88634346698338, 26.855048720597974, 26.84091941897083, 26.968191982487728, 26.95297071726646, 26.929069978205984, 26.86692262454396, 26.68381122308976, 26.859002787124865, 26.844570639339672, 27.15822138075869, 27.124068458873218, 27.12435199384081, 27.316729076922066, 27.32424798861382, 27.243387716987215, 27.035393087328412, 26.957914681831397, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.92936332143105, 8.013803837577463, 8.098149439304999, 8.224365047232157, 8.308800587400125, 8.29460208676393, 8.336607668790197, 9.063792021484005, 9.791192397417133, 10.452309356727735, 10.484630943026547, 10.39337979970911, 10.975270623930951, 11.926213205488697, 11.839343516812207, 11.769754933199362, 11.789816022449802, 11.764702931020619, 12.195727012286522, 12.043724138601245, 11.984842580256153, 11.984156416986844, 12.0470266851411, 12.036693115203084, 12.786512570225414, 12.773134007358145, 13.188159014958497, 13.21320116137703, 13.470406621181152, 13.428298290732025, 13.416885245956076, 13.681786677622565, 14.240039088895411, 14.233232420470646, 14.47207296284273, 15.02774441163816, 14.954742143284966, 14.964376927622702, 14.99879456416893, 15.419268991224918, 15.432861822951379, 15.513608129945647, 15.83144308223422, 15.892589796726384, 16.008717995320545, 16.211735215260077, 16.30669767186493, 16.64746440488554, 16.986111021907067, 17.218864536308196, 17.47937897537953, 17.594628868217402, 17.91308088998731, 18.096723224003092, 18.67611994992658, 18.854309547009773, 19.056215118424575, 19.227742364729075, 19.738418468130792, 20.21434799500568, 20.692638178199495, 20.949349021385906, 21.315019906101494, 21.739147529028592, 22.10742512024269, 22.723198961304643, 23.03353424957794, 23.489748829581046, 23.848149734093465, 24.238965808735358, 24.58096132424391, 24.849147863906953, 25.35037698581983, 25.525676109587327, 25.869434183359363, 26.25522738715108, 26.34888568507805, 26.851736101914835, 27.067687499687622, 26.952699860948236, 26.867287089925078, 26.77230846635817, 26.770281976526572, 26.882760542805524, 26.851603959924798, 27.15150315588032, 27.11423301465674, 27.12178870785596, 27.332615475369447, 27.48037448453513, 27.421787355147924, 27.424957038276865, 27.256641274286363, 27.2852400472141, 27.228173136805655, 27.150624485819037, 27.058366561844636, 27.206727729769227, 27.077983173963883, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.9924686782349985, 9.408079792481058, 9.981908012803869, 11.12464153453914, 11.117187508646795, 11.067623546636424, 11.00324728204128, 10.986741684159378, 11.401950592853094, 11.808474269319685, 12.161989651682978, 12.634800137272695, 12.56109404878282, 12.859204012888684, 12.810244041307978, 12.724440294467096, 13.000976320411164, 12.933446183450448, 12.90454398593213, 13.281211283266355, 13.288983245653416, 13.24948455158731, 13.200427130844233, 13.523896319083182, 13.52567418356892, 13.86710126379779, 14.106827558646163, 14.182655684555428, 14.362287230502723, 14.430454620910139, 14.488552561547014, 14.710404334787231, 14.757248037715145, 14.796971634697394, 15.091723664988502, 15.765635476779922, 16.04304107735794, 16.019588536316416, 16.111331704911986, 16.378750520207245, 16.66886467369542, 16.906747015019967, 17.176634780945616, 17.27579547163922, 17.53645206120759, 17.59528265769567, 17.719143352385245, 17.82397364890919, 18.021845799249743, 18.32393070577974, 18.635428193157992, 18.782808628018582, 18.94284598198258, 18.935573847270387, 19.279311904905178, 19.64404166981759, 19.701102238700287, 19.755017446708607, 19.890409687042673, 20.22884141317396, 20.85437153824212, 21.169627825742506, 21.691923023856692, 21.924279045551636, 22.088164189560107, 22.67368686843172, 22.952809791382805, 23.403696335290746, 23.815979463119817, 24.062276567441415, 24.481173652387884, 24.811585447174718, 25.281544155717828, 25.665321464554204, 25.93830728370866, 26.338163209892755, 26.748006362253303, 26.84252569921362, 27.067687499687622, 26.952699860948236, 26.851291568075332, 26.788618533836612, 26.786580359015826, 26.859525682791553, 26.85173767622161, 27.05574903528366, 27.08793882256134, 27.047140046758457, 27.35602719406025, 27.269123750451378, 27.44454678951093, 27.43250433723056, 27.391012084745725, 27.31517316196791, 27.242133087978292, 27.113786461727408, 27.061694655342283, 26.980850870726535, 27.104102576948954, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2029, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 10.602774921912797, 11.66103333161233, 11.34058612746552, 11.611948305190948, 11.578943479914122, 11.527252207290664, 11.752072233769951, 12.296161148595857, 12.25525995608922, 12.21056440818788, 12.178750420576268, 12.135104264052881, 12.142866607520732, 12.128867777539936, 12.659780032308444, 13.37977392274593, 13.554808821959307, 13.701165192592477, 13.843160803060762, 13.722285745762195, 13.69126199303925, 13.676989732472839, 13.743985256481789, 13.687988757987322, 13.734393835567813, 13.767543758872144, 13.852792220391407, 14.19604895973896, 14.402232774030526, 14.845602109227961, 14.982703160386187, 15.258884247681593, 15.332472019748703, 15.392684685760742, 15.702937293691562, 15.98858046525169, 16.11449752087004, 16.255386952806287, 16.431953385642757, 16.60675621606354, 16.726535785189185, 17.03858895222293, 17.24813856580085, 17.688117413446065, 17.863586013221838, 18.307695818737376, 18.437004238328488, 18.65687962162302, 19.053541977251804, 19.278360169453315, 18.940929124272362, 18.85874050520021, 18.961755067893176, 19.10133990051684, 19.45086411568949, 19.752772046745424, 20.167777702736696, 20.4147352827216, 20.76686638201753, 20.99234173327476, 21.156922386503314, 21.299617814742714, 21.481989449090012, 22.008557291871806, 22.44920402178467, 22.781954403190046, 23.212446037949398, 23.65949858324548, 23.973632343497076, 24.199643634467908, 24.54038707048618, 24.794035202894896, 25.05832289574887, 25.675434301383536, 26.038816187312268, 26.341600849107838, 26.703055448507047, 26.84611060196523, 27.067687499687622, 27.00121975110067, 26.952699860948236, 26.88634346698338, 26.855048720597974, 26.84091941897083, 26.968191982487728, 26.95297071726646, 26.929069978205984, 26.86692262454396, 26.68381122308976, 26.859002787124865, 26.844570639339672, 27.15822138075869, 27.124068458873218, 27.12435199384081, 27.316729076922066, 27.32424798861382, 27.243387716987215, 27.035393087328412, 26.957914681831397, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2029, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.99598581532562, 9.332835061625087, 9.850954674850055, 10.724840181247194, 10.577102816897074, 10.938011657000397, 11.998264283539843, 12.18667430096978, 12.12137804184065, 12.002677743244996, 12.280915842933625, 12.237016532552524, 12.238203172169563, 12.74821824567686, 12.995191654407446, 13.040108600920982, 13.050627687399963, 13.086252991113021, 13.113373007709663, 13.24666551094592, 13.295000838595245, 13.551894637528111, 13.554549416173703, 13.642875955259218, 13.565976853328907, 13.867662701442004, 14.168702794776026, 14.203116004177751, 14.475533825098546, 15.05108882816094, 15.484899131584687, 15.507586186382255, 15.567145209891315, 15.752139083856221, 15.879011918241021, 16.022446947090895, 16.11676111123047, 16.456951837351415, 16.554415878392142, 16.558578495646973, 16.90821298803818, 17.037003339963672, 17.212915840883138, 17.41456601321829, 17.74635538243883, 18.00857994151524, 18.3300940459569, 18.692046446650387, 18.99419076554856, 19.244242352451643, 19.266652775358313, 18.973669838144733, 19.210362576786263, 19.37991686596236, 19.47093825306342, 19.58360312608807, 19.775475633365367, 19.90437604604645, 20.1503058549764, 20.333053391413745, 20.463534394691095, 20.57266442441348, 20.771547841671413, 21.007457620214833, 21.253141050663, 21.772982237126104, 22.054146778452854, 22.46005092370801, 22.96800676493658, 23.346653947034252, 24.262777422828165, 24.60330268972762, 24.956635638465354, 25.28877847427288, 25.60127898846939, 25.890833147715973, 26.33344480282485, 26.807815753332875, 27.03438042302786, 26.973006398760447, 26.87170023604978, 26.703883318929527, 26.656594166264437, 26.61502629767623, 26.689319126287675, 26.607969229630253, 26.732192105189217, 26.666055688667733, 26.64258851359297, 26.61187208629378, 26.86557664001273, 26.81106785066753, 26.770341778771407, 26.90644012288953, 26.97604025246988, 27.16112495222065, 27.10994188662465, 27.050731153379644, 26.908369297882345, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2029, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.946269905459646, 9.832865762786474, 10.639321751002537, 11.035076125674067, 11.799846983874444, 12.03114856550731, 12.175473963424036, 12.323992439445092, 12.277066484574723, 12.215649844107457, 12.182403805733138, 12.172782415434085, 12.132389205070288, 12.288880548983245, 12.544499507915209, 12.517364664043873, 12.459070748261274, 12.746894960584326, 12.770366018146834, 12.896577298396005, 13.15351566361744, 13.227406132241581, 13.330469148673622, 13.385205943993098, 13.40820389388942, 14.221915173037056, 14.67615976351848, 14.598153277222693, 14.600192571457763, 14.856427686954838, 14.937789981512099, 15.226649696612268, 15.28535522295808, 15.434808315499104, 15.665951544397904, 15.763189515268621, 16.08564405117862, 16.178617937579897, 16.3318189801639, 16.804256627159305, 16.963342692105055, 17.074759177233002, 17.216367007947852, 17.63861035375032, 17.792962983525616, 17.965999280893126, 18.319826360722736, 18.50266021530378, 18.953771671162976, 19.12237532808399, 19.346990932342607, 19.519560208453555, 19.30555507361424, 19.54313651670459, 19.93719529707494, 20.067709337593538, 20.223688061241777, 20.15930861173053, 20.38282958572374, 20.666755077023016, 20.995819320825447, 21.495969372820866, 21.79849872553487, 21.927904620450647, 22.14271928633209, 22.560496299682896, 22.76113011981049, 23.023069170989892, 23.3233017625871, 23.74357067547175, 24.05074365099082, 24.377640609495, 24.73819924164155, 25.169486763594474, 25.5078354786602, 25.813607870906672, 26.657431606684955, 27.17684856231796, 26.99327013726692, 26.961772379732196, 26.916668953292632, 26.876412394260363, 26.788328799497425, 26.779310727894106, 26.725273736254113, 26.97500892728658, 26.905424492288898, 27.07205941896953, 27.113042486962573, 27.080983778155687, 27.134032113661057, 27.352490920527227, 27.278102513648548, 27.136117860589103, 27.04588245154022, 27.173543859449268, 27.112813502867073, 27.009637599828224, 26.907974413626974, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2029, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.993254711103641, 8.82098246269819, 10.287240762735742, 10.84758859489668, 10.702886355289731, 10.482758903959443, 10.93913602977707, 10.844707761824273, 10.880898226794411, 10.826270923067256, 11.554369324150594, 11.593122852270227, 12.229006480824708, 12.539452628562245, 12.504917396860169, 12.506118058881613, 12.558019933349826, 12.54940160007147, 12.54864953250904, 12.591539050454973, 13.13354605290406, 13.109276972804828, 13.150604820169672, 13.43471161387973, 13.696755095614172, 13.745812098224553, 13.60091883472573, 13.779518131104984, 14.03039295564259, 14.081729633614422, 14.371746736581622, 14.444860521241672, 14.573949727812114, 14.75024892869588, 15.539228701995322, 15.667676962497007, 15.738690300456565, 15.834073617389802, 16.007503343290434, 16.221131510976747, 16.683915066490723, 17.11369520335986, 17.50598530267316, 17.675475686228893, 17.87892483550234, 18.16434280411482, 18.06981079024423, 18.07291191131189, 18.359200420483504, 18.51631575633601, 18.64279471259292, 18.957549657480072, 19.18719411903106, 19.15961275493198, 19.32189760833122, 19.54136099831546, 19.751814054532062, 20.156920908845326, 20.396940524434353, 20.70915340692232, 20.93403834945987, 21.18776047118143, 21.554797739633415, 21.878023929867542, 22.23869221553098, 22.555187381785952, 22.795645749011246, 23.11119193709216, 23.429226145612432, 23.507545210559265, 24.27813738465702, 24.730228000009085, 25.057186115788433, 25.325966776653747, 25.546660049332594, 25.80007355483871, 26.09941911192023, 26.286842176500514, 26.55691226408826, 26.691833570302055, 26.850869498672687, 26.73316779009522, 26.936342514575685, 26.9126517338531, 26.86854152998583, 26.751312192428124, 26.792181316858827, 26.84861267278775, 27.08444654961239, 27.278337143817243, 27.257363131130525, 27.136930031567957, 27.323283357762524, 27.443275113832357, 27.36777526684898, 27.25485289657264, 27.225380783315412, 27.147455446121686, 27.100310028081303, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_difference", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.34614471559377, 10.412031466827372, 10.826805512421572, 11.219688984504065, 11.012301445487296, 11.334871437086408, 11.859533252511065, 12.131526994276498, 12.182195939409397, 12.979805746363837, 12.669176230070962, 12.620005704488843, 12.752567279744142, 12.697989241385258, 13.127140631992479, 13.00900712353573, 13.00914268685442, 13.188798794253028, 13.439356153392925, 13.408362067233295, 13.656569218531923, 13.709536802495597, 13.851997264123241, 13.935221398442154, 14.016516693351317, 14.091408790514482, 14.194088799292814, 14.190991830121938, 14.542687864312976, 14.698471465743095, 15.148318933021912, 15.294532955813159, 15.55397025822232, 15.599122027649889, 15.69355726690523, 15.7806256339094, 15.962602892962671, 16.117339908090482, 16.201826437513844, 16.350611136325327, 16.43257150345843, 16.60991339541927, 17.061421173245645, 17.41511069699801, 17.48556803494466, 17.671767554413904, 17.854477161453687, 18.41005163409505, 18.71684780347736, 18.984401921177966, 18.733648938741307, 18.859689240009953, 19.367515373826734, 19.55644809205553, 19.64092099089235, 19.759689234353317, 19.91736886995124, 20.175024972731944, 20.278823752650876, 20.45555250947324, 20.43631966886877, 20.664866770947604, 20.80397381311118, 21.281280867634923, 21.512018653781585, 21.840595432957194, 22.08768816327956, 22.834723270394967, 23.136614415273076, 23.505526865419085, 23.964898684887633, 24.357278939320032, 24.79603755381875, 25.206350679208146, 25.550242192386573, 25.87992345785018, 26.224319946321074, 26.6041422483153, 26.989568494530463, 26.98205664892778, 26.88757111564729, 26.804732297512558, 26.631272688921076, 26.70586878293465, 26.714949265764194, 26.69907927274699, 26.70597149667569, 26.689767315089536, 26.689767315089536, 26.59611697568256, 26.87439642373229, 26.940820582807206, 26.903028861707114, 27.09952508171018, 27.0467032967033, 27.083462349169157, 27.284290007283502, 27.10649411599102, 26.917128572913935, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_RH", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.883365135935739, 7.972067790314169, 8.763127281916317, 8.819900529281036, 9.534610302351862, 10.23652719128938, 11.468375359189253, 11.471871769793784, 11.428097852705763, 11.441502685025123, 11.91067976673795, 11.820469924204122, 12.219108466895797, 12.250605725946656, 12.254370622482993, 12.336082370914033, 12.709459081118865, 12.663904395699241, 12.660514549397547, 12.643261646373603, 12.635654307468847, 12.578516015949228, 12.938015740038422, 12.98672969149236, 13.314433473728293, 13.650386010105805, 14.035930828426094, 13.92915855274017, 13.904496145095996, 13.983559842994303, 14.15191071072845, 14.195107322054302, 14.266474179381897, 14.529766059698574, 14.58398980085942, 14.644771803250903, 14.877167007281708, 15.098601450929966, 15.29358255962611, 15.498700936845268, 15.66040461112481, 15.699833704310512, 15.955822182509255, 16.008972064468136, 16.0398582583954, 16.35408269773825, 16.57799586291013, 16.65059111061916, 16.78617579226191, 17.164182878397927, 17.34463357248339, 17.66143385987732, 18.28003753003993, 18.472064312219906, 18.635700981485385, 18.84841786728383, 18.913283740887195, 19.372189642188154, 19.66403307187077, 20.362271400094187, 20.69713194515514, 20.917585925355002, 21.320585077183978, 21.623698851836934, 21.927552038199337, 22.222726633829094, 22.64507429455404, 23.07572232247794, 23.54465122845893, 23.90959385103152, 24.03740050320082, 24.423228455289443, 24.948466974459052, 25.306605712612694, 25.487510667125758, 25.812165583566276, 26.129854697747813, 26.466452044738297, 26.906738920012906, 26.989568494530463, 26.84502720046788, 26.75678565474376, 26.784164200848476, 26.72935550683303, 26.653654261333156, 26.642810848790493, 26.70101562847409, 26.910625678586513, 26.871459956693037, 27.02062463214879, 26.9528350865327, 26.993974570944896, 26.951862271479655, 26.885944946051453, 27.14310802769132, 27.1129158412921, 27.1172976899382, 26.967332896536433, 27.104949495361915, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_error_ratio", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.901949074690205, 8.705567442582783, 9.458160118520174, 9.561192104647255, 10.19794178763187, 10.18764511293078, 11.352519517667838, 11.273185675935324, 11.624416251915406, 11.601201462393803, 12.501294656670597, 12.461614305682476, 12.820477452959139, 12.801870937037965, 12.749515718759973, 13.13737256798383, 13.442105993669118, 13.7528842455326, 13.9900545372037, 14.046794475753726, 14.003266604551243, 14.038841612898, 14.030088174489821, 14.00134097301601, 13.87414874476726, 13.86313508511071, 13.814329253663265, 14.132104128449562, 14.15256593885445, 14.440985020184216, 14.490641471861773, 14.768097318542656, 14.945115613495808, 15.167070245558492, 15.21211640431305, 15.293614396718002, 15.272721178540422, 15.696654290262419, 15.686432123152946, 15.78560567331692, 16.256136721543736, 16.585837100536924, 16.7124374280733, 16.75017537417285, 16.926284847538614, 17.041895885059812, 17.119955586882213, 17.41889306320172, 17.62197050675045, 17.74852177766037, 17.956660137263402, 18.250299416267943, 18.385671551872296, 18.485665408771997, 18.884441605249503, 19.10283461642978, 19.404970824172622, 19.65521559441477, 20.385620393589846, 20.487347992655046, 20.62102889778362, 20.992875006961825, 21.20559944469904, 21.526647050581165, 21.745622002322236, 22.311889499389498, 22.70803967608455, 22.990309729168498, 23.352603050432265, 23.67972326023472, 24.37602990373825, 24.73088294802885, 25.07910576874362, 25.304894809086054, 25.49491879382666, 25.815367419262593, 26.14656344117713, 26.466452044738297, 26.906738920012906, 26.94509805563471, 26.84502720046788, 26.75678565474376, 26.784164200848476, 26.71316307240944, 26.653654261333156, 26.635512963717588, 26.724423050587802, 26.887410896650948, 26.871459956693037, 27.038607920024166, 26.986359517295575, 26.98689146247215, 26.939619231147798, 26.885944946051453, 27.16064167803653, 27.1129158412921, 27.11042955807007, 27.00648795220118, 26.865441491447267, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2030, "method": "4S_normalized", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.385691199033147, 10.607014724065985, 10.886995681134353, 11.251480662221043, 11.132795060465142, 11.681096088731294, 11.950485050656404, 12.257252647244954, 12.506060671997915, 12.514939040453227, 12.465605022280354, 12.420055756946361, 12.29887380374021, 12.419722833745093, 12.575536441831794, 13.044082832538306, 13.007265914495877, 13.307566938348394, 13.54675677097075, 13.497074182497135, 13.555566828996898, 13.489687173219936, 13.716701233693632, 13.75804044864708, 13.887624658384269, 13.92250071040878, 14.330481760984783, 14.811091624801936, 15.11196029225728, 15.340141855596729, 15.470228260094428, 15.602397136523424, 15.70686053663336, 15.77343114114952, 15.878079775290255, 16.185375446708754, 16.290856126262142, 16.331182187887638, 16.50044815857474, 16.56092601480848, 16.77465374797612, 16.94284498489534, 16.999451677577518, 17.163645586166435, 17.629775924543, 17.811488078872795, 17.905962000293275, 17.8983029118888, 18.211534969415936, 18.346538800999543, 18.494315902646907, 18.413902519057483, 18.548729286284622, 18.669420736132167, 18.829851654407225, 19.133282926264325, 19.193379699569622, 19.540879014531914, 19.73800209959461, 19.923183286960267, 20.164179388371707, 20.751172953972016, 21.47132790658189, 21.93063742480102, 22.362366590401837, 22.721608381678863, 22.974930007524037, 23.20854560052026, 23.2383924026918, 23.619367202525236, 24.009757143686, 24.453429561409283, 24.902699748790198, 25.246829349184345, 25.60144526845346, 25.8921080715241, 26.21796012896782, 26.5317341035184, 26.961660151404732, 26.989568494530463, 26.869480281200865, 26.740175462729944, 26.640355050620595, 26.682225655918423, 26.602846423093073, 26.624796634848057, 26.757955225427448, 26.684087621965713, 26.906854172574672, 26.869063981199858, 26.948143779905827, 27.17690106225915, 27.13203615169023, 27.060893323128965, 27.091464853822067, 27.12052075905354, 26.97763910098948, 27.056400498241338, 26.88989959621687, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2030, "method": "three", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 9.337511206841318, 9.850631955036674, 10.200797561353186, 10.99905844569189, 11.26746580898171, 11.192869059192157, 11.484987879455975, 11.80645219083362, 12.24002257972993, 12.706661155341445, 12.875679617404497, 12.849077272837897, 12.767168507963225, 12.765180140606452, 12.656887760747018, 12.896233115751194, 12.828042901771003, 12.79935672940589, 13.020058890847444, 13.024569441730801, 13.06330824710027, 13.111590183661544, 13.331855136733187, 13.588286471042013, 13.522894013475332, 13.611643155624176, 13.641973775017252, 14.037087098884246, 14.098870263282661, 14.1619660040608, 14.476148443912365, 14.740954314897554, 14.855599083410716, 14.968429409661402, 15.220407781094384, 15.440088403187497, 15.657923690482475, 15.529351957089371, 15.833756224387157, 16.19877713601904, 16.513678533143633, 16.75001694117731, 16.934389087314276, 17.17896505784124, 17.441776645024916, 17.550410399464077, 18.033546672589676, 18.265028423224123, 18.718732937233877, 18.89026771282617, 19.271683996130466, 19.552388536315956, 19.00502910782773, 19.132176610627166, 19.482166572995883, 19.740672598007784, 19.80512866950612, 19.920369756522316, 20.088409878012037, 20.245152812242313, 20.397061571101776, 20.57788077238477, 20.745103435914974, 21.08716897785692, 21.517031783109257, 21.875821519496863, 22.245596537822994, 22.718037596824587, 23.09630953459604, 23.592329209325865, 23.900937425596638, 24.318102751667002, 24.65371261723616, 25.091648089613667, 25.45463974696177, 25.80256476273603, 26.050567529845814, 26.329969487241456, 26.452758224286416, 26.692556363232388, 26.746565654258525, 26.779074866179236, 26.723020608673092, 26.70030223050594, 26.651442180324075, 26.604710421250346, 26.603183652902956, 26.67427916556741, 26.59417663404615, 26.80840598814293, 26.777064799933363, 27.058170705678748, 27.222868555405753, 27.208728283912542, 27.187917284032615, 27.174478978730054, 27.155744605437793, 27.032729480369987, 26.948482713452147, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2030, "method": "gain", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 8.002833941270199, 9.336627071131392, 9.856551732792788, 9.864272934971433, 10.642158440324824, 11.340902421276194, 11.58786208804158, 12.17407632697183, 12.4436986206082, 12.366183587226645, 12.313081363316728, 12.379744009430697, 12.418031216417525, 12.413307717524253, 12.457682312944835, 12.53097347083027, 12.637973701932035, 12.70272702929758, 12.702675529456206, 12.821721736414435, 12.8564681369712, 13.233218209682557, 13.483770738623416, 13.598800707363532, 13.74389086784398, 13.748760912559943, 13.917962863895756, 13.913864596461758, 14.019599085462906, 14.102753330228182, 14.394657272945055, 14.695921388893138, 14.837266044709512, 15.28642093509393, 15.550721376596051, 15.747871745724018, 16.010219207932682, 16.39353657079284, 16.584441946385613, 16.616704400876507, 16.702181609802857, 17.201567782063137, 17.41454616155646, 17.515472341203896, 17.812262399741478, 17.941395385730555, 18.10195392965119, 18.379448736446726, 18.564490462603885, 19.042174859402937, 19.26888851341838, 19.209264885549597, 19.535578628999456, 19.702529127506498, 19.937184903162596, 20.11948263490557, 20.32145085870079, 20.254974773911833, 20.444594804010613, 20.709706661522716, 21.256677236079575, 21.495231516063168, 21.742994226324942, 22.203474970819713, 22.46621210529944, 22.673432907221958, 23.057470627627758, 23.4113887627231, 23.745477224411204, 24.039325579310848, 24.32288937849009, 24.49594701910464, 24.956189433092263, 25.630670955901845, 26.08334062061938, 26.440888954521213, 26.74759036592733, 26.921204670290827, 27.017380559026776, 26.99327013726692, 26.935896527166815, 26.880681011331447, 26.833177604075992, 26.800221694163017, 26.859503867469353, 26.874146521091273, 26.878448130163612, 26.847150003900584, 26.81187583786235, 26.773617663244746, 26.750128920056614, 26.90821398439742, 27.091968407075722, 27.042071012583335, 27.14946555729176, 27.038402245859473, 27.172072908827683, 27.158557112216453, 26.928591262569633, 27.001009067886244]}, {"task": "construction", "expert": "rtdetr_x_fp32", "endpoint": "Qwen3.8", "seed": 2030, "method": "posthoc", "calls": [0.0, 0.9986684420772304, 1.9973368841544608, 2.9960053262316912, 3.9946737683089215, 4.993342210386151, 5.9920106524633825, 6.990679094540613, 7.989347536617843, 8.988015978695072, 9.986684420772303, 10.985352862849535, 11.984021304926765, 12.982689747003995, 13.981358189081226, 14.980026631158456, 15.978695073235686, 16.977363515312916, 17.976031957390145, 18.974700399467377, 19.973368841544605, 20.972037283621837, 21.97070572569907, 22.969374167776298, 23.96804260985353, 25.0, 25.998668442077232, 26.99733688415446, 27.996005326231693, 28.99467376830892, 29.993342210386153, 30.99201065246338, 31.990679094540614, 32.98934753661784, 33.988015978695074, 34.9866844207723, 35.98535286284953, 36.98402130492676, 37.982689747003995, 38.98135818908123, 39.98002663115845, 40.97869507323569, 41.977363515312916, 42.97603195739015, 43.97470039946737, 44.973368841544605, 45.97203728362184, 46.97070572569907, 47.969374167776294, 48.968042609853526, 50.0, 50.99866844207723, 51.997336884154464, 52.99600532623169, 53.99467376830892, 54.99334221038615, 55.992010652463385, 56.99067909454061, 57.98934753661784, 58.988015978695074, 59.986684420772306, 60.98535286284953, 61.98402130492676, 62.982689747003995, 63.98135818908123, 64.98002663115847, 65.97869507323568, 66.97736351531292, 67.97603195739015, 68.97470039946738, 69.9733688415446, 70.97203728362183, 71.97070572569906, 72.96937416777631, 73.96804260985353, 75.0, 75.99866844207723, 76.99733688415446, 77.99600532623168, 78.99467376830891, 79.99334221038616, 80.99201065246339, 81.99067909454061, 82.98934753661784, 83.98801597869507, 84.9866844207723, 85.98535286284952, 86.98402130492676, 87.98268974700399, 88.98135818908123, 89.98002663115847, 90.97869507323568, 91.97736351531292, 92.97603195739015, 93.97470039946738, 94.9733688415446, 95.97203728362183, 96.97070572569906, 97.96937416777631, 98.96804260985353, 100.0], "score": [7.8873978710439285, 7.956445724806113, 8.889461770019176, 8.95763422791293, 8.97493314977383, 9.033078296977052, 9.088909018731496, 9.124475335533752, 9.214300719500027, 9.2369490745475, 9.26796633126708, 9.315065211292007, 9.388880645396437, 9.43046623931187, 9.999471639594306, 10.529400794387533, 10.615356689061883, 10.677723250409864, 10.641848793951162, 10.63524725421832, 11.101306105134123, 11.160788704400137, 11.203861084600549, 11.279511470159752, 11.706087908984191, 12.210368762859265, 12.19350112751136, 12.519354852984105, 12.59507611859871, 12.678877260246955, 13.006317924108668, 13.371700645332387, 13.713983360410564, 13.9313144791061, 14.204015815705667, 14.666436713475948, 15.023722254045571, 15.099318014689647, 15.172811503013275, 15.274119746171813, 15.662330621434627, 15.757992116326799, 15.874836021910237, 16.220620325078887, 16.756343621425938, 16.908761586095803, 16.973373948879914, 17.326760836281778, 17.4647243085359, 17.585104085062135, 18.001285289391745, 18.0050561763018, 18.46532772298736, 18.769587811754857, 18.89174983839938, 19.14406646040475, 19.511592821110497, 19.67155210622832, 20.001497737414535, 20.134423231109526, 20.491779453383263, 20.79225936970764, 21.00573978843503, 21.12071734440927, 21.54023083463192, 21.876285410373093, 22.122267737063677, 22.47148146953056, 23.045186196997232, 23.460465280180422, 23.780580782779285, 24.407682610875604, 24.697049626199167, 25.228639376642807, 25.547474042499495, 25.943108535085102, 26.302253451389834, 26.763769609484022, 26.88230343780954, 26.981655361083874, 26.891072982812776, 26.822428524887815, 27.043884418210627, 27.288617874131393, 27.266368088283997, 27.228565427030947, 27.10603295522345, 27.14945508462951, 26.988975703273166, 27.232479513886883, 27.123911149764968, 27.039619917349178, 26.97176422904261, 27.14118595562369, 27.03644633401031, 26.964228427479036, 26.93562937977274, 26.963908174730005, 26.93932659314816, 26.92626234191639, 27.001009067886244]}] \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/lr_control_summary.json b/releases/20260913_lr/paper_data/lr_control_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..c0b6582555365d1dfb0baf49e048e843c2947e04 --- /dev/null +++ b/releases/20260913_lr/paper_data/lr_control_summary.json @@ -0,0 +1,156 @@ +{ + "4S_difference": { + "cub": { + "full": 71.30477624717177, + "middle_high": 66.65309176574908 + }, + "grefcoco": { + "full": 60.10334108982569, + "middle_high": 62.55288392181684 + }, + "nlvr2": { + "full": 86.42688661827373, + "middle_high": 87.82059502279398 + }, + "construction": { + "full": 21.68664457361428, + "middle_high": 23.37179473046792 + }, + "overall": { + "mean": 59.880412132221366, + "std": 0.06230299379475004 + } + }, + "4S_RH": { + "cub": { + "full": 67.80147679642886, + "middle_high": 62.716073314651304 + }, + "grefcoco": { + "full": 59.83696029450736, + "middle_high": 62.2898964392558 + }, + "nlvr2": { + "full": 87.36051668387572, + "middle_high": 89.08451490248436 + }, + "construction": { + "full": 21.11319259968111, + "middle_high": 22.83140793334926 + }, + "overall": { + "mean": 59.028036593623256, + "std": 0.09215446832595518 + } + }, + "4S_error_ratio": { + "cub": { + "full": 67.63433680158339, + "middle_high": 62.495205558823486 + }, + "grefcoco": { + "full": 59.9030469001769, + "middle_high": 62.255919404292584 + }, + "nlvr2": { + "full": 87.36099315635701, + "middle_high": 89.08882695682252 + }, + "construction": { + "full": 21.29756392944788, + "middle_high": 22.972511633206114 + }, + "overall": { + "mean": 59.048985196891294, + "std": 0.10043056570886838 + } + }, + "4S_normalized": { + "cub": { + "full": 71.3715854777698, + "middle_high": 66.74755638479657 + }, + "grefcoco": { + "full": 60.65894058616111, + "middle_high": 63.22949855130163 + }, + "nlvr2": { + "full": 87.36085314912711, + "middle_high": 89.0886402849806 + }, + "construction": { + "full": 21.625047131135076, + "middle_high": 23.29240515579527 + }, + "overall": { + "mean": 60.25410658604827, + "std": 0.055784402767809056 + } + }, + "three": { + "cub": { + "full": 71.27395612317933, + "middle_high": 66.60007988650366 + }, + "grefcoco": { + "full": 60.12755657790998, + "middle_high": 62.59349605468297 + }, + "nlvr2": { + "full": 86.340413568538, + "middle_high": 87.71263160064547 + }, + "construction": { + "full": 21.76578337259674, + "middle_high": 23.45434250634451 + }, + "overall": { + "mean": 59.876927410556014, + "std": 0.06459211304335916 + } + }, + "gain": { + "cub": { + "full": 71.01409198260043, + "middle_high": 66.28415678198927 + }, + "grefcoco": { + "full": 59.96696456319969, + "middle_high": 62.41541160228904 + }, + "nlvr2": { + "full": 86.29461067562258, + "middle_high": 87.67252770031322 + }, + "construction": { + "full": 21.60783215955454, + "middle_high": 23.264626260993804 + }, + "overall": { + "mean": 59.72087484524432, + "std": 0.039237511453832113 + } + }, + "posthoc": { + "cub": { + "full": 65.85980220810953, + "middle_high": 60.37970343902742 + }, + "grefcoco": { + "full": 59.367755773800184, + "middle_high": 61.58463842152389 + }, + "nlvr2": { + "full": 87.32516457516526, + "middle_high": 89.03494498253306 + }, + "construction": { + "full": 21.23728513819176, + "middle_high": 22.948980499229855 + }, + "overall": { + "mean": 58.44750192381669, + "std": 0.0661360515853849 + } + } +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/lr_main_summary.json b/releases/20260913_lr/paper_data/lr_main_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..10c2a90caca1b7d1ae4bb5436a9b8d892ad17bdf --- /dev/null +++ b/releases/20260913_lr/paper_data/lr_main_summary.json @@ -0,0 +1,210 @@ +[ + { + "key": [ + "cub", + "cub", + "MiniCPM" + ], + "confidence_score": 81.429064549534, + "confidence_calls": 0.0, + "lr_score": 81.42906454953402, + "lr_calls": 0.5937176389368313, + "rescued": 4.6, + "harmed": 4.6 + }, + { + "key": [ + "cub", + "cub", + "Qwen3.8" + ], + "confidence_score": 81.429064549534, + "confidence_calls": 0.0, + "lr_score": 81.42906454953402, + "lr_calls": 0.0, + "rescued": 0.0, + "harmed": 0.0 + }, + { + "key": [ + "cub", + "glsim", + "MiniCPM" + ], + "confidence_score": 90.16223679668623, + "confidence_calls": 0.0, + "lr_score": 90.13117017604418, + "lr_calls": 0.8491542975491888, + "rescued": 1.0, + "harmed": 2.8 + }, + { + "key": [ + "cub", + "glsim", + "Qwen3.8" + ], + "confidence_score": 90.16223679668623, + "confidence_calls": 0.0, + "lr_score": 90.16223679668623, + "lr_calls": 0.0, + "rescued": 0.0, + "harmed": 0.0 + }, + { + "key": [ + "grefcoco", + "groundingdino", + "MiniCPM" + ], + "confidence_score": 50.00708958398321, + "confidence_calls": 100.0, + "lr_score": 50.22828460425942, + "lr_calls": 58.18676800045374, + "rescued": 9475.4, + "harmed": 1174.4 + }, + { + "key": [ + "grefcoco", + "groundingdino", + "Qwen3.8" + ], + "confidence_score": 74.47182599325072, + "confidence_calls": 100.0, + "lr_score": 75.12179905283158, + "lr_calls": 88.46666477611093, + "rescued": 18720.8, + "harmed": 1641.6 + }, + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "confidence_score": 61.877889005473165, + "confidence_calls": 52.09426310864078, + "lr_score": 67.91367722542041, + "lr_calls": 19.038085245157816, + "rescued": 727.6, + "harmed": 388.2 + }, + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "confidence_score": 72.43286163967898, + "confidence_calls": 68.38612710206165, + "lr_score": 76.24365482233502, + "lr_calls": 62.00153135014037, + "rescued": 4753.6, + "harmed": 1476.8 + }, + { + "key": [ + "nlvr2", + "nlvr2", + "MiniCPM" + ], + "confidence_score": 90.77276314845376, + "confidence_calls": 85.80038745784601, + "lr_score": 90.80002870058121, + "lr_calls": 91.10425486116092, + "rescued": 2189.4, + "harmed": 684.6 + }, + { + "key": [ + "nlvr2", + "nlvr2", + "Qwen3.8" + ], + "confidence_score": 90.24180239649853, + "confidence_calls": 85.80038745784601, + "lr_score": 90.60055966133314, + "lr_calls": 84.34096290449882, + "rescued": 2163.4, + "harmed": 686.4 + }, + { + "key": [ + "nlvr2", + "vilt", + "MiniCPM" + ], + "confidence_score": 90.64361053311329, + "confidence_calls": 95.93169261677549, + "lr_score": 90.69383655019013, + "lr_calls": 98.05984071177441, + "rescued": 2693.8, + "harmed": 704.8 + }, + { + "key": [ + "nlvr2", + "vilt", + "Qwen3.8" + ], + "confidence_score": 89.96197172992753, + "confidence_calls": 95.93169261677549, + "lr_score": 90.0652938221999, + "lr_calls": 99.61684724115663, + "rescued": 2788.2, + "harmed": 886.8 + }, + { + "key": [ + "construction", + "yolo26x", + "MiniCPM" + ], + "confidence_score": 25.23047077304935, + "confidence_calls": 92.54327563249002, + "lr_score": 25.880436219165098, + "lr_calls": 45.31957390146471, + "rescued": 1139.6, + "harmed": 803.4 + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "confidence_score": 25.974900047621052, + "confidence_calls": 92.21038615179761, + "lr_score": 29.79678330127048, + "lr_calls": 44.766977363515316, + "rescued": 1231.6, + "harmed": 315.0 + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "MiniCPM" + ], + "confidence_score": 22.361115727328066, + "confidence_calls": 100.0, + "lr_score": 22.79931433315267, + "lr_calls": 78.43541944074566, + "rescued": 3288.6, + "harmed": 1452.4 + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "Qwen3.8" + ], + "confidence_score": 27.001009067886244, + "confidence_calls": 100.0, + "lr_score": 26.783631605767233, + "lr_calls": 83.56857523302264, + "rescued": 3448.0, + "harmed": 574.8 + } +] \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/build_figures.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/build_figures.py new file mode 100644 index 0000000000000000000000000000000000000000..a743bd68af655c8172473ef8e5dee264706c2d7d --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/build_figures.py @@ -0,0 +1,101 @@ +"""Main-table curves plus fresh measured component costs; no threshold re-selection.""" +from pathlib import Path +import csv +import json +import numpy as np +import matplotlib +matplotlib.use('Agg') +import matplotlib.pyplot as plt +from matplotlib.ticker import MaxNLocator,LogLocator,NullLocator + +root=Path(__file__).resolve().parent +groups=json.loads((root/'main_table_results.json').read_text(encoding='utf-8'))['groups'] +components={} +curves=[] +selected=[] +for g,tag in zip(groups,['minicpm','qwen']): + expert=g['key'][1] + spec=json.loads((root/f'components/{expert}_specialist.json').read_text()) + router=json.loads((root/f'components/{expert}_router.json').read_text()) + large=json.loads((root/f'components/{tag}_latency.json').read_text()) + flops=json.loads((root/f'components/{tag}_flops.json').read_text()) + assert spec['completed'] and router['status']==large['status']==flops['status']=='complete' + assert router['run']==g['run'] and router['selected_decisions_match_main'] + assert abs(router['actual_call_fraction']-g['methods']['learned']['test_point']['actual_fraction'])<1e-12 + key='|'.join(g['key']) + c={'specialist_gflops':spec['mean_counted_gflops_per_sample'], + 'router_gflops':router['mean_gflops_per_input'], + 'large_gflops':flops['mean_prefill_gflops_counted']+large['mean_completion_tokens']*flops['mean_decode_one_token_gflops_counted'], + 'specialist_ms':spec['mean_end_to_end_latency_ms_per_sample'], + 'router_ms':router['mean_latency_ms_per_input'], + 'large_ms':large['mean_batch_wall_ms_per_sample'], + 'large_samples':large['measured_count'], 'flops_probes':flops['sample_count'], + 'specialist_samples':spec['sample_count'],'router_samples':router['sample_count'],'batch_size':large['batch_size']} + components[key]=c + for method in ['learned','confidence']: + for kind,points in [('curve',g['methods'][method]['test_curve']),('selected',[g['methods'][method]['test_point']])]: + for p in points: + rate=p['actual_fraction'] + row={'pair':key,'method':method,'nominal_validation_budget':p['budget'],'actual_test_call_fraction':rate, + 'test_metric':p['metric'],'estimated_gflops_per_input':c['specialist_gflops']+(c['router_gflops'] if method=='learned' else 0)+rate*c['large_gflops'], + 'estimated_time_ms_per_input':c['specialist_ms']+(c['router_ms'] if method=='learned' else 0)+rate*c['large_ms']} + (curves if kind=='curve' else selected).append(row) +for name,rows in [('full_cost_performance_curves',curves),('selected_points',selected),('fixed_budget_cost_performance',[p for p in curves if p['nominal_validation_budget'] in [0,.05,.1,.25,.5,.75,1]])]: + with (root/(name+'.csv')).open('w',encoding='utf-8',newline='') as f: + writer=csv.DictWriter(f,fieldnames=list(rows[0]));writer.writeheader();writer.writerows(rows) +summary={'components':components,'selected_points':selected,'protocol': + {'method':'original main-table logit contrast; original checkpoints and validation thresholds; seed 42', + 'large_timing':'64 measured examples; batch size one; same full task prompt; two separate warmup examples', + 'cost':'serial component estimates, not a timed integrated service; includes active distilled encoders and routing head only for learned', + 'flops':'multiply-add=2; supported operators; mean prefill (including visual embedding) plus average generated length times one-token decode probe', + 'scope_limits':'mean call cost extrapolated across policies; no policy-specific input/output-length costs; specialist-feature extraction overhead is not separately measured; confidence threshold overhead not separately timed'}} +(root/'cost_summary.json').write_text(json.dumps(summary,indent=2),encoding='utf-8') + +plt.rcParams.update({'font.family':'serif','font.serif':['Times New Roman'],'font.size':7, + 'axes.labelsize':7,'xtick.labelsize':6.5,'ytick.labelsize':6.5,'legend.fontsize':7, + 'pdf.fonttype':42,'ps.fonttype':42,'axes.spines.top':False,'axes.spines.right':False, + 'axes.linewidth':.55,'mathtext.fontset':'stix'}) +styles=[('learned','learned','#176f91','-'),('confidence','conf.','#c78324','--')] +keys=list(components) + +def panel(ax,key,field): + for method,label,color,ls in styles: + pts=[p for p in curves if p['pair']==key and p['method']==method] + assert len(pts)==101 + x=np.array([p[field] for p in pts]);y=np.array([p['test_metric']*100 for p in pts]) + assert np.all(np.diff(x)>=-1e-7) + ax.plot(x,y,color=color,ls=ls,lw=1.05,label=label) + p=next(p for p in selected if p['pair']==key and p['method']==method) + ax.scatter(p[field],100*p['test_metric'],color=color,s=13,edgecolor='white',linewidth=.4,zorder=5) + ax.grid(axis='y',color='#dce1e4',lw=.4) + ax.tick_params(length=2.3,pad=1.5) + ax.yaxis.set_major_locator(MaxNLocator(nbins=3)) + if 'gflops' in field: + ax.set_xscale('log');ax.xaxis.set_major_locator(LogLocator(base=10,numticks=4));ax.xaxis.set_minor_locator(NullLocator()) + ax.set_xlabel('Est. GFLOPs/input',labelpad=1) + else: + ax.xaxis.set_major_locator(MaxNLocator(nbins=3));ax.set_xlabel('Est. time (ms/input)',labelpad=1) + ax.set_ylabel('Acc. (%)' if key.startswith('grefcoco') else 'Macro-F1 (%)',labelpad=1) + +for field,stem in [('estimated_gflops_per_input','flops_performance'),('estimated_time_ms_per_input','latency_performance')]: + fig,axes=plt.subplots(1,2,figsize=(3.44,1.50)) + fig.subplots_adjust(left=.115,right=.995,bottom=.31,top=.83,wspace=.43) + for ax,key,title in zip(axes,keys,['gRefCOCO / M','Construction / Q']): + panel(ax,key,field) + ax.text(.5,1.10,title,transform=ax.transAxes,ha='center',fontsize=7) + h,l=axes[0].get_legend_handles_labels() + fig.legend(h,l,loc='lower center',bbox_to_anchor=(.53,-.025),ncol=2,frameon=False,handlelength=2.2,columnspacing=1.8) + fig.savefig(root/(stem+'.pdf'));fig.savefig(root/(stem+'.png'),dpi=400) + plt.close(fig) + +fig,axes=plt.subplots(2,2,figsize=(3.44,2.05)) +fig.subplots_adjust(left=.115,right=.995,bottom=.20,top=.89,wspace=.43,hspace=.64) +for col,key in enumerate(keys): + for row,field in enumerate(['estimated_gflops_per_input','estimated_time_ms_per_input']): + panel(axes[row,col],key,field) + axes[0,col].text(.5,1.14,'gRefCOCO / M' if col==0 else 'Construction / Q',transform=axes[0,col].transAxes,ha='center',fontsize=7) +h,l=axes[0,0].get_legend_handles_labels() +fig.legend(h,l,loc='lower center',bbox_to_anchor=(.53,-.005),ncol=2,frameon=False,handlelength=2.2,columnspacing=1.8) +fig.savefig(root/'cost_compact.pdf');fig.savefig(root/'cost_compact.png',dpi=400) +plt.close(fig) +print(json.dumps(summary,indent=2)) diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/components/instancevg_router.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/instancevg_router.json new file mode 100644 index 0000000000000000000000000000000000000000..267d00e8ec46bb0f9ba4ea6238f34bcd29d67a12 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/instancevg_router.json @@ -0,0 +1,239 @@ +{ + "status": "complete", + "run": "shared_12_instancevg_MiniCPM", + "config": { + "fraction": 1.0, + "seed": 42, + "freeze_encoder": true, + "fusion": "concat", + "harm_cost": 1.0, + "batch_size": 256, + "weight_decay": 0.01, + "max_epochs": 50, + "patience": 8, + "cpu_threads": 1, + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90, + "expert": "instancevg", + "task": "grefcoco", + "endpoint": "MiniCPM", + "branches": [ + "image", + "text", + "hidden", + "confidence", + "output" + ], + "hidden_layer": "joint_encoder__layer_11", + "id": "shared_12_instancevg_MiniCPM", + "stage": "shared_mandatory_inputs" + }, + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_shared_image_hidden_20260910/runs/shared_12_instancevg_MiniCPM/best.pt", + "score": "logit_rescue-logit_harm", + "test_rows_recomputed": 35263, + "selected_decisions_match_main": true, + "max_score_abs_difference": 0.0, + "actual_call_fraction": 0.30904914499617164, + "sample_count": 64, + "sample_ids": [ + "testB-5056-14012", + "testB-31505-86708", + "testA-57489-158264", + "testA-38865-107108", + "testB-47361-130361", + "testA-5476-15123", + "testA-71404-196892", + "testA-61030-168115", + "testB-35969-99063", + "testB-1675-4724", + "testA-30414-83641", + "testB-47742-131425", + "testB-77456-213280", + "testA-59283-163253", + "testA-57390-157996", + "testA-77229-212665", + "testA-6784-18703", + "testB-11434-31542", + "testB-11675-32181", + "testB-73046-201304", + "testA-28837-79356", + "testA-4563-12619", + "testB-30168-82996", + "testB-92461-254721", + "testA-58293-160462", + "testA-13050-35910", + "testA-39320-108309", + "testB-36423-100313", + "testA-43418-119528", + "testB-23411-64351", + "testB-77730-214031", + "testA-7232-19907", + "testA-17738-48695", + "testA-44914-123593", + "testB-92861-255854", + "testA-69678-192132", + "testA-68785-189658", + "testB-62911-173380", + "testB-19029-52242", + "testA-66284-182759", + "testB-19170-52626", + "testB-8468-23291", + "testB-72940-201009", + "testB-74045-203983", + "testB-93380-257275", + "testB-49369-135990", + "testA-58046-159759", + "testA-57437-158132", + "testA-5754-15895", + "testA-85590-235577", + "testA-92278-254227", + "testB-80373-221257", + "testA-80383-221294", + "testB-71197-196311", + "testB-48080-132352", + "testB-18351-50380", + "testB-1089-3077", + "testB-92466-254737", + "testB-53065-146099", + "testB-42611-117349", + "testA-57690-158796", + "testA-65173-179681", + "testB-90675-249707", + "testB-74709-205815" + ], + "batch_size": 1, + "latency_ms": [ + 13.662165962159634, + 9.513642988167703, + 8.86400998570025, + 7.89595500100404, + 9.713676990941167, + 9.162381989881396, + 9.548410074785352, + 9.09417902585119, + 11.833172990009189, + 14.292752020992339, + 10.746326064690948, + 10.920883971266448, + 19.845983013510704, + 14.003381016664207, + 15.885940054431558, + 14.8980199592188, + 15.25757904164493, + 17.458743997849524, + 17.598357051610947, + 15.274635050445795, + 15.84710506722331, + 31.497937976382673, + 17.706500948406756, + 13.621860067360103, + 12.55292899440974, + 10.675732046365738, + 9.522471926175058, + 11.19849900715053, + 13.095670961774886, + 10.178428026847541, + 11.139138019643724, + 10.480726952664554, + 9.46063396986574, + 10.675741010345519, + 9.826205903664231, + 9.403666015714407, + 10.096908081322908, + 28.78253604285419, + 21.029149065725505, + 10.848066071048379, + 10.954314027912915, + 11.451196041889489, + 11.626868974417448, + 9.91171004716307, + 12.449359055608511, + 11.346090934239328, + 10.699236067011952, + 10.140218073502183, + 12.354035046882927, + 14.74389701616019, + 11.097895912826061, + 11.268700007349253, + 12.365077971480787, + 10.821785079315305, + 11.905013001523912, + 25.439211982302368, + 15.189048950560391, + 12.554107001051307, + 10.584128089249134, + 23.863457958213985, + 14.576193992979825, + 11.179295019246638, + 12.35828804783523, + 11.567718931473792 + ], + "mean_latency_ms_per_input": 13.180577338061994, + "head_mean_latency_ms": 0.6150530008599162, + "gflops_per_input": [ + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392 + ], + "mean_gflops_per_input": 3.642171392, + "operators": [ + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + } + ], + "scope": "image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/components/instancevg_specialist.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/instancevg_specialist.json new file mode 100644 index 0000000000000000000000000000000000000000..aebe35a841d23d1c4030450bea8661304d1e417f --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/instancevg_specialist.json @@ -0,0 +1,254 @@ +{ + "component": "specialist", + "expert": "instancevg", + "task": "grefcoco", + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/models/grefcoco/instancevg/InstanceVG-grefcoco.pth", + "sample_count": 64, + "batch_size": 1, + "sample_ids": [ + "testA-30844-84861", + "testB-44353-122077", + "testB-36889-101643", + "testA-49055-135087", + "testA-81245-223633", + "testB-37669-103833", + "testA-5038-13943", + "testB-79646-219274", + "testA-469-1309", + "testB-46644-128438", + "testA-24562-67515", + "testA-47070-129599", + "testA-54554-150141", + "testB-12382-34099", + "testB-23014-63279", + "testB-79647-219275", + "testB-69479-191564", + "testA-15582-42797", + "testB-56692-156000", + "testB-42726-117660", + "testA-80455-221494", + "testB-33115-91174", + "testA-62940-173455", + "testB-31494-86670", + "testA-54586-150233", + "testA-81445-224177", + "testB-8417-23141", + "testA-24438-67190", + "testA-59285-163255", + "testA-73258-201893", + "testB-945-2633", + "testA-54307-149426", + "testB-60557-166822", + "testA-81622-224665", + "testA-69466-191525", + "testA-60731-167321", + "testA-7857-21644", + "testA-44142-121499", + "testB-66162-182407", + "testA-31764-87406", + "testB-38637-106482", + "testB-52921-145709", + "testB-65796-181394", + "testB-2936-8113", + "testA-79438-218690", + "testA-57397-158016", + "testB-23920-65743", + "testB-15419-42358", + "testA-10078-27739", + "testA-43890-120793", + "testA-55116-151689", + "testB-27487-75582", + "testA-26856-73830", + "testB-50845-140107", + "testA-29199-80374", + "testB-65935-181778", + "testA-17736-48684", + "testB-69477-191553", + "testB-22290-61227", + "testA-58476-160950", + "testB-36265-99859", + "testA-5734-15837", + "testB-67952-187353", + "testB-47054-129555" + ], + "model_load_seconds_excluded_from_latency": true, + "forward_latency_ms_per_sample": [ + 41.24195098876953, + 41.74086380004883, + 40.36540985107422, + 44.46102523803711, + 41.2421760559082, + 40.76591873168945, + 41.14704132080078, + 40.88172912597656, + 41.70716857910156, + 41.191871643066406, + 39.91231918334961, + 39.29497528076172, + 40.70723342895508, + 38.920894622802734, + 39.32780838012695, + 38.89958572387695, + 38.59721755981445, + 38.47331237792969, + 39.8744010925293, + 39.87260818481445, + 41.48284912109375, + 40.55779266357422, + 39.05420684814453, + 39.052734375, + 39.143775939941406, + 39.82249450683594, + 39.543006896972656, + 39.368160247802734, + 40.661216735839844, + 39.092193603515625, + 39.85852813720703, + 40.33164978027344, + 38.89593505859375, + 39.29216003417969, + 39.20943832397461, + 40.946590423583984, + 39.44643020629883, + 40.220863342285156, + 39.61030578613281, + 39.639007568359375, + 39.78163146972656, + 39.223201751708984, + 39.00921630859375, + 38.99894332885742, + 41.13721466064453, + 39.94972610473633, + 41.41766357421875, + 39.28073501586914, + 39.721214294433594, + 39.24937438964844, + 39.40460968017578, + 41.30179214477539, + 42.94054412841797, + 41.5063362121582, + 44.03267288208008, + 44.55561447143555, + 44.24995040893555, + 44.87868881225586, + 45.14652633666992, + 44.85055923461914, + 42.39228820800781, + 41.121856689453125, + 45.708351135253906, + 41.8752326965332 + ], + "end_to_end_latency_ms_per_sample": [ + 47.29630902875215, + 46.27182998228818, + 47.56885801907629, + 45.48592597711831, + 45.399176073260605, + 46.442538034170866, + 45.95420998521149, + 46.45739798434079, + 44.90085388533771, + 43.80481690168381, + 44.848884106613696, + 43.67590101901442, + 46.40063806436956, + 44.26892101764679, + 44.44277100265026, + 44.71806308720261, + 44.62323593907058, + 44.6264409692958, + 44.96095096692443, + 44.103492982685566, + 45.13338196557015, + 44.1846780013293, + 48.1606520479545, + 46.43543600104749, + 46.19023704435676, + 45.247678994201124, + 44.801001087762415, + 45.467636082321405, + 44.22409099061042, + 44.7448689956218, + 58.0457029864192, + 46.10048606991768, + 44.56539999227971, + 44.47658592835069, + 46.05250502936542, + 44.743441976606846, + 44.77924096863717, + 43.877715012058616, + 44.05005706939846, + 42.87725198082626, + 43.836933095008135, + 45.634721987880766, + 44.748114072717726, + 46.21323198080063, + 52.24333005025983, + 48.90036000870168, + 48.12136595137417, + 47.65231907367706, + 46.71858798246831, + 48.516221111640334, + 43.90157596208155, + 46.83465394191444, + 46.49809293914586, + 42.6300170365721, + 47.39421408157796, + 48.35387098137289, + 48.176085110753775, + 48.29539393540472, + 47.32040502130985, + 46.35261802468449, + 47.482443042099476, + 47.437409055419266, + 47.27676196489483, + 48.198972013778985 + ], + "mean_forward_latency_ms_per_sample": 40.71232491731644, + "median_forward_latency_ms_per_sample": 40.08529472351074, + "mean_end_to_end_latency_ms_per_sample": 46.08042127620138, + "median_end_to_end_latency_ms_per_sample": 45.79446598654613, + "flop_profiles": [ + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + } + ], + "mean_counted_gflops_per_sample": 7.502680064, + "flops_scope": "GPU FlopCounterMode dense products, convolutions and supported attention; multiply-add counted as 2 FLOPs. Elementwise, normalization, custom ops and postprocessing may be missing.", + "latency_scope": "A800 wall-clock CUDA forward latency and end-to-end latency including image read/preprocess, excluding model load; batch processing uses the recorded batch size.", + "completed": true +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/components/minicpm_flops.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/minicpm_flops.json new file mode 100644 index 0000000000000000000000000000000000000000..b373baf7cb4a2c5a1df109891d3ad7dcd70b15c7 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/minicpm_flops.json @@ -0,0 +1,261 @@ +{ + "status": "complete", + "model_kind": "MiniCPM", + "task": "grefcoco", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/MiniCPM-V-4_5", + "sample_count": 8, + "records": [ + { + "sample_id": "testA-5038-13945", + "prompt_tokens_in_reference_tensor": 528, + "input_shapes": { + "input_ids": [ + 1, + 528 + ], + "attention_mask": [ + 1, + 528 + ] + }, + "prefill_flops_counted": 10720928684032, + "decode_one_token_flops_counted": 15446671488, + "prefill_gflops_counted": 10720.928684032, + "decode_one_token_gflops_counted": 15.446671488, + "prefill_operators": { + "aten.bmm": 164433561600, + "aten.mm": 7335973519360, + "vision:aten.convolution": 4210569216, + "vision:aten.addmm": 2568098906112, + "vision:aten.bmm": 610809421824, + "vision:aten.mm": 35773218816, + "vision:aten.baddbmm": 1629487104 + }, + "decode_operators": { + "aten.bmm": 312017024, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testA-73305-202014", + "prompt_tokens_in_reference_tensor": 528, + "input_shapes": { + "input_ids": [ + 1, + 528 + ], + "attention_mask": [ + 1, + 528 + ] + }, + "prefill_flops_counted": 10720928684032, + "decode_one_token_flops_counted": 15446671488, + "prefill_gflops_counted": 10720.928684032, + "decode_one_token_gflops_counted": 15.446671488, + "prefill_operators": { + "aten.bmm": 164433561600, + "aten.mm": 7335973519360, + "vision:aten.convolution": 4210569216, + "vision:aten.addmm": 2568098906112, + "vision:aten.bmm": 610809421824, + "vision:aten.mm": 35773218816, + "vision:aten.baddbmm": 1629487104 + }, + "decode_operators": { + "aten.bmm": 312017024, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testA-26952-74099", + "prompt_tokens_in_reference_tensor": 523, + "input_shapes": { + "input_ids": [ + 1, + 523 + ], + "attention_mask": [ + 1, + 523 + ] + }, + "prefill_flops_counted": 10648371484032, + "decode_one_token_flops_counted": 15443722368, + "prefill_gflops_counted": 10648.371484032, + "decode_one_token_gflops_counted": 15.443722368, + "prefill_operators": { + "aten.bmm": 161334035840, + "aten.mm": 7266515845120, + "vision:aten.convolution": 4210569216, + "vision:aten.addmm": 2568098906112, + "vision:aten.bmm": 610809421824, + "vision:aten.mm": 35773218816, + "vision:aten.baddbmm": 1629487104 + }, + "decode_operators": { + "aten.bmm": 309067904, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testB-53254-146620", + "prompt_tokens_in_reference_tensor": 520, + "input_shapes": { + "input_ids": [ + 1, + 520 + ], + "attention_mask": [ + 1, + 520 + ] + }, + "prefill_flops_counted": 10604851319808, + "decode_one_token_flops_counted": 15441952896, + "prefill_gflops_counted": 10604.851319808, + "decode_one_token_gflops_counted": 15.441952896, + "prefill_operators": { + "aten.bmm": 159488476160, + "aten.mm": 7224841240576, + "vision:aten.convolution": 4210569216, + "vision:aten.addmm": 2568098906112, + "vision:aten.bmm": 610809421824, + "vision:aten.mm": 35773218816, + "vision:aten.baddbmm": 1629487104 + }, + "decode_operators": { + "aten.bmm": 307298432, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testB-37889-104439", + "prompt_tokens_in_reference_tensor": 528, + "input_shapes": { + "input_ids": [ + 1, + 528 + ], + "attention_mask": [ + 1, + 528 + ] + }, + "prefill_flops_counted": 10637704296448, + "decode_one_token_flops_counted": 15446671488, + "prefill_gflops_counted": 10637.704296448, + "decode_one_token_gflops_counted": 15.446671488, + "prefill_operators": { + "aten.bmm": 164433561600, + "aten.mm": 7335973519360, + "vision:aten.convolution": 4113027072, + "vision:aten.addmm": 2508904759296, + "vision:aten.bmm": 587593949184, + "vision:aten.mm": 35093741568, + "vision:aten.baddbmm": 1591738368 + }, + "decode_operators": { + "aten.bmm": 312017024, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testA-58523-161084", + "prompt_tokens_in_reference_tensor": 396, + "input_shapes": { + "input_ids": [ + 1, + 396 + ], + "attention_mask": [ + 1, + 396 + ] + }, + "prefill_flops_counted": 6668292011520, + "decode_one_token_flops_counted": 15368814720, + "prefill_gflops_counted": 6668.29201152, + "decode_one_token_gflops_counted": 15.36881472, + "prefill_operators": { + "aten.bmm": 92493891072, + "aten.mm": 5502290919424, + "vision:aten.convolution": 1403523072, + "vision:aten.addmm": 925557751808, + "vision:aten.bmm": 134078357504, + "vision:aten.mm": 11924406272, + "vision:aten.baddbmm": 543162368 + }, + "decode_operators": { + "aten.bmm": 234160256, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testB-3419-9449", + "prompt_tokens_in_reference_tensor": 528, + "input_shapes": { + "input_ids": [ + 1, + 528 + ], + "attention_mask": [ + 1, + 528 + ] + }, + "prefill_flops_counted": 10720928684032, + "decode_one_token_flops_counted": 15446671488, + "prefill_gflops_counted": 10720.928684032, + "decode_one_token_gflops_counted": 15.446671488, + "prefill_operators": { + "aten.bmm": 164433561600, + "aten.mm": 7335973519360, + "vision:aten.convolution": 4210569216, + "vision:aten.addmm": 2568098906112, + "vision:aten.bmm": 610809421824, + "vision:aten.mm": 35773218816, + "vision:aten.baddbmm": 1629487104 + }, + "decode_operators": { + "aten.bmm": 312017024, + "aten.mm": 15134654464 + } + }, + { + "sample_id": "testB-24035-66084", + "prompt_tokens_in_reference_tensor": 521, + "input_shapes": { + "input_ids": [ + 1, + 521 + ], + "attention_mask": [ + 1, + 521 + ] + }, + "prefill_flops_counted": 10619356861568, + "decode_one_token_flops_counted": 15442542720, + "prefill_gflops_counted": 10619.356861568, + "decode_one_token_gflops_counted": 15.44254272, + "prefill_operators": { + "aten.bmm": 160102483072, + "aten.mm": 7238732775424, + "vision:aten.convolution": 4210569216, + "vision:aten.addmm": 2568098906112, + "vision:aten.bmm": 610809421824, + "vision:aten.mm": 35773218816, + "vision:aten.baddbmm": 1629487104 + }, + "decode_operators": { + "aten.bmm": 307888256, + "aten.mm": 15134654464 + } + } + ], + "mean_prefill_gflops_counted": 10167.670253184, + "mean_decode_one_token_gflops_counted": 15.435464832, + "scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "answer_generation": "not performed; one known-token decode probe only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/components/minicpm_latency.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/minicpm_latency.json new file mode 100644 index 0000000000000000000000000000000000000000..48f59b93141dabdd57edef7b48240386f4a9a6b8 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/minicpm_latency.json @@ -0,0 +1,2105 @@ +{ + "status": "complete", + "model_kind": "MiniCPM", + "task": "grefcoco", + "expert": "instancevg", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/MiniCPM-V-4_5", + "device": "cuda:0", + "gpu_memory_utilization": 0.85, + "sample_count_requested": 66, + "warmup_count": 2, + "measured_count": 64, + "batch_size": 1, + "sample_ids": [ + "testA-29199-80373", + "testB-44351-122075", + "testA-91850-253019", + "testB-36888-101641", + "testA-54586-150233", + "testB-79644-219272", + "testB-65935-181776", + "testA-81622-224664", + "testA-10078-27739", + "testA-15582-42797", + "testA-5038-13943", + "testA-30844-84861", + "testB-15419-42356", + "testB-27486-75580", + "testB-945-2632", + "testA-24438-67190", + "testB-66161-182405", + "testB-42726-117659", + "testB-69439-191450", + "testA-54307-149426", + "testB-37669-103831", + "testB-50844-140105", + "testA-79438-218690", + "testA-62940-173454", + "testA-81245-223632", + "testA-47070-129599", + "testB-12381-34098", + "testA-44142-121498", + "testA-73258-201893", + "testB-46644-128436", + "testA-43889-120792", + "testB-36082-99363", + "testB-33108-91152", + "testA-26856-73830", + "testA-81271-223709", + "testA-54554-150141", + "testB-38636-106481", + "testA-5734-15837", + "testB-67558-186325", + "testB-47053-129554", + "testA-93064-256422", + "testB-60557-166821", + "testA-55116-151688", + "testB-65796-181392", + "testB-23014-63278", + "testA-24562-67515", + "testA-17736-48684", + "testA-58475-160949", + "testA-80455-221493", + "testA-31763-87405", + "testB-23920-65742", + "testA-469-1309", + "testB-56691-155998", + "testB-31493-86668", + "testA-59284-163254", + "testA-57397-158015", + "testB-2935-8111", + "testB-22290-61226", + "testB-8416-23140", + "testA-60731-167320", + "testA-49055-135086", + "testA-7857-21644", + "testB-79645-219273", + "testB-52921-145707", + "testB-69479-191562", + "testA-69466-191524" + ], + "batches": [ + { + "sample_ids": [ + "testA-91850-253019" + ], + "batch_size": 1, + "generation_latency_seconds": 0.8861644599819556, + "batch_wall_latency_seconds": 0.9864439839730039, + "amortized_generation_ms_per_sample": 886.1644599819556, + "amortized_batch_wall_ms_per_sample": 986.4439839730039, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[420 ,220 ,710 ,640 ],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 420, + 220, + 710, + 640 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 530 + } + ] + }, + { + "sample_ids": [ + "testB-36888-101641" + ], + "batch_size": 1, + "generation_latency_seconds": 1.3824354230891913, + "batch_wall_latency_seconds": 1.3937634559115395, + "amortized_generation_ms_per_sample": 1382.4354230891913, + "amortized_batch_wall_ms_per_sample": 1393.7634559115395, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[30,570,720,900],\"score\":0.95},{\"bbox_2d\":[390,260,960,860],\"score\":0.93}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 30, + 570, + 720, + 900 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 390, + 260, + 960, + 860 + ], + "score": 0.93 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 60, + "prompt_tokens": 528 + } + ] + }, + { + "sample_ids": [ + "testA-54586-150233" + ], + "batch_size": 1, + "generation_latency_seconds": 10.028413712978363, + "batch_wall_latency_seconds": 10.048457957920618, + "amortized_generation_ms_per_sample": 10028.413712978363, + "amortized_batch_wall_ms_per_sample": 10048.457957920618, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[480 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 660 + } + ] + }, + { + "sample_ids": [ + "testB-79644-219272" + ], + "batch_size": 1, + "generation_latency_seconds": 7.480462849023752, + "batch_wall_latency_seconds": 7.510920200962573, + "amortized_generation_ms_per_sample": 7480.462849023752, + "amortized_batch_wall_ms_per_sample": 7510.920200962573, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\":[120,390,270,790],\"score\":0.92},{\"bbox_2d\":[160,430,300,730],\"score\":0.91},{\"bbox_2d\":[270,560,320,740],\"score\":0.90},{\"bbox_2d\":[300,550,350,720],\"score\":0.89},{\"bbox_2d\":[340,540,380,700],\"score\":0.88},{\"bbox_2d\":[450,560,500,770],\"score\":0.87},{\"bbox_2d\":[480,580,530,760],\"score\":0.86},{\"bbox_2d\":[530,560,570,710],\"score\":0.85},{\"bbox_2d\":[570,530,740,780],\"score\":0.84},{\"bbox_2d\":[670,550,740,810],\"score\":0.83},{\"bbox_2d\":[740,540,800,760],\"score\":0.82},{\"bbox_2d\":[800,540,840,700],\"score\":0.81},{\"bbox_2d\":[840,540,900,780],\"score\":0.80}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 120, + 390, + 270, + 790 + ], + "score": 0.92 + }, + { + "bbox_2d": [ + 160, + 430, + 300, + 730 + ], + "score": 0.91 + }, + { + "bbox_2d": [ + 270, + 560, + 320, + 740 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 300, + 550, + 350, + 720 + ], + "score": 0.89 + }, + { + "bbox_2d": [ + 340, + 540, + 380, + 700 + ], + "score": 0.88 + }, + { + "bbox_2d": [ + 450, + 560, + 500, + 770 + ], + "score": 0.87 + }, + { + "bbox_2d": [ + 480, + 580, + 530, + 760 + ], + "score": 0.86 + }, + { + "bbox_2d": [ + 530, + 560, + 570, + 710 + ], + "score": 0.85 + }, + { + "bbox_2d": [ + 570, + 530, + 740, + 780 + ], + "score": 0.84 + }, + { + "bbox_2d": [ + 670, + 550, + 740, + 810 + ], + "score": 0.83 + }, + { + "bbox_2d": [ + 740, + 540, + 800, + 760 + ], + "score": 0.82 + }, + { + "bbox_2d": [ + 800, + 540, + 840, + 700 + ], + "score": 0.81 + }, + { + "bbox_2d": [ + 840, + 540, + 900, + 780 + ], + "score": 0.8 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 370, + "prompt_tokens": 520 + } + ] + }, + { + "sample_ids": [ + "testB-65935-181776" + ], + "batch_size": 1, + "generation_latency_seconds": 10.180860323016532, + "batch_wall_latency_seconds": 10.195898448000662, + "amortized_generation_ms_per_sample": 10180.860323016532, + "amortized_batch_wall_ms_per_sample": 10195.898448000662, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[340 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 524 + } + ] + }, + { + "sample_ids": [ + "testA-81622-224664" + ], + "batch_size": 1, + "generation_latency_seconds": 0.2510422539198771, + "batch_wall_latency_seconds": 0.26205689006019384, + "amortized_generation_ms_per_sample": 251.0422539198771, + "amortized_batch_wall_ms_per_sample": 262.05689006019384, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 524 + } + ] + }, + { + "sample_ids": [ + "testA-10078-27739" + ], + "batch_size": 1, + "generation_latency_seconds": 0.26944356702733785, + "batch_wall_latency_seconds": 0.2834722528932616, + "amortized_generation_ms_per_sample": 269.44356702733785, + "amortized_batch_wall_ms_per_sample": 283.4722528932616, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testA-15582-42797" + ], + "batch_size": 1, + "generation_latency_seconds": 1.9161002039909363, + "batch_wall_latency_seconds": 1.9308117219479755, + "amortized_generation_ms_per_sample": 1916.1002039909363, + "amortized_batch_wall_ms_per_sample": 1930.8117219479755, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[80,280,400,980],\"score\":0.95},{\"bbox_2d\":[310,390,510,980],\"score\":0.93},{\"bbox_2d\":[540,400,640,870],\"score\":0.88}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 80, + 280, + 400, + 980 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 310, + 390, + 510, + 980 + ], + "score": 0.93 + }, + { + "bbox_2d": [ + 540, + 400, + 640, + 870 + ], + "score": 0.88 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 88, + "prompt_tokens": 539 + } + ] + }, + { + "sample_ids": [ + "testA-5038-13943" + ], + "batch_size": 1, + "generation_latency_seconds": 0.7828407019842416, + "batch_wall_latency_seconds": 0.7960362600861117, + "amortized_generation_ms_per_sample": 782.8407019842416, + "amortized_batch_wall_ms_per_sample": 796.0362600861117, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[490,240,780,980],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 490, + 240, + 780, + 980 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 33, + "prompt_tokens": 524 + } + ] + }, + { + "sample_ids": [ + "testA-30844-84861" + ], + "batch_size": 1, + "generation_latency_seconds": 10.480434139957651, + "batch_wall_latency_seconds": 10.485813157982193, + "amortized_generation_ms_per_sample": 10480.434139957651, + "amortized_batch_wall_ms_per_sample": 10485.813157982193, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[470 \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 499 column 1 (char 2020)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 543 + } + ] + }, + { + "sample_ids": [ + "testB-15419-42356" + ], + "batch_size": 1, + "generation_latency_seconds": 10.714943284052424, + "batch_wall_latency_seconds": 10.730338808032684, + "amortized_generation_ms_per_sample": 10714.943284052424, + "amortized_batch_wall_ms_per_sample": 10730.338808032684, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[110 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 526 + } + ] + }, + { + "sample_ids": [ + "testB-27486-75580" + ], + "batch_size": 1, + "generation_latency_seconds": 10.180824494105764, + "batch_wall_latency_seconds": 10.188868912984617, + "amortized_generation_ms_per_sample": 10180.824494105764, + "amortized_batch_wall_ms_per_sample": 10188.868912984617, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[530 \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 499 column 1 (char 2020)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 530 + } + ] + }, + { + "sample_ids": [ + "testB-945-2632" + ], + "batch_size": 1, + "generation_latency_seconds": 3.843070151982829, + "batch_wall_latency_seconds": 3.859969821991399, + "amortized_generation_ms_per_sample": 3843.070151982829, + "amortized_batch_wall_ms_per_sample": 3859.969821991399, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[120,530,220,910] ,\"score\":0.92},{\"bbox_2d\":[250,530,350,910] ,\"score\":0.92},{\"bbox_2d\":[380,530,480,910] ,\"score\":0.92},{\"bbox_2d\":[500,520,600,910] ,\"score\":0.92},{\"bbox_2d\":[620,530,720,910] ,\"score\":0.92},{\"bbox_2d\":[740,530,840,910] ,\"score\":0.92}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 120, + 530, + 220, + 910 + ], + "score": 0.92 + }, + { + "bbox_2d": [ + 250, + 530, + 350, + 910 + ], + "score": 0.92 + }, + { + "bbox_2d": [ + 380, + 530, + 480, + 910 + ], + "score": 0.92 + }, + { + "bbox_2d": [ + 500, + 520, + 600, + 910 + ], + "score": 0.92 + }, + { + "bbox_2d": [ + 620, + 530, + 720, + 910 + ], + "score": 0.92 + }, + { + "bbox_2d": [ + 740, + 530, + 840, + 910 + ], + "score": 0.92 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 179, + "prompt_tokens": 530 + } + ] + }, + { + "sample_ids": [ + "testA-24438-67190" + ], + "batch_size": 1, + "generation_latency_seconds": 10.143939319998026, + "batch_wall_latency_seconds": 10.162826716084965, + "amortized_generation_ms_per_sample": 10143.939319998026, + "amortized_batch_wall_ms_per_sample": 10162.826716084965, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[290,240,590,920],\"score\":0.95},{\"bbox_2d\":[730 \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 472 column 1 (char 1954)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testB-66161-182405" + ], + "batch_size": 1, + "generation_latency_seconds": 10.25821736396756, + "batch_wall_latency_seconds": 10.271643411950208, + "amortized_generation_ms_per_sample": 10258.21736396756, + "amortized_batch_wall_ms_per_sample": 10271.643411950208, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[100 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 523 + } + ] + }, + { + "sample_ids": [ + "testB-42726-117659" + ], + "batch_size": 1, + "generation_latency_seconds": 0.9900629689218476, + "batch_wall_latency_seconds": 1.0045652500120923, + "amortized_generation_ms_per_sample": 990.0629689218476, + "amortized_batch_wall_ms_per_sample": 1004.5652500120923, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[775 ,775 ,999 ,999 ],\"score\":0.92}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 775, + 775, + 999, + 999 + ], + "score": 0.92 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 43, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testB-69439-191450" + ], + "batch_size": 1, + "generation_latency_seconds": 0.19075044698547572, + "batch_wall_latency_seconds": 0.20009391102939844, + "amortized_generation_ms_per_sample": 190.75044698547572, + "amortized_batch_wall_ms_per_sample": 200.09391102939844, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 388 + } + ] + }, + { + "sample_ids": [ + "testA-54307-149426" + ], + "batch_size": 1, + "generation_latency_seconds": 1.2734235550742596, + "batch_wall_latency_seconds": 1.2819705069996417, + "amortized_generation_ms_per_sample": 1273.4235550742596, + "amortized_batch_wall_ms_per_sample": 1281.9705069996417, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[170,0,560,420] ,\"score\":0.95},{\"bbox_2d\":[540,40,830,910] ,\"score\":0.96}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 170, + 0, + 560, + 420 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 540, + 40, + 830, + 910 + ], + "score": 0.96 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 60, + "prompt_tokens": 524 + } + ] + }, + { + "sample_ids": [ + "testB-37669-103831" + ], + "batch_size": 1, + "generation_latency_seconds": 0.21837474894709885, + "batch_wall_latency_seconds": 0.23633889202028513, + "amortized_generation_ms_per_sample": 218.37474894709885, + "amortized_batch_wall_ms_per_sample": 236.33889202028513, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 528 + } + ] + }, + { + "sample_ids": [ + "testB-50844-140105" + ], + "batch_size": 1, + "generation_latency_seconds": 0.23954864998813719, + "batch_wall_latency_seconds": 0.2525435209972784, + "amortized_generation_ms_per_sample": 239.54864998813719, + "amortized_batch_wall_ms_per_sample": 252.5435209972784, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 522 + } + ] + }, + { + "sample_ids": [ + "testA-79438-218690" + ], + "batch_size": 1, + "generation_latency_seconds": 0.7543689489830285, + "batch_wall_latency_seconds": 0.7701102939900011, + "amortized_generation_ms_per_sample": 754.3689489830285, + "amortized_batch_wall_ms_per_sample": 770.1102939900011, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[260,170,520,910],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 260, + 170, + 520, + 910 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 33, + "prompt_tokens": 522 + } + ] + }, + { + "sample_ids": [ + "testA-62940-173454" + ], + "batch_size": 1, + "generation_latency_seconds": 0.8060770370066166, + "batch_wall_latency_seconds": 0.8215904459357262, + "amortized_generation_ms_per_sample": 806.0770370066166, + "amortized_batch_wall_ms_per_sample": 821.5904459357262, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[620,270,750,690],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 620, + 270, + 750, + 690 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 33, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testA-81245-223632" + ], + "batch_size": 1, + "generation_latency_seconds": 2.46771560003981, + "batch_wall_latency_seconds": 2.4794425220461562, + "amortized_generation_ms_per_sample": 2467.71560003981, + "amortized_batch_wall_ms_per_sample": 2479.4425220461562, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[530,110,760,880] ,\"score\":0.95},{\"bbox_2d\":[560,180,630,610] ,\"score\":0.93},{\"bbox_2d\":[690,130,790,740] ,\"score\":0.91},{\"bbox_2d\":[790,110,920,660] ,\"score\":0.90}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 530, + 110, + 760, + 880 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 560, + 180, + 630, + 610 + ], + "score": 0.93 + }, + { + "bbox_2d": [ + 690, + 130, + 790, + 740 + ], + "score": 0.91 + }, + { + "bbox_2d": [ + 790, + 110, + 920, + 660 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 121, + "prompt_tokens": 530 + } + ] + }, + { + "sample_ids": [ + "testA-47070-129599" + ], + "batch_size": 1, + "generation_latency_seconds": 0.22282294591423124, + "batch_wall_latency_seconds": 0.2299416270107031, + "amortized_generation_ms_per_sample": 222.82294591423124, + "amortized_batch_wall_ms_per_sample": 229.9416270107031, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 521 + } + ] + }, + { + "sample_ids": [ + "testB-12381-34098" + ], + "batch_size": 1, + "generation_latency_seconds": 10.451599241001531, + "batch_wall_latency_seconds": 10.457325526047498, + "amortized_generation_ms_per_sample": 10451.599241001531, + "amortized_batch_wall_ms_per_sample": 10457.325526047498, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[290 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 534 + } + ] + }, + { + "sample_ids": [ + "testA-44142-121498" + ], + "batch_size": 1, + "generation_latency_seconds": 1.4406404530163854, + "batch_wall_latency_seconds": 1.4515395000344142, + "amortized_generation_ms_per_sample": 1440.6404530163854, + "amortized_batch_wall_ms_per_sample": 1451.5395000344142, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,100,580,980] ,\"score\":0.95},{\"bbox_2d\":[570,120,1000,900] ,\"score\":0.93}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 100, + 580, + 980 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 570, + 120, + 1000, + 900 + ], + "score": 0.93 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 62, + "prompt_tokens": 523 + } + ] + }, + { + "sample_ids": [ + "testA-73258-201893" + ], + "batch_size": 1, + "generation_latency_seconds": 11.113033535890281, + "batch_wall_latency_seconds": 11.123388323001564, + "amortized_generation_ms_per_sample": 11113.033535890281, + "amortized_batch_wall_ms_per_sample": 11123.388323001564, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[520 \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 251 column 1 (char 1401)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 526 + } + ] + }, + { + "sample_ids": [ + "testB-46644-128436" + ], + "batch_size": 1, + "generation_latency_seconds": 1.2878015459282324, + "batch_wall_latency_seconds": 1.2976085640257224, + "amortized_generation_ms_per_sample": 1287.8015459282324, + "amortized_batch_wall_ms_per_sample": 1297.6085640257224, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[100,180,520,720],\"score\":0.95},{\"bbox_2d\":[450,0,990,740],\"score\":0.96}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 100, + 180, + 520, + 720 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 450, + 0, + 990, + 740 + ], + "score": 0.96 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 59, + "prompt_tokens": 393 + } + ] + }, + { + "sample_ids": [ + "testA-43889-120792" + ], + "batch_size": 1, + "generation_latency_seconds": 0.7570710440631956, + "batch_wall_latency_seconds": 0.7649242469342425, + "amortized_generation_ms_per_sample": 757.0710440631956, + "amortized_batch_wall_ms_per_sample": 764.9242469342425, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[120,260,340,830],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 120, + 260, + 340, + 830 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 33, + "prompt_tokens": 531 + } + ] + }, + { + "sample_ids": [ + "testB-36082-99363" + ], + "batch_size": 1, + "generation_latency_seconds": 1.2985183690907434, + "batch_wall_latency_seconds": 1.3064349760534242, + "amortized_generation_ms_per_sample": 1298.5183690907434, + "amortized_batch_wall_ms_per_sample": 1306.4349760534242, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,100,420,890] ,\"score\":0.95},{\"bbox_2d\":[440,100,990,890] ,\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 100, + 420, + 890 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 440, + 100, + 990, + 890 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 392 + } + ] + }, + { + "sample_ids": [ + "testB-33108-91152" + ], + "batch_size": 1, + "generation_latency_seconds": 0.9395933509804308, + "batch_wall_latency_seconds": 0.9509444029536098, + "amortized_generation_ms_per_sample": 939.5933509804308, + "amortized_batch_wall_ms_per_sample": 950.9444029536098, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0 ,270 ,390 ,450 ],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 270, + 390, + 450 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 41, + "prompt_tokens": 522 + } + ] + }, + { + "sample_ids": [ + "testA-26856-73830" + ], + "batch_size": 1, + "generation_latency_seconds": 10.264511390007101, + "batch_wall_latency_seconds": 10.279962090076879, + "amortized_generation_ms_per_sample": 10264.511390007101, + "amortized_batch_wall_ms_per_sample": 10279.962090076879, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[575 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 522 + } + ] + }, + { + "sample_ids": [ + "testA-81271-223709" + ], + "batch_size": 1, + "generation_latency_seconds": 0.23597171693108976, + "batch_wall_latency_seconds": 0.2577737799147144, + "amortized_generation_ms_per_sample": 235.97171693108976, + "amortized_batch_wall_ms_per_sample": 257.7737799147144, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 520 + } + ] + }, + { + "sample_ids": [ + "testA-54554-150141" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6926666980143636, + "batch_wall_latency_seconds": 0.710583986947313, + "amortized_generation_ms_per_sample": 692.6666980143636, + "amortized_batch_wall_ms_per_sample": 710.583986947313, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,0,1000,250] ,\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 0, + 1000, + 250 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 31, + "prompt_tokens": 530 + } + ] + }, + { + "sample_ids": [ + "testB-38636-106481" + ], + "batch_size": 1, + "generation_latency_seconds": 9.935319383977912, + "batch_wall_latency_seconds": 9.945194306899793, + "amortized_generation_ms_per_sample": 9935.319383977912, + "amortized_batch_wall_ms_per_sample": 9945.194306899793, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[430 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 397 + } + ] + }, + { + "sample_ids": [ + "testA-5734-15837" + ], + "batch_size": 1, + "generation_latency_seconds": 1.2923860140144825, + "batch_wall_latency_seconds": 1.3085041570011526, + "amortized_generation_ms_per_sample": 1292.3860140144825, + "amortized_batch_wall_ms_per_sample": 1308.5041570011526, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,190,310,740],\"score\":0.95},{\"bbox_2d\":[400,230,660,690],\"score\":0.93}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 190, + 310, + 740 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 400, + 230, + 660, + 690 + ], + "score": 0.93 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 59, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testB-67558-186325" + ], + "batch_size": 1, + "generation_latency_seconds": 0.23707255406770855, + "batch_wall_latency_seconds": 0.24415827100165188, + "amortized_generation_ms_per_sample": 237.07255406770855, + "amortized_batch_wall_ms_per_sample": 244.15827100165188, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 520 + } + ] + }, + { + "sample_ids": [ + "testB-47053-129554" + ], + "batch_size": 1, + "generation_latency_seconds": 0.8615183549700305, + "batch_wall_latency_seconds": 0.8810114429797977, + "amortized_generation_ms_per_sample": 861.5183549700305, + "amortized_batch_wall_ms_per_sample": 881.0114429797977, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[420 ,300 ,800 ,800 ],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 420, + 300, + 800, + 800 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testA-93064-256422" + ], + "batch_size": 1, + "generation_latency_seconds": 0.7549799080006778, + "batch_wall_latency_seconds": 0.7621725639328361, + "amortized_generation_ms_per_sample": 754.9799080006778, + "amortized_batch_wall_ms_per_sample": 762.172563932836, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[100,80,750,980],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 100, + 80, + 750, + 980 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 530 + } + ] + }, + { + "sample_ids": [ + "testB-60557-166821" + ], + "batch_size": 1, + "generation_latency_seconds": 0.23407667502760887, + "batch_wall_latency_seconds": 0.24112907599192113, + "amortized_generation_ms_per_sample": 234.07667502760887, + "amortized_batch_wall_ms_per_sample": 241.12907599192113, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 520 + } + ] + }, + { + "sample_ids": [ + "testA-55116-151688" + ], + "batch_size": 1, + "generation_latency_seconds": 0.22477965708822012, + "batch_wall_latency_seconds": 0.23706394899636507, + "amortized_generation_ms_per_sample": 224.77965708822012, + "amortized_batch_wall_ms_per_sample": 237.06394899636507, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 520 + } + ] + }, + { + "sample_ids": [ + "testB-65796-181392" + ], + "batch_size": 1, + "generation_latency_seconds": 0.24896689096931368, + "batch_wall_latency_seconds": 0.26310552388895303, + "amortized_generation_ms_per_sample": 248.96689096931368, + "amortized_batch_wall_ms_per_sample": 263.10552388895303, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 521 + } + ] + }, + { + "sample_ids": [ + "testB-23014-63278" + ], + "batch_size": 1, + "generation_latency_seconds": 0.8068696269765496, + "batch_wall_latency_seconds": 0.8200291059911251, + "amortized_generation_ms_per_sample": 806.8696269765496, + "amortized_batch_wall_ms_per_sample": 820.0291059911251, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[30,300,340,700],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 30, + 300, + 340, + 700 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 523 + } + ] + }, + { + "sample_ids": [ + "testA-24562-67515" + ], + "batch_size": 1, + "generation_latency_seconds": 0.2276058099232614, + "batch_wall_latency_seconds": 0.24499452603049576, + "amortized_generation_ms_per_sample": 227.6058099232614, + "amortized_batch_wall_ms_per_sample": 244.99452603049576, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 526 + } + ] + }, + { + "sample_ids": [ + "testA-17736-48684" + ], + "batch_size": 1, + "generation_latency_seconds": 9.981450814986601, + "batch_wall_latency_seconds": 9.995843482087366, + "amortized_generation_ms_per_sample": 9981.450814986601, + "amortized_batch_wall_ms_per_sample": 9995.843482087366, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[500 \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 499 column 1 (char 2020)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testA-58475-160949" + ], + "batch_size": 1, + "generation_latency_seconds": 0.7565385290654376, + "batch_wall_latency_seconds": 0.76229489792604, + "amortized_generation_ms_per_sample": 756.5385290654376, + "amortized_batch_wall_ms_per_sample": 762.29489792604, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[40,10,580,980],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 40, + 10, + 580, + 980 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 31, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testA-80455-221493" + ], + "batch_size": 1, + "generation_latency_seconds": 1.0085041749989614, + "batch_wall_latency_seconds": 1.020079139037989, + "amortized_generation_ms_per_sample": 1008.5041749989614, + "amortized_batch_wall_ms_per_sample": 1020.0791390379891, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0 , 0 , 560 , 890 ],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 0, + 560, + 890 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 45, + "prompt_tokens": 522 + } + ] + }, + { + "sample_ids": [ + "testA-31763-87405" + ], + "batch_size": 1, + "generation_latency_seconds": 1.3390386949758977, + "batch_wall_latency_seconds": 1.3469314540270716, + "amortized_generation_ms_per_sample": 1339.0386949758977, + "amortized_batch_wall_ms_per_sample": 1346.9314540270716, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[350,880,420,930] ,\"score\":0.92},{\"bbox_2d\":[580,750,640,810] ,\"score\":0.91}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 350, + 880, + 420, + 930 + ], + "score": 0.92 + }, + { + "bbox_2d": [ + 580, + 750, + 640, + 810 + ], + "score": 0.91 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 63, + "prompt_tokens": 389 + } + ] + }, + { + "sample_ids": [ + "testB-23920-65742" + ], + "batch_size": 1, + "generation_latency_seconds": 10.071104103000835, + "batch_wall_latency_seconds": 10.078898616950028, + "amortized_generation_ms_per_sample": 10071.104103000835, + "amortized_batch_wall_ms_per_sample": 10078.898616950028, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[510 \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 499 column 1 (char 2020)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 529 + } + ] + }, + { + "sample_ids": [ + "testA-469-1309" + ], + "batch_size": 1, + "generation_latency_seconds": 0.26207074394915253, + "batch_wall_latency_seconds": 0.2735602979082614, + "amortized_generation_ms_per_sample": 262.07074394915253, + "amortized_batch_wall_ms_per_sample": 273.5602979082614, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 522 + } + ] + }, + { + "sample_ids": [ + "testB-56691-155998" + ], + "batch_size": 1, + "generation_latency_seconds": 10.205810929066502, + "batch_wall_latency_seconds": 10.223061772994697, + "amortized_generation_ms_per_sample": 10205.810929066502, + "amortized_batch_wall_ms_per_sample": 10223.061772994697, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[690 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 523 + } + ] + }, + { + "sample_ids": [ + "testB-31493-86668" + ], + "batch_size": 1, + "generation_latency_seconds": 10.434396203025244, + "batch_wall_latency_seconds": 10.448493773932569, + "amortized_generation_ms_per_sample": 10434.396203025244, + "amortized_batch_wall_ms_per_sample": 10448.493773932569, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[158 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 535 + } + ] + }, + { + "sample_ids": [ + "testA-59284-163254" + ], + "batch_size": 1, + "generation_latency_seconds": 1.32271799701266, + "batch_wall_latency_seconds": 1.3361050819512457, + "amortized_generation_ms_per_sample": 1322.71799701266, + "amortized_batch_wall_ms_per_sample": 1336.1050819512457, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[490,360,680,970],\"score\":0.95},{\"bbox_2d\":[580,310,780,370],\"score\":0.92}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 490, + 360, + 680, + 970 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 580, + 310, + 780, + 370 + ], + "score": 0.92 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 543 + } + ] + }, + { + "sample_ids": [ + "testA-57397-158015" + ], + "batch_size": 1, + "generation_latency_seconds": 0.22141807794105262, + "batch_wall_latency_seconds": 0.2377260560169816, + "amortized_generation_ms_per_sample": 221.41807794105262, + "amortized_batch_wall_ms_per_sample": 237.7260560169816, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 522 + } + ] + }, + { + "sample_ids": [ + "testB-2935-8111" + ], + "batch_size": 1, + "generation_latency_seconds": 10.570877194055356, + "batch_wall_latency_seconds": 10.580568464938551, + "amortized_generation_ms_per_sample": 10570.877194055356, + "amortized_batch_wall_ms_per_sample": 10580.568464938551, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[380 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 530 + } + ] + }, + { + "sample_ids": [ + "testB-22290-61226" + ], + "batch_size": 1, + "generation_latency_seconds": 0.22713651705998927, + "batch_wall_latency_seconds": 0.23939726105891168, + "amortized_generation_ms_per_sample": 227.13651705998927, + "amortized_batch_wall_ms_per_sample": 239.39726105891168, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 5, + "prompt_tokens": 526 + } + ] + }, + { + "sample_ids": [ + "testB-8416-23140" + ], + "batch_size": 1, + "generation_latency_seconds": 1.8887132819509134, + "batch_wall_latency_seconds": 1.9046463000122458, + "amortized_generation_ms_per_sample": 1888.7132819509134, + "amortized_batch_wall_ms_per_sample": 1904.6463000122458, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,280,330,720] ,\"score\":0.95},{\"bbox_2d\":[170,270,560,720] ,\"score\":0.94},{\"bbox_2d\":[730,190,960,790] ,\"score\":0.93}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 280, + 330, + 720 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 170, + 270, + 560, + 720 + ], + "score": 0.94 + }, + { + "bbox_2d": [ + 730, + 190, + 960, + 790 + ], + "score": 0.93 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 90, + "prompt_tokens": 525 + } + ] + }, + { + "sample_ids": [ + "testA-60731-167320" + ], + "batch_size": 1, + "generation_latency_seconds": 1.2317780090961605, + "batch_wall_latency_seconds": 1.246546880924143, + "amortized_generation_ms_per_sample": 1231.7780090961605, + "amortized_batch_wall_ms_per_sample": 1246.546880924143, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,80,420,910],\"score\":0.95},{\"bbox_2d\":[460,0,760,200],\"score\":0.85}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 80, + 420, + 910 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 460, + 0, + 760, + 200 + ], + "score": 0.85 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 56, + "prompt_tokens": 526 + } + ] + }, + { + "sample_ids": [ + "testA-49055-135086" + ], + "batch_size": 1, + "generation_latency_seconds": 1.8898930319119245, + "batch_wall_latency_seconds": 1.899978275061585, + "amortized_generation_ms_per_sample": 1889.8930319119245, + "amortized_batch_wall_ms_per_sample": 1899.978275061585, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[120,140,320,980] ,\"score\":0.95},{\"bbox_2d\":[320,160,620,970] ,\"score\":0.94},{\"bbox_2d\":[620,250,860,980] ,\"score\":0.93}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 120, + 140, + 320, + 980 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 320, + 160, + 620, + 970 + ], + "score": 0.94 + }, + { + "bbox_2d": [ + 620, + 250, + 860, + 980 + ], + "score": 0.93 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 92, + "prompt_tokens": 526 + } + ] + }, + { + "sample_ids": [ + "testA-7857-21644" + ], + "batch_size": 1, + "generation_latency_seconds": 9.8525026650168, + "batch_wall_latency_seconds": 9.865253204014152, + "amortized_generation_ms_per_sample": 9852.5026650168, + "amortized_batch_wall_ms_per_sample": 9865.253204014152, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[650 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 527 + } + ] + }, + { + "sample_ids": [ + "testB-79645-219273" + ], + "batch_size": 1, + "generation_latency_seconds": 2.319699679967016, + "batch_wall_latency_seconds": 2.332257404923439, + "amortized_generation_ms_per_sample": 2319.699679967016, + "amortized_batch_wall_ms_per_sample": 2332.257404923439, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[120,420,540,700] ,\"score\":0.95},{\"bbox_2d\":[540,400,780,690] ,\"score\":0.94},{\"bbox_2d\":[0,450,120,670] ,\"score\":0.93},{\"bbox_2d\":[140,460,260,600] ,\"score\":0.92}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 120, + 420, + 540, + 700 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 540, + 400, + 780, + 690 + ], + "score": 0.94 + }, + { + "bbox_2d": [ + 0, + 450, + 120, + 670 + ], + "score": 0.93 + }, + { + "bbox_2d": [ + 140, + 460, + 260, + 600 + ], + "score": 0.92 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 119, + "prompt_tokens": 521 + } + ] + }, + { + "sample_ids": [ + "testB-52921-145707" + ], + "batch_size": 1, + "generation_latency_seconds": 10.00177818897646, + "batch_wall_latency_seconds": 10.020861182012595, + "amortized_generation_ms_per_sample": 10001.77818897646, + "amortized_batch_wall_ms_per_sample": 10020.861182012595, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[380 ", + "valid_prediction": null, + "error": "Expecting ',' delimiter: line 1 column 529 (char 528)", + "finish_reason": "length", + "completion_tokens": 512, + "prompt_tokens": 531 + } + ] + }, + { + "sample_ids": [ + "testB-69479-191562" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6840293509885669, + "batch_wall_latency_seconds": 0.70091468305327, + "amortized_generation_ms_per_sample": 684.0293509885669, + "amortized_batch_wall_ms_per_sample": 700.91468305327, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,30,974,986],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 30, + 974, + 986 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 30, + "prompt_tokens": 522 + } + ] + }, + { + "sample_ids": [ + "testA-69466-191524" + ], + "batch_size": 1, + "generation_latency_seconds": 0.24351457098964602, + "batch_wall_latency_seconds": 0.2538871930446476, + "amortized_generation_ms_per_sample": 243.51457098964602, + "amortized_batch_wall_ms_per_sample": 253.88719304464757, + "outputs": [ + { + "raw_response": "{\"objects\":[]}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 522 + } + ] + } + ], + "mean_generation_ms_per_sample": 3652.817064514238, + "median_generation_ms_per_sample": 1280.612550501246, + "mean_batch_wall_ms_per_sample": 3667.173011115665, + "median_batch_wall_ms_per_sample": 1289.789535512682, + "mean_prompt_tokens": 517.609375, + "mean_completion_tokens": 178.0625, + "latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.", + "flops_scope": "This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/components/qwen_flops.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/qwen_flops.json new file mode 100644 index 0000000000000000000000000000000000000000..58b8c3f787acbc76f88e2d4185de8db41426bb45 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/qwen_flops.json @@ -0,0 +1,341 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "construction", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "sample_count": 8, + "records": [ + { + "sample_id": "0000184", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0001955", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0000840", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003887", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003433", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0001613", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0002591", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003105", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + } + ], + "mean_prefill_gflops_counted": 90083.398713024, + "mean_decode_one_token_gflops_counted": 51.922600128, + "scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "answer_generation": "not performed; one known-token decode probe only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/components/qwen_latency.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/qwen_latency.json new file mode 100644 index 0000000000000000000000000000000000000000..d4d3a8551f8fc5c964291e5c4768033d935c2b3e --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/qwen_latency.json @@ -0,0 +1,3066 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "construction", + "expert": "yolo26x", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "device": "cuda:0", + "gpu_memory_utilization": 0.85, + "sample_count_requested": 66, + "warmup_count": 2, + "measured_count": 64, + "batch_size": 1, + "sample_ids": [ + "0000894", + "0003510", + "0002450", + "0003360", + "0001470", + "0004484", + "0004166", + "0002167", + "0000380", + "0000516", + "0000176", + "0000943", + "0002853", + "0003176", + "0002495", + "0000756", + "0004213", + "0003472", + "0004290", + "0001443", + "0003365", + "0003782", + "0002089", + "0001710", + "0002148", + "0001289", + "0002751", + "0001252", + "0001922", + "0003581", + "0001233", + "0003383", + "0003266", + "0000824", + "0002144", + "0001450", + "0003429", + "0000205", + "0004209", + "0003623", + "0002496", + "0004049", + "0001483", + "0004197", + "0003082", + "0000758", + "0000564", + "0001580", + "0002136", + "0000949", + "0003045", + "0000013", + "0003889", + "0003261", + "0001629", + "0001569", + "0002536", + "0003053", + "0002661", + "0001664", + "0001351", + "0000304", + "0004537", + "0003801", + "0004321", + "0001878" + ], + "batches": [ + { + "sample_ids": [ + "0002450" + ], + "batch_size": 1, + "generation_latency_seconds": 10.705359333078377, + "batch_wall_latency_seconds": 10.720781230949797, + "amortized_generation_ms_per_sample": 10705.359333078377, + "amortized_batch_wall_ms_per_sample": 10720.781230949797, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker standing in the middle of the site is clearly bareheaded, missing a hard hat.\", \"boxes\": [[518, 668, 532, 745]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker standing in the middle of the site is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 518, + 668, + 532, + 745 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1473 + } + ] + }, + { + "sample_ids": [ + "0003360" + ], + "batch_size": 1, + "generation_latency_seconds": 11.7157509700628, + "batch_wall_latency_seconds": 11.733416861039586, + "amortized_generation_ms_per_sample": 11715.7509700628, + "amortized_batch_wall_ms_per_sample": 11733.416861039586, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the excavation is visible on the right side of the image, bordering a deep drop with no guardrails or fencing.\", \"boxes\": [[590, 315, 1000, 495]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the excavation is visible on the right side of the image, bordering a deep drop with no guardrails or fencing.", + "boxes": [ + [ + 590, + 315, + 1000, + 495 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 158, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001470" + ], + "batch_size": 1, + "generation_latency_seconds": 11.342743383022025, + "batch_wall_latency_seconds": 11.35942490492016, + "amortized_generation_ms_per_sample": 11342.743383022025, + "amortized_batch_wall_ms_per_sample": 11359.42490492016, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the ground is clearly visible without a hard hat.\", \"boxes\": [[560, 849, 582, 905]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the ground is clearly visible without a hard hat.", + "boxes": [ + [ + 560, + 849, + 582, + 905 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 141, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004484" + ], + "batch_size": 1, + "generation_latency_seconds": 13.103367771022022, + "batch_wall_latency_seconds": 13.121180747984909, + "amortized_generation_ms_per_sample": 13103.367771022022, + "amortized_batch_wall_ms_per_sample": 13121.180747984909, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"Workers are standing directly beside the tracks and rear of the yellow excavator, within its normal swing and travel zone.\", \"boxes\": [[608, 505, 640, 655], [658, 502, 690, 646]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "Workers are standing directly beside the tracks and rear of the yellow excavator, within its normal swing and travel zone.", + "boxes": [ + [ + 608, + 505, + 640, + 655 + ], + [ + 658, + 502, + 690, + 646 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 171, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004166" + ], + "batch_size": 1, + "generation_latency_seconds": 11.62961833900772, + "batch_wall_latency_seconds": 11.636370092979632, + "amortized_generation_ms_per_sample": 11629.61833900772, + "amortized_batch_wall_ms_per_sample": 11636.370092979632, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the concrete structure where the workers are standing lacks guardrails or fencing, bordering a deep drop/excavation area.\", \"boxes\": [[0, 620, 510, 835]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the concrete structure where the workers are standing lacks guardrails or fencing, bordering a deep drop/excavation area.", + "boxes": [ + [ + 0, + 620, + 510, + 835 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 154, + "prompt_tokens": 1129 + } + ] + }, + { + "sample_ids": [ + "0002167" + ], + "batch_size": 1, + "generation_latency_seconds": 11.212190569960512, + "batch_wall_latency_seconds": 11.235628957045265, + "amortized_generation_ms_per_sample": 11212.190569960512, + "amortized_batch_wall_ms_per_sample": 11235.628957045265, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the concrete platform borders a deep excavation area without guardrails or fencing, creating a significant fall hazard.\", \"boxes\": [[0, 315, 515, 620]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the concrete platform borders a deep excavation area without guardrails or fencing, creating a significant fall hazard.", + "boxes": [ + [ + 0, + 315, + 515, + 620 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 151, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0000380" + ], + "batch_size": 1, + "generation_latency_seconds": 8.91931619006209, + "batch_wall_latency_seconds": 8.942373727913946, + "amortized_generation_ms_per_sample": 8919.31619006209, + "amortized_batch_wall_ms_per_sample": 8942.373727913946, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0000516" + ], + "batch_size": 1, + "generation_latency_seconds": 8.323322776006535, + "batch_wall_latency_seconds": 8.3395675040083, + "amortized_generation_ms_per_sample": 8323.322776006535, + "amortized_batch_wall_ms_per_sample": 8339.5675040083, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0000176" + ], + "batch_size": 1, + "generation_latency_seconds": 8.365567820030265, + "batch_wall_latency_seconds": 8.383442353922874, + "amortized_generation_ms_per_sample": 8365.567820030265, + "amortized_batch_wall_ms_per_sample": 8383.442353922874, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000943" + ], + "batch_size": 1, + "generation_latency_seconds": 8.261911686044186, + "batch_wall_latency_seconds": 8.284466639976017, + "amortized_generation_ms_per_sample": 8261.911686044186, + "amortized_batch_wall_ms_per_sample": 8284.466639976017, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002853" + ], + "batch_size": 1, + "generation_latency_seconds": 10.991469252971001, + "batch_wall_latency_seconds": 11.013957520015538, + "amortized_generation_ms_per_sample": 10991.469252971001, + "amortized_batch_wall_ms_per_sample": 11013.957520015538, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation in the foreground lacks guardrails or fencing, exposing a significant drop.\", \"boxes\": [[0, 580, 998, 998]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation in the foreground lacks guardrails or fencing, exposing a significant drop.", + "boxes": [ + [ + 0, + 580, + 998, + 998 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003176" + ], + "batch_size": 1, + "generation_latency_seconds": 10.860358019010164, + "batch_wall_latency_seconds": 10.878267464926466, + "amortized_generation_ms_per_sample": 10860.358019010164, + "amortized_batch_wall_ms_per_sample": 10878.267464926466, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing a significant drop.\", \"boxes\": [[0, 500, 640, 998]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing a significant drop.", + "boxes": [ + [ + 0, + 500, + 640, + 998 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002495" + ], + "batch_size": 1, + "generation_latency_seconds": 10.56970894197002, + "batch_wall_latency_seconds": 10.593795233988203, + "amortized_generation_ms_per_sample": 10569.70894197002, + "amortized_batch_wall_ms_per_sample": 10593.795233988203, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker crouching on the ground is clearly bareheaded, missing a hard hat.\", \"boxes\": [[475, 525, 571, 688]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker crouching on the ground is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 475, + 525, + 571, + 688 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 146, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000756" + ], + "batch_size": 1, + "generation_latency_seconds": 8.038495411048643, + "batch_wall_latency_seconds": 8.052992713986896, + "amortized_generation_ms_per_sample": 8038.495411048643, + "amortized_batch_wall_ms_per_sample": 8052.992713986896, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004213" + ], + "batch_size": 1, + "generation_latency_seconds": 8.011779504013248, + "batch_wall_latency_seconds": 8.047042324091308, + "amortized_generation_ms_per_sample": 8011.779504013248, + "amortized_batch_wall_ms_per_sample": 8047.042324091308, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003472" + ], + "batch_size": 1, + "generation_latency_seconds": 8.286275054095313, + "batch_wall_latency_seconds": 8.304093755898066, + "amortized_generation_ms_per_sample": 8286.275054095313, + "amortized_batch_wall_ms_per_sample": 8304.093755898066, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004290" + ], + "batch_size": 1, + "generation_latency_seconds": 8.341199293034151, + "batch_wall_latency_seconds": 8.358467961079441, + "amortized_generation_ms_per_sample": 8341.199293034151, + "amortized_batch_wall_ms_per_sample": 8358.467961079441, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001443" + ], + "batch_size": 1, + "generation_latency_seconds": 15.180880905943923, + "batch_wall_latency_seconds": 15.206637792987749, + "amortized_generation_ms_per_sample": 15180.880905943923, + "amortized_batch_wall_ms_per_sample": 15206.637792987749, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the left side of the image, standing on the ground near the scaffolding, is clearly bareheaded and not wearing a hard hat.\", \"boxes\": [[238, 458, 278, 688]]}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"A worker is standing on a low concrete beam/ledge without a safety harness, exposed to a fall from a height of approximately 3 meters or more.\", \"boxes\": [[315, 325, 368, 505]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the left side of the image, standing on the ground near the scaffolding, is clearly bareheaded and not wearing a hard hat.", + "boxes": [ + [ + 238, + 458, + 278, + 688 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "A worker is standing on a low concrete beam/ledge without a safety harness, exposed to a fall from a height of approximately 3 meters or more.", + "boxes": [ + [ + 315, + 325, + 368, + 505 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 207, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003365" + ], + "batch_size": 1, + "generation_latency_seconds": 8.154706532950513, + "batch_wall_latency_seconds": 8.171265691053122, + "amortized_generation_ms_per_sample": 8154.706532950513, + "amortized_batch_wall_ms_per_sample": 8171.265691053122, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003782" + ], + "batch_size": 1, + "generation_latency_seconds": 8.263969205901958, + "batch_wall_latency_seconds": 8.278776310035028, + "amortized_generation_ms_per_sample": 8263.969205901958, + "amortized_batch_wall_ms_per_sample": 8278.776310035028, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002089" + ], + "batch_size": 1, + "generation_latency_seconds": 8.21782910998445, + "batch_wall_latency_seconds": 8.233122498961166, + "amortized_generation_ms_per_sample": 8217.82910998445, + "amortized_batch_wall_ms_per_sample": 8233.122498961166, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1531 + } + ] + }, + { + "sample_ids": [ + "0001710" + ], + "batch_size": 1, + "generation_latency_seconds": 8.018760634935461, + "batch_wall_latency_seconds": 8.032283170963638, + "amortized_generation_ms_per_sample": 8018.760634935461, + "amortized_batch_wall_ms_per_sample": 8032.2831709636375, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0002148" + ], + "batch_size": 1, + "generation_latency_seconds": 10.378318366943859, + "batch_wall_latency_seconds": 10.404688362963498, + "amortized_generation_ms_per_sample": 10378.318366943859, + "amortized_batch_wall_ms_per_sample": 10404.688362963498, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker in the middle distance is clearly bareheaded, missing a hard hat.\", \"boxes\": [[561, 300, 582, 408]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker in the middle distance is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 561, + 300, + 582, + 408 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 144, + "prompt_tokens": 1531 + } + ] + }, + { + "sample_ids": [ + "0001289" + ], + "batch_size": 1, + "generation_latency_seconds": 11.48702211806085, + "batch_wall_latency_seconds": 11.514577882946469, + "amortized_generation_ms_per_sample": 11487.02211806085, + "amortized_batch_wall_ms_per_sample": 11514.577882946469, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation on the right side lacks guardrails or fencing, bordering a steep drop of several meters.\", \"boxes\": [[430, 240, 998, 480]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation on the right side lacks guardrails or fencing, bordering a steep drop of several meters.", + "boxes": [ + [ + 430, + 240, + 998, + 480 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 156, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002751" + ], + "batch_size": 1, + "generation_latency_seconds": 8.446506896987557, + "batch_wall_latency_seconds": 8.467677644919604, + "amortized_generation_ms_per_sample": 8446.506896987557, + "amortized_batch_wall_ms_per_sample": 8467.677644919604, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001252" + ], + "batch_size": 1, + "generation_latency_seconds": 8.315838309004903, + "batch_wall_latency_seconds": 8.333076176000759, + "amortized_generation_ms_per_sample": 8315.838309004903, + "amortized_batch_wall_ms_per_sample": 8333.076176000759, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001922" + ], + "batch_size": 1, + "generation_latency_seconds": 8.236222366103902, + "batch_wall_latency_seconds": 8.253340549999848, + "amortized_generation_ms_per_sample": 8236.222366103902, + "amortized_batch_wall_ms_per_sample": 8253.340549999848, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1107 + } + ] + }, + { + "sample_ids": [ + "0003581" + ], + "batch_size": 1, + "generation_latency_seconds": 8.227389317005873, + "batch_wall_latency_seconds": 8.240366365062073, + "amortized_generation_ms_per_sample": 8227.389317005873, + "amortized_batch_wall_ms_per_sample": 8240.366365062073, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1694 + } + ] + }, + { + "sample_ids": [ + "0001233" + ], + "batch_size": 1, + "generation_latency_seconds": 11.756323368987069, + "batch_wall_latency_seconds": 11.764942721929401, + "amortized_generation_ms_per_sample": 11756.323368987069, + "amortized_batch_wall_ms_per_sample": 11764.942721929401, + "outputs": [ + { + "raw_response": "{\n \"events\": {\n \"rule_1_ppe_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_2_fall_protection_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_3_unprotected_edge_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_4_excavator_proximity_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n }\n }\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 163, + "prompt_tokens": 1453 + } + ] + }, + { + "sample_ids": [ + "0003383" + ], + "batch_size": 1, + "generation_latency_seconds": 8.106548791052774, + "batch_wall_latency_seconds": 8.117786544025876, + "amortized_generation_ms_per_sample": 8106.548791052774, + "amortized_batch_wall_ms_per_sample": 8117.786544025876, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1573 + } + ] + }, + { + "sample_ids": [ + "0003266" + ], + "batch_size": 1, + "generation_latency_seconds": 8.274491983000189, + "batch_wall_latency_seconds": 8.288345888955519, + "amortized_generation_ms_per_sample": 8274.491983000189, + "amortized_batch_wall_ms_per_sample": 8288.345888955519, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000824" + ], + "batch_size": 1, + "generation_latency_seconds": 11.22929672000464, + "batch_wall_latency_seconds": 11.251729905954562, + "amortized_generation_ms_per_sample": 11229.29672000464, + "amortized_batch_wall_ms_per_sample": 11251.729905954562, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing on the ground within the normal reach of the excavator's boom and bucket, which are positioned close to the worker.\", \"boxes\": [[155, 592, 191, 700]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing on the ground within the normal reach of the excavator's boom and bucket, which are positioned close to the worker.", + "boxes": [ + [ + 155, + 592, + 191, + 700 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 156, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002144" + ], + "batch_size": 1, + "generation_latency_seconds": 10.366267680074088, + "batch_wall_latency_seconds": 10.379591763019562, + "amortized_generation_ms_per_sample": 10366.267680074088, + "amortized_batch_wall_ms_per_sample": 10379.591763019562, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"Worker on the left is clearly bareheaded, missing a hard hat.\", \"boxes\": [[46, 383, 112, 545]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "Worker on the left is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 46, + 383, + 112, + 545 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 141, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001450" + ], + "batch_size": 1, + "generation_latency_seconds": 8.482930272002704, + "batch_wall_latency_seconds": 8.493541297968477, + "amortized_generation_ms_per_sample": 8482.930272002704, + "amortized_batch_wall_ms_per_sample": 8493.541297968477, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0003429" + ], + "batch_size": 1, + "generation_latency_seconds": 10.942747003980912, + "batch_wall_latency_seconds": 11.023728625965305, + "amortized_generation_ms_per_sample": 10942.747003980912, + "amortized_batch_wall_ms_per_sample": 11023.728625965305, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the left is clearly bareheaded, missing a hard hat.\", \"boxes\": [[129, 331, 202, 555]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the left is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 129, + 331, + 202, + 555 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 143, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000205" + ], + "batch_size": 1, + "generation_latency_seconds": 12.731149753089994, + "batch_wall_latency_seconds": 12.753692528931424, + "amortized_generation_ms_per_sample": 12731.149753089994, + "amortized_batch_wall_ms_per_sample": 12753.692528931424, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The circular concrete structure has a deep opening in the center and the worker is standing near the edge without guardrails or fencing, indicating an unprotected edge over a deep drop.\", \"boxes\": [[0, 100, 1000, 1000]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The circular concrete structure has a deep opening in the center and the worker is standing near the edge without guardrails or fencing, indicating an unprotected edge over a deep drop.", + "boxes": [ + [ + 0, + 100, + 1000, + 1000 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 164, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004209" + ], + "batch_size": 1, + "generation_latency_seconds": 8.481906582019292, + "batch_wall_latency_seconds": 8.510338544961996, + "amortized_generation_ms_per_sample": 8481.906582019292, + "amortized_batch_wall_ms_per_sample": 8510.338544961996, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003623" + ], + "batch_size": 1, + "generation_latency_seconds": 11.558857796015218, + "batch_wall_latency_seconds": 11.585797557956539, + "amortized_generation_ms_per_sample": 11558.857796015218, + "amortized_batch_wall_ms_per_sample": 11585.79755795654, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"Worker on the upper level is exposed to a fall hazard without a safety harness or effective edge protection.\", \"boxes\": [[508, 31, 646, 152]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "Worker on the upper level is exposed to a fall hazard without a safety harness or effective edge protection.", + "boxes": [ + [ + 508, + 31, + 646, + 152 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1694 + } + ] + }, + { + "sample_ids": [ + "0002496" + ], + "batch_size": 1, + "generation_latency_seconds": 11.345404008985497, + "batch_wall_latency_seconds": 11.363981842063367, + "amortized_generation_ms_per_sample": 11345.404008985497, + "amortized_batch_wall_ms_per_sample": 11363.981842063367, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation pit lacks guardrails or fencing, exposing workers to a significant fall hazard.\", \"boxes\": [[42, 442, 998, 685]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation pit lacks guardrails or fencing, exposing workers to a significant fall hazard.", + "boxes": [ + [ + 42, + 442, + 998, + 685 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 149, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004049" + ], + "batch_size": 1, + "generation_latency_seconds": 8.281572339008562, + "batch_wall_latency_seconds": 8.321087789023295, + "amortized_generation_ms_per_sample": 8281.572339008562, + "amortized_batch_wall_ms_per_sample": 8321.087789023295, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001483" + ], + "batch_size": 1, + "generation_latency_seconds": 8.35940665891394, + "batch_wall_latency_seconds": 8.414654169930145, + "amortized_generation_ms_per_sample": 8359.40665891394, + "amortized_batch_wall_ms_per_sample": 8414.654169930145, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004197" + ], + "batch_size": 1, + "generation_latency_seconds": 11.471120469039306, + "batch_wall_latency_seconds": 11.495890544028953, + "amortized_generation_ms_per_sample": 11471.120469039306, + "amortized_batch_wall_ms_per_sample": 11495.890544028953, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing on the ground directly in the path of the excavator's bucket and within the machine's active swing zone.\", \"boxes\": [[638, 545, 668, 682]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing on the ground directly in the path of the excavator's bucket and within the machine's active swing zone.", + "boxes": [ + [ + 638, + 545, + 668, + 682 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 154, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003082" + ], + "batch_size": 1, + "generation_latency_seconds": 11.027026457944885, + "batch_wall_latency_seconds": 11.062949666986242, + "amortized_generation_ms_per_sample": 11027.026457944885, + "amortized_batch_wall_ms_per_sample": 11062.949666986242, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing workers to a significant drop.\", \"boxes\": [[0, 0, 670, 640]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing workers to a significant drop.", + "boxes": [ + [ + 0, + 0, + 670, + 640 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000758" + ], + "batch_size": 1, + "generation_latency_seconds": 11.055003611021675, + "batch_wall_latency_seconds": 11.07016703102272, + "amortized_generation_ms_per_sample": 11055.003611021675, + "amortized_batch_wall_ms_per_sample": 11070.16703102272, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing within the normal reach of the excavator's boom and bucket in the active work area.\", \"boxes\": [[636, 330, 725, 582]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing within the normal reach of the excavator's boom and bucket in the active work area.", + "boxes": [ + [ + 636, + 330, + 725, + 582 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 150, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000564" + ], + "batch_size": 1, + "generation_latency_seconds": 13.539180523017421, + "batch_wall_latency_seconds": 13.570679377065971, + "amortized_generation_ms_per_sample": 13539.180523017421, + "amortized_batch_wall_ms_per_sample": 13570.679377065971, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the right is walking on the construction site without a hard hat.\", \"boxes\": [[576, 515, 611, 632]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing very close to the excavator's tracks and rear counterweight, within its normal swing and travel zone.\", \"boxes\": [[472, 466, 508, 603]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the right is walking on the construction site without a hard hat.", + "boxes": [ + [ + 576, + 515, + 611, + 632 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing very close to the excavator's tracks and rear counterweight, within its normal swing and travel zone.", + "boxes": [ + [ + 472, + 466, + 508, + 603 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 187, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001580" + ], + "batch_size": 1, + "generation_latency_seconds": 8.109316511079669, + "batch_wall_latency_seconds": 8.123522659065202, + "amortized_generation_ms_per_sample": 8109.316511079669, + "amortized_batch_wall_ms_per_sample": 8123.522659065202, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1333 + } + ] + }, + { + "sample_ids": [ + "0002136" + ], + "batch_size": 1, + "generation_latency_seconds": 10.921932367025875, + "batch_wall_latency_seconds": 10.956237377016805, + "amortized_generation_ms_per_sample": 10921.932367025875, + "amortized_batch_wall_ms_per_sample": 10956.237377016805, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation is accessible and lacks guardrails or fencing, exposing workers to a significant drop.\", \"boxes\": [[0, 62, 1000, 312]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation is accessible and lacks guardrails or fencing, exposing workers to a significant drop.", + "boxes": [ + [ + 0, + 62, + 1000, + 312 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 149, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000949" + ], + "batch_size": 1, + "generation_latency_seconds": 10.729247406939976, + "batch_wall_latency_seconds": 10.767821826972067, + "amortized_generation_ms_per_sample": 10729.247406939976, + "amortized_batch_wall_ms_per_sample": 10767.821826972067, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"The driver inside the truck cab is clearly visible with a bare head, missing a hard hat.\", \"boxes\": [[378, 208, 408, 258]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "The driver inside the truck cab is clearly visible with a bare head, missing a hard hat.", + "boxes": [ + [ + 378, + 208, + 408, + 258 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003045" + ], + "batch_size": 1, + "generation_latency_seconds": 7.96182514692191, + "batch_wall_latency_seconds": 7.970436848932877, + "amortized_generation_ms_per_sample": 7961.82514692191, + "amortized_batch_wall_ms_per_sample": 7970.436848932877, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1129 + } + ] + }, + { + "sample_ids": [ + "0000013" + ], + "batch_size": 1, + "generation_latency_seconds": 8.21879173698835, + "batch_wall_latency_seconds": 8.24058064003475, + "amortized_generation_ms_per_sample": 8218.79173698835, + "amortized_batch_wall_ms_per_sample": 8240.58064003475, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003889" + ], + "batch_size": 1, + "generation_latency_seconds": 8.273510461091064, + "batch_wall_latency_seconds": 8.289232861017808, + "amortized_generation_ms_per_sample": 8273.510461091064, + "amortized_batch_wall_ms_per_sample": 8289.232861017808, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003261" + ], + "batch_size": 1, + "generation_latency_seconds": 8.405189790995792, + "batch_wall_latency_seconds": 8.4461084890645, + "amortized_generation_ms_per_sample": 8405.189790995792, + "amortized_batch_wall_ms_per_sample": 8446.1084890645, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001629" + ], + "batch_size": 1, + "generation_latency_seconds": 8.233813347993419, + "batch_wall_latency_seconds": 8.26478240697179, + "amortized_generation_ms_per_sample": 8233.813347993419, + "amortized_batch_wall_ms_per_sample": 8264.78240697179, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001569" + ], + "batch_size": 1, + "generation_latency_seconds": 8.24229718092829, + "batch_wall_latency_seconds": 8.27361940103583, + "amortized_generation_ms_per_sample": 8242.29718092829, + "amortized_batch_wall_ms_per_sample": 8273.61940103583, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002536" + ], + "batch_size": 1, + "generation_latency_seconds": 8.316832296084613, + "batch_wall_latency_seconds": 8.332275224965997, + "amortized_generation_ms_per_sample": 8316.832296084613, + "amortized_batch_wall_ms_per_sample": 8332.275224965997, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003053" + ], + "batch_size": 1, + "generation_latency_seconds": 8.387920169043355, + "batch_wall_latency_seconds": 8.417040045023896, + "amortized_generation_ms_per_sample": 8387.920169043355, + "amortized_batch_wall_ms_per_sample": 8417.040045023896, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002661" + ], + "batch_size": 1, + "generation_latency_seconds": 11.386560487910174, + "batch_wall_latency_seconds": 11.400971814990044, + "amortized_generation_ms_per_sample": 11386.560487910174, + "amortized_batch_wall_ms_per_sample": 11400.971814990044, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation trench lacks guardrails or fencing, exposing workers to a significant fall hazard.\", \"boxes\": [[0, 200, 780, 650]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation trench lacks guardrails or fencing, exposing workers to a significant fall hazard.", + "boxes": [ + [ + 0, + 200, + 780, + 650 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 150, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001664" + ], + "batch_size": 1, + "generation_latency_seconds": 8.633664096007124, + "batch_wall_latency_seconds": 8.660372731043026, + "amortized_generation_ms_per_sample": 8633.664096007124, + "amortized_batch_wall_ms_per_sample": 8660.372731043026, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001351" + ], + "batch_size": 1, + "generation_latency_seconds": 12.240043157013133, + "batch_wall_latency_seconds": 12.259637883980758, + "amortized_generation_ms_per_sample": 12240.043157013133, + "amortized_batch_wall_ms_per_sample": 12259.637883980758, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the concrete retaining wall borders a deep excavation area and lacks guardrails or fencing, creating a fall hazard for workers on the upper level.\", \"boxes\": [[0, 500, 998, 580]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the concrete retaining wall borders a deep excavation area and lacks guardrails or fencing, creating a fall hazard for workers on the upper level.", + "boxes": [ + [ + 0, + 500, + 998, + 580 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 159, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0000304" + ], + "batch_size": 1, + "generation_latency_seconds": 8.286614268901758, + "batch_wall_latency_seconds": 8.305405738996342, + "amortized_generation_ms_per_sample": 8286.614268901758, + "amortized_batch_wall_ms_per_sample": 8305.405738996342, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1243 + } + ] + }, + { + "sample_ids": [ + "0004537" + ], + "batch_size": 1, + "generation_latency_seconds": 10.83758004498668, + "batch_wall_latency_seconds": 10.849978967919014, + "amortized_generation_ms_per_sample": 10837.58004498668, + "amortized_batch_wall_ms_per_sample": 10849.978967919014, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"Worker is standing on rebar at a significant height without a safety harness, exposed to a fall.\", \"boxes\": [[538, 292, 706, 568]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "Worker is standing on rebar at a significant height without a safety harness, exposed to a fall.", + "boxes": [ + [ + 538, + 292, + 706, + 568 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0003801" + ], + "batch_size": 1, + "generation_latency_seconds": 8.35220990900416, + "batch_wall_latency_seconds": 8.370414639008231, + "amortized_generation_ms_per_sample": 8352.20990900416, + "amortized_batch_wall_ms_per_sample": 8370.414639008231, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004321" + ], + "batch_size": 1, + "generation_latency_seconds": 11.045618020929396, + "batch_wall_latency_seconds": 11.056004835991189, + "amortized_generation_ms_per_sample": 11045.618020929396, + "amortized_batch_wall_ms_per_sample": 11056.004835991189, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on foot near the concrete pump hose is clearly bareheaded, missing a hard hat.\", \"boxes\": [[451, 686, 468, 758]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on foot near the concrete pump hose is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 451, + 686, + 468, + 758 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1393 + } + ] + }, + { + "sample_ids": [ + "0001878" + ], + "batch_size": 1, + "generation_latency_seconds": 12.032795147038996, + "batch_wall_latency_seconds": 12.044023220078088, + "amortized_generation_ms_per_sample": 12032.795147038996, + "amortized_batch_wall_ms_per_sample": 12044.023220078088, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"A worker is standing on the top of the concrete pump truck chassis, which is elevated above the ground, without any visible safety harness or fall protection equipment.\", \"boxes\": [[483, 526, 518, 625]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "A worker is standing on the top of the concrete pump truck chassis, which is elevated above the ground, without any visible safety harness or fall protection equipment.", + "boxes": [ + [ + 483, + 526, + 518, + 625 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 159, + "prompt_tokens": 1705 + } + ] + } + ], + "mean_generation_ms_per_sample": 9800.482400676628, + "median_generation_ms_per_sample": 8558.297184004914, + "mean_batch_wall_ms_per_sample": 9822.475872695577, + "median_batch_wall_ms_per_sample": 8585.355638002511, + "mean_prompt_tokens": 1624.5625, + "mean_completion_tokens": 130.796875, + "latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.", + "flops_scope": "This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/components/yolo26x_router.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/yolo26x_router.json new file mode 100644 index 0000000000000000000000000000000000000000..cef08acee599ed6dc1e24e3c873932ba815f9e87 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/yolo26x_router.json @@ -0,0 +1,238 @@ +{ + "status": "complete", + "run": "shared_12_yolo26x_Qwen3.8", + "config": { + "fraction": 1.0, + "seed": 42, + "freeze_encoder": true, + "fusion": "concat", + "harm_cost": 1.0, + "batch_size": 256, + "weight_decay": 0.01, + "max_epochs": 50, + "patience": 8, + "cpu_threads": 1, + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90, + "expert": "yolo26x", + "task": "construction", + "endpoint": "Qwen3.8", + "branches": [ + "image", + "hidden", + "confidence", + "output" + ], + "hidden_layer": "feature_blocks__layer_16", + "id": "shared_12_yolo26x_Qwen3.8", + "stage": "shared_mandatory_inputs" + }, + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_shared_image_hidden_20260910/runs/shared_12_yolo26x_Qwen3.8/best.pt", + "score": "logit_rescue-logit_harm", + "test_rows_recomputed": 3004, + "selected_decisions_match_main": true, + "max_score_abs_difference": 0.0, + "actual_call_fraction": 0.4876830892143808, + "sample_count": 64, + "sample_ids": [ + "0002628", + "0003242", + "0001568", + "0001119", + "0003648", + "0000187", + "0001891", + "0001689", + "0003372", + "0002549", + "0000915", + "0003633", + "0004424", + "0001627", + "0001562", + "0002018", + "0000254", + "0002719", + "0002730", + "0004413", + "0000891", + "0000150", + "0003223", + "0004731", + "0001587", + "0000483", + "0001116", + "0003376", + "0001225", + "0003082", + "0004447", + "0000283", + "0000563", + "0001254", + "0004759", + "0001868", + "0001860", + "0004087", + "0002962", + "0001802", + "0002973", + "0002648", + "0004350", + "0004377", + "0004809", + "0003694", + "0001572", + "0001550", + "0000213", + "0002266", + "0002445", + "0004513", + "0002124", + "0004391", + "0003679", + "0002902", + "0002508", + "0004767", + "0003840", + "0003498", + "0001579", + "0001748", + "0004706", + "0004387" + ], + "batch_size": 1, + "latency_ms": [ + 17.35539303626865, + 13.579263002611697, + 15.319477999582887, + 14.879688038490713, + 24.25052202306688, + 20.667891018092632, + 20.015343907289207, + 19.221558002755046, + 20.937222987413406, + 26.26270893961191, + 17.899635946378112, + 16.57471200451255, + 18.844021018594503, + 21.3212959934026, + 19.545868039131165, + 64.57463896367699, + 20.962884067557752, + 24.004184058867395, + 20.19065897911787, + 18.50342599209398, + 19.576229038648307, + 25.253725005313754, + 21.430211956612766, + 25.169604923576117, + 17.8982320940122, + 21.415626979433, + 22.9903319850564, + 229.59673800505698, + 19.89229000173509, + 18.98771501146257, + 18.157949903979897, + 22.423187969252467, + 19.864327972754836, + 21.502194926142693, + 18.544791964814067, + 50.5554600385949, + 20.046600024215877, + 16.299219918437302, + 18.668118049390614, + 24.017272982746363, + 24.73958197515458, + 18.68593308608979, + 20.250934991054237, + 23.676551994867623, + 23.003229987807572, + 24.6098650386557, + 21.308812079951167, + 17.95686304103583, + 18.23423698078841, + 14.343310962431133, + 25.8766250917688, + 18.20633898023516, + 30.77617404051125, + 19.3530690157786, + 15.836970997042954, + 19.045730936340988, + 16.162591986358166, + 25.458221905864775, + 18.277363968081772, + 52.543785073794425, + 22.9536690749228, + 17.81571004539728, + 18.550690030679107, + 21.854626946151257 + ], + "mean_latency_ms_per_input": 25.261267359383055, + "head_mean_latency_ms": 0.486134999897331, + "gflops_per_input": [ + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752 + ], + "mean_gflops_per_input": 3.183754752, + "operators": [ + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + } + ], + "scope": "image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/components/yolo26x_specialist.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/yolo26x_specialist.json new file mode 100644 index 0000000000000000000000000000000000000000..251decaacc2a6eb1affbe81ae0c03c9f13b031aa --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/components/yolo26x_specialist.json @@ -0,0 +1,254 @@ +{ + "component": "specialist", + "expert": "yolo26x", + "task": "construction", + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/models/construction/yolo26x.pt", + "sample_count": 64, + "batch_size": 1, + "sample_ids": [ + "0000944", + "0003511", + "0003365", + "0001351", + "0002149", + "0003368", + "0000176", + "0004487", + "0000013", + "0003583", + "0000758", + "0001291", + "0001451", + "0002755", + "0003088", + "0004538", + "0004324", + "0000516", + "0003893", + "0003475", + "0002137", + "0003272", + "0001711", + "0003262", + "0001470", + "0002145", + "0002662", + "0000756", + "0001631", + "0001924", + "0002496", + "0001443", + "0004053", + "0002168", + "0001881", + "0001666", + "0000304", + "0001252", + "0004844", + "0000949", + "0003430", + "0003807", + "0004199", + "0002538", + "0002091", + "0001569", + "0003048", + "0002854", + "0000380", + "0001234", + "0001484", + "0003180", + "0000824", + "0003784", + "0000894", + "0004168", + "0000564", + "0004296", + "0003054", + "0001583", + "0003386", + "0000205", + "0004214", + "0003625" + ], + "model_load_seconds_excluded_from_latency": true, + "forward_latency_ms_per_sample": [ + 41.741790771484375, + 51.50377655029297, + 78.82457733154297, + 87.29910278320312, + 49.47385787963867, + 49.40816116333008, + 168.38259887695312, + 42.57542419433594, + 40.342529296875, + 54.95475387573242, + 48.86323165893555, + 40.684993743896484, + 282.5484619140625, + 47.938270568847656, + 45.019649505615234, + 268.04327392578125, + 51.39228820800781, + 41.36457443237305, + 54.01919937133789, + 43.2599983215332, + 47.13923263549805, + 41.1484489440918, + 35.99542236328125, + 80.46995544433594, + 83.43251037597656, + 48.88550567626953, + 171.4989776611328, + 58.917728424072266, + 46.90486526489258, + 56.416385650634766, + 44.49577713012695, + 49.42009735107422, + 43.6354866027832, + 55.22761535644531, + 48.02000045776367, + 39.44412612915039, + 30.26905632019043, + 492.74456787109375, + 45.74156951904297, + 39.223392486572266, + 50.18822479248047, + 43.890113830566406, + 51.966495513916016, + 47.814815521240234, + 54.13961410522461, + 39.215328216552734, + 56.004703521728516, + 39.528831481933594, + 396.5307312011719, + 83.4832992553711, + 50.0503044128418, + 153.1714630126953, + 61.07219314575195, + 52.57244873046875, + 42.75983810424805, + 38.4617919921875, + 44.08700942993164, + 53.36006546020508, + 45.519649505615234, + 290.8823547363281, + 48.10774230957031, + 53.34476852416992, + 51.016319274902344, + 42.9719352722168 + ], + "end_to_end_latency_ms_per_sample": [ + 35.17372906208038, + 48.07086195796728, + 46.07532289810479, + 45.34715902991593, + 46.541396994143724, + 45.594630064442754, + 45.38595397025347, + 39.57494895439595, + 39.812199072912335, + 51.36631103232503, + 42.81833197455853, + 33.49219704978168, + 37.3103239107877, + 38.96563302259892, + 40.327758993953466, + 44.32409000582993, + 43.25311293359846, + 33.99710892699659, + 49.12354901898652, + 45.043769059702754, + 47.554407035931945, + 37.903546006418765, + 32.56980108562857, + 35.192839917726815, + 39.34567596297711, + 46.67255701497197, + 49.70169102307409, + 52.48237494379282, + 43.415413005277514, + 48.36420400533825, + 48.96194895263761, + 51.6092429170385, + 37.311508087441325, + 50.858690054155886, + 43.94579504150897, + 36.09434806276113, + 26.66287892498076, + 41.9749760767445, + 34.900374012067914, + 39.04577391222119, + 51.60050210542977, + 46.00996105000377, + 53.40714694466442, + 51.78673507180065, + 54.96632098220289, + 46.47863702848554, + 47.48446703888476, + 36.593522992916405, + 37.32479899190366, + 26.43477905075997, + 39.97986298054457, + 55.87899999227375, + 40.821335976943374, + 45.88699492160231, + 40.707007981836796, + 34.627774031832814, + 43.4188200160861, + 49.56604307517409, + 39.82124803587794, + 249.81214595027268, + 38.2277499884367, + 48.29735495150089, + 47.6501100929454, + 40.76927900314331 + ], + "mean_forward_latency_ms_per_sample": 78.70017620921135, + "median_forward_latency_ms_per_sample": 49.41412925720215, + "mean_end_to_end_latency_ms_per_sample": 46.30809425361804, + "median_end_to_end_latency_ms_per_sample": 43.68230752879754, + "flop_profiles": [ + { + "batch_size": 1, + "flops_counted": 116518072320, + "gflops_counted_per_sample": 116.51807232, + "operators": { + "aten.convolution": 116319006720, + "aten.bmm": 199065600 + } + }, + { + "batch_size": 1, + "flops_counted": 145709798400, + "gflops_counted_per_sample": 145.7097984, + "operators": { + "aten.convolution": 145398758400, + "aten.bmm": 311040000 + } + }, + { + "batch_size": 1, + "flops_counted": 145709798400, + "gflops_counted_per_sample": 145.7097984, + "operators": { + "aten.convolution": 145398758400, + "aten.bmm": 311040000 + } + }, + { + "batch_size": 1, + "flops_counted": 135976458240, + "gflops_counted_per_sample": 135.97645824, + "operators": { + "aten.convolution": 135705507840, + "aten.bmm": 270950400 + } + } + ], + "mean_counted_gflops_per_sample": 135.97853184000002, + "flops_scope": "GPU FlopCounterMode around the real Ultralytics predict path; multiply-add counted as 2 FLOPs. Elementwise, postprocessing and unsupported custom ops may be missing.", + "latency_scope": "A800 wall-clock latency for real Ultralytics prediction including image loading/preprocess, forward and result postprocessing, excluding model load; batch processing uses the recorded batch size.", + "completed": true +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact.pdf b/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact.pdf new file mode 100644 index 0000000000000000000000000000000000000000..d72c2188e497b9dcdc6951b6d236413d7b00087e Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact.pdf differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact.png b/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact.png new file mode 100644 index 0000000000000000000000000000000000000000..a233dc81de8c5347c013da8b3915cc9ba8a56a86 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3e5c4bfdf5310e20a021a38d0faf8c2394864d2f65b05b76736154cf041dfc4 +size 120520 diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact_verified.png b/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact_verified.png new file mode 100644 index 0000000000000000000000000000000000000000..e5a557bffc4f771e07f14a14bdebfdb1314fd866 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_compact_verified.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_summary.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..3850a0b23a86eea72e94b84fe2f5abdffdbe373d --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/cost_summary.json @@ -0,0 +1,75 @@ +{ + "components": { + "grefcoco|instancevg|MiniCPM": { + "specialist_gflops": 7.502680064, + "router_gflops": 3.642171392, + "large_gflops": 12916.147709832, + "specialist_ms": 46.08042127620138, + "router_ms": 13.180577338061994, + "large_ms": 3667.173011115665, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + }, + "construction|yolo26x|Qwen3.8": { + "specialist_gflops": 135.97853184000002, + "router_gflops": 3.183754752, + "large_gflops": 96874.712551641, + "specialist_ms": 46.30809425361804, + "router_ms": 25.261267359383055, + "large_ms": 9822.475872695577, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + } + }, + "selected_points": [ + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "learned", + "nominal_validation_budget": 0.23, + "actual_test_call_fraction": 0.30904914499617164, + "test_metric": 0.6671865694921022, + "estimated_gflops_per_input": 4002.86925782384, + "estimated_time_ms_per_input": 1192.5976822525959 + }, + { + "pair": "grefcoco|instancevg|MiniCPM", + "method": "confidence", + "nominal_validation_budget": 0.65, + "actual_test_call_fraction": 0.5209426310864078, + "test_metric": 0.6187788900547316, + "estimated_gflops_per_input": 6736.074651524562, + "estimated_time_ms_per_input": 1956.4671783358606 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "nominal_validation_budget": 0.5, + "actual_test_call_fraction": 0.4876830892143808, + "test_metric": 0.29875800068449215, + "estimated_gflops_per_input": 47383.321370531434, + "estimated_time_ms_per_input": 4861.824738942901 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "nominal_validation_budget": 0.93, + "actual_test_call_fraction": 0.9221038615179761, + "test_metric": 0.2597490004762105, + "estimated_gflops_per_input": 89464.5250591521, + "estimated_time_ms_per_input": 9103.651026133362 + } + ], + "protocol": { + "method": "original main-table logit contrast; original checkpoints and validation thresholds; seed 42", + "large_timing": "64 measured examples; batch size one; same full task prompt; two separate warmup examples", + "cost": "serial component estimates, not a timed integrated service; includes active distilled encoders and routing head only for learned", + "flops": "multiply-add=2; supported operators; mean prefill (including visual embedding) plus average generated length times one-token decode probe", + "scope_limits": "mean call cost extrapolated across policies; no policy-specific input/output-length costs; specialist-feature extraction overhead is not separately measured; confidence threshold overhead not separately timed" + } +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/fixed_budget_cost_performance.csv b/releases/20260913_lr/paper_data/main_table_cost_20260912/fixed_budget_cost_performance.csv new file mode 100644 index 0000000000000000000000000000000000000000..73e6759efc8f937df9e85647bc7b74b3c9dcb374 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/fixed_budget_cost_performance.csv @@ -0,0 +1,29 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|MiniCPM,learned,0.0,0.0,0.6695119530385957,11.144851456,59.26099861426337 +grefcoco|instancevg|MiniCPM,learned,0.05,0.09125712503190313,0.6796075206306894,1189.835357942668,393.91666460326627 +grefcoco|instancevg|MiniCPM,learned,0.1,0.14868275529591923,0.681535887474123,1931.5532807628988,604.5063860537731 +grefcoco|instancevg|MiniCPM,learned,0.25,0.3278223633837166,0.6640104358676233,4245.346919506304,1261.4423220551812 +grefcoco|instancevg|MiniCPM,learned,0.5,0.5738876442730341,0.6134475228993562,7423.56243373403,2163.806279105081 +grefcoco|instancevg|MiniCPM,learned,0.75,0.7862348637381958,0.5578651844709752,10166.270486118172,2942.5202713131775 +grefcoco|instancevg|MiniCPM,learned,1.0,1.0,0.5000708958398321,12927.292561287999,3726.434009729928 +grefcoco|instancevg|MiniCPM,confidence,0.0,0.0,0.6695119530385957,7.502680064,46.08042127620138 +grefcoco|instancevg|MiniCPM,confidence,0.05,0.012534384482318578,0.6662791027422511,169.39864148945287,92.04617776070705 +grefcoco|instancevg|MiniCPM,confidence,0.1,0.0345971698380739,0.6645208859144145,454.3648360347069,172.9542287673709 +grefcoco|instancevg|MiniCPM,confidence,0.25,0.12477667810452883,0.6596999688058305,1619.1366852042547,503.6580876377964 +grefcoco|instancevg|MiniCPM,confidence,0.5,0.2345234381646485,0.6527521765022829,3036.6420488162507,906.1184441876538 +grefcoco|instancevg|MiniCPM,confidence,0.75,0.7478093185491875,0.567762243711539,9666.318297234116,2788.426571720579 +grefcoco|instancevg|MiniCPM,confidence,1.0,1.0,0.5000708958398321,12923.650389896,3713.253432391866 +construction|yolo26x|Qwen3.8,learned,0.0,0.0,0.18129624718318377,139.16228659200002,71.56936161300109 +construction|yolo26x|Qwen3.8,learned,0.05,0.045938748335552594,0.19531128485766117,4589.465326580834,522.8016087608005 +construction|yolo26x|Qwen3.8,learned,0.1,0.09920106524633822,0.20211074868345033,9749.236967147599,1045.9694315408578 +construction|yolo26x|Qwen3.8,learned,0.25,0.25432756324900135,0.2504846177578749,24777.071870298303,2569.695715387775 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,learned,0.75,0.7709720372836218,0.28730364578560735,74826.85678379593,7644.4235963543315 +construction|yolo26x|Qwen3.8,learned,1.0,1.0,0.27001009067886245,97013.87483823301,9894.045234308578 +construction|yolo26x|Qwen3.8,confidence,0.0,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.05,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.1,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.25,0.24800266311584554,0.18026802885144627,24161.165233228996,2482.3082690732604 +construction|yolo26x|Qwen3.8,confidence,0.5,0.5013315579227696,0.18015785955296285,48702.32909867467,4970.625226170908 +construction|yolo26x|Qwen3.8,confidence,0.75,0.7486684420772304,0.22103445955201576,72663.01865455658,7400.0858032057995 +construction|yolo26x|Qwen3.8,confidence,1.0,1.0,0.27001009067886245,97010.691083481,9868.783966949195 diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance.pdf b/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance.pdf new file mode 100644 index 0000000000000000000000000000000000000000..3ef904fe3a999380f46e7be53b9c73ad23b1c75a Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance.pdf differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance.png b/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance.png new file mode 100644 index 0000000000000000000000000000000000000000..7f8b55d487e12c8652346e487744fd1818546c34 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance_verified.png b/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance_verified.png new file mode 100644 index 0000000000000000000000000000000000000000..1e164638ab8ceb9ba90f92442f99bd6ac160aff5 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_20260912/flops_performance_verified.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/full_cost_performance_curves.csv b/releases/20260913_lr/paper_data/main_table_cost_20260912/full_cost_performance_curves.csv new file mode 100644 index 0000000000000000000000000000000000000000..0964f7686d09a37d8665a51e314684b9bc23ab19 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/full_cost_performance_curves.csv @@ -0,0 +1,405 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|MiniCPM,learned,0.0,0.0,0.6695119530385957,11.144851456,59.26099861426337 +grefcoco|instancevg|MiniCPM,learned,0.01,0.04055242038397187,0.67373734509259,534.9259031265826,207.97374018178175 +grefcoco|instancevg|MiniCPM,learned,0.02,0.05498681337379122,0.6756089952641579,721.362655084853,260.9071565858844 +grefcoco|instancevg|MiniCPM,learned,0.03,0.0680600062388339,0.6773104954201288,890.2179451688661,308.8488166296788 +grefcoco|instancevg|MiniCPM,learned,0.04,0.08110484076794373,0.6788134872245697,1058.7069547971653,356.6864817493001 +grefcoco|instancevg|MiniCPM,learned,0.05,0.09125712503190313,0.6796075206306894,1189.835357942668,393.91666460326627 +grefcoco|instancevg|MiniCPM,learned,0.06,0.10106910926466835,0.6802597623571449,1316.5683956196062,429.89890836715534 +grefcoco|instancevg|MiniCPM,learned,0.07,0.11527663556702493,0.6812523041147945,1500.074903932167,482.0003653778733 +grefcoco|instancevg|MiniCPM,learned,0.08,0.12605280322150697,0.681167229106996,1639.261477103371,521.5184365636475 +grefcoco|instancevg|MiniCPM,learned,0.09,0.13674389586819047,0.6818194708334515,1777.3492089074336,560.7245229769023 +grefcoco|instancevg|MiniCPM,learned,0.1,0.14868275529591923,0.681535887474123,1931.5532807628988,604.5063860537731 +grefcoco|instancevg|MiniCPM,learned,0.11,0.1613305731219692,0.6816493208178545,2094.9143640112065,650.8881222349711 +grefcoco|instancevg|MiniCPM,learned,0.12,0.17383659926835493,0.6807985707398689,2256.444044980946,696.7498837953037 +grefcoco|instancevg|MiniCPM,learned,0.13,0.18634262541474067,0.6802881206930778,2417.973725950685,742.6116453556364 +grefcoco|instancevg|MiniCPM,learned,0.14,0.19865014320959645,0.6797209539744208,2576.939443730428,787.7454424467572 +grefcoco|instancevg|MiniCPM,learned,0.15,0.21149646938717637,0.6788418455605025,2742.8644901687317,834.855143097167 +grefcoco|instancevg|MiniCPM,learned,0.16,0.2222726370416584,0.6784448288574426,2882.0510633399354,874.373214282941 +grefcoco|instancevg|MiniCPM,learned,0.17,0.23472194651617842,0.6770836287326659,3042.8481833982473,920.0269859949274 +grefcoco|instancevg|MiniCPM,learned,0.18,0.24966678955278904,0.6750701868814338,3235.877983559364,974.8323110341457 +grefcoco|instancevg|MiniCPM,learned,0.19,0.2611235572696594,0.6740492867878513,3383.8552876676963,1016.846260400074 +grefcoco|instancevg|MiniCPM,learned,0.2,0.27226838329126846,0.6728298783427389,3527.803506763178,1057.7162656000983 +grefcoco|instancevg|MiniCPM,learned,0.21,0.28545500950004254,0.6707030031477753,3698.1239186700464,1106.073905340585 +grefcoco|instancevg|MiniCPM,learned,0.22,0.29620281881859173,0.6688313529762073,3836.944211385536,1145.4879816021862 +grefcoco|instancevg|MiniCPM,learned,0.23,0.30904914499617164,0.6671865694921022,4002.86925782384,1192.5976822525959 +grefcoco|instancevg|MiniCPM,learned,0.24,0.31880441255707115,0.6657119360235941,4128.86973458935,1228.3719361681387 +grefcoco|instancevg|MiniCPM,learned,0.25,0.3278223633837166,0.6640104358676233,4245.346919506304,1261.4423220551812 +grefcoco|instancevg|MiniCPM,learned,0.26,0.3389388310693928,0.6622805773757195,4388.928858146072,1302.2083323310324 +grefcoco|instancevg|MiniCPM,learned,0.27,0.35082097382525596,0.6600402688370246,4542.400369090111,1345.782205559557 +grefcoco|instancevg|MiniCPM,learned,0.28,0.36142699146414087,0.658111901993591,4679.389259527033,1384.6763072002925 +grefcoco|instancevg|MiniCPM,learned,0.29,0.37316734254033973,0.6560133851345603,4831.029368192502,1427.7302058079517 +grefcoco|instancevg|MiniCPM,learned,0.3,0.38351813515582905,0.6541417349629923,4964.7217345280005,1465.6883531311296 +grefcoco|instancevg|MiniCPM,learned,0.31,0.39270623599807164,0.6522417264554916,5083.396602179238,1499.3827085632106 +grefcoco|instancevg|MiniCPM,learned,0.32,0.4026032952386354,0.6502282846042594,5211.2284812233165,1535.676937099619 +grefcoco|instancevg|MiniCPM,learned,0.33,0.4103167626123699,0.6486685761279528,5310.856765177441,1563.9635564746993 +grefcoco|instancevg|MiniCPM,learned,0.34,0.4194481467827468,0.6470805093157134,5428.799071917251,1597.4499220584341 +grefcoco|instancevg|MiniCPM,learned,0.35,0.4308198394918186,0.6448685591129513,5575.677534658443,1639.1518866518434 +grefcoco|instancevg|MiniCPM,learned,0.36,0.4392139069279415,0.6431387006210476,5684.096549549696,1669.934384207078 +grefcoco|instancevg|MiniCPM,learned,0.37,0.4474661826844001,0.6415222754728752,5790.684162162381,1700.196907141447 +grefcoco|instancevg|MiniCPM,learned,0.38,0.46124833394776393,0.6382043501687321,5968.696463639236,1750.7384402895686 +grefcoco|instancevg|MiniCPM,learned,0.39,0.46918866800896125,0.6371550917392167,6071.2549912390705,1779.8570190580338 +grefcoco|instancevg|MiniCPM,learned,0.4,0.47786631880441255,0.6349714998723874,6183.336810687461,1811.679465854999 +grefcoco|instancevg|MiniCPM,learned,0.41,0.48657232793579674,0.6325043246462297,6295.784910591566,1843.605907576138 +grefcoco|instancevg|MiniCPM,learned,0.42,0.4953634120749794,0.6308878994980575,6409.33185186281,1875.8443340697954 +grefcoco|instancevg|MiniCPM,learned,0.43,0.5059694297138644,0.6287610243030939,6546.320742299732,1914.7384357105311 +grefcoco|instancevg|MiniCPM,learned,0.44,0.51439185548592,0.6270311658111902,6655.1060376466985,1945.6249281899386 +grefcoco|instancevg|MiniCPM,learned,0.45,0.524629214757678,0.6243087655616368,6787.333282159342,1983.167095816424 +grefcoco|instancevg|MiniCPM,learned,0.46,0.5349232907013016,0.6222669653744718,6920.293087583414,2020.9172532912557 +grefcoco|instancevg|MiniCPM,learned,0.47,0.5420412330204464,0.620934123585628,7012.229481967552,2047.0199792587014 +grefcoco|instancevg|MiniCPM,learned,0.48,0.5530159090264584,0.6183818733516717,7153.980018328752,2087.2660149136873 +grefcoco|instancevg|MiniCPM,learned,0.49,0.5640189433684032,0.6162549981567081,7296.096835145665,2127.6160454928463 +grefcoco|instancevg|MiniCPM,learned,0.5,0.5738876442730341,0.6134475228993562,7423.56243373403,2163.806279105081 +grefcoco|instancevg|MiniCPM,learned,0.51,0.5832458951308738,0.6109519893372657,7544.434984119549,2198.124604082201 +grefcoco|instancevg|MiniCPM,learned,0.52,0.5937101210900945,0.6086549641267051,7679.592472277903,2236.498731102071 +grefcoco|instancevg|MiniCPM,learned,0.53,0.6031250886197997,0.6062728639083459,7801.197583574848,2271.0250459275367 +grefcoco|instancevg|MiniCPM,learned,0.54,0.611263931032527,0.6044579304086436,7906.320074364678,2300.871589165213 +grefcoco|instancevg|MiniCPM,learned,0.55,0.6218983070073448,0.6018489635028217,8043.675245257315,2339.8696857301225 +grefcoco|instancevg|MiniCPM,learned,0.56,0.6296117743810793,0.5999489549953209,8143.303529211438,2368.1563051052026 +grefcoco|instancevg|MiniCPM,learned,0.57,0.6381192751609336,0.5979638714800215,8253.187665925547,2399.35478235713 +grefcoco|instancevg|MiniCPM,learned,0.58,0.6469387176360492,0.5955534129257295,8367.100887652507,2431.6972037749606 +grefcoco|instancevg|MiniCPM,learned,0.59,0.654935768369112,0.5933414627229674,8470.391976163766,2461.0237723917717 +grefcoco|instancevg|MiniCPM,learned,0.6,0.6636701358364291,0.5907608541530783,8583.206356523586,2493.054209037083 +grefcoco|instancevg|MiniCPM,learned,0.61,0.6710149448430366,0.5888324873096447,8678.072994553431,2519.9888943979136 +grefcoco|instancevg|MiniCPM,learned,0.62,0.6805433457164733,0.5859966537163599,8801.143227673234,2554.9311889200717 +grefcoco|instancevg|MiniCPM,learned,0.63,0.6900150299180444,0.5836712701698664,8923.480899881608,2589.665493593884 +grefcoco|instancevg|MiniCPM,learned,0.64,0.6979837223151746,0.5813458866233729,9026.405707937156,2618.888067286522 +grefcoco|instancevg|MiniCPM,learned,0.65,0.7061225647279018,0.579105578084678,9131.528198726986,2648.734610524199 +grefcoco|instancevg|MiniCPM,learned,0.66,0.7130703570314494,0.5778578113036327,9221.26691037684,2674.2133669466057 +grefcoco|instancevg|MiniCPM,learned,0.67,0.7218614411706321,0.5755891444290049,9334.813851648087,2706.451793440264 +grefcoco|instancevg|MiniCPM,learned,0.68,0.7307092419816805,0.5725548024841902,9449.093353830758,2738.8982097822677 +grefcoco|instancevg|MiniCPM,learned,0.69,0.7391033094178033,0.5702861356095624,9557.512368722011,2769.680707337502 +grefcoco|instancevg|MiniCPM,learned,0.7,0.7464481184244108,0.5685279187817259,9652.379006751858,2796.6153926983325 +grefcoco|instancevg|MiniCPM,learned,0.71,0.7537078524232198,0.5663726852508295,9746.146803414566,2823.2380932866436 +grefcoco|instancevg|MiniCPM,learned,0.72,0.7623855032186712,0.5639622266965374,9858.228622862955,2855.060540083609 +grefcoco|instancevg|MiniCPM,learned,0.73,0.7701556872642713,0.5618637098375068,9958.589467728507,2883.5551493070357 +grefcoco|instancevg|MiniCPM,learned,0.74,0.7780393046536029,0.5599069846581403,10060.415434416915,2912.4657382271544 +grefcoco|instancevg|MiniCPM,learned,0.75,0.7862348637381958,0.5578651844709752,10166.270486118172,2942.5202713131775 +grefcoco|instancevg|MiniCPM,learned,0.76,0.7948557978617815,0.5555965175963474,10277.619744655136,2974.134728261797 +grefcoco|instancevg|MiniCPM,learned,0.77,0.8029662819385759,0.5536681507529139,10382.375954989253,3003.8772765753006 +grefcoco|instancevg|MiniCPM,learned,0.78,0.8114737827184301,0.5513144088704874,10492.26009170336,3035.0757538272273 +grefcoco|instancevg|MiniCPM,learned,0.79,0.8210872585996654,0.5488188753083969,10616.429166190303,3070.3300331219048 +grefcoco|instancevg|MiniCPM,learned,0.8,0.8279499758954144,0.5472591668320903,10705.069036473016,3095.4968047717925 +grefcoco|instancevg|MiniCPM,learned,0.81,0.8352664265660891,0.5455293083401866,10799.569394047148,3122.3274952084494 +grefcoco|instancevg|MiniCPM,learned,0.82,0.8424978022289652,0.5440546748716786,10892.970910254142,3148.8462008725874 +grefcoco|instancevg|MiniCPM,learned,0.83,0.850891869665088,0.5418143663329836,11001.389925145395,3179.628698427822 +grefcoco|instancevg|MiniCPM,learned,0.84,0.8593710121090095,0.53948898278649,11110.90778140379,3210.723180755576 +grefcoco|instancevg|MiniCPM,learned,0.85,0.8669994044749454,0.5374188242633923,11209.437223990773,3238.697815358137 +grefcoco|instancevg|MiniCPM,learned,0.86,0.877293480418569,0.5350934407168987,11342.397029414844,3276.447972832969 +grefcoco|instancevg|MiniCPM,learned,0.87,0.8848651561126393,0.5330799988656666,11440.1939110904,3304.2146175871835 +grefcoco|instancevg|MiniCPM,learned,0.88,0.893089073533165,0.5305561069676431,11546.415243247371,3334.37314559738 +grefcoco|instancevg|MiniCPM,learned,0.89,0.901653290984885,0.5283725151008138,11657.031940872906,3365.7796126976527 +grefcoco|instancevg|MiniCPM,learned,0.9,0.9106996001474633,0.5261889232339846,11773.875406245576,3398.9539935088683 +grefcoco|instancevg|MiniCPM,learned,0.91,0.9177324674588095,0.5241754813827525,11864.712959262572,3424.744734703795 +grefcoco|instancevg|MiniCPM,learned,0.92,0.9268638516291864,0.5215948728128633,11982.65526600238,3458.2311002875294 +grefcoco|instancevg|MiniCPM,learned,0.93,0.9340668689561297,0.5198650143209597,12075.690501753661,3484.6458110274943 +grefcoco|instancevg|MiniCPM,learned,0.94,0.9422907863766554,0.5175963474463319,12181.911833910632,3514.8043390376906 +grefcoco|instancevg|MiniCPM,learned,0.95,0.9508266454924426,0.5152426055639056,12292.162251080454,3546.106811213791 +grefcoco|instancevg|MiniCPM,learned,0.96,0.9601281796784165,0.5127187136658821,12412.302240554545,3580.217146342564 +grefcoco|instancevg|MiniCPM,learned,0.97,0.9683237387630094,0.509939596744463,12518.157292255803,3610.271679428587 +grefcoco|instancevg|MiniCPM,learned,0.98,0.976944672886595,0.506791821455917,12629.506550792767,3641.886136377206 +grefcoco|instancevg|MiniCPM,learned,0.99,0.9883163655956668,0.5033604628080425,12776.385013533956,3683.588100970615 +grefcoco|instancevg|MiniCPM,learned,1.0,1.0,0.5000708958398321,12927.292561287999,3726.434009729928 +grefcoco|instancevg|MiniCPM,confidence,0.0,0.0,0.6695119530385957,7.502680064,46.08042127620138 +grefcoco|instancevg|MiniCPM,confidence,0.01,0.002467175226157729,0.6686044862887446,39.3690797110914,55.12797967926019 +grefcoco|instancevg|MiniCPM,confidence,0.02,0.004849275444516916,0.6678388112185577,70.13663799104171,63.86355330979973 +grefcoco|instancevg|MiniCPM,confidence,0.03,0.007600034030003119,0.667214927828035,105.66584219527005,73.95106095458944 +grefcoco|instancevg|MiniCPM,confidence,0.04,0.009811984232765221,0.6666761194453109,134.23571774093818,82.06266504009045 +grefcoco|instancevg|MiniCPM,confidence,0.05,0.012534384482318578,0.6662791027422511,169.39864148945287,92.04617776070705 +grefcoco|instancevg|MiniCPM,confidence,0.06,0.01607917647392451,0.6658537277032583,215.18369845366465,105.04554328234329 +grefcoco|instancevg|MiniCPM,confidence,0.07,0.019709043473328986,0.6654850693361314,262.0675967850175,118.35689357649878 +grefcoco|instancevg|MiniCPM,confidence,0.08,0.02353741882426339,0.6652865609846014,311.5154583063662,132.3962083398659 +grefcoco|instancevg|MiniCPM,confidence,0.09,0.028868785979638715,0.6648611859456087,380.3761839805407,151.94725408440678 +grefcoco|instancevg|MiniCPM,confidence,0.1,0.0345971698380739,0.6645208859144145,454.3648360347069,172.9542287673709 +grefcoco|instancevg|MiniCPM,confidence,0.11,0.03981510364971783,0.6642089442191532,521.7604398860266,192.08929481521943 +grefcoco|instancevg|MiniCPM,confidence,0.12,0.045855429203414344,0.6640387942035562,599.7781769530435,214.24021366408755 +grefcoco|instancevg|MiniCPM,confidence,0.13,0.05317187987408899,0.6637268525082948,694.2785345271766,241.0709041007447 +grefcoco|instancevg|MiniCPM,confidence,0.14,0.05960922213084536,0.6633581941411678,777.4241979741852,264.6777518880361 +grefcoco|instancevg|MiniCPM,confidence,0.15,0.0675211978561098,0.6630178941099736,879.6164451183058,293.69233573232816 +grefcoco|instancevg|MiniCPM,confidence,0.16,0.07551824858917279,0.6626492357428466,982.9075336295676,323.01890434913946 +grefcoco|instancevg|MiniCPM,confidence,0.17,0.08212574086152624,0.6621671440319882,1068.2508798108586,347.24972168146934 +grefcoco|instancevg|MiniCPM,confidence,0.18,0.08856308311828262,0.6619969940163911,1151.396543257867,370.85656946876077 +grefcoco|instancevg|MiniCPM,confidence,0.19,0.09480191702350906,0.6615716189773984,1231.9782435148798,393.7354527868405 +grefcoco|instancevg|MiniCPM,confidence,0.2,0.10007656750701868,0.6612029606102714,1300.1064082776268,413.0785086830352 +grefcoco|instancevg|MiniCPM,confidence,0.21,0.10554972634205825,0.6608910189150101,1370.79853623037,433.14952904844154 +grefcoco|instancevg|MiniCPM,confidence,0.22,0.11005870175538099,0.6605790772197487,1429.0371286888474,449.68472199196276 +grefcoco|instancevg|MiniCPM,confidence,0.23,0.11527663556702493,0.6602671355244875,1496.432732540167,468.8197880398113 +grefcoco|instancevg|MiniCPM,confidence,0.24,0.12029606102713893,0.6599835521651589,1561.264373201491,487.2268896184482 +grefcoco|instancevg|MiniCPM,confidence,0.25,0.12477667810452883,0.6596999688058305,1619.1366852042547,503.6580876377964 +grefcoco|instancevg|MiniCPM,confidence,0.26,0.12914386183818732,0.659217877094972,1675.5438753841634,519.6733059604521 +grefcoco|instancevg|MiniCPM,confidence,0.27,0.1322916371267334,0.6587925020559794,1716.2010059683835,531.2167425436651 +grefcoco|instancevg|MiniCPM,confidence,0.28,0.13626180415733205,0.6582253353373224,1767.4802697683008,545.7760319278977 +grefcoco|instancevg|MiniCPM,confidence,0.29,0.14023197118793068,0.6578283186342625,1818.7595335682179,560.3353213121303 +grefcoco|instancevg|MiniCPM,confidence,0.3,0.14326631313274538,0.6573178685874713,1857.9515423295834,571.462778198651 +grefcoco|instancevg|MiniCPM,confidence,0.31,0.14766185520233674,0.6569208518844114,1914.725012965206,587.5819914454798 +grefcoco|instancevg|MiniCPM,confidence,0.32,0.15066783881121856,0.6565805518532173,1953.5507412708575,598.6054534078274 +grefcoco|instancevg|MiniCPM,confidence,0.33,0.15452457249808582,0.6561268184782917,2003.36488324792,612.7487630953676 +grefcoco|instancevg|MiniCPM,confidence,0.34,0.15806936448969175,0.6558715934548961,2049.1499402121317,625.7481286170038 +grefcoco|instancevg|MiniCPM,confidence,0.35,0.16201117318435754,0.6555596517596347,2100.0629235563347,640.2034230770632 +grefcoco|instancevg|MiniCPM,confidence,0.36,0.1657544735274934,0.6551626350565749,2148.4119437105423,653.930753067911 +grefcoco|instancevg|MiniCPM,confidence,0.37,0.17012165726115192,0.6545954683379179,2204.8191338904517,669.9459713905669 +grefcoco|instancevg|MiniCPM,confidence,0.38,0.17536794940872868,0.6540850182911266,2272.581018197485,689.1850323625886 +grefcoco|instancevg|MiniCPM,confidence,0.39,0.17965005813458867,0.6534044182287383,2327.889367010253,704.8882659127252 +grefcoco|instancevg|MiniCPM,confidence,0.4,0.18401724186824717,0.653092476533477,2384.2965571901614,720.9034842353809 +grefcoco|instancevg|MiniCPM,confidence,0.41,0.18858293395343562,0.6527521765022829,2443.2677105600665,737.6466670272484 +grefcoco|instancevg|MiniCPM,confidence,0.42,0.19238295096843716,0.6523835181351558,2492.3492916257014,751.5819868664424 +grefcoco|instancevg|MiniCPM,confidence,0.43,0.19689192638175992,0.6517596347446332,2550.5878840841788,768.1171798099638 +grefcoco|instancevg|MiniCPM,confidence,0.44,0.20182627683407536,0.6511641096900433,2614.3206833783615,786.2122966160813 +grefcoco|instancevg|MiniCPM,confidence,0.45,0.20718600232538353,0.6505402262995208,2683.5476895082497,805.8673372847952 +grefcoco|instancevg|MiniCPM,confidence,0.46,0.21189348609023623,0.6500864929245952,2744.350245156723,823.1304946975282 +grefcoco|instancevg|MiniCPM,confidence,0.47,0.217139778237813,0.6500581345886624,2812.1121294637564,842.3695556695498 +grefcoco|instancevg|MiniCPM,confidence,0.48,0.22198905368232993,0.6507670929869835,2874.7460873907985,860.1526877031482 +grefcoco|instancevg|MiniCPM,confidence,0.49,0.22763236253296656,0.6515611263931033,2947.635898077823,880.8476776135931 +grefcoco|instancevg|MiniCPM,confidence,0.5,0.2345234381646485,0.6527521765022829,3036.6420488162507,906.1184441876538 +grefcoco|instancevg|MiniCPM,confidence,0.51,0.24382497235062248,0.6537447182599325,3156.782038290343,940.2287793164273 +grefcoco|instancevg|MiniCPM,confidence,0.52,0.252644414825738,0.6545387516660522,3270.6952600173017,972.5712007342581 +grefcoco|instancevg|MiniCPM,confidence,0.53,0.26401610753480986,0.6538581516036639,3417.573722758493,1014.2731653276672 +grefcoco|instancevg|MiniCPM,confidence,0.54,0.2760116836344043,0.6537163599239997,3572.5103555253854,1058.2630182528842 +grefcoco|instancevg|MiniCPM,confidence,0.55,0.2893401015228426,0.6533477015568726,3744.6621697108217,1107.1406326142362 +grefcoco|instancevg|MiniCPM,confidence,0.56,0.3036043444970649,0.6525536681507529,3928.90123893481,1159.4500794733005 +grefcoco|instancevg|MiniCPM,confidence,0.57,0.3175850041119587,0.6509088846666478,4109.477503601662,1210.719577090634 +grefcoco|instancevg|MiniCPM,confidence,0.58,0.33462836400760004,0.6498312679011996,4329.612057485591,1273.2205265186608 +grefcoco|instancevg|MiniCPM,confidence,0.59,0.35637920766809406,0.6465984176048549,4610.549167017995,1352.9846333594207 +grefcoco|instancevg|MiniCPM,confidence,0.6,0.37968975980489467,0.6442730340583614,4911.631701614652,1438.4684610297004 +grefcoco|instancevg|MiniCPM,confidence,0.61,0.4046450954258004,0.6405580920511584,5233.958502642702,1529.9839943020195 +grefcoco|instancevg|MiniCPM,confidence,0.62,0.43141536454640844,0.6357371749425744,5579.72725283643,1628.1552027214163 +grefcoco|instancevg|MiniCPM,confidence,0.63,0.4610214672603012,0.630604316138729,5962.124048601527,1736.725903558122 +grefcoco|instancevg|MiniCPM,confidence,0.64,0.4916484700677764,0.6244221989053682,6357.706940772317,1849.0404216650588 +grefcoco|instancevg|MiniCPM,confidence,0.65,0.5209426310864078,0.6187788900547316,6736.074651524562,1956.4671783358606 +grefcoco|instancevg|MiniCPM,confidence,0.66,0.5529308340186598,0.6118310977511839,7149.2390055696105,2073.773452803106 +grefcoco|instancevg|MiniCPM,confidence,0.67,0.5888041289737118,0.6045146470805093,7612.583782047433,2205.3270318820646 +grefcoco|instancevg|MiniCPM,confidence,0.68,0.6175027649377535,0.5983608881830814,7983.2596032296915,2310.5698951452314 +grefcoco|instancevg|MiniCPM,confidence,0.69,0.6421461588633979,0.5925474293168477,8301.557319244892,2400.941484251646 +grefcoco|instancevg|MiniCPM,confidence,0.7,0.665910444375124,0.5877548705441965,8608.500341132967,2488.0892307086956 +grefcoco|instancevg|MiniCPM,confidence,0.71,0.6838612710206166,0.5829339534356124,8840.355869599738,2553.9180177102617 +grefcoco|instancevg|MiniCPM,confidence,0.72,0.7029464311034228,0.5779996029832969,9086.862616295053,2623.9066016787506 +grefcoco|instancevg|MiniCPM,confidence,0.73,0.7180614241556306,0.5745966026713553,9282.090099190453,2679.3358962630077 +grefcoco|instancevg|MiniCPM,confidence,0.74,0.7352465757309361,0.5710234523438165,9504.05605535295,2742.3568203119 +grefcoco|instancevg|MiniCPM,confidence,0.75,0.7478093185491875,0.567762243711539,9666.318297234116,2788.426571720579 +grefcoco|instancevg|MiniCPM,confidence,0.76,0.759804894648782,0.565153276805717,9821.254930001009,2832.416424645796 +grefcoco|instancevg|MiniCPM,confidence,0.77,0.7729064458497575,0.5620622181890367,9990.476500540737,2880.4620796137633 +grefcoco|instancevg|MiniCPM,confidence,0.78,0.7855259053398747,0.5588860845645578,10153.47130333333,2926.739820870788 +grefcoco|instancevg|MiniCPM,confidence,0.79,0.7964722230099538,0.5562771176587358,10294.855559238817,2966.881861601601 +grefcoco|instancevg|MiniCPM,confidence,0.8,0.8082125740861527,0.5536114340810481,10446.495667904286,3009.93576020926 +grefcoco|instancevg|MiniCPM,confidence,0.81,0.8197544168108215,0.5504920171284349,10595.57181337976,3052.2616943477074 +grefcoco|instancevg|MiniCPM,confidence,0.82,0.8333380597226555,0.5469188668008961,10771.02015166662,3102.075263026618 +grefcoco|instancevg|MiniCPM,confidence,0.83,0.8426112355726966,0.5450188582933954,10890.793860684997,3136.0816032312177 +grefcoco|instancevg|MiniCPM,confidence,0.84,0.8529903865241187,0.5423531747157077,11024.85250747621,3174.1437454785687 +grefcoco|instancevg|MiniCPM,confidence,0.85,0.8635396874911381,0.5395456994583557,11161.108837001704,3212.829857270958 +grefcoco|instancevg|MiniCPM,confidence,0.86,0.8710262881774098,0.5378725576383178,11257.806877310119,3240.284517252654 +grefcoco|instancevg|MiniCPM,confidence,0.87,0.8809517057539064,0.535774040779287,11386.005036809911,3276.6827407132355 +grefcoco|instancevg|MiniCPM,confidence,0.88,0.8913024983693957,0.5330516405297338,11519.69740314541,3314.640888036413 +grefcoco|instancevg|MiniCPM,confidence,0.89,0.900915974250631,0.5304710319598446,11643.866477632353,3349.8951673310903 +grefcoco|instancevg|MiniCPM,confidence,0.9,0.910954825170859,0.5277486317102913,11773.529758955001,3386.7093704883646 +grefcoco|instancevg|MiniCPM,confidence,0.91,0.921901142840938,0.5245724980858123,11914.914014860486,3426.851411219177 +grefcoco|instancevg|MiniCPM,confidence,0.92,0.9303235686129938,0.5217366644925275,12023.699310207454,3457.737903698585 +grefcoco|instancevg|MiniCPM,confidence,0.93,0.9393698777755721,0.5187306808836457,12140.542775580121,3490.9122845098004 +grefcoco|instancevg|MiniCPM,confidence,0.94,0.9490117119927403,0.515526188923234,12265.078130522777,3526.270558728651 +grefcoco|instancevg|MiniCPM,confidence,0.95,0.9587953378895726,0.5122366219550237,12391.444887744003,3562.1488075683665 +grefcoco|instancevg|MiniCPM,confidence,0.96,0.9673595553412926,0.5097694467288659,12502.061585369538,3593.55527466864 +grefcoco|instancevg|MiniCPM,confidence,0.97,0.9746192893401016,0.5075574965261038,12595.829382032243,3620.177975256951 +grefcoco|instancevg|MiniCPM,confidence,0.98,0.9830984317840229,0.5046365879250205,12705.347238290638,3651.2724575847046 +grefcoco|instancevg|MiniCPM,confidence,0.99,0.9908118991577575,0.5019425460114001,12804.975522244764,3679.559076959785 +grefcoco|instancevg|MiniCPM,confidence,1.0,1.0,0.5000708958398321,12923.650389896,3713.253432391866 +construction|yolo26x|Qwen3.8,learned,0.0,0.0,0.18129624718318377,139.16228659200002,71.56936161300109 +construction|yolo26x|Qwen3.8,learned,0.01,0.013648468708388815,0.18693471712069915,1461.3537694872334,205.6311162003908 +construction|yolo26x|Qwen3.8,learned,0.02,0.019640479360852198,0.18781313491575913,2041.8280790509946,264.48749626314725 +construction|yolo26x|Qwen3.8,learned,0.03,0.031291611185086554,0.1913128749335621,3170.5281254249744,378.9304574962848 +construction|yolo26x|Qwen3.8,learned,0.04,0.04061251664447404,0.19266008228682668,4073.4881625241583,470.48482648279486 +construction|yolo26x|Qwen3.8,learned,0.05,0.045938748335552594,0.19531128485766117,4589.465326580834,522.8016087608005 +construction|yolo26x|Qwen3.8,learned,0.06,0.0559254327563249,0.19800731707634547,5556.922509187103,620.8955755320613 +construction|yolo26x|Qwen3.8,learned,0.07,0.07456724367509987,0.19763843353437874,7362.842583385471,804.0043135050814 +construction|yolo26x|Qwen3.8,learned,0.08,0.08688415446071904,0.1974112776278713,8556.039775266534,924.9868725229696 +construction|yolo26x|Qwen3.8,learned,0.09,0.09587217043941411,0.20030220967092682,9426.751239612175,1013.2714426171043 +construction|yolo26x|Qwen3.8,learned,0.1,0.09920106524633822,0.20211074868345033,9749.236967147599,1045.9694315408578 +construction|yolo26x|Qwen3.8,learned,0.11,0.11284953395472704,0.20834032699131655,11071.428450042833,1180.0311861282478 +construction|yolo26x|Qwen3.8,learned,0.12,0.12350199733688415,0.21121674052765757,12103.382778156185,1284.664750684259 +construction|yolo26x|Qwen3.8,learned,0.13,0.13315579227696406,0.21276215593410724,13038.591388008912,1379.4889185631446 +construction|yolo26x|Qwen3.8,learned,0.14,0.14347536617842876,0.21511521078428622,14038.297143368722,1480.8526842267806 +construction|yolo26x|Qwen3.8,learned,0.15,0.15246338215712382,0.21738179676765032,14909.008607714362,1569.1372543209152 +construction|yolo26x|Qwen3.8,learned,0.16,0.1604527296937417,0.22260454951009195,15682.974353799378,1647.612427737924 +construction|yolo26x|Qwen3.8,learned,0.17,0.1780958721704394,0.22627631882842034,17392.148709737117,1820.9117690338178 +construction|yolo26x|Qwen3.8,learned,0.18,0.18442077230359522,0.2270967395967396,18004.871592054424,1883.0379479889498 +construction|yolo26x|Qwen3.8,learned,0.19,0.2000665778961385,0.23397114982462924,19520.55451147091,2036.7184959305916 +construction|yolo26x|Qwen3.8,learned,0.2,0.20439414114513982,0.23512276426997616,19939.78595726696,2079.2258815314713 +construction|yolo26x|Qwen3.8,learned,0.21,0.21937416777629826,0.24202469545559713,21390.97173117636,2226.3668316883623 +construction|yolo26x|Qwen3.8,learned,0.22,0.22769640479360853,0.24353835097564847,22197.18605001492,2308.1118039977464 +construction|yolo26x|Qwen3.8,learned,0.23,0.23601864181091878,0.2454657390209296,23003.400368853476,2389.8567763071305 +construction|yolo26x|Qwen3.8,learned,0.24,0.24533954727030627,0.2483414219175777,23906.36040595266,2481.4111452936404 +construction|yolo26x|Qwen3.8,learned,0.25,0.25432756324900135,0.2504846177578749,24777.071870298303,2569.695715387775 +construction|yolo26x|Qwen3.8,learned,0.26,0.2636484687083888,0.252142216376585,25680.031907397482,2661.250084374285 +construction|yolo26x|Qwen3.8,learned,0.27,0.275965379494008,0.25276746625574176,26873.22909927855,2782.2326433921735 +construction|yolo26x|Qwen3.8,learned,0.28,0.29427430093209056,0.25702642690899347,28646.900600723373,2962.071582472818 +construction|yolo26x|Qwen3.8,learned,0.29,0.31091877496671105,0.26183035419940204,30259.32923840049,3125.561527091586 +construction|yolo26x|Qwen3.8,learned,0.3,0.322237017310253,0.2672887365132105,31355.780712020925,3236.734689432348 +construction|yolo26x|Qwen3.8,learned,0.31,0.33388814913448733,0.27039536303602457,32484.480758394904,3351.1776506654855 +construction|yolo26x|Qwen3.8,learned,0.32,0.34354194407456723,0.2713201570387918,33419.68936824763,3446.001818544371 +construction|yolo26x|Qwen3.8,learned,0.33,0.3492010652463382,0.2722481828097158,33967.91510505785,3501.588399714752 +construction|yolo26x|Qwen3.8,learned,0.34,0.3545272969374168,0.274417471554269,34483.89226911453,3553.905181992758 +construction|yolo26x|Qwen3.8,learned,0.35,0.3648468708388815,0.2782223498666452,35483.59802447434,3655.268947656394 +construction|yolo26x|Qwen3.8,learned,0.36,0.37217043941411454,0.2795152621397537,36193.06662505227,3727.204523288652 +construction|yolo26x|Qwen3.8,learned,0.37,0.37683089214380827,0.28090947933574756,36644.54664360186,3772.981707781907 +construction|yolo26x|Qwen3.8,learned,0.38,0.3854860186418109,0.28295009794869297,37483.00953519396,3857.9964789836663 +construction|yolo26x|Qwen3.8,learned,0.39,0.39980026631158455,0.28644294895435674,38869.69816359627,3998.5978313558066 +construction|yolo26x|Qwen3.8,learned,0.4,0.4054593874833555,0.2862200047733393,39417.923900406495,4054.1844125261878 +construction|yolo26x|Qwen3.8,learned,0.41,0.4104527296937417,0.28760386086589435,39901.65249170963,4103.231395911818 +construction|yolo26x|Qwen3.8,learned,0.42,0.4181091877496671,0.2930744270093899,40643.3696650411,4178.436770436451 +construction|yolo26x|Qwen3.8,learned,0.43,0.42809587217043943,0.2961685540536353,41610.82684764737,4276.530737207712 +construction|yolo26x|Qwen3.8,learned,0.44,0.4370838881491345,0.2977058723546182,42481.53831199301,4364.8153073018475 +construction|yolo26x|Qwen3.8,learned,0.45,0.44673768308921435,0.2999402430635043,43416.74692184573,4459.639475180732 +construction|yolo26x|Qwen3.8,learned,0.46,0.45206391478029295,0.2993046120720782,43932.72408590242,4511.956257458738 +construction|yolo26x|Qwen3.8,learned,0.47,0.45872170439414117,0.2998310238962732,44577.69554097326,4577.352235306245 +construction|yolo26x|Qwen3.8,learned,0.48,0.4673768308921438,0.30393211265289466,45416.15843256536,4662.367006508005 +construction|yolo26x|Qwen3.8,learned,0.49,0.47503328894806923,0.30350001249166053,46157.87560589683,4737.572381032637 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,learned,0.51,0.5046604527296937,0.29989697064322696,49027.99858096209,5028.5844824540445 +construction|yolo26x|Qwen3.8,learned,0.52,0.5206391478029294,0.297424793956044,50575.93007313212,5185.534829288062 +construction|yolo26x|Qwen3.8,learned,0.53,0.5249667110519307,0.297424793956044,50995.161518928166,5228.042214888941 +construction|yolo26x|Qwen3.8,learned,0.54,0.5436085219707057,0.2978194775414123,52801.08159312654,5411.150952861962 +construction|yolo26x|Qwen3.8,learned,0.55,0.5529294274300932,0.29630990060928936,53704.04163022572,5502.705321848472 +construction|yolo26x|Qwen3.8,learned,0.56,0.5612516644474035,0.29578344205458756,54510.25594906428,5584.450294157856 +construction|yolo26x|Qwen3.8,learned,0.57,0.5692410119840213,0.2950949302773625,55284.221695149296,5662.925467574864 +construction|yolo26x|Qwen3.8,learned,0.58,0.5902130492676432,0.2947373376338526,57315.88177862246,5868.922797794512 +construction|yolo26x|Qwen3.8,learned,0.59,0.6021970705725699,0.29722091277674884,58476.830397749975,5986.635557920024 +construction|yolo26x|Qwen3.8,learned,0.6,0.6208388814913449,0.29665021956289706,60282.750471948355,6169.744295893045 +construction|yolo26x|Qwen3.8,learned,0.61,0.6288282290279628,0.295988506182091,61056.71621803337,6248.219469310054 +construction|yolo26x|Qwen3.8,learned,0.62,0.6421438082556591,0.2939669175161438,62346.65912817505,6379.011425005067 +construction|yolo26x|Qwen3.8,learned,0.63,0.6491344873501997,0.29277362668991463,63023.87915599944,6447.67720174495 +construction|yolo26x|Qwen3.8,learned,0.64,0.6571238348868176,0.2930821542583563,63797.84490208446,6526.152375161959 +construction|yolo26x|Qwen3.8,learned,0.65,0.6677762982689747,0.2892305220657417,64829.79923019781,6630.78593971797 +construction|yolo26x|Qwen3.8,learned,0.66,0.6744340878828229,0.2915915088317972,65474.77068526865,6696.181917565477 +construction|yolo26x|Qwen3.8,learned,0.67,0.6854194407456724,0.2936540392680743,66538.97358613556,6804.085281013864 +construction|yolo26x|Qwen3.8,learned,0.68,0.7047270306258322,0.29632332439424064,68409.390805841,6993.733616771635 +construction|yolo26x|Qwen3.8,learned,0.69,0.7157123834886817,0.2937246709263134,69473.5937067079,7101.6369802200215 +construction|yolo26x|Qwen3.8,learned,0.7,0.7253661784287616,0.2952254059747369,70408.80231656063,7196.461148098907 +construction|yolo26x|Qwen3.8,learned,0.71,0.7283621837549934,0.2945683782087344,70699.03947134252,7225.889338130286 +construction|yolo26x|Qwen3.8,learned,0.72,0.741011984021305,0.29252963134197957,71924.48523597712,7350.1416960405495 +construction|yolo26x|Qwen3.8,learned,0.73,0.7526631158455392,0.2900395106945919,73053.1852823511,7464.584657273686 +construction|yolo26x|Qwen3.8,learned,0.74,0.7616511318242344,0.2887744568626224,73923.89674669674,7552.869227367822 +construction|yolo26x|Qwen3.8,learned,0.75,0.7709720372836218,0.28730364578560735,74826.85678379593,7644.4235963543315 +construction|yolo26x|Qwen3.8,learned,0.76,0.7749667110519307,0.2858070539975931,75213.83965683842,7683.661183062835 +construction|yolo26x|Qwen3.8,learned,0.77,0.7792942743009321,0.28488735368132817,75633.07110263448,7726.168568663716 +construction|yolo26x|Qwen3.8,learned,0.78,0.7846205059920106,0.2830658409372236,76149.04826669116,7778.485350941721 +construction|yolo26x|Qwen3.8,learned,0.79,0.7972703062583223,0.2809995077380739,77374.49403132577,7902.737708851985 +construction|yolo26x|Qwen3.8,learned,0.8,0.8129161118508655,0.28013559922601305,78890.17695074226,8056.418256793627 +construction|yolo26x|Qwen3.8,learned,0.81,0.8199067909454061,0.27814841049220523,79567.39697856663,8125.084033533509 +construction|yolo26x|Qwen3.8,learned,0.82,0.831225033288948,0.2772348763322643,80663.84845218707,8236.25719587427 +construction|yolo26x|Qwen3.8,learned,0.83,0.84254327563249,0.27700595116508253,81760.29992580751,8347.430358215033 +construction|yolo26x|Qwen3.8,learned,0.84,0.857190412782956,0.2763733409861609,83179.23712696337,8491.301509479548 +construction|yolo26x|Qwen3.8,learned,0.85,0.8618508655126498,0.2761345157638265,83630.71714551296,8537.078693972804 +construction|yolo26x|Qwen3.8,learned,0.86,0.87017310252996,0.2761345157638265,84436.93146435152,8618.823666282187 +construction|yolo26x|Qwen3.8,learned,0.87,0.8848202396804261,0.27546832609774896,85855.86866550738,8762.694817546704 +construction|yolo26x|Qwen3.8,learned,0.88,0.8984687083888149,0.27423981143437465,87178.06014840261,8896.756572134092 +construction|yolo26x|Qwen3.8,learned,0.89,0.9087882822902796,0.2736575311942959,88177.76590376243,8998.12033779773 +construction|yolo26x|Qwen3.8,learned,0.9,0.9161118508655126,0.272975172975173,88887.23450434035,9070.055913429987 +construction|yolo26x|Qwen3.8,learned,0.91,0.9217709720372836,0.2727190231806067,89435.46024115058,9125.64249460037 +construction|yolo26x|Qwen3.8,learned,0.92,0.9264314247669774,0.27264729817760425,89886.94025970016,9171.419679093624 +construction|yolo26x|Qwen3.8,learned,0.93,0.933422103861518,0.2741441928343702,90564.16028752456,9240.085455833507 +construction|yolo26x|Qwen3.8,learned,0.94,0.9417443408788282,0.2733716828250215,91370.37460636311,9321.83042814289 +construction|yolo26x|Qwen3.8,learned,0.95,0.9530625832223701,0.2727312493845365,92466.82607998354,9433.003590483651 +construction|yolo26x|Qwen3.8,learned,0.96,0.9613848202396804,0.27223682114986464,93273.04039882211,9514.748562793036 +construction|yolo26x|Qwen3.8,learned,0.97,0.9673768308921438,0.27453093162048386,93853.51470838586,9573.604942855793 +construction|yolo26x|Qwen3.8,learned,0.98,0.9760319573901465,0.2737331615154065,94691.97759997797,9658.619714057553 +construction|yolo26x|Qwen3.8,learned,0.99,0.9890146471371505,0.2725383412883413,95949.67193736612,9786.141870860192 +construction|yolo26x|Qwen3.8,learned,1.0,1.0,0.27001009067886245,97013.87483823301,9894.045234308578 +construction|yolo26x|Qwen3.8,confidence,0.0,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.01,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.02,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.03,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.04,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.05,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.06,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.07,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.08,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.09,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.1,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.11,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.12,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.13,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.14,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.15,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.16,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.17,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.18,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.19,0.17942743009320905,0.1784481207469488,17517.959245999285,1808.7296972439362 +construction|yolo26x|Qwen3.8,confidence,0.2,0.19340878828229027,0.1780928651788194,18872.39930164806,1946.0612507237013 +construction|yolo26x|Qwen3.8,confidence,0.21,0.20306258322237017,0.17829989216687975,19807.60791150079,2040.8854186025867 +construction|yolo26x|Qwen3.8,confidence,0.22,0.20838881491344874,0.1779854683890517,20323.585075557465,2093.2022008805925 +construction|yolo26x|Qwen3.8,confidence,0.23,0.21537949400798934,0.17917657728013692,21000.805103381852,2161.867977620475 +construction|yolo26x|Qwen3.8,confidence,0.24,0.23135818908122505,0.17803687432776102,22548.736595551884,2318.8183244544925 +construction|yolo26x|Qwen3.8,confidence,0.25,0.24800266311584554,0.18026802885144627,24161.165233228996,2482.3082690732604 +construction|yolo26x|Qwen3.8,confidence,0.26,0.2589880159786951,0.18119827871024874,25225.368134095894,2590.211632521647 +construction|yolo26x|Qwen3.8,confidence,0.27,0.27430093209054596,0.17963665085316138,26708.802480758837,2740.6223815709136 +construction|yolo26x|Qwen3.8,confidence,0.28,0.281624500665779,0.17924095839307458,27418.271081336767,2812.5579572031716 +construction|yolo26x|Qwen3.8,confidence,0.29,0.28761651131824234,0.1789157174618718,27998.745390900527,2871.4143372659278 +construction|yolo26x|Qwen3.8,confidence,0.3,0.2969374167776298,0.1787206845800119,28901.70542799971,2962.9687062524376 +construction|yolo26x|Qwen3.8,confidence,0.31,0.30459387483355527,0.17813242189706796,29643.422601331185,3038.174080777071 +construction|yolo26x|Qwen3.8,confidence,0.32,0.3129161118508655,0.17772000842137498,30449.636920169738,3119.919053086455 +construction|yolo26x|Qwen3.8,confidence,0.33,0.31990679094540614,0.17791259734473341,31126.85694799413,3188.5848298263377 +construction|yolo26x|Qwen3.8,confidence,0.34,0.3322237017310253,0.1789653934599337,32320.05413987519,3309.5673888442257 +construction|yolo26x|Qwen3.8,confidence,0.35,0.34054593874833555,0.18042682200794885,33126.268458713756,3391.31236115361 +construction|yolo26x|Qwen3.8,confidence,0.36,0.3488681757656458,0.17962877325912951,33932.48277755231,3473.057333462994 +construction|yolo26x|Qwen3.8,confidence,0.37,0.3581890812250333,0.18158304685510065,34835.44281465149,3564.611702449504 +construction|yolo26x|Qwen3.8,confidence,0.38,0.37017310252996005,0.18069239723097547,35996.39143377901,3682.3244625750167 +construction|yolo26x|Qwen3.8,confidence,0.39,0.37683089214380827,0.1812845803851709,36641.362888849864,3747.720440422524 +construction|yolo26x|Qwen3.8,confidence,0.4,0.38348868175765644,0.1824781664689562,37286.3343439207,3813.116418270031 +construction|yolo26x|Qwen3.8,confidence,0.41,0.4021304926764314,0.18220594501399923,39092.254418119075,3996.225156243051 +construction|yolo26x|Qwen3.8,confidence,0.42,0.41311584553928093,0.18162498930772586,40156.457318985966,4104.128519691438 +construction|yolo26x|Qwen3.8,confidence,0.43,0.42676431424766975,0.18066434967152634,41478.6488018812,4238.190274278827 +construction|yolo26x|Qwen3.8,confidence,0.44,0.4324234354194407,0.1803913789289444,42026.874538691416,4293.776855449209 +construction|yolo26x|Qwen3.8,confidence,0.45,0.44107856191744343,0.17959468902865128,42865.33743028352,4378.791626650968 +construction|yolo26x|Qwen3.8,confidence,0.46,0.45772303595206393,0.17863383514615866,44477.766067960634,4542.281571269737 +construction|yolo26x|Qwen3.8,confidence,0.47,0.4717043941411451,0.17957308538379646,45832.20612360941,4679.6131247495005 +construction|yolo26x|Qwen3.8,confidence,0.48,0.4826897470039947,0.17910245145856155,46896.40902447631,4787.516488197888 +construction|yolo26x|Qwen3.8,confidence,0.49,0.4896804260985353,0.17871833907402393,47573.62905230069,4856.182264937771 +construction|yolo26x|Qwen3.8,confidence,0.5,0.5013315579227696,0.18015785955296285,48702.32909867467,4970.625226170908 +construction|yolo26x|Qwen3.8,confidence,0.51,0.5116511318242344,0.1810878167493375,49702.034854034486,5071.988991834544 +construction|yolo26x|Qwen3.8,confidence,0.52,0.5183089214380826,0.18177962418520882,50347.00630910533,5137.384969682052 +construction|yolo26x|Qwen3.8,confidence,0.53,0.5252996005326231,0.18246323539519374,51024.226336929714,5206.050746421934 +construction|yolo26x|Qwen3.8,confidence,0.54,0.5396138482023968,0.1839876166906015,52410.914965332035,5346.652098794074 +construction|yolo26x|Qwen3.8,confidence,0.55,0.5452729693741678,0.1846674213907964,52959.14070214226,5402.2386799644555 +construction|yolo26x|Qwen3.8,confidence,0.56,0.559254327563249,0.18676294233791663,54313.58075779103,5539.57023344422 +construction|yolo26x|Qwen3.8,confidence,0.57,0.572237017310253,0.18938663221175195,55571.275095179175,5667.092390246859 +construction|yolo26x|Qwen3.8,confidence,0.58,0.5862183754993342,0.1909571002676625,56925.71515082795,5804.423943726624 +construction|yolo26x|Qwen3.8,confidence,0.59,0.5998668442077231,0.1934300101790686,58247.90663372319,5938.485698314014 +construction|yolo26x|Qwen3.8,confidence,0.6,0.6191744340878829,0.1973701682130997,60118.32385342864,6128.134034071785 +construction|yolo26x|Qwen3.8,confidence,0.61,0.6264980026631158,0.1986727924998457,60827.792454006565,6200.069609704043 +construction|yolo26x|Qwen3.8,confidence,0.62,0.6384820239680427,0.20186289191698192,61988.74107313409,6317.782369829556 +construction|yolo26x|Qwen3.8,confidence,0.63,0.6461384820239681,0.20246031058520178,62730.45824646557,6392.987744354189 +construction|yolo26x|Qwen3.8,confidence,0.64,0.653462050599201,0.20349944137622245,63439.92684704349,6464.923319986447 +construction|yolo26x|Qwen3.8,confidence,0.65,0.6597869507323568,0.20442990090722246,64052.64972936079,6527.049498941578 +construction|yolo26x|Qwen3.8,confidence,0.66,0.6661118508655126,0.2044993942500819,64665.372611678096,6589.17567789671 +construction|yolo26x|Qwen3.8,confidence,0.67,0.6734354194407457,0.2064379542733177,65374.84121225603,6661.1112535289685 +construction|yolo26x|Qwen3.8,confidence,0.68,0.6820905459387483,0.2065831206227621,66213.30410384812,6746.126024730727 +construction|yolo26x|Qwen3.8,confidence,0.69,0.6927430093209055,0.20802802394105854,67245.25843196148,6850.759589286739 +construction|yolo26x|Qwen3.8,confidence,0.7,0.7090545938748336,0.21169680596278434,68825.43849688505,7010.979735013132 +construction|yolo26x|Qwen3.8,confidence,0.71,0.7170439414114514,0.2139280527103987,69599.40424297006,7089.454908430141 +construction|yolo26x|Qwen3.8,confidence,0.72,0.7243675099866844,0.2146900231025185,70308.87284354799,7161.390484062398 +construction|yolo26x|Qwen3.8,confidence,0.73,0.7330226364846871,0.21653561577692484,71147.33573514009,7246.405255264158 +construction|yolo26x|Qwen3.8,confidence,0.74,0.7363515312916112,0.21721689648133474,71469.82146267552,7279.1032441879115 +construction|yolo26x|Qwen3.8,confidence,0.75,0.7486684420772304,0.22103445955201576,72663.01865455658,7400.0858032057995 +construction|yolo26x|Qwen3.8,confidence,0.76,0.757323568575233,0.22340134248367294,73501.48154614867,7485.100574407558 +construction|yolo26x|Qwen3.8,confidence,0.77,0.7639813581890812,0.2269322568854726,74146.45300121952,7550.496552255066 +construction|yolo26x|Qwen3.8,confidence,0.78,0.7733022636484687,0.22757452798038888,75049.4130383187,7642.050921241575 +construction|yolo26x|Qwen3.8,confidence,0.79,0.784953395472703,0.227821050912831,76178.11308469268,7756.493882474713 +construction|yolo26x|Qwen3.8,confidence,0.8,0.7939414114513982,0.22991865671924006,77048.82454903833,7844.7784525688485 +construction|yolo26x|Qwen3.8,confidence,0.81,0.8039280958721704,0.23486191981046528,78016.28173164459,7942.8724193401085 +construction|yolo26x|Qwen3.8,confidence,0.82,0.8122503328894807,0.23803916047544685,78822.49605048315,8024.617391649493 +construction|yolo26x|Qwen3.8,confidence,0.83,0.8215712383488681,0.24022947987632814,79725.45608758234,8116.171760636003 +construction|yolo26x|Qwen3.8,confidence,0.84,0.8388814913448736,0.24528627405740036,81402.38187076653,8286.201303039523 +construction|yolo26x|Qwen3.8,confidence,0.85,0.8535286284953395,0.249537136326177,82821.3190719224,8430.072454304036 +construction|yolo26x|Qwen3.8,confidence,0.86,0.8638482023968043,0.25211074301800224,83821.0248272822,8531.436219967674 +construction|yolo26x|Qwen3.8,confidence,0.87,0.8745006657789614,0.2540042317315216,84852.97915539556,8636.069784523685 +construction|yolo26x|Qwen3.8,confidence,0.88,0.8764980026631158,0.2540996517958595,85046.4705919168,8655.688577877936 +construction|yolo26x|Qwen3.8,confidence,0.89,0.8841544607190412,0.25436194678023316,85788.18776524828,8730.89395240257 +construction|yolo26x|Qwen3.8,confidence,0.9,0.8891478029294274,0.2534339447803836,86271.91635655142,8779.9409357882 +construction|yolo26x|Qwen3.8,confidence,0.91,0.8981358189081226,0.2563240880787248,87142.62782089706,8868.225505882336 +construction|yolo26x|Qwen3.8,confidence,0.92,0.9054593874833555,0.25551845731687656,87852.09642147498,8940.161081514594 +construction|yolo26x|Qwen3.8,confidence,0.93,0.9221038615179761,0.2597490004762105,89464.5250591521,9103.651026133362 +construction|yolo26x|Qwen3.8,confidence,0.94,0.9254327563249002,0.26306323674307786,89787.01078668753,9136.349015057116 +construction|yolo26x|Qwen3.8,confidence,0.95,0.9364181091877497,0.26218185491449153,90851.21368755442,9244.252378505502 +construction|yolo26x|Qwen3.8,confidence,0.96,0.953728362183755,0.26357052137555514,92528.13947073862,9414.281920909021 +construction|yolo26x|Qwen3.8,confidence,0.97,0.96271637816245,0.26400290307863034,93398.85093508427,9502.566491003154 +construction|yolo26x|Qwen3.8,confidence,0.98,0.9700399467376831,0.2647475724296899,94108.3195356622,9574.502066635414 +construction|yolo26x|Qwen3.8,confidence,0.99,0.987017310252996,0.2649306125590882,95752.99674609285,9741.261810146556 +construction|yolo26x|Qwen3.8,confidence,1.0,1.0,0.27001009067886245,97010.691083481,9868.783966949195 diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance.pdf b/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance.pdf new file mode 100644 index 0000000000000000000000000000000000000000..653813eba6017c7f65c30c46e4aed4cba3176700 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance.pdf differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance.png b/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance.png new file mode 100644 index 0000000000000000000000000000000000000000..894da76b7f008c1ae6a68dc84d7b1ddefc6c6eb9 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance_verified.png b/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance_verified.png new file mode 100644 index 0000000000000000000000000000000000000000..203ecc9659c889ee14888f02e1d4469fa23c9d3d Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_20260912/latency_performance_verified.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/latex_includes.tex b/releases/20260913_lr/paper_data/main_table_cost_20260912/latex_includes.tex new file mode 100644 index 0000000000000000000000000000000000000000..63ac7aabc093c05924f302e5198d0feadfc761e3 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/latex_includes.tex @@ -0,0 +1,7 @@ +% Copy cost_compact.pdf alongside this snippet, or adjust the path. +\begin{figure}[t] + \centering + \includegraphics[width=\columnwidth]{cost_compact.pdf} + \caption{Test performance versus estimated computation (top) and serial latency (bottom) for InstanceVG--MiniCPM on gRefCOCO and YOLO26x--Qwen on Construction. Only the original main-table learned router and confidence baseline are compared. Dots mark the operating points selected on validation. Costs combine A800 component measurements with each policy's actual test call rate; learned routing includes the distilled encoders and routing head. FLOPs use supported-operator counts and generation-length estimates. Latency is estimated from batch-one measurements, not timed end-to-end for each policy.} + \label{fig:main-cost} +\end{figure} diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/main_table_results.json b/releases/20260913_lr/paper_data/main_table_cost_20260912/main_table_results.json new file mode 100644 index 0000000000000000000000000000000000000000..c8a3401b634ec03b7aeb47b67efa15e2dba31663 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/main_table_results.json @@ -0,0 +1,10760 @@ +{ + "source": "C:\\Users\\csh10\\Documents\\CodexProjects\\ICASSP-COLLA\\paper_materials\\draft_20260912_revision19\\data\\results.json", + "protocol": "Confidence and learned routing each select a fixed threshold from the same 1..99% validation budget grid; threshold unchanged on test. No inference or training.", + "groups": [ + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "expert_display": "InstanceVG", + "run": "shared_12_instancevg_MiniCPM", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detections_above_07", + "max_score", + "mean_score", + "query_count", + "detections_above_07_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 65, + "validation_area_0_25": 0.7465176751704266, + "validation_point": { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.7716635041113219, + "rescued": 1117, + "harmed": 813, + "missed_rescue": 252, + "avoided_harm": 2108, + "selected_both_correct": 5984, + "selected_both_wrong": 1334 + }, + "test_point": { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.6187788900547316, + "rescued": 1563, + "harmed": 3352, + "missed_rescue": 1064, + "avoided_harm": 5250, + "selected_both_correct": 8439, + "selected_both_wrong": 5016 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1369, + "avoided_harm": 2921, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7498770117365943, + "rescued": 0, + "harmed": 6, + "missed_rescue": 1369, + "avoided_harm": 2915, + "selected_both_correct": 136, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7497364537212735, + "rescued": 0, + "harmed": 8, + "missed_rescue": 1369, + "avoided_harm": 2913, + "selected_both_correct": 276, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7494553376906318, + "rescued": 0, + "harmed": 12, + "missed_rescue": 1369, + "avoided_harm": 2909, + "selected_both_correct": 414, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.749314779675311, + "rescued": 0, + "harmed": 14, + "missed_rescue": 1369, + "avoided_harm": 2907, + "selected_both_correct": 555, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7492445006676506, + "rescued": 0, + "harmed": 15, + "missed_rescue": 1369, + "avoided_harm": 2906, + "selected_both_correct": 696, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7491742216599901, + "rescued": 0, + "harmed": 16, + "missed_rescue": 1369, + "avoided_harm": 2905, + "selected_both_correct": 837, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7491039426523297, + "rescued": 0, + "harmed": 17, + "missed_rescue": 1369, + "avoided_harm": 2904, + "selected_both_correct": 979, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7488931056293485, + "rescued": 0, + "harmed": 20, + "missed_rescue": 1369, + "avoided_harm": 2901, + "selected_both_correct": 1118, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.7485417105910465, + "rescued": 0, + "harmed": 25, + "missed_rescue": 1369, + "avoided_harm": 2896, + "selected_both_correct": 1255, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.748120036545084, + "rescued": 0, + "harmed": 31, + "missed_rescue": 1369, + "avoided_harm": 2890, + "selected_both_correct": 1391, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7477686415067819, + "rescued": 0, + "harmed": 36, + "missed_rescue": 1369, + "avoided_harm": 2885, + "selected_both_correct": 1529, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7472766884531591, + "rescued": 0, + "harmed": 43, + "missed_rescue": 1369, + "avoided_harm": 2878, + "selected_both_correct": 1664, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.746573898376555, + "rescued": 0, + "harmed": 53, + "missed_rescue": 1369, + "avoided_harm": 2868, + "selected_both_correct": 1796, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7463630613535737, + "rescued": 0, + "harmed": 56, + "missed_rescue": 1369, + "avoided_harm": 2865, + "selected_both_correct": 1936, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7461522243305925, + "rescued": 0, + "harmed": 59, + "missed_rescue": 1369, + "avoided_harm": 2862, + "selected_both_correct": 2075, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.74573055028463, + "rescued": 0, + "harmed": 65, + "missed_rescue": 1369, + "avoided_harm": 2856, + "selected_both_correct": 2211, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7454494342539884, + "rescued": 0, + "harmed": 69, + "missed_rescue": 1369, + "avoided_harm": 2852, + "selected_both_correct": 2349, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7451683182233467, + "rescued": 0, + "harmed": 73, + "missed_rescue": 1369, + "avoided_harm": 2848, + "selected_both_correct": 2488, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7444655281467426, + "rescued": 0, + "harmed": 83, + "missed_rescue": 1369, + "avoided_harm": 2838, + "selected_both_correct": 2620, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7439032960854592, + "rescued": 0, + "harmed": 91, + "missed_rescue": 1369, + "avoided_harm": 2830, + "selected_both_correct": 2754, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7430599479935344, + "rescued": 0, + "harmed": 103, + "missed_rescue": 1369, + "avoided_harm": 2818, + "selected_both_correct": 2885, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7422868789092698, + "rescued": 0, + "harmed": 114, + "missed_rescue": 1369, + "avoided_harm": 2807, + "selected_both_correct": 3016, + "selected_both_wrong": 0 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7413732518096845, + "rescued": 0, + "harmed": 127, + "missed_rescue": 1369, + "avoided_harm": 2794, + "selected_both_correct": 3144, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7406704617330804, + "rescued": 0, + "harmed": 137, + "missed_rescue": 1369, + "avoided_harm": 2784, + "selected_both_correct": 3276, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7400379506641366, + "rescued": 0, + "harmed": 146, + "missed_rescue": 1369, + "avoided_harm": 2775, + "selected_both_correct": 3410, + "selected_both_wrong": 1 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7391946025722117, + "rescued": 0, + "harmed": 158, + "missed_rescue": 1369, + "avoided_harm": 2763, + "selected_both_correct": 3540, + "selected_both_wrong": 1 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.738210696464966, + "rescued": 0, + "harmed": 172, + "missed_rescue": 1369, + "avoided_harm": 2749, + "selected_both_correct": 3668, + "selected_both_wrong": 1 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7374376273807014, + "rescued": 0, + "harmed": 183, + "missed_rescue": 1369, + "avoided_harm": 2738, + "selected_both_correct": 3799, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7362428842504743, + "rescued": 0, + "harmed": 200, + "missed_rescue": 1369, + "avoided_harm": 2721, + "selected_both_correct": 3924, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7352589781432286, + "rescued": 1, + "harmed": 215, + "missed_rescue": 1368, + "avoided_harm": 2706, + "selected_both_correct": 4050, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7342047930283224, + "rescued": 1, + "harmed": 230, + "missed_rescue": 1368, + "avoided_harm": 2691, + "selected_both_correct": 4177, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7327289338674537, + "rescued": 1, + "harmed": 251, + "missed_rescue": 1368, + "avoided_harm": 2670, + "selected_both_correct": 4299, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7313233537142455, + "rescued": 1, + "harmed": 271, + "missed_rescue": 1368, + "avoided_harm": 2650, + "selected_both_correct": 4421, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.7298474945533769, + "rescued": 1, + "harmed": 292, + "missed_rescue": 1368, + "avoided_harm": 2629, + "selected_both_correct": 4541, + "selected_both_wrong": 3 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7284419144001687, + "rescued": 1, + "harmed": 312, + "missed_rescue": 1368, + "avoided_harm": 2609, + "selected_both_correct": 4664, + "selected_both_wrong": 3 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7268957762316396, + "rescued": 1, + "harmed": 334, + "missed_rescue": 1368, + "avoided_harm": 2587, + "selected_both_correct": 4784, + "selected_both_wrong": 3 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.7252793590554502, + "rescued": 1, + "harmed": 357, + "missed_rescue": 1368, + "avoided_harm": 2564, + "selected_both_correct": 4903, + "selected_both_wrong": 3 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7234521048562794, + "rescued": 1, + "harmed": 383, + "missed_rescue": 1368, + "avoided_harm": 2538, + "selected_both_correct": 5019, + "selected_both_wrong": 4 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7216248506571087, + "rescued": 1, + "harmed": 409, + "missed_rescue": 1368, + "avoided_harm": 2512, + "selected_both_correct": 5134, + "selected_both_wrong": 5 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.719446201419636, + "rescued": 1, + "harmed": 440, + "missed_rescue": 1368, + "avoided_harm": 2481, + "selected_both_correct": 5242, + "selected_both_wrong": 8 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7169161571438611, + "rescued": 1, + "harmed": 476, + "missed_rescue": 1368, + "avoided_harm": 2445, + "selected_both_correct": 5347, + "selected_both_wrong": 9 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.7148077869140488, + "rescued": 3, + "harmed": 508, + "missed_rescue": 1366, + "avoided_harm": 2413, + "selected_both_correct": 5450, + "selected_both_wrong": 15 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7126994166842364, + "rescued": 5, + "harmed": 540, + "missed_rescue": 1364, + "avoided_harm": 2381, + "selected_both_correct": 5552, + "selected_both_wrong": 21 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7100288143931408, + "rescued": 5, + "harmed": 578, + "missed_rescue": 1364, + "avoided_harm": 2343, + "selected_both_correct": 5648, + "selected_both_wrong": 29 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7070770960714035, + "rescued": 5, + "harmed": 620, + "missed_rescue": 1364, + "avoided_harm": 2301, + "selected_both_correct": 5742, + "selected_both_wrong": 36 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7045470517956286, + "rescued": 6, + "harmed": 657, + "missed_rescue": 1363, + "avoided_harm": 2264, + "selected_both_correct": 5841, + "selected_both_wrong": 41 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7017358914892122, + "rescued": 7, + "harmed": 698, + "missed_rescue": 1362, + "avoided_harm": 2223, + "selected_both_correct": 5930, + "selected_both_wrong": 52 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7059526319488368, + "rescued": 76, + "harmed": 707, + "missed_rescue": 1293, + "avoided_harm": 2214, + "selected_both_correct": 5945, + "selected_both_wrong": 101 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7123480216459344, + "rescued": 167, + "harmed": 707, + "missed_rescue": 1202, + "avoided_harm": 2214, + "selected_both_correct": 5945, + "selected_both_wrong": 153 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7179000632511069, + "rescued": 247, + "harmed": 708, + "missed_rescue": 1122, + "avoided_harm": 2213, + "selected_both_correct": 5945, + "selected_both_wrong": 214 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.723381825848619, + "rescued": 326, + "harmed": 709, + "missed_rescue": 1043, + "avoided_harm": 2212, + "selected_both_correct": 5945, + "selected_both_wrong": 276 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.729004146461452, + "rescued": 407, + "harmed": 710, + "missed_rescue": 962, + "avoided_harm": 2211, + "selected_both_correct": 5945, + "selected_both_wrong": 337 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.7339939560053412, + "rescued": 478, + "harmed": 710, + "missed_rescue": 891, + "avoided_harm": 2211, + "selected_both_correct": 5946, + "selected_both_wrong": 407 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7390540445568908, + "rescued": 552, + "harmed": 712, + "missed_rescue": 817, + "avoided_harm": 2209, + "selected_both_correct": 5946, + "selected_both_wrong": 473 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.743692459062478, + "rescued": 620, + "harmed": 714, + "missed_rescue": 749, + "avoided_harm": 2207, + "selected_both_correct": 5947, + "selected_both_wrong": 544 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7482605945604048, + "rescued": 690, + "harmed": 719, + "missed_rescue": 679, + "avoided_harm": 2202, + "selected_both_correct": 5948, + "selected_both_wrong": 611 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7525476140276899, + "rescued": 755, + "harmed": 723, + "missed_rescue": 614, + "avoided_harm": 2198, + "selected_both_correct": 5948, + "selected_both_wrong": 684 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.7554993323494272, + "rescued": 802, + "harmed": 728, + "missed_rescue": 567, + "avoided_harm": 2193, + "selected_both_correct": 5949, + "selected_both_wrong": 772 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7591538407477686, + "rescued": 859, + "harmed": 733, + "missed_rescue": 510, + "avoided_harm": 2188, + "selected_both_correct": 5953, + "selected_both_wrong": 850 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7622461170848268, + "rescued": 911, + "harmed": 741, + "missed_rescue": 458, + "avoided_harm": 2180, + "selected_both_correct": 5956, + "selected_both_wrong": 929 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7658303464755077, + "rescued": 970, + "harmed": 749, + "missed_rescue": 399, + "avoided_harm": 2172, + "selected_both_correct": 5959, + "selected_both_wrong": 1001 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7682198327359617, + "rescued": 1014, + "harmed": 759, + "missed_rescue": 355, + "avoided_harm": 2162, + "selected_both_correct": 5961, + "selected_both_wrong": 1087 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.769976807927472, + "rescued": 1049, + "harmed": 769, + "missed_rescue": 320, + "avoided_harm": 2152, + "selected_both_correct": 5971, + "selected_both_wrong": 1175 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.7713121090730198, + "rescued": 1089, + "harmed": 790, + "missed_rescue": 280, + "avoided_harm": 2131, + "selected_both_correct": 5977, + "selected_both_wrong": 1250 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.7716635041113219, + "rescued": 1117, + "harmed": 813, + "missed_rescue": 252, + "avoided_harm": 2108, + "selected_both_correct": 5984, + "selected_both_wrong": 1334 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.7709607140347178, + "rescued": 1141, + "harmed": 847, + "missed_rescue": 228, + "avoided_harm": 2074, + "selected_both_correct": 5993, + "selected_both_wrong": 1410 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.7709607140347178, + "rescued": 1164, + "harmed": 870, + "missed_rescue": 205, + "avoided_harm": 2051, + "selected_both_correct": 6013, + "selected_both_wrong": 1486 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.7703282029657741, + "rescued": 1191, + "harmed": 906, + "missed_rescue": 178, + "avoided_harm": 2015, + "selected_both_correct": 6033, + "selected_both_wrong": 1545 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.7687117857895847, + "rescued": 1208, + "harmed": 946, + "missed_rescue": 161, + "avoided_harm": 1975, + "selected_both_correct": 6063, + "selected_both_wrong": 1601 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.6999086372900415, + "metric": 0.7656195094525265, + "rescued": 1224, + "harmed": 1006, + "missed_rescue": 145, + "avoided_harm": 1915, + "selected_both_correct": 6082, + "selected_both_wrong": 1647 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.7630191861690913, + "rescued": 1234, + "harmed": 1053, + "missed_rescue": 135, + "avoided_harm": 1868, + "selected_both_correct": 6118, + "selected_both_wrong": 1697 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.7605594209009768, + "rescued": 1247, + "harmed": 1101, + "missed_rescue": 122, + "avoided_harm": 1820, + "selected_both_correct": 6158, + "selected_both_wrong": 1738 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.7559210063953897, + "rescued": 1253, + "harmed": 1173, + "missed_rescue": 116, + "avoided_harm": 1748, + "selected_both_correct": 6185, + "selected_both_wrong": 1776 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.7533206831119544, + "rescued": 1265, + "harmed": 1222, + "missed_rescue": 104, + "avoided_harm": 1699, + "selected_both_correct": 6233, + "selected_both_wrong": 1809 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.7491742216599901, + "rescued": 1272, + "harmed": 1288, + "missed_rescue": 97, + "avoided_harm": 1633, + "selected_both_correct": 6272, + "selected_both_wrong": 1839 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.7455899922693091, + "rescued": 1280, + "harmed": 1347, + "missed_rescue": 89, + "avoided_harm": 1574, + "selected_both_correct": 6315, + "selected_both_wrong": 1872 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.7415840888326657, + "rescued": 1288, + "harmed": 1412, + "missed_rescue": 81, + "avoided_harm": 1509, + "selected_both_correct": 6353, + "selected_both_wrong": 1903 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.7378593014266639, + "rescued": 1295, + "harmed": 1472, + "missed_rescue": 74, + "avoided_harm": 1449, + "selected_both_correct": 6399, + "selected_both_wrong": 1932 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.7342750720359829, + "rescued": 1306, + "harmed": 1534, + "missed_rescue": 63, + "avoided_harm": 1387, + "selected_both_correct": 6446, + "selected_both_wrong": 1954 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.7297069365380561, + "rescued": 1316, + "harmed": 1609, + "missed_rescue": 53, + "avoided_harm": 1312, + "selected_both_correct": 6491, + "selected_both_wrong": 1967 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.7263335441703563, + "rescued": 1324, + "harmed": 1665, + "missed_rescue": 45, + "avoided_harm": 1256, + "selected_both_correct": 6553, + "selected_both_wrong": 1983 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.7221168037107316, + "rescued": 1329, + "harmed": 1730, + "missed_rescue": 40, + "avoided_harm": 1191, + "selected_both_correct": 6599, + "selected_both_wrong": 2009 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.718251458289409, + "rescued": 1333, + "harmed": 1789, + "missed_rescue": 36, + "avoided_harm": 1132, + "selected_both_correct": 6654, + "selected_both_wrong": 2034 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.7131913697378593, + "rescued": 1337, + "harmed": 1865, + "missed_rescue": 32, + "avoided_harm": 1056, + "selected_both_correct": 6699, + "selected_both_wrong": 2051 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.7093260243165367, + "rescued": 1340, + "harmed": 1923, + "missed_rescue": 29, + "avoided_harm": 998, + "selected_both_correct": 6763, + "selected_both_wrong": 2068 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.7037037037037037, + "rescued": 1343, + "harmed": 2006, + "missed_rescue": 26, + "avoided_harm": 915, + "selected_both_correct": 6810, + "selected_both_wrong": 2077 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.6985030571368332, + "rescued": 1347, + "harmed": 2084, + "missed_rescue": 22, + "avoided_harm": 837, + "selected_both_correct": 6856, + "selected_both_wrong": 2092 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.6937943636235856, + "rescued": 1353, + "harmed": 2157, + "missed_rescue": 16, + "avoided_harm": 764, + "selected_both_correct": 6910, + "selected_both_wrong": 2101 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.6895776231639609, + "rescued": 1356, + "harmed": 2220, + "missed_rescue": 13, + "avoided_harm": 701, + "selected_both_correct": 6973, + "selected_both_wrong": 2114 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.6846580926277321, + "rescued": 1359, + "harmed": 2293, + "missed_rescue": 10, + "avoided_harm": 628, + "selected_both_correct": 7029, + "selected_both_wrong": 2125 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.6794574460608617, + "rescued": 1359, + "harmed": 2367, + "missed_rescue": 10, + "avoided_harm": 554, + "selected_both_correct": 7084, + "selected_both_wrong": 2138 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.6744676365169724, + "rescued": 1361, + "harmed": 2440, + "missed_rescue": 8, + "avoided_harm": 481, + "selected_both_correct": 7144, + "selected_both_wrong": 2145 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.6701806170496872, + "rescued": 1364, + "harmed": 2504, + "missed_rescue": 5, + "avoided_harm": 417, + "selected_both_correct": 7216, + "selected_both_wrong": 2148 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.6651908075057981, + "rescued": 1367, + "harmed": 2578, + "missed_rescue": 2, + "avoided_harm": 343, + "selected_both_correct": 7278, + "selected_both_wrong": 2152 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.6606226720078713, + "rescued": 1367, + "harmed": 2643, + "missed_rescue": 2, + "avoided_harm": 278, + "selected_both_correct": 7349, + "selected_both_wrong": 2158 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.6557031414716424, + "rescued": 1368, + "harmed": 2714, + "missed_rescue": 1, + "avoided_harm": 207, + "selected_both_correct": 7414, + "selected_both_wrong": 2163 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.6521891910886218, + "rescued": 1368, + "harmed": 2764, + "missed_rescue": 1, + "avoided_harm": 157, + "selected_both_correct": 7498, + "selected_both_wrong": 2172 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.6481832876519783, + "rescued": 1369, + "harmed": 2822, + "missed_rescue": 0, + "avoided_harm": 99, + "selected_both_correct": 7574, + "selected_both_wrong": 2179 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6447396162766181, + "rescued": 1369, + "harmed": 2871, + "missed_rescue": 0, + "avoided_harm": 50, + "selected_both_correct": 7664, + "selected_both_wrong": 2182 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.6412256658935975, + "rescued": 1369, + "harmed": 2921, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7755, + "selected_both_wrong": 2184 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2627, + "avoided_harm": 8602, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.002467175226157729, + "metric": 0.6686044862887446, + "rescued": 0, + "harmed": 32, + "missed_rescue": 2627, + "avoided_harm": 8570, + "selected_both_correct": 55, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.004849275444516916, + "metric": 0.6678388112185577, + "rescued": 0, + "harmed": 59, + "missed_rescue": 2627, + "avoided_harm": 8543, + "selected_both_correct": 112, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.007600034030003119, + "metric": 0.667214927828035, + "rescued": 0, + "harmed": 81, + "missed_rescue": 2627, + "avoided_harm": 8521, + "selected_both_correct": 186, + "selected_both_wrong": 1 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.009811984232765221, + "metric": 0.6666761194453109, + "rescued": 0, + "harmed": 100, + "missed_rescue": 2627, + "avoided_harm": 8502, + "selected_both_correct": 245, + "selected_both_wrong": 1 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.012534384482318578, + "metric": 0.6662791027422511, + "rescued": 0, + "harmed": 114, + "missed_rescue": 2627, + "avoided_harm": 8488, + "selected_both_correct": 326, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.01607917647392451, + "metric": 0.6658537277032583, + "rescued": 0, + "harmed": 129, + "missed_rescue": 2627, + "avoided_harm": 8473, + "selected_both_correct": 436, + "selected_both_wrong": 2 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.019709043473328986, + "metric": 0.6654850693361314, + "rescued": 0, + "harmed": 142, + "missed_rescue": 2627, + "avoided_harm": 8460, + "selected_both_correct": 550, + "selected_both_wrong": 3 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.02353741882426339, + "metric": 0.6652865609846014, + "rescued": 0, + "harmed": 149, + "missed_rescue": 2627, + "avoided_harm": 8453, + "selected_both_correct": 678, + "selected_both_wrong": 3 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.028868785979638715, + "metric": 0.6648611859456087, + "rescued": 0, + "harmed": 164, + "missed_rescue": 2627, + "avoided_harm": 8438, + "selected_both_correct": 850, + "selected_both_wrong": 4 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.0345971698380739, + "metric": 0.6645208859144145, + "rescued": 0, + "harmed": 176, + "missed_rescue": 2627, + "avoided_harm": 8426, + "selected_both_correct": 1036, + "selected_both_wrong": 8 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.03981510364971783, + "metric": 0.6642089442191532, + "rescued": 0, + "harmed": 187, + "missed_rescue": 2627, + "avoided_harm": 8415, + "selected_both_correct": 1207, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.045855429203414344, + "metric": 0.6640387942035562, + "rescued": 0, + "harmed": 193, + "missed_rescue": 2627, + "avoided_harm": 8409, + "selected_both_correct": 1413, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.05317187987408899, + "metric": 0.6637268525082948, + "rescued": 1, + "harmed": 205, + "missed_rescue": 2626, + "avoided_harm": 8397, + "selected_both_correct": 1657, + "selected_both_wrong": 12 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.05960922213084536, + "metric": 0.6633581941411678, + "rescued": 1, + "harmed": 218, + "missed_rescue": 2626, + "avoided_harm": 8384, + "selected_both_correct": 1869, + "selected_both_wrong": 14 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.0675211978561098, + "metric": 0.6630178941099736, + "rescued": 1, + "harmed": 230, + "missed_rescue": 2626, + "avoided_harm": 8372, + "selected_both_correct": 2122, + "selected_both_wrong": 28 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.07551824858917279, + "metric": 0.6626492357428466, + "rescued": 1, + "harmed": 243, + "missed_rescue": 2626, + "avoided_harm": 8359, + "selected_both_correct": 2388, + "selected_both_wrong": 31 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.08212574086152624, + "metric": 0.6621671440319882, + "rescued": 1, + "harmed": 260, + "missed_rescue": 2626, + "avoided_harm": 8342, + "selected_both_correct": 2598, + "selected_both_wrong": 37 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.08856308311828262, + "metric": 0.6619969940163911, + "rescued": 1, + "harmed": 266, + "missed_rescue": 2626, + "avoided_harm": 8336, + "selected_both_correct": 2814, + "selected_both_wrong": 42 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.09480191702350906, + "metric": 0.6615716189773984, + "rescued": 1, + "harmed": 281, + "missed_rescue": 2626, + "avoided_harm": 8321, + "selected_both_correct": 3009, + "selected_both_wrong": 52 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.10007656750701868, + "metric": 0.6612029606102714, + "rescued": 1, + "harmed": 294, + "missed_rescue": 2626, + "avoided_harm": 8308, + "selected_both_correct": 3174, + "selected_both_wrong": 60 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.10554972634205825, + "metric": 0.6608910189150101, + "rescued": 1, + "harmed": 305, + "missed_rescue": 2626, + "avoided_harm": 8297, + "selected_both_correct": 3344, + "selected_both_wrong": 72 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.11005870175538099, + "metric": 0.6605790772197487, + "rescued": 1, + "harmed": 316, + "missed_rescue": 2626, + "avoided_harm": 8286, + "selected_both_correct": 3488, + "selected_both_wrong": 76 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6602671355244875, + "rescued": 3, + "harmed": 329, + "missed_rescue": 2624, + "avoided_harm": 8273, + "selected_both_correct": 3638, + "selected_both_wrong": 95 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.12029606102713893, + "metric": 0.6599835521651589, + "rescued": 4, + "harmed": 340, + "missed_rescue": 2623, + "avoided_harm": 8262, + "selected_both_correct": 3778, + "selected_both_wrong": 120 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.12477667810452883, + "metric": 0.6596999688058305, + "rescued": 4, + "harmed": 350, + "missed_rescue": 2623, + "avoided_harm": 8252, + "selected_both_correct": 3906, + "selected_both_wrong": 140 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.12914386183818732, + "metric": 0.659217877094972, + "rescued": 4, + "harmed": 367, + "missed_rescue": 2623, + "avoided_harm": 8235, + "selected_both_correct": 4023, + "selected_both_wrong": 160 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.1322916371267334, + "metric": 0.6587925020559794, + "rescued": 4, + "harmed": 382, + "missed_rescue": 2623, + "avoided_harm": 8220, + "selected_both_correct": 4102, + "selected_both_wrong": 177 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.13626180415733205, + "metric": 0.6582253353373224, + "rescued": 5, + "harmed": 403, + "missed_rescue": 2622, + "avoided_harm": 8199, + "selected_both_correct": 4197, + "selected_both_wrong": 200 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.14023197118793068, + "metric": 0.6578283186342625, + "rescued": 6, + "harmed": 418, + "missed_rescue": 2621, + "avoided_harm": 8184, + "selected_both_correct": 4295, + "selected_both_wrong": 226 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.14326631313274538, + "metric": 0.6573178685874713, + "rescued": 7, + "harmed": 437, + "missed_rescue": 2620, + "avoided_harm": 8165, + "selected_both_correct": 4366, + "selected_both_wrong": 242 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.14766185520233674, + "metric": 0.6569208518844114, + "rescued": 8, + "harmed": 452, + "missed_rescue": 2619, + "avoided_harm": 8150, + "selected_both_correct": 4472, + "selected_both_wrong": 275 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.15066783881121856, + "metric": 0.6565805518532173, + "rescued": 9, + "harmed": 465, + "missed_rescue": 2618, + "avoided_harm": 8137, + "selected_both_correct": 4550, + "selected_both_wrong": 289 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.15452457249808582, + "metric": 0.6561268184782917, + "rescued": 12, + "harmed": 484, + "missed_rescue": 2615, + "avoided_harm": 8118, + "selected_both_correct": 4633, + "selected_both_wrong": 320 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.15806936448969175, + "metric": 0.6558715934548961, + "rescued": 16, + "harmed": 497, + "missed_rescue": 2611, + "avoided_harm": 8105, + "selected_both_correct": 4709, + "selected_both_wrong": 352 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.16201117318435754, + "metric": 0.6555596517596347, + "rescued": 17, + "harmed": 509, + "missed_rescue": 2610, + "avoided_harm": 8093, + "selected_both_correct": 4781, + "selected_both_wrong": 406 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.1657544735274934, + "metric": 0.6551626350565749, + "rescued": 19, + "harmed": 525, + "missed_rescue": 2608, + "avoided_harm": 8077, + "selected_both_correct": 4844, + "selected_both_wrong": 457 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.17012165726115192, + "metric": 0.6545954683379179, + "rescued": 23, + "harmed": 549, + "missed_rescue": 2604, + "avoided_harm": 8053, + "selected_both_correct": 4930, + "selected_both_wrong": 497 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.17536794940872868, + "metric": 0.6540850182911266, + "rescued": 33, + "harmed": 577, + "missed_rescue": 2594, + "avoided_harm": 8025, + "selected_both_correct": 5012, + "selected_both_wrong": 562 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.17965005813458867, + "metric": 0.6534044182287383, + "rescued": 39, + "harmed": 607, + "missed_rescue": 2588, + "avoided_harm": 7995, + "selected_both_correct": 5077, + "selected_both_wrong": 612 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.18401724186824717, + "metric": 0.653092476533477, + "rescued": 44, + "harmed": 623, + "missed_rescue": 2583, + "avoided_harm": 7979, + "selected_both_correct": 5133, + "selected_both_wrong": 689 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.18858293395343562, + "metric": 0.6527521765022829, + "rescued": 46, + "harmed": 637, + "missed_rescue": 2581, + "avoided_harm": 7965, + "selected_both_correct": 5199, + "selected_both_wrong": 768 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.19238295096843716, + "metric": 0.6523835181351558, + "rescued": 52, + "harmed": 656, + "missed_rescue": 2575, + "avoided_harm": 7946, + "selected_both_correct": 5260, + "selected_both_wrong": 816 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.19689192638175992, + "metric": 0.6517596347446332, + "rescued": 63, + "harmed": 689, + "missed_rescue": 2564, + "avoided_harm": 7913, + "selected_both_correct": 5317, + "selected_both_wrong": 874 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.20182627683407536, + "metric": 0.6511641096900433, + "rescued": 77, + "harmed": 724, + "missed_rescue": 2550, + "avoided_harm": 7878, + "selected_both_correct": 5395, + "selected_both_wrong": 921 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.20718600232538353, + "metric": 0.6505402262995208, + "rescued": 97, + "harmed": 766, + "missed_rescue": 2530, + "avoided_harm": 7836, + "selected_both_correct": 5459, + "selected_both_wrong": 984 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.21189348609023623, + "metric": 0.6500864929245952, + "rescued": 122, + "harmed": 807, + "missed_rescue": 2505, + "avoided_harm": 7795, + "selected_both_correct": 5501, + "selected_both_wrong": 1042 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.217139778237813, + "metric": 0.6500581345886624, + "rescued": 151, + "harmed": 837, + "missed_rescue": 2476, + "avoided_harm": 7765, + "selected_both_correct": 5555, + "selected_both_wrong": 1114 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.22198905368232993, + "metric": 0.6507670929869835, + "rescued": 203, + "harmed": 864, + "missed_rescue": 2424, + "avoided_harm": 7738, + "selected_both_correct": 5574, + "selected_both_wrong": 1187 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.22763236253296656, + "metric": 0.6515611263931033, + "rescued": 267, + "harmed": 900, + "missed_rescue": 2360, + "avoided_harm": 7702, + "selected_both_correct": 5592, + "selected_both_wrong": 1268 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.2345234381646485, + "metric": 0.6527521765022829, + "rescued": 344, + "harmed": 935, + "missed_rescue": 2283, + "avoided_harm": 7667, + "selected_both_correct": 5617, + "selected_both_wrong": 1374 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.24382497235062248, + "metric": 0.6537447182599325, + "rescued": 430, + "harmed": 986, + "missed_rescue": 2197, + "avoided_harm": 7616, + "selected_both_correct": 5653, + "selected_both_wrong": 1529 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.252644414825738, + "metric": 0.6545387516660522, + "rescued": 501, + "harmed": 1029, + "missed_rescue": 2126, + "avoided_harm": 7573, + "selected_both_correct": 5683, + "selected_both_wrong": 1696 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.26401610753480986, + "metric": 0.6538581516036639, + "rescued": 571, + "harmed": 1123, + "missed_rescue": 2056, + "avoided_harm": 7479, + "selected_both_correct": 5730, + "selected_both_wrong": 1886 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.2760116836344043, + "metric": 0.6537163599239997, + "rescued": 647, + "harmed": 1204, + "missed_rescue": 1980, + "avoided_harm": 7398, + "selected_both_correct": 5804, + "selected_both_wrong": 2078 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.2893401015228426, + "metric": 0.6533477015568726, + "rescued": 728, + "harmed": 1298, + "missed_rescue": 1899, + "avoided_harm": 7304, + "selected_both_correct": 5894, + "selected_both_wrong": 2283 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.3036043444970649, + "metric": 0.6525536681507529, + "rescued": 811, + "harmed": 1409, + "missed_rescue": 1816, + "avoided_harm": 7193, + "selected_both_correct": 5983, + "selected_both_wrong": 2503 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.6509088846666478, + "rescued": 876, + "harmed": 1532, + "missed_rescue": 1751, + "avoided_harm": 7070, + "selected_both_correct": 6081, + "selected_both_wrong": 2710 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.33462836400760004, + "metric": 0.6498312679011996, + "rescued": 971, + "harmed": 1665, + "missed_rescue": 1656, + "avoided_harm": 6937, + "selected_both_correct": 6236, + "selected_both_wrong": 2928 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.35637920766809406, + "metric": 0.6465984176048549, + "rescued": 1048, + "harmed": 1856, + "missed_rescue": 1579, + "avoided_harm": 6746, + "selected_both_correct": 6453, + "selected_both_wrong": 3210 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.37968975980489467, + "metric": 0.6442730340583614, + "rescued": 1153, + "harmed": 2043, + "missed_rescue": 1474, + "avoided_harm": 6559, + "selected_both_correct": 6702, + "selected_both_wrong": 3491 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.4046450954258004, + "metric": 0.6405580920511584, + "rescued": 1230, + "harmed": 2251, + "missed_rescue": 1397, + "avoided_harm": 6351, + "selected_both_correct": 6991, + "selected_both_wrong": 3797 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.43141536454640844, + "metric": 0.6357371749425744, + "rescued": 1303, + "harmed": 2494, + "missed_rescue": 1324, + "avoided_harm": 6108, + "selected_both_correct": 7308, + "selected_both_wrong": 4108 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.4610214672603012, + "metric": 0.630604316138729, + "rescued": 1384, + "harmed": 2756, + "missed_rescue": 1243, + "avoided_harm": 5846, + "selected_both_correct": 7664, + "selected_both_wrong": 4453 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.4916484700677764, + "metric": 0.6244221989053682, + "rescued": 1476, + "harmed": 3066, + "missed_rescue": 1151, + "avoided_harm": 5536, + "selected_both_correct": 8035, + "selected_both_wrong": 4760 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.6187788900547316, + "rescued": 1563, + "harmed": 3352, + "missed_rescue": 1064, + "avoided_harm": 5250, + "selected_both_correct": 8439, + "selected_both_wrong": 5016 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.5529308340186598, + "metric": 0.6118310977511839, + "rescued": 1647, + "harmed": 3681, + "missed_rescue": 980, + "avoided_harm": 4921, + "selected_both_correct": 8872, + "selected_both_wrong": 5298 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.5888041289737118, + "metric": 0.6045146470805093, + "rescued": 1741, + "harmed": 4033, + "missed_rescue": 886, + "avoided_harm": 4569, + "selected_both_correct": 9334, + "selected_both_wrong": 5655 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6175027649377535, + "metric": 0.5983608881830814, + "rescued": 1805, + "harmed": 4314, + "missed_rescue": 822, + "avoided_harm": 4288, + "selected_both_correct": 9696, + "selected_both_wrong": 5960 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6421461588633979, + "metric": 0.5925474293168477, + "rescued": 1860, + "harmed": 4574, + "missed_rescue": 767, + "avoided_harm": 4028, + "selected_both_correct": 10008, + "selected_both_wrong": 6202 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.665910444375124, + "metric": 0.5877548705441965, + "rescued": 1925, + "harmed": 4808, + "missed_rescue": 702, + "avoided_harm": 3794, + "selected_both_correct": 10303, + "selected_both_wrong": 6446 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.5829339534356124, + "rescued": 1961, + "harmed": 5014, + "missed_rescue": 666, + "avoided_harm": 3588, + "selected_both_correct": 10525, + "selected_both_wrong": 6615 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7029464311034228, + "metric": 0.5779996029832969, + "rescued": 2000, + "harmed": 5227, + "missed_rescue": 627, + "avoided_harm": 3375, + "selected_both_correct": 10747, + "selected_both_wrong": 6814 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7180614241556306, + "metric": 0.5745966026713553, + "rescued": 2042, + "harmed": 5389, + "missed_rescue": 585, + "avoided_harm": 3213, + "selected_both_correct": 10923, + "selected_both_wrong": 6967 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7352465757309361, + "metric": 0.5710234523438165, + "rescued": 2087, + "harmed": 5560, + "missed_rescue": 540, + "avoided_harm": 3042, + "selected_both_correct": 11139, + "selected_both_wrong": 7141 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7478093185491875, + "metric": 0.567762243711539, + "rescued": 2117, + "harmed": 5705, + "missed_rescue": 510, + "avoided_harm": 2897, + "selected_both_correct": 11290, + "selected_both_wrong": 7258 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.759804894648782, + "metric": 0.565153276805717, + "rescued": 2149, + "harmed": 5829, + "missed_rescue": 478, + "avoided_harm": 2773, + "selected_both_correct": 11421, + "selected_both_wrong": 7394 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.7729064458497575, + "metric": 0.5620622181890367, + "rescued": 2183, + "harmed": 5972, + "missed_rescue": 444, + "avoided_harm": 2630, + "selected_both_correct": 11577, + "selected_both_wrong": 7523 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7855259053398747, + "metric": 0.5588860845645578, + "rescued": 2221, + "harmed": 6122, + "missed_rescue": 406, + "avoided_harm": 2480, + "selected_both_correct": 11724, + "selected_both_wrong": 7633 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.7964722230099538, + "metric": 0.5562771176587358, + "rescued": 2244, + "harmed": 6237, + "missed_rescue": 383, + "avoided_harm": 2365, + "selected_both_correct": 11855, + "selected_both_wrong": 7750 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.8082125740861527, + "metric": 0.5536114340810481, + "rescued": 2274, + "harmed": 6361, + "missed_rescue": 353, + "avoided_harm": 2241, + "selected_both_correct": 12000, + "selected_both_wrong": 7865 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8197544168108215, + "metric": 0.5504920171284349, + "rescued": 2300, + "harmed": 6497, + "missed_rescue": 327, + "avoided_harm": 2105, + "selected_both_correct": 12140, + "selected_both_wrong": 7970 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8333380597226555, + "metric": 0.5469188668008961, + "rescued": 2329, + "harmed": 6652, + "missed_rescue": 298, + "avoided_harm": 1950, + "selected_both_correct": 12321, + "selected_both_wrong": 8084 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8426112355726966, + "metric": 0.5450188582933954, + "rescued": 2356, + "harmed": 6746, + "missed_rescue": 271, + "avoided_harm": 1856, + "selected_both_correct": 12437, + "selected_both_wrong": 8174 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8529903865241187, + "metric": 0.5423531747157077, + "rescued": 2381, + "harmed": 6865, + "missed_rescue": 246, + "avoided_harm": 1737, + "selected_both_correct": 12586, + "selected_both_wrong": 8247 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8635396874911381, + "metric": 0.5395456994583557, + "rescued": 2414, + "harmed": 6997, + "missed_rescue": 213, + "avoided_harm": 1605, + "selected_both_correct": 12707, + "selected_both_wrong": 8333 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8710262881774098, + "metric": 0.5378725576383178, + "rescued": 2435, + "harmed": 7077, + "missed_rescue": 192, + "avoided_harm": 1525, + "selected_both_correct": 12808, + "selected_both_wrong": 8395 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8809517057539064, + "metric": 0.535774040779287, + "rescued": 2462, + "harmed": 7178, + "missed_rescue": 165, + "avoided_harm": 1424, + "selected_both_correct": 12942, + "selected_both_wrong": 8483 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8913024983693957, + "metric": 0.5330516405297338, + "rescued": 2480, + "harmed": 7292, + "missed_rescue": 147, + "avoided_harm": 1310, + "selected_both_correct": 13082, + "selected_both_wrong": 8576 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.900915974250631, + "metric": 0.5304710319598446, + "rescued": 2498, + "harmed": 7401, + "missed_rescue": 129, + "avoided_harm": 1201, + "selected_both_correct": 13224, + "selected_both_wrong": 8646 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.910954825170859, + "metric": 0.5277486317102913, + "rescued": 2518, + "harmed": 7517, + "missed_rescue": 109, + "avoided_harm": 1085, + "selected_both_correct": 13379, + "selected_both_wrong": 8709 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.921901142840938, + "metric": 0.5245724980858123, + "rescued": 2542, + "harmed": 7653, + "missed_rescue": 85, + "avoided_harm": 949, + "selected_both_correct": 13541, + "selected_both_wrong": 8773 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9303235686129938, + "metric": 0.5217366644925275, + "rescued": 2553, + "harmed": 7764, + "missed_rescue": 74, + "avoided_harm": 838, + "selected_both_correct": 13673, + "selected_both_wrong": 8816 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9393698777755721, + "metric": 0.5187306808836457, + "rescued": 2567, + "harmed": 7884, + "missed_rescue": 60, + "avoided_harm": 718, + "selected_both_correct": 13819, + "selected_both_wrong": 8855 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9490117119927403, + "metric": 0.515526188923234, + "rescued": 2578, + "harmed": 8008, + "missed_rescue": 49, + "avoided_harm": 594, + "selected_both_correct": 13977, + "selected_both_wrong": 8902 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9587953378895726, + "metric": 0.5122366219550237, + "rescued": 2588, + "harmed": 8134, + "missed_rescue": 39, + "avoided_harm": 468, + "selected_both_correct": 14140, + "selected_both_wrong": 8948 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9673595553412926, + "metric": 0.5097694467288659, + "rescued": 2604, + "harmed": 8237, + "missed_rescue": 23, + "avoided_harm": 365, + "selected_both_correct": 14295, + "selected_both_wrong": 8976 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9746192893401016, + "metric": 0.5075574965261038, + "rescued": 2615, + "harmed": 8326, + "missed_rescue": 12, + "avoided_harm": 276, + "selected_both_correct": 14432, + "selected_both_wrong": 8995 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9830984317840229, + "metric": 0.5046365879250205, + "rescued": 2623, + "harmed": 8437, + "missed_rescue": 4, + "avoided_harm": 165, + "selected_both_correct": 14600, + "selected_both_wrong": 9007 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9908118991577575, + "metric": 0.5019425460114001, + "rescued": 2626, + "harmed": 8535, + "missed_rescue": 1, + "avoided_harm": 67, + "selected_both_correct": 14761, + "selected_both_wrong": 9017 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5000708958398321, + "rescued": 2627, + "harmed": 8602, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 15007, + "selected_both_wrong": 9027 + } + ] + }, + "learned": { + "selected_budget_index": 23, + "validation_area_0_25": 0.7607140347178298, + "validation_point": { + "budget": 0.23, + "threshold": -5.381783485412598, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7729988052568698, + "rescued": 773, + "harmed": 450, + "missed_rescue": 596, + "avoided_harm": 2471, + "selected_both_correct": 843, + "selected_both_wrong": 1206 + }, + "test_point": { + "budget": 0.23, + "threshold": -5.381783485412598, + "operator": "ge", + "actual_fraction": 0.30904914499617164, + "metric": 0.6671865694921022, + "rescued": 1570, + "harmed": 1652, + "missed_rescue": 1057, + "avoided_harm": 6950, + "selected_both_correct": 1879, + "selected_both_wrong": 5797 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1369, + "avoided_harm": 2921, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.430821418762207, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7526881720430108, + "rescued": 38, + "harmed": 4, + "missed_rescue": 1331, + "avoided_harm": 2917, + "selected_both_correct": 21, + "selected_both_wrong": 79 + }, + { + "budget": 0.02, + "threshold": 0.8396652936935425, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7528287300583316, + "rescued": 59, + "harmed": 23, + "missed_rescue": 1310, + "avoided_harm": 2898, + "selected_both_correct": 71, + "selected_both_wrong": 131 + }, + { + "budget": 0.03, + "threshold": -0.16024979948997498, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7518448239510858, + "rescued": 70, + "harmed": 48, + "missed_rescue": 1299, + "avoided_harm": 2873, + "selected_both_correct": 129, + "selected_both_wrong": 179 + }, + { + "budget": 0.04, + "threshold": -0.9098284244537354, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7521962189893878, + "rescued": 92, + "harmed": 65, + "missed_rescue": 1277, + "avoided_harm": 2856, + "selected_both_correct": 177, + "selected_both_wrong": 235 + }, + { + "budget": 0.05, + "threshold": -1.4125885963439941, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7518448239510858, + "rescued": 113, + "harmed": 91, + "missed_rescue": 1256, + "avoided_harm": 2830, + "selected_both_correct": 214, + "selected_both_wrong": 293 + }, + { + "budget": 0.06, + "threshold": -1.8043409585952759, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7523367770047087, + "rescued": 137, + "harmed": 108, + "missed_rescue": 1232, + "avoided_harm": 2813, + "selected_both_correct": 265, + "selected_both_wrong": 343 + }, + { + "budget": 0.07, + "threshold": -2.2280092239379883, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7530395670813128, + "rescued": 170, + "harmed": 131, + "missed_rescue": 1199, + "avoided_harm": 2790, + "selected_both_correct": 297, + "selected_both_wrong": 398 + }, + { + "budget": 0.08, + "threshold": -2.5369863510131836, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.754093752196219, + "rescued": 196, + "harmed": 142, + "missed_rescue": 1173, + "avoided_harm": 2779, + "selected_both_correct": 331, + "selected_both_wrong": 469 + }, + { + "budget": 0.09, + "threshold": -2.807616949081421, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.754937100288144, + "rescued": 233, + "harmed": 167, + "missed_rescue": 1136, + "avoided_harm": 2754, + "selected_both_correct": 359, + "selected_both_wrong": 521 + }, + { + "budget": 0.1, + "threshold": -3.0437393188476562, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.7566237964719938, + "rescued": 272, + "harmed": 182, + "missed_rescue": 1097, + "avoided_harm": 2739, + "selected_both_correct": 384, + "selected_both_wrong": 584 + }, + { + "budget": 0.11, + "threshold": -3.299111843109131, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7583807716635041, + "rescued": 319, + "harmed": 204, + "missed_rescue": 1050, + "avoided_harm": 2717, + "selected_both_correct": 413, + "selected_both_wrong": 629 + }, + { + "budget": 0.12, + "threshold": -3.5267717838287354, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7603485838779956, + "rescued": 359, + "harmed": 216, + "missed_rescue": 1010, + "avoided_harm": 2705, + "selected_both_correct": 441, + "selected_both_wrong": 691 + }, + { + "budget": 0.13, + "threshold": -3.745974540710449, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.761613606015883, + "rescued": 395, + "harmed": 234, + "missed_rescue": 974, + "avoided_harm": 2687, + "selected_both_correct": 463, + "selected_both_wrong": 757 + }, + { + "budget": 0.14, + "threshold": -3.9486825466156006, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7631597441844121, + "rescued": 432, + "harmed": 249, + "missed_rescue": 937, + "avoided_harm": 2672, + "selected_both_correct": 494, + "selected_both_wrong": 817 + }, + { + "budget": 0.15, + "threshold": -4.13905668258667, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7641436502916579, + "rescued": 468, + "harmed": 271, + "missed_rescue": 901, + "avoided_harm": 2650, + "selected_both_correct": 519, + "selected_both_wrong": 876 + }, + { + "budget": 0.16, + "threshold": -4.306515693664551, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.7654086724295452, + "rescued": 505, + "harmed": 290, + "missed_rescue": 864, + "avoided_harm": 2631, + "selected_both_correct": 558, + "selected_both_wrong": 923 + }, + { + "budget": 0.17, + "threshold": -4.4712700843811035, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.767235926628716, + "rescued": 551, + "harmed": 310, + "missed_rescue": 818, + "avoided_harm": 2611, + "selected_both_correct": 591, + "selected_both_wrong": 966 + }, + { + "budget": 0.18, + "threshold": -4.667325019836426, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.768079274720641, + "rescued": 591, + "harmed": 338, + "missed_rescue": 778, + "avoided_harm": 2583, + "selected_both_correct": 623, + "selected_both_wrong": 1009 + }, + { + "budget": 0.19, + "threshold": -4.808548450469971, + "operator": "gt", + "actual_fraction": 0.18989387869843277, + "metric": 0.7678684376976597, + "rescued": 618, + "harmed": 368, + "missed_rescue": 751, + "avoided_harm": 2553, + "selected_both_correct": 659, + "selected_both_wrong": 1057 + }, + { + "budget": 0.2, + "threshold": -4.945236682891846, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7694848548738492, + "rescued": 658, + "harmed": 385, + "missed_rescue": 711, + "avoided_harm": 2536, + "selected_both_correct": 708, + "selected_both_wrong": 1094 + }, + { + "budget": 0.21, + "threshold": -5.0980048179626465, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7709607140347178, + "rescued": 696, + "harmed": 402, + "missed_rescue": 673, + "avoided_harm": 2519, + "selected_both_correct": 755, + "selected_both_wrong": 1135 + }, + { + "budget": 0.22, + "threshold": -5.24181604385376, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7718040621266428, + "rescued": 731, + "harmed": 425, + "missed_rescue": 638, + "avoided_harm": 2496, + "selected_both_correct": 800, + "selected_both_wrong": 1174 + }, + { + "budget": 0.23, + "threshold": -5.381783485412598, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7729988052568698, + "rescued": 773, + "harmed": 450, + "missed_rescue": 596, + "avoided_harm": 2471, + "selected_both_correct": 843, + "selected_both_wrong": 1206 + }, + { + "budget": 0.24, + "threshold": -5.480440616607666, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7727176892262281, + "rescued": 796, + "harmed": 477, + "missed_rescue": 573, + "avoided_harm": 2444, + "selected_both_correct": 898, + "selected_both_wrong": 1243 + }, + { + "budget": 0.25, + "threshold": -5.592668533325195, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7726474102185678, + "rescued": 823, + "harmed": 505, + "missed_rescue": 546, + "avoided_harm": 2416, + "selected_both_correct": 951, + "selected_both_wrong": 1278 + }, + { + "budget": 0.26, + "threshold": -5.701513290405273, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7725068522032469, + "rescued": 849, + "harmed": 533, + "missed_rescue": 520, + "avoided_harm": 2388, + "selected_both_correct": 1019, + "selected_both_wrong": 1298 + }, + { + "budget": 0.27, + "threshold": -5.817866802215576, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7726474102185678, + "rescued": 879, + "harmed": 561, + "missed_rescue": 490, + "avoided_harm": 2360, + "selected_both_correct": 1077, + "selected_both_wrong": 1324 + }, + { + "budget": 0.28, + "threshold": -5.915761947631836, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7726474102185678, + "rescued": 901, + "harmed": 583, + "missed_rescue": 468, + "avoided_harm": 2338, + "selected_both_correct": 1152, + "selected_both_wrong": 1348 + }, + { + "budget": 0.29, + "threshold": -6.032350063323975, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7715932251036616, + "rescued": 915, + "harmed": 612, + "missed_rescue": 454, + "avoided_harm": 2309, + "selected_both_correct": 1218, + "selected_both_wrong": 1381 + }, + { + "budget": 0.3, + "threshold": -6.134230136871338, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7706093189964157, + "rescued": 930, + "harmed": 641, + "missed_rescue": 439, + "avoided_harm": 2280, + "selected_both_correct": 1281, + "selected_both_wrong": 1416 + }, + { + "budget": 0.31, + "threshold": -6.218080520629883, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7701173659427929, + "rescued": 954, + "harmed": 672, + "missed_rescue": 415, + "avoided_harm": 2249, + "selected_both_correct": 1336, + "selected_both_wrong": 1448 + }, + { + "budget": 0.32, + "threshold": -6.302128791809082, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7697659709044908, + "rescued": 972, + "harmed": 695, + "missed_rescue": 397, + "avoided_harm": 2226, + "selected_both_correct": 1412, + "selected_both_wrong": 1474 + }, + { + "budget": 0.33, + "threshold": -6.377316474914551, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7685712277742638, + "rescued": 986, + "harmed": 726, + "missed_rescue": 383, + "avoided_harm": 2195, + "selected_both_correct": 1485, + "selected_both_wrong": 1498 + }, + { + "budget": 0.34, + "threshold": -6.449829578399658, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.768430669758943, + "rescued": 1008, + "harmed": 750, + "missed_rescue": 361, + "avoided_harm": 2171, + "selected_both_correct": 1563, + "selected_both_wrong": 1516 + }, + { + "budget": 0.35, + "threshold": -6.539102077484131, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7677981586899993, + "rescued": 1025, + "harmed": 776, + "missed_rescue": 344, + "avoided_harm": 2145, + "selected_both_correct": 1641, + "selected_both_wrong": 1538 + }, + { + "budget": 0.36, + "threshold": -6.614948272705078, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7666034155597723, + "rescued": 1035, + "harmed": 803, + "missed_rescue": 334, + "avoided_harm": 2118, + "selected_both_correct": 1725, + "selected_both_wrong": 1559 + }, + { + "budget": 0.37, + "threshold": -6.686672210693359, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.7658303464755077, + "rescued": 1047, + "harmed": 826, + "missed_rescue": 322, + "avoided_harm": 2095, + "selected_both_correct": 1797, + "selected_both_wrong": 1594 + }, + { + "budget": 0.38, + "threshold": -6.78654146194458, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7642139292993183, + "rescued": 1056, + "harmed": 858, + "missed_rescue": 313, + "avoided_harm": 2063, + "selected_both_correct": 1876, + "selected_both_wrong": 1617 + }, + { + "budget": 0.39, + "threshold": -6.846216201782227, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7630894651767517, + "rescued": 1067, + "harmed": 885, + "missed_rescue": 302, + "avoided_harm": 2036, + "selected_both_correct": 1960, + "selected_both_wrong": 1637 + }, + { + "budget": 0.4, + "threshold": -6.915761470794678, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.7618947220465248, + "rescued": 1083, + "harmed": 918, + "missed_rescue": 286, + "avoided_harm": 2003, + "selected_both_correct": 2035, + "selected_both_wrong": 1655 + }, + { + "budget": 0.41, + "threshold": -6.9817047119140625, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7602783048703352, + "rescued": 1096, + "harmed": 954, + "missed_rescue": 273, + "avoided_harm": 1967, + "selected_both_correct": 2111, + "selected_both_wrong": 1672 + }, + { + "budget": 0.42, + "threshold": -7.047516822814941, + "operator": "gt", + "actual_fraction": 0.4198467917633003, + "metric": 0.7592943987630895, + "rescued": 1106, + "harmed": 978, + "missed_rescue": 263, + "avoided_harm": 1943, + "selected_both_correct": 2195, + "selected_both_wrong": 1695 + }, + { + "budget": 0.43, + "threshold": -7.1213059425354, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7578185396022208, + "rescued": 1114, + "harmed": 1007, + "missed_rescue": 255, + "avoided_harm": 1914, + "selected_both_correct": 2279, + "selected_both_wrong": 1718 + }, + { + "budget": 0.44, + "threshold": -7.181412696838379, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7569049125026355, + "rescued": 1125, + "harmed": 1031, + "missed_rescue": 244, + "avoided_harm": 1890, + "selected_both_correct": 2363, + "selected_both_wrong": 1741 + }, + { + "budget": 0.45, + "threshold": -7.25251579284668, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7559210063953897, + "rescued": 1135, + "harmed": 1055, + "missed_rescue": 234, + "avoided_harm": 1866, + "selected_both_correct": 2455, + "selected_both_wrong": 1758 + }, + { + "budget": 0.46, + "threshold": -7.320802211761475, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7554290533417668, + "rescued": 1144, + "harmed": 1071, + "missed_rescue": 225, + "avoided_harm": 1850, + "selected_both_correct": 2558, + "selected_both_wrong": 1772 + }, + { + "budget": 0.47, + "threshold": -7.374159812927246, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7533206831119544, + "rescued": 1148, + "harmed": 1105, + "missed_rescue": 221, + "avoided_harm": 1816, + "selected_both_correct": 2651, + "selected_both_wrong": 1783 + }, + { + "budget": 0.48, + "threshold": -7.4538164138793945, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7522664979970483, + "rescued": 1156, + "harmed": 1128, + "missed_rescue": 213, + "avoided_harm": 1793, + "selected_both_correct": 2751, + "selected_both_wrong": 1794 + }, + { + "budget": 0.49, + "threshold": -7.532625675201416, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7510014758591609, + "rescued": 1162, + "harmed": 1152, + "missed_rescue": 207, + "avoided_harm": 1769, + "selected_both_correct": 2852, + "selected_both_wrong": 1806 + }, + { + "budget": 0.5, + "threshold": -7.600386619567871, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7501581277672359, + "rescued": 1172, + "harmed": 1174, + "missed_rescue": 197, + "avoided_harm": 1747, + "selected_both_correct": 2947, + "selected_both_wrong": 1821 + }, + { + "budget": 0.51, + "threshold": -7.662148952484131, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.7490336636446694, + "rescued": 1183, + "harmed": 1201, + "missed_rescue": 186, + "avoided_harm": 1720, + "selected_both_correct": 3036, + "selected_both_wrong": 1836 + }, + { + "budget": 0.52, + "threshold": -7.733202934265137, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.7479091995221028, + "rescued": 1192, + "harmed": 1226, + "missed_rescue": 177, + "avoided_harm": 1695, + "selected_both_correct": 3132, + "selected_both_wrong": 1849 + }, + { + "budget": 0.53, + "threshold": -7.79387092590332, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.7471361304378382, + "rescued": 1203, + "harmed": 1248, + "missed_rescue": 166, + "avoided_harm": 1673, + "selected_both_correct": 3232, + "selected_both_wrong": 1858 + }, + { + "budget": 0.54, + "threshold": -7.855954647064209, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7453088762386675, + "rescued": 1211, + "harmed": 1282, + "missed_rescue": 158, + "avoided_harm": 1639, + "selected_both_correct": 3315, + "selected_both_wrong": 1875 + }, + { + "budget": 0.55, + "threshold": -7.925965785980225, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.7439032960854592, + "rescued": 1219, + "harmed": 1310, + "missed_rescue": 150, + "avoided_harm": 1611, + "selected_both_correct": 3410, + "selected_both_wrong": 1886 + }, + { + "budget": 0.56, + "threshold": -7.977593421936035, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7429193899782135, + "rescued": 1229, + "harmed": 1334, + "missed_rescue": 140, + "avoided_harm": 1587, + "selected_both_correct": 3507, + "selected_both_wrong": 1898 + }, + { + "budget": 0.57, + "threshold": -8.031231880187988, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7415138098250053, + "rescued": 1240, + "harmed": 1365, + "missed_rescue": 129, + "avoided_harm": 1556, + "selected_both_correct": 3597, + "selected_both_wrong": 1908 + }, + { + "budget": 0.58, + "threshold": -8.088851928710938, + "operator": "ge", + "actual_fraction": 0.5799423712137185, + "metric": 0.7399676716564763, + "rescued": 1245, + "harmed": 1392, + "missed_rescue": 124, + "avoided_harm": 1529, + "selected_both_correct": 3698, + "selected_both_wrong": 1917 + }, + { + "budget": 0.59, + "threshold": -8.145581245422363, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7384215334879471, + "rescued": 1251, + "harmed": 1420, + "missed_rescue": 118, + "avoided_harm": 1501, + "selected_both_correct": 3793, + "selected_both_wrong": 1931 + }, + { + "budget": 0.6, + "threshold": -8.199271202087402, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7371565113500598, + "rescued": 1254, + "harmed": 1441, + "missed_rescue": 115, + "avoided_harm": 1480, + "selected_both_correct": 3894, + "selected_both_wrong": 1948 + }, + { + "budget": 0.61, + "threshold": -8.256582260131836, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7357509311968515, + "rescued": 1260, + "harmed": 1467, + "missed_rescue": 109, + "avoided_harm": 1454, + "selected_both_correct": 3992, + "selected_both_wrong": 1960 + }, + { + "budget": 0.62, + "threshold": -8.316590309143066, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7338533979900204, + "rescued": 1265, + "harmed": 1499, + "missed_rescue": 104, + "avoided_harm": 1422, + "selected_both_correct": 4091, + "selected_both_wrong": 1966 + }, + { + "budget": 0.63, + "threshold": -8.380763053894043, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.731745027760208, + "rescued": 1271, + "harmed": 1535, + "missed_rescue": 98, + "avoided_harm": 1386, + "selected_both_correct": 4186, + "selected_both_wrong": 1972 + }, + { + "budget": 0.64, + "threshold": -8.436734199523926, + "operator": "gt", + "actual_fraction": 0.6398903647480497, + "metric": 0.7303394476069998, + "rescued": 1277, + "harmed": 1561, + "missed_rescue": 92, + "avoided_harm": 1360, + "selected_both_correct": 4287, + "selected_both_wrong": 1980 + }, + { + "budget": 0.65, + "threshold": -8.49548625946045, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.7289338674537915, + "rescued": 1285, + "harmed": 1589, + "missed_rescue": 84, + "avoided_harm": 1332, + "selected_both_correct": 4382, + "selected_both_wrong": 1992 + }, + { + "budget": 0.66, + "threshold": -8.5530366897583, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.7272471712699417, + "rescued": 1290, + "harmed": 1618, + "missed_rescue": 79, + "avoided_harm": 1303, + "selected_both_correct": 4487, + "selected_both_wrong": 1996 + }, + { + "budget": 0.67, + "threshold": -8.610180854797363, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.7263335441703563, + "rescued": 1294, + "harmed": 1635, + "missed_rescue": 75, + "avoided_harm": 1286, + "selected_both_correct": 4602, + "selected_both_wrong": 2002 + }, + { + "budget": 0.68, + "threshold": -8.674345016479492, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.724225173940544, + "rescued": 1296, + "harmed": 1667, + "missed_rescue": 73, + "avoided_harm": 1254, + "selected_both_correct": 4703, + "selected_both_wrong": 2009 + }, + { + "budget": 0.69, + "threshold": -8.73198127746582, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.7227493147796753, + "rescued": 1303, + "harmed": 1695, + "missed_rescue": 66, + "avoided_harm": 1226, + "selected_both_correct": 4805, + "selected_both_wrong": 2015 + }, + { + "budget": 0.7, + "threshold": -8.787148475646973, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.7210626185958254, + "rescued": 1304, + "harmed": 1720, + "missed_rescue": 65, + "avoided_harm": 1201, + "selected_both_correct": 4914, + "selected_both_wrong": 2022 + }, + { + "budget": 0.71, + "threshold": -8.846220016479492, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.7190948063813339, + "rescued": 1307, + "harmed": 1751, + "missed_rescue": 62, + "avoided_harm": 1170, + "selected_both_correct": 5009, + "selected_both_wrong": 2035 + }, + { + "budget": 0.72, + "threshold": -8.907856941223145, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.7169864361515216, + "rescued": 1310, + "harmed": 1784, + "missed_rescue": 59, + "avoided_harm": 1137, + "selected_both_correct": 5109, + "selected_both_wrong": 2041 + }, + { + "budget": 0.73, + "threshold": -8.959864616394043, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.71501862393703, + "rescued": 1315, + "harmed": 1817, + "missed_rescue": 54, + "avoided_harm": 1104, + "selected_both_correct": 5205, + "selected_both_wrong": 2050 + }, + { + "budget": 0.74, + "threshold": -9.021697998046875, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.7132616487455197, + "rescued": 1318, + "harmed": 1845, + "missed_rescue": 51, + "avoided_harm": 1076, + "selected_both_correct": 5313, + "selected_both_wrong": 2053 + }, + { + "budget": 0.75, + "threshold": -9.08212661743164, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.7112938365310282, + "rescued": 1323, + "harmed": 1878, + "missed_rescue": 46, + "avoided_harm": 1043, + "selected_both_correct": 5412, + "selected_both_wrong": 2058 + }, + { + "budget": 0.76, + "threshold": -9.147140502929688, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.7094665823318574, + "rescued": 1326, + "harmed": 1907, + "missed_rescue": 43, + "avoided_harm": 1014, + "selected_both_correct": 5518, + "selected_both_wrong": 2063 + }, + { + "budget": 0.77, + "threshold": -9.211226463317871, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.7072879330943848, + "rescued": 1328, + "harmed": 1940, + "missed_rescue": 41, + "avoided_harm": 981, + "selected_both_correct": 5623, + "selected_both_wrong": 2065 + }, + { + "budget": 0.78, + "threshold": -9.274300575256348, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.7053201208798932, + "rescued": 1331, + "harmed": 1971, + "missed_rescue": 38, + "avoided_harm": 950, + "selected_both_correct": 5721, + "selected_both_wrong": 2075 + }, + { + "budget": 0.79, + "threshold": -9.347284317016602, + "operator": "gt", + "actual_fraction": 0.7898657670953686, + "metric": 0.7028603556117787, + "rescued": 1331, + "harmed": 2006, + "missed_rescue": 38, + "avoided_harm": 915, + "selected_both_correct": 5822, + "selected_both_wrong": 2080 + }, + { + "budget": 0.8, + "threshold": -9.402959823608398, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.7004005903436643, + "rescued": 1334, + "harmed": 2044, + "missed_rescue": 35, + "avoided_harm": 877, + "selected_both_correct": 5924, + "selected_both_wrong": 2081 + }, + { + "budget": 0.81, + "threshold": -9.468785285949707, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.6978705460678896, + "rescued": 1337, + "harmed": 2083, + "missed_rescue": 32, + "avoided_harm": 838, + "selected_both_correct": 6021, + "selected_both_wrong": 2084 + }, + { + "budget": 0.82, + "threshold": -9.52985954284668, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.6950593857614731, + "rescued": 1340, + "harmed": 2126, + "missed_rescue": 29, + "avoided_harm": 795, + "selected_both_correct": 6112, + "selected_both_wrong": 2089 + }, + { + "budget": 0.83, + "threshold": -9.608570098876953, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.6924590624780378, + "rescued": 1341, + "harmed": 2164, + "missed_rescue": 28, + "avoided_harm": 757, + "selected_both_correct": 6210, + "selected_both_wrong": 2095 + }, + { + "budget": 0.84, + "threshold": -9.684678077697754, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.6905615292712067, + "rescued": 1341, + "harmed": 2191, + "missed_rescue": 28, + "avoided_harm": 730, + "selected_both_correct": 6320, + "selected_both_wrong": 2100 + }, + { + "budget": 0.85, + "threshold": -9.755184173583984, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.6873989739264882, + "rescued": 1343, + "harmed": 2238, + "missed_rescue": 26, + "avoided_harm": 683, + "selected_both_correct": 6407, + "selected_both_wrong": 2106 + }, + { + "budget": 0.86, + "threshold": -9.839400291442871, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.6847283716353925, + "rescued": 1343, + "harmed": 2276, + "missed_rescue": 26, + "avoided_harm": 645, + "selected_both_correct": 6508, + "selected_both_wrong": 2109 + }, + { + "budget": 0.87, + "threshold": -9.916629791259766, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.6819874903366364, + "rescued": 1346, + "harmed": 2318, + "missed_rescue": 23, + "avoided_harm": 603, + "selected_both_correct": 6602, + "selected_both_wrong": 2113 + }, + { + "budget": 0.88, + "threshold": -10.001229286193848, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.6798088410991637, + "rescued": 1349, + "harmed": 2352, + "missed_rescue": 20, + "avoided_harm": 569, + "selected_both_correct": 6702, + "selected_both_wrong": 2118 + }, + { + "budget": 0.89, + "threshold": -10.100711822509766, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.6767165647621055, + "rescued": 1350, + "harmed": 2397, + "missed_rescue": 19, + "avoided_harm": 524, + "selected_both_correct": 6795, + "selected_both_wrong": 2121 + }, + { + "budget": 0.9, + "threshold": -10.199607849121094, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.6744676365169724, + "rescued": 1351, + "harmed": 2430, + "missed_rescue": 18, + "avoided_harm": 491, + "selected_both_correct": 6899, + "selected_both_wrong": 2126 + }, + { + "budget": 0.91, + "threshold": -10.293088912963867, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.6717970342258768, + "rescued": 1352, + "harmed": 2469, + "missed_rescue": 17, + "avoided_harm": 452, + "selected_both_correct": 6997, + "selected_both_wrong": 2130 + }, + { + "budget": 0.92, + "threshold": -10.41290283203125, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.6689858739194603, + "rescued": 1356, + "harmed": 2513, + "missed_rescue": 13, + "avoided_harm": 408, + "selected_both_correct": 7083, + "selected_both_wrong": 2138 + }, + { + "budget": 0.93, + "threshold": -10.520768165588379, + "operator": "gt", + "actual_fraction": 0.929861550354909, + "metric": 0.6666666666666666, + "rescued": 1358, + "harmed": 2548, + "missed_rescue": 11, + "avoided_harm": 373, + "selected_both_correct": 7181, + "selected_both_wrong": 2144 + }, + { + "budget": 0.94, + "threshold": -10.65395736694336, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.663644669337269, + "rescued": 1362, + "harmed": 2595, + "missed_rescue": 7, + "avoided_harm": 326, + "selected_both_correct": 7272, + "selected_both_wrong": 2146 + }, + { + "budget": 0.95, + "threshold": -10.800084114074707, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.6603415559772297, + "rescued": 1364, + "harmed": 2644, + "missed_rescue": 5, + "avoided_harm": 277, + "selected_both_correct": 7355, + "selected_both_wrong": 2154 + }, + { + "budget": 0.96, + "threshold": -10.987924575805664, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.6574601166631527, + "rescued": 1365, + "harmed": 2686, + "missed_rescue": 4, + "avoided_harm": 235, + "selected_both_correct": 7449, + "selected_both_wrong": 2159 + }, + { + "budget": 0.97, + "threshold": -11.214167594909668, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.6540164452877926, + "rescued": 1365, + "harmed": 2735, + "missed_rescue": 4, + "avoided_harm": 186, + "selected_both_correct": 7538, + "selected_both_wrong": 2164 + }, + { + "budget": 0.98, + "threshold": -11.459996223449707, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.6504322158971115, + "rescued": 1368, + "harmed": 2789, + "missed_rescue": 1, + "avoided_harm": 132, + "selected_both_correct": 7617, + "selected_both_wrong": 2170 + }, + { + "budget": 0.99, + "threshold": -11.947364807128906, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.6460046384145056, + "rescued": 1368, + "harmed": 2852, + "missed_rescue": 1, + "avoided_harm": 69, + "selected_both_correct": 7688, + "selected_both_wrong": 2178 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.6412256658935975, + "rescued": 1369, + "harmed": 2921, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 7755, + "selected_both_wrong": 2184 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 2627, + "avoided_harm": 8602, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.430821418762207, + "operator": "ge", + "actual_fraction": 0.04055242038397187, + "metric": 0.67373734509259, + "rescued": 158, + "harmed": 9, + "missed_rescue": 2469, + "avoided_harm": 8593, + "selected_both_correct": 13, + "selected_both_wrong": 1250 + }, + { + "budget": 0.02, + "threshold": 0.8396652936935425, + "operator": "ge", + "actual_fraction": 0.05498681337379122, + "metric": 0.6756089952641579, + "rescued": 250, + "harmed": 35, + "missed_rescue": 2377, + "avoided_harm": 8567, + "selected_both_correct": 54, + "selected_both_wrong": 1600 + }, + { + "budget": 0.03, + "threshold": -0.16024979948997498, + "operator": "ge", + "actual_fraction": 0.0680600062388339, + "metric": 0.6773104954201288, + "rescued": 339, + "harmed": 64, + "missed_rescue": 2288, + "avoided_harm": 8538, + "selected_both_correct": 90, + "selected_both_wrong": 1907 + }, + { + "budget": 0.04, + "threshold": -0.9098284244537354, + "operator": "ge", + "actual_fraction": 0.08110484076794373, + "metric": 0.6788134872245697, + "rescued": 431, + "harmed": 103, + "missed_rescue": 2196, + "avoided_harm": 8499, + "selected_both_correct": 146, + "selected_both_wrong": 2180 + }, + { + "budget": 0.05, + "threshold": -1.4125885963439941, + "operator": "ge", + "actual_fraction": 0.09125712503190313, + "metric": 0.6796075206306894, + "rescued": 498, + "harmed": 142, + "missed_rescue": 2129, + "avoided_harm": 8460, + "selected_both_correct": 194, + "selected_both_wrong": 2384 + }, + { + "budget": 0.06, + "threshold": -1.8043409585952759, + "operator": "ge", + "actual_fraction": 0.10106910926466835, + "metric": 0.6802597623571449, + "rescued": 561, + "harmed": 182, + "missed_rescue": 2066, + "avoided_harm": 8420, + "selected_both_correct": 234, + "selected_both_wrong": 2587 + }, + { + "budget": 0.07, + "threshold": -2.2280092239379883, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6812523041147945, + "rescued": 655, + "harmed": 241, + "missed_rescue": 1972, + "avoided_harm": 8361, + "selected_both_correct": 306, + "selected_both_wrong": 2863 + }, + { + "budget": 0.08, + "threshold": -2.5369863510131836, + "operator": "ge", + "actual_fraction": 0.12605280322150697, + "metric": 0.681167229106996, + "rescued": 713, + "harmed": 302, + "missed_rescue": 1914, + "avoided_harm": 8300, + "selected_both_correct": 378, + "selected_both_wrong": 3052 + }, + { + "budget": 0.09, + "threshold": -2.807616949081421, + "operator": "ge", + "actual_fraction": 0.13674389586819047, + "metric": 0.6818194708334515, + "rescued": 784, + "harmed": 350, + "missed_rescue": 1843, + "avoided_harm": 8252, + "selected_both_correct": 444, + "selected_both_wrong": 3244 + }, + { + "budget": 0.1, + "threshold": -3.0437393188476562, + "operator": "ge", + "actual_fraction": 0.14868275529591923, + "metric": 0.681535887474123, + "rescued": 843, + "harmed": 419, + "missed_rescue": 1784, + "avoided_harm": 8183, + "selected_both_correct": 513, + "selected_both_wrong": 3468 + }, + { + "budget": 0.11, + "threshold": -3.299111843109131, + "operator": "ge", + "actual_fraction": 0.1613305731219692, + "metric": 0.6816493208178545, + "rescued": 910, + "harmed": 482, + "missed_rescue": 1717, + "avoided_harm": 8120, + "selected_both_correct": 594, + "selected_both_wrong": 3703 + }, + { + "budget": 0.12, + "threshold": -3.5267717838287354, + "operator": "ge", + "actual_fraction": 0.17383659926835493, + "metric": 0.6807985707398689, + "rescued": 966, + "harmed": 568, + "missed_rescue": 1661, + "avoided_harm": 8034, + "selected_both_correct": 669, + "selected_both_wrong": 3927 + }, + { + "budget": 0.13, + "threshold": -3.745974540710449, + "operator": "ge", + "actual_fraction": 0.18634262541474067, + "metric": 0.6802881206930778, + "rescued": 1037, + "harmed": 657, + "missed_rescue": 1590, + "avoided_harm": 7945, + "selected_both_correct": 761, + "selected_both_wrong": 4116 + }, + { + "budget": 0.14, + "threshold": -3.9486825466156006, + "operator": "ge", + "actual_fraction": 0.19865014320959645, + "metric": 0.6797209539744208, + "rescued": 1108, + "harmed": 748, + "missed_rescue": 1519, + "avoided_harm": 7854, + "selected_both_correct": 842, + "selected_both_wrong": 4307 + }, + { + "budget": 0.15, + "threshold": -4.13905668258667, + "operator": "ge", + "actual_fraction": 0.21149646938717637, + "metric": 0.6788418455605025, + "rescued": 1175, + "harmed": 846, + "missed_rescue": 1452, + "avoided_harm": 7756, + "selected_both_correct": 934, + "selected_both_wrong": 4503 + }, + { + "budget": 0.16, + "threshold": -4.306515693664551, + "operator": "ge", + "actual_fraction": 0.2222726370416584, + "metric": 0.6784448288574426, + "rescued": 1233, + "harmed": 918, + "missed_rescue": 1394, + "avoided_harm": 7684, + "selected_both_correct": 1022, + "selected_both_wrong": 4665 + }, + { + "budget": 0.17, + "threshold": -4.4712700843811035, + "operator": "ge", + "actual_fraction": 0.23472194651617842, + "metric": 0.6770836287326659, + "rescued": 1283, + "harmed": 1016, + "missed_rescue": 1344, + "avoided_harm": 7586, + "selected_both_correct": 1129, + "selected_both_wrong": 4849 + }, + { + "budget": 0.18, + "threshold": -4.667325019836426, + "operator": "ge", + "actual_fraction": 0.24966678955278904, + "metric": 0.6750701868814338, + "rescued": 1337, + "harmed": 1141, + "missed_rescue": 1290, + "avoided_harm": 7461, + "selected_both_correct": 1257, + "selected_both_wrong": 5069 + }, + { + "budget": 0.19, + "threshold": -4.808548450469971, + "operator": "gt", + "actual_fraction": 0.2611235572696594, + "metric": 0.6740492867878513, + "rescued": 1394, + "harmed": 1234, + "missed_rescue": 1233, + "avoided_harm": 7368, + "selected_both_correct": 1363, + "selected_both_wrong": 5217 + }, + { + "budget": 0.2, + "threshold": -4.945236682891846, + "operator": "ge", + "actual_fraction": 0.27226838329126846, + "metric": 0.6728298783427389, + "rescued": 1441, + "harmed": 1324, + "missed_rescue": 1186, + "avoided_harm": 7278, + "selected_both_correct": 1462, + "selected_both_wrong": 5374 + }, + { + "budget": 0.21, + "threshold": -5.0980048179626465, + "operator": "ge", + "actual_fraction": 0.28545500950004254, + "metric": 0.6707030031477753, + "rescued": 1484, + "harmed": 1442, + "missed_rescue": 1143, + "avoided_harm": 7160, + "selected_both_correct": 1602, + "selected_both_wrong": 5538 + }, + { + "budget": 0.22, + "threshold": -5.24181604385376, + "operator": "ge", + "actual_fraction": 0.29620281881859173, + "metric": 0.6688313529762073, + "rescued": 1521, + "harmed": 1545, + "missed_rescue": 1106, + "avoided_harm": 7057, + "selected_both_correct": 1712, + "selected_both_wrong": 5667 + }, + { + "budget": 0.23, + "threshold": -5.381783485412598, + "operator": "ge", + "actual_fraction": 0.30904914499617164, + "metric": 0.6671865694921022, + "rescued": 1570, + "harmed": 1652, + "missed_rescue": 1057, + "avoided_harm": 6950, + "selected_both_correct": 1879, + "selected_both_wrong": 5797 + }, + { + "budget": 0.24, + "threshold": -5.480440616607666, + "operator": "ge", + "actual_fraction": 0.31880441255707115, + "metric": 0.6657119360235941, + "rescued": 1608, + "harmed": 1742, + "missed_rescue": 1019, + "avoided_harm": 6860, + "selected_both_correct": 1986, + "selected_both_wrong": 5906 + }, + { + "budget": 0.25, + "threshold": -5.592668533325195, + "operator": "ge", + "actual_fraction": 0.3278223633837166, + "metric": 0.6640104358676233, + "rescued": 1630, + "harmed": 1824, + "missed_rescue": 997, + "avoided_harm": 6778, + "selected_both_correct": 2094, + "selected_both_wrong": 6012 + }, + { + "budget": 0.26, + "threshold": -5.701513290405273, + "operator": "ge", + "actual_fraction": 0.3389388310693928, + "metric": 0.6622805773757195, + "rescued": 1667, + "harmed": 1922, + "missed_rescue": 960, + "avoided_harm": 6680, + "selected_both_correct": 2235, + "selected_both_wrong": 6128 + }, + { + "budget": 0.27, + "threshold": -5.817866802215576, + "operator": "ge", + "actual_fraction": 0.35082097382525596, + "metric": 0.6600402688370246, + "rescued": 1699, + "harmed": 2033, + "missed_rescue": 928, + "avoided_harm": 6569, + "selected_both_correct": 2400, + "selected_both_wrong": 6239 + }, + { + "budget": 0.28, + "threshold": -5.915761947631836, + "operator": "ge", + "actual_fraction": 0.36142699146414087, + "metric": 0.658111901993591, + "rescued": 1731, + "harmed": 2133, + "missed_rescue": 896, + "avoided_harm": 6469, + "selected_both_correct": 2541, + "selected_both_wrong": 6340 + }, + { + "budget": 0.29, + "threshold": -6.032350063323975, + "operator": "ge", + "actual_fraction": 0.37316734254033973, + "metric": 0.6560133851345603, + "rescued": 1770, + "harmed": 2246, + "missed_rescue": 857, + "avoided_harm": 6356, + "selected_both_correct": 2702, + "selected_both_wrong": 6441 + }, + { + "budget": 0.3, + "threshold": -6.134230136871338, + "operator": "ge", + "actual_fraction": 0.38351813515582905, + "metric": 0.6541417349629923, + "rescued": 1793, + "harmed": 2335, + "missed_rescue": 834, + "avoided_harm": 6267, + "selected_both_correct": 2873, + "selected_both_wrong": 6523 + }, + { + "budget": 0.31, + "threshold": -6.218080520629883, + "operator": "ge", + "actual_fraction": 0.39270623599807164, + "metric": 0.6522417264554916, + "rescued": 1815, + "harmed": 2424, + "missed_rescue": 812, + "avoided_harm": 6178, + "selected_both_correct": 3002, + "selected_both_wrong": 6607 + }, + { + "budget": 0.32, + "threshold": -6.302128791809082, + "operator": "ge", + "actual_fraction": 0.4026032952386354, + "metric": 0.6502282846042594, + "rescued": 1845, + "harmed": 2525, + "missed_rescue": 782, + "avoided_harm": 6077, + "selected_both_correct": 3150, + "selected_both_wrong": 6677 + }, + { + "budget": 0.33, + "threshold": -6.377316474914551, + "operator": "ge", + "actual_fraction": 0.4103167626123699, + "metric": 0.6486685761279528, + "rescued": 1864, + "harmed": 2599, + "missed_rescue": 763, + "avoided_harm": 6003, + "selected_both_correct": 3271, + "selected_both_wrong": 6735 + }, + { + "budget": 0.34, + "threshold": -6.449829578399658, + "operator": "ge", + "actual_fraction": 0.4194481467827468, + "metric": 0.6470805093157134, + "rescued": 1889, + "harmed": 2680, + "missed_rescue": 738, + "avoided_harm": 5922, + "selected_both_correct": 3412, + "selected_both_wrong": 6810 + }, + { + "budget": 0.35, + "threshold": -6.539102077484131, + "operator": "ge", + "actual_fraction": 0.4308198394918186, + "metric": 0.6448685591129513, + "rescued": 1916, + "harmed": 2785, + "missed_rescue": 711, + "avoided_harm": 5817, + "selected_both_correct": 3593, + "selected_both_wrong": 6898 + }, + { + "budget": 0.36, + "threshold": -6.614948272705078, + "operator": "ge", + "actual_fraction": 0.4392139069279415, + "metric": 0.6431387006210476, + "rescued": 1935, + "harmed": 2865, + "missed_rescue": 692, + "avoided_harm": 5737, + "selected_both_correct": 3729, + "selected_both_wrong": 6959 + }, + { + "budget": 0.37, + "threshold": -6.686672210693359, + "operator": "ge", + "actual_fraction": 0.4474661826844001, + "metric": 0.6415222754728752, + "rescued": 1954, + "harmed": 2941, + "missed_rescue": 673, + "avoided_harm": 5661, + "selected_both_correct": 3881, + "selected_both_wrong": 7003 + }, + { + "budget": 0.38, + "threshold": -6.78654146194458, + "operator": "ge", + "actual_fraction": 0.46124833394776393, + "metric": 0.6382043501687321, + "rescued": 1979, + "harmed": 3083, + "missed_rescue": 648, + "avoided_harm": 5519, + "selected_both_correct": 4094, + "selected_both_wrong": 7109 + }, + { + "budget": 0.39, + "threshold": -6.846216201782227, + "operator": "ge", + "actual_fraction": 0.46918866800896125, + "metric": 0.6371550917392167, + "rescued": 1999, + "harmed": 3140, + "missed_rescue": 628, + "avoided_harm": 5462, + "selected_both_correct": 4243, + "selected_both_wrong": 7163 + }, + { + "budget": 0.4, + "threshold": -6.915761470794678, + "operator": "ge", + "actual_fraction": 0.47786631880441255, + "metric": 0.6349714998723874, + "rescued": 2015, + "harmed": 3233, + "missed_rescue": 612, + "avoided_harm": 5369, + "selected_both_correct": 4390, + "selected_both_wrong": 7213 + }, + { + "budget": 0.41, + "threshold": -6.9817047119140625, + "operator": "ge", + "actual_fraction": 0.48657232793579674, + "metric": 0.6325043246462297, + "rescued": 2033, + "harmed": 3338, + "missed_rescue": 594, + "avoided_harm": 5264, + "selected_both_correct": 4523, + "selected_both_wrong": 7264 + }, + { + "budget": 0.42, + "threshold": -7.047516822814941, + "operator": "gt", + "actual_fraction": 0.4953634120749794, + "metric": 0.6308878994980575, + "rescued": 2055, + "harmed": 3417, + "missed_rescue": 572, + "avoided_harm": 5185, + "selected_both_correct": 4686, + "selected_both_wrong": 7310 + }, + { + "budget": 0.43, + "threshold": -7.1213059425354, + "operator": "ge", + "actual_fraction": 0.5059694297138644, + "metric": 0.6287610243030939, + "rescued": 2086, + "harmed": 3523, + "missed_rescue": 541, + "avoided_harm": 5079, + "selected_both_correct": 4852, + "selected_both_wrong": 7381 + }, + { + "budget": 0.44, + "threshold": -7.181412696838379, + "operator": "ge", + "actual_fraction": 0.51439185548592, + "metric": 0.6270311658111902, + "rescued": 2107, + "harmed": 3605, + "missed_rescue": 520, + "avoided_harm": 4997, + "selected_both_correct": 5002, + "selected_both_wrong": 7425 + }, + { + "budget": 0.45, + "threshold": -7.25251579284668, + "operator": "ge", + "actual_fraction": 0.524629214757678, + "metric": 0.6243087655616368, + "rescued": 2122, + "harmed": 3716, + "missed_rescue": 505, + "avoided_harm": 4886, + "selected_both_correct": 5184, + "selected_both_wrong": 7478 + }, + { + "budget": 0.46, + "threshold": -7.320802211761475, + "operator": "ge", + "actual_fraction": 0.5349232907013016, + "metric": 0.6222669653744718, + "rescued": 2145, + "harmed": 3811, + "missed_rescue": 482, + "avoided_harm": 4791, + "selected_both_correct": 5373, + "selected_both_wrong": 7534 + }, + { + "budget": 0.47, + "threshold": -7.374159812927246, + "operator": "ge", + "actual_fraction": 0.5420412330204464, + "metric": 0.620934123585628, + "rescued": 2158, + "harmed": 3871, + "missed_rescue": 469, + "avoided_harm": 4731, + "selected_both_correct": 5508, + "selected_both_wrong": 7577 + }, + { + "budget": 0.48, + "threshold": -7.4538164138793945, + "operator": "ge", + "actual_fraction": 0.5530159090264584, + "metric": 0.6183818733516717, + "rescued": 2178, + "harmed": 3981, + "missed_rescue": 449, + "avoided_harm": 4621, + "selected_both_correct": 5711, + "selected_both_wrong": 7631 + }, + { + "budget": 0.49, + "threshold": -7.532625675201416, + "operator": "ge", + "actual_fraction": 0.5640189433684032, + "metric": 0.6162549981567081, + "rescued": 2202, + "harmed": 4080, + "missed_rescue": 425, + "avoided_harm": 4522, + "selected_both_correct": 5928, + "selected_both_wrong": 7679 + }, + { + "budget": 0.5, + "threshold": -7.600386619567871, + "operator": "ge", + "actual_fraction": 0.5738876442730341, + "metric": 0.6134475228993562, + "rescued": 2214, + "harmed": 4191, + "missed_rescue": 413, + "avoided_harm": 4411, + "selected_both_correct": 6117, + "selected_both_wrong": 7715 + }, + { + "budget": 0.51, + "threshold": -7.662148952484131, + "operator": "ge", + "actual_fraction": 0.5832458951308738, + "metric": 0.6109519893372657, + "rescued": 2221, + "harmed": 4286, + "missed_rescue": 406, + "avoided_harm": 4316, + "selected_both_correct": 6292, + "selected_both_wrong": 7768 + }, + { + "budget": 0.52, + "threshold": -7.733202934265137, + "operator": "ge", + "actual_fraction": 0.5937101210900945, + "metric": 0.6086549641267051, + "rescued": 2242, + "harmed": 4388, + "missed_rescue": 385, + "avoided_harm": 4214, + "selected_both_correct": 6483, + "selected_both_wrong": 7823 + }, + { + "budget": 0.53, + "threshold": -7.79387092590332, + "operator": "ge", + "actual_fraction": 0.6031250886197997, + "metric": 0.6062728639083459, + "rescued": 2260, + "harmed": 4490, + "missed_rescue": 367, + "avoided_harm": 4112, + "selected_both_correct": 6652, + "selected_both_wrong": 7866 + }, + { + "budget": 0.54, + "threshold": -7.855954647064209, + "operator": "ge", + "actual_fraction": 0.611263931032527, + "metric": 0.6044579304086436, + "rescued": 2271, + "harmed": 4565, + "missed_rescue": 356, + "avoided_harm": 4037, + "selected_both_correct": 6810, + "selected_both_wrong": 7909 + }, + { + "budget": 0.55, + "threshold": -7.925965785980225, + "operator": "ge", + "actual_fraction": 0.6218983070073448, + "metric": 0.6018489635028217, + "rescued": 2283, + "harmed": 4669, + "missed_rescue": 344, + "avoided_harm": 3933, + "selected_both_correct": 7013, + "selected_both_wrong": 7965 + }, + { + "budget": 0.56, + "threshold": -7.977593421936035, + "operator": "ge", + "actual_fraction": 0.6296117743810793, + "metric": 0.5999489549953209, + "rescued": 2291, + "harmed": 4744, + "missed_rescue": 336, + "avoided_harm": 3858, + "selected_both_correct": 7173, + "selected_both_wrong": 7994 + }, + { + "budget": 0.57, + "threshold": -8.031231880187988, + "operator": "ge", + "actual_fraction": 0.6381192751609336, + "metric": 0.5979638714800215, + "rescued": 2306, + "harmed": 4829, + "missed_rescue": 321, + "avoided_harm": 3773, + "selected_both_correct": 7339, + "selected_both_wrong": 8028 + }, + { + "budget": 0.58, + "threshold": -8.088851928710938, + "operator": "ge", + "actual_fraction": 0.6469387176360492, + "metric": 0.5955534129257295, + "rescued": 2318, + "harmed": 4926, + "missed_rescue": 309, + "avoided_harm": 3676, + "selected_both_correct": 7520, + "selected_both_wrong": 8049 + }, + { + "budget": 0.59, + "threshold": -8.145581245422363, + "operator": "ge", + "actual_fraction": 0.654935768369112, + "metric": 0.5933414627229674, + "rescued": 2329, + "harmed": 5015, + "missed_rescue": 298, + "avoided_harm": 3587, + "selected_both_correct": 7677, + "selected_both_wrong": 8074 + }, + { + "budget": 0.6, + "threshold": -8.199271202087402, + "operator": "ge", + "actual_fraction": 0.6636701358364291, + "metric": 0.5907608541530783, + "rescued": 2339, + "harmed": 5116, + "missed_rescue": 288, + "avoided_harm": 3486, + "selected_both_correct": 7851, + "selected_both_wrong": 8097 + }, + { + "budget": 0.61, + "threshold": -8.256582260131836, + "operator": "ge", + "actual_fraction": 0.6710149448430366, + "metric": 0.5888324873096447, + "rescued": 2347, + "harmed": 5192, + "missed_rescue": 280, + "avoided_harm": 3410, + "selected_both_correct": 8006, + "selected_both_wrong": 8117 + }, + { + "budget": 0.62, + "threshold": -8.316590309143066, + "operator": "ge", + "actual_fraction": 0.6805433457164733, + "metric": 0.5859966537163599, + "rescued": 2355, + "harmed": 5300, + "missed_rescue": 272, + "avoided_harm": 3302, + "selected_both_correct": 8192, + "selected_both_wrong": 8151 + }, + { + "budget": 0.63, + "threshold": -8.380763053894043, + "operator": "ge", + "actual_fraction": 0.6900150299180444, + "metric": 0.5836712701698664, + "rescued": 2370, + "harmed": 5397, + "missed_rescue": 257, + "avoided_harm": 3205, + "selected_both_correct": 8387, + "selected_both_wrong": 8178 + }, + { + "budget": 0.64, + "threshold": -8.436734199523926, + "operator": "gt", + "actual_fraction": 0.6979837223151746, + "metric": 0.5813458866233729, + "rescued": 2380, + "harmed": 5489, + "missed_rescue": 247, + "avoided_harm": 3113, + "selected_both_correct": 8531, + "selected_both_wrong": 8213 + }, + { + "budget": 0.65, + "threshold": -8.49548625946045, + "operator": "ge", + "actual_fraction": 0.7061225647279018, + "metric": 0.579105578084678, + "rescued": 2387, + "harmed": 5575, + "missed_rescue": 240, + "avoided_harm": 3027, + "selected_both_correct": 8704, + "selected_both_wrong": 8234 + }, + { + "budget": 0.66, + "threshold": -8.5530366897583, + "operator": "ge", + "actual_fraction": 0.7130703570314494, + "metric": 0.5778578113036327, + "rescued": 2398, + "harmed": 5630, + "missed_rescue": 229, + "avoided_harm": 2972, + "selected_both_correct": 8865, + "selected_both_wrong": 8252 + }, + { + "budget": 0.67, + "threshold": -8.610180854797363, + "operator": "ge", + "actual_fraction": 0.7218614411706321, + "metric": 0.5755891444290049, + "rescued": 2408, + "harmed": 5720, + "missed_rescue": 219, + "avoided_harm": 2882, + "selected_both_correct": 9038, + "selected_both_wrong": 8289 + }, + { + "budget": 0.68, + "threshold": -8.674345016479492, + "operator": "ge", + "actual_fraction": 0.7307092419816805, + "metric": 0.5725548024841902, + "rescued": 2415, + "harmed": 5834, + "missed_rescue": 212, + "avoided_harm": 2768, + "selected_both_correct": 9193, + "selected_both_wrong": 8325 + }, + { + "budget": 0.69, + "threshold": -8.73198127746582, + "operator": "ge", + "actual_fraction": 0.7391033094178033, + "metric": 0.5702861356095624, + "rescued": 2426, + "harmed": 5925, + "missed_rescue": 201, + "avoided_harm": 2677, + "selected_both_correct": 9357, + "selected_both_wrong": 8355 + }, + { + "budget": 0.7, + "threshold": -8.787148475646973, + "operator": "ge", + "actual_fraction": 0.7464481184244108, + "metric": 0.5685279187817259, + "rescued": 2432, + "harmed": 5993, + "missed_rescue": 195, + "avoided_harm": 2609, + "selected_both_correct": 9512, + "selected_both_wrong": 8385 + }, + { + "budget": 0.71, + "threshold": -8.846220016479492, + "operator": "ge", + "actual_fraction": 0.7537078524232198, + "metric": 0.5663726852508295, + "rescued": 2438, + "harmed": 6075, + "missed_rescue": 189, + "avoided_harm": 2527, + "selected_both_correct": 9662, + "selected_both_wrong": 8403 + }, + { + "budget": 0.72, + "threshold": -8.907856941223145, + "operator": "ge", + "actual_fraction": 0.7623855032186712, + "metric": 0.5639622266965374, + "rescued": 2445, + "harmed": 6167, + "missed_rescue": 182, + "avoided_harm": 2435, + "selected_both_correct": 9844, + "selected_both_wrong": 8428 + }, + { + "budget": 0.73, + "threshold": -8.959864616394043, + "operator": "ge", + "actual_fraction": 0.7701556872642713, + "metric": 0.5618637098375068, + "rescued": 2455, + "harmed": 6251, + "missed_rescue": 172, + "avoided_harm": 2351, + "selected_both_correct": 10001, + "selected_both_wrong": 8451 + }, + { + "budget": 0.74, + "threshold": -9.021697998046875, + "operator": "ge", + "actual_fraction": 0.7780393046536029, + "metric": 0.5599069846581403, + "rescued": 2463, + "harmed": 6328, + "missed_rescue": 164, + "avoided_harm": 2274, + "selected_both_correct": 10172, + "selected_both_wrong": 8473 + }, + { + "budget": 0.75, + "threshold": -9.08212661743164, + "operator": "ge", + "actual_fraction": 0.7862348637381958, + "metric": 0.5578651844709752, + "rescued": 2472, + "harmed": 6409, + "missed_rescue": 155, + "avoided_harm": 2193, + "selected_both_correct": 10346, + "selected_both_wrong": 8498 + }, + { + "budget": 0.76, + "threshold": -9.147140502929688, + "operator": "ge", + "actual_fraction": 0.7948557978617815, + "metric": 0.5555965175963474, + "rescued": 2478, + "harmed": 6495, + "missed_rescue": 149, + "avoided_harm": 2107, + "selected_both_correct": 10525, + "selected_both_wrong": 8531 + }, + { + "budget": 0.77, + "threshold": -9.211226463317871, + "operator": "ge", + "actual_fraction": 0.8029662819385759, + "metric": 0.5536681507529139, + "rescued": 2489, + "harmed": 6574, + "missed_rescue": 138, + "avoided_harm": 2028, + "selected_both_correct": 10712, + "selected_both_wrong": 8540 + }, + { + "budget": 0.78, + "threshold": -9.274300575256348, + "operator": "ge", + "actual_fraction": 0.8114737827184301, + "metric": 0.5513144088704874, + "rescued": 2498, + "harmed": 6666, + "missed_rescue": 129, + "avoided_harm": 1936, + "selected_both_correct": 10890, + "selected_both_wrong": 8561 + }, + { + "budget": 0.79, + "threshold": -9.347284317016602, + "operator": "gt", + "actual_fraction": 0.8210872585996654, + "metric": 0.5488188753083969, + "rescued": 2507, + "harmed": 6763, + "missed_rescue": 120, + "avoided_harm": 1839, + "selected_both_correct": 11095, + "selected_both_wrong": 8589 + }, + { + "budget": 0.8, + "threshold": -9.402959823608398, + "operator": "ge", + "actual_fraction": 0.8279499758954144, + "metric": 0.5472591668320903, + "rescued": 2518, + "harmed": 6829, + "missed_rescue": 109, + "avoided_harm": 1773, + "selected_both_correct": 11239, + "selected_both_wrong": 8610 + }, + { + "budget": 0.81, + "threshold": -9.468785285949707, + "operator": "ge", + "actual_fraction": 0.8352664265660891, + "metric": 0.5455293083401866, + "rescued": 2521, + "harmed": 6893, + "missed_rescue": 106, + "avoided_harm": 1709, + "selected_both_correct": 11407, + "selected_both_wrong": 8633 + }, + { + "budget": 0.82, + "threshold": -9.52985954284668, + "operator": "ge", + "actual_fraction": 0.8424978022289652, + "metric": 0.5440546748716786, + "rescued": 2526, + "harmed": 6950, + "missed_rescue": 101, + "avoided_harm": 1652, + "selected_both_correct": 11577, + "selected_both_wrong": 8656 + }, + { + "budget": 0.83, + "threshold": -9.608570098876953, + "operator": "ge", + "actual_fraction": 0.850891869665088, + "metric": 0.5418143663329836, + "rescued": 2528, + "harmed": 7031, + "missed_rescue": 99, + "avoided_harm": 1571, + "selected_both_correct": 11770, + "selected_both_wrong": 8676 + }, + { + "budget": 0.84, + "threshold": -9.684678077697754, + "operator": "ge", + "actual_fraction": 0.8593710121090095, + "metric": 0.53948898278649, + "rescued": 2534, + "harmed": 7119, + "missed_rescue": 93, + "avoided_harm": 1483, + "selected_both_correct": 11952, + "selected_both_wrong": 8699 + }, + { + "budget": 0.85, + "threshold": -9.755184173583984, + "operator": "ge", + "actual_fraction": 0.8669994044749454, + "metric": 0.5374188242633923, + "rescued": 2538, + "harmed": 7196, + "missed_rescue": 89, + "avoided_harm": 1406, + "selected_both_correct": 12113, + "selected_both_wrong": 8726 + }, + { + "budget": 0.86, + "threshold": -9.839400291442871, + "operator": "ge", + "actual_fraction": 0.877293480418569, + "metric": 0.5350934407168987, + "rescued": 2549, + "harmed": 7289, + "missed_rescue": 78, + "avoided_harm": 1313, + "selected_both_correct": 12347, + "selected_both_wrong": 8751 + }, + { + "budget": 0.87, + "threshold": -9.916629791259766, + "operator": "ge", + "actual_fraction": 0.8848651561126393, + "metric": 0.5330799988656666, + "rescued": 2555, + "harmed": 7366, + "missed_rescue": 72, + "avoided_harm": 1236, + "selected_both_correct": 12510, + "selected_both_wrong": 8772 + }, + { + "budget": 0.88, + "threshold": -10.001229286193848, + "operator": "ge", + "actual_fraction": 0.893089073533165, + "metric": 0.5305561069676431, + "rescued": 2559, + "harmed": 7459, + "missed_rescue": 68, + "avoided_harm": 1143, + "selected_both_correct": 12685, + "selected_both_wrong": 8790 + }, + { + "budget": 0.89, + "threshold": -10.100711822509766, + "operator": "ge", + "actual_fraction": 0.901653290984885, + "metric": 0.5283725151008138, + "rescued": 2567, + "harmed": 7544, + "missed_rescue": 60, + "avoided_harm": 1058, + "selected_both_correct": 12873, + "selected_both_wrong": 8811 + }, + { + "budget": 0.9, + "threshold": -10.199607849121094, + "operator": "ge", + "actual_fraction": 0.9106996001474633, + "metric": 0.5261889232339846, + "rescued": 2572, + "harmed": 7626, + "missed_rescue": 55, + "avoided_harm": 976, + "selected_both_correct": 13080, + "selected_both_wrong": 8836 + }, + { + "budget": 0.91, + "threshold": -10.293088912963867, + "operator": "ge", + "actual_fraction": 0.9177324674588095, + "metric": 0.5241754813827525, + "rescued": 2575, + "harmed": 7700, + "missed_rescue": 52, + "avoided_harm": 902, + "selected_both_correct": 13240, + "selected_both_wrong": 8847 + }, + { + "budget": 0.92, + "threshold": -10.41290283203125, + "operator": "ge", + "actual_fraction": 0.9268638516291864, + "metric": 0.5215948728128633, + "rescued": 2580, + "harmed": 7796, + "missed_rescue": 47, + "avoided_harm": 806, + "selected_both_correct": 13438, + "selected_both_wrong": 8870 + }, + { + "budget": 0.93, + "threshold": -10.520768165588379, + "operator": "gt", + "actual_fraction": 0.9340668689561297, + "metric": 0.5198650143209597, + "rescued": 2589, + "harmed": 7866, + "missed_rescue": 38, + "avoided_harm": 736, + "selected_both_correct": 13595, + "selected_both_wrong": 8888 + }, + { + "budget": 0.94, + "threshold": -10.65395736694336, + "operator": "ge", + "actual_fraction": 0.9422907863766554, + "metric": 0.5175963474463319, + "rescued": 2594, + "harmed": 7951, + "missed_rescue": 33, + "avoided_harm": 651, + "selected_both_correct": 13777, + "selected_both_wrong": 8906 + }, + { + "budget": 0.95, + "threshold": -10.800084114074707, + "operator": "ge", + "actual_fraction": 0.9508266454924426, + "metric": 0.5152426055639056, + "rescued": 2600, + "harmed": 8040, + "missed_rescue": 27, + "avoided_harm": 562, + "selected_both_correct": 13959, + "selected_both_wrong": 8930 + }, + { + "budget": 0.96, + "threshold": -10.987924575805664, + "operator": "ge", + "actual_fraction": 0.9601281796784165, + "metric": 0.5127187136658821, + "rescued": 2612, + "harmed": 8141, + "missed_rescue": 15, + "avoided_harm": 461, + "selected_both_correct": 14155, + "selected_both_wrong": 8949 + }, + { + "budget": 0.97, + "threshold": -11.214167594909668, + "operator": "ge", + "actual_fraction": 0.9683237387630094, + "metric": 0.509939596744463, + "rescued": 2614, + "harmed": 8241, + "missed_rescue": 13, + "avoided_harm": 361, + "selected_both_correct": 14328, + "selected_both_wrong": 8963 + }, + { + "budget": 0.98, + "threshold": -11.459996223449707, + "operator": "ge", + "actual_fraction": 0.976944672886595, + "metric": 0.506791821455917, + "rescued": 2617, + "harmed": 8355, + "missed_rescue": 10, + "avoided_harm": 247, + "selected_both_correct": 14505, + "selected_both_wrong": 8973 + }, + { + "budget": 0.99, + "threshold": -11.947364807128906, + "operator": "ge", + "actual_fraction": 0.9883163655956668, + "metric": 0.5033604628080425, + "rescued": 2622, + "harmed": 8481, + "missed_rescue": 5, + "avoided_harm": 121, + "selected_both_correct": 14753, + "selected_both_wrong": 8995 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.5000708958398321, + "rescued": 2627, + "harmed": 8602, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 15007, + "selected_both_wrong": 9027 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 7755, + "rescuable": 1369, + "harmful": 2921, + "both_wrong": 2184, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 15007, + "rescuable": 2627, + "harmful": 8602, + "both_wrong": 9027, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.7502986857825568, + "test": 0.6695119530385957 + }, + "large": { + "validation": 0.6412256658935975, + "test": 0.5000708958398321 + }, + "learned_minus_confidence_test": 0.048407679437370565, + "learned_minus_small_test": -0.002325383546493476, + "learned_minus_large_test": 0.1671156736522701 + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "expert_display": "YOLO26x", + "run": "shared_12_yolo26x_Qwen3.8", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 93, + "validation_area_0_25": 0.17942553980644182, + "validation_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + "test_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 498, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 526, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.17237009889855048, + "rescued": 0, + "harmed": 36, + "missed_rescue": 307, + "avoided_harm": 184, + "selected_both_correct": 552, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.17209204025345443, + "rescued": 0, + "harmed": 37, + "missed_rescue": 307, + "avoided_harm": 183, + "selected_both_correct": 578, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.1716297194027841, + "rescued": 0, + "harmed": 39, + "missed_rescue": 307, + "avoided_harm": 181, + "selected_both_correct": 604, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 629, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24821683309557774, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 652, + "selected_both_wrong": 2 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 681, + "selected_both_wrong": 2 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.268188302425107, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 705, + "selected_both_wrong": 2 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 734, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 762, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.16880965944961934, + "rescued": 0, + "harmed": 51, + "missed_rescue": 307, + "avoided_harm": 169, + "selected_both_correct": 787, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30813124108416545, + "metric": 0.16830460894456886, + "rescued": 0, + "harmed": 56, + "missed_rescue": 307, + "avoided_harm": 164, + "selected_both_correct": 806, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3181169757489301, + "metric": 0.16811053970066264, + "rescued": 0, + "harmed": 57, + "missed_rescue": 307, + "avoided_harm": 163, + "selected_both_correct": 833, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.1671811299174608, + "rescued": 0, + "harmed": 61, + "missed_rescue": 307, + "avoided_harm": 159, + "selected_both_correct": 861, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.16645634059427164, + "rescued": 0, + "harmed": 64, + "missed_rescue": 307, + "avoided_harm": 156, + "selected_both_correct": 886, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34807417974322397, + "metric": 0.16600354762332126, + "rescued": 0, + "harmed": 66, + "missed_rescue": 307, + "avoided_harm": 154, + "selected_both_correct": 908, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.16502549737843858, + "rescued": 0, + "harmed": 70, + "missed_rescue": 307, + "avoided_harm": 150, + "selected_both_correct": 936, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3666191155492154, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 72, + "missed_rescue": 307, + "avoided_harm": 148, + "selected_both_correct": 954, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37660485021398005, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 76, + "missed_rescue": 307, + "avoided_harm": 144, + "selected_both_correct": 978, + "selected_both_wrong": 2 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.16749463738508683, + "rescued": 1, + "harmed": 81, + "missed_rescue": 306, + "avoided_harm": 139, + "selected_both_correct": 1008, + "selected_both_wrong": 2 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.16607474321791724, + "rescued": 1, + "harmed": 87, + "missed_rescue": 306, + "avoided_harm": 133, + "selected_both_correct": 1030, + "selected_both_wrong": 2 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.16514206572141965, + "rescued": 1, + "harmed": 90, + "missed_rescue": 306, + "avoided_harm": 130, + "selected_both_correct": 1055, + "selected_both_wrong": 2 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.16444252141114554, + "rescued": 1, + "harmed": 93, + "missed_rescue": 306, + "avoided_harm": 127, + "selected_both_correct": 1080, + "selected_both_wrong": 2 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.1639139801849299, + "rescued": 1, + "harmed": 96, + "missed_rescue": 306, + "avoided_harm": 124, + "selected_both_correct": 1105, + "selected_both_wrong": 2 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.16237039262040495, + "rescued": 1, + "harmed": 103, + "missed_rescue": 306, + "avoided_harm": 117, + "selected_both_correct": 1126, + "selected_both_wrong": 2 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.16187473569847557, + "rescued": 1, + "harmed": 105, + "missed_rescue": 306, + "avoided_harm": 115, + "selected_both_correct": 1152, + "selected_both_wrong": 2 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.1640341119762699, + "rescued": 2, + "harmed": 110, + "missed_rescue": 305, + "avoided_harm": 110, + "selected_both_correct": 1174, + "selected_both_wrong": 2 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.46790299572039945, + "metric": 0.16656779979633995, + "rescued": 3, + "harmed": 113, + "missed_rescue": 304, + "avoided_harm": 107, + "selected_both_correct": 1194, + "selected_both_wrong": 2 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.16532395129425315, + "rescued": 3, + "harmed": 119, + "missed_rescue": 304, + "avoided_harm": 101, + "selected_both_correct": 1220, + "selected_both_wrong": 2 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.16467625831496033, + "rescued": 5, + "harmed": 123, + "missed_rescue": 302, + "avoided_harm": 97, + "selected_both_correct": 1241, + "selected_both_wrong": 3 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.16569589650413527, + "rescued": 11, + "harmed": 128, + "missed_rescue": 296, + "avoided_harm": 92, + "selected_both_correct": 1257, + "selected_both_wrong": 4 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.1671927745457157, + "rescued": 18, + "harmed": 129, + "missed_rescue": 289, + "avoided_harm": 91, + "selected_both_correct": 1276, + "selected_both_wrong": 5 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5178316690442225, + "metric": 0.16789087940773334, + "rescued": 23, + "harmed": 131, + "missed_rescue": 284, + "avoided_harm": 89, + "selected_both_correct": 1293, + "selected_both_wrong": 5 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.16946103210148153, + "rescued": 30, + "harmed": 132, + "missed_rescue": 277, + "avoided_harm": 88, + "selected_both_correct": 1316, + "selected_both_wrong": 6 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5378031383737518, + "metric": 0.17052688690619724, + "rescued": 35, + "harmed": 133, + "missed_rescue": 272, + "avoided_harm": 87, + "selected_both_correct": 1334, + "selected_both_wrong": 6 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.17103667850460152, + "rescued": 42, + "harmed": 134, + "missed_rescue": 265, + "avoided_harm": 86, + "selected_both_correct": 1358, + "selected_both_wrong": 6 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.17291017019862398, + "rescued": 49, + "harmed": 135, + "missed_rescue": 258, + "avoided_harm": 85, + "selected_both_correct": 1376, + "selected_both_wrong": 8 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.17468909977784217, + "rescued": 55, + "harmed": 135, + "missed_rescue": 252, + "avoided_harm": 85, + "selected_both_correct": 1397, + "selected_both_wrong": 9 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.1754433851238893, + "rescued": 60, + "harmed": 136, + "missed_rescue": 247, + "avoided_harm": 84, + "selected_both_correct": 1416, + "selected_both_wrong": 12 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5877318116975749, + "metric": 0.17567949038614306, + "rescued": 65, + "harmed": 140, + "missed_rescue": 242, + "avoided_harm": 80, + "selected_both_correct": 1430, + "selected_both_wrong": 13 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.5977175463623395, + "metric": 0.17726242688944494, + "rescued": 73, + "harmed": 142, + "missed_rescue": 234, + "avoided_harm": 78, + "selected_both_correct": 1448, + "selected_both_wrong": 13 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.17927753989081613, + "rescued": 81, + "harmed": 143, + "missed_rescue": 226, + "avoided_harm": 77, + "selected_both_correct": 1471, + "selected_both_wrong": 13 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6176890156918687, + "metric": 0.1803426895170312, + "rescued": 86, + "harmed": 144, + "missed_rescue": 221, + "avoided_harm": 76, + "selected_both_correct": 1489, + "selected_both_wrong": 13 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.18256297675338154, + "rescued": 94, + "harmed": 145, + "missed_rescue": 213, + "avoided_harm": 75, + "selected_both_correct": 1509, + "selected_both_wrong": 16 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.18741054744283442, + "rescued": 103, + "harmed": 149, + "missed_rescue": 204, + "avoided_harm": 71, + "selected_both_correct": 1524, + "selected_both_wrong": 16 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6476462196861626, + "metric": 0.18874453577558464, + "rescued": 109, + "harmed": 150, + "missed_rescue": 198, + "avoided_harm": 70, + "selected_both_correct": 1540, + "selected_both_wrong": 17 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6562054208273894, + "metric": 0.18973030032445837, + "rescued": 114, + "harmed": 152, + "missed_rescue": 193, + "avoided_harm": 68, + "selected_both_correct": 1556, + "selected_both_wrong": 18 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.1910322331722189, + "rescued": 122, + "harmed": 157, + "missed_rescue": 185, + "avoided_harm": 63, + "selected_both_correct": 1576, + "selected_both_wrong": 21 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.19273815380416676, + "rescued": 128, + "harmed": 158, + "missed_rescue": 179, + "avoided_harm": 62, + "selected_both_correct": 1595, + "selected_both_wrong": 23 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.19362404817095658, + "rescued": 133, + "harmed": 161, + "missed_rescue": 174, + "avoided_harm": 59, + "selected_both_correct": 1612, + "selected_both_wrong": 26 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.6975748930099858, + "metric": 0.19489832797986031, + "rescued": 138, + "harmed": 163, + "missed_rescue": 169, + "avoided_harm": 57, + "selected_both_correct": 1626, + "selected_both_wrong": 29 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.19709243659719292, + "rescued": 147, + "harmed": 166, + "missed_rescue": 160, + "avoided_harm": 54, + "selected_both_correct": 1646, + "selected_both_wrong": 29 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.20000323882160753, + "rescued": 155, + "harmed": 166, + "missed_rescue": 152, + "avoided_harm": 54, + "selected_both_correct": 1666, + "selected_both_wrong": 29 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2015690267335004, + "rescued": 160, + "harmed": 167, + "missed_rescue": 147, + "avoided_harm": 53, + "selected_both_correct": 1684, + "selected_both_wrong": 33 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2034984942593638, + "rescued": 166, + "harmed": 168, + "missed_rescue": 141, + "avoided_harm": 52, + "selected_both_correct": 1703, + "selected_both_wrong": 35 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.20679879821670866, + "rescued": 176, + "harmed": 170, + "missed_rescue": 131, + "avoided_harm": 50, + "selected_both_correct": 1718, + "selected_both_wrong": 36 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.20972162293572166, + "rescued": 184, + "harmed": 171, + "missed_rescue": 123, + "avoided_harm": 49, + "selected_both_correct": 1735, + "selected_both_wrong": 38 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.2126226551226551, + "rescued": 191, + "harmed": 174, + "missed_rescue": 116, + "avoided_harm": 46, + "selected_both_correct": 1752, + "selected_both_wrong": 39 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.21516614155218836, + "rescued": 198, + "harmed": 175, + "missed_rescue": 109, + "avoided_harm": 45, + "selected_both_correct": 1771, + "selected_both_wrong": 40 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2162880860385455, + "rescued": 204, + "harmed": 177, + "missed_rescue": 103, + "avoided_harm": 43, + "selected_both_correct": 1788, + "selected_both_wrong": 43 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.21779120256650406, + "rescued": 209, + "harmed": 179, + "missed_rescue": 98, + "avoided_harm": 41, + "selected_both_correct": 1806, + "selected_both_wrong": 46 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.2201843117420812, + "rescued": 216, + "harmed": 182, + "missed_rescue": 91, + "avoided_harm": 38, + "selected_both_correct": 1820, + "selected_both_wrong": 50 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.22188537527520574, + "rescued": 222, + "harmed": 184, + "missed_rescue": 85, + "avoided_harm": 36, + "selected_both_correct": 1838, + "selected_both_wrong": 52 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.2260242392475367, + "rescued": 231, + "harmed": 187, + "missed_rescue": 76, + "avoided_harm": 33, + "selected_both_correct": 1854, + "selected_both_wrong": 52 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2282665192342907, + "rescued": 238, + "harmed": 190, + "missed_rescue": 69, + "avoided_harm": 30, + "selected_both_correct": 1869, + "selected_both_wrong": 55 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.23641538775702553, + "rescued": 246, + "harmed": 193, + "missed_rescue": 61, + "avoided_harm": 27, + "selected_both_correct": 1884, + "selected_both_wrong": 57 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2387681003514125, + "rescued": 251, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 27, + "selected_both_correct": 1904, + "selected_both_wrong": 60 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.24246700995212897, + "rescued": 258, + "harmed": 193, + "missed_rescue": 49, + "avoided_harm": 27, + "selected_both_correct": 1923, + "selected_both_wrong": 62 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.2423696527366133, + "rescued": 259, + "harmed": 197, + "missed_rescue": 48, + "avoided_harm": 23, + "selected_both_correct": 1941, + "selected_both_wrong": 67 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.24478740599615373, + "rescued": 266, + "harmed": 199, + "missed_rescue": 41, + "avoided_harm": 21, + "selected_both_correct": 1959, + "selected_both_wrong": 68 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.25736315359477124, + "rescued": 273, + "harmed": 199, + "missed_rescue": 34, + "avoided_harm": 21, + "selected_both_correct": 1977, + "selected_both_wrong": 71 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.9072753209700428, + "metric": 0.2597551406926407, + "rescued": 278, + "harmed": 200, + "missed_rescue": 29, + "avoided_harm": 20, + "selected_both_correct": 1992, + "selected_both_wrong": 74 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2629241291812946, + "rescued": 284, + "harmed": 201, + "missed_rescue": 23, + "avoided_harm": 19, + "selected_both_correct": 2013, + "selected_both_wrong": 78 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2639590174546567, + "rescued": 290, + "harmed": 203, + "missed_rescue": 17, + "avoided_harm": 17, + "selected_both_correct": 2057, + "selected_both_wrong": 82 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26132796745180287, + "rescued": 293, + "harmed": 207, + "missed_rescue": 14, + "avoided_harm": 13, + "selected_both_correct": 2078, + "selected_both_wrong": 82 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.2584345323365104, + "rescued": 296, + "harmed": 212, + "missed_rescue": 11, + "avoided_harm": 8, + "selected_both_correct": 2096, + "selected_both_wrong": 84 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2503146324574896, + "rescued": 299, + "harmed": 215, + "missed_rescue": 8, + "avoided_harm": 5, + "selected_both_correct": 2116, + "selected_both_wrong": 86 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.2545437366865938, + "rescued": 302, + "harmed": 216, + "missed_rescue": 5, + "avoided_harm": 4, + "selected_both_correct": 2139, + "selected_both_wrong": 87 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.2541444987255433, + "rescued": 304, + "harmed": 218, + "missed_rescue": 3, + "avoided_harm": 2, + "selected_both_correct": 2163, + "selected_both_wrong": 87 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.17942743009320905, + "metric": 0.1784481207469488, + "rescued": 4, + "harmed": 94, + "missed_rescue": 1278, + "avoided_harm": 683, + "selected_both_correct": 2057, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19340878828229027, + "metric": 0.1780928651788194, + "rescued": 4, + "harmed": 99, + "missed_rescue": 1278, + "avoided_harm": 678, + "selected_both_correct": 2220, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20306258322237017, + "metric": 0.17829989216687975, + "rescued": 5, + "harmed": 108, + "missed_rescue": 1277, + "avoided_harm": 669, + "selected_both_correct": 2325, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.1779854683890517, + "rescued": 5, + "harmed": 112, + "missed_rescue": 1277, + "avoided_harm": 665, + "selected_both_correct": 2385, + "selected_both_wrong": 2 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.21537949400798934, + "metric": 0.17917657728013692, + "rescued": 6, + "harmed": 118, + "missed_rescue": 1276, + "avoided_harm": 659, + "selected_both_correct": 2462, + "selected_both_wrong": 2 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.17803687432776102, + "rescued": 6, + "harmed": 133, + "missed_rescue": 1276, + "avoided_harm": 644, + "selected_both_correct": 2638, + "selected_both_wrong": 3 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24800266311584554, + "metric": 0.18026802885144627, + "rescued": 10, + "harmed": 153, + "missed_rescue": 1272, + "avoided_harm": 624, + "selected_both_correct": 2814, + "selected_both_wrong": 3 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.2589880159786951, + "metric": 0.18119827871024874, + "rescued": 11, + "harmed": 163, + "missed_rescue": 1271, + "avoided_harm": 614, + "selected_both_correct": 2935, + "selected_both_wrong": 3 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.27430093209054596, + "metric": 0.17963665085316138, + "rescued": 11, + "harmed": 185, + "missed_rescue": 1271, + "avoided_harm": 592, + "selected_both_correct": 3097, + "selected_both_wrong": 3 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.281624500665779, + "metric": 0.17924095839307458, + "rescued": 11, + "harmed": 191, + "missed_rescue": 1271, + "avoided_harm": 586, + "selected_both_correct": 3179, + "selected_both_wrong": 3 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.28761651131824234, + "metric": 0.1789157174618718, + "rescued": 11, + "harmed": 196, + "missed_rescue": 1271, + "avoided_harm": 581, + "selected_both_correct": 3246, + "selected_both_wrong": 3 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2969374167776298, + "metric": 0.1787206845800119, + "rescued": 11, + "harmed": 199, + "missed_rescue": 1271, + "avoided_harm": 578, + "selected_both_correct": 3355, + "selected_both_wrong": 3 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30459387483355527, + "metric": 0.17813242189706796, + "rescued": 11, + "harmed": 207, + "missed_rescue": 1271, + "avoided_harm": 570, + "selected_both_correct": 3438, + "selected_both_wrong": 4 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3129161118508655, + "metric": 0.17772000842137498, + "rescued": 12, + "harmed": 218, + "missed_rescue": 1270, + "avoided_harm": 559, + "selected_both_correct": 3526, + "selected_both_wrong": 4 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.31990679094540614, + "metric": 0.17791259734473341, + "rescued": 13, + "harmed": 227, + "missed_rescue": 1269, + "avoided_harm": 550, + "selected_both_correct": 3600, + "selected_both_wrong": 4 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.3322237017310253, + "metric": 0.1789653934599337, + "rescued": 15, + "harmed": 238, + "missed_rescue": 1267, + "avoided_harm": 539, + "selected_both_correct": 3735, + "selected_both_wrong": 4 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34054593874833555, + "metric": 0.18042682200794885, + "rescued": 16, + "harmed": 249, + "missed_rescue": 1266, + "avoided_harm": 528, + "selected_both_correct": 3822, + "selected_both_wrong": 5 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3488681757656458, + "metric": 0.17962877325912951, + "rescued": 16, + "harmed": 260, + "missed_rescue": 1266, + "avoided_harm": 517, + "selected_both_correct": 3910, + "selected_both_wrong": 6 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3581890812250333, + "metric": 0.18158304685510065, + "rescued": 18, + "harmed": 271, + "missed_rescue": 1264, + "avoided_harm": 506, + "selected_both_correct": 4008, + "selected_both_wrong": 7 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37017310252996005, + "metric": 0.18069239723097547, + "rescued": 18, + "harmed": 285, + "missed_rescue": 1264, + "avoided_harm": 492, + "selected_both_correct": 4138, + "selected_both_wrong": 7 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.1812845803851709, + "rescued": 20, + "harmed": 292, + "missed_rescue": 1262, + "avoided_harm": 485, + "selected_both_correct": 4209, + "selected_both_wrong": 7 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.38348868175765644, + "metric": 0.1824781664689562, + "rescued": 21, + "harmed": 301, + "missed_rescue": 1261, + "avoided_harm": 476, + "selected_both_correct": 4279, + "selected_both_wrong": 7 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4021304926764314, + "metric": 0.18220594501399923, + "rescued": 23, + "harmed": 321, + "missed_rescue": 1259, + "avoided_harm": 456, + "selected_both_correct": 4480, + "selected_both_wrong": 8 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.41311584553928093, + "metric": 0.18162498930772586, + "rescued": 23, + "harmed": 329, + "missed_rescue": 1259, + "avoided_harm": 448, + "selected_both_correct": 4603, + "selected_both_wrong": 9 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42676431424766975, + "metric": 0.18066434967152634, + "rescued": 23, + "harmed": 351, + "missed_rescue": 1259, + "avoided_harm": 426, + "selected_both_correct": 4745, + "selected_both_wrong": 9 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.4324234354194407, + "metric": 0.1803913789289444, + "rescued": 23, + "harmed": 355, + "missed_rescue": 1259, + "avoided_harm": 422, + "selected_both_correct": 4809, + "selected_both_wrong": 9 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44107856191744343, + "metric": 0.17959468902865128, + "rescued": 23, + "harmed": 368, + "missed_rescue": 1259, + "avoided_harm": 409, + "selected_both_correct": 4900, + "selected_both_wrong": 9 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.45772303595206393, + "metric": 0.17863383514615866, + "rescued": 24, + "harmed": 387, + "missed_rescue": 1258, + "avoided_harm": 390, + "selected_both_correct": 5078, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.4717043941411451, + "metric": 0.17957308538379646, + "rescued": 26, + "harmed": 405, + "missed_rescue": 1256, + "avoided_harm": 372, + "selected_both_correct": 5225, + "selected_both_wrong": 12 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4826897470039947, + "metric": 0.17910245145856155, + "rescued": 27, + "harmed": 417, + "missed_rescue": 1255, + "avoided_harm": 360, + "selected_both_correct": 5344, + "selected_both_wrong": 12 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4896804260985353, + "metric": 0.17871833907402393, + "rescued": 29, + "harmed": 425, + "missed_rescue": 1253, + "avoided_harm": 352, + "selected_both_correct": 5416, + "selected_both_wrong": 14 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.5013315579227696, + "metric": 0.18015785955296285, + "rescued": 57, + "harmed": 431, + "missed_rescue": 1225, + "avoided_harm": 346, + "selected_both_correct": 5516, + "selected_both_wrong": 20 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5116511318242344, + "metric": 0.1810878167493375, + "rescued": 76, + "harmed": 435, + "missed_rescue": 1206, + "avoided_harm": 342, + "selected_both_correct": 5606, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5183089214380826, + "metric": 0.18177962418520882, + "rescued": 92, + "harmed": 439, + "missed_rescue": 1190, + "avoided_harm": 338, + "selected_both_correct": 5662, + "selected_both_wrong": 35 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5252996005326231, + "metric": 0.18246323539519374, + "rescued": 109, + "harmed": 445, + "missed_rescue": 1173, + "avoided_harm": 332, + "selected_both_correct": 5719, + "selected_both_wrong": 39 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5396138482023968, + "metric": 0.1839876166906015, + "rescued": 150, + "harmed": 461, + "missed_rescue": 1132, + "avoided_harm": 316, + "selected_both_correct": 5829, + "selected_both_wrong": 44 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5452729693741678, + "metric": 0.1846674213907964, + "rescued": 164, + "harmed": 464, + "missed_rescue": 1118, + "avoided_harm": 313, + "selected_both_correct": 5875, + "selected_both_wrong": 49 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.559254327563249, + "metric": 0.18676294233791663, + "rescued": 196, + "harmed": 473, + "missed_rescue": 1086, + "avoided_harm": 304, + "selected_both_correct": 5990, + "selected_both_wrong": 61 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.572237017310253, + "metric": 0.18938663221175195, + "rescued": 230, + "harmed": 483, + "missed_rescue": 1052, + "avoided_harm": 294, + "selected_both_correct": 6093, + "selected_both_wrong": 70 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.5862183754993342, + "metric": 0.1909571002676625, + "rescued": 267, + "harmed": 496, + "missed_rescue": 1015, + "avoided_harm": 281, + "selected_both_correct": 6208, + "selected_both_wrong": 73 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5998668442077231, + "metric": 0.1934300101790686, + "rescued": 299, + "harmed": 503, + "missed_rescue": 983, + "avoided_harm": 274, + "selected_both_correct": 6315, + "selected_both_wrong": 91 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.6191744340878829, + "metric": 0.1973701682130997, + "rescued": 350, + "harmed": 520, + "missed_rescue": 932, + "avoided_harm": 257, + "selected_both_correct": 6465, + "selected_both_wrong": 105 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.6264980026631158, + "metric": 0.1986727924998457, + "rescued": 370, + "harmed": 523, + "missed_rescue": 912, + "avoided_harm": 254, + "selected_both_correct": 6527, + "selected_both_wrong": 108 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6384820239680427, + "metric": 0.20186289191698192, + "rescued": 402, + "harmed": 532, + "missed_rescue": 880, + "avoided_harm": 245, + "selected_both_correct": 6621, + "selected_both_wrong": 117 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6461384820239681, + "metric": 0.20246031058520178, + "rescued": 425, + "harmed": 537, + "missed_rescue": 857, + "avoided_harm": 240, + "selected_both_correct": 6682, + "selected_both_wrong": 120 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.653462050599201, + "metric": 0.20349944137622245, + "rescued": 444, + "harmed": 539, + "missed_rescue": 838, + "avoided_harm": 238, + "selected_both_correct": 6744, + "selected_both_wrong": 125 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6597869507323568, + "metric": 0.20442990090722246, + "rescued": 461, + "harmed": 545, + "missed_rescue": 821, + "avoided_harm": 232, + "selected_both_correct": 6792, + "selected_both_wrong": 130 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6661118508655126, + "metric": 0.2044993942500819, + "rescued": 480, + "harmed": 556, + "missed_rescue": 802, + "avoided_harm": 221, + "selected_both_correct": 6834, + "selected_both_wrong": 134 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6734354194407457, + "metric": 0.2064379542733177, + "rescued": 500, + "harmed": 561, + "missed_rescue": 782, + "avoided_harm": 216, + "selected_both_correct": 6889, + "selected_both_wrong": 142 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6820905459387483, + "metric": 0.2065831206227621, + "rescued": 523, + "harmed": 566, + "missed_rescue": 759, + "avoided_harm": 211, + "selected_both_correct": 6958, + "selected_both_wrong": 149 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6927430093209055, + "metric": 0.20802802394105854, + "rescued": 553, + "harmed": 577, + "missed_rescue": 729, + "avoided_harm": 200, + "selected_both_correct": 7035, + "selected_both_wrong": 159 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.7090545938748336, + "metric": 0.21169680596278434, + "rescued": 602, + "harmed": 584, + "missed_rescue": 680, + "avoided_harm": 193, + "selected_both_correct": 7159, + "selected_both_wrong": 175 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7170439414114514, + "metric": 0.2139280527103987, + "rescued": 625, + "harmed": 588, + "missed_rescue": 657, + "avoided_harm": 189, + "selected_both_correct": 7220, + "selected_both_wrong": 183 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7243675099866844, + "metric": 0.2146900231025185, + "rescued": 647, + "harmed": 594, + "missed_rescue": 635, + "avoided_harm": 183, + "selected_both_correct": 7268, + "selected_both_wrong": 195 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7330226364846871, + "metric": 0.21653561577692484, + "rescued": 670, + "harmed": 599, + "missed_rescue": 612, + "avoided_harm": 178, + "selected_both_correct": 7338, + "selected_both_wrong": 201 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.7363515312916112, + "metric": 0.21721689648133474, + "rescued": 681, + "harmed": 601, + "missed_rescue": 601, + "avoided_harm": 176, + "selected_both_correct": 7365, + "selected_both_wrong": 201 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7486684420772304, + "metric": 0.22103445955201576, + "rescued": 718, + "harmed": 610, + "missed_rescue": 564, + "avoided_harm": 167, + "selected_both_correct": 7460, + "selected_both_wrong": 208 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.757323568575233, + "metric": 0.22340134248367294, + "rescued": 744, + "harmed": 618, + "missed_rescue": 538, + "avoided_harm": 159, + "selected_both_correct": 7522, + "selected_both_wrong": 216 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7639813581890812, + "metric": 0.2269322568854726, + "rescued": 767, + "harmed": 620, + "missed_rescue": 515, + "avoided_harm": 157, + "selected_both_correct": 7571, + "selected_both_wrong": 222 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7733022636484687, + "metric": 0.22757452798038888, + "rescued": 798, + "harmed": 628, + "missed_rescue": 484, + "avoided_harm": 149, + "selected_both_correct": 7636, + "selected_both_wrong": 230 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.784953395472703, + "metric": 0.227821050912831, + "rescued": 824, + "harmed": 634, + "missed_rescue": 458, + "avoided_harm": 143, + "selected_both_correct": 7728, + "selected_both_wrong": 246 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7939414114513982, + "metric": 0.22991865671924006, + "rescued": 852, + "harmed": 642, + "missed_rescue": 430, + "avoided_harm": 135, + "selected_both_correct": 7793, + "selected_both_wrong": 253 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8039280958721704, + "metric": 0.23486191981046528, + "rescued": 884, + "harmed": 646, + "missed_rescue": 398, + "avoided_harm": 131, + "selected_both_correct": 7868, + "selected_both_wrong": 262 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.8122503328894807, + "metric": 0.23803916047544685, + "rescued": 906, + "harmed": 655, + "missed_rescue": 376, + "avoided_harm": 122, + "selected_both_correct": 7924, + "selected_both_wrong": 275 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8215712383488681, + "metric": 0.24022947987632814, + "rescued": 933, + "harmed": 664, + "missed_rescue": 349, + "avoided_harm": 113, + "selected_both_correct": 7987, + "selected_both_wrong": 288 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388814913448736, + "metric": 0.24528627405740036, + "rescued": 988, + "harmed": 672, + "missed_rescue": 294, + "avoided_harm": 105, + "selected_both_correct": 8116, + "selected_both_wrong": 304 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8535286284953395, + "metric": 0.249537136326177, + "rescued": 1028, + "harmed": 680, + "missed_rescue": 254, + "avoided_harm": 97, + "selected_both_correct": 8224, + "selected_both_wrong": 324 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8638482023968043, + "metric": 0.25211074301800224, + "rescued": 1056, + "harmed": 688, + "missed_rescue": 226, + "avoided_harm": 89, + "selected_both_correct": 8299, + "selected_both_wrong": 337 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8745006657789614, + "metric": 0.2540042317315216, + "rescued": 1076, + "harmed": 695, + "missed_rescue": 206, + "avoided_harm": 82, + "selected_both_correct": 8389, + "selected_both_wrong": 348 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8764980026631158, + "metric": 0.2540996517958595, + "rescued": 1080, + "harmed": 698, + "missed_rescue": 202, + "avoided_harm": 79, + "selected_both_correct": 8404, + "selected_both_wrong": 350 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8841544607190412, + "metric": 0.25436194678023316, + "rescued": 1090, + "harmed": 705, + "missed_rescue": 192, + "avoided_harm": 72, + "selected_both_correct": 8468, + "selected_both_wrong": 361 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8891478029294274, + "metric": 0.2534339447803836, + "rescued": 1101, + "harmed": 709, + "missed_rescue": 181, + "avoided_harm": 68, + "selected_both_correct": 8506, + "selected_both_wrong": 368 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.8981358189081226, + "metric": 0.2563240880787248, + "rescued": 1126, + "harmed": 715, + "missed_rescue": 156, + "avoided_harm": 62, + "selected_both_correct": 8573, + "selected_both_wrong": 378 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9054593874833555, + "metric": 0.25551845731687656, + "rescued": 1143, + "harmed": 721, + "missed_rescue": 139, + "avoided_harm": 56, + "selected_both_correct": 8632, + "selected_both_wrong": 384 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.26306323674307786, + "rescued": 1189, + "harmed": 729, + "missed_rescue": 93, + "avoided_harm": 48, + "selected_both_correct": 8800, + "selected_both_wrong": 402 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.9364181091877497, + "metric": 0.26218185491449153, + "rescued": 1203, + "harmed": 738, + "missed_rescue": 79, + "avoided_harm": 39, + "selected_both_correct": 8897, + "selected_both_wrong": 414 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.953728362183755, + "metric": 0.26357052137555514, + "rescued": 1223, + "harmed": 754, + "missed_rescue": 59, + "avoided_harm": 23, + "selected_both_correct": 9055, + "selected_both_wrong": 428 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.96271637816245, + "metric": 0.26400290307863034, + "rescued": 1231, + "harmed": 758, + "missed_rescue": 51, + "avoided_harm": 19, + "selected_both_correct": 9150, + "selected_both_wrong": 429 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.9700399467376831, + "metric": 0.2647475724296899, + "rescued": 1237, + "harmed": 760, + "missed_rescue": 45, + "avoided_harm": 17, + "selected_both_correct": 9227, + "selected_both_wrong": 432 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.987017310252996, + "metric": 0.2649306125590882, + "rescued": 1257, + "harmed": 772, + "missed_rescue": 25, + "avoided_harm": 5, + "selected_both_correct": 9387, + "selected_both_wrong": 444 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + }, + "learned": { + "selected_budget_index": 50, + "validation_area_0_25": 0.23466194499464404, + "validation_point": { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.33126660689663334, + "rescued": 304, + "harmed": 95, + "missed_rescue": 3, + "avoided_harm": 125, + "selected_both_correct": 916, + "selected_both_wrong": 85 + }, + "test_point": { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4876830892143808, + "metric": 0.29875800068449215, + "rescued": 1249, + "harmed": 337, + "missed_rescue": 33, + "avoided_harm": 440, + "selected_both_correct": 3843, + "selected_both_wrong": 431 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.3274500370025635, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.18570977475658457, + "rescued": 12, + "harmed": 1, + "missed_rescue": 295, + "avoided_harm": 219, + "selected_both_correct": 12, + "selected_both_wrong": 3 + }, + { + "budget": 0.02, + "threshold": 2.223353862762451, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.19072678038582708, + "rescued": 24, + "harmed": 1, + "missed_rescue": 283, + "avoided_harm": 219, + "selected_both_correct": 26, + "selected_both_wrong": 5 + }, + { + "budget": 0.03, + "threshold": 2.105621099472046, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.1947361119928147, + "rescued": 33, + "harmed": 1, + "missed_rescue": 274, + "avoided_harm": 219, + "selected_both_correct": 40, + "selected_both_wrong": 10 + }, + { + "budget": 0.04, + "threshold": 1.9981958866119385, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.20027934358652552, + "rescued": 44, + "harmed": 1, + "missed_rescue": 263, + "avoided_harm": 219, + "selected_both_correct": 52, + "selected_both_wrong": 15 + }, + { + "budget": 0.05, + "threshold": 1.949358582496643, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.21085585634765963, + "rescued": 57, + "harmed": 1, + "missed_rescue": 250, + "avoided_harm": 219, + "selected_both_correct": 65, + "selected_both_wrong": 17 + }, + { + "budget": 0.06, + "threshold": 1.8504760265350342, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.2155935788086818, + "rescued": 68, + "harmed": 2, + "missed_rescue": 239, + "avoided_harm": 218, + "selected_both_correct": 78, + "selected_both_wrong": 20 + }, + { + "budget": 0.07, + "threshold": 1.6561405658721924, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.2201008442387753, + "rescued": 76, + "harmed": 4, + "missed_rescue": 231, + "avoided_harm": 216, + "selected_both_correct": 93, + "selected_both_wrong": 23 + }, + { + "budget": 0.08, + "threshold": 1.5329420566558838, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.22831044506816361, + "rescued": 90, + "harmed": 5, + "missed_rescue": 217, + "avoided_harm": 215, + "selected_both_correct": 106, + "selected_both_wrong": 23 + }, + { + "budget": 0.09, + "threshold": 1.380273461341858, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.23143136333139114, + "rescued": 96, + "harmed": 6, + "missed_rescue": 211, + "avoided_harm": 214, + "selected_both_correct": 124, + "selected_both_wrong": 26 + }, + { + "budget": 0.1, + "threshold": 1.32539701461792, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.23760484070252644, + "rescued": 104, + "harmed": 8, + "missed_rescue": 203, + "avoided_harm": 212, + "selected_both_correct": 140, + "selected_both_wrong": 28 + }, + { + "budget": 0.11, + "threshold": 1.2173664569854736, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.23824320355020984, + "rescued": 110, + "harmed": 9, + "missed_rescue": 197, + "avoided_harm": 211, + "selected_both_correct": 161, + "selected_both_wrong": 28 + }, + { + "budget": 0.12, + "threshold": 1.1719112396240234, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.2407100587309389, + "rescued": 115, + "harmed": 9, + "missed_rescue": 192, + "avoided_harm": 211, + "selected_both_correct": 181, + "selected_both_wrong": 31 + }, + { + "budget": 0.13, + "threshold": 1.1186922788619995, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.24134727061556333, + "rescued": 121, + "harmed": 11, + "missed_rescue": 186, + "avoided_harm": 209, + "selected_both_correct": 199, + "selected_both_wrong": 33 + }, + { + "budget": 0.14, + "threshold": 1.0748379230499268, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.24423106111883697, + "rescued": 128, + "harmed": 16, + "missed_rescue": 179, + "avoided_harm": 204, + "selected_both_correct": 213, + "selected_both_wrong": 35 + }, + { + "budget": 0.15, + "threshold": 1.0427184104919434, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.24892068530842207, + "rescued": 135, + "harmed": 16, + "missed_rescue": 172, + "avoided_harm": 204, + "selected_both_correct": 234, + "selected_both_wrong": 35 + }, + { + "budget": 0.16, + "threshold": 1.0111123323440552, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.252253667728857, + "rescued": 141, + "harmed": 16, + "missed_rescue": 166, + "avoided_harm": 204, + "selected_both_correct": 254, + "selected_both_wrong": 37 + }, + { + "budget": 0.17, + "threshold": 0.9669641852378845, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.25275417712816833, + "rescued": 146, + "harmed": 18, + "missed_rescue": 161, + "avoided_harm": 202, + "selected_both_correct": 272, + "selected_both_wrong": 40 + }, + { + "budget": 0.18, + "threshold": 0.9490972757339478, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.25643535926629674, + "rescued": 152, + "harmed": 19, + "missed_rescue": 155, + "avoided_harm": 201, + "selected_both_correct": 292, + "selected_both_wrong": 41 + }, + { + "budget": 0.19, + "threshold": 0.9165292382240295, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.2576337529504327, + "rescued": 159, + "harmed": 22, + "missed_rescue": 148, + "avoided_harm": 198, + "selected_both_correct": 310, + "selected_both_wrong": 41 + }, + { + "budget": 0.2, + "threshold": 0.8978593349456787, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.2581122448979592, + "rescued": 164, + "harmed": 24, + "missed_rescue": 143, + "avoided_harm": 196, + "selected_both_correct": 330, + "selected_both_wrong": 42 + }, + { + "budget": 0.21, + "threshold": 0.854847252368927, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.2602060885029815, + "rescued": 169, + "harmed": 25, + "missed_rescue": 138, + "avoided_harm": 195, + "selected_both_correct": 350, + "selected_both_wrong": 44 + }, + { + "budget": 0.22, + "threshold": 0.8254434466362, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.2635325636000918, + "rescued": 174, + "harmed": 25, + "missed_rescue": 133, + "avoided_harm": 195, + "selected_both_correct": 371, + "selected_both_wrong": 46 + }, + { + "budget": 0.23, + "threshold": 0.8046320080757141, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.24799578059071728, + "rescued": 178, + "harmed": 29, + "missed_rescue": 129, + "avoided_harm": 191, + "selected_both_correct": 391, + "selected_both_wrong": 46 + }, + { + "budget": 0.24, + "threshold": 0.7761231660842896, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.25061334511913563, + "rescued": 184, + "harmed": 31, + "missed_rescue": 123, + "avoided_harm": 189, + "selected_both_correct": 410, + "selected_both_wrong": 47 + }, + { + "budget": 0.25, + "threshold": 0.7452236413955688, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.25309147374364765, + "rescued": 190, + "harmed": 32, + "missed_rescue": 117, + "avoided_harm": 188, + "selected_both_correct": 430, + "selected_both_wrong": 48 + }, + { + "budget": 0.26, + "threshold": 0.7167489528656006, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.25499133574730426, + "rescued": 196, + "harmed": 34, + "missed_rescue": 111, + "avoided_harm": 186, + "selected_both_correct": 448, + "selected_both_wrong": 50 + }, + { + "budget": 0.27, + "threshold": 0.6837625503540039, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.25870776744375423, + "rescued": 201, + "harmed": 34, + "missed_rescue": 106, + "avoided_harm": 186, + "selected_both_correct": 468, + "selected_both_wrong": 53 + }, + { + "budget": 0.28, + "threshold": 0.633089005947113, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.25965649053463696, + "rescued": 205, + "harmed": 36, + "missed_rescue": 102, + "avoided_harm": 184, + "selected_both_correct": 488, + "selected_both_wrong": 55 + }, + { + "budget": 0.29, + "threshold": 0.5875619649887085, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.2629594041536389, + "rescued": 213, + "harmed": 39, + "missed_rescue": 94, + "avoided_harm": 181, + "selected_both_correct": 505, + "selected_both_wrong": 55 + }, + { + "budget": 0.3, + "threshold": 0.5604347586631775, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.26353457364905875, + "rescued": 219, + "harmed": 45, + "missed_rescue": 88, + "avoided_harm": 175, + "selected_both_correct": 521, + "selected_both_wrong": 55 + }, + { + "budget": 0.31, + "threshold": 0.5320743918418884, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.26608946608946615, + "rescued": 225, + "harmed": 47, + "missed_rescue": 82, + "avoided_harm": 173, + "selected_both_correct": 540, + "selected_both_wrong": 56 + }, + { + "budget": 0.32, + "threshold": 0.49417412281036377, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.2673115302724549, + "rescued": 230, + "harmed": 51, + "missed_rescue": 77, + "avoided_harm": 169, + "selected_both_correct": 558, + "selected_both_wrong": 57 + }, + { + "budget": 0.33, + "threshold": 0.46743881702423096, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.26933684672815106, + "rescued": 236, + "harmed": 53, + "missed_rescue": 71, + "avoided_harm": 167, + "selected_both_correct": 577, + "selected_both_wrong": 58 + }, + { + "budget": 0.34, + "threshold": 0.4429023861885071, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.27352180339985216, + "rescued": 243, + "harmed": 54, + "missed_rescue": 64, + "avoided_harm": 166, + "selected_both_correct": 597, + "selected_both_wrong": 58 + }, + { + "budget": 0.35, + "threshold": 0.40694165229797363, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.275880713431076, + "rescued": 247, + "harmed": 60, + "missed_rescue": 60, + "avoided_harm": 160, + "selected_both_correct": 612, + "selected_both_wrong": 61 + }, + { + "budget": 0.36, + "threshold": 0.3611782193183899, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.27761867831050957, + "rescued": 253, + "harmed": 64, + "missed_rescue": 54, + "avoided_harm": 156, + "selected_both_correct": 629, + "selected_both_wrong": 62 + }, + { + "budget": 0.37, + "threshold": 0.3311936855316162, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.28017336393459635, + "rescued": 257, + "harmed": 66, + "missed_rescue": 50, + "avoided_harm": 154, + "selected_both_correct": 650, + "selected_both_wrong": 63 + }, + { + "budget": 0.38, + "threshold": 0.28899651765823364, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.2825320512820513, + "rescued": 263, + "harmed": 70, + "missed_rescue": 44, + "avoided_harm": 150, + "selected_both_correct": 667, + "selected_both_wrong": 64 + }, + { + "budget": 0.39, + "threshold": 0.19986218214035034, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.28580355455355455, + "rescued": 267, + "harmed": 71, + "missed_rescue": 40, + "avoided_harm": 149, + "selected_both_correct": 689, + "selected_both_wrong": 65 + }, + { + "budget": 0.4, + "threshold": 0.157886803150177, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.3015271169266015, + "rescued": 271, + "harmed": 72, + "missed_rescue": 36, + "avoided_harm": 148, + "selected_both_correct": 709, + "selected_both_wrong": 68 + }, + { + "budget": 0.41, + "threshold": 0.12043046951293945, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.30656010656010657, + "rescued": 275, + "harmed": 74, + "missed_rescue": 32, + "avoided_harm": 146, + "selected_both_correct": 730, + "selected_both_wrong": 69 + }, + { + "budget": 0.42, + "threshold": 0.06623327732086182, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.30573593073593075, + "rescued": 277, + "harmed": 77, + "missed_rescue": 30, + "avoided_harm": 143, + "selected_both_correct": 750, + "selected_both_wrong": 72 + }, + { + "budget": 0.43, + "threshold": -0.01827472448348999, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.3122803645750104, + "rescued": 283, + "harmed": 79, + "missed_rescue": 24, + "avoided_harm": 141, + "selected_both_correct": 770, + "selected_both_wrong": 72 + }, + { + "budget": 0.44, + "threshold": -0.14177680015563965, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.32014301599541106, + "rescued": 290, + "harmed": 81, + "missed_rescue": 17, + "avoided_harm": 139, + "selected_both_correct": 788, + "selected_both_wrong": 73 + }, + { + "budget": 0.45, + "threshold": -0.270923376083374, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.3274352926559858, + "rescued": 295, + "harmed": 82, + "missed_rescue": 12, + "avoided_harm": 138, + "selected_both_correct": 808, + "selected_both_wrong": 75 + }, + { + "budget": 0.46, + "threshold": -0.4213377833366394, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.3269493177387914, + "rescued": 299, + "harmed": 85, + "missed_rescue": 8, + "avoided_harm": 135, + "selected_both_correct": 826, + "selected_both_wrong": 78 + }, + { + "budget": 0.47, + "threshold": -0.6602996587753296, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.3206497235233282, + "rescued": 299, + "harmed": 88, + "missed_rescue": 8, + "avoided_harm": 132, + "selected_both_correct": 849, + "selected_both_wrong": 80 + }, + { + "budget": 0.48, + "threshold": -0.8612939715385437, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.3201660892787428, + "rescued": 300, + "harmed": 89, + "missed_rescue": 7, + "avoided_harm": 131, + "selected_both_correct": 873, + "selected_both_wrong": 82 + }, + { + "budget": 0.49, + "threshold": -0.9715782999992371, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.325869430652523, + "rescued": 303, + "harmed": 91, + "missed_rescue": 4, + "avoided_harm": 129, + "selected_both_correct": 894, + "selected_both_wrong": 84 + }, + { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.33126660689663334, + "rescued": 304, + "harmed": 95, + "missed_rescue": 3, + "avoided_harm": 125, + "selected_both_correct": 916, + "selected_both_wrong": 85 + }, + { + "budget": 0.51, + "threshold": -1.6505446434020996, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.3264586577163139, + "rescued": 304, + "harmed": 98, + "missed_rescue": 3, + "avoided_harm": 122, + "selected_both_correct": 941, + "selected_both_wrong": 85 + }, + { + "budget": 0.52, + "threshold": -2.6140921115875244, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.3264586577163139, + "rescued": 304, + "harmed": 98, + "missed_rescue": 3, + "avoided_harm": 122, + "selected_both_correct": 968, + "selected_both_wrong": 86 + }, + { + "budget": 0.53, + "threshold": -2.6907107830047607, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.3235189795049722, + "rescued": 304, + "harmed": 102, + "missed_rescue": 3, + "avoided_harm": 118, + "selected_both_correct": 992, + "selected_both_wrong": 86 + }, + { + "budget": 0.54, + "threshold": -2.8344802856445312, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.3235189795049722, + "rescued": 304, + "harmed": 102, + "missed_rescue": 3, + "avoided_harm": 118, + "selected_both_correct": 1020, + "selected_both_wrong": 86 + }, + { + "budget": 0.55, + "threshold": -2.905412197113037, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.3203578942095772, + "rescued": 304, + "harmed": 106, + "missed_rescue": 3, + "avoided_harm": 114, + "selected_both_correct": 1044, + "selected_both_wrong": 86 + }, + { + "budget": 0.56, + "threshold": -2.9482638835906982, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.31783773291925466, + "rescued": 304, + "harmed": 107, + "missed_rescue": 3, + "avoided_harm": 113, + "selected_both_correct": 1071, + "selected_both_wrong": 86 + }, + { + "budget": 0.57, + "threshold": -2.9957408905029297, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.31989933342152455, + "rescued": 305, + "harmed": 111, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 1094, + "selected_both_wrong": 86 + }, + { + "budget": 0.58, + "threshold": -3.1037726402282715, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.31989933342152455, + "rescued": 305, + "harmed": 111, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 1122, + "selected_both_wrong": 86 + }, + { + "budget": 0.59, + "threshold": -3.1757895946502686, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.3184090005781182, + "rescued": 305, + "harmed": 113, + "missed_rescue": 2, + "avoided_harm": 107, + "selected_both_correct": 1148, + "selected_both_wrong": 86 + }, + { + "budget": 0.6, + "threshold": -3.253206729888916, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.3177705940408353, + "rescued": 305, + "harmed": 114, + "missed_rescue": 2, + "avoided_harm": 106, + "selected_both_correct": 1175, + "selected_both_wrong": 86 + }, + { + "budget": 0.61, + "threshold": -3.2825026512145996, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.31694234643839414, + "rescued": 305, + "harmed": 115, + "missed_rescue": 2, + "avoided_harm": 105, + "selected_both_correct": 1202, + "selected_both_wrong": 86 + }, + { + "budget": 0.62, + "threshold": -3.322240114212036, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.3111129112312657, + "rescued": 305, + "harmed": 121, + "missed_rescue": 2, + "avoided_harm": 99, + "selected_both_correct": 1224, + "selected_both_wrong": 86 + }, + { + "budget": 0.63, + "threshold": -3.3503003120422363, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.30995657450601133, + "rescued": 305, + "harmed": 123, + "missed_rescue": 2, + "avoided_harm": 97, + "selected_both_correct": 1250, + "selected_both_wrong": 86 + }, + { + "budget": 0.64, + "threshold": -3.377089738845825, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.3083114768968865, + "rescued": 305, + "harmed": 126, + "missed_rescue": 2, + "avoided_harm": 94, + "selected_both_correct": 1275, + "selected_both_wrong": 86 + }, + { + "budget": 0.65, + "threshold": -3.4055864810943604, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.30666904687523244, + "rescued": 305, + "harmed": 128, + "missed_rescue": 2, + "avoided_harm": 92, + "selected_both_correct": 1301, + "selected_both_wrong": 86 + }, + { + "budget": 0.66, + "threshold": -3.430248498916626, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.30339954163483573, + "rescued": 305, + "harmed": 131, + "missed_rescue": 2, + "avoided_harm": 89, + "selected_both_correct": 1326, + "selected_both_wrong": 86 + }, + { + "budget": 0.67, + "threshold": -3.4584414958953857, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.3011204481792717, + "rescued": 305, + "harmed": 135, + "missed_rescue": 2, + "avoided_harm": 85, + "selected_both_correct": 1350, + "selected_both_wrong": 86 + }, + { + "budget": 0.68, + "threshold": -3.502922773361206, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.2968337073049115, + "rescued": 305, + "harmed": 140, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 1373, + "selected_both_wrong": 86 + }, + { + "budget": 0.69, + "threshold": -3.532865285873413, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.2968337073049115, + "rescued": 305, + "harmed": 140, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 1401, + "selected_both_wrong": 86 + }, + { + "budget": 0.7, + "threshold": -3.5550942420959473, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.2951499808548299, + "rescued": 305, + "harmed": 143, + "missed_rescue": 2, + "avoided_harm": 77, + "selected_both_correct": 1426, + "selected_both_wrong": 86 + }, + { + "budget": 0.71, + "threshold": -3.566344976425171, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.2985697751322751, + "rescued": 306, + "harmed": 144, + "missed_rescue": 1, + "avoided_harm": 76, + "selected_both_correct": 1452, + "selected_both_wrong": 86 + }, + { + "budget": 0.72, + "threshold": -3.6056067943573, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.29288998820248824, + "rescued": 306, + "harmed": 150, + "missed_rescue": 1, + "avoided_harm": 70, + "selected_both_correct": 1474, + "selected_both_wrong": 86 + }, + { + "budget": 0.73, + "threshold": -3.628530979156494, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2916863054994102, + "rescued": 306, + "harmed": 156, + "missed_rescue": 1, + "avoided_harm": 64, + "selected_both_correct": 1496, + "selected_both_wrong": 86 + }, + { + "budget": 0.74, + "threshold": -3.6580705642700195, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2908398260842505, + "rescued": 306, + "harmed": 158, + "missed_rescue": 1, + "avoided_harm": 62, + "selected_both_correct": 1521, + "selected_both_wrong": 87 + }, + { + "budget": 0.75, + "threshold": -3.6900978088378906, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.29144523554686896, + "rescued": 307, + "harmed": 161, + "missed_rescue": 0, + "avoided_harm": 59, + "selected_both_correct": 1545, + "selected_both_wrong": 87 + }, + { + "budget": 0.76, + "threshold": -3.7050747871398926, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.28771367521367525, + "rescued": 307, + "harmed": 164, + "missed_rescue": 0, + "avoided_harm": 56, + "selected_both_correct": 1569, + "selected_both_wrong": 88 + }, + { + "budget": 0.77, + "threshold": -3.720116138458252, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.28342343829529665, + "rescued": 307, + "harmed": 170, + "missed_rescue": 0, + "avoided_harm": 50, + "selected_both_correct": 1591, + "selected_both_wrong": 88 + }, + { + "budget": 0.78, + "threshold": -3.7399919033050537, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.27967055835908294, + "rescued": 307, + "harmed": 175, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 1614, + "selected_both_wrong": 88 + }, + { + "budget": 0.79, + "threshold": -3.799569606781006, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2754483860214125, + "rescued": 307, + "harmed": 180, + "missed_rescue": 0, + "avoided_harm": 40, + "selected_both_correct": 1637, + "selected_both_wrong": 88 + }, + { + "budget": 0.8, + "threshold": -3.8980000019073486, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.2750612892472189, + "rescued": 307, + "harmed": 181, + "missed_rescue": 0, + "avoided_harm": 39, + "selected_both_correct": 1664, + "selected_both_wrong": 88 + }, + { + "budget": 0.81, + "threshold": -4.0607757568359375, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.27434520884520885, + "rescued": 307, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 1687, + "selected_both_wrong": 88 + }, + { + "budget": 0.82, + "threshold": -8.964031219482422, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.27434520884520885, + "rescued": 307, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 1715, + "selected_both_wrong": 88 + }, + { + "budget": 0.83, + "threshold": -9.02132511138916, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.27363625362132826, + "rescued": 307, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 1742, + "selected_both_wrong": 88 + }, + { + "budget": 0.84, + "threshold": -9.09000015258789, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2722230802027571, + "rescued": 307, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1768, + "selected_both_wrong": 88 + }, + { + "budget": 0.85, + "threshold": -9.12938404083252, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2722230802027571, + "rescued": 307, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1796, + "selected_both_wrong": 88 + }, + { + "budget": 0.86, + "threshold": -9.17154312133789, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2708398536448072, + "rescued": 307, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 29, + "selected_both_correct": 1822, + "selected_both_wrong": 88 + }, + { + "budget": 0.87, + "threshold": -9.273024559020996, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.2708398536448072, + "rescued": 307, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 29, + "selected_both_correct": 1850, + "selected_both_wrong": 88 + }, + { + "budget": 0.88, + "threshold": -9.444879531860352, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1877, + "selected_both_wrong": 88 + }, + { + "budget": 0.89, + "threshold": -9.566991806030273, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1905, + "selected_both_wrong": 88 + }, + { + "budget": 0.9, + "threshold": -9.675031661987305, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1933, + "selected_both_wrong": 88 + }, + { + "budget": 0.91, + "threshold": -9.739017486572266, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.2694835977031099, + "rescued": 307, + "harmed": 193, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 1960, + "selected_both_wrong": 88 + }, + { + "budget": 0.92, + "threshold": -9.787779808044434, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2680528165105032, + "rescued": 307, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 1985, + "selected_both_wrong": 88 + }, + { + "budget": 0.93, + "threshold": -9.832305908203125, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2680528165105032, + "rescued": 307, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 2013, + "selected_both_wrong": 88 + }, + { + "budget": 0.94, + "threshold": -9.88642692565918, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2673844613676863, + "rescued": 307, + "harmed": 201, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 2036, + "selected_both_wrong": 88 + }, + { + "budget": 0.95, + "threshold": -9.976344108581543, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26575274061687104, + "rescued": 307, + "harmed": 203, + "missed_rescue": 0, + "avoided_harm": 17, + "selected_both_correct": 2062, + "selected_both_wrong": 88 + }, + { + "budget": 0.96, + "threshold": -10.03441047668457, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.261865308604439, + "rescued": 307, + "harmed": 209, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 2084, + "selected_both_wrong": 88 + }, + { + "budget": 0.97, + "threshold": -10.085777282714844, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2589514652014652, + "rescued": 307, + "harmed": 212, + "missed_rescue": 0, + "avoided_harm": 8, + "selected_both_correct": 2109, + "selected_both_wrong": 88 + }, + { + "budget": 0.98, + "threshold": -10.1488676071167, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.25830826953457287, + "rescued": 307, + "harmed": 213, + "missed_rescue": 0, + "avoided_harm": 7, + "selected_both_correct": 2136, + "selected_both_wrong": 88 + }, + { + "budget": 0.99, + "threshold": -10.265552520751953, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.25668767695229905, + "rescued": 307, + "harmed": 216, + "missed_rescue": 0, + "avoided_harm": 4, + "selected_both_correct": 2161, + "selected_both_wrong": 88 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.3274500370025635, + "operator": "ge", + "actual_fraction": 0.013648468708388815, + "metric": 0.18693471712069915, + "rescued": 72, + "harmed": 3, + "missed_rescue": 1210, + "avoided_harm": 774, + "selected_both_correct": 69, + "selected_both_wrong": 20 + }, + { + "budget": 0.02, + "threshold": 2.223353862762451, + "operator": "ge", + "actual_fraction": 0.019640479360852198, + "metric": 0.18781313491575913, + "rescued": 98, + "harmed": 7, + "missed_rescue": 1184, + "avoided_harm": 770, + "selected_both_correct": 104, + "selected_both_wrong": 27 + }, + { + "budget": 0.03, + "threshold": 2.105621099472046, + "operator": "ge", + "actual_fraction": 0.031291611185086554, + "metric": 0.1913128749335621, + "rescued": 154, + "harmed": 10, + "missed_rescue": 1128, + "avoided_harm": 767, + "selected_both_correct": 170, + "selected_both_wrong": 42 + }, + { + "budget": 0.04, + "threshold": 1.9981958866119385, + "operator": "ge", + "actual_fraction": 0.04061251664447404, + "metric": 0.19266008228682668, + "rescued": 193, + "harmed": 16, + "missed_rescue": 1089, + "avoided_harm": 761, + "selected_both_correct": 219, + "selected_both_wrong": 60 + }, + { + "budget": 0.05, + "threshold": 1.949358582496643, + "operator": "ge", + "actual_fraction": 0.045938748335552594, + "metric": 0.19531128485766117, + "rescued": 224, + "harmed": 18, + "missed_rescue": 1058, + "avoided_harm": 759, + "selected_both_correct": 249, + "selected_both_wrong": 61 + }, + { + "budget": 0.06, + "threshold": 1.8504760265350342, + "operator": "ge", + "actual_fraction": 0.0559254327563249, + "metric": 0.19800731707634547, + "rescued": 268, + "harmed": 23, + "missed_rescue": 1014, + "avoided_harm": 754, + "selected_both_correct": 302, + "selected_both_wrong": 79 + }, + { + "budget": 0.07, + "threshold": 1.6561405658721924, + "operator": "ge", + "actual_fraction": 0.07456724367509987, + "metric": 0.19763843353437874, + "rescued": 336, + "harmed": 37, + "missed_rescue": 946, + "avoided_harm": 740, + "selected_both_correct": 409, + "selected_both_wrong": 114 + }, + { + "budget": 0.08, + "threshold": 1.5329420566558838, + "operator": "ge", + "actual_fraction": 0.08688415446071904, + "metric": 0.1974112776278713, + "rescued": 386, + "harmed": 48, + "missed_rescue": 896, + "avoided_harm": 729, + "selected_both_correct": 472, + "selected_both_wrong": 138 + }, + { + "budget": 0.09, + "threshold": 1.380273461341858, + "operator": "ge", + "actual_fraction": 0.09587217043941411, + "metric": 0.20030220967092682, + "rescued": 421, + "harmed": 54, + "missed_rescue": 861, + "avoided_harm": 723, + "selected_both_correct": 529, + "selected_both_wrong": 148 + }, + { + "budget": 0.1, + "threshold": 1.32539701461792, + "operator": "ge", + "actual_fraction": 0.09920106524633822, + "metric": 0.20211074868345033, + "rescued": 432, + "harmed": 56, + "missed_rescue": 850, + "avoided_harm": 721, + "selected_both_correct": 552, + "selected_both_wrong": 152 + }, + { + "budget": 0.11, + "threshold": 1.2173664569854736, + "operator": "ge", + "actual_fraction": 0.11284953395472704, + "metric": 0.20834032699131655, + "rescued": 471, + "harmed": 62, + "missed_rescue": 811, + "avoided_harm": 715, + "selected_both_correct": 659, + "selected_both_wrong": 164 + }, + { + "budget": 0.12, + "threshold": 1.1719112396240234, + "operator": "ge", + "actual_fraction": 0.12350199733688415, + "metric": 0.21121674052765757, + "rescued": 503, + "harmed": 67, + "missed_rescue": 779, + "avoided_harm": 710, + "selected_both_correct": 745, + "selected_both_wrong": 169 + }, + { + "budget": 0.13, + "threshold": 1.1186922788619995, + "operator": "ge", + "actual_fraction": 0.13315579227696406, + "metric": 0.21276215593410724, + "rescued": 531, + "harmed": 77, + "missed_rescue": 751, + "avoided_harm": 700, + "selected_both_correct": 818, + "selected_both_wrong": 174 + }, + { + "budget": 0.14, + "threshold": 1.0748379230499268, + "operator": "ge", + "actual_fraction": 0.14347536617842876, + "metric": 0.21511521078428622, + "rescued": 556, + "harmed": 79, + "missed_rescue": 726, + "avoided_harm": 698, + "selected_both_correct": 907, + "selected_both_wrong": 182 + }, + { + "budget": 0.15, + "threshold": 1.0427184104919434, + "operator": "ge", + "actual_fraction": 0.15246338215712382, + "metric": 0.21738179676765032, + "rescued": 577, + "harmed": 81, + "missed_rescue": 705, + "avoided_harm": 696, + "selected_both_correct": 988, + "selected_both_wrong": 186 + }, + { + "budget": 0.16, + "threshold": 1.0111123323440552, + "operator": "ge", + "actual_fraction": 0.1604527296937417, + "metric": 0.22260454951009195, + "rescued": 597, + "harmed": 83, + "missed_rescue": 685, + "avoided_harm": 694, + "selected_both_correct": 1055, + "selected_both_wrong": 193 + }, + { + "budget": 0.17, + "threshold": 0.9669641852378845, + "operator": "ge", + "actual_fraction": 0.1780958721704394, + "metric": 0.22627631882842034, + "rescued": 641, + "harmed": 97, + "missed_rescue": 641, + "avoided_harm": 680, + "selected_both_correct": 1196, + "selected_both_wrong": 206 + }, + { + "budget": 0.18, + "threshold": 0.9490972757339478, + "operator": "ge", + "actual_fraction": 0.18442077230359522, + "metric": 0.2270967395967396, + "rescued": 657, + "harmed": 103, + "missed_rescue": 625, + "avoided_harm": 674, + "selected_both_correct": 1246, + "selected_both_wrong": 210 + }, + { + "budget": 0.19, + "threshold": 0.9165292382240295, + "operator": "ge", + "actual_fraction": 0.2000665778961385, + "metric": 0.23397114982462924, + "rescued": 697, + "harmed": 110, + "missed_rescue": 585, + "avoided_harm": 667, + "selected_both_correct": 1378, + "selected_both_wrong": 219 + }, + { + "budget": 0.2, + "threshold": 0.8978593349456787, + "operator": "ge", + "actual_fraction": 0.20439414114513982, + "metric": 0.23512276426997616, + "rescued": 707, + "harmed": 112, + "missed_rescue": 575, + "avoided_harm": 665, + "selected_both_correct": 1413, + "selected_both_wrong": 224 + }, + { + "budget": 0.21, + "threshold": 0.854847252368927, + "operator": "ge", + "actual_fraction": 0.21937416777629826, + "metric": 0.24202469545559713, + "rescued": 747, + "harmed": 119, + "missed_rescue": 535, + "avoided_harm": 658, + "selected_both_correct": 1539, + "selected_both_wrong": 231 + }, + { + "budget": 0.22, + "threshold": 0.8254434466362, + "operator": "ge", + "actual_fraction": 0.22769640479360853, + "metric": 0.24353835097564847, + "rescued": 766, + "harmed": 124, + "missed_rescue": 516, + "avoided_harm": 653, + "selected_both_correct": 1611, + "selected_both_wrong": 235 + }, + { + "budget": 0.23, + "threshold": 0.8046320080757141, + "operator": "ge", + "actual_fraction": 0.23601864181091878, + "metric": 0.2454657390209296, + "rescued": 787, + "harmed": 129, + "missed_rescue": 495, + "avoided_harm": 648, + "selected_both_correct": 1680, + "selected_both_wrong": 240 + }, + { + "budget": 0.24, + "threshold": 0.7761231660842896, + "operator": "ge", + "actual_fraction": 0.24533954727030627, + "metric": 0.2483414219175777, + "rescued": 812, + "harmed": 133, + "missed_rescue": 470, + "avoided_harm": 644, + "selected_both_correct": 1760, + "selected_both_wrong": 243 + }, + { + "budget": 0.25, + "threshold": 0.7452236413955688, + "operator": "ge", + "actual_fraction": 0.25432756324900135, + "metric": 0.2504846177578749, + "rescued": 831, + "harmed": 135, + "missed_rescue": 451, + "avoided_harm": 642, + "selected_both_correct": 1839, + "selected_both_wrong": 251 + }, + { + "budget": 0.26, + "threshold": 0.7167489528656006, + "operator": "ge", + "actual_fraction": 0.2636484687083888, + "metric": 0.252142216376585, + "rescued": 855, + "harmed": 144, + "missed_rescue": 427, + "avoided_harm": 633, + "selected_both_correct": 1914, + "selected_both_wrong": 255 + }, + { + "budget": 0.27, + "threshold": 0.6837625503540039, + "operator": "ge", + "actual_fraction": 0.275965379494008, + "metric": 0.25276746625574176, + "rescued": 880, + "harmed": 155, + "missed_rescue": 402, + "avoided_harm": 622, + "selected_both_correct": 2019, + "selected_both_wrong": 262 + }, + { + "budget": 0.28, + "threshold": 0.633089005947113, + "operator": "ge", + "actual_fraction": 0.29427430093209056, + "metric": 0.25702642690899347, + "rescued": 927, + "harmed": 172, + "missed_rescue": 355, + "avoided_harm": 605, + "selected_both_correct": 2167, + "selected_both_wrong": 270 + }, + { + "budget": 0.29, + "threshold": 0.5875619649887085, + "operator": "ge", + "actual_fraction": 0.31091877496671105, + "metric": 0.26183035419940204, + "rescued": 969, + "harmed": 193, + "missed_rescue": 313, + "avoided_harm": 584, + "selected_both_correct": 2296, + "selected_both_wrong": 278 + }, + { + "budget": 0.3, + "threshold": 0.5604347586631775, + "operator": "ge", + "actual_fraction": 0.322237017310253, + "metric": 0.2672887365132105, + "rescued": 995, + "harmed": 204, + "missed_rescue": 287, + "avoided_harm": 573, + "selected_both_correct": 2389, + "selected_both_wrong": 284 + }, + { + "budget": 0.31, + "threshold": 0.5320743918418884, + "operator": "ge", + "actual_fraction": 0.33388814913448733, + "metric": 0.27039536303602457, + "rescued": 1020, + "harmed": 216, + "missed_rescue": 262, + "avoided_harm": 561, + "selected_both_correct": 2480, + "selected_both_wrong": 296 + }, + { + "budget": 0.32, + "threshold": 0.49417412281036377, + "operator": "ge", + "actual_fraction": 0.34354194407456723, + "metric": 0.2713201570387918, + "rescued": 1038, + "harmed": 227, + "missed_rescue": 244, + "avoided_harm": 550, + "selected_both_correct": 2562, + "selected_both_wrong": 301 + }, + { + "budget": 0.33, + "threshold": 0.46743881702423096, + "operator": "ge", + "actual_fraction": 0.3492010652463382, + "metric": 0.2722481828097158, + "rescued": 1046, + "harmed": 230, + "missed_rescue": 236, + "avoided_harm": 547, + "selected_both_correct": 2610, + "selected_both_wrong": 310 + }, + { + "budget": 0.34, + "threshold": 0.4429023861885071, + "operator": "ge", + "actual_fraction": 0.3545272969374168, + "metric": 0.274417471554269, + "rescued": 1061, + "harmed": 239, + "missed_rescue": 221, + "avoided_harm": 538, + "selected_both_correct": 2648, + "selected_both_wrong": 312 + }, + { + "budget": 0.35, + "threshold": 0.40694165229797363, + "operator": "ge", + "actual_fraction": 0.3648468708388815, + "metric": 0.2782223498666452, + "rescued": 1081, + "harmed": 252, + "missed_rescue": 201, + "avoided_harm": 525, + "selected_both_correct": 2730, + "selected_both_wrong": 321 + }, + { + "budget": 0.36, + "threshold": 0.3611782193183899, + "operator": "ge", + "actual_fraction": 0.37217043941411454, + "metric": 0.2795152621397537, + "rescued": 1094, + "harmed": 257, + "missed_rescue": 188, + "avoided_harm": 520, + "selected_both_correct": 2793, + "selected_both_wrong": 328 + }, + { + "budget": 0.37, + "threshold": 0.3311936855316162, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.28090947933574756, + "rescued": 1104, + "harmed": 261, + "missed_rescue": 178, + "avoided_harm": 516, + "selected_both_correct": 2831, + "selected_both_wrong": 332 + }, + { + "budget": 0.38, + "threshold": 0.28899651765823364, + "operator": "ge", + "actual_fraction": 0.3854860186418109, + "metric": 0.28295009794869297, + "rescued": 1121, + "harmed": 269, + "missed_rescue": 161, + "avoided_harm": 508, + "selected_both_correct": 2904, + "selected_both_wrong": 338 + }, + { + "budget": 0.39, + "threshold": 0.19986218214035034, + "operator": "ge", + "actual_fraction": 0.39980026631158455, + "metric": 0.28644294895435674, + "rescued": 1145, + "harmed": 278, + "missed_rescue": 137, + "avoided_harm": 499, + "selected_both_correct": 3027, + "selected_both_wrong": 354 + }, + { + "budget": 0.4, + "threshold": 0.157886803150177, + "operator": "ge", + "actual_fraction": 0.4054593874833555, + "metric": 0.2862200047733393, + "rescued": 1160, + "harmed": 284, + "missed_rescue": 122, + "avoided_harm": 493, + "selected_both_correct": 3072, + "selected_both_wrong": 356 + }, + { + "budget": 0.41, + "threshold": 0.12043046951293945, + "operator": "ge", + "actual_fraction": 0.4104527296937417, + "metric": 0.28760386086589435, + "rescued": 1168, + "harmed": 287, + "missed_rescue": 114, + "avoided_harm": 490, + "selected_both_correct": 3115, + "selected_both_wrong": 362 + }, + { + "budget": 0.42, + "threshold": 0.06623327732086182, + "operator": "ge", + "actual_fraction": 0.4181091877496671, + "metric": 0.2930744270093899, + "rescued": 1181, + "harmed": 293, + "missed_rescue": 101, + "avoided_harm": 484, + "selected_both_correct": 3180, + "selected_both_wrong": 370 + }, + { + "budget": 0.43, + "threshold": -0.01827472448348999, + "operator": "ge", + "actual_fraction": 0.42809587217043943, + "metric": 0.2961685540536353, + "rescued": 1200, + "harmed": 302, + "missed_rescue": 82, + "avoided_harm": 475, + "selected_both_correct": 3262, + "selected_both_wrong": 380 + }, + { + "budget": 0.44, + "threshold": -0.14177680015563965, + "operator": "ge", + "actual_fraction": 0.4370838881491345, + "metric": 0.2977058723546182, + "rescued": 1215, + "harmed": 309, + "missed_rescue": 67, + "avoided_harm": 468, + "selected_both_correct": 3338, + "selected_both_wrong": 390 + }, + { + "budget": 0.45, + "threshold": -0.270923376083374, + "operator": "ge", + "actual_fraction": 0.44673768308921435, + "metric": 0.2999402430635043, + "rescued": 1230, + "harmed": 313, + "missed_rescue": 52, + "avoided_harm": 464, + "selected_both_correct": 3422, + "selected_both_wrong": 403 + }, + { + "budget": 0.46, + "threshold": -0.4213377833366394, + "operator": "ge", + "actual_fraction": 0.45206391478029295, + "metric": 0.2993046120720782, + "rescued": 1235, + "harmed": 316, + "missed_rescue": 47, + "avoided_harm": 461, + "selected_both_correct": 3475, + "selected_both_wrong": 406 + }, + { + "budget": 0.47, + "threshold": -0.6602996587753296, + "operator": "ge", + "actual_fraction": 0.45872170439414117, + "metric": 0.2998310238962732, + "rescued": 1243, + "harmed": 321, + "missed_rescue": 39, + "avoided_harm": 456, + "selected_both_correct": 3536, + "selected_both_wrong": 412 + }, + { + "budget": 0.48, + "threshold": -0.8612939715385437, + "operator": "ge", + "actual_fraction": 0.4673768308921438, + "metric": 0.30393211265289466, + "rescued": 1247, + "harmed": 326, + "missed_rescue": 35, + "avoided_harm": 451, + "selected_both_correct": 3623, + "selected_both_wrong": 420 + }, + { + "budget": 0.49, + "threshold": -0.9715782999992371, + "operator": "ge", + "actual_fraction": 0.47503328894806923, + "metric": 0.30350001249166053, + "rescued": 1248, + "harmed": 328, + "missed_rescue": 34, + "avoided_harm": 449, + "selected_both_correct": 3710, + "selected_both_wrong": 422 + }, + { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4876830892143808, + "metric": 0.29875800068449215, + "rescued": 1249, + "harmed": 337, + "missed_rescue": 33, + "avoided_harm": 440, + "selected_both_correct": 3843, + "selected_both_wrong": 431 + }, + { + "budget": 0.51, + "threshold": -1.6505446434020996, + "operator": "ge", + "actual_fraction": 0.5046604527296937, + "metric": 0.29989697064322696, + "rescued": 1255, + "harmed": 348, + "missed_rescue": 27, + "avoided_harm": 429, + "selected_both_correct": 4026, + "selected_both_wrong": 435 + }, + { + "budget": 0.52, + "threshold": -2.6140921115875244, + "operator": "ge", + "actual_fraction": 0.5206391478029294, + "metric": 0.297424793956044, + "rescued": 1255, + "harmed": 359, + "missed_rescue": 27, + "avoided_harm": 418, + "selected_both_correct": 4205, + "selected_both_wrong": 437 + }, + { + "budget": 0.53, + "threshold": -2.6907107830047607, + "operator": "ge", + "actual_fraction": 0.5249667110519307, + "metric": 0.297424793956044, + "rescued": 1255, + "harmed": 359, + "missed_rescue": 27, + "avoided_harm": 418, + "selected_both_correct": 4257, + "selected_both_wrong": 437 + }, + { + "budget": 0.54, + "threshold": -2.8344802856445312, + "operator": "ge", + "actual_fraction": 0.5436085219707057, + "metric": 0.2978194775414123, + "rescued": 1257, + "harmed": 377, + "missed_rescue": 25, + "avoided_harm": 400, + "selected_both_correct": 4461, + "selected_both_wrong": 437 + }, + { + "budget": 0.55, + "threshold": -2.905412197113037, + "operator": "ge", + "actual_fraction": 0.5529294274300932, + "metric": 0.29630990060928936, + "rescued": 1257, + "harmed": 385, + "missed_rescue": 25, + "avoided_harm": 392, + "selected_both_correct": 4565, + "selected_both_wrong": 437 + }, + { + "budget": 0.56, + "threshold": -2.9482638835906982, + "operator": "ge", + "actual_fraction": 0.5612516644474035, + "metric": 0.29578344205458756, + "rescued": 1257, + "harmed": 388, + "missed_rescue": 25, + "avoided_harm": 389, + "selected_both_correct": 4660, + "selected_both_wrong": 439 + }, + { + "budget": 0.57, + "threshold": -2.9957408905029297, + "operator": "ge", + "actual_fraction": 0.5692410119840213, + "metric": 0.2950949302773625, + "rescued": 1257, + "harmed": 396, + "missed_rescue": 25, + "avoided_harm": 381, + "selected_both_correct": 4748, + "selected_both_wrong": 439 + }, + { + "budget": 0.58, + "threshold": -3.1037726402282715, + "operator": "ge", + "actual_fraction": 0.5902130492676432, + "metric": 0.2947373376338526, + "rescued": 1260, + "harmed": 407, + "missed_rescue": 22, + "avoided_harm": 370, + "selected_both_correct": 4985, + "selected_both_wrong": 440 + }, + { + "budget": 0.59, + "threshold": -3.1757895946502686, + "operator": "ge", + "actual_fraction": 0.6021970705725699, + "metric": 0.29722091277674884, + "rescued": 1261, + "harmed": 417, + "missed_rescue": 21, + "avoided_harm": 360, + "selected_both_correct": 5115, + "selected_both_wrong": 443 + }, + { + "budget": 0.6, + "threshold": -3.253206729888916, + "operator": "ge", + "actual_fraction": 0.6208388814913449, + "metric": 0.29665021956289706, + "rescued": 1264, + "harmed": 430, + "missed_rescue": 18, + "avoided_harm": 347, + "selected_both_correct": 5323, + "selected_both_wrong": 443 + }, + { + "budget": 0.61, + "threshold": -3.2825026512145996, + "operator": "ge", + "actual_fraction": 0.6288282290279628, + "metric": 0.295988506182091, + "rescued": 1264, + "harmed": 435, + "missed_rescue": 18, + "avoided_harm": 342, + "selected_both_correct": 5413, + "selected_both_wrong": 444 + }, + { + "budget": 0.62, + "threshold": -3.322240114212036, + "operator": "ge", + "actual_fraction": 0.6421438082556591, + "metric": 0.2939669175161438, + "rescued": 1264, + "harmed": 452, + "missed_rescue": 18, + "avoided_harm": 325, + "selected_both_correct": 5556, + "selected_both_wrong": 444 + }, + { + "budget": 0.63, + "threshold": -3.3503003120422363, + "operator": "ge", + "actual_fraction": 0.6491344873501997, + "metric": 0.29277362668991463, + "rescued": 1265, + "harmed": 463, + "missed_rescue": 17, + "avoided_harm": 314, + "selected_both_correct": 5628, + "selected_both_wrong": 444 + }, + { + "budget": 0.64, + "threshold": -3.377089738845825, + "operator": "ge", + "actual_fraction": 0.6571238348868176, + "metric": 0.2930821542583563, + "rescued": 1266, + "harmed": 468, + "missed_rescue": 16, + "avoided_harm": 309, + "selected_both_correct": 5718, + "selected_both_wrong": 444 + }, + { + "budget": 0.65, + "threshold": -3.4055864810943604, + "operator": "ge", + "actual_fraction": 0.6677762982689747, + "metric": 0.2892305220657417, + "rescued": 1266, + "harmed": 488, + "missed_rescue": 16, + "avoided_harm": 289, + "selected_both_correct": 5826, + "selected_both_wrong": 444 + }, + { + "budget": 0.66, + "threshold": -3.430248498916626, + "operator": "ge", + "actual_fraction": 0.6744340878828229, + "metric": 0.2915915088317972, + "rescued": 1267, + "harmed": 492, + "missed_rescue": 15, + "avoided_harm": 285, + "selected_both_correct": 5901, + "selected_both_wrong": 444 + }, + { + "budget": 0.67, + "threshold": -3.4584414958953857, + "operator": "ge", + "actual_fraction": 0.6854194407456724, + "metric": 0.2936540392680743, + "rescued": 1269, + "harmed": 501, + "missed_rescue": 13, + "avoided_harm": 276, + "selected_both_correct": 6022, + "selected_both_wrong": 444 + }, + { + "budget": 0.68, + "threshold": -3.502922773361206, + "operator": "ge", + "actual_fraction": 0.7047270306258322, + "metric": 0.29632332439424064, + "rescued": 1272, + "harmed": 523, + "missed_rescue": 10, + "avoided_harm": 254, + "selected_both_correct": 6229, + "selected_both_wrong": 444 + }, + { + "budget": 0.69, + "threshold": -3.532865285873413, + "operator": "ge", + "actual_fraction": 0.7157123834886817, + "metric": 0.2937246709263134, + "rescued": 1273, + "harmed": 540, + "missed_rescue": 9, + "avoided_harm": 237, + "selected_both_correct": 6342, + "selected_both_wrong": 445 + }, + { + "budget": 0.7, + "threshold": -3.5550942420959473, + "operator": "ge", + "actual_fraction": 0.7253661784287616, + "metric": 0.2952254059747369, + "rescued": 1274, + "harmed": 550, + "missed_rescue": 8, + "avoided_harm": 227, + "selected_both_correct": 6446, + "selected_both_wrong": 446 + }, + { + "budget": 0.71, + "threshold": -3.566344976425171, + "operator": "ge", + "actual_fraction": 0.7283621837549934, + "metric": 0.2945683782087344, + "rescued": 1274, + "harmed": 553, + "missed_rescue": 8, + "avoided_harm": 224, + "selected_both_correct": 6478, + "selected_both_wrong": 447 + }, + { + "budget": 0.72, + "threshold": -3.6056067943573, + "operator": "ge", + "actual_fraction": 0.741011984021305, + "metric": 0.29252963134197957, + "rescued": 1275, + "harmed": 568, + "missed_rescue": 7, + "avoided_harm": 209, + "selected_both_correct": 6614, + "selected_both_wrong": 447 + }, + { + "budget": 0.73, + "threshold": -3.628530979156494, + "operator": "ge", + "actual_fraction": 0.7526631158455392, + "metric": 0.2900395106945919, + "rescued": 1276, + "harmed": 588, + "missed_rescue": 6, + "avoided_harm": 189, + "selected_both_correct": 6733, + "selected_both_wrong": 447 + }, + { + "budget": 0.74, + "threshold": -3.6580705642700195, + "operator": "ge", + "actual_fraction": 0.7616511318242344, + "metric": 0.2887744568626224, + "rescued": 1276, + "harmed": 597, + "missed_rescue": 6, + "avoided_harm": 180, + "selected_both_correct": 6832, + "selected_both_wrong": 447 + }, + { + "budget": 0.75, + "threshold": -3.6900978088378906, + "operator": "ge", + "actual_fraction": 0.7709720372836218, + "metric": 0.28730364578560735, + "rescued": 1276, + "harmed": 606, + "missed_rescue": 6, + "avoided_harm": 171, + "selected_both_correct": 6935, + "selected_both_wrong": 447 + }, + { + "budget": 0.76, + "threshold": -3.7050747871398926, + "operator": "ge", + "actual_fraction": 0.7749667110519307, + "metric": 0.2858070539975931, + "rescued": 1276, + "harmed": 612, + "missed_rescue": 6, + "avoided_harm": 165, + "selected_both_correct": 6977, + "selected_both_wrong": 447 + }, + { + "budget": 0.77, + "threshold": -3.720116138458252, + "operator": "ge", + "actual_fraction": 0.7792942743009321, + "metric": 0.28488735368132817, + "rescued": 1276, + "harmed": 617, + "missed_rescue": 6, + "avoided_harm": 160, + "selected_both_correct": 7023, + "selected_both_wrong": 448 + }, + { + "budget": 0.78, + "threshold": -3.7399919033050537, + "operator": "ge", + "actual_fraction": 0.7846205059920106, + "metric": 0.2830658409372236, + "rescued": 1276, + "harmed": 627, + "missed_rescue": 6, + "avoided_harm": 150, + "selected_both_correct": 7077, + "selected_both_wrong": 448 + }, + { + "budget": 0.79, + "threshold": -3.799569606781006, + "operator": "ge", + "actual_fraction": 0.7972703062583223, + "metric": 0.2809995077380739, + "rescued": 1277, + "harmed": 643, + "missed_rescue": 5, + "avoided_harm": 134, + "selected_both_correct": 7212, + "selected_both_wrong": 448 + }, + { + "budget": 0.8, + "threshold": -3.8980000019073486, + "operator": "ge", + "actual_fraction": 0.8129161118508655, + "metric": 0.28013559922601305, + "rescued": 1278, + "harmed": 667, + "missed_rescue": 4, + "avoided_harm": 110, + "selected_both_correct": 7375, + "selected_both_wrong": 448 + }, + { + "budget": 0.81, + "threshold": -4.0607757568359375, + "operator": "ge", + "actual_fraction": 0.8199067909454061, + "metric": 0.27814841049220523, + "rescued": 1278, + "harmed": 681, + "missed_rescue": 4, + "avoided_harm": 96, + "selected_both_correct": 7445, + "selected_both_wrong": 448 + }, + { + "budget": 0.82, + "threshold": -8.964031219482422, + "operator": "ge", + "actual_fraction": 0.831225033288948, + "metric": 0.2772348763322643, + "rescued": 1278, + "harmed": 686, + "missed_rescue": 4, + "avoided_harm": 91, + "selected_both_correct": 7576, + "selected_both_wrong": 448 + }, + { + "budget": 0.83, + "threshold": -9.02132511138916, + "operator": "ge", + "actual_fraction": 0.84254327563249, + "metric": 0.27700595116508253, + "rescued": 1279, + "harmed": 689, + "missed_rescue": 3, + "avoided_harm": 88, + "selected_both_correct": 7708, + "selected_both_wrong": 448 + }, + { + "budget": 0.84, + "threshold": -9.09000015258789, + "operator": "ge", + "actual_fraction": 0.857190412782956, + "metric": 0.2763733409861609, + "rescued": 1279, + "harmed": 692, + "missed_rescue": 3, + "avoided_harm": 85, + "selected_both_correct": 7881, + "selected_both_wrong": 448 + }, + { + "budget": 0.85, + "threshold": -9.12938404083252, + "operator": "ge", + "actual_fraction": 0.8618508655126498, + "metric": 0.2761345157638265, + "rescued": 1279, + "harmed": 694, + "missed_rescue": 3, + "avoided_harm": 83, + "selected_both_correct": 7935, + "selected_both_wrong": 448 + }, + { + "budget": 0.86, + "threshold": -9.17154312133789, + "operator": "ge", + "actual_fraction": 0.87017310252996, + "metric": 0.2761345157638265, + "rescued": 1279, + "harmed": 694, + "missed_rescue": 3, + "avoided_harm": 83, + "selected_both_correct": 8035, + "selected_both_wrong": 448 + }, + { + "budget": 0.87, + "threshold": -9.273024559020996, + "operator": "ge", + "actual_fraction": 0.8848202396804261, + "metric": 0.27546832609774896, + "rescued": 1279, + "harmed": 699, + "missed_rescue": 3, + "avoided_harm": 78, + "selected_both_correct": 8206, + "selected_both_wrong": 448 + }, + { + "budget": 0.88, + "threshold": -9.444879531860352, + "operator": "ge", + "actual_fraction": 0.8984687083888149, + "metric": 0.27423981143437465, + "rescued": 1279, + "harmed": 709, + "missed_rescue": 3, + "avoided_harm": 68, + "selected_both_correct": 8360, + "selected_both_wrong": 448 + }, + { + "budget": 0.89, + "threshold": -9.566991806030273, + "operator": "ge", + "actual_fraction": 0.9087882822902796, + "metric": 0.2736575311942959, + "rescued": 1279, + "harmed": 713, + "missed_rescue": 3, + "avoided_harm": 64, + "selected_both_correct": 8480, + "selected_both_wrong": 448 + }, + { + "budget": 0.9, + "threshold": -9.675031661987305, + "operator": "ge", + "actual_fraction": 0.9161118508655126, + "metric": 0.272975172975173, + "rescued": 1279, + "harmed": 717, + "missed_rescue": 3, + "avoided_harm": 60, + "selected_both_correct": 8563, + "selected_both_wrong": 449 + }, + { + "budget": 0.91, + "threshold": -9.739017486572266, + "operator": "ge", + "actual_fraction": 0.9217709720372836, + "metric": 0.2727190231806067, + "rescued": 1279, + "harmed": 719, + "missed_rescue": 3, + "avoided_harm": 58, + "selected_both_correct": 8629, + "selected_both_wrong": 449 + }, + { + "budget": 0.92, + "threshold": -9.787779808044434, + "operator": "ge", + "actual_fraction": 0.9264314247669774, + "metric": 0.27264729817760425, + "rescued": 1279, + "harmed": 720, + "missed_rescue": 3, + "avoided_harm": 57, + "selected_both_correct": 8684, + "selected_both_wrong": 449 + }, + { + "budget": 0.93, + "threshold": -9.832305908203125, + "operator": "ge", + "actual_fraction": 0.933422103861518, + "metric": 0.2741441928343702, + "rescued": 1280, + "harmed": 724, + "missed_rescue": 2, + "avoided_harm": 53, + "selected_both_correct": 8763, + "selected_both_wrong": 449 + }, + { + "budget": 0.94, + "threshold": -9.88642692565918, + "operator": "ge", + "actual_fraction": 0.9417443408788282, + "metric": 0.2733716828250215, + "rescued": 1280, + "harmed": 728, + "missed_rescue": 2, + "avoided_harm": 49, + "selected_both_correct": 8859, + "selected_both_wrong": 449 + }, + { + "budget": 0.95, + "threshold": -9.976344108581543, + "operator": "ge", + "actual_fraction": 0.9530625832223701, + "metric": 0.2727312493845365, + "rescued": 1280, + "harmed": 737, + "missed_rescue": 2, + "avoided_harm": 40, + "selected_both_correct": 8986, + "selected_both_wrong": 449 + }, + { + "budget": 0.96, + "threshold": -10.03441047668457, + "operator": "ge", + "actual_fraction": 0.9613848202396804, + "metric": 0.27223682114986464, + "rescued": 1280, + "harmed": 740, + "missed_rescue": 2, + "avoided_harm": 37, + "selected_both_correct": 9083, + "selected_both_wrong": 449 + }, + { + "budget": 0.97, + "threshold": -10.085777282714844, + "operator": "ge", + "actual_fraction": 0.9673768308921438, + "metric": 0.27453093162048386, + "rescued": 1282, + "harmed": 744, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 9149, + "selected_both_wrong": 449 + }, + { + "budget": 0.98, + "threshold": -10.1488676071167, + "operator": "ge", + "actual_fraction": 0.9760319573901465, + "metric": 0.2737331615154065, + "rescued": 1282, + "harmed": 750, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 9247, + "selected_both_wrong": 449 + }, + { + "budget": 0.99, + "threshold": -10.265552520751953, + "operator": "ge", + "actual_fraction": 0.9890146471371505, + "metric": 0.2725383412883413, + "rescued": 1282, + "harmed": 758, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 9395, + "selected_both_wrong": 449 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 2189, + "rescuable": 307, + "harmful": 220, + "both_wrong": 88, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 9508, + "rescuable": 1282, + "harmful": 777, + "both_wrong": 449, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.18254072168965788, + "test": 0.18129624718318377 + }, + "large": { + "validation": 0.25448626222520154, + "test": 0.27001009067886245 + }, + "learned_minus_confidence_test": 0.03900900020828163, + "learned_minus_small_test": 0.11746175350130839, + "learned_minus_large_test": 0.028747910005629707 + } + ] +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/launch.sh b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/launch.sh new file mode 100644 index 0000000000000000000000000000000000000000..2d1c81b09ce00216016922f994ee0d7578a72dcc --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/launch.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail +OUT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_20260912 +WORK=/tmp/codex_task_main_cost_20260912 +mkdir -p "$OUT/logs" "$OUT/components" +bash -n "$WORK/run_large.sh" +python -m py_compile "$WORK/measure_large.py" "$WORK/measure_flops.py" "$WORK/measure_specialist.py" +nohup bash "$WORK/run_large.sh" 0 MiniCPM grefcoco minicpm > "$OUT/logs/minicpm.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/minicpm.pid" +nohup bash "$WORK/run_large.sh" 1 Qwen3.8 construction qwen > "$OUT/logs/qwen.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/qwen.pid" +cat "$OUT/logs/minicpm.pid" "$OUT/logs/qwen.pid" diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/launch_small.sh b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/launch_small.sh new file mode 100644 index 0000000000000000000000000000000000000000..72bb571f0e76e178cb624d31546b29dc6b97cc59 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/launch_small.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +set -euo pipefail +WORK=/tmp/codex_task_main_cost_20260912 +OUT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_20260912 +bash -n "$WORK/run_small.sh" +python -m py_compile "$WORK/measure_router.py" +nohup bash "$WORK/run_small.sh" 0 instancevg MiniCPM minicpm > "$OUT/logs/instancevg_small.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/instancevg_small.pid" +nohup bash "$WORK/run_small.sh" 1 yolo26x Qwen3.8 qwen > "$OUT/logs/yolo26x_small.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/yolo26x_small.pid" +cat "$OUT/logs/instancevg_small.pid" "$OUT/logs/yolo26x_small.pid" diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_flops.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_flops.py new file mode 100644 index 0000000000000000000000000000000000000000..4b5eb79777bb9eca67245173f1c6813f000b8e8d --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_flops.py @@ -0,0 +1,152 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import json +import re +from pathlib import Path + +import numpy as np +import torch +from PIL import Image +from torch.utils.flop_counter import FlopCounterMode + + +def count(call): + with torch.inference_mode(), FlopCounterMode(display=False) as counter: + value = call() + torch.cuda.synchronize() + counts = counter.get_flop_counts().get('Global', {}) + return int(counter.get_total_flops()), value, {str(k): int(v) for k, v in counts.items()} + + +def make_text(root: Path, model_kind: str, task: str, row: dict, processor): + prompt = (root / 'outputs/large_model_labels_all_20260909/prompts' / task / f'{model_kind}.txt').read_text(encoding='utf-8').strip() + if task == 'grefcoco': + prompt = prompt.replace('{expression}', row['expression']) + if model_kind == 'Qwen3.8': + content = [{'type': 'image'}, {'type': 'text', 'text': '\n'}, {'type': 'text', 'text': prompt}] + return processor.apply_chat_template([{'role': 'user', 'content': content}], tokenize=False, add_generation_prompt=True) + message = '(./)\n' + prompt + return processor.tokenizer.apply_chat_template([{'role': 'user', 'content': message}], tokenize=False, add_generation_prompt=True, enable_thinking=False) + + +def prepare(root: Path, model_kind: str, task: str, row: dict, processor): + image = Image.open(row['image_path']).convert('RGB') + text = make_text(root, model_kind, task, row, processor) + if model_kind == 'Qwen3.8': + data = processor(text=[text], images=[image], return_tensors='pt', size={'shortest_edge': 65536, 'longest_edge': 1003520}) + else: + data = processor([text], [[image]], return_tensors='pt', max_length=16384) + image.close() + return data + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--model-kind', choices=['Qwen3.8', 'MiniCPM'], required=True) + parser.add_argument('--task', choices=['grefcoco', 'construction'], required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--samples', type=int, default=4) + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + expert = 'instancevg' if args.task == 'grefcoco' else 'yolo26x' + rows = [json.loads(line) for line in (args.root / f'outputs/router_full_20260910/data/{expert}/test/records.jsonl').open(encoding='utf-8')] + if args.samples > len(rows): + raise ValueError('sample count exceeds test rows') + selected_indices = np.random.default_rng(20260911).choice(len(rows), args.samples, replace=False) + selected = [rows[int(index)] for index in selected_indices] + checkpoint_root = args.root.parents[1] / 'checkpoints' + if not checkpoint_root.exists(): + checkpoint_root = args.root / 'checkpoints' + model_path = checkpoint_root / ('Qwen3.8-27B-FP8' if args.model_kind == 'Qwen3.8' else 'MiniCPM-V-4_5') + from transformers import AutoProcessor + processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + if args.model_kind == 'Qwen3.8': + from transformers import FineGrainedFP8Config, Qwen3_5ForConditionalGeneration + import transformers.models.qwen3_5.modeling_qwen3_5 as implementation + implementation.causal_conv1d_fn = None + implementation.causal_conv1d_update = None + implementation.chunk_gated_delta_rule = None + implementation.fused_recurrent_gated_delta_rule = None + implementation.FusedRMSNormGated = None + implementation.is_fast_path_available = False + model = Qwen3_5ForConditionalGeneration.from_pretrained( + model_path, local_files_only=True, dtype=torch.bfloat16, device_map={'': 'cuda:0'}, + attn_implementation='eager', quantization_config=FineGrainedFP8Config(dequantize=False), + ) + else: + model = processor # replaced below to keep imports local to the selected environment + from transformers import AutoModel + from transformers.modeling_utils import PreTrainedModel + if not hasattr(PreTrainedModel, 'all_tied_weights_keys'): + def get_tied_weights(self): + return getattr(self, '_all_tied_weights_keys', getattr(self, '_tied_weights_keys', {}) or {}) + def set_tied_weights(self, value): + self._all_tied_weights_keys = value + PreTrainedModel.all_tied_weights_keys = property(get_tied_weights, set_tied_weights) + model = AutoModel.from_pretrained( + model_path, trust_remote_code=True, local_files_only=True, torch_dtype=torch.bfloat16, + device_map={'': 'cuda:0'}, attn_implementation='eager', + ) + torch.set_num_threads(4) + torch.backends.mha.set_fastpath_enabled(False) + model.eval().requires_grad_(False) + records = [] + for row in selected: + data = prepare(args.root, args.model_kind, args.task, row, processor).to('cuda') + prompt_tokens = int(data['input_ids'].shape[-1]) + if args.model_kind == 'MiniCPM': + vision_flops, embedding_result, vision_ops = count(lambda: model.get_vllm_embedding(data)) + embeddings, _ = embedding_result + prefill_flops, prefill_output, prefill_ops = count(lambda: model.llm(inputs_embeds=embeddings, attention_mask=data['attention_mask'], use_cache=True, logits_to_keep=1)) + prefill_flops += vision_flops + prefill_ops.update({'vision:'+k:v for k,v in vision_ops.items()}) + length = int(data['input_ids'].shape[-1]) + known_token = data['input_ids'][:, -1:] + kwargs = { + 'input_ids': known_token, + 'past_key_values': prefill_output.past_key_values, + 'attention_mask': torch.ones((1, length + 1), device='cuda', dtype=torch.long), + 'use_cache': True, + 'logits_to_keep': 1, + } + decode_flops, _, decode_ops = count(lambda: model.llm(**kwargs)) + else: + prefill_flops, prefill_output, prefill_ops = count(lambda: model(**data, use_cache=True, logits_to_keep=1)) + length = int(data['input_ids'].shape[-1]) + known_token = data['input_ids'][:, -1:] + kwargs = { + 'input_ids': known_token, + 'past_key_values': prefill_output.past_key_values, + 'attention_mask': torch.ones((1, length + 1), device='cuda', dtype=torch.long), + 'use_cache': True, + 'logits_to_keep': 1, + 'cache_position': torch.tensor([length], device='cuda'), + } + kwargs.pop('attention_mask') + decode_flops, _, decode_ops = count(lambda: model(**kwargs)) + records.append({ + 'sample_id': row['sample_id'], 'prompt_tokens_in_reference_tensor': prompt_tokens, + 'input_shapes': {key: list(value.shape) for key, value in data.items() if hasattr(value, 'shape')}, + 'prefill_flops_counted': prefill_flops, 'decode_one_token_flops_counted': decode_flops, + 'prefill_gflops_counted': prefill_flops / 1e9, 'decode_one_token_gflops_counted': decode_flops / 1e9, + 'prefill_operators': prefill_ops, 'decode_operators': decode_ops, + }) + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps({'status': 'running', 'model_kind': args.model_kind, 'task': args.task, 'records': records}, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'completed': len(records), 'sample_id': row['sample_id'], 'prefill_gflops': prefill_flops / 1e9, 'decode_gflops': decode_flops / 1e9}), flush=True) + args.output.write_text(json.dumps({ + 'status': 'complete', 'model_kind': args.model_kind, 'task': args.task, 'model_path': str(model_path), + 'sample_count': len(records), 'records': records, + 'mean_prefill_gflops_counted': float(np.mean([row['prefill_gflops_counted'] for row in records])), + 'mean_decode_one_token_gflops_counted': float(np.mean([row['decode_one_token_gflops_counted'] for row in records])), + 'scope': 'HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.', + 'answer_generation': 'not performed; one known-token decode probe only', + }, ensure_ascii=False, indent=2), encoding='utf-8') + + +if __name__ == '__main__': + main() diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_large.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_large.py new file mode 100644 index 0000000000000000000000000000000000000000..61b54474c6d7e15ee107c15c3ca4eb10c14ebca3 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_large.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import json +import time +from pathlib import Path + +import numpy as np +import jsonschema +from PIL import Image +from vllm import LLM, SamplingParams +from vllm.sampling_params import StructuredOutputsParams + + +EVENTS = [ + 'rule_1_ppe_violation', + 'rule_2_fall_protection_violation', + 'rule_3_unprotected_edge_violation', + 'rule_4_excavator_proximity_violation', +] + + +def gref_schema(): + return { + 'type': 'object', + 'properties': { + 'objects': { + 'type': 'array', 'maxItems': 18, + 'items': { + 'type': 'object', + 'properties': { + 'bbox_2d': {'type': 'array', 'items': {'type': 'number', 'minimum': 0, 'maximum': 1000}, 'minItems': 4, 'maxItems': 4}, + 'score': {'type': 'number', 'minimum': 0, 'maximum': 1}, + }, + 'required': ['bbox_2d', 'score'], 'additionalProperties': False, + }, + }, + }, + 'required': ['objects'], 'additionalProperties': False, + } + + +def construction_schema(): + box = {'type': 'array', 'items': {'type': 'integer', 'minimum': 0, 'maximum': 1000}, 'minItems': 4, 'maxItems': 4} + max_boxes = [13, 6, 3, 2] + events = {} + for event, limit in zip(EVENTS, max_boxes): + events[event] = { + 'type': 'object', + 'properties': { + 'present': {'type': 'boolean'}, + 'reason': {'type': ['string', 'null']}, + 'boxes': {'type': 'array', 'items': box, 'maxItems': limit}, + }, + 'required': ['present', 'reason', 'boxes'], 'additionalProperties': False, + } + return {'type': 'object', 'properties': {'events': {'type': 'object', 'properties': events, 'required': EVENTS, 'additionalProperties': False}}, 'required': ['events'], 'additionalProperties': False} + + +def make_prompt(root: Path, model_kind: str, task: str, row: dict, processor_or_tokenizer): + prompt = (root / 'outputs/large_model_labels_all_20260909/prompts' / task / f'{model_kind}.txt').read_text(encoding='utf-8').strip() + if task == 'grefcoco': + prompt = prompt.replace('{expression}', row['expression']) + if model_kind == 'Qwen3.8': + converted = [{'type': 'image'}] + converted.append({'type': 'text', 'text': '\n'}) + converted.append({'type': 'text', 'text': prompt}) + messages = [{'role': 'user', 'content': converted}] + return processor_or_tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + message = '(./)\n' + prompt + return processor_or_tokenizer.apply_chat_template([{'role': 'user', 'content': message}], tokenize=False, add_generation_prompt=True, enable_thinking=False) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--model-kind', choices=['Qwen3.8', 'MiniCPM'], required=True) + parser.add_argument('--task', choices=['grefcoco', 'construction'], required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--samples', type=int, default=32) + parser.add_argument('--batch-size', type=int, default=8) + parser.add_argument('--gpu-memory-utilization', type=float, default=0.90) + parser.add_argument('--max-model-len', type=int, default=16384) + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + if args.samples < 3 or args.batch_size < 1: + raise ValueError('samples must include warmup rows and batch size must be positive') + expert = 'instancevg' if args.task == 'grefcoco' else 'yolo26x' + rows = [json.loads(line) for line in (args.root / f'outputs/router_full_20260910/data/{expert}/test/records.jsonl').open(encoding='utf-8')] + if args.samples > len(rows): + raise ValueError(f'samples {args.samples} exceeds {len(rows)}') + selected_indices = np.random.default_rng(20260911).choice(len(rows), args.samples, replace=False) + selected = [rows[int(index)] for index in selected_indices] + checkpoint_root = args.root.parents[1] / 'checkpoints' + if not checkpoint_root.exists(): + checkpoint_root = args.root / 'checkpoints' + model_path = checkpoint_root / ('Qwen3.8-27B-FP8' if args.model_kind == 'Qwen3.8' else 'MiniCPM-V-4_5') + if args.model_kind == 'Qwen3.8': + from transformers import AutoProcessor + processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + model_kwargs = {'mm_processor_kwargs': {'size': {'shortest_edge': 65536, 'longest_edge': 1003520}}} + else: + from transformers import AutoTokenizer + processor = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + model_kwargs = {} + model = LLM(model=str(model_path), trust_remote_code=True, tensor_parallel_size=1, max_model_len=args.max_model_len, + limit_mm_per_prompt={'image': 1}, max_num_seqs=args.batch_size, + gpu_memory_utilization=args.gpu_memory_utilization, enforce_eager=True, **model_kwargs) + schema = gref_schema() if args.task == 'grefcoco' else construction_schema() + sampling = SamplingParams(temperature=0, max_tokens=512 if args.task == 'grefcoco' else 768, + structured_outputs=StructuredOutputsParams(json=schema, disable_additional_properties=True)) + + def generate(batch_rows): + images = [] + requests = [] + for row in batch_rows: + required = {'sample_id', 'image_path'} + if args.task == 'grefcoco': + required.add('expression') + missing = required - row.keys() + if missing: + raise ValueError(f'missing fields {sorted(missing)} for {row["sample_id"]}') + image = Image.open(row['image_path']).convert('RGB') + images.append(image) + requests.append({'prompt': make_prompt(args.root, args.model_kind, args.task, row, processor), 'multi_modal_data': {'image': image}}) + started = time.perf_counter() + generated = model.generate(requests, sampling, use_tqdm=False) + elapsed = time.perf_counter() - started + for image in images: + image.close() + if len(generated) != len(batch_rows): + raise ValueError('vLLM returned an unexpected number of results') + return elapsed, generated + + warmup_count = min(2, len(selected) - 1) + for row in selected[:warmup_count]: + generate([row]) + measured = selected[warmup_count:] + batches = [] + for start in range(0, len(measured), args.batch_size): + batch_rows = measured[start:start + args.batch_size] + batch_started = time.perf_counter() + elapsed, generated = generate(batch_rows) + wall = time.perf_counter() - batch_started + output_records = [] + for result in generated: + raw = result.outputs[0].text + error = None + try: + parsed = json.loads(raw) + jsonschema.validate(parsed, schema) + boxes = [o['bbox_2d'] for o in parsed['objects']] if args.task == 'grefcoco' else [b for e in parsed['events'].values() for b in e['boxes']] + if any(b[0] >= b[2] or b[1] >= b[3] for b in boxes): + raise ValueError('non-positive box size') + except (json.JSONDecodeError, jsonschema.ValidationError, ValueError) as exc: + parsed = None + error = str(exc) + output_records.append({'raw_response':raw,'valid_prediction':parsed,'error':error, + 'finish_reason':result.outputs[0].finish_reason,'completion_tokens':len(result.outputs[0].token_ids), + 'prompt_tokens':len(result.prompt_token_ids)}) + item = { + 'sample_ids': [row['sample_id'] for row in batch_rows], + 'batch_size': len(batch_rows), + 'generation_latency_seconds': elapsed, + 'batch_wall_latency_seconds': wall, + 'amortized_generation_ms_per_sample': elapsed * 1000 / len(batch_rows), + 'amortized_batch_wall_ms_per_sample': wall * 1000 / len(batch_rows), + 'outputs': [ + { + 'finish_reason': result.outputs[0].finish_reason, + 'stop_reason': result.outputs[0].stop_reason, + 'prompt_tokens': len(result.prompt_token_ids) if result.prompt_token_ids is not None else None, + 'completion_tokens': len(result.outputs[0].token_ids), + 'text_characters': len(result.outputs[0].text), + } + for result in generated + ], + } + batches.append(item) + item['outputs'] = output_records + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps({ + 'status': 'running', 'model_kind': args.model_kind, 'task': args.task, 'model_path': str(model_path), + 'sample_count_requested': args.samples, 'warmup_count': warmup_count, 'batch_size': args.batch_size, + 'sample_ids': [row['sample_id'] for row in selected], 'batches': batches, + }, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'completed_samples': sum(item['batch_size'] for item in batches), 'total_measured': len(measured), 'batch_latency_seconds': elapsed}), flush=True) + all_ms = [x['amortized_generation_ms_per_sample'] for x in batches] + all_wall_ms = [x['amortized_batch_wall_ms_per_sample'] for x in batches] + output = { + 'status': 'complete', 'model_kind': args.model_kind, 'task': args.task, 'expert': expert, + 'model_path': str(model_path), 'device': 'cuda:0', 'gpu_memory_utilization': args.gpu_memory_utilization, + 'sample_count_requested': args.samples, 'warmup_count': warmup_count, 'measured_count': len(measured), + 'batch_size': args.batch_size, 'sample_ids': [row['sample_id'] for row in selected], 'batches': batches, + 'mean_generation_ms_per_sample': float(np.mean(all_ms)), 'median_generation_ms_per_sample': float(np.median(all_ms)), + 'mean_batch_wall_ms_per_sample': sum(b['batch_wall_latency_seconds'] for b in batches)*1000/len(measured), 'median_batch_wall_ms_per_sample': float(np.median(all_wall_ms)), + 'mean_prompt_tokens': float(np.mean([x['prompt_tokens'] for batch in batches for x in batch['outputs'] if x['prompt_tokens'] is not None])), + 'mean_completion_tokens': float(np.mean([x['completion_tokens'] for batch in batches for x in batch['outputs']])), + 'latency_scope': 'A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.', + 'flops_scope': 'This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs.', + } + args.output.write_text(json.dumps(output, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'status': 'complete', 'mean_generation_ms_per_sample': output['mean_generation_ms_per_sample'], 'mean_completion_tokens': output['mean_completion_tokens']}), flush=True) + + +if __name__ == '__main__': + main() diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_router.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_router.py new file mode 100644 index 0000000000000000000000000000000000000000..192677ef8450d4b93944d341658a5a745e82b4c8 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_router.py @@ -0,0 +1,104 @@ +from pathlib import Path +import argparse +import json +import sys +import time +import numpy as np +import torch +from PIL import Image +from torch.utils.flop_counter import FlopCounterMode + +p=argparse.ArgumentParser() +p.add_argument('--root',type=Path,required=True) +p.add_argument('--expert',required=True) +p.add_argument('--endpoint',required=True) +p.add_argument('--output',type=Path,required=True) +a=p.parse_args() +sys.path.insert(0,str(a.root/'code')) +from routing import train_router_formal as formal +from routing.router import prepare_images,load_distilled_router +from siglip_distillation.mini_siglip import MiniSiglip +from tokenizers import Tokenizer +torch.set_num_threads(4) +torch.backends.cuda.matmul.allow_tf32=False +torch.backends.mha.set_fastpath_enabled(False) +work=a.root/'outputs/router_shared_image_hidden_20260910' +run=f'shared_12_{a.expert}_{a.endpoint}' +dest=work/'runs'/run +cfg=json.loads((dest/'config.json').read_text()) +assert cfg['seed']==42 and cfg['student']=='8M' and cfg['target']=='four_state' +records,data=formal.load_data(cfg,'test',torch.device('cuda')) +for key in ['hidden','confidence']: + norm=np.load(dest/(key+'_normalization.npz')) + data[key]=(data[key]-torch.from_numpy(norm['mean']).cuda())/torch.from_numpy(norm['std']).cuda() +model=formal.FeatureRouter(cfg,data['hidden'].shape[1],data['confidence'].shape[1],data['output'].shape[1]).cuda() +ckpt=torch.load(dest/'best.pt',map_location='cuda',weights_only=True) +model.load_state_dict(ckpt['state_dict'],strict=True) +model.eval().requires_grad_(False) +with torch.inference_mode(): + raw=torch.cat([model(data,torch.arange(i,min(i+8192,len(records)),device='cuda')).cpu() for i in range(0,len(records),8192)]).numpy() +scores=raw[:,1]-raw[:,2] +saved=np.load(work/'test_evaluation'/run/'paired_test_arrays.npz')['scores'] +reference=json.loads((work/'test_results.json').read_text()) +ref=next(g for g in reference['groups'] if g['id']==run) +point=ref['best_test_point'] +threshold=point['threshold'] +mask=scores>=threshold if point['operator']=='ge' else scores>threshold +saved_mask=saved>=threshold if point['operator']=='ge' else saved>threshold +assert np.array_equal(mask,saved_mask), 'Recomputed selected decisions differ from main-table archive' +indices=np.random.default_rng(20260912).choice(len(records),64,replace=False) +encoder_dir=a.root/'outputs/mini_siglip_strict_20260910_300ep/8M' +checkpoint=torch.load(encoder_dir/'epoch_300.pt',map_location='cpu',weights_only=True) +encoder=MiniSiglip(**checkpoint['model_config']).cuda() +encoder.load_state_dict(checkpoint['model'],strict=True) +encoder.eval().requires_grad_(False) +tokenizer=Tokenizer.from_file(str(encoder_dir/'student_tokenizer.json')) + +def prepare(i): + with Image.open(records[i]['image_path']) as im: + pixels,valid=prepare_images([[im]]) + pixels=pixels.flatten(0,1).cuda() + tokens=torch.tensor([tokenizer.encode(records[i]['expression']).ids],device='cuda') if 'text' in cfg['branches'] else None + return pixels,valid.cuda(),tokens + +def forward(i,prepared): + pixels,valid,tokens=prepared + with torch.autocast('cuda',dtype=torch.bfloat16): + image=encoder.encode_image(pixels).to(torch.float16).float().reshape(1,2,768) + text=encoder.encode_text(tokens).to(torch.float16).float() if tokens is not None else None + row={k:v[i:i+1] for k,v in data.items()} + row['image']=image;row['valid']=valid + if text is not None:row['text']=text + return model(row,torch.zeros(1,device='cuda',dtype=torch.long)) + +times=[];head_times=[];flops=[];ops=[] +with torch.inference_mode(): + for i in indices[:4]: + forward(int(i),prepare(int(i))) + torch.cuda.synchronize() + for i in indices: + i=int(i) + started=time.perf_counter() + forward(i,prepare(i)) + torch.cuda.synchronize() + times.append((time.perf_counter()-started)*1000) + start=torch.cuda.Event(enable_timing=True);end=torch.cuda.Event(enable_timing=True) + idx=torch.tensor([i],device='cuda') + start.record();model(data,idx);end.record();torch.cuda.synchronize() + head_times.append(start.elapsed_time(end)) + for i in indices[:8]: + prepared=prepare(int(i)) + with FlopCounterMode(display=False) as counter: + forward(int(i),prepared) + flops.append(counter.get_total_flops()/1e9) + ops.append({str(k):int(v) for k,v in counter.get_flop_counts().get('Global',{}).items()}) +report={'status':'complete','run':run,'config':cfg,'checkpoint':str(dest/'best.pt'), + 'score':'logit_rescue-logit_harm','test_rows_recomputed':len(records),'selected_decisions_match_main':True, + 'max_score_abs_difference':float(np.max(np.abs(saved-scores))),'actual_call_fraction':float(mask.mean()), + 'sample_count':len(indices),'sample_ids':[records[int(i)]['sample_id'] for i in indices], + 'batch_size':1,'latency_ms':times,'mean_latency_ms_per_input':float(np.mean(times)), + 'head_mean_latency_ms':float(np.mean(head_times)), 'gflops_per_input':flops,'mean_gflops_per_input':float(np.mean(flops)), + 'operators':ops,'scope':'image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only'} +a.output.parent.mkdir(parents=True,exist_ok=True) +a.output.write_text(json.dumps(report,indent=2),encoding='utf-8') +print(json.dumps({k:v for k,v in report.items() if k in ['run','actual_call_fraction','selected_decisions_match_main','mean_latency_ms_per_input','mean_gflops_per_input']},indent=2)) diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_specialist.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_specialist.py new file mode 100644 index 0000000000000000000000000000000000000000..fbaf817087f52f7e7dfcd6b7ac42c3d4d8360a20 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/measure_specialist.py @@ -0,0 +1,258 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import importlib.util +import json +import sys +import time +from pathlib import Path + +import numpy as np +import torch +from torch.utils.flop_counter import FlopCounterMode + + +def timed_cuda(call): + start = torch.cuda.Event(enable_timing=True) + end = torch.cuda.Event(enable_timing=True) + start.record() + value = call() + end.record() + torch.cuda.synchronize() + return float(start.elapsed_time(end)), value + + +def count_flops(call): + with torch.inference_mode(), FlopCounterMode(display=False) as counter: + value = call() + torch.cuda.synchronize() + global_counts = counter.get_flop_counts().get('Global', {}) + return int(counter.get_total_flops()), value, {str(k): int(v) for k, v in global_counts.items()} + + +def instancevg_measure(root: Path, output: Path, samples: int, batch_size: int, device: str) -> None: + import mmcv + from mmcv import Config + from transformers import XLMRobertaTokenizer + from instancevg.models import build_model + from instancevg.datasets.pipelines.transforms import Normalize, Resize + + torch.set_num_threads(4) + torch.backends.cuda.matmul.allow_tf32 = False + torch.backends.cudnn.allow_tf32 = False + cfg = Config.fromfile(str(root / 'code/third_party/InstanceVG/configs/gres/InstanceVG-grefcoco.py')) + cfg.model.vis_enc.pretrain = None + cfg.model.process_visual = False + model = build_model(cfg.model) + checkpoint_path = root / 'models/grefcoco/instancevg/InstanceVG-grefcoco.pth' + checkpoint = torch.load(checkpoint_path, map_location='cpu') + state = checkpoint['state_dict'] + if all(key.startswith('module.') for key in state): + state = {key[7:]: value for key, value in state.items()} + model.load_state_dict(state, strict=True) + del state, checkpoint + model.eval().to(device).requires_grad_(False) + tokenizer_path = root / 'models/nlvr2/Raghavan--beit3_base_patch16_224_nlvr2/sentencepiece.bpe.model' + tokenizer = XLMRobertaTokenizer(str(tokenizer_path)) + resize = Resize(img_scale=(320, 320), keep_ratio=False) + normalize = Normalize(mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375]) + rows = [json.loads(line) for line in (root / 'outputs/router_full_20260910/data/instancevg/test/records.jsonl').open(encoding='utf-8')] + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chosen = [rows[int(index)] for index in indices] + + def prepare(batch_rows): + images, metas, tokens, masks = [], [], [], [] + for row in batch_rows: + image = mmcv.imread(row['image_path']) + if image.shape[:2] != (row['height'], row['width']): + raise ValueError(f'image shape mismatch for {row["sample_id"]}') + expression = row['expression'] + expression = __import__('re').sub(r"([.,'!?\"()*#:;])", '', expression.lower()).replace('-', ' ').replace('/', ' ') + token_ids = tokenizer.convert_tokens_to_ids(tokenizer.tokenize(expression)) + if not token_ids: + raise ValueError(f'empty token sequence for {row["sample_id"]}') + token_ids = [tokenizer.bos_token_id] + token_ids[:48] + [tokenizer.eos_token_id] + if len(token_ids) > 50: + raise ValueError(f'token sequence exceeds fixed length for {row["sample_id"]}') + masks.append([0] * len(token_ids) + [1] * (50 - len(token_ids))) + tokens.append(token_ids + [tokenizer.pad_token_id] * (50 - len(token_ids))) + data = dict(img=image, filename=row['image_path'], img_shape=image.shape, ori_shape=image.shape, + expression=expression, empty=None, with_bbox=False, with_mask=False) + data = normalize(resize(data)) + images.append(torch.from_numpy(np.ascontiguousarray(data['img'].transpose(2, 0, 1)))) + metas.append({key: data[key] for key in ['filename', 'expression', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'empty']}) + image_tensor = torch.stack(images).to(device) + token_tensor = torch.tensor(tokens, device=device) + mask_tensor = torch.tensor(masks, device=device) + + def forward(): + return model(img=image_tensor, ref_expr_inds=token_tensor, text_attention_mask=mask_tensor, + img_metas=metas, return_loss=False, rescale=True, with_bbox=True, with_mask=False) + + return forward + + batches = [chosen[start:start + batch_size] for start in range(0, len(chosen), batch_size)] + prepared = [prepare(batch) for batch in batches] + for forward in prepared[:2]: + forward() + torch.cuda.synchronize() + forward_ms = [] + end_to_end_ms = [] + for batch in batches: + forward = prepare(batch) + elapsed, _ = timed_cuda(forward) + forward_ms.append(elapsed / len(batch)) + started = time.perf_counter() + forward = prepare(batch) + forward() + torch.cuda.synchronize() + end_to_end_ms.append((time.perf_counter() - started) * 1000 / len(batch)) + flop_rows = [] + for batch in batches[:min(4, len(batches))]: + forward = prepare(batch) + total, _, operators = count_flops(forward) + flop_rows.append({'batch_size': len(batch), 'flops_counted': total, 'gflops_counted_per_sample': total / len(batch) / 1e9, 'operators': operators}) + output.write_text(json.dumps({ + 'component': 'specialist', 'expert': 'instancevg', 'task': 'grefcoco', 'device': device, + 'checkpoint': str(checkpoint_path), 'sample_count': len(chosen), 'batch_size': batch_size, + 'sample_ids': [row['sample_id'] for row in chosen], + 'model_load_seconds_excluded_from_latency': True, + 'forward_latency_ms_per_sample': forward_ms, + 'end_to_end_latency_ms_per_sample': end_to_end_ms, + 'mean_forward_latency_ms_per_sample': float(np.mean(forward_ms)), + 'median_forward_latency_ms_per_sample': float(np.median(forward_ms)), + 'mean_end_to_end_latency_ms_per_sample': float(np.mean(end_to_end_ms)), + 'median_end_to_end_latency_ms_per_sample': float(np.median(end_to_end_ms)), + 'flop_profiles': flop_rows, + 'mean_counted_gflops_per_sample': float(np.mean([row['gflops_counted_per_sample'] for row in flop_rows])), + 'flops_scope': 'GPU FlopCounterMode dense products, convolutions and supported attention; multiply-add counted as 2 FLOPs. Elementwise, normalization, custom ops and postprocessing may be missing.', + 'latency_scope': 'A800 wall-clock CUDA forward latency and end-to-end latency including image read/preprocess, excluding model load; batch processing uses the recorded batch size.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def yolo_measure(root: Path, output: Path, samples: int, batch_size: int, device: str) -> None: + from ultralytics import YOLO + + torch.set_num_threads(4) + torch.backends.cuda.matmul.allow_tf32 = False + torch.backends.cudnn.allow_tf32 = False + checkpoint_path = root / 'models/construction/yolo26x.pt' + model = YOLO(str(checkpoint_path)) + rows = [json.loads(line) for line in (root / 'outputs/router_full_20260910/data/yolo26x/test/records.jsonl').open(encoding='utf-8')] + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chosen = [rows[int(index)] for index in indices] + batches = [chosen[start:start + batch_size] for start in range(0, len(chosen), batch_size)] + + def predict(batch_rows): + return model.predict(source=[row['image_path'] for row in batch_rows], imgsz=640, batch=len(batch_rows), + device=device, half=False, conf=0.001, iou=0.7, max_det=300, save=False, verbose=False) + + for batch in batches[:2]: + predict(batch) + torch.cuda.synchronize() + forward_ms, end_to_end_ms = [], [] + for batch in batches: + elapsed, _ = timed_cuda(lambda batch=batch: predict(batch)) + forward_ms.append(elapsed / len(batch)) + started = time.perf_counter() + predict(batch) + torch.cuda.synchronize() + end_to_end_ms.append((time.perf_counter() - started) * 1000 / len(batch)) + flop_rows = [] + for batch in batches[:min(4, len(batches))]: + total, _, operators = count_flops(lambda batch=batch: predict(batch)) + flop_rows.append({'batch_size': len(batch), 'flops_counted': total, 'gflops_counted_per_sample': total / len(batch) / 1e9, 'operators': operators}) + output.write_text(json.dumps({ + 'component': 'specialist', 'expert': 'yolo26x', 'task': 'construction', 'device': device, + 'checkpoint': str(checkpoint_path), 'sample_count': len(chosen), 'batch_size': batch_size, + 'sample_ids': [row['sample_id'] for row in chosen], + 'model_load_seconds_excluded_from_latency': True, + 'forward_latency_ms_per_sample': forward_ms, + 'end_to_end_latency_ms_per_sample': end_to_end_ms, + 'mean_forward_latency_ms_per_sample': float(np.mean(forward_ms)), + 'median_forward_latency_ms_per_sample': float(np.median(forward_ms)), + 'mean_end_to_end_latency_ms_per_sample': float(np.mean(end_to_end_ms)), + 'median_end_to_end_latency_ms_per_sample': float(np.median(end_to_end_ms)), + 'flop_profiles': flop_rows, + 'mean_counted_gflops_per_sample': float(np.mean([row['gflops_counted_per_sample'] for row in flop_rows])), + 'flops_scope': 'GPU FlopCounterMode around the real Ultralytics predict path; multiply-add counted as 2 FLOPs. Elementwise, postprocessing and unsupported custom ops may be missing.', + 'latency_scope': 'A800 wall-clock latency for real Ultralytics prediction including image loading/preprocess, forward and result postprocessing, excluding model load; batch processing uses the recorded batch size.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def router_measure(root: Path, output: Path, expert: str, endpoint: str, condition: str, seed: int, samples: int, batch_size: int, device: str) -> None: + work = root / 'outputs/router_method_improvement_20260911_r1/reproducible' + sys.path.insert(0, str(work)) + common = __import__('method_improvement_common') + spec = json.loads((root / 'outputs/router_method_improvement_20260911_r1/method_improvement_spec.json').read_text(encoding='utf-8')) + trajectory_id = f'{expert}_{endpoint}_seed{seed}' + cfg = common.make_config(spec, expert, endpoint, seed, trajectory_id) + rows, data = common.load_split(root, spec, cfg, 'test', device) + checkpoint_path = root / f'outputs/router_method_improvement_20260911_r1/conditions/{condition}_{expert}_{endpoint}_seed{seed}/best.pt' + checkpoint = torch.load(checkpoint_path, map_location=device, weights_only=True) + model = common.FeatureRouter(cfg, data['hidden'].shape[1], data['confidence'].shape[1], data['output'].shape[1]).to(device) + model.load_state_dict(checkpoint['state_dict'], strict=True) + model.eval().requires_grad_(False) + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chunks = [torch.as_tensor(indices[start:start + batch_size], device=device, dtype=torch.long) for start in range(0, samples, batch_size)] + for index in chunks[:2]: + model(data, index) + torch.cuda.synchronize() + latency = [] + for index in chunks: + elapsed, _ = timed_cuda(lambda index=index: model(data, index)) + latency.append(elapsed / len(index)) + index = chunks[0] + total, _, operators = count_flops(lambda: model(data, index)) + output.write_text(json.dumps({ + 'component': 'router', 'expert': expert, 'task': cfg['task'], 'endpoint': endpoint, + 'condition': condition, 'seed': seed, 'device': device, 'checkpoint': str(checkpoint_path), + 'sample_count': samples, 'batch_size': batch_size, 'row_count_loaded': len(rows), + 'mean_latency_ms_per_sample': float(np.mean(latency)), 'median_latency_ms_per_sample': float(np.median(latency)), + 'latency_ms_per_sample': latency, 'flops_counted_per_batch': total, + 'gflops_counted_per_sample': total / len(index) / 1e9, 'operators': operators, + 'flops_scope': 'GPU FlopCounterMode for the router forward; multiply-add counted as 2 FLOPs.', + 'latency_scope': 'A800 CUDA forward latency on cached router features, excluding feature loading and model load.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--component', choices=['specialist', 'router'], required=True) + parser.add_argument('--expert', choices=['instancevg', 'yolo26x'], required=True) + parser.add_argument('--endpoint', choices=['MiniCPM', 'Qwen3.8']) + parser.add_argument('--condition', default='both') + parser.add_argument('--seed', type=int, default=42) + parser.add_argument('--samples', type=int, default=64) + parser.add_argument('--batch-size', type=int, default=8) + parser.add_argument('--device', default='cuda:0') + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + args.output.parent.mkdir(parents=True, exist_ok=True) + if not torch.cuda.is_available(): + raise RuntimeError('CUDA is unavailable') + if args.component == 'specialist' and args.expert == 'instancevg': + instancevg_measure(args.root, args.output, args.samples, args.batch_size, args.device) + elif args.component == 'specialist': + yolo_measure(args.root, args.output, args.samples, args.batch_size, args.device) + else: + if args.endpoint is None: + raise ValueError('--endpoint is required for router') + router_measure(args.root, args.output, args.expert, args.endpoint, args.condition, args.seed, args.samples, args.batch_size, args.device) + + +if __name__ == '__main__': + main() diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/prepare_output.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/prepare_output.py new file mode 100644 index 0000000000000000000000000000000000000000..2f5852b9ecd89acd1afbda9caaf75a202b0fa4ed --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/prepare_output.py @@ -0,0 +1,19 @@ +from pathlib import Path +import json +import shutil +root=Path(__file__).resolve().parents[2] +out=root/'dse_results/main_table_cost_20260912' +out.mkdir(exist_ok=True) +(out/'components').mkdir(exist_ok=True) +source=root/'paper_materials/draft_20260912_revision19/data/results.json' +data=json.loads(source.read_text(encoding='utf-8')) +groups=[g for g in data['groups'] if g['key'] in [['grefcoco','instancevg','MiniCPM'],['construction','yolo26x','Qwen3.8']]] +assert len(groups)==2 +(out/'main_table_results.json').write_text(json.dumps({'source':str(source),'protocol':data['protocol'],'groups':groups},indent=2),encoding='utf-8') +scripts=out/'reproducible' +scripts.mkdir(exist_ok=True) +for p in Path(__file__).resolve().parent.glob('*.py'): + shutil.copy2(p,scripts/p.name) +for p in Path(__file__).resolve().parent.glob('*.sh'): + shutil.copy2(p,scripts/p.name) +print(out) diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/probe.sh b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/probe.sh new file mode 100644 index 0000000000000000000000000000000000000000..82ac85243879576913be1d49e120698b7d9ed971 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/probe.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash +set -eu +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +command -v python +python -c 'import torch; print(torch.__version__); import vllm; print(vllm.__version__)' +ls "$ROOT/envs" +ls "$ROOT/outputs/router_full_20260910" +ls "$ROOT/outputs/mini_siglip_strict_20260910_300ep/8M" +sed -n '1,170p' "$ROOT/code/routing/raw_student.py" +sed -n '1,150p' "$ROOT/code/routing/student_artifacts.py" diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/read_main.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/read_main.py new file mode 100644 index 0000000000000000000000000000000000000000..d1e604ed318b6c4d5d3c4d4a9e2c1633662abcdc --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/read_main.py @@ -0,0 +1,10 @@ +from pathlib import Path +import json +root=Path(__file__).resolve().parents[2] +data=json.loads((root/'paper_materials/draft_20260912_revision19/data/results.json').read_text(encoding='utf-8')) +print('TOP',list(data)) +for g in data['groups']: + if g['key'] in [['grefcoco','instancevg','MiniCPM'],['construction','yolo26x','Qwen3.8']]: + print(json.dumps({k:v for k,v in g.items() if k!='methods'},ensure_ascii=False)[:6000]) + for m,v in g['methods'].items(): + print(m, {k:v1 for k,v1 in v.items() if 'curve' not in k}) diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/resume_yolo.sh b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/resume_yolo.sh new file mode 100644 index 0000000000000000000000000000000000000000..c8e59d36337adb23926f1495dff0837607c08484 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/resume_yolo.sh @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +set -euo pipefail +WORK=/tmp/codex_task_main_cost_20260912 +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +python -m pip install --no-deps --target "$WORK/yolo_deps" ultralytics==8.3.222 +export PYTHONPATH="$WORK/yolo_deps${PYTHONPATH:+:$PYTHONPATH}" +bash "$WORK/run_small.sh" 1 yolo26x Qwen3.8 qwen diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/retry_minicpm.sh b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/retry_minicpm.sh new file mode 100644 index 0000000000000000000000000000000000000000..578297b0147ac2bb99abe300e95b8e740a6c243c --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/retry_minicpm.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +set -euo pipefail +OUT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_20260912 +WORK=/tmp/codex_task_main_cost_20260912 +ps -p 6908 -o args= | grep -F "$WORK/measure_large.py" | grep -F 'MiniCPM' +test "$(ps -p 7122 -o ppid= | tr -d ' ')" = 6908 +kill -TERM 7122 6908 +export VLLM_WORKER_MULTIPROC_METHOD=spawn +nohup bash "$WORK/run_large.sh" 0 MiniCPM grefcoco minicpm > "$OUT/logs/minicpm_spawn.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/minicpm.pid" +cat "$OUT/logs/minicpm.pid" diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/run_large.sh b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/run_large.sh new file mode 100644 index 0000000000000000000000000000000000000000..5cf848660977a5fed4c41abd0df9780d39d90bdd --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/run_large.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +WORK=/tmp/codex_task_main_cost_20260912 +OUT="$ROOT/outputs/main_table_cost_20260912" +mkdir -p "$OUT/components" "$OUT/logs" +GPU=$1 +KIND=$2 +TASK=$3 +TAG=$4 +export CUDA_VISIBLE_DEVICES="$GPU" +export OMP_NUM_THREADS=4 +python "$WORK/measure_large.py" --root "$ROOT" --model-kind "$KIND" --task "$TASK" --output "$OUT/components/${TAG}_latency.json" --samples 66 --batch-size 1 --gpu-memory-utilization .85 +python "$WORK/measure_flops.py" --root "$ROOT" --model-kind "$KIND" --task "$TASK" --output "$OUT/components/${TAG}_flops.json" --samples 8 +echo complete > "$OUT/logs/${TAG}.done" diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/run_small.sh b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/run_small.sh new file mode 100644 index 0000000000000000000000000000000000000000..036d1cc65fa02e3042e3867c9224d13a08029ce7 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/run_small.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +WORK=/tmp/codex_task_main_cost_20260912 +OUT="$ROOT/outputs/main_table_cost_20260912" +GPU=$1 +EXPERT=$2 +ENDPOINT=$3 +TAG=$4 +while [[ ! -f "$OUT/logs/${TAG}.done" ]]; do + PID=$(cat "$OUT/logs/${TAG}.pid") + STATE=$(ps -p "$PID" -o stat= || true) + if [[ -z "$STATE" || "$STATE" == Z* ]]; then + echo "Large-model stage stopped without completion: $TAG" >&2 + exit 1 + fi + sleep 5 +done +export CUDA_VISIBLE_DEVICES="$GPU" +export OMP_NUM_THREADS=4 +PYTHON=python +if [[ "$EXPERT" == instancevg ]]; then PYTHON="$ROOT/envs/instancevg/bin/python"; fi +"$PYTHON" "$WORK/measure_specialist.py" --root "$ROOT" --output "$OUT/components/${EXPERT}_specialist.json" --component specialist --expert "$EXPERT" --samples 64 --batch-size 1 --device cuda:0 +python "$WORK/measure_router.py" --root "$ROOT" --expert "$EXPERT" --endpoint "$ENDPOINT" --output "$OUT/components/${EXPERT}_router.json" +echo complete > "$OUT/logs/${EXPERT}_small.done" diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/setup_local.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/setup_local.py new file mode 100644 index 0000000000000000000000000000000000000000..2e533df9c9a4e92f6b0daa229e641d8f205e7919 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/reproducible/setup_local.py @@ -0,0 +1,6 @@ +from pathlib import Path +import shutil +root=Path(__file__).resolve().parents[2] +dest=Path(__file__).resolve().parent +for source,target in [('measure_large_vllm_runtime_20260911.py','measure_large.py'),('measure_large_flops_reference_20260911.py','measure_flops.py'),('measure_expert_router_runtime_20260911.py','measure_specialist.py')]: + shutil.copy2(root/'tmp'/source,dest/target) diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/selected_points.csv b/releases/20260913_lr/paper_data/main_table_cost_20260912/selected_points.csv new file mode 100644 index 0000000000000000000000000000000000000000..84c2ac9357a0ed0d0803cefbbef5f8de863870e5 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/selected_points.csv @@ -0,0 +1,5 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|MiniCPM,learned,0.23,0.30904914499617164,0.6671865694921022,4002.86925782384,1192.5976822525959 +grefcoco|instancevg|MiniCPM,confidence,0.65,0.5209426310864078,0.6187788900547316,6736.074651524562,1956.4671783358606 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,confidence,0.93,0.9221038615179761,0.2597490004762105,89464.5250591521,9103.651026133362 diff --git a/releases/20260913_lr/paper_data/main_table_cost_20260912/verify_and_report.py b/releases/20260913_lr/paper_data/main_table_cost_20260912/verify_and_report.py new file mode 100644 index 0000000000000000000000000000000000000000..aec745ff6f19d27eb1e2641aad7b460cff1fd8b3 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_20260912/verify_and_report.py @@ -0,0 +1,56 @@ +from pathlib import Path +import csv +import json +import fitz + +root = Path(__file__).resolve().parent +summary = json.loads((root/'cost_summary.json').read_text()) +groups = json.loads((root/'main_table_results.json').read_text())['groups'] +rows = list(csv.DictReader((root/'full_cost_performance_curves.csv').open())) +assert len(rows) == 404 +lines = ['# 主表方法:成本—性能曲线', '', '## 1 做了什么', '', + '- 使用主表原检查点、种子42及原验证集阈值;未重新训练,也未改用后续改进方法。', + '- 在 A800 的0、1号卡测量两个模型对;每个大模型预热2张、正式测量64张,每次1张。专家和路由部分各测64张。', + '- 重算原路由头,并核对全测试集的选定阈值调用决定与主表完全一致。性能数值沿用主表固定的预测记录。', + '- 只画学习路由与置信度路由;共404个曲线点。另提供单栏四面板紧凑版。', '', + '## 2 为什么做', '', '让成本曲线与主实验比较同一个方法,避免混入后续改进结果;补计蒸馏编码器及路由头开销。', '', + '## 3 做完有什么效果', '', '| 数据集 | 方法 | 测试性能(%) | 调用率(%) | 估算GFLOPs/输入 | 估算毫秒/输入 |', + '|---|---|---:|---:|---:|---:|'] +comparisons = [] +for group in groups: + key = '|'.join(group['key']) + chosen = {p['method']:p for p in summary['selected_points'] if p['pair']==key} + for method,label in [('learned','学习路由'),('confidence','置信度路由')]: + p = chosen[method] + original = group['methods'][method]['test_point'] + assert p['test_metric'] == original['metric'] + assert p['actual_test_call_fraction'] == original['actual_fraction'] + lines.append(f"| {group['key'][0]} | {label} | {100*p['test_metric']:.2f} | {100*p['actual_test_call_fraction']:.2f} | {p['estimated_gflops_per_input']:,.1f} | {p['estimated_time_ms_per_input']:,.1f} |") + a,b = chosen['learned'],chosen['confidence'] + comparisons.append(f"{group['key'][0]}:相对置信度路由,性能提高 {100*(a['test_metric']-b['test_metric']):.2f} 个百分点,估算 FLOPs 减少 {100*(1-a['estimated_gflops_per_input']/b['estimated_gflops_per_input']):.1f}%,估算耗时减少 {100*(1-a['estimated_time_ms_per_input']/b['estimated_time_ms_per_input']):.1f}%。") +for comparison in comparisons: + lines += ['', comparison] +lines += [''] +lines += ['### 数值口径', '', + '- 每个曲线点使用验证集确定的阈值,横轴按该阈值在测试集上的实际调用率计算;圆点对应主表选定点,没有按测试集重新选点。', + '- 置信度路由成本 = 专家成本 + 实际调用率 × 大模型平均成本。学习路由再加编码器和路由头成本。', + '- 耗时为分项实测后相加的串行估算,并非各策略整套系统的直接端到端计时。加载模型和预热不计入。', + '- FLOPs 按乘加计2次统计支持的算子;大模型用8个输入估计预填充成本(含视觉编码),再加平均输出长度乘单词元解码成本。专家统计4个输入,编码器与路由头统计8个输入。', + '- 各策略共用大模型平均单次成本,尚未测量所选样本的长度差异对成本的影响;专家隐藏特征来自缓存,其额外提取开销未单独测量。置信度阈值判断开销未单独计时。', + '- 本轮单张测量与上一轮批量测量口径不同,绝对耗时不宜直接比较。', '', + '- YOLO 使用权重记录的依赖版本8.3.222,安装在任务临时目录;结束时0、1号卡显存均为0,未操作2、3号卡服务。', '', + '## 4 对应文件在哪', '', + '- `cost_compact.pdf`:推荐插入正文的单栏四面板图,宽3.44英寸、高2.05英寸。', + '- `flops_performance.pdf`、`latency_performance.pdf`:两张独立双面板图。', + '- `full_cost_performance_curves.csv`:全部404个点;`selected_points.csv`:主表4个运行点。', + '- `components/`:8份实测记录,含原始回答、错误记录及路由核验。', + '- `build_figures.py`:绘图及成本计算;`reproducible/`:远端测量脚本;`latex_includes.tex`:插入代码及图注。', + '- 远端结果:`/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_20260912`。', '', + '### 检查记录', '', '已核对404个曲线点、4个主表运行点、两组全测试集调用决定及单位。PDF文字为嵌入字体,另渲染检查排版。'] +(root/'成本性能报告.md').write_text('\n'.join(lines)+'\n',encoding='utf-8') +for stem in ['cost_compact','flops_performance','latency_performance']: + doc = fitz.open(root/(stem+'.pdf')) + assert len(doc)==1 + assert all(font[1] != 'n/a' for font in doc[0].get_fonts()) + doc[0].get_pixmap(matrix=fitz.Matrix(4,4)).save(root/(stem+'_verified.png')) +print('\n'.join(lines)) diff --git "a/releases/20260913_lr/paper_data/main_table_cost_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" "b/releases/20260913_lr/paper_data/main_table_cost_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" new file mode 100644 index 0000000000000000000000000000000000000000..9cdfb36b24c18e308dd08c41ae23d497d6e1c1e7 --- /dev/null +++ "b/releases/20260913_lr/paper_data/main_table_cost_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" @@ -0,0 +1,49 @@ +# 主表方法:成本—性能曲线 + +## 1 做了什么 + +- 使用主表原检查点、种子42及原验证集阈值;未重新训练,也未改用后续改进方法。 +- 在 A800 的0、1号卡测量两个模型对;每个大模型预热2张、正式测量64张,每次1张。专家和路由部分各测64张。 +- 重算原路由头,并核对全测试集的选定阈值调用决定与主表完全一致。性能数值沿用主表固定的预测记录。 +- 只画学习路由与置信度路由;共404个曲线点。另提供单栏四面板紧凑版。 + +## 2 为什么做 + +让成本曲线与主实验比较同一个方法,避免混入后续改进结果;补计蒸馏编码器及路由头开销。 + +## 3 做完有什么效果 + +| 数据集 | 方法 | 测试性能(%) | 调用率(%) | 估算GFLOPs/输入 | 估算毫秒/输入 | +|---|---|---:|---:|---:|---:| +| grefcoco | 学习路由 | 66.72 | 30.90 | 4,002.9 | 1,192.6 | +| grefcoco | 置信度路由 | 61.88 | 52.09 | 6,736.1 | 1,956.5 | +| construction | 学习路由 | 29.88 | 48.77 | 47,383.3 | 4,861.8 | +| construction | 置信度路由 | 25.97 | 92.21 | 89,464.5 | 9,103.7 | + +grefcoco:相对置信度路由,性能提高 4.84 个百分点,估算 FLOPs 减少 40.6%,估算耗时减少 39.0%。 + +construction:相对置信度路由,性能提高 3.90 个百分点,估算 FLOPs 减少 47.0%,估算耗时减少 46.6%。 + +### 数值口径 + +- 每个曲线点使用验证集确定的阈值,横轴按该阈值在测试集上的实际调用率计算;圆点对应主表选定点,没有按测试集重新选点。 +- 置信度路由成本 = 专家成本 + 实际调用率 × 大模型平均成本。学习路由再加编码器和路由头成本。 +- 耗时为分项实测后相加的串行估算,并非各策略整套系统的直接端到端计时。加载模型和预热不计入。 +- FLOPs 按乘加计2次统计支持的算子;大模型用8个输入估计预填充成本(含视觉编码),再加平均输出长度乘单词元解码成本。专家统计4个输入,编码器与路由头统计8个输入。 +- 各策略共用大模型平均单次成本,尚未测量所选样本的长度差异对成本的影响;专家隐藏特征来自缓存,其额外提取开销未单独测量。置信度阈值判断开销未单独计时。 +- 本轮单张测量与上一轮批量测量口径不同,绝对耗时不宜直接比较。 + +- YOLO 使用权重记录的依赖版本8.3.222,安装在任务临时目录;结束时0、1号卡显存均为0,未操作2、3号卡服务。 + +## 4 对应文件在哪 + +- `cost_compact.pdf`:推荐插入正文的单栏四面板图,宽3.44英寸、高2.05英寸。 +- `flops_performance.pdf`、`latency_performance.pdf`:两张独立双面板图。 +- `full_cost_performance_curves.csv`:全部404个点;`selected_points.csv`:主表4个运行点。 +- `components/`:8份实测记录,含原始回答、错误记录及路由核验。 +- `build_figures.py`:绘图及成本计算;`reproducible/`:远端测量脚本;`latex_includes.tex`:插入代码及图注。 +- 远端结果:`/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_20260912`。 + +### 检查记录 + +已核对404个曲线点、4个主表运行点、两组全测试集调用决定及单位。PDF文字为嵌入字体,另渲染检查排版。 diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/build_figures.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/build_figures.py new file mode 100644 index 0000000000000000000000000000000000000000..5953cf95b5fd0da7cd8d67f73fce7c796cae7941 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/build_figures.py @@ -0,0 +1,101 @@ +"""Main-table curves plus fresh measured component costs; no threshold re-selection.""" +from pathlib import Path +import csv +import json +import numpy as np +import matplotlib +matplotlib.use('Agg') +import matplotlib.pyplot as plt +from matplotlib.ticker import MaxNLocator,LogLocator,NullLocator + +root=Path(__file__).resolve().parent +groups=json.loads((root/'main_table_results.json').read_text(encoding='utf-8'))['groups'] +components={} +curves=[] +selected=[] +for g,tag in zip(groups,['qwen_gref','qwen']): + expert=g['key'][1] + spec=json.loads((root/f'components/{expert}_specialist.json').read_text()) + router=json.loads((root/f'components/{expert}_router.json').read_text()) + large=json.loads((root/f'components/{tag}_latency.json').read_text()) + flops=json.loads((root/f'components/{tag}_flops.json').read_text()) + assert spec['completed'] and router['status']==large['status']==flops['status']=='complete' + assert router['run']==g['run'] and router['selected_decisions_match_main'] + assert abs(router['actual_call_fraction']-g['methods']['learned']['test_point']['actual_fraction'])<1e-12 + key='|'.join(g['key']) + c={'specialist_gflops':spec['mean_counted_gflops_per_sample'], + 'router_gflops':router['mean_gflops_per_input'], + 'large_gflops':flops['mean_prefill_gflops_counted']+large['mean_completion_tokens']*flops['mean_decode_one_token_gflops_counted'], + 'specialist_ms':spec['mean_end_to_end_latency_ms_per_sample'], + 'router_ms':router['mean_latency_ms_per_input'], + 'large_ms':large['mean_batch_wall_ms_per_sample'], + 'large_samples':large['measured_count'], 'flops_probes':flops['sample_count'], + 'specialist_samples':spec['sample_count'],'router_samples':router['sample_count'],'batch_size':large['batch_size']} + components[key]=c + for method in ['learned','confidence']: + for kind,points in [('curve',g['methods'][method]['test_curve']),('selected',[g['methods'][method]['test_point']])]: + for p in points: + rate=p['actual_fraction'] + row={'pair':key,'method':method,'nominal_validation_budget':p['budget'],'actual_test_call_fraction':rate, + 'test_metric':p['metric'],'estimated_gflops_per_input':c['specialist_gflops']+(c['router_gflops'] if method=='learned' else 0)+rate*c['large_gflops'], + 'estimated_time_ms_per_input':c['specialist_ms']+(c['router_ms'] if method=='learned' else 0)+rate*c['large_ms']} + (curves if kind=='curve' else selected).append(row) +for name,rows in [('full_cost_performance_curves',curves),('selected_points',selected),('fixed_budget_cost_performance',[p for p in curves if p['nominal_validation_budget'] in [0,.05,.1,.25,.5,.75,1]])]: + with (root/(name+'.csv')).open('w',encoding='utf-8',newline='') as f: + writer=csv.DictWriter(f,fieldnames=list(rows[0]));writer.writeheader();writer.writerows(rows) +summary={'components':components,'selected_points':selected,'protocol': + {'method':'original main-table logit contrast; original checkpoints and validation thresholds; seed 42', + 'large_timing':'64 measured examples; batch size one; same full task prompt; two separate warmup examples', + 'cost':'serial component estimates, not a timed integrated service; includes active distilled encoders and routing head only for learned', + 'flops':'multiply-add=2; supported operators; mean prefill (including visual embedding) plus average generated length times one-token decode probe', + 'scope_limits':'mean call cost extrapolated across policies; no policy-specific input/output-length costs; specialist-feature extraction overhead is not separately measured; confidence threshold overhead not separately timed'}} +(root/'cost_summary.json').write_text(json.dumps(summary,indent=2),encoding='utf-8') + +plt.rcParams.update({'font.family':'serif','font.serif':['Times New Roman'],'font.size':7, + 'axes.labelsize':7,'xtick.labelsize':6.5,'ytick.labelsize':6.5,'legend.fontsize':7, + 'pdf.fonttype':42,'ps.fonttype':42,'axes.spines.top':False,'axes.spines.right':False, + 'axes.linewidth':.55,'mathtext.fontset':'stix'}) +styles=[('learned','learned','#176f91','-'),('confidence','conf.','#c78324','--')] +keys=list(components) + +def panel(ax,key,field): + for method,label,color,ls in styles: + pts=[p for p in curves if p['pair']==key and p['method']==method] + assert len(pts)==101 + x=np.array([p[field] for p in pts]);y=np.array([p['test_metric']*100 for p in pts]) + assert np.all(np.diff(x)>=-1e-7) + ax.plot(x,y,color=color,ls=ls,lw=1.05,label=label) + p=next(p for p in selected if p['pair']==key and p['method']==method) + ax.scatter(p[field],100*p['test_metric'],color=color,s=13,edgecolor='white',linewidth=.4,zorder=5) + ax.grid(axis='y',color='#dce1e4',lw=.4) + ax.tick_params(length=2.3,pad=1.5) + ax.yaxis.set_major_locator(MaxNLocator(nbins=3)) + if 'gflops' in field: + ax.set_xscale('log');ax.xaxis.set_major_locator(LogLocator(base=10,numticks=4));ax.xaxis.set_minor_locator(NullLocator()) + ax.set_xlabel('Est. GFLOPs/input',labelpad=1) + else: + ax.xaxis.set_major_locator(MaxNLocator(nbins=3));ax.set_xlabel('Est. time (ms/input)',labelpad=1) + ax.set_ylabel('Acc. (%)' if key.startswith('grefcoco') else 'Macro-F1 (%)',labelpad=1) + +for field,stem in [('estimated_gflops_per_input','flops_performance'),('estimated_time_ms_per_input','latency_performance')]: + fig,axes=plt.subplots(1,2,figsize=(3.44,1.50)) + fig.subplots_adjust(left=.115,right=.995,bottom=.31,top=.83,wspace=.43) + for ax,key,title in zip(axes,keys,['gRefCOCO / Q','Construction / Q']): + panel(ax,key,field) + ax.text(.5,1.10,title,transform=ax.transAxes,ha='center',fontsize=7) + h,l=axes[0].get_legend_handles_labels() + fig.legend(h,l,loc='lower center',bbox_to_anchor=(.53,-.025),ncol=2,frameon=False,handlelength=2.2,columnspacing=1.8) + fig.savefig(root/(stem+'.pdf'));fig.savefig(root/(stem+'.png'),dpi=400) + plt.close(fig) + +fig,axes=plt.subplots(2,2,figsize=(3.44,2.05)) +fig.subplots_adjust(left=.115,right=.995,bottom=.20,top=.89,wspace=.43,hspace=.64) +for col,key in enumerate(keys): + for row,field in enumerate(['estimated_gflops_per_input','estimated_time_ms_per_input']): + panel(axes[row,col],key,field) + axes[0,col].text(.5,1.14,'gRefCOCO / Q' if col==0 else 'Construction / Q',transform=axes[0,col].transAxes,ha='center',fontsize=7) +h,l=axes[0,0].get_legend_handles_labels() +fig.legend(h,l,loc='lower center',bbox_to_anchor=(.53,-.005),ncol=2,frameon=False,handlelength=2.2,columnspacing=1.8) +fig.savefig(root/'cost_compact.pdf');fig.savefig(root/'cost_compact.png',dpi=400) +plt.close(fig) +print(json.dumps(summary,indent=2)) diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/instancevg_router.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/instancevg_router.json new file mode 100644 index 0000000000000000000000000000000000000000..193120495a2aa267d7efb03a46542590614d2e9e --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/instancevg_router.json @@ -0,0 +1,239 @@ +{ + "status": "complete", + "run": "shared_12_instancevg_Qwen3.8", + "config": { + "fraction": 1.0, + "seed": 42, + "freeze_encoder": true, + "fusion": "concat", + "harm_cost": 1.0, + "batch_size": 256, + "weight_decay": 0.01, + "max_epochs": 50, + "patience": 8, + "cpu_threads": 1, + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90, + "expert": "instancevg", + "task": "grefcoco", + "endpoint": "Qwen3.8", + "branches": [ + "image", + "text", + "hidden", + "confidence", + "output" + ], + "hidden_layer": "joint_encoder__layer_11", + "id": "shared_12_instancevg_Qwen3.8", + "stage": "shared_mandatory_inputs" + }, + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_shared_image_hidden_20260910/runs/shared_12_instancevg_Qwen3.8/best.pt", + "score": "logit_rescue-logit_harm", + "test_rows_recomputed": 35263, + "selected_decisions_match_main": true, + "max_score_abs_difference": 0.0, + "actual_call_fraction": 0.5355755324277571, + "sample_count": 64, + "sample_ids": [ + "testB-5056-14012", + "testB-31505-86708", + "testA-57489-158264", + "testA-38865-107108", + "testB-47361-130361", + "testA-5476-15123", + "testA-71404-196892", + "testA-61030-168115", + "testB-35969-99063", + "testB-1675-4724", + "testA-30414-83641", + "testB-47742-131425", + "testB-77456-213280", + "testA-59283-163253", + "testA-57390-157996", + "testA-77229-212665", + "testA-6784-18703", + "testB-11434-31542", + "testB-11675-32181", + "testB-73046-201304", + "testA-28837-79356", + "testA-4563-12619", + "testB-30168-82996", + "testB-92461-254721", + "testA-58293-160462", + "testA-13050-35910", + "testA-39320-108309", + "testB-36423-100313", + "testA-43418-119528", + "testB-23411-64351", + "testB-77730-214031", + "testA-7232-19907", + "testA-17738-48695", + "testA-44914-123593", + "testB-92861-255854", + "testA-69678-192132", + "testA-68785-189658", + "testB-62911-173380", + "testB-19029-52242", + "testA-66284-182759", + "testB-19170-52626", + "testB-8468-23291", + "testB-72940-201009", + "testB-74045-203983", + "testB-93380-257275", + "testB-49369-135990", + "testA-58046-159759", + "testA-57437-158132", + "testA-5754-15895", + "testA-85590-235577", + "testA-92278-254227", + "testB-80373-221257", + "testA-80383-221294", + "testB-71197-196311", + "testB-48080-132352", + "testB-18351-50380", + "testB-1089-3077", + "testB-92466-254737", + "testB-53065-146099", + "testB-42611-117349", + "testA-57690-158796", + "testA-65173-179681", + "testB-90675-249707", + "testB-74709-205815" + ], + "batch_size": 1, + "latency_ms": [ + 10.095497942529619, + 10.442811995744705, + 9.535272954963148, + 8.56779795140028, + 28.526668902486563, + 14.110558084212244, + 19.977235002443194, + 21.489594015292823, + 16.842726967297494, + 12.316308100707829, + 12.318658991716802, + 13.136874069459736, + 16.18148700799793, + 11.749238008633256, + 12.584974057972431, + 12.680649990215898, + 11.369985993951559, + 14.251800021156669, + 14.000480994582176, + 12.42606493178755, + 13.526970986276865, + 11.346371960826218, + 11.965820915065706, + 13.418521964922547, + 13.269996969029307, + 13.868742040358484, + 11.83428696822375, + 13.831221032887697, + 12.576585984788835, + 12.408879934810102, + 12.892312020994723, + 15.224601957015693, + 13.460875954478979, + 11.89423305913806, + 10.926949908025563, + 157.88777999114245, + 16.57543599139899, + 29.21812597196549, + 23.506848025135696, + 13.830928015522659, + 14.57549398764968, + 13.596423901617527, + 16.056897002272308, + 12.610359000973403, + 17.024922999553382, + 12.336026062257588, + 14.77054797578603, + 13.220850960351527, + 19.9308970477432, + 16.4189770584926, + 17.153396038338542, + 31.296705012209713, + 20.386314019560814, + 14.672313001938164, + 15.437994967214763, + 10.980968945659697, + 15.224996954202652, + 12.009209021925926, + 12.397779966704547, + 13.347716070711613, + 15.58609597850591, + 17.80865201726556, + 13.338097953237593, + 13.845606008544564 + ], + "mean_latency_ms_per_input": 17.032772149832454, + "head_mean_latency_ms": 0.6086819972842932, + "gflops_per_input": [ + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392, + 3.642171392 + ], + "mean_gflops_per_input": 3.642171392, + "operators": [ + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 691709440, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2485631488 + } + ], + "scope": "image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/instancevg_specialist.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/instancevg_specialist.json new file mode 100644 index 0000000000000000000000000000000000000000..aebe35a841d23d1c4030450bea8661304d1e417f --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/instancevg_specialist.json @@ -0,0 +1,254 @@ +{ + "component": "specialist", + "expert": "instancevg", + "task": "grefcoco", + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/models/grefcoco/instancevg/InstanceVG-grefcoco.pth", + "sample_count": 64, + "batch_size": 1, + "sample_ids": [ + "testA-30844-84861", + "testB-44353-122077", + "testB-36889-101643", + "testA-49055-135087", + "testA-81245-223633", + "testB-37669-103833", + "testA-5038-13943", + "testB-79646-219274", + "testA-469-1309", + "testB-46644-128438", + "testA-24562-67515", + "testA-47070-129599", + "testA-54554-150141", + "testB-12382-34099", + "testB-23014-63279", + "testB-79647-219275", + "testB-69479-191564", + "testA-15582-42797", + "testB-56692-156000", + "testB-42726-117660", + "testA-80455-221494", + "testB-33115-91174", + "testA-62940-173455", + "testB-31494-86670", + "testA-54586-150233", + "testA-81445-224177", + "testB-8417-23141", + "testA-24438-67190", + "testA-59285-163255", + "testA-73258-201893", + "testB-945-2633", + "testA-54307-149426", + "testB-60557-166822", + "testA-81622-224665", + "testA-69466-191525", + "testA-60731-167321", + "testA-7857-21644", + "testA-44142-121499", + "testB-66162-182407", + "testA-31764-87406", + "testB-38637-106482", + "testB-52921-145709", + "testB-65796-181394", + "testB-2936-8113", + "testA-79438-218690", + "testA-57397-158016", + "testB-23920-65743", + "testB-15419-42358", + "testA-10078-27739", + "testA-43890-120793", + "testA-55116-151689", + "testB-27487-75582", + "testA-26856-73830", + "testB-50845-140107", + "testA-29199-80374", + "testB-65935-181778", + "testA-17736-48684", + "testB-69477-191553", + "testB-22290-61227", + "testA-58476-160950", + "testB-36265-99859", + "testA-5734-15837", + "testB-67952-187353", + "testB-47054-129555" + ], + "model_load_seconds_excluded_from_latency": true, + "forward_latency_ms_per_sample": [ + 41.24195098876953, + 41.74086380004883, + 40.36540985107422, + 44.46102523803711, + 41.2421760559082, + 40.76591873168945, + 41.14704132080078, + 40.88172912597656, + 41.70716857910156, + 41.191871643066406, + 39.91231918334961, + 39.29497528076172, + 40.70723342895508, + 38.920894622802734, + 39.32780838012695, + 38.89958572387695, + 38.59721755981445, + 38.47331237792969, + 39.8744010925293, + 39.87260818481445, + 41.48284912109375, + 40.55779266357422, + 39.05420684814453, + 39.052734375, + 39.143775939941406, + 39.82249450683594, + 39.543006896972656, + 39.368160247802734, + 40.661216735839844, + 39.092193603515625, + 39.85852813720703, + 40.33164978027344, + 38.89593505859375, + 39.29216003417969, + 39.20943832397461, + 40.946590423583984, + 39.44643020629883, + 40.220863342285156, + 39.61030578613281, + 39.639007568359375, + 39.78163146972656, + 39.223201751708984, + 39.00921630859375, + 38.99894332885742, + 41.13721466064453, + 39.94972610473633, + 41.41766357421875, + 39.28073501586914, + 39.721214294433594, + 39.24937438964844, + 39.40460968017578, + 41.30179214477539, + 42.94054412841797, + 41.5063362121582, + 44.03267288208008, + 44.55561447143555, + 44.24995040893555, + 44.87868881225586, + 45.14652633666992, + 44.85055923461914, + 42.39228820800781, + 41.121856689453125, + 45.708351135253906, + 41.8752326965332 + ], + "end_to_end_latency_ms_per_sample": [ + 47.29630902875215, + 46.27182998228818, + 47.56885801907629, + 45.48592597711831, + 45.399176073260605, + 46.442538034170866, + 45.95420998521149, + 46.45739798434079, + 44.90085388533771, + 43.80481690168381, + 44.848884106613696, + 43.67590101901442, + 46.40063806436956, + 44.26892101764679, + 44.44277100265026, + 44.71806308720261, + 44.62323593907058, + 44.6264409692958, + 44.96095096692443, + 44.103492982685566, + 45.13338196557015, + 44.1846780013293, + 48.1606520479545, + 46.43543600104749, + 46.19023704435676, + 45.247678994201124, + 44.801001087762415, + 45.467636082321405, + 44.22409099061042, + 44.7448689956218, + 58.0457029864192, + 46.10048606991768, + 44.56539999227971, + 44.47658592835069, + 46.05250502936542, + 44.743441976606846, + 44.77924096863717, + 43.877715012058616, + 44.05005706939846, + 42.87725198082626, + 43.836933095008135, + 45.634721987880766, + 44.748114072717726, + 46.21323198080063, + 52.24333005025983, + 48.90036000870168, + 48.12136595137417, + 47.65231907367706, + 46.71858798246831, + 48.516221111640334, + 43.90157596208155, + 46.83465394191444, + 46.49809293914586, + 42.6300170365721, + 47.39421408157796, + 48.35387098137289, + 48.176085110753775, + 48.29539393540472, + 47.32040502130985, + 46.35261802468449, + 47.482443042099476, + 47.437409055419266, + 47.27676196489483, + 48.198972013778985 + ], + "mean_forward_latency_ms_per_sample": 40.71232491731644, + "median_forward_latency_ms_per_sample": 40.08529472351074, + "mean_end_to_end_latency_ms_per_sample": 46.08042127620138, + "median_end_to_end_latency_ms_per_sample": 45.79446598654613, + "flop_profiles": [ + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + }, + { + "batch_size": 1, + "flops_counted": 7502680064, + "gflops_counted_per_sample": 7.502680064, + "operators": { + "aten.bmm": 7500632064, + "aten.baddbmm": 2048000 + } + } + ], + "mean_counted_gflops_per_sample": 7.502680064, + "flops_scope": "GPU FlopCounterMode dense products, convolutions and supported attention; multiply-add counted as 2 FLOPs. Elementwise, normalization, custom ops and postprocessing may be missing.", + "latency_scope": "A800 wall-clock CUDA forward latency and end-to-end latency including image read/preprocess, excluding model load; batch processing uses the recorded batch size.", + "completed": true +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_flops.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_flops.json new file mode 100644 index 0000000000000000000000000000000000000000..58b8c3f787acbc76f88e2d4185de8db41426bb45 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_flops.json @@ -0,0 +1,341 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "construction", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "sample_count": 8, + "records": [ + { + "sample_id": "0000184", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0001955", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0000840", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003887", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003433", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0001613", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0002591", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "0003105", + "prompt_tokens_in_reference_tensor": 1705, + "input_shapes": { + "input_ids": [ + 1, + 1705 + ], + "attention_mask": [ + 1, + 1705 + ], + "mm_token_type_ids": [ + 1, + 1705 + ], + "pixel_values": [ + 3888, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 90083398713024, + "decode_one_token_flops_counted": 51922600128, + "prefill_gflops_counted": 90083.398713024, + "decode_one_token_gflops_counted": 51.922600128, + "prefill_operators": { + "aten.convolution": 20475543552, + "aten.addmm": 3283626885120, + "aten.bmm": 3741356981952, + "aten.mm": 83037939302400 + }, + "decode_operators": { + "aten.bmm": 670826688, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + } + ], + "mean_prefill_gflops_counted": 90083.398713024, + "mean_decode_one_token_gflops_counted": 51.922600128, + "scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "answer_generation": "not performed; one known-token decode probe only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_gref_flops.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_gref_flops.json new file mode 100644 index 0000000000000000000000000000000000000000..ec5d0f5375ca215c84bf4c0eb314781a873d6ea1 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_gref_flops.json @@ -0,0 +1,341 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "grefcoco", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "sample_count": 8, + "records": [ + { + "sample_id": "testA-5038-13945", + "prompt_tokens_in_reference_tensor": 608, + "input_shapes": { + "input_ids": [ + 1, + 608 + ], + "attention_mask": [ + 1, + 608 + ], + "mm_token_type_ids": [ + 1, + 608 + ], + "pixel_values": [ + 1200, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 31223201515520, + "decode_one_token_flops_counted": 51491242176, + "prefill_gflops_counted": 31223.20151552, + "decode_one_token_gflops_counted": 51.491242176, + "prefill_operators": { + "aten.convolution": 6649282560, + "aten.addmm": 1013465088000, + "aten.bmm": 590268057600, + "aten.mm": 29612819087360 + }, + "decode_operators": { + "aten.bmm": 239468736, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testA-73305-202014", + "prompt_tokens_in_reference_tensor": 608, + "input_shapes": { + "input_ids": [ + 1, + 608 + ], + "attention_mask": [ + 1, + 608 + ], + "mm_token_type_ids": [ + 1, + 608 + ], + "pixel_values": [ + 1200, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 31223201515520, + "decode_one_token_flops_counted": 51491242176, + "prefill_gflops_counted": 31223.20151552, + "decode_one_token_gflops_counted": 51.491242176, + "prefill_operators": { + "aten.convolution": 6649282560, + "aten.addmm": 1013465088000, + "aten.bmm": 590268057600, + "aten.mm": 29612819087360 + }, + "decode_operators": { + "aten.bmm": 239468736, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testA-26952-74099", + "prompt_tokens_in_reference_tensor": 562, + "input_shapes": { + "input_ids": [ + 1, + 562 + ], + "attention_mask": [ + 1, + 562 + ], + "mm_token_type_ids": [ + 1, + 562 + ], + "pixel_values": [ + 1040, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 28754745861504, + "decode_one_token_flops_counted": 51473154240, + "prefill_gflops_counted": 28754.745861504, + "decode_one_token_gflops_counted": 51.47315424, + "prefill_operators": { + "aten.convolution": 5902172160, + "aten.addmm": 878336409600, + "aten.bmm": 497939359104, + "aten.mm": 27372567920640 + }, + "decode_operators": { + "aten.bmm": 221380800, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testB-53254-146620", + "prompt_tokens_in_reference_tensor": 560, + "input_shapes": { + "input_ids": [ + 1, + 560 + ], + "attention_mask": [ + 1, + 560 + ], + "mm_token_type_ids": [ + 1, + 560 + ], + "pixel_values": [ + 1040, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 28656453395456, + "decode_one_token_flops_counted": 51472367808, + "prefill_gflops_counted": 28656.453395456, + "decode_one_token_gflops_counted": 51.472367808, + "prefill_operators": { + "aten.convolution": 5894307840, + "aten.addmm": 878336409600, + "aten.bmm": 497056982016, + "aten.mm": 27275165696000 + }, + "decode_operators": { + "aten.bmm": 220594368, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testB-37889-104439", + "prompt_tokens_in_reference_tensor": 687, + "input_shapes": { + "input_ids": [ + 1, + 687 + ], + "attention_mask": [ + 1, + 687 + ], + "mm_token_type_ids": [ + 1, + 687 + ], + "pixel_values": [ + 1520, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 35517384622912, + "decode_one_token_flops_counted": 51522306240, + "prefill_gflops_counted": 35517.384622912, + "decode_one_token_gflops_counted": 51.52230624, + "prefill_operators": { + "aten.convolution": 8092385280, + "aten.addmm": 1283722444800, + "aten.bmm": 765362832192, + "aten.mm": 33460206960640 + }, + "decode_operators": { + "aten.bmm": 270532800, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testA-58523-161084", + "prompt_tokens_in_reference_tensor": 500, + "input_shapes": { + "input_ids": [ + 1, + 500 + ], + "attention_mask": [ + 1, + 500 + ], + "mm_token_type_ids": [ + 1, + 500 + ], + "pixel_values": [ + 768, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 25390700918528, + "decode_one_token_flops_counted": 51448774848, + "prefill_gflops_counted": 25390.700918528, + "decode_one_token_gflops_counted": 51.448774848, + "prefill_operators": { + "aten.convolution": 4695785472, + "aten.addmm": 648617656320, + "aten.bmm": 384288519936, + "aten.mm": 24353098956800 + }, + "decode_operators": { + "aten.bmm": 197001408, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testB-3419-9449", + "prompt_tokens_in_reference_tensor": 568, + "input_shapes": { + "input_ids": [ + 1, + 568 + ], + "attention_mask": [ + 1, + 568 + ], + "mm_token_type_ids": [ + 1, + 568 + ], + "pixel_values": [ + 1040, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 29049642134016, + "decode_one_token_flops_counted": 51475513536, + "prefill_gflops_counted": 29049.642134016, + "decode_one_token_gflops_counted": 51.475513536, + "prefill_operators": { + "aten.convolution": 5925765120, + "aten.addmm": 878336409600, + "aten.bmm": 500605364736, + "aten.mm": 27664774594560 + }, + "decode_operators": { + "aten.bmm": 223740096, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + }, + { + "sample_id": "testB-24035-66084", + "prompt_tokens_in_reference_tensor": 561, + "input_shapes": { + "input_ids": [ + 1, + 561 + ], + "attention_mask": [ + 1, + 561 + ], + "mm_token_type_ids": [ + 1, + 561 + ], + "pixel_values": [ + 1040, + 1536 + ], + "image_grid_thw": [ + 1, + 3 + ] + }, + "prefill_flops_counted": 28705599235264, + "decode_one_token_flops_counted": 51472761024, + "prefill_gflops_counted": 28705.599235264, + "decode_one_token_gflops_counted": 51.472761024, + "prefill_operators": { + "aten.convolution": 5898240000, + "aten.addmm": 878336409600, + "aten.bmm": 497497777344, + "aten.mm": 27323866808320 + }, + "decode_operators": { + "aten.bmm": 220987584, + "aten.mm": 51243909120, + "aten.convolution": 7864320 + } + } + ], + "mean_prefill_gflops_counted": 29815.11614984, + "mean_decode_one_token_gflops_counted": 51.480920256000005, + "scope": "HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.", + "answer_generation": "not performed; one known-token decode probe only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_gref_latency.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_gref_latency.json new file mode 100644 index 0000000000000000000000000000000000000000..d23d091ce482278c9b6b8d326d0f9d0663f7c403 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_gref_latency.json @@ -0,0 +1,2446 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "grefcoco", + "expert": "instancevg", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "device": "cuda:0", + "gpu_memory_utilization": 0.85, + "sample_count_requested": 66, + "warmup_count": 2, + "measured_count": 64, + "batch_size": 1, + "sample_ids": [ + "testA-29199-80373", + "testB-44351-122075", + "testA-91850-253019", + "testB-36888-101641", + "testA-54586-150233", + "testB-79644-219272", + "testB-65935-181776", + "testA-81622-224664", + "testA-10078-27739", + "testA-15582-42797", + "testA-5038-13943", + "testA-30844-84861", + "testB-15419-42356", + "testB-27486-75580", + "testB-945-2632", + "testA-24438-67190", + "testB-66161-182405", + "testB-42726-117659", + "testB-69439-191450", + "testA-54307-149426", + "testB-37669-103831", + "testB-50844-140105", + "testA-79438-218690", + "testA-62940-173454", + "testA-81245-223632", + "testA-47070-129599", + "testB-12381-34098", + "testA-44142-121498", + "testA-73258-201893", + "testB-46644-128436", + "testA-43889-120792", + "testB-36082-99363", + "testB-33108-91152", + "testA-26856-73830", + "testA-81271-223709", + "testA-54554-150141", + "testB-38636-106481", + "testA-5734-15837", + "testB-67558-186325", + "testB-47053-129554", + "testA-93064-256422", + "testB-60557-166821", + "testA-55116-151688", + "testB-65796-181392", + "testB-23014-63278", + "testA-24562-67515", + "testA-17736-48684", + "testA-58475-160949", + "testA-80455-221493", + "testA-31763-87405", + "testB-23920-65742", + "testA-469-1309", + "testB-56691-155998", + "testB-31493-86668", + "testA-59284-163254", + "testA-57397-158015", + "testB-2935-8111", + "testB-22290-61226", + "testB-8416-23140", + "testA-60731-167320", + "testA-49055-135086", + "testA-7857-21644", + "testB-79645-219273", + "testB-52921-145707", + "testB-69479-191562", + "testA-69466-191524" + ], + "batches": [ + { + "sample_ids": [ + "testA-91850-253019" + ], + "batch_size": 1, + "generation_latency_seconds": 3.147620694944635, + "batch_wall_latency_seconds": 3.1645371050108224, + "amortized_generation_ms_per_sample": 3147.620694944635, + "amortized_batch_wall_ms_per_sample": 3164.5371050108224, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [496, 292, 713, 655], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 496, + 292, + 713, + 655 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 570 + } + ] + }, + { + "sample_ids": [ + "testB-36888-101641" + ], + "batch_size": 1, + "generation_latency_seconds": 4.945294587058015, + "batch_wall_latency_seconds": 4.966181440046057, + "amortized_generation_ms_per_sample": 4945.294587058015, + "amortized_batch_wall_ms_per_sample": 4966.181440046057, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[21,567,693,915],\"score\":0.95},{\"bbox_2d\":[381,233,959,885],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 21, + 567, + 693, + 915 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 381, + 233, + 959, + 885 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 60, + "prompt_tokens": 608 + } + ] + }, + { + "sample_ids": [ + "testA-54586-150233" + ], + "batch_size": 1, + "generation_latency_seconds": 3.2111774859949946, + "batch_wall_latency_seconds": 3.2314598000375554, + "amortized_generation_ms_per_sample": 3211.1774859949946, + "amortized_batch_wall_ms_per_sample": 3231.4598000375554, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [693, 438, 983, 813], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 693, + 438, + 983, + 813 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 707 + } + ] + }, + { + "sample_ids": [ + "testB-79644-219272" + ], + "batch_size": 1, + "generation_latency_seconds": 33.390286731068045, + "batch_wall_latency_seconds": 33.40712711191736, + "amortized_generation_ms_per_sample": 33390.286731068045, + "amortized_batch_wall_ms_per_sample": 33407.12711191736, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [121, 581, 170, 764], \"score\": 0.9}, {\"bbox_2d\": [173, 595, 259, 837], \"score\": 0.9}, {\"bbox_2d\": [265, 581, 315, 806], \"score\": 0.9}, {\"bbox_2d\": [310, 571, 351, 764], \"score\": 0.9}, {\"bbox_2d\": [338, 581, 387, 764], \"score\": 0.9}, {\"bbox_2d\": [449, 575, 492, 756], \"score\": 0.9}, {\"bbox_2d\": [475, 604, 542, 806], \"score\": 0.9}, {\"bbox_2d\": [668, 575, 743, 818], \"score\": 0.9}, {\"bbox_2d\": [740, 571, 783, 775], \"score\": 0.9}, {\"bbox_2d\": [771, 571, 814, 787], \"score\": 0.9}, {\"bbox_2d\": [822, 587, 851, 764], \"score\": 0.9}, {\"bbox_2d\": [847, 591, 921, 903], \"score\": 0.9}, {\"bbox_2d\": [902, 604, 959, 795], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 121, + 581, + 170, + 764 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 173, + 595, + 259, + 837 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 265, + 581, + 315, + 806 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 310, + 571, + 351, + 764 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 338, + 581, + 387, + 764 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 449, + 575, + 492, + 756 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 475, + 604, + 542, + 806 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 668, + 575, + 743, + 818 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 740, + 571, + 783, + 775 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 771, + 571, + 814, + 787 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 822, + 587, + 851, + 764 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 847, + 591, + 921, + 903 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 902, + 604, + 959, + 795 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 447, + "prompt_tokens": 600 + } + ] + }, + { + "sample_ids": [ + "testB-65935-181776" + ], + "batch_size": 1, + "generation_latency_seconds": 4.57236669503618, + "batch_wall_latency_seconds": 4.583186280098744, + "amortized_generation_ms_per_sample": 4572.36669503618, + "amortized_batch_wall_ms_per_sample": 4583.186280098744, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[347,333,951,761],\"score\":0.95},{\"bbox_2d\":[246,421,777,987],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 347, + 333, + 951, + 761 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 246, + 421, + 777, + 987 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testA-81622-224664" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6317770309979096, + "batch_wall_latency_seconds": 0.6445289449766278, + "amortized_generation_ms_per_sample": 631.7770309979096, + "amortized_batch_wall_ms_per_sample": 644.5289449766278, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testA-10078-27739" + ], + "batch_size": 1, + "generation_latency_seconds": 2.393106485949829, + "batch_wall_latency_seconds": 2.4061070699244738, + "amortized_generation_ms_per_sample": 2393.106485949829, + "amortized_batch_wall_ms_per_sample": 2406.1070699244738, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[283,97,775,810],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 283, + 97, + 775, + 810 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testA-15582-42797" + ], + "batch_size": 1, + "generation_latency_seconds": 7.63340066303499, + "batch_wall_latency_seconds": 7.651124170981348, + "amortized_generation_ms_per_sample": 7633.40066303499, + "amortized_batch_wall_ms_per_sample": 7651.124170981348, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [85, 297, 446, 999], \"score\": 0.9}, {\"bbox_2d\": [315, 413, 542, 999], \"score\": 0.9}, {\"bbox_2d\": [542, 476, 642, 876], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 85, + 297, + 446, + 999 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 315, + 413, + 542, + 999 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 542, + 476, + 642, + 876 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 106, + "prompt_tokens": 619 + } + ] + }, + { + "sample_ids": [ + "testA-5038-13943" + ], + "batch_size": 1, + "generation_latency_seconds": 3.06427818199154, + "batch_wall_latency_seconds": 3.0877813909901306, + "amortized_generation_ms_per_sample": 3064.27818199154, + "amortized_batch_wall_ms_per_sample": 3087.7813909901306, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [521, 246, 793, 999], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 521, + 246, + 793, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 604 + } + ] + }, + { + "sample_ids": [ + "testA-30844-84861" + ], + "batch_size": 1, + "generation_latency_seconds": 2.516077253036201, + "batch_wall_latency_seconds": 2.5232016970403492, + "amortized_generation_ms_per_sample": 2516.077253036201, + "amortized_batch_wall_ms_per_sample": 2523.2016970403492, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[508,536,807,722],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 508, + 536, + 807, + 722 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 623 + } + ] + }, + { + "sample_ids": [ + "testB-15419-42356" + ], + "batch_size": 1, + "generation_latency_seconds": 2.673266503959894, + "batch_wall_latency_seconds": 2.6889539459953085, + "amortized_generation_ms_per_sample": 2673.266503959894, + "amortized_batch_wall_ms_per_sample": 2688.9539459953085, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[107,722,587,999],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 107, + 722, + 587, + 999 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 566 + } + ] + }, + { + "sample_ids": [ + "testB-27486-75580" + ], + "batch_size": 1, + "generation_latency_seconds": 4.617897133924998, + "batch_wall_latency_seconds": 4.632912510889582, + "amortized_generation_ms_per_sample": 4617.897133924998, + "amortized_batch_wall_ms_per_sample": 4632.912510889582, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[542,393,999,999],\"score\":0.95},{\"bbox_2d\":[646,271,911,471],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 542, + 393, + 999, + 999 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 646, + 271, + 911, + 471 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 670 + } + ] + }, + { + "sample_ids": [ + "testB-945-2632" + ], + "batch_size": 1, + "generation_latency_seconds": 12.483138478011824, + "batch_wall_latency_seconds": 12.503725816030055, + "amortized_generation_ms_per_sample": 12483.138478011824, + "amortized_batch_wall_ms_per_sample": 12503.725816030055, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [213, 692, 283, 983], \"score\": 0.9}, {\"bbox_2d\": [321, 661, 415, 973], \"score\": 0.9}, {\"bbox_2d\": [473, 661, 555, 966], \"score\": 0.9}, {\"bbox_2d\": [638, 626, 713, 973], \"score\": 0.9}, {\"bbox_2d\": [775, 638, 871, 988], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 213, + 692, + 283, + 983 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 321, + 661, + 415, + 973 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 473, + 661, + 555, + 966 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 638, + 626, + 713, + 973 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 775, + 638, + 871, + 988 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 175, + "prompt_tokens": 670 + } + ] + }, + { + "sample_ids": [ + "testA-24438-67190" + ], + "batch_size": 1, + "generation_latency_seconds": 4.798777529038489, + "batch_wall_latency_seconds": 4.815012337057851, + "amortized_generation_ms_per_sample": 4798.777529038489, + "amortized_batch_wall_ms_per_sample": 4815.012337057851, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[731,375,999,999],\"score\":0.95},{\"bbox_2d\":[305,247,585,928],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 731, + 375, + 999, + 999 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 305, + 247, + 585, + 928 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 605 + } + ] + }, + { + "sample_ids": [ + "testB-66161-182405" + ], + "batch_size": 1, + "generation_latency_seconds": 2.5812068269588053, + "batch_wall_latency_seconds": 2.593027200899087, + "amortized_generation_ms_per_sample": 2581.2068269588053, + "amortized_batch_wall_ms_per_sample": 2593.027200899087, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[103,193,755,713],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 103, + 193, + 755, + 713 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 33, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testB-42726-117659" + ], + "batch_size": 1, + "generation_latency_seconds": 3.0745598620269448, + "batch_wall_latency_seconds": 3.0872938209213316, + "amortized_generation_ms_per_sample": 3074.5598620269448, + "amortized_batch_wall_ms_per_sample": 3087.2938209213316, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [743, 458, 999, 878], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 743, + 458, + 999, + 878 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 604 + } + ] + }, + { + "sample_ids": [ + "testB-69439-191450" + ], + "batch_size": 1, + "generation_latency_seconds": 0.5888717719353735, + "batch_wall_latency_seconds": 0.5951321049360558, + "amortized_generation_ms_per_sample": 588.8717719353735, + "amortized_batch_wall_ms_per_sample": 595.1321049360558, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 492 + } + ] + }, + { + "sample_ids": [ + "testA-54307-149426" + ], + "batch_size": 1, + "generation_latency_seconds": 4.398384497035295, + "batch_wall_latency_seconds": 4.4204836380667984, + "amortized_generation_ms_per_sample": 4398.384497035295, + "amortized_batch_wall_ms_per_sample": 4420.483638066798, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[542,48,838,923],\"score\":0.95},{\"bbox_2d\":[177,0,567,477],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 542, + 48, + 838, + 923 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 177, + 0, + 567, + 477 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 58, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testB-37669-103831" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6747266949387267, + "batch_wall_latency_seconds": 0.699499664013274, + "amortized_generation_ms_per_sample": 674.7266949387267, + "amortized_batch_wall_ms_per_sample": 699.499664013274, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 648 + } + ] + }, + { + "sample_ids": [ + "testB-50844-140105" + ], + "batch_size": 1, + "generation_latency_seconds": 0.652067915070802, + "batch_wall_latency_seconds": 0.6656363360816613, + "amortized_generation_ms_per_sample": 652.067915070802, + "amortized_batch_wall_ms_per_sample": 665.6363360816613, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 561 + } + ] + }, + { + "sample_ids": [ + "testA-79438-218690" + ], + "batch_size": 1, + "generation_latency_seconds": 3.1018499380443245, + "batch_wall_latency_seconds": 3.1217426829971373, + "amortized_generation_ms_per_sample": 3101.8499380443245, + "amortized_batch_wall_ms_per_sample": 3121.7426829971373, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [255, 173, 536, 923], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 255, + 173, + 536, + 923 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 562 + } + ] + }, + { + "sample_ids": [ + "testA-62940-173454" + ], + "batch_size": 1, + "generation_latency_seconds": 3.057670062989928, + "batch_wall_latency_seconds": 3.073035439942032, + "amortized_generation_ms_per_sample": 3057.670062989928, + "amortized_batch_wall_ms_per_sample": 3073.035439942032, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [626, 266, 751, 693], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 626, + 266, + 751, + 693 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 605 + } + ] + }, + { + "sample_ids": [ + "testA-81245-223632" + ], + "batch_size": 1, + "generation_latency_seconds": 10.252644294989295, + "batch_wall_latency_seconds": 10.266198730911128, + "amortized_generation_ms_per_sample": 10252.644294989295, + "amortized_batch_wall_ms_per_sample": 10266.198730911128, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [584, 117, 783, 882], \"score\": 0.9}, {\"bbox_2d\": [707, 131, 792, 761], \"score\": 0.9}, {\"bbox_2d\": [798, 105, 915, 655], \"score\": 0.9}, {\"bbox_2d\": [533, 178, 655, 705], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 584, + 117, + 783, + 882 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 707, + 131, + 792, + 761 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 798, + 105, + 915, + 655 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 533, + 178, + 655, + 705 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 141, + "prompt_tokens": 570 + } + ] + }, + { + "sample_ids": [ + "testA-47070-129599" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6474383240565658, + "batch_wall_latency_seconds": 0.6609540630597621, + "amortized_generation_ms_per_sample": 647.4383240565658, + "amortized_batch_wall_ms_per_sample": 660.9540630597621, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 560 + } + ] + }, + { + "sample_ids": [ + "testB-12381-34098" + ], + "batch_size": 1, + "generation_latency_seconds": 6.267876331927255, + "batch_wall_latency_seconds": 6.293664561933838, + "amortized_generation_ms_per_sample": 6267.876331927255, + "amortized_batch_wall_ms_per_sample": 6293.664561933838, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[458,242,771,473],\"score\":0.9},{\"bbox_2d\":[298,343,826,817],\"score\":0.9},{\"bbox_2d\":[305,466,793,840],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 458, + 242, + 771, + 473 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 298, + 343, + 826, + 817 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 305, + 466, + 793, + 840 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 86, + "prompt_tokens": 574 + } + ] + }, + { + "sample_ids": [ + "testA-44142-121498" + ], + "batch_size": 1, + "generation_latency_seconds": 5.1425224100239575, + "batch_wall_latency_seconds": 5.153767512994818, + "amortized_generation_ms_per_sample": 5142.5224100239575, + "amortized_batch_wall_ms_per_sample": 5153.767512994818, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [0, 95, 585, 987], \"score\": 0.9}, {\"bbox_2d\": [567, 126, 999, 987], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 95, + 585, + 987 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 567, + 126, + 999, + 987 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 70, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testA-73258-201893" + ], + "batch_size": 1, + "generation_latency_seconds": 2.4450040519004688, + "batch_wall_latency_seconds": 2.4576197939459234, + "amortized_generation_ms_per_sample": 2445.0040519004688, + "amortized_batch_wall_ms_per_sample": 2457.6197939459234, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[401,274,999,987],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 401, + 274, + 999, + 987 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 606 + } + ] + }, + { + "sample_ids": [ + "testB-46644-128436" + ], + "batch_size": 1, + "generation_latency_seconds": 4.464105351013131, + "batch_wall_latency_seconds": 4.473494493984617, + "amortized_generation_ms_per_sample": 4464.105351013131, + "amortized_batch_wall_ms_per_sample": 4473.494493984617, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[10,183,537,826],\"score\":0.95},{\"bbox_2d\":[483,0,999,743],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 10, + 183, + 537, + 826 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 483, + 0, + 999, + 743 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 58, + "prompt_tokens": 497 + } + ] + }, + { + "sample_ids": [ + "testA-43889-120792" + ], + "batch_size": 1, + "generation_latency_seconds": 3.0225026920670643, + "batch_wall_latency_seconds": 3.039261986967176, + "amortized_generation_ms_per_sample": 3022.5026920670643, + "amortized_batch_wall_ms_per_sample": 3039.261986967176, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [305, 303, 475, 882], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 305, + 303, + 475, + 882 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 571 + } + ] + }, + { + "sample_ids": [ + "testB-36082-99363" + ], + "batch_size": 1, + "generation_latency_seconds": 4.3171981680206954, + "batch_wall_latency_seconds": 4.324737289105542, + "amortized_generation_ms_per_sample": 4317.198168020695, + "amortized_batch_wall_ms_per_sample": 4324.737289105542, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,107,417,876],\"score\":0.95},{\"bbox_2d\":[442,0,999,986],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 107, + 417, + 876 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 442, + 0, + 999, + 986 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 57, + "prompt_tokens": 412 + } + ] + }, + { + "sample_ids": [ + "testB-33108-91152" + ], + "batch_size": 1, + "generation_latency_seconds": 2.9848218420520425, + "batch_wall_latency_seconds": 2.9953485280275345, + "amortized_generation_ms_per_sample": 2984.8218420520425, + "amortized_batch_wall_ms_per_sample": 2995.3485280275345, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [58, 201, 385, 458], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 58, + 201, + 385, + 458 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 602 + } + ] + }, + { + "sample_ids": [ + "testA-26856-73830" + ], + "batch_size": 1, + "generation_latency_seconds": 3.0266257260227576, + "batch_wall_latency_seconds": 3.0415181359276175, + "amortized_generation_ms_per_sample": 3026.6257260227576, + "amortized_batch_wall_ms_per_sample": 3041.5181359276175, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [576, 343, 783, 713], \"score\": 0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 576, + 343, + 783, + 713 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 40, + "prompt_tokens": 562 + } + ] + }, + { + "sample_ids": [ + "testA-81271-223709" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6367205450078472, + "batch_wall_latency_seconds": 0.6504157679155469, + "amortized_generation_ms_per_sample": 636.7205450078472, + "amortized_batch_wall_ms_per_sample": 650.4157679155469, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 560 + } + ] + }, + { + "sample_ids": [ + "testA-54554-150141" + ], + "batch_size": 1, + "generation_latency_seconds": 11.380541390972212, + "batch_wall_latency_seconds": 11.400562496040948, + "amortized_generation_ms_per_sample": 11380.541390972212, + "amortized_batch_wall_ms_per_sample": 11400.562496040948, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[27,0,121,243],\"score\":0.9},{\"bbox_2d\":[62,0,185,258],\"score\":0.9},{\"bbox_2d\":[142,0,222,255],\"score\":0.9},{\"bbox_2d\":[185,0,283,261],\"score\":0.9},{\"bbox_2d\":[277,312,597,826],\"score\":0.9},{\"bbox_2d\":[655,500,951,871],\"score\":0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 27, + 0, + 121, + 243 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 62, + 0, + 185, + 258 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 142, + 0, + 222, + 255 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 185, + 0, + 283, + 261 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 277, + 312, + 597, + 826 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 655, + 500, + 951, + 871 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 157, + "prompt_tokens": 610 + } + ] + }, + { + "sample_ids": [ + "testB-38636-106481" + ], + "batch_size": 1, + "generation_latency_seconds": 4.233924768050201, + "batch_wall_latency_seconds": 4.251373155973852, + "amortized_generation_ms_per_sample": 4233.924768050201, + "amortized_batch_wall_ms_per_sample": 4251.373155973852, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[494,177,881,985],\"score\":0.95},{\"bbox_2d\":[0,42,307,722],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 494, + 177, + 881, + 985 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 0, + 42, + 307, + 722 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 58, + "prompt_tokens": 469 + } + ] + }, + { + "sample_ids": [ + "testA-5734-15837" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6504808579338714, + "batch_wall_latency_seconds": 0.6668235529214144, + "amortized_generation_ms_per_sample": 650.4808579338714, + "amortized_batch_wall_ms_per_sample": 666.8235529214144, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 565 + } + ] + }, + { + "sample_ids": [ + "testB-67558-186325" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6280788460280746, + "batch_wall_latency_seconds": 0.6436845100251958, + "amortized_generation_ms_per_sample": 628.0788460280746, + "amortized_batch_wall_ms_per_sample": 643.6845100251958, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 600 + } + ] + }, + { + "sample_ids": [ + "testB-47053-129554" + ], + "batch_size": 1, + "generation_latency_seconds": 4.556327463011257, + "batch_wall_latency_seconds": 4.573514219024219, + "amortized_generation_ms_per_sample": 4556.327463011257, + "amortized_batch_wall_ms_per_sample": 4573.514219024219, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[438,321,871,542],\"score\":0.95},{\"bbox_2d\":[438,452,851,863],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 438, + 321, + 871, + 542 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 438, + 452, + 851, + 863 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 604 + } + ] + }, + { + "sample_ids": [ + "testA-93064-256422" + ], + "batch_size": 1, + "generation_latency_seconds": 2.830894450074993, + "batch_wall_latency_seconds": 2.8475917810574174, + "amortized_generation_ms_per_sample": 2830.894450074993, + "amortized_batch_wall_ms_per_sample": 2847.5917810574174, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [0, 226, 173, 755], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 226, + 173, + 755 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 37, + "prompt_tokens": 549 + } + ] + }, + { + "sample_ids": [ + "testB-60557-166821" + ], + "batch_size": 1, + "generation_latency_seconds": 5.58066594589036, + "batch_wall_latency_seconds": 5.594979963963851, + "amortized_generation_ms_per_sample": 5580.66594589036, + "amortized_batch_wall_ms_per_sample": 5594.979963963851, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [857, 442, 875, 477], \"score\": 0.9}, {\"bbox_2d\": [347, 442, 365, 477], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 857, + 442, + 875, + 477 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 347, + 442, + 365, + 477 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 73, + "prompt_tokens": 500 + } + ] + }, + { + "sample_ids": [ + "testA-55116-151688" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6620599570451304, + "batch_wall_latency_seconds": 0.6773106920300052, + "amortized_generation_ms_per_sample": 662.0599570451304, + "amortized_batch_wall_ms_per_sample": 677.3106920300052, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 600 + } + ] + }, + { + "sample_ids": [ + "testB-65796-181392" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6342472250107676, + "batch_wall_latency_seconds": 0.6483092680573463, + "amortized_generation_ms_per_sample": 634.2472250107676, + "amortized_batch_wall_ms_per_sample": 648.3092680573463, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 581 + } + ] + }, + { + "sample_ids": [ + "testB-23014-63278" + ], + "batch_size": 1, + "generation_latency_seconds": 2.7931598050054163, + "batch_wall_latency_seconds": 2.8087078949902207, + "amortized_generation_ms_per_sample": 2793.1598050054163, + "amortized_batch_wall_ms_per_sample": 2808.7078949902207, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[33, 303, 466, 761],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 33, + 303, + 466, + 761 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 35, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testA-24562-67515" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6919117919169366, + "batch_wall_latency_seconds": 0.7147108289645985, + "amortized_generation_ms_per_sample": 691.9117919169366, + "amortized_batch_wall_ms_per_sample": 714.7108289645985, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 606 + } + ] + }, + { + "sample_ids": [ + "testA-17736-48684" + ], + "batch_size": 1, + "generation_latency_seconds": 3.0041811569826677, + "batch_wall_latency_seconds": 3.0170002209488302, + "amortized_generation_ms_per_sample": 3004.1811569826677, + "amortized_batch_wall_ms_per_sample": 3017.0002209488302, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [508, 426, 765, 983], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 508, + 426, + 765, + 983 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 605 + } + ] + }, + { + "sample_ids": [ + "testA-58475-160949" + ], + "batch_size": 1, + "generation_latency_seconds": 2.428418811992742, + "batch_wall_latency_seconds": 2.442452701041475, + "amortized_generation_ms_per_sample": 2428.418811992742, + "amortized_batch_wall_ms_per_sample": 2442.452701041475, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[46,36,576,999],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 46, + 36, + 576, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 31, + "prompt_tokens": 564 + } + ] + }, + { + "sample_ids": [ + "testA-80455-221493" + ], + "batch_size": 1, + "generation_latency_seconds": 2.4254801200004295, + "batch_wall_latency_seconds": 2.4414882709970698, + "amortized_generation_ms_per_sample": 2425.4801200004295, + "amortized_batch_wall_ms_per_sample": 2441.48827099707, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,483,581,938],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 483, + 581, + 938 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 31, + "prompt_tokens": 541 + } + ] + }, + { + "sample_ids": [ + "testA-31763-87405" + ], + "batch_size": 1, + "generation_latency_seconds": 7.767380379955284, + "batch_wall_latency_seconds": 7.780358829069883, + "amortized_generation_ms_per_sample": 7767.380379955284, + "amortized_batch_wall_ms_per_sample": 7780.358829069883, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [626, 737, 652, 821], \"score\": 0.9}, {\"bbox_2d\": [426, 913, 452, 972], \"score\": 0.9}, {\"bbox_2d\": [356, 852, 378, 936], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 626, + 737, + 652, + 821 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 426, + 913, + 452, + 972 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 356, + 852, + 378, + 936 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 107, + "prompt_tokens": 445 + } + ] + }, + { + "sample_ids": [ + "testB-23920-65742" + ], + "batch_size": 1, + "generation_latency_seconds": 2.9294577399268746, + "batch_wall_latency_seconds": 2.9447807769756764, + "amortized_generation_ms_per_sample": 2929.4577399268746, + "amortized_batch_wall_ms_per_sample": 2944.7807769756764, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [537, 417, 999, 993], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 537, + 417, + 999, + 993 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 39, + "prompt_tokens": 568 + } + ] + }, + { + "sample_ids": [ + "testA-469-1309" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6524032501038164, + "batch_wall_latency_seconds": 0.6627495769644156, + "amortized_generation_ms_per_sample": 652.4032501038164, + "amortized_batch_wall_ms_per_sample": 662.7495769644156, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 621 + } + ] + }, + { + "sample_ids": [ + "testB-56691-155998" + ], + "batch_size": 1, + "generation_latency_seconds": 0.655415665009059, + "batch_wall_latency_seconds": 0.67216857499443, + "amortized_generation_ms_per_sample": 655.415665009059, + "amortized_batch_wall_ms_per_sample": 672.16857499443, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testB-31493-86668" + ], + "batch_size": 1, + "generation_latency_seconds": 4.518256991053931, + "batch_wall_latency_seconds": 4.5385722850915045, + "amortized_generation_ms_per_sample": 4518.256991053931, + "amortized_batch_wall_ms_per_sample": 4538.572285091504, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[537,426,747,765],\"score\":0.95},{\"bbox_2d\":[152,426,466,462],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 537, + 426, + 747, + 765 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 152, + 426, + 466, + 462 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 614 + } + ] + }, + { + "sample_ids": [ + "testA-59284-163254" + ], + "batch_size": 1, + "generation_latency_seconds": 5.580998525954783, + "batch_wall_latency_seconds": 5.5959404041059315, + "amortized_generation_ms_per_sample": 5580.998525954783, + "amortized_batch_wall_ms_per_sample": 5595.9404041059315, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [501, 381, 696, 999], \"score\": 0.9}, {\"bbox_2d\": [561, 300, 814, 426], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 501, + 381, + 696, + 999 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 561, + 300, + 814, + 426 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 73, + "prompt_tokens": 623 + } + ] + }, + { + "sample_ids": [ + "testA-57397-158015" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6980972309829667, + "batch_wall_latency_seconds": 0.709314426057972, + "amortized_generation_ms_per_sample": 698.0972309829667, + "amortized_batch_wall_ms_per_sample": 709.314426057972, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 602 + } + ] + }, + { + "sample_ids": [ + "testB-2935-8111" + ], + "batch_size": 1, + "generation_latency_seconds": 4.602234860998578, + "batch_wall_latency_seconds": 4.6172516259830445, + "amortized_generation_ms_per_sample": 4602.234860998578, + "amortized_batch_wall_ms_per_sample": 4617.2516259830445, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[371,91,707,431],\"score\":0.95},{\"bbox_2d\":[585,181,999,705],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 371, + 91, + 707, + 431 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 585, + 181, + 999, + 705 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 60, + "prompt_tokens": 570 + } + ] + }, + { + "sample_ids": [ + "testB-22290-61226" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6584999479819089, + "batch_wall_latency_seconds": 0.6694054200779647, + "amortized_generation_ms_per_sample": 658.4999479819089, + "amortized_batch_wall_ms_per_sample": 669.4054200779647, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 586 + } + ] + }, + { + "sample_ids": [ + "testB-8416-23140" + ], + "batch_size": 1, + "generation_latency_seconds": 6.426246937015094, + "batch_wall_latency_seconds": 6.434293171972968, + "amortized_generation_ms_per_sample": 6426.246937015094, + "amortized_batch_wall_ms_per_sample": 6434.293171972968, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,292,295,642],\"score\":0.95},{\"bbox_2d\":[234,252,581,727],\"score\":0.95},{\"bbox_2d\":[722,183,959,808],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 292, + 295, + 642 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 234, + 252, + 581, + 727 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 722, + 183, + 959, + 808 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 87, + "prompt_tokens": 563 + } + ] + }, + { + "sample_ids": [ + "testA-60731-167320" + ], + "batch_size": 1, + "generation_latency_seconds": 5.329908382031135, + "batch_wall_latency_seconds": 5.342693028971553, + "amortized_generation_ms_per_sample": 5329.908382031135, + "amortized_batch_wall_ms_per_sample": 5342.693028971553, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [0, 81, 446, 987], \"score\": 0.9}, {\"bbox_2d\": [483, 0, 792, 201], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 81, + 446, + 987 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 483, + 0, + 792, + 201 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 68, + "prompt_tokens": 606 + } + ] + }, + { + "sample_ids": [ + "testA-49055-135086" + ], + "batch_size": 1, + "generation_latency_seconds": 4.7276995739666745, + "batch_wall_latency_seconds": 4.746967880986631, + "amortized_generation_ms_per_sample": 4727.6995739666745, + "amortized_batch_wall_ms_per_sample": 4746.967880986631, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[615,242,851,999],\"score\":0.95},{\"bbox_2d\":[137,142,326,999],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 615, + 242, + 851, + 999 + ], + "score": 0.95 + }, + { + "bbox_2d": [ + 137, + 142, + 326, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 61, + "prompt_tokens": 566 + } + ] + }, + { + "sample_ids": [ + "testA-7857-21644" + ], + "batch_size": 1, + "generation_latency_seconds": 2.446160126943141, + "batch_wall_latency_seconds": 2.4602624271064997, + "amortized_generation_ms_per_sample": 2446.160126943141, + "amortized_batch_wall_ms_per_sample": 2460.2624271064997, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[668,36,999,999],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 668, + 36, + 999, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 32, + "prompt_tokens": 607 + } + ] + }, + { + "sample_ids": [ + "testB-79645-219273" + ], + "batch_size": 1, + "generation_latency_seconds": 15.058248669025488, + "batch_wall_latency_seconds": 15.065027808072045, + "amortized_generation_ms_per_sample": 15058.248669025488, + "amortized_batch_wall_ms_per_sample": 15065.027808072045, + "outputs": [ + { + "raw_response": "{\"objects\": [{\"bbox_2d\": [542, 417, 798, 722], \"score\": 0.9}, {\"bbox_2d\": [295, 421, 552, 715], \"score\": 0.9}, {\"bbox_2d\": [147, 452, 310, 665], \"score\": 0.9}, {\"bbox_2d\": [55, 476, 115, 645], \"score\": 0.9}, {\"bbox_2d\": [0, 448, 62, 677], \"score\": 0.9}, {\"bbox_2d\": [113, 478, 162, 595], \"score\": 0.9}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 542, + 417, + 798, + 722 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 295, + 421, + 552, + 715 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 147, + 452, + 310, + 665 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 55, + 476, + 115, + 645 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 0, + 448, + 62, + 677 + ], + "score": 0.9 + }, + { + "bbox_2d": [ + 113, + 478, + 162, + 595 + ], + "score": 0.9 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 205, + "prompt_tokens": 601 + } + ] + }, + { + "sample_ids": [ + "testB-52921-145707" + ], + "batch_size": 1, + "generation_latency_seconds": 0.65654409409035, + "batch_wall_latency_seconds": 0.6844046280020848, + "amortized_generation_ms_per_sample": 656.54409409035, + "amortized_batch_wall_ms_per_sample": 684.4046280020848, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 571 + } + ] + }, + { + "sample_ids": [ + "testB-69479-191562" + ], + "batch_size": 1, + "generation_latency_seconds": 2.294087173999287, + "batch_wall_latency_seconds": 2.3162708869203925, + "amortized_generation_ms_per_sample": 2294.087173999287, + "amortized_batch_wall_ms_per_sample": 2316.2708869203925, + "outputs": [ + { + "raw_response": "{\"objects\":[{\"bbox_2d\":[0,31,972,999],\"score\":0.95}]}", + "valid_prediction": { + "objects": [ + { + "bbox_2d": [ + 0, + 31, + 972, + 999 + ], + "score": 0.95 + } + ] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 30, + "prompt_tokens": 561 + } + ] + }, + { + "sample_ids": [ + "testA-69466-191524" + ], + "batch_size": 1, + "generation_latency_seconds": 0.6702312079723924, + "batch_wall_latency_seconds": 0.683375352062285, + "amortized_generation_ms_per_sample": 670.2312079723924, + "amortized_batch_wall_ms_per_sample": 683.375352062285, + "outputs": [ + { + "raw_response": "{\"objects\": []}", + "valid_prediction": { + "objects": [] + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 6, + "prompt_tokens": 602 + } + ] + } + ], + "mean_generation_ms_per_sample": 4056.054782922729, + "median_generation_ms_per_sample": 3024.564209044911, + "mean_batch_wall_ms_per_sample": 4071.4221567668574, + "median_batch_wall_ms_per_sample": 3040.390061447397, + "mean_prompt_tokens": 578.6875, + "mean_completion_tokens": 53.203125, + "latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.", + "flops_scope": "This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_latency.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_latency.json new file mode 100644 index 0000000000000000000000000000000000000000..d4d3a8551f8fc5c964291e5c4768033d935c2b3e --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/qwen_latency.json @@ -0,0 +1,3066 @@ +{ + "status": "complete", + "model_kind": "Qwen3.8", + "task": "construction", + "expert": "yolo26x", + "model_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/checkpoints/Qwen3.8-27B-FP8", + "device": "cuda:0", + "gpu_memory_utilization": 0.85, + "sample_count_requested": 66, + "warmup_count": 2, + "measured_count": 64, + "batch_size": 1, + "sample_ids": [ + "0000894", + "0003510", + "0002450", + "0003360", + "0001470", + "0004484", + "0004166", + "0002167", + "0000380", + "0000516", + "0000176", + "0000943", + "0002853", + "0003176", + "0002495", + "0000756", + "0004213", + "0003472", + "0004290", + "0001443", + "0003365", + "0003782", + "0002089", + "0001710", + "0002148", + "0001289", + "0002751", + "0001252", + "0001922", + "0003581", + "0001233", + "0003383", + "0003266", + "0000824", + "0002144", + "0001450", + "0003429", + "0000205", + "0004209", + "0003623", + "0002496", + "0004049", + "0001483", + "0004197", + "0003082", + "0000758", + "0000564", + "0001580", + "0002136", + "0000949", + "0003045", + "0000013", + "0003889", + "0003261", + "0001629", + "0001569", + "0002536", + "0003053", + "0002661", + "0001664", + "0001351", + "0000304", + "0004537", + "0003801", + "0004321", + "0001878" + ], + "batches": [ + { + "sample_ids": [ + "0002450" + ], + "batch_size": 1, + "generation_latency_seconds": 10.705359333078377, + "batch_wall_latency_seconds": 10.720781230949797, + "amortized_generation_ms_per_sample": 10705.359333078377, + "amortized_batch_wall_ms_per_sample": 10720.781230949797, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker standing in the middle of the site is clearly bareheaded, missing a hard hat.\", \"boxes\": [[518, 668, 532, 745]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker standing in the middle of the site is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 518, + 668, + 532, + 745 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1473 + } + ] + }, + { + "sample_ids": [ + "0003360" + ], + "batch_size": 1, + "generation_latency_seconds": 11.7157509700628, + "batch_wall_latency_seconds": 11.733416861039586, + "amortized_generation_ms_per_sample": 11715.7509700628, + "amortized_batch_wall_ms_per_sample": 11733.416861039586, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the excavation is visible on the right side of the image, bordering a deep drop with no guardrails or fencing.\", \"boxes\": [[590, 315, 1000, 495]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the excavation is visible on the right side of the image, bordering a deep drop with no guardrails or fencing.", + "boxes": [ + [ + 590, + 315, + 1000, + 495 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 158, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001470" + ], + "batch_size": 1, + "generation_latency_seconds": 11.342743383022025, + "batch_wall_latency_seconds": 11.35942490492016, + "amortized_generation_ms_per_sample": 11342.743383022025, + "amortized_batch_wall_ms_per_sample": 11359.42490492016, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the ground is clearly visible without a hard hat.\", \"boxes\": [[560, 849, 582, 905]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the ground is clearly visible without a hard hat.", + "boxes": [ + [ + 560, + 849, + 582, + 905 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 141, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004484" + ], + "batch_size": 1, + "generation_latency_seconds": 13.103367771022022, + "batch_wall_latency_seconds": 13.121180747984909, + "amortized_generation_ms_per_sample": 13103.367771022022, + "amortized_batch_wall_ms_per_sample": 13121.180747984909, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"Workers are standing directly beside the tracks and rear of the yellow excavator, within its normal swing and travel zone.\", \"boxes\": [[608, 505, 640, 655], [658, 502, 690, 646]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "Workers are standing directly beside the tracks and rear of the yellow excavator, within its normal swing and travel zone.", + "boxes": [ + [ + 608, + 505, + 640, + 655 + ], + [ + 658, + 502, + 690, + 646 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 171, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004166" + ], + "batch_size": 1, + "generation_latency_seconds": 11.62961833900772, + "batch_wall_latency_seconds": 11.636370092979632, + "amortized_generation_ms_per_sample": 11629.61833900772, + "amortized_batch_wall_ms_per_sample": 11636.370092979632, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the concrete structure where the workers are standing lacks guardrails or fencing, bordering a deep drop/excavation area.\", \"boxes\": [[0, 620, 510, 835]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the concrete structure where the workers are standing lacks guardrails or fencing, bordering a deep drop/excavation area.", + "boxes": [ + [ + 0, + 620, + 510, + 835 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 154, + "prompt_tokens": 1129 + } + ] + }, + { + "sample_ids": [ + "0002167" + ], + "batch_size": 1, + "generation_latency_seconds": 11.212190569960512, + "batch_wall_latency_seconds": 11.235628957045265, + "amortized_generation_ms_per_sample": 11212.190569960512, + "amortized_batch_wall_ms_per_sample": 11235.628957045265, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The edge of the concrete platform borders a deep excavation area without guardrails or fencing, creating a significant fall hazard.\", \"boxes\": [[0, 315, 515, 620]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The edge of the concrete platform borders a deep excavation area without guardrails or fencing, creating a significant fall hazard.", + "boxes": [ + [ + 0, + 315, + 515, + 620 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 151, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0000380" + ], + "batch_size": 1, + "generation_latency_seconds": 8.91931619006209, + "batch_wall_latency_seconds": 8.942373727913946, + "amortized_generation_ms_per_sample": 8919.31619006209, + "amortized_batch_wall_ms_per_sample": 8942.373727913946, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0000516" + ], + "batch_size": 1, + "generation_latency_seconds": 8.323322776006535, + "batch_wall_latency_seconds": 8.3395675040083, + "amortized_generation_ms_per_sample": 8323.322776006535, + "amortized_batch_wall_ms_per_sample": 8339.5675040083, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0000176" + ], + "batch_size": 1, + "generation_latency_seconds": 8.365567820030265, + "batch_wall_latency_seconds": 8.383442353922874, + "amortized_generation_ms_per_sample": 8365.567820030265, + "amortized_batch_wall_ms_per_sample": 8383.442353922874, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000943" + ], + "batch_size": 1, + "generation_latency_seconds": 8.261911686044186, + "batch_wall_latency_seconds": 8.284466639976017, + "amortized_generation_ms_per_sample": 8261.911686044186, + "amortized_batch_wall_ms_per_sample": 8284.466639976017, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002853" + ], + "batch_size": 1, + "generation_latency_seconds": 10.991469252971001, + "batch_wall_latency_seconds": 11.013957520015538, + "amortized_generation_ms_per_sample": 10991.469252971001, + "amortized_batch_wall_ms_per_sample": 11013.957520015538, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation in the foreground lacks guardrails or fencing, exposing a significant drop.\", \"boxes\": [[0, 580, 998, 998]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation in the foreground lacks guardrails or fencing, exposing a significant drop.", + "boxes": [ + [ + 0, + 580, + 998, + 998 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003176" + ], + "batch_size": 1, + "generation_latency_seconds": 10.860358019010164, + "batch_wall_latency_seconds": 10.878267464926466, + "amortized_generation_ms_per_sample": 10860.358019010164, + "amortized_batch_wall_ms_per_sample": 10878.267464926466, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing a significant drop.\", \"boxes\": [[0, 500, 640, 998]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing a significant drop.", + "boxes": [ + [ + 0, + 500, + 640, + 998 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002495" + ], + "batch_size": 1, + "generation_latency_seconds": 10.56970894197002, + "batch_wall_latency_seconds": 10.593795233988203, + "amortized_generation_ms_per_sample": 10569.70894197002, + "amortized_batch_wall_ms_per_sample": 10593.795233988203, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker crouching on the ground is clearly bareheaded, missing a hard hat.\", \"boxes\": [[475, 525, 571, 688]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker crouching on the ground is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 475, + 525, + 571, + 688 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 146, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000756" + ], + "batch_size": 1, + "generation_latency_seconds": 8.038495411048643, + "batch_wall_latency_seconds": 8.052992713986896, + "amortized_generation_ms_per_sample": 8038.495411048643, + "amortized_batch_wall_ms_per_sample": 8052.992713986896, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004213" + ], + "batch_size": 1, + "generation_latency_seconds": 8.011779504013248, + "batch_wall_latency_seconds": 8.047042324091308, + "amortized_generation_ms_per_sample": 8011.779504013248, + "amortized_batch_wall_ms_per_sample": 8047.042324091308, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003472" + ], + "batch_size": 1, + "generation_latency_seconds": 8.286275054095313, + "batch_wall_latency_seconds": 8.304093755898066, + "amortized_generation_ms_per_sample": 8286.275054095313, + "amortized_batch_wall_ms_per_sample": 8304.093755898066, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004290" + ], + "batch_size": 1, + "generation_latency_seconds": 8.341199293034151, + "batch_wall_latency_seconds": 8.358467961079441, + "amortized_generation_ms_per_sample": 8341.199293034151, + "amortized_batch_wall_ms_per_sample": 8358.467961079441, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001443" + ], + "batch_size": 1, + "generation_latency_seconds": 15.180880905943923, + "batch_wall_latency_seconds": 15.206637792987749, + "amortized_generation_ms_per_sample": 15180.880905943923, + "amortized_batch_wall_ms_per_sample": 15206.637792987749, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the left side of the image, standing on the ground near the scaffolding, is clearly bareheaded and not wearing a hard hat.\", \"boxes\": [[238, 458, 278, 688]]}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"A worker is standing on a low concrete beam/ledge without a safety harness, exposed to a fall from a height of approximately 3 meters or more.\", \"boxes\": [[315, 325, 368, 505]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the left side of the image, standing on the ground near the scaffolding, is clearly bareheaded and not wearing a hard hat.", + "boxes": [ + [ + 238, + 458, + 278, + 688 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "A worker is standing on a low concrete beam/ledge without a safety harness, exposed to a fall from a height of approximately 3 meters or more.", + "boxes": [ + [ + 315, + 325, + 368, + 505 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 207, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003365" + ], + "batch_size": 1, + "generation_latency_seconds": 8.154706532950513, + "batch_wall_latency_seconds": 8.171265691053122, + "amortized_generation_ms_per_sample": 8154.706532950513, + "amortized_batch_wall_ms_per_sample": 8171.265691053122, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003782" + ], + "batch_size": 1, + "generation_latency_seconds": 8.263969205901958, + "batch_wall_latency_seconds": 8.278776310035028, + "amortized_generation_ms_per_sample": 8263.969205901958, + "amortized_batch_wall_ms_per_sample": 8278.776310035028, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002089" + ], + "batch_size": 1, + "generation_latency_seconds": 8.21782910998445, + "batch_wall_latency_seconds": 8.233122498961166, + "amortized_generation_ms_per_sample": 8217.82910998445, + "amortized_batch_wall_ms_per_sample": 8233.122498961166, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1531 + } + ] + }, + { + "sample_ids": [ + "0001710" + ], + "batch_size": 1, + "generation_latency_seconds": 8.018760634935461, + "batch_wall_latency_seconds": 8.032283170963638, + "amortized_generation_ms_per_sample": 8018.760634935461, + "amortized_batch_wall_ms_per_sample": 8032.2831709636375, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0002148" + ], + "batch_size": 1, + "generation_latency_seconds": 10.378318366943859, + "batch_wall_latency_seconds": 10.404688362963498, + "amortized_generation_ms_per_sample": 10378.318366943859, + "amortized_batch_wall_ms_per_sample": 10404.688362963498, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker in the middle distance is clearly bareheaded, missing a hard hat.\", \"boxes\": [[561, 300, 582, 408]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker in the middle distance is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 561, + 300, + 582, + 408 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 144, + "prompt_tokens": 1531 + } + ] + }, + { + "sample_ids": [ + "0001289" + ], + "batch_size": 1, + "generation_latency_seconds": 11.48702211806085, + "batch_wall_latency_seconds": 11.514577882946469, + "amortized_generation_ms_per_sample": 11487.02211806085, + "amortized_batch_wall_ms_per_sample": 11514.577882946469, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation on the right side lacks guardrails or fencing, bordering a steep drop of several meters.\", \"boxes\": [[430, 240, 998, 480]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation on the right side lacks guardrails or fencing, bordering a steep drop of several meters.", + "boxes": [ + [ + 430, + 240, + 998, + 480 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 156, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002751" + ], + "batch_size": 1, + "generation_latency_seconds": 8.446506896987557, + "batch_wall_latency_seconds": 8.467677644919604, + "amortized_generation_ms_per_sample": 8446.506896987557, + "amortized_batch_wall_ms_per_sample": 8467.677644919604, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001252" + ], + "batch_size": 1, + "generation_latency_seconds": 8.315838309004903, + "batch_wall_latency_seconds": 8.333076176000759, + "amortized_generation_ms_per_sample": 8315.838309004903, + "amortized_batch_wall_ms_per_sample": 8333.076176000759, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001922" + ], + "batch_size": 1, + "generation_latency_seconds": 8.236222366103902, + "batch_wall_latency_seconds": 8.253340549999848, + "amortized_generation_ms_per_sample": 8236.222366103902, + "amortized_batch_wall_ms_per_sample": 8253.340549999848, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1107 + } + ] + }, + { + "sample_ids": [ + "0003581" + ], + "batch_size": 1, + "generation_latency_seconds": 8.227389317005873, + "batch_wall_latency_seconds": 8.240366365062073, + "amortized_generation_ms_per_sample": 8227.389317005873, + "amortized_batch_wall_ms_per_sample": 8240.366365062073, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1694 + } + ] + }, + { + "sample_ids": [ + "0001233" + ], + "batch_size": 1, + "generation_latency_seconds": 11.756323368987069, + "batch_wall_latency_seconds": 11.764942721929401, + "amortized_generation_ms_per_sample": 11756.323368987069, + "amortized_batch_wall_ms_per_sample": 11764.942721929401, + "outputs": [ + { + "raw_response": "{\n \"events\": {\n \"rule_1_ppe_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_2_fall_protection_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_3_unprotected_edge_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n },\n \"rule_4_excavator_proximity_violation\": {\n \"present\": false,\n \"reason\": null,\n \"boxes\": []\n }\n }\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 163, + "prompt_tokens": 1453 + } + ] + }, + { + "sample_ids": [ + "0003383" + ], + "batch_size": 1, + "generation_latency_seconds": 8.106548791052774, + "batch_wall_latency_seconds": 8.117786544025876, + "amortized_generation_ms_per_sample": 8106.548791052774, + "amortized_batch_wall_ms_per_sample": 8117.786544025876, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1573 + } + ] + }, + { + "sample_ids": [ + "0003266" + ], + "batch_size": 1, + "generation_latency_seconds": 8.274491983000189, + "batch_wall_latency_seconds": 8.288345888955519, + "amortized_generation_ms_per_sample": 8274.491983000189, + "amortized_batch_wall_ms_per_sample": 8288.345888955519, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000824" + ], + "batch_size": 1, + "generation_latency_seconds": 11.22929672000464, + "batch_wall_latency_seconds": 11.251729905954562, + "amortized_generation_ms_per_sample": 11229.29672000464, + "amortized_batch_wall_ms_per_sample": 11251.729905954562, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing on the ground within the normal reach of the excavator's boom and bucket, which are positioned close to the worker.\", \"boxes\": [[155, 592, 191, 700]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing on the ground within the normal reach of the excavator's boom and bucket, which are positioned close to the worker.", + "boxes": [ + [ + 155, + 592, + 191, + 700 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 156, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002144" + ], + "batch_size": 1, + "generation_latency_seconds": 10.366267680074088, + "batch_wall_latency_seconds": 10.379591763019562, + "amortized_generation_ms_per_sample": 10366.267680074088, + "amortized_batch_wall_ms_per_sample": 10379.591763019562, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"Worker on the left is clearly bareheaded, missing a hard hat.\", \"boxes\": [[46, 383, 112, 545]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "Worker on the left is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 46, + 383, + 112, + 545 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 141, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001450" + ], + "batch_size": 1, + "generation_latency_seconds": 8.482930272002704, + "batch_wall_latency_seconds": 8.493541297968477, + "amortized_generation_ms_per_sample": 8482.930272002704, + "amortized_batch_wall_ms_per_sample": 8493.541297968477, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0003429" + ], + "batch_size": 1, + "generation_latency_seconds": 10.942747003980912, + "batch_wall_latency_seconds": 11.023728625965305, + "amortized_generation_ms_per_sample": 10942.747003980912, + "amortized_batch_wall_ms_per_sample": 11023.728625965305, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the left is clearly bareheaded, missing a hard hat.\", \"boxes\": [[129, 331, 202, 555]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the left is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 129, + 331, + 202, + 555 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 143, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000205" + ], + "batch_size": 1, + "generation_latency_seconds": 12.731149753089994, + "batch_wall_latency_seconds": 12.753692528931424, + "amortized_generation_ms_per_sample": 12731.149753089994, + "amortized_batch_wall_ms_per_sample": 12753.692528931424, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The circular concrete structure has a deep opening in the center and the worker is standing near the edge without guardrails or fencing, indicating an unprotected edge over a deep drop.\", \"boxes\": [[0, 100, 1000, 1000]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The circular concrete structure has a deep opening in the center and the worker is standing near the edge without guardrails or fencing, indicating an unprotected edge over a deep drop.", + "boxes": [ + [ + 0, + 100, + 1000, + 1000 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 164, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004209" + ], + "batch_size": 1, + "generation_latency_seconds": 8.481906582019292, + "batch_wall_latency_seconds": 8.510338544961996, + "amortized_generation_ms_per_sample": 8481.906582019292, + "amortized_batch_wall_ms_per_sample": 8510.338544961996, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003623" + ], + "batch_size": 1, + "generation_latency_seconds": 11.558857796015218, + "batch_wall_latency_seconds": 11.585797557956539, + "amortized_generation_ms_per_sample": 11558.857796015218, + "amortized_batch_wall_ms_per_sample": 11585.79755795654, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"Worker on the upper level is exposed to a fall hazard without a safety harness or effective edge protection.\", \"boxes\": [[508, 31, 646, 152]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "Worker on the upper level is exposed to a fall hazard without a safety harness or effective edge protection.", + "boxes": [ + [ + 508, + 31, + 646, + 152 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1694 + } + ] + }, + { + "sample_ids": [ + "0002496" + ], + "batch_size": 1, + "generation_latency_seconds": 11.345404008985497, + "batch_wall_latency_seconds": 11.363981842063367, + "amortized_generation_ms_per_sample": 11345.404008985497, + "amortized_batch_wall_ms_per_sample": 11363.981842063367, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation pit lacks guardrails or fencing, exposing workers to a significant fall hazard.\", \"boxes\": [[42, 442, 998, 685]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation pit lacks guardrails or fencing, exposing workers to a significant fall hazard.", + "boxes": [ + [ + 42, + 442, + 998, + 685 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 149, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004049" + ], + "batch_size": 1, + "generation_latency_seconds": 8.281572339008562, + "batch_wall_latency_seconds": 8.321087789023295, + "amortized_generation_ms_per_sample": 8281.572339008562, + "amortized_batch_wall_ms_per_sample": 8321.087789023295, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001483" + ], + "batch_size": 1, + "generation_latency_seconds": 8.35940665891394, + "batch_wall_latency_seconds": 8.414654169930145, + "amortized_generation_ms_per_sample": 8359.40665891394, + "amortized_batch_wall_ms_per_sample": 8414.654169930145, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004197" + ], + "batch_size": 1, + "generation_latency_seconds": 11.471120469039306, + "batch_wall_latency_seconds": 11.495890544028953, + "amortized_generation_ms_per_sample": 11471.120469039306, + "amortized_batch_wall_ms_per_sample": 11495.890544028953, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing on the ground directly in the path of the excavator's bucket and within the machine's active swing zone.\", \"boxes\": [[638, 545, 668, 682]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing on the ground directly in the path of the excavator's bucket and within the machine's active swing zone.", + "boxes": [ + [ + 638, + 545, + 668, + 682 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 154, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003082" + ], + "batch_size": 1, + "generation_latency_seconds": 11.027026457944885, + "batch_wall_latency_seconds": 11.062949666986242, + "amortized_generation_ms_per_sample": 11027.026457944885, + "amortized_batch_wall_ms_per_sample": 11062.949666986242, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing workers to a significant drop.\", \"boxes\": [[0, 0, 670, 640]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation on the left side lacks guardrails or fencing, exposing workers to a significant drop.", + "boxes": [ + [ + 0, + 0, + 670, + 640 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000758" + ], + "batch_size": 1, + "generation_latency_seconds": 11.055003611021675, + "batch_wall_latency_seconds": 11.07016703102272, + "amortized_generation_ms_per_sample": 11055.003611021675, + "amortized_batch_wall_ms_per_sample": 11070.16703102272, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing within the normal reach of the excavator's boom and bucket in the active work area.\", \"boxes\": [[636, 330, 725, 582]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing within the normal reach of the excavator's boom and bucket in the active work area.", + "boxes": [ + [ + 636, + 330, + 725, + 582 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 150, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000564" + ], + "batch_size": 1, + "generation_latency_seconds": 13.539180523017421, + "batch_wall_latency_seconds": 13.570679377065971, + "amortized_generation_ms_per_sample": 13539.180523017421, + "amortized_batch_wall_ms_per_sample": 13570.679377065971, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on the right is walking on the construction site without a hard hat.\", \"boxes\": [[576, 515, 611, 632]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": true, \"reason\": \"A worker is standing very close to the excavator's tracks and rear counterweight, within its normal swing and travel zone.\", \"boxes\": [[472, 466, 508, 603]]}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on the right is walking on the construction site without a hard hat.", + "boxes": [ + [ + 576, + 515, + 611, + 632 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": true, + "reason": "A worker is standing very close to the excavator's tracks and rear counterweight, within its normal swing and travel zone.", + "boxes": [ + [ + 472, + 466, + 508, + 603 + ] + ] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 187, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001580" + ], + "batch_size": 1, + "generation_latency_seconds": 8.109316511079669, + "batch_wall_latency_seconds": 8.123522659065202, + "amortized_generation_ms_per_sample": 8109.316511079669, + "amortized_batch_wall_ms_per_sample": 8123.522659065202, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1333 + } + ] + }, + { + "sample_ids": [ + "0002136" + ], + "batch_size": 1, + "generation_latency_seconds": 10.921932367025875, + "batch_wall_latency_seconds": 10.956237377016805, + "amortized_generation_ms_per_sample": 10921.932367025875, + "amortized_batch_wall_ms_per_sample": 10956.237377016805, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The upper edge of the deep excavation is accessible and lacks guardrails or fencing, exposing workers to a significant drop.\", \"boxes\": [[0, 62, 1000, 312]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The upper edge of the deep excavation is accessible and lacks guardrails or fencing, exposing workers to a significant drop.", + "boxes": [ + [ + 0, + 62, + 1000, + 312 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 149, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0000949" + ], + "batch_size": 1, + "generation_latency_seconds": 10.729247406939976, + "batch_wall_latency_seconds": 10.767821826972067, + "amortized_generation_ms_per_sample": 10729.247406939976, + "amortized_batch_wall_ms_per_sample": 10767.821826972067, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"The driver inside the truck cab is clearly visible with a bare head, missing a hard hat.\", \"boxes\": [[378, 208, 408, 258]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "The driver inside the truck cab is clearly visible with a bare head, missing a hard hat.", + "boxes": [ + [ + 378, + 208, + 408, + 258 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003045" + ], + "batch_size": 1, + "generation_latency_seconds": 7.96182514692191, + "batch_wall_latency_seconds": 7.970436848932877, + "amortized_generation_ms_per_sample": 7961.82514692191, + "amortized_batch_wall_ms_per_sample": 7970.436848932877, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1129 + } + ] + }, + { + "sample_ids": [ + "0000013" + ], + "batch_size": 1, + "generation_latency_seconds": 8.21879173698835, + "batch_wall_latency_seconds": 8.24058064003475, + "amortized_generation_ms_per_sample": 8218.79173698835, + "amortized_batch_wall_ms_per_sample": 8240.58064003475, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003889" + ], + "batch_size": 1, + "generation_latency_seconds": 8.273510461091064, + "batch_wall_latency_seconds": 8.289232861017808, + "amortized_generation_ms_per_sample": 8273.510461091064, + "amortized_batch_wall_ms_per_sample": 8289.232861017808, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003261" + ], + "batch_size": 1, + "generation_latency_seconds": 8.405189790995792, + "batch_wall_latency_seconds": 8.4461084890645, + "amortized_generation_ms_per_sample": 8405.189790995792, + "amortized_batch_wall_ms_per_sample": 8446.1084890645, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001629" + ], + "batch_size": 1, + "generation_latency_seconds": 8.233813347993419, + "batch_wall_latency_seconds": 8.26478240697179, + "amortized_generation_ms_per_sample": 8233.813347993419, + "amortized_batch_wall_ms_per_sample": 8264.78240697179, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001569" + ], + "batch_size": 1, + "generation_latency_seconds": 8.24229718092829, + "batch_wall_latency_seconds": 8.27361940103583, + "amortized_generation_ms_per_sample": 8242.29718092829, + "amortized_batch_wall_ms_per_sample": 8273.61940103583, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002536" + ], + "batch_size": 1, + "generation_latency_seconds": 8.316832296084613, + "batch_wall_latency_seconds": 8.332275224965997, + "amortized_generation_ms_per_sample": 8316.832296084613, + "amortized_batch_wall_ms_per_sample": 8332.275224965997, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0003053" + ], + "batch_size": 1, + "generation_latency_seconds": 8.387920169043355, + "batch_wall_latency_seconds": 8.417040045023896, + "amortized_generation_ms_per_sample": 8387.920169043355, + "amortized_batch_wall_ms_per_sample": 8417.040045023896, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0002661" + ], + "batch_size": 1, + "generation_latency_seconds": 11.386560487910174, + "batch_wall_latency_seconds": 11.400971814990044, + "amortized_generation_ms_per_sample": 11386.560487910174, + "amortized_batch_wall_ms_per_sample": 11400.971814990044, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the deep excavation trench lacks guardrails or fencing, exposing workers to a significant fall hazard.\", \"boxes\": [[0, 200, 780, 650]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the deep excavation trench lacks guardrails or fencing, exposing workers to a significant fall hazard.", + "boxes": [ + [ + 0, + 200, + 780, + 650 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 150, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001664" + ], + "batch_size": 1, + "generation_latency_seconds": 8.633664096007124, + "batch_wall_latency_seconds": 8.660372731043026, + "amortized_generation_ms_per_sample": 8633.664096007124, + "amortized_batch_wall_ms_per_sample": 8660.372731043026, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0001351" + ], + "batch_size": 1, + "generation_latency_seconds": 12.240043157013133, + "batch_wall_latency_seconds": 12.259637883980758, + "amortized_generation_ms_per_sample": 12240.043157013133, + "amortized_batch_wall_ms_per_sample": 12259.637883980758, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": true, \"reason\": \"The top edge of the concrete retaining wall borders a deep excavation area and lacks guardrails or fencing, creating a fall hazard for workers on the upper level.\", \"boxes\": [[0, 500, 998, 580]]}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}\n}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": true, + "reason": "The top edge of the concrete retaining wall borders a deep excavation area and lacks guardrails or fencing, creating a fall hazard for workers on the upper level.", + "boxes": [ + [ + 0, + 500, + 998, + 580 + ] + ] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 159, + "prompt_tokens": 1683 + } + ] + }, + { + "sample_ids": [ + "0000304" + ], + "batch_size": 1, + "generation_latency_seconds": 8.286614268901758, + "batch_wall_latency_seconds": 8.305405738996342, + "amortized_generation_ms_per_sample": 8286.614268901758, + "amortized_batch_wall_ms_per_sample": 8305.405738996342, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1243 + } + ] + }, + { + "sample_ids": [ + "0004537" + ], + "batch_size": 1, + "generation_latency_seconds": 10.83758004498668, + "batch_wall_latency_seconds": 10.849978967919014, + "amortized_generation_ms_per_sample": 10837.58004498668, + "amortized_batch_wall_ms_per_sample": 10849.978967919014, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"Worker is standing on rebar at a significant height without a safety harness, exposed to a fall.\", \"boxes\": [[538, 292, 706, 568]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "Worker is standing on rebar at a significant height without a safety harness, exposed to a fall.", + "boxes": [ + [ + 538, + 292, + 706, + 568 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 148, + "prompt_tokens": 1405 + } + ] + }, + { + "sample_ids": [ + "0003801" + ], + "batch_size": 1, + "generation_latency_seconds": 8.35220990900416, + "batch_wall_latency_seconds": 8.370414639008231, + "amortized_generation_ms_per_sample": 8352.20990900416, + "amortized_batch_wall_ms_per_sample": 8370.414639008231, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 110, + "prompt_tokens": 1705 + } + ] + }, + { + "sample_ids": [ + "0004321" + ], + "batch_size": 1, + "generation_latency_seconds": 11.045618020929396, + "batch_wall_latency_seconds": 11.056004835991189, + "amortized_generation_ms_per_sample": 11045.618020929396, + "amortized_batch_wall_ms_per_sample": 11056.004835991189, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": true, \"reason\": \"A worker on foot near the concrete pump hose is clearly bareheaded, missing a hard hat.\", \"boxes\": [[451, 686, 468, 758]]}, \"rule_2_fall_protection_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": true, + "reason": "A worker on foot near the concrete pump hose is clearly bareheaded, missing a hard hat.", + "boxes": [ + [ + 451, + 686, + 468, + 758 + ] + ] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 147, + "prompt_tokens": 1393 + } + ] + }, + { + "sample_ids": [ + "0001878" + ], + "batch_size": 1, + "generation_latency_seconds": 12.032795147038996, + "batch_wall_latency_seconds": 12.044023220078088, + "amortized_generation_ms_per_sample": 12032.795147038996, + "amortized_batch_wall_ms_per_sample": 12044.023220078088, + "outputs": [ + { + "raw_response": "{\"events\": {\"rule_1_ppe_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_2_fall_protection_violation\": {\"present\": true, \"reason\": \"A worker is standing on the top of the concrete pump truck chassis, which is elevated above the ground, without any visible safety harness or fall protection equipment.\", \"boxes\": [[483, 526, 518, 625]]}, \"rule_3_unprotected_edge_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}, \"rule_4_excavator_proximity_violation\": {\"present\": false, \"reason\": null, \"boxes\": []}}}", + "valid_prediction": { + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": true, + "reason": "A worker is standing on the top of the concrete pump truck chassis, which is elevated above the ground, without any visible safety harness or fall protection equipment.", + "boxes": [ + [ + 483, + 526, + 518, + 625 + ] + ] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + }, + "error": null, + "finish_reason": "stop", + "completion_tokens": 159, + "prompt_tokens": 1705 + } + ] + } + ], + "mean_generation_ms_per_sample": 9800.482400676628, + "median_generation_ms_per_sample": 8558.297184004914, + "mean_batch_wall_ms_per_sample": 9822.475872695577, + "median_batch_wall_ms_per_sample": 8585.355638002511, + "mean_prompt_tokens": 1624.5625, + "mean_completion_tokens": 130.796875, + "latency_scope": "A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.", + "flops_scope": "This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/yolo26x_router.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/yolo26x_router.json new file mode 100644 index 0000000000000000000000000000000000000000..cef08acee599ed6dc1e24e3c873932ba815f9e87 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/yolo26x_router.json @@ -0,0 +1,238 @@ +{ + "status": "complete", + "run": "shared_12_yolo26x_Qwen3.8", + "config": { + "fraction": 1.0, + "seed": 42, + "freeze_encoder": true, + "fusion": "concat", + "harm_cost": 1.0, + "batch_size": 256, + "weight_decay": 0.01, + "max_epochs": 50, + "patience": 8, + "cpu_threads": 1, + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90, + "expert": "yolo26x", + "task": "construction", + "endpoint": "Qwen3.8", + "branches": [ + "image", + "hidden", + "confidence", + "output" + ], + "hidden_layer": "feature_blocks__layer_16", + "id": "shared_12_yolo26x_Qwen3.8", + "stage": "shared_mandatory_inputs" + }, + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_shared_image_hidden_20260910/runs/shared_12_yolo26x_Qwen3.8/best.pt", + "score": "logit_rescue-logit_harm", + "test_rows_recomputed": 3004, + "selected_decisions_match_main": true, + "max_score_abs_difference": 0.0, + "actual_call_fraction": 0.4876830892143808, + "sample_count": 64, + "sample_ids": [ + "0002628", + "0003242", + "0001568", + "0001119", + "0003648", + "0000187", + "0001891", + "0001689", + "0003372", + "0002549", + "0000915", + "0003633", + "0004424", + "0001627", + "0001562", + "0002018", + "0000254", + "0002719", + "0002730", + "0004413", + "0000891", + "0000150", + "0003223", + "0004731", + "0001587", + "0000483", + "0001116", + "0003376", + "0001225", + "0003082", + "0004447", + "0000283", + "0000563", + "0001254", + "0004759", + "0001868", + "0001860", + "0004087", + "0002962", + "0001802", + "0002973", + "0002648", + "0004350", + "0004377", + "0004809", + "0003694", + "0001572", + "0001550", + "0000213", + "0002266", + "0002445", + "0004513", + "0002124", + "0004391", + "0003679", + "0002902", + "0002508", + "0004767", + "0003840", + "0003498", + "0001579", + "0001748", + "0004706", + "0004387" + ], + "batch_size": 1, + "latency_ms": [ + 17.35539303626865, + 13.579263002611697, + 15.319477999582887, + 14.879688038490713, + 24.25052202306688, + 20.667891018092632, + 20.015343907289207, + 19.221558002755046, + 20.937222987413406, + 26.26270893961191, + 17.899635946378112, + 16.57471200451255, + 18.844021018594503, + 21.3212959934026, + 19.545868039131165, + 64.57463896367699, + 20.962884067557752, + 24.004184058867395, + 20.19065897911787, + 18.50342599209398, + 19.576229038648307, + 25.253725005313754, + 21.430211956612766, + 25.169604923576117, + 17.8982320940122, + 21.415626979433, + 22.9903319850564, + 229.59673800505698, + 19.89229000173509, + 18.98771501146257, + 18.157949903979897, + 22.423187969252467, + 19.864327972754836, + 21.502194926142693, + 18.544791964814067, + 50.5554600385949, + 20.046600024215877, + 16.299219918437302, + 18.668118049390614, + 24.017272982746363, + 24.73958197515458, + 18.68593308608979, + 20.250934991054237, + 23.676551994867623, + 23.003229987807572, + 24.6098650386557, + 21.308812079951167, + 17.95686304103583, + 18.23423698078841, + 14.343310962431133, + 25.8766250917688, + 18.20633898023516, + 30.77617404051125, + 19.3530690157786, + 15.836970997042954, + 19.045730936340988, + 16.162591986358166, + 25.458221905864775, + 18.277363968081772, + 52.543785073794425, + 22.9536690749228, + 17.81571004539728, + 18.550690030679107, + 21.854626946151257 + ], + "mean_latency_ms_per_input": 25.261267359383055, + "head_mean_latency_ms": 0.486134999897331, + "gflops_per_input": [ + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752, + 3.183754752 + ], + "mean_gflops_per_input": 3.183754752, + "operators": [ + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + }, + { + "aten.convolution": 228802560, + "aten.bmm": 679257600, + "aten._scaled_dot_product_flash_attention": 236027904, + "aten.addmm": 2039666688 + } + ], + "scope": "image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only" +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/yolo26x_specialist.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/yolo26x_specialist.json new file mode 100644 index 0000000000000000000000000000000000000000..251decaacc2a6eb1affbe81ae0c03c9f13b031aa --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/components/yolo26x_specialist.json @@ -0,0 +1,254 @@ +{ + "component": "specialist", + "expert": "yolo26x", + "task": "construction", + "device": "cuda:0", + "checkpoint": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/models/construction/yolo26x.pt", + "sample_count": 64, + "batch_size": 1, + "sample_ids": [ + "0000944", + "0003511", + "0003365", + "0001351", + "0002149", + "0003368", + "0000176", + "0004487", + "0000013", + "0003583", + "0000758", + "0001291", + "0001451", + "0002755", + "0003088", + "0004538", + "0004324", + "0000516", + "0003893", + "0003475", + "0002137", + "0003272", + "0001711", + "0003262", + "0001470", + "0002145", + "0002662", + "0000756", + "0001631", + "0001924", + "0002496", + "0001443", + "0004053", + "0002168", + "0001881", + "0001666", + "0000304", + "0001252", + "0004844", + "0000949", + "0003430", + "0003807", + "0004199", + "0002538", + "0002091", + "0001569", + "0003048", + "0002854", + "0000380", + "0001234", + "0001484", + "0003180", + "0000824", + "0003784", + "0000894", + "0004168", + "0000564", + "0004296", + "0003054", + "0001583", + "0003386", + "0000205", + "0004214", + "0003625" + ], + "model_load_seconds_excluded_from_latency": true, + "forward_latency_ms_per_sample": [ + 41.741790771484375, + 51.50377655029297, + 78.82457733154297, + 87.29910278320312, + 49.47385787963867, + 49.40816116333008, + 168.38259887695312, + 42.57542419433594, + 40.342529296875, + 54.95475387573242, + 48.86323165893555, + 40.684993743896484, + 282.5484619140625, + 47.938270568847656, + 45.019649505615234, + 268.04327392578125, + 51.39228820800781, + 41.36457443237305, + 54.01919937133789, + 43.2599983215332, + 47.13923263549805, + 41.1484489440918, + 35.99542236328125, + 80.46995544433594, + 83.43251037597656, + 48.88550567626953, + 171.4989776611328, + 58.917728424072266, + 46.90486526489258, + 56.416385650634766, + 44.49577713012695, + 49.42009735107422, + 43.6354866027832, + 55.22761535644531, + 48.02000045776367, + 39.44412612915039, + 30.26905632019043, + 492.74456787109375, + 45.74156951904297, + 39.223392486572266, + 50.18822479248047, + 43.890113830566406, + 51.966495513916016, + 47.814815521240234, + 54.13961410522461, + 39.215328216552734, + 56.004703521728516, + 39.528831481933594, + 396.5307312011719, + 83.4832992553711, + 50.0503044128418, + 153.1714630126953, + 61.07219314575195, + 52.57244873046875, + 42.75983810424805, + 38.4617919921875, + 44.08700942993164, + 53.36006546020508, + 45.519649505615234, + 290.8823547363281, + 48.10774230957031, + 53.34476852416992, + 51.016319274902344, + 42.9719352722168 + ], + "end_to_end_latency_ms_per_sample": [ + 35.17372906208038, + 48.07086195796728, + 46.07532289810479, + 45.34715902991593, + 46.541396994143724, + 45.594630064442754, + 45.38595397025347, + 39.57494895439595, + 39.812199072912335, + 51.36631103232503, + 42.81833197455853, + 33.49219704978168, + 37.3103239107877, + 38.96563302259892, + 40.327758993953466, + 44.32409000582993, + 43.25311293359846, + 33.99710892699659, + 49.12354901898652, + 45.043769059702754, + 47.554407035931945, + 37.903546006418765, + 32.56980108562857, + 35.192839917726815, + 39.34567596297711, + 46.67255701497197, + 49.70169102307409, + 52.48237494379282, + 43.415413005277514, + 48.36420400533825, + 48.96194895263761, + 51.6092429170385, + 37.311508087441325, + 50.858690054155886, + 43.94579504150897, + 36.09434806276113, + 26.66287892498076, + 41.9749760767445, + 34.900374012067914, + 39.04577391222119, + 51.60050210542977, + 46.00996105000377, + 53.40714694466442, + 51.78673507180065, + 54.96632098220289, + 46.47863702848554, + 47.48446703888476, + 36.593522992916405, + 37.32479899190366, + 26.43477905075997, + 39.97986298054457, + 55.87899999227375, + 40.821335976943374, + 45.88699492160231, + 40.707007981836796, + 34.627774031832814, + 43.4188200160861, + 49.56604307517409, + 39.82124803587794, + 249.81214595027268, + 38.2277499884367, + 48.29735495150089, + 47.6501100929454, + 40.76927900314331 + ], + "mean_forward_latency_ms_per_sample": 78.70017620921135, + "median_forward_latency_ms_per_sample": 49.41412925720215, + "mean_end_to_end_latency_ms_per_sample": 46.30809425361804, + "median_end_to_end_latency_ms_per_sample": 43.68230752879754, + "flop_profiles": [ + { + "batch_size": 1, + "flops_counted": 116518072320, + "gflops_counted_per_sample": 116.51807232, + "operators": { + "aten.convolution": 116319006720, + "aten.bmm": 199065600 + } + }, + { + "batch_size": 1, + "flops_counted": 145709798400, + "gflops_counted_per_sample": 145.7097984, + "operators": { + "aten.convolution": 145398758400, + "aten.bmm": 311040000 + } + }, + { + "batch_size": 1, + "flops_counted": 145709798400, + "gflops_counted_per_sample": 145.7097984, + "operators": { + "aten.convolution": 145398758400, + "aten.bmm": 311040000 + } + }, + { + "batch_size": 1, + "flops_counted": 135976458240, + "gflops_counted_per_sample": 135.97645824, + "operators": { + "aten.convolution": 135705507840, + "aten.bmm": 270950400 + } + } + ], + "mean_counted_gflops_per_sample": 135.97853184000002, + "flops_scope": "GPU FlopCounterMode around the real Ultralytics predict path; multiply-add counted as 2 FLOPs. Elementwise, postprocessing and unsupported custom ops may be missing.", + "latency_scope": "A800 wall-clock latency for real Ultralytics prediction including image loading/preprocess, forward and result postprocessing, excluding model load; batch processing uses the recorded batch size.", + "completed": true +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact.pdf b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact.pdf new file mode 100644 index 0000000000000000000000000000000000000000..698d2658133ba6a567b843856aa53bd530e36781 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact.pdf differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact.png b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact.png new file mode 100644 index 0000000000000000000000000000000000000000..72f5ad4950aebbc9164ef99c61a2067e15ae10c4 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bdd66e43cb5992efd68d905c50e4d88a4c7254b49c0769888657fc998285fd71 +size 123334 diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact_verified.png b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact_verified.png new file mode 100644 index 0000000000000000000000000000000000000000..3e1080fb94128fd94859f696c0c628ce2caa9b1b Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_compact_verified.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_summary.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_summary.json new file mode 100644 index 0000000000000000000000000000000000000000..dff4f5c3e165c1e5a0fdf467d55e836ed5c10706 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/cost_summary.json @@ -0,0 +1,75 @@ +{ + "components": { + "grefcoco|instancevg|Qwen3.8": { + "specialist_gflops": 7.502680064, + "router_gflops": 3.642171392, + "large_gflops": 32554.061985335, + "specialist_ms": 46.08042127620138, + "router_ms": 17.032772149832454, + "large_ms": 4071.4221567668574, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + }, + "construction|yolo26x|Qwen3.8": { + "specialist_gflops": 135.97853184000002, + "router_gflops": 3.183754752, + "large_gflops": 96874.712551641, + "specialist_ms": 46.30809425361804, + "router_ms": 25.261267359383055, + "large_ms": 9822.475872695577, + "large_samples": 64, + "flops_probes": 8, + "specialist_samples": 64, + "router_samples": 64, + "batch_size": 1 + } + }, + "selected_points": [ + { + "pair": "grefcoco|instancevg|Qwen3.8", + "method": "learned", + "nominal_validation_budget": 0.56, + "actual_test_call_fraction": 0.5355755324277571, + "test_metric": 0.7528287440093016, + "estimated_gflops_per_input": 17446.303931938, + "estimated_time_ms_per_input": 2243.6672827746106 + }, + { + "pair": "grefcoco|instancevg|Qwen3.8", + "method": "confidence", + "nominal_validation_budget": 0.71, + "actual_test_call_fraction": 0.6838612710206166, + "test_metric": 0.7243286163967898, + "estimated_gflops_per_input": 22269.96488623913, + "estimated_time_ms_per_input": 2830.3683522642846 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "learned", + "nominal_validation_budget": 0.5, + "actual_test_call_fraction": 0.4876830892143808, + "test_metric": 0.29875800068449215, + "estimated_gflops_per_input": 47383.321370531434, + "estimated_time_ms_per_input": 4861.824738942901 + }, + { + "pair": "construction|yolo26x|Qwen3.8", + "method": "confidence", + "nominal_validation_budget": 0.93, + "actual_test_call_fraction": 0.9221038615179761, + "test_metric": 0.2597490004762105, + "estimated_gflops_per_input": 89464.5250591521, + "estimated_time_ms_per_input": 9103.651026133362 + } + ], + "protocol": { + "method": "original main-table logit contrast; original checkpoints and validation thresholds; seed 42", + "large_timing": "64 measured examples; batch size one; same full task prompt; two separate warmup examples", + "cost": "serial component estimates, not a timed integrated service; includes active distilled encoders and routing head only for learned", + "flops": "multiply-add=2; supported operators; mean prefill (including visual embedding) plus average generated length times one-token decode probe", + "scope_limits": "mean call cost extrapolated across policies; no policy-specific input/output-length costs; specialist-feature extraction overhead is not separately measured; confidence threshold overhead not separately timed" + } +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/fixed_budget_cost_performance.csv b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/fixed_budget_cost_performance.csv new file mode 100644 index 0000000000000000000000000000000000000000..f4e38dc91cc3bc9b3bfe25187fb66aea99cd8d67 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/fixed_budget_cost_performance.csv @@ -0,0 +1,29 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|Qwen3.8,learned,0.0,0.0,0.6695119530385957,11.144851456,63.11319342603383 +grefcoco|instancevg|Qwen3.8,learned,0.05,0.11998411933187761,0.7164449990074583,3917.1153094417755,551.619195333999 +grefcoco|instancevg|Qwen3.8,learned,0.1,0.16822164875365114,0.7290077418257097,5487.442832257611,748.0145414095009 +grefcoco|instancevg|Qwen3.8,learned,0.25,0.30176105266142983,0.748092901908516,9834.692864556126,1291.7098292810697 +grefcoco|instancevg|Qwen3.8,learned,0.5,0.487451436349715,0.7535093440716899,15879.6691252252,2047.733771728093 +grefcoco|instancevg|Qwen3.8,learned,0.75,0.6870657629810283,0.7512973938689278,22377.92628754188,2860.4479639829183 +grefcoco|instancevg|Qwen3.8,learned,1.0,1.0,0.7447182599325072,32565.206836791,4134.535350192891 +grefcoco|instancevg|Qwen3.8,confidence,0.0,0.0,0.6695119530385957,7.502680064,46.08042127620138 +grefcoco|instancevg|Qwen3.8,confidence,0.05,0.012534384482318578,0.6666194027734452,415.54780944942013,97.11319197894791 +grefcoco|instancevg|Qwen3.8,confidence,0.1,0.0345971698380739,0.6647761109378102,1133.7810914898203,186.94010511636148 +grefcoco|instancevg|Qwen3.8,confidence,0.25,0.12477667810452883,0.6594731021183677,4069.490393403024,554.0989531587461 +grefcoco|instancevg|Qwen3.8,confidence,0.5,0.2345234381646485,0.662762669086578,7642.1932230898465,1000.9243437008932 +grefcoco|instancevg|Qwen3.8,confidence,0.75,0.7478093185491875,0.7311629753566061,24351.733589325377,3090.727849854088 +grefcoco|instancevg|Qwen3.8,confidence,1.0,1.0,0.7447182599325072,32561.564665399,4117.502578043059 +construction|yolo26x|Qwen3.8,learned,0.0,0.0,0.18129624718318377,139.16228659200002,71.56936161300109 +construction|yolo26x|Qwen3.8,learned,0.05,0.045938748335552594,0.19531128485766117,4589.465326580834,522.8016087608005 +construction|yolo26x|Qwen3.8,learned,0.1,0.09920106524633822,0.20211074868345033,9749.236967147599,1045.9694315408578 +construction|yolo26x|Qwen3.8,learned,0.25,0.25432756324900135,0.2504846177578749,24777.071870298303,2569.695715387775 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,learned,0.75,0.7709720372836218,0.28730364578560735,74826.85678379593,7644.4235963543315 +construction|yolo26x|Qwen3.8,learned,1.0,1.0,0.27001009067886245,97013.87483823301,9894.045234308578 +construction|yolo26x|Qwen3.8,confidence,0.0,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.05,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.1,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.25,0.24800266311584554,0.18026802885144627,24161.165233228996,2482.3082690732604 +construction|yolo26x|Qwen3.8,confidence,0.5,0.5013315579227696,0.18015785955296285,48702.32909867467,4970.625226170908 +construction|yolo26x|Qwen3.8,confidence,0.75,0.7486684420772304,0.22103445955201576,72663.01865455658,7400.0858032057995 +construction|yolo26x|Qwen3.8,confidence,1.0,1.0,0.27001009067886245,97010.691083481,9868.783966949195 diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance.pdf b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance.pdf new file mode 100644 index 0000000000000000000000000000000000000000..975781f1e8a8b319f3165e1ae4b4197ef21b44d2 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance.pdf differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance.png b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance.png new file mode 100644 index 0000000000000000000000000000000000000000..a43c56fd38f9b57fd84824226597046a8071c9a8 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance_verified.png b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance_verified.png new file mode 100644 index 0000000000000000000000000000000000000000..f9c9f1a1bdd83aed00380b3d1f535b1fa798c489 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/flops_performance_verified.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/full_cost_performance_curves.csv b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/full_cost_performance_curves.csv new file mode 100644 index 0000000000000000000000000000000000000000..e25f1dec04edc9639248485b8cfa9becbfeb6fd6 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/full_cost_performance_curves.csv @@ -0,0 +1,405 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|Qwen3.8,learned,0.0,0.0,0.6695119530385957,11.144851456,63.11319342603383 +grefcoco|instancevg|Qwen3.8,learned,0.01,0.04701812097666109,0.6878314380512152,1541.775676164205,254.54381293995635 +grefcoco|instancevg|Qwen3.8,learned,0.02,0.06706746448118424,0.6980404389870403,2194.4632473757256,336.17315431290154 +grefcoco|instancevg|Qwen3.8,learned,0.03,0.08720188299350594,0.7065195814309616,2849.9203556645225,418.1488719575849 +grefcoco|instancevg|Qwen3.8,learned,0.04,0.1055780846779911,0.712389756969061,3448.14036435627,492.9661466530142 +grefcoco|instancevg|Qwen3.8,learned,0.05,0.11998411933187761,0.7164449990074583,3917.1153094417755,551.619195333999 +grefcoco|instancevg|Qwen3.8,learned,0.06,0.13243342880639764,0.7199047159912656,4322.390901749918,602.3055897650073 +grefcoco|instancevg|Qwen3.8,learned,0.07,0.14235884638289426,0.7222017412018262,4645.503560765523,642.7161548011189 +grefcoco|instancevg|Qwen3.8,learned,0.08,0.1511215721861441,0.7239032413577972,4930.765879725012,678.392910790143 +grefcoco|instancevg|Qwen3.8,learned,0.09,0.15957235629413266,0.7263136999120892,5205.873229401156,712.7996204494608 +grefcoco|instancevg|Qwen3.8,learned,0.1,0.16822164875365114,0.7290077418257097,5487.442832257611,748.0145414095009 +grefcoco|instancevg|Qwen3.8,learned,0.11,0.17695601622096815,0.7316450670674645,5771.781972191343,783.575838641279 +grefcoco|instancevg|Qwen3.8,learned,0.12,0.18855457561750277,0.7345943340044806,6149.362193726721,830.7984703549065 +grefcoco|instancevg|Qwen3.8,learned,0.13,0.19751580977228256,0.7366361341916456,6441.086765866523,867.2834376446528 +grefcoco|instancevg|Qwen3.8,learned,0.14,0.20664719394265946,0.7385361426991465,6738.350412160879,904.4611574778754 +grefcoco|instancevg|Qwen3.8,learned,0.15,0.2172532115815444,0.7402376428551173,7083.619367794696,947.6427326878917 +grefcoco|instancevg|Qwen3.8,learned,0.16,0.22717862915804102,0.7420242180188866,7406.7320268103,988.0532977240033 +grefcoco|instancevg|Qwen3.8,learned,0.17,0.2361398633128208,0.7436122848311261,7698.4565989501025,1024.5382650137497 +grefcoco|instancevg|Qwen3.8,learned,0.18,0.24362646399909255,0.7448033349403057,7942.175861750445,1055.0193769267023 +grefcoco|instancevg|Qwen3.8,learned,0.19,0.2507727646541701,0.7456540850182911,8174.816976241679,1084.1149837527025 +grefcoco|instancevg|Qwen3.8,learned,0.2,0.25942205711368854,0.7459943850494853,8456.386579098133,1119.3299047127425 +grefcoco|instancevg|Qwen3.8,learned,0.21,0.2690922496667896,0.746419760088478,8771.19062688191,1158.7013409336398 +grefcoco|instancevg|Qwen3.8,learned,0.22,0.2781385588293679,0.7467600601196722,9065.684736098987,1195.5326844951244 +grefcoco|instancevg|Qwen3.8,learned,0.23,0.2868445679607521,0.7473555851742619,9349.100697006961,1230.9785229696565 +grefcoco|instancevg|Qwen3.8,learned,0.24,0.29430281031109096,0.7480645435725831,9591.896780781542,1261.344176125363 +grefcoco|instancevg|Qwen3.8,learned,0.25,0.30176105266142983,0.748092901908516,9834.692864556126,1291.7098292810697 +grefcoco|instancevg|Qwen3.8,learned,0.26,0.30995661174602274,0.7486033519553073,10101.491603000439,1325.0774101251732 +grefcoco|instancevg|Qwen3.8,learned,0.27,0.3175850041119587,0.7494541020332927,10349.826760929574,1356.135815824356 +grefcoco|instancevg|Qwen3.8,learned,0.28,0.3262342965714772,0.7501630604316138,10631.396363786029,1391.350736784396 +grefcoco|instancevg|Qwen3.8,learned,0.29,0.3324731304767036,0.7505600771346738,10834.49574945298,1416.7516633785233 +grefcoco|instancevg|Qwen3.8,learned,0.3,0.3399313728270425,0.7509570938377336,11077.291833227562,1447.11731653423 +grefcoco|instancevg|Qwen3.8,learned,0.31,0.34824036525536683,0.7511839605251964,11347.783287774913,1480.9467324073175 +grefcoco|instancevg|Qwen3.8,learned,0.32,0.3557553242775714,0.751269035532995,11592.425729601013,1511.5433030775164 +grefcoco|instancevg|Qwen3.8,learned,0.33,0.3627598332529847,0.7511272438533307,11820.450948963453,1540.0616161172863 +grefcoco|instancevg|Qwen3.8,learned,0.34,0.37013300059552506,0.7512123188611293,12060.47749566076,1570.080893001255 +grefcoco|instancevg|Qwen3.8,learned,0.35,0.3783569180160508,0.751439185548592,12328.199413130831,1603.5639326026044 +grefcoco|instancevg|Qwen3.8,learned,0.36,0.3854748603351955,0.7517511272438533,12559.917348596306,1632.5440806713586 +grefcoco|instancevg|Qwen3.8,learned,0.37,0.39279131100587017,0.752176502282846,12798.097537242094,1662.3324400408353 +grefcoco|instancevg|Qwen3.8,learned,0.38,0.40013612001247767,0.7522332189547117,13037.200904913643,1692.2362581675577 +grefcoco|instancevg|Qwen3.8,learned,0.39,0.406573462269234,0.7525735189859059,13246.762543760904,1718.445396062407 +grefcoco|instancevg|Qwen3.8,learned,0.4,0.4153929047443496,0.7528003856733687,13533.871220771913,1754.3530695659233 +grefcoco|instancevg|Qwen3.8,learned,0.41,0.42151830530584466,0.75311232736863,13733.27789033583,1779.2921611310667 +grefcoco|instancevg|Qwen3.8,learned,0.42,0.42920341434364634,0.7535093440716899,13983.459406316484,1810.5814843447415 +grefcoco|instancevg|Qwen3.8,learned,0.43,0.4367750900377166,0.7537362107591526,14229.948206194102,1841.4089725294323 +grefcoco|instancevg|Qwen3.8,learned,0.44,0.4450273657941752,0.753679494087287,14498.593302689931,1875.0074708880277 +grefcoco|instancevg|Qwen3.8,learned,0.45,0.45143634971499874,0.7537929274310183,14707.231762511437,1901.1011500256313 +grefcoco|instancevg|Qwen3.8,learned,0.46,0.4578736919717551,0.7536227774154213,14916.7934013587,1927.3102879204807 +grefcoco|instancevg|Qwen3.8,learned,0.47,0.4659274593766838,0.7537929274310183,15178.976244674219,1960.100574978354 +grefcoco|instancevg|Qwen3.8,learned,0.48,0.4741797351331424,0.7538780024388169,15447.621341170048,1993.6990733369496 +grefcoco|instancevg|Qwen3.8,learned,0.49,0.4806454357258316,0.7537362107591526,15658.106159043073,2020.0236699890452 +grefcoco|instancevg|Qwen3.8,learned,0.5,0.487451436349715,0.7535093440716899,15879.6691252252,2047.733771728093 +grefcoco|instancevg|Qwen3.8,learned,0.51,0.49482460369225534,0.7537645690950855,16119.695671922505,2077.7530486120613 +grefcoco|instancevg|Qwen3.8,learned,0.52,0.501573887644273,0.7535660607435556,16339.412280053117,2105.232232836617 +grefcoco|instancevg|Qwen3.8,learned,0.53,0.5105351217990528,0.7532257607123614,16631.136852192918,2141.7172001263634 +grefcoco|instancevg|Qwen3.8,learned,0.54,0.519241130930437,0.7530556106967643,16914.552813100894,2177.163038600896 +grefcoco|instancevg|Qwen3.8,learned,0.55,0.5268128066245072,0.7528571023452344,17161.041612978508,2207.9905267855866 +grefcoco|instancevg|Qwen3.8,learned,0.56,0.5355755324277571,0.7528287440093016,17446.303931938,2243.6672827746106 +grefcoco|instancevg|Qwen3.8,learned,0.57,0.5419561580126478,0.753282477384227,17654.019212733747,2269.645503154968 +grefcoco|instancevg|Qwen3.8,learned,0.58,0.5495845503785838,0.7528287440093016,17902.35437066288,2300.703908854151 +grefcoco|instancevg|Qwen3.8,learned,0.59,0.5562771176587358,0.7527153106655702,18120.224620741974,2327.952175564215 +grefcoco|instancevg|Qwen3.8,learned,0.6,0.564047301704336,0.7526585939937045,18373.175673799906,2359.587875049628 +grefcoco|instancevg|Qwen3.8,learned,0.61,0.5725831608201231,0.7528287440093016,18651.052560553326,2394.3409609806836 +grefcoco|instancevg|Qwen3.8,learned,0.62,0.5818846950060971,0.7528571023452344,18953.855281002234,2432.2114333573827 +grefcoco|instancevg|Qwen3.8,learned,0.63,0.5903638374500184,0.7528571023452344,19229.885809704134,2466.7336017739462 +grefcoco|instancevg|Qwen3.8,learned,0.64,0.5973683464254318,0.7526869523296373,19457.911029066578,2495.2519148137167 +grefcoco|instancevg|Qwen3.8,learned,0.65,0.6042594220571137,0.7526018773218388,19682.24353232598,2523.3083928245023 +grefcoco|instancevg|Qwen3.8,learned,0.66,0.6122848311261095,0.752545160649973,19943.503196615744,2555.9832211251296 +grefcoco|instancevg|Qwen3.8,learned,0.67,0.6198848651561126,0.7522048606187789,20190.91517551912,2586.9261680670666 +grefcoco|instancevg|Qwen3.8,learned,0.68,0.6280520659047727,0.752176502282846,20456.790734937673,2620.178290153924 +grefcoco|instancevg|Qwen3.8,learned,0.69,0.6363894166690298,0.7522048606187789,20728.20536851078,2654.1231647842574 +grefcoco|instancevg|Qwen3.8,learned,0.7,0.6444999007458242,0.7521197856109804,20992.234569877815,2687.144369356623 +grefcoco|instancevg|Qwen3.8,learned,0.71,0.6527521765022829,0.7519212772594505,21260.87966637365,2720.742867715219 +grefcoco|instancevg|Qwen3.8,learned,0.72,0.6601253438448231,0.7518362022516518,21500.906213070954,2750.7621445991867 +grefcoco|instancevg|Qwen3.8,learned,0.73,0.6697955363979242,0.7518362022516518,21815.71026085473,2790.1335808200847 +grefcoco|instancevg|Qwen3.8,learned,0.74,0.6775373621075915,0.7518645605875848,22067.7381348869,2821.6538215482515 +grefcoco|instancevg|Qwen3.8,learned,0.75,0.6870657629810283,0.7512973938689278,22377.92628754188,2860.4479639829183 +grefcoco|instancevg|Qwen3.8,learned,0.76,0.6956299804327482,0.7514108272126592,22656.726353321057,2895.31650867122 +grefcoco|instancevg|Qwen3.8,learned,0.77,0.7044494229078637,0.7514959022204577,22943.835030332066,2931.224182174736 +grefcoco|instancevg|Qwen3.8,learned,0.78,0.7129852820236509,0.7510705271814649,23221.711917085486,2965.9772681057925 +grefcoco|instancevg|Qwen3.8,learned,0.79,0.7243853330686555,0.7506735104784051,23592.82988544055,3012.3916885186977 +grefcoco|instancevg|Qwen3.8,learned,0.8,0.7337152255905623,0.7507302271502708,23896.555784915217,3050.3776196526424 +grefcoco|instancevg|Qwen3.8,learned,0.81,0.744377959901313,0.7505033604628081,24243.671098600553,3093.7901123771508 +grefcoco|instancevg|Qwen3.8,learned,0.82,0.7522048606187789,0.7503048521112781,24498.46850971,3125.656729377056 +grefcoco|instancevg|Qwen3.8,learned,0.83,0.7607123613986331,0.7503332104472109,24775.42221743766,3160.2943565508654 +grefcoco|instancevg|Qwen3.8,learned,0.84,0.770779570654794,0.749964552080084,25103.15077158206,3201.2822153732072 +grefcoco|instancevg|Qwen3.8,learned,0.85,0.7827467884184556,0.749964552080084,25492.7323204523,3250.005810931033 +grefcoco|instancevg|Qwen3.8,learned,0.86,0.793466239401072,0.7498511187363526,25841.693992189157,3293.649221170034 +grefcoco|instancevg|Qwen3.8,learned,0.87,0.8059439072115249,0.7499361937441511,26247.89276352306,3344.451074358288 +grefcoco|instancevg|Qwen3.8,learned,0.88,0.8164932081785441,0.7497093270566884,26591.315361105357,3387.4017320538123 +grefcoco|instancevg|Qwen3.8,learned,0.89,0.8270425091455633,0.7494257436973598,26934.737958687656,3430.3523897493365 +grefcoco|instancevg|Qwen3.8,learned,0.9,0.8367410600345971,0.749397385361427,27250.46518549719,3469.83928472748 +grefcoco|instancevg|Qwen3.8,learned,0.91,0.8491903695091172,0.74883021864277,27655.740777805335,3520.5256791584884 +grefcoco|instancevg|Qwen3.8,learned,0.92,0.8618098289992343,0.7487167852990386,28066.55544426803,3571.904826132973 +grefcoco|instancevg|Qwen3.8,learned,0.93,0.8745710801690156,0.7486884269631058,28481.98600585952,3623.8612668936876 +grefcoco|instancevg|Qwen3.8,learned,0.94,0.8879845730652526,0.7486884269631058,28918.649685043467,3678.4732590710614 +grefcoco|instancevg|Qwen3.8,learned,0.95,0.9000085075007799,0.7484332019397102,29310.07759196523,3727.4277721433796 +grefcoco|instancevg|Qwen3.8,learned,0.96,0.9125996086549641,0.7477809602132547,29719.969079402166,3778.691460360618 +grefcoco|instancevg|Qwen3.8,learned,0.97,0.9269489266369849,0.7476391685335905,30187.09766643615,3837.11359152711 +grefcoco|instancevg|Qwen3.8,learned,0.98,0.9459773700479256,0.7472988685023962,30806.550792720354,3914.5864176391988 +grefcoco|instancevg|Qwen3.8,learned,0.99,0.9658282052009188,0.7472421518305306,31452.77611075156,3995.4075477114216 +grefcoco|instancevg|Qwen3.8,learned,1.0,1.0,0.7447182599325072,32565.206836791,4134.535350192891 +grefcoco|instancevg|Qwen3.8,confidence,0.0,0.0,0.6695119530385957,7.502680064,46.08042127620138 +grefcoco|instancevg|Qwen3.8,confidence,0.01,0.002467175226157729,0.6686895612965431,87.81925530502161,56.12533315660624 +grefcoco|instancevg|Qwen3.8,confidence,0.02,0.004849275444516916,0.6680373195700876,165.3662934687666,65.823868765273 +grefcoco|instancevg|Qwen3.8,confidence,0.03,0.007600034030003119,0.6675835861951621,254.9146589673769,77.02336821813819 +grefcoco|instancevg|Qwen3.8,confidence,0.04,0.009811984232765221,0.6671014944843037,326.9226229765687,86.02915128332876 +grefcoco|instancevg|Qwen3.8,confidence,0.05,0.012534384482318578,0.6666194027734452,415.54780944942013,97.11319197894791 +grefcoco|instancevg|Qwen3.8,confidence,0.06,0.01607917647392451,0.6662223860703854,530.9451876692788,111.54553663470202 +grefcoco|instancevg|Qwen3.8,confidence,0.07,0.019709043473328986,0.6658253693673255,649.112102966414,126.32425756219423 +grefcoco|instancevg|Qwen3.8,confidence,0.08,0.02353741882426339,0.6655985026798628,773.7412714438613,141.91118979040866 +grefcoco|instancevg|Qwen3.8,confidence,0.09,0.028868785979638715,0.6651164109690043,947.2989282865286,163.61743615266283 +grefcoco|instancevg|Qwen3.8,confidence,0.1,0.0345971698380739,0.6647761109378102,1133.7810914898203,186.94010511636148 +grefcoco|instancevg|Qwen3.8,confidence,0.11,0.03981510364971783,0.664435810906616,1303.6460322294522,208.18451644963153 +grefcoco|instancevg|Qwen3.8,confidence,0.12,0.045855429203414344,0.664067152539489,1500.2831647160913,232.77723174303654 +grefcoco|instancevg|Qwen3.8,confidence,0.13,0.05317187987408899,0.6637552108442276,1738.4633533618796,262.565591112513 +grefcoco|instancevg|Qwen3.8,confidence,0.14,0.05960922213084536,0.6636134191645634,1948.0249922091427,288.77472900736245 +grefcoco|instancevg|Qwen3.8,confidence,0.15,0.0675211978561098,0.6632164024615036,2205.591940395867,320.98772227900565 +grefcoco|instancevg|Qwen3.8,confidence,0.16,0.07551824858917279,0.6626208774069138,2465.928425659868,353.54709182238696 +grefcoco|instancevg|Qwen3.8,confidence,0.17,0.08212574086152624,0.6622522190397867,2681.0291386616846,380.44898226071257 +grefcoco|instancevg|Qwen3.8,confidence,0.18,0.08856308311828262,0.6619969940163911,2890.590777508948,406.6581201555621 +grefcoco|instancevg|Qwen3.8,confidence,0.19,0.09480191702350906,0.6615149023055327,3093.6901631758988,432.0590467496893 +grefcoco|instancevg|Qwen3.8,confidence,0.2,0.10007656750701868,0.6612596772821371,3265.4014619670484,453.5343755974514 +grefcoco|instancevg|Qwen3.8,confidence,0.21,0.10554972634205825,0.6608343022431443,3443.575013938511,475.81791574593575 +grefcoco|instancevg|Qwen3.8,confidence,0.22,0.11005870175538099,0.6605507188838159,3590.3604790341706,494.17585814805494 +grefcoco|instancevg|Qwen3.8,confidence,0.23,0.11527663556702493,0.6600969855088903,3760.2254197738025,515.420269481325 +grefcoco|instancevg|Qwen3.8,confidence,0.24,0.12029606102713893,0.6598417604854947,3923.6281073331224,535.8564695138729 +grefcoco|instancevg|Qwen3.8,confidence,0.25,0.12477667810452883,0.6594731021183677,4069.490393403024,554.0989531587461 +grefcoco|instancevg|Qwen3.8,confidence,0.26,0.12914386183818732,0.6590193687434421,4211.659963369889,571.879601774635 +grefcoco|instancevg|Qwen3.8,confidence,0.27,0.1322916371267334,0.658679068712248,4314.132835229124,584.6955238289447 +grefcoco|instancevg|Qwen3.8,confidence,0.28,0.13626180415733205,0.6582820520091881,4443.377898835366,600.8597498433894 +grefcoco|instancevg|Qwen3.8,confidence,0.29,0.14023197118793068,0.657743243626464,4572.622962441607,617.0239758578339 +grefcoco|instancevg|Qwen3.8,confidence,0.3,0.14326631313274538,0.657374585259337,4671.403118197806,629.3780628831595 +grefcoco|instancevg|Qwen3.8,confidence,0.31,0.14766185520233674,0.6569775685562771,4814.495867190431,647.2741702562946 +grefcoco|instancevg|Qwen3.8,confidence,0.32,0.15066783881121856,0.6567507018688143,4912.352843920871,659.512798524374 +grefcoco|instancevg|Qwen3.8,confidence,0.33,0.15452457249808582,0.6565238351813516,5037.905191424078,675.2151895098345 +grefcoco|instancevg|Qwen3.8,confidence,0.34,0.15806936448969175,0.6564104018376201,5153.302569643936,689.6475341655887 +grefcoco|instancevg|Qwen3.8,confidence,0.35,0.16201117318435754,0.6562686101579559,5281.6244542244185,705.6963014227872 +grefcoco|instancevg|Qwen3.8,confidence,0.36,0.1657544735274934,0.6558999517908289,5403.484085624589,720.9368573792635 +grefcoco|instancevg|Qwen3.8,confidence,0.37,0.17012165726115192,0.6558715934548961,5545.653655591455,738.7175059951526 +grefcoco|instancevg|Qwen3.8,confidence,0.38,0.17536794940872868,0.6555596517596347,5716.441775356846,760.0773760856687 +grefcoco|instancevg|Qwen3.8,confidence,0.39,0.17965005813458867,0.6553044267362391,5855.841808246436,777.5116484298197 +grefcoco|instancevg|Qwen3.8,confidence,0.4,0.18401724186824717,0.6552193517284406,5998.011378213301,795.2922970457087 +grefcoco|instancevg|Qwen3.8,confidence,0.41,0.18858293395343562,0.6552760684003063,6146.643201360479,813.88115696232 +grefcoco|instancevg|Qwen3.8,confidence,0.42,0.19238295096843716,0.6554745767518362,6270.349190812167,829.3526304332884 +grefcoco|instancevg|Qwen3.8,confidence,0.43,0.19689192638175992,0.655502935087769,6417.134655907828,847.7105728354077 +grefcoco|instancevg|Qwen3.8,confidence,0.44,0.20182627683407536,0.6555880100955676,6577.76780638987,867.8003965962173 +grefcoco|instancevg|Qwen3.8,confidence,0.45,0.20718600232538353,0.6559283101267618,6752.248642258297,889.6221017157176 +grefcoco|instancevg|Qwen3.8,confidence,0.46,0.21189348609023623,0.656070101806426,6905.49636053427,908.7882554185591 +grefcoco|instancevg|Qwen3.8,confidence,0.47,0.217139778237813,0.6567790602047472,7076.28448029966,930.148125509075 +grefcoco|instancevg|Qwen3.8,confidence,0.48,0.22198905368232993,0.6584238436888523,7234.148093704427,949.8915729981468 +grefcoco|instancevg|Qwen3.8,confidence,0.49,0.22763236253296656,0.6604372855400845,7417.860719830442,972.8678656901072 +grefcoco|instancevg|Qwen3.8,confidence,0.5,0.2345234381646485,0.662762669086578,7642.1932230898465,1000.9243437008932 +grefcoco|instancevg|Qwen3.8,confidence,0.51,0.24382497235062248,0.6654567110001985,7944.995943538756,1038.7948160775923 +grefcoco|instancevg|Qwen3.8,confidence,0.52,0.252644414825738,0.6678955278904234,8232.104620549764,1074.7024895811082 +grefcoco|instancevg|Qwen3.8,confidence,0.53,0.26401610753480986,0.6703627031165811,8602.299409879071,1121.0014512367675 +grefcoco|instancevg|Qwen3.8,confidence,0.54,0.2760116836344043,0.6731134617020673,8992.804137775072,1169.8405055518392 +grefcoco|instancevg|Qwen3.8,confidence,0.55,0.2893401015228426,0.6758358619516207,9426.69827988174,1224.1061214574747 +grefcoco|instancevg|Qwen3.8,confidence,0.56,0.3036043444970649,0.679295578935428,9891.057329838453,1282.1818763522294 +grefcoco|instancevg|Qwen3.8,confidence,0.57,0.3175850041119587,0.6822448458724442,10346.184589537575,1339.1030436745236 +grefcoco|instancevg|Qwen3.8,confidence,0.58,0.33462836400760004,0.6853926211609903,10901.015184018655,1408.4937567793893 +grefcoco|instancevg|Qwen3.8,confidence,0.59,0.35637920766809406,0.6889090548166633,11609.09349677571,1497.0506235870967 +grefcoco|instancevg|Qwen3.8,confidence,0.6,0.37968975980489467,0.6922553384567394,12367.9466559495,1591.9577220433357 +grefcoco|instancevg|Qwen3.8,confidence,0.61,0.4046450954258004,0.6949493803703599,13180.344198617304,1693.5614284198446 +grefcoco|instancevg|Qwen3.8,confidence,0.62,0.43141536454640844,0.6983807390182344,14051.825198933677,1802.5544952600997 +grefcoco|instancevg|Qwen3.8,confidence,0.63,0.4610214672603012,0.7015568726427134,15015.624101825935,1923.093437824958 +grefcoco|instancevg|Qwen3.8,confidence,0.64,0.4916484700677764,0.7053285313217821,16012.657449645512,2047.7888956506733 +grefcoco|instancevg|Qwen3.8,confidence,0.65,0.5209426310864078,0.7086180982899923,16966.301383254424,2167.0577918858253 +grefcoco|instancevg|Qwen3.8,confidence,0.66,0.5529308340186598,0.71148229021921,18007.64732431043,2297.2952700593505 +grefcoco|instancevg|Qwen3.8,confidence,0.67,0.5888041289737118,0.7155091739216742,19175.4687918954,2443.350597975582 +grefcoco|instancevg|Qwen3.8,confidence,0.68,0.6175027649377535,0.7182882908430933,20109.725965963375,2560.1948603085675 +grefcoco|instancevg|Qwen3.8,confidence,0.69,0.6421461588633979,0.7211241244363781,20911.968539347832,2660.52852035537 +grefcoco|instancevg|Qwen3.8,confidence,0.7,0.665910444375124,0.7233077163032073,21685.59256293376,2757.282958927545 +grefcoco|instancevg|Qwen3.8,confidence,0.71,0.6838612710206166,0.7243286163967898,22269.96488623913,2830.3683522642846 +grefcoco|instancevg|Qwen3.8,confidence,0.72,0.7029464311034228,0.7260017582168279,22891.264370574845,2908.072095890864 +grefcoco|instancevg|Qwen3.8,confidence,0.73,0.7180614241556306,0.7278450500524629,23383.318791304326,2969.611613503 +grefcoco|instancevg|Qwen3.8,confidence,0.74,0.7352465757309361,0.7297450585599637,23942.7652809142,3039.579620394096 +grefcoco|instancevg|Qwen3.8,confidence,0.75,0.7478093185491875,0.7311629753566061,24351.733589325377,3090.727849854088 +grefcoco|instancevg|Qwen3.8,confidence,0.76,0.759804894648782,0.7328928338485098,24742.238317221378,3139.5669041691604 +grefcoco|instancevg|Qwen3.8,confidence,0.77,0.7729064458497575,0.7342540339732865,25168.74702712198,3192.9088500168273 +grefcoco|instancevg|Qwen3.8,confidence,0.78,0.7855259053398747,0.735274934066869,25579.561693584674,3244.287996991312 +grefcoco|instancevg|Qwen3.8,confidence,0.79,0.7964722230099538,0.7362958341604514,25935.908797527598,3288.855077288281 +grefcoco|instancevg|Qwen3.8,confidence,0.8,0.8082125740861527,0.7377704676289595,26318.104914191772,3336.6550027881385 +grefcoco|instancevg|Qwen3.8,confidence,0.81,0.8197544168108215,0.738422709355415,26693.838777675628,3383.6467169872735 +grefcoco|instancevg|Qwen3.8,confidence,0.82,0.8333380597226555,0.7389048010662734,27136.04153101413,3438.9514617081236 +grefcoco|instancevg|Qwen3.8,confidence,0.83,0.8426112355726966,0.7398689844879902,27437.921072437275,3476.7064753275763 +grefcoco|instancevg|Qwen3.8,confidence,0.84,0.8529903865241187,0.7403227178629158,27775.804595865022,3518.964380479624 +grefcoco|instancevg|Qwen3.8,confidence,0.85,0.8635396874911381,0.7406913762300428,28119.227193447325,3561.9150381751488 +grefcoco|instancevg|Qwen3.8,confidence,0.86,0.8710262881774098,0.7413152596205654,28362.946456247664,3592.3961500881014 +grefcoco|instancevg|Qwen3.8,confidence,0.87,0.8809517057539064,0.7422227263704165,28686.05911526327,3632.8067151242126 +grefcoco|instancevg|Qwen3.8,confidence,0.88,0.8913024983693957,0.7430167597765364,29023.019459665255,3674.9491615190145 +grefcoco|instancevg|Qwen3.8,confidence,0.89,0.900915974250631,0.743442134815529,29335.977149397513,3714.0896802254197 +grefcoco|instancevg|Qwen3.8,confidence,0.9,0.910954825170859,0.7443212432294473,29662.782524516155,3754.9620802905156 +grefcoco|instancevg|Qwen3.8,confidence,0.91,0.921901142840938,0.7448033349403057,30019.129628459075,3799.529160587484 +grefcoco|instancevg|Qwen3.8,confidence,0.92,0.9303235686129938,0.7450018432918356,30293.31379910946,3833.820411489556 +grefcoco|instancevg|Qwen3.8,confidence,0.93,0.9393698777755721,0.7448884099481042,30587.80790832654,3870.6517550510403 +grefcoco|instancevg|Qwen3.8,confidence,0.94,0.9490117119927403,0.7448884099481042,30901.688777084553,3909.907732514692 +grefcoco|instancevg|Qwen3.8,confidence,0.95,0.9587953378895726,0.7449734849559028,31220.185540971364,3949.7410037645727 +grefcoco|instancevg|Qwen3.8,confidence,0.96,0.9673595553412926,0.7452570683152313,31498.985606750543,3984.609548452875 +grefcoco|instancevg|Qwen3.8,confidence,0.97,0.9746192893401016,0.745313784987097,31735.319437344813,4014.1669903078596 +grefcoco|instancevg|Qwen3.8,confidence,0.98,0.9830984317840229,0.7450585599637013,32011.349966046713,4048.689158724423 +grefcoco|instancevg|Qwen3.8,confidence,0.99,0.9908118991577575,0.7448884099481042,32262.45466105313,4080.0939406953444 +grefcoco|instancevg|Qwen3.8,confidence,1.0,1.0,0.7447182599325072,32561.564665399,4117.502578043059 +construction|yolo26x|Qwen3.8,learned,0.0,0.0,0.18129624718318377,139.16228659200002,71.56936161300109 +construction|yolo26x|Qwen3.8,learned,0.01,0.013648468708388815,0.18693471712069915,1461.3537694872334,205.6311162003908 +construction|yolo26x|Qwen3.8,learned,0.02,0.019640479360852198,0.18781313491575913,2041.8280790509946,264.48749626314725 +construction|yolo26x|Qwen3.8,learned,0.03,0.031291611185086554,0.1913128749335621,3170.5281254249744,378.9304574962848 +construction|yolo26x|Qwen3.8,learned,0.04,0.04061251664447404,0.19266008228682668,4073.4881625241583,470.48482648279486 +construction|yolo26x|Qwen3.8,learned,0.05,0.045938748335552594,0.19531128485766117,4589.465326580834,522.8016087608005 +construction|yolo26x|Qwen3.8,learned,0.06,0.0559254327563249,0.19800731707634547,5556.922509187103,620.8955755320613 +construction|yolo26x|Qwen3.8,learned,0.07,0.07456724367509987,0.19763843353437874,7362.842583385471,804.0043135050814 +construction|yolo26x|Qwen3.8,learned,0.08,0.08688415446071904,0.1974112776278713,8556.039775266534,924.9868725229696 +construction|yolo26x|Qwen3.8,learned,0.09,0.09587217043941411,0.20030220967092682,9426.751239612175,1013.2714426171043 +construction|yolo26x|Qwen3.8,learned,0.1,0.09920106524633822,0.20211074868345033,9749.236967147599,1045.9694315408578 +construction|yolo26x|Qwen3.8,learned,0.11,0.11284953395472704,0.20834032699131655,11071.428450042833,1180.0311861282478 +construction|yolo26x|Qwen3.8,learned,0.12,0.12350199733688415,0.21121674052765757,12103.382778156185,1284.664750684259 +construction|yolo26x|Qwen3.8,learned,0.13,0.13315579227696406,0.21276215593410724,13038.591388008912,1379.4889185631446 +construction|yolo26x|Qwen3.8,learned,0.14,0.14347536617842876,0.21511521078428622,14038.297143368722,1480.8526842267806 +construction|yolo26x|Qwen3.8,learned,0.15,0.15246338215712382,0.21738179676765032,14909.008607714362,1569.1372543209152 +construction|yolo26x|Qwen3.8,learned,0.16,0.1604527296937417,0.22260454951009195,15682.974353799378,1647.612427737924 +construction|yolo26x|Qwen3.8,learned,0.17,0.1780958721704394,0.22627631882842034,17392.148709737117,1820.9117690338178 +construction|yolo26x|Qwen3.8,learned,0.18,0.18442077230359522,0.2270967395967396,18004.871592054424,1883.0379479889498 +construction|yolo26x|Qwen3.8,learned,0.19,0.2000665778961385,0.23397114982462924,19520.55451147091,2036.7184959305916 +construction|yolo26x|Qwen3.8,learned,0.2,0.20439414114513982,0.23512276426997616,19939.78595726696,2079.2258815314713 +construction|yolo26x|Qwen3.8,learned,0.21,0.21937416777629826,0.24202469545559713,21390.97173117636,2226.3668316883623 +construction|yolo26x|Qwen3.8,learned,0.22,0.22769640479360853,0.24353835097564847,22197.18605001492,2308.1118039977464 +construction|yolo26x|Qwen3.8,learned,0.23,0.23601864181091878,0.2454657390209296,23003.400368853476,2389.8567763071305 +construction|yolo26x|Qwen3.8,learned,0.24,0.24533954727030627,0.2483414219175777,23906.36040595266,2481.4111452936404 +construction|yolo26x|Qwen3.8,learned,0.25,0.25432756324900135,0.2504846177578749,24777.071870298303,2569.695715387775 +construction|yolo26x|Qwen3.8,learned,0.26,0.2636484687083888,0.252142216376585,25680.031907397482,2661.250084374285 +construction|yolo26x|Qwen3.8,learned,0.27,0.275965379494008,0.25276746625574176,26873.22909927855,2782.2326433921735 +construction|yolo26x|Qwen3.8,learned,0.28,0.29427430093209056,0.25702642690899347,28646.900600723373,2962.071582472818 +construction|yolo26x|Qwen3.8,learned,0.29,0.31091877496671105,0.26183035419940204,30259.32923840049,3125.561527091586 +construction|yolo26x|Qwen3.8,learned,0.3,0.322237017310253,0.2672887365132105,31355.780712020925,3236.734689432348 +construction|yolo26x|Qwen3.8,learned,0.31,0.33388814913448733,0.27039536303602457,32484.480758394904,3351.1776506654855 +construction|yolo26x|Qwen3.8,learned,0.32,0.34354194407456723,0.2713201570387918,33419.68936824763,3446.001818544371 +construction|yolo26x|Qwen3.8,learned,0.33,0.3492010652463382,0.2722481828097158,33967.91510505785,3501.588399714752 +construction|yolo26x|Qwen3.8,learned,0.34,0.3545272969374168,0.274417471554269,34483.89226911453,3553.905181992758 +construction|yolo26x|Qwen3.8,learned,0.35,0.3648468708388815,0.2782223498666452,35483.59802447434,3655.268947656394 +construction|yolo26x|Qwen3.8,learned,0.36,0.37217043941411454,0.2795152621397537,36193.06662505227,3727.204523288652 +construction|yolo26x|Qwen3.8,learned,0.37,0.37683089214380827,0.28090947933574756,36644.54664360186,3772.981707781907 +construction|yolo26x|Qwen3.8,learned,0.38,0.3854860186418109,0.28295009794869297,37483.00953519396,3857.9964789836663 +construction|yolo26x|Qwen3.8,learned,0.39,0.39980026631158455,0.28644294895435674,38869.69816359627,3998.5978313558066 +construction|yolo26x|Qwen3.8,learned,0.4,0.4054593874833555,0.2862200047733393,39417.923900406495,4054.1844125261878 +construction|yolo26x|Qwen3.8,learned,0.41,0.4104527296937417,0.28760386086589435,39901.65249170963,4103.231395911818 +construction|yolo26x|Qwen3.8,learned,0.42,0.4181091877496671,0.2930744270093899,40643.3696650411,4178.436770436451 +construction|yolo26x|Qwen3.8,learned,0.43,0.42809587217043943,0.2961685540536353,41610.82684764737,4276.530737207712 +construction|yolo26x|Qwen3.8,learned,0.44,0.4370838881491345,0.2977058723546182,42481.53831199301,4364.8153073018475 +construction|yolo26x|Qwen3.8,learned,0.45,0.44673768308921435,0.2999402430635043,43416.74692184573,4459.639475180732 +construction|yolo26x|Qwen3.8,learned,0.46,0.45206391478029295,0.2993046120720782,43932.72408590242,4511.956257458738 +construction|yolo26x|Qwen3.8,learned,0.47,0.45872170439414117,0.2998310238962732,44577.69554097326,4577.352235306245 +construction|yolo26x|Qwen3.8,learned,0.48,0.4673768308921438,0.30393211265289466,45416.15843256536,4662.367006508005 +construction|yolo26x|Qwen3.8,learned,0.49,0.47503328894806923,0.30350001249166053,46157.87560589683,4737.572381032637 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,learned,0.51,0.5046604527296937,0.29989697064322696,49027.99858096209,5028.5844824540445 +construction|yolo26x|Qwen3.8,learned,0.52,0.5206391478029294,0.297424793956044,50575.93007313212,5185.534829288062 +construction|yolo26x|Qwen3.8,learned,0.53,0.5249667110519307,0.297424793956044,50995.161518928166,5228.042214888941 +construction|yolo26x|Qwen3.8,learned,0.54,0.5436085219707057,0.2978194775414123,52801.08159312654,5411.150952861962 +construction|yolo26x|Qwen3.8,learned,0.55,0.5529294274300932,0.29630990060928936,53704.04163022572,5502.705321848472 +construction|yolo26x|Qwen3.8,learned,0.56,0.5612516644474035,0.29578344205458756,54510.25594906428,5584.450294157856 +construction|yolo26x|Qwen3.8,learned,0.57,0.5692410119840213,0.2950949302773625,55284.221695149296,5662.925467574864 +construction|yolo26x|Qwen3.8,learned,0.58,0.5902130492676432,0.2947373376338526,57315.88177862246,5868.922797794512 +construction|yolo26x|Qwen3.8,learned,0.59,0.6021970705725699,0.29722091277674884,58476.830397749975,5986.635557920024 +construction|yolo26x|Qwen3.8,learned,0.6,0.6208388814913449,0.29665021956289706,60282.750471948355,6169.744295893045 +construction|yolo26x|Qwen3.8,learned,0.61,0.6288282290279628,0.295988506182091,61056.71621803337,6248.219469310054 +construction|yolo26x|Qwen3.8,learned,0.62,0.6421438082556591,0.2939669175161438,62346.65912817505,6379.011425005067 +construction|yolo26x|Qwen3.8,learned,0.63,0.6491344873501997,0.29277362668991463,63023.87915599944,6447.67720174495 +construction|yolo26x|Qwen3.8,learned,0.64,0.6571238348868176,0.2930821542583563,63797.84490208446,6526.152375161959 +construction|yolo26x|Qwen3.8,learned,0.65,0.6677762982689747,0.2892305220657417,64829.79923019781,6630.78593971797 +construction|yolo26x|Qwen3.8,learned,0.66,0.6744340878828229,0.2915915088317972,65474.77068526865,6696.181917565477 +construction|yolo26x|Qwen3.8,learned,0.67,0.6854194407456724,0.2936540392680743,66538.97358613556,6804.085281013864 +construction|yolo26x|Qwen3.8,learned,0.68,0.7047270306258322,0.29632332439424064,68409.390805841,6993.733616771635 +construction|yolo26x|Qwen3.8,learned,0.69,0.7157123834886817,0.2937246709263134,69473.5937067079,7101.6369802200215 +construction|yolo26x|Qwen3.8,learned,0.7,0.7253661784287616,0.2952254059747369,70408.80231656063,7196.461148098907 +construction|yolo26x|Qwen3.8,learned,0.71,0.7283621837549934,0.2945683782087344,70699.03947134252,7225.889338130286 +construction|yolo26x|Qwen3.8,learned,0.72,0.741011984021305,0.29252963134197957,71924.48523597712,7350.1416960405495 +construction|yolo26x|Qwen3.8,learned,0.73,0.7526631158455392,0.2900395106945919,73053.1852823511,7464.584657273686 +construction|yolo26x|Qwen3.8,learned,0.74,0.7616511318242344,0.2887744568626224,73923.89674669674,7552.869227367822 +construction|yolo26x|Qwen3.8,learned,0.75,0.7709720372836218,0.28730364578560735,74826.85678379593,7644.4235963543315 +construction|yolo26x|Qwen3.8,learned,0.76,0.7749667110519307,0.2858070539975931,75213.83965683842,7683.661183062835 +construction|yolo26x|Qwen3.8,learned,0.77,0.7792942743009321,0.28488735368132817,75633.07110263448,7726.168568663716 +construction|yolo26x|Qwen3.8,learned,0.78,0.7846205059920106,0.2830658409372236,76149.04826669116,7778.485350941721 +construction|yolo26x|Qwen3.8,learned,0.79,0.7972703062583223,0.2809995077380739,77374.49403132577,7902.737708851985 +construction|yolo26x|Qwen3.8,learned,0.8,0.8129161118508655,0.28013559922601305,78890.17695074226,8056.418256793627 +construction|yolo26x|Qwen3.8,learned,0.81,0.8199067909454061,0.27814841049220523,79567.39697856663,8125.084033533509 +construction|yolo26x|Qwen3.8,learned,0.82,0.831225033288948,0.2772348763322643,80663.84845218707,8236.25719587427 +construction|yolo26x|Qwen3.8,learned,0.83,0.84254327563249,0.27700595116508253,81760.29992580751,8347.430358215033 +construction|yolo26x|Qwen3.8,learned,0.84,0.857190412782956,0.2763733409861609,83179.23712696337,8491.301509479548 +construction|yolo26x|Qwen3.8,learned,0.85,0.8618508655126498,0.2761345157638265,83630.71714551296,8537.078693972804 +construction|yolo26x|Qwen3.8,learned,0.86,0.87017310252996,0.2761345157638265,84436.93146435152,8618.823666282187 +construction|yolo26x|Qwen3.8,learned,0.87,0.8848202396804261,0.27546832609774896,85855.86866550738,8762.694817546704 +construction|yolo26x|Qwen3.8,learned,0.88,0.8984687083888149,0.27423981143437465,87178.06014840261,8896.756572134092 +construction|yolo26x|Qwen3.8,learned,0.89,0.9087882822902796,0.2736575311942959,88177.76590376243,8998.12033779773 +construction|yolo26x|Qwen3.8,learned,0.9,0.9161118508655126,0.272975172975173,88887.23450434035,9070.055913429987 +construction|yolo26x|Qwen3.8,learned,0.91,0.9217709720372836,0.2727190231806067,89435.46024115058,9125.64249460037 +construction|yolo26x|Qwen3.8,learned,0.92,0.9264314247669774,0.27264729817760425,89886.94025970016,9171.419679093624 +construction|yolo26x|Qwen3.8,learned,0.93,0.933422103861518,0.2741441928343702,90564.16028752456,9240.085455833507 +construction|yolo26x|Qwen3.8,learned,0.94,0.9417443408788282,0.2733716828250215,91370.37460636311,9321.83042814289 +construction|yolo26x|Qwen3.8,learned,0.95,0.9530625832223701,0.2727312493845365,92466.82607998354,9433.003590483651 +construction|yolo26x|Qwen3.8,learned,0.96,0.9613848202396804,0.27223682114986464,93273.04039882211,9514.748562793036 +construction|yolo26x|Qwen3.8,learned,0.97,0.9673768308921438,0.27453093162048386,93853.51470838586,9573.604942855793 +construction|yolo26x|Qwen3.8,learned,0.98,0.9760319573901465,0.2737331615154065,94691.97759997797,9658.619714057553 +construction|yolo26x|Qwen3.8,learned,0.99,0.9890146471371505,0.2725383412883413,95949.67193736612,9786.141870860192 +construction|yolo26x|Qwen3.8,learned,1.0,1.0,0.27001009067886245,97013.87483823301,9894.045234308578 +construction|yolo26x|Qwen3.8,confidence,0.0,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.01,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.02,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.03,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.04,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.05,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.06,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.07,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.08,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.09,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.1,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.11,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.12,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.13,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.14,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.15,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.16,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.17,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.18,0.0,0.18129624718318377,135.97853184000002,46.30809425361804 +construction|yolo26x|Qwen3.8,confidence,0.19,0.17942743009320905,0.1784481207469488,17517.959245999285,1808.7296972439362 +construction|yolo26x|Qwen3.8,confidence,0.2,0.19340878828229027,0.1780928651788194,18872.39930164806,1946.0612507237013 +construction|yolo26x|Qwen3.8,confidence,0.21,0.20306258322237017,0.17829989216687975,19807.60791150079,2040.8854186025867 +construction|yolo26x|Qwen3.8,confidence,0.22,0.20838881491344874,0.1779854683890517,20323.585075557465,2093.2022008805925 +construction|yolo26x|Qwen3.8,confidence,0.23,0.21537949400798934,0.17917657728013692,21000.805103381852,2161.867977620475 +construction|yolo26x|Qwen3.8,confidence,0.24,0.23135818908122505,0.17803687432776102,22548.736595551884,2318.8183244544925 +construction|yolo26x|Qwen3.8,confidence,0.25,0.24800266311584554,0.18026802885144627,24161.165233228996,2482.3082690732604 +construction|yolo26x|Qwen3.8,confidence,0.26,0.2589880159786951,0.18119827871024874,25225.368134095894,2590.211632521647 +construction|yolo26x|Qwen3.8,confidence,0.27,0.27430093209054596,0.17963665085316138,26708.802480758837,2740.6223815709136 +construction|yolo26x|Qwen3.8,confidence,0.28,0.281624500665779,0.17924095839307458,27418.271081336767,2812.5579572031716 +construction|yolo26x|Qwen3.8,confidence,0.29,0.28761651131824234,0.1789157174618718,27998.745390900527,2871.4143372659278 +construction|yolo26x|Qwen3.8,confidence,0.3,0.2969374167776298,0.1787206845800119,28901.70542799971,2962.9687062524376 +construction|yolo26x|Qwen3.8,confidence,0.31,0.30459387483355527,0.17813242189706796,29643.422601331185,3038.174080777071 +construction|yolo26x|Qwen3.8,confidence,0.32,0.3129161118508655,0.17772000842137498,30449.636920169738,3119.919053086455 +construction|yolo26x|Qwen3.8,confidence,0.33,0.31990679094540614,0.17791259734473341,31126.85694799413,3188.5848298263377 +construction|yolo26x|Qwen3.8,confidence,0.34,0.3322237017310253,0.1789653934599337,32320.05413987519,3309.5673888442257 +construction|yolo26x|Qwen3.8,confidence,0.35,0.34054593874833555,0.18042682200794885,33126.268458713756,3391.31236115361 +construction|yolo26x|Qwen3.8,confidence,0.36,0.3488681757656458,0.17962877325912951,33932.48277755231,3473.057333462994 +construction|yolo26x|Qwen3.8,confidence,0.37,0.3581890812250333,0.18158304685510065,34835.44281465149,3564.611702449504 +construction|yolo26x|Qwen3.8,confidence,0.38,0.37017310252996005,0.18069239723097547,35996.39143377901,3682.3244625750167 +construction|yolo26x|Qwen3.8,confidence,0.39,0.37683089214380827,0.1812845803851709,36641.362888849864,3747.720440422524 +construction|yolo26x|Qwen3.8,confidence,0.4,0.38348868175765644,0.1824781664689562,37286.3343439207,3813.116418270031 +construction|yolo26x|Qwen3.8,confidence,0.41,0.4021304926764314,0.18220594501399923,39092.254418119075,3996.225156243051 +construction|yolo26x|Qwen3.8,confidence,0.42,0.41311584553928093,0.18162498930772586,40156.457318985966,4104.128519691438 +construction|yolo26x|Qwen3.8,confidence,0.43,0.42676431424766975,0.18066434967152634,41478.6488018812,4238.190274278827 +construction|yolo26x|Qwen3.8,confidence,0.44,0.4324234354194407,0.1803913789289444,42026.874538691416,4293.776855449209 +construction|yolo26x|Qwen3.8,confidence,0.45,0.44107856191744343,0.17959468902865128,42865.33743028352,4378.791626650968 +construction|yolo26x|Qwen3.8,confidence,0.46,0.45772303595206393,0.17863383514615866,44477.766067960634,4542.281571269737 +construction|yolo26x|Qwen3.8,confidence,0.47,0.4717043941411451,0.17957308538379646,45832.20612360941,4679.6131247495005 +construction|yolo26x|Qwen3.8,confidence,0.48,0.4826897470039947,0.17910245145856155,46896.40902447631,4787.516488197888 +construction|yolo26x|Qwen3.8,confidence,0.49,0.4896804260985353,0.17871833907402393,47573.62905230069,4856.182264937771 +construction|yolo26x|Qwen3.8,confidence,0.5,0.5013315579227696,0.18015785955296285,48702.32909867467,4970.625226170908 +construction|yolo26x|Qwen3.8,confidence,0.51,0.5116511318242344,0.1810878167493375,49702.034854034486,5071.988991834544 +construction|yolo26x|Qwen3.8,confidence,0.52,0.5183089214380826,0.18177962418520882,50347.00630910533,5137.384969682052 +construction|yolo26x|Qwen3.8,confidence,0.53,0.5252996005326231,0.18246323539519374,51024.226336929714,5206.050746421934 +construction|yolo26x|Qwen3.8,confidence,0.54,0.5396138482023968,0.1839876166906015,52410.914965332035,5346.652098794074 +construction|yolo26x|Qwen3.8,confidence,0.55,0.5452729693741678,0.1846674213907964,52959.14070214226,5402.2386799644555 +construction|yolo26x|Qwen3.8,confidence,0.56,0.559254327563249,0.18676294233791663,54313.58075779103,5539.57023344422 +construction|yolo26x|Qwen3.8,confidence,0.57,0.572237017310253,0.18938663221175195,55571.275095179175,5667.092390246859 +construction|yolo26x|Qwen3.8,confidence,0.58,0.5862183754993342,0.1909571002676625,56925.71515082795,5804.423943726624 +construction|yolo26x|Qwen3.8,confidence,0.59,0.5998668442077231,0.1934300101790686,58247.90663372319,5938.485698314014 +construction|yolo26x|Qwen3.8,confidence,0.6,0.6191744340878829,0.1973701682130997,60118.32385342864,6128.134034071785 +construction|yolo26x|Qwen3.8,confidence,0.61,0.6264980026631158,0.1986727924998457,60827.792454006565,6200.069609704043 +construction|yolo26x|Qwen3.8,confidence,0.62,0.6384820239680427,0.20186289191698192,61988.74107313409,6317.782369829556 +construction|yolo26x|Qwen3.8,confidence,0.63,0.6461384820239681,0.20246031058520178,62730.45824646557,6392.987744354189 +construction|yolo26x|Qwen3.8,confidence,0.64,0.653462050599201,0.20349944137622245,63439.92684704349,6464.923319986447 +construction|yolo26x|Qwen3.8,confidence,0.65,0.6597869507323568,0.20442990090722246,64052.64972936079,6527.049498941578 +construction|yolo26x|Qwen3.8,confidence,0.66,0.6661118508655126,0.2044993942500819,64665.372611678096,6589.17567789671 +construction|yolo26x|Qwen3.8,confidence,0.67,0.6734354194407457,0.2064379542733177,65374.84121225603,6661.1112535289685 +construction|yolo26x|Qwen3.8,confidence,0.68,0.6820905459387483,0.2065831206227621,66213.30410384812,6746.126024730727 +construction|yolo26x|Qwen3.8,confidence,0.69,0.6927430093209055,0.20802802394105854,67245.25843196148,6850.759589286739 +construction|yolo26x|Qwen3.8,confidence,0.7,0.7090545938748336,0.21169680596278434,68825.43849688505,7010.979735013132 +construction|yolo26x|Qwen3.8,confidence,0.71,0.7170439414114514,0.2139280527103987,69599.40424297006,7089.454908430141 +construction|yolo26x|Qwen3.8,confidence,0.72,0.7243675099866844,0.2146900231025185,70308.87284354799,7161.390484062398 +construction|yolo26x|Qwen3.8,confidence,0.73,0.7330226364846871,0.21653561577692484,71147.33573514009,7246.405255264158 +construction|yolo26x|Qwen3.8,confidence,0.74,0.7363515312916112,0.21721689648133474,71469.82146267552,7279.1032441879115 +construction|yolo26x|Qwen3.8,confidence,0.75,0.7486684420772304,0.22103445955201576,72663.01865455658,7400.0858032057995 +construction|yolo26x|Qwen3.8,confidence,0.76,0.757323568575233,0.22340134248367294,73501.48154614867,7485.100574407558 +construction|yolo26x|Qwen3.8,confidence,0.77,0.7639813581890812,0.2269322568854726,74146.45300121952,7550.496552255066 +construction|yolo26x|Qwen3.8,confidence,0.78,0.7733022636484687,0.22757452798038888,75049.4130383187,7642.050921241575 +construction|yolo26x|Qwen3.8,confidence,0.79,0.784953395472703,0.227821050912831,76178.11308469268,7756.493882474713 +construction|yolo26x|Qwen3.8,confidence,0.8,0.7939414114513982,0.22991865671924006,77048.82454903833,7844.7784525688485 +construction|yolo26x|Qwen3.8,confidence,0.81,0.8039280958721704,0.23486191981046528,78016.28173164459,7942.8724193401085 +construction|yolo26x|Qwen3.8,confidence,0.82,0.8122503328894807,0.23803916047544685,78822.49605048315,8024.617391649493 +construction|yolo26x|Qwen3.8,confidence,0.83,0.8215712383488681,0.24022947987632814,79725.45608758234,8116.171760636003 +construction|yolo26x|Qwen3.8,confidence,0.84,0.8388814913448736,0.24528627405740036,81402.38187076653,8286.201303039523 +construction|yolo26x|Qwen3.8,confidence,0.85,0.8535286284953395,0.249537136326177,82821.3190719224,8430.072454304036 +construction|yolo26x|Qwen3.8,confidence,0.86,0.8638482023968043,0.25211074301800224,83821.0248272822,8531.436219967674 +construction|yolo26x|Qwen3.8,confidence,0.87,0.8745006657789614,0.2540042317315216,84852.97915539556,8636.069784523685 +construction|yolo26x|Qwen3.8,confidence,0.88,0.8764980026631158,0.2540996517958595,85046.4705919168,8655.688577877936 +construction|yolo26x|Qwen3.8,confidence,0.89,0.8841544607190412,0.25436194678023316,85788.18776524828,8730.89395240257 +construction|yolo26x|Qwen3.8,confidence,0.9,0.8891478029294274,0.2534339447803836,86271.91635655142,8779.9409357882 +construction|yolo26x|Qwen3.8,confidence,0.91,0.8981358189081226,0.2563240880787248,87142.62782089706,8868.225505882336 +construction|yolo26x|Qwen3.8,confidence,0.92,0.9054593874833555,0.25551845731687656,87852.09642147498,8940.161081514594 +construction|yolo26x|Qwen3.8,confidence,0.93,0.9221038615179761,0.2597490004762105,89464.5250591521,9103.651026133362 +construction|yolo26x|Qwen3.8,confidence,0.94,0.9254327563249002,0.26306323674307786,89787.01078668753,9136.349015057116 +construction|yolo26x|Qwen3.8,confidence,0.95,0.9364181091877497,0.26218185491449153,90851.21368755442,9244.252378505502 +construction|yolo26x|Qwen3.8,confidence,0.96,0.953728362183755,0.26357052137555514,92528.13947073862,9414.281920909021 +construction|yolo26x|Qwen3.8,confidence,0.97,0.96271637816245,0.26400290307863034,93398.85093508427,9502.566491003154 +construction|yolo26x|Qwen3.8,confidence,0.98,0.9700399467376831,0.2647475724296899,94108.3195356622,9574.502066635414 +construction|yolo26x|Qwen3.8,confidence,0.99,0.987017310252996,0.2649306125590882,95752.99674609285,9741.261810146556 +construction|yolo26x|Qwen3.8,confidence,1.0,1.0,0.27001009067886245,97010.691083481,9868.783966949195 diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance.pdf b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance.pdf new file mode 100644 index 0000000000000000000000000000000000000000..9d670d8dfb496dd6b05275823ddae1021cc0c4a7 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance.pdf differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance.png b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance.png new file mode 100644 index 0000000000000000000000000000000000000000..9642b63e33d52693ac5233a2c5b2c032c4023278 Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance_verified.png b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance_verified.png new file mode 100644 index 0000000000000000000000000000000000000000..0ab2e51ba1c6956d0c8a450bc6456558c286305a Binary files /dev/null and b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latency_performance_verified.png differ diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latex_includes.tex b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latex_includes.tex new file mode 100644 index 0000000000000000000000000000000000000000..49ea953ee4c37a03a823ba8d9b6920bdf5134434 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/latex_includes.tex @@ -0,0 +1,7 @@ +% Copy cost_compact.pdf alongside this snippet, or adjust the path. +\begin{figure}[t] + \centering + \includegraphics[width=\columnwidth]{cost_compact.pdf} + \caption{Test performance versus estimated computation (top) and serial latency (bottom) for InstanceVG--Qwen on gRefCOCO and YOLO26x--Qwen on Construction. Only the original main-table learned router and confidence baseline are compared. Dots mark the operating points selected on validation. Costs combine A800 component measurements with each policy's actual test call rate; learned routing includes the distilled encoders and routing head. FLOPs use supported-operator counts and generation-length estimates. Latency is estimated from batch-one measurements, not timed end-to-end for each policy.} + \label{fig:main-cost} +\end{figure} diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/main_table_results.json b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/main_table_results.json new file mode 100644 index 0000000000000000000000000000000000000000..ef8e3f2eb2ef3bd0d56909cef076cc4eabac62e8 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/main_table_results.json @@ -0,0 +1,10760 @@ +{ + "source": "C:\\Users\\csh10\\Documents\\CodexProjects\\ICASSP-COLLA\\paper_materials\\draft_20260912_revision19\\data\\results.json", + "protocol": "Confidence and learned routing each select a fixed threshold from the same 1..99% validation budget grid; threshold unchanged on test. No inference or training.", + "groups": [ + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "expert_display": "InstanceVG", + "run": "shared_12_instancevg_Qwen3.8", + "validation_rows": 14229, + "test_rows": 35263, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detections_above_07", + "max_score", + "mean_score", + "query_count", + "detections_above_07_valid", + "max_score_valid", + "mean_score_valid", + "query_count_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 71, + "validation_area_0_25": 0.7477616136060159, + "validation_point": { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8201560193970061, + "rescued": 1660, + "harmed": 666, + "missed_rescue": 304, + "avoided_harm": 505, + "selected_both_correct": 6505, + "selected_both_wrong": 1271 + }, + "test_point": { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.7243286163967898, + "rescued": 3794, + "harmed": 1861, + "missed_rescue": 1629, + "avoided_harm": 910, + "selected_both_correct": 13678, + "selected_both_wrong": 4782 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1964, + "avoided_harm": 1171, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7502284067748963, + "rescued": 0, + "harmed": 1, + "missed_rescue": 1964, + "avoided_harm": 1170, + "selected_both_correct": 141, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7501581277672359, + "rescued": 0, + "harmed": 2, + "missed_rescue": 1964, + "avoided_harm": 1169, + "selected_both_correct": 282, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.7498770117365943, + "rescued": 0, + "harmed": 6, + "missed_rescue": 1964, + "avoided_harm": 1165, + "selected_both_correct": 420, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7495958957059526, + "rescued": 0, + "harmed": 10, + "missed_rescue": 1964, + "avoided_harm": 1161, + "selected_both_correct": 559, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.7494553376906318, + "rescued": 0, + "harmed": 12, + "missed_rescue": 1964, + "avoided_harm": 1159, + "selected_both_correct": 699, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7493850586829714, + "rescued": 0, + "harmed": 13, + "missed_rescue": 1964, + "avoided_harm": 1158, + "selected_both_correct": 840, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.749314779675311, + "rescued": 0, + "harmed": 14, + "missed_rescue": 1964, + "avoided_harm": 1157, + "selected_both_correct": 982, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7491742216599901, + "rescued": 0, + "harmed": 16, + "missed_rescue": 1964, + "avoided_harm": 1155, + "selected_both_correct": 1122, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.748963384637009, + "rescued": 0, + "harmed": 19, + "missed_rescue": 1964, + "avoided_harm": 1152, + "selected_both_correct": 1261, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.7486119895987069, + "rescued": 0, + "harmed": 24, + "missed_rescue": 1964, + "avoided_harm": 1147, + "selected_both_correct": 1398, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.7483308735680653, + "rescued": 0, + "harmed": 28, + "missed_rescue": 1964, + "avoided_harm": 1143, + "selected_both_correct": 1537, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7481903155527444, + "rescued": 0, + "harmed": 30, + "missed_rescue": 1964, + "avoided_harm": 1141, + "selected_both_correct": 1677, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.7477686415067819, + "rescued": 0, + "harmed": 36, + "missed_rescue": 1964, + "avoided_harm": 1135, + "selected_both_correct": 1813, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.7476280834914611, + "rescued": 0, + "harmed": 38, + "missed_rescue": 1964, + "avoided_harm": 1133, + "selected_both_correct": 1954, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7474875254761403, + "rescued": 0, + "harmed": 40, + "missed_rescue": 1964, + "avoided_harm": 1131, + "selected_both_correct": 2094, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.15995502143509735, + "metric": 0.7472766884531591, + "rescued": 0, + "harmed": 43, + "missed_rescue": 1964, + "avoided_harm": 1128, + "selected_both_correct": 2233, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7471361304378382, + "rescued": 0, + "harmed": 45, + "missed_rescue": 1964, + "avoided_harm": 1126, + "selected_both_correct": 2373, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7470658514301778, + "rescued": 0, + "harmed": 46, + "missed_rescue": 1964, + "avoided_harm": 1125, + "selected_both_correct": 2515, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7465036193688945, + "rescued": 0, + "harmed": 54, + "missed_rescue": 1964, + "avoided_harm": 1117, + "selected_both_correct": 2649, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7462225033382529, + "rescued": 0, + "harmed": 58, + "missed_rescue": 1964, + "avoided_harm": 1113, + "selected_both_correct": 2787, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7456602712769695, + "rescued": 0, + "harmed": 66, + "missed_rescue": 1964, + "avoided_harm": 1105, + "selected_both_correct": 2922, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.7453088762386675, + "rescued": 0, + "harmed": 71, + "missed_rescue": 1964, + "avoided_harm": 1100, + "selected_both_correct": 3059, + "selected_both_wrong": 0 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.7446763651697238, + "rescued": 1, + "harmed": 81, + "missed_rescue": 1963, + "avoided_harm": 1090, + "selected_both_correct": 3190, + "selected_both_wrong": 0 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7442546911237613, + "rescued": 1, + "harmed": 87, + "missed_rescue": 1963, + "avoided_harm": 1084, + "selected_both_correct": 3326, + "selected_both_wrong": 0 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7437627380701385, + "rescued": 1, + "harmed": 94, + "missed_rescue": 1963, + "avoided_harm": 1077, + "selected_both_correct": 3462, + "selected_both_wrong": 0 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7431302270011948, + "rescued": 1, + "harmed": 103, + "missed_rescue": 1963, + "avoided_harm": 1068, + "selected_both_correct": 3595, + "selected_both_wrong": 0 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7422868789092698, + "rescued": 1, + "harmed": 115, + "missed_rescue": 1963, + "avoided_harm": 1056, + "selected_both_correct": 3725, + "selected_both_wrong": 0 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7419354838709677, + "rescued": 1, + "harmed": 120, + "missed_rescue": 1963, + "avoided_harm": 1051, + "selected_both_correct": 3862, + "selected_both_wrong": 1 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7411624147867032, + "rescued": 1, + "harmed": 131, + "missed_rescue": 1963, + "avoided_harm": 1040, + "selected_both_correct": 3993, + "selected_both_wrong": 1 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.7404596247100991, + "rescued": 2, + "harmed": 142, + "missed_rescue": 1962, + "avoided_harm": 1029, + "selected_both_correct": 4123, + "selected_both_wrong": 1 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7394757186028533, + "rescued": 2, + "harmed": 156, + "missed_rescue": 1962, + "avoided_harm": 1015, + "selected_both_correct": 4251, + "selected_both_wrong": 1 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.7388432075339096, + "rescued": 2, + "harmed": 165, + "missed_rescue": 1962, + "avoided_harm": 1006, + "selected_both_correct": 4385, + "selected_both_wrong": 1 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.7377890224190035, + "rescued": 2, + "harmed": 180, + "missed_rescue": 1962, + "avoided_harm": 991, + "selected_both_correct": 4512, + "selected_both_wrong": 1 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.7361023262351536, + "rescued": 2, + "harmed": 204, + "missed_rescue": 1962, + "avoided_harm": 967, + "selected_both_correct": 4629, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.7348373040972661, + "rescued": 2, + "harmed": 222, + "missed_rescue": 1962, + "avoided_harm": 949, + "selected_both_correct": 4754, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.7337128399746996, + "rescued": 2, + "harmed": 238, + "missed_rescue": 1962, + "avoided_harm": 933, + "selected_both_correct": 4880, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.732588375852133, + "rescued": 2, + "harmed": 254, + "missed_rescue": 1962, + "avoided_harm": 917, + "selected_both_correct": 5006, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.7311827956989247, + "rescued": 2, + "harmed": 274, + "missed_rescue": 1962, + "avoided_harm": 897, + "selected_both_correct": 5128, + "selected_both_wrong": 3 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.7297069365380561, + "rescued": 2, + "harmed": 295, + "missed_rescue": 1962, + "avoided_harm": 876, + "selected_both_correct": 5248, + "selected_both_wrong": 4 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.7282310773771874, + "rescued": 3, + "harmed": 317, + "missed_rescue": 1961, + "avoided_harm": 854, + "selected_both_correct": 5365, + "selected_both_wrong": 6 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.7266849392086584, + "rescued": 3, + "harmed": 339, + "missed_rescue": 1961, + "avoided_harm": 832, + "selected_both_correct": 5484, + "selected_both_wrong": 7 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.7249279640171481, + "rescued": 7, + "harmed": 368, + "missed_rescue": 1957, + "avoided_harm": 803, + "selected_both_correct": 5590, + "selected_both_wrong": 11 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.7233115468409586, + "rescued": 12, + "harmed": 396, + "missed_rescue": 1952, + "avoided_harm": 775, + "selected_both_correct": 5696, + "selected_both_wrong": 14 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.7212734556188066, + "rescued": 15, + "harmed": 428, + "missed_rescue": 1949, + "avoided_harm": 743, + "selected_both_correct": 5798, + "selected_both_wrong": 19 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.7190245273736735, + "rescued": 16, + "harmed": 461, + "missed_rescue": 1948, + "avoided_harm": 710, + "selected_both_correct": 5901, + "selected_both_wrong": 25 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.7173378311898236, + "rescued": 21, + "harmed": 490, + "missed_rescue": 1943, + "avoided_harm": 681, + "selected_both_correct": 6008, + "selected_both_wrong": 26 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.7155808559983133, + "rescued": 27, + "harmed": 521, + "missed_rescue": 1937, + "avoided_harm": 650, + "selected_both_correct": 6107, + "selected_both_wrong": 32 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.7206409445498629, + "rescued": 105, + "harmed": 527, + "missed_rescue": 1859, + "avoided_harm": 644, + "selected_both_correct": 6125, + "selected_both_wrong": 72 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.7280202403542062, + "rescued": 210, + "harmed": 527, + "missed_rescue": 1754, + "avoided_harm": 644, + "selected_both_correct": 6125, + "selected_both_wrong": 110 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.7346264670742849, + "rescued": 305, + "harmed": 528, + "missed_rescue": 1659, + "avoided_harm": 643, + "selected_both_correct": 6125, + "selected_both_wrong": 156 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.7414435308173448, + "rescued": 403, + "harmed": 529, + "missed_rescue": 1561, + "avoided_harm": 642, + "selected_both_correct": 6125, + "selected_both_wrong": 199 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.7478389205144423, + "rescued": 495, + "harmed": 530, + "missed_rescue": 1469, + "avoided_harm": 641, + "selected_both_correct": 6125, + "selected_both_wrong": 249 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.754093752196219, + "rescued": 585, + "harmed": 531, + "missed_rescue": 1379, + "avoided_harm": 640, + "selected_both_correct": 6125, + "selected_both_wrong": 300 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.7602783048703352, + "rescued": 673, + "harmed": 531, + "missed_rescue": 1291, + "avoided_harm": 640, + "selected_both_correct": 6127, + "selected_both_wrong": 352 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.7662520205214702, + "rescued": 759, + "harmed": 532, + "missed_rescue": 1205, + "avoided_harm": 639, + "selected_both_correct": 6129, + "selected_both_wrong": 405 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.7718743411343032, + "rescued": 844, + "harmed": 537, + "missed_rescue": 1120, + "avoided_harm": 634, + "selected_both_correct": 6130, + "selected_both_wrong": 457 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.7774263827394757, + "rescued": 924, + "harmed": 538, + "missed_rescue": 1040, + "avoided_harm": 633, + "selected_both_correct": 6133, + "selected_both_wrong": 515 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.7818539602220816, + "rescued": 991, + "harmed": 542, + "missed_rescue": 973, + "avoided_harm": 629, + "selected_both_correct": 6135, + "selected_both_wrong": 583 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.7864220957200084, + "rescued": 1061, + "harmed": 547, + "missed_rescue": 903, + "avoided_harm": 624, + "selected_both_correct": 6139, + "selected_both_wrong": 648 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.7909902312179352, + "rescued": 1130, + "harmed": 551, + "missed_rescue": 834, + "avoided_harm": 620, + "selected_both_correct": 6146, + "selected_both_wrong": 710 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.7954880877082016, + "rescued": 1200, + "harmed": 557, + "missed_rescue": 764, + "avoided_harm": 614, + "selected_both_correct": 6151, + "selected_both_wrong": 771 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.7996345491601659, + "rescued": 1263, + "harmed": 561, + "missed_rescue": 701, + "avoided_harm": 610, + "selected_both_correct": 6159, + "selected_both_wrong": 838 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.8028673835125448, + "rescued": 1317, + "harmed": 569, + "missed_rescue": 647, + "avoided_harm": 602, + "selected_both_correct": 6171, + "selected_both_wrong": 907 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.8064516129032258, + "rescued": 1380, + "harmed": 581, + "missed_rescue": 584, + "avoided_harm": 590, + "selected_both_correct": 6186, + "selected_both_wrong": 959 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.809895284278586, + "rescued": 1434, + "harmed": 586, + "missed_rescue": 530, + "avoided_harm": 585, + "selected_both_correct": 6211, + "selected_both_wrong": 1017 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.8123550495467003, + "rescued": 1479, + "harmed": 596, + "missed_rescue": 485, + "avoided_harm": 575, + "selected_both_correct": 6244, + "selected_both_wrong": 1072 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.8149553728301356, + "rescued": 1525, + "harmed": 605, + "missed_rescue": 439, + "avoided_harm": 566, + "selected_both_correct": 6278, + "selected_both_wrong": 1125 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.8169231850446271, + "rescued": 1569, + "harmed": 621, + "missed_rescue": 395, + "avoided_harm": 550, + "selected_both_correct": 6318, + "selected_both_wrong": 1167 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.818609881228477, + "rescued": 1605, + "harmed": 633, + "missed_rescue": 359, + "avoided_harm": 538, + "selected_both_correct": 6376, + "selected_both_wrong": 1204 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.6999086372900415, + "metric": 0.8196640663433832, + "rescued": 1636, + "harmed": 649, + "missed_rescue": 328, + "avoided_harm": 522, + "selected_both_correct": 6439, + "selected_both_wrong": 1235 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8201560193970061, + "rescued": 1660, + "harmed": 666, + "missed_rescue": 304, + "avoided_harm": 505, + "selected_both_correct": 6505, + "selected_both_wrong": 1271 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.8200154613816852, + "rescued": 1686, + "harmed": 694, + "missed_rescue": 278, + "avoided_harm": 477, + "selected_both_correct": 6565, + "selected_both_wrong": 1299 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.8199451823740249, + "rescued": 1703, + "harmed": 712, + "missed_rescue": 261, + "avoided_harm": 459, + "selected_both_correct": 6646, + "selected_both_wrong": 1326 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8200154613816852, + "rescued": 1722, + "harmed": 730, + "missed_rescue": 242, + "avoided_harm": 441, + "selected_both_correct": 6725, + "selected_both_wrong": 1352 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.8198749033663645, + "rescued": 1742, + "harmed": 752, + "missed_rescue": 222, + "avoided_harm": 419, + "selected_both_correct": 6808, + "selected_both_wrong": 1369 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8200154613816852, + "rescued": 1763, + "harmed": 771, + "missed_rescue": 201, + "avoided_harm": 400, + "selected_both_correct": 6891, + "selected_both_wrong": 1389 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8195235083280624, + "rescued": 1784, + "harmed": 799, + "missed_rescue": 180, + "avoided_harm": 372, + "selected_both_correct": 6966, + "selected_both_wrong": 1407 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.8191018342820999, + "rescued": 1800, + "harmed": 821, + "missed_rescue": 164, + "avoided_harm": 350, + "selected_both_correct": 7050, + "selected_both_wrong": 1427 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.8192423922974208, + "rescued": 1818, + "harmed": 837, + "missed_rescue": 146, + "avoided_harm": 334, + "selected_both_correct": 7143, + "selected_both_wrong": 1442 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.8190315552744395, + "rescued": 1834, + "harmed": 856, + "missed_rescue": 130, + "avoided_harm": 315, + "selected_both_correct": 7244, + "selected_both_wrong": 1449 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.8190315552744395, + "rescued": 1849, + "harmed": 871, + "missed_rescue": 115, + "avoided_harm": 300, + "selected_both_correct": 7347, + "selected_both_wrong": 1458 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.8187504392437979, + "rescued": 1859, + "harmed": 885, + "missed_rescue": 105, + "avoided_harm": 286, + "selected_both_correct": 7444, + "selected_both_wrong": 1479 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.818609881228477, + "rescued": 1874, + "harmed": 902, + "missed_rescue": 90, + "avoided_harm": 269, + "selected_both_correct": 7541, + "selected_both_wrong": 1493 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.8176962541288917, + "rescued": 1880, + "harmed": 921, + "missed_rescue": 84, + "avoided_harm": 250, + "selected_both_correct": 7643, + "selected_both_wrong": 1508 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.8175556961135709, + "rescued": 1889, + "harmed": 932, + "missed_rescue": 75, + "avoided_harm": 239, + "selected_both_correct": 7754, + "selected_both_wrong": 1519 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.8165717900063251, + "rescued": 1895, + "harmed": 952, + "missed_rescue": 69, + "avoided_harm": 219, + "selected_both_correct": 7864, + "selected_both_wrong": 1525 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.8160798369527023, + "rescued": 1907, + "harmed": 971, + "missed_rescue": 57, + "avoided_harm": 200, + "selected_both_correct": 7969, + "selected_both_wrong": 1532 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.8155878838990793, + "rescued": 1914, + "harmed": 985, + "missed_rescue": 50, + "avoided_harm": 186, + "selected_both_correct": 8082, + "selected_both_wrong": 1540 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.8148148148148148, + "rescued": 1919, + "harmed": 1001, + "missed_rescue": 45, + "avoided_harm": 170, + "selected_both_correct": 8192, + "selected_both_wrong": 1551 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.8144634197765127, + "rescued": 1927, + "harmed": 1014, + "missed_rescue": 37, + "avoided_harm": 157, + "selected_both_correct": 8308, + "selected_both_wrong": 1557 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.8129172816079837, + "rescued": 1932, + "harmed": 1041, + "missed_rescue": 32, + "avoided_harm": 130, + "selected_both_correct": 8410, + "selected_both_wrong": 1565 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9199522102747909, + "metric": 0.8123550495467003, + "rescued": 1937, + "harmed": 1054, + "missed_rescue": 27, + "avoided_harm": 117, + "selected_both_correct": 8530, + "selected_both_wrong": 1569 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.8113711434394546, + "rescued": 1942, + "harmed": 1073, + "missed_rescue": 22, + "avoided_harm": 98, + "selected_both_correct": 8647, + "selected_both_wrong": 1570 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.8102466793168881, + "rescued": 1945, + "harmed": 1092, + "missed_rescue": 19, + "avoided_harm": 79, + "selected_both_correct": 8764, + "selected_both_wrong": 1574 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.8087708201560194, + "rescued": 1948, + "harmed": 1116, + "missed_rescue": 16, + "avoided_harm": 55, + "selected_both_correct": 8876, + "selected_both_wrong": 1577 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8079977510717549, + "rescued": 1951, + "harmed": 1130, + "missed_rescue": 13, + "avoided_harm": 41, + "selected_both_correct": 8998, + "selected_both_wrong": 1580 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8075057980181319, + "rescued": 1956, + "harmed": 1142, + "missed_rescue": 8, + "avoided_harm": 29, + "selected_both_correct": 9120, + "selected_both_wrong": 1584 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8070138449645091, + "rescued": 1961, + "harmed": 1154, + "missed_rescue": 3, + "avoided_harm": 17, + "selected_both_correct": 9242, + "selected_both_wrong": 1587 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.806311054887905, + "rescued": 1962, + "harmed": 1165, + "missed_rescue": 2, + "avoided_harm": 6, + "selected_both_correct": 9370, + "selected_both_wrong": 1589 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 1964, + "harmed": 1171, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9505, + "selected_both_wrong": 1589 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 5423, + "avoided_harm": 2771, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": -2.0005727208172175e-07, + "operator": "ge", + "actual_fraction": 0.002467175226157729, + "metric": 0.6686895612965431, + "rescued": 0, + "harmed": 29, + "missed_rescue": 5423, + "avoided_harm": 2742, + "selected_both_correct": 58, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": -2.4156264544217265e-07, + "operator": "ge", + "actual_fraction": 0.004849275444516916, + "metric": 0.6680373195700876, + "rescued": 0, + "harmed": 52, + "missed_rescue": 5423, + "avoided_harm": 2719, + "selected_both_correct": 119, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": -2.7729177531909954e-07, + "operator": "ge", + "actual_fraction": 0.007600034030003119, + "metric": 0.6675835861951621, + "rescued": 0, + "harmed": 68, + "missed_rescue": 5423, + "avoided_harm": 2703, + "selected_both_correct": 199, + "selected_both_wrong": 1 + }, + { + "budget": 0.04, + "threshold": -3.064858447032748e-07, + "operator": "ge", + "actual_fraction": 0.009811984232765221, + "metric": 0.6671014944843037, + "rescued": 0, + "harmed": 85, + "missed_rescue": 5423, + "avoided_harm": 2686, + "selected_both_correct": 260, + "selected_both_wrong": 1 + }, + { + "budget": 0.05, + "threshold": -3.3848826319626824e-07, + "operator": "ge", + "actual_fraction": 0.012534384482318578, + "metric": 0.6666194027734452, + "rescued": 0, + "harmed": 102, + "missed_rescue": 5423, + "avoided_harm": 2669, + "selected_both_correct": 338, + "selected_both_wrong": 2 + }, + { + "budget": 0.06, + "threshold": -3.668622525765386e-07, + "operator": "ge", + "actual_fraction": 0.01607917647392451, + "metric": 0.6662223860703854, + "rescued": 0, + "harmed": 116, + "missed_rescue": 5423, + "avoided_harm": 2655, + "selected_both_correct": 449, + "selected_both_wrong": 2 + }, + { + "budget": 0.07, + "threshold": -3.98026287484754e-07, + "operator": "ge", + "actual_fraction": 0.019709043473328986, + "metric": 0.6658253693673255, + "rescued": 0, + "harmed": 130, + "missed_rescue": 5423, + "avoided_harm": 2641, + "selected_both_correct": 562, + "selected_both_wrong": 3 + }, + { + "budget": 0.08, + "threshold": -4.302527258914779e-07, + "operator": "ge", + "actual_fraction": 0.02353741882426339, + "metric": 0.6655985026798628, + "rescued": 0, + "harmed": 138, + "missed_rescue": 5423, + "avoided_harm": 2633, + "selected_both_correct": 689, + "selected_both_wrong": 3 + }, + { + "budget": 0.09, + "threshold": -4.7191912244670675e-07, + "operator": "ge", + "actual_fraction": 0.028868785979638715, + "metric": 0.6651164109690043, + "rescued": 0, + "harmed": 155, + "missed_rescue": 5423, + "avoided_harm": 2616, + "selected_both_correct": 859, + "selected_both_wrong": 4 + }, + { + "budget": 0.1, + "threshold": -5.115426802149159e-07, + "operator": "ge", + "actual_fraction": 0.0345971698380739, + "metric": 0.6647761109378102, + "rescued": 0, + "harmed": 167, + "missed_rescue": 5423, + "avoided_harm": 2604, + "selected_both_correct": 1045, + "selected_both_wrong": 8 + }, + { + "budget": 0.11, + "threshold": -5.534793672268279e-07, + "operator": "ge", + "actual_fraction": 0.03981510364971783, + "metric": 0.664435810906616, + "rescued": 0, + "harmed": 179, + "missed_rescue": 5423, + "avoided_harm": 2592, + "selected_both_correct": 1215, + "selected_both_wrong": 10 + }, + { + "budget": 0.12, + "threshold": -5.941800509390305e-07, + "operator": "ge", + "actual_fraction": 0.045855429203414344, + "metric": 0.664067152539489, + "rescued": 0, + "harmed": 192, + "missed_rescue": 5423, + "avoided_harm": 2579, + "selected_both_correct": 1414, + "selected_both_wrong": 11 + }, + { + "budget": 0.13, + "threshold": -6.464504735959054e-07, + "operator": "ge", + "actual_fraction": 0.05317187987408899, + "metric": 0.6637552108442276, + "rescued": 1, + "harmed": 204, + "missed_rescue": 5422, + "avoided_harm": 2567, + "selected_both_correct": 1658, + "selected_both_wrong": 12 + }, + { + "budget": 0.14, + "threshold": -6.999837864896108e-07, + "operator": "ge", + "actual_fraction": 0.05960922213084536, + "metric": 0.6636134191645634, + "rescued": 1, + "harmed": 209, + "missed_rescue": 5422, + "avoided_harm": 2562, + "selected_both_correct": 1878, + "selected_both_wrong": 14 + }, + { + "budget": 0.15, + "threshold": -7.663418841730163e-07, + "operator": "ge", + "actual_fraction": 0.0675211978561098, + "metric": 0.6632164024615036, + "rescued": 2, + "harmed": 224, + "missed_rescue": 5421, + "avoided_harm": 2547, + "selected_both_correct": 2128, + "selected_both_wrong": 27 + }, + { + "budget": 0.16, + "threshold": -8.471382670904859e-07, + "operator": "ge", + "actual_fraction": 0.07551824858917279, + "metric": 0.6626208774069138, + "rescued": 2, + "harmed": 245, + "missed_rescue": 5421, + "avoided_harm": 2526, + "selected_both_correct": 2386, + "selected_both_wrong": 30 + }, + { + "budget": 0.17, + "threshold": -9.292756431023008e-07, + "operator": "ge", + "actual_fraction": 0.08212574086152624, + "metric": 0.6622522190397867, + "rescued": 3, + "harmed": 259, + "missed_rescue": 5420, + "avoided_harm": 2512, + "selected_both_correct": 2599, + "selected_both_wrong": 35 + }, + { + "budget": 0.18, + "threshold": -1.0245233852401725e-06, + "operator": "ge", + "actual_fraction": 0.08856308311828262, + "metric": 0.6619969940163911, + "rescued": 3, + "harmed": 268, + "missed_rescue": 5420, + "avoided_harm": 2503, + "selected_both_correct": 2812, + "selected_both_wrong": 40 + }, + { + "budget": 0.19, + "threshold": -1.1468601996966754e-06, + "operator": "ge", + "actual_fraction": 0.09480191702350906, + "metric": 0.6615149023055327, + "rescued": 3, + "harmed": 285, + "missed_rescue": 5420, + "avoided_harm": 2486, + "selected_both_correct": 3005, + "selected_both_wrong": 50 + }, + { + "budget": 0.2, + "threshold": -1.3018121762797819e-06, + "operator": "ge", + "actual_fraction": 0.10007656750701868, + "metric": 0.6612596772821371, + "rescued": 3, + "harmed": 294, + "missed_rescue": 5420, + "avoided_harm": 2477, + "selected_both_correct": 3174, + "selected_both_wrong": 58 + }, + { + "budget": 0.21, + "threshold": -1.4898060953782988e-06, + "operator": "ge", + "actual_fraction": 0.10554972634205825, + "metric": 0.6608343022431443, + "rescued": 3, + "harmed": 309, + "missed_rescue": 5420, + "avoided_harm": 2462, + "selected_both_correct": 3340, + "selected_both_wrong": 70 + }, + { + "budget": 0.22, + "threshold": -1.735574414851726e-06, + "operator": "ge", + "actual_fraction": 0.11005870175538099, + "metric": 0.6605507188838159, + "rescued": 3, + "harmed": 319, + "missed_rescue": 5420, + "avoided_harm": 2452, + "selected_both_correct": 3485, + "selected_both_wrong": 74 + }, + { + "budget": 0.23, + "threshold": -2.0881013824691763e-06, + "operator": "ge", + "actual_fraction": 0.11527663556702493, + "metric": 0.6600969855088903, + "rescued": 5, + "harmed": 337, + "missed_rescue": 5418, + "avoided_harm": 2434, + "selected_both_correct": 3630, + "selected_both_wrong": 93 + }, + { + "budget": 0.24, + "threshold": -2.5644872039265465e-06, + "operator": "ge", + "actual_fraction": 0.12029606102713893, + "metric": 0.6598417604854947, + "rescued": 8, + "harmed": 349, + "missed_rescue": 5415, + "avoided_harm": 2422, + "selected_both_correct": 3769, + "selected_both_wrong": 116 + }, + { + "budget": 0.25, + "threshold": -3.2399550491390983e-06, + "operator": "ge", + "actual_fraction": 0.12477667810452883, + "metric": 0.6594731021183677, + "rescued": 8, + "harmed": 362, + "missed_rescue": 5415, + "avoided_harm": 2409, + "selected_both_correct": 3894, + "selected_both_wrong": 136 + }, + { + "budget": 0.26, + "threshold": -4.2517635847616475e-06, + "operator": "ge", + "actual_fraction": 0.12914386183818732, + "metric": 0.6590193687434421, + "rescued": 8, + "harmed": 378, + "missed_rescue": 5415, + "avoided_harm": 2393, + "selected_both_correct": 4012, + "selected_both_wrong": 156 + }, + { + "budget": 0.27, + "threshold": -5.607922958006384e-06, + "operator": "ge", + "actual_fraction": 0.1322916371267334, + "metric": 0.658679068712248, + "rescued": 10, + "harmed": 392, + "missed_rescue": 5413, + "avoided_harm": 2379, + "selected_both_correct": 4092, + "selected_both_wrong": 171 + }, + { + "budget": 0.28, + "threshold": -8.438555596512742e-06, + "operator": "ge", + "actual_fraction": 0.13626180415733205, + "metric": 0.6582820520091881, + "rescued": 12, + "harmed": 408, + "missed_rescue": 5411, + "avoided_harm": 2363, + "selected_both_correct": 4192, + "selected_both_wrong": 193 + }, + { + "budget": 0.29, + "threshold": -1.3145449884177651e-05, + "operator": "ge", + "actual_fraction": 0.14023197118793068, + "metric": 0.657743243626464, + "rescued": 13, + "harmed": 428, + "missed_rescue": 5410, + "avoided_harm": 2343, + "selected_both_correct": 4285, + "selected_both_wrong": 219 + }, + { + "budget": 0.3, + "threshold": -2.076632881653495e-05, + "operator": "ge", + "actual_fraction": 0.14326631313274538, + "metric": 0.657374585259337, + "rescued": 16, + "harmed": 444, + "missed_rescue": 5407, + "avoided_harm": 2327, + "selected_both_correct": 4359, + "selected_both_wrong": 233 + }, + { + "budget": 0.31, + "threshold": -4.0114435250870883e-05, + "operator": "ge", + "actual_fraction": 0.14766185520233674, + "metric": 0.6569775685562771, + "rescued": 18, + "harmed": 460, + "missed_rescue": 5405, + "avoided_harm": 2311, + "selected_both_correct": 4464, + "selected_both_wrong": 265 + }, + { + "budget": 0.32, + "threshold": -7.415730942739174e-05, + "operator": "ge", + "actual_fraction": 0.15066783881121856, + "metric": 0.6567507018688143, + "rescued": 20, + "harmed": 470, + "missed_rescue": 5403, + "avoided_harm": 2301, + "selected_both_correct": 4545, + "selected_both_wrong": 278 + }, + { + "budget": 0.33, + "threshold": -0.00015223574882838875, + "operator": "ge", + "actual_fraction": 0.15452457249808582, + "metric": 0.6565238351813516, + "rescued": 27, + "harmed": 485, + "missed_rescue": 5396, + "avoided_harm": 2286, + "selected_both_correct": 4632, + "selected_both_wrong": 305 + }, + { + "budget": 0.34, + "threshold": -0.00032258269493468106, + "operator": "ge", + "actual_fraction": 0.15806936448969175, + "metric": 0.6564104018376201, + "rescued": 39, + "harmed": 501, + "missed_rescue": 5384, + "avoided_harm": 2270, + "selected_both_correct": 4705, + "selected_both_wrong": 329 + }, + { + "budget": 0.35, + "threshold": -0.0006007538759149611, + "operator": "ge", + "actual_fraction": 0.16201117318435754, + "metric": 0.6562686101579559, + "rescued": 46, + "harmed": 513, + "missed_rescue": 5377, + "avoided_harm": 2258, + "selected_both_correct": 4777, + "selected_both_wrong": 377 + }, + { + "budget": 0.36, + "threshold": -0.0011453087208792567, + "operator": "ge", + "actual_fraction": 0.1657544735274934, + "metric": 0.6558999517908289, + "rescued": 48, + "harmed": 528, + "missed_rescue": 5375, + "avoided_harm": 2243, + "selected_both_correct": 4841, + "selected_both_wrong": 428 + }, + { + "budget": 0.37, + "threshold": -0.002608026610687375, + "operator": "ge", + "actual_fraction": 0.17012165726115192, + "metric": 0.6558715934548961, + "rescued": 61, + "harmed": 542, + "missed_rescue": 5362, + "avoided_harm": 2229, + "selected_both_correct": 4937, + "selected_both_wrong": 459 + }, + { + "budget": 0.38, + "threshold": -0.006219128146767616, + "operator": "ge", + "actual_fraction": 0.17536794940872868, + "metric": 0.6555596517596347, + "rescued": 79, + "harmed": 571, + "missed_rescue": 5344, + "avoided_harm": 2200, + "selected_both_correct": 5018, + "selected_both_wrong": 516 + }, + { + "budget": 0.39, + "threshold": -0.01280172262340784, + "operator": "ge", + "actual_fraction": 0.17965005813458867, + "metric": 0.6553044267362391, + "rescued": 92, + "harmed": 593, + "missed_rescue": 5331, + "avoided_harm": 2178, + "selected_both_correct": 5091, + "selected_both_wrong": 559 + }, + { + "budget": 0.4, + "threshold": -0.03794781118631363, + "operator": "ge", + "actual_fraction": 0.18401724186824717, + "metric": 0.6552193517284406, + "rescued": 107, + "harmed": 611, + "missed_rescue": 5316, + "avoided_harm": 2160, + "selected_both_correct": 5145, + "selected_both_wrong": 626 + }, + { + "budget": 0.41, + "threshold": -0.08479184657335281, + "operator": "ge", + "actual_fraction": 0.18858293395343562, + "metric": 0.6552760684003063, + "rescued": 127, + "harmed": 629, + "missed_rescue": 5296, + "avoided_harm": 2142, + "selected_both_correct": 5207, + "selected_both_wrong": 687 + }, + { + "budget": 0.42, + "threshold": -0.14941807091236115, + "operator": "ge", + "actual_fraction": 0.19238295096843716, + "metric": 0.6554745767518362, + "rescued": 144, + "harmed": 639, + "missed_rescue": 5279, + "avoided_harm": 2132, + "selected_both_correct": 5277, + "selected_both_wrong": 724 + }, + { + "budget": 0.43, + "threshold": -0.2939925789833069, + "operator": "ge", + "actual_fraction": 0.19689192638175992, + "metric": 0.655502935087769, + "rescued": 168, + "harmed": 662, + "missed_rescue": 5255, + "avoided_harm": 2109, + "selected_both_correct": 5344, + "selected_both_wrong": 769 + }, + { + "budget": 0.44, + "threshold": -0.4386065602302551, + "operator": "ge", + "actual_fraction": 0.20182627683407536, + "metric": 0.6555880100955676, + "rescued": 199, + "harmed": 690, + "missed_rescue": 5224, + "avoided_harm": 2081, + "selected_both_correct": 5429, + "selected_both_wrong": 799 + }, + { + "budget": 0.45, + "threshold": -0.5420411825180054, + "operator": "ge", + "actual_fraction": 0.20718600232538353, + "metric": 0.6559283101267618, + "rescued": 240, + "harmed": 719, + "missed_rescue": 5183, + "avoided_harm": 2052, + "selected_both_correct": 5506, + "selected_both_wrong": 841 + }, + { + "budget": 0.46, + "threshold": -0.6230618953704834, + "operator": "ge", + "actual_fraction": 0.21189348609023623, + "metric": 0.656070101806426, + "rescued": 280, + "harmed": 754, + "missed_rescue": 5143, + "avoided_harm": 2017, + "selected_both_correct": 5554, + "selected_both_wrong": 884 + }, + { + "budget": 0.47, + "threshold": -0.6914289593696594, + "operator": "ge", + "actual_fraction": 0.217139778237813, + "metric": 0.6567790602047472, + "rescued": 329, + "harmed": 778, + "missed_rescue": 5094, + "avoided_harm": 1993, + "selected_both_correct": 5614, + "selected_both_wrong": 936 + }, + { + "budget": 0.48, + "threshold": -0.7387455701828003, + "operator": "ge", + "actual_fraction": 0.22198905368232993, + "metric": 0.6584238436888523, + "rescued": 399, + "harmed": 790, + "missed_rescue": 5024, + "avoided_harm": 1981, + "selected_both_correct": 5648, + "selected_both_wrong": 991 + }, + { + "budget": 0.49, + "threshold": -0.7823225855827332, + "operator": "ge", + "actual_fraction": 0.22763236253296656, + "metric": 0.6604372855400845, + "rescued": 483, + "harmed": 803, + "missed_rescue": 4940, + "avoided_harm": 1968, + "selected_both_correct": 5689, + "selected_both_wrong": 1052 + }, + { + "budget": 0.5, + "threshold": -0.8206579089164734, + "operator": "ge", + "actual_fraction": 0.2345234381646485, + "metric": 0.662762669086578, + "rescued": 579, + "harmed": 817, + "missed_rescue": 4844, + "avoided_harm": 1954, + "selected_both_correct": 5735, + "selected_both_wrong": 1139 + }, + { + "budget": 0.51, + "threshold": -0.8575630187988281, + "operator": "ge", + "actual_fraction": 0.24382497235062248, + "metric": 0.6654567110001985, + "rescued": 702, + "harmed": 845, + "missed_rescue": 4721, + "avoided_harm": 1926, + "selected_both_correct": 5794, + "selected_both_wrong": 1257 + }, + { + "budget": 0.52, + "threshold": -0.8823798894882202, + "operator": "ge", + "actual_fraction": 0.252644414825738, + "metric": 0.6678955278904234, + "rescued": 813, + "harmed": 870, + "missed_rescue": 4610, + "avoided_harm": 1901, + "selected_both_correct": 5842, + "selected_both_wrong": 1384 + }, + { + "budget": 0.53, + "threshold": -0.9040287733078003, + "operator": "ge", + "actual_fraction": 0.26401610753480986, + "metric": 0.6703627031165811, + "rescued": 937, + "harmed": 907, + "missed_rescue": 4486, + "avoided_harm": 1864, + "selected_both_correct": 5946, + "selected_both_wrong": 1520 + }, + { + "budget": 0.54, + "threshold": -0.9208687543869019, + "operator": "ge", + "actual_fraction": 0.2760116836344043, + "metric": 0.6731134617020673, + "rescued": 1067, + "harmed": 940, + "missed_rescue": 4356, + "avoided_harm": 1831, + "selected_both_correct": 6068, + "selected_both_wrong": 1658 + }, + { + "budget": 0.55, + "threshold": -0.9344452619552612, + "operator": "ge", + "actual_fraction": 0.2893401015228426, + "metric": 0.6758358619516207, + "rescued": 1197, + "harmed": 974, + "missed_rescue": 4226, + "avoided_harm": 1797, + "selected_both_correct": 6218, + "selected_both_wrong": 1814 + }, + { + "budget": 0.56, + "threshold": -0.9448322653770447, + "operator": "ge", + "actual_fraction": 0.3036043444970649, + "metric": 0.679295578935428, + "rescued": 1352, + "harmed": 1007, + "missed_rescue": 4071, + "avoided_harm": 1764, + "selected_both_correct": 6385, + "selected_both_wrong": 1962 + }, + { + "budget": 0.57, + "threshold": -0.9531905055046082, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.6822448458724442, + "rescued": 1489, + "harmed": 1040, + "missed_rescue": 3934, + "avoided_harm": 1731, + "selected_both_correct": 6573, + "selected_both_wrong": 2097 + }, + { + "budget": 0.58, + "threshold": -0.9603281617164612, + "operator": "gt", + "actual_fraction": 0.33462836400760004, + "metric": 0.6853926211609903, + "rescued": 1645, + "harmed": 1085, + "missed_rescue": 3778, + "avoided_harm": 1686, + "selected_both_correct": 6816, + "selected_both_wrong": 2254 + }, + { + "budget": 0.59, + "threshold": -0.9670801758766174, + "operator": "ge", + "actual_fraction": 0.35637920766809406, + "metric": 0.6889090548166633, + "rescued": 1818, + "harmed": 1134, + "missed_rescue": 3605, + "avoided_harm": 1637, + "selected_both_correct": 7175, + "selected_both_wrong": 2440 + }, + { + "budget": 0.6, + "threshold": -0.9724565148353577, + "operator": "ge", + "actual_fraction": 0.37968975980489467, + "metric": 0.6922553384567394, + "rescued": 1978, + "harmed": 1176, + "missed_rescue": 3445, + "avoided_harm": 1595, + "selected_both_correct": 7569, + "selected_both_wrong": 2666 + }, + { + "budget": 0.61, + "threshold": -0.9766654968261719, + "operator": "ge", + "actual_fraction": 0.4046450954258004, + "metric": 0.6949493803703599, + "rescued": 2126, + "harmed": 1229, + "missed_rescue": 3297, + "avoided_harm": 1542, + "selected_both_correct": 8013, + "selected_both_wrong": 2901 + }, + { + "budget": 0.62, + "threshold": -0.9801702499389648, + "operator": "ge", + "actual_fraction": 0.43141536454640844, + "metric": 0.6983807390182344, + "rescued": 2296, + "harmed": 1278, + "missed_rescue": 3127, + "avoided_harm": 1493, + "selected_both_correct": 8524, + "selected_both_wrong": 3115 + }, + { + "budget": 0.63, + "threshold": -0.9831460118293762, + "operator": "ge", + "actual_fraction": 0.4610214672603012, + "metric": 0.7015568726427134, + "rescued": 2479, + "harmed": 1349, + "missed_rescue": 2944, + "avoided_harm": 1422, + "selected_both_correct": 9071, + "selected_both_wrong": 3358 + }, + { + "budget": 0.64, + "threshold": -0.985751748085022, + "operator": "ge", + "actual_fraction": 0.4916484700677764, + "metric": 0.7053285313217821, + "rescued": 2674, + "harmed": 1411, + "missed_rescue": 2749, + "avoided_harm": 1360, + "selected_both_correct": 9690, + "selected_both_wrong": 3562 + }, + { + "budget": 0.65, + "threshold": -0.9877713322639465, + "operator": "ge", + "actual_fraction": 0.5209426310864078, + "metric": 0.7086180982899923, + "rescued": 2849, + "harmed": 1470, + "missed_rescue": 2574, + "avoided_harm": 1301, + "selected_both_correct": 10321, + "selected_both_wrong": 3730 + }, + { + "budget": 0.66, + "threshold": -0.9895955324172974, + "operator": "ge", + "actual_fraction": 0.5529308340186598, + "metric": 0.71148229021921, + "rescued": 3014, + "harmed": 1534, + "missed_rescue": 2409, + "avoided_harm": 1237, + "selected_both_correct": 11019, + "selected_both_wrong": 3931 + }, + { + "budget": 0.67, + "threshold": -0.9913432002067566, + "operator": "ge", + "actual_fraction": 0.5888041289737118, + "metric": 0.7155091739216742, + "rescued": 3227, + "harmed": 1605, + "missed_rescue": 2196, + "avoided_harm": 1166, + "selected_both_correct": 11762, + "selected_both_wrong": 4169 + }, + { + "budget": 0.68, + "threshold": -0.9925730228424072, + "operator": "ge", + "actual_fraction": 0.6175027649377535, + "metric": 0.7182882908430933, + "rescued": 3404, + "harmed": 1684, + "missed_rescue": 2019, + "avoided_harm": 1087, + "selected_both_correct": 12326, + "selected_both_wrong": 4361 + }, + { + "budget": 0.69, + "threshold": -0.9935283660888672, + "operator": "ge", + "actual_fraction": 0.6421461588633979, + "metric": 0.7211241244363781, + "rescued": 3553, + "harmed": 1733, + "missed_rescue": 1870, + "avoided_harm": 1038, + "selected_both_correct": 12849, + "selected_both_wrong": 4509 + }, + { + "budget": 0.7, + "threshold": -0.99436354637146, + "operator": "gt", + "actual_fraction": 0.665910444375124, + "metric": 0.7233077163032073, + "rescued": 3694, + "harmed": 1797, + "missed_rescue": 1729, + "avoided_harm": 974, + "selected_both_correct": 13314, + "selected_both_wrong": 4677 + }, + { + "budget": 0.71, + "threshold": -0.9949791431427002, + "operator": "ge", + "actual_fraction": 0.6838612710206166, + "metric": 0.7243286163967898, + "rescued": 3794, + "harmed": 1861, + "missed_rescue": 1629, + "avoided_harm": 910, + "selected_both_correct": 13678, + "selected_both_wrong": 4782 + }, + { + "budget": 0.72, + "threshold": -0.9955723881721497, + "operator": "ge", + "actual_fraction": 0.7029464311034228, + "metric": 0.7260017582168279, + "rescued": 3908, + "harmed": 1916, + "missed_rescue": 1515, + "avoided_harm": 855, + "selected_both_correct": 14058, + "selected_both_wrong": 4906 + }, + { + "budget": 0.73, + "threshold": -0.9960037469863892, + "operator": "ge", + "actual_fraction": 0.7180614241556306, + "metric": 0.7278450500524629, + "rescued": 4015, + "harmed": 1958, + "missed_rescue": 1408, + "avoided_harm": 813, + "selected_both_correct": 14354, + "selected_both_wrong": 4994 + }, + { + "budget": 0.74, + "threshold": -0.9964693188667297, + "operator": "ge", + "actual_fraction": 0.7352465757309361, + "metric": 0.7297450585599637, + "rescued": 4134, + "harmed": 2010, + "missed_rescue": 1289, + "avoided_harm": 761, + "selected_both_correct": 14689, + "selected_both_wrong": 5094 + }, + { + "budget": 0.75, + "threshold": -0.9968041777610779, + "operator": "ge", + "actual_fraction": 0.7478093185491875, + "metric": 0.7311629753566061, + "rescued": 4208, + "harmed": 2034, + "missed_rescue": 1215, + "avoided_harm": 737, + "selected_both_correct": 14961, + "selected_both_wrong": 5167 + }, + { + "budget": 0.76, + "threshold": -0.9971042275428772, + "operator": "ge", + "actual_fraction": 0.759804894648782, + "metric": 0.7328928338485098, + "rescued": 4299, + "harmed": 2064, + "missed_rescue": 1124, + "avoided_harm": 707, + "selected_both_correct": 15186, + "selected_both_wrong": 5244 + }, + { + "budget": 0.77, + "threshold": -0.9974007606506348, + "operator": "ge", + "actual_fraction": 0.7729064458497575, + "metric": 0.7342540339732865, + "rescued": 4385, + "harmed": 2102, + "missed_rescue": 1038, + "avoided_harm": 669, + "selected_both_correct": 15447, + "selected_both_wrong": 5321 + }, + { + "budget": 0.78, + "threshold": -0.9976577162742615, + "operator": "ge", + "actual_fraction": 0.7855259053398747, + "metric": 0.735274934066869, + "rescued": 4462, + "harmed": 2143, + "missed_rescue": 961, + "avoided_harm": 628, + "selected_both_correct": 15703, + "selected_both_wrong": 5392 + }, + { + "budget": 0.79, + "threshold": -0.9978766441345215, + "operator": "ge", + "actual_fraction": 0.7964722230099538, + "metric": 0.7362958341604514, + "rescued": 4527, + "harmed": 2172, + "missed_rescue": 896, + "avoided_harm": 599, + "selected_both_correct": 15920, + "selected_both_wrong": 5467 + }, + { + "budget": 0.8, + "threshold": -0.9980920553207397, + "operator": "ge", + "actual_fraction": 0.8082125740861527, + "metric": 0.7377704676289595, + "rescued": 4609, + "harmed": 2202, + "missed_rescue": 814, + "avoided_harm": 569, + "selected_both_correct": 16159, + "selected_both_wrong": 5530 + }, + { + "budget": 0.81, + "threshold": -0.9982843399047852, + "operator": "ge", + "actual_fraction": 0.8197544168108215, + "metric": 0.738422709355415, + "rescued": 4678, + "harmed": 2248, + "missed_rescue": 745, + "avoided_harm": 523, + "selected_both_correct": 16389, + "selected_both_wrong": 5592 + }, + { + "budget": 0.82, + "threshold": -0.9984900951385498, + "operator": "ge", + "actual_fraction": 0.8333380597226555, + "metric": 0.7389048010662734, + "rescued": 4746, + "harmed": 2299, + "missed_rescue": 677, + "avoided_harm": 472, + "selected_both_correct": 16674, + "selected_both_wrong": 5667 + }, + { + "budget": 0.83, + "threshold": -0.9986395835876465, + "operator": "ge", + "actual_fraction": 0.8426112355726966, + "metric": 0.7398689844879902, + "rescued": 4807, + "harmed": 2326, + "missed_rescue": 616, + "avoided_harm": 445, + "selected_both_correct": 16857, + "selected_both_wrong": 5723 + }, + { + "budget": 0.84, + "threshold": -0.9987816214561462, + "operator": "ge", + "actual_fraction": 0.8529903865241187, + "metric": 0.7403227178629158, + "rescued": 4859, + "harmed": 2362, + "missed_rescue": 564, + "avoided_harm": 409, + "selected_both_correct": 17089, + "selected_both_wrong": 5769 + }, + { + "budget": 0.85, + "threshold": -0.9989162087440491, + "operator": "ge", + "actual_fraction": 0.8635396874911381, + "metric": 0.7406913762300428, + "rescued": 4914, + "harmed": 2404, + "missed_rescue": 509, + "avoided_harm": 367, + "selected_both_correct": 17300, + "selected_both_wrong": 5833 + }, + { + "budget": 0.86, + "threshold": -0.9990254044532776, + "operator": "ge", + "actual_fraction": 0.8710262881774098, + "metric": 0.7413152596205654, + "rescued": 4962, + "harmed": 2430, + "missed_rescue": 461, + "avoided_harm": 341, + "selected_both_correct": 17455, + "selected_both_wrong": 5868 + }, + { + "budget": 0.87, + "threshold": -0.9991487264633179, + "operator": "ge", + "actual_fraction": 0.8809517057539064, + "metric": 0.7422227263704165, + "rescued": 5018, + "harmed": 2454, + "missed_rescue": 405, + "avoided_harm": 317, + "selected_both_correct": 17666, + "selected_both_wrong": 5927 + }, + { + "budget": 0.88, + "threshold": -0.9992602467536926, + "operator": "ge", + "actual_fraction": 0.8913024983693957, + "metric": 0.7430167597765364, + "rescued": 5074, + "harmed": 2482, + "missed_rescue": 349, + "avoided_harm": 289, + "selected_both_correct": 17892, + "selected_both_wrong": 5982 + }, + { + "budget": 0.89, + "threshold": -0.999350905418396, + "operator": "ge", + "actual_fraction": 0.900915974250631, + "metric": 0.743442134815529, + "rescued": 5120, + "harmed": 2513, + "missed_rescue": 303, + "avoided_harm": 258, + "selected_both_correct": 18112, + "selected_both_wrong": 6024 + }, + { + "budget": 0.9, + "threshold": -0.999435544013977, + "operator": "ge", + "actual_fraction": 0.910954825170859, + "metric": 0.7443212432294473, + "rescued": 5172, + "harmed": 2534, + "missed_rescue": 251, + "avoided_harm": 237, + "selected_both_correct": 18362, + "selected_both_wrong": 6055 + }, + { + "budget": 0.91, + "threshold": -0.9995256066322327, + "operator": "ge", + "actual_fraction": 0.921901142840938, + "metric": 0.7448033349403057, + "rescued": 5223, + "harmed": 2568, + "missed_rescue": 200, + "avoided_harm": 203, + "selected_both_correct": 18626, + "selected_both_wrong": 6092 + }, + { + "budget": 0.92, + "threshold": -0.9995898604393005, + "operator": "ge", + "actual_fraction": 0.9303235686129938, + "metric": 0.7450018432918356, + "rescued": 5253, + "harmed": 2591, + "missed_rescue": 170, + "avoided_harm": 180, + "selected_both_correct": 18846, + "selected_both_wrong": 6116 + }, + { + "budget": 0.93, + "threshold": -0.9996523857116699, + "operator": "ge", + "actual_fraction": 0.9393698777755721, + "metric": 0.7448884099481042, + "rescued": 5284, + "harmed": 2626, + "missed_rescue": 139, + "avoided_harm": 145, + "selected_both_correct": 19077, + "selected_both_wrong": 6138 + }, + { + "budget": 0.94, + "threshold": -0.9997095465660095, + "operator": "ge", + "actual_fraction": 0.9490117119927403, + "metric": 0.7448884099481042, + "rescued": 5314, + "harmed": 2656, + "missed_rescue": 109, + "avoided_harm": 115, + "selected_both_correct": 19329, + "selected_both_wrong": 6166 + }, + { + "budget": 0.95, + "threshold": -0.9997634887695312, + "operator": "ge", + "actual_fraction": 0.9587953378895726, + "metric": 0.7449734849559028, + "rescued": 5347, + "harmed": 2686, + "missed_rescue": 76, + "avoided_harm": 85, + "selected_both_correct": 19588, + "selected_both_wrong": 6189 + }, + { + "budget": 0.96, + "threshold": -0.9998065829277039, + "operator": "ge", + "actual_fraction": 0.9673595553412926, + "metric": 0.7452570683152313, + "rescued": 5375, + "harmed": 2704, + "missed_rescue": 48, + "avoided_harm": 67, + "selected_both_correct": 19828, + "selected_both_wrong": 6205 + }, + { + "budget": 0.97, + "threshold": -0.9998431205749512, + "operator": "ge", + "actual_fraction": 0.9746192893401016, + "metric": 0.745313784987097, + "rescued": 5398, + "harmed": 2725, + "missed_rescue": 25, + "avoided_harm": 46, + "selected_both_correct": 20033, + "selected_both_wrong": 6212 + }, + { + "budget": 0.98, + "threshold": -0.9998764991760254, + "operator": "ge", + "actual_fraction": 0.9830984317840229, + "metric": 0.7450585599637013, + "rescued": 5411, + "harmed": 2747, + "missed_rescue": 12, + "avoided_harm": 24, + "selected_both_correct": 20290, + "selected_both_wrong": 6219 + }, + { + "budget": 0.99, + "threshold": -0.9999123811721802, + "operator": "ge", + "actual_fraction": 0.9908118991577575, + "metric": 0.7448884099481042, + "rescued": 5419, + "harmed": 2761, + "missed_rescue": 4, + "avoided_harm": 10, + "selected_both_correct": 20535, + "selected_both_wrong": 6224 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 5423, + "harmed": 2771, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 20838, + "selected_both_wrong": 6231 + } + ] + }, + "learned": { + "selected_budget_index": 56, + "validation_area_0_25": 0.777812917281608, + "validation_point": { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.8081383090870757, + "rescued": 1513, + "harmed": 690, + "missed_rescue": 451, + "avoided_harm": 481, + "selected_both_correct": 4599, + "selected_both_wrong": 1166 + }, + "test_point": { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5355755324277571, + "metric": 0.7528287440093016, + "rescued": 4361, + "harmed": 1423, + "missed_rescue": 1062, + "avoided_harm": 1348, + "selected_both_correct": 8197, + "selected_both_wrong": 4905 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.7502986857825568, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1964, + "avoided_harm": 1171, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 5.873929023742676, + "operator": "ge", + "actual_fraction": 0.009979619087778481, + "metric": 0.7559210063953897, + "rescued": 80, + "harmed": 0, + "missed_rescue": 1884, + "avoided_harm": 1171, + "selected_both_correct": 1, + "selected_both_wrong": 61 + }, + { + "budget": 0.02, + "threshold": 3.9572336673736572, + "operator": "ge", + "actual_fraction": 0.019959238175556962, + "metric": 0.7606999789162977, + "rescued": 149, + "harmed": 1, + "missed_rescue": 1815, + "avoided_harm": 1170, + "selected_both_correct": 3, + "selected_both_wrong": 131 + }, + { + "budget": 0.03, + "threshold": 2.499727725982666, + "operator": "ge", + "actual_fraction": 0.029938857263335443, + "metric": 0.765197835406564, + "rescued": 217, + "harmed": 5, + "missed_rescue": 1747, + "avoided_harm": 1166, + "selected_both_correct": 16, + "selected_both_wrong": 188 + }, + { + "budget": 0.04, + "threshold": 1.2223308086395264, + "operator": "ge", + "actual_fraction": 0.03998875535877434, + "metric": 0.7671656476210555, + "rescued": 258, + "harmed": 18, + "missed_rescue": 1706, + "avoided_harm": 1153, + "selected_both_correct": 51, + "selected_both_wrong": 242 + }, + { + "budget": 0.05, + "threshold": 0.36731216311454773, + "operator": "ge", + "actual_fraction": 0.04996837444655281, + "metric": 0.767587321667018, + "rescued": 283, + "harmed": 37, + "missed_rescue": 1681, + "avoided_harm": 1134, + "selected_both_correct": 113, + "selected_both_wrong": 278 + }, + { + "budget": 0.06, + "threshold": -0.14144867658615112, + "operator": "ge", + "actual_fraction": 0.05994799353433129, + "metric": 0.7663925785367911, + "rescued": 293, + "harmed": 64, + "missed_rescue": 1671, + "avoided_harm": 1107, + "selected_both_correct": 199, + "selected_both_wrong": 297 + }, + { + "budget": 0.07, + "threshold": -0.42117512226104736, + "operator": "ge", + "actual_fraction": 0.06999789162977019, + "metric": 0.7665331365521119, + "rescued": 320, + "harmed": 89, + "missed_rescue": 1644, + "avoided_harm": 1082, + "selected_both_correct": 259, + "selected_both_wrong": 328 + }, + { + "budget": 0.08, + "threshold": -0.6775990128517151, + "operator": "ge", + "actual_fraction": 0.07997751071754867, + "metric": 0.7686415067819242, + "rescued": 368, + "harmed": 107, + "missed_rescue": 1596, + "avoided_harm": 1064, + "selected_both_correct": 304, + "selected_both_wrong": 359 + }, + { + "budget": 0.09, + "threshold": -0.9111056327819824, + "operator": "ge", + "actual_fraction": 0.08995712980532715, + "metric": 0.7707498770117366, + "rescued": 412, + "harmed": 121, + "missed_rescue": 1552, + "avoided_harm": 1050, + "selected_both_correct": 343, + "selected_both_wrong": 404 + }, + { + "budget": 0.1, + "threshold": -1.1332147121429443, + "operator": "ge", + "actual_fraction": 0.09993674889310562, + "metric": 0.772858247241549, + "rescued": 454, + "harmed": 133, + "missed_rescue": 1510, + "avoided_harm": 1038, + "selected_both_correct": 393, + "selected_both_wrong": 442 + }, + { + "budget": 0.11, + "threshold": -1.3457791805267334, + "operator": "ge", + "actual_fraction": 0.10998664698854452, + "metric": 0.774404385410078, + "rescued": 491, + "harmed": 148, + "missed_rescue": 1473, + "avoided_harm": 1023, + "selected_both_correct": 446, + "selected_both_wrong": 480 + }, + { + "budget": 0.12, + "threshold": -1.5916669368743896, + "operator": "ge", + "actual_fraction": 0.119966266076323, + "metric": 0.7769344296858528, + "rescued": 541, + "harmed": 162, + "missed_rescue": 1423, + "avoided_harm": 1009, + "selected_both_correct": 492, + "selected_both_wrong": 512 + }, + { + "budget": 0.13, + "threshold": -1.7836966514587402, + "operator": "ge", + "actual_fraction": 0.1299458851641015, + "metric": 0.7791130789233256, + "rescued": 584, + "harmed": 174, + "missed_rescue": 1380, + "avoided_harm": 997, + "selected_both_correct": 543, + "selected_both_wrong": 548 + }, + { + "budget": 0.14, + "threshold": -1.9795069694519043, + "operator": "ge", + "actual_fraction": 0.13999578325954037, + "metric": 0.781221449153138, + "rescued": 632, + "harmed": 192, + "missed_rescue": 1332, + "avoided_harm": 979, + "selected_both_correct": 597, + "selected_both_wrong": 571 + }, + { + "budget": 0.15, + "threshold": -2.1732606887817383, + "operator": "ge", + "actual_fraction": 0.14997540234731885, + "metric": 0.7838920514442336, + "rescued": 678, + "harmed": 200, + "missed_rescue": 1286, + "avoided_harm": 971, + "selected_both_correct": 654, + "selected_both_wrong": 602 + }, + { + "budget": 0.16, + "threshold": -2.3698818683624268, + "operator": "gt", + "actual_fraction": 0.15988474242743692, + "metric": 0.786351816712348, + "rescued": 724, + "harmed": 211, + "missed_rescue": 1240, + "avoided_harm": 960, + "selected_both_correct": 713, + "selected_both_wrong": 627 + }, + { + "budget": 0.17, + "threshold": -2.5250070095062256, + "operator": "ge", + "actual_fraction": 0.16993464052287582, + "metric": 0.7876168388502355, + "rescued": 756, + "harmed": 225, + "missed_rescue": 1208, + "avoided_harm": 946, + "selected_both_correct": 785, + "selected_both_wrong": 652 + }, + { + "budget": 0.18, + "threshold": -2.659658908843994, + "operator": "ge", + "actual_fraction": 0.1799845386183147, + "metric": 0.7886710239651417, + "rescued": 793, + "harmed": 247, + "missed_rescue": 1171, + "avoided_harm": 924, + "selected_both_correct": 845, + "selected_both_wrong": 676 + }, + { + "budget": 0.19, + "threshold": -2.7833452224731445, + "operator": "ge", + "actual_fraction": 0.18996415770609318, + "metric": 0.7901468831260102, + "rescued": 833, + "harmed": 266, + "missed_rescue": 1131, + "avoided_harm": 905, + "selected_both_correct": 904, + "selected_both_wrong": 700 + }, + { + "budget": 0.2, + "threshold": -2.928896903991699, + "operator": "ge", + "actual_fraction": 0.19994377679387168, + "metric": 0.7914119052638977, + "rescued": 870, + "harmed": 285, + "missed_rescue": 1094, + "avoided_harm": 886, + "selected_both_correct": 974, + "selected_both_wrong": 716 + }, + { + "budget": 0.21, + "threshold": -3.0729546546936035, + "operator": "ge", + "actual_fraction": 0.20999367488931056, + "metric": 0.7926066483941246, + "rescued": 905, + "harmed": 303, + "missed_rescue": 1059, + "avoided_harm": 868, + "selected_both_correct": 1051, + "selected_both_wrong": 729 + }, + { + "budget": 0.22, + "threshold": -3.207036018371582, + "operator": "ge", + "actual_fraction": 0.21997329397708903, + "metric": 0.792817485417106, + "rescued": 925, + "harmed": 320, + "missed_rescue": 1039, + "avoided_harm": 851, + "selected_both_correct": 1140, + "selected_both_wrong": 745 + }, + { + "budget": 0.23, + "threshold": -3.3137195110321045, + "operator": "ge", + "actual_fraction": 0.22995291306486754, + "metric": 0.794363623585635, + "rescued": 957, + "harmed": 330, + "missed_rescue": 1007, + "avoided_harm": 841, + "selected_both_correct": 1218, + "selected_both_wrong": 767 + }, + { + "budget": 0.24, + "threshold": -3.4117960929870605, + "operator": "ge", + "actual_fraction": 0.239932532152646, + "metric": 0.7959800407618245, + "rescued": 989, + "harmed": 339, + "missed_rescue": 975, + "avoided_harm": 832, + "selected_both_correct": 1299, + "selected_both_wrong": 787 + }, + { + "budget": 0.25, + "threshold": -3.505338191986084, + "operator": "ge", + "actual_fraction": 0.2499824302480849, + "metric": 0.7963314358001266, + "rescued": 1018, + "harmed": 363, + "missed_rescue": 946, + "avoided_harm": 808, + "selected_both_correct": 1374, + "selected_both_wrong": 802 + }, + { + "budget": 0.26, + "threshold": -3.594294548034668, + "operator": "ge", + "actual_fraction": 0.25996204933586337, + "metric": 0.7976667369456744, + "rescued": 1044, + "harmed": 370, + "missed_rescue": 920, + "avoided_harm": 801, + "selected_both_correct": 1463, + "selected_both_wrong": 822 + }, + { + "budget": 0.27, + "threshold": -3.6820778846740723, + "operator": "ge", + "actual_fraction": 0.26994166842364187, + "metric": 0.7978072949609951, + "rescued": 1066, + "harmed": 390, + "missed_rescue": 898, + "avoided_harm": 781, + "selected_both_correct": 1552, + "selected_both_wrong": 833 + }, + { + "budget": 0.28, + "threshold": -3.7688918113708496, + "operator": "ge", + "actual_fraction": 0.27999156651908075, + "metric": 0.7990723170988826, + "rescued": 1096, + "harmed": 402, + "missed_rescue": 868, + "avoided_harm": 769, + "selected_both_correct": 1637, + "selected_both_wrong": 849 + }, + { + "budget": 0.29, + "threshold": -3.8456528186798096, + "operator": "ge", + "actual_fraction": 0.28997118560685925, + "metric": 0.7996345491601659, + "rescued": 1115, + "harmed": 413, + "missed_rescue": 849, + "avoided_harm": 758, + "selected_both_correct": 1736, + "selected_both_wrong": 862 + }, + { + "budget": 0.3, + "threshold": -3.9210715293884277, + "operator": "ge", + "actual_fraction": 0.2999508046946377, + "metric": 0.799845386183147, + "rescued": 1134, + "harmed": 429, + "missed_rescue": 830, + "avoided_harm": 742, + "selected_both_correct": 1830, + "selected_both_wrong": 875 + }, + { + "budget": 0.31, + "threshold": -4.0025529861450195, + "operator": "ge", + "actual_fraction": 0.3099304237824162, + "metric": 0.7997751071754867, + "rescued": 1147, + "harmed": 443, + "missed_rescue": 817, + "avoided_harm": 728, + "selected_both_correct": 1933, + "selected_both_wrong": 887 + }, + { + "budget": 0.32, + "threshold": -4.074873924255371, + "operator": "ge", + "actual_fraction": 0.3199803218778551, + "metric": 0.8008995712980532, + "rescued": 1169, + "harmed": 449, + "missed_rescue": 795, + "avoided_harm": 722, + "selected_both_correct": 2035, + "selected_both_wrong": 900 + }, + { + "budget": 0.33, + "threshold": -4.144405841827393, + "operator": "ge", + "actual_fraction": 0.3299599409656336, + "metric": 0.8016726403823178, + "rescued": 1192, + "harmed": 461, + "missed_rescue": 772, + "avoided_harm": 710, + "selected_both_correct": 2134, + "selected_both_wrong": 908 + }, + { + "budget": 0.34, + "threshold": -4.213827610015869, + "operator": "ge", + "actual_fraction": 0.339939560053412, + "metric": 0.8027971045048844, + "rescued": 1215, + "harmed": 468, + "missed_rescue": 749, + "avoided_harm": 703, + "selected_both_correct": 2231, + "selected_both_wrong": 923 + }, + { + "budget": 0.35, + "threshold": -4.29035758972168, + "operator": "ge", + "actual_fraction": 0.34998945814885096, + "metric": 0.8037810106121301, + "rescued": 1235, + "harmed": 474, + "missed_rescue": 729, + "avoided_harm": 697, + "selected_both_correct": 2335, + "selected_both_wrong": 936 + }, + { + "budget": 0.36, + "threshold": -4.3551530838012695, + "operator": "ge", + "actual_fraction": 0.3599690772366294, + "metric": 0.8041324056504322, + "rescued": 1253, + "harmed": 487, + "missed_rescue": 711, + "avoided_harm": 684, + "selected_both_correct": 2435, + "selected_both_wrong": 947 + }, + { + "budget": 0.37, + "threshold": -4.414309978485107, + "operator": "ge", + "actual_fraction": 0.3699486963244079, + "metric": 0.8043432426734135, + "rescued": 1272, + "harmed": 503, + "missed_rescue": 692, + "avoided_harm": 668, + "selected_both_correct": 2530, + "selected_both_wrong": 959 + }, + { + "budget": 0.38, + "threshold": -4.483922481536865, + "operator": "ge", + "actual_fraction": 0.3799985944198468, + "metric": 0.8037810106121301, + "rescued": 1285, + "harmed": 524, + "missed_rescue": 679, + "avoided_harm": 647, + "selected_both_correct": 2631, + "selected_both_wrong": 967 + }, + { + "budget": 0.39, + "threshold": -4.539813041687012, + "operator": "ge", + "actual_fraction": 0.3899782135076253, + "metric": 0.8041324056504322, + "rescued": 1300, + "harmed": 534, + "missed_rescue": 664, + "avoided_harm": 637, + "selected_both_correct": 2738, + "selected_both_wrong": 977 + }, + { + "budget": 0.4, + "threshold": -4.609446048736572, + "operator": "ge", + "actual_fraction": 0.39995783259540374, + "metric": 0.8051865907653384, + "rescued": 1323, + "harmed": 542, + "missed_rescue": 641, + "avoided_harm": 629, + "selected_both_correct": 2839, + "selected_both_wrong": 987 + }, + { + "budget": 0.41, + "threshold": -4.663720607757568, + "operator": "ge", + "actual_fraction": 0.40993745168318224, + "metric": 0.8053271487806593, + "rescued": 1336, + "harmed": 553, + "missed_rescue": 628, + "avoided_harm": 618, + "selected_both_correct": 2941, + "selected_both_wrong": 1003 + }, + { + "budget": 0.42, + "threshold": -4.7261199951171875, + "operator": "ge", + "actual_fraction": 0.4199873497786211, + "metric": 0.8050460327500175, + "rescued": 1348, + "harmed": 569, + "missed_rescue": 616, + "avoided_harm": 602, + "selected_both_correct": 3045, + "selected_both_wrong": 1014 + }, + { + "budget": 0.43, + "threshold": -4.786887168884277, + "operator": "ge", + "actual_fraction": 0.4299669688663996, + "metric": 0.805116311757678, + "rescued": 1357, + "harmed": 577, + "missed_rescue": 607, + "avoided_harm": 594, + "selected_both_correct": 3159, + "selected_both_wrong": 1025 + }, + { + "budget": 0.44, + "threshold": -4.843598365783691, + "operator": "ge", + "actual_fraction": 0.43994658795417807, + "metric": 0.8055379858036404, + "rescued": 1371, + "harmed": 585, + "missed_rescue": 593, + "avoided_harm": 586, + "selected_both_correct": 3266, + "selected_both_wrong": 1038 + }, + { + "budget": 0.45, + "threshold": -4.890982151031494, + "operator": "ge", + "actual_fraction": 0.449996486049617, + "metric": 0.8057488228266216, + "rescued": 1383, + "harmed": 594, + "missed_rescue": 581, + "avoided_harm": 577, + "selected_both_correct": 3378, + "selected_both_wrong": 1048 + }, + { + "budget": 0.46, + "threshold": -4.9397172927856445, + "operator": "ge", + "actual_fraction": 0.45997610513739545, + "metric": 0.8056082648113009, + "rescued": 1392, + "harmed": 605, + "missed_rescue": 572, + "avoided_harm": 566, + "selected_both_correct": 3488, + "selected_both_wrong": 1060 + }, + { + "budget": 0.47, + "threshold": -4.991722583770752, + "operator": "ge", + "actual_fraction": 0.46995572422517395, + "metric": 0.8059596598496029, + "rescued": 1407, + "harmed": 615, + "missed_rescue": 557, + "avoided_harm": 556, + "selected_both_correct": 3593, + "selected_both_wrong": 1072 + }, + { + "budget": 0.48, + "threshold": -5.048281669616699, + "operator": "ge", + "actual_fraction": 0.4799353433129524, + "metric": 0.8059596598496029, + "rescued": 1420, + "harmed": 628, + "missed_rescue": 544, + "avoided_harm": 543, + "selected_both_correct": 3697, + "selected_both_wrong": 1084 + }, + { + "budget": 0.49, + "threshold": -5.091976165771484, + "operator": "ge", + "actual_fraction": 0.48998524140839134, + "metric": 0.8063813338955654, + "rescued": 1435, + "harmed": 637, + "missed_rescue": 529, + "avoided_harm": 534, + "selected_both_correct": 3803, + "selected_both_wrong": 1097 + }, + { + "budget": 0.5, + "threshold": -5.137824535369873, + "operator": "ge", + "actual_fraction": 0.4999648604961698, + "metric": 0.8070841239721696, + "rescued": 1449, + "harmed": 641, + "missed_rescue": 515, + "avoided_harm": 530, + "selected_both_correct": 3913, + "selected_both_wrong": 1111 + }, + { + "budget": 0.51, + "threshold": -5.183772563934326, + "operator": "ge", + "actual_fraction": 0.5099444795839483, + "metric": 0.8065921709185466, + "rescued": 1454, + "harmed": 653, + "missed_rescue": 510, + "avoided_harm": 518, + "selected_both_correct": 4024, + "selected_both_wrong": 1125 + }, + { + "budget": 0.52, + "threshold": -5.225483417510986, + "operator": "ge", + "actual_fraction": 0.5199943776793872, + "metric": 0.8068732869491883, + "rescued": 1465, + "harmed": 660, + "missed_rescue": 499, + "avoided_harm": 511, + "selected_both_correct": 4140, + "selected_both_wrong": 1134 + }, + { + "budget": 0.53, + "threshold": -5.285849571228027, + "operator": "ge", + "actual_fraction": 0.5299739967671656, + "metric": 0.8071544029798299, + "rescued": 1479, + "harmed": 670, + "missed_rescue": 485, + "avoided_harm": 501, + "selected_both_correct": 4252, + "selected_both_wrong": 1140 + }, + { + "budget": 0.54, + "threshold": -5.33848237991333, + "operator": "ge", + "actual_fraction": 0.5399536158549442, + "metric": 0.807857193056434, + "rescued": 1493, + "harmed": 674, + "missed_rescue": 471, + "avoided_harm": 497, + "selected_both_correct": 4367, + "selected_both_wrong": 1149 + }, + { + "budget": 0.55, + "threshold": -5.383656978607178, + "operator": "ge", + "actual_fraction": 0.5499332349427226, + "metric": 0.8079977510717549, + "rescued": 1506, + "harmed": 685, + "missed_rescue": 458, + "avoided_harm": 486, + "selected_both_correct": 4474, + "selected_both_wrong": 1160 + }, + { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5599831330381615, + "metric": 0.8081383090870757, + "rescued": 1513, + "harmed": 690, + "missed_rescue": 451, + "avoided_harm": 481, + "selected_both_correct": 4599, + "selected_both_wrong": 1166 + }, + { + "budget": 0.57, + "threshold": -5.478402614593506, + "operator": "ge", + "actual_fraction": 0.5699627521259399, + "metric": 0.8079274720640944, + "rescued": 1523, + "harmed": 703, + "missed_rescue": 441, + "avoided_harm": 468, + "selected_both_correct": 4705, + "selected_both_wrong": 1179 + }, + { + "budget": 0.58, + "threshold": -5.5247578620910645, + "operator": "gt", + "actual_fraction": 0.5798720922060581, + "metric": 0.8079274720640944, + "rescued": 1532, + "harmed": 712, + "missed_rescue": 432, + "avoided_harm": 459, + "selected_both_correct": 4819, + "selected_both_wrong": 1188 + }, + { + "budget": 0.59, + "threshold": -5.566192626953125, + "operator": "ge", + "actual_fraction": 0.5899922693091574, + "metric": 0.8080680300794153, + "rescued": 1543, + "harmed": 721, + "missed_rescue": 421, + "avoided_harm": 450, + "selected_both_correct": 4933, + "selected_both_wrong": 1198 + }, + { + "budget": 0.6, + "threshold": -5.613105773925781, + "operator": "ge", + "actual_fraction": 0.5999718883969358, + "metric": 0.807857193056434, + "rescued": 1551, + "harmed": 732, + "missed_rescue": 413, + "avoided_harm": 439, + "selected_both_correct": 5051, + "selected_both_wrong": 1203 + }, + { + "budget": 0.61, + "threshold": -5.660335540771484, + "operator": "ge", + "actual_fraction": 0.6099515074847143, + "metric": 0.8075057980181319, + "rescued": 1557, + "harmed": 743, + "missed_rescue": 407, + "avoided_harm": 428, + "selected_both_correct": 5166, + "selected_both_wrong": 1213 + }, + { + "budget": 0.62, + "threshold": -5.713016986846924, + "operator": "ge", + "actual_fraction": 0.6199311265724928, + "metric": 0.8070841239721696, + "rescued": 1560, + "harmed": 752, + "missed_rescue": 404, + "avoided_harm": 419, + "selected_both_correct": 5282, + "selected_both_wrong": 1227 + }, + { + "budget": 0.63, + "threshold": -5.765469074249268, + "operator": "ge", + "actual_fraction": 0.6299810246679317, + "metric": 0.8065921709185466, + "rescued": 1568, + "harmed": 767, + "missed_rescue": 396, + "avoided_harm": 404, + "selected_both_correct": 5389, + "selected_both_wrong": 1240 + }, + { + "budget": 0.64, + "threshold": -5.808561325073242, + "operator": "ge", + "actual_fraction": 0.6399606437557102, + "metric": 0.8065218919108862, + "rescued": 1579, + "harmed": 779, + "missed_rescue": 385, + "avoided_harm": 392, + "selected_both_correct": 5501, + "selected_both_wrong": 1247 + }, + { + "budget": 0.65, + "threshold": -5.852151870727539, + "operator": "ge", + "actual_fraction": 0.6499402628434886, + "metric": 0.8058893808419425, + "rescued": 1584, + "harmed": 793, + "missed_rescue": 380, + "avoided_harm": 378, + "selected_both_correct": 5618, + "selected_both_wrong": 1253 + }, + { + "budget": 0.66, + "threshold": -5.899008274078369, + "operator": "ge", + "actual_fraction": 0.6599901609389276, + "metric": 0.8056785438189613, + "rescued": 1593, + "harmed": 805, + "missed_rescue": 371, + "avoided_harm": 366, + "selected_both_correct": 5729, + "selected_both_wrong": 1264 + }, + { + "budget": 0.67, + "threshold": -5.944814682006836, + "operator": "ge", + "actual_fraction": 0.669969780026706, + "metric": 0.8057488228266216, + "rescued": 1603, + "harmed": 814, + "missed_rescue": 361, + "avoided_harm": 357, + "selected_both_correct": 5849, + "selected_both_wrong": 1267 + }, + { + "budget": 0.68, + "threshold": -5.992681980133057, + "operator": "ge", + "actual_fraction": 0.6799493991144845, + "metric": 0.8060299388572634, + "rescued": 1620, + "harmed": 827, + "missed_rescue": 344, + "avoided_harm": 344, + "selected_both_correct": 5954, + "selected_both_wrong": 1274 + }, + { + "budget": 0.69, + "threshold": -6.03839635848999, + "operator": "ge", + "actual_fraction": 0.6899992972099234, + "metric": 0.8056082648113009, + "rescued": 1629, + "harmed": 842, + "missed_rescue": 335, + "avoided_harm": 329, + "selected_both_correct": 6066, + "selected_both_wrong": 1281 + }, + { + "budget": 0.7, + "threshold": -6.091725826263428, + "operator": "ge", + "actual_fraction": 0.6999789162977019, + "metric": 0.8055379858036404, + "rescued": 1635, + "harmed": 849, + "missed_rescue": 329, + "avoided_harm": 322, + "selected_both_correct": 6187, + "selected_both_wrong": 1289 + }, + { + "budget": 0.71, + "threshold": -6.139003276824951, + "operator": "ge", + "actual_fraction": 0.7099585353854804, + "metric": 0.8053974277883196, + "rescued": 1644, + "harmed": 860, + "missed_rescue": 320, + "avoided_harm": 311, + "selected_both_correct": 6302, + "selected_both_wrong": 1296 + }, + { + "budget": 0.72, + "threshold": -6.183269500732422, + "operator": "ge", + "actual_fraction": 0.7199381544732588, + "metric": 0.80546770679598, + "rescued": 1655, + "harmed": 870, + "missed_rescue": 309, + "avoided_harm": 301, + "selected_both_correct": 6417, + "selected_both_wrong": 1302 + }, + { + "budget": 0.73, + "threshold": -6.240170955657959, + "operator": "ge", + "actual_fraction": 0.7299880525686977, + "metric": 0.8055379858036404, + "rescued": 1664, + "harmed": 878, + "missed_rescue": 300, + "avoided_harm": 293, + "selected_both_correct": 6531, + "selected_both_wrong": 1314 + }, + { + "budget": 0.74, + "threshold": -6.2856950759887695, + "operator": "ge", + "actual_fraction": 0.7399676716564763, + "metric": 0.8058191018342821, + "rescued": 1676, + "harmed": 886, + "missed_rescue": 288, + "avoided_harm": 285, + "selected_both_correct": 6650, + "selected_both_wrong": 1317 + }, + { + "budget": 0.75, + "threshold": -6.338753700256348, + "operator": "ge", + "actual_fraction": 0.7499472907442547, + "metric": 0.8057488228266216, + "rescued": 1684, + "harmed": 895, + "missed_rescue": 280, + "avoided_harm": 276, + "selected_both_correct": 6764, + "selected_both_wrong": 1328 + }, + { + "budget": 0.76, + "threshold": -6.385316848754883, + "operator": "ge", + "actual_fraction": 0.7599971888396936, + "metric": 0.8058893808419425, + "rescued": 1695, + "harmed": 904, + "missed_rescue": 269, + "avoided_harm": 267, + "selected_both_correct": 6872, + "selected_both_wrong": 1343 + }, + { + "budget": 0.77, + "threshold": -6.438819408416748, + "operator": "ge", + "actual_fraction": 0.769976807927472, + "metric": 0.8061704968725841, + "rescued": 1707, + "harmed": 912, + "missed_rescue": 257, + "avoided_harm": 259, + "selected_both_correct": 6987, + "selected_both_wrong": 1350 + }, + { + "budget": 0.78, + "threshold": -6.489089012145996, + "operator": "ge", + "actual_fraction": 0.7799564270152506, + "metric": 0.8062407758802446, + "rescued": 1719, + "harmed": 923, + "missed_rescue": 245, + "avoided_harm": 248, + "selected_both_correct": 7096, + "selected_both_wrong": 1360 + }, + { + "budget": 0.79, + "threshold": -6.554056644439697, + "operator": "ge", + "actual_fraction": 0.789936046103029, + "metric": 0.8066624499262071, + "rescued": 1729, + "harmed": 927, + "missed_rescue": 235, + "avoided_harm": 244, + "selected_both_correct": 7218, + "selected_both_wrong": 1366 + }, + { + "budget": 0.8, + "threshold": -6.614213943481445, + "operator": "ge", + "actual_fraction": 0.7999859441984679, + "metric": 0.8068030079415278, + "rescued": 1740, + "harmed": 936, + "missed_rescue": 224, + "avoided_harm": 235, + "selected_both_correct": 7333, + "selected_both_wrong": 1374 + }, + { + "budget": 0.81, + "threshold": -6.680136680603027, + "operator": "ge", + "actual_fraction": 0.8099655632862464, + "metric": 0.8072949609951507, + "rescued": 1756, + "harmed": 945, + "missed_rescue": 208, + "avoided_harm": 226, + "selected_both_correct": 7439, + "selected_both_wrong": 1385 + }, + { + "budget": 0.82, + "threshold": -6.735454559326172, + "operator": "ge", + "actual_fraction": 0.8199451823740249, + "metric": 0.8072246819874903, + "rescued": 1761, + "harmed": 951, + "missed_rescue": 203, + "avoided_harm": 220, + "selected_both_correct": 7561, + "selected_both_wrong": 1394 + }, + { + "budget": 0.83, + "threshold": -6.790655136108398, + "operator": "ge", + "actual_fraction": 0.8299950804694638, + "metric": 0.8075760770257924, + "rescued": 1770, + "harmed": 955, + "missed_rescue": 194, + "avoided_harm": 216, + "selected_both_correct": 7684, + "selected_both_wrong": 1401 + }, + { + "budget": 0.84, + "threshold": -6.856695175170898, + "operator": "ge", + "actual_fraction": 0.8399746995572422, + "metric": 0.8075760770257924, + "rescued": 1783, + "harmed": 968, + "missed_rescue": 181, + "avoided_harm": 203, + "selected_both_correct": 7793, + "selected_both_wrong": 1408 + }, + { + "budget": 0.85, + "threshold": -6.932329177856445, + "operator": "ge", + "actual_fraction": 0.8499543186450207, + "metric": 0.8075760770257924, + "rescued": 1792, + "harmed": 977, + "missed_rescue": 172, + "avoided_harm": 194, + "selected_both_correct": 7906, + "selected_both_wrong": 1419 + }, + { + "budget": 0.86, + "threshold": -6.999267578125, + "operator": "ge", + "actual_fraction": 0.8599339377327992, + "metric": 0.8077166350411132, + "rescued": 1799, + "harmed": 982, + "missed_rescue": 165, + "avoided_harm": 189, + "selected_both_correct": 8025, + "selected_both_wrong": 1430 + }, + { + "budget": 0.87, + "threshold": -7.081692695617676, + "operator": "ge", + "actual_fraction": 0.8699838358282381, + "metric": 0.8077166350411132, + "rescued": 1807, + "harmed": 990, + "missed_rescue": 157, + "avoided_harm": 181, + "selected_both_correct": 8145, + "selected_both_wrong": 1437 + }, + { + "budget": 0.88, + "threshold": -7.155144214630127, + "operator": "ge", + "actual_fraction": 0.8799634549160166, + "metric": 0.8077869140487737, + "rescued": 1819, + "harmed": 1001, + "missed_rescue": 145, + "avoided_harm": 170, + "selected_both_correct": 8259, + "selected_both_wrong": 1442 + }, + { + "budget": 0.89, + "threshold": -7.227406978607178, + "operator": "ge", + "actual_fraction": 0.889943074003795, + "metric": 0.8076463560334528, + "rescued": 1829, + "harmed": 1013, + "missed_rescue": 135, + "avoided_harm": 158, + "selected_both_correct": 8365, + "selected_both_wrong": 1456 + }, + { + "budget": 0.9, + "threshold": -7.308749198913574, + "operator": "ge", + "actual_fraction": 0.899992972099234, + "metric": 0.8070138449645091, + "rescued": 1837, + "harmed": 1030, + "missed_rescue": 127, + "avoided_harm": 141, + "selected_both_correct": 8472, + "selected_both_wrong": 1467 + }, + { + "budget": 0.91, + "threshold": -7.405246257781982, + "operator": "ge", + "actual_fraction": 0.9099725911870125, + "metric": 0.8070138449645091, + "rescued": 1848, + "harmed": 1041, + "missed_rescue": 116, + "avoided_harm": 130, + "selected_both_correct": 8582, + "selected_both_wrong": 1477 + }, + { + "budget": 0.92, + "threshold": -7.509663105010986, + "operator": "gt", + "actual_fraction": 0.9198819312671305, + "metric": 0.8070138449645091, + "rescued": 1856, + "harmed": 1049, + "missed_rescue": 108, + "avoided_harm": 122, + "selected_both_correct": 8695, + "selected_both_wrong": 1489 + }, + { + "budget": 0.93, + "threshold": -7.625953674316406, + "operator": "ge", + "actual_fraction": 0.9299318293625694, + "metric": 0.8068030079415278, + "rescued": 1865, + "harmed": 1061, + "missed_rescue": 99, + "avoided_harm": 110, + "selected_both_correct": 8805, + "selected_both_wrong": 1501 + }, + { + "budget": 0.94, + "threshold": -7.762190818786621, + "operator": "ge", + "actual_fraction": 0.9399817274580083, + "metric": 0.8068030079415278, + "rescued": 1878, + "harmed": 1074, + "missed_rescue": 86, + "avoided_harm": 97, + "selected_both_correct": 8911, + "selected_both_wrong": 1512 + }, + { + "budget": 0.95, + "threshold": -7.891505241394043, + "operator": "ge", + "actual_fraction": 0.9499613465457868, + "metric": 0.8068732869491883, + "rescued": 1894, + "harmed": 1089, + "missed_rescue": 70, + "avoided_harm": 82, + "selected_both_correct": 9012, + "selected_both_wrong": 1522 + }, + { + "budget": 0.96, + "threshold": -8.049057960510254, + "operator": "ge", + "actual_fraction": 0.9599409656335652, + "metric": 0.8068732869491883, + "rescued": 1913, + "harmed": 1108, + "missed_rescue": 51, + "avoided_harm": 63, + "selected_both_correct": 9108, + "selected_both_wrong": 1530 + }, + { + "budget": 0.97, + "threshold": -8.248269081115723, + "operator": "ge", + "actual_fraction": 0.9699908637290041, + "metric": 0.8071544029798299, + "rescued": 1927, + "harmed": 1118, + "missed_rescue": 37, + "avoided_harm": 53, + "selected_both_correct": 9215, + "selected_both_wrong": 1542 + }, + { + "budget": 0.98, + "threshold": -8.543853759765625, + "operator": "ge", + "actual_fraction": 0.9799704828167827, + "metric": 0.8068030079415278, + "rescued": 1939, + "harmed": 1135, + "missed_rescue": 25, + "avoided_harm": 36, + "selected_both_correct": 9310, + "selected_both_wrong": 1560 + }, + { + "budget": 0.99, + "threshold": -8.96818733215332, + "operator": "ge", + "actual_fraction": 0.9899501019045611, + "metric": 0.8065218919108862, + "rescued": 1948, + "harmed": 1148, + "missed_rescue": 16, + "avoided_harm": 23, + "selected_both_correct": 9420, + "selected_both_wrong": 1570 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.8060299388572634, + "rescued": 1964, + "harmed": 1171, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9505, + "selected_both_wrong": 1589 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.6695119530385957, + "rescued": 0, + "harmed": 0, + "missed_rescue": 5423, + "avoided_harm": 2771, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 5.873929023742676, + "operator": "ge", + "actual_fraction": 0.04701812097666109, + "metric": 0.6878314380512152, + "rescued": 655, + "harmed": 9, + "missed_rescue": 4768, + "avoided_harm": 2762, + "selected_both_correct": 18, + "selected_both_wrong": 976 + }, + { + "budget": 0.02, + "threshold": 3.9572336673736572, + "operator": "ge", + "actual_fraction": 0.06706746448118424, + "metric": 0.6980404389870403, + "rescued": 1022, + "harmed": 16, + "missed_rescue": 4401, + "avoided_harm": 2755, + "selected_both_correct": 64, + "selected_both_wrong": 1263 + }, + { + "budget": 0.03, + "threshold": 2.499727725982666, + "operator": "ge", + "actual_fraction": 0.08720188299350594, + "metric": 0.7065195814309616, + "rescued": 1340, + "harmed": 35, + "missed_rescue": 4083, + "avoided_harm": 2736, + "selected_both_correct": 157, + "selected_both_wrong": 1543 + }, + { + "budget": 0.04, + "threshold": 1.2223308086395264, + "operator": "ge", + "actual_fraction": 0.1055780846779911, + "metric": 0.712389756969061, + "rescued": 1582, + "harmed": 70, + "missed_rescue": 3841, + "avoided_harm": 2701, + "selected_both_correct": 311, + "selected_both_wrong": 1760 + }, + { + "budget": 0.05, + "threshold": 0.36731216311454773, + "operator": "ge", + "actual_fraction": 0.11998411933187761, + "metric": 0.7164449990074583, + "rescued": 1757, + "harmed": 102, + "missed_rescue": 3666, + "avoided_harm": 2669, + "selected_both_correct": 475, + "selected_both_wrong": 1897 + }, + { + "budget": 0.06, + "threshold": -0.14144867658615112, + "operator": "ge", + "actual_fraction": 0.13243342880639764, + "metric": 0.7199047159912656, + "rescued": 1911, + "harmed": 134, + "missed_rescue": 3512, + "avoided_harm": 2637, + "selected_both_correct": 596, + "selected_both_wrong": 2029 + }, + { + "budget": 0.07, + "threshold": -0.42117512226104736, + "operator": "ge", + "actual_fraction": 0.14235884638289426, + "metric": 0.7222017412018262, + "rescued": 2025, + "harmed": 167, + "missed_rescue": 3398, + "avoided_harm": 2604, + "selected_both_correct": 705, + "selected_both_wrong": 2123 + }, + { + "budget": 0.08, + "threshold": -0.6775990128517151, + "operator": "ge", + "actual_fraction": 0.1511215721861441, + "metric": 0.7239032413577972, + "rescued": 2118, + "harmed": 200, + "missed_rescue": 3305, + "avoided_harm": 2571, + "selected_both_correct": 789, + "selected_both_wrong": 2222 + }, + { + "budget": 0.09, + "threshold": -0.9111056327819824, + "operator": "ge", + "actual_fraction": 0.15957235629413266, + "metric": 0.7263136999120892, + "rescued": 2226, + "harmed": 223, + "missed_rescue": 3197, + "avoided_harm": 2548, + "selected_both_correct": 857, + "selected_both_wrong": 2321 + }, + { + "budget": 0.1, + "threshold": -1.1332147121429443, + "operator": "ge", + "actual_fraction": 0.16822164875365114, + "metric": 0.7290077418257097, + "rescued": 2339, + "harmed": 241, + "missed_rescue": 3084, + "avoided_harm": 2530, + "selected_both_correct": 914, + "selected_both_wrong": 2438 + }, + { + "budget": 0.11, + "threshold": -1.3457791805267334, + "operator": "ge", + "actual_fraction": 0.17695601622096815, + "metric": 0.7316450670674645, + "rescued": 2454, + "harmed": 263, + "missed_rescue": 2969, + "avoided_harm": 2508, + "selected_both_correct": 997, + "selected_both_wrong": 2526 + }, + { + "budget": 0.12, + "threshold": -1.5916669368743896, + "operator": "ge", + "actual_fraction": 0.18855457561750277, + "metric": 0.7345943340044806, + "rescued": 2587, + "harmed": 292, + "missed_rescue": 2836, + "avoided_harm": 2479, + "selected_both_correct": 1103, + "selected_both_wrong": 2667 + }, + { + "budget": 0.13, + "threshold": -1.7836966514587402, + "operator": "ge", + "actual_fraction": 0.19751580977228256, + "metric": 0.7366361341916456, + "rescued": 2685, + "harmed": 318, + "missed_rescue": 2738, + "avoided_harm": 2453, + "selected_both_correct": 1187, + "selected_both_wrong": 2775 + }, + { + "budget": 0.14, + "threshold": -1.9795069694519043, + "operator": "ge", + "actual_fraction": 0.20664719394265946, + "metric": 0.7385361426991465, + "rescued": 2776, + "harmed": 342, + "missed_rescue": 2647, + "avoided_harm": 2429, + "selected_both_correct": 1271, + "selected_both_wrong": 2898 + }, + { + "budget": 0.15, + "threshold": -2.1732606887817383, + "operator": "ge", + "actual_fraction": 0.2172532115815444, + "metric": 0.7402376428551173, + "rescued": 2874, + "harmed": 380, + "missed_rescue": 2549, + "avoided_harm": 2391, + "selected_both_correct": 1386, + "selected_both_wrong": 3021 + }, + { + "budget": 0.16, + "threshold": -2.3698818683624268, + "operator": "gt", + "actual_fraction": 0.22717862915804102, + "metric": 0.7420242180188866, + "rescued": 2970, + "harmed": 413, + "missed_rescue": 2453, + "avoided_harm": 2358, + "selected_both_correct": 1489, + "selected_both_wrong": 3139 + }, + { + "budget": 0.17, + "threshold": -2.5250070095062256, + "operator": "ge", + "actual_fraction": 0.2361398633128208, + "metric": 0.7436122848311261, + "rescued": 3053, + "harmed": 440, + "missed_rescue": 2370, + "avoided_harm": 2331, + "selected_both_correct": 1581, + "selected_both_wrong": 3253 + }, + { + "budget": 0.18, + "threshold": -2.659658908843994, + "operator": "ge", + "actual_fraction": 0.24362646399909255, + "metric": 0.7448033349403057, + "rescued": 3124, + "harmed": 469, + "missed_rescue": 2299, + "avoided_harm": 2302, + "selected_both_correct": 1685, + "selected_both_wrong": 3313 + }, + { + "budget": 0.19, + "threshold": -2.7833452224731445, + "operator": "ge", + "actual_fraction": 0.2507727646541701, + "metric": 0.7456540850182911, + "rescued": 3179, + "harmed": 494, + "missed_rescue": 2244, + "avoided_harm": 2277, + "selected_both_correct": 1781, + "selected_both_wrong": 3389 + }, + { + "budget": 0.2, + "threshold": -2.928896903991699, + "operator": "ge", + "actual_fraction": 0.25942205711368854, + "metric": 0.7459943850494853, + "rescued": 3237, + "harmed": 540, + "missed_rescue": 2186, + "avoided_harm": 2231, + "selected_both_correct": 1913, + "selected_both_wrong": 3458 + }, + { + "budget": 0.21, + "threshold": -3.0729546546936035, + "operator": "ge", + "actual_fraction": 0.2690922496667896, + "metric": 0.746419760088478, + "rescued": 3296, + "harmed": 584, + "missed_rescue": 2127, + "avoided_harm": 2187, + "selected_both_correct": 2081, + "selected_both_wrong": 3528 + }, + { + "budget": 0.22, + "threshold": -3.207036018371582, + "operator": "ge", + "actual_fraction": 0.2781385588293679, + "metric": 0.7467600601196722, + "rescued": 3347, + "harmed": 623, + "missed_rescue": 2076, + "avoided_harm": 2148, + "selected_both_correct": 2230, + "selected_both_wrong": 3608 + }, + { + "budget": 0.23, + "threshold": -3.3137195110321045, + "operator": "ge", + "actual_fraction": 0.2868445679607521, + "metric": 0.7473555851742619, + "rescued": 3396, + "harmed": 651, + "missed_rescue": 2027, + "avoided_harm": 2120, + "selected_both_correct": 2391, + "selected_both_wrong": 3677 + }, + { + "budget": 0.24, + "threshold": -3.4117960929870605, + "operator": "ge", + "actual_fraction": 0.29430281031109096, + "metric": 0.7480645435725831, + "rescued": 3446, + "harmed": 676, + "missed_rescue": 1977, + "avoided_harm": 2095, + "selected_both_correct": 2535, + "selected_both_wrong": 3721 + }, + { + "budget": 0.25, + "threshold": -3.505338191986084, + "operator": "ge", + "actual_fraction": 0.30176105266142983, + "metric": 0.748092901908516, + "rescued": 3479, + "harmed": 708, + "missed_rescue": 1944, + "avoided_harm": 2063, + "selected_both_correct": 2673, + "selected_both_wrong": 3781 + }, + { + "budget": 0.26, + "threshold": -3.594294548034668, + "operator": "ge", + "actual_fraction": 0.30995661174602274, + "metric": 0.7486033519553073, + "rescued": 3520, + "harmed": 731, + "missed_rescue": 1903, + "avoided_harm": 2040, + "selected_both_correct": 2847, + "selected_both_wrong": 3832 + }, + { + "budget": 0.27, + "threshold": -3.6820778846740723, + "operator": "ge", + "actual_fraction": 0.3175850041119587, + "metric": 0.7494541020332927, + "rescued": 3569, + "harmed": 750, + "missed_rescue": 1854, + "avoided_harm": 2021, + "selected_both_correct": 3000, + "selected_both_wrong": 3880 + }, + { + "budget": 0.28, + "threshold": -3.7688918113708496, + "operator": "ge", + "actual_fraction": 0.3262342965714772, + "metric": 0.7501630604316138, + "rescued": 3623, + "harmed": 779, + "missed_rescue": 1800, + "avoided_harm": 1992, + "selected_both_correct": 3177, + "selected_both_wrong": 3925 + }, + { + "budget": 0.29, + "threshold": -3.8456528186798096, + "operator": "ge", + "actual_fraction": 0.3324731304767036, + "metric": 0.7505600771346738, + "rescued": 3655, + "harmed": 797, + "missed_rescue": 1768, + "avoided_harm": 1974, + "selected_both_correct": 3301, + "selected_both_wrong": 3971 + }, + { + "budget": 0.3, + "threshold": -3.9210715293884277, + "operator": "ge", + "actual_fraction": 0.3399313728270425, + "metric": 0.7509570938377336, + "rescued": 3698, + "harmed": 826, + "missed_rescue": 1725, + "avoided_harm": 1945, + "selected_both_correct": 3454, + "selected_both_wrong": 4009 + }, + { + "budget": 0.31, + "threshold": -4.0025529861450195, + "operator": "ge", + "actual_fraction": 0.34824036525536683, + "metric": 0.7511839605251964, + "rescued": 3732, + "harmed": 852, + "missed_rescue": 1691, + "avoided_harm": 1919, + "selected_both_correct": 3635, + "selected_both_wrong": 4061 + }, + { + "budget": 0.32, + "threshold": -4.074873924255371, + "operator": "ge", + "actual_fraction": 0.3557553242775714, + "metric": 0.751269035532995, + "rescued": 3761, + "harmed": 878, + "missed_rescue": 1662, + "avoided_harm": 1893, + "selected_both_correct": 3798, + "selected_both_wrong": 4108 + }, + { + "budget": 0.33, + "threshold": -4.144405841827393, + "operator": "ge", + "actual_fraction": 0.3627598332529847, + "metric": 0.7511272438533307, + "rescued": 3779, + "harmed": 901, + "missed_rescue": 1644, + "avoided_harm": 1870, + "selected_both_correct": 3962, + "selected_both_wrong": 4150 + }, + { + "budget": 0.34, + "threshold": -4.213827610015869, + "operator": "ge", + "actual_fraction": 0.37013300059552506, + "metric": 0.7512123188611293, + "rescued": 3806, + "harmed": 925, + "missed_rescue": 1617, + "avoided_harm": 1846, + "selected_both_correct": 4131, + "selected_both_wrong": 4190 + }, + { + "budget": 0.35, + "threshold": -4.29035758972168, + "operator": "ge", + "actual_fraction": 0.3783569180160508, + "metric": 0.751439185548592, + "rescued": 3839, + "harmed": 950, + "missed_rescue": 1584, + "avoided_harm": 1821, + "selected_both_correct": 4321, + "selected_both_wrong": 4232 + }, + { + "budget": 0.36, + "threshold": -4.3551530838012695, + "operator": "ge", + "actual_fraction": 0.3854748603351955, + "metric": 0.7517511272438533, + "rescued": 3869, + "harmed": 969, + "missed_rescue": 1554, + "avoided_harm": 1802, + "selected_both_correct": 4491, + "selected_both_wrong": 4264 + }, + { + "budget": 0.37, + "threshold": -4.414309978485107, + "operator": "ge", + "actual_fraction": 0.39279131100587017, + "metric": 0.752176502282846, + "rescued": 3898, + "harmed": 983, + "missed_rescue": 1525, + "avoided_harm": 1788, + "selected_both_correct": 4671, + "selected_both_wrong": 4299 + }, + { + "budget": 0.38, + "threshold": -4.483922481536865, + "operator": "ge", + "actual_fraction": 0.40013612001247767, + "metric": 0.7522332189547117, + "rescued": 3921, + "harmed": 1004, + "missed_rescue": 1502, + "avoided_harm": 1767, + "selected_both_correct": 4853, + "selected_both_wrong": 4332 + }, + { + "budget": 0.39, + "threshold": -4.539813041687012, + "operator": "ge", + "actual_fraction": 0.406573462269234, + "metric": 0.7525735189859059, + "rescued": 3947, + "harmed": 1018, + "missed_rescue": 1476, + "avoided_harm": 1753, + "selected_both_correct": 5011, + "selected_both_wrong": 4361 + }, + { + "budget": 0.4, + "threshold": -4.609446048736572, + "operator": "ge", + "actual_fraction": 0.4153929047443496, + "metric": 0.7528003856733687, + "rescued": 3980, + "harmed": 1043, + "missed_rescue": 1443, + "avoided_harm": 1728, + "selected_both_correct": 5214, + "selected_both_wrong": 4411 + }, + { + "budget": 0.41, + "threshold": -4.663720607757568, + "operator": "ge", + "actual_fraction": 0.42151830530584466, + "metric": 0.75311232736863, + "rescued": 4010, + "harmed": 1062, + "missed_rescue": 1413, + "avoided_harm": 1709, + "selected_both_correct": 5358, + "selected_both_wrong": 4434 + }, + { + "budget": 0.42, + "threshold": -4.7261199951171875, + "operator": "ge", + "actual_fraction": 0.42920341434364634, + "metric": 0.7535093440716899, + "rescued": 4045, + "harmed": 1083, + "missed_rescue": 1378, + "avoided_harm": 1688, + "selected_both_correct": 5538, + "selected_both_wrong": 4469 + }, + { + "budget": 0.43, + "threshold": -4.786887168884277, + "operator": "ge", + "actual_fraction": 0.4367750900377166, + "metric": 0.7537362107591526, + "rescued": 4070, + "harmed": 1100, + "missed_rescue": 1353, + "avoided_harm": 1671, + "selected_both_correct": 5721, + "selected_both_wrong": 4511 + }, + { + "budget": 0.44, + "threshold": -4.843598365783691, + "operator": "ge", + "actual_fraction": 0.4450273657941752, + "metric": 0.753679494087287, + "rescued": 4099, + "harmed": 1131, + "missed_rescue": 1324, + "avoided_harm": 1640, + "selected_both_correct": 5910, + "selected_both_wrong": 4553 + }, + { + "budget": 0.45, + "threshold": -4.890982151031494, + "operator": "ge", + "actual_fraction": 0.45143634971499874, + "metric": 0.7537929274310183, + "rescued": 4122, + "harmed": 1150, + "missed_rescue": 1301, + "avoided_harm": 1621, + "selected_both_correct": 6067, + "selected_both_wrong": 4580 + }, + { + "budget": 0.46, + "threshold": -4.9397172927856445, + "operator": "ge", + "actual_fraction": 0.4578736919717551, + "metric": 0.7536227774154213, + "rescued": 4144, + "harmed": 1178, + "missed_rescue": 1279, + "avoided_harm": 1593, + "selected_both_correct": 6218, + "selected_both_wrong": 4606 + }, + { + "budget": 0.47, + "threshold": -4.991722583770752, + "operator": "ge", + "actual_fraction": 0.4659274593766838, + "metric": 0.7537929274310183, + "rescued": 4172, + "harmed": 1200, + "missed_rescue": 1251, + "avoided_harm": 1571, + "selected_both_correct": 6425, + "selected_both_wrong": 4633 + }, + { + "budget": 0.48, + "threshold": -5.048281669616699, + "operator": "ge", + "actual_fraction": 0.4741797351331424, + "metric": 0.7538780024388169, + "rescued": 4202, + "harmed": 1227, + "missed_rescue": 1221, + "avoided_harm": 1544, + "selected_both_correct": 6628, + "selected_both_wrong": 4664 + }, + { + "budget": 0.49, + "threshold": -5.091976165771484, + "operator": "ge", + "actual_fraction": 0.4806454357258316, + "metric": 0.7537362107591526, + "rescued": 4222, + "harmed": 1252, + "missed_rescue": 1201, + "avoided_harm": 1519, + "selected_both_correct": 6781, + "selected_both_wrong": 4694 + }, + { + "budget": 0.5, + "threshold": -5.137824535369873, + "operator": "ge", + "actual_fraction": 0.487451436349715, + "metric": 0.7535093440716899, + "rescued": 4237, + "harmed": 1275, + "missed_rescue": 1186, + "avoided_harm": 1496, + "selected_both_correct": 6963, + "selected_both_wrong": 4714 + }, + { + "budget": 0.51, + "threshold": -5.183772563934326, + "operator": "ge", + "actual_fraction": 0.49482460369225534, + "metric": 0.7537645690950855, + "rescued": 4267, + "harmed": 1296, + "missed_rescue": 1156, + "avoided_harm": 1475, + "selected_both_correct": 7140, + "selected_both_wrong": 4746 + }, + { + "budget": 0.52, + "threshold": -5.225483417510986, + "operator": "ge", + "actual_fraction": 0.501573887644273, + "metric": 0.7535660607435556, + "rescued": 4282, + "harmed": 1318, + "missed_rescue": 1141, + "avoided_harm": 1453, + "selected_both_correct": 7308, + "selected_both_wrong": 4779 + }, + { + "budget": 0.53, + "threshold": -5.285849571228027, + "operator": "ge", + "actual_fraction": 0.5105351217990528, + "metric": 0.7532257607123614, + "rescued": 4298, + "harmed": 1346, + "missed_rescue": 1125, + "avoided_harm": 1425, + "selected_both_correct": 7555, + "selected_both_wrong": 4804 + }, + { + "budget": 0.54, + "threshold": -5.33848237991333, + "operator": "ge", + "actual_fraction": 0.519241130930437, + "metric": 0.7530556106967643, + "rescued": 4319, + "harmed": 1373, + "missed_rescue": 1104, + "avoided_harm": 1398, + "selected_both_correct": 7769, + "selected_both_wrong": 4849 + }, + { + "budget": 0.55, + "threshold": -5.383656978607178, + "operator": "ge", + "actual_fraction": 0.5268128066245072, + "metric": 0.7528571023452344, + "rescued": 4337, + "harmed": 1398, + "missed_rescue": 1086, + "avoided_harm": 1373, + "selected_both_correct": 7963, + "selected_both_wrong": 4879 + }, + { + "budget": 0.56, + "threshold": -5.435944080352783, + "operator": "ge", + "actual_fraction": 0.5355755324277571, + "metric": 0.7528287440093016, + "rescued": 4361, + "harmed": 1423, + "missed_rescue": 1062, + "avoided_harm": 1348, + "selected_both_correct": 8197, + "selected_both_wrong": 4905 + }, + { + "budget": 0.57, + "threshold": -5.478402614593506, + "operator": "ge", + "actual_fraction": 0.5419561580126478, + "metric": 0.753282477384227, + "rescued": 4390, + "harmed": 1436, + "missed_rescue": 1033, + "avoided_harm": 1335, + "selected_both_correct": 8356, + "selected_both_wrong": 4929 + }, + { + "budget": 0.58, + "threshold": -5.5247578620910645, + "operator": "gt", + "actual_fraction": 0.5495845503785838, + "metric": 0.7528287440093016, + "rescued": 4406, + "harmed": 1468, + "missed_rescue": 1017, + "avoided_harm": 1303, + "selected_both_correct": 8545, + "selected_both_wrong": 4961 + }, + { + "budget": 0.59, + "threshold": -5.566192626953125, + "operator": "ge", + "actual_fraction": 0.5562771176587358, + "metric": 0.7527153106655702, + "rescued": 4430, + "harmed": 1496, + "missed_rescue": 993, + "avoided_harm": 1275, + "selected_both_correct": 8706, + "selected_both_wrong": 4984 + }, + { + "budget": 0.6, + "threshold": -5.613105773925781, + "operator": "ge", + "actual_fraction": 0.564047301704336, + "metric": 0.7526585939937045, + "rescued": 4452, + "harmed": 1520, + "missed_rescue": 971, + "avoided_harm": 1251, + "selected_both_correct": 8905, + "selected_both_wrong": 5013 + }, + { + "budget": 0.61, + "threshold": -5.660335540771484, + "operator": "ge", + "actual_fraction": 0.5725831608201231, + "metric": 0.7528287440093016, + "rescued": 4481, + "harmed": 1543, + "missed_rescue": 942, + "avoided_harm": 1228, + "selected_both_correct": 9128, + "selected_both_wrong": 5039 + }, + { + "budget": 0.62, + "threshold": -5.713016986846924, + "operator": "ge", + "actual_fraction": 0.5818846950060971, + "metric": 0.7528571023452344, + "rescued": 4511, + "harmed": 1572, + "missed_rescue": 912, + "avoided_harm": 1199, + "selected_both_correct": 9371, + "selected_both_wrong": 5065 + }, + { + "budget": 0.63, + "threshold": -5.765469074249268, + "operator": "ge", + "actual_fraction": 0.5903638374500184, + "metric": 0.7528571023452344, + "rescued": 4535, + "harmed": 1596, + "missed_rescue": 888, + "avoided_harm": 1175, + "selected_both_correct": 9596, + "selected_both_wrong": 5091 + }, + { + "budget": 0.64, + "threshold": -5.808561325073242, + "operator": "ge", + "actual_fraction": 0.5973683464254318, + "metric": 0.7526869523296373, + "rescued": 4548, + "harmed": 1615, + "missed_rescue": 875, + "avoided_harm": 1156, + "selected_both_correct": 9794, + "selected_both_wrong": 5108 + }, + { + "budget": 0.65, + "threshold": -5.852151870727539, + "operator": "ge", + "actual_fraction": 0.6042594220571137, + "metric": 0.7526018773218388, + "rescued": 4570, + "harmed": 1640, + "missed_rescue": 853, + "avoided_harm": 1131, + "selected_both_correct": 9969, + "selected_both_wrong": 5129 + }, + { + "budget": 0.66, + "threshold": -5.899008274078369, + "operator": "ge", + "actual_fraction": 0.6122848311261095, + "metric": 0.752545160649973, + "rescued": 4589, + "harmed": 1661, + "missed_rescue": 834, + "avoided_harm": 1110, + "selected_both_correct": 10189, + "selected_both_wrong": 5152 + }, + { + "budget": 0.67, + "threshold": -5.944814682006836, + "operator": "ge", + "actual_fraction": 0.6198848651561126, + "metric": 0.7522048606187789, + "rescued": 4603, + "harmed": 1687, + "missed_rescue": 820, + "avoided_harm": 1084, + "selected_both_correct": 10390, + "selected_both_wrong": 5179 + }, + { + "budget": 0.68, + "threshold": -5.992681980133057, + "operator": "ge", + "actual_fraction": 0.6280520659047727, + "metric": 0.752176502282846, + "rescued": 4622, + "harmed": 1707, + "missed_rescue": 801, + "avoided_harm": 1064, + "selected_both_correct": 10612, + "selected_both_wrong": 5206 + }, + { + "budget": 0.69, + "threshold": -6.03839635848999, + "operator": "ge", + "actual_fraction": 0.6363894166690298, + "metric": 0.7522048606187789, + "rescued": 4651, + "harmed": 1735, + "missed_rescue": 772, + "avoided_harm": 1036, + "selected_both_correct": 10825, + "selected_both_wrong": 5230 + }, + { + "budget": 0.7, + "threshold": -6.091725826263428, + "operator": "ge", + "actual_fraction": 0.6444999007458242, + "metric": 0.7521197856109804, + "rescued": 4673, + "harmed": 1760, + "missed_rescue": 750, + "avoided_harm": 1011, + "selected_both_correct": 11035, + "selected_both_wrong": 5259 + }, + { + "budget": 0.71, + "threshold": -6.139003276824951, + "operator": "ge", + "actual_fraction": 0.6527521765022829, + "metric": 0.7519212772594505, + "rescued": 4686, + "harmed": 1780, + "missed_rescue": 737, + "avoided_harm": 991, + "selected_both_correct": 11264, + "selected_both_wrong": 5288 + }, + { + "budget": 0.72, + "threshold": -6.183269500732422, + "operator": "ge", + "actual_fraction": 0.6601253438448231, + "metric": 0.7518362022516518, + "rescued": 4704, + "harmed": 1801, + "missed_rescue": 719, + "avoided_harm": 970, + "selected_both_correct": 11458, + "selected_both_wrong": 5315 + }, + { + "budget": 0.73, + "threshold": -6.240170955657959, + "operator": "ge", + "actual_fraction": 0.6697955363979242, + "metric": 0.7518362022516518, + "rescued": 4732, + "harmed": 1829, + "missed_rescue": 691, + "avoided_harm": 942, + "selected_both_correct": 11714, + "selected_both_wrong": 5344 + }, + { + "budget": 0.74, + "threshold": -6.2856950759887695, + "operator": "ge", + "actual_fraction": 0.6775373621075915, + "metric": 0.7518645605875848, + "rescued": 4754, + "harmed": 1850, + "missed_rescue": 669, + "avoided_harm": 921, + "selected_both_correct": 11918, + "selected_both_wrong": 5370 + }, + { + "budget": 0.75, + "threshold": -6.338753700256348, + "operator": "ge", + "actual_fraction": 0.6870657629810283, + "metric": 0.7512973938689278, + "rescued": 4769, + "harmed": 1885, + "missed_rescue": 654, + "avoided_harm": 886, + "selected_both_correct": 12181, + "selected_both_wrong": 5393 + }, + { + "budget": 0.76, + "threshold": -6.385316848754883, + "operator": "ge", + "actual_fraction": 0.6956299804327482, + "metric": 0.7514108272126592, + "rescued": 4786, + "harmed": 1898, + "missed_rescue": 637, + "avoided_harm": 873, + "selected_both_correct": 12430, + "selected_both_wrong": 5416 + }, + { + "budget": 0.77, + "threshold": -6.438819408416748, + "operator": "ge", + "actual_fraction": 0.7044494229078637, + "metric": 0.7514959022204577, + "rescued": 4805, + "harmed": 1914, + "missed_rescue": 618, + "avoided_harm": 857, + "selected_both_correct": 12668, + "selected_both_wrong": 5454 + }, + { + "budget": 0.78, + "threshold": -6.489089012145996, + "operator": "ge", + "actual_fraction": 0.7129852820236509, + "metric": 0.7510705271814649, + "rescued": 4813, + "harmed": 1937, + "missed_rescue": 610, + "avoided_harm": 834, + "selected_both_correct": 12915, + "selected_both_wrong": 5477 + }, + { + "budget": 0.79, + "threshold": -6.554056644439697, + "operator": "ge", + "actual_fraction": 0.7243853330686555, + "metric": 0.7506735104784051, + "rescued": 4834, + "harmed": 1972, + "missed_rescue": 589, + "avoided_harm": 799, + "selected_both_correct": 13238, + "selected_both_wrong": 5500 + }, + { + "budget": 0.8, + "threshold": -6.614213943481445, + "operator": "ge", + "actual_fraction": 0.7337152255905623, + "metric": 0.7507302271502708, + "rescued": 4858, + "harmed": 1994, + "missed_rescue": 565, + "avoided_harm": 777, + "selected_both_correct": 13488, + "selected_both_wrong": 5533 + }, + { + "budget": 0.81, + "threshold": -6.680136680603027, + "operator": "ge", + "actual_fraction": 0.744377959901313, + "metric": 0.7505033604628081, + "rescued": 4880, + "harmed": 2024, + "missed_rescue": 543, + "avoided_harm": 747, + "selected_both_correct": 13784, + "selected_both_wrong": 5561 + }, + { + "budget": 0.82, + "threshold": -6.735454559326172, + "operator": "ge", + "actual_fraction": 0.7522048606187789, + "metric": 0.7503048521112781, + "rescued": 4897, + "harmed": 2048, + "missed_rescue": 526, + "avoided_harm": 723, + "selected_both_correct": 13998, + "selected_both_wrong": 5582 + }, + { + "budget": 0.83, + "threshold": -6.790655136108398, + "operator": "ge", + "actual_fraction": 0.7607123613986331, + "metric": 0.7503332104472109, + "rescued": 4923, + "harmed": 2073, + "missed_rescue": 500, + "avoided_harm": 698, + "selected_both_correct": 14223, + "selected_both_wrong": 5606 + }, + { + "budget": 0.84, + "threshold": -6.856695175170898, + "operator": "ge", + "actual_fraction": 0.770779570654794, + "metric": 0.749964552080084, + "rescued": 4937, + "harmed": 2100, + "missed_rescue": 486, + "avoided_harm": 671, + "selected_both_correct": 14511, + "selected_both_wrong": 5632 + }, + { + "budget": 0.85, + "threshold": -6.932329177856445, + "operator": "ge", + "actual_fraction": 0.7827467884184556, + "metric": 0.749964552080084, + "rescued": 4965, + "harmed": 2128, + "missed_rescue": 458, + "avoided_harm": 643, + "selected_both_correct": 14849, + "selected_both_wrong": 5660 + }, + { + "budget": 0.86, + "threshold": -6.999267578125, + "operator": "ge", + "actual_fraction": 0.793466239401072, + "metric": 0.7498511187363526, + "rescued": 4987, + "harmed": 2154, + "missed_rescue": 436, + "avoided_harm": 617, + "selected_both_correct": 15148, + "selected_both_wrong": 5691 + }, + { + "budget": 0.87, + "threshold": -7.081692695617676, + "operator": "ge", + "actual_fraction": 0.8059439072115249, + "metric": 0.7499361937441511, + "rescued": 5015, + "harmed": 2179, + "missed_rescue": 408, + "avoided_harm": 592, + "selected_both_correct": 15496, + "selected_both_wrong": 5730 + }, + { + "budget": 0.88, + "threshold": -7.155144214630127, + "operator": "ge", + "actual_fraction": 0.8164932081785441, + "metric": 0.7497093270566884, + "rescued": 5038, + "harmed": 2210, + "missed_rescue": 385, + "avoided_harm": 561, + "selected_both_correct": 15779, + "selected_both_wrong": 5765 + }, + { + "budget": 0.89, + "threshold": -7.227406978607178, + "operator": "ge", + "actual_fraction": 0.8270425091455633, + "metric": 0.7494257436973598, + "rescued": 5061, + "harmed": 2243, + "missed_rescue": 362, + "avoided_harm": 528, + "selected_both_correct": 16073, + "selected_both_wrong": 5787 + }, + { + "budget": 0.9, + "threshold": -7.308749198913574, + "operator": "ge", + "actual_fraction": 0.8367410600345971, + "metric": 0.749397385361427, + "rescued": 5087, + "harmed": 2270, + "missed_rescue": 336, + "avoided_harm": 501, + "selected_both_correct": 16348, + "selected_both_wrong": 5801 + }, + { + "budget": 0.91, + "threshold": -7.405246257781982, + "operator": "ge", + "actual_fraction": 0.8491903695091172, + "metric": 0.74883021864277, + "rescued": 5113, + "harmed": 2316, + "missed_rescue": 310, + "avoided_harm": 455, + "selected_both_correct": 16682, + "selected_both_wrong": 5834 + }, + { + "budget": 0.92, + "threshold": -7.509663105010986, + "operator": "gt", + "actual_fraction": 0.8618098289992343, + "metric": 0.7487167852990386, + "rescued": 5142, + "harmed": 2349, + "missed_rescue": 281, + "avoided_harm": 422, + "selected_both_correct": 17025, + "selected_both_wrong": 5874 + }, + { + "budget": 0.93, + "threshold": -7.625953674316406, + "operator": "ge", + "actual_fraction": 0.8745710801690156, + "metric": 0.7486884269631058, + "rescued": 5170, + "harmed": 2378, + "missed_rescue": 253, + "avoided_harm": 393, + "selected_both_correct": 17383, + "selected_both_wrong": 5909 + }, + { + "budget": 0.94, + "threshold": -7.762190818786621, + "operator": "ge", + "actual_fraction": 0.8879845730652526, + "metric": 0.7486884269631058, + "rescued": 5198, + "harmed": 2406, + "missed_rescue": 225, + "avoided_harm": 365, + "selected_both_correct": 17763, + "selected_both_wrong": 5946 + }, + { + "budget": 0.95, + "threshold": -7.891505241394043, + "operator": "ge", + "actual_fraction": 0.9000085075007799, + "metric": 0.7484332019397102, + "rescued": 5216, + "harmed": 2433, + "missed_rescue": 207, + "avoided_harm": 338, + "selected_both_correct": 18108, + "selected_both_wrong": 5980 + }, + { + "budget": 0.96, + "threshold": -8.049057960510254, + "operator": "ge", + "actual_fraction": 0.9125996086549641, + "metric": 0.7477809602132547, + "rescued": 5244, + "harmed": 2484, + "missed_rescue": 179, + "avoided_harm": 287, + "selected_both_correct": 18442, + "selected_both_wrong": 6011 + }, + { + "budget": 0.97, + "threshold": -8.248269081115723, + "operator": "ge", + "actual_fraction": 0.9269489266369849, + "metric": 0.7476391685335905, + "rescued": 5273, + "harmed": 2518, + "missed_rescue": 150, + "avoided_harm": 253, + "selected_both_correct": 18844, + "selected_both_wrong": 6052 + }, + { + "budget": 0.98, + "threshold": -8.543853759765625, + "operator": "ge", + "actual_fraction": 0.9459773700479256, + "metric": 0.7472988685023962, + "rescued": 5315, + "harmed": 2572, + "missed_rescue": 108, + "avoided_harm": 199, + "selected_both_correct": 19374, + "selected_both_wrong": 6097 + }, + { + "budget": 0.99, + "threshold": -8.96818733215332, + "operator": "ge", + "actual_fraction": 0.9658282052009188, + "metric": 0.7472421518305306, + "rescued": 5353, + "harmed": 2612, + "missed_rescue": 70, + "avoided_harm": 159, + "selected_both_correct": 19948, + "selected_both_wrong": 6145 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.7447182599325072, + "rescued": 5423, + "harmed": 2771, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 20838, + "selected_both_wrong": 6231 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 9505, + "rescuable": 1964, + "harmful": 1171, + "both_wrong": 1589, + "unit": "sample", + "total": 14229 + }, + "test": { + "both_correct": 20838, + "rescuable": 5423, + "harmful": 2771, + "both_wrong": 6231, + "unit": "sample", + "total": 35263 + } + }, + "small": { + "validation": 0.7502986857825568, + "test": 0.6695119530385957 + }, + "large": { + "validation": 0.8060299388572634, + "test": 0.7447182599325072 + }, + "learned_minus_confidence_test": 0.02850012761251175, + "learned_minus_small_test": 0.08331679097070588, + "learned_minus_large_test": 0.008110484076794378 + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "expert_display": "YOLO26x", + "run": "shared_12_yolo26x_Qwen3.8", + "validation_rows": 701, + "test_rows": 3004, + "confidence_rule": { + "column": "max_score", + "columns": [ + "detection_count", + "has_detections", + "max_score", + "mean_score", + "detection_count_valid", + "has_detections_valid", + "max_score_valid", + "mean_score_valid" + ], + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", + "missing": "routes first, score=1; valid score=-confidence" + }, + "methods": { + "confidence": { + "selected_budget_index": 93, + "validation_area_0_25": 0.17942553980644182, + "validation_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + "test_point": { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 498, + "selected_both_wrong": 0 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.1728507294858811, + "rescued": 0, + "harmed": 34, + "missed_rescue": 307, + "avoided_harm": 186, + "selected_both_correct": 526, + "selected_both_wrong": 0 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.17237009889855048, + "rescued": 0, + "harmed": 36, + "missed_rescue": 307, + "avoided_harm": 184, + "selected_both_correct": 552, + "selected_both_wrong": 0 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.17209204025345443, + "rescued": 0, + "harmed": 37, + "missed_rescue": 307, + "avoided_harm": 183, + "selected_both_correct": 578, + "selected_both_wrong": 1 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.1716297194027841, + "rescued": 0, + "harmed": 39, + "missed_rescue": 307, + "avoided_harm": 181, + "selected_both_correct": 604, + "selected_both_wrong": 1 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 629, + "selected_both_wrong": 1 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24821683309557774, + "metric": 0.17076888726505096, + "rescued": 0, + "harmed": 42, + "missed_rescue": 307, + "avoided_harm": 178, + "selected_both_correct": 652, + "selected_both_wrong": 2 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 681, + "selected_both_wrong": 2 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.268188302425107, + "metric": 0.17010751160367527, + "rescued": 0, + "harmed": 45, + "missed_rescue": 307, + "avoided_harm": 175, + "selected_both_correct": 705, + "selected_both_wrong": 2 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 734, + "selected_both_wrong": 2 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.16948188597804964, + "rescued": 0, + "harmed": 48, + "missed_rescue": 307, + "avoided_harm": 172, + "selected_both_correct": 762, + "selected_both_wrong": 2 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.16880965944961934, + "rescued": 0, + "harmed": 51, + "missed_rescue": 307, + "avoided_harm": 169, + "selected_both_correct": 787, + "selected_both_wrong": 2 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30813124108416545, + "metric": 0.16830460894456886, + "rescued": 0, + "harmed": 56, + "missed_rescue": 307, + "avoided_harm": 164, + "selected_both_correct": 806, + "selected_both_wrong": 2 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3181169757489301, + "metric": 0.16811053970066264, + "rescued": 0, + "harmed": 57, + "missed_rescue": 307, + "avoided_harm": 163, + "selected_both_correct": 833, + "selected_both_wrong": 2 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.1671811299174608, + "rescued": 0, + "harmed": 61, + "missed_rescue": 307, + "avoided_harm": 159, + "selected_both_correct": 861, + "selected_both_wrong": 2 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.16645634059427164, + "rescued": 0, + "harmed": 64, + "missed_rescue": 307, + "avoided_harm": 156, + "selected_both_correct": 886, + "selected_both_wrong": 2 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34807417974322397, + "metric": 0.16600354762332126, + "rescued": 0, + "harmed": 66, + "missed_rescue": 307, + "avoided_harm": 154, + "selected_both_correct": 908, + "selected_both_wrong": 2 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.16502549737843858, + "rescued": 0, + "harmed": 70, + "missed_rescue": 307, + "avoided_harm": 150, + "selected_both_correct": 936, + "selected_both_wrong": 2 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3666191155492154, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 72, + "missed_rescue": 307, + "avoided_harm": 148, + "selected_both_correct": 954, + "selected_both_wrong": 2 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37660485021398005, + "metric": 0.16467825104188744, + "rescued": 0, + "harmed": 76, + "missed_rescue": 307, + "avoided_harm": 144, + "selected_both_correct": 978, + "selected_both_wrong": 2 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.16749463738508683, + "rescued": 1, + "harmed": 81, + "missed_rescue": 306, + "avoided_harm": 139, + "selected_both_correct": 1008, + "selected_both_wrong": 2 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.16607474321791724, + "rescued": 1, + "harmed": 87, + "missed_rescue": 306, + "avoided_harm": 133, + "selected_both_correct": 1030, + "selected_both_wrong": 2 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.16514206572141965, + "rescued": 1, + "harmed": 90, + "missed_rescue": 306, + "avoided_harm": 130, + "selected_both_correct": 1055, + "selected_both_wrong": 2 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.16444252141114554, + "rescued": 1, + "harmed": 93, + "missed_rescue": 306, + "avoided_harm": 127, + "selected_both_correct": 1080, + "selected_both_wrong": 2 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.1639139801849299, + "rescued": 1, + "harmed": 96, + "missed_rescue": 306, + "avoided_harm": 124, + "selected_both_correct": 1105, + "selected_both_wrong": 2 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.16237039262040495, + "rescued": 1, + "harmed": 103, + "missed_rescue": 306, + "avoided_harm": 117, + "selected_both_correct": 1126, + "selected_both_wrong": 2 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.16187473569847557, + "rescued": 1, + "harmed": 105, + "missed_rescue": 306, + "avoided_harm": 115, + "selected_both_correct": 1152, + "selected_both_wrong": 2 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.1640341119762699, + "rescued": 2, + "harmed": 110, + "missed_rescue": 305, + "avoided_harm": 110, + "selected_both_correct": 1174, + "selected_both_wrong": 2 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.46790299572039945, + "metric": 0.16656779979633995, + "rescued": 3, + "harmed": 113, + "missed_rescue": 304, + "avoided_harm": 107, + "selected_both_correct": 1194, + "selected_both_wrong": 2 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.16532395129425315, + "rescued": 3, + "harmed": 119, + "missed_rescue": 304, + "avoided_harm": 101, + "selected_both_correct": 1220, + "selected_both_wrong": 2 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.16467625831496033, + "rescued": 5, + "harmed": 123, + "missed_rescue": 302, + "avoided_harm": 97, + "selected_both_correct": 1241, + "selected_both_wrong": 3 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.16569589650413527, + "rescued": 11, + "harmed": 128, + "missed_rescue": 296, + "avoided_harm": 92, + "selected_both_correct": 1257, + "selected_both_wrong": 4 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.1671927745457157, + "rescued": 18, + "harmed": 129, + "missed_rescue": 289, + "avoided_harm": 91, + "selected_both_correct": 1276, + "selected_both_wrong": 5 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5178316690442225, + "metric": 0.16789087940773334, + "rescued": 23, + "harmed": 131, + "missed_rescue": 284, + "avoided_harm": 89, + "selected_both_correct": 1293, + "selected_both_wrong": 5 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.16946103210148153, + "rescued": 30, + "harmed": 132, + "missed_rescue": 277, + "avoided_harm": 88, + "selected_both_correct": 1316, + "selected_both_wrong": 6 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5378031383737518, + "metric": 0.17052688690619724, + "rescued": 35, + "harmed": 133, + "missed_rescue": 272, + "avoided_harm": 87, + "selected_both_correct": 1334, + "selected_both_wrong": 6 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.17103667850460152, + "rescued": 42, + "harmed": 134, + "missed_rescue": 265, + "avoided_harm": 86, + "selected_both_correct": 1358, + "selected_both_wrong": 6 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.17291017019862398, + "rescued": 49, + "harmed": 135, + "missed_rescue": 258, + "avoided_harm": 85, + "selected_both_correct": 1376, + "selected_both_wrong": 8 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.17468909977784217, + "rescued": 55, + "harmed": 135, + "missed_rescue": 252, + "avoided_harm": 85, + "selected_both_correct": 1397, + "selected_both_wrong": 9 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.1754433851238893, + "rescued": 60, + "harmed": 136, + "missed_rescue": 247, + "avoided_harm": 84, + "selected_both_correct": 1416, + "selected_both_wrong": 12 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5877318116975749, + "metric": 0.17567949038614306, + "rescued": 65, + "harmed": 140, + "missed_rescue": 242, + "avoided_harm": 80, + "selected_both_correct": 1430, + "selected_both_wrong": 13 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.5977175463623395, + "metric": 0.17726242688944494, + "rescued": 73, + "harmed": 142, + "missed_rescue": 234, + "avoided_harm": 78, + "selected_both_correct": 1448, + "selected_both_wrong": 13 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.17927753989081613, + "rescued": 81, + "harmed": 143, + "missed_rescue": 226, + "avoided_harm": 77, + "selected_both_correct": 1471, + "selected_both_wrong": 13 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6176890156918687, + "metric": 0.1803426895170312, + "rescued": 86, + "harmed": 144, + "missed_rescue": 221, + "avoided_harm": 76, + "selected_both_correct": 1489, + "selected_both_wrong": 13 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.18256297675338154, + "rescued": 94, + "harmed": 145, + "missed_rescue": 213, + "avoided_harm": 75, + "selected_both_correct": 1509, + "selected_both_wrong": 16 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.18741054744283442, + "rescued": 103, + "harmed": 149, + "missed_rescue": 204, + "avoided_harm": 71, + "selected_both_correct": 1524, + "selected_both_wrong": 16 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6476462196861626, + "metric": 0.18874453577558464, + "rescued": 109, + "harmed": 150, + "missed_rescue": 198, + "avoided_harm": 70, + "selected_both_correct": 1540, + "selected_both_wrong": 17 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6562054208273894, + "metric": 0.18973030032445837, + "rescued": 114, + "harmed": 152, + "missed_rescue": 193, + "avoided_harm": 68, + "selected_both_correct": 1556, + "selected_both_wrong": 18 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.1910322331722189, + "rescued": 122, + "harmed": 157, + "missed_rescue": 185, + "avoided_harm": 63, + "selected_both_correct": 1576, + "selected_both_wrong": 21 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.19273815380416676, + "rescued": 128, + "harmed": 158, + "missed_rescue": 179, + "avoided_harm": 62, + "selected_both_correct": 1595, + "selected_both_wrong": 23 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.19362404817095658, + "rescued": 133, + "harmed": 161, + "missed_rescue": 174, + "avoided_harm": 59, + "selected_both_correct": 1612, + "selected_both_wrong": 26 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.6975748930099858, + "metric": 0.19489832797986031, + "rescued": 138, + "harmed": 163, + "missed_rescue": 169, + "avoided_harm": 57, + "selected_both_correct": 1626, + "selected_both_wrong": 29 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.19709243659719292, + "rescued": 147, + "harmed": 166, + "missed_rescue": 160, + "avoided_harm": 54, + "selected_both_correct": 1646, + "selected_both_wrong": 29 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.20000323882160753, + "rescued": 155, + "harmed": 166, + "missed_rescue": 152, + "avoided_harm": 54, + "selected_both_correct": 1666, + "selected_both_wrong": 29 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2015690267335004, + "rescued": 160, + "harmed": 167, + "missed_rescue": 147, + "avoided_harm": 53, + "selected_both_correct": 1684, + "selected_both_wrong": 33 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2034984942593638, + "rescued": 166, + "harmed": 168, + "missed_rescue": 141, + "avoided_harm": 52, + "selected_both_correct": 1703, + "selected_both_wrong": 35 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.20679879821670866, + "rescued": 176, + "harmed": 170, + "missed_rescue": 131, + "avoided_harm": 50, + "selected_both_correct": 1718, + "selected_both_wrong": 36 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.20972162293572166, + "rescued": 184, + "harmed": 171, + "missed_rescue": 123, + "avoided_harm": 49, + "selected_both_correct": 1735, + "selected_both_wrong": 38 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.2126226551226551, + "rescued": 191, + "harmed": 174, + "missed_rescue": 116, + "avoided_harm": 46, + "selected_both_correct": 1752, + "selected_both_wrong": 39 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.21516614155218836, + "rescued": 198, + "harmed": 175, + "missed_rescue": 109, + "avoided_harm": 45, + "selected_both_correct": 1771, + "selected_both_wrong": 40 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2162880860385455, + "rescued": 204, + "harmed": 177, + "missed_rescue": 103, + "avoided_harm": 43, + "selected_both_correct": 1788, + "selected_both_wrong": 43 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.21779120256650406, + "rescued": 209, + "harmed": 179, + "missed_rescue": 98, + "avoided_harm": 41, + "selected_both_correct": 1806, + "selected_both_wrong": 46 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.2201843117420812, + "rescued": 216, + "harmed": 182, + "missed_rescue": 91, + "avoided_harm": 38, + "selected_both_correct": 1820, + "selected_both_wrong": 50 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.22188537527520574, + "rescued": 222, + "harmed": 184, + "missed_rescue": 85, + "avoided_harm": 36, + "selected_both_correct": 1838, + "selected_both_wrong": 52 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.2260242392475367, + "rescued": 231, + "harmed": 187, + "missed_rescue": 76, + "avoided_harm": 33, + "selected_both_correct": 1854, + "selected_both_wrong": 52 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2282665192342907, + "rescued": 238, + "harmed": 190, + "missed_rescue": 69, + "avoided_harm": 30, + "selected_both_correct": 1869, + "selected_both_wrong": 55 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.23641538775702553, + "rescued": 246, + "harmed": 193, + "missed_rescue": 61, + "avoided_harm": 27, + "selected_both_correct": 1884, + "selected_both_wrong": 57 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2387681003514125, + "rescued": 251, + "harmed": 193, + "missed_rescue": 56, + "avoided_harm": 27, + "selected_both_correct": 1904, + "selected_both_wrong": 60 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.24246700995212897, + "rescued": 258, + "harmed": 193, + "missed_rescue": 49, + "avoided_harm": 27, + "selected_both_correct": 1923, + "selected_both_wrong": 62 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.2423696527366133, + "rescued": 259, + "harmed": 197, + "missed_rescue": 48, + "avoided_harm": 23, + "selected_both_correct": 1941, + "selected_both_wrong": 67 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.24478740599615373, + "rescued": 266, + "harmed": 199, + "missed_rescue": 41, + "avoided_harm": 21, + "selected_both_correct": 1959, + "selected_both_wrong": 68 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.25736315359477124, + "rescued": 273, + "harmed": 199, + "missed_rescue": 34, + "avoided_harm": 21, + "selected_both_correct": 1977, + "selected_both_wrong": 71 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.9072753209700428, + "metric": 0.2597551406926407, + "rescued": 278, + "harmed": 200, + "missed_rescue": 29, + "avoided_harm": 20, + "selected_both_correct": 1992, + "selected_both_wrong": 74 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2629241291812946, + "rescued": 284, + "harmed": 201, + "missed_rescue": 23, + "avoided_harm": 19, + "selected_both_correct": 2013, + "selected_both_wrong": 78 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2642878791936722, + "rescued": 288, + "harmed": 202, + "missed_rescue": 19, + "avoided_harm": 18, + "selected_both_correct": 2035, + "selected_both_wrong": 79 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2639590174546567, + "rescued": 290, + "harmed": 203, + "missed_rescue": 17, + "avoided_harm": 17, + "selected_both_correct": 2057, + "selected_both_wrong": 82 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26132796745180287, + "rescued": 293, + "harmed": 207, + "missed_rescue": 14, + "avoided_harm": 13, + "selected_both_correct": 2078, + "selected_both_wrong": 82 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.2584345323365104, + "rescued": 296, + "harmed": 212, + "missed_rescue": 11, + "avoided_harm": 8, + "selected_both_correct": 2096, + "selected_both_wrong": 84 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2503146324574896, + "rescued": 299, + "harmed": 215, + "missed_rescue": 8, + "avoided_harm": 5, + "selected_both_correct": 2116, + "selected_both_wrong": 86 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.2545437366865938, + "rescued": 302, + "harmed": 216, + "missed_rescue": 5, + "avoided_harm": 4, + "selected_both_correct": 2139, + "selected_both_wrong": 87 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.2541444987255433, + "rescued": 304, + "harmed": 218, + "missed_rescue": 3, + "avoided_harm": 2, + "selected_both_correct": 2163, + "selected_both_wrong": 87 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.02, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.03, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.04, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.05, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.06, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.07, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.08, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.09, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.1, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.11, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.12, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.13, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.14, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.15, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.16, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.17, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.18, + "threshold": 1.0, + "operator": "gt", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.19, + "threshold": 1.0, + "operator": "ge", + "actual_fraction": 0.17942743009320905, + "metric": 0.1784481207469488, + "rescued": 4, + "harmed": 94, + "missed_rescue": 1278, + "avoided_harm": 683, + "selected_both_correct": 2057, + "selected_both_wrong": 1 + }, + { + "budget": 0.2, + "threshold": -0.0010652542114257812, + "operator": "ge", + "actual_fraction": 0.19340878828229027, + "metric": 0.1780928651788194, + "rescued": 4, + "harmed": 99, + "missed_rescue": 1278, + "avoided_harm": 678, + "selected_both_correct": 2220, + "selected_both_wrong": 1 + }, + { + "budget": 0.21, + "threshold": -0.0011425018310546875, + "operator": "ge", + "actual_fraction": 0.20306258322237017, + "metric": 0.17829989216687975, + "rescued": 5, + "harmed": 108, + "missed_rescue": 1277, + "avoided_harm": 669, + "selected_both_correct": 2325, + "selected_both_wrong": 2 + }, + { + "budget": 0.22, + "threshold": -0.0011968612670898438, + "operator": "ge", + "actual_fraction": 0.20838881491344874, + "metric": 0.1779854683890517, + "rescued": 5, + "harmed": 112, + "missed_rescue": 1277, + "avoided_harm": 665, + "selected_both_correct": 2385, + "selected_both_wrong": 2 + }, + { + "budget": 0.23, + "threshold": -0.0012493133544921875, + "operator": "ge", + "actual_fraction": 0.21537949400798934, + "metric": 0.17917657728013692, + "rescued": 6, + "harmed": 118, + "missed_rescue": 1276, + "avoided_harm": 659, + "selected_both_correct": 2462, + "selected_both_wrong": 2 + }, + { + "budget": 0.24, + "threshold": -0.0013513565063476562, + "operator": "ge", + "actual_fraction": 0.23135818908122505, + "metric": 0.17803687432776102, + "rescued": 6, + "harmed": 133, + "missed_rescue": 1276, + "avoided_harm": 644, + "selected_both_correct": 2638, + "selected_both_wrong": 3 + }, + { + "budget": 0.25, + "threshold": -0.0015010833740234375, + "operator": "gt", + "actual_fraction": 0.24800266311584554, + "metric": 0.18026802885144627, + "rescued": 10, + "harmed": 153, + "missed_rescue": 1272, + "avoided_harm": 624, + "selected_both_correct": 2814, + "selected_both_wrong": 3 + }, + { + "budget": 0.26, + "threshold": -0.00157928466796875, + "operator": "ge", + "actual_fraction": 0.2589880159786951, + "metric": 0.18119827871024874, + "rescued": 11, + "harmed": 163, + "missed_rescue": 1271, + "avoided_harm": 614, + "selected_both_correct": 2935, + "selected_both_wrong": 3 + }, + { + "budget": 0.27, + "threshold": -0.0017137527465820312, + "operator": "gt", + "actual_fraction": 0.27430093209054596, + "metric": 0.17963665085316138, + "rescued": 11, + "harmed": 185, + "missed_rescue": 1271, + "avoided_harm": 592, + "selected_both_correct": 3097, + "selected_both_wrong": 3 + }, + { + "budget": 0.28, + "threshold": -0.0017614364624023438, + "operator": "ge", + "actual_fraction": 0.281624500665779, + "metric": 0.17924095839307458, + "rescued": 11, + "harmed": 191, + "missed_rescue": 1271, + "avoided_harm": 586, + "selected_both_correct": 3179, + "selected_both_wrong": 3 + }, + { + "budget": 0.29, + "threshold": -0.0018463134765625, + "operator": "ge", + "actual_fraction": 0.28761651131824234, + "metric": 0.1789157174618718, + "rescued": 11, + "harmed": 196, + "missed_rescue": 1271, + "avoided_harm": 581, + "selected_both_correct": 3246, + "selected_both_wrong": 3 + }, + { + "budget": 0.3, + "threshold": -0.001934051513671875, + "operator": "ge", + "actual_fraction": 0.2969374167776298, + "metric": 0.1787206845800119, + "rescued": 11, + "harmed": 199, + "missed_rescue": 1271, + "avoided_harm": 578, + "selected_both_correct": 3355, + "selected_both_wrong": 3 + }, + { + "budget": 0.31, + "threshold": -0.0020427703857421875, + "operator": "gt", + "actual_fraction": 0.30459387483355527, + "metric": 0.17813242189706796, + "rescued": 11, + "harmed": 207, + "missed_rescue": 1271, + "avoided_harm": 570, + "selected_both_correct": 3438, + "selected_both_wrong": 4 + }, + { + "budget": 0.32, + "threshold": -0.0021495819091796875, + "operator": "gt", + "actual_fraction": 0.3129161118508655, + "metric": 0.17772000842137498, + "rescued": 12, + "harmed": 218, + "missed_rescue": 1270, + "avoided_harm": 559, + "selected_both_correct": 3526, + "selected_both_wrong": 4 + }, + { + "budget": 0.33, + "threshold": -0.002208709716796875, + "operator": "ge", + "actual_fraction": 0.31990679094540614, + "metric": 0.17791259734473341, + "rescued": 13, + "harmed": 227, + "missed_rescue": 1269, + "avoided_harm": 550, + "selected_both_correct": 3600, + "selected_both_wrong": 4 + }, + { + "budget": 0.34, + "threshold": -0.0023326873779296875, + "operator": "ge", + "actual_fraction": 0.3322237017310253, + "metric": 0.1789653934599337, + "rescued": 15, + "harmed": 238, + "missed_rescue": 1267, + "avoided_harm": 539, + "selected_both_correct": 3735, + "selected_both_wrong": 4 + }, + { + "budget": 0.35, + "threshold": -0.0024623870849609375, + "operator": "gt", + "actual_fraction": 0.34054593874833555, + "metric": 0.18042682200794885, + "rescued": 16, + "harmed": 249, + "missed_rescue": 1266, + "avoided_harm": 528, + "selected_both_correct": 3822, + "selected_both_wrong": 5 + }, + { + "budget": 0.36, + "threshold": -0.0025615692138671875, + "operator": "ge", + "actual_fraction": 0.3488681757656458, + "metric": 0.17962877325912951, + "rescued": 16, + "harmed": 260, + "missed_rescue": 1266, + "avoided_harm": 517, + "selected_both_correct": 3910, + "selected_both_wrong": 6 + }, + { + "budget": 0.37, + "threshold": -0.0026721954345703125, + "operator": "gt", + "actual_fraction": 0.3581890812250333, + "metric": 0.18158304685510065, + "rescued": 18, + "harmed": 271, + "missed_rescue": 1264, + "avoided_harm": 506, + "selected_both_correct": 4008, + "selected_both_wrong": 7 + }, + { + "budget": 0.38, + "threshold": -0.0028228759765625, + "operator": "gt", + "actual_fraction": 0.37017310252996005, + "metric": 0.18069239723097547, + "rescued": 18, + "harmed": 285, + "missed_rescue": 1264, + "avoided_harm": 492, + "selected_both_correct": 4138, + "selected_both_wrong": 7 + }, + { + "budget": 0.39, + "threshold": -0.0029354095458984375, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.1812845803851709, + "rescued": 20, + "harmed": 292, + "missed_rescue": 1262, + "avoided_harm": 485, + "selected_both_correct": 4209, + "selected_both_wrong": 7 + }, + { + "budget": 0.4, + "threshold": -0.00307464599609375, + "operator": "ge", + "actual_fraction": 0.38348868175765644, + "metric": 0.1824781664689562, + "rescued": 21, + "harmed": 301, + "missed_rescue": 1261, + "avoided_harm": 476, + "selected_both_correct": 4279, + "selected_both_wrong": 7 + }, + { + "budget": 0.41, + "threshold": -0.0033626556396484375, + "operator": "ge", + "actual_fraction": 0.4021304926764314, + "metric": 0.18220594501399923, + "rescued": 23, + "harmed": 321, + "missed_rescue": 1259, + "avoided_harm": 456, + "selected_both_correct": 4480, + "selected_both_wrong": 8 + }, + { + "budget": 0.42, + "threshold": -0.0035381317138671875, + "operator": "ge", + "actual_fraction": 0.41311584553928093, + "metric": 0.18162498930772586, + "rescued": 23, + "harmed": 329, + "missed_rescue": 1259, + "avoided_harm": 448, + "selected_both_correct": 4603, + "selected_both_wrong": 9 + }, + { + "budget": 0.43, + "threshold": -0.003765106201171875, + "operator": "ge", + "actual_fraction": 0.42676431424766975, + "metric": 0.18066434967152634, + "rescued": 23, + "harmed": 351, + "missed_rescue": 1259, + "avoided_harm": 426, + "selected_both_correct": 4745, + "selected_both_wrong": 9 + }, + { + "budget": 0.44, + "threshold": -0.00392913818359375, + "operator": "ge", + "actual_fraction": 0.4324234354194407, + "metric": 0.1803913789289444, + "rescued": 23, + "harmed": 355, + "missed_rescue": 1259, + "avoided_harm": 422, + "selected_both_correct": 4809, + "selected_both_wrong": 9 + }, + { + "budget": 0.45, + "threshold": -0.004085540771484375, + "operator": "ge", + "actual_fraction": 0.44107856191744343, + "metric": 0.17959468902865128, + "rescued": 23, + "harmed": 368, + "missed_rescue": 1259, + "avoided_harm": 409, + "selected_both_correct": 4900, + "selected_both_wrong": 9 + }, + { + "budget": 0.46, + "threshold": -0.004364013671875, + "operator": "ge", + "actual_fraction": 0.45772303595206393, + "metric": 0.17863383514615866, + "rescued": 24, + "harmed": 387, + "missed_rescue": 1258, + "avoided_harm": 390, + "selected_both_correct": 5078, + "selected_both_wrong": 11 + }, + { + "budget": 0.47, + "threshold": -0.004680633544921875, + "operator": "gt", + "actual_fraction": 0.4717043941411451, + "metric": 0.17957308538379646, + "rescued": 26, + "harmed": 405, + "missed_rescue": 1256, + "avoided_harm": 372, + "selected_both_correct": 5225, + "selected_both_wrong": 12 + }, + { + "budget": 0.48, + "threshold": -0.00490570068359375, + "operator": "ge", + "actual_fraction": 0.4826897470039947, + "metric": 0.17910245145856155, + "rescued": 27, + "harmed": 417, + "missed_rescue": 1255, + "avoided_harm": 360, + "selected_both_correct": 5344, + "selected_both_wrong": 12 + }, + { + "budget": 0.49, + "threshold": -0.005039215087890625, + "operator": "ge", + "actual_fraction": 0.4896804260985353, + "metric": 0.17871833907402393, + "rescued": 29, + "harmed": 425, + "missed_rescue": 1253, + "avoided_harm": 352, + "selected_both_correct": 5416, + "selected_both_wrong": 14 + }, + { + "budget": 0.5, + "threshold": -0.005321502685546875, + "operator": "ge", + "actual_fraction": 0.5013315579227696, + "metric": 0.18015785955296285, + "rescued": 57, + "harmed": 431, + "missed_rescue": 1225, + "avoided_harm": 346, + "selected_both_correct": 5516, + "selected_both_wrong": 20 + }, + { + "budget": 0.51, + "threshold": -0.005619049072265625, + "operator": "ge", + "actual_fraction": 0.5116511318242344, + "metric": 0.1810878167493375, + "rescued": 76, + "harmed": 435, + "missed_rescue": 1206, + "avoided_harm": 342, + "selected_both_correct": 5606, + "selected_both_wrong": 31 + }, + { + "budget": 0.52, + "threshold": -0.00586700439453125, + "operator": "gt", + "actual_fraction": 0.5183089214380826, + "metric": 0.18177962418520882, + "rescued": 92, + "harmed": 439, + "missed_rescue": 1190, + "avoided_harm": 338, + "selected_both_correct": 5662, + "selected_both_wrong": 35 + }, + { + "budget": 0.53, + "threshold": -0.00605010986328125, + "operator": "ge", + "actual_fraction": 0.5252996005326231, + "metric": 0.18246323539519374, + "rescued": 109, + "harmed": 445, + "missed_rescue": 1173, + "avoided_harm": 332, + "selected_both_correct": 5719, + "selected_both_wrong": 39 + }, + { + "budget": 0.54, + "threshold": -0.006439208984375, + "operator": "gt", + "actual_fraction": 0.5396138482023968, + "metric": 0.1839876166906015, + "rescued": 150, + "harmed": 461, + "missed_rescue": 1132, + "avoided_harm": 316, + "selected_both_correct": 5829, + "selected_both_wrong": 44 + }, + { + "budget": 0.55, + "threshold": -0.006641387939453125, + "operator": "ge", + "actual_fraction": 0.5452729693741678, + "metric": 0.1846674213907964, + "rescued": 164, + "harmed": 464, + "missed_rescue": 1118, + "avoided_harm": 313, + "selected_both_correct": 5875, + "selected_both_wrong": 49 + }, + { + "budget": 0.56, + "threshold": -0.0070648193359375, + "operator": "ge", + "actual_fraction": 0.559254327563249, + "metric": 0.18676294233791663, + "rescued": 196, + "harmed": 473, + "missed_rescue": 1086, + "avoided_harm": 304, + "selected_both_correct": 5990, + "selected_both_wrong": 61 + }, + { + "budget": 0.57, + "threshold": -0.007518768310546875, + "operator": "ge", + "actual_fraction": 0.572237017310253, + "metric": 0.18938663221175195, + "rescued": 230, + "harmed": 483, + "missed_rescue": 1052, + "avoided_harm": 294, + "selected_both_correct": 6093, + "selected_both_wrong": 70 + }, + { + "budget": 0.58, + "threshold": -0.00818634033203125, + "operator": "ge", + "actual_fraction": 0.5862183754993342, + "metric": 0.1909571002676625, + "rescued": 267, + "harmed": 496, + "missed_rescue": 1015, + "avoided_harm": 281, + "selected_both_correct": 6208, + "selected_both_wrong": 73 + }, + { + "budget": 0.59, + "threshold": -0.00878143310546875, + "operator": "gt", + "actual_fraction": 0.5998668442077231, + "metric": 0.1934300101790686, + "rescued": 299, + "harmed": 503, + "missed_rescue": 983, + "avoided_harm": 274, + "selected_both_correct": 6315, + "selected_both_wrong": 91 + }, + { + "budget": 0.6, + "threshold": -0.00955963134765625, + "operator": "gt", + "actual_fraction": 0.6191744340878829, + "metric": 0.1973701682130997, + "rescued": 350, + "harmed": 520, + "missed_rescue": 932, + "avoided_harm": 257, + "selected_both_correct": 6465, + "selected_both_wrong": 105 + }, + { + "budget": 0.61, + "threshold": -0.0099334716796875, + "operator": "ge", + "actual_fraction": 0.6264980026631158, + "metric": 0.1986727924998457, + "rescued": 370, + "harmed": 523, + "missed_rescue": 912, + "avoided_harm": 254, + "selected_both_correct": 6527, + "selected_both_wrong": 108 + }, + { + "budget": 0.62, + "threshold": -0.010650634765625, + "operator": "gt", + "actual_fraction": 0.6384820239680427, + "metric": 0.20186289191698192, + "rescued": 402, + "harmed": 532, + "missed_rescue": 880, + "avoided_harm": 245, + "selected_both_correct": 6621, + "selected_both_wrong": 117 + }, + { + "budget": 0.63, + "threshold": -0.01129150390625, + "operator": "ge", + "actual_fraction": 0.6461384820239681, + "metric": 0.20246031058520178, + "rescued": 425, + "harmed": 537, + "missed_rescue": 857, + "avoided_harm": 240, + "selected_both_correct": 6682, + "selected_both_wrong": 120 + }, + { + "budget": 0.64, + "threshold": -0.011871337890625, + "operator": "ge", + "actual_fraction": 0.653462050599201, + "metric": 0.20349944137622245, + "rescued": 444, + "harmed": 539, + "missed_rescue": 838, + "avoided_harm": 238, + "selected_both_correct": 6744, + "selected_both_wrong": 125 + }, + { + "budget": 0.65, + "threshold": -0.0125274658203125, + "operator": "gt", + "actual_fraction": 0.6597869507323568, + "metric": 0.20442990090722246, + "rescued": 461, + "harmed": 545, + "missed_rescue": 821, + "avoided_harm": 232, + "selected_both_correct": 6792, + "selected_both_wrong": 130 + }, + { + "budget": 0.66, + "threshold": -0.012969970703125, + "operator": "gt", + "actual_fraction": 0.6661118508655126, + "metric": 0.2044993942500819, + "rescued": 480, + "harmed": 556, + "missed_rescue": 802, + "avoided_harm": 221, + "selected_both_correct": 6834, + "selected_both_wrong": 134 + }, + { + "budget": 0.67, + "threshold": -0.01363372802734375, + "operator": "ge", + "actual_fraction": 0.6734354194407457, + "metric": 0.2064379542733177, + "rescued": 500, + "harmed": 561, + "missed_rescue": 782, + "avoided_harm": 216, + "selected_both_correct": 6889, + "selected_both_wrong": 142 + }, + { + "budget": 0.68, + "threshold": -0.01473236083984375, + "operator": "ge", + "actual_fraction": 0.6820905459387483, + "metric": 0.2065831206227621, + "rescued": 523, + "harmed": 566, + "missed_rescue": 759, + "avoided_harm": 211, + "selected_both_correct": 6958, + "selected_both_wrong": 149 + }, + { + "budget": 0.69, + "threshold": -0.015716552734375, + "operator": "ge", + "actual_fraction": 0.6927430093209055, + "metric": 0.20802802394105854, + "rescued": 553, + "harmed": 577, + "missed_rescue": 729, + "avoided_harm": 200, + "selected_both_correct": 7035, + "selected_both_wrong": 159 + }, + { + "budget": 0.7, + "threshold": -0.017578125, + "operator": "gt", + "actual_fraction": 0.7090545938748336, + "metric": 0.21169680596278434, + "rescued": 602, + "harmed": 584, + "missed_rescue": 680, + "avoided_harm": 193, + "selected_both_correct": 7159, + "selected_both_wrong": 175 + }, + { + "budget": 0.71, + "threshold": -0.0189056396484375, + "operator": "ge", + "actual_fraction": 0.7170439414114514, + "metric": 0.2139280527103987, + "rescued": 625, + "harmed": 588, + "missed_rescue": 657, + "avoided_harm": 189, + "selected_both_correct": 7220, + "selected_both_wrong": 183 + }, + { + "budget": 0.72, + "threshold": -0.019866943359375, + "operator": "ge", + "actual_fraction": 0.7243675099866844, + "metric": 0.2146900231025185, + "rescued": 647, + "harmed": 594, + "missed_rescue": 635, + "avoided_harm": 183, + "selected_both_correct": 7268, + "selected_both_wrong": 195 + }, + { + "budget": 0.73, + "threshold": -0.020965576171875, + "operator": "ge", + "actual_fraction": 0.7330226364846871, + "metric": 0.21653561577692484, + "rescued": 670, + "harmed": 599, + "missed_rescue": 612, + "avoided_harm": 178, + "selected_both_correct": 7338, + "selected_both_wrong": 201 + }, + { + "budget": 0.74, + "threshold": -0.021453857421875, + "operator": "ge", + "actual_fraction": 0.7363515312916112, + "metric": 0.21721689648133474, + "rescued": 681, + "harmed": 601, + "missed_rescue": 601, + "avoided_harm": 176, + "selected_both_correct": 7365, + "selected_both_wrong": 201 + }, + { + "budget": 0.75, + "threshold": -0.0248870849609375, + "operator": "ge", + "actual_fraction": 0.7486684420772304, + "metric": 0.22103445955201576, + "rescued": 718, + "harmed": 610, + "missed_rescue": 564, + "avoided_harm": 167, + "selected_both_correct": 7460, + "selected_both_wrong": 208 + }, + { + "budget": 0.76, + "threshold": -0.0269622802734375, + "operator": "ge", + "actual_fraction": 0.757323568575233, + "metric": 0.22340134248367294, + "rescued": 744, + "harmed": 618, + "missed_rescue": 538, + "avoided_harm": 159, + "selected_both_correct": 7522, + "selected_both_wrong": 216 + }, + { + "budget": 0.77, + "threshold": -0.0282135009765625, + "operator": "ge", + "actual_fraction": 0.7639813581890812, + "metric": 0.2269322568854726, + "rescued": 767, + "harmed": 620, + "missed_rescue": 515, + "avoided_harm": 157, + "selected_both_correct": 7571, + "selected_both_wrong": 222 + }, + { + "budget": 0.78, + "threshold": -0.0299835205078125, + "operator": "ge", + "actual_fraction": 0.7733022636484687, + "metric": 0.22757452798038888, + "rescued": 798, + "harmed": 628, + "missed_rescue": 484, + "avoided_harm": 149, + "selected_both_correct": 7636, + "selected_both_wrong": 230 + }, + { + "budget": 0.79, + "threshold": -0.033966064453125, + "operator": "ge", + "actual_fraction": 0.784953395472703, + "metric": 0.227821050912831, + "rescued": 824, + "harmed": 634, + "missed_rescue": 458, + "avoided_harm": 143, + "selected_both_correct": 7728, + "selected_both_wrong": 246 + }, + { + "budget": 0.8, + "threshold": -0.037200927734375, + "operator": "ge", + "actual_fraction": 0.7939414114513982, + "metric": 0.22991865671924006, + "rescued": 852, + "harmed": 642, + "missed_rescue": 430, + "avoided_harm": 135, + "selected_both_correct": 7793, + "selected_both_wrong": 253 + }, + { + "budget": 0.81, + "threshold": -0.04132080078125, + "operator": "ge", + "actual_fraction": 0.8039280958721704, + "metric": 0.23486191981046528, + "rescued": 884, + "harmed": 646, + "missed_rescue": 398, + "avoided_harm": 131, + "selected_both_correct": 7868, + "selected_both_wrong": 262 + }, + { + "budget": 0.82, + "threshold": -0.0455322265625, + "operator": "ge", + "actual_fraction": 0.8122503328894807, + "metric": 0.23803916047544685, + "rescued": 906, + "harmed": 655, + "missed_rescue": 376, + "avoided_harm": 122, + "selected_both_correct": 7924, + "selected_both_wrong": 275 + }, + { + "budget": 0.83, + "threshold": -0.051849365234375, + "operator": "ge", + "actual_fraction": 0.8215712383488681, + "metric": 0.24022947987632814, + "rescued": 933, + "harmed": 664, + "missed_rescue": 349, + "avoided_harm": 113, + "selected_both_correct": 7987, + "selected_both_wrong": 288 + }, + { + "budget": 0.84, + "threshold": -0.06878662109375, + "operator": "ge", + "actual_fraction": 0.8388814913448736, + "metric": 0.24528627405740036, + "rescued": 988, + "harmed": 672, + "missed_rescue": 294, + "avoided_harm": 105, + "selected_both_correct": 8116, + "selected_both_wrong": 304 + }, + { + "budget": 0.85, + "threshold": -0.08477783203125, + "operator": "ge", + "actual_fraction": 0.8535286284953395, + "metric": 0.249537136326177, + "rescued": 1028, + "harmed": 680, + "missed_rescue": 254, + "avoided_harm": 97, + "selected_both_correct": 8224, + "selected_both_wrong": 324 + }, + { + "budget": 0.86, + "threshold": -0.09912109375, + "operator": "ge", + "actual_fraction": 0.8638482023968043, + "metric": 0.25211074301800224, + "rescued": 1056, + "harmed": 688, + "missed_rescue": 226, + "avoided_harm": 89, + "selected_both_correct": 8299, + "selected_both_wrong": 337 + }, + { + "budget": 0.87, + "threshold": -0.1298828125, + "operator": "ge", + "actual_fraction": 0.8745006657789614, + "metric": 0.2540042317315216, + "rescued": 1076, + "harmed": 695, + "missed_rescue": 206, + "avoided_harm": 82, + "selected_both_correct": 8389, + "selected_both_wrong": 348 + }, + { + "budget": 0.88, + "threshold": -0.1441650390625, + "operator": "ge", + "actual_fraction": 0.8764980026631158, + "metric": 0.2540996517958595, + "rescued": 1080, + "harmed": 698, + "missed_rescue": 202, + "avoided_harm": 79, + "selected_both_correct": 8404, + "selected_both_wrong": 350 + }, + { + "budget": 0.89, + "threshold": -0.182373046875, + "operator": "ge", + "actual_fraction": 0.8841544607190412, + "metric": 0.25436194678023316, + "rescued": 1090, + "harmed": 705, + "missed_rescue": 192, + "avoided_harm": 72, + "selected_both_correct": 8468, + "selected_both_wrong": 361 + }, + { + "budget": 0.9, + "threshold": -0.2049560546875, + "operator": "ge", + "actual_fraction": 0.8891478029294274, + "metric": 0.2534339447803836, + "rescued": 1101, + "harmed": 709, + "missed_rescue": 181, + "avoided_harm": 68, + "selected_both_correct": 8506, + "selected_both_wrong": 368 + }, + { + "budget": 0.91, + "threshold": -0.25830078125, + "operator": "gt", + "actual_fraction": 0.8981358189081226, + "metric": 0.2563240880787248, + "rescued": 1126, + "harmed": 715, + "missed_rescue": 156, + "avoided_harm": 62, + "selected_both_correct": 8573, + "selected_both_wrong": 378 + }, + { + "budget": 0.92, + "threshold": -0.32080078125, + "operator": "ge", + "actual_fraction": 0.9054593874833555, + "metric": 0.25551845731687656, + "rescued": 1143, + "harmed": 721, + "missed_rescue": 139, + "avoided_harm": 56, + "selected_both_correct": 8632, + "selected_both_wrong": 384 + }, + { + "budget": 0.93, + "threshold": -0.453369140625, + "operator": "ge", + "actual_fraction": 0.9221038615179761, + "metric": 0.2597490004762105, + "rescued": 1180, + "harmed": 728, + "missed_rescue": 102, + "avoided_harm": 49, + "selected_both_correct": 8774, + "selected_both_wrong": 398 + }, + { + "budget": 0.94, + "threshold": -0.490234375, + "operator": "ge", + "actual_fraction": 0.9254327563249002, + "metric": 0.26306323674307786, + "rescued": 1189, + "harmed": 729, + "missed_rescue": 93, + "avoided_harm": 48, + "selected_both_correct": 8800, + "selected_both_wrong": 402 + }, + { + "budget": 0.95, + "threshold": -0.611328125, + "operator": "ge", + "actual_fraction": 0.9364181091877497, + "metric": 0.26218185491449153, + "rescued": 1203, + "harmed": 738, + "missed_rescue": 79, + "avoided_harm": 39, + "selected_both_correct": 8897, + "selected_both_wrong": 414 + }, + { + "budget": 0.96, + "threshold": -0.70068359375, + "operator": "ge", + "actual_fraction": 0.953728362183755, + "metric": 0.26357052137555514, + "rescued": 1223, + "harmed": 754, + "missed_rescue": 59, + "avoided_harm": 23, + "selected_both_correct": 9055, + "selected_both_wrong": 428 + }, + { + "budget": 0.97, + "threshold": -0.75048828125, + "operator": "ge", + "actual_fraction": 0.96271637816245, + "metric": 0.26400290307863034, + "rescued": 1231, + "harmed": 758, + "missed_rescue": 51, + "avoided_harm": 19, + "selected_both_correct": 9150, + "selected_both_wrong": 429 + }, + { + "budget": 0.98, + "threshold": -0.7841796875, + "operator": "ge", + "actual_fraction": 0.9700399467376831, + "metric": 0.2647475724296899, + "rescued": 1237, + "harmed": 760, + "missed_rescue": 45, + "avoided_harm": 17, + "selected_both_correct": 9227, + "selected_both_wrong": 432 + }, + { + "budget": 0.99, + "threshold": -0.85107421875, + "operator": "ge", + "actual_fraction": 0.987017310252996, + "metric": 0.2649306125590882, + "rescued": 1257, + "harmed": 772, + "missed_rescue": 25, + "avoided_harm": 5, + "selected_both_correct": 9387, + "selected_both_wrong": 444 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + }, + "learned": { + "selected_budget_index": 50, + "validation_area_0_25": 0.23466194499464404, + "validation_point": { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.33126660689663334, + "rescued": 304, + "harmed": 95, + "missed_rescue": 3, + "avoided_harm": 125, + "selected_both_correct": 916, + "selected_both_wrong": 85 + }, + "test_point": { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4876830892143808, + "metric": 0.29875800068449215, + "rescued": 1249, + "harmed": 337, + "missed_rescue": 33, + "avoided_harm": 440, + "selected_both_correct": 3843, + "selected_both_wrong": 431 + }, + "validation_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18254072168965788, + "rescued": 0, + "harmed": 0, + "missed_rescue": 307, + "avoided_harm": 220, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.3274500370025635, + "operator": "ge", + "actual_fraction": 0.009985734664764621, + "metric": 0.18570977475658457, + "rescued": 12, + "harmed": 1, + "missed_rescue": 295, + "avoided_harm": 219, + "selected_both_correct": 12, + "selected_both_wrong": 3 + }, + { + "budget": 0.02, + "threshold": 2.223353862762451, + "operator": "ge", + "actual_fraction": 0.019971469329529243, + "metric": 0.19072678038582708, + "rescued": 24, + "harmed": 1, + "missed_rescue": 283, + "avoided_harm": 219, + "selected_both_correct": 26, + "selected_both_wrong": 5 + }, + { + "budget": 0.03, + "threshold": 2.105621099472046, + "operator": "ge", + "actual_fraction": 0.029957203994293864, + "metric": 0.1947361119928147, + "rescued": 33, + "harmed": 1, + "missed_rescue": 274, + "avoided_harm": 219, + "selected_both_correct": 40, + "selected_both_wrong": 10 + }, + { + "budget": 0.04, + "threshold": 1.9981958866119385, + "operator": "ge", + "actual_fraction": 0.039942938659058486, + "metric": 0.20027934358652552, + "rescued": 44, + "harmed": 1, + "missed_rescue": 263, + "avoided_harm": 219, + "selected_both_correct": 52, + "selected_both_wrong": 15 + }, + { + "budget": 0.05, + "threshold": 1.949358582496643, + "operator": "ge", + "actual_fraction": 0.04992867332382311, + "metric": 0.21085585634765963, + "rescued": 57, + "harmed": 1, + "missed_rescue": 250, + "avoided_harm": 219, + "selected_both_correct": 65, + "selected_both_wrong": 17 + }, + { + "budget": 0.06, + "threshold": 1.8504760265350342, + "operator": "ge", + "actual_fraction": 0.05991440798858773, + "metric": 0.2155935788086818, + "rescued": 68, + "harmed": 2, + "missed_rescue": 239, + "avoided_harm": 218, + "selected_both_correct": 78, + "selected_both_wrong": 20 + }, + { + "budget": 0.07, + "threshold": 1.6561405658721924, + "operator": "ge", + "actual_fraction": 0.06990014265335236, + "metric": 0.2201008442387753, + "rescued": 76, + "harmed": 4, + "missed_rescue": 231, + "avoided_harm": 216, + "selected_both_correct": 93, + "selected_both_wrong": 23 + }, + { + "budget": 0.08, + "threshold": 1.5329420566558838, + "operator": "ge", + "actual_fraction": 0.07988587731811697, + "metric": 0.22831044506816361, + "rescued": 90, + "harmed": 5, + "missed_rescue": 217, + "avoided_harm": 215, + "selected_both_correct": 106, + "selected_both_wrong": 23 + }, + { + "budget": 0.09, + "threshold": 1.380273461341858, + "operator": "ge", + "actual_fraction": 0.0898716119828816, + "metric": 0.23143136333139114, + "rescued": 96, + "harmed": 6, + "missed_rescue": 211, + "avoided_harm": 214, + "selected_both_correct": 124, + "selected_both_wrong": 26 + }, + { + "budget": 0.1, + "threshold": 1.32539701461792, + "operator": "ge", + "actual_fraction": 0.09985734664764621, + "metric": 0.23760484070252644, + "rescued": 104, + "harmed": 8, + "missed_rescue": 203, + "avoided_harm": 212, + "selected_both_correct": 140, + "selected_both_wrong": 28 + }, + { + "budget": 0.11, + "threshold": 1.2173664569854736, + "operator": "ge", + "actual_fraction": 0.10984308131241084, + "metric": 0.23824320355020984, + "rescued": 110, + "harmed": 9, + "missed_rescue": 197, + "avoided_harm": 211, + "selected_both_correct": 161, + "selected_both_wrong": 28 + }, + { + "budget": 0.12, + "threshold": 1.1719112396240234, + "operator": "ge", + "actual_fraction": 0.11982881597717546, + "metric": 0.2407100587309389, + "rescued": 115, + "harmed": 9, + "missed_rescue": 192, + "avoided_harm": 211, + "selected_both_correct": 181, + "selected_both_wrong": 31 + }, + { + "budget": 0.13, + "threshold": 1.1186922788619995, + "operator": "ge", + "actual_fraction": 0.12981455064194009, + "metric": 0.24134727061556333, + "rescued": 121, + "harmed": 11, + "missed_rescue": 186, + "avoided_harm": 209, + "selected_both_correct": 199, + "selected_both_wrong": 33 + }, + { + "budget": 0.14, + "threshold": 1.0748379230499268, + "operator": "ge", + "actual_fraction": 0.1398002853067047, + "metric": 0.24423106111883697, + "rescued": 128, + "harmed": 16, + "missed_rescue": 179, + "avoided_harm": 204, + "selected_both_correct": 213, + "selected_both_wrong": 35 + }, + { + "budget": 0.15, + "threshold": 1.0427184104919434, + "operator": "ge", + "actual_fraction": 0.14978601997146934, + "metric": 0.24892068530842207, + "rescued": 135, + "harmed": 16, + "missed_rescue": 172, + "avoided_harm": 204, + "selected_both_correct": 234, + "selected_both_wrong": 35 + }, + { + "budget": 0.16, + "threshold": 1.0111123323440552, + "operator": "ge", + "actual_fraction": 0.15977175463623394, + "metric": 0.252253667728857, + "rescued": 141, + "harmed": 16, + "missed_rescue": 166, + "avoided_harm": 204, + "selected_both_correct": 254, + "selected_both_wrong": 37 + }, + { + "budget": 0.17, + "threshold": 0.9669641852378845, + "operator": "ge", + "actual_fraction": 0.16975748930099857, + "metric": 0.25275417712816833, + "rescued": 146, + "harmed": 18, + "missed_rescue": 161, + "avoided_harm": 202, + "selected_both_correct": 272, + "selected_both_wrong": 40 + }, + { + "budget": 0.18, + "threshold": 0.9490972757339478, + "operator": "ge", + "actual_fraction": 0.1797432239657632, + "metric": 0.25643535926629674, + "rescued": 152, + "harmed": 19, + "missed_rescue": 155, + "avoided_harm": 201, + "selected_both_correct": 292, + "selected_both_wrong": 41 + }, + { + "budget": 0.19, + "threshold": 0.9165292382240295, + "operator": "ge", + "actual_fraction": 0.18972895863052783, + "metric": 0.2576337529504327, + "rescued": 159, + "harmed": 22, + "missed_rescue": 148, + "avoided_harm": 198, + "selected_both_correct": 310, + "selected_both_wrong": 41 + }, + { + "budget": 0.2, + "threshold": 0.8978593349456787, + "operator": "ge", + "actual_fraction": 0.19971469329529243, + "metric": 0.2581122448979592, + "rescued": 164, + "harmed": 24, + "missed_rescue": 143, + "avoided_harm": 196, + "selected_both_correct": 330, + "selected_both_wrong": 42 + }, + { + "budget": 0.21, + "threshold": 0.854847252368927, + "operator": "ge", + "actual_fraction": 0.20970042796005706, + "metric": 0.2602060885029815, + "rescued": 169, + "harmed": 25, + "missed_rescue": 138, + "avoided_harm": 195, + "selected_both_correct": 350, + "selected_both_wrong": 44 + }, + { + "budget": 0.22, + "threshold": 0.8254434466362, + "operator": "ge", + "actual_fraction": 0.21968616262482168, + "metric": 0.2635325636000918, + "rescued": 174, + "harmed": 25, + "missed_rescue": 133, + "avoided_harm": 195, + "selected_both_correct": 371, + "selected_both_wrong": 46 + }, + { + "budget": 0.23, + "threshold": 0.8046320080757141, + "operator": "ge", + "actual_fraction": 0.2296718972895863, + "metric": 0.24799578059071728, + "rescued": 178, + "harmed": 29, + "missed_rescue": 129, + "avoided_harm": 191, + "selected_both_correct": 391, + "selected_both_wrong": 46 + }, + { + "budget": 0.24, + "threshold": 0.7761231660842896, + "operator": "ge", + "actual_fraction": 0.2396576319543509, + "metric": 0.25061334511913563, + "rescued": 184, + "harmed": 31, + "missed_rescue": 123, + "avoided_harm": 189, + "selected_both_correct": 410, + "selected_both_wrong": 47 + }, + { + "budget": 0.25, + "threshold": 0.7452236413955688, + "operator": "ge", + "actual_fraction": 0.24964336661911554, + "metric": 0.25309147374364765, + "rescued": 190, + "harmed": 32, + "missed_rescue": 117, + "avoided_harm": 188, + "selected_both_correct": 430, + "selected_both_wrong": 48 + }, + { + "budget": 0.26, + "threshold": 0.7167489528656006, + "operator": "ge", + "actual_fraction": 0.25962910128388017, + "metric": 0.25499133574730426, + "rescued": 196, + "harmed": 34, + "missed_rescue": 111, + "avoided_harm": 186, + "selected_both_correct": 448, + "selected_both_wrong": 50 + }, + { + "budget": 0.27, + "threshold": 0.6837625503540039, + "operator": "ge", + "actual_fraction": 0.26961483594864477, + "metric": 0.25870776744375423, + "rescued": 201, + "harmed": 34, + "missed_rescue": 106, + "avoided_harm": 186, + "selected_both_correct": 468, + "selected_both_wrong": 53 + }, + { + "budget": 0.28, + "threshold": 0.633089005947113, + "operator": "ge", + "actual_fraction": 0.2796005706134094, + "metric": 0.25965649053463696, + "rescued": 205, + "harmed": 36, + "missed_rescue": 102, + "avoided_harm": 184, + "selected_both_correct": 488, + "selected_both_wrong": 55 + }, + { + "budget": 0.29, + "threshold": 0.5875619649887085, + "operator": "ge", + "actual_fraction": 0.289586305278174, + "metric": 0.2629594041536389, + "rescued": 213, + "harmed": 39, + "missed_rescue": 94, + "avoided_harm": 181, + "selected_both_correct": 505, + "selected_both_wrong": 55 + }, + { + "budget": 0.3, + "threshold": 0.5604347586631775, + "operator": "ge", + "actual_fraction": 0.2995720399429387, + "metric": 0.26353457364905875, + "rescued": 219, + "harmed": 45, + "missed_rescue": 88, + "avoided_harm": 175, + "selected_both_correct": 521, + "selected_both_wrong": 55 + }, + { + "budget": 0.31, + "threshold": 0.5320743918418884, + "operator": "ge", + "actual_fraction": 0.3095577746077033, + "metric": 0.26608946608946615, + "rescued": 225, + "harmed": 47, + "missed_rescue": 82, + "avoided_harm": 173, + "selected_both_correct": 540, + "selected_both_wrong": 56 + }, + { + "budget": 0.32, + "threshold": 0.49417412281036377, + "operator": "ge", + "actual_fraction": 0.3195435092724679, + "metric": 0.2673115302724549, + "rescued": 230, + "harmed": 51, + "missed_rescue": 77, + "avoided_harm": 169, + "selected_both_correct": 558, + "selected_both_wrong": 57 + }, + { + "budget": 0.33, + "threshold": 0.46743881702423096, + "operator": "ge", + "actual_fraction": 0.32952924393723254, + "metric": 0.26933684672815106, + "rescued": 236, + "harmed": 53, + "missed_rescue": 71, + "avoided_harm": 167, + "selected_both_correct": 577, + "selected_both_wrong": 58 + }, + { + "budget": 0.34, + "threshold": 0.4429023861885071, + "operator": "ge", + "actual_fraction": 0.33951497860199714, + "metric": 0.27352180339985216, + "rescued": 243, + "harmed": 54, + "missed_rescue": 64, + "avoided_harm": 166, + "selected_both_correct": 597, + "selected_both_wrong": 58 + }, + { + "budget": 0.35, + "threshold": 0.40694165229797363, + "operator": "ge", + "actual_fraction": 0.34950071326676174, + "metric": 0.275880713431076, + "rescued": 247, + "harmed": 60, + "missed_rescue": 60, + "avoided_harm": 160, + "selected_both_correct": 612, + "selected_both_wrong": 61 + }, + { + "budget": 0.36, + "threshold": 0.3611782193183899, + "operator": "ge", + "actual_fraction": 0.3594864479315264, + "metric": 0.27761867831050957, + "rescued": 253, + "harmed": 64, + "missed_rescue": 54, + "avoided_harm": 156, + "selected_both_correct": 629, + "selected_both_wrong": 62 + }, + { + "budget": 0.37, + "threshold": 0.3311936855316162, + "operator": "ge", + "actual_fraction": 0.369472182596291, + "metric": 0.28017336393459635, + "rescued": 257, + "harmed": 66, + "missed_rescue": 50, + "avoided_harm": 154, + "selected_both_correct": 650, + "selected_both_wrong": 63 + }, + { + "budget": 0.38, + "threshold": 0.28899651765823364, + "operator": "ge", + "actual_fraction": 0.37945791726105566, + "metric": 0.2825320512820513, + "rescued": 263, + "harmed": 70, + "missed_rescue": 44, + "avoided_harm": 150, + "selected_both_correct": 667, + "selected_both_wrong": 64 + }, + { + "budget": 0.39, + "threshold": 0.19986218214035034, + "operator": "ge", + "actual_fraction": 0.38944365192582026, + "metric": 0.28580355455355455, + "rescued": 267, + "harmed": 71, + "missed_rescue": 40, + "avoided_harm": 149, + "selected_both_correct": 689, + "selected_both_wrong": 65 + }, + { + "budget": 0.4, + "threshold": 0.157886803150177, + "operator": "ge", + "actual_fraction": 0.39942938659058486, + "metric": 0.3015271169266015, + "rescued": 271, + "harmed": 72, + "missed_rescue": 36, + "avoided_harm": 148, + "selected_both_correct": 709, + "selected_both_wrong": 68 + }, + { + "budget": 0.41, + "threshold": 0.12043046951293945, + "operator": "ge", + "actual_fraction": 0.4094151212553495, + "metric": 0.30656010656010657, + "rescued": 275, + "harmed": 74, + "missed_rescue": 32, + "avoided_harm": 146, + "selected_both_correct": 730, + "selected_both_wrong": 69 + }, + { + "budget": 0.42, + "threshold": 0.06623327732086182, + "operator": "ge", + "actual_fraction": 0.4194008559201141, + "metric": 0.30573593073593075, + "rescued": 277, + "harmed": 77, + "missed_rescue": 30, + "avoided_harm": 143, + "selected_both_correct": 750, + "selected_both_wrong": 72 + }, + { + "budget": 0.43, + "threshold": -0.01827472448348999, + "operator": "ge", + "actual_fraction": 0.42938659058487877, + "metric": 0.3122803645750104, + "rescued": 283, + "harmed": 79, + "missed_rescue": 24, + "avoided_harm": 141, + "selected_both_correct": 770, + "selected_both_wrong": 72 + }, + { + "budget": 0.44, + "threshold": -0.14177680015563965, + "operator": "ge", + "actual_fraction": 0.43937232524964337, + "metric": 0.32014301599541106, + "rescued": 290, + "harmed": 81, + "missed_rescue": 17, + "avoided_harm": 139, + "selected_both_correct": 788, + "selected_both_wrong": 73 + }, + { + "budget": 0.45, + "threshold": -0.270923376083374, + "operator": "ge", + "actual_fraction": 0.44935805991440797, + "metric": 0.3274352926559858, + "rescued": 295, + "harmed": 82, + "missed_rescue": 12, + "avoided_harm": 138, + "selected_both_correct": 808, + "selected_both_wrong": 75 + }, + { + "budget": 0.46, + "threshold": -0.4213377833366394, + "operator": "ge", + "actual_fraction": 0.4593437945791726, + "metric": 0.3269493177387914, + "rescued": 299, + "harmed": 85, + "missed_rescue": 8, + "avoided_harm": 135, + "selected_both_correct": 826, + "selected_both_wrong": 78 + }, + { + "budget": 0.47, + "threshold": -0.6602996587753296, + "operator": "ge", + "actual_fraction": 0.4693295292439372, + "metric": 0.3206497235233282, + "rescued": 299, + "harmed": 88, + "missed_rescue": 8, + "avoided_harm": 132, + "selected_both_correct": 849, + "selected_both_wrong": 80 + }, + { + "budget": 0.48, + "threshold": -0.8612939715385437, + "operator": "ge", + "actual_fraction": 0.4793152639087018, + "metric": 0.3201660892787428, + "rescued": 300, + "harmed": 89, + "missed_rescue": 7, + "avoided_harm": 131, + "selected_both_correct": 873, + "selected_both_wrong": 82 + }, + { + "budget": 0.49, + "threshold": -0.9715782999992371, + "operator": "ge", + "actual_fraction": 0.4893009985734665, + "metric": 0.325869430652523, + "rescued": 303, + "harmed": 91, + "missed_rescue": 4, + "avoided_harm": 129, + "selected_both_correct": 894, + "selected_both_wrong": 84 + }, + { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4992867332382311, + "metric": 0.33126660689663334, + "rescued": 304, + "harmed": 95, + "missed_rescue": 3, + "avoided_harm": 125, + "selected_both_correct": 916, + "selected_both_wrong": 85 + }, + { + "budget": 0.51, + "threshold": -1.6505446434020996, + "operator": "ge", + "actual_fraction": 0.5092724679029957, + "metric": 0.3264586577163139, + "rescued": 304, + "harmed": 98, + "missed_rescue": 3, + "avoided_harm": 122, + "selected_both_correct": 941, + "selected_both_wrong": 85 + }, + { + "budget": 0.52, + "threshold": -2.6140921115875244, + "operator": "ge", + "actual_fraction": 0.5192582025677603, + "metric": 0.3264586577163139, + "rescued": 304, + "harmed": 98, + "missed_rescue": 3, + "avoided_harm": 122, + "selected_both_correct": 968, + "selected_both_wrong": 86 + }, + { + "budget": 0.53, + "threshold": -2.6907107830047607, + "operator": "ge", + "actual_fraction": 0.5292439372325249, + "metric": 0.3235189795049722, + "rescued": 304, + "harmed": 102, + "missed_rescue": 3, + "avoided_harm": 118, + "selected_both_correct": 992, + "selected_both_wrong": 86 + }, + { + "budget": 0.54, + "threshold": -2.8344802856445312, + "operator": "ge", + "actual_fraction": 0.5392296718972895, + "metric": 0.3235189795049722, + "rescued": 304, + "harmed": 102, + "missed_rescue": 3, + "avoided_harm": 118, + "selected_both_correct": 1020, + "selected_both_wrong": 86 + }, + { + "budget": 0.55, + "threshold": -2.905412197113037, + "operator": "ge", + "actual_fraction": 0.5492154065620543, + "metric": 0.3203578942095772, + "rescued": 304, + "harmed": 106, + "missed_rescue": 3, + "avoided_harm": 114, + "selected_both_correct": 1044, + "selected_both_wrong": 86 + }, + { + "budget": 0.56, + "threshold": -2.9482638835906982, + "operator": "ge", + "actual_fraction": 0.5592011412268189, + "metric": 0.31783773291925466, + "rescued": 304, + "harmed": 107, + "missed_rescue": 3, + "avoided_harm": 113, + "selected_both_correct": 1071, + "selected_both_wrong": 86 + }, + { + "budget": 0.57, + "threshold": -2.9957408905029297, + "operator": "ge", + "actual_fraction": 0.5691868758915835, + "metric": 0.31989933342152455, + "rescued": 305, + "harmed": 111, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 1094, + "selected_both_wrong": 86 + }, + { + "budget": 0.58, + "threshold": -3.1037726402282715, + "operator": "ge", + "actual_fraction": 0.579172610556348, + "metric": 0.31989933342152455, + "rescued": 305, + "harmed": 111, + "missed_rescue": 2, + "avoided_harm": 109, + "selected_both_correct": 1122, + "selected_both_wrong": 86 + }, + { + "budget": 0.59, + "threshold": -3.1757895946502686, + "operator": "ge", + "actual_fraction": 0.5891583452211127, + "metric": 0.3184090005781182, + "rescued": 305, + "harmed": 113, + "missed_rescue": 2, + "avoided_harm": 107, + "selected_both_correct": 1148, + "selected_both_wrong": 86 + }, + { + "budget": 0.6, + "threshold": -3.253206729888916, + "operator": "ge", + "actual_fraction": 0.5991440798858774, + "metric": 0.3177705940408353, + "rescued": 305, + "harmed": 114, + "missed_rescue": 2, + "avoided_harm": 106, + "selected_both_correct": 1175, + "selected_both_wrong": 86 + }, + { + "budget": 0.61, + "threshold": -3.2825026512145996, + "operator": "ge", + "actual_fraction": 0.609129814550642, + "metric": 0.31694234643839414, + "rescued": 305, + "harmed": 115, + "missed_rescue": 2, + "avoided_harm": 105, + "selected_both_correct": 1202, + "selected_both_wrong": 86 + }, + { + "budget": 0.62, + "threshold": -3.322240114212036, + "operator": "ge", + "actual_fraction": 0.6191155492154066, + "metric": 0.3111129112312657, + "rescued": 305, + "harmed": 121, + "missed_rescue": 2, + "avoided_harm": 99, + "selected_both_correct": 1224, + "selected_both_wrong": 86 + }, + { + "budget": 0.63, + "threshold": -3.3503003120422363, + "operator": "ge", + "actual_fraction": 0.6291012838801712, + "metric": 0.30995657450601133, + "rescued": 305, + "harmed": 123, + "missed_rescue": 2, + "avoided_harm": 97, + "selected_both_correct": 1250, + "selected_both_wrong": 86 + }, + { + "budget": 0.64, + "threshold": -3.377089738845825, + "operator": "ge", + "actual_fraction": 0.6390870185449358, + "metric": 0.3083114768968865, + "rescued": 305, + "harmed": 126, + "missed_rescue": 2, + "avoided_harm": 94, + "selected_both_correct": 1275, + "selected_both_wrong": 86 + }, + { + "budget": 0.65, + "threshold": -3.4055864810943604, + "operator": "ge", + "actual_fraction": 0.6490727532097005, + "metric": 0.30666904687523244, + "rescued": 305, + "harmed": 128, + "missed_rescue": 2, + "avoided_harm": 92, + "selected_both_correct": 1301, + "selected_both_wrong": 86 + }, + { + "budget": 0.66, + "threshold": -3.430248498916626, + "operator": "ge", + "actual_fraction": 0.6590584878744651, + "metric": 0.30339954163483573, + "rescued": 305, + "harmed": 131, + "missed_rescue": 2, + "avoided_harm": 89, + "selected_both_correct": 1326, + "selected_both_wrong": 86 + }, + { + "budget": 0.67, + "threshold": -3.4584414958953857, + "operator": "ge", + "actual_fraction": 0.6690442225392297, + "metric": 0.3011204481792717, + "rescued": 305, + "harmed": 135, + "missed_rescue": 2, + "avoided_harm": 85, + "selected_both_correct": 1350, + "selected_both_wrong": 86 + }, + { + "budget": 0.68, + "threshold": -3.502922773361206, + "operator": "ge", + "actual_fraction": 0.6790299572039943, + "metric": 0.2968337073049115, + "rescued": 305, + "harmed": 140, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 1373, + "selected_both_wrong": 86 + }, + { + "budget": 0.69, + "threshold": -3.532865285873413, + "operator": "ge", + "actual_fraction": 0.6890156918687589, + "metric": 0.2968337073049115, + "rescued": 305, + "harmed": 140, + "missed_rescue": 2, + "avoided_harm": 80, + "selected_both_correct": 1401, + "selected_both_wrong": 86 + }, + { + "budget": 0.7, + "threshold": -3.5550942420959473, + "operator": "ge", + "actual_fraction": 0.6990014265335235, + "metric": 0.2951499808548299, + "rescued": 305, + "harmed": 143, + "missed_rescue": 2, + "avoided_harm": 77, + "selected_both_correct": 1426, + "selected_both_wrong": 86 + }, + { + "budget": 0.71, + "threshold": -3.566344976425171, + "operator": "ge", + "actual_fraction": 0.7089871611982882, + "metric": 0.2985697751322751, + "rescued": 306, + "harmed": 144, + "missed_rescue": 1, + "avoided_harm": 76, + "selected_both_correct": 1452, + "selected_both_wrong": 86 + }, + { + "budget": 0.72, + "threshold": -3.6056067943573, + "operator": "ge", + "actual_fraction": 0.7189728958630528, + "metric": 0.29288998820248824, + "rescued": 306, + "harmed": 150, + "missed_rescue": 1, + "avoided_harm": 70, + "selected_both_correct": 1474, + "selected_both_wrong": 86 + }, + { + "budget": 0.73, + "threshold": -3.628530979156494, + "operator": "ge", + "actual_fraction": 0.7289586305278174, + "metric": 0.2916863054994102, + "rescued": 306, + "harmed": 156, + "missed_rescue": 1, + "avoided_harm": 64, + "selected_both_correct": 1496, + "selected_both_wrong": 86 + }, + { + "budget": 0.74, + "threshold": -3.6580705642700195, + "operator": "ge", + "actual_fraction": 0.738944365192582, + "metric": 0.2908398260842505, + "rescued": 306, + "harmed": 158, + "missed_rescue": 1, + "avoided_harm": 62, + "selected_both_correct": 1521, + "selected_both_wrong": 87 + }, + { + "budget": 0.75, + "threshold": -3.6900978088378906, + "operator": "ge", + "actual_fraction": 0.7489300998573466, + "metric": 0.29144523554686896, + "rescued": 307, + "harmed": 161, + "missed_rescue": 0, + "avoided_harm": 59, + "selected_both_correct": 1545, + "selected_both_wrong": 87 + }, + { + "budget": 0.76, + "threshold": -3.7050747871398926, + "operator": "ge", + "actual_fraction": 0.7589158345221113, + "metric": 0.28771367521367525, + "rescued": 307, + "harmed": 164, + "missed_rescue": 0, + "avoided_harm": 56, + "selected_both_correct": 1569, + "selected_both_wrong": 88 + }, + { + "budget": 0.77, + "threshold": -3.720116138458252, + "operator": "ge", + "actual_fraction": 0.7689015691868759, + "metric": 0.28342343829529665, + "rescued": 307, + "harmed": 170, + "missed_rescue": 0, + "avoided_harm": 50, + "selected_both_correct": 1591, + "selected_both_wrong": 88 + }, + { + "budget": 0.78, + "threshold": -3.7399919033050537, + "operator": "ge", + "actual_fraction": 0.7788873038516405, + "metric": 0.27967055835908294, + "rescued": 307, + "harmed": 175, + "missed_rescue": 0, + "avoided_harm": 45, + "selected_both_correct": 1614, + "selected_both_wrong": 88 + }, + { + "budget": 0.79, + "threshold": -3.799569606781006, + "operator": "ge", + "actual_fraction": 0.7888730385164051, + "metric": 0.2754483860214125, + "rescued": 307, + "harmed": 180, + "missed_rescue": 0, + "avoided_harm": 40, + "selected_both_correct": 1637, + "selected_both_wrong": 88 + }, + { + "budget": 0.8, + "threshold": -3.8980000019073486, + "operator": "ge", + "actual_fraction": 0.7988587731811697, + "metric": 0.2750612892472189, + "rescued": 307, + "harmed": 181, + "missed_rescue": 0, + "avoided_harm": 39, + "selected_both_correct": 1664, + "selected_both_wrong": 88 + }, + { + "budget": 0.81, + "threshold": -4.0607757568359375, + "operator": "ge", + "actual_fraction": 0.8088445078459344, + "metric": 0.27434520884520885, + "rescued": 307, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 1687, + "selected_both_wrong": 88 + }, + { + "budget": 0.82, + "threshold": -8.964031219482422, + "operator": "ge", + "actual_fraction": 0.818830242510699, + "metric": 0.27434520884520885, + "rescued": 307, + "harmed": 186, + "missed_rescue": 0, + "avoided_harm": 34, + "selected_both_correct": 1715, + "selected_both_wrong": 88 + }, + { + "budget": 0.83, + "threshold": -9.02132511138916, + "operator": "ge", + "actual_fraction": 0.8288159771754636, + "metric": 0.27363625362132826, + "rescued": 307, + "harmed": 187, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 1742, + "selected_both_wrong": 88 + }, + { + "budget": 0.84, + "threshold": -9.09000015258789, + "operator": "ge", + "actual_fraction": 0.8388017118402282, + "metric": 0.2722230802027571, + "rescued": 307, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1768, + "selected_both_wrong": 88 + }, + { + "budget": 0.85, + "threshold": -9.12938404083252, + "operator": "ge", + "actual_fraction": 0.8487874465049928, + "metric": 0.2722230802027571, + "rescued": 307, + "harmed": 189, + "missed_rescue": 0, + "avoided_harm": 31, + "selected_both_correct": 1796, + "selected_both_wrong": 88 + }, + { + "budget": 0.86, + "threshold": -9.17154312133789, + "operator": "ge", + "actual_fraction": 0.8587731811697575, + "metric": 0.2708398536448072, + "rescued": 307, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 29, + "selected_both_correct": 1822, + "selected_both_wrong": 88 + }, + { + "budget": 0.87, + "threshold": -9.273024559020996, + "operator": "ge", + "actual_fraction": 0.8687589158345221, + "metric": 0.2708398536448072, + "rescued": 307, + "harmed": 191, + "missed_rescue": 0, + "avoided_harm": 29, + "selected_both_correct": 1850, + "selected_both_wrong": 88 + }, + { + "budget": 0.88, + "threshold": -9.444879531860352, + "operator": "ge", + "actual_fraction": 0.8787446504992867, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1877, + "selected_both_wrong": 88 + }, + { + "budget": 0.89, + "threshold": -9.566991806030273, + "operator": "ge", + "actual_fraction": 0.8887303851640513, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1905, + "selected_both_wrong": 88 + }, + { + "budget": 0.9, + "threshold": -9.675031661987305, + "operator": "ge", + "actual_fraction": 0.8987161198288159, + "metric": 0.27016508981214865, + "rescued": 307, + "harmed": 192, + "missed_rescue": 0, + "avoided_harm": 28, + "selected_both_correct": 1933, + "selected_both_wrong": 88 + }, + { + "budget": 0.91, + "threshold": -9.739017486572266, + "operator": "ge", + "actual_fraction": 0.9087018544935807, + "metric": 0.2694835977031099, + "rescued": 307, + "harmed": 193, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 1960, + "selected_both_wrong": 88 + }, + { + "budget": 0.92, + "threshold": -9.787779808044434, + "operator": "ge", + "actual_fraction": 0.9186875891583453, + "metric": 0.2680528165105032, + "rescued": 307, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 1985, + "selected_both_wrong": 88 + }, + { + "budget": 0.93, + "threshold": -9.832305908203125, + "operator": "ge", + "actual_fraction": 0.9286733238231099, + "metric": 0.2680528165105032, + "rescued": 307, + "harmed": 196, + "missed_rescue": 0, + "avoided_harm": 24, + "selected_both_correct": 2013, + "selected_both_wrong": 88 + }, + { + "budget": 0.94, + "threshold": -9.88642692565918, + "operator": "ge", + "actual_fraction": 0.9386590584878745, + "metric": 0.2673844613676863, + "rescued": 307, + "harmed": 201, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 2036, + "selected_both_wrong": 88 + }, + { + "budget": 0.95, + "threshold": -9.976344108581543, + "operator": "ge", + "actual_fraction": 0.948644793152639, + "metric": 0.26575274061687104, + "rescued": 307, + "harmed": 203, + "missed_rescue": 0, + "avoided_harm": 17, + "selected_both_correct": 2062, + "selected_both_wrong": 88 + }, + { + "budget": 0.96, + "threshold": -10.03441047668457, + "operator": "ge", + "actual_fraction": 0.9586305278174037, + "metric": 0.261865308604439, + "rescued": 307, + "harmed": 209, + "missed_rescue": 0, + "avoided_harm": 11, + "selected_both_correct": 2084, + "selected_both_wrong": 88 + }, + { + "budget": 0.97, + "threshold": -10.085777282714844, + "operator": "ge", + "actual_fraction": 0.9686162624821684, + "metric": 0.2589514652014652, + "rescued": 307, + "harmed": 212, + "missed_rescue": 0, + "avoided_harm": 8, + "selected_both_correct": 2109, + "selected_both_wrong": 88 + }, + { + "budget": 0.98, + "threshold": -10.1488676071167, + "operator": "ge", + "actual_fraction": 0.978601997146933, + "metric": 0.25830826953457287, + "rescued": 307, + "harmed": 213, + "missed_rescue": 0, + "avoided_harm": 7, + "selected_both_correct": 2136, + "selected_both_wrong": 88 + }, + { + "budget": 0.99, + "threshold": -10.265552520751953, + "operator": "ge", + "actual_fraction": 0.9885877318116976, + "metric": 0.25668767695229905, + "rescued": 307, + "harmed": 216, + "missed_rescue": 0, + "avoided_harm": 4, + "selected_both_correct": 2161, + "selected_both_wrong": 88 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.25448626222520154, + "rescued": 307, + "harmed": 220, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 2189, + "selected_both_wrong": 88 + } + ], + "test_curve": [ + { + "budget": 0.0, + "threshold": null, + "operator": "never", + "actual_fraction": 0.0, + "metric": 0.18129624718318377, + "rescued": 0, + "harmed": 0, + "missed_rescue": 1282, + "avoided_harm": 777, + "selected_both_correct": 0, + "selected_both_wrong": 0 + }, + { + "budget": 0.01, + "threshold": 2.3274500370025635, + "operator": "ge", + "actual_fraction": 0.013648468708388815, + "metric": 0.18693471712069915, + "rescued": 72, + "harmed": 3, + "missed_rescue": 1210, + "avoided_harm": 774, + "selected_both_correct": 69, + "selected_both_wrong": 20 + }, + { + "budget": 0.02, + "threshold": 2.223353862762451, + "operator": "ge", + "actual_fraction": 0.019640479360852198, + "metric": 0.18781313491575913, + "rescued": 98, + "harmed": 7, + "missed_rescue": 1184, + "avoided_harm": 770, + "selected_both_correct": 104, + "selected_both_wrong": 27 + }, + { + "budget": 0.03, + "threshold": 2.105621099472046, + "operator": "ge", + "actual_fraction": 0.031291611185086554, + "metric": 0.1913128749335621, + "rescued": 154, + "harmed": 10, + "missed_rescue": 1128, + "avoided_harm": 767, + "selected_both_correct": 170, + "selected_both_wrong": 42 + }, + { + "budget": 0.04, + "threshold": 1.9981958866119385, + "operator": "ge", + "actual_fraction": 0.04061251664447404, + "metric": 0.19266008228682668, + "rescued": 193, + "harmed": 16, + "missed_rescue": 1089, + "avoided_harm": 761, + "selected_both_correct": 219, + "selected_both_wrong": 60 + }, + { + "budget": 0.05, + "threshold": 1.949358582496643, + "operator": "ge", + "actual_fraction": 0.045938748335552594, + "metric": 0.19531128485766117, + "rescued": 224, + "harmed": 18, + "missed_rescue": 1058, + "avoided_harm": 759, + "selected_both_correct": 249, + "selected_both_wrong": 61 + }, + { + "budget": 0.06, + "threshold": 1.8504760265350342, + "operator": "ge", + "actual_fraction": 0.0559254327563249, + "metric": 0.19800731707634547, + "rescued": 268, + "harmed": 23, + "missed_rescue": 1014, + "avoided_harm": 754, + "selected_both_correct": 302, + "selected_both_wrong": 79 + }, + { + "budget": 0.07, + "threshold": 1.6561405658721924, + "operator": "ge", + "actual_fraction": 0.07456724367509987, + "metric": 0.19763843353437874, + "rescued": 336, + "harmed": 37, + "missed_rescue": 946, + "avoided_harm": 740, + "selected_both_correct": 409, + "selected_both_wrong": 114 + }, + { + "budget": 0.08, + "threshold": 1.5329420566558838, + "operator": "ge", + "actual_fraction": 0.08688415446071904, + "metric": 0.1974112776278713, + "rescued": 386, + "harmed": 48, + "missed_rescue": 896, + "avoided_harm": 729, + "selected_both_correct": 472, + "selected_both_wrong": 138 + }, + { + "budget": 0.09, + "threshold": 1.380273461341858, + "operator": "ge", + "actual_fraction": 0.09587217043941411, + "metric": 0.20030220967092682, + "rescued": 421, + "harmed": 54, + "missed_rescue": 861, + "avoided_harm": 723, + "selected_both_correct": 529, + "selected_both_wrong": 148 + }, + { + "budget": 0.1, + "threshold": 1.32539701461792, + "operator": "ge", + "actual_fraction": 0.09920106524633822, + "metric": 0.20211074868345033, + "rescued": 432, + "harmed": 56, + "missed_rescue": 850, + "avoided_harm": 721, + "selected_both_correct": 552, + "selected_both_wrong": 152 + }, + { + "budget": 0.11, + "threshold": 1.2173664569854736, + "operator": "ge", + "actual_fraction": 0.11284953395472704, + "metric": 0.20834032699131655, + "rescued": 471, + "harmed": 62, + "missed_rescue": 811, + "avoided_harm": 715, + "selected_both_correct": 659, + "selected_both_wrong": 164 + }, + { + "budget": 0.12, + "threshold": 1.1719112396240234, + "operator": "ge", + "actual_fraction": 0.12350199733688415, + "metric": 0.21121674052765757, + "rescued": 503, + "harmed": 67, + "missed_rescue": 779, + "avoided_harm": 710, + "selected_both_correct": 745, + "selected_both_wrong": 169 + }, + { + "budget": 0.13, + "threshold": 1.1186922788619995, + "operator": "ge", + "actual_fraction": 0.13315579227696406, + "metric": 0.21276215593410724, + "rescued": 531, + "harmed": 77, + "missed_rescue": 751, + "avoided_harm": 700, + "selected_both_correct": 818, + "selected_both_wrong": 174 + }, + { + "budget": 0.14, + "threshold": 1.0748379230499268, + "operator": "ge", + "actual_fraction": 0.14347536617842876, + "metric": 0.21511521078428622, + "rescued": 556, + "harmed": 79, + "missed_rescue": 726, + "avoided_harm": 698, + "selected_both_correct": 907, + "selected_both_wrong": 182 + }, + { + "budget": 0.15, + "threshold": 1.0427184104919434, + "operator": "ge", + "actual_fraction": 0.15246338215712382, + "metric": 0.21738179676765032, + "rescued": 577, + "harmed": 81, + "missed_rescue": 705, + "avoided_harm": 696, + "selected_both_correct": 988, + "selected_both_wrong": 186 + }, + { + "budget": 0.16, + "threshold": 1.0111123323440552, + "operator": "ge", + "actual_fraction": 0.1604527296937417, + "metric": 0.22260454951009195, + "rescued": 597, + "harmed": 83, + "missed_rescue": 685, + "avoided_harm": 694, + "selected_both_correct": 1055, + "selected_both_wrong": 193 + }, + { + "budget": 0.17, + "threshold": 0.9669641852378845, + "operator": "ge", + "actual_fraction": 0.1780958721704394, + "metric": 0.22627631882842034, + "rescued": 641, + "harmed": 97, + "missed_rescue": 641, + "avoided_harm": 680, + "selected_both_correct": 1196, + "selected_both_wrong": 206 + }, + { + "budget": 0.18, + "threshold": 0.9490972757339478, + "operator": "ge", + "actual_fraction": 0.18442077230359522, + "metric": 0.2270967395967396, + "rescued": 657, + "harmed": 103, + "missed_rescue": 625, + "avoided_harm": 674, + "selected_both_correct": 1246, + "selected_both_wrong": 210 + }, + { + "budget": 0.19, + "threshold": 0.9165292382240295, + "operator": "ge", + "actual_fraction": 0.2000665778961385, + "metric": 0.23397114982462924, + "rescued": 697, + "harmed": 110, + "missed_rescue": 585, + "avoided_harm": 667, + "selected_both_correct": 1378, + "selected_both_wrong": 219 + }, + { + "budget": 0.2, + "threshold": 0.8978593349456787, + "operator": "ge", + "actual_fraction": 0.20439414114513982, + "metric": 0.23512276426997616, + "rescued": 707, + "harmed": 112, + "missed_rescue": 575, + "avoided_harm": 665, + "selected_both_correct": 1413, + "selected_both_wrong": 224 + }, + { + "budget": 0.21, + "threshold": 0.854847252368927, + "operator": "ge", + "actual_fraction": 0.21937416777629826, + "metric": 0.24202469545559713, + "rescued": 747, + "harmed": 119, + "missed_rescue": 535, + "avoided_harm": 658, + "selected_both_correct": 1539, + "selected_both_wrong": 231 + }, + { + "budget": 0.22, + "threshold": 0.8254434466362, + "operator": "ge", + "actual_fraction": 0.22769640479360853, + "metric": 0.24353835097564847, + "rescued": 766, + "harmed": 124, + "missed_rescue": 516, + "avoided_harm": 653, + "selected_both_correct": 1611, + "selected_both_wrong": 235 + }, + { + "budget": 0.23, + "threshold": 0.8046320080757141, + "operator": "ge", + "actual_fraction": 0.23601864181091878, + "metric": 0.2454657390209296, + "rescued": 787, + "harmed": 129, + "missed_rescue": 495, + "avoided_harm": 648, + "selected_both_correct": 1680, + "selected_both_wrong": 240 + }, + { + "budget": 0.24, + "threshold": 0.7761231660842896, + "operator": "ge", + "actual_fraction": 0.24533954727030627, + "metric": 0.2483414219175777, + "rescued": 812, + "harmed": 133, + "missed_rescue": 470, + "avoided_harm": 644, + "selected_both_correct": 1760, + "selected_both_wrong": 243 + }, + { + "budget": 0.25, + "threshold": 0.7452236413955688, + "operator": "ge", + "actual_fraction": 0.25432756324900135, + "metric": 0.2504846177578749, + "rescued": 831, + "harmed": 135, + "missed_rescue": 451, + "avoided_harm": 642, + "selected_both_correct": 1839, + "selected_both_wrong": 251 + }, + { + "budget": 0.26, + "threshold": 0.7167489528656006, + "operator": "ge", + "actual_fraction": 0.2636484687083888, + "metric": 0.252142216376585, + "rescued": 855, + "harmed": 144, + "missed_rescue": 427, + "avoided_harm": 633, + "selected_both_correct": 1914, + "selected_both_wrong": 255 + }, + { + "budget": 0.27, + "threshold": 0.6837625503540039, + "operator": "ge", + "actual_fraction": 0.275965379494008, + "metric": 0.25276746625574176, + "rescued": 880, + "harmed": 155, + "missed_rescue": 402, + "avoided_harm": 622, + "selected_both_correct": 2019, + "selected_both_wrong": 262 + }, + { + "budget": 0.28, + "threshold": 0.633089005947113, + "operator": "ge", + "actual_fraction": 0.29427430093209056, + "metric": 0.25702642690899347, + "rescued": 927, + "harmed": 172, + "missed_rescue": 355, + "avoided_harm": 605, + "selected_both_correct": 2167, + "selected_both_wrong": 270 + }, + { + "budget": 0.29, + "threshold": 0.5875619649887085, + "operator": "ge", + "actual_fraction": 0.31091877496671105, + "metric": 0.26183035419940204, + "rescued": 969, + "harmed": 193, + "missed_rescue": 313, + "avoided_harm": 584, + "selected_both_correct": 2296, + "selected_both_wrong": 278 + }, + { + "budget": 0.3, + "threshold": 0.5604347586631775, + "operator": "ge", + "actual_fraction": 0.322237017310253, + "metric": 0.2672887365132105, + "rescued": 995, + "harmed": 204, + "missed_rescue": 287, + "avoided_harm": 573, + "selected_both_correct": 2389, + "selected_both_wrong": 284 + }, + { + "budget": 0.31, + "threshold": 0.5320743918418884, + "operator": "ge", + "actual_fraction": 0.33388814913448733, + "metric": 0.27039536303602457, + "rescued": 1020, + "harmed": 216, + "missed_rescue": 262, + "avoided_harm": 561, + "selected_both_correct": 2480, + "selected_both_wrong": 296 + }, + { + "budget": 0.32, + "threshold": 0.49417412281036377, + "operator": "ge", + "actual_fraction": 0.34354194407456723, + "metric": 0.2713201570387918, + "rescued": 1038, + "harmed": 227, + "missed_rescue": 244, + "avoided_harm": 550, + "selected_both_correct": 2562, + "selected_both_wrong": 301 + }, + { + "budget": 0.33, + "threshold": 0.46743881702423096, + "operator": "ge", + "actual_fraction": 0.3492010652463382, + "metric": 0.2722481828097158, + "rescued": 1046, + "harmed": 230, + "missed_rescue": 236, + "avoided_harm": 547, + "selected_both_correct": 2610, + "selected_both_wrong": 310 + }, + { + "budget": 0.34, + "threshold": 0.4429023861885071, + "operator": "ge", + "actual_fraction": 0.3545272969374168, + "metric": 0.274417471554269, + "rescued": 1061, + "harmed": 239, + "missed_rescue": 221, + "avoided_harm": 538, + "selected_both_correct": 2648, + "selected_both_wrong": 312 + }, + { + "budget": 0.35, + "threshold": 0.40694165229797363, + "operator": "ge", + "actual_fraction": 0.3648468708388815, + "metric": 0.2782223498666452, + "rescued": 1081, + "harmed": 252, + "missed_rescue": 201, + "avoided_harm": 525, + "selected_both_correct": 2730, + "selected_both_wrong": 321 + }, + { + "budget": 0.36, + "threshold": 0.3611782193183899, + "operator": "ge", + "actual_fraction": 0.37217043941411454, + "metric": 0.2795152621397537, + "rescued": 1094, + "harmed": 257, + "missed_rescue": 188, + "avoided_harm": 520, + "selected_both_correct": 2793, + "selected_both_wrong": 328 + }, + { + "budget": 0.37, + "threshold": 0.3311936855316162, + "operator": "ge", + "actual_fraction": 0.37683089214380827, + "metric": 0.28090947933574756, + "rescued": 1104, + "harmed": 261, + "missed_rescue": 178, + "avoided_harm": 516, + "selected_both_correct": 2831, + "selected_both_wrong": 332 + }, + { + "budget": 0.38, + "threshold": 0.28899651765823364, + "operator": "ge", + "actual_fraction": 0.3854860186418109, + "metric": 0.28295009794869297, + "rescued": 1121, + "harmed": 269, + "missed_rescue": 161, + "avoided_harm": 508, + "selected_both_correct": 2904, + "selected_both_wrong": 338 + }, + { + "budget": 0.39, + "threshold": 0.19986218214035034, + "operator": "ge", + "actual_fraction": 0.39980026631158455, + "metric": 0.28644294895435674, + "rescued": 1145, + "harmed": 278, + "missed_rescue": 137, + "avoided_harm": 499, + "selected_both_correct": 3027, + "selected_both_wrong": 354 + }, + { + "budget": 0.4, + "threshold": 0.157886803150177, + "operator": "ge", + "actual_fraction": 0.4054593874833555, + "metric": 0.2862200047733393, + "rescued": 1160, + "harmed": 284, + "missed_rescue": 122, + "avoided_harm": 493, + "selected_both_correct": 3072, + "selected_both_wrong": 356 + }, + { + "budget": 0.41, + "threshold": 0.12043046951293945, + "operator": "ge", + "actual_fraction": 0.4104527296937417, + "metric": 0.28760386086589435, + "rescued": 1168, + "harmed": 287, + "missed_rescue": 114, + "avoided_harm": 490, + "selected_both_correct": 3115, + "selected_both_wrong": 362 + }, + { + "budget": 0.42, + "threshold": 0.06623327732086182, + "operator": "ge", + "actual_fraction": 0.4181091877496671, + "metric": 0.2930744270093899, + "rescued": 1181, + "harmed": 293, + "missed_rescue": 101, + "avoided_harm": 484, + "selected_both_correct": 3180, + "selected_both_wrong": 370 + }, + { + "budget": 0.43, + "threshold": -0.01827472448348999, + "operator": "ge", + "actual_fraction": 0.42809587217043943, + "metric": 0.2961685540536353, + "rescued": 1200, + "harmed": 302, + "missed_rescue": 82, + "avoided_harm": 475, + "selected_both_correct": 3262, + "selected_both_wrong": 380 + }, + { + "budget": 0.44, + "threshold": -0.14177680015563965, + "operator": "ge", + "actual_fraction": 0.4370838881491345, + "metric": 0.2977058723546182, + "rescued": 1215, + "harmed": 309, + "missed_rescue": 67, + "avoided_harm": 468, + "selected_both_correct": 3338, + "selected_both_wrong": 390 + }, + { + "budget": 0.45, + "threshold": -0.270923376083374, + "operator": "ge", + "actual_fraction": 0.44673768308921435, + "metric": 0.2999402430635043, + "rescued": 1230, + "harmed": 313, + "missed_rescue": 52, + "avoided_harm": 464, + "selected_both_correct": 3422, + "selected_both_wrong": 403 + }, + { + "budget": 0.46, + "threshold": -0.4213377833366394, + "operator": "ge", + "actual_fraction": 0.45206391478029295, + "metric": 0.2993046120720782, + "rescued": 1235, + "harmed": 316, + "missed_rescue": 47, + "avoided_harm": 461, + "selected_both_correct": 3475, + "selected_both_wrong": 406 + }, + { + "budget": 0.47, + "threshold": -0.6602996587753296, + "operator": "ge", + "actual_fraction": 0.45872170439414117, + "metric": 0.2998310238962732, + "rescued": 1243, + "harmed": 321, + "missed_rescue": 39, + "avoided_harm": 456, + "selected_both_correct": 3536, + "selected_both_wrong": 412 + }, + { + "budget": 0.48, + "threshold": -0.8612939715385437, + "operator": "ge", + "actual_fraction": 0.4673768308921438, + "metric": 0.30393211265289466, + "rescued": 1247, + "harmed": 326, + "missed_rescue": 35, + "avoided_harm": 451, + "selected_both_correct": 3623, + "selected_both_wrong": 420 + }, + { + "budget": 0.49, + "threshold": -0.9715782999992371, + "operator": "ge", + "actual_fraction": 0.47503328894806923, + "metric": 0.30350001249166053, + "rescued": 1248, + "harmed": 328, + "missed_rescue": 34, + "avoided_harm": 449, + "selected_both_correct": 3710, + "selected_both_wrong": 422 + }, + { + "budget": 0.5, + "threshold": -1.1963446140289307, + "operator": "ge", + "actual_fraction": 0.4876830892143808, + "metric": 0.29875800068449215, + "rescued": 1249, + "harmed": 337, + "missed_rescue": 33, + "avoided_harm": 440, + "selected_both_correct": 3843, + "selected_both_wrong": 431 + }, + { + "budget": 0.51, + "threshold": -1.6505446434020996, + "operator": "ge", + "actual_fraction": 0.5046604527296937, + "metric": 0.29989697064322696, + "rescued": 1255, + "harmed": 348, + "missed_rescue": 27, + "avoided_harm": 429, + "selected_both_correct": 4026, + "selected_both_wrong": 435 + }, + { + "budget": 0.52, + "threshold": -2.6140921115875244, + "operator": "ge", + "actual_fraction": 0.5206391478029294, + "metric": 0.297424793956044, + "rescued": 1255, + "harmed": 359, + "missed_rescue": 27, + "avoided_harm": 418, + "selected_both_correct": 4205, + "selected_both_wrong": 437 + }, + { + "budget": 0.53, + "threshold": -2.6907107830047607, + "operator": "ge", + "actual_fraction": 0.5249667110519307, + "metric": 0.297424793956044, + "rescued": 1255, + "harmed": 359, + "missed_rescue": 27, + "avoided_harm": 418, + "selected_both_correct": 4257, + "selected_both_wrong": 437 + }, + { + "budget": 0.54, + "threshold": -2.8344802856445312, + "operator": "ge", + "actual_fraction": 0.5436085219707057, + "metric": 0.2978194775414123, + "rescued": 1257, + "harmed": 377, + "missed_rescue": 25, + "avoided_harm": 400, + "selected_both_correct": 4461, + "selected_both_wrong": 437 + }, + { + "budget": 0.55, + "threshold": -2.905412197113037, + "operator": "ge", + "actual_fraction": 0.5529294274300932, + "metric": 0.29630990060928936, + "rescued": 1257, + "harmed": 385, + "missed_rescue": 25, + "avoided_harm": 392, + "selected_both_correct": 4565, + "selected_both_wrong": 437 + }, + { + "budget": 0.56, + "threshold": -2.9482638835906982, + "operator": "ge", + "actual_fraction": 0.5612516644474035, + "metric": 0.29578344205458756, + "rescued": 1257, + "harmed": 388, + "missed_rescue": 25, + "avoided_harm": 389, + "selected_both_correct": 4660, + "selected_both_wrong": 439 + }, + { + "budget": 0.57, + "threshold": -2.9957408905029297, + "operator": "ge", + "actual_fraction": 0.5692410119840213, + "metric": 0.2950949302773625, + "rescued": 1257, + "harmed": 396, + "missed_rescue": 25, + "avoided_harm": 381, + "selected_both_correct": 4748, + "selected_both_wrong": 439 + }, + { + "budget": 0.58, + "threshold": -3.1037726402282715, + "operator": "ge", + "actual_fraction": 0.5902130492676432, + "metric": 0.2947373376338526, + "rescued": 1260, + "harmed": 407, + "missed_rescue": 22, + "avoided_harm": 370, + "selected_both_correct": 4985, + "selected_both_wrong": 440 + }, + { + "budget": 0.59, + "threshold": -3.1757895946502686, + "operator": "ge", + "actual_fraction": 0.6021970705725699, + "metric": 0.29722091277674884, + "rescued": 1261, + "harmed": 417, + "missed_rescue": 21, + "avoided_harm": 360, + "selected_both_correct": 5115, + "selected_both_wrong": 443 + }, + { + "budget": 0.6, + "threshold": -3.253206729888916, + "operator": "ge", + "actual_fraction": 0.6208388814913449, + "metric": 0.29665021956289706, + "rescued": 1264, + "harmed": 430, + "missed_rescue": 18, + "avoided_harm": 347, + "selected_both_correct": 5323, + "selected_both_wrong": 443 + }, + { + "budget": 0.61, + "threshold": -3.2825026512145996, + "operator": "ge", + "actual_fraction": 0.6288282290279628, + "metric": 0.295988506182091, + "rescued": 1264, + "harmed": 435, + "missed_rescue": 18, + "avoided_harm": 342, + "selected_both_correct": 5413, + "selected_both_wrong": 444 + }, + { + "budget": 0.62, + "threshold": -3.322240114212036, + "operator": "ge", + "actual_fraction": 0.6421438082556591, + "metric": 0.2939669175161438, + "rescued": 1264, + "harmed": 452, + "missed_rescue": 18, + "avoided_harm": 325, + "selected_both_correct": 5556, + "selected_both_wrong": 444 + }, + { + "budget": 0.63, + "threshold": -3.3503003120422363, + "operator": "ge", + "actual_fraction": 0.6491344873501997, + "metric": 0.29277362668991463, + "rescued": 1265, + "harmed": 463, + "missed_rescue": 17, + "avoided_harm": 314, + "selected_both_correct": 5628, + "selected_both_wrong": 444 + }, + { + "budget": 0.64, + "threshold": -3.377089738845825, + "operator": "ge", + "actual_fraction": 0.6571238348868176, + "metric": 0.2930821542583563, + "rescued": 1266, + "harmed": 468, + "missed_rescue": 16, + "avoided_harm": 309, + "selected_both_correct": 5718, + "selected_both_wrong": 444 + }, + { + "budget": 0.65, + "threshold": -3.4055864810943604, + "operator": "ge", + "actual_fraction": 0.6677762982689747, + "metric": 0.2892305220657417, + "rescued": 1266, + "harmed": 488, + "missed_rescue": 16, + "avoided_harm": 289, + "selected_both_correct": 5826, + "selected_both_wrong": 444 + }, + { + "budget": 0.66, + "threshold": -3.430248498916626, + "operator": "ge", + "actual_fraction": 0.6744340878828229, + "metric": 0.2915915088317972, + "rescued": 1267, + "harmed": 492, + "missed_rescue": 15, + "avoided_harm": 285, + "selected_both_correct": 5901, + "selected_both_wrong": 444 + }, + { + "budget": 0.67, + "threshold": -3.4584414958953857, + "operator": "ge", + "actual_fraction": 0.6854194407456724, + "metric": 0.2936540392680743, + "rescued": 1269, + "harmed": 501, + "missed_rescue": 13, + "avoided_harm": 276, + "selected_both_correct": 6022, + "selected_both_wrong": 444 + }, + { + "budget": 0.68, + "threshold": -3.502922773361206, + "operator": "ge", + "actual_fraction": 0.7047270306258322, + "metric": 0.29632332439424064, + "rescued": 1272, + "harmed": 523, + "missed_rescue": 10, + "avoided_harm": 254, + "selected_both_correct": 6229, + "selected_both_wrong": 444 + }, + { + "budget": 0.69, + "threshold": -3.532865285873413, + "operator": "ge", + "actual_fraction": 0.7157123834886817, + "metric": 0.2937246709263134, + "rescued": 1273, + "harmed": 540, + "missed_rescue": 9, + "avoided_harm": 237, + "selected_both_correct": 6342, + "selected_both_wrong": 445 + }, + { + "budget": 0.7, + "threshold": -3.5550942420959473, + "operator": "ge", + "actual_fraction": 0.7253661784287616, + "metric": 0.2952254059747369, + "rescued": 1274, + "harmed": 550, + "missed_rescue": 8, + "avoided_harm": 227, + "selected_both_correct": 6446, + "selected_both_wrong": 446 + }, + { + "budget": 0.71, + "threshold": -3.566344976425171, + "operator": "ge", + "actual_fraction": 0.7283621837549934, + "metric": 0.2945683782087344, + "rescued": 1274, + "harmed": 553, + "missed_rescue": 8, + "avoided_harm": 224, + "selected_both_correct": 6478, + "selected_both_wrong": 447 + }, + { + "budget": 0.72, + "threshold": -3.6056067943573, + "operator": "ge", + "actual_fraction": 0.741011984021305, + "metric": 0.29252963134197957, + "rescued": 1275, + "harmed": 568, + "missed_rescue": 7, + "avoided_harm": 209, + "selected_both_correct": 6614, + "selected_both_wrong": 447 + }, + { + "budget": 0.73, + "threshold": -3.628530979156494, + "operator": "ge", + "actual_fraction": 0.7526631158455392, + "metric": 0.2900395106945919, + "rescued": 1276, + "harmed": 588, + "missed_rescue": 6, + "avoided_harm": 189, + "selected_both_correct": 6733, + "selected_both_wrong": 447 + }, + { + "budget": 0.74, + "threshold": -3.6580705642700195, + "operator": "ge", + "actual_fraction": 0.7616511318242344, + "metric": 0.2887744568626224, + "rescued": 1276, + "harmed": 597, + "missed_rescue": 6, + "avoided_harm": 180, + "selected_both_correct": 6832, + "selected_both_wrong": 447 + }, + { + "budget": 0.75, + "threshold": -3.6900978088378906, + "operator": "ge", + "actual_fraction": 0.7709720372836218, + "metric": 0.28730364578560735, + "rescued": 1276, + "harmed": 606, + "missed_rescue": 6, + "avoided_harm": 171, + "selected_both_correct": 6935, + "selected_both_wrong": 447 + }, + { + "budget": 0.76, + "threshold": -3.7050747871398926, + "operator": "ge", + "actual_fraction": 0.7749667110519307, + "metric": 0.2858070539975931, + "rescued": 1276, + "harmed": 612, + "missed_rescue": 6, + "avoided_harm": 165, + "selected_both_correct": 6977, + "selected_both_wrong": 447 + }, + { + "budget": 0.77, + "threshold": -3.720116138458252, + "operator": "ge", + "actual_fraction": 0.7792942743009321, + "metric": 0.28488735368132817, + "rescued": 1276, + "harmed": 617, + "missed_rescue": 6, + "avoided_harm": 160, + "selected_both_correct": 7023, + "selected_both_wrong": 448 + }, + { + "budget": 0.78, + "threshold": -3.7399919033050537, + "operator": "ge", + "actual_fraction": 0.7846205059920106, + "metric": 0.2830658409372236, + "rescued": 1276, + "harmed": 627, + "missed_rescue": 6, + "avoided_harm": 150, + "selected_both_correct": 7077, + "selected_both_wrong": 448 + }, + { + "budget": 0.79, + "threshold": -3.799569606781006, + "operator": "ge", + "actual_fraction": 0.7972703062583223, + "metric": 0.2809995077380739, + "rescued": 1277, + "harmed": 643, + "missed_rescue": 5, + "avoided_harm": 134, + "selected_both_correct": 7212, + "selected_both_wrong": 448 + }, + { + "budget": 0.8, + "threshold": -3.8980000019073486, + "operator": "ge", + "actual_fraction": 0.8129161118508655, + "metric": 0.28013559922601305, + "rescued": 1278, + "harmed": 667, + "missed_rescue": 4, + "avoided_harm": 110, + "selected_both_correct": 7375, + "selected_both_wrong": 448 + }, + { + "budget": 0.81, + "threshold": -4.0607757568359375, + "operator": "ge", + "actual_fraction": 0.8199067909454061, + "metric": 0.27814841049220523, + "rescued": 1278, + "harmed": 681, + "missed_rescue": 4, + "avoided_harm": 96, + "selected_both_correct": 7445, + "selected_both_wrong": 448 + }, + { + "budget": 0.82, + "threshold": -8.964031219482422, + "operator": "ge", + "actual_fraction": 0.831225033288948, + "metric": 0.2772348763322643, + "rescued": 1278, + "harmed": 686, + "missed_rescue": 4, + "avoided_harm": 91, + "selected_both_correct": 7576, + "selected_both_wrong": 448 + }, + { + "budget": 0.83, + "threshold": -9.02132511138916, + "operator": "ge", + "actual_fraction": 0.84254327563249, + "metric": 0.27700595116508253, + "rescued": 1279, + "harmed": 689, + "missed_rescue": 3, + "avoided_harm": 88, + "selected_both_correct": 7708, + "selected_both_wrong": 448 + }, + { + "budget": 0.84, + "threshold": -9.09000015258789, + "operator": "ge", + "actual_fraction": 0.857190412782956, + "metric": 0.2763733409861609, + "rescued": 1279, + "harmed": 692, + "missed_rescue": 3, + "avoided_harm": 85, + "selected_both_correct": 7881, + "selected_both_wrong": 448 + }, + { + "budget": 0.85, + "threshold": -9.12938404083252, + "operator": "ge", + "actual_fraction": 0.8618508655126498, + "metric": 0.2761345157638265, + "rescued": 1279, + "harmed": 694, + "missed_rescue": 3, + "avoided_harm": 83, + "selected_both_correct": 7935, + "selected_both_wrong": 448 + }, + { + "budget": 0.86, + "threshold": -9.17154312133789, + "operator": "ge", + "actual_fraction": 0.87017310252996, + "metric": 0.2761345157638265, + "rescued": 1279, + "harmed": 694, + "missed_rescue": 3, + "avoided_harm": 83, + "selected_both_correct": 8035, + "selected_both_wrong": 448 + }, + { + "budget": 0.87, + "threshold": -9.273024559020996, + "operator": "ge", + "actual_fraction": 0.8848202396804261, + "metric": 0.27546832609774896, + "rescued": 1279, + "harmed": 699, + "missed_rescue": 3, + "avoided_harm": 78, + "selected_both_correct": 8206, + "selected_both_wrong": 448 + }, + { + "budget": 0.88, + "threshold": -9.444879531860352, + "operator": "ge", + "actual_fraction": 0.8984687083888149, + "metric": 0.27423981143437465, + "rescued": 1279, + "harmed": 709, + "missed_rescue": 3, + "avoided_harm": 68, + "selected_both_correct": 8360, + "selected_both_wrong": 448 + }, + { + "budget": 0.89, + "threshold": -9.566991806030273, + "operator": "ge", + "actual_fraction": 0.9087882822902796, + "metric": 0.2736575311942959, + "rescued": 1279, + "harmed": 713, + "missed_rescue": 3, + "avoided_harm": 64, + "selected_both_correct": 8480, + "selected_both_wrong": 448 + }, + { + "budget": 0.9, + "threshold": -9.675031661987305, + "operator": "ge", + "actual_fraction": 0.9161118508655126, + "metric": 0.272975172975173, + "rescued": 1279, + "harmed": 717, + "missed_rescue": 3, + "avoided_harm": 60, + "selected_both_correct": 8563, + "selected_both_wrong": 449 + }, + { + "budget": 0.91, + "threshold": -9.739017486572266, + "operator": "ge", + "actual_fraction": 0.9217709720372836, + "metric": 0.2727190231806067, + "rescued": 1279, + "harmed": 719, + "missed_rescue": 3, + "avoided_harm": 58, + "selected_both_correct": 8629, + "selected_both_wrong": 449 + }, + { + "budget": 0.92, + "threshold": -9.787779808044434, + "operator": "ge", + "actual_fraction": 0.9264314247669774, + "metric": 0.27264729817760425, + "rescued": 1279, + "harmed": 720, + "missed_rescue": 3, + "avoided_harm": 57, + "selected_both_correct": 8684, + "selected_both_wrong": 449 + }, + { + "budget": 0.93, + "threshold": -9.832305908203125, + "operator": "ge", + "actual_fraction": 0.933422103861518, + "metric": 0.2741441928343702, + "rescued": 1280, + "harmed": 724, + "missed_rescue": 2, + "avoided_harm": 53, + "selected_both_correct": 8763, + "selected_both_wrong": 449 + }, + { + "budget": 0.94, + "threshold": -9.88642692565918, + "operator": "ge", + "actual_fraction": 0.9417443408788282, + "metric": 0.2733716828250215, + "rescued": 1280, + "harmed": 728, + "missed_rescue": 2, + "avoided_harm": 49, + "selected_both_correct": 8859, + "selected_both_wrong": 449 + }, + { + "budget": 0.95, + "threshold": -9.976344108581543, + "operator": "ge", + "actual_fraction": 0.9530625832223701, + "metric": 0.2727312493845365, + "rescued": 1280, + "harmed": 737, + "missed_rescue": 2, + "avoided_harm": 40, + "selected_both_correct": 8986, + "selected_both_wrong": 449 + }, + { + "budget": 0.96, + "threshold": -10.03441047668457, + "operator": "ge", + "actual_fraction": 0.9613848202396804, + "metric": 0.27223682114986464, + "rescued": 1280, + "harmed": 740, + "missed_rescue": 2, + "avoided_harm": 37, + "selected_both_correct": 9083, + "selected_both_wrong": 449 + }, + { + "budget": 0.97, + "threshold": -10.085777282714844, + "operator": "ge", + "actual_fraction": 0.9673768308921438, + "metric": 0.27453093162048386, + "rescued": 1282, + "harmed": 744, + "missed_rescue": 0, + "avoided_harm": 33, + "selected_both_correct": 9149, + "selected_both_wrong": 449 + }, + { + "budget": 0.98, + "threshold": -10.1488676071167, + "operator": "ge", + "actual_fraction": 0.9760319573901465, + "metric": 0.2737331615154065, + "rescued": 1282, + "harmed": 750, + "missed_rescue": 0, + "avoided_harm": 27, + "selected_both_correct": 9247, + "selected_both_wrong": 449 + }, + { + "budget": 0.99, + "threshold": -10.265552520751953, + "operator": "ge", + "actual_fraction": 0.9890146471371505, + "metric": 0.2725383412883413, + "rescued": 1282, + "harmed": 758, + "missed_rescue": 0, + "avoided_harm": 19, + "selected_both_correct": 9395, + "selected_both_wrong": 449 + }, + { + "budget": 1.0, + "threshold": null, + "operator": "always", + "actual_fraction": 1.0, + "metric": 0.27001009067886245, + "rescued": 1282, + "harmed": 777, + "missed_rescue": 0, + "avoided_harm": 0, + "selected_both_correct": 9508, + "selected_both_wrong": 449 + } + ] + } + }, + "opportunity": { + "val": { + "both_correct": 2189, + "rescuable": 307, + "harmful": 220, + "both_wrong": 88, + "unit": "event", + "total": 2804 + }, + "test": { + "both_correct": 9508, + "rescuable": 1282, + "harmful": 777, + "both_wrong": 449, + "unit": "event", + "total": 12016 + } + }, + "small": { + "validation": 0.18254072168965788, + "test": 0.18129624718318377 + }, + "large": { + "validation": 0.25448626222520154, + "test": 0.27001009067886245 + }, + "learned_minus_confidence_test": 0.03900900020828163, + "learned_minus_small_test": 0.11746175350130839, + "learned_minus_large_test": 0.028747910005629707 + } + ] +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch.sh new file mode 100644 index 0000000000000000000000000000000000000000..2d1c81b09ce00216016922f994ee0d7578a72dcc --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail +OUT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_20260912 +WORK=/tmp/codex_task_main_cost_20260912 +mkdir -p "$OUT/logs" "$OUT/components" +bash -n "$WORK/run_large.sh" +python -m py_compile "$WORK/measure_large.py" "$WORK/measure_flops.py" "$WORK/measure_specialist.py" +nohup bash "$WORK/run_large.sh" 0 MiniCPM grefcoco minicpm > "$OUT/logs/minicpm.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/minicpm.pid" +nohup bash "$WORK/run_large.sh" 1 Qwen3.8 construction qwen > "$OUT/logs/qwen.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/qwen.pid" +cat "$OUT/logs/minicpm.pid" "$OUT/logs/qwen.pid" diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch_qwen_gref.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch_qwen_gref.sh new file mode 100644 index 0000000000000000000000000000000000000000..85271ae71f6b5dd74c522d428bd1d948720025ab --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch_qwen_gref.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +WORK=/tmp/codex_task_main_cost_20260912 +OUT="$ROOT/outputs/main_table_cost_qwen_20260912" +mkdir -p "$OUT/components" "$OUT/logs" +nohup bash "$WORK/run_qwen_gref.sh" > "$OUT/logs/qwen_gref.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/qwen_gref.pid" +CUDA_VISIBLE_DEVICES=1 OMP_NUM_THREADS=4 nohup python "$WORK/measure_router.py" --root "$ROOT" --expert instancevg --endpoint Qwen3.8 --output "$OUT/components/instancevg_router.json" > "$OUT/logs/router.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/router.pid" +cat "$OUT/logs/"*.pid diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch_small.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch_small.sh new file mode 100644 index 0000000000000000000000000000000000000000..72bb571f0e76e178cb624d31546b29dc6b97cc59 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/launch_small.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +set -euo pipefail +WORK=/tmp/codex_task_main_cost_20260912 +OUT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_20260912 +bash -n "$WORK/run_small.sh" +python -m py_compile "$WORK/measure_router.py" +nohup bash "$WORK/run_small.sh" 0 instancevg MiniCPM minicpm > "$OUT/logs/instancevg_small.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/instancevg_small.pid" +nohup bash "$WORK/run_small.sh" 1 yolo26x Qwen3.8 qwen > "$OUT/logs/yolo26x_small.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/yolo26x_small.pid" +cat "$OUT/logs/instancevg_small.pid" "$OUT/logs/yolo26x_small.pid" diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_flops.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_flops.py new file mode 100644 index 0000000000000000000000000000000000000000..4b5eb79777bb9eca67245173f1c6813f000b8e8d --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_flops.py @@ -0,0 +1,152 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import json +import re +from pathlib import Path + +import numpy as np +import torch +from PIL import Image +from torch.utils.flop_counter import FlopCounterMode + + +def count(call): + with torch.inference_mode(), FlopCounterMode(display=False) as counter: + value = call() + torch.cuda.synchronize() + counts = counter.get_flop_counts().get('Global', {}) + return int(counter.get_total_flops()), value, {str(k): int(v) for k, v in counts.items()} + + +def make_text(root: Path, model_kind: str, task: str, row: dict, processor): + prompt = (root / 'outputs/large_model_labels_all_20260909/prompts' / task / f'{model_kind}.txt').read_text(encoding='utf-8').strip() + if task == 'grefcoco': + prompt = prompt.replace('{expression}', row['expression']) + if model_kind == 'Qwen3.8': + content = [{'type': 'image'}, {'type': 'text', 'text': '\n'}, {'type': 'text', 'text': prompt}] + return processor.apply_chat_template([{'role': 'user', 'content': content}], tokenize=False, add_generation_prompt=True) + message = '(./)\n' + prompt + return processor.tokenizer.apply_chat_template([{'role': 'user', 'content': message}], tokenize=False, add_generation_prompt=True, enable_thinking=False) + + +def prepare(root: Path, model_kind: str, task: str, row: dict, processor): + image = Image.open(row['image_path']).convert('RGB') + text = make_text(root, model_kind, task, row, processor) + if model_kind == 'Qwen3.8': + data = processor(text=[text], images=[image], return_tensors='pt', size={'shortest_edge': 65536, 'longest_edge': 1003520}) + else: + data = processor([text], [[image]], return_tensors='pt', max_length=16384) + image.close() + return data + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--model-kind', choices=['Qwen3.8', 'MiniCPM'], required=True) + parser.add_argument('--task', choices=['grefcoco', 'construction'], required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--samples', type=int, default=4) + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + expert = 'instancevg' if args.task == 'grefcoco' else 'yolo26x' + rows = [json.loads(line) for line in (args.root / f'outputs/router_full_20260910/data/{expert}/test/records.jsonl').open(encoding='utf-8')] + if args.samples > len(rows): + raise ValueError('sample count exceeds test rows') + selected_indices = np.random.default_rng(20260911).choice(len(rows), args.samples, replace=False) + selected = [rows[int(index)] for index in selected_indices] + checkpoint_root = args.root.parents[1] / 'checkpoints' + if not checkpoint_root.exists(): + checkpoint_root = args.root / 'checkpoints' + model_path = checkpoint_root / ('Qwen3.8-27B-FP8' if args.model_kind == 'Qwen3.8' else 'MiniCPM-V-4_5') + from transformers import AutoProcessor + processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + if args.model_kind == 'Qwen3.8': + from transformers import FineGrainedFP8Config, Qwen3_5ForConditionalGeneration + import transformers.models.qwen3_5.modeling_qwen3_5 as implementation + implementation.causal_conv1d_fn = None + implementation.causal_conv1d_update = None + implementation.chunk_gated_delta_rule = None + implementation.fused_recurrent_gated_delta_rule = None + implementation.FusedRMSNormGated = None + implementation.is_fast_path_available = False + model = Qwen3_5ForConditionalGeneration.from_pretrained( + model_path, local_files_only=True, dtype=torch.bfloat16, device_map={'': 'cuda:0'}, + attn_implementation='eager', quantization_config=FineGrainedFP8Config(dequantize=False), + ) + else: + model = processor # replaced below to keep imports local to the selected environment + from transformers import AutoModel + from transformers.modeling_utils import PreTrainedModel + if not hasattr(PreTrainedModel, 'all_tied_weights_keys'): + def get_tied_weights(self): + return getattr(self, '_all_tied_weights_keys', getattr(self, '_tied_weights_keys', {}) or {}) + def set_tied_weights(self, value): + self._all_tied_weights_keys = value + PreTrainedModel.all_tied_weights_keys = property(get_tied_weights, set_tied_weights) + model = AutoModel.from_pretrained( + model_path, trust_remote_code=True, local_files_only=True, torch_dtype=torch.bfloat16, + device_map={'': 'cuda:0'}, attn_implementation='eager', + ) + torch.set_num_threads(4) + torch.backends.mha.set_fastpath_enabled(False) + model.eval().requires_grad_(False) + records = [] + for row in selected: + data = prepare(args.root, args.model_kind, args.task, row, processor).to('cuda') + prompt_tokens = int(data['input_ids'].shape[-1]) + if args.model_kind == 'MiniCPM': + vision_flops, embedding_result, vision_ops = count(lambda: model.get_vllm_embedding(data)) + embeddings, _ = embedding_result + prefill_flops, prefill_output, prefill_ops = count(lambda: model.llm(inputs_embeds=embeddings, attention_mask=data['attention_mask'], use_cache=True, logits_to_keep=1)) + prefill_flops += vision_flops + prefill_ops.update({'vision:'+k:v for k,v in vision_ops.items()}) + length = int(data['input_ids'].shape[-1]) + known_token = data['input_ids'][:, -1:] + kwargs = { + 'input_ids': known_token, + 'past_key_values': prefill_output.past_key_values, + 'attention_mask': torch.ones((1, length + 1), device='cuda', dtype=torch.long), + 'use_cache': True, + 'logits_to_keep': 1, + } + decode_flops, _, decode_ops = count(lambda: model.llm(**kwargs)) + else: + prefill_flops, prefill_output, prefill_ops = count(lambda: model(**data, use_cache=True, logits_to_keep=1)) + length = int(data['input_ids'].shape[-1]) + known_token = data['input_ids'][:, -1:] + kwargs = { + 'input_ids': known_token, + 'past_key_values': prefill_output.past_key_values, + 'attention_mask': torch.ones((1, length + 1), device='cuda', dtype=torch.long), + 'use_cache': True, + 'logits_to_keep': 1, + 'cache_position': torch.tensor([length], device='cuda'), + } + kwargs.pop('attention_mask') + decode_flops, _, decode_ops = count(lambda: model(**kwargs)) + records.append({ + 'sample_id': row['sample_id'], 'prompt_tokens_in_reference_tensor': prompt_tokens, + 'input_shapes': {key: list(value.shape) for key, value in data.items() if hasattr(value, 'shape')}, + 'prefill_flops_counted': prefill_flops, 'decode_one_token_flops_counted': decode_flops, + 'prefill_gflops_counted': prefill_flops / 1e9, 'decode_one_token_gflops_counted': decode_flops / 1e9, + 'prefill_operators': prefill_ops, 'decode_operators': decode_ops, + }) + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps({'status': 'running', 'model_kind': args.model_kind, 'task': args.task, 'records': records}, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'completed': len(records), 'sample_id': row['sample_id'], 'prefill_gflops': prefill_flops / 1e9, 'decode_gflops': decode_flops / 1e9}), flush=True) + args.output.write_text(json.dumps({ + 'status': 'complete', 'model_kind': args.model_kind, 'task': args.task, 'model_path': str(model_path), + 'sample_count': len(records), 'records': records, + 'mean_prefill_gflops_counted': float(np.mean([row['prefill_gflops_counted'] for row in records])), + 'mean_decode_one_token_gflops_counted': float(np.mean([row['decode_one_token_gflops_counted'] for row in records])), + 'scope': 'HF reference execution on the A800 with real task inputs; dense products/convolutions/supported attention counted with multiply-add=2 FLOPs. vLLM custom kernels, elementwise/normalization/nonlinearities and other unsupported operations are not complete.', + 'answer_generation': 'not performed; one known-token decode probe only', + }, ensure_ascii=False, indent=2), encoding='utf-8') + + +if __name__ == '__main__': + main() diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_large.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_large.py new file mode 100644 index 0000000000000000000000000000000000000000..61b54474c6d7e15ee107c15c3ca4eb10c14ebca3 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_large.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import json +import time +from pathlib import Path + +import numpy as np +import jsonschema +from PIL import Image +from vllm import LLM, SamplingParams +from vllm.sampling_params import StructuredOutputsParams + + +EVENTS = [ + 'rule_1_ppe_violation', + 'rule_2_fall_protection_violation', + 'rule_3_unprotected_edge_violation', + 'rule_4_excavator_proximity_violation', +] + + +def gref_schema(): + return { + 'type': 'object', + 'properties': { + 'objects': { + 'type': 'array', 'maxItems': 18, + 'items': { + 'type': 'object', + 'properties': { + 'bbox_2d': {'type': 'array', 'items': {'type': 'number', 'minimum': 0, 'maximum': 1000}, 'minItems': 4, 'maxItems': 4}, + 'score': {'type': 'number', 'minimum': 0, 'maximum': 1}, + }, + 'required': ['bbox_2d', 'score'], 'additionalProperties': False, + }, + }, + }, + 'required': ['objects'], 'additionalProperties': False, + } + + +def construction_schema(): + box = {'type': 'array', 'items': {'type': 'integer', 'minimum': 0, 'maximum': 1000}, 'minItems': 4, 'maxItems': 4} + max_boxes = [13, 6, 3, 2] + events = {} + for event, limit in zip(EVENTS, max_boxes): + events[event] = { + 'type': 'object', + 'properties': { + 'present': {'type': 'boolean'}, + 'reason': {'type': ['string', 'null']}, + 'boxes': {'type': 'array', 'items': box, 'maxItems': limit}, + }, + 'required': ['present', 'reason', 'boxes'], 'additionalProperties': False, + } + return {'type': 'object', 'properties': {'events': {'type': 'object', 'properties': events, 'required': EVENTS, 'additionalProperties': False}}, 'required': ['events'], 'additionalProperties': False} + + +def make_prompt(root: Path, model_kind: str, task: str, row: dict, processor_or_tokenizer): + prompt = (root / 'outputs/large_model_labels_all_20260909/prompts' / task / f'{model_kind}.txt').read_text(encoding='utf-8').strip() + if task == 'grefcoco': + prompt = prompt.replace('{expression}', row['expression']) + if model_kind == 'Qwen3.8': + converted = [{'type': 'image'}] + converted.append({'type': 'text', 'text': '\n'}) + converted.append({'type': 'text', 'text': prompt}) + messages = [{'role': 'user', 'content': converted}] + return processor_or_tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + message = '(./)\n' + prompt + return processor_or_tokenizer.apply_chat_template([{'role': 'user', 'content': message}], tokenize=False, add_generation_prompt=True, enable_thinking=False) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--model-kind', choices=['Qwen3.8', 'MiniCPM'], required=True) + parser.add_argument('--task', choices=['grefcoco', 'construction'], required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--samples', type=int, default=32) + parser.add_argument('--batch-size', type=int, default=8) + parser.add_argument('--gpu-memory-utilization', type=float, default=0.90) + parser.add_argument('--max-model-len', type=int, default=16384) + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + if args.samples < 3 or args.batch_size < 1: + raise ValueError('samples must include warmup rows and batch size must be positive') + expert = 'instancevg' if args.task == 'grefcoco' else 'yolo26x' + rows = [json.loads(line) for line in (args.root / f'outputs/router_full_20260910/data/{expert}/test/records.jsonl').open(encoding='utf-8')] + if args.samples > len(rows): + raise ValueError(f'samples {args.samples} exceeds {len(rows)}') + selected_indices = np.random.default_rng(20260911).choice(len(rows), args.samples, replace=False) + selected = [rows[int(index)] for index in selected_indices] + checkpoint_root = args.root.parents[1] / 'checkpoints' + if not checkpoint_root.exists(): + checkpoint_root = args.root / 'checkpoints' + model_path = checkpoint_root / ('Qwen3.8-27B-FP8' if args.model_kind == 'Qwen3.8' else 'MiniCPM-V-4_5') + if args.model_kind == 'Qwen3.8': + from transformers import AutoProcessor + processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + model_kwargs = {'mm_processor_kwargs': {'size': {'shortest_edge': 65536, 'longest_edge': 1003520}}} + else: + from transformers import AutoTokenizer + processor = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True, local_files_only=True) + model_kwargs = {} + model = LLM(model=str(model_path), trust_remote_code=True, tensor_parallel_size=1, max_model_len=args.max_model_len, + limit_mm_per_prompt={'image': 1}, max_num_seqs=args.batch_size, + gpu_memory_utilization=args.gpu_memory_utilization, enforce_eager=True, **model_kwargs) + schema = gref_schema() if args.task == 'grefcoco' else construction_schema() + sampling = SamplingParams(temperature=0, max_tokens=512 if args.task == 'grefcoco' else 768, + structured_outputs=StructuredOutputsParams(json=schema, disable_additional_properties=True)) + + def generate(batch_rows): + images = [] + requests = [] + for row in batch_rows: + required = {'sample_id', 'image_path'} + if args.task == 'grefcoco': + required.add('expression') + missing = required - row.keys() + if missing: + raise ValueError(f'missing fields {sorted(missing)} for {row["sample_id"]}') + image = Image.open(row['image_path']).convert('RGB') + images.append(image) + requests.append({'prompt': make_prompt(args.root, args.model_kind, args.task, row, processor), 'multi_modal_data': {'image': image}}) + started = time.perf_counter() + generated = model.generate(requests, sampling, use_tqdm=False) + elapsed = time.perf_counter() - started + for image in images: + image.close() + if len(generated) != len(batch_rows): + raise ValueError('vLLM returned an unexpected number of results') + return elapsed, generated + + warmup_count = min(2, len(selected) - 1) + for row in selected[:warmup_count]: + generate([row]) + measured = selected[warmup_count:] + batches = [] + for start in range(0, len(measured), args.batch_size): + batch_rows = measured[start:start + args.batch_size] + batch_started = time.perf_counter() + elapsed, generated = generate(batch_rows) + wall = time.perf_counter() - batch_started + output_records = [] + for result in generated: + raw = result.outputs[0].text + error = None + try: + parsed = json.loads(raw) + jsonschema.validate(parsed, schema) + boxes = [o['bbox_2d'] for o in parsed['objects']] if args.task == 'grefcoco' else [b for e in parsed['events'].values() for b in e['boxes']] + if any(b[0] >= b[2] or b[1] >= b[3] for b in boxes): + raise ValueError('non-positive box size') + except (json.JSONDecodeError, jsonschema.ValidationError, ValueError) as exc: + parsed = None + error = str(exc) + output_records.append({'raw_response':raw,'valid_prediction':parsed,'error':error, + 'finish_reason':result.outputs[0].finish_reason,'completion_tokens':len(result.outputs[0].token_ids), + 'prompt_tokens':len(result.prompt_token_ids)}) + item = { + 'sample_ids': [row['sample_id'] for row in batch_rows], + 'batch_size': len(batch_rows), + 'generation_latency_seconds': elapsed, + 'batch_wall_latency_seconds': wall, + 'amortized_generation_ms_per_sample': elapsed * 1000 / len(batch_rows), + 'amortized_batch_wall_ms_per_sample': wall * 1000 / len(batch_rows), + 'outputs': [ + { + 'finish_reason': result.outputs[0].finish_reason, + 'stop_reason': result.outputs[0].stop_reason, + 'prompt_tokens': len(result.prompt_token_ids) if result.prompt_token_ids is not None else None, + 'completion_tokens': len(result.outputs[0].token_ids), + 'text_characters': len(result.outputs[0].text), + } + for result in generated + ], + } + batches.append(item) + item['outputs'] = output_records + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps({ + 'status': 'running', 'model_kind': args.model_kind, 'task': args.task, 'model_path': str(model_path), + 'sample_count_requested': args.samples, 'warmup_count': warmup_count, 'batch_size': args.batch_size, + 'sample_ids': [row['sample_id'] for row in selected], 'batches': batches, + }, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'completed_samples': sum(item['batch_size'] for item in batches), 'total_measured': len(measured), 'batch_latency_seconds': elapsed}), flush=True) + all_ms = [x['amortized_generation_ms_per_sample'] for x in batches] + all_wall_ms = [x['amortized_batch_wall_ms_per_sample'] for x in batches] + output = { + 'status': 'complete', 'model_kind': args.model_kind, 'task': args.task, 'expert': expert, + 'model_path': str(model_path), 'device': 'cuda:0', 'gpu_memory_utilization': args.gpu_memory_utilization, + 'sample_count_requested': args.samples, 'warmup_count': warmup_count, 'measured_count': len(measured), + 'batch_size': args.batch_size, 'sample_ids': [row['sample_id'] for row in selected], 'batches': batches, + 'mean_generation_ms_per_sample': float(np.mean(all_ms)), 'median_generation_ms_per_sample': float(np.median(all_ms)), + 'mean_batch_wall_ms_per_sample': sum(b['batch_wall_latency_seconds'] for b in batches)*1000/len(measured), 'median_batch_wall_ms_per_sample': float(np.median(all_wall_ms)), + 'mean_prompt_tokens': float(np.mean([x['prompt_tokens'] for batch in batches for x in batch['outputs'] if x['prompt_tokens'] is not None])), + 'mean_completion_tokens': float(np.mean([x['completion_tokens'] for batch in batches for x in batch['outputs']])), + 'latency_scope': 'A800 vLLM-0.18.0 native generation latency with the frozen task prompt and structured-output schema; model loading and warmup excluded. Batch wall time includes the real request/image preparation path.', + 'flops_scope': 'This file measures latency only. Counted FLOPs are measured separately by the HF reference probe and are not claimed to be complete vLLM kernel FLOPs.', + } + args.output.write_text(json.dumps(output, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({'status': 'complete', 'mean_generation_ms_per_sample': output['mean_generation_ms_per_sample'], 'mean_completion_tokens': output['mean_completion_tokens']}), flush=True) + + +if __name__ == '__main__': + main() diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_router.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_router.py new file mode 100644 index 0000000000000000000000000000000000000000..192677ef8450d4b93944d341658a5a745e82b4c8 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_router.py @@ -0,0 +1,104 @@ +from pathlib import Path +import argparse +import json +import sys +import time +import numpy as np +import torch +from PIL import Image +from torch.utils.flop_counter import FlopCounterMode + +p=argparse.ArgumentParser() +p.add_argument('--root',type=Path,required=True) +p.add_argument('--expert',required=True) +p.add_argument('--endpoint',required=True) +p.add_argument('--output',type=Path,required=True) +a=p.parse_args() +sys.path.insert(0,str(a.root/'code')) +from routing import train_router_formal as formal +from routing.router import prepare_images,load_distilled_router +from siglip_distillation.mini_siglip import MiniSiglip +from tokenizers import Tokenizer +torch.set_num_threads(4) +torch.backends.cuda.matmul.allow_tf32=False +torch.backends.mha.set_fastpath_enabled(False) +work=a.root/'outputs/router_shared_image_hidden_20260910' +run=f'shared_12_{a.expert}_{a.endpoint}' +dest=work/'runs'/run +cfg=json.loads((dest/'config.json').read_text()) +assert cfg['seed']==42 and cfg['student']=='8M' and cfg['target']=='four_state' +records,data=formal.load_data(cfg,'test',torch.device('cuda')) +for key in ['hidden','confidence']: + norm=np.load(dest/(key+'_normalization.npz')) + data[key]=(data[key]-torch.from_numpy(norm['mean']).cuda())/torch.from_numpy(norm['std']).cuda() +model=formal.FeatureRouter(cfg,data['hidden'].shape[1],data['confidence'].shape[1],data['output'].shape[1]).cuda() +ckpt=torch.load(dest/'best.pt',map_location='cuda',weights_only=True) +model.load_state_dict(ckpt['state_dict'],strict=True) +model.eval().requires_grad_(False) +with torch.inference_mode(): + raw=torch.cat([model(data,torch.arange(i,min(i+8192,len(records)),device='cuda')).cpu() for i in range(0,len(records),8192)]).numpy() +scores=raw[:,1]-raw[:,2] +saved=np.load(work/'test_evaluation'/run/'paired_test_arrays.npz')['scores'] +reference=json.loads((work/'test_results.json').read_text()) +ref=next(g for g in reference['groups'] if g['id']==run) +point=ref['best_test_point'] +threshold=point['threshold'] +mask=scores>=threshold if point['operator']=='ge' else scores>threshold +saved_mask=saved>=threshold if point['operator']=='ge' else saved>threshold +assert np.array_equal(mask,saved_mask), 'Recomputed selected decisions differ from main-table archive' +indices=np.random.default_rng(20260912).choice(len(records),64,replace=False) +encoder_dir=a.root/'outputs/mini_siglip_strict_20260910_300ep/8M' +checkpoint=torch.load(encoder_dir/'epoch_300.pt',map_location='cpu',weights_only=True) +encoder=MiniSiglip(**checkpoint['model_config']).cuda() +encoder.load_state_dict(checkpoint['model'],strict=True) +encoder.eval().requires_grad_(False) +tokenizer=Tokenizer.from_file(str(encoder_dir/'student_tokenizer.json')) + +def prepare(i): + with Image.open(records[i]['image_path']) as im: + pixels,valid=prepare_images([[im]]) + pixels=pixels.flatten(0,1).cuda() + tokens=torch.tensor([tokenizer.encode(records[i]['expression']).ids],device='cuda') if 'text' in cfg['branches'] else None + return pixels,valid.cuda(),tokens + +def forward(i,prepared): + pixels,valid,tokens=prepared + with torch.autocast('cuda',dtype=torch.bfloat16): + image=encoder.encode_image(pixels).to(torch.float16).float().reshape(1,2,768) + text=encoder.encode_text(tokens).to(torch.float16).float() if tokens is not None else None + row={k:v[i:i+1] for k,v in data.items()} + row['image']=image;row['valid']=valid + if text is not None:row['text']=text + return model(row,torch.zeros(1,device='cuda',dtype=torch.long)) + +times=[];head_times=[];flops=[];ops=[] +with torch.inference_mode(): + for i in indices[:4]: + forward(int(i),prepare(int(i))) + torch.cuda.synchronize() + for i in indices: + i=int(i) + started=time.perf_counter() + forward(i,prepare(i)) + torch.cuda.synchronize() + times.append((time.perf_counter()-started)*1000) + start=torch.cuda.Event(enable_timing=True);end=torch.cuda.Event(enable_timing=True) + idx=torch.tensor([i],device='cuda') + start.record();model(data,idx);end.record();torch.cuda.synchronize() + head_times.append(start.elapsed_time(end)) + for i in indices[:8]: + prepared=prepare(int(i)) + with FlopCounterMode(display=False) as counter: + forward(int(i),prepared) + flops.append(counter.get_total_flops()/1e9) + ops.append({str(k):int(v) for k,v in counter.get_flop_counts().get('Global',{}).items()}) +report={'status':'complete','run':run,'config':cfg,'checkpoint':str(dest/'best.pt'), + 'score':'logit_rescue-logit_harm','test_rows_recomputed':len(records),'selected_decisions_match_main':True, + 'max_score_abs_difference':float(np.max(np.abs(saved-scores))),'actual_call_fraction':float(mask.mean()), + 'sample_count':len(indices),'sample_ids':[records[int(i)]['sample_id'] for i in indices], + 'batch_size':1,'latency_ms':times,'mean_latency_ms_per_input':float(np.mean(times)), + 'head_mean_latency_ms':float(np.mean(head_times)), 'gflops_per_input':flops,'mean_gflops_per_input':float(np.mean(flops)), + 'operators':ops,'scope':'image loading, bilinear resize, tokenize when used, two image slots (including gray placeholder), active distilled encoders, feature assembly and original routing head; cached specialist features; supported FLOPs only'} +a.output.parent.mkdir(parents=True,exist_ok=True) +a.output.write_text(json.dumps(report,indent=2),encoding='utf-8') +print(json.dumps({k:v for k,v in report.items() if k in ['run','actual_call_fraction','selected_decisions_match_main','mean_latency_ms_per_input','mean_gflops_per_input']},indent=2)) diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_specialist.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_specialist.py new file mode 100644 index 0000000000000000000000000000000000000000..fbaf817087f52f7e7dfcd6b7ac42c3d4d8360a20 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/measure_specialist.py @@ -0,0 +1,258 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import importlib.util +import json +import sys +import time +from pathlib import Path + +import numpy as np +import torch +from torch.utils.flop_counter import FlopCounterMode + + +def timed_cuda(call): + start = torch.cuda.Event(enable_timing=True) + end = torch.cuda.Event(enable_timing=True) + start.record() + value = call() + end.record() + torch.cuda.synchronize() + return float(start.elapsed_time(end)), value + + +def count_flops(call): + with torch.inference_mode(), FlopCounterMode(display=False) as counter: + value = call() + torch.cuda.synchronize() + global_counts = counter.get_flop_counts().get('Global', {}) + return int(counter.get_total_flops()), value, {str(k): int(v) for k, v in global_counts.items()} + + +def instancevg_measure(root: Path, output: Path, samples: int, batch_size: int, device: str) -> None: + import mmcv + from mmcv import Config + from transformers import XLMRobertaTokenizer + from instancevg.models import build_model + from instancevg.datasets.pipelines.transforms import Normalize, Resize + + torch.set_num_threads(4) + torch.backends.cuda.matmul.allow_tf32 = False + torch.backends.cudnn.allow_tf32 = False + cfg = Config.fromfile(str(root / 'code/third_party/InstanceVG/configs/gres/InstanceVG-grefcoco.py')) + cfg.model.vis_enc.pretrain = None + cfg.model.process_visual = False + model = build_model(cfg.model) + checkpoint_path = root / 'models/grefcoco/instancevg/InstanceVG-grefcoco.pth' + checkpoint = torch.load(checkpoint_path, map_location='cpu') + state = checkpoint['state_dict'] + if all(key.startswith('module.') for key in state): + state = {key[7:]: value for key, value in state.items()} + model.load_state_dict(state, strict=True) + del state, checkpoint + model.eval().to(device).requires_grad_(False) + tokenizer_path = root / 'models/nlvr2/Raghavan--beit3_base_patch16_224_nlvr2/sentencepiece.bpe.model' + tokenizer = XLMRobertaTokenizer(str(tokenizer_path)) + resize = Resize(img_scale=(320, 320), keep_ratio=False) + normalize = Normalize(mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375]) + rows = [json.loads(line) for line in (root / 'outputs/router_full_20260910/data/instancevg/test/records.jsonl').open(encoding='utf-8')] + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chosen = [rows[int(index)] for index in indices] + + def prepare(batch_rows): + images, metas, tokens, masks = [], [], [], [] + for row in batch_rows: + image = mmcv.imread(row['image_path']) + if image.shape[:2] != (row['height'], row['width']): + raise ValueError(f'image shape mismatch for {row["sample_id"]}') + expression = row['expression'] + expression = __import__('re').sub(r"([.,'!?\"()*#:;])", '', expression.lower()).replace('-', ' ').replace('/', ' ') + token_ids = tokenizer.convert_tokens_to_ids(tokenizer.tokenize(expression)) + if not token_ids: + raise ValueError(f'empty token sequence for {row["sample_id"]}') + token_ids = [tokenizer.bos_token_id] + token_ids[:48] + [tokenizer.eos_token_id] + if len(token_ids) > 50: + raise ValueError(f'token sequence exceeds fixed length for {row["sample_id"]}') + masks.append([0] * len(token_ids) + [1] * (50 - len(token_ids))) + tokens.append(token_ids + [tokenizer.pad_token_id] * (50 - len(token_ids))) + data = dict(img=image, filename=row['image_path'], img_shape=image.shape, ori_shape=image.shape, + expression=expression, empty=None, with_bbox=False, with_mask=False) + data = normalize(resize(data)) + images.append(torch.from_numpy(np.ascontiguousarray(data['img'].transpose(2, 0, 1)))) + metas.append({key: data[key] for key in ['filename', 'expression', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'empty']}) + image_tensor = torch.stack(images).to(device) + token_tensor = torch.tensor(tokens, device=device) + mask_tensor = torch.tensor(masks, device=device) + + def forward(): + return model(img=image_tensor, ref_expr_inds=token_tensor, text_attention_mask=mask_tensor, + img_metas=metas, return_loss=False, rescale=True, with_bbox=True, with_mask=False) + + return forward + + batches = [chosen[start:start + batch_size] for start in range(0, len(chosen), batch_size)] + prepared = [prepare(batch) for batch in batches] + for forward in prepared[:2]: + forward() + torch.cuda.synchronize() + forward_ms = [] + end_to_end_ms = [] + for batch in batches: + forward = prepare(batch) + elapsed, _ = timed_cuda(forward) + forward_ms.append(elapsed / len(batch)) + started = time.perf_counter() + forward = prepare(batch) + forward() + torch.cuda.synchronize() + end_to_end_ms.append((time.perf_counter() - started) * 1000 / len(batch)) + flop_rows = [] + for batch in batches[:min(4, len(batches))]: + forward = prepare(batch) + total, _, operators = count_flops(forward) + flop_rows.append({'batch_size': len(batch), 'flops_counted': total, 'gflops_counted_per_sample': total / len(batch) / 1e9, 'operators': operators}) + output.write_text(json.dumps({ + 'component': 'specialist', 'expert': 'instancevg', 'task': 'grefcoco', 'device': device, + 'checkpoint': str(checkpoint_path), 'sample_count': len(chosen), 'batch_size': batch_size, + 'sample_ids': [row['sample_id'] for row in chosen], + 'model_load_seconds_excluded_from_latency': True, + 'forward_latency_ms_per_sample': forward_ms, + 'end_to_end_latency_ms_per_sample': end_to_end_ms, + 'mean_forward_latency_ms_per_sample': float(np.mean(forward_ms)), + 'median_forward_latency_ms_per_sample': float(np.median(forward_ms)), + 'mean_end_to_end_latency_ms_per_sample': float(np.mean(end_to_end_ms)), + 'median_end_to_end_latency_ms_per_sample': float(np.median(end_to_end_ms)), + 'flop_profiles': flop_rows, + 'mean_counted_gflops_per_sample': float(np.mean([row['gflops_counted_per_sample'] for row in flop_rows])), + 'flops_scope': 'GPU FlopCounterMode dense products, convolutions and supported attention; multiply-add counted as 2 FLOPs. Elementwise, normalization, custom ops and postprocessing may be missing.', + 'latency_scope': 'A800 wall-clock CUDA forward latency and end-to-end latency including image read/preprocess, excluding model load; batch processing uses the recorded batch size.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def yolo_measure(root: Path, output: Path, samples: int, batch_size: int, device: str) -> None: + from ultralytics import YOLO + + torch.set_num_threads(4) + torch.backends.cuda.matmul.allow_tf32 = False + torch.backends.cudnn.allow_tf32 = False + checkpoint_path = root / 'models/construction/yolo26x.pt' + model = YOLO(str(checkpoint_path)) + rows = [json.loads(line) for line in (root / 'outputs/router_full_20260910/data/yolo26x/test/records.jsonl').open(encoding='utf-8')] + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chosen = [rows[int(index)] for index in indices] + batches = [chosen[start:start + batch_size] for start in range(0, len(chosen), batch_size)] + + def predict(batch_rows): + return model.predict(source=[row['image_path'] for row in batch_rows], imgsz=640, batch=len(batch_rows), + device=device, half=False, conf=0.001, iou=0.7, max_det=300, save=False, verbose=False) + + for batch in batches[:2]: + predict(batch) + torch.cuda.synchronize() + forward_ms, end_to_end_ms = [], [] + for batch in batches: + elapsed, _ = timed_cuda(lambda batch=batch: predict(batch)) + forward_ms.append(elapsed / len(batch)) + started = time.perf_counter() + predict(batch) + torch.cuda.synchronize() + end_to_end_ms.append((time.perf_counter() - started) * 1000 / len(batch)) + flop_rows = [] + for batch in batches[:min(4, len(batches))]: + total, _, operators = count_flops(lambda batch=batch: predict(batch)) + flop_rows.append({'batch_size': len(batch), 'flops_counted': total, 'gflops_counted_per_sample': total / len(batch) / 1e9, 'operators': operators}) + output.write_text(json.dumps({ + 'component': 'specialist', 'expert': 'yolo26x', 'task': 'construction', 'device': device, + 'checkpoint': str(checkpoint_path), 'sample_count': len(chosen), 'batch_size': batch_size, + 'sample_ids': [row['sample_id'] for row in chosen], + 'model_load_seconds_excluded_from_latency': True, + 'forward_latency_ms_per_sample': forward_ms, + 'end_to_end_latency_ms_per_sample': end_to_end_ms, + 'mean_forward_latency_ms_per_sample': float(np.mean(forward_ms)), + 'median_forward_latency_ms_per_sample': float(np.median(forward_ms)), + 'mean_end_to_end_latency_ms_per_sample': float(np.mean(end_to_end_ms)), + 'median_end_to_end_latency_ms_per_sample': float(np.median(end_to_end_ms)), + 'flop_profiles': flop_rows, + 'mean_counted_gflops_per_sample': float(np.mean([row['gflops_counted_per_sample'] for row in flop_rows])), + 'flops_scope': 'GPU FlopCounterMode around the real Ultralytics predict path; multiply-add counted as 2 FLOPs. Elementwise, postprocessing and unsupported custom ops may be missing.', + 'latency_scope': 'A800 wall-clock latency for real Ultralytics prediction including image loading/preprocess, forward and result postprocessing, excluding model load; batch processing uses the recorded batch size.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def router_measure(root: Path, output: Path, expert: str, endpoint: str, condition: str, seed: int, samples: int, batch_size: int, device: str) -> None: + work = root / 'outputs/router_method_improvement_20260911_r1/reproducible' + sys.path.insert(0, str(work)) + common = __import__('method_improvement_common') + spec = json.loads((root / 'outputs/router_method_improvement_20260911_r1/method_improvement_spec.json').read_text(encoding='utf-8')) + trajectory_id = f'{expert}_{endpoint}_seed{seed}' + cfg = common.make_config(spec, expert, endpoint, seed, trajectory_id) + rows, data = common.load_split(root, spec, cfg, 'test', device) + checkpoint_path = root / f'outputs/router_method_improvement_20260911_r1/conditions/{condition}_{expert}_{endpoint}_seed{seed}/best.pt' + checkpoint = torch.load(checkpoint_path, map_location=device, weights_only=True) + model = common.FeatureRouter(cfg, data['hidden'].shape[1], data['confidence'].shape[1], data['output'].shape[1]).to(device) + model.load_state_dict(checkpoint['state_dict'], strict=True) + model.eval().requires_grad_(False) + if samples > len(rows): + raise ValueError(f'samples {samples} exceeds {len(rows)}') + indices = np.random.default_rng(20260911).choice(len(rows), samples, replace=False) + chunks = [torch.as_tensor(indices[start:start + batch_size], device=device, dtype=torch.long) for start in range(0, samples, batch_size)] + for index in chunks[:2]: + model(data, index) + torch.cuda.synchronize() + latency = [] + for index in chunks: + elapsed, _ = timed_cuda(lambda index=index: model(data, index)) + latency.append(elapsed / len(index)) + index = chunks[0] + total, _, operators = count_flops(lambda: model(data, index)) + output.write_text(json.dumps({ + 'component': 'router', 'expert': expert, 'task': cfg['task'], 'endpoint': endpoint, + 'condition': condition, 'seed': seed, 'device': device, 'checkpoint': str(checkpoint_path), + 'sample_count': samples, 'batch_size': batch_size, 'row_count_loaded': len(rows), + 'mean_latency_ms_per_sample': float(np.mean(latency)), 'median_latency_ms_per_sample': float(np.median(latency)), + 'latency_ms_per_sample': latency, 'flops_counted_per_batch': total, + 'gflops_counted_per_sample': total / len(index) / 1e9, 'operators': operators, + 'flops_scope': 'GPU FlopCounterMode for the router forward; multiply-add counted as 2 FLOPs.', + 'latency_scope': 'A800 CUDA forward latency on cached router features, excluding feature loading and model load.', + 'completed': True, + }, ensure_ascii=False, indent=2)) + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--output', type=Path, required=True) + parser.add_argument('--component', choices=['specialist', 'router'], required=True) + parser.add_argument('--expert', choices=['instancevg', 'yolo26x'], required=True) + parser.add_argument('--endpoint', choices=['MiniCPM', 'Qwen3.8']) + parser.add_argument('--condition', default='both') + parser.add_argument('--seed', type=int, default=42) + parser.add_argument('--samples', type=int, default=64) + parser.add_argument('--batch-size', type=int, default=8) + parser.add_argument('--device', default='cuda:0') + args = parser.parse_args() + if args.output.exists(): + raise FileExistsError(args.output) + args.output.parent.mkdir(parents=True, exist_ok=True) + if not torch.cuda.is_available(): + raise RuntimeError('CUDA is unavailable') + if args.component == 'specialist' and args.expert == 'instancevg': + instancevg_measure(args.root, args.output, args.samples, args.batch_size, args.device) + elif args.component == 'specialist': + yolo_measure(args.root, args.output, args.samples, args.batch_size, args.device) + else: + if args.endpoint is None: + raise ValueError('--endpoint is required for router') + router_measure(args.root, args.output, args.expert, args.endpoint, args.condition, args.seed, args.samples, args.batch_size, args.device) + + +if __name__ == '__main__': + main() diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/prepare_output.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/prepare_output.py new file mode 100644 index 0000000000000000000000000000000000000000..2f5852b9ecd89acd1afbda9caaf75a202b0fa4ed --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/prepare_output.py @@ -0,0 +1,19 @@ +from pathlib import Path +import json +import shutil +root=Path(__file__).resolve().parents[2] +out=root/'dse_results/main_table_cost_20260912' +out.mkdir(exist_ok=True) +(out/'components').mkdir(exist_ok=True) +source=root/'paper_materials/draft_20260912_revision19/data/results.json' +data=json.loads(source.read_text(encoding='utf-8')) +groups=[g for g in data['groups'] if g['key'] in [['grefcoco','instancevg','MiniCPM'],['construction','yolo26x','Qwen3.8']]] +assert len(groups)==2 +(out/'main_table_results.json').write_text(json.dumps({'source':str(source),'protocol':data['protocol'],'groups':groups},indent=2),encoding='utf-8') +scripts=out/'reproducible' +scripts.mkdir(exist_ok=True) +for p in Path(__file__).resolve().parent.glob('*.py'): + shutil.copy2(p,scripts/p.name) +for p in Path(__file__).resolve().parent.glob('*.sh'): + shutil.copy2(p,scripts/p.name) +print(out) diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/probe.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/probe.sh new file mode 100644 index 0000000000000000000000000000000000000000..82ac85243879576913be1d49e120698b7d9ed971 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/probe.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash +set -eu +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +command -v python +python -c 'import torch; print(torch.__version__); import vllm; print(vllm.__version__)' +ls "$ROOT/envs" +ls "$ROOT/outputs/router_full_20260910" +ls "$ROOT/outputs/mini_siglip_strict_20260910_300ep/8M" +sed -n '1,170p' "$ROOT/code/routing/raw_student.py" +sed -n '1,150p' "$ROOT/code/routing/student_artifacts.py" diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/read_main.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/read_main.py new file mode 100644 index 0000000000000000000000000000000000000000..d1e604ed318b6c4d5d3c4d4a9e2c1633662abcdc --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/read_main.py @@ -0,0 +1,10 @@ +from pathlib import Path +import json +root=Path(__file__).resolve().parents[2] +data=json.loads((root/'paper_materials/draft_20260912_revision19/data/results.json').read_text(encoding='utf-8')) +print('TOP',list(data)) +for g in data['groups']: + if g['key'] in [['grefcoco','instancevg','MiniCPM'],['construction','yolo26x','Qwen3.8']]: + print(json.dumps({k:v for k,v in g.items() if k!='methods'},ensure_ascii=False)[:6000]) + for m,v in g['methods'].items(): + print(m, {k:v1 for k,v1 in v.items() if 'curve' not in k}) diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/resume_yolo.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/resume_yolo.sh new file mode 100644 index 0000000000000000000000000000000000000000..c8e59d36337adb23926f1495dff0837607c08484 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/resume_yolo.sh @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +set -euo pipefail +WORK=/tmp/codex_task_main_cost_20260912 +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +python -m pip install --no-deps --target "$WORK/yolo_deps" ultralytics==8.3.222 +export PYTHONPATH="$WORK/yolo_deps${PYTHONPATH:+:$PYTHONPATH}" +bash "$WORK/run_small.sh" 1 yolo26x Qwen3.8 qwen diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/retry_minicpm.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/retry_minicpm.sh new file mode 100644 index 0000000000000000000000000000000000000000..578297b0147ac2bb99abe300e95b8e740a6c243c --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/retry_minicpm.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +set -euo pipefail +OUT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_20260912 +WORK=/tmp/codex_task_main_cost_20260912 +ps -p 6908 -o args= | grep -F "$WORK/measure_large.py" | grep -F 'MiniCPM' +test "$(ps -p 7122 -o ppid= | tr -d ' ')" = 6908 +kill -TERM 7122 6908 +export VLLM_WORKER_MULTIPROC_METHOD=spawn +nohup bash "$WORK/run_large.sh" 0 MiniCPM grefcoco minicpm > "$OUT/logs/minicpm_spawn.log" 2>&1 < /dev/null & +echo $! > "$OUT/logs/minicpm.pid" +cat "$OUT/logs/minicpm.pid" diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_large.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_large.sh new file mode 100644 index 0000000000000000000000000000000000000000..5cf848660977a5fed4c41abd0df9780d39d90bdd --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_large.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +WORK=/tmp/codex_task_main_cost_20260912 +OUT="$ROOT/outputs/main_table_cost_20260912" +mkdir -p "$OUT/components" "$OUT/logs" +GPU=$1 +KIND=$2 +TASK=$3 +TAG=$4 +export CUDA_VISIBLE_DEVICES="$GPU" +export OMP_NUM_THREADS=4 +python "$WORK/measure_large.py" --root "$ROOT" --model-kind "$KIND" --task "$TASK" --output "$OUT/components/${TAG}_latency.json" --samples 66 --batch-size 1 --gpu-memory-utilization .85 +python "$WORK/measure_flops.py" --root "$ROOT" --model-kind "$KIND" --task "$TASK" --output "$OUT/components/${TAG}_flops.json" --samples 8 +echo complete > "$OUT/logs/${TAG}.done" diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_qwen_gref.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_qwen_gref.sh new file mode 100644 index 0000000000000000000000000000000000000000..f8829f9d7ad18801d2e8e1f7b422023cf7102022 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_qwen_gref.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +WORK=/tmp/codex_task_main_cost_20260912 +OUT="$ROOT/outputs/main_table_cost_qwen_20260912" +mkdir -p "$OUT/components" "$OUT/logs" +export CUDA_VISIBLE_DEVICES=0 OMP_NUM_THREADS=4 VLLM_WORKER_MULTIPROC_METHOD=spawn +python "$WORK/measure_large.py" --root "$ROOT" --model-kind Qwen3.8 --task grefcoco --output "$OUT/components/qwen_gref_latency.json" --samples 66 --batch-size 1 --gpu-memory-utilization .85 +python "$WORK/measure_flops.py" --root "$ROOT" --model-kind Qwen3.8 --task grefcoco --output "$OUT/components/qwen_gref_flops.json" --samples 8 +echo complete > "$OUT/logs/qwen_gref.done" diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_small.sh b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_small.sh new file mode 100644 index 0000000000000000000000000000000000000000..036d1cc65fa02e3042e3867c9224d13a08029ce7 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/run_small.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT=/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration +WORK=/tmp/codex_task_main_cost_20260912 +OUT="$ROOT/outputs/main_table_cost_20260912" +GPU=$1 +EXPERT=$2 +ENDPOINT=$3 +TAG=$4 +while [[ ! -f "$OUT/logs/${TAG}.done" ]]; do + PID=$(cat "$OUT/logs/${TAG}.pid") + STATE=$(ps -p "$PID" -o stat= || true) + if [[ -z "$STATE" || "$STATE" == Z* ]]; then + echo "Large-model stage stopped without completion: $TAG" >&2 + exit 1 + fi + sleep 5 +done +export CUDA_VISIBLE_DEVICES="$GPU" +export OMP_NUM_THREADS=4 +PYTHON=python +if [[ "$EXPERT" == instancevg ]]; then PYTHON="$ROOT/envs/instancevg/bin/python"; fi +"$PYTHON" "$WORK/measure_specialist.py" --root "$ROOT" --output "$OUT/components/${EXPERT}_specialist.json" --component specialist --expert "$EXPERT" --samples 64 --batch-size 1 --device cuda:0 +python "$WORK/measure_router.py" --root "$ROOT" --expert "$EXPERT" --endpoint "$ENDPOINT" --output "$OUT/components/${EXPERT}_router.json" +echo complete > "$OUT/logs/${EXPERT}_small.done" diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/setup_local.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/setup_local.py new file mode 100644 index 0000000000000000000000000000000000000000..2e533df9c9a4e92f6b0daa229e641d8f205e7919 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/setup_local.py @@ -0,0 +1,6 @@ +from pathlib import Path +import shutil +root=Path(__file__).resolve().parents[2] +dest=Path(__file__).resolve().parent +for source,target in [('measure_large_vllm_runtime_20260911.py','measure_large.py'),('measure_large_flops_reference_20260911.py','measure_flops.py'),('measure_expert_router_runtime_20260911.py','measure_specialist.py')]: + shutil.copy2(root/'tmp'/source,dest/target) diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/switch_qwen.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/switch_qwen.py new file mode 100644 index 0000000000000000000000000000000000000000..b0e16427d5d8eb9cc13b09e3ae31ad0571fec2bc --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/reproducible/switch_qwen.py @@ -0,0 +1,23 @@ +from pathlib import Path +import json +import shutil +root=Path(__file__).resolve().parents[2] +old=root/'dse_results/main_table_cost_20260912' +out=root/'dse_results/main_table_cost_qwen_20260912' +out.mkdir(exist_ok=True) +(out/'components').mkdir(exist_ok=True) +(out/'reproducible').mkdir(exist_ok=True) +(out/'logs').mkdir(exist_ok=True) +source=root/'paper_materials/draft_20260912_revision19/data/results.json' +data=json.loads(source.read_text(encoding='utf-8')) +groups=[next(g for g in data['groups'] if g['key']==key) for key in [['grefcoco','instancevg','Qwen3.8'],['construction','yolo26x','Qwen3.8']]] +(out/'main_table_results.json').write_text(json.dumps({'source':str(source),'protocol':data['protocol'],'groups':groups},indent=2),encoding='utf-8') +for name in ['instancevg_specialist.json','yolo26x_specialist.json','yolo26x_router.json','qwen_latency.json','qwen_flops.json']: + shutil.copy2(old/'components'/name,out/'components'/name) +for name in ['build_figures.py','verify_and_report.py','latex_includes.tex']: + text=(old/name).read_text(encoding='utf-8').replace("['minicpm','qwen']","['qwen_gref','qwen']").replace('gRefCOCO / M','gRefCOCO / Q').replace('InstanceVG--MiniCPM','InstanceVG--Qwen').replace('main_table_cost_20260912','main_table_cost_qwen_20260912') + (out/name).write_text(text,encoding='utf-8') +for p in (old/'reproducible').iterdir(): + if p.is_file(): shutil.copy2(p,out/'reproducible'/p.name) +for g in groups: + print(g['key'],g['run'],{m:g['methods'][m]['test_point'] for m in ['learned','confidence']}) diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/selected_points.csv b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/selected_points.csv new file mode 100644 index 0000000000000000000000000000000000000000..dd2dc668ab26f4f391bb1747866a2a167ad1d971 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/selected_points.csv @@ -0,0 +1,5 @@ +pair,method,nominal_validation_budget,actual_test_call_fraction,test_metric,estimated_gflops_per_input,estimated_time_ms_per_input +grefcoco|instancevg|Qwen3.8,learned,0.56,0.5355755324277571,0.7528287440093016,17446.303931938,2243.6672827746106 +grefcoco|instancevg|Qwen3.8,confidence,0.71,0.6838612710206166,0.7243286163967898,22269.96488623913,2830.3683522642846 +construction|yolo26x|Qwen3.8,learned,0.5,0.4876830892143808,0.29875800068449215,47383.321370531434,4861.824738942901 +construction|yolo26x|Qwen3.8,confidence,0.93,0.9221038615179761,0.2597490004762105,89464.5250591521,9103.651026133362 diff --git a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/verify_and_report.py b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/verify_and_report.py new file mode 100644 index 0000000000000000000000000000000000000000..0b2067240b13934a01d0088d8a37d55b66773d54 --- /dev/null +++ b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/verify_and_report.py @@ -0,0 +1,57 @@ +from pathlib import Path +import csv +import json +import fitz + +root = Path(__file__).resolve().parent +summary = json.loads((root/'cost_summary.json').read_text()) +groups = json.loads((root/'main_table_results.json').read_text())['groups'] +rows = list(csv.DictReader((root/'full_cost_performance_curves.csv').open())) +assert len(rows) == 404 +lines = ['# 主表方法:成本—性能曲线', '', '## 1 做了什么', '', + '- 使用主表原检查点、种子42及原验证集阈值;未重新训练,也未改用后续改进方法。', + '- gRefCOCO 改为 InstanceVG + Qwen,重新测量该任务上的 Qwen 成本及对应路由器;Construction 和 InstanceVG 专家复用上一轮同口径实测。', + '- 在 A800 的0、1号卡测量;每个大模型预热2张、正式测量64张,每次1张。专家和路由部分各测64张。', + '- 重算原路由头,并核对全测试集的选定阈值调用决定与主表完全一致。性能数值沿用主表固定的预测记录。', + '- 只画学习路由与置信度路由;共404个曲线点。另提供单栏四面板紧凑版。', '', + '## 2 为什么做', '', '让成本曲线与主实验比较同一个方法,避免混入后续改进结果;补计蒸馏编码器及路由头开销。', '', + '## 3 做完有什么效果', '', '| 数据集 | 方法 | 测试性能(%) | 调用率(%) | 估算GFLOPs/输入 | 估算毫秒/输入 |', + '|---|---|---:|---:|---:|---:|'] +comparisons = [] +for group in groups: + key = '|'.join(group['key']) + chosen = {p['method']:p for p in summary['selected_points'] if p['pair']==key} + for method,label in [('learned','学习路由'),('confidence','置信度路由')]: + p = chosen[method] + original = group['methods'][method]['test_point'] + assert p['test_metric'] == original['metric'] + assert p['actual_test_call_fraction'] == original['actual_fraction'] + lines.append(f"| {group['key'][0]} | {label} | {100*p['test_metric']:.2f} | {100*p['actual_test_call_fraction']:.2f} | {p['estimated_gflops_per_input']:,.1f} | {p['estimated_time_ms_per_input']:,.1f} |") + a,b = chosen['learned'],chosen['confidence'] + comparisons.append(f"{group['key'][0]}:相对置信度路由,性能提高 {100*(a['test_metric']-b['test_metric']):.2f} 个百分点,估算 FLOPs 减少 {100*(1-a['estimated_gflops_per_input']/b['estimated_gflops_per_input']):.1f}%,估算耗时减少 {100*(1-a['estimated_time_ms_per_input']/b['estimated_time_ms_per_input']):.1f}%。") +for comparison in comparisons: + lines += ['', comparison] +lines += [''] +lines += ['### 数值口径', '', + '- 每个曲线点使用验证集确定的阈值,横轴按该阈值在测试集上的实际调用率计算;圆点对应主表选定点,没有按测试集重新选点。', + '- 置信度路由成本 = 专家成本 + 实际调用率 × 大模型平均成本。学习路由再加编码器和路由头成本。', + '- 耗时为分项实测后相加的串行估算,并非各策略整套系统的直接端到端计时。加载模型和预热不计入。', + '- FLOPs 按乘加计2次统计支持的算子;大模型用8个输入估计预填充成本(含视觉编码),再加平均输出长度乘单词元解码成本。专家统计4个输入,编码器与路由头统计8个输入。', + '- 各策略共用大模型平均单次成本,尚未测量所选样本的长度差异对成本的影响;专家隐藏特征来自缓存,其额外提取开销未单独测量。置信度阈值判断开销未单独计时。', + '- 本轮单张测量与上一轮批量测量口径不同,绝对耗时不宜直接比较。', '', + '- YOLO 使用权重记录的依赖版本8.3.222,安装在任务临时目录;结束时0、1号卡显存均为0,未操作2、3号卡服务。', '', + '## 4 对应文件在哪', '', + '- `cost_compact.pdf`:推荐插入正文的单栏四面板图,宽3.44英寸、高2.05英寸。', + '- `flops_performance.pdf`、`latency_performance.pdf`:两张独立双面板图。', + '- `full_cost_performance_curves.csv`:全部404个点;`selected_points.csv`:主表4个运行点。', + '- `components/`:8份实测记录,含原始回答、错误记录及路由核验。', + '- `build_figures.py`:绘图及成本计算;`reproducible/`:远端测量脚本;`latex_includes.tex`:插入代码及图注。', + '- 远端结果:`/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_qwen_20260912`。', '', + '### 检查记录', '', '已核对404个曲线点、4个主表运行点、两组全测试集调用决定及单位。PDF文字为嵌入字体,另渲染检查排版。'] +(root/'成本性能报告.md').write_text('\n'.join(lines)+'\n',encoding='utf-8') +for stem in ['cost_compact','flops_performance','latency_performance']: + doc = fitz.open(root/(stem+'.pdf')) + assert len(doc)==1 + assert all(font[1] != 'n/a' for font in doc[0].get_fonts()) + doc[0].get_pixmap(matrix=fitz.Matrix(4,4)).save(root/(stem+'_verified.png')) +print('\n'.join(lines)) diff --git "a/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" "b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" new file mode 100644 index 0000000000000000000000000000000000000000..51fdd1d3a68014b53f5e26765afc5b7a13112c0d --- /dev/null +++ "b/releases/20260913_lr/paper_data/main_table_cost_qwen_20260912/\346\210\220\346\234\254\346\200\247\350\203\275\346\212\245\345\221\212.md" @@ -0,0 +1,50 @@ +# 主表方法:成本—性能曲线 + +## 1 做了什么 + +- 使用主表原检查点、种子42及原验证集阈值;未重新训练,也未改用后续改进方法。 +- gRefCOCO 改为 InstanceVG + Qwen,重新测量该任务上的 Qwen 成本及对应路由器;Construction 和 InstanceVG 专家复用上一轮同口径实测。 +- 在 A800 的0、1号卡测量;每个大模型预热2张、正式测量64张,每次1张。专家和路由部分各测64张。 +- 重算原路由头,并核对全测试集的选定阈值调用决定与主表完全一致。性能数值沿用主表固定的预测记录。 +- 只画学习路由与置信度路由;共404个曲线点。另提供单栏四面板紧凑版。 + +## 2 为什么做 + +让成本曲线与主实验比较同一个方法,避免混入后续改进结果;补计蒸馏编码器及路由头开销。 + +## 3 做完有什么效果 + +| 数据集 | 方法 | 测试性能(%) | 调用率(%) | 估算GFLOPs/输入 | 估算毫秒/输入 | +|---|---|---:|---:|---:|---:| +| grefcoco | 学习路由 | 75.28 | 53.56 | 17,446.3 | 2,243.7 | +| grefcoco | 置信度路由 | 72.43 | 68.39 | 22,270.0 | 2,830.4 | +| construction | 学习路由 | 29.88 | 48.77 | 47,383.3 | 4,861.8 | +| construction | 置信度路由 | 25.97 | 92.21 | 89,464.5 | 9,103.7 | + +grefcoco:相对置信度路由,性能提高 2.85 个百分点,估算 FLOPs 减少 21.7%,估算耗时减少 20.7%。 + +construction:相对置信度路由,性能提高 3.90 个百分点,估算 FLOPs 减少 47.0%,估算耗时减少 46.6%。 + +### 数值口径 + +- 每个曲线点使用验证集确定的阈值,横轴按该阈值在测试集上的实际调用率计算;圆点对应主表选定点,没有按测试集重新选点。 +- 置信度路由成本 = 专家成本 + 实际调用率 × 大模型平均成本。学习路由再加编码器和路由头成本。 +- 耗时为分项实测后相加的串行估算,并非各策略整套系统的直接端到端计时。加载模型和预热不计入。 +- FLOPs 按乘加计2次统计支持的算子;大模型用8个输入估计预填充成本(含视觉编码),再加平均输出长度乘单词元解码成本。专家统计4个输入,编码器与路由头统计8个输入。 +- 各策略共用大模型平均单次成本,尚未测量所选样本的长度差异对成本的影响;专家隐藏特征来自缓存,其额外提取开销未单独测量。置信度阈值判断开销未单独计时。 +- 本轮单张测量与上一轮批量测量口径不同,绝对耗时不宜直接比较。 + +- YOLO 使用权重记录的依赖版本8.3.222,安装在任务临时目录;结束时0、1号卡显存均为0,未操作2、3号卡服务。 + +## 4 对应文件在哪 + +- `cost_compact.pdf`:推荐插入正文的单栏四面板图,宽3.44英寸、高2.05英寸。 +- `flops_performance.pdf`、`latency_performance.pdf`:两张独立双面板图。 +- `full_cost_performance_curves.csv`:全部404个点;`selected_points.csv`:主表4个运行点。 +- `components/`:8份实测记录,含原始回答、错误记录及路由核验。 +- `build_figures.py`:绘图及成本计算;`reproducible/`:远端测量脚本;`latex_includes.tex`:插入代码及图注。 +- 远端结果:`/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/main_table_cost_qwen_20260912`。 + +### 检查记录 + +已核对404个曲线点、4个主表运行点、两组全测试集调用决定及单位。PDF文字为嵌入字体,另渲染检查排版。 diff --git a/releases/20260913_lr/paper_data/method_improvement_spec.json b/releases/20260913_lr/paper_data/method_improvement_spec.json new file mode 100644 index 0000000000000000000000000000000000000000..8a745d12752d9ef54290b788165fc3f86f83ccf2 --- /dev/null +++ b/releases/20260913_lr/paper_data/method_improvement_spec.json @@ -0,0 +1,114 @@ +{ + "name": "router_method_improvement_20260911", + "remote_root": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration", + "remote_work": "outputs/router_method_improvement_20260911", + "data_work": "outputs/router_full_20260910", + "student": "8M", + "student_checkpoint": "outputs/mini_siglip_strict_20260910_300ep/8M/epoch_300.pt", + "fixed": { + "fraction": 1.0, + "freeze_encoder": true, + "fusion": "concat", + "harm_cost": 1.0, + "batch_size": 256, + "weight_decay": 0.01, + "max_epochs": 50, + "patience": 8, + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90, + "cpu_threads": 1 + }, + "seeds": [42, 43, 44], + "experts": { + "cub": { + "task": "cub", + "hidden_layer": "residual_blocks__layer_15", + "hidden_depth": 16, + "branches": ["image", "hidden", "confidence", "output"], + "confidence_column": "max_probability", + "confidence_valid_column": "max_probability_valid", + "expected_rows": {"train": 4792, "val": 1200, "test": 5794} + }, + "glsim": { + "task": "cub", + "hidden_layer": "global_local_encoder__layer_11", + "hidden_depth": 12, + "branches": ["image", "hidden", "confidence", "output"], + "confidence_column": "max_probability", + "confidence_valid_column": "max_probability_valid", + "expected_rows": {"train": 4792, "val": 1200, "test": 5794} + }, + "groundingdino": { + "task": "grefcoco", + "hidden_layer": "multimodal_encoder__layer_06", + "hidden_depth": 6, + "branches": ["image", "text", "hidden", "confidence", "output"], + "confidence_column": "max_score", + "confidence_valid_column": "max_score_valid", + "expected_rows": {"train": 209344, "val": 14229, "test": 35263} + }, + "instancevg": { + "task": "grefcoco", + "hidden_layer": "joint_encoder__layer_11", + "hidden_depth": 12, + "branches": ["image", "text", "hidden", "confidence", "output"], + "confidence_column": "max_score", + "confidence_valid_column": "max_score_valid", + "expected_rows": {"train": 209344, "val": 14229, "test": 35263} + }, + "nlvr2": { + "task": "nlvr2", + "hidden_layer": "multimodal_encoder__layer_11", + "hidden_depth": 12, + "branches": ["image", "text", "hidden", "confidence", "output"], + "confidence_column": "max_probability", + "confidence_valid_column": "max_probability_valid", + "expected_rows": {"train": 85042, "val": 6982, "test": 13937} + }, + "vilt": { + "task": "nlvr2", + "hidden_layer": "ordered_image_text__layer_11", + "hidden_depth": 12, + "branches": ["image", "text", "hidden", "confidence", "output"], + "confidence_column": "max_probability", + "confidence_valid_column": "max_probability_valid", + "expected_rows": {"train": 85042, "val": 6982, "test": 13937} + }, + "yolo26x": { + "task": "construction", + "hidden_layer": "feature_blocks__layer_16", + "hidden_depth": 17, + "branches": ["image", "hidden", "confidence", "output"], + "confidence_column": "max_score", + "confidence_valid_column": "max_score_valid", + "expected_rows": {"train": 6308, "val": 701, "test": 3004} + }, + "rtdetr_x_fp32": { + "task": "construction", + "hidden_layer": "query_decoder__layer_06", + "hidden_depth": 6, + "branches": ["image", "hidden", "confidence", "output"], + "confidence_column": "max_score", + "confidence_valid_column": "max_score_valid", + "expected_rows": {"train": 6308, "val": 701, "test": 3004} + } + }, + "conditions": [ + {"name": "original", "selection_mode": "a25", "score_mode": "s_log"}, + {"name": "score_gain", "selection_mode": "a25", "score_mode": "s_gain"}, + {"name": "checkpoint_perf", "selection_mode": "max_task_performance", "score_mode": "s_log"}, + {"name": "both", "selection_mode": "max_task_performance", "score_mode": "s_gain"} + ], + "score_only_control": {"name": "score_only_gain", "source_condition": "original", "score_mode": "s_gain"}, + "budgets": { + "a25": [0.0, 0.05, 0.10, 0.15, 0.20, 0.25], + "historical_threshold": [0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.30, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.40, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.50, 0.51, 0.52, 0.53, 0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.70, 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79, 0.80, 0.81, 0.82, 0.83, 0.84, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99], + "endpoint": [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.10, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.30, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.40, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.50, 0.51, 0.52, 0.53, 0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.70, 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79, 0.80, 0.81, 0.82, 0.83, 0.84, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99, 1.0], + "batch_diagnostic": [0.0, 0.05, 0.10, 0.15, 0.20, 0.25, 0.50, 0.75, 1.0] + }, + "fusion_alphas": [0.0, 0.25, 0.5, 0.75, 1.0], + "tie_epsilon": 1e-12 +} diff --git a/releases/20260913_lr/paper_data/method_lock_five_seeds.json b/releases/20260913_lr/paper_data/method_lock_five_seeds.json new file mode 100644 index 0000000000000000000000000000000000000000..75c0103e724d2a4693f463cc9c644dd1607bf60d --- /dev/null +++ b/releases/20260913_lr/paper_data/method_lock_five_seeds.json @@ -0,0 +1,81 @@ +{ + "selected_method": "four_difference", + "selection": "task-balanced validation Score25 mean minus sample standard deviation over five router seeds", + "repeat_seeds": [ + 2026, + 2027, + 2028, + 2029, + 2030 + ], + "comparison": [ + { + "method": "error", + "validation_seed_scores": [ + 0.652854712535587, + 0.653201861091734, + 0.653549705081995, + 0.6529296233301471, + 0.6533734477104217 + ], + "validation_mean": 0.653181869949977, + "validation_std": 0.00029285862604605935, + "selection_value": 0.6528890113239308 + }, + { + "method": "four_difference", + "validation_seed_scores": [ + 0.6659279432360811, + 0.6666052399411169, + 0.6664902552160231, + 0.666460246201781, + 0.6671766800294149 + ], + "validation_mean": 0.6665320729248834, + "validation_std": 0.0004451831677229395, + "selection_value": 0.6660868897571605 + }, + { + "method": "four_ratio", + "validation_seed_scores": [ + 0.6669800355113474, + 0.6654694961331916, + 0.6669276041336858, + 0.6664081580190494, + 0.6645222674050018 + ], + "validation_mean": 0.6660615122404552, + "validation_std": 0.001052820164752406, + "selection_value": 0.6650086920757028 + }, + { + "method": "gain", + "validation_seed_scores": [ + 0.6656365114137299, + 0.6665939365548585, + 0.6663926476912081, + 0.6648075719596137, + 0.6656891621835576 + ], + "validation_mean": 0.6658239659605936, + "validation_std": 0.0007075700773489459, + "selection_value": 0.6651163958832447 + }, + { + "method": "route", + "validation_seed_scores": [ + 0.6522091985724042, + 0.6522614154026475, + 0.6528619080142624, + 0.6518457956309271, + 0.6514639634291153 + ], + "validation_mean": 0.6521284562098713, + "validation_std": 0.0005204797508024464, + "selection_value": 0.6516079764590688 + } + ], + "selector_reads_test_records": false, + "historical_pilot_selected_method": "four_ratio", + "rule_amendment": "User changed selection criterion after the seed-42 pilot and partial CUB test results had been reported. The final numerical criterion uses validation records only; this is not a prospectively untouched test set." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/objective_mean_std.csv b/releases/20260913_lr/paper_data/objective_mean_std.csv new file mode 100644 index 0000000000000000000000000000000000000000..f50fea50083e04f07dce158210470f30cc2f0c2f --- /dev/null +++ b/releases/20260913_lr/paper_data/objective_mean_std.csv @@ -0,0 +1,6 @@ +method,validation_score25_mean,validation_score25_std,test_fixed_score25_mean,test_fixed_score25_std,test_ranked_score25_mean,test_ranked_score25_std,validation_metric_mean,validation_metric_std,test_metric_mean,test_metric_std,validation_call_mean,validation_call_std,test_call_mean,test_call_std,validation_mean_minus_std +error,0.653181869949977,0.00029285862604605935,0.5623434954185125,0.0005865578956539791,0.5824310668534718,0.0004537183262375207,0.7327345991681659,0.0007294476116559371,0.6680335653281058,0.0011201126898523841,0.4987538747529051,0.010232411153173287,0.5340324580222858,0.011615989761374094,0.6528890113239308 +four_difference,0.6665320729248834,0.0004451831677229395,0.5966008930470663,0.0006800302851502677,0.5919492621796143,0.0004530494994020287,0.7311304050310635,0.0003132384785791997,0.6742025885780812,0.0004984749682483765,0.5472576238775222,0.0043082642498178045,0.5666366527967134,0.0036071787229059135,0.6660868897571605 +four_ratio,0.6660615122404552,0.001052820164752406,0.5832913753284377,0.0017921552388281987,0.5874390286163175,0.0010028619279574815,0.7345261185173315,0.001016396826168321,0.6696234867607541,0.0031125297264799506,0.49825541694902187,0.01359322573042264,0.5282950880372231,0.00858481729720522,0.6650086920757028 +gain,0.6658239659605936,0.0007075700773489459,0.5954408739908841,0.0006431142672200772,0.5912682221089238,0.0004560314657539302,0.7304688805207613,0.0003308377568018673,0.6733461562700767,0.0003218018899692329,0.5495056533388056,0.011611002500379745,0.5665729657740511,0.010940868975414547,0.6651163958832447 +route,0.6521284562098713,0.0005204797508024464,0.5698550507031748,0.002651025242534859,0.5845427411273968,0.0004120921867447789,0.7333419016671792,0.0006129222977505627,0.6726232865905931,0.0011190488025117787,0.4890204854293394,0.008068411702983952,0.5207177256972306,0.007986317831949974,0.6516079764590688 diff --git a/releases/20260913_lr/paper_data/operation_measurements.json b/releases/20260913_lr/paper_data/operation_measurements.json new file mode 100644 index 0000000000000000000000000000000000000000..325a3b87dc347865f6656f58c6b6d6a7d8c9d925 --- /dev/null +++ b/releases/20260913_lr/paper_data/operation_measurements.json @@ -0,0 +1,129 @@ +{ + "small_models": [ + { + "model": "ResNet", + "expert": "cub", + "samples": 32, + "mean_gflops_counted": 8.175091712, + "min_gflops_counted": 8.175091712, + "max_gflops_counted": 8.175091712, + "operator_inventory_source": "cub_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": false + }, + { + "model": "YOLO26x", + "expert": "yolo26x", + "samples": 32, + "mean_gflops_counted": 143.2219032, + "min_gflops_counted": 116.2238976, + "max_gflops_counted": 193.92768, + "operator_inventory_source": "yolo26x_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": false + }, + { + "model": "RT-DETR-X", + "expert": "rtdetr_x_fp32", + "samples": 32, + "mean_gflops_counted": 224.6444544, + "min_gflops_counted": 224.6444544, + "max_gflops_counted": 224.6444544, + "operator_inventory_source": "rtdetr_x_fp32_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": true + }, + { + "model": "ViLT", + "expert": "vilt", + "samples": 32, + "mean_gflops_counted": 84.895706112, + "min_gflops_counted": 57.611655168, + "max_gflops_counted": 120.690972672, + "operator_inventory_source": "vilt_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": false + }, + { + "model": "GroundingDINO", + "expert": "groundingdino", + "samples": 32, + "mean_gflops_counted": 668.259162288, + "min_gflops_counted": 499.212779776, + "max_gflops_counted": 820.17636864, + "operator_inventory_source": "groundingdino_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": true + }, + { + "model": "BEiT3", + "expert": "nlvr2", + "samples": 32, + "mean_gflops_counted": 76.799401728, + "min_gflops_counted": 73.586743296, + "max_gflops_counted": 83.259561984, + "operator_inventory_source": "nlvr2_operator_check.json", + "operator_check_samples": 1, + "uncovered_spatial_sampling_seen": false + }, + { + "model": "GLSim", + "expert": "glsim", + "samples": 32, + "mean_gflops_counted": 70.280871936, + "min_gflops_counted": 70.280871936, + "max_gflops_counted": 70.280871936, + "operator_inventory_source": "glsim_ops.json", + "operator_check_samples": 32, + "uncovered_spatial_sampling_seen": false + } + ], + "large_input_cases": 16, + "large_input_check_complete": true, + "large_model_probes": [ + { + "model": "MiniCPM", + "sample_id": "cub-test-00517", + "task": "cub", + "prompt_tokens": 1996, + "prefill_gflops_counted": 33243.606087168, + "one_known_token_gflops_counted": 16.31253312 + }, + { + "model": "Qwen3.8", + "sample_id": "cub-test-00517", + "task": "cub", + "prompt_tokens": 4864, + "prefill_gflops_counted": 249018.23668224, + "one_known_token_gflops_counted": 53.164769472 + } + ], + "large_gpu_probe_status": { + "stage": "finished", + "occupancy_original_pid": 157057, + "stopped_process_ids": [ + 157068, + 157342, + 157343, + 157587, + 157588, + 157589, + 157590, + 157057 + ], + "started_at": 1789028871.3039298, + "measurement_exit_code": 0, + "occupancy_restored": true, + "measurement_pid": 159330, + "occupancy_new_pid": 159642, + "finished_at": 1789029043.4601417 + }, + "whole_system_flops_complete": false, + "missing_components": [ + "InstanceVG", + "full generation and all-task large-model coverage", + "uncovered operations", + "per-example routing alignment", + "95%/99% resampling intervals" + ] +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/pair_mean_std.csv b/releases/20260913_lr/paper_data/pair_mean_std.csv new file mode 100644 index 0000000000000000000000000000000000000000..a7cee106bd58870840e37c05916ff0d2812ab220 --- /dev/null +++ b/releases/20260913_lr/paper_data/pair_mean_std.csv @@ -0,0 +1,81 @@ +task,expert,endpoint,method,validation_score25_mean,validation_score25_std,test_fixed_score25_mean,test_fixed_score25_std,test_ranked_score25_mean,test_ranked_score25_std,validation_metric_mean,validation_metric_std,test_metric_mean,test_metric_std,validation_call_mean,validation_call_std,test_call_mean,test_call_std +cub,cub,MiniCPM,error,0.9043833333333332,0.00048805168214491085,0.6549085260614429,0.005037074161490198,0.7764515015533311,0.005093383797235659,0.9818333333333333,0.0010865337342004667,0.7956851915774938,0.013595084245079669,0.01,0.0,0.06948567483603726,0.027759372885887968 +cub,cub,MiniCPM,four_difference,0.9743333333333333,0.0004750730937912844,0.8064480497065931,0.0021628719115412928,0.8088850535036245,0.0012530132339606866,0.9865,0.0006972166887784124,0.8139454608215395,0.000585289090708094,0.01,0.0,0.013841905419399378,0.0046301734514364885 +cub,cub,MiniCPM,four_ratio,0.9492166666666666,0.01094757076047265,0.7502726958923025,0.025634637867592076,0.7956955471177081,0.004983207347578584,0.985,0.0011785113019776065,0.811460131170176,0.003664489341274483,0.01,0.0,0.04449430445288229,0.00864203455894296 +cub,cub,MiniCPM,gain,0.9737333333333333,0.00039264063297966096,0.8057991025198483,0.002817936601711226,0.8078633068691751,0.002142113573427962,0.9858333333333335,1.2412670766236366e-16,0.8139799792889196,0.0013936230057429231,0.01,0.0,0.024646185709354507,0.008823672451964179 +cub,cub,MiniCPM,route,0.9076666666666666,0.0015822364621706617,0.6833931653434588,0.014706459242787717,0.7798412150500518,0.00103943009491982,0.9814999999999999,0.0010865337342004665,0.8081118398343113,0.00264691605283824,0.01,0.0,0.02982395581636175,0.005011720100208917 +cub,cub,Qwen3.8,error,0.9195666666666666,0.0015142838424666554,0.7454677252329998,0.004138251756163409,0.7961132205730065,0.0037080129051114594,0.9804999999999999,0.00045643546458764894,0.8081808767690715,0.004054267100566989,0.01,0.0,0.04697963410424577,0.013784931688472545 +cub,cub,Qwen3.8,four_difference,0.9834833333333333,0.0003084459253887113,0.8129651363479461,0.0001327951954551148,0.8117500862961684,0.0005625027599448041,0.9858333333333335,1.2412670766236366e-16,0.8142906454953401,0.00012204121180300483,0.01,0.0,0.008146358301691404,0.0018118085567153201 +cub,cub,Qwen3.8,four_ratio,0.9644833333333332,0.00836938037000215,0.7995305488436314,0.007857354044301107,0.8046254746289264,0.0028245356848001254,0.9843333333333334,0.0014907119849998601,0.812875388332758,0.0010239832222431171,0.01,0.0,0.022333448394891264,0.012564674710770952 +cub,cub,Qwen3.8,gain,0.9817499999999999,0.0005137011669140923,0.8125163962720056,0.00031171915298276585,0.8114981014842941,0.0008376864683897515,0.9855,0.00045643546458764894,0.8142216085605799,0.00026174923866248607,0.01,0.0,0.011114946496375561,0.007037874481504933 +cub,cub,Qwen3.8,route,0.9208500000000001,0.0015694567354484427,0.7514739385571282,0.006819227851398293,0.8002174663444942,0.00292223343038228,0.9793333333333333,0.0006972166887783726,0.8127027959958578,0.0005903566232822391,0.01,0.0,0.022989299275112184,0.002065346952247372 +cub,glsim,MiniCPM,error,0.9108,0.001903578501431243,0.7618018639972386,0.004596620314287549,0.8451950293406973,0.0015486544105787873,0.9868333333333335,0.00037267799624997355,0.8867794269934415,0.0023946214921490576,0.01,0.0,0.039523645150155336,0.002715245551010056 +cub,glsim,MiniCPM,four_difference,0.9787166666666666,0.0002738612787525529,0.8915395236451502,0.0008214379924671345,0.8924439074905074,0.0006071964231972181,0.9893333333333333,0.00037267799624997355,0.901484294097342,0.00014440110917050722,0.01,0.0,0.013289609941318605,0.00372375716437546 +cub,glsim,MiniCPM,four_ratio,0.9700666666666666,0.0012766601392348224,0.8716775975146703,0.002696586118271531,0.8807939247497412,0.0012344515327090803,0.9890000000000001,0.0014907119849998508,0.8995167414566794,0.0011474470237207235,0.014000000000000002,0.005477225575051661,0.028132550914739384,0.008585492994896726 +cub,glsim,MiniCPM,gain,0.9785333333333334,0.00026744677559799074,0.8913807386952021,0.0011066520510041504,0.8929444252675183,0.0004230096988763061,0.9891666666666665,1.2412670766236366e-16,0.9015188125647221,0.0002617492386624421,0.01,0.0,0.01591301346220228,0.002039950889484623 +cub,glsim,MiniCPM,route,0.9012833333333333,0.0008220063530438317,0.7546530894028305,0.017484619999829788,0.8502657921988263,0.0007734563392064111,0.9846666666666666,0.001263812574008621,0.8903003106662064,0.0036928307483028735,0.01,0.0,0.04266482568173973,0.008699578332443097 +cub,glsim,Qwen3.8,error,0.9357333333333333,0.0014981470036163184,0.8150569554711771,0.001975959909892733,0.8662064204349326,0.0005027446207673551,0.9848333333333334,0.0006972166887784124,0.8947531929582325,0.0018882895266286348,0.01,0.0,0.038246461857093544,0.003534141553118471 +cub,glsim,Qwen3.8,four_difference,0.9862166666666667,0.00034661057238474823,0.8974387297204004,0.0003871996662839683,0.8969036934760097,0.00018532591916529473,0.9890000000000001,0.00037267799624997355,0.9014152571625818,0.00037417127008418876,0.01,0.0,0.008215395236451501,0.002613506210765189 +cub,glsim,Qwen3.8,four_ratio,0.9762333333333334,0.001975896421036943,0.8879254401104593,0.0043313902986515445,0.891408353469106,0.0023735180300587344,0.9875,0.0010206207261596358,0.9005177770107007,0.0004972337000651875,0.01,0.0,0.017190196755264068,0.005837092226331363 +cub,glsim,Qwen3.8,gain,0.9852000000000001,0.0006417207769393092,0.8956092509492578,0.0020768029516458754,0.8952916810493614,0.0016337302893772345,0.9884999999999999,0.0009128709291752371,0.9008629616845012,0.0011487443073727145,0.01,0.0,0.011391094235415948,0.00538504686290048 +cub,glsim,Qwen3.8,route,0.92515,0.002299909418506208,0.7938384535726615,0.007721293913312285,0.8631204694511563,0.0036321646217745583,0.9828333333333333,0.0007453559924998913,0.8952709699689333,0.0022216864349369875,0.01,0.0,0.024956851915774937,0.0049702397897606635 +grefcoco,groundingdino,MiniCPM,error,0.6210682409164383,0.0009278213191392739,0.32351813515582906,0.00600652764795926,0.33151745455576664,0.0012940834975593042,0.6770398481973434,0.005618058332376438,0.45265575816011117,0.012077372145864079,0.3479654227282311,0.013046877094502269,0.4526103848226186,0.03314363208971995 +grefcoco,groundingdino,MiniCPM,four_difference,0.6137648464403682,0.0020091906683944055,0.3761943113178119,0.00451225219660107,0.34934180302299855,0.0006879617740834799,0.6562513177313936,0.0022540834511329062,0.49913507075404817,0.005887753308252541,0.4439665471923536,0.03360550791504779,0.5624138615546039,0.03987669902841215 +grefcoco,groundingdino,MiniCPM,four_ratio,0.625737578185396,0.0006219672325087774,0.3358925786234864,0.009258320790970132,0.3379752148143947,0.0034946718480593076,0.6801742919389978,0.004772353442217688,0.4605960922213085,0.008720430379345945,0.3379717478389205,0.013044188980017753,0.4383631568499561,0.021818814102269945 +grefcoco,groundingdino,MiniCPM,gain,0.614033312249631,0.0028476200552817986,0.37808864815812615,0.0022935649879668374,0.34947962453563225,0.0008158284668645607,0.6562372619298615,0.0034606970465091472,0.4952443070640615,0.005071323680743823,0.4299669688663997,0.036734414444423956,0.5468337917930975,0.03868040637573763 +grefcoco,groundingdino,MiniCPM,route,0.6111223557523368,0.0013447057578513206,0.37844142585713075,0.001884887095658619,0.34925162351473216,0.0007971331001151746,0.6562934851359898,0.003268011640403439,0.49978731248050356,0.009799169535157469,0.47595755147937313,0.07797406967337053,0.5931145960355046,0.07913626739982761 +grefcoco,groundingdino,Qwen3.8,error,0.6641844121161009,0.0006409353025584052,0.35656410401837624,0.008431473714078902,0.3702696877747214,0.0014255436768525348,0.808545927331506,0.001509281573748924,0.7479681252304115,0.0014444111780072732,0.8079696394686907,0.07461922413173061,0.9098942234069705,0.04580720870865557 +grefcoco,groundingdino,Qwen3.8,four_difference,0.6484278585986366,0.002552755685953187,0.42230780137821516,0.0038522524472683673,0.3748864248645889,0.0009344435306377577,0.8127205003865345,0.0014669638122516435,0.7511102288517709,0.0012934323753779572,0.7859723100709818,0.05939858081016824,0.8917222017412019,0.03061690111015076 +grefcoco,groundingdino,Qwen3.8,four_ratio,0.6681059807435519,0.00033921544682756597,0.35779655729801785,0.0034213256361630895,0.37301534185973967,0.001546992842135703,0.8140698573336145,0.002380996356522431,0.7164279840058987,0.04781699237960433,0.6399746995572422,0.18357088955100645,0.7694240421972038,0.16649655087770804 +grefcoco,groundingdino,Qwen3.8,gain,0.64726403823178,0.003378401153004879,0.4091189065025665,0.005116857858126231,0.3739528684456796,0.0012501282781362643,0.8086021505376344,0.001488853769272928,0.7481099169100757,0.0011598524728590547,0.8599620493358634,0.09026851710833313,0.9264441482573803,0.04489441381375845 +grefcoco,groundingdino,Qwen3.8,route,0.6464755077658303,0.0020374064926579048,0.4191600260896691,0.003996773503553001,0.3745648413351105,0.0006566084717211617,0.8095860566448803,0.0015797933077241991,0.7482800669256728,0.0008087347957697223,0.7599690772366294,0.019994383855045934,0.8708447948274396,0.009520603360520385 +grefcoco,instancevg,MiniCPM,error,0.7575753742357159,0.0025670440546817206,0.6707773019879193,0.0019246865481410181,0.6760292658026827,0.00037120942583228534,0.7623163960924872,0.005124828894220894,0.670192553100984,0.007641979616213665,0.16397498067327287,0.05224783155686308,0.23946913195133712,0.05264343975067549 +grefcoco,instancevg,MiniCPM,four_difference,0.75516761543327,0.00038324296992454167,0.6789513087372032,0.00107169487051,0.6784754558602502,0.0009978758068675264,0.7565394616628013,0.0005053251250895244,0.6802540906899583,0.0012240819123290802,0.08195937873357229,0.04968689558802639,0.1512350055298755,0.050698419588036114 +grefcoco,instancevg,MiniCPM,four_ratio,0.7599339377327993,0.0029760483711252735,0.6762561324901454,0.001277555466059089,0.6778685874712872,0.000527070018222977,0.7671375360179915,0.005231090837092384,0.671258826532059,0.0030551813437642737,0.21197554290533419,0.008374769277990949,0.27478093185491875,0.01505522661579686 +grefcoco,instancevg,MiniCPM,gain,0.7532785157073583,0.001429301208093621,0.677202733743584,0.0008135309205015226,0.6768822845475427,0.0007797330613034966,0.7548808770820156,0.0013485514263493986,0.6782916938434053,0.0009934730945914698,0.05996204933586339,0.03937905952946572,0.11832232084621275,0.0368431354195028 +grefcoco,instancevg,MiniCPM,route,0.7677700470869351,0.00470575293954817,0.670389359952358,0.0040411039649331385,0.6744820349941866,0.0020093242842834365,0.7760910815939279,0.008715872080531073,0.6654396959986388,0.012248907138640627,0.18396233045189403,0.03910916588884175,0.25921787709497207,0.0625546079947896 +grefcoco,instancevg,Qwen3.8,error,0.7835561177876167,0.0028567720730928994,0.7282692907580184,0.0009976928491790967,0.7136675835861952,0.0003243070505697559,0.8261297350481411,0.004737712191726519,0.7569066727164449,0.0011860431161500352,0.4899571298053271,0.10367254926841935,0.6187108300484928,0.11032541276487592 +grefcoco,instancevg,Qwen3.8,four_difference,0.7676983624991215,0.0017218313848783413,0.7132609250489181,0.004416137043455413,0.7061696395655502,0.0023474909898597036,0.8083069787054606,0.0006490828527103093,0.7475030485211128,0.0016083702167531654,0.9679527725068523,0.008378985030784162,0.9685052321129796,0.00781786913048959 +grefcoco,instancevg,Qwen3.8,four_ratio,0.782085880947361,0.002500229406303351,0.7288443978107365,0.0009825121903040395,0.7136772254204123,0.0005213886555517868,0.8248366013071895,0.008700075636824258,0.7599466863284462,0.0027812863554784835,0.4839412467495959,0.02791311081846456,0.5978107364659841,0.029397396446080807 +grefcoco,instancevg,Qwen3.8,gain,0.7688368824232201,0.002960871357556939,0.7126971613305731,0.0060107898302351,0.7036780761704904,0.004089673699329882,0.8065359477124183,0.0009195545566826218,0.7466806567790603,0.00204016498940818,0.96996275212594,0.02915560864754022,0.9692652355159799,0.030481559938283696 +grefcoco,instancevg,Qwen3.8,route,0.7877250685220325,0.0021416863123372054,0.7251901426424298,0.00426943454547805,0.7128888636814792,0.0014029711714627744,0.8295593506219692,0.00411159224312457,0.7575192127725945,0.002329495328277004,0.3879682338885375,0.014823507124941973,0.4996341774664663,0.02131782709435361 +nlvr2,nlvr2,MiniCPM,error,0.8357146949298195,0.0004079537160868926,0.8414091985362704,0.0012511877563891084,0.8410059553705963,0.0009410935450967526,0.8997708393010599,0.0002935247999415425,0.907885484681065,0.0005925471140722553,0.8899169292466343,0.046325469934854695,0.8867905575087895,0.04527361464443895 +nlvr2,nlvr2,MiniCPM,four_difference,0.8364508736751646,0.0002926499103652411,0.8425256511444358,0.0011731132163255802,0.8416545885054173,0.0009248230676831484,0.8987109710684618,0.0005104133394788471,0.9070962186984287,0.0007583283398502093,0.9899742194213692,0.0,0.9904427064648059,0.0008793578933689391 +nlvr2,nlvr2,MiniCPM,four_ratio,0.8360355199083358,0.0003105053478756761,0.8420061706249552,0.0005010954216740357,0.8412829159790485,0.00046070914567228506,0.8996562589515898,0.0002791977755602646,0.9078711343904715,0.0002870058118676702,0.9419650529934117,0.016413035153506183,0.9379923943459856,0.016810019462330374 +nlvr2,nlvr2,MiniCPM,gain,0.835030077341736,0.0004510350706267802,0.8411221927244027,0.0015812968669033293,0.840622802611753,0.0010549977028042586,0.8985677456316242,0.0004793239911395297,0.9067087608524073,0.0002932168578848269,0.9899742194213692,0.0,0.9903422544306523,0.0012076858760089696 +nlvr2,nlvr2,MiniCPM,route,0.836459467201375,0.0002674905191619506,0.8416502834182392,0.0008439630263159055,0.8411422831312333,0.0007332352561280011,0.8997135491263248,0.00018674312246354723,0.9075554279974171,0.00029843774618762816,0.8879404182182755,0.037037361339264246,0.8849967711846165,0.03641866552327483 +nlvr2,nlvr2,Qwen3.8,error,0.8396132913205385,0.0004260084064604224,0.8421783741120759,0.0007576253552531098,0.8416675037669513,0.0006538325315131123,0.907877399026067,0.0012705961109262095,0.903336442562962,0.0012021313033542167,0.7639358350042967,0.05361984298731853,0.7649852909521417,0.04966345118649264 +nlvr2,nlvr2,Qwen3.8,four_difference,0.8407676883414494,0.0004412877746031485,0.8432948267202413,0.0007906508752431857,0.8426691540503697,0.0005200470866397635,0.9091950730449728,0.0006243052053194811,0.9061060486474851,0.0013050088305247716,0.9699226582641078,0.010025780578630794,0.9687737676687954,0.00929609232811199 +nlvr2,nlvr2,Qwen3.8,four_ratio,0.8400887997708393,0.00022938437951945014,0.8431814594245534,0.0003181996948668016,0.8420506565257947,0.0004083092698993979,0.910426811801776,0.0010888901637781795,0.9060342971945182,0.0007637402075473725,0.845918075050129,0.05896396588126329,0.8468249982062137,0.05730505605210457 +nlvr2,nlvr2,Qwen3.8,gain,0.8392208536236035,0.0007300705032751239,0.8421482385018297,0.0008448013756582421,0.8415039104541867,0.0006065990219145921,0.907877399026067,0.00027549962849714773,0.9054889861519696,0.001801947424734317,0.9719278143798338,0.01647403853853979,0.9705245031211882,0.017668451101310333 +nlvr2,nlvr2,Qwen3.8,route,0.8395645946720137,0.0003798310501410037,0.842644758556361,0.00050150621462047,0.8423462725120183,0.0003603109072443614,0.9082784302492124,0.0009532833739498352,0.9039248044772906,0.0008133589760174149,0.77393297049556,0.05317200699648349,0.7705962545741551,0.05152617166919074 +construction,rtdetr_x_fp32,MiniCPM,error,0.10689476721964583,0.0018360489113105247,0.11228041200099251,0.0007874020019033298,0.11341845104454558,0.0007858958535639363,0.21889058834744546,0.0006909688623161181,0.22865624886444885,0.0017162835197550568,0.7689015691868758,0.012229977317747697,0.77982689747004,0.014104853740125097 +construction,rtdetr_x_fp32,MiniCPM,four_difference,0.10406870592389231,0.0016309496616220727,0.11028216638595059,0.0030870855901712468,0.11295458120332329,0.0032538914151414734,0.21913219001083611,0.001307931612591827,0.22907685973193592,0.005805161470975003,0.7908701854493579,0.034156467798975576,0.8000665778961386,0.031921480323791725 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,0.10885158276665383,0.0030031928416340567,0.10446897690901666,0.008056790308998755,0.10521271910177055,0.00882421402109394,0.21899066225807357,0.0016940473764087934,0.23106183250713333,0.0017439773513269028,0.7728958630527817,0.005469412129152859,0.7847536617842876,0.0036663208794296713 +construction,rtdetr_x_fp32,MiniCPM,gain,0.10587160696739821,0.005457444202487199,0.11589092443741347,0.0036994266330316137,0.11735790506183426,0.002461993985841927,0.2187875682038221,0.001987443903799894,0.22867251554212525,0.001307889147807483,0.790870185449358,0.03828588490407007,0.799866844207723,0.033566090024000295 +construction,rtdetr_x_fp32,MiniCPM,route,0.10807869319402193,0.0026729953594518017,0.11030261812792752,0.002285774048247044,0.1135660437443529,0.001697345602479432,0.22210407360282552,0.0040419006526536735,0.2329352390844493,0.0030020764824269853,0.76490727532097,0.013397268909413866,0.7770972037283622,0.014344408196966299 +construction,rtdetr_x_fp32,Qwen3.8,error,0.12058049527949839,0.0007119376697686785,0.11710283387557004,0.002446399717461908,0.12041798453640595,0.0026454146265488363,0.26852261197687727,0.0006085000523326687,0.2701640351341744,0.0014756278001663904,0.7788873038516405,0.017295799790274107,0.7904127829560585,0.015301332966387055 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,0.11567459806423948,0.0038660720993751122,0.1165967027385137,0.005546977531866126,0.12052216301974265,0.005741396785076693,0.2713761319061188,0.0019541144340920026,0.26849490913617513,0.0013607443068895128,0.8088445078459342,0.01578883282537709,0.8147137150466046,0.013308503446227726 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,0.12609139028910016,0.0039303176874141265,0.10414412907604517,0.006671835621388073,0.10418376707234697,0.005495393076564938,0.2712168529841502,0.0024562657131376934,0.2689660211469896,0.001368100927643748,0.8048502139800287,0.0240488336808976,0.8093209054593874,0.017341273546869682 +construction,rtdetr_x_fp32,Qwen3.8,gain,0.11285010972278928,0.005293518165238395,0.11549777204401543,0.00567335294760609,0.11798156421014253,0.005515904530759242,0.27047768072162165,0.0018838084284716512,0.26944377853139817,0.0018206679282355342,0.8108416547788874,0.02164855687885188,0.8151131824234354,0.01884078000452992 +construction,rtdetr_x_fp32,Qwen3.8,route,0.11682085983824413,0.0033608523722885733,0.117581447669038,0.0022038507276840764,0.12024573167998012,0.0021788847122375696,0.2684545810328028,0.0005469921208345313,0.27204727695052544,0.0013249582738802692,0.7728958630527817,0.005469412129152859,0.7850199733688414,0.0060399032916333886 +nlvr2,vilt,MiniCPM,error,0.7925809223718132,0.00039756347412721625,0.7990241802396498,0.0005851153917737499,0.8007562603142713,0.0004232301891437592,0.8988828415926668,0.00028645087367516753,0.9069957666642751,0.00021163638091056884,0.9859639071899169,0.008967330760526202,0.9833393126210807,0.010326278064507578 +nlvr2,vilt,MiniCPM,four_difference,0.7937295903752506,0.00023477615660432653,0.8002367797947908,0.0004140500264405131,0.8015713568199756,0.00025509634547792867,0.8975938126611286,0.00045291859927933527,0.9061778001004519,0.0005525327777334696,0.9899742194213692,0.0,0.9900839491999711,0.001108554143190012 +nlvr2,vilt,MiniCPM,four_ratio,0.7940876539673446,0.0004357679519371168,0.8007261247040253,0.0008061172759119159,0.8018224869053597,0.0009210354794995672,0.8989401317674017,0.00021721212959187555,0.9070388175360552,0.00026167155813750523,0.9839587510741907,0.005491346179513221,0.9842577312190572,0.004544082452848853 +nlvr2,vilt,MiniCPM,gain,0.7932769979948439,0.0004186251910059456,0.7992394345985507,0.0007226087367763718,0.800533830810074,0.0003855464027399567,0.8970209109137783,0.00028645087367512584,0.9055463873143431,0.0004382128574184761,0.9899742194213692,0.0,0.9904857573365862,0.001875583529533495 +nlvr2,vilt,MiniCPM,route,0.7930850759094815,0.00011386197029134972,0.7995494008753677,0.0007321004366172784,0.8011494582765302,0.0007251228958352712,0.8988828415926668,0.0001432254368375907,0.9065078567840998,0.0005248166333618459,0.9859639071899169,0.008967330760526202,0.9838989739542227,0.00991944050277271 +nlvr2,vilt,Qwen3.8,error,0.7940618733887139,0.0003271690349065673,0.7993714572720099,0.0005320891534888945,0.801004520341537,0.0003532256329238195,0.9045545688914352,0.00038697382085668073,0.9003515821195378,0.0001380166754801895,0.9839587510741907,0.008967330760526197,0.9829662050656525,0.008406655728505972 +nlvr2,vilt,Qwen3.8,four_difference,0.7964737897450587,0.0005440116139392647,0.8014723398148813,0.0006701639482073403,0.8025299562316137,0.0006035999025271126,0.9066170151818964,0.0007089297433838111,0.9030637870416875,0.0007973778708076688,0.987969063305643,0.004483665380263076,0.9871995407907009,0.00543220578901596 +nlvr2,vilt,Qwen3.8,four_ratio,0.7966055571469493,0.00037853266360360615,0.8020377412642606,0.0013289717266528848,0.8031283633493578,0.0012157478618395412,0.9050988255514181,0.0002601819267342169,0.9004520341536916,0.00013034298718937417,0.9819535949584646,0.013072018572448725,0.9837267704671019,0.013156700629261588 +nlvr2,vilt,Qwen3.8,gain,0.7955485534230878,0.0002872375334038346,0.801462294611466,0.0007599796569634845,0.8024826002726556,0.0005869163592895192,0.9061873388713835,0.0005453862149716374,0.9028198321015999,0.0011272059632147719,0.9759381266112861,0.015204849071429653,0.9745712850685226,0.01539612977766457 +nlvr2,vilt,Qwen3.8,route,0.7942366084216557,0.00017264458556634163,0.799490564683935,0.0007399450122506702,0.8008882829877304,0.00024996912841740015,0.9049269550272129,0.00035662846170121177,0.9003802827007246,0.00017280038141341425,0.9899742194213692,0.0,0.9886919710124129,0.0010888741028969617 +construction,yolo26x,MiniCPM,error,0.2263763416931531,0.0036466564252099136,0.2134960675350596,0.0037893501865690687,0.21196187733187508,0.0035581956334776953,0.2871498749765999,0.00021775480476007544,0.25992143490750746,0.006392136679927825,0.4733238231098431,0.019465780431858117,0.46631158455392807,0.014971517038550188 +construction,yolo26x,MiniCPM,four_difference,0.2305961827838297,0.0021773102343294495,0.21690726805479663,0.002422512373997175,0.21618459881785138,0.0026516339340236957,0.2840018248312731,0.0024167487582089655,0.25938275238549513,0.004678859568347879,0.46733238231098434,0.028594792423091146,0.4645139813581891,0.028448517663268665 +construction,yolo26x,MiniCPM,four_ratio,0.22826564625561377,0.0015297086341046872,0.21520805285746542,0.002190142314047735,0.21446524128543043,0.00247894706463491,0.28867145328814314,0.0012529026117471522,0.2614745132215918,0.004496654069936857,0.4473609129814551,0.028594792423091146,0.44081225033288945,0.022793086409624682 +construction,yolo26x,MiniCPM,gain,0.23123445568440806,0.007110678273988894,0.21497730456389003,0.002641220315936139,0.21553188436650422,0.0032684260180297496,0.28890182177370444,0.0023112254090902297,0.2590138696350914,0.004352712921232588,0.42339514978601994,0.05586659350350672,0.42303595206391476,0.0532880056226722 +construction,yolo26x,MiniCPM,route,0.23452541392857143,0.0018493760447112907,0.21490095615841512,0.002705808474604684,0.21537259075758963,0.0024415913341775154,0.30299452217106587,0.0056360005318521565,0.26085137014586846,0.003726770003063279,0.33951497860199714,0.0440958103169008,0.3418109187749668,0.045606101839235295 +construction,yolo26x,Qwen3.8,error,0.2382200546072438,0.003931288140586608,0.21626950044157126,0.0035285101392354924,0.21321435332803368,0.0035430039227462365,0.33007295590902413,0.0015909500660411025,0.2981042528095301,0.00144280990411268,0.4853067047075606,0.02603961017769959,0.4749667110519308,0.02170863143103973 +construction,yolo26x,Qwen3.8,four_difference,0.23894305491785256,0.003208020446717338,0.2151927681972142,0.0035320630378739256,0.21424573214583562,0.0024759945835623456,0.32697453663947174,0.0035407217767539855,0.2987049451139474,0.002939193920775561,0.43138373751783166,0.020464666294355817,0.4330226364846871,0.015864550132763944 +construction,yolo26x,Qwen3.8,four_ratio,0.2310946681333383,0.0015058698881701732,0.21269340181123103,0.0007505886085794798,0.2118186381106563,0.000722417789109846,0.32736527974362695,0.0006793007886723935,0.29847751096358843,0.0015818331608369054,0.475320970042796,0.032047830456824106,0.4725033288948069,0.030788767929039392 +construction,yolo26x,Qwen3.8,gain,0.23752138533297384,0.001377134849463997,0.21430288420141394,0.0004297028231840248,0.212686687585937,0.0005346193370841685,0.3244253859282494,0.0021590794479529832,0.2969344334969651,0.0016956622846636454,0.4793152639087019,0.03235747852272113,0.4772969374167776,0.03403084728891011 +construction,yolo26x,Qwen3.8,route,0.243241607065442,0.003475917853283145,0.21502118034384718,0.001304817857724834,0.21334088837887535,0.0007021415234397854,0.328252166542654,0.0007458187903913553,0.30035812266639594,0.002261991506365177,0.46134094151212557,0.02770928915223432,0.4561251664447403,0.027400666801026683 diff --git a/releases/20260913_lr/paper_data/paired_confidence_differences.csv b/releases/20260913_lr/paper_data/paired_confidence_differences.csv new file mode 100644 index 0000000000000000000000000000000000000000..6d8d088eaf10ae270bb5030c01b26510e70837e0 --- /dev/null +++ b/releases/20260913_lr/paper_data/paired_confidence_differences.csv @@ -0,0 +1,33 @@ +task,expert,endpoint,comparison,measure,mean_difference,seed_std,descriptive_t95_low,descriptive_t95_high,scope +cub,cub,MiniCPM,selected_method_minus_fixed_confidence,performance,0.014670348636520525,0.0005852890907080939,0.01394361624409864,0.015397081028942411,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +cub,cub,MiniCPM,selected_method_minus_fixed_confidence,call_fraction,-0.09972385226095962,0.004630173451436489,-0.10547297194063761,-0.09397473258128163,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +cub,cub,Qwen3.8,selected_method_minus_fixed_confidence,performance,-0.001553331032102201,0.00012204121180300483,-0.001704865211084357,-0.0014017968531200447,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +cub,cub,Qwen3.8,selected_method_minus_fixed_confidence,call_fraction,-0.1054193993786676,0.0018118085567153175,-0.10766905683501826,-0.10316974192231693,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +cub,glsim,MiniCPM,selected_method_minus_fixed_confidence,performance,0.02454263030721433,0.00014440110917050722,0.024363332648669394,0.024721927965759265,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +cub,glsim,MiniCPM,selected_method_minus_fixed_confidence,call_fraction,-0.07369692785640318,0.0037237571643754627,-0.07832058306513213,-0.06907327264767422,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +cub,glsim,Qwen3.8,selected_method_minus_fixed_confidence,performance,0.013600276147739043,0.00037417127008418876,0.013135681152546427,0.01406487114293166,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +cub,glsim,Qwen3.8,selected_method_minus_fixed_confidence,call_fraction,-0.07877114256127028,0.002613506210765189,-0.08201623913202817,-0.0755260459905124,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +grefcoco,groundingdino,MiniCPM,selected_method_minus_fixed_confidence,performance,0.002297025210560644,0.0058877533082525415,-0.00501358610208084,0.009607636523202127,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +grefcoco,groundingdino,MiniCPM,selected_method_minus_fixed_confidence,call_fraction,-0.4249666789552789,0.03987669902841215,-0.47448014050253795,-0.37545321740801985,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +grefcoco,groundingdino,Qwen3.8,selected_method_minus_fixed_confidence,performance,0.012403936137027439,0.0012934323753779574,0.010797927722523864,0.014009944551531013,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +grefcoco,groundingdino,Qwen3.8,selected_method_minus_fixed_confidence,call_fraction,-0.09565833876868106,0.03061690111015076,-0.13367424261101185,-0.05764243492635025,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +grefcoco,instancevg,MiniCPM,selected_method_minus_fixed_confidence,performance,0.06147520063522671,0.0012240819123290802,0.05995530219394356,0.06299509907650987,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +grefcoco,instancevg,MiniCPM,selected_method_minus_fixed_confidence,call_fraction,-0.3697076255565323,0.05069841958803612,-0.4326580279203407,-0.3067572231927239,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +grefcoco,instancevg,Qwen3.8,selected_method_minus_fixed_confidence,performance,0.023174432124322974,0.0016083702167531656,0.021177376730539654,0.025171487518106293,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +grefcoco,instancevg,Qwen3.8,selected_method_minus_fixed_confidence,call_fraction,0.2846439610923631,0.00781786913048959,0.2749367944510572,0.29435112773366906,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +nlvr2,nlvr2,MiniCPM,selected_method_minus_fixed_confidence,performance,-0.0006314127861089425,0.0007583283398502093,-0.0015730017844514978,0.00031017621223361264,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +nlvr2,nlvr2,MiniCPM,selected_method_minus_fixed_confidence,call_fraction,0.13243883188634573,0.0008793578933689391,0.131346964848691,0.13353069892400046,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +nlvr2,nlvr2,Qwen3.8,selected_method_minus_fixed_confidence,performance,0.0036880246824998243,0.0013050088305247716,0.0020676422004146534,0.005308407164584995,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +nlvr2,nlvr2,Qwen3.8,selected_method_minus_fixed_confidence,call_fraction,0.11076989309033511,0.00929609232811199,0.09922726992336253,0.12231251625730769,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +nlvr2,vilt,MiniCPM,selected_method_minus_fixed_confidence,performance,-0.0002583052306809108,0.0005525327777334696,-0.0009443652885712077,0.0004277548272093861,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +nlvr2,vilt,MiniCPM,selected_method_minus_fixed_confidence,call_fraction,0.03076702303221639,0.0011085541431900122,0.02939057126265579,0.03214347480177699,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +nlvr2,vilt,Qwen3.8,selected_method_minus_fixed_confidence,performance,0.003444069742412359,0.0007973778708076688,0.0024539943473366375,0.00443414513748808,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +nlvr2,vilt,Qwen3.8,selected_method_minus_fixed_confidence,call_fraction,0.027882614622946122,0.00543220578901596,0.02113764026437144,0.0346275889815208,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +construction,yolo26x,MiniCPM,selected_method_minus_fixed_confidence,performance,0.0070780446550015915,0.004678859568347879,0.0012684731767466989,0.012887616133256483,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +construction,yolo26x,MiniCPM,selected_method_minus_fixed_confidence,call_fraction,-0.4609187749667111,0.028448517663268665,-0.4962422751489392,-0.42559527478448295,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +construction,yolo26x,Qwen3.8,selected_method_minus_fixed_confidence,performance,0.038955944637736904,0.002939193920775561,0.03530645336283051,0.0426054359126433,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +construction,yolo26x,Qwen3.8,selected_method_minus_fixed_confidence,call_fraction,-0.489081225033289,0.015864550132763944,-0.5087796657794375,-0.4693827842871406,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +construction,rtdetr_x_fp32,MiniCPM,selected_method_minus_fixed_confidence,performance,0.01298173100779495,0.005805161470975002,0.005773671006039186,0.020189791009550713,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +construction,rtdetr_x_fp32,MiniCPM,selected_method_minus_fixed_confidence,call_fraction,-0.17896138482023966,0.03192148032379172,-0.21859713770483802,-0.1393256319356413,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +construction,rtdetr_x_fp32,Qwen3.8,selected_method_minus_fixed_confidence,performance,0.00031059630104366407,0.0013607443068895128,-0.0013789909154614125,0.0020001835175487404,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty +construction,rtdetr_x_fp32,Qwen3.8,selected_method_minus_fixed_confidence,call_fraction,-0.17130492676431425,0.013308503446227724,-0.18782961636228873,-0.15478023716633976,router-seed variation only; approximate t interval; no multiplicity correction; not test-sample uncertainty diff --git a/releases/20260913_lr/paper_data/per_seed.csv b/releases/20260913_lr/paper_data/per_seed.csv new file mode 100644 index 0000000000000000000000000000000000000000..e86eb7b419d9453cb37279d9021e8839530d0df0 --- /dev/null +++ b/releases/20260913_lr/paper_data/per_seed.csv @@ -0,0 +1,561 @@ +task,expert,endpoint,method,seed,ranking,same_checkpoint_control,validation_score25,test_fixed_score25,test_ranked_score25,validation_metric,test_metric,validation_call,test_call,threshold,operator,nominal_budget +cub,cub,MiniCPM,error,2026,probability,False,0.905,0.6580082844321712,0.7830169140490163,0.9825,0.8065239903348291,0.01,0.05730065585088022,0.16549015045166016,ge,0.01 +cub,cub,MiniCPM,error,2027,probability,False,0.9045833333333334,0.6511218501898516,0.7711425612702796,0.98,0.7799447704521919,0.01,0.09458060062133242,0.10670345276594162,ge,0.01 +cub,cub,MiniCPM,error,2028,probability,False,0.9044166666666666,0.6622022782188471,0.7806178805661029,0.9825,0.8082499137038316,0.01,0.039868829823955815,0.1944640427827835,ge,0.01 +cub,cub,MiniCPM,error,2029,probability,False,0.90425,0.6504832585433207,0.774197445633414,0.9816666666666667,0.782188470831895,0.01,0.1032102174663445,0.08395212143659592,ge,0.01 +cub,cub,MiniCPM,error,2030,probability,False,0.9036666666666666,0.6527269589230238,0.7732827062478427,0.9825,0.8015188125647221,0.01,0.05246807041767346,0.19942712783813477,ge,0.01 +cub,cub,MiniCPM,four_difference,2026,log_ratio,True,0.9165833333333333,0.6840869865377978,0.7851052813255092,0.9825,0.8094580600621333,0.01,0.035208836727649295,-1.1720302104949951,ge,0.01 +cub,cub,MiniCPM,four_difference,2026,probability_difference,False,0.9745,0.80711080428029,0.8094062823610632,0.9866666666666667,0.8139454608215395,0.01,0.013634794615119088,-0.004982936196029186,ge,0.01 +cub,cub,MiniCPM,four_difference,2027,log_ratio,True,0.91,0.6659302726958923,0.7853296513634795,0.9816666666666667,0.8073869520193303,0.01,0.04263030721435968,-2.2670059204101562,ge,0.01 +cub,cub,MiniCPM,four_difference,2027,probability_difference,False,0.9744999999999999,0.80712806351398,0.8096651708664135,0.9858333333333333,0.81429064549534,0.01,0.006731101139109423,-0.0061280992813408375,ge,0.01 +cub,cub,MiniCPM,four_difference,2028,log_ratio,True,0.9084166666666667,0.687141870900932,0.7804798066965827,0.9808333333333333,0.8066965826717294,0.01,0.031239212978943734,-1.4453094005584717,ge,0.01 +cub,cub,MiniCPM,four_difference,2028,probability_difference,False,0.9740833333333334,0.8026579219882638,0.8066620642043494,0.9875,0.8130824991370383,0.01,0.019330341732827064,-0.000379358563804999,ge,0.01 +cub,cub,MiniCPM,four_difference,2029,log_ratio,True,0.9175,0.6836209872281671,0.7869865377977218,0.985,0.813600276147739,0.01,0.029340697273041078,-1.1837186813354492,ge,0.01 +cub,cub,MiniCPM,four_difference,2029,probability_difference,False,0.9736666666666667,0.8081463583016915,0.8094753192958233,0.9858333333333333,0.8137728684846393,0.01,0.013462202278218847,-0.003421929432079196,ge,0.01 +cub,cub,MiniCPM,four_difference,2030,log_ratio,True,0.9103333333333333,0.6719019675526408,0.7847083189506386,0.9833333333333333,0.8098032447359337,0.01,0.03348291335864688,-1.2544234991073608,ge,0.01 +cub,cub,MiniCPM,four_difference,2030,probability_difference,False,0.9749166666666667,0.8071971004487402,0.8092164307904729,0.9866666666666667,0.8146358301691405,0.01,0.01605108733172247,-0.004842729773372412,ge,0.01 +cub,cub,MiniCPM,four_ratio,2026,log_ratio,False,0.9430833333333333,0.736434242319641,0.8000862961684501,0.9858333333333333,0.8139454608215395,0.01,0.04953400069036935,-2.668729066848755,ge,0.01 +cub,cub,MiniCPM,four_ratio,2026,probability_difference,True,0.97125,0.807852951328961,0.8067138419054194,0.985,0.81429064549534,0.01,0.008802209181912324,-0.019993318244814873,ge,0.01 +cub,cub,MiniCPM,four_ratio,2027,log_ratio,False,0.95025,0.7628926475664481,0.790093199861926,0.9841666666666666,0.8084225060407317,0.01,0.041767345529858474,-3.035402297973633,ge,0.01 +cub,cub,MiniCPM,four_ratio,2027,probability_difference,True,0.9720833333333334,0.8029858474283742,0.8040386606834656,0.9866666666666667,0.8123921297894373,0.01,0.023990334829133587,-2.745852454915862e-14,ge,0.01 +cub,cub,MiniCPM,four_ratio,2028,log_ratio,False,0.9560833333333333,0.7569036934760097,0.7904729030031066,0.9833333333333333,0.8066965826717294,0.01,0.05350362443907491,-3.7635207176208496,ge,0.01 +cub,cub,MiniCPM,four_ratio,2028,probability_difference,True,0.9708333333333333,0.8005695547117708,0.8029340697273042,0.9858333333333333,0.8130824991370383,0.01,0.027269589230238176,-3.485907396072463e-11,ge,0.01 +cub,cub,MiniCPM,four_ratio,2029,log_ratio,False,0.9623333333333334,0.7808767690714532,0.7983258543320677,0.9858333333333333,0.8132550914739386,0.01,0.031066620642043494,-1.9326815605163574,ge,0.01 +cub,cub,MiniCPM,four_ratio,2029,probability_difference,True,0.973,0.8036244390749051,0.8048671039005868,0.9858333333333333,0.8139454608215395,0.01,0.022782188470831896,-2.99900972111633e-11,ge,0.01 +cub,cub,MiniCPM,four_ratio,2030,log_ratio,False,0.9343333333333333,0.71425612702796,0.7994994822229893,0.9858333333333333,0.8149810148429409,0.01,0.04659993096306524,-1.5429136753082275,ge,0.01 +cub,cub,MiniCPM,four_ratio,2030,probability_difference,True,0.9726666666666667,0.809250949257853,0.8090956161546428,0.9858333333333333,0.8149810148429409,0.01,0.015188125647221263,-0.015334374271333218,ge,0.01 +cub,cub,MiniCPM,gain,2026,regression,False,0.97375,0.8086813945460822,0.8095098377632034,0.9858333333333333,0.8146358301691405,0.01,0.013289609941318605,0.06945275515317917,ge,0.01 +cub,cub,MiniCPM,gain,2027,regression,False,0.9740833333333333,0.8012254055919917,0.8041076976182258,0.9858333333333333,0.8117017604418364,0.01,0.021919226786330688,0.20536428689956665,ge,0.01 +cub,cub,MiniCPM,gain,2028,regression,False,0.9730833333333333,0.8073179150845703,0.8087504314808422,0.9858333333333333,0.8149810148429409,0.01,0.026924404556437694,0.02451162599027157,ge,0.01 +cub,cub,MiniCPM,gain,2029,regression,False,0.974,0.8055574732481878,0.8084225060407317,0.9858333333333333,0.813600276147739,0.01,0.037625129444252677,0.06620025634765625,ge,0.01 +cub,cub,MiniCPM,gain,2030,regression,False,0.97375,0.8062133241284087,0.808526061442872,0.9858333333333333,0.8149810148429409,0.01,0.02347255781843286,0.18179216980934143,ge,0.01 +cub,cub,MiniCPM,route,2026,probability,False,0.9069166666666666,0.688488091128754,0.779306178805661,0.9808333333333333,0.8084225060407317,0.01,0.03348291335864688,0.09526840597391129,ge,0.01 +cub,cub,MiniCPM,route,2027,probability,False,0.9094166666666667,0.6993614083534692,0.7795823265447014,0.9816666666666667,0.809285467725233,0.01,0.02537107352433552,0.06959006190299988,ge,0.01 +cub,cub,MiniCPM,route,2028,probability,False,0.9061666666666667,0.6632896099413186,0.7816879530548844,0.9825,0.8104936140835347,0.01,0.027096996893337937,0.005073598586022854,ge,0.01 +cub,cub,MiniCPM,route,2029,probability,False,0.9093333333333334,0.6923886779426994,0.779271660338281,0.98,0.803589920607525,0.01,0.03676216775975147,0.08050795644521713,ge,0.01 +cub,cub,MiniCPM,route,2030,probability,False,0.9065,0.6734380393510528,0.7793579565067311,0.9825,0.8087676907145323,0.01,0.026406627545736968,0.10346683859825134,ge,0.01 +cub,cub,Qwen3.8,error,2026,probability,False,0.9205833333333333,0.7482740766309977,0.794356230583362,0.9808333333333333,0.808595098377632,0.01,0.03158439765274422,0.5351537466049194,ge,0.01 +cub,cub,Qwen3.8,error,2027,probability,False,0.9194166666666667,0.749206075250259,0.7945288229202624,0.98,0.8073869520193303,0.01,0.04297549188816017,0.4191480576992035,ge,0.01 +cub,cub,Qwen3.8,error,2028,probability,False,0.9189166666666666,0.7391957197100449,0.8024853296513634,0.9808333333333333,0.8146358301691405,0.01,0.039868829823955815,0.1944640427827835,ge,0.01 +cub,cub,Qwen3.8,error,2029,probability,False,0.9214166666666667,0.7472385226095962,0.7931480842250604,0.98,0.803589920607525,0.01,0.06748360372799447,0.10496433079242706,ge,0.01 +cub,cub,Qwen3.8,error,2030,probability,False,0.9175,0.7434242319641009,0.7960476354849845,0.9808333333333333,0.8066965826717294,0.01,0.05298584742837418,0.3316056728363037,ge,0.01 +cub,cub,Qwen3.8,four_difference,2026,log_ratio,True,0.9197500000000001,0.7397652744218157,0.797186744908526,0.9825,0.8104936140835347,0.01,0.032274767000345185,-2.029417037963867,ge,0.01 +cub,cub,Qwen3.8,four_difference,2026,probability_difference,False,0.9836666666666667,0.8131515360717985,0.8127545736969278,0.9858333333333333,0.8144632378322403,0.01,0.009492578529513289,-0.001442005392163992,ge,0.01 +cub,cub,Qwen3.8,four_difference,2027,log_ratio,True,0.9186666666666666,0.7403693476009665,0.8032447359337246,0.9791666666666666,0.8144632378322403,0.01,0.030031066620642043,-2.2460412979125977,ge,0.01 +cub,cub,Qwen3.8,four_difference,2027,probability_difference,False,0.9831666666666666,0.812806351397998,0.811477390403866,0.9858333333333333,0.8141180531584398,0.01,0.006731101139109423,-0.008533324114978313,ge,0.01 +cub,cub,Qwen3.8,four_difference,2028,log_ratio,True,0.9118333333333334,0.7356230583362099,0.7940800828443217,0.98,0.8120469451156369,0.01,0.02847773558853987,-2.7140793800354004,ge,0.01 +cub,cub,Qwen3.8,four_difference,2028,probability_difference,False,0.9839166666666667,0.8128753883327581,0.811460131170176,0.9858333333333333,0.81429064549534,0.01,0.008111839834311357,-0.0035147510934621096,ge,0.01 +cub,cub,Qwen3.8,four_difference,2029,log_ratio,True,0.9155,0.7356230583362099,0.7988091128753882,0.98,0.8127373144632378,0.01,0.038833275802554364,-2.7343597412109375,ge,0.01 +cub,cub,Qwen3.8,four_difference,2029,probability_difference,False,0.9834166666666667,0.8129789437348982,0.8115464273386261,0.9858333333333333,0.81429064549534,0.01,0.010355540214014497,-0.014309233985841274,ge,0.01 +cub,cub,Qwen3.8,four_difference,2030,log_ratio,True,0.9219166666666667,0.7439765274421816,0.8031584397652743,0.9816666666666667,0.8123921297894373,0.01,0.02847773558853987,-2.5535104274749756,ge,0.01 +cub,cub,Qwen3.8,four_difference,2030,probability_difference,False,0.98325,0.8130134622022783,0.8115119088712461,0.9858333333333333,0.81429064549534,0.01,0.0060407317915084574,-0.00484488345682621,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2026,log_ratio,False,0.9755833333333334,0.8073351743182603,0.808577839143942,0.9858333333333333,0.8132550914739386,0.01,0.02105626510182948,3.954883575439453,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2026,probability_difference,True,0.9784999999999999,0.8094580600621333,0.8093890231273732,0.9841666666666666,0.8120469451156369,0.01,0.030031066620642043,1.199191856926518e-09,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2027,log_ratio,False,0.9529166666666667,0.7864342423196411,0.8006040731791509,0.9825,0.8111839834311356,0.01,0.043148084225060404,-3.265713691711426,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2027,probability_difference,True,0.97925,0.8102692440455644,0.8096479116327235,0.985,0.8127373144632378,0.01,0.019675526406627546,-3.3625896798993374e-14,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2028,log_ratio,False,0.9682499999999999,0.8025025888850535,0.8046254746289265,0.9858333333333333,0.8129099068001381,0.01,0.016396272005522954,-1.5082664489746094,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2028,probability_difference,True,0.9801666666666666,0.810631687953055,0.8093545046599931,0.9858333333333333,0.8127373144632378,0.01,0.012426648256817397,-6.139429649344372e-14,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2029,log_ratio,False,0.9641666666666666,0.8019157749395928,0.8049016223679668,0.9841666666666666,0.8139454608215395,0.01,0.009665170866413532,2.535764694213867,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2029,probability_difference,True,0.978,0.8123748705557474,0.8117017604418364,0.9841666666666666,0.8139454608215395,0.01,0.01812219537452537,7.519151100365828e-12,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2030,log_ratio,False,0.9615,0.7994649637556093,0.8044183638246463,0.9833333333333333,0.8130824991370383,0.01,0.021401449775629963,-3.623467445373535,ge,0.01 +cub,cub,Qwen3.8,four_ratio,2030,probability_difference,True,0.9794166666666667,0.8099240593717639,0.8088367276492924,0.9858333333333333,0.8129099068001381,0.01,0.01156368657231619,-5.734394841703247e-15,ge,0.01 +cub,cub,Qwen3.8,gain,2026,regression,False,0.982,0.8122022782188472,0.8107697618225751,0.985,0.8141180531584398,0.01,0.00638591646530894,0.10205373167991638,ge,0.01 +cub,cub,Qwen3.8,gain,2027,regression,False,0.98125,0.8128926475664481,0.8124093890231274,0.9858333333333333,0.8146358301691405,0.01,0.01725923369002416,0.08088664710521698,ge,0.01 +cub,cub,Qwen3.8,gain,2028,regression,False,0.9813333333333334,0.8124439074905074,0.8104590956161546,0.9858333333333333,0.81429064549534,0.01,0.005350362443907491,0.16180923581123352,ge,0.01 +cub,cub,Qwen3.8,gain,2029,regression,False,0.9825,0.812789092164308,0.8119261304798067,0.9858333333333333,0.8141180531584398,0.01,0.00638591646530894,0.052189622074365616,ge,0.01 +cub,cub,Qwen3.8,gain,2030,regression,False,0.9816666666666667,0.8122540559199173,0.8119261304798067,0.985,0.8139454608215395,0.01,0.020193303417328272,0.061363235116004944,ge,0.01 +cub,cub,Qwen3.8,route,2026,probability,False,0.9196666666666666,0.7508802209181913,0.8007766655160511,0.98,0.8123921297894373,0.01,0.024680704176734552,0.06744378060102463,ge,0.01 +cub,cub,Qwen3.8,route,2027,probability,False,0.9233333333333333,0.7612357611322058,0.8025198481187436,0.9791666666666666,0.8125647221263376,0.01,0.02347255781843286,0.09797634929418564,ge,0.01 +cub,cub,Qwen3.8,route,2028,probability,False,0.92025,0.7445288229202625,0.7988263721090784,0.9791666666666666,0.813600276147739,0.01,0.021574042112530202,0.0361652746796608,ge,0.01 +cub,cub,Qwen3.8,route,2029,probability,False,0.9214166666666667,0.7548153261995167,0.8030548843631343,0.98,0.8129099068001381,0.01,0.025025888850535038,0.05318247154355049,ge,0.01 +cub,cub,Qwen3.8,route,2030,probability,False,0.9195833333333333,0.7459095616154643,0.7959095616154642,0.9783333333333334,0.8120469451156369,0.01,0.020193303417328272,0.11371302604675293,ge,0.01 +cub,glsim,MiniCPM,error,2026,probability,False,0.9094166666666667,0.7611322057300656,0.8433724542630308,0.9875,0.8828098032447359,0.01,0.04263030721435968,0.0131531385704875,ge,0.01 +cub,glsim,MiniCPM,error,2027,probability,False,0.9126666666666666,0.7677252329996549,0.8445115636865723,0.9866666666666667,0.8878149810148429,0.01,0.03779772178115292,0.018917348235845566,ge,0.01 +cub,glsim,MiniCPM,error,2028,probability,False,0.9130833333333334,0.7650845702450811,0.845685191577494,0.9866666666666667,0.8872972040041422,0.01,0.039868829823955815,0.007665780372917652,ge,0.01 +cub,glsim,MiniCPM,error,2029,probability,False,0.9095833333333333,0.7584742837418019,0.8475319295823266,0.9866666666666667,0.8891957197100449,0.01,0.03589920607525026,0.13308997452259064,ge,0.01 +cub,glsim,MiniCPM,error,2030,probability,False,0.9092499999999999,0.7565930272695893,0.8448740075940628,0.9866666666666667,0.8867794269934415,0.01,0.04142216085605799,0.03516277298331261,ge,0.01 +cub,glsim,MiniCPM,four_difference,2026,log_ratio,True,0.9288333333333334,0.7791681049361409,0.8593545046599931,0.9866666666666667,0.8904038660683465,0.01,0.04401104590956161,-3.2388687133789062,ge,0.01 +cub,glsim,MiniCPM,four_difference,2026,probability_difference,False,0.9784166666666667,0.8924749741111495,0.8925785295132896,0.9891666666666666,0.9016223679668622,0.01,0.014152571625819814,-0.0050849150866270065,ge,0.01 +cub,glsim,MiniCPM,four_difference,2027,log_ratio,True,0.9278333333333334,0.7789437348981705,0.8601656886434242,0.9875,0.8931653434587504,0.01,0.03727994477045219,-1.9760479927062988,ge,0.01 +cub,glsim,MiniCPM,four_difference,2027,probability_difference,False,0.9786666666666667,0.8919054193993787,0.89288919571971,0.9891666666666666,0.9016223679668622,0.01,0.008802209181912324,-0.005811857525259256,ge,0.01 +cub,glsim,MiniCPM,four_difference,2028,log_ratio,True,0.9224166666666667,0.7610804280289956,0.8570245081118398,0.9866666666666667,0.8929927511218502,0.01,0.03727994477045219,-2.430908203125,ge,0.01 +cub,glsim,MiniCPM,four_difference,2028,probability_difference,False,0.9785,0.8917846047635485,0.8931135657576803,0.9891666666666666,0.901449775629962,0.01,0.01052813255091474,-0.0011190498480573297,ge,0.01 +cub,glsim,MiniCPM,four_difference,2029,log_ratio,True,0.9264166666666667,0.7758025543665862,0.8551259924059371,0.9866666666666667,0.8919571971004487,0.01,0.034691059716948566,-3.817823648452759,ge,0.01 +cub,glsim,MiniCPM,four_difference,2029,probability_difference,False,0.979,0.8903003106662065,0.8916983085950984,0.9891666666666666,0.901449775629962,0.01,0.014670348636520539,-0.004320982377976179,ge,0.01 +cub,glsim,MiniCPM,four_difference,2030,log_ratio,True,0.93825,0.7904038660683467,0.8627373144632379,0.9883333333333333,0.8955816361753538,0.01,0.03072143596824301,-1.7466963529586792,ge,0.01 +cub,glsim,MiniCPM,four_difference,2030,probability_difference,False,0.979,0.8912323092854677,0.8919399378667587,0.99,0.9012771832930618,0.01,0.018294787711425613,-0.002133682370185852,ge,0.01 +cub,glsim,MiniCPM,four_ratio,2026,log_ratio,False,0.9708333333333332,0.8690714532274767,0.879979288919572,0.99,0.8988608905764585,0.02,0.036589575422851225,-2.0773916244506836,ge,0.02 +cub,glsim,MiniCPM,four_ratio,2026,probability_difference,True,0.9737499999999999,0.8880566102865033,0.8898343113565758,0.9891666666666666,0.9000690369347601,0.01,0.02312737314463238,-7.376906377420944e-12,ge,0.01 +cub,glsim,MiniCPM,four_ratio,2027,log_ratio,False,0.968,0.868519157749396,0.8793924749741111,0.99,0.9000690369347601,0.01,0.024508111839834312,-1.8984746932983398,ge,0.01 +cub,glsim,MiniCPM,four_ratio,2027,probability_difference,True,0.9760833333333333,0.8887814981014843,0.8905419399378668,0.9891666666666666,0.9009319986192613,0.01,0.02122885743872972,-6.550278985884006e-09,ge,0.01 +cub,glsim,MiniCPM,four_ratio,2028,log_ratio,False,0.97075,0.873524335519503,0.8809630652399034,0.9866666666666667,0.8978253365550569,0.02,0.03779772178115292,-0.5955595970153809,ge,0.02 +cub,glsim,MiniCPM,four_ratio,2028,probability_difference,True,0.9741666666666666,0.8867621677597516,0.8893855712806351,0.9858333333333333,0.8983431135657577,0.01,0.02416292716603383,8.301383885855884e-11,ge,0.01 +cub,glsim,MiniCPM,four_ratio,2029,log_ratio,False,0.9696666666666667,0.8744218156713842,0.8809975837072834,0.99,0.9002416292716603,0.01,0.02329996548153262,-1.2633552551269531,ge,0.01 +cub,glsim,MiniCPM,four_ratio,2029,probability_difference,True,0.9755833333333332,0.8876078702105626,0.8901794960303762,0.99,0.9004142216085605,0.01,0.02002071108042803,-1.0354328952946225e-14,ge,0.01 +cub,glsim,MiniCPM,four_ratio,2030,log_ratio,False,0.9710833333333333,0.872851225405592,0.8826372109078356,0.9883333333333333,0.9005868139454608,0.01,0.018467380048325856,2.274503707885742,ge,0.01 +cub,glsim,MiniCPM,four_ratio,2030,probability_difference,True,0.9755,0.8858646876078702,0.8894373489817052,0.9891666666666666,0.8986882982395582,0.01,0.02657921988263721,1.9104025028582328e-08,ge,0.01 +cub,glsim,MiniCPM,gain,2026,regression,False,0.9782500000000001,0.8915257162581982,0.8927511218501899,0.9891666666666666,0.9011045909561616,0.01,0.017086641353123922,0.06638844311237335,ge,0.01 +cub,glsim,MiniCPM,gain,2027,regression,False,0.9786666666666667,0.8915429754918882,0.8931998619261305,0.9891666666666666,0.901449775629962,0.01,0.014670348636520539,0.08822585642337799,ge,0.01 +cub,glsim,MiniCPM,gain,2028,regression,False,0.9788333333333333,0.8914566793234381,0.8929582326544702,0.9891666666666666,0.9016223679668622,0.01,0.016396272005522954,0.025555070489645004,ge,0.01 +cub,glsim,MiniCPM,gain,2029,regression,False,0.9782500000000001,0.8927338626165,0.8934587504314808,0.9891666666666666,0.9016223679668622,0.01,0.013117017604418364,0.06589554250240326,ge,0.01 +cub,glsim,MiniCPM,gain,2030,regression,False,0.9786666666666667,0.8896444597859856,0.8923541594753193,0.9891666666666666,0.9017949603037625,0.01,0.018294787711425613,0.16952542960643768,ge,0.01 +cub,glsim,MiniCPM,route,2026,probability,False,0.9014166666666666,0.7645322747670005,0.8499482222989299,0.9841666666666666,0.8935105281325509,0.01,0.0340006903693476,0.006400675512850285,ge,0.01 +cub,glsim,MiniCPM,route,2027,probability,False,0.9013333333333333,0.7772868484639283,0.8505868139454609,0.9866666666666667,0.8883327580255437,0.01,0.05108733172247152,0.06703241169452667,ge,0.01 +cub,glsim,MiniCPM,route,2028,probability,False,0.9009166666666667,0.7466862271315153,0.8501898515705902,0.9833333333333333,0.890749050742147,0.01,0.04452882292026234,0.020855436101555824,ge,0.01 +cub,glsim,MiniCPM,route,2029,probability,False,0.9025,0.7535208836727649,0.8513462202278219,0.985,0.8938557128063513,0.01,0.03313772868484639,0.03475888818502426,ge,0.01 +cub,glsim,MiniCPM,route,2030,probability,False,0.90025,0.7312392129789438,0.8492578529513289,0.9841666666666666,0.885053503624439,0.01,0.0505695547117708,0.006442188285291195,ge,0.01 +cub,glsim,Qwen3.8,error,2026,probability,False,0.9371666666666667,0.8147911632723508,0.8659647911632724,0.985,0.8917846047635485,0.01,0.04263030721435968,0.0131531385704875,ge,0.01 +cub,glsim,Qwen3.8,error,2027,probability,False,0.9338333333333333,0.8157749395926822,0.8658439765274422,0.9858333333333333,0.8966171901967552,0.01,0.035208836727649295,0.01910049468278885,ge,0.01 +cub,glsim,Qwen3.8,error,2028,probability,False,0.9361666666666667,0.8180531584397653,0.8659302726958923,0.9841666666666666,0.8948912668277529,0.01,0.035208836727649295,0.22747483849525452,ge,0.01 +cub,glsim,Qwen3.8,error,2029,probability,False,0.9345,0.8130997583707283,0.8662236796686227,0.9841666666666666,0.8960994131860546,0.01,0.03676216775975147,0.023573948070406914,ge,0.01 +cub,glsim,Qwen3.8,error,2030,probability,False,0.937,0.813565757680359,0.867069382119434,0.985,0.8943734898170521,0.01,0.04142216085605799,0.02723637968301773,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2026,log_ratio,True,0.9530833333333334,0.8487918536416983,0.8794097342078012,0.9858333333333333,0.8957542285122541,0.01,0.03935105281325509,-3.026237964630127,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2026,probability_difference,False,0.9861666666666666,0.8969623748705557,0.8968243010010355,0.9883333333333333,0.9007594062823611,0.01,0.012426648256817397,-0.001153669785708189,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2027,log_ratio,True,0.9509166666666666,0.8343976527442182,0.8812392129789437,0.9875,0.8990334829133586,0.01,0.024680704176734552,-1.5695973634719849,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2027,probability_difference,False,0.9865833333333334,0.8975146703486365,0.8968760787021056,0.9891666666666666,0.9016223679668622,0.01,0.0075940628236106315,-0.005822875536978245,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2028,log_ratio,True,0.9498333333333333,0.8387124611667242,0.877873662409389,0.9841666666666666,0.896444597859855,0.01,0.04504659993096306,-2.472076892852783,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2028,probability_difference,False,0.98625,0.897117707973766,0.8967380048325855,0.9891666666666666,0.9016223679668622,0.01,0.005350362443907491,-0.01126081869006157,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2029,log_ratio,True,0.9350833333333334,0.8065585088022093,0.8781325509147394,0.985,0.897134967207456,0.01,0.04124956851915775,-2.668215751647949,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2029,probability_difference,False,0.9856666666666667,0.8977562996202969,0.8968588194684156,0.9891666666666666,0.9016223679668622,0.01,0.007248878149810148,-0.009776376187801361,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2030,log_ratio,True,0.9451666666666667,0.8248360372799448,0.8789955125992406,0.9866666666666667,0.8973075595443563,0.01,0.03141180531584398,-2.2285304069519043,ge,0.01 +cub,glsim,Qwen3.8,four_difference,2030,probability_difference,False,0.9864166666666666,0.897842595788747,0.8972212633759061,0.9891666666666666,0.901449775629962,0.01,0.00845702450811184,-0.00619181152433157,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2026,log_ratio,False,0.97775,0.8907835692095272,0.8929927511218502,0.9866666666666667,0.9007594062823611,0.01,0.013289609941318605,7.8850555419921875,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2026,probability_difference,True,0.98,0.8945115636865724,0.8949085260614429,0.9833333333333333,0.9000690369347601,0.01,0.021401449775629963,3.262804426640287e-08,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2027,log_ratio,False,0.9788333333333333,0.8919226786330687,0.8939247497411115,0.9891666666666666,0.9012771832930618,0.01,0.010700724887814981,5.494701862335205,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2027,probability_difference,True,0.9826666666666667,0.8942526751812221,0.8946323783224025,0.9866666666666667,0.8998964445978599,0.01,0.02036589575422851,2.1654463466802554e-07,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2028,log_ratio,False,0.9742500000000001,0.8885226095961339,0.8900586813945461,0.9875,0.9002416292716603,0.01,0.015360717984121504,4.342945575714111,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2028,probability_difference,True,0.9825833333333334,0.8935277873662409,0.8939420089748016,0.9858333333333333,0.8986882982395582,0.01,0.025716258198136003,1.3274492616233147e-08,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2029,log_ratio,False,0.97475,0.8875733517431827,0.8920434932688989,0.9866666666666667,0.9000690369347601,0.01,0.022609596133931653,1.6047245264053345,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2029,probability_difference,True,0.9834166666666667,0.8941146013117018,0.8947531929582326,0.9866666666666667,0.898515705902658,0.01,0.02865032792544011,2.6156001808885776e-07,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2030,log_ratio,False,0.9755833333333334,0.8808249913703833,0.8880220918191233,0.9875,0.9002416292716603,0.01,0.023990334829133587,2.1992859840393066,ge,0.01 +cub,glsim,Qwen3.8,four_ratio,2030,probability_difference,True,0.9823333333333333,0.8923886779426994,0.8940800828443217,0.9858333333333333,0.8981705212288574,0.01,0.02744218156713842,1.076633907359792e-05,ge,0.01 +cub,glsim,Qwen3.8,gain,2026,regression,False,0.98425,0.8925785295132896,0.893562305833621,0.9875,0.8990334829133586,0.01,0.019848118743527786,0.26899492740631104,ge,0.01 +cub,glsim,Qwen3.8,gain,2027,regression,False,0.9848333333333333,0.8949430445288229,0.8939074905074215,0.9891666666666666,0.9016223679668622,0.01,0.008974801518812565,0.13488280773162842,ge,0.01 +cub,glsim,Qwen3.8,gain,2028,regression,False,0.9855,0.8969623748705559,0.8970141525716258,0.9891666666666666,0.9016223679668622,0.01,0.012426648256817397,0.15440733730793,ge,0.01 +cub,glsim,Qwen3.8,gain,2029,regression,False,0.98575,0.8980151881256473,0.8969451156368657,0.9891666666666666,0.9016223679668622,0.01,0.005350362443907491,0.08522714674472809,ge,0.01 +cub,glsim,Qwen3.8,gain,2030,regression,False,0.9856666666666667,0.8955471177079739,0.895029340697273,0.9875,0.9004142216085605,0.01,0.010355540214014497,0.11318457126617432,ge,0.01 +cub,glsim,Qwen3.8,route,2026,probability,False,0.927,0.7939592682084916,0.8622367966862271,0.9816666666666667,0.8967897825336555,0.01,0.019848118743527786,0.02219533920288086,ge,0.01 +cub,glsim,Qwen3.8,route,2027,probability,False,0.9275833333333334,0.8065239903348291,0.8693648602002071,0.9825,0.8954090438384535,0.01,0.028995512599240592,0.022180303931236267,ge,0.01 +cub,glsim,Qwen3.8,route,2028,probability,False,0.92425,0.7911114946496376,0.8623921297894374,0.9833333333333333,0.891439420089748,0.01,0.03089402830514325,0.0007422103662975132,ge,0.01 +cub,glsim,Qwen3.8,route,2029,probability,False,0.9250833333333334,0.7919054193993786,0.8617362789092164,0.9833333333333333,0.8967897825336555,0.01,0.02036589575422851,0.012569688260555267,ge,0.01 +cub,glsim,Qwen3.8,route,2030,probability,False,0.9218333333333333,0.7856920952709701,0.8598722816706938,0.9833333333333333,0.8959268208491543,0.01,0.024680704176734552,0.0028624904807657003,ge,0.01 +grefcoco,groundingdino,MiniCPM,error,2026,probability,False,0.6196570384426172,0.3284746051101721,0.3309162578339903,0.672007871248858,0.45007514959022205,0.339939560053412,0.45194679976179,0.9987803101539612,ge,0.34 +grefcoco,groundingdino,MiniCPM,error,2027,probability,False,0.6219059666877503,0.3199415818279784,0.33172447040807646,0.6781221449153138,0.4620140090179508,0.34998945814885096,0.47097524317273065,0.9986439347267151,ge,0.35 +grefcoco,groundingdino,MiniCPM,error,2028,probability,False,0.6211328976034858,0.32172248532456116,0.3314182003800017,0.6761543327008223,0.438363156849956,0.3299599409656336,0.4045600204180019,0.997635006904602,ge,0.33 +grefcoco,groundingdino,MiniCPM,error,2029,probability,False,0.6207674467636517,0.3165272381816635,0.33000879108413916,0.6860636727809404,0.4450840824660409,0.3599690772366294,0.4424183988883532,0.9989563226699829,ge,0.36 +grefcoco,groundingdino,MiniCPM,error,2030,probability,False,0.6218778550846862,0.33092476533477017,0.3335195530726257,0.672851219340783,0.467742392876386,0.3599690772366294,0.4931514618722173,0.9941957592964172,ge,0.36 +grefcoco,groundingdino,MiniCPM,four_difference,2026,log_ratio,True,0.6254691123761332,0.3268269857924737,0.33397045061395797,0.686344788811582,0.4573916002608967,0.34998945814885096,0.4425034738961518,6.270540237426758,ge,0.35 +grefcoco,groundingdino,MiniCPM,four_difference,2026,probability_difference,False,0.6121863799283154,0.3787539347191107,0.3491052945013186,0.6554923044486611,0.5033037461361767,0.449996486049617,0.5800697615063948,0.2908308804035187,ge,0.45 +grefcoco,groundingdino,MiniCPM,four_difference,2027,log_ratio,True,0.6236910534823248,0.3298783427388481,0.33427388480843945,0.6797385620915033,0.453052774863171,0.3299599409656336,0.4178317216345745,5.889019966125488,ge,0.33 +grefcoco,groundingdino,MiniCPM,four_difference,2027,probability_difference,False,0.6151521540515847,0.3754161585798146,0.3483906644358109,0.6602712769695692,0.49249922014576186,0.39995783259540374,0.5114993052207697,0.3213382363319397,ge,0.4 +grefcoco,groundingdino,MiniCPM,four_difference,2028,log_ratio,True,0.6240354206198608,0.33228880129314015,0.33633837166435077,0.6694075479654227,0.43830644017809034,0.2999508046946377,0.38292261010123924,7.289079189300537,ge,0.3 +grefcoco,groundingdino,MiniCPM,four_difference,2028,probability_difference,False,0.615187293555415,0.37666108952726657,0.3502850012761251,0.6552814674256799,0.5009783625896832,0.46995572422517395,0.5808070782406488,0.24761390686035156,ge,0.47 +grefcoco,groundingdino,MiniCPM,four_difference,2029,log_ratio,True,0.6243235645512686,0.3337634347616482,0.3369395683861271,0.6740459624710099,0.4451407991379066,0.3099304237824162,0.390777869154638,5.797358512878418,ge,0.31 +grefcoco,groundingdino,MiniCPM,four_difference,2029,probability_difference,False,0.6152646004638415,0.36908941383319627,0.3493832061934606,0.6550003513950383,0.4933783285596801,0.4199873497786211,0.5310665570144344,0.31439468264579773,ge,0.42 +grefcoco,groundingdino,MiniCPM,four_difference,2030,log_ratio,True,0.6234591327570455,0.3296911777216913,0.3353770240762272,0.675240705601237,0.46825284292317726,0.3599690772366294,0.4682244845872444,5.167756080627441,ge,0.36 +grefcoco,groundingdino,MiniCPM,four_difference,2030,probability_difference,False,0.6110338042026846,0.38105095992967136,0.3495448487082778,0.6552111884180195,0.5055156963389388,0.4799353433129524,0.6086266057907722,0.16629476845264435,ge,0.48 +grefcoco,groundingdino,MiniCPM,four_ratio,2026,log_ratio,False,0.6254691123761332,0.3268269857924737,0.33397045061395797,0.686344788811582,0.4573916002608967,0.34998945814885096,0.4425034738961518,6.270540237426758,ge,0.35 +grefcoco,groundingdino,MiniCPM,four_ratio,2026,probability_difference,True,0.6121863799283154,0.3787539347191107,0.3491052945013186,0.6554923044486611,0.5033037461361767,0.449996486049617,0.5800697615063948,0.2908308804035187,ge,0.45 +grefcoco,groundingdino,MiniCPM,four_ratio,2027,log_ratio,False,0.6256237261929861,0.33940390777869156,0.3395825652950685,0.6751001475859161,0.4591781754246661,0.3199803218778551,0.42540339732864474,5.838075160980225,ge,0.32 +grefcoco,groundingdino,MiniCPM,four_ratio,2027,probability_difference,True,0.611961487103802,0.37634347616481867,0.348869920313076,0.6555625834563216,0.5022828460425942,0.449996486049617,0.5760712361398633,0.1986093968153,ge,0.45 +grefcoco,groundingdino,MiniCPM,four_ratio,2028,log_ratio,False,0.6255464192845597,0.3335053739046593,0.33636956583387684,0.6811441422447115,0.4548960666988061,0.339939560053412,0.43184073958540115,5.855062961578369,ge,0.34 +grefcoco,groundingdino,MiniCPM,four_ratio,2028,probability_difference,True,0.6106261859582542,0.38387545018858293,0.3494427586989195,0.6584440227703985,0.5022261293707285,0.45997610513739545,0.5888892039815103,0.26440566778182983,ge,0.46 +grefcoco,groundingdino,MiniCPM,four_ratio,2029,log_ratio,False,0.6268184693232132,0.3295890877123331,0.33684315004395543,0.6826200014055801,0.4556050250971273,0.3299599409656336,0.418115304993903,6.247387886047363,ge,0.33 +grefcoco,groundingdino,MiniCPM,four_ratio,2029,probability_difference,True,0.6111181390118772,0.379942148994697,0.3487649944701245,0.6553517464333404,0.49859626237132404,0.43994658795417807,0.566457760258628,0.1935059130191803,ge,0.44 +grefcoco,groundingdino,MiniCPM,four_ratio,2030,log_ratio,False,0.6252301637500879,0.3501375379292743,0.3431103422851147,0.6756623796471993,0.4759095936250461,0.34998945814885096,0.47395286844567963,4.759512424468994,ge,0.35 +grefcoco,groundingdino,MiniCPM,four_ratio,2030,probability_difference,True,0.6018553658022349,0.379862745654085,0.34663244760797435,0.6539461662801321,0.5139664804469274,0.6099515074847143,0.7495391770410912,0.10030291974544525,ge,0.61 +grefcoco,groundingdino,MiniCPM,gain,2026,regression,False,0.610070981797737,0.37849020219493523,0.3483764852678445,0.652400028111603,0.4986529790431897,0.45997610513739545,0.5797578198111335,0.21678990125656128,ge,0.46 +grefcoco,groundingdino,MiniCPM,gain,2027,regression,False,0.6174221659990161,0.3740889884581573,0.348969174488841,0.6599901609389276,0.49513654538751667,0.40993745168318224,0.5304710319598446,0.3141249120235443,ge,0.41 +grefcoco,groundingdino,MiniCPM,gain,2028,regression,False,0.6159744184412116,0.378717068882398,0.3501176870941213,0.6553517464333404,0.4867991946232595,0.3799985944198468,0.49170518673964214,0.33114850521087646,ge,0.38 +grefcoco,groundingdino,MiniCPM,gain,2029,regression,False,0.6138730761121652,0.37937498227604005,0.35035589711595727,0.6597090449082859,0.4959872954655021,0.4299669688663996,0.5455009500042538,0.22381940484046936,ge,0.43 +grefcoco,groundingdino,MiniCPM,gain,2030,regression,False,0.6128259188980252,0.3797719989790999,0.3495788787113972,0.6537353292571509,0.4996455208008394,0.46995572422517395,0.5867339704506139,0.22586381435394287,ge,0.47 +grefcoco,groundingdino,MiniCPM,route,2026,probability,False,0.6102888467214843,0.37863482970819273,0.3489464878200947,0.6519080750579802,0.4875365113575135,0.3899782135076253,0.5070470464793126,0.3191682696342468,ge,0.39 +grefcoco,groundingdino,MiniCPM,route,2027,probability,False,0.6112095017218357,0.37822646967075974,0.34943708703173293,0.6542272823107738,0.5000141791679664,0.46995572422517395,0.5829339534356124,0.2239772230386734,ge,0.47 +grefcoco,groundingdino,MiniCPM,route,2028,probability,False,0.6121512404244852,0.381246632447608,0.3505345546323342,0.6566870475788882,0.5091455633383433,0.5599831330381615,0.679125428919831,0.1410127878189087,ge,0.56 +grefcoco,groundingdino,MiniCPM,route,2029,probability,False,0.6093330522173027,0.37595780279613195,0.34845588860845644,0.6587251388010401,0.509570938377336,0.5499332349427226,0.6692567280152001,0.16817709803581238,ge,0.55 +grefcoco,groundingdino,MiniCPM,route,2030,probability,False,0.612629137676576,0.3781413946629612,0.3488840994810425,0.6599198819312672,0.4926693701613589,0.40993745168318224,0.5272098233275672,0.252823144197464,ge,0.41 +grefcoco,groundingdino,Qwen3.8,error,2026,probability,False,0.6632581347951367,0.3435243739897343,0.36802881206930776,0.8076463560334528,0.7476391685335905,0.8499543186450207,0.9376683776196013,0.022591590881347656,ge,0.85 +grefcoco,groundingdino,Qwen3.8,error,2027,probability,False,0.664171761894722,0.35875847205285993,0.37080509315713356,0.8063813338955654,0.7456540850182911,0.9199522102747909,0.9775401979411849,0.02014699950814247,ge,0.92 +grefcoco,groundingdino,Qwen3.8,error,2028,probability,False,0.6648604961697941,0.3547599466863285,0.37165017156793245,0.809051936186661,0.7488018603068372,0.7499472907442547,0.8757904886141281,0.06771374493837357,ge,0.75 +grefcoco,groundingdino,Qwen3.8,error,2029,probability,False,0.6639328132686766,0.35926608626605794,0.3697756855627712,0.8100358422939068,0.749397385361427,0.7599971888396936,0.8780307971528231,0.11147670447826385,ge,0.76 +grefcoco,groundingdino,Qwen3.8,error,2030,probability,False,0.6646988544521751,0.3665116410969005,0.37108867651646205,0.8096141682479443,0.7483481269319117,0.7599971888396936,0.8804412557071151,0.09122030436992645,ge,0.76 +grefcoco,groundingdino,Qwen3.8,four_difference,2026,log_ratio,True,0.6653875887272471,0.3563536851657545,0.37075404815245444,0.8131983976386253,0.7484332019397102,0.7199381544732588,0.8543799449848283,1.1629548072814941,ge,0.72 +grefcoco,groundingdino,Qwen3.8,four_difference,2026,probability_difference,False,0.64471853257432,0.4197714318123813,0.3737997334316422,0.8122144915313796,0.7510988855173978,0.889943074003795,0.9439639281966934,-0.005598927848041058,ge,0.89 +grefcoco,groundingdino,Qwen3.8,four_difference,2027,log_ratio,True,0.6657249279640172,0.36034086719791286,0.37154808155857416,0.8140417457305503,0.7490570853302329,0.7299880525686977,0.864475512576922,0.9775638580322266,ge,0.73 +grefcoco,groundingdino,Qwen3.8,four_difference,2027,probability_difference,False,0.6501792114695341,0.42674191078467516,0.3756628761024303,0.8137606296999086,0.7514108272126592,0.7599971888396936,0.8772651220826362,0.007067253813147545,ge,0.76 +grefcoco,groundingdino,Qwen3.8,four_difference,2028,log_ratio,True,0.6657319558647832,0.3570966735671951,0.3724158466381193,0.8109494693934922,0.7490003686583672,0.7599971888396936,0.8856024728468933,0.5702824592590332,ge,0.76 +grefcoco,groundingdino,Qwen3.8,four_difference,2028,probability_difference,False,0.6468269028041324,0.42378413634687917,0.37404928678785127,0.8108791903858318,0.7490003686583672,0.7399676716564763,0.8646456625925191,0.04995465278625488,ge,0.74 +grefcoco,groundingdino,Qwen3.8,four_difference,2029,log_ratio,True,0.6677208517815728,0.3597396704761365,0.37265689249354844,0.8180476491671937,0.6975016306043161,0.4999648604961698,0.6610611689306072,4.503515243530273,ge,0.5 +grefcoco,groundingdino,Qwen3.8,four_difference,2029,probability_difference,False,0.6497926769274018,0.42418398888353237,0.37587839945551993,0.8146039777918336,0.7525168023140402,0.769976807927472,0.8896009982134249,0.0001585048739798367,ge,0.77 +grefcoco,groundingdino,Qwen3.8,four_difference,2030,log_ratio,True,0.6672710661325463,0.35784533363582227,0.37289510251538444,0.8139011877152295,0.68598814621558,0.45997610513739545,0.6289311743186909,5.074526786804199,ge,0.46 +grefcoco,groundingdino,Qwen3.8,four_difference,2030,probability_difference,False,0.6506219692177946,0.41705753906360776,0.37504182854550094,0.8121442125237192,0.7515242605563905,0.769976807927472,0.8831352976207356,0.0009468437638133764,ge,0.77 +grefcoco,groundingdino,Qwen3.8,four_ratio,2026,log_ratio,False,0.6681987490336636,0.35673085103366137,0.3720074866006863,0.814674256799494,0.6775090037716587,0.449996486049617,0.6187788900547316,4.9009013175964355,ge,0.45 +grefcoco,groundingdino,Qwen3.8,four_ratio,2026,probability_difference,True,0.6444866118490407,0.4191759067577915,0.37435272098233274,0.8118630964930775,0.7504750021268752,0.7799564270152506,0.8855173978390948,0.0017387100961059332,ge,0.78 +grefcoco,groundingdino,Qwen3.8,four_ratio,2027,log_ratio,False,0.6675029868578255,0.3584578736919718,0.3718742024218019,0.8115819804624359,0.7508436604940022,0.769976807927472,0.8901681649320818,0.7685046792030334,ge,0.77 +grefcoco,groundingdino,Qwen3.8,four_ratio,2027,probability_difference,True,0.636355330662731,0.42979893939823616,0.3723761449678133,0.8112305854241338,0.7503048521112781,0.7999859441984679,0.9094518333664181,0.0002940092235803604,ge,0.8 +grefcoco,groundingdino,Qwen3.8,four_ratio,2028,log_ratio,False,0.6682479443390259,0.35434875081530215,0.37289793834897766,0.8178368121442126,0.6524118764710887,0.4299669688663996,0.559963701330006,4.766618251800537,ge,0.43 +grefcoco,groundingdino,Qwen3.8,four_ratio,2028,probability_difference,True,0.6435589289479232,0.4260783257238465,0.37392451010974675,0.8148148148148148,0.7518362022516518,0.7199381544732588,0.8465246859314296,0.02629547379910946,ge,0.72 +grefcoco,groundingdino,Qwen3.8,four_ratio,2029,log_ratio,False,0.6682830838428562,0.3561211468111051,0.37262002665683575,0.8134092346616065,0.7514675438845249,0.7999859441984679,0.9124578169752999,0.5637103319168091,ge,0.8 +grefcoco,groundingdino,Qwen3.8,four_ratio,2029,probability_difference,True,0.6365942792887764,0.439707341973173,0.37266823582792163,0.81228477053904,0.7514108272126592,0.769976807927472,0.882312905878683,0.026074310764670372,ge,0.77 +grefcoco,groundingdino,Qwen3.8,four_ratio,2030,log_ratio,False,0.6682971396443882,0.3633241641380484,0.37567705527039674,0.8128470026003233,0.7499078354082183,0.7499472907442547,0.8657516376939001,0.4791870713233948,ge,0.75 +grefcoco,groundingdino,Qwen3.8,four_ratio,2030,probability_difference,True,0.6425398833368473,0.4160763406403312,0.37284972917789183,0.8117225384777567,0.7500212687519496,0.7599971888396936,0.8725576383177835,0.006772097200155258,ge,0.76 +grefcoco,groundingdino,Qwen3.8,gain,2026,regression,False,0.6500948766603415,0.4085528741173468,0.37506167938065393,0.8111603064164734,0.7486884269631058,0.8399746995572422,0.9279981850665003,-0.02583235502243042,ge,0.84 +grefcoco,groundingdino,Qwen3.8,gain,2027,regression,False,0.6490266357439033,0.40404957037121064,0.3745710801690157,0.8083491461100569,0.7489152936505686,0.8599339377327992,0.931032527011315,-0.046560950577259064,ge,0.86 +grefcoco,groundingdino,Qwen3.8,gain,2028,regression,False,0.6493920865837374,0.41194736692850864,0.37493406686895614,0.8083491461100569,0.7489152936505686,0.889943074003795,0.9355982190965034,-0.011892817914485931,ge,0.89 +grefcoco,groundingdino,Qwen3.8,gain,2029,regression,False,0.6419776512755639,0.41630037149420074,0.37266823582792163,0.8077869140487737,0.7461928934010152,0.7299880525686977,0.8561948784845306,0.2810576260089874,ge,0.73 +grefcoco,groundingdino,Qwen3.8,gain,2030,regression,False,0.6458289408953546,0.4047443496015654,0.37252927998185065,0.8073652400028112,0.7478376768851204,0.9799704828167827,0.9813969316280521,-0.14250685274600983,ge,0.98 +grefcoco,groundingdino,Qwen3.8,route,2026,probability,False,0.6444866118490405,0.41589201145676774,0.373618240081672,0.8079977510717549,0.7483764852678445,0.7799564270152506,0.8789099055667413,0.03565872833132744,ge,0.78 +grefcoco,groundingdino,Qwen3.8,route,2027,probability,False,0.64471853257432,0.4247710064373423,0.37423928763860137,0.8113008644317943,0.7493690270254941,0.7399676716564763,0.8599948954995321,0.052881352603435516,ge,0.74 +grefcoco,groundingdino,Qwen3.8,route,2028,probability,False,0.6485838779956428,0.42200323285029634,0.3751977993931316,0.809051936186661,0.7477526018773218,0.7799564270152506,0.8825964892380115,0.04733765125274658,ge,0.78 +grefcoco,groundingdino,Qwen3.8,route,2029,probability,False,0.645920303605313,0.4168845532144174,0.3746419760088478,0.8083491461100569,0.7472705101664634,0.7599971888396936,0.8672262711624081,0.07822275161743164,ge,0.76 +grefcoco,groundingdino,Qwen3.8,route,2030,probability,False,0.6486682128048352,0.4162493264895216,0.3751269035532995,0.8112305854241338,0.7486317102912401,0.7399676716564763,0.8654964126705045,0.059102341532707214,ge,0.74 +grefcoco,instancevg,MiniCPM,error,2026,probability,False,0.7587954178087006,0.6688625471457335,0.6756260102657177,0.7621758380771664,0.6750701868814338,0.119966266076323,0.2011740351076199,0.09462917596101761,ge,0.12 +grefcoco,instancevg,MiniCPM,error,2027,probability,False,0.754782486471291,0.6689220996511925,0.6757366077758558,0.7564129594490125,0.6766298953577404,0.119966266076323,0.19453818449933358,0.11577890813350677,ge,0.12 +grefcoco,instancevg,MiniCPM,error,2028,probability,False,0.7610302902523016,0.6724186824717127,0.6764540736749567,0.7705390399887554,0.6594447437824349,0.22995291306486754,0.3116013952301279,0.01688133366405964,ge,0.23 +grefcoco,instancevg,MiniCPM,error,2029,probability,False,0.7553376906318083,0.6730142075263024,0.6763718345007514,0.7609108159392789,0.6647761109378102,0.20999367488931056,0.27918781725888325,0.07360158115625381,ge,0.21 +grefcoco,instancevg,MiniCPM,error,2030,probability,False,0.7579309860144775,0.6706689731446558,0.6759578027961319,0.7615433270082227,0.675041828545501,0.13999578325954037,0.21084422766072086,0.09310838580131531,ge,0.14 +grefcoco,instancevg,MiniCPM,four_difference,2026,log_ratio,True,0.764670742849111,0.6763945211694977,0.6784221421886965,0.7743341064024176,0.6692567280152001,0.21997329397708903,0.290389359952358,-5.367876052856445,ge,0.22 +grefcoco,instancevg,MiniCPM,four_difference,2026,probability_difference,False,0.7550636025019327,0.6777075121231887,0.6773587045912146,0.7559210063953897,0.6787851288886368,0.06999789162977019,0.13972152114113945,-0.03360140696167946,ge,0.07 +grefcoco,instancevg,MiniCPM,four_difference,2027,log_ratio,True,0.7550987420057629,0.6750758585486204,0.6768822845475427,0.7571157495256167,0.6745597368346425,0.16993464052287582,0.23001446275132575,-3.7348968982696533,ge,0.17 +grefcoco,instancevg,MiniCPM,four_difference,2027,probability_difference,False,0.7550214350973364,0.6781584096645209,0.6777131837903752,0.7562021224260314,0.6794090122791595,0.16993464052287582,0.2406771970620764,-0.018624423071742058,ge,0.17 +grefcoco,instancevg,MiniCPM,four_difference,2028,log_ratio,True,0.7558507273877293,0.6748291410260046,0.6773048237529421,0.7574671445639187,0.6803731957008763,0.10998664698854452,0.17284405751070528,-3.0022528171539307,ge,0.11 +grefcoco,instancevg,MiniCPM,four_difference,2028,probability_difference,False,0.7546911237613325,0.6797408048095738,0.6792870714346482,0.756483238456673,0.681167229106996,0.05994799353433129,0.1307319286504268,-0.0012479540891945362,ge,0.06 +grefcoco,instancevg,MiniCPM,four_difference,2029,log_ratio,True,0.7576147304800056,0.677741542126308,0.6783342313473046,0.7604188628856561,0.6748149618580381,0.20999367488931056,0.25596234012988117,-5.01124382019043,ge,0.21 +grefcoco,instancevg,MiniCPM,four_difference,2029,probability_difference,False,0.7553517464333404,0.6788616963956555,0.678328559680118,0.7571157495256167,0.6801463290134135,0.04996837444655281,0.1162124606528089,-0.0008232841501012444,ge,0.05 +grefcoco,instancevg,MiniCPM,four_difference,2030,log_ratio,True,0.7548035701735891,0.6733771942262428,0.6768397470436435,0.7573968655562583,0.6815642458100558,0.04996837444655281,0.11862291920710093,-0.6041420102119446,ge,0.05 +grefcoco,instancevg,MiniCPM,four_difference,2030,probability_difference,False,0.7557101693724084,0.6802881206930778,0.6796897598048947,0.7569751915102959,0.6817627541615858,0.05994799353433129,0.12883192014292602,-0.0011288531823083758,ge,0.06 +grefcoco,instancevg,MiniCPM,four_ratio,2026,log_ratio,False,0.764670742849111,0.6763945211694977,0.6784221421886965,0.7743341064024176,0.6692567280152001,0.21997329397708903,0.290389359952358,-5.367876052856445,ge,0.22 +grefcoco,instancevg,MiniCPM,four_ratio,2026,probability_difference,True,0.7550636025019327,0.6777075121231887,0.6773587045912146,0.7559210063953897,0.6787851288886368,0.06999789162977019,0.13972152114113945,-0.03360140696167946,ge,0.07 +grefcoco,instancevg,MiniCPM,four_ratio,2027,log_ratio,False,0.7594560404807085,0.6767433287014718,0.6778181096333267,0.7694848548738492,0.6726313699912089,0.21997329397708903,0.27853557553242775,-4.522805690765381,ge,0.22 +grefcoco,instancevg,MiniCPM,four_ratio,2027,probability_difference,True,0.7519361866610443,0.6786490088761592,0.6782860221762187,0.7528287300583316,0.6800896123415477,0.05994799353433129,0.13016476193176985,-0.00909316074103117,ge,0.06 +grefcoco,instancevg,MiniCPM,four_ratio,2028,log_ratio,False,0.7606226720078713,0.6761591469812551,0.6776252729489833,0.7669548105980744,0.6724895783115447,0.19994377679387168,0.2623713240507047,-4.883466720581055,ge,0.2 +grefcoco,instancevg,MiniCPM,four_ratio,2028,probability_difference,True,0.7536158549441282,0.6785128888636816,0.678229305504353,0.7547965422728231,0.6799478206618835,0.04996837444655281,0.10764824320108896,-0.002954066265374422,ge,0.05 +grefcoco,instancevg,MiniCPM,four_ratio,2029,log_ratio,False,0.7576147304800056,0.677741542126308,0.6783342313473046,0.7604188628856561,0.6748149618580381,0.20999367488931056,0.25596234012988117,-5.01124382019043,ge,0.21 +grefcoco,instancevg,MiniCPM,four_ratio,2029,probability_difference,True,0.7553517464333404,0.6788616963956555,0.678328559680118,0.7571157495256167,0.6801463290134135,0.04996837444655281,0.1162124606528089,-0.0008232841501012444,ge,0.05 +grefcoco,instancevg,MiniCPM,four_ratio,2030,log_ratio,False,0.7573055028462998,0.6742421234721947,0.677143181238125,0.7644950453299599,0.6671014944843037,0.20999367488931056,0.2866460596092221,-4.427010536193848,ge,0.21 +grefcoco,instancevg,MiniCPM,four_ratio,2030,probability_difference,True,0.7545786773490759,0.6790715480815586,0.6788276663925361,0.7561318434183709,0.6804582707086748,0.03998875535877434,0.1021750843660494,-0.00392303429543972,ge,0.04 +grefcoco,instancevg,MiniCPM,gain,2026,regression,False,0.753236348302762,0.678237813005133,0.6778748263051924,0.7551479373111252,0.6793806539432267,0.03998875535877434,0.10106910926466835,0.07657840847969055,ge,0.04 +grefcoco,instancevg,MiniCPM,gain,2027,regression,False,0.754930072387378,0.6774466154326064,0.6772310920795168,0.7560615644107105,0.6790403539120324,0.07997751071754867,0.15177381391259961,-0.015288867056369781,ge,0.08 +grefcoco,instancevg,MiniCPM,gain,2028,regression,False,0.754079696394687,0.6769701953889347,0.6766866120296061,0.7557101693724084,0.6783313955137112,0.029938857263335443,0.0887899498057454,0.08195191621780396,ge,0.03 +grefcoco,instancevg,MiniCPM,gain,2029,regression,False,0.7510998664698855,0.6760031761336245,0.6757479511102288,0.7526178930353503,0.676885120381136,0.119966266076323,0.16405297337152255,-0.07293874025344849,ge,0.12 +grefcoco,instancevg,MiniCPM,gain,2030,regression,False,0.7530465949820788,0.6773558687576213,0.6768709412131697,0.7548668212804835,0.67782094546692,0.029938857263335443,0.0859257578765278,0.08320517092943192,ge,0.03 +grefcoco,instancevg,MiniCPM,route,2026,probability,False,0.7737156511350061,0.6649320817854409,0.6720897257748916,0.7869140487736314,0.6456909508550038,0.22995291306486754,0.3532030740436151,3.4431228414177895e-05,ge,0.23 +grefcoco,instancevg,MiniCPM,route,2027,probability,False,0.7712910253707217,0.67043076312282,0.6732552533817315,0.783610935413592,0.6626775940787795,0.21997329397708903,0.29064458497575363,0.0035944085102528334,ge,0.22 +grefcoco,instancevg,MiniCPM,route,2028,probability,False,0.7668142525827535,0.6693162805206591,0.6747554093525792,0.7727879682338885,0.6686612029606103,0.15995502143509735,0.22978759606386298,0.0013828137889504433,ge,0.16 +grefcoco,instancevg,MiniCPM,route,2029,probability,False,0.7621547543748682,0.6710518106797494,0.6748801860306838,0.7668142525827535,0.6733403283895301,0.13999578325954037,0.1978277514675439,0.00121789809782058,ge,0.14 +grefcoco,instancevg,MiniCPM,route,2030,probability,False,0.7648745519713261,0.6762158636531208,0.6774296004310467,0.7703282029657741,0.6768284037092703,0.16993464052287582,0.22462637892408474,0.004773556720465422,ge,0.17 +grefcoco,instancevg,Qwen3.8,error,2026,probability,False,0.7873005833157636,0.7298925219068145,0.7142075263023566,0.8309087075690491,0.7584436945240053,0.4799353433129524,0.629129682670221,0.0005966582684777677,ge,0.48 +grefcoco,instancevg,Qwen3.8,error,2027,probability,False,0.7805748822826621,0.7284774409437654,0.7136687179196325,0.8188207182514583,0.7566854776961688,0.6499402628434886,0.766866120296061,0.000342582497978583,ge,0.65 +grefcoco,instancevg,Qwen3.8,error,2028,probability,False,0.7852976315974418,0.7277684825454442,0.7133539403907779,0.8253566659638766,0.7554944275869891,0.3599690772366294,0.4618722173382866,0.002208717865869403,ge,0.36 +grefcoco,instancevg,Qwen3.8,error,2029,probability,False,0.7809684447255605,0.7278932592235487,0.713603493746987,0.8258486190174995,0.7577347361256841,0.46995572422517395,0.5859966537163599,0.0004450614214874804,ge,0.47 +grefcoco,instancevg,Qwen3.8,error,2030,probability,False,0.7836390470166561,0.7273147491705186,0.7135042395712219,0.8297139644388222,0.7561750276493775,0.48998524140839134,0.6496894762215353,1.606099795026239e-05,ge,0.49 +grefcoco,instancevg,Qwen3.8,four_difference,2026,log_ratio,True,0.7851219340782908,0.7299690894138332,0.7140175254516065,0.8315412186379928,0.7620168448515441,0.4698854452175135,0.6019907551824859,-5.802664279937744,gt,0.47 +grefcoco,instancevg,Qwen3.8,four_difference,2026,probability_difference,False,0.7670883407126291,0.7108782576638404,0.7054618155006664,0.8080680300794153,0.7457108016901568,0.9699908637290041,0.9670192553100984,-0.7546408772468567,ge,0.97 +grefcoco,instancevg,Qwen3.8,four_difference,2027,log_ratio,True,0.779865064305292,0.7305475994668633,0.7134191645634235,0.8200154613816852,0.7604571363752375,0.40993745168318224,0.5371068825681309,-5.039910793304443,ge,0.41 +grefcoco,instancevg,Qwen3.8,four_difference,2027,probability_difference,False,0.7705390399887554,0.7202620310240195,0.7091909366758358,0.8084194251177174,0.7500212687519496,0.9599409656335652,0.9593057879363639,-0.6507218480110168,ge,0.96 +grefcoco,instancevg,Qwen3.8,four_difference,2028,log_ratio,True,0.7839271909480638,0.7292969968522247,0.7135184187391884,0.8321737297069366,0.76127952811729,0.48998524140839134,0.6452939341519439,-6.389966011047363,ge,0.49 +grefcoco,instancevg,Qwen3.8,four_difference,2028,probability_difference,False,0.7674678473539953,0.7092334741797351,0.7034115078127215,0.8073652400028112,0.7466749851118737,0.9699205847213437,0.9723789808014066,-0.7309830784797668,gt,0.97 +grefcoco,instancevg,Qwen3.8,four_difference,2029,log_ratio,True,0.7787406001827254,0.7273487791736382,0.7128576695119531,0.8107386323705109,0.7550690525479965,0.5299037177595053,0.5922922042934521,-4.511953353881836,gt,0.53 +grefcoco,instancevg,Qwen3.8,four_difference,2029,probability_difference,False,0.765872513880104,0.7148257380256927,0.7079261548932309,0.8085599831330381,0.7477809602132547,0.9799704828167827,0.9796387148002155,-0.6767090559005737,ge,0.98 +grefcoco,instancevg,Qwen3.8,four_difference,2030,log_ratio,True,0.7802094314428281,0.7272722116666194,0.7127442361682217,0.820647972450629,0.7574795111022885,0.6599901609389276,0.8281768425828773,-7.195610523223877,ge,0.66 +grefcoco,instancevg,Qwen3.8,four_difference,2030,probability_difference,False,0.7675240705601237,0.711105124351303,0.7048577829452968,0.8091222151943215,0.7473272268383291,0.9599409656335652,0.9641834217168137,-0.5991365909576416,ge,0.96 +grefcoco,instancevg,Qwen3.8,four_ratio,2026,log_ratio,False,0.7851219340782908,0.7299690894138332,0.7140175254516065,0.8315412186379928,0.7620168448515441,0.4698854452175135,0.6019907551824859,-5.802664279937744,gt,0.47 +grefcoco,instancevg,Qwen3.8,four_ratio,2026,probability_difference,True,0.7670883407126291,0.7108782576638404,0.7054618155006664,0.8080680300794153,0.7457108016901568,0.9699908637290041,0.9670192553100984,-0.7546408772468567,ge,0.97 +grefcoco,instancevg,Qwen3.8,four_ratio,2027,log_ratio,False,0.7811933375500738,0.7285284859484445,0.7138076737657034,0.8232482957340642,0.7607123613986331,0.46995572422517395,0.5680458270708675,-5.170469760894775,ge,0.47 +grefcoco,instancevg,Qwen3.8,four_ratio,2027,probability_difference,True,0.7647129102537072,0.7124719961432664,0.7069761506394805,0.8080680300794153,0.7476958852054562,0.9599409656335652,0.9606386297252077,-0.5886308550834656,ge,0.96 +grefcoco,instancevg,Qwen3.8,four_ratio,2028,log_ratio,False,0.7839271909480638,0.7292969968522247,0.7135184187391884,0.8321737297069366,0.76127952811729,0.48998524140839134,0.6452939341519439,-6.389966011047363,ge,0.49 +grefcoco,instancevg,Qwen3.8,four_ratio,2028,probability_difference,True,0.7674678473539953,0.7092334741797351,0.7034115078127215,0.8073652400028112,0.7466749851118737,0.9699205847213437,0.9723789808014066,-0.7309830784797668,gt,0.97 +grefcoco,instancevg,Qwen3.8,four_ratio,2029,log_ratio,False,0.7787406001827254,0.7273487791736382,0.7128576695119531,0.8107386323705109,0.7550690525479965,0.5299037177595053,0.5922922042934521,-4.511953353881836,gt,0.53 +grefcoco,instancevg,Qwen3.8,four_ratio,2029,probability_difference,True,0.765872513880104,0.7148257380256927,0.7079261548932309,0.8085599831330381,0.7477809602132547,0.9799704828167827,0.9796387148002155,-0.6767090559005737,ge,0.98 +grefcoco,instancevg,Qwen3.8,four_ratio,2030,log_ratio,False,0.7814463419776514,0.7290786376655418,0.7141848396336103,0.8264811300864432,0.7606556447267674,0.45997610513739545,0.5814309616311715,-5.810272693634033,ge,0.46 +grefcoco,instancevg,Qwen3.8,four_ratio,2030,probability_difference,True,0.7640241759786353,0.7115418427246689,0.7059609222130845,0.8084194251177174,0.7497376853926212,0.9699908637290041,0.9668491052945013,-0.6625540852546692,ge,0.97 +grefcoco,instancevg,Qwen3.8,gain,2026,regression,False,0.769126431934781,0.7125315486487254,0.7037234495079828,0.8061002178649237,0.7502481354394124,0.9199522102747909,0.9159458922950401,-0.4572525918483734,ge,0.92 +grefcoco,instancevg,Qwen3.8,gain,2027,regression,False,0.7719375922411975,0.7199132234920456,0.7080452599041488,0.8053271487806593,0.7461928934010152,0.9899501019045611,0.9879760655644727,-0.7287997007369995,ge,0.99 +grefcoco,instancevg,Qwen3.8,gain,2028,regression,False,0.7640312038794013,0.7033349403057029,0.6972889430848197,0.8069435659568487,0.7452287099792985,0.9799704828167827,0.9803193148626038,-0.6322726607322693,ge,0.98 +grefcoco,instancevg,Qwen3.8,gain,2029,regression,False,0.7687469252934149,0.7148767830303718,0.7062558489067862,0.8077869140487737,0.7462212517369481,0.9699908637290041,0.9731730142075263,-0.8551065921783447,ge,0.97 +grefcoco,instancevg,Qwen3.8,gain,2030,regression,False,0.7703422587673062,0.7128293111760202,0.7030768794487139,0.8065218919108862,0.7455122933386269,0.9899501019045611,0.9889118906502566,-0.850878119468689,ge,0.99 +grefcoco,instancevg,Qwen3.8,route,2026,probability,False,0.789085670110338,0.7282959475937951,0.7137452854266512,0.831400660622672,0.7565436860165046,0.3699486963244079,0.4927260868332246,0.0005373849417082965,ge,0.37 +grefcoco,instancevg,Qwen3.8,route,2027,probability,False,0.7869070208728652,0.7302895386098744,0.7145761846694836,0.8283786632932743,0.7587839945551995,0.3799985944198468,0.4800499106712418,0.0014764259103685617,ge,0.38 +grefcoco,instancevg,Qwen3.8,route,2028,probability,False,0.788888888888889,0.7240620480390211,0.712389756969061,0.8275353152013494,0.756458611008706,0.40993745168318224,0.5148172305249128,0.00015934875409584492,ge,0.41 +grefcoco,instancevg,Qwen3.8,route,2029,probability,False,0.7894230093471081,0.7193120267702692,0.7108782576638403,0.8248647129102538,0.7549272608683323,0.3899782135076253,0.48192156084280974,0.0003914774861186743,ge,0.39 +grefcoco,instancevg,Qwen3.8,route,2030,probability,False,0.7843207533909622,0.723991152199189,0.7128548336783598,0.8356174010822968,0.7608825114142302,0.3899782135076253,0.5286560984601424,0.0004941682564094663,ge,0.39 +nlvr2,nlvr2,MiniCPM,error,2026,probability,False,0.8352621025494128,0.8419745999856497,0.8415799669943317,0.8993125179031796,0.908301643108273,0.8998854196505299,0.8963191504627969,1.560075907036662e-05,ge,0.9 +nlvr2,nlvr2,MiniCPM,error,2027,probability,False,0.8357920366657118,0.8416588935925953,0.8409700796441129,0.8997421942136924,0.9085886489201407,0.9099112002291607,0.908301643108273,0.00012601532216649503,ge,0.91 +nlvr2,nlvr2,MiniCPM,error,2028,probability,False,0.8353766828988828,0.83924086962761,0.8393915476788405,0.8997421942136924,0.9071536198608022,0.8599255227728445,0.8617349501327402,1.4528410247294232e-05,ge,0.86 +nlvr2,nlvr2,MiniCPM,error,2029,probability,False,0.8358636493841306,0.8424481595752314,0.8416588935925953,0.9000286450873675,0.9079428858434383,0.8299914064737898,0.8246394489488412,0.001538369688205421,ge,0.83 +nlvr2,nlvr2,MiniCPM,error,2030,probability,False,0.8362790031509596,0.8417234699002656,0.841429288943101,0.9000286450873675,0.9074406256726699,0.9498710971068461,0.9429575948912966,0.00030057248659431934,ge,0.95 +nlvr2,nlvr2,MiniCPM,four_difference,2026,log_ratio,True,0.8359925522772844,0.8425127358829017,0.8415153906866615,0.8998854196505299,0.9081581402023391,0.879977083930106,0.8691253497883332,-2.989978551864624,ge,0.88 +nlvr2,nlvr2,MiniCPM,four_difference,2026,probability_difference,False,0.8365654540246348,0.8430221711989668,0.8424266341393414,0.8994557433400172,0.9071536198608022,0.9899742194213692,0.9901700509435316,-0.10858634114265442,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_difference,2027,log_ratio,True,0.8359066170151819,0.8422329052163307,0.8412283848747937,0.8997421942136924,0.9077276314845376,0.939988541965053,0.9338451603644974,-5.144361972808838,ge,0.94 +nlvr2,nlvr2,MiniCPM,four_difference,2027,probability_difference,False,0.8359352621025494,0.842577312190572,0.8416301930114085,0.8983099398453165,0.9067948625959675,0.9899742194213692,0.9910310683791347,-0.18117938935756683,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_difference,2028,log_ratio,True,0.8346748782583787,0.8403243165674106,0.8385735811150176,0.899169292466342,0.9075123771256368,0.9899742194213692,0.9906723111143001,-7.820161819458008,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_difference,2028,probability_difference,False,0.8365368089372672,0.8405252206357179,0.8401018870632131,0.8984531652821541,0.9080146372964053,0.9899742194213692,0.9903853053024324,-0.3423033654689789,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_difference,2029,log_ratio,True,0.8357633915783443,0.8424768601564182,0.8407189495587286,0.9006015468347178,0.9071536198608022,0.8498997421942137,0.8412857860371673,-3.638676881790161,ge,0.85 +nlvr2,nlvr2,MiniCPM,four_difference,2029,probability_difference,False,0.836665711830421,0.8429360694554064,0.8418454473703093,0.8990260670295045,0.9060055966133315,0.9899742194213692,0.9891655306019946,-0.20271095633506775,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_difference,2030,log_ratio,True,0.8345316528215411,0.8423907584128579,0.8411638085671235,0.9000286450873675,0.9074406256726699,0.8498997421942137,0.8453756188562819,-3.771627902984619,ge,0.85 +nlvr2,nlvr2,MiniCPM,four_difference,2030,probability_difference,False,0.836551131480951,0.8435674822415155,0.8422687809428141,0.8983099398453165,0.9075123771256368,0.9899742194213692,0.9914615770969362,-0.19141565263271332,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_ratio,2026,log_ratio,False,0.8362933256946434,0.8420033005668366,0.8414651646695845,0.8993125179031796,0.9077993829375045,0.939988541965053,0.932338379852192,-7.357630252838135,ge,0.94 +nlvr2,nlvr2,MiniCPM,four_ratio,2026,probability_difference,True,0.8347894586078488,0.8397646552342686,0.8391906436105331,0.8977370380979662,0.9075841285786037,0.9899742194213692,0.9922508430795723,-0.432656854391098,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_ratio,2027,log_ratio,False,0.8359066170151819,0.8422329052163307,0.8412283848747937,0.8997421942136924,0.9077276314845376,0.939988541965053,0.9338451603644974,-5.144361972808838,ge,0.94 +nlvr2,nlvr2,MiniCPM,four_ratio,2027,probability_difference,True,0.8359352621025494,0.842577312190572,0.8416301930114085,0.8983099398453165,0.9067948625959675,0.9899742194213692,0.9910310683791347,-0.18117938935756683,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_ratio,2028,log_ratio,False,0.8358779719278144,0.8413862380713211,0.840733299849322,0.8994557433400172,0.9075123771256368,0.9699226582641077,0.9676400947119179,-7.614771842956543,ge,0.97 +nlvr2,nlvr2,MiniCPM,four_ratio,2028,probability_difference,True,0.8354912632483529,0.8412642606012772,0.8406256726698715,0.8975938126611286,0.9067948625959675,0.9899742194213692,0.9903853053024324,-0.22819791734218597,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_ratio,2029,log_ratio,False,0.8364222285877971,0.8426992896606157,0.8419530745497597,0.9000286450873675,0.908229891655306,0.9299627613864222,0.9264547607089043,-6.099917411804199,ge,0.93 +nlvr2,nlvr2,MiniCPM,four_ratio,2029,probability_difference,True,0.8346032655399599,0.8417019444643755,0.8411709837124202,0.898166714408479,0.906436105331133,0.9899742194213692,0.9916050800028701,-0.16860052943229675,ge,0.99 +nlvr2,nlvr2,MiniCPM,four_ratio,2030,log_ratio,False,0.8356774563162417,0.8417091196096722,0.8410346559517831,0.8997421942136924,0.9080863887493722,0.9299627613864222,0.9296835760924159,-4.7400689125061035,ge,0.93 +nlvr2,nlvr2,MiniCPM,four_ratio,2030,probability_difference,True,0.8357347464909768,0.8416804190284853,0.8413360120542441,0.8985963907189917,0.9076558800315706,0.9899742194213692,0.9911745712850685,-0.1725480854511261,ge,0.99 +nlvr2,nlvr2,MiniCPM,gain,2026,regression,False,0.8348324262389001,0.840733299849322,0.8403530171485973,0.8985963907189917,0.9067231111430006,0.9899742194213692,0.9903135538494654,-0.21196576952934265,ge,0.99 +nlvr2,nlvr2,MiniCPM,gain,2027,regression,False,0.8349040389573188,0.8431656741049007,0.8420320011480232,0.8987396161558293,0.9069383655019014,0.9899742194213692,0.9885197675252924,-0.23289483785629272,ge,0.99 +nlvr2,nlvr2,MiniCPM,gain,2028,regression,False,0.8344457175594385,0.8393987228241373,0.8397861806701585,0.8977370380979662,0.9069383655019014,0.9899742194213692,0.9908875654732008,-0.3863624930381775,ge,0.99 +nlvr2,nlvr2,MiniCPM,gain,2029,regression,False,0.8354339730736179,0.842319006959891,0.8413790629260243,0.8988828415926668,0.9067231111430006,0.9899742194213692,0.9918203343617709,-0.3368431627750397,ge,0.99 +nlvr2,nlvr2,MiniCPM,gain,2030,regression,False,0.8355342308794041,0.8399942598837628,0.8395637511659612,0.8988828415926668,0.9062208509722322,0.9899742194213692,0.9901700509435316,-0.2702803313732147,ge,0.99 +nlvr2,nlvr2,MiniCPM,route,2026,probability,False,0.8362360355199083,0.8426777642247256,0.8420894023103968,0.8995989687768547,0.907297122766736,0.879977083930106,0.8772332639735955,9.101400064537302e-05,ge,0.88 +nlvr2,nlvr2,MiniCPM,route,2027,probability,False,0.8369235176167288,0.8403530171485973,0.8400875367726197,0.8998854196505299,0.9075841285786037,0.8398739616155829,0.8386309822773911,1.2550027349789161e-05,ge,0.84 +nlvr2,nlvr2,MiniCPM,route,2028,probability,False,0.836379260956746,0.8418741479514961,0.8408768027552558,0.8998854196505299,0.9077276314845376,0.9199369808077915,0.9158355456697999,0.0004337143909651786,ge,0.92 +nlvr2,nlvr2,MiniCPM,route,2029,probability,False,0.8364079060441134,0.8415010403960681,0.8413073114730574,0.8994557433400172,0.9079428858434383,0.8699513033514752,0.8660400373107555,0.00018013599037658423,ge,0.87 +nlvr2,nlvr2,MiniCPM,route,2030,probability,False,0.8363506158693784,0.8418454473703093,0.8413503623448375,0.8997421942136924,0.9072253713137691,0.9299627613864222,0.9272440266915405,0.00011133222869830206,ge,0.93 +nlvr2,nlvr2,Qwen3.8,error,2026,probability,False,0.8395875107419077,0.8425701370452752,0.8420176508574299,0.907046691492409,0.9022745210590515,0.7999140647378975,0.8033292674176652,7.230258779600263e-05,ge,0.8 +nlvr2,nlvr2,Qwen3.8,error,2027,probability,False,0.8398739616155829,0.8425557867546818,0.8420678768745067,0.9069034660555715,0.9022027696060845,0.7399026067029505,0.7407620004305088,0.00032515457132831216,ge,0.74 +nlvr2,nlvr2,Qwen3.8,error,2028,probability,False,0.8388857061014037,0.8408409270287723,0.8405395709263113,0.9069034660555715,0.9036377986654229,0.7199942709825266,0.727559733084595,0.0005172103410586715,ge,0.72 +nlvr2,nlvr2,Qwen3.8,error,2029,probability,False,0.8398023488971641,0.8426203630623521,0.8416517184472986,0.9091950730449728,0.9051445791777284,0.7199942709825266,0.7212456052235058,0.004085870459675789,ge,0.72 +nlvr2,nlvr2,Qwen3.8,error,2030,probability,False,0.8399169292466342,0.8423046566692977,0.84206070172921,0.9093382984818104,0.9034225443065222,0.8398739616155829,0.8320298486044342,0.0013198753586038947,ge,0.84 +nlvr2,nlvr2,Qwen3.8,four_difference,2026,log_ratio,True,0.8395875107419077,0.842505560737605,0.8414149386525077,0.9100544256659983,0.9060773480662984,0.8398739616155829,0.8376264619358542,-5.0886969566345215,ge,0.84 +nlvr2,nlvr2,Qwen3.8,four_difference,2026,probability_difference,False,0.8402893153824119,0.8423118318145943,0.8417952213532324,0.9099112002291607,0.9067948625959675,0.9699226582641077,0.9679988519767525,-0.28458863496780396,ge,0.97 +nlvr2,nlvr2,Qwen3.8,four_difference,2027,log_ratio,True,0.8398309939845316,0.8434455047714717,0.8417952213532324,0.911200229160699,0.9073688742197029,0.8299914064737898,0.8323886058692689,-4.49802303314209,ge,0.83 +nlvr2,nlvr2,Qwen3.8,four_difference,2027,probability_difference,False,0.8411057003723861,0.8443280476429649,0.8430436966348569,0.9084789458607849,0.9039248044772906,0.9799484388427385,0.9771830379565186,-0.25909605622291565,ge,0.98 +nlvr2,nlvr2,Qwen3.8,four_difference,2028,log_ratio,True,0.8402320252076769,0.842857142857143,0.8421898543445505,0.9109137782870238,0.9057185908014638,0.8898596390718991,0.8899332711487408,-5.5232343673706055,ge,0.89 +nlvr2,nlvr2,Qwen3.8,four_difference,2028,probability_difference,False,0.8405327986250357,0.8428427925665496,0.8427064648059124,0.9086221712976225,0.9059338451603645,0.9598968776854769,0.9595321805266557,-0.23020051419734955,ge,0.96 +nlvr2,nlvr2,Qwen3.8,four_difference,2029,log_ratio,True,0.8377112575193354,0.8403243165674106,0.8391188921575662,0.9110570037238613,0.9075841285786037,0.8699513033514752,0.8727846738896463,-4.178714275360107,ge,0.87 +nlvr2,nlvr2,Qwen3.8,four_difference,2029,probability_difference,False,0.84134918361501,0.8437970868910096,0.842713639951209,0.9093382984818104,0.9066513596900337,0.9799484388427385,0.9792638300925594,-0.3751336932182312,ge,0.98 +nlvr2,nlvr2,Qwen3.8,four_difference,2030,log_ratio,True,0.8398739616155829,0.8430508717801537,0.8415225658319581,0.909481523918648,0.9058620937073976,0.7499283872815812,0.7526727416230179,-3.552999496459961,ge,0.75 +nlvr2,nlvr2,Qwen3.8,four_difference,2030,probability_difference,False,0.8405614437124033,0.8431943746860875,0.843086747506637,0.9096247493554855,0.9072253713137691,0.9598968776854769,0.9598909377914903,-0.29492709040641785,ge,0.96 +nlvr2,nlvr2,Qwen3.8,four_ratio,2026,log_ratio,False,0.8401460899455743,0.8429647700365933,0.8425916624811652,0.911486680034374,0.9057903422544307,0.8699513033514752,0.8740762000430509,-4.613958835601807,ge,0.87 +nlvr2,nlvr2,Qwen3.8,four_ratio,2026,probability_difference,True,0.8399598968776855,0.8428643180024397,0.8431728492501973,0.9093382984818104,0.9052163306306953,0.9699226582641077,0.9662050656525795,-0.29177480936050415,ge,0.97 +nlvr2,nlvr2,Qwen3.8,four_ratio,2027,log_ratio,False,0.8398309939845316,0.8434455047714717,0.8417952213532324,0.911200229160699,0.9073688742197029,0.8299914064737898,0.8323886058692689,-4.49802303314209,ge,0.83 +nlvr2,nlvr2,Qwen3.8,four_ratio,2027,probability_difference,True,0.8411057003723861,0.8443280476429649,0.8430436966348569,0.9084789458607849,0.9039248044772906,0.9799484388427385,0.9771830379565186,-0.25909605622291565,ge,0.98 +nlvr2,nlvr2,Qwen3.8,four_ratio,2028,log_ratio,False,0.8402320252076769,0.842857142857143,0.8421898543445505,0.9109137782870238,0.9057185908014638,0.8898596390718991,0.8899332711487408,-5.5232343673706055,ge,0.89 +nlvr2,nlvr2,Qwen3.8,four_ratio,2028,probability_difference,True,0.8405327986250357,0.8428427925665496,0.8427064648059124,0.9086221712976225,0.9059338451603645,0.9598968776854769,0.9595321805266557,-0.23020051419734955,ge,0.96 +nlvr2,nlvr2,Qwen3.8,four_ratio,2029,log_ratio,False,0.8403609281008307,0.8435890076774055,0.8421539786180671,0.9090518476081352,0.9054315849895961,0.8898596390718991,0.88505417234699,-4.287631988525391,ge,0.89 +nlvr2,nlvr2,Qwen3.8,four_ratio,2029,probability_difference,True,0.839931251790318,0.8430939226519338,0.8421468034727704,0.9089086221712976,0.9070101169548683,0.9598968776854769,0.9522852837769965,-0.22549447417259216,ge,0.96 +nlvr2,nlvr2,Qwen3.8,four_ratio,2030,log_ratio,False,0.8398739616155829,0.8430508717801537,0.8415225658319581,0.909481523918648,0.9058620937073976,0.7499283872815812,0.7526727416230179,-3.552999496459961,ge,0.75 +nlvr2,nlvr2,Qwen3.8,four_ratio,2030,probability_difference,True,0.8405614437124033,0.8431943746860875,0.843086747506637,0.9096247493554855,0.9072253713137691,0.9598968776854769,0.9598909377914903,-0.29492709040641785,ge,0.96 +nlvr2,nlvr2,Qwen3.8,gain,2026,regression,False,0.8387138355771985,0.8416086675755184,0.8416947693190787,0.9077628186765969,0.908229891655306,0.9598968776854769,0.9565903709550119,-0.3106471598148346,ge,0.96 +nlvr2,nlvr2,Qwen3.8,gain,2027,regression,False,0.8397593812661128,0.8434096290449882,0.8422759560881108,0.9074763678029218,0.9042118102891583,0.9899742194213692,0.9909593169261678,-0.5174528360366821,ge,0.99 +nlvr2,nlvr2,Qwen3.8,gain,2028,regression,False,0.8382555141793182,0.841178158857717,0.8405897969433881,0.9079060441134346,0.9044270646480591,0.9598968776854769,0.9580971514673172,-0.30899009108543396,ge,0.96 +nlvr2,nlvr2,Qwen3.8,gain,2029,regression,False,0.8393583500429677,0.8422903063787043,0.8414436392336946,0.9080492695502721,0.906436105331133,0.9598968776854769,0.958240654373251,-0.26885250210762024,ge,0.96 +nlvr2,nlvr2,Qwen3.8,gain,2030,regression,False,0.8400171870524206,0.8422544306522207,0.8415153906866614,0.9081924949871097,0.9041400588361914,0.9899742194213692,0.9887350218841932,-0.38928309082984924,ge,0.99 +nlvr2,nlvr2,Qwen3.8,route,2026,probability,False,0.8397593812661128,0.8429575948912966,0.8427351653870991,0.907046691492409,0.9026332783238861,0.7999140647378975,0.798880677333716,5.027784573030658e-05,ge,0.8 +nlvr2,nlvr2,Qwen3.8,route,2027,probability,False,0.8394299627613864,0.8425557867546819,0.8420822271651001,0.9089086221712976,0.9038530530243237,0.7099684904038958,0.7066800602712205,0.002013248624280095,ge,0.71 +nlvr2,nlvr2,Qwen3.8,route,2028,probability,False,0.8389573188198224,0.8418454473703093,0.841967424840353,0.9077628186765969,0.9040683073832245,0.8498997421942137,0.8416445433020019,0.00028116285102441907,ge,0.85 +nlvr2,nlvr2,Qwen3.8,route,2029,probability,False,0.8398023488971642,0.8431513238143074,0.8427208150965058,0.909481523918648,0.9042118102891583,0.759954167860212,0.7583411064074047,0.0012565479846671224,ge,0.76 +nlvr2,nlvr2,Qwen3.8,route,2030,probability,False,0.8398739616155829,0.8427136399512091,0.842225730071034,0.9081924949871097,0.9048575733658607,0.7499283872815812,0.7474348855564326,0.0009504365734755993,ge,0.75 +construction,rtdetr_x_fp32,MiniCPM,error,2026,probability,False,0.10798165974959693,0.11265667917538302,0.11450855730399546,0.21812181923959445,0.22911930648923418,0.7788873038516405,0.7922769640479361,0.059129804372787476,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,error,2027,probability,False,0.10705388899535563,0.11345416132823648,0.11315340055462031,0.21943098196525596,0.2291253763040382,0.7689015691868759,0.7792942743009321,0.19002777338027954,ge,0.77 +construction,rtdetr_x_fp32,MiniCPM,error,2028,probability,False,0.10857857718364279,0.11163610251969724,0.11395379812222817,0.21932024888034976,0.22564073213107894,0.7489300998573466,0.7566577896138482,0.22662235796451569,ge,0.75 +construction,rtdetr_x_fp32,MiniCPM,error,2029,probability,False,0.10382509712996263,0.11156731257861378,0.11273905726146491,0.21814890968677128,0.22990350770895734,0.7788873038516405,0.7899467376830892,0.008460866287350655,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,error,2030,probability,False,0.10703461303967125,0.11208780440303191,0.112737441980419,0.21943098196525596,0.22949232168893552,0.7689015691868759,0.7809587217043942,0.18727901577949524,ge,0.77 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2026,log_ratio,True,0.10417398823129682,0.10403198946200061,0.1039483271683989,0.2207269708303888,0.22271604145525356,0.7589158345221113,0.7699733688415447,-0.08936488628387451,ge,0.76 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2026,probability_difference,False,0.10455797932281947,0.10763255589308596,0.1118377754010868,0.2207269708303888,0.22151919702262796,0.7589158345221113,0.7683089214380826,-0.01648801565170288,ge,0.76 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2027,log_ratio,True,0.10039291708298954,0.10867828547189472,0.11387033192391746,0.21840948617358796,0.23009498853111496,0.7788873038516405,0.7892809587217043,-1.0986450910568237,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2027,probability_difference,False,0.10304587640569697,0.11123146409341321,0.11297984898228418,0.2191270620579996,0.22590669255486498,0.7888730385164051,0.7939414114513982,-0.08512895554304123,ge,0.79 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2028,log_ratio,True,0.110867343614074,0.1010151913629908,0.10126861974876791,0.2180381294788074,0.23358243652893326,0.7689015691868759,0.7806258322237017,-0.316249281167984,ge,0.77 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2028,probability_difference,False,0.1022278994548627,0.11147913524581093,0.11367374377343961,0.2193613683209817,0.23706753735104394,0.7788873038516405,0.7942743009320905,-0.0833086222410202,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2029,log_ratio,True,0.10509645695324267,0.09858743445299838,0.09880324326759393,0.21754518867203937,0.2312189038829086,0.7689015691868759,0.7809587217043942,-0.3547515273094177,ge,0.77 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2029,probability_difference,False,0.10651028970648257,0.10674970199694214,0.10863983473996808,0.2170841805238287,0.23031420481265008,0.8487874465049928,0.8538615179760319,-0.15434026718139648,ge,0.85 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2030,log_ratio,True,0.10781922601926675,0.10607949881755499,0.10838834043973575,0.2209509348306372,0.22460153595485094,0.7689015691868759,0.7736351531291611,-0.2931842505931854,ge,0.77 +construction,rtdetr_x_fp32,MiniCPM,four_difference,2030,probability_difference,False,0.10400148472959976,0.11431797470050066,0.11764170311983775,0.2193613683209817,0.23057666691849263,0.7788873038516405,0.7899467376830892,-0.17031188309192657,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2026,log_ratio,False,0.10591010867846867,0.11347960659845076,0.11371991501789111,0.21929256887137727,0.2301404923989437,0.7788873038516405,0.7902796271637816,-1.3436588048934937,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2026,probability_difference,True,0.10190966800748241,0.11328196655631823,0.11665197577374897,0.21943098196525596,0.22955418669669722,0.7788873038516405,0.7912782956058588,-0.10884908586740494,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2027,log_ratio,False,0.11125107118984306,0.0993285476071516,0.09925464014723498,0.22179252080355524,0.23149038799614485,0.7689015691868759,0.7819573901464714,-0.2739071249961853,ge,0.77 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2027,probability_difference,True,0.10085403372851058,0.11361688725214963,0.11408743030663096,0.21707572173874692,0.23467218901838918,0.7888730385164051,0.7969374167776299,-0.06965917348861694,ge,0.79 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2028,log_ratio,False,0.11164238532826157,0.09862809226873387,0.09857810065933705,0.2180381294788074,0.23000559721303715,0.7689015691868759,0.783288948069241,-0.4775036573410034,ge,0.77 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2028,probability_difference,True,0.09963952312941046,0.10495152345227951,0.1096796379008783,0.21789807345639844,0.22815985212860396,0.7788873038516405,0.7976031957390146,-0.10890354216098785,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2029,log_ratio,False,0.10534498133534209,0.11306924805448078,0.1159512188647017,0.2183127947773642,0.22973658816108528,0.7788873038516405,0.7866178428761651,-0.9724206328392029,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2029,probability_difference,True,0.10212169500654901,0.11081968938551227,0.11279806994672872,0.2162842419001542,0.2255677122416803,0.8687589158345221,0.8731691078561917,-0.09747463464736938,ge,0.87 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2030,log_ratio,False,0.11010936730135382,0.09783939001626635,0.09855972081968792,0.21751729735926367,0.23393609676645558,0.7689015691868759,0.7816245006657789,-0.37025827169418335,ge,0.77 +construction,rtdetr_x_fp32,MiniCPM,four_ratio,2030,probability_difference,True,0.10228453674116161,0.10547291811342843,0.10829981057514294,0.2168469081294761,0.23555935306461712,0.7988587731811697,0.8112516644474035,-0.1065693274140358,ge,0.8 +construction,rtdetr_x_fp32,MiniCPM,gain,2026,regression,False,0.10227600542887572,0.1133331577300786,0.1146512704435424,0.22059418705946607,0.22715438341479213,0.7589158345221113,0.7726364846870839,-0.026349209249019623,ge,0.76 +construction,rtdetr_x_fp32,MiniCPM,gain,2027,regression,False,0.11502105264565711,0.11865112377735795,0.11799666857398437,0.21822410339870008,0.22820625410619372,0.8088445078459344,0.813249001331558,-0.014458496123552322,ge,0.81 +construction,rtdetr_x_fp32,MiniCPM,gain,2028,regression,False,0.10140327488071836,0.11124410594674533,0.1153412637266712,0.22088915363632028,0.22799329390180714,0.7589158345221113,0.7733022636484687,0.0010181870311498642,ge,0.76 +construction,rtdetr_x_fp32,MiniCPM,gain,2029,regression,False,0.10631346314988693,0.12024770793777992,0.1208325202716409,0.2160644686273638,0.2303989154626947,0.8487874465049928,0.8521970705725699,-0.18803671002388,ge,0.85 +construction,rtdetr_x_fp32,MiniCPM,gain,2030,regression,False,0.10434423873185297,0.11597852679510554,0.11796780229333237,0.2181659282972604,0.2296097308251386,0.7788873038516405,0.7879494007989347,-0.05549885332584381,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,route,2026,probability,False,0.1094914144002768,0.10955210009460677,0.11203517976395429,0.2173202614379085,0.22985341505614795,0.7788873038516405,0.7912782956058588,0.07960143685340881,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,route,2027,probability,False,0.10750863454602093,0.10813482872103479,0.11247483815388529,0.22421375899802484,0.23355419176449477,0.7589158345221113,0.7726364846870839,0.1589115858078003,ge,0.76 +construction,rtdetr_x_fp32,MiniCPM,route,2028,probability,False,0.11029595989967605,0.11415672797621425,0.11543694541684765,0.22544113943695793,0.2365291826102373,0.7489300998573466,0.7579893475366178,0.14587557315826416,ge,0.75 +construction,rtdetr_x_fp32,MiniCPM,route,2029,probability,False,0.10366246683590916,0.10941972417007215,0.11249123903253086,0.21812181923959445,0.22985842654118172,0.7788873038516405,0.7916111850865513,0.04558877646923065,ge,0.78 +construction,rtdetr_x_fp32,MiniCPM,route,2030,probability,False,0.10943499028822667,0.11024970967770964,0.1153920163545464,0.22542338890164193,0.23488097945018452,0.7589158345221113,0.771970705725699,0.17386986315250397,ge,0.76 +construction,rtdetr_x_fp32,Qwen3.8,error,2026,probability,False,0.12039378593731341,0.11331040657120152,0.1157610948909677,0.26944205800380894,0.2699784556262035,0.8088445078459344,0.8152463382157124,0.0440121591091156,ge,0.81 +construction,rtdetr_x_fp32,Qwen3.8,error,2027,probability,False,0.1199614628289603,0.11991445869575061,0.12168465202843265,0.26803712803712804,0.27268636672844154,0.7788873038516405,0.7952729693741678,0.014608846046030521,ge,0.78 +construction,rtdetr_x_fp32,Qwen3.8,error,2028,probability,False,0.12177569282139204,0.11762245618432621,0.12187637338050852,0.2686150371322641,0.26886816407237546,0.7689015691868759,0.7792942743009321,0.20549169182777405,ge,0.77 +construction,rtdetr_x_fp32,Qwen3.8,error,2029,probability,False,0.12061492278324251,0.1165348638837826,0.1207885363902005,0.2686150371322641,0.2694287519691638,0.7689015691868759,0.7809587217043942,0.20376545190811157,ge,0.77 +construction,rtdetr_x_fp32,Qwen3.8,error,2030,probability,False,0.12015661202658363,0.11813198404278928,0.12197926599192044,0.2679037995789213,0.26985843727468783,0.7689015691868759,0.7812916111850865,0.19904620945453644,ge,0.77 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2026,log_ratio,True,0.11034137605622484,0.09502280825624436,0.09312195995743873,0.2699586861016554,0.26910436104030006,0.7988587731811697,0.8102529960053262,-0.08189237117767334,ge,0.8 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2026,probability_difference,False,0.11325378160549135,0.11911714184391976,0.12435106299175834,0.2699586861016554,0.2692688947733183,0.7988587731811697,0.8099201065246339,-0.0004259832203388214,ge,0.8 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2027,log_ratio,True,0.13096425070410553,0.10976185583726535,0.10823245045562856,0.27453926046943244,0.2677107689812078,0.818830242510699,0.8195739014647138,-0.17494702339172363,ge,0.82 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2027,probability_difference,False,0.1184969737010214,0.11915988751172547,0.12297109326457535,0.2738229314922339,0.26776779485961383,0.818830242510699,0.8202396804260985,-0.007923062890768051,ge,0.82 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2028,log_ratio,True,0.11221843316112176,0.10602473813026195,0.10449699179671162,0.2696894259027477,0.26993270137266917,0.7888730385164051,0.7986018641810919,-1.3913800716400146,ge,0.79 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2028,probability_difference,False,0.11510912726571669,0.10672096491235614,0.11036458545687358,0.2699586861016554,0.27017380559026777,0.7888730385164051,0.7936085219707057,-0.0023735486902296543,ge,0.79 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2029,log_ratio,True,0.10926994271756382,0.10212893130727205,0.10596226564035197,0.2700096064066652,0.2677230846635817,0.818830242510699,0.8242343541944075,-1.4155470132827759,ge,0.82 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2029,probability_difference,False,0.11098022125339418,0.11974805231661778,0.12196493903895703,0.2699586861016554,0.268624728799547,0.8088445078459344,0.8239014647137151,-0.00946833472698927,ge,0.81 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2030,log_ratio,True,0.11709328685605169,0.11238358271130114,0.11294932185723938,0.2722115247468264,0.2673676620788535,0.8388017118402282,0.8362183754993342,-0.5787290930747986,ge,0.84 +construction,rtdetr_x_fp32,Qwen3.8,four_difference,2030,probability_difference,False,0.12053288649557378,0.11823746710794934,0.12295913434654906,0.27318166973339386,0.26663932165812865,0.8288159771754636,0.8258988015978695,-0.015626760199666023,ge,0.83 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2026,log_ratio,False,0.12820312351577445,0.09902454720960482,0.09900601359401771,0.2693115545451994,0.27051015600234996,0.7788873038516405,0.7909454061251664,-0.9575721621513367,ge,0.78 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2026,probability_difference,True,0.10371633787367057,0.11486852991210213,0.11545107659649317,0.2720493854628622,0.26846091394620314,0.8288159771754636,0.8288948069241012,-0.02662026882171631,ge,0.83 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2027,log_ratio,False,0.13096425070410553,0.10976185583726535,0.10823245045562856,0.27453926046943244,0.2677107689812078,0.818830242510699,0.8195739014647138,-0.17494702339172363,ge,0.82 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2027,probability_difference,True,0.1184969737010214,0.11915988751172547,0.12297109326457535,0.2738229314922339,0.26776779485961383,0.818830242510699,0.8202396804260985,-0.007923062890768051,ge,0.82 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2028,log_ratio,False,0.12490734525868222,0.09916794146260717,0.10077312127534066,0.2697705209900332,0.2684788215570028,0.818830242510699,0.8212383488681758,-1.3686586618423462,ge,0.82 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2028,probability_difference,True,0.10654845382696372,0.11768072431881688,0.1198051991826763,0.27020636527215475,0.26884986632848556,0.8587731811697575,0.8641810918774967,-0.0331578254699707,ge,0.86 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2029,log_ratio,False,0.12596242054282145,0.1129213349435225,0.11172901112275058,0.2731513743708866,0.26778655496410875,0.8288159771754636,0.8249001331557922,0.05315309762954712,ge,0.83 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2029,probability_difference,True,0.10851950827289983,0.11795652459841306,0.11887033683261619,0.2731513743708866,0.2675134251843376,0.8288159771754636,0.8272303595206392,0.0009714104235172272,ge,0.83 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2030,log_ratio,False,0.12041981142411713,0.099844965927226,0.10117823891399735,0.2693115545451994,0.2703438042302786,0.7788873038516405,0.7899467376830892,-0.8490487337112427,ge,0.78 +construction,rtdetr_x_fp32,Qwen3.8,four_ratio,2030,probability_difference,True,0.10424930254708085,0.11704944317152478,0.12051950836714966,0.27005354160526573,0.2690915793229438,0.818830242510699,0.8235685752330226,-0.02198227494955063,ge,0.82 +construction,rtdetr_x_fp32,Qwen3.8,gain,2026,regression,False,0.11461043447688607,0.10736279456956735,0.108425186515875,0.267925415931338,0.27082164595102265,0.7788873038516405,0.7926098535286284,0.18016111850738525,ge,0.78 +construction,rtdetr_x_fp32,Qwen3.8,gain,2027,regression,False,0.11340701932374346,0.11664880682569982,0.12127650863040235,0.27174992117664104,0.26793884750142727,0.8288159771754636,0.8249001331557922,-0.04774220287799835,ge,0.83 +construction,rtdetr_x_fp32,Qwen3.8,gain,2028,regression,False,0.12040957038651784,0.12294365489488415,0.12179059597820625,0.2695166476972549,0.2719292081853759,0.7988587731811697,0.7989347536617842,0.08590328693389893,ge,0.8 +construction,rtdetr_x_fp32,Qwen3.8,gain,2029,regression,False,0.10908574026810577,0.11695457110058574,0.12020874903351958,0.2704520130990719,0.26852697407753484,0.818830242510699,0.8212383488681758,-0.021960698068141937,ge,0.82 +construction,rtdetr_x_fp32,Qwen3.8,gain,2030,regression,False,0.10673778415869332,0.11357903282934012,0.11820678089270939,0.27274440570380254,0.2680022169416302,0.8288159771754636,0.8378828229027963,-0.016868319362401962,ge,0.83 +construction,rtdetr_x_fp32,Qwen3.8,route,2026,probability,False,0.11588539743356142,0.11588329698868416,0.11759697784415424,0.26845414221561015,0.2721126205953094,0.7788873038516405,0.7929427430093209,0.07033984363079071,ge,0.78 +construction,rtdetr_x_fp32,Qwen3.8,route,2027,probability,False,0.11614938507087924,0.11777232314428779,0.12005583116546811,0.2685446644347383,0.2722277873783231,0.7689015691868759,0.7802929427430093,0.13405601680278778,ge,0.77 +construction,rtdetr_x_fp32,Qwen3.8,route,2028,probability,False,0.1226251153324313,0.12091240794202765,0.12345536042150987,0.267925415931338,0.27012282177874536,0.7689015691868759,0.7822902796271638,0.10965022444725037,ge,0.77 +construction,rtdetr_x_fp32,Qwen3.8,route,2029,probability,False,0.11554597977650932,0.11803881530628267,0.1209455883754125,0.26803712803712804,0.27385426168681254,0.7788873038516405,0.7899467376830892,0.014448748901486397,ge,0.78 +construction,rtdetr_x_fp32,Qwen3.8,route,2030,probability,False,0.11389842157783937,0.11530039496390766,0.11917490059335592,0.2693115545451994,0.2719188933134367,0.7689015691868759,0.7796271637816246,0.13787509500980377,ge,0.77 +nlvr2,vilt,MiniCPM,error,2026,probability,False,0.7926382125465483,0.799138982564397,0.8006242376408123,0.8984531652821541,0.9066513596900337,0.9899742194213692,0.9853627035947478,2.4283915536216227e-06,ge,0.99 +nlvr2,vilt,MiniCPM,error,2027,probability,False,0.7919793755370954,0.7990815814020235,0.8008753677261964,0.8990260670295045,0.9069383655019014,0.9699226582641077,0.9651287938580756,4.422461643116549e-06,ge,0.97 +nlvr2,vilt,MiniCPM,error,2028,probability,False,0.7929962761386422,0.7981201119322666,0.8001148023247471,0.8990260670295045,0.9071536198608022,0.9899742194213692,0.9902418023964985,4.1207282689917306e-17,ge,0.99 +nlvr2,vilt,MiniCPM,error,2029,probability,False,0.7928530507018046,0.7997560450599126,0.8012556504269212,0.899169292466342,0.9071536198608022,0.9899742194213692,0.9881610102604578,2.1061083316453733e-05,ge,0.99 +nlvr2,vilt,MiniCPM,error,2030,probability,False,0.7924376969349757,0.7990241802396498,0.8009112434526799,0.8987396161558293,0.9070818684078352,0.9899742194213692,0.9878022529956232,2.7660298655973747e-06,ge,0.99 +nlvr2,vilt,MiniCPM,four_difference,2026,log_ratio,True,0.7941420796333428,0.801026045777427,0.8017866111788764,0.8985963907189917,0.9067948625959675,0.9799484388427385,0.9812728707756332,-7.513500213623047,ge,0.98 +nlvr2,vilt,MiniCPM,four_difference,2026,probability_difference,False,0.7935118877112576,0.8008466671450097,0.8019659898112936,0.897450587224291,0.9056468393484968,0.9899742194213692,0.9898830451316639,-0.15563204884529114,ge,0.99 +nlvr2,vilt,MiniCPM,four_difference,2027,log_ratio,True,0.7934689200802062,0.8007820908373395,0.801750735452393,0.8993125179031796,0.9070818684078352,0.9899742194213692,0.9895960393197962,-4.813183784484863,ge,0.99 +nlvr2,vilt,MiniCPM,four_difference,2027,probability_difference,False,0.7938126611286165,0.8001578531965273,0.8012915261534046,0.8983099398453165,0.9057185908014638,0.9899742194213692,0.9882327617134247,-0.1142226904630661,ge,0.99 +nlvr2,vilt,MiniCPM,four_difference,2028,log_ratio,True,0.7933400171870524,0.7994905646839349,0.8016144076917557,0.8985963907189917,0.9071536198608022,0.9899742194213692,0.989811293678697,-10.380939483642578,ge,0.99 +nlvr2,vilt,MiniCPM,four_difference,2028,probability_difference,False,0.7934689200802063,0.7997416947693191,0.8014493793499319,0.8977370380979662,0.906364353878166,0.9899742194213692,0.9908875654732008,-0.2333981692790985,ge,0.99 +nlvr2,vilt,MiniCPM,four_difference,2029,log_ratio,True,0.7941277570896591,0.799232259453254,0.8014063284781516,0.8988828415926668,0.9067231111430006,0.9799484388427385,0.978689818468824,-11.480510711669922,ge,0.98 +nlvr2,vilt,MiniCPM,four_difference,2029,probability_difference,False,0.7938269836723002,0.8003946329913181,0.8014996053670087,0.8973073617874534,0.9061490995192653,0.9899742194213692,0.9906005596613331,-0.18295258283615112,ge,0.99 +nlvr2,vilt,MiniCPM,four_difference,2030,log_ratio,True,0.7938269836723002,0.8004879098801752,0.8016718088541294,0.8987396161558293,0.9072253713137691,0.9899742194213692,0.9913180741910024,-6.187008857727051,ge,0.99 +nlvr2,vilt,MiniCPM,four_difference,2030,probability_difference,False,0.7940274992838728,0.8000430508717802,0.8016502834182392,0.8971641363506159,0.9070101169548683,0.9899742194213692,0.9908158140202339,-0.15542080998420715,ge,0.99 +nlvr2,vilt,MiniCPM,four_ratio,2026,log_ratio,False,0.7941420796333428,0.801026045777427,0.8017866111788764,0.8985963907189917,0.9067948625959675,0.9799484388427385,0.9812728707756332,-7.513500213623047,ge,0.98 +nlvr2,vilt,MiniCPM,four_ratio,2026,probability_difference,True,0.7935118877112576,0.8008466671450097,0.8019659898112936,0.897450587224291,0.9056468393484968,0.9899742194213692,0.9898830451316639,-0.15563204884529114,ge,0.99 +nlvr2,vilt,MiniCPM,four_ratio,2027,log_ratio,False,0.7937124033228302,0.7994116380856712,0.8002726555212742,0.8988828415926668,0.9068666140489345,0.9799484388427385,0.979550835904427,-18.18950843811035,ge,0.98 +nlvr2,vilt,MiniCPM,four_ratio,2027,probability_difference,True,0.7930249212260096,0.7987730501542657,0.7994833895386382,0.8968776854769407,0.9062926024251992,0.9899742194213692,0.9906005596613331,-0.35407698154449463,ge,0.99 +nlvr2,vilt,MiniCPM,four_ratio,2028,log_ratio,False,0.7939272414780865,0.8005237856066586,0.8021955944607878,0.8990260670295045,0.9071536198608022,0.9899742194213692,0.98973954222573,-7.835921287536621,ge,0.99 +nlvr2,vilt,MiniCPM,four_ratio,2028,probability_difference,True,0.7929389859639072,0.7995407907010117,0.8009758197603501,0.8965912346032655,0.9043553131950922,0.9799484388427385,0.9812728707756332,-0.16742494702339172,ge,0.98 +nlvr2,vilt,MiniCPM,four_ratio,2029,log_ratio,False,0.7948152391864796,0.8013417521704815,0.8026691540503694,0.899169292466342,0.9074406256726699,0.9899742194213692,0.9884480160723255,-11.320954322814941,ge,0.99 +nlvr2,vilt,MiniCPM,four_ratio,2029,probability_difference,True,0.7936694356917788,0.8000717514529669,0.8014493793499319,0.8970209109137783,0.9059338451603645,0.9899742194213692,0.9914615770969362,-0.22513195872306824,ge,0.99 +nlvr2,vilt,MiniCPM,four_ratio,2030,log_ratio,False,0.793841306215984,0.8013274018798882,0.8021884193154911,0.8990260670295045,0.9069383655019014,0.9799484388427385,0.9822773911171702,-9.19042682647705,ge,0.98 +nlvr2,vilt,MiniCPM,four_ratio,2030,probability_difference,True,0.7934402749928386,0.8007390399655594,0.8019014135036234,0.8984531652821541,0.9069383655019014,0.9899742194213692,0.9911028198321016,-0.1698741316795349,ge,0.99 +nlvr2,vilt,MiniCPM,gain,2026,regression,False,0.7930249212260098,0.7991318074191003,0.8004305087178015,0.8973073617874534,0.9052880820836622,0.9899742194213692,0.9904570567553993,-0.15253251791000366,ge,0.99 +nlvr2,vilt,MiniCPM,gain,2027,regression,False,0.7927527928960183,0.7987945755901558,0.8002296046494941,0.8970209109137783,0.9053598335366291,0.9899742194213692,0.9872999928248547,-0.11346326023340225,ge,0.99 +nlvr2,vilt,MiniCPM,gain,2028,regression,False,0.7936551131480951,0.8001148023247471,0.8008681925808998,0.8973073617874534,0.9062926024251992,0.9899742194213692,0.9911745712850685,-0.1503293514251709,ge,0.99 +nlvr2,vilt,MiniCPM,gain,2029,regression,False,0.7937410484101977,0.7983497165817608,0.8001363277606371,0.8967344600401032,0.9055750878955299,0.9899742194213692,0.9921073401736386,-0.2225644588470459,ge,0.99 +nlvr2,vilt,MiniCPM,gain,2030,regression,False,0.7932111142938986,0.7998062710769894,0.8010045203415369,0.8967344600401032,0.9052163306306953,0.9899742194213692,0.9913898256439693,-0.11570084095001221,ge,0.99 +nlvr2,vilt,MiniCPM,route,2026,probability,False,0.7929676310512747,0.799418813230968,0.8005022601707685,0.8990260670295045,0.906436105331133,0.9899742194213692,0.9868694841070532,2.094150131082273e-10,ge,0.99 +nlvr2,vilt,MiniCPM,route,2027,probability,False,0.7931395015754799,0.800373107555428,0.8020090406830739,0.8987396161558293,0.9056468393484968,0.9699226582641077,0.9662768171055464,1.948114913830068e-08,ge,0.97 +nlvr2,vilt,MiniCPM,route,2028,probability,False,0.7930535663133772,0.8002439549400876,0.8018511874865466,0.8990260670295045,0.9070101169548683,0.9899742194213692,0.9883045131663917,3.0138309625726833e-08,ge,0.99 +nlvr2,vilt,MiniCPM,route,2029,probability,False,0.7930105986823259,0.7988950276243094,0.800559661333142,0.8987396161558293,0.9067948625959675,0.9899742194213692,0.9900982994905647,5.493770913744811e-07,ge,0.99 +nlvr2,vilt,MiniCPM,route,2030,probability,False,0.7932540819249498,0.7988161010260458,0.8008251417091197,0.8988828415926668,0.9066513596900337,0.9899742194213692,0.987945755901557,2.1658902369381394e-06,ge,0.99 +nlvr2,vilt,Qwen3.8,error,2026,probability,False,0.7942423374391292,0.7992824854703309,0.8009471191791634,0.9040389573188198,0.9001937289230106,0.9899742194213692,0.9853627035947478,2.4283915536216227e-06,ge,0.99 +nlvr2,vilt,Qwen3.8,error,2027,probability,False,0.7938699513033515,0.7992824854703309,0.8009829949056468,0.904325408192495,0.9005524861878453,0.9899742194213692,0.9904570567553993,2.1695223040296696e-06,ge,0.99 +nlvr2,vilt,Qwen3.8,error,2028,probability,False,0.7936264680607276,0.7988950276243094,0.8006098873502188,0.9047550845030078,0.9003372318289445,0.9799484388427385,0.9822773911171702,0.00011319875193294138,ge,0.98 +nlvr2,vilt,Qwen3.8,error,2029,probability,False,0.7944714981380694,0.800279830666571,0.8015785319652723,0.9046118590661701,0.9002654803759776,0.9699226582641077,0.9689316208653225,0.00015893796808086336,ge,0.97 +nlvr2,vilt,Qwen3.8,error,2030,probability,False,0.7940991120022916,0.7991174571285069,0.8009040683073833,0.9050415353766829,0.9004089832819114,0.9899742194213692,0.9878022529956232,2.7660298655973747e-06,ge,0.99 +nlvr2,vilt,Qwen3.8,four_difference,2026,log_ratio,True,0.7971068461758808,0.8012484752816246,0.8016287579823491,0.9046118590661701,0.9002654803759776,0.9899742194213692,0.9915333285499032,-18.6831111907959,ge,0.99 +nlvr2,vilt,Qwen3.8,four_difference,2026,probability_difference,False,0.7970925236321971,0.801284351008108,0.8021094927172275,0.9054712116871957,0.9028485326827869,0.9899742194213692,0.9898830451316639,-0.7896829843521118,ge,0.99 +nlvr2,vilt,Qwen3.8,four_difference,2027,log_ratio,True,0.7958034947006588,0.8028772332639736,0.8027193800674464,0.9047550845030078,0.8996914687522423,0.9799484388427385,0.977757049580254,-11.819299697875977,ge,0.98 +nlvr2,vilt,Qwen3.8,four_difference,2027,probability_difference,False,0.7967487825837869,0.8018655377771401,0.8026476286144795,0.9067602406187338,0.9034942957594891,0.9899742194213692,0.9890937791490277,-0.6514462232589722,ge,0.99 +nlvr2,vilt,Qwen3.8,four_difference,2028,log_ratio,True,0.796089945574334,0.8021381932984143,0.8035301714859726,0.9046118590661701,0.9000502260170768,0.9899742194213692,0.9908158140202339,-12.309871673583984,ge,0.99 +nlvr2,vilt,Qwen3.8,four_difference,2028,probability_difference,False,0.7958178172443426,0.8010116954868337,0.8027839563751166,0.9071899169292467,0.9034225443065222,0.9899742194213692,0.9881610102604578,-0.6827153563499451,ge,0.99 +nlvr2,vilt,Qwen3.8,four_difference,2029,log_ratio,True,0.7963477513606416,0.8034655951783025,0.8040109062208509,0.9046118590661701,0.8997632202052092,0.939988541965053,0.9383655019014135,-9.78353500366211,ge,0.94 +nlvr2,vilt,Qwen3.8,four_difference,2029,probability_difference,False,0.7959896877685476,0.802425199110282,0.8033292674176653,0.9064737897450588,0.9037813015713568,0.9799484388427385,0.9776852981272871,-0.4852448105812073,ge,0.98 +nlvr2,vilt,Qwen3.8,four_difference,2030,log_ratio,True,0.7963334288169579,0.802145368443711,0.8023606228026117,0.905327986250358,0.9004089832819114,0.9699226582641077,0.9715146731721317,-12.531599044799805,ge,0.97 +nlvr2,vilt,Qwen3.8,four_difference,2030,probability_difference,False,0.7967201374964193,0.8007749156920428,0.8017794360335797,0.9071899169292467,0.901772260888283,0.9899742194213692,0.9911745712850685,-0.5994459390640259,ge,0.99 +nlvr2,vilt,Qwen3.8,four_ratio,2026,log_ratio,False,0.7971927814379833,0.801700509435316,0.8024682499820621,0.9051847608135205,0.9002654803759776,0.9799484388427385,0.9817033794934348,-12.83373737335205,ge,0.98 +nlvr2,vilt,Qwen3.8,four_ratio,2026,probability_difference,True,0.7964623317101117,0.8008968931620866,0.8023247470761282,0.9060441134345459,0.9038530530243237,0.9899742194213692,0.9901700509435316,-0.49320393800735474,ge,0.99 +nlvr2,vilt,Qwen3.8,four_ratio,2027,log_ratio,False,0.7963191062732742,0.803896103896104,0.8042620363062352,0.9047550845030078,0.9006242376408122,0.9899742194213692,0.9911745712850685,-8.281852722167969,ge,0.99 +nlvr2,vilt,Qwen3.8,four_ratio,2027,probability_difference,True,0.7959610426811801,0.8023749730932052,0.8035229963406759,0.9060441134345459,0.9032790414005883,0.9899742194213692,0.9892372820549616,-0.2945282459259033,ge,0.99 +nlvr2,vilt,Qwen3.8,four_ratio,2028,log_ratio,False,0.7962331710111715,0.8024180239649854,0.8040467819473345,0.905327986250358,0.9004807347348783,0.9899742194213692,0.9923943459855062,-9.808849334716797,ge,0.99 +nlvr2,vilt,Qwen3.8,four_ratio,2028,probability_difference,True,0.7948868519048983,0.8011336729568774,0.8023032216402382,0.9069034660555715,0.9025615268709192,0.9899742194213692,0.9868694841070532,-0.6343045234680176,ge,0.99 +nlvr2,vilt,Qwen3.8,four_ratio,2029,log_ratio,False,0.7966914924090518,0.8019659898112937,0.8035229963406759,0.9048983099398453,0.9004807347348783,0.9899742194213692,0.9918203343617709,-12.624320983886719,ge,0.99 +nlvr2,vilt,Qwen3.8,four_ratio,2029,probability_difference,True,0.7959037525064452,0.8012915261534046,0.8029418095716438,0.9061873388713836,0.902705029776853,0.9899742194213692,0.9886632704312263,-0.3853158950805664,ge,0.99 +nlvr2,vilt,Qwen3.8,four_ratio,2030,log_ratio,False,0.7965912346032655,0.8002080792136042,0.8013417521704814,0.905327986250358,0.9004089832819114,0.9598968776854769,0.9615412212097295,-13.448873519897461,ge,0.96 +nlvr2,vilt,Qwen3.8,four_ratio,2030,probability_difference,True,0.7953308507590948,0.7990815814020235,0.8005955370596255,0.9066170151818963,0.902705029776853,0.9899742194213692,0.9900265480375977,-0.6531928777694702,ge,0.99 +nlvr2,vilt,Qwen3.8,gain,2026,regression,False,0.7956029790890862,0.8007820908373395,0.8018296620506565,0.9056144371240332,0.9025615268709192,0.9799484388427385,0.9791920786395925,-0.33801835775375366,ge,0.98 +nlvr2,vilt,Qwen3.8,gain,2027,regression,False,0.7954597536522485,0.8019875152471838,0.8030279113152041,0.9066170151818963,0.9013417521704814,0.9799484388427385,0.9799095931692616,-0.484546959400177,ge,0.98 +nlvr2,vilt,Qwen3.8,gain,2028,regression,False,0.7952305929533086,0.8020807921360408,0.802590227452106,0.9059008879977084,0.9036377986654229,0.9899742194213692,0.9882327617134247,-0.6079308390617371,ge,0.99 +nlvr2,vilt,Qwen3.8,gain,2029,regression,False,0.7954454311085649,0.8019659898112937,0.8030494367510943,0.9069034660555715,0.9042118102891583,0.9799484388427385,0.9774700437683863,-0.34608492255210876,ge,0.98 +nlvr2,vilt,Qwen3.8,gain,2030,regression,False,0.7960040103122314,0.8004950850254718,0.8019157637942169,0.9059008879977084,0.9023462725120184,0.9498710971068461,0.948051948051948,-0.10941234230995178,ge,0.95 +nlvr2,vilt,Qwen3.8,route,2026,probability,False,0.7940561443712404,0.7990241802396499,0.8008538422903064,0.9048983099398453,0.9005524861878453,0.9899742194213692,0.9894525364138623,3.1904770025903417e-07,ge,0.99 +nlvr2,vilt,Qwen3.8,route,2027,probability,False,0.7945001432254368,0.7991748582908804,0.8008466671450097,0.904325408192495,0.9001937289230106,0.9899742194213692,0.9899547965846308,9.3228453579286e-07,ge,0.99 +nlvr2,vilt,Qwen3.8,route,2028,probability,False,0.7942996276138642,0.8007605654014495,0.8013130515892947,0.9051847608135205,0.9004807347348783,0.9899742194213692,0.9878022529956232,1.9875805951153856e-19,ge,0.99 +nlvr2,vilt,Qwen3.8,route,2029,probability,False,0.7941993698080779,0.7989811293678698,0.8007677405467462,0.9051847608135205,0.9004807347348783,0.9899742194213692,0.988878524790127,0.00021692014706786722,ge,0.99 +nlvr2,vilt,Qwen3.8,route,2030,probability,False,0.7941277570896591,0.799512090119825,0.8006601133672957,0.9050415353766829,0.9001937289230106,0.9899742194213692,0.9873717442778216,1.2126680952809221e-12,ge,0.99 +construction,yolo26x,MiniCPM,error,2026,probability,False,0.22030583243822807,0.2092676631629135,0.20832058587154184,0.28676728360055675,0.2513295188283841,0.4992867332382311,0.4890146471371505,0.022617846727371216,ge,0.5 +construction,yolo26x,MiniCPM,error,2027,probability,False,0.22724739896667534,0.20991613366682121,0.20803919105935909,0.28722189653104074,0.2650749805378385,0.4593437945791726,0.4567243675099867,0.12995079159736633,ge,0.46 +construction,yolo26x,MiniCPM,error,2028,probability,False,0.22606176068978506,0.21596039844107484,0.21492808643434902,0.28731640168932066,0.25479259910760677,0.4893009985734665,0.4743675099866844,0.037664178758859634,ge,0.49 +construction,yolo26x,MiniCPM,error,2029,probability,False,0.22875829295123679,0.21438072186576557,0.21311437189137622,0.28722189653104074,0.2643695414493677,0.4593437945791726,0.4560585885486019,0.1535213738679886,ge,0.46 +construction,yolo26x,MiniCPM,error,2030,probability,False,0.22950842341984,0.2179554205387229,0.21540715140274938,0.28722189653104074,0.2640405346143402,0.4593437945791726,0.45539280958721706,0.1269768625497818,ge,0.46 +construction,yolo26x,MiniCPM,four_difference,2026,log_ratio,True,0.22951439243789146,0.21732146019959078,0.21717681664243893,0.28996084144947726,0.26673201439315736,0.4094151212553495,0.414780292942743,-0.6827399134635925,ge,0.41 +construction,yolo26x,MiniCPM,four_difference,2026,probability_difference,False,0.23090631810158493,0.21784441318843004,0.21783657212241375,0.2828447999106682,0.26215437873972275,0.4693295292439372,0.4663781624500666,-0.10248661041259766,ge,0.47 +construction,yolo26x,MiniCPM,four_difference,2027,log_ratio,True,0.2258579190074036,0.21236247216777693,0.21151986488139016,0.28892789420049103,0.2594174069021474,0.44935805991440797,0.4400798934753662,-0.9231481552124023,ge,0.45 +construction,yolo26x,MiniCPM,four_difference,2027,probability_difference,False,0.22700398036250496,0.21267365992667417,0.21242061512355645,0.2848598219459613,0.2635851498163025,0.42938659058487877,0.42809587217043943,-0.06325958669185638,ge,0.43 +construction,yolo26x,MiniCPM,four_difference,2028,log_ratio,True,0.22966955071522963,0.2164794985024573,0.21581980585771837,0.287614783141149,0.26515528472125294,0.43937232524964337,0.43142476697736354,-0.5946990847587585,ge,0.44 +construction,yolo26x,MiniCPM,four_difference,2028,probability_difference,False,0.23231046411106632,0.21723184089928244,0.21910381550319696,0.28728969883242916,0.26237361710647344,0.4693295292439372,0.46238348868175766,-0.08103815466165543,ge,0.47 +construction,yolo26x,MiniCPM,four_difference,2029,log_ratio,True,0.2279013295273578,0.21854747687650594,0.21549421736895213,0.287054322169918,0.26025754776915677,0.4593437945791726,0.4543941411451398,-1.810979962348938,ge,0.46 +construction,yolo26x,MiniCPM,four_difference,2029,probability_difference,False,0.2323274089517411,0.21811828317375476,0.2168719686239992,0.2807737471705083,0.255747541850064,0.4593437945791726,0.4580559254327563,-0.10648070275783539,ge,0.46 +construction,yolo26x,MiniCPM,four_difference,2030,log_ratio,True,0.22810730263174128,0.21649743756080791,0.215649842402169,0.28713951489686784,0.2556066597365839,0.4893009985734665,0.4770306258322237,-2.1627044677734375,ge,0.49 +construction,yolo26x,MiniCPM,four_difference,2030,probability_difference,False,0.23043274239225123,0.2186681430858416,0.2146900227160903,0.28424105629679847,0.25305307441491287,0.5092724679029957,0.5076564580559254,-0.1446637064218521,ge,0.51 +construction,yolo26x,MiniCPM,four_ratio,2026,log_ratio,False,0.22951439243789146,0.21732146019959078,0.21717681664243893,0.28996084144947726,0.26673201439315736,0.4094151212553495,0.414780292942743,-0.6827399134635925,ge,0.41 +construction,yolo26x,MiniCPM,four_ratio,2026,probability_difference,True,0.23090631810158493,0.21784441318843004,0.21783657212241375,0.2828447999106682,0.26215437873972275,0.4693295292439372,0.4663781624500666,-0.10248661041259766,ge,0.47 +construction,yolo26x,MiniCPM,four_ratio,2027,log_ratio,False,0.2258579190074036,0.21236247216777693,0.21151986488139016,0.28892789420049103,0.2594174069021474,0.44935805991440797,0.4400798934753662,-0.9231481552124023,ge,0.45 +construction,yolo26x,MiniCPM,four_ratio,2027,probability_difference,True,0.22700398036250496,0.21267365992667417,0.21242061512355645,0.2848598219459613,0.2635851498163025,0.42938659058487877,0.42809587217043943,-0.06325958669185638,ge,0.43 +construction,yolo26x,MiniCPM,four_ratio,2028,log_ratio,False,0.22966955071522963,0.2164794985024573,0.21581980585771837,0.287614783141149,0.26515528472125294,0.43937232524964337,0.43142476697736354,-0.5946990847587585,ge,0.44 +construction,yolo26x,MiniCPM,four_ratio,2028,probability_difference,True,0.23231046411106632,0.21723184089928244,0.21910381550319696,0.28728969883242916,0.26237361710647344,0.4693295292439372,0.46238348868175766,-0.08103815466165543,ge,0.47 +construction,yolo26x,MiniCPM,four_ratio,2029,log_ratio,False,0.22817906648580283,0.2133793958566942,0.21215987664343575,0.28971423275273056,0.2604612003548174,0.44935805991440797,0.440745672436751,-0.9343714714050293,ge,0.45 +construction,yolo26x,MiniCPM,four_ratio,2029,probability_difference,True,0.230615206477237,0.21352954914864786,0.21365252195081794,0.284704737193425,0.25535310475057427,0.4593437945791726,0.45572569906790944,-0.07427139580249786,ge,0.46 +construction,yolo26x,MiniCPM,four_ratio,2030,log_ratio,False,0.22810730263174128,0.21649743756080791,0.215649842402169,0.28713951489686784,0.2556066597365839,0.4893009985734665,0.4770306258322237,-2.1627044677734375,ge,0.49 +construction,yolo26x,MiniCPM,four_ratio,2030,probability_difference,True,0.23043274239225123,0.2186681430858416,0.2146900227160903,0.28424105629679847,0.25305307441491287,0.5092724679029957,0.5076564580559254,-0.1446637064218521,ge,0.51 +construction,yolo26x,MiniCPM,gain,2026,regression,False,0.2307441700914365,0.21705883184489042,0.21677467948808923,0.2853430389496839,0.2587616184765472,0.42938659058487877,0.42543275632490013,-0.07304845750331879,ge,0.43 +construction,yolo26x,MiniCPM,gain,2027,regression,False,0.22383550615447997,0.2136303916366499,0.21445551093433052,0.2915794573643411,0.25826370096754736,0.43937232524964337,0.4344207723035952,-0.05811157077550888,ge,0.44 +construction,yolo26x,MiniCPM,gain,2028,regression,False,0.23969954747629152,0.21710478884039042,0.21935789642951004,0.2883615567439097,0.2663681988649724,0.32952924393723254,0.3358854860186418,-0.04107583314180374,ge,0.33 +construction,yolo26x,MiniCPM,gain,2029,regression,False,0.2248049248123454,0.21098351695323345,0.21058446577560175,0.2892731967187423,0.25650818370892253,0.43937232524964337,0.4380825565912117,-0.05632615461945534,ge,0.44 +construction,yolo26x,MiniCPM,gain,2030,regression,False,0.23708812988748706,0.21610899354428587,0.21648686920498958,0.28995185909184523,0.25516764615746745,0.4793152639087018,0.481358189081225,-0.112443707883358,ge,0.48 +construction,yolo26x,MiniCPM,route,2026,probability,False,0.23449286127969596,0.2133840954400366,0.21490540685523601,0.2956914061777062,0.26411718033799025,0.369472182596291,0.3801597869507324,0.1022382527589798,ge,0.37 +construction,yolo26x,MiniCPM,route,2027,probability,False,0.23164013282647233,0.21436902744290343,0.21446348688810374,0.3068150571279571,0.262497753382933,0.3195435092724679,0.3229027962716378,0.15746349096298218,ge,0.32 +construction,yolo26x,MiniCPM,route,2028,probability,False,0.2355746599794164,0.21514552017118374,0.21491922219527276,0.30379299504010265,0.25450082952315023,0.289586305278174,0.28894806924101196,0.18077737092971802,ge,0.29 +construction,yolo26x,MiniCPM,route,2029,probability,False,0.23433985666483198,0.21226651990380008,0.21305312007294325,0.30957519440086917,0.26093203880008864,0.3195435092724679,0.3189081225033289,0.17545486986637115,ge,0.32 +construction,yolo26x,MiniCPM,route,2030,probability,False,0.2365795588924405,0.2193396178341517,0.2195217177763924,0.2990979581086944,0.26220904868518,0.39942938659058486,0.3981358189081225,0.11110042035579681,ge,0.4 +construction,yolo26x,Qwen3.8,error,2026,probability,False,0.2340861181383704,0.21559995562093154,0.2108414220432617,0.33055989058265917,0.29897628913217833,0.4992867332382311,0.4833555259653795,0.03377731144428253,ge,0.5 +construction,yolo26x,Qwen3.8,error,2027,probability,False,0.2435166042185849,0.22119934978174002,0.21816610219714874,0.3276728380176656,0.29593471190959275,0.43937232524964337,0.4374167776298269,0.18083004653453827,ge,0.44 +construction,yolo26x,Qwen3.8,error,2028,probability,False,0.23458946846108567,0.21150812074024367,0.20949679565053572,0.3317914176762552,0.2985763967138555,0.4992867332382311,0.4890146471371505,0.027080882340669632,ge,0.5 +construction,yolo26x,Qwen3.8,error,2029,probability,False,0.23992871790704787,0.21758153593591198,0.21545589080880334,0.33093016797323793,0.299588361311237,0.4893009985734665,0.47569906790945404,0.02594057470560074,ge,0.49 +construction,yolo26x,Qwen3.8,error,2030,probability,False,0.23897936431113018,0.2154585401290291,0.212111555940419,0.3294104652953028,0.29744550498078726,0.4992867332382311,0.48934753661784286,0.026845572516322136,ge,0.5 +construction,yolo26x,Qwen3.8,four_difference,2026,log_ratio,True,0.22956812850068237,0.2123814905577035,0.21268061165082322,0.3281285807463818,0.2994456979113117,0.44935805991440797,0.4484021304926764,0.10543298721313477,ge,0.45 +construction,yolo26x,Qwen3.8,four_difference,2026,probability_difference,False,0.2368629762797031,0.21551128655882565,0.21401065405899017,0.3281285807463818,0.2992560160564012,0.44935805991440797,0.4477363515312916,0.009329468011856079,ge,0.45 +construction,yolo26x,Qwen3.8,four_difference,2027,log_ratio,True,0.22912272949022852,0.2128072844063923,0.21202893742710108,0.32788133528265107,0.2995124307318383,0.44935805991440797,0.4434087882822903,0.2047330141067505,ge,0.45 +construction,yolo26x,Qwen3.8,four_difference,2027,probability_difference,False,0.23572609512743017,0.21328856707004673,0.2115182013715633,0.3270447636073488,0.2994603011846414,0.44935805991440797,0.4444074567243675,0.013585388660430908,ge,0.45 +construction,yolo26x,Qwen3.8,four_difference,2028,log_ratio,True,0.22313070157132972,0.21051323403313987,0.21403391931992122,0.3306060705009355,0.29613258844874296,0.5392296718972895,0.5322902796271638,-3.7614707946777344,ge,0.54 +construction,yolo26x,Qwen3.8,four_difference,2028,probability_difference,False,0.24091708026299674,0.21403453268118733,0.2138369411714542,0.32187620296097463,0.2952522823078242,0.4094151212553495,0.41344873501997337,0.039804454892873764,ge,0.41 +construction,yolo26x,Qwen3.8,four_difference,2029,log_ratio,True,0.22457887533837526,0.212148758894345,0.21002872666831796,0.3290740844772266,0.295205827629145,0.42938659058487877,0.43608521970705727,0.4606260657310486,ge,0.43 +construction,yolo26x,Qwen3.8,four_difference,2029,probability_difference,False,0.237682914725911,0.22109619443245102,0.21829545748579043,0.33167859760757346,0.2966589960709296,0.4094151212553495,0.41844207723035953,0.0639844611287117,ge,0.41 +construction,yolo26x,Qwen3.8,four_difference,2030,log_ratio,True,0.22569949498210512,0.20900398995848302,0.2114406454856609,0.3254578754578754,0.3018106075277844,0.43937232524964337,0.44107856191744343,0.07300722599029541,ge,0.44 +construction,yolo26x,Qwen3.8,four_difference,2030,probability_difference,False,0.2435262081932218,0.2120332602435602,0.21356740664138002,0.32614453827508016,0.3028971299499408,0.43937232524964337,0.44107856191744343,0.0036868713796138763,ge,0.44 +construction,yolo26x,Qwen3.8,four_ratio,2026,log_ratio,False,0.22956812850068237,0.2123814905577035,0.21268061165082322,0.3281285807463818,0.2994456979113117,0.44935805991440797,0.4484021304926764,0.10543298721313477,ge,0.45 +construction,yolo26x,Qwen3.8,four_ratio,2026,probability_difference,True,0.2368629762797031,0.21551128655882565,0.21401065405899017,0.3281285807463818,0.2992560160564012,0.44935805991440797,0.4477363515312916,0.009329468011856079,ge,0.45 +construction,yolo26x,Qwen3.8,four_ratio,2027,log_ratio,False,0.22989348555230152,0.21210124224567403,0.21150463331583857,0.3264586577163139,0.2973078947731702,0.5192582025677603,0.5163115845539281,-1.5303850173950195,ge,0.52 +construction,yolo26x,Qwen3.8,four_ratio,2027,probability_difference,True,0.2318634041838887,0.21224699544489786,0.21128368096133335,0.29850296625880607,0.2897476393716847,0.39942938659058486,0.41278295605858856,-0.016872283071279526,ge,0.4 +construction,yolo26x,Qwen3.8,four_ratio,2028,log_ratio,False,0.23067441529919713,0.21212878285104259,0.2108049402577198,0.3269493177387914,0.3005042852186894,0.4593437945791726,0.4540612516644474,-0.1898920238018036,ge,0.46 +construction,yolo26x,Qwen3.8,four_ratio,2028,probability_difference,True,0.23163970072423956,0.21435518602832812,0.2115977025124468,0.3263512316143895,0.3002620371566739,0.4593437945791726,0.45272969374167776,-0.010172035545110703,ge,0.46 +construction,yolo26x,Qwen3.8,four_ratio,2029,log_ratio,False,0.23238062116119743,0.2129759325509759,0.2118170700354305,0.3274085072339964,0.2965825858147407,0.4992867332382311,0.4936750998668442,-1.27814519405365,ge,0.5 +construction,yolo26x,Qwen3.8,four_ratio,2029,probability_difference,True,0.23647553559932344,0.21591364683230602,0.21245346447351982,0.32788133528265107,0.29983745518329075,0.44935805991440797,0.4444074567243675,0.004357501864433289,ge,0.45 +construction,yolo26x,Qwen3.8,four_ratio,2030,log_ratio,False,0.2329566901533132,0.21387956085075915,0.21228593529346934,0.32788133528265107,0.2985470911000302,0.44935805991440797,0.45006657789613846,-0.3354406952857971,ge,0.45 +construction,yolo26x,Qwen3.8,four_ratio,2030,probability_difference,True,0.23793834805397718,0.21368445025146315,0.21270060003068342,0.31697887604623354,0.2948692962915945,0.5592011412268189,0.5605858854860186,-0.03164273127913475,ge,0.56 +construction,yolo26x,Qwen3.8,gain,2026,regression,False,0.23960077179566422,0.21400610735220266,0.21241159528412765,0.32586392690019633,0.29646409334649515,0.43937232524964337,0.43541944074567246,0.03868880867958069,ge,0.44 +construction,yolo26x,Qwen3.8,gain,2027,regression,False,0.23821364027732878,0.21409588787720268,0.2125057588947129,0.32335633116883117,0.29977910001068875,0.4992867332382311,0.4936750998668442,0.004932999610900879,ge,0.5 +construction,yolo26x,Qwen3.8,gain,2028,regression,False,0.23695562717660318,0.21466654111641606,0.21351268007648552,0.32723359493146115,0.2954033522460231,0.44935805991440797,0.4470705725699068,-0.007363259792327881,ge,0.45 +construction,yolo26x,Qwen3.8,gain,2029,regression,False,0.2365407013371036,0.21485543142572305,0.21212476860198715,0.3217186593475253,0.29700677143840626,0.4992867332382311,0.4966711051930759,-0.037876881659030914,ge,0.5 +construction,yolo26x,Qwen3.8,gain,2030,regression,False,0.23629618607816952,0.21389045323552514,0.2128786350723718,0.32395441729323304,0.29601885044321186,0.5092724679029957,0.5136484687083888,-0.026152916252613068,ge,0.51 +construction,yolo26x,Qwen3.8,route,2026,probability,False,0.23988153202052687,0.21526584183204825,0.21292677083735004,0.32788133528265107,0.3000955196994197,0.44935805991440797,0.4447403462050599,0.055466752499341965,ge,0.45 +construction,yolo26x,Qwen3.8,route,2027,probability,False,0.2410986216135448,0.2141749273260206,0.213116352929275,0.32872666687078367,0.30000656372593065,0.44935805991440797,0.4420772303595206,0.059321802109479904,ge,0.45 +construction,yolo26x,Qwen3.8,route,2028,probability,False,0.24658342608775974,0.21709641059985313,0.21429341402277668,0.3275339265335182,0.298343122279109,0.5092724679029957,0.503994673768309,0.012455511838197708,ge,0.51 +construction,yolo26x,Qwen3.8,route,2029,probability,False,0.24739975050797705,0.21485084530964332,0.21381384817331112,0.3293202407013377,0.30419831211135995,0.43937232524964337,0.4367509986684421,0.07076191157102585,ge,0.44 +construction,yolo26x,Qwen3.8,route,2030,probability,False,0.2412447050974016,0.2137178766516705,0.21255405593166388,0.3277986633249791,0.2991470955161605,0.4593437945791726,0.4530625832223702,0.03224967047572136,ge,0.46 diff --git a/releases/20260913_lr/paper_data/reproduction/DSE_STATE.json b/releases/20260913_lr/paper_data/reproduction/DSE_STATE.json new file mode 100644 index 0000000000000000000000000000000000000000..9392aa0d85468e9f65ad61fa23018b5535b43532 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/DSE_STATE.json @@ -0,0 +1,720 @@ +{ + "status": "search_complete", + "started_at": 1789041525.1799183, + "completed_stages": [ + "hidden_percent", + "student", + "representation_dim", + "lr", + "target", + "strategy" + ], + "selected": "shared_12", + "candidates": [ + { + "id": "shared_00", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_00_cub_MiniCPM", + "shared_00_cub_Qwen3.8", + "shared_00_glsim_MiniCPM", + "shared_00_glsim_Qwen3.8", + "shared_00_groundingdino_MiniCPM", + "shared_00_groundingdino_Qwen3.8", + "shared_00_instancevg_MiniCPM", + "shared_00_instancevg_Qwen3.8", + "shared_00_nlvr2_MiniCPM", + "shared_00_nlvr2_Qwen3.8", + "shared_00_vilt_MiniCPM", + "shared_00_vilt_Qwen3.8", + "shared_00_yolo26x_MiniCPM", + "shared_00_yolo26x_Qwen3.8", + "shared_00_rtdetr_x_fp32_MiniCPM", + "shared_00_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9498541666666667, + "grefcoco": 0.7090097687820648, + "nlvr2": 0.8164207963334288, + "construction": 0.175986962717605 + }, + "validation_mean": 0.6628179236249413, + "mean_active_parameters": 4881924.0, + "mean_fit_seconds": 37.63353890167855 + }, + { + "id": "shared_01", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 30 + }, + "runs": [ + "shared_01_cub_MiniCPM", + "shared_01_cub_Qwen3.8", + "shared_01_glsim_MiniCPM", + "shared_01_glsim_Qwen3.8", + "shared_01_groundingdino_MiniCPM", + "shared_01_groundingdino_Qwen3.8", + "shared_01_instancevg_MiniCPM", + "shared_01_instancevg_Qwen3.8", + "shared_01_nlvr2_MiniCPM", + "shared_01_nlvr2_Qwen3.8", + "shared_01_vilt_MiniCPM", + "shared_01_vilt_Qwen3.8", + "shared_01_yolo26x_MiniCPM", + "shared_01_yolo26x_Qwen3.8", + "shared_01_rtdetr_x_fp32_MiniCPM", + "shared_01_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9346041666666667, + "grefcoco": 0.7090431513107035, + "nlvr2": 0.8157691205958177, + "construction": 0.18298715773892013 + }, + "validation_mean": 0.660600899078027, + "mean_active_parameters": 4783620.0, + "mean_fit_seconds": 39.49221354418114 + }, + { + "id": "shared_02", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 60 + }, + "runs": [ + "shared_02_cub_MiniCPM", + "shared_02_cub_Qwen3.8", + "shared_02_glsim_MiniCPM", + "shared_02_glsim_Qwen3.8", + "shared_02_groundingdino_MiniCPM", + "shared_02_groundingdino_Qwen3.8", + "shared_02_instancevg_MiniCPM", + "shared_02_instancevg_Qwen3.8", + "shared_02_nlvr2_MiniCPM", + "shared_02_nlvr2_Qwen3.8", + "shared_02_vilt_MiniCPM", + "shared_02_vilt_Qwen3.8", + "shared_02_yolo26x_MiniCPM", + "shared_02_yolo26x_Qwen3.8", + "shared_02_rtdetr_x_fp32_MiniCPM", + "shared_02_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9472291666666667, + "grefcoco": 0.708183990442055, + "nlvr2": 0.8155900887997709, + "construction": 0.1758169352446146 + }, + "validation_mean": 0.6617050452882768, + "mean_active_parameters": 4816388.0, + "mean_fit_seconds": 34.18548409075447 + }, + { + "id": "shared_03", + "settings": { + "student": "1M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_03_cub_MiniCPM", + "shared_03_cub_Qwen3.8", + "shared_03_glsim_MiniCPM", + "shared_03_glsim_Qwen3.8", + "shared_03_groundingdino_MiniCPM", + "shared_03_groundingdino_Qwen3.8", + "shared_03_instancevg_MiniCPM", + "shared_03_instancevg_Qwen3.8", + "shared_03_nlvr2_MiniCPM", + "shared_03_nlvr2_Qwen3.8", + "shared_03_vilt_MiniCPM", + "shared_03_vilt_Qwen3.8", + "shared_03_yolo26x_MiniCPM", + "shared_03_yolo26x_Qwen3.8", + "shared_03_rtdetr_x_fp32_MiniCPM", + "shared_03_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9559375, + "grefcoco": 0.7093348091924943, + "nlvr2": 0.8166929246634202, + "construction": 0.173092415345378 + }, + "validation_mean": 0.6637644123003231, + "mean_active_parameters": 2615822.0, + "mean_fit_seconds": 57.72181743498368 + }, + { + "id": "shared_04", + "settings": { + "student": "2M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_04_cub_MiniCPM", + "shared_04_cub_Qwen3.8", + "shared_04_glsim_MiniCPM", + "shared_04_glsim_Qwen3.8", + "shared_04_groundingdino_MiniCPM", + "shared_04_groundingdino_Qwen3.8", + "shared_04_instancevg_MiniCPM", + "shared_04_instancevg_Qwen3.8", + "shared_04_nlvr2_MiniCPM", + "shared_04_nlvr2_Qwen3.8", + "shared_04_vilt_MiniCPM", + "shared_04_vilt_Qwen3.8", + "shared_04_yolo26x_MiniCPM", + "shared_04_yolo26x_Qwen3.8", + "shared_04_rtdetr_x_fp32_MiniCPM", + "shared_04_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9448958333333333, + "grefcoco": 0.7087286527514232, + "nlvr2": 0.8167144084789458, + "construction": 0.1764152469711854 + }, + "validation_mean": 0.661688535383722, + "mean_active_parameters": 3402222.0, + "mean_fit_seconds": 57.64163886100141 + }, + { + "id": "shared_05", + "settings": { + "student": "8M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_05_cub_MiniCPM", + "shared_05_cub_Qwen3.8", + "shared_05_glsim_MiniCPM", + "shared_05_glsim_Qwen3.8", + "shared_05_groundingdino_MiniCPM", + "shared_05_groundingdino_Qwen3.8", + "shared_05_instancevg_MiniCPM", + "shared_05_instancevg_Qwen3.8", + "shared_05_nlvr2_MiniCPM", + "shared_05_nlvr2_Qwen3.8", + "shared_05_vilt_MiniCPM", + "shared_05_vilt_Qwen3.8", + "shared_05_yolo26x_MiniCPM", + "shared_05_yolo26x_Qwen3.8", + "shared_05_rtdetr_x_fp32_MiniCPM", + "shared_05_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9542916666666666, + "grefcoco": 0.7095351043643263, + "nlvr2": 0.8168110856488112, + "construction": 0.17955466204144832 + }, + "validation_mean": 0.6650481296803131, + "mean_active_parameters": 7994758.0, + "mean_fit_seconds": 46.836150376082514 + }, + { + "id": "shared_06", + "settings": { + "student": "8M", + "representation_dim": 32, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_06_cub_MiniCPM", + "shared_06_cub_Qwen3.8", + "shared_06_glsim_MiniCPM", + "shared_06_glsim_Qwen3.8", + "shared_06_groundingdino_MiniCPM", + "shared_06_groundingdino_Qwen3.8", + "shared_06_instancevg_MiniCPM", + "shared_06_instancevg_Qwen3.8", + "shared_06_nlvr2_MiniCPM", + "shared_06_nlvr2_Qwen3.8", + "shared_06_vilt_MiniCPM", + "shared_06_vilt_Qwen3.8", + "shared_06_yolo26x_MiniCPM", + "shared_06_yolo26x_Qwen3.8", + "shared_06_rtdetr_x_fp32_MiniCPM", + "shared_06_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.966125, + "grefcoco": 0.7070665542202544, + "nlvr2": 0.8169149240905185, + "construction": 0.17258963439949287 + }, + "validation_mean": 0.6656740281775665, + "mean_active_parameters": 6222718.0, + "mean_fit_seconds": 63.611915712936025 + }, + { + "id": "shared_07", + "settings": { + "student": "8M", + "representation_dim": 64, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_07_cub_MiniCPM", + "shared_07_cub_Qwen3.8", + "shared_07_glsim_MiniCPM", + "shared_07_glsim_Qwen3.8", + "shared_07_groundingdino_MiniCPM", + "shared_07_groundingdino_Qwen3.8", + "shared_07_instancevg_MiniCPM", + "shared_07_instancevg_Qwen3.8", + "shared_07_nlvr2_MiniCPM", + "shared_07_nlvr2_Qwen3.8", + "shared_07_vilt_MiniCPM", + "shared_07_vilt_Qwen3.8", + "shared_07_yolo26x_MiniCPM", + "shared_07_yolo26x_Qwen3.8", + "shared_07_rtdetr_x_fp32_MiniCPM", + "shared_07_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9693333333333334, + "grefcoco": 0.7083755007379295, + "nlvr2": 0.8167716986536809, + "construction": 0.17603867039292925 + }, + "validation_mean": 0.6676298007794683, + "mean_active_parameters": 6340854.0, + "mean_fit_seconds": 62.082761554156605 + }, + { + "id": "shared_08", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_08_cub_MiniCPM", + "shared_08_cub_Qwen3.8", + "shared_08_glsim_MiniCPM", + "shared_08_glsim_Qwen3.8", + "shared_08_groundingdino_MiniCPM", + "shared_08_groundingdino_Qwen3.8", + "shared_08_instancevg_MiniCPM", + "shared_08_instancevg_Qwen3.8", + "shared_08_nlvr2_MiniCPM", + "shared_08_nlvr2_Qwen3.8", + "shared_08_vilt_MiniCPM", + "shared_08_vilt_Qwen3.8", + "shared_08_yolo26x_MiniCPM", + "shared_08_yolo26x_Qwen3.8", + "shared_08_rtdetr_x_fp32_MiniCPM", + "shared_08_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9704166666666667, + "grefcoco": 0.7082156159955021, + "nlvr2": 0.8169006015468347, + "construction": 0.17553875298671762 + }, + "validation_mean": 0.6677679092989303, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 58.37819891100662 + }, + { + "id": "shared_09", + "settings": { + "student": "8M", + "representation_dim": 256, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_09_cub_MiniCPM", + "shared_09_cub_Qwen3.8", + "shared_09_glsim_MiniCPM", + "shared_09_glsim_Qwen3.8", + "shared_09_groundingdino_MiniCPM", + "shared_09_groundingdino_Qwen3.8", + "shared_09_instancevg_MiniCPM", + "shared_09_instancevg_Qwen3.8", + "shared_09_nlvr2_MiniCPM", + "shared_09_nlvr2_Qwen3.8", + "shared_09_vilt_MiniCPM", + "shared_09_vilt_Qwen3.8", + "shared_09_yolo26x_MiniCPM", + "shared_09_yolo26x_Qwen3.8", + "shared_09_rtdetr_x_fp32_MiniCPM", + "shared_09_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9678541666666667, + "grefcoco": 0.7090659919881931, + "nlvr2": 0.8167609567459181, + "construction": 0.17265650809871438 + }, + "validation_mean": 0.6665844058748731, + "mean_active_parameters": 7049670.0, + "mean_fit_seconds": 50.47316628225235 + }, + { + "id": "shared_10", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.0001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_10_cub_MiniCPM", + "shared_10_cub_Qwen3.8", + "shared_10_glsim_MiniCPM", + "shared_10_glsim_Qwen3.8", + "shared_10_groundingdino_MiniCPM", + "shared_10_groundingdino_Qwen3.8", + "shared_10_instancevg_MiniCPM", + "shared_10_instancevg_Qwen3.8", + "shared_10_nlvr2_MiniCPM", + "shared_10_nlvr2_Qwen3.8", + "shared_10_vilt_MiniCPM", + "shared_10_vilt_Qwen3.8", + "shared_10_yolo26x_MiniCPM", + "shared_10_yolo26x_Qwen3.8", + "shared_10_rtdetr_x_fp32_MiniCPM", + "shared_10_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9694166666666667, + "grefcoco": 0.7047965422728231, + "nlvr2": 0.8161844743626469, + "construction": 0.17403212405686308 + }, + "validation_mean": 0.66610745183975, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 91.61671701988234 + }, + { + "id": "shared_11", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.0003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_11_cub_MiniCPM", + "shared_11_cub_Qwen3.8", + "shared_11_glsim_MiniCPM", + "shared_11_glsim_Qwen3.8", + "shared_11_groundingdino_MiniCPM", + "shared_11_groundingdino_Qwen3.8", + "shared_11_instancevg_MiniCPM", + "shared_11_instancevg_Qwen3.8", + "shared_11_nlvr2_MiniCPM", + "shared_11_nlvr2_Qwen3.8", + "shared_11_vilt_MiniCPM", + "shared_11_vilt_Qwen3.8", + "shared_11_yolo26x_MiniCPM", + "shared_11_yolo26x_Qwen3.8", + "shared_11_rtdetr_x_fp32_MiniCPM", + "shared_11_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9692083333333333, + "grefcoco": 0.7071210204511913, + "nlvr2": 0.8166034087653967, + "construction": 0.17173346213113433 + }, + "validation_mean": 0.6661665561702639, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 75.3759171601705 + }, + { + "id": "shared_12", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_12_cub_MiniCPM", + "shared_12_cub_Qwen3.8", + "shared_12_glsim_MiniCPM", + "shared_12_glsim_Qwen3.8", + "shared_12_groundingdino_MiniCPM", + "shared_12_groundingdino_Qwen3.8", + "shared_12_instancevg_MiniCPM", + "shared_12_instancevg_Qwen3.8", + "shared_12_nlvr2_MiniCPM", + "shared_12_nlvr2_Qwen3.8", + "shared_12_vilt_MiniCPM", + "shared_12_vilt_Qwen3.8", + "shared_12_yolo26x_MiniCPM", + "shared_12_yolo26x_Qwen3.8", + "shared_12_rtdetr_x_fp32_MiniCPM", + "shared_12_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9698125, + "grefcoco": 0.7082964368543117, + "nlvr2": 0.8164959896877686, + "construction": 0.1768778191883302 + }, + "validation_mean": 0.6678706864326026, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 67.41994623254868 + }, + { + "id": "shared_13", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.01, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_13_cub_MiniCPM", + "shared_13_cub_Qwen3.8", + "shared_13_glsim_MiniCPM", + "shared_13_glsim_Qwen3.8", + "shared_13_groundingdino_MiniCPM", + "shared_13_groundingdino_Qwen3.8", + "shared_13_instancevg_MiniCPM", + "shared_13_instancevg_Qwen3.8", + "shared_13_nlvr2_MiniCPM", + "shared_13_nlvr2_Qwen3.8", + "shared_13_vilt_MiniCPM", + "shared_13_vilt_Qwen3.8", + "shared_13_yolo26x_MiniCPM", + "shared_13_yolo26x_Qwen3.8", + "shared_13_rtdetr_x_fp32_MiniCPM", + "shared_13_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9509375, + "grefcoco": 0.7100024597652681, + "nlvr2": 0.8169936980807792, + "construction": 0.17572600052962756 + }, + "validation_mean": 0.6634149145939187, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 58.952753572441 + }, + { + "id": "shared_14", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "gain", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_14_cub_MiniCPM", + "shared_14_cub_Qwen3.8", + "shared_14_glsim_MiniCPM", + "shared_14_glsim_Qwen3.8", + "shared_14_groundingdino_MiniCPM", + "shared_14_groundingdino_Qwen3.8", + "shared_14_instancevg_MiniCPM", + "shared_14_instancevg_Qwen3.8", + "shared_14_nlvr2_MiniCPM", + "shared_14_nlvr2_Qwen3.8", + "shared_14_vilt_MiniCPM", + "shared_14_vilt_Qwen3.8", + "shared_14_yolo26x_MiniCPM", + "shared_14_yolo26x_Qwen3.8", + "shared_14_rtdetr_x_fp32_MiniCPM", + "shared_14_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.97975, + "grefcoco": 0.6943653805608265, + "nlvr2": 0.815819249498711, + "construction": 0.17119724948093312 + }, + "validation_mean": 0.6652829698851177, + "mean_active_parameters": 6576739.0, + "mean_fit_seconds": 46.42365771972982 + }, + { + "id": "shared_15", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "error", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_15_cub_MiniCPM", + "shared_15_cub_Qwen3.8", + "shared_15_glsim_MiniCPM", + "shared_15_glsim_Qwen3.8", + "shared_15_groundingdino_MiniCPM", + "shared_15_groundingdino_Qwen3.8", + "shared_15_instancevg_MiniCPM", + "shared_15_instancevg_Qwen3.8", + "shared_15_nlvr2_MiniCPM", + "shared_15_nlvr2_Qwen3.8", + "shared_15_vilt_MiniCPM", + "shared_15_vilt_Qwen3.8", + "shared_15_yolo26x_MiniCPM", + "shared_15_yolo26x_Qwen3.8", + "shared_15_rtdetr_x_fp32_MiniCPM", + "shared_15_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9172083333333333, + "grefcoco": 0.707660411834985, + "nlvr2": 0.8152570896591235, + "construction": 0.17163397920287538 + }, + "validation_mean": 0.6529399535075793, + "mean_active_parameters": 6576739.0, + "mean_fit_seconds": 39.98559669695533 + }, + { + "id": "shared_16", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "weighted", + "hidden_percent": 90 + }, + "runs": [ + "shared_16_cub_MiniCPM", + "shared_16_cub_Qwen3.8", + "shared_16_glsim_MiniCPM", + "shared_16_glsim_Qwen3.8", + "shared_16_groundingdino_MiniCPM", + "shared_16_groundingdino_Qwen3.8", + "shared_16_instancevg_MiniCPM", + "shared_16_instancevg_Qwen3.8", + "shared_16_nlvr2_MiniCPM", + "shared_16_nlvr2_Qwen3.8", + "shared_16_vilt_MiniCPM", + "shared_16_vilt_Qwen3.8", + "shared_16_yolo26x_MiniCPM", + "shared_16_yolo26x_Qwen3.8", + "shared_16_rtdetr_x_fp32_MiniCPM", + "shared_16_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.958625, + "grefcoco": 0.7108879752617893, + "nlvr2": 0.8163420223431682, + "construction": 0.17381623998937223 + }, + "validation_mean": 0.6649178093985824, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 39.95715373130224 + }, + { + "id": "shared_17", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "balanced", + "hidden_percent": 90 + }, + "runs": [ + "shared_17_cub_MiniCPM", + "shared_17_cub_Qwen3.8", + "shared_17_glsim_MiniCPM", + "shared_17_glsim_Qwen3.8", + "shared_17_groundingdino_MiniCPM", + "shared_17_groundingdino_Qwen3.8", + "shared_17_instancevg_MiniCPM", + "shared_17_instancevg_Qwen3.8", + "shared_17_nlvr2_MiniCPM", + "shared_17_nlvr2_Qwen3.8", + "shared_17_vilt_MiniCPM", + "shared_17_vilt_Qwen3.8", + "shared_17_yolo26x_MiniCPM", + "shared_17_yolo26x_Qwen3.8", + "shared_17_rtdetr_x_fp32_MiniCPM", + "shared_17_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9554583333333334, + "grefcoco": 0.7062583456321596, + "nlvr2": 0.8156903466055572, + "construction": 0.17510649266382114 + }, + "validation_mean": 0.6631283795587178, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 35.547329766683106 + } + ], + "failures": [], + "test_loaded": false, + "finished_at": 1789043234.371715 +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/reproduction/completion.json b/releases/20260913_lr/paper_data/reproduction/completion.json new file mode 100644 index 0000000000000000000000000000000000000000..ced7c666386d856c1c69680215c0e2f19b4ab65d --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/completion.json @@ -0,0 +1 @@ +{"status": "complete", "epochs": 300, "unix_time": 1788991077.698775} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/reproduction/config.json b/releases/20260913_lr/paper_data/reproduction/config.json new file mode 100644 index 0000000000000000000000000000000000000000..167d24c4082d3ab88fe51883a802b3948fa491ff --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/config.json @@ -0,0 +1,38 @@ +{ + "model": { + "width": 380, + "depth": 2, + "vocab_size": 4096, + "text_length": 64, + "output_dim": 768 + }, + "parameters": { + "total": 8072328, + "vision": 4135548, + "text": 3936780 + }, + "root": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration", + "output": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/mini_siglip_strict_20260910_300ep", + "experiment": [ + 1, + 2, + 4, + 8 + ], + "gpus_per_experiment": 1, + "one_gpu_per_experiment": true, + "cache_dir": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/data/siglip_distillation_strict_20260910", + "settings_file": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/code/siglip_distillation/exp_settings_strict_20260910.md", + "batch_size": 2048, + "lr": 0.0001, + "epochs": 300, + "weight_decay": 0.01, + "seed": 42, + "smoke_steps": 0, + "resume": false, + "gpus": [ + 3 + ], + "world_size": 1, + "size_millions": 8 +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/reproduction/construction_MiniCPM_display.txt b/releases/20260913_lr/paper_data/reproduction/construction_MiniCPM_display.txt new file mode 100644 index 0000000000000000000000000000000000000000..fc487abd7a2b3bffaddf9be35c3382da125298ac --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/construction_MiniCPM_display.txt @@ -0,0 +1,15 @@ +Look at this one construction-site image and report all visible violations from the 4 named categories in one JSON answer. Mark a category true when the image gives clear visual evidence. Do not invent details that are hidden, but do not reject a plainly visible violation merely because the scene is not a close-up. + +rule_1_ppe_violation: box every worker on foot who is visibly missing a hard hat or another clearly required item of personal protection. A bare head is positive. Do not assume hidden footwear or clothing is missing, and do not use the lack of a reflective vest in ordinary daylight by itself. + +rule_2_fall_protection_violation: box every worker clearly about three metres or more above the lower level when the exposed side has no effective edge protection and no safety harness is being used. Ground workers, low platforms, dirt slopes, effective guardrails, and uncertain heights are negative. + +rule_3_unprotected_edge_violation: box every people-accessible, continuous upper edge of a visibly deep excavation or steep drop of about three metres or more when guardrails, fencing, and warning protection are absent. The image must show both a substantial lower level and its hazardous upper edge; excavation activity or exposed soil alone is not enough. Soil piles, excavation walls without an accessible upper edge, shallow trenches, gentle slopes, ordinary uneven ground, uncertain distant cuts, and protected edges are negative. Box the hazardous upper edge and only enough adjacent drop to identify it, never a worker or the whole image. + +rule_4_excavator_proximity_violation: box a worker when the worker is clearly in the excavator's rear blind spot, swing radius, boom or bucket path, or close enough beside the body or tracks to be reached by normal machine movement. A worker merely visible in the same scene is not enough. The cab operator, a worker behind a barrier, and a worker separated from the machine by a clearly safe gap are negative. Box only the endangered worker. + +Use exactly this JSON shape: +{"events":{"rule_1_ppe_violation":{"present":false,"reason":null,"boxes":[]},"rule_2_fall_protection_violation":{"present":false,"reason":null,"boxes":[]},"rule_3_unprotected_edge_violation":{"present":false,"reason":null,"boxes":[]},"rule_4_excavator_proximity_violation":{"present":false,"reason":null,"boxes":[]}}} + +Change a visible category to present=true, give it a short positive reason, and add one or more worker or edge boxes. The word false must always keep reason=null and boxes=[]. Coordinates are integer image-relative [x1,y1,x2,y2] values from 0 to 1000 with x1 < x2 and y1 < y2. Use at most 13, 6, 3, and 2 boxes for rules 1 through 4. Return only the JSON object, with no Markdown or extra text. + diff --git a/releases/20260913_lr/paper_data/reproduction/construction_MiniCPM_prompt.txt b/releases/20260913_lr/paper_data/reproduction/construction_MiniCPM_prompt.txt new file mode 100644 index 0000000000000000000000000000000000000000..e3ccf75b8266b01e4b6b48e6b8645c9bb41becc6 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/construction_MiniCPM_prompt.txt @@ -0,0 +1,15 @@ +Look at this one construction-site image and report all visible violations from the four named categories in one JSON answer. Mark a category true when the image gives clear visual evidence. Do not invent details that are hidden, but do not reject a plainly visible violation merely because the scene is not a close-up. + +rule_1_ppe_violation: box every worker on foot who is visibly missing a hard hat or another clearly required item of personal protection. A bare head is positive. Do not assume hidden footwear or clothing is missing, and do not use the lack of a reflective vest in ordinary daylight by itself. + +rule_2_fall_protection_violation: box every worker clearly about three metres or more above the lower level when the exposed side has no effective edge protection and no safety harness is being used. Ground workers, low platforms, dirt slopes, effective guardrails, and uncertain heights are negative. + +rule_3_unprotected_edge_violation: box every people-accessible, continuous upper edge of a visibly deep excavation or steep drop of about three metres or more when guardrails, fencing, and warning protection are absent. The image must show both a substantial lower level and its hazardous upper edge; excavation activity or exposed soil alone is not enough. Soil piles, excavation walls without an accessible upper edge, shallow trenches, gentle slopes, ordinary uneven ground, uncertain distant cuts, and protected edges are negative. Box the hazardous upper edge and only enough adjacent drop to identify it, never a worker or the whole image. + +rule_4_excavator_proximity_violation: box a worker when the worker is clearly in the excavator's rear blind spot, swing radius, boom or bucket path, or close enough beside the body or tracks to be reached by normal machine movement. A worker merely visible in the same scene is not enough. The cab operator, a worker behind a barrier, and a worker separated from the machine by a clearly safe gap are negative. Box only the endangered worker. + +Use exactly this JSON shape: +{"events":{"rule_1_ppe_violation":{"present":false,"reason":null,"boxes":[]},"rule_2_fall_protection_violation":{"present":false,"reason":null,"boxes":[]},"rule_3_unprotected_edge_violation":{"present":false,"reason":null,"boxes":[]},"rule_4_excavator_proximity_violation":{"present":false,"reason":null,"boxes":[]}}} + +Change a visible category to present=true, give it a short positive reason, and add one or more worker or edge boxes. The word false must always keep reason=null and boxes=[]. Coordinates are integer image-relative [x1,y1,x2,y2] values from 0 to 1000 with x1 < x2 and y1 < y2. Use at most 13, 6, 3, and 2 boxes for rules 1 through 4. Return only the JSON object, with no Markdown or extra text. + diff --git a/releases/20260913_lr/paper_data/reproduction/construction_Qwen3.8_display.txt b/releases/20260913_lr/paper_data/reproduction/construction_Qwen3.8_display.txt new file mode 100644 index 0000000000000000000000000000000000000000..d0b8bf808b4c456c75f8c3b9baec7567c47176be --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/construction_Qwen3.8_display.txt @@ -0,0 +1,12 @@ +Look at this construction-site image and report every clearly visible violation from the 4 categories below in one JSON answer. Most categories will usually be absent in a single image. A dangerous-looking construction scene is not enough by itself; mark a category true only when its specific target and violation are visible. + +rule_1_ppe_violation means a worker on foot is visibly missing required personal protection. A clearly bare head is a violation. A worker who visibly wears a hard hat should normally be treated as compliant unless another required item is unmistakably missing for the task shown; ordinary work clothes or the absence of a reflective vest in ordinary daylight are not enough by themselves. Do not guess about small, distant, blurred, or hidden clothing, footwear, eyes, or faces. Put a tight box around each violating worker's visible body, not around the missing item. + +rule_2_fall_protection_violation means a worker is clearly about three metres or more above a lower level, is exposed because effective edge protection is absent, and is not using a safety harness. Workers on the ground, on a low platform or dirt slope, behind effective guardrails, or at an uncertain height are not violations. Box each exposed worker, not the structure or edge. + +rule_3_unprotected_edge_violation means an accessible upper edge borders a visibly deep excavation or steep drop of about three metres or more and lacks guardrails, fencing, or warning protection. The substantial height difference and the hazardous upper edge must both be discernible in the image rather than inferred merely because excavation work or exposed soil is present. Soil piles, excavation walls without an accessible upper edge, shallow trenches, gentle slopes, ordinary uneven ground, distant cuts with uncertain depth, and protected edges are not violations. Put one box around each continuous hazardous upper edge and only enough adjacent drop to show the hazard; do not box workers or the whole image. + +rule_4_excavator_proximity_violation means a visible worker is inside the normal reach of an excavator's rear counterweight, rotating body, boom, or bucket. A still photograph need not show actual motion: count a worker who is clearly within the machine's ordinary swing or travel zone, including directly beside its rear or tracks in the active work area. Do not count the cab operator, a worker behind a barrier, a worker separated by a clearly safe gap, or simple distant co-appearance. Box each endangered worker, not the excavator or a combined worker-machine area. + +Return exactly one JSON object whose "events" object contains "rule_1_ppe_violation", "rule_2_fall_protection_violation", "rule_3_unprotected_edge_violation", and "rule_4_excavator_proximity_violation" exactly once. A visible event is {"present":true,"reason":"short visible evidence","boxes":[[x1,y1,x2,y2]]}; an absent or uncertain event is {"present":false,"reason":null,"boxes":[]}. Coordinates are integer image-relative values from 0 to 1000 with x1 < x2 and y1 < y2. Return at most 13, 6, 3, and 2 boxes for rules 1 through 4. Return no Markdown or text outside the JSON object. + diff --git a/releases/20260913_lr/paper_data/reproduction/construction_Qwen3.8_prompt.txt b/releases/20260913_lr/paper_data/reproduction/construction_Qwen3.8_prompt.txt new file mode 100644 index 0000000000000000000000000000000000000000..749973379a44202d1b5fd9abf198528c18b3ad05 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/construction_Qwen3.8_prompt.txt @@ -0,0 +1,12 @@ +Look at this construction-site image and report every clearly visible violation from the four categories below in one JSON answer. Most categories will usually be absent in a single image. A dangerous-looking construction scene is not enough by itself; mark a category true only when its specific target and violation are visible. + +rule_1_ppe_violation means a worker on foot is visibly missing required personal protection. A clearly bare head is a violation. A worker who visibly wears a hard hat should normally be treated as compliant unless another required item is unmistakably missing for the task shown; ordinary work clothes or the absence of a reflective vest in ordinary daylight are not enough by themselves. Do not guess about small, distant, blurred, or hidden clothing, footwear, eyes, or faces. Put a tight box around each violating worker's visible body, not around the missing item. + +rule_2_fall_protection_violation means a worker is clearly about three metres or more above a lower level, is exposed because effective edge protection is absent, and is not using a safety harness. Workers on the ground, on a low platform or dirt slope, behind effective guardrails, or at an uncertain height are not violations. Box each exposed worker, not the structure or edge. + +rule_3_unprotected_edge_violation means an accessible upper edge borders a visibly deep excavation or steep drop of about three metres or more and lacks guardrails, fencing, or warning protection. The substantial height difference and the hazardous upper edge must both be discernible in the image rather than inferred merely because excavation work or exposed soil is present. Soil piles, excavation walls without an accessible upper edge, shallow trenches, gentle slopes, ordinary uneven ground, distant cuts with uncertain depth, and protected edges are not violations. Put one box around each continuous hazardous upper edge and only enough adjacent drop to show the hazard; do not box workers or the whole image. + +rule_4_excavator_proximity_violation means a visible worker is inside the normal reach of an excavator's rear counterweight, rotating body, boom, or bucket. A still photograph need not show actual motion: count a worker who is clearly within the machine's ordinary swing or travel zone, including directly beside its rear or tracks in the active work area. Do not count the cab operator, a worker behind a barrier, a worker separated by a clearly safe gap, or simple distant co-appearance. Box each endangered worker, not the excavator or a combined worker-machine area. + +Return exactly one JSON object whose "events" object contains "rule_1_ppe_violation", "rule_2_fall_protection_violation", "rule_3_unprotected_edge_violation", and "rule_4_excavator_proximity_violation" exactly once. A visible event is {"present":true,"reason":"short visible evidence","boxes":[[x1,y1,x2,y2]]}; an absent or uncertain event is {"present":false,"reason":null,"boxes":[]}. Coordinates are integer image-relative values from 0 to 1000 with x1 < x2 and y1 < y2. Return at most 13, 6, 3, and 2 boxes for rules 1 through 4. Return no Markdown or text outside the JSON object. + diff --git a/releases/20260913_lr/paper_data/reproduction/construction_image_path.jpg b/releases/20260913_lr/paper_data/reproduction/construction_image_path.jpg new file mode 100644 index 0000000000000000000000000000000000000000..d82eece29aeb7b735c36722f9245c2bc32da9888 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/construction_image_path.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:070910857b069ca9714c7a921b8804d9a9d28a0dd4cdcdaf3fb1d0b0e7287cb4 +size 236188 diff --git a/releases/20260913_lr/paper_data/reproduction/cub_MiniCPM_prompt.txt b/releases/20260913_lr/paper_data/reproduction/cub_MiniCPM_prompt.txt new file mode 100644 index 0000000000000000000000000000000000000000..69d173df03524f2d11209449de2fdfb0d21eaa9d --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/cub_MiniCPM_prompt.txt @@ -0,0 +1,4 @@ +Identify the bird species in this image using the fixed CUB-200-2011 class list below. Choose the class whose visible shape, bill, head pattern, wing pattern, body color, and tail best match the bird. Ignore the background and any non-bird objects. Reply with exactly one class number from 1 to 200 and no other text. + +{class_names} + diff --git a/releases/20260913_lr/paper_data/reproduction/cub_Qwen3.8_prompt.txt b/releases/20260913_lr/paper_data/reproduction/cub_Qwen3.8_prompt.txt new file mode 100644 index 0000000000000000000000000000000000000000..c2ffd213c2ab35c74720ebc67089247ce5128e7a --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/cub_Qwen3.8_prompt.txt @@ -0,0 +1,3 @@ +Identify the bird species in the image from the fixed CUB-200-2011 reference below. Each class is paired with three visual traits learned only from official training images. Use the species name to establish the likely bird group, then distinguish close candidates by the visible bill, crown and face markings, wing pattern, breast and belly color, tail, and overall form. The traits are clues rather than absolute rules because pose, sex, age, lighting, and occlusion can change what is visible. Choose the strongest overall match, ignore the background, and reply with exactly one class number from 1 to 200 and no other text. + +{class_traits} diff --git a/releases/20260913_lr/paper_data/reproduction/cub_image_path.jpg b/releases/20260913_lr/paper_data/reproduction/cub_image_path.jpg new file mode 100644 index 0000000000000000000000000000000000000000..d42d73fbae235d8ccd43ae8f798178389036600c Binary files /dev/null and b/releases/20260913_lr/paper_data/reproduction/cub_image_path.jpg differ diff --git a/releases/20260913_lr/paper_data/reproduction/examples.json b/releases/20260913_lr/paper_data/reproduction/examples.json new file mode 100644 index 0000000000000000000000000000000000000000..8ed9e0ac5254d3b00c687746beefb4f2ee414c63 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/examples.json @@ -0,0 +1,108 @@ +[ + { + "task": "cub", + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/data/router_splits_strict_20260910/cub/val.jsonl", + "record": { + "sample_id": "cub-train-00003", + "split": "val", + "image_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260827_cub_native_classification/inputs/train_images/00003.jpg", + "width": 500, + "height": 333, + "ground_truth": { + "sample_id": "cub-train-00003", + "category_id": 172, + "official_name": "172.Nashville_Warbler", + "prompt_name": "Nashville Warbler" + }, + "source_split": "train" + } + }, + { + "task": "grefcoco", + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/data/router_splits_strict_20260910/grefcoco/val.jsonl", + "record": { + "sample_id": "val-101-307", + "split": "val", + "ref_id": 101, + "sentence_id": 307, + "image_id": 716, + "image_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/data/grefcoco/images/train2014/COCO_train2014_000000000716.jpg", + "width": 640, + "height": 480, + "expression": "a mouse beside a laptop and a little boy sitting far from us is looking at a laptop", + "ground_truth": { + "sample_id": "val-101-307", + "split": "val", + "target_type": "multi_target", + "target_boxes": [ + { + "annotation_id": 209737, + "bbox_xyxy": [ + 231.76, + 148.4, + 348.26, + 267.79 + ] + }, + { + "annotation_id": 1971716, + "bbox_xyxy": [ + 454.39, + 335.2, + 491.24, + 361.65999999999997 + ] + } + ] + } + } + }, + { + "task": "nlvr2", + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/data/router_splits_strict_20260910/nlvr2/dev.jsonl", + "record": { + "sample_id": "dev-850-0-0", + "split": "dev", + "sentence": "The right image shows a curving walkway of dark glass circles embedded in dirt and flanked by foliage.", + "image1": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/data/nlvr2/images/eval_original_20260908/dev/dev/dev-850-0-img0.png", + "image2": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/data/nlvr2/images/eval_original_20260908/dev/dev/dev-850-0-img1.png", + "ground_truth": "False", + "image_source": "original_archive" + } + }, + { + "task": "construction", + "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/data/router_splits_strict_20260910/construction/val.jsonl", + "record": { + "sample_id": "0004857", + "split": "val", + "image_path": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260901_construction_yolo4_events/dataset_event4_yolo/images/val/0004857.jpg", + "ground_truth": { + "sample_id": "0004857", + "split": "val", + "events": { + "rule_1_ppe_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_2_fall_protection_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_3_unprotected_edge_violation": { + "present": false, + "reason": null, + "boxes": [] + }, + "rule_4_excavator_proximity_violation": { + "present": false, + "reason": null, + "boxes": [] + } + } + } + } + } +] \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/reproduction/exp_settings.md b/releases/20260913_lr/paper_data/reproduction/exp_settings.md new file mode 100644 index 0000000000000000000000000000000000000000..37c53a288a9805041f835f18208ded831e8bdf63 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/exp_settings.md @@ -0,0 +1,13 @@ +# 严格排除验证与测试数据后的重新蒸馏 + +四个尺寸全部随机初始化,从第1轮训练至300轮,不加载任何旧学生权重或优化器。旧学生目录已改名作废。教师仍是固定的SigLIP2;只复用通过新划分检查的逐样本教师表示,教师不在本项目训练。 + +数据清单:`data/siglip_distillation_strict_20260910/`;固定训练、验证和测试划分:`data/router_splits_strict_20260910/`。CUB按类别从原训练集固定留出20%验证,种子42;工地排除路由验证701张;gRefCOCO按COCO图片编号排除非训练图片;NLVR2排除验证、测试来源网址。验证/测试文本的规范化重复项也不进入蒸馏或词表训练。具体数量及检查范围以`split_audit.json`为准,不宣称检查了外部教师预训练集或任意未知近重复图片。 + +文字从允许的训练样本重建,并重新训练4096词词表,最大长度64。图像224×224、16×16图块;四档学生各两层Transformer,维度及总参数与上次相同。 + +仍为图像与文本两个教师表示的联合蒸馏,两个余弦损失权重均1,不加入新的关系损失。两路独立采样。每轮以较多的一路完整遍历计,另一端循环补齐。 + +四张A800每卡负责一个尺寸,共用父进程保有的0号卡图文缓存;每个尺寸批量2048,AdamW,学习率0.0001,权重衰减0.01,随机种子42,总300轮,逐步余弦衰减。正式启动前使用同配置做3步接线检查;正式训练重新随机初始化,不继承检查权重。 + +每轮保存可恢复权重、逐步及逐轮日志,每25轮及最后一轮保存独立权重。每个模型的`training_started.json`记录真正进入训练的时间;正式运行5分钟后的预计工期依据这次实测,不沿用上次混合硬件耗时。 diff --git a/releases/20260913_lr/paper_data/reproduction/grefcoco_MiniCPM_display.txt b/releases/20260913_lr/paper_data/reproduction/grefcoco_MiniCPM_display.txt new file mode 100644 index 0000000000000000000000000000000000000000..ac32d686126d5119dc1708a7c3f2191c0ab022c3 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/grefcoco_MiniCPM_display.txt @@ -0,0 +1,6 @@ +Your entire reply is one complete JSON line that begins with {"objects": and ends with the final }. A no-match answer is exactly {"objects":[]}. A one-match answer looks like {"objects":[{"bbox_2d":[120,85,640,910],"score":0.92}]}. Several matches use separate complete entries, for example {"objects":[{"bbox_2d":[80,100,300,700],"score":0.93},{"bbox_2d":[560,120,820,740],"score":0.91}]}. Every bbox_2d contains all 4 xyxy numbers from 0 to 1000 with x1 smaller than x2 and y1 smaller than y2. Never end after only one coordinate; finish every object, the objects list, and the final brace. Return no other text. + +Locate all and only the visible objects that satisfy the entire referring expression below. Every stated type, appearance, position, and spatial relation matters. Interpret left and right from the viewer's perspective, and use front, back, nearest, farthest, and ordinal words to distinguish among similar objects. Some expressions intentionally have no matching object, so use the no-match answer whenever the full expression is not visibly satisfied instead of choosing the closest-looking object. Return one tight box for every distinct match without merging objects into a group box. + +Referring expression: {expression} + diff --git a/releases/20260913_lr/paper_data/reproduction/grefcoco_MiniCPM_prompt.txt b/releases/20260913_lr/paper_data/reproduction/grefcoco_MiniCPM_prompt.txt new file mode 100644 index 0000000000000000000000000000000000000000..b7555aaaab009bf50ce3b69d6f953db6b0aee512 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/grefcoco_MiniCPM_prompt.txt @@ -0,0 +1,6 @@ +Your entire reply is one complete JSON line that begins with {"objects": and ends with the final }. A no-match answer is exactly {"objects":[]}. A one-match answer looks like {"objects":[{"bbox_2d":[120,85,640,910],"score":0.92}]}. Several matches use separate complete entries, for example {"objects":[{"bbox_2d":[80,100,300,700],"score":0.93},{"bbox_2d":[560,120,820,740],"score":0.91}]}. Every bbox_2d contains all four xyxy numbers from 0 to 1000 with x1 smaller than x2 and y1 smaller than y2. Never end after only one coordinate; finish every object, the objects list, and the final brace. Return no other text. + +Locate all and only the visible objects that satisfy the entire referring expression below. Every stated type, appearance, position, and spatial relation matters. Interpret left and right from the viewer's perspective, and use front, back, nearest, farthest, and ordinal words to distinguish among similar objects. Some expressions intentionally have no matching object, so use the no-match answer whenever the full expression is not visibly satisfied instead of choosing the closest-looking object. Return one tight box for every distinct match without merging objects into a group box. + +Referring expression: {expression} + diff --git a/releases/20260913_lr/paper_data/reproduction/grefcoco_Qwen3.8_prompt.txt b/releases/20260913_lr/paper_data/reproduction/grefcoco_Qwen3.8_prompt.txt new file mode 100644 index 0000000000000000000000000000000000000000..7e1a2b7165857469c89d2a22a4cade6df33963aa --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/grefcoco_Qwen3.8_prompt.txt @@ -0,0 +1,8 @@ +Locate all and only the visible objects that satisfy the entire referring expression below. Every stated object type, appearance, position, and spatial relation matters. Do not return the closest-looking object when part of the expression is missing or contradicted. Some expressions intentionally have no matching object, so an empty list is a normal answer whenever the full expression is not visibly satisfied. + +The expression may identify zero, one, or multiple objects. When separate target descriptions are joined by words such as "and", return a separate tight box for every described target rather than one box around the group. When "and" only joins properties of one object, all those properties must hold for that one box. Do not merge neighboring objects into one box. + +Coordinates must be xyxy numbers normalized from 0 to 1000, and each box must have a confidence score from 0 to 1. Reply with exactly one complete JSON object and no Markdown or explanation, using this form: {"objects":[{"bbox_2d":[120,85,640,910],"score":0.92}]}. An answer with no match is exactly {"objects":[]}. + +Referring expression: {expression} + diff --git a/releases/20260913_lr/paper_data/reproduction/grefcoco_image_path.jpg b/releases/20260913_lr/paper_data/reproduction/grefcoco_image_path.jpg new file mode 100644 index 0000000000000000000000000000000000000000..1b1d703cb395ffa79a50250fa3e94c89cbd74f63 Binary files /dev/null and b/releases/20260913_lr/paper_data/reproduction/grefcoco_image_path.jpg differ diff --git a/releases/20260913_lr/paper_data/reproduction/loss_epochs.csv b/releases/20260913_lr/paper_data/reproduction/loss_epochs.csv new file mode 100644 index 0000000000000000000000000000000000000000..414e022bcd5b9bc9a84814cac665bcd837445086 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/loss_epochs.csv @@ -0,0 +1,301 @@ +epoch,image_loss,text_loss,total_loss,seconds,peak_gpu_gib +1,0.38169994950294495,0.29440152645111084,0.6761013269424438,26.331963401986286,16.537731647491455 +2,0.30366095900535583,0.18708381056785583,0.49074476957321167,24.59355512697948,16.537731647491455 +3,0.2973741590976715,0.17778776586055756,0.4751618802547455,24.588273482979275,16.537795066833496 +4,0.29009467363357544,0.16689662635326385,0.45699137449264526,24.60026482900139,16.537795066833496 +5,0.2836175858974457,0.15723110735416412,0.4408486485481262,24.575577274954412,16.537795066833496 +6,0.27890950441360474,0.14786240458488464,0.42677196860313416,24.629834071965888,16.537795066833496 +7,0.274699866771698,0.13933546841144562,0.4140353202819824,24.619091781962197,16.537915229797363 +8,0.2708246409893036,0.13187596201896667,0.40270060300827026,24.59842898795614,16.537915229797363 +9,0.26745671033859253,0.12555359303951263,0.39301031827926636,24.54948589199921,16.537915229797363 +10,0.264181911945343,0.1200122982263565,0.3841942548751831,24.550708913011476,16.537915229797363 +11,0.2610059678554535,0.11548355221748352,0.37648966908454895,24.553155623958446,16.538061141967773 +12,0.2583882212638855,0.11170694231987,0.3700951635837555,24.580371932999697,16.538061141967773 +13,0.2553453743457794,0.10830622911453247,0.3636515736579895,24.645826621970627,16.538061141967773 +14,0.252975195646286,0.10555939376354218,0.35853469371795654,24.608030108967796,16.538061141967773 +15,0.25128793716430664,0.10278020799160004,0.3540681302547455,24.613570243003778,16.538061141967773 +16,0.2489832043647766,0.10057397931814194,0.34955716133117676,24.637701046012808,16.538061141967773 +17,0.24709868431091309,0.09846373647451401,0.34556230902671814,24.57681101298658,16.538061141967773 +18,0.24493275582790375,0.0965164452791214,0.34144917130470276,24.579220557003282,16.538061141967773 +19,0.24332989752292633,0.09513607621192932,0.33846598863601685,24.575602344993968,16.538061141967773 +20,0.24169297516345978,0.09327544271945953,0.3349684178829193,24.600441598973703,16.538061141967773 +21,0.23992949724197388,0.09183112531900406,0.3317604959011078,24.558528236986604,16.538061141967773 +22,0.23852334916591644,0.09055673331022263,0.32908007502555847,24.524785312998574,16.538061141967773 +23,0.23724085092544556,0.08932065963745117,0.32656151056289673,24.56060989201069,16.538061141967773 +24,0.2356327325105667,0.08850018680095673,0.3241329789161682,24.57394596398808,16.538061141967773 +25,0.23443114757537842,0.08713757991790771,0.32156872749328613,24.54459476401098,16.538061141967773 +26,0.23280662298202515,0.08620020747184753,0.31900689005851746,24.554826711013447,16.538061141967773 +27,0.2318182736635208,0.08509977161884308,0.31691810488700867,24.616897800005972,16.538061141967773 +28,0.23042860627174377,0.08435436338186264,0.31478291749954224,24.639545881014783,16.538061141967773 +29,0.22887255251407623,0.08354092389345169,0.3124134838581085,24.56846531596966,16.538061141967773 +30,0.2278919219970703,0.08265107870101929,0.3105429410934448,24.530594648967963,16.538061141967773 +31,0.22699986398220062,0.08181413263082504,0.30881401896476746,24.574196753033902,16.538061141967773 +32,0.22566719353199005,0.08112400025129318,0.3067912459373474,24.57827706396347,16.538061141967773 +33,0.22460484504699707,0.08031459897756577,0.3049194812774658,24.597571842954494,16.538061141967773 +34,0.22335034608840942,0.07969635725021362,0.3030467927455902,24.556553228991106,16.538061141967773 +35,0.22223418951034546,0.07912632822990417,0.30136051774024963,24.607842475001235,16.538061141967773 +36,0.2212960422039032,0.07847446948289871,0.2997705638408661,24.58042472100351,16.538061141967773 +37,0.2204824686050415,0.07763839513063431,0.29812091588974,24.575828389963135,16.538061141967773 +38,0.2191540151834488,0.07711657881736755,0.29627057909965515,24.552013123990037,16.538061141967773 +39,0.21828193962574005,0.07649268954992294,0.2947746515274048,24.502926516986918,16.538061141967773 +40,0.2174306958913803,0.07595304399728775,0.29338374733924866,24.556589913961943,16.538061141967773 +41,0.21614877879619598,0.0753927230834961,0.29154151678085327,24.536056321987417,16.538061141967773 +42,0.21514026820659637,0.07493691891431808,0.2900771498680115,24.578196354035754,16.538061141967773 +43,0.21437673270702362,0.07434365898370743,0.28872042894363403,24.592609788989648,16.538061141967773 +44,0.21339477598667145,0.07398340851068497,0.28737813234329224,24.606362355989404,16.538061141967773 +45,0.21242891252040863,0.07335265725851059,0.2857815623283386,24.57683062000433,16.538061141967773 +46,0.2118227183818817,0.07284755259752274,0.284670352935791,24.52285299199866,16.538061141967773 +47,0.210783451795578,0.07235153764486313,0.28313499689102173,24.588152919954155,16.538061141967773 +48,0.20971404016017914,0.07203736901283264,0.28175148367881775,24.58494030201109,16.538061141967773 +49,0.20877404510974884,0.07147669792175293,0.28025078773498535,24.605456740013324,16.538061141967773 +50,0.20790454745292664,0.07100217789411545,0.2789066433906555,24.58449192601256,16.538061141967773 +51,0.2074689120054245,0.07067886739969254,0.27814775705337524,24.62764697399689,16.538061141967773 +52,0.2065799981355667,0.07028350234031677,0.2768634855747223,24.640694460016675,16.538061141967773 +53,0.20535270869731903,0.06991709768772125,0.27526986598968506,24.536149800987914,16.538061141967773 +54,0.20463639497756958,0.06942184269428253,0.2740582227706909,24.538073144038208,16.538061141967773 +55,0.20392414927482605,0.0689614787697792,0.27288559079170227,24.58334651699988,16.538061141967773 +56,0.20320290327072144,0.06862780451774597,0.2718307375907898,24.608183332020417,16.538061141967773 +57,0.20232294499874115,0.06827466934919357,0.2705976366996765,24.589062219951302,16.538061141967773 +58,0.20145128667354584,0.06806720048189163,0.26951852440834045,24.638605396961793,16.538061141967773 +59,0.2008747160434723,0.06756433844566345,0.26843905448913574,24.57679158897372,16.538061141967773 +60,0.19996777176856995,0.06720567494630814,0.26717352867126465,24.57199823798146,16.538061141967773 +61,0.1989816129207611,0.06691036373376846,0.26589202880859375,24.561299055989366,16.538061141967773 +62,0.19817695021629333,0.06656637787818909,0.26474320888519287,24.578324441972654,16.538061141967773 +63,0.1975945234298706,0.06631047278642654,0.2639048993587494,24.568218585976865,16.538061141967773 +64,0.1967531442642212,0.06590515375137329,0.26265832781791687,24.606933967967052,16.538061141967773 +65,0.1961655616760254,0.06570370495319366,0.2618691921234131,24.60413408401655,16.538061141967773 +66,0.19560213387012482,0.06533174961805344,0.26093384623527527,24.61445846699644,16.538061141967773 +67,0.19462761282920837,0.06506501883268356,0.2596926689147949,24.612582314002793,16.538061141967773 +68,0.19394227862358093,0.06479647755622864,0.25873884558677673,24.529169813962653,16.538061141967773 +69,0.19333340227603912,0.0645025447010994,0.2578359842300415,24.618322068999987,16.538061141967773 +70,0.1924489438533783,0.06420794874429703,0.2566568851470947,24.604794248996768,16.538061141967773 +71,0.19211576879024506,0.06393476575613022,0.25605058670043945,24.54316502797883,16.538061141967773 +72,0.19109836220741272,0.06369583308696747,0.254794180393219,24.578262674971484,16.538061141967773 +73,0.19051635265350342,0.06337708234786987,0.2538934051990509,24.662088126991875,16.538061141967773 +74,0.18996688723564148,0.06309347599744797,0.2530602812767029,24.601354965998325,16.538061141967773 +75,0.18924769759178162,0.06295687705278397,0.2522045075893402,24.55083378398558,16.538061141967773 +76,0.18846574425697327,0.06266548484563828,0.2511310875415802,24.558917244023178,16.538061141967773 +77,0.18775574862957,0.062493354082107544,0.25024908781051636,24.557256483996753,16.538061141967773 +78,0.1875632405281067,0.06214962154626846,0.24971291422843933,24.576127021980938,16.538061141967773 +79,0.18670663237571716,0.061992861330509186,0.24869953095912933,24.509278555982746,16.538061141967773 +80,0.18595336377620697,0.06174543499946594,0.24769887328147888,24.594442485948093,16.538061141967773 +81,0.18548421561717987,0.061540476977825165,0.24702470004558563,24.576886782015208,16.538061141967773 +82,0.184661403298378,0.06126108393073082,0.2459225356578827,24.527889075980056,16.538061141967773 +83,0.18413899838924408,0.06109264865517616,0.24523161351680756,24.53612045699265,16.538061141967773 +84,0.18341723084449768,0.06086864694952965,0.24428583681583405,24.66846833296586,16.538061141967773 +85,0.18308743834495544,0.06065921485424042,0.2437465786933899,24.722954003023915,16.538061141967773 +86,0.18216858804225922,0.06048444285988808,0.242653027176857,24.644223303999752,16.538061141967773 +87,0.18191535770893097,0.060250379145145416,0.2421656996011734,24.613704594958108,16.538061141967773 +88,0.18111921846866608,0.0600358322262764,0.24115504324436188,24.589966397965327,16.538061141967773 +89,0.18053145706653595,0.059874411672353745,0.24040587246418,24.531438074016478,16.538061141967773 +90,0.1802002191543579,0.05968380719423294,0.23988407850265503,24.564661459007766,16.538061141967773 +91,0.1794125884771347,0.05943508818745613,0.23884765803813934,24.563521293981466,16.538061141967773 +92,0.17912575602531433,0.059325236827135086,0.23845097422599792,24.60010905994568,16.538061141967773 +93,0.1785309612751007,0.059117089956998825,0.23764804005622864,24.66342087398516,16.538061141967773 +94,0.1779632270336151,0.0590134896337986,0.23697669804096222,24.607082493952475,16.538061141967773 +95,0.1771625131368637,0.05883527174592018,0.23599779605865479,24.55027315899497,16.538061141967773 +96,0.1767023503780365,0.05857545882463455,0.23527784645557404,24.554933384002652,16.538061141967773 +97,0.1762712299823761,0.05847092345356941,0.2347421944141388,24.5644650059985,16.538061141967773 +98,0.17575272917747498,0.05822795256972313,0.2339806705713272,24.56440415000543,16.538061141967773 +99,0.17502360045909882,0.05808337405323982,0.23310695588588715,24.59411642100895,16.538061141967773 +100,0.17478297650814056,0.05794910341501236,0.23273208737373352,24.616461983008776,16.538061141967773 +101,0.17401571571826935,0.057795241475105286,0.23181091248989105,24.628891633998137,16.538061141967773 +102,0.17342686653137207,0.05761715769767761,0.2310439497232437,24.62149364798097,16.538061141967773 +103,0.17320235073566437,0.057497404515743256,0.23069977760314941,24.621449952945113,16.538061141967773 +104,0.17263327538967133,0.05733602121472359,0.2299693375825882,24.295152470993344,16.538061141967773 +105,0.1724015176296234,0.05717705190181732,0.22957861423492432,23.695310420996975,16.538061141967773 +106,0.17176277935504913,0.057046566158533096,0.2288093864917755,23.83085748803569,16.538061141967773 +107,0.1712954193353653,0.056936196982860565,0.22823160886764526,23.69165078899823,16.538061141967773 +108,0.17059653997421265,0.05674746632575989,0.22734399139881134,23.716627044021152,16.538061141967773 +109,0.17012152075767517,0.05661861225962639,0.22674015164375305,23.719600152981002,16.538061141967773 +110,0.16968640685081482,0.056468356400728226,0.22615471482276917,23.765634846000466,16.538061141967773 +111,0.16913358867168427,0.05633510649204254,0.22546865046024323,23.804762543004472,16.538061141967773 +112,0.1686526983976364,0.05629553645849228,0.2249481976032257,23.796736487012822,16.538061141967773 +113,0.16843630373477936,0.05603635683655739,0.22447268664836884,23.778096321038902,16.538061141967773 +114,0.16786900162696838,0.05593954771757126,0.22380849719047546,23.757725485949777,16.538061141967773 +115,0.16759014129638672,0.055801160633563995,0.2233913540840149,23.77862852095859,16.538061141967773 +116,0.1670101135969162,0.05567202717065811,0.22268210351467133,23.77680180297466,16.538061141967773 +117,0.16650104522705078,0.055556513369083405,0.22205758094787598,23.78127462300472,16.538061141967773 +118,0.16620169579982758,0.05549774318933487,0.22169941663742065,23.776538699981757,16.538061141967773 +119,0.16562703251838684,0.05533304065465927,0.22096014022827148,23.787270841014106,16.538061141967773 +120,0.16519032418727875,0.05519219487905502,0.22038252651691437,23.73171545000514,16.538061141967773 +121,0.16487720608711243,0.055109698325395584,0.2199869006872177,23.678009944967926,16.538061141967773 +122,0.16442511975765228,0.05497782304883003,0.21940290927886963,23.70919721701648,16.538061141967773 +123,0.16415289044380188,0.05492546781897545,0.21907836198806763,23.694206412008498,16.538061141967773 +124,0.16366304457187653,0.054783597588539124,0.21844665706157684,23.715028506994713,16.538061141967773 +125,0.16323766112327576,0.05465357005596161,0.21789129078388214,23.747699711006135,16.538061141967773 +126,0.16291284561157227,0.054551735520362854,0.21746456623077393,23.786654688010458,16.538061141967773 +127,0.16237473487854004,0.05444422364234924,0.21681900322437286,23.787919082969893,16.538061141967773 +128,0.16212289035320282,0.054343000054359436,0.21646589040756226,23.77168129698839,16.538061141967773 +129,0.16162851452827454,0.05420926213264465,0.215837761759758,23.786362488986924,16.538061141967773 +130,0.16136407852172852,0.05413622409105301,0.21550023555755615,23.752092355978675,16.538061141967773 +131,0.16097722947597504,0.05407073348760605,0.21504797041416168,23.77449477801565,16.538061141967773 +132,0.1606559157371521,0.053918223828077316,0.21457412838935852,23.77315111196367,16.538061141967773 +133,0.16039039194583893,0.05385313928127289,0.21424353122711182,23.764979821047746,16.538061141967773 +134,0.15971188247203827,0.05375809967517853,0.21347001194953918,23.792087958019692,16.538061141967773 +135,0.1593227982521057,0.0536891408264637,0.2130119651556015,23.761413988948334,16.538061141967773 +136,0.15932977199554443,0.05354268103837967,0.21287250518798828,23.769856000028085,16.538061141967773 +137,0.1586971879005432,0.053461939096450806,0.21215912699699402,23.69219572696602,16.538061141967773 +138,0.15871387720108032,0.05337854102253914,0.21209242939949036,23.704744291026145,16.538061141967773 +139,0.15818555653095245,0.05331449210643768,0.21149997413158417,23.727004889049567,16.538061141967773 +140,0.1578405797481537,0.05315692350268364,0.21099749207496643,23.693568659015,16.538061141967773 +141,0.15747974812984467,0.05312183126807213,0.2106015831232071,23.71492793399375,16.538061141967773 +142,0.15730968117713928,0.053108297288417816,0.21041804552078247,23.767243537004106,16.538061141967773 +143,0.15671616792678833,0.052934326231479645,0.20965047180652618,23.777266509016044,16.538061141967773 +144,0.1564960926771164,0.052833762019872665,0.20932984352111816,23.746426391007844,16.538061141967773 +145,0.1561020314693451,0.052782438695430756,0.20888449251651764,23.698487881978508,16.538061141967773 +146,0.15576757490634918,0.05271069332957268,0.20847824215888977,23.700259822013322,16.538061141967773 +147,0.15552616119384766,0.05262487754225731,0.20815105736255646,23.70104228099808,16.538061141967773 +148,0.15522246062755585,0.052528221160173416,0.20775067806243896,23.711551253974903,16.538061141967773 +149,0.15482836961746216,0.05246708542108536,0.2072954773902893,23.733055965043604,16.538061141967773 +150,0.1546069085597992,0.05242108926177025,0.20702798664569855,23.774564865976572,16.538061141967773 +151,0.15423183143138885,0.05233398824930191,0.20656581223011017,23.778908938984387,16.538061141967773 +152,0.15396040678024292,0.05224936828017235,0.20620977878570557,23.770856415037997,16.538061141967773 +153,0.15364739298820496,0.052177298814058304,0.20582468807697296,23.68188015499618,16.538061141967773 +154,0.15337620675563812,0.05209318920969963,0.20546939969062805,23.669382177002262,16.538061141967773 +155,0.1531168818473816,0.05202173441648483,0.20513860881328583,23.673730089969467,16.538061141967773 +156,0.15280543267726898,0.05196481943130493,0.2047702670097351,23.64563782903133,16.538061141967773 +157,0.15269321203231812,0.05190523341298103,0.20459845662117004,23.65414381294977,16.538061141967773 +158,0.15227821469306946,0.05184098705649376,0.20411920547485352,23.683104049006943,16.538061141967773 +159,0.15189817547798157,0.05175719037652016,0.203655406832695,23.732676362036727,16.538061141967773 +160,0.15180115401744843,0.05168185010552406,0.2034829705953598,23.7458754350082,16.538061141967773 +161,0.1515081822872162,0.051611557602882385,0.20311973989009857,23.800021574948914,16.538061141967773 +162,0.1512991040945053,0.05158821493387222,0.20288732647895813,23.827102078008465,16.538061141967773 +163,0.15092939138412476,0.051498379558324814,0.20242777466773987,23.84149757999694,16.538061141967773 +164,0.15076136589050293,0.05143243819475174,0.20219384133815765,23.855513966991566,16.538061141967773 +165,0.15062077343463898,0.05138722062110901,0.20200802385807037,23.842798421974294,16.538061141967773 +166,0.15029919147491455,0.051304373890161514,0.20160357654094696,23.832781545992475,16.538061141967773 +167,0.15003839135169983,0.05125632509589195,0.2012947052717209,23.840878254035488,16.538061141967773 +168,0.1497495472431183,0.05121706426143646,0.20096665620803833,23.869667230988853,16.538061141967773 +169,0.14945629239082336,0.05111052095890045,0.2005668431520462,23.857379608030897,16.538061141967773 +170,0.14924299716949463,0.051114678382873535,0.20035766065120697,23.85956668795552,16.538061141967773 +171,0.14895597100257874,0.051036082208156586,0.19999197125434875,23.861297063995153,16.538061141967773 +172,0.14886784553527832,0.05094881355762482,0.19981661438941956,23.864013378042728,16.538061141967773 +173,0.14871758222579956,0.05092042312026024,0.19963808357715607,23.855510408000555,16.538061141967773 +174,0.14824113249778748,0.050866659730672836,0.1991077959537506,23.85922439501155,16.538061141967773 +175,0.14815518260002136,0.05079418420791626,0.19894936680793762,23.875450732011814,16.538061141967773 +176,0.148061603307724,0.050750598311424255,0.19881221652030945,23.86231814196799,16.538061141967773 +177,0.1478702425956726,0.05069762095808983,0.19856786727905273,23.848046708037145,16.538061141967773 +178,0.14758825302124023,0.05065985023975372,0.19824807345867157,23.86471866298234,16.538061141967773 +179,0.14746706187725067,0.050603728741407394,0.19807074964046478,23.877899574988987,16.538061141967773 +180,0.14712682366371155,0.05054830387234688,0.19767510890960693,23.84560279600555,16.538061141967773 +181,0.14688171446323395,0.05050280690193176,0.1973845213651657,23.784910069021862,16.538061141967773 +182,0.14678223431110382,0.0504707396030426,0.19725292921066284,23.77994823397603,16.538061141967773 +183,0.14638876914978027,0.05040860176086426,0.19679732620716095,23.84384081902681,16.538061141967773 +184,0.1463737040758133,0.05038299411535263,0.19675667583942413,23.785318643029314,16.538061141967773 +185,0.14623357355594635,0.05030802637338638,0.19654159247875214,23.79943421902135,16.538061141967773 +186,0.14590610563755035,0.05027090385556221,0.19617699086666107,23.796898415952455,16.538061141967773 +187,0.1457560807466507,0.050236303359270096,0.1959923952817917,23.77103706804337,16.538061141967773 +188,0.14554555714130402,0.05016549676656723,0.19571104645729065,23.788181602023542,16.538061141967773 +189,0.14541082084178925,0.050144631415605545,0.1955554485321045,23.8268586039776,16.538061141967773 +190,0.14527957141399384,0.05008481815457344,0.19536441564559937,23.861469737021253,16.538061141967773 +191,0.14509576559066772,0.05005105584859848,0.1951468139886856,23.881612269033212,16.538061141967773 +192,0.1448913812637329,0.05000263452529907,0.19489403069019318,23.854751775972545,16.538061141967773 +193,0.14474359154701233,0.0499662347137928,0.1947098821401596,23.85181849001674,16.538061141967773 +194,0.1445516049861908,0.04993106797337532,0.19448265433311462,23.80967182200402,16.538061141967773 +195,0.14447973668575287,0.049892161041498184,0.19437189400196075,23.698770824994426,16.538061141967773 +196,0.1442750245332718,0.0498434454202652,0.1941184401512146,23.677202372986358,16.538061141967773 +197,0.14410382509231567,0.04980795457959175,0.1939118653535843,23.69062947202474,16.538061141967773 +198,0.1439322978258133,0.04978352040052414,0.19371575117111206,23.677275185007602,16.538061141967773 +199,0.14386077225208282,0.04974578320980072,0.19360655546188354,23.707563119009137,16.538061141967773 +200,0.14377175271511078,0.049703724682331085,0.19347545504570007,23.720960870035924,16.538061141967773 +201,0.1435139924287796,0.04966637119650841,0.19318035244941711,23.76091081404593,16.538061141967773 +202,0.14334307610988617,0.04965586215257645,0.1929989606142044,23.77442016999703,16.538061141967773 +203,0.143225759267807,0.04960157722234726,0.1928272843360901,23.793196846963838,16.538061141967773 +204,0.14303679764270782,0.04954684525728226,0.19258363544940948,23.797604288964067,16.538061141967773 +205,0.14306218922138214,0.04953490197658539,0.19259706139564514,23.825505372020416,16.538061141967773 +206,0.14276860654354095,0.04950505122542381,0.19227363169193268,23.87863384501543,16.538061141967773 +207,0.14262402057647705,0.04947471618652344,0.1920987069606781,23.895645098004024,16.538061141967773 +208,0.14253586530685425,0.04942796751856804,0.191963791847229,23.87932059902232,16.538061141967773 +209,0.1424424797296524,0.04939675331115723,0.19183920323848724,23.8517420309945,16.538061141967773 +210,0.14229585230350494,0.049366697669029236,0.1916625052690506,23.831401505041867,16.538061141967773 +211,0.1421985924243927,0.0493338368833065,0.19153238832950592,23.860068388981745,16.538061141967773 +212,0.14221687614917755,0.049312468618154526,0.19152940809726715,23.861139524029568,16.538061141967773 +213,0.14195741713047028,0.049289770424366,0.19124718010425568,23.8420457339962,16.538061141967773 +214,0.1418737918138504,0.049245402216911316,0.19111917912960052,23.82056774396915,16.538061141967773 +215,0.14164824783802032,0.04922390356659889,0.19087211787700653,23.815267500001937,16.538061141967773 +216,0.14167577028274536,0.04919085279107094,0.1908666044473648,23.791405806026887,16.538061141967773 +217,0.14155089855194092,0.04917354881763458,0.19072438776493073,23.801227214979008,16.538061141967773 +218,0.141236811876297,0.04914287105202675,0.19037964940071106,23.814554023963865,16.538061141967773 +219,0.14118385314941406,0.04912382364273071,0.19030770659446716,23.82072806096403,16.538061141967773 +220,0.14111948013305664,0.049090541899204254,0.19021008908748627,23.796734970004763,16.538061141967773 +221,0.14110469818115234,0.04906576871871948,0.1901704967021942,23.79326643096283,16.538061141967773 +222,0.14110082387924194,0.04904753342270851,0.19014830887317657,23.7910252790316,16.538061141967773 +223,0.14088870584964752,0.04901157319545746,0.18990030884742737,23.778661962016486,16.538061141967773 +224,0.140785813331604,0.04900513216853142,0.18979091942310333,23.816003638028633,16.538061141967773 +225,0.14066115021705627,0.04896515607833862,0.1896263211965561,23.78999009699328,16.538061141967773 +226,0.14073871076107025,0.04894005507230759,0.18967872858047485,23.78718847502023,16.538061141967773 +227,0.14050818979740143,0.04892006143927574,0.18942826986312866,23.77054475498153,16.538061141967773 +228,0.14039196074008942,0.04890028387308121,0.1892922967672348,23.721191425982397,16.538061141967773 +229,0.14037659764289856,0.048878103494644165,0.18925471603870392,23.763547400012612,16.538061141967773 +230,0.14023607969284058,0.04885786026716232,0.1890939325094223,23.799041750025935,16.538061141967773 +231,0.14016130566596985,0.04884228855371475,0.1890036016702652,23.753545788000338,16.538061141967773 +232,0.1401616334915161,0.04881012812256813,0.18897172808647156,23.772803690982983,16.538061141967773 +233,0.14000864326953888,0.04879477247595787,0.18880338966846466,23.801001053012442,16.538061141967773 +234,0.13992351293563843,0.04877138510346413,0.18869487941265106,23.865786383044906,16.538061141967773 +235,0.13991044461727142,0.04875841364264488,0.1886688470840454,23.876142910972703,16.538061141967773 +236,0.13981367647647858,0.048735588788986206,0.18854926526546478,23.874316830013413,16.538061141967773 +237,0.13969406485557556,0.0487193688750267,0.18841348588466644,23.84539108799072,16.538061141967773 +238,0.1395992934703827,0.04869788512587547,0.18829722702503204,23.852499878965318,16.538061141967773 +239,0.13957557082176208,0.048683688044548035,0.1882593035697937,23.84107511700131,16.538061141967773 +240,0.13960789144039154,0.04867210611701012,0.18828000128269196,23.813290061953012,16.538061141967773 +241,0.13933981955051422,0.04865274205803871,0.18799252808094025,23.676253671990708,16.538061141967773 +242,0.13933274149894714,0.048638805747032166,0.1879715472459793,23.660605672979727,16.538061141967773 +243,0.13914217054843903,0.04861382767558098,0.1877559870481491,23.673258927999996,16.538061141967773 +244,0.1393701732158661,0.04860866069793701,0.1879788041114807,23.70421232195804,16.538061141967773 +245,0.13916251063346863,0.04858509078621864,0.18774765729904175,23.7329161480302,16.538061141967773 +246,0.13909317553043365,0.048568785190582275,0.18766196072101593,23.756109088019002,16.538061141967773 +247,0.13912880420684814,0.048560407012701035,0.1876891851425171,23.801507944997866,16.538061141967773 +248,0.13904748857021332,0.04854503646492958,0.1875925362110138,23.864072584023234,16.538061141967773 +249,0.13900533318519592,0.048534464091062546,0.18753981590270996,23.858778066991363,16.538061141967773 +250,0.13886626064777374,0.048519283533096313,0.18738558888435364,23.854837904975284,16.538061141967773 +251,0.13891996443271637,0.04850304126739502,0.187422975897789,23.836097771010827,16.538061141967773 +252,0.13876789808273315,0.048490893095731735,0.18725880980491638,23.837332123017404,16.538061141967773 +253,0.13873623311519623,0.04848138988018036,0.18721766769886017,23.855241929995827,16.538061141967773 +254,0.13871686160564423,0.04847310483455658,0.18718993663787842,23.796150182024576,16.538061141967773 +255,0.138605996966362,0.04845300316810608,0.18705898523330688,23.80368453101255,16.538061141967773 +256,0.13863694667816162,0.04844534024596214,0.18708229064941406,23.823650475009345,16.538061141967773 +257,0.1386219710111618,0.04843727871775627,0.18705925345420837,23.797417136956938,16.538061141967773 +258,0.13858631253242493,0.048422232270240784,0.18700852990150452,23.8170631119865,16.538061141967773 +259,0.1384792923927307,0.04841376096010208,0.18689313530921936,23.82631194201531,16.538061141967773 +260,0.13850802183151245,0.04840456321835518,0.18691256642341614,23.799606874992605,16.538061141967773 +261,0.13842622935771942,0.04839333891868591,0.18681956827640533,23.864344649016857,16.538061141967773 +262,0.13845708966255188,0.048383768647909164,0.18684087693691254,23.888249506999273,16.538061141967773 +263,0.13848459720611572,0.048378653824329376,0.1868632733821869,23.86052856297465,16.538061141967773 +264,0.13840946555137634,0.04836936295032501,0.18677878379821777,23.842982928967103,16.538061141967773 +265,0.13840030133724213,0.04836057871580124,0.18676088750362396,23.8179495499935,16.538061141967773 +266,0.1384143829345703,0.04835169389843941,0.18676608800888062,23.828313963022083,16.538061141967773 +267,0.13829363882541656,0.048344485461711884,0.18663816154003143,23.84927254798822,16.538061141967773 +268,0.13823345303535461,0.04833903908729553,0.18657250702381134,23.8390042160172,16.538061141967773 +269,0.13828979432582855,0.048330578953027725,0.18662036955356598,23.801049580972176,16.538061141967773 +270,0.13831481337547302,0.04832201823592186,0.18663686513900757,23.757539115962572,16.538061141967773 +271,0.13816916942596436,0.04831814020872116,0.18648725748062134,23.843181282980368,16.538061141967773 +272,0.13815978169441223,0.048311103135347366,0.1864708662033081,23.839611345960293,16.538061141967773 +273,0.1382344514131546,0.048303596675395966,0.1865379959344864,23.845436306961346,16.538061141967773 +274,0.13807271420955658,0.048300325870513916,0.1863730400800705,23.85302015801426,16.538061141967773 +275,0.1379932463169098,0.04829475283622742,0.186287984251976,23.854387201019563,16.538061141967773 +276,0.13806293904781342,0.04828783869743347,0.18635083734989166,23.81611063098535,16.538061141967773 +277,0.13801302015781403,0.04828232154250145,0.18629537522792816,23.815382472996134,16.538061141967773 +278,0.13812442123889923,0.04827794060111046,0.1864023357629776,23.786231826001313,16.538061141967773 +279,0.13798189163208008,0.04827413707971573,0.1862560510635376,23.79814125102712,16.538061141967773 +280,0.13804753124713898,0.04826877638697624,0.18631628155708313,23.831617349991575,16.538061141967773 +281,0.13802313804626465,0.048266030848026276,0.18628916144371033,23.820580891042482,16.538061141967773 +282,0.1378949135541916,0.048262353986501694,0.1861572563648224,23.885267710022163,16.538061141967773 +283,0.13794371485710144,0.048258230090141296,0.18620193004608154,23.888307827000972,16.538061141967773 +284,0.13793954253196716,0.048254627734422684,0.18619415163993835,23.866968904971145,16.538061141967773 +285,0.13792921602725983,0.048251111060380936,0.18618029356002808,23.878962003043853,16.538061141967773 +286,0.1379459947347641,0.04824871942400932,0.1861947476863861,23.880564541032072,16.538061141967773 +287,0.1379547417163849,0.048246633261442184,0.18620139360427856,23.87580955796875,16.538061141967773 +288,0.13796377182006836,0.048243675380945206,0.18620745837688446,23.885554094973486,16.538061141967773 +289,0.13794752955436707,0.048241257667541504,0.18618878722190857,23.886619200988207,16.538061141967773 +290,0.1378968507051468,0.04823939874768257,0.18613626062870026,23.86679526604712,16.538061141967773 +291,0.13780684769153595,0.04823680594563484,0.18604369461536407,23.872018941037823,16.538061141967773 +292,0.13784006237983704,0.04823537543416023,0.18607543408870697,23.87162912101485,16.538061141967773 +293,0.1379750370979309,0.04823374003171921,0.1862087845802307,23.886490018980112,16.538061141967773 +294,0.1379300355911255,0.048232246190309525,0.1861623227596283,23.87077501701424,16.538061141967773 +295,0.1379389613866806,0.04823022335767746,0.18616920709609985,23.872567207028624,16.538061141967773 +296,0.13782326877117157,0.048229340463876724,0.1860525757074356,23.872934015991632,16.538061141967773 +297,0.13787662982940674,0.04822789877653122,0.18610455095767975,23.866060052008834,16.538061141967773 +298,0.1377841830253601,0.048227425664663315,0.18601161241531372,23.838517934957054,16.538061141967773 +299,0.13793353736400604,0.0482260063290596,0.18615959584712982,23.86191218602471,16.538061141967773 +300,0.13796520233154297,0.048226166516542435,0.1861913800239563,23.868265053955838,16.538061141967773 diff --git a/releases/20260913_lr/paper_data/reproduction/nlvr2_MiniCPM_prompt.txt b/releases/20260913_lr/paper_data/reproduction/nlvr2_MiniCPM_prompt.txt new file mode 100644 index 0000000000000000000000000000000000000000..e38d6cb4756c4bfb7057b16c13c35aae67cbfa0b --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/nlvr2_MiniCPM_prompt.txt @@ -0,0 +1,3 @@ +The first image is the left image and the second is the right image. Determine whether the sentence below is true for this ordered pair. A sentence is True only when every part of it is supported by the images. Interpret counts and comparisons exactly, distinguish claims about one image from claims about both images together, and handle words such as all, at least, more, fewer, same, different, and no literally. Ignore details that the sentence does not mention. Answer with exactly True or False and nothing else. + +Sentence: {sentence} diff --git a/releases/20260913_lr/paper_data/reproduction/nlvr2_Qwen3.8_prompt.txt b/releases/20260913_lr/paper_data/reproduction/nlvr2_Qwen3.8_prompt.txt new file mode 100644 index 0000000000000000000000000000000000000000..c9484bde2ffeda72cc84043a2f9678d29b7dc355 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/nlvr2_Qwen3.8_prompt.txt @@ -0,0 +1,3 @@ +The first image is the left image and the second is the right image. Decide whether the sentence below is true about the pair. Count each distinct matching object once in the exact image scope named by the sentence before evaluating more, fewer, equal, at least, at most, or exactly. An identifiable partly visible object still counts, but background patterns do not. Check the named colors, shapes, identities, negations, and spatial relations, and require every stated clause to hold without adding unstated conditions. Reply with exactly True or False and no other text. + +Sentence: {sentence} diff --git a/releases/20260913_lr/paper_data/reproduction/nlvr2_image1.jpg b/releases/20260913_lr/paper_data/reproduction/nlvr2_image1.jpg new file mode 100644 index 0000000000000000000000000000000000000000..014ea0bd8bfd39cff8fc850d686a6b9c9a442db5 Binary files /dev/null and b/releases/20260913_lr/paper_data/reproduction/nlvr2_image1.jpg differ diff --git a/releases/20260913_lr/paper_data/reproduction/nlvr2_image2.jpg b/releases/20260913_lr/paper_data/reproduction/nlvr2_image2.jpg new file mode 100644 index 0000000000000000000000000000000000000000..dbab8223b68fbb082d0a30c4e800f7335bf3180b --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/nlvr2_image2.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0fcf0b34809760ef9f030c321b497762d40bfc4cc6ccdd208edf5db0562a5cde +size 399183 diff --git a/releases/20260913_lr/paper_data/reproduction/prompt_experiment_ledger.csv b/releases/20260913_lr/paper_data/reproduction/prompt_experiment_ledger.csv new file mode 100644 index 0000000000000000000000000000000000000000..7c673b5680bd6ffca76cf46b29f30b14ac5002df --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/prompt_experiment_ledger.csv @@ -0,0 +1,89 @@ +record_id,phase,dataset,model,round,status,prompt_id,prompt_path,input_path,output_path,metrics_path,primary_metric,metric_value,valid_outputs,notes +tune_grefcoco_qwen38_r01,tune,gRefCOCO,Qwen3.8,1,done,grefcoco_qwen_round01,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_01.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_01.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_01_metrics.json,F1_score,0.7533333333333333,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r02,tune,gRefCOCO,Qwen3.8,2,done,grefcoco_qwen_round02,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_02.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_02.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_02_metrics.json,F1_score,0.8033333333333333,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r03,tune,gRefCOCO,Qwen3.8,3,done,grefcoco_qwen_round03,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_03.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_03.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_03_metrics.json,F1_score,0.8133333333333334,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r04,tune,gRefCOCO,Qwen3.8,4,done,grefcoco_qwen_round04,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_04.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_04.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_04_metrics.json,F1_score,0.81,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r05,tune,gRefCOCO,Qwen3.8,5,done,grefcoco_qwen_round05,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_05.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_05.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_05_metrics.json,F1_score,0.8233333333333334,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r06,tune,gRefCOCO,Qwen3.8,6,done,grefcoco_qwen_round06,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_06.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_06.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_06_metrics.json,F1_score,0.8166666666666667,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r07,tune,gRefCOCO,Qwen3.8,7,done,grefcoco_qwen_round07,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_07.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_07.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_07_metrics.json,F1_score,0.8233333333333334,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r08,tune,gRefCOCO,Qwen3.8,8,done,grefcoco_qwen_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_08.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_08.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_08_metrics.json,F1_score,0.8166666666666667,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r09,tune,gRefCOCO,Qwen3.8,9,done,grefcoco_qwen_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_09.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_09.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_09_metrics.json,F1_score,0.8166666666666667,300,vllm_http_gpus_0_1 +tune_grefcoco_qwen38_r10,tune,gRefCOCO,Qwen3.8,10,done,grefcoco_qwen_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/Qwen3.8/round_10.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/round_10.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/round_10_metrics.json,F1_score,0.82,300,vllm_http_gpus_0_1 +tune_grefcoco_minicpm_r01,tune,gRefCOCO,MiniCPM,1,done,grefcoco_minicpm_round01,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_01.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_01.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_01_metrics.json,F1_score,0.35333333333333333,144,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r02,tune,gRefCOCO,MiniCPM,2,done,grefcoco_minicpm_round02,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_02.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_02.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_02_metrics.json,F1_score,0.47,188,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r03,tune,gRefCOCO,MiniCPM,3,done,grefcoco_minicpm_round03,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_03.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_03.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_03_metrics.json,F1_score,0.47333333333333333,197,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r04,tune,gRefCOCO,MiniCPM,4,done,grefcoco_minicpm_round04,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_04.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_04.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_04_metrics.json,F1_score,0.49666666666666665,203,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r05,tune,gRefCOCO,MiniCPM,5,done,grefcoco_minicpm_round05,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_05.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_05.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_05_metrics.json,F1_score,0.53,211,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r06,tune,gRefCOCO,MiniCPM,6,done,grefcoco_minicpm_round06,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_06.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_06.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_06_metrics.json,F1_score,0.49333333333333335,244,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r07,tune,gRefCOCO,MiniCPM,7,done,grefcoco_minicpm_round07,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_07.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_07.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_07_metrics.json,F1_score,0.4666666666666667,184,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r08,tune,gRefCOCO,MiniCPM,8,done,grefcoco_minicpm_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_08.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_08.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_08_metrics.json,F1_score,0.47,241,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r09,tune,gRefCOCO,MiniCPM,9,done,grefcoco_minicpm_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_09.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_09.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_09_metrics.json,F1_score,0.55,219,vllm_http_gpus_2_3 +tune_grefcoco_minicpm_r10,tune,gRefCOCO,MiniCPM,10,done,grefcoco_minicpm_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/gRefCOCO/MiniCPM/round_10.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/train_tune300_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/round_10.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/round_10_metrics.json,F1_score,0.6066666666666667,245,vllm_http_gpus_2_3 +tune_nlvr2_qwen38_r01,tune,NLVR2,Qwen3.8,1,done,nlvr2_Qwen3.8_round01,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_01.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_01.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_01_metrics.json,accuracy,0.9075,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r02,tune,NLVR2,Qwen3.8,2,done,nlvr2_Qwen3.8_round02,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_02.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_02.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_02_metrics.json,accuracy,0.885,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r03,tune,NLVR2,Qwen3.8,3,done,nlvr2_Qwen3.8_round03,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_03.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_03.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_03_metrics.json,accuracy,0.91,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r04,tune,NLVR2,Qwen3.8,4,done,nlvr2_Qwen3.8_round04,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_04.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_04.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_04_metrics.json,accuracy,0.89,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r05,tune,NLVR2,Qwen3.8,5,done,nlvr2_Qwen3.8_round05,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_05.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_05.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_05_metrics.json,accuracy,0.9125,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r06,tune,NLVR2,Qwen3.8,6,done,nlvr2_Qwen3.8_round06,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_06.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_06.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_06_metrics.json,accuracy,0.88,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r07,tune,NLVR2,Qwen3.8,7,done,nlvr2_Qwen3.8_round07,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_07.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_07.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_07_metrics.json,accuracy,0.9075,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r08,tune,NLVR2,Qwen3.8,8,done,nlvr2_Qwen3.8_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_08.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_08.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_08_metrics.json,accuracy,0.915,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r09,tune,NLVR2,Qwen3.8,9,done,nlvr2_Qwen3.8_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_09.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_09.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_09_metrics.json,accuracy,0.9075,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_qwen38_r10,tune,NLVR2,Qwen3.8,10,done,nlvr2_Qwen3.8_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/Qwen3.8/round_10.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/round_10.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/round_10_metrics.json,accuracy,0.905,400,vllm_http_gpus_0_1_two_images_free_decode_thinking_off +tune_nlvr2_minicpm_r01,tune,NLVR2,MiniCPM,1,done,nlvr2_MiniCPM_round01,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_01.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_01.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_01_metrics.json,accuracy,0.89,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r02,tune,NLVR2,MiniCPM,2,done,nlvr2_MiniCPM_round02,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_02.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_02.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_02_metrics.json,accuracy,0.905,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r03,tune,NLVR2,MiniCPM,3,done,nlvr2_MiniCPM_round03,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_03.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_03.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_03_metrics.json,accuracy,0.91,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r04,tune,NLVR2,MiniCPM,4,done,nlvr2_MiniCPM_round04,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_04.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_04.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_04_metrics.json,accuracy,0.8925,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r05,tune,NLVR2,MiniCPM,5,done,nlvr2_MiniCPM_round05,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_05.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_05.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_05_metrics.json,accuracy,0.895,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r06,tune,NLVR2,MiniCPM,6,done,nlvr2_MiniCPM_round06,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_06.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_06.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_06_metrics.json,accuracy,0.825,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r07,tune,NLVR2,MiniCPM,7,done,nlvr2_MiniCPM_round07,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_07.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_07.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_07_metrics.json,accuracy,0.9025,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r08,tune,NLVR2,MiniCPM,8,done,nlvr2_MiniCPM_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_08.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_08.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_08_metrics.json,accuracy,0.905,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r09,tune,NLVR2,MiniCPM,9,done,nlvr2_MiniCPM_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_09.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_09.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_09_metrics.json,accuracy,0.875,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_nlvr2_minicpm_r10,tune,NLVR2,MiniCPM,10,done,nlvr2_MiniCPM_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/NLVR2/MiniCPM/round_10.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/train_tune400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/round_10.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/round_10_metrics.json,accuracy,0.905,400,vllm_http_gpus_2_3_two_images_choice_constrained +tune_cub_qwen38_r01,tune,CUB,Qwen3.8,1,done,cub_qwen_round01,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_01.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_01.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_01_metrics.json,top1_accuracy,0.0675,400,vllm_http_gpus_0_1 +tune_cub_qwen38_r02,tune,CUB,Qwen3.8,2,done,cub_qwen_round02,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_02.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_02.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_02_metrics.json,top1_accuracy,0.07,400,vllm_http_gpus_0_1 +tune_cub_qwen38_r03,tune,CUB,Qwen3.8,3,done,cub_qwen_round03,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_03.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_03.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_03_metrics.json,top1_accuracy,0.4725,400,vllm_http_gpus_0_1 +tune_cub_qwen38_r04,tune,CUB,Qwen3.8,4,done,cub_qwen_round04,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_04.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_04.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_04_metrics.json,top1_accuracy,0.54,400,vllm_http_gpus_0_1_free_decode_thinking_off +tune_cub_qwen38_r05,tune,CUB,Qwen3.8,5,done,cub_Qwen3.8_round05,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_05.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_05.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_05_metrics.json,top1_accuracy,0.5375,400,vllm_http_gpus_0_1_free_decode_thinking_off +tune_cub_qwen38_r06,tune,CUB,Qwen3.8,6,done,cub_Qwen3.8_round06,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_06.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_06.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_06_metrics.json,top1_accuracy,0.54,400,vllm_http_gpus_0_1_free_decode_thinking_off +tune_cub_qwen38_r07,tune,CUB,Qwen3.8,7,done,cub_Qwen3.8_round07,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_07.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_07.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_07_metrics.json,top1_accuracy,0.53,400,vllm_http_gpus_0_1_free_decode_thinking_off +tune_cub_qwen38_r08,tune,CUB,Qwen3.8,8,done,cub_Qwen3.8_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_08.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_08.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_08_metrics.json,top1_accuracy,0.535,400,vllm_http_gpus_0_1_free_decode_thinking_off +tune_cub_qwen38_r09,tune,CUB,Qwen3.8,9,done,cub_Qwen3.8_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_09.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_09.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_09_metrics.json,top1_accuracy,0.535,400,vllm_http_gpus_0_1_free_decode_thinking_off +tune_cub_qwen38_r10,tune,CUB,Qwen3.8,10,done,cub_Qwen3.8_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/Qwen3.8/round_10.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/round_10.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/round_10_metrics.json,top1_accuracy,0.54,400,vllm_http_gpus_0_1_free_decode_thinking_off +tune_cub_minicpm_r01,tune,CUB,MiniCPM,1,done,cub_minicpm_round01,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_01.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_01.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_01_metrics.json,top1_accuracy,0.265,400,vllm_http_gpus_2_3 +tune_cub_minicpm_r02,tune,CUB,MiniCPM,2,done,cub_minicpm_round02,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_02.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_02.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_02_metrics.json,top1_accuracy,0.245,400,vllm_http_gpus_2_3 +tune_cub_minicpm_r03,tune,CUB,MiniCPM,3,done,cub_minicpm_round03,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_03.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_03.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_03_metrics.json,top1_accuracy,0.0025,6,vllm_http_gpus_2_3 +tune_cub_minicpm_r04,tune,CUB,MiniCPM,4,done,cub_minicpm_round04,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_04.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_04.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_04_metrics.json,top1_accuracy,0.2575,400,vllm_http_gpus_2_3_integer_constrained +tune_cub_minicpm_r05,tune,CUB,MiniCPM,5,done,cub_MiniCPM_round05,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_05.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_05.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_05_metrics.json,top1_accuracy,0.1975,400,vllm_http_gpus_2_3_integer_constrained +tune_cub_minicpm_r06,tune,CUB,MiniCPM,6,done,cub_MiniCPM_round06,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_06.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_06.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_06_metrics.json,top1_accuracy,0.265,400,vllm_http_gpus_2_3_integer_constrained +tune_cub_minicpm_r07,tune,CUB,MiniCPM,7,done,cub_MiniCPM_round07,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_07.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_07.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_07_metrics.json,top1_accuracy,0.2525,400,vllm_http_gpus_2_3_integer_constrained +tune_cub_minicpm_r08,tune,CUB,MiniCPM,8,done,cub_MiniCPM_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_08.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_08.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_08_metrics.json,top1_accuracy,0.2375,400,vllm_http_gpus_2_3_integer_constrained +tune_cub_minicpm_r09,tune,CUB,MiniCPM,9,done,cub_MiniCPM_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_09.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_09.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_09_metrics.json,top1_accuracy,0.21,400,vllm_http_gpus_2_3_integer_constrained +tune_cub_minicpm_r10,tune,CUB,MiniCPM,10,done,cub_MiniCPM_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/CUB/MiniCPM/round_10.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260831_cub_qwen_prompt_enhancement/inputs_v2/cub_train_selection400_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/round_10.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/round_10_metrics.json,top1_accuracy,0.2275,400,vllm_http_gpus_2_3_integer_constrained +tune_construction10k_qwen38_r01,tune,Construction10K,Qwen3.8,1,done_existing,v0_baseline,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v0_baseline.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/qwen_tune365_v0.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/qwen_tune365_v0_metrics.json,macro_presence_f1,0.5720962835310611,359,计入既有10轮预算 +tune_construction10k_qwen38_r02,tune,Construction10K,Qwen3.8,2,done_existing,v2_natural,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v2_natural.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/qwen_tune365_v2.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/qwen_tune365_v2_metrics.json,macro_presence_f1,0.6514138475832085,364,计入既有10轮预算 +tune_construction10k_qwen38_r03,tune,Construction10K,Qwen3.8,3,done_existing,v3_qwen_precision,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v3_qwen_precision.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/qwen_tune365_v3.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/qwen_tune365_v3_metrics.json,macro_presence_f1,0.6575004369482635,365,计入既有10轮预算 +tune_construction10k_qwen38_r04,tune,Construction10K,Qwen3.8,4,done_existing,v4_qwen_hybrid,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v4_qwen_hybrid.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/qwen_tune365_v4.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/qwen_tune365_v4_metrics.json,macro_presence_f1,0.6583285943217521,365,计入既有10轮预算 +tune_construction10k_qwen38_r05,tune,Construction10K,Qwen3.8,5,done_existing,v5_qwen_best_parts,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v5_qwen_best_parts.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/qwen_tune365_v5.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/qwen_tune365_v5_metrics.json,macro_presence_f1,0.6427125433562845,365,计入既有10轮预算 +tune_construction10k_qwen38_r06,tune,Construction10K,Qwen3.8,6,done,construction_qwen_round06,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/Qwen3.8/round_06.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/Qwen3.8/round_06.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/Qwen3.8/round_06_metrics.json,macro_presence_f1,0.6885384156138877,364,completed_on_gpus_0_1 +tune_construction10k_qwen38_r07,tune,Construction10K,Qwen3.8,7,done,construction_qwen_round07,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/Qwen3.8/round_07.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/Qwen3.8/round_07.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/Qwen3.8/round_07_metrics.json,macro_presence_f1,0.6993289087907485,365,completed_by_resident_worker_gpus_0_1 +tune_construction10k_qwen38_r08,tune,Construction10K,Qwen3.8,8,done,construction_qwen_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/Qwen3.8/round_08.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/Qwen3.8/round_08.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/Qwen3.8/round_08_metrics.json,macro_presence_f1,0.7167519315871013,364,vllm_http_gpus_0_1 +tune_construction10k_qwen38_r09,tune,Construction10K,Qwen3.8,9,done,construction_qwen_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/Qwen3.8/round_09.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/Qwen3.8/round_09.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/Qwen3.8/round_09_metrics.json,macro_presence_f1,0.7234805012433111,363,vllm_http_gpus_0_1 +tune_construction10k_qwen38_r10,tune,Construction10K,Qwen3.8,10,done,construction_qwen_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/Qwen3.8/round_10.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/Qwen3.8/round_10.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/Qwen3.8/round_10_metrics.json,macro_presence_f1,0.7163038548752834,363,vllm_http_gpus_0_1 +tune_construction10k_minicpm_r01,tune,Construction10K,MiniCPM,1,done_existing,v0_baseline,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v0_baseline.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/minicpm_tune365_v0.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/minicpm_tune365_v0_metrics.json,macro_presence_f1,0.42788608450373156,180,计入既有10轮预算 +tune_construction10k_minicpm_r02,tune,Construction10K,MiniCPM,2,done_existing,v2_natural,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v2_natural.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/minicpm_tune365_v2.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/minicpm_tune365_v2_metrics.json,macro_presence_f1,0.41786938337890406,158,计入既有10轮预算 +tune_construction10k_minicpm_r03,tune,Construction10K,MiniCPM,3,done_existing,v3_minicpm_balanced,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v3_minicpm_balanced.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/minicpm_tune365_v3.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/minicpm_tune365_v3_metrics.json,macro_presence_f1,0.5937555950976168,344,计入既有10轮预算 +tune_construction10k_minicpm_r04,tune,Construction10K,MiniCPM,4,done_existing,v4_minicpm_precision,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v4_minicpm_precision.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/minicpm_tune365_v4.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/minicpm_tune365_v4_metrics.json,macro_presence_f1,0.49105057835006544,197,计入既有10轮预算 +tune_construction10k_minicpm_r05,tune,Construction10K,MiniCPM,5,done_existing,v5_minicpm_moderate,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/prompts/prompt_v5_minicpm_moderate.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/model_outputs/minicpm_tune365_v5.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/audits/minicpm_tune365_v5_metrics.json,macro_presence_f1,0.5977736835475554,335,计入既有10轮预算 +tune_construction10k_minicpm_r06,tune,Construction10K,MiniCPM,6,done,construction_minicpm_round06,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/MiniCPM/round_06.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/MiniCPM/round_06.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/MiniCPM/round_06_metrics.json,macro_presence_f1,0.5538669375412117,362,completed_on_gpus_2_3 +tune_construction10k_minicpm_r07,tune,Construction10K,MiniCPM,7,done,construction_minicpm_round07,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/MiniCPM/round_07.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/MiniCPM/round_07.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/MiniCPM/round_07_metrics.json,macro_presence_f1,0.568082928988004,356,completed_by_resident_worker_gpus_2_3 +tune_construction10k_minicpm_r08,tune,Construction10K,MiniCPM,8,done,construction_minicpm_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/MiniCPM/round_08.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/MiniCPM/round_08.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/MiniCPM/round_08_metrics.json,macro_presence_f1,0.6061216594572615,360,vllm_http_gpus_2_3 +tune_construction10k_minicpm_r09,tune,Construction10K,MiniCPM,9,done,construction_minicpm_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/MiniCPM/round_09.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/MiniCPM/round_09.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/MiniCPM/round_09_metrics.json,macro_presence_f1,0.6185847801297606,362,vllm_http_gpus_2_3 +tune_construction10k_minicpm_r10,tune,Construction10K,MiniCPM,10,done,construction_minicpm_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/prompts/Construction10K/MiniCPM/round_10.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_construction_event_prompt_optimization/inputs/prepared/train_tune_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/MiniCPM/round_10.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/MiniCPM/round_10_metrics.json,macro_presence_f1,0.6027582276653498,351,vllm_http_gpus_2_3 +test_grefcoco_qwen38,test,gRefCOCO,Qwen3.8,final,running,grefcoco_qwen_best_round05,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/best_prompts/gRefCOCO/Qwen3.8.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/final_testA_testB_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/Qwen3.8/final_test.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/Qwen3.8/final_test_metrics.json,,,,vllm_http_gpus_0_1_final_test +test_grefcoco_minicpm,test,gRefCOCO,MiniCPM,final,running,grefcoco_minicpm_best_round10,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/best_prompts/gRefCOCO/MiniCPM.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/gRefCOCO/final_testA_testB_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/gRefCOCO/MiniCPM/final_test.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/gRefCOCO/MiniCPM/final_test_metrics.json,,,,vllm_http_gpus_2_3_final_test +test_nlvr2_qwen38,test,NLVR2,Qwen3.8,final,running,nlvr2_qwen_best_round08,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/best_prompts/NLVR2/Qwen3.8.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/final_test_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/Qwen3.8/final_test.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/Qwen3.8/final_test_metrics.json,,,,vllm_http_gpus_0_1_final_test +test_nlvr2_minicpm,test,NLVR2,MiniCPM,final,running,nlvr2_minicpm_best_round03,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/best_prompts/NLVR2/MiniCPM.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/NLVR2/final_test_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/NLVR2/MiniCPM/final_test.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/NLVR2/MiniCPM/final_test_metrics.json,,,,vllm_http_gpus_2_3_final_test +test_cub_qwen38,test,CUB,Qwen3.8,final,running,cub_qwen_best_round04,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/best_prompts/CUB/Qwen3.8.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/CUB/final_test_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/Qwen3.8/final_test.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/Qwen3.8/final_test_metrics.json,,,,vllm_http_gpus_0_1_final_test +test_cub_minicpm,test,CUB,MiniCPM,final,running,cub_minicpm_best_round01,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/best_prompts/CUB/MiniCPM.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/CUB/final_test_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/CUB/MiniCPM/final_test.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/CUB/MiniCPM/final_test_metrics.json,,,,vllm_http_gpus_2_3_final_test +test_construction10k_qwen38,test,Construction10K,Qwen3.8,final,running,construction10k_qwen_best_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/best_prompts/Construction10K/Qwen3.8.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/Construction10K/final_test_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/Qwen3.8/final_test.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/Qwen3.8/final_test_metrics.json,,,,vllm_http_gpus_0_1_final_test +test_construction10k_minicpm,test,Construction10K,MiniCPM,final,running,construction10k_minicpm_best_round09,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/best_prompts/Construction10K/MiniCPM.txt,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/inputs/Construction10K/final_test_inputs.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/model_outputs/Construction10K/MiniCPM/final_test.jsonl,/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/experiments/20260902_four_dataset_prompt_optimization_70plus8/audits/Construction10K/MiniCPM/final_test_metrics.json,,,,vllm_http_gpus_2_3_final_test diff --git a/releases/20260913_lr/paper_data/reproduction/selected_config.json b/releases/20260913_lr/paper_data/reproduction/selected_config.json new file mode 100644 index 0000000000000000000000000000000000000000..fbd5cb2ee2185fb5dce8a3265abc7382cb1a4f63 --- /dev/null +++ b/releases/20260913_lr/paper_data/reproduction/selected_config.json @@ -0,0 +1,54 @@ +{ + "selected": { + "id": "shared_12", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_12_cub_MiniCPM", + "shared_12_cub_Qwen3.8", + "shared_12_glsim_MiniCPM", + "shared_12_glsim_Qwen3.8", + "shared_12_groundingdino_MiniCPM", + "shared_12_groundingdino_Qwen3.8", + "shared_12_instancevg_MiniCPM", + "shared_12_instancevg_Qwen3.8", + "shared_12_nlvr2_MiniCPM", + "shared_12_nlvr2_Qwen3.8", + "shared_12_vilt_MiniCPM", + "shared_12_vilt_Qwen3.8", + "shared_12_yolo26x_MiniCPM", + "shared_12_yolo26x_Qwen3.8", + "shared_12_rtdetr_x_fp32_MiniCPM", + "shared_12_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9698125, + "grefcoco": 0.7082964368543117, + "nlvr2": 0.8164959896877686, + "construction": 0.1768778191883302 + }, + "validation_mean": 0.6678706864326026, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 67.41994623254868 + }, + "selection_locked_at": 1789043234.3553715, + "completed_stages": [ + "hidden_percent", + "student", + "representation_dim", + "lr", + "target", + "strategy" + ], + "test_loaded_during_selection": false, + "shared_settings_tested": 18, + "individual_fits": 288, + "boundary_note": "No expansion beyond declared ranges: 288-fit cap and only four already-distilled encoder sizes. This is the best observed coordinate-search setting, not a global optimum." +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/results.json b/releases/20260913_lr/paper_data/results.json new file mode 100644 index 0000000000000000000000000000000000000000..5df17fef354b5f63a77c4bba09fb955539545d91 --- /dev/null +++ b/releases/20260913_lr/paper_data/results.json @@ -0,0 +1 @@ +{"protocol": "Five-seed probability-difference router, selected by task-balanced validation Score25 mean minus sample SD. Each seed fixes its own validation-selected threshold on test; frozen endpoint and encoder records.", "source": "Five-seed router study: dse_results/objective_multiseed_20260912/results_five_seeds.json and runs/repeat_*/test_evaluation.json; original specialist/VLM outcomes are reused.", "groups": [{"key": ["cub", "cub", "MiniCPM"], "expert_display": "ResNet", "run": "outputs/full_budget_20260913/runs/full_cub_MiniCPM_s{2026..2030}", "validation_rows": 1200, "test_rows": 5794, "confidence_rule": {"column": "max_probability", "columns": ["entropy", "max_probability", "top_two_margin", "entropy_valid", "max_probability_valid", "top_two_margin_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/cub/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 0, "validation_area_0_25": 0.9058333333333333, "validation_point": {"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9858333333333333, "rescued": 0, "harmed": 0, "missed_rescue": 6, "avoided_harm": 856, "selected_both_correct": 0, "selected_both_wrong": 0}, "test_point": {"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.81429064549534, "rescued": 0, "harmed": 0, "missed_rescue": 232, "avoided_harm": 3411, "selected_both_correct": 0, "selected_both_wrong": 0}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9858333333333333, "rescued": 0, "harmed": 0, "missed_rescue": 6, "avoided_harm": 856, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.305810809135437, "operator": "ge", "actual_fraction": 0.01, "metric": 0.985, "rescued": 4, "harmed": 5, "missed_rescue": 2, "avoided_harm": 851, "selected_both_correct": 1, "selected_both_wrong": 2}, {"budget": 0.02, "threshold": -0.3627302646636963, "operator": "ge", "actual_fraction": 0.02, "metric": 0.98, "rescued": 5, "harmed": 12, "missed_rescue": 1, "avoided_harm": 844, "selected_both_correct": 2, "selected_both_wrong": 5}, {"budget": 0.03, "threshold": -0.4222676455974579, "operator": "ge", "actual_fraction": 0.03, "metric": 0.9741666666666666, "rescued": 5, "harmed": 19, "missed_rescue": 1, "avoided_harm": 837, "selected_both_correct": 5, "selected_both_wrong": 7}, {"budget": 0.04, "threshold": -0.46618229150772095, "operator": "ge", "actual_fraction": 0.04, "metric": 0.9683333333333334, "rescued": 5, "harmed": 26, "missed_rescue": 1, "avoided_harm": 830, "selected_both_correct": 8, "selected_both_wrong": 9}, {"budget": 0.05, "threshold": -0.5066347122192383, "operator": "ge", "actual_fraction": 0.05, "metric": 0.9625, "rescued": 5, "harmed": 33, "missed_rescue": 1, "avoided_harm": 823, "selected_both_correct": 13, "selected_both_wrong": 9}, {"budget": 0.06, "threshold": -0.5393391251564026, "operator": "ge", "actual_fraction": 0.06, "metric": 0.9566666666666667, "rescued": 5, "harmed": 40, "missed_rescue": 1, "avoided_harm": 816, "selected_both_correct": 16, "selected_both_wrong": 11}, {"budget": 0.07, "threshold": -0.5679479837417603, "operator": "ge", "actual_fraction": 0.07, "metric": 0.95, "rescued": 6, "harmed": 49, "missed_rescue": 0, "avoided_harm": 807, "selected_both_correct": 18, "selected_both_wrong": 11}, {"budget": 0.08, "threshold": -0.5907968282699585, "operator": "ge", "actual_fraction": 0.08, "metric": 0.9416666666666667, "rescued": 6, "harmed": 59, "missed_rescue": 0, "avoided_harm": 797, "selected_both_correct": 20, "selected_both_wrong": 11}, {"budget": 0.09, "threshold": -0.6168767809867859, "operator": "ge", "actual_fraction": 0.09, "metric": 0.9333333333333333, "rescued": 6, "harmed": 69, "missed_rescue": 0, "avoided_harm": 787, "selected_both_correct": 22, "selected_both_wrong": 11}, {"budget": 0.1, "threshold": -0.6361178159713745, "operator": "ge", "actual_fraction": 0.1, "metric": 0.9258333333333333, "rescued": 6, "harmed": 78, "missed_rescue": 0, "avoided_harm": 778, "selected_both_correct": 25, "selected_both_wrong": 11}, {"budget": 0.11, "threshold": -0.6547113656997681, "operator": "ge", "actual_fraction": 0.11, "metric": 0.9183333333333333, "rescued": 6, "harmed": 87, "missed_rescue": 0, "avoided_harm": 769, "selected_both_correct": 28, "selected_both_wrong": 11}, {"budget": 0.12, "threshold": -0.6680066585540771, "operator": "ge", "actual_fraction": 0.12, "metric": 0.9091666666666667, "rescued": 6, "harmed": 98, "missed_rescue": 0, "avoided_harm": 758, "selected_both_correct": 29, "selected_both_wrong": 11}, {"budget": 0.13, "threshold": -0.6809748411178589, "operator": "ge", "actual_fraction": 0.13, "metric": 0.9025, "rescued": 6, "harmed": 106, "missed_rescue": 0, "avoided_harm": 750, "selected_both_correct": 33, "selected_both_wrong": 11}, {"budget": 0.14, "threshold": -0.6963143348693848, "operator": "ge", "actual_fraction": 0.14, "metric": 0.895, "rescued": 6, "harmed": 115, "missed_rescue": 0, "avoided_harm": 741, "selected_both_correct": 36, "selected_both_wrong": 11}, {"budget": 0.15, "threshold": -0.7065573930740356, "operator": "ge", "actual_fraction": 0.15, "metric": 0.8866666666666667, "rescued": 6, "harmed": 125, "missed_rescue": 0, "avoided_harm": 731, "selected_both_correct": 38, "selected_both_wrong": 11}, {"budget": 0.16, "threshold": -0.7129085659980774, "operator": "ge", "actual_fraction": 0.16, "metric": 0.8791666666666667, "rescued": 6, "harmed": 134, "missed_rescue": 0, "avoided_harm": 722, "selected_both_correct": 41, "selected_both_wrong": 11}, {"budget": 0.17, "threshold": -0.7240402102470398, "operator": "ge", "actual_fraction": 0.17, "metric": 0.8733333333333333, "rescued": 6, "harmed": 141, "missed_rescue": 0, "avoided_harm": 715, "selected_both_correct": 46, "selected_both_wrong": 11}, {"budget": 0.18, "threshold": -0.7345248460769653, "operator": "ge", "actual_fraction": 0.18, "metric": 0.865, "rescued": 6, "harmed": 151, "missed_rescue": 0, "avoided_harm": 705, "selected_both_correct": 48, "selected_both_wrong": 11}, {"budget": 0.19, "threshold": -0.7443157434463501, "operator": "ge", "actual_fraction": 0.19, "metric": 0.8591666666666666, "rescued": 6, "harmed": 158, "missed_rescue": 0, "avoided_harm": 698, "selected_both_correct": 53, "selected_both_wrong": 11}, {"budget": 0.2, "threshold": -0.7533873319625854, "operator": "ge", "actual_fraction": 0.2, "metric": 0.8525, "rescued": 6, "harmed": 166, "missed_rescue": 0, "avoided_harm": 690, "selected_both_correct": 57, "selected_both_wrong": 11}, {"budget": 0.21, "threshold": -0.7606735825538635, "operator": "ge", "actual_fraction": 0.21, "metric": 0.845, "rescued": 6, "harmed": 175, "missed_rescue": 0, "avoided_harm": 681, "selected_both_correct": 60, "selected_both_wrong": 11}, {"budget": 0.22, "threshold": -0.7657199501991272, "operator": "ge", "actual_fraction": 0.22, "metric": 0.8383333333333334, "rescued": 6, "harmed": 183, "missed_rescue": 0, "avoided_harm": 673, "selected_both_correct": 64, "selected_both_wrong": 11}, {"budget": 0.23, "threshold": -0.7712029218673706, "operator": "ge", "actual_fraction": 0.23, "metric": 0.8316666666666667, "rescued": 6, "harmed": 191, "missed_rescue": 0, "avoided_harm": 665, "selected_both_correct": 68, "selected_both_wrong": 11}, {"budget": 0.24, "threshold": -0.7754930257797241, "operator": "ge", "actual_fraction": 0.24, "metric": 0.825, "rescued": 6, "harmed": 199, "missed_rescue": 0, "avoided_harm": 657, "selected_both_correct": 72, "selected_both_wrong": 11}, {"budget": 0.25, "threshold": -0.7811619639396667, "operator": "ge", "actual_fraction": 0.25, "metric": 0.8175, "rescued": 6, "harmed": 208, "missed_rescue": 0, "avoided_harm": 648, "selected_both_correct": 75, "selected_both_wrong": 11}, {"budget": 0.26, "threshold": -0.7841500043869019, "operator": "ge", "actual_fraction": 0.26, "metric": 0.8108333333333333, "rescued": 6, "harmed": 216, "missed_rescue": 0, "avoided_harm": 640, "selected_both_correct": 79, "selected_both_wrong": 11}, {"budget": 0.27, "threshold": -0.78773033618927, "operator": "ge", "actual_fraction": 0.27, "metric": 0.8033333333333333, "rescued": 6, "harmed": 225, "missed_rescue": 0, "avoided_harm": 631, "selected_both_correct": 82, "selected_both_wrong": 11}, {"budget": 0.28, "threshold": -0.7923226356506348, "operator": "ge", "actual_fraction": 0.28, "metric": 0.7975, "rescued": 6, "harmed": 232, "missed_rescue": 0, "avoided_harm": 624, "selected_both_correct": 87, "selected_both_wrong": 11}, {"budget": 0.29, "threshold": -0.7950519323348999, "operator": "ge", "actual_fraction": 0.29, "metric": 0.79, "rescued": 6, "harmed": 241, "missed_rescue": 0, "avoided_harm": 615, "selected_both_correct": 90, "selected_both_wrong": 11}, {"budget": 0.3, "threshold": -0.800890326499939, "operator": "ge", "actual_fraction": 0.3, "metric": 0.7816666666666666, "rescued": 6, "harmed": 251, "missed_rescue": 0, "avoided_harm": 605, "selected_both_correct": 92, "selected_both_wrong": 11}, {"budget": 0.31, "threshold": -0.8055772185325623, "operator": "ge", "actual_fraction": 0.31, "metric": 0.7733333333333333, "rescued": 6, "harmed": 261, "missed_rescue": 0, "avoided_harm": 595, "selected_both_correct": 94, "selected_both_wrong": 11}, {"budget": 0.32, "threshold": -0.809782087802887, "operator": "ge", "actual_fraction": 0.32, "metric": 0.7675, "rescued": 6, "harmed": 268, "missed_rescue": 0, "avoided_harm": 588, "selected_both_correct": 99, "selected_both_wrong": 11}, {"budget": 0.33, "threshold": -0.8134240508079529, "operator": "ge", "actual_fraction": 0.33, "metric": 0.7575, "rescued": 6, "harmed": 280, "missed_rescue": 0, "avoided_harm": 576, "selected_both_correct": 99, "selected_both_wrong": 11}, {"budget": 0.34, "threshold": -0.8167235255241394, "operator": "ge", "actual_fraction": 0.34, "metric": 0.7516666666666667, "rescued": 6, "harmed": 287, "missed_rescue": 0, "avoided_harm": 569, "selected_both_correct": 104, "selected_both_wrong": 11}, {"budget": 0.35, "threshold": -0.8208380937576294, "operator": "ge", "actual_fraction": 0.35, "metric": 0.7425, "rescued": 6, "harmed": 298, "missed_rescue": 0, "avoided_harm": 558, "selected_both_correct": 105, "selected_both_wrong": 11}, {"budget": 0.36, "threshold": -0.8250406384468079, "operator": "ge", "actual_fraction": 0.36, "metric": 0.7375, "rescued": 6, "harmed": 304, "missed_rescue": 0, "avoided_harm": 552, "selected_both_correct": 111, "selected_both_wrong": 11}, {"budget": 0.37, "threshold": -0.8295461535453796, "operator": "ge", "actual_fraction": 0.37, "metric": 0.7291666666666666, "rescued": 6, "harmed": 314, "missed_rescue": 0, "avoided_harm": 542, "selected_both_correct": 113, "selected_both_wrong": 11}, {"budget": 0.38, "threshold": -0.8328037261962891, "operator": "ge", "actual_fraction": 0.38, "metric": 0.7191666666666666, "rescued": 6, "harmed": 326, "missed_rescue": 0, "avoided_harm": 530, "selected_both_correct": 113, "selected_both_wrong": 11}, {"budget": 0.39, "threshold": -0.8359338641166687, "operator": "ge", "actual_fraction": 0.39, "metric": 0.7116666666666667, "rescued": 6, "harmed": 335, "missed_rescue": 0, "avoided_harm": 521, "selected_both_correct": 116, "selected_both_wrong": 11}, {"budget": 0.4, "threshold": -0.8391940593719482, "operator": "ge", "actual_fraction": 0.4, "metric": 0.7033333333333334, "rescued": 6, "harmed": 345, "missed_rescue": 0, "avoided_harm": 511, "selected_both_correct": 118, "selected_both_wrong": 11}, {"budget": 0.41, "threshold": -0.8424465656280518, "operator": "ge", "actual_fraction": 0.4091666666666667, "metric": 0.6966666666666667, "rescued": 6, "harmed": 353, "missed_rescue": 0, "avoided_harm": 503, "selected_both_correct": 121, "selected_both_wrong": 11}, {"budget": 0.42, "threshold": -0.8460794687271118, "operator": "ge", "actual_fraction": 0.42, "metric": 0.6883333333333334, "rescued": 6, "harmed": 363, "missed_rescue": 0, "avoided_harm": 493, "selected_both_correct": 124, "selected_both_wrong": 11}, {"budget": 0.43, "threshold": -0.8500295281410217, "operator": "ge", "actual_fraction": 0.43, "metric": 0.68, "rescued": 6, "harmed": 373, "missed_rescue": 0, "avoided_harm": 483, "selected_both_correct": 126, "selected_both_wrong": 11}, {"budget": 0.44, "threshold": -0.852453887462616, "operator": "ge", "actual_fraction": 0.44, "metric": 0.6741666666666667, "rescued": 6, "harmed": 380, "missed_rescue": 0, "avoided_harm": 476, "selected_both_correct": 131, "selected_both_wrong": 11}, {"budget": 0.45, "threshold": -0.8539360165596008, "operator": "ge", "actual_fraction": 0.45, "metric": 0.6683333333333333, "rescued": 6, "harmed": 387, "missed_rescue": 0, "avoided_harm": 469, "selected_both_correct": 136, "selected_both_wrong": 11}, {"budget": 0.46, "threshold": -0.8562509417533875, "operator": "ge", "actual_fraction": 0.46, "metric": 0.6608333333333334, "rescued": 6, "harmed": 396, "missed_rescue": 0, "avoided_harm": 460, "selected_both_correct": 139, "selected_both_wrong": 11}, {"budget": 0.47, "threshold": -0.8592684864997864, "operator": "ge", "actual_fraction": 0.47, "metric": 0.6533333333333333, "rescued": 6, "harmed": 405, "missed_rescue": 0, "avoided_harm": 451, "selected_both_correct": 142, "selected_both_wrong": 11}, {"budget": 0.48, "threshold": -0.8621411919593811, "operator": "ge", "actual_fraction": 0.48, "metric": 0.645, "rescued": 6, "harmed": 415, "missed_rescue": 0, "avoided_harm": 441, "selected_both_correct": 144, "selected_both_wrong": 11}, {"budget": 0.49, "threshold": -0.863978385925293, "operator": "ge", "actual_fraction": 0.49, "metric": 0.6391666666666667, "rescued": 6, "harmed": 422, "missed_rescue": 0, "avoided_harm": 434, "selected_both_correct": 149, "selected_both_wrong": 11}, {"budget": 0.5, "threshold": -0.8661247491836548, "operator": "ge", "actual_fraction": 0.5, "metric": 0.6341666666666667, "rescued": 6, "harmed": 428, "missed_rescue": 0, "avoided_harm": 428, "selected_both_correct": 155, "selected_both_wrong": 11}, {"budget": 0.51, "threshold": -0.8688956499099731, "operator": "ge", "actual_fraction": 0.51, "metric": 0.6258333333333334, "rescued": 6, "harmed": 438, "missed_rescue": 0, "avoided_harm": 418, "selected_both_correct": 157, "selected_both_wrong": 11}, {"budget": 0.52, "threshold": -0.8706188201904297, "operator": "ge", "actual_fraction": 0.52, "metric": 0.6191666666666666, "rescued": 6, "harmed": 446, "missed_rescue": 0, "avoided_harm": 410, "selected_both_correct": 161, "selected_both_wrong": 11}, {"budget": 0.53, "threshold": -0.8727357983589172, "operator": "ge", "actual_fraction": 0.53, "metric": 0.6108333333333333, "rescued": 6, "harmed": 456, "missed_rescue": 0, "avoided_harm": 400, "selected_both_correct": 163, "selected_both_wrong": 11}, {"budget": 0.54, "threshold": -0.874894380569458, "operator": "ge", "actual_fraction": 0.54, "metric": 0.6033333333333334, "rescued": 6, "harmed": 465, "missed_rescue": 0, "avoided_harm": 391, "selected_both_correct": 166, "selected_both_wrong": 11}, {"budget": 0.55, "threshold": -0.8764559626579285, "operator": "ge", "actual_fraction": 0.55, "metric": 0.5966666666666667, "rescued": 6, "harmed": 473, "missed_rescue": 0, "avoided_harm": 383, "selected_both_correct": 170, "selected_both_wrong": 11}, {"budget": 0.56, "threshold": -0.879269540309906, "operator": "ge", "actual_fraction": 0.56, "metric": 0.5891666666666666, "rescued": 6, "harmed": 482, "missed_rescue": 0, "avoided_harm": 374, "selected_both_correct": 173, "selected_both_wrong": 11}, {"budget": 0.57, "threshold": -0.881222128868103, "operator": "ge", "actual_fraction": 0.5691666666666667, "metric": 0.5816666666666667, "rescued": 6, "harmed": 491, "missed_rescue": 0, "avoided_harm": 365, "selected_both_correct": 175, "selected_both_wrong": 11}, {"budget": 0.58, "threshold": -0.882451593875885, "operator": "ge", "actual_fraction": 0.58, "metric": 0.575, "rescued": 6, "harmed": 499, "missed_rescue": 0, "avoided_harm": 357, "selected_both_correct": 180, "selected_both_wrong": 11}, {"budget": 0.59, "threshold": -0.8843295574188232, "operator": "ge", "actual_fraction": 0.59, "metric": 0.5675, "rescued": 6, "harmed": 508, "missed_rescue": 0, "avoided_harm": 348, "selected_both_correct": 183, "selected_both_wrong": 11}, {"budget": 0.6, "threshold": -0.8863861560821533, "operator": "ge", "actual_fraction": 0.6, "metric": 0.5608333333333333, "rescued": 6, "harmed": 516, "missed_rescue": 0, "avoided_harm": 340, "selected_both_correct": 187, "selected_both_wrong": 11}, {"budget": 0.61, "threshold": -0.8876380920410156, "operator": "ge", "actual_fraction": 0.61, "metric": 0.5525, "rescued": 6, "harmed": 526, "missed_rescue": 0, "avoided_harm": 330, "selected_both_correct": 189, "selected_both_wrong": 11}, {"budget": 0.62, "threshold": -0.8888679146766663, "operator": "ge", "actual_fraction": 0.62, "metric": 0.5466666666666666, "rescued": 6, "harmed": 533, "missed_rescue": 0, "avoided_harm": 323, "selected_both_correct": 194, "selected_both_wrong": 11}, {"budget": 0.63, "threshold": -0.8905391693115234, "operator": "ge", "actual_fraction": 0.63, "metric": 0.5408333333333334, "rescued": 6, "harmed": 540, "missed_rescue": 0, "avoided_harm": 316, "selected_both_correct": 199, "selected_both_wrong": 11}, {"budget": 0.64, "threshold": -0.8919612169265747, "operator": "ge", "actual_fraction": 0.64, "metric": 0.5325, "rescued": 6, "harmed": 550, "missed_rescue": 0, "avoided_harm": 306, "selected_both_correct": 201, "selected_both_wrong": 11}, {"budget": 0.65, "threshold": -0.8940081596374512, "operator": "ge", "actual_fraction": 0.65, "metric": 0.525, "rescued": 6, "harmed": 559, "missed_rescue": 0, "avoided_harm": 297, "selected_both_correct": 204, "selected_both_wrong": 11}, {"budget": 0.66, "threshold": -0.895609974861145, "operator": "ge", "actual_fraction": 0.66, "metric": 0.5175, "rescued": 6, "harmed": 568, "missed_rescue": 0, "avoided_harm": 288, "selected_both_correct": 207, "selected_both_wrong": 11}, {"budget": 0.67, "threshold": -0.8972456455230713, "operator": "ge", "actual_fraction": 0.67, "metric": 0.5091666666666667, "rescued": 6, "harmed": 578, "missed_rescue": 0, "avoided_harm": 278, "selected_both_correct": 209, "selected_both_wrong": 11}, {"budget": 0.68, "threshold": -0.8986453413963318, "operator": "ge", "actual_fraction": 0.68, "metric": 0.5008333333333334, "rescued": 6, "harmed": 588, "missed_rescue": 0, "avoided_harm": 268, "selected_both_correct": 211, "selected_both_wrong": 11}, {"budget": 0.69, "threshold": -0.8996874094009399, "operator": "ge", "actual_fraction": 0.6891666666666667, "metric": 0.49333333333333335, "rescued": 6, "harmed": 597, "missed_rescue": 0, "avoided_harm": 259, "selected_both_correct": 213, "selected_both_wrong": 11}, {"budget": 0.7, "threshold": -0.9008907079696655, "operator": "ge", "actual_fraction": 0.7, "metric": 0.48333333333333334, "rescued": 6, "harmed": 609, "missed_rescue": 0, "avoided_harm": 247, "selected_both_correct": 214, "selected_both_wrong": 11}, {"budget": 0.71, "threshold": -0.9031620621681213, "operator": "ge", "actual_fraction": 0.71, "metric": 0.47583333333333333, "rescued": 6, "harmed": 618, "missed_rescue": 0, "avoided_harm": 238, "selected_both_correct": 217, "selected_both_wrong": 11}, {"budget": 0.72, "threshold": -0.9056642055511475, "operator": "ge", "actual_fraction": 0.72, "metric": 0.4716666666666667, "rescued": 6, "harmed": 623, "missed_rescue": 0, "avoided_harm": 233, "selected_both_correct": 224, "selected_both_wrong": 11}, {"budget": 0.73, "threshold": -0.9068957567214966, "operator": "ge", "actual_fraction": 0.73, "metric": 0.465, "rescued": 6, "harmed": 631, "missed_rescue": 0, "avoided_harm": 225, "selected_both_correct": 228, "selected_both_wrong": 11}, {"budget": 0.74, "threshold": -0.9093236327171326, "operator": "ge", "actual_fraction": 0.74, "metric": 0.45916666666666667, "rescued": 6, "harmed": 638, "missed_rescue": 0, "avoided_harm": 218, "selected_both_correct": 233, "selected_both_wrong": 11}, {"budget": 0.75, "threshold": -0.9112368822097778, "operator": "ge", "actual_fraction": 0.75, "metric": 0.45166666666666666, "rescued": 6, "harmed": 647, "missed_rescue": 0, "avoided_harm": 209, "selected_both_correct": 236, "selected_both_wrong": 11}, {"budget": 0.76, "threshold": -0.9129221439361572, "operator": "ge", "actual_fraction": 0.76, "metric": 0.44416666666666665, "rescued": 6, "harmed": 656, "missed_rescue": 0, "avoided_harm": 200, "selected_both_correct": 239, "selected_both_wrong": 11}, {"budget": 0.77, "threshold": -0.9144289493560791, "operator": "ge", "actual_fraction": 0.77, "metric": 0.4375, "rescued": 6, "harmed": 664, "missed_rescue": 0, "avoided_harm": 192, "selected_both_correct": 243, "selected_both_wrong": 11}, {"budget": 0.78, "threshold": -0.9158422946929932, "operator": "ge", "actual_fraction": 0.78, "metric": 0.43416666666666665, "rescued": 6, "harmed": 668, "missed_rescue": 0, "avoided_harm": 188, "selected_both_correct": 251, "selected_both_wrong": 11}, {"budget": 0.79, "threshold": -0.9167781472206116, "operator": "ge", "actual_fraction": 0.79, "metric": 0.4266666666666667, "rescued": 6, "harmed": 677, "missed_rescue": 0, "avoided_harm": 179, "selected_both_correct": 254, "selected_both_wrong": 11}, {"budget": 0.8, "threshold": -0.9187189936637878, "operator": "ge", "actual_fraction": 0.8, "metric": 0.42, "rescued": 6, "harmed": 685, "missed_rescue": 0, "avoided_harm": 171, "selected_both_correct": 258, "selected_both_wrong": 11}, {"budget": 0.81, "threshold": -0.9207327365875244, "operator": "ge", "actual_fraction": 0.81, "metric": 0.41333333333333333, "rescued": 6, "harmed": 693, "missed_rescue": 0, "avoided_harm": 163, "selected_both_correct": 262, "selected_both_wrong": 11}, {"budget": 0.82, "threshold": -0.9226345419883728, "operator": "ge", "actual_fraction": 0.8191666666666667, "metric": 0.4075, "rescued": 6, "harmed": 700, "missed_rescue": 0, "avoided_harm": 156, "selected_both_correct": 266, "selected_both_wrong": 11}, {"budget": 0.83, "threshold": -0.9243993759155273, "operator": "ge", "actual_fraction": 0.83, "metric": 0.39916666666666667, "rescued": 6, "harmed": 710, "missed_rescue": 0, "avoided_harm": 146, "selected_both_correct": 269, "selected_both_wrong": 11}, {"budget": 0.84, "threshold": -0.9264947175979614, "operator": "ge", "actual_fraction": 0.84, "metric": 0.39, "rescued": 6, "harmed": 721, "missed_rescue": 0, "avoided_harm": 135, "selected_both_correct": 270, "selected_both_wrong": 11}, {"budget": 0.85, "threshold": -0.9274600148200989, "operator": "ge", "actual_fraction": 0.85, "metric": 0.38416666666666666, "rescued": 6, "harmed": 728, "missed_rescue": 0, "avoided_harm": 128, "selected_both_correct": 275, "selected_both_wrong": 11}, {"budget": 0.86, "threshold": -0.930335283279419, "operator": "ge", "actual_fraction": 0.86, "metric": 0.37833333333333335, "rescued": 6, "harmed": 735, "missed_rescue": 0, "avoided_harm": 121, "selected_both_correct": 280, "selected_both_wrong": 11}, {"budget": 0.87, "threshold": -0.9319414496421814, "operator": "ge", "actual_fraction": 0.87, "metric": 0.37166666666666665, "rescued": 6, "harmed": 743, "missed_rescue": 0, "avoided_harm": 113, "selected_both_correct": 284, "selected_both_wrong": 11}, {"budget": 0.88, "threshold": -0.9340114593505859, "operator": "ge", "actual_fraction": 0.88, "metric": 0.36583333333333334, "rescued": 6, "harmed": 750, "missed_rescue": 0, "avoided_harm": 106, "selected_both_correct": 289, "selected_both_wrong": 11}, {"budget": 0.89, "threshold": -0.9363258481025696, "operator": "ge", "actual_fraction": 0.89, "metric": 0.3591666666666667, "rescued": 6, "harmed": 758, "missed_rescue": 0, "avoided_harm": 98, "selected_both_correct": 293, "selected_both_wrong": 11}, {"budget": 0.9, "threshold": -0.9400071501731873, "operator": "ge", "actual_fraction": 0.9, "metric": 0.35, "rescued": 6, "harmed": 769, "missed_rescue": 0, "avoided_harm": 87, "selected_both_correct": 294, "selected_both_wrong": 11}, {"budget": 0.91, "threshold": -0.9423198103904724, "operator": "ge", "actual_fraction": 0.91, "metric": 0.3425, "rescued": 6, "harmed": 778, "missed_rescue": 0, "avoided_harm": 78, "selected_both_correct": 297, "selected_both_wrong": 11}, {"budget": 0.92, "threshold": -0.9443356394767761, "operator": "ge", "actual_fraction": 0.92, "metric": 0.335, "rescued": 6, "harmed": 787, "missed_rescue": 0, "avoided_harm": 69, "selected_both_correct": 300, "selected_both_wrong": 11}, {"budget": 0.93, "threshold": -0.9468213319778442, "operator": "ge", "actual_fraction": 0.93, "metric": 0.3283333333333333, "rescued": 6, "harmed": 795, "missed_rescue": 0, "avoided_harm": 61, "selected_both_correct": 304, "selected_both_wrong": 11}, {"budget": 0.94, "threshold": -0.9496435523033142, "operator": "ge", "actual_fraction": 0.94, "metric": 0.32083333333333336, "rescued": 6, "harmed": 804, "missed_rescue": 0, "avoided_harm": 52, "selected_both_correct": 307, "selected_both_wrong": 11}, {"budget": 0.95, "threshold": -0.9529175758361816, "operator": "ge", "actual_fraction": 0.95, "metric": 0.3125, "rescued": 6, "harmed": 814, "missed_rescue": 0, "avoided_harm": 42, "selected_both_correct": 309, "selected_both_wrong": 11}, {"budget": 0.96, "threshold": -0.9560379981994629, "operator": "ge", "actual_fraction": 0.96, "metric": 0.305, "rescued": 6, "harmed": 823, "missed_rescue": 0, "avoided_harm": 33, "selected_both_correct": 312, "selected_both_wrong": 11}, {"budget": 0.97, "threshold": -0.9591385126113892, "operator": "ge", "actual_fraction": 0.97, "metric": 0.2991666666666667, "rescued": 6, "harmed": 830, "missed_rescue": 0, "avoided_harm": 26, "selected_both_correct": 317, "selected_both_wrong": 11}, {"budget": 0.98, "threshold": -0.9627243280410767, "operator": "ge", "actual_fraction": 0.98, "metric": 0.2925, "rescued": 6, "harmed": 838, "missed_rescue": 0, "avoided_harm": 18, "selected_both_correct": 321, "selected_both_wrong": 11}, {"budget": 0.99, "threshold": -0.9702208638191223, "operator": "ge", "actual_fraction": 0.99, "metric": 0.28583333333333333, "rescued": 6, "harmed": 846, "missed_rescue": 0, "avoided_harm": 10, "selected_both_correct": 325, "selected_both_wrong": 11}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.2775, "rescued": 6, "harmed": 856, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 327, "selected_both_wrong": 11}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.81429064549534, "rescued": 0, "harmed": 0, "missed_rescue": 232, "avoided_harm": 3411, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.305810809135437, "operator": "ge", "actual_fraction": 0.113565757680359, "metric": 0.799275112185019, "rescued": 88, "harmed": 175, "missed_rescue": 144, "avoided_harm": 3236, "selected_both_correct": 56, "selected_both_wrong": 339}, {"budget": 0.02, "threshold": -0.3627302646636963, "operator": "ge", "actual_fraction": 0.15947531929582326, "metric": 0.7839143942008975, "rescued": 113, "harmed": 289, "missed_rescue": 119, "avoided_harm": 3122, "selected_both_correct": 88, "selected_both_wrong": 434}, {"budget": 0.03, "threshold": -0.4222676455974579, "operator": "ge", "actual_fraction": 0.2143596824301001, "metric": 0.7611322057300656, "rescued": 142, "harmed": 450, "missed_rescue": 90, "avoided_harm": 2961, "selected_both_correct": 128, "selected_both_wrong": 522}, {"budget": 0.04, "threshold": -0.46618229150772095, "operator": "ge", "actual_fraction": 0.2545736969278564, "metric": 0.7430100103555403, "rescued": 159, "harmed": 572, "missed_rescue": 73, "avoided_harm": 2839, "selected_both_correct": 159, "selected_both_wrong": 585}, {"budget": 0.05, "threshold": -0.5066347122192383, "operator": "ge", "actual_fraction": 0.29375215740421123, "metric": 0.7229892992751122, "rescued": 170, "harmed": 699, "missed_rescue": 62, "avoided_harm": 2712, "selected_both_correct": 209, "selected_both_wrong": 624}, {"budget": 0.06, "threshold": -0.5393391251564026, "operator": "ge", "actual_fraction": 0.3208491542975492, "metric": 0.7102174663444943, "rescued": 176, "harmed": 779, "missed_rescue": 56, "avoided_harm": 2632, "selected_both_correct": 243, "selected_both_wrong": 661}, {"budget": 0.07, "threshold": -0.5679479837417603, "operator": "ge", "actual_fraction": 0.350707628581291, "metric": 0.6948567483603728, "rescued": 190, "harmed": 882, "missed_rescue": 42, "avoided_harm": 2529, "selected_both_correct": 270, "selected_both_wrong": 690}, {"budget": 0.08, "threshold": -0.5907968282699585, "operator": "ge", "actual_fraction": 0.36882982395581637, "metric": 0.684328615809458, "rescued": 195, "harmed": 948, "missed_rescue": 37, "avoided_harm": 2463, "selected_both_correct": 289, "selected_both_wrong": 705}, {"budget": 0.09, "threshold": -0.6168767809867859, "operator": "ge", "actual_fraction": 0.391439420089748, "metric": 0.6720745598895409, "rescued": 201, "harmed": 1025, "missed_rescue": 31, "avoided_harm": 2386, "selected_both_correct": 319, "selected_both_wrong": 723}, {"budget": 0.1, "threshold": -0.6361178159713745, "operator": "ge", "actual_fraction": 0.4138764238867794, "metric": 0.6601656886434243, "rescued": 208, "harmed": 1101, "missed_rescue": 24, "avoided_harm": 2310, "selected_both_correct": 349, "selected_both_wrong": 740}, {"budget": 0.11, "threshold": -0.6547113656997681, "operator": "ge", "actual_fraction": 0.43372454263030724, "metric": 0.6482568173973076, "rescued": 212, "harmed": 1174, "missed_rescue": 20, "avoided_harm": 2237, "selected_both_correct": 373, "selected_both_wrong": 754}, {"budget": 0.12, "threshold": -0.6680066585540771, "operator": "ge", "actual_fraction": 0.4480497065930273, "metric": 0.6384190541939938, "rescued": 212, "harmed": 1231, "missed_rescue": 20, "avoided_harm": 2180, "selected_both_correct": 394, "selected_both_wrong": 759}, {"budget": 0.13, "threshold": -0.6809748411178589, "operator": "ge", "actual_fraction": 0.4630652399033483, "metric": 0.6287538833275803, "rescued": 214, "harmed": 1289, "missed_rescue": 18, "avoided_harm": 2122, "selected_both_correct": 412, "selected_both_wrong": 768}, {"budget": 0.14, "threshold": -0.6963143348693848, "operator": "ge", "actual_fraction": 0.479288919571971, "metric": 0.617880566102865, "rescued": 217, "harmed": 1355, "missed_rescue": 15, "avoided_harm": 2056, "selected_both_correct": 433, "selected_both_wrong": 772}, {"budget": 0.15, "threshold": -0.7065573930740356, "operator": "ge", "actual_fraction": 0.4932688988608906, "metric": 0.6102865032792544, "rescued": 220, "harmed": 1402, "missed_rescue": 12, "avoided_harm": 2009, "selected_both_correct": 457, "selected_both_wrong": 779}, {"budget": 0.16, "threshold": -0.7129085659980774, "operator": "ge", "actual_fraction": 0.5008629616845012, "metric": 0.6054539178460476, "rescued": 221, "harmed": 1431, "missed_rescue": 11, "avoided_harm": 1980, "selected_both_correct": 468, "selected_both_wrong": 782}, {"budget": 0.17, "threshold": -0.7240402102470398, "operator": "ge", "actual_fraction": 0.5155333103210218, "metric": 0.5944080082844322, "rescued": 221, "harmed": 1495, "missed_rescue": 11, "avoided_harm": 1916, "selected_both_correct": 487, "selected_both_wrong": 784}, {"budget": 0.18, "threshold": -0.7345248460769653, "operator": "ge", "actual_fraction": 0.5298584742837418, "metric": 0.5845702450811184, "rescued": 223, "harmed": 1554, "missed_rescue": 9, "avoided_harm": 1857, "selected_both_correct": 502, "selected_both_wrong": 791}, {"budget": 0.19, "threshold": -0.7443157434463501, "operator": "ge", "actual_fraction": 0.5455643769416638, "metric": 0.5745598895409044, "rescued": 224, "harmed": 1613, "missed_rescue": 8, "avoided_harm": 1798, "selected_both_correct": 529, "selected_both_wrong": 795}, {"budget": 0.2, "threshold": -0.7533873319625854, "operator": "ge", "actual_fraction": 0.5560925094925785, "metric": 0.5674836037279944, "rescued": 226, "harmed": 1656, "missed_rescue": 6, "avoided_harm": 1755, "selected_both_correct": 542, "selected_both_wrong": 798}, {"budget": 0.21, "threshold": -0.7606735825538635, "operator": "ge", "actual_fraction": 0.5676561960648947, "metric": 0.5598895409043838, "rescued": 226, "harmed": 1700, "missed_rescue": 6, "avoided_harm": 1711, "selected_both_correct": 563, "selected_both_wrong": 800}, {"budget": 0.22, "threshold": -0.7657199501991272, "operator": "ge", "actual_fraction": 0.5766309975837073, "metric": 0.5541939937866759, "rescued": 226, "harmed": 1733, "missed_rescue": 6, "avoided_harm": 1678, "selected_both_correct": 580, "selected_both_wrong": 802}, {"budget": 0.23, "threshold": -0.7712029218673706, "operator": "ge", "actual_fraction": 0.5861235761132205, "metric": 0.5479806696582672, "rescued": 226, "harmed": 1769, "missed_rescue": 6, "avoided_harm": 1642, "selected_both_correct": 597, "selected_both_wrong": 804}, {"budget": 0.24, "threshold": -0.7754930257797241, "operator": "ge", "actual_fraction": 0.5923369002416292, "metric": 0.5434932688988608, "rescued": 226, "harmed": 1795, "missed_rescue": 6, "avoided_harm": 1616, "selected_both_correct": 604, "selected_both_wrong": 807}, {"budget": 0.25, "threshold": -0.7811619639396667, "operator": "ge", "actual_fraction": 0.6013117017604418, "metric": 0.5367621677597515, "rescued": 226, "harmed": 1834, "missed_rescue": 6, "avoided_harm": 1577, "selected_both_correct": 616, "selected_both_wrong": 808}, {"budget": 0.26, "threshold": -0.7841500043869019, "operator": "ge", "actual_fraction": 0.6052813255091474, "metric": 0.5345184673800484, "rescued": 226, "harmed": 1847, "missed_rescue": 6, "avoided_harm": 1564, "selected_both_correct": 625, "selected_both_wrong": 809}, {"budget": 0.27, "threshold": -0.78773033618927, "operator": "ge", "actual_fraction": 0.6123576113220573, "metric": 0.5291681049361409, "rescued": 227, "harmed": 1879, "missed_rescue": 5, "avoided_harm": 1532, "selected_both_correct": 632, "selected_both_wrong": 810}, {"budget": 0.28, "threshold": -0.7923226356506348, "operator": "ge", "actual_fraction": 0.6206420434932689, "metric": 0.5241629271660339, "rescued": 227, "harmed": 1908, "missed_rescue": 5, "avoided_harm": 1503, "selected_both_correct": 649, "selected_both_wrong": 812}, {"budget": 0.29, "threshold": -0.7950519323348999, "operator": "ge", "actual_fraction": 0.6251294442526751, "metric": 0.52140144977563, "rescued": 227, "harmed": 1924, "missed_rescue": 5, "avoided_harm": 1487, "selected_both_correct": 658, "selected_both_wrong": 813}, {"budget": 0.3, "threshold": -0.800890326499939, "operator": "ge", "actual_fraction": 0.6373835001725924, "metric": 0.5117362789092165, "rescued": 227, "harmed": 1980, "missed_rescue": 5, "avoided_harm": 1431, "selected_both_correct": 672, "selected_both_wrong": 814}, {"budget": 0.31, "threshold": -0.8055772185325623, "operator": "ge", "actual_fraction": 0.6470486710390059, "metric": 0.5055229547808078, "rescued": 227, "harmed": 2016, "missed_rescue": 5, "avoided_harm": 1395, "selected_both_correct": 691, "selected_both_wrong": 815}, {"budget": 0.32, "threshold": -0.809782087802887, "operator": "ge", "actual_fraction": 0.6556782878840179, "metric": 0.5005177770107008, "rescued": 227, "harmed": 2045, "missed_rescue": 5, "avoided_harm": 1366, "selected_both_correct": 708, "selected_both_wrong": 819}, {"budget": 0.33, "threshold": -0.8134240508079529, "operator": "ge", "actual_fraction": 0.6636175353814291, "metric": 0.4955125992405937, "rescued": 227, "harmed": 2074, "missed_rescue": 5, "avoided_harm": 1337, "selected_both_correct": 724, "selected_both_wrong": 820}, {"budget": 0.34, "threshold": -0.8167235255241394, "operator": "ge", "actual_fraction": 0.6708664135312392, "metric": 0.489989644459786, "rescued": 227, "harmed": 2106, "missed_rescue": 5, "avoided_harm": 1305, "selected_both_correct": 733, "selected_both_wrong": 821}, {"budget": 0.35, "threshold": -0.8208380937576294, "operator": "ge", "actual_fraction": 0.6812219537452537, "metric": 0.4830859509837763, "rescued": 228, "harmed": 2147, "missed_rescue": 4, "avoided_harm": 1264, "selected_both_correct": 751, "selected_both_wrong": 821}, {"budget": 0.36, "threshold": -0.8250406384468079, "operator": "ge", "actual_fraction": 0.689333793579565, "metric": 0.4777355885398688, "rescued": 228, "harmed": 2178, "missed_rescue": 4, "avoided_harm": 1233, "selected_both_correct": 765, "selected_both_wrong": 823}, {"budget": 0.37, "threshold": -0.8295461535453796, "operator": "ge", "actual_fraction": 0.7017604418363824, "metric": 0.468588194684156, "rescued": 228, "harmed": 2231, "missed_rescue": 4, "avoided_harm": 1180, "selected_both_correct": 782, "selected_both_wrong": 825}, {"budget": 0.38, "threshold": -0.8328037261962891, "operator": "ge", "actual_fraction": 0.7088367276492924, "metric": 0.463583016914049, "rescued": 228, "harmed": 2260, "missed_rescue": 4, "avoided_harm": 1151, "selected_both_correct": 793, "selected_both_wrong": 826}, {"budget": 0.39, "threshold": -0.8359338641166687, "operator": "ge", "actual_fraction": 0.7166033828098033, "metric": 0.4577148774594408, "rescued": 228, "harmed": 2294, "missed_rescue": 4, "avoided_harm": 1117, "selected_both_correct": 804, "selected_both_wrong": 826}, {"budget": 0.4, "threshold": -0.8391940593719482, "operator": "ge", "actual_fraction": 0.7238522609596134, "metric": 0.4527096996893338, "rescued": 229, "harmed": 2324, "missed_rescue": 3, "avoided_harm": 1087, "selected_both_correct": 814, "selected_both_wrong": 827}, {"budget": 0.41, "threshold": -0.8424465656280518, "operator": "ge", "actual_fraction": 0.7302381774249224, "metric": 0.44839489126682774, "rescued": 229, "harmed": 2349, "missed_rescue": 3, "avoided_harm": 1062, "selected_both_correct": 825, "selected_both_wrong": 828}, {"budget": 0.42, "threshold": -0.8460794687271118, "operator": "ge", "actual_fraction": 0.7392129789437349, "metric": 0.4418363824646186, "rescued": 229, "harmed": 2387, "missed_rescue": 3, "avoided_harm": 1024, "selected_both_correct": 839, "selected_both_wrong": 828}, {"budget": 0.43, "threshold": -0.8500295281410217, "operator": "ge", "actual_fraction": 0.747152226441146, "metric": 0.43596824301001036, "rescued": 229, "harmed": 2421, "missed_rescue": 3, "avoided_harm": 990, "selected_both_correct": 851, "selected_both_wrong": 828}, {"budget": 0.44, "threshold": -0.852453887462616, "operator": "ge", "actual_fraction": 0.7523299965481532, "metric": 0.4325163962720055, "rescued": 229, "harmed": 2441, "missed_rescue": 3, "avoided_harm": 970, "selected_both_correct": 860, "selected_both_wrong": 829}, {"budget": 0.45, "threshold": -0.8539360165596008, "operator": "ge", "actual_fraction": 0.7562996202968588, "metric": 0.42975491888160167, "rescued": 229, "harmed": 2457, "missed_rescue": 3, "avoided_harm": 954, "selected_both_correct": 867, "selected_both_wrong": 829}, {"budget": 0.46, "threshold": -0.8562509417533875, "operator": "ge", "actual_fraction": 0.7613047980669658, "metric": 0.42595788746979635, "rescued": 229, "harmed": 2479, "missed_rescue": 3, "avoided_harm": 932, "selected_both_correct": 874, "selected_both_wrong": 829}, {"budget": 0.47, "threshold": -0.8592684864997864, "operator": "ge", "actual_fraction": 0.7668277528477736, "metric": 0.4219882637210908, "rescued": 229, "harmed": 2502, "missed_rescue": 3, "avoided_harm": 909, "selected_both_correct": 882, "selected_both_wrong": 830}, {"budget": 0.48, "threshold": -0.8621411919593811, "operator": "ge", "actual_fraction": 0.774249223334484, "metric": 0.4161201242664826, "rescued": 230, "harmed": 2537, "missed_rescue": 2, "avoided_harm": 874, "selected_both_correct": 888, "selected_both_wrong": 831}, {"budget": 0.49, "threshold": -0.863978385925293, "operator": "ge", "actual_fraction": 0.7790818087676907, "metric": 0.41266827752847773, "rescued": 230, "harmed": 2557, "missed_rescue": 2, "avoided_harm": 854, "selected_both_correct": 895, "selected_both_wrong": 832}, {"budget": 0.5, "threshold": -0.8661247491836548, "operator": "ge", "actual_fraction": 0.7849499482222989, "metric": 0.40852606144287196, "rescued": 230, "harmed": 2581, "missed_rescue": 2, "avoided_harm": 830, "selected_both_correct": 904, "selected_both_wrong": 833}, {"budget": 0.51, "threshold": -0.8688956499099731, "operator": "ge", "actual_fraction": 0.7928891957197101, "metric": 0.402830514325164, "rescued": 230, "harmed": 2614, "missed_rescue": 2, "avoided_harm": 797, "selected_both_correct": 916, "selected_both_wrong": 834}, {"budget": 0.52, "threshold": -0.8706188201904297, "operator": "ge", "actual_fraction": 0.7970314118053159, "metric": 0.39972385226095963, "rescued": 230, "harmed": 2632, "missed_rescue": 2, "avoided_harm": 779, "selected_both_correct": 922, "selected_both_wrong": 834}, {"budget": 0.53, "threshold": -0.8727357983589172, "operator": "ge", "actual_fraction": 0.8016914049016224, "metric": 0.39748015188125646, "rescued": 230, "harmed": 2645, "missed_rescue": 2, "avoided_harm": 766, "selected_both_correct": 936, "selected_both_wrong": 834}, {"budget": 0.54, "threshold": -0.874894380569458, "operator": "ge", "actual_fraction": 0.8070417673455299, "metric": 0.39368312046945114, "rescued": 230, "harmed": 2667, "missed_rescue": 2, "avoided_harm": 744, "selected_both_correct": 945, "selected_both_wrong": 834}, {"budget": 0.55, "threshold": -0.8764559626579285, "operator": "ge", "actual_fraction": 0.8120469451156369, "metric": 0.39040386606834654, "rescued": 230, "harmed": 2686, "missed_rescue": 2, "avoided_harm": 725, "selected_both_correct": 955, "selected_both_wrong": 834}, {"budget": 0.56, "threshold": -0.879269540309906, "operator": "ge", "actual_fraction": 0.8180876769071453, "metric": 0.3860890576458405, "rescued": 230, "harmed": 2711, "missed_rescue": 2, "avoided_harm": 700, "selected_both_correct": 964, "selected_both_wrong": 835}, {"budget": 0.57, "threshold": -0.881222128868103, "operator": "ge", "actual_fraction": 0.823610631687953, "metric": 0.3833275802554367, "rescued": 231, "harmed": 2728, "missed_rescue": 1, "avoided_harm": 683, "selected_both_correct": 976, "selected_both_wrong": 837}, {"budget": 0.58, "threshold": -0.882451593875885, "operator": "ge", "actual_fraction": 0.8272350707628582, "metric": 0.38073869520193304, "rescued": 231, "harmed": 2743, "missed_rescue": 1, "avoided_harm": 668, "selected_both_correct": 982, "selected_both_wrong": 837}, {"budget": 0.59, "threshold": -0.8843295574188232, "operator": "ge", "actual_fraction": 0.8313772868484639, "metric": 0.37797721781152915, "rescued": 232, "harmed": 2760, "missed_rescue": 0, "avoided_harm": 651, "selected_both_correct": 988, "selected_both_wrong": 837}, {"budget": 0.6, "threshold": -0.8863861560821533, "operator": "ge", "actual_fraction": 0.8355195029340697, "metric": 0.3748705557473248, "rescued": 232, "harmed": 2778, "missed_rescue": 0, "avoided_harm": 633, "selected_both_correct": 994, "selected_both_wrong": 837}, {"budget": 0.61, "threshold": -0.8876380920410156, "operator": "ge", "actual_fraction": 0.8391439420089748, "metric": 0.3726268553676217, "rescued": 232, "harmed": 2791, "missed_rescue": 0, "avoided_harm": 620, "selected_both_correct": 1002, "selected_both_wrong": 837}, {"budget": 0.62, "threshold": -0.8888679146766663, "operator": "ge", "actual_fraction": 0.8425957887469796, "metric": 0.3702105626510183, "rescued": 232, "harmed": 2805, "missed_rescue": 0, "avoided_harm": 606, "selected_both_correct": 1008, "selected_both_wrong": 837}, {"budget": 0.63, "threshold": -0.8905391693115234, "operator": "ge", "actual_fraction": 0.8469105971694857, "metric": 0.3669313082499137, "rescued": 232, "harmed": 2824, "missed_rescue": 0, "avoided_harm": 587, "selected_both_correct": 1014, "selected_both_wrong": 837}, {"budget": 0.64, "threshold": -0.8919612169265747, "operator": "ge", "actual_fraction": 0.8503624439074905, "metric": 0.36468760787021054, "rescued": 232, "harmed": 2837, "missed_rescue": 0, "avoided_harm": 574, "selected_both_correct": 1021, "selected_both_wrong": 837}, {"budget": 0.65, "threshold": -0.8940081596374512, "operator": "ge", "actual_fraction": 0.8560579910251985, "metric": 0.360717984121505, "rescued": 232, "harmed": 2860, "missed_rescue": 0, "avoided_harm": 551, "selected_both_correct": 1031, "selected_both_wrong": 837}, {"budget": 0.66, "threshold": -0.895609974861145, "operator": "ge", "actual_fraction": 0.8603727994477045, "metric": 0.3574387297204004, "rescued": 232, "harmed": 2879, "missed_rescue": 0, "avoided_harm": 532, "selected_both_correct": 1036, "selected_both_wrong": 838}, {"budget": 0.67, "threshold": -0.8972456455230713, "operator": "ge", "actual_fraction": 0.8643424231964101, "metric": 0.3548498446668968, "rescued": 232, "harmed": 2894, "missed_rescue": 0, "avoided_harm": 517, "selected_both_correct": 1043, "selected_both_wrong": 839}, {"budget": 0.68, "threshold": -0.8986453413963318, "operator": "ge", "actual_fraction": 0.8688298239558163, "metric": 0.35174318260269244, "rescued": 232, "harmed": 2912, "missed_rescue": 0, "avoided_harm": 499, "selected_both_correct": 1051, "selected_both_wrong": 839}, {"budget": 0.69, "threshold": -0.8996874094009399, "operator": "ge", "actual_fraction": 0.8712461166724197, "metric": 0.3496720745598895, "rescued": 232, "harmed": 2924, "missed_rescue": 0, "avoided_harm": 487, "selected_both_correct": 1053, "selected_both_wrong": 839}, {"budget": 0.7, "threshold": -0.9008907079696655, "operator": "ge", "actual_fraction": 0.8746979634104246, "metric": 0.3470831895063859, "rescued": 232, "harmed": 2939, "missed_rescue": 0, "avoided_harm": 472, "selected_both_correct": 1058, "selected_both_wrong": 839}, {"budget": 0.71, "threshold": -0.9031620621681213, "operator": "ge", "actual_fraction": 0.8817742492233345, "metric": 0.34311356575768037, "rescued": 232, "harmed": 2962, "missed_rescue": 0, "avoided_harm": 449, "selected_both_correct": 1076, "selected_both_wrong": 839}, {"budget": 0.72, "threshold": -0.9056642055511475, "operator": "ge", "actual_fraction": 0.8893683120469451, "metric": 0.3384535726613738, "rescued": 232, "harmed": 2989, "missed_rescue": 0, "avoided_harm": 422, "selected_both_correct": 1092, "selected_both_wrong": 840}, {"budget": 0.73, "threshold": -0.9068957567214966, "operator": "ge", "actual_fraction": 0.8933379357956507, "metric": 0.3369002416292717, "rescued": 232, "harmed": 2998, "missed_rescue": 0, "avoided_harm": 413, "selected_both_correct": 1106, "selected_both_wrong": 840}, {"budget": 0.74, "threshold": -0.9093236327171326, "operator": "ge", "actual_fraction": 0.8981705212288574, "metric": 0.3336209872281671, "rescued": 232, "harmed": 3017, "missed_rescue": 0, "avoided_harm": 394, "selected_both_correct": 1115, "selected_both_wrong": 840}, {"budget": 0.75, "threshold": -0.9112368822097778, "operator": "ge", "actual_fraction": 0.9023127373144633, "metric": 0.3308595098377632, "rescued": 232, "harmed": 3033, "missed_rescue": 0, "avoided_harm": 378, "selected_both_correct": 1123, "selected_both_wrong": 840}, {"budget": 0.76, "threshold": -0.9129221439361572, "operator": "ge", "actual_fraction": 0.9081808767690714, "metric": 0.32688988608905767, "rescued": 232, "harmed": 3056, "missed_rescue": 0, "avoided_harm": 355, "selected_both_correct": 1133, "selected_both_wrong": 841}, {"budget": 0.77, "threshold": -0.9144289493560791, "operator": "ge", "actual_fraction": 0.9123230928546773, "metric": 0.32361063168795307, "rescued": 232, "harmed": 3075, "missed_rescue": 0, "avoided_harm": 336, "selected_both_correct": 1138, "selected_both_wrong": 841}, {"budget": 0.78, "threshold": -0.9158422946929932, "operator": "ge", "actual_fraction": 0.9149119779081809, "metric": 0.3217121159820504, "rescued": 232, "harmed": 3086, "missed_rescue": 0, "avoided_harm": 325, "selected_both_correct": 1142, "selected_both_wrong": 841}, {"budget": 0.79, "threshold": -0.9167781472206116, "operator": "ge", "actual_fraction": 0.9168104936140835, "metric": 0.32033137728684846, "rescued": 232, "harmed": 3094, "missed_rescue": 0, "avoided_harm": 317, "selected_both_correct": 1145, "selected_both_wrong": 841}, {"budget": 0.8, "threshold": -0.9187189936637878, "operator": "ge", "actual_fraction": 0.922851225405592, "metric": 0.3161891612012427, "rescued": 232, "harmed": 3118, "missed_rescue": 0, "avoided_harm": 293, "selected_both_correct": 1156, "selected_both_wrong": 841}, {"budget": 0.81, "threshold": -0.9207327365875244, "operator": "ge", "actual_fraction": 0.9280289955125992, "metric": 0.3127373144632378, "rescued": 232, "harmed": 3138, "missed_rescue": 0, "avoided_harm": 273, "selected_both_correct": 1166, "selected_both_wrong": 841}, {"budget": 0.82, "threshold": -0.9226345419883728, "operator": "ge", "actual_fraction": 0.9342423196410079, "metric": 0.3079047290300311, "rescued": 232, "harmed": 3166, "missed_rescue": 0, "avoided_harm": 245, "selected_both_correct": 1174, "selected_both_wrong": 841}, {"budget": 0.83, "threshold": -0.9243993759155273, "operator": "ge", "actual_fraction": 0.9387297204004142, "metric": 0.30479806696582673, "rescued": 232, "harmed": 3184, "missed_rescue": 0, "avoided_harm": 227, "selected_both_correct": 1182, "selected_both_wrong": 841}, {"budget": 0.84, "threshold": -0.9264947175979614, "operator": "ge", "actual_fraction": 0.9440800828443218, "metric": 0.3015188125647221, "rescued": 232, "harmed": 3203, "missed_rescue": 0, "avoided_harm": 208, "selected_both_correct": 1194, "selected_both_wrong": 841}, {"budget": 0.85, "threshold": -0.9274600148200989, "operator": "ge", "actual_fraction": 0.9463237832240249, "metric": 0.2999654815326199, "rescued": 232, "harmed": 3212, "missed_rescue": 0, "avoided_harm": 199, "selected_both_correct": 1198, "selected_both_wrong": 841}, {"budget": 0.86, "threshold": -0.930335283279419, "operator": "ge", "actual_fraction": 0.9534000690369347, "metric": 0.29530548843631343, "rescued": 232, "harmed": 3239, "missed_rescue": 0, "avoided_harm": 172, "selected_both_correct": 1211, "selected_both_wrong": 842}, {"budget": 0.87, "threshold": -0.9319414496421814, "operator": "ge", "actual_fraction": 0.9558163617535381, "metric": 0.2939247497411115, "rescued": 232, "harmed": 3247, "missed_rescue": 0, "avoided_harm": 164, "selected_both_correct": 1217, "selected_both_wrong": 842}, {"budget": 0.88, "threshold": -0.9340114593505859, "operator": "ge", "actual_fraction": 0.958577839143942, "metric": 0.29202623403520883, "rescued": 232, "harmed": 3258, "missed_rescue": 0, "avoided_harm": 153, "selected_both_correct": 1222, "selected_both_wrong": 842}, {"budget": 0.89, "threshold": -0.9363258481025696, "operator": "ge", "actual_fraction": 0.9628926475664481, "metric": 0.28874697963410423, "rescued": 232, "harmed": 3277, "missed_rescue": 0, "avoided_harm": 134, "selected_both_correct": 1228, "selected_both_wrong": 842}, {"budget": 0.9, "threshold": -0.9400071501731873, "operator": "ge", "actual_fraction": 0.9694511563686572, "metric": 0.2839143942008975, "rescued": 232, "harmed": 3305, "missed_rescue": 0, "avoided_harm": 106, "selected_both_correct": 1238, "selected_both_wrong": 842}, {"budget": 0.91, "threshold": -0.9423198103904724, "operator": "ge", "actual_fraction": 0.9734207801173628, "metric": 0.2823610631687953, "rescued": 232, "harmed": 3314, "missed_rescue": 0, "avoided_harm": 97, "selected_both_correct": 1251, "selected_both_wrong": 843}, {"budget": 0.92, "threshold": -0.9443356394767761, "operator": "ge", "actual_fraction": 0.9761822575077667, "metric": 0.2806351397997929, "rescued": 232, "harmed": 3324, "missed_rescue": 0, "avoided_harm": 87, "selected_both_correct": 1256, "selected_both_wrong": 844}, {"budget": 0.93, "threshold": -0.9468213319778442, "operator": "ge", "actual_fraction": 0.9803244735933725, "metric": 0.27839143942008976, "rescued": 232, "harmed": 3337, "missed_rescue": 0, "avoided_harm": 74, "selected_both_correct": 1267, "selected_both_wrong": 844}, {"budget": 0.94, "threshold": -0.9496435523033142, "operator": "ge", "actual_fraction": 0.9837763203313773, "metric": 0.27632033137728684, "rescued": 232, "harmed": 3349, "missed_rescue": 0, "avoided_harm": 62, "selected_both_correct": 1275, "selected_both_wrong": 844}, {"budget": 0.95, "threshold": -0.9529175758361816, "operator": "ge", "actual_fraction": 0.9860200207110804, "metric": 0.2752847773558854, "rescued": 232, "harmed": 3355, "missed_rescue": 0, "avoided_harm": 56, "selected_both_correct": 1282, "selected_both_wrong": 844}, {"budget": 0.96, "threshold": -0.9560379981994629, "operator": "ge", "actual_fraction": 0.988608905764584, "metric": 0.27338626164998275, "rescued": 232, "harmed": 3366, "missed_rescue": 0, "avoided_harm": 45, "selected_both_correct": 1286, "selected_both_wrong": 844}, {"budget": 0.97, "threshold": -0.9591385126113892, "operator": "ge", "actual_fraction": 0.9920607525025888, "metric": 0.2711425612702796, "rescued": 232, "harmed": 3379, "missed_rescue": 0, "avoided_harm": 32, "selected_both_correct": 1293, "selected_both_wrong": 844}, {"budget": 0.98, "threshold": -0.9627243280410767, "operator": "ge", "actual_fraction": 0.9946496375560925, "metric": 0.2697618225750777, "rescued": 232, "harmed": 3387, "missed_rescue": 0, "avoided_harm": 24, "selected_both_correct": 1300, "selected_both_wrong": 844}, {"budget": 0.99, "threshold": -0.9702208638191223, "operator": "ge", "actual_fraction": 0.9972385226095961, "metric": 0.2676907145322748, "rescued": 232, "harmed": 3399, "missed_rescue": 0, "avoided_harm": 12, "selected_both_correct": 1303, "selected_both_wrong": 844}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.26561960648947186, "rescued": 232, "harmed": 3411, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 1307, "selected_both_wrong": 844}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.9861666666666666, "actual_fraction": 0.004, "metric_std": 0.00045643546458764894, "actual_fraction_std": 0.005477225575051661, "per_seed": [{"seed": 2026, "budget": 0.01, "threshold": -0.004982936196029186, "operator": "ge", "alpha": 0.0, "epoch": 3, "metric": 0.9866666666666667, "actual_fraction": 0.01, "rescue": 1, "harm": 0, "missed_rescue": 5}, {"seed": 2027, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.9858333333333333, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 6}, {"seed": 2028, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.25, "epoch": 3, "metric": 0.9858333333333333, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 6}, {"seed": 2029, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 4, "metric": 0.9858333333333333, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 6}, {"seed": 2030, "budget": 0.01, "threshold": -0.004842729773372412, "operator": "ge", "alpha": 0.0, "epoch": 3, "metric": 0.9866666666666667, "actual_fraction": 0.01, "rescue": 1, "harm": 0, "missed_rescue": 5}], "rescued": 0.4, "harmed": 0.0, "missed_rescue": 5.6, "avoided_harm": 856.0}, "validation_curve": [{"budget": 0.0, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.9861666666666666, "metric_std": 0.00045643546458764894, "actual_fraction": 0.01, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.9861666666666666, "metric_std": 0.00045643546458764894, "actual_fraction": 0.02, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.9861666666666666, "metric_std": 0.00045643546458764894, "actual_fraction": 0.03, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.9861666666666666, "metric_std": 0.00045643546458764894, "actual_fraction": 0.04, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.9861666666666666, "metric_std": 0.00045643546458764894, "actual_fraction": 0.05, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.9861666666666666, "metric_std": 0.00045643546458764894, "actual_fraction": 0.06, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.9858333333333335, "metric_std": 0.0005892556509888032, "actual_fraction": 0.07, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.9855, "metric_std": 0.0009501461875826759, "actual_fraction": 0.08, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.9848333333333332, "metric_std": 0.0016029486718059536, "actual_fraction": 0.09, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.9835, "metric_std": 0.0018066236157232162, "actual_fraction": 0.1, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.9819999999999999, "metric_std": 0.0016244657241347908, "actual_fraction": 0.11000000000000001, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.12, "metric": 0.9800000000000001, "metric_std": 0.0013176156917368376, "actual_fraction": 0.12, "actual_fraction_std": 0.0}, {"budget": 0.13, "metric": 0.9775, "metric_std": 0.0014433756729740495, "actual_fraction": 0.13, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.9758333333333333, "metric_std": 0.0019543398999263793, "actual_fraction": 0.14, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.9726666666666667, "metric_std": 0.0021570298509241506, "actual_fraction": 0.15, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.9711666666666666, "metric_std": 0.0022515426810177227, "actual_fraction": 0.16, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.9693333333333334, "metric_std": 0.0017078251276599129, "actual_fraction": 0.17, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.9668333333333333, "metric_std": 0.0021570298509241506, "actual_fraction": 0.18, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.9646666666666667, "metric_std": 0.0017280367794439738, "actual_fraction": 0.19, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.9631666666666666, "metric_std": 0.0017078251276599129, "actual_fraction": 0.2, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.9603333333333334, "metric_std": 0.002327373340628129, "actual_fraction": 0.21000000000000002, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.9565000000000001, "metric_std": 0.0031950482521134575, "actual_fraction": 0.22000000000000003, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.23, "metric": 0.9531666666666666, "metric_std": 0.0027888667551136097, "actual_fraction": 0.23000000000000004, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.24, "metric": 0.9491666666666667, "metric_std": 0.0015590239111557857, "actual_fraction": 0.24, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.9435, "metric_std": 0.001086533734200424, "actual_fraction": 0.25, "actual_fraction_std": 0.0}, {"budget": 0.26, "metric": 0.9369999999999999, "metric_std": 0.0030390970881350923, "actual_fraction": 0.26, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.9328333333333333, "metric_std": 0.0018257418583505502, "actual_fraction": 0.27, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.9271666666666667, "metric_std": 0.0013944333775568248, "actual_fraction": 0.28, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.9211666666666666, "metric_std": 0.001118033988749838, "actual_fraction": 0.29, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.9151666666666667, "metric_std": 0.0016029486718059072, "actual_fraction": 0.3, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.9076666666666666, "metric_std": 0.002595401917066253, "actual_fraction": 0.31, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.9001666666666667, "metric_std": 0.002595401917066166, "actual_fraction": 0.32, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.8934999999999998, "metric_std": 0.003791437722025819, "actual_fraction": 0.33, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.8851666666666667, "metric_std": 0.0027888667551135603, "actual_fraction": 0.34, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.8779999999999999, "metric_std": 0.0030390970881350784, "actual_fraction": 0.35, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.8709999999999999, "metric_std": 0.0029697549468675473, "actual_fraction": 0.36, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.8628333333333333, "metric_std": 0.0028625940062196095, "actual_fraction": 0.37, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.8543333333333333, "metric_std": 0.0029697549468675876, "actual_fraction": 0.38, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.8460000000000001, "metric_std": 0.0025276251480171688, "actual_fraction": 0.39, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.8376666666666666, "metric_std": 0.0017078251276599237, "actual_fraction": 0.4, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.8301666666666666, "metric_std": 0.002312406154251912, "actual_fraction": 0.4091666666666668, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.42, "metric": 0.8206666666666667, "metric_std": 0.0023124061542519263, "actual_fraction": 0.42000000000000004, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.8121666666666666, "metric_std": 0.0022515426810177062, "actual_fraction": 0.43, "actual_fraction_std": 0.0}, {"budget": 0.44, "metric": 0.8035, "metric_std": 0.002236067977499802, "actual_fraction": 0.44000000000000006, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.45, "metric": 0.7951666666666666, "metric_std": 0.002074983266331376, "actual_fraction": 0.45, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.7873333333333333, "metric_std": 0.002527625148017198, "actual_fraction": 0.4600000000000001, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.47, "metric": 0.7788333333333333, "metric_std": 0.001825741858350596, "actual_fraction": 0.4699999999999999, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.48, "metric": 0.7699999999999999, "metric_std": 0.0013176156917368727, "actual_fraction": 0.48, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.7611666666666667, "metric_std": 0.001394433377556775, "actual_fraction": 0.49000000000000005, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.5, "metric": 0.7531666666666668, "metric_std": 0.0016029486718059998, "actual_fraction": 0.5, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.7436666666666667, "metric_std": 0.0012638125740085477, "actual_fraction": 0.51, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.7348333333333332, "metric_std": 0.0016029486718059536, "actual_fraction": 0.52, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.7258333333333333, "metric_std": 0.0016666666666666496, "actual_fraction": 0.53, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.7170000000000001, "metric_std": 0.0026087459737497314, "actual_fraction": 0.54, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.7073333333333334, "metric_std": 0.002788866755113615, "actual_fraction": 0.55, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.6990000000000001, "metric_std": 0.002236067977499829, "actual_fraction": 0.56, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.6908333333333335, "metric_std": 0.0015590239111558152, "actual_fraction": 0.5691666666666667, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.6813333333333333, "metric_std": 0.0017280367794439229, "actual_fraction": 0.58, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.6716666666666666, "metric_std": 0.0016666666666666496, "actual_fraction": 0.59, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.6621666666666667, "metric_std": 0.0017280367794439658, "actual_fraction": 0.6, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.6530000000000001, "metric_std": 0.001118033988749871, "actual_fraction": 0.61, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.6438333333333333, "metric_std": 0.0009501461875826369, "actual_fraction": 0.62, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.6346666666666667, "metric_std": 0.0007453559924999471, "actual_fraction": 0.63, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.6251666666666666, "metric_std": 0.0010865337342004667, "actual_fraction": 0.64, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.6156666666666667, "metric_std": 0.0006972166887784124, "actual_fraction": 0.65, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.6058333333333333, "metric_std": 0.0010206207261596358, "actual_fraction": 0.66, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.5970000000000001, "metric_std": 0.0007453559924999471, "actual_fraction": 0.67, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.5875, "metric_std": 0.0008333333333333526, "actual_fraction": 0.68, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.5793333333333333, "metric_std": 0.0014907119849998415, "actual_fraction": 0.6891666666666667, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.5686666666666668, "metric_std": 0.0013944333775568248, "actual_fraction": 0.7, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.5589999999999999, "metric_std": 0.0018066236157232188, "actual_fraction": 0.71, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.5493333333333333, "metric_std": 0.001989556064385579, "actual_fraction": 0.72, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.54, "metric_std": 0.0013176156917368376, "actual_fraction": 0.73, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.53, "metric_std": 0.0013176156917368376, "actual_fraction": 0.74, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.5205, "metric_std": 0.0009501461875826174, "actual_fraction": 0.75, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.5108333333333333, "metric_std": 0.00131761569173682, "actual_fraction": 0.76, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.5011666666666666, "metric_std": 0.0011180339887498876, "actual_fraction": 0.77, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.4915, "metric_std": 0.0012360330811826185, "actual_fraction": 0.78, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.48166666666666663, "metric_std": 0.0013176156917368814, "actual_fraction": 0.79, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.4718333333333334, "metric_std": 0.0012360330811826278, "actual_fraction": 0.8, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.462, "metric_std": 0.000950146187582654, "actual_fraction": 0.8100000000000002, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.82, "metric": 0.4531666666666666, "metric_std": 0.0006972166887784124, "actual_fraction": 0.8191666666666666, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.83, "metric": 0.4423333333333333, "metric_std": 0.0006972166887784124, "actual_fraction": 0.8299999999999998, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.84, "metric": 0.4325, "metric_std": 0.0005892556509888032, "actual_fraction": 0.8400000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.42283333333333334, "metric_std": 0.000456435464587649, "actual_fraction": 0.85, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.4133333333333334, "metric_std": 0.0005892556509888229, "actual_fraction": 0.86, "actual_fraction_std": 0.0}, {"budget": 0.87, "metric": 0.4036666666666666, "metric_std": 0.0007453559924999037, "actual_fraction": 0.8699999999999999, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.39399999999999996, "metric_std": 0.0009128709291753209, "actual_fraction": 0.8800000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.89, "metric": 0.3843333333333333, "metric_std": 0.0006972166887784124, "actual_fraction": 0.89, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.3745, "metric_std": 0.000456435464587649, "actual_fraction": 0.9, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.36466666666666664, "metric_std": 0.0007453559924999471, "actual_fraction": 0.9099999999999999, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.3551666666666667, "metric_std": 0.0006972166887784124, "actual_fraction": 0.9200000000000002, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.93, "metric": 0.3453333333333334, "metric_std": 0.0009501461875826637, "actual_fraction": 0.9298333333333334, "actual_fraction_std": 0.00037267799624997355}, {"budget": 0.94, "metric": 0.33533333333333326, "metric_std": 0.0007453559924998696, "actual_fraction": 0.9399999999999998, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.95, "metric": 0.3255, "metric_std": 0.000950146187582598, "actual_fraction": 0.95, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.31566666666666665, "metric_std": 0.0009128709291752523, "actual_fraction": 0.96, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.30633333333333324, "metric_std": 0.0007453559924999471, "actual_fraction": 0.97, "actual_fraction_std": 0.0}, {"budget": 0.98, "metric": 0.2966666666666667, "metric_std": 0.000589255650988764, "actual_fraction": 0.9800000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.99, "metric": 0.28733333333333333, "metric_std": 0.00037267799624997355, "actual_fraction": 0.99, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.27749999999999997, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.8142906454953401, "actual_fraction": 0.0059371763893683125, "metric_std": 0.00024408242360600966, "actual_fraction_std": 0.008174575085817906, "per_seed": [{"seed": 2026, "budget": 0.01, "threshold": -0.004982936196029186, "operator": "ge", "alpha": 0.0, "epoch": 3, "metric": 0.8139454608215395, "actual_fraction": 0.013634794615119088, "rescue": 6, "harm": 8, "missed_rescue": 226}, {"seed": 2027, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.81429064549534, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 232}, {"seed": 2028, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.25, "epoch": 3, "metric": 0.81429064549534, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 232}, {"seed": 2029, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 4, "metric": 0.81429064549534, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 232}, {"seed": 2030, "budget": 0.01, "threshold": -0.004842729773372412, "operator": "ge", "alpha": 0.0, "epoch": 3, "metric": 0.8146358301691405, "actual_fraction": 0.01605108733172247, "rescue": 17, "harm": 15, "missed_rescue": 215}], "rescued": 4.6, "harmed": 4.6, "missed_rescue": 227.4, "avoided_harm": 3406.4}, "test_curve": [{"budget": 0.0, "metric": 0.8142906454953401, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.81432516396272, "metric_std": 0.000255995805560798, "actual_fraction": 0.010390058681394546, "actual_fraction_std": 0.004511234864398906}, {"budget": 0.02, "metric": 0.8142216085605799, "metric_std": 0.00026174923866248607, "actual_fraction": 0.01636175353814291, "actual_fraction_std": 0.0051581725543715954}, {"budget": 0.03, "metric": 0.8141525716258198, "metric_std": 0.00022503287556794013, "actual_fraction": 0.02347255781843286, "actual_fraction_std": 0.004350901852245609}, {"budget": 0.04, "metric": 0.8141870900931998, "metric_std": 0.00019678554109410403, "actual_fraction": 0.0299275112185019, "actual_fraction_std": 0.003906462746507447}, {"budget": 0.05, "metric": 0.8139454608215395, "metric_std": 0.0005319653091964963, "actual_fraction": 0.037245426303072135, "actual_fraction_std": 0.009005781479029477}, {"budget": 0.06, "metric": 0.8140490162236796, "metric_std": 0.00035789508721656085, "actual_fraction": 0.04794615119088712, "actual_fraction_std": 0.008835480311949565}, {"budget": 0.07, "metric": 0.8140835346910598, "metric_std": 0.00047892429842981475, "actual_fraction": 0.06358301691404902, "actual_fraction_std": 0.008376153454087607}, {"budget": 0.08, "metric": 0.8139454608215395, "metric_std": 0.0005031888069421181, "actual_fraction": 0.07449085260614428, "actual_fraction_std": 0.00583326353652593}, {"budget": 0.09, "metric": 0.8138764238867795, "metric_std": 0.0006411520752850164, "actual_fraction": 0.08539868829823957, "actual_fraction_std": 0.006321923249221419}, {"budget": 0.1, "metric": 0.8135312392129789, "metric_std": 0.0011357047895314356, "actual_fraction": 0.09534000690369349, "actual_fraction_std": 0.008733069919080258}, {"budget": 0.11, "metric": 0.8129789437348982, "metric_std": 0.0011091659656928088, "actual_fraction": 0.10824991370383155, "actual_fraction_std": 0.009321104779521973}, {"budget": 0.12, "metric": 0.8123576113220572, "metric_std": 0.0020068266280633368, "actual_fraction": 0.12402485329651365, "actual_fraction_std": 0.012648557496221135}, {"budget": 0.13, "metric": 0.8110113910942355, "metric_std": 0.0029593421124408945, "actual_fraction": 0.13921297894373488, "actual_fraction_std": 0.016030009804289102}, {"budget": 0.14, "metric": 0.8103900586813946, "metric_std": 0.0031650818303281185, "actual_fraction": 0.1495685191577494, "actual_fraction_std": 0.015650137889613103}, {"budget": 0.15, "metric": 0.8092164307904728, "metric_std": 0.003348024429860245, "actual_fraction": 0.16189161201242666, "actual_fraction_std": 0.01618001379910227}, {"budget": 0.16, "metric": 0.8079392474974112, "metric_std": 0.003474211291704854, "actual_fraction": 0.17276492923714187, "actual_fraction_std": 0.014557219802526444}, {"budget": 0.17, "metric": 0.8066275457369694, "metric_std": 0.0033122443546464903, "actual_fraction": 0.18505350362443906, "actual_fraction_std": 0.012370428574954467}, {"budget": 0.18, "metric": 0.8046599930963065, "metric_std": 0.002990383261382592, "actual_fraction": 0.1961684501208146, "actual_fraction_std": 0.011492368911943125}, {"budget": 0.19, "metric": 0.8021746634449431, "metric_std": 0.0027053539132095117, "actual_fraction": 0.20856057991025198, "actual_fraction_std": 0.00908416282936675}, {"budget": 0.2, "metric": 0.799275112185019, "metric_std": 0.0021933491589815187, "actual_fraction": 0.22005522954780807, "actual_fraction_std": 0.007062591462067699}, {"budget": 0.21, "metric": 0.7962374870555747, "metric_std": 0.002115928554542914, "actual_fraction": 0.23579565067311012, "actual_fraction_std": 0.0065669059482571426}, {"budget": 0.22, "metric": 0.7917846047635485, "metric_std": 0.0026837969647808256, "actual_fraction": 0.24963755609250954, "actual_fraction_std": 0.006357398375268952}, {"budget": 0.23, "metric": 0.7875388332758027, "metric_std": 0.0029111370026636526, "actual_fraction": 0.26178805661028653, "actual_fraction_std": 0.005897760727715467}, {"budget": 0.24, "metric": 0.7826026924404557, "metric_std": 0.004433305551894902, "actual_fraction": 0.27721781152916813, "actual_fraction_std": 0.00774880009517945}, {"budget": 0.25, "metric": 0.7772178115291681, "metric_std": 0.0037464863103113875, "actual_fraction": 0.29195719710044876, "actual_fraction_std": 0.005351197502952279}, {"budget": 0.26, "metric": 0.770693821194339, "metric_std": 0.004911457556567518, "actual_fraction": 0.30724887814981017, "actual_fraction_std": 0.0074158490723446446}, {"budget": 0.27, "metric": 0.7634104245771489, "metric_std": 0.004684538839763328, "actual_fraction": 0.3265792198826372, "actual_fraction_std": 0.007374561757235656}, {"budget": 0.28, "metric": 0.7558508802209183, "metric_std": 0.0040799017789691985, "actual_fraction": 0.3438384535726614, "actual_fraction_std": 0.010251752361647295}, {"budget": 0.29, "metric": 0.74984466689679, "metric_std": 0.004628886577189381, "actual_fraction": 0.35937176389368314, "actual_fraction_std": 0.011008360843157}, {"budget": 0.3, "metric": 0.7425267518122196, "metric_std": 0.0044043183728647625, "actual_fraction": 0.37493959268208493, "actual_fraction_std": 0.01076220825892271}, {"budget": 0.31, "metric": 0.73576113220573, "metric_std": 0.0028854424399897645, "actual_fraction": 0.3901277183293062, "actual_fraction_std": 0.0049416896379205705}, {"budget": 0.32, "metric": 0.7287193648602003, "metric_std": 0.0017533206902589091, "actual_fraction": 0.40524680704176735, "actual_fraction_std": 0.003995801485257226}, {"budget": 0.33, "metric": 0.7205039696237489, "metric_std": 0.002619198890589451, "actual_fraction": 0.42029685881946843, "actual_fraction_std": 0.0038056512444916693}, {"budget": 0.34, "metric": 0.7117707973765965, "metric_std": 0.0028438483005322674, "actual_fraction": 0.43593372454263035, "actual_fraction_std": 0.005279463891138156}, {"budget": 0.35, "metric": 0.7017259233690025, "metric_std": 0.0025582120342231497, "actual_fraction": 0.4545046599930963, "actual_fraction_std": 0.007565177121655602}, {"budget": 0.36, "metric": 0.6946151190887125, "metric_std": 0.0028032311129861888, "actual_fraction": 0.467138419054194, "actual_fraction_std": 0.006070982983706016}, {"budget": 0.37, "metric": 0.68567483603728, "metric_std": 0.00447976036318683, "actual_fraction": 0.48267172937521574, "actual_fraction_std": 0.0036928307483028327}, {"budget": 0.38, "metric": 0.6764238867794271, "metric_std": 0.0028238764704506133, "actual_fraction": 0.4972730410769762, "actual_fraction_std": 0.00269432061310538}, {"budget": 0.39, "metric": 0.6677942699344148, "metric_std": 0.004191187334398027, "actual_fraction": 0.5117707973765965, "actual_fraction_std": 0.006145111315269389}, {"budget": 0.4, "metric": 0.6613047980669658, "metric_std": 0.003720956293603548, "actual_fraction": 0.522057300655851, "actual_fraction_std": 0.006497362251958563}, {"budget": 0.41, "metric": 0.6540214014497756, "metric_std": 0.0027684800752208892, "actual_fraction": 0.5343803935105281, "actual_fraction_std": 0.0067668522965983234}, {"budget": 0.42, "metric": 0.6457714877459442, "metric_std": 0.004002505229996091, "actual_fraction": 0.5468070417673455, "actual_fraction_std": 0.007118675496472304}, {"budget": 0.43, "metric": 0.6374180186399724, "metric_std": 0.00437547955704542, "actual_fraction": 0.5583707283396617, "actual_fraction_std": 0.006111084966718894}, {"budget": 0.44, "metric": 0.6297549188816017, "metric_std": 0.004800735817273054, "actual_fraction": 0.5709699689333794, "actual_fraction_std": 0.006959987946636393}, {"budget": 0.45, "metric": 0.6226441146013117, "metric_std": 0.007497940315866409, "actual_fraction": 0.5816016568864343, "actual_fraction_std": 0.010531951524740003}, {"budget": 0.46, "metric": 0.613600276147739, "metric_std": 0.006344265178823681, "actual_fraction": 0.5955816361753538, "actual_fraction_std": 0.010596804055859553}, {"budget": 0.47, "metric": 0.6065239903348292, "metric_std": 0.007317792966331814, "actual_fraction": 0.606454953400069, "actual_fraction_std": 0.012675494253209213}, {"budget": 0.48, "metric": 0.5993786675871592, "metric_std": 0.007436907466732374, "actual_fraction": 0.6163962720055229, "actual_fraction_std": 0.011679414107963403}, {"budget": 0.49, "metric": 0.5911632723507076, "metric_std": 0.006999465146732437, "actual_fraction": 0.6281325509147393, "actual_fraction_std": 0.012225216078610797}, {"budget": 0.5, "metric": 0.5833275802554366, "metric_std": 0.005638251153357398, "actual_fraction": 0.6394200897480152, "actual_fraction_std": 0.012789546346603401}, {"budget": 0.51, "metric": 0.5774249223334483, "metric_std": 0.006806574386655489, "actual_fraction": 0.6472903003106663, "actual_fraction_std": 0.013929067730638916}, {"budget": 0.52, "metric": 0.5681739730755954, "metric_std": 0.0066026438603343385, "actual_fraction": 0.6591991715567829, "actual_fraction_std": 0.013678745747369353}, {"budget": 0.53, "metric": 0.561511908871246, "metric_std": 0.005807674434659391, "actual_fraction": 0.6676561960648948, "actual_fraction_std": 0.012888268476320299}, {"budget": 0.54, "metric": 0.5550914739385571, "metric_std": 0.004524421781929145, "actual_fraction": 0.6763548498446669, "actual_fraction_std": 0.011492887298416233}, {"budget": 0.55, "metric": 0.5468415602347255, "metric_std": 0.007856838436055637, "actual_fraction": 0.6862616499827408, "actual_fraction_std": 0.013869488406724105}, {"budget": 0.56, "metric": 0.5384535726613737, "metric_std": 0.006706050763912324, "actual_fraction": 0.6968243010010355, "actual_fraction_std": 0.011165670454171597}, {"budget": 0.57, "metric": 0.5320676561960649, "metric_std": 0.006163262442710574, "actual_fraction": 0.7051777701070072, "actual_fraction_std": 0.01146888740598236}, {"budget": 0.58, "metric": 0.5274767000345184, "metric_std": 0.00561442633182816, "actual_fraction": 0.7110804280289955, "actual_fraction_std": 0.009378928663578581}, {"budget": 0.59, "metric": 0.5213324128408698, "metric_std": 0.00601181500333199, "actual_fraction": 0.7183983431135658, "actual_fraction_std": 0.009901596946891071}, {"budget": 0.6, "metric": 0.5145667932343804, "metric_std": 0.007756676748811692, "actual_fraction": 0.7266482568173972, "actual_fraction_std": 0.010817974230168729}, {"budget": 0.61, "metric": 0.5054193993786676, "metric_std": 0.006685140878804049, "actual_fraction": 0.7375215740421127, "actual_fraction_std": 0.009518745119784046}, {"budget": 0.62, "metric": 0.49972385226095967, "metric_std": 0.006780265297464003, "actual_fraction": 0.7446668967897825, "actual_fraction_std": 0.00909645124051809}, {"budget": 0.63, "metric": 0.4900586813945461, "metric_std": 0.0068414957953053844, "actual_fraction": 0.7559199171556783, "actual_fraction_std": 0.009027255703693539}, {"budget": 0.64, "metric": 0.4826717293752158, "metric_std": 0.006559190052367931, "actual_fraction": 0.7648256817397308, "actual_fraction_std": 0.007675606384239241}, {"budget": 0.65, "metric": 0.4743872972040041, "metric_std": 0.0077959406448527835, "actual_fraction": 0.7740766309975837, "actual_fraction_std": 0.007742646895463583}, {"budget": 0.66, "metric": 0.46758715913013466, "metric_std": 0.006738178130039443, "actual_fraction": 0.7819123230928546, "actual_fraction_std": 0.00709268413233613}, {"budget": 0.67, "metric": 0.4625129444252676, "metric_std": 0.006659246686996921, "actual_fraction": 0.7877459440800829, "actual_fraction_std": 0.006958917887139144}, {"budget": 0.68, "metric": 0.45540214014497754, "metric_std": 0.00489170651041599, "actual_fraction": 0.7957197100448741, "actual_fraction_std": 0.005721117067028697}, {"budget": 0.69, "metric": 0.45029340697273046, "metric_std": 0.006564183726522827, "actual_fraction": 0.8015533310321021, "actual_fraction_std": 0.007540329875813645}, {"budget": 0.7, "metric": 0.4453917846047636, "metric_std": 0.007956675203399077, "actual_fraction": 0.8072488781498102, "actual_fraction_std": 0.009287488568011173}, {"budget": 0.71, "metric": 0.44000690369347606, "metric_std": 0.007941685927706551, "actual_fraction": 0.8132550914739387, "actual_fraction_std": 0.008898972023815519}, {"budget": 0.72, "metric": 0.43369002416292723, "metric_std": 0.006484742215540577, "actual_fraction": 0.820607525025889, "actual_fraction_std": 0.00830382610841396}, {"budget": 0.73, "metric": 0.42816706938211946, "metric_std": 0.007534401787256738, "actual_fraction": 0.8266137383500173, "actual_fraction_std": 0.0091949772220438}, {"budget": 0.74, "metric": 0.4235761132205731, "metric_std": 0.006776749701765843, "actual_fraction": 0.8317915084570245, "actual_fraction_std": 0.00804232140437032}, {"budget": 0.75, "metric": 0.4171556782878841, "metric_std": 0.0070583724703757265, "actual_fraction": 0.8388677942699344, "actual_fraction_std": 0.00843021258252143}, {"budget": 0.76, "metric": 0.4117707973765965, "metric_std": 0.008469695269432836, "actual_fraction": 0.8448049706593027, "actual_fraction_std": 0.009813054497654695}, {"budget": 0.77, "metric": 0.4057991025198482, "metric_std": 0.0076345533035279195, "actual_fraction": 0.8510873317224716, "actual_fraction_std": 0.008660111807760334}, {"budget": 0.78, "metric": 0.3993441491197791, "metric_std": 0.007749761091313179, "actual_fraction": 0.8580945806006213, "actual_fraction_std": 0.0087516398841813}, {"budget": 0.79, "metric": 0.3915774939592683, "metric_std": 0.00683147417394673, "actual_fraction": 0.8662409389023127, "actual_fraction_std": 0.007922345472922691}, {"budget": 0.8, "metric": 0.38173973075595447, "metric_std": 0.00755907148572337, "actual_fraction": 0.8768381083879875, "actual_fraction_std": 0.007821689720724204}, {"budget": 0.81, "metric": 0.3760441836382465, "metric_std": 0.006501945282966423, "actual_fraction": 0.8830859509837763, "actual_fraction_std": 0.006468874987986184}, {"budget": 0.82, "metric": 0.3711770797376596, "metric_std": 0.006196040856742114, "actual_fraction": 0.8883672764929237, "actual_fraction_std": 0.0062030079663992245}, {"budget": 0.83, "metric": 0.36572316189161197, "metric_std": 0.005319653091964946, "actual_fraction": 0.8944080082844321, "actual_fraction_std": 0.005248339710375846}, {"budget": 0.84, "metric": 0.35868139454608217, "metric_std": 0.004570276366631299, "actual_fraction": 0.9018639972385227, "actual_fraction_std": 0.004434985025210346}, {"budget": 0.85, "metric": 0.3523299965481533, "metric_std": 0.0039064627465074196, "actual_fraction": 0.9087331722471523, "actual_fraction_std": 0.0038271158887421344}, {"budget": 0.86, "metric": 0.34597859855022445, "metric_std": 0.0024396035186861884, "actual_fraction": 0.9155678287884019, "actual_fraction_std": 0.002284486311049287}, {"budget": 0.87, "metric": 0.3396962374870556, "metric_std": 0.0037484735200783957, "actual_fraction": 0.9224370037970313, "actual_fraction_std": 0.0034249904512542646}, {"budget": 0.88, "metric": 0.33451846738004826, "metric_std": 0.001880385391779754, "actual_fraction": 0.9280635139799793, "actual_fraction_std": 0.0017362480276091343}, {"budget": 0.89, "metric": 0.3278909216430791, "metric_std": 0.0014253243232352556, "actual_fraction": 0.9351052813255091, "actual_fraction_std": 0.0012019665410065872}, {"budget": 0.9, "metric": 0.3218501898515706, "metric_std": 0.003369752119258756, "actual_fraction": 0.9415257162581983, "actual_fraction_std": 0.0031338700412237067}, {"budget": 0.91, "metric": 0.31636175353814294, "metric_std": 0.0010355540214014418, "actual_fraction": 0.9472557818432861, "actual_fraction_std": 0.0008419959211356756}, {"budget": 0.92, "metric": 0.3101484294097342, "metric_std": 0.0024921520673444044, "actual_fraction": 0.9539523645150154, "actual_fraction_std": 0.0028735457905788636}, {"budget": 0.93, "metric": 0.30503969623748706, "metric_std": 0.0021715106644876305, "actual_fraction": 0.9594062823610632, "actual_fraction_std": 0.0023360613483864932}, {"budget": 0.94, "metric": 0.2994822229892993, "metric_std": 0.002307192256189294, "actual_fraction": 0.9652744218156715, "actual_fraction_std": 0.0025986468816176357}, {"budget": 0.95, "metric": 0.2930272695892303, "metric_std": 0.0024723513399337355, "actual_fraction": 0.9719364860200207, "actual_fraction_std": 0.002771168703683248}, {"budget": 0.96, "metric": 0.28802209181912325, "metric_std": 0.002738187079850087, "actual_fraction": 0.9770452191922679, "actual_fraction_std": 0.0029794056785695452}, {"budget": 0.97, "metric": 0.28194684156023475, "metric_std": 0.0035509588812749223, "actual_fraction": 0.9832585433206766, "actual_fraction_std": 0.0037177527146251546}, {"budget": 0.98, "metric": 0.2773904038660684, "metric_std": 0.0021605086397357147, "actual_fraction": 0.987849499482223, "actual_fraction_std": 0.002272065203143106}, {"budget": 0.99, "metric": 0.27197100448740075, "metric_std": 0.001466527151277056, "actual_fraction": 0.9936140835346909, "actual_fraction_std": 0.0014284557608945981}, {"budget": 1.0, "metric": 0.2656196064894719, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 327, "rescuable": 6, "harmful": 856, "both_wrong": 11, "unit": "sample", "total": 1200}, "test": {"both_correct": 1307, "rescuable": 232, "harmful": 3411, "both_wrong": 844, "unit": "sample", "total": 5794}}, "small": {"validation": 0.9858333333333333, "test": 0.81429064549534}, "large": {"validation": 0.2775, "test": 0.26561960648947186}, "learned_minus_confidence_test": 1.1102230246251565e-16, "learned_minus_small_test": 1.1102230246251565e-16, "learned_minus_large_test": 0.5486710390058682, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_cub_MiniCPM"}, {"key": ["cub", "cub", "Qwen3.8"], "expert_display": "ResNet", "run": "outputs/full_budget_20260913/runs/full_cub_Qwen3.8_s{2026..2030}", "validation_rows": 1200, "test_rows": 5794, "confidence_rule": {"column": "max_probability", "columns": ["entropy", "max_probability", "top_two_margin", "entropy_valid", "max_probability_valid", "top_two_margin_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/cub/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 0, "validation_area_0_25": 0.9171666666666667, "validation_point": {"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9858333333333333, "rescued": 0, "harmed": 0, "missed_rescue": 2, "avoided_harm": 524, "selected_both_correct": 0, "selected_both_wrong": 0}, "test_point": {"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.81429064549534, "rescued": 0, "harmed": 0, "missed_rescue": 365, "avoided_harm": 1907, "selected_both_correct": 0, "selected_both_wrong": 0}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9858333333333333, "rescued": 0, "harmed": 0, "missed_rescue": 2, "avoided_harm": 524, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.305810809135437, "operator": "ge", "actual_fraction": 0.01, "metric": 0.9825, "rescued": 1, "harmed": 5, "missed_rescue": 1, "avoided_harm": 519, "selected_both_correct": 1, "selected_both_wrong": 5}, {"budget": 0.02, "threshold": -0.3627302646636963, "operator": "ge", "actual_fraction": 0.02, "metric": 0.9775, "rescued": 1, "harmed": 11, "missed_rescue": 1, "avoided_harm": 513, "selected_both_correct": 3, "selected_both_wrong": 9}, {"budget": 0.03, "threshold": -0.4222676455974579, "operator": "ge", "actual_fraction": 0.03, "metric": 0.9725, "rescued": 2, "harmed": 18, "missed_rescue": 0, "avoided_harm": 506, "selected_both_correct": 6, "selected_both_wrong": 10}, {"budget": 0.04, "threshold": -0.46618229150772095, "operator": "ge", "actual_fraction": 0.04, "metric": 0.9675, "rescued": 2, "harmed": 24, "missed_rescue": 0, "avoided_harm": 500, "selected_both_correct": 10, "selected_both_wrong": 12}, {"budget": 0.05, "threshold": -0.5066347122192383, "operator": "ge", "actual_fraction": 0.05, "metric": 0.9608333333333333, "rescued": 2, "harmed": 32, "missed_rescue": 0, "avoided_harm": 492, "selected_both_correct": 14, "selected_both_wrong": 12}, {"budget": 0.06, "threshold": -0.5393391251564026, "operator": "ge", "actual_fraction": 0.06, "metric": 0.955, "rescued": 2, "harmed": 39, "missed_rescue": 0, "avoided_harm": 485, "selected_both_correct": 17, "selected_both_wrong": 14}, {"budget": 0.07, "threshold": -0.5679479837417603, "operator": "ge", "actual_fraction": 0.07, "metric": 0.9508333333333333, "rescued": 2, "harmed": 44, "missed_rescue": 0, "avoided_harm": 480, "selected_both_correct": 23, "selected_both_wrong": 15}, {"budget": 0.08, "threshold": -0.5907968282699585, "operator": "ge", "actual_fraction": 0.08, "metric": 0.945, "rescued": 2, "harmed": 51, "missed_rescue": 0, "avoided_harm": 473, "selected_both_correct": 28, "selected_both_wrong": 15}, {"budget": 0.09, "threshold": -0.6168767809867859, "operator": "ge", "actual_fraction": 0.09, "metric": 0.9391666666666667, "rescued": 2, "harmed": 58, "missed_rescue": 0, "avoided_harm": 466, "selected_both_correct": 33, "selected_both_wrong": 15}, {"budget": 0.1, "threshold": -0.6361178159713745, "operator": "ge", "actual_fraction": 0.1, "metric": 0.9333333333333333, "rescued": 2, "harmed": 65, "missed_rescue": 0, "avoided_harm": 459, "selected_both_correct": 38, "selected_both_wrong": 15}, {"budget": 0.11, "threshold": -0.6547113656997681, "operator": "ge", "actual_fraction": 0.11, "metric": 0.9291666666666667, "rescued": 2, "harmed": 70, "missed_rescue": 0, "avoided_harm": 454, "selected_both_correct": 45, "selected_both_wrong": 15}, {"budget": 0.12, "threshold": -0.6680066585540771, "operator": "ge", "actual_fraction": 0.12, "metric": 0.9225, "rescued": 2, "harmed": 78, "missed_rescue": 0, "avoided_harm": 446, "selected_both_correct": 49, "selected_both_wrong": 15}, {"budget": 0.13, "threshold": -0.6809748411178589, "operator": "ge", "actual_fraction": 0.13, "metric": 0.9141666666666667, "rescued": 2, "harmed": 88, "missed_rescue": 0, "avoided_harm": 436, "selected_both_correct": 51, "selected_both_wrong": 15}, {"budget": 0.14, "threshold": -0.6963143348693848, "operator": "ge", "actual_fraction": 0.14, "metric": 0.9083333333333333, "rescued": 2, "harmed": 95, "missed_rescue": 0, "avoided_harm": 429, "selected_both_correct": 56, "selected_both_wrong": 15}, {"budget": 0.15, "threshold": -0.7065573930740356, "operator": "ge", "actual_fraction": 0.15, "metric": 0.9033333333333333, "rescued": 2, "harmed": 101, "missed_rescue": 0, "avoided_harm": 423, "selected_both_correct": 62, "selected_both_wrong": 15}, {"budget": 0.16, "threshold": -0.7129085659980774, "operator": "ge", "actual_fraction": 0.16, "metric": 0.8966666666666666, "rescued": 2, "harmed": 109, "missed_rescue": 0, "avoided_harm": 415, "selected_both_correct": 66, "selected_both_wrong": 15}, {"budget": 0.17, "threshold": -0.7240402102470398, "operator": "ge", "actual_fraction": 0.17, "metric": 0.8891666666666667, "rescued": 2, "harmed": 118, "missed_rescue": 0, "avoided_harm": 406, "selected_both_correct": 69, "selected_both_wrong": 15}, {"budget": 0.18, "threshold": -0.7345248460769653, "operator": "ge", "actual_fraction": 0.18, "metric": 0.8833333333333333, "rescued": 2, "harmed": 125, "missed_rescue": 0, "avoided_harm": 399, "selected_both_correct": 74, "selected_both_wrong": 15}, {"budget": 0.19, "threshold": -0.7443157434463501, "operator": "ge", "actual_fraction": 0.19, "metric": 0.8775, "rescued": 2, "harmed": 132, "missed_rescue": 0, "avoided_harm": 392, "selected_both_correct": 79, "selected_both_wrong": 15}, {"budget": 0.2, "threshold": -0.7533873319625854, "operator": "ge", "actual_fraction": 0.2, "metric": 0.8716666666666667, "rescued": 2, "harmed": 139, "missed_rescue": 0, "avoided_harm": 385, "selected_both_correct": 84, "selected_both_wrong": 15}, {"budget": 0.21, "threshold": -0.7606735825538635, "operator": "ge", "actual_fraction": 0.21, "metric": 0.8666666666666667, "rescued": 2, "harmed": 145, "missed_rescue": 0, "avoided_harm": 379, "selected_both_correct": 90, "selected_both_wrong": 15}, {"budget": 0.22, "threshold": -0.7657199501991272, "operator": "ge", "actual_fraction": 0.22, "metric": 0.8641666666666666, "rescued": 2, "harmed": 148, "missed_rescue": 0, "avoided_harm": 376, "selected_both_correct": 99, "selected_both_wrong": 15}, {"budget": 0.23, "threshold": -0.7712029218673706, "operator": "ge", "actual_fraction": 0.23, "metric": 0.86, "rescued": 2, "harmed": 153, "missed_rescue": 0, "avoided_harm": 371, "selected_both_correct": 106, "selected_both_wrong": 15}, {"budget": 0.24, "threshold": -0.7754930257797241, "operator": "ge", "actual_fraction": 0.24, "metric": 0.855, "rescued": 2, "harmed": 159, "missed_rescue": 0, "avoided_harm": 365, "selected_both_correct": 112, "selected_both_wrong": 15}, {"budget": 0.25, "threshold": -0.7811619639396667, "operator": "ge", "actual_fraction": 0.25, "metric": 0.8475, "rescued": 2, "harmed": 168, "missed_rescue": 0, "avoided_harm": 356, "selected_both_correct": 115, "selected_both_wrong": 15}, {"budget": 0.26, "threshold": -0.7841500043869019, "operator": "ge", "actual_fraction": 0.26, "metric": 0.8408333333333333, "rescued": 2, "harmed": 176, "missed_rescue": 0, "avoided_harm": 348, "selected_both_correct": 119, "selected_both_wrong": 15}, {"budget": 0.27, "threshold": -0.78773033618927, "operator": "ge", "actual_fraction": 0.27, "metric": 0.8358333333333333, "rescued": 2, "harmed": 182, "missed_rescue": 0, "avoided_harm": 342, "selected_both_correct": 125, "selected_both_wrong": 15}, {"budget": 0.28, "threshold": -0.7923226356506348, "operator": "ge", "actual_fraction": 0.28, "metric": 0.8316666666666667, "rescued": 2, "harmed": 187, "missed_rescue": 0, "avoided_harm": 337, "selected_both_correct": 132, "selected_both_wrong": 15}, {"budget": 0.29, "threshold": -0.7950519323348999, "operator": "ge", "actual_fraction": 0.29, "metric": 0.8291666666666667, "rescued": 2, "harmed": 190, "missed_rescue": 0, "avoided_harm": 334, "selected_both_correct": 141, "selected_both_wrong": 15}, {"budget": 0.3, "threshold": -0.800890326499939, "operator": "ge", "actual_fraction": 0.3, "metric": 0.825, "rescued": 2, "harmed": 195, "missed_rescue": 0, "avoided_harm": 329, "selected_both_correct": 148, "selected_both_wrong": 15}, {"budget": 0.31, "threshold": -0.8055772185325623, "operator": "ge", "actual_fraction": 0.31, "metric": 0.8208333333333333, "rescued": 2, "harmed": 200, "missed_rescue": 0, "avoided_harm": 324, "selected_both_correct": 155, "selected_both_wrong": 15}, {"budget": 0.32, "threshold": -0.809782087802887, "operator": "ge", "actual_fraction": 0.32, "metric": 0.8183333333333334, "rescued": 2, "harmed": 203, "missed_rescue": 0, "avoided_harm": 321, "selected_both_correct": 164, "selected_both_wrong": 15}, {"budget": 0.33, "threshold": -0.8134240508079529, "operator": "ge", "actual_fraction": 0.33, "metric": 0.8125, "rescued": 2, "harmed": 210, "missed_rescue": 0, "avoided_harm": 314, "selected_both_correct": 169, "selected_both_wrong": 15}, {"budget": 0.34, "threshold": -0.8167235255241394, "operator": "ge", "actual_fraction": 0.34, "metric": 0.81, "rescued": 2, "harmed": 213, "missed_rescue": 0, "avoided_harm": 311, "selected_both_correct": 178, "selected_both_wrong": 15}, {"budget": 0.35, "threshold": -0.8208380937576294, "operator": "ge", "actual_fraction": 0.35, "metric": 0.8041666666666667, "rescued": 2, "harmed": 220, "missed_rescue": 0, "avoided_harm": 304, "selected_both_correct": 183, "selected_both_wrong": 15}, {"budget": 0.36, "threshold": -0.8250406384468079, "operator": "ge", "actual_fraction": 0.36, "metric": 0.8016666666666666, "rescued": 2, "harmed": 223, "missed_rescue": 0, "avoided_harm": 301, "selected_both_correct": 192, "selected_both_wrong": 15}, {"budget": 0.37, "threshold": -0.8295461535453796, "operator": "ge", "actual_fraction": 0.37, "metric": 0.7966666666666666, "rescued": 2, "harmed": 229, "missed_rescue": 0, "avoided_harm": 295, "selected_both_correct": 198, "selected_both_wrong": 15}, {"budget": 0.38, "threshold": -0.8328037261962891, "operator": "ge", "actual_fraction": 0.38, "metric": 0.7916666666666666, "rescued": 2, "harmed": 235, "missed_rescue": 0, "avoided_harm": 289, "selected_both_correct": 204, "selected_both_wrong": 15}, {"budget": 0.39, "threshold": -0.8359338641166687, "operator": "ge", "actual_fraction": 0.39, "metric": 0.7875, "rescued": 2, "harmed": 240, "missed_rescue": 0, "avoided_harm": 284, "selected_both_correct": 211, "selected_both_wrong": 15}, {"budget": 0.4, "threshold": -0.8391940593719482, "operator": "ge", "actual_fraction": 0.4, "metric": 0.78, "rescued": 2, "harmed": 249, "missed_rescue": 0, "avoided_harm": 275, "selected_both_correct": 214, "selected_both_wrong": 15}, {"budget": 0.41, "threshold": -0.8424465656280518, "operator": "ge", "actual_fraction": 0.4091666666666667, "metric": 0.7775, "rescued": 2, "harmed": 252, "missed_rescue": 0, "avoided_harm": 272, "selected_both_correct": 222, "selected_both_wrong": 15}, {"budget": 0.42, "threshold": -0.8460794687271118, "operator": "ge", "actual_fraction": 0.42, "metric": 0.7741666666666667, "rescued": 2, "harmed": 256, "missed_rescue": 0, "avoided_harm": 268, "selected_both_correct": 231, "selected_both_wrong": 15}, {"budget": 0.43, "threshold": -0.8500295281410217, "operator": "ge", "actual_fraction": 0.43, "metric": 0.7716666666666666, "rescued": 2, "harmed": 259, "missed_rescue": 0, "avoided_harm": 265, "selected_both_correct": 240, "selected_both_wrong": 15}, {"budget": 0.44, "threshold": -0.852453887462616, "operator": "ge", "actual_fraction": 0.44, "metric": 0.7666666666666667, "rescued": 2, "harmed": 265, "missed_rescue": 0, "avoided_harm": 259, "selected_both_correct": 246, "selected_both_wrong": 15}, {"budget": 0.45, "threshold": -0.8539360165596008, "operator": "ge", "actual_fraction": 0.45, "metric": 0.7625, "rescued": 2, "harmed": 270, "missed_rescue": 0, "avoided_harm": 254, "selected_both_correct": 253, "selected_both_wrong": 15}, {"budget": 0.46, "threshold": -0.8562509417533875, "operator": "ge", "actual_fraction": 0.46, "metric": 0.76, "rescued": 2, "harmed": 273, "missed_rescue": 0, "avoided_harm": 251, "selected_both_correct": 262, "selected_both_wrong": 15}, {"budget": 0.47, "threshold": -0.8592684864997864, "operator": "ge", "actual_fraction": 0.47, "metric": 0.7558333333333334, "rescued": 2, "harmed": 278, "missed_rescue": 0, "avoided_harm": 246, "selected_both_correct": 269, "selected_both_wrong": 15}, {"budget": 0.48, "threshold": -0.8621411919593811, "operator": "ge", "actual_fraction": 0.48, "metric": 0.7525, "rescued": 2, "harmed": 282, "missed_rescue": 0, "avoided_harm": 242, "selected_both_correct": 277, "selected_both_wrong": 15}, {"budget": 0.49, "threshold": -0.863978385925293, "operator": "ge", "actual_fraction": 0.49, "metric": 0.7483333333333333, "rescued": 2, "harmed": 287, "missed_rescue": 0, "avoided_harm": 237, "selected_both_correct": 284, "selected_both_wrong": 15}, {"budget": 0.5, "threshold": -0.8661247491836548, "operator": "ge", "actual_fraction": 0.5, "metric": 0.7425, "rescued": 2, "harmed": 294, "missed_rescue": 0, "avoided_harm": 230, "selected_both_correct": 289, "selected_both_wrong": 15}, {"budget": 0.51, "threshold": -0.8688956499099731, "operator": "ge", "actual_fraction": 0.51, "metric": 0.7383333333333333, "rescued": 2, "harmed": 299, "missed_rescue": 0, "avoided_harm": 225, "selected_both_correct": 296, "selected_both_wrong": 15}, {"budget": 0.52, "threshold": -0.8706188201904297, "operator": "ge", "actual_fraction": 0.52, "metric": 0.7358333333333333, "rescued": 2, "harmed": 302, "missed_rescue": 0, "avoided_harm": 222, "selected_both_correct": 305, "selected_both_wrong": 15}, {"budget": 0.53, "threshold": -0.8727357983589172, "operator": "ge", "actual_fraction": 0.53, "metric": 0.73, "rescued": 2, "harmed": 309, "missed_rescue": 0, "avoided_harm": 215, "selected_both_correct": 310, "selected_both_wrong": 15}, {"budget": 0.54, "threshold": -0.874894380569458, "operator": "ge", "actual_fraction": 0.54, "metric": 0.725, "rescued": 2, "harmed": 315, "missed_rescue": 0, "avoided_harm": 209, "selected_both_correct": 316, "selected_both_wrong": 15}, {"budget": 0.55, "threshold": -0.8764559626579285, "operator": "ge", "actual_fraction": 0.55, "metric": 0.72, "rescued": 2, "harmed": 321, "missed_rescue": 0, "avoided_harm": 203, "selected_both_correct": 322, "selected_both_wrong": 15}, {"budget": 0.56, "threshold": -0.879269540309906, "operator": "ge", "actual_fraction": 0.56, "metric": 0.7166666666666667, "rescued": 2, "harmed": 325, "missed_rescue": 0, "avoided_harm": 199, "selected_both_correct": 330, "selected_both_wrong": 15}, {"budget": 0.57, "threshold": -0.881222128868103, "operator": "ge", "actual_fraction": 0.5691666666666667, "metric": 0.7116666666666667, "rescued": 2, "harmed": 331, "missed_rescue": 0, "avoided_harm": 193, "selected_both_correct": 335, "selected_both_wrong": 15}, {"budget": 0.58, "threshold": -0.882451593875885, "operator": "ge", "actual_fraction": 0.58, "metric": 0.7075, "rescued": 2, "harmed": 336, "missed_rescue": 0, "avoided_harm": 188, "selected_both_correct": 343, "selected_both_wrong": 15}, {"budget": 0.59, "threshold": -0.8843295574188232, "operator": "ge", "actual_fraction": 0.59, "metric": 0.7025, "rescued": 2, "harmed": 342, "missed_rescue": 0, "avoided_harm": 182, "selected_both_correct": 349, "selected_both_wrong": 15}, {"budget": 0.6, "threshold": -0.8863861560821533, "operator": "ge", "actual_fraction": 0.6, "metric": 0.6983333333333334, "rescued": 2, "harmed": 347, "missed_rescue": 0, "avoided_harm": 177, "selected_both_correct": 356, "selected_both_wrong": 15}, {"budget": 0.61, "threshold": -0.8876380920410156, "operator": "ge", "actual_fraction": 0.61, "metric": 0.6933333333333334, "rescued": 2, "harmed": 353, "missed_rescue": 0, "avoided_harm": 171, "selected_both_correct": 362, "selected_both_wrong": 15}, {"budget": 0.62, "threshold": -0.8888679146766663, "operator": "ge", "actual_fraction": 0.62, "metric": 0.69, "rescued": 2, "harmed": 357, "missed_rescue": 0, "avoided_harm": 167, "selected_both_correct": 370, "selected_both_wrong": 15}, {"budget": 0.63, "threshold": -0.8905391693115234, "operator": "ge", "actual_fraction": 0.63, "metric": 0.6883333333333334, "rescued": 2, "harmed": 359, "missed_rescue": 0, "avoided_harm": 165, "selected_both_correct": 380, "selected_both_wrong": 15}, {"budget": 0.64, "threshold": -0.8919612169265747, "operator": "ge", "actual_fraction": 0.64, "metric": 0.6875, "rescued": 2, "harmed": 360, "missed_rescue": 0, "avoided_harm": 164, "selected_both_correct": 391, "selected_both_wrong": 15}, {"budget": 0.65, "threshold": -0.8940081596374512, "operator": "ge", "actual_fraction": 0.65, "metric": 0.6841666666666667, "rescued": 2, "harmed": 364, "missed_rescue": 0, "avoided_harm": 160, "selected_both_correct": 399, "selected_both_wrong": 15}, {"budget": 0.66, "threshold": -0.895609974861145, "operator": "ge", "actual_fraction": 0.66, "metric": 0.68, "rescued": 2, "harmed": 369, "missed_rescue": 0, "avoided_harm": 155, "selected_both_correct": 406, "selected_both_wrong": 15}, {"budget": 0.67, "threshold": -0.8972456455230713, "operator": "ge", "actual_fraction": 0.67, "metric": 0.6733333333333333, "rescued": 2, "harmed": 377, "missed_rescue": 0, "avoided_harm": 147, "selected_both_correct": 410, "selected_both_wrong": 15}, {"budget": 0.68, "threshold": -0.8986453413963318, "operator": "ge", "actual_fraction": 0.68, "metric": 0.6716666666666666, "rescued": 2, "harmed": 379, "missed_rescue": 0, "avoided_harm": 145, "selected_both_correct": 420, "selected_both_wrong": 15}, {"budget": 0.69, "threshold": -0.8996874094009399, "operator": "ge", "actual_fraction": 0.6891666666666667, "metric": 0.67, "rescued": 2, "harmed": 381, "missed_rescue": 0, "avoided_harm": 143, "selected_both_correct": 429, "selected_both_wrong": 15}, {"budget": 0.7, "threshold": -0.9008907079696655, "operator": "ge", "actual_fraction": 0.7, "metric": 0.665, "rescued": 2, "harmed": 387, "missed_rescue": 0, "avoided_harm": 137, "selected_both_correct": 436, "selected_both_wrong": 15}, {"budget": 0.71, "threshold": -0.9031620621681213, "operator": "ge", "actual_fraction": 0.71, "metric": 0.6583333333333333, "rescued": 2, "harmed": 395, "missed_rescue": 0, "avoided_harm": 129, "selected_both_correct": 440, "selected_both_wrong": 15}, {"budget": 0.72, "threshold": -0.9056642055511475, "operator": "ge", "actual_fraction": 0.72, "metric": 0.6541666666666667, "rescued": 2, "harmed": 400, "missed_rescue": 0, "avoided_harm": 124, "selected_both_correct": 447, "selected_both_wrong": 15}, {"budget": 0.73, "threshold": -0.9068957567214966, "operator": "ge", "actual_fraction": 0.73, "metric": 0.6508333333333334, "rescued": 2, "harmed": 404, "missed_rescue": 0, "avoided_harm": 120, "selected_both_correct": 455, "selected_both_wrong": 15}, {"budget": 0.74, "threshold": -0.9093236327171326, "operator": "ge", "actual_fraction": 0.74, "metric": 0.6491666666666667, "rescued": 2, "harmed": 406, "missed_rescue": 0, "avoided_harm": 118, "selected_both_correct": 465, "selected_both_wrong": 15}, {"budget": 0.75, "threshold": -0.9112368822097778, "operator": "ge", "actual_fraction": 0.75, "metric": 0.645, "rescued": 2, "harmed": 411, "missed_rescue": 0, "avoided_harm": 113, "selected_both_correct": 472, "selected_both_wrong": 15}, {"budget": 0.76, "threshold": -0.9129221439361572, "operator": "ge", "actual_fraction": 0.76, "metric": 0.6425, "rescued": 2, "harmed": 414, "missed_rescue": 0, "avoided_harm": 110, "selected_both_correct": 481, "selected_both_wrong": 15}, {"budget": 0.77, "threshold": -0.9144289493560791, "operator": "ge", "actual_fraction": 0.77, "metric": 0.64, "rescued": 2, "harmed": 417, "missed_rescue": 0, "avoided_harm": 107, "selected_both_correct": 490, "selected_both_wrong": 15}, {"budget": 0.78, "threshold": -0.9158422946929932, "operator": "ge", "actual_fraction": 0.78, "metric": 0.6383333333333333, "rescued": 2, "harmed": 419, "missed_rescue": 0, "avoided_harm": 105, "selected_both_correct": 500, "selected_both_wrong": 15}, {"budget": 0.79, "threshold": -0.9167781472206116, "operator": "ge", "actual_fraction": 0.79, "metric": 0.6341666666666667, "rescued": 2, "harmed": 424, "missed_rescue": 0, "avoided_harm": 100, "selected_both_correct": 507, "selected_both_wrong": 15}, {"budget": 0.8, "threshold": -0.9187189936637878, "operator": "ge", "actual_fraction": 0.8, "metric": 0.6283333333333333, "rescued": 2, "harmed": 431, "missed_rescue": 0, "avoided_harm": 93, "selected_both_correct": 512, "selected_both_wrong": 15}, {"budget": 0.81, "threshold": -0.9207327365875244, "operator": "ge", "actual_fraction": 0.81, "metric": 0.625, "rescued": 2, "harmed": 435, "missed_rescue": 0, "avoided_harm": 89, "selected_both_correct": 520, "selected_both_wrong": 15}, {"budget": 0.82, "threshold": -0.9226345419883728, "operator": "ge", "actual_fraction": 0.8191666666666667, "metric": 0.6216666666666667, "rescued": 2, "harmed": 439, "missed_rescue": 0, "avoided_harm": 85, "selected_both_correct": 527, "selected_both_wrong": 15}, {"budget": 0.83, "threshold": -0.9243993759155273, "operator": "ge", "actual_fraction": 0.83, "metric": 0.6191666666666666, "rescued": 2, "harmed": 442, "missed_rescue": 0, "avoided_harm": 82, "selected_both_correct": 537, "selected_both_wrong": 15}, {"budget": 0.84, "threshold": -0.9264947175979614, "operator": "ge", "actual_fraction": 0.84, "metric": 0.6166666666666667, "rescued": 2, "harmed": 445, "missed_rescue": 0, "avoided_harm": 79, "selected_both_correct": 546, "selected_both_wrong": 15}, {"budget": 0.85, "threshold": -0.9274600148200989, "operator": "ge", "actual_fraction": 0.85, "metric": 0.6141666666666666, "rescued": 2, "harmed": 448, "missed_rescue": 0, "avoided_harm": 76, "selected_both_correct": 555, "selected_both_wrong": 15}, {"budget": 0.86, "threshold": -0.930335283279419, "operator": "ge", "actual_fraction": 0.86, "metric": 0.6108333333333333, "rescued": 2, "harmed": 452, "missed_rescue": 0, "avoided_harm": 72, "selected_both_correct": 563, "selected_both_wrong": 15}, {"budget": 0.87, "threshold": -0.9319414496421814, "operator": "ge", "actual_fraction": 0.87, "metric": 0.6066666666666667, "rescued": 2, "harmed": 457, "missed_rescue": 0, "avoided_harm": 67, "selected_both_correct": 570, "selected_both_wrong": 15}, {"budget": 0.88, "threshold": -0.9340114593505859, "operator": "ge", "actual_fraction": 0.88, "metric": 0.6033333333333334, "rescued": 2, "harmed": 461, "missed_rescue": 0, "avoided_harm": 63, "selected_both_correct": 578, "selected_both_wrong": 15}, {"budget": 0.89, "threshold": -0.9363258481025696, "operator": "ge", "actual_fraction": 0.89, "metric": 0.5983333333333334, "rescued": 2, "harmed": 467, "missed_rescue": 0, "avoided_harm": 57, "selected_both_correct": 584, "selected_both_wrong": 15}, {"budget": 0.9, "threshold": -0.9400071501731873, "operator": "ge", "actual_fraction": 0.9, "metric": 0.595, "rescued": 2, "harmed": 471, "missed_rescue": 0, "avoided_harm": 53, "selected_both_correct": 592, "selected_both_wrong": 15}, {"budget": 0.91, "threshold": -0.9423198103904724, "operator": "ge", "actual_fraction": 0.91, "metric": 0.5908333333333333, "rescued": 2, "harmed": 476, "missed_rescue": 0, "avoided_harm": 48, "selected_both_correct": 599, "selected_both_wrong": 15}, {"budget": 0.92, "threshold": -0.9443356394767761, "operator": "ge", "actual_fraction": 0.92, "metric": 0.5866666666666667, "rescued": 2, "harmed": 481, "missed_rescue": 0, "avoided_harm": 43, "selected_both_correct": 606, "selected_both_wrong": 15}, {"budget": 0.93, "threshold": -0.9468213319778442, "operator": "ge", "actual_fraction": 0.93, "metric": 0.5833333333333334, "rescued": 2, "harmed": 485, "missed_rescue": 0, "avoided_harm": 39, "selected_both_correct": 614, "selected_both_wrong": 15}, {"budget": 0.94, "threshold": -0.9496435523033142, "operator": "ge", "actual_fraction": 0.94, "metric": 0.5791666666666667, "rescued": 2, "harmed": 490, "missed_rescue": 0, "avoided_harm": 34, "selected_both_correct": 621, "selected_both_wrong": 15}, {"budget": 0.95, "threshold": -0.9529175758361816, "operator": "ge", "actual_fraction": 0.95, "metric": 0.5766666666666667, "rescued": 2, "harmed": 493, "missed_rescue": 0, "avoided_harm": 31, "selected_both_correct": 630, "selected_both_wrong": 15}, {"budget": 0.96, "threshold": -0.9560379981994629, "operator": "ge", "actual_fraction": 0.96, "metric": 0.57, "rescued": 2, "harmed": 501, "missed_rescue": 0, "avoided_harm": 23, "selected_both_correct": 634, "selected_both_wrong": 15}, {"budget": 0.97, "threshold": -0.9591385126113892, "operator": "ge", "actual_fraction": 0.97, "metric": 0.5666666666666667, "rescued": 2, "harmed": 505, "missed_rescue": 0, "avoided_harm": 19, "selected_both_correct": 642, "selected_both_wrong": 15}, {"budget": 0.98, "threshold": -0.9627243280410767, "operator": "ge", "actual_fraction": 0.98, "metric": 0.56, "rescued": 2, "harmed": 513, "missed_rescue": 0, "avoided_harm": 11, "selected_both_correct": 646, "selected_both_wrong": 15}, {"budget": 0.99, "threshold": -0.9702208638191223, "operator": "ge", "actual_fraction": 0.99, "metric": 0.5558333333333333, "rescued": 2, "harmed": 518, "missed_rescue": 0, "avoided_harm": 6, "selected_both_correct": 653, "selected_both_wrong": 15}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.5508333333333333, "rescued": 2, "harmed": 524, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 659, "selected_both_wrong": 15}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.81429064549534, "rescued": 0, "harmed": 0, "missed_rescue": 365, "avoided_harm": 1907, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.305810809135437, "operator": "ge", "actual_fraction": 0.113565757680359, "metric": 0.8158439765274422, "rescued": 132, "harmed": 123, "missed_rescue": 233, "avoided_harm": 1784, "selected_both_correct": 108, "selected_both_wrong": 295}, {"budget": 0.02, "threshold": -0.3627302646636963, "operator": "ge", "actual_fraction": 0.15947531929582326, "metric": 0.8099758370728339, "rescued": 182, "harmed": 207, "missed_rescue": 183, "avoided_harm": 1700, "selected_both_correct": 170, "selected_both_wrong": 365}, {"budget": 0.03, "threshold": -0.4222676455974579, "operator": "ge", "actual_fraction": 0.2143596824301001, "metric": 0.798584742837418, "rescued": 226, "harmed": 317, "missed_rescue": 139, "avoided_harm": 1590, "selected_both_correct": 261, "selected_both_wrong": 438}, {"budget": 0.04, "threshold": -0.46618229150772095, "operator": "ge", "actual_fraction": 0.2545736969278564, "metric": 0.789264756644805, "rescued": 257, "harmed": 402, "missed_rescue": 108, "avoided_harm": 1505, "selected_both_correct": 329, "selected_both_wrong": 487}, {"budget": 0.05, "threshold": -0.5066347122192383, "operator": "ge", "actual_fraction": 0.29375215740421123, "metric": 0.7773558853986883, "rescued": 277, "harmed": 491, "missed_rescue": 88, "avoided_harm": 1416, "selected_both_correct": 417, "selected_both_wrong": 517}, {"budget": 0.06, "threshold": -0.5393391251564026, "operator": "ge", "actual_fraction": 0.3208491542975492, "metric": 0.7709699689333793, "rescued": 291, "harmed": 542, "missed_rescue": 74, "avoided_harm": 1365, "selected_both_correct": 480, "selected_both_wrong": 546}, {"budget": 0.07, "threshold": -0.5679479837417603, "operator": "ge", "actual_fraction": 0.350707628581291, "metric": 0.7604418363824647, "rescued": 303, "harmed": 615, "missed_rescue": 62, "avoided_harm": 1292, "selected_both_correct": 537, "selected_both_wrong": 577}, {"budget": 0.08, "threshold": -0.5907968282699585, "operator": "ge", "actual_fraction": 0.36882982395581637, "metric": 0.7540559199171557, "rescued": 312, "harmed": 661, "missed_rescue": 53, "avoided_harm": 1246, "selected_both_correct": 576, "selected_both_wrong": 588}, {"budget": 0.09, "threshold": -0.6168767809867859, "operator": "ge", "actual_fraction": 0.391439420089748, "metric": 0.7452537107352434, "rescued": 317, "harmed": 717, "missed_rescue": 48, "avoided_harm": 1190, "selected_both_correct": 627, "selected_both_wrong": 607}, {"budget": 0.1, "threshold": -0.6361178159713745, "operator": "ge", "actual_fraction": 0.4138764238867794, "metric": 0.7392129789437349, "rescued": 327, "harmed": 762, "missed_rescue": 38, "avoided_harm": 1145, "selected_both_correct": 688, "selected_both_wrong": 621}, {"budget": 0.11, "threshold": -0.6547113656997681, "operator": "ge", "actual_fraction": 0.43372454263030724, "metric": 0.732827062478426, "rescued": 336, "harmed": 808, "missed_rescue": 29, "avoided_harm": 1099, "selected_both_correct": 739, "selected_both_wrong": 630}, {"budget": 0.12, "threshold": -0.6680066585540771, "operator": "ge", "actual_fraction": 0.4480497065930273, "metric": 0.7267863306869174, "rescued": 338, "harmed": 845, "missed_rescue": 27, "avoided_harm": 1062, "selected_both_correct": 780, "selected_both_wrong": 633}, {"budget": 0.13, "threshold": -0.6809748411178589, "operator": "ge", "actual_fraction": 0.4630652399033483, "metric": 0.7216085605799103, "rescued": 343, "harmed": 880, "missed_rescue": 22, "avoided_harm": 1027, "selected_both_correct": 821, "selected_both_wrong": 639}, {"budget": 0.14, "threshold": -0.6963143348693848, "operator": "ge", "actual_fraction": 0.479288919571971, "metric": 0.7152226441146013, "rescued": 346, "harmed": 920, "missed_rescue": 19, "avoided_harm": 987, "selected_both_correct": 868, "selected_both_wrong": 643}, {"budget": 0.15, "threshold": -0.7065573930740356, "operator": "ge", "actual_fraction": 0.4932688988608906, "metric": 0.710735243355195, "rescued": 348, "harmed": 948, "missed_rescue": 17, "avoided_harm": 959, "selected_both_correct": 911, "selected_both_wrong": 651}, {"budget": 0.16, "threshold": -0.7129085659980774, "operator": "ge", "actual_fraction": 0.5008629616845012, "metric": 0.7086641353123921, "rescued": 350, "harmed": 962, "missed_rescue": 15, "avoided_harm": 945, "selected_both_correct": 937, "selected_both_wrong": 653}, {"budget": 0.17, "threshold": -0.7240402102470398, "operator": "ge", "actual_fraction": 0.5155333103210218, "metric": 0.701415257162582, "rescued": 350, "harmed": 1004, "missed_rescue": 15, "avoided_harm": 903, "selected_both_correct": 978, "selected_both_wrong": 655}, {"budget": 0.18, "threshold": -0.7345248460769653, "operator": "ge", "actual_fraction": 0.5298584742837418, "metric": 0.6967552640662754, "rescued": 355, "harmed": 1036, "missed_rescue": 10, "avoided_harm": 871, "selected_both_correct": 1020, "selected_both_wrong": 659}, {"budget": 0.19, "threshold": -0.7443157434463501, "operator": "ge", "actual_fraction": 0.5455643769416638, "metric": 0.6908871246116672, "rescued": 355, "harmed": 1070, "missed_rescue": 10, "avoided_harm": 837, "selected_both_correct": 1072, "selected_both_wrong": 664}, {"budget": 0.2, "threshold": -0.7533873319625854, "operator": "ge", "actual_fraction": 0.5560925094925785, "metric": 0.6872626855367622, "rescued": 356, "harmed": 1092, "missed_rescue": 9, "avoided_harm": 815, "selected_both_correct": 1106, "selected_both_wrong": 668}, {"budget": 0.21, "threshold": -0.7606735825538635, "operator": "ge", "actual_fraction": 0.5676561960648947, "metric": 0.6841560234725578, "rescued": 356, "harmed": 1110, "missed_rescue": 9, "avoided_harm": 797, "selected_both_correct": 1153, "selected_both_wrong": 670}, {"budget": 0.22, "threshold": -0.7657199501991272, "operator": "ge", "actual_fraction": 0.5766309975837073, "metric": 0.6808767690714532, "rescued": 356, "harmed": 1129, "missed_rescue": 9, "avoided_harm": 778, "selected_both_correct": 1184, "selected_both_wrong": 672}, {"budget": 0.23, "threshold": -0.7712029218673706, "operator": "ge", "actual_fraction": 0.5861235761132205, "metric": 0.6779426993441491, "rescued": 357, "harmed": 1147, "missed_rescue": 8, "avoided_harm": 760, "selected_both_correct": 1219, "selected_both_wrong": 673}, {"budget": 0.24, "threshold": -0.7754930257797241, "operator": "ge", "actual_fraction": 0.5923369002416292, "metric": 0.6755264066275457, "rescued": 357, "harmed": 1161, "missed_rescue": 8, "avoided_harm": 746, "selected_both_correct": 1238, "selected_both_wrong": 676}, {"budget": 0.25, "threshold": -0.7811619639396667, "operator": "ge", "actual_fraction": 0.6013117017604418, "metric": 0.6727649292371418, "rescued": 358, "harmed": 1178, "missed_rescue": 7, "avoided_harm": 729, "selected_both_correct": 1272, "selected_both_wrong": 676}, {"budget": 0.26, "threshold": -0.7841500043869019, "operator": "ge", "actual_fraction": 0.6052813255091474, "metric": 0.6712115982050397, "rescued": 359, "harmed": 1188, "missed_rescue": 6, "avoided_harm": 719, "selected_both_correct": 1284, "selected_both_wrong": 676}, {"budget": 0.27, "threshold": -0.78773033618927, "operator": "ge", "actual_fraction": 0.6123576113220573, "metric": 0.6681049361408353, "rescued": 359, "harmed": 1206, "missed_rescue": 6, "avoided_harm": 701, "selected_both_correct": 1305, "selected_both_wrong": 678}, {"budget": 0.28, "threshold": -0.7923226356506348, "operator": "ge", "actual_fraction": 0.6206420434932689, "metric": 0.664998274076631, "rescued": 359, "harmed": 1224, "missed_rescue": 6, "avoided_harm": 683, "selected_both_correct": 1333, "selected_both_wrong": 680}, {"budget": 0.29, "threshold": -0.7950519323348999, "operator": "ge", "actual_fraction": 0.6251294442526751, "metric": 0.6632723507076286, "rescued": 359, "harmed": 1234, "missed_rescue": 6, "avoided_harm": 673, "selected_both_correct": 1348, "selected_both_wrong": 681}, {"budget": 0.3, "threshold": -0.800890326499939, "operator": "ge", "actual_fraction": 0.6373835001725924, "metric": 0.6580945806006213, "rescued": 359, "harmed": 1264, "missed_rescue": 6, "avoided_harm": 643, "selected_both_correct": 1388, "selected_both_wrong": 682}, {"budget": 0.31, "threshold": -0.8055772185325623, "operator": "ge", "actual_fraction": 0.6470486710390059, "metric": 0.6546427338626165, "rescued": 359, "harmed": 1284, "missed_rescue": 6, "avoided_harm": 623, "selected_both_correct": 1423, "selected_both_wrong": 683}, {"budget": 0.32, "threshold": -0.809782087802887, "operator": "ge", "actual_fraction": 0.6556782878840179, "metric": 0.6527442181567138, "rescued": 360, "harmed": 1296, "missed_rescue": 5, "avoided_harm": 611, "selected_both_correct": 1457, "selected_both_wrong": 686}, {"budget": 0.33, "threshold": -0.8134240508079529, "operator": "ge", "actual_fraction": 0.6636175353814291, "metric": 0.6498101484294098, "rescued": 360, "harmed": 1313, "missed_rescue": 5, "avoided_harm": 594, "selected_both_correct": 1485, "selected_both_wrong": 687}, {"budget": 0.34, "threshold": -0.8167235255241394, "operator": "ge", "actual_fraction": 0.6708664135312392, "metric": 0.6472212633759061, "rescued": 360, "harmed": 1328, "missed_rescue": 5, "avoided_harm": 579, "selected_both_correct": 1511, "selected_both_wrong": 688}, {"budget": 0.35, "threshold": -0.8208380937576294, "operator": "ge", "actual_fraction": 0.6812219537452537, "metric": 0.6435968243010011, "rescued": 360, "harmed": 1349, "missed_rescue": 5, "avoided_harm": 558, "selected_both_correct": 1549, "selected_both_wrong": 689}, {"budget": 0.36, "threshold": -0.8250406384468079, "operator": "ge", "actual_fraction": 0.689333793579565, "metric": 0.6408353469105972, "rescued": 360, "harmed": 1365, "missed_rescue": 5, "avoided_harm": 542, "selected_both_correct": 1578, "selected_both_wrong": 691}, {"budget": 0.37, "threshold": -0.8295461535453796, "operator": "ge", "actual_fraction": 0.7017604418363824, "metric": 0.6347946151190887, "rescued": 360, "harmed": 1400, "missed_rescue": 5, "avoided_harm": 507, "selected_both_correct": 1613, "selected_both_wrong": 693}, {"budget": 0.38, "threshold": -0.8328037261962891, "operator": "ge", "actual_fraction": 0.7088367276492924, "metric": 0.6320331377286849, "rescued": 360, "harmed": 1416, "missed_rescue": 5, "avoided_harm": 491, "selected_both_correct": 1637, "selected_both_wrong": 694}, {"budget": 0.39, "threshold": -0.8359338641166687, "operator": "ge", "actual_fraction": 0.7166033828098033, "metric": 0.6290990680013807, "rescued": 360, "harmed": 1433, "missed_rescue": 5, "avoided_harm": 474, "selected_both_correct": 1665, "selected_both_wrong": 694}, {"budget": 0.4, "threshold": -0.8391940593719482, "operator": "ge", "actual_fraction": 0.7238522609596134, "metric": 0.6265101829478771, "rescued": 361, "harmed": 1449, "missed_rescue": 4, "avoided_harm": 458, "selected_both_correct": 1689, "selected_both_wrong": 695}, {"budget": 0.41, "threshold": -0.8424465656280518, "operator": "ge", "actual_fraction": 0.7302381774249224, "metric": 0.6244390749050742, "rescued": 361, "harmed": 1461, "missed_rescue": 4, "avoided_harm": 446, "selected_both_correct": 1713, "selected_both_wrong": 696}, {"budget": 0.42, "threshold": -0.8460794687271118, "operator": "ge", "actual_fraction": 0.7392129789437349, "metric": 0.6211598205039697, "rescued": 361, "harmed": 1480, "missed_rescue": 4, "avoided_harm": 427, "selected_both_correct": 1746, "selected_both_wrong": 696}, {"budget": 0.43, "threshold": -0.8500295281410217, "operator": "ge", "actual_fraction": 0.747152226441146, "metric": 0.618570935450466, "rescued": 361, "harmed": 1495, "missed_rescue": 4, "avoided_harm": 412, "selected_both_correct": 1777, "selected_both_wrong": 696}, {"budget": 0.44, "threshold": -0.852453887462616, "operator": "ge", "actual_fraction": 0.7523299965481532, "metric": 0.6175353814290645, "rescued": 361, "harmed": 1501, "missed_rescue": 4, "avoided_harm": 406, "selected_both_correct": 1800, "selected_both_wrong": 697}, {"budget": 0.45, "threshold": -0.8539360165596008, "operator": "ge", "actual_fraction": 0.7562996202968588, "metric": 0.6171901967552641, "rescued": 361, "harmed": 1503, "missed_rescue": 4, "avoided_harm": 404, "selected_both_correct": 1821, "selected_both_wrong": 697}, {"budget": 0.46, "threshold": -0.8562509417533875, "operator": "ge", "actual_fraction": 0.7613047980669658, "metric": 0.6159820503969624, "rescued": 361, "harmed": 1510, "missed_rescue": 4, "avoided_harm": 397, "selected_both_correct": 1843, "selected_both_wrong": 697}, {"budget": 0.47, "threshold": -0.8592684864997864, "operator": "ge", "actual_fraction": 0.7668277528477736, "metric": 0.6144287193648602, "rescued": 361, "harmed": 1519, "missed_rescue": 4, "avoided_harm": 388, "selected_both_correct": 1865, "selected_both_wrong": 698}, {"budget": 0.48, "threshold": -0.8621411919593811, "operator": "ge", "actual_fraction": 0.774249223334484, "metric": 0.6127027959958578, "rescued": 362, "harmed": 1530, "missed_rescue": 3, "avoided_harm": 377, "selected_both_correct": 1895, "selected_both_wrong": 699}, {"budget": 0.49, "threshold": -0.863978385925293, "operator": "ge", "actual_fraction": 0.7790818087676907, "metric": 0.6104590956161546, "rescued": 362, "harmed": 1543, "missed_rescue": 3, "avoided_harm": 364, "selected_both_correct": 1909, "selected_both_wrong": 700}, {"budget": 0.5, "threshold": -0.8661247491836548, "operator": "ge", "actual_fraction": 0.7849499482222989, "metric": 0.6080428028995513, "rescued": 362, "harmed": 1557, "missed_rescue": 3, "avoided_harm": 350, "selected_both_correct": 1928, "selected_both_wrong": 701}, {"budget": 0.51, "threshold": -0.8688956499099731, "operator": "ge", "actual_fraction": 0.7928891957197101, "metric": 0.6056265101829479, "rescued": 363, "harmed": 1572, "missed_rescue": 2, "avoided_harm": 335, "selected_both_correct": 1958, "selected_both_wrong": 701}, {"budget": 0.52, "threshold": -0.8706188201904297, "operator": "ge", "actual_fraction": 0.7970314118053159, "metric": 0.6044183638246462, "rescued": 363, "harmed": 1579, "missed_rescue": 2, "avoided_harm": 328, "selected_both_correct": 1975, "selected_both_wrong": 701}, {"budget": 0.53, "threshold": -0.8727357983589172, "operator": "ge", "actual_fraction": 0.8016914049016224, "metric": 0.6037279944770452, "rescued": 363, "harmed": 1583, "missed_rescue": 2, "avoided_harm": 324, "selected_both_correct": 1998, "selected_both_wrong": 701}, {"budget": 0.54, "threshold": -0.874894380569458, "operator": "ge", "actual_fraction": 0.8070417673455299, "metric": 0.6018294787711426, "rescued": 363, "harmed": 1594, "missed_rescue": 2, "avoided_harm": 313, "selected_both_correct": 2018, "selected_both_wrong": 701}, {"budget": 0.55, "threshold": -0.8764559626579285, "operator": "ge", "actual_fraction": 0.8120469451156369, "metric": 0.6007939247497411, "rescued": 363, "harmed": 1600, "missed_rescue": 2, "avoided_harm": 307, "selected_both_correct": 2041, "selected_both_wrong": 701}, {"budget": 0.56, "threshold": -0.879269540309906, "operator": "ge", "actual_fraction": 0.8180876769071453, "metric": 0.5990680013807387, "rescued": 363, "harmed": 1610, "missed_rescue": 2, "avoided_harm": 297, "selected_both_correct": 2065, "selected_both_wrong": 702}, {"budget": 0.57, "threshold": -0.881222128868103, "operator": "ge", "actual_fraction": 0.823610631687953, "metric": 0.5976872626855367, "rescued": 363, "harmed": 1618, "missed_rescue": 2, "avoided_harm": 289, "selected_both_correct": 2086, "selected_both_wrong": 705}, {"budget": 0.58, "threshold": -0.882451593875885, "operator": "ge", "actual_fraction": 0.8272350707628582, "metric": 0.5963065239903348, "rescued": 363, "harmed": 1626, "missed_rescue": 2, "avoided_harm": 281, "selected_both_correct": 2099, "selected_both_wrong": 705}, {"budget": 0.59, "threshold": -0.8843295574188232, "operator": "ge", "actual_fraction": 0.8313772868484639, "metric": 0.5957887469796341, "rescued": 364, "harmed": 1630, "missed_rescue": 1, "avoided_harm": 277, "selected_both_correct": 2118, "selected_both_wrong": 705}, {"budget": 0.6, "threshold": -0.8863861560821533, "operator": "ge", "actual_fraction": 0.8355195029340697, "metric": 0.5942354159475319, "rescued": 364, "harmed": 1639, "missed_rescue": 1, "avoided_harm": 268, "selected_both_correct": 2133, "selected_both_wrong": 705}, {"budget": 0.61, "threshold": -0.8876380920410156, "operator": "ge", "actual_fraction": 0.8391439420089748, "metric": 0.5931998619261305, "rescued": 364, "harmed": 1645, "missed_rescue": 1, "avoided_harm": 262, "selected_both_correct": 2148, "selected_both_wrong": 705}, {"budget": 0.62, "threshold": -0.8888679146766663, "operator": "ge", "actual_fraction": 0.8425957887469796, "metric": 0.5923369002416292, "rescued": 364, "harmed": 1650, "missed_rescue": 1, "avoided_harm": 257, "selected_both_correct": 2163, "selected_both_wrong": 705}, {"budget": 0.63, "threshold": -0.8905391693115234, "operator": "ge", "actual_fraction": 0.8469105971694857, "metric": 0.5913013462202278, "rescued": 364, "harmed": 1656, "missed_rescue": 1, "avoided_harm": 251, "selected_both_correct": 2182, "selected_both_wrong": 705}, {"budget": 0.64, "threshold": -0.8919612169265747, "operator": "ge", "actual_fraction": 0.8503624439074905, "metric": 0.5900931998619261, "rescued": 364, "harmed": 1663, "missed_rescue": 1, "avoided_harm": 244, "selected_both_correct": 2195, "selected_both_wrong": 705}, {"budget": 0.65, "threshold": -0.8940081596374512, "operator": "ge", "actual_fraction": 0.8560579910251985, "metric": 0.5881946841560235, "rescued": 364, "harmed": 1674, "missed_rescue": 1, "avoided_harm": 233, "selected_both_correct": 2217, "selected_both_wrong": 705}, {"budget": 0.66, "threshold": -0.895609974861145, "operator": "ge", "actual_fraction": 0.8603727994477045, "metric": 0.586468760787021, "rescued": 365, "harmed": 1685, "missed_rescue": 0, "avoided_harm": 222, "selected_both_correct": 2230, "selected_both_wrong": 705}, {"budget": 0.67, "threshold": -0.8972456455230713, "operator": "ge", "actual_fraction": 0.8643424231964101, "metric": 0.5861235761132205, "rescued": 365, "harmed": 1687, "missed_rescue": 0, "avoided_harm": 220, "selected_both_correct": 2250, "selected_both_wrong": 706}, {"budget": 0.68, "threshold": -0.8986453413963318, "operator": "ge", "actual_fraction": 0.8688298239558163, "metric": 0.5849154297549188, "rescued": 365, "harmed": 1694, "missed_rescue": 0, "avoided_harm": 213, "selected_both_correct": 2269, "selected_both_wrong": 706}, {"budget": 0.69, "threshold": -0.8996874094009399, "operator": "ge", "actual_fraction": 0.8712461166724197, "metric": 0.5840524680704177, "rescued": 365, "harmed": 1699, "missed_rescue": 0, "avoided_harm": 208, "selected_both_correct": 2278, "selected_both_wrong": 706}, {"budget": 0.7, "threshold": -0.9008907079696655, "operator": "ge", "actual_fraction": 0.8746979634104246, "metric": 0.5826717293752157, "rescued": 365, "harmed": 1707, "missed_rescue": 0, "avoided_harm": 200, "selected_both_correct": 2290, "selected_both_wrong": 706}, {"budget": 0.71, "threshold": -0.9031620621681213, "operator": "ge", "actual_fraction": 0.8817742492233345, "metric": 0.5806006213324129, "rescued": 365, "harmed": 1719, "missed_rescue": 0, "avoided_harm": 188, "selected_both_correct": 2319, "selected_both_wrong": 706}, {"budget": 0.72, "threshold": -0.9056642055511475, "operator": "ge", "actual_fraction": 0.8893683120469451, "metric": 0.5781843286158095, "rescued": 365, "harmed": 1733, "missed_rescue": 0, "avoided_harm": 174, "selected_both_correct": 2348, "selected_both_wrong": 707}, {"budget": 0.73, "threshold": -0.9068957567214966, "operator": "ge", "actual_fraction": 0.8933379357956507, "metric": 0.5773213669313082, "rescued": 365, "harmed": 1738, "missed_rescue": 0, "avoided_harm": 169, "selected_both_correct": 2366, "selected_both_wrong": 707}, {"budget": 0.74, "threshold": -0.9093236327171326, "operator": "ge", "actual_fraction": 0.8981705212288574, "metric": 0.5762858129099068, "rescued": 365, "harmed": 1744, "missed_rescue": 0, "avoided_harm": 163, "selected_both_correct": 2388, "selected_both_wrong": 707}, {"budget": 0.75, "threshold": -0.9112368822097778, "operator": "ge", "actual_fraction": 0.9023127373144633, "metric": 0.5754228512254056, "rescued": 365, "harmed": 1749, "missed_rescue": 0, "avoided_harm": 158, "selected_both_correct": 2407, "selected_both_wrong": 707}, {"budget": 0.76, "threshold": -0.9129221439361572, "operator": "ge", "actual_fraction": 0.9081808767690714, "metric": 0.5738695201933034, "rescued": 365, "harmed": 1758, "missed_rescue": 0, "avoided_harm": 149, "selected_both_correct": 2431, "selected_both_wrong": 708}, {"budget": 0.77, "threshold": -0.9144289493560791, "operator": "ge", "actual_fraction": 0.9123230928546773, "metric": 0.5723161891612012, "rescued": 365, "harmed": 1767, "missed_rescue": 0, "avoided_harm": 140, "selected_both_correct": 2446, "selected_both_wrong": 708}, {"budget": 0.78, "threshold": -0.9158422946929932, "operator": "ge", "actual_fraction": 0.9149119779081809, "metric": 0.5711080428028995, "rescued": 365, "harmed": 1774, "missed_rescue": 0, "avoided_harm": 133, "selected_both_correct": 2454, "selected_both_wrong": 708}, {"budget": 0.79, "threshold": -0.9167781472206116, "operator": "ge", "actual_fraction": 0.9168104936140835, "metric": 0.5705902657921988, "rescued": 365, "harmed": 1777, "missed_rescue": 0, "avoided_harm": 130, "selected_both_correct": 2462, "selected_both_wrong": 708}, {"budget": 0.8, "threshold": -0.9187189936637878, "operator": "ge", "actual_fraction": 0.922851225405592, "metric": 0.5683465654124957, "rescued": 365, "harmed": 1790, "missed_rescue": 0, "avoided_harm": 117, "selected_both_correct": 2484, "selected_both_wrong": 708}, {"budget": 0.81, "threshold": -0.9207327365875244, "operator": "ge", "actual_fraction": 0.9280289955125992, "metric": 0.5667932343803935, "rescued": 365, "harmed": 1799, "missed_rescue": 0, "avoided_harm": 108, "selected_both_correct": 2505, "selected_both_wrong": 708}, {"budget": 0.82, "threshold": -0.9226345419883728, "operator": "ge", "actual_fraction": 0.9342423196410079, "metric": 0.5655850880220918, "rescued": 365, "harmed": 1806, "missed_rescue": 0, "avoided_harm": 101, "selected_both_correct": 2534, "selected_both_wrong": 708}, {"budget": 0.83, "threshold": -0.9243993759155273, "operator": "ge", "actual_fraction": 0.9387297204004142, "metric": 0.5642043493268899, "rescued": 365, "harmed": 1814, "missed_rescue": 0, "avoided_harm": 93, "selected_both_correct": 2552, "selected_both_wrong": 708}, {"budget": 0.84, "threshold": -0.9264947175979614, "operator": "ge", "actual_fraction": 0.9440800828443218, "metric": 0.5626510182947877, "rescued": 365, "harmed": 1823, "missed_rescue": 0, "avoided_harm": 84, "selected_both_correct": 2574, "selected_both_wrong": 708}, {"budget": 0.85, "threshold": -0.9274600148200989, "operator": "ge", "actual_fraction": 0.9463237832240249, "metric": 0.5624784259578874, "rescued": 365, "harmed": 1824, "missed_rescue": 0, "avoided_harm": 83, "selected_both_correct": 2586, "selected_both_wrong": 708}, {"budget": 0.86, "threshold": -0.930335283279419, "operator": "ge", "actual_fraction": 0.9534000690369347, "metric": 0.5607525025888851, "rescued": 365, "harmed": 1834, "missed_rescue": 0, "avoided_harm": 73, "selected_both_correct": 2616, "selected_both_wrong": 709}, {"budget": 0.87, "threshold": -0.9319414496421814, "operator": "ge", "actual_fraction": 0.9558163617535381, "metric": 0.5600621332412841, "rescued": 365, "harmed": 1838, "missed_rescue": 0, "avoided_harm": 69, "selected_both_correct": 2626, "selected_both_wrong": 709}, {"budget": 0.88, "threshold": -0.9340114593505859, "operator": "ge", "actual_fraction": 0.958577839143942, "metric": 0.5591991715567829, "rescued": 365, "harmed": 1843, "missed_rescue": 0, "avoided_harm": 64, "selected_both_correct": 2637, "selected_both_wrong": 709}, {"budget": 0.89, "threshold": -0.9363258481025696, "operator": "ge", "actual_fraction": 0.9628926475664481, "metric": 0.5574732481877804, "rescued": 365, "harmed": 1853, "missed_rescue": 0, "avoided_harm": 54, "selected_both_correct": 2652, "selected_both_wrong": 709}, {"budget": 0.9, "threshold": -0.9400071501731873, "operator": "ge", "actual_fraction": 0.9694511563686572, "metric": 0.5562651018294787, "rescued": 365, "harmed": 1860, "missed_rescue": 0, "avoided_harm": 47, "selected_both_correct": 2683, "selected_both_wrong": 709}, {"budget": 0.91, "threshold": -0.9423198103904724, "operator": "ge", "actual_fraction": 0.9734207801173628, "metric": 0.5550569554711771, "rescued": 365, "harmed": 1867, "missed_rescue": 0, "avoided_harm": 40, "selected_both_correct": 2698, "selected_both_wrong": 710}, {"budget": 0.92, "threshold": -0.9443356394767761, "operator": "ge", "actual_fraction": 0.9761822575077667, "metric": 0.5545391784604764, "rescued": 365, "harmed": 1870, "missed_rescue": 0, "avoided_harm": 37, "selected_both_correct": 2710, "selected_both_wrong": 711}, {"budget": 0.93, "threshold": -0.9468213319778442, "operator": "ge", "actual_fraction": 0.9803244735933725, "metric": 0.5536762167759751, "rescued": 365, "harmed": 1875, "missed_rescue": 0, "avoided_harm": 32, "selected_both_correct": 2729, "selected_both_wrong": 711}, {"budget": 0.94, "threshold": -0.9496435523033142, "operator": "ge", "actual_fraction": 0.9837763203313773, "metric": 0.5526406627545737, "rescued": 365, "harmed": 1881, "missed_rescue": 0, "avoided_harm": 26, "selected_both_correct": 2743, "selected_both_wrong": 711}, {"budget": 0.95, "threshold": -0.9529175758361816, "operator": "ge", "actual_fraction": 0.9860200207110804, "metric": 0.5524680704176734, "rescued": 365, "harmed": 1882, "missed_rescue": 0, "avoided_harm": 25, "selected_both_correct": 2755, "selected_both_wrong": 711}, {"budget": 0.96, "threshold": -0.9560379981994629, "operator": "ge", "actual_fraction": 0.988608905764584, "metric": 0.5519502934069728, "rescued": 365, "harmed": 1885, "missed_rescue": 0, "avoided_harm": 22, "selected_both_correct": 2767, "selected_both_wrong": 711}, {"budget": 0.97, "threshold": -0.9591385126113892, "operator": "ge", "actual_fraction": 0.9920607525025888, "metric": 0.5505695547117708, "rescued": 365, "harmed": 1893, "missed_rescue": 0, "avoided_harm": 14, "selected_both_correct": 2779, "selected_both_wrong": 711}, {"budget": 0.98, "threshold": -0.9627243280410767, "operator": "ge", "actual_fraction": 0.9946496375560925, "metric": 0.5498791853641698, "rescued": 365, "harmed": 1897, "missed_rescue": 0, "avoided_harm": 10, "selected_both_correct": 2790, "selected_both_wrong": 711}, {"budget": 0.99, "threshold": -0.9702208638191223, "operator": "ge", "actual_fraction": 0.9972385226095961, "metric": 0.5488436313427684, "rescued": 365, "harmed": 1903, "missed_rescue": 0, "avoided_harm": 4, "selected_both_correct": 2799, "selected_both_wrong": 711}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.5481532619951675, "rescued": 365, "harmed": 1907, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 2811, "selected_both_wrong": 711}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.9858333333333335, "actual_fraction": 0.0, "metric_std": 1.2412670766236366e-16, "actual_fraction_std": 0.0, "per_seed": [{"seed": 2026, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.9858333333333333, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}, {"seed": 2027, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.9858333333333333, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}, {"seed": 2028, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.9858333333333333, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}, {"seed": 2029, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.9858333333333333, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}, {"seed": 2030, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.25, "epoch": 2, "metric": 0.9858333333333333, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}], "rescued": 0.0, "harmed": 0.0, "missed_rescue": 2.0, "avoided_harm": 524.0}, "validation_curve": [{"budget": 0.0, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.01, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.02, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.03, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.04, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.05, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.06, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.07, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.9858333333333335, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.08, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.9856666666666667, "metric_std": 0.00037267799624997355, "actual_fraction": 0.09, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.9853333333333334, "metric_std": 0.00045643546458764894, "actual_fraction": 0.1, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.9853333333333334, "metric_std": 0.00045643546458764894, "actual_fraction": 0.11000000000000001, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.12, "metric": 0.985, "metric_std": 0.0008333333333334081, "actual_fraction": 0.12, "actual_fraction_std": 0.0}, {"budget": 0.13, "metric": 0.9844999999999999, "metric_std": 0.0009501461875826223, "actual_fraction": 0.13, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.9844999999999999, "metric_std": 0.0009501461875826223, "actual_fraction": 0.14, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.9841666666666666, "metric_std": 0.000589255650988764, "actual_fraction": 0.15, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.9838333333333333, "metric_std": 0.0009501461875825639, "actual_fraction": 0.16, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.9831666666666667, "metric_std": 0.001236033081182598, "actual_fraction": 0.17, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.9825000000000002, "metric_std": 0.001020620726159681, "actual_fraction": 0.18, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.9813333333333333, "metric_std": 0.001263812574008621, "actual_fraction": 0.19, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.9806666666666667, "metric_std": 0.0012360330811826727, "actual_fraction": 0.2, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.9804999999999999, "metric_std": 0.0012638125740086503, "actual_fraction": 0.21000000000000002, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.9791666666666666, "metric_std": 0.0010206207261596358, "actual_fraction": 0.22000000000000003, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.23, "metric": 0.9778333333333332, "metric_std": 0.001513825177048741, "actual_fraction": 0.23000000000000004, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.24, "metric": 0.9775, "metric_std": 0.0016666666666666633, "actual_fraction": 0.24, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.977, "metric_std": 0.0009501461875826368, "actual_fraction": 0.25, "actual_fraction_std": 0.0}, {"budget": 0.26, "metric": 0.9755, "metric_std": 0.0013944333775567817, "actual_fraction": 0.26, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.9746666666666666, "metric_std": 0.0020916500663351973, "actual_fraction": 0.27, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.9738333333333333, "metric_std": 0.002400810048481322, "actual_fraction": 0.28, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.9731666666666665, "metric_std": 0.002386303510546062, "actual_fraction": 0.29, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.9719999999999999, "metric_std": 0.002922612986125006, "actual_fraction": 0.3, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.9709999999999999, "metric_std": 0.0027259045389660076, "actual_fraction": 0.31, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.9693333333333334, "metric_std": 0.0021570298509241354, "actual_fraction": 0.32, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.9671666666666667, "metric_std": 0.001918477405536893, "actual_fraction": 0.33, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.9640000000000001, "metric_std": 0.001707825127659913, "actual_fraction": 0.34, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.9628333333333334, "metric_std": 0.001513825177048732, "actual_fraction": 0.35, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.9608333333333332, "metric_std": 0.0016666666666666774, "actual_fraction": 0.36, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.9576666666666668, "metric_std": 0.0018066236157232595, "actual_fraction": 0.37, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.9560000000000001, "metric_std": 0.0024579802006254006, "actual_fraction": 0.38, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.9536666666666667, "metric_std": 0.0029814239699997276, "actual_fraction": 0.39, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.9513333333333334, "metric_std": 0.002674467755980245, "actual_fraction": 0.4, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.9491666666666667, "metric_std": 0.002041241452319249, "actual_fraction": 0.4091666666666668, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.42, "metric": 0.9461666666666668, "metric_std": 0.0020069324297987945, "actual_fraction": 0.42000000000000004, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.9436666666666668, "metric_std": 0.0019184774055369, "actual_fraction": 0.43, "actual_fraction_std": 0.0}, {"budget": 0.44, "metric": 0.9406666666666667, "metric_std": 0.0017078251276599237, "actual_fraction": 0.44000000000000006, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.45, "metric": 0.9365, "metric_std": 0.0021570298509241076, "actual_fraction": 0.45, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.933, "metric_std": 0.0028012893856619718, "actual_fraction": 0.4600000000000001, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.47, "metric": 0.9299999999999999, "metric_std": 0.002204792759220533, "actual_fraction": 0.4699999999999999, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.48, "metric": 0.9251666666666667, "metric_std": 0.0025954019170662495, "actual_fraction": 0.48, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.9216666666666666, "metric_std": 0.0020412414523193058, "actual_fraction": 0.49000000000000005, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.5, "metric": 0.9189999999999999, "metric_std": 0.001806623615723193, "actual_fraction": 0.5, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.9145, "metric_std": 0.0024008100484813276, "actual_fraction": 0.51, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.9098333333333333, "metric_std": 0.0031950482521134545, "actual_fraction": 0.52, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.9048333333333334, "metric_std": 0.0034054694569504358, "actual_fraction": 0.53, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.8996666666666666, "metric_std": 0.004022506128715728, "actual_fraction": 0.54, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.8939999999999999, "metric_std": 0.003301935459359824, "actual_fraction": 0.55, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.8893333333333334, "metric_std": 0.003505947327993752, "actual_fraction": 0.56, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.8848333333333332, "metric_std": 0.0040138648595974555, "actual_fraction": 0.5691666666666667, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.8791666666666667, "metric_std": 0.00338501600193166, "actual_fraction": 0.58, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.8744999999999999, "metric_std": 0.002922612986125049, "actual_fraction": 0.59, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.8691666666666666, "metric_std": 0.002429563289518855, "actual_fraction": 0.6, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.8643333333333334, "metric_std": 0.0018066236157232188, "actual_fraction": 0.61, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.858, "metric_std": 0.002862594006219571, "actual_fraction": 0.62, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.8526666666666667, "metric_std": 0.0025954019170662587, "actual_fraction": 0.63, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.8459999999999999, "metric_std": 0.0026614532371118686, "actual_fraction": 0.64, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.8408333333333333, "metric_std": 0.002357022603955174, "actual_fraction": 0.65, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.8345, "metric_std": 0.002674467755980205, "actual_fraction": 0.66, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.8283333333333334, "metric_std": 0.001954339899926415, "actual_fraction": 0.67, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.8216666666666669, "metric_std": 0.0026352313834736227, "actual_fraction": 0.68, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.8143333333333335, "metric_std": 0.0024579802006255056, "actual_fraction": 0.6891666666666667, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.8068333333333333, "metric_std": 0.002312406154251934, "actual_fraction": 0.7, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.7998333333333333, "metric_std": 0.0013693063937629064, "actual_fraction": 0.71, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.7931666666666667, "metric_std": 0.0014907119849998385, "actual_fraction": 0.72, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.7853333333333333, "metric_std": 0.0011180339887498876, "actual_fraction": 0.73, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.7781666666666667, "metric_std": 0.0016029486718059822, "actual_fraction": 0.74, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.7703333333333333, "metric_std": 0.001825741858350558, "actual_fraction": 0.75, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.7625, "metric_std": 0.0016666666666666217, "actual_fraction": 0.76, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.7550000000000001, "metric_std": 0.0013176156917368551, "actual_fraction": 0.77, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.7473333333333333, "metric_std": 0.0014907119849999317, "actual_fraction": 0.78, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.7391666666666667, "metric_std": 0.0014433756729740495, "actual_fraction": 0.79, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.7310000000000001, "metric_std": 0.0013693063937629097, "actual_fraction": 0.8, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.7226666666666667, "metric_std": 0.0014907119849998385, "actual_fraction": 0.8100000000000002, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.82, "metric": 0.7146666666666668, "metric_std": 0.0018257418583505808, "actual_fraction": 0.8191666666666666, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.83, "metric": 0.7053333333333334, "metric_std": 0.0013944333775567452, "actual_fraction": 0.8299999999999998, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.84, "metric": 0.6975, "metric_std": 0.002204792759220522, "actual_fraction": 0.8400000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.6883333333333335, "metric_std": 0.002568505834570412, "actual_fraction": 0.85, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.6795, "metric_std": 0.0015138251770487013, "actual_fraction": 0.86, "actual_fraction_std": 0.0}, {"budget": 0.87, "metric": 0.6723333333333332, "metric_std": 0.0010865337342004452, "actual_fraction": 0.8699999999999999, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.6633333333333333, "metric_std": 0.001020620726159681, "actual_fraction": 0.8800000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.89, "metric": 0.6548333333333334, "metric_std": 0.0009128709291752675, "actual_fraction": 0.89, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.6463333333333333, "metric_std": 0.0011180339887498876, "actual_fraction": 0.9, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.6371666666666667, "metric_std": 0.00045643546458758817, "actual_fraction": 0.9099999999999999, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.6275000000000001, "metric_std": 0.0008333333333333526, "actual_fraction": 0.9200000000000002, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.93, "metric": 0.6178333333333333, "metric_std": 0.0007453559924999595, "actual_fraction": 0.93, "actual_fraction_std": 0.0}, {"budget": 0.94, "metric": 0.6086666666666667, "metric_std": 0.00045643546458758817, "actual_fraction": 0.9399999999999998, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.95, "metric": 0.5986666666666667, "metric_std": 0.00045643546458758817, "actual_fraction": 0.95, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.5888333333333333, "metric_std": 0.00045643546458758817, "actual_fraction": 0.96, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.579, "metric_std": 0.0006972166887783991, "actual_fraction": 0.97, "actual_fraction_std": 0.0}, {"budget": 0.98, "metric": 0.5696666666666668, "metric_std": 0.00045643546458770976, "actual_fraction": 0.9800000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.99, "metric": 0.5605, "metric_std": 0.00045643546458758817, "actual_fraction": 0.99, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.5508333333333333, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.8142906454953401, "actual_fraction": 0.0, "metric_std": 1.2412670766236366e-16, "actual_fraction_std": 0.0, "per_seed": [{"seed": 2026, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.81429064549534, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 365}, {"seed": 2027, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.81429064549534, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 365}, {"seed": 2028, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.81429064549534, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 365}, {"seed": 2029, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.81429064549534, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 365}, {"seed": 2030, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.25, "epoch": 2, "metric": 0.81429064549534, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 365}], "rescued": 0.0, "harmed": 0.0, "missed_rescue": 365.0, "avoided_harm": 1907.0}, "test_curve": [{"budget": 0.0, "metric": 0.8142906454953401, "metric_std": 1.2412670766236366e-16, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.8142906454953401, "metric_std": 0.00012204121180300483, "actual_fraction": 0.008146358301691404, "actual_fraction_std": 0.0018118085567153204}, {"budget": 0.02, "metric": 0.8142906454953401, "metric_std": 0.00012204121180300483, "actual_fraction": 0.015464273386261652, "actual_fraction_std": 0.002026031148198714}, {"budget": 0.03, "metric": 0.8142906454953401, "metric_std": 0.00012204121180300483, "actual_fraction": 0.02323092854677252, "actual_fraction_std": 0.0019434847442539515}, {"budget": 0.04, "metric": 0.81425612702796, "metric_std": 0.00014440110917054703, "actual_fraction": 0.030928546772523303, "actual_fraction_std": 0.003155184297578292}, {"budget": 0.05, "metric": 0.8142216085605799, "metric_std": 0.0001543712790818079, "actual_fraction": 0.03772868484639282, "actual_fraction_std": 0.0037903572057459045}, {"budget": 0.06, "metric": 0.8140490162236796, "metric_std": 0.0001543712790818079, "actual_fraction": 0.045253710735243355, "actual_fraction_std": 0.0037385269082671193}, {"budget": 0.07, "metric": 0.8139109423541594, "metric_std": 0.00014440110917054703, "actual_fraction": 0.05129444252675182, "actual_fraction_std": 0.0035316120467322873}, {"budget": 0.08, "metric": 0.8138419054193994, "metric_std": 9.453271617280195e-05, "actual_fraction": 0.059337245426303066, "actual_fraction_std": 0.004552317150224353}, {"budget": 0.09, "metric": 0.8136693130824991, "metric_std": 0.00015437127908175203, "actual_fraction": 0.06582671729375215, "actual_fraction_std": 0.005272688830418076}, {"budget": 0.1, "metric": 0.813634794615119, "metric_std": 0.00018906543234555831, "actual_fraction": 0.07324818778046256, "actual_fraction_std": 0.005328884479658877}, {"budget": 0.11, "metric": 0.8134622022782189, "metric_std": 0.0001890654323455583, "actual_fraction": 0.0806351397997929, "actual_fraction_std": 0.005045780582084395}, {"budget": 0.12, "metric": 0.8133241284086987, "metric_std": 0.00035789508721654746, "actual_fraction": 0.08847083189506386, "actual_fraction_std": 0.006501028935236854}, {"budget": 0.13, "metric": 0.8132550914739385, "metric_std": 0.00040476490851081254, "actual_fraction": 0.09744563341387642, "actual_fraction_std": 0.008137138408627615}, {"budget": 0.14, "metric": 0.8128408698653781, "metric_std": 0.0001543712790818079, "actual_fraction": 0.10507421470486711, "actual_fraction_std": 0.00892687877127766}, {"budget": 0.15, "metric": 0.812875388332758, "metric_std": 0.00014440110917054703, "actual_fraction": 0.11108042802899551, "actual_fraction_std": 0.008783237421182847}, {"budget": 0.16, "metric": 0.812806351397998, "metric_std": 0.0002888022183410543, "actual_fraction": 0.11788056610286504, "actual_fraction_std": 0.007868582878878056}, {"budget": 0.17, "metric": 0.8125302036589576, "metric_std": 0.0003319879886377293, "actual_fraction": 0.1252675181221954, "actual_fraction_std": 0.007808158199761985}, {"budget": 0.18, "metric": 0.8123921297894373, "metric_std": 0.00045663640163350375, "actual_fraction": 0.1322057300655851, "actual_fraction_std": 0.00830687473324741}, {"budget": 0.19, "metric": 0.812219537452537, "metric_std": 0.00032289069613168175, "actual_fraction": 0.1402830514325164, "actual_fraction_std": 0.012124166358551047}, {"budget": 0.2, "metric": 0.8119779081808767, "metric_std": 0.00028880221834113717, "actual_fraction": 0.14704867103900587, "actual_fraction_std": 0.01257972017355305}, {"budget": 0.21, "metric": 0.8114601311701761, "metric_std": 0.00035789508721653944, "actual_fraction": 0.1575077666551605, "actual_fraction_std": 0.012394605448930564}, {"budget": 0.22, "metric": 0.8113565757680359, "metric_std": 0.00042276315892015784, "actual_fraction": 0.16479116327235072, "actual_fraction_std": 0.01332900123909626}, {"budget": 0.23, "metric": 0.8109768726268554, "metric_std": 0.0004120585566555266, "actual_fraction": 0.17172937521574042, "actual_fraction_std": 0.0135981514314311}, {"budget": 0.24, "metric": 0.8105281325509146, "metric_std": 0.0004467441872814894, "actual_fraction": 0.18118743527787365, "actual_fraction_std": 0.014703003312664557}, {"budget": 0.25, "metric": 0.809941318605454, "metric_std": 0.0005234984773249501, "actual_fraction": 0.19095616154642733, "actual_fraction_std": 0.013792600816039993}, {"budget": 0.26, "metric": 0.8096306523990334, "metric_std": 0.0004726635808639489, "actual_fraction": 0.1986882982395582, "actual_fraction_std": 0.014531311333658899}, {"budget": 0.27, "metric": 0.8092164307904728, "metric_std": 0.0007871421643763273, "actual_fraction": 0.20845702450811182, "actual_fraction_std": 0.01601364866312133}, {"budget": 0.28, "metric": 0.8088712461166724, "metric_std": 0.0008851143531210506, "actual_fraction": 0.21674145667932346, "actual_fraction_std": 0.016039669906799014}, {"budget": 0.29, "metric": 0.8081808767690714, "metric_std": 0.0007483425401684249, "actual_fraction": 0.22616499827407663, "actual_fraction_std": 0.017132169481185372}, {"budget": 0.3, "metric": 0.8075250258888506, "metric_std": 0.0006150683355611166, "actual_fraction": 0.23500172592336904, "actual_fraction_std": 0.017909979233221546}, {"budget": 0.31, "metric": 0.8069382119433899, "metric_std": 0.0008595029063160439, "actual_fraction": 0.2450465999309631, "actual_fraction_std": 0.019529415887583185}, {"budget": 0.32, "metric": 0.806385916465309, "metric_std": 0.0011539188574695672, "actual_fraction": 0.25643769416637896, "actual_fraction_std": 0.020543346208403045}, {"budget": 0.33, "metric": 0.8057991025198481, "metric_std": 0.0013335480684902891, "actual_fraction": 0.2661028650327925, "actual_fraction_std": 0.021139708020754735}, {"budget": 0.34, "metric": 0.8043838453572661, "metric_std": 0.0014776565269591308, "actual_fraction": 0.277148774594408, "actual_fraction_std": 0.02241389107215589}, {"budget": 0.35, "metric": 0.8035554021401449, "metric_std": 0.0018035692754729917, "actual_fraction": 0.28753883327580254, "actual_fraction_std": 0.024923014724623643}, {"budget": 0.36, "metric": 0.8026234035208837, "metric_std": 0.002000135948335218, "actual_fraction": 0.29906800138073864, "actual_fraction_std": 0.02625656473140242}, {"budget": 0.37, "metric": 0.8022782188470832, "metric_std": 0.0020260311481986775, "actual_fraction": 0.3119088712461166, "actual_fraction_std": 0.025071197959088302}, {"budget": 0.38, "metric": 0.8011391094235416, "metric_std": 0.002118742290565961, "actual_fraction": 0.3262340352088367, "actual_fraction_std": 0.02598830282071733}, {"budget": 0.39, "metric": 0.8007939247497411, "metric_std": 0.002118742290565965, "actual_fraction": 0.337107352433552, "actual_fraction_std": 0.02632737559725449}, {"budget": 0.4, "metric": 0.7993786675871591, "metric_std": 0.003013207158135345, "actual_fraction": 0.3468760787021056, "actual_fraction_std": 0.029724589642601128}, {"budget": 0.41, "metric": 0.798550224370038, "metric_std": 0.003102824308490747, "actual_fraction": 0.356437694166379, "actual_fraction_std": 0.030952163258326944}, {"budget": 0.42, "metric": 0.7982050396962375, "metric_std": 0.0033983602062123133, "actual_fraction": 0.3681049361408354, "actual_fraction_std": 0.029229418005828966}, {"budget": 0.43, "metric": 0.7969623748705558, "metric_std": 0.0035968011119367265, "actual_fraction": 0.38522609596133933, "actual_fraction_std": 0.030488816131446633}, {"budget": 0.44, "metric": 0.7954090438384536, "metric_std": 0.0037883919662267886, "actual_fraction": 0.39934414911977906, "actual_fraction_std": 0.03089966837785876}, {"budget": 0.45, "metric": 0.7939247497411115, "metric_std": 0.004302709997348254, "actual_fraction": 0.4130824991370384, "actual_fraction_std": 0.03495082182691313}, {"budget": 0.46, "metric": 0.7921643079047289, "metric_std": 0.0044864049238887375, "actual_fraction": 0.42640662754573694, "actual_fraction_std": 0.037202677569188145}, {"budget": 0.47, "metric": 0.7906454953400069, "metric_std": 0.004276669552421099, "actual_fraction": 0.43848809112875387, "actual_fraction_std": 0.03520156002424398}, {"budget": 0.48, "metric": 0.7887469796341043, "metric_std": 0.004422205031176709, "actual_fraction": 0.4510528132550914, "actual_fraction_std": 0.03858869011778491}, {"budget": 0.49, "metric": 0.7867794269934414, "metric_std": 0.004581993374123574, "actual_fraction": 0.4657921988263721, "actual_fraction_std": 0.038033297309923386}, {"budget": 0.5, "metric": 0.7853986882982397, "metric_std": 0.004627277481101205, "actual_fraction": 0.4781152916810494, "actual_fraction_std": 0.03596415985972227}, {"budget": 0.51, "metric": 0.7831895063859166, "metric_std": 0.0032159653983713514, "actual_fraction": 0.49371763893683107, "actual_fraction_std": 0.033962251514410596}, {"budget": 0.52, "metric": 0.781325509147394, "metric_std": 0.0037911430163080437, "actual_fraction": 0.5088367276492923, "actual_fraction_std": 0.035810185755465296}, {"budget": 0.53, "metric": 0.7787366240938903, "metric_std": 0.00460048427175071, "actual_fraction": 0.5198826372109079, "actual_fraction_std": 0.037544566077752674}, {"budget": 0.54, "metric": 0.7767690714532275, "metric_std": 0.004478430268037823, "actual_fraction": 0.5355195029340697, "actual_fraction_std": 0.03561354107264749}, {"budget": 0.55, "metric": 0.774249223334484, "metric_std": 0.004352613121476875, "actual_fraction": 0.5502243700379704, "actual_fraction_std": 0.03488073621817781}, {"budget": 0.56, "metric": 0.7707283396617189, "metric_std": 0.0047118024075887, "actual_fraction": 0.5645150155333104, "actual_fraction_std": 0.03467418277876143}, {"budget": 0.57, "metric": 0.7685881946841561, "metric_std": 0.004604367630015375, "actual_fraction": 0.5772868484639282, "actual_fraction_std": 0.03308001280633379}, {"budget": 0.58, "metric": 0.7662754573696928, "metric_std": 0.005458394670563711, "actual_fraction": 0.5884363134276838, "actual_fraction_std": 0.03398592482374056}, {"budget": 0.59, "metric": 0.7637556092509492, "metric_std": 0.0067116009368153595, "actual_fraction": 0.6017949603037624, "actual_fraction_std": 0.03515028431525718}, {"budget": 0.6, "metric": 0.7608905764584052, "metric_std": 0.006117418456246462, "actual_fraction": 0.614981014842941, "actual_fraction_std": 0.031691590821805886}, {"budget": 0.61, "metric": 0.7583362098722818, "metric_std": 0.0069706795071288975, "actual_fraction": 0.6290645495340007, "actual_fraction_std": 0.032582809478218434}, {"budget": 0.62, "metric": 0.7554021401449778, "metric_std": 0.00750667547488394, "actual_fraction": 0.6419744563341389, "actual_fraction_std": 0.03175455109123171}, {"budget": 0.63, "metric": 0.7521228857438731, "metric_std": 0.007215309290686041, "actual_fraction": 0.657887469796341, "actual_fraction_std": 0.02781185070167844}, {"budget": 0.64, "metric": 0.7479461511908871, "metric_std": 0.00725975973700272, "actual_fraction": 0.672212633759061, "actual_fraction_std": 0.025663035416305182}, {"budget": 0.65, "metric": 0.7429754918881601, "metric_std": 0.008625646307829712, "actual_fraction": 0.6880911287538833, "actual_fraction_std": 0.026739115318441996}, {"budget": 0.66, "metric": 0.7379357956506731, "metric_std": 0.008578203767889966, "actual_fraction": 0.702865032792544, "actual_fraction_std": 0.02418775537861567}, {"budget": 0.67, "metric": 0.7332758025543666, "metric_std": 0.008589482093197615, "actual_fraction": 0.7158784949948223, "actual_fraction_std": 0.02208959418809807}, {"budget": 0.68, "metric": 0.7276492923714187, "metric_std": 0.010305800462116751, "actual_fraction": 0.7305143251639628, "actual_fraction_std": 0.022848318265145013}, {"budget": 0.69, "metric": 0.722091819123231, "metric_std": 0.011605727831347253, "actual_fraction": 0.7426648256817396, "actual_fraction_std": 0.02449808241598392}, {"budget": 0.7, "metric": 0.7152916810493614, "metric_std": 0.011455373496208312, "actual_fraction": 0.7573351743182603, "actual_fraction_std": 0.023744151994594957}, {"budget": 0.71, "metric": 0.708560579910252, "metric_std": 0.012219732470570574, "actual_fraction": 0.7710044874007596, "actual_fraction_std": 0.024587715359705605}, {"budget": 0.72, "metric": 0.7028995512599241, "metric_std": 0.012410217150658452, "actual_fraction": 0.7822920262340352, "actual_fraction_std": 0.024126099949641287}, {"budget": 0.73, "metric": 0.6963410424577148, "metric_std": 0.013302827975529228, "actual_fraction": 0.793545046599931, "actual_fraction_std": 0.026074412644816893}, {"budget": 0.74, "metric": 0.6882982395581636, "metric_std": 0.009921432619005719, "actual_fraction": 0.8072833966171903, "actual_fraction_std": 0.020668824111692165}, {"budget": 0.75, "metric": 0.6826026924404557, "metric_std": 0.009033523143045721, "actual_fraction": 0.8174663444943044, "actual_fraction_std": 0.018802507342553624}, {"budget": 0.76, "metric": 0.6767345529858475, "metric_std": 0.00852717937662109, "actual_fraction": 0.8281325509147394, "actual_fraction_std": 0.016895282397133834}, {"budget": 0.77, "metric": 0.6715567828788401, "metric_std": 0.008358709419629788, "actual_fraction": 0.83569209527097, "actual_fraction_std": 0.01612376418641068}, {"budget": 0.78, "metric": 0.6661719019675527, "metric_std": 0.009292298329541888, "actual_fraction": 0.8440110459095618, "actual_fraction_std": 0.016948092935911756}, {"budget": 0.79, "metric": 0.6588194684156023, "metric_std": 0.009514049837537317, "actual_fraction": 0.8562996202968589, "actual_fraction_std": 0.016723476068915163}, {"budget": 0.8, "metric": 0.6538142906454953, "metric_std": 0.009376069766695005, "actual_fraction": 0.8655505695547117, "actual_fraction_std": 0.01443030894356102}, {"budget": 0.81, "metric": 0.649292371418709, "metric_std": 0.008566389011893934, "actual_fraction": 0.8726268553676217, "actual_fraction_std": 0.012690525695652865}, {"budget": 0.82, "metric": 0.6454608215395237, "metric_std": 0.00828209465354153, "actual_fraction": 0.8787021056265102, "actual_fraction_std": 0.01225612199203093}, {"budget": 0.83, "metric": 0.6404901622367968, "metric_std": 0.009541094208550392, "actual_fraction": 0.8861235761132207, "actual_fraction_std": 0.013941786342156063}, {"budget": 0.84, "metric": 0.6355195029340698, "metric_std": 0.009163986748247217, "actual_fraction": 0.8938211943389713, "actual_fraction_std": 0.012727090461391146}, {"budget": 0.85, "metric": 0.6280980324473594, "metric_std": 0.007456907791860313, "actual_fraction": 0.9037970314118053, "actual_fraction_std": 0.011578231852900304}, {"budget": 0.86, "metric": 0.622160856057991, "metric_std": 0.006201807298992233, "actual_fraction": 0.9118398343113565, "actual_fraction_std": 0.009726921922694132}, {"budget": 0.87, "metric": 0.6172247152226442, "metric_std": 0.006306591061986477, "actual_fraction": 0.9188125647221265, "actual_fraction_std": 0.010050892178028277}, {"budget": 0.88, "metric": 0.6122540559199171, "metric_std": 0.005461395362530534, "actual_fraction": 0.9260269244045565, "actual_fraction_std": 0.008495330742972878}, {"budget": 0.89, "metric": 0.6073869520193304, "metric_std": 0.004725690386926347, "actual_fraction": 0.9324128408698652, "actual_fraction_std": 0.006673991908571178}, {"budget": 0.9, "metric": 0.6003451846738005, "metric_std": 0.004861163588738741, "actual_fraction": 0.9416983085950985, "actual_fraction_std": 0.006098886693214954}, {"budget": 0.91, "metric": 0.5926475664480497, "metric_std": 0.004068569204313806, "actual_fraction": 0.9508802209181912, "actual_fraction_std": 0.005483169177029308}, {"budget": 0.92, "metric": 0.5875388332758026, "metric_std": 0.0015837169136285211, "actual_fraction": 0.9570245081118397, "actual_fraction_std": 0.0021797257404755394}, {"budget": 0.93, "metric": 0.582809803244736, "metric_std": 0.00201792846435127, "actual_fraction": 0.9626855367621678, "actual_fraction_std": 0.00251120367433436}, {"budget": 0.94, "metric": 0.5770797376596478, "metric_std": 0.0014675424003784351, "actual_fraction": 0.9693821194338972, "actual_fraction_std": 0.0016309747538732894}, {"budget": 0.95, "metric": 0.572868484639282, "metric_std": 0.0017059597919435992, "actual_fraction": 0.9741456679323438, "actual_fraction_std": 0.001919580630086153}, {"budget": 0.96, "metric": 0.5667932343803935, "metric_std": 0.0011705781814161675, "actual_fraction": 0.9806351397997929, "actual_fraction_std": 0.0010874686918897522}, {"budget": 0.97, "metric": 0.5616154642733863, "metric_std": 0.0011957547860330625, "actual_fraction": 0.9859855022437003, "actual_fraction_std": 0.0011278087160997592}, {"budget": 0.98, "metric": 0.5567138419054194, "metric_std": 0.0010254367122894542, "actual_fraction": 0.9910597169485676, "actual_fraction_std": 0.0009484730178829254}, {"budget": 0.99, "metric": 0.5525371073524336, "metric_std": 0.001351299905492833, "actual_fraction": 0.9954780807732136, "actual_fraction_std": 0.0013882690750563287}, {"budget": 1.0, "metric": 0.5481532619951675, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 659, "rescuable": 2, "harmful": 524, "both_wrong": 15, "unit": "sample", "total": 1200}, "test": {"both_correct": 2811, "rescuable": 365, "harmful": 1907, "both_wrong": 711, "unit": "sample", "total": 5794}}, "small": {"validation": 0.9858333333333333, "test": 0.81429064549534}, "large": {"validation": 0.5508333333333333, "test": 0.5481532619951675}, "learned_minus_confidence_test": 1.1102230246251565e-16, "learned_minus_small_test": 1.1102230246251565e-16, "learned_minus_large_test": 0.2661373835001727, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_cub_Qwen3.8"}, {"key": ["cub", "glsim", "MiniCPM"], "expert_display": "GLSim", "run": "outputs/full_budget_20260913/runs/full_glsim_MiniCPM_s{2026..2030}", "validation_rows": 1200, "test_rows": 5794, "confidence_rule": {"column": "max_probability", "columns": ["entropy", "max_probability", "top_two_margin", "entropy_valid", "max_probability_valid", "top_two_margin_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/glsim/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 0, "validation_area_0_25": 0.9005, "validation_point": {"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9891666666666666, "rescued": 0, "harmed": 0, "missed_rescue": 3, "avoided_harm": 857, "selected_both_correct": 0, "selected_both_wrong": 0}, "test_point": {"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9016223679668622, "rescued": 0, "harmed": 0, "missed_rescue": 106, "avoided_harm": 3791, "selected_both_correct": 0, "selected_both_wrong": 0}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9891666666666666, "rescued": 0, "harmed": 0, "missed_rescue": 3, "avoided_harm": 857, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.48843976855278015, "operator": "ge", "actual_fraction": 0.01, "metric": 0.985, "rescued": 1, "harmed": 6, "missed_rescue": 2, "avoided_harm": 851, "selected_both_correct": 0, "selected_both_wrong": 5}, {"budget": 0.02, "threshold": -0.6014261841773987, "operator": "ge", "actual_fraction": 0.02, "metric": 0.9791666666666666, "rescued": 2, "harmed": 14, "missed_rescue": 1, "avoided_harm": 843, "selected_both_correct": 2, "selected_both_wrong": 6}, {"budget": 0.03, "threshold": -0.6401472687721252, "operator": "ge", "actual_fraction": 0.03, "metric": 0.9716666666666667, "rescued": 2, "harmed": 23, "missed_rescue": 1, "avoided_harm": 834, "selected_both_correct": 3, "selected_both_wrong": 8}, {"budget": 0.04, "threshold": -0.6884650588035583, "operator": "ge", "actual_fraction": 0.04, "metric": 0.9658333333333333, "rescued": 3, "harmed": 31, "missed_rescue": 0, "avoided_harm": 826, "selected_both_correct": 5, "selected_both_wrong": 9}, {"budget": 0.05, "threshold": -0.7208121418952942, "operator": "ge", "actual_fraction": 0.05, "metric": 0.9591666666666666, "rescued": 3, "harmed": 39, "missed_rescue": 0, "avoided_harm": 818, "selected_both_correct": 9, "selected_both_wrong": 9}, {"budget": 0.06, "threshold": -0.7578181624412537, "operator": "ge", "actual_fraction": 0.06, "metric": 0.9516666666666667, "rescued": 3, "harmed": 48, "missed_rescue": 0, "avoided_harm": 809, "selected_both_correct": 12, "selected_both_wrong": 9}, {"budget": 0.07, "threshold": -0.7790470719337463, "operator": "ge", "actual_fraction": 0.07, "metric": 0.9441666666666667, "rescued": 3, "harmed": 57, "missed_rescue": 0, "avoided_harm": 800, "selected_both_correct": 15, "selected_both_wrong": 9}, {"budget": 0.08, "threshold": -0.7927228808403015, "operator": "ge", "actual_fraction": 0.08, "metric": 0.9375, "rescued": 3, "harmed": 65, "missed_rescue": 0, "avoided_harm": 792, "selected_both_correct": 18, "selected_both_wrong": 10}, {"budget": 0.09, "threshold": -0.8049252033233643, "operator": "ge", "actual_fraction": 0.09, "metric": 0.93, "rescued": 3, "harmed": 74, "missed_rescue": 0, "avoided_harm": 783, "selected_both_correct": 21, "selected_both_wrong": 10}, {"budget": 0.1, "threshold": -0.80850750207901, "operator": "ge", "actual_fraction": 0.1, "metric": 0.9216666666666666, "rescued": 3, "harmed": 84, "missed_rescue": 0, "avoided_harm": 773, "selected_both_correct": 23, "selected_both_wrong": 10}, {"budget": 0.11, "threshold": -0.819871723651886, "operator": "ge", "actual_fraction": 0.11, "metric": 0.9133333333333333, "rescued": 3, "harmed": 94, "missed_rescue": 0, "avoided_harm": 763, "selected_both_correct": 25, "selected_both_wrong": 10}, {"budget": 0.12, "threshold": -0.8310170769691467, "operator": "ge", "actual_fraction": 0.12, "metric": 0.905, "rescued": 3, "harmed": 104, "missed_rescue": 0, "avoided_harm": 753, "selected_both_correct": 27, "selected_both_wrong": 10}, {"budget": 0.13, "threshold": -0.8376906514167786, "operator": "ge", "actual_fraction": 0.13, "metric": 0.8966666666666666, "rescued": 3, "harmed": 114, "missed_rescue": 0, "avoided_harm": 743, "selected_both_correct": 29, "selected_both_wrong": 10}, {"budget": 0.14, "threshold": -0.8419997692108154, "operator": "ge", "actual_fraction": 0.14, "metric": 0.8875, "rescued": 3, "harmed": 125, "missed_rescue": 0, "avoided_harm": 732, "selected_both_correct": 30, "selected_both_wrong": 10}, {"budget": 0.15, "threshold": -0.8447275161743164, "operator": "ge", "actual_fraction": 0.15, "metric": 0.8808333333333334, "rescued": 3, "harmed": 133, "missed_rescue": 0, "avoided_harm": 724, "selected_both_correct": 34, "selected_both_wrong": 10}, {"budget": 0.16, "threshold": -0.8506776690483093, "operator": "ge", "actual_fraction": 0.16, "metric": 0.8716666666666667, "rescued": 3, "harmed": 144, "missed_rescue": 0, "avoided_harm": 713, "selected_both_correct": 35, "selected_both_wrong": 10}, {"budget": 0.17, "threshold": -0.8545639514923096, "operator": "ge", "actual_fraction": 0.17, "metric": 0.8633333333333333, "rescued": 3, "harmed": 154, "missed_rescue": 0, "avoided_harm": 703, "selected_both_correct": 37, "selected_both_wrong": 10}, {"budget": 0.18, "threshold": -0.8578683137893677, "operator": "ge", "actual_fraction": 0.18, "metric": 0.855, "rescued": 3, "harmed": 164, "missed_rescue": 0, "avoided_harm": 693, "selected_both_correct": 39, "selected_both_wrong": 10}, {"budget": 0.19, "threshold": -0.86090087890625, "operator": "ge", "actual_fraction": 0.19, "metric": 0.8491666666666666, "rescued": 3, "harmed": 171, "missed_rescue": 0, "avoided_harm": 686, "selected_both_correct": 44, "selected_both_wrong": 10}, {"budget": 0.2, "threshold": -0.8640955090522766, "operator": "ge", "actual_fraction": 0.2, "metric": 0.8425, "rescued": 3, "harmed": 179, "missed_rescue": 0, "avoided_harm": 678, "selected_both_correct": 48, "selected_both_wrong": 10}, {"budget": 0.21, "threshold": -0.8664237260818481, "operator": "ge", "actual_fraction": 0.21, "metric": 0.8358333333333333, "rescued": 3, "harmed": 187, "missed_rescue": 0, "avoided_harm": 670, "selected_both_correct": 52, "selected_both_wrong": 10}, {"budget": 0.22, "threshold": -0.8686786890029907, "operator": "ge", "actual_fraction": 0.22, "metric": 0.8283333333333334, "rescued": 3, "harmed": 196, "missed_rescue": 0, "avoided_harm": 661, "selected_both_correct": 55, "selected_both_wrong": 10}, {"budget": 0.23, "threshold": -0.8709638118743896, "operator": "ge", "actual_fraction": 0.23, "metric": 0.8208333333333333, "rescued": 3, "harmed": 205, "missed_rescue": 0, "avoided_harm": 652, "selected_both_correct": 58, "selected_both_wrong": 10}, {"budget": 0.24, "threshold": -0.8724074959754944, "operator": "ge", "actual_fraction": 0.24, "metric": 0.8158333333333333, "rescued": 3, "harmed": 211, "missed_rescue": 0, "avoided_harm": 646, "selected_both_correct": 64, "selected_both_wrong": 10}, {"budget": 0.25, "threshold": -0.8736594319343567, "operator": "ge", "actual_fraction": 0.25, "metric": 0.8075, "rescued": 3, "harmed": 221, "missed_rescue": 0, "avoided_harm": 636, "selected_both_correct": 66, "selected_both_wrong": 10}, {"budget": 0.26, "threshold": -0.8752554059028625, "operator": "ge", "actual_fraction": 0.26, "metric": 0.8016666666666666, "rescued": 3, "harmed": 228, "missed_rescue": 0, "avoided_harm": 629, "selected_both_correct": 71, "selected_both_wrong": 10}, {"budget": 0.27, "threshold": -0.8766024112701416, "operator": "ge", "actual_fraction": 0.27, "metric": 0.7933333333333333, "rescued": 3, "harmed": 238, "missed_rescue": 0, "avoided_harm": 619, "selected_both_correct": 73, "selected_both_wrong": 10}, {"budget": 0.28, "threshold": -0.878311276435852, "operator": "ge", "actual_fraction": 0.28, "metric": 0.7866666666666666, "rescued": 3, "harmed": 246, "missed_rescue": 0, "avoided_harm": 611, "selected_both_correct": 77, "selected_both_wrong": 10}, {"budget": 0.29, "threshold": -0.880103588104248, "operator": "ge", "actual_fraction": 0.29, "metric": 0.78, "rescued": 3, "harmed": 254, "missed_rescue": 0, "avoided_harm": 603, "selected_both_correct": 81, "selected_both_wrong": 10}, {"budget": 0.3, "threshold": -0.8824952244758606, "operator": "ge", "actual_fraction": 0.3, "metric": 0.7725, "rescued": 3, "harmed": 263, "missed_rescue": 0, "avoided_harm": 594, "selected_both_correct": 84, "selected_both_wrong": 10}, {"budget": 0.31, "threshold": -0.8831692337989807, "operator": "ge", "actual_fraction": 0.31, "metric": 0.7658333333333334, "rescued": 3, "harmed": 271, "missed_rescue": 0, "avoided_harm": 586, "selected_both_correct": 88, "selected_both_wrong": 10}, {"budget": 0.32, "threshold": -0.8845130205154419, "operator": "ge", "actual_fraction": 0.32, "metric": 0.76, "rescued": 3, "harmed": 278, "missed_rescue": 0, "avoided_harm": 579, "selected_both_correct": 93, "selected_both_wrong": 10}, {"budget": 0.33, "threshold": -0.8855781555175781, "operator": "ge", "actual_fraction": 0.33, "metric": 0.7508333333333334, "rescued": 3, "harmed": 289, "missed_rescue": 0, "avoided_harm": 568, "selected_both_correct": 94, "selected_both_wrong": 10}, {"budget": 0.34, "threshold": -0.8865919709205627, "operator": "ge", "actual_fraction": 0.34, "metric": 0.7433333333333333, "rescued": 3, "harmed": 298, "missed_rescue": 0, "avoided_harm": 559, "selected_both_correct": 97, "selected_both_wrong": 10}, {"budget": 0.35, "threshold": -0.8877584338188171, "operator": "ge", "actual_fraction": 0.35, "metric": 0.735, "rescued": 3, "harmed": 308, "missed_rescue": 0, "avoided_harm": 549, "selected_both_correct": 99, "selected_both_wrong": 10}, {"budget": 0.36, "threshold": -0.8890330195426941, "operator": "ge", "actual_fraction": 0.36, "metric": 0.7291666666666666, "rescued": 3, "harmed": 315, "missed_rescue": 0, "avoided_harm": 542, "selected_both_correct": 104, "selected_both_wrong": 10}, {"budget": 0.37, "threshold": -0.890850305557251, "operator": "ge", "actual_fraction": 0.37, "metric": 0.7225, "rescued": 3, "harmed": 323, "missed_rescue": 0, "avoided_harm": 534, "selected_both_correct": 108, "selected_both_wrong": 10}, {"budget": 0.38, "threshold": -0.8916605710983276, "operator": "ge", "actual_fraction": 0.38, "metric": 0.7183333333333334, "rescued": 3, "harmed": 328, "missed_rescue": 0, "avoided_harm": 529, "selected_both_correct": 115, "selected_both_wrong": 10}, {"budget": 0.39, "threshold": -0.8928589224815369, "operator": "ge", "actual_fraction": 0.39, "metric": 0.7116666666666667, "rescued": 3, "harmed": 336, "missed_rescue": 0, "avoided_harm": 521, "selected_both_correct": 119, "selected_both_wrong": 10}, {"budget": 0.4, "threshold": -0.8940943479537964, "operator": "ge", "actual_fraction": 0.4, "metric": 0.7033333333333334, "rescued": 3, "harmed": 346, "missed_rescue": 0, "avoided_harm": 511, "selected_both_correct": 121, "selected_both_wrong": 10}, {"budget": 0.41, "threshold": -0.8949463963508606, "operator": "ge", "actual_fraction": 0.4091666666666667, "metric": 0.6983333333333334, "rescued": 3, "harmed": 352, "missed_rescue": 0, "avoided_harm": 505, "selected_both_correct": 126, "selected_both_wrong": 10}, {"budget": 0.42, "threshold": -0.8960229158401489, "operator": "ge", "actual_fraction": 0.42, "metric": 0.6916666666666667, "rescued": 3, "harmed": 360, "missed_rescue": 0, "avoided_harm": 497, "selected_both_correct": 131, "selected_both_wrong": 10}, {"budget": 0.43, "threshold": -0.8966536521911621, "operator": "ge", "actual_fraction": 0.43, "metric": 0.6841666666666667, "rescued": 3, "harmed": 369, "missed_rescue": 0, "avoided_harm": 488, "selected_both_correct": 134, "selected_both_wrong": 10}, {"budget": 0.44, "threshold": -0.8975363373756409, "operator": "ge", "actual_fraction": 0.44, "metric": 0.68, "rescued": 3, "harmed": 374, "missed_rescue": 0, "avoided_harm": 483, "selected_both_correct": 141, "selected_both_wrong": 10}, {"budget": 0.45, "threshold": -0.8988122940063477, "operator": "ge", "actual_fraction": 0.45, "metric": 0.6741666666666667, "rescued": 3, "harmed": 381, "missed_rescue": 0, "avoided_harm": 476, "selected_both_correct": 146, "selected_both_wrong": 10}, {"budget": 0.46, "threshold": -0.9000959396362305, "operator": "ge", "actual_fraction": 0.46, "metric": 0.665, "rescued": 3, "harmed": 392, "missed_rescue": 0, "avoided_harm": 465, "selected_both_correct": 147, "selected_both_wrong": 10}, {"budget": 0.47, "threshold": -0.9013051986694336, "operator": "ge", "actual_fraction": 0.47, "metric": 0.6583333333333333, "rescued": 3, "harmed": 400, "missed_rescue": 0, "avoided_harm": 457, "selected_both_correct": 151, "selected_both_wrong": 10}, {"budget": 0.48, "threshold": -0.9029324054718018, "operator": "ge", "actual_fraction": 0.48, "metric": 0.6525, "rescued": 3, "harmed": 407, "missed_rescue": 0, "avoided_harm": 450, "selected_both_correct": 156, "selected_both_wrong": 10}, {"budget": 0.49, "threshold": -0.9036871790885925, "operator": "ge", "actual_fraction": 0.49, "metric": 0.6475, "rescued": 3, "harmed": 413, "missed_rescue": 0, "avoided_harm": 444, "selected_both_correct": 162, "selected_both_wrong": 10}, {"budget": 0.5, "threshold": -0.9044621586799622, "operator": "ge", "actual_fraction": 0.5, "metric": 0.6425, "rescued": 3, "harmed": 419, "missed_rescue": 0, "avoided_harm": 438, "selected_both_correct": 168, "selected_both_wrong": 10}, {"budget": 0.51, "threshold": -0.9054575562477112, "operator": "ge", "actual_fraction": 0.51, "metric": 0.6358333333333334, "rescued": 3, "harmed": 427, "missed_rescue": 0, "avoided_harm": 430, "selected_both_correct": 172, "selected_both_wrong": 10}, {"budget": 0.52, "threshold": -0.906682550907135, "operator": "ge", "actual_fraction": 0.52, "metric": 0.6266666666666667, "rescued": 3, "harmed": 438, "missed_rescue": 0, "avoided_harm": 419, "selected_both_correct": 173, "selected_both_wrong": 10}, {"budget": 0.53, "threshold": -0.9072114825248718, "operator": "ge", "actual_fraction": 0.53, "metric": 0.6225, "rescued": 3, "harmed": 443, "missed_rescue": 0, "avoided_harm": 414, "selected_both_correct": 180, "selected_both_wrong": 10}, {"budget": 0.54, "threshold": -0.9083563089370728, "operator": "ge", "actual_fraction": 0.54, "metric": 0.6158333333333333, "rescued": 3, "harmed": 451, "missed_rescue": 0, "avoided_harm": 406, "selected_both_correct": 184, "selected_both_wrong": 10}, {"budget": 0.55, "threshold": -0.9095192551612854, "operator": "ge", "actual_fraction": 0.55, "metric": 0.6091666666666666, "rescued": 3, "harmed": 459, "missed_rescue": 0, "avoided_harm": 398, "selected_both_correct": 188, "selected_both_wrong": 10}, {"budget": 0.56, "threshold": -0.9106979370117188, "operator": "ge", "actual_fraction": 0.56, "metric": 0.6016666666666667, "rescued": 3, "harmed": 468, "missed_rescue": 0, "avoided_harm": 389, "selected_both_correct": 191, "selected_both_wrong": 10}, {"budget": 0.57, "threshold": -0.9115362167358398, "operator": "ge", "actual_fraction": 0.5691666666666667, "metric": 0.595, "rescued": 3, "harmed": 476, "missed_rescue": 0, "avoided_harm": 381, "selected_both_correct": 194, "selected_both_wrong": 10}, {"budget": 0.58, "threshold": -0.9124338030815125, "operator": "ge", "actual_fraction": 0.58, "metric": 0.5866666666666667, "rescued": 3, "harmed": 486, "missed_rescue": 0, "avoided_harm": 371, "selected_both_correct": 197, "selected_both_wrong": 10}, {"budget": 0.59, "threshold": -0.9134801626205444, "operator": "ge", "actual_fraction": 0.59, "metric": 0.5783333333333334, "rescued": 3, "harmed": 496, "missed_rescue": 0, "avoided_harm": 361, "selected_both_correct": 199, "selected_both_wrong": 10}, {"budget": 0.6, "threshold": -0.9143747091293335, "operator": "ge", "actual_fraction": 0.6, "metric": 0.5733333333333334, "rescued": 3, "harmed": 502, "missed_rescue": 0, "avoided_harm": 355, "selected_both_correct": 205, "selected_both_wrong": 10}, {"budget": 0.61, "threshold": -0.915804386138916, "operator": "ge", "actual_fraction": 0.61, "metric": 0.5641666666666667, "rescued": 3, "harmed": 513, "missed_rescue": 0, "avoided_harm": 344, "selected_both_correct": 206, "selected_both_wrong": 10}, {"budget": 0.62, "threshold": -0.9165146350860596, "operator": "ge", "actual_fraction": 0.62, "metric": 0.555, "rescued": 3, "harmed": 524, "missed_rescue": 0, "avoided_harm": 333, "selected_both_correct": 207, "selected_both_wrong": 10}, {"budget": 0.63, "threshold": -0.9172822833061218, "operator": "ge", "actual_fraction": 0.63, "metric": 0.55, "rescued": 3, "harmed": 530, "missed_rescue": 0, "avoided_harm": 327, "selected_both_correct": 213, "selected_both_wrong": 10}, {"budget": 0.64, "threshold": -0.9181071519851685, "operator": "ge", "actual_fraction": 0.64, "metric": 0.5416666666666666, "rescued": 3, "harmed": 540, "missed_rescue": 0, "avoided_harm": 317, "selected_both_correct": 215, "selected_both_wrong": 10}, {"budget": 0.65, "threshold": -0.9192029237747192, "operator": "ge", "actual_fraction": 0.65, "metric": 0.5383333333333333, "rescued": 3, "harmed": 544, "missed_rescue": 0, "avoided_harm": 313, "selected_both_correct": 223, "selected_both_wrong": 10}, {"budget": 0.66, "threshold": -0.9200543761253357, "operator": "ge", "actual_fraction": 0.66, "metric": 0.5333333333333333, "rescued": 3, "harmed": 550, "missed_rescue": 0, "avoided_harm": 307, "selected_both_correct": 229, "selected_both_wrong": 10}, {"budget": 0.67, "threshold": -0.9208071231842041, "operator": "ge", "actual_fraction": 0.67, "metric": 0.5266666666666666, "rescued": 3, "harmed": 558, "missed_rescue": 0, "avoided_harm": 299, "selected_both_correct": 233, "selected_both_wrong": 10}, {"budget": 0.68, "threshold": -0.9215553998947144, "operator": "ge", "actual_fraction": 0.68, "metric": 0.5175, "rescued": 3, "harmed": 569, "missed_rescue": 0, "avoided_harm": 288, "selected_both_correct": 234, "selected_both_wrong": 10}, {"budget": 0.69, "threshold": -0.9220835566520691, "operator": "ge", "actual_fraction": 0.6891666666666667, "metric": 0.5116666666666667, "rescued": 3, "harmed": 576, "missed_rescue": 0, "avoided_harm": 281, "selected_both_correct": 238, "selected_both_wrong": 10}, {"budget": 0.7, "threshold": -0.9232237935066223, "operator": "ge", "actual_fraction": 0.7, "metric": 0.5041666666666667, "rescued": 3, "harmed": 585, "missed_rescue": 0, "avoided_harm": 272, "selected_both_correct": 242, "selected_both_wrong": 10}, {"budget": 0.71, "threshold": -0.9243364334106445, "operator": "ge", "actual_fraction": 0.71, "metric": 0.49833333333333335, "rescued": 3, "harmed": 592, "missed_rescue": 0, "avoided_harm": 265, "selected_both_correct": 247, "selected_both_wrong": 10}, {"budget": 0.72, "threshold": -0.9252541065216064, "operator": "ge", "actual_fraction": 0.72, "metric": 0.49083333333333334, "rescued": 3, "harmed": 601, "missed_rescue": 0, "avoided_harm": 256, "selected_both_correct": 250, "selected_both_wrong": 10}, {"budget": 0.73, "threshold": -0.925781786441803, "operator": "ge", "actual_fraction": 0.73, "metric": 0.485, "rescued": 3, "harmed": 608, "missed_rescue": 0, "avoided_harm": 249, "selected_both_correct": 255, "selected_both_wrong": 10}, {"budget": 0.74, "threshold": -0.9274963140487671, "operator": "ge", "actual_fraction": 0.74, "metric": 0.47833333333333333, "rescued": 3, "harmed": 616, "missed_rescue": 0, "avoided_harm": 241, "selected_both_correct": 259, "selected_both_wrong": 10}, {"budget": 0.75, "threshold": -0.9286423325538635, "operator": "ge", "actual_fraction": 0.75, "metric": 0.4725, "rescued": 3, "harmed": 623, "missed_rescue": 0, "avoided_harm": 234, "selected_both_correct": 264, "selected_both_wrong": 10}, {"budget": 0.76, "threshold": -0.9301291704177856, "operator": "ge", "actual_fraction": 0.76, "metric": 0.4666666666666667, "rescued": 3, "harmed": 630, "missed_rescue": 0, "avoided_harm": 227, "selected_both_correct": 269, "selected_both_wrong": 10}, {"budget": 0.77, "threshold": -0.9318777918815613, "operator": "ge", "actual_fraction": 0.77, "metric": 0.4583333333333333, "rescued": 3, "harmed": 640, "missed_rescue": 0, "avoided_harm": 217, "selected_both_correct": 271, "selected_both_wrong": 10}, {"budget": 0.78, "threshold": -0.9332055449485779, "operator": "ge", "actual_fraction": 0.78, "metric": 0.45, "rescued": 3, "harmed": 650, "missed_rescue": 0, "avoided_harm": 207, "selected_both_correct": 273, "selected_both_wrong": 10}, {"budget": 0.79, "threshold": -0.9344666600227356, "operator": "ge", "actual_fraction": 0.79, "metric": 0.4425, "rescued": 3, "harmed": 659, "missed_rescue": 0, "avoided_harm": 198, "selected_both_correct": 276, "selected_both_wrong": 10}, {"budget": 0.8, "threshold": -0.9362465143203735, "operator": "ge", "actual_fraction": 0.8, "metric": 0.43583333333333335, "rescued": 3, "harmed": 667, "missed_rescue": 0, "avoided_harm": 190, "selected_both_correct": 280, "selected_both_wrong": 10}, {"budget": 0.81, "threshold": -0.937842071056366, "operator": "ge", "actual_fraction": 0.81, "metric": 0.4275, "rescued": 3, "harmed": 677, "missed_rescue": 0, "avoided_harm": 180, "selected_both_correct": 282, "selected_both_wrong": 10}, {"budget": 0.82, "threshold": -0.9384457468986511, "operator": "ge", "actual_fraction": 0.8191666666666667, "metric": 0.4216666666666667, "rescued": 3, "harmed": 684, "missed_rescue": 0, "avoided_harm": 173, "selected_both_correct": 286, "selected_both_wrong": 10}, {"budget": 0.83, "threshold": -0.9396296143531799, "operator": "ge", "actual_fraction": 0.83, "metric": 0.41333333333333333, "rescued": 3, "harmed": 694, "missed_rescue": 0, "avoided_harm": 163, "selected_both_correct": 289, "selected_both_wrong": 10}, {"budget": 0.84, "threshold": -0.9407488703727722, "operator": "ge", "actual_fraction": 0.84, "metric": 0.4058333333333333, "rescued": 3, "harmed": 703, "missed_rescue": 0, "avoided_harm": 154, "selected_both_correct": 292, "selected_both_wrong": 10}, {"budget": 0.85, "threshold": -0.942738950252533, "operator": "ge", "actual_fraction": 0.85, "metric": 0.4, "rescued": 3, "harmed": 710, "missed_rescue": 0, "avoided_harm": 147, "selected_both_correct": 297, "selected_both_wrong": 10}, {"budget": 0.86, "threshold": -0.9438527226448059, "operator": "gt", "actual_fraction": 0.8591666666666666, "metric": 0.39416666666666667, "rescued": 3, "harmed": 717, "missed_rescue": 0, "avoided_harm": 140, "selected_both_correct": 301, "selected_both_wrong": 10}, {"budget": 0.87, "threshold": -0.9450756907463074, "operator": "ge", "actual_fraction": 0.87, "metric": 0.38916666666666666, "rescued": 3, "harmed": 723, "missed_rescue": 0, "avoided_harm": 134, "selected_both_correct": 308, "selected_both_wrong": 10}, {"budget": 0.88, "threshold": -0.9481598138809204, "operator": "ge", "actual_fraction": 0.88, "metric": 0.38083333333333336, "rescued": 3, "harmed": 733, "missed_rescue": 0, "avoided_harm": 124, "selected_both_correct": 310, "selected_both_wrong": 10}, {"budget": 0.89, "threshold": -0.9500836730003357, "operator": "ge", "actual_fraction": 0.89, "metric": 0.37333333333333335, "rescued": 3, "harmed": 742, "missed_rescue": 0, "avoided_harm": 115, "selected_both_correct": 313, "selected_both_wrong": 10}, {"budget": 0.9, "threshold": -0.9519118666648865, "operator": "ge", "actual_fraction": 0.9, "metric": 0.365, "rescued": 3, "harmed": 752, "missed_rescue": 0, "avoided_harm": 105, "selected_both_correct": 315, "selected_both_wrong": 10}, {"budget": 0.91, "threshold": -0.9543898701667786, "operator": "ge", "actual_fraction": 0.91, "metric": 0.3566666666666667, "rescued": 3, "harmed": 762, "missed_rescue": 0, "avoided_harm": 95, "selected_both_correct": 317, "selected_both_wrong": 10}, {"budget": 0.92, "threshold": -0.9570822715759277, "operator": "ge", "actual_fraction": 0.92, "metric": 0.34833333333333333, "rescued": 3, "harmed": 772, "missed_rescue": 0, "avoided_harm": 85, "selected_both_correct": 319, "selected_both_wrong": 10}, {"budget": 0.93, "threshold": -0.9593446254730225, "operator": "ge", "actual_fraction": 0.93, "metric": 0.3383333333333333, "rescued": 3, "harmed": 784, "missed_rescue": 0, "avoided_harm": 73, "selected_both_correct": 319, "selected_both_wrong": 10}, {"budget": 0.94, "threshold": -0.9610995054244995, "operator": "ge", "actual_fraction": 0.94, "metric": 0.33, "rescued": 3, "harmed": 794, "missed_rescue": 0, "avoided_harm": 63, "selected_both_correct": 321, "selected_both_wrong": 10}, {"budget": 0.95, "threshold": -0.9635111093521118, "operator": "ge", "actual_fraction": 0.95, "metric": 0.32, "rescued": 3, "harmed": 806, "missed_rescue": 0, "avoided_harm": 51, "selected_both_correct": 321, "selected_both_wrong": 10}, {"budget": 0.96, "threshold": -0.9648061990737915, "operator": "ge", "actual_fraction": 0.96, "metric": 0.31333333333333335, "rescued": 3, "harmed": 814, "missed_rescue": 0, "avoided_harm": 43, "selected_both_correct": 325, "selected_both_wrong": 10}, {"budget": 0.97, "threshold": -0.9674390554428101, "operator": "ge", "actual_fraction": 0.97, "metric": 0.30416666666666664, "rescued": 3, "harmed": 825, "missed_rescue": 0, "avoided_harm": 32, "selected_both_correct": 326, "selected_both_wrong": 10}, {"budget": 0.98, "threshold": -0.9724470973014832, "operator": "ge", "actual_fraction": 0.98, "metric": 0.29583333333333334, "rescued": 3, "harmed": 835, "missed_rescue": 0, "avoided_harm": 22, "selected_both_correct": 328, "selected_both_wrong": 10}, {"budget": 0.99, "threshold": -0.9774226546287537, "operator": "ge", "actual_fraction": 0.99, "metric": 0.2866666666666667, "rescued": 3, "harmed": 846, "missed_rescue": 0, "avoided_harm": 11, "selected_both_correct": 329, "selected_both_wrong": 10}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.2775, "rescued": 3, "harmed": 857, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 330, "selected_both_wrong": 10}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9016223679668622, "rescued": 0, "harmed": 0, "missed_rescue": 106, "avoided_harm": 3791, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.48843976855278015, "operator": "ge", "actual_fraction": 0.08698653779772178, "metric": 0.8769416637901277, "rescued": 50, "harmed": 193, "missed_rescue": 56, "avoided_harm": 3598, "selected_both_correct": 42, "selected_both_wrong": 219}, {"budget": 0.02, "threshold": -0.6014261841773987, "operator": "ge", "actual_fraction": 0.13910942354159475, "metric": 0.8493268898860891, "rescued": 66, "harmed": 369, "missed_rescue": 40, "avoided_harm": 3422, "selected_both_correct": 85, "selected_both_wrong": 286}, {"budget": 0.03, "threshold": -0.6401472687721252, "operator": "ge", "actual_fraction": 0.15947531929582326, "metric": 0.8381083879875734, "rescued": 74, "harmed": 442, "missed_rescue": 32, "avoided_harm": 3349, "selected_both_correct": 104, "selected_both_wrong": 304}, {"budget": 0.04, "threshold": -0.6884650588035583, "operator": "ge", "actual_fraction": 0.1886434242319641, "metric": 0.8208491542975492, "rescued": 81, "harmed": 549, "missed_rescue": 25, "avoided_harm": 3242, "selected_both_correct": 139, "selected_both_wrong": 324}, {"budget": 0.05, "threshold": -0.7208121418952942, "operator": "ge", "actual_fraction": 0.2143596824301001, "metric": 0.8041076976182258, "rescued": 84, "harmed": 649, "missed_rescue": 22, "avoided_harm": 3142, "selected_both_correct": 165, "selected_both_wrong": 344}, {"budget": 0.06, "threshold": -0.7578181624412537, "operator": "ge", "actual_fraction": 0.24525371073524335, "metric": 0.7825336555056955, "rescued": 88, "harmed": 778, "missed_rescue": 18, "avoided_harm": 3013, "selected_both_correct": 198, "selected_both_wrong": 357}, {"budget": 0.07, "threshold": -0.7790470719337463, "operator": "ge", "actual_fraction": 0.26734552985847426, "metric": 0.767172937521574, "rescued": 90, "harmed": 869, "missed_rescue": 16, "avoided_harm": 2922, "selected_both_correct": 221, "selected_both_wrong": 369}, {"budget": 0.08, "threshold": -0.7927228808403015, "operator": "ge", "actual_fraction": 0.2827062478425958, "metric": 0.75716258198136, "rescued": 92, "harmed": 929, "missed_rescue": 14, "avoided_harm": 2862, "selected_both_correct": 244, "selected_both_wrong": 373}, {"budget": 0.09, "threshold": -0.8049252033233643, "operator": "ge", "actual_fraction": 0.2999654815326199, "metric": 0.7449085260614429, "rescued": 93, "harmed": 1001, "missed_rescue": 13, "avoided_harm": 2790, "selected_both_correct": 264, "selected_both_wrong": 380}, {"budget": 0.1, "threshold": -0.80850750207901, "operator": "ge", "actual_fraction": 0.30531584397652745, "metric": 0.7416292716603383, "rescued": 93, "harmed": 1020, "missed_rescue": 13, "avoided_harm": 2771, "selected_both_correct": 274, "selected_both_wrong": 382}, {"budget": 0.11, "threshold": -0.819871723651886, "operator": "ge", "actual_fraction": 0.3258543320676562, "metric": 0.7279944770452192, "rescued": 96, "harmed": 1102, "missed_rescue": 10, "avoided_harm": 2689, "selected_both_correct": 302, "selected_both_wrong": 388}, {"budget": 0.12, "threshold": -0.8310170769691467, "operator": "ge", "actual_fraction": 0.35053503624439075, "metric": 0.710735243355195, "rescued": 99, "harmed": 1205, "missed_rescue": 7, "avoided_harm": 2586, "selected_both_correct": 332, "selected_both_wrong": 395}, {"budget": 0.13, "threshold": -0.8376906514167786, "operator": "ge", "actual_fraction": 0.36796686227131514, "metric": 0.6976182257507767, "rescued": 99, "harmed": 1281, "missed_rescue": 7, "avoided_harm": 2510, "selected_both_correct": 353, "selected_both_wrong": 399}, {"budget": 0.14, "threshold": -0.8419997692108154, "operator": "ge", "actual_fraction": 0.3798757335174318, "metric": 0.6896789782533655, "rescued": 99, "harmed": 1327, "missed_rescue": 7, "avoided_harm": 2464, "selected_both_correct": 374, "selected_both_wrong": 401}, {"budget": 0.15, "threshold": -0.8447275161743164, "operator": "ge", "actual_fraction": 0.3867794269934415, "metric": 0.6855367621677597, "rescued": 99, "harmed": 1351, "missed_rescue": 7, "avoided_harm": 2440, "selected_both_correct": 390, "selected_both_wrong": 401}, {"budget": 0.16, "threshold": -0.8506776690483093, "operator": "ge", "actual_fraction": 0.4050742147048671, "metric": 0.6717293752157404, "rescued": 100, "harmed": 1432, "missed_rescue": 6, "avoided_harm": 2359, "selected_both_correct": 411, "selected_both_wrong": 404}, {"budget": 0.17, "threshold": -0.8545639514923096, "operator": "ge", "actual_fraction": 0.4175008629616845, "metric": 0.6632723507076286, "rescued": 100, "harmed": 1481, "missed_rescue": 6, "avoided_harm": 2310, "selected_both_correct": 430, "selected_both_wrong": 408}, {"budget": 0.18, "threshold": -0.8578683137893677, "operator": "ge", "actual_fraction": 0.4290645495340007, "metric": 0.6544701415257163, "rescued": 101, "harmed": 1533, "missed_rescue": 5, "avoided_harm": 2258, "selected_both_correct": 443, "selected_both_wrong": 409}, {"budget": 0.19, "threshold": -0.86090087890625, "operator": "ge", "actual_fraction": 0.43993786675871593, "metric": 0.6473938557128064, "rescued": 101, "harmed": 1574, "missed_rescue": 5, "avoided_harm": 2217, "selected_both_correct": 462, "selected_both_wrong": 412}, {"budget": 0.2, "threshold": -0.8640955090522766, "operator": "ge", "actual_fraction": 0.4520193303417328, "metric": 0.6389368312046945, "rescued": 101, "harmed": 1623, "missed_rescue": 5, "avoided_harm": 2168, "selected_both_correct": 482, "selected_both_wrong": 413}, {"budget": 0.21, "threshold": -0.8664237260818481, "operator": "ge", "actual_fraction": 0.46047635484984467, "metric": 0.6332412840869865, "rescued": 101, "harmed": 1656, "missed_rescue": 5, "avoided_harm": 2135, "selected_both_correct": 496, "selected_both_wrong": 415}, {"budget": 0.22, "threshold": -0.8686786890029907, "operator": "ge", "actual_fraction": 0.46945115636865725, "metric": 0.6272005522954781, "rescued": 101, "harmed": 1691, "missed_rescue": 5, "avoided_harm": 2100, "selected_both_correct": 513, "selected_both_wrong": 415}, {"budget": 0.23, "threshold": -0.8709638118743896, "operator": "ge", "actual_fraction": 0.48066965826717295, "metric": 0.6197790818087677, "rescued": 101, "harmed": 1734, "missed_rescue": 5, "avoided_harm": 2057, "selected_both_correct": 534, "selected_both_wrong": 416}, {"budget": 0.24, "threshold": -0.8724074959754944, "operator": "ge", "actual_fraction": 0.48912668277528476, "metric": 0.61425612702796, "rescued": 101, "harmed": 1766, "missed_rescue": 5, "avoided_harm": 2025, "selected_both_correct": 550, "selected_both_wrong": 417}, {"budget": 0.25, "threshold": -0.8736594319343567, "operator": "ge", "actual_fraction": 0.49413186054539177, "metric": 0.6113220573006558, "rescued": 101, "harmed": 1783, "missed_rescue": 5, "avoided_harm": 2008, "selected_both_correct": 561, "selected_both_wrong": 418}, {"budget": 0.26, "threshold": -0.8752554059028625, "operator": "ge", "actual_fraction": 0.5005177770107008, "metric": 0.6071798412150501, "rescued": 101, "harmed": 1807, "missed_rescue": 5, "avoided_harm": 1984, "selected_both_correct": 574, "selected_both_wrong": 418}, {"budget": 0.27, "threshold": -0.8766024112701416, "operator": "ge", "actual_fraction": 0.5081118398343114, "metric": 0.6020020711080428, "rescued": 101, "harmed": 1837, "missed_rescue": 5, "avoided_harm": 1954, "selected_both_correct": 587, "selected_both_wrong": 419}, {"budget": 0.28, "threshold": -0.878311276435852, "operator": "ge", "actual_fraction": 0.5169140490162237, "metric": 0.596479116327235, "rescued": 102, "harmed": 1870, "missed_rescue": 4, "avoided_harm": 1921, "selected_both_correct": 603, "selected_both_wrong": 420}, {"budget": 0.29, "threshold": -0.880103588104248, "operator": "ge", "actual_fraction": 0.5277873662409389, "metric": 0.5894028305143252, "rescued": 102, "harmed": 1911, "missed_rescue": 4, "avoided_harm": 1880, "selected_both_correct": 624, "selected_both_wrong": 421}, {"budget": 0.3, "threshold": -0.8824952244758606, "operator": "ge", "actual_fraction": 0.5419399378667588, "metric": 0.5788746979634104, "rescued": 102, "harmed": 1972, "missed_rescue": 4, "avoided_harm": 1819, "selected_both_correct": 645, "selected_both_wrong": 421}, {"budget": 0.31, "threshold": -0.8831692337989807, "operator": "ge", "actual_fraction": 0.5453917846047636, "metric": 0.5766309975837073, "rescued": 102, "harmed": 1985, "missed_rescue": 4, "avoided_harm": 1806, "selected_both_correct": 651, "selected_both_wrong": 422}, {"budget": 0.32, "threshold": -0.8845130205154419, "operator": "ge", "actual_fraction": 0.5566102865032793, "metric": 0.5704176734552986, "rescued": 102, "harmed": 2021, "missed_rescue": 4, "avoided_harm": 1770, "selected_both_correct": 677, "selected_both_wrong": 425}, {"budget": 0.33, "threshold": -0.8855781555175781, "operator": "ge", "actual_fraction": 0.5633413876423887, "metric": 0.5661028650327925, "rescued": 102, "harmed": 2046, "missed_rescue": 4, "avoided_harm": 1745, "selected_both_correct": 691, "selected_both_wrong": 425}, {"budget": 0.34, "threshold": -0.8865919709205627, "operator": "ge", "actual_fraction": 0.5716258198136003, "metric": 0.5607525025888851, "rescued": 102, "harmed": 2077, "missed_rescue": 4, "avoided_harm": 1714, "selected_both_correct": 707, "selected_both_wrong": 426}, {"budget": 0.35, "threshold": -0.8877584338188171, "operator": "ge", "actual_fraction": 0.5819813600276148, "metric": 0.5545391784604764, "rescued": 103, "harmed": 2114, "missed_rescue": 3, "avoided_harm": 1677, "selected_both_correct": 728, "selected_both_wrong": 427}, {"budget": 0.36, "threshold": -0.8890330195426941, "operator": "ge", "actual_fraction": 0.5902657921988264, "metric": 0.5493614083534691, "rescued": 104, "harmed": 2145, "missed_rescue": 2, "avoided_harm": 1646, "selected_both_correct": 743, "selected_both_wrong": 428}, {"budget": 0.37, "threshold": -0.890850305557251, "operator": "ge", "actual_fraction": 0.6047635484984467, "metric": 0.5383154987918537, "rescued": 104, "harmed": 2209, "missed_rescue": 2, "avoided_harm": 1582, "selected_both_correct": 763, "selected_both_wrong": 428}, {"budget": 0.38, "threshold": -0.8916605710983276, "operator": "ge", "actual_fraction": 0.6113220573006558, "metric": 0.5343458750431481, "rescued": 104, "harmed": 2232, "missed_rescue": 2, "avoided_harm": 1559, "selected_both_correct": 778, "selected_both_wrong": 428}, {"budget": 0.39, "threshold": -0.8928589224815369, "operator": "ge", "actual_fraction": 0.6206420434932689, "metric": 0.5279599585778392, "rescued": 104, "harmed": 2269, "missed_rescue": 2, "avoided_harm": 1522, "selected_both_correct": 794, "selected_both_wrong": 429}, {"budget": 0.4, "threshold": -0.8940943479537964, "operator": "ge", "actual_fraction": 0.6289264756644805, "metric": 0.5222644114601311, "rescued": 104, "harmed": 2302, "missed_rescue": 2, "avoided_harm": 1489, "selected_both_correct": 809, "selected_both_wrong": 429}, {"budget": 0.41, "threshold": -0.8949463963508606, "operator": "ge", "actual_fraction": 0.6370383154987919, "metric": 0.5169140490162237, "rescued": 104, "harmed": 2333, "missed_rescue": 2, "avoided_harm": 1458, "selected_both_correct": 824, "selected_both_wrong": 430}, {"budget": 0.42, "threshold": -0.8960229158401489, "operator": "ge", "actual_fraction": 0.6435968243010011, "metric": 0.5127718329306179, "rescued": 104, "harmed": 2357, "missed_rescue": 2, "avoided_harm": 1434, "selected_both_correct": 836, "selected_both_wrong": 432}, {"budget": 0.43, "threshold": -0.8966536521911621, "operator": "ge", "actual_fraction": 0.6491197790818087, "metric": 0.5082844321712116, "rescued": 104, "harmed": 2383, "missed_rescue": 2, "avoided_harm": 1408, "selected_both_correct": 842, "selected_both_wrong": 432}, {"budget": 0.44, "threshold": -0.8975363373756409, "operator": "ge", "actual_fraction": 0.6558508802209182, "metric": 0.5044874007594062, "rescued": 104, "harmed": 2405, "missed_rescue": 2, "avoided_harm": 1386, "selected_both_correct": 859, "selected_both_wrong": 432}, {"budget": 0.45, "threshold": -0.8988122940063477, "operator": "ge", "actual_fraction": 0.6641353123921298, "metric": 0.49896444597859857, "rescued": 104, "harmed": 2437, "missed_rescue": 2, "avoided_harm": 1354, "selected_both_correct": 875, "selected_both_wrong": 432}, {"budget": 0.46, "threshold": -0.9000959396362305, "operator": "ge", "actual_fraction": 0.6739730755954436, "metric": 0.4917155678287884, "rescued": 104, "harmed": 2479, "missed_rescue": 2, "avoided_harm": 1312, "selected_both_correct": 890, "selected_both_wrong": 432}, {"budget": 0.47, "threshold": -0.9013051986694336, "operator": "ge", "actual_fraction": 0.6848463928201588, "metric": 0.48412150500517775, "rescued": 104, "harmed": 2523, "missed_rescue": 2, "avoided_harm": 1268, "selected_both_correct": 907, "selected_both_wrong": 434}, {"budget": 0.48, "threshold": -0.9029324054718018, "operator": "ge", "actual_fraction": 0.696410079392475, "metric": 0.4775629962029686, "rescued": 104, "harmed": 2561, "missed_rescue": 2, "avoided_harm": 1230, "selected_both_correct": 935, "selected_both_wrong": 435}, {"budget": 0.49, "threshold": -0.9036871790885925, "operator": "ge", "actual_fraction": 0.7026234035208837, "metric": 0.47411114946496374, "rescued": 104, "harmed": 2581, "missed_rescue": 2, "avoided_harm": 1210, "selected_both_correct": 949, "selected_both_wrong": 437}, {"budget": 0.5, "threshold": -0.9044621586799622, "operator": "ge", "actual_fraction": 0.7088367276492924, "metric": 0.46996893337935797, "rescued": 104, "harmed": 2605, "missed_rescue": 2, "avoided_harm": 1186, "selected_both_correct": 961, "selected_both_wrong": 437}, {"budget": 0.51, "threshold": -0.9054575562477112, "operator": "ge", "actual_fraction": 0.717121159820504, "metric": 0.46496375560925096, "rescued": 104, "harmed": 2634, "missed_rescue": 2, "avoided_harm": 1157, "selected_both_correct": 980, "selected_both_wrong": 437}, {"budget": 0.52, "threshold": -0.906682550907135, "operator": "ge", "actual_fraction": 0.7262685536762168, "metric": 0.45875043148084227, "rescued": 104, "harmed": 2670, "missed_rescue": 2, "avoided_harm": 1121, "selected_both_correct": 997, "selected_both_wrong": 437}, {"budget": 0.53, "threshold": -0.9072114825248718, "operator": "ge", "actual_fraction": 0.7300655850880221, "metric": 0.4559889540904384, "rescued": 104, "harmed": 2686, "missed_rescue": 2, "avoided_harm": 1105, "selected_both_correct": 1002, "selected_both_wrong": 438}, {"budget": 0.54, "threshold": -0.9083563089370728, "operator": "ge", "actual_fraction": 0.7381774249223334, "metric": 0.4518467380048326, "rescued": 105, "harmed": 2711, "missed_rescue": 1, "avoided_harm": 1080, "selected_both_correct": 1023, "selected_both_wrong": 438}, {"budget": 0.55, "threshold": -0.9095192551612854, "operator": "ge", "actual_fraction": 0.7487055574732482, "metric": 0.4458060062133241, "rescued": 105, "harmed": 2746, "missed_rescue": 1, "avoided_harm": 1045, "selected_both_correct": 1047, "selected_both_wrong": 440}, {"budget": 0.56, "threshold": -0.9106979370117188, "operator": "ge", "actual_fraction": 0.759233690024163, "metric": 0.43924749741111496, "rescued": 105, "harmed": 2784, "missed_rescue": 1, "avoided_harm": 1007, "selected_both_correct": 1069, "selected_both_wrong": 441}, {"budget": 0.57, "threshold": -0.9115362167358398, "operator": "ge", "actual_fraction": 0.767863306869175, "metric": 0.4340697273041077, "rescued": 105, "harmed": 2814, "missed_rescue": 1, "avoided_harm": 977, "selected_both_correct": 1089, "selected_both_wrong": 441}, {"budget": 0.58, "threshold": -0.9124338030815125, "operator": "ge", "actual_fraction": 0.7751121850189852, "metric": 0.4301001035554021, "rescued": 105, "harmed": 2837, "missed_rescue": 1, "avoided_harm": 954, "selected_both_correct": 1108, "selected_both_wrong": 441}, {"budget": 0.59, "threshold": -0.9134801626205444, "operator": "ge", "actual_fraction": 0.7830514325163963, "metric": 0.42440455643769415, "rescued": 105, "harmed": 2870, "missed_rescue": 1, "avoided_harm": 921, "selected_both_correct": 1121, "selected_both_wrong": 441}, {"budget": 0.6, "threshold": -0.9143747091293335, "operator": "ge", "actual_fraction": 0.7904729030031067, "metric": 0.41939937866758714, "rescued": 105, "harmed": 2899, "missed_rescue": 1, "avoided_harm": 892, "selected_both_correct": 1134, "selected_both_wrong": 442}, {"budget": 0.61, "threshold": -0.915804386138916, "operator": "ge", "actual_fraction": 0.8010010355540214, "metric": 0.4133586468760787, "rescued": 105, "harmed": 2934, "missed_rescue": 1, "avoided_harm": 857, "selected_both_correct": 1158, "selected_both_wrong": 444}, {"budget": 0.62, "threshold": -0.9165146350860596, "operator": "ge", "actual_fraction": 0.8063513979979289, "metric": 0.41025198481187436, "rescued": 105, "harmed": 2952, "missed_rescue": 1, "avoided_harm": 839, "selected_both_correct": 1171, "selected_both_wrong": 444}, {"budget": 0.63, "threshold": -0.9172822833061218, "operator": "ge", "actual_fraction": 0.8110113910942354, "metric": 0.4062823610631688, "rescued": 105, "harmed": 2975, "missed_rescue": 1, "avoided_harm": 816, "selected_both_correct": 1175, "selected_both_wrong": 444}, {"budget": 0.64, "threshold": -0.9181071519851685, "operator": "ge", "actual_fraction": 0.8172247152226441, "metric": 0.40248532965136347, "rescued": 105, "harmed": 2997, "missed_rescue": 1, "avoided_harm": 794, "selected_both_correct": 1188, "selected_both_wrong": 445}, {"budget": 0.65, "threshold": -0.9192029237747192, "operator": "ge", "actual_fraction": 0.824301001035554, "metric": 0.3979979288919572, "rescued": 105, "harmed": 3023, "missed_rescue": 1, "avoided_harm": 768, "selected_both_correct": 1202, "selected_both_wrong": 446}, {"budget": 0.66, "threshold": -0.9200543761253357, "operator": "ge", "actual_fraction": 0.829306178805661, "metric": 0.39420089748015186, "rescued": 105, "harmed": 3045, "missed_rescue": 1, "avoided_harm": 746, "selected_both_correct": 1209, "selected_both_wrong": 446}, {"budget": 0.67, "threshold": -0.9208071231842041, "operator": "ge", "actual_fraction": 0.8348291335864687, "metric": 0.3898860890576458, "rescued": 105, "harmed": 3070, "missed_rescue": 1, "avoided_harm": 721, "selected_both_correct": 1216, "selected_both_wrong": 446}, {"budget": 0.68, "threshold": -0.9215553998947144, "operator": "ge", "actual_fraction": 0.840006903693476, "metric": 0.3867794269934415, "rescued": 105, "harmed": 3088, "missed_rescue": 1, "avoided_harm": 703, "selected_both_correct": 1228, "selected_both_wrong": 446}, {"budget": 0.69, "threshold": -0.9220835566520691, "operator": "ge", "actual_fraction": 0.8443217121159821, "metric": 0.38401794960303764, "rescued": 105, "harmed": 3104, "missed_rescue": 1, "avoided_harm": 687, "selected_both_correct": 1236, "selected_both_wrong": 447}, {"budget": 0.7, "threshold": -0.9232237935066223, "operator": "ge", "actual_fraction": 0.852088367276493, "metric": 0.37849499482222987, "rescued": 105, "harmed": 3136, "missed_rescue": 1, "avoided_harm": 655, "selected_both_correct": 1247, "selected_both_wrong": 449}, {"budget": 0.71, "threshold": -0.9243364334106445, "operator": "ge", "actual_fraction": 0.8598550224370038, "metric": 0.37279944770452195, "rescued": 105, "harmed": 3169, "missed_rescue": 1, "avoided_harm": 622, "selected_both_correct": 1257, "selected_both_wrong": 451}, {"budget": 0.72, "threshold": -0.9252541065216064, "operator": "ge", "actual_fraction": 0.8641698308595098, "metric": 0.37003797031411806, "rescued": 105, "harmed": 3185, "missed_rescue": 1, "avoided_harm": 606, "selected_both_correct": 1266, "selected_both_wrong": 451}, {"budget": 0.73, "threshold": -0.925781786441803, "operator": "ge", "actual_fraction": 0.8681394546082154, "metric": 0.3674490852606144, "rescued": 105, "harmed": 3200, "missed_rescue": 1, "avoided_harm": 591, "selected_both_correct": 1274, "selected_both_wrong": 451}, {"budget": 0.74, "threshold": -0.9274963140487671, "operator": "ge", "actual_fraction": 0.877114256127028, "metric": 0.36123576113220573, "rescued": 105, "harmed": 3236, "missed_rescue": 1, "avoided_harm": 555, "selected_both_correct": 1290, "selected_both_wrong": 451}, {"budget": 0.75, "threshold": -0.9286423325538635, "operator": "ge", "actual_fraction": 0.8828098032447359, "metric": 0.35726613738350016, "rescued": 105, "harmed": 3259, "missed_rescue": 1, "avoided_harm": 532, "selected_both_correct": 1299, "selected_both_wrong": 452}, {"budget": 0.76, "threshold": -0.9301291704177856, "operator": "ge", "actual_fraction": 0.8900586813945461, "metric": 0.3520883672764929, "rescued": 105, "harmed": 3289, "missed_rescue": 1, "avoided_harm": 502, "selected_both_correct": 1310, "selected_both_wrong": 453}, {"budget": 0.77, "threshold": -0.9318777918815613, "operator": "ge", "actual_fraction": 0.8979979288919572, "metric": 0.34673800483258543, "rescued": 105, "harmed": 3320, "missed_rescue": 1, "avoided_harm": 471, "selected_both_correct": 1325, "selected_both_wrong": 453}, {"budget": 0.78, "threshold": -0.9332055449485779, "operator": "ge", "actual_fraction": 0.9052468070417673, "metric": 0.34173282706247843, "rescued": 105, "harmed": 3349, "missed_rescue": 1, "avoided_harm": 442, "selected_both_correct": 1338, "selected_both_wrong": 453}, {"budget": 0.79, "threshold": -0.9344666600227356, "operator": "ge", "actual_fraction": 0.9100793924749742, "metric": 0.3379357956506731, "rescued": 105, "harmed": 3371, "missed_rescue": 1, "avoided_harm": 420, "selected_both_correct": 1344, "selected_both_wrong": 453}, {"budget": 0.8, "threshold": -0.9362465143203735, "operator": "ge", "actual_fraction": 0.917155678287884, "metric": 0.33224024853296513, "rescued": 105, "harmed": 3404, "missed_rescue": 1, "avoided_harm": 387, "selected_both_correct": 1351, "selected_both_wrong": 454}, {"budget": 0.81, "threshold": -0.937842071056366, "operator": "ge", "actual_fraction": 0.9237141870900932, "metric": 0.3274076630997584, "rescued": 105, "harmed": 3432, "missed_rescue": 1, "avoided_harm": 359, "selected_both_correct": 1361, "selected_both_wrong": 454}, {"budget": 0.82, "threshold": -0.9384457468986511, "operator": "ge", "actual_fraction": 0.9249223334483949, "metric": 0.3267172937521574, "rescued": 105, "harmed": 3436, "missed_rescue": 1, "avoided_harm": 355, "selected_both_correct": 1363, "selected_both_wrong": 455}, {"budget": 0.83, "threshold": -0.9396296143531799, "operator": "ge", "actual_fraction": 0.927166033828098, "metric": 0.324991370383155, "rescued": 105, "harmed": 3446, "missed_rescue": 1, "avoided_harm": 345, "selected_both_correct": 1365, "selected_both_wrong": 456}, {"budget": 0.84, "threshold": -0.9407488703727722, "operator": "ge", "actual_fraction": 0.9316534345875043, "metric": 0.3211943389713497, "rescued": 105, "harmed": 3468, "missed_rescue": 1, "avoided_harm": 323, "selected_both_correct": 1369, "selected_both_wrong": 456}, {"budget": 0.85, "threshold": -0.942738950252533, "operator": "ge", "actual_fraction": 0.9397652744218157, "metric": 0.31429064549534, "rescued": 105, "harmed": 3508, "missed_rescue": 1, "avoided_harm": 283, "selected_both_correct": 1376, "selected_both_wrong": 456}, {"budget": 0.86, "threshold": -0.9438527226448059, "operator": "gt", "actual_fraction": 0.9432171211598205, "metric": 0.31239212978943737, "rescued": 105, "harmed": 3519, "missed_rescue": 1, "avoided_harm": 272, "selected_both_correct": 1384, "selected_both_wrong": 457}, {"budget": 0.87, "threshold": -0.9450756907463074, "operator": "ge", "actual_fraction": 0.9456334138764239, "metric": 0.3106662064204349, "rescued": 105, "harmed": 3529, "missed_rescue": 1, "avoided_harm": 262, "selected_both_correct": 1388, "selected_both_wrong": 457}, {"budget": 0.88, "threshold": -0.9481598138809204, "operator": "ge", "actual_fraction": 0.9539178460476355, "metric": 0.3044528822920262, "rescued": 105, "harmed": 3565, "missed_rescue": 1, "avoided_harm": 226, "selected_both_correct": 1398, "selected_both_wrong": 459}, {"budget": 0.89, "threshold": -0.9500836730003357, "operator": "ge", "actual_fraction": 0.9606489471867449, "metric": 0.29858474283741804, "rescued": 105, "harmed": 3599, "missed_rescue": 1, "avoided_harm": 192, "selected_both_correct": 1402, "selected_both_wrong": 460}, {"budget": 0.9, "threshold": -0.9519118666648865, "operator": "ge", "actual_fraction": 0.9647911632723507, "metric": 0.2956506731101139, "rescued": 105, "harmed": 3616, "missed_rescue": 1, "avoided_harm": 175, "selected_both_correct": 1408, "selected_both_wrong": 461}, {"budget": 0.91, "threshold": -0.9543898701667786, "operator": "ge", "actual_fraction": 0.9697963410424577, "metric": 0.2911632723507076, "rescued": 105, "harmed": 3642, "missed_rescue": 1, "avoided_harm": 149, "selected_both_correct": 1410, "selected_both_wrong": 462}, {"budget": 0.92, "threshold": -0.9570822715759277, "operator": "ge", "actual_fraction": 0.9760096651708664, "metric": 0.28633068691750085, "rescued": 106, "harmed": 3671, "missed_rescue": 0, "avoided_harm": 120, "selected_both_correct": 1416, "selected_both_wrong": 462}, {"budget": 0.93, "threshold": -0.9593446254730225, "operator": "ge", "actual_fraction": 0.9801518812564722, "metric": 0.2827062478425958, "rescued": 106, "harmed": 3692, "missed_rescue": 0, "avoided_harm": 99, "selected_both_correct": 1419, "selected_both_wrong": 462}, {"budget": 0.94, "threshold": -0.9610995054244995, "operator": "ge", "actual_fraction": 0.9817052122885744, "metric": 0.28132550914739385, "rescued": 106, "harmed": 3700, "missed_rescue": 0, "avoided_harm": 91, "selected_both_correct": 1420, "selected_both_wrong": 462}, {"budget": 0.95, "threshold": -0.9635111093521118, "operator": "ge", "actual_fraction": 0.9853296513634795, "metric": 0.2787366240938902, "rescued": 106, "harmed": 3715, "missed_rescue": 0, "avoided_harm": 76, "selected_both_correct": 1426, "selected_both_wrong": 462}, {"budget": 0.96, "threshold": -0.9648061990737915, "operator": "ge", "actual_fraction": 0.9875733517431826, "metric": 0.27666551605108736, "rescued": 106, "harmed": 3727, "missed_rescue": 0, "avoided_harm": 64, "selected_both_correct": 1427, "selected_both_wrong": 462}, {"budget": 0.97, "threshold": -0.9674390554428101, "operator": "ge", "actual_fraction": 0.9906800138073869, "metric": 0.27390403866068347, "rescued": 106, "harmed": 3743, "missed_rescue": 0, "avoided_harm": 48, "selected_both_correct": 1429, "selected_both_wrong": 462}, {"budget": 0.98, "threshold": -0.9724470973014832, "operator": "ge", "actual_fraction": 0.9937866758715913, "metric": 0.2711425612702796, "rescued": 106, "harmed": 3759, "missed_rescue": 0, "avoided_harm": 32, "selected_both_correct": 1430, "selected_both_wrong": 463}, {"budget": 0.99, "threshold": -0.9774226546287537, "operator": "ge", "actual_fraction": 0.9965481532619952, "metric": 0.26889886089057646, "rescued": 106, "harmed": 3772, "missed_rescue": 0, "avoided_harm": 19, "selected_both_correct": 1432, "selected_both_wrong": 464}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.26561960648947186, "rescued": 106, "harmed": 3791, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 1433, "selected_both_wrong": 464}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.9894999999999999, "actual_fraction": 0.004, "metric_std": 0.00045643546458770976, "actual_fraction_std": 0.005477225575051661, "per_seed": [{"seed": 2026, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.5, "epoch": 3, "metric": 0.9891666666666665, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 3}, {"seed": 2027, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.25, "epoch": 2, "metric": 0.9891666666666665, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 3}, {"seed": 2028, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.5, "epoch": 3, "metric": 0.9891666666666665, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 3}, {"seed": 2029, "budget": 0.01, "threshold": -0.053622053345030285, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.99, "actual_fraction": 0.01, "rescue": 1, "harm": 0, "missed_rescue": 2}, {"seed": 2030, "budget": 0.01, "threshold": -0.04478147329272479, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.99, "actual_fraction": 0.01, "rescue": 1, "harm": 0, "missed_rescue": 2}], "rescued": 0.4, "harmed": 0.0, "missed_rescue": 2.6, "avoided_harm": 857.0}, "validation_curve": [{"budget": 0.0, "metric": 0.9891666666666665, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.9893333333333333, "metric_std": 0.0006972166887784256, "actual_fraction": 0.01, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.9893333333333333, "metric_std": 0.0006972166887784256, "actual_fraction": 0.02, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.9891666666666665, "metric_std": 0.0005892556509888032, "actual_fraction": 0.03, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.9889999999999999, "metric_std": 0.0003726779962499239, "actual_fraction": 0.04, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.9888333333333333, "metric_std": 0.00045643546458758817, "actual_fraction": 0.05, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.9888333333333333, "metric_std": 0.00045643546458758817, "actual_fraction": 0.06, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.9886666666666667, "metric_std": 0.0007453559924998479, "actual_fraction": 0.07, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.9881666666666666, "metric_std": 0.00108653373420036, "actual_fraction": 0.08, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.9873333333333333, "metric_std": 0.0016029486718059016, "actual_fraction": 0.09, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.986, "metric_std": 0.0012360330811826802, "actual_fraction": 0.1, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.984, "metric_std": 0.0013693063937629164, "actual_fraction": 0.11000000000000001, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.12, "metric": 0.9828333333333333, "metric_std": 0.0009501461875825445, "actual_fraction": 0.12, "actual_fraction_std": 0.0}, {"budget": 0.13, "metric": 0.9813333333333334, "metric_std": 0.0009501461875826369, "actual_fraction": 0.13, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.9796666666666667, "metric_std": 0.0015138251770488297, "actual_fraction": 0.14, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.9776666666666667, "metric_std": 0.0012360330811825716, "actual_fraction": 0.15, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.9751666666666665, "metric_std": 0.0013693063937629468, "actual_fraction": 0.16, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.9731666666666665, "metric_std": 0.0006972166887784654, "actual_fraction": 0.17, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.9726666666666667, "metric_std": 0.001086533734200424, "actual_fraction": 0.18, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.9704999999999998, "metric_std": 0.0012638125740085332, "actual_fraction": 0.19, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.9685, "metric_std": 0.0018066236157232188, "actual_fraction": 0.2, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.9663333333333334, "metric_std": 0.0009501461875826369, "actual_fraction": 0.20983333333333332, "actual_fraction_std": 0.0003726779962499612}, {"budget": 0.22, "metric": 0.9633333333333333, "metric_std": 0.0021245914639969883, "actual_fraction": 0.22000000000000003, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.23, "metric": 0.9594999999999999, "metric_std": 0.0007453559924999471, "actual_fraction": 0.23000000000000004, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.24, "metric": 0.9555, "metric_std": 0.0007453559924999471, "actual_fraction": 0.24, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.9504999999999999, "metric_std": 0.001263812574008621, "actual_fraction": 0.25, "actual_fraction_std": 0.0}, {"budget": 0.26, "metric": 0.946, "metric_std": 0.001369306393762947, "actual_fraction": 0.26, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.9401666666666667, "metric_std": 0.0010865337342005048, "actual_fraction": 0.27, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.9343333333333333, "metric_std": 0.0010865337342004708, "actual_fraction": 0.28, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.9276666666666668, "metric_std": 0.0009128709291753282, "actual_fraction": 0.29, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.9215, "metric_std": 0.0023863035105460267, "actual_fraction": 0.3, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.9151666666666667, "metric_std": 0.0013693063937628592, "actual_fraction": 0.31, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.9076666666666666, "metric_std": 0.001236033081182639, "actual_fraction": 0.32, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.901, "metric_std": 0.0018066236157232342, "actual_fraction": 0.33, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.8936666666666667, "metric_std": 0.0017280367794439257, "actual_fraction": 0.34, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.8856666666666667, "metric_std": 0.0016029486718059306, "actual_fraction": 0.35, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.8775000000000001, "metric_std": 0.002124591463996999, "actual_fraction": 0.36, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.8694999999999998, "metric_std": 0.002251542681017729, "actual_fraction": 0.37, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.8615, "metric_std": 0.002969754946867524, "actual_fraction": 0.38, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.8536666666666666, "metric_std": 0.0029226129861250442, "actual_fraction": 0.39, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.8455, "metric_std": 0.0022515426810176963, "actual_fraction": 0.4, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.8370000000000001, "metric_std": 0.002006932429798707, "actual_fraction": 0.4091666666666668, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.42, "metric": 0.828, "metric_std": 0.001728036779443971, "actual_fraction": 0.42000000000000004, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.8188333333333334, "metric_std": 0.0015138251770487135, "actual_fraction": 0.43, "actual_fraction_std": 0.0}, {"budget": 0.44, "metric": 0.8098333333333333, "metric_std": 0.0014907119849999317, "actual_fraction": 0.44000000000000006, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.45, "metric": 0.8008333333333333, "metric_std": 0.0020412414523192715, "actual_fraction": 0.45, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.792, "metric_std": 0.001825741858350601, "actual_fraction": 0.4600000000000001, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.47, "metric": 0.7831666666666667, "metric_std": 0.0010865337342004667, "actual_fraction": 0.4699999999999999, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.48, "metric": 0.7741666666666667, "metric_std": 0.0015590239111557857, "actual_fraction": 0.48, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.7653333333333332, "metric_std": 0.002173067468400878, "actual_fraction": 0.49000000000000005, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.5, "metric": 0.7558333333333334, "metric_std": 0.0021245914639969666, "actual_fraction": 0.5, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.7461666666666666, "metric_std": 0.0017280367794440192, "actual_fraction": 0.51, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.7369999999999999, "metric_std": 0.0016244657241347908, "actual_fraction": 0.52, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.7283333333333333, "metric_std": 0.0010206207261596358, "actual_fraction": 0.53, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.7188333333333333, "metric_std": 0.0007453559924998974, "actual_fraction": 0.54, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.7095, "metric_std": 0.001513825177048744, "actual_fraction": 0.55, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.7, "metric_std": 0.0013176156917368376, "actual_fraction": 0.56, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.6915, "metric_std": 0.0010865337342004196, "actual_fraction": 0.5691666666666667, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.6809999999999999, "metric_std": 0.0010865337342003687, "actual_fraction": 0.58, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.6714999999999999, "metric_std": 0.0003726779962499239, "actual_fraction": 0.59, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.6615, "metric_std": 0.0003726779962499239, "actual_fraction": 0.6, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.6521666666666667, "metric_std": 0.0012638125740085477, "actual_fraction": 0.61, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.6428333333333334, "metric_std": 0.0007453559924999471, "actual_fraction": 0.62, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.6331666666666667, "metric_std": 0.0006972166887784124, "actual_fraction": 0.63, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.6234999999999999, "metric_std": 0.00037267799624997355, "actual_fraction": 0.64, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.6146666666666667, "metric_std": 0.0009501461875826029, "actual_fraction": 0.65, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.6053333333333333, "metric_std": 0.0009501461875826223, "actual_fraction": 0.66, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.5961666666666667, "metric_std": 0.0013944333775567717, "actual_fraction": 0.67, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.5864999999999999, "metric_std": 0.0016029486718059651, "actual_fraction": 0.68, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.5786666666666667, "metric_std": 0.0012638125740085699, "actual_fraction": 0.6891666666666667, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.5678333333333334, "metric_std": 0.001263812574008599, "actual_fraction": 0.7, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.5583333333333333, "metric_std": 0.00155902391115583, "actual_fraction": 0.71, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.5489999999999999, "metric_std": 0.0010865337342004621, "actual_fraction": 0.72, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.5399999999999999, "metric_std": 0.0008333333333333526, "actual_fraction": 0.73, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.5299999999999999, "metric_std": 0.0008333333333333526, "actual_fraction": 0.74, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.5203333333333333, "metric_std": 0.0009501461875825834, "actual_fraction": 0.75, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.5105, "metric_std": 0.0009501461875826174, "actual_fraction": 0.76, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.5006666666666666, "metric_std": 0.0006972166887784256, "actual_fraction": 0.77, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.4913333333333333, "metric_std": 0.0009501461875826174, "actual_fraction": 0.78, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.48166666666666663, "metric_std": 0.000589255650988764, "actual_fraction": 0.79, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.472, "metric_std": 0.00045643546458755776, "actual_fraction": 0.8, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.4621666666666666, "metric_std": 0.0007453559924999036, "actual_fraction": 0.8100000000000002, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.82, "metric": 0.4535, "metric_std": 0.0009128709291752372, "actual_fraction": 0.8191666666666666, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.83, "metric": 0.44316666666666665, "metric_std": 0.0006972166887783726, "actual_fraction": 0.8299999999999998, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.84, "metric": 0.4333333333333334, "metric_std": 0.0005892556509887639, "actual_fraction": 0.8400000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.42366666666666675, "metric_std": 0.000456435464587649, "actual_fraction": 0.85, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.41400000000000003, "metric_std": 0.0003726779962499984, "actual_fraction": 0.86, "actual_fraction_std": 0.0}, {"budget": 0.87, "metric": 0.40449999999999997, "metric_std": 0.0007453559924999223, "actual_fraction": 0.8699999999999999, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.39466666666666667, "metric_std": 0.0007453559924999036, "actual_fraction": 0.8800000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.89, "metric": 0.38466666666666666, "metric_std": 0.0007453559924999254, "actual_fraction": 0.89, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.375, "metric_std": 0.0010206207261596358, "actual_fraction": 0.9, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.36499999999999994, "metric_std": 0.0010206207261596358, "actual_fraction": 0.9099999999999999, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.35583333333333333, "metric_std": 0.0011785113019775672, "actual_fraction": 0.9200000000000002, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.93, "metric": 0.346, "metric_std": 0.0010865337342004346, "actual_fraction": 0.93, "actual_fraction_std": 0.0}, {"budget": 0.94, "metric": 0.336, "metric_std": 0.001086533734200443, "actual_fraction": 0.9399999999999998, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.95, "metric": 0.32616666666666666, "metric_std": 0.0009501461875826369, "actual_fraction": 0.95, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.3165, "metric_std": 0.0006972166887784124, "actual_fraction": 0.96, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.30700000000000005, "metric_std": 0.0007453559924999471, "actual_fraction": 0.97, "actual_fraction_std": 0.0}, {"budget": 0.98, "metric": 0.2973333333333333, "metric_std": 0.00037267799624997355, "actual_fraction": 0.9800000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.99, "metric": 0.28733333333333333, "metric_std": 0.00037267799624997355, "actual_fraction": 0.99, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.27749999999999997, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.9013117017604418, "actual_fraction": 0.008491542975491888, "metric_std": 0.00042975145315806427, "actual_fraction_std": 0.011668441889060724, "per_seed": [{"seed": 2026, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.5, "epoch": 3, "metric": 0.9016223679668623, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 106}, {"seed": 2027, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.25, "epoch": 2, "metric": 0.9016223679668623, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 106}, {"seed": 2028, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.5, "epoch": 3, "metric": 0.9016223679668623, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 106}, {"seed": 2029, "budget": 0.01, "threshold": -0.053622053345030285, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.9009319986192614, "actual_fraction": 0.019848118743527786, "rescue": 1, "harm": 5, "missed_rescue": 105}, {"seed": 2030, "budget": 0.01, "threshold": -0.04478147329272479, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.9007594062823611, "actual_fraction": 0.022609596133931653, "rescue": 4, "harm": 9, "missed_rescue": 102}], "rescued": 1.0, "harmed": 2.8, "missed_rescue": 105.0, "avoided_harm": 3788.2}, "test_curve": [{"budget": 0.0, "metric": 0.9016223679668623, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.9013462202278217, "metric_std": 0.00048202416443791224, "actual_fraction": 0.017293752157404214, "actual_fraction_std": 0.004977426548241186}, {"budget": 0.02, "metric": 0.9012081463583017, "metric_std": 0.0005539111679627347, "actual_fraction": 0.026337590610976874, "actual_fraction_std": 0.005038691318896594}, {"budget": 0.03, "metric": 0.9011391094235416, "metric_std": 0.000661729013209592, "actual_fraction": 0.03462202278218847, "actual_fraction_std": 0.004730730452762932}, {"budget": 0.04, "metric": 0.9010700724887816, "metric_std": 0.0006150683355611353, "actual_fraction": 0.04425267518122195, "actual_fraction_std": 0.0018772144192169445}, {"budget": 0.05, "metric": 0.9007594062823611, "metric_std": 0.0007814450412614042, "actual_fraction": 0.05322747670003451, "actual_fraction_std": 0.002795249988941753}, {"budget": 0.06, "metric": 0.9004832585433207, "metric_std": 0.0009655920421608053, "actual_fraction": 0.06258198136002761, "actual_fraction_std": 0.0041144362452497606}, {"budget": 0.07, "metric": 0.9001035554021402, "metric_std": 0.0005903566232822699, "actual_fraction": 0.07328270624784258, "actual_fraction_std": 0.00334357285970927}, {"budget": 0.08, "metric": 0.8996202968588195, "metric_std": 0.0008058417348932683, "actual_fraction": 0.0873662409389023, "actual_fraction_std": 0.0032550933041855185}, {"budget": 0.09, "metric": 0.8985157059026581, "metric_std": 0.0005852890907080777, "actual_fraction": 0.10024162927166033, "actual_fraction_std": 0.00393835896710885}, {"budget": 0.1, "metric": 0.8973420780117364, "metric_std": 0.0009001315022716231, "actual_fraction": 0.1131515360717984, "actual_fraction_std": 0.004508262492189997}, {"budget": 0.11, "metric": 0.8962374870555747, "metric_std": 0.0013936230057429314, "actual_fraction": 0.1257852951328961, "actual_fraction_std": 0.0046043676300154155}, {"budget": 0.12, "metric": 0.8949257852951329, "metric_std": 0.002083297463169618, "actual_fraction": 0.13476009665170868, "actual_fraction_std": 0.005149502864976609}, {"budget": 0.13, "metric": 0.8937176389368313, "metric_std": 0.0017362480276091586, "actual_fraction": 0.1443562305833621, "actual_fraction_std": 0.0033391153549147895}, {"budget": 0.14, "metric": 0.8921297894373488, "metric_std": 0.0016418965818468078, "actual_fraction": 0.15602347255781843, "actual_fraction_std": 0.0026457904240172396}, {"budget": 0.15, "metric": 0.8909561615464273, "metric_std": 0.0012938670910591294, "actual_fraction": 0.1663444943044529, "actual_fraction_std": 0.0018322446706372387}, {"budget": 0.16, "metric": 0.8887124611667241, "metric_std": 0.0015930936498742838, "actual_fraction": 0.177148774594408, "actual_fraction_std": 0.0032669682455329797}, {"budget": 0.17, "metric": 0.8859164653089403, "metric_std": 0.0014996675178618212, "actual_fraction": 0.1882982395581636, "actual_fraction_std": 0.0011769228250764453}, {"budget": 0.18, "metric": 0.8832240248532965, "metric_std": 0.0013234580264190842, "actual_fraction": 0.19879185364169832, "actual_fraction_std": 0.0030201193252084317}, {"budget": 0.19, "metric": 0.8811529168104937, "metric_std": 0.001187003718608877, "actual_fraction": 0.20714532274767, "actual_fraction_std": 0.003730550524295833}, {"budget": 0.2, "metric": 0.8785295132896099, "metric_std": 0.0010597225704527984, "actual_fraction": 0.21629271660338278, "actual_fraction_std": 0.003563101810122719}, {"budget": 0.21, "metric": 0.8754918881601658, "metric_std": 0.0017836397214723851, "actual_fraction": 0.22802899551259928, "actual_fraction_std": 0.004201834791646161}, {"budget": 0.22, "metric": 0.8723852260959614, "metric_std": 0.002087582618461681, "actual_fraction": 0.23845357266137385, "actual_fraction_std": 0.004926596787210022}, {"budget": 0.23, "metric": 0.868519157749396, "metric_std": 0.002865760499137761, "actual_fraction": 0.2504314808422506, "actual_fraction_std": 0.005250609508172691}, {"budget": 0.24, "metric": 0.8648947186744907, "metric_std": 0.00287613620469058, "actual_fraction": 0.25930272695892304, "actual_fraction_std": 0.005195860699055066}, {"budget": 0.25, "metric": 0.859268208491543, "metric_std": 0.0040192156738039315, "actual_fraction": 0.2725578184328616, "actual_fraction_std": 0.007287199199743697}, {"budget": 0.26, "metric": 0.8528132550914738, "metric_std": 0.004923270150775339, "actual_fraction": 0.2854677252329996, "actual_fraction_std": 0.008852826098192733}, {"budget": 0.27, "metric": 0.8480497065930273, "metric_std": 0.005515668735611658, "actual_fraction": 0.29527096996893337, "actual_fraction_std": 0.011293527834724842}, {"budget": 0.28, "metric": 0.8405937176389369, "metric_std": 0.007174113511549884, "actual_fraction": 0.30880220918191237, "actual_fraction_std": 0.013671557470302386}, {"budget": 0.29, "metric": 0.8345875043148083, "metric_std": 0.006468874987986197, "actual_fraction": 0.32033137728684846, "actual_fraction_std": 0.012348496242031987}, {"budget": 0.3, "metric": 0.8285467725233, "metric_std": 0.006546689202331235, "actual_fraction": 0.3315153607179841, "actual_fraction_std": 0.01285691278681645}, {"budget": 0.31, "metric": 0.8238177424922333, "metric_std": 0.005585169090833507, "actual_fraction": 0.34100793924749745, "actual_fraction_std": 0.011622399235405144}, {"budget": 0.32, "metric": 0.8183293061788056, "metric_std": 0.006332751289678691, "actual_fraction": 0.3503624439074905, "actual_fraction_std": 0.01229228262379662}, {"budget": 0.33, "metric": 0.8106662064204349, "metric_std": 0.006267027150626544, "actual_fraction": 0.3631342768381084, "actual_fraction_std": 0.012226677958727429}, {"budget": 0.34, "metric": 0.802174663444943, "metric_std": 0.008317983778858642, "actual_fraction": 0.3774249223334484, "actual_fraction_std": 0.015135275355997681}, {"budget": 0.35, "metric": 0.7926130479806697, "metric_std": 0.009907611950299062, "actual_fraction": 0.39223334483948913, "actual_fraction_std": 0.017385883264031333}, {"budget": 0.36, "metric": 0.7836382464618571, "metric_std": 0.011049010394932616, "actual_fraction": 0.40642043493268903, "actual_fraction_std": 0.017520630131094197}, {"budget": 0.37, "metric": 0.775733517431826, "metric_std": 0.011246744937594982, "actual_fraction": 0.4175353814290646, "actual_fraction_std": 0.017795182256552113}, {"budget": 0.38, "metric": 0.7685191577493959, "metric_std": 0.013158623584790016, "actual_fraction": 0.4274076630997584, "actual_fraction_std": 0.020390964719901254}, {"budget": 0.39, "metric": 0.7595098377632032, "metric_std": 0.011597511567610769, "actual_fraction": 0.4390403866068347, "actual_fraction_std": 0.019594133679867204}, {"budget": 0.4, "metric": 0.7502243700379703, "metric_std": 0.009933734834113259, "actual_fraction": 0.45184673800483255, "actual_fraction_std": 0.01633164962676369}, {"budget": 0.41, "metric": 0.7433206765619605, "metric_std": 0.0109159707149524, "actual_fraction": 0.46144287193648603, "actual_fraction_std": 0.01748284553198961}, {"budget": 0.42, "metric": 0.7337245426303072, "metric_std": 0.011746046616566136, "actual_fraction": 0.4747324818778047, "actual_fraction_std": 0.01701483851516919}, {"budget": 0.43, "metric": 0.7213324128408697, "metric_std": 0.01118965516701828, "actual_fraction": 0.49074905074214703, "actual_fraction_std": 0.015446580255127912}, {"budget": 0.44, "metric": 0.7119433897134966, "metric_std": 0.01218152263308793, "actual_fraction": 0.5037970314118053, "actual_fraction_std": 0.0159925217930416}, {"budget": 0.45, "metric": 0.7041076976182258, "metric_std": 0.012779177654707754, "actual_fraction": 0.5137038315498792, "actual_fraction_std": 0.01678880812512295}, {"budget": 0.46, "metric": 0.6953400069036935, "metric_std": 0.011658225902783547, "actual_fraction": 0.5245081118398344, "actual_fraction_std": 0.014931481671167826}, {"budget": 0.47, "metric": 0.6861926130479806, "metric_std": 0.010581332001972041, "actual_fraction": 0.5357956506731101, "actual_fraction_std": 0.013363707734037049}, {"budget": 0.48, "metric": 0.6769761822575078, "metric_std": 0.008310101469923588, "actual_fraction": 0.5473593372454263, "actual_fraction_std": 0.010990893583907998}, {"budget": 0.49, "metric": 0.6702795995857784, "metric_std": 0.008984421704687993, "actual_fraction": 0.5563341387642389, "actual_fraction_std": 0.011644931721863537}, {"budget": 0.5, "metric": 0.6588885053503624, "metric_std": 0.010022697211528145, "actual_fraction": 0.5699344149119779, "actual_fraction_std": 0.012502763286916395}, {"budget": 0.51, "metric": 0.6515705902657922, "metric_std": 0.00941016093046247, "actual_fraction": 0.5784604763548498, "actual_fraction_std": 0.011977211822676045}, {"budget": 0.52, "metric": 0.6449430445288229, "metric_std": 0.00815523904108602, "actual_fraction": 0.5867449085260614, "actual_fraction_std": 0.01061940887459674}, {"budget": 0.53, "metric": 0.6365895754228512, "metric_std": 0.008665441689761416, "actual_fraction": 0.5965481532619952, "actual_fraction_std": 0.01133775337575874}, {"budget": 0.54, "metric": 0.6287538833275803, "metric_std": 0.008773396635925985, "actual_fraction": 0.6060407317915084, "actual_fraction_std": 0.012052210751062655}, {"budget": 0.55, "metric": 0.619330341732827, "metric_std": 0.010292495931949851, "actual_fraction": 0.6169830859509837, "actual_fraction_std": 0.01332732500765606}, {"budget": 0.56, "metric": 0.6115982050396962, "metric_std": 0.012571192672879076, "actual_fraction": 0.6263030721435967, "actual_fraction_std": 0.015266082451692984}, {"budget": 0.57, "metric": 0.6018639972385225, "metric_std": 0.011704763875830757, "actual_fraction": 0.637176389368312, "actual_fraction_std": 0.01450001626122001}, {"budget": 0.58, "metric": 0.5939247497411115, "metric_std": 0.011256540487224248, "actual_fraction": 0.6458405246807042, "actual_fraction_std": 0.01421662261664556}, {"budget": 0.59, "metric": 0.5868139454608216, "metric_std": 0.012492991125733634, "actual_fraction": 0.6541249568519159, "actual_fraction_std": 0.015730065963829143}, {"budget": 0.6, "metric": 0.5776665516051087, "metric_std": 0.012675259245482478, "actual_fraction": 0.6644804970659303, "actual_fraction_std": 0.015837639354041328}, {"budget": 0.61, "metric": 0.5693821194338972, "metric_std": 0.01280967720659687, "actual_fraction": 0.6740766309975837, "actual_fraction_std": 0.015898461879845865}, {"budget": 0.62, "metric": 0.5602692440455643, "metric_std": 0.01669664725546424, "actual_fraction": 0.6841215050051778, "actual_fraction_std": 0.019640446778641834}, {"budget": 0.63, "metric": 0.5521574042112529, "metric_std": 0.013542395878222282, "actual_fraction": 0.6933379357956507, "actual_fraction_std": 0.016073063891191286}, {"budget": 0.64, "metric": 0.5454263030721437, "metric_std": 0.012110645756007295, "actual_fraction": 0.7011736278909216, "actual_fraction_std": 0.013899095676753187}, {"budget": 0.65, "metric": 0.5387297204004142, "metric_std": 0.012055299841767638, "actual_fraction": 0.7088712461166724, "actual_fraction_std": 0.01326908474207424}, {"budget": 0.66, "metric": 0.5298929927511218, "metric_std": 0.01072186049475577, "actual_fraction": 0.7187435277873663, "actual_fraction_std": 0.011444967322783366}, {"budget": 0.67, "metric": 0.5228512254055919, "metric_std": 0.011022693067920088, "actual_fraction": 0.7263375906109768, "actual_fraction_std": 0.011692159554712806}, {"budget": 0.68, "metric": 0.5141180531584397, "metric_std": 0.011814320518565846, "actual_fraction": 0.7358992060752502, "actual_fraction_std": 0.012749306090639262}, {"budget": 0.69, "metric": 0.5084225060407318, "metric_std": 0.011807384752105103, "actual_fraction": 0.7424231964100795, "actual_fraction_std": 0.012847757792752235}, {"budget": 0.7, "metric": 0.4998274076630997, "metric_std": 0.010741706685160649, "actual_fraction": 0.7515360717984121, "actual_fraction_std": 0.011706163518214881}, {"budget": 0.71, "metric": 0.49140490162236794, "metric_std": 0.008212566555717482, "actual_fraction": 0.7609250949257852, "actual_fraction_std": 0.009023130014189935}, {"budget": 0.72, "metric": 0.4842595788746979, "metric_std": 0.006992013582557909, "actual_fraction": 0.7688988608905765, "actual_fraction_std": 0.00736304073444995}, {"budget": 0.73, "metric": 0.4764929237141871, "metric_std": 0.008313506111049893, "actual_fraction": 0.7772868484639281, "actual_fraction_std": 0.008975299367514586}, {"budget": 0.74, "metric": 0.46979634104245765, "metric_std": 0.008735116252761188, "actual_fraction": 0.7844666896789783, "actual_fraction_std": 0.009646197907230713}, {"budget": 0.75, "metric": 0.46213324128408695, "metric_std": 0.007811209602623734, "actual_fraction": 0.7927856403175699, "actual_fraction_std": 0.008448743065773047}, {"budget": 0.76, "metric": 0.45522954780807723, "metric_std": 0.007980972734211216, "actual_fraction": 0.8000345184673799, "actual_fraction_std": 0.008572993382662377}, {"budget": 0.77, "metric": 0.44798066965826716, "metric_std": 0.01009082319103756, "actual_fraction": 0.8076976182257509, "actual_fraction_std": 0.010850966832441888}, {"budget": 0.78, "metric": 0.440731791508457, "metric_std": 0.01016215706621783, "actual_fraction": 0.8158784949948222, "actual_fraction_std": 0.010747528673699937}, {"budget": 0.79, "metric": 0.4333103210217466, "metric_std": 0.010684427555187498, "actual_fraction": 0.8236796686227132, "actual_fraction_std": 0.011179001647675286}, {"budget": 0.8, "metric": 0.42471522264411454, "metric_std": 0.00961991351908102, "actual_fraction": 0.8327925440110459, "actual_fraction_std": 0.01007738263779321}, {"budget": 0.81, "metric": 0.41539523645150156, "metric_std": 0.007979852938204359, "actual_fraction": 0.8430100103555402, "actual_fraction_std": 0.008234481502841902}, {"budget": 0.82, "metric": 0.40662754573696924, "metric_std": 0.010125008490620464, "actual_fraction": 0.8523299965481534, "actual_fraction_std": 0.010309846259100332}, {"budget": 0.83, "metric": 0.39727304107697614, "metric_std": 0.011803599889015796, "actual_fraction": 0.8620987228167069, "actual_fraction_std": 0.012286222834114194}, {"budget": 0.84, "metric": 0.39078356920952706, "metric_std": 0.012297370539479976, "actual_fraction": 0.8689678978253366, "actual_fraction_std": 0.012835463617836489}, {"budget": 0.85, "metric": 0.38405246807041765, "metric_std": 0.013044943969619919, "actual_fraction": 0.8761477390403867, "actual_fraction_std": 0.013857240856535244}, {"budget": 0.86, "metric": 0.37562996202968585, "metric_std": 0.011118885406131145, "actual_fraction": 0.8854332067656194, "actual_fraction_std": 0.011987031707300535}, {"budget": 0.87, "metric": 0.3686917500862961, "metric_std": 0.009413325924810784, "actual_fraction": 0.8927166033828099, "actual_fraction_std": 0.009979509209677079}, {"budget": 0.88, "metric": 0.3625819813600276, "metric_std": 0.009958443234158702, "actual_fraction": 0.8991025198481187, "actual_fraction_std": 0.010582176515481686}, {"budget": 0.89, "metric": 0.35547117707973763, "metric_std": 0.011087361813249067, "actual_fraction": 0.9068001380738696, "actual_fraction_std": 0.011713413531900039}, {"budget": 0.9, "metric": 0.347186744908526, "metric_std": 0.009906108541851958, "actual_fraction": 0.9154642733862618, "actual_fraction_std": 0.010554554123959024}, {"budget": 0.91, "metric": 0.34021401449775623, "metric_std": 0.009185901843110777, "actual_fraction": 0.9226441146013118, "actual_fraction_std": 0.009734575011985396}, {"budget": 0.92, "metric": 0.33227476700034514, "metric_std": 0.008649786577388427, "actual_fraction": 0.9308595098377632, "actual_fraction_std": 0.009306872734140771}, {"budget": 0.93, "metric": 0.32454263030721425, "metric_std": 0.007511832385216758, "actual_fraction": 0.9388677942699346, "actual_fraction_std": 0.008099344807894947}, {"budget": 0.94, "metric": 0.3172937521574042, "metric_std": 0.007539539733311608, "actual_fraction": 0.9463237832240248, "actual_fraction_std": 0.008072267403292007}, {"budget": 0.95, "metric": 0.309319986192613, "metric_std": 0.007060904167899901, "actual_fraction": 0.9547462892647566, "actual_fraction_std": 0.007499727880635131}, {"budget": 0.96, "metric": 0.3002761477390403, "metric_std": 0.005955561118886025, "actual_fraction": 0.9643079047290299, "actual_fraction_std": 0.006343091249687047}, {"budget": 0.97, "metric": 0.29085260614428715, "metric_std": 0.0032205933550012262, "actual_fraction": 0.9741456679323438, "actual_fraction_std": 0.0036457450273480675}, {"budget": 0.98, "metric": 0.2827062478425958, "metric_std": 0.0025657695456193772, "actual_fraction": 0.9824991370383156, "actual_fraction_std": 0.0028244038542391227}, {"budget": 0.99, "metric": 0.2741456679323438, "metric_std": 0.0020660679670894597, "actual_fraction": 0.9913013462202278, "actual_fraction_std": 0.002212281007133522}, {"budget": 1.0, "metric": 0.2656196064894718, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 330, "rescuable": 3, "harmful": 857, "both_wrong": 10, "unit": "sample", "total": 1200}, "test": {"both_correct": 1433, "rescuable": 106, "harmful": 3791, "both_wrong": 464, "unit": "sample", "total": 5794}}, "small": {"validation": 0.9891666666666666, "test": 0.9016223679668622}, "large": {"validation": 0.2775, "test": 0.26561960648947186}, "learned_minus_confidence_test": -0.0003106662064203958, "learned_minus_small_test": -0.0003106662064203958, "learned_minus_large_test": 0.6356920952709699, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_glsim_MiniCPM"}, {"key": ["cub", "glsim", "Qwen3.8"], "expert_display": "GLSim", "run": "outputs/full_budget_20260913/runs/full_glsim_Qwen3.8_s{2026..2030}", "validation_rows": 1200, "test_rows": 5794, "confidence_rule": {"column": "max_probability", "columns": ["entropy", "max_probability", "top_two_margin", "entropy_valid", "max_probability_valid", "top_two_margin_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/glsim/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 0, "validation_area_0_25": 0.9213333333333333, "validation_point": {"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9891666666666666, "rescued": 0, "harmed": 0, "missed_rescue": 2, "avoided_harm": 528, "selected_both_correct": 0, "selected_both_wrong": 0}, "test_point": {"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9016223679668622, "rescued": 0, "harmed": 0, "missed_rescue": 144, "avoided_harm": 2192, "selected_both_correct": 0, "selected_both_wrong": 0}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9891666666666666, "rescued": 0, "harmed": 0, "missed_rescue": 2, "avoided_harm": 528, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.48843976855278015, "operator": "ge", "actual_fraction": 0.01, "metric": 0.9858333333333333, "rescued": 1, "harmed": 5, "missed_rescue": 1, "avoided_harm": 523, "selected_both_correct": 1, "selected_both_wrong": 5}, {"budget": 0.02, "threshold": -0.6014261841773987, "operator": "ge", "actual_fraction": 0.02, "metric": 0.98, "rescued": 1, "harmed": 12, "missed_rescue": 1, "avoided_harm": 516, "selected_both_correct": 4, "selected_both_wrong": 7}, {"budget": 0.03, "threshold": -0.6401472687721252, "operator": "ge", "actual_fraction": 0.03, "metric": 0.9741666666666666, "rescued": 2, "harmed": 20, "missed_rescue": 0, "avoided_harm": 508, "selected_both_correct": 6, "selected_both_wrong": 8}, {"budget": 0.04, "threshold": -0.6884650588035583, "operator": "ge", "actual_fraction": 0.04, "metric": 0.9675, "rescued": 2, "harmed": 28, "missed_rescue": 0, "avoided_harm": 500, "selected_both_correct": 8, "selected_both_wrong": 10}, {"budget": 0.05, "threshold": -0.7208121418952942, "operator": "ge", "actual_fraction": 0.05, "metric": 0.96, "rescued": 2, "harmed": 37, "missed_rescue": 0, "avoided_harm": 491, "selected_both_correct": 11, "selected_both_wrong": 10}, {"budget": 0.06, "threshold": -0.7578181624412537, "operator": "ge", "actual_fraction": 0.06, "metric": 0.9558333333333333, "rescued": 2, "harmed": 42, "missed_rescue": 0, "avoided_harm": 486, "selected_both_correct": 18, "selected_both_wrong": 10}, {"budget": 0.07, "threshold": -0.7790470719337463, "operator": "ge", "actual_fraction": 0.07, "metric": 0.95, "rescued": 2, "harmed": 49, "missed_rescue": 0, "avoided_harm": 479, "selected_both_correct": 23, "selected_both_wrong": 10}, {"budget": 0.08, "threshold": -0.7927228808403015, "operator": "ge", "actual_fraction": 0.08, "metric": 0.945, "rescued": 2, "harmed": 55, "missed_rescue": 0, "avoided_harm": 473, "selected_both_correct": 28, "selected_both_wrong": 11}, {"budget": 0.09, "threshold": -0.8049252033233643, "operator": "ge", "actual_fraction": 0.09, "metric": 0.9408333333333333, "rescued": 2, "harmed": 60, "missed_rescue": 0, "avoided_harm": 468, "selected_both_correct": 35, "selected_both_wrong": 11}, {"budget": 0.1, "threshold": -0.80850750207901, "operator": "ge", "actual_fraction": 0.1, "metric": 0.9333333333333333, "rescued": 2, "harmed": 69, "missed_rescue": 0, "avoided_harm": 459, "selected_both_correct": 38, "selected_both_wrong": 11}, {"budget": 0.11, "threshold": -0.819871723651886, "operator": "ge", "actual_fraction": 0.11, "metric": 0.9258333333333333, "rescued": 2, "harmed": 78, "missed_rescue": 0, "avoided_harm": 450, "selected_both_correct": 41, "selected_both_wrong": 11}, {"budget": 0.12, "threshold": -0.8310170769691467, "operator": "ge", "actual_fraction": 0.12, "metric": 0.9225, "rescued": 2, "harmed": 82, "missed_rescue": 0, "avoided_harm": 446, "selected_both_correct": 49, "selected_both_wrong": 11}, {"budget": 0.13, "threshold": -0.8376906514167786, "operator": "ge", "actual_fraction": 0.13, "metric": 0.9183333333333333, "rescued": 2, "harmed": 87, "missed_rescue": 0, "avoided_harm": 441, "selected_both_correct": 56, "selected_both_wrong": 11}, {"budget": 0.14, "threshold": -0.8419997692108154, "operator": "ge", "actual_fraction": 0.14, "metric": 0.9108333333333334, "rescued": 2, "harmed": 96, "missed_rescue": 0, "avoided_harm": 432, "selected_both_correct": 59, "selected_both_wrong": 11}, {"budget": 0.15, "threshold": -0.8447275161743164, "operator": "ge", "actual_fraction": 0.15, "metric": 0.9058333333333334, "rescued": 2, "harmed": 102, "missed_rescue": 0, "avoided_harm": 426, "selected_both_correct": 65, "selected_both_wrong": 11}, {"budget": 0.16, "threshold": -0.8506776690483093, "operator": "ge", "actual_fraction": 0.16, "metric": 0.9016666666666666, "rescued": 2, "harmed": 107, "missed_rescue": 0, "avoided_harm": 421, "selected_both_correct": 72, "selected_both_wrong": 11}, {"budget": 0.17, "threshold": -0.8545639514923096, "operator": "ge", "actual_fraction": 0.17, "metric": 0.8966666666666666, "rescued": 2, "harmed": 113, "missed_rescue": 0, "avoided_harm": 415, "selected_both_correct": 78, "selected_both_wrong": 11}, {"budget": 0.18, "threshold": -0.8578683137893677, "operator": "ge", "actual_fraction": 0.18, "metric": 0.8916666666666667, "rescued": 2, "harmed": 119, "missed_rescue": 0, "avoided_harm": 409, "selected_both_correct": 84, "selected_both_wrong": 11}, {"budget": 0.19, "threshold": -0.86090087890625, "operator": "ge", "actual_fraction": 0.19, "metric": 0.8866666666666667, "rescued": 2, "harmed": 125, "missed_rescue": 0, "avoided_harm": 403, "selected_both_correct": 90, "selected_both_wrong": 11}, {"budget": 0.2, "threshold": -0.8640955090522766, "operator": "ge", "actual_fraction": 0.2, "metric": 0.8841666666666667, "rescued": 2, "harmed": 128, "missed_rescue": 0, "avoided_harm": 400, "selected_both_correct": 99, "selected_both_wrong": 11}, {"budget": 0.21, "threshold": -0.8664237260818481, "operator": "ge", "actual_fraction": 0.21, "metric": 0.8791666666666667, "rescued": 2, "harmed": 134, "missed_rescue": 0, "avoided_harm": 394, "selected_both_correct": 105, "selected_both_wrong": 11}, {"budget": 0.22, "threshold": -0.8686786890029907, "operator": "ge", "actual_fraction": 0.22, "metric": 0.8741666666666666, "rescued": 2, "harmed": 140, "missed_rescue": 0, "avoided_harm": 388, "selected_both_correct": 111, "selected_both_wrong": 11}, {"budget": 0.23, "threshold": -0.8709638118743896, "operator": "ge", "actual_fraction": 0.23, "metric": 0.8683333333333333, "rescued": 2, "harmed": 147, "missed_rescue": 0, "avoided_harm": 381, "selected_both_correct": 116, "selected_both_wrong": 11}, {"budget": 0.24, "threshold": -0.8724074959754944, "operator": "ge", "actual_fraction": 0.24, "metric": 0.8625, "rescued": 2, "harmed": 154, "missed_rescue": 0, "avoided_harm": 374, "selected_both_correct": 121, "selected_both_wrong": 11}, {"budget": 0.25, "threshold": -0.8736594319343567, "operator": "ge", "actual_fraction": 0.25, "metric": 0.8575, "rescued": 2, "harmed": 160, "missed_rescue": 0, "avoided_harm": 368, "selected_both_correct": 127, "selected_both_wrong": 11}, {"budget": 0.26, "threshold": -0.8752554059028625, "operator": "ge", "actual_fraction": 0.26, "metric": 0.8533333333333334, "rescued": 2, "harmed": 165, "missed_rescue": 0, "avoided_harm": 363, "selected_both_correct": 134, "selected_both_wrong": 11}, {"budget": 0.27, "threshold": -0.8766024112701416, "operator": "ge", "actual_fraction": 0.27, "metric": 0.8508333333333333, "rescued": 2, "harmed": 168, "missed_rescue": 0, "avoided_harm": 360, "selected_both_correct": 143, "selected_both_wrong": 11}, {"budget": 0.28, "threshold": -0.878311276435852, "operator": "ge", "actual_fraction": 0.28, "metric": 0.8475, "rescued": 2, "harmed": 172, "missed_rescue": 0, "avoided_harm": 356, "selected_both_correct": 151, "selected_both_wrong": 11}, {"budget": 0.29, "threshold": -0.880103588104248, "operator": "ge", "actual_fraction": 0.29, "metric": 0.8425, "rescued": 2, "harmed": 178, "missed_rescue": 0, "avoided_harm": 350, "selected_both_correct": 157, "selected_both_wrong": 11}, {"budget": 0.3, "threshold": -0.8824952244758606, "operator": "ge", "actual_fraction": 0.3, "metric": 0.8375, "rescued": 2, "harmed": 184, "missed_rescue": 0, "avoided_harm": 344, "selected_both_correct": 163, "selected_both_wrong": 11}, {"budget": 0.31, "threshold": -0.8831692337989807, "operator": "ge", "actual_fraction": 0.31, "metric": 0.8358333333333333, "rescued": 2, "harmed": 186, "missed_rescue": 0, "avoided_harm": 342, "selected_both_correct": 173, "selected_both_wrong": 11}, {"budget": 0.32, "threshold": -0.8845130205154419, "operator": "ge", "actual_fraction": 0.32, "metric": 0.8316666666666667, "rescued": 2, "harmed": 191, "missed_rescue": 0, "avoided_harm": 337, "selected_both_correct": 180, "selected_both_wrong": 11}, {"budget": 0.33, "threshold": -0.8855781555175781, "operator": "ge", "actual_fraction": 0.33, "metric": 0.8291666666666667, "rescued": 2, "harmed": 194, "missed_rescue": 0, "avoided_harm": 334, "selected_both_correct": 189, "selected_both_wrong": 11}, {"budget": 0.34, "threshold": -0.8865919709205627, "operator": "ge", "actual_fraction": 0.34, "metric": 0.8266666666666667, "rescued": 2, "harmed": 197, "missed_rescue": 0, "avoided_harm": 331, "selected_both_correct": 198, "selected_both_wrong": 11}, {"budget": 0.35, "threshold": -0.8877584338188171, "operator": "ge", "actual_fraction": 0.35, "metric": 0.8208333333333333, "rescued": 2, "harmed": 204, "missed_rescue": 0, "avoided_harm": 324, "selected_both_correct": 203, "selected_both_wrong": 11}, {"budget": 0.36, "threshold": -0.8890330195426941, "operator": "ge", "actual_fraction": 0.36, "metric": 0.8166666666666667, "rescued": 2, "harmed": 209, "missed_rescue": 0, "avoided_harm": 319, "selected_both_correct": 210, "selected_both_wrong": 11}, {"budget": 0.37, "threshold": -0.890850305557251, "operator": "ge", "actual_fraction": 0.37, "metric": 0.815, "rescued": 2, "harmed": 211, "missed_rescue": 0, "avoided_harm": 317, "selected_both_correct": 220, "selected_both_wrong": 11}, {"budget": 0.38, "threshold": -0.8916605710983276, "operator": "ge", "actual_fraction": 0.38, "metric": 0.8125, "rescued": 2, "harmed": 214, "missed_rescue": 0, "avoided_harm": 314, "selected_both_correct": 229, "selected_both_wrong": 11}, {"budget": 0.39, "threshold": -0.8928589224815369, "operator": "ge", "actual_fraction": 0.39, "metric": 0.8066666666666666, "rescued": 2, "harmed": 221, "missed_rescue": 0, "avoided_harm": 307, "selected_both_correct": 234, "selected_both_wrong": 11}, {"budget": 0.4, "threshold": -0.8940943479537964, "operator": "ge", "actual_fraction": 0.4, "metric": 0.8, "rescued": 2, "harmed": 229, "missed_rescue": 0, "avoided_harm": 299, "selected_both_correct": 238, "selected_both_wrong": 11}, {"budget": 0.41, "threshold": -0.8949463963508606, "operator": "ge", "actual_fraction": 0.4091666666666667, "metric": 0.7991666666666667, "rescued": 2, "harmed": 230, "missed_rescue": 0, "avoided_harm": 298, "selected_both_correct": 248, "selected_both_wrong": 11}, {"budget": 0.42, "threshold": -0.8960229158401489, "operator": "ge", "actual_fraction": 0.42, "metric": 0.7941666666666667, "rescued": 2, "harmed": 236, "missed_rescue": 0, "avoided_harm": 292, "selected_both_correct": 255, "selected_both_wrong": 11}, {"budget": 0.43, "threshold": -0.8966536521911621, "operator": "ge", "actual_fraction": 0.43, "metric": 0.7916666666666666, "rescued": 2, "harmed": 239, "missed_rescue": 0, "avoided_harm": 289, "selected_both_correct": 264, "selected_both_wrong": 11}, {"budget": 0.44, "threshold": -0.8975363373756409, "operator": "ge", "actual_fraction": 0.44, "metric": 0.7891666666666667, "rescued": 2, "harmed": 242, "missed_rescue": 0, "avoided_harm": 286, "selected_both_correct": 273, "selected_both_wrong": 11}, {"budget": 0.45, "threshold": -0.8988122940063477, "operator": "ge", "actual_fraction": 0.45, "metric": 0.7875, "rescued": 2, "harmed": 244, "missed_rescue": 0, "avoided_harm": 284, "selected_both_correct": 283, "selected_both_wrong": 11}, {"budget": 0.46, "threshold": -0.9000959396362305, "operator": "ge", "actual_fraction": 0.46, "metric": 0.7816666666666666, "rescued": 2, "harmed": 251, "missed_rescue": 0, "avoided_harm": 277, "selected_both_correct": 288, "selected_both_wrong": 11}, {"budget": 0.47, "threshold": -0.9013051986694336, "operator": "ge", "actual_fraction": 0.47, "metric": 0.7791666666666667, "rescued": 2, "harmed": 254, "missed_rescue": 0, "avoided_harm": 274, "selected_both_correct": 297, "selected_both_wrong": 11}, {"budget": 0.48, "threshold": -0.9029324054718018, "operator": "ge", "actual_fraction": 0.48, "metric": 0.775, "rescued": 2, "harmed": 259, "missed_rescue": 0, "avoided_harm": 269, "selected_both_correct": 304, "selected_both_wrong": 11}, {"budget": 0.49, "threshold": -0.9036871790885925, "operator": "ge", "actual_fraction": 0.49, "metric": 0.7733333333333333, "rescued": 2, "harmed": 261, "missed_rescue": 0, "avoided_harm": 267, "selected_both_correct": 314, "selected_both_wrong": 11}, {"budget": 0.5, "threshold": -0.9044621586799622, "operator": "ge", "actual_fraction": 0.5, "metric": 0.77, "rescued": 2, "harmed": 265, "missed_rescue": 0, "avoided_harm": 263, "selected_both_correct": 322, "selected_both_wrong": 11}, {"budget": 0.51, "threshold": -0.9054575562477112, "operator": "ge", "actual_fraction": 0.51, "metric": 0.7666666666666667, "rescued": 2, "harmed": 269, "missed_rescue": 0, "avoided_harm": 259, "selected_both_correct": 330, "selected_both_wrong": 11}, {"budget": 0.52, "threshold": -0.906682550907135, "operator": "ge", "actual_fraction": 0.52, "metric": 0.7616666666666667, "rescued": 2, "harmed": 275, "missed_rescue": 0, "avoided_harm": 253, "selected_both_correct": 336, "selected_both_wrong": 11}, {"budget": 0.53, "threshold": -0.9072114825248718, "operator": "ge", "actual_fraction": 0.53, "metric": 0.7575, "rescued": 2, "harmed": 280, "missed_rescue": 0, "avoided_harm": 248, "selected_both_correct": 343, "selected_both_wrong": 11}, {"budget": 0.54, "threshold": -0.9083563089370728, "operator": "ge", "actual_fraction": 0.54, "metric": 0.7533333333333333, "rescued": 2, "harmed": 285, "missed_rescue": 0, "avoided_harm": 243, "selected_both_correct": 350, "selected_both_wrong": 11}, {"budget": 0.55, "threshold": -0.9095192551612854, "operator": "ge", "actual_fraction": 0.55, "metric": 0.75, "rescued": 2, "harmed": 289, "missed_rescue": 0, "avoided_harm": 239, "selected_both_correct": 358, "selected_both_wrong": 11}, {"budget": 0.56, "threshold": -0.9106979370117188, "operator": "ge", "actual_fraction": 0.56, "metric": 0.7466666666666667, "rescued": 2, "harmed": 293, "missed_rescue": 0, "avoided_harm": 235, "selected_both_correct": 366, "selected_both_wrong": 11}, {"budget": 0.57, "threshold": -0.9115362167358398, "operator": "ge", "actual_fraction": 0.5691666666666667, "metric": 0.7425, "rescued": 2, "harmed": 298, "missed_rescue": 0, "avoided_harm": 230, "selected_both_correct": 372, "selected_both_wrong": 11}, {"budget": 0.58, "threshold": -0.9124338030815125, "operator": "ge", "actual_fraction": 0.58, "metric": 0.7375, "rescued": 2, "harmed": 304, "missed_rescue": 0, "avoided_harm": 224, "selected_both_correct": 379, "selected_both_wrong": 11}, {"budget": 0.59, "threshold": -0.9134801626205444, "operator": "ge", "actual_fraction": 0.59, "metric": 0.735, "rescued": 2, "harmed": 307, "missed_rescue": 0, "avoided_harm": 221, "selected_both_correct": 388, "selected_both_wrong": 11}, {"budget": 0.6, "threshold": -0.9143747091293335, "operator": "ge", "actual_fraction": 0.6, "metric": 0.7316666666666667, "rescued": 2, "harmed": 311, "missed_rescue": 0, "avoided_harm": 217, "selected_both_correct": 396, "selected_both_wrong": 11}, {"budget": 0.61, "threshold": -0.915804386138916, "operator": "ge", "actual_fraction": 0.61, "metric": 0.7283333333333334, "rescued": 2, "harmed": 315, "missed_rescue": 0, "avoided_harm": 213, "selected_both_correct": 404, "selected_both_wrong": 11}, {"budget": 0.62, "threshold": -0.9165146350860596, "operator": "ge", "actual_fraction": 0.62, "metric": 0.7266666666666667, "rescued": 2, "harmed": 317, "missed_rescue": 0, "avoided_harm": 211, "selected_both_correct": 414, "selected_both_wrong": 11}, {"budget": 0.63, "threshold": -0.9172822833061218, "operator": "ge", "actual_fraction": 0.63, "metric": 0.725, "rescued": 2, "harmed": 319, "missed_rescue": 0, "avoided_harm": 209, "selected_both_correct": 424, "selected_both_wrong": 11}, {"budget": 0.64, "threshold": -0.9181071519851685, "operator": "ge", "actual_fraction": 0.64, "metric": 0.7208333333333333, "rescued": 2, "harmed": 324, "missed_rescue": 0, "avoided_harm": 204, "selected_both_correct": 431, "selected_both_wrong": 11}, {"budget": 0.65, "threshold": -0.9192029237747192, "operator": "ge", "actual_fraction": 0.65, "metric": 0.7175, "rescued": 2, "harmed": 328, "missed_rescue": 0, "avoided_harm": 200, "selected_both_correct": 439, "selected_both_wrong": 11}, {"budget": 0.66, "threshold": -0.9200543761253357, "operator": "ge", "actual_fraction": 0.66, "metric": 0.7133333333333334, "rescued": 2, "harmed": 333, "missed_rescue": 0, "avoided_harm": 195, "selected_both_correct": 446, "selected_both_wrong": 11}, {"budget": 0.67, "threshold": -0.9208071231842041, "operator": "ge", "actual_fraction": 0.67, "metric": 0.71, "rescued": 2, "harmed": 337, "missed_rescue": 0, "avoided_harm": 191, "selected_both_correct": 454, "selected_both_wrong": 11}, {"budget": 0.68, "threshold": -0.9215553998947144, "operator": "ge", "actual_fraction": 0.68, "metric": 0.7058333333333333, "rescued": 2, "harmed": 342, "missed_rescue": 0, "avoided_harm": 186, "selected_both_correct": 461, "selected_both_wrong": 11}, {"budget": 0.69, "threshold": -0.9220835566520691, "operator": "ge", "actual_fraction": 0.6891666666666667, "metric": 0.7016666666666667, "rescued": 2, "harmed": 347, "missed_rescue": 0, "avoided_harm": 181, "selected_both_correct": 467, "selected_both_wrong": 11}, {"budget": 0.7, "threshold": -0.9232237935066223, "operator": "ge", "actual_fraction": 0.7, "metric": 0.6975, "rescued": 2, "harmed": 352, "missed_rescue": 0, "avoided_harm": 176, "selected_both_correct": 475, "selected_both_wrong": 11}, {"budget": 0.71, "threshold": -0.9243364334106445, "operator": "ge", "actual_fraction": 0.71, "metric": 0.6933333333333334, "rescued": 2, "harmed": 357, "missed_rescue": 0, "avoided_harm": 171, "selected_both_correct": 482, "selected_both_wrong": 11}, {"budget": 0.72, "threshold": -0.9252541065216064, "operator": "ge", "actual_fraction": 0.72, "metric": 0.69, "rescued": 2, "harmed": 361, "missed_rescue": 0, "avoided_harm": 167, "selected_both_correct": 490, "selected_both_wrong": 11}, {"budget": 0.73, "threshold": -0.925781786441803, "operator": "ge", "actual_fraction": 0.73, "metric": 0.6866666666666666, "rescued": 2, "harmed": 365, "missed_rescue": 0, "avoided_harm": 163, "selected_both_correct": 498, "selected_both_wrong": 11}, {"budget": 0.74, "threshold": -0.9274963140487671, "operator": "ge", "actual_fraction": 0.74, "metric": 0.6833333333333333, "rescued": 2, "harmed": 369, "missed_rescue": 0, "avoided_harm": 159, "selected_both_correct": 506, "selected_both_wrong": 11}, {"budget": 0.75, "threshold": -0.9286423325538635, "operator": "ge", "actual_fraction": 0.75, "metric": 0.6783333333333333, "rescued": 2, "harmed": 375, "missed_rescue": 0, "avoided_harm": 153, "selected_both_correct": 512, "selected_both_wrong": 11}, {"budget": 0.76, "threshold": -0.9301291704177856, "operator": "ge", "actual_fraction": 0.76, "metric": 0.6741666666666667, "rescued": 2, "harmed": 380, "missed_rescue": 0, "avoided_harm": 148, "selected_both_correct": 519, "selected_both_wrong": 11}, {"budget": 0.77, "threshold": -0.9318777918815613, "operator": "ge", "actual_fraction": 0.77, "metric": 0.67, "rescued": 2, "harmed": 385, "missed_rescue": 0, "avoided_harm": 143, "selected_both_correct": 526, "selected_both_wrong": 11}, {"budget": 0.78, "threshold": -0.9332055449485779, "operator": "ge", "actual_fraction": 0.78, "metric": 0.665, "rescued": 2, "harmed": 391, "missed_rescue": 0, "avoided_harm": 137, "selected_both_correct": 532, "selected_both_wrong": 11}, {"budget": 0.79, "threshold": -0.9344666600227356, "operator": "ge", "actual_fraction": 0.79, "metric": 0.6616666666666666, "rescued": 2, "harmed": 395, "missed_rescue": 0, "avoided_harm": 133, "selected_both_correct": 540, "selected_both_wrong": 11}, {"budget": 0.8, "threshold": -0.9362465143203735, "operator": "ge", "actual_fraction": 0.8, "metric": 0.6583333333333333, "rescued": 2, "harmed": 399, "missed_rescue": 0, "avoided_harm": 129, "selected_both_correct": 548, "selected_both_wrong": 11}, {"budget": 0.81, "threshold": -0.937842071056366, "operator": "ge", "actual_fraction": 0.81, "metric": 0.6533333333333333, "rescued": 2, "harmed": 405, "missed_rescue": 0, "avoided_harm": 123, "selected_both_correct": 554, "selected_both_wrong": 11}, {"budget": 0.82, "threshold": -0.9384457468986511, "operator": "ge", "actual_fraction": 0.8191666666666667, "metric": 0.65, "rescued": 2, "harmed": 409, "missed_rescue": 0, "avoided_harm": 119, "selected_both_correct": 561, "selected_both_wrong": 11}, {"budget": 0.83, "threshold": -0.9396296143531799, "operator": "ge", "actual_fraction": 0.83, "metric": 0.6458333333333334, "rescued": 2, "harmed": 414, "missed_rescue": 0, "avoided_harm": 114, "selected_both_correct": 569, "selected_both_wrong": 11}, {"budget": 0.84, "threshold": -0.9407488703727722, "operator": "ge", "actual_fraction": 0.84, "metric": 0.6383333333333333, "rescued": 2, "harmed": 423, "missed_rescue": 0, "avoided_harm": 105, "selected_both_correct": 572, "selected_both_wrong": 11}, {"budget": 0.85, "threshold": -0.942738950252533, "operator": "ge", "actual_fraction": 0.85, "metric": 0.6358333333333334, "rescued": 2, "harmed": 426, "missed_rescue": 0, "avoided_harm": 102, "selected_both_correct": 581, "selected_both_wrong": 11}, {"budget": 0.86, "threshold": -0.9438527226448059, "operator": "gt", "actual_fraction": 0.8591666666666666, "metric": 0.6325, "rescued": 2, "harmed": 430, "missed_rescue": 0, "avoided_harm": 98, "selected_both_correct": 588, "selected_both_wrong": 11}, {"budget": 0.87, "threshold": -0.9450756907463074, "operator": "ge", "actual_fraction": 0.87, "metric": 0.6308333333333334, "rescued": 2, "harmed": 432, "missed_rescue": 0, "avoided_harm": 96, "selected_both_correct": 599, "selected_both_wrong": 11}, {"budget": 0.88, "threshold": -0.9481598138809204, "operator": "ge", "actual_fraction": 0.88, "metric": 0.625, "rescued": 2, "harmed": 439, "missed_rescue": 0, "avoided_harm": 89, "selected_both_correct": 604, "selected_both_wrong": 11}, {"budget": 0.89, "threshold": -0.9500836730003357, "operator": "ge", "actual_fraction": 0.89, "metric": 0.62, "rescued": 2, "harmed": 445, "missed_rescue": 0, "avoided_harm": 83, "selected_both_correct": 610, "selected_both_wrong": 11}, {"budget": 0.9, "threshold": -0.9519118666648865, "operator": "ge", "actual_fraction": 0.9, "metric": 0.6133333333333333, "rescued": 2, "harmed": 453, "missed_rescue": 0, "avoided_harm": 75, "selected_both_correct": 614, "selected_both_wrong": 11}, {"budget": 0.91, "threshold": -0.9543898701667786, "operator": "ge", "actual_fraction": 0.91, "metric": 0.6066666666666667, "rescued": 2, "harmed": 461, "missed_rescue": 0, "avoided_harm": 67, "selected_both_correct": 618, "selected_both_wrong": 11}, {"budget": 0.92, "threshold": -0.9570822715759277, "operator": "ge", "actual_fraction": 0.92, "metric": 0.6008333333333333, "rescued": 2, "harmed": 468, "missed_rescue": 0, "avoided_harm": 60, "selected_both_correct": 623, "selected_both_wrong": 11}, {"budget": 0.93, "threshold": -0.9593446254730225, "operator": "ge", "actual_fraction": 0.93, "metric": 0.5941666666666666, "rescued": 2, "harmed": 476, "missed_rescue": 0, "avoided_harm": 52, "selected_both_correct": 627, "selected_both_wrong": 11}, {"budget": 0.94, "threshold": -0.9610995054244995, "operator": "ge", "actual_fraction": 0.94, "metric": 0.5883333333333334, "rescued": 2, "harmed": 483, "missed_rescue": 0, "avoided_harm": 45, "selected_both_correct": 632, "selected_both_wrong": 11}, {"budget": 0.95, "threshold": -0.9635111093521118, "operator": "ge", "actual_fraction": 0.95, "metric": 0.5825, "rescued": 2, "harmed": 490, "missed_rescue": 0, "avoided_harm": 38, "selected_both_correct": 637, "selected_both_wrong": 11}, {"budget": 0.96, "threshold": -0.9648061990737915, "operator": "ge", "actual_fraction": 0.96, "metric": 0.5775, "rescued": 2, "harmed": 496, "missed_rescue": 0, "avoided_harm": 32, "selected_both_correct": 643, "selected_both_wrong": 11}, {"budget": 0.97, "threshold": -0.9674390554428101, "operator": "ge", "actual_fraction": 0.97, "metric": 0.57, "rescued": 2, "harmed": 505, "missed_rescue": 0, "avoided_harm": 23, "selected_both_correct": 646, "selected_both_wrong": 11}, {"budget": 0.98, "threshold": -0.9724470973014832, "operator": "ge", "actual_fraction": 0.98, "metric": 0.5641666666666667, "rescued": 2, "harmed": 512, "missed_rescue": 0, "avoided_harm": 16, "selected_both_correct": 651, "selected_both_wrong": 11}, {"budget": 0.99, "threshold": -0.9774226546287537, "operator": "ge", "actual_fraction": 0.99, "metric": 0.5575, "rescued": 2, "harmed": 520, "missed_rescue": 0, "avoided_harm": 8, "selected_both_correct": 655, "selected_both_wrong": 11}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.5508333333333333, "rescued": 2, "harmed": 528, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 659, "selected_both_wrong": 11}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.9016223679668622, "rescued": 0, "harmed": 0, "missed_rescue": 144, "avoided_harm": 2192, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.48843976855278015, "operator": "ge", "actual_fraction": 0.08698653779772178, "metric": 0.8878149810148429, "rescued": 70, "harmed": 150, "missed_rescue": 74, "avoided_harm": 2042, "selected_both_correct": 85, "selected_both_wrong": 199}, {"budget": 0.02, "threshold": -0.6014261841773987, "operator": "ge", "actual_fraction": 0.13910942354159475, "metric": 0.8683120469451157, "rescued": 93, "harmed": 286, "missed_rescue": 51, "avoided_harm": 1906, "selected_both_correct": 168, "selected_both_wrong": 259}, {"budget": 0.03, "threshold": -0.6401472687721252, "operator": "ge", "actual_fraction": 0.15947531929582326, "metric": 0.8596824301001036, "rescued": 100, "harmed": 343, "missed_rescue": 44, "avoided_harm": 1849, "selected_both_correct": 203, "selected_both_wrong": 278}, {"budget": 0.04, "threshold": -0.6884650588035583, "operator": "ge", "actual_fraction": 0.1886434242319641, "metric": 0.845702450811184, "rescued": 106, "harmed": 430, "missed_rescue": 38, "avoided_harm": 1762, "selected_both_correct": 258, "selected_both_wrong": 299}, {"budget": 0.05, "threshold": -0.7208121418952942, "operator": "ge", "actual_fraction": 0.2143596824301001, "metric": 0.8329306178805661, "rescued": 111, "harmed": 509, "missed_rescue": 33, "avoided_harm": 1683, "selected_both_correct": 305, "selected_both_wrong": 317}, {"budget": 0.06, "threshold": -0.7578181624412537, "operator": "ge", "actual_fraction": 0.24525371073524335, "metric": 0.819986192613048, "rescued": 119, "harmed": 592, "missed_rescue": 25, "avoided_harm": 1600, "selected_both_correct": 384, "selected_both_wrong": 326}, {"budget": 0.07, "threshold": -0.7790470719337463, "operator": "ge", "actual_fraction": 0.26734552985847426, "metric": 0.8087676907145323, "rescued": 122, "harmed": 660, "missed_rescue": 22, "avoided_harm": 1532, "selected_both_correct": 430, "selected_both_wrong": 337}, {"budget": 0.08, "threshold": -0.7927228808403015, "operator": "ge", "actual_fraction": 0.2827062478425958, "metric": 0.8015188125647221, "rescued": 125, "harmed": 705, "missed_rescue": 19, "avoided_harm": 1487, "selected_both_correct": 468, "selected_both_wrong": 340}, {"budget": 0.09, "threshold": -0.8049252033233643, "operator": "ge", "actual_fraction": 0.2999654815326199, "metric": 0.7951328960994132, "rescued": 127, "harmed": 744, "missed_rescue": 17, "avoided_harm": 1448, "selected_both_correct": 521, "selected_both_wrong": 346}, {"budget": 0.1, "threshold": -0.80850750207901, "operator": "ge", "actual_fraction": 0.30531584397652745, "metric": 0.793579565067311, "rescued": 127, "harmed": 753, "missed_rescue": 17, "avoided_harm": 1439, "selected_both_correct": 541, "selected_both_wrong": 348}, {"budget": 0.11, "threshold": -0.819871723651886, "operator": "ge", "actual_fraction": 0.3258543320676562, "metric": 0.7839143942008975, "rescued": 129, "harmed": 811, "missed_rescue": 15, "avoided_harm": 1381, "selected_both_correct": 593, "selected_both_wrong": 355}, {"budget": 0.12, "threshold": -0.8310170769691467, "operator": "ge", "actual_fraction": 0.35053503624439075, "metric": 0.7733862616499827, "rescued": 132, "harmed": 875, "missed_rescue": 12, "avoided_harm": 1317, "selected_both_correct": 662, "selected_both_wrong": 362}, {"budget": 0.13, "threshold": -0.8376906514167786, "operator": "ge", "actual_fraction": 0.36796686227131514, "metric": 0.7654470141525717, "rescued": 133, "harmed": 922, "missed_rescue": 11, "avoided_harm": 1270, "selected_both_correct": 712, "selected_both_wrong": 365}, {"budget": 0.14, "threshold": -0.8419997692108154, "operator": "ge", "actual_fraction": 0.3798757335174318, "metric": 0.7609596133931653, "rescued": 133, "harmed": 948, "missed_rescue": 11, "avoided_harm": 1244, "selected_both_correct": 753, "selected_both_wrong": 367}, {"budget": 0.15, "threshold": -0.8447275161743164, "operator": "ge", "actual_fraction": 0.3867794269934415, "metric": 0.7581981360027614, "rescued": 133, "harmed": 964, "missed_rescue": 11, "avoided_harm": 1228, "selected_both_correct": 777, "selected_both_wrong": 367}, {"budget": 0.16, "threshold": -0.8506776690483093, "operator": "ge", "actual_fraction": 0.4050742147048671, "metric": 0.7509492578529513, "rescued": 135, "harmed": 1008, "missed_rescue": 9, "avoided_harm": 1184, "selected_both_correct": 835, "selected_both_wrong": 369}, {"budget": 0.17, "threshold": -0.8545639514923096, "operator": "ge", "actual_fraction": 0.4175008629616845, "metric": 0.7462892647566448, "rescued": 135, "harmed": 1035, "missed_rescue": 9, "avoided_harm": 1157, "selected_both_correct": 876, "selected_both_wrong": 373}, {"budget": 0.18, "threshold": -0.8578683137893677, "operator": "ge", "actual_fraction": 0.4290645495340007, "metric": 0.74283741801864, "rescued": 136, "harmed": 1056, "missed_rescue": 8, "avoided_harm": 1136, "selected_both_correct": 920, "selected_both_wrong": 374}, {"budget": 0.19, "threshold": -0.86090087890625, "operator": "ge", "actual_fraction": 0.43993786675871593, "metric": 0.7399033482913359, "rescued": 137, "harmed": 1074, "missed_rescue": 7, "avoided_harm": 1118, "selected_both_correct": 962, "selected_both_wrong": 376}, {"budget": 0.2, "threshold": -0.8640955090522766, "operator": "ge", "actual_fraction": 0.4520193303417328, "metric": 0.7347255781843286, "rescued": 138, "harmed": 1105, "missed_rescue": 6, "avoided_harm": 1087, "selected_both_correct": 1000, "selected_both_wrong": 376}, {"budget": 0.21, "threshold": -0.8664237260818481, "operator": "ge", "actual_fraction": 0.46047635484984467, "metric": 0.731446323783224, "rescued": 138, "harmed": 1124, "missed_rescue": 6, "avoided_harm": 1068, "selected_both_correct": 1028, "selected_both_wrong": 378}, {"budget": 0.22, "threshold": -0.8686786890029907, "operator": "ge", "actual_fraction": 0.46945115636865725, "metric": 0.7279944770452192, "rescued": 138, "harmed": 1144, "missed_rescue": 6, "avoided_harm": 1048, "selected_both_correct": 1060, "selected_both_wrong": 378}, {"budget": 0.23, "threshold": -0.8709638118743896, "operator": "ge", "actual_fraction": 0.48066965826717295, "metric": 0.7241974456334139, "rescued": 138, "harmed": 1166, "missed_rescue": 6, "avoided_harm": 1026, "selected_both_correct": 1102, "selected_both_wrong": 379}, {"budget": 0.24, "threshold": -0.8724074959754944, "operator": "ge", "actual_fraction": 0.48912668277528476, "metric": 0.7210907835692095, "rescued": 138, "harmed": 1184, "missed_rescue": 6, "avoided_harm": 1008, "selected_both_correct": 1132, "selected_both_wrong": 380}, {"budget": 0.25, "threshold": -0.8736594319343567, "operator": "ge", "actual_fraction": 0.49413186054539177, "metric": 0.7200552295478081, "rescued": 138, "harmed": 1190, "missed_rescue": 6, "avoided_harm": 1002, "selected_both_correct": 1154, "selected_both_wrong": 381}, {"budget": 0.26, "threshold": -0.8752554059028625, "operator": "ge", "actual_fraction": 0.5005177770107008, "metric": 0.717811529168105, "rescued": 138, "harmed": 1203, "missed_rescue": 6, "avoided_harm": 989, "selected_both_correct": 1178, "selected_both_wrong": 381}, {"budget": 0.27, "threshold": -0.8766024112701416, "operator": "ge", "actual_fraction": 0.5081118398343114, "metric": 0.7153952364515016, "rescued": 138, "harmed": 1217, "missed_rescue": 6, "avoided_harm": 975, "selected_both_correct": 1207, "selected_both_wrong": 382}, {"budget": 0.28, "threshold": -0.878311276435852, "operator": "ge", "actual_fraction": 0.5169140490162237, "metric": 0.7131515360717984, "rescued": 138, "harmed": 1230, "missed_rescue": 6, "avoided_harm": 962, "selected_both_correct": 1243, "selected_both_wrong": 384}, {"budget": 0.29, "threshold": -0.880103588104248, "operator": "ge", "actual_fraction": 0.5277873662409389, "metric": 0.7105626510182947, "rescued": 138, "harmed": 1245, "missed_rescue": 6, "avoided_harm": 947, "selected_both_correct": 1290, "selected_both_wrong": 385}, {"budget": 0.3, "threshold": -0.8824952244758606, "operator": "ge", "actual_fraction": 0.5419399378667588, "metric": 0.704349326889886, "rescued": 138, "harmed": 1281, "missed_rescue": 6, "avoided_harm": 911, "selected_both_correct": 1336, "selected_both_wrong": 385}, {"budget": 0.31, "threshold": -0.8831692337989807, "operator": "ge", "actual_fraction": 0.5453917846047636, "metric": 0.7034863652053849, "rescued": 138, "harmed": 1286, "missed_rescue": 6, "avoided_harm": 906, "selected_both_correct": 1350, "selected_both_wrong": 386}, {"budget": 0.32, "threshold": -0.8845130205154419, "operator": "ge", "actual_fraction": 0.5566102865032793, "metric": 0.7003797031411805, "rescued": 138, "harmed": 1304, "missed_rescue": 6, "avoided_harm": 888, "selected_both_correct": 1394, "selected_both_wrong": 389}, {"budget": 0.33, "threshold": -0.8855781555175781, "operator": "ge", "actual_fraction": 0.5633413876423887, "metric": 0.6976182257507767, "rescued": 138, "harmed": 1320, "missed_rescue": 6, "avoided_harm": 872, "selected_both_correct": 1417, "selected_both_wrong": 389}, {"budget": 0.34, "threshold": -0.8865919709205627, "operator": "ge", "actual_fraction": 0.5716258198136003, "metric": 0.6958923023817742, "rescued": 138, "harmed": 1330, "missed_rescue": 6, "avoided_harm": 862, "selected_both_correct": 1454, "selected_both_wrong": 390}, {"budget": 0.35, "threshold": -0.8877584338188171, "operator": "ge", "actual_fraction": 0.5819813600276148, "metric": 0.6929582326544701, "rescued": 139, "harmed": 1348, "missed_rescue": 5, "avoided_harm": 844, "selected_both_correct": 1494, "selected_both_wrong": 391}, {"budget": 0.36, "threshold": -0.8890330195426941, "operator": "ge", "actual_fraction": 0.5902657921988264, "metric": 0.6910597169485675, "rescued": 139, "harmed": 1359, "missed_rescue": 5, "avoided_harm": 833, "selected_both_correct": 1529, "selected_both_wrong": 393}, {"budget": 0.37, "threshold": -0.890850305557251, "operator": "ge", "actual_fraction": 0.6047635484984467, "metric": 0.6853641698308595, "rescued": 139, "harmed": 1392, "missed_rescue": 5, "avoided_harm": 800, "selected_both_correct": 1580, "selected_both_wrong": 393}, {"budget": 0.38, "threshold": -0.8916605710983276, "operator": "ge", "actual_fraction": 0.6113220573006558, "metric": 0.6834656541249569, "rescued": 139, "harmed": 1403, "missed_rescue": 5, "avoided_harm": 789, "selected_both_correct": 1607, "selected_both_wrong": 393}, {"budget": 0.39, "threshold": -0.8928589224815369, "operator": "ge", "actual_fraction": 0.6206420434932689, "metric": 0.681394546082154, "rescued": 140, "harmed": 1416, "missed_rescue": 4, "avoided_harm": 776, "selected_both_correct": 1647, "selected_both_wrong": 393}, {"budget": 0.4, "threshold": -0.8940943479537964, "operator": "ge", "actual_fraction": 0.6289264756644805, "metric": 0.6789782533655506, "rescued": 140, "harmed": 1430, "missed_rescue": 4, "avoided_harm": 762, "selected_both_correct": 1681, "selected_both_wrong": 393}, {"budget": 0.41, "threshold": -0.8949463963508606, "operator": "ge", "actual_fraction": 0.6370383154987919, "metric": 0.6762167759751467, "rescued": 140, "harmed": 1446, "missed_rescue": 4, "avoided_harm": 746, "selected_both_correct": 1711, "selected_both_wrong": 394}, {"budget": 0.42, "threshold": -0.8960229158401489, "operator": "ge", "actual_fraction": 0.6435968243010011, "metric": 0.6739730755954436, "rescued": 140, "harmed": 1459, "missed_rescue": 4, "avoided_harm": 733, "selected_both_correct": 1734, "selected_both_wrong": 396}, {"budget": 0.43, "threshold": -0.8966536521911621, "operator": "ge", "actual_fraction": 0.6491197790818087, "metric": 0.6724197445633414, "rescued": 140, "harmed": 1468, "missed_rescue": 4, "avoided_harm": 724, "selected_both_correct": 1757, "selected_both_wrong": 396}, {"budget": 0.44, "threshold": -0.8975363373756409, "operator": "ge", "actual_fraction": 0.6558508802209182, "metric": 0.670003451846738, "rescued": 140, "harmed": 1482, "missed_rescue": 4, "avoided_harm": 710, "selected_both_correct": 1782, "selected_both_wrong": 396}, {"budget": 0.45, "threshold": -0.8988122940063477, "operator": "ge", "actual_fraction": 0.6641353123921298, "metric": 0.6681049361408353, "rescued": 140, "harmed": 1493, "missed_rescue": 4, "avoided_harm": 699, "selected_both_correct": 1819, "selected_both_wrong": 396}, {"budget": 0.46, "threshold": -0.9000959396362305, "operator": "ge", "actual_fraction": 0.6739730755954436, "metric": 0.6644804970659303, "rescued": 140, "harmed": 1514, "missed_rescue": 4, "avoided_harm": 678, "selected_both_correct": 1855, "selected_both_wrong": 396}, {"budget": 0.47, "threshold": -0.9013051986694336, "operator": "ge", "actual_fraction": 0.6848463928201588, "metric": 0.6596479116327235, "rescued": 141, "harmed": 1543, "missed_rescue": 3, "avoided_harm": 649, "selected_both_correct": 1887, "selected_both_wrong": 397}, {"budget": 0.48, "threshold": -0.9029324054718018, "operator": "ge", "actual_fraction": 0.696410079392475, "metric": 0.6575768035899207, "rescued": 141, "harmed": 1555, "missed_rescue": 3, "avoided_harm": 637, "selected_both_correct": 1941, "selected_both_wrong": 398}, {"budget": 0.49, "threshold": -0.9036871790885925, "operator": "ge", "actual_fraction": 0.7026234035208837, "metric": 0.6561960648947187, "rescued": 141, "harmed": 1563, "missed_rescue": 3, "avoided_harm": 629, "selected_both_correct": 1967, "selected_both_wrong": 400}, {"budget": 0.5, "threshold": -0.9044621586799622, "operator": "ge", "actual_fraction": 0.7088367276492924, "metric": 0.654297549188816, "rescued": 141, "harmed": 1574, "missed_rescue": 3, "avoided_harm": 618, "selected_both_correct": 1992, "selected_both_wrong": 400}, {"budget": 0.51, "threshold": -0.9054575562477112, "operator": "ge", "actual_fraction": 0.717121159820504, "metric": 0.6525716258198136, "rescued": 141, "harmed": 1584, "missed_rescue": 3, "avoided_harm": 608, "selected_both_correct": 2030, "selected_both_wrong": 400}, {"budget": 0.52, "threshold": -0.906682550907135, "operator": "ge", "actual_fraction": 0.7262685536762168, "metric": 0.64998274076631, "rescued": 141, "harmed": 1599, "missed_rescue": 3, "avoided_harm": 593, "selected_both_correct": 2068, "selected_both_wrong": 400}, {"budget": 0.53, "threshold": -0.9072114825248718, "operator": "ge", "actual_fraction": 0.7300655850880221, "metric": 0.6484294097342078, "rescued": 141, "harmed": 1608, "missed_rescue": 3, "avoided_harm": 584, "selected_both_correct": 2080, "selected_both_wrong": 401}, {"budget": 0.54, "threshold": -0.9083563089370728, "operator": "ge", "actual_fraction": 0.7381774249223334, "metric": 0.6460131170176044, "rescued": 141, "harmed": 1622, "missed_rescue": 3, "avoided_harm": 570, "selected_both_correct": 2112, "selected_both_wrong": 402}, {"budget": 0.55, "threshold": -0.9095192551612854, "operator": "ge", "actual_fraction": 0.7487055574732482, "metric": 0.6435968243010011, "rescued": 141, "harmed": 1636, "missed_rescue": 3, "avoided_harm": 556, "selected_both_correct": 2157, "selected_both_wrong": 404}, {"budget": 0.56, "threshold": -0.9106979370117188, "operator": "ge", "actual_fraction": 0.759233690024163, "metric": 0.6394546082153952, "rescued": 141, "harmed": 1660, "missed_rescue": 3, "avoided_harm": 532, "selected_both_correct": 2193, "selected_both_wrong": 405}, {"budget": 0.57, "threshold": -0.9115362167358398, "operator": "ge", "actual_fraction": 0.767863306869175, "metric": 0.6372109078356921, "rescued": 141, "harmed": 1673, "missed_rescue": 3, "avoided_harm": 519, "selected_both_correct": 2230, "selected_both_wrong": 405}, {"budget": 0.58, "threshold": -0.9124338030815125, "operator": "ge", "actual_fraction": 0.7751121850189852, "metric": 0.6339316534345875, "rescued": 141, "harmed": 1692, "missed_rescue": 3, "avoided_harm": 500, "selected_both_correct": 2253, "selected_both_wrong": 405}, {"budget": 0.59, "threshold": -0.9134801626205444, "operator": "ge", "actual_fraction": 0.7830514325163963, "metric": 0.6316879530548843, "rescued": 141, "harmed": 1705, "missed_rescue": 3, "avoided_harm": 487, "selected_both_correct": 2286, "selected_both_wrong": 405}, {"budget": 0.6, "threshold": -0.9143747091293335, "operator": "ge", "actual_fraction": 0.7904729030031067, "metric": 0.629271660338281, "rescued": 141, "harmed": 1719, "missed_rescue": 3, "avoided_harm": 473, "selected_both_correct": 2314, "selected_both_wrong": 406}, {"budget": 0.61, "threshold": -0.915804386138916, "operator": "ge", "actual_fraction": 0.8010010355540214, "metric": 0.6272005522954781, "rescued": 141, "harmed": 1731, "missed_rescue": 3, "avoided_harm": 461, "selected_both_correct": 2361, "selected_both_wrong": 408}, {"budget": 0.62, "threshold": -0.9165146350860596, "operator": "ge", "actual_fraction": 0.8063513979979289, "metric": 0.6253020365895754, "rescued": 141, "harmed": 1742, "missed_rescue": 3, "avoided_harm": 450, "selected_both_correct": 2381, "selected_both_wrong": 408}, {"budget": 0.63, "threshold": -0.9172822833061218, "operator": "ge", "actual_fraction": 0.8110113910942354, "metric": 0.6240938902312737, "rescued": 141, "harmed": 1749, "missed_rescue": 3, "avoided_harm": 443, "selected_both_correct": 2401, "selected_both_wrong": 408}, {"budget": 0.64, "threshold": -0.9181071519851685, "operator": "ge", "actual_fraction": 0.8172247152226441, "metric": 0.6220227821884708, "rescued": 141, "harmed": 1761, "missed_rescue": 3, "avoided_harm": 431, "selected_both_correct": 2424, "selected_both_wrong": 409}, {"budget": 0.65, "threshold": -0.9192029237747192, "operator": "ge", "actual_fraction": 0.824301001035554, "metric": 0.619951674145668, "rescued": 141, "harmed": 1773, "missed_rescue": 3, "avoided_harm": 419, "selected_both_correct": 2452, "selected_both_wrong": 410}, {"budget": 0.66, "threshold": -0.9200543761253357, "operator": "ge", "actual_fraction": 0.829306178805661, "metric": 0.618570935450466, "rescued": 141, "harmed": 1781, "missed_rescue": 3, "avoided_harm": 411, "selected_both_correct": 2473, "selected_both_wrong": 410}, {"budget": 0.67, "threshold": -0.9208071231842041, "operator": "ge", "actual_fraction": 0.8348291335864687, "metric": 0.6164998274076631, "rescued": 141, "harmed": 1793, "missed_rescue": 3, "avoided_harm": 399, "selected_both_correct": 2493, "selected_both_wrong": 410}, {"budget": 0.68, "threshold": -0.9215553998947144, "operator": "ge", "actual_fraction": 0.840006903693476, "metric": 0.6151190887124611, "rescued": 141, "harmed": 1801, "missed_rescue": 3, "avoided_harm": 391, "selected_both_correct": 2515, "selected_both_wrong": 410}, {"budget": 0.69, "threshold": -0.9220835566520691, "operator": "ge", "actual_fraction": 0.8443217121159821, "metric": 0.613565757680359, "rescued": 141, "harmed": 1810, "missed_rescue": 3, "avoided_harm": 382, "selected_both_correct": 2530, "selected_both_wrong": 411}, {"budget": 0.7, "threshold": -0.9232237935066223, "operator": "ge", "actual_fraction": 0.852088367276493, "metric": 0.6113220573006558, "rescued": 141, "harmed": 1823, "missed_rescue": 3, "avoided_harm": 369, "selected_both_correct": 2560, "selected_both_wrong": 413}, {"budget": 0.71, "threshold": -0.9243364334106445, "operator": "ge", "actual_fraction": 0.8598550224370038, "metric": 0.6082153952364515, "rescued": 141, "harmed": 1841, "missed_rescue": 3, "avoided_harm": 351, "selected_both_correct": 2585, "selected_both_wrong": 415}, {"budget": 0.72, "threshold": -0.9252541065216064, "operator": "ge", "actual_fraction": 0.8641698308595098, "metric": 0.6071798412150501, "rescued": 141, "harmed": 1847, "missed_rescue": 3, "avoided_harm": 345, "selected_both_correct": 2604, "selected_both_wrong": 415}, {"budget": 0.73, "threshold": -0.925781786441803, "operator": "ge", "actual_fraction": 0.8681394546082154, "metric": 0.6057991025198481, "rescued": 141, "harmed": 1855, "missed_rescue": 3, "avoided_harm": 337, "selected_both_correct": 2619, "selected_both_wrong": 415}, {"budget": 0.74, "threshold": -0.9274963140487671, "operator": "ge", "actual_fraction": 0.877114256127028, "metric": 0.6018294787711426, "rescued": 141, "harmed": 1878, "missed_rescue": 3, "avoided_harm": 314, "selected_both_correct": 2648, "selected_both_wrong": 415}, {"budget": 0.75, "threshold": -0.9286423325538635, "operator": "ge", "actual_fraction": 0.8828098032447359, "metric": 0.5995857783914395, "rescued": 141, "harmed": 1891, "missed_rescue": 3, "avoided_harm": 301, "selected_both_correct": 2667, "selected_both_wrong": 416}, {"budget": 0.76, "threshold": -0.9301291704177856, "operator": "ge", "actual_fraction": 0.8900586813945461, "metric": 0.5969968933379358, "rescued": 141, "harmed": 1906, "missed_rescue": 3, "avoided_harm": 286, "selected_both_correct": 2693, "selected_both_wrong": 417}, {"budget": 0.77, "threshold": -0.9318777918815613, "operator": "ge", "actual_fraction": 0.8979979288919572, "metric": 0.5945806006213324, "rescued": 141, "harmed": 1920, "missed_rescue": 3, "avoided_harm": 272, "selected_both_correct": 2725, "selected_both_wrong": 417}, {"budget": 0.78, "threshold": -0.9332055449485779, "operator": "ge", "actual_fraction": 0.9052468070417673, "metric": 0.5916465308940283, "rescued": 141, "harmed": 1937, "missed_rescue": 3, "avoided_harm": 255, "selected_both_correct": 2750, "selected_both_wrong": 417}, {"budget": 0.79, "threshold": -0.9344666600227356, "operator": "ge", "actual_fraction": 0.9100793924749742, "metric": 0.5888850535036244, "rescued": 141, "harmed": 1953, "missed_rescue": 3, "avoided_harm": 239, "selected_both_correct": 2762, "selected_both_wrong": 417}, {"budget": 0.8, "threshold": -0.9362465143203735, "operator": "ge", "actual_fraction": 0.917155678287884, "metric": 0.5862961684501208, "rescued": 141, "harmed": 1968, "missed_rescue": 3, "avoided_harm": 224, "selected_both_correct": 2787, "selected_both_wrong": 418}, {"budget": 0.81, "threshold": -0.937842071056366, "operator": "ge", "actual_fraction": 0.9237141870900932, "metric": 0.5830169140490162, "rescued": 141, "harmed": 1987, "missed_rescue": 3, "avoided_harm": 205, "selected_both_correct": 2806, "selected_both_wrong": 418}, {"budget": 0.82, "threshold": -0.9384457468986511, "operator": "ge", "actual_fraction": 0.9249223334483949, "metric": 0.5823265447014152, "rescued": 141, "harmed": 1991, "missed_rescue": 3, "avoided_harm": 201, "selected_both_correct": 2808, "selected_both_wrong": 419}, {"budget": 0.83, "threshold": -0.9396296143531799, "operator": "ge", "actual_fraction": 0.927166033828098, "metric": 0.5819813600276148, "rescued": 142, "harmed": 1994, "missed_rescue": 2, "avoided_harm": 198, "selected_both_correct": 2817, "selected_both_wrong": 419}, {"budget": 0.84, "threshold": -0.9407488703727722, "operator": "ge", "actual_fraction": 0.9316534345875043, "metric": 0.5802554366586123, "rescued": 142, "harmed": 2004, "missed_rescue": 2, "avoided_harm": 188, "selected_both_correct": 2833, "selected_both_wrong": 419}, {"budget": 0.85, "threshold": -0.942738950252533, "operator": "ge", "actual_fraction": 0.9397652744218157, "metric": 0.576458405246807, "rescued": 142, "harmed": 2026, "missed_rescue": 2, "avoided_harm": 166, "selected_both_correct": 2858, "selected_both_wrong": 419}, {"budget": 0.86, "threshold": -0.9438527226448059, "operator": "gt", "actual_fraction": 0.9432171211598205, "metric": 0.5749050742147048, "rescued": 142, "harmed": 2035, "missed_rescue": 2, "avoided_harm": 157, "selected_both_correct": 2868, "selected_both_wrong": 420}, {"budget": 0.87, "threshold": -0.9450756907463074, "operator": "ge", "actual_fraction": 0.9456334138764239, "metric": 0.5742147048671039, "rescued": 142, "harmed": 2039, "missed_rescue": 2, "avoided_harm": 153, "selected_both_correct": 2878, "selected_both_wrong": 420}, {"budget": 0.88, "threshold": -0.9481598138809204, "operator": "ge", "actual_fraction": 0.9539178460476355, "metric": 0.5704176734552986, "rescued": 142, "harmed": 2061, "missed_rescue": 2, "avoided_harm": 131, "selected_both_correct": 2902, "selected_both_wrong": 422}, {"budget": 0.89, "threshold": -0.9500836730003357, "operator": "ge", "actual_fraction": 0.9606489471867449, "metric": 0.5676561960648947, "rescued": 142, "harmed": 2077, "missed_rescue": 2, "avoided_harm": 115, "selected_both_correct": 2924, "selected_both_wrong": 423}, {"budget": 0.9, "threshold": -0.9519118666648865, "operator": "ge", "actual_fraction": 0.9647911632723507, "metric": 0.5662754573696928, "rescued": 143, "harmed": 2086, "missed_rescue": 1, "avoided_harm": 106, "selected_both_correct": 2938, "selected_both_wrong": 423}, {"budget": 0.91, "threshold": -0.9543898701667786, "operator": "ge", "actual_fraction": 0.9697963410424577, "metric": 0.5636865723161891, "rescued": 143, "harmed": 2101, "missed_rescue": 1, "avoided_harm": 91, "selected_both_correct": 2951, "selected_both_wrong": 424}, {"budget": 0.92, "threshold": -0.9570822715759277, "operator": "ge", "actual_fraction": 0.9760096651708664, "metric": 0.5605799102519848, "rescued": 143, "harmed": 2119, "missed_rescue": 1, "avoided_harm": 73, "selected_both_correct": 2968, "selected_both_wrong": 425}, {"budget": 0.93, "threshold": -0.9593446254730225, "operator": "ge", "actual_fraction": 0.9801518812564722, "metric": 0.5586813945460821, "rescued": 143, "harmed": 2130, "missed_rescue": 1, "avoided_harm": 62, "selected_both_correct": 2981, "selected_both_wrong": 425}, {"budget": 0.94, "threshold": -0.9610995054244995, "operator": "ge", "actual_fraction": 0.9817052122885744, "metric": 0.5579910251984812, "rescued": 143, "harmed": 2134, "missed_rescue": 1, "avoided_harm": 58, "selected_both_correct": 2986, "selected_both_wrong": 425}, {"budget": 0.95, "threshold": -0.9635111093521118, "operator": "ge", "actual_fraction": 0.9853296513634795, "metric": 0.556437694166379, "rescued": 143, "harmed": 2143, "missed_rescue": 1, "avoided_harm": 49, "selected_both_correct": 2998, "selected_both_wrong": 425}, {"budget": 0.96, "threshold": -0.9648061990737915, "operator": "ge", "actual_fraction": 0.9875733517431826, "metric": 0.5548843631342768, "rescued": 143, "harmed": 2152, "missed_rescue": 1, "avoided_harm": 40, "selected_both_correct": 3002, "selected_both_wrong": 425}, {"budget": 0.97, "threshold": -0.9674390554428101, "operator": "ge", "actual_fraction": 0.9906800138073869, "metric": 0.5533310321021747, "rescued": 143, "harmed": 2161, "missed_rescue": 1, "avoided_harm": 31, "selected_both_correct": 3011, "selected_both_wrong": 425}, {"budget": 0.98, "threshold": -0.9724470973014832, "operator": "ge", "actual_fraction": 0.9937866758715913, "metric": 0.5516051087331723, "rescued": 143, "harmed": 2171, "missed_rescue": 1, "avoided_harm": 21, "selected_both_correct": 3018, "selected_both_wrong": 426}, {"budget": 0.99, "threshold": -0.9774226546287537, "operator": "ge", "actual_fraction": 0.9965481532619952, "metric": 0.5502243700379703, "rescued": 144, "harmed": 2180, "missed_rescue": 0, "avoided_harm": 12, "selected_both_correct": 3024, "selected_both_wrong": 426}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.5481532619951675, "rescued": 144, "harmed": 2192, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 3032, "selected_both_wrong": 426}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.9891666666666665, "actual_fraction": 0.0, "metric_std": 0.0, "actual_fraction_std": 0.0, "per_seed": [{"seed": 2026, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.9891666666666665, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}, {"seed": 2027, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.9891666666666665, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}, {"seed": 2028, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.9891666666666665, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}, {"seed": 2029, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.9891666666666665, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}, {"seed": 2030, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.9891666666666665, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 2}], "rescued": 0.0, "harmed": 0.0, "missed_rescue": 2.0, "avoided_harm": 528.0}, "validation_curve": [{"budget": 0.0, "metric": 0.9891666666666665, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.9888333333333333, "metric_std": 0.00045643546458758817, "actual_fraction": 0.01, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.9886666666666667, "metric_std": 0.0007453559924998479, "actual_fraction": 0.02, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.9884999999999999, "metric_std": 0.0006972166887783195, "actual_fraction": 0.03, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.9883333333333333, "metric_std": 0.0005892556509887247, "actual_fraction": 0.04, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.9881666666666666, "metric_std": 0.0003726779962499239, "actual_fraction": 0.05, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.9881666666666666, "metric_std": 0.0003726779962499239, "actual_fraction": 0.06, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.9879999999999999, "metric_std": 0.00045643546458758817, "actual_fraction": 0.07, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.9879999999999999, "metric_std": 0.00045643546458758817, "actual_fraction": 0.08, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.9878333333333333, "metric_std": 0.0007453559924998913, "actual_fraction": 0.09, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.9876666666666667, "metric_std": 0.0006972166887783593, "actual_fraction": 0.1, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.9876666666666667, "metric_std": 0.0006972166887783593, "actual_fraction": 0.11000000000000001, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.12, "metric": 0.9876666666666667, "metric_std": 0.0006972166887783593, "actual_fraction": 0.12, "actual_fraction_std": 0.0}, {"budget": 0.13, "metric": 0.9871666666666666, "metric_std": 0.000456435464587649, "actual_fraction": 0.13, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.9866666666666667, "metric_std": 0.0005892556509888032, "actual_fraction": 0.14, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.9865, "metric_std": 0.0006972166887784124, "actual_fraction": 0.15, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.9861666666666666, "metric_std": 0.0007453559924999471, "actual_fraction": 0.16, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.9856666666666666, "metric_std": 0.0010865337342005048, "actual_fraction": 0.17, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.985, "metric_std": 0.0015590239111558004, "actual_fraction": 0.18, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.9836666666666666, "metric_std": 0.0018257418583505376, "actual_fraction": 0.19, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.9823333333333333, "metric_std": 0.0016029486718059016, "actual_fraction": 0.2, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.982, "metric_std": 0.001513825177048732, "actual_fraction": 0.21000000000000002, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.9806666666666667, "metric_std": 0.0017078251276600213, "actual_fraction": 0.22000000000000003, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.23, "metric": 0.9801666666666666, "metric_std": 0.0013693063937630077, "actual_fraction": 0.23000000000000004, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.24, "metric": 0.9798333333333332, "metric_std": 0.0017078251276599833, "actual_fraction": 0.24, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.9794999999999998, "metric_std": 0.0020916500663352945, "actual_fraction": 0.25, "actual_fraction_std": 0.0}, {"budget": 0.26, "metric": 0.9781666666666666, "metric_std": 0.0021570298509241397, "actual_fraction": 0.26, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.9766666666666666, "metric_std": 0.002357022603955164, "actual_fraction": 0.27, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.976, "metric_std": 0.00231240615425194, "actual_fraction": 0.28, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.9739999999999999, "metric_std": 0.002312406154251894, "actual_fraction": 0.29, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.9719999999999999, "metric_std": 0.0019184774055368834, "actual_fraction": 0.3, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.9703333333333333, "metric_std": 0.001263812574008621, "actual_fraction": 0.3098333333333333, "actual_fraction_std": 0.00037267799624997355}, {"budget": 0.32, "metric": 0.9684999999999999, "metric_std": 0.0016029486718059363, "actual_fraction": 0.32, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.9673333333333334, "metric_std": 0.0016029486718059335, "actual_fraction": 0.33, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.9654999999999999, "metric_std": 0.0018257418583505528, "actual_fraction": 0.34, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.9635, "metric_std": 0.0013693063937629064, "actual_fraction": 0.35, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.9608333333333334, "metric_std": 0.001020620726159681, "actual_fraction": 0.36, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.9586666666666666, "metric_std": 0.0016244657241348106, "actual_fraction": 0.37, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.9571666666666667, "metric_std": 0.001513825177048744, "actual_fraction": 0.38, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.9551666666666667, "metric_std": 0.0010865337342004667, "actual_fraction": 0.39, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.953, "metric_std": 0.0012638125740085699, "actual_fraction": 0.4, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.9513333333333334, "metric_std": 0.0017280367794440168, "actual_fraction": 0.4091666666666668, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.42, "metric": 0.9484999999999999, "metric_std": 0.0016029486718059016, "actual_fraction": 0.42000000000000004, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.9458333333333334, "metric_std": 0.0017677669529663834, "actual_fraction": 0.43, "actual_fraction_std": 0.0}, {"budget": 0.44, "metric": 0.9425000000000001, "metric_std": 0.0021245914639969883, "actual_fraction": 0.44000000000000006, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.45, "metric": 0.9390000000000001, "metric_std": 0.0021570298509241354, "actual_fraction": 0.45, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.9359999999999999, "metric_std": 0.0021570298509241354, "actual_fraction": 0.4600000000000001, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.47, "metric": 0.9326666666666666, "metric_std": 0.001490711984999857, "actual_fraction": 0.4699999999999999, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.48, "metric": 0.9298333333333334, "metric_std": 0.0018066236157231624, "actual_fraction": 0.48, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.9258333333333333, "metric_std": 0.0019543398999264266, "actual_fraction": 0.49000000000000005, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.5, "metric": 0.9215, "metric_std": 0.002595401917066212, "actual_fraction": 0.5, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.9176666666666666, "metric_std": 0.003084459253886972, "actual_fraction": 0.51, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.9136666666666666, "metric_std": 0.0032058973436119333, "actual_fraction": 0.52, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.9093333333333333, "metric_std": 0.004013864859597433, "actual_fraction": 0.53, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.9041666666666666, "metric_std": 0.003486083443892002, "actual_fraction": 0.54, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.8988333333333334, "metric_std": 0.0022515426810177084, "actual_fraction": 0.55, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.8943333333333333, "metric_std": 0.0019002923751652398, "actual_fraction": 0.56, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.8886666666666667, "metric_std": 0.00045643546458764894, "actual_fraction": 0.5691666666666667, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.8826666666666666, "metric_std": 0.0010865337342003728, "actual_fraction": 0.58, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.8768333333333331, "metric_std": 0.0006972166887784124, "actual_fraction": 0.59, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.8728333333333333, "metric_std": 0.001394433377556818, "actual_fraction": 0.6, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.866, "metric_std": 0.0009128709291753131, "actual_fraction": 0.61, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.859, "metric_std": 0.0009128709291752372, "actual_fraction": 0.62, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.8525, "metric_std": 0.0008333333333333526, "actual_fraction": 0.63, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.8458333333333332, "metric_std": 0.0010206207261595904, "actual_fraction": 0.64, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.8396666666666667, "metric_std": 0.0017280367794440168, "actual_fraction": 0.65, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.8331666666666667, "metric_std": 0.0014907119849998508, "actual_fraction": 0.66, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.8261666666666667, "metric_std": 0.001263812574008621, "actual_fraction": 0.67, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.8196666666666668, "metric_std": 0.0012638125740085733, "actual_fraction": 0.68, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.8143333333333335, "metric_std": 0.0009128709291752827, "actual_fraction": 0.6891666666666667, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.808, "metric_std": 0.0020916500663351843, "actual_fraction": 0.7, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.8019999999999999, "metric_std": 0.0012638125740085696, "actual_fraction": 0.71, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.7945, "metric_std": 0.0007453559924999471, "actual_fraction": 0.72, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.787, "metric_std": 0.0007453559924999471, "actual_fraction": 0.73, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.7801666666666666, "metric_std": 0.0012360330811825905, "actual_fraction": 0.74, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.7715, "metric_std": 0.0018066236157232391, "actual_fraction": 0.75, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.7628333333333333, "metric_std": 0.0019184774055368932, "actual_fraction": 0.76, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.7551666666666667, "metric_std": 0.0014907119849998508, "actual_fraction": 0.77, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.7478333333333333, "metric_std": 0.0019184774055368858, "actual_fraction": 0.78, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.7396666666666667, "metric_std": 0.0019184774055369172, "actual_fraction": 0.79, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.7318333333333333, "metric_std": 0.0012360330811826052, "actual_fraction": 0.8, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.7224999999999998, "metric_std": 0.0005892556509888032, "actual_fraction": 0.8100000000000002, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.82, "metric": 0.7151666666666665, "metric_std": 0.0009128709291752523, "actual_fraction": 0.8191666666666666, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.83, "metric": 0.7065, "metric_std": 0.0016029486718059508, "actual_fraction": 0.8299999999999998, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.84, "metric": 0.6983333333333335, "metric_std": 0.0013176156917367849, "actual_fraction": 0.8400000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.6900000000000001, "metric_std": 0.001020620726159681, "actual_fraction": 0.85, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.6806666666666666, "metric_std": 0.0014907119849997949, "actual_fraction": 0.86, "actual_fraction_std": 0.0}, {"budget": 0.87, "metric": 0.6714999999999999, "metric_std": 0.0010865337342004623, "actual_fraction": 0.8699999999999999, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.6628333333333334, "metric_std": 0.0015138251770487443, "actual_fraction": 0.8800000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.89, "metric": 0.6533333333333333, "metric_std": 0.0015590239111558004, "actual_fraction": 0.89, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.6439999999999999, "metric_std": 0.0010865337342004196, "actual_fraction": 0.9, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.6356666666666667, "metric_std": 0.0013693063937629097, "actual_fraction": 0.9099999999999999, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.6268333333333332, "metric_std": 0.0010865337342004623, "actual_fraction": 0.9200000000000002, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.93, "metric": 0.6174999999999999, "metric_std": 0.000589255650988764, "actual_fraction": 0.93, "actual_fraction_std": 0.0}, {"budget": 0.94, "metric": 0.6078333333333333, "metric_std": 0.00045643546458764894, "actual_fraction": 0.9399999999999998, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.95, "metric": 0.5983333333333333, "metric_std": 0.0008333333333333526, "actual_fraction": 0.95, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.589, "metric_std": 0.0006972166887783593, "actual_fraction": 0.96, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.5795, "metric_std": 0.00045643546458764894, "actual_fraction": 0.97, "actual_fraction_std": 0.0}, {"budget": 0.98, "metric": 0.5701666666666666, "metric_std": 0.0006972166887784124, "actual_fraction": 0.9800000000000001, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.99, "metric": 0.5603333333333333, "metric_std": 0.00045643546458758817, "actual_fraction": 0.99, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.5508333333333333, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.9016223679668623, "actual_fraction": 0.0, "metric_std": 0.0, "actual_fraction_std": 0.0, "per_seed": [{"seed": 2026, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.9016223679668623, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 144}, {"seed": 2027, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.9016223679668623, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 144}, {"seed": 2028, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 3, "metric": 0.9016223679668623, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 144}, {"seed": 2029, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.9016223679668623, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 144}, {"seed": 2030, "budget": 0.0, "threshold": null, "operator": "never", "alpha": 0.0, "epoch": 2, "metric": 0.9016223679668623, "actual_fraction": 0.0, "rescue": 0, "harm": 0, "missed_rescue": 144}], "rescued": 0.0, "harmed": 0.0, "missed_rescue": 144.0, "avoided_harm": 2192.0}, "test_curve": [{"budget": 0.0, "metric": 0.9016223679668623, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.9015188125647221, "metric_std": 0.00015437127908186376, "actual_fraction": 0.008457024508111841, "actual_fraction_std": 0.0011513344894827702}, {"budget": 0.02, "metric": 0.9015188125647221, "metric_std": 0.00015437127908186376, "actual_fraction": 0.018812564722126338, "actual_fraction_std": 0.0016005554876589066}, {"budget": 0.03, "metric": 0.901484294097342, "metric_std": 0.0001444011091706001, "actual_fraction": 0.027373144632378322, "actual_fraction_std": 0.0015893495938172364}, {"budget": 0.04, "metric": 0.9013807386952019, "metric_std": 0.00015437127908185134, "actual_fraction": 0.036382464618570934, "actual_fraction_std": 0.0024663197346076965}, {"budget": 0.05, "metric": 0.9012426648256817, "metric_std": 7.718563954090395e-05, "actual_fraction": 0.04539178460476355, "actual_fraction_std": 0.0019411843077669986}, {"budget": 0.06, "metric": 0.9011391094235416, "metric_std": 0.00014440110917046743, "actual_fraction": 0.0556437694166379, "actual_fraction_std": 0.004256772275620626}, {"budget": 0.07, "metric": 0.9008974801518812, "metric_std": 0.0001444011091705205, "actual_fraction": 0.06161546427338627, "actual_fraction_std": 0.0052477721075287415}, {"budget": 0.08, "metric": 0.9007594062823611, "metric_std": 0.0001725923369003124, "actual_fraction": 0.06986537797721781, "actual_fraction_std": 0.004494696811575262}, {"budget": 0.09, "metric": 0.9004832585433207, "metric_std": 0.00041565738967190525, "actual_fraction": 0.07870210562651017, "actual_fraction_std": 0.0038282832262372846}, {"budget": 0.1, "metric": 0.9002761477390404, "metric_std": 0.0003087425581636158, "actual_fraction": 0.08729720400414222, "actual_fraction_std": 0.004180512758950508}, {"budget": 0.11, "metric": 0.90006903693476, "metric_std": 0.0004227631589201012, "actual_fraction": 0.09551259924059372, "actual_fraction_std": 0.005272688830418069}, {"budget": 0.12, "metric": 0.8995167414566794, "metric_std": 0.0003741712700841811, "actual_fraction": 0.10797376596479116, "actual_fraction_std": 0.0035781184612060923}, {"budget": 0.13, "metric": 0.8988608905764582, "metric_std": 0.00024408242360600966, "actual_fraction": 0.11960648947186744, "actual_fraction_std": 0.0042046695599170895}, {"budget": 0.14, "metric": 0.8981360027614773, "metric_std": 0.00044674418728147224, "actual_fraction": 0.13010010355540214, "actual_fraction_std": 0.005722158311384777}, {"budget": 0.15, "metric": 0.8974456334138763, "metric_std": 0.00035370903575973096, "actual_fraction": 0.13969623748705556, "actual_fraction_std": 0.004695970757402041}, {"budget": 0.16, "metric": 0.8968933379357956, "metric_std": 0.00023155691862270357, "actual_fraction": 0.14970659302726957, "actual_fraction_std": 0.005283975772223241}, {"budget": 0.17, "metric": 0.8963410424577148, "metric_std": 0.0003135295499583305, "actual_fraction": 0.15971694856748359, "actual_fraction_std": 0.003944405194375617}, {"budget": 0.18, "metric": 0.895685191577494, "metric_std": 0.0005801772976863372, "actual_fraction": 0.1691404901622368, "actual_fraction_std": 0.004186920770006282}, {"budget": 0.19, "metric": 0.8950293406972731, "metric_std": 0.0005645642814565547, "actual_fraction": 0.17877114256127027, "actual_fraction_std": 0.003881217545108293}, {"budget": 0.2, "metric": 0.8943389713496721, "metric_std": 0.00025599580556075684, "actual_fraction": 0.18860890576458406, "actual_fraction_std": 0.005532923778206849}, {"budget": 0.21, "metric": 0.8929237141870899, "metric_std": 0.0005119916111215886, "actual_fraction": 0.1999309630652399, "actual_fraction_std": 0.006330398943366078}, {"budget": 0.22, "metric": 0.8921297894373488, "metric_std": 0.00044002584687547315, "actual_fraction": 0.20952709699689334, "actual_fraction_std": 0.004936261527813291}, {"budget": 0.23, "metric": 0.8910597169485674, "metric_std": 0.0006617290132095761, "actual_fraction": 0.2195719710044874, "actual_fraction_std": 0.0022483422947161893}, {"budget": 0.24, "metric": 0.8900931998619261, "metric_std": 0.0009326376310739572, "actual_fraction": 0.22837418018639974, "actual_fraction_std": 0.003842651231232095}, {"budget": 0.25, "metric": 0.8894373489817052, "metric_std": 0.0008595029063161575, "actual_fraction": 0.23555402140144976, "actual_fraction_std": 0.003548441363141197}, {"budget": 0.26, "metric": 0.8881946841560234, "metric_std": 0.0011145242865698106, "actual_fraction": 0.2453572661373835, "actual_fraction_std": 0.0058663624997437646}, {"budget": 0.27, "metric": 0.8870555747324819, "metric_std": 0.0015320911992524674, "actual_fraction": 0.2555056955471177, "actual_fraction_std": 0.006678453730107497}, {"budget": 0.28, "metric": 0.8857438729720399, "metric_std": 0.001428455760894608, "actual_fraction": 0.2655850880220918, "actual_fraction_std": 0.003983481985761557}, {"budget": 0.29, "metric": 0.8847428374180186, "metric_std": 0.001198243352275457, "actual_fraction": 0.272143596824301, "actual_fraction_std": 0.004583293414829049}, {"budget": 0.3, "metric": 0.8833620987228166, "metric_std": 0.0009944674001303268, "actual_fraction": 0.2834311356575768, "actual_fraction_std": 0.005258262842151995}, {"budget": 0.31, "metric": 0.8822575077666551, "metric_std": 0.0013882690750562825, "actual_fraction": 0.2918881601656887, "actual_fraction_std": 0.004731989630441026}, {"budget": 0.32, "metric": 0.8809458060062132, "metric_std": 0.0013166883757514908, "actual_fraction": 0.3023472557818433, "actual_fraction_std": 0.004496353354148589}, {"budget": 0.33, "metric": 0.8793234380393511, "metric_std": 0.0017490681477608757, "actual_fraction": 0.31297894373489815, "actual_fraction_std": 0.006360911591431476}, {"budget": 0.34, "metric": 0.8777701070072489, "metric_std": 0.002658146124705383, "actual_fraction": 0.3245771487745944, "actual_fraction_std": 0.0095118579126522}, {"budget": 0.35, "metric": 0.8766655160510874, "metric_std": 0.0025733048615372545, "actual_fraction": 0.3345529858474284, "actual_fraction_std": 0.007761475663328928}, {"budget": 0.36, "metric": 0.8748360372799446, "metric_std": 0.002326478121816323, "actual_fraction": 0.3443562305833621, "actual_fraction_std": 0.007044008997028087}, {"budget": 0.37, "metric": 0.8730410769761823, "metric_std": 0.0022055382879848373, "actual_fraction": 0.3544701415257162, "actual_fraction_std": 0.0065850253341432595}, {"budget": 0.38, "metric": 0.8710044874007593, "metric_std": 0.0015799506373967225, "actual_fraction": 0.36551605108733176, "actual_fraction_std": 0.004077710835405344}, {"budget": 0.39, "metric": 0.8692095270969968, "metric_std": 0.00156479488528036, "actual_fraction": 0.3752847773558854, "actual_fraction_std": 0.004021068103120029}, {"budget": 0.4, "metric": 0.8666206420434932, "metric_std": 0.0015456412272860038, "actual_fraction": 0.3856057991025198, "actual_fraction_std": 0.00542692445681123}, {"budget": 0.41, "metric": 0.8641007939247496, "metric_std": 0.002487965081434898, "actual_fraction": 0.3973420780117363, "actual_fraction_std": 0.007222530481895513}, {"budget": 0.42, "metric": 0.8617880566102866, "metric_std": 0.002475361631247107, "actual_fraction": 0.4099758370728339, "actual_fraction_std": 0.00870060549796205}, {"budget": 0.43, "metric": 0.8601311701760442, "metric_std": 0.002058846456119639, "actual_fraction": 0.4179150845702451, "actual_fraction_std": 0.006534164436311594}, {"budget": 0.44, "metric": 0.8583707283396617, "metric_std": 0.002258916567255288, "actual_fraction": 0.4273386261649983, "actual_fraction_std": 0.007970141449098245}, {"budget": 0.45, "metric": 0.8559199171556783, "metric_std": 0.0024693373788718664, "actual_fraction": 0.4358301691404902, "actual_fraction_std": 0.008381308502806594}, {"budget": 0.46, "metric": 0.8541939937866758, "metric_std": 0.002702599810985626, "actual_fraction": 0.44521919226786333, "actual_fraction_std": 0.009433240970771882}, {"budget": 0.47, "metric": 0.852088367276493, "metric_std": 0.003399674767620345, "actual_fraction": 0.4532619951674145, "actual_fraction_std": 0.011724470698828204}, {"budget": 0.48, "metric": 0.8497756299620297, "metric_std": 0.0033301824460402105, "actual_fraction": 0.462823610631688, "actual_fraction_std": 0.010841491742695438}, {"budget": 0.49, "metric": 0.8460476354849844, "metric_std": 0.0026345076842377397, "actual_fraction": 0.47773558853986886, "actual_fraction_std": 0.013406214618022046}, {"budget": 0.5, "metric": 0.8445978598550224, "metric_std": 0.0030960967892980635, "actual_fraction": 0.4859855022437004, "actual_fraction_std": 0.014234107634149548}, {"budget": 0.51, "metric": 0.8423886779426993, "metric_std": 0.003441908371635498, "actual_fraction": 0.49468415602347254, "actual_fraction_std": 0.014694390339064542}, {"budget": 0.52, "metric": 0.8398343113565758, "metric_std": 0.003322122291149458, "actual_fraction": 0.5039005868139455, "actual_fraction_std": 0.01281990505218611}, {"budget": 0.53, "metric": 0.8366586123576113, "metric_std": 0.002933054302554056, "actual_fraction": 0.5139109423541595, "actual_fraction_std": 0.012083682423596104}, {"budget": 0.54, "metric": 0.833586468760787, "metric_std": 0.002778683023472319, "actual_fraction": 0.5255091473938558, "actual_fraction_std": 0.011322504558548501}, {"budget": 0.55, "metric": 0.8298584742837416, "metric_std": 0.0025112036743344095, "actual_fraction": 0.5403520883672766, "actual_fraction_std": 0.011011742765795265}, {"budget": 0.56, "metric": 0.8256817397307559, "metric_std": 0.0042099796056784345, "actual_fraction": 0.5531584397652745, "actual_fraction_std": 0.014216098781548342}, {"budget": 0.57, "metric": 0.8225405591991715, "metric_std": 0.004697239253037608, "actual_fraction": 0.5639282015878495, "actual_fraction_std": 0.015380872854771688}, {"budget": 0.58, "metric": 0.8198826372109078, "metric_std": 0.005348413465771533, "actual_fraction": 0.5725232999654816, "actual_fraction_std": 0.015697555315725486}, {"budget": 0.59, "metric": 0.8171901967552639, "metric_std": 0.005053154655206181, "actual_fraction": 0.5825681739730756, "actual_fraction_std": 0.014684352389557512}, {"budget": 0.6, "metric": 0.8148429409734208, "metric_std": 0.004570276366631302, "actual_fraction": 0.5897825336555057, "actual_fraction_std": 0.01564385548407137}, {"budget": 0.61, "metric": 0.8111149464963756, "metric_std": 0.005123696471793347, "actual_fraction": 0.601449775629962, "actual_fraction_std": 0.015991311039177725}, {"budget": 0.62, "metric": 0.8060407317915084, "metric_std": 0.00461083262565959, "actual_fraction": 0.6136693130824991, "actual_fraction_std": 0.01582851463642558}, {"budget": 0.63, "metric": 0.8012426648256816, "metric_std": 0.004103562022433026, "actual_fraction": 0.6246461857093545, "actual_fraction_std": 0.01334240350944725}, {"budget": 0.64, "metric": 0.7964791163272349, "metric_std": 0.004731989630440974, "actual_fraction": 0.6360372799447704, "actual_fraction_std": 0.015258275410538227}, {"budget": 0.65, "metric": 0.7920607525025888, "metric_std": 0.005386982587308418, "actual_fraction": 0.6474628926475665, "actual_fraction_std": 0.014691450639878926}, {"budget": 0.66, "metric": 0.787884017949603, "metric_std": 0.003671392369033954, "actual_fraction": 0.6567828788401796, "actual_fraction_std": 0.01244018458847833}, {"budget": 0.67, "metric": 0.7825681739730755, "metric_std": 0.0043839812546376195, "actual_fraction": 0.6687262685536762, "actual_fraction_std": 0.012188001094053274}, {"budget": 0.68, "metric": 0.7789782533655505, "metric_std": 0.0054994429638340935, "actual_fraction": 0.6767690714532275, "actual_fraction_std": 0.014624794753293431}, {"budget": 0.69, "metric": 0.774318260269244, "metric_std": 0.006446964907613874, "actual_fraction": 0.6866068346565413, "actual_fraction_std": 0.016035955171427963}, {"budget": 0.7, "metric": 0.7678287884017949, "metric_std": 0.006523670711560935, "actual_fraction": 0.6994131860545393, "actual_fraction_std": 0.014003185416908382}, {"budget": 0.71, "metric": 0.7627890921643078, "metric_std": 0.005442271814469875, "actual_fraction": 0.7087676907145323, "actual_fraction_std": 0.011767076689794568}, {"budget": 0.72, "metric": 0.7558508802209182, "metric_std": 0.005754861074472984, "actual_fraction": 0.7211943389713497, "actual_fraction_std": 0.011721421487697843}, {"budget": 0.73, "metric": 0.7495685191577494, "metric_std": 0.005572621432848767, "actual_fraction": 0.7324128408698654, "actual_fraction_std": 0.010070137986452172}, {"budget": 0.74, "metric": 0.7432171211598204, "metric_std": 0.006691598770267963, "actual_fraction": 0.7423886779426994, "actual_fraction_std": 0.011673674116956079}, {"budget": 0.75, "metric": 0.7357266137383501, "metric_std": 0.005290736379452145, "actual_fraction": 0.7551605108733173, "actual_fraction_std": 0.008768811813548895}, {"budget": 0.76, "metric": 0.7294442526751812, "metric_std": 0.0048381298634704705, "actual_fraction": 0.7644114601311702, "actual_fraction_std": 0.009104143524268539}, {"budget": 0.77, "metric": 0.7236451501553332, "metric_std": 0.004555587728381775, "actual_fraction": 0.7736278909216431, "actual_fraction_std": 0.007839758744851185}, {"budget": 0.78, "metric": 0.7168104936140836, "metric_std": 0.005647489218226469, "actual_fraction": 0.784294097342078, "actual_fraction_std": 0.008832951385053207}, {"budget": 0.79, "metric": 0.7117362789092164, "metric_std": 0.005330002349291387, "actual_fraction": 0.7924404556437694, "actual_fraction_std": 0.008832276882254629}, {"budget": 0.8, "metric": 0.7055919917155677, "metric_std": 0.004562121850684946, "actual_fraction": 0.8011045909561616, "actual_fraction_std": 0.009353963303050472}, {"budget": 0.81, "metric": 0.6992060752502589, "metric_std": 0.0037879987959734186, "actual_fraction": 0.8104245771487746, "actual_fraction_std": 0.0080182097802038}, {"budget": 0.82, "metric": 0.6937866758715913, "metric_std": 0.003166963567611171, "actual_fraction": 0.8185364169830859, "actual_fraction_std": 0.00691511826499799}, {"budget": 0.83, "metric": 0.6852951328960993, "metric_std": 0.0030646684295015277, "actual_fraction": 0.8304107697618225, "actual_fraction_std": 0.007482032081075329}, {"budget": 0.84, "metric": 0.6800483258543321, "metric_std": 0.0030859780115406843, "actual_fraction": 0.8379703141180531, "actual_fraction_std": 0.006995208082023662}, {"budget": 0.85, "metric": 0.6740421125302036, "metric_std": 0.002858475096578458, "actual_fraction": 0.8463583016914049, "actual_fraction_std": 0.0060670564036307975}, {"budget": 0.86, "metric": 0.6662064204349327, "metric_std": 0.004874014928009321, "actual_fraction": 0.857921988263721, "actual_fraction_std": 0.00693962860152726}, {"budget": 0.87, "metric": 0.658577839143942, "metric_std": 0.0060510784707288635, "actual_fraction": 0.8682430100103555, "actual_fraction_std": 0.00818822866646688}, {"budget": 0.88, "metric": 0.6517777010700725, "metric_std": 0.00461277035746433, "actual_fraction": 0.8773904038660684, "actual_fraction_std": 0.005649071369600258}, {"budget": 0.89, "metric": 0.6441491197790818, "metric_std": 0.0045441304062862585, "actual_fraction": 0.8873662409389024, "actual_fraction_std": 0.0065728002275759765}, {"budget": 0.9, "metric": 0.6331377286848463, "metric_std": 0.004136098923474281, "actual_fraction": 0.9017604418363824, "actual_fraction_std": 0.0072503162766271514}, {"budget": 0.91, "metric": 0.6248187780462546, "metric_std": 0.005538304954131735, "actual_fraction": 0.9121159820503969, "actual_fraction_std": 0.009045386382139285}, {"budget": 0.92, "metric": 0.6149119779081809, "metric_std": 0.007437908760271144, "actual_fraction": 0.9237487055574732, "actual_fraction_std": 0.011131469846942505}, {"budget": 0.93, "metric": 0.6061442871936485, "metric_std": 0.0027749284071131356, "actual_fraction": 0.9340006903693476, "actual_fraction_std": 0.005032775971331334}, {"budget": 0.94, "metric": 0.5968588194684156, "metric_std": 0.001844397653764937, "actual_fraction": 0.9450811183983431, "actual_fraction_std": 0.0038869694885079035}, {"budget": 0.95, "metric": 0.5881601656886434, "metric_std": 0.001066726761502415, "actual_fraction": 0.9558163617535381, "actual_fraction_std": 0.0012973158766286894}, {"budget": 0.96, "metric": 0.5813600276147739, "metric_std": 0.002985897310135814, "actual_fraction": 0.9635484984466689, "actual_fraction_std": 0.003396168139799451}, {"budget": 0.97, "metric": 0.573627890921643, "metric_std": 0.0025558821502842063, "actual_fraction": 0.972178115291681, "actual_fraction_std": 0.0033095452551722723}, {"budget": 0.98, "metric": 0.5665516051087331, "metric_std": 0.002779218983169373, "actual_fraction": 0.9797721781152917, "actual_fraction_std": 0.003600526009086712}, {"budget": 0.99, "metric": 0.5597514670348637, "metric_std": 0.00209754726992328, "actual_fraction": 0.987228167069382, "actual_fraction_std": 0.002637332894362457}, {"budget": 1.0, "metric": 0.5481532619951675, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 659, "rescuable": 2, "harmful": 528, "both_wrong": 11, "unit": "sample", "total": 1200}, "test": {"both_correct": 3032, "rescuable": 144, "harmful": 2192, "both_wrong": 426, "unit": "sample", "total": 5794}}, "small": {"validation": 0.9891666666666666, "test": 0.9016223679668622}, "large": {"validation": 0.5508333333333333, "test": 0.5481532619951675}, "learned_minus_confidence_test": 1.1102230246251565e-16, "learned_minus_small_test": 1.1102230246251565e-16, "learned_minus_large_test": 0.3534691059716949, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_glsim_Qwen3.8"}, {"key": ["grefcoco", "groundingdino", "MiniCPM"], "expert_display": "GroundingDINO", "run": "outputs/full_budget_20260913/runs/full_groundingdino_MiniCPM_s{2026..2030}", "validation_rows": 14229, "test_rows": 35263, "confidence_rule": {"column": "max_score", "columns": ["detection_count", "max_score", "mean_score", "query_count", "detection_count_valid", "max_score_valid", "mean_score_valid", "query_count_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/groundingdino/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 100, "validation_area_0_25": 0.5566729917773561, "validation_point": {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.6412256658935975, "rescued": 2589, "harmed": 1441, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 6535, "selected_both_wrong": 3664}, "test_point": {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.5000708958398321, "rescued": 10723, "harmed": 2500, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 6911, "selected_both_wrong": 15129}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.5605453650994447, "rescued": 0, "harmed": 0, "missed_rescue": 2589, "avoided_harm": 1441, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.1584138125181198, "operator": "ge", "actual_fraction": 0.009979619087778481, "metric": 0.5602642490688031, "rescued": 0, "harmed": 4, "missed_rescue": 2589, "avoided_harm": 1437, "selected_both_correct": 138, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -0.21403978765010834, "operator": "ge", "actual_fraction": 0.019959238175556962, "metric": 0.5599128540305011, "rescued": 0, "harmed": 9, "missed_rescue": 2589, "avoided_harm": 1432, "selected_both_correct": 275, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -0.24253316223621368, "operator": "ge", "actual_fraction": 0.029938857263335443, "metric": 0.559561458992199, "rescued": 0, "harmed": 14, "missed_rescue": 2589, "avoided_harm": 1427, "selected_both_correct": 409, "selected_both_wrong": 3}, {"budget": 0.04, "threshold": -0.2599165141582489, "operator": "ge", "actual_fraction": 0.03998875535877434, "metric": 0.5589289479232553, "rescued": 0, "harmed": 23, "missed_rescue": 2589, "avoided_harm": 1418, "selected_both_correct": 542, "selected_both_wrong": 4}, {"budget": 0.05, "threshold": -0.2725968658924103, "operator": "ge", "actual_fraction": 0.04996837444655281, "metric": 0.5583667158619721, "rescued": 1, "harmed": 32, "missed_rescue": 2588, "avoided_harm": 1409, "selected_both_correct": 666, "selected_both_wrong": 12}, {"budget": 0.06, "threshold": -0.2842792272567749, "operator": "ge", "actual_fraction": 0.05994799353433129, "metric": 0.5578747628083491, "rescued": 2, "harmed": 40, "missed_rescue": 2587, "avoided_harm": 1401, "selected_both_correct": 794, "selected_both_wrong": 17}, {"budget": 0.07, "threshold": -0.2935555577278137, "operator": "ge", "actual_fraction": 0.06999789162977019, "metric": 0.5569611357087638, "rescued": 3, "harmed": 54, "missed_rescue": 2586, "avoided_harm": 1387, "selected_both_correct": 911, "selected_both_wrong": 28}, {"budget": 0.08, "threshold": -0.30114513635635376, "operator": "ge", "actual_fraction": 0.07997751071754867, "metric": 0.5566800196781222, "rescued": 8, "harmed": 63, "missed_rescue": 2581, "avoided_harm": 1378, "selected_both_correct": 1032, "selected_both_wrong": 35}, {"budget": 0.09, "threshold": -0.308601438999176, "operator": "ge", "actual_fraction": 0.08995712980532715, "metric": 0.5562583456321597, "rescued": 14, "harmed": 75, "missed_rescue": 2575, "avoided_harm": 1366, "selected_both_correct": 1140, "selected_both_wrong": 51}, {"budget": 0.1, "threshold": -0.3146189749240875, "operator": "ge", "actual_fraction": 0.09993674889310562, "metric": 0.555625834563216, "rescued": 18, "harmed": 88, "missed_rescue": 2571, "avoided_harm": 1353, "selected_both_correct": 1249, "selected_both_wrong": 67}, {"budget": 0.11, "threshold": -0.32043084502220154, "operator": "ge", "actual_fraction": 0.10998664698854452, "metric": 0.555625834563216, "rescued": 24, "harmed": 94, "missed_rescue": 2565, "avoided_harm": 1347, "selected_both_correct": 1360, "selected_both_wrong": 87}, {"budget": 0.12, "threshold": -0.32644838094711304, "operator": "ge", "actual_fraction": 0.119966266076323, "metric": 0.5553447185325743, "rescued": 35, "harmed": 109, "missed_rescue": 2554, "avoided_harm": 1332, "selected_both_correct": 1461, "selected_both_wrong": 102}, {"budget": 0.13, "threshold": -0.33121857047080994, "operator": "ge", "actual_fraction": 0.1299458851641015, "metric": 0.5553447185325743, "rescued": 46, "harmed": 120, "missed_rescue": 2543, "avoided_harm": 1321, "selected_both_correct": 1562, "selected_both_wrong": 121}, {"budget": 0.14, "threshold": -0.33570271730422974, "operator": "ge", "actual_fraction": 0.13999578325954037, "metric": 0.5554149975402347, "rescued": 57, "harmed": 130, "missed_rescue": 2532, "avoided_harm": 1311, "selected_both_correct": 1662, "selected_both_wrong": 143}, {"budget": 0.15, "threshold": -0.3407592475414276, "operator": "ge", "actual_fraction": 0.14997540234731885, "metric": 0.5549933234942722, "rescued": 67, "harmed": 146, "missed_rescue": 2522, "avoided_harm": 1295, "selected_both_correct": 1754, "selected_both_wrong": 167}, {"budget": 0.16, "threshold": -0.34456855058670044, "operator": "ge", "actual_fraction": 0.15995502143509735, "metric": 0.5546419284559702, "rescued": 81, "harmed": 165, "missed_rescue": 2508, "avoided_harm": 1276, "selected_both_correct": 1844, "selected_both_wrong": 186}, {"budget": 0.17, "threshold": -0.3487326204776764, "operator": "ge", "actual_fraction": 0.16993464052287582, "metric": 0.5551338815095931, "rescued": 99, "harmed": 176, "missed_rescue": 2490, "avoided_harm": 1265, "selected_both_correct": 1931, "selected_both_wrong": 212}, {"budget": 0.18, "threshold": -0.35280776023864746, "operator": "ge", "actual_fraction": 0.1799845386183147, "metric": 0.5553447185325743, "rescued": 114, "harmed": 188, "missed_rescue": 2475, "avoided_harm": 1253, "selected_both_correct": 2017, "selected_both_wrong": 242}, {"budget": 0.19, "threshold": -0.3564819097518921, "operator": "ge", "actual_fraction": 0.18996415770609318, "metric": 0.555274439524914, "rescued": 126, "harmed": 201, "missed_rescue": 2463, "avoided_harm": 1240, "selected_both_correct": 2103, "selected_both_wrong": 273}, {"budget": 0.2, "threshold": -0.35992002487182617, "operator": "ge", "actual_fraction": 0.19994377679387168, "metric": 0.5554852765478951, "rescued": 142, "harmed": 214, "missed_rescue": 2447, "avoided_harm": 1227, "selected_both_correct": 2186, "selected_both_wrong": 303}, {"budget": 0.21, "threshold": -0.36340460181236267, "operator": "ge", "actual_fraction": 0.20999367488931056, "metric": 0.5566097406704618, "rescued": 166, "harmed": 222, "missed_rescue": 2423, "avoided_harm": 1219, "selected_both_correct": 2269, "selected_both_wrong": 331}, {"budget": 0.22, "threshold": -0.3674520254135132, "operator": "ge", "actual_fraction": 0.21997329397708903, "metric": 0.5566800196781222, "rescued": 182, "harmed": 237, "missed_rescue": 2407, "avoided_harm": 1204, "selected_both_correct": 2349, "selected_both_wrong": 362}, {"budget": 0.23, "threshold": -0.37108901143074036, "operator": "ge", "actual_fraction": 0.22995291306486754, "metric": 0.556820577693443, "rescued": 197, "harmed": 250, "missed_rescue": 2392, "avoided_harm": 1191, "selected_both_correct": 2436, "selected_both_wrong": 389}, {"budget": 0.24, "threshold": -0.3745429813861847, "operator": "ge", "actual_fraction": 0.239932532152646, "metric": 0.5575233677700471, "rescued": 219, "harmed": 262, "missed_rescue": 2370, "avoided_harm": 1179, "selected_both_correct": 2514, "selected_both_wrong": 419}, {"budget": 0.25, "threshold": -0.3773327171802521, "operator": "ge", "actual_fraction": 0.2499824302480849, "metric": 0.5572422517394054, "rescued": 233, "harmed": 280, "missed_rescue": 2356, "avoided_harm": 1161, "selected_both_correct": 2590, "selected_both_wrong": 454}, {"budget": 0.26, "threshold": -0.3803459703922272, "operator": "ge", "actual_fraction": 0.25996204933586337, "metric": 0.5584369948696324, "rescued": 260, "harmed": 290, "missed_rescue": 2329, "avoided_harm": 1151, "selected_both_correct": 2663, "selected_both_wrong": 486}, {"budget": 0.27, "threshold": -0.38361650705337524, "operator": "ge", "actual_fraction": 0.26994166842364187, "metric": 0.5585775528849533, "rescued": 278, "harmed": 306, "missed_rescue": 2311, "avoided_harm": 1135, "selected_both_correct": 2737, "selected_both_wrong": 520}, {"budget": 0.28, "threshold": -0.38652583956718445, "operator": "ge", "actual_fraction": 0.27999156651908075, "metric": 0.5589992269309157, "rescued": 298, "harmed": 320, "missed_rescue": 2291, "avoided_harm": 1121, "selected_both_correct": 2802, "selected_both_wrong": 564}, {"budget": 0.29, "threshold": -0.38952407240867615, "operator": "ge", "actual_fraction": 0.28997118560685925, "metric": 0.5599831330381615, "rescued": 323, "harmed": 331, "missed_rescue": 2266, "avoided_harm": 1110, "selected_both_correct": 2873, "selected_both_wrong": 599}, {"budget": 0.3, "threshold": -0.39262744784355164, "operator": "ge", "actual_fraction": 0.2999508046946377, "metric": 0.5608264811300865, "rescued": 348, "harmed": 344, "missed_rescue": 2241, "avoided_harm": 1097, "selected_both_correct": 2941, "selected_both_wrong": 635}, {"budget": 0.31, "threshold": -0.3954668939113617, "operator": "ge", "actual_fraction": 0.3099304237824162, "metric": 0.5613184341837093, "rescued": 370, "harmed": 359, "missed_rescue": 2219, "avoided_harm": 1082, "selected_both_correct": 3011, "selected_both_wrong": 670}, {"budget": 0.32, "threshold": -0.3986983895301819, "operator": "ge", "actual_fraction": 0.3199803218778551, "metric": 0.5613887131913697, "rescued": 394, "harmed": 382, "missed_rescue": 2195, "avoided_harm": 1059, "selected_both_correct": 3076, "selected_both_wrong": 701}, {"budget": 0.33, "threshold": -0.4014112949371338, "operator": "ge", "actual_fraction": 0.3299599409656336, "metric": 0.5616698292220114, "rescued": 415, "harmed": 399, "missed_rescue": 2174, "avoided_harm": 1042, "selected_both_correct": 3145, "selected_both_wrong": 736}, {"budget": 0.34, "threshold": -0.4042409658432007, "operator": "ge", "actual_fraction": 0.339939560053412, "metric": 0.562302340290955, "rescued": 437, "harmed": 412, "missed_rescue": 2152, "avoided_harm": 1029, "selected_both_correct": 3220, "selected_both_wrong": 768}, {"budget": 0.35, "threshold": -0.40724706649780273, "operator": "ge", "actual_fraction": 0.34998945814885096, "metric": 0.5628645723522384, "rescued": 461, "harmed": 428, "missed_rescue": 2128, "avoided_harm": 1013, "selected_both_correct": 3288, "selected_both_wrong": 803}, {"budget": 0.36, "threshold": -0.40968337655067444, "operator": "ge", "actual_fraction": 0.3599690772366294, "metric": 0.5624428983062759, "rescued": 478, "harmed": 451, "missed_rescue": 2111, "avoided_harm": 990, "selected_both_correct": 3361, "selected_both_wrong": 832}, {"budget": 0.37, "threshold": -0.4123799502849579, "operator": "ge", "actual_fraction": 0.3699486963244079, "metric": 0.5635673624288425, "rescued": 508, "harmed": 465, "missed_rescue": 2081, "avoided_harm": 976, "selected_both_correct": 3425, "selected_both_wrong": 866}, {"budget": 0.38, "threshold": -0.41514647006988525, "operator": "ge", "actual_fraction": 0.3799985944198468, "metric": 0.5650432215897112, "rescued": 538, "harmed": 474, "missed_rescue": 2051, "avoided_harm": 967, "selected_both_correct": 3487, "selected_both_wrong": 908}, {"budget": 0.39, "threshold": -0.4179389476776123, "operator": "ge", "actual_fraction": 0.3899782135076253, "metric": 0.565183779605032, "rescued": 557, "harmed": 491, "missed_rescue": 2032, "avoided_harm": 950, "selected_both_correct": 3556, "selected_both_wrong": 945}, {"budget": 0.4, "threshold": -0.42099592089653015, "operator": "ge", "actual_fraction": 0.39995783259540374, "metric": 0.5658865696816361, "rescued": 580, "harmed": 504, "missed_rescue": 2009, "avoided_harm": 937, "selected_both_correct": 3625, "selected_both_wrong": 982}, {"budget": 0.41, "threshold": -0.4243226647377014, "operator": "ge", "actual_fraction": 0.40993745168318224, "metric": 0.5672218708271839, "rescued": 614, "harmed": 519, "missed_rescue": 1975, "avoided_harm": 922, "selected_both_correct": 3695, "selected_both_wrong": 1005}, {"budget": 0.42, "threshold": -0.4275209903717041, "operator": "ge", "actual_fraction": 0.4199873497786211, "metric": 0.5686977299880526, "rescued": 645, "harmed": 529, "missed_rescue": 1944, "avoided_harm": 912, "selected_both_correct": 3759, "selected_both_wrong": 1043}, {"budget": 0.43, "threshold": -0.4307159185409546, "operator": "ge", "actual_fraction": 0.4299669688663996, "metric": 0.5695410780799776, "rescued": 672, "harmed": 544, "missed_rescue": 1917, "avoided_harm": 897, "selected_both_correct": 3828, "selected_both_wrong": 1074}, {"budget": 0.44, "threshold": -0.4336964190006256, "operator": "ge", "actual_fraction": 0.43994658795417807, "metric": 0.5705952631948837, "rescued": 702, "harmed": 559, "missed_rescue": 1887, "avoided_harm": 882, "selected_both_correct": 3892, "selected_both_wrong": 1107}, {"budget": 0.45, "threshold": -0.4373021125793457, "operator": "ge", "actual_fraction": 0.449996486049617, "metric": 0.5709466582331858, "rescued": 725, "harmed": 577, "missed_rescue": 1864, "avoided_harm": 864, "selected_both_correct": 3955, "selected_both_wrong": 1146}, {"budget": 0.46, "threshold": -0.4404309391975403, "operator": "ge", "actual_fraction": 0.45997610513739545, "metric": 0.5722116803710732, "rescued": 759, "harmed": 593, "missed_rescue": 1830, "avoided_harm": 848, "selected_both_correct": 4019, "selected_both_wrong": 1174}, {"budget": 0.47, "threshold": -0.44344601035118103, "operator": "ge", "actual_fraction": 0.46995572422517395, "metric": 0.5731955864783189, "rescued": 784, "harmed": 604, "missed_rescue": 1805, "avoided_harm": 837, "selected_both_correct": 4091, "selected_both_wrong": 1208}, {"budget": 0.48, "threshold": -0.4465356171131134, "operator": "ge", "actual_fraction": 0.4799353433129524, "metric": 0.5732658654859794, "rescued": 805, "harmed": 624, "missed_rescue": 1784, "avoided_harm": 817, "selected_both_correct": 4159, "selected_both_wrong": 1241}, {"budget": 0.49, "threshold": -0.44965147972106934, "operator": "ge", "actual_fraction": 0.48998524140839134, "metric": 0.5741794925855647, "rescued": 836, "harmed": 642, "missed_rescue": 1753, "avoided_harm": 799, "selected_both_correct": 4222, "selected_both_wrong": 1272}, {"budget": 0.5, "threshold": -0.453143447637558, "operator": "ge", "actual_fraction": 0.4999648604961698, "metric": 0.574741724646848, "rescued": 860, "harmed": 658, "missed_rescue": 1729, "avoided_harm": 783, "selected_both_correct": 4278, "selected_both_wrong": 1318}, {"budget": 0.51, "threshold": -0.4565357565879822, "operator": "ge", "actual_fraction": 0.5099444795839483, "metric": 0.5757256307540938, "rescued": 892, "harmed": 676, "missed_rescue": 1697, "avoided_harm": 765, "selected_both_correct": 4336, "selected_both_wrong": 1352}, {"budget": 0.52, "threshold": -0.45961302518844604, "operator": "ge", "actual_fraction": 0.5199943776793872, "metric": 0.5763581418230375, "rescued": 916, "harmed": 691, "missed_rescue": 1673, "avoided_harm": 750, "selected_both_correct": 4403, "selected_both_wrong": 1389}, {"budget": 0.53, "threshold": -0.46368208527565, "operator": "ge", "actual_fraction": 0.5299739967671656, "metric": 0.5774826059456041, "rescued": 948, "harmed": 707, "missed_rescue": 1641, "avoided_harm": 734, "selected_both_correct": 4468, "selected_both_wrong": 1418}, {"budget": 0.54, "threshold": -0.46793216466903687, "operator": "ge", "actual_fraction": 0.5399536158549442, "metric": 0.5792395811371144, "rescued": 986, "harmed": 720, "missed_rescue": 1603, "avoided_harm": 721, "selected_both_correct": 4528, "selected_both_wrong": 1449}, {"budget": 0.55, "threshold": -0.4718930721282959, "operator": "ge", "actual_fraction": 0.5499332349427226, "metric": 0.5795909761754164, "rescued": 1010, "harmed": 739, "missed_rescue": 1579, "avoided_harm": 702, "selected_both_correct": 4579, "selected_both_wrong": 1497}, {"budget": 0.56, "threshold": -0.47629430890083313, "operator": "ge", "actual_fraction": 0.5599831330381615, "metric": 0.5794504181600956, "rescued": 1031, "harmed": 762, "missed_rescue": 1558, "avoided_harm": 679, "selected_both_correct": 4644, "selected_both_wrong": 1531}, {"budget": 0.57, "threshold": -0.4799485504627228, "operator": "ge", "actual_fraction": 0.5699627521259399, "metric": 0.5802234872443601, "rescued": 1060, "harmed": 780, "missed_rescue": 1529, "avoided_harm": 661, "selected_both_correct": 4700, "selected_both_wrong": 1570}, {"budget": 0.58, "threshold": -0.4838995635509491, "operator": "ge", "actual_fraction": 0.5799423712137185, "metric": 0.5812776723592663, "rescued": 1093, "harmed": 798, "missed_rescue": 1496, "avoided_harm": 643, "selected_both_correct": 4756, "selected_both_wrong": 1605}, {"budget": 0.59, "threshold": -0.4886912405490875, "operator": "ge", "actual_fraction": 0.5899922693091574, "metric": 0.5816290673975684, "rescued": 1120, "harmed": 820, "missed_rescue": 1469, "avoided_harm": 621, "selected_both_correct": 4815, "selected_both_wrong": 1640}, {"budget": 0.6, "threshold": -0.49332454800605774, "operator": "ge", "actual_fraction": 0.5999718883969358, "metric": 0.5819804624358704, "rescued": 1142, "harmed": 837, "missed_rescue": 1447, "avoided_harm": 604, "selected_both_correct": 4882, "selected_both_wrong": 1676}, {"budget": 0.61, "threshold": -0.49750789999961853, "operator": "ge", "actual_fraction": 0.6099515074847143, "metric": 0.582261578466512, "rescued": 1166, "harmed": 857, "missed_rescue": 1423, "avoided_harm": 584, "selected_both_correct": 4941, "selected_both_wrong": 1715}, {"budget": 0.62, "threshold": -0.5024909377098083, "operator": "ge", "actual_fraction": 0.6199311265724928, "metric": 0.5823318574741725, "rescued": 1183, "harmed": 873, "missed_rescue": 1406, "avoided_harm": 568, "selected_both_correct": 5010, "selected_both_wrong": 1755}, {"budget": 0.63, "threshold": -0.5067122578620911, "operator": "ge", "actual_fraction": 0.6299810246679317, "metric": 0.5824021364818329, "rescued": 1205, "harmed": 894, "missed_rescue": 1384, "avoided_harm": 547, "selected_both_correct": 5076, "selected_both_wrong": 1789}, {"budget": 0.64, "threshold": -0.512323796749115, "operator": "ge", "actual_fraction": 0.6399606437557102, "metric": 0.5825426944971537, "rescued": 1223, "harmed": 910, "missed_rescue": 1366, "avoided_harm": 531, "selected_both_correct": 5140, "selected_both_wrong": 1833}, {"budget": 0.65, "threshold": -0.5170255303382874, "operator": "ge", "actual_fraction": 0.6499402628434886, "metric": 0.5833157635814182, "rescued": 1250, "harmed": 926, "missed_rescue": 1339, "avoided_harm": 515, "selected_both_correct": 5200, "selected_both_wrong": 1872}, {"budget": 0.66, "threshold": -0.5215396881103516, "operator": "ge", "actual_fraction": 0.6599901609389276, "metric": 0.5833157635814182, "rescued": 1269, "harmed": 945, "missed_rescue": 1320, "avoided_harm": 496, "selected_both_correct": 5264, "selected_both_wrong": 1913}, {"budget": 0.67, "threshold": -0.5264327526092529, "operator": "ge", "actual_fraction": 0.669969780026706, "metric": 0.5826129735048141, "rescued": 1288, "harmed": 974, "missed_rescue": 1301, "avoided_harm": 467, "selected_both_correct": 5321, "selected_both_wrong": 1950}, {"budget": 0.68, "threshold": -0.5312389135360718, "operator": "ge", "actual_fraction": 0.6799493991144845, "metric": 0.583104926558437, "rescued": 1316, "harmed": 995, "missed_rescue": 1273, "avoided_harm": 446, "selected_both_correct": 5379, "selected_both_wrong": 1985}, {"budget": 0.69, "threshold": -0.5368650555610657, "operator": "ge", "actual_fraction": 0.6899992972099234, "metric": 0.5835266006043994, "rescued": 1336, "harmed": 1009, "missed_rescue": 1253, "avoided_harm": 432, "selected_both_correct": 5445, "selected_both_wrong": 2028}, {"budget": 0.7, "threshold": -0.5431069135665894, "operator": "ge", "actual_fraction": 0.6999789162977019, "metric": 0.5843699486963244, "rescued": 1363, "harmed": 1024, "missed_rescue": 1226, "avoided_harm": 417, "selected_both_correct": 5492, "selected_both_wrong": 2081}, {"budget": 0.71, "threshold": -0.5487136244773865, "operator": "ge", "actual_fraction": 0.7099585353854804, "metric": 0.5852835757959097, "rescued": 1393, "harmed": 1041, "missed_rescue": 1196, "avoided_harm": 400, "selected_both_correct": 5550, "selected_both_wrong": 2118}, {"budget": 0.72, "threshold": -0.5549414753913879, "operator": "ge", "actual_fraction": 0.7199381544732588, "metric": 0.5849321807576077, "rescued": 1415, "harmed": 1068, "missed_rescue": 1174, "avoided_harm": 373, "selected_both_correct": 5611, "selected_both_wrong": 2150}, {"budget": 0.73, "threshold": -0.5604677200317383, "operator": "ge", "actual_fraction": 0.7299880525686977, "metric": 0.5845105067116453, "rescued": 1432, "harmed": 1091, "missed_rescue": 1157, "avoided_harm": 350, "selected_both_correct": 5683, "selected_both_wrong": 2181}, {"budget": 0.74, "threshold": -0.5666090250015259, "operator": "ge", "actual_fraction": 0.7399676716564763, "metric": 0.584651064726966, "rescued": 1452, "harmed": 1109, "missed_rescue": 1137, "avoided_harm": 332, "selected_both_correct": 5738, "selected_both_wrong": 2230}, {"budget": 0.75, "threshold": -0.574927568435669, "operator": "ge", "actual_fraction": 0.7499472907442547, "metric": 0.5847213437346265, "rescued": 1473, "harmed": 1129, "missed_rescue": 1116, "avoided_harm": 312, "selected_both_correct": 5796, "selected_both_wrong": 2273}, {"budget": 0.76, "threshold": -0.5812664031982422, "operator": "ge", "actual_fraction": 0.7599971888396936, "metric": 0.5850727387729285, "rescued": 1499, "harmed": 1150, "missed_rescue": 1090, "avoided_harm": 291, "selected_both_correct": 5856, "selected_both_wrong": 2309}, {"budget": 0.77, "threshold": -0.5883752107620239, "operator": "ge", "actual_fraction": 0.769976807927472, "metric": 0.5853538548035702, "rescued": 1522, "harmed": 1169, "missed_rescue": 1067, "avoided_harm": 272, "selected_both_correct": 5923, "selected_both_wrong": 2342}, {"budget": 0.78, "threshold": -0.5972744226455688, "operator": "ge", "actual_fraction": 0.7799564270152506, "metric": 0.5852132967882494, "rescued": 1540, "harmed": 1189, "missed_rescue": 1049, "avoided_harm": 252, "selected_both_correct": 5976, "selected_both_wrong": 2393}, {"budget": 0.79, "threshold": -0.6058458089828491, "operator": "ge", "actual_fraction": 0.789936046103029, "metric": 0.5857755288495327, "rescued": 1569, "harmed": 1210, "missed_rescue": 1020, "avoided_harm": 231, "selected_both_correct": 6037, "selected_both_wrong": 2424}, {"budget": 0.8, "threshold": -0.6140543818473816, "operator": "ge", "actual_fraction": 0.7999859441984679, "metric": 0.5853538548035702, "rescued": 1596, "harmed": 1243, "missed_rescue": 993, "avoided_harm": 198, "selected_both_correct": 6083, "selected_both_wrong": 2461}, {"budget": 0.81, "threshold": -0.6230268478393555, "operator": "ge", "actual_fraction": 0.8099655632862464, "metric": 0.5852132967882494, "rescued": 1619, "harmed": 1268, "missed_rescue": 970, "avoided_harm": 173, "selected_both_correct": 6141, "selected_both_wrong": 2497}, {"budget": 0.82, "threshold": -0.6330366134643555, "operator": "ge", "actual_fraction": 0.8199451823740249, "metric": 0.5852835757959097, "rescued": 1639, "harmed": 1287, "missed_rescue": 950, "avoided_harm": 154, "selected_both_correct": 6203, "selected_both_wrong": 2538}, {"budget": 0.83, "threshold": -0.6429333686828613, "operator": "ge", "actual_fraction": 0.8299950804694638, "metric": 0.584299669688664, "rescued": 1655, "harmed": 1317, "missed_rescue": 934, "avoided_harm": 124, "selected_both_correct": 6256, "selected_both_wrong": 2582}, {"budget": 0.84, "threshold": -0.6513019800186157, "operator": "ge", "actual_fraction": 0.8399746995572422, "metric": 0.5847213437346265, "rescued": 1679, "harmed": 1335, "missed_rescue": 910, "avoided_harm": 106, "selected_both_correct": 6315, "selected_both_wrong": 2623}, {"budget": 0.85, "threshold": -0.6624053120613098, "operator": "ge", "actual_fraction": 0.8499543186450207, "metric": 0.5849321807576077, "rescued": 1702, "harmed": 1355, "missed_rescue": 887, "avoided_harm": 86, "selected_both_correct": 6367, "selected_both_wrong": 2670}, {"budget": 0.86, "threshold": -0.6759263277053833, "operator": "ge", "actual_fraction": 0.8599339377327992, "metric": 0.584651064726966, "rescued": 1724, "harmed": 1381, "missed_rescue": 865, "avoided_harm": 60, "selected_both_correct": 6420, "selected_both_wrong": 2711}, {"budget": 0.87, "threshold": -0.6904275417327881, "operator": "ge", "actual_fraction": 0.8699838358282381, "metric": 0.5847213437346265, "rescued": 1741, "harmed": 1397, "missed_rescue": 848, "avoided_harm": 44, "selected_both_correct": 6489, "selected_both_wrong": 2752}, {"budget": 0.88, "threshold": -0.7045450210571289, "operator": "ge", "actual_fraction": 0.8799634549160166, "metric": 0.5859160868648534, "rescued": 1778, "harmed": 1417, "missed_rescue": 811, "avoided_harm": 24, "selected_both_correct": 6525, "selected_both_wrong": 2801}, {"budget": 0.89, "threshold": -0.7205488681793213, "operator": "ge", "actual_fraction": 0.889943074003795, "metric": 0.5913275704547052, "rescued": 1856, "harmed": 1418, "missed_rescue": 733, "avoided_harm": 23, "selected_both_correct": 6525, "selected_both_wrong": 2864}, {"budget": 0.9, "threshold": -0.7357243299484253, "operator": "ge", "actual_fraction": 0.899992972099234, "metric": 0.5963876590062548, "rescued": 1928, "harmed": 1418, "missed_rescue": 661, "avoided_harm": 23, "selected_both_correct": 6525, "selected_both_wrong": 2935}, {"budget": 0.91, "threshold": -0.7513164281845093, "operator": "ge", "actual_fraction": 0.9099725911870125, "metric": 0.6009557945041816, "rescued": 1993, "harmed": 1418, "missed_rescue": 596, "avoided_harm": 23, "selected_both_correct": 6525, "selected_both_wrong": 3012}, {"budget": 0.92, "threshold": -0.7671180963516235, "operator": "ge", "actual_fraction": 0.9199522102747909, "metric": 0.606156441071052, "rescued": 2069, "harmed": 1420, "missed_rescue": 520, "avoided_harm": 21, "selected_both_correct": 6525, "selected_both_wrong": 3076}, {"budget": 0.93, "threshold": -0.7833515405654907, "operator": "ge", "actual_fraction": 0.9299318293625694, "metric": 0.6112165296226018, "rescued": 2142, "harmed": 1421, "missed_rescue": 447, "avoided_harm": 20, "selected_both_correct": 6526, "selected_both_wrong": 3143}, {"budget": 0.94, "threshold": -0.8013419508934021, "operator": "ge", "actual_fraction": 0.9399817274580083, "metric": 0.6155738280975472, "rescued": 2206, "harmed": 1423, "missed_rescue": 383, "avoided_harm": 18, "selected_both_correct": 6527, "selected_both_wrong": 3219}, {"budget": 0.95, "threshold": -0.8188987374305725, "operator": "ge", "actual_fraction": 0.9499613465457868, "metric": 0.6207744746644177, "rescued": 2280, "harmed": 1423, "missed_rescue": 309, "avoided_harm": 18, "selected_both_correct": 6528, "selected_both_wrong": 3286}, {"budget": 0.96, "threshold": -0.8395025134086609, "operator": "ge", "actual_fraction": 0.9599409656335652, "metric": 0.6249912151240424, "rescued": 2344, "harmed": 1427, "missed_rescue": 245, "avoided_harm": 14, "selected_both_correct": 6529, "selected_both_wrong": 3359}, {"budget": 0.97, "threshold": -0.8628664612770081, "operator": "ge", "actual_fraction": 0.9699908637290041, "metric": 0.6291376765760067, "rescued": 2405, "harmed": 1429, "missed_rescue": 184, "avoided_harm": 12, "selected_both_correct": 6531, "selected_both_wrong": 3437}, {"budget": 0.98, "threshold": -0.8857419490814209, "operator": "ge", "actual_fraction": 0.9799704828167827, "metric": 0.6332138590203107, "rescued": 2467, "harmed": 1433, "missed_rescue": 122, "avoided_harm": 8, "selected_both_correct": 6533, "selected_both_wrong": 3511}, {"budget": 0.99, "threshold": -0.9162307977676392, "operator": "ge", "actual_fraction": 0.9899501019045611, "metric": 0.6372197624569541, "rescued": 2529, "harmed": 1438, "missed_rescue": 60, "avoided_harm": 3, "selected_both_correct": 6534, "selected_both_wrong": 3585}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.6412256658935975, "rescued": 2589, "harmed": 1441, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 6535, "selected_both_wrong": 3664}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.26688029946402747, "rescued": 0, "harmed": 0, "missed_rescue": 10723, "avoided_harm": 2500, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.1584138125181198, "operator": "ge", "actual_fraction": 0.029095652667101495, "metric": 0.2671071661514902, "rescued": 14, "harmed": 6, "missed_rescue": 10709, "avoided_harm": 2494, "selected_both_correct": 996, "selected_both_wrong": 10}, {"budget": 0.02, "threshold": -0.21403978765010834, "operator": "ge", "actual_fraction": 0.04622408757054136, "metric": 0.2674474661826844, "rescued": 34, "harmed": 14, "missed_rescue": 10689, "avoided_harm": 2486, "selected_both_correct": 1522, "selected_both_wrong": 60}, {"budget": 0.03, "threshold": -0.24253316223621368, "operator": "ge", "actual_fraction": 0.056149505147037974, "metric": 0.26801463290134137, "rescued": 58, "harmed": 18, "missed_rescue": 10665, "avoided_harm": 2482, "selected_both_correct": 1796, "selected_both_wrong": 108}, {"budget": 0.04, "threshold": -0.2599165141582489, "operator": "ge", "actual_fraction": 0.06372118084110824, "metric": 0.2684683662762669, "rescued": 81, "harmed": 25, "missed_rescue": 10642, "avoided_harm": 2475, "selected_both_correct": 1988, "selected_both_wrong": 153}, {"budget": 0.05, "threshold": -0.2725968658924103, "operator": "ge", "actual_fraction": 0.06882568130902078, "metric": 0.26883702464339393, "rescued": 101, "harmed": 32, "missed_rescue": 10622, "avoided_harm": 2468, "selected_both_correct": 2095, "selected_both_wrong": 199}, {"budget": 0.06, "threshold": -0.2842792272567749, "operator": "ge", "actual_fraction": 0.07475257351898591, "metric": 0.26920568301052095, "rescued": 119, "harmed": 37, "missed_rescue": 10604, "avoided_harm": 2463, "selected_both_correct": 2209, "selected_both_wrong": 271}, {"budget": 0.07, "threshold": -0.2935555577278137, "operator": "ge", "actual_fraction": 0.08033916569775686, "metric": 0.2694609080339166, "rescued": 140, "harmed": 49, "missed_rescue": 10583, "avoided_harm": 2451, "selected_both_correct": 2306, "selected_both_wrong": 338}, {"budget": 0.08, "threshold": -0.30114513635635376, "operator": "ge", "actual_fraction": 0.08632277457958767, "metric": 0.269942999744775, "rescued": 170, "harmed": 62, "missed_rescue": 10553, "avoided_harm": 2438, "selected_both_correct": 2387, "selected_both_wrong": 425}, {"budget": 0.09, "threshold": -0.308601438999176, "operator": "ge", "actual_fraction": 0.09227802512548564, "metric": 0.27028329977596915, "rescued": 200, "harmed": 80, "missed_rescue": 10523, "avoided_harm": 2420, "selected_both_correct": 2473, "selected_both_wrong": 501}, {"budget": 0.1, "threshold": -0.3146189749240875, "operator": "ge", "actual_fraction": 0.09783625896832374, "metric": 0.27076539148682754, "rescued": 226, "harmed": 89, "missed_rescue": 10497, "avoided_harm": 2411, "selected_both_correct": 2547, "selected_both_wrong": 588}, {"budget": 0.11, "threshold": -0.32043084502220154, "operator": "ge", "actual_fraction": 0.10362135949862462, "metric": 0.2713325582054845, "rescued": 258, "harmed": 101, "missed_rescue": 10465, "avoided_harm": 2399, "selected_both_correct": 2611, "selected_both_wrong": 684}, {"budget": 0.12, "threshold": -0.32644838094711304, "operator": "ge", "actual_fraction": 0.11031392677877662, "metric": 0.27221166661940277, "rescued": 305, "harmed": 117, "missed_rescue": 10418, "avoided_harm": 2383, "selected_both_correct": 2671, "selected_both_wrong": 797}, {"budget": 0.13, "threshold": -0.33121857047080994, "operator": "ge", "actual_fraction": 0.11658111901993591, "metric": 0.2733176417207838, "rescued": 353, "harmed": 126, "missed_rescue": 10370, "avoided_harm": 2374, "selected_both_correct": 2743, "selected_both_wrong": 889}, {"budget": 0.14, "threshold": -0.33570271730422974, "operator": "ge", "actual_fraction": 0.12262144457363242, "metric": 0.2742534668065678, "rescued": 399, "harmed": 139, "missed_rescue": 10324, "avoided_harm": 2361, "selected_both_correct": 2797, "selected_both_wrong": 989}, {"budget": 0.15, "threshold": -0.3407592475414276, "operator": "ge", "actual_fraction": 0.13024983693956838, "metric": 0.27535944190794887, "rescued": 455, "harmed": 156, "missed_rescue": 10268, "avoided_harm": 2344, "selected_both_correct": 2866, "selected_both_wrong": 1116}, {"budget": 0.16, "threshold": -0.34456855058670044, "operator": "ge", "actual_fraction": 0.13714091257125033, "metric": 0.2768907920483226, "rescued": 526, "harmed": 173, "missed_rescue": 10197, "avoided_harm": 2327, "selected_both_correct": 2922, "selected_both_wrong": 1215}, {"budget": 0.17, "threshold": -0.3487326204776764, "operator": "ge", "actual_fraction": 0.143521538156141, "metric": 0.27833706718089785, "rescued": 589, "harmed": 185, "missed_rescue": 10134, "avoided_harm": 2315, "selected_both_correct": 2969, "selected_both_wrong": 1318}, {"budget": 0.18, "threshold": -0.35280776023864746, "operator": "ge", "actual_fraction": 0.15095142217054702, "metric": 0.2800669256728015, "rescued": 665, "harmed": 200, "missed_rescue": 10058, "avoided_harm": 2300, "selected_both_correct": 3012, "selected_both_wrong": 1446}, {"budget": 0.19, "threshold": -0.3564819097518921, "operator": "ge", "actual_fraction": 0.15823951450528884, "metric": 0.28097439242265265, "rescued": 720, "harmed": 223, "missed_rescue": 10003, "avoided_harm": 2277, "selected_both_correct": 3066, "selected_both_wrong": 1571}, {"budget": 0.2, "threshold": -0.35992002487182617, "operator": "ge", "actual_fraction": 0.16496044012137367, "metric": 0.2826191759067578, "rescued": 797, "harmed": 242, "missed_rescue": 9926, "avoided_harm": 2258, "selected_both_correct": 3110, "selected_both_wrong": 1668}, {"budget": 0.21, "threshold": -0.36340460181236267, "operator": "ge", "actual_fraction": 0.17290077418257097, "metric": 0.28457590108612424, "rescued": 880, "harmed": 256, "missed_rescue": 9843, "avoided_harm": 2244, "selected_both_correct": 3166, "selected_both_wrong": 1795}, {"budget": 0.22, "threshold": -0.3674520254135132, "operator": "ge", "actual_fraction": 0.1829679834387318, "metric": 0.28667441794515497, "rescued": 979, "harmed": 281, "missed_rescue": 9744, "avoided_harm": 2219, "selected_both_correct": 3230, "selected_both_wrong": 1962}, {"budget": 0.23, "threshold": -0.37108901143074036, "operator": "ge", "actual_fraction": 0.19122025919519042, "metric": 0.28814905141366304, "rescued": 1052, "harmed": 302, "missed_rescue": 9671, "avoided_harm": 2198, "selected_both_correct": 3292, "selected_both_wrong": 2097}, {"budget": 0.24, "threshold": -0.3745429813861847, "operator": "ge", "actual_fraction": 0.1999262683265746, "metric": 0.28993562657743244, "rescued": 1139, "harmed": 326, "missed_rescue": 9584, "avoided_harm": 2174, "selected_both_correct": 3343, "selected_both_wrong": 2242}, {"budget": 0.25, "threshold": -0.3773327171802521, "operator": "ge", "actual_fraction": 0.2080651107393018, "metric": 0.2917789184130675, "rescued": 1224, "harmed": 346, "missed_rescue": 9499, "avoided_harm": 2154, "selected_both_correct": 3394, "selected_both_wrong": 2373}, {"budget": 0.26, "threshold": -0.3803459703922272, "operator": "ge", "actual_fraction": 0.21603380313643195, "metric": 0.29379236026429967, "rescued": 1315, "harmed": 366, "missed_rescue": 9408, "avoided_harm": 2134, "selected_both_correct": 3435, "selected_both_wrong": 2502}, {"budget": 0.27, "threshold": -0.38361650705337524, "operator": "ge", "actual_fraction": 0.2243144372288234, "metric": 0.29623117715452457, "rescued": 1420, "harmed": 385, "missed_rescue": 9303, "avoided_harm": 2115, "selected_both_correct": 3485, "selected_both_wrong": 2620}, {"budget": 0.28, "threshold": -0.38652583956718445, "operator": "ge", "actual_fraction": 0.2324249213056178, "metric": 0.29813118566202534, "rescued": 1512, "harmed": 410, "missed_rescue": 9211, "avoided_harm": 2090, "selected_both_correct": 3520, "selected_both_wrong": 2754}, {"budget": 0.29, "threshold": -0.38952407240867615, "operator": "ge", "actual_fraction": 0.2419816805149874, "metric": 0.3005983608881831, "rescued": 1618, "harmed": 429, "missed_rescue": 9105, "avoided_harm": 2071, "selected_both_correct": 3587, "selected_both_wrong": 2899}, {"budget": 0.3, "threshold": -0.39262744784355164, "operator": "ge", "actual_fraction": 0.2512832147009613, "metric": 0.3034625528174007, "rescued": 1743, "harmed": 453, "missed_rescue": 8980, "avoided_harm": 2047, "selected_both_correct": 3641, "selected_both_wrong": 3024}, {"budget": 0.31, "threshold": -0.3954668939113617, "operator": "ge", "actual_fraction": 0.2596772821370842, "metric": 0.30561778634829706, "rescued": 1837, "harmed": 471, "missed_rescue": 8886, "avoided_harm": 2029, "selected_both_correct": 3675, "selected_both_wrong": 3174}, {"budget": 0.32, "threshold": -0.3986983895301819, "operator": "ge", "actual_fraction": 0.26985792473697645, "metric": 0.308793919972776, "rescued": 1971, "harmed": 493, "missed_rescue": 8752, "avoided_harm": 2007, "selected_both_correct": 3725, "selected_both_wrong": 3327}, {"budget": 0.33, "threshold": -0.4014112949371338, "operator": "ge", "actual_fraction": 0.27757139211071097, "metric": 0.31109094518333663, "rescued": 2067, "harmed": 508, "missed_rescue": 8656, "avoided_harm": 1992, "selected_both_correct": 3757, "selected_both_wrong": 3456}, {"budget": 0.34, "threshold": -0.4042409658432007, "operator": "ge", "actual_fraction": 0.28729830133567763, "metric": 0.3132178203783002, "rescued": 2174, "harmed": 540, "missed_rescue": 8549, "avoided_harm": 1960, "selected_both_correct": 3810, "selected_both_wrong": 3607}, {"budget": 0.35, "threshold": -0.40724706649780273, "operator": "ge", "actual_fraction": 0.29793267731049544, "metric": 0.31659246235430905, "rescued": 2314, "harmed": 561, "missed_rescue": 8409, "avoided_harm": 1939, "selected_both_correct": 3868, "selected_both_wrong": 3763}, {"budget": 0.36, "threshold": -0.40968337655067444, "operator": "ge", "actual_fraction": 0.3067237614496781, "metric": 0.31928650426792954, "rescued": 2427, "harmed": 579, "missed_rescue": 8296, "avoided_harm": 1921, "selected_both_correct": 3912, "selected_both_wrong": 3898}, {"budget": 0.37, "threshold": -0.4123799502849579, "operator": "ge", "actual_fraction": 0.3161387289793835, "metric": 0.32192382950968434, "rescued": 2547, "harmed": 606, "missed_rescue": 8176, "avoided_harm": 1894, "selected_both_correct": 3952, "selected_both_wrong": 4043}, {"budget": 0.38, "threshold": -0.41514647006988525, "operator": "ge", "actual_fraction": 0.32538354649349177, "metric": 0.3242492130561779, "rescued": 2654, "harmed": 631, "missed_rescue": 8069, "avoided_harm": 1869, "selected_both_correct": 4003, "selected_both_wrong": 4186}, {"budget": 0.39, "threshold": -0.4179389476776123, "operator": "ge", "actual_fraction": 0.3354507557496526, "metric": 0.3270283299775969, "rescued": 2782, "harmed": 661, "missed_rescue": 7941, "avoided_harm": 1839, "selected_both_correct": 4053, "selected_both_wrong": 4333}, {"budget": 0.4, "threshold": -0.42099592089653015, "operator": "ge", "actual_fraction": 0.34653886509939597, "metric": 0.33009103025834446, "rescued": 2921, "harmed": 692, "missed_rescue": 7802, "avoided_harm": 1808, "selected_both_correct": 4098, "selected_both_wrong": 4509}, {"budget": 0.41, "threshold": -0.4243226647377014, "operator": "ge", "actual_fraction": 0.3577404077928707, "metric": 0.33306865553129344, "rescued": 3055, "harmed": 721, "missed_rescue": 7668, "avoided_harm": 1779, "selected_both_correct": 4157, "selected_both_wrong": 4682}, {"budget": 0.42, "threshold": -0.4275209903717041, "operator": "ge", "actual_fraction": 0.3693673255253382, "metric": 0.33661344752289935, "rescued": 3203, "harmed": 744, "missed_rescue": 7520, "avoided_harm": 1756, "selected_both_correct": 4207, "selected_both_wrong": 4871}, {"budget": 0.43, "threshold": -0.4307159185409546, "operator": "ge", "actual_fraction": 0.38039871820321586, "metric": 0.33953435612398264, "rescued": 3331, "harmed": 769, "missed_rescue": 7392, "avoided_harm": 1731, "selected_both_correct": 4259, "selected_both_wrong": 5055}, {"budget": 0.44, "threshold": -0.4336964190006256, "operator": "ge", "actual_fraction": 0.39049428579530954, "metric": 0.3427388480843944, "rescued": 3465, "harmed": 790, "missed_rescue": 7258, "avoided_harm": 1710, "selected_both_correct": 4309, "selected_both_wrong": 5206}, {"budget": 0.45, "threshold": -0.4373021125793457, "operator": "ge", "actual_fraction": 0.4023764285511726, "metric": 0.34597169838073905, "rescued": 3606, "harmed": 817, "missed_rescue": 7117, "avoided_harm": 1683, "selected_both_correct": 4365, "selected_both_wrong": 5401}, {"budget": 0.46, "threshold": -0.4404309391975403, "operator": "ge", "actual_fraction": 0.41196154609647506, "metric": 0.3488358903099566, "rescued": 3729, "harmed": 839, "missed_rescue": 6994, "avoided_harm": 1661, "selected_both_correct": 4405, "selected_both_wrong": 5554}, {"budget": 0.47, "threshold": -0.44344601035118103, "operator": "ge", "actual_fraction": 0.42157502197771035, "metric": 0.35172844057510705, "rescued": 3850, "harmed": 858, "missed_rescue": 6873, "avoided_harm": 1642, "selected_both_correct": 4436, "selected_both_wrong": 5722}, {"budget": 0.48, "threshold": -0.4465356171131134, "operator": "ge", "actual_fraction": 0.43184073958540115, "metric": 0.35459263250432466, "rescued": 3979, "harmed": 886, "missed_rescue": 6744, "avoided_harm": 1614, "selected_both_correct": 4479, "selected_both_wrong": 5884}, {"budget": 0.49, "threshold": -0.44965147972106934, "operator": "ge", "actual_fraction": 0.44165272381816634, "metric": 0.35751354110540795, "rescued": 4106, "harmed": 910, "missed_rescue": 6617, "avoided_harm": 1590, "selected_both_correct": 4530, "selected_both_wrong": 6028}, {"budget": 0.5, "threshold": -0.453143447637558, "operator": "ge", "actual_fraction": 0.452315458128917, "metric": 0.3607463914017525, "rescued": 4242, "harmed": 932, "missed_rescue": 6481, "avoided_harm": 1568, "selected_both_correct": 4574, "selected_both_wrong": 6202}, {"budget": 0.51, "threshold": -0.4565357565879822, "operator": "ge", "actual_fraction": 0.4631483424552647, "metric": 0.36440461673708985, "rescued": 4392, "harmed": 953, "missed_rescue": 6331, "avoided_harm": 1547, "selected_both_correct": 4610, "selected_both_wrong": 6377}, {"budget": 0.52, "threshold": -0.45961302518844604, "operator": "ge", "actual_fraction": 0.47176927657885037, "metric": 0.36641805858832205, "rescued": 4484, "harmed": 974, "missed_rescue": 6239, "avoided_harm": 1526, "selected_both_correct": 4654, "selected_both_wrong": 6524}, {"budget": 0.53, "threshold": -0.46368208527565, "operator": "ge", "actual_fraction": 0.4840484360377733, "metric": 0.36970762555653236, "rescued": 4626, "harmed": 1000, "missed_rescue": 6097, "avoided_harm": 1500, "selected_both_correct": 4715, "selected_both_wrong": 6728}, {"budget": 0.54, "threshold": -0.46793216466903687, "operator": "ge", "actual_fraction": 0.495079828715651, "metric": 0.3731389842044069, "rescued": 4772, "harmed": 1025, "missed_rescue": 5951, "avoided_harm": 1475, "selected_both_correct": 4766, "selected_both_wrong": 6895}, {"budget": 0.55, "threshold": -0.4718930721282959, "operator": "ge", "actual_fraction": 0.5065365964325214, "metric": 0.37634347616481867, "rescued": 4911, "harmed": 1051, "missed_rescue": 5812, "avoided_harm": 1449, "selected_both_correct": 4826, "selected_both_wrong": 7074}, {"budget": 0.56, "threshold": -0.47629430890083313, "operator": "ge", "actual_fraction": 0.5183620225165188, "metric": 0.379434534781499, "rescued": 5056, "harmed": 1087, "missed_rescue": 5667, "avoided_harm": 1413, "selected_both_correct": 4873, "selected_both_wrong": 7263}, {"budget": 0.57, "threshold": -0.4799485504627228, "operator": "ge", "actual_fraction": 0.5273232566712985, "metric": 0.3818733516717239, "rescued": 5170, "harmed": 1115, "missed_rescue": 5553, "avoided_harm": 1385, "selected_both_correct": 4915, "selected_both_wrong": 7395}, {"budget": 0.58, "threshold": -0.4838995635509491, "operator": "ge", "actual_fraction": 0.5381561409976463, "metric": 0.3852763519836656, "rescued": 5317, "harmed": 1142, "missed_rescue": 5406, "avoided_harm": 1358, "selected_both_correct": 4957, "selected_both_wrong": 7561}, {"budget": 0.59, "threshold": -0.4886912405490875, "operator": "ge", "actual_fraction": 0.5503502254487707, "metric": 0.3888211439752715, "rescued": 5470, "harmed": 1170, "missed_rescue": 5253, "avoided_harm": 1330, "selected_both_correct": 4996, "selected_both_wrong": 7771}, {"budget": 0.6, "threshold": -0.49332454800605774, "operator": "ge", "actual_fraction": 0.5619487848453053, "metric": 0.39256444431840737, "rescued": 5631, "harmed": 1199, "missed_rescue": 5092, "avoided_harm": 1301, "selected_both_correct": 5037, "selected_both_wrong": 7949}, {"budget": 0.61, "threshold": -0.49750789999961853, "operator": "ge", "actual_fraction": 0.5722712191248618, "metric": 0.3955987862632221, "rescued": 5771, "harmed": 1232, "missed_rescue": 4952, "avoided_harm": 1268, "selected_both_correct": 5078, "selected_both_wrong": 8099}, {"budget": 0.62, "threshold": -0.5024909377098083, "operator": "ge", "actual_fraction": 0.5818563366701642, "metric": 0.39840626152057396, "rescued": 5893, "harmed": 1255, "missed_rescue": 4830, "avoided_harm": 1245, "selected_both_correct": 5118, "selected_both_wrong": 8252}, {"budget": 0.63, "threshold": -0.5067122578620911, "operator": "ge", "actual_fraction": 0.590817570824944, "metric": 0.4011003034341945, "rescued": 6008, "harmed": 1275, "missed_rescue": 4715, "avoided_harm": 1225, "selected_both_correct": 5160, "selected_both_wrong": 8391}, {"budget": 0.64, "threshold": -0.512323796749115, "operator": "ge", "actual_fraction": 0.604202705385248, "metric": 0.40532569548818875, "rescued": 6184, "harmed": 1302, "missed_rescue": 4539, "avoided_harm": 1198, "selected_both_correct": 5218, "selected_both_wrong": 8602}, {"budget": 0.65, "threshold": -0.5170255303382874, "operator": "ge", "actual_fraction": 0.6130505061962964, "metric": 0.4078495873862122, "rescued": 6296, "harmed": 1325, "missed_rescue": 4427, "avoided_harm": 1175, "selected_both_correct": 5258, "selected_both_wrong": 8739}, {"budget": 0.66, "threshold": -0.5215396881103516, "operator": "ge", "actual_fraction": 0.6222102487026061, "metric": 0.4103167626123699, "rescued": 6411, "harmed": 1353, "missed_rescue": 4312, "avoided_harm": 1147, "selected_both_correct": 5298, "selected_both_wrong": 8879}, {"budget": 0.67, "threshold": -0.5264327526092529, "operator": "ge", "actual_fraction": 0.6320505912713043, "metric": 0.41216005444800496, "rescued": 6514, "harmed": 1391, "missed_rescue": 4209, "avoided_harm": 1109, "selected_both_correct": 5333, "selected_both_wrong": 9050}, {"budget": 0.68, "threshold": -0.5312389135360718, "operator": "ge", "actual_fraction": 0.6424297422227264, "metric": 0.4155630547599467, "rescued": 6659, "harmed": 1416, "missed_rescue": 4064, "avoided_harm": 1084, "selected_both_correct": 5378, "selected_both_wrong": 9201}, {"budget": 0.69, "threshold": -0.5368650555610657, "operator": "ge", "actual_fraction": 0.6538865099395967, "metric": 0.4177182882908431, "rescued": 6784, "harmed": 1465, "missed_rescue": 3939, "avoided_harm": 1035, "selected_both_correct": 5424, "selected_both_wrong": 9385}, {"budget": 0.7, "threshold": -0.5431069135665894, "operator": "ge", "actual_fraction": 0.6647761109378102, "metric": 0.4212347219465162, "rescued": 6935, "harmed": 1492, "missed_rescue": 3788, "avoided_harm": 1008, "selected_both_correct": 5463, "selected_both_wrong": 9552}, {"budget": 0.71, "threshold": -0.5487136244773865, "operator": "ge", "actual_fraction": 0.6740492867878513, "metric": 0.42384368885233814, "rescued": 7054, "harmed": 1519, "missed_rescue": 3669, "avoided_harm": 981, "selected_both_correct": 5498, "selected_both_wrong": 9698}, {"budget": 0.72, "threshold": -0.5549414753913879, "operator": "ge", "actual_fraction": 0.6839179876924822, "metric": 0.42628250574256304, "rescued": 7167, "harmed": 1546, "missed_rescue": 3556, "avoided_harm": 954, "selected_both_correct": 5541, "selected_both_wrong": 9863}, {"budget": 0.73, "threshold": -0.5604677200317383, "operator": "ge", "actual_fraction": 0.6935314635737175, "metric": 0.429260131015512, "rescued": 7298, "harmed": 1572, "missed_rescue": 3425, "avoided_harm": 928, "selected_both_correct": 5579, "selected_both_wrong": 10007}, {"budget": 0.74, "threshold": -0.5666090250015259, "operator": "ge", "actual_fraction": 0.7039673311970054, "metric": 0.4321243229447296, "rescued": 7424, "harmed": 1597, "missed_rescue": 3299, "avoided_harm": 903, "selected_both_correct": 5614, "selected_both_wrong": 10189}, {"budget": 0.75, "threshold": -0.574927568435669, "operator": "ge", "actual_fraction": 0.7186853075461532, "metric": 0.4360094149675297, "rescued": 7607, "harmed": 1643, "missed_rescue": 3116, "avoided_harm": 857, "selected_both_correct": 5687, "selected_both_wrong": 10406}, {"budget": 0.76, "threshold": -0.5812664031982422, "operator": "ge", "actual_fraction": 0.7283555000992542, "metric": 0.438363156849956, "rescued": 7724, "harmed": 1677, "missed_rescue": 2999, "avoided_harm": 823, "selected_both_correct": 5729, "selected_both_wrong": 10554}, {"budget": 0.77, "threshold": -0.5883752107620239, "operator": "ge", "actual_fraction": 0.7390465927459376, "metric": 0.44173779882596487, "rescued": 7869, "harmed": 1703, "missed_rescue": 2854, "avoided_harm": 797, "selected_both_correct": 5774, "selected_both_wrong": 10715}, {"budget": 0.78, "threshold": -0.5972744226455688, "operator": "ge", "actual_fraction": 0.752176502282846, "metric": 0.44596319087995917, "rescued": 8052, "harmed": 1737, "missed_rescue": 2671, "avoided_harm": 763, "selected_both_correct": 5823, "selected_both_wrong": 10912}, {"budget": 0.79, "threshold": -0.6058458089828491, "operator": "ge", "actual_fraction": 0.7635765533278507, "metric": 0.44888409948104246, "rescued": 8191, "harmed": 1773, "missed_rescue": 2532, "avoided_harm": 727, "selected_both_correct": 5871, "selected_both_wrong": 11091}, {"budget": 0.8, "threshold": -0.6140543818473816, "operator": "ge", "actual_fraction": 0.7745228709979298, "metric": 0.4523721748007827, "rescued": 8338, "harmed": 1797, "missed_rescue": 2385, "avoided_harm": 703, "selected_both_correct": 5916, "selected_both_wrong": 11261}, {"budget": 0.81, "threshold": -0.6230268478393555, "operator": "ge", "actual_fraction": 0.7864050137537929, "metric": 0.4548109916910076, "rescued": 8468, "harmed": 1841, "missed_rescue": 2255, "avoided_harm": 659, "selected_both_correct": 5956, "selected_both_wrong": 11466}, {"budget": 0.82, "threshold": -0.6330366134643555, "operator": "ge", "actual_fraction": 0.798287156509656, "metric": 0.45827070867481495, "rescued": 8625, "harmed": 1876, "missed_rescue": 2098, "avoided_harm": 624, "selected_both_correct": 5991, "selected_both_wrong": 11658}, {"budget": 0.83, "threshold": -0.6429333686828613, "operator": "ge", "actual_fraction": 0.8092334741797351, "metric": 0.4612766922836968, "rescued": 8762, "harmed": 1907, "missed_rescue": 1961, "avoided_harm": 593, "selected_both_correct": 6039, "selected_both_wrong": 11828}, {"budget": 0.84, "threshold": -0.6513019800186157, "operator": "ge", "actual_fraction": 0.8177126166236566, "metric": 0.46340356747866035, "rescued": 8858, "harmed": 1928, "missed_rescue": 1865, "avoided_harm": 572, "selected_both_correct": 6071, "selected_both_wrong": 11978}, {"budget": 0.85, "threshold": -0.6624053120613098, "operator": "ge", "actual_fraction": 0.8288858009811985, "metric": 0.4659274593766838, "rescued": 8984, "harmed": 1965, "missed_rescue": 1739, "avoided_harm": 535, "selected_both_correct": 6119, "selected_both_wrong": 12161}, {"budget": 0.86, "threshold": -0.6759263277053833, "operator": "ge", "actual_fraction": 0.8421291438618382, "metric": 0.4693021013526926, "rescued": 9143, "harmed": 2005, "missed_rescue": 1580, "avoided_harm": 495, "selected_both_correct": 6163, "selected_both_wrong": 12385}, {"budget": 0.87, "threshold": -0.6904275417327881, "operator": "ge", "actual_fraction": 0.8543799449848283, "metric": 0.47202450160224596, "rescued": 9273, "harmed": 2039, "missed_rescue": 1450, "avoided_harm": 461, "selected_both_correct": 6209, "selected_both_wrong": 12607}, {"budget": 0.88, "threshold": -0.7045450210571289, "operator": "ge", "actual_fraction": 0.8657516376939001, "metric": 0.47395286844567963, "rescued": 9384, "harmed": 2082, "missed_rescue": 1339, "avoided_harm": 418, "selected_both_correct": 6264, "selected_both_wrong": 12799}, {"budget": 0.89, "threshold": -0.7205488681793213, "operator": "ge", "actual_fraction": 0.8767830303717777, "metric": 0.4759663102969118, "rescued": 9501, "harmed": 2128, "missed_rescue": 1222, "avoided_harm": 372, "selected_both_correct": 6312, "selected_both_wrong": 12977}, {"budget": 0.9, "threshold": -0.7357243299484253, "operator": "ge", "actual_fraction": 0.8868785979638715, "metric": 0.4782066188356067, "rescued": 9610, "harmed": 2158, "missed_rescue": 1113, "avoided_harm": 342, "selected_both_correct": 6365, "selected_both_wrong": 13141}, {"budget": 0.91, "threshold": -0.7513164281845093, "operator": "ge", "actual_fraction": 0.8974562572668235, "metric": 0.4802200606868389, "rescued": 9710, "harmed": 2187, "missed_rescue": 1013, "avoided_harm": 313, "selected_both_correct": 6424, "selected_both_wrong": 13326}, {"budget": 0.92, "threshold": -0.7671180963516235, "operator": "ge", "actual_fraction": 0.9084025749369027, "metric": 0.4826021609051981, "rescued": 9825, "harmed": 2218, "missed_rescue": 898, "avoided_harm": 282, "selected_both_correct": 6474, "selected_both_wrong": 13516}, {"budget": 0.93, "threshold": -0.7833515405654907, "operator": "ge", "actual_fraction": 0.9181011258259365, "metric": 0.4839917193659076, "rescued": 9916, "harmed": 2260, "missed_rescue": 807, "avoided_harm": 240, "selected_both_correct": 6514, "selected_both_wrong": 13685}, {"budget": 0.94, "threshold": -0.8013419508934021, "operator": "ge", "actual_fraction": 0.9289623684882171, "metric": 0.48634546124833394, "rescued": 10031, "harmed": 2292, "missed_rescue": 692, "avoided_harm": 208, "selected_both_correct": 6566, "selected_both_wrong": 13869}, {"budget": 0.95, "threshold": -0.8188987374305725, "operator": "ge", "actual_fraction": 0.9394549527833707, "metric": 0.48770666137311064, "rescued": 10126, "harmed": 2339, "missed_rescue": 597, "avoided_harm": 161, "selected_both_correct": 6606, "selected_both_wrong": 14057}, {"budget": 0.96, "threshold": -0.8395025134086609, "operator": "ge", "actual_fraction": 0.9509684371721068, "metric": 0.4894648782009472, "rescued": 10222, "harmed": 2373, "missed_rescue": 501, "avoided_harm": 127, "selected_both_correct": 6654, "selected_both_wrong": 14285}, {"budget": 0.97, "threshold": -0.8628664612770081, "operator": "ge", "actual_fraction": 0.9640699883730822, "metric": 0.49218727845050053, "rescued": 10358, "harmed": 2413, "missed_rescue": 365, "avoided_harm": 87, "selected_both_correct": 6713, "selected_both_wrong": 14512}, {"budget": 0.98, "threshold": -0.8857419490814209, "operator": "ge", "actual_fraction": 0.9760372061367439, "metric": 0.49439922865326263, "rescued": 10476, "harmed": 2453, "missed_rescue": 247, "avoided_harm": 47, "selected_both_correct": 6784, "selected_both_wrong": 14705}, {"budget": 0.99, "threshold": -0.9162307977676392, "operator": "ge", "actual_fraction": 0.9873805405098829, "metric": 0.4968380455434875, "rescued": 10590, "harmed": 2481, "missed_rescue": 133, "avoided_harm": 19, "selected_both_correct": 6845, "selected_both_wrong": 14902}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.5000708958398321, "rescued": 10723, "harmed": 2500, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 6911, "selected_both_wrong": 15129}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.6590765338393421, "actual_fraction": 0.4579520697167756, "metric_std": 0.002458258828082453, "actual_fraction_std": 0.04146715758880001, "per_seed": [{"seed": 2026, "budget": 0.51, "threshold": 0.11353585124015808, "operator": "ge", "alpha": 0.0, "epoch": 16, "metric": 0.6606226720078712, "actual_fraction": 0.5099444795839483, "rescue": 2249, "harm": 825, "missed_rescue": 340}, {"seed": 2027, "budget": 0.4, "threshold": 0.3213382363319397, "operator": "ge", "alpha": 0.0, "epoch": 9, "metric": 0.6602712769695691, "actual_fraction": 0.39995783259540374, "rescue": 2044, "harm": 625, "missed_rescue": 545}, {"seed": 2028, "budget": 0.44, "threshold": 0.2752467393875122, "operator": "ge", "alpha": 0.0, "epoch": 5, "metric": 0.6580926277320964, "actual_fraction": 0.43994658795417807, "rescue": 2099, "harm": 711, "missed_rescue": 490}, {"seed": 2029, "budget": 0.46, "threshold": 0.20826171338558197, "operator": "ge", "alpha": 0.0, "epoch": 9, "metric": 0.6611849040691545, "actual_fraction": 0.45997610513739545, "rescue": 2163, "harm": 731, "missed_rescue": 426}, {"seed": 2030, "budget": 0.48, "threshold": 0.16629476845264435, "operator": "ge", "alpha": 0.0, "epoch": 4, "metric": 0.6552111884180195, "actual_fraction": 0.47993534331295234, "rescue": 2163, "harm": 816, "missed_rescue": 426}], "rescued": 2143.6, "harmed": 741.6, "missed_rescue": 445.4, "avoided_harm": 699.4}, "validation_curve": [{"budget": 0.0, "metric": 0.5605453650994447, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.5673624288425047, "metric_std": 0.0009750816589719443, "actual_fraction": 0.009979619087778481, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.573546981516621, "metric_std": 0.001360039530561778, "actual_fraction": 0.019959238175556962, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.5793801391524351, "metric_std": 0.0015373281927210246, "actual_fraction": 0.02993885726333544, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.5848337901468831, "metric_std": 0.0019492766785172788, "actual_fraction": 0.03998875535877434, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.5894019256448099, "metric_std": 0.002429255684135887, "actual_fraction": 0.049968374446552805, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.5939278937381404, "metric_std": 0.002595094586778652, "actual_fraction": 0.05994799353433129, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.5981305783962328, "metric_std": 0.002566579352870163, "actual_fraction": 0.06999789162977019, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.6021083702298122, "metric_std": 0.0027316755072500803, "actual_fraction": 0.07997751071754867, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.6063672780940333, "metric_std": 0.003031380536396027, "actual_fraction": 0.08995712980532716, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.6097828378663293, "metric_std": 0.002799981286386802, "actual_fraction": 0.09992269309157353, "actual_fraction_std": 3.142972770397328e-05}, {"budget": 0.11, "metric": 0.6130016164171761, "metric_std": 0.00249396424174447, "actual_fraction": 0.10998664698854452, "actual_fraction_std": 0.0}, {"budget": 0.12, "metric": 0.6160938927542343, "metric_std": 0.0025185980449534983, "actual_fraction": 0.11995221027479093, "actual_fraction_std": 3.142972770398569e-05}, {"budget": 0.13, "metric": 0.6189753320683111, "metric_std": 0.0027800655708934987, "actual_fraction": 0.1299458851641015, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.621688101764003, "metric_std": 0.0033513582170402395, "actual_fraction": 0.13999578325954037, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.6246398200857403, "metric_std": 0.0035835403145637753, "actual_fraction": 0.14997540234731885, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.6277883196289269, "metric_std": 0.0038481848062619037, "actual_fraction": 0.15995502143509735, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.6301637500878486, "metric_std": 0.003790704004978321, "actual_fraction": 0.16993464052287582, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.6325954037528989, "metric_std": 0.003880776003320619, "actual_fraction": 0.1799845386183147, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.634984890013353, "metric_std": 0.0035312666319039813, "actual_fraction": 0.18996415770609318, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.6366294187926066, "metric_std": 0.0033497366752185333, "actual_fraction": 0.19994377679387168, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.6385972310070981, "metric_std": 0.0035036041374904084, "actual_fraction": 0.20997961908777846, "actual_fraction_std": 3.142972770397948e-05}, {"budget": 0.22, "metric": 0.6403120387940122, "metric_std": 0.003895891925373607, "actual_fraction": 0.219959238175557, "actual_fraction_std": 3.1429727703979486e-05}, {"budget": 0.23, "metric": 0.6421111813901188, "metric_std": 0.004118553724934033, "actual_fraction": 0.22995291306486756, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.24, "metric": 0.6437978775739686, "metric_std": 0.004267562176624231, "actual_fraction": 0.23993253215264604, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.25, "metric": 0.6454705179562864, "metric_std": 0.0038095500891801664, "actual_fraction": 0.24998243024808492, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.26, "metric": 0.6469885445217514, "metric_std": 0.0033027395143746864, "actual_fraction": 0.2599479935343313, "actual_fraction_std": 3.142972770396707e-05}, {"budget": 0.27, "metric": 0.6481130086443179, "metric_std": 0.003382530901991374, "actual_fraction": 0.26994166842364187, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.6495045329959941, "metric_std": 0.003671498050425002, "actual_fraction": 0.27999156651908075, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.6507695551338815, "metric_std": 0.0037821599825411294, "actual_fraction": 0.2899571298053271, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.3, "metric": 0.6519361866610442, "metric_std": 0.0036279219104016617, "actual_fraction": 0.2999508046946377, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.652737367348373, "metric_std": 0.0036097695622797025, "actual_fraction": 0.309902312179352, "actual_fraction_std": 6.285945540795896e-05}, {"budget": 0.32, "metric": 0.6528779253636939, "metric_std": 0.003879503075151953, "actual_fraction": 0.3199803218778551, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.6535947712418301, "metric_std": 0.004324586629030935, "actual_fraction": 0.3299599409656336, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.6542553939138378, "metric_std": 0.003936440082943139, "actual_fraction": 0.339939560053412, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.6552393000210837, "metric_std": 0.003869368398243031, "actual_fraction": 0.34998945814885096, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.6559139784946236, "metric_std": 0.0034622666207755454, "actual_fraction": 0.3599690772366294, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.6562372619298615, "metric_std": 0.0032748056937888462, "actual_fraction": 0.3699346405228758, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.38, "metric": 0.6561950945252653, "metric_std": 0.0028912787168071286, "actual_fraction": 0.37999859441984685, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.6566870475788882, "metric_std": 0.0028465167124997, "actual_fraction": 0.3899782135076253, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.6573055028462997, "metric_std": 0.0028914495406361916, "actual_fraction": 0.39995783259540374, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.6576147304800056, "metric_std": 0.0025656169697875794, "actual_fraction": 0.40992339588165017, "actual_fraction_std": 3.142972770401672e-05}, {"budget": 0.42, "metric": 0.6577412326937944, "metric_std": 0.0024340289915734483, "actual_fraction": 0.4199873497786212, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.6580082929229039, "metric_std": 0.0022731755620126345, "actual_fraction": 0.42993885726333547, "actual_fraction_std": 3.8493397814698215e-05}, {"budget": 0.44, "metric": 0.6581488509382248, "metric_std": 0.0022834730030044346, "actual_fraction": 0.43994658795417807, "actual_fraction_std": 0.0}, {"budget": 0.45, "metric": 0.6580504603275001, "metric_std": 0.0022354911668270206, "actual_fraction": 0.44999648604961695, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.46, "metric": 0.658191018342821, "metric_std": 0.002243761194700636, "actual_fraction": 0.4599761051373955, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.47, "metric": 0.6581629067397567, "metric_std": 0.002216297284590783, "actual_fraction": 0.4699557242251739, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.48, "metric": 0.658008292922904, "metric_std": 0.0022298500282488098, "actual_fraction": 0.47993534331295234, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.6577412326937944, "metric_std": 0.0024013422676603963, "actual_fraction": 0.4899852414083913, "actual_fraction_std": 0.0}, {"budget": 0.5, "metric": 0.6575585072738772, "metric_std": 0.0025013798553964674, "actual_fraction": 0.49996486049616984, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.51, "metric": 0.6576568978846018, "metric_std": 0.0027338443646836314, "actual_fraction": 0.5099444795839483, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.6572633354417036, "metric_std": 0.0027433129549777504, "actual_fraction": 0.5199943776793872, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.6570806100217864, "metric_std": 0.0027339346964044705, "actual_fraction": 0.5299739967671656, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.6566027127696956, "metric_std": 0.002790616478771012, "actual_fraction": 0.5399536158549442, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.6564621547543747, "metric_std": 0.0024554443338987366, "actual_fraction": 0.5499191791411906, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.56, "metric": 0.6565043221589711, "metric_std": 0.0024520223907125486, "actual_fraction": 0.5599831330381615, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.6560123691053482, "metric_std": 0.0024890082250791676, "actual_fraction": 0.5699627521259399, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.655660974067046, "metric_std": 0.0022393543138980432, "actual_fraction": 0.5799423712137185, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.6554220254410007, "metric_std": 0.0021153864562924443, "actual_fraction": 0.5899922693091574, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.6545224541429475, "metric_std": 0.0017885972931364481, "actual_fraction": 0.5999718883969358, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.6540164452877925, "metric_std": 0.0019703205803466473, "actual_fraction": 0.6099515074847143, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.6538056082648113, "metric_std": 0.002157004603728004, "actual_fraction": 0.6199311265724928, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.6533979900203808, "metric_std": 0.002200305425881029, "actual_fraction": 0.6299810246679317, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.6526530325391805, "metric_std": 0.002298671532470384, "actual_fraction": 0.6399465879541781, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.65, "metric": 0.6518377960503198, "metric_std": 0.0025168324660157773, "actual_fraction": 0.6499402628434886, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.6513036755921007, "metric_std": 0.002597757783800734, "actual_fraction": 0.6599901609389276, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.650685220324689, "metric_std": 0.0026560444492132983, "actual_fraction": 0.669969780026706, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.6502635462787264, "metric_std": 0.0027818416303418566, "actual_fraction": 0.6799493991144846, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.6499262070419565, "metric_std": 0.0030319507505372025, "actual_fraction": 0.6899992972099234, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.6497856490266357, "metric_std": 0.003069592043473762, "actual_fraction": 0.6999789162977019, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.6497013142174433, "metric_std": 0.003084920236681349, "actual_fraction": 0.7099585353854804, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.6496732026143789, "metric_std": 0.0029971339559118487, "actual_fraction": 0.7199381544732588, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.6496029236067187, "metric_std": 0.0029543944041742656, "actual_fraction": 0.7299880525686977, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.6495467004005903, "metric_std": 0.00298979156686745, "actual_fraction": 0.7399676716564763, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.6494061423852694, "metric_std": 0.0029644915038368504, "actual_fraction": 0.7499472907442546, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.6493218075760769, "metric_std": 0.003013978151462454, "actual_fraction": 0.7599971888396937, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.6492374727668845, "metric_std": 0.0030313805363959626, "actual_fraction": 0.769976807927472, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.6491953053622882, "metric_std": 0.002927186053957235, "actual_fraction": 0.7799564270152506, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.6490969147515637, "metric_std": 0.002892986501195139, "actual_fraction": 0.7899360461030291, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.6489282451331786, "metric_std": 0.0029085661653625045, "actual_fraction": 0.7999859441984679, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.6486892965071334, "metric_std": 0.002763940332940743, "actual_fraction": 0.8099655632862464, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.6484222362780238, "metric_std": 0.0028411326497102653, "actual_fraction": 0.8199451823740249, "actual_fraction_std": 0.0}, {"budget": 0.83, "metric": 0.6482535666596387, "metric_std": 0.002790881952584806, "actual_fraction": 0.8299950804694639, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.84, "metric": 0.6479302832244007, "metric_std": 0.002603455460883478, "actual_fraction": 0.8399746995572424, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.6475648323845667, "metric_std": 0.0023777788625370406, "actual_fraction": 0.8499543186450207, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.647255604750861, "metric_std": 0.0022458514338193187, "actual_fraction": 0.8599339377327994, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.87, "metric": 0.647129102537072, "metric_std": 0.001992753795594551, "actual_fraction": 0.8699838358282379, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.6465387588727247, "metric_std": 0.002088717646316282, "actual_fraction": 0.8799634549160166, "actual_fraction_std": 0.0}, {"budget": 0.89, "metric": 0.6462154754374867, "metric_std": 0.0018666982901529682, "actual_fraction": 0.889943074003795, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.6456251317731394, "metric_std": 0.0016092204741945445, "actual_fraction": 0.8999929720992339, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.91, "metric": 0.6449223416965352, "metric_std": 0.0013893219529863342, "actual_fraction": 0.9099725911870123, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.6442757748260594, "metric_std": 0.001343230228459575, "actual_fraction": 0.919952210274791, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.93, "metric": 0.6439524913908214, "metric_std": 0.0012217232832923584, "actual_fraction": 0.9299318293625692, "actual_fraction_std": 0.0}, {"budget": 0.94, "metric": 0.6433059245203456, "metric_std": 0.001436513919121058, "actual_fraction": 0.9399817274580083, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.642673413451402, "metric_std": 0.001589921789930374, "actual_fraction": 0.9499613465457868, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.6419284559702018, "metric_std": 0.0012815083652267007, "actual_fraction": 0.9599409656335652, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.6413521681073863, "metric_std": 0.0014661218424800262, "actual_fraction": 0.9699908637290042, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.98, "metric": 0.6411132194813408, "metric_std": 0.0013687276542452742, "actual_fraction": 0.9799704828167826, "actual_fraction_std": 0.0}, {"budget": 0.99, "metric": 0.6408321034506993, "metric_std": 0.0010803325998321718, "actual_fraction": 0.9899501019045612, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 1.0, "metric": 0.6412256658935975, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.5022828460425942, "actual_fraction": 0.5818676800045374, "metric_std": 0.00594393021203905, "actual_fraction_std": 0.04760049787234513, "per_seed": [{"seed": 2026, "budget": 0.51, "threshold": 0.11353585124015808, "operator": "ge", "alpha": 0.0, "epoch": 16, "metric": 0.5081813799166265, "actual_fraction": 0.6393103252701131, "rescue": 9851, "harm": 1342, "missed_rescue": 872}, {"seed": 2027, "budget": 0.4, "threshold": 0.3213382363319397, "operator": "ge", "alpha": 0.0, "epoch": 9, "metric": 0.49249922014576186, "actual_fraction": 0.5114993052207697, "rescue": 8938, "harm": 982, "missed_rescue": 1785}, {"seed": 2028, "budget": 0.44, "threshold": 0.2752467393875122, "operator": "ge", "alpha": 0.0, "epoch": 5, "metric": 0.5023395627144599, "actual_fraction": 0.5746816776791538, "rescue": 9443, "harm": 1140, "missed_rescue": 1280}, {"seed": 2029, "budget": 0.46, "threshold": 0.20826171338558197, "operator": "ge", "alpha": 0.0, "epoch": 9, "metric": 0.502878371097184, "actual_fraction": 0.5752204860618779, "rescue": 9457, "harm": 1135, "missed_rescue": 1266}, {"seed": 2030, "budget": 0.48, "threshold": 0.16629476845264435, "operator": "ge", "alpha": 0.0, "epoch": 4, "metric": 0.5055156963389389, "actual_fraction": 0.6086266057907722, "rescue": 9688, "harm": 1273, "missed_rescue": 1035}], "rescued": 9475.4, "harmed": 1174.4, "missed_rescue": 1247.6, "avoided_harm": 1325.6}, "test_curve": [{"budget": 0.0, "metric": 0.26688029946402747, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.2834018659785044, "metric_std": 0.004857750663371017, "actual_fraction": 0.020486061877889008, "actual_fraction_std": 0.006225756028511027}, {"budget": 0.02, "metric": 0.29684938887786066, "metric_std": 0.005320699480450805, "actual_fraction": 0.03822703683747838, "actual_fraction_std": 0.007222640467902761}, {"budget": 0.03, "metric": 0.3082097382525594, "metric_std": 0.006907499227193402, "actual_fraction": 0.053739046592745934, "actual_fraction_std": 0.010055299698800644}, {"budget": 0.04, "metric": 0.3190823242492131, "metric_std": 0.0073096712064342475, "actual_fraction": 0.06928508635113292, "actual_fraction_std": 0.011018124077792546}, {"budget": 0.05, "metric": 0.32901341349289626, "metric_std": 0.00766121321358462, "actual_fraction": 0.08453052774863172, "actual_fraction_std": 0.011173569419681416}, {"budget": 0.06, "metric": 0.33797464764767604, "metric_std": 0.0064965661140867015, "actual_fraction": 0.09889686073221224, "actual_fraction_std": 0.009588569425290409}, {"budget": 0.07, "metric": 0.3471911068258514, "metric_std": 0.006169779452575021, "actual_fraction": 0.11438051215154695, "actual_fraction_std": 0.009347150149283912}, {"budget": 0.08, "metric": 0.3559821909650342, "metric_std": 0.005574306852643363, "actual_fraction": 0.12969401355528457, "actual_fraction_std": 0.008580648585323685}, {"budget": 0.09, "metric": 0.36393954002779116, "metric_std": 0.006651967414363128, "actual_fraction": 0.14433825823100702, "actual_fraction_std": 0.010211391421218852}, {"budget": 0.1, "metric": 0.37176076907807054, "metric_std": 0.006579030036524582, "actual_fraction": 0.15927742960043106, "actual_fraction_std": 0.010710774922225372}, {"budget": 0.11, "metric": 0.3788787113972153, "metric_std": 0.0068288144799950575, "actual_fraction": 0.17353600090746674, "actual_fraction_std": 0.011218878119582887}, {"budget": 0.12, "metric": 0.38541247199614326, "metric_std": 0.00727581729312998, "actual_fraction": 0.1872727788333381, "actual_fraction_std": 0.011854961435964548}, {"budget": 0.13, "metric": 0.3917023509060488, "metric_std": 0.006736421079229098, "actual_fraction": 0.20038567336868676, "actual_fraction_std": 0.011189410242254757}, {"budget": 0.14, "metric": 0.39794685647846184, "metric_std": 0.007023524581012364, "actual_fraction": 0.21404871962113264, "actual_fraction_std": 0.012054277691524921}, {"budget": 0.15, "metric": 0.4034597169838075, "metric_std": 0.007459546935346484, "actual_fraction": 0.22619743073476445, "actual_fraction_std": 0.01354370492000012}, {"budget": 0.16, "metric": 0.4088024274735559, "metric_std": 0.006998553489304444, "actual_fraction": 0.23881121855769502, "actual_fraction_std": 0.013212449699252457}, {"budget": 0.17, "metric": 0.41454215466636424, "metric_std": 0.007023971117735099, "actual_fraction": 0.25227008479142443, "actual_fraction_std": 0.013359175328840924}, {"budget": 0.18, "metric": 0.4196239684655305, "metric_std": 0.006879728790506289, "actual_fraction": 0.26442446757224286, "actual_fraction_std": 0.012377050865294198}, {"budget": 0.19, "metric": 0.4241839888835323, "metric_std": 0.00684869958638423, "actual_fraction": 0.2762612369906134, "actual_fraction_std": 0.012629244102215009}, {"budget": 0.2, "metric": 0.4290332643280492, "metric_std": 0.006803696118937311, "actual_fraction": 0.288795621472932, "actual_fraction_std": 0.013439733164256912}, {"budget": 0.21, "metric": 0.43334940305702857, "metric_std": 0.006108486932904156, "actual_fraction": 0.3009046309162578, "actual_fraction_std": 0.013028570748704362}, {"budget": 0.22, "metric": 0.43775061679380656, "metric_std": 0.00607565682939586, "actual_fraction": 0.31287184867991946, "actual_fraction_std": 0.012799154089080323}, {"budget": 0.23, "metric": 0.442197203868077, "metric_std": 0.0054060167515537225, "actual_fraction": 0.3251453364716558, "actual_fraction_std": 0.01157901797773864}, {"budget": 0.24, "metric": 0.4466267759407878, "metric_std": 0.004466442410744368, "actual_fraction": 0.33829793267731045, "actual_fraction_std": 0.010903800448455833}, {"budget": 0.25, "metric": 0.4507557496526104, "metric_std": 0.0038420574310445957, "actual_fraction": 0.35155261889232337, "actual_fraction_std": 0.010200202121318374}, {"budget": 0.26, "metric": 0.4545444233332388, "metric_std": 0.003721895198595346, "actual_fraction": 0.3631568499560446, "actual_fraction_std": 0.01036674645214236}, {"budget": 0.27, "metric": 0.4583217536794941, "metric_std": 0.0038883715118285664, "actual_fraction": 0.37526018773218384, "actual_fraction_std": 0.010433813753643871}, {"budget": 0.28, "metric": 0.461826844000794, "metric_std": 0.0036969634795930373, "actual_fraction": 0.38702889714431554, "actual_fraction_std": 0.010234106580209682}, {"budget": 0.29, "metric": 0.4651844709752432, "metric_std": 0.0041437826936582154, "actual_fraction": 0.39812267816124547, "actual_fraction_std": 0.011494102684503446}, {"budget": 0.3, "metric": 0.46829821626066986, "metric_std": 0.0035880075655415956, "actual_fraction": 0.4095737742109294, "actual_fraction_std": 0.010599710962652312}, {"budget": 0.31, "metric": 0.4717239032413579, "metric_std": 0.003482168094890985, "actual_fraction": 0.4219720386807702, "actual_fraction_std": 0.010609429189445484}, {"budget": 0.32, "metric": 0.4746051101721351, "metric_std": 0.003155876370115487, "actual_fraction": 0.4330317896945807, "actual_fraction_std": 0.010401267809207429}, {"budget": 0.33, "metric": 0.4774352720982332, "metric_std": 0.002907621546298061, "actual_fraction": 0.44419363071774953, "actual_fraction_std": 0.009844909897737937}, {"budget": 0.34, "metric": 0.48002722400249553, "metric_std": 0.0028508521008349208, "actual_fraction": 0.4553100984034256, "actual_fraction_std": 0.009799571658592196}, {"budget": 0.35, "metric": 0.482630519241131, "metric_std": 0.002518549140505866, "actual_fraction": 0.4662223860703854, "actual_fraction_std": 0.00876654737992713}, {"budget": 0.36, "metric": 0.4850750077985424, "metric_std": 0.002447654613541852, "actual_fraction": 0.47726512208263616, "actual_fraction_std": 0.009434537504901292}, {"budget": 0.37, "metric": 0.4873436746731702, "metric_std": 0.002333100118832695, "actual_fraction": 0.4875194963559538, "actual_fraction_std": 0.009481031896400783}, {"budget": 0.38, "metric": 0.4893854748603353, "metric_std": 0.002381627532528859, "actual_fraction": 0.4981935740010776, "actual_fraction_std": 0.010020657863091732}, {"budget": 0.39, "metric": 0.491693843405269, "metric_std": 0.0020444177167818693, "actual_fraction": 0.5097864617304256, "actual_fraction_std": 0.009395977334480303}, {"budget": 0.4, "metric": 0.4935087769049712, "metric_std": 0.0016502261247845508, "actual_fraction": 0.519899044324079, "actual_fraction_std": 0.008870481133617492}, {"budget": 0.41, "metric": 0.4955449054249497, "metric_std": 0.001665264653551539, "actual_fraction": 0.5317585004111958, "actual_fraction_std": 0.009175457804823995}, {"budget": 0.42, "metric": 0.49727476391685344, "metric_std": 0.0013098051977949096, "actual_fraction": 0.5427785497547004, "actual_fraction_std": 0.008899797368668494}, {"budget": 0.43, "metric": 0.4988514873947197, "metric_std": 0.0012137897830038582, "actual_fraction": 0.5531803873748687, "actual_fraction_std": 0.009515508520578203}, {"budget": 0.44, "metric": 0.5004225392053995, "metric_std": 0.0014031789442148252, "actual_fraction": 0.5635368516575447, "actual_fraction_std": 0.008981649542154323}, {"budget": 0.45, "metric": 0.5019198593426538, "metric_std": 0.0013790142962843836, "actual_fraction": 0.5744377959901313, "actual_fraction_std": 0.009075168186567461}, {"budget": 0.46, "metric": 0.5033264328049231, "metric_std": 0.001126400813952091, "actual_fraction": 0.584771573604061, "actual_fraction_std": 0.009253021263705618}, {"budget": 0.47, "metric": 0.5047897229390579, "metric_std": 0.00100213720958762, "actual_fraction": 0.5960298329694014, "actual_fraction_std": 0.009233706722129163}, {"budget": 0.48, "metric": 0.5060034597169838, "metric_std": 0.0010268694018098707, "actual_fraction": 0.6059892805490174, "actual_fraction_std": 0.007895466960813412}, {"budget": 0.49, "metric": 0.5072568981652157, "metric_std": 0.0008995897102271735, "actual_fraction": 0.6160791764739245, "actual_fraction_std": 0.007785935935074421}, {"budget": 0.5, "metric": 0.5085160082806341, "metric_std": 0.0008892045530903895, "actual_fraction": 0.6267532541190483, "actual_fraction_std": 0.00782958197041126}, {"budget": 0.51, "metric": 0.5094234750304852, "metric_std": 0.0008365468771857441, "actual_fraction": 0.6376995717891274, "actual_fraction_std": 0.007121652791622189}, {"budget": 0.52, "metric": 0.5104500467912543, "metric_std": 0.0006701813401070066, "actual_fraction": 0.6488443978107366, "actual_fraction_std": 0.007338000945631228}, {"budget": 0.53, "metric": 0.511244080197374, "metric_std": 0.0006803045989220967, "actual_fraction": 0.6602557921901143, "actual_fraction_std": 0.006930280665217966}, {"budget": 0.54, "metric": 0.5118112469160309, "metric_std": 0.0009597945722582114, "actual_fraction": 0.6712418115304994, "actual_fraction_std": 0.0074137390224431575}, {"budget": 0.55, "metric": 0.5124861753112329, "metric_std": 0.0010577317172722968, "actual_fraction": 0.682182457533392, "actual_fraction_std": 0.00707142272987947}, {"budget": 0.56, "metric": 0.513076028698636, "metric_std": 0.0009186087529913924, "actual_fraction": 0.6934577319002921, "actual_fraction_std": 0.007364043401613089}, {"budget": 0.57, "metric": 0.5135694637438675, "metric_std": 0.000988278852804749, "actual_fraction": 0.7039446445282591, "actual_fraction_std": 0.00778371492224358}, {"budget": 0.58, "metric": 0.5140572271219125, "metric_std": 0.0008553223888078463, "actual_fraction": 0.7155488755919802, "actual_fraction_std": 0.008532056744452858}, {"budget": 0.59, "metric": 0.5143634971499873, "metric_std": 0.0009248035538167541, "actual_fraction": 0.7268865382979326, "actual_fraction_std": 0.008023965970919563}, {"budget": 0.6, "metric": 0.5146811105124351, "metric_std": 0.0007368803230087642, "actual_fraction": 0.7386949493803704, "actual_fraction_std": 0.00889603755395143}, {"budget": 0.61, "metric": 0.514959022204577, "metric_std": 0.0006522417264554866, "actual_fraction": 0.7496923120551287, "actual_fraction_std": 0.009471378537928333}, {"budget": 0.62, "metric": 0.514998723874883, "metric_std": 0.000732556802046194, "actual_fraction": 0.7611263931032528, "actual_fraction_std": 0.008918180415039497}, {"budget": 0.63, "metric": 0.5149193205342711, "metric_std": 0.0006804818923966039, "actual_fraction": 0.7737118225902504, "actual_fraction_std": 0.008353530183107525}, {"budget": 0.64, "metric": 0.5148172305249128, "metric_std": 0.0007830701455624663, "actual_fraction": 0.7848396336102997, "actual_fraction_std": 0.008837806123881561}, {"budget": 0.65, "metric": 0.5148399171936591, "metric_std": 0.0010298020355873879, "actual_fraction": 0.7951450528882965, "actual_fraction_std": 0.00778252667844889}, {"budget": 0.66, "metric": 0.51478887218898, "metric_std": 0.0010680594155498292, "actual_fraction": 0.8052746504835095, "actual_fraction_std": 0.009594857628237379}, {"budget": 0.67, "metric": 0.5147605138530471, "metric_std": 0.0010256155925488807, "actual_fraction": 0.8162096248192157, "actual_fraction_std": 0.01425241269870363}, {"budget": 0.68, "metric": 0.5146357371749426, "metric_std": 0.0010674192162763358, "actual_fraction": 0.8249723506224654, "actual_fraction_std": 0.014930993769667339}, {"budget": 0.69, "metric": 0.5145563338343306, "metric_std": 0.0010758242217639928, "actual_fraction": 0.8377676317953663, "actual_fraction_std": 0.014556991536186}, {"budget": 0.7, "metric": 0.5144882738280918, "metric_std": 0.0010302314532368662, "actual_fraction": 0.8476023026968778, "actual_fraction_std": 0.014164882347847374}, {"budget": 0.71, "metric": 0.514459915492159, "metric_std": 0.0010321811000609084, "actual_fraction": 0.8554915917533957, "actual_fraction_std": 0.01352442012970462}, {"budget": 0.72, "metric": 0.5144088704874799, "metric_std": 0.001026712759386298, "actual_fraction": 0.862025352352324, "actual_fraction_std": 0.012465974185070674}, {"budget": 0.73, "metric": 0.5143918554859201, "metric_std": 0.0010034203563482635, "actual_fraction": 0.867492839520177, "actual_fraction_std": 0.011528118746096746}, {"budget": 0.74, "metric": 0.5143691688171739, "metric_std": 0.0010132699706351586, "actual_fraction": 0.8725462949834103, "actual_fraction_std": 0.010129794482082182}, {"budget": 0.75, "metric": 0.5143124521453082, "metric_std": 0.0010221596870209548, "actual_fraction": 0.8773104954201288, "actual_fraction_std": 0.008887920517189883}, {"budget": 0.76, "metric": 0.5142954371437485, "metric_std": 0.001028864502244793, "actual_fraction": 0.8816890224881604, "actual_fraction_std": 0.00811685228677371}, {"budget": 0.77, "metric": 0.5142273771375097, "metric_std": 0.0010816019517343853, "actual_fraction": 0.8858633695374756, "actual_fraction_std": 0.007163901660037523}, {"budget": 0.78, "metric": 0.5142046904687634, "metric_std": 0.0010781017331202842, "actual_fraction": 0.8900944332586563, "actual_fraction_std": 0.006076020818124062}, {"budget": 0.79, "metric": 0.5141366304625246, "metric_std": 0.0010710669805757303, "actual_fraction": 0.8941723619657997, "actual_fraction_std": 0.005191140371968665}, {"budget": 0.8, "metric": 0.514130958795338, "metric_std": 0.001113294960132202, "actual_fraction": 0.8974449139324502, "actual_fraction_std": 0.004075893025003039}, {"budget": 0.81, "metric": 0.5140061821172335, "metric_std": 0.001099155345853835, "actual_fraction": 0.9010180642599892, "actual_fraction_std": 0.0032690040227062204}, {"budget": 0.82, "metric": 0.5138246887672631, "metric_std": 0.0011050658065620484, "actual_fraction": 0.9048804696140431, "actual_fraction_std": 0.002490000775819463}, {"budget": 0.83, "metric": 0.5136602104188526, "metric_std": 0.0010454961208840646, "actual_fraction": 0.9084025749369028, "actual_fraction_std": 0.0018746554610825164}, {"budget": 0.84, "metric": 0.5136318520829197, "metric_std": 0.0010934335960339984, "actual_fraction": 0.9117318435754189, "actual_fraction_std": 0.0017091162295976652}, {"budget": 0.85, "metric": 0.5135637920766809, "metric_std": 0.001198018031402747, "actual_fraction": 0.915027082210816, "actual_fraction_std": 0.0015403302075729034}, {"budget": 0.86, "metric": 0.5134333437313898, "metric_std": 0.0011301428420554833, "actual_fraction": 0.9185548592008621, "actual_fraction_std": 0.0014454408232282522}, {"budget": 0.87, "metric": 0.5133539403907779, "metric_std": 0.0010698649691366037, "actual_fraction": 0.9220032328502965, "actual_fraction_std": 0.001936544230033898}, {"budget": 0.88, "metric": 0.5132291637126734, "metric_std": 0.001024634986537714, "actual_fraction": 0.9253722031591186, "actual_fraction_std": 0.002188796967633264}, {"budget": 0.89, "metric": 0.5129455803533449, "metric_std": 0.000997995879804198, "actual_fraction": 0.9292062501772396, "actual_fraction_std": 0.002850443041828021}, {"budget": 0.9, "metric": 0.5127527436690016, "metric_std": 0.0009994855168578545, "actual_fraction": 0.9330062671922411, "actual_fraction_std": 0.0027191932778973128}, {"budget": 0.91, "metric": 0.5125542353174717, "metric_std": 0.00108605392343023, "actual_fraction": 0.9372997192524742, "actual_fraction_std": 0.0033844577958121414}, {"budget": 0.92, "metric": 0.512287666959703, "metric_std": 0.0009694652907494479, "actual_fraction": 0.9417122763236254, "actual_fraction_std": 0.003807554127424176}, {"budget": 0.93, "metric": 0.5118339335847772, "metric_std": 0.0009121955880763828, "actual_fraction": 0.9465615517681423, "actual_fraction_std": 0.003959041049996964}, {"budget": 0.94, "metric": 0.5116240818988742, "metric_std": 0.0007865541178748974, "actual_fraction": 0.9512179905283158, "actual_fraction_std": 0.004068823364775011}, {"budget": 0.95, "metric": 0.5112100501942547, "metric_std": 0.0007161251914327148, "actual_fraction": 0.9562544309899895, "actual_fraction_std": 0.003955433870684926}, {"budget": 0.96, "metric": 0.5107790034880754, "metric_std": 0.0006613027740297585, "actual_fraction": 0.9618977398406262, "actual_fraction_std": 0.003646147609301087}, {"budget": 0.97, "metric": 0.5099225817429034, "metric_std": 0.0005126509503321123, "actual_fraction": 0.968459858775487, "actual_fraction_std": 0.003451886335027423}, {"budget": 0.98, "metric": 0.5086464566259252, "metric_std": 0.000535664870402053, "actual_fraction": 0.9760428778039303, "actual_fraction_std": 0.0033925507638685073}, {"budget": 0.99, "metric": 0.5066897314465588, "metric_std": 0.00041707004856640054, "actual_fraction": 0.984703513597822, "actual_fraction_std": 0.0016818429880142293}, {"budget": 1.0, "metric": 0.5000708958398321, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 6535, "rescuable": 2589, "harmful": 1441, "both_wrong": 3664, "unit": "sample", "total": 14229}, "test": {"both_correct": 6911, "rescuable": 10723, "harmful": 2500, "both_wrong": 15129, "unit": "sample", "total": 35263}}, "small": {"validation": 0.5605453650994447, "test": 0.26688029946402747}, "large": {"validation": 0.6412256658935975, "test": 0.5000708958398321}, "learned_minus_confidence_test": 0.002211950202762103, "learned_minus_small_test": 0.23540254657856674, "learned_minus_large_test": 0.002211950202762103, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_groundingdino_MiniCPM"}, {"key": ["grefcoco", "groundingdino", "Qwen3.8"], "expert_display": "GroundingDINO", "run": "outputs/full_budget_20260913/runs/full_groundingdino_Qwen3.8_s{2026..2030}", "validation_rows": 14229, "test_rows": 35263, "confidence_rule": {"column": "max_score", "columns": ["detection_count", "max_score", "mean_score", "query_count", "detection_count_valid", "max_score_valid", "mean_score_valid", "query_count_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/groundingdino/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 100, "validation_area_0_25": 0.5648886077728582, "validation_point": {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.8060299388572634, "rescued": 4626, "harmed": 1133, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 6843, "selected_both_wrong": 1627}, "test_point": {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.7447182599325072, "rescued": 18820, "harmed": 1970, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 7441, "selected_both_wrong": 7032}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.5605453650994447, "rescued": 0, "harmed": 0, "missed_rescue": 4626, "avoided_harm": 1133, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.1584138125181198, "operator": "ge", "actual_fraction": 0.009979619087778481, "metric": 0.5602642490688031, "rescued": 0, "harmed": 4, "missed_rescue": 4626, "avoided_harm": 1129, "selected_both_correct": 138, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -0.21403978765010834, "operator": "ge", "actual_fraction": 0.019959238175556962, "metric": 0.5601939700611427, "rescued": 0, "harmed": 5, "missed_rescue": 4626, "avoided_harm": 1128, "selected_both_correct": 279, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -0.24253316223621368, "operator": "ge", "actual_fraction": 0.029938857263335443, "metric": 0.5596317379998594, "rescued": 0, "harmed": 13, "missed_rescue": 4626, "avoided_harm": 1120, "selected_both_correct": 410, "selected_both_wrong": 3}, {"budget": 0.04, "threshold": -0.2599165141582489, "operator": "ge", "actual_fraction": 0.03998875535877434, "metric": 0.5594209009768782, "rescued": 0, "harmed": 16, "missed_rescue": 4626, "avoided_harm": 1117, "selected_both_correct": 549, "selected_both_wrong": 4}, {"budget": 0.05, "threshold": -0.2725968658924103, "operator": "ge", "actual_fraction": 0.04996837444655281, "metric": 0.5591397849462365, "rescued": 3, "harmed": 23, "missed_rescue": 4623, "avoided_harm": 1110, "selected_both_correct": 675, "selected_both_wrong": 10}, {"budget": 0.06, "threshold": -0.2842792272567749, "operator": "ge", "actual_fraction": 0.05994799353433129, "metric": 0.5588586689155949, "rescued": 5, "harmed": 29, "missed_rescue": 4621, "avoided_harm": 1104, "selected_both_correct": 805, "selected_both_wrong": 14}, {"budget": 0.07, "threshold": -0.2935555577278137, "operator": "ge", "actual_fraction": 0.06999789162977019, "metric": 0.5586478318926137, "rescued": 12, "harmed": 39, "missed_rescue": 4614, "avoided_harm": 1094, "selected_both_correct": 926, "selected_both_wrong": 19}, {"budget": 0.08, "threshold": -0.30114513635635376, "operator": "ge", "actual_fraction": 0.07997751071754867, "metric": 0.5589289479232553, "rescued": 19, "harmed": 42, "missed_rescue": 4607, "avoided_harm": 1091, "selected_both_correct": 1053, "selected_both_wrong": 24}, {"budget": 0.09, "threshold": -0.308601438999176, "operator": "ge", "actual_fraction": 0.08995712980532715, "metric": 0.5594209009768782, "rescued": 34, "harmed": 50, "missed_rescue": 4592, "avoided_harm": 1083, "selected_both_correct": 1165, "selected_both_wrong": 31}, {"budget": 0.1, "threshold": -0.3146189749240875, "operator": "ge", "actual_fraction": 0.09993674889310562, "metric": 0.559561458992199, "rescued": 48, "harmed": 62, "missed_rescue": 4578, "avoided_harm": 1071, "selected_both_correct": 1275, "selected_both_wrong": 37}, {"budget": 0.11, "threshold": -0.32043084502220154, "operator": "ge", "actual_fraction": 0.10998664698854452, "metric": 0.5599831330381615, "rescued": 64, "harmed": 72, "missed_rescue": 4562, "avoided_harm": 1061, "selected_both_correct": 1382, "selected_both_wrong": 47}, {"budget": 0.12, "threshold": -0.32644838094711304, "operator": "ge", "actual_fraction": 0.119966266076323, "metric": 0.5604750860917844, "rescued": 84, "harmed": 85, "missed_rescue": 4542, "avoided_harm": 1048, "selected_both_correct": 1485, "selected_both_wrong": 53}, {"budget": 0.13, "threshold": -0.33121857047080994, "operator": "ge", "actual_fraction": 0.1299458851641015, "metric": 0.5616698292220114, "rescued": 106, "harmed": 90, "missed_rescue": 4520, "avoided_harm": 1043, "selected_both_correct": 1592, "selected_both_wrong": 61}, {"budget": 0.14, "threshold": -0.33570271730422974, "operator": "ge", "actual_fraction": 0.13999578325954037, "metric": 0.562794293344578, "rescued": 126, "harmed": 94, "missed_rescue": 4500, "avoided_harm": 1039, "selected_both_correct": 1698, "selected_both_wrong": 74}, {"budget": 0.15, "threshold": -0.3407592475414276, "operator": "ge", "actual_fraction": 0.14997540234731885, "metric": 0.5637781994518237, "rescued": 151, "harmed": 105, "missed_rescue": 4475, "avoided_harm": 1028, "selected_both_correct": 1795, "selected_both_wrong": 83}, {"budget": 0.16, "threshold": -0.34456855058670044, "operator": "ge", "actual_fraction": 0.15995502143509735, "metric": 0.5646918265514091, "rescued": 180, "harmed": 121, "missed_rescue": 4446, "avoided_harm": 1012, "selected_both_correct": 1888, "selected_both_wrong": 87}, {"budget": 0.17, "threshold": -0.3487326204776764, "operator": "ge", "actual_fraction": 0.16993464052287582, "metric": 0.5665190807505798, "rescued": 211, "harmed": 126, "missed_rescue": 4415, "avoided_harm": 1007, "selected_both_correct": 1981, "selected_both_wrong": 100}, {"budget": 0.18, "threshold": -0.35280776023864746, "operator": "ge", "actual_fraction": 0.1799845386183147, "metric": 0.5680652189191089, "rescued": 242, "harmed": 135, "missed_rescue": 4384, "avoided_harm": 998, "selected_both_correct": 2070, "selected_both_wrong": 114}, {"budget": 0.19, "threshold": -0.3564819097518921, "operator": "ge", "actual_fraction": 0.18996415770609318, "metric": 0.5697519151029588, "rescued": 276, "harmed": 145, "missed_rescue": 4350, "avoided_harm": 988, "selected_both_correct": 2159, "selected_both_wrong": 123}, {"budget": 0.2, "threshold": -0.35992002487182617, "operator": "ge", "actual_fraction": 0.19994377679387168, "metric": 0.5714386112868086, "rescued": 309, "harmed": 154, "missed_rescue": 4317, "avoided_harm": 979, "selected_both_correct": 2246, "selected_both_wrong": 136}, {"budget": 0.21, "threshold": -0.36340460181236267, "operator": "ge", "actual_fraction": 0.20999367488931056, "metric": 0.5737578185396022, "rescued": 351, "harmed": 163, "missed_rescue": 4275, "avoided_harm": 970, "selected_both_correct": 2328, "selected_both_wrong": 146}, {"budget": 0.22, "threshold": -0.3674520254135132, "operator": "ge", "actual_fraction": 0.21997329397708903, "metric": 0.5751633986928104, "rescued": 384, "harmed": 176, "missed_rescue": 4242, "avoided_harm": 957, "selected_both_correct": 2410, "selected_both_wrong": 160}, {"budget": 0.23, "threshold": -0.37108901143074036, "operator": "ge", "actual_fraction": 0.22995291306486754, "metric": 0.5764986998383583, "rescued": 414, "harmed": 187, "missed_rescue": 4212, "avoided_harm": 946, "selected_both_correct": 2499, "selected_both_wrong": 172}, {"budget": 0.24, "threshold": -0.3745429813861847, "operator": "ge", "actual_fraction": 0.239932532152646, "metric": 0.578325954037529, "rescued": 452, "harmed": 199, "missed_rescue": 4174, "avoided_harm": 934, "selected_both_correct": 2577, "selected_both_wrong": 186}, {"budget": 0.25, "threshold": -0.3773327171802521, "operator": "ge", "actual_fraction": 0.2499824302480849, "metric": 0.5805046032750018, "rescued": 490, "harmed": 206, "missed_rescue": 4136, "avoided_harm": 927, "selected_both_correct": 2664, "selected_both_wrong": 197}, {"budget": 0.26, "threshold": -0.3803459703922272, "operator": "ge", "actual_fraction": 0.25996204933586337, "metric": 0.5829643685431162, "rescued": 536, "harmed": 217, "missed_rescue": 4090, "avoided_harm": 916, "selected_both_correct": 2736, "selected_both_wrong": 210}, {"budget": 0.27, "threshold": -0.38361650705337524, "operator": "ge", "actual_fraction": 0.26994166842364187, "metric": 0.584651064726966, "rescued": 575, "harmed": 232, "missed_rescue": 4051, "avoided_harm": 901, "selected_both_correct": 2811, "selected_both_wrong": 223}, {"budget": 0.28, "threshold": -0.38652583956718445, "operator": "ge", "actual_fraction": 0.27999156651908075, "metric": 0.5876730620563637, "rescued": 626, "harmed": 240, "missed_rescue": 4000, "avoided_harm": 893, "selected_both_correct": 2882, "selected_both_wrong": 236}, {"budget": 0.29, "threshold": -0.38952407240867615, "operator": "ge", "actual_fraction": 0.28997118560685925, "metric": 0.5904842223627802, "rescued": 676, "harmed": 250, "missed_rescue": 3950, "avoided_harm": 883, "selected_both_correct": 2954, "selected_both_wrong": 246}, {"budget": 0.3, "threshold": -0.39262744784355164, "operator": "ge", "actual_fraction": 0.2999508046946377, "metric": 0.5935764986998383, "rescued": 726, "harmed": 256, "missed_rescue": 3900, "avoided_harm": 877, "selected_both_correct": 3029, "selected_both_wrong": 257}, {"budget": 0.31, "threshold": -0.3954668939113617, "operator": "ge", "actual_fraction": 0.3099304237824162, "metric": 0.5956145899219903, "rescued": 765, "harmed": 266, "missed_rescue": 3861, "avoided_harm": 867, "selected_both_correct": 3104, "selected_both_wrong": 275}, {"budget": 0.32, "threshold": -0.3986983895301819, "operator": "ge", "actual_fraction": 0.3199803218778551, "metric": 0.5973715651135006, "rescued": 804, "harmed": 280, "missed_rescue": 3822, "avoided_harm": 853, "selected_both_correct": 3178, "selected_both_wrong": 291}, {"budget": 0.33, "threshold": -0.4014112949371338, "operator": "ge", "actual_fraction": 0.3299599409656336, "metric": 0.5994096563356526, "rescued": 849, "harmed": 296, "missed_rescue": 3777, "avoided_harm": 837, "selected_both_correct": 3248, "selected_both_wrong": 302}, {"budget": 0.34, "threshold": -0.4042409658432007, "operator": "ge", "actual_fraction": 0.339939560053412, "metric": 0.6013774685501441, "rescued": 890, "harmed": 309, "missed_rescue": 3736, "avoided_harm": 824, "selected_both_correct": 3323, "selected_both_wrong": 315}, {"budget": 0.35, "threshold": -0.40724706649780273, "operator": "ge", "actual_fraction": 0.34998945814885096, "metric": 0.6035561177876169, "rescued": 932, "harmed": 320, "missed_rescue": 3694, "avoided_harm": 813, "selected_both_correct": 3396, "selected_both_wrong": 332}, {"budget": 0.36, "threshold": -0.40968337655067444, "operator": "ge", "actual_fraction": 0.3599690772366294, "metric": 0.6046805819101835, "rescued": 965, "harmed": 337, "missed_rescue": 3661, "avoided_harm": 796, "selected_both_correct": 3475, "selected_both_wrong": 345}, {"budget": 0.37, "threshold": -0.4123799502849579, "operator": "ge", "actual_fraction": 0.3699486963244079, "metric": 0.6079836952702228, "rescued": 1019, "harmed": 344, "missed_rescue": 3607, "avoided_harm": 789, "selected_both_correct": 3546, "selected_both_wrong": 355}, {"budget": 0.38, "threshold": -0.41514647006988525, "operator": "ge", "actual_fraction": 0.3799985944198468, "metric": 0.611778761683885, "rescued": 1079, "harmed": 350, "missed_rescue": 3547, "avoided_harm": 783, "selected_both_correct": 3611, "selected_both_wrong": 367}, {"budget": 0.39, "threshold": -0.4179389476776123, "operator": "ge", "actual_fraction": 0.3899782135076253, "metric": 0.6131843418370932, "rescued": 1121, "harmed": 372, "missed_rescue": 3505, "avoided_harm": 761, "selected_both_correct": 3675, "selected_both_wrong": 381}, {"budget": 0.4, "threshold": -0.42099592089653015, "operator": "ge", "actual_fraction": 0.39995783259540374, "metric": 0.6156441071052077, "rescued": 1166, "harmed": 382, "missed_rescue": 3460, "avoided_harm": 751, "selected_both_correct": 3747, "selected_both_wrong": 396}, {"budget": 0.41, "threshold": -0.4243226647377014, "operator": "ge", "actual_fraction": 0.40993745168318224, "metric": 0.617401082296718, "rescued": 1208, "harmed": 399, "missed_rescue": 3418, "avoided_harm": 734, "selected_both_correct": 3815, "selected_both_wrong": 411}, {"budget": 0.42, "threshold": -0.4275209903717041, "operator": "ge", "actual_fraction": 0.4199873497786211, "metric": 0.6203528006184552, "rescued": 1261, "harmed": 410, "missed_rescue": 3365, "avoided_harm": 723, "selected_both_correct": 3878, "selected_both_wrong": 427}, {"budget": 0.43, "threshold": -0.4307159185409546, "operator": "ge", "actual_fraction": 0.4299669688663996, "metric": 0.6228828448942301, "rescued": 1310, "harmed": 423, "missed_rescue": 3316, "avoided_harm": 710, "selected_both_correct": 3949, "selected_both_wrong": 436}, {"budget": 0.44, "threshold": -0.4336964190006256, "operator": "ge", "actual_fraction": 0.43994658795417807, "metric": 0.625764284208307, "rescued": 1364, "harmed": 436, "missed_rescue": 3262, "avoided_harm": 697, "selected_both_correct": 4015, "selected_both_wrong": 445}, {"budget": 0.45, "threshold": -0.4373021125793457, "operator": "ge", "actual_fraction": 0.449996486049617, "metric": 0.627661817415138, "rescued": 1410, "harmed": 455, "missed_rescue": 3216, "avoided_harm": 678, "selected_both_correct": 4077, "selected_both_wrong": 461}, {"budget": 0.46, "threshold": -0.4404309391975403, "operator": "ge", "actual_fraction": 0.45997610513739545, "metric": 0.6299107456602713, "rescued": 1459, "harmed": 472, "missed_rescue": 3167, "avoided_harm": 661, "selected_both_correct": 4140, "selected_both_wrong": 474}, {"budget": 0.47, "threshold": -0.44344601035118103, "operator": "ge", "actual_fraction": 0.46995572422517395, "metric": 0.6319488368824232, "rescued": 1501, "harmed": 485, "missed_rescue": 3125, "avoided_harm": 648, "selected_both_correct": 4210, "selected_both_wrong": 491}, {"budget": 0.48, "threshold": -0.4465356171131134, "operator": "ge", "actual_fraction": 0.4799353433129524, "metric": 0.6338463700892544, "rescued": 1545, "harmed": 502, "missed_rescue": 3081, "avoided_harm": 631, "selected_both_correct": 4281, "selected_both_wrong": 501}, {"budget": 0.49, "threshold": -0.44965147972106934, "operator": "ge", "actual_fraction": 0.48998524140839134, "metric": 0.6358844613114063, "rescued": 1592, "harmed": 520, "missed_rescue": 3034, "avoided_harm": 613, "selected_both_correct": 4344, "selected_both_wrong": 516}, {"budget": 0.5, "threshold": -0.453143447637558, "operator": "ge", "actual_fraction": 0.4999648604961698, "metric": 0.6390470166561248, "rescued": 1648, "harmed": 531, "missed_rescue": 2978, "avoided_harm": 602, "selected_both_correct": 4405, "selected_both_wrong": 530}, {"budget": 0.51, "threshold": -0.4565357565879822, "operator": "ge", "actual_fraction": 0.5099444795839483, "metric": 0.6419987349778621, "rescued": 1702, "harmed": 543, "missed_rescue": 2924, "avoided_harm": 590, "selected_both_correct": 4469, "selected_both_wrong": 542}, {"budget": 0.52, "threshold": -0.45961302518844604, "operator": "ge", "actual_fraction": 0.5199943776793872, "metric": 0.6445990582612974, "rescued": 1747, "harmed": 551, "missed_rescue": 2879, "avoided_harm": 582, "selected_both_correct": 4543, "selected_both_wrong": 558}, {"budget": 0.53, "threshold": -0.46368208527565, "operator": "ge", "actual_fraction": 0.5299739967671656, "metric": 0.6468479865064305, "rescued": 1795, "harmed": 567, "missed_rescue": 2831, "avoided_harm": 566, "selected_both_correct": 4608, "selected_both_wrong": 571}, {"budget": 0.54, "threshold": -0.46793216466903687, "operator": "ge", "actual_fraction": 0.5399536158549442, "metric": 0.6500808208588095, "rescued": 1850, "harmed": 576, "missed_rescue": 2776, "avoided_harm": 557, "selected_both_correct": 4672, "selected_both_wrong": 585}, {"budget": 0.55, "threshold": -0.4718930721282959, "operator": "ge", "actual_fraction": 0.5499332349427226, "metric": 0.653243376203528, "rescued": 1906, "harmed": 587, "missed_rescue": 2720, "avoided_harm": 546, "selected_both_correct": 4731, "selected_both_wrong": 601}, {"budget": 0.56, "threshold": -0.47629430890083313, "operator": "ge", "actual_fraction": 0.5599831330381615, "metric": 0.654438119333755, "rescued": 1942, "harmed": 606, "missed_rescue": 2684, "avoided_harm": 527, "selected_both_correct": 4800, "selected_both_wrong": 620}, {"budget": 0.57, "threshold": -0.4799485504627228, "operator": "ge", "actual_fraction": 0.5699627521259399, "metric": 0.6574601166631527, "rescued": 1998, "harmed": 619, "missed_rescue": 2628, "avoided_harm": 514, "selected_both_correct": 4861, "selected_both_wrong": 632}, {"budget": 0.58, "threshold": -0.4838995635509491, "operator": "ge", "actual_fraction": 0.5799423712137185, "metric": 0.6599901609389276, "rescued": 2048, "harmed": 633, "missed_rescue": 2578, "avoided_harm": 500, "selected_both_correct": 4921, "selected_both_wrong": 650}, {"budget": 0.59, "threshold": -0.4886912405490875, "operator": "ge", "actual_fraction": 0.5899922693091574, "metric": 0.6618876941457587, "rescued": 2098, "harmed": 656, "missed_rescue": 2528, "avoided_harm": 477, "selected_both_correct": 4979, "selected_both_wrong": 662}, {"budget": 0.6, "threshold": -0.49332454800605774, "operator": "ge", "actual_fraction": 0.5999718883969358, "metric": 0.6642771804062126, "rescued": 2146, "harmed": 670, "missed_rescue": 2480, "avoided_harm": 463, "selected_both_correct": 5049, "selected_both_wrong": 672}, {"budget": 0.61, "threshold": -0.49750789999961853, "operator": "ge", "actual_fraction": 0.6099515074847143, "metric": 0.6670883407126291, "rescued": 2199, "harmed": 683, "missed_rescue": 2427, "avoided_harm": 450, "selected_both_correct": 5115, "selected_both_wrong": 682}, {"budget": 0.62, "threshold": -0.5024909377098083, "operator": "ge", "actual_fraction": 0.6199311265724928, "metric": 0.668775036896479, "rescued": 2238, "harmed": 698, "missed_rescue": 2388, "avoided_harm": 435, "selected_both_correct": 5185, "selected_both_wrong": 700}, {"budget": 0.63, "threshold": -0.5067122578620911, "operator": "ge", "actual_fraction": 0.6299810246679317, "metric": 0.6707428491109706, "rescued": 2280, "harmed": 712, "missed_rescue": 2346, "avoided_harm": 421, "selected_both_correct": 5258, "selected_both_wrong": 714}, {"budget": 0.64, "threshold": -0.512323796749115, "operator": "ge", "actual_fraction": 0.6399606437557102, "metric": 0.6732728933867453, "rescued": 2329, "harmed": 725, "missed_rescue": 2297, "avoided_harm": 408, "selected_both_correct": 5325, "selected_both_wrong": 727}, {"budget": 0.65, "threshold": -0.5170255303382874, "operator": "ge", "actual_fraction": 0.6499402628434886, "metric": 0.675592100639539, "rescued": 2377, "harmed": 740, "missed_rescue": 2249, "avoided_harm": 393, "selected_both_correct": 5386, "selected_both_wrong": 745}, {"budget": 0.66, "threshold": -0.5215396881103516, "operator": "ge", "actual_fraction": 0.6599901609389276, "metric": 0.6772787968233889, "rescued": 2417, "harmed": 756, "missed_rescue": 2209, "avoided_harm": 377, "selected_both_correct": 5453, "selected_both_wrong": 765}, {"budget": 0.67, "threshold": -0.5264327526092529, "operator": "ge", "actual_fraction": 0.669969780026706, "metric": 0.6788952139995783, "rescued": 2457, "harmed": 773, "missed_rescue": 2169, "avoided_harm": 360, "selected_both_correct": 5522, "selected_both_wrong": 781}, {"budget": 0.68, "threshold": -0.5312389135360718, "operator": "ge", "actual_fraction": 0.6799493991144845, "metric": 0.6810738632370511, "rescued": 2504, "harmed": 789, "missed_rescue": 2122, "avoided_harm": 344, "selected_both_correct": 5585, "selected_both_wrong": 797}, {"budget": 0.69, "threshold": -0.5368650555610657, "operator": "ge", "actual_fraction": 0.6899992972099234, "metric": 0.6833930704898447, "rescued": 2550, "harmed": 802, "missed_rescue": 2076, "avoided_harm": 331, "selected_both_correct": 5652, "selected_both_wrong": 814}, {"budget": 0.7, "threshold": -0.5431069135665894, "operator": "ge", "actual_fraction": 0.6999789162977019, "metric": 0.6867664628575445, "rescued": 2614, "harmed": 818, "missed_rescue": 2012, "avoided_harm": 315, "selected_both_correct": 5698, "selected_both_wrong": 830}, {"budget": 0.71, "threshold": -0.5487136244773865, "operator": "ge", "actual_fraction": 0.7099585353854804, "metric": 0.6895073441563006, "rescued": 2664, "harmed": 829, "missed_rescue": 1962, "avoided_harm": 304, "selected_both_correct": 5762, "selected_both_wrong": 847}, {"budget": 0.72, "threshold": -0.5549414753913879, "operator": "ge", "actual_fraction": 0.7199381544732588, "metric": 0.6909129243095088, "rescued": 2704, "harmed": 849, "missed_rescue": 1922, "avoided_harm": 284, "selected_both_correct": 5830, "selected_both_wrong": 861}, {"budget": 0.73, "threshold": -0.5604677200317383, "operator": "ge", "actual_fraction": 0.7299880525686977, "metric": 0.6918265514090941, "rescued": 2739, "harmed": 871, "missed_rescue": 1887, "avoided_harm": 262, "selected_both_correct": 5903, "selected_both_wrong": 874}, {"budget": 0.74, "threshold": -0.5666090250015259, "operator": "ge", "actual_fraction": 0.7399676716564763, "metric": 0.6940754796542273, "rescued": 2787, "harmed": 887, "missed_rescue": 1839, "avoided_harm": 246, "selected_both_correct": 5960, "selected_both_wrong": 895}, {"budget": 0.75, "threshold": -0.574927568435669, "operator": "ge", "actual_fraction": 0.7499472907442547, "metric": 0.6962541288917, "rescued": 2831, "harmed": 900, "missed_rescue": 1795, "avoided_harm": 233, "selected_both_correct": 6025, "selected_both_wrong": 915}, {"budget": 0.76, "threshold": -0.5812664031982422, "operator": "ge", "actual_fraction": 0.7599971888396936, "metric": 0.6987841731674749, "rescued": 2879, "harmed": 912, "missed_rescue": 1747, "avoided_harm": 221, "selected_both_correct": 6094, "selected_both_wrong": 929}, {"budget": 0.77, "threshold": -0.5883752107620239, "operator": "ge", "actual_fraction": 0.769976807927472, "metric": 0.7008925433972872, "rescued": 2923, "harmed": 926, "missed_rescue": 1703, "avoided_harm": 207, "selected_both_correct": 6166, "selected_both_wrong": 941}, {"budget": 0.78, "threshold": -0.5972744226455688, "operator": "ge", "actual_fraction": 0.7799564270152506, "metric": 0.7033523086654017, "rescued": 2977, "harmed": 945, "missed_rescue": 1649, "avoided_harm": 188, "selected_both_correct": 6220, "selected_both_wrong": 956}, {"budget": 0.79, "threshold": -0.6058458089828491, "operator": "ge", "actual_fraction": 0.789936046103029, "metric": 0.7058120739335161, "rescued": 3026, "harmed": 959, "missed_rescue": 1600, "avoided_harm": 174, "selected_both_correct": 6288, "selected_both_wrong": 967}, {"budget": 0.8, "threshold": -0.6140543818473816, "operator": "ge", "actual_fraction": 0.7999859441984679, "metric": 0.7068662590484223, "rescued": 3067, "harmed": 985, "missed_rescue": 1559, "avoided_harm": 148, "selected_both_correct": 6341, "selected_both_wrong": 990}, {"budget": 0.81, "threshold": -0.6230268478393555, "operator": "ge", "actual_fraction": 0.8099655632862464, "metric": 0.7082718392016305, "rescued": 3109, "harmed": 1007, "missed_rescue": 1517, "avoided_harm": 126, "selected_both_correct": 6402, "selected_both_wrong": 1007}, {"budget": 0.82, "threshold": -0.6330366134643555, "operator": "ge", "actual_fraction": 0.8199451823740249, "metric": 0.7100990934008011, "rescued": 3152, "harmed": 1024, "missed_rescue": 1474, "avoided_harm": 109, "selected_both_correct": 6466, "selected_both_wrong": 1025}, {"budget": 0.83, "threshold": -0.6429333686828613, "operator": "ge", "actual_fraction": 0.8299950804694638, "metric": 0.711434394546349, "rescued": 3192, "harmed": 1045, "missed_rescue": 1434, "avoided_harm": 88, "selected_both_correct": 6528, "selected_both_wrong": 1045}, {"budget": 0.84, "threshold": -0.6513019800186157, "operator": "ge", "actual_fraction": 0.8399746995572422, "metric": 0.7131913697378593, "rescued": 3238, "harmed": 1066, "missed_rescue": 1388, "avoided_harm": 67, "selected_both_correct": 6584, "selected_both_wrong": 1064}, {"budget": 0.85, "threshold": -0.6624053120613098, "operator": "ge", "actual_fraction": 0.8499543186450207, "metric": 0.7150889029446904, "rescued": 3282, "harmed": 1083, "missed_rescue": 1344, "avoided_harm": 50, "selected_both_correct": 6639, "selected_both_wrong": 1090}, {"budget": 0.86, "threshold": -0.6759263277053833, "operator": "ge", "actual_fraction": 0.8599339377327992, "metric": 0.7167755991285403, "rescued": 3326, "harmed": 1103, "missed_rescue": 1300, "avoided_harm": 30, "selected_both_correct": 6698, "selected_both_wrong": 1109}, {"budget": 0.87, "threshold": -0.6904275417327881, "operator": "ge", "actual_fraction": 0.8699838358282381, "metric": 0.7191650853889943, "rescued": 3371, "harmed": 1114, "missed_rescue": 1255, "avoided_harm": 19, "selected_both_correct": 6772, "selected_both_wrong": 1122}, {"budget": 0.88, "threshold": -0.7045450210571289, "operator": "ge", "actual_fraction": 0.8799634549160166, "metric": 0.7228195937873357, "rescued": 3437, "harmed": 1128, "missed_rescue": 1189, "avoided_harm": 5, "selected_both_correct": 6814, "selected_both_wrong": 1142}, {"budget": 0.89, "threshold": -0.7205488681793213, "operator": "ge", "actual_fraction": 0.889943074003795, "metric": 0.7301286105840186, "rescued": 3541, "harmed": 1128, "missed_rescue": 1085, "avoided_harm": 5, "selected_both_correct": 6815, "selected_both_wrong": 1179}, {"budget": 0.9, "threshold": -0.7357243299484253, "operator": "ge", "actual_fraction": 0.899992972099234, "metric": 0.7374376273807014, "rescued": 3645, "harmed": 1128, "missed_rescue": 981, "avoided_harm": 5, "selected_both_correct": 6815, "selected_both_wrong": 1218}, {"budget": 0.91, "threshold": -0.7513164281845093, "operator": "ge", "actual_fraction": 0.9099725911870125, "metric": 0.7445358071544029, "rescued": 3746, "harmed": 1128, "missed_rescue": 880, "avoided_harm": 5, "selected_both_correct": 6815, "selected_both_wrong": 1259}, {"budget": 0.92, "threshold": -0.7671180963516235, "operator": "ge", "actual_fraction": 0.9199522102747909, "metric": 0.7521962189893878, "rescued": 3855, "harmed": 1128, "missed_rescue": 771, "avoided_harm": 5, "selected_both_correct": 6817, "selected_both_wrong": 1290}, {"budget": 0.93, "threshold": -0.7833515405654907, "operator": "ge", "actual_fraction": 0.9299318293625694, "metric": 0.758872724717127, "rescued": 3951, "harmed": 1129, "missed_rescue": 675, "avoided_harm": 4, "selected_both_correct": 6818, "selected_both_wrong": 1334}, {"budget": 0.94, "threshold": -0.8013419508934021, "operator": "ge", "actual_fraction": 0.9399817274580083, "metric": 0.7654789514372057, "rescued": 4045, "harmed": 1129, "missed_rescue": 581, "avoided_harm": 4, "selected_both_correct": 6821, "selected_both_wrong": 1380}, {"budget": 0.95, "threshold": -0.8188987374305725, "operator": "ge", "actual_fraction": 0.9499613465457868, "metric": 0.7727879682338885, "rescued": 4149, "harmed": 1129, "missed_rescue": 477, "avoided_harm": 4, "selected_both_correct": 6822, "selected_both_wrong": 1417}, {"budget": 0.96, "threshold": -0.8395025134086609, "operator": "ge", "actual_fraction": 0.9599409656335652, "metric": 0.7791130789233256, "rescued": 4240, "harmed": 1130, "missed_rescue": 386, "avoided_harm": 3, "selected_both_correct": 6826, "selected_both_wrong": 1463}, {"budget": 0.97, "threshold": -0.8628664612770081, "operator": "ge", "actual_fraction": 0.9699908637290041, "metric": 0.7857193056434043, "rescued": 4335, "harmed": 1131, "missed_rescue": 291, "avoided_harm": 2, "selected_both_correct": 6829, "selected_both_wrong": 1507}, {"budget": 0.98, "threshold": -0.8857419490814209, "operator": "ge", "actual_fraction": 0.9799704828167827, "metric": 0.7927472064094455, "rescued": 4435, "harmed": 1131, "missed_rescue": 191, "avoided_harm": 2, "selected_both_correct": 6835, "selected_both_wrong": 1543}, {"budget": 0.99, "threshold": -0.9162307977676392, "operator": "ge", "actual_fraction": 0.9899501019045611, "metric": 0.7994237121371847, "rescued": 4531, "harmed": 1132, "missed_rescue": 95, "avoided_harm": 1, "selected_both_correct": 6840, "selected_both_wrong": 1583}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.8060299388572634, "rescued": 4626, "harmed": 1133, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 6843, "selected_both_wrong": 1627}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.26688029946402747, "rescued": 0, "harmed": 0, "missed_rescue": 18820, "avoided_harm": 1970, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.1584138125181198, "operator": "ge", "actual_fraction": 0.029095652667101495, "metric": 0.2670788078155574, "rescued": 15, "harmed": 8, "missed_rescue": 18805, "avoided_harm": 1962, "selected_both_correct": 994, "selected_both_wrong": 9}, {"budget": 0.02, "threshold": -0.21403978765010834, "operator": "ge", "actual_fraction": 0.04622408757054136, "metric": 0.26747582451861723, "rescued": 38, "harmed": 17, "missed_rescue": 18782, "avoided_harm": 1953, "selected_both_correct": 1519, "selected_both_wrong": 56}, {"budget": 0.03, "threshold": -0.24253316223621368, "operator": "ge", "actual_fraction": 0.056149505147037974, "metric": 0.26821314125287127, "rescued": 68, "harmed": 21, "missed_rescue": 18752, "avoided_harm": 1949, "selected_both_correct": 1793, "selected_both_wrong": 98}, {"budget": 0.04, "threshold": -0.2599165141582489, "operator": "ge", "actual_fraction": 0.06372118084110824, "metric": 0.26883702464339393, "rescued": 98, "harmed": 29, "missed_rescue": 18722, "avoided_harm": 1941, "selected_both_correct": 1984, "selected_both_wrong": 136}, {"budget": 0.05, "threshold": -0.2725968658924103, "operator": "ge", "actual_fraction": 0.06882568130902078, "metric": 0.2692907580183195, "rescued": 125, "harmed": 40, "missed_rescue": 18695, "avoided_harm": 1930, "selected_both_correct": 2087, "selected_both_wrong": 175}, {"budget": 0.06, "threshold": -0.2842792272567749, "operator": "ge", "actual_fraction": 0.07475257351898591, "metric": 0.2701415080963049, "rescued": 161, "harmed": 46, "missed_rescue": 18659, "avoided_harm": 1924, "selected_both_correct": 2200, "selected_both_wrong": 229}, {"budget": 0.07, "threshold": -0.2935555577278137, "operator": "ge", "actual_fraction": 0.08033916569775686, "metric": 0.27119076652582025, "rescued": 209, "harmed": 57, "missed_rescue": 18611, "avoided_harm": 1913, "selected_both_correct": 2298, "selected_both_wrong": 269}, {"budget": 0.08, "threshold": -0.30114513635635376, "operator": "ge", "actual_fraction": 0.08632277457958767, "metric": 0.27260868332246263, "rescued": 270, "harmed": 68, "missed_rescue": 18550, "avoided_harm": 1902, "selected_both_correct": 2381, "selected_both_wrong": 325}, {"budget": 0.09, "threshold": -0.308601438999176, "operator": "ge", "actual_fraction": 0.09227802512548564, "metric": 0.27419675013470207, "rescued": 338, "harmed": 80, "missed_rescue": 18482, "avoided_harm": 1890, "selected_both_correct": 2473, "selected_both_wrong": 363}, {"budget": 0.1, "threshold": -0.3146189749240875, "operator": "ge", "actual_fraction": 0.09783625896832374, "metric": 0.275699741939143, "rescued": 405, "harmed": 94, "missed_rescue": 18415, "avoided_harm": 1876, "selected_both_correct": 2542, "selected_both_wrong": 409}, {"budget": 0.11, "threshold": -0.32043084502220154, "operator": "ge", "actual_fraction": 0.10362135949862462, "metric": 0.2776564671185095, "rescued": 482, "harmed": 102, "missed_rescue": 18338, "avoided_harm": 1868, "selected_both_correct": 2610, "selected_both_wrong": 460}, {"budget": 0.12, "threshold": -0.32644838094711304, "operator": "ge", "actual_fraction": 0.11031392677877662, "metric": 0.28032215069619715, "rescued": 589, "harmed": 115, "missed_rescue": 18231, "avoided_harm": 1855, "selected_both_correct": 2673, "selected_both_wrong": 513}, {"budget": 0.13, "threshold": -0.33121857047080994, "operator": "ge", "actual_fraction": 0.11658111901993591, "metric": 0.28267589257862347, "rescued": 680, "harmed": 123, "missed_rescue": 18140, "avoided_harm": 1847, "selected_both_correct": 2746, "selected_both_wrong": 562}, {"budget": 0.14, "threshold": -0.33570271730422974, "operator": "ge", "actual_fraction": 0.12262144457363242, "metric": 0.2851997844766469, "rescued": 777, "harmed": 131, "missed_rescue": 18043, "avoided_harm": 1839, "selected_both_correct": 2805, "selected_both_wrong": 611}, {"budget": 0.15, "threshold": -0.3407592475414276, "operator": "ge", "actual_fraction": 0.13024983693956838, "metric": 0.2880072597339988, "rescued": 897, "harmed": 152, "missed_rescue": 17923, "avoided_harm": 1818, "selected_both_correct": 2870, "selected_both_wrong": 674}, {"budget": 0.16, "threshold": -0.34456855058670044, "operator": "ge", "actual_fraction": 0.13714091257125033, "metric": 0.29106996001474633, "rescued": 1018, "harmed": 165, "missed_rescue": 17802, "avoided_harm": 1805, "selected_both_correct": 2930, "selected_both_wrong": 723}, {"budget": 0.17, "threshold": -0.3487326204776764, "operator": "ge", "actual_fraction": 0.143521538156141, "metric": 0.2940475852876953, "rescued": 1137, "harmed": 179, "missed_rescue": 17683, "avoided_harm": 1791, "selected_both_correct": 2975, "selected_both_wrong": 770}, {"budget": 0.18, "threshold": -0.35280776023864746, "operator": "ge", "actual_fraction": 0.15095142217054702, "metric": 0.29747894393556984, "rescued": 1269, "harmed": 190, "missed_rescue": 17551, "avoided_harm": 1780, "selected_both_correct": 3022, "selected_both_wrong": 842}, {"budget": 0.19, "threshold": -0.3564819097518921, "operator": "ge", "actual_fraction": 0.15823951450528884, "metric": 0.3003714942007203, "rescued": 1395, "harmed": 214, "missed_rescue": 17425, "avoided_harm": 1756, "selected_both_correct": 3075, "selected_both_wrong": 896}, {"budget": 0.2, "threshold": -0.35992002487182617, "operator": "ge", "actual_fraction": 0.16496044012137367, "metric": 0.30334911947366927, "rescued": 1511, "harmed": 225, "missed_rescue": 17309, "avoided_harm": 1745, "selected_both_correct": 3127, "selected_both_wrong": 954}, {"budget": 0.21, "threshold": -0.36340460181236267, "operator": "ge", "actual_fraction": 0.17290077418257097, "metric": 0.30697898647307376, "rescued": 1651, "harmed": 237, "missed_rescue": 17169, "avoided_harm": 1733, "selected_both_correct": 3185, "selected_both_wrong": 1024}, {"budget": 0.22, "threshold": -0.3674520254135132, "operator": "ge", "actual_fraction": 0.1829679834387318, "metric": 0.3114879618863965, "rescued": 1832, "harmed": 259, "missed_rescue": 16988, "avoided_harm": 1711, "selected_both_correct": 3252, "selected_both_wrong": 1109}, {"budget": 0.23, "threshold": -0.37108901143074036, "operator": "ge", "actual_fraction": 0.19122025919519042, "metric": 0.31480588719053965, "rescued": 1970, "harmed": 280, "missed_rescue": 16850, "avoided_harm": 1690, "selected_both_correct": 3314, "selected_both_wrong": 1179}, {"budget": 0.24, "threshold": -0.3745429813861847, "operator": "ge", "actual_fraction": 0.1999262683265746, "metric": 0.3192581459319967, "rescued": 2144, "harmed": 297, "missed_rescue": 16676, "avoided_harm": 1673, "selected_both_correct": 3372, "selected_both_wrong": 1237}, {"budget": 0.25, "threshold": -0.3773327171802521, "operator": "ge", "actual_fraction": 0.2080651107393018, "metric": 0.32294472960326687, "rescued": 2294, "harmed": 317, "missed_rescue": 16526, "avoided_harm": 1653, "selected_both_correct": 3423, "selected_both_wrong": 1303}, {"budget": 0.26, "threshold": -0.3803459703922272, "operator": "ge", "actual_fraction": 0.21603380313643195, "metric": 0.3270283299775969, "rescued": 2457, "harmed": 336, "missed_rescue": 16363, "avoided_harm": 1634, "selected_both_correct": 3465, "selected_both_wrong": 1360}, {"budget": 0.27, "threshold": -0.38361650705337524, "operator": "ge", "actual_fraction": 0.2243144372288234, "metric": 0.33170745540651675, "rescued": 2634, "harmed": 348, "missed_rescue": 16186, "avoided_harm": 1622, "selected_both_correct": 3522, "selected_both_wrong": 1406}, {"budget": 0.28, "threshold": -0.38652583956718445, "operator": "ge", "actual_fraction": 0.2324249213056178, "metric": 0.33542239741371976, "rescued": 2793, "harmed": 376, "missed_rescue": 16027, "avoided_harm": 1594, "selected_both_correct": 3554, "selected_both_wrong": 1473}, {"budget": 0.29, "threshold": -0.38952407240867615, "operator": "ge", "actual_fraction": 0.2419816805149874, "metric": 0.3401015228426396, "rescued": 2975, "harmed": 393, "missed_rescue": 15845, "avoided_harm": 1577, "selected_both_correct": 3623, "selected_both_wrong": 1542}, {"budget": 0.3, "threshold": -0.39262744784355164, "operator": "ge", "actual_fraction": 0.2512832147009613, "metric": 0.3448373649434251, "rescued": 3169, "harmed": 420, "missed_rescue": 15651, "avoided_harm": 1550, "selected_both_correct": 3674, "selected_both_wrong": 1598}, {"budget": 0.31, "threshold": -0.3954668939113617, "operator": "ge", "actual_fraction": 0.2596772821370842, "metric": 0.3491194736692851, "rescued": 3340, "harmed": 440, "missed_rescue": 15480, "avoided_harm": 1530, "selected_both_correct": 3706, "selected_both_wrong": 1671}, {"budget": 0.32, "threshold": -0.3986983895301819, "operator": "ge", "actual_fraction": 0.26985792473697645, "metric": 0.35459263250432466, "rescued": 3553, "harmed": 460, "missed_rescue": 15267, "avoided_harm": 1510, "selected_both_correct": 3758, "selected_both_wrong": 1745}, {"budget": 0.33, "threshold": -0.4014112949371338, "operator": "ge", "actual_fraction": 0.27757139211071097, "metric": 0.35878966622238606, "rescued": 3718, "harmed": 477, "missed_rescue": 15102, "avoided_harm": 1493, "selected_both_correct": 3788, "selected_both_wrong": 1805}, {"budget": 0.34, "threshold": -0.4042409658432007, "operator": "ge", "actual_fraction": 0.28729830133567763, "metric": 0.3635255083231716, "rescued": 3913, "harmed": 505, "missed_rescue": 14907, "avoided_harm": 1465, "selected_both_correct": 3845, "selected_both_wrong": 1868}, {"budget": 0.35, "threshold": -0.40724706649780273, "operator": "ge", "actual_fraction": 0.29793267731049544, "metric": 0.36928225051753966, "rescued": 4137, "harmed": 526, "missed_rescue": 14683, "avoided_harm": 1444, "selected_both_correct": 3903, "selected_both_wrong": 1940}, {"budget": 0.36, "threshold": -0.40968337655067444, "operator": "ge", "actual_fraction": 0.3067237614496781, "metric": 0.3743583926495193, "rescued": 4332, "harmed": 542, "missed_rescue": 14488, "avoided_harm": 1428, "selected_both_correct": 3949, "selected_both_wrong": 1993}, {"budget": 0.37, "threshold": -0.4123799502849579, "operator": "ge", "actual_fraction": 0.3161387289793835, "metric": 0.3794912514533647, "rescued": 4536, "harmed": 565, "missed_rescue": 14284, "avoided_harm": 1405, "selected_both_correct": 3993, "selected_both_wrong": 2054}, {"budget": 0.38, "threshold": -0.41514647006988525, "operator": "ge", "actual_fraction": 0.32538354649349177, "metric": 0.3843121685619488, "rescued": 4725, "harmed": 584, "missed_rescue": 14095, "avoided_harm": 1386, "selected_both_correct": 4050, "selected_both_wrong": 2115}, {"budget": 0.39, "threshold": -0.4179389476776123, "operator": "ge", "actual_fraction": 0.3354507557496526, "metric": 0.389671894053257, "rescued": 4940, "harmed": 610, "missed_rescue": 13880, "avoided_harm": 1360, "selected_both_correct": 4104, "selected_both_wrong": 2175}, {"budget": 0.4, "threshold": -0.42099592089653015, "operator": "ge", "actual_fraction": 0.34653886509939597, "metric": 0.395428636247625, "rescued": 5169, "harmed": 636, "missed_rescue": 13651, "avoided_harm": 1334, "selected_both_correct": 4154, "selected_both_wrong": 2261}, {"budget": 0.41, "threshold": -0.4243226647377014, "operator": "ge", "actual_fraction": 0.3577404077928707, "metric": 0.4010719450982616, "rescued": 5399, "harmed": 667, "missed_rescue": 13421, "avoided_harm": 1303, "selected_both_correct": 4211, "selected_both_wrong": 2338}, {"budget": 0.42, "threshold": -0.4275209903717041, "operator": "ge", "actual_fraction": 0.3693673255253382, "metric": 0.40776451237841366, "rescued": 5657, "harmed": 689, "missed_rescue": 13163, "avoided_harm": 1281, "selected_both_correct": 4262, "selected_both_wrong": 2417}, {"budget": 0.43, "threshold": -0.4307159185409546, "operator": "ge", "actual_fraction": 0.38039871820321586, "metric": 0.41423021297110285, "rescued": 5901, "harmed": 705, "missed_rescue": 12919, "avoided_harm": 1265, "selected_both_correct": 4323, "selected_both_wrong": 2485}, {"budget": 0.44, "threshold": -0.4336964190006256, "operator": "ge", "actual_fraction": 0.39049428579530954, "metric": 0.42007203017326944, "rescued": 6128, "harmed": 726, "missed_rescue": 12692, "avoided_harm": 1244, "selected_both_correct": 4373, "selected_both_wrong": 2543}, {"budget": 0.45, "threshold": -0.4373021125793457, "operator": "ge", "actual_fraction": 0.4023764285511726, "metric": 0.42665116410969006, "rescued": 6384, "harmed": 750, "missed_rescue": 12436, "avoided_harm": 1220, "selected_both_correct": 4432, "selected_both_wrong": 2623}, {"budget": 0.46, "threshold": -0.4404309391975403, "operator": "ge", "actual_fraction": 0.41196154609647506, "metric": 0.4321810396165953, "rescued": 6599, "harmed": 770, "missed_rescue": 12221, "avoided_harm": 1200, "selected_both_correct": 4474, "selected_both_wrong": 2684}, {"budget": 0.47, "threshold": -0.44344601035118103, "operator": "ge", "actual_fraction": 0.42157502197771035, "metric": 0.43782434846723195, "rescued": 6813, "harmed": 785, "missed_rescue": 12007, "avoided_harm": 1185, "selected_both_correct": 4509, "selected_both_wrong": 2759}, {"budget": 0.48, "threshold": -0.4465356171131134, "operator": "ge", "actual_fraction": 0.43184073958540115, "metric": 0.4434960156538014, "rescued": 7036, "harmed": 808, "missed_rescue": 11784, "avoided_harm": 1162, "selected_both_correct": 4557, "selected_both_wrong": 2827}, {"budget": 0.49, "threshold": -0.44965147972106934, "operator": "ge", "actual_fraction": 0.44165272381816634, "metric": 0.4488557411451096, "rescued": 7253, "harmed": 836, "missed_rescue": 11567, "avoided_harm": 1134, "selected_both_correct": 4604, "selected_both_wrong": 2881}, {"budget": 0.5, "threshold": -0.453143447637558, "operator": "ge", "actual_fraction": 0.452315458128917, "metric": 0.4548393500269404, "rescued": 7483, "harmed": 855, "missed_rescue": 11337, "avoided_harm": 1115, "selected_both_correct": 4651, "selected_both_wrong": 2961}, {"budget": 0.51, "threshold": -0.4565357565879822, "operator": "ge", "actual_fraction": 0.4631483424552647, "metric": 0.4610781839321669, "rescued": 7721, "harmed": 873, "missed_rescue": 11099, "avoided_harm": 1097, "selected_both_correct": 4690, "selected_both_wrong": 3048}, {"budget": 0.52, "threshold": -0.45961302518844604, "operator": "ge", "actual_fraction": 0.47176927657885037, "metric": 0.46538865099395965, "rescued": 7891, "harmed": 891, "missed_rescue": 10929, "avoided_harm": 1079, "selected_both_correct": 4737, "selected_both_wrong": 3117}, {"budget": 0.53, "threshold": -0.46368208527565, "operator": "ge", "actual_fraction": 0.4840484360377733, "metric": 0.4715707682273204, "rescued": 8142, "harmed": 924, "missed_rescue": 10678, "avoided_harm": 1046, "selected_both_correct": 4791, "selected_both_wrong": 3212}, {"budget": 0.54, "threshold": -0.46793216466903687, "operator": "ge", "actual_fraction": 0.495079828715651, "metric": 0.47809318549187535, "rescued": 8393, "harmed": 945, "missed_rescue": 10427, "avoided_harm": 1025, "selected_both_correct": 4846, "selected_both_wrong": 3274}, {"budget": 0.55, "threshold": -0.4718930721282959, "operator": "ge", "actual_fraction": 0.5065365964325214, "metric": 0.48393500269404194, "rescued": 8631, "harmed": 977, "missed_rescue": 10189, "avoided_harm": 993, "selected_both_correct": 4900, "selected_both_wrong": 3354}, {"budget": 0.56, "threshold": -0.47629430890083313, "operator": "ge", "actual_fraction": 0.5183620225165188, "metric": 0.4900887615914698, "rescued": 8881, "harmed": 1010, "missed_rescue": 9939, "avoided_harm": 960, "selected_both_correct": 4950, "selected_both_wrong": 3438}, {"budget": 0.57, "threshold": -0.4799485504627228, "operator": "ge", "actual_fraction": 0.5273232566712985, "metric": 0.4943708703173298, "rescued": 9063, "harmed": 1041, "missed_rescue": 9757, "avoided_harm": 929, "selected_both_correct": 4989, "selected_both_wrong": 3502}, {"budget": 0.58, "threshold": -0.4838995635509491, "operator": "ge", "actual_fraction": 0.5381561409976463, "metric": 0.5008082125740861, "rescued": 9310, "harmed": 1061, "missed_rescue": 9510, "avoided_harm": 909, "selected_both_correct": 5038, "selected_both_wrong": 3568}, {"budget": 0.59, "threshold": -0.4886912405490875, "operator": "ge", "actual_fraction": 0.5503502254487707, "metric": 0.5080395882369623, "rescued": 9590, "harmed": 1086, "missed_rescue": 9230, "avoided_harm": 884, "selected_both_correct": 5080, "selected_both_wrong": 3651}, {"budget": 0.6, "threshold": -0.49332454800605774, "operator": "ge", "actual_fraction": 0.5619487848453053, "metric": 0.5147321555171143, "rescued": 9849, "harmed": 1109, "missed_rescue": 8971, "avoided_harm": 861, "selected_both_correct": 5127, "selected_both_wrong": 3731}, {"budget": 0.61, "threshold": -0.49750789999961853, "operator": "ge", "actual_fraction": 0.5722712191248618, "metric": 0.5204038227036838, "rescued": 10085, "harmed": 1145, "missed_rescue": 8735, "avoided_harm": 825, "selected_both_correct": 5165, "selected_both_wrong": 3785}, {"budget": 0.62, "threshold": -0.5024909377098083, "operator": "ge", "actual_fraction": 0.5818563366701642, "metric": 0.5257635481949919, "rescued": 10296, "harmed": 1167, "missed_rescue": 8524, "avoided_harm": 803, "selected_both_correct": 5206, "selected_both_wrong": 3849}, {"budget": 0.63, "threshold": -0.5067122578620911, "operator": "ge", "actual_fraction": 0.590817570824944, "metric": 0.5306128236395088, "rescued": 10487, "harmed": 1187, "missed_rescue": 8333, "avoided_harm": 783, "selected_both_correct": 5248, "selected_both_wrong": 3912}, {"budget": 0.64, "threshold": -0.512323796749115, "operator": "ge", "actual_fraction": 0.604202705385248, "metric": 0.5381277826617135, "rescued": 10781, "harmed": 1216, "missed_rescue": 8039, "avoided_harm": 754, "selected_both_correct": 5304, "selected_both_wrong": 4005}, {"budget": 0.65, "threshold": -0.5170255303382874, "operator": "ge", "actual_fraction": 0.6130505061962964, "metric": 0.5431472081218274, "rescued": 10982, "harmed": 1240, "missed_rescue": 7838, "avoided_harm": 730, "selected_both_correct": 5343, "selected_both_wrong": 4053}, {"budget": 0.66, "threshold": -0.5215396881103516, "operator": "ge", "actual_fraction": 0.6222102487026061, "metric": 0.548421858605337, "rescued": 11191, "harmed": 1263, "missed_rescue": 7629, "avoided_harm": 707, "selected_both_correct": 5388, "selected_both_wrong": 4099}, {"budget": 0.67, "threshold": -0.5264327526092529, "operator": "ge", "actual_fraction": 0.6320505912713043, "metric": 0.552902475682727, "rescued": 11382, "harmed": 1296, "missed_rescue": 7438, "avoided_harm": 674, "selected_both_correct": 5428, "selected_both_wrong": 4182}, {"budget": 0.68, "threshold": -0.5312389135360718, "operator": "ge", "actual_fraction": 0.6424297422227264, "metric": 0.5588010095567592, "rescued": 11615, "harmed": 1321, "missed_rescue": 7205, "avoided_harm": 649, "selected_both_correct": 5473, "selected_both_wrong": 4245}, {"budget": 0.69, "threshold": -0.5368650555610657, "operator": "ge", "actual_fraction": 0.6538865099395967, "metric": 0.5642174517199331, "rescued": 11846, "harmed": 1361, "missed_rescue": 6974, "avoided_harm": 609, "selected_both_correct": 5528, "selected_both_wrong": 4323}, {"budget": 0.7, "threshold": -0.5431069135665894, "operator": "ge", "actual_fraction": 0.6647761109378102, "metric": 0.5704562856251595, "rescued": 12087, "harmed": 1382, "missed_rescue": 6733, "avoided_harm": 588, "selected_both_correct": 5573, "selected_both_wrong": 4400}, {"budget": 0.71, "threshold": -0.5487136244773865, "operator": "ge", "actual_fraction": 0.6740492867878513, "metric": 0.5753339194056093, "rescued": 12285, "harmed": 1408, "missed_rescue": 6535, "avoided_harm": 562, "selected_both_correct": 5609, "selected_both_wrong": 4467}, {"budget": 0.72, "threshold": -0.5549414753913879, "operator": "ge", "actual_fraction": 0.6839179876924822, "metric": 0.5810906615999774, "rescued": 12503, "harmed": 1423, "missed_rescue": 6317, "avoided_harm": 547, "selected_both_correct": 5664, "selected_both_wrong": 4527}, {"budget": 0.73, "threshold": -0.5604677200317383, "operator": "ge", "actual_fraction": 0.6935314635737175, "metric": 0.5866205371068826, "rescued": 12724, "harmed": 1449, "missed_rescue": 6096, "avoided_harm": 521, "selected_both_correct": 5702, "selected_both_wrong": 4581}, {"budget": 0.74, "threshold": -0.5666090250015259, "operator": "ge", "actual_fraction": 0.7039673311970054, "metric": 0.5924623543090491, "rescued": 12953, "harmed": 1472, "missed_rescue": 5867, "avoided_harm": 498, "selected_both_correct": 5739, "selected_both_wrong": 4660}, {"budget": 0.75, "threshold": -0.574927568435669, "operator": "ge", "actual_fraction": 0.7186853075461532, "metric": 0.6001758216827836, "rescued": 13270, "harmed": 1517, "missed_rescue": 5550, "avoided_harm": 453, "selected_both_correct": 5813, "selected_both_wrong": 4743}, {"budget": 0.76, "threshold": -0.5812664031982422, "operator": "ge", "actual_fraction": 0.7283555000992542, "metric": 0.6051668888069648, "rescued": 13473, "harmed": 1544, "missed_rescue": 5347, "avoided_harm": 426, "selected_both_correct": 5862, "selected_both_wrong": 4805}, {"budget": 0.77, "threshold": -0.5883752107620239, "operator": "ge", "actual_fraction": 0.7390465927459376, "metric": 0.6112072143606613, "rescued": 13710, "harmed": 1568, "missed_rescue": 5110, "avoided_harm": 402, "selected_both_correct": 5909, "selected_both_wrong": 4874}, {"budget": 0.78, "threshold": -0.5972744226455688, "operator": "ge", "actual_fraction": 0.752176502282846, "metric": 0.618722173382866, "rescued": 14008, "harmed": 1601, "missed_rescue": 4812, "avoided_harm": 369, "selected_both_correct": 5959, "selected_both_wrong": 4956}, {"budget": 0.79, "threshold": -0.6058458089828491, "operator": "ge", "actual_fraction": 0.7635765533278507, "metric": 0.6246490655928311, "rescued": 14250, "harmed": 1634, "missed_rescue": 4570, "avoided_harm": 336, "selected_both_correct": 6010, "selected_both_wrong": 5032}, {"budget": 0.8, "threshold": -0.6140543818473816, "operator": "ge", "actual_fraction": 0.7745228709979298, "metric": 0.6308878994980575, "rescued": 14487, "harmed": 1651, "missed_rescue": 4333, "avoided_harm": 319, "selected_both_correct": 6062, "selected_both_wrong": 5112}, {"budget": 0.81, "threshold": -0.6230268478393555, "operator": "ge", "actual_fraction": 0.7864050137537929, "metric": 0.6366730000283584, "rescued": 14732, "harmed": 1692, "missed_rescue": 4088, "avoided_harm": 278, "selected_both_correct": 6105, "selected_both_wrong": 5202}, {"budget": 0.82, "threshold": -0.6330366134643555, "operator": "ge", "actual_fraction": 0.798287156509656, "metric": 0.6438476590193687, "rescued": 15012, "harmed": 1719, "missed_rescue": 3808, "avoided_harm": 251, "selected_both_correct": 6148, "selected_both_wrong": 5271}, {"budget": 0.83, "threshold": -0.6429333686828613, "operator": "ge", "actual_fraction": 0.8092334741797351, "metric": 0.6499730595808638, "rescued": 15248, "harmed": 1739, "missed_rescue": 3572, "avoided_harm": 231, "selected_both_correct": 6207, "selected_both_wrong": 5342}, {"budget": 0.84, "threshold": -0.6513019800186157, "operator": "ge", "actual_fraction": 0.8177126166236566, "metric": 0.6548223350253807, "rescued": 15437, "harmed": 1757, "missed_rescue": 3383, "avoided_harm": 213, "selected_both_correct": 6242, "selected_both_wrong": 5399}, {"budget": 0.85, "threshold": -0.6624053120613098, "operator": "ge", "actual_fraction": 0.8288858009811985, "metric": 0.6607492272353458, "rescued": 15678, "harmed": 1789, "missed_rescue": 3142, "avoided_harm": 181, "selected_both_correct": 6295, "selected_both_wrong": 5467}, {"budget": 0.86, "threshold": -0.6759263277053833, "operator": "ge", "actual_fraction": 0.8421291438618382, "metric": 0.6682925445934833, "rescued": 15980, "harmed": 1825, "missed_rescue": 2840, "avoided_harm": 145, "selected_both_correct": 6343, "selected_both_wrong": 5548}, {"budget": 0.87, "threshold": -0.6904275417327881, "operator": "ge", "actual_fraction": 0.8543799449848283, "metric": 0.6742194368034484, "rescued": 16226, "harmed": 1862, "missed_rescue": 2594, "avoided_harm": 108, "selected_both_correct": 6386, "selected_both_wrong": 5654}, {"budget": 0.88, "threshold": -0.7045450210571289, "operator": "ge", "actual_fraction": 0.8657516376939001, "metric": 0.6793806539432267, "rescued": 16441, "harmed": 1895, "missed_rescue": 2379, "avoided_harm": 75, "selected_both_correct": 6451, "selected_both_wrong": 5742}, {"budget": 0.89, "threshold": -0.7205488681793213, "operator": "ge", "actual_fraction": 0.8767830303717777, "metric": 0.6845985877548706, "rescued": 16636, "harmed": 1906, "missed_rescue": 2184, "avoided_harm": 64, "selected_both_correct": 6534, "selected_both_wrong": 5842}, {"budget": 0.9, "threshold": -0.7357243299484253, "operator": "ge", "actual_fraction": 0.8868785979638715, "metric": 0.6893911465275218, "rescued": 16811, "harmed": 1912, "missed_rescue": 2009, "avoided_harm": 58, "selected_both_correct": 6611, "selected_both_wrong": 5940}, {"budget": 0.91, "threshold": -0.7513164281845093, "operator": "ge", "actual_fraction": 0.8974562572668235, "metric": 0.6947508720188299, "rescued": 17004, "harmed": 1916, "missed_rescue": 1816, "avoided_harm": 54, "selected_both_correct": 6695, "selected_both_wrong": 6032}, {"budget": 0.92, "threshold": -0.7671180963516235, "operator": "ge", "actual_fraction": 0.9084025749369027, "metric": 0.7000255225023395, "rescued": 17196, "harmed": 1922, "missed_rescue": 1624, "avoided_harm": 48, "selected_both_correct": 6770, "selected_both_wrong": 6145}, {"budget": 0.93, "threshold": -0.7833515405654907, "operator": "ge", "actual_fraction": 0.9181011258259365, "metric": 0.704959872954655, "rescued": 17376, "harmed": 1928, "missed_rescue": 1444, "avoided_harm": 42, "selected_both_correct": 6846, "selected_both_wrong": 6225}, {"budget": 0.94, "threshold": -0.8013419508934021, "operator": "ge", "actual_fraction": 0.9289623684882171, "metric": 0.710546465133426, "rescued": 17577, "harmed": 1932, "missed_rescue": 1243, "avoided_harm": 38, "selected_both_correct": 6926, "selected_both_wrong": 6323}, {"budget": 0.95, "threshold": -0.8188987374305725, "operator": "ge", "actual_fraction": 0.9394549527833707, "metric": 0.7157360406091371, "rescued": 17763, "harmed": 1935, "missed_rescue": 1057, "avoided_harm": 35, "selected_both_correct": 7010, "selected_both_wrong": 6420}, {"budget": 0.96, "threshold": -0.8395025134086609, "operator": "ge", "actual_fraction": 0.9509684371721068, "metric": 0.7211808411082438, "rescued": 17963, "harmed": 1943, "missed_rescue": 857, "avoided_harm": 27, "selected_both_correct": 7084, "selected_both_wrong": 6544}, {"budget": 0.97, "threshold": -0.8628664612770081, "operator": "ge", "actual_fraction": 0.9640699883730822, "metric": 0.727476391685336, "rescued": 18191, "harmed": 1949, "missed_rescue": 629, "avoided_harm": 21, "selected_both_correct": 7177, "selected_both_wrong": 6679}, {"budget": 0.98, "threshold": -0.8857419490814209, "operator": "ge", "actual_fraction": 0.9760372061367439, "metric": 0.7329779088563083, "rescued": 18395, "harmed": 1959, "missed_rescue": 425, "avoided_harm": 11, "selected_both_correct": 7278, "selected_both_wrong": 6786}, {"budget": 0.99, "threshold": -0.9162307977676392, "operator": "ge", "actual_fraction": 0.9873805405098829, "metric": 0.7387062927147435, "rescued": 18604, "harmed": 1966, "missed_rescue": 216, "avoided_harm": 4, "selected_both_correct": 7360, "selected_both_wrong": 6888}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.7447182599325072, "rescued": 18820, "harmed": 1970, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 7441, "selected_both_wrong": 7032}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.8140698573336145, "actual_fraction": 0.7759786351816713, "metric_std": 0.002855871162158936, "actual_fraction_std": 0.06948144675241769, "per_seed": [{"seed": 2026, "budget": 0.89, "threshold": -0.005598927848041058, "operator": "ge", "alpha": 0.0, "epoch": 3, "metric": 0.8122144915313796, "actual_fraction": 0.889943074003795, "rescue": 4618, "harm": 1037, "missed_rescue": 8}, {"seed": 2027, "budget": 0.76, "threshold": 0.007067253813147545, "operator": "ge", "alpha": 0.0, "epoch": 8, "metric": 0.8137606296999086, "actual_fraction": 0.7599971888396937, "rescue": 4596, "harm": 993, "missed_rescue": 30}, {"seed": 2028, "budget": 0.76, "threshold": 0.019726920872926712, "operator": "ge", "alpha": 0.0, "epoch": 1, "metric": 0.8131983976386252, "actual_fraction": 0.7599971888396937, "rescue": 4625, "harm": 1030, "missed_rescue": 1}, {"seed": 2029, "budget": 0.7, "threshold": 0.006556786596775055, "operator": "ge", "alpha": 0.0, "epoch": 6, "metric": 0.8190315552744394, "actual_fraction": 0.6999789162977019, "rescue": 4601, "harm": 923, "missed_rescue": 25}, {"seed": 2030, "budget": 0.77, "threshold": 0.0009468437638133764, "operator": "ge", "alpha": 0.0, "epoch": 10, "metric": 0.812144212523719, "actual_fraction": 0.769976807927472, "rescue": 4604, "harm": 1024, "missed_rescue": 22}], "rescued": 4608.8, "harmed": 1001.4, "missed_rescue": 17.2, "avoided_harm": 131.60000000000002}, "validation_curve": [{"budget": 0.0, "metric": 0.5605453650994447, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.5700892543397287, "metric_std": 0.0002454740206138481, "actual_fraction": 0.009979619087778481, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.5789162977018764, "metric_std": 0.0005273267687765411, "actual_fraction": 0.019959238175556962, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.5874903366364467, "metric_std": 0.0011140940759317099, "actual_fraction": 0.029924801461803357, "actual_fraction_std": 3.142972770398104e-05}, {"budget": 0.04, "metric": 0.5961065429756132, "metric_std": 0.0012795798338627537, "actual_fraction": 0.03998875535877434, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.604090238245836, "metric_std": 0.0014671321482224078, "actual_fraction": 0.04995431864502072, "actual_fraction_std": 3.1429727703976376e-05}, {"budget": 0.06, "metric": 0.6118349848900133, "metric_std": 0.001629654870230064, "actual_fraction": 0.0599339377327992, "actual_fraction_std": 3.142972770398259e-05}, {"budget": 0.07, "metric": 0.6193829503127416, "metric_std": 0.0016387220110717319, "actual_fraction": 0.06999789162977019, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.6263124604680581, "metric_std": 0.00227697477411224, "actual_fraction": 0.07997751071754867, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.6331576358141823, "metric_std": 0.0026461702975562667, "actual_fraction": 0.08994307400379507, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.1, "metric": 0.6396935835266007, "metric_std": 0.0027415119352490708, "actual_fraction": 0.09992269309157353, "actual_fraction_std": 3.142972770397328e-05}, {"budget": 0.11, "metric": 0.6455970201700751, "metric_std": 0.0026260285141136495, "actual_fraction": 0.10998664698854452, "actual_fraction_std": 0.0}, {"budget": 0.12, "metric": 0.6513036755921007, "metric_std": 0.0028786954121912786, "actual_fraction": 0.11996626607632302, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.13, "metric": 0.6565464895635673, "metric_std": 0.003220205297458369, "actual_fraction": 0.1299458851641015, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.662295312390189, "metric_std": 0.003464191944510799, "actual_fraction": 0.13998172745800833, "actual_fraction_std": 3.1429727703979486e-05}, {"budget": 0.15, "metric": 0.6671445639187574, "metric_std": 0.003776278840773885, "actual_fraction": 0.14997540234731885, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.6721203176611147, "metric_std": 0.003798513725178707, "actual_fraction": 0.15995502143509735, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.676772787968234, "metric_std": 0.00423293211047109, "actual_fraction": 0.16992058472134375, "actual_fraction_std": 3.1429727703979486e-05}, {"budget": 0.18, "metric": 0.6816923185044625, "metric_std": 0.004530516482744597, "actual_fraction": 0.1799845386183147, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.6862604540023896, "metric_std": 0.004321558986723308, "actual_fraction": 0.18996415770609318, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.6906599198819313, "metric_std": 0.004299846358969907, "actual_fraction": 0.19994377679387168, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.6949328835476842, "metric_std": 0.004383404865827688, "actual_fraction": 0.2099936748893106, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.6992761262210976, "metric_std": 0.004826791716548652, "actual_fraction": 0.21997329397708903, "actual_fraction_std": 0.0}, {"budget": 0.23, "metric": 0.7033523086654017, "metric_std": 0.0053792141944278195, "actual_fraction": 0.22995291306486756, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.24, "metric": 0.7074144353081734, "metric_std": 0.005462672935410186, "actual_fraction": 0.23993253215264604, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.25, "metric": 0.7111392227141753, "metric_std": 0.005648263989706566, "actual_fraction": 0.24998243024808492, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.26, "metric": 0.7146250614941316, "metric_std": 0.005459326512608638, "actual_fraction": 0.25996204933586337, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.7184763511139223, "metric_std": 0.0050526158892283115, "actual_fraction": 0.26994166842364187, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.7226509241689507, "metric_std": 0.0047837759508080075, "actual_fraction": 0.27999156651908075, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.7266006043994659, "metric_std": 0.004746149056035001, "actual_fraction": 0.28997118560685925, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.7304237824161922, "metric_std": 0.0042151002971300994, "actual_fraction": 0.2999508046946377, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.733628505165507, "metric_std": 0.003937631888654014, "actual_fraction": 0.3099304237824162, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.7368613395178858, "metric_std": 0.003838224804505319, "actual_fraction": 0.3199803218778551, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.7397849462365591, "metric_std": 0.0038353927359114012, "actual_fraction": 0.3299599409656336, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.7428209993674889, "metric_std": 0.0038111703883802975, "actual_fraction": 0.339939560053412, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.7457727176892262, "metric_std": 0.003339029541498041, "actual_fraction": 0.34998945814885096, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.7485417105910465, "metric_std": 0.002541244603522975, "actual_fraction": 0.3599690772366294, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.7512685360882704, "metric_std": 0.0025116266095356714, "actual_fraction": 0.3699486963244079, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.7543327008222643, "metric_std": 0.002309924624923089, "actual_fraction": 0.37999859441984685, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.757171972731745, "metric_std": 0.0022546311836152333, "actual_fraction": 0.3899782135076253, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.7600253004427577, "metric_std": 0.00225145248034875, "actual_fraction": 0.39995783259540374, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.762625623726193, "metric_std": 0.0016298064026001474, "actual_fraction": 0.4099374516831823, "actual_fraction_std": 0.0}, {"budget": 0.42, "metric": 0.7649448309789866, "metric_std": 0.001391985703025056, "actual_fraction": 0.4199873497786212, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.7677278796823389, "metric_std": 0.0011749422656054463, "actual_fraction": 0.42995291306486755, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.44, "metric": 0.770145477545857, "metric_std": 0.0011660810195057976, "actual_fraction": 0.43994658795417807, "actual_fraction_std": 0.0}, {"budget": 0.45, "metric": 0.7725490196078431, "metric_std": 0.0017046100508039653, "actual_fraction": 0.44999648604961695, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.46, "metric": 0.7753742357157917, "metric_std": 0.002310352230373846, "actual_fraction": 0.4599761051373955, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.47, "metric": 0.7779605031976947, "metric_std": 0.002521048186635445, "actual_fraction": 0.4699557242251739, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.48, "metric": 0.7799283154121863, "metric_std": 0.0025729220531210276, "actual_fraction": 0.47993534331295234, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.7821772436573196, "metric_std": 0.002546874779147548, "actual_fraction": 0.4899852414083913, "actual_fraction_std": 0.0}, {"budget": 0.5, "metric": 0.7848478459484152, "metric_std": 0.0032073725286989336, "actual_fraction": 0.4999508046946377, "actual_fraction_std": 3.142972770396707e-05}, {"budget": 0.51, "metric": 0.7867875465598425, "metric_std": 0.003552880146029157, "actual_fraction": 0.5099444795839483, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.7888256377819944, "metric_std": 0.003467754536151893, "actual_fraction": 0.5199943776793872, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.7910323986225315, "metric_std": 0.003906526772979988, "actual_fraction": 0.5299739967671656, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.7927050390048492, "metric_std": 0.004455544978118954, "actual_fraction": 0.5399536158549442, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.7941106191580575, "metric_std": 0.0048005243841016815, "actual_fraction": 0.5499332349427226, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.795389697097477, "metric_std": 0.004891489615563991, "actual_fraction": 0.5599831330381615, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.7972872303043081, "metric_std": 0.0050976751159587065, "actual_fraction": 0.5699627521259399, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.7991144845034788, "metric_std": 0.005372920913838329, "actual_fraction": 0.5799283154121865, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.59, "metric": 0.8009557945041816, "metric_std": 0.005685741116542411, "actual_fraction": 0.5899922693091574, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.8024597652681142, "metric_std": 0.005675524846194789, "actual_fraction": 0.5999718883969358, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.8033031133600395, "metric_std": 0.005414540319730133, "actual_fraction": 0.6099515074847143, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.8047227493147796, "metric_std": 0.0054760381765678046, "actual_fraction": 0.6199311265724928, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.8056925996204933, "metric_std": 0.005159503563803555, "actual_fraction": 0.6299810246679317, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.8065500035139503, "metric_std": 0.004743338435315904, "actual_fraction": 0.6399606437557102, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.8076182444303888, "metric_std": 0.004679125451889847, "actual_fraction": 0.6499402628434886, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.8084334809192493, "metric_std": 0.004584702115971454, "actual_fraction": 0.6599901609389276, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.8091643825989177, "metric_std": 0.004555847174125413, "actual_fraction": 0.669969780026706, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.8096703914540726, "metric_std": 0.00474796986598617, "actual_fraction": 0.6799493991144846, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.8105137395459977, "metric_std": 0.004906562025778376, "actual_fraction": 0.6899992972099234, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.8112165296226017, "metric_std": 0.004973301313848519, "actual_fraction": 0.6999789162977019, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.8116382036685643, "metric_std": 0.00449631887415267, "actual_fraction": 0.7099585353854804, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.8123972169512965, "metric_std": 0.003919779757241607, "actual_fraction": 0.7199381544732588, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.8127205003865345, "metric_std": 0.0035778158607014227, "actual_fraction": 0.7299880525686977, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.8130859512263686, "metric_std": 0.0032845945165275482, "actual_fraction": 0.7399676716564763, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.8134092346616064, "metric_std": 0.003128402749884717, "actual_fraction": 0.7499472907442546, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.813676294890716, "metric_std": 0.002920767149005142, "actual_fraction": 0.7599971888396937, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.8137044064937802, "metric_std": 0.002670787229449218, "actual_fraction": 0.769976807927472, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.8136060158830556, "metric_std": 0.0026578104673096437, "actual_fraction": 0.7799564270152506, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.8134232904631385, "metric_std": 0.0027134432709500175, "actual_fraction": 0.7899360461030291, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.8133670672570104, "metric_std": 0.0026989335071230195, "actual_fraction": 0.7999859441984679, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.8133530114554782, "metric_std": 0.0027079770219034364, "actual_fraction": 0.8099655632862464, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.8132967882493498, "metric_std": 0.002628472468587764, "actual_fraction": 0.8199451823740249, "actual_fraction_std": 0.0}, {"budget": 0.83, "metric": 0.8132967882493499, "metric_std": 0.002645796966530483, "actual_fraction": 0.8299950804694639, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.84, "metric": 0.8132686766462858, "metric_std": 0.002656230401147516, "actual_fraction": 0.8399746995572424, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.8131843418370932, "metric_std": 0.002555006831217253, "actual_fraction": 0.8499543186450207, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.8130578396233045, "metric_std": 0.002480759377991837, "actual_fraction": 0.8599339377327994, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.87, "metric": 0.813001616417176, "metric_std": 0.0023731005444351423, "actual_fraction": 0.8699838358282379, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.8127907793941948, "metric_std": 0.002112699656826451, "actual_fraction": 0.8799634549160166, "actual_fraction_std": 0.0}, {"budget": 0.89, "metric": 0.8126642771804061, "metric_std": 0.001971323033008991, "actual_fraction": 0.889943074003795, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.8125377749666175, "metric_std": 0.0019333753730043325, "actual_fraction": 0.8999929720992339, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.91, "metric": 0.8122004357298476, "metric_std": 0.0017181744181335367, "actual_fraction": 0.9099725911870123, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.8119193196992059, "metric_std": 0.0017066371084055337, "actual_fraction": 0.919952210274791, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.93, "metric": 0.8116663152716284, "metric_std": 0.0017196111391958166, "actual_fraction": 0.9299318293625692, "actual_fraction_std": 0.0}, {"budget": 0.94, "metric": 0.8114554782486472, "metric_std": 0.0015788550952444992, "actual_fraction": 0.9399817274580083, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.811075971607281, "metric_std": 0.0014745199054559384, "actual_fraction": 0.9499613465457868, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.8106121301567223, "metric_std": 0.0013558567421302489, "actual_fraction": 0.9599409656335652, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.8098671726755218, "metric_std": 0.0010871687739322131, "actual_fraction": 0.9699908637290042, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.98, "metric": 0.8089535455759365, "metric_std": 0.0007787979578154683, "actual_fraction": 0.9799704828167826, "actual_fraction_std": 0.0}, {"budget": 0.99, "metric": 0.8077728582472414, "metric_std": 0.0005520370197949574, "actual_fraction": 0.9899501019045612, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 1.0, "metric": 0.806029938857263, "metric_std": 1.2412670766236366e-16, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.7512179905283157, "actual_fraction": 0.8846666477611093, "metric_std": 0.0008064444629924049, "actual_fraction_std": 0.0370614917629929, "per_seed": [{"seed": 2026, "budget": 0.89, "threshold": -0.005598927848041058, "operator": "ge", "alpha": 0.0, "epoch": 3, "metric": 0.7510988855173979, "actual_fraction": 0.9439639281966934, "rescue": 18744, "harm": 1669, "missed_rescue": 76}, {"seed": 2027, "budget": 0.76, "threshold": 0.007067253813147545, "operator": "ge", "alpha": 0.0, "epoch": 8, "metric": 0.7514108272126592, "actual_fraction": 0.8772651220826362, "rescue": 18710, "harm": 1624, "missed_rescue": 110}, {"seed": 2028, "budget": 0.76, "threshold": 0.019726920872926712, "operator": "ge", "alpha": 0.0, "epoch": 1, "metric": 0.749936193744151, "actual_fraction": 0.8775487054419646, "rescue": 18745, "harm": 1711, "missed_rescue": 75}, {"seed": 2029, "budget": 0.7, "threshold": 0.006556786596775055, "operator": "ge", "alpha": 0.0, "epoch": 6, "metric": 0.7521197856109804, "actual_fraction": 0.841420185463517, "rescue": 18660, "harm": 1549, "missed_rescue": 160}, {"seed": 2030, "budget": 0.77, "threshold": 0.0009468437638133764, "operator": "ge", "alpha": 0.0, "epoch": 10, "metric": 0.7515242605563905, "actual_fraction": 0.8831352976207356, "rescue": 18745, "harm": 1655, "missed_rescue": 75}], "rescued": 18720.8, "harmed": 1641.6, "missed_rescue": 99.2, "avoided_harm": 328.4000000000001}, "test_curve": [{"budget": 0.0, "metric": 0.26688029946402747, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.2802767773587046, "metric_std": 0.002230559339671242, "actual_fraction": 0.01404304795394606, "actual_fraction_std": 0.0022437519686261224}, {"budget": 0.02, "metric": 0.2952159487281286, "metric_std": 0.002786082029009796, "actual_fraction": 0.029935059410713775, "actual_fraction_std": 0.002989928977317969}, {"budget": 0.03, "metric": 0.3094064600289255, "metric_std": 0.00461329204344182, "actual_fraction": 0.04528259081757083, "actual_fraction_std": 0.004854761586334819}, {"budget": 0.04, "metric": 0.3232566712985282, "metric_std": 0.00555005309813909, "actual_fraction": 0.060856988911890654, "actual_fraction_std": 0.006280715228252985}, {"budget": 0.05, "metric": 0.3370898675665712, "metric_std": 0.0038657623383648505, "actual_fraction": 0.07666959702804639, "actual_fraction_std": 0.004538556537725048}, {"budget": 0.06, "metric": 0.3507756004877634, "metric_std": 0.004427107734080273, "actual_fraction": 0.09272608683322461, "actual_fraction_std": 0.0051712341165676395}, {"budget": 0.07, "metric": 0.3641210333777614, "metric_std": 0.004474510489366405, "actual_fraction": 0.1086237699571789, "actual_fraction_std": 0.005410262162313685}, {"budget": 0.08, "metric": 0.37727930125060266, "metric_std": 0.003636617981256096, "actual_fraction": 0.12480503644046168, "actual_fraction_std": 0.0038783070067903783}, {"budget": 0.09, "metric": 0.3900972690922497, "metric_std": 0.004884924008915436, "actual_fraction": 0.14087286958001305, "actual_fraction_std": 0.004969170282377531}, {"budget": 0.1, "metric": 0.4020928451918442, "metric_std": 0.004635308588109036, "actual_fraction": 0.15607293764001928, "actual_fraction_std": 0.00486936658462405}, {"budget": 0.11, "metric": 0.4139636446133341, "metric_std": 0.00514137490753296, "actual_fraction": 0.1713297223718912, "actual_fraction_std": 0.005834847355979504}, {"budget": 0.12, "metric": 0.42570966735671956, "metric_std": 0.005646670948258889, "actual_fraction": 0.18679635878966622, "actual_fraction_std": 0.006504730973939501}, {"budget": 0.13, "metric": 0.4366786716955449, "metric_std": 0.005479018380319435, "actual_fraction": 0.20149732013725438, "actual_fraction_std": 0.00682666493075311}, {"budget": 0.14, "metric": 0.4467572242860789, "metric_std": 0.006213420771134132, "actual_fraction": 0.2152851430678048, "actual_fraction_std": 0.007680335820725634}, {"budget": 0.15, "metric": 0.4575674219436804, "metric_std": 0.005791644268695864, "actual_fraction": 0.2303434194481468, "actual_fraction_std": 0.0074400135908241315}, {"budget": 0.16, "metric": 0.46828120125911016, "metric_std": 0.00576354836242943, "actual_fraction": 0.24516915747383944, "actual_fraction_std": 0.006990821603072464}, {"budget": 0.17, "metric": 0.47771318379037514, "metric_std": 0.006138666038410859, "actual_fraction": 0.25862235204038225, "actual_fraction_std": 0.007656839977591084}, {"budget": 0.18, "metric": 0.4865326262654907, "metric_std": 0.005505484445785032, "actual_fraction": 0.2712304681961263, "actual_fraction_std": 0.006475599775491907}, {"budget": 0.19, "metric": 0.4952613220656212, "metric_std": 0.004750871942656999, "actual_fraction": 0.2841561977143181, "actual_fraction_std": 0.005874845271314996}, {"budget": 0.2, "metric": 0.503575986161132, "metric_std": 0.0053645847175930374, "actual_fraction": 0.296633865524771, "actual_fraction_std": 0.006396153360775978}, {"budget": 0.21, "metric": 0.511470946884837, "metric_std": 0.005609971275773111, "actual_fraction": 0.3088619799790148, "actual_fraction_std": 0.006403573476558177}, {"budget": 0.22, "metric": 0.519320534271049, "metric_std": 0.005515692693907304, "actual_fraction": 0.32109576610044516, "actual_fraction_std": 0.006734445047292585}, {"budget": 0.23, "metric": 0.5265689249354848, "metric_std": 0.005374731402044185, "actual_fraction": 0.3320477554377109, "actual_fraction_std": 0.006927796947328311}, {"budget": 0.24, "metric": 0.5343901539857641, "metric_std": 0.0059130496601453525, "actual_fraction": 0.34443467657317867, "actual_fraction_std": 0.007760511351450168}, {"budget": 0.25, "metric": 0.5413946629611776, "metric_std": 0.006147299738929747, "actual_fraction": 0.35576099594475796, "actual_fraction_std": 0.007948594194120393}, {"budget": 0.26, "metric": 0.5483311119303519, "metric_std": 0.0065426804933728135, "actual_fraction": 0.3669285086351133, "actual_fraction_std": 0.008390785466797586}, {"budget": 0.27, "metric": 0.5549386042027054, "metric_std": 0.006244419286235234, "actual_fraction": 0.3781130363270283, "actual_fraction_std": 0.008073827911743162}, {"budget": 0.28, "metric": 0.5616538581516036, "metric_std": 0.005813064602828289, "actual_fraction": 0.38921248901114486, "actual_fraction_std": 0.007384431177310305}, {"budget": 0.29, "metric": 0.5674276153475314, "metric_std": 0.005498388093543556, "actual_fraction": 0.39938745994385044, "actual_fraction_std": 0.007396270458252986}, {"budget": 0.3, "metric": 0.5734962992371607, "metric_std": 0.005568439262153627, "actual_fraction": 0.4099310892436832, "actual_fraction_std": 0.007536320478398078}, {"budget": 0.31, "metric": 0.5797805064798798, "metric_std": 0.005203093937504007, "actual_fraction": 0.4208377052434564, "actual_fraction_std": 0.007351211477295298}, {"budget": 0.32, "metric": 0.5858435187023225, "metric_std": 0.005639053495864899, "actual_fraction": 0.4315004395542069, "actual_fraction_std": 0.0070679703433865326}, {"budget": 0.33, "metric": 0.5915095142217055, "metric_std": 0.005848510615373134, "actual_fraction": 0.44165272381816634, "actual_fraction_std": 0.007503794666879848}, {"budget": 0.34, "metric": 0.596772821370842, "metric_std": 0.006214197295046492, "actual_fraction": 0.45130590136970766, "actual_fraction_std": 0.008479583456740448}, {"budget": 0.35, "metric": 0.6027734452542325, "metric_std": 0.006232469368484458, "actual_fraction": 0.46174744066018203, "actual_fraction_std": 0.008353289504381108}, {"budget": 0.36, "metric": 0.6088988458157276, "metric_std": 0.006251408467126378, "actual_fraction": 0.4734651050676346, "actual_fraction_std": 0.008162088740353249}, {"budget": 0.37, "metric": 0.6144797663273118, "metric_std": 0.006346555033529579, "actual_fraction": 0.48378753934719115, "actual_fraction_std": 0.008269346190793716}, {"budget": 0.38, "metric": 0.6199018801576723, "metric_std": 0.0062873570951490374, "actual_fraction": 0.49464311034228514, "actual_fraction_std": 0.008522578779452297}, {"budget": 0.39, "metric": 0.6254998156708165, "metric_std": 0.005692648992956946, "actual_fraction": 0.5057822646967075, "actual_fraction_std": 0.00759782747347306}, {"budget": 0.4, "metric": 0.6307971528230724, "metric_std": 0.005569486210319508, "actual_fraction": 0.5159799222981596, "actual_fraction_std": 0.007439673097776658}, {"budget": 0.41, "metric": 0.6362249383206193, "metric_std": 0.00561432029409292, "actual_fraction": 0.5265575816011117, "actual_fraction_std": 0.007783652931400327}, {"budget": 0.42, "metric": 0.6413521254572782, "metric_std": 0.004936020703786802, "actual_fraction": 0.5371012109009443, "actual_fraction_std": 0.0063214095792167874}, {"budget": 0.43, "metric": 0.6463261775798996, "metric_std": 0.005015522668932812, "actual_fraction": 0.5478717068882398, "actual_fraction_std": 0.006575881702693823}, {"budget": 0.44, "metric": 0.651050676346312, "metric_std": 0.005408530199269447, "actual_fraction": 0.558290559509968, "actual_fraction_std": 0.007586027130345651}, {"budget": 0.45, "metric": 0.6560644301392393, "metric_std": 0.004943607122580101, "actual_fraction": 0.5692538921816068, "actual_fraction_std": 0.007230224556446873}, {"budget": 0.46, "metric": 0.6608626605790773, "metric_std": 0.004181628713579798, "actual_fraction": 0.5794969231205512, "actual_fraction_std": 0.006424818676716126}, {"budget": 0.47, "metric": 0.6659161160423107, "metric_std": 0.004202462361442581, "actual_fraction": 0.5905056291296826, "actual_fraction_std": 0.006265523881688423}, {"budget": 0.48, "metric": 0.6708221081586933, "metric_std": 0.004225696562065504, "actual_fraction": 0.601673141820038, "actual_fraction_std": 0.006499363114836915}, {"budget": 0.49, "metric": 0.6756657119360236, "metric_std": 0.004622582731301273, "actual_fraction": 0.612897371182259, "actual_fraction_std": 0.006640896231463029}, {"budget": 0.5, "metric": 0.6805660323852197, "metric_std": 0.0046559689759777464, "actual_fraction": 0.6236905538383007, "actual_fraction_std": 0.007118292552924193}, {"budget": 0.51, "metric": 0.6858860562062218, "metric_std": 0.004572143012434143, "actual_fraction": 0.6357995632816266, "actual_fraction_std": 0.007372114736859272}, {"budget": 0.52, "metric": 0.6907296599835522, "metric_std": 0.004316544893655242, "actual_fraction": 0.6471882709922582, "actual_fraction_std": 0.006959137949113234}, {"budget": 0.53, "metric": 0.6952159487281286, "metric_std": 0.0037316596222665686, "actual_fraction": 0.6575107052718147, "actual_fraction_std": 0.006344584330149168}, {"budget": 0.54, "metric": 0.7001673141820038, "metric_std": 0.0036668435920596517, "actual_fraction": 0.6696650880526331, "actual_fraction_std": 0.006773327837270835}, {"budget": 0.55, "metric": 0.7046989762640729, "metric_std": 0.003682130904846671, "actual_fraction": 0.6808609590789213, "actual_fraction_std": 0.006795428933299371}, {"budget": 0.56, "metric": 0.709122876669597, "metric_std": 0.0037348800480656815, "actual_fraction": 0.6923744434676574, "actual_fraction_std": 0.007062632035130183}, {"budget": 0.57, "metric": 0.7138814054391288, "metric_std": 0.0037908317806251987, "actual_fraction": 0.7039559878626322, "actual_fraction_std": 0.006808971692679445}, {"budget": 0.58, "metric": 0.718055752488444, "metric_std": 0.003858891236146277, "actual_fraction": 0.7149703655389501, "actual_fraction_std": 0.006552647474270297}, {"budget": 0.59, "metric": 0.7219578595128039, "metric_std": 0.003688143049947665, "actual_fraction": 0.7255933981793949, "actual_fraction_std": 0.00625237320951913}, {"budget": 0.6, "metric": 0.725973399880895, "metric_std": 0.0038702635710776896, "actual_fraction": 0.7369367325525338, "actual_fraction_std": 0.007435634634587727}, {"budget": 0.61, "metric": 0.7298981935740011, "metric_std": 0.0038546792454942614, "actual_fraction": 0.7481893202506876, "actual_fraction_std": 0.008453795455123122}, {"budget": 0.62, "metric": 0.7331537305390919, "metric_std": 0.003917886540424116, "actual_fraction": 0.7587556362192667, "actual_fraction_std": 0.00891198776434956}, {"budget": 0.63, "metric": 0.7358534441198991, "metric_std": 0.003889467506394551, "actual_fraction": 0.7683917987692482, "actual_fraction_std": 0.009126680054534993}, {"budget": 0.64, "metric": 0.7387800243881688, "metric_std": 0.003692501450988525, "actual_fraction": 0.7790488614128123, "actual_fraction_std": 0.008363848778768529}, {"budget": 0.65, "metric": 0.7412131696112072, "metric_std": 0.0035332369875507234, "actual_fraction": 0.7886850239627938, "actual_fraction_std": 0.008667394957299644}, {"budget": 0.66, "metric": 0.7433797464764768, "metric_std": 0.003481694621974773, "actual_fraction": 0.7981680514987379, "actual_fraction_std": 0.008451150474521316}, {"budget": 0.67, "metric": 0.7449621416215296, "metric_std": 0.0030240279545186103, "actual_fraction": 0.8069988373082267, "actual_fraction_std": 0.008265722828618018}, {"budget": 0.68, "metric": 0.7466749851118737, "metric_std": 0.0026318308081997592, "actual_fraction": 0.8159487281286335, "actual_fraction_std": 0.007662037181383831}, {"budget": 0.69, "metric": 0.747934095227292, "metric_std": 0.002340516391385588, "actual_fraction": 0.8244619005756743, "actual_fraction_std": 0.007890168707115274}, {"budget": 0.7, "metric": 0.7490570853302329, "metric_std": 0.0019916558204824616, "actual_fraction": 0.8330885063664463, "actual_fraction_std": 0.008019956002692575}, {"budget": 0.71, "metric": 0.7498170887332332, "metric_std": 0.0016329329688048845, "actual_fraction": 0.8423446672149278, "actual_fraction_std": 0.008737603217008288}, {"budget": 0.72, "metric": 0.7503785837847035, "metric_std": 0.00127338086046386, "actual_fraction": 0.8503417179479907, "actual_fraction_std": 0.008097648217681998}, {"budget": 0.73, "metric": 0.7507699288205769, "metric_std": 0.0010769822474425463, "actual_fraction": 0.8576241386155461, "actual_fraction_std": 0.0065231446769578524}, {"budget": 0.74, "metric": 0.7510364971783456, "metric_std": 0.0009586208188481664, "actual_fraction": 0.8647023792643846, "actual_fraction_std": 0.005385807216681773}, {"budget": 0.75, "metric": 0.7511669455236367, "metric_std": 0.0010276913828231229, "actual_fraction": 0.8712134531945666, "actual_fraction_std": 0.004629736101990483}, {"budget": 0.76, "metric": 0.7512917222017412, "metric_std": 0.0009974719656834412, "actual_fraction": 0.8779967671497036, "actual_fraction_std": 0.0036054364957235846}, {"budget": 0.77, "metric": 0.7513370955392338, "metric_std": 0.0010589095291365037, "actual_fraction": 0.8851714261407141, "actual_fraction_std": 0.0025989606510334906}, {"budget": 0.78, "metric": 0.75135978220798, "metric_std": 0.001071892581538305, "actual_fraction": 0.8929869835238067, "actual_fraction_std": 0.0023445501841379774}, {"budget": 0.79, "metric": 0.7513938122110995, "metric_std": 0.0011100758221065951, "actual_fraction": 0.8997929841476902, "actual_fraction_std": 0.0031038843179445256}, {"budget": 0.8, "metric": 0.7513767972095398, "metric_std": 0.00110386439050134, "actual_fraction": 0.9058503247029464, "actual_fraction_std": 0.0033940082920629697}, {"budget": 0.81, "metric": 0.75135978220798, "metric_std": 0.0011145583661320163, "actual_fraction": 0.9111930351926947, "actual_fraction_std": 0.0034434423962434874}, {"budget": 0.82, "metric": 0.7513314238720472, "metric_std": 0.0011027710582737704, "actual_fraction": 0.9162011173184357, "actual_fraction_std": 0.003120383564303812}, {"budget": 0.83, "metric": 0.7513030655361144, "metric_std": 0.0010884578051114013, "actual_fraction": 0.9215438278081842, "actual_fraction_std": 0.0027871786731423686}, {"budget": 0.84, "metric": 0.7513257522048606, "metric_std": 0.0011148469436153, "actual_fraction": 0.926291013243343, "actual_fraction_std": 0.0036899306170216817}, {"budget": 0.85, "metric": 0.7512973938689279, "metric_std": 0.0010915196809073732, "actual_fraction": 0.9308737203300911, "actual_fraction_std": 0.004904008270971833}, {"budget": 0.86, "metric": 0.751229333862689, "metric_std": 0.0010505222968241698, "actual_fraction": 0.9359555341292574, "actual_fraction_std": 0.00649878153683167}, {"budget": 0.87, "metric": 0.7512009755267561, "metric_std": 0.0010011336068666535, "actual_fraction": 0.9403907778691547, "actual_fraction_std": 0.007386570833915809}, {"budget": 0.88, "metric": 0.7511045571845845, "metric_std": 0.001020387939606713, "actual_fraction": 0.9447069165981341, "actual_fraction_std": 0.007641805978327978}, {"budget": 0.89, "metric": 0.7510875421830248, "metric_std": 0.001090524592442763, "actual_fraction": 0.9486373819584266, "actual_fraction_std": 0.008082931674436593}, {"budget": 0.9, "metric": 0.7510478405127188, "metric_std": 0.0010717050008040552, "actual_fraction": 0.9525678473187194, "actual_fraction_std": 0.00812377483295943}, {"budget": 0.91, "metric": 0.7509797805064798, "metric_std": 0.0010634186917564447, "actual_fraction": 0.9567592093695941, "actual_fraction_std": 0.007920874628421481}, {"budget": 0.92, "metric": 0.7509741088392934, "metric_std": 0.001074365577812494, "actual_fraction": 0.9608087797408048, "actual_fraction_std": 0.007693737267003936}, {"budget": 0.93, "metric": 0.7509344071689874, "metric_std": 0.0011179804139944995, "actual_fraction": 0.9646768567620452, "actual_fraction_std": 0.00776025746218664}, {"budget": 0.94, "metric": 0.750860675495562, "metric_std": 0.0011147748062479157, "actual_fraction": 0.9683861271020617, "actual_fraction_std": 0.0070632013432873975}, {"budget": 0.95, "metric": 0.7505941071377932, "metric_std": 0.000980394881267413, "actual_fraction": 0.9724583841420186, "actual_fraction_std": 0.006344457575864208}, {"budget": 0.96, "metric": 0.7503048521112781, "metric_std": 0.0009028022221439949, "actual_fraction": 0.9763831778351246, "actual_fraction_std": 0.004906459269039596}, {"budget": 0.97, "metric": 0.7499758954144571, "metric_std": 0.0009033810414714414, "actual_fraction": 0.9808751382468877, "actual_fraction_std": 0.0036165829543790503}, {"budget": 0.98, "metric": 0.7495051470379719, "metric_std": 0.0008709287735293698, "actual_fraction": 0.985140231971188, "actual_fraction_std": 0.0028757505998935062}, {"budget": 0.99, "metric": 0.7481609619147549, "metric_std": 0.0008242476079554807, "actual_fraction": 0.990806227490571, "actual_fraction_std": 0.002033353990092406}, {"budget": 1.0, "metric": 0.7447182599325072, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 6843, "rescuable": 4626, "harmful": 1133, "both_wrong": 1627, "unit": "sample", "total": 14229}, "test": {"both_correct": 7441, "rescuable": 18820, "harmful": 1970, "both_wrong": 7032, "unit": "sample", "total": 35263}}, "small": {"validation": 0.5605453650994447, "test": 0.26688029946402747}, "large": {"validation": 0.8060299388572634, "test": 0.7447182599325072}, "learned_minus_confidence_test": 0.0064997305958085505, "learned_minus_small_test": 0.48433769106428826, "learned_minus_large_test": 0.0064997305958085505, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_groundingdino_Qwen3.8"}, {"key": ["grefcoco", "instancevg", "MiniCPM"], "expert_display": "InstanceVG", "run": "outputs/full_budget_20260913/runs/full_instancevg_MiniCPM_s{2026..2030}", "validation_rows": 14229, "test_rows": 35263, "confidence_rule": {"column": "max_score", "columns": ["detections_above_07", "max_score", "mean_score", "query_count", "detections_above_07_valid", "max_score_valid", "mean_score_valid", "query_count_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 65, "validation_area_0_25": 0.7465176751704266, "validation_point": {"budget": 0.65, "threshold": -0.9877713322639465, "operator": "ge", "actual_fraction": 0.6499402628434886, "metric": 0.7716635041113219, "rescued": 1117, "harmed": 813, "missed_rescue": 252, "avoided_harm": 2108, "selected_both_correct": 5984, "selected_both_wrong": 1334}, "test_point": {"budget": 0.65, "threshold": -0.9877713322639465, "operator": "ge", "actual_fraction": 0.5209426310864078, "metric": 0.6187788900547316, "rescued": 1563, "harmed": 3352, "missed_rescue": 1064, "avoided_harm": 5250, "selected_both_correct": 8439, "selected_both_wrong": 5016}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.7502986857825568, "rescued": 0, "harmed": 0, "missed_rescue": 1369, "avoided_harm": 2921, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -2.0005727208172175e-07, "operator": "ge", "actual_fraction": 0.009979619087778481, "metric": 0.7498770117365943, "rescued": 0, "harmed": 6, "missed_rescue": 1369, "avoided_harm": 2915, "selected_both_correct": 136, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -2.4156264544217265e-07, "operator": "ge", "actual_fraction": 0.019959238175556962, "metric": 0.7497364537212735, "rescued": 0, "harmed": 8, "missed_rescue": 1369, "avoided_harm": 2913, "selected_both_correct": 276, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -2.7729177531909954e-07, "operator": "ge", "actual_fraction": 0.029938857263335443, "metric": 0.7494553376906318, "rescued": 0, "harmed": 12, "missed_rescue": 1369, "avoided_harm": 2909, "selected_both_correct": 414, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": -3.064858447032748e-07, "operator": "ge", "actual_fraction": 0.03998875535877434, "metric": 0.749314779675311, "rescued": 0, "harmed": 14, "missed_rescue": 1369, "avoided_harm": 2907, "selected_both_correct": 555, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": -3.3848826319626824e-07, "operator": "ge", "actual_fraction": 0.04996837444655281, "metric": 0.7492445006676506, "rescued": 0, "harmed": 15, "missed_rescue": 1369, "avoided_harm": 2906, "selected_both_correct": 696, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": -3.668622525765386e-07, "operator": "ge", "actual_fraction": 0.05994799353433129, "metric": 0.7491742216599901, "rescued": 0, "harmed": 16, "missed_rescue": 1369, "avoided_harm": 2905, "selected_both_correct": 837, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": -3.98026287484754e-07, "operator": "ge", "actual_fraction": 0.06999789162977019, "metric": 0.7491039426523297, "rescued": 0, "harmed": 17, "missed_rescue": 1369, "avoided_harm": 2904, "selected_both_correct": 979, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": -4.302527258914779e-07, "operator": "ge", "actual_fraction": 0.07997751071754867, "metric": 0.7488931056293485, "rescued": 0, "harmed": 20, "missed_rescue": 1369, "avoided_harm": 2901, "selected_both_correct": 1118, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": -4.7191912244670675e-07, "operator": "ge", "actual_fraction": 0.08995712980532715, "metric": 0.7485417105910465, "rescued": 0, "harmed": 25, "missed_rescue": 1369, "avoided_harm": 2896, "selected_both_correct": 1255, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": -5.115426802149159e-07, "operator": "ge", "actual_fraction": 0.09993674889310562, "metric": 0.748120036545084, "rescued": 0, "harmed": 31, "missed_rescue": 1369, "avoided_harm": 2890, "selected_both_correct": 1391, "selected_both_wrong": 0}, {"budget": 0.11, "threshold": -5.534793672268279e-07, "operator": "ge", "actual_fraction": 0.10998664698854452, "metric": 0.7477686415067819, "rescued": 0, "harmed": 36, "missed_rescue": 1369, "avoided_harm": 2885, "selected_both_correct": 1529, "selected_both_wrong": 0}, {"budget": 0.12, "threshold": -5.941800509390305e-07, "operator": "ge", "actual_fraction": 0.119966266076323, "metric": 0.7472766884531591, "rescued": 0, "harmed": 43, "missed_rescue": 1369, "avoided_harm": 2878, "selected_both_correct": 1664, "selected_both_wrong": 0}, {"budget": 0.13, "threshold": -6.464504735959054e-07, "operator": "ge", "actual_fraction": 0.1299458851641015, "metric": 0.746573898376555, "rescued": 0, "harmed": 53, "missed_rescue": 1369, "avoided_harm": 2868, "selected_both_correct": 1796, "selected_both_wrong": 0}, {"budget": 0.14, "threshold": -6.999837864896108e-07, "operator": "ge", "actual_fraction": 0.13999578325954037, "metric": 0.7463630613535737, "rescued": 0, "harmed": 56, "missed_rescue": 1369, "avoided_harm": 2865, "selected_both_correct": 1936, "selected_both_wrong": 0}, {"budget": 0.15, "threshold": -7.663418841730163e-07, "operator": "ge", "actual_fraction": 0.14997540234731885, "metric": 0.7461522243305925, "rescued": 0, "harmed": 59, "missed_rescue": 1369, "avoided_harm": 2862, "selected_both_correct": 2075, "selected_both_wrong": 0}, {"budget": 0.16, "threshold": -8.471382670904859e-07, "operator": "ge", "actual_fraction": 0.15995502143509735, "metric": 0.74573055028463, "rescued": 0, "harmed": 65, "missed_rescue": 1369, "avoided_harm": 2856, "selected_both_correct": 2211, "selected_both_wrong": 0}, {"budget": 0.17, "threshold": -9.292756431023008e-07, "operator": "ge", "actual_fraction": 0.16993464052287582, "metric": 0.7454494342539884, "rescued": 0, "harmed": 69, "missed_rescue": 1369, "avoided_harm": 2852, "selected_both_correct": 2349, "selected_both_wrong": 0}, {"budget": 0.18, "threshold": -1.0245233852401725e-06, "operator": "ge", "actual_fraction": 0.1799845386183147, "metric": 0.7451683182233467, "rescued": 0, "harmed": 73, "missed_rescue": 1369, "avoided_harm": 2848, "selected_both_correct": 2488, "selected_both_wrong": 0}, {"budget": 0.19, "threshold": -1.1468601996966754e-06, "operator": "ge", "actual_fraction": 0.18996415770609318, "metric": 0.7444655281467426, "rescued": 0, "harmed": 83, "missed_rescue": 1369, "avoided_harm": 2838, "selected_both_correct": 2620, "selected_both_wrong": 0}, {"budget": 0.2, "threshold": -1.3018121762797819e-06, "operator": "ge", "actual_fraction": 0.19994377679387168, "metric": 0.7439032960854592, "rescued": 0, "harmed": 91, "missed_rescue": 1369, "avoided_harm": 2830, "selected_both_correct": 2754, "selected_both_wrong": 0}, {"budget": 0.21, "threshold": -1.4898060953782988e-06, "operator": "ge", "actual_fraction": 0.20999367488931056, "metric": 0.7430599479935344, "rescued": 0, "harmed": 103, "missed_rescue": 1369, "avoided_harm": 2818, "selected_both_correct": 2885, "selected_both_wrong": 0}, {"budget": 0.22, "threshold": -1.735574414851726e-06, "operator": "ge", "actual_fraction": 0.21997329397708903, "metric": 0.7422868789092698, "rescued": 0, "harmed": 114, "missed_rescue": 1369, "avoided_harm": 2807, "selected_both_correct": 3016, "selected_both_wrong": 0}, {"budget": 0.23, "threshold": -2.0881013824691763e-06, "operator": "ge", "actual_fraction": 0.22995291306486754, "metric": 0.7413732518096845, "rescued": 0, "harmed": 127, "missed_rescue": 1369, "avoided_harm": 2794, "selected_both_correct": 3144, "selected_both_wrong": 1}, {"budget": 0.24, "threshold": -2.5644872039265465e-06, "operator": "ge", "actual_fraction": 0.239932532152646, "metric": 0.7406704617330804, "rescued": 0, "harmed": 137, "missed_rescue": 1369, "avoided_harm": 2784, "selected_both_correct": 3276, "selected_both_wrong": 1}, {"budget": 0.25, "threshold": -3.2399550491390983e-06, "operator": "ge", "actual_fraction": 0.2499824302480849, "metric": 0.7400379506641366, "rescued": 0, "harmed": 146, "missed_rescue": 1369, "avoided_harm": 2775, "selected_both_correct": 3410, "selected_both_wrong": 1}, {"budget": 0.26, "threshold": -4.2517635847616475e-06, "operator": "ge", "actual_fraction": 0.25996204933586337, "metric": 0.7391946025722117, "rescued": 0, "harmed": 158, "missed_rescue": 1369, "avoided_harm": 2763, "selected_both_correct": 3540, "selected_both_wrong": 1}, {"budget": 0.27, "threshold": -5.607922958006384e-06, "operator": "ge", "actual_fraction": 0.26994166842364187, "metric": 0.738210696464966, "rescued": 0, "harmed": 172, "missed_rescue": 1369, "avoided_harm": 2749, "selected_both_correct": 3668, "selected_both_wrong": 1}, {"budget": 0.28, "threshold": -8.438555596512742e-06, "operator": "ge", "actual_fraction": 0.27999156651908075, "metric": 0.7374376273807014, "rescued": 0, "harmed": 183, "missed_rescue": 1369, "avoided_harm": 2738, "selected_both_correct": 3799, "selected_both_wrong": 2}, {"budget": 0.29, "threshold": -1.3145449884177651e-05, "operator": "ge", "actual_fraction": 0.28997118560685925, "metric": 0.7362428842504743, "rescued": 0, "harmed": 200, "missed_rescue": 1369, "avoided_harm": 2721, "selected_both_correct": 3924, "selected_both_wrong": 2}, {"budget": 0.3, "threshold": -2.076632881653495e-05, "operator": "ge", "actual_fraction": 0.2999508046946377, "metric": 0.7352589781432286, "rescued": 1, "harmed": 215, "missed_rescue": 1368, "avoided_harm": 2706, "selected_both_correct": 4050, "selected_both_wrong": 2}, {"budget": 0.31, "threshold": -4.0114435250870883e-05, "operator": "ge", "actual_fraction": 0.3099304237824162, "metric": 0.7342047930283224, "rescued": 1, "harmed": 230, "missed_rescue": 1368, "avoided_harm": 2691, "selected_both_correct": 4177, "selected_both_wrong": 2}, {"budget": 0.32, "threshold": -7.415730942739174e-05, "operator": "ge", "actual_fraction": 0.3199803218778551, "metric": 0.7327289338674537, "rescued": 1, "harmed": 251, "missed_rescue": 1368, "avoided_harm": 2670, "selected_both_correct": 4299, "selected_both_wrong": 2}, {"budget": 0.33, "threshold": -0.00015223574882838875, "operator": "ge", "actual_fraction": 0.3299599409656336, "metric": 0.7313233537142455, "rescued": 1, "harmed": 271, "missed_rescue": 1368, "avoided_harm": 2650, "selected_both_correct": 4421, "selected_both_wrong": 2}, {"budget": 0.34, "threshold": -0.00032258269493468106, "operator": "ge", "actual_fraction": 0.339939560053412, "metric": 0.7298474945533769, "rescued": 1, "harmed": 292, "missed_rescue": 1368, "avoided_harm": 2629, "selected_both_correct": 4541, "selected_both_wrong": 3}, {"budget": 0.35, "threshold": -0.0006007538759149611, "operator": "ge", "actual_fraction": 0.34998945814885096, "metric": 0.7284419144001687, "rescued": 1, "harmed": 312, "missed_rescue": 1368, "avoided_harm": 2609, "selected_both_correct": 4664, "selected_both_wrong": 3}, {"budget": 0.36, "threshold": -0.0011453087208792567, "operator": "ge", "actual_fraction": 0.3599690772366294, "metric": 0.7268957762316396, "rescued": 1, "harmed": 334, "missed_rescue": 1368, "avoided_harm": 2587, "selected_both_correct": 4784, "selected_both_wrong": 3}, {"budget": 0.37, "threshold": -0.002608026610687375, "operator": "ge", "actual_fraction": 0.3699486963244079, "metric": 0.7252793590554502, "rescued": 1, "harmed": 357, "missed_rescue": 1368, "avoided_harm": 2564, "selected_both_correct": 4903, "selected_both_wrong": 3}, {"budget": 0.38, "threshold": -0.006219128146767616, "operator": "ge", "actual_fraction": 0.3799985944198468, "metric": 0.7234521048562794, "rescued": 1, "harmed": 383, "missed_rescue": 1368, "avoided_harm": 2538, "selected_both_correct": 5019, "selected_both_wrong": 4}, {"budget": 0.39, "threshold": -0.01280172262340784, "operator": "ge", "actual_fraction": 0.3899782135076253, "metric": 0.7216248506571087, "rescued": 1, "harmed": 409, "missed_rescue": 1368, "avoided_harm": 2512, "selected_both_correct": 5134, "selected_both_wrong": 5}, {"budget": 0.4, "threshold": -0.03794781118631363, "operator": "ge", "actual_fraction": 0.39995783259540374, "metric": 0.719446201419636, "rescued": 1, "harmed": 440, "missed_rescue": 1368, "avoided_harm": 2481, "selected_both_correct": 5242, "selected_both_wrong": 8}, {"budget": 0.41, "threshold": -0.08479184657335281, "operator": "ge", "actual_fraction": 0.40993745168318224, "metric": 0.7169161571438611, "rescued": 1, "harmed": 476, "missed_rescue": 1368, "avoided_harm": 2445, "selected_both_correct": 5347, "selected_both_wrong": 9}, {"budget": 0.42, "threshold": -0.14941807091236115, "operator": "ge", "actual_fraction": 0.4199873497786211, "metric": 0.7148077869140488, "rescued": 3, "harmed": 508, "missed_rescue": 1366, "avoided_harm": 2413, "selected_both_correct": 5450, "selected_both_wrong": 15}, {"budget": 0.43, "threshold": -0.2939925789833069, "operator": "ge", "actual_fraction": 0.4299669688663996, "metric": 0.7126994166842364, "rescued": 5, "harmed": 540, "missed_rescue": 1364, "avoided_harm": 2381, "selected_both_correct": 5552, "selected_both_wrong": 21}, {"budget": 0.44, "threshold": -0.4386065602302551, "operator": "ge", "actual_fraction": 0.43994658795417807, "metric": 0.7100288143931408, "rescued": 5, "harmed": 578, "missed_rescue": 1364, "avoided_harm": 2343, "selected_both_correct": 5648, "selected_both_wrong": 29}, {"budget": 0.45, "threshold": -0.5420411825180054, "operator": "ge", "actual_fraction": 0.449996486049617, "metric": 0.7070770960714035, "rescued": 5, "harmed": 620, "missed_rescue": 1364, "avoided_harm": 2301, "selected_both_correct": 5742, "selected_both_wrong": 36}, {"budget": 0.46, "threshold": -0.6230618953704834, "operator": "ge", "actual_fraction": 0.45997610513739545, "metric": 0.7045470517956286, "rescued": 6, "harmed": 657, "missed_rescue": 1363, "avoided_harm": 2264, "selected_both_correct": 5841, "selected_both_wrong": 41}, {"budget": 0.47, "threshold": -0.6914289593696594, "operator": "ge", "actual_fraction": 0.46995572422517395, "metric": 0.7017358914892122, "rescued": 7, "harmed": 698, "missed_rescue": 1362, "avoided_harm": 2223, "selected_both_correct": 5930, "selected_both_wrong": 52}, {"budget": 0.48, "threshold": -0.7387455701828003, "operator": "ge", "actual_fraction": 0.4799353433129524, "metric": 0.7059526319488368, "rescued": 76, "harmed": 707, "missed_rescue": 1293, "avoided_harm": 2214, "selected_both_correct": 5945, "selected_both_wrong": 101}, {"budget": 0.49, "threshold": -0.7823225855827332, "operator": "ge", "actual_fraction": 0.48998524140839134, "metric": 0.7123480216459344, "rescued": 167, "harmed": 707, "missed_rescue": 1202, "avoided_harm": 2214, "selected_both_correct": 5945, "selected_both_wrong": 153}, {"budget": 0.5, "threshold": -0.8206579089164734, "operator": "ge", "actual_fraction": 0.4999648604961698, "metric": 0.7179000632511069, "rescued": 247, "harmed": 708, "missed_rescue": 1122, "avoided_harm": 2213, "selected_both_correct": 5945, "selected_both_wrong": 214}, {"budget": 0.51, "threshold": -0.8575630187988281, "operator": "ge", "actual_fraction": 0.5099444795839483, "metric": 0.723381825848619, "rescued": 326, "harmed": 709, "missed_rescue": 1043, "avoided_harm": 2212, "selected_both_correct": 5945, "selected_both_wrong": 276}, {"budget": 0.52, "threshold": -0.8823798894882202, "operator": "ge", "actual_fraction": 0.5199943776793872, "metric": 0.729004146461452, "rescued": 407, "harmed": 710, "missed_rescue": 962, "avoided_harm": 2211, "selected_both_correct": 5945, "selected_both_wrong": 337}, {"budget": 0.53, "threshold": -0.9040287733078003, "operator": "ge", "actual_fraction": 0.5299739967671656, "metric": 0.7339939560053412, "rescued": 478, "harmed": 710, "missed_rescue": 891, "avoided_harm": 2211, "selected_both_correct": 5946, "selected_both_wrong": 407}, {"budget": 0.54, "threshold": -0.9208687543869019, "operator": "ge", "actual_fraction": 0.5399536158549442, "metric": 0.7390540445568908, "rescued": 552, "harmed": 712, "missed_rescue": 817, "avoided_harm": 2209, "selected_both_correct": 5946, "selected_both_wrong": 473}, {"budget": 0.55, "threshold": -0.9344452619552612, "operator": "ge", "actual_fraction": 0.5499332349427226, "metric": 0.743692459062478, "rescued": 620, "harmed": 714, "missed_rescue": 749, "avoided_harm": 2207, "selected_both_correct": 5947, "selected_both_wrong": 544}, {"budget": 0.56, "threshold": -0.9448322653770447, "operator": "ge", "actual_fraction": 0.5599831330381615, "metric": 0.7482605945604048, "rescued": 690, "harmed": 719, "missed_rescue": 679, "avoided_harm": 2202, "selected_both_correct": 5948, "selected_both_wrong": 611}, {"budget": 0.57, "threshold": -0.9531905055046082, "operator": "ge", "actual_fraction": 0.5699627521259399, "metric": 0.7525476140276899, "rescued": 755, "harmed": 723, "missed_rescue": 614, "avoided_harm": 2198, "selected_both_correct": 5948, "selected_both_wrong": 684}, {"budget": 0.58, "threshold": -0.9603281617164612, "operator": "gt", "actual_fraction": 0.5798720922060581, "metric": 0.7554993323494272, "rescued": 802, "harmed": 728, "missed_rescue": 567, "avoided_harm": 2193, "selected_both_correct": 5949, "selected_both_wrong": 772}, {"budget": 0.59, "threshold": -0.9670801758766174, "operator": "ge", "actual_fraction": 0.5899922693091574, "metric": 0.7591538407477686, "rescued": 859, "harmed": 733, "missed_rescue": 510, "avoided_harm": 2188, "selected_both_correct": 5953, "selected_both_wrong": 850}, {"budget": 0.6, "threshold": -0.9724565148353577, "operator": "ge", "actual_fraction": 0.5999718883969358, "metric": 0.7622461170848268, "rescued": 911, "harmed": 741, "missed_rescue": 458, "avoided_harm": 2180, "selected_both_correct": 5956, "selected_both_wrong": 929}, {"budget": 0.61, "threshold": -0.9766654968261719, "operator": "ge", "actual_fraction": 0.6099515074847143, "metric": 0.7658303464755077, "rescued": 970, "harmed": 749, "missed_rescue": 399, "avoided_harm": 2172, "selected_both_correct": 5959, "selected_both_wrong": 1001}, {"budget": 0.62, "threshold": -0.9801702499389648, "operator": "ge", "actual_fraction": 0.6199311265724928, "metric": 0.7682198327359617, "rescued": 1014, "harmed": 759, "missed_rescue": 355, "avoided_harm": 2162, "selected_both_correct": 5961, "selected_both_wrong": 1087}, {"budget": 0.63, "threshold": -0.9831460118293762, "operator": "ge", "actual_fraction": 0.6299810246679317, "metric": 0.769976807927472, "rescued": 1049, "harmed": 769, "missed_rescue": 320, "avoided_harm": 2152, "selected_both_correct": 5971, "selected_both_wrong": 1175}, {"budget": 0.64, "threshold": -0.985751748085022, "operator": "ge", "actual_fraction": 0.6399606437557102, "metric": 0.7713121090730198, "rescued": 1089, "harmed": 790, "missed_rescue": 280, "avoided_harm": 2131, "selected_both_correct": 5977, "selected_both_wrong": 1250}, {"budget": 0.65, "threshold": -0.9877713322639465, "operator": "ge", "actual_fraction": 0.6499402628434886, "metric": 0.7716635041113219, "rescued": 1117, "harmed": 813, "missed_rescue": 252, "avoided_harm": 2108, "selected_both_correct": 5984, "selected_both_wrong": 1334}, {"budget": 0.66, "threshold": -0.9895955324172974, "operator": "ge", "actual_fraction": 0.6599901609389276, "metric": 0.7709607140347178, "rescued": 1141, "harmed": 847, "missed_rescue": 228, "avoided_harm": 2074, "selected_both_correct": 5993, "selected_both_wrong": 1410}, {"budget": 0.67, "threshold": -0.9913432002067566, "operator": "ge", "actual_fraction": 0.669969780026706, "metric": 0.7709607140347178, "rescued": 1164, "harmed": 870, "missed_rescue": 205, "avoided_harm": 2051, "selected_both_correct": 6013, "selected_both_wrong": 1486}, {"budget": 0.68, "threshold": -0.9925730228424072, "operator": "ge", "actual_fraction": 0.6799493991144845, "metric": 0.7703282029657741, "rescued": 1191, "harmed": 906, "missed_rescue": 178, "avoided_harm": 2015, "selected_both_correct": 6033, "selected_both_wrong": 1545}, {"budget": 0.69, "threshold": -0.9935283660888672, "operator": "ge", "actual_fraction": 0.6899992972099234, "metric": 0.7687117857895847, "rescued": 1208, "harmed": 946, "missed_rescue": 161, "avoided_harm": 1975, "selected_both_correct": 6063, "selected_both_wrong": 1601}, {"budget": 0.7, "threshold": -0.99436354637146, "operator": "gt", "actual_fraction": 0.6999086372900415, "metric": 0.7656195094525265, "rescued": 1224, "harmed": 1006, "missed_rescue": 145, "avoided_harm": 1915, "selected_both_correct": 6082, "selected_both_wrong": 1647}, {"budget": 0.71, "threshold": -0.9949791431427002, "operator": "ge", "actual_fraction": 0.7099585353854804, "metric": 0.7630191861690913, "rescued": 1234, "harmed": 1053, "missed_rescue": 135, "avoided_harm": 1868, "selected_both_correct": 6118, "selected_both_wrong": 1697}, {"budget": 0.72, "threshold": -0.9955723881721497, "operator": "ge", "actual_fraction": 0.7199381544732588, "metric": 0.7605594209009768, "rescued": 1247, "harmed": 1101, "missed_rescue": 122, "avoided_harm": 1820, "selected_both_correct": 6158, "selected_both_wrong": 1738}, {"budget": 0.73, "threshold": -0.9960037469863892, "operator": "ge", "actual_fraction": 0.7299880525686977, "metric": 0.7559210063953897, "rescued": 1253, "harmed": 1173, "missed_rescue": 116, "avoided_harm": 1748, "selected_both_correct": 6185, "selected_both_wrong": 1776}, {"budget": 0.74, "threshold": -0.9964693188667297, "operator": "ge", "actual_fraction": 0.7399676716564763, "metric": 0.7533206831119544, "rescued": 1265, "harmed": 1222, "missed_rescue": 104, "avoided_harm": 1699, "selected_both_correct": 6233, "selected_both_wrong": 1809}, {"budget": 0.75, "threshold": -0.9968041777610779, "operator": "ge", "actual_fraction": 0.7499472907442547, "metric": 0.7491742216599901, "rescued": 1272, "harmed": 1288, "missed_rescue": 97, "avoided_harm": 1633, "selected_both_correct": 6272, "selected_both_wrong": 1839}, {"budget": 0.76, "threshold": -0.9971042275428772, "operator": "ge", "actual_fraction": 0.7599971888396936, "metric": 0.7455899922693091, "rescued": 1280, "harmed": 1347, "missed_rescue": 89, "avoided_harm": 1574, "selected_both_correct": 6315, "selected_both_wrong": 1872}, {"budget": 0.77, "threshold": -0.9974007606506348, "operator": "ge", "actual_fraction": 0.769976807927472, "metric": 0.7415840888326657, "rescued": 1288, "harmed": 1412, "missed_rescue": 81, "avoided_harm": 1509, "selected_both_correct": 6353, "selected_both_wrong": 1903}, {"budget": 0.78, "threshold": -0.9976577162742615, "operator": "ge", "actual_fraction": 0.7799564270152506, "metric": 0.7378593014266639, "rescued": 1295, "harmed": 1472, "missed_rescue": 74, "avoided_harm": 1449, "selected_both_correct": 6399, "selected_both_wrong": 1932}, {"budget": 0.79, "threshold": -0.9978766441345215, "operator": "ge", "actual_fraction": 0.789936046103029, "metric": 0.7342750720359829, "rescued": 1306, "harmed": 1534, "missed_rescue": 63, "avoided_harm": 1387, "selected_both_correct": 6446, "selected_both_wrong": 1954}, {"budget": 0.8, "threshold": -0.9980920553207397, "operator": "ge", "actual_fraction": 0.7999859441984679, "metric": 0.7297069365380561, "rescued": 1316, "harmed": 1609, "missed_rescue": 53, "avoided_harm": 1312, "selected_both_correct": 6491, "selected_both_wrong": 1967}, {"budget": 0.81, "threshold": -0.9982843399047852, "operator": "ge", "actual_fraction": 0.8099655632862464, "metric": 0.7263335441703563, "rescued": 1324, "harmed": 1665, "missed_rescue": 45, "avoided_harm": 1256, "selected_both_correct": 6553, "selected_both_wrong": 1983}, {"budget": 0.82, "threshold": -0.9984900951385498, "operator": "ge", "actual_fraction": 0.8199451823740249, "metric": 0.7221168037107316, "rescued": 1329, "harmed": 1730, "missed_rescue": 40, "avoided_harm": 1191, "selected_both_correct": 6599, "selected_both_wrong": 2009}, {"budget": 0.83, "threshold": -0.9986395835876465, "operator": "ge", "actual_fraction": 0.8299950804694638, "metric": 0.718251458289409, "rescued": 1333, "harmed": 1789, "missed_rescue": 36, "avoided_harm": 1132, "selected_both_correct": 6654, "selected_both_wrong": 2034}, {"budget": 0.84, "threshold": -0.9987816214561462, "operator": "ge", "actual_fraction": 0.8399746995572422, "metric": 0.7131913697378593, "rescued": 1337, "harmed": 1865, "missed_rescue": 32, "avoided_harm": 1056, "selected_both_correct": 6699, "selected_both_wrong": 2051}, {"budget": 0.85, "threshold": -0.9989162087440491, "operator": "ge", "actual_fraction": 0.8499543186450207, "metric": 0.7093260243165367, "rescued": 1340, "harmed": 1923, "missed_rescue": 29, "avoided_harm": 998, "selected_both_correct": 6763, "selected_both_wrong": 2068}, {"budget": 0.86, "threshold": -0.9990254044532776, "operator": "ge", "actual_fraction": 0.8599339377327992, "metric": 0.7037037037037037, "rescued": 1343, "harmed": 2006, "missed_rescue": 26, "avoided_harm": 915, "selected_both_correct": 6810, "selected_both_wrong": 2077}, {"budget": 0.87, "threshold": -0.9991487264633179, "operator": "ge", "actual_fraction": 0.8699838358282381, "metric": 0.6985030571368332, "rescued": 1347, "harmed": 2084, "missed_rescue": 22, "avoided_harm": 837, "selected_both_correct": 6856, "selected_both_wrong": 2092}, {"budget": 0.88, "threshold": -0.9992602467536926, "operator": "ge", "actual_fraction": 0.8799634549160166, "metric": 0.6937943636235856, "rescued": 1353, "harmed": 2157, "missed_rescue": 16, "avoided_harm": 764, "selected_both_correct": 6910, "selected_both_wrong": 2101}, {"budget": 0.89, "threshold": -0.999350905418396, "operator": "ge", "actual_fraction": 0.889943074003795, "metric": 0.6895776231639609, "rescued": 1356, "harmed": 2220, "missed_rescue": 13, "avoided_harm": 701, "selected_both_correct": 6973, "selected_both_wrong": 2114}, {"budget": 0.9, "threshold": -0.999435544013977, "operator": "ge", "actual_fraction": 0.899992972099234, "metric": 0.6846580926277321, "rescued": 1359, "harmed": 2293, "missed_rescue": 10, "avoided_harm": 628, "selected_both_correct": 7029, "selected_both_wrong": 2125}, {"budget": 0.91, "threshold": -0.9995256066322327, "operator": "ge", "actual_fraction": 0.9099725911870125, "metric": 0.6794574460608617, "rescued": 1359, "harmed": 2367, "missed_rescue": 10, "avoided_harm": 554, "selected_both_correct": 7084, "selected_both_wrong": 2138}, {"budget": 0.92, "threshold": -0.9995898604393005, "operator": "ge", "actual_fraction": 0.9199522102747909, "metric": 0.6744676365169724, "rescued": 1361, "harmed": 2440, "missed_rescue": 8, "avoided_harm": 481, "selected_both_correct": 7144, "selected_both_wrong": 2145}, {"budget": 0.93, "threshold": -0.9996523857116699, "operator": "ge", "actual_fraction": 0.9299318293625694, "metric": 0.6701806170496872, "rescued": 1364, "harmed": 2504, "missed_rescue": 5, "avoided_harm": 417, "selected_both_correct": 7216, "selected_both_wrong": 2148}, {"budget": 0.94, "threshold": -0.9997095465660095, "operator": "ge", "actual_fraction": 0.9399817274580083, "metric": 0.6651908075057981, "rescued": 1367, "harmed": 2578, "missed_rescue": 2, "avoided_harm": 343, "selected_both_correct": 7278, "selected_both_wrong": 2152}, {"budget": 0.95, "threshold": -0.9997634887695312, "operator": "ge", "actual_fraction": 0.9499613465457868, "metric": 0.6606226720078713, "rescued": 1367, "harmed": 2643, "missed_rescue": 2, "avoided_harm": 278, "selected_both_correct": 7349, "selected_both_wrong": 2158}, {"budget": 0.96, "threshold": -0.9998065829277039, "operator": "ge", "actual_fraction": 0.9599409656335652, "metric": 0.6557031414716424, "rescued": 1368, "harmed": 2714, "missed_rescue": 1, "avoided_harm": 207, "selected_both_correct": 7414, "selected_both_wrong": 2163}, {"budget": 0.97, "threshold": -0.9998431205749512, "operator": "ge", "actual_fraction": 0.9699908637290041, "metric": 0.6521891910886218, "rescued": 1368, "harmed": 2764, "missed_rescue": 1, "avoided_harm": 157, "selected_both_correct": 7498, "selected_both_wrong": 2172}, {"budget": 0.98, "threshold": -0.9998764991760254, "operator": "ge", "actual_fraction": 0.9799704828167827, "metric": 0.6481832876519783, "rescued": 1369, "harmed": 2822, "missed_rescue": 0, "avoided_harm": 99, "selected_both_correct": 7574, "selected_both_wrong": 2179}, {"budget": 0.99, "threshold": -0.9999123811721802, "operator": "ge", "actual_fraction": 0.9899501019045611, "metric": 0.6447396162766181, "rescued": 1369, "harmed": 2871, "missed_rescue": 0, "avoided_harm": 50, "selected_both_correct": 7664, "selected_both_wrong": 2182}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.6412256658935975, "rescued": 1369, "harmed": 2921, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 7755, "selected_both_wrong": 2184}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.6695119530385957, "rescued": 0, "harmed": 0, "missed_rescue": 2627, "avoided_harm": 8602, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -2.0005727208172175e-07, "operator": "ge", "actual_fraction": 0.002467175226157729, "metric": 0.6686044862887446, "rescued": 0, "harmed": 32, "missed_rescue": 2627, "avoided_harm": 8570, "selected_both_correct": 55, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -2.4156264544217265e-07, "operator": "ge", "actual_fraction": 0.004849275444516916, "metric": 0.6678388112185577, "rescued": 0, "harmed": 59, "missed_rescue": 2627, "avoided_harm": 8543, "selected_both_correct": 112, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -2.7729177531909954e-07, "operator": "ge", "actual_fraction": 0.007600034030003119, "metric": 0.667214927828035, "rescued": 0, "harmed": 81, "missed_rescue": 2627, "avoided_harm": 8521, "selected_both_correct": 186, "selected_both_wrong": 1}, {"budget": 0.04, "threshold": -3.064858447032748e-07, "operator": "ge", "actual_fraction": 0.009811984232765221, "metric": 0.6666761194453109, "rescued": 0, "harmed": 100, "missed_rescue": 2627, "avoided_harm": 8502, "selected_both_correct": 245, "selected_both_wrong": 1}, {"budget": 0.05, "threshold": -3.3848826319626824e-07, "operator": "ge", "actual_fraction": 0.012534384482318578, "metric": 0.6662791027422511, "rescued": 0, "harmed": 114, "missed_rescue": 2627, "avoided_harm": 8488, "selected_both_correct": 326, "selected_both_wrong": 2}, {"budget": 0.06, "threshold": -3.668622525765386e-07, "operator": "ge", "actual_fraction": 0.01607917647392451, "metric": 0.6658537277032583, "rescued": 0, "harmed": 129, "missed_rescue": 2627, "avoided_harm": 8473, "selected_both_correct": 436, "selected_both_wrong": 2}, {"budget": 0.07, "threshold": -3.98026287484754e-07, "operator": "ge", "actual_fraction": 0.019709043473328986, "metric": 0.6654850693361314, "rescued": 0, "harmed": 142, "missed_rescue": 2627, "avoided_harm": 8460, "selected_both_correct": 550, "selected_both_wrong": 3}, {"budget": 0.08, "threshold": -4.302527258914779e-07, "operator": "ge", "actual_fraction": 0.02353741882426339, "metric": 0.6652865609846014, "rescued": 0, "harmed": 149, "missed_rescue": 2627, "avoided_harm": 8453, "selected_both_correct": 678, "selected_both_wrong": 3}, {"budget": 0.09, "threshold": -4.7191912244670675e-07, "operator": "ge", "actual_fraction": 0.028868785979638715, "metric": 0.6648611859456087, "rescued": 0, "harmed": 164, "missed_rescue": 2627, "avoided_harm": 8438, "selected_both_correct": 850, "selected_both_wrong": 4}, {"budget": 0.1, "threshold": -5.115426802149159e-07, "operator": "ge", "actual_fraction": 0.0345971698380739, "metric": 0.6645208859144145, "rescued": 0, "harmed": 176, "missed_rescue": 2627, "avoided_harm": 8426, "selected_both_correct": 1036, "selected_both_wrong": 8}, {"budget": 0.11, "threshold": -5.534793672268279e-07, "operator": "ge", "actual_fraction": 0.03981510364971783, "metric": 0.6642089442191532, "rescued": 0, "harmed": 187, "missed_rescue": 2627, "avoided_harm": 8415, "selected_both_correct": 1207, "selected_both_wrong": 10}, {"budget": 0.12, "threshold": -5.941800509390305e-07, "operator": "ge", "actual_fraction": 0.045855429203414344, "metric": 0.6640387942035562, "rescued": 0, "harmed": 193, "missed_rescue": 2627, "avoided_harm": 8409, "selected_both_correct": 1413, "selected_both_wrong": 11}, {"budget": 0.13, "threshold": -6.464504735959054e-07, "operator": "ge", "actual_fraction": 0.05317187987408899, "metric": 0.6637268525082948, "rescued": 1, "harmed": 205, "missed_rescue": 2626, "avoided_harm": 8397, "selected_both_correct": 1657, "selected_both_wrong": 12}, {"budget": 0.14, "threshold": -6.999837864896108e-07, "operator": "ge", "actual_fraction": 0.05960922213084536, "metric": 0.6633581941411678, "rescued": 1, "harmed": 218, "missed_rescue": 2626, "avoided_harm": 8384, "selected_both_correct": 1869, "selected_both_wrong": 14}, {"budget": 0.15, "threshold": -7.663418841730163e-07, "operator": "ge", "actual_fraction": 0.0675211978561098, "metric": 0.6630178941099736, "rescued": 1, "harmed": 230, "missed_rescue": 2626, "avoided_harm": 8372, "selected_both_correct": 2122, "selected_both_wrong": 28}, {"budget": 0.16, "threshold": -8.471382670904859e-07, "operator": "ge", "actual_fraction": 0.07551824858917279, "metric": 0.6626492357428466, "rescued": 1, "harmed": 243, "missed_rescue": 2626, "avoided_harm": 8359, "selected_both_correct": 2388, "selected_both_wrong": 31}, {"budget": 0.17, "threshold": -9.292756431023008e-07, "operator": "ge", "actual_fraction": 0.08212574086152624, "metric": 0.6621671440319882, "rescued": 1, "harmed": 260, "missed_rescue": 2626, "avoided_harm": 8342, "selected_both_correct": 2598, "selected_both_wrong": 37}, {"budget": 0.18, "threshold": -1.0245233852401725e-06, "operator": "ge", "actual_fraction": 0.08856308311828262, "metric": 0.6619969940163911, "rescued": 1, "harmed": 266, "missed_rescue": 2626, "avoided_harm": 8336, "selected_both_correct": 2814, "selected_both_wrong": 42}, {"budget": 0.19, "threshold": -1.1468601996966754e-06, "operator": "ge", "actual_fraction": 0.09480191702350906, "metric": 0.6615716189773984, "rescued": 1, "harmed": 281, "missed_rescue": 2626, "avoided_harm": 8321, "selected_both_correct": 3009, "selected_both_wrong": 52}, {"budget": 0.2, "threshold": -1.3018121762797819e-06, "operator": "ge", "actual_fraction": 0.10007656750701868, "metric": 0.6612029606102714, "rescued": 1, "harmed": 294, "missed_rescue": 2626, "avoided_harm": 8308, "selected_both_correct": 3174, "selected_both_wrong": 60}, {"budget": 0.21, "threshold": -1.4898060953782988e-06, "operator": "ge", "actual_fraction": 0.10554972634205825, "metric": 0.6608910189150101, "rescued": 1, "harmed": 305, "missed_rescue": 2626, "avoided_harm": 8297, "selected_both_correct": 3344, "selected_both_wrong": 72}, {"budget": 0.22, "threshold": -1.735574414851726e-06, "operator": "ge", "actual_fraction": 0.11005870175538099, "metric": 0.6605790772197487, "rescued": 1, "harmed": 316, "missed_rescue": 2626, "avoided_harm": 8286, "selected_both_correct": 3488, "selected_both_wrong": 76}, {"budget": 0.23, "threshold": -2.0881013824691763e-06, "operator": "ge", "actual_fraction": 0.11527663556702493, "metric": 0.6602671355244875, "rescued": 3, "harmed": 329, "missed_rescue": 2624, "avoided_harm": 8273, "selected_both_correct": 3638, "selected_both_wrong": 95}, {"budget": 0.24, "threshold": -2.5644872039265465e-06, "operator": "ge", "actual_fraction": 0.12029606102713893, "metric": 0.6599835521651589, "rescued": 4, "harmed": 340, "missed_rescue": 2623, "avoided_harm": 8262, "selected_both_correct": 3778, "selected_both_wrong": 120}, {"budget": 0.25, "threshold": -3.2399550491390983e-06, "operator": "ge", "actual_fraction": 0.12477667810452883, "metric": 0.6596999688058305, "rescued": 4, "harmed": 350, "missed_rescue": 2623, "avoided_harm": 8252, "selected_both_correct": 3906, "selected_both_wrong": 140}, {"budget": 0.26, "threshold": -4.2517635847616475e-06, "operator": "ge", "actual_fraction": 0.12914386183818732, "metric": 0.659217877094972, "rescued": 4, "harmed": 367, "missed_rescue": 2623, "avoided_harm": 8235, "selected_both_correct": 4023, "selected_both_wrong": 160}, {"budget": 0.27, "threshold": -5.607922958006384e-06, "operator": "ge", "actual_fraction": 0.1322916371267334, "metric": 0.6587925020559794, "rescued": 4, "harmed": 382, "missed_rescue": 2623, "avoided_harm": 8220, "selected_both_correct": 4102, "selected_both_wrong": 177}, {"budget": 0.28, "threshold": -8.438555596512742e-06, "operator": "ge", "actual_fraction": 0.13626180415733205, "metric": 0.6582253353373224, "rescued": 5, "harmed": 403, "missed_rescue": 2622, "avoided_harm": 8199, "selected_both_correct": 4197, "selected_both_wrong": 200}, {"budget": 0.29, "threshold": -1.3145449884177651e-05, "operator": "ge", "actual_fraction": 0.14023197118793068, "metric": 0.6578283186342625, "rescued": 6, "harmed": 418, "missed_rescue": 2621, "avoided_harm": 8184, "selected_both_correct": 4295, "selected_both_wrong": 226}, {"budget": 0.3, "threshold": -2.076632881653495e-05, "operator": "ge", "actual_fraction": 0.14326631313274538, "metric": 0.6573178685874713, "rescued": 7, "harmed": 437, "missed_rescue": 2620, "avoided_harm": 8165, "selected_both_correct": 4366, "selected_both_wrong": 242}, {"budget": 0.31, "threshold": -4.0114435250870883e-05, "operator": "ge", "actual_fraction": 0.14766185520233674, "metric": 0.6569208518844114, "rescued": 8, "harmed": 452, "missed_rescue": 2619, "avoided_harm": 8150, "selected_both_correct": 4472, "selected_both_wrong": 275}, {"budget": 0.32, "threshold": -7.415730942739174e-05, "operator": "ge", "actual_fraction": 0.15066783881121856, "metric": 0.6565805518532173, "rescued": 9, "harmed": 465, "missed_rescue": 2618, "avoided_harm": 8137, "selected_both_correct": 4550, "selected_both_wrong": 289}, {"budget": 0.33, "threshold": -0.00015223574882838875, "operator": "ge", "actual_fraction": 0.15452457249808582, "metric": 0.6561268184782917, "rescued": 12, "harmed": 484, "missed_rescue": 2615, "avoided_harm": 8118, "selected_both_correct": 4633, "selected_both_wrong": 320}, {"budget": 0.34, "threshold": -0.00032258269493468106, "operator": "ge", "actual_fraction": 0.15806936448969175, "metric": 0.6558715934548961, "rescued": 16, "harmed": 497, "missed_rescue": 2611, "avoided_harm": 8105, "selected_both_correct": 4709, "selected_both_wrong": 352}, {"budget": 0.35, "threshold": -0.0006007538759149611, "operator": "ge", "actual_fraction": 0.16201117318435754, "metric": 0.6555596517596347, "rescued": 17, "harmed": 509, "missed_rescue": 2610, "avoided_harm": 8093, "selected_both_correct": 4781, "selected_both_wrong": 406}, {"budget": 0.36, "threshold": -0.0011453087208792567, "operator": "ge", "actual_fraction": 0.1657544735274934, "metric": 0.6551626350565749, "rescued": 19, "harmed": 525, "missed_rescue": 2608, "avoided_harm": 8077, "selected_both_correct": 4844, "selected_both_wrong": 457}, {"budget": 0.37, "threshold": -0.002608026610687375, "operator": "ge", "actual_fraction": 0.17012165726115192, "metric": 0.6545954683379179, "rescued": 23, "harmed": 549, "missed_rescue": 2604, "avoided_harm": 8053, "selected_both_correct": 4930, "selected_both_wrong": 497}, {"budget": 0.38, "threshold": -0.006219128146767616, "operator": "ge", "actual_fraction": 0.17536794940872868, "metric": 0.6540850182911266, "rescued": 33, "harmed": 577, "missed_rescue": 2594, "avoided_harm": 8025, "selected_both_correct": 5012, "selected_both_wrong": 562}, {"budget": 0.39, "threshold": -0.01280172262340784, "operator": "ge", "actual_fraction": 0.17965005813458867, "metric": 0.6534044182287383, "rescued": 39, "harmed": 607, "missed_rescue": 2588, "avoided_harm": 7995, "selected_both_correct": 5077, "selected_both_wrong": 612}, {"budget": 0.4, "threshold": -0.03794781118631363, "operator": "ge", "actual_fraction": 0.18401724186824717, "metric": 0.653092476533477, "rescued": 44, "harmed": 623, "missed_rescue": 2583, "avoided_harm": 7979, "selected_both_correct": 5133, "selected_both_wrong": 689}, {"budget": 0.41, "threshold": -0.08479184657335281, "operator": "ge", "actual_fraction": 0.18858293395343562, "metric": 0.6527521765022829, "rescued": 46, "harmed": 637, "missed_rescue": 2581, "avoided_harm": 7965, "selected_both_correct": 5199, "selected_both_wrong": 768}, {"budget": 0.42, "threshold": -0.14941807091236115, "operator": "ge", "actual_fraction": 0.19238295096843716, "metric": 0.6523835181351558, "rescued": 52, "harmed": 656, "missed_rescue": 2575, "avoided_harm": 7946, "selected_both_correct": 5260, "selected_both_wrong": 816}, {"budget": 0.43, "threshold": -0.2939925789833069, "operator": "ge", "actual_fraction": 0.19689192638175992, "metric": 0.6517596347446332, "rescued": 63, "harmed": 689, "missed_rescue": 2564, "avoided_harm": 7913, "selected_both_correct": 5317, "selected_both_wrong": 874}, {"budget": 0.44, "threshold": -0.4386065602302551, "operator": "ge", "actual_fraction": 0.20182627683407536, "metric": 0.6511641096900433, "rescued": 77, "harmed": 724, "missed_rescue": 2550, "avoided_harm": 7878, "selected_both_correct": 5395, "selected_both_wrong": 921}, {"budget": 0.45, "threshold": -0.5420411825180054, "operator": "ge", "actual_fraction": 0.20718600232538353, "metric": 0.6505402262995208, "rescued": 97, "harmed": 766, "missed_rescue": 2530, "avoided_harm": 7836, "selected_both_correct": 5459, "selected_both_wrong": 984}, {"budget": 0.46, "threshold": -0.6230618953704834, "operator": "ge", "actual_fraction": 0.21189348609023623, "metric": 0.6500864929245952, "rescued": 122, "harmed": 807, "missed_rescue": 2505, "avoided_harm": 7795, "selected_both_correct": 5501, "selected_both_wrong": 1042}, {"budget": 0.47, "threshold": -0.6914289593696594, "operator": "ge", "actual_fraction": 0.217139778237813, "metric": 0.6500581345886624, "rescued": 151, "harmed": 837, "missed_rescue": 2476, "avoided_harm": 7765, "selected_both_correct": 5555, "selected_both_wrong": 1114}, {"budget": 0.48, "threshold": -0.7387455701828003, "operator": "ge", "actual_fraction": 0.22198905368232993, "metric": 0.6507670929869835, "rescued": 203, "harmed": 864, "missed_rescue": 2424, "avoided_harm": 7738, "selected_both_correct": 5574, "selected_both_wrong": 1187}, {"budget": 0.49, "threshold": -0.7823225855827332, "operator": "ge", "actual_fraction": 0.22763236253296656, "metric": 0.6515611263931033, "rescued": 267, "harmed": 900, "missed_rescue": 2360, "avoided_harm": 7702, "selected_both_correct": 5592, "selected_both_wrong": 1268}, {"budget": 0.5, "threshold": -0.8206579089164734, "operator": "ge", "actual_fraction": 0.2345234381646485, "metric": 0.6527521765022829, "rescued": 344, "harmed": 935, "missed_rescue": 2283, "avoided_harm": 7667, "selected_both_correct": 5617, "selected_both_wrong": 1374}, {"budget": 0.51, "threshold": -0.8575630187988281, "operator": "ge", "actual_fraction": 0.24382497235062248, "metric": 0.6537447182599325, "rescued": 430, "harmed": 986, "missed_rescue": 2197, "avoided_harm": 7616, "selected_both_correct": 5653, "selected_both_wrong": 1529}, {"budget": 0.52, "threshold": -0.8823798894882202, "operator": "ge", "actual_fraction": 0.252644414825738, "metric": 0.6545387516660522, "rescued": 501, "harmed": 1029, "missed_rescue": 2126, "avoided_harm": 7573, "selected_both_correct": 5683, "selected_both_wrong": 1696}, {"budget": 0.53, "threshold": -0.9040287733078003, "operator": "ge", "actual_fraction": 0.26401610753480986, "metric": 0.6538581516036639, "rescued": 571, "harmed": 1123, "missed_rescue": 2056, "avoided_harm": 7479, "selected_both_correct": 5730, "selected_both_wrong": 1886}, {"budget": 0.54, "threshold": -0.9208687543869019, "operator": "ge", "actual_fraction": 0.2760116836344043, "metric": 0.6537163599239997, "rescued": 647, "harmed": 1204, "missed_rescue": 1980, "avoided_harm": 7398, "selected_both_correct": 5804, "selected_both_wrong": 2078}, {"budget": 0.55, "threshold": -0.9344452619552612, "operator": "ge", "actual_fraction": 0.2893401015228426, "metric": 0.6533477015568726, "rescued": 728, "harmed": 1298, "missed_rescue": 1899, "avoided_harm": 7304, "selected_both_correct": 5894, "selected_both_wrong": 2283}, {"budget": 0.56, "threshold": -0.9448322653770447, "operator": "ge", "actual_fraction": 0.3036043444970649, "metric": 0.6525536681507529, "rescued": 811, "harmed": 1409, "missed_rescue": 1816, "avoided_harm": 7193, "selected_both_correct": 5983, "selected_both_wrong": 2503}, {"budget": 0.57, "threshold": -0.9531905055046082, "operator": "ge", "actual_fraction": 0.3175850041119587, "metric": 0.6509088846666478, "rescued": 876, "harmed": 1532, "missed_rescue": 1751, "avoided_harm": 7070, "selected_both_correct": 6081, "selected_both_wrong": 2710}, {"budget": 0.58, "threshold": -0.9603281617164612, "operator": "gt", "actual_fraction": 0.33462836400760004, "metric": 0.6498312679011996, "rescued": 971, "harmed": 1665, "missed_rescue": 1656, "avoided_harm": 6937, "selected_both_correct": 6236, "selected_both_wrong": 2928}, {"budget": 0.59, "threshold": -0.9670801758766174, "operator": "ge", "actual_fraction": 0.35637920766809406, "metric": 0.6465984176048549, "rescued": 1048, "harmed": 1856, "missed_rescue": 1579, "avoided_harm": 6746, "selected_both_correct": 6453, "selected_both_wrong": 3210}, {"budget": 0.6, "threshold": -0.9724565148353577, "operator": "ge", "actual_fraction": 0.37968975980489467, "metric": 0.6442730340583614, "rescued": 1153, "harmed": 2043, "missed_rescue": 1474, "avoided_harm": 6559, "selected_both_correct": 6702, "selected_both_wrong": 3491}, {"budget": 0.61, "threshold": -0.9766654968261719, "operator": "ge", "actual_fraction": 0.4046450954258004, "metric": 0.6405580920511584, "rescued": 1230, "harmed": 2251, "missed_rescue": 1397, "avoided_harm": 6351, "selected_both_correct": 6991, "selected_both_wrong": 3797}, {"budget": 0.62, "threshold": -0.9801702499389648, "operator": "ge", "actual_fraction": 0.43141536454640844, "metric": 0.6357371749425744, "rescued": 1303, "harmed": 2494, "missed_rescue": 1324, "avoided_harm": 6108, "selected_both_correct": 7308, "selected_both_wrong": 4108}, {"budget": 0.63, "threshold": -0.9831460118293762, "operator": "ge", "actual_fraction": 0.4610214672603012, "metric": 0.630604316138729, "rescued": 1384, "harmed": 2756, "missed_rescue": 1243, "avoided_harm": 5846, "selected_both_correct": 7664, "selected_both_wrong": 4453}, {"budget": 0.64, "threshold": -0.985751748085022, "operator": "ge", "actual_fraction": 0.4916484700677764, "metric": 0.6244221989053682, "rescued": 1476, "harmed": 3066, "missed_rescue": 1151, "avoided_harm": 5536, "selected_both_correct": 8035, "selected_both_wrong": 4760}, {"budget": 0.65, "threshold": -0.9877713322639465, "operator": "ge", "actual_fraction": 0.5209426310864078, "metric": 0.6187788900547316, "rescued": 1563, "harmed": 3352, "missed_rescue": 1064, "avoided_harm": 5250, "selected_both_correct": 8439, "selected_both_wrong": 5016}, {"budget": 0.66, "threshold": -0.9895955324172974, "operator": "ge", "actual_fraction": 0.5529308340186598, "metric": 0.6118310977511839, "rescued": 1647, "harmed": 3681, "missed_rescue": 980, "avoided_harm": 4921, "selected_both_correct": 8872, "selected_both_wrong": 5298}, {"budget": 0.67, "threshold": -0.9913432002067566, "operator": "ge", "actual_fraction": 0.5888041289737118, "metric": 0.6045146470805093, "rescued": 1741, "harmed": 4033, "missed_rescue": 886, "avoided_harm": 4569, "selected_both_correct": 9334, "selected_both_wrong": 5655}, {"budget": 0.68, "threshold": -0.9925730228424072, "operator": "ge", "actual_fraction": 0.6175027649377535, "metric": 0.5983608881830814, "rescued": 1805, "harmed": 4314, "missed_rescue": 822, "avoided_harm": 4288, "selected_both_correct": 9696, "selected_both_wrong": 5960}, {"budget": 0.69, "threshold": -0.9935283660888672, "operator": "ge", "actual_fraction": 0.6421461588633979, "metric": 0.5925474293168477, "rescued": 1860, "harmed": 4574, "missed_rescue": 767, "avoided_harm": 4028, "selected_both_correct": 10008, "selected_both_wrong": 6202}, {"budget": 0.7, "threshold": -0.99436354637146, "operator": "gt", "actual_fraction": 0.665910444375124, "metric": 0.5877548705441965, "rescued": 1925, "harmed": 4808, "missed_rescue": 702, "avoided_harm": 3794, "selected_both_correct": 10303, "selected_both_wrong": 6446}, {"budget": 0.71, "threshold": -0.9949791431427002, "operator": "ge", "actual_fraction": 0.6838612710206166, "metric": 0.5829339534356124, "rescued": 1961, "harmed": 5014, "missed_rescue": 666, "avoided_harm": 3588, "selected_both_correct": 10525, "selected_both_wrong": 6615}, {"budget": 0.72, "threshold": -0.9955723881721497, "operator": "ge", "actual_fraction": 0.7029464311034228, "metric": 0.5779996029832969, "rescued": 2000, "harmed": 5227, "missed_rescue": 627, "avoided_harm": 3375, "selected_both_correct": 10747, "selected_both_wrong": 6814}, {"budget": 0.73, "threshold": -0.9960037469863892, "operator": "ge", "actual_fraction": 0.7180614241556306, "metric": 0.5745966026713553, "rescued": 2042, "harmed": 5389, "missed_rescue": 585, "avoided_harm": 3213, "selected_both_correct": 10923, "selected_both_wrong": 6967}, {"budget": 0.74, "threshold": -0.9964693188667297, "operator": "ge", "actual_fraction": 0.7352465757309361, "metric": 0.5710234523438165, "rescued": 2087, "harmed": 5560, "missed_rescue": 540, "avoided_harm": 3042, "selected_both_correct": 11139, "selected_both_wrong": 7141}, {"budget": 0.75, "threshold": -0.9968041777610779, "operator": "ge", "actual_fraction": 0.7478093185491875, "metric": 0.567762243711539, "rescued": 2117, "harmed": 5705, "missed_rescue": 510, "avoided_harm": 2897, "selected_both_correct": 11290, "selected_both_wrong": 7258}, {"budget": 0.76, "threshold": -0.9971042275428772, "operator": "ge", "actual_fraction": 0.759804894648782, "metric": 0.565153276805717, "rescued": 2149, "harmed": 5829, "missed_rescue": 478, "avoided_harm": 2773, "selected_both_correct": 11421, "selected_both_wrong": 7394}, {"budget": 0.77, "threshold": -0.9974007606506348, "operator": "ge", "actual_fraction": 0.7729064458497575, "metric": 0.5620622181890367, "rescued": 2183, "harmed": 5972, "missed_rescue": 444, "avoided_harm": 2630, "selected_both_correct": 11577, "selected_both_wrong": 7523}, {"budget": 0.78, "threshold": -0.9976577162742615, "operator": "ge", "actual_fraction": 0.7855259053398747, "metric": 0.5588860845645578, "rescued": 2221, "harmed": 6122, "missed_rescue": 406, "avoided_harm": 2480, "selected_both_correct": 11724, "selected_both_wrong": 7633}, {"budget": 0.79, "threshold": -0.9978766441345215, "operator": "ge", "actual_fraction": 0.7964722230099538, "metric": 0.5562771176587358, "rescued": 2244, "harmed": 6237, "missed_rescue": 383, "avoided_harm": 2365, "selected_both_correct": 11855, "selected_both_wrong": 7750}, {"budget": 0.8, "threshold": -0.9980920553207397, "operator": "ge", "actual_fraction": 0.8082125740861527, "metric": 0.5536114340810481, "rescued": 2274, "harmed": 6361, "missed_rescue": 353, "avoided_harm": 2241, "selected_both_correct": 12000, "selected_both_wrong": 7865}, {"budget": 0.81, "threshold": -0.9982843399047852, "operator": "ge", "actual_fraction": 0.8197544168108215, "metric": 0.5504920171284349, "rescued": 2300, "harmed": 6497, "missed_rescue": 327, "avoided_harm": 2105, "selected_both_correct": 12140, "selected_both_wrong": 7970}, {"budget": 0.82, "threshold": -0.9984900951385498, "operator": "ge", "actual_fraction": 0.8333380597226555, "metric": 0.5469188668008961, "rescued": 2329, "harmed": 6652, "missed_rescue": 298, "avoided_harm": 1950, "selected_both_correct": 12321, "selected_both_wrong": 8084}, {"budget": 0.83, "threshold": -0.9986395835876465, "operator": "ge", "actual_fraction": 0.8426112355726966, "metric": 0.5450188582933954, "rescued": 2356, "harmed": 6746, "missed_rescue": 271, "avoided_harm": 1856, "selected_both_correct": 12437, "selected_both_wrong": 8174}, {"budget": 0.84, "threshold": -0.9987816214561462, "operator": "ge", "actual_fraction": 0.8529903865241187, "metric": 0.5423531747157077, "rescued": 2381, "harmed": 6865, "missed_rescue": 246, "avoided_harm": 1737, "selected_both_correct": 12586, "selected_both_wrong": 8247}, {"budget": 0.85, "threshold": -0.9989162087440491, "operator": "ge", "actual_fraction": 0.8635396874911381, "metric": 0.5395456994583557, "rescued": 2414, "harmed": 6997, "missed_rescue": 213, "avoided_harm": 1605, "selected_both_correct": 12707, "selected_both_wrong": 8333}, {"budget": 0.86, "threshold": -0.9990254044532776, "operator": "ge", "actual_fraction": 0.8710262881774098, "metric": 0.5378725576383178, "rescued": 2435, "harmed": 7077, "missed_rescue": 192, "avoided_harm": 1525, "selected_both_correct": 12808, "selected_both_wrong": 8395}, {"budget": 0.87, "threshold": -0.9991487264633179, "operator": "ge", "actual_fraction": 0.8809517057539064, "metric": 0.535774040779287, "rescued": 2462, "harmed": 7178, "missed_rescue": 165, "avoided_harm": 1424, "selected_both_correct": 12942, "selected_both_wrong": 8483}, {"budget": 0.88, "threshold": -0.9992602467536926, "operator": "ge", "actual_fraction": 0.8913024983693957, "metric": 0.5330516405297338, "rescued": 2480, "harmed": 7292, "missed_rescue": 147, "avoided_harm": 1310, "selected_both_correct": 13082, "selected_both_wrong": 8576}, {"budget": 0.89, "threshold": -0.999350905418396, "operator": "ge", "actual_fraction": 0.900915974250631, "metric": 0.5304710319598446, "rescued": 2498, "harmed": 7401, "missed_rescue": 129, "avoided_harm": 1201, "selected_both_correct": 13224, "selected_both_wrong": 8646}, {"budget": 0.9, "threshold": -0.999435544013977, "operator": "ge", "actual_fraction": 0.910954825170859, "metric": 0.5277486317102913, "rescued": 2518, "harmed": 7517, "missed_rescue": 109, "avoided_harm": 1085, "selected_both_correct": 13379, "selected_both_wrong": 8709}, {"budget": 0.91, "threshold": -0.9995256066322327, "operator": "ge", "actual_fraction": 0.921901142840938, "metric": 0.5245724980858123, "rescued": 2542, "harmed": 7653, "missed_rescue": 85, "avoided_harm": 949, "selected_both_correct": 13541, "selected_both_wrong": 8773}, {"budget": 0.92, "threshold": -0.9995898604393005, "operator": "ge", "actual_fraction": 0.9303235686129938, "metric": 0.5217366644925275, "rescued": 2553, "harmed": 7764, "missed_rescue": 74, "avoided_harm": 838, "selected_both_correct": 13673, "selected_both_wrong": 8816}, {"budget": 0.93, "threshold": -0.9996523857116699, "operator": "ge", "actual_fraction": 0.9393698777755721, "metric": 0.5187306808836457, "rescued": 2567, "harmed": 7884, "missed_rescue": 60, "avoided_harm": 718, "selected_both_correct": 13819, "selected_both_wrong": 8855}, {"budget": 0.94, "threshold": -0.9997095465660095, "operator": "ge", "actual_fraction": 0.9490117119927403, "metric": 0.515526188923234, "rescued": 2578, "harmed": 8008, "missed_rescue": 49, "avoided_harm": 594, "selected_both_correct": 13977, "selected_both_wrong": 8902}, {"budget": 0.95, "threshold": -0.9997634887695312, "operator": "ge", "actual_fraction": 0.9587953378895726, "metric": 0.5122366219550237, "rescued": 2588, "harmed": 8134, "missed_rescue": 39, "avoided_harm": 468, "selected_both_correct": 14140, "selected_both_wrong": 8948}, {"budget": 0.96, "threshold": -0.9998065829277039, "operator": "ge", "actual_fraction": 0.9673595553412926, "metric": 0.5097694467288659, "rescued": 2604, "harmed": 8237, "missed_rescue": 23, "avoided_harm": 365, "selected_both_correct": 14295, "selected_both_wrong": 8976}, {"budget": 0.97, "threshold": -0.9998431205749512, "operator": "ge", "actual_fraction": 0.9746192893401016, "metric": 0.5075574965261038, "rescued": 2615, "harmed": 8326, "missed_rescue": 12, "avoided_harm": 276, "selected_both_correct": 14432, "selected_both_wrong": 8995}, {"budget": 0.98, "threshold": -0.9998764991760254, "operator": "ge", "actual_fraction": 0.9830984317840229, "metric": 0.5046365879250205, "rescued": 2623, "harmed": 8437, "missed_rescue": 4, "avoided_harm": 165, "selected_both_correct": 14600, "selected_both_wrong": 9007}, {"budget": 0.99, "threshold": -0.9999123811721802, "operator": "ge", "actual_fraction": 0.9908118991577575, "metric": 0.5019425460114001, "rescued": 2626, "harmed": 8535, "missed_rescue": 1, "avoided_harm": 67, "selected_both_correct": 14761, "selected_both_wrong": 9017}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.5000708958398321, "rescued": 2627, "harmed": 8602, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 15007, "selected_both_wrong": 9027}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.7566800196781222, "actual_fraction": 0.14395951929158762, "metric_std": 0.003176346811551161, "actual_fraction_std": 0.1433058123344503, "per_seed": [{"seed": 2026, "budget": 0.32, "threshold": -0.617985546511672, "operator": "ge", "alpha": 0.75, "epoch": 3, "metric": 0.7621055590695058, "actual_fraction": 0.3199803218778551, "rescue": 358, "harm": 190, "missed_rescue": 1011}, {"seed": 2027, "budget": 0.28, "threshold": -0.5145840417649868, "operator": "ge", "alpha": 0.75, "epoch": 3, "metric": 0.7549371002881441, "actual_fraction": 0.27999156651908075, "rescue": 341, "harm": 275, "missed_rescue": 1028}, {"seed": 2028, "budget": 0.03, "threshold": -0.011228500772213757, "operator": "ge", "alpha": 0.75, "epoch": 1, "metric": 0.7565535174643334, "actual_fraction": 0.02993885726333544, "rescue": 95, "harm": 6, "missed_rescue": 1274}, {"seed": 2029, "budget": 0.05, "threshold": 0.0020421056906284726, "operator": "gt", "alpha": 0.75, "epoch": 3, "metric": 0.7540234731885586, "actual_fraction": 0.049898095438892405, "rescue": 119, "harm": 66, "missed_rescue": 1250}, {"seed": 2030, "budget": 0.04, "threshold": -0.02570354767256211, "operator": "ge", "alpha": 0.75, "epoch": 4, "metric": 0.7557804483800689, "actual_fraction": 0.03998875535877434, "rescue": 103, "harm": 25, "missed_rescue": 1266}], "rescued": 203.2, "harmed": 112.4, "missed_rescue": 1165.8, "avoided_harm": 2808.6}, "validation_curve": [{"budget": 0.0, "metric": 0.7502986857825568, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.7518167123480217, "metric_std": 0.0014962993254720382, "actual_fraction": 0.009979619087778481, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.7530957902874411, "metric_std": 0.0021492051358265626, "actual_fraction": 0.019959238175556962, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.7541780870054116, "metric_std": 0.0025694643393775036, "actual_fraction": 0.02993885726333544, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.7544873146391173, "metric_std": 0.0019320976150345279, "actual_fraction": 0.03998875535877434, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.7543045892192003, "metric_std": 0.0011386500629713549, "actual_fraction": 0.04995431864502072, "actual_fraction_std": 3.142972770397638e-05}, {"budget": 0.06, "metric": 0.7535174643334036, "metric_std": 0.0016581974054682247, "actual_fraction": 0.05994799353433129, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.7529552322721205, "metric_std": 0.002203669980717225, "actual_fraction": 0.06999789162977019, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.7529973996767165, "metric_std": 0.0024062735823794884, "actual_fraction": 0.07997751071754867, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.7530255112797808, "metric_std": 0.0025689837332832253, "actual_fraction": 0.08995712980532716, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.7533487947150187, "metric_std": 0.0024149814148473566, "actual_fraction": 0.09993674889310561, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.7535315201349357, "metric_std": 0.002171268775848654, "actual_fraction": 0.10998664698854452, "actual_fraction_std": 0.0}, {"budget": 0.12, "metric": 0.753911026776302, "metric_std": 0.0022294069823416847, "actual_fraction": 0.11996626607632302, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.13, "metric": 0.7539813057839624, "metric_std": 0.00215792033486824, "actual_fraction": 0.1299458851641015, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.7540375289900907, "metric_std": 0.0022176340140758204, "actual_fraction": 0.13998172745800827, "actual_fraction_std": 3.142972770397948e-05}, {"budget": 0.15, "metric": 0.7541640312038795, "metric_std": 0.0021042666275324926, "actual_fraction": 0.14997540234731885, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.7542764776161361, "metric_std": 0.0023747650023247515, "actual_fraction": 0.15995502143509735, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.7544310914329891, "metric_std": 0.002459464053924464, "actual_fraction": 0.16993464052287582, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.7546559842575022, "metric_std": 0.0025030576735551417, "actual_fraction": 0.1799845386183147, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.7545857052498418, "metric_std": 0.0027241615634709887, "actual_fraction": 0.18996415770609318, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.7548668212804835, "metric_std": 0.002676144882234864, "actual_fraction": 0.19994377679387168, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.7549933234942723, "metric_std": 0.0027901739662840957, "actual_fraction": 0.2099936748893106, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.7550917141049969, "metric_std": 0.0029723118581581704, "actual_fraction": 0.21997329397708903, "actual_fraction_std": 0.0}, {"budget": 0.23, "metric": 0.7552463279218498, "metric_std": 0.0027294144481217033, "actual_fraction": 0.22995291306486756, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.24, "metric": 0.7552463279218498, "metric_std": 0.003054752879902307, "actual_fraction": 0.23993253215264604, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.25, "metric": 0.7553306627310423, "metric_std": 0.0032123734124192248, "actual_fraction": 0.2499683744465528, "actual_fraction_std": 3.142972770397948e-05}, {"budget": 0.26, "metric": 0.7552463279218499, "metric_std": 0.003179144533888868, "actual_fraction": 0.25996204933586337, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.7550776583034647, "metric_std": 0.003054510339742882, "actual_fraction": 0.26994166842364187, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.7550917141049969, "metric_std": 0.0031603681303667627, "actual_fraction": 0.27999156651908075, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.7552182163187857, "metric_std": 0.003177745980613287, "actual_fraction": 0.28997118560685925, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.7552182163187856, "metric_std": 0.003381435581901547, "actual_fraction": 0.2999508046946377, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.7551479373111252, "metric_std": 0.003748579760515731, "actual_fraction": 0.3099304237824162, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.7552041605172535, "metric_std": 0.0038635560869708842, "actual_fraction": 0.3199803218778551, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.7552041605172535, "metric_std": 0.003838546498269714, "actual_fraction": 0.3299599409656336, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.754951156089676, "metric_std": 0.003911201996317789, "actual_fraction": 0.339939560053412, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.7545435378452455, "metric_std": 0.0037282997004229484, "actual_fraction": 0.34998945814885096, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.7548527654789515, "metric_std": 0.003910254767610842, "actual_fraction": 0.3599690772366294, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.7545013704406494, "metric_std": 0.003762848593215405, "actual_fraction": 0.3699486963244079, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.7544592030360532, "metric_std": 0.003584711665150221, "actual_fraction": 0.37999859441984685, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.7543045892192002, "metric_std": 0.0036494366814160614, "actual_fraction": 0.3899782135076253, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.754107807997751, "metric_std": 0.003640086259225188, "actual_fraction": 0.39995783259540374, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.7540515847916228, "metric_std": 0.0033308839293649988, "actual_fraction": 0.4099374516831823, "actual_fraction_std": 0.0}, {"budget": 0.42, "metric": 0.7539531941808981, "metric_std": 0.003341764966749262, "actual_fraction": 0.4199873497786212, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.7536861339517886, "metric_std": 0.0031399855266508853, "actual_fraction": 0.4299669688663997, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.44, "metric": 0.7533066273104224, "metric_std": 0.0030707985967390423, "actual_fraction": 0.43994658795417807, "actual_fraction_std": 0.0}, {"budget": 0.45, "metric": 0.7527022278445428, "metric_std": 0.0030143878096475065, "actual_fraction": 0.44999648604961695, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.46, "metric": 0.7526460046384146, "metric_std": 0.0030897196798726954, "actual_fraction": 0.4599761051373955, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.47, "metric": 0.7519151029587462, "metric_std": 0.003233980002767178, "actual_fraction": 0.4699557242251739, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.48, "metric": 0.7515496521189121, "metric_std": 0.003111146177388658, "actual_fraction": 0.47993534331295234, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.7510155316606929, "metric_std": 0.0028961432475091546, "actual_fraction": 0.4899852414083913, "actual_fraction_std": 0.0}, {"budget": 0.5, "metric": 0.7505938576147304, "metric_std": 0.0029409896417248223, "actual_fraction": 0.4999508046946377, "actual_fraction_std": 3.142972770396707e-05}, {"budget": 0.51, "metric": 0.7502002951718322, "metric_std": 0.003232681568317226, "actual_fraction": 0.5099444795839483, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.7497926769274018, "metric_std": 0.003181784568394876, "actual_fraction": 0.5199943776793872, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.7493147796753109, "metric_std": 0.002991195438193848, "actual_fraction": 0.5299739967671656, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.7488649940262844, "metric_std": 0.0028885441616366106, "actual_fraction": 0.5399536158549442, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.7480216459343595, "metric_std": 0.002947783421143101, "actual_fraction": 0.5499332349427226, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.7477405299037179, "metric_std": 0.003226181557404566, "actual_fraction": 0.5599831330381615, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.747276688453159, "metric_std": 0.003393828510127117, "actual_fraction": 0.5699627521259399, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.7466020099796191, "metric_std": 0.003110431692234, "actual_fraction": 0.5799423712137185, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.745899219903015, "metric_std": 0.0030087296004160176, "actual_fraction": 0.5899922693091574, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.7450699276126223, "metric_std": 0.002577716738903816, "actual_fraction": 0.5999718883969358, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.7445217513528709, "metric_std": 0.002667548957147145, "actual_fraction": 0.6099515074847143, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.7438751844823951, "metric_std": 0.0031068568018053225, "actual_fraction": 0.6199311265724928, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.7432567292149835, "metric_std": 0.003121052670742916, "actual_fraction": 0.6299810246679317, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.7424696043291868, "metric_std": 0.00276224216898386, "actual_fraction": 0.6399606437557102, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.7420198186801603, "metric_std": 0.002685081205143302, "actual_fraction": 0.6499402628434886, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.7409234661606579, "metric_std": 0.0026015576216890293, "actual_fraction": 0.6599901609389276, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.7393632721905967, "metric_std": 0.002910263805246433, "actual_fraction": 0.669969780026706, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.738210696464966, "metric_std": 0.002550944045664694, "actual_fraction": 0.6799493991144846, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.737353292571509, "metric_std": 0.0024669833753857276, "actual_fraction": 0.6899992972099234, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.7367067257010331, "metric_std": 0.0021653462943841775, "actual_fraction": 0.6999508046946377, "actual_fraction_std": 3.8493397814698215e-05}, {"budget": 0.71, "metric": 0.7357228195937874, "metric_std": 0.0027733952182450094, "actual_fraction": 0.7099585353854804, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.7340220676084054, "metric_std": 0.0028437391142653837, "actual_fraction": 0.7199381544732588, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.7321948134092346, "metric_std": 0.003129823352129992, "actual_fraction": 0.7299880525686977, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.730381615011596, "metric_std": 0.0037253177758473214, "actual_fraction": 0.7399676716564763, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.7287370862323425, "metric_std": 0.004190715481845605, "actual_fraction": 0.7499472907442546, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.7271628364607492, "metric_std": 0.004622521258565811, "actual_fraction": 0.7599971888396937, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.7255885866891558, "metric_std": 0.004322758872231455, "actual_fraction": 0.7699486963244079, "actual_fraction_std": 3.8493397814637405e-05}, {"budget": 0.78, "metric": 0.7236207744746644, "metric_std": 0.004760488535458411, "actual_fraction": 0.7799564270152506, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.72183568768009, "metric_std": 0.004955542220480493, "actual_fraction": 0.7899360461030291, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.7195164804272963, "metric_std": 0.0053024568163437735, "actual_fraction": 0.7999859441984679, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.7174081101974841, "metric_std": 0.0056242968625967735, "actual_fraction": 0.8099655632862464, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.7148077869140488, "metric_std": 0.00590490060794687, "actual_fraction": 0.8199451823740249, "actual_fraction_std": 0.0}, {"budget": 0.83, "metric": 0.7122074636306135, "metric_std": 0.006009574341536832, "actual_fraction": 0.8299950804694639, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.84, "metric": 0.7090167966828308, "metric_std": 0.006320931612287248, "actual_fraction": 0.8399746995572424, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.7061494131702861, "metric_std": 0.006162030368137583, "actual_fraction": 0.8499543186450207, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.7035069224822547, "metric_std": 0.005921272281807843, "actual_fraction": 0.8599198819312672, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.87, "metric": 0.700498980954389, "metric_std": 0.005374069872420316, "actual_fraction": 0.869969780026706, "actual_fraction_std": 3.1429727703942244e-05}, {"budget": 0.88, "metric": 0.6970693653805609, "metric_std": 0.005038715687436518, "actual_fraction": 0.8799634549160166, "actual_fraction_std": 0.0}, {"budget": 0.89, "metric": 0.6934851359898797, "metric_std": 0.004479039456852837, "actual_fraction": 0.889943074003795, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.6897322369808139, "metric_std": 0.0034074086713972904, "actual_fraction": 0.8999929720992339, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.91, "metric": 0.6851078782767587, "metric_std": 0.003466401182647379, "actual_fraction": 0.9099725911870123, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.6813830908707569, "metric_std": 0.00337873227417546, "actual_fraction": 0.919952210274791, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.93, "metric": 0.6770960714034718, "metric_std": 0.003263852756753131, "actual_fraction": 0.9299318293625692, "actual_fraction_std": 0.0}, {"budget": 0.94, "metric": 0.6726544381193338, "metric_std": 0.0030053624305326205, "actual_fraction": 0.9399817274580083, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.6683533628505165, "metric_std": 0.0028308572719267385, "actual_fraction": 0.9499472907442547, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.96, "metric": 0.6636446693372691, "metric_std": 0.002178649237472756, "actual_fraction": 0.9599409656335652, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.6585986365872515, "metric_std": 0.001897142727722813, "actual_fraction": 0.9699908637290042, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.98, "metric": 0.6541851149061776, "metric_std": 0.0013377032667577864, "actual_fraction": 0.9799564270152505, "actual_fraction_std": 3.1429727703942244e-05}, {"budget": 0.99, "metric": 0.6481832876519784, "metric_std": 0.0008059136229657814, "actual_fraction": 0.9899360461030291, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 1.0, "metric": 0.6412256658935976, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.6791367722542041, "actual_fraction": 0.19038085245157815, "metric_std": 0.001853581189232703, "actual_fraction_std": 0.1267464230826633, "per_seed": [{"seed": 2026, "budget": 0.32, "threshold": -0.617985546511672, "operator": "ge", "alpha": 0.75, "epoch": 3, "metric": 0.6792388622635623, "actual_fraction": 0.33170745540651675, "rescue": 1001, "harm": 658, "missed_rescue": 1626}, {"seed": 2027, "budget": 0.28, "threshold": -0.5145840417649868, "operator": "ge", "alpha": 0.75, "epoch": 3, "metric": 0.6762612369906134, "actual_fraction": 0.3264895215948728, "rescue": 1052, "harm": 814, "missed_rescue": 1575}, {"seed": 2028, "budget": 0.03, "threshold": -0.011228500772213757, "operator": "ge", "alpha": 0.75, "epoch": 1, "metric": 0.6786716955449053, "actual_fraction": 0.08969741655559652, "rescue": 458, "harm": 135, "missed_rescue": 2169}, {"seed": 2029, "budget": 0.05, "threshold": 0.0020421056906284726, "operator": "gt", "alpha": 0.75, "epoch": 3, "metric": 0.6807985707398688, "actual_fraction": 0.1009840342568698, "rescue": 582, "harm": 184, "missed_rescue": 2045}, {"seed": 2030, "budget": 0.04, "threshold": -0.02570354767256211, "operator": "ge", "alpha": 0.75, "epoch": 4, "metric": 0.6807134957320704, "actual_fraction": 0.10302583444403482, "rescue": 545, "harm": 150, "missed_rescue": 2082}], "rescued": 727.6, "harmed": 388.2, "missed_rescue": 1899.4, "avoided_harm": 8213.8}, "test_curve": [{"budget": 0.0, "metric": 0.6695119530385957, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.6736239117488585, "metric_std": 0.0012217145012357968, "actual_fraction": 0.020332926863851627, "actual_fraction_std": 0.007757521148664225}, {"budget": 0.02, "metric": 0.6766298953577403, "metric_std": 0.0013258848909691505, "actual_fraction": 0.04462467742392876, "actual_fraction_std": 0.011769283398187027}, {"budget": 0.03, "metric": 0.6786603522105323, "metric_std": 0.0009584530224493218, "actual_fraction": 0.07091285483367836, "actual_fraction_std": 0.017046032730533737}, {"budget": 0.04, "metric": 0.6797663273119132, "metric_std": 0.0005936992197726307, "actual_fraction": 0.09045174829141027, "actual_fraction_std": 0.015691279990398856}, {"budget": 0.05, "metric": 0.6804072257039957, "metric_std": 0.00034985498711487224, "actual_fraction": 0.11004735842100785, "actual_fraction_std": 0.006786968019007509}, {"budget": 0.06, "metric": 0.6805717040524062, "metric_std": 0.0005139826152168853, "actual_fraction": 0.12548563650285, "actual_fraction_std": 0.005940377607248918}, {"budget": 0.07, "metric": 0.6804015540368091, "metric_std": 0.0005707005018716308, "actual_fraction": 0.13746986926807134, "actual_fraction_std": 0.006909041666318466}, {"budget": 0.08, "metric": 0.6803391656977569, "metric_std": 0.000768139330216263, "actual_fraction": 0.15023112043785272, "actual_fraction_std": 0.010144966284797581}, {"budget": 0.09, "metric": 0.6802767773587046, "metric_std": 0.0008079886614695994, "actual_fraction": 0.16403595836996285, "actual_fraction_std": 0.010543440903610113}, {"budget": 0.1, "metric": 0.6802881206930778, "metric_std": 0.0009862424219311545, "actual_fraction": 0.1779372146442447, "actual_fraction_std": 0.008464110582874215}, {"budget": 0.11, "metric": 0.6802824490258912, "metric_std": 0.0010639857173864841, "actual_fraction": 0.18960383404701814, "actual_fraction_std": 0.007059203829865779}, {"budget": 0.12, "metric": 0.6801293140118538, "metric_std": 0.0011508034945890183, "actual_fraction": 0.2002041800187165, "actual_fraction_std": 0.005919132686066076}, {"budget": 0.13, "metric": 0.6800839406743613, "metric_std": 0.001201537037857467, "actual_fraction": 0.2103564642826759, "actual_fraction_std": 0.005864055441009514}, {"budget": 0.14, "metric": 0.6799818506650028, "metric_std": 0.0012635221978859702, "actual_fraction": 0.21922695176247053, "actual_fraction_std": 0.006065257394334894}, {"budget": 0.15, "metric": 0.6798684173212717, "metric_std": 0.001277762548093442, "actual_fraction": 0.2273317641720784, "actual_fraction_std": 0.007193090651604836}, {"budget": 0.16, "metric": 0.6796925956384878, "metric_std": 0.00139850018500276, "actual_fraction": 0.2348580665286561, "actual_fraction_std": 0.007301415195949497}, {"budget": 0.17, "metric": 0.6796018489635027, "metric_std": 0.0014572708427796155, "actual_fraction": 0.24202705385247994, "actual_fraction_std": 0.007701014058382026}, {"budget": 0.18, "metric": 0.6794316989479057, "metric_std": 0.0016751113248791808, "actual_fraction": 0.24939454952783371, "actual_fraction_std": 0.008078308902113232}, {"budget": 0.19, "metric": 0.6793693106088534, "metric_std": 0.0015190382930639185, "actual_fraction": 0.25641040183762015, "actual_fraction_std": 0.008130201872973203}, {"budget": 0.2, "metric": 0.6792615489323086, "metric_std": 0.0015934255987445649, "actual_fraction": 0.2633184924708618, "actual_fraction_std": 0.008784165641960402}, {"budget": 0.21, "metric": 0.6791027422510847, "metric_std": 0.0016228056188951204, "actual_fraction": 0.2692510563480135, "actual_fraction_std": 0.009265467330671778}, {"budget": 0.22, "metric": 0.6789496072370473, "metric_std": 0.0016038638471624527, "actual_fraction": 0.27534242690638916, "actual_fraction_std": 0.009714367250674856}, {"budget": 0.23, "metric": 0.6787681138870771, "metric_std": 0.0015904198237112024, "actual_fraction": 0.2815245441397499, "actual_fraction_std": 0.010395420997827}, {"budget": 0.24, "metric": 0.6786490088761592, "metric_std": 0.0015622071644523974, "actual_fraction": 0.28715650965601336, "actual_fraction_std": 0.010451499237374571}, {"budget": 0.25, "metric": 0.6784902021949353, "metric_std": 0.0016493730868613046, "actual_fraction": 0.29289056518163514, "actual_fraction_std": 0.010755651764764076}, {"budget": 0.26, "metric": 0.6783540821824576, "metric_std": 0.0017545997062751318, "actual_fraction": 0.2986983523806823, "actual_fraction_std": 0.011484828461471926}, {"budget": 0.27, "metric": 0.6781215438278082, "metric_std": 0.0017887794522866427, "actual_fraction": 0.30438703456881144, "actual_fraction_std": 0.011973408628346779}, {"budget": 0.28, "metric": 0.6777301987919349, "metric_std": 0.0016981174250232183, "actual_fraction": 0.31002467175226156, "actual_fraction_std": 0.012752734525216363}, {"budget": 0.29, "metric": 0.6775146754388452, "metric_std": 0.0015893829039171302, "actual_fraction": 0.31561126393103256, "actual_fraction_std": 0.012872085778165118}, {"budget": 0.3, "metric": 0.6773048237529421, "metric_std": 0.0016584657008131838, "actual_fraction": 0.3206477043927063, "actual_fraction_std": 0.013041643826102175}, {"budget": 0.31, "metric": 0.6772084054107704, "metric_std": 0.0015048099187908164, "actual_fraction": 0.3267560899526416, "actual_fraction_std": 0.0127435559033785}, {"budget": 0.32, "metric": 0.6769191503842554, "metric_std": 0.0014708657221007941, "actual_fraction": 0.3326829821626067, "actual_fraction_std": 0.013201964331969067}, {"budget": 0.33, "metric": 0.6765788503530613, "metric_std": 0.0014320819257598292, "actual_fraction": 0.33824688767263134, "actual_fraction_std": 0.013896027337327784}, {"budget": 0.34, "metric": 0.6762782519921732, "metric_std": 0.0014730783936056794, "actual_fraction": 0.3451209483027536, "actual_fraction_std": 0.013992159281335926}, {"budget": 0.35, "metric": 0.6760173553015909, "metric_std": 0.0016355410572372695, "actual_fraction": 0.3511272438533307, "actual_fraction_std": 0.014260259283880656}, {"budget": 0.36, "metric": 0.6754955619204265, "metric_std": 0.001751342501391816, "actual_fraction": 0.35775175112724383, "actual_fraction_std": 0.014157494140310466}, {"budget": 0.37, "metric": 0.6750021268751949, "metric_std": 0.0017974129867042348, "actual_fraction": 0.36507387346510506, "actual_fraction_std": 0.014843341963049443}, {"budget": 0.38, "metric": 0.6744406318237245, "metric_std": 0.0017097983660078237, "actual_fraction": 0.3725207724810708, "actual_fraction_std": 0.014446469480468472}, {"budget": 0.39, "metric": 0.6740152567847318, "metric_std": 0.0015652928056018982, "actual_fraction": 0.37905453307999887, "actual_fraction_std": 0.014057965380366374}, {"budget": 0.4, "metric": 0.6733346567223435, "metric_std": 0.001494594611902563, "actual_fraction": 0.38662053710688254, "actual_fraction_std": 0.014352254394619535}, {"budget": 0.41, "metric": 0.6728752516802314, "metric_std": 0.0015259051121983985, "actual_fraction": 0.39437937781810967, "actual_fraction_std": 0.014722742946863546}, {"budget": 0.42, "metric": 0.6721946516178432, "metric_std": 0.0016874520508534593, "actual_fraction": 0.40177523182939623, "actual_fraction_std": 0.01506136557332565}, {"budget": 0.43, "metric": 0.6716388282335591, "metric_std": 0.0016803360992570851, "actual_fraction": 0.4089045174829141, "actual_fraction_std": 0.015142480432927263}, {"budget": 0.44, "metric": 0.6709298698352381, "metric_std": 0.0018169263510922039, "actual_fraction": 0.4166690298613277, "actual_fraction_std": 0.014876075947167118}, {"budget": 0.45, "metric": 0.670351359782208, "metric_std": 0.0016734542132194857, "actual_fraction": 0.42493832061934605, "actual_fraction_std": 0.014562462486030828}, {"budget": 0.46, "metric": 0.6695743413776479, "metric_std": 0.0016395680380714972, "actual_fraction": 0.43286731134617024, "actual_fraction_std": 0.014137894726986918}, {"budget": 0.47, "metric": 0.6687065762981028, "metric_std": 0.0015613317907892449, "actual_fraction": 0.4409891387573377, "actual_fraction_std": 0.014066000479579212}, {"budget": 0.48, "metric": 0.6676913478717068, "metric_std": 0.0016203259315794802, "actual_fraction": 0.4495646995434308, "actual_fraction_std": 0.01448876477589292}, {"budget": 0.49, "metric": 0.6667838811218558, "metric_std": 0.0019899794187281453, "actual_fraction": 0.45795309531236705, "actual_fraction_std": 0.014357657729073072}, {"budget": 0.5, "metric": 0.665910444375124, "metric_std": 0.002158682279589188, "actual_fraction": 0.46621671440319884, "actual_fraction_std": 0.014630199905532477}, {"budget": 0.51, "metric": 0.6649065592831013, "metric_std": 0.0020266391467580976, "actual_fraction": 0.47530839690326976, "actual_fraction_std": 0.01500628178078082}, {"budget": 0.52, "metric": 0.6638005841817202, "metric_std": 0.00203455991306422, "actual_fraction": 0.4846496327595496, "actual_fraction_std": 0.014558096386315987}, {"budget": 0.53, "metric": 0.6626946090803392, "metric_std": 0.0021030722416973315, "actual_fraction": 0.4937072852565011, "actual_fraction_std": 0.014503266459337364}, {"budget": 0.54, "metric": 0.6615659473102118, "metric_std": 0.002412576158634996, "actual_fraction": 0.5040467345376174, "actual_fraction_std": 0.014615185893016426}, {"budget": 0.55, "metric": 0.6605280322150696, "metric_std": 0.0023247263702377203, "actual_fraction": 0.5130249836939569, "actual_fraction_std": 0.014026564446105455}, {"budget": 0.56, "metric": 0.6591214587528004, "metric_std": 0.0023443786747717296, "actual_fraction": 0.5229674162720132, "actual_fraction_std": 0.013513522217293778}, {"budget": 0.57, "metric": 0.6579020503076879, "metric_std": 0.0023635954249031925, "actual_fraction": 0.5324504438079574, "actual_fraction_std": 0.013200834313786249}, {"budget": 0.58, "metric": 0.6562969684938889, "metric_std": 0.002475635625015206, "actual_fraction": 0.5418654113376628, "actual_fraction_std": 0.012887926168840554}, {"budget": 0.59, "metric": 0.6548733800300598, "metric_std": 0.002704722283228844, "actual_fraction": 0.5520120239344355, "actual_fraction_std": 0.012914546219988637}, {"budget": 0.6, "metric": 0.6530811331991039, "metric_std": 0.0025790187668516635, "actual_fraction": 0.5628449082607834, "actual_fraction_std": 0.01185423556668943}, {"budget": 0.61, "metric": 0.6512435130306553, "metric_std": 0.0026337704308165857, "actual_fraction": 0.5729631625216232, "actual_fraction_std": 0.011530385705309431}, {"budget": 0.62, "metric": 0.6493945495278337, "metric_std": 0.002848679184200044, "actual_fraction": 0.5823667867169554, "actual_fraction_std": 0.011143563174287817}, {"budget": 0.63, "metric": 0.647755437710915, "metric_std": 0.0030781141607676135, "actual_fraction": 0.5923942943028104, "actual_fraction_std": 0.011545911115296374}, {"budget": 0.64, "metric": 0.6456966225221904, "metric_std": 0.003257038523706834, "actual_fraction": 0.6029662819385759, "actual_fraction_std": 0.011817885258729831}, {"budget": 0.65, "metric": 0.6438760173553015, "metric_std": 0.003216202881834712, "actual_fraction": 0.6129314011853785, "actual_fraction_std": 0.012131128004008895}, {"budget": 0.66, "metric": 0.641459887133823, "metric_std": 0.003108596237061996, "actual_fraction": 0.6240138388679352, "actual_fraction_std": 0.01138071641455083}, {"budget": 0.67, "metric": 0.6394010719450983, "metric_std": 0.0033686432650003044, "actual_fraction": 0.6343873181521709, "actual_fraction_std": 0.011933327934003009}, {"budget": 0.68, "metric": 0.6372458384142019, "metric_std": 0.0034978257205084858, "actual_fraction": 0.6447324391004736, "actual_fraction_std": 0.012026064466199035}, {"budget": 0.69, "metric": 0.6348240365255367, "metric_std": 0.003250550675812357, "actual_fraction": 0.655871593454896, "actual_fraction_std": 0.012418571529718307}, {"budget": 0.7, "metric": 0.6324419363071774, "metric_std": 0.003303861758269954, "actual_fraction": 0.6665116410969005, "actual_fraction_std": 0.012452868292296103}, {"budget": 0.71, "metric": 0.6297138643904374, "metric_std": 0.0037099380450165875, "actual_fraction": 0.6780421404871962, "actual_fraction_std": 0.012566957453082683}, {"budget": 0.72, "metric": 0.6270595241471231, "metric_std": 0.0037900043359217895, "actual_fraction": 0.687848453052775, "actual_fraction_std": 0.01319178763550053}, {"budget": 0.73, "metric": 0.624359810566316, "metric_std": 0.003756457401865809, "actual_fraction": 0.6985792473697643, "actual_fraction_std": 0.013166882564396394}, {"budget": 0.74, "metric": 0.62159203697927, "metric_std": 0.004087802729216404, "actual_fraction": 0.7093157133539403, "actual_fraction_std": 0.013513046125645497}, {"budget": 0.75, "metric": 0.6185803817032016, "metric_std": 0.004626017382024357, "actual_fraction": 0.7205569577177211, "actual_fraction_std": 0.013357303040817102}, {"budget": 0.76, "metric": 0.6154042480787226, "metric_std": 0.004551535003099835, "actual_fraction": 0.7306752119785611, "actual_fraction_std": 0.013217257275983689}, {"budget": 0.77, "metric": 0.6121713977823781, "metric_std": 0.004783573005725944, "actual_fraction": 0.7409692879221847, "actual_fraction_std": 0.013393954581550661}, {"budget": 0.78, "metric": 0.6087967558063693, "metric_std": 0.004858677648693181, "actual_fraction": 0.7520970989422342, "actual_fraction_std": 0.013134156475540304}, {"budget": 0.79, "metric": 0.6053710688256813, "metric_std": 0.005011544628138265, "actual_fraction": 0.7639338683606047, "actual_fraction_std": 0.013372039460809667}, {"budget": 0.8, "metric": 0.6012931401185378, "metric_std": 0.00547887160110957, "actual_fraction": 0.7769787028897144, "actual_fraction_std": 0.014142035148271307}, {"budget": 0.81, "metric": 0.5976405864503871, "metric_std": 0.004953771916838412, "actual_fraction": 0.7888154723080849, "actual_fraction_std": 0.013292877330968782}, {"budget": 0.82, "metric": 0.5939540027791168, "metric_std": 0.004652003282975198, "actual_fraction": 0.799671043303179, "actual_fraction_std": 0.012881388202967352}, {"budget": 0.83, "metric": 0.5896775657204435, "metric_std": 0.00466537290941536, "actual_fraction": 0.8114284093809374, "actual_fraction_std": 0.01249226703524317}, {"budget": 0.84, "metric": 0.5853273969883448, "metric_std": 0.00490708207006902, "actual_fraction": 0.823225477129002, "actual_fraction_std": 0.012302625989874081}, {"budget": 0.85, "metric": 0.5808467799109548, "metric_std": 0.005033608622210647, "actual_fraction": 0.8346992598474321, "actual_fraction_std": 0.012170963678021932}, {"budget": 0.86, "metric": 0.5766554178600799, "metric_std": 0.005095299347939179, "actual_fraction": 0.8458951308737204, "actual_fraction_std": 0.011964637011219446}, {"budget": 0.87, "metric": 0.5724016674701529, "metric_std": 0.005250528858431239, "actual_fraction": 0.8569038368828519, "actual_fraction_std": 0.012152564429515877}, {"budget": 0.88, "metric": 0.5677111987068598, "metric_std": 0.004623948201767571, "actual_fraction": 0.868638516291864, "actual_fraction_std": 0.010846894825775983}, {"budget": 0.89, "metric": 0.5631284916201118, "metric_std": 0.005002719111192073, "actual_fraction": 0.8793125939369878, "actual_fraction_std": 0.010566104382526292}, {"budget": 0.9, "metric": 0.5585287695318039, "metric_std": 0.0046967774223608365, "actual_fraction": 0.8898675665711936, "actual_fraction_std": 0.009962731824854363}, {"budget": 0.91, "metric": 0.5539857641153618, "metric_std": 0.004565225293685162, "actual_fraction": 0.9006267192241161, "actual_fraction_std": 0.009053049534789314}, {"budget": 0.92, "metric": 0.5498227604004197, "metric_std": 0.004808582133467798, "actual_fraction": 0.9102855684428437, "actual_fraction_std": 0.009120426169679334}, {"budget": 0.93, "metric": 0.5444630349091115, "metric_std": 0.0046166900579897785, "actual_fraction": 0.9222641295408784, "actual_fraction_std": 0.008220430561756715}, {"budget": 0.94, "metric": 0.539069279414684, "metric_std": 0.003858265983716658, "actual_fraction": 0.9331140288687859, "actual_fraction_std": 0.006348138756678345}, {"budget": 0.95, "metric": 0.5336868672546294, "metric_std": 0.003610407100503806, "actual_fraction": 0.9439185548592007, "actual_fraction_std": 0.005901280448372811}, {"budget": 0.96, "metric": 0.5275557950259478, "metric_std": 0.0032589515140259663, "actual_fraction": 0.9553469642401383, "actual_fraction_std": 0.004924259887326958}, {"budget": 0.97, "metric": 0.520874571080169, "metric_std": 0.002536494232914186, "actual_fraction": 0.966695970280464, "actual_fraction_std": 0.0038749671163683013}, {"budget": 0.98, "metric": 0.5142840938093752, "metric_std": 0.002416489666605443, "actual_fraction": 0.977761392961461, "actual_fraction_std": 0.003646301998205248}, {"budget": 0.99, "metric": 0.5079318265604174, "metric_std": 0.001739524458740192, "actual_fraction": 0.9880441255707115, "actual_fraction_std": 0.0023563541830349576}, {"budget": 1.0, "metric": 0.5000708958398321, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 7755, "rescuable": 1369, "harmful": 2921, "both_wrong": 2184, "unit": "sample", "total": 14229}, "test": {"both_correct": 15007, "rescuable": 2627, "harmful": 8602, "both_wrong": 9027, "unit": "sample", "total": 35263}}, "small": {"validation": 0.7502986857825568, "test": 0.6695119530385957}, "large": {"validation": 0.6412256658935975, "test": 0.5000708958398321}, "learned_minus_confidence_test": 0.06035788219947247, "learned_minus_small_test": 0.009624819215608427, "learned_minus_large_test": 0.179065876414372, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_instancevg_MiniCPM"}, {"key": ["grefcoco", "instancevg", "Qwen3.8"], "expert_display": "InstanceVG", "run": "outputs/full_budget_20260913/runs/full_instancevg_Qwen3.8_s{2026..2030}", "validation_rows": 14229, "test_rows": 35263, "confidence_rule": {"column": "max_score", "columns": ["detections_above_07", "max_score", "mean_score", "query_count", "detections_above_07_valid", "max_score_valid", "mean_score_valid", "query_count_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/instancevg/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 71, "validation_area_0_25": 0.7477616136060159, "validation_point": {"budget": 0.71, "threshold": -0.9949791431427002, "operator": "ge", "actual_fraction": 0.7099585353854804, "metric": 0.8201560193970061, "rescued": 1660, "harmed": 666, "missed_rescue": 304, "avoided_harm": 505, "selected_both_correct": 6505, "selected_both_wrong": 1271}, "test_point": {"budget": 0.71, "threshold": -0.9949791431427002, "operator": "ge", "actual_fraction": 0.6838612710206166, "metric": 0.7243286163967898, "rescued": 3794, "harmed": 1861, "missed_rescue": 1629, "avoided_harm": 910, "selected_both_correct": 13678, "selected_both_wrong": 4782}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.7502986857825568, "rescued": 0, "harmed": 0, "missed_rescue": 1964, "avoided_harm": 1171, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -2.0005727208172175e-07, "operator": "ge", "actual_fraction": 0.009979619087778481, "metric": 0.7502284067748963, "rescued": 0, "harmed": 1, "missed_rescue": 1964, "avoided_harm": 1170, "selected_both_correct": 141, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -2.4156264544217265e-07, "operator": "ge", "actual_fraction": 0.019959238175556962, "metric": 0.7501581277672359, "rescued": 0, "harmed": 2, "missed_rescue": 1964, "avoided_harm": 1169, "selected_both_correct": 282, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -2.7729177531909954e-07, "operator": "ge", "actual_fraction": 0.029938857263335443, "metric": 0.7498770117365943, "rescued": 0, "harmed": 6, "missed_rescue": 1964, "avoided_harm": 1165, "selected_both_correct": 420, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": -3.064858447032748e-07, "operator": "ge", "actual_fraction": 0.03998875535877434, "metric": 0.7495958957059526, "rescued": 0, "harmed": 10, "missed_rescue": 1964, "avoided_harm": 1161, "selected_both_correct": 559, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": -3.3848826319626824e-07, "operator": "ge", "actual_fraction": 0.04996837444655281, "metric": 0.7494553376906318, "rescued": 0, "harmed": 12, "missed_rescue": 1964, "avoided_harm": 1159, "selected_both_correct": 699, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": -3.668622525765386e-07, "operator": "ge", "actual_fraction": 0.05994799353433129, "metric": 0.7493850586829714, "rescued": 0, "harmed": 13, "missed_rescue": 1964, "avoided_harm": 1158, "selected_both_correct": 840, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": -3.98026287484754e-07, "operator": "ge", "actual_fraction": 0.06999789162977019, "metric": 0.749314779675311, "rescued": 0, "harmed": 14, "missed_rescue": 1964, "avoided_harm": 1157, "selected_both_correct": 982, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": -4.302527258914779e-07, "operator": "ge", "actual_fraction": 0.07997751071754867, "metric": 0.7491742216599901, "rescued": 0, "harmed": 16, "missed_rescue": 1964, "avoided_harm": 1155, "selected_both_correct": 1122, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": -4.7191912244670675e-07, "operator": "ge", "actual_fraction": 0.08995712980532715, "metric": 0.748963384637009, "rescued": 0, "harmed": 19, "missed_rescue": 1964, "avoided_harm": 1152, "selected_both_correct": 1261, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": -5.115426802149159e-07, "operator": "ge", "actual_fraction": 0.09993674889310562, "metric": 0.7486119895987069, "rescued": 0, "harmed": 24, "missed_rescue": 1964, "avoided_harm": 1147, "selected_both_correct": 1398, "selected_both_wrong": 0}, {"budget": 0.11, "threshold": -5.534793672268279e-07, "operator": "ge", "actual_fraction": 0.10998664698854452, "metric": 0.7483308735680653, "rescued": 0, "harmed": 28, "missed_rescue": 1964, "avoided_harm": 1143, "selected_both_correct": 1537, "selected_both_wrong": 0}, {"budget": 0.12, "threshold": -5.941800509390305e-07, "operator": "ge", "actual_fraction": 0.119966266076323, "metric": 0.7481903155527444, "rescued": 0, "harmed": 30, "missed_rescue": 1964, "avoided_harm": 1141, "selected_both_correct": 1677, "selected_both_wrong": 0}, {"budget": 0.13, "threshold": -6.464504735959054e-07, "operator": "ge", "actual_fraction": 0.1299458851641015, "metric": 0.7477686415067819, "rescued": 0, "harmed": 36, "missed_rescue": 1964, "avoided_harm": 1135, "selected_both_correct": 1813, "selected_both_wrong": 0}, {"budget": 0.14, "threshold": -6.999837864896108e-07, "operator": "ge", "actual_fraction": 0.13999578325954037, "metric": 0.7476280834914611, "rescued": 0, "harmed": 38, "missed_rescue": 1964, "avoided_harm": 1133, "selected_both_correct": 1954, "selected_both_wrong": 0}, {"budget": 0.15, "threshold": -7.663418841730163e-07, "operator": "ge", "actual_fraction": 0.14997540234731885, "metric": 0.7474875254761403, "rescued": 0, "harmed": 40, "missed_rescue": 1964, "avoided_harm": 1131, "selected_both_correct": 2094, "selected_both_wrong": 0}, {"budget": 0.16, "threshold": -8.471382670904859e-07, "operator": "ge", "actual_fraction": 0.15995502143509735, "metric": 0.7472766884531591, "rescued": 0, "harmed": 43, "missed_rescue": 1964, "avoided_harm": 1128, "selected_both_correct": 2233, "selected_both_wrong": 0}, {"budget": 0.17, "threshold": -9.292756431023008e-07, "operator": "ge", "actual_fraction": 0.16993464052287582, "metric": 0.7471361304378382, "rescued": 0, "harmed": 45, "missed_rescue": 1964, "avoided_harm": 1126, "selected_both_correct": 2373, "selected_both_wrong": 0}, {"budget": 0.18, "threshold": -1.0245233852401725e-06, "operator": "ge", "actual_fraction": 0.1799845386183147, "metric": 0.7470658514301778, "rescued": 0, "harmed": 46, "missed_rescue": 1964, "avoided_harm": 1125, "selected_both_correct": 2515, "selected_both_wrong": 0}, {"budget": 0.19, "threshold": -1.1468601996966754e-06, "operator": "ge", "actual_fraction": 0.18996415770609318, "metric": 0.7465036193688945, "rescued": 0, "harmed": 54, "missed_rescue": 1964, "avoided_harm": 1117, "selected_both_correct": 2649, "selected_both_wrong": 0}, {"budget": 0.2, "threshold": -1.3018121762797819e-06, "operator": "ge", "actual_fraction": 0.19994377679387168, "metric": 0.7462225033382529, "rescued": 0, "harmed": 58, "missed_rescue": 1964, "avoided_harm": 1113, "selected_both_correct": 2787, "selected_both_wrong": 0}, {"budget": 0.21, "threshold": -1.4898060953782988e-06, "operator": "ge", "actual_fraction": 0.20999367488931056, "metric": 0.7456602712769695, "rescued": 0, "harmed": 66, "missed_rescue": 1964, "avoided_harm": 1105, "selected_both_correct": 2922, "selected_both_wrong": 0}, {"budget": 0.22, "threshold": -1.735574414851726e-06, "operator": "ge", "actual_fraction": 0.21997329397708903, "metric": 0.7453088762386675, "rescued": 0, "harmed": 71, "missed_rescue": 1964, "avoided_harm": 1100, "selected_both_correct": 3059, "selected_both_wrong": 0}, {"budget": 0.23, "threshold": -2.0881013824691763e-06, "operator": "ge", "actual_fraction": 0.22995291306486754, "metric": 0.7446763651697238, "rescued": 1, "harmed": 81, "missed_rescue": 1963, "avoided_harm": 1090, "selected_both_correct": 3190, "selected_both_wrong": 0}, {"budget": 0.24, "threshold": -2.5644872039265465e-06, "operator": "ge", "actual_fraction": 0.239932532152646, "metric": 0.7442546911237613, "rescued": 1, "harmed": 87, "missed_rescue": 1963, "avoided_harm": 1084, "selected_both_correct": 3326, "selected_both_wrong": 0}, {"budget": 0.25, "threshold": -3.2399550491390983e-06, "operator": "ge", "actual_fraction": 0.2499824302480849, "metric": 0.7437627380701385, "rescued": 1, "harmed": 94, "missed_rescue": 1963, "avoided_harm": 1077, "selected_both_correct": 3462, "selected_both_wrong": 0}, {"budget": 0.26, "threshold": -4.2517635847616475e-06, "operator": "ge", "actual_fraction": 0.25996204933586337, "metric": 0.7431302270011948, "rescued": 1, "harmed": 103, "missed_rescue": 1963, "avoided_harm": 1068, "selected_both_correct": 3595, "selected_both_wrong": 0}, {"budget": 0.27, "threshold": -5.607922958006384e-06, "operator": "ge", "actual_fraction": 0.26994166842364187, "metric": 0.7422868789092698, "rescued": 1, "harmed": 115, "missed_rescue": 1963, "avoided_harm": 1056, "selected_both_correct": 3725, "selected_both_wrong": 0}, {"budget": 0.28, "threshold": -8.438555596512742e-06, "operator": "ge", "actual_fraction": 0.27999156651908075, "metric": 0.7419354838709677, "rescued": 1, "harmed": 120, "missed_rescue": 1963, "avoided_harm": 1051, "selected_both_correct": 3862, "selected_both_wrong": 1}, {"budget": 0.29, "threshold": -1.3145449884177651e-05, "operator": "ge", "actual_fraction": 0.28997118560685925, "metric": 0.7411624147867032, "rescued": 1, "harmed": 131, "missed_rescue": 1963, "avoided_harm": 1040, "selected_both_correct": 3993, "selected_both_wrong": 1}, {"budget": 0.3, "threshold": -2.076632881653495e-05, "operator": "ge", "actual_fraction": 0.2999508046946377, "metric": 0.7404596247100991, "rescued": 2, "harmed": 142, "missed_rescue": 1962, "avoided_harm": 1029, "selected_both_correct": 4123, "selected_both_wrong": 1}, {"budget": 0.31, "threshold": -4.0114435250870883e-05, "operator": "ge", "actual_fraction": 0.3099304237824162, "metric": 0.7394757186028533, "rescued": 2, "harmed": 156, "missed_rescue": 1962, "avoided_harm": 1015, "selected_both_correct": 4251, "selected_both_wrong": 1}, {"budget": 0.32, "threshold": -7.415730942739174e-05, "operator": "ge", "actual_fraction": 0.3199803218778551, "metric": 0.7388432075339096, "rescued": 2, "harmed": 165, "missed_rescue": 1962, "avoided_harm": 1006, "selected_both_correct": 4385, "selected_both_wrong": 1}, {"budget": 0.33, "threshold": -0.00015223574882838875, "operator": "ge", "actual_fraction": 0.3299599409656336, "metric": 0.7377890224190035, "rescued": 2, "harmed": 180, "missed_rescue": 1962, "avoided_harm": 991, "selected_both_correct": 4512, "selected_both_wrong": 1}, {"budget": 0.34, "threshold": -0.00032258269493468106, "operator": "ge", "actual_fraction": 0.339939560053412, "metric": 0.7361023262351536, "rescued": 2, "harmed": 204, "missed_rescue": 1962, "avoided_harm": 967, "selected_both_correct": 4629, "selected_both_wrong": 2}, {"budget": 0.35, "threshold": -0.0006007538759149611, "operator": "ge", "actual_fraction": 0.34998945814885096, "metric": 0.7348373040972661, "rescued": 2, "harmed": 222, "missed_rescue": 1962, "avoided_harm": 949, "selected_both_correct": 4754, "selected_both_wrong": 2}, {"budget": 0.36, "threshold": -0.0011453087208792567, "operator": "ge", "actual_fraction": 0.3599690772366294, "metric": 0.7337128399746996, "rescued": 2, "harmed": 238, "missed_rescue": 1962, "avoided_harm": 933, "selected_both_correct": 4880, "selected_both_wrong": 2}, {"budget": 0.37, "threshold": -0.002608026610687375, "operator": "ge", "actual_fraction": 0.3699486963244079, "metric": 0.732588375852133, "rescued": 2, "harmed": 254, "missed_rescue": 1962, "avoided_harm": 917, "selected_both_correct": 5006, "selected_both_wrong": 2}, {"budget": 0.38, "threshold": -0.006219128146767616, "operator": "ge", "actual_fraction": 0.3799985944198468, "metric": 0.7311827956989247, "rescued": 2, "harmed": 274, "missed_rescue": 1962, "avoided_harm": 897, "selected_both_correct": 5128, "selected_both_wrong": 3}, {"budget": 0.39, "threshold": -0.01280172262340784, "operator": "ge", "actual_fraction": 0.3899782135076253, "metric": 0.7297069365380561, "rescued": 2, "harmed": 295, "missed_rescue": 1962, "avoided_harm": 876, "selected_both_correct": 5248, "selected_both_wrong": 4}, {"budget": 0.4, "threshold": -0.03794781118631363, "operator": "ge", "actual_fraction": 0.39995783259540374, "metric": 0.7282310773771874, "rescued": 3, "harmed": 317, "missed_rescue": 1961, "avoided_harm": 854, "selected_both_correct": 5365, "selected_both_wrong": 6}, {"budget": 0.41, "threshold": -0.08479184657335281, "operator": "ge", "actual_fraction": 0.40993745168318224, "metric": 0.7266849392086584, "rescued": 3, "harmed": 339, "missed_rescue": 1961, "avoided_harm": 832, "selected_both_correct": 5484, "selected_both_wrong": 7}, {"budget": 0.42, "threshold": -0.14941807091236115, "operator": "ge", "actual_fraction": 0.4199873497786211, "metric": 0.7249279640171481, "rescued": 7, "harmed": 368, "missed_rescue": 1957, "avoided_harm": 803, "selected_both_correct": 5590, "selected_both_wrong": 11}, {"budget": 0.43, "threshold": -0.2939925789833069, "operator": "ge", "actual_fraction": 0.4299669688663996, "metric": 0.7233115468409586, "rescued": 12, "harmed": 396, "missed_rescue": 1952, "avoided_harm": 775, "selected_both_correct": 5696, "selected_both_wrong": 14}, {"budget": 0.44, "threshold": -0.4386065602302551, "operator": "ge", "actual_fraction": 0.43994658795417807, "metric": 0.7212734556188066, "rescued": 15, "harmed": 428, "missed_rescue": 1949, "avoided_harm": 743, "selected_both_correct": 5798, "selected_both_wrong": 19}, {"budget": 0.45, "threshold": -0.5420411825180054, "operator": "ge", "actual_fraction": 0.449996486049617, "metric": 0.7190245273736735, "rescued": 16, "harmed": 461, "missed_rescue": 1948, "avoided_harm": 710, "selected_both_correct": 5901, "selected_both_wrong": 25}, {"budget": 0.46, "threshold": -0.6230618953704834, "operator": "ge", "actual_fraction": 0.45997610513739545, "metric": 0.7173378311898236, "rescued": 21, "harmed": 490, "missed_rescue": 1943, "avoided_harm": 681, "selected_both_correct": 6008, "selected_both_wrong": 26}, {"budget": 0.47, "threshold": -0.6914289593696594, "operator": "ge", "actual_fraction": 0.46995572422517395, "metric": 0.7155808559983133, "rescued": 27, "harmed": 521, "missed_rescue": 1937, "avoided_harm": 650, "selected_both_correct": 6107, "selected_both_wrong": 32}, {"budget": 0.48, "threshold": -0.7387455701828003, "operator": "ge", "actual_fraction": 0.4799353433129524, "metric": 0.7206409445498629, "rescued": 105, "harmed": 527, "missed_rescue": 1859, "avoided_harm": 644, "selected_both_correct": 6125, "selected_both_wrong": 72}, {"budget": 0.49, "threshold": -0.7823225855827332, "operator": "ge", "actual_fraction": 0.48998524140839134, "metric": 0.7280202403542062, "rescued": 210, "harmed": 527, "missed_rescue": 1754, "avoided_harm": 644, "selected_both_correct": 6125, "selected_both_wrong": 110}, {"budget": 0.5, "threshold": -0.8206579089164734, "operator": "ge", "actual_fraction": 0.4999648604961698, "metric": 0.7346264670742849, "rescued": 305, "harmed": 528, "missed_rescue": 1659, "avoided_harm": 643, "selected_both_correct": 6125, "selected_both_wrong": 156}, {"budget": 0.51, "threshold": -0.8575630187988281, "operator": "ge", "actual_fraction": 0.5099444795839483, "metric": 0.7414435308173448, "rescued": 403, "harmed": 529, "missed_rescue": 1561, "avoided_harm": 642, "selected_both_correct": 6125, "selected_both_wrong": 199}, {"budget": 0.52, "threshold": -0.8823798894882202, "operator": "ge", "actual_fraction": 0.5199943776793872, "metric": 0.7478389205144423, "rescued": 495, "harmed": 530, "missed_rescue": 1469, "avoided_harm": 641, "selected_both_correct": 6125, "selected_both_wrong": 249}, {"budget": 0.53, "threshold": -0.9040287733078003, "operator": "ge", "actual_fraction": 0.5299739967671656, "metric": 0.754093752196219, "rescued": 585, "harmed": 531, "missed_rescue": 1379, "avoided_harm": 640, "selected_both_correct": 6125, "selected_both_wrong": 300}, {"budget": 0.54, "threshold": -0.9208687543869019, "operator": "ge", "actual_fraction": 0.5399536158549442, "metric": 0.7602783048703352, "rescued": 673, "harmed": 531, "missed_rescue": 1291, "avoided_harm": 640, "selected_both_correct": 6127, "selected_both_wrong": 352}, {"budget": 0.55, "threshold": -0.9344452619552612, "operator": "ge", "actual_fraction": 0.5499332349427226, "metric": 0.7662520205214702, "rescued": 759, "harmed": 532, "missed_rescue": 1205, "avoided_harm": 639, "selected_both_correct": 6129, "selected_both_wrong": 405}, {"budget": 0.56, "threshold": -0.9448322653770447, "operator": "ge", "actual_fraction": 0.5599831330381615, "metric": 0.7718743411343032, "rescued": 844, "harmed": 537, "missed_rescue": 1120, "avoided_harm": 634, "selected_both_correct": 6130, "selected_both_wrong": 457}, {"budget": 0.57, "threshold": -0.9531905055046082, "operator": "ge", "actual_fraction": 0.5699627521259399, "metric": 0.7774263827394757, "rescued": 924, "harmed": 538, "missed_rescue": 1040, "avoided_harm": 633, "selected_both_correct": 6133, "selected_both_wrong": 515}, {"budget": 0.58, "threshold": -0.9603281617164612, "operator": "gt", "actual_fraction": 0.5798720922060581, "metric": 0.7818539602220816, "rescued": 991, "harmed": 542, "missed_rescue": 973, "avoided_harm": 629, "selected_both_correct": 6135, "selected_both_wrong": 583}, {"budget": 0.59, "threshold": -0.9670801758766174, "operator": "ge", "actual_fraction": 0.5899922693091574, "metric": 0.7864220957200084, "rescued": 1061, "harmed": 547, "missed_rescue": 903, "avoided_harm": 624, "selected_both_correct": 6139, "selected_both_wrong": 648}, {"budget": 0.6, "threshold": -0.9724565148353577, "operator": "ge", "actual_fraction": 0.5999718883969358, "metric": 0.7909902312179352, "rescued": 1130, "harmed": 551, "missed_rescue": 834, "avoided_harm": 620, "selected_both_correct": 6146, "selected_both_wrong": 710}, {"budget": 0.61, "threshold": -0.9766654968261719, "operator": "ge", "actual_fraction": 0.6099515074847143, "metric": 0.7954880877082016, "rescued": 1200, "harmed": 557, "missed_rescue": 764, "avoided_harm": 614, "selected_both_correct": 6151, "selected_both_wrong": 771}, {"budget": 0.62, "threshold": -0.9801702499389648, "operator": "ge", "actual_fraction": 0.6199311265724928, "metric": 0.7996345491601659, "rescued": 1263, "harmed": 561, "missed_rescue": 701, "avoided_harm": 610, "selected_both_correct": 6159, "selected_both_wrong": 838}, {"budget": 0.63, "threshold": -0.9831460118293762, "operator": "ge", "actual_fraction": 0.6299810246679317, "metric": 0.8028673835125448, "rescued": 1317, "harmed": 569, "missed_rescue": 647, "avoided_harm": 602, "selected_both_correct": 6171, "selected_both_wrong": 907}, {"budget": 0.64, "threshold": -0.985751748085022, "operator": "ge", "actual_fraction": 0.6399606437557102, "metric": 0.8064516129032258, "rescued": 1380, "harmed": 581, "missed_rescue": 584, "avoided_harm": 590, "selected_both_correct": 6186, "selected_both_wrong": 959}, {"budget": 0.65, "threshold": -0.9877713322639465, "operator": "ge", "actual_fraction": 0.6499402628434886, "metric": 0.809895284278586, "rescued": 1434, "harmed": 586, "missed_rescue": 530, "avoided_harm": 585, "selected_both_correct": 6211, "selected_both_wrong": 1017}, {"budget": 0.66, "threshold": -0.9895955324172974, "operator": "ge", "actual_fraction": 0.6599901609389276, "metric": 0.8123550495467003, "rescued": 1479, "harmed": 596, "missed_rescue": 485, "avoided_harm": 575, "selected_both_correct": 6244, "selected_both_wrong": 1072}, {"budget": 0.67, "threshold": -0.9913432002067566, "operator": "ge", "actual_fraction": 0.669969780026706, "metric": 0.8149553728301356, "rescued": 1525, "harmed": 605, "missed_rescue": 439, "avoided_harm": 566, "selected_both_correct": 6278, "selected_both_wrong": 1125}, {"budget": 0.68, "threshold": -0.9925730228424072, "operator": "ge", "actual_fraction": 0.6799493991144845, "metric": 0.8169231850446271, "rescued": 1569, "harmed": 621, "missed_rescue": 395, "avoided_harm": 550, "selected_both_correct": 6318, "selected_both_wrong": 1167}, {"budget": 0.69, "threshold": -0.9935283660888672, "operator": "ge", "actual_fraction": 0.6899992972099234, "metric": 0.818609881228477, "rescued": 1605, "harmed": 633, "missed_rescue": 359, "avoided_harm": 538, "selected_both_correct": 6376, "selected_both_wrong": 1204}, {"budget": 0.7, "threshold": -0.99436354637146, "operator": "gt", "actual_fraction": 0.6999086372900415, "metric": 0.8196640663433832, "rescued": 1636, "harmed": 649, "missed_rescue": 328, "avoided_harm": 522, "selected_both_correct": 6439, "selected_both_wrong": 1235}, {"budget": 0.71, "threshold": -0.9949791431427002, "operator": "ge", "actual_fraction": 0.7099585353854804, "metric": 0.8201560193970061, "rescued": 1660, "harmed": 666, "missed_rescue": 304, "avoided_harm": 505, "selected_both_correct": 6505, "selected_both_wrong": 1271}, {"budget": 0.72, "threshold": -0.9955723881721497, "operator": "ge", "actual_fraction": 0.7199381544732588, "metric": 0.8200154613816852, "rescued": 1686, "harmed": 694, "missed_rescue": 278, "avoided_harm": 477, "selected_both_correct": 6565, "selected_both_wrong": 1299}, {"budget": 0.73, "threshold": -0.9960037469863892, "operator": "ge", "actual_fraction": 0.7299880525686977, "metric": 0.8199451823740249, "rescued": 1703, "harmed": 712, "missed_rescue": 261, "avoided_harm": 459, "selected_both_correct": 6646, "selected_both_wrong": 1326}, {"budget": 0.74, "threshold": -0.9964693188667297, "operator": "ge", "actual_fraction": 0.7399676716564763, "metric": 0.8200154613816852, "rescued": 1722, "harmed": 730, "missed_rescue": 242, "avoided_harm": 441, "selected_both_correct": 6725, "selected_both_wrong": 1352}, {"budget": 0.75, "threshold": -0.9968041777610779, "operator": "ge", "actual_fraction": 0.7499472907442547, "metric": 0.8198749033663645, "rescued": 1742, "harmed": 752, "missed_rescue": 222, "avoided_harm": 419, "selected_both_correct": 6808, "selected_both_wrong": 1369}, {"budget": 0.76, "threshold": -0.9971042275428772, "operator": "ge", "actual_fraction": 0.7599971888396936, "metric": 0.8200154613816852, "rescued": 1763, "harmed": 771, "missed_rescue": 201, "avoided_harm": 400, "selected_both_correct": 6891, "selected_both_wrong": 1389}, {"budget": 0.77, "threshold": -0.9974007606506348, "operator": "ge", "actual_fraction": 0.769976807927472, "metric": 0.8195235083280624, "rescued": 1784, "harmed": 799, "missed_rescue": 180, "avoided_harm": 372, "selected_both_correct": 6966, "selected_both_wrong": 1407}, {"budget": 0.78, "threshold": -0.9976577162742615, "operator": "ge", "actual_fraction": 0.7799564270152506, "metric": 0.8191018342820999, "rescued": 1800, "harmed": 821, "missed_rescue": 164, "avoided_harm": 350, "selected_both_correct": 7050, "selected_both_wrong": 1427}, {"budget": 0.79, "threshold": -0.9978766441345215, "operator": "ge", "actual_fraction": 0.789936046103029, "metric": 0.8192423922974208, "rescued": 1818, "harmed": 837, "missed_rescue": 146, "avoided_harm": 334, "selected_both_correct": 7143, "selected_both_wrong": 1442}, {"budget": 0.8, "threshold": -0.9980920553207397, "operator": "ge", "actual_fraction": 0.7999859441984679, "metric": 0.8190315552744395, "rescued": 1834, "harmed": 856, "missed_rescue": 130, "avoided_harm": 315, "selected_both_correct": 7244, "selected_both_wrong": 1449}, {"budget": 0.81, "threshold": -0.9982843399047852, "operator": "ge", "actual_fraction": 0.8099655632862464, "metric": 0.8190315552744395, "rescued": 1849, "harmed": 871, "missed_rescue": 115, "avoided_harm": 300, "selected_both_correct": 7347, "selected_both_wrong": 1458}, {"budget": 0.82, "threshold": -0.9984900951385498, "operator": "ge", "actual_fraction": 0.8199451823740249, "metric": 0.8187504392437979, "rescued": 1859, "harmed": 885, "missed_rescue": 105, "avoided_harm": 286, "selected_both_correct": 7444, "selected_both_wrong": 1479}, {"budget": 0.83, "threshold": -0.9986395835876465, "operator": "ge", "actual_fraction": 0.8299950804694638, "metric": 0.818609881228477, "rescued": 1874, "harmed": 902, "missed_rescue": 90, "avoided_harm": 269, "selected_both_correct": 7541, "selected_both_wrong": 1493}, {"budget": 0.84, "threshold": -0.9987816214561462, "operator": "ge", "actual_fraction": 0.8399746995572422, "metric": 0.8176962541288917, "rescued": 1880, "harmed": 921, "missed_rescue": 84, "avoided_harm": 250, "selected_both_correct": 7643, "selected_both_wrong": 1508}, {"budget": 0.85, "threshold": -0.9989162087440491, "operator": "ge", "actual_fraction": 0.8499543186450207, "metric": 0.8175556961135709, "rescued": 1889, "harmed": 932, "missed_rescue": 75, "avoided_harm": 239, "selected_both_correct": 7754, "selected_both_wrong": 1519}, {"budget": 0.86, "threshold": -0.9990254044532776, "operator": "ge", "actual_fraction": 0.8599339377327992, "metric": 0.8165717900063251, "rescued": 1895, "harmed": 952, "missed_rescue": 69, "avoided_harm": 219, "selected_both_correct": 7864, "selected_both_wrong": 1525}, {"budget": 0.87, "threshold": -0.9991487264633179, "operator": "ge", "actual_fraction": 0.8699838358282381, "metric": 0.8160798369527023, "rescued": 1907, "harmed": 971, "missed_rescue": 57, "avoided_harm": 200, "selected_both_correct": 7969, "selected_both_wrong": 1532}, {"budget": 0.88, "threshold": -0.9992602467536926, "operator": "ge", "actual_fraction": 0.8799634549160166, "metric": 0.8155878838990793, "rescued": 1914, "harmed": 985, "missed_rescue": 50, "avoided_harm": 186, "selected_both_correct": 8082, "selected_both_wrong": 1540}, {"budget": 0.89, "threshold": -0.999350905418396, "operator": "ge", "actual_fraction": 0.889943074003795, "metric": 0.8148148148148148, "rescued": 1919, "harmed": 1001, "missed_rescue": 45, "avoided_harm": 170, "selected_both_correct": 8192, "selected_both_wrong": 1551}, {"budget": 0.9, "threshold": -0.999435544013977, "operator": "ge", "actual_fraction": 0.899992972099234, "metric": 0.8144634197765127, "rescued": 1927, "harmed": 1014, "missed_rescue": 37, "avoided_harm": 157, "selected_both_correct": 8308, "selected_both_wrong": 1557}, {"budget": 0.91, "threshold": -0.9995256066322327, "operator": "ge", "actual_fraction": 0.9099725911870125, "metric": 0.8129172816079837, "rescued": 1932, "harmed": 1041, "missed_rescue": 32, "avoided_harm": 130, "selected_both_correct": 8410, "selected_both_wrong": 1565}, {"budget": 0.92, "threshold": -0.9995898604393005, "operator": "ge", "actual_fraction": 0.9199522102747909, "metric": 0.8123550495467003, "rescued": 1937, "harmed": 1054, "missed_rescue": 27, "avoided_harm": 117, "selected_both_correct": 8530, "selected_both_wrong": 1569}, {"budget": 0.93, "threshold": -0.9996523857116699, "operator": "ge", "actual_fraction": 0.9299318293625694, "metric": 0.8113711434394546, "rescued": 1942, "harmed": 1073, "missed_rescue": 22, "avoided_harm": 98, "selected_both_correct": 8647, "selected_both_wrong": 1570}, {"budget": 0.94, "threshold": -0.9997095465660095, "operator": "ge", "actual_fraction": 0.9399817274580083, "metric": 0.8102466793168881, "rescued": 1945, "harmed": 1092, "missed_rescue": 19, "avoided_harm": 79, "selected_both_correct": 8764, "selected_both_wrong": 1574}, {"budget": 0.95, "threshold": -0.9997634887695312, "operator": "ge", "actual_fraction": 0.9499613465457868, "metric": 0.8087708201560194, "rescued": 1948, "harmed": 1116, "missed_rescue": 16, "avoided_harm": 55, "selected_both_correct": 8876, "selected_both_wrong": 1577}, {"budget": 0.96, "threshold": -0.9998065829277039, "operator": "ge", "actual_fraction": 0.9599409656335652, "metric": 0.8079977510717549, "rescued": 1951, "harmed": 1130, "missed_rescue": 13, "avoided_harm": 41, "selected_both_correct": 8998, "selected_both_wrong": 1580}, {"budget": 0.97, "threshold": -0.9998431205749512, "operator": "ge", "actual_fraction": 0.9699908637290041, "metric": 0.8075057980181319, "rescued": 1956, "harmed": 1142, "missed_rescue": 8, "avoided_harm": 29, "selected_both_correct": 9120, "selected_both_wrong": 1584}, {"budget": 0.98, "threshold": -0.9998764991760254, "operator": "ge", "actual_fraction": 0.9799704828167827, "metric": 0.8070138449645091, "rescued": 1961, "harmed": 1154, "missed_rescue": 3, "avoided_harm": 17, "selected_both_correct": 9242, "selected_both_wrong": 1587}, {"budget": 0.99, "threshold": -0.9999123811721802, "operator": "ge", "actual_fraction": 0.9899501019045611, "metric": 0.806311054887905, "rescued": 1962, "harmed": 1165, "missed_rescue": 2, "avoided_harm": 6, "selected_both_correct": 9370, "selected_both_wrong": 1589}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.8060299388572634, "rescued": 1964, "harmed": 1171, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 9505, "selected_both_wrong": 1589}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.6695119530385957, "rescued": 0, "harmed": 0, "missed_rescue": 5423, "avoided_harm": 2771, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -2.0005727208172175e-07, "operator": "ge", "actual_fraction": 0.002467175226157729, "metric": 0.6686895612965431, "rescued": 0, "harmed": 29, "missed_rescue": 5423, "avoided_harm": 2742, "selected_both_correct": 58, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -2.4156264544217265e-07, "operator": "ge", "actual_fraction": 0.004849275444516916, "metric": 0.6680373195700876, "rescued": 0, "harmed": 52, "missed_rescue": 5423, "avoided_harm": 2719, "selected_both_correct": 119, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -2.7729177531909954e-07, "operator": "ge", "actual_fraction": 0.007600034030003119, "metric": 0.6675835861951621, "rescued": 0, "harmed": 68, "missed_rescue": 5423, "avoided_harm": 2703, "selected_both_correct": 199, "selected_both_wrong": 1}, {"budget": 0.04, "threshold": -3.064858447032748e-07, "operator": "ge", "actual_fraction": 0.009811984232765221, "metric": 0.6671014944843037, "rescued": 0, "harmed": 85, "missed_rescue": 5423, "avoided_harm": 2686, "selected_both_correct": 260, "selected_both_wrong": 1}, {"budget": 0.05, "threshold": -3.3848826319626824e-07, "operator": "ge", "actual_fraction": 0.012534384482318578, "metric": 0.6666194027734452, "rescued": 0, "harmed": 102, "missed_rescue": 5423, "avoided_harm": 2669, "selected_both_correct": 338, "selected_both_wrong": 2}, {"budget": 0.06, "threshold": -3.668622525765386e-07, "operator": "ge", "actual_fraction": 0.01607917647392451, "metric": 0.6662223860703854, "rescued": 0, "harmed": 116, "missed_rescue": 5423, "avoided_harm": 2655, "selected_both_correct": 449, "selected_both_wrong": 2}, {"budget": 0.07, "threshold": -3.98026287484754e-07, "operator": "ge", "actual_fraction": 0.019709043473328986, "metric": 0.6658253693673255, "rescued": 0, "harmed": 130, "missed_rescue": 5423, "avoided_harm": 2641, "selected_both_correct": 562, "selected_both_wrong": 3}, {"budget": 0.08, "threshold": -4.302527258914779e-07, "operator": "ge", "actual_fraction": 0.02353741882426339, "metric": 0.6655985026798628, "rescued": 0, "harmed": 138, "missed_rescue": 5423, "avoided_harm": 2633, "selected_both_correct": 689, "selected_both_wrong": 3}, {"budget": 0.09, "threshold": -4.7191912244670675e-07, "operator": "ge", "actual_fraction": 0.028868785979638715, "metric": 0.6651164109690043, "rescued": 0, "harmed": 155, "missed_rescue": 5423, "avoided_harm": 2616, "selected_both_correct": 859, "selected_both_wrong": 4}, {"budget": 0.1, "threshold": -5.115426802149159e-07, "operator": "ge", "actual_fraction": 0.0345971698380739, "metric": 0.6647761109378102, "rescued": 0, "harmed": 167, "missed_rescue": 5423, "avoided_harm": 2604, "selected_both_correct": 1045, "selected_both_wrong": 8}, {"budget": 0.11, "threshold": -5.534793672268279e-07, "operator": "ge", "actual_fraction": 0.03981510364971783, "metric": 0.664435810906616, "rescued": 0, "harmed": 179, "missed_rescue": 5423, "avoided_harm": 2592, "selected_both_correct": 1215, "selected_both_wrong": 10}, {"budget": 0.12, "threshold": -5.941800509390305e-07, "operator": "ge", "actual_fraction": 0.045855429203414344, "metric": 0.664067152539489, "rescued": 0, "harmed": 192, "missed_rescue": 5423, "avoided_harm": 2579, "selected_both_correct": 1414, "selected_both_wrong": 11}, {"budget": 0.13, "threshold": -6.464504735959054e-07, "operator": "ge", "actual_fraction": 0.05317187987408899, "metric": 0.6637552108442276, "rescued": 1, "harmed": 204, "missed_rescue": 5422, "avoided_harm": 2567, "selected_both_correct": 1658, "selected_both_wrong": 12}, {"budget": 0.14, "threshold": -6.999837864896108e-07, "operator": "ge", "actual_fraction": 0.05960922213084536, "metric": 0.6636134191645634, "rescued": 1, "harmed": 209, "missed_rescue": 5422, "avoided_harm": 2562, "selected_both_correct": 1878, "selected_both_wrong": 14}, {"budget": 0.15, "threshold": -7.663418841730163e-07, "operator": "ge", "actual_fraction": 0.0675211978561098, "metric": 0.6632164024615036, "rescued": 2, "harmed": 224, "missed_rescue": 5421, "avoided_harm": 2547, "selected_both_correct": 2128, "selected_both_wrong": 27}, {"budget": 0.16, "threshold": -8.471382670904859e-07, "operator": "ge", "actual_fraction": 0.07551824858917279, "metric": 0.6626208774069138, "rescued": 2, "harmed": 245, "missed_rescue": 5421, "avoided_harm": 2526, "selected_both_correct": 2386, "selected_both_wrong": 30}, {"budget": 0.17, "threshold": -9.292756431023008e-07, "operator": "ge", "actual_fraction": 0.08212574086152624, "metric": 0.6622522190397867, "rescued": 3, "harmed": 259, "missed_rescue": 5420, "avoided_harm": 2512, "selected_both_correct": 2599, "selected_both_wrong": 35}, {"budget": 0.18, "threshold": -1.0245233852401725e-06, "operator": "ge", "actual_fraction": 0.08856308311828262, "metric": 0.6619969940163911, "rescued": 3, "harmed": 268, "missed_rescue": 5420, "avoided_harm": 2503, "selected_both_correct": 2812, "selected_both_wrong": 40}, {"budget": 0.19, "threshold": -1.1468601996966754e-06, "operator": "ge", "actual_fraction": 0.09480191702350906, "metric": 0.6615149023055327, "rescued": 3, "harmed": 285, "missed_rescue": 5420, "avoided_harm": 2486, "selected_both_correct": 3005, "selected_both_wrong": 50}, {"budget": 0.2, "threshold": -1.3018121762797819e-06, "operator": "ge", "actual_fraction": 0.10007656750701868, "metric": 0.6612596772821371, "rescued": 3, "harmed": 294, "missed_rescue": 5420, "avoided_harm": 2477, "selected_both_correct": 3174, "selected_both_wrong": 58}, {"budget": 0.21, "threshold": -1.4898060953782988e-06, "operator": "ge", "actual_fraction": 0.10554972634205825, "metric": 0.6608343022431443, "rescued": 3, "harmed": 309, "missed_rescue": 5420, "avoided_harm": 2462, "selected_both_correct": 3340, "selected_both_wrong": 70}, {"budget": 0.22, "threshold": -1.735574414851726e-06, "operator": "ge", "actual_fraction": 0.11005870175538099, "metric": 0.6605507188838159, "rescued": 3, "harmed": 319, "missed_rescue": 5420, "avoided_harm": 2452, "selected_both_correct": 3485, "selected_both_wrong": 74}, {"budget": 0.23, "threshold": -2.0881013824691763e-06, "operator": "ge", "actual_fraction": 0.11527663556702493, "metric": 0.6600969855088903, "rescued": 5, "harmed": 337, "missed_rescue": 5418, "avoided_harm": 2434, "selected_both_correct": 3630, "selected_both_wrong": 93}, {"budget": 0.24, "threshold": -2.5644872039265465e-06, "operator": "ge", "actual_fraction": 0.12029606102713893, "metric": 0.6598417604854947, "rescued": 8, "harmed": 349, "missed_rescue": 5415, "avoided_harm": 2422, "selected_both_correct": 3769, "selected_both_wrong": 116}, {"budget": 0.25, "threshold": -3.2399550491390983e-06, "operator": "ge", "actual_fraction": 0.12477667810452883, "metric": 0.6594731021183677, "rescued": 8, "harmed": 362, "missed_rescue": 5415, "avoided_harm": 2409, "selected_both_correct": 3894, "selected_both_wrong": 136}, {"budget": 0.26, "threshold": -4.2517635847616475e-06, "operator": "ge", "actual_fraction": 0.12914386183818732, "metric": 0.6590193687434421, "rescued": 8, "harmed": 378, "missed_rescue": 5415, "avoided_harm": 2393, "selected_both_correct": 4012, "selected_both_wrong": 156}, {"budget": 0.27, "threshold": -5.607922958006384e-06, "operator": "ge", "actual_fraction": 0.1322916371267334, "metric": 0.658679068712248, "rescued": 10, "harmed": 392, "missed_rescue": 5413, "avoided_harm": 2379, "selected_both_correct": 4092, "selected_both_wrong": 171}, {"budget": 0.28, "threshold": -8.438555596512742e-06, "operator": "ge", "actual_fraction": 0.13626180415733205, "metric": 0.6582820520091881, "rescued": 12, "harmed": 408, "missed_rescue": 5411, "avoided_harm": 2363, "selected_both_correct": 4192, "selected_both_wrong": 193}, {"budget": 0.29, "threshold": -1.3145449884177651e-05, "operator": "ge", "actual_fraction": 0.14023197118793068, "metric": 0.657743243626464, "rescued": 13, "harmed": 428, "missed_rescue": 5410, "avoided_harm": 2343, "selected_both_correct": 4285, "selected_both_wrong": 219}, {"budget": 0.3, "threshold": -2.076632881653495e-05, "operator": "ge", "actual_fraction": 0.14326631313274538, "metric": 0.657374585259337, "rescued": 16, "harmed": 444, "missed_rescue": 5407, "avoided_harm": 2327, "selected_both_correct": 4359, "selected_both_wrong": 233}, {"budget": 0.31, "threshold": -4.0114435250870883e-05, "operator": "ge", "actual_fraction": 0.14766185520233674, "metric": 0.6569775685562771, "rescued": 18, "harmed": 460, "missed_rescue": 5405, "avoided_harm": 2311, "selected_both_correct": 4464, "selected_both_wrong": 265}, {"budget": 0.32, "threshold": -7.415730942739174e-05, "operator": "ge", "actual_fraction": 0.15066783881121856, "metric": 0.6567507018688143, "rescued": 20, "harmed": 470, "missed_rescue": 5403, "avoided_harm": 2301, "selected_both_correct": 4545, "selected_both_wrong": 278}, {"budget": 0.33, "threshold": -0.00015223574882838875, "operator": "ge", "actual_fraction": 0.15452457249808582, "metric": 0.6565238351813516, "rescued": 27, "harmed": 485, "missed_rescue": 5396, "avoided_harm": 2286, "selected_both_correct": 4632, "selected_both_wrong": 305}, {"budget": 0.34, "threshold": -0.00032258269493468106, "operator": "ge", "actual_fraction": 0.15806936448969175, "metric": 0.6564104018376201, "rescued": 39, "harmed": 501, "missed_rescue": 5384, "avoided_harm": 2270, "selected_both_correct": 4705, "selected_both_wrong": 329}, {"budget": 0.35, "threshold": -0.0006007538759149611, "operator": "ge", "actual_fraction": 0.16201117318435754, "metric": 0.6562686101579559, "rescued": 46, "harmed": 513, "missed_rescue": 5377, "avoided_harm": 2258, "selected_both_correct": 4777, "selected_both_wrong": 377}, {"budget": 0.36, "threshold": -0.0011453087208792567, "operator": "ge", "actual_fraction": 0.1657544735274934, "metric": 0.6558999517908289, "rescued": 48, "harmed": 528, "missed_rescue": 5375, "avoided_harm": 2243, "selected_both_correct": 4841, "selected_both_wrong": 428}, {"budget": 0.37, "threshold": -0.002608026610687375, "operator": "ge", "actual_fraction": 0.17012165726115192, "metric": 0.6558715934548961, "rescued": 61, "harmed": 542, "missed_rescue": 5362, "avoided_harm": 2229, "selected_both_correct": 4937, "selected_both_wrong": 459}, {"budget": 0.38, "threshold": -0.006219128146767616, "operator": "ge", "actual_fraction": 0.17536794940872868, "metric": 0.6555596517596347, "rescued": 79, "harmed": 571, "missed_rescue": 5344, "avoided_harm": 2200, "selected_both_correct": 5018, "selected_both_wrong": 516}, {"budget": 0.39, "threshold": -0.01280172262340784, "operator": "ge", "actual_fraction": 0.17965005813458867, "metric": 0.6553044267362391, "rescued": 92, "harmed": 593, "missed_rescue": 5331, "avoided_harm": 2178, "selected_both_correct": 5091, "selected_both_wrong": 559}, {"budget": 0.4, "threshold": -0.03794781118631363, "operator": "ge", "actual_fraction": 0.18401724186824717, "metric": 0.6552193517284406, "rescued": 107, "harmed": 611, "missed_rescue": 5316, "avoided_harm": 2160, "selected_both_correct": 5145, "selected_both_wrong": 626}, {"budget": 0.41, "threshold": -0.08479184657335281, "operator": "ge", "actual_fraction": 0.18858293395343562, "metric": 0.6552760684003063, "rescued": 127, "harmed": 629, "missed_rescue": 5296, "avoided_harm": 2142, "selected_both_correct": 5207, "selected_both_wrong": 687}, {"budget": 0.42, "threshold": -0.14941807091236115, "operator": "ge", "actual_fraction": 0.19238295096843716, "metric": 0.6554745767518362, "rescued": 144, "harmed": 639, "missed_rescue": 5279, "avoided_harm": 2132, "selected_both_correct": 5277, "selected_both_wrong": 724}, {"budget": 0.43, "threshold": -0.2939925789833069, "operator": "ge", "actual_fraction": 0.19689192638175992, "metric": 0.655502935087769, "rescued": 168, "harmed": 662, "missed_rescue": 5255, "avoided_harm": 2109, "selected_both_correct": 5344, "selected_both_wrong": 769}, {"budget": 0.44, "threshold": -0.4386065602302551, "operator": "ge", "actual_fraction": 0.20182627683407536, "metric": 0.6555880100955676, "rescued": 199, "harmed": 690, "missed_rescue": 5224, "avoided_harm": 2081, "selected_both_correct": 5429, "selected_both_wrong": 799}, {"budget": 0.45, "threshold": -0.5420411825180054, "operator": "ge", "actual_fraction": 0.20718600232538353, "metric": 0.6559283101267618, "rescued": 240, "harmed": 719, "missed_rescue": 5183, "avoided_harm": 2052, "selected_both_correct": 5506, "selected_both_wrong": 841}, {"budget": 0.46, "threshold": -0.6230618953704834, "operator": "ge", "actual_fraction": 0.21189348609023623, "metric": 0.656070101806426, "rescued": 280, "harmed": 754, "missed_rescue": 5143, "avoided_harm": 2017, "selected_both_correct": 5554, "selected_both_wrong": 884}, {"budget": 0.47, "threshold": -0.6914289593696594, "operator": "ge", "actual_fraction": 0.217139778237813, "metric": 0.6567790602047472, "rescued": 329, "harmed": 778, "missed_rescue": 5094, "avoided_harm": 1993, "selected_both_correct": 5614, "selected_both_wrong": 936}, {"budget": 0.48, "threshold": -0.7387455701828003, "operator": "ge", "actual_fraction": 0.22198905368232993, "metric": 0.6584238436888523, "rescued": 399, "harmed": 790, "missed_rescue": 5024, "avoided_harm": 1981, "selected_both_correct": 5648, "selected_both_wrong": 991}, {"budget": 0.49, "threshold": -0.7823225855827332, "operator": "ge", "actual_fraction": 0.22763236253296656, "metric": 0.6604372855400845, "rescued": 483, "harmed": 803, "missed_rescue": 4940, "avoided_harm": 1968, "selected_both_correct": 5689, "selected_both_wrong": 1052}, {"budget": 0.5, "threshold": -0.8206579089164734, "operator": "ge", "actual_fraction": 0.2345234381646485, "metric": 0.662762669086578, "rescued": 579, "harmed": 817, "missed_rescue": 4844, "avoided_harm": 1954, "selected_both_correct": 5735, "selected_both_wrong": 1139}, {"budget": 0.51, "threshold": -0.8575630187988281, "operator": "ge", "actual_fraction": 0.24382497235062248, "metric": 0.6654567110001985, "rescued": 702, "harmed": 845, "missed_rescue": 4721, "avoided_harm": 1926, "selected_both_correct": 5794, "selected_both_wrong": 1257}, {"budget": 0.52, "threshold": -0.8823798894882202, "operator": "ge", "actual_fraction": 0.252644414825738, "metric": 0.6678955278904234, "rescued": 813, "harmed": 870, "missed_rescue": 4610, "avoided_harm": 1901, "selected_both_correct": 5842, "selected_both_wrong": 1384}, {"budget": 0.53, "threshold": -0.9040287733078003, "operator": "ge", "actual_fraction": 0.26401610753480986, "metric": 0.6703627031165811, "rescued": 937, "harmed": 907, "missed_rescue": 4486, "avoided_harm": 1864, "selected_both_correct": 5946, "selected_both_wrong": 1520}, {"budget": 0.54, "threshold": -0.9208687543869019, "operator": "ge", "actual_fraction": 0.2760116836344043, "metric": 0.6731134617020673, "rescued": 1067, "harmed": 940, "missed_rescue": 4356, "avoided_harm": 1831, "selected_both_correct": 6068, "selected_both_wrong": 1658}, {"budget": 0.55, "threshold": -0.9344452619552612, "operator": "ge", "actual_fraction": 0.2893401015228426, "metric": 0.6758358619516207, "rescued": 1197, "harmed": 974, "missed_rescue": 4226, "avoided_harm": 1797, "selected_both_correct": 6218, "selected_both_wrong": 1814}, {"budget": 0.56, "threshold": -0.9448322653770447, "operator": "ge", "actual_fraction": 0.3036043444970649, "metric": 0.679295578935428, "rescued": 1352, "harmed": 1007, "missed_rescue": 4071, "avoided_harm": 1764, "selected_both_correct": 6385, "selected_both_wrong": 1962}, {"budget": 0.57, "threshold": -0.9531905055046082, "operator": "ge", "actual_fraction": 0.3175850041119587, "metric": 0.6822448458724442, "rescued": 1489, "harmed": 1040, "missed_rescue": 3934, "avoided_harm": 1731, "selected_both_correct": 6573, "selected_both_wrong": 2097}, {"budget": 0.58, "threshold": -0.9603281617164612, "operator": "gt", "actual_fraction": 0.33462836400760004, "metric": 0.6853926211609903, "rescued": 1645, "harmed": 1085, "missed_rescue": 3778, "avoided_harm": 1686, "selected_both_correct": 6816, "selected_both_wrong": 2254}, {"budget": 0.59, "threshold": -0.9670801758766174, "operator": "ge", "actual_fraction": 0.35637920766809406, "metric": 0.6889090548166633, "rescued": 1818, "harmed": 1134, "missed_rescue": 3605, "avoided_harm": 1637, "selected_both_correct": 7175, "selected_both_wrong": 2440}, {"budget": 0.6, "threshold": -0.9724565148353577, "operator": "ge", "actual_fraction": 0.37968975980489467, "metric": 0.6922553384567394, "rescued": 1978, "harmed": 1176, "missed_rescue": 3445, "avoided_harm": 1595, "selected_both_correct": 7569, "selected_both_wrong": 2666}, {"budget": 0.61, "threshold": -0.9766654968261719, "operator": "ge", "actual_fraction": 0.4046450954258004, "metric": 0.6949493803703599, "rescued": 2126, "harmed": 1229, "missed_rescue": 3297, "avoided_harm": 1542, "selected_both_correct": 8013, "selected_both_wrong": 2901}, {"budget": 0.62, "threshold": -0.9801702499389648, "operator": "ge", "actual_fraction": 0.43141536454640844, "metric": 0.6983807390182344, "rescued": 2296, "harmed": 1278, "missed_rescue": 3127, "avoided_harm": 1493, "selected_both_correct": 8524, "selected_both_wrong": 3115}, {"budget": 0.63, "threshold": -0.9831460118293762, "operator": "ge", "actual_fraction": 0.4610214672603012, "metric": 0.7015568726427134, "rescued": 2479, "harmed": 1349, "missed_rescue": 2944, "avoided_harm": 1422, "selected_both_correct": 9071, "selected_both_wrong": 3358}, {"budget": 0.64, "threshold": -0.985751748085022, "operator": "ge", "actual_fraction": 0.4916484700677764, "metric": 0.7053285313217821, "rescued": 2674, "harmed": 1411, "missed_rescue": 2749, "avoided_harm": 1360, "selected_both_correct": 9690, "selected_both_wrong": 3562}, {"budget": 0.65, "threshold": -0.9877713322639465, "operator": "ge", "actual_fraction": 0.5209426310864078, "metric": 0.7086180982899923, "rescued": 2849, "harmed": 1470, "missed_rescue": 2574, "avoided_harm": 1301, "selected_both_correct": 10321, "selected_both_wrong": 3730}, {"budget": 0.66, "threshold": -0.9895955324172974, "operator": "ge", "actual_fraction": 0.5529308340186598, "metric": 0.71148229021921, "rescued": 3014, "harmed": 1534, "missed_rescue": 2409, "avoided_harm": 1237, "selected_both_correct": 11019, "selected_both_wrong": 3931}, {"budget": 0.67, "threshold": -0.9913432002067566, "operator": "ge", "actual_fraction": 0.5888041289737118, "metric": 0.7155091739216742, "rescued": 3227, "harmed": 1605, "missed_rescue": 2196, "avoided_harm": 1166, "selected_both_correct": 11762, "selected_both_wrong": 4169}, {"budget": 0.68, "threshold": -0.9925730228424072, "operator": "ge", "actual_fraction": 0.6175027649377535, "metric": 0.7182882908430933, "rescued": 3404, "harmed": 1684, "missed_rescue": 2019, "avoided_harm": 1087, "selected_both_correct": 12326, "selected_both_wrong": 4361}, {"budget": 0.69, "threshold": -0.9935283660888672, "operator": "ge", "actual_fraction": 0.6421461588633979, "metric": 0.7211241244363781, "rescued": 3553, "harmed": 1733, "missed_rescue": 1870, "avoided_harm": 1038, "selected_both_correct": 12849, "selected_both_wrong": 4509}, {"budget": 0.7, "threshold": -0.99436354637146, "operator": "gt", "actual_fraction": 0.665910444375124, "metric": 0.7233077163032073, "rescued": 3694, "harmed": 1797, "missed_rescue": 1729, "avoided_harm": 974, "selected_both_correct": 13314, "selected_both_wrong": 4677}, {"budget": 0.71, "threshold": -0.9949791431427002, "operator": "ge", "actual_fraction": 0.6838612710206166, "metric": 0.7243286163967898, "rescued": 3794, "harmed": 1861, "missed_rescue": 1629, "avoided_harm": 910, "selected_both_correct": 13678, "selected_both_wrong": 4782}, {"budget": 0.72, "threshold": -0.9955723881721497, "operator": "ge", "actual_fraction": 0.7029464311034228, "metric": 0.7260017582168279, "rescued": 3908, "harmed": 1916, "missed_rescue": 1515, "avoided_harm": 855, "selected_both_correct": 14058, "selected_both_wrong": 4906}, {"budget": 0.73, "threshold": -0.9960037469863892, "operator": "ge", "actual_fraction": 0.7180614241556306, "metric": 0.7278450500524629, "rescued": 4015, "harmed": 1958, "missed_rescue": 1408, "avoided_harm": 813, "selected_both_correct": 14354, "selected_both_wrong": 4994}, {"budget": 0.74, "threshold": -0.9964693188667297, "operator": "ge", "actual_fraction": 0.7352465757309361, "metric": 0.7297450585599637, "rescued": 4134, "harmed": 2010, "missed_rescue": 1289, "avoided_harm": 761, "selected_both_correct": 14689, "selected_both_wrong": 5094}, {"budget": 0.75, "threshold": -0.9968041777610779, "operator": "ge", "actual_fraction": 0.7478093185491875, "metric": 0.7311629753566061, "rescued": 4208, "harmed": 2034, "missed_rescue": 1215, "avoided_harm": 737, "selected_both_correct": 14961, "selected_both_wrong": 5167}, {"budget": 0.76, "threshold": -0.9971042275428772, "operator": "ge", "actual_fraction": 0.759804894648782, "metric": 0.7328928338485098, "rescued": 4299, "harmed": 2064, "missed_rescue": 1124, "avoided_harm": 707, "selected_both_correct": 15186, "selected_both_wrong": 5244}, {"budget": 0.77, "threshold": -0.9974007606506348, "operator": "ge", "actual_fraction": 0.7729064458497575, "metric": 0.7342540339732865, "rescued": 4385, "harmed": 2102, "missed_rescue": 1038, "avoided_harm": 669, "selected_both_correct": 15447, "selected_both_wrong": 5321}, {"budget": 0.78, "threshold": -0.9976577162742615, "operator": "ge", "actual_fraction": 0.7855259053398747, "metric": 0.735274934066869, "rescued": 4462, "harmed": 2143, "missed_rescue": 961, "avoided_harm": 628, "selected_both_correct": 15703, "selected_both_wrong": 5392}, {"budget": 0.79, "threshold": -0.9978766441345215, "operator": "ge", "actual_fraction": 0.7964722230099538, "metric": 0.7362958341604514, "rescued": 4527, "harmed": 2172, "missed_rescue": 896, "avoided_harm": 599, "selected_both_correct": 15920, "selected_both_wrong": 5467}, {"budget": 0.8, "threshold": -0.9980920553207397, "operator": "ge", "actual_fraction": 0.8082125740861527, "metric": 0.7377704676289595, "rescued": 4609, "harmed": 2202, "missed_rescue": 814, "avoided_harm": 569, "selected_both_correct": 16159, "selected_both_wrong": 5530}, {"budget": 0.81, "threshold": -0.9982843399047852, "operator": "ge", "actual_fraction": 0.8197544168108215, "metric": 0.738422709355415, "rescued": 4678, "harmed": 2248, "missed_rescue": 745, "avoided_harm": 523, "selected_both_correct": 16389, "selected_both_wrong": 5592}, {"budget": 0.82, "threshold": -0.9984900951385498, "operator": "ge", "actual_fraction": 0.8333380597226555, "metric": 0.7389048010662734, "rescued": 4746, "harmed": 2299, "missed_rescue": 677, "avoided_harm": 472, "selected_both_correct": 16674, "selected_both_wrong": 5667}, {"budget": 0.83, "threshold": -0.9986395835876465, "operator": "ge", "actual_fraction": 0.8426112355726966, "metric": 0.7398689844879902, "rescued": 4807, "harmed": 2326, "missed_rescue": 616, "avoided_harm": 445, "selected_both_correct": 16857, "selected_both_wrong": 5723}, {"budget": 0.84, "threshold": -0.9987816214561462, "operator": "ge", "actual_fraction": 0.8529903865241187, "metric": 0.7403227178629158, "rescued": 4859, "harmed": 2362, "missed_rescue": 564, "avoided_harm": 409, "selected_both_correct": 17089, "selected_both_wrong": 5769}, {"budget": 0.85, "threshold": -0.9989162087440491, "operator": "ge", "actual_fraction": 0.8635396874911381, "metric": 0.7406913762300428, "rescued": 4914, "harmed": 2404, "missed_rescue": 509, "avoided_harm": 367, "selected_both_correct": 17300, "selected_both_wrong": 5833}, {"budget": 0.86, "threshold": -0.9990254044532776, "operator": "ge", "actual_fraction": 0.8710262881774098, "metric": 0.7413152596205654, "rescued": 4962, "harmed": 2430, "missed_rescue": 461, "avoided_harm": 341, "selected_both_correct": 17455, "selected_both_wrong": 5868}, {"budget": 0.87, "threshold": -0.9991487264633179, "operator": "ge", "actual_fraction": 0.8809517057539064, "metric": 0.7422227263704165, "rescued": 5018, "harmed": 2454, "missed_rescue": 405, "avoided_harm": 317, "selected_both_correct": 17666, "selected_both_wrong": 5927}, {"budget": 0.88, "threshold": -0.9992602467536926, "operator": "ge", "actual_fraction": 0.8913024983693957, "metric": 0.7430167597765364, "rescued": 5074, "harmed": 2482, "missed_rescue": 349, "avoided_harm": 289, "selected_both_correct": 17892, "selected_both_wrong": 5982}, {"budget": 0.89, "threshold": -0.999350905418396, "operator": "ge", "actual_fraction": 0.900915974250631, "metric": 0.743442134815529, "rescued": 5120, "harmed": 2513, "missed_rescue": 303, "avoided_harm": 258, "selected_both_correct": 18112, "selected_both_wrong": 6024}, {"budget": 0.9, "threshold": -0.999435544013977, "operator": "ge", "actual_fraction": 0.910954825170859, "metric": 0.7443212432294473, "rescued": 5172, "harmed": 2534, "missed_rescue": 251, "avoided_harm": 237, "selected_both_correct": 18362, "selected_both_wrong": 6055}, {"budget": 0.91, "threshold": -0.9995256066322327, "operator": "ge", "actual_fraction": 0.921901142840938, "metric": 0.7448033349403057, "rescued": 5223, "harmed": 2568, "missed_rescue": 200, "avoided_harm": 203, "selected_both_correct": 18626, "selected_both_wrong": 6092}, {"budget": 0.92, "threshold": -0.9995898604393005, "operator": "ge", "actual_fraction": 0.9303235686129938, "metric": 0.7450018432918356, "rescued": 5253, "harmed": 2591, "missed_rescue": 170, "avoided_harm": 180, "selected_both_correct": 18846, "selected_both_wrong": 6116}, {"budget": 0.93, "threshold": -0.9996523857116699, "operator": "ge", "actual_fraction": 0.9393698777755721, "metric": 0.7448884099481042, "rescued": 5284, "harmed": 2626, "missed_rescue": 139, "avoided_harm": 145, "selected_both_correct": 19077, "selected_both_wrong": 6138}, {"budget": 0.94, "threshold": -0.9997095465660095, "operator": "ge", "actual_fraction": 0.9490117119927403, "metric": 0.7448884099481042, "rescued": 5314, "harmed": 2656, "missed_rescue": 109, "avoided_harm": 115, "selected_both_correct": 19329, "selected_both_wrong": 6166}, {"budget": 0.95, "threshold": -0.9997634887695312, "operator": "ge", "actual_fraction": 0.9587953378895726, "metric": 0.7449734849559028, "rescued": 5347, "harmed": 2686, "missed_rescue": 76, "avoided_harm": 85, "selected_both_correct": 19588, "selected_both_wrong": 6189}, {"budget": 0.96, "threshold": -0.9998065829277039, "operator": "ge", "actual_fraction": 0.9673595553412926, "metric": 0.7452570683152313, "rescued": 5375, "harmed": 2704, "missed_rescue": 48, "avoided_harm": 67, "selected_both_correct": 19828, "selected_both_wrong": 6205}, {"budget": 0.97, "threshold": -0.9998431205749512, "operator": "ge", "actual_fraction": 0.9746192893401016, "metric": 0.745313784987097, "rescued": 5398, "harmed": 2725, "missed_rescue": 25, "avoided_harm": 46, "selected_both_correct": 20033, "selected_both_wrong": 6212}, {"budget": 0.98, "threshold": -0.9998764991760254, "operator": "ge", "actual_fraction": 0.9830984317840229, "metric": 0.7450585599637013, "rescued": 5411, "harmed": 2747, "missed_rescue": 12, "avoided_harm": 24, "selected_both_correct": 20290, "selected_both_wrong": 6219}, {"budget": 0.99, "threshold": -0.9999123811721802, "operator": "ge", "actual_fraction": 0.9908118991577575, "metric": 0.7448884099481042, "rescued": 5419, "harmed": 2761, "missed_rescue": 4, "avoided_harm": 10, "selected_both_correct": 20535, "selected_both_wrong": 6224}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.7447182599325072, "rescued": 5423, "harmed": 2771, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 20838, "selected_both_wrong": 6231}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.8338042026846582, "actual_fraction": 0.45595614589921996, "metric_std": 0.00507942712956473, "actual_fraction_std": 0.023006871968910717, "per_seed": [{"seed": 2026, "budget": 0.48, "threshold": -0.9890743071566287, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.8369527022278446, "actual_fraction": 0.47993534331295234, "rescue": 1807, "harm": 574, "missed_rescue": 157}, {"seed": 2027, "budget": 0.48, "threshold": -0.9816381658305964, "operator": "ge", "alpha": 1.0, "epoch": 9, "metric": 0.8276055942090097, "actual_fraction": 0.47993534331295234, "rescue": 1714, "harm": 614, "missed_rescue": 250}, {"seed": 2028, "budget": 0.43, "threshold": -0.9910323303127401, "operator": "ge", "alpha": 1.0, "epoch": 2, "metric": 0.8401855365802235, "actual_fraction": 0.4299669688663996, "rescue": 1797, "harm": 518, "missed_rescue": 167}, {"seed": 2029, "budget": 0.44, "threshold": -0.9642834129211594, "operator": "ge", "alpha": 1.0, "epoch": 9, "metric": 0.8342118209290885, "actual_fraction": 0.43994658795417807, "rescue": 1726, "harm": 532, "missed_rescue": 238}, {"seed": 2030, "budget": 0.45, "threshold": -0.9911641218451251, "operator": "ge", "alpha": 1.0, "epoch": 8, "metric": 0.8300653594771242, "actual_fraction": 0.449996486049617, "rescue": 1696, "harm": 561, "missed_rescue": 268}], "rescued": 1748.0, "harmed": 559.8, "missed_rescue": 216.0, "avoided_harm": 611.2}, "validation_curve": [{"budget": 0.0, "metric": 0.7502986857825568, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.7541780870054116, "metric_std": 0.0013235983431991573, "actual_fraction": 0.009979619087778481, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.7566659638765901, "metric_std": 0.0028258803587889727, "actual_fraction": 0.019959238175556962, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.7587602783048704, "metric_std": 0.004349810727612077, "actual_fraction": 0.029924801461803357, "actual_fraction_std": 3.142972770398104e-05}, {"budget": 0.04, "metric": 0.7601939700611428, "metric_std": 0.005359759301807694, "actual_fraction": 0.03998875535877434, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.7621898938786984, "metric_std": 0.00566136558305043, "actual_fraction": 0.049968374446552805, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.7637641436502917, "metric_std": 0.00645087026094694, "actual_fraction": 0.05994799353433129, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.7660552393000211, "metric_std": 0.007025862379210046, "actual_fraction": 0.06999789162977019, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.768768008995713, "metric_std": 0.0069090734471567525, "actual_fraction": 0.07997751071754867, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.7720008433480918, "metric_std": 0.007131570743088253, "actual_fraction": 0.08995712980532716, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.7748541710591047, "metric_std": 0.0067401195788737815, "actual_fraction": 0.09993674889310561, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.7778761683885024, "metric_std": 0.006420555672920232, "actual_fraction": 0.10998664698854452, "actual_fraction_std": 0.0}, {"budget": 0.12, "metric": 0.7806873286949187, "metric_std": 0.0056783959527873405, "actual_fraction": 0.11996626607632302, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.13, "metric": 0.7839342188488299, "metric_std": 0.004802838794659355, "actual_fraction": 0.1299458851641015, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.7866610443460539, "metric_std": 0.004658761439072706, "actual_fraction": 0.13999578325954037, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.7894440930494061, "metric_std": 0.003970608445238772, "actual_fraction": 0.14997540234731885, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.7921709185466301, "metric_std": 0.003336365894037006, "actual_fraction": 0.15995502143509735, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.794897744043854, "metric_std": 0.0026270627723271886, "actual_fraction": 0.16993464052287582, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.7977510717548667, "metric_std": 0.0021543697085210267, "actual_fraction": 0.17997048281678266, "actual_fraction_std": 3.142972770396707e-05}, {"budget": 0.19, "metric": 0.8007168458781362, "metric_std": 0.002115736656850455, "actual_fraction": 0.18995010190456113, "actual_fraction_std": 3.1429727703979486e-05}, {"budget": 0.2, "metric": 0.8038231780167264, "metric_std": 0.0021075501485649955, "actual_fraction": 0.19992972099233958, "actual_fraction_std": 3.142972770400431e-05}, {"budget": 0.21, "metric": 0.8062548316817766, "metric_std": 0.001940897022655915, "actual_fraction": 0.2099936748893106, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.8087848759575514, "metric_std": 0.0022845542441289834, "actual_fraction": 0.21997329397708903, "actual_fraction_std": 0.0}, {"budget": 0.23, "metric": 0.8105559069505939, "metric_std": 0.0029364517231817155, "actual_fraction": 0.22995291306486756, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.24, "metric": 0.8131562302340292, "metric_std": 0.00282719092219934, "actual_fraction": 0.23993253215264604, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.25, "metric": 0.814828870616347, "metric_std": 0.0027684042079590145, "actual_fraction": 0.2499683744465528, "actual_fraction_std": 3.142972770397948e-05}, {"budget": 0.26, "metric": 0.8164733993956006, "metric_std": 0.003126665581712756, "actual_fraction": 0.25996204933586337, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.8185396022208167, "metric_std": 0.0026590181196700113, "actual_fraction": 0.26994166842364187, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.8204371354276478, "metric_std": 0.0035381135633260788, "actual_fraction": 0.27997751071754867, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.29, "metric": 0.8217864923747277, "metric_std": 0.0034481147316708085, "actual_fraction": 0.28997118560685925, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.8230234029095509, "metric_std": 0.0034663299388787445, "actual_fraction": 0.2999508046946377, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.8242603134443742, "metric_std": 0.0030193811641378316, "actual_fraction": 0.3099304237824162, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.825398833368473, "metric_std": 0.0028359125338668924, "actual_fraction": 0.3199803218778551, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.8262984046665262, "metric_std": 0.0029711484290865227, "actual_fraction": 0.3299599409656336, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.8274650361936888, "metric_std": 0.003126823545972994, "actual_fraction": 0.339939560053412, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.8284629981024668, "metric_std": 0.0034609824765930606, "actual_fraction": 0.34998945814885096, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.8291376765760068, "metric_std": 0.0038175800194039456, "actual_fraction": 0.35995502143509733, "actual_fraction_std": 3.142972770396707e-05}, {"budget": 0.37, "metric": 0.8298264108510788, "metric_std": 0.003960519836452524, "actual_fraction": 0.3699486963244079, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.8306697589430037, "metric_std": 0.004286616208110916, "actual_fraction": 0.37999859441984685, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.8313866048211398, "metric_std": 0.00506408896926681, "actual_fraction": 0.3899641577060932, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.4, "metric": 0.8318082788671024, "metric_std": 0.0051845715173080885, "actual_fraction": 0.39995783259540374, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.8323002319207253, "metric_std": 0.0053099964538272666, "actual_fraction": 0.4099374516831823, "actual_fraction_std": 0.0}, {"budget": 0.42, "metric": 0.8324407899360461, "metric_std": 0.005324394392426987, "actual_fraction": 0.41997329397708905, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.43, "metric": 0.8327781291728161, "metric_std": 0.0052273127163553, "actual_fraction": 0.4299669688663997, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.44, "metric": 0.833284138027971, "metric_std": 0.004982727124553848, "actual_fraction": 0.43994658795417807, "actual_fraction_std": 0.0}, {"budget": 0.45, "metric": 0.8332419706233749, "metric_std": 0.004968830215258723, "actual_fraction": 0.44999648604961695, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.46, "metric": 0.8330451894019257, "metric_std": 0.004744379600093259, "actual_fraction": 0.4599761051373955, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.47, "metric": 0.832932742989669, "metric_std": 0.004490823090357092, "actual_fraction": 0.4699557242251739, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.48, "metric": 0.833284138027971, "metric_std": 0.004661093257748563, "actual_fraction": 0.47993534331295234, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.8328905755850726, "metric_std": 0.004342082573692857, "actual_fraction": 0.4899852414083913, "actual_fraction_std": 0.0}, {"budget": 0.5, "metric": 0.832258064516129, "metric_std": 0.004081449966352085, "actual_fraction": 0.49996486049616984, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.51, "metric": 0.8321456181038724, "metric_std": 0.003897983203224346, "actual_fraction": 0.5099444795839483, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.8317801672640381, "metric_std": 0.003596816170895546, "actual_fraction": 0.5199943776793872, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.8315412186379927, "metric_std": 0.0034544116178071263, "actual_fraction": 0.5299739967671656, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.8311617119966266, "metric_std": 0.003274051495189731, "actual_fraction": 0.5399536158549442, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.8310914329889663, "metric_std": 0.0029902045386178346, "actual_fraction": 0.5499332349427226, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.8305151451261509, "metric_std": 0.0028722541207713737, "actual_fraction": 0.5599831330381615, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.8300934710801885, "metric_std": 0.002483047947786163, "actual_fraction": 0.5699627521259399, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.8295734064235013, "metric_std": 0.0026216048146172166, "actual_fraction": 0.5799423712137185, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.8290252301637502, "metric_std": 0.0022856349737623064, "actual_fraction": 0.5899922693091574, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.8287300583315764, "metric_std": 0.002071145130127401, "actual_fraction": 0.5999718883969358, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.8282240494764214, "metric_std": 0.0018187196961614047, "actual_fraction": 0.6099515074847143, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.827465036193689, "metric_std": 0.0015707004454724195, "actual_fraction": 0.6199311265724928, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.8268887483308737, "metric_std": 0.0013576769292058147, "actual_fraction": 0.6299669688663997, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.64, "metric": 0.8266216881017641, "metric_std": 0.0013100957117371036, "actual_fraction": 0.6399606437557102, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.8256518377960502, "metric_std": 0.0013235983431992154, "actual_fraction": 0.6499402628434886, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.8251177173378311, "metric_std": 0.0011565114867228363, "actual_fraction": 0.6599901609389276, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.8245554852765478, "metric_std": 0.0013237849100255694, "actual_fraction": 0.669969780026706, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.8241338112305854, "metric_std": 0.001235991774751961, "actual_fraction": 0.6799493991144846, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.8235715791693021, "metric_std": 0.001242965266395935, "actual_fraction": 0.6899992972099234, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.8231920725279359, "metric_std": 0.0011325609815524638, "actual_fraction": 0.6999789162977019, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.8225455056574603, "metric_std": 0.001369088462912235, "actual_fraction": 0.7099585353854804, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.8220957200084336, "metric_std": 0.0012859330113411192, "actual_fraction": 0.7199381544732588, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.8216599901609388, "metric_std": 0.0012926371091179507, "actual_fraction": 0.7299880525686977, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.8211399255042519, "metric_std": 0.001199899576847578, "actual_fraction": 0.7399676716564763, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.8206339166490968, "metric_std": 0.0013458017094403001, "actual_fraction": 0.7499472907442546, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.8200997961908778, "metric_std": 0.00143203717781433, "actual_fraction": 0.7599971888396937, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.8196640663433833, "metric_std": 0.0015847877270053002, "actual_fraction": 0.769976807927472, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.8190596668775036, "metric_std": 0.0018829003996888101, "actual_fraction": 0.7799564270152506, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.8184131000070278, "metric_std": 0.0017740403936717353, "actual_fraction": 0.789921990301497, "actual_fraction_std": 3.142972770404154e-05}, {"budget": 0.8, "metric": 0.8179492585564692, "metric_std": 0.001822246716114879, "actual_fraction": 0.7999718883969359, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.81, "metric": 0.8174573055028465, "metric_std": 0.0017103952703901486, "actual_fraction": 0.8099655632862464, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.8167404596247101, "metric_std": 0.0017053342765453944, "actual_fraction": 0.8199451823740249, "actual_fraction_std": 0.0}, {"budget": 0.83, "metric": 0.8162344507695553, "metric_std": 0.0016529766027652808, "actual_fraction": 0.8299950804694639, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.84, "metric": 0.8158127767235929, "metric_std": 0.0018268486983633039, "actual_fraction": 0.8399746995572424, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.8153067678684378, "metric_std": 0.0015769769957352687, "actual_fraction": 0.8499543186450207, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.814688312601026, "metric_std": 0.0012387858821198152, "actual_fraction": 0.8599198819312672, "actual_fraction_std": 3.1429727703991894e-05}, {"budget": 0.87, "metric": 0.814168247944339, "metric_std": 0.001026169779939215, "actual_fraction": 0.8699838358282379, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.8134373462646707, "metric_std": 0.0009357934227776585, "actual_fraction": 0.8799634549160166, "actual_fraction_std": 0.0}, {"budget": 0.89, "metric": 0.8130016164171762, "metric_std": 0.0008741194372471055, "actual_fraction": 0.889943074003795, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.8124534401574252, "metric_std": 0.0009771057019573805, "actual_fraction": 0.8999929720992339, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.91, "metric": 0.812102045119123, "metric_std": 0.0012656076607782257, "actual_fraction": 0.9099725911870123, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.8115257572563076, "metric_std": 0.0014551319026231452, "actual_fraction": 0.919952210274791, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.93, "metric": 0.8110338042026847, "metric_std": 0.0012387858821198078, "actual_fraction": 0.9299318293625692, "actual_fraction_std": 0.0}, {"budget": 0.94, "metric": 0.8103731815306767, "metric_std": 0.0011487984674510832, "actual_fraction": 0.9399817274580083, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.8097547262632652, "metric_std": 0.0007966676539992106, "actual_fraction": 0.9499613465457868, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.8092346616065781, "metric_std": 0.0008006873494202611, "actual_fraction": 0.9599409656335652, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.8086162063391665, "metric_std": 0.0006371791106412322, "actual_fraction": 0.9699908637290042, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.98, "metric": 0.8079977510717548, "metric_std": 0.0005043467599412447, "actual_fraction": 0.9799704828167826, "actual_fraction_std": 0.0}, {"budget": 0.99, "metric": 0.8071403471782979, "metric_std": 0.00041696245622863066, "actual_fraction": 0.9899501019045612, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 1.0, "metric": 0.8060299388572634, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.7624365482233502, "actual_fraction": 0.6200153135014037, "metric_std": 0.0012512101456992367, "actual_fraction_std": 0.03516548916697877, "per_seed": [{"seed": 2026, "budget": 0.48, "threshold": -0.9890743071566287, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.7624705782264697, "actual_fraction": 0.6720925616084849, "rescue": 4847, "harm": 1569, "missed_rescue": 576}, {"seed": 2027, "budget": 0.48, "threshold": -0.9816381658305964, "operator": "ge", "alpha": 1.0, "epoch": 9, "metric": 0.7609959447579616, "actual_fraction": 0.621671440319882, "rescue": 4711, "harm": 1485, "missed_rescue": 712}, {"seed": 2028, "budget": 0.43, "threshold": -0.9910323303127401, "operator": "ge", "alpha": 1.0, "epoch": 2, "metric": 0.7627258032498652, "actual_fraction": 0.6292147576780195, "rescue": 4812, "harm": 1525, "missed_rescue": 611}, {"seed": 2029, "budget": 0.44, "threshold": -0.9642834129211594, "operator": "ge", "alpha": 1.0, "epoch": 9, "metric": 0.7643138700621047, "actual_fraction": 0.5976802881206931, "rescue": 4701, "harm": 1358, "missed_rescue": 722}, {"seed": 2030, "budget": 0.45, "threshold": -0.9911641218451251, "operator": "ge", "alpha": 1.0, "epoch": 8, "metric": 0.7616765448203499, "actual_fraction": 0.5794175197799393, "rescue": 4697, "harm": 1447, "missed_rescue": 726}], "rescued": 4753.6, "harmed": 1476.8, "missed_rescue": 669.4, "avoided_harm": 1294.2}, "test_curve": [{"budget": 0.0, "metric": 0.6695119530385957, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.6806851373961376, "metric_std": 0.003175310619708847, "actual_fraction": 0.01987919348892607, "actual_fraction_std": 0.004840934901183682}, {"budget": 0.02, "metric": 0.6892096531775516, "metric_std": 0.005911424195374723, "actual_fraction": 0.039440773615404254, "actual_fraction_std": 0.011579413851929937}, {"budget": 0.03, "metric": 0.6964126705044948, "metric_std": 0.007958533435403975, "actual_fraction": 0.0597510138105096, "actual_fraction_std": 0.01772466323080814}, {"budget": 0.04, "metric": 0.7029407594362362, "metric_std": 0.009353080478305865, "actual_fraction": 0.07992513399313728, "actual_fraction_std": 0.02272529713402944}, {"budget": 0.05, "metric": 0.7085783966196864, "metric_std": 0.00948620459599248, "actual_fraction": 0.09742789893089074, "actual_fraction_std": 0.023387914124506505}, {"budget": 0.06, "metric": 0.7131667753736212, "metric_std": 0.008315198940511049, "actual_fraction": 0.11330856705328532, "actual_fraction_std": 0.020165135916394576}, {"budget": 0.07, "metric": 0.7175339591072795, "metric_std": 0.006596303773164903, "actual_fraction": 0.1289623684882171, "actual_fraction_std": 0.01618483051393864}, {"budget": 0.08, "metric": 0.7218217395003261, "metric_std": 0.005235266797887147, "actual_fraction": 0.14501885829339536, "actual_fraction_std": 0.012723690417940602}, {"budget": 0.09, "metric": 0.7255423531747157, "metric_std": 0.0042760665789532745, "actual_fraction": 0.15882936789269206, "actual_fraction_std": 0.010624377797215258}, {"budget": 0.1, "metric": 0.7289907268241499, "metric_std": 0.003843762959064929, "actual_fraction": 0.17294614752006351, "actual_fraction_std": 0.010236958633687837}, {"budget": 0.11, "metric": 0.7317755154127557, "metric_std": 0.0034184206671273587, "actual_fraction": 0.18605337038822561, "actual_fraction_std": 0.009384946956813323}, {"budget": 0.12, "metric": 0.7342767206420328, "metric_std": 0.0029524963831335034, "actual_fraction": 0.19781640813317075, "actual_fraction_std": 0.009362344734222603}, {"budget": 0.13, "metric": 0.7366304625244591, "metric_std": 0.002790624521805661, "actual_fraction": 0.21022601593738477, "actual_fraction_std": 0.008996241269431765}, {"budget": 0.14, "metric": 0.7387800243881688, "metric_std": 0.0026534826892880767, "actual_fraction": 0.22213651702918075, "actual_fraction_std": 0.009213740739838657}, {"budget": 0.15, "metric": 0.7410203329268639, "metric_std": 0.0023949449983986164, "actual_fraction": 0.23394492811161843, "actual_fraction_std": 0.008722910735739981}, {"budget": 0.16, "metric": 0.7428182514250063, "metric_std": 0.002189476578505238, "actual_fraction": 0.2467175226157729, "actual_fraction_std": 0.008055889008012025}, {"budget": 0.17, "metric": 0.7443042282278876, "metric_std": 0.0021546364305854927, "actual_fraction": 0.2592519070980915, "actual_fraction_std": 0.0075069287876522355}, {"budget": 0.18, "metric": 0.74610214672603, "metric_std": 0.0019536814833429274, "actual_fraction": 0.27111703485239486, "actual_fraction_std": 0.00775279251419728}, {"budget": 0.19, "metric": 0.7473896151773813, "metric_std": 0.0017019018516464192, "actual_fraction": 0.28366843433627315, "actual_fraction_std": 0.007669789685640884}, {"budget": 0.2, "metric": 0.748739471967785, "metric_std": 0.0014968796589143348, "actual_fraction": 0.2962028188185918, "actual_fraction_std": 0.00762942015459885}, {"budget": 0.21, "metric": 0.7499078354082181, "metric_std": 0.0014112368527656692, "actual_fraction": 0.3083174999291042, "actual_fraction_std": 0.007252236215169684}, {"budget": 0.22, "metric": 0.751070527181465, "metric_std": 0.0012335061209160611, "actual_fraction": 0.3215097978050648, "actual_fraction_std": 0.007345427608348847}, {"budget": 0.23, "metric": 0.7522105322859655, "metric_std": 0.001430536811268888, "actual_fraction": 0.3342256756373536, "actual_fraction_std": 0.00775413569808463}, {"budget": 0.24, "metric": 0.7532881490514136, "metric_std": 0.0014580983831231512, "actual_fraction": 0.34700394180869465, "actual_fraction_std": 0.008075919349990534}, {"budget": 0.25, "metric": 0.7542239741371977, "metric_std": 0.0013387717803497877, "actual_fraction": 0.3601962396846553, "actual_fraction_std": 0.008977968793176638}, {"budget": 0.26, "metric": 0.7552675608995264, "metric_std": 0.0013043293224144444, "actual_fraction": 0.37428466097609386, "actual_fraction_std": 0.009692155717111448}, {"budget": 0.27, "metric": 0.7559595042962879, "metric_std": 0.0013226359496397765, "actual_fraction": 0.38759039219578595, "actual_fraction_std": 0.010701686044361726}, {"budget": 0.28, "metric": 0.7568499560445792, "metric_std": 0.001300995646790546, "actual_fraction": 0.40182627683407535, "actual_fraction_std": 0.011750614499609105}, {"budget": 0.29, "metric": 0.7577007061225648, "metric_std": 0.0010722676445616389, "actual_fraction": 0.41473499135070757, "actual_fraction_std": 0.01245601612631104}, {"budget": 0.3, "metric": 0.7582962311771545, "metric_std": 0.00121765952604251, "actual_fraction": 0.42861356095624303, "actual_fraction_std": 0.012580932117190975}, {"budget": 0.31, "metric": 0.7592093695941922, "metric_std": 0.00114368855911726, "actual_fraction": 0.442645265575816, "actual_fraction_std": 0.012928151384364376}, {"budget": 0.32, "metric": 0.7597425063097298, "metric_std": 0.0012366640788593769, "actual_fraction": 0.455911295125202, "actual_fraction_std": 0.01326460064391915}, {"budget": 0.33, "metric": 0.7600771346737374, "metric_std": 0.0011638324637985203, "actual_fraction": 0.4686895612965431, "actual_fraction_std": 0.013434484422015806}, {"budget": 0.34, "metric": 0.76070101806426, "metric_std": 0.0010801511100998055, "actual_fraction": 0.4828857442645266, "actual_fraction_std": 0.01504980922347571}, {"budget": 0.35, "metric": 0.7610866914329467, "metric_std": 0.00118111703176359, "actual_fraction": 0.49637864050137537, "actual_fraction_std": 0.016937850306250127}, {"budget": 0.36, "metric": 0.7613362447891557, "metric_std": 0.001211966403672812, "actual_fraction": 0.5085613816181266, "actual_fraction_std": 0.017892880770571423}, {"budget": 0.37, "metric": 0.7615007231375662, "metric_std": 0.0013769717000862458, "actual_fraction": 0.521810396165953, "actual_fraction_std": 0.019032971330781436}, {"budget": 0.38, "metric": 0.7617843064968948, "metric_std": 0.0015536900535529443, "actual_fraction": 0.5341519439639282, "actual_fraction_std": 0.01963025178160358}, {"budget": 0.39, "metric": 0.7620111731843575, "metric_std": 0.0013556352420967574, "actual_fraction": 0.5466352834415676, "actual_fraction_std": 0.02150309974581159}, {"budget": 0.4, "metric": 0.7621983382015143, "metric_std": 0.0013039593348911675, "actual_fraction": 0.5584039928536992, "actual_fraction_std": 0.023167207117998852}, {"budget": 0.41, "metric": 0.7622890848764994, "metric_std": 0.0012538105159702352, "actual_fraction": 0.569395683861271, "actual_fraction_std": 0.024818660877924392}, {"budget": 0.42, "metric": 0.7624592348920965, "metric_std": 0.0012082448333027476, "actual_fraction": 0.580727674900037, "actual_fraction_std": 0.02660474733907795}, {"budget": 0.43, "metric": 0.7624308765561636, "metric_std": 0.001102078053521725, "actual_fraction": 0.5924396676403029, "actual_fraction_std": 0.028078113837679927}, {"budget": 0.44, "metric": 0.762544309899895, "metric_std": 0.001221319486567495, "actual_fraction": 0.603215835294785, "actual_fraction_std": 0.028780137320276216}, {"budget": 0.45, "metric": 0.7626520715764399, "metric_std": 0.0013494218130673807, "actual_fraction": 0.6151263363865809, "actual_fraction_std": 0.030876904650452827}, {"budget": 0.46, "metric": 0.7625329665655218, "metric_std": 0.0013591899249984069, "actual_fraction": 0.6255905623458016, "actual_fraction_std": 0.03301792971771287}, {"budget": 0.47, "metric": 0.7624592348920964, "metric_std": 0.0012687939096444774, "actual_fraction": 0.6363724016674701, "actual_fraction_std": 0.033885248516717925}, {"budget": 0.48, "metric": 0.7623855032186713, "metric_std": 0.0011056114731939173, "actual_fraction": 0.6471258826532058, "actual_fraction_std": 0.03600385654588272}, {"budget": 0.49, "metric": 0.7624478915577233, "metric_std": 0.0009274953495822915, "actual_fraction": 0.6574766752686951, "actual_fraction_std": 0.0377434462171188}, {"budget": 0.5, "metric": 0.7624365482233502, "metric_std": 0.0008458678883007974, "actual_fraction": 0.6675155261889232, "actual_fraction_std": 0.03814376765419754}, {"budget": 0.51, "metric": 0.7623287865468054, "metric_std": 0.0007518970923383005, "actual_fraction": 0.6773331820888749, "actual_fraction_std": 0.038597676695068694}, {"budget": 0.52, "metric": 0.7623741598842979, "metric_std": 0.000599428413492512, "actual_fraction": 0.6872585996653717, "actual_fraction_std": 0.039516587640257235}, {"budget": 0.53, "metric": 0.7622323682046337, "metric_std": 0.0006567266515171124, "actual_fraction": 0.6963786405013754, "actual_fraction_std": 0.04022538538634575}, {"budget": 0.54, "metric": 0.7620905765249695, "metric_std": 0.0006889966883260858, "actual_fraction": 0.7052321129796104, "actual_fraction_std": 0.040266014073030314}, {"budget": 0.55, "metric": 0.7618296798343873, "metric_std": 0.0008259532689441228, "actual_fraction": 0.7140345404531663, "actual_fraction_std": 0.04115738899021608}, {"budget": 0.56, "metric": 0.7617105748234694, "metric_std": 0.0007509338759680432, "actual_fraction": 0.7222017412018263, "actual_fraction_std": 0.04117098624315265}, {"budget": 0.57, "metric": 0.7616084848141111, "metric_std": 0.0009124600309252318, "actual_fraction": 0.7301307319286504, "actual_fraction_std": 0.04111855347718663}, {"budget": 0.58, "metric": 0.7613702747922751, "metric_std": 0.0009326832077931762, "actual_fraction": 0.7377137509570939, "actual_fraction_std": 0.042266242248414486}, {"budget": 0.59, "metric": 0.7609222130845362, "metric_std": 0.0010424533546256256, "actual_fraction": 0.7455066216714403, "actual_fraction_std": 0.04209161610888662}, {"budget": 0.6, "metric": 0.7608144514079914, "metric_std": 0.0011732950917205211, "actual_fraction": 0.752862774012421, "actual_fraction_std": 0.04278981312033327}, {"budget": 0.61, "metric": 0.7606443013923941, "metric_std": 0.0011051749614416717, "actual_fraction": 0.7606896747298868, "actual_fraction_std": 0.04249511731784607}, {"budget": 0.62, "metric": 0.7604798230439838, "metric_std": 0.0012230960479211776, "actual_fraction": 0.7682329920880243, "actual_fraction_std": 0.04241099072105209}, {"budget": 0.63, "metric": 0.7602302696877746, "metric_std": 0.0013107871989246336, "actual_fraction": 0.7752091427275049, "actual_fraction_std": 0.04261383466155539}, {"budget": 0.64, "metric": 0.7600090746674985, "metric_std": 0.0012219448674884938, "actual_fraction": 0.7818563366701642, "actual_fraction_std": 0.04252574585341542}, {"budget": 0.65, "metric": 0.7598275813175283, "metric_std": 0.0012118668677154915, "actual_fraction": 0.7883957689362788, "actual_fraction_std": 0.042581056077459684}, {"budget": 0.66, "metric": 0.7595893712956924, "metric_std": 0.0012491517038825432, "actual_fraction": 0.7948387828602217, "actual_fraction_std": 0.04199663991771922}, {"budget": 0.67, "metric": 0.7593511612738564, "metric_std": 0.001327703248245402, "actual_fraction": 0.8010209000935824, "actual_fraction_std": 0.04185193987309903}, {"budget": 0.68, "metric": 0.7591356379207667, "metric_std": 0.0013428503024576718, "actual_fraction": 0.8071860023253835, "actual_fraction_std": 0.04114161465175317}, {"budget": 0.69, "metric": 0.7588577262286249, "metric_std": 0.0013668894422658472, "actual_fraction": 0.8134702095681025, "actual_fraction_std": 0.04069562303930431}, {"budget": 0.7, "metric": 0.7585174261974307, "metric_std": 0.001347781939294748, "actual_fraction": 0.8194084451124407, "actual_fraction_std": 0.039956927531879745}, {"budget": 0.71, "metric": 0.7582622011740351, "metric_std": 0.0013909467469211349, "actual_fraction": 0.8254260839973909, "actual_fraction_std": 0.039151864080056706}, {"budget": 0.72, "metric": 0.7579048861412812, "metric_std": 0.001285230320009594, "actual_fraction": 0.8320052179338117, "actual_fraction_std": 0.03861446944093918}, {"budget": 0.73, "metric": 0.7575872727788333, "metric_std": 0.0012502456730779407, "actual_fraction": 0.8382100218359186, "actual_fraction_std": 0.03721919182446757}, {"budget": 0.74, "metric": 0.7572810027507586, "metric_std": 0.0013616431516801921, "actual_fraction": 0.8437228823412642, "actual_fraction_std": 0.03630905031936013}, {"budget": 0.75, "metric": 0.7570427927289225, "metric_std": 0.001430143242857705, "actual_fraction": 0.8494739528684455, "actual_fraction_std": 0.034980929840314395}, {"budget": 0.76, "metric": 0.7566968210305418, "metric_std": 0.0014607433617676651, "actual_fraction": 0.8555256217565151, "actual_fraction_std": 0.03353644266037327}, {"budget": 0.77, "metric": 0.756509656013385, "metric_std": 0.001471822222861949, "actual_fraction": 0.8615602756430253, "actual_fraction_std": 0.03274432193619756}, {"budget": 0.78, "metric": 0.756260102657176, "metric_std": 0.0015060652738877812, "actual_fraction": 0.8671411961546097, "actual_fraction_std": 0.031439068527776705}, {"budget": 0.79, "metric": 0.7559084592916088, "metric_std": 0.0015174492296261936, "actual_fraction": 0.87296032668803, "actual_fraction_std": 0.03086751392713356}, {"budget": 0.8, "metric": 0.7555454725916684, "metric_std": 0.001628247638256894, "actual_fraction": 0.8780761704903156, "actual_fraction_std": 0.03035138656453905}, {"budget": 0.81, "metric": 0.7552448742307802, "metric_std": 0.0016577381865430444, "actual_fraction": 0.8835493293253552, "actual_fraction_std": 0.029832764530544578}, {"budget": 0.82, "metric": 0.7549669625386383, "metric_std": 0.0017534764249051728, "actual_fraction": 0.8881887530839692, "actual_fraction_std": 0.029218689097751264}, {"budget": 0.83, "metric": 0.7546493491761904, "metric_std": 0.0016723485598834603, "actual_fraction": 0.8935031052377844, "actual_fraction_std": 0.028858927128069003}, {"budget": 0.84, "metric": 0.7543771091512348, "metric_std": 0.001622210839625022, "actual_fraction": 0.8984034256869806, "actual_fraction_std": 0.028101698680347516}, {"budget": 0.85, "metric": 0.7540765107903468, "metric_std": 0.0016848765876429175, "actual_fraction": 0.9041771828829084, "actual_fraction_std": 0.027055717704505374}, {"budget": 0.86, "metric": 0.7537135240904063, "metric_std": 0.0015135755396829822, "actual_fraction": 0.9097524317273061, "actual_fraction_std": 0.025473470943408134}, {"budget": 0.87, "metric": 0.7533675523920256, "metric_std": 0.001556482602167957, "actual_fraction": 0.9147378271843009, "actual_fraction_std": 0.024435660794696064}, {"budget": 0.88, "metric": 0.7528797890139807, "metric_std": 0.0014622290597301943, "actual_fraction": 0.9204605393755495, "actual_fraction_std": 0.023162246148749715}, {"budget": 0.89, "metric": 0.7524600856421746, "metric_std": 0.0014573536179598545, "actual_fraction": 0.9253948898278649, "actual_fraction_std": 0.0224548459644044}, {"budget": 0.9, "metric": 0.7521084422766072, "metric_std": 0.0015429384629759384, "actual_fraction": 0.9307262569832403, "actual_fraction_std": 0.021843894374773024}, {"budget": 0.91, "metric": 0.751796500581346, "metric_std": 0.0015226076222134564, "actual_fraction": 0.9359555341292574, "actual_fraction_std": 0.02032611893247424}, {"budget": 0.92, "metric": 0.7513030655361144, "metric_std": 0.001330939819909972, "actual_fraction": 0.9413322746221251, "actual_fraction_std": 0.018948116131351465}, {"budget": 0.93, "metric": 0.7508550038283752, "metric_std": 0.001345841330536271, "actual_fraction": 0.9471230468196126, "actual_fraction_std": 0.0176715966419986}, {"budget": 0.94, "metric": 0.7501914187675467, "metric_std": 0.0012131270530702008, "actual_fraction": 0.9537248674247796, "actual_fraction_std": 0.015803474969149738}, {"budget": 0.95, "metric": 0.7496526103848227, "metric_std": 0.0011841090999695414, "actual_fraction": 0.9604854947111704, "actual_fraction_std": 0.014298769605183172}, {"budget": 0.96, "metric": 0.7490400703286731, "metric_std": 0.001090893249561738, "actual_fraction": 0.9669682103054195, "actual_fraction_std": 0.012649111688425944}, {"budget": 0.97, "metric": 0.7483651419334713, "metric_std": 0.000963891453464428, "actual_fraction": 0.9737118225902505, "actual_fraction_std": 0.011376005752724263}, {"budget": 0.98, "metric": 0.7475767801945382, "metric_std": 0.000858606868895236, "actual_fraction": 0.9810112582593653, "actual_fraction_std": 0.00883362391903167}, {"budget": 0.99, "metric": 0.7465048350962764, "metric_std": 0.0005703481088143168, "actual_fraction": 0.9892124890111447, "actual_fraction_std": 0.0052645247665775476}, {"budget": 1.0, "metric": 0.7447182599325072, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 9505, "rescuable": 1964, "harmful": 1171, "both_wrong": 1589, "unit": "sample", "total": 14229}, "test": {"both_correct": 20838, "rescuable": 5423, "harmful": 2771, "both_wrong": 6231, "unit": "sample", "total": 35263}}, "small": {"validation": 0.7502986857825568, "test": 0.6695119530385957}, "large": {"validation": 0.8060299388572634, "test": 0.7447182599325072}, "learned_minus_confidence_test": 0.03810793182656036, "learned_minus_small_test": 0.09292459518475449, "learned_minus_large_test": 0.017718288290842987, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_instancevg_Qwen3.8"}, {"key": ["nlvr2", "nlvr2", "MiniCPM"], "expert_display": "BEiT3", "run": "outputs/full_budget_20260913/runs/full_nlvr2_MiniCPM_s{2026..2030}", "validation_rows": 6982, "test_rows": 13937, "confidence_rule": {"column": "max_probability", "columns": ["entropy", "max_probability", "top_two_margin", "entropy_valid", "max_probability_valid", "top_two_margin_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/nlvr2/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 86, "validation_area_0_25": 0.8284302492122602, "validation_point": {"budget": 0.86, "threshold": -0.9997387528419495, "operator": "gt", "actual_fraction": 0.8587797192781438, "metric": 0.9004583213978803, "rescued": 1073, "harmed": 359, "missed_rescue": 14, "avoided_harm": 24, "selected_both_correct": 4254, "selected_both_wrong": 310}, "test_point": {"budget": 0.86, "threshold": -0.9997387528419495, "operator": "gt", "actual_fraction": 0.8580038745784602, "metric": 0.9077276314845376, "rescued": 2191, "harmed": 690, "missed_rescue": 34, "avoided_harm": 39, "selected_both_correct": 8533, "selected_both_wrong": 544}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.7981953594958464, "rescued": 0, "harmed": 0, "missed_rescue": 1087, "avoided_harm": 383, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.5245773792266846, "operator": "gt", "actual_fraction": 0.0097393297049556, "metric": 0.8009166427957606, "rescued": 26, "harmed": 7, "missed_rescue": 1061, "avoided_harm": 376, "selected_both_correct": 33, "selected_both_wrong": 2}, {"budget": 0.02, "threshold": -0.5456489324569702, "operator": "gt", "actual_fraction": 0.019765110283586365, "metric": 0.8037811515325122, "rescued": 51, "harmed": 12, "missed_rescue": 1036, "avoided_harm": 371, "selected_both_correct": 63, "selected_both_wrong": 12}, {"budget": 0.03, "threshold": -0.5652987360954285, "operator": "ge", "actual_fraction": 0.029934116299054712, "metric": 0.8066456602692639, "rescued": 80, "harmed": 21, "missed_rescue": 1007, "avoided_harm": 362, "selected_both_correct": 86, "selected_both_wrong": 22}, {"budget": 0.04, "threshold": -0.5851011276245117, "operator": "ge", "actual_fraction": 0.039959896877685475, "metric": 0.8083643655113149, "rescued": 101, "harmed": 30, "missed_rescue": 986, "avoided_harm": 353, "selected_both_correct": 118, "selected_both_wrong": 30}, {"budget": 0.05, "threshold": -0.6055658459663391, "operator": "ge", "actual_fraction": 0.04998567745631624, "metric": 0.809653394442853, "rescued": 121, "harmed": 41, "missed_rescue": 966, "avoided_harm": 342, "selected_both_correct": 151, "selected_both_wrong": 36}, {"budget": 0.06, "threshold": -0.6293190717697144, "operator": "ge", "actual_fraction": 0.059868232598109423, "metric": 0.8125179031796047, "rescued": 145, "harmed": 45, "missed_rescue": 942, "avoided_harm": 338, "selected_both_correct": 183, "selected_both_wrong": 45}, {"budget": 0.07, "threshold": -0.6477982401847839, "operator": "ge", "actual_fraction": 0.06989401317674018, "metric": 0.8168146662847322, "rescued": 177, "harmed": 47, "missed_rescue": 910, "avoided_harm": 336, "selected_both_correct": 212, "selected_both_wrong": 52}, {"budget": 0.08, "threshold": -0.6693127155303955, "operator": "ge", "actual_fraction": 0.07991979375537095, "metric": 0.8188198224004584, "rescued": 201, "harmed": 57, "missed_rescue": 886, "avoided_harm": 326, "selected_both_correct": 239, "selected_both_wrong": 61}, {"budget": 0.09, "threshold": -0.6888871788978577, "operator": "gt", "actual_fraction": 0.08951589802348897, "metric": 0.8205385276425093, "rescued": 222, "harmed": 66, "missed_rescue": 865, "avoided_harm": 317, "selected_both_correct": 267, "selected_both_wrong": 70}, {"budget": 0.1, "threshold": -0.7061904072761536, "operator": "gt", "actual_fraction": 0.09925522772844457, "metric": 0.8222572328845603, "rescued": 240, "harmed": 72, "missed_rescue": 847, "avoided_harm": 311, "selected_both_correct": 305, "selected_both_wrong": 76}, {"budget": 0.11, "threshold": -0.7233095169067383, "operator": "ge", "actual_fraction": 0.10999713549126325, "metric": 0.8254081924949871, "rescued": 270, "harmed": 80, "missed_rescue": 817, "avoided_harm": 303, "selected_both_correct": 339, "selected_both_wrong": 79}, {"budget": 0.12, "threshold": -0.7424216866493225, "operator": "ge", "actual_fraction": 0.11987969063305644, "metric": 0.8282727012317388, "rescued": 296, "harmed": 86, "missed_rescue": 791, "avoided_harm": 297, "selected_both_correct": 369, "selected_both_wrong": 86}, {"budget": 0.13, "threshold": -0.7627776861190796, "operator": "gt", "actual_fraction": 0.1297622457748496, "metric": 0.8308507590948152, "rescued": 320, "harmed": 92, "missed_rescue": 767, "avoided_harm": 291, "selected_both_correct": 404, "selected_both_wrong": 90}, {"budget": 0.14, "threshold": -0.7786493301391602, "operator": "gt", "actual_fraction": 0.1396448009166428, "metric": 0.8337152678315669, "rescued": 346, "harmed": 98, "missed_rescue": 741, "avoided_harm": 285, "selected_both_correct": 436, "selected_both_wrong": 95}, {"budget": 0.15, "threshold": -0.7950222492218018, "operator": "ge", "actual_fraction": 0.14995703236894872, "metric": 0.8362933256946433, "rescued": 367, "harmed": 101, "missed_rescue": 720, "avoided_harm": 282, "selected_both_correct": 474, "selected_both_wrong": 105}, {"budget": 0.16, "threshold": -0.8140527606010437, "operator": "ge", "actual_fraction": 0.1599828129475795, "metric": 0.8394442853050702, "rescued": 392, "harmed": 104, "missed_rescue": 695, "avoided_harm": 279, "selected_both_correct": 513, "selected_both_wrong": 108}, {"budget": 0.17, "threshold": -0.8327813148498535, "operator": "ge", "actual_fraction": 0.16986536808937266, "metric": 0.840876539673446, "rescued": 409, "harmed": 111, "missed_rescue": 678, "avoided_harm": 272, "selected_both_correct": 552, "selected_both_wrong": 114}, {"budget": 0.18, "threshold": -0.8464508652687073, "operator": "gt", "actual_fraction": 0.17960469779432828, "metric": 0.8428816957891722, "rescued": 426, "harmed": 114, "missed_rescue": 661, "avoided_harm": 269, "selected_both_correct": 596, "selected_both_wrong": 118}, {"budget": 0.19, "threshold": -0.858244001865387, "operator": "ge", "actual_fraction": 0.1899169292466342, "metric": 0.8451733027785735, "rescued": 449, "harmed": 121, "missed_rescue": 638, "avoided_harm": 262, "selected_both_correct": 631, "selected_both_wrong": 125}, {"budget": 0.2, "threshold": -0.8701565861701965, "operator": "ge", "actual_fraction": 0.19994270982526496, "metric": 0.8467487825837868, "rescued": 467, "harmed": 128, "missed_rescue": 620, "avoided_harm": 255, "selected_both_correct": 672, "selected_both_wrong": 129}, {"budget": 0.21, "threshold": -0.8840392827987671, "operator": "gt", "actual_fraction": 0.2093955886565454, "metric": 0.8481810369521627, "rescued": 483, "harmed": 134, "missed_rescue": 604, "avoided_harm": 249, "selected_both_correct": 712, "selected_both_wrong": 133}, {"budget": 0.22, "threshold": -0.8933094143867493, "operator": "ge", "actual_fraction": 0.2199942709825265, "metric": 0.8497565167573761, "rescued": 500, "harmed": 140, "missed_rescue": 587, "avoided_harm": 243, "selected_both_correct": 759, "selected_both_wrong": 137}, {"budget": 0.23, "threshold": -0.9046505093574524, "operator": "gt", "actual_fraction": 0.2297336006874821, "metric": 0.851475221999427, "rescued": 516, "harmed": 144, "missed_rescue": 571, "avoided_harm": 239, "selected_both_correct": 797, "selected_both_wrong": 147}, {"budget": 0.24, "threshold": -0.9136765599250793, "operator": "ge", "actual_fraction": 0.23990260670295044, "metric": 0.8543397307361787, "rescued": 540, "harmed": 148, "missed_rescue": 547, "avoided_harm": 235, "selected_both_correct": 834, "selected_both_wrong": 153}, {"budget": 0.25, "threshold": -0.9230391979217529, "operator": "gt", "actual_fraction": 0.24906903466055572, "metric": 0.8562016614150673, "rescued": 555, "harmed": 150, "missed_rescue": 532, "avoided_harm": 233, "selected_both_correct": 874, "selected_both_wrong": 160}, {"budget": 0.26, "threshold": -0.9314625263214111, "operator": "gt", "actual_fraction": 0.2595244915496992, "metric": 0.8589229447149814, "rescued": 581, "harmed": 157, "missed_rescue": 506, "avoided_harm": 226, "selected_both_correct": 913, "selected_both_wrong": 161}, {"budget": 0.27, "threshold": -0.9376689791679382, "operator": "ge", "actual_fraction": 0.26997994843884277, "metric": 0.8607848753938699, "rescued": 601, "harmed": 164, "missed_rescue": 486, "avoided_harm": 219, "selected_both_correct": 957, "selected_both_wrong": 163}, {"budget": 0.28, "threshold": -0.9449946880340576, "operator": "gt", "actual_fraction": 0.2794328272701232, "metric": 0.8625035806359209, "rescued": 616, "harmed": 167, "missed_rescue": 471, "avoided_harm": 216, "selected_both_correct": 1000, "selected_both_wrong": 168}, {"budget": 0.29, "threshold": -0.9511421918869019, "operator": "gt", "actual_fraction": 0.2896018332855915, "metric": 0.8642222858779719, "rescued": 633, "harmed": 172, "missed_rescue": 454, "avoided_harm": 211, "selected_both_correct": 1041, "selected_both_wrong": 176}, {"budget": 0.3, "threshold": -0.9559813141822815, "operator": "gt", "actual_fraction": 0.29977083930105985, "metric": 0.8660842165568605, "rescued": 652, "harmed": 178, "missed_rescue": 435, "avoided_harm": 205, "selected_both_correct": 1082, "selected_both_wrong": 181}, {"budget": 0.31, "threshold": -0.9597620368003845, "operator": "ge", "actual_fraction": 0.30993984531652824, "metric": 0.8672300200515611, "rescued": 664, "harmed": 182, "missed_rescue": 423, "avoided_harm": 201, "selected_both_correct": 1131, "selected_both_wrong": 187}, {"budget": 0.32, "threshold": -0.9643214344978333, "operator": "gt", "actual_fraction": 0.3196791750214838, "metric": 0.8692351761672873, "rescued": 682, "harmed": 186, "missed_rescue": 405, "avoided_harm": 197, "selected_both_correct": 1172, "selected_both_wrong": 192}, {"budget": 0.33, "threshold": -0.9686195850372314, "operator": "gt", "actual_fraction": 0.3297049556001146, "metric": 0.8695216270409625, "rescued": 693, "harmed": 195, "missed_rescue": 394, "avoided_harm": 188, "selected_both_correct": 1217, "selected_both_wrong": 197}, {"budget": 0.34, "threshold": -0.9719924926757812, "operator": "gt", "actual_fraction": 0.339157834431395, "metric": 0.8712403322830135, "rescued": 709, "harmed": 199, "missed_rescue": 378, "avoided_harm": 184, "selected_both_correct": 1257, "selected_both_wrong": 203}, {"budget": 0.35, "threshold": -0.974821150302887, "operator": "ge", "actual_fraction": 0.3498997421942137, "metric": 0.872099684904039, "rescued": 722, "harmed": 206, "missed_rescue": 365, "avoided_harm": 177, "selected_both_correct": 1305, "selected_both_wrong": 210}, {"budget": 0.36, "threshold": -0.9777138829231262, "operator": "gt", "actual_fraction": 0.3587797192781438, "metric": 0.8739616155829275, "rescued": 739, "harmed": 210, "missed_rescue": 348, "avoided_harm": 173, "selected_both_correct": 1340, "selected_both_wrong": 216}, {"budget": 0.37, "threshold": -0.9802808165550232, "operator": "gt", "actual_fraction": 0.3685190489830994, "metric": 0.8746777427671154, "rescued": 749, "harmed": 215, "missed_rescue": 338, "avoided_harm": 168, "selected_both_correct": 1389, "selected_both_wrong": 220}, {"budget": 0.38, "threshold": -0.982557475566864, "operator": "gt", "actual_fraction": 0.3795474076195932, "metric": 0.8771125751933544, "rescued": 769, "harmed": 218, "missed_rescue": 318, "avoided_harm": 165, "selected_both_correct": 1439, "selected_both_wrong": 224}, {"budget": 0.39, "threshold": -0.9843363761901855, "operator": "gt", "actual_fraction": 0.3892867373245488, "metric": 0.8781151532512174, "rescued": 780, "harmed": 222, "missed_rescue": 307, "avoided_harm": 161, "selected_both_correct": 1488, "selected_both_wrong": 228}, {"budget": 0.4, "threshold": -0.9863631129264832, "operator": "gt", "actual_fraction": 0.3993125179031796, "metric": 0.8795474076195933, "rescued": 796, "harmed": 228, "missed_rescue": 291, "avoided_harm": 155, "selected_both_correct": 1535, "selected_both_wrong": 229}, {"budget": 0.41, "threshold": -0.9879462122917175, "operator": "gt", "actual_fraction": 0.40962474935548554, "metric": 0.8811228874248066, "rescued": 812, "harmed": 233, "missed_rescue": 275, "avoided_harm": 150, "selected_both_correct": 1583, "selected_both_wrong": 232}, {"budget": 0.42, "threshold": -0.9890130758285522, "operator": "gt", "actual_fraction": 0.41850472643941566, "metric": 0.8825551417931825, "rescued": 826, "harmed": 237, "missed_rescue": 261, "avoided_harm": 146, "selected_both_correct": 1624, "selected_both_wrong": 235}, {"budget": 0.43, "threshold": -0.9899863600730896, "operator": "gt", "actual_fraction": 0.4270982526496706, "metric": 0.8832712689773704, "rescued": 834, "harmed": 240, "missed_rescue": 253, "avoided_harm": 143, "selected_both_correct": 1671, "selected_both_wrong": 237}, {"budget": 0.44, "threshold": -0.9911527037620544, "operator": "gt", "actual_fraction": 0.4395588656545402, "metric": 0.8864222285877972, "rescued": 859, "harmed": 243, "missed_rescue": 228, "avoided_harm": 140, "selected_both_correct": 1727, "selected_both_wrong": 240}, {"budget": 0.45, "threshold": -0.9919379353523254, "operator": "gt", "actual_fraction": 0.4497278716700086, "metric": 0.8865654540246348, "rescued": 866, "harmed": 249, "missed_rescue": 221, "avoided_harm": 134, "selected_both_correct": 1781, "selected_both_wrong": 244}, {"budget": 0.46, "threshold": -0.9926542043685913, "operator": "gt", "actual_fraction": 0.4586078487539387, "metric": 0.8877112575193354, "rescued": 876, "harmed": 251, "missed_rescue": 211, "avoided_harm": 132, "selected_both_correct": 1828, "selected_both_wrong": 247}, {"budget": 0.47, "threshold": -0.9933071732521057, "operator": "gt", "actual_fraction": 0.46863362933256947, "metric": 0.888570610140361, "rescued": 886, "harmed": 255, "missed_rescue": 201, "avoided_harm": 128, "selected_both_correct": 1882, "selected_both_wrong": 249}, {"budget": 0.48, "threshold": -0.9939024448394775, "operator": "gt", "actual_fraction": 0.4798052134059009, "metric": 0.8882841592666858, "rescued": 893, "harmed": 264, "missed_rescue": 194, "avoided_harm": 119, "selected_both_correct": 1942, "selected_both_wrong": 251}, {"budget": 0.49, "threshold": -0.9944451451301575, "operator": "gt", "actual_fraction": 0.4891148668003437, "metric": 0.8897164136350616, "rescued": 908, "harmed": 269, "missed_rescue": 179, "avoided_harm": 114, "selected_both_correct": 1986, "selected_both_wrong": 252}, {"budget": 0.5, "threshold": -0.9949396848678589, "operator": "gt", "actual_fraction": 0.4992838728158121, "metric": 0.890289315382412, "rescued": 915, "harmed": 272, "missed_rescue": 172, "avoided_harm": 111, "selected_both_correct": 2043, "selected_both_wrong": 256}, {"budget": 0.51, "threshold": -0.9952448010444641, "operator": "gt", "actual_fraction": 0.5065883700945288, "metric": 0.890289315382412, "rescued": 918, "harmed": 275, "missed_rescue": 169, "avoided_harm": 108, "selected_both_correct": 2085, "selected_both_wrong": 259}, {"budget": 0.52, "threshold": -0.9956685304641724, "operator": "gt", "actual_fraction": 0.5179031796046978, "metric": 0.890575766256087, "rescued": 923, "harmed": 278, "missed_rescue": 164, "avoided_harm": 105, "selected_both_correct": 2153, "selected_both_wrong": 262}, {"budget": 0.53, "threshold": -0.9961755275726318, "operator": "gt", "actual_fraction": 0.528788312804354, "metric": 0.8911486680034374, "rescued": 931, "harmed": 282, "missed_rescue": 156, "avoided_harm": 101, "selected_both_correct": 2215, "selected_both_wrong": 264}, {"budget": 0.54, "threshold": -0.9965165853500366, "operator": "gt", "actual_fraction": 0.5380979661987969, "metric": 0.8921512460613005, "rescued": 939, "harmed": 283, "missed_rescue": 148, "avoided_harm": 100, "selected_both_correct": 2270, "selected_both_wrong": 265}, {"budget": 0.55, "threshold": -0.9968273043632507, "operator": "gt", "actual_fraction": 0.5489830993984531, "metric": 0.8920080206244629, "rescued": 943, "harmed": 288, "missed_rescue": 144, "avoided_harm": 95, "selected_both_correct": 2337, "selected_both_wrong": 265}, {"budget": 0.56, "threshold": -0.9971103668212891, "operator": "gt", "actual_fraction": 0.5592953308507591, "metric": 0.8922944714981381, "rescued": 950, "harmed": 293, "missed_rescue": 137, "avoided_harm": 90, "selected_both_correct": 2393, "selected_both_wrong": 269}, {"budget": 0.57, "threshold": -0.997285008430481, "operator": "gt", "actual_fraction": 0.5690346605557147, "metric": 0.8925809223718132, "rescued": 957, "harmed": 298, "missed_rescue": 130, "avoided_harm": 85, "selected_both_correct": 2445, "selected_both_wrong": 273}, {"budget": 0.58, "threshold": -0.9974491000175476, "operator": "gt", "actual_fraction": 0.5783443139501575, "metric": 0.8928673732454884, "rescued": 961, "harmed": 300, "missed_rescue": 126, "avoided_harm": 83, "selected_both_correct": 2499, "selected_both_wrong": 278}, {"budget": 0.59, "threshold": -0.9976769089698792, "operator": "gt", "actual_fraction": 0.5893726725866514, "metric": 0.8927241478086508, "rescued": 965, "harmed": 305, "missed_rescue": 122, "avoided_harm": 78, "selected_both_correct": 2565, "selected_both_wrong": 280}, {"budget": 0.6, "threshold": -0.9978172779083252, "operator": "gt", "actual_fraction": 0.5976797479232312, "metric": 0.8935835004296763, "rescued": 972, "harmed": 306, "missed_rescue": 115, "avoided_harm": 77, "selected_both_correct": 2614, "selected_both_wrong": 281}, {"budget": 0.61, "threshold": -0.9980121850967407, "operator": "gt", "actual_fraction": 0.6084216556860499, "metric": 0.8944428530507018, "rescued": 980, "harmed": 308, "missed_rescue": 107, "avoided_harm": 75, "selected_both_correct": 2678, "selected_both_wrong": 282}, {"budget": 0.62, "threshold": -0.9981897473335266, "operator": "gt", "actual_fraction": 0.6187338871383558, "metric": 0.8941564021770266, "rescued": 982, "harmed": 312, "missed_rescue": 105, "avoided_harm": 71, "selected_both_correct": 2743, "selected_both_wrong": 283}, {"budget": 0.63, "threshold": -0.9982993006706238, "operator": "gt", "actual_fraction": 0.6280435405327986, "metric": 0.8954454311085649, "rescued": 992, "harmed": 313, "missed_rescue": 95, "avoided_harm": 70, "selected_both_correct": 2796, "selected_both_wrong": 284}, {"budget": 0.64, "threshold": -0.9984021782875061, "operator": "gt", "actual_fraction": 0.6360641649957033, "metric": 0.8958751074190776, "rescued": 997, "harmed": 315, "missed_rescue": 90, "avoided_harm": 68, "selected_both_correct": 2843, "selected_both_wrong": 286}, {"budget": 0.65, "threshold": -0.9985449314117432, "operator": "gt", "actual_fraction": 0.6465196218848468, "metric": 0.8961615582927528, "rescued": 1000, "harmed": 316, "missed_rescue": 87, "avoided_harm": 67, "selected_both_correct": 2912, "selected_both_wrong": 286}, {"budget": 0.66, "threshold": -0.9986749291419983, "operator": "gt", "actual_fraction": 0.6586937840160413, "metric": 0.8971641363506159, "rescued": 1011, "harmed": 320, "missed_rescue": 76, "avoided_harm": 63, "selected_both_correct": 2982, "selected_both_wrong": 286}, {"budget": 0.67, "threshold": -0.9987550973892212, "operator": "gt", "actual_fraction": 0.6672873102262962, "metric": 0.8973073617874534, "rescued": 1015, "harmed": 323, "missed_rescue": 72, "avoided_harm": 60, "selected_both_correct": 3034, "selected_both_wrong": 287}, {"budget": 0.68, "threshold": -0.9988664388656616, "operator": "gt", "actual_fraction": 0.6793182469206531, "metric": 0.8977370380979662, "rescued": 1023, "harmed": 328, "missed_rescue": 64, "avoided_harm": 55, "selected_both_correct": 3102, "selected_both_wrong": 290}, {"budget": 0.69, "threshold": -0.9989677667617798, "operator": "gt", "actual_fraction": 0.6894872529361215, "metric": 0.8984531652821541, "rescued": 1028, "harmed": 328, "missed_rescue": 59, "avoided_harm": 55, "selected_both_correct": 3166, "selected_both_wrong": 292}, {"budget": 0.7, "threshold": -0.9990302324295044, "operator": "gt", "actual_fraction": 0.6969349756516757, "metric": 0.8975938126611286, "rescued": 1028, "harmed": 334, "missed_rescue": 59, "avoided_harm": 49, "selected_both_correct": 3212, "selected_both_wrong": 292}, {"budget": 0.71, "threshold": -0.9991169571876526, "operator": "gt", "actual_fraction": 0.7098252649670581, "metric": 0.8977370380979662, "rescued": 1031, "harmed": 336, "missed_rescue": 56, "avoided_harm": 47, "selected_both_correct": 3292, "selected_both_wrong": 297}, {"budget": 0.72, "threshold": -0.9991704225540161, "operator": "gt", "actual_fraction": 0.7189916929246635, "metric": 0.8978802635348038, "rescued": 1034, "harmed": 338, "missed_rescue": 53, "avoided_harm": 45, "selected_both_correct": 3351, "selected_both_wrong": 297}, {"budget": 0.73, "threshold": -0.9992445707321167, "operator": "gt", "actual_fraction": 0.7295903752506445, "metric": 0.898166714408479, "rescued": 1038, "harmed": 340, "missed_rescue": 49, "avoided_harm": 43, "selected_both_correct": 3419, "selected_both_wrong": 297}, {"budget": 0.74, "threshold": -0.9992903470993042, "operator": "gt", "actual_fraction": 0.7384703523345746, "metric": 0.8985963907189917, "rescued": 1044, "harmed": 343, "missed_rescue": 43, "avoided_harm": 40, "selected_both_correct": 3471, "selected_both_wrong": 298}, {"budget": 0.75, "threshold": -0.9993333220481873, "operator": "gt", "actual_fraction": 0.7484961329132054, "metric": 0.8988828415926668, "rescued": 1049, "harmed": 346, "missed_rescue": 38, "avoided_harm": 37, "selected_both_correct": 3533, "selected_both_wrong": 298}, {"budget": 0.76, "threshold": -0.999373733997345, "operator": "gt", "actual_fraction": 0.7553709538814093, "metric": 0.8987396161558293, "rescued": 1050, "harmed": 348, "missed_rescue": 37, "avoided_harm": 35, "selected_both_correct": 3578, "selected_both_wrong": 298}, {"budget": 0.77, "threshold": -0.9994296431541443, "operator": "gt", "actual_fraction": 0.7694070466914924, "metric": 0.8994557433400172, "rescued": 1056, "harmed": 349, "missed_rescue": 31, "avoided_harm": 34, "selected_both_correct": 3667, "selected_both_wrong": 300}, {"budget": 0.78, "threshold": -0.999464213848114, "operator": "gt", "actual_fraction": 0.7782870237754225, "metric": 0.8995989687768547, "rescued": 1058, "harmed": 350, "missed_rescue": 29, "avoided_harm": 33, "selected_both_correct": 3723, "selected_both_wrong": 303}, {"budget": 0.79, "threshold": -0.9994966983795166, "operator": "gt", "actual_fraction": 0.7885992552277284, "metric": 0.8998854196505299, "rescued": 1060, "harmed": 350, "missed_rescue": 27, "avoided_harm": 33, "selected_both_correct": 3792, "selected_both_wrong": 304}, {"budget": 0.8, "threshold": -0.9995417594909668, "operator": "gt", "actual_fraction": 0.7977656831853337, "metric": 0.8997421942136924, "rescued": 1061, "harmed": 352, "missed_rescue": 26, "avoided_harm": 31, "selected_both_correct": 3852, "selected_both_wrong": 305}, {"budget": 0.81, "threshold": -0.9995827078819275, "operator": "gt", "actual_fraction": 0.8090804926955028, "metric": 0.9001718705242051, "rescued": 1064, "harmed": 352, "missed_rescue": 23, "avoided_harm": 31, "selected_both_correct": 3928, "selected_both_wrong": 305}, {"budget": 0.82, "threshold": -0.9996140599250793, "operator": "gt", "actual_fraction": 0.8198224004583214, "metric": 0.9003150959610426, "rescued": 1066, "harmed": 353, "missed_rescue": 21, "avoided_harm": 30, "selected_both_correct": 3998, "selected_both_wrong": 307}, {"budget": 0.83, "threshold": -0.9996429681777954, "operator": "gt", "actual_fraction": 0.8281294757949011, "metric": 0.9001718705242051, "rescued": 1066, "harmed": 354, "missed_rescue": 21, "avoided_harm": 29, "selected_both_correct": 4054, "selected_both_wrong": 308}, {"budget": 0.84, "threshold": -0.9996750354766846, "operator": "gt", "actual_fraction": 0.8395875107419077, "metric": 0.9003150959610426, "rescued": 1070, "harmed": 357, "missed_rescue": 17, "avoided_harm": 26, "selected_both_correct": 4127, "selected_both_wrong": 308}, {"budget": 0.85, "threshold": -0.9997040629386902, "operator": "gt", "actual_fraction": 0.8458894299627614, "metric": 0.9003150959610426, "rescued": 1071, "harmed": 358, "missed_rescue": 16, "avoided_harm": 25, "selected_both_correct": 4167, "selected_both_wrong": 310}, {"budget": 0.86, "threshold": -0.9997387528419495, "operator": "gt", "actual_fraction": 0.8587797192781438, "metric": 0.9004583213978803, "rescued": 1073, "harmed": 359, "missed_rescue": 14, "avoided_harm": 24, "selected_both_correct": 4254, "selected_both_wrong": 310}, {"budget": 0.87, "threshold": -0.9997695088386536, "operator": "gt", "actual_fraction": 0.8693784016041249, "metric": 0.9000286450873675, "rescued": 1073, "harmed": 362, "missed_rescue": 14, "avoided_harm": 21, "selected_both_correct": 4323, "selected_both_wrong": 312}, {"budget": 0.88, "threshold": -0.999783456325531, "operator": "gt", "actual_fraction": 0.8749641936407906, "metric": 0.8998854196505299, "rescued": 1073, "harmed": 363, "missed_rescue": 14, "avoided_harm": 20, "selected_both_correct": 4361, "selected_both_wrong": 312}, {"budget": 0.89, "threshold": -0.9998205304145813, "operator": "gt", "actual_fraction": 0.8894299627613864, "metric": 0.8994557433400172, "rescued": 1074, "harmed": 367, "missed_rescue": 13, "avoided_harm": 16, "selected_both_correct": 4456, "selected_both_wrong": 313}, {"budget": 0.9, "threshold": -0.999841570854187, "operator": "gt", "actual_fraction": 0.8988828415926668, "metric": 0.8993125179031796, "rescued": 1075, "harmed": 369, "missed_rescue": 12, "avoided_harm": 14, "selected_both_correct": 4517, "selected_both_wrong": 315}, {"budget": 0.91, "threshold": -0.9998601675033569, "operator": "gt", "actual_fraction": 0.9093382984818104, "metric": 0.8994557433400172, "rescued": 1077, "harmed": 370, "missed_rescue": 10, "avoided_harm": 13, "selected_both_correct": 4586, "selected_both_wrong": 316}, {"budget": 0.92, "threshold": -0.9998766183853149, "operator": "gt", "actual_fraction": 0.9185047264394156, "metric": 0.8993125179031796, "rescued": 1077, "harmed": 371, "missed_rescue": 10, "avoided_harm": 12, "selected_both_correct": 4646, "selected_both_wrong": 319}, {"budget": 0.93, "threshold": -0.9998977184295654, "operator": "gt", "actual_fraction": 0.9276711543970209, "metric": 0.899169292466342, "rescued": 1078, "harmed": 373, "missed_rescue": 9, "avoided_harm": 10, "selected_both_correct": 4707, "selected_both_wrong": 319}, {"budget": 0.94, "threshold": -0.9999151229858398, "operator": "gt", "actual_fraction": 0.9394156402177026, "metric": 0.8993125179031796, "rescued": 1081, "harmed": 375, "missed_rescue": 6, "avoided_harm": 8, "selected_both_correct": 4784, "selected_both_wrong": 319}, {"budget": 0.95, "threshold": -0.999931812286377, "operator": "gt", "actual_fraction": 0.9497278716700086, "metric": 0.8993125179031796, "rescued": 1082, "harmed": 376, "missed_rescue": 5, "avoided_harm": 7, "selected_both_correct": 4854, "selected_both_wrong": 319}, {"budget": 0.96, "threshold": -0.9999452829360962, "operator": "gt", "actual_fraction": 0.9590375250644515, "metric": 0.8995989687768547, "rescued": 1085, "harmed": 377, "missed_rescue": 2, "avoided_harm": 6, "selected_both_correct": 4914, "selected_both_wrong": 320}, {"budget": 0.97, "threshold": -0.9999599456787109, "operator": "gt", "actual_fraction": 0.969492981953595, "metric": 0.8994557433400172, "rescued": 1085, "harmed": 378, "missed_rescue": 2, "avoided_harm": 5, "selected_both_correct": 4986, "selected_both_wrong": 320}, {"budget": 0.98, "threshold": -0.9999687671661377, "operator": "gt", "actual_fraction": 0.9783729590375251, "metric": 0.899169292466342, "rescued": 1085, "harmed": 380, "missed_rescue": 2, "avoided_harm": 3, "selected_both_correct": 5046, "selected_both_wrong": 320}, {"budget": 0.99, "threshold": -0.9999842643737793, "operator": "gt", "actual_fraction": 0.9898309939845317, "metric": 0.899169292466342, "rescued": 1087, "harmed": 382, "missed_rescue": 0, "avoided_harm": 1, "selected_both_correct": 5122, "selected_both_wrong": 320}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.8990260670295045, "rescued": 1087, "harmed": 383, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 5190, "selected_both_wrong": 322}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.8000287005811868, "rescued": 0, "harmed": 0, "missed_rescue": 2225, "avoided_harm": 729, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.5245773792266846, "operator": "gt", "actual_fraction": 0.011623735380641457, "metric": 0.8034727703235991, "rescued": 65, "harmed": 17, "missed_rescue": 2160, "avoided_harm": 712, "selected_both_correct": 65, "selected_both_wrong": 15}, {"budget": 0.02, "threshold": -0.5456489324569702, "operator": "gt", "actual_fraction": 0.021453684437109852, "metric": 0.8064145798952429, "rescued": 115, "harmed": 26, "missed_rescue": 2110, "avoided_harm": 703, "selected_both_correct": 126, "selected_both_wrong": 32}, {"budget": 0.03, "threshold": -0.5652987360954285, "operator": "ge", "actual_fraction": 0.02970510152830595, "metric": 0.8094281409198536, "rescued": 165, "harmed": 34, "missed_rescue": 2060, "avoided_harm": 695, "selected_both_correct": 174, "selected_both_wrong": 41}, {"budget": 0.04, "threshold": -0.5851011276245117, "operator": "ge", "actual_fraction": 0.03795651861950204, "metric": 0.8113654301499605, "rescued": 204, "harmed": 46, "missed_rescue": 2021, "avoided_harm": 683, "selected_both_correct": 229, "selected_both_wrong": 50}, {"budget": 0.05, "threshold": -0.6055658459663391, "operator": "ge", "actual_fraction": 0.047212456052235056, "metric": 0.8140202339097367, "rescued": 253, "harmed": 58, "missed_rescue": 1972, "avoided_harm": 671, "selected_both_correct": 289, "selected_both_wrong": 58}, {"budget": 0.06, "threshold": -0.6293190717697144, "operator": "ge", "actual_fraction": 0.057759919638372674, "metric": 0.817464303652149, "rescued": 312, "harmed": 69, "missed_rescue": 1913, "avoided_harm": 660, "selected_both_correct": 351, "selected_both_wrong": 73}, {"budget": 0.07, "threshold": -0.6477982401847839, "operator": "ge", "actual_fraction": 0.06601133672956877, "metric": 0.8196885986941236, "rescued": 356, "harmed": 82, "missed_rescue": 1869, "avoided_harm": 647, "selected_both_correct": 401, "selected_both_wrong": 81}, {"budget": 0.08, "threshold": -0.6693127155303955, "operator": "ge", "actual_fraction": 0.07483676544450026, "metric": 0.8226304082657674, "rescued": 406, "harmed": 91, "missed_rescue": 1819, "avoided_harm": 638, "selected_both_correct": 453, "selected_both_wrong": 93}, {"budget": 0.09, "threshold": -0.6888871788978577, "operator": "gt", "actual_fraction": 0.08488196885986941, "metric": 0.8254287149314774, "rescued": 455, "harmed": 101, "missed_rescue": 1770, "avoided_harm": 628, "selected_both_correct": 523, "selected_both_wrong": 104}, {"budget": 0.1, "threshold": -0.7061904072761536, "operator": "gt", "actual_fraction": 0.09356389466886704, "metric": 0.8282987730501543, "rescued": 503, "harmed": 109, "missed_rescue": 1722, "avoided_harm": 620, "selected_both_correct": 576, "selected_both_wrong": 116}, {"budget": 0.11, "threshold": -0.7233095169067383, "operator": "ge", "actual_fraction": 0.1026045777426993, "metric": 0.8310970797158642, "rescued": 551, "harmed": 118, "missed_rescue": 1674, "avoided_harm": 611, "selected_both_correct": 638, "selected_both_wrong": 123}, {"budget": 0.12, "threshold": -0.7424216866493225, "operator": "ge", "actual_fraction": 0.11322379278180383, "metric": 0.8338953863815742, "rescued": 603, "harmed": 131, "missed_rescue": 1622, "avoided_harm": 598, "selected_both_correct": 708, "selected_both_wrong": 136}, {"budget": 0.13, "threshold": -0.7627776861190796, "operator": "gt", "actual_fraction": 0.12255148166750376, "metric": 0.8363349357824496, "rescued": 646, "harmed": 140, "missed_rescue": 1579, "avoided_harm": 589, "selected_both_correct": 776, "selected_both_wrong": 146}, {"budget": 0.14, "threshold": -0.7786493301391602, "operator": "gt", "actual_fraction": 0.13137691038243524, "metric": 0.8390614909951927, "rescued": 693, "harmed": 149, "missed_rescue": 1532, "avoided_harm": 580, "selected_both_correct": 832, "selected_both_wrong": 157}, {"budget": 0.15, "threshold": -0.7950222492218018, "operator": "ge", "actual_fraction": 0.14056109636220132, "metric": 0.8419315491138696, "rescued": 743, "harmed": 159, "missed_rescue": 1482, "avoided_harm": 570, "selected_both_correct": 891, "selected_both_wrong": 166}, {"budget": 0.16, "threshold": -0.8140527606010437, "operator": "ge", "actual_fraction": 0.15096505704240512, "metric": 0.8448016072325465, "rescued": 793, "harmed": 169, "missed_rescue": 1432, "avoided_harm": 560, "selected_both_correct": 965, "selected_both_wrong": 177}, {"budget": 0.17, "threshold": -0.8327813148498535, "operator": "ge", "actual_fraction": 0.1613690177226089, "metric": 0.8465236421037526, "rescued": 831, "harmed": 183, "missed_rescue": 1394, "avoided_harm": 546, "selected_both_correct": 1041, "selected_both_wrong": 194}, {"budget": 0.18, "threshold": -0.8464508652687073, "operator": "gt", "actual_fraction": 0.1702661978905073, "metric": 0.8483891798808926, "rescued": 871, "harmed": 197, "missed_rescue": 1354, "avoided_harm": 532, "selected_both_correct": 1104, "selected_both_wrong": 201}, {"budget": 0.19, "threshold": -0.858244001865387, "operator": "ge", "actual_fraction": 0.17916337805840568, "metric": 0.8509722321877018, "rescued": 915, "harmed": 205, "missed_rescue": 1310, "avoided_harm": 524, "selected_both_correct": 1164, "selected_both_wrong": 213}, {"budget": 0.2, "threshold": -0.8701565861701965, "operator": "ge", "actual_fraction": 0.18798880677333715, "metric": 0.8528377699648418, "rescued": 954, "harmed": 218, "missed_rescue": 1271, "avoided_harm": 511, "selected_both_correct": 1230, "selected_both_wrong": 218}, {"budget": 0.21, "threshold": -0.8840392827987671, "operator": "gt", "actual_fraction": 0.19659898112936788, "metric": 0.8549903135538495, "rescued": 992, "harmed": 226, "missed_rescue": 1233, "avoided_harm": 503, "selected_both_correct": 1299, "selected_both_wrong": 223}, {"budget": 0.22, "threshold": -0.8933094143867493, "operator": "ge", "actual_fraction": 0.20642893018583627, "metric": 0.8574298629547248, "rescued": 1035, "harmed": 235, "missed_rescue": 1190, "avoided_harm": 494, "selected_both_correct": 1377, "selected_both_wrong": 230}, {"budget": 0.23, "threshold": -0.9046505093574524, "operator": "gt", "actual_fraction": 0.21711989667790774, "metric": 0.8599411638085671, "rescued": 1079, "harmed": 244, "missed_rescue": 1146, "avoided_harm": 485, "selected_both_correct": 1461, "selected_both_wrong": 242}, {"budget": 0.24, "threshold": -0.9136765599250793, "operator": "ge", "actual_fraction": 0.22766736026404535, "metric": 0.8620937073975748, "rescued": 1120, "harmed": 255, "missed_rescue": 1105, "avoided_harm": 474, "selected_both_correct": 1539, "selected_both_wrong": 259}, {"budget": 0.25, "threshold": -0.9230391979217529, "operator": "gt", "actual_fraction": 0.23785606658534836, "metric": 0.8648202626103179, "rescued": 1170, "harmed": 267, "missed_rescue": 1055, "avoided_harm": 462, "selected_both_correct": 1612, "selected_both_wrong": 266}, {"budget": 0.26, "threshold": -0.9314625263214111, "operator": "gt", "actual_fraction": 0.24804477290665136, "metric": 0.8671163091052594, "rescued": 1211, "harmed": 276, "missed_rescue": 1014, "avoided_harm": 453, "selected_both_correct": 1693, "selected_both_wrong": 277}, {"budget": 0.27, "threshold": -0.9376689791679382, "operator": "ge", "actual_fraction": 0.25751596469828514, "metric": 0.8688383439764655, "rescued": 1245, "harmed": 286, "missed_rescue": 980, "avoided_harm": 443, "selected_both_correct": 1771, "selected_both_wrong": 287}, {"budget": 0.28, "threshold": -0.9449946880340576, "operator": "gt", "actual_fraction": 0.26863743990815814, "metric": 0.8704886273947048, "rescued": 1278, "harmed": 296, "missed_rescue": 947, "avoided_harm": 433, "selected_both_correct": 1869, "selected_both_wrong": 301}, {"budget": 0.29, "threshold": -0.9511421918869019, "operator": "gt", "actual_fraction": 0.2793284064002296, "metric": 0.8725694195307455, "rescued": 1322, "harmed": 311, "missed_rescue": 903, "avoided_harm": 418, "selected_both_correct": 1951, "selected_both_wrong": 309}, {"budget": 0.3, "threshold": -0.9559813141822815, "operator": "gt", "actual_fraction": 0.2897323670804334, "metric": 0.8739326971371171, "rescued": 1350, "harmed": 320, "missed_rescue": 875, "avoided_harm": 409, "selected_both_correct": 2050, "selected_both_wrong": 318}, {"budget": 0.31, "threshold": -0.9597620368003845, "operator": "ge", "actual_fraction": 0.2999928248547033, "metric": 0.8755829805553562, "rescued": 1377, "harmed": 324, "missed_rescue": 848, "avoided_harm": 405, "selected_both_correct": 2158, "selected_both_wrong": 322}, {"budget": 0.32, "threshold": -0.9643214344978333, "operator": "gt", "actual_fraction": 0.3091770108344694, "metric": 0.8770897610676616, "rescued": 1408, "harmed": 334, "missed_rescue": 817, "avoided_harm": 395, "selected_both_correct": 2234, "selected_both_wrong": 333}, {"budget": 0.33, "threshold": -0.9686195850372314, "operator": "gt", "actual_fraction": 0.3218770180096147, "metric": 0.878596541579967, "rescued": 1442, "harmed": 347, "missed_rescue": 783, "avoided_harm": 382, "selected_both_correct": 2358, "selected_both_wrong": 339}, {"budget": 0.34, "threshold": -0.9719924926757812, "operator": "gt", "actual_fraction": 0.33127645834828157, "metric": 0.8806055822630409, "rescued": 1476, "harmed": 353, "missed_rescue": 749, "avoided_harm": 376, "selected_both_correct": 2445, "selected_both_wrong": 343}, {"budget": 0.35, "threshold": -0.974821150302887, "operator": "ge", "actual_fraction": 0.3431871995407907, "metric": 0.8816101026045777, "rescued": 1507, "harmed": 370, "missed_rescue": 718, "avoided_harm": 359, "selected_both_correct": 2556, "selected_both_wrong": 350}, {"budget": 0.36, "threshold": -0.9777138829231262, "operator": "gt", "actual_fraction": 0.35280189423835834, "metric": 0.8823993685872139, "rescued": 1526, "harmed": 378, "missed_rescue": 699, "avoided_harm": 351, "selected_both_correct": 2652, "selected_both_wrong": 361}, {"budget": 0.37, "threshold": -0.9802808165550232, "operator": "gt", "actual_fraction": 0.36421037526009903, "metric": 0.8837626461935855, "rescued": 1555, "harmed": 388, "missed_rescue": 670, "avoided_harm": 341, "selected_both_correct": 2768, "selected_both_wrong": 365}, {"budget": 0.38, "threshold": -0.982557475566864, "operator": "gt", "actual_fraction": 0.3761928679055751, "metric": 0.8851259237999569, "rescued": 1587, "harmed": 401, "missed_rescue": 638, "avoided_harm": 328, "selected_both_correct": 2882, "selected_both_wrong": 373}, {"budget": 0.39, "threshold": -0.9843363761901855, "operator": "gt", "actual_fraction": 0.38803185764511733, "metric": 0.8864892014063285, "rescued": 1614, "harmed": 409, "missed_rescue": 611, "avoided_harm": 320, "selected_both_correct": 3004, "selected_both_wrong": 381}, {"budget": 0.4, "threshold": -0.9863631129264832, "operator": "gt", "actual_fraction": 0.4008036162732295, "metric": 0.8881394848245677, "rescued": 1647, "harmed": 419, "missed_rescue": 578, "avoided_harm": 310, "selected_both_correct": 3133, "selected_both_wrong": 387}, {"budget": 0.41, "threshold": -0.9879462122917175, "operator": "gt", "actual_fraction": 0.41156633421826794, "metric": 0.8890722537131377, "rescued": 1670, "harmed": 429, "missed_rescue": 555, "avoided_harm": 300, "selected_both_correct": 3241, "selected_both_wrong": 396}, {"budget": 0.42, "threshold": -0.9890130758285522, "operator": "gt", "actual_fraction": 0.4204635143861663, "metric": 0.8895745138839062, "rescued": 1684, "harmed": 436, "missed_rescue": 541, "avoided_harm": 293, "selected_both_correct": 3336, "selected_both_wrong": 404}, {"budget": 0.43, "threshold": -0.9899863600730896, "operator": "gt", "actual_fraction": 0.4293606945540647, "metric": 0.890579034225443, "rescued": 1706, "harmed": 444, "missed_rescue": 519, "avoided_harm": 285, "selected_both_correct": 3428, "selected_both_wrong": 406}, {"budget": 0.44, "threshold": -0.9911527037620544, "operator": "gt", "actual_fraction": 0.44119968429360695, "metric": 0.89251632345555, "rescued": 1742, "harmed": 453, "missed_rescue": 483, "avoided_harm": 276, "selected_both_correct": 3542, "selected_both_wrong": 412}, {"budget": 0.45, "threshold": -0.9919379353523254, "operator": "gt", "actual_fraction": 0.4516753964267776, "metric": 0.8933773408911531, "rescued": 1765, "harmed": 464, "missed_rescue": 460, "avoided_harm": 265, "selected_both_correct": 3649, "selected_both_wrong": 417}, {"budget": 0.46, "threshold": -0.9926542043685913, "operator": "gt", "actual_fraction": 0.4611465882184114, "metric": 0.8938796010619215, "rescued": 1781, "harmed": 473, "missed_rescue": 444, "avoided_harm": 256, "selected_both_correct": 3750, "selected_both_wrong": 423}, {"budget": 0.47, "threshold": -0.9933071732521057, "operator": "gt", "actual_fraction": 0.4706895314630121, "metric": 0.8947406184975246, "rescued": 1797, "harmed": 477, "missed_rescue": 428, "avoided_harm": 252, "selected_both_correct": 3860, "selected_both_wrong": 426}, {"budget": 0.48, "threshold": -0.9939024448394775, "operator": "gt", "actual_fraction": 0.481021740690249, "metric": 0.8955298844801607, "rescued": 1817, "harmed": 486, "missed_rescue": 408, "avoided_harm": 243, "selected_both_correct": 3969, "selected_both_wrong": 432}, {"budget": 0.49, "threshold": -0.9944451451301575, "operator": "gt", "actual_fraction": 0.4912821984645189, "metric": 0.8963909019157638, "rescued": 1835, "harmed": 492, "missed_rescue": 390, "avoided_harm": 237, "selected_both_correct": 4079, "selected_both_wrong": 441}, {"budget": 0.5, "threshold": -0.9949396848678589, "operator": "gt", "actual_fraction": 0.4992466097438473, "metric": 0.8968931620865322, "rescued": 1850, "harmed": 500, "missed_rescue": 375, "avoided_harm": 229, "selected_both_correct": 4163, "selected_both_wrong": 445}, {"budget": 0.51, "threshold": -0.9952448010444641, "operator": "gt", "actual_fraction": 0.5059912463227381, "metric": 0.897108416445433, "rescued": 1858, "harmed": 505, "missed_rescue": 367, "avoided_harm": 224, "selected_both_correct": 4239, "selected_both_wrong": 450}, {"budget": 0.52, "threshold": -0.9956685304641724, "operator": "gt", "actual_fraction": 0.5140991605080003, "metric": 0.8974671737102676, "rescued": 1869, "harmed": 511, "missed_rescue": 356, "avoided_harm": 218, "selected_both_correct": 4331, "selected_both_wrong": 454}, {"budget": 0.53, "threshold": -0.9961755275726318, "operator": "gt", "actual_fraction": 0.529166965631054, "metric": 0.8992609600344407, "rescued": 1903, "harmed": 520, "missed_rescue": 322, "avoided_harm": 209, "selected_both_correct": 4495, "selected_both_wrong": 457}, {"budget": 0.54, "threshold": -0.9965165853500366, "operator": "gt", "actual_fraction": 0.5406471981057617, "metric": 0.899834971658176, "rescued": 1920, "harmed": 529, "missed_rescue": 305, "avoided_harm": 200, "selected_both_correct": 4619, "selected_both_wrong": 467}, {"budget": 0.55, "threshold": -0.9968273043632507, "operator": "gt", "actual_fraction": 0.5504771471622301, "metric": 0.89997847456411, "rescued": 1927, "harmed": 534, "missed_rescue": 298, "avoided_harm": 195, "selected_both_correct": 4743, "selected_both_wrong": 468}, {"budget": 0.56, "threshold": -0.9971103668212891, "operator": "gt", "actual_fraction": 0.5621726339958384, "metric": 0.9006959890937791, "rescued": 1946, "harmed": 543, "missed_rescue": 279, "avoided_harm": 186, "selected_both_correct": 4874, "selected_both_wrong": 472}, {"budget": 0.57, "threshold": -0.997285008430481, "operator": "gt", "actual_fraction": 0.5677692473272584, "metric": 0.9011264978115807, "rescued": 1956, "harmed": 547, "missed_rescue": 269, "avoided_harm": 182, "selected_both_correct": 4935, "selected_both_wrong": 475}, {"budget": 0.58, "threshold": -0.9974491000175476, "operator": "gt", "actual_fraction": 0.5759489129654876, "metric": 0.9014135036234484, "rescued": 1968, "harmed": 555, "missed_rescue": 257, "avoided_harm": 174, "selected_both_correct": 5025, "selected_both_wrong": 479}, {"budget": 0.59, "threshold": -0.9976769089698792, "operator": "gt", "actual_fraction": 0.5864963765516251, "metric": 0.9019875152471838, "rescued": 1985, "harmed": 564, "missed_rescue": 240, "avoided_harm": 165, "selected_both_correct": 5144, "selected_both_wrong": 481}, {"budget": 0.6, "threshold": -0.9978172779083252, "operator": "gt", "actual_fraction": 0.5942455334720528, "metric": 0.9022745210590515, "rescued": 1994, "harmed": 569, "missed_rescue": 231, "avoided_harm": 160, "selected_both_correct": 5235, "selected_both_wrong": 484}, {"budget": 0.61, "threshold": -0.9980121850967407, "operator": "gt", "actual_fraction": 0.6054387601348927, "metric": 0.9023462725120184, "rescued": 2002, "harmed": 576, "missed_rescue": 223, "avoided_harm": 153, "selected_both_correct": 5372, "selected_both_wrong": 488}, {"budget": 0.62, "threshold": -0.9981897473335266, "operator": "gt", "actual_fraction": 0.6165602353447658, "metric": 0.9033507928535552, "rescued": 2018, "harmed": 578, "missed_rescue": 207, "avoided_harm": 151, "selected_both_correct": 5504, "selected_both_wrong": 493}, {"budget": 0.63, "threshold": -0.9982993006706238, "operator": "gt", "actual_fraction": 0.6242376408122264, "metric": 0.9037095501183899, "rescued": 2027, "harmed": 582, "missed_rescue": 198, "avoided_harm": 147, "selected_both_correct": 5596, "selected_both_wrong": 495}, {"budget": 0.64, "threshold": -0.9984021782875061, "operator": "gt", "actual_fraction": 0.6322020520915549, "metric": 0.9040683073832245, "rescued": 2037, "harmed": 587, "missed_rescue": 188, "avoided_harm": 142, "selected_both_correct": 5691, "selected_both_wrong": 496}, {"budget": 0.65, "threshold": -0.9985449314117432, "operator": "gt", "actual_fraction": 0.6454043194374686, "metric": 0.9046423190069599, "rescued": 2051, "harmed": 593, "missed_rescue": 174, "avoided_harm": 136, "selected_both_correct": 5853, "selected_both_wrong": 498}, {"budget": 0.66, "threshold": -0.9986749291419983, "operator": "gt", "actual_fraction": 0.6566692975532754, "metric": 0.9050010762717945, "rescued": 2062, "harmed": 599, "missed_rescue": 163, "avoided_harm": 130, "selected_both_correct": 5989, "selected_both_wrong": 502}, {"budget": 0.67, "threshold": -0.9987550973892212, "operator": "gt", "actual_fraction": 0.6646337088326039, "metric": 0.9047858219128937, "rescued": 2066, "harmed": 606, "missed_rescue": 159, "avoided_harm": 123, "selected_both_correct": 6087, "selected_both_wrong": 504}, {"budget": 0.68, "threshold": -0.9988664388656616, "operator": "gt", "actual_fraction": 0.6776207218196169, "metric": 0.9051445791777284, "rescued": 2077, "harmed": 612, "missed_rescue": 148, "avoided_harm": 117, "selected_both_correct": 6248, "selected_both_wrong": 507}, {"budget": 0.69, "threshold": -0.9989677667617798, "operator": "gt", "actual_fraction": 0.690464231900696, "metric": 0.9052163306306953, "rescued": 2084, "harmed": 618, "missed_rescue": 141, "avoided_harm": 111, "selected_both_correct": 6409, "selected_both_wrong": 512}, {"budget": 0.7, "threshold": -0.9990302324295044, "operator": "gt", "actual_fraction": 0.7000789265982637, "metric": 0.9053598335366291, "rescued": 2094, "harmed": 626, "missed_rescue": 131, "avoided_harm": 103, "selected_both_correct": 6523, "selected_both_wrong": 514}, {"budget": 0.71, "threshold": -0.9991169571876526, "operator": "gt", "actual_fraction": 0.7121331706967066, "metric": 0.9056468393484968, "rescued": 2103, "harmed": 631, "missed_rescue": 122, "avoided_harm": 98, "selected_both_correct": 6676, "selected_both_wrong": 515}, {"budget": 0.72, "threshold": -0.9991704225540161, "operator": "gt", "actual_fraction": 0.7201693334290019, "metric": 0.9060055966133315, "rescued": 2110, "harmed": 633, "missed_rescue": 115, "avoided_harm": 96, "selected_both_correct": 6779, "selected_both_wrong": 515}, {"budget": 0.73, "threshold": -0.9992445707321167, "operator": "gt", "actual_fraction": 0.7310755542799742, "metric": 0.9062208509722322, "rescued": 2117, "harmed": 637, "missed_rescue": 108, "avoided_harm": 92, "selected_both_correct": 6917, "selected_both_wrong": 518}, {"budget": 0.74, "threshold": -0.9992903470993042, "operator": "gt", "actual_fraction": 0.7394704742771041, "metric": 0.9062208509722322, "rescued": 2121, "harmed": 641, "missed_rescue": 104, "avoided_harm": 88, "selected_both_correct": 7024, "selected_both_wrong": 520}, {"budget": 0.75, "threshold": -0.9993333220481873, "operator": "gt", "actual_fraction": 0.7467173710267633, "metric": 0.9065796082370668, "rescued": 2127, "harmed": 642, "missed_rescue": 98, "avoided_harm": 87, "selected_both_correct": 7118, "selected_both_wrong": 520}, {"budget": 0.76, "threshold": -0.999373733997345, "operator": "gt", "actual_fraction": 0.7550405395709263, "metric": 0.9065796082370668, "rescued": 2129, "harmed": 644, "missed_rescue": 96, "avoided_harm": 85, "selected_both_correct": 7227, "selected_both_wrong": 523}, {"budget": 0.77, "threshold": -0.9994296431541443, "operator": "gt", "actual_fraction": 0.7659467604218986, "metric": 0.9066513596900337, "rescued": 2136, "harmed": 650, "missed_rescue": 89, "avoided_harm": 79, "selected_both_correct": 7362, "selected_both_wrong": 527}, {"budget": 0.78, "threshold": -0.999464213848114, "operator": "gt", "actual_fraction": 0.7743416804190285, "metric": 0.9071536198608022, "rescued": 2145, "harmed": 652, "missed_rescue": 80, "avoided_harm": 77, "selected_both_correct": 7467, "selected_both_wrong": 528}, {"budget": 0.79, "threshold": -0.9994966983795166, "operator": "gt", "actual_fraction": 0.7820908373394562, "metric": 0.9071536198608022, "rescued": 2150, "harmed": 657, "missed_rescue": 75, "avoided_harm": 72, "selected_both_correct": 7563, "selected_both_wrong": 530}, {"budget": 0.8, "threshold": -0.9995417594909668, "operator": "gt", "actual_fraction": 0.7935710698141637, "metric": 0.9070101169548683, "rescued": 2157, "harmed": 666, "missed_rescue": 68, "avoided_harm": 63, "selected_both_correct": 7706, "selected_both_wrong": 531}, {"budget": 0.81, "threshold": -0.9995827078819275, "operator": "gt", "actual_fraction": 0.8052665566477721, "metric": 0.907297122766736, "rescued": 2166, "harmed": 671, "missed_rescue": 59, "avoided_harm": 58, "selected_both_correct": 7854, "selected_both_wrong": 532}, {"budget": 0.82, "threshold": -0.9996140599250793, "operator": "gt", "actual_fraction": 0.8185405754466528, "metric": 0.907297122766736, "rescued": 2172, "harmed": 677, "missed_rescue": 53, "avoided_harm": 52, "selected_both_correct": 8022, "selected_both_wrong": 537}, {"budget": 0.83, "threshold": -0.9996429681777954, "operator": "gt", "actual_fraction": 0.8264332352730143, "metric": 0.9075123771256368, "rescued": 2176, "harmed": 678, "missed_rescue": 49, "avoided_harm": 51, "selected_both_correct": 8126, "selected_both_wrong": 538}, {"budget": 0.84, "threshold": -0.9996750354766846, "operator": "gt", "actual_fraction": 0.836837195953218, "metric": 0.9075841285786037, "rescued": 2179, "harmed": 680, "missed_rescue": 46, "avoided_harm": 49, "selected_both_correct": 8265, "selected_both_wrong": 539}, {"budget": 0.85, "threshold": -0.9997040629386902, "operator": "gt", "actual_fraction": 0.844299347061778, "metric": 0.9075123771256368, "rescued": 2183, "harmed": 685, "missed_rescue": 42, "avoided_harm": 44, "selected_both_correct": 8359, "selected_both_wrong": 540}, {"budget": 0.86, "threshold": -0.9997387528419495, "operator": "gt", "actual_fraction": 0.8580038745784602, "metric": 0.9077276314845376, "rescued": 2191, "harmed": 690, "missed_rescue": 34, "avoided_harm": 39, "selected_both_correct": 8533, "selected_both_wrong": 544}, {"budget": 0.87, "threshold": -0.9997695088386536, "operator": "gt", "actual_fraction": 0.870273373035804, "metric": 0.9081581402023391, "rescued": 2199, "harmed": 692, "missed_rescue": 26, "avoided_harm": 37, "selected_both_correct": 8692, "selected_both_wrong": 546}, {"budget": 0.88, "threshold": -0.999783456325531, "operator": "gt", "actual_fraction": 0.8750807203845878, "metric": 0.9080863887493722, "rescued": 2199, "harmed": 693, "missed_rescue": 26, "avoided_harm": 36, "selected_both_correct": 8757, "selected_both_wrong": 547}, {"budget": 0.89, "threshold": -0.9998205304145813, "operator": "gt", "actual_fraction": 0.8908660400373107, "metric": 0.9080146372964053, "rescued": 2202, "harmed": 697, "missed_rescue": 23, "avoided_harm": 32, "selected_both_correct": 8966, "selected_both_wrong": 551}, {"budget": 0.9, "threshold": -0.999841570854187, "operator": "gt", "actual_fraction": 0.8992609600344407, "metric": 0.9077276314845376, "rescued": 2204, "harmed": 703, "missed_rescue": 21, "avoided_harm": 26, "selected_both_correct": 9075, "selected_both_wrong": 551}, {"budget": 0.91, "threshold": -0.9998601675033569, "operator": "gt", "actual_fraction": 0.9091626605438761, "metric": 0.9076558800315706, "rescued": 2206, "harmed": 706, "missed_rescue": 19, "avoided_harm": 23, "selected_both_correct": 9206, "selected_both_wrong": 553}, {"budget": 0.92, "threshold": -0.9998766183853149, "operator": "gt", "actual_fraction": 0.9171270718232044, "metric": 0.9077276314845376, "rescued": 2209, "harmed": 708, "missed_rescue": 16, "avoided_harm": 21, "selected_both_correct": 9311, "selected_both_wrong": 554}, {"budget": 0.93, "threshold": -0.9998977184295654, "operator": "gt", "actual_fraction": 0.9273875295974744, "metric": 0.9075123771256368, "rescued": 2210, "harmed": 712, "missed_rescue": 15, "avoided_harm": 17, "selected_both_correct": 9448, "selected_both_wrong": 555}, {"budget": 0.94, "threshold": -0.9999151229858398, "operator": "gt", "actual_fraction": 0.9375044844658105, "metric": 0.9073688742197029, "rescued": 2213, "harmed": 717, "missed_rescue": 12, "avoided_harm": 12, "selected_both_correct": 9580, "selected_both_wrong": 556}, {"budget": 0.95, "threshold": -0.999931812286377, "operator": "gt", "actual_fraction": 0.948984716940518, "metric": 0.9073688742197029, "rescued": 2217, "harmed": 721, "missed_rescue": 8, "avoided_harm": 8, "selected_both_correct": 9731, "selected_both_wrong": 557}, {"budget": 0.96, "threshold": -0.9999452829360962, "operator": "gt", "actual_fraction": 0.9576666427495156, "metric": 0.9073688742197029, "rescued": 2217, "harmed": 721, "missed_rescue": 8, "avoided_harm": 8, "selected_both_correct": 9850, "selected_both_wrong": 559}, {"budget": 0.97, "threshold": -0.9999599456787109, "operator": "gt", "actual_fraction": 0.9709406615483963, "metric": 0.9072253713137691, "rescued": 2217, "harmed": 723, "missed_rescue": 8, "avoided_harm": 6, "selected_both_correct": 10033, "selected_both_wrong": 559}, {"budget": 0.98, "threshold": -0.9999687671661377, "operator": "gt", "actual_fraction": 0.9784745641099232, "metric": 0.907297122766736, "rescued": 2218, "harmed": 723, "missed_rescue": 7, "avoided_harm": 6, "selected_both_correct": 10136, "selected_both_wrong": 560}, {"budget": 0.99, "threshold": -0.9999842643737793, "operator": "gt", "actual_fraction": 0.98973954222573, "metric": 0.9072253713137691, "rescued": 2222, "harmed": 728, "missed_rescue": 3, "avoided_harm": 1, "selected_both_correct": 10282, "selected_both_wrong": 562}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.9073688742197029, "rescued": 2225, "harmed": 729, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 10421, "selected_both_wrong": 562}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.8999140647378974, "actual_fraction": 0.9139215124606132, "metric_std": 0.000567505378291589, "actual_fraction_std": 0.0433799324816244, "per_seed": [{"seed": 2026, "budget": 0.95, "threshold": -0.9836896605081553, "operator": "ge", "alpha": 1.0, "epoch": 1, "metric": 0.9000286450873675, "actual_fraction": 0.9498710971068463, "rescue": 1084, "harm": 373, "missed_rescue": 3}, {"seed": 2027, "budget": 0.94, "threshold": -0.9859202056551734, "operator": "ge", "alpha": 1.0, "epoch": 2, "metric": 0.8997421942136923, "actual_fraction": 0.9399885419650531, "rescue": 1082, "harm": 373, "missed_rescue": 5}, {"seed": 2028, "budget": 0.92, "threshold": -0.9773874937376431, "operator": "ge", "alpha": 1.0, "epoch": 1, "metric": 0.9007447722715554, "actual_fraction": 0.9199369808077915, "rescue": 1077, "harm": 361, "missed_rescue": 10}, {"seed": 2029, "budget": 0.84, "threshold": -0.9954741443489745, "operator": "ge", "alpha": 1.0, "epoch": 9, "metric": 0.8991692924663419, "actual_fraction": 0.8398739616155829, "rescue": 1063, "harm": 358, "missed_rescue": 24}, {"seed": 2030, "budget": 0.92, "threshold": -0.9790576549262332, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.8998854196505299, "actual_fraction": 0.9199369808077915, "rescue": 1078, "harm": 368, "missed_rescue": 9}], "rescued": 1076.8, "harmed": 366.6, "missed_rescue": 10.2, "avoided_harm": 16.399999999999977}, "validation_curve": [{"budget": 0.0, "metric": 0.7981953594958465, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.8018619306788886, "metric_std": 0.0006292146272268612, "actual_fraction": 0.009882555141793183, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.805327986250358, "metric_std": 0.0007538067812626663, "actual_fraction": 0.01990833572042395, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.8089372672586652, "metric_std": 0.000803850770575919, "actual_fraction": 0.029934116299054715, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.8125751933543397, "metric_std": 0.0009092277426585235, "actual_fraction": 0.039959896877685475, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.8158980234889717, "metric_std": 0.0009586479822790792, "actual_fraction": 0.04998567745631624, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.8196791750214837, "metric_std": 0.0012770377041053176, "actual_fraction": 0.05986823259810943, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.8225150386708678, "metric_std": 0.0012212015896471535, "actual_fraction": 0.06989401317674018, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.8255227728444572, "metric_std": 0.0017523911680530052, "actual_fraction": 0.07991979375537095, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.8280148954454312, "metric_std": 0.0014725079533056047, "actual_fraction": 0.08994557433400172, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.8316241764537381, "metric_std": 0.0018358573697123373, "actual_fraction": 0.09997135491263248, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.8346319106273272, "metric_std": 0.0016127972798922056, "actual_fraction": 0.10999713549126325, "actual_fraction_std": 0.0}, {"budget": 0.12, "metric": 0.8369235176167287, "metric_std": 0.00187729159298894, "actual_fraction": 0.11987969063305641, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.13, "metric": 0.8390146089945574, "metric_std": 0.0018701754307076026, "actual_fraction": 0.1299054712116872, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.8413348610713263, "metric_std": 0.0015929604942308466, "actual_fraction": 0.13993125179031796, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.8439988541965053, "metric_std": 0.0016529977709829372, "actual_fraction": 0.14995703236894872, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.8460613004869664, "metric_std": 0.0013142446000520905, "actual_fraction": 0.1599828129475795, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.8483529074763678, "metric_std": 0.0014655258942751566, "actual_fraction": 0.16986536808937266, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.8499856774563161, "metric_std": 0.000741459702408792, "actual_fraction": 0.17989114866800343, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.8521054139215124, "metric_std": 0.00037348624492706894, "actual_fraction": 0.18991692924663417, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.8544829561730163, "metric_std": 0.00039223901282235853, "actual_fraction": 0.19994270982526496, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.8564594672013749, "metric_std": 0.0006193568693909739, "actual_fraction": 0.20996849040389573, "actual_fraction_std": 0.0}, {"budget": 0.22, "metric": 0.8588370094528788, "metric_std": 0.00097456464257829, "actual_fraction": 0.2199942709825265, "actual_fraction_std": 0.0}, {"budget": 0.23, "metric": 0.8606416499570322, "metric_std": 0.0010377668824255277, "actual_fraction": 0.22987682612431967, "actual_fraction_std": 0.0}, {"budget": 0.24, "metric": 0.8623603551990835, "metric_std": 0.0009714021746097157, "actual_fraction": 0.23990260670295044, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.8632769979948438, "metric_std": 0.0009424625537875798, "actual_fraction": 0.24992838728158123, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.26, "metric": 0.864451446576912, "metric_std": 0.0005957214077079581, "actual_fraction": 0.25995416786021197, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.8659982812947579, "metric_std": 0.0003297297173652583, "actual_fraction": 0.26997994843884277, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.8672873102262961, "metric_std": 0.0004248753071954316, "actual_fraction": 0.2798625035806359, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.869006015468347, "metric_std": 0.001192303373719132, "actual_fraction": 0.2898882841592667, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.8700945287883128, "metric_std": 0.0013587557978380642, "actual_fraction": 0.29991406473789745, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.8712403322830135, "metric_std": 0.0014069666789163554, "actual_fraction": 0.30993984531652824, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.8722142652535089, "metric_std": 0.0013297616027998103, "actual_fraction": 0.319965625895159, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.8735319392724147, "metric_std": 0.001293001295135145, "actual_fraction": 0.3299914064737898, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.8746490976797479, "metric_std": 0.001378986525569832, "actual_fraction": 0.3398739616155829, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.8757662560870809, "metric_std": 0.00139083625165401, "actual_fraction": 0.3498997421942137, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.8770552850186192, "metric_std": 0.0011029288632585883, "actual_fraction": 0.35992552277284445, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.8786307648238326, "metric_std": 0.0010982692359026746, "actual_fraction": 0.3699513033514752, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.8797479232311659, "metric_std": 0.0011167911022410448, "actual_fraction": 0.379977083930106, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.8810083070753365, "metric_std": 0.0009682293773220312, "actual_fraction": 0.3898596390718992, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.8821827556574048, "metric_std": 0.001069885385103567, "actual_fraction": 0.3998854196505299, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.8832712689773704, "metric_std": 0.000771292581944249, "actual_fraction": 0.40991120022916067, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.42, "metric": 0.8842738470352334, "metric_std": 0.0012608720181652312, "actual_fraction": 0.41993698080779146, "actual_fraction_std": 0.0}, {"budget": 0.43, "metric": 0.8852764250930966, "metric_std": 0.0009282069175605585, "actual_fraction": 0.42996276138642225, "actual_fraction_std": 0.0}, {"budget": 0.44, "metric": 0.8862217129762244, "metric_std": 0.0004706868153868586, "actual_fraction": 0.439988541965053, "actual_fraction_std": 0.0}, {"budget": 0.45, "metric": 0.8872242910340876, "metric_std": 0.001060255257709631, "actual_fraction": 0.4498710971068462, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.8878258378688054, "metric_std": 0.0006972593208042485, "actual_fraction": 0.459896877685477, "actual_fraction_std": 0.0}, {"budget": 0.47, "metric": 0.8884846748782584, "metric_std": 0.0002969985871287149, "actual_fraction": 0.46992265826410773, "actual_fraction_std": 0.0}, {"budget": 0.48, "metric": 0.8890862217129761, "metric_std": 0.0005124189006014944, "actual_fraction": 0.4799484388427385, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.49, "metric": 0.8897164136350615, "metric_std": 0.0005991549889243532, "actual_fraction": 0.4899742194213693, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.5, "metric": 0.890403895731882, "metric_std": 0.000558395551120512, "actual_fraction": 0.5, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.8910054425665997, "metric_std": 0.0003358934230753153, "actual_fraction": 0.5098825551417931, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.8914064737897449, "metric_std": 0.0008909957613861769, "actual_fraction": 0.5199083357204239, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.8919507304497278, "metric_std": 0.0008449086911345113, "actual_fraction": 0.5299341162990547, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.8924376969349757, "metric_std": 0.0011679680915425376, "actual_fraction": 0.5399598968776855, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.8929819535949586, "metric_std": 0.0009839910648706924, "actual_fraction": 0.5499856774563162, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.8936980807791464, "metric_std": 0.0010926514536098784, "actual_fraction": 0.5598682325981095, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.8944142079633343, "metric_std": 0.0010784789800639834, "actual_fraction": 0.5698940131767402, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.8945574334001719, "metric_std": 0.0012665534786269665, "actual_fraction": 0.579919793755371, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.8948725293612145, "metric_std": 0.0008165106166565485, "actual_fraction": 0.5899455743340017, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.8949871097106847, "metric_std": 0.0005203638534685193, "actual_fraction": 0.5999713549126325, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.8950730449727871, "metric_std": 0.0002601819267342535, "actual_fraction": 0.6099971354912632, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.8952162704096247, "metric_std": 0.0003594929933546951, "actual_fraction": 0.6198796906330565, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.8954740761959323, "metric_std": 0.0004899063699939194, "actual_fraction": 0.6299054712116872, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.8957318819822401, "metric_std": 0.00041757031615905963, "actual_fraction": 0.639931251790318, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.89561730163277, "metric_std": 0.0005301276288548173, "actual_fraction": 0.6499570323689486, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.8957891721569752, "metric_std": 0.0006292146272268093, "actual_fraction": 0.6599828129475795, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.8956459467201375, "metric_std": 0.0004596621751899439, "actual_fraction": 0.6698653680893727, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.8959610426811802, "metric_std": 0.0006532084933251926, "actual_fraction": 0.6798911486680035, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.8962474935548552, "metric_std": 0.0009259942638352771, "actual_fraction": 0.6899169292466341, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.8963334288169579, "metric_std": 0.0009682293773220353, "actual_fraction": 0.699942709825265, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.8965339444285304, "metric_std": 0.0008921461760181668, "actual_fraction": 0.7099684904038958, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.8970209109137782, "metric_std": 0.0007909883283577151, "actual_fraction": 0.7199942709825266, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.8972787167000857, "metric_std": 0.0007400750892445687, "actual_fraction": 0.7298768261243197, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.8973073617874533, "metric_std": 0.0004857010873049192, "actual_fraction": 0.7399026067029504, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.8974219421369234, "metric_std": 0.00034194604372140766, "actual_fraction": 0.7499283872815812, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.897507877399026, "metric_std": 0.00047068681538685856, "actual_fraction": 0.759954167860212, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.8973646519621884, "metric_std": 0.0004706868153868315, "actual_fraction": 0.7699799484388428, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.897679747923231, "metric_std": 0.0005223312096480064, "actual_fraction": 0.7798625035806359, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.8979375537095388, "metric_std": 0.0003449325287537358, "actual_fraction": 0.7898882841592667, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.8981667144084788, "metric_std": 0.0006641090300412365, "actual_fraction": 0.7999140647378975, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.8985104554568892, "metric_std": 0.000550999256994268, "actual_fraction": 0.8099398453165282, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.8985963907189916, "metric_std": 0.000640523625751861, "actual_fraction": 0.8199656258951592, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.83, "metric": 0.8986536808937267, "metric_std": 0.0005509992569943012, "actual_fraction": 0.8299914064737898, "actual_fraction_std": 0.0}, {"budget": 0.84, "metric": 0.8989687768547693, "metric_std": 0.0005509992569942479, "actual_fraction": 0.8398739616155829, "actual_fraction_std": 0.0}, {"budget": 0.85, "metric": 0.8989114866800344, "metric_std": 0.0005203638534684887, "actual_fraction": 0.8498997421942137, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.8991406473789745, "metric_std": 0.0004344242591837548, "actual_fraction": 0.8599255227728445, "actual_fraction_std": 0.0}, {"budget": 0.87, "metric": 0.8989974219421368, "metric_std": 0.0005203638534684704, "actual_fraction": 0.8699513033514752, "actual_fraction_std": 0.0}, {"budget": 0.88, "metric": 0.8991406473789745, "metric_std": 0.0006671907590291084, "actual_fraction": 0.879977083930106, "actual_fraction_std": 0.0}, {"budget": 0.89, "metric": 0.8991406473789745, "metric_std": 0.0007045759883270306, "actual_fraction": 0.8898596390718991, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.8991692924663418, "metric_std": 0.0008770730960981962, "actual_fraction": 0.8998854196505299, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.8992265826410769, "metric_std": 0.0008688485183673769, "actual_fraction": 0.9099112002291608, "actual_fraction_std": 0.0}, {"budget": 0.92, "metric": 0.8994557433400171, "metric_std": 0.0010954639459165342, "actual_fraction": 0.9199369808077915, "actual_fraction_std": 0.0}, {"budget": 0.93, "metric": 0.8993698080779146, "metric_std": 0.0009692881303745918, "actual_fraction": 0.9299627613864223, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.94, "metric": 0.8993698080779147, "metric_std": 0.0008863791719716555, "actual_fraction": 0.9399885419650531, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.8994270982526495, "metric_std": 0.000767292766312196, "actual_fraction": 0.9498710971068463, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.8993411629905472, "metric_std": 0.0006594594347305215, "actual_fraction": 0.959896877685477, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.97, "metric": 0.8992838728158119, "metric_std": 0.0003566284617013121, "actual_fraction": 0.9699226582641078, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.98, "metric": 0.8992552277284446, "metric_std": 0.00021721212959182798, "actual_fraction": 0.9799484388427386, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.99, "metric": 0.8990833572042394, "metric_std": 0.00012810472515038213, "actual_fraction": 0.9899742194213692, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.8990260670295044, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.908000287005812, "actual_fraction": 0.9110425486116093, "metric_std": 0.00042266417378264046, "actual_fraction_std": 0.04045195853065919, "per_seed": [{"seed": 2026, "budget": 0.95, "threshold": -0.9836896605081553, "operator": "ge", "alpha": 1.0, "epoch": 1, "metric": 0.9080863887493722, "actual_fraction": 0.9448231326684364, "rescue": 2200, "harm": 694, "missed_rescue": 25}, {"seed": 2027, "budget": 0.94, "threshold": -0.9859202056551734, "operator": "ge", "alpha": 1.0, "epoch": 2, "metric": 0.9078711343904715, "actual_fraction": 0.9334146516466959, "rescue": 2199, "harm": 696, "missed_rescue": 26}, {"seed": 2028, "budget": 0.92, "threshold": -0.9773874937376431, "operator": "ge", "alpha": 1.0, "epoch": 1, "metric": 0.9085168974671738, "actual_fraction": 0.9177010834469398, "rescue": 2192, "harm": 680, "missed_rescue": 33}, {"seed": 2029, "budget": 0.84, "threshold": -0.9954741443489745, "operator": "ge", "alpha": 1.0, "epoch": 9, "metric": 0.9073688742197031, "actual_fraction": 0.8416445433020019, "rescue": 2158, "harm": 662, "missed_rescue": 67}, {"seed": 2030, "budget": 0.92, "threshold": -0.9790576549262332, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.9081581402023392, "actual_fraction": 0.917629331993973, "rescue": 2198, "harm": 691, "missed_rescue": 27}], "rescued": 2189.4, "harmed": 684.6, "missed_rescue": 35.6, "avoided_harm": 44.39999999999998}, "test_curve": [{"budget": 0.0, "metric": 0.8000287005811867, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.806084523211595, "metric_std": 0.0006912015256535936, "actual_fraction": 0.010719667073258232, "actual_fraction_std": 0.0008970366535140704}, {"budget": 0.02, "metric": 0.8108918705603788, "metric_std": 0.0012760870339463654, "actual_fraction": 0.02132453182176939, "actual_fraction_std": 0.002415371752017907}, {"budget": 0.03, "metric": 0.8151826074478008, "metric_std": 0.0018644335818510803, "actual_fraction": 0.03192939657028055, "actual_fraction_std": 0.0024506036245875106}, {"budget": 0.04, "metric": 0.8194733443352227, "metric_std": 0.0012935177138064433, "actual_fraction": 0.043180024395494, "actual_fraction_std": 0.0015096819443075151}, {"budget": 0.05, "metric": 0.8231326684365359, "metric_std": 0.0011713301081519808, "actual_fraction": 0.05306737461433595, "actual_fraction_std": 0.00186870870422412}, {"budget": 0.06, "metric": 0.8260601277175862, "metric_std": 0.001427294936526257, "actual_fraction": 0.06292602425199109, "actual_fraction_std": 0.002491749351249961}, {"budget": 0.07, "metric": 0.8290593384516036, "metric_std": 0.0015343742469195747, "actual_fraction": 0.0724689674965918, "actual_fraction_std": 0.002438494004156769}, {"budget": 0.08, "metric": 0.8321159503479947, "metric_std": 0.0008837379199370029, "actual_fraction": 0.08260027265552126, "actual_fraction_std": 0.0014054862446139789}, {"budget": 0.09, "metric": 0.8357322235775275, "metric_std": 0.0007167966560230283, "actual_fraction": 0.09346344263471336, "actual_fraction_std": 0.001950267319347856}, {"budget": 0.1, "metric": 0.838544880533831, "metric_std": 0.00074289612430478, "actual_fraction": 0.10448446581043266, "actual_fraction_std": 0.001535045158297004}, {"budget": 0.11, "metric": 0.8419602496950563, "metric_std": 0.0003817015449026813, "actual_fraction": 0.11663916194302934, "actual_fraction_std": 0.0022309971774005888}, {"budget": 0.12, "metric": 0.8441558441558442, "metric_std": 0.00027789218240706944, "actual_fraction": 0.12714357465738682, "actual_fraction_std": 0.0026294729427866373}, {"budget": 0.13, "metric": 0.8464949415225659, "metric_std": 0.0005731140287784869, "actual_fraction": 0.137002224295042, "actual_fraction_std": 0.002570364989252703}, {"budget": 0.14, "metric": 0.8485470330774199, "metric_std": 0.0007767736874634958, "actual_fraction": 0.14599985649709407, "actual_fraction_std": 0.002765643908560144}, {"budget": 0.15, "metric": 0.8516036449738108, "metric_std": 0.0013842638719789714, "actual_fraction": 0.15703522996340677, "actual_fraction_std": 0.003839406772211292}, {"budget": 0.16, "metric": 0.8542871493147738, "metric_std": 0.0015517236700004653, "actual_fraction": 0.1674391906436105, "actual_fraction_std": 0.004546347811408668}, {"budget": 0.17, "metric": 0.8562100882542871, "metric_std": 0.001672747826199969, "actual_fraction": 0.17632202052091553, "actual_fraction_std": 0.0052158420662858475}, {"budget": 0.18, "metric": 0.8582334792279545, "metric_std": 0.001828184218031569, "actual_fraction": 0.18529095214178087, "actual_fraction_std": 0.006172626996506406}, {"budget": 0.19, "metric": 0.860342971945182, "metric_std": 0.0014823231982339271, "actual_fraction": 0.19539355671952358, "actual_fraction_std": 0.005641142856098058}, {"budget": 0.2, "metric": 0.8623663629188492, "metric_std": 0.0017549061837926353, "actual_fraction": 0.20459209298988307, "actual_fraction_std": 0.005332489590249711}, {"budget": 0.21, "metric": 0.8643897538925163, "metric_std": 0.0014359256723964766, "actual_fraction": 0.21450814378991176, "actual_fraction_std": 0.004473287352041418}, {"budget": 0.22, "metric": 0.8662839922508432, "metric_std": 0.0013099308093387331, "actual_fraction": 0.2239075841285786, "actual_fraction_std": 0.00453387433220611}, {"budget": 0.23, "metric": 0.8679773265408626, "metric_std": 0.0011920246628281473, "actual_fraction": 0.23459855062065005, "actual_fraction_std": 0.003759461182691456}, {"budget": 0.24, "metric": 0.8696276099591017, "metric_std": 0.0007228758584921753, "actual_fraction": 0.24361053311329556, "actual_fraction_std": 0.0026486881461590486}, {"budget": 0.25, "metric": 0.8708904355313196, "metric_std": 0.001035560162374292, "actual_fraction": 0.2521633063069527, "actual_fraction_std": 0.0025085317711365084}, {"budget": 0.26, "metric": 0.8724976680777786, "metric_std": 0.0009464666684561164, "actual_fraction": 0.26195020449164097, "actual_fraction_std": 0.00204422395373529}, {"budget": 0.27, "metric": 0.8742484035301714, "metric_std": 0.0009335964228642283, "actual_fraction": 0.27089043553131953, "actual_fraction_std": 0.0023870706621234423}, {"budget": 0.28, "metric": 0.8757695343330703, "metric_std": 0.0006226273416062942, "actual_fraction": 0.27918490349429576, "actual_fraction_std": 0.0020218128440274673}, {"budget": 0.29, "metric": 0.876501399153333, "metric_std": 0.000785014674412438, "actual_fraction": 0.28911530458491785, "actual_fraction_std": 0.0013769924510885967}, {"budget": 0.3, "metric": 0.8779364282126714, "metric_std": 0.0009764526561644058, "actual_fraction": 0.2985290952141781, "actual_fraction_std": 0.002731083123771609}, {"budget": 0.31, "metric": 0.8794145081437901, "metric_std": 0.0009335964228642523, "actual_fraction": 0.3080289875869987, "actual_fraction_std": 0.001892933336912241}, {"budget": 0.32, "metric": 0.8807060342971946, "metric_std": 0.0015936254533964642, "actual_fraction": 0.31826074478008176, "actual_fraction_std": 0.0016080971751541934}, {"budget": 0.33, "metric": 0.8816675037669514, "metric_std": 0.0012748761334359052, "actual_fraction": 0.32768888569993543, "actual_fraction_std": 0.0015629634003581745}, {"budget": 0.34, "metric": 0.8831168831168832, "metric_std": 0.0014757095358544268, "actual_fraction": 0.337863241730645, "actual_fraction_std": 0.0028350450948215907}, {"budget": 0.35, "metric": 0.8843940589796944, "metric_std": 0.0013038247908157665, "actual_fraction": 0.34673172131735663, "actual_fraction_std": 0.003661293219240807}, {"budget": 0.36, "metric": 0.8859008394919998, "metric_std": 0.0010874547573844426, "actual_fraction": 0.35549974886991464, "actual_fraction_std": 0.003767327102960007}, {"budget": 0.37, "metric": 0.8874076200043051, "metric_std": 0.0013484692500059638, "actual_fraction": 0.36652077204563394, "actual_fraction_std": 0.0038242245537935178}, {"budget": 0.38, "metric": 0.8888282987730503, "metric_std": 0.0011553989229018747, "actual_fraction": 0.37688168185405757, "actual_fraction_std": 0.004436305939602267}, {"budget": 0.39, "metric": 0.8897180167898402, "metric_std": 0.0011536152106489176, "actual_fraction": 0.387012987012987, "actual_fraction_std": 0.0041675798591465614}, {"budget": 0.4, "metric": 0.8907655880031571, "metric_std": 0.00109005547907488, "actual_fraction": 0.39711559159072973, "actual_fraction_std": 0.005065113906771969}, {"budget": 0.41, "metric": 0.8915548539857931, "metric_std": 0.0010613395759260038, "actual_fraction": 0.4068450886130444, "actual_fraction_std": 0.004903675088449402}, {"budget": 0.42, "metric": 0.892631125780297, "metric_std": 0.0010829478909220563, "actual_fraction": 0.4183396713783454, "actual_fraction_std": 0.004843515387180374}, {"budget": 0.43, "metric": 0.8938652507713283, "metric_std": 0.001177248722221087, "actual_fraction": 0.4295472483317787, "actual_fraction_std": 0.006050906850891982}, {"budget": 0.44, "metric": 0.8948123699504915, "metric_std": 0.0009464666684561458, "actual_fraction": 0.4392336944823133, "actual_fraction_std": 0.0060402621584745965}, {"budget": 0.45, "metric": 0.8955442347707541, "metric_std": 0.0009711659218304272, "actual_fraction": 0.44873358685513376, "actual_fraction_std": 0.00681601038217001}, {"budget": 0.46, "metric": 0.8964913539499175, "metric_std": 0.0008221727207078053, "actual_fraction": 0.4576451173136256, "actual_fraction_std": 0.004614851314782487}, {"budget": 0.47, "metric": 0.8972949702231471, "metric_std": 0.0010065682954294034, "actual_fraction": 0.4679486259596756, "actual_fraction_std": 0.005084844799065775}, {"budget": 0.48, "metric": 0.897625026906795, "metric_std": 0.0008645975651226163, "actual_fraction": 0.4782377843151323, "actual_fraction_std": 0.00571436247449081}, {"budget": 0.49, "metric": 0.8983568917270576, "metric_std": 0.0003388309952820932, "actual_fraction": 0.48869914615770976, "actual_fraction_std": 0.004704236697468905}, {"budget": 0.5, "metric": 0.8986869484107054, "metric_std": 0.000344107879982316, "actual_fraction": 0.4975389251632346, "actual_fraction_std": 0.00567163352836711}, {"budget": 0.51, "metric": 0.8991318074191004, "metric_std": 0.0004165293900752963, "actual_fraction": 0.5074262753820765, "actual_fraction_std": 0.0040866846138761905}, {"budget": 0.52, "metric": 0.8992896606156275, "metric_std": 0.00028520638465022414, "actual_fraction": 0.517858936643467, "actual_fraction_std": 0.004535009702552253}, {"budget": 0.53, "metric": 0.8999641242735166, "metric_std": 0.0004903307616637183, "actual_fraction": 0.5289517112721532, "actual_fraction_std": 0.004306581740524544}, {"budget": 0.54, "metric": 0.9001363277606371, "metric_std": 0.00034560076282672247, "actual_fraction": 0.5384372533543804, "actual_fraction_std": 0.005020141055112137}, {"budget": 0.55, "metric": 0.900767740546746, "metric_std": 0.00024855432411119774, "actual_fraction": 0.5481380497955084, "actual_fraction_std": 0.005648029002674221}, {"budget": 0.56, "metric": 0.9014565544952285, "metric_std": 0.00033883099528212146, "actual_fraction": 0.5588864174499534, "actual_fraction_std": 0.00394613422254794}, {"budget": 0.57, "metric": 0.9017866111788765, "metric_std": 0.0003672665248495053, "actual_fraction": 0.5690320728994762, "actual_fraction_std": 0.0033771237396112326}, {"budget": 0.58, "metric": 0.9024180239649853, "metric_std": 0.0005248166333618004, "actual_fraction": 0.57850326469111, "actual_fraction_std": 0.003823955299089685}, {"budget": 0.59, "metric": 0.90277678122982, "metric_std": 0.0006900833774008377, "actual_fraction": 0.5884480160723256, "actual_fraction_std": 0.002455011377377616}, {"budget": 0.6, "metric": 0.9031785893664347, "metric_std": 0.00044114016356281165, "actual_fraction": 0.5976322020520916, "actual_fraction_std": 0.003424732252122163}, {"budget": 0.61, "metric": 0.9036234483748297, "metric_std": 0.0005159124061777214, "actual_fraction": 0.6093994403386669, "actual_fraction_std": 0.0024945370634974094}, {"budget": 0.62, "metric": 0.9039965559302576, "metric_std": 0.0006045884891423499, "actual_fraction": 0.6210805768816818, "actual_fraction_std": 0.0011984855484315077}, {"budget": 0.63, "metric": 0.9041257085455982, "metric_std": 0.00043170291903550385, "actual_fraction": 0.6312979837841717, "actual_fraction_std": 0.0022997432033256887}, {"budget": 0.64, "metric": 0.9045131663916195, "metric_std": 0.0004287111787698866, "actual_fraction": 0.641386238071321, "actual_fraction_std": 0.00354897802310864}, {"budget": 0.65, "metric": 0.90464231900696, "metric_std": 0.00038971014175218685, "actual_fraction": 0.6510009327688886, "actual_fraction_std": 0.0033196195688108003}, {"budget": 0.66, "metric": 0.9048001722034872, "metric_std": 0.0005726647037163128, "actual_fraction": 0.6605295257228959, "actual_fraction_std": 0.0032901777212975916}, {"budget": 0.67, "metric": 0.9050728277247615, "metric_std": 0.00046222640190591007, "actual_fraction": 0.6708186840783525, "actual_fraction_std": 0.0028264062190606508}, {"budget": 0.68, "metric": 0.9052163306306953, "metric_std": 0.0006316573459732414, "actual_fraction": 0.6812369950491497, "actual_fraction_std": 0.0025248968261785965}, {"budget": 0.69, "metric": 0.9054315849895961, "metric_std": 0.0009437430177201248, "actual_fraction": 0.6908516897467173, "actual_fraction_std": 0.0016390142797714712}, {"budget": 0.7, "metric": 0.9056324890579035, "metric_std": 0.0010488972844431579, "actual_fraction": 0.7013848030422617, "actual_fraction_std": 0.001926229195024413}, {"budget": 0.71, "metric": 0.9057329410920572, "metric_std": 0.0010063125291197204, "actual_fraction": 0.710913395996269, "actual_fraction_std": 0.0015505620137026635}, {"budget": 0.72, "metric": 0.9059338451603646, "metric_std": 0.0010323236399467995, "actual_fraction": 0.720915548539858, "actual_fraction_std": 0.001135170573974203}, {"budget": 0.73, "metric": 0.9061490995192653, "metric_std": 0.0008227986612024645, "actual_fraction": 0.7302145368443711, "actual_fraction_std": 0.00265189332174448}, {"budget": 0.74, "metric": 0.9061634498098587, "metric_std": 0.0008837379199370683, "actual_fraction": 0.7399440338666857, "actual_fraction_std": 0.0018070828141280764}, {"budget": 0.75, "metric": 0.906321303006386, "metric_std": 0.0008422777120372781, "actual_fraction": 0.7497883332137476, "actual_fraction_std": 0.002565252423419872}, {"budget": 0.76, "metric": 0.9064504556217263, "metric_std": 0.000796408806802715, "actual_fraction": 0.7592451747147879, "actual_fraction_std": 0.0023550440232373675}, {"budget": 0.77, "metric": 0.9065509076558799, "metric_std": 0.0008095522828482035, "actual_fraction": 0.7687020162158283, "actual_fraction_std": 0.001839553894340363}, {"budget": 0.78, "metric": 0.9064648059123197, "metric_std": 0.0006079850804137321, "actual_fraction": 0.7781158068450885, "actual_fraction_std": 0.0015839041826119999}, {"budget": 0.79, "metric": 0.9066226591088469, "metric_std": 0.00042630266994376424, "actual_fraction": 0.7886632704312262, "actual_fraction_std": 0.0007274903841938365}, {"budget": 0.8, "metric": 0.9071105689890221, "metric_std": 0.00029843774618768155, "actual_fraction": 0.7984501686159144, "actual_fraction_std": 0.0017062670518179803}, {"budget": 0.81, "metric": 0.9073401736385162, "metric_std": 0.00029843774618764285, "actual_fraction": 0.8092559374327329, "actual_fraction_std": 0.0021348921094134603}, {"budget": 0.82, "metric": 0.9073688742197031, "metric_std": 0.00018293102940344894, "actual_fraction": 0.818325321087752, "actual_fraction_std": 0.00270237347854869}, {"budget": 0.83, "metric": 0.9074836765444502, "metric_std": 0.0002616715581374261, "actual_fraction": 0.827911315204133, "actual_fraction_std": 0.0018919811913154993}, {"budget": 0.84, "metric": 0.9074836765444502, "metric_std": 0.00020671393113134295, "actual_fraction": 0.8382291741407762, "actual_fraction_std": 0.003527735179293429}, {"budget": 0.85, "metric": 0.9074693262538567, "metric_std": 0.00016518424959784003, "actual_fraction": 0.8475138121546962, "actual_fraction_std": 0.0028697890381145894}, {"budget": 0.86, "metric": 0.9075410777068236, "metric_std": 0.00021284920677612257, "actual_fraction": 0.8582047786467676, "actual_fraction_std": 0.002825039771160835}, {"budget": 0.87, "metric": 0.9077706823563177, "metric_std": 0.0003069419478539884, "actual_fraction": 0.8678912247973022, "actual_fraction_std": 0.0026031047412600175}, {"budget": 0.88, "metric": 0.9078280835186913, "metric_std": 0.00025670580210946454, "actual_fraction": 0.8773911171701227, "actual_fraction_std": 0.0041153696210818255}, {"budget": 0.89, "metric": 0.9079572361340318, "metric_std": 0.0003493050568885119, "actual_fraction": 0.887106263901844, "actual_fraction_std": 0.0042598856358940585}, {"budget": 0.9, "metric": 0.9079141852622516, "metric_std": 0.0003388309952821379, "actual_fraction": 0.8969505632489059, "actual_fraction_std": 0.004708994888884231}, {"budget": 0.91, "metric": 0.9079715864246252, "metric_std": 0.0003426084921091421, "actual_fraction": 0.9075554279974171, "actual_fraction_std": 0.004474725735890536}, {"budget": 0.92, "metric": 0.9081868407835259, "metric_std": 0.0002984377461876642, "actual_fraction": 0.9175001793786324, "actual_fraction_std": 0.0037672587744401677}, {"budget": 0.93, "metric": 0.9081150893305591, "metric_std": 0.00025670580210949555, "actual_fraction": 0.9264404104183109, "actual_fraction_std": 0.00306790955659338}, {"budget": 0.94, "metric": 0.9081294396211523, "metric_std": 0.00027133056470098754, "actual_fraction": 0.9370883260386023, "actual_fraction_std": 0.002670465313979533}, {"budget": 0.95, "metric": 0.9079715864246252, "metric_std": 0.0003916867062913264, "actual_fraction": 0.9461577096936213, "actual_fraction_std": 0.0027150127848504266}, {"budget": 0.96, "metric": 0.907813733228098, "metric_std": 0.0001857240310761599, "actual_fraction": 0.9569060773480663, "actual_fraction_std": 0.0029764315449425627}, {"budget": 0.97, "metric": 0.9077419817751309, "metric_std": 0.00018572403107620063, "actual_fraction": 0.9670660830881825, "actual_fraction_std": 0.0027547325565691985}, {"budget": 0.98, "metric": 0.9075410777068237, "metric_std": 0.0002188125232399875, "actual_fraction": 0.9773839420248261, "actual_fraction_std": 0.0022435384016104456}, {"budget": 0.99, "metric": 0.9073832245102963, "metric_std": 0.00017130424605460827, "actual_fraction": 0.989179880892588, "actual_fraction_std": 0.0012627032626817544}, {"budget": 1.0, "metric": 0.9073688742197031, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 5190, "rescuable": 1087, "harmful": 383, "both_wrong": 322, "unit": "sample", "total": 6982}, "test": {"both_correct": 10421, "rescuable": 2225, "harmful": 729, "both_wrong": 562, "unit": "sample", "total": 13937}}, "small": {"validation": 0.7981953594958464, "test": 0.8000287005811868}, "large": {"validation": 0.8990260670295045, "test": 0.9073688742197029}, "learned_minus_confidence_test": 0.00027265552127442305, "learned_minus_small_test": 0.10797158642462523, "learned_minus_large_test": 0.0006314127861091201, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_nlvr2_MiniCPM"}, {"key": ["nlvr2", "nlvr2", "Qwen3.8"], "expert_display": "BEiT3", "run": "outputs/full_budget_20260913/runs/full_nlvr2_Qwen3.8_s{2026..2030}", "validation_rows": 6982, "test_rows": 13937, "confidence_rule": {"column": "max_probability", "columns": ["entropy", "max_probability", "top_two_margin", "entropy_valid", "max_probability_valid", "top_two_margin_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/nlvr2/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 86, "validation_area_0_25": 0.8312804354053279, "validation_point": {"budget": 0.86, "threshold": -0.9997387528419495, "operator": "gt", "actual_fraction": 0.8587797192781438, "metric": 0.9057576625608708, "rescued": 1134, "harmed": 383, "missed_rescue": 20, "avoided_harm": 25, "selected_both_correct": 4230, "selected_both_wrong": 249}, "test_point": {"budget": 0.86, "threshold": -0.9997387528419495, "operator": "gt", "actual_fraction": 0.8580038745784602, "metric": 0.9024180239649853, "rescued": 2213, "harmed": 786, "missed_rescue": 34, "avoided_harm": 59, "selected_both_correct": 8437, "selected_both_wrong": 522}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.7981953594958464, "rescued": 0, "harmed": 0, "missed_rescue": 1154, "avoided_harm": 408, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.5245773792266846, "operator": "gt", "actual_fraction": 0.0097393297049556, "metric": 0.8004869664852478, "rescued": 26, "harmed": 10, "missed_rescue": 1128, "avoided_harm": 398, "selected_both_correct": 30, "selected_both_wrong": 2}, {"budget": 0.02, "threshold": -0.5456489324569702, "operator": "gt", "actual_fraction": 0.019765110283586365, "metric": 0.8033514752219995, "rescued": 51, "harmed": 15, "missed_rescue": 1103, "avoided_harm": 393, "selected_both_correct": 60, "selected_both_wrong": 12}, {"budget": 0.03, "threshold": -0.5652987360954285, "operator": "ge", "actual_fraction": 0.029934116299054712, "metric": 0.8067888857061014, "rescued": 83, "harmed": 23, "missed_rescue": 1071, "avoided_harm": 385, "selected_both_correct": 84, "selected_both_wrong": 19}, {"budget": 0.04, "threshold": -0.5851011276245117, "operator": "ge", "actual_fraction": 0.039959896877685475, "metric": 0.8087940418218276, "rescued": 105, "harmed": 31, "missed_rescue": 1049, "avoided_harm": 377, "selected_both_correct": 117, "selected_both_wrong": 26}, {"budget": 0.05, "threshold": -0.6055658459663391, "operator": "ge", "actual_fraction": 0.04998567745631624, "metric": 0.8110856488112289, "rescued": 125, "harmed": 35, "missed_rescue": 1029, "avoided_harm": 373, "selected_both_correct": 157, "selected_both_wrong": 32}, {"budget": 0.06, "threshold": -0.6293190717697144, "operator": "ge", "actual_fraction": 0.059868232598109423, "metric": 0.8140933829848181, "rescued": 152, "harmed": 41, "missed_rescue": 1002, "avoided_harm": 367, "selected_both_correct": 187, "selected_both_wrong": 38}, {"budget": 0.07, "threshold": -0.6477982401847839, "operator": "ge", "actual_fraction": 0.06989401317674018, "metric": 0.8179604697794328, "rescued": 183, "harmed": 45, "missed_rescue": 971, "avoided_harm": 363, "selected_both_correct": 214, "selected_both_wrong": 46}, {"budget": 0.08, "threshold": -0.6693127155303955, "operator": "ge", "actual_fraction": 0.07991979375537095, "metric": 0.8212546548266972, "rescued": 211, "harmed": 50, "missed_rescue": 943, "avoided_harm": 358, "selected_both_correct": 246, "selected_both_wrong": 51}, {"budget": 0.09, "threshold": -0.6888871788978577, "operator": "gt", "actual_fraction": 0.08951589802348897, "metric": 0.8239759381266113, "rescued": 238, "harmed": 58, "missed_rescue": 916, "avoided_harm": 350, "selected_both_correct": 275, "selected_both_wrong": 54}, {"budget": 0.1, "threshold": -0.7061904072761536, "operator": "gt", "actual_fraction": 0.09925522772844457, "metric": 0.8252649670581496, "rescued": 255, "harmed": 66, "missed_rescue": 899, "avoided_harm": 342, "selected_both_correct": 311, "selected_both_wrong": 61}, {"budget": 0.11, "threshold": -0.7233095169067383, "operator": "ge", "actual_fraction": 0.10999713549126325, "metric": 0.8289888284159267, "rescued": 284, "harmed": 69, "missed_rescue": 870, "avoided_harm": 339, "selected_both_correct": 350, "selected_both_wrong": 65}, {"budget": 0.12, "threshold": -0.7424216866493225, "operator": "ge", "actual_fraction": 0.11987969063305644, "metric": 0.8319965625895159, "rescued": 314, "harmed": 78, "missed_rescue": 840, "avoided_harm": 330, "selected_both_correct": 377, "selected_both_wrong": 68}, {"budget": 0.13, "threshold": -0.7627776861190796, "operator": "gt", "actual_fraction": 0.1297622457748496, "metric": 0.8344313950157548, "rescued": 338, "harmed": 85, "missed_rescue": 816, "avoided_harm": 323, "selected_both_correct": 411, "selected_both_wrong": 72}, {"budget": 0.14, "threshold": -0.7786493301391602, "operator": "gt", "actual_fraction": 0.1396448009166428, "metric": 0.8371526783156689, "rescued": 366, "harmed": 94, "missed_rescue": 788, "avoided_harm": 314, "selected_both_correct": 440, "selected_both_wrong": 75}, {"budget": 0.15, "threshold": -0.7950222492218018, "operator": "ge", "actual_fraction": 0.14995703236894872, "metric": 0.8400171870524205, "rescued": 390, "harmed": 98, "missed_rescue": 764, "avoided_harm": 310, "selected_both_correct": 477, "selected_both_wrong": 82}, {"budget": 0.16, "threshold": -0.8140527606010437, "operator": "ge", "actual_fraction": 0.1599828129475795, "metric": 0.8424520194786594, "rescued": 412, "harmed": 103, "missed_rescue": 742, "avoided_harm": 305, "selected_both_correct": 514, "selected_both_wrong": 88}, {"budget": 0.17, "threshold": -0.8327813148498535, "operator": "ge", "actual_fraction": 0.16986536808937266, "metric": 0.844313950157548, "rescued": 429, "harmed": 107, "missed_rescue": 725, "avoided_harm": 301, "selected_both_correct": 556, "selected_both_wrong": 94}, {"budget": 0.18, "threshold": -0.8464508652687073, "operator": "gt", "actual_fraction": 0.17960469779432828, "metric": 0.8463191062732741, "rescued": 446, "harmed": 110, "missed_rescue": 708, "avoided_harm": 298, "selected_both_correct": 600, "selected_both_wrong": 98}, {"budget": 0.19, "threshold": -0.858244001865387, "operator": "ge", "actual_fraction": 0.1899169292466342, "metric": 0.8490403895731882, "rescued": 470, "harmed": 115, "missed_rescue": 684, "avoided_harm": 293, "selected_both_correct": 637, "selected_both_wrong": 104}, {"budget": 0.2, "threshold": -0.8701565861701965, "operator": "ge", "actual_fraction": 0.19994270982526496, "metric": 0.8509023202520768, "rescued": 488, "harmed": 120, "missed_rescue": 666, "avoided_harm": 288, "selected_both_correct": 680, "selected_both_wrong": 108}, {"budget": 0.21, "threshold": -0.8840392827987671, "operator": "gt", "actual_fraction": 0.2093955886565454, "metric": 0.852191349183615, "rescued": 505, "harmed": 128, "missed_rescue": 649, "avoided_harm": 280, "selected_both_correct": 718, "selected_both_wrong": 111}, {"budget": 0.22, "threshold": -0.8933094143867493, "operator": "ge", "actual_fraction": 0.2199942709825265, "metric": 0.8533371526783157, "rescued": 522, "harmed": 137, "missed_rescue": 632, "avoided_harm": 271, "selected_both_correct": 762, "selected_both_wrong": 115}, {"budget": 0.23, "threshold": -0.9046505093574524, "operator": "gt", "actual_fraction": 0.2297336006874821, "metric": 0.855628759667717, "rescued": 543, "harmed": 142, "missed_rescue": 611, "avoided_harm": 266, "selected_both_correct": 799, "selected_both_wrong": 120}, {"budget": 0.24, "threshold": -0.9136765599250793, "operator": "ge", "actual_fraction": 0.23990260670295044, "metric": 0.8583500429676311, "rescued": 566, "harmed": 146, "missed_rescue": 588, "avoided_harm": 262, "selected_both_correct": 836, "selected_both_wrong": 127}, {"budget": 0.25, "threshold": -0.9230391979217529, "operator": "gt", "actual_fraction": 0.24906903466055572, "metric": 0.860068748209682, "rescued": 585, "harmed": 153, "missed_rescue": 569, "avoided_harm": 255, "selected_both_correct": 871, "selected_both_wrong": 130}, {"budget": 0.26, "threshold": -0.9314625263214111, "operator": "gt", "actual_fraction": 0.2595244915496992, "metric": 0.8632197078201088, "rescued": 610, "harmed": 156, "missed_rescue": 544, "avoided_harm": 252, "selected_both_correct": 914, "selected_both_wrong": 132}, {"budget": 0.27, "threshold": -0.9376689791679382, "operator": "ge", "actual_fraction": 0.26997994843884277, "metric": 0.8656545402463478, "rescued": 630, "harmed": 159, "missed_rescue": 524, "avoided_harm": 249, "selected_both_correct": 962, "selected_both_wrong": 134}, {"budget": 0.28, "threshold": -0.9449946880340576, "operator": "gt", "actual_fraction": 0.2794328272701232, "metric": 0.8673732454883988, "rescued": 645, "harmed": 162, "missed_rescue": 509, "avoided_harm": 246, "selected_both_correct": 1005, "selected_both_wrong": 139}, {"budget": 0.29, "threshold": -0.9511421918869019, "operator": "gt", "actual_fraction": 0.2896018332855915, "metric": 0.8689487252936121, "rescued": 664, "harmed": 170, "missed_rescue": 490, "avoided_harm": 238, "selected_both_correct": 1043, "selected_both_wrong": 145}, {"budget": 0.3, "threshold": -0.9559813141822815, "operator": "gt", "actual_fraction": 0.29977083930105985, "metric": 0.870380979661988, "rescued": 681, "harmed": 177, "missed_rescue": 473, "avoided_harm": 231, "selected_both_correct": 1083, "selected_both_wrong": 152}, {"budget": 0.31, "threshold": -0.9597620368003845, "operator": "ge", "actual_fraction": 0.30993984531652824, "metric": 0.8716700085935262, "rescued": 697, "harmed": 184, "missed_rescue": 457, "avoided_harm": 224, "selected_both_correct": 1129, "selected_both_wrong": 154}, {"budget": 0.32, "threshold": -0.9643214344978333, "operator": "gt", "actual_fraction": 0.3196791750214838, "metric": 0.8743912918934402, "rescued": 719, "harmed": 187, "missed_rescue": 435, "avoided_harm": 221, "selected_both_correct": 1171, "selected_both_wrong": 155}, {"budget": 0.33, "threshold": -0.9686195850372314, "operator": "gt", "actual_fraction": 0.3297049556001146, "metric": 0.8746777427671154, "rescued": 731, "harmed": 197, "missed_rescue": 423, "avoided_harm": 211, "selected_both_correct": 1215, "selected_both_wrong": 159}, {"budget": 0.34, "threshold": -0.9719924926757812, "operator": "gt", "actual_fraction": 0.339157834431395, "metric": 0.8771125751933544, "rescued": 751, "harmed": 200, "missed_rescue": 403, "avoided_harm": 208, "selected_both_correct": 1256, "selected_both_wrong": 161}, {"budget": 0.35, "threshold": -0.974821150302887, "operator": "ge", "actual_fraction": 0.3498997421942137, "metric": 0.8786880549985677, "rescued": 767, "harmed": 205, "missed_rescue": 387, "avoided_harm": 203, "selected_both_correct": 1306, "selected_both_wrong": 165}, {"budget": 0.36, "threshold": -0.9777138829231262, "operator": "gt", "actual_fraction": 0.3587797192781438, "metric": 0.8814093382984818, "rescued": 786, "harmed": 205, "missed_rescue": 368, "avoided_harm": 203, "selected_both_correct": 1345, "selected_both_wrong": 169}, {"budget": 0.37, "threshold": -0.9802808165550232, "operator": "gt", "actual_fraction": 0.3685190489830994, "metric": 0.881695789172157, "rescued": 797, "harmed": 214, "missed_rescue": 357, "avoided_harm": 194, "selected_both_correct": 1390, "selected_both_wrong": 172}, {"budget": 0.38, "threshold": -0.982557475566864, "operator": "gt", "actual_fraction": 0.3795474076195932, "metric": 0.8839873961615583, "rescued": 817, "harmed": 218, "missed_rescue": 337, "avoided_harm": 190, "selected_both_correct": 1439, "selected_both_wrong": 176}, {"budget": 0.39, "threshold": -0.9843363761901855, "operator": "gt", "actual_fraction": 0.3892867373245488, "metric": 0.8847035233457462, "rescued": 827, "harmed": 223, "missed_rescue": 327, "avoided_harm": 185, "selected_both_correct": 1487, "selected_both_wrong": 181}, {"budget": 0.4, "threshold": -0.9863631129264832, "operator": "gt", "actual_fraction": 0.3993125179031796, "metric": 0.8864222285877972, "rescued": 843, "harmed": 227, "missed_rescue": 311, "avoided_harm": 181, "selected_both_correct": 1536, "selected_both_wrong": 182}, {"budget": 0.41, "threshold": -0.9879462122917175, "operator": "gt", "actual_fraction": 0.40962474935548554, "metric": 0.8879977083930106, "rescued": 857, "harmed": 230, "missed_rescue": 297, "avoided_harm": 178, "selected_both_correct": 1586, "selected_both_wrong": 187}, {"budget": 0.42, "threshold": -0.9890130758285522, "operator": "gt", "actual_fraction": 0.41850472643941566, "metric": 0.8897164136350616, "rescued": 872, "harmed": 233, "missed_rescue": 282, "avoided_harm": 175, "selected_both_correct": 1628, "selected_both_wrong": 189}, {"budget": 0.43, "threshold": -0.9899863600730896, "operator": "gt", "actual_fraction": 0.4270982526496706, "metric": 0.890575766256087, "rescued": 882, "harmed": 237, "missed_rescue": 272, "avoided_harm": 171, "selected_both_correct": 1674, "selected_both_wrong": 189}, {"budget": 0.44, "threshold": -0.9911527037620544, "operator": "gt", "actual_fraction": 0.4395588656545402, "metric": 0.8928673732454884, "rescued": 903, "harmed": 242, "missed_rescue": 251, "avoided_harm": 166, "selected_both_correct": 1728, "selected_both_wrong": 196}, {"budget": 0.45, "threshold": -0.9919379353523254, "operator": "gt", "actual_fraction": 0.4497278716700086, "metric": 0.8937267258665139, "rescued": 913, "harmed": 246, "missed_rescue": 241, "avoided_harm": 162, "selected_both_correct": 1784, "selected_both_wrong": 197}, {"budget": 0.46, "threshold": -0.9926542043685913, "operator": "gt", "actual_fraction": 0.4586078487539387, "metric": 0.8953022056717272, "rescued": 925, "harmed": 247, "missed_rescue": 229, "avoided_harm": 161, "selected_both_correct": 1832, "selected_both_wrong": 198}, {"budget": 0.47, "threshold": -0.9933071732521057, "operator": "gt", "actual_fraction": 0.46863362933256947, "metric": 0.8961615582927528, "rescued": 935, "harmed": 251, "missed_rescue": 219, "avoided_harm": 157, "selected_both_correct": 1886, "selected_both_wrong": 200}, {"budget": 0.48, "threshold": -0.9939024448394775, "operator": "gt", "actual_fraction": 0.4798052134059009, "metric": 0.8960183328559153, "rescued": 941, "harmed": 258, "missed_rescue": 213, "avoided_harm": 150, "selected_both_correct": 1948, "selected_both_wrong": 203}, {"budget": 0.49, "threshold": -0.9944451451301575, "operator": "gt", "actual_fraction": 0.4891148668003437, "metric": 0.8970209109137783, "rescued": 953, "harmed": 263, "missed_rescue": 201, "avoided_harm": 145, "selected_both_correct": 1992, "selected_both_wrong": 207}, {"budget": 0.5, "threshold": -0.9949396848678589, "operator": "gt", "actual_fraction": 0.4992838728158121, "metric": 0.897450587224291, "rescued": 960, "harmed": 267, "missed_rescue": 194, "avoided_harm": 141, "selected_both_correct": 2048, "selected_both_wrong": 211}, {"budget": 0.51, "threshold": -0.9952448010444641, "operator": "gt", "actual_fraction": 0.5065883700945288, "metric": 0.8978802635348038, "rescued": 965, "harmed": 269, "missed_rescue": 189, "avoided_harm": 139, "selected_both_correct": 2091, "selected_both_wrong": 212}, {"budget": 0.52, "threshold": -0.9956685304641724, "operator": "gt", "actual_fraction": 0.5179031796046978, "metric": 0.8977370380979662, "rescued": 972, "harmed": 277, "missed_rescue": 182, "avoided_harm": 131, "selected_both_correct": 2154, "selected_both_wrong": 213}, {"budget": 0.53, "threshold": -0.9961755275726318, "operator": "gt", "actual_fraction": 0.528788312804354, "metric": 0.8980234889716414, "rescued": 980, "harmed": 283, "missed_rescue": 174, "avoided_harm": 125, "selected_both_correct": 2214, "selected_both_wrong": 215}, {"budget": 0.54, "threshold": -0.9965165853500366, "operator": "gt", "actual_fraction": 0.5380979661987969, "metric": 0.8988828415926668, "rescued": 989, "harmed": 286, "missed_rescue": 165, "avoided_harm": 122, "selected_both_correct": 2267, "selected_both_wrong": 215}, {"budget": 0.55, "threshold": -0.9968273043632507, "operator": "gt", "actual_fraction": 0.5489830993984531, "metric": 0.898166714408479, "rescued": 993, "harmed": 295, "missed_rescue": 161, "avoided_harm": 113, "selected_both_correct": 2330, "selected_both_wrong": 215}, {"budget": 0.56, "threshold": -0.9971103668212891, "operator": "gt", "actual_fraction": 0.5592953308507591, "metric": 0.8987396161558293, "rescued": 1001, "harmed": 299, "missed_rescue": 153, "avoided_harm": 109, "selected_both_correct": 2387, "selected_both_wrong": 218}, {"budget": 0.57, "threshold": -0.997285008430481, "operator": "gt", "actual_fraction": 0.5690346605557147, "metric": 0.899169292466342, "rescued": 1008, "harmed": 303, "missed_rescue": 146, "avoided_harm": 105, "selected_both_correct": 2440, "selected_both_wrong": 222}, {"budget": 0.58, "threshold": -0.9974491000175476, "operator": "gt", "actual_fraction": 0.5783443139501575, "metric": 0.899169292466342, "rescued": 1012, "harmed": 307, "missed_rescue": 142, "avoided_harm": 101, "selected_both_correct": 2492, "selected_both_wrong": 227}, {"budget": 0.59, "threshold": -0.9976769089698792, "operator": "gt", "actual_fraction": 0.5893726725866514, "metric": 0.899169292466342, "rescued": 1017, "harmed": 312, "missed_rescue": 137, "avoided_harm": 96, "selected_both_correct": 2558, "selected_both_wrong": 228}, {"budget": 0.6, "threshold": -0.9978172779083252, "operator": "gt", "actual_fraction": 0.5976797479232312, "metric": 0.8998854196505299, "rescued": 1024, "harmed": 314, "missed_rescue": 130, "avoided_harm": 94, "selected_both_correct": 2606, "selected_both_wrong": 229}, {"budget": 0.61, "threshold": -0.9980121850967407, "operator": "gt", "actual_fraction": 0.6084216556860499, "metric": 0.9007447722715555, "rescued": 1033, "harmed": 317, "missed_rescue": 121, "avoided_harm": 91, "selected_both_correct": 2669, "selected_both_wrong": 229}, {"budget": 0.62, "threshold": -0.9981897473335266, "operator": "gt", "actual_fraction": 0.6187338871383558, "metric": 0.9004583213978803, "rescued": 1036, "harmed": 322, "missed_rescue": 118, "avoided_harm": 86, "selected_both_correct": 2733, "selected_both_wrong": 229}, {"budget": 0.63, "threshold": -0.9982993006706238, "operator": "gt", "actual_fraction": 0.6280435405327986, "metric": 0.9014608994557434, "rescued": 1047, "harmed": 326, "missed_rescue": 107, "avoided_harm": 82, "selected_both_correct": 2783, "selected_both_wrong": 229}, {"budget": 0.64, "threshold": -0.9984021782875061, "operator": "gt", "actual_fraction": 0.6360641649957033, "metric": 0.9018905757662561, "rescued": 1054, "harmed": 330, "missed_rescue": 100, "avoided_harm": 78, "selected_both_correct": 2828, "selected_both_wrong": 229}, {"budget": 0.65, "threshold": -0.9985449314117432, "operator": "gt", "actual_fraction": 0.6465196218848468, "metric": 0.9021770266399313, "rescued": 1057, "harmed": 331, "missed_rescue": 97, "avoided_harm": 77, "selected_both_correct": 2897, "selected_both_wrong": 229}, {"budget": 0.66, "threshold": -0.9986749291419983, "operator": "gt", "actual_fraction": 0.6586937840160413, "metric": 0.9024634775136064, "rescued": 1065, "harmed": 337, "missed_rescue": 89, "avoided_harm": 71, "selected_both_correct": 2965, "selected_both_wrong": 232}, {"budget": 0.67, "threshold": -0.9987550973892212, "operator": "gt", "actual_fraction": 0.6672873102262962, "metric": 0.902606702950444, "rescued": 1069, "harmed": 340, "missed_rescue": 85, "avoided_harm": 68, "selected_both_correct": 3017, "selected_both_wrong": 233}, {"budget": 0.68, "threshold": -0.9988664388656616, "operator": "gt", "actual_fraction": 0.6793182469206531, "metric": 0.9034660555714695, "rescued": 1079, "harmed": 344, "missed_rescue": 75, "avoided_harm": 64, "selected_both_correct": 3086, "selected_both_wrong": 234}, {"budget": 0.69, "threshold": -0.9989677667617798, "operator": "gt", "actual_fraction": 0.6894872529361215, "metric": 0.9038957318819822, "rescued": 1085, "harmed": 347, "missed_rescue": 69, "avoided_harm": 61, "selected_both_correct": 3147, "selected_both_wrong": 235}, {"budget": 0.7, "threshold": -0.9990302324295044, "operator": "gt", "actual_fraction": 0.6969349756516757, "metric": 0.9033228301346319, "rescued": 1085, "harmed": 351, "missed_rescue": 69, "avoided_harm": 57, "selected_both_correct": 3195, "selected_both_wrong": 235}, {"budget": 0.71, "threshold": -0.9991169571876526, "operator": "gt", "actual_fraction": 0.7098252649670581, "metric": 0.9037525064451447, "rescued": 1091, "harmed": 354, "missed_rescue": 63, "avoided_harm": 54, "selected_both_correct": 3274, "selected_both_wrong": 237}, {"budget": 0.72, "threshold": -0.9991704225540161, "operator": "gt", "actual_fraction": 0.7189916929246635, "metric": 0.9040389573188198, "rescued": 1094, "harmed": 355, "missed_rescue": 60, "avoided_harm": 53, "selected_both_correct": 3334, "selected_both_wrong": 237}, {"budget": 0.73, "threshold": -0.9992445707321167, "operator": "gt", "actual_fraction": 0.7295903752506445, "metric": 0.9038957318819822, "rescued": 1098, "harmed": 360, "missed_rescue": 56, "avoided_harm": 48, "selected_both_correct": 3399, "selected_both_wrong": 237}, {"budget": 0.74, "threshold": -0.9992903470993042, "operator": "gt", "actual_fraction": 0.7384703523345746, "metric": 0.9040389573188198, "rescued": 1103, "harmed": 364, "missed_rescue": 51, "avoided_harm": 44, "selected_both_correct": 3450, "selected_both_wrong": 239}, {"budget": 0.75, "threshold": -0.9993333220481873, "operator": "gt", "actual_fraction": 0.7484961329132054, "metric": 0.9044686336293326, "rescued": 1108, "harmed": 366, "missed_rescue": 46, "avoided_harm": 42, "selected_both_correct": 3513, "selected_both_wrong": 239}, {"budget": 0.76, "threshold": -0.999373733997345, "operator": "gt", "actual_fraction": 0.7553709538814093, "metric": 0.9041821827556574, "rescued": 1109, "harmed": 369, "missed_rescue": 45, "avoided_harm": 39, "selected_both_correct": 3557, "selected_both_wrong": 239}, {"budget": 0.77, "threshold": -0.9994296431541443, "operator": "gt", "actual_fraction": 0.7694070466914924, "metric": 0.9047550845030078, "rescued": 1115, "harmed": 371, "missed_rescue": 39, "avoided_harm": 37, "selected_both_correct": 3645, "selected_both_wrong": 241}, {"budget": 0.78, "threshold": -0.999464213848114, "operator": "gt", "actual_fraction": 0.7782870237754225, "metric": 0.9050415353766829, "rescued": 1117, "harmed": 371, "missed_rescue": 37, "avoided_harm": 37, "selected_both_correct": 3702, "selected_both_wrong": 244}, {"budget": 0.79, "threshold": -0.9994966983795166, "operator": "gt", "actual_fraction": 0.7885992552277284, "metric": 0.9051847608135205, "rescued": 1119, "harmed": 372, "missed_rescue": 35, "avoided_harm": 36, "selected_both_correct": 3770, "selected_both_wrong": 245}, {"budget": 0.8, "threshold": -0.9995417594909668, "operator": "gt", "actual_fraction": 0.7977656831853337, "metric": 0.9048983099398453, "rescued": 1120, "harmed": 375, "missed_rescue": 34, "avoided_harm": 33, "selected_both_correct": 3829, "selected_both_wrong": 246}, {"budget": 0.81, "threshold": -0.9995827078819275, "operator": "gt", "actual_fraction": 0.8090804926955028, "metric": 0.9050415353766829, "rescued": 1123, "harmed": 377, "missed_rescue": 31, "avoided_harm": 31, "selected_both_correct": 3903, "selected_both_wrong": 246}, {"budget": 0.82, "threshold": -0.9996140599250793, "operator": "gt", "actual_fraction": 0.8198224004583214, "metric": 0.905327986250358, "rescued": 1127, "harmed": 379, "missed_rescue": 27, "avoided_harm": 29, "selected_both_correct": 3972, "selected_both_wrong": 246}, {"budget": 0.83, "threshold": -0.9996429681777954, "operator": "gt", "actual_fraction": 0.8281294757949011, "metric": 0.9051847608135205, "rescued": 1128, "harmed": 381, "missed_rescue": 26, "avoided_harm": 27, "selected_both_correct": 4027, "selected_both_wrong": 246}, {"budget": 0.84, "threshold": -0.9996750354766846, "operator": "gt", "actual_fraction": 0.8395875107419077, "metric": 0.9054712116871957, "rescued": 1131, "harmed": 382, "missed_rescue": 23, "avoided_harm": 26, "selected_both_correct": 4102, "selected_both_wrong": 247}, {"budget": 0.85, "threshold": -0.9997040629386902, "operator": "gt", "actual_fraction": 0.8458894299627614, "metric": 0.9056144371240332, "rescued": 1132, "harmed": 382, "missed_rescue": 22, "avoided_harm": 26, "selected_both_correct": 4143, "selected_both_wrong": 249}, {"budget": 0.86, "threshold": -0.9997387528419495, "operator": "gt", "actual_fraction": 0.8587797192781438, "metric": 0.9057576625608708, "rescued": 1134, "harmed": 383, "missed_rescue": 20, "avoided_harm": 25, "selected_both_correct": 4230, "selected_both_wrong": 249}, {"budget": 0.87, "threshold": -0.9997695088386536, "operator": "gt", "actual_fraction": 0.8693784016041249, "metric": 0.9057576625608708, "rescued": 1136, "harmed": 385, "missed_rescue": 18, "avoided_harm": 23, "selected_both_correct": 4300, "selected_both_wrong": 249}, {"budget": 0.88, "threshold": -0.999783456325531, "operator": "gt", "actual_fraction": 0.8749641936407906, "metric": 0.9057576625608708, "rescued": 1136, "harmed": 385, "missed_rescue": 18, "avoided_harm": 23, "selected_both_correct": 4339, "selected_both_wrong": 249}, {"budget": 0.89, "threshold": -0.9998205304145813, "operator": "gt", "actual_fraction": 0.8894299627613864, "metric": 0.905327986250358, "rescued": 1137, "harmed": 389, "missed_rescue": 17, "avoided_harm": 19, "selected_both_correct": 4434, "selected_both_wrong": 250}, {"budget": 0.9, "threshold": -0.999841570854187, "operator": "gt", "actual_fraction": 0.8988828415926668, "metric": 0.905327986250358, "rescued": 1140, "harmed": 392, "missed_rescue": 14, "avoided_harm": 16, "selected_both_correct": 4494, "selected_both_wrong": 250}, {"budget": 0.91, "threshold": -0.9998601675033569, "operator": "gt", "actual_fraction": 0.9093382984818104, "metric": 0.9054712116871957, "rescued": 1143, "harmed": 394, "missed_rescue": 11, "avoided_harm": 14, "selected_both_correct": 4562, "selected_both_wrong": 250}, {"budget": 0.92, "threshold": -0.9998766183853149, "operator": "gt", "actual_fraction": 0.9185047264394156, "metric": 0.9051847608135205, "rescued": 1144, "harmed": 397, "missed_rescue": 10, "avoided_harm": 11, "selected_both_correct": 4620, "selected_both_wrong": 252}, {"budget": 0.93, "threshold": -0.9998977184295654, "operator": "gt", "actual_fraction": 0.9276711543970209, "metric": 0.905327986250358, "rescued": 1145, "harmed": 397, "missed_rescue": 9, "avoided_harm": 11, "selected_both_correct": 4683, "selected_both_wrong": 252}, {"budget": 0.94, "threshold": -0.9999151229858398, "operator": "gt", "actual_fraction": 0.9394156402177026, "metric": 0.905327986250358, "rescued": 1147, "harmed": 399, "missed_rescue": 7, "avoided_harm": 9, "selected_both_correct": 4760, "selected_both_wrong": 253}, {"budget": 0.95, "threshold": -0.999931812286377, "operator": "gt", "actual_fraction": 0.9497278716700086, "metric": 0.9051847608135205, "rescued": 1148, "harmed": 401, "missed_rescue": 6, "avoided_harm": 7, "selected_both_correct": 4829, "selected_both_wrong": 253}, {"budget": 0.96, "threshold": -0.9999452829360962, "operator": "gt", "actual_fraction": 0.9590375250644515, "metric": 0.9054712116871957, "rescued": 1151, "harmed": 402, "missed_rescue": 3, "avoided_harm": 6, "selected_both_correct": 4889, "selected_both_wrong": 254}, {"budget": 0.97, "threshold": -0.9999599456787109, "operator": "gt", "actual_fraction": 0.969492981953595, "metric": 0.9050415353766829, "rescued": 1151, "harmed": 405, "missed_rescue": 3, "avoided_harm": 3, "selected_both_correct": 4959, "selected_both_wrong": 254}, {"budget": 0.98, "threshold": -0.9999687671661377, "operator": "gt", "actual_fraction": 0.9783729590375251, "metric": 0.9047550845030078, "rescued": 1151, "harmed": 407, "missed_rescue": 3, "avoided_harm": 1, "selected_both_correct": 5019, "selected_both_wrong": 254}, {"budget": 0.99, "threshold": -0.9999842643737793, "operator": "gt", "actual_fraction": 0.9898309939845317, "metric": 0.9048983099398453, "rescued": 1153, "harmed": 408, "missed_rescue": 1, "avoided_harm": 0, "selected_both_correct": 5096, "selected_both_wrong": 254}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.9050415353766829, "rescued": 1154, "harmed": 408, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 5165, "selected_both_wrong": 255}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.8000287005811868, "rescued": 0, "harmed": 0, "missed_rescue": 2247, "avoided_harm": 845, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.5245773792266846, "operator": "gt", "actual_fraction": 0.011623735380641457, "metric": 0.8039032790414006, "rescued": 70, "harmed": 16, "missed_rescue": 2177, "avoided_harm": 829, "selected_both_correct": 66, "selected_both_wrong": 10}, {"budget": 0.02, "threshold": -0.5456489324569702, "operator": "gt", "actual_fraction": 0.021453684437109852, "metric": 0.8067733371600775, "rescued": 122, "harmed": 28, "missed_rescue": 2125, "avoided_harm": 817, "selected_both_correct": 124, "selected_both_wrong": 25}, {"budget": 0.03, "threshold": -0.5652987360954285, "operator": "ge", "actual_fraction": 0.02970510152830595, "metric": 0.8093563894668867, "rescued": 169, "harmed": 39, "missed_rescue": 2078, "avoided_harm": 806, "selected_both_correct": 169, "selected_both_wrong": 37}, {"budget": 0.04, "threshold": -0.5851011276245117, "operator": "ge", "actual_fraction": 0.03795651861950204, "metric": 0.8113654301499605, "rescued": 209, "harmed": 51, "missed_rescue": 2038, "avoided_harm": 794, "selected_both_correct": 224, "selected_both_wrong": 45}, {"budget": 0.05, "threshold": -0.6055658459663391, "operator": "ge", "actual_fraction": 0.047212456052235056, "metric": 0.8140202339097367, "rescued": 259, "harmed": 64, "missed_rescue": 1988, "avoided_harm": 781, "selected_both_correct": 283, "selected_both_wrong": 52}, {"budget": 0.06, "threshold": -0.6293190717697144, "operator": "ge", "actual_fraction": 0.057759919638372674, "metric": 0.817464303652149, "rescued": 318, "harmed": 75, "missed_rescue": 1929, "avoided_harm": 770, "selected_both_correct": 345, "selected_both_wrong": 67}, {"budget": 0.07, "threshold": -0.6477982401847839, "operator": "ge", "actual_fraction": 0.06601133672956877, "metric": 0.8201908588648921, "rescued": 365, "harmed": 84, "missed_rescue": 1882, "avoided_harm": 761, "selected_both_correct": 399, "selected_both_wrong": 72}, {"budget": 0.08, "threshold": -0.6693127155303955, "operator": "ge", "actual_fraction": 0.07483676544450026, "metric": 0.8232761713424697, "rescued": 417, "harmed": 93, "missed_rescue": 1830, "avoided_harm": 752, "selected_both_correct": 451, "selected_both_wrong": 82}, {"budget": 0.09, "threshold": -0.6888871788978577, "operator": "gt", "actual_fraction": 0.08488196885986941, "metric": 0.8261462294611466, "rescued": 464, "harmed": 100, "missed_rescue": 1783, "avoided_harm": 745, "selected_both_correct": 524, "selected_both_wrong": 95}, {"budget": 0.1, "threshold": -0.7061904072761536, "operator": "gt", "actual_fraction": 0.09356389466886704, "metric": 0.8290162875798235, "rescued": 511, "harmed": 107, "missed_rescue": 1736, "avoided_harm": 738, "selected_both_correct": 578, "selected_both_wrong": 108}, {"budget": 0.11, "threshold": -0.7233095169067383, "operator": "ge", "actual_fraction": 0.1026045777426993, "metric": 0.8319580971514673, "rescued": 558, "harmed": 113, "missed_rescue": 1689, "avoided_harm": 732, "selected_both_correct": 643, "selected_both_wrong": 116}, {"budget": 0.12, "threshold": -0.7424216866493225, "operator": "ge", "actual_fraction": 0.11322379278180383, "metric": 0.8348999067231111, "rescued": 617, "harmed": 131, "missed_rescue": 1630, "avoided_harm": 714, "selected_both_correct": 708, "selected_both_wrong": 122}, {"budget": 0.13, "threshold": -0.7627776861190796, "operator": "gt", "actual_fraction": 0.12255148166750376, "metric": 0.8375547104828873, "rescued": 664, "harmed": 141, "missed_rescue": 1583, "avoided_harm": 704, "selected_both_correct": 775, "selected_both_wrong": 128}, {"budget": 0.14, "threshold": -0.7786493301391602, "operator": "gt", "actual_fraction": 0.13137691038243524, "metric": 0.839635502618928, "rescued": 705, "harmed": 153, "missed_rescue": 1542, "avoided_harm": 692, "selected_both_correct": 828, "selected_both_wrong": 145}, {"budget": 0.15, "threshold": -0.7950222492218018, "operator": "ge", "actual_fraction": 0.14056109636220132, "metric": 0.8421468034727704, "rescued": 753, "harmed": 166, "missed_rescue": 1494, "avoided_harm": 679, "selected_both_correct": 884, "selected_both_wrong": 156}, {"budget": 0.16, "threshold": -0.8140527606010437, "operator": "ge", "actual_fraction": 0.15096505704240512, "metric": 0.8448733586855134, "rescued": 804, "harmed": 179, "missed_rescue": 1443, "avoided_harm": 666, "selected_both_correct": 955, "selected_both_wrong": 166}, {"budget": 0.17, "threshold": -0.8327813148498535, "operator": "ge", "actual_fraction": 0.1613690177226089, "metric": 0.8468823993685872, "rescued": 844, "harmed": 191, "missed_rescue": 1403, "avoided_harm": 654, "selected_both_correct": 1033, "selected_both_wrong": 181}, {"budget": 0.18, "threshold": -0.8464508652687073, "operator": "gt", "actual_fraction": 0.1702661978905073, "metric": 0.8490349429575949, "rescued": 882, "harmed": 199, "missed_rescue": 1365, "avoided_harm": 646, "selected_both_correct": 1102, "selected_both_wrong": 190}, {"budget": 0.19, "threshold": -0.858244001865387, "operator": "ge", "actual_fraction": 0.17916337805840568, "metric": 0.851761498170338, "rescued": 932, "harmed": 211, "missed_rescue": 1315, "avoided_harm": 634, "selected_both_correct": 1158, "selected_both_wrong": 196}, {"budget": 0.2, "threshold": -0.8701565861701965, "operator": "ge", "actual_fraction": 0.18798880677333715, "metric": 0.8539857932123125, "rescued": 972, "harmed": 220, "missed_rescue": 1275, "avoided_harm": 625, "selected_both_correct": 1228, "selected_both_wrong": 200}, {"budget": 0.21, "threshold": -0.8840392827987671, "operator": "gt", "actual_fraction": 0.19659898112936788, "metric": 0.855420822271651, "rescued": 1004, "harmed": 232, "missed_rescue": 1243, "avoided_harm": 613, "selected_both_correct": 1293, "selected_both_wrong": 211}, {"budget": 0.22, "threshold": -0.8933094143867493, "operator": "ge", "actual_fraction": 0.20642893018583627, "metric": 0.8580038745784602, "rescued": 1047, "harmed": 239, "missed_rescue": 1200, "avoided_harm": 606, "selected_both_correct": 1373, "selected_both_wrong": 218}, {"budget": 0.23, "threshold": -0.9046505093574524, "operator": "gt", "actual_fraction": 0.21711989667790774, "metric": 0.860012915261534, "rescued": 1088, "harmed": 252, "missed_rescue": 1159, "avoided_harm": 593, "selected_both_correct": 1453, "selected_both_wrong": 233}, {"budget": 0.24, "threshold": -0.9136765599250793, "operator": "ge", "actual_fraction": 0.22766736026404535, "metric": 0.8627394704742771, "rescued": 1134, "harmed": 260, "missed_rescue": 1113, "avoided_harm": 585, "selected_both_correct": 1534, "selected_both_wrong": 245}, {"budget": 0.25, "threshold": -0.9230391979217529, "operator": "gt", "actual_fraction": 0.23785606658534836, "metric": 0.865609528592954, "rescued": 1183, "harmed": 269, "missed_rescue": 1064, "avoided_harm": 576, "selected_both_correct": 1610, "selected_both_wrong": 253}, {"budget": 0.26, "threshold": -0.9314625263214111, "operator": "gt", "actual_fraction": 0.24804477290665136, "metric": 0.8672598120111932, "rescued": 1221, "harmed": 284, "missed_rescue": 1026, "avoided_harm": 561, "selected_both_correct": 1685, "selected_both_wrong": 267}, {"budget": 0.27, "threshold": -0.9376689791679382, "operator": "ge", "actual_fraction": 0.25751596469828514, "metric": 0.8686230896175647, "rescued": 1253, "harmed": 297, "missed_rescue": 994, "avoided_harm": 548, "selected_both_correct": 1760, "selected_both_wrong": 279}, {"budget": 0.28, "threshold": -0.9449946880340576, "operator": "gt", "actual_fraction": 0.26863743990815814, "metric": 0.8702016215828371, "rescued": 1285, "harmed": 307, "missed_rescue": 962, "avoided_harm": 538, "selected_both_correct": 1858, "selected_both_wrong": 294}, {"budget": 0.29, "threshold": -0.9511421918869019, "operator": "gt", "actual_fraction": 0.2793284064002296, "metric": 0.872067159359977, "rescued": 1328, "harmed": 324, "missed_rescue": 919, "avoided_harm": 521, "selected_both_correct": 1938, "selected_both_wrong": 303}, {"budget": 0.3, "threshold": -0.9559813141822815, "operator": "gt", "actual_fraction": 0.2897323670804334, "metric": 0.8733586855133817, "rescued": 1356, "harmed": 334, "missed_rescue": 891, "avoided_harm": 511, "selected_both_correct": 2036, "selected_both_wrong": 312}, {"budget": 0.31, "threshold": -0.9597620368003845, "operator": "ge", "actual_fraction": 0.2999928248547033, "metric": 0.8741479514960178, "rescued": 1382, "harmed": 349, "missed_rescue": 865, "avoided_harm": 496, "selected_both_correct": 2133, "selected_both_wrong": 317}, {"budget": 0.32, "threshold": -0.9643214344978333, "operator": "gt", "actual_fraction": 0.3091770108344694, "metric": 0.8758699863672239, "rescued": 1416, "harmed": 359, "missed_rescue": 831, "avoided_harm": 486, "selected_both_correct": 2209, "selected_both_wrong": 325}, {"budget": 0.33, "threshold": -0.9686195850372314, "operator": "gt", "actual_fraction": 0.3218770180096147, "metric": 0.8774485183324963, "rescued": 1451, "harmed": 372, "missed_rescue": 796, "avoided_harm": 473, "selected_both_correct": 2333, "selected_both_wrong": 330}, {"budget": 0.34, "threshold": -0.9719924926757812, "operator": "gt", "actual_fraction": 0.33127645834828157, "metric": 0.8790988017507354, "rescued": 1484, "harmed": 382, "missed_rescue": 763, "avoided_harm": 463, "selected_both_correct": 2416, "selected_both_wrong": 335}, {"budget": 0.35, "threshold": -0.974821150302887, "operator": "ge", "actual_fraction": 0.3431871995407907, "metric": 0.8801750735452393, "rescued": 1514, "harmed": 397, "missed_rescue": 733, "avoided_harm": 448, "selected_both_correct": 2529, "selected_both_wrong": 343}, {"budget": 0.36, "threshold": -0.9777138829231262, "operator": "gt", "actual_fraction": 0.35280189423835834, "metric": 0.8812513453397431, "rescued": 1539, "harmed": 407, "missed_rescue": 708, "avoided_harm": 438, "selected_both_correct": 2623, "selected_both_wrong": 348}, {"budget": 0.37, "threshold": -0.9802808165550232, "operator": "gt", "actual_fraction": 0.36421037526009903, "metric": 0.8827581258520485, "rescued": 1569, "harmed": 416, "missed_rescue": 678, "avoided_harm": 429, "selected_both_correct": 2740, "selected_both_wrong": 351}, {"budget": 0.38, "threshold": -0.982557475566864, "operator": "gt", "actual_fraction": 0.3761928679055751, "metric": 0.8843366578173208, "rescued": 1602, "harmed": 427, "missed_rescue": 645, "avoided_harm": 418, "selected_both_correct": 2856, "selected_both_wrong": 358}, {"budget": 0.39, "threshold": -0.9843363761901855, "operator": "gt", "actual_fraction": 0.38803185764511733, "metric": 0.8857716868766593, "rescued": 1630, "harmed": 435, "missed_rescue": 617, "avoided_harm": 410, "selected_both_correct": 2978, "selected_both_wrong": 365}, {"budget": 0.4, "threshold": -0.9863631129264832, "operator": "gt", "actual_fraction": 0.4008036162732295, "metric": 0.8869197101241301, "rescued": 1658, "harmed": 447, "missed_rescue": 589, "avoided_harm": 398, "selected_both_correct": 3105, "selected_both_wrong": 376}, {"budget": 0.41, "threshold": -0.9879462122917175, "operator": "gt", "actual_fraction": 0.41156633421826794, "metric": 0.8880677333716007, "rescued": 1685, "harmed": 458, "missed_rescue": 562, "avoided_harm": 387, "selected_both_correct": 3212, "selected_both_wrong": 381}, {"budget": 0.42, "threshold": -0.9890130758285522, "operator": "gt", "actual_fraction": 0.4204635143861663, "metric": 0.8884264906364354, "rescued": 1701, "harmed": 469, "missed_rescue": 546, "avoided_harm": 376, "selected_both_correct": 3303, "selected_both_wrong": 387}, {"budget": 0.43, "threshold": -0.9899863600730896, "operator": "gt", "actual_fraction": 0.4293606945540647, "metric": 0.8889287508072038, "rescued": 1720, "harmed": 481, "missed_rescue": 527, "avoided_harm": 364, "selected_both_correct": 3391, "selected_both_wrong": 392}, {"budget": 0.44, "threshold": -0.9911527037620544, "operator": "gt", "actual_fraction": 0.44119968429360695, "metric": 0.890579034225443, "rescued": 1755, "harmed": 493, "missed_rescue": 492, "avoided_harm": 352, "selected_both_correct": 3502, "selected_both_wrong": 399}, {"budget": 0.45, "threshold": -0.9919379353523254, "operator": "gt", "actual_fraction": 0.4516753964267776, "metric": 0.8913683002080792, "rescued": 1778, "harmed": 505, "missed_rescue": 469, "avoided_harm": 340, "selected_both_correct": 3608, "selected_both_wrong": 404}, {"budget": 0.46, "threshold": -0.9926542043685913, "operator": "gt", "actual_fraction": 0.4611465882184114, "metric": 0.89251632345555, "rescued": 1797, "harmed": 508, "missed_rescue": 450, "avoided_harm": 337, "selected_both_correct": 3715, "selected_both_wrong": 407}, {"budget": 0.47, "threshold": -0.9933071732521057, "operator": "gt", "actual_fraction": 0.4706895314630121, "metric": 0.8933055894381862, "rescued": 1814, "harmed": 514, "missed_rescue": 433, "avoided_harm": 331, "selected_both_correct": 3823, "selected_both_wrong": 409}, {"budget": 0.48, "threshold": -0.9939024448394775, "operator": "gt", "actual_fraction": 0.481021740690249, "metric": 0.8941666068737892, "rescued": 1835, "harmed": 523, "missed_rescue": 412, "avoided_harm": 322, "selected_both_correct": 3932, "selected_both_wrong": 414}, {"budget": 0.49, "threshold": -0.9944451451301575, "operator": "gt", "actual_fraction": 0.4912821984645189, "metric": 0.8946688670445576, "rescued": 1853, "harmed": 534, "missed_rescue": 394, "avoided_harm": 311, "selected_both_correct": 4037, "selected_both_wrong": 423}, {"budget": 0.5, "threshold": -0.9949396848678589, "operator": "gt", "actual_fraction": 0.4992466097438473, "metric": 0.89531463012126, "rescued": 1869, "harmed": 541, "missed_rescue": 378, "avoided_harm": 304, "selected_both_correct": 4122, "selected_both_wrong": 426}, {"budget": 0.51, "threshold": -0.9952448010444641, "operator": "gt", "actual_fraction": 0.5059912463227381, "metric": 0.8957451388390615, "rescued": 1882, "harmed": 548, "missed_rescue": 365, "avoided_harm": 297, "selected_both_correct": 4196, "selected_both_wrong": 426}, {"budget": 0.52, "threshold": -0.9956685304641724, "operator": "gt", "actual_fraction": 0.5140991605080003, "metric": 0.8960321446509292, "rescued": 1894, "harmed": 556, "missed_rescue": 353, "avoided_harm": 289, "selected_both_correct": 4286, "selected_both_wrong": 429}, {"budget": 0.53, "threshold": -0.9961755275726318, "operator": "gt", "actual_fraction": 0.529166965631054, "metric": 0.8973954222573007, "rescued": 1921, "harmed": 564, "missed_rescue": 326, "avoided_harm": 281, "selected_both_correct": 4451, "selected_both_wrong": 439}, {"budget": 0.54, "threshold": -0.9965165853500366, "operator": "gt", "actual_fraction": 0.5406471981057617, "metric": 0.8981129367869699, "rescued": 1940, "harmed": 573, "missed_rescue": 307, "avoided_harm": 272, "selected_both_correct": 4575, "selected_both_wrong": 447}, {"budget": 0.55, "threshold": -0.9968273043632507, "operator": "gt", "actual_fraction": 0.5504771471622301, "metric": 0.8979694338810361, "rescued": 1947, "harmed": 582, "missed_rescue": 300, "avoided_harm": 263, "selected_both_correct": 4695, "selected_both_wrong": 448}, {"budget": 0.56, "threshold": -0.9971103668212891, "operator": "gt", "actual_fraction": 0.5621726339958384, "metric": 0.8986151969577384, "rescued": 1966, "harmed": 592, "missed_rescue": 281, "avoided_harm": 253, "selected_both_correct": 4825, "selected_both_wrong": 452}, {"budget": 0.57, "threshold": -0.997285008430481, "operator": "gt", "actual_fraction": 0.5677692473272584, "metric": 0.898973954222573, "rescued": 1977, "harmed": 598, "missed_rescue": 270, "avoided_harm": 247, "selected_both_correct": 4884, "selected_both_wrong": 454}, {"budget": 0.58, "threshold": -0.9974491000175476, "operator": "gt", "actual_fraction": 0.5759489129654876, "metric": 0.8992609600344407, "rescued": 1990, "harmed": 607, "missed_rescue": 257, "avoided_harm": 238, "selected_both_correct": 4973, "selected_both_wrong": 457}, {"budget": 0.59, "threshold": -0.9976769089698792, "operator": "gt", "actual_fraction": 0.5864963765516251, "metric": 0.899834971658176, "rescued": 2008, "harmed": 617, "missed_rescue": 239, "avoided_harm": 228, "selected_both_correct": 5091, "selected_both_wrong": 458}, {"budget": 0.6, "threshold": -0.9978172779083252, "operator": "gt", "actual_fraction": 0.5942455334720528, "metric": 0.899834971658176, "rescued": 2016, "harmed": 625, "missed_rescue": 231, "avoided_harm": 220, "selected_both_correct": 5179, "selected_both_wrong": 462}, {"budget": 0.61, "threshold": -0.9980121850967407, "operator": "gt", "actual_fraction": 0.6054387601348927, "metric": 0.8994044629403746, "rescued": 2025, "harmed": 640, "missed_rescue": 222, "avoided_harm": 205, "selected_both_correct": 5308, "selected_both_wrong": 465}, {"budget": 0.62, "threshold": -0.9981897473335266, "operator": "gt", "actual_fraction": 0.6165602353447658, "metric": 0.9001219774700437, "rescued": 2039, "harmed": 644, "missed_rescue": 208, "avoided_harm": 201, "selected_both_correct": 5438, "selected_both_wrong": 472}, {"budget": 0.63, "threshold": -0.9982993006706238, "operator": "gt", "actual_fraction": 0.6242376408122264, "metric": 0.9004089832819114, "rescued": 2049, "harmed": 650, "missed_rescue": 198, "avoided_harm": 195, "selected_both_correct": 5528, "selected_both_wrong": 473}, {"budget": 0.64, "threshold": -0.9984021782875061, "operator": "gt", "actual_fraction": 0.6322020520915549, "metric": 0.900767740546746, "rescued": 2058, "harmed": 654, "missed_rescue": 189, "avoided_harm": 191, "selected_both_correct": 5624, "selected_both_wrong": 475}, {"budget": 0.65, "threshold": -0.9985449314117432, "operator": "gt", "actual_fraction": 0.6454043194374686, "metric": 0.9011982492645476, "rescued": 2072, "harmed": 662, "missed_rescue": 175, "avoided_harm": 183, "selected_both_correct": 5784, "selected_both_wrong": 477}, {"budget": 0.66, "threshold": -0.9986749291419983, "operator": "gt", "actual_fraction": 0.6566692975532754, "metric": 0.9017005094353161, "rescued": 2085, "harmed": 668, "missed_rescue": 162, "avoided_harm": 177, "selected_both_correct": 5920, "selected_both_wrong": 479}, {"budget": 0.67, "threshold": -0.9987550973892212, "operator": "gt", "actual_fraction": 0.6646337088326039, "metric": 0.9017005094353161, "rescued": 2089, "harmed": 672, "missed_rescue": 158, "avoided_harm": 173, "selected_both_correct": 6021, "selected_both_wrong": 481}, {"budget": 0.68, "threshold": -0.9988664388656616, "operator": "gt", "actual_fraction": 0.6776207218196169, "metric": 0.9021310181531176, "rescued": 2102, "harmed": 679, "missed_rescue": 145, "avoided_harm": 166, "selected_both_correct": 6181, "selected_both_wrong": 482}, {"budget": 0.69, "threshold": -0.9989677667617798, "operator": "gt", "actual_fraction": 0.690464231900696, "metric": 0.9020592667001507, "rescued": 2113, "harmed": 691, "missed_rescue": 134, "avoided_harm": 154, "selected_both_correct": 6336, "selected_both_wrong": 483}, {"budget": 0.7, "threshold": -0.9990302324295044, "operator": "gt", "actual_fraction": 0.7000789265982637, "metric": 0.9024180239649853, "rescued": 2123, "harmed": 696, "missed_rescue": 124, "avoided_harm": 149, "selected_both_correct": 6453, "selected_both_wrong": 485}, {"budget": 0.71, "threshold": -0.9991169571876526, "operator": "gt", "actual_fraction": 0.7121331706967066, "metric": 0.901772260888283, "rescued": 2130, "harmed": 712, "missed_rescue": 117, "avoided_harm": 133, "selected_both_correct": 6595, "selected_both_wrong": 488}, {"budget": 0.72, "threshold": -0.9991704225540161, "operator": "gt", "actual_fraction": 0.7201693334290019, "metric": 0.9017005094353161, "rescued": 2136, "harmed": 719, "missed_rescue": 111, "avoided_harm": 126, "selected_both_correct": 6693, "selected_both_wrong": 489}, {"budget": 0.73, "threshold": -0.9992445707321167, "operator": "gt", "actual_fraction": 0.7310755542799742, "metric": 0.9020592667001507, "rescued": 2143, "harmed": 721, "missed_rescue": 104, "avoided_harm": 124, "selected_both_correct": 6833, "selected_both_wrong": 492}, {"budget": 0.74, "threshold": -0.9992903470993042, "operator": "gt", "actual_fraction": 0.7394704742771041, "metric": 0.9018440123412499, "rescued": 2147, "harmed": 728, "missed_rescue": 100, "avoided_harm": 117, "selected_both_correct": 6937, "selected_both_wrong": 494}, {"budget": 0.75, "threshold": -0.9993333220481873, "operator": "gt", "actual_fraction": 0.7467173710267633, "metric": 0.9018440123412499, "rescued": 2151, "harmed": 732, "missed_rescue": 96, "avoided_harm": 113, "selected_both_correct": 7028, "selected_both_wrong": 496}, {"budget": 0.76, "threshold": -0.999373733997345, "operator": "gt", "actual_fraction": 0.7550405395709263, "metric": 0.9016287579823491, "rescued": 2154, "harmed": 738, "missed_rescue": 93, "avoided_harm": 107, "selected_both_correct": 7133, "selected_both_wrong": 498}, {"budget": 0.77, "threshold": -0.9994296431541443, "operator": "gt", "actual_fraction": 0.7659467604218986, "metric": 0.9018440123412499, "rescued": 2164, "harmed": 745, "missed_rescue": 83, "avoided_harm": 100, "selected_both_correct": 7267, "selected_both_wrong": 499}, {"budget": 0.78, "threshold": -0.999464213848114, "operator": "gt", "actual_fraction": 0.7743416804190285, "metric": 0.9022027696060845, "rescued": 2172, "harmed": 748, "missed_rescue": 75, "avoided_harm": 97, "selected_both_correct": 7371, "selected_both_wrong": 501}, {"budget": 0.79, "threshold": -0.9994966983795166, "operator": "gt", "actual_fraction": 0.7820908373394562, "metric": 0.9022745210590515, "rescued": 2178, "harmed": 753, "missed_rescue": 69, "avoided_harm": 92, "selected_both_correct": 7467, "selected_both_wrong": 502}, {"budget": 0.8, "threshold": -0.9995417594909668, "operator": "gt", "actual_fraction": 0.7935710698141637, "metric": 0.9023462725120184, "rescued": 2182, "harmed": 756, "missed_rescue": 65, "avoided_harm": 89, "selected_both_correct": 7616, "selected_both_wrong": 506}, {"budget": 0.81, "threshold": -0.9995827078819275, "operator": "gt", "actual_fraction": 0.8052665566477721, "metric": 0.9025615268709192, "rescued": 2189, "harmed": 760, "missed_rescue": 58, "avoided_harm": 85, "selected_both_correct": 7765, "selected_both_wrong": 509}, {"budget": 0.82, "threshold": -0.9996140599250793, "operator": "gt", "actual_fraction": 0.8185405754466528, "metric": 0.9026332783238861, "rescued": 2196, "harmed": 766, "missed_rescue": 51, "avoided_harm": 79, "selected_both_correct": 7933, "selected_both_wrong": 513}, {"budget": 0.83, "threshold": -0.9996429681777954, "operator": "gt", "actual_fraction": 0.8264332352730143, "metric": 0.9026332783238861, "rescued": 2200, "harmed": 770, "missed_rescue": 47, "avoided_harm": 75, "selected_both_correct": 8034, "selected_both_wrong": 514}, {"budget": 0.84, "threshold": -0.9996750354766846, "operator": "gt", "actual_fraction": 0.836837195953218, "metric": 0.9025615268709192, "rescued": 2202, "harmed": 773, "missed_rescue": 45, "avoided_harm": 72, "selected_both_correct": 8172, "selected_both_wrong": 516}, {"budget": 0.85, "threshold": -0.9997040629386902, "operator": "gt", "actual_fraction": 0.844299347061778, "metric": 0.9025615268709192, "rescued": 2206, "harmed": 777, "missed_rescue": 41, "avoided_harm": 68, "selected_both_correct": 8267, "selected_both_wrong": 517}, {"budget": 0.86, "threshold": -0.9997387528419495, "operator": "gt", "actual_fraction": 0.8580038745784602, "metric": 0.9024180239649853, "rescued": 2213, "harmed": 786, "missed_rescue": 34, "avoided_harm": 59, "selected_both_correct": 8437, "selected_both_wrong": 522}, {"budget": 0.87, "threshold": -0.9997695088386536, "operator": "gt", "actual_fraction": 0.870273373035804, "metric": 0.9026332783238861, "rescued": 2220, "harmed": 790, "missed_rescue": 27, "avoided_harm": 55, "selected_both_correct": 8594, "selected_both_wrong": 525}, {"budget": 0.88, "threshold": -0.999783456325531, "operator": "gt", "actual_fraction": 0.8750807203845878, "metric": 0.9024180239649853, "rescued": 2220, "harmed": 793, "missed_rescue": 27, "avoided_harm": 52, "selected_both_correct": 8657, "selected_both_wrong": 526}, {"budget": 0.89, "threshold": -0.9998205304145813, "operator": "gt", "actual_fraction": 0.8908660400373107, "metric": 0.9022745210590515, "rescued": 2223, "harmed": 798, "missed_rescue": 24, "avoided_harm": 47, "selected_both_correct": 8865, "selected_both_wrong": 530}, {"budget": 0.9, "threshold": -0.999841570854187, "operator": "gt", "actual_fraction": 0.8992609600344407, "metric": 0.9023462725120184, "rescued": 2225, "harmed": 799, "missed_rescue": 22, "avoided_harm": 46, "selected_both_correct": 8979, "selected_both_wrong": 530}, {"budget": 0.91, "threshold": -0.9998601675033569, "operator": "gt", "actual_fraction": 0.9091626605438761, "metric": 0.9023462725120184, "rescued": 2227, "harmed": 801, "missed_rescue": 20, "avoided_harm": 44, "selected_both_correct": 9111, "selected_both_wrong": 532}, {"budget": 0.92, "threshold": -0.9998766183853149, "operator": "gt", "actual_fraction": 0.9171270718232044, "metric": 0.9023462725120184, "rescued": 2230, "harmed": 804, "missed_rescue": 17, "avoided_harm": 41, "selected_both_correct": 9215, "selected_both_wrong": 533}, {"budget": 0.93, "threshold": -0.9998977184295654, "operator": "gt", "actual_fraction": 0.9273875295974744, "metric": 0.9020592667001507, "rescued": 2232, "harmed": 810, "missed_rescue": 15, "avoided_harm": 35, "selected_both_correct": 9350, "selected_both_wrong": 533}, {"budget": 0.94, "threshold": -0.9999151229858398, "operator": "gt", "actual_fraction": 0.9375044844658105, "metric": 0.9018440123412499, "rescued": 2235, "harmed": 816, "missed_rescue": 12, "avoided_harm": 29, "selected_both_correct": 9481, "selected_both_wrong": 534}, {"budget": 0.95, "threshold": -0.999931812286377, "operator": "gt", "actual_fraction": 0.948984716940518, "metric": 0.9016287579823491, "rescued": 2239, "harmed": 823, "missed_rescue": 8, "avoided_harm": 22, "selected_both_correct": 9629, "selected_both_wrong": 535}, {"budget": 0.96, "threshold": -0.9999452829360962, "operator": "gt", "actual_fraction": 0.9576666427495156, "metric": 0.9015570065293822, "rescued": 2241, "harmed": 826, "missed_rescue": 6, "avoided_harm": 19, "selected_both_correct": 9745, "selected_both_wrong": 535}, {"budget": 0.97, "threshold": -0.9999599456787109, "operator": "gt", "actual_fraction": 0.9709406615483963, "metric": 0.9009829949056468, "rescued": 2241, "harmed": 834, "missed_rescue": 6, "avoided_harm": 11, "selected_both_correct": 9922, "selected_both_wrong": 535}, {"budget": 0.98, "threshold": -0.9999687671661377, "operator": "gt", "actual_fraction": 0.9784745641099232, "metric": 0.900839491999713, "rescued": 2241, "harmed": 836, "missed_rescue": 6, "avoided_harm": 9, "selected_both_correct": 10023, "selected_both_wrong": 537}, {"budget": 0.99, "threshold": -0.9999842643737793, "operator": "gt", "actual_fraction": 0.98973954222573, "metric": 0.900839491999713, "rescued": 2245, "harmed": 840, "missed_rescue": 2, "avoided_harm": 5, "selected_both_correct": 10170, "selected_both_wrong": 539}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.9006242376408122, "rescued": 2247, "harmed": 845, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 10305, "selected_both_wrong": 540}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.9108851331996561, "actual_fraction": 0.8419077628186766, "metric_std": 0.0008967330760526022, "actual_fraction_std": 0.03832550514043027, "per_seed": [{"seed": 2026, "budget": 0.85, "threshold": -0.9734646185051644, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.911486680034374, "actual_fraction": 0.8498997421942137, "rescue": 1119, "harm": 328, "missed_rescue": 35}, {"seed": 2027, "budget": 0.79, "threshold": -0.9487407074812001, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.911486680034374, "actual_fraction": 0.7898882841592667, "rescue": 1108, "harm": 317, "missed_rescue": 46}, {"seed": 2028, "budget": 0.89, "threshold": -0.9906119153114747, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.9109137782870236, "actual_fraction": 0.8898596390718991, "rescue": 1135, "harm": 348, "missed_rescue": 19}, {"seed": 2029, "budget": 0.82, "threshold": -0.9948301155311453, "operator": "ge", "alpha": 1.0, "epoch": 8, "metric": 0.9093382984818104, "actual_fraction": 0.8199656258951591, "rescue": 1114, "harm": 338, "missed_rescue": 40}, {"seed": 2030, "budget": 0.86, "threshold": -0.9742640089127141, "operator": "ge", "alpha": 1.0, "epoch": 1, "metric": 0.9112002291606989, "actual_fraction": 0.8599255227728445, "rescue": 1121, "harm": 332, "missed_rescue": 33}], "rescued": 1119.4, "harmed": 332.6, "missed_rescue": 34.6, "avoided_harm": 75.39999999999998}, "validation_curve": [{"budget": 0.0, "metric": 0.7981953594958465, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.8030650243483242, "metric_std": 0.00045291859927929136, "actual_fraction": 0.009882555141793183, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.8078201088513319, "metric_std": 0.00027549962849715635, "actual_fraction": 0.01990833572042395, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.8117158407333142, "metric_std": 0.0007619270168015102, "actual_fraction": 0.029934116299054715, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.8159553136637067, "metric_std": 0.0005262438576589548, "actual_fraction": 0.039959896877685475, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.8197364651962188, "metric_std": 0.0006913502369605734, "actual_fraction": 0.04998567745631624, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.8234603265539959, "metric_std": 0.0005223312096480034, "actual_fraction": 0.05986823259810943, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.8268690919507306, "metric_std": 0.0010973349360858154, "actual_fraction": 0.06989401317674018, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.8301059868232599, "metric_std": 0.0007469724898541644, "actual_fraction": 0.07991979375537095, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.8331137209968491, "metric_std": 0.0005852998207304509, "actual_fraction": 0.08994557433400172, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.8356344886851904, "metric_std": 0.0003734862449271264, "actual_fraction": 0.09997135491263248, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.8386422228587798, "metric_std": 0.0007060302230802777, "actual_fraction": 0.10999713549126325, "actual_fraction_std": 0.0}, {"budget": 0.12, "metric": 0.8412202807218561, "metric_std": 0.0008139943992198583, "actual_fraction": 0.11987969063305641, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.13, "metric": 0.8439129189344026, "metric_std": 0.0006594594347305902, "actual_fraction": 0.1299054712116872, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.8466342022343168, "metric_std": 0.0004224543455671916, "actual_fraction": 0.13993125179031796, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.8496705814952735, "metric_std": 0.0005957214077080155, "actual_fraction": 0.14995703236894872, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.8518762532225723, "metric_std": 0.0007935774951128567, "actual_fraction": 0.1599828129475795, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.8541965052993412, "metric_std": 0.0014745961244610585, "actual_fraction": 0.16986536808937266, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.8564308221140073, "metric_std": 0.0015545074249459286, "actual_fraction": 0.17989114866800343, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.8588656545402464, "metric_std": 0.001084170201533979, "actual_fraction": 0.18991692924663417, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.8611572615296476, "metric_std": 0.0012428469292418284, "actual_fraction": 0.19994270982526496, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.8631624176453737, "metric_std": 0.0008863791719716313, "actual_fraction": 0.20996849040389573, "actual_fraction_std": 0.0}, {"budget": 0.22, "metric": 0.8649384130621598, "metric_std": 0.0008165106166565389, "actual_fraction": 0.2199942709825265, "actual_fraction_std": 0.0}, {"budget": 0.23, "metric": 0.8668003437410483, "metric_std": 0.0009170902660316023, "actual_fraction": 0.22987682612431967, "actual_fraction_std": 0.0}, {"budget": 0.24, "metric": 0.8687195645946719, "metric_std": 0.0005416118705582703, "actual_fraction": 0.23990260670295044, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.8708393010598682, "metric_std": 0.0009304143093045583, "actual_fraction": 0.24992838728158123, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.26, "metric": 0.8727298768261242, "metric_std": 0.0008202704705400134, "actual_fraction": 0.25995416786021197, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.8743912918934402, "metric_std": 0.0006160358970955153, "actual_fraction": 0.26997994843884277, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.8756803208249785, "metric_std": 0.0009337156123178, "actual_fraction": 0.2798625035806359, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.8778573474649096, "metric_std": 0.0006748335141443383, "actual_fraction": 0.2898882841592667, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.8792323116585503, "metric_std": 0.0008252569630987206, "actual_fraction": 0.29991406473789745, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.8802348897164135, "metric_std": 0.0009735116280702912, "actual_fraction": 0.30993984531652824, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.8815239186479518, "metric_std": 0.0005491346179513601, "actual_fraction": 0.319965625895159, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.882526496705815, "metric_std": 0.0007189859867092641, "actual_fraction": 0.3299914064737898, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.8836436551131479, "metric_std": 0.000826498885491423, "actual_fraction": 0.3398739616155829, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.884531652821541, "metric_std": 0.0009682293773219983, "actual_fraction": 0.3498997421942137, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.8854482956173015, "metric_std": 0.0006516363887755815, "actual_fraction": 0.35992552277284445, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.8864795187625323, "metric_std": 0.00048145907516386135, "actual_fraction": 0.3699513033514752, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.8880263534803781, "metric_std": 0.0005301276288548339, "actual_fraction": 0.379977083930106, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.888857061014036, "metric_std": 0.0005729017473503212, "actual_fraction": 0.3898596390718992, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.8900028645087368, "metric_std": 0.00028645087367515365, "actual_fraction": 0.3998854196505299, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.8913491836150099, "metric_std": 0.0007551662175000917, "actual_fraction": 0.40991120022916067, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.42, "metric": 0.892294471498138, "metric_std": 0.0010860606479593156, "actual_fraction": 0.41993698080779146, "actual_fraction_std": 0.0}, {"budget": 0.43, "metric": 0.8936407906044114, "metric_std": 0.0013797301150148815, "actual_fraction": 0.42996276138642225, "actual_fraction_std": 0.0}, {"budget": 0.44, "metric": 0.894786594099112, "metric_std": 0.0009532833739498283, "actual_fraction": 0.439988541965053, "actual_fraction_std": 0.0}, {"budget": 0.45, "metric": 0.8956745918075049, "metric_std": 0.0007483443413126336, "actual_fraction": 0.4498710971068462, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.8965339444285305, "metric_std": 0.0006610129337848152, "actual_fraction": 0.459896877685477, "actual_fraction_std": 0.0}, {"budget": 0.47, "metric": 0.8970495560011458, "metric_std": 0.0006972593208042485, "actual_fraction": 0.46992265826410773, "actual_fraction_std": 0.0}, {"budget": 0.48, "metric": 0.8979089086221712, "metric_std": 0.0014585104116987514, "actual_fraction": 0.4799484388427385, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.49, "metric": 0.8985677456316242, "metric_std": 0.0014443772254917144, "actual_fraction": 0.4899742194213693, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.5, "metric": 0.899341162990547, "metric_std": 0.001426513846804812, "actual_fraction": 0.5, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.9002291606989401, "metric_std": 0.0012008750396021533, "actual_fraction": 0.5098825551417931, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.9007734173589229, "metric_std": 0.0009080989678686974, "actual_fraction": 0.5199083357204239, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.9011744485820682, "metric_std": 0.0010178079635886693, "actual_fraction": 0.5299341162990547, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.9018046405041537, "metric_std": 0.00071325692340239, "actual_fraction": 0.5399598968776855, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.9024061873388712, "metric_std": 0.0008388169844942069, "actual_fraction": 0.5499856774563162, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.9028931538241192, "metric_std": 0.0007779132405614789, "actual_fraction": 0.5598682325981095, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.9030077341735894, "metric_std": 0.0006025690050977797, "actual_fraction": 0.5698940131767402, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.9032082497851619, "metric_std": 0.0006671907590290463, "actual_fraction": 0.579919793755371, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.9038384417072471, "metric_std": 0.0006292146272268245, "actual_fraction": 0.5899455743340017, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.9040103122314523, "metric_std": 0.0005491346179513065, "actual_fraction": 0.5999713549126325, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.904554568891435, "metric_std": 0.0004706868153868856, "actual_fraction": 0.6099971354912632, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.9054139215124606, "metric_std": 0.00041262848154937184, "actual_fraction": 0.6198796906330565, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.9057863076482382, "metric_std": 0.0005104133394788549, "actual_fraction": 0.6299054712116872, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.9058722429103409, "metric_std": 0.0005675053782916114, "actual_fraction": 0.639931251790318, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.9061586937840161, "metric_std": 0.0006516363887755693, "actual_fraction": 0.6499570323689486, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.9063019192208536, "metric_std": 0.0005939971742574164, "actual_fraction": 0.6599828129475795, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.9065024348324261, "metric_std": 0.0007118174528667299, "actual_fraction": 0.6698653680893727, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.9067315955313664, "metric_std": 0.0009024339158993248, "actual_fraction": 0.6798911486680035, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.9070753365797766, "metric_std": 0.0010446624192959283, "actual_fraction": 0.6899169292466341, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.9076768834144943, "metric_std": 0.0008326807134774255, "actual_fraction": 0.699942709825265, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.907877399026067, "metric_std": 0.0008852212626962448, "actual_fraction": 0.7099684904038958, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.9081924949871099, "metric_std": 0.0006160358970955476, "actual_fraction": 0.7199942709825266, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.9085075909481523, "metric_std": 0.0007331127981707052, "actual_fraction": 0.7298768261243197, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.9088513319965624, "metric_std": 0.0011965968819367472, "actual_fraction": 0.7399026067029504, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.9091377828702377, "metric_std": 0.0009315160436269695, "actual_fraction": 0.7499283872815812, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.9095101690060154, "metric_std": 0.0013220258675697074, "actual_fraction": 0.759954167860212, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.9096533944428531, "metric_std": 0.0014372585166773975, "actual_fraction": 0.7699799484388428, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.9099112002291607, "metric_std": 0.001136816661872493, "actual_fraction": 0.7798625035806359, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.9099971354912633, "metric_std": 0.0009798127399878015, "actual_fraction": 0.7898882841592667, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.9100830707533657, "metric_std": 0.0008557644942016954, "actual_fraction": 0.7999140647378975, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.9100544256659981, "metric_std": 0.0008712063205812408, "actual_fraction": 0.8099398453165282, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.9099398453165282, "metric_std": 0.0009024339158993697, "actual_fraction": 0.8199656258951592, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.83, "metric": 0.9098825551417932, "metric_std": 0.001039741702256189, "actual_fraction": 0.8299914064737898, "actual_fraction_std": 0.0}, {"budget": 0.84, "metric": 0.9100544256659981, "metric_std": 0.0010426969191178472, "actual_fraction": 0.8398739616155829, "actual_fraction_std": 0.0}, {"budget": 0.85, "metric": 0.9106273274133485, "metric_std": 0.0009661083903775274, "actual_fraction": 0.8498997421942137, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.9106273274133485, "metric_std": 0.0009446366340785411, "actual_fraction": 0.8599255227728445, "actual_fraction_std": 0.0}, {"budget": 0.87, "metric": 0.9105700372386135, "metric_std": 0.0008921461760182371, "actual_fraction": 0.8699513033514752, "actual_fraction_std": 0.0}, {"budget": 0.88, "metric": 0.9103122314523059, "metric_std": 0.0011784590335561763, "actual_fraction": 0.879977083930106, "actual_fraction_std": 0.0}, {"budget": 0.89, "metric": 0.9101976511028358, "metric_std": 0.001324351332634348, "actual_fraction": 0.8898596390718991, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.909882555141793, "metric_std": 0.0012420213915026468, "actual_fraction": 0.8998854196505299, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.9094242337439129, "metric_std": 0.0012551648410315858, "actual_fraction": 0.9099112002291608, "actual_fraction_std": 0.0}, {"budget": 0.92, "metric": 0.9091377828702377, "metric_std": 0.00117497246072722, "actual_fraction": 0.9199369808077915, "actual_fraction_std": 0.0}, {"budget": 0.93, "metric": 0.9087940418218274, "metric_std": 0.001106642438962032, "actual_fraction": 0.9299627613864223, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.94, "metric": 0.9084503007734174, "metric_std": 0.0012705961109261913, "actual_fraction": 0.9399885419650531, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.9082211400744772, "metric_std": 0.0011914428154159217, "actual_fraction": 0.9498710971068463, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.9075336579776568, "metric_std": 0.0007818587260931975, "actual_fraction": 0.959896877685477, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.97, "metric": 0.907046691492409, "metric_std": 0.0007016584768785971, "actual_fraction": 0.9699226582641078, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.98, "metric": 0.9065024348324261, "metric_std": 0.0007118174528667299, "actual_fraction": 0.9799484388427386, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.99, "metric": 0.9061873388713835, "metric_std": 0.0005359005137171772, "actual_fraction": 0.9899742194213692, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.9050415353766829, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.9060055966133314, "actual_fraction": 0.8434096290449882, "metric_std": 0.0012438068324144567, "actual_fraction_std": 0.03732074421572776, "per_seed": [{"seed": 2026, "budget": 0.85, "threshold": -0.9734646185051644, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.9062926024251992, "actual_fraction": 0.8557078280835186, "rescue": 2174, "harm": 693, "missed_rescue": 73}, {"seed": 2027, "budget": 0.79, "threshold": -0.9487407074812001, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.9067231111430005, "actual_fraction": 0.7902705029776853, "rescue": 2136, "harm": 649, "missed_rescue": 111}, {"seed": 2028, "budget": 0.89, "threshold": -0.9906119153114747, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.905503336442563, "actual_fraction": 0.8894310109779723, "rescue": 2191, "harm": 721, "missed_rescue": 56}, {"seed": 2029, "budget": 0.82, "threshold": -0.9948301155311453, "operator": "ge", "alpha": 1.0, "epoch": 8, "metric": 0.9041400588361914, "actual_fraction": 0.8255722178374111, "rescue": 2155, "harm": 704, "missed_rescue": 92}, {"seed": 2030, "budget": 0.86, "threshold": -0.9742640089127141, "operator": "ge", "alpha": 1.0, "epoch": 1, "metric": 0.9073688742197031, "actual_fraction": 0.8560665853483533, "rescue": 2161, "harm": 665, "missed_rescue": 86}], "rescued": 2163.4, "harmed": 686.4, "missed_rescue": 83.6, "avoided_harm": 158.60000000000002}, "test_curve": [{"budget": 0.0, "metric": 0.8000287005811867, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.8058836191432877, "metric_std": 0.0009183064794877472, "actual_fraction": 0.012312549329123915, "actual_fraction_std": 0.001044224017348146}, {"budget": 0.02, "metric": 0.810375260099017, "metric_std": 0.0006242788759174815, "actual_fraction": 0.02211379780440554, "actual_fraction_std": 0.0006242788759175202}, {"budget": 0.03, "metric": 0.8147807993111862, "metric_std": 0.0006837126641067988, "actual_fraction": 0.031699791920786397, "actual_fraction_std": 0.0007306679841056737}, {"budget": 0.04, "metric": 0.8182966205065654, "metric_std": 0.0008079608673463904, "actual_fraction": 0.040410418310970794, "actual_fraction_std": 0.00139075713759671}, {"budget": 0.05, "metric": 0.8222286001291528, "metric_std": 0.0007004500171839504, "actual_fraction": 0.05077132811939441, "actual_fraction_std": 0.0005088791914040971}, {"budget": 0.06, "metric": 0.8254287149314774, "metric_std": 0.001026070865040858, "actual_fraction": 0.06002726555212743, "actual_fraction_std": 0.0012413208735686937}, {"budget": 0.07, "metric": 0.8284853268278682, "metric_std": 0.0012130028246540223, "actual_fraction": 0.07024467245461721, "actual_fraction_std": 0.0021084430300516753}, {"budget": 0.08, "metric": 0.832316854416302, "metric_std": 0.0011691304252565505, "actual_fraction": 0.08107914185262252, "actual_fraction_std": 0.0015388972224673306}, {"budget": 0.09, "metric": 0.8355313195092201, "metric_std": 0.0012667738833718, "actual_fraction": 0.09049293248188275, "actual_fraction_std": 0.001933565205075898}, {"budget": 0.1, "metric": 0.8391332424481595, "metric_std": 0.0019406082135751679, "actual_fraction": 0.10200186553777713, "actual_fraction_std": 0.0019323666728593787}, {"budget": 0.11, "metric": 0.8421898543445504, "metric_std": 0.0014622164502869324, "actual_fraction": 0.11214752098729999, "actual_fraction_std": 0.002550458863051761}, {"budget": 0.12, "metric": 0.8449164095572936, "metric_std": 0.00150899975631775, "actual_fraction": 0.12286718806055823, "actual_fraction_std": 0.0024535430062767247}, {"budget": 0.13, "metric": 0.847470761282916, "metric_std": 0.0013532331638409275, "actual_fraction": 0.13245318217693908, "actual_fraction_std": 0.0028767769529639813}, {"budget": 0.14, "metric": 0.8494654516753964, "metric_std": 0.0010557464999326426, "actual_fraction": 0.1406758986869484, "actual_fraction_std": 0.0021409123461389604}, {"budget": 0.15, "metric": 0.8519050010762719, "metric_std": 0.0009693087526803884, "actual_fraction": 0.1509794073329985, "actual_fraction_std": 0.003259917330870963}, {"budget": 0.16, "metric": 0.8545741551266414, "metric_std": 0.0014245871083683034, "actual_fraction": 0.16080935638946686, "actual_fraction_std": 0.004242568217508688}, {"budget": 0.17, "metric": 0.8569563033651431, "metric_std": 0.0008422777120372951, "actual_fraction": 0.1716438257874722, "actual_fraction_std": 0.0032436894809081502}, {"budget": 0.18, "metric": 0.8591375475353376, "metric_std": 0.0005450277395374459, "actual_fraction": 0.18141637368156707, "actual_fraction_std": 0.003812156789389181}, {"budget": 0.19, "metric": 0.8612183396713784, "metric_std": 0.0006525044152623798, "actual_fraction": 0.19147592738752958, "actual_fraction_std": 0.0027939798355828376}, {"budget": 0.2, "metric": 0.863442634713353, "metric_std": 0.0009067408004261602, "actual_fraction": 0.2025974025974026, "actual_fraction_std": 0.0018776410686714385}, {"budget": 0.21, "metric": 0.8654803759776135, "metric_std": 0.0006544739471043028, "actual_fraction": 0.2129726626964196, "actual_fraction_std": 0.0020486264963470438}, {"budget": 0.22, "metric": 0.8672024108488197, "metric_std": 0.001219141434997197, "actual_fraction": 0.2230896175647557, "actual_fraction_std": 0.0027380489831942975}, {"budget": 0.23, "metric": 0.8690248977541796, "metric_std": 0.001274068227059752, "actual_fraction": 0.2323599052880821, "actual_fraction_std": 0.00218022902855291}, {"budget": 0.24, "metric": 0.8707325823347922, "metric_std": 0.0014038369349381875, "actual_fraction": 0.2416158427208151, "actual_fraction_std": 0.0013946385857569645}, {"budget": 0.25, "metric": 0.8723685154624382, "metric_std": 0.001122400068781625, "actual_fraction": 0.25059912463227374, "actual_fraction_std": 0.002304216110796497}, {"budget": 0.26, "metric": 0.8740331491712707, "metric_std": 0.0011741835352989192, "actual_fraction": 0.26019946903924807, "actual_fraction_std": 0.0026179941149883142}, {"budget": 0.27, "metric": 0.8758412857860371, "metric_std": 0.0008984703073831852, "actual_fraction": 0.2698141637368156, "actual_fraction_std": 0.0020774492970229794}, {"budget": 0.28, "metric": 0.877893377340891, "metric_std": 0.001011415566554696, "actual_fraction": 0.2793858075626032, "actual_fraction_std": 0.003083392946250911}, {"budget": 0.29, "metric": 0.8795580110497238, "metric_std": 0.0008589221354176864, "actual_fraction": 0.2897323670804334, "actual_fraction_std": 0.002919411352523344}, {"budget": 0.3, "metric": 0.8810647915620292, "metric_std": 0.0006079850804137216, "actual_fraction": 0.29974886991461575, "actual_fraction_std": 0.0032106670691673437}, {"budget": 0.31, "metric": 0.8824711200401808, "metric_std": 0.0011963357968381582, "actual_fraction": 0.3085025471765803, "actual_fraction_std": 0.0014918436763073839}, {"budget": 0.32, "metric": 0.883934849680706, "metric_std": 0.0009027576085269384, "actual_fraction": 0.31853340030135613, "actual_fraction_std": 0.0026667033461292323}, {"budget": 0.33, "metric": 0.8851115735093635, "metric_std": 0.0008953132385629201, "actual_fraction": 0.32733012843510084, "actual_fraction_std": 0.0034589856621063037}, {"budget": 0.34, "metric": 0.8865035516969219, "metric_std": 0.00044923515369156406, "actual_fraction": 0.3374614335940303, "actual_fraction_std": 0.00447668119624916}, {"budget": 0.35, "metric": 0.8881681854057545, "metric_std": 0.0005311546394008433, "actual_fraction": 0.3478223434024539, "actual_fraction_std": 0.0037945598349230737}, {"budget": 0.36, "metric": 0.8896175647556863, "metric_std": 0.0006780417395380592, "actual_fraction": 0.3571643825787472, "actual_fraction_std": 0.003684700841147648}, {"budget": 0.37, "metric": 0.8910669441056183, "metric_std": 0.0007341825209033413, "actual_fraction": 0.3673243883188635, "actual_fraction_std": 0.004006726028596458}, {"budget": 0.38, "metric": 0.8917988089258808, "metric_std": 0.0006477531063093878, "actual_fraction": 0.3771543373753318, "actual_fraction_std": 0.0035328392819720202}, {"budget": 0.39, "metric": 0.8929324818827581, "metric_std": 0.0008139916941029958, "actual_fraction": 0.3864246250986582, "actual_fraction_std": 0.004103530771929138}, {"budget": 0.4, "metric": 0.8943962115232834, "metric_std": 0.000694545169854628, "actual_fraction": 0.39745999856497094, "actual_fraction_std": 0.0042662257882614485}, {"budget": 0.41, "metric": 0.8954868336083806, "metric_std": 0.0011191846742146586, "actual_fraction": 0.40793571069814166, "actual_fraction_std": 0.0036436741432497857}, {"budget": 0.42, "metric": 0.896477003659324, "metric_std": 0.001043977476436403, "actual_fraction": 0.41849752457487266, "actual_fraction_std": 0.0037074052817904417}, {"budget": 0.43, "metric": 0.897036664992466, "metric_std": 0.0008149398501543045, "actual_fraction": 0.4278539140417593, "actual_fraction_std": 0.005305242461250971}, {"budget": 0.44, "metric": 0.8979837841716296, "metric_std": 0.0009291743900799315, "actual_fraction": 0.43677979479084456, "actual_fraction_std": 0.005286431640043159}, {"budget": 0.45, "metric": 0.8988735021884194, "metric_std": 0.001040519770068069, "actual_fraction": 0.4473416086675755, "actual_fraction_std": 0.004895321426028515}, {"budget": 0.46, "metric": 0.8997775704958025, "metric_std": 0.000622213773083942, "actual_fraction": 0.45771686876659257, "actual_fraction_std": 0.0037246537921555778}, {"budget": 0.47, "metric": 0.9004233335725049, "metric_std": 0.0008012424065519946, "actual_fraction": 0.46840783525866403, "actual_fraction_std": 0.0037978824190232027}, {"budget": 0.48, "metric": 0.9011695486833607, "metric_std": 0.0006509244991055393, "actual_fraction": 0.4781229819903853, "actual_fraction_std": 0.003982237960007365}, {"budget": 0.49, "metric": 0.9019301140848104, "metric_std": 0.0008108231663351545, "actual_fraction": 0.48842649063643534, "actual_fraction_std": 0.003741203599844267}, {"budget": 0.5, "metric": 0.9026476286144796, "metric_std": 0.0007583283398501295, "actual_fraction": 0.49786898184688244, "actual_fraction_std": 0.0028925715261433707}, {"budget": 0.51, "metric": 0.903006385879314, "metric_std": 0.0005497303956196059, "actual_fraction": 0.5086891009542944, "actual_fraction_std": 0.002544193611323946}, {"budget": 0.52, "metric": 0.9039104541866972, "metric_std": 0.000607561544952728, "actual_fraction": 0.5189065078567842, "actual_fraction_std": 0.0029149993636753643}, {"budget": 0.53, "metric": 0.9043266126139053, "metric_std": 0.0009335964228642779, "actual_fraction": 0.5290952141780871, "actual_fraction_std": 0.002971411242139429}, {"budget": 0.54, "metric": 0.90468536987874, "metric_std": 0.0008854838641874004, "actual_fraction": 0.5393987228241371, "actual_fraction_std": 0.00286134502915243}, {"budget": 0.55, "metric": 0.9049006242376407, "metric_std": 0.0008392159872118325, "actual_fraction": 0.5496591805984071, "actual_fraction_std": 0.0028676354846589704}, {"budget": 0.56, "metric": 0.9050441271435746, "metric_std": 0.000937723139998845, "actual_fraction": 0.5601922938939513, "actual_fraction_std": 0.002224642199333085}, {"budget": 0.57, "metric": 0.9051732797589152, "metric_std": 0.0009898060273837214, "actual_fraction": 0.569921790916266, "actual_fraction_std": 0.00316398687814442}, {"budget": 0.58, "metric": 0.9052306809212887, "metric_std": 0.000992143848314187, "actual_fraction": 0.5801248475281625, "actual_fraction_std": 0.0032674102250314636}, {"budget": 0.59, "metric": 0.9055463873143431, "metric_std": 0.0009280655896967843, "actual_fraction": 0.5894955872856426, "actual_fraction_std": 0.0036336993052599486}, {"budget": 0.6, "metric": 0.9056468393484968, "metric_std": 0.0008860650810955314, "actual_fraction": 0.6000430508717801, "actual_fraction_std": 0.0031445642170580316}, {"budget": 0.61, "metric": 0.905804692545024, "metric_std": 0.0010024682038644183, "actual_fraction": 0.6112793284064002, "actual_fraction_std": 0.0028518833304196608}, {"budget": 0.62, "metric": 0.905876443997991, "metric_std": 0.0010177583828081607, "actual_fraction": 0.6210805768816818, "actual_fraction_std": 0.005001801969132752}, {"budget": 0.63, "metric": 0.9060342971945182, "metric_std": 0.0010491426692364335, "actual_fraction": 0.6310827294252709, "actual_fraction_std": 0.005019935946286672}, {"budget": 0.64, "metric": 0.9061921503910455, "metric_std": 0.0012720462162202643, "actual_fraction": 0.6421898543445504, "actual_fraction_std": 0.005059215249911949}, {"budget": 0.65, "metric": 0.9061203989380786, "metric_std": 0.001365730055442052, "actual_fraction": 0.6522637583411064, "actual_fraction_std": 0.006370260318533224}, {"budget": 0.66, "metric": 0.9061921503910455, "metric_std": 0.001319719697613531, "actual_fraction": 0.6629834254143646, "actual_fraction_std": 0.006242376408122206}, {"budget": 0.67, "metric": 0.906321303006386, "metric_std": 0.0015200473762635122, "actual_fraction": 0.6722393628470976, "actual_fraction_std": 0.006440067639324633}, {"budget": 0.68, "metric": 0.9064504556217263, "metric_std": 0.0016607012904898395, "actual_fraction": 0.6822271651000932, "actual_fraction_std": 0.007102741527837655}, {"budget": 0.69, "metric": 0.9064074047499462, "metric_std": 0.001507292935551763, "actual_fraction": 0.6932625385664059, "actual_fraction_std": 0.007007439534189425}, {"budget": 0.7, "metric": 0.9064074047499462, "metric_std": 0.001546073163414328, "actual_fraction": 0.7043266126139054, "actual_fraction_std": 0.006740131525680831}, {"budget": 0.71, "metric": 0.9066513596900337, "metric_std": 0.0017894760584544942, "actual_fraction": 0.7137691038243525, "actual_fraction_std": 0.006494081272525432}, {"budget": 0.72, "metric": 0.9065939585276601, "metric_std": 0.0016645718529825816, "actual_fraction": 0.723943459855062, "actual_fraction_std": 0.00688544827960958}, {"budget": 0.73, "metric": 0.9067948625959676, "metric_std": 0.0015963690508206633, "actual_fraction": 0.7330415440912679, "actual_fraction_std": 0.006277452567501143}, {"budget": 0.74, "metric": 0.9067518117241875, "metric_std": 0.001762371100542691, "actual_fraction": 0.7429288943101099, "actual_fraction_std": 0.007696225538675239}, {"budget": 0.75, "metric": 0.9065222070746932, "metric_std": 0.0018281842180315675, "actual_fraction": 0.7524000861017435, "actual_fraction_std": 0.006980092570289338}, {"budget": 0.76, "metric": 0.9067518117241875, "metric_std": 0.001741802543914808, "actual_fraction": 0.7627896964913539, "actual_fraction_std": 0.006076674564928275}, {"budget": 0.77, "metric": 0.9065365573652867, "metric_std": 0.001964077254704228, "actual_fraction": 0.7734089115304584, "actual_fraction_std": 0.0056685918413993215}, {"budget": 0.78, "metric": 0.9066226591088469, "metric_std": 0.001941669090086632, "actual_fraction": 0.7830810073903998, "actual_fraction_std": 0.005111101321900722}, {"budget": 0.79, "metric": 0.9064791562029132, "metric_std": 0.0017985156855306504, "actual_fraction": 0.7919638372677047, "actual_fraction_std": 0.0049411178468930185}, {"budget": 0.8, "metric": 0.9065078567840998, "metric_std": 0.0017648523895025907, "actual_fraction": 0.8021525435890077, "actual_fraction_std": 0.003232241690547464}, {"budget": 0.81, "metric": 0.9065939585276602, "metric_std": 0.0017335067880951867, "actual_fraction": 0.8125565042692113, "actual_fraction_std": 0.003825301383086226}, {"budget": 0.82, "metric": 0.9066083088182536, "metric_std": 0.0015741228775666644, "actual_fraction": 0.8229461146588217, "actual_fraction_std": 0.002906863807783438}, {"budget": 0.83, "metric": 0.9065509076558801, "metric_std": 0.0018346497163417171, "actual_fraction": 0.8328334648776637, "actual_fraction_std": 0.00414155701400372}, {"budget": 0.84, "metric": 0.9064504556217263, "metric_std": 0.0017110878759810052, "actual_fraction": 0.8424051087034512, "actual_fraction_std": 0.0037216116776212184}, {"budget": 0.85, "metric": 0.9059625457415512, "metric_std": 0.0015912007036225347, "actual_fraction": 0.851991102819832, "actual_fraction_std": 0.003022261142469728}, {"budget": 0.86, "metric": 0.9056468393484968, "metric_std": 0.0015288279940204573, "actual_fraction": 0.8613331419961255, "actual_fraction_std": 0.0033606958990786426}, {"budget": 0.87, "metric": 0.9055607376049364, "metric_std": 0.001764414768537161, "actual_fraction": 0.8714500968644614, "actual_fraction_std": 0.003902909714978781}, {"budget": 0.88, "metric": 0.9054028844084094, "metric_std": 0.0020525180051936282, "actual_fraction": 0.8802898758699864, "actual_fraction_std": 0.0037498630095143346}, {"budget": 0.89, "metric": 0.9051732797589152, "metric_std": 0.0018590412379027746, "actual_fraction": 0.8893592595250054, "actual_fraction_std": 0.002794624681660476}, {"budget": 0.9, "metric": 0.9050441271435747, "metric_std": 0.001524275140384002, "actual_fraction": 0.8990170050943531, "actual_fraction_std": 0.0012567774439727925}, {"budget": 0.91, "metric": 0.9048288727846738, "metric_std": 0.0015326956836263584, "actual_fraction": 0.9093492143215899, "actual_fraction_std": 0.0012298626023428405}, {"budget": 0.92, "metric": 0.9047284207505202, "metric_std": 0.0014344908225028333, "actual_fraction": 0.9188347564038171, "actual_fraction_std": 0.0014073165439239604}, {"budget": 0.93, "metric": 0.904455765229246, "metric_std": 0.0013000682175333762, "actual_fraction": 0.9295687737676689, "actual_fraction_std": 0.0020737287114502203}, {"budget": 0.94, "metric": 0.9040683073832245, "metric_std": 0.0012551378119061165, "actual_fraction": 0.940360192293894, "actual_fraction_std": 0.0022320353602647267}, {"budget": 0.95, "metric": 0.9038674033149172, "metric_std": 0.0010190222084320495, "actual_fraction": 0.9496591805984071, "actual_fraction_std": 0.0019875281986673414}, {"budget": 0.96, "metric": 0.9035086460500825, "metric_std": 0.0007734527073666856, "actual_fraction": 0.9592882255865682, "actual_fraction_std": 0.001483191221496449}, {"budget": 0.97, "metric": 0.9031642390758414, "metric_std": 0.0007310201981459439, "actual_fraction": 0.9692760278395637, "actual_fraction_std": 0.0010402723512629762}, {"budget": 0.98, "metric": 0.9024467245461721, "metric_std": 0.0005951479185858236, "actual_fraction": 0.9798808925880749, "actual_fraction_std": 0.00202664516037845}, {"budget": 0.99, "metric": 0.9015713568199756, "metric_std": 0.000525796683951016, "actual_fraction": 0.9904570567553993, "actual_fraction_std": 0.0013423467700272894}, {"budget": 1.0, "metric": 0.9006242376408121, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 5165, "rescuable": 1154, "harmful": 408, "both_wrong": 255, "unit": "sample", "total": 6982}, "test": {"both_correct": 10305, "rescuable": 2247, "harmful": 845, "both_wrong": 540, "unit": "sample", "total": 13937}}, "small": {"validation": 0.7981953594958464, "test": 0.8000287005811868}, "large": {"validation": 0.9050415353766829, "test": 0.9006242376408122}, "learned_minus_confidence_test": 0.0035875726483460824, "learned_minus_small_test": 0.10597689603214455, "learned_minus_large_test": 0.0053813589725191235, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_nlvr2_Qwen3.8"}, {"key": ["nlvr2", "vilt", "MiniCPM"], "expert_display": "ViLT", "run": "outputs/full_budget_20260913/runs/full_vilt_MiniCPM_s{2026..2030}", "validation_rows": 6982, "test_rows": 13937, "confidence_rule": {"column": "max_probability", "columns": ["entropy", "max_probability", "top_two_margin", "entropy_valid", "max_probability_valid", "top_two_margin_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/vilt/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 96, "validation_area_0_25": 0.7926095674591808, "validation_point": {"budget": 0.96, "threshold": -0.9998401403427124, "operator": "ge", "actual_fraction": 0.9598968776854769, "metric": 0.8997421942136924, "rescued": 1385, "harmed": 379, "missed_rescue": 4, "avoided_harm": 9, "selected_both_correct": 4621, "selected_both_wrong": 317}, "test_point": {"budget": 0.96, "threshold": -0.9998401403427124, "operator": "ge", "actual_fraction": 0.9593169261677549, "metric": 0.906436105331133, "rescued": 2682, "harmed": 700, "missed_rescue": 24, "avoided_harm": 11, "selected_both_correct": 9413, "selected_both_wrong": 575}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.7556574047550845, "rescued": 0, "harmed": 0, "missed_rescue": 1389, "avoided_harm": 388, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.5127503275871277, "operator": "ge", "actual_fraction": 0.009882555141793183, "metric": 0.7590948152391864, "rescued": 31, "harmed": 7, "missed_rescue": 1358, "avoided_harm": 381, "selected_both_correct": 27, "selected_both_wrong": 4}, {"budget": 0.02, "threshold": -0.5296345353126526, "operator": "ge", "actual_fraction": 0.01990833572042395, "metric": 0.7619593239759381, "rescued": 55, "harmed": 11, "missed_rescue": 1334, "avoided_harm": 377, "selected_both_correct": 64, "selected_both_wrong": 9}, {"budget": 0.03, "threshold": -0.5434877276420593, "operator": "ge", "actual_fraction": 0.029934116299054712, "metric": 0.7639644800916643, "rescued": 80, "harmed": 22, "missed_rescue": 1309, "avoided_harm": 366, "selected_both_correct": 93, "selected_both_wrong": 14}, {"budget": 0.04, "threshold": -0.56125408411026, "operator": "ge", "actual_fraction": 0.039959896877685475, "metric": 0.7672586651389287, "rescued": 109, "harmed": 28, "missed_rescue": 1280, "avoided_harm": 360, "selected_both_correct": 124, "selected_both_wrong": 18}, {"budget": 0.05, "threshold": -0.5779195427894592, "operator": "ge", "actual_fraction": 0.04998567745631624, "metric": 0.771268977370381, "rescued": 137, "harmed": 28, "missed_rescue": 1252, "avoided_harm": 360, "selected_both_correct": 158, "selected_both_wrong": 26}, {"budget": 0.06, "threshold": -0.5911363363265991, "operator": "ge", "actual_fraction": 0.059868232598109423, "metric": 0.7734173589229447, "rescued": 160, "harmed": 36, "missed_rescue": 1229, "avoided_harm": 352, "selected_both_correct": 189, "selected_both_wrong": 33}, {"budget": 0.07, "threshold": -0.6087464094161987, "operator": "ge", "actual_fraction": 0.06989401317674018, "metric": 0.7769979948438843, "rescued": 192, "harmed": 43, "missed_rescue": 1197, "avoided_harm": 345, "selected_both_correct": 215, "selected_both_wrong": 38}, {"budget": 0.08, "threshold": -0.6274068355560303, "operator": "ge", "actual_fraction": 0.07991979375537095, "metric": 0.7805786307648238, "rescued": 223, "harmed": 49, "missed_rescue": 1166, "avoided_harm": 339, "selected_both_correct": 245, "selected_both_wrong": 41}, {"budget": 0.09, "threshold": -0.6465112566947937, "operator": "ge", "actual_fraction": 0.08994557433400172, "metric": 0.783586364938413, "rescued": 249, "harmed": 54, "missed_rescue": 1140, "avoided_harm": 334, "selected_both_correct": 279, "selected_both_wrong": 46}, {"budget": 0.1, "threshold": -0.6620025038719177, "operator": "ge", "actual_fraction": 0.09997135491263248, "metric": 0.7877399026067029, "rescued": 281, "harmed": 57, "missed_rescue": 1108, "avoided_harm": 331, "selected_both_correct": 307, "selected_both_wrong": 53}, {"budget": 0.11, "threshold": -0.6800539493560791, "operator": "ge", "actual_fraction": 0.10999713549126325, "metric": 0.7901747350329419, "rescued": 303, "harmed": 62, "missed_rescue": 1086, "avoided_harm": 326, "selected_both_correct": 345, "selected_both_wrong": 58}, {"budget": 0.12, "threshold": -0.6957284212112427, "operator": "ge", "actual_fraction": 0.11987969063305644, "metric": 0.7926095674591808, "rescued": 329, "harmed": 71, "missed_rescue": 1060, "avoided_harm": 317, "selected_both_correct": 372, "selected_both_wrong": 65}, {"budget": 0.13, "threshold": -0.7128273248672485, "operator": "ge", "actual_fraction": 0.1299054712116872, "metric": 0.7954740761959324, "rescued": 355, "harmed": 77, "missed_rescue": 1034, "avoided_harm": 311, "selected_both_correct": 405, "selected_both_wrong": 70}, {"budget": 0.14, "threshold": -0.7332167625427246, "operator": "ge", "actual_fraction": 0.13993125179031796, "metric": 0.7977656831853337, "rescued": 379, "harmed": 85, "missed_rescue": 1010, "avoided_harm": 303, "selected_both_correct": 439, "selected_both_wrong": 74}, {"budget": 0.15, "threshold": -0.7503995895385742, "operator": "ge", "actual_fraction": 0.14995703236894872, "metric": 0.8010598682325981, "rescued": 408, "harmed": 91, "missed_rescue": 981, "avoided_harm": 297, "selected_both_correct": 468, "selected_both_wrong": 80}, {"budget": 0.16, "threshold": -0.7655540704727173, "operator": "ge", "actual_fraction": 0.1599828129475795, "metric": 0.8026353480378116, "rescued": 428, "harmed": 100, "missed_rescue": 961, "avoided_harm": 288, "selected_both_correct": 507, "selected_both_wrong": 82}, {"budget": 0.17, "threshold": -0.7812193036079407, "operator": "ge", "actual_fraction": 0.16986536808937266, "metric": 0.8059295330850759, "rescued": 453, "harmed": 102, "missed_rescue": 936, "avoided_harm": 286, "selected_both_correct": 543, "selected_both_wrong": 88}, {"budget": 0.18, "threshold": -0.7954917550086975, "operator": "ge", "actual_fraction": 0.17989114866800343, "metric": 0.806932111142939, "rescued": 470, "harmed": 112, "missed_rescue": 919, "avoided_harm": 276, "selected_both_correct": 584, "selected_both_wrong": 90}, {"budget": 0.19, "threshold": -0.8097342252731323, "operator": "ge", "actual_fraction": 0.1899169292466342, "metric": 0.8093669435691779, "rescued": 494, "harmed": 119, "missed_rescue": 895, "avoided_harm": 269, "selected_both_correct": 619, "selected_both_wrong": 94}, {"budget": 0.2, "threshold": -0.8232117891311646, "operator": "ge", "actual_fraction": 0.19994270982526496, "metric": 0.8122314523059295, "rescued": 517, "harmed": 122, "missed_rescue": 872, "avoided_harm": 266, "selected_both_correct": 658, "selected_both_wrong": 99}, {"budget": 0.21, "threshold": -0.8359466791152954, "operator": "ge", "actual_fraction": 0.20996849040389573, "metric": 0.8146662847321684, "rescued": 539, "harmed": 127, "missed_rescue": 850, "avoided_harm": 261, "selected_both_correct": 697, "selected_both_wrong": 103}, {"budget": 0.22, "threshold": -0.8473842144012451, "operator": "ge", "actual_fraction": 0.2199942709825265, "metric": 0.8168146662847322, "rescued": 560, "harmed": 133, "missed_rescue": 829, "avoided_harm": 255, "selected_both_correct": 734, "selected_both_wrong": 109}, {"budget": 0.23, "threshold": -0.8595863580703735, "operator": "ge", "actual_fraction": 0.22987682612431967, "metric": 0.8203953022056717, "rescued": 588, "harmed": 136, "missed_rescue": 801, "avoided_harm": 252, "selected_both_correct": 767, "selected_both_wrong": 114}, {"budget": 0.24, "threshold": -0.8704345226287842, "operator": "ge", "actual_fraction": 0.23990260670295044, "metric": 0.8225436837582355, "rescued": 610, "harmed": 143, "missed_rescue": 779, "avoided_harm": 245, "selected_both_correct": 805, "selected_both_wrong": 117}, {"budget": 0.25, "threshold": -0.8811548352241516, "operator": "ge", "actual_fraction": 0.2499283872815812, "metric": 0.8258378688054998, "rescued": 637, "harmed": 147, "missed_rescue": 752, "avoided_harm": 241, "selected_both_correct": 836, "selected_both_wrong": 125}, {"budget": 0.26, "threshold": -0.8929013609886169, "operator": "ge", "actual_fraction": 0.25995416786021197, "metric": 0.8279862503580636, "rescued": 656, "harmed": 151, "missed_rescue": 733, "avoided_harm": 237, "selected_both_correct": 880, "selected_both_wrong": 128}, {"budget": 0.27, "threshold": -0.903549313545227, "operator": "ge", "actual_fraction": 0.26997994843884277, "metric": 0.829561730163277, "rescued": 675, "harmed": 159, "missed_rescue": 714, "avoided_harm": 229, "selected_both_correct": 915, "selected_both_wrong": 136}, {"budget": 0.28, "threshold": -0.9115740656852722, "operator": "ge", "actual_fraction": 0.2798625035806359, "metric": 0.8321397880263535, "rescued": 697, "harmed": 163, "missed_rescue": 692, "avoided_harm": 225, "selected_both_correct": 953, "selected_both_wrong": 141}, {"budget": 0.29, "threshold": -0.9182595014572144, "operator": "ge", "actual_fraction": 0.2898882841592667, "metric": 0.83471784588943, "rescued": 718, "harmed": 166, "missed_rescue": 671, "avoided_harm": 222, "selected_both_correct": 997, "selected_both_wrong": 143}, {"budget": 0.3, "threshold": -0.9258768558502197, "operator": "ge", "actual_fraction": 0.29991406473789745, "metric": 0.8361501002578058, "rescued": 735, "harmed": 173, "missed_rescue": 654, "avoided_harm": 215, "selected_both_correct": 1041, "selected_both_wrong": 145}, {"budget": 0.31, "threshold": -0.9331877827644348, "operator": "ge", "actual_fraction": 0.30993984531652824, "metric": 0.8381552563735319, "rescued": 754, "harmed": 178, "missed_rescue": 635, "avoided_harm": 210, "selected_both_correct": 1080, "selected_both_wrong": 152}, {"budget": 0.32, "threshold": -0.9399217367172241, "operator": "ge", "actual_fraction": 0.319965625895159, "metric": 0.8405900887997708, "rescued": 774, "harmed": 181, "missed_rescue": 615, "avoided_harm": 207, "selected_both_correct": 1123, "selected_both_wrong": 156}, {"budget": 0.33, "threshold": -0.9453923106193542, "operator": "ge", "actual_fraction": 0.3299914064737898, "metric": 0.8423087940418218, "rescued": 792, "harmed": 187, "missed_rescue": 597, "avoided_harm": 201, "selected_both_correct": 1162, "selected_both_wrong": 163}, {"budget": 0.34, "threshold": -0.9515207409858704, "operator": "ge", "actual_fraction": 0.3398739616155829, "metric": 0.8438842738470352, "rescued": 807, "harmed": 191, "missed_rescue": 582, "avoided_harm": 197, "selected_both_correct": 1206, "selected_both_wrong": 169}, {"budget": 0.35, "threshold": -0.9567410349845886, "operator": "ge", "actual_fraction": 0.3498997421942137, "metric": 0.8466055571469493, "rescued": 829, "harmed": 194, "missed_rescue": 560, "avoided_harm": 194, "selected_both_correct": 1245, "selected_both_wrong": 175}, {"budget": 0.36, "threshold": -0.9606627225875854, "operator": "ge", "actual_fraction": 0.35992552277284445, "metric": 0.8498997421942137, "rescued": 854, "harmed": 196, "missed_rescue": 535, "avoided_harm": 192, "selected_both_correct": 1287, "selected_both_wrong": 176}, {"budget": 0.37, "threshold": -0.9650874733924866, "operator": "ge", "actual_fraction": 0.36995130335147525, "metric": 0.8519048983099399, "rescued": 872, "harmed": 200, "missed_rescue": 517, "avoided_harm": 188, "selected_both_correct": 1329, "selected_both_wrong": 182}, {"budget": 0.38, "threshold": -0.9686882495880127, "operator": "ge", "actual_fraction": 0.379977083930106, "metric": 0.8547694070466915, "rescued": 893, "harmed": 201, "missed_rescue": 496, "avoided_harm": 187, "selected_both_correct": 1372, "selected_both_wrong": 187}, {"budget": 0.39, "threshold": -0.971785306930542, "operator": "ge", "actual_fraction": 0.3898596390718992, "metric": 0.8557719851045545, "rescued": 903, "harmed": 204, "missed_rescue": 486, "avoided_harm": 184, "selected_both_correct": 1423, "selected_both_wrong": 192}, {"budget": 0.4, "threshold": -0.9748144745826721, "operator": "ge", "actual_fraction": 0.3998854196505299, "metric": 0.8584932684044686, "rescued": 923, "harmed": 205, "missed_rescue": 466, "avoided_harm": 183, "selected_both_correct": 1468, "selected_both_wrong": 196}, {"budget": 0.41, "threshold": -0.977255642414093, "operator": "ge", "actual_fraction": 0.4099112002291607, "metric": 0.8596390718991693, "rescued": 935, "harmed": 209, "missed_rescue": 454, "avoided_harm": 179, "selected_both_correct": 1520, "selected_both_wrong": 198}, {"budget": 0.42, "threshold": -0.9794421792030334, "operator": "ge", "actual_fraction": 0.41993698080779146, "metric": 0.861787453451733, "rescued": 954, "harmed": 213, "missed_rescue": 435, "avoided_harm": 175, "selected_both_correct": 1560, "selected_both_wrong": 205}, {"budget": 0.43, "threshold": -0.981472373008728, "operator": "ge", "actual_fraction": 0.42996276138642225, "metric": 0.8632197078201088, "rescued": 969, "harmed": 218, "missed_rescue": 420, "avoided_harm": 170, "selected_both_correct": 1606, "selected_both_wrong": 209}, {"budget": 0.44, "threshold": -0.9835578799247742, "operator": "ge", "actual_fraction": 0.439988541965053, "metric": 0.8649384130621598, "rescued": 986, "harmed": 223, "missed_rescue": 403, "avoided_harm": 165, "selected_both_correct": 1653, "selected_both_wrong": 210}, {"budget": 0.45, "threshold": -0.9851564168930054, "operator": "ge", "actual_fraction": 0.4498710971068462, "metric": 0.8663706674305357, "rescued": 1001, "harmed": 228, "missed_rescue": 388, "avoided_harm": 160, "selected_both_correct": 1699, "selected_both_wrong": 213}, {"budget": 0.46, "threshold": -0.986689031124115, "operator": "ge", "actual_fraction": 0.45989687768547693, "metric": 0.8673732454883988, "rescued": 1016, "harmed": 236, "missed_rescue": 373, "avoided_harm": 152, "selected_both_correct": 1742, "selected_both_wrong": 217}, {"budget": 0.47, "threshold": -0.9879001379013062, "operator": "ge", "actual_fraction": 0.46992265826410773, "metric": 0.8680893726725867, "rescued": 1024, "harmed": 239, "missed_rescue": 365, "avoided_harm": 149, "selected_both_correct": 1797, "selected_both_wrong": 221}, {"budget": 0.48, "threshold": -0.9891784191131592, "operator": "ge", "actual_fraction": 0.47994843884273847, "metric": 0.8700945287883128, "rescued": 1043, "harmed": 244, "missed_rescue": 346, "avoided_harm": 144, "selected_both_correct": 1839, "selected_both_wrong": 225}, {"budget": 0.49, "threshold": -0.990114688873291, "operator": "ge", "actual_fraction": 0.48997421942136926, "metric": 0.8715267831566886, "rescued": 1057, "harmed": 248, "missed_rescue": 332, "avoided_harm": 140, "selected_both_correct": 1889, "selected_both_wrong": 227}, {"budget": 0.5, "threshold": -0.9910277724266052, "operator": "ge", "actual_fraction": 0.5, "metric": 0.8722429103408765, "rescued": 1066, "harmed": 252, "missed_rescue": 323, "avoided_harm": 136, "selected_both_correct": 1942, "selected_both_wrong": 231}, {"budget": 0.51, "threshold": -0.9918718934059143, "operator": "ge", "actual_fraction": 0.5098825551417931, "metric": 0.8725293612145517, "rescued": 1074, "harmed": 258, "missed_rescue": 315, "avoided_harm": 130, "selected_both_correct": 1996, "selected_both_wrong": 232}, {"budget": 0.52, "threshold": -0.9926809668540955, "operator": "ge", "actual_fraction": 0.5199083357204239, "metric": 0.8729590375250644, "rescued": 1083, "harmed": 264, "missed_rescue": 306, "avoided_harm": 124, "selected_both_correct": 2050, "selected_both_wrong": 233}, {"budget": 0.53, "threshold": -0.9932646751403809, "operator": "ge", "actual_fraction": 0.5299341162990547, "metric": 0.87381839014609, "rescued": 1090, "harmed": 265, "missed_rescue": 299, "avoided_harm": 123, "selected_both_correct": 2108, "selected_both_wrong": 237}, {"budget": 0.54, "threshold": -0.9938417673110962, "operator": "ge", "actual_fraction": 0.5399598968776855, "metric": 0.8743912918934402, "rescued": 1097, "harmed": 268, "missed_rescue": 292, "avoided_harm": 120, "selected_both_correct": 2166, "selected_both_wrong": 239}, {"budget": 0.55, "threshold": -0.9944412112236023, "operator": "ge", "actual_fraction": 0.5499856774563162, "metric": 0.8751074190776282, "rescued": 1107, "harmed": 273, "missed_rescue": 282, "avoided_harm": 115, "selected_both_correct": 2220, "selected_both_wrong": 240}, {"budget": 0.56, "threshold": -0.9949513673782349, "operator": "ge", "actual_fraction": 0.5598682325981095, "metric": 0.8761099971354913, "rescued": 1118, "harmed": 277, "missed_rescue": 271, "avoided_harm": 111, "selected_both_correct": 2272, "selected_both_wrong": 242}, {"budget": 0.57, "threshold": -0.9954326152801514, "operator": "ge", "actual_fraction": 0.5698940131767402, "metric": 0.8776854769407046, "rescued": 1131, "harmed": 279, "missed_rescue": 258, "avoided_harm": 109, "selected_both_correct": 2323, "selected_both_wrong": 246}, {"budget": 0.58, "threshold": -0.9958608746528625, "operator": "ge", "actual_fraction": 0.579919793755371, "metric": 0.878258378688055, "rescued": 1142, "harmed": 286, "missed_rescue": 247, "avoided_harm": 102, "selected_both_correct": 2371, "selected_both_wrong": 250}, {"budget": 0.59, "threshold": -0.9962128400802612, "operator": "ge", "actual_fraction": 0.5899455743340017, "metric": 0.8792609567459181, "rescued": 1154, "harmed": 291, "missed_rescue": 235, "avoided_harm": 97, "selected_both_correct": 2421, "selected_both_wrong": 253}, {"budget": 0.6, "threshold": -0.9965764880180359, "operator": "ge", "actual_fraction": 0.5999713549126325, "metric": 0.8796906330564308, "rescued": 1161, "harmed": 295, "missed_rescue": 228, "avoided_harm": 93, "selected_both_correct": 2479, "selected_both_wrong": 254}, {"budget": 0.61, "threshold": -0.9968799352645874, "operator": "ge", "actual_fraction": 0.6099971354912632, "metric": 0.8808364365511315, "rescued": 1174, "harmed": 300, "missed_rescue": 215, "avoided_harm": 88, "selected_both_correct": 2530, "selected_both_wrong": 255}, {"budget": 0.62, "threshold": -0.9971383810043335, "operator": "ge", "actual_fraction": 0.6198796906330565, "metric": 0.8811228874248066, "rescued": 1178, "harmed": 302, "missed_rescue": 211, "avoided_harm": 86, "selected_both_correct": 2591, "selected_both_wrong": 257}, {"budget": 0.63, "threshold": -0.997377872467041, "operator": "ge", "actual_fraction": 0.6299054712116872, "metric": 0.8822686909195073, "rescued": 1188, "harmed": 304, "missed_rescue": 201, "avoided_harm": 84, "selected_both_correct": 2645, "selected_both_wrong": 261}, {"budget": 0.64, "threshold": -0.9976021647453308, "operator": "ge", "actual_fraction": 0.639931251790318, "metric": 0.8832712689773704, "rescued": 1200, "harmed": 309, "missed_rescue": 189, "avoided_harm": 79, "selected_both_correct": 2695, "selected_both_wrong": 264}, {"budget": 0.65, "threshold": -0.9978214502334595, "operator": "ge", "actual_fraction": 0.6499570323689488, "metric": 0.8834144944142079, "rescued": 1205, "harmed": 313, "missed_rescue": 184, "avoided_harm": 75, "selected_both_correct": 2753, "selected_both_wrong": 267}, {"budget": 0.66, "threshold": -0.9980514049530029, "operator": "ge", "actual_fraction": 0.6599828129475795, "metric": 0.8847035233457462, "rescued": 1215, "harmed": 314, "missed_rescue": 174, "avoided_harm": 74, "selected_both_correct": 2810, "selected_both_wrong": 269}, {"budget": 0.67, "threshold": -0.9982016086578369, "operator": "ge", "actual_fraction": 0.6698653680893727, "metric": 0.8855628759667717, "rescued": 1223, "harmed": 316, "missed_rescue": 166, "avoided_harm": 72, "selected_both_correct": 2867, "selected_both_wrong": 271}, {"budget": 0.68, "threshold": -0.9983693957328796, "operator": "ge", "actual_fraction": 0.6798911486680035, "metric": 0.8862790031509596, "rescued": 1231, "harmed": 319, "missed_rescue": 158, "avoided_harm": 69, "selected_both_correct": 2925, "selected_both_wrong": 272}, {"budget": 0.69, "threshold": -0.9984915256500244, "operator": "ge", "actual_fraction": 0.6899169292466342, "metric": 0.887854482956173, "rescued": 1243, "harmed": 320, "missed_rescue": 146, "avoided_harm": 68, "selected_both_correct": 2980, "selected_both_wrong": 274}, {"budget": 0.7, "threshold": -0.9986109733581543, "operator": "gt", "actual_fraction": 0.6997994843884274, "metric": 0.8881409338298482, "rescued": 1248, "harmed": 323, "missed_rescue": 141, "avoided_harm": 65, "selected_both_correct": 3039, "selected_both_wrong": 276}, {"budget": 0.71, "threshold": -0.9986966252326965, "operator": "ge", "actual_fraction": 0.7099684904038958, "metric": 0.888857061014036, "rescued": 1253, "harmed": 323, "missed_rescue": 136, "avoided_harm": 65, "selected_both_correct": 3105, "selected_both_wrong": 276}, {"budget": 0.72, "threshold": -0.9987963438034058, "operator": "ge", "actual_fraction": 0.7199942709825266, "metric": 0.8890002864508737, "rescued": 1258, "harmed": 327, "missed_rescue": 131, "avoided_harm": 61, "selected_both_correct": 3164, "selected_both_wrong": 278}, {"budget": 0.73, "threshold": -0.998875081539154, "operator": "ge", "actual_fraction": 0.7298768261243197, "metric": 0.8897164136350616, "rescued": 1264, "harmed": 328, "missed_rescue": 125, "avoided_harm": 60, "selected_both_correct": 3223, "selected_both_wrong": 281}, {"budget": 0.74, "threshold": -0.9989551305770874, "operator": "ge", "actual_fraction": 0.7399026067029505, "metric": 0.8901460899455743, "rescued": 1272, "harmed": 333, "missed_rescue": 117, "avoided_harm": 55, "selected_both_correct": 3277, "selected_both_wrong": 284}, {"budget": 0.75, "threshold": -0.9990480542182922, "operator": "ge", "actual_fraction": 0.7499283872815812, "metric": 0.8911486680034374, "rescued": 1280, "harmed": 334, "missed_rescue": 109, "avoided_harm": 54, "selected_both_correct": 3335, "selected_both_wrong": 287}, {"budget": 0.76, "threshold": -0.999126136302948, "operator": "ge", "actual_fraction": 0.759954167860212, "metric": 0.8915783443139501, "rescued": 1286, "harmed": 337, "missed_rescue": 103, "avoided_harm": 51, "selected_both_correct": 3393, "selected_both_wrong": 290}, {"budget": 0.77, "threshold": -0.999184787273407, "operator": "gt", "actual_fraction": 0.7698367230020051, "metric": 0.8920080206244629, "rescued": 1291, "harmed": 339, "missed_rescue": 98, "avoided_harm": 49, "selected_both_correct": 3454, "selected_both_wrong": 291}, {"budget": 0.78, "threshold": -0.9992268085479736, "operator": "ge", "actual_fraction": 0.7798625035806359, "metric": 0.893010598682326, "rescued": 1301, "harmed": 342, "missed_rescue": 88, "avoided_harm": 46, "selected_both_correct": 3510, "selected_both_wrong": 292}, {"budget": 0.79, "threshold": -0.9992831349372864, "operator": "ge", "actual_fraction": 0.7898882841592667, "metric": 0.8941564021770266, "rescued": 1312, "harmed": 345, "missed_rescue": 77, "avoided_harm": 43, "selected_both_correct": 3564, "selected_both_wrong": 294}, {"budget": 0.8, "threshold": -0.9993318915367126, "operator": "ge", "actual_fraction": 0.7999140647378975, "metric": 0.8942996276138642, "rescued": 1314, "harmed": 346, "missed_rescue": 75, "avoided_harm": 42, "selected_both_correct": 3631, "selected_both_wrong": 294}, {"budget": 0.81, "threshold": -0.9993773102760315, "operator": "ge", "actual_fraction": 0.8099398453165282, "metric": 0.8944428530507018, "rescued": 1318, "harmed": 349, "missed_rescue": 71, "avoided_harm": 39, "selected_both_correct": 3693, "selected_both_wrong": 295}, {"budget": 0.82, "threshold": -0.9994175434112549, "operator": "ge", "actual_fraction": 0.819965625895159, "metric": 0.8945860784875393, "rescued": 1323, "harmed": 353, "missed_rescue": 66, "avoided_harm": 35, "selected_both_correct": 3752, "selected_both_wrong": 297}, {"budget": 0.83, "threshold": -0.999452531337738, "operator": "ge", "actual_fraction": 0.8299914064737898, "metric": 0.894729303924377, "rescued": 1327, "harmed": 356, "missed_rescue": 62, "avoided_harm": 32, "selected_both_correct": 3815, "selected_both_wrong": 297}, {"budget": 0.84, "threshold": -0.9994945526123047, "operator": "ge", "actual_fraction": 0.8398739616155829, "metric": 0.8951589802348897, "rescued": 1332, "harmed": 358, "missed_rescue": 57, "avoided_harm": 30, "selected_both_correct": 3876, "selected_both_wrong": 298}, {"budget": 0.85, "threshold": -0.9995274543762207, "operator": "gt", "actual_fraction": 0.8497565167573761, "metric": 0.8954454311085649, "rescued": 1335, "harmed": 359, "missed_rescue": 54, "avoided_harm": 29, "selected_both_correct": 3938, "selected_both_wrong": 301}, {"budget": 0.86, "threshold": -0.9995617270469666, "operator": "ge", "actual_fraction": 0.8599255227728445, "metric": 0.8958751074190776, "rescued": 1340, "harmed": 361, "missed_rescue": 49, "avoided_harm": 27, "selected_both_correct": 3999, "selected_both_wrong": 304}, {"budget": 0.87, "threshold": -0.9995949864387512, "operator": "ge", "actual_fraction": 0.8699513033514752, "metric": 0.896448009166428, "rescued": 1347, "harmed": 364, "missed_rescue": 42, "avoided_harm": 24, "selected_both_correct": 4059, "selected_both_wrong": 304}, {"budget": 0.88, "threshold": -0.9996248483657837, "operator": "ge", "actual_fraction": 0.879977083930106, "metric": 0.8968776854769407, "rescued": 1351, "harmed": 365, "missed_rescue": 38, "avoided_harm": 23, "selected_both_correct": 4123, "selected_both_wrong": 305}, {"budget": 0.89, "threshold": -0.9996511936187744, "operator": "ge", "actual_fraction": 0.8898596390718991, "metric": 0.897450587224291, "rescued": 1358, "harmed": 368, "missed_rescue": 31, "avoided_harm": 20, "selected_both_correct": 4180, "selected_both_wrong": 307}, {"budget": 0.9, "threshold": -0.9996830224990845, "operator": "ge", "actual_fraction": 0.8998854196505299, "metric": 0.8977370380979662, "rescued": 1361, "harmed": 369, "missed_rescue": 28, "avoided_harm": 19, "selected_both_correct": 4244, "selected_both_wrong": 309}, {"budget": 0.91, "threshold": -0.9997097849845886, "operator": "gt", "actual_fraction": 0.9097679747923231, "metric": 0.8984531652821541, "rescued": 1369, "harmed": 372, "missed_rescue": 20, "avoided_harm": 16, "selected_both_correct": 4302, "selected_both_wrong": 309}, {"budget": 0.92, "threshold": -0.9997391104698181, "operator": "ge", "actual_fraction": 0.9199369808077915, "metric": 0.8985963907189917, "rescued": 1372, "harmed": 374, "missed_rescue": 17, "avoided_harm": 14, "selected_both_correct": 4367, "selected_both_wrong": 310}, {"budget": 0.93, "threshold": -0.9997641444206238, "operator": "ge", "actual_fraction": 0.9299627613864222, "metric": 0.8987396161558293, "rescued": 1374, "harmed": 375, "missed_rescue": 15, "avoided_harm": 13, "selected_both_correct": 4429, "selected_both_wrong": 315}, {"budget": 0.94, "threshold": -0.9997876286506653, "operator": "ge", "actual_fraction": 0.939988541965053, "metric": 0.8990260670295045, "rescued": 1377, "harmed": 376, "missed_rescue": 12, "avoided_harm": 12, "selected_both_correct": 4494, "selected_both_wrong": 316}, {"budget": 0.95, "threshold": -0.9998131394386292, "operator": "ge", "actual_fraction": 0.9498710971068461, "metric": 0.899169292466342, "rescued": 1380, "harmed": 378, "missed_rescue": 9, "avoided_harm": 10, "selected_both_correct": 4557, "selected_both_wrong": 317}, {"budget": 0.96, "threshold": -0.9998401403427124, "operator": "ge", "actual_fraction": 0.9598968776854769, "metric": 0.8997421942136924, "rescued": 1385, "harmed": 379, "missed_rescue": 4, "avoided_harm": 9, "selected_both_correct": 4621, "selected_both_wrong": 317}, {"budget": 0.97, "threshold": -0.999868631362915, "operator": "ge", "actual_fraction": 0.9699226582641077, "metric": 0.8993125179031796, "rescued": 1385, "harmed": 382, "missed_rescue": 4, "avoided_harm": 6, "selected_both_correct": 4688, "selected_both_wrong": 317}, {"budget": 0.98, "threshold": -0.9998925924301147, "operator": "ge", "actual_fraction": 0.9799484388427385, "metric": 0.899169292466342, "rescued": 1385, "harmed": 383, "missed_rescue": 4, "avoided_harm": 5, "selected_both_correct": 4757, "selected_both_wrong": 317}, {"budget": 0.99, "threshold": -0.99992835521698, "operator": "ge", "actual_fraction": 0.9899742194213692, "metric": 0.8987396161558293, "rescued": 1385, "harmed": 386, "missed_rescue": 4, "avoided_harm": 2, "selected_both_correct": 4824, "selected_both_wrong": 317}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.8990260670295045, "rescued": 1389, "harmed": 388, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 4888, "selected_both_wrong": 317}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.7642247255506924, "rescued": 0, "harmed": 0, "missed_rescue": 2706, "avoided_harm": 711, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.5127503275871277, "operator": "ge", "actual_fraction": 0.007749156920427639, "metric": 0.7658032575159647, "rescued": 33, "harmed": 11, "missed_rescue": 2673, "avoided_harm": 700, "selected_both_correct": 57, "selected_both_wrong": 7}, {"budget": 0.02, "threshold": -0.5296345353126526, "operator": "ge", "actual_fraction": 0.017650857429862953, "metric": 0.7694625816172778, "rescued": 96, "harmed": 23, "missed_rescue": 2610, "avoided_harm": 688, "selected_both_correct": 117, "selected_both_wrong": 10}, {"budget": 0.03, "threshold": -0.5434877276420593, "operator": "ge", "actual_fraction": 0.025113008538422903, "metric": 0.7724043911889216, "rescued": 142, "harmed": 28, "missed_rescue": 2564, "avoided_harm": 683, "selected_both_correct": 160, "selected_both_wrong": 20}, {"budget": 0.04, "threshold": -0.56125408411026, "operator": "ge", "actual_fraction": 0.0349429575948913, "metric": 0.7759919638372677, "rescued": 199, "harmed": 35, "missed_rescue": 2507, "avoided_harm": 676, "selected_both_correct": 224, "selected_both_wrong": 29}, {"budget": 0.05, "threshold": -0.5779195427894592, "operator": "ge", "actual_fraction": 0.043840137762789694, "metric": 0.7782880103322092, "rescued": 238, "harmed": 42, "missed_rescue": 2468, "avoided_harm": 669, "selected_both_correct": 291, "selected_both_wrong": 40}, {"budget": 0.06, "threshold": -0.5911363363265991, "operator": "ge", "actual_fraction": 0.05187630049508503, "metric": 0.7806558082801177, "rescued": 281, "harmed": 52, "missed_rescue": 2425, "avoided_harm": 659, "selected_both_correct": 346, "selected_both_wrong": 44}, {"budget": 0.07, "threshold": -0.6087464094161987, "operator": "ge", "actual_fraction": 0.06177800100452034, "metric": 0.7828801033220922, "rescued": 326, "harmed": 66, "missed_rescue": 2380, "avoided_harm": 645, "selected_both_correct": 409, "selected_both_wrong": 60}, {"budget": 0.08, "threshold": -0.6274068355560303, "operator": "ge", "actual_fraction": 0.07132094424912104, "metric": 0.7859654157996699, "rescued": 376, "harmed": 73, "missed_rescue": 2330, "avoided_harm": 638, "selected_both_correct": 477, "selected_both_wrong": 68}, {"budget": 0.09, "threshold": -0.6465112566947937, "operator": "ge", "actual_fraction": 0.08251417091196096, "metric": 0.788691971012413, "rescued": 423, "harmed": 82, "missed_rescue": 2283, "avoided_harm": 629, "selected_both_correct": 560, "selected_both_wrong": 85}, {"budget": 0.1, "threshold": -0.6620025038719177, "operator": "ge", "actual_fraction": 0.09090909090909091, "metric": 0.7908445146014207, "rescued": 462, "harmed": 91, "missed_rescue": 2244, "avoided_harm": 620, "selected_both_correct": 619, "selected_both_wrong": 95}, {"budget": 0.11, "threshold": -0.6800539493560791, "operator": "ge", "actual_fraction": 0.10030853124775777, "metric": 0.7934275669082299, "rescued": 508, "harmed": 101, "missed_rescue": 2198, "avoided_harm": 610, "selected_both_correct": 682, "selected_both_wrong": 107}, {"budget": 0.12, "threshold": -0.6957284212112427, "operator": "ge", "actual_fraction": 0.11078424338092846, "metric": 0.7968716366506422, "rescued": 564, "harmed": 109, "missed_rescue": 2142, "avoided_harm": 602, "selected_both_correct": 747, "selected_both_wrong": 124}, {"budget": 0.13, "threshold": -0.7128273248672485, "operator": "ge", "actual_fraction": 0.12075769534333071, "metric": 0.7998851976752529, "rescued": 613, "harmed": 116, "missed_rescue": 2093, "avoided_harm": 595, "selected_both_correct": 821, "selected_both_wrong": 133}, {"budget": 0.14, "threshold": -0.7332167625427246, "operator": "ge", "actual_fraction": 0.13431871995407907, "metric": 0.8045490421181029, "rescued": 690, "harmed": 128, "missed_rescue": 2016, "avoided_harm": 583, "selected_both_correct": 902, "selected_both_wrong": 152}, {"budget": 0.15, "threshold": -0.7503995895385742, "operator": "ge", "actual_fraction": 0.144579177728349, "metric": 0.8069168400660114, "rescued": 738, "harmed": 143, "missed_rescue": 1968, "avoided_harm": 568, "selected_both_correct": 974, "selected_both_wrong": 160}, {"budget": 0.16, "threshold": -0.7655540704727173, "operator": "ge", "actual_fraction": 0.1534763578962474, "metric": 0.8095716438257875, "rescued": 788, "harmed": 156, "missed_rescue": 1918, "avoided_harm": 555, "selected_both_correct": 1026, "selected_both_wrong": 169}, {"budget": 0.17, "threshold": -0.7812193036079407, "operator": "ge", "actual_fraction": 0.16280404678194735, "metric": 0.8120111932266628, "rescued": 836, "harmed": 170, "missed_rescue": 1870, "avoided_harm": 541, "selected_both_correct": 1083, "selected_both_wrong": 180}, {"budget": 0.18, "threshold": -0.7954917550086975, "operator": "ge", "actual_fraction": 0.17306450455621727, "metric": 0.8153117600631413, "rescued": 891, "harmed": 179, "missed_rescue": 1815, "avoided_harm": 532, "selected_both_correct": 1151, "selected_both_wrong": 191}, {"budget": 0.19, "threshold": -0.8097342252731323, "operator": "ge", "actual_fraction": 0.18196168472411567, "metric": 0.8172490492932482, "rescued": 928, "harmed": 189, "missed_rescue": 1778, "avoided_harm": 522, "selected_both_correct": 1219, "selected_both_wrong": 200}, {"budget": 0.2, "threshold": -0.8232117891311646, "operator": "ge", "actual_fraction": 0.1928679055750879, "metric": 0.8200473559589582, "rescued": 976, "harmed": 198, "missed_rescue": 1730, "avoided_harm": 513, "selected_both_correct": 1308, "selected_both_wrong": 206}, {"budget": 0.21, "threshold": -0.8359466791152954, "operator": "ge", "actual_fraction": 0.20104757121331707, "metric": 0.8223434024538997, "rescued": 1015, "harmed": 205, "missed_rescue": 1691, "avoided_harm": 506, "selected_both_correct": 1369, "selected_both_wrong": 213}, {"budget": 0.22, "threshold": -0.8473842144012451, "operator": "ge", "actual_fraction": 0.21051876300495084, "metric": 0.8249264547607089, "rescued": 1060, "harmed": 214, "missed_rescue": 1646, "avoided_harm": 497, "selected_both_correct": 1434, "selected_both_wrong": 226}, {"budget": 0.23, "threshold": -0.8595863580703735, "operator": "ge", "actual_fraction": 0.21998995479658462, "metric": 0.8267919925378489, "rescued": 1097, "harmed": 225, "missed_rescue": 1609, "avoided_harm": 486, "selected_both_correct": 1510, "selected_both_wrong": 234}, {"budget": 0.24, "threshold": -0.8704345226287842, "operator": "ge", "actual_fraction": 0.22838487479371458, "metric": 0.8283705245031212, "rescued": 1128, "harmed": 234, "missed_rescue": 1578, "avoided_harm": 477, "selected_both_correct": 1582, "selected_both_wrong": 239}, {"budget": 0.25, "threshold": -0.8811548352241516, "operator": "ge", "actual_fraction": 0.23900408983281912, "metric": 0.8309535768099304, "rescued": 1174, "harmed": 244, "missed_rescue": 1532, "avoided_harm": 467, "selected_both_correct": 1665, "selected_both_wrong": 248}, {"budget": 0.26, "threshold": -0.8929013609886169, "operator": "ge", "actual_fraction": 0.2506995766664275, "metric": 0.8342541436464088, "rescued": 1227, "harmed": 251, "missed_rescue": 1479, "avoided_harm": 460, "selected_both_correct": 1757, "selected_both_wrong": 259}, {"budget": 0.27, "threshold": -0.903549313545227, "operator": "ge", "actual_fraction": 0.2646911099949774, "metric": 0.8369806988591519, "rescued": 1280, "harmed": 266, "missed_rescue": 1426, "avoided_harm": 445, "selected_both_correct": 1872, "selected_both_wrong": 271}, {"budget": 0.28, "threshold": -0.9115740656852722, "operator": "ge", "actual_fraction": 0.27437755614551196, "metric": 0.8399225084307957, "rescued": 1326, "harmed": 271, "missed_rescue": 1380, "avoided_harm": 440, "selected_both_correct": 1952, "selected_both_wrong": 275}, {"budget": 0.29, "threshold": -0.9182595014572144, "operator": "ge", "actual_fraction": 0.2819832101600057, "metric": 0.8419315491138696, "rescued": 1362, "harmed": 279, "missed_rescue": 1344, "avoided_harm": 432, "selected_both_correct": 2007, "selected_both_wrong": 282}, {"budget": 0.3, "threshold": -0.9258768558502197, "operator": "ge", "actual_fraction": 0.2907368874219703, "metric": 0.8444428499677118, "rescued": 1406, "harmed": 288, "missed_rescue": 1300, "avoided_harm": 423, "selected_both_correct": 2069, "selected_both_wrong": 289}, {"budget": 0.31, "threshold": -0.9331877827644348, "operator": "ge", "actual_fraction": 0.302145368443711, "metric": 0.8466671450096864, "rescued": 1448, "harmed": 299, "missed_rescue": 1258, "avoided_harm": 412, "selected_both_correct": 2169, "selected_both_wrong": 295}, {"budget": 0.32, "threshold": -0.9399217367172241, "operator": "ge", "actual_fraction": 0.31412786108918705, "metric": 0.8487479371457272, "rescued": 1492, "harmed": 314, "missed_rescue": 1214, "avoided_harm": 397, "selected_both_correct": 2272, "selected_both_wrong": 300}, {"budget": 0.33, "threshold": -0.9453923106193542, "operator": "ge", "actual_fraction": 0.3238143072397216, "metric": 0.8506852263758341, "rescued": 1527, "harmed": 322, "missed_rescue": 1179, "avoided_harm": 389, "selected_both_correct": 2358, "selected_both_wrong": 306}, {"budget": 0.34, "threshold": -0.9515207409858704, "operator": "ge", "actual_fraction": 0.33529453971442924, "metric": 0.8536987874004449, "rescued": 1578, "harmed": 331, "missed_rescue": 1128, "avoided_harm": 380, "selected_both_correct": 2453, "selected_both_wrong": 311}, {"budget": 0.35, "threshold": -0.9567410349845886, "operator": "ge", "actual_fraction": 0.3452679916768315, "metric": 0.8557078280835186, "rescued": 1615, "harmed": 340, "missed_rescue": 1091, "avoided_harm": 371, "selected_both_correct": 2540, "selected_both_wrong": 317}, {"budget": 0.36, "threshold": -0.9606627225875854, "operator": "ge", "actual_fraction": 0.35595895816890294, "metric": 0.8581473774843941, "rescued": 1655, "harmed": 346, "missed_rescue": 1051, "avoided_harm": 365, "selected_both_correct": 2634, "selected_both_wrong": 326}, {"budget": 0.37, "threshold": -0.9650874733924866, "operator": "ge", "actual_fraction": 0.36837195953218055, "metric": 0.8605151754323025, "rescued": 1702, "harmed": 360, "missed_rescue": 1004, "avoided_harm": 351, "selected_both_correct": 2738, "selected_both_wrong": 334}, {"budget": 0.38, "threshold": -0.9686882495880127, "operator": "ge", "actual_fraction": 0.3797086891009543, "metric": 0.8622372103035086, "rescued": 1741, "harmed": 375, "missed_rescue": 965, "avoided_harm": 336, "selected_both_correct": 2832, "selected_both_wrong": 344}, {"budget": 0.39, "threshold": -0.971785306930542, "operator": "ge", "actual_fraction": 0.391117170122695, "metric": 0.8652507713281193, "rescued": 1786, "harmed": 378, "missed_rescue": 920, "avoided_harm": 333, "selected_both_correct": 2936, "selected_both_wrong": 351}, {"budget": 0.4, "threshold": -0.9748144745826721, "operator": "ge", "actual_fraction": 0.40231039678553493, "metric": 0.8673315634641602, "rescued": 1820, "harmed": 383, "missed_rescue": 886, "avoided_harm": 328, "selected_both_correct": 3042, "selected_both_wrong": 362}, {"budget": 0.41, "threshold": -0.977255642414093, "operator": "ge", "actual_fraction": 0.41235560020090406, "metric": 0.8697711128650355, "rescued": 1863, "harmed": 392, "missed_rescue": 843, "avoided_harm": 319, "selected_both_correct": 3125, "selected_both_wrong": 367}, {"budget": 0.42, "threshold": -0.9794421792030334, "operator": "ge", "actual_fraction": 0.4231183181459425, "metric": 0.8717801535481093, "rescued": 1903, "harmed": 404, "missed_rescue": 803, "avoided_harm": 307, "selected_both_correct": 3218, "selected_both_wrong": 372}, {"budget": 0.43, "threshold": -0.981472373008728, "operator": "ge", "actual_fraction": 0.43172849250197315, "metric": 0.8732151826074478, "rescued": 1925, "harmed": 406, "missed_rescue": 781, "avoided_harm": 305, "selected_both_correct": 3309, "selected_both_wrong": 377}, {"budget": 0.44, "threshold": -0.9835578799247742, "operator": "ge", "actual_fraction": 0.4432804764296477, "metric": 0.8749372174786539, "rescued": 1956, "harmed": 413, "missed_rescue": 750, "avoided_harm": 298, "selected_both_correct": 3425, "selected_both_wrong": 384}, {"budget": 0.45, "threshold": -0.9851564168930054, "operator": "ge", "actual_fraction": 0.4547607089043553, "metric": 0.8768027552557939, "rescued": 1994, "harmed": 425, "missed_rescue": 712, "avoided_harm": 286, "selected_both_correct": 3527, "selected_both_wrong": 392}, {"budget": 0.46, "threshold": -0.986689031124115, "operator": "ge", "actual_fraction": 0.4649494152256583, "metric": 0.8787400444859008, "rescued": 2026, "harmed": 430, "missed_rescue": 680, "avoided_harm": 281, "selected_both_correct": 3626, "selected_both_wrong": 398}, {"budget": 0.47, "threshold": -0.9879001379013062, "operator": "ge", "actual_fraction": 0.47399009829949057, "metric": 0.8798880677333716, "rescued": 2047, "harmed": 435, "missed_rescue": 659, "avoided_harm": 276, "selected_both_correct": 3721, "selected_both_wrong": 403}, {"budget": 0.48, "threshold": -0.9891784191131592, "operator": "ge", "actual_fraction": 0.48446581043266124, "metric": 0.8814665996986439, "rescued": 2076, "harmed": 442, "missed_rescue": 630, "avoided_harm": 269, "selected_both_correct": 3824, "selected_both_wrong": 410}, {"budget": 0.49, "threshold": -0.990114688873291, "operator": "ge", "actual_fraction": 0.491210447011552, "metric": 0.8825428714931477, "rescued": 2095, "harmed": 446, "missed_rescue": 611, "avoided_harm": 265, "selected_both_correct": 3892, "selected_both_wrong": 413}, {"budget": 0.5, "threshold": -0.9910277724266052, "operator": "ge", "actual_fraction": 0.5008251417091196, "metric": 0.8833321374757839, "rescued": 2116, "harmed": 456, "missed_rescue": 590, "avoided_harm": 255, "selected_both_correct": 3984, "selected_both_wrong": 424}, {"budget": 0.51, "threshold": -0.9918718934059143, "operator": "ge", "actual_fraction": 0.5107268422185549, "metric": 0.8839061490995193, "rescued": 2133, "harmed": 465, "missed_rescue": 573, "avoided_harm": 246, "selected_both_correct": 4093, "selected_both_wrong": 427}, {"budget": 0.52, "threshold": -0.9926809668540955, "operator": "ge", "actual_fraction": 0.5217048145224941, "metric": 0.88505417234699, "rescued": 2158, "harmed": 474, "missed_rescue": 548, "avoided_harm": 237, "selected_both_correct": 4209, "selected_both_wrong": 430}, {"budget": 0.53, "threshold": -0.9932646751403809, "operator": "ge", "actual_fraction": 0.5306737461433594, "metric": 0.8867044557652293, "rescued": 2184, "harmed": 477, "missed_rescue": 522, "avoided_harm": 234, "selected_both_correct": 4302, "selected_both_wrong": 433}, {"budget": 0.54, "threshold": -0.9938417673110962, "operator": "ge", "actual_fraction": 0.5398579321231255, "metric": 0.8876372246537992, "rescued": 2203, "harmed": 483, "missed_rescue": 503, "avoided_harm": 228, "selected_both_correct": 4401, "selected_both_wrong": 437}, {"budget": 0.55, "threshold": -0.9944412112236023, "operator": "ge", "actual_fraction": 0.5504771471622301, "metric": 0.8882112362775346, "rescued": 2220, "harmed": 492, "missed_rescue": 486, "avoided_harm": 219, "selected_both_correct": 4514, "selected_both_wrong": 446}, {"budget": 0.56, "threshold": -0.9949513673782349, "operator": "ge", "actual_fraction": 0.5604505991246322, "metric": 0.8893592595250054, "rescued": 2242, "harmed": 498, "missed_rescue": 464, "avoided_harm": 213, "selected_both_correct": 4617, "selected_both_wrong": 454}, {"budget": 0.57, "threshold": -0.9954326152801514, "operator": "ge", "actual_fraction": 0.570926311257803, "metric": 0.8905072827724761, "rescued": 2262, "harmed": 502, "missed_rescue": 444, "avoided_harm": 209, "selected_both_correct": 4734, "selected_both_wrong": 459}, {"budget": 0.58, "threshold": -0.9958608746528625, "operator": "ge", "actual_fraction": 0.5816172777498745, "metric": 0.8916553060199469, "rescued": 2287, "harmed": 511, "missed_rescue": 419, "avoided_harm": 200, "selected_both_correct": 4841, "selected_both_wrong": 467}, {"budget": 0.59, "threshold": -0.9962128400802612, "operator": "ge", "actual_fraction": 0.5920929898830452, "metric": 0.8924445720025831, "rescued": 2308, "harmed": 521, "missed_rescue": 398, "avoided_harm": 190, "selected_both_correct": 4951, "selected_both_wrong": 472}, {"budget": 0.6, "threshold": -0.9965764880180359, "operator": "ge", "actual_fraction": 0.6034297194518189, "metric": 0.8928750807203846, "rescued": 2326, "harmed": 533, "missed_rescue": 380, "avoided_harm": 178, "selected_both_correct": 5075, "selected_both_wrong": 476}, {"budget": 0.61, "threshold": -0.9968799352645874, "operator": "ge", "actual_fraction": 0.6144794432087249, "metric": 0.8936643467030208, "rescued": 2346, "harmed": 542, "missed_rescue": 360, "avoided_harm": 169, "selected_both_correct": 5197, "selected_both_wrong": 479}, {"budget": 0.62, "threshold": -0.9971383810043335, "operator": "ge", "actual_fraction": 0.6232331204706896, "metric": 0.8946688670445576, "rescued": 2363, "harmed": 545, "missed_rescue": 343, "avoided_harm": 166, "selected_both_correct": 5293, "selected_both_wrong": 485}, {"budget": 0.63, "threshold": -0.997377872467041, "operator": "ge", "actual_fraction": 0.6337805840568271, "metric": 0.8953863815742269, "rescued": 2381, "harmed": 553, "missed_rescue": 325, "avoided_harm": 158, "selected_both_correct": 5411, "selected_both_wrong": 488}, {"budget": 0.64, "threshold": -0.9976021647453308, "operator": "ge", "actual_fraction": 0.6445433020018655, "metric": 0.8958886417449954, "rescued": 2398, "harmed": 563, "missed_rescue": 308, "avoided_harm": 148, "selected_both_correct": 5525, "selected_both_wrong": 497}, {"budget": 0.65, "threshold": -0.9978214502334595, "operator": "ge", "actual_fraction": 0.6565257946473416, "metric": 0.8963909019157638, "rescued": 2411, "harmed": 569, "missed_rescue": 295, "avoided_harm": 142, "selected_both_correct": 5668, "selected_both_wrong": 502}, {"budget": 0.66, "threshold": -0.9980514049530029, "operator": "ge", "actual_fraction": 0.6692258018224869, "metric": 0.8972519193513668, "rescued": 2430, "harmed": 576, "missed_rescue": 276, "avoided_harm": 135, "selected_both_correct": 5815, "selected_both_wrong": 506}, {"budget": 0.67, "threshold": -0.9982016086578369, "operator": "ge", "actual_fraction": 0.6777642247255506, "metric": 0.8978259309751022, "rescued": 2443, "harmed": 581, "missed_rescue": 263, "avoided_harm": 130, "selected_both_correct": 5912, "selected_both_wrong": 510}, {"budget": 0.68, "threshold": -0.9983693957328796, "operator": "ge", "actual_fraction": 0.6885269426705891, "metric": 0.8986869484107053, "rescued": 2462, "harmed": 588, "missed_rescue": 244, "avoided_harm": 123, "selected_both_correct": 6035, "selected_both_wrong": 511}, {"budget": 0.69, "threshold": -0.9984915256500244, "operator": "ge", "actual_fraction": 0.6977828801033221, "metric": 0.898973954222573, "rescued": 2473, "harmed": 595, "missed_rescue": 233, "avoided_harm": 116, "selected_both_correct": 6142, "selected_both_wrong": 515}, {"budget": 0.7, "threshold": -0.9986109733581543, "operator": "gt", "actual_fraction": 0.7064648059123197, "metric": 0.8988304513166392, "rescued": 2481, "harmed": 605, "missed_rescue": 225, "avoided_harm": 106, "selected_both_correct": 6242, "selected_both_wrong": 518}, {"budget": 0.71, "threshold": -0.9986966252326965, "operator": "ge", "actual_fraction": 0.7137117026619789, "metric": 0.8996197172992753, "rescued": 2497, "harmed": 610, "missed_rescue": 209, "avoided_harm": 101, "selected_both_correct": 6320, "selected_both_wrong": 520}, {"budget": 0.72, "threshold": -0.9987963438034058, "operator": "ge", "actual_fraction": 0.7256941953074549, "metric": 0.9004089832819114, "rescued": 2512, "harmed": 614, "missed_rescue": 194, "avoided_harm": 97, "selected_both_correct": 6462, "selected_both_wrong": 526}, {"budget": 0.73, "threshold": -0.998875081539154, "operator": "ge", "actual_fraction": 0.7345196240223865, "metric": 0.9009829949056468, "rescued": 2521, "harmed": 615, "missed_rescue": 185, "avoided_harm": 96, "selected_both_correct": 6569, "selected_both_wrong": 532}, {"budget": 0.74, "threshold": -0.9989551305770874, "operator": "ge", "actual_fraction": 0.745067087608524, "metric": 0.901772260888283, "rescued": 2535, "harmed": 618, "missed_rescue": 171, "avoided_harm": 93, "selected_both_correct": 6698, "selected_both_wrong": 533}, {"budget": 0.75, "threshold": -0.9990480542182922, "operator": "ge", "actual_fraction": 0.7568343258950994, "metric": 0.9023462725120184, "rescued": 2548, "harmed": 623, "missed_rescue": 158, "avoided_harm": 88, "selected_both_correct": 6839, "selected_both_wrong": 538}, {"budget": 0.76, "threshold": -0.999126136302948, "operator": "ge", "actual_fraction": 0.7663772691397001, "metric": 0.9028485326827869, "rescued": 2561, "harmed": 629, "missed_rescue": 145, "avoided_harm": 82, "selected_both_correct": 6951, "selected_both_wrong": 540}, {"budget": 0.77, "threshold": -0.999184787273407, "operator": "gt", "actual_fraction": 0.7746286862308962, "metric": 0.9030637870416875, "rescued": 2567, "harmed": 632, "missed_rescue": 139, "avoided_harm": 79, "selected_both_correct": 7054, "selected_both_wrong": 543}, {"budget": 0.78, "threshold": -0.9992268085479736, "operator": "ge", "actual_fraction": 0.7817320800746215, "metric": 0.9028485326827869, "rescued": 2570, "harmed": 638, "missed_rescue": 136, "avoided_harm": 73, "selected_both_correct": 7143, "selected_both_wrong": 544}, {"budget": 0.79, "threshold": -0.9992831349372864, "operator": "ge", "actual_fraction": 0.7928535552844945, "metric": 0.9028485326827869, "rescued": 2579, "harmed": 647, "missed_rescue": 127, "avoided_harm": 64, "selected_both_correct": 7279, "selected_both_wrong": 545}, {"budget": 0.8, "threshold": -0.9993318915367126, "operator": "ge", "actual_fraction": 0.8023247470761283, "metric": 0.9028485326827869, "rescued": 2581, "harmed": 649, "missed_rescue": 125, "avoided_harm": 62, "selected_both_correct": 7406, "selected_both_wrong": 546}, {"budget": 0.81, "threshold": -0.9993773102760315, "operator": "ge", "actual_fraction": 0.811795938867762, "metric": 0.9027767812298199, "rescued": 2588, "harmed": 657, "missed_rescue": 118, "avoided_harm": 54, "selected_both_correct": 7521, "selected_both_wrong": 548}, {"budget": 0.82, "threshold": -0.9994175434112549, "operator": "ge", "actual_fraction": 0.8214106335653297, "metric": 0.9034225443065222, "rescued": 2601, "harmed": 661, "missed_rescue": 105, "avoided_harm": 50, "selected_both_correct": 7637, "selected_both_wrong": 549}, {"budget": 0.83, "threshold": -0.999452531337738, "operator": "ge", "actual_fraction": 0.8298773050154266, "metric": 0.9037095501183899, "rescued": 2607, "harmed": 663, "missed_rescue": 99, "avoided_harm": 48, "selected_both_correct": 7745, "selected_both_wrong": 551}, {"budget": 0.84, "threshold": -0.9994945526123047, "operator": "ge", "actual_fraction": 0.8420750520198034, "metric": 0.9037095501183899, "rescued": 2612, "harmed": 668, "missed_rescue": 94, "avoided_harm": 43, "selected_both_correct": 7903, "selected_both_wrong": 553}, {"budget": 0.85, "threshold": -0.9995274543762207, "operator": "gt", "actual_fraction": 0.8502547176580326, "metric": 0.9039248044772906, "rescued": 2616, "harmed": 669, "missed_rescue": 90, "avoided_harm": 42, "selected_both_correct": 8008, "selected_both_wrong": 557}, {"budget": 0.86, "threshold": -0.9995617270469666, "operator": "ge", "actual_fraction": 0.8604434239793356, "metric": 0.9042835617421253, "rescued": 2623, "harmed": 671, "missed_rescue": 83, "avoided_harm": 40, "selected_both_correct": 8140, "selected_both_wrong": 558}, {"budget": 0.87, "threshold": -0.9995949864387512, "operator": "ge", "actual_fraction": 0.8690535983353663, "metric": 0.904570567553993, "rescued": 2629, "harmed": 673, "missed_rescue": 77, "avoided_harm": 38, "selected_both_correct": 8251, "selected_both_wrong": 559}, {"budget": 0.88, "threshold": -0.9996248483657837, "operator": "ge", "actual_fraction": 0.8778790270502977, "metric": 0.904498816101026, "rescued": 2634, "harmed": 679, "missed_rescue": 72, "avoided_harm": 32, "selected_both_correct": 8361, "selected_both_wrong": 561}, {"budget": 0.89, "threshold": -0.9996511936187744, "operator": "ge", "actual_fraction": 0.886058692688527, "metric": 0.9050010762717945, "rescued": 2642, "harmed": 680, "missed_rescue": 64, "avoided_harm": 31, "selected_both_correct": 8464, "selected_both_wrong": 563}, {"budget": 0.9, "threshold": -0.9996830224990845, "operator": "ge", "actual_fraction": 0.8961756475568631, "metric": 0.9052163306306953, "rescued": 2648, "harmed": 683, "missed_rescue": 58, "avoided_harm": 28, "selected_both_correct": 8590, "selected_both_wrong": 569}, {"budget": 0.91, "threshold": -0.9997097849845886, "operator": "gt", "actual_fraction": 0.9053598335366291, "metric": 0.9055750878955299, "rescued": 2654, "harmed": 684, "missed_rescue": 52, "avoided_harm": 27, "selected_both_correct": 8711, "selected_both_wrong": 569}, {"budget": 0.92, "threshold": -0.9997391104698181, "operator": "ge", "actual_fraction": 0.9164813087465021, "metric": 0.9057185908014638, "rescued": 2659, "harmed": 687, "missed_rescue": 47, "avoided_harm": 24, "selected_both_correct": 8855, "selected_both_wrong": 572}, {"budget": 0.93, "threshold": -0.9997641444206238, "operator": "ge", "actual_fraction": 0.9265265121618713, "metric": 0.9060773480662984, "rescued": 2667, "harmed": 690, "missed_rescue": 39, "avoided_harm": 21, "selected_both_correct": 8984, "selected_both_wrong": 572}, {"budget": 0.94, "threshold": -0.9997876286506653, "operator": "ge", "actual_fraction": 0.9362847097653727, "metric": 0.906364353878166, "rescued": 2674, "harmed": 693, "missed_rescue": 32, "avoided_harm": 18, "selected_both_correct": 9109, "selected_both_wrong": 573}, {"budget": 0.95, "threshold": -0.9998131394386292, "operator": "ge", "actual_fraction": 0.9475496878811795, "metric": 0.9065078567840998, "rescued": 2680, "harmed": 697, "missed_rescue": 26, "avoided_harm": 14, "selected_both_correct": 9256, "selected_both_wrong": 573}, {"budget": 0.96, "threshold": -0.9998401403427124, "operator": "ge", "actual_fraction": 0.9593169261677549, "metric": 0.906436105331133, "rescued": 2682, "harmed": 700, "missed_rescue": 24, "avoided_harm": 11, "selected_both_correct": 9413, "selected_both_wrong": 575}, {"budget": 0.97, "threshold": -0.999868631362915, "operator": "ge", "actual_fraction": 0.9700796441127932, "metric": 0.9070101169548683, "rescued": 2690, "harmed": 700, "missed_rescue": 16, "avoided_harm": 11, "selected_both_correct": 9554, "selected_both_wrong": 576}, {"budget": 0.98, "threshold": -0.9998925924301147, "operator": "ge", "actual_fraction": 0.9793355815455262, "metric": 0.9071536198608022, "rescued": 2696, "harmed": 704, "missed_rescue": 10, "avoided_harm": 7, "selected_both_correct": 9672, "selected_both_wrong": 577}, {"budget": 0.99, "threshold": -0.99992835521698, "operator": "ge", "actual_fraction": 0.9909593169261678, "metric": 0.9071536198608022, "rescued": 2702, "harmed": 710, "missed_rescue": 4, "avoided_harm": 1, "selected_both_correct": 9821, "selected_both_wrong": 578}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.9073688742197029, "rescued": 2706, "harmed": 711, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 9940, "selected_both_wrong": 580}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.8993125179031795, "actual_fraction": 0.9819535949584648, "metric_std": 0.00041757031615907866, "actual_fraction_std": 0.019284973994800794, "per_seed": [{"seed": 2026, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 2, "metric": 0.8990260670295044, "actual_fraction": 1.0, "rescue": 1389, "harm": 388, "missed_rescue": 0}, {"seed": 2027, "budget": 0.95, "threshold": -0.9996432517530086, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.9000286450873675, "actual_fraction": 0.9498710971068463, "rescue": 1377, "harm": 369, "missed_rescue": 12}, {"seed": 2028, "budget": 0.98, "threshold": -0.9980580462347883, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.8991692924663419, "actual_fraction": 0.9799484388427385, "rescue": 1384, "harm": 382, "missed_rescue": 5}, {"seed": 2029, "budget": 0.99, "threshold": -0.9999548796893659, "operator": "ge", "alpha": 1.0, "epoch": 5, "metric": 0.8993125179031796, "actual_fraction": 0.9899742194213692, "rescue": 1387, "harm": 384, "missed_rescue": 2}, {"seed": 2030, "budget": 0.99, "threshold": -0.9927928274576011, "operator": "ge", "alpha": 1.0, "epoch": 1, "metric": 0.8990260670295044, "actual_fraction": 0.9899742194213692, "rescue": 1387, "harm": 386, "missed_rescue": 2}], "rescued": 1384.8, "harmed": 381.8, "missed_rescue": 4.2, "avoided_harm": 6.199999999999989}, "validation_curve": [{"budget": 0.0, "metric": 0.7556574047550845, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.7596390718991692, "metric_std": 0.0004899063699939308, "actual_fraction": 0.009882555141793183, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.7631910627327413, "metric_std": 0.0009639827367782965, "actual_fraction": 0.01990833572042395, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.7669149240905184, "metric_std": 0.0008202704705399941, "actual_fraction": 0.029934116299054715, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.7700658837009453, "metric_std": 0.001574177217305558, "actual_fraction": 0.039959896877685475, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.7735319392724146, "metric_std": 0.000978765371714101, "actual_fraction": 0.04998567745631624, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.7765110283586365, "metric_std": 0.0008012947998109031, "actual_fraction": 0.05986823259810943, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.7798625035806358, "metric_std": 0.0009607854386277666, "actual_fraction": 0.06989401317674018, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.7829561730163277, "metric_std": 0.0007204111337702149, "actual_fraction": 0.07991979375537095, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.7859925522772844, "metric_std": 0.0010544349652996672, "actual_fraction": 0.08994557433400172, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.7886279003150959, "metric_std": 0.00153391701351932, "actual_fraction": 0.09997135491263248, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.7916356344886852, "metric_std": 0.001698897605523744, "actual_fraction": 0.10999713549126325, "actual_fraction_std": 0.0}, {"budget": 0.12, "metric": 0.7945574334001718, "metric_std": 0.0020371269623430917, "actual_fraction": 0.11987969063305641, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.13, "metric": 0.7972214265253508, "metric_std": 0.0018240871354835346, "actual_fraction": 0.1299054712116872, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.7999999999999999, "metric_std": 0.0012093859543158593, "actual_fraction": 0.13993125179031796, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.802663993125179, "metric_std": 0.0008852212626962115, "actual_fraction": 0.14995703236894872, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.8053566313377256, "metric_std": 0.001086060647959257, "actual_fraction": 0.1599828129475795, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.8074477227155542, "metric_std": 0.0016721224718071089, "actual_fraction": 0.16986536808937266, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.8106273274133485, "metric_std": 0.0010495600664887361, "actual_fraction": 0.17989114866800343, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.8125179031796048, "metric_std": 0.0005547097316252281, "actual_fraction": 0.18991692924663417, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.814924090518476, "metric_std": 0.0009413736307736741, "actual_fraction": 0.19994270982526496, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.8175021483815523, "metric_std": 0.0006110205960384551, "actual_fraction": 0.20996849040389573, "actual_fraction_std": 0.0}, {"budget": 0.22, "metric": 0.8195645946720138, "metric_std": 0.000326604246662617, "actual_fraction": 0.2199942709825265, "actual_fraction_std": 0.0}, {"budget": 0.23, "metric": 0.8212260097393298, "metric_std": 0.0005675053782915637, "actual_fraction": 0.22987682612431967, "actual_fraction_std": 0.0}, {"budget": 0.24, "metric": 0.8232025207676884, "metric_std": 0.0010157905028069865, "actual_fraction": 0.23990260670295044, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.8253795474076195, "metric_std": 0.0009080989678686956, "actual_fraction": 0.24992838728158123, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.26, "metric": 0.827785734746491, "metric_std": 0.001025838131164114, "actual_fraction": 0.25995416786021197, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.8302205671727299, "metric_std": 0.0011836696368602532, "actual_fraction": 0.26997994843884277, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.8320252076768833, "metric_std": 0.0014479234552964653, "actual_fraction": 0.2798625035806359, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.8334861071326267, "metric_std": 0.001420028021534846, "actual_fraction": 0.2898882841592667, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.835720423947293, "metric_std": 0.0008038507705758695, "actual_fraction": 0.29991406473789745, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.8378974505872243, "metric_std": 0.000787088608652805, "actual_fraction": 0.30993984531652824, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.8396734460040103, "metric_std": 0.0007619270168014935, "actual_fraction": 0.319965625895159, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.8422515038670868, "metric_std": 0.0008139943992198798, "actual_fraction": 0.3299914064737898, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.8444285305070179, "metric_std": 0.0008617364054134285, "actual_fraction": 0.3398739616155829, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.8468347178458895, "metric_std": 0.0005957214077079554, "actual_fraction": 0.3498997421942137, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.8492122600973933, "metric_std": 0.0009522068254709154, "actual_fraction": 0.35992552277284445, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.851532512174162, "metric_std": 0.0011923033737191015, "actual_fraction": 0.3699513033514752, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.8535949584646234, "metric_std": 0.0010689262765422717, "actual_fraction": 0.379977083930106, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.8557146949298196, "metric_std": 0.0015774316702673236, "actual_fraction": 0.3898596390718992, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.857347464909768, "metric_std": 0.0012689806034567046, "actual_fraction": 0.3998854196505299, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.8598682325981095, "metric_std": 0.0010746680883588146, "actual_fraction": 0.40991120022916067, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.42, "metric": 0.8613577771412203, "metric_std": 0.001236227191466107, "actual_fraction": 0.41993698080779146, "actual_fraction_std": 0.0}, {"budget": 0.43, "metric": 0.8625895158980235, "metric_std": 0.0011749724607272554, "actual_fraction": 0.42996276138642225, "actual_fraction_std": 0.0}, {"budget": 0.44, "metric": 0.8643368662274419, "metric_std": 0.0015604343970169157, "actual_fraction": 0.439988541965053, "actual_fraction_std": 0.0}, {"budget": 0.45, "metric": 0.8655399598968776, "metric_std": 0.0015538474769539255, "actual_fraction": 0.4498710971068462, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.8664852477800057, "metric_std": 0.001903874996242619, "actual_fraction": 0.459896877685477, "actual_fraction_std": 0.0}, {"budget": 0.47, "metric": 0.8673159553136637, "metric_std": 0.001401854403121363, "actual_fraction": 0.46992265826410773, "actual_fraction_std": 0.0}, {"budget": 0.48, "metric": 0.8683185333715266, "metric_std": 0.0016598091785526025, "actual_fraction": 0.4799484388427385, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.49, "metric": 0.8694356917788599, "metric_std": 0.0017175112820304902, "actual_fraction": 0.4899742194213693, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.5, "metric": 0.8705242050988256, "metric_std": 0.00098711320195432, "actual_fraction": 0.5, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.8712403322830135, "metric_std": 0.0008770730960982279, "actual_fraction": 0.5098825551417931, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.8718132340303637, "metric_std": 0.0010178079635886107, "actual_fraction": 0.5199083357204239, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.8727585219134918, "metric_std": 0.0009204393590982168, "actual_fraction": 0.5299341162990547, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.8736465196218848, "metric_std": 0.00112502646156714, "actual_fraction": 0.5399598968776855, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.8745631624176454, "metric_std": 0.001493258266282228, "actual_fraction": 0.5499856774563162, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.8753938699513034, "metric_std": 0.00155911923926361, "actual_fraction": 0.5598682325981095, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.8765969636207391, "metric_std": 0.0015177842213393146, "actual_fraction": 0.5698940131767402, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.8778573474649096, "metric_std": 0.0014156876302665782, "actual_fraction": 0.579919793755371, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.8784588942996276, "metric_std": 0.0011213737686086745, "actual_fraction": 0.5899455743340017, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.8791177313090804, "metric_std": 0.001047603755251222, "actual_fraction": 0.5999713549126325, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.8796906330564308, "metric_std": 0.0008227674944912574, "actual_fraction": 0.6099971354912632, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.8804067602406185, "metric_std": 0.0008652997689483318, "actual_fraction": 0.6198796906330565, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.8812088226869091, "metric_std": 0.0005782477581834809, "actual_fraction": 0.6299054712116872, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.8819822400458321, "metric_std": 0.0007232530033201291, "actual_fraction": 0.639931251790318, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.8825837868805501, "metric_std": 0.0009024339158993169, "actual_fraction": 0.6499570323689486, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.8831566886279003, "metric_std": 0.000549134617951311, "actual_fraction": 0.6599828129475795, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.8838728158120881, "metric_std": 0.00039743445790627773, "actual_fraction": 0.6698653680893727, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.8847608135204812, "metric_std": 0.0005957214077079193, "actual_fraction": 0.6798911486680035, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.8855628759667716, "metric_std": 0.000671786846150607, "actual_fraction": 0.6899169292466341, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.8863649384130621, "metric_std": 0.0006126969245547722, "actual_fraction": 0.699942709825265, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.88731022629619, "metric_std": 0.0006748335141443843, "actual_fraction": 0.7099684904038958, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.8880549985677456, "metric_std": 0.0006292146272268536, "actual_fraction": 0.7199942709825266, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.8888857061014036, "metric_std": 0.00044607308800909763, "actual_fraction": 0.7298768261243197, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.8892867373245487, "metric_std": 0.00039223901282233825, "actual_fraction": 0.7399026067029504, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.8897164136350615, "metric_std": 0.0006641090300412724, "actual_fraction": 0.7499283872815812, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.8901460899455742, "metric_std": 0.0009226653341969089, "actual_fraction": 0.759954167860212, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.8907476367802921, "metric_std": 0.0008063986399461364, "actual_fraction": 0.7699799484388428, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.8914924090518476, "metric_std": 0.0006210106957513343, "actual_fraction": 0.7798625035806359, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.8919220853623603, "metric_std": 0.0006532084933252571, "actual_fraction": 0.7898882841592667, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.8924663420223432, "metric_std": 0.0005764712631766782, "actual_fraction": 0.7999140647378975, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.8930105986823259, "metric_std": 0.0005729017473503212, "actual_fraction": 0.8099398453165282, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.8935262102549413, "metric_std": 0.0009148507260195003, "actual_fraction": 0.8199656258951592, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.83, "metric": 0.8938985963907189, "metric_std": 0.0008252569630987379, "actual_fraction": 0.8299914064737898, "actual_fraction_std": 0.0}, {"budget": 0.84, "metric": 0.8944714981380691, "metric_std": 0.0007805457802027484, "actual_fraction": 0.8398739616155829, "actual_fraction_std": 0.0}, {"budget": 0.85, "metric": 0.8946433686622743, "metric_std": 0.0008629258282092264, "actual_fraction": 0.8498997421942137, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.8949298195359494, "metric_std": 0.0006292146272268739, "actual_fraction": 0.8599255227728445, "actual_fraction_std": 0.0}, {"budget": 0.87, "metric": 0.8953308507590947, "metric_std": 0.00047932399113953295, "actual_fraction": 0.8699513033514752, "actual_fraction_std": 0.0}, {"budget": 0.88, "metric": 0.8955313663706674, "metric_std": 0.0004814590751638993, "actual_fraction": 0.879977083930106, "actual_fraction_std": 0.0}, {"budget": 0.89, "metric": 0.8959323975938126, "metric_std": 0.0006453096720749605, "actual_fraction": 0.8898596390718991, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.8964766542537955, "metric_std": 0.0005939971742574057, "actual_fraction": 0.8998854196505299, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.8969636207390431, "metric_std": 0.0008388169844941652, "actual_fraction": 0.9099112002291608, "actual_fraction_std": 0.0}, {"budget": 0.92, "metric": 0.8972214265253509, "metric_std": 0.0005782477581835537, "actual_fraction": 0.9199369808077915, "actual_fraction_std": 0.0}, {"budget": 0.93, "metric": 0.8979089086221712, "metric_std": 0.0006748335141443407, "actual_fraction": 0.9299627613864223, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.94, "metric": 0.8982526496705816, "metric_std": 0.000909227742658458, "actual_fraction": 0.9399885419650531, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.8986823259810943, "metric_std": 0.0007752717618288885, "actual_fraction": 0.9498710971068463, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.8986823259810943, "metric_std": 0.0005223312096479837, "actual_fraction": 0.959896877685477, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.97, "metric": 0.8987682612431968, "metric_std": 0.0003974344579063698, "actual_fraction": 0.9699226582641078, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.98, "metric": 0.8991120022916069, "metric_std": 0.0003297297173653403, "actual_fraction": 0.9799484388427386, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.99, "metric": 0.8989974219421368, "metric_std": 0.0002754996284971362, "actual_fraction": 0.9899742194213692, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.8990260670295044, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.9069383655019013, "actual_fraction": 0.9805984071177442, "metric_std": 0.0006172293368043996, "actual_fraction_std": 0.02160484980174617, "per_seed": [{"seed": 2026, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 2, "metric": 0.9073688742197031, "actual_fraction": 1.0, "rescue": 2706, "harm": 711, "missed_rescue": 0}, {"seed": 2027, "budget": 0.95, "threshold": -0.9996432517530086, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.9058620937073976, "actual_fraction": 0.9439621152328335, "rescue": 2666, "harm": 692, "missed_rescue": 40}, {"seed": 2028, "budget": 0.98, "threshold": -0.9980580462347883, "operator": "ge", "alpha": 1.0, "epoch": 3, "metric": 0.9070101169548682, "actual_fraction": 0.9809141135107986, "rescue": 2696, "harm": 706, "missed_rescue": 10}, {"seed": 2029, "budget": 0.99, "threshold": -0.9999548796893659, "operator": "ge", "alpha": 1.0, "epoch": 5, "metric": 0.9072971227667359, "actual_fraction": 0.9874434957307886, "rescue": 2699, "harm": 705, "missed_rescue": 7}, {"seed": 2030, "budget": 0.99, "threshold": -0.9927928274576011, "operator": "ge", "alpha": 1.0, "epoch": 1, "metric": 0.9071536198608022, "actual_fraction": 0.9906723111143001, "rescue": 2702, "harm": 710, "missed_rescue": 4}], "rescued": 2693.8, "harmed": 704.8, "missed_rescue": 12.2, "avoided_harm": 6.2000000000000455}, "test_curve": [{"budget": 0.0, "metric": 0.7642247255506924, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.7683289086604004, "metric_std": 0.0006054394233975682, "actual_fraction": 0.011408481021740689, "actual_fraction_std": 0.000781069852087151}, {"budget": 0.02, "metric": 0.7713998708473845, "metric_std": 0.0005369387080108769, "actual_fraction": 0.021109277462868625, "actual_fraction_std": 0.001730534380579098}, {"budget": 0.03, "metric": 0.7750735452392912, "metric_std": 0.0011892140448577523, "actual_fraction": 0.03111143000645763, "actual_fraction_std": 0.0021612557185347043}, {"budget": 0.04, "metric": 0.7785176149817035, "metric_std": 0.0009921438483142496, "actual_fraction": 0.04140058836191433, "actual_fraction_std": 0.0023294371931904916}, {"budget": 0.05, "metric": 0.781818181818182, "metric_std": 0.0011428544760496452, "actual_fraction": 0.05204850398220564, "actual_fraction_std": 0.0024925756651016386}, {"budget": 0.06, "metric": 0.7843868838343978, "metric_std": 0.0007491071614343555, "actual_fraction": 0.06064432804764297, "actual_fraction_std": 0.0022135061584342064}, {"budget": 0.07, "metric": 0.7871421396283275, "metric_std": 0.0011145751625580172, "actual_fraction": 0.0695845590873215, "actual_fraction_std": 0.002343429256187513}, {"budget": 0.08, "metric": 0.7898256439692904, "metric_std": 0.0015156379915659427, "actual_fraction": 0.07858219128937362, "actual_fraction_std": 0.0038423556257736685}, {"budget": 0.09, "metric": 0.7928822558656814, "metric_std": 0.0014409363953538225, "actual_fraction": 0.08842649063643539, "actual_fraction_std": 0.00412386755410044}, {"budget": 0.1, "metric": 0.7955370596254575, "metric_std": 0.0013274988377699297, "actual_fraction": 0.09728061993255363, "actual_fraction_std": 0.0036023945809586866}, {"budget": 0.11, "metric": 0.798392767453541, "metric_std": 0.0014182487372880403, "actual_fraction": 0.10581904283561742, "actual_fraction_std": 0.0038767038462541926}, {"budget": 0.12, "metric": 0.8012341249910311, "metric_std": 0.0012064063214812257, "actual_fraction": 0.11541938724259168, "actual_fraction_std": 0.002753610998855659}, {"budget": 0.13, "metric": 0.8041615842720816, "metric_std": 0.0010757933451383652, "actual_fraction": 0.1250197316495659, "actual_fraction_std": 0.0032835992388382477}, {"budget": 0.14, "metric": 0.8066728851259238, "metric_std": 0.0008912789775048377, "actual_fraction": 0.134017363851618, "actual_fraction_std": 0.00249763087228096}, {"budget": 0.15, "metric": 0.8096864461505344, "metric_std": 0.0014301776356236253, "actual_fraction": 0.1436607591303724, "actual_fraction_std": 0.00354948570897391}, {"budget": 0.16, "metric": 0.81212599555141, "metric_std": 0.0014918436763073542, "actual_fraction": 0.15316065150319294, "actual_fraction_std": 0.0032933838751100513}, {"budget": 0.17, "metric": 0.8143933414651647, "metric_std": 0.0017789441209495153, "actual_fraction": 0.16194302934634425, "actual_fraction_std": 0.0024124925634576144}, {"budget": 0.18, "metric": 0.8166176365071394, "metric_std": 0.0015291647031961885, "actual_fraction": 0.17119896677907728, "actual_fraction_std": 0.0020900497472382093}, {"budget": 0.19, "metric": 0.8191719882327618, "metric_std": 0.0016198992963020891, "actual_fraction": 0.18141637368156704, "actual_fraction_std": 0.0021713559322283703}, {"budget": 0.2, "metric": 0.8217980914113511, "metric_std": 0.0016854720227226004, "actual_fraction": 0.1924086962760996, "actual_fraction_std": 0.0030553821053174762}, {"budget": 0.21, "metric": 0.824065437325106, "metric_std": 0.0018286065777964764, "actual_fraction": 0.20262610317858937, "actual_fraction_std": 0.0036314316975316337}, {"budget": 0.22, "metric": 0.8264332352730144, "metric_std": 0.002006476494813785, "actual_fraction": 0.2123412499103107, "actual_fraction_std": 0.002825586429618151}, {"budget": 0.23, "metric": 0.8289158355456699, "metric_std": 0.0012256694002200858, "actual_fraction": 0.22257300710339384, "actual_fraction_std": 0.0023355070386395687}, {"budget": 0.24, "metric": 0.8312405826217983, "metric_std": 0.0012815219272075437, "actual_fraction": 0.23312047068953148, "actual_fraction_std": 0.002485543241111982}, {"budget": 0.25, "metric": 0.8330056683647845, "metric_std": 0.0015912007036225045, "actual_fraction": 0.2420750520198034, "actual_fraction_std": 0.004024290756606788}, {"budget": 0.26, "metric": 0.8349860084666714, "metric_std": 0.0009908457398840039, "actual_fraction": 0.25101528305948195, "actual_fraction_std": 0.0033919546042365674}, {"budget": 0.27, "metric": 0.8374829590299203, "metric_std": 0.000893298385448008, "actual_fraction": 0.26150534548324605, "actual_fraction_std": 0.0029227600098285724}, {"budget": 0.28, "metric": 0.8400516610461363, "metric_std": 0.0009053202437016064, "actual_fraction": 0.2725694195307455, "actual_fraction_std": 0.003001321337963004}, {"budget": 0.29, "metric": 0.8425055607376049, "metric_std": 0.0011558444208644306, "actual_fraction": 0.28343258950993755, "actual_fraction_std": 0.0032979921388250426}, {"budget": 0.3, "metric": 0.8446150534548325, "metric_std": 0.0012119413010196658, "actual_fraction": 0.29412355600200907, "actual_fraction_std": 0.0029910116217891267}, {"budget": 0.31, "metric": 0.847470761282916, "metric_std": 0.0006075615449528151, "actual_fraction": 0.30541723469900267, "actual_fraction_std": 0.0027059907330540083}, {"budget": 0.32, "metric": 0.8492788978976826, "metric_std": 0.0009322167909255057, "actual_fraction": 0.3153476357896247, "actual_fraction_std": 0.003585419458606296}, {"budget": 0.33, "metric": 0.8519624022386454, "metric_std": 0.0008660849179597364, "actual_fraction": 0.3271148740762, "actual_fraction_std": 0.0029601279105598488}, {"budget": 0.34, "metric": 0.8540575446652795, "metric_std": 0.0009746055612546767, "actual_fraction": 0.3380641457989525, "actual_fraction_std": 0.002700753664601509}, {"budget": 0.35, "metric": 0.8566979981344623, "metric_std": 0.0012524688317769796, "actual_fraction": 0.3482815527014422, "actual_fraction_std": 0.0021626844906509892}, {"budget": 0.36, "metric": 0.8585348353304154, "metric_std": 0.0007185899954684577, "actual_fraction": 0.35835545669799806, "actual_fraction_std": 0.001780535111578533}, {"budget": 0.37, "metric": 0.860974384731291, "metric_std": 0.0008079608673464303, "actual_fraction": 0.3687737676687952, "actual_fraction_std": 0.0014866582463799916}, {"budget": 0.38, "metric": 0.8632704312262325, "metric_std": 0.0005548572843275627, "actual_fraction": 0.3790916266054388, "actual_fraction_std": 0.001912280547758269}, {"budget": 0.39, "metric": 0.8652794719093062, "metric_std": 0.0005708638668555959, "actual_fraction": 0.38939513525148883, "actual_fraction_std": 0.0019617167416936653}, {"budget": 0.4, "metric": 0.8670732582334792, "metric_std": 0.0005359790312687727, "actual_fraction": 0.4008036162732296, "actual_fraction_std": 0.0012785053943607278}, {"budget": 0.41, "metric": 0.8688813948482457, "metric_std": 0.00040778567090145063, "actual_fraction": 0.4107340173638517, "actual_fraction_std": 0.0024733965376995217}, {"budget": 0.42, "metric": 0.8707756332065726, "metric_std": 0.00046222640190591435, "actual_fraction": 0.4207792207792208, "actual_fraction_std": 0.0028864245986922705}, {"budget": 0.43, "metric": 0.8724115663342182, "metric_std": 0.0006242788759174885, "actual_fraction": 0.43114013058764444, "actual_fraction_std": 0.002305779578506619}, {"budget": 0.44, "metric": 0.8740762000430509, "metric_std": 0.0006788006001443909, "actual_fraction": 0.4418741479514961, "actual_fraction_std": 0.0016374429877989251}, {"budget": 0.45, "metric": 0.8756403817177297, "metric_std": 0.0010139574543697755, "actual_fraction": 0.4522207074693263, "actual_fraction_std": 0.001923019256626829}, {"budget": 0.46, "metric": 0.8769606084523212, "metric_std": 0.0011073925017275495, "actual_fraction": 0.46091698356891725, "actual_fraction_std": 0.0018295917047424603}, {"budget": 0.47, "metric": 0.8781660328621654, "metric_std": 0.0015380606378866564, "actual_fraction": 0.4703164239075841, "actual_fraction_std": 0.0013972202275540574}, {"budget": 0.48, "metric": 0.8795149601779437, "metric_std": 0.001520724608365006, "actual_fraction": 0.48079213604075477, "actual_fraction_std": 0.0018183013056255685}, {"budget": 0.49, "metric": 0.880462079357107, "metric_std": 0.0015212323345851861, "actual_fraction": 0.49036377986654234, "actual_fraction_std": 0.0015062679147357406}, {"budget": 0.5, "metric": 0.8817105546387314, "metric_std": 0.0014935681522920553, "actual_fraction": 0.5003802827007247, "actual_fraction_std": 0.0024408152745578186}, {"budget": 0.51, "metric": 0.8826146229461147, "metric_std": 0.0013171815850388244, "actual_fraction": 0.5104398364066872, "actual_fraction_std": 0.003472725849273217}, {"budget": 0.52, "metric": 0.8833895386381576, "metric_std": 0.0012473201261366796, "actual_fraction": 0.5190787113439047, "actual_fraction_std": 0.0022223267925865066}, {"budget": 0.53, "metric": 0.8842792566549473, "metric_std": 0.0013008599765699362, "actual_fraction": 0.5279041400588362, "actual_fraction_std": 0.003456231069813812}, {"budget": 0.54, "metric": 0.8852120255435173, "metric_std": 0.0015002746690030416, "actual_fraction": 0.5381789481236995, "actual_fraction_std": 0.0034495215275340135}, {"budget": 0.55, "metric": 0.8858434383296263, "metric_std": 0.0019419342186588785, "actual_fraction": 0.5475783884623664, "actual_fraction_std": 0.004041588083648339}, {"budget": 0.56, "metric": 0.886661404893449, "metric_std": 0.001699161669296478, "actual_fraction": 0.5572504843223075, "actual_fraction_std": 0.003355713510501984}, {"budget": 0.57, "metric": 0.8875080720384588, "metric_std": 0.0015484023398438782, "actual_fraction": 0.5681567051732798, "actual_fraction_std": 0.004932253538681449}, {"budget": 0.58, "metric": 0.8884551912176221, "metric_std": 0.0014814546663739472, "actual_fraction": 0.5777426992896606, "actual_fraction_std": 0.005506661076085231}, {"budget": 0.59, "metric": 0.8891440051661046, "metric_std": 0.0015016466625856627, "actual_fraction": 0.5865681280045921, "actual_fraction_std": 0.005970695316237054}, {"budget": 0.6, "metric": 0.889789768242807, "metric_std": 0.0015547068006041905, "actual_fraction": 0.5971155915907298, "actual_fraction_std": 0.006139467904262766}, {"budget": 0.61, "metric": 0.8906077348066299, "metric_std": 0.0015593358825644007, "actual_fraction": 0.6069024897754179, "actual_fraction_std": 0.00497460642227795}, {"budget": 0.62, "metric": 0.8911960967209586, "metric_std": 0.001497011144763779, "actual_fraction": 0.6168902920284135, "actual_fraction_std": 0.005482674961323662}, {"budget": 0.63, "metric": 0.8920858147377485, "metric_std": 0.001410605098036736, "actual_fraction": 0.6270359474779363, "actual_fraction_std": 0.00538972347469356}, {"budget": 0.64, "metric": 0.892631125780297, "metric_std": 0.0013419631875733126, "actual_fraction": 0.6369663485685585, "actual_fraction_std": 0.006594227882890779}, {"budget": 0.65, "metric": 0.8932768888569994, "metric_std": 0.0014622164502869107, "actual_fraction": 0.6467388964626534, "actual_fraction_std": 0.006790963259321838}, {"budget": 0.66, "metric": 0.8939944033866686, "metric_std": 0.001440936395353793, "actual_fraction": 0.6565544952285285, "actual_fraction_std": 0.0067252587490345085}, {"budget": 0.67, "metric": 0.8945540647198105, "metric_std": 0.0016654994495175761, "actual_fraction": 0.6660113367295688, "actual_fraction_std": 0.00668375471917215}, {"budget": 0.68, "metric": 0.8950993757623593, "metric_std": 0.0017568120178941987, "actual_fraction": 0.6763865968285858, "actual_fraction_std": 0.006572292093952725}, {"budget": 0.69, "metric": 0.8957451388390616, "metric_std": 0.0018045169462523217, "actual_fraction": 0.6856999354236923, "actual_fraction_std": 0.006799486604665345}, {"budget": 0.7, "metric": 0.8965057042405109, "metric_std": 0.0017306831221995719, "actual_fraction": 0.6955729353519409, "actual_fraction_std": 0.007356781128442898}, {"budget": 0.71, "metric": 0.8974241228384875, "metric_std": 0.0015855285335492552, "actual_fraction": 0.7050728277247614, "actual_fraction_std": 0.007430970671623275}, {"budget": 0.72, "metric": 0.8980124847528164, "metric_std": 0.0014674882488449985, "actual_fraction": 0.7158355456697998, "actual_fraction_std": 0.0070788181324413805}, {"budget": 0.73, "metric": 0.8988591518978261, "metric_std": 0.0014436135570018273, "actual_fraction": 0.7256080935638944, "actual_fraction_std": 0.006487339282565483}, {"budget": 0.74, "metric": 0.8994044629403746, "metric_std": 0.0011625064035316315, "actual_fraction": 0.7370452751668222, "actual_fraction_std": 0.005145984993899063}, {"budget": 0.75, "metric": 0.8998636722393629, "metric_std": 0.0011385669231632203, "actual_fraction": 0.7473631341034654, "actual_fraction_std": 0.005805885323639694}, {"budget": 0.76, "metric": 0.9003085312477577, "metric_std": 0.0009294513834949608, "actual_fraction": 0.7578388462366362, "actual_fraction_std": 0.006152451751014484}, {"budget": 0.77, "metric": 0.9008825428714932, "metric_std": 0.0008825720382685082, "actual_fraction": 0.768730716797015, "actual_fraction_std": 0.006343696867607343}, {"budget": 0.78, "metric": 0.9014278539140419, "metric_std": 0.0010177583828081904, "actual_fraction": 0.7801535481093491, "actual_fraction_std": 0.00751508701888562}, {"budget": 0.79, "metric": 0.9016574585635361, "metric_std": 0.001018011273452653, "actual_fraction": 0.7904570567553993, "actual_fraction_std": 0.006872538298298523}, {"budget": 0.8, "metric": 0.9020879672813373, "metric_std": 0.0009027576085270337, "actual_fraction": 0.8014924302217119, "actual_fraction_std": 0.0066442000417930094}, {"budget": 0.81, "metric": 0.9024036736743918, "metric_std": 0.0008895444076269025, "actual_fraction": 0.8118389897395423, "actual_fraction_std": 0.006803460511574524}, {"budget": 0.82, "metric": 0.9027624309392266, "metric_std": 0.0008281000528809535, "actual_fraction": 0.8216976393771974, "actual_fraction_std": 0.007528775713324053}, {"budget": 0.83, "metric": 0.9032072899476216, "metric_std": 0.0009666494643974403, "actual_fraction": 0.8310253282628974, "actual_fraction_std": 0.007448167169939611}, {"budget": 0.84, "metric": 0.9036090980842362, "metric_std": 0.0009486399587307864, "actual_fraction": 0.8405252206357178, "actual_fraction_std": 0.007027724255531087}, {"budget": 0.85, "metric": 0.9040396068020378, "metric_std": 0.0008999016772684649, "actual_fraction": 0.8513309894525365, "actual_fraction_std": 0.007216641148413703}, {"budget": 0.86, "metric": 0.9044127143574657, "metric_std": 0.0011629491787917916, "actual_fraction": 0.8617923512951137, "actual_fraction_std": 0.007519093540590981}, {"budget": 0.87, "metric": 0.90464231900696, "metric_std": 0.001007079633179194, "actual_fraction": 0.8713352945397144, "actual_fraction_std": 0.00690948237306986}, {"budget": 0.88, "metric": 0.9049580254000145, "metric_std": 0.0009112717008455755, "actual_fraction": 0.8808208366219417, "actual_fraction_std": 0.007348834112976636}, {"budget": 0.89, "metric": 0.9055176867331565, "metric_std": 0.0007866525119417492, "actual_fraction": 0.8912678481739256, "actual_fraction_std": 0.007315411637124395}, {"budget": 0.9, "metric": 0.9057759919638372, "metric_std": 0.0007128354284003694, "actual_fraction": 0.9008681925808997, "actual_fraction_std": 0.007198712819708353}, {"budget": 0.91, "metric": 0.9059912463227381, "metric_std": 0.0007376803135524008, "actual_fraction": 0.9108129439621153, "actual_fraction_std": 0.007218959289710402}, {"budget": 0.92, "metric": 0.9062065006816388, "metric_std": 0.0007515087018885744, "actual_fraction": 0.9210016502834183, "actual_fraction_std": 0.006195396231223314}, {"budget": 0.93, "metric": 0.9063500035875727, "metric_std": 0.0007497941007455484, "actual_fraction": 0.9307598478869196, "actual_fraction_std": 0.0050963739275538935}, {"budget": 0.94, "metric": 0.9064361053311331, "metric_std": 0.0005960123314140744, "actual_fraction": 0.9407476501399152, "actual_fraction_std": 0.005049793652357328}, {"budget": 0.95, "metric": 0.9065365573652866, "metric_std": 0.000561773096966716, "actual_fraction": 0.9503049436751094, "actual_fraction_std": 0.0057559246333438365}, {"budget": 0.96, "metric": 0.9068235631771543, "metric_std": 0.00036796674764896217, "actual_fraction": 0.9600631412786109, "actual_fraction_std": 0.004503682726068271}, {"budget": 0.97, "metric": 0.9069383655019013, "metric_std": 0.00026363167210837396, "actual_fraction": 0.9695773839420248, "actual_fraction_std": 0.00276359549007254}, {"budget": 0.98, "metric": 0.9070388175360551, "metric_std": 0.00016518424959781835, "actual_fraction": 0.980627107698931, "actual_fraction_std": 0.0014835382885940477}, {"budget": 0.99, "metric": 0.9072110210231756, "metric_std": 0.00016361848677611313, "actual_fraction": 0.9902131018153117, "actual_fraction_std": 0.001596852725744829}, {"budget": 1.0, "metric": 0.9073688742197031, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 4888, "rescuable": 1389, "harmful": 388, "both_wrong": 317, "unit": "sample", "total": 6982}, "test": {"both_correct": 9940, "rescuable": 2706, "harmful": 711, "both_wrong": 580, "unit": "sample", "total": 13937}}, "small": {"validation": 0.7556574047550845, "test": 0.7642247255506924}, "large": {"validation": 0.8990260670295045, "test": 0.9073688742197029}, "learned_minus_confidence_test": 0.000502260170768376, "learned_minus_small_test": 0.1427136399512089, "learned_minus_large_test": -0.00043050871780159206, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_vilt_MiniCPM"}, {"key": ["nlvr2", "vilt", "Qwen3.8"], "expert_display": "ViLT", "run": "outputs/full_budget_20260913/runs/full_vilt_Qwen3.8_s{2026..2030}", "validation_rows": 6982, "test_rows": 13937, "confidence_rule": {"column": "max_probability", "columns": ["entropy", "max_probability", "top_two_margin", "entropy_valid", "max_probability_valid", "top_two_margin_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/vilt/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 96, "validation_area_0_25": 0.7940561443712404, "validation_point": {"budget": 0.96, "threshold": -0.9998401403427124, "operator": "ge", "actual_fraction": 0.9598968776854769, "metric": 0.9059008879977084, "rescued": 1446, "harmed": 397, "missed_rescue": 3, "avoided_harm": 9, "selected_both_correct": 4603, "selected_both_wrong": 256}, "test_point": {"budget": 0.96, "threshold": -0.9998401403427124, "operator": "ge", "actual_fraction": 0.9593169261677549, "metric": 0.8996197172992753, "rescued": 2770, "harmed": 883, "missed_rescue": 21, "avoided_harm": 7, "selected_both_correct": 9230, "selected_both_wrong": 487}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.7556574047550845, "rescued": 0, "harmed": 0, "missed_rescue": 1449, "avoided_harm": 406, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.5127503275871277, "operator": "ge", "actual_fraction": 0.009882555141793183, "metric": 0.7588083643655114, "rescued": 28, "harmed": 6, "missed_rescue": 1421, "avoided_harm": 400, "selected_both_correct": 28, "selected_both_wrong": 7}, {"budget": 0.02, "threshold": -0.5296345353126526, "operator": "ge", "actual_fraction": 0.01990833572042395, "metric": 0.7610999713549126, "rescued": 51, "harmed": 13, "missed_rescue": 1398, "avoided_harm": 393, "selected_both_correct": 62, "selected_both_wrong": 13}, {"budget": 0.03, "threshold": -0.5434877276420593, "operator": "ge", "actual_fraction": 0.029934116299054712, "metric": 0.7632483529074764, "rescued": 77, "harmed": 24, "missed_rescue": 1372, "avoided_harm": 382, "selected_both_correct": 91, "selected_both_wrong": 17}, {"budget": 0.04, "threshold": -0.56125408411026, "operator": "ge", "actual_fraction": 0.039959896877685475, "metric": 0.7656831853337153, "rescued": 102, "harmed": 32, "missed_rescue": 1347, "avoided_harm": 374, "selected_both_correct": 120, "selected_both_wrong": 25}, {"budget": 0.05, "threshold": -0.5779195427894592, "operator": "ge", "actual_fraction": 0.04998567745631624, "metric": 0.7698367230020051, "rescued": 132, "harmed": 33, "missed_rescue": 1317, "avoided_harm": 373, "selected_both_correct": 153, "selected_both_wrong": 31}, {"budget": 0.06, "threshold": -0.5911363363265991, "operator": "ge", "actual_fraction": 0.059868232598109423, "metric": 0.7727012317387568, "rescued": 156, "harmed": 37, "missed_rescue": 1293, "avoided_harm": 369, "selected_both_correct": 188, "selected_both_wrong": 37}, {"budget": 0.07, "threshold": -0.6087464094161987, "operator": "ge", "actual_fraction": 0.06989401317674018, "metric": 0.7772844457175594, "rescued": 191, "harmed": 40, "missed_rescue": 1258, "avoided_harm": 366, "selected_both_correct": 218, "selected_both_wrong": 39}, {"budget": 0.08, "threshold": -0.6274068355560303, "operator": "ge", "actual_fraction": 0.07991979375537095, "metric": 0.780865081638499, "rescued": 221, "harmed": 45, "missed_rescue": 1228, "avoided_harm": 361, "selected_both_correct": 249, "selected_both_wrong": 43}, {"budget": 0.09, "threshold": -0.6465112566947937, "operator": "ge", "actual_fraction": 0.08994557433400172, "metric": 0.784302492122601, "rescued": 250, "harmed": 50, "missed_rescue": 1199, "avoided_harm": 356, "selected_both_correct": 283, "selected_both_wrong": 45}, {"budget": 0.1, "threshold": -0.6620025038719177, "operator": "ge", "actual_fraction": 0.09997135491263248, "metric": 0.7884560297908909, "rescued": 284, "harmed": 55, "missed_rescue": 1165, "avoided_harm": 351, "selected_both_correct": 309, "selected_both_wrong": 50}, {"budget": 0.11, "threshold": -0.6800539493560791, "operator": "ge", "actual_fraction": 0.10999713549126325, "metric": 0.7910340876539673, "rescued": 306, "harmed": 59, "missed_rescue": 1143, "avoided_harm": 347, "selected_both_correct": 348, "selected_both_wrong": 55}, {"budget": 0.12, "threshold": -0.6957284212112427, "operator": "ge", "actual_fraction": 0.11987969063305644, "metric": 0.7944714981380693, "rescued": 334, "harmed": 63, "missed_rescue": 1115, "avoided_harm": 343, "selected_both_correct": 380, "selected_both_wrong": 60}, {"budget": 0.13, "threshold": -0.7128273248672485, "operator": "ge", "actual_fraction": 0.1299054712116872, "metric": 0.7974792323116585, "rescued": 362, "harmed": 70, "missed_rescue": 1087, "avoided_harm": 336, "selected_both_correct": 412, "selected_both_wrong": 63}, {"budget": 0.14, "threshold": -0.7332167625427246, "operator": "ge", "actual_fraction": 0.13993125179031796, "metric": 0.8006301919220854, "rescued": 386, "harmed": 72, "missed_rescue": 1063, "avoided_harm": 334, "selected_both_correct": 452, "selected_both_wrong": 67}, {"budget": 0.15, "threshold": -0.7503995895385742, "operator": "ge", "actual_fraction": 0.14995703236894872, "metric": 0.8039243769693497, "rescued": 417, "harmed": 80, "missed_rescue": 1032, "avoided_harm": 326, "selected_both_correct": 479, "selected_both_wrong": 71}, {"budget": 0.16, "threshold": -0.7655540704727173, "operator": "ge", "actual_fraction": 0.1599828129475795, "metric": 0.8054998567745631, "rescued": 438, "harmed": 90, "missed_rescue": 1011, "avoided_harm": 316, "selected_both_correct": 517, "selected_both_wrong": 72}, {"budget": 0.17, "threshold": -0.7812193036079407, "operator": "ge", "actual_fraction": 0.16986536808937266, "metric": 0.8086508163849899, "rescued": 464, "harmed": 94, "missed_rescue": 985, "avoided_harm": 312, "selected_both_correct": 551, "selected_both_wrong": 77}, {"budget": 0.18, "threshold": -0.7954917550086975, "operator": "ge", "actual_fraction": 0.17989114866800343, "metric": 0.810369521627041, "rescued": 481, "harmed": 99, "missed_rescue": 968, "avoided_harm": 307, "selected_both_correct": 597, "selected_both_wrong": 79}, {"budget": 0.19, "threshold": -0.8097342252731323, "operator": "ge", "actual_fraction": 0.1899169292466342, "metric": 0.8132340303637926, "rescued": 505, "harmed": 103, "missed_rescue": 944, "avoided_harm": 303, "selected_both_correct": 635, "selected_both_wrong": 83}, {"budget": 0.2, "threshold": -0.8232117891311646, "operator": "ge", "actual_fraction": 0.19994270982526496, "metric": 0.8158120882268691, "rescued": 527, "harmed": 107, "missed_rescue": 922, "avoided_harm": 299, "selected_both_correct": 673, "selected_both_wrong": 89}, {"budget": 0.21, "threshold": -0.8359466791152954, "operator": "ge", "actual_fraction": 0.20996849040389573, "metric": 0.8181036952162704, "rescued": 550, "harmed": 114, "missed_rescue": 899, "avoided_harm": 292, "selected_both_correct": 710, "selected_both_wrong": 92}, {"budget": 0.22, "threshold": -0.8473842144012451, "operator": "ge", "actual_fraction": 0.2199942709825265, "metric": 0.8203953022056717, "rescued": 574, "harmed": 122, "missed_rescue": 875, "avoided_harm": 284, "selected_both_correct": 745, "selected_both_wrong": 95}, {"budget": 0.23, "threshold": -0.8595863580703735, "operator": "ge", "actual_fraction": 0.22987682612431967, "metric": 0.8234030363792609, "rescued": 597, "harmed": 124, "missed_rescue": 852, "avoided_harm": 282, "selected_both_correct": 779, "selected_both_wrong": 105}, {"budget": 0.24, "threshold": -0.8704345226287842, "operator": "ge", "actual_fraction": 0.23990260670295044, "metric": 0.8256946433686623, "rescued": 618, "harmed": 129, "missed_rescue": 831, "avoided_harm": 277, "selected_both_correct": 819, "selected_both_wrong": 109}, {"budget": 0.25, "threshold": -0.8811548352241516, "operator": "ge", "actual_fraction": 0.2499283872815812, "metric": 0.828845602979089, "rescued": 647, "harmed": 136, "missed_rescue": 802, "avoided_harm": 270, "selected_both_correct": 847, "selected_both_wrong": 115}, {"budget": 0.26, "threshold": -0.8929013609886169, "operator": "ge", "actual_fraction": 0.25995416786021197, "metric": 0.8304210827843025, "rescued": 667, "harmed": 145, "missed_rescue": 782, "avoided_harm": 261, "selected_both_correct": 886, "selected_both_wrong": 117}, {"budget": 0.27, "threshold": -0.903549313545227, "operator": "ge", "actual_fraction": 0.26997994843884277, "metric": 0.8327126897737038, "rescued": 693, "harmed": 155, "missed_rescue": 756, "avoided_harm": 251, "selected_both_correct": 919, "selected_both_wrong": 118}, {"budget": 0.28, "threshold": -0.9115740656852722, "operator": "ge", "actual_fraction": 0.2798625035806359, "metric": 0.8350042967631052, "rescued": 716, "harmed": 162, "missed_rescue": 733, "avoided_harm": 244, "selected_both_correct": 954, "selected_both_wrong": 122}, {"budget": 0.29, "threshold": -0.9182595014572144, "operator": "ge", "actual_fraction": 0.2898882841592667, "metric": 0.8371526783156689, "rescued": 738, "harmed": 169, "missed_rescue": 711, "avoided_harm": 237, "selected_both_correct": 994, "selected_both_wrong": 123}, {"budget": 0.3, "threshold": -0.9258768558502197, "operator": "ge", "actual_fraction": 0.29991406473789745, "metric": 0.8377255800630192, "rescued": 751, "harmed": 178, "missed_rescue": 698, "avoided_harm": 228, "selected_both_correct": 1036, "selected_both_wrong": 129}, {"budget": 0.31, "threshold": -0.9331877827644348, "operator": "ge", "actual_fraction": 0.30993984531652824, "metric": 0.8401604124892581, "rescued": 771, "harmed": 181, "missed_rescue": 678, "avoided_harm": 225, "selected_both_correct": 1077, "selected_both_wrong": 135}, {"budget": 0.32, "threshold": -0.9399217367172241, "operator": "ge", "actual_fraction": 0.319965625895159, "metric": 0.842595244915497, "rescued": 793, "harmed": 186, "missed_rescue": 656, "avoided_harm": 220, "selected_both_correct": 1118, "selected_both_wrong": 137}, {"budget": 0.33, "threshold": -0.9453923106193542, "operator": "ge", "actual_fraction": 0.3299914064737898, "metric": 0.8447436264680608, "rescued": 814, "harmed": 192, "missed_rescue": 635, "avoided_harm": 214, "selected_both_correct": 1157, "selected_both_wrong": 141}, {"budget": 0.34, "threshold": -0.9515207409858704, "operator": "ge", "actual_fraction": 0.3398739616155829, "metric": 0.8466055571469493, "rescued": 830, "harmed": 195, "missed_rescue": 619, "avoided_harm": 211, "selected_both_correct": 1202, "selected_both_wrong": 146}, {"budget": 0.35, "threshold": -0.9567410349845886, "operator": "ge", "actual_fraction": 0.3498997421942137, "metric": 0.8491836150100258, "rescued": 854, "harmed": 201, "missed_rescue": 595, "avoided_harm": 205, "selected_both_correct": 1238, "selected_both_wrong": 150}, {"budget": 0.36, "threshold": -0.9606627225875854, "operator": "ge", "actual_fraction": 0.35992552277284445, "metric": 0.8519048983099399, "rescued": 876, "harmed": 204, "missed_rescue": 573, "avoided_harm": 202, "selected_both_correct": 1279, "selected_both_wrong": 154}, {"budget": 0.37, "threshold": -0.9650874733924866, "operator": "ge", "actual_fraction": 0.36995130335147525, "metric": 0.8534803781151532, "rescued": 894, "harmed": 211, "missed_rescue": 555, "avoided_harm": 195, "selected_both_correct": 1318, "selected_both_wrong": 160}, {"budget": 0.38, "threshold": -0.9686882495880127, "operator": "ge", "actual_fraction": 0.379977083930106, "metric": 0.8562016614150673, "rescued": 916, "harmed": 214, "missed_rescue": 533, "avoided_harm": 192, "selected_both_correct": 1359, "selected_both_wrong": 164}, {"budget": 0.39, "threshold": -0.971785306930542, "operator": "ge", "actual_fraction": 0.3898596390718992, "metric": 0.8567745631624176, "rescued": 927, "harmed": 221, "missed_rescue": 522, "avoided_harm": 185, "selected_both_correct": 1406, "selected_both_wrong": 168}, {"budget": 0.4, "threshold": -0.9748144745826721, "operator": "ge", "actual_fraction": 0.3998854196505299, "metric": 0.859066170151819, "rescued": 948, "harmed": 226, "missed_rescue": 501, "avoided_harm": 180, "selected_both_correct": 1447, "selected_both_wrong": 171}, {"budget": 0.41, "threshold": -0.977255642414093, "operator": "ge", "actual_fraction": 0.4099112002291607, "metric": 0.860068748209682, "rescued": 960, "harmed": 231, "missed_rescue": 489, "avoided_harm": 175, "selected_both_correct": 1498, "selected_both_wrong": 173}, {"budget": 0.42, "threshold": -0.9794421792030334, "operator": "ge", "actual_fraction": 0.41993698080779146, "metric": 0.8627900315095961, "rescued": 982, "harmed": 234, "missed_rescue": 467, "avoided_harm": 172, "selected_both_correct": 1539, "selected_both_wrong": 177}, {"budget": 0.43, "threshold": -0.981472373008728, "operator": "ge", "actual_fraction": 0.42996276138642225, "metric": 0.8642222858779719, "rescued": 998, "harmed": 240, "missed_rescue": 451, "avoided_harm": 166, "selected_both_correct": 1584, "selected_both_wrong": 180}, {"budget": 0.44, "threshold": -0.9835578799247742, "operator": "ge", "actual_fraction": 0.439988541965053, "metric": 0.8653680893726726, "rescued": 1014, "harmed": 248, "missed_rescue": 435, "avoided_harm": 158, "selected_both_correct": 1628, "selected_both_wrong": 182}, {"budget": 0.45, "threshold": -0.9851564168930054, "operator": "ge", "actual_fraction": 0.4498710971068462, "metric": 0.8668003437410484, "rescued": 1030, "harmed": 254, "missed_rescue": 419, "avoided_harm": 152, "selected_both_correct": 1673, "selected_both_wrong": 184}, {"budget": 0.46, "threshold": -0.986689031124115, "operator": "ge", "actual_fraction": 0.45989687768547693, "metric": 0.8686622744199369, "rescued": 1048, "harmed": 259, "missed_rescue": 401, "avoided_harm": 147, "selected_both_correct": 1719, "selected_both_wrong": 185}, {"budget": 0.47, "threshold": -0.9879001379013062, "operator": "ge", "actual_fraction": 0.46992265826410773, "metric": 0.8696648524778, "rescued": 1057, "harmed": 261, "missed_rescue": 392, "avoided_harm": 145, "selected_both_correct": 1775, "selected_both_wrong": 188}, {"budget": 0.48, "threshold": -0.9891784191131592, "operator": "ge", "actual_fraction": 0.47994843884273847, "metric": 0.8723861357777141, "rescued": 1077, "harmed": 262, "missed_rescue": 372, "avoided_harm": 144, "selected_both_correct": 1821, "selected_both_wrong": 191}, {"budget": 0.49, "threshold": -0.990114688873291, "operator": "ge", "actual_fraction": 0.48997421942136926, "metric": 0.8741048410197652, "rescued": 1092, "harmed": 265, "missed_rescue": 357, "avoided_harm": 141, "selected_both_correct": 1872, "selected_both_wrong": 192}, {"budget": 0.5, "threshold": -0.9910277724266052, "operator": "ge", "actual_fraction": 0.5, "metric": 0.8746777427671154, "rescued": 1103, "harmed": 272, "missed_rescue": 346, "avoided_harm": 134, "selected_both_correct": 1922, "selected_both_wrong": 194}, {"budget": 0.51, "threshold": -0.9918718934059143, "operator": "ge", "actual_fraction": 0.5098825551417931, "metric": 0.8749641936407906, "rescued": 1110, "harmed": 277, "missed_rescue": 339, "avoided_harm": 129, "selected_both_correct": 1977, "selected_both_wrong": 196}, {"budget": 0.52, "threshold": -0.9926809668540955, "operator": "ge", "actual_fraction": 0.5199083357204239, "metric": 0.8758235462618161, "rescued": 1120, "harmed": 281, "missed_rescue": 329, "avoided_harm": 125, "selected_both_correct": 2033, "selected_both_wrong": 196}, {"budget": 0.53, "threshold": -0.9932646751403809, "operator": "ge", "actual_fraction": 0.5299341162990547, "metric": 0.8762532225723288, "rescued": 1129, "harmed": 287, "missed_rescue": 320, "avoided_harm": 119, "selected_both_correct": 2086, "selected_both_wrong": 198}, {"budget": 0.54, "threshold": -0.9938417673110962, "operator": "ge", "actual_fraction": 0.5399598968776855, "metric": 0.8768261243196792, "rescued": 1136, "harmed": 290, "missed_rescue": 313, "avoided_harm": 116, "selected_both_correct": 2144, "selected_both_wrong": 200}, {"budget": 0.55, "threshold": -0.9944412112236023, "operator": "ge", "actual_fraction": 0.5499856774563162, "metric": 0.8778287023775423, "rescued": 1146, "harmed": 293, "missed_rescue": 303, "avoided_harm": 113, "selected_both_correct": 2200, "selected_both_wrong": 201}, {"budget": 0.56, "threshold": -0.9949513673782349, "operator": "ge", "actual_fraction": 0.5598682325981095, "metric": 0.878258378688055, "rescued": 1156, "harmed": 300, "missed_rescue": 293, "avoided_harm": 106, "selected_both_correct": 2249, "selected_both_wrong": 204}, {"budget": 0.57, "threshold": -0.9954326152801514, "operator": "ge", "actual_fraction": 0.5698940131767402, "metric": 0.8792609567459181, "rescued": 1170, "harmed": 307, "missed_rescue": 279, "avoided_harm": 99, "selected_both_correct": 2295, "selected_both_wrong": 207}, {"budget": 0.58, "threshold": -0.9958608746528625, "operator": "ge", "actual_fraction": 0.579919793755371, "metric": 0.8805499856774563, "rescued": 1182, "harmed": 310, "missed_rescue": 267, "avoided_harm": 96, "selected_both_correct": 2347, "selected_both_wrong": 210}, {"budget": 0.59, "threshold": -0.9962128400802612, "operator": "ge", "actual_fraction": 0.5899455743340017, "metric": 0.8819822400458321, "rescued": 1195, "harmed": 313, "missed_rescue": 254, "avoided_harm": 93, "selected_both_correct": 2399, "selected_both_wrong": 212}, {"budget": 0.6, "threshold": -0.9965764880180359, "operator": "ge", "actual_fraction": 0.5999713549126325, "metric": 0.8819822400458321, "rescued": 1202, "harmed": 320, "missed_rescue": 247, "avoided_harm": 86, "selected_both_correct": 2454, "selected_both_wrong": 213}, {"budget": 0.61, "threshold": -0.9968799352645874, "operator": "ge", "actual_fraction": 0.6099971354912632, "metric": 0.8832712689773704, "rescued": 1214, "harmed": 323, "missed_rescue": 235, "avoided_harm": 83, "selected_both_correct": 2507, "selected_both_wrong": 215}, {"budget": 0.62, "threshold": -0.9971383810043335, "operator": "ge", "actual_fraction": 0.6198796906330565, "metric": 0.8832712689773704, "rescued": 1218, "harmed": 327, "missed_rescue": 231, "avoided_harm": 79, "selected_both_correct": 2566, "selected_both_wrong": 217}, {"budget": 0.63, "threshold": -0.997377872467041, "operator": "ge", "actual_fraction": 0.6299054712116872, "metric": 0.8842738470352335, "rescued": 1228, "harmed": 330, "missed_rescue": 221, "avoided_harm": 76, "selected_both_correct": 2619, "selected_both_wrong": 221}, {"budget": 0.64, "threshold": -0.9976021647453308, "operator": "ge", "actual_fraction": 0.639931251790318, "metric": 0.886135777714122, "rescued": 1243, "harmed": 332, "missed_rescue": 206, "avoided_harm": 74, "selected_both_correct": 2672, "selected_both_wrong": 221}, {"budget": 0.65, "threshold": -0.9978214502334595, "operator": "ge", "actual_fraction": 0.6499570323689488, "metric": 0.8867086794614724, "rescued": 1249, "harmed": 334, "missed_rescue": 200, "avoided_harm": 72, "selected_both_correct": 2732, "selected_both_wrong": 223}, {"budget": 0.66, "threshold": -0.9980514049530029, "operator": "ge", "actual_fraction": 0.6599828129475795, "metric": 0.8882841592666858, "rescued": 1261, "harmed": 335, "missed_rescue": 188, "avoided_harm": 71, "selected_both_correct": 2789, "selected_both_wrong": 223}, {"budget": 0.67, "threshold": -0.9982016086578369, "operator": "ge", "actual_fraction": 0.6698653680893727, "metric": 0.889573188198224, "rescued": 1271, "harmed": 336, "missed_rescue": 178, "avoided_harm": 70, "selected_both_correct": 2847, "selected_both_wrong": 223}, {"budget": 0.68, "threshold": -0.9983693957328796, "operator": "ge", "actual_fraction": 0.6798911486680035, "metric": 0.8907189916929247, "rescued": 1280, "harmed": 337, "missed_rescue": 169, "avoided_harm": 69, "selected_both_correct": 2907, "selected_both_wrong": 223}, {"budget": 0.69, "threshold": -0.9984915256500244, "operator": "ge", "actual_fraction": 0.6899169292466342, "metric": 0.8921512460613005, "rescued": 1293, "harmed": 340, "missed_rescue": 156, "avoided_harm": 66, "selected_both_correct": 2960, "selected_both_wrong": 224}, {"budget": 0.7, "threshold": -0.9986109733581543, "operator": "gt", "actual_fraction": 0.6997994843884274, "metric": 0.8928673732454884, "rescued": 1299, "harmed": 341, "missed_rescue": 150, "avoided_harm": 65, "selected_both_correct": 3021, "selected_both_wrong": 225}, {"budget": 0.71, "threshold": -0.9986966252326965, "operator": "ge", "actual_fraction": 0.7099684904038958, "metric": 0.8931538241191636, "rescued": 1304, "harmed": 344, "missed_rescue": 145, "avoided_harm": 62, "selected_both_correct": 3084, "selected_both_wrong": 225}, {"budget": 0.72, "threshold": -0.9987963438034058, "operator": "ge", "actual_fraction": 0.7199942709825266, "metric": 0.8935835004296763, "rescued": 1310, "harmed": 347, "missed_rescue": 139, "avoided_harm": 59, "selected_both_correct": 3144, "selected_both_wrong": 226}, {"budget": 0.73, "threshold": -0.998875081539154, "operator": "ge", "actual_fraction": 0.7298768261243197, "metric": 0.8940131767401891, "rescued": 1318, "harmed": 352, "missed_rescue": 131, "avoided_harm": 54, "selected_both_correct": 3199, "selected_both_wrong": 227}, {"budget": 0.74, "threshold": -0.9989551305770874, "operator": "ge", "actual_fraction": 0.7399026067029505, "metric": 0.8951589802348897, "rescued": 1327, "harmed": 353, "missed_rescue": 122, "avoided_harm": 53, "selected_both_correct": 3257, "selected_both_wrong": 229}, {"budget": 0.75, "threshold": -0.9990480542182922, "operator": "ge", "actual_fraction": 0.7499283872815812, "metric": 0.8961615582927528, "rescued": 1336, "harmed": 355, "missed_rescue": 113, "avoided_harm": 51, "selected_both_correct": 3314, "selected_both_wrong": 231}, {"budget": 0.76, "threshold": -0.999126136302948, "operator": "ge", "actual_fraction": 0.759954167860212, "metric": 0.8973073617874534, "rescued": 1345, "harmed": 356, "missed_rescue": 104, "avoided_harm": 50, "selected_both_correct": 3374, "selected_both_wrong": 231}, {"budget": 0.77, "threshold": -0.999184787273407, "operator": "gt", "actual_fraction": 0.7698367230020051, "metric": 0.8977370380979662, "rescued": 1350, "harmed": 358, "missed_rescue": 99, "avoided_harm": 48, "selected_both_correct": 3435, "selected_both_wrong": 232}, {"budget": 0.78, "threshold": -0.9992268085479736, "operator": "ge", "actual_fraction": 0.7798625035806359, "metric": 0.8987396161558293, "rescued": 1359, "harmed": 360, "missed_rescue": 90, "avoided_harm": 46, "selected_both_correct": 3492, "selected_both_wrong": 234}, {"budget": 0.79, "threshold": -0.9992831349372864, "operator": "ge", "actual_fraction": 0.7898882841592667, "metric": 0.8997421942136924, "rescued": 1370, "harmed": 364, "missed_rescue": 79, "avoided_harm": 42, "selected_both_correct": 3545, "selected_both_wrong": 236}, {"budget": 0.8, "threshold": -0.9993318915367126, "operator": "ge", "actual_fraction": 0.7999140647378975, "metric": 0.8997421942136924, "rescued": 1372, "harmed": 366, "missed_rescue": 77, "avoided_harm": 40, "selected_both_correct": 3611, "selected_both_wrong": 236}, {"budget": 0.81, "threshold": -0.9993773102760315, "operator": "ge", "actual_fraction": 0.8099398453165282, "metric": 0.9000286450873675, "rescued": 1376, "harmed": 368, "missed_rescue": 73, "avoided_harm": 38, "selected_both_correct": 3674, "selected_both_wrong": 237}, {"budget": 0.82, "threshold": -0.9994175434112549, "operator": "ge", "actual_fraction": 0.819965625895159, "metric": 0.9003150959610426, "rescued": 1381, "harmed": 371, "missed_rescue": 68, "avoided_harm": 35, "selected_both_correct": 3734, "selected_both_wrong": 239}, {"budget": 0.83, "threshold": -0.999452531337738, "operator": "ge", "actual_fraction": 0.8299914064737898, "metric": 0.9003150959610426, "rescued": 1383, "harmed": 373, "missed_rescue": 66, "avoided_harm": 33, "selected_both_correct": 3798, "selected_both_wrong": 241}, {"budget": 0.84, "threshold": -0.9994945526123047, "operator": "ge", "actual_fraction": 0.8398739616155829, "metric": 0.9011744485820682, "rescued": 1389, "harmed": 373, "missed_rescue": 60, "avoided_harm": 33, "selected_both_correct": 3861, "selected_both_wrong": 241}, {"budget": 0.85, "threshold": -0.9995274543762207, "operator": "gt", "actual_fraction": 0.8497565167573761, "metric": 0.9016041248925809, "rescued": 1394, "harmed": 375, "missed_rescue": 55, "avoided_harm": 31, "selected_both_correct": 3922, "selected_both_wrong": 242}, {"budget": 0.86, "threshold": -0.9995617270469666, "operator": "ge", "actual_fraction": 0.8599255227728445, "metric": 0.902606702950444, "rescued": 1401, "harmed": 375, "missed_rescue": 48, "avoided_harm": 31, "selected_both_correct": 3985, "selected_both_wrong": 243}, {"budget": 0.87, "threshold": -0.9995949864387512, "operator": "ge", "actual_fraction": 0.8699513033514752, "metric": 0.9027499283872816, "rescued": 1407, "harmed": 380, "missed_rescue": 42, "avoided_harm": 26, "selected_both_correct": 4043, "selected_both_wrong": 244}, {"budget": 0.88, "threshold": -0.9996248483657837, "operator": "ge", "actual_fraction": 0.879977083930106, "metric": 0.9031796046977943, "rescued": 1412, "harmed": 382, "missed_rescue": 37, "avoided_harm": 24, "selected_both_correct": 4106, "selected_both_wrong": 244}, {"budget": 0.89, "threshold": -0.9996511936187744, "operator": "ge", "actual_fraction": 0.8898596390718991, "metric": 0.9034660555714695, "rescued": 1419, "harmed": 387, "missed_rescue": 30, "avoided_harm": 19, "selected_both_correct": 4161, "selected_both_wrong": 246}, {"budget": 0.9, "threshold": -0.9996830224990845, "operator": "ge", "actual_fraction": 0.8998854196505299, "metric": 0.9038957318819822, "rescued": 1423, "harmed": 388, "missed_rescue": 26, "avoided_harm": 18, "selected_both_correct": 4225, "selected_both_wrong": 247}, {"budget": 0.91, "threshold": -0.9997097849845886, "operator": "gt", "actual_fraction": 0.9097679747923231, "metric": 0.9046118590661701, "rescued": 1431, "harmed": 391, "missed_rescue": 18, "avoided_harm": 15, "selected_both_correct": 4283, "selected_both_wrong": 247}, {"budget": 0.92, "threshold": -0.9997391104698181, "operator": "ge", "actual_fraction": 0.9199369808077915, "metric": 0.9050415353766829, "rescued": 1434, "harmed": 391, "missed_rescue": 15, "avoided_harm": 15, "selected_both_correct": 4350, "selected_both_wrong": 248}, {"budget": 0.93, "threshold": -0.9997641444206238, "operator": "ge", "actual_fraction": 0.9299627613864222, "metric": 0.9051847608135205, "rescued": 1437, "harmed": 393, "missed_rescue": 12, "avoided_harm": 13, "selected_both_correct": 4411, "selected_both_wrong": 252}, {"budget": 0.94, "threshold": -0.9997876286506653, "operator": "ge", "actual_fraction": 0.939988541965053, "metric": 0.905327986250358, "rescued": 1439, "harmed": 394, "missed_rescue": 10, "avoided_harm": 12, "selected_both_correct": 4476, "selected_both_wrong": 254}, {"budget": 0.95, "threshold": -0.9998131394386292, "operator": "ge", "actual_fraction": 0.9498710971068461, "metric": 0.9056144371240332, "rescued": 1442, "harmed": 395, "missed_rescue": 7, "avoided_harm": 11, "selected_both_correct": 4540, "selected_both_wrong": 255}, {"budget": 0.96, "threshold": -0.9998401403427124, "operator": "ge", "actual_fraction": 0.9598968776854769, "metric": 0.9059008879977084, "rescued": 1446, "harmed": 397, "missed_rescue": 3, "avoided_harm": 9, "selected_both_correct": 4603, "selected_both_wrong": 256}, {"budget": 0.97, "threshold": -0.999868631362915, "operator": "ge", "actual_fraction": 0.9699226582641077, "metric": 0.9054712116871957, "rescued": 1446, "harmed": 400, "missed_rescue": 3, "avoided_harm": 6, "selected_both_correct": 4670, "selected_both_wrong": 256}, {"budget": 0.98, "threshold": -0.9998925924301147, "operator": "ge", "actual_fraction": 0.9799484388427385, "metric": 0.9051847608135205, "rescued": 1446, "harmed": 402, "missed_rescue": 3, "avoided_harm": 4, "selected_both_correct": 4738, "selected_both_wrong": 256}, {"budget": 0.99, "threshold": -0.99992835521698, "operator": "ge", "actual_fraction": 0.9899742194213692, "metric": 0.9047550845030078, "rescued": 1446, "harmed": 405, "missed_rescue": 3, "avoided_harm": 1, "selected_both_correct": 4805, "selected_both_wrong": 256}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.9050415353766829, "rescued": 1449, "harmed": 406, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 4870, "selected_both_wrong": 257}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.7642247255506924, "rescued": 0, "harmed": 0, "missed_rescue": 2791, "avoided_harm": 890, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.5127503275871277, "operator": "ge", "actual_fraction": 0.007749156920427639, "metric": 0.7660185118748655, "rescued": 36, "harmed": 11, "missed_rescue": 2755, "avoided_harm": 879, "selected_both_correct": 57, "selected_both_wrong": 4}, {"budget": 0.02, "threshold": -0.5296345353126526, "operator": "ge", "actual_fraction": 0.017650857429862953, "metric": 0.7696778359761786, "rescued": 96, "harmed": 20, "missed_rescue": 2695, "avoided_harm": 870, "selected_both_correct": 120, "selected_both_wrong": 10}, {"budget": 0.03, "threshold": -0.5434877276420593, "operator": "ge", "actual_fraction": 0.025113008538422903, "metric": 0.7726196455478224, "rescued": 144, "harmed": 27, "missed_rescue": 2647, "avoided_harm": 863, "selected_both_correct": 161, "selected_both_wrong": 18}, {"budget": 0.04, "threshold": -0.56125408411026, "operator": "ge", "actual_fraction": 0.0349429575948913, "metric": 0.776565975461003, "rescued": 205, "harmed": 33, "missed_rescue": 2586, "avoided_harm": 857, "selected_both_correct": 226, "selected_both_wrong": 23}, {"budget": 0.05, "threshold": -0.5779195427894592, "operator": "ge", "actual_fraction": 0.043840137762789694, "metric": 0.7785750161440769, "rescued": 242, "harmed": 42, "missed_rescue": 2549, "avoided_harm": 848, "selected_both_correct": 291, "selected_both_wrong": 36}, {"budget": 0.06, "threshold": -0.5911363363265991, "operator": "ge", "actual_fraction": 0.05187630049508503, "metric": 0.7809428140919854, "rescued": 286, "harmed": 53, "missed_rescue": 2505, "avoided_harm": 837, "selected_both_correct": 345, "selected_both_wrong": 39}, {"budget": 0.07, "threshold": -0.6087464094161987, "operator": "ge", "actual_fraction": 0.06177800100452034, "metric": 0.7835976178517615, "rescued": 336, "harmed": 66, "missed_rescue": 2455, "avoided_harm": 824, "selected_both_correct": 409, "selected_both_wrong": 50}, {"budget": 0.08, "threshold": -0.6274068355560303, "operator": "ge", "actual_fraction": 0.07132094424912104, "metric": 0.7864676759704384, "rescued": 387, "harmed": 77, "missed_rescue": 2404, "avoided_harm": 813, "selected_both_correct": 473, "selected_both_wrong": 57}, {"budget": 0.09, "threshold": -0.6465112566947937, "operator": "ge", "actual_fraction": 0.08251417091196096, "metric": 0.7888354739183469, "rescued": 439, "harmed": 96, "missed_rescue": 2352, "avoided_harm": 794, "selected_both_correct": 546, "selected_both_wrong": 69}, {"budget": 0.1, "threshold": -0.6620025038719177, "operator": "ge", "actual_fraction": 0.09090909090909091, "metric": 0.7910597689603215, "rescued": 483, "harmed": 109, "missed_rescue": 2308, "avoided_harm": 781, "selected_both_correct": 601, "selected_both_wrong": 74}, {"budget": 0.11, "threshold": -0.6800539493560791, "operator": "ge", "actual_fraction": 0.10030853124775777, "metric": 0.7934275669082299, "rescued": 529, "harmed": 122, "missed_rescue": 2262, "avoided_harm": 768, "selected_both_correct": 661, "selected_both_wrong": 86}, {"budget": 0.12, "threshold": -0.6957284212112427, "operator": "ge", "actual_fraction": 0.11078424338092846, "metric": 0.7971586424625099, "rescued": 592, "harmed": 133, "missed_rescue": 2199, "avoided_harm": 757, "selected_both_correct": 723, "selected_both_wrong": 96}, {"budget": 0.13, "threshold": -0.7128273248672485, "operator": "ge", "actual_fraction": 0.12075769534333071, "metric": 0.7997416947693191, "rescued": 640, "harmed": 145, "missed_rescue": 2151, "avoided_harm": 745, "selected_both_correct": 792, "selected_both_wrong": 106}, {"budget": 0.14, "threshold": -0.7332167625427246, "operator": "ge", "actual_fraction": 0.13431871995407907, "metric": 0.8044055392121691, "rescued": 720, "harmed": 160, "missed_rescue": 2071, "avoided_harm": 730, "selected_both_correct": 870, "selected_both_wrong": 122}, {"budget": 0.15, "threshold": -0.7503995895385742, "operator": "ge", "actual_fraction": 0.144579177728349, "metric": 0.8072038458778791, "rescued": 769, "harmed": 170, "missed_rescue": 2022, "avoided_harm": 720, "selected_both_correct": 947, "selected_both_wrong": 129}, {"budget": 0.16, "threshold": -0.7655540704727173, "operator": "ge", "actual_fraction": 0.1534763578962474, "metric": 0.8099304010906221, "rescued": 817, "harmed": 180, "missed_rescue": 1974, "avoided_harm": 710, "selected_both_correct": 1002, "selected_both_wrong": 140}, {"budget": 0.17, "threshold": -0.7812193036079407, "operator": "ge", "actual_fraction": 0.16280404678194735, "metric": 0.8125852048503982, "rescued": 868, "harmed": 194, "missed_rescue": 1923, "avoided_harm": 696, "selected_both_correct": 1059, "selected_both_wrong": 148}, {"budget": 0.18, "threshold": -0.7954917550086975, "operator": "ge", "actual_fraction": 0.17306450455621727, "metric": 0.8159575231398436, "rescued": 927, "harmed": 206, "missed_rescue": 1864, "avoided_harm": 684, "selected_both_correct": 1124, "selected_both_wrong": 155}, {"budget": 0.19, "threshold": -0.8097342252731323, "operator": "ge", "actual_fraction": 0.18196168472411567, "metric": 0.8179665638229174, "rescued": 966, "harmed": 217, "missed_rescue": 1825, "avoided_harm": 673, "selected_both_correct": 1191, "selected_both_wrong": 162}, {"budget": 0.2, "threshold": -0.8232117891311646, "operator": "ge", "actual_fraction": 0.1928679055750879, "metric": 0.820262610317859, "rescued": 1011, "harmed": 230, "missed_rescue": 1780, "avoided_harm": 660, "selected_both_correct": 1276, "selected_both_wrong": 171}, {"budget": 0.21, "threshold": -0.8359466791152954, "operator": "ge", "actual_fraction": 0.20104757121331707, "metric": 0.8221998995479658, "rescued": 1049, "harmed": 241, "missed_rescue": 1742, "avoided_harm": 649, "selected_both_correct": 1333, "selected_both_wrong": 179}, {"budget": 0.22, "threshold": -0.8473842144012451, "operator": "ge", "actual_fraction": 0.21051876300495084, "metric": 0.8254287149314774, "rescued": 1098, "harmed": 245, "missed_rescue": 1693, "avoided_harm": 645, "selected_both_correct": 1403, "selected_both_wrong": 188}, {"budget": 0.23, "threshold": -0.8595863580703735, "operator": "ge", "actual_fraction": 0.21998995479658462, "metric": 0.8272225012556504, "rescued": 1135, "harmed": 257, "missed_rescue": 1656, "avoided_harm": 633, "selected_both_correct": 1478, "selected_both_wrong": 196}, {"budget": 0.24, "threshold": -0.8704345226287842, "operator": "ge", "actual_fraction": 0.22838487479371458, "metric": 0.8280117672382866, "rescued": 1161, "harmed": 272, "missed_rescue": 1630, "avoided_harm": 618, "selected_both_correct": 1544, "selected_both_wrong": 206}, {"budget": 0.25, "threshold": -0.8811548352241516, "operator": "ge", "actual_fraction": 0.23900408983281912, "metric": 0.8303078137332282, "rescued": 1207, "harmed": 286, "missed_rescue": 1584, "avoided_harm": 604, "selected_both_correct": 1623, "selected_both_wrong": 215}, {"budget": 0.26, "threshold": -0.8929013609886169, "operator": "ge", "actual_fraction": 0.2506995766664275, "metric": 0.8333931262108057, "rescued": 1260, "harmed": 296, "missed_rescue": 1531, "avoided_harm": 594, "selected_both_correct": 1712, "selected_both_wrong": 226}, {"budget": 0.27, "threshold": -0.903549313545227, "operator": "ge", "actual_fraction": 0.2646911099949774, "metric": 0.8365501901413503, "rescued": 1318, "harmed": 310, "missed_rescue": 1473, "avoided_harm": 580, "selected_both_correct": 1828, "selected_both_wrong": 233}, {"budget": 0.28, "threshold": -0.9115740656852722, "operator": "ge", "actual_fraction": 0.27437755614551196, "metric": 0.8389897395422258, "rescued": 1366, "harmed": 324, "missed_rescue": 1425, "avoided_harm": 566, "selected_both_correct": 1899, "selected_both_wrong": 235}, {"budget": 0.29, "threshold": -0.9182595014572144, "operator": "ge", "actual_fraction": 0.2819832101600057, "metric": 0.8409987802252996, "rescued": 1401, "harmed": 331, "missed_rescue": 1390, "avoided_harm": 559, "selected_both_correct": 1955, "selected_both_wrong": 243}, {"budget": 0.3, "threshold": -0.9258768558502197, "operator": "ge", "actual_fraction": 0.2907368874219703, "metric": 0.8434383296261749, "rescued": 1446, "harmed": 342, "missed_rescue": 1345, "avoided_harm": 548, "selected_both_correct": 2015, "selected_both_wrong": 249}, {"budget": 0.31, "threshold": -0.9331877827644348, "operator": "ge", "actual_fraction": 0.302145368443711, "metric": 0.8450168615914472, "rescued": 1488, "harmed": 362, "missed_rescue": 1303, "avoided_harm": 528, "selected_both_correct": 2106, "selected_both_wrong": 255}, {"budget": 0.32, "threshold": -0.9399217367172241, "operator": "ge", "actual_fraction": 0.31412786108918705, "metric": 0.847025902274521, "rescued": 1531, "harmed": 377, "missed_rescue": 1260, "avoided_harm": 513, "selected_both_correct": 2209, "selected_both_wrong": 261}, {"budget": 0.33, "threshold": -0.9453923106193542, "operator": "ge", "actual_fraction": 0.3238143072397216, "metric": 0.8486761856927603, "rescued": 1564, "harmed": 387, "missed_rescue": 1227, "avoided_harm": 503, "selected_both_correct": 2293, "selected_both_wrong": 269}, {"budget": 0.34, "threshold": -0.9515207409858704, "operator": "ge", "actual_fraction": 0.33529453971442924, "metric": 0.8514744923584703, "rescued": 1615, "harmed": 399, "missed_rescue": 1176, "avoided_harm": 491, "selected_both_correct": 2385, "selected_both_wrong": 274}, {"budget": 0.35, "threshold": -0.9567410349845886, "operator": "ge", "actual_fraction": 0.3452679916768315, "metric": 0.8531247757767095, "rescued": 1654, "harmed": 415, "missed_rescue": 1137, "avoided_harm": 475, "selected_both_correct": 2465, "selected_both_wrong": 278}, {"budget": 0.36, "threshold": -0.9606627225875854, "operator": "ge", "actual_fraction": 0.35595895816890294, "metric": 0.855420822271651, "rescued": 1697, "harmed": 426, "missed_rescue": 1094, "avoided_harm": 464, "selected_both_correct": 2554, "selected_both_wrong": 284}, {"budget": 0.37, "threshold": -0.9650874733924866, "operator": "ge", "actual_fraction": 0.36837195953218055, "metric": 0.8573581115017579, "rescued": 1745, "harmed": 447, "missed_rescue": 1046, "avoided_harm": 443, "selected_both_correct": 2651, "selected_both_wrong": 291}, {"budget": 0.38, "threshold": -0.9686882495880127, "operator": "ge", "actual_fraction": 0.3797086891009543, "metric": 0.8592954007318648, "rescued": 1785, "harmed": 460, "missed_rescue": 1006, "avoided_harm": 430, "selected_both_correct": 2747, "selected_both_wrong": 300}, {"budget": 0.39, "threshold": -0.971785306930542, "operator": "ge", "actual_fraction": 0.391117170122695, "metric": 0.8620219559446078, "rescued": 1832, "harmed": 469, "missed_rescue": 959, "avoided_harm": 421, "selected_both_correct": 2845, "selected_both_wrong": 305}, {"budget": 0.4, "threshold": -0.9748144745826721, "operator": "ge", "actual_fraction": 0.40231039678553493, "metric": 0.8641744995336156, "rescued": 1873, "harmed": 480, "missed_rescue": 918, "avoided_harm": 410, "selected_both_correct": 2945, "selected_both_wrong": 309}, {"budget": 0.41, "threshold": -0.977255642414093, "operator": "ge", "actual_fraction": 0.41235560020090406, "metric": 0.8662552916696563, "rescued": 1912, "harmed": 490, "missed_rescue": 879, "avoided_harm": 400, "selected_both_correct": 3027, "selected_both_wrong": 318}, {"budget": 0.42, "threshold": -0.9794421792030334, "operator": "ge", "actual_fraction": 0.4231183181459425, "metric": 0.8680490779938294, "rescued": 1950, "harmed": 503, "missed_rescue": 841, "avoided_harm": 387, "selected_both_correct": 3119, "selected_both_wrong": 325}, {"budget": 0.43, "threshold": -0.981472373008728, "operator": "ge", "actual_fraction": 0.43172849250197315, "metric": 0.869268852694267, "rescued": 1972, "harmed": 508, "missed_rescue": 819, "avoided_harm": 382, "selected_both_correct": 3207, "selected_both_wrong": 330}, {"budget": 0.44, "threshold": -0.9835578799247742, "operator": "ge", "actual_fraction": 0.4432804764296477, "metric": 0.8709191361125063, "rescued": 2005, "harmed": 518, "missed_rescue": 786, "avoided_harm": 372, "selected_both_correct": 3320, "selected_both_wrong": 335}, {"budget": 0.45, "threshold": -0.9851564168930054, "operator": "ge", "actual_fraction": 0.4547607089043553, "metric": 0.8727129224366793, "rescued": 2042, "harmed": 530, "missed_rescue": 749, "avoided_harm": 360, "selected_both_correct": 3422, "selected_both_wrong": 344}, {"budget": 0.46, "threshold": -0.986689031124115, "operator": "ge", "actual_fraction": 0.4649494152256583, "metric": 0.8740762000430509, "rescued": 2074, "harmed": 543, "missed_rescue": 717, "avoided_harm": 347, "selected_both_correct": 3513, "selected_both_wrong": 350}, {"budget": 0.47, "threshold": -0.9879001379013062, "operator": "ge", "actual_fraction": 0.47399009829949057, "metric": 0.8749372174786539, "rescued": 2096, "harmed": 553, "missed_rescue": 695, "avoided_harm": 337, "selected_both_correct": 3603, "selected_both_wrong": 354}, {"budget": 0.48, "threshold": -0.9891784191131592, "operator": "ge", "actual_fraction": 0.48446581043266124, "metric": 0.8764439979909593, "rescued": 2126, "harmed": 562, "missed_rescue": 665, "avoided_harm": 328, "selected_both_correct": 3704, "selected_both_wrong": 360}, {"budget": 0.49, "threshold": -0.990114688873291, "operator": "ge", "actual_fraction": 0.491210447011552, "metric": 0.8771615125206286, "rescued": 2145, "harmed": 571, "missed_rescue": 646, "avoided_harm": 319, "selected_both_correct": 3767, "selected_both_wrong": 363}, {"budget": 0.5, "threshold": -0.9910277724266052, "operator": "ge", "actual_fraction": 0.5008251417091196, "metric": 0.8780942814091985, "rescued": 2168, "harmed": 581, "missed_rescue": 623, "avoided_harm": 309, "selected_both_correct": 3859, "selected_both_wrong": 372}, {"budget": 0.51, "threshold": -0.9918718934059143, "operator": "ge", "actual_fraction": 0.5107268422185549, "metric": 0.878596541579967, "rescued": 2183, "harmed": 589, "missed_rescue": 608, "avoided_harm": 301, "selected_both_correct": 3969, "selected_both_wrong": 377}, {"budget": 0.52, "threshold": -0.9926809668540955, "operator": "ge", "actual_fraction": 0.5217048145224941, "metric": 0.8796010619215039, "rescued": 2208, "harmed": 600, "missed_rescue": 583, "avoided_harm": 290, "selected_both_correct": 4083, "selected_both_wrong": 380}, {"budget": 0.53, "threshold": -0.9932646751403809, "operator": "ge", "actual_fraction": 0.5306737461433594, "metric": 0.8807490851689747, "rescued": 2231, "harmed": 607, "missed_rescue": 560, "avoided_harm": 283, "selected_both_correct": 4172, "selected_both_wrong": 386}, {"budget": 0.54, "threshold": -0.9938417673110962, "operator": "ge", "actual_fraction": 0.5398579321231255, "metric": 0.8816101026045777, "rescued": 2253, "harmed": 617, "missed_rescue": 538, "avoided_harm": 273, "selected_both_correct": 4267, "selected_both_wrong": 387}, {"budget": 0.55, "threshold": -0.9944412112236023, "operator": "ge", "actual_fraction": 0.5504771471622301, "metric": 0.882327617134247, "rescued": 2273, "harmed": 627, "missed_rescue": 518, "avoided_harm": 263, "selected_both_correct": 4379, "selected_both_wrong": 393}, {"budget": 0.56, "threshold": -0.9949513673782349, "operator": "ge", "actual_fraction": 0.5604505991246322, "metric": 0.8834756403817178, "rescued": 2298, "harmed": 636, "missed_rescue": 493, "avoided_harm": 254, "selected_both_correct": 4479, "selected_both_wrong": 398}, {"budget": 0.57, "threshold": -0.9954326152801514, "operator": "ge", "actual_fraction": 0.570926311257803, "metric": 0.8845519121762215, "rescued": 2321, "harmed": 644, "missed_rescue": 470, "avoided_harm": 246, "selected_both_correct": 4592, "selected_both_wrong": 400}, {"budget": 0.58, "threshold": -0.9958608746528625, "operator": "ge", "actual_fraction": 0.5816172777498745, "metric": 0.8858434383296262, "rescued": 2349, "harmed": 654, "missed_rescue": 442, "avoided_harm": 236, "selected_both_correct": 4698, "selected_both_wrong": 405}, {"budget": 0.59, "threshold": -0.9962128400802612, "operator": "ge", "actual_fraction": 0.5920929898830452, "metric": 0.8866327043122624, "rescued": 2374, "harmed": 668, "missed_rescue": 417, "avoided_harm": 222, "selected_both_correct": 4804, "selected_both_wrong": 406}, {"budget": 0.6, "threshold": -0.9965764880180359, "operator": "ge", "actual_fraction": 0.6034297194518189, "metric": 0.8874219702948984, "rescued": 2395, "harmed": 678, "missed_rescue": 396, "avoided_harm": 212, "selected_both_correct": 4930, "selected_both_wrong": 407}, {"budget": 0.61, "threshold": -0.9968799352645874, "operator": "ge", "actual_fraction": 0.6144794432087249, "metric": 0.8879242304656669, "rescued": 2415, "harmed": 691, "missed_rescue": 376, "avoided_harm": 199, "selected_both_correct": 5048, "selected_both_wrong": 410}, {"budget": 0.62, "threshold": -0.9971383810043335, "operator": "ge", "actual_fraction": 0.6232331204706896, "metric": 0.8888569993542369, "rescued": 2435, "harmed": 698, "missed_rescue": 356, "avoided_harm": 192, "selected_both_correct": 5140, "selected_both_wrong": 413}, {"budget": 0.63, "threshold": -0.997377872467041, "operator": "ge", "actual_fraction": 0.6337805840568271, "metric": 0.8894310109779723, "rescued": 2451, "harmed": 706, "missed_rescue": 340, "avoided_harm": 184, "selected_both_correct": 5258, "selected_both_wrong": 418}, {"budget": 0.64, "threshold": -0.9976021647453308, "operator": "ge", "actual_fraction": 0.6445433020018655, "metric": 0.8904355313195093, "rescued": 2472, "harmed": 713, "missed_rescue": 319, "avoided_harm": 177, "selected_both_correct": 5375, "selected_both_wrong": 423}, {"budget": 0.65, "threshold": -0.9978214502334595, "operator": "ge", "actual_fraction": 0.6565257946473416, "metric": 0.8904355313195093, "rescued": 2488, "harmed": 729, "missed_rescue": 303, "avoided_harm": 161, "selected_both_correct": 5508, "selected_both_wrong": 425}, {"budget": 0.66, "threshold": -0.9980514049530029, "operator": "ge", "actual_fraction": 0.6692258018224869, "metric": 0.8909377914902776, "rescued": 2507, "harmed": 741, "missed_rescue": 284, "avoided_harm": 149, "selected_both_correct": 5650, "selected_both_wrong": 429}, {"budget": 0.67, "threshold": -0.9982016086578369, "operator": "ge", "actual_fraction": 0.6777642247255506, "metric": 0.8917988089258807, "rescued": 2523, "harmed": 745, "missed_rescue": 268, "avoided_harm": 145, "selected_both_correct": 5748, "selected_both_wrong": 430}, {"budget": 0.68, "threshold": -0.9983693957328796, "operator": "ge", "actual_fraction": 0.6885269426705891, "metric": 0.8920858147377484, "rescued": 2541, "harmed": 759, "missed_rescue": 250, "avoided_harm": 131, "selected_both_correct": 5864, "selected_both_wrong": 432}, {"budget": 0.69, "threshold": -0.9984915256500244, "operator": "ge", "actual_fraction": 0.6977828801033221, "metric": 0.89251632345555, "rescued": 2552, "harmed": 764, "missed_rescue": 239, "avoided_harm": 126, "selected_both_correct": 5973, "selected_both_wrong": 436}, {"budget": 0.7, "threshold": -0.9986109733581543, "operator": "gt", "actual_fraction": 0.7064648059123197, "metric": 0.8928750807203846, "rescued": 2563, "harmed": 770, "missed_rescue": 228, "avoided_harm": 120, "selected_both_correct": 6077, "selected_both_wrong": 436}, {"budget": 0.71, "threshold": -0.9986966252326965, "operator": "ge", "actual_fraction": 0.7137117026619789, "metric": 0.8933055894381862, "rescued": 2575, "harmed": 776, "missed_rescue": 216, "avoided_harm": 114, "selected_both_correct": 6154, "selected_both_wrong": 442}, {"budget": 0.72, "threshold": -0.9987963438034058, "operator": "ge", "actual_fraction": 0.7256941953074549, "metric": 0.8941666068737892, "rescued": 2592, "harmed": 781, "missed_rescue": 199, "avoided_harm": 109, "selected_both_correct": 6295, "selected_both_wrong": 446}, {"budget": 0.73, "threshold": -0.998875081539154, "operator": "ge", "actual_fraction": 0.7345196240223865, "metric": 0.8950993757623592, "rescued": 2605, "harmed": 781, "missed_rescue": 186, "avoided_harm": 109, "selected_both_correct": 6403, "selected_both_wrong": 448}, {"budget": 0.74, "threshold": -0.9989551305770874, "operator": "ge", "actual_fraction": 0.745067087608524, "metric": 0.8956733873860946, "rescued": 2618, "harmed": 786, "missed_rescue": 173, "avoided_harm": 104, "selected_both_correct": 6530, "selected_both_wrong": 450}, {"budget": 0.75, "threshold": -0.9990480542182922, "operator": "ge", "actual_fraction": 0.7568343258950994, "metric": 0.8963191504627969, "rescued": 2632, "harmed": 791, "missed_rescue": 159, "avoided_harm": 99, "selected_both_correct": 6671, "selected_both_wrong": 454}, {"budget": 0.76, "threshold": -0.999126136302948, "operator": "ge", "actual_fraction": 0.7663772691397001, "metric": 0.8967496591805985, "rescued": 2644, "harmed": 797, "missed_rescue": 147, "avoided_harm": 93, "selected_both_correct": 6783, "selected_both_wrong": 457}, {"budget": 0.77, "threshold": -0.999184787273407, "operator": "gt", "actual_fraction": 0.7746286862308962, "metric": 0.8969649135394991, "rescued": 2653, "harmed": 803, "missed_rescue": 138, "avoided_harm": 87, "selected_both_correct": 6883, "selected_both_wrong": 457}, {"budget": 0.78, "threshold": -0.9992268085479736, "operator": "ge", "actual_fraction": 0.7817320800746215, "metric": 0.8966061562746646, "rescued": 2657, "harmed": 812, "missed_rescue": 134, "avoided_harm": 78, "selected_both_correct": 6969, "selected_both_wrong": 457}, {"budget": 0.79, "threshold": -0.9992831349372864, "operator": "ge", "actual_fraction": 0.7928535552844945, "metric": 0.8967496591805985, "rescued": 2667, "harmed": 820, "missed_rescue": 124, "avoided_harm": 70, "selected_both_correct": 7106, "selected_both_wrong": 457}, {"budget": 0.8, "threshold": -0.9993318915367126, "operator": "ge", "actual_fraction": 0.8023247470761283, "metric": 0.8966061562746646, "rescued": 2669, "harmed": 824, "missed_rescue": 122, "avoided_harm": 66, "selected_both_correct": 7231, "selected_both_wrong": 458}, {"budget": 0.81, "threshold": -0.9993773102760315, "operator": "ge", "actual_fraction": 0.811795938867762, "metric": 0.8968214106335654, "rescued": 2677, "harmed": 829, "missed_rescue": 114, "avoided_harm": 61, "selected_both_correct": 7349, "selected_both_wrong": 459}, {"budget": 0.82, "threshold": -0.9994175434112549, "operator": "ge", "actual_fraction": 0.8214106335653297, "metric": 0.8973954222573007, "rescued": 2689, "harmed": 833, "missed_rescue": 102, "avoided_harm": 57, "selected_both_correct": 7465, "selected_both_wrong": 461}, {"budget": 0.83, "threshold": -0.999452531337738, "operator": "ge", "actual_fraction": 0.8298773050154266, "metric": 0.8976824280691684, "rescued": 2696, "harmed": 836, "missed_rescue": 95, "avoided_harm": 54, "selected_both_correct": 7572, "selected_both_wrong": 462}, {"budget": 0.84, "threshold": -0.9994945526123047, "operator": "ge", "actual_fraction": 0.8420750520198034, "metric": 0.8973954222573007, "rescued": 2702, "harmed": 846, "missed_rescue": 89, "avoided_harm": 44, "selected_both_correct": 7725, "selected_both_wrong": 463}, {"budget": 0.85, "threshold": -0.9995274543762207, "operator": "gt", "actual_fraction": 0.8502547176580326, "metric": 0.8976106766162015, "rescued": 2707, "harmed": 848, "missed_rescue": 84, "avoided_harm": 42, "selected_both_correct": 7829, "selected_both_wrong": 466}, {"budget": 0.86, "threshold": -0.9995617270469666, "operator": "ge", "actual_fraction": 0.8604434239793356, "metric": 0.8976824280691684, "rescued": 2712, "harmed": 852, "missed_rescue": 79, "avoided_harm": 38, "selected_both_correct": 7959, "selected_both_wrong": 469}, {"budget": 0.87, "threshold": -0.9995949864387512, "operator": "ge", "actual_fraction": 0.8690535983353663, "metric": 0.8978259309751022, "rescued": 2717, "harmed": 855, "missed_rescue": 74, "avoided_harm": 35, "selected_both_correct": 8069, "selected_both_wrong": 471}, {"budget": 0.88, "threshold": -0.9996248483657837, "operator": "ge", "actual_fraction": 0.8778790270502977, "metric": 0.8978976824280692, "rescued": 2723, "harmed": 860, "missed_rescue": 68, "avoided_harm": 30, "selected_both_correct": 8180, "selected_both_wrong": 472}, {"budget": 0.89, "threshold": -0.9996511936187744, "operator": "ge", "actual_fraction": 0.886058692688527, "metric": 0.8981129367869699, "rescued": 2731, "harmed": 865, "missed_rescue": 60, "avoided_harm": 25, "selected_both_correct": 8279, "selected_both_wrong": 474}, {"budget": 0.9, "threshold": -0.9996830224990845, "operator": "ge", "actual_fraction": 0.8961756475568631, "metric": 0.8986869484107053, "rescued": 2740, "harmed": 866, "missed_rescue": 51, "avoided_harm": 24, "selected_both_correct": 8407, "selected_both_wrong": 477}, {"budget": 0.91, "threshold": -0.9997097849845886, "operator": "gt", "actual_fraction": 0.9053598335366291, "metric": 0.8990457056755399, "rescued": 2746, "harmed": 867, "missed_rescue": 45, "avoided_harm": 23, "selected_both_correct": 8528, "selected_both_wrong": 477}, {"budget": 0.92, "threshold": -0.9997391104698181, "operator": "ge", "actual_fraction": 0.9164813087465021, "metric": 0.8992609600344407, "rescued": 2752, "harmed": 870, "missed_rescue": 39, "avoided_harm": 20, "selected_both_correct": 8672, "selected_both_wrong": 479}, {"budget": 0.93, "threshold": -0.9997641444206238, "operator": "ge", "actual_fraction": 0.9265265121618713, "metric": 0.8995479658463084, "rescued": 2759, "harmed": 873, "missed_rescue": 32, "avoided_harm": 17, "selected_both_correct": 8801, "selected_both_wrong": 480}, {"budget": 0.94, "threshold": -0.9997876286506653, "operator": "ge", "actual_fraction": 0.9362847097653727, "metric": 0.8997632202052092, "rescued": 2765, "harmed": 876, "missed_rescue": 26, "avoided_harm": 14, "selected_both_correct": 8926, "selected_both_wrong": 482}, {"budget": 0.95, "threshold": -0.9998131394386292, "operator": "ge", "actual_fraction": 0.9475496878811795, "metric": 0.899906723111143, "rescued": 2769, "harmed": 878, "missed_rescue": 22, "avoided_harm": 12, "selected_both_correct": 9075, "selected_both_wrong": 484}, {"budget": 0.96, "threshold": -0.9998401403427124, "operator": "ge", "actual_fraction": 0.9593169261677549, "metric": 0.8996197172992753, "rescued": 2770, "harmed": 883, "missed_rescue": 21, "avoided_harm": 7, "selected_both_correct": 9230, "selected_both_wrong": 487}, {"budget": 0.97, "threshold": -0.999868631362915, "operator": "ge", "actual_fraction": 0.9700796441127932, "metric": 0.89997847456411, "rescued": 2775, "harmed": 883, "missed_rescue": 16, "avoided_harm": 7, "selected_both_correct": 9371, "selected_both_wrong": 491}, {"budget": 0.98, "threshold": -0.9998925924301147, "operator": "ge", "actual_fraction": 0.9793355815455262, "metric": 0.9001937289230106, "rescued": 2780, "harmed": 885, "missed_rescue": 11, "avoided_harm": 5, "selected_both_correct": 9491, "selected_both_wrong": 493}, {"budget": 0.99, "threshold": -0.99992835521698, "operator": "ge", "actual_fraction": 0.9909593169261678, "metric": 0.9004089832819114, "rescued": 2786, "harmed": 888, "missed_rescue": 5, "avoided_harm": 2, "selected_both_correct": 9643, "selected_both_wrong": 494}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.9006242376408122, "rescued": 2791, "harmed": 890, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 9761, "selected_both_wrong": 495}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.9050415353766829, "actual_fraction": 0.9959896877685477, "metric_std": 0.0, "actual_fraction_std": 0.008967330760526202, "per_seed": [{"seed": 2026, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 3, "metric": 0.9050415353766829, "actual_fraction": 1.0, "rescue": 1449, "harm": 406, "missed_rescue": 0}, {"seed": 2027, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 2, "metric": 0.9050415353766829, "actual_fraction": 1.0, "rescue": 1449, "harm": 406, "missed_rescue": 0}, {"seed": 2028, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 2, "metric": 0.9050415353766829, "actual_fraction": 1.0, "rescue": 1449, "harm": 406, "missed_rescue": 0}, {"seed": 2029, "budget": 0.98, "threshold": -0.9986718987255754, "operator": "ge", "alpha": 1.0, "epoch": 2, "metric": 0.9050415353766829, "actual_fraction": 0.9799484388427385, "rescue": 1439, "harm": 396, "missed_rescue": 10}, {"seed": 2030, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 1, "metric": 0.9050415353766829, "actual_fraction": 1.0, "rescue": 1449, "harm": 406, "missed_rescue": 0}], "rescued": 1447.0, "harmed": 404.0, "missed_rescue": 2.0, "avoided_harm": 2.0}, "validation_curve": [{"budget": 0.0, "metric": 0.7556574047550845, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.7600973932970495, "metric_std": 0.0004961474670779084, "actual_fraction": 0.009882555141793183, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.7637926095674592, "metric_std": 0.0005104133394788145, "actual_fraction": 0.01990833572042395, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.767287310226296, "metric_std": 0.0006898650575073909, "actual_fraction": 0.029934116299054715, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.7712116871956459, "metric_std": 0.0006453096720749825, "actual_fraction": 0.039959896877685475, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.774820968203953, "metric_std": 0.0007739476417133235, "actual_fraction": 0.04998567745631624, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.7776568318533372, "metric_std": 0.001060255257709673, "actual_fraction": 0.05986823259810943, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.7807791463763964, "metric_std": 0.0010456437839268988, "actual_fraction": 0.06989401317674018, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.7838155256373532, "metric_std": 0.0014939449797084332, "actual_fraction": 0.07991979375537095, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.78719564594672, "metric_std": 0.0016311358897173947, "actual_fraction": 0.08994557433400172, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.7901747350329418, "metric_std": 0.0012567981076184907, "actual_fraction": 0.09997135491263248, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.793182469206531, "metric_std": 0.0010907724299432566, "actual_fraction": 0.10999713549126325, "actual_fraction_std": 0.0}, {"budget": 0.12, "metric": 0.7961042681180177, "metric_std": 0.0011573820596296996, "actual_fraction": 0.11987969063305641, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.13, "metric": 0.7990833572042395, "metric_std": 0.0008497506143907444, "actual_fraction": 0.1299054712116872, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.8018046405041535, "metric_std": 0.0009522068254709271, "actual_fraction": 0.13993125179031796, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.8045832139788025, "metric_std": 0.0008509567904619315, "actual_fraction": 0.14995703236894872, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.8074190776281867, "metric_std": 0.001462723738950024, "actual_fraction": 0.1599828129475795, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.809596104268118, "metric_std": 0.0013111191625718352, "actual_fraction": 0.16986536808937266, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.8126038384417071, "metric_std": 0.0011304833721381084, "actual_fraction": 0.17989114866800343, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.815353766828989, "metric_std": 0.0011204587347813076, "actual_fraction": 0.18991692924663417, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.8181609853910053, "metric_std": 0.0010407277069369371, "actual_fraction": 0.19994270982526496, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.821025494127757, "metric_std": 0.0008688485183673925, "actual_fraction": 0.20996849040389573, "actual_fraction_std": 0.0}, {"budget": 0.22, "metric": 0.8238040676024061, "metric_std": 0.0008127333691842586, "actual_fraction": 0.2199942709825265, "actual_fraction_std": 0.0}, {"budget": 0.23, "metric": 0.8264107705528501, "metric_std": 0.0008652997689483365, "actual_fraction": 0.22987682612431967, "actual_fraction_std": 0.0}, {"budget": 0.24, "metric": 0.8288742480664565, "metric_std": 0.0006193568693910085, "actual_fraction": 0.23990260670295044, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.8309653394442853, "metric_std": 0.0009304143093045334, "actual_fraction": 0.24992838728158123, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.26, "metric": 0.8331423660842165, "metric_std": 0.0006324664040624722, "actual_fraction": 0.25995416786021197, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.8354339730736179, "metric_std": 0.0010813283350429542, "actual_fraction": 0.26997994843884277, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.8379833858493267, "metric_std": 0.0006516363887755486, "actual_fraction": 0.2798625035806359, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.840103122314523, "metric_std": 0.0007060302230802462, "actual_fraction": 0.2898882841592667, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.8417072472071039, "metric_std": 0.0009891891520957701, "actual_fraction": 0.29991406473789745, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.8435118877112575, "metric_std": 0.000974564642578312, "actual_fraction": 0.30993984531652824, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.8449154969922658, "metric_std": 0.0005002649440438859, "actual_fraction": 0.319965625895159, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.8469492981953595, "metric_std": 0.00037348624492707745, "actual_fraction": 0.3299914064737898, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.8491549699226584, "metric_std": 0.0008794088474940523, "actual_fraction": 0.3398739616155829, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.8509023202520767, "metric_std": 0.000784478025644788, "actual_fraction": 0.3498997421942137, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.8531079919793754, "metric_std": 0.0013572452295023676, "actual_fraction": 0.35992552277284445, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.8554282440561444, "metric_std": 0.0014236349057335168, "actual_fraction": 0.3699513033514752, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.8572328845602979, "metric_std": 0.0013024855393170803, "actual_fraction": 0.379977083930106, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.8586937840160414, "metric_std": 0.0012178373915486306, "actual_fraction": 0.3898596390718992, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.860068748209682, "metric_std": 0.0010907724299432748, "actual_fraction": 0.3998854196505299, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.8618160985391006, "metric_std": 0.0013715284583069858, "actual_fraction": 0.40991120022916067, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.42, "metric": 0.8633629332569465, "metric_std": 0.0018756517980104443, "actual_fraction": 0.41993698080779146, "actual_fraction_std": 0.0}, {"budget": 0.43, "metric": 0.8653107991979375, "metric_std": 0.002234041364154911, "actual_fraction": 0.42996276138642225, "actual_fraction_std": 0.0}, {"budget": 0.44, "metric": 0.8665998281294758, "metric_std": 0.002479910285079566, "actual_fraction": 0.439988541965053, "actual_fraction_std": 0.0}, {"budget": 0.45, "metric": 0.8685763391578345, "metric_std": 0.0018854691760256473, "actual_fraction": 0.4498710971068462, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.8693497565167574, "metric_std": 0.0016837366954666616, "actual_fraction": 0.459896877685477, "actual_fraction_std": 0.0}, {"budget": 0.47, "metric": 0.8704669149240905, "metric_std": 0.0014870635751838967, "actual_fraction": 0.46992265826410773, "actual_fraction_std": 0.0}, {"budget": 0.48, "metric": 0.8713549126324835, "metric_std": 0.0009839910648707336, "actual_fraction": 0.4799484388427385, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.49, "metric": 0.872156975078774, "metric_std": 0.0010841702015339746, "actual_fraction": 0.4899742194213693, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.5, "metric": 0.8732168433113721, "metric_std": 0.0010879478095006192, "actual_fraction": 0.5, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.8746204525923804, "metric_std": 0.0014091519766541714, "actual_fraction": 0.5098825551417931, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.8756803208249785, "metric_std": 0.0014322543683758543, "actual_fraction": 0.5199083357204239, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.8771125751933543, "metric_std": 0.0021146982326315225, "actual_fraction": 0.5299341162990547, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.8781151532512173, "metric_std": 0.0021956043707757325, "actual_fraction": 0.5399598968776855, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.8790317960469778, "metric_std": 0.001986139723970027, "actual_fraction": 0.5499856774563162, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.8798052134059009, "metric_std": 0.0020641364537729977, "actual_fraction": 0.5598682325981095, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.8811515325121742, "metric_std": 0.0017957522794222343, "actual_fraction": 0.5698940131767402, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.8826124319679174, "metric_std": 0.0017204946285242191, "actual_fraction": 0.579919793755371, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.8838155256373532, "metric_std": 0.00178429235203638, "actual_fraction": 0.5899455743340017, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.8847321684331136, "metric_std": 0.002091287567853793, "actual_fraction": 0.5999713549126325, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.8855055857920366, "metric_std": 0.0017819915227723696, "actual_fraction": 0.6099971354912632, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.886193067888857, "metric_std": 0.001732376638694712, "actual_fraction": 0.6198796906330565, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.8868232598109425, "metric_std": 0.001772758343943389, "actual_fraction": 0.6299054712116872, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.8873961615582928, "metric_std": 0.001728820610370266, "actual_fraction": 0.639931251790318, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.8881409338298483, "metric_std": 0.0019295608114160798, "actual_fraction": 0.6499570323689486, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.8885419650529933, "metric_std": 0.0015472325151310304, "actual_fraction": 0.6599828129475795, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.8891148668003437, "metric_std": 0.0013258993768012072, "actual_fraction": 0.6698653680893727, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.8894013176740188, "metric_std": 0.0013677841748855244, "actual_fraction": 0.6798911486680035, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.8900315095961042, "metric_std": 0.001646781134552284, "actual_fraction": 0.6899169292466341, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.8909481523918646, "metric_std": 0.001322801476828283, "actual_fraction": 0.699942709825265, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.8917215697507876, "metric_std": 0.0013165838091413514, "actual_fraction": 0.7099684904038958, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.8924376969349757, "metric_std": 0.0013282180600824491, "actual_fraction": 0.7199942709825266, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.8932684044686336, "metric_std": 0.0010592874297966025, "actual_fraction": 0.7298768261243197, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.8936407906044114, "metric_std": 0.0012262305810944129, "actual_fraction": 0.7399026067029504, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.8938413062159839, "metric_std": 0.0012127735966781217, "actual_fraction": 0.7499283872815812, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.8947293039243769, "metric_std": 0.0010670054730558133, "actual_fraction": 0.759954167860212, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.8954740761959323, "metric_std": 0.0010737132520189628, "actual_fraction": 0.7699799484388428, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.8962761386422228, "metric_std": 0.0010097139348046462, "actual_fraction": 0.7798625035806359, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.8965052993411631, "metric_std": 0.0008012947998108703, "actual_fraction": 0.7898882841592667, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.8968490403895732, "metric_std": 0.0006594594347305432, "actual_fraction": 0.7999140647378975, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.8972500716127184, "metric_std": 0.0007948689158127235, "actual_fraction": 0.8099398453165282, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.8981094242337437, "metric_std": 0.0012008750396021433, "actual_fraction": 0.8199656258951592, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.83, "metric": 0.8985677456316242, "metric_std": 0.0011697231159608053, "actual_fraction": 0.8299914064737898, "actual_fraction_std": 0.0}, {"budget": 0.84, "metric": 0.8992552277284445, "metric_std": 0.0012262305810944653, "actual_fraction": 0.8398739616155829, "actual_fraction_std": 0.0}, {"budget": 0.85, "metric": 0.8994270982526495, "metric_std": 0.001479457062001305, "actual_fraction": 0.8498997421942137, "actual_fraction_std": 0.0}, {"budget": 0.86, "metric": 0.9000286450873676, "metric_std": 0.0014358305396564549, "actual_fraction": 0.8599255227728445, "actual_fraction_std": 0.0}, {"budget": 0.87, "metric": 0.9007734173589228, "metric_std": 0.0008967330760526287, "actual_fraction": 0.8699513033514752, "actual_fraction_std": 0.0}, {"budget": 0.88, "metric": 0.9017187052420509, "metric_std": 0.0007870886086527585, "actual_fraction": 0.879977083930106, "actual_fraction_std": 0.0}, {"budget": 0.89, "metric": 0.9018905757662561, "metric_std": 0.0005991549889244527, "actual_fraction": 0.8898596390718991, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.9022343168146663, "metric_std": 0.0003869738208566787, "actual_fraction": 0.8998854196505299, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.9023488971641364, "metric_std": 0.0005939971742574164, "actual_fraction": 0.9099112002291608, "actual_fraction_std": 0.0}, {"budget": 0.92, "metric": 0.9023775422515039, "metric_std": 0.000595721407707926, "actual_fraction": 0.9199369808077915, "actual_fraction_std": 0.0}, {"budget": 0.93, "metric": 0.9028645087367517, "metric_std": 0.00021243765359771114, "actual_fraction": 0.9299627613864223, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.94, "metric": 0.9030363792609567, "metric_std": 0.0004296763105127536, "actual_fraction": 0.9399885419650531, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.9029790890862216, "metric_std": 0.0006913502369605792, "actual_fraction": 0.9498710971068463, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.9036665711830422, "metric_std": 0.0006913502369606044, "actual_fraction": 0.959896877685477, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.97, "metric": 0.9038670867946147, "metric_std": 0.0005583955511205206, "actual_fraction": 0.9699226582641078, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.98, "metric": 0.9044399885419651, "metric_std": 0.0003974344579063878, "actual_fraction": 0.9799484388427386, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.99, "metric": 0.9046977943282727, "metric_std": 0.0001633021233313243, "actual_fraction": 0.9899742194213692, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.9050415353766829, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.900652938221999, "actual_fraction": 0.9961684724115664, "metric_std": 6.417645052744179e-05, "actual_fraction_std": 0.008567556145403497, "per_seed": [{"seed": 2026, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 3, "metric": 0.9006242376408121, "actual_fraction": 1.0, "rescue": 2791, "harm": 890, "missed_rescue": 0}, {"seed": 2027, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 2, "metric": 0.9006242376408121, "actual_fraction": 1.0, "rescue": 2791, "harm": 890, "missed_rescue": 0}, {"seed": 2028, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 2, "metric": 0.9006242376408121, "actual_fraction": 1.0, "rescue": 2791, "harm": 890, "missed_rescue": 0}, {"seed": 2029, "budget": 0.98, "threshold": -0.9986718987255754, "operator": "ge", "alpha": 1.0, "epoch": 2, "metric": 0.9007677405467461, "actual_fraction": 0.9808423620578317, "rescue": 2777, "harm": 874, "missed_rescue": 14}, {"seed": 2030, "budget": 1.0, "threshold": null, "operator": "always", "alpha": 1.0, "epoch": 1, "metric": 0.9006242376408121, "actual_fraction": 1.0, "rescue": 2791, "harm": 890, "missed_rescue": 0}], "rescued": 2788.2, "harmed": 886.8, "missed_rescue": 2.8, "avoided_harm": 3.2000000000000455}, "test_curve": [{"budget": 0.0, "metric": 0.7642247255506924, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.7689459711559159, "metric_std": 0.00027509922526627547, "actual_fraction": 0.013173566764726984, "actual_fraction_std": 0.0003388309952821303}, {"budget": 0.02, "metric": 0.7731362560091842, "metric_std": 0.00064056006660449, "actual_fraction": 0.02379278180383153, "actual_fraction_std": 0.0007449722237162907}, {"budget": 0.03, "metric": 0.7765803257515964, "metric_std": 0.0007583283398502671, "actual_fraction": 0.03303436894597116, "actual_fraction_std": 0.0017870289597756515}, {"budget": 0.04, "metric": 0.7797373896821411, "metric_std": 0.0006852169842183487, "actual_fraction": 0.04220420463514386, "actual_fraction_std": 0.001527480415094802}, {"budget": 0.05, "metric": 0.7829949056468394, "metric_std": 0.0008361430512711168, "actual_fraction": 0.051905001076271794, "actual_fraction_std": 0.0024250505080455815}, {"budget": 0.06, "metric": 0.7858506134749228, "metric_std": 0.0008498837695249191, "actual_fraction": 0.05999856497094066, "actual_fraction_std": 0.0015375584686493624}, {"budget": 0.07, "metric": 0.7886202195594461, "metric_std": 0.001039777337030208, "actual_fraction": 0.06888139484824568, "actual_fraction_std": 0.0018052300531547852}, {"budget": 0.08, "metric": 0.7920355887206716, "metric_std": 0.000991105497586689, "actual_fraction": 0.07889789768242808, "actual_fraction_std": 0.0013807261684927736}, {"budget": 0.09, "metric": 0.7945181889933272, "metric_std": 0.0008450237796046193, "actual_fraction": 0.08783812872210661, "actual_fraction_std": 0.0010779446949387079}, {"budget": 0.1, "metric": 0.7976609026332784, "metric_std": 0.0010905276712040708, "actual_fraction": 0.09753892516323456, "actual_fraction_std": 0.00047486236945610497}, {"budget": 0.11, "metric": 0.8002296046494941, "metric_std": 0.0014362841608984648, "actual_fraction": 0.10607734806629834, "actual_fraction_std": 0.0015004462368084462}, {"budget": 0.12, "metric": 0.8031570639305446, "metric_std": 0.0011653814356491561, "actual_fraction": 0.11464447155054891, "actual_fraction_std": 0.0026868021140784826}, {"budget": 0.13, "metric": 0.806199325536342, "metric_std": 0.0015866645897607216, "actual_fraction": 0.12358470259022744, "actual_fraction_std": 0.0033272101132119027}, {"budget": 0.14, "metric": 0.8093563894668868, "metric_std": 0.001328855505306109, "actual_fraction": 0.13477792925306736, "actual_fraction_std": 0.004140997591926502}, {"budget": 0.15, "metric": 0.8122551481667504, "metric_std": 0.001346750115763567, "actual_fraction": 0.14473703092487625, "actual_fraction_std": 0.004273701093532679}, {"budget": 0.16, "metric": 0.8149099519265265, "metric_std": 0.0014445048416190998, "actual_fraction": 0.15445217765659752, "actual_fraction_std": 0.004323464161664857}, {"budget": 0.17, "metric": 0.8173064504556218, "metric_std": 0.0015484023398439043, "actual_fraction": 0.1638803185764512, "actual_fraction_std": 0.00329236762779176}, {"budget": 0.18, "metric": 0.8197316495659038, "metric_std": 0.0014056693817890988, "actual_fraction": 0.17287795077850326, "actual_fraction_std": 0.0034681270945558424}, {"budget": 0.19, "metric": 0.8221281480949989, "metric_std": 0.0013014534797457934, "actual_fraction": 0.18168902920284136, "actual_fraction_std": 0.002814084301616819}, {"budget": 0.2, "metric": 0.8248260027265552, "metric_std": 0.001329436510437119, "actual_fraction": 0.19227954366075914, "actual_fraction_std": 0.0024659962912753092}, {"budget": 0.21, "metric": 0.8271794503838704, "metric_std": 0.001150934459053618, "actual_fraction": 0.2023247470761283, "actual_fraction_std": 0.001950135326026384}, {"budget": 0.22, "metric": 0.8295759489129655, "metric_std": 0.0013175723821353556, "actual_fraction": 0.2122264475855636, "actual_fraction_std": 0.0021889485654912073}, {"budget": 0.23, "metric": 0.832058549185621, "metric_std": 0.0011840076832859109, "actual_fraction": 0.22270215971873428, "actual_fraction_std": 0.0033774286167211347}, {"budget": 0.24, "metric": 0.8343689459711559, "metric_std": 0.0011464526100242246, "actual_fraction": 0.23294826720241088, "actual_fraction_std": 0.002524896826178664}, {"budget": 0.25, "metric": 0.8368228456626247, "metric_std": 0.0012369584237359155, "actual_fraction": 0.2438544880533831, "actual_fraction_std": 0.0024393383580616165}, {"budget": 0.26, "metric": 0.8388318863456986, "metric_std": 0.0013340754422783796, "actual_fraction": 0.25457415512664133, "actual_fraction_std": 0.0021822352480082696}, {"budget": 0.27, "metric": 0.8409844299347062, "metric_std": 0.0014073165439239094, "actual_fraction": 0.26477721173853774, "actual_fraction_std": 0.0021353743529501125}, {"budget": 0.28, "metric": 0.8431800243954941, "metric_std": 0.0012700209861248352, "actual_fraction": 0.27446365788907223, "actual_fraction_std": 0.0020182447863097263}, {"budget": 0.29, "metric": 0.8448590083949201, "metric_std": 0.0010340676415471809, "actual_fraction": 0.28512592379995694, "actual_fraction_std": 0.001101565919668523}, {"budget": 0.3, "metric": 0.846796297625027, "metric_std": 0.0011551761094925877, "actual_fraction": 0.29479801965989816, "actual_fraction_std": 0.002217108286380039}, {"budget": 0.31, "metric": 0.848963191504628, "metric_std": 0.0015654318884460233, "actual_fraction": 0.3060199469039248, "actual_fraction_std": 0.00303823137830812}, {"budget": 0.32, "metric": 0.8506134749228673, "metric_std": 0.0013876070607601021, "actual_fraction": 0.31655306019946905, "actual_fraction_std": 0.0021605409781573653}, {"budget": 0.33, "metric": 0.8522063571787328, "metric_std": 0.0016944588334620613, "actual_fraction": 0.3261964554782234, "actual_fraction_std": 0.0037846425966225005}, {"budget": 0.34, "metric": 0.8538996914687523, "metric_std": 0.0014795421016036917, "actual_fraction": 0.33576809930401086, "actual_fraction_std": 0.0030600123038984485}, {"budget": 0.35, "metric": 0.8559948338953864, "metric_std": 0.0015497317260856297, "actual_fraction": 0.3468465236421038, "actual_fraction_std": 0.0027877078539230017}, {"budget": 0.36, "metric": 0.8577742699289661, "metric_std": 0.0013311780043202236, "actual_fraction": 0.3584128578603717, "actual_fraction_std": 0.0025747676730898806}, {"budget": 0.37, "metric": 0.8602138193298414, "metric_std": 0.0014091444659100393, "actual_fraction": 0.3690894740618497, "actual_fraction_std": 0.004269060710524073}, {"budget": 0.38, "metric": 0.862251560594102, "metric_std": 0.001217028170390499, "actual_fraction": 0.37969433881036097, "actual_fraction_std": 0.003929792428016096}, {"budget": 0.39, "metric": 0.8637583411064075, "metric_std": 0.001037794918950388, "actual_fraction": 0.3889359259525006, "actual_fraction_std": 0.0028859786604260047}, {"budget": 0.4, "metric": 0.8656525794647342, "metric_std": 0.0008589221354176846, "actual_fraction": 0.3985936715218483, "actual_fraction_std": 0.003270953499845734}, {"budget": 0.41, "metric": 0.867173710267633, "metric_std": 0.000975133659673918, "actual_fraction": 0.40773480662983425, "actual_fraction_std": 0.002766202304886153}, {"budget": 0.42, "metric": 0.8685800387457846, "metric_std": 0.0011180340779669094, "actual_fraction": 0.4173925521991821, "actual_fraction_std": 0.0023712742543469494}, {"budget": 0.43, "metric": 0.8699433163521562, "metric_std": 0.000911271700845646, "actual_fraction": 0.4275669082298917, "actual_fraction_std": 0.001463448235227903}, {"budget": 0.44, "metric": 0.8715648991892084, "metric_std": 0.0010134495868426132, "actual_fraction": 0.43810002152543587, "actual_fraction_std": 0.0023783198588178157}, {"budget": 0.45, "metric": 0.8724115663342182, "metric_std": 0.001003751281573572, "actual_fraction": 0.44685369878740044, "actual_fraction_std": 0.0014496634862509806}, {"budget": 0.46, "metric": 0.8738752959747436, "metric_std": 0.0012462878325507792, "actual_fraction": 0.45798952428786677, "actual_fraction_std": 0.0015118969294472089}, {"budget": 0.47, "metric": 0.8751524718375547, "metric_std": 0.001633823281443033, "actual_fraction": 0.46812082944679634, "actual_fraction_std": 0.0008690519870066593}, {"budget": 0.48, "metric": 0.8762000430508717, "metric_std": 0.0015700293488248873, "actual_fraction": 0.4783812872210662, "actual_fraction_std": 0.0019724473115560488}, {"budget": 0.49, "metric": 0.8774485183324963, "metric_std": 0.0018881678075541135, "actual_fraction": 0.48832603860228174, "actual_fraction_std": 0.0031592646285573444}, {"budget": 0.5, "metric": 0.8784673889646266, "metric_std": 0.0016544895563958854, "actual_fraction": 0.49715146731721316, "actual_fraction_std": 0.0025572121202275056}, {"budget": 0.51, "metric": 0.8792997058190428, "metric_std": 0.001922215934476669, "actual_fraction": 0.5063930544593529, "actual_fraction_std": 0.0036220628095878873}, {"budget": 0.52, "metric": 0.8800889718016791, "metric_std": 0.00197244731155603, "actual_fraction": 0.5147879744564827, "actual_fraction_std": 0.0049639872583449015}, {"budget": 0.53, "metric": 0.8813230967927101, "metric_std": 0.002080173497068858, "actual_fraction": 0.5246466240941379, "actual_fraction_std": 0.005509231497894011}, {"budget": 0.54, "metric": 0.8819114587070389, "metric_std": 0.0023842652623739698, "actual_fraction": 0.5343761211164526, "actual_fraction_std": 0.005272047561457054}, {"budget": 0.55, "metric": 0.8825572217837412, "metric_std": 0.0022082668076827817, "actual_fraction": 0.5431010977972304, "actual_fraction_std": 0.00484452505626286}, {"budget": 0.56, "metric": 0.88318863456985, "metric_std": 0.0020752177461404835, "actual_fraction": 0.5517686733156346, "actual_fraction_std": 0.006141144781837013}, {"budget": 0.57, "metric": 0.8840353017148598, "metric_std": 0.0018345094043292704, "actual_fraction": 0.5617708258592236, "actual_fraction_std": 0.005113417523471749}, {"budget": 0.58, "metric": 0.8847671665351223, "metric_std": 0.0018922532818024973, "actual_fraction": 0.5720599842146804, "actual_fraction_std": 0.004329651761473454}, {"budget": 0.59, "metric": 0.8856425342613189, "metric_std": 0.001494257385440149, "actual_fraction": 0.5826074478008179, "actual_fraction_std": 0.004140313751183346}, {"budget": 0.6, "metric": 0.8862882973380211, "metric_std": 0.0015071221471611627, "actual_fraction": 0.5916337805840568, "actual_fraction_std": 0.004990879641221526}, {"budget": 0.61, "metric": 0.886991461577097, "metric_std": 0.0014187931357742868, "actual_fraction": 0.6005166104613618, "actual_fraction_std": 0.006096889462831221}, {"budget": 0.62, "metric": 0.8876515749443927, "metric_std": 0.0015942714305017857, "actual_fraction": 0.6107340173638516, "actual_fraction_std": 0.005439971633994595}, {"budget": 0.63, "metric": 0.8882973380210949, "metric_std": 0.0012708314654582104, "actual_fraction": 0.6210949271722753, "actual_fraction_std": 0.005393161103182769}, {"budget": 0.64, "metric": 0.8884982420894024, "metric_std": 0.001324003880838514, "actual_fraction": 0.6311114300064576, "actual_fraction_std": 0.00584640067410865}, {"budget": 0.65, "metric": 0.888971801678984, "metric_std": 0.0012371665082565092, "actual_fraction": 0.6408839779005524, "actual_fraction_std": 0.006472164013280806}, {"budget": 0.66, "metric": 0.8894453612685658, "metric_std": 0.0008202920334090444, "actual_fraction": 0.651316639161943, "actual_fraction_std": 0.0066637363303935675}, {"budget": 0.67, "metric": 0.8900767740546748, "metric_std": 0.0007404666513794327, "actual_fraction": 0.6610030853124775, "actual_fraction_std": 0.006190117237994721}, {"budget": 0.68, "metric": 0.8905503336442564, "metric_std": 0.0005731140287785174, "actual_fraction": 0.6714070459926813, "actual_fraction_std": 0.005263985089142735}, {"budget": 0.69, "metric": 0.891095644686805, "metric_std": 0.0006603472979748377, "actual_fraction": 0.6824711200401807, "actual_fraction_std": 0.005805574958740383}, {"budget": 0.7, "metric": 0.8917270574729139, "metric_std": 0.0009586273384435456, "actual_fraction": 0.6926885269426706, "actual_fraction_std": 0.005737200594088865}, {"budget": 0.71, "metric": 0.8924445720025831, "metric_std": 0.0010702758047051425, "actual_fraction": 0.7038817536055105, "actual_fraction_std": 0.004970827564726834}, {"budget": 0.72, "metric": 0.8930903350792854, "metric_std": 0.0011022667357204904, "actual_fraction": 0.7141135107985938, "actual_fraction_std": 0.004091280180043034}, {"budget": 0.73, "metric": 0.8933486403099664, "metric_std": 0.0010982895319409343, "actual_fraction": 0.7237712563679415, "actual_fraction_std": 0.004586876835866309}, {"budget": 0.74, "metric": 0.8940518045490421, "metric_std": 0.001057695272241694, "actual_fraction": 0.7347779292530674, "actual_fraction_std": 0.005080843954468902}, {"budget": 0.75, "metric": 0.894310109779723, "metric_std": 0.0008904121149452119, "actual_fraction": 0.7442921719164813, "actual_fraction_std": 0.0032197939869606867}, {"budget": 0.76, "metric": 0.8947119179163379, "metric_std": 0.0006912015256536327, "actual_fraction": 0.7545382794001578, "actual_fraction_std": 0.002789092591745268}, {"budget": 0.77, "metric": 0.89531463012126, "metric_std": 0.0008889654655495203, "actual_fraction": 0.7650426921145153, "actual_fraction_std": 0.002671043607987233}, {"budget": 0.78, "metric": 0.8957881897108416, "metric_std": 0.0007687792343246937, "actual_fraction": 0.775217048145225, "actual_fraction_std": 0.0023901958773035036}, {"budget": 0.79, "metric": 0.8964196024969506, "metric_std": 0.0007869796704137521, "actual_fraction": 0.7859797660902632, "actual_fraction_std": 0.0022395190579654624}, {"budget": 0.8, "metric": 0.8969505632489059, "metric_std": 0.0012839300507104122, "actual_fraction": 0.796154122120973, "actual_fraction_std": 0.0027863224279181054}, {"budget": 0.81, "metric": 0.8974671737102675, "metric_std": 0.0010798534077127312, "actual_fraction": 0.8065006816388032, "actual_fraction_std": 0.001771694338887651}, {"budget": 0.82, "metric": 0.8975676257444214, "metric_std": 0.0009845909962210154, "actual_fraction": 0.8162301786611179, "actual_fraction_std": 0.002036022546905236}, {"budget": 0.83, "metric": 0.8981559876587502, "metric_std": 0.0010865074969629985, "actual_fraction": 0.8265623878883549, "actual_fraction_std": 0.00223434070622135}, {"budget": 0.84, "metric": 0.8984716940518046, "metric_std": 0.0008444143141235652, "actual_fraction": 0.8370093994403385, "actual_fraction_std": 0.0029077492099568447}, {"budget": 0.85, "metric": 0.8988735021884194, "metric_std": 0.0008678663766496755, "actual_fraction": 0.8481308746502118, "actual_fraction_std": 0.0033097569457431575}, {"budget": 0.86, "metric": 0.8992035588720672, "metric_std": 0.0009208258435620933, "actual_fraction": 0.8577886202195593, "actual_fraction_std": 0.004132410303503479}, {"budget": 0.87, "metric": 0.8995479658463085, "metric_std": 0.0007843585817993587, "actual_fraction": 0.867733371600775, "actual_fraction_std": 0.004385827738067517}, {"budget": 0.88, "metric": 0.8999641242735166, "metric_std": 0.0009658502501521895, "actual_fraction": 0.8766449020592667, "actual_fraction_std": 0.003811413948757421}, {"budget": 0.89, "metric": 0.9001363277606373, "metric_std": 0.0010827101677514232, "actual_fraction": 0.8878668293032934, "actual_fraction_std": 0.003561431632048686}, {"budget": 0.9, "metric": 0.9001219774700437, "metric_std": 0.0009838063572362881, "actual_fraction": 0.89811293678697, "actual_fraction_std": 0.002324459166667864}, {"budget": 0.91, "metric": 0.9004950850254719, "metric_std": 0.0006908290106571425, "actual_fraction": 0.9073545239291096, "actual_fraction_std": 0.0017446082130075653}, {"budget": 0.92, "metric": 0.9004807347348784, "metric_std": 0.0006109416074451668, "actual_fraction": 0.9180598407117744, "actual_fraction_std": 0.002323905394374911}, {"budget": 0.93, "metric": 0.9008538422903065, "metric_std": 0.0006833360666318412, "actual_fraction": 0.9286647054602856, "actual_fraction_std": 0.0013833337745333706}, {"budget": 0.94, "metric": 0.9008251417091196, "metric_std": 0.0007650871930869838, "actual_fraction": 0.9384372533543803, "actual_fraction_std": 0.0026328970555943624}, {"budget": 0.95, "metric": 0.9008825428714932, "metric_std": 0.0006742346250030199, "actual_fraction": 0.9489416660687381, "actual_fraction_std": 0.002291894610191045}, {"budget": 0.96, "metric": 0.9009686446150535, "metric_std": 0.0004850525588335469, "actual_fraction": 0.9599626892444573, "actual_fraction_std": 0.0022752113074130504}, {"budget": 0.97, "metric": 0.900839491999713, "metric_std": 0.00030441563371739225, "actual_fraction": 0.9699217909162661, "actual_fraction_std": 0.0015665825184347948}, {"budget": 0.98, "metric": 0.900724689674966, "metric_std": 0.00015719955729497269, "actual_fraction": 0.9809284638013921, "actual_fraction_std": 0.0022193131524055987}, {"budget": 0.99, "metric": 0.9006816388031857, "metric_std": 0.0001064246033880566, "actual_fraction": 0.9903422544306523, "actual_fraction_std": 0.0015030174037771208}, {"budget": 1.0, "metric": 0.9006242376408121, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 4870, "rescuable": 1449, "harmful": 406, "both_wrong": 257, "unit": "sample", "total": 6982}, "test": {"both_correct": 9761, "rescuable": 2791, "harmful": 890, "both_wrong": 495, "unit": "sample", "total": 13937}}, "small": {"validation": 0.7556574047550845, "test": 0.7642247255506924}, "large": {"validation": 0.9050415353766829, "test": 0.9006242376408122}, "learned_minus_confidence_test": 0.0010332209227237321, "learned_minus_small_test": 0.13642821267130656, "learned_minus_large_test": 2.8700581186758e-05, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_vilt_Qwen3.8"}, {"key": ["construction", "yolo26x", "MiniCPM"], "expert_display": "YOLO26x", "run": "outputs/full_budget_20260913/runs/full_yolo26x_MiniCPM_s{2026..2030}", "validation_rows": 701, "test_rows": 3004, "confidence_rule": {"column": "max_score", "columns": ["detection_count", "has_detections", "max_score", "mean_score", "detection_count_valid", "has_detections_valid", "max_score_valid", "mean_score_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 94, "validation_area_0_25": 0.1745164052515915, "validation_point": {"budget": 0.94, "threshold": -0.490234375, "operator": "ge", "actual_fraction": 0.9386590584878745, "metric": 0.22641590025257546, "rescued": 272, "harmed": 478, "missed_rescue": 14, "avoided_harm": 40, "selected_both_correct": 1782, "selected_both_wrong": 100}, "test_point": {"budget": 0.94, "threshold": -0.490234375, "operator": "ge", "actual_fraction": 0.9254327563249002, "metric": 0.2523047077304935, "rescued": 1113, "harmed": 1932, "missed_rescue": 85, "avoided_harm": 177, "selected_both_correct": 7597, "selected_both_wrong": 478}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.11, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.12, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.13, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.14, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.15, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.16, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.17, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.18, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 286, "avoided_harm": 518, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.19, "threshold": 1.0, "operator": "ge", "actual_fraction": 0.18972895863052783, "metric": 0.1583943405474877, "rescued": 0, "harmed": 88, "missed_rescue": 286, "avoided_harm": 430, "selected_both_correct": 444, "selected_both_wrong": 0}, {"budget": 0.2, "threshold": -0.0010652542114257812, "operator": "ge", "actual_fraction": 0.19971469329529243, "metric": 0.15744253541872588, "rescued": 0, "harmed": 94, "missed_rescue": 286, "avoided_harm": 424, "selected_both_correct": 466, "selected_both_wrong": 0}, {"budget": 0.21, "threshold": -0.0011425018310546875, "operator": "ge", "actual_fraction": 0.20970042796005706, "metric": 0.1562524865065214, "rescued": 0, "harmed": 100, "missed_rescue": 286, "avoided_harm": 418, "selected_both_correct": 488, "selected_both_wrong": 0}, {"budget": 0.22, "threshold": -0.0011968612670898438, "operator": "ge", "actual_fraction": 0.21968616262482168, "metric": 0.15524778086857327, "rescued": 0, "harmed": 106, "missed_rescue": 286, "avoided_harm": 412, "selected_both_correct": 509, "selected_both_wrong": 1}, {"budget": 0.23, "threshold": -0.0012493133544921875, "operator": "ge", "actual_fraction": 0.2296718972895863, "metric": 0.1542743955978643, "rescued": 0, "harmed": 112, "missed_rescue": 286, "avoided_harm": 406, "selected_both_correct": 531, "selected_both_wrong": 1}, {"budget": 0.24, "threshold": -0.0013513565063476562, "operator": "ge", "actual_fraction": 0.2396576319543509, "metric": 0.15300917812583928, "rescued": 0, "harmed": 118, "missed_rescue": 286, "avoided_harm": 400, "selected_both_correct": 553, "selected_both_wrong": 1}, {"budget": 0.25, "threshold": -0.0015010833740234375, "operator": "gt", "actual_fraction": 0.24821683309557774, "metric": 0.15249392985085833, "rescued": 0, "harmed": 122, "missed_rescue": 286, "avoided_harm": 396, "selected_both_correct": 572, "selected_both_wrong": 2}, {"budget": 0.26, "threshold": -0.00157928466796875, "operator": "ge", "actual_fraction": 0.25962910128388017, "metric": 0.15123140165156973, "rescued": 0, "harmed": 131, "missed_rescue": 286, "avoided_harm": 387, "selected_both_correct": 595, "selected_both_wrong": 2}, {"budget": 0.27, "threshold": -0.0017137527465820312, "operator": "gt", "actual_fraction": 0.268188302425107, "metric": 0.15063802192285164, "rescued": 0, "harmed": 136, "missed_rescue": 286, "avoided_harm": 382, "selected_both_correct": 614, "selected_both_wrong": 2}, {"budget": 0.28, "threshold": -0.0017614364624023438, "operator": "ge", "actual_fraction": 0.2796005706134094, "metric": 0.14987516228902426, "rescued": 0, "harmed": 143, "missed_rescue": 286, "avoided_harm": 375, "selected_both_correct": 639, "selected_both_wrong": 2}, {"budget": 0.29, "threshold": -0.0018463134765625, "operator": "ge", "actual_fraction": 0.289586305278174, "metric": 0.14942913195896182, "rescued": 0, "harmed": 147, "missed_rescue": 286, "avoided_harm": 371, "selected_both_correct": 663, "selected_both_wrong": 2}, {"budget": 0.3, "threshold": -0.001934051513671875, "operator": "ge", "actual_fraction": 0.2995720399429387, "metric": 0.14874977232993136, "rescued": 0, "harmed": 152, "missed_rescue": 286, "avoided_harm": 366, "selected_both_correct": 686, "selected_both_wrong": 2}, {"budget": 0.31, "threshold": -0.0020427703857421875, "operator": "gt", "actual_fraction": 0.30813124108416545, "metric": 0.1481550075018755, "rescued": 0, "harmed": 156, "missed_rescue": 286, "avoided_harm": 362, "selected_both_correct": 706, "selected_both_wrong": 2}, {"budget": 0.32, "threshold": -0.0021495819091796875, "operator": "gt", "actual_fraction": 0.3181169757489301, "metric": 0.14737954730376035, "rescued": 0, "harmed": 162, "missed_rescue": 286, "avoided_harm": 356, "selected_both_correct": 728, "selected_both_wrong": 2}, {"budget": 0.33, "threshold": -0.002208709716796875, "operator": "ge", "actual_fraction": 0.32952924393723254, "metric": 0.14624628747548335, "rescued": 0, "harmed": 170, "missed_rescue": 286, "avoided_harm": 348, "selected_both_correct": 752, "selected_both_wrong": 2}, {"budget": 0.34, "threshold": -0.0023326873779296875, "operator": "ge", "actual_fraction": 0.33951497860199714, "metric": 0.14552266133559724, "rescued": 0, "harmed": 176, "missed_rescue": 286, "avoided_harm": 342, "selected_both_correct": 774, "selected_both_wrong": 2}, {"budget": 0.35, "threshold": -0.0024623870849609375, "operator": "gt", "actual_fraction": 0.34807417974322397, "metric": 0.14490257033424028, "rescued": 0, "harmed": 181, "missed_rescue": 286, "avoided_harm": 337, "selected_both_correct": 793, "selected_both_wrong": 2}, {"budget": 0.36, "threshold": -0.0025615692138671875, "operator": "ge", "actual_fraction": 0.3594864479315264, "metric": 0.1443034349839422, "rescued": 0, "harmed": 188, "missed_rescue": 286, "avoided_harm": 330, "selected_both_correct": 818, "selected_both_wrong": 2}, {"budget": 0.37, "threshold": -0.0026721954345703125, "operator": "gt", "actual_fraction": 0.3666191155492154, "metric": 0.14397079647073363, "rescued": 0, "harmed": 192, "missed_rescue": 286, "avoided_harm": 326, "selected_both_correct": 834, "selected_both_wrong": 2}, {"budget": 0.38, "threshold": -0.0028228759765625, "operator": "gt", "actual_fraction": 0.37660485021398005, "metric": 0.14328749409772273, "rescued": 0, "harmed": 198, "missed_rescue": 286, "avoided_harm": 320, "selected_both_correct": 856, "selected_both_wrong": 2}, {"budget": 0.39, "threshold": -0.0029354095458984375, "operator": "ge", "actual_fraction": 0.38944365192582026, "metric": 0.14535108686369555, "rescued": 1, "harmed": 209, "missed_rescue": 285, "avoided_harm": 309, "selected_both_correct": 880, "selected_both_wrong": 2}, {"budget": 0.4, "threshold": -0.00307464599609375, "operator": "ge", "actual_fraction": 0.39942938659058486, "metric": 0.14474378206514643, "rescued": 1, "harmed": 214, "missed_rescue": 285, "avoided_harm": 304, "selected_both_correct": 903, "selected_both_wrong": 2}, {"budget": 0.41, "threshold": -0.0033626556396484375, "operator": "ge", "actual_fraction": 0.4094151212553495, "metric": 0.14399177962039386, "rescued": 1, "harmed": 221, "missed_rescue": 285, "avoided_harm": 297, "selected_both_correct": 924, "selected_both_wrong": 2}, {"budget": 0.42, "threshold": -0.0035381317138671875, "operator": "ge", "actual_fraction": 0.4194008559201141, "metric": 0.14353714231992915, "rescued": 1, "harmed": 227, "missed_rescue": 285, "avoided_harm": 291, "selected_both_correct": 946, "selected_both_wrong": 2}, {"budget": 0.43, "threshold": -0.003765106201171875, "operator": "ge", "actual_fraction": 0.42938659058487877, "metric": 0.1430174966224799, "rescued": 1, "harmed": 234, "missed_rescue": 285, "avoided_harm": 284, "selected_both_correct": 967, "selected_both_wrong": 2}, {"budget": 0.44, "threshold": -0.00392913818359375, "operator": "ge", "actual_fraction": 0.43937232524964337, "metric": 0.14225189964081136, "rescued": 1, "harmed": 242, "missed_rescue": 285, "avoided_harm": 276, "selected_both_correct": 987, "selected_both_wrong": 2}, {"budget": 0.45, "threshold": -0.004085540771484375, "operator": "ge", "actual_fraction": 0.44935805991440797, "metric": 0.14143790301195314, "rescued": 1, "harmed": 248, "missed_rescue": 285, "avoided_harm": 270, "selected_both_correct": 1009, "selected_both_wrong": 2}, {"budget": 0.46, "threshold": -0.004364013671875, "operator": "ge", "actual_fraction": 0.4593437945791726, "metric": 0.14328380481065045, "rescued": 2, "harmed": 254, "missed_rescue": 284, "avoided_harm": 264, "selected_both_correct": 1030, "selected_both_wrong": 2}, {"budget": 0.47, "threshold": -0.004680633544921875, "operator": "gt", "actual_fraction": 0.46790299572039945, "metric": 0.145146248248267, "rescued": 3, "harmed": 259, "missed_rescue": 283, "avoided_harm": 259, "selected_both_correct": 1048, "selected_both_wrong": 2}, {"budget": 0.48, "threshold": -0.00490570068359375, "operator": "ge", "actual_fraction": 0.4793152639087018, "metric": 0.14408640672685616, "rescued": 3, "harmed": 268, "missed_rescue": 283, "avoided_harm": 250, "selected_both_correct": 1071, "selected_both_wrong": 2}, {"budget": 0.49, "threshold": -0.005039215087890625, "operator": "ge", "actual_fraction": 0.4893009985734665, "metric": 0.14363247863247863, "rescued": 5, "harmed": 274, "missed_rescue": 281, "avoided_harm": 244, "selected_both_correct": 1090, "selected_both_wrong": 3}, {"budget": 0.5, "threshold": -0.005321502685546875, "operator": "ge", "actual_fraction": 0.4992867332382311, "metric": 0.14422472705491574, "rescued": 8, "harmed": 281, "missed_rescue": 278, "avoided_harm": 237, "selected_both_correct": 1104, "selected_both_wrong": 7}, {"budget": 0.51, "threshold": -0.005619049072265625, "operator": "ge", "actual_fraction": 0.5092724679029957, "metric": 0.14584304986884322, "rescued": 15, "harmed": 285, "missed_rescue": 271, "avoided_harm": 233, "selected_both_correct": 1120, "selected_both_wrong": 8}, {"budget": 0.52, "threshold": -0.00586700439453125, "operator": "gt", "actual_fraction": 0.5178316690442225, "metric": 0.14475873599710132, "rescued": 18, "harmed": 291, "missed_rescue": 268, "avoided_harm": 227, "selected_both_correct": 1133, "selected_both_wrong": 10}, {"budget": 0.53, "threshold": -0.00605010986328125, "operator": "ge", "actual_fraction": 0.5292439372325249, "metric": 0.14599834427394986, "rescued": 24, "harmed": 297, "missed_rescue": 262, "avoided_harm": 221, "selected_both_correct": 1151, "selected_both_wrong": 12}, {"budget": 0.54, "threshold": -0.006439208984375, "operator": "gt", "actual_fraction": 0.5378031383737518, "metric": 0.14694159872855866, "rescued": 29, "harmed": 301, "missed_rescue": 257, "avoided_harm": 217, "selected_both_correct": 1166, "selected_both_wrong": 12}, {"budget": 0.55, "threshold": -0.006641387939453125, "operator": "ge", "actual_fraction": 0.5492154065620543, "metric": 0.14684487529484538, "rescued": 35, "harmed": 305, "missed_rescue": 251, "avoided_harm": 213, "selected_both_correct": 1187, "selected_both_wrong": 13}, {"budget": 0.56, "threshold": -0.0070648193359375, "operator": "ge", "actual_fraction": 0.5592011412268189, "metric": 0.14849467509528708, "rescued": 42, "harmed": 308, "missed_rescue": 244, "avoided_harm": 210, "selected_both_correct": 1203, "selected_both_wrong": 15}, {"budget": 0.57, "threshold": -0.007518768310546875, "operator": "ge", "actual_fraction": 0.5691868758915835, "metric": 0.1496557181286853, "rescued": 48, "harmed": 311, "missed_rescue": 238, "avoided_harm": 207, "selected_both_correct": 1221, "selected_both_wrong": 16}, {"budget": 0.58, "threshold": -0.00818634033203125, "operator": "ge", "actual_fraction": 0.579172610556348, "metric": 0.15076631577130084, "rescued": 53, "harmed": 315, "missed_rescue": 233, "avoided_harm": 203, "selected_both_correct": 1237, "selected_both_wrong": 19}, {"budget": 0.59, "threshold": -0.00878143310546875, "operator": "gt", "actual_fraction": 0.5877318116975749, "metric": 0.15169442750270445, "rescued": 57, "harmed": 320, "missed_rescue": 229, "avoided_harm": 198, "selected_both_correct": 1250, "selected_both_wrong": 21}, {"budget": 0.6, "threshold": -0.00955963134765625, "operator": "gt", "actual_fraction": 0.5977175463623395, "metric": 0.1523976411340631, "rescued": 62, "harmed": 322, "missed_rescue": 224, "avoided_harm": 196, "selected_both_correct": 1268, "selected_both_wrong": 24}, {"budget": 0.61, "threshold": -0.0099334716796875, "operator": "ge", "actual_fraction": 0.609129814550642, "metric": 0.15402113497563988, "rescued": 68, "harmed": 328, "missed_rescue": 218, "avoided_harm": 190, "selected_both_correct": 1286, "selected_both_wrong": 26}, {"budget": 0.62, "threshold": -0.010650634765625, "operator": "gt", "actual_fraction": 0.6176890156918687, "metric": 0.1533732028512746, "rescued": 71, "harmed": 331, "missed_rescue": 215, "avoided_harm": 187, "selected_both_correct": 1302, "selected_both_wrong": 28}, {"budget": 0.63, "threshold": -0.01129150390625, "operator": "ge", "actual_fraction": 0.6291012838801712, "metric": 0.15514650916956785, "rescued": 79, "harmed": 337, "missed_rescue": 207, "avoided_harm": 181, "selected_both_correct": 1317, "selected_both_wrong": 31}, {"budget": 0.64, "threshold": -0.011871337890625, "operator": "ge", "actual_fraction": 0.6390870185449358, "metric": 0.1583928135738368, "rescued": 85, "harmed": 341, "missed_rescue": 201, "avoided_harm": 177, "selected_both_correct": 1332, "selected_both_wrong": 34}, {"budget": 0.65, "threshold": -0.0125274658203125, "operator": "gt", "actual_fraction": 0.6476462196861626, "metric": 0.15975557497296627, "rescued": 92, "harmed": 346, "missed_rescue": 194, "avoided_harm": 172, "selected_both_correct": 1344, "selected_both_wrong": 34}, {"budget": 0.66, "threshold": -0.012969970703125, "operator": "gt", "actual_fraction": 0.6562054208273894, "metric": 0.16074742775562448, "rescued": 95, "harmed": 347, "missed_rescue": 191, "avoided_harm": 171, "selected_both_correct": 1361, "selected_both_wrong": 37}, {"budget": 0.67, "threshold": -0.01363372802734375, "operator": "ge", "actual_fraction": 0.6690442225392297, "metric": 0.16246869830774383, "rescued": 102, "harmed": 352, "missed_rescue": 184, "avoided_harm": 166, "selected_both_correct": 1381, "selected_both_wrong": 41}, {"budget": 0.68, "threshold": -0.01473236083984375, "operator": "ge", "actual_fraction": 0.6790299572039943, "metric": 0.16459532061840704, "rescued": 109, "harmed": 357, "missed_rescue": 177, "avoided_harm": 161, "selected_both_correct": 1396, "selected_both_wrong": 42}, {"budget": 0.69, "threshold": -0.015716552734375, "operator": "ge", "actual_fraction": 0.6890156918687589, "metric": 0.16603896761133602, "rescued": 114, "harmed": 362, "missed_rescue": 172, "avoided_harm": 156, "selected_both_correct": 1411, "selected_both_wrong": 45}, {"budget": 0.7, "threshold": -0.017578125, "operator": "gt", "actual_fraction": 0.6975748930099858, "metric": 0.16707723655265538, "rescued": 119, "harmed": 366, "missed_rescue": 167, "avoided_harm": 152, "selected_both_correct": 1423, "selected_both_wrong": 48}, {"budget": 0.71, "threshold": -0.0189056396484375, "operator": "ge", "actual_fraction": 0.7089871611982882, "metric": 0.16940425786435637, "rescued": 126, "harmed": 371, "missed_rescue": 160, "avoided_harm": 147, "selected_both_correct": 1441, "selected_both_wrong": 50}, {"budget": 0.72, "threshold": -0.019866943359375, "operator": "ge", "actual_fraction": 0.7189728958630528, "metric": 0.1708064509820028, "rescued": 134, "harmed": 379, "missed_rescue": 152, "avoided_harm": 139, "selected_both_correct": 1453, "selected_both_wrong": 50}, {"budget": 0.73, "threshold": -0.020965576171875, "operator": "ge", "actual_fraction": 0.7289586305278174, "metric": 0.17255814544390938, "rescued": 139, "harmed": 381, "missed_rescue": 147, "avoided_harm": 137, "selected_both_correct": 1470, "selected_both_wrong": 54}, {"budget": 0.74, "threshold": -0.021453857421875, "operator": "ge", "actual_fraction": 0.738944365192582, "metric": 0.17530145923526386, "rescued": 146, "harmed": 384, "missed_rescue": 140, "avoided_harm": 134, "selected_both_correct": 1487, "selected_both_wrong": 55}, {"budget": 0.75, "threshold": -0.0248870849609375, "operator": "ge", "actual_fraction": 0.7489300998573466, "metric": 0.17900218396288498, "rescued": 155, "harmed": 389, "missed_rescue": 131, "avoided_harm": 129, "selected_both_correct": 1499, "selected_both_wrong": 57}, {"budget": 0.76, "threshold": -0.0269622802734375, "operator": "ge", "actual_fraction": 0.7589158345221113, "metric": 0.18402223394204145, "rescued": 163, "harmed": 392, "missed_rescue": 123, "avoided_harm": 126, "selected_both_correct": 1514, "selected_both_wrong": 59}, {"budget": 0.77, "threshold": -0.0282135009765625, "operator": "ge", "actual_fraction": 0.7689015691868759, "metric": 0.18675324648763036, "rescued": 169, "harmed": 395, "missed_rescue": 117, "avoided_harm": 123, "selected_both_correct": 1531, "selected_both_wrong": 61}, {"budget": 0.78, "threshold": -0.0299835205078125, "operator": "ge", "actual_fraction": 0.7788873038516405, "metric": 0.18924798946215277, "rescued": 175, "harmed": 399, "missed_rescue": 111, "avoided_harm": 119, "selected_both_correct": 1547, "selected_both_wrong": 63}, {"budget": 0.79, "threshold": -0.033966064453125, "operator": "ge", "actual_fraction": 0.7888730385164051, "metric": 0.19150730156013804, "rescued": 181, "harmed": 403, "missed_rescue": 105, "avoided_harm": 115, "selected_both_correct": 1562, "selected_both_wrong": 66}, {"budget": 0.8, "threshold": -0.037200927734375, "operator": "ge", "actual_fraction": 0.7988587731811697, "metric": 0.19267082077365763, "rescued": 185, "harmed": 410, "missed_rescue": 101, "avoided_harm": 108, "selected_both_correct": 1575, "selected_both_wrong": 70}, {"budget": 0.81, "threshold": -0.04132080078125, "operator": "ge", "actual_fraction": 0.8088445078459344, "metric": 0.1985395820585592, "rescued": 193, "harmed": 413, "missed_rescue": 93, "avoided_harm": 105, "selected_both_correct": 1589, "selected_both_wrong": 73}, {"budget": 0.82, "threshold": -0.0455322265625, "operator": "ge", "actual_fraction": 0.818830242510699, "metric": 0.20050380794511347, "rescued": 198, "harmed": 417, "missed_rescue": 88, "avoided_harm": 101, "selected_both_correct": 1605, "selected_both_wrong": 76}, {"budget": 0.83, "threshold": -0.051849365234375, "operator": "ge", "actual_fraction": 0.8288159771754636, "metric": 0.20479272888770159, "rescued": 205, "harmed": 422, "missed_rescue": 81, "avoided_harm": 96, "selected_both_correct": 1619, "selected_both_wrong": 78}, {"budget": 0.84, "threshold": -0.06878662109375, "operator": "ge", "actual_fraction": 0.8388017118402282, "metric": 0.20996516586206454, "rescued": 211, "harmed": 428, "missed_rescue": 75, "avoided_harm": 90, "selected_both_correct": 1631, "selected_both_wrong": 82}, {"budget": 0.85, "threshold": -0.08477783203125, "operator": "ge", "actual_fraction": 0.8487874465049928, "metric": 0.2153593554624366, "rescued": 218, "harmed": 430, "missed_rescue": 68, "avoided_harm": 88, "selected_both_correct": 1647, "selected_both_wrong": 85}, {"budget": 0.86, "threshold": -0.09912109375, "operator": "ge", "actual_fraction": 0.8587731811697575, "metric": 0.20675118175118173, "rescued": 225, "harmed": 439, "missed_rescue": 61, "avoided_harm": 79, "selected_both_correct": 1658, "selected_both_wrong": 86}, {"budget": 0.87, "threshold": -0.1298828125, "operator": "ge", "actual_fraction": 0.8687589158345221, "metric": 0.2103979829447119, "rescued": 232, "harmed": 441, "missed_rescue": 54, "avoided_harm": 77, "selected_both_correct": 1675, "selected_both_wrong": 88}, {"budget": 0.88, "threshold": -0.1441650390625, "operator": "ge", "actual_fraction": 0.8787446504992867, "metric": 0.2127029758032997, "rescued": 236, "harmed": 447, "missed_rescue": 50, "avoided_harm": 71, "selected_both_correct": 1691, "selected_both_wrong": 90}, {"budget": 0.89, "threshold": -0.182373046875, "operator": "ge", "actual_fraction": 0.8887303851640513, "metric": 0.21408261584117613, "rescued": 244, "harmed": 453, "missed_rescue": 42, "avoided_harm": 65, "selected_both_correct": 1705, "selected_both_wrong": 90}, {"budget": 0.9, "threshold": -0.2049560546875, "operator": "ge", "actual_fraction": 0.8987161198288159, "metric": 0.21447015425888666, "rescued": 251, "harmed": 460, "missed_rescue": 35, "avoided_harm": 58, "selected_both_correct": 1716, "selected_both_wrong": 93}, {"budget": 0.91, "threshold": -0.25830078125, "operator": "gt", "actual_fraction": 0.9072753209700428, "metric": 0.21778616982331747, "rescued": 255, "harmed": 462, "missed_rescue": 31, "avoided_harm": 56, "selected_both_correct": 1730, "selected_both_wrong": 97}, {"budget": 0.92, "threshold": -0.32080078125, "operator": "ge", "actual_fraction": 0.9186875891583453, "metric": 0.22341697071831912, "rescued": 265, "harmed": 468, "missed_rescue": 21, "avoided_harm": 50, "selected_both_correct": 1746, "selected_both_wrong": 97}, {"budget": 0.93, "threshold": -0.453369140625, "operator": "ge", "actual_fraction": 0.9286733238231099, "metric": 0.2260324221654167, "rescued": 269, "harmed": 475, "missed_rescue": 17, "avoided_harm": 43, "selected_both_correct": 1762, "selected_both_wrong": 98}, {"budget": 0.94, "threshold": -0.490234375, "operator": "ge", "actual_fraction": 0.9386590584878745, "metric": 0.22641590025257546, "rescued": 272, "harmed": 478, "missed_rescue": 14, "avoided_harm": 40, "selected_both_correct": 1782, "selected_both_wrong": 100}, {"budget": 0.95, "threshold": -0.611328125, "operator": "ge", "actual_fraction": 0.948644793152639, "metric": 0.2213594019034288, "rescued": 273, "harmed": 486, "missed_rescue": 13, "avoided_harm": 32, "selected_both_correct": 1799, "selected_both_wrong": 102}, {"budget": 0.96, "threshold": -0.70068359375, "operator": "ge", "actual_fraction": 0.9586305278174037, "metric": 0.21682778770996153, "rescued": 278, "harmed": 495, "missed_rescue": 8, "avoided_harm": 23, "selected_both_correct": 1813, "selected_both_wrong": 102}, {"budget": 0.97, "threshold": -0.75048828125, "operator": "ge", "actual_fraction": 0.9686162624821684, "metric": 0.21297215749387616, "rescued": 280, "harmed": 500, "missed_rescue": 6, "avoided_harm": 18, "selected_both_correct": 1831, "selected_both_wrong": 105}, {"budget": 0.98, "threshold": -0.7841796875, "operator": "ge", "actual_fraction": 0.978601997146933, "metric": 0.21025141881293422, "rescued": 282, "harmed": 504, "missed_rescue": 4, "avoided_harm": 14, "selected_both_correct": 1851, "selected_both_wrong": 107}, {"budget": 0.99, "threshold": -0.85107421875, "operator": "ge", "actual_fraction": 0.9885877318116976, "metric": 0.20718702865761687, "rescued": 284, "harmed": 512, "missed_rescue": 2, "avoided_harm": 6, "selected_both_correct": 1869, "selected_both_wrong": 107}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.20666016863629594, "rescued": 286, "harmed": 518, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 1891, "selected_both_wrong": 109}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.11, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.12, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.13, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.14, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.15, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.16, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.17, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.18, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1198, "avoided_harm": 2109, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.19, "threshold": 1.0, "operator": "ge", "actual_fraction": 0.17942743009320905, "metric": 0.16785638110940382, "rescued": 3, "harmed": 361, "missed_rescue": 1195, "avoided_harm": 1748, "selected_both_correct": 1790, "selected_both_wrong": 2}, {"budget": 0.2, "threshold": -0.0010652542114257812, "operator": "ge", "actual_fraction": 0.19340878828229027, "metric": 0.1667569975208824, "rescued": 3, "harmed": 392, "missed_rescue": 1195, "avoided_harm": 1717, "selected_both_correct": 1927, "selected_both_wrong": 2}, {"budget": 0.21, "threshold": -0.0011425018310546875, "operator": "ge", "actual_fraction": 0.20306258322237017, "metric": 0.16670205561344506, "rescued": 4, "harmed": 415, "missed_rescue": 1194, "avoided_harm": 1694, "selected_both_correct": 2018, "selected_both_wrong": 3}, {"budget": 0.22, "threshold": -0.0011968612670898438, "operator": "ge", "actual_fraction": 0.20838881491344874, "metric": 0.16636966208367632, "rescued": 4, "harmed": 426, "missed_rescue": 1194, "avoided_harm": 1683, "selected_both_correct": 2071, "selected_both_wrong": 3}, {"budget": 0.23, "threshold": -0.0012493133544921875, "operator": "ge", "actual_fraction": 0.21537949400798934, "metric": 0.1674861536267502, "rescued": 5, "harmed": 440, "missed_rescue": 1193, "avoided_harm": 1669, "selected_both_correct": 2140, "selected_both_wrong": 3}, {"budget": 0.24, "threshold": -0.0013513565063476562, "operator": "ge", "actual_fraction": 0.23135818908122505, "metric": 0.1659144502174876, "rescued": 5, "harmed": 482, "missed_rescue": 1193, "avoided_harm": 1627, "selected_both_correct": 2289, "selected_both_wrong": 4}, {"budget": 0.25, "threshold": -0.0015010833740234375, "operator": "gt", "actual_fraction": 0.24800266311584554, "metric": 0.16716244725738397, "rescued": 7, "harmed": 526, "missed_rescue": 1191, "avoided_harm": 1583, "selected_both_correct": 2441, "selected_both_wrong": 6}, {"budget": 0.26, "threshold": -0.00157928466796875, "operator": "ge", "actual_fraction": 0.2589880159786951, "metric": 0.1678664772207598, "rescued": 8, "harmed": 550, "missed_rescue": 1190, "avoided_harm": 1559, "selected_both_correct": 2548, "selected_both_wrong": 6}, {"budget": 0.27, "threshold": -0.0017137527465820312, "operator": "gt", "actual_fraction": 0.27430093209054596, "metric": 0.16649682507763983, "rescued": 8, "harmed": 594, "missed_rescue": 1190, "avoided_harm": 1515, "selected_both_correct": 2688, "selected_both_wrong": 6}, {"budget": 0.28, "threshold": -0.0017614364624023438, "operator": "ge", "actual_fraction": 0.281624500665779, "metric": 0.16581256695877195, "rescued": 8, "harmed": 621, "missed_rescue": 1190, "avoided_harm": 1488, "selected_both_correct": 2749, "selected_both_wrong": 6}, {"budget": 0.29, "threshold": -0.0018463134765625, "operator": "ge", "actual_fraction": 0.28761651131824234, "metric": 0.1651274175371034, "rescued": 8, "harmed": 636, "missed_rescue": 1190, "avoided_harm": 1473, "selected_both_correct": 2806, "selected_both_wrong": 6}, {"budget": 0.3, "threshold": -0.001934051513671875, "operator": "ge", "actual_fraction": 0.2969374167776298, "metric": 0.16471286313286057, "rescued": 8, "harmed": 657, "missed_rescue": 1190, "avoided_harm": 1452, "selected_both_correct": 2897, "selected_both_wrong": 6}, {"budget": 0.31, "threshold": -0.0020427703857421875, "operator": "gt", "actual_fraction": 0.30459387483355527, "metric": 0.16408718171666042, "rescued": 8, "harmed": 676, "missed_rescue": 1190, "avoided_harm": 1433, "selected_both_correct": 2969, "selected_both_wrong": 7}, {"budget": 0.32, "threshold": -0.0021495819091796875, "operator": "gt", "actual_fraction": 0.3129161118508655, "metric": 0.1636354375797962, "rescued": 9, "harmed": 701, "missed_rescue": 1189, "avoided_harm": 1408, "selected_both_correct": 3043, "selected_both_wrong": 7}, {"budget": 0.33, "threshold": -0.002208709716796875, "operator": "ge", "actual_fraction": 0.31990679094540614, "metric": 0.16348828966450218, "rescued": 10, "harmed": 720, "missed_rescue": 1188, "avoided_harm": 1389, "selected_both_correct": 3107, "selected_both_wrong": 7}, {"budget": 0.34, "threshold": -0.0023326873779296875, "operator": "ge", "actual_fraction": 0.3322237017310253, "metric": 0.16484645578128018, "rescued": 12, "harmed": 749, "missed_rescue": 1186, "avoided_harm": 1360, "selected_both_correct": 3224, "selected_both_wrong": 7}, {"budget": 0.35, "threshold": -0.0024623870849609375, "operator": "gt", "actual_fraction": 0.34054593874833555, "metric": 0.16526596702267776, "rescued": 13, "harmed": 767, "missed_rescue": 1185, "avoided_harm": 1342, "selected_both_correct": 3304, "selected_both_wrong": 8}, {"budget": 0.36, "threshold": -0.0025615692138671875, "operator": "ge", "actual_fraction": 0.3488681757656458, "metric": 0.16520260386826902, "rescued": 14, "harmed": 792, "missed_rescue": 1184, "avoided_harm": 1317, "selected_both_correct": 3378, "selected_both_wrong": 8}, {"budget": 0.37, "threshold": -0.0026721954345703125, "operator": "gt", "actual_fraction": 0.3581890812250333, "metric": 0.16665740832732326, "rescued": 17, "harmed": 812, "missed_rescue": 1181, "avoided_harm": 1297, "selected_both_correct": 3467, "selected_both_wrong": 8}, {"budget": 0.38, "threshold": -0.0028228759765625, "operator": "gt", "actual_fraction": 0.37017310252996005, "metric": 0.1659164559036785, "rescued": 17, "harmed": 844, "missed_rescue": 1181, "avoided_harm": 1265, "selected_both_correct": 3579, "selected_both_wrong": 8}, {"budget": 0.39, "threshold": -0.0029354095458984375, "operator": "ge", "actual_fraction": 0.37683089214380827, "metric": 0.16604558576074954, "rescued": 18, "harmed": 868, "missed_rescue": 1180, "avoided_harm": 1241, "selected_both_correct": 3633, "selected_both_wrong": 9}, {"budget": 0.4, "threshold": -0.00307464599609375, "operator": "ge", "actual_fraction": 0.38348868175765644, "metric": 0.16627533009733186, "rescued": 19, "harmed": 882, "missed_rescue": 1179, "avoided_harm": 1227, "selected_both_correct": 3698, "selected_both_wrong": 9}, {"budget": 0.41, "threshold": -0.0033626556396484375, "operator": "ge", "actual_fraction": 0.4021304926764314, "metric": 0.16596184167496, "rescued": 21, "harmed": 925, "missed_rescue": 1177, "avoided_harm": 1184, "selected_both_correct": 3876, "selected_both_wrong": 10}, {"budget": 0.42, "threshold": -0.0035381317138671875, "operator": "ge", "actual_fraction": 0.41311584553928093, "metric": 0.16543665040595484, "rescued": 22, "harmed": 952, "missed_rescue": 1176, "avoided_harm": 1157, "selected_both_correct": 3980, "selected_both_wrong": 10}, {"budget": 0.43, "threshold": -0.003765106201171875, "operator": "ge", "actual_fraction": 0.42676431424766975, "metric": 0.16433504278528627, "rescued": 22, "harmed": 996, "missed_rescue": 1176, "avoided_harm": 1113, "selected_both_correct": 4100, "selected_both_wrong": 10}, {"budget": 0.44, "threshold": -0.00392913818359375, "operator": "ge", "actual_fraction": 0.4324234354194407, "metric": 0.1640338494529538, "rescued": 22, "harmed": 1010, "missed_rescue": 1176, "avoided_harm": 1099, "selected_both_correct": 4154, "selected_both_wrong": 10}, {"budget": 0.45, "threshold": -0.004085540771484375, "operator": "ge", "actual_fraction": 0.44107856191744343, "metric": 0.16355335117560724, "rescued": 22, "harmed": 1030, "missed_rescue": 1176, "avoided_harm": 1079, "selected_both_correct": 4238, "selected_both_wrong": 10}, {"budget": 0.46, "threshold": -0.004364013671875, "operator": "ge", "actual_fraction": 0.45772303595206393, "metric": 0.16328618395901012, "rescued": 25, "harmed": 1072, "missed_rescue": 1173, "avoided_harm": 1037, "selected_both_correct": 4393, "selected_both_wrong": 10}, {"budget": 0.47, "threshold": -0.004680633544921875, "operator": "gt", "actual_fraction": 0.4717043941411451, "metric": 0.16343963675474008, "rescued": 27, "harmed": 1106, "missed_rescue": 1171, "avoided_harm": 1003, "selected_both_correct": 4524, "selected_both_wrong": 11}, {"budget": 0.48, "threshold": -0.00490570068359375, "operator": "ge", "actual_fraction": 0.4826897470039947, "metric": 0.16288089231855363, "rescued": 27, "harmed": 1135, "missed_rescue": 1171, "avoided_harm": 974, "selected_both_correct": 4626, "selected_both_wrong": 12}, {"budget": 0.49, "threshold": -0.005039215087890625, "operator": "ge", "actual_fraction": 0.4896804260985353, "metric": 0.16253980428982567, "rescued": 28, "harmed": 1150, "missed_rescue": 1170, "avoided_harm": 959, "selected_both_correct": 4691, "selected_both_wrong": 15}, {"budget": 0.5, "threshold": -0.005321502685546875, "operator": "ge", "actual_fraction": 0.5013315579227696, "metric": 0.16396334310527208, "rescued": 56, "harmed": 1172, "missed_rescue": 1142, "avoided_harm": 937, "selected_both_correct": 4775, "selected_both_wrong": 21}, {"budget": 0.51, "threshold": -0.005619049072265625, "operator": "ge", "actual_fraction": 0.5116511318242344, "metric": 0.16505546455344486, "rescued": 76, "harmed": 1183, "missed_rescue": 1122, "avoided_harm": 926, "selected_both_correct": 4858, "selected_both_wrong": 31}, {"budget": 0.52, "threshold": -0.00586700439453125, "operator": "gt", "actual_fraction": 0.5183089214380826, "metric": 0.16558890552522137, "rescued": 91, "harmed": 1197, "missed_rescue": 1107, "avoided_harm": 912, "selected_both_correct": 4904, "selected_both_wrong": 36}, {"budget": 0.53, "threshold": -0.00605010986328125, "operator": "ge", "actual_fraction": 0.5252996005326231, "metric": 0.1665272377685939, "rescued": 107, "harmed": 1214, "missed_rescue": 1091, "avoided_harm": 895, "selected_both_correct": 4950, "selected_both_wrong": 41}, {"budget": 0.54, "threshold": -0.006439208984375, "operator": "gt", "actual_fraction": 0.5396138482023968, "metric": 0.16808344009752285, "rescued": 144, "harmed": 1241, "missed_rescue": 1054, "avoided_harm": 868, "selected_both_correct": 5049, "selected_both_wrong": 50}, {"budget": 0.55, "threshold": -0.006641387939453125, "operator": "ge", "actual_fraction": 0.5452729693741678, "metric": 0.16859171218698246, "rescued": 156, "harmed": 1251, "missed_rescue": 1042, "avoided_harm": 858, "selected_both_correct": 5088, "selected_both_wrong": 57}, {"budget": 0.56, "threshold": -0.0070648193359375, "operator": "ge", "actual_fraction": 0.559254327563249, "metric": 0.1696257903027952, "rescued": 182, "harmed": 1277, "missed_rescue": 1016, "avoided_harm": 832, "selected_both_correct": 5186, "selected_both_wrong": 75}, {"budget": 0.57, "threshold": -0.007518768310546875, "operator": "ge", "actual_fraction": 0.572237017310253, "metric": 0.17132213898552048, "rescued": 211, "harmed": 1300, "missed_rescue": 987, "avoided_harm": 809, "selected_both_correct": 5276, "selected_both_wrong": 89}, {"budget": 0.58, "threshold": -0.00818634033203125, "operator": "ge", "actual_fraction": 0.5862183754993342, "metric": 0.17290558782298915, "rescued": 244, "harmed": 1328, "missed_rescue": 954, "avoided_harm": 781, "selected_both_correct": 5376, "selected_both_wrong": 96}, {"budget": 0.59, "threshold": -0.00878143310546875, "operator": "gt", "actual_fraction": 0.5998668442077231, "metric": 0.1757330216164513, "rescued": 281, "harmed": 1358, "missed_rescue": 917, "avoided_harm": 751, "selected_both_correct": 5460, "selected_both_wrong": 109}, {"budget": 0.6, "threshold": -0.00955963134765625, "operator": "gt", "actual_fraction": 0.6191744340878829, "metric": 0.17918696387117494, "rescued": 329, "harmed": 1394, "missed_rescue": 869, "avoided_harm": 715, "selected_both_correct": 5591, "selected_both_wrong": 126}, {"budget": 0.61, "threshold": -0.0099334716796875, "operator": "ge", "actual_fraction": 0.6264980026631158, "metric": 0.18029020519999098, "rescued": 347, "harmed": 1407, "missed_rescue": 851, "avoided_harm": 702, "selected_both_correct": 5643, "selected_both_wrong": 131}, {"budget": 0.62, "threshold": -0.010650634765625, "operator": "gt", "actual_fraction": 0.6384820239680427, "metric": 0.18360411382005598, "rescued": 376, "harmed": 1424, "missed_rescue": 822, "avoided_harm": 685, "selected_both_correct": 5729, "selected_both_wrong": 143}, {"budget": 0.63, "threshold": -0.01129150390625, "operator": "ge", "actual_fraction": 0.6461384820239681, "metric": 0.18430872796712833, "rescued": 393, "harmed": 1439, "missed_rescue": 805, "avoided_harm": 670, "selected_both_correct": 5780, "selected_both_wrong": 152}, {"budget": 0.64, "threshold": -0.011871337890625, "operator": "ge", "actual_fraction": 0.653462050599201, "metric": 0.18413764925256976, "rescued": 406, "harmed": 1452, "missed_rescue": 792, "avoided_harm": 657, "selected_both_correct": 5831, "selected_both_wrong": 163}, {"budget": 0.65, "threshold": -0.0125274658203125, "operator": "gt", "actual_fraction": 0.6597869507323568, "metric": 0.18503793034066174, "rescued": 422, "harmed": 1464, "missed_rescue": 776, "avoided_harm": 645, "selected_both_correct": 5873, "selected_both_wrong": 169}, {"budget": 0.66, "threshold": -0.012969970703125, "operator": "gt", "actual_fraction": 0.6661118508655126, "metric": 0.186106537681911, "rescued": 441, "harmed": 1475, "missed_rescue": 757, "avoided_harm": 634, "selected_both_correct": 5915, "selected_both_wrong": 173}, {"budget": 0.67, "threshold": -0.01363372802734375, "operator": "ge", "actual_fraction": 0.6734354194407457, "metric": 0.1880916072476674, "rescued": 461, "harmed": 1484, "missed_rescue": 737, "avoided_harm": 625, "selected_both_correct": 5966, "selected_both_wrong": 181}, {"budget": 0.68, "threshold": -0.01473236083984375, "operator": "ge", "actual_fraction": 0.6820905459387483, "metric": 0.18987442927689613, "rescued": 484, "harmed": 1494, "missed_rescue": 714, "avoided_harm": 615, "selected_both_correct": 6030, "selected_both_wrong": 188}, {"budget": 0.69, "threshold": -0.015716552734375, "operator": "ge", "actual_fraction": 0.6927430093209055, "metric": 0.19160618326349385, "rescued": 511, "harmed": 1512, "missed_rescue": 687, "avoided_harm": 597, "selected_both_correct": 6100, "selected_both_wrong": 201}, {"budget": 0.7, "threshold": -0.017578125, "operator": "gt", "actual_fraction": 0.7090545938748336, "metric": 0.19565888484893704, "rescued": 558, "harmed": 1543, "missed_rescue": 640, "avoided_harm": 566, "selected_both_correct": 6200, "selected_both_wrong": 219}, {"budget": 0.71, "threshold": -0.0189056396484375, "operator": "ge", "actual_fraction": 0.7170439414114514, "metric": 0.1973992538335802, "rescued": 581, "harmed": 1558, "missed_rescue": 617, "avoided_harm": 551, "selected_both_correct": 6250, "selected_both_wrong": 227}, {"budget": 0.72, "threshold": -0.019866943359375, "operator": "ge", "actual_fraction": 0.7243675099866844, "metric": 0.1992894526144666, "rescued": 603, "harmed": 1570, "missed_rescue": 595, "avoided_harm": 539, "selected_both_correct": 6292, "selected_both_wrong": 239}, {"budget": 0.73, "threshold": -0.020965576171875, "operator": "ge", "actual_fraction": 0.7330226364846871, "metric": 0.2014755547124751, "rescued": 624, "harmed": 1582, "missed_rescue": 574, "avoided_harm": 527, "selected_both_correct": 6355, "selected_both_wrong": 247}, {"budget": 0.74, "threshold": -0.021453857421875, "operator": "ge", "actual_fraction": 0.7363515312916112, "metric": 0.20209035743657205, "rescued": 634, "harmed": 1592, "missed_rescue": 564, "avoided_harm": 517, "selected_both_correct": 6374, "selected_both_wrong": 248}, {"budget": 0.75, "threshold": -0.0248870849609375, "operator": "ge", "actual_fraction": 0.7486684420772304, "metric": 0.20546211362830763, "rescued": 665, "harmed": 1612, "missed_rescue": 533, "avoided_harm": 497, "selected_both_correct": 6458, "selected_both_wrong": 261}, {"budget": 0.76, "threshold": -0.0269622802734375, "operator": "ge", "actual_fraction": 0.757323568575233, "metric": 0.20773020664342898, "rescued": 689, "harmed": 1626, "missed_rescue": 509, "avoided_harm": 483, "selected_both_correct": 6514, "selected_both_wrong": 271}, {"budget": 0.77, "threshold": -0.0282135009765625, "operator": "ge", "actual_fraction": 0.7639813581890812, "metric": 0.2112193086129685, "rescued": 707, "harmed": 1633, "missed_rescue": 491, "avoided_harm": 476, "selected_both_correct": 6558, "selected_both_wrong": 282}, {"budget": 0.78, "threshold": -0.0299835205078125, "operator": "ge", "actual_fraction": 0.7733022636484687, "metric": 0.2156222025883268, "rescued": 738, "harmed": 1654, "missed_rescue": 460, "avoided_harm": 455, "selected_both_correct": 6610, "selected_both_wrong": 290}, {"budget": 0.79, "threshold": -0.033966064453125, "operator": "ge", "actual_fraction": 0.784953395472703, "metric": 0.21617077858796782, "rescued": 763, "harmed": 1672, "missed_rescue": 435, "avoided_harm": 437, "selected_both_correct": 6690, "selected_both_wrong": 307}, {"budget": 0.8, "threshold": -0.037200927734375, "operator": "ge", "actual_fraction": 0.7939414114513982, "metric": 0.21811777255990603, "rescued": 788, "harmed": 1693, "missed_rescue": 410, "avoided_harm": 416, "selected_both_correct": 6742, "selected_both_wrong": 317}, {"budget": 0.81, "threshold": -0.04132080078125, "operator": "ge", "actual_fraction": 0.8039280958721704, "metric": 0.22157947433765235, "rescued": 814, "harmed": 1705, "missed_rescue": 384, "avoided_harm": 404, "selected_both_correct": 6809, "selected_both_wrong": 332}, {"budget": 0.82, "threshold": -0.0455322265625, "operator": "ge", "actual_fraction": 0.8122503328894807, "metric": 0.22355438821966017, "rescued": 835, "harmed": 1722, "missed_rescue": 363, "avoided_harm": 387, "selected_both_correct": 6857, "selected_both_wrong": 346}, {"budget": 0.83, "threshold": -0.051849365234375, "operator": "ge", "actual_fraction": 0.8215712383488681, "metric": 0.22762279758152631, "rescued": 863, "harmed": 1740, "missed_rescue": 335, "avoided_harm": 369, "selected_both_correct": 6911, "selected_both_wrong": 358}, {"budget": 0.84, "threshold": -0.06878662109375, "operator": "ge", "actual_fraction": 0.8388814913448736, "metric": 0.23330777488772814, "rescued": 909, "harmed": 1767, "missed_rescue": 289, "avoided_harm": 342, "selected_both_correct": 7021, "selected_both_wrong": 383}, {"budget": 0.85, "threshold": -0.08477783203125, "operator": "ge", "actual_fraction": 0.8535286284953395, "metric": 0.23874788475407085, "rescued": 947, "harmed": 1794, "missed_rescue": 251, "avoided_harm": 315, "selected_both_correct": 7110, "selected_both_wrong": 405}, {"budget": 0.86, "threshold": -0.09912109375, "operator": "ge", "actual_fraction": 0.8638482023968043, "metric": 0.2419731256831217, "rescued": 974, "harmed": 1808, "missed_rescue": 224, "avoided_harm": 301, "selected_both_correct": 7179, "selected_both_wrong": 419}, {"budget": 0.87, "threshold": -0.1298828125, "operator": "ge", "actual_fraction": 0.8745006657789614, "metric": 0.2400850101807912, "rescued": 996, "harmed": 1832, "missed_rescue": 202, "avoided_harm": 277, "selected_both_correct": 7252, "selected_both_wrong": 428}, {"budget": 0.88, "threshold": -0.1441650390625, "operator": "ge", "actual_fraction": 0.8764980026631158, "metric": 0.24058030940047917, "rescued": 1000, "harmed": 1834, "missed_rescue": 198, "avoided_harm": 275, "selected_both_correct": 7268, "selected_both_wrong": 430}, {"budget": 0.89, "threshold": -0.182373046875, "operator": "ge", "actual_fraction": 0.8841544607190412, "metric": 0.24227931512045134, "rescued": 1017, "harmed": 1852, "missed_rescue": 181, "avoided_harm": 257, "selected_both_correct": 7321, "selected_both_wrong": 434}, {"budget": 0.9, "threshold": -0.2049560546875, "operator": "ge", "actual_fraction": 0.8891478029294274, "metric": 0.2414044779527529, "rescued": 1030, "harmed": 1861, "missed_rescue": 168, "avoided_harm": 248, "selected_both_correct": 7354, "selected_both_wrong": 439}, {"budget": 0.91, "threshold": -0.25830078125, "operator": "gt", "actual_fraction": 0.8981358189081226, "metric": 0.24325574507940786, "rescued": 1055, "harmed": 1885, "missed_rescue": 143, "avoided_harm": 224, "selected_both_correct": 7403, "selected_both_wrong": 449}, {"budget": 0.92, "threshold": -0.32080078125, "operator": "ge", "actual_fraction": 0.9054593874833555, "metric": 0.24437346394419443, "rescued": 1069, "harmed": 1898, "missed_rescue": 129, "avoided_harm": 211, "selected_both_correct": 7455, "selected_both_wrong": 458}, {"budget": 0.93, "threshold": -0.453369140625, "operator": "ge", "actual_fraction": 0.9221038615179761, "metric": 0.24877865873563798, "rescued": 1105, "harmed": 1927, "missed_rescue": 93, "avoided_harm": 182, "selected_both_correct": 7575, "selected_both_wrong": 473}, {"budget": 0.94, "threshold": -0.490234375, "operator": "ge", "actual_fraction": 0.9254327563249002, "metric": 0.2523047077304935, "rescued": 1113, "harmed": 1932, "missed_rescue": 85, "avoided_harm": 177, "selected_both_correct": 7597, "selected_both_wrong": 478}, {"budget": 0.95, "threshold": -0.611328125, "operator": "ge", "actual_fraction": 0.9364181091877497, "metric": 0.2527194116218445, "rescued": 1129, "harmed": 1959, "missed_rescue": 69, "avoided_harm": 150, "selected_both_correct": 7676, "selected_both_wrong": 488}, {"budget": 0.96, "threshold": -0.70068359375, "operator": "ge", "actual_fraction": 0.953728362183755, "metric": 0.24014411561115867, "rescued": 1148, "harmed": 2005, "missed_rescue": 50, "avoided_harm": 104, "selected_both_correct": 7804, "selected_both_wrong": 503}, {"budget": 0.97, "threshold": -0.75048828125, "operator": "ge", "actual_fraction": 0.96271637816245, "metric": 0.2331497088327334, "rescued": 1153, "harmed": 2028, "missed_rescue": 45, "avoided_harm": 81, "selected_both_correct": 7880, "selected_both_wrong": 507}, {"budget": 0.98, "threshold": -0.7841796875, "operator": "ge", "actual_fraction": 0.9700399467376831, "metric": 0.23063278452496924, "rescued": 1156, "harmed": 2044, "missed_rescue": 42, "avoided_harm": 65, "selected_both_correct": 7943, "selected_both_wrong": 513}, {"budget": 0.99, "threshold": -0.85107421875, "operator": "ge", "actual_fraction": 0.987017310252996, "metric": 0.2242415410777232, "rescued": 1174, "harmed": 2087, "missed_rescue": 24, "avoided_harm": 22, "selected_both_correct": 8072, "selected_both_wrong": 527}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.22361115727328065, "rescued": 1198, "harmed": 2109, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 8176, "selected_both_wrong": 533}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.2870655827112144, "actual_fraction": 0.4573466476462197, "metric_std": 0.0009388358187959056, "actual_fraction_std": 0.004465756303137945, "per_seed": [{"seed": 2026, "budget": 0.46, "threshold": -0.3365704427855552, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.2885771696562181, "actual_fraction": 0.45934379457917257, "rescue": 274, "harm": 208, "missed_rescue": 12}, {"seed": 2027, "budget": 0.46, "threshold": -0.27194246924330945, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.2864514777488093, "actual_fraction": 0.45934379457917257, "rescue": 274, "harm": 209, "missed_rescue": 12}, {"seed": 2028, "budget": 0.45, "threshold": -0.21528634559799156, "operator": "ge", "alpha": 0.75, "epoch": 5, "metric": 0.2873963106534258, "actual_fraction": 0.44935805991440797, "rescue": 272, "harm": 204, "missed_rescue": 14}, {"seed": 2029, "budget": 0.46, "threshold": -0.32222263090817316, "operator": "ge", "alpha": 0.75, "epoch": 4, "metric": 0.2864514777488093, "actual_fraction": 0.45934379457917257, "rescue": 274, "harm": 209, "missed_rescue": 12}, {"seed": 2030, "budget": 0.46, "threshold": -0.27538920052695537, "operator": "ge", "alpha": 0.5, "epoch": 4, "metric": 0.2864514777488093, "actual_fraction": 0.45934379457917257, "rescue": 274, "harm": 209, "missed_rescue": 12}], "rescued": 273.6, "harmed": 207.8, "missed_rescue": 12.4, "avoided_harm": 310.2}, "validation_curve": [{"budget": 0.0, "metric": 0.18254072168965788, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.18434422127117828, "metric_std": 0.0005515030561238408, "actual_fraction": 0.009985734664764621, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.18997710102107082, "metric_std": 0.002624894491280287, "actual_fraction": 0.019971469329529243, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.19764676372636134, "metric_std": 0.0007334807981670814, "actual_fraction": 0.029957203994293864, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.20207394394731062, "metric_std": 0.00041998264186959773, "actual_fraction": 0.039942938659058486, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.20610157637498566, "metric_std": 0.00047827083038268133, "actual_fraction": 0.04992867332382311, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.20995850066288096, "metric_std": 0.0006552703907508667, "actual_fraction": 0.05991440798858773, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.2144647207667772, "metric_std": 0.0007603008164758542, "actual_fraction": 0.06990014265335236, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.21914548075155887, "metric_std": 0.001545090668262203, "actual_fraction": 0.07988587731811697, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.22386486747809173, "metric_std": 0.001168357183349575, "actual_fraction": 0.0898716119828816, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.2259888489290806, "metric_std": 0.0011947047915756095, "actual_fraction": 0.09985734664764621, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.23175358146526367, "metric_std": 0.007519269761779462, "actual_fraction": 0.10984308131241086, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.12, "metric": 0.2323717338784051, "metric_std": 0.006928865801909536, "actual_fraction": 0.11982881597717546, "actual_fraction_std": 0.0}, {"budget": 0.13, "metric": 0.2394843025589184, "metric_std": 0.007293619826326822, "actual_fraction": 0.12981455064194009, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.2416449704124526, "metric_std": 0.004708419880954656, "actual_fraction": 0.1398002853067047, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.24271855527724306, "metric_std": 0.006216415885590686, "actual_fraction": 0.14978601997146934, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.244039721330656, "metric_std": 0.00845736249960185, "actual_fraction": 0.15977175463623394, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.2451257033456206, "metric_std": 0.007940751950235796, "actual_fraction": 0.16975748930099857, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.24647662899334763, "metric_std": 0.006655452811919424, "actual_fraction": 0.1797432239657632, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.24664687378316055, "metric_std": 0.007084985029861769, "actual_fraction": 0.18972895863052786, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.2475344618872933, "metric_std": 0.005780392663676702, "actual_fraction": 0.19971469329529243, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.24817756922689876, "metric_std": 0.004851820709146102, "actual_fraction": 0.20970042796005703, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.24845781845105935, "metric_std": 0.004995007288815537, "actual_fraction": 0.2196861626248217, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.23, "metric": 0.2508943862470604, "metric_std": 0.004121710639286569, "actual_fraction": 0.22967189728958629, "actual_fraction_std": 0.0}, {"budget": 0.24, "metric": 0.2540784065722141, "metric_std": 0.0017180097595275454, "actual_fraction": 0.2396576319543509, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.2530134253201837, "metric_std": 0.00927035017136823, "actual_fraction": 0.24964336661911551, "actual_fraction_std": 0.0}, {"budget": 0.26, "metric": 0.2508917661084983, "metric_std": 0.010597839295142691, "actual_fraction": 0.25962910128388017, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.2534983533216329, "metric_std": 0.013844304911045303, "actual_fraction": 0.26961483594864477, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.2563931837209062, "metric_std": 0.01116880193840821, "actual_fraction": 0.2796005706134094, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.2578710402839143, "metric_std": 0.010969361516562604, "actual_fraction": 0.289586305278174, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.2614141810318732, "metric_std": 0.010610497666577916, "actual_fraction": 0.2995720399429387, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.2623732869581253, "metric_std": 0.009839034193628297, "actual_fraction": 0.3095577746077033, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.2637219117385352, "metric_std": 0.01171958089205847, "actual_fraction": 0.3195435092724679, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.26649147277126956, "metric_std": 0.012684187329627266, "actual_fraction": 0.3295292439372325, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.26818195737625744, "metric_std": 0.01233479777526885, "actual_fraction": 0.33951497860199714, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.27161298161417013, "metric_std": 0.012803440495996063, "actual_fraction": 0.34950071326676174, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.2717587392672331, "metric_std": 0.007866720766423288, "actual_fraction": 0.3594864479315264, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.2685859831749695, "metric_std": 0.005796195300748171, "actual_fraction": 0.36947218259629094, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.2713005618709616, "metric_std": 0.004861194021787204, "actual_fraction": 0.3794579172610557, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.2721368841262496, "metric_std": 0.002717384517395365, "actual_fraction": 0.38944365192582026, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.2721377066716427, "metric_std": 0.004423200546548374, "actual_fraction": 0.39942938659058486, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.274310633412408, "metric_std": 0.005291481832144318, "actual_fraction": 0.4094151212553495, "actual_fraction_std": 0.0}, {"budget": 0.42, "metric": 0.2772570671476548, "metric_std": 0.004457786575880585, "actual_fraction": 0.41940085592011406, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.2808832751362349, "metric_std": 0.003545700139488753, "actual_fraction": 0.4293865905848787, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.44, "metric": 0.28059524227729, "metric_std": 0.002198279801936177, "actual_fraction": 0.4393723252496434, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.45, "metric": 0.28536483839269866, "metric_std": 0.0017697866956918947, "actual_fraction": 0.44935805991440797, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.28687661613029103, "metric_std": 0.0009506383208374531, "actual_fraction": 0.45934379457917257, "actual_fraction_std": 0.0}, {"budget": 0.47, "metric": 0.28562294740066746, "metric_std": 0.001831885777502801, "actual_fraction": 0.4693295292439372, "actual_fraction_std": 0.0}, {"budget": 0.48, "metric": 0.28536664622318386, "metric_std": 0.0012589813745197491, "actual_fraction": 0.4793152639087018, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.28452382276782323, "metric_std": 0.0010795013878046138, "actual_fraction": 0.4893009985734665, "actual_fraction_std": 0.0}, {"budget": 0.5, "metric": 0.28338562813203794, "metric_std": 0.0015902502133405935, "actual_fraction": 0.49928673323823103, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.28195285909023154, "metric_std": 0.002138153523802313, "actual_fraction": 0.5092724679029957, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.28059279629856626, "metric_std": 0.0028225916253023318, "actual_fraction": 0.5192582025677603, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.2791310204702136, "metric_std": 0.0018235121440001167, "actual_fraction": 0.5292439372325249, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.2787700394613709, "metric_std": 0.003318791334185605, "actual_fraction": 0.5392296718972895, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.2777378969095856, "metric_std": 0.0035050734377836735, "actual_fraction": 0.5492154065620543, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.2776185101265654, "metric_std": 0.0034134603904219327, "actual_fraction": 0.5592011412268189, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.27622342375785164, "metric_std": 0.0053962023150829555, "actual_fraction": 0.5691868758915835, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.27486932707331624, "metric_std": 0.003291222150567497, "actual_fraction": 0.579172610556348, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.2720509348014768, "metric_std": 0.00338461441967693, "actual_fraction": 0.5891583452211127, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.27149801076781843, "metric_std": 0.0033277101145030605, "actual_fraction": 0.5991440798858774, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.27035730180261824, "metric_std": 0.003217157143841743, "actual_fraction": 0.609129814550642, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.2700322082850198, "metric_std": 0.002726540781464275, "actual_fraction": 0.6191155492154066, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.26851166105449237, "metric_std": 0.003465900324037174, "actual_fraction": 0.6291012838801712, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.26850904466961917, "metric_std": 0.0032583162348606733, "actual_fraction": 0.6390870185449358, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.2642508394405022, "metric_std": 0.003062139317796734, "actual_fraction": 0.6490727532097005, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.2636867547430462, "metric_std": 0.0033977646106750545, "actual_fraction": 0.659058487874465, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.2603441807437822, "metric_std": 0.0031457156032240296, "actual_fraction": 0.6690442225392297, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.25803019103433045, "metric_std": 0.004923241567805681, "actual_fraction": 0.6790299572039943, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.2568785829203348, "metric_std": 0.004028408399418567, "actual_fraction": 0.6890156918687589, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.25553286134077347, "metric_std": 0.0020065934297446826, "actual_fraction": 0.6990014265335235, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.251662654410262, "metric_std": 0.0013638304883785116, "actual_fraction": 0.7089871611982882, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.2528094223953553, "metric_std": 0.0011740596213345585, "actual_fraction": 0.7189728958630528, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.2512948594908143, "metric_std": 0.002110275300191361, "actual_fraction": 0.7289586305278174, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.24789179078676987, "metric_std": 0.0028620705858186696, "actual_fraction": 0.7389443651925819, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.24679339739924222, "metric_std": 0.00362058861933755, "actual_fraction": 0.7489300998573466, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.24547162161439479, "metric_std": 0.003109561597102724, "actual_fraction": 0.7589158345221114, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.2437995365085793, "metric_std": 0.004149422297930581, "actual_fraction": 0.7689015691868759, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.24247457715580487, "metric_std": 0.0031013335892182024, "actual_fraction": 0.7788873038516405, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.24001175627285204, "metric_std": 0.003158867544997836, "actual_fraction": 0.7888730385164051, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.2385910479395957, "metric_std": 0.004021562629976042, "actual_fraction": 0.7988587731811697, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.23624029018185225, "metric_std": 0.00520835199331964, "actual_fraction": 0.8088445078459344, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.2339832776118122, "metric_std": 0.004026209770235739, "actual_fraction": 0.818830242510699, "actual_fraction_std": 0.0}, {"budget": 0.83, "metric": 0.23190792616712796, "metric_std": 0.002685786146576833, "actual_fraction": 0.8288159771754635, "actual_fraction_std": 0.0}, {"budget": 0.84, "metric": 0.22936905878879554, "metric_std": 0.003015168951690643, "actual_fraction": 0.8388017118402281, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.22806304194754098, "metric_std": 0.0037539743426702418, "actual_fraction": 0.8487874465049929, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.86, "metric": 0.22629480542568353, "metric_std": 0.0044827850909483595, "actual_fraction": 0.8587731811697574, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.87, "metric": 0.2254302184302867, "metric_std": 0.004680064465334867, "actual_fraction": 0.8687589158345222, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.22349874016511176, "metric_std": 0.002934100156020553, "actual_fraction": 0.8787446504992869, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.89, "metric": 0.22275194969701087, "metric_std": 0.0028272198960455344, "actual_fraction": 0.8887303851640513, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.2219360294991783, "metric_std": 0.0036157314869176677, "actual_fraction": 0.8987161198288159, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.22122647898545775, "metric_std": 0.0035279878472835743, "actual_fraction": 0.9087018544935808, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.21854567524413024, "metric_std": 0.002178987475374233, "actual_fraction": 0.9186875891583451, "actual_fraction_std": 0.0}, {"budget": 0.93, "metric": 0.21647097470368865, "metric_std": 0.002392808103681056, "actual_fraction": 0.9286733238231101, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.94, "metric": 0.2144932810684775, "metric_std": 0.0017002459273824713, "actual_fraction": 0.9386590584878745, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.21310408672849612, "metric_std": 0.0015109189412829272, "actual_fraction": 0.948644793152639, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.21192541020125724, "metric_std": 0.0015486007169792605, "actual_fraction": 0.9586305278174037, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.21012130666285045, "metric_std": 0.0012673860367408976, "actual_fraction": 0.9686162624821684, "actual_fraction_std": 0.0}, {"budget": 0.98, "metric": 0.20927860075986607, "metric_std": 0.001127610407678773, "actual_fraction": 0.978601997146933, "actual_fraction_std": 0.0}, {"budget": 0.99, "metric": 0.20810175412097193, "metric_std": 0.0013947764704022997, "actual_fraction": 0.9885877318116976, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.20666016863629594, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.258804362191651, "actual_fraction": 0.4531957390146471, "metric_std": 0.0009732953941336366, "actual_fraction_std": 0.007211495698030496, "per_seed": [{"seed": 2026, "budget": 0.46, "threshold": -0.3365704427855552, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.2592463564955129, "actual_fraction": 0.46171770972037285, "rescue": 1144, "harm": 813, "missed_rescue": 54}, {"seed": 2027, "budget": 0.46, "threshold": -0.27194246924330945, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.2593611863895939, "actual_fraction": 0.45505992010652463, "rescue": 1145, "harm": 810, "missed_rescue": 53}, {"seed": 2028, "budget": 0.45, "threshold": -0.21528634559799156, "operator": "ge", "alpha": 0.75, "epoch": 5, "metric": 0.25707853728930236, "actual_fraction": 0.44174434087882825, "rescue": 1124, "harm": 782, "missed_rescue": 74}, {"seed": 2029, "budget": 0.46, "threshold": -0.32222263090817316, "operator": "ge", "alpha": 0.75, "epoch": 4, "metric": 0.2593111532108809, "actual_fraction": 0.4533954727030626, "rescue": 1142, "harm": 805, "missed_rescue": 56}, {"seed": 2030, "budget": 0.46, "threshold": -0.27538920052695537, "operator": "ge", "alpha": 0.5, "epoch": 4, "metric": 0.25902457757296465, "actual_fraction": 0.4540612516644474, "rescue": 1143, "harm": 807, "missed_rescue": 55}], "rescued": 1139.6, "harmed": 803.4, "missed_rescue": 58.4, "avoided_harm": 1305.6}, "test_curve": [{"budget": 0.0, "metric": 0.18129624718318374, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.18284451454595305, "metric_std": 0.001178128207896284, "actual_fraction": 0.008388814913448735, "actual_fraction_std": 0.0013398543140810992}, {"budget": 0.02, "metric": 0.18476705780992775, "metric_std": 0.0012848978533708632, "actual_fraction": 0.014980026631158456, "actual_fraction_std": 0.002968134903483191}, {"budget": 0.03, "metric": 0.1875827851042889, "metric_std": 0.001345901143805384, "actual_fraction": 0.02197070572569907, "actual_fraction_std": 0.0016137416302983787}, {"budget": 0.04, "metric": 0.19038259848278677, "metric_std": 0.001278540888525995, "actual_fraction": 0.03162450066577896, "actual_fraction_std": 0.0025461149368805505}, {"budget": 0.05, "metric": 0.19390844047803088, "metric_std": 0.0021103487181895948, "actual_fraction": 0.042942743009320904, "actual_fraction_std": 0.001630818736872957}, {"budget": 0.06, "metric": 0.19783434529209248, "metric_std": 0.0026003633991075695, "actual_fraction": 0.053928095872170435, "actual_fraction_std": 0.0021829023050272942}, {"budget": 0.07, "metric": 0.20121507544967768, "metric_std": 0.0021000892759842353, "actual_fraction": 0.06611185086551266, "actual_fraction_std": 0.004907156794018859}, {"budget": 0.08, "metric": 0.20545424121560324, "metric_std": 0.0009175355620607567, "actual_fraction": 0.07862849533954727, "actual_fraction_std": 0.005006640082695263}, {"budget": 0.09, "metric": 0.20847607280936326, "metric_std": 0.0015141295593193997, "actual_fraction": 0.09027962716378161, "actual_fraction_std": 0.007381594360643187}, {"budget": 0.1, "metric": 0.21127611876456998, "metric_std": 0.0007844927959688787, "actual_fraction": 0.10725699067909453, "actual_fraction_std": 0.0033817377095074013}, {"budget": 0.11, "metric": 0.21225833352267384, "metric_std": 0.0032091397012058583, "actual_fraction": 0.11830892143808255, "actual_fraction_std": 0.004734760469733455}, {"budget": 0.12, "metric": 0.21479788399710315, "metric_std": 0.003368898716092574, "actual_fraction": 0.1273635153129161, "actual_fraction_std": 0.00433140257838193}, {"budget": 0.13, "metric": 0.2175707125604182, "metric_std": 0.003391123381105952, "actual_fraction": 0.137017310252996, "actual_fraction_std": 0.0036964225412759793}, {"budget": 0.14, "metric": 0.21972986199678587, "metric_std": 0.004349846149853621, "actual_fraction": 0.14620505992010652, "actual_fraction_std": 0.005001103599654079}, {"budget": 0.15, "metric": 0.22210424298031733, "metric_std": 0.005820945762125215, "actual_fraction": 0.153595206391478, "actual_fraction_std": 0.008367387509581908}, {"budget": 0.16, "metric": 0.22393727679793116, "metric_std": 0.006599082529845246, "actual_fraction": 0.16364846870838884, "actual_fraction_std": 0.013504813800276896}, {"budget": 0.17, "metric": 0.22565325698446684, "metric_std": 0.006275143272393994, "actual_fraction": 0.17137150466045273, "actual_fraction_std": 0.011878138548275545}, {"budget": 0.18, "metric": 0.22641873975159948, "metric_std": 0.006582677113863102, "actual_fraction": 0.1826231691078562, "actual_fraction_std": 0.008576669909637936}, {"budget": 0.19, "metric": 0.22830350472387498, "metric_std": 0.006130604924052559, "actual_fraction": 0.19207723035952062, "actual_fraction_std": 0.005086786547914824}, {"budget": 0.2, "metric": 0.23073833365715446, "metric_std": 0.006708022435185637, "actual_fraction": 0.2036617842876165, "actual_fraction_std": 0.007343967474962084}, {"budget": 0.21, "metric": 0.23233407233081582, "metric_std": 0.005961571990709476, "actual_fraction": 0.21797603195739015, "actual_fraction_std": 0.005451668733937228}, {"budget": 0.22, "metric": 0.23534019820111554, "metric_std": 0.005688868734575132, "actual_fraction": 0.23275632490013315, "actual_fraction_std": 0.005704952659458104}, {"budget": 0.23, "metric": 0.2367590112106815, "metric_std": 0.005762449862657553, "actual_fraction": 0.24280958721704393, "actual_fraction_std": 0.0065173265420819144}, {"budget": 0.24, "metric": 0.23836925639846793, "metric_std": 0.004751826368742579, "actual_fraction": 0.25146471371504664, "actual_fraction_std": 0.008940422395185783}, {"budget": 0.25, "metric": 0.2410030288786155, "metric_std": 0.00400127190115729, "actual_fraction": 0.26131824234354195, "actual_fraction_std": 0.008988015978695082}, {"budget": 0.26, "metric": 0.2422802441121794, "metric_std": 0.004574852578103662, "actual_fraction": 0.27203728362183754, "actual_fraction_std": 0.011019094370405375}, {"budget": 0.27, "metric": 0.24262436004694724, "metric_std": 0.004339912221458983, "actual_fraction": 0.28382157123834884, "actual_fraction_std": 0.014183201583473113}, {"budget": 0.28, "metric": 0.24525036499337735, "metric_std": 0.0027526729137504835, "actual_fraction": 0.29707057256990677, "actual_fraction_std": 0.010462478597259997}, {"budget": 0.29, "metric": 0.24976067858231416, "metric_std": 0.003242101235913108, "actual_fraction": 0.30679094540612517, "actual_fraction_std": 0.009124451479130631}, {"budget": 0.3, "metric": 0.24984923057300074, "metric_std": 0.0053445933199205205, "actual_fraction": 0.3144474034620506, "actual_fraction_std": 0.012322757456396034}, {"budget": 0.31, "metric": 0.25081200762249806, "metric_std": 0.0050076777522744115, "actual_fraction": 0.3228362183754993, "actual_fraction_std": 0.007392845082551959}, {"budget": 0.32, "metric": 0.2518534614570558, "metric_std": 0.004423218713491799, "actual_fraction": 0.3329560585885486, "actual_fraction_std": 0.006499448661446494}, {"budget": 0.33, "metric": 0.2524345018272001, "metric_std": 0.0034032715271313857, "actual_fraction": 0.3434753661784288, "actual_fraction_std": 0.007071061543196649}, {"budget": 0.34, "metric": 0.2531785629653481, "metric_std": 0.0030179654238542118, "actual_fraction": 0.35173102529960054, "actual_fraction_std": 0.00817309775734964}, {"budget": 0.35, "metric": 0.2551813777526542, "metric_std": 0.003720537124957887, "actual_fraction": 0.36378162450066576, "actual_fraction_std": 0.0074339520545648385}, {"budget": 0.36, "metric": 0.2578445076807395, "metric_std": 0.003935125883351638, "actual_fraction": 0.3699733688415446, "actual_fraction_std": 0.006857038494620512}, {"budget": 0.37, "metric": 0.2591210983072941, "metric_std": 0.0029425373799771993, "actual_fraction": 0.3788282290279627, "actual_fraction_std": 0.005037531940886015}, {"budget": 0.38, "metric": 0.2606117629803265, "metric_std": 0.0024297961266108088, "actual_fraction": 0.38508655126498004, "actual_fraction_std": 0.0035182496912424493}, {"budget": 0.39, "metric": 0.260643968774322, "metric_std": 0.0016635750296879598, "actual_fraction": 0.3934753661784287, "actual_fraction_std": 0.0017614855599631268}, {"budget": 0.4, "metric": 0.25980204334964574, "metric_std": 0.0023117130498130586, "actual_fraction": 0.401930758988016, "actual_fraction_std": 0.0012140201417317335}, {"budget": 0.41, "metric": 0.26140183383963894, "metric_std": 0.00168581533658272, "actual_fraction": 0.41105193075898805, "actual_fraction_std": 0.003470682162178004}, {"budget": 0.42, "metric": 0.2620846424654545, "metric_std": 0.00197577693731104, "actual_fraction": 0.4192410119840213, "actual_fraction_std": 0.002869422362118536}, {"budget": 0.43, "metric": 0.2622701701802889, "metric_std": 0.0019549591066573865, "actual_fraction": 0.4255659121171771, "actual_fraction_std": 0.0024754983279065305}, {"budget": 0.44, "metric": 0.2603384859498789, "metric_std": 0.003286217048778142, "actual_fraction": 0.43621837549933423, "actual_fraction_std": 0.0019353451203393426}, {"budget": 0.45, "metric": 0.25814288339417285, "metric_std": 0.0007694921814045058, "actual_fraction": 0.44407456724367506, "actual_fraction_std": 0.0014510316057059485}, {"budget": 0.46, "metric": 0.2592298032114456, "metric_std": 0.0001292766984256986, "actual_fraction": 0.4551930758988016, "actual_fraction_std": 0.0038433961111509042}, {"budget": 0.47, "metric": 0.261388298884564, "metric_std": 0.0031582167529478, "actual_fraction": 0.46704394141145145, "actual_fraction_std": 0.004869753275075815}, {"budget": 0.48, "metric": 0.26032560612228384, "metric_std": 0.002942082978190543, "actual_fraction": 0.47470039946737685, "actual_fraction_std": 0.005305385303098938}, {"budget": 0.49, "metric": 0.2589339662007738, "metric_std": 0.002731899976509469, "actual_fraction": 0.48508655126498007, "actual_fraction_std": 0.007129588129432321}, {"budget": 0.5, "metric": 0.25576817523437734, "metric_std": 0.003783071495670512, "actual_fraction": 0.4946737683089214, "actual_fraction_std": 0.005540385145503754}, {"budget": 0.51, "metric": 0.25489961013444623, "metric_std": 0.0033987743437838536, "actual_fraction": 0.5017310252996005, "actual_fraction_std": 0.006746249876062309}, {"budget": 0.52, "metric": 0.2536076495310996, "metric_std": 0.003573660937825687, "actual_fraction": 0.511318242343542, "actual_fraction_std": 0.004515532611934283}, {"budget": 0.53, "metric": 0.25100327394097455, "metric_std": 0.002794855911192852, "actual_fraction": 0.5233022636484687, "actual_fraction_std": 0.005289696598558826}, {"budget": 0.54, "metric": 0.24969176611674, "metric_std": 0.0025421978022182284, "actual_fraction": 0.5336884154460719, "actual_fraction_std": 0.0054921720302637455}, {"budget": 0.55, "metric": 0.2490012131648555, "metric_std": 0.0029179021175637144, "actual_fraction": 0.5454061251664448, "actual_fraction_std": 0.0011190237226346125}, {"budget": 0.56, "metric": 0.24777943114391204, "metric_std": 0.0028616863393727088, "actual_fraction": 0.5549933422103861, "actual_fraction_std": 0.0019324800630758701}, {"budget": 0.57, "metric": 0.2468952108887433, "metric_std": 0.0034145549399608125, "actual_fraction": 0.5674434087882823, "actual_fraction_std": 0.007386096705899034}, {"budget": 0.58, "metric": 0.24712282121707788, "metric_std": 0.003260403898193415, "actual_fraction": 0.5754327563249001, "actual_fraction_std": 0.009434935925882282}, {"budget": 0.59, "metric": 0.24723916111936103, "metric_std": 0.00322438461292551, "actual_fraction": 0.5885486018641812, "actual_fraction_std": 0.00914689192012921}, {"budget": 0.6, "metric": 0.24714673277109767, "metric_std": 0.00242510207951213, "actual_fraction": 0.6059254327563249, "actual_fraction_std": 0.005446584643738767}, {"budget": 0.61, "metric": 0.24717051062019055, "metric_std": 0.004121201916738283, "actual_fraction": 0.6139147802929428, "actual_fraction_std": 0.005597098835713557}, {"budget": 0.62, "metric": 0.24720733689109725, "metric_std": 0.005026154350201276, "actual_fraction": 0.6261651131824234, "actual_fraction_std": 0.008904404710449587}, {"budget": 0.63, "metric": 0.24779225260449475, "metric_std": 0.004468695700283776, "actual_fraction": 0.6378828229027962, "actual_fraction_std": 0.0075228603619791595}, {"budget": 0.64, "metric": 0.24649953933928667, "metric_std": 0.004507789107578152, "actual_fraction": 0.6474700399467378, "actual_fraction_std": 0.007916193043005452}, {"budget": 0.65, "metric": 0.24506131732120745, "metric_std": 0.0037783583029782275, "actual_fraction": 0.657589880159787, "actual_fraction_std": 0.008210299122818267}, {"budget": 0.66, "metric": 0.2436059531433486, "metric_std": 0.004481148410320459, "actual_fraction": 0.6705725699067909, "actual_fraction_std": 0.006491771640858802}, {"budget": 0.67, "metric": 0.24176074113596685, "metric_std": 0.0043580704913624355, "actual_fraction": 0.6830226364846871, "actual_fraction_std": 0.008320239440688153}, {"budget": 0.68, "metric": 0.2403635259228471, "metric_std": 0.00365225277836571, "actual_fraction": 0.6916111850865512, "actual_fraction_std": 0.007356029018666958}, {"budget": 0.69, "metric": 0.23982087042341188, "metric_std": 0.0026510140601274388, "actual_fraction": 0.7018641810918774, "actual_fraction_std": 0.007977548988884742}, {"budget": 0.7, "metric": 0.23798055004644342, "metric_std": 0.0037637858083510727, "actual_fraction": 0.7117177097203727, "actual_fraction_std": 0.008177841875191896}, {"budget": 0.71, "metric": 0.23822661932038963, "metric_std": 0.005209066668420255, "actual_fraction": 0.7194407456724368, "actual_fraction_std": 0.006452390887711536}, {"budget": 0.72, "metric": 0.2381972444901154, "metric_std": 0.005759186340966202, "actual_fraction": 0.7301597869507324, "actual_fraction_std": 0.008524830994690985}, {"budget": 0.73, "metric": 0.2366319109793776, "metric_std": 0.0061796981063572546, "actual_fraction": 0.7398801597869508, "actual_fraction_std": 0.01213244016108159}, {"budget": 0.74, "metric": 0.23556796104146477, "metric_std": 0.007041984070124362, "actual_fraction": 0.7486684420772304, "actual_fraction_std": 0.014003137560787724}, {"budget": 0.75, "metric": 0.2354779825922572, "metric_std": 0.007325212193098403, "actual_fraction": 0.7551930758988016, "actual_fraction_std": 0.011039691295953757}, {"budget": 0.76, "metric": 0.2342491702620657, "metric_std": 0.006817968137638322, "actual_fraction": 0.7639813581890813, "actual_fraction_std": 0.009394917304707139}, {"budget": 0.77, "metric": 0.23271886437324535, "metric_std": 0.00685712419126016, "actual_fraction": 0.7745672436750998, "actual_fraction_std": 0.01016103902667295}, {"budget": 0.78, "metric": 0.23200464406954707, "metric_std": 0.007972991230733122, "actual_fraction": 0.7840878828229028, "actual_fraction_std": 0.008986782970118155}, {"budget": 0.79, "metric": 0.2307022213773588, "metric_std": 0.007627851241876857, "actual_fraction": 0.7916777629826898, "actual_fraction_std": 0.00721456834226795}, {"budget": 0.8, "metric": 0.23080082395325302, "metric_std": 0.0069330845060770595, "actual_fraction": 0.8008655126498002, "actual_fraction_std": 0.007610730116230952}, {"budget": 0.81, "metric": 0.2305430963843432, "metric_std": 0.006465747911822796, "actual_fraction": 0.8103195739014648, "actual_fraction_std": 0.0085209303625971}, {"budget": 0.82, "metric": 0.22919080843217002, "metric_std": 0.006649110781475433, "actual_fraction": 0.8198402130492676, "actual_fraction_std": 0.006387662531799443}, {"budget": 0.83, "metric": 0.2280897612130083, "metric_std": 0.006421242085169682, "actual_fraction": 0.831624500665779, "actual_fraction_std": 0.007098434001237099}, {"budget": 0.84, "metric": 0.227594018444977, "metric_std": 0.005400598147289165, "actual_fraction": 0.8415446071904128, "actual_fraction_std": 0.010268439956848965}, {"budget": 0.85, "metric": 0.2274837255009144, "metric_std": 0.006747706906918706, "actual_fraction": 0.854260985352863, "actual_fraction_std": 0.008553381277366708}, {"budget": 0.86, "metric": 0.22662375329768358, "metric_std": 0.007095065823016244, "actual_fraction": 0.8627829560585886, "actual_fraction_std": 0.009617568283358944}, {"budget": 0.87, "metric": 0.2259642236297772, "metric_std": 0.005831321357448076, "actual_fraction": 0.8709054593874834, "actual_fraction_std": 0.010702798024341448}, {"budget": 0.88, "metric": 0.22694627990755306, "metric_std": 0.005235673525128533, "actual_fraction": 0.8819573901464715, "actual_fraction_std": 0.009977248094003235}, {"budget": 0.89, "metric": 0.22610202769983268, "metric_std": 0.004886709368992784, "actual_fraction": 0.89034620505992, "actual_fraction_std": 0.008875110750723326}, {"budget": 0.9, "metric": 0.22544030490585315, "metric_std": 0.003857787034363843, "actual_fraction": 0.9027296937416777, "actual_fraction_std": 0.010556848685658506}, {"budget": 0.91, "metric": 0.22499062983016221, "metric_std": 0.0036031980308357746, "actual_fraction": 0.9117842876165113, "actual_fraction_std": 0.013099528874800944}, {"budget": 0.92, "metric": 0.22434630929719535, "metric_std": 0.0034306265529017877, "actual_fraction": 0.9217709720372836, "actual_fraction_std": 0.014226887913687146}, {"budget": 0.93, "metric": 0.22578134146554701, "metric_std": 0.0015673729856210915, "actual_fraction": 0.935352862849534, "actual_fraction_std": 0.011506136014019197}, {"budget": 0.94, "metric": 0.22556482611846768, "metric_std": 0.0012424614561356824, "actual_fraction": 0.9416777629826898, "actual_fraction_std": 0.007949021619293695}, {"budget": 0.95, "metric": 0.22579321590758727, "metric_std": 0.003360549462268617, "actual_fraction": 0.953462050599201, "actual_fraction_std": 0.006099240273476362}, {"budget": 0.96, "metric": 0.22521328994032652, "metric_std": 0.0036520097910981144, "actual_fraction": 0.9631824234354195, "actual_fraction_std": 0.0056669484523746115}, {"budget": 0.97, "metric": 0.22454370104623358, "metric_std": 0.002672341705492152, "actual_fraction": 0.9747003994673769, "actual_fraction_std": 0.003654210785635844}, {"budget": 0.98, "metric": 0.22502599298847956, "metric_std": 0.0011226069263489632, "actual_fraction": 0.9813581890812252, "actual_fraction_std": 0.003585329432180081}, {"budget": 0.99, "metric": 0.22460619880524044, "metric_std": 0.0008083266010042632, "actual_fraction": 0.9907456724367509, "actual_fraction_std": 0.0033071862069294377}, {"budget": 1.0, "metric": 0.22361115727328068, "metric_std": 3.1031676915590914e-17, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 1891, "rescuable": 286, "harmful": 518, "both_wrong": 109, "unit": "event", "total": 2804}, "test": {"both_correct": 8176, "rescuable": 1198, "harmful": 2109, "both_wrong": 533, "unit": "event", "total": 12016}}, "small": {"validation": 0.18254072168965788, "test": 0.18129624718318377}, "large": {"validation": 0.20666016863629594, "test": 0.22361115727328065}, "learned_minus_confidence_test": 0.006499654461157467, "learned_minus_small_test": 0.07750811500846722, "learned_minus_large_test": 0.03519320491837033, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_yolo26x_MiniCPM"}, {"key": ["construction", "yolo26x", "Qwen3.8"], "expert_display": "YOLO26x", "run": "outputs/full_budget_20260913/runs/full_yolo26x_Qwen3.8_s{2026..2030}", "validation_rows": 701, "test_rows": 3004, "confidence_rule": {"column": "max_score", "columns": ["detection_count", "has_detections", "max_score", "mean_score", "detection_count_valid", "has_detections_valid", "max_score_valid", "mean_score_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/yolo26x/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 93, "validation_area_0_25": 0.17942553980644182, "validation_point": {"budget": 0.93, "threshold": -0.453369140625, "operator": "ge", "actual_fraction": 0.9286733238231099, "metric": 0.2642878791936722, "rescued": 288, "harmed": 202, "missed_rescue": 19, "avoided_harm": 18, "selected_both_correct": 2035, "selected_both_wrong": 79}, "test_point": {"budget": 0.93, "threshold": -0.453369140625, "operator": "ge", "actual_fraction": 0.9221038615179761, "metric": 0.2597490004762105, "rescued": 1180, "harmed": 728, "missed_rescue": 102, "avoided_harm": 49, "selected_both_correct": 8774, "selected_both_wrong": 398}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.11, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.12, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.13, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.14, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.15, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.16, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.17, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.18, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18254072168965788, "rescued": 0, "harmed": 0, "missed_rescue": 307, "avoided_harm": 220, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.19, "threshold": 1.0, "operator": "ge", "actual_fraction": 0.18972895863052783, "metric": 0.1728507294858811, "rescued": 0, "harmed": 34, "missed_rescue": 307, "avoided_harm": 186, "selected_both_correct": 498, "selected_both_wrong": 0}, {"budget": 0.2, "threshold": -0.0010652542114257812, "operator": "ge", "actual_fraction": 0.19971469329529243, "metric": 0.1728507294858811, "rescued": 0, "harmed": 34, "missed_rescue": 307, "avoided_harm": 186, "selected_both_correct": 526, "selected_both_wrong": 0}, {"budget": 0.21, "threshold": -0.0011425018310546875, "operator": "ge", "actual_fraction": 0.20970042796005706, "metric": 0.17237009889855048, "rescued": 0, "harmed": 36, "missed_rescue": 307, "avoided_harm": 184, "selected_both_correct": 552, "selected_both_wrong": 0}, {"budget": 0.22, "threshold": -0.0011968612670898438, "operator": "ge", "actual_fraction": 0.21968616262482168, "metric": 0.17209204025345443, "rescued": 0, "harmed": 37, "missed_rescue": 307, "avoided_harm": 183, "selected_both_correct": 578, "selected_both_wrong": 1}, {"budget": 0.23, "threshold": -0.0012493133544921875, "operator": "ge", "actual_fraction": 0.2296718972895863, "metric": 0.1716297194027841, "rescued": 0, "harmed": 39, "missed_rescue": 307, "avoided_harm": 181, "selected_both_correct": 604, "selected_both_wrong": 1}, {"budget": 0.24, "threshold": -0.0013513565063476562, "operator": "ge", "actual_fraction": 0.2396576319543509, "metric": 0.17076888726505096, "rescued": 0, "harmed": 42, "missed_rescue": 307, "avoided_harm": 178, "selected_both_correct": 629, "selected_both_wrong": 1}, {"budget": 0.25, "threshold": -0.0015010833740234375, "operator": "gt", "actual_fraction": 0.24821683309557774, "metric": 0.17076888726505096, "rescued": 0, "harmed": 42, "missed_rescue": 307, "avoided_harm": 178, "selected_both_correct": 652, "selected_both_wrong": 2}, {"budget": 0.26, "threshold": -0.00157928466796875, "operator": "ge", "actual_fraction": 0.25962910128388017, "metric": 0.17010751160367527, "rescued": 0, "harmed": 45, "missed_rescue": 307, "avoided_harm": 175, "selected_both_correct": 681, "selected_both_wrong": 2}, {"budget": 0.27, "threshold": -0.0017137527465820312, "operator": "gt", "actual_fraction": 0.268188302425107, "metric": 0.17010751160367527, "rescued": 0, "harmed": 45, "missed_rescue": 307, "avoided_harm": 175, "selected_both_correct": 705, "selected_both_wrong": 2}, {"budget": 0.28, "threshold": -0.0017614364624023438, "operator": "ge", "actual_fraction": 0.2796005706134094, "metric": 0.16948188597804964, "rescued": 0, "harmed": 48, "missed_rescue": 307, "avoided_harm": 172, "selected_both_correct": 734, "selected_both_wrong": 2}, {"budget": 0.29, "threshold": -0.0018463134765625, "operator": "ge", "actual_fraction": 0.289586305278174, "metric": 0.16948188597804964, "rescued": 0, "harmed": 48, "missed_rescue": 307, "avoided_harm": 172, "selected_both_correct": 762, "selected_both_wrong": 2}, {"budget": 0.3, "threshold": -0.001934051513671875, "operator": "ge", "actual_fraction": 0.2995720399429387, "metric": 0.16880965944961934, "rescued": 0, "harmed": 51, "missed_rescue": 307, "avoided_harm": 169, "selected_both_correct": 787, "selected_both_wrong": 2}, {"budget": 0.31, "threshold": -0.0020427703857421875, "operator": "gt", "actual_fraction": 0.30813124108416545, "metric": 0.16830460894456886, "rescued": 0, "harmed": 56, "missed_rescue": 307, "avoided_harm": 164, "selected_both_correct": 806, "selected_both_wrong": 2}, {"budget": 0.32, "threshold": -0.0021495819091796875, "operator": "gt", "actual_fraction": 0.3181169757489301, "metric": 0.16811053970066264, "rescued": 0, "harmed": 57, "missed_rescue": 307, "avoided_harm": 163, "selected_both_correct": 833, "selected_both_wrong": 2}, {"budget": 0.33, "threshold": -0.002208709716796875, "operator": "ge", "actual_fraction": 0.32952924393723254, "metric": 0.1671811299174608, "rescued": 0, "harmed": 61, "missed_rescue": 307, "avoided_harm": 159, "selected_both_correct": 861, "selected_both_wrong": 2}, {"budget": 0.34, "threshold": -0.0023326873779296875, "operator": "ge", "actual_fraction": 0.33951497860199714, "metric": 0.16645634059427164, "rescued": 0, "harmed": 64, "missed_rescue": 307, "avoided_harm": 156, "selected_both_correct": 886, "selected_both_wrong": 2}, {"budget": 0.35, "threshold": -0.0024623870849609375, "operator": "gt", "actual_fraction": 0.34807417974322397, "metric": 0.16600354762332126, "rescued": 0, "harmed": 66, "missed_rescue": 307, "avoided_harm": 154, "selected_both_correct": 908, "selected_both_wrong": 2}, {"budget": 0.36, "threshold": -0.0025615692138671875, "operator": "ge", "actual_fraction": 0.3594864479315264, "metric": 0.16502549737843858, "rescued": 0, "harmed": 70, "missed_rescue": 307, "avoided_harm": 150, "selected_both_correct": 936, "selected_both_wrong": 2}, {"budget": 0.37, "threshold": -0.0026721954345703125, "operator": "gt", "actual_fraction": 0.3666191155492154, "metric": 0.16467825104188744, "rescued": 0, "harmed": 72, "missed_rescue": 307, "avoided_harm": 148, "selected_both_correct": 954, "selected_both_wrong": 2}, {"budget": 0.38, "threshold": -0.0028228759765625, "operator": "gt", "actual_fraction": 0.37660485021398005, "metric": 0.16467825104188744, "rescued": 0, "harmed": 76, "missed_rescue": 307, "avoided_harm": 144, "selected_both_correct": 978, "selected_both_wrong": 2}, {"budget": 0.39, "threshold": -0.0029354095458984375, "operator": "ge", "actual_fraction": 0.38944365192582026, "metric": 0.16749463738508683, "rescued": 1, "harmed": 81, "missed_rescue": 306, "avoided_harm": 139, "selected_both_correct": 1008, "selected_both_wrong": 2}, {"budget": 0.4, "threshold": -0.00307464599609375, "operator": "ge", "actual_fraction": 0.39942938659058486, "metric": 0.16607474321791724, "rescued": 1, "harmed": 87, "missed_rescue": 306, "avoided_harm": 133, "selected_both_correct": 1030, "selected_both_wrong": 2}, {"budget": 0.41, "threshold": -0.0033626556396484375, "operator": "ge", "actual_fraction": 0.4094151212553495, "metric": 0.16514206572141965, "rescued": 1, "harmed": 90, "missed_rescue": 306, "avoided_harm": 130, "selected_both_correct": 1055, "selected_both_wrong": 2}, {"budget": 0.42, "threshold": -0.0035381317138671875, "operator": "ge", "actual_fraction": 0.4194008559201141, "metric": 0.16444252141114554, "rescued": 1, "harmed": 93, "missed_rescue": 306, "avoided_harm": 127, "selected_both_correct": 1080, "selected_both_wrong": 2}, {"budget": 0.43, "threshold": -0.003765106201171875, "operator": "ge", "actual_fraction": 0.42938659058487877, "metric": 0.1639139801849299, "rescued": 1, "harmed": 96, "missed_rescue": 306, "avoided_harm": 124, "selected_both_correct": 1105, "selected_both_wrong": 2}, {"budget": 0.44, "threshold": -0.00392913818359375, "operator": "ge", "actual_fraction": 0.43937232524964337, "metric": 0.16237039262040495, "rescued": 1, "harmed": 103, "missed_rescue": 306, "avoided_harm": 117, "selected_both_correct": 1126, "selected_both_wrong": 2}, {"budget": 0.45, "threshold": -0.004085540771484375, "operator": "ge", "actual_fraction": 0.44935805991440797, "metric": 0.16187473569847557, "rescued": 1, "harmed": 105, "missed_rescue": 306, "avoided_harm": 115, "selected_both_correct": 1152, "selected_both_wrong": 2}, {"budget": 0.46, "threshold": -0.004364013671875, "operator": "ge", "actual_fraction": 0.4593437945791726, "metric": 0.1640341119762699, "rescued": 2, "harmed": 110, "missed_rescue": 305, "avoided_harm": 110, "selected_both_correct": 1174, "selected_both_wrong": 2}, {"budget": 0.47, "threshold": -0.004680633544921875, "operator": "gt", "actual_fraction": 0.46790299572039945, "metric": 0.16656779979633995, "rescued": 3, "harmed": 113, "missed_rescue": 304, "avoided_harm": 107, "selected_both_correct": 1194, "selected_both_wrong": 2}, {"budget": 0.48, "threshold": -0.00490570068359375, "operator": "ge", "actual_fraction": 0.4793152639087018, "metric": 0.16532395129425315, "rescued": 3, "harmed": 119, "missed_rescue": 304, "avoided_harm": 101, "selected_both_correct": 1220, "selected_both_wrong": 2}, {"budget": 0.49, "threshold": -0.005039215087890625, "operator": "ge", "actual_fraction": 0.4893009985734665, "metric": 0.16467625831496033, "rescued": 5, "harmed": 123, "missed_rescue": 302, "avoided_harm": 97, "selected_both_correct": 1241, "selected_both_wrong": 3}, {"budget": 0.5, "threshold": -0.005321502685546875, "operator": "ge", "actual_fraction": 0.4992867332382311, "metric": 0.16569589650413527, "rescued": 11, "harmed": 128, "missed_rescue": 296, "avoided_harm": 92, "selected_both_correct": 1257, "selected_both_wrong": 4}, {"budget": 0.51, "threshold": -0.005619049072265625, "operator": "ge", "actual_fraction": 0.5092724679029957, "metric": 0.1671927745457157, "rescued": 18, "harmed": 129, "missed_rescue": 289, "avoided_harm": 91, "selected_both_correct": 1276, "selected_both_wrong": 5}, {"budget": 0.52, "threshold": -0.00586700439453125, "operator": "gt", "actual_fraction": 0.5178316690442225, "metric": 0.16789087940773334, "rescued": 23, "harmed": 131, "missed_rescue": 284, "avoided_harm": 89, "selected_both_correct": 1293, "selected_both_wrong": 5}, {"budget": 0.53, "threshold": -0.00605010986328125, "operator": "ge", "actual_fraction": 0.5292439372325249, "metric": 0.16946103210148153, "rescued": 30, "harmed": 132, "missed_rescue": 277, "avoided_harm": 88, "selected_both_correct": 1316, "selected_both_wrong": 6}, {"budget": 0.54, "threshold": -0.006439208984375, "operator": "gt", "actual_fraction": 0.5378031383737518, "metric": 0.17052688690619724, "rescued": 35, "harmed": 133, "missed_rescue": 272, "avoided_harm": 87, "selected_both_correct": 1334, "selected_both_wrong": 6}, {"budget": 0.55, "threshold": -0.006641387939453125, "operator": "ge", "actual_fraction": 0.5492154065620543, "metric": 0.17103667850460152, "rescued": 42, "harmed": 134, "missed_rescue": 265, "avoided_harm": 86, "selected_both_correct": 1358, "selected_both_wrong": 6}, {"budget": 0.56, "threshold": -0.0070648193359375, "operator": "ge", "actual_fraction": 0.5592011412268189, "metric": 0.17291017019862398, "rescued": 49, "harmed": 135, "missed_rescue": 258, "avoided_harm": 85, "selected_both_correct": 1376, "selected_both_wrong": 8}, {"budget": 0.57, "threshold": -0.007518768310546875, "operator": "ge", "actual_fraction": 0.5691868758915835, "metric": 0.17468909977784217, "rescued": 55, "harmed": 135, "missed_rescue": 252, "avoided_harm": 85, "selected_both_correct": 1397, "selected_both_wrong": 9}, {"budget": 0.58, "threshold": -0.00818634033203125, "operator": "ge", "actual_fraction": 0.579172610556348, "metric": 0.1754433851238893, "rescued": 60, "harmed": 136, "missed_rescue": 247, "avoided_harm": 84, "selected_both_correct": 1416, "selected_both_wrong": 12}, {"budget": 0.59, "threshold": -0.00878143310546875, "operator": "gt", "actual_fraction": 0.5877318116975749, "metric": 0.17567949038614306, "rescued": 65, "harmed": 140, "missed_rescue": 242, "avoided_harm": 80, "selected_both_correct": 1430, "selected_both_wrong": 13}, {"budget": 0.6, "threshold": -0.00955963134765625, "operator": "gt", "actual_fraction": 0.5977175463623395, "metric": 0.17726242688944494, "rescued": 73, "harmed": 142, "missed_rescue": 234, "avoided_harm": 78, "selected_both_correct": 1448, "selected_both_wrong": 13}, {"budget": 0.61, "threshold": -0.0099334716796875, "operator": "ge", "actual_fraction": 0.609129814550642, "metric": 0.17927753989081613, "rescued": 81, "harmed": 143, "missed_rescue": 226, "avoided_harm": 77, "selected_both_correct": 1471, "selected_both_wrong": 13}, {"budget": 0.62, "threshold": -0.010650634765625, "operator": "gt", "actual_fraction": 0.6176890156918687, "metric": 0.1803426895170312, "rescued": 86, "harmed": 144, "missed_rescue": 221, "avoided_harm": 76, "selected_both_correct": 1489, "selected_both_wrong": 13}, {"budget": 0.63, "threshold": -0.01129150390625, "operator": "ge", "actual_fraction": 0.6291012838801712, "metric": 0.18256297675338154, "rescued": 94, "harmed": 145, "missed_rescue": 213, "avoided_harm": 75, "selected_both_correct": 1509, "selected_both_wrong": 16}, {"budget": 0.64, "threshold": -0.011871337890625, "operator": "ge", "actual_fraction": 0.6390870185449358, "metric": 0.18741054744283442, "rescued": 103, "harmed": 149, "missed_rescue": 204, "avoided_harm": 71, "selected_both_correct": 1524, "selected_both_wrong": 16}, {"budget": 0.65, "threshold": -0.0125274658203125, "operator": "gt", "actual_fraction": 0.6476462196861626, "metric": 0.18874453577558464, "rescued": 109, "harmed": 150, "missed_rescue": 198, "avoided_harm": 70, "selected_both_correct": 1540, "selected_both_wrong": 17}, {"budget": 0.66, "threshold": -0.012969970703125, "operator": "gt", "actual_fraction": 0.6562054208273894, "metric": 0.18973030032445837, "rescued": 114, "harmed": 152, "missed_rescue": 193, "avoided_harm": 68, "selected_both_correct": 1556, "selected_both_wrong": 18}, {"budget": 0.67, "threshold": -0.01363372802734375, "operator": "ge", "actual_fraction": 0.6690442225392297, "metric": 0.1910322331722189, "rescued": 122, "harmed": 157, "missed_rescue": 185, "avoided_harm": 63, "selected_both_correct": 1576, "selected_both_wrong": 21}, {"budget": 0.68, "threshold": -0.01473236083984375, "operator": "ge", "actual_fraction": 0.6790299572039943, "metric": 0.19273815380416676, "rescued": 128, "harmed": 158, "missed_rescue": 179, "avoided_harm": 62, "selected_both_correct": 1595, "selected_both_wrong": 23}, {"budget": 0.69, "threshold": -0.015716552734375, "operator": "ge", "actual_fraction": 0.6890156918687589, "metric": 0.19362404817095658, "rescued": 133, "harmed": 161, "missed_rescue": 174, "avoided_harm": 59, "selected_both_correct": 1612, "selected_both_wrong": 26}, {"budget": 0.7, "threshold": -0.017578125, "operator": "gt", "actual_fraction": 0.6975748930099858, "metric": 0.19489832797986031, "rescued": 138, "harmed": 163, "missed_rescue": 169, "avoided_harm": 57, "selected_both_correct": 1626, "selected_both_wrong": 29}, {"budget": 0.71, "threshold": -0.0189056396484375, "operator": "ge", "actual_fraction": 0.7089871611982882, "metric": 0.19709243659719292, "rescued": 147, "harmed": 166, "missed_rescue": 160, "avoided_harm": 54, "selected_both_correct": 1646, "selected_both_wrong": 29}, {"budget": 0.72, "threshold": -0.019866943359375, "operator": "ge", "actual_fraction": 0.7189728958630528, "metric": 0.20000323882160753, "rescued": 155, "harmed": 166, "missed_rescue": 152, "avoided_harm": 54, "selected_both_correct": 1666, "selected_both_wrong": 29}, {"budget": 0.73, "threshold": -0.020965576171875, "operator": "ge", "actual_fraction": 0.7289586305278174, "metric": 0.2015690267335004, "rescued": 160, "harmed": 167, "missed_rescue": 147, "avoided_harm": 53, "selected_both_correct": 1684, "selected_both_wrong": 33}, {"budget": 0.74, "threshold": -0.021453857421875, "operator": "ge", "actual_fraction": 0.738944365192582, "metric": 0.2034984942593638, "rescued": 166, "harmed": 168, "missed_rescue": 141, "avoided_harm": 52, "selected_both_correct": 1703, "selected_both_wrong": 35}, {"budget": 0.75, "threshold": -0.0248870849609375, "operator": "ge", "actual_fraction": 0.7489300998573466, "metric": 0.20679879821670866, "rescued": 176, "harmed": 170, "missed_rescue": 131, "avoided_harm": 50, "selected_both_correct": 1718, "selected_both_wrong": 36}, {"budget": 0.76, "threshold": -0.0269622802734375, "operator": "ge", "actual_fraction": 0.7589158345221113, "metric": 0.20972162293572166, "rescued": 184, "harmed": 171, "missed_rescue": 123, "avoided_harm": 49, "selected_both_correct": 1735, "selected_both_wrong": 38}, {"budget": 0.77, "threshold": -0.0282135009765625, "operator": "ge", "actual_fraction": 0.7689015691868759, "metric": 0.2126226551226551, "rescued": 191, "harmed": 174, "missed_rescue": 116, "avoided_harm": 46, "selected_both_correct": 1752, "selected_both_wrong": 39}, {"budget": 0.78, "threshold": -0.0299835205078125, "operator": "ge", "actual_fraction": 0.7788873038516405, "metric": 0.21516614155218836, "rescued": 198, "harmed": 175, "missed_rescue": 109, "avoided_harm": 45, "selected_both_correct": 1771, "selected_both_wrong": 40}, {"budget": 0.79, "threshold": -0.033966064453125, "operator": "ge", "actual_fraction": 0.7888730385164051, "metric": 0.2162880860385455, "rescued": 204, "harmed": 177, "missed_rescue": 103, "avoided_harm": 43, "selected_both_correct": 1788, "selected_both_wrong": 43}, {"budget": 0.8, "threshold": -0.037200927734375, "operator": "ge", "actual_fraction": 0.7988587731811697, "metric": 0.21779120256650406, "rescued": 209, "harmed": 179, "missed_rescue": 98, "avoided_harm": 41, "selected_both_correct": 1806, "selected_both_wrong": 46}, {"budget": 0.81, "threshold": -0.04132080078125, "operator": "ge", "actual_fraction": 0.8088445078459344, "metric": 0.2201843117420812, "rescued": 216, "harmed": 182, "missed_rescue": 91, "avoided_harm": 38, "selected_both_correct": 1820, "selected_both_wrong": 50}, {"budget": 0.82, "threshold": -0.0455322265625, "operator": "ge", "actual_fraction": 0.818830242510699, "metric": 0.22188537527520574, "rescued": 222, "harmed": 184, "missed_rescue": 85, "avoided_harm": 36, "selected_both_correct": 1838, "selected_both_wrong": 52}, {"budget": 0.83, "threshold": -0.051849365234375, "operator": "ge", "actual_fraction": 0.8288159771754636, "metric": 0.2260242392475367, "rescued": 231, "harmed": 187, "missed_rescue": 76, "avoided_harm": 33, "selected_both_correct": 1854, "selected_both_wrong": 52}, {"budget": 0.84, "threshold": -0.06878662109375, "operator": "ge", "actual_fraction": 0.8388017118402282, "metric": 0.2282665192342907, "rescued": 238, "harmed": 190, "missed_rescue": 69, "avoided_harm": 30, "selected_both_correct": 1869, "selected_both_wrong": 55}, {"budget": 0.85, "threshold": -0.08477783203125, "operator": "ge", "actual_fraction": 0.8487874465049928, "metric": 0.23641538775702553, "rescued": 246, "harmed": 193, "missed_rescue": 61, "avoided_harm": 27, "selected_both_correct": 1884, "selected_both_wrong": 57}, {"budget": 0.86, "threshold": -0.09912109375, "operator": "ge", "actual_fraction": 0.8587731811697575, "metric": 0.2387681003514125, "rescued": 251, "harmed": 193, "missed_rescue": 56, "avoided_harm": 27, "selected_both_correct": 1904, "selected_both_wrong": 60}, {"budget": 0.87, "threshold": -0.1298828125, "operator": "ge", "actual_fraction": 0.8687589158345221, "metric": 0.24246700995212897, "rescued": 258, "harmed": 193, "missed_rescue": 49, "avoided_harm": 27, "selected_both_correct": 1923, "selected_both_wrong": 62}, {"budget": 0.88, "threshold": -0.1441650390625, "operator": "ge", "actual_fraction": 0.8787446504992867, "metric": 0.2423696527366133, "rescued": 259, "harmed": 197, "missed_rescue": 48, "avoided_harm": 23, "selected_both_correct": 1941, "selected_both_wrong": 67}, {"budget": 0.89, "threshold": -0.182373046875, "operator": "ge", "actual_fraction": 0.8887303851640513, "metric": 0.24478740599615373, "rescued": 266, "harmed": 199, "missed_rescue": 41, "avoided_harm": 21, "selected_both_correct": 1959, "selected_both_wrong": 68}, {"budget": 0.9, "threshold": -0.2049560546875, "operator": "ge", "actual_fraction": 0.8987161198288159, "metric": 0.25736315359477124, "rescued": 273, "harmed": 199, "missed_rescue": 34, "avoided_harm": 21, "selected_both_correct": 1977, "selected_both_wrong": 71}, {"budget": 0.91, "threshold": -0.25830078125, "operator": "gt", "actual_fraction": 0.9072753209700428, "metric": 0.2597551406926407, "rescued": 278, "harmed": 200, "missed_rescue": 29, "avoided_harm": 20, "selected_both_correct": 1992, "selected_both_wrong": 74}, {"budget": 0.92, "threshold": -0.32080078125, "operator": "ge", "actual_fraction": 0.9186875891583453, "metric": 0.2629241291812946, "rescued": 284, "harmed": 201, "missed_rescue": 23, "avoided_harm": 19, "selected_both_correct": 2013, "selected_both_wrong": 78}, {"budget": 0.93, "threshold": -0.453369140625, "operator": "ge", "actual_fraction": 0.9286733238231099, "metric": 0.2642878791936722, "rescued": 288, "harmed": 202, "missed_rescue": 19, "avoided_harm": 18, "selected_both_correct": 2035, "selected_both_wrong": 79}, {"budget": 0.94, "threshold": -0.490234375, "operator": "ge", "actual_fraction": 0.9386590584878745, "metric": 0.2639590174546567, "rescued": 290, "harmed": 203, "missed_rescue": 17, "avoided_harm": 17, "selected_both_correct": 2057, "selected_both_wrong": 82}, {"budget": 0.95, "threshold": -0.611328125, "operator": "ge", "actual_fraction": 0.948644793152639, "metric": 0.26132796745180287, "rescued": 293, "harmed": 207, "missed_rescue": 14, "avoided_harm": 13, "selected_both_correct": 2078, "selected_both_wrong": 82}, {"budget": 0.96, "threshold": -0.70068359375, "operator": "ge", "actual_fraction": 0.9586305278174037, "metric": 0.2584345323365104, "rescued": 296, "harmed": 212, "missed_rescue": 11, "avoided_harm": 8, "selected_both_correct": 2096, "selected_both_wrong": 84}, {"budget": 0.97, "threshold": -0.75048828125, "operator": "ge", "actual_fraction": 0.9686162624821684, "metric": 0.2503146324574896, "rescued": 299, "harmed": 215, "missed_rescue": 8, "avoided_harm": 5, "selected_both_correct": 2116, "selected_both_wrong": 86}, {"budget": 0.98, "threshold": -0.7841796875, "operator": "ge", "actual_fraction": 0.978601997146933, "metric": 0.2545437366865938, "rescued": 302, "harmed": 216, "missed_rescue": 5, "avoided_harm": 4, "selected_both_correct": 2139, "selected_both_wrong": 87}, {"budget": 0.99, "threshold": -0.85107421875, "operator": "ge", "actual_fraction": 0.9885877318116976, "metric": 0.2541444987255433, "rescued": 304, "harmed": 218, "missed_rescue": 3, "avoided_harm": 2, "selected_both_correct": 2163, "selected_both_wrong": 87}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.25448626222520154, "rescued": 307, "harmed": 220, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 2189, "selected_both_wrong": 88}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.11, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.12, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.13, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.14, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.15, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.16, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.17, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.18, "threshold": 1.0, "operator": "gt", "actual_fraction": 0.0, "metric": 0.18129624718318377, "rescued": 0, "harmed": 0, "missed_rescue": 1282, "avoided_harm": 777, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.19, "threshold": 1.0, "operator": "ge", "actual_fraction": 0.17942743009320905, "metric": 0.1784481207469488, "rescued": 4, "harmed": 94, "missed_rescue": 1278, "avoided_harm": 683, "selected_both_correct": 2057, "selected_both_wrong": 1}, {"budget": 0.2, "threshold": -0.0010652542114257812, "operator": "ge", "actual_fraction": 0.19340878828229027, "metric": 0.1780928651788194, "rescued": 4, "harmed": 99, "missed_rescue": 1278, "avoided_harm": 678, "selected_both_correct": 2220, "selected_both_wrong": 1}, {"budget": 0.21, "threshold": -0.0011425018310546875, "operator": "ge", "actual_fraction": 0.20306258322237017, "metric": 0.17829989216687975, "rescued": 5, "harmed": 108, "missed_rescue": 1277, "avoided_harm": 669, "selected_both_correct": 2325, "selected_both_wrong": 2}, {"budget": 0.22, "threshold": -0.0011968612670898438, "operator": "ge", "actual_fraction": 0.20838881491344874, "metric": 0.1779854683890517, "rescued": 5, "harmed": 112, "missed_rescue": 1277, "avoided_harm": 665, "selected_both_correct": 2385, "selected_both_wrong": 2}, {"budget": 0.23, "threshold": -0.0012493133544921875, "operator": "ge", "actual_fraction": 0.21537949400798934, "metric": 0.17917657728013692, "rescued": 6, "harmed": 118, "missed_rescue": 1276, "avoided_harm": 659, "selected_both_correct": 2462, "selected_both_wrong": 2}, {"budget": 0.24, "threshold": -0.0013513565063476562, "operator": "ge", "actual_fraction": 0.23135818908122505, "metric": 0.17803687432776102, "rescued": 6, "harmed": 133, "missed_rescue": 1276, "avoided_harm": 644, "selected_both_correct": 2638, "selected_both_wrong": 3}, {"budget": 0.25, "threshold": -0.0015010833740234375, "operator": "gt", "actual_fraction": 0.24800266311584554, "metric": 0.18026802885144627, "rescued": 10, "harmed": 153, "missed_rescue": 1272, "avoided_harm": 624, "selected_both_correct": 2814, "selected_both_wrong": 3}, {"budget": 0.26, "threshold": -0.00157928466796875, "operator": "ge", "actual_fraction": 0.2589880159786951, "metric": 0.18119827871024874, "rescued": 11, "harmed": 163, "missed_rescue": 1271, "avoided_harm": 614, "selected_both_correct": 2935, "selected_both_wrong": 3}, {"budget": 0.27, "threshold": -0.0017137527465820312, "operator": "gt", "actual_fraction": 0.27430093209054596, "metric": 0.17963665085316138, "rescued": 11, "harmed": 185, "missed_rescue": 1271, "avoided_harm": 592, "selected_both_correct": 3097, "selected_both_wrong": 3}, {"budget": 0.28, "threshold": -0.0017614364624023438, "operator": "ge", "actual_fraction": 0.281624500665779, "metric": 0.17924095839307458, "rescued": 11, "harmed": 191, "missed_rescue": 1271, "avoided_harm": 586, "selected_both_correct": 3179, "selected_both_wrong": 3}, {"budget": 0.29, "threshold": -0.0018463134765625, "operator": "ge", "actual_fraction": 0.28761651131824234, "metric": 0.1789157174618718, "rescued": 11, "harmed": 196, "missed_rescue": 1271, "avoided_harm": 581, "selected_both_correct": 3246, "selected_both_wrong": 3}, {"budget": 0.3, "threshold": -0.001934051513671875, "operator": "ge", "actual_fraction": 0.2969374167776298, "metric": 0.1787206845800119, "rescued": 11, "harmed": 199, "missed_rescue": 1271, "avoided_harm": 578, "selected_both_correct": 3355, "selected_both_wrong": 3}, {"budget": 0.31, "threshold": -0.0020427703857421875, "operator": "gt", "actual_fraction": 0.30459387483355527, "metric": 0.17813242189706796, "rescued": 11, "harmed": 207, "missed_rescue": 1271, "avoided_harm": 570, "selected_both_correct": 3438, "selected_both_wrong": 4}, {"budget": 0.32, "threshold": -0.0021495819091796875, "operator": "gt", "actual_fraction": 0.3129161118508655, "metric": 0.17772000842137498, "rescued": 12, "harmed": 218, "missed_rescue": 1270, "avoided_harm": 559, "selected_both_correct": 3526, "selected_both_wrong": 4}, {"budget": 0.33, "threshold": -0.002208709716796875, "operator": "ge", "actual_fraction": 0.31990679094540614, "metric": 0.17791259734473341, "rescued": 13, "harmed": 227, "missed_rescue": 1269, "avoided_harm": 550, "selected_both_correct": 3600, "selected_both_wrong": 4}, {"budget": 0.34, "threshold": -0.0023326873779296875, "operator": "ge", "actual_fraction": 0.3322237017310253, "metric": 0.1789653934599337, "rescued": 15, "harmed": 238, "missed_rescue": 1267, "avoided_harm": 539, "selected_both_correct": 3735, "selected_both_wrong": 4}, {"budget": 0.35, "threshold": -0.0024623870849609375, "operator": "gt", "actual_fraction": 0.34054593874833555, "metric": 0.18042682200794885, "rescued": 16, "harmed": 249, "missed_rescue": 1266, "avoided_harm": 528, "selected_both_correct": 3822, "selected_both_wrong": 5}, {"budget": 0.36, "threshold": -0.0025615692138671875, "operator": "ge", "actual_fraction": 0.3488681757656458, "metric": 0.17962877325912951, "rescued": 16, "harmed": 260, "missed_rescue": 1266, "avoided_harm": 517, "selected_both_correct": 3910, "selected_both_wrong": 6}, {"budget": 0.37, "threshold": -0.0026721954345703125, "operator": "gt", "actual_fraction": 0.3581890812250333, "metric": 0.18158304685510065, "rescued": 18, "harmed": 271, "missed_rescue": 1264, "avoided_harm": 506, "selected_both_correct": 4008, "selected_both_wrong": 7}, {"budget": 0.38, "threshold": -0.0028228759765625, "operator": "gt", "actual_fraction": 0.37017310252996005, "metric": 0.18069239723097547, "rescued": 18, "harmed": 285, "missed_rescue": 1264, "avoided_harm": 492, "selected_both_correct": 4138, "selected_both_wrong": 7}, {"budget": 0.39, "threshold": -0.0029354095458984375, "operator": "ge", "actual_fraction": 0.37683089214380827, "metric": 0.1812845803851709, "rescued": 20, "harmed": 292, "missed_rescue": 1262, "avoided_harm": 485, "selected_both_correct": 4209, "selected_both_wrong": 7}, {"budget": 0.4, "threshold": -0.00307464599609375, "operator": "ge", "actual_fraction": 0.38348868175765644, "metric": 0.1824781664689562, "rescued": 21, "harmed": 301, "missed_rescue": 1261, "avoided_harm": 476, "selected_both_correct": 4279, "selected_both_wrong": 7}, {"budget": 0.41, "threshold": -0.0033626556396484375, "operator": "ge", "actual_fraction": 0.4021304926764314, "metric": 0.18220594501399923, "rescued": 23, "harmed": 321, "missed_rescue": 1259, "avoided_harm": 456, "selected_both_correct": 4480, "selected_both_wrong": 8}, {"budget": 0.42, "threshold": -0.0035381317138671875, "operator": "ge", "actual_fraction": 0.41311584553928093, "metric": 0.18162498930772586, "rescued": 23, "harmed": 329, "missed_rescue": 1259, "avoided_harm": 448, "selected_both_correct": 4603, "selected_both_wrong": 9}, {"budget": 0.43, "threshold": -0.003765106201171875, "operator": "ge", "actual_fraction": 0.42676431424766975, "metric": 0.18066434967152634, "rescued": 23, "harmed": 351, "missed_rescue": 1259, "avoided_harm": 426, "selected_both_correct": 4745, "selected_both_wrong": 9}, {"budget": 0.44, "threshold": -0.00392913818359375, "operator": "ge", "actual_fraction": 0.4324234354194407, "metric": 0.1803913789289444, "rescued": 23, "harmed": 355, "missed_rescue": 1259, "avoided_harm": 422, "selected_both_correct": 4809, "selected_both_wrong": 9}, {"budget": 0.45, "threshold": -0.004085540771484375, "operator": "ge", "actual_fraction": 0.44107856191744343, "metric": 0.17959468902865128, "rescued": 23, "harmed": 368, "missed_rescue": 1259, "avoided_harm": 409, "selected_both_correct": 4900, "selected_both_wrong": 9}, {"budget": 0.46, "threshold": -0.004364013671875, "operator": "ge", "actual_fraction": 0.45772303595206393, "metric": 0.17863383514615866, "rescued": 24, "harmed": 387, "missed_rescue": 1258, "avoided_harm": 390, "selected_both_correct": 5078, "selected_both_wrong": 11}, {"budget": 0.47, "threshold": -0.004680633544921875, "operator": "gt", "actual_fraction": 0.4717043941411451, "metric": 0.17957308538379646, "rescued": 26, "harmed": 405, "missed_rescue": 1256, "avoided_harm": 372, "selected_both_correct": 5225, "selected_both_wrong": 12}, {"budget": 0.48, "threshold": -0.00490570068359375, "operator": "ge", "actual_fraction": 0.4826897470039947, "metric": 0.17910245145856155, "rescued": 27, "harmed": 417, "missed_rescue": 1255, "avoided_harm": 360, "selected_both_correct": 5344, "selected_both_wrong": 12}, {"budget": 0.49, "threshold": -0.005039215087890625, "operator": "ge", "actual_fraction": 0.4896804260985353, "metric": 0.17871833907402393, "rescued": 29, "harmed": 425, "missed_rescue": 1253, "avoided_harm": 352, "selected_both_correct": 5416, "selected_both_wrong": 14}, {"budget": 0.5, "threshold": -0.005321502685546875, "operator": "ge", "actual_fraction": 0.5013315579227696, "metric": 0.18015785955296285, "rescued": 57, "harmed": 431, "missed_rescue": 1225, "avoided_harm": 346, "selected_both_correct": 5516, "selected_both_wrong": 20}, {"budget": 0.51, "threshold": -0.005619049072265625, "operator": "ge", "actual_fraction": 0.5116511318242344, "metric": 0.1810878167493375, "rescued": 76, "harmed": 435, "missed_rescue": 1206, "avoided_harm": 342, "selected_both_correct": 5606, "selected_both_wrong": 31}, {"budget": 0.52, "threshold": -0.00586700439453125, "operator": "gt", "actual_fraction": 0.5183089214380826, "metric": 0.18177962418520882, "rescued": 92, "harmed": 439, "missed_rescue": 1190, "avoided_harm": 338, "selected_both_correct": 5662, "selected_both_wrong": 35}, {"budget": 0.53, "threshold": -0.00605010986328125, "operator": "ge", "actual_fraction": 0.5252996005326231, "metric": 0.18246323539519374, "rescued": 109, "harmed": 445, "missed_rescue": 1173, "avoided_harm": 332, "selected_both_correct": 5719, "selected_both_wrong": 39}, {"budget": 0.54, "threshold": -0.006439208984375, "operator": "gt", "actual_fraction": 0.5396138482023968, "metric": 0.1839876166906015, "rescued": 150, "harmed": 461, "missed_rescue": 1132, "avoided_harm": 316, "selected_both_correct": 5829, "selected_both_wrong": 44}, {"budget": 0.55, "threshold": -0.006641387939453125, "operator": "ge", "actual_fraction": 0.5452729693741678, "metric": 0.1846674213907964, "rescued": 164, "harmed": 464, "missed_rescue": 1118, "avoided_harm": 313, "selected_both_correct": 5875, "selected_both_wrong": 49}, {"budget": 0.56, "threshold": -0.0070648193359375, "operator": "ge", "actual_fraction": 0.559254327563249, "metric": 0.18676294233791663, "rescued": 196, "harmed": 473, "missed_rescue": 1086, "avoided_harm": 304, "selected_both_correct": 5990, "selected_both_wrong": 61}, {"budget": 0.57, "threshold": -0.007518768310546875, "operator": "ge", "actual_fraction": 0.572237017310253, "metric": 0.18938663221175195, "rescued": 230, "harmed": 483, "missed_rescue": 1052, "avoided_harm": 294, "selected_both_correct": 6093, "selected_both_wrong": 70}, {"budget": 0.58, "threshold": -0.00818634033203125, "operator": "ge", "actual_fraction": 0.5862183754993342, "metric": 0.1909571002676625, "rescued": 267, "harmed": 496, "missed_rescue": 1015, "avoided_harm": 281, "selected_both_correct": 6208, "selected_both_wrong": 73}, {"budget": 0.59, "threshold": -0.00878143310546875, "operator": "gt", "actual_fraction": 0.5998668442077231, "metric": 0.1934300101790686, "rescued": 299, "harmed": 503, "missed_rescue": 983, "avoided_harm": 274, "selected_both_correct": 6315, "selected_both_wrong": 91}, {"budget": 0.6, "threshold": -0.00955963134765625, "operator": "gt", "actual_fraction": 0.6191744340878829, "metric": 0.1973701682130997, "rescued": 350, "harmed": 520, "missed_rescue": 932, "avoided_harm": 257, "selected_both_correct": 6465, "selected_both_wrong": 105}, {"budget": 0.61, "threshold": -0.0099334716796875, "operator": "ge", "actual_fraction": 0.6264980026631158, "metric": 0.1986727924998457, "rescued": 370, "harmed": 523, "missed_rescue": 912, "avoided_harm": 254, "selected_both_correct": 6527, "selected_both_wrong": 108}, {"budget": 0.62, "threshold": -0.010650634765625, "operator": "gt", "actual_fraction": 0.6384820239680427, "metric": 0.20186289191698192, "rescued": 402, "harmed": 532, "missed_rescue": 880, "avoided_harm": 245, "selected_both_correct": 6621, "selected_both_wrong": 117}, {"budget": 0.63, "threshold": -0.01129150390625, "operator": "ge", "actual_fraction": 0.6461384820239681, "metric": 0.20246031058520178, "rescued": 425, "harmed": 537, "missed_rescue": 857, "avoided_harm": 240, "selected_both_correct": 6682, "selected_both_wrong": 120}, {"budget": 0.64, "threshold": -0.011871337890625, "operator": "ge", "actual_fraction": 0.653462050599201, "metric": 0.20349944137622245, "rescued": 444, "harmed": 539, "missed_rescue": 838, "avoided_harm": 238, "selected_both_correct": 6744, "selected_both_wrong": 125}, {"budget": 0.65, "threshold": -0.0125274658203125, "operator": "gt", "actual_fraction": 0.6597869507323568, "metric": 0.20442990090722246, "rescued": 461, "harmed": 545, "missed_rescue": 821, "avoided_harm": 232, "selected_both_correct": 6792, "selected_both_wrong": 130}, {"budget": 0.66, "threshold": -0.012969970703125, "operator": "gt", "actual_fraction": 0.6661118508655126, "metric": 0.2044993942500819, "rescued": 480, "harmed": 556, "missed_rescue": 802, "avoided_harm": 221, "selected_both_correct": 6834, "selected_both_wrong": 134}, {"budget": 0.67, "threshold": -0.01363372802734375, "operator": "ge", "actual_fraction": 0.6734354194407457, "metric": 0.2064379542733177, "rescued": 500, "harmed": 561, "missed_rescue": 782, "avoided_harm": 216, "selected_both_correct": 6889, "selected_both_wrong": 142}, {"budget": 0.68, "threshold": -0.01473236083984375, "operator": "ge", "actual_fraction": 0.6820905459387483, "metric": 0.2065831206227621, "rescued": 523, "harmed": 566, "missed_rescue": 759, "avoided_harm": 211, "selected_both_correct": 6958, "selected_both_wrong": 149}, {"budget": 0.69, "threshold": -0.015716552734375, "operator": "ge", "actual_fraction": 0.6927430093209055, "metric": 0.20802802394105854, "rescued": 553, "harmed": 577, "missed_rescue": 729, "avoided_harm": 200, "selected_both_correct": 7035, "selected_both_wrong": 159}, {"budget": 0.7, "threshold": -0.017578125, "operator": "gt", "actual_fraction": 0.7090545938748336, "metric": 0.21169680596278434, "rescued": 602, "harmed": 584, "missed_rescue": 680, "avoided_harm": 193, "selected_both_correct": 7159, "selected_both_wrong": 175}, {"budget": 0.71, "threshold": -0.0189056396484375, "operator": "ge", "actual_fraction": 0.7170439414114514, "metric": 0.2139280527103987, "rescued": 625, "harmed": 588, "missed_rescue": 657, "avoided_harm": 189, "selected_both_correct": 7220, "selected_both_wrong": 183}, {"budget": 0.72, "threshold": -0.019866943359375, "operator": "ge", "actual_fraction": 0.7243675099866844, "metric": 0.2146900231025185, "rescued": 647, "harmed": 594, "missed_rescue": 635, "avoided_harm": 183, "selected_both_correct": 7268, "selected_both_wrong": 195}, {"budget": 0.73, "threshold": -0.020965576171875, "operator": "ge", "actual_fraction": 0.7330226364846871, "metric": 0.21653561577692484, "rescued": 670, "harmed": 599, "missed_rescue": 612, "avoided_harm": 178, "selected_both_correct": 7338, "selected_both_wrong": 201}, {"budget": 0.74, "threshold": -0.021453857421875, "operator": "ge", "actual_fraction": 0.7363515312916112, "metric": 0.21721689648133474, "rescued": 681, "harmed": 601, "missed_rescue": 601, "avoided_harm": 176, "selected_both_correct": 7365, "selected_both_wrong": 201}, {"budget": 0.75, "threshold": -0.0248870849609375, "operator": "ge", "actual_fraction": 0.7486684420772304, "metric": 0.22103445955201576, "rescued": 718, "harmed": 610, "missed_rescue": 564, "avoided_harm": 167, "selected_both_correct": 7460, "selected_both_wrong": 208}, {"budget": 0.76, "threshold": -0.0269622802734375, "operator": "ge", "actual_fraction": 0.757323568575233, "metric": 0.22340134248367294, "rescued": 744, "harmed": 618, "missed_rescue": 538, "avoided_harm": 159, "selected_both_correct": 7522, "selected_both_wrong": 216}, {"budget": 0.77, "threshold": -0.0282135009765625, "operator": "ge", "actual_fraction": 0.7639813581890812, "metric": 0.2269322568854726, "rescued": 767, "harmed": 620, "missed_rescue": 515, "avoided_harm": 157, "selected_both_correct": 7571, "selected_both_wrong": 222}, {"budget": 0.78, "threshold": -0.0299835205078125, "operator": "ge", "actual_fraction": 0.7733022636484687, "metric": 0.22757452798038888, "rescued": 798, "harmed": 628, "missed_rescue": 484, "avoided_harm": 149, "selected_both_correct": 7636, "selected_both_wrong": 230}, {"budget": 0.79, "threshold": -0.033966064453125, "operator": "ge", "actual_fraction": 0.784953395472703, "metric": 0.227821050912831, "rescued": 824, "harmed": 634, "missed_rescue": 458, "avoided_harm": 143, "selected_both_correct": 7728, "selected_both_wrong": 246}, {"budget": 0.8, "threshold": -0.037200927734375, "operator": "ge", "actual_fraction": 0.7939414114513982, "metric": 0.22991865671924006, "rescued": 852, "harmed": 642, "missed_rescue": 430, "avoided_harm": 135, "selected_both_correct": 7793, "selected_both_wrong": 253}, {"budget": 0.81, "threshold": -0.04132080078125, "operator": "ge", "actual_fraction": 0.8039280958721704, "metric": 0.23486191981046528, "rescued": 884, "harmed": 646, "missed_rescue": 398, "avoided_harm": 131, "selected_both_correct": 7868, "selected_both_wrong": 262}, {"budget": 0.82, "threshold": -0.0455322265625, "operator": "ge", "actual_fraction": 0.8122503328894807, "metric": 0.23803916047544685, "rescued": 906, "harmed": 655, "missed_rescue": 376, "avoided_harm": 122, "selected_both_correct": 7924, "selected_both_wrong": 275}, {"budget": 0.83, "threshold": -0.051849365234375, "operator": "ge", "actual_fraction": 0.8215712383488681, "metric": 0.24022947987632814, "rescued": 933, "harmed": 664, "missed_rescue": 349, "avoided_harm": 113, "selected_both_correct": 7987, "selected_both_wrong": 288}, {"budget": 0.84, "threshold": -0.06878662109375, "operator": "ge", "actual_fraction": 0.8388814913448736, "metric": 0.24528627405740036, "rescued": 988, "harmed": 672, "missed_rescue": 294, "avoided_harm": 105, "selected_both_correct": 8116, "selected_both_wrong": 304}, {"budget": 0.85, "threshold": -0.08477783203125, "operator": "ge", "actual_fraction": 0.8535286284953395, "metric": 0.249537136326177, "rescued": 1028, "harmed": 680, "missed_rescue": 254, "avoided_harm": 97, "selected_both_correct": 8224, "selected_both_wrong": 324}, {"budget": 0.86, "threshold": -0.09912109375, "operator": "ge", "actual_fraction": 0.8638482023968043, "metric": 0.25211074301800224, "rescued": 1056, "harmed": 688, "missed_rescue": 226, "avoided_harm": 89, "selected_both_correct": 8299, "selected_both_wrong": 337}, {"budget": 0.87, "threshold": -0.1298828125, "operator": "ge", "actual_fraction": 0.8745006657789614, "metric": 0.2540042317315216, "rescued": 1076, "harmed": 695, "missed_rescue": 206, "avoided_harm": 82, "selected_both_correct": 8389, "selected_both_wrong": 348}, {"budget": 0.88, "threshold": -0.1441650390625, "operator": "ge", "actual_fraction": 0.8764980026631158, "metric": 0.2540996517958595, "rescued": 1080, "harmed": 698, "missed_rescue": 202, "avoided_harm": 79, "selected_both_correct": 8404, "selected_both_wrong": 350}, {"budget": 0.89, "threshold": -0.182373046875, "operator": "ge", "actual_fraction": 0.8841544607190412, "metric": 0.25436194678023316, "rescued": 1090, "harmed": 705, "missed_rescue": 192, "avoided_harm": 72, "selected_both_correct": 8468, "selected_both_wrong": 361}, {"budget": 0.9, "threshold": -0.2049560546875, "operator": "ge", "actual_fraction": 0.8891478029294274, "metric": 0.2534339447803836, "rescued": 1101, "harmed": 709, "missed_rescue": 181, "avoided_harm": 68, "selected_both_correct": 8506, "selected_both_wrong": 368}, {"budget": 0.91, "threshold": -0.25830078125, "operator": "gt", "actual_fraction": 0.8981358189081226, "metric": 0.2563240880787248, "rescued": 1126, "harmed": 715, "missed_rescue": 156, "avoided_harm": 62, "selected_both_correct": 8573, "selected_both_wrong": 378}, {"budget": 0.92, "threshold": -0.32080078125, "operator": "ge", "actual_fraction": 0.9054593874833555, "metric": 0.25551845731687656, "rescued": 1143, "harmed": 721, "missed_rescue": 139, "avoided_harm": 56, "selected_both_correct": 8632, "selected_both_wrong": 384}, {"budget": 0.93, "threshold": -0.453369140625, "operator": "ge", "actual_fraction": 0.9221038615179761, "metric": 0.2597490004762105, "rescued": 1180, "harmed": 728, "missed_rescue": 102, "avoided_harm": 49, "selected_both_correct": 8774, "selected_both_wrong": 398}, {"budget": 0.94, "threshold": -0.490234375, "operator": "ge", "actual_fraction": 0.9254327563249002, "metric": 0.26306323674307786, "rescued": 1189, "harmed": 729, "missed_rescue": 93, "avoided_harm": 48, "selected_both_correct": 8800, "selected_both_wrong": 402}, {"budget": 0.95, "threshold": -0.611328125, "operator": "ge", "actual_fraction": 0.9364181091877497, "metric": 0.26218185491449153, "rescued": 1203, "harmed": 738, "missed_rescue": 79, "avoided_harm": 39, "selected_both_correct": 8897, "selected_both_wrong": 414}, {"budget": 0.96, "threshold": -0.70068359375, "operator": "ge", "actual_fraction": 0.953728362183755, "metric": 0.26357052137555514, "rescued": 1223, "harmed": 754, "missed_rescue": 59, "avoided_harm": 23, "selected_both_correct": 9055, "selected_both_wrong": 428}, {"budget": 0.97, "threshold": -0.75048828125, "operator": "ge", "actual_fraction": 0.96271637816245, "metric": 0.26400290307863034, "rescued": 1231, "harmed": 758, "missed_rescue": 51, "avoided_harm": 19, "selected_both_correct": 9150, "selected_both_wrong": 429}, {"budget": 0.98, "threshold": -0.7841796875, "operator": "ge", "actual_fraction": 0.9700399467376831, "metric": 0.2647475724296899, "rescued": 1237, "harmed": 760, "missed_rescue": 45, "avoided_harm": 17, "selected_both_correct": 9227, "selected_both_wrong": 432}, {"budget": 0.99, "threshold": -0.85107421875, "operator": "ge", "actual_fraction": 0.987017310252996, "metric": 0.2649306125590882, "rescued": 1257, "harmed": 772, "missed_rescue": 25, "avoided_harm": 5, "selected_both_correct": 9387, "selected_both_wrong": 444}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.27001009067886245, "rescued": 1282, "harmed": 777, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 9508, "selected_both_wrong": 449}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.32798023346814337, "actual_fraction": 0.45135520684736086, "metric_std": 0.00013542191772614485, "actual_fraction_std": 0.004465756303137945, "per_seed": [{"seed": 2026, "budget": 0.45, "threshold": 0.015573756125582155, "operator": "ge", "alpha": 0.75, "epoch": 4, "metric": 0.3281285807463818, "actual_fraction": 0.44935805991440797, "rescue": 298, "harm": 84, "missed_rescue": 9}, {"seed": 2027, "budget": 0.45, "threshold": 0.023982490021399305, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.32788133528265107, "actual_fraction": 0.44935805991440797, "rescue": 298, "harm": 84, "missed_rescue": 9}, {"seed": 2028, "budget": 0.46, "threshold": -0.028658352427870838, "operator": "ge", "alpha": 0.25, "epoch": 4, "metric": 0.3281285807463818, "actual_fraction": 0.45934379457917257, "rescue": 298, "harm": 84, "missed_rescue": 9}, {"seed": 2029, "budget": 0.45, "threshold": 0.046315565722561215, "operator": "ge", "alpha": 0.25, "epoch": 4, "metric": 0.32788133528265107, "actual_fraction": 0.44935805991440797, "rescue": 298, "harm": 84, "missed_rescue": 9}, {"seed": 2030, "budget": 0.45, "threshold": -0.005500473325143033, "operator": "ge", "alpha": 0.75, "epoch": 4, "metric": 0.32788133528265107, "actual_fraction": 0.44935805991440797, "rescue": 298, "harm": 84, "missed_rescue": 9}], "rescued": 298.0, "harmed": 84.0, "missed_rescue": 9.0, "avoided_harm": 136.0}, "validation_curve": [{"budget": 0.0, "metric": 0.18254072168965788, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.185788780099517, "metric_std": 0.0004061326320205983, "actual_fraction": 0.009985734664764621, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.19000982905854907, "metric_std": 0.0008155498257606541, "actual_fraction": 0.019971469329529243, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.19514621055740253, "metric_std": 0.0005311438182522545, "actual_fraction": 0.029957203994293864, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.20096838698744873, "metric_std": 0.0020165242161687123, "actual_fraction": 0.039942938659058486, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.20546303233629257, "metric_std": 0.002006283548997381, "actual_fraction": 0.04992867332382311, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.21162842133897541, "metric_std": 0.004237377020063226, "actual_fraction": 0.05991440798858773, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.21802295496993493, "metric_std": 0.0050096293875118835, "actual_fraction": 0.06990014265335236, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.22392374348527655, "metric_std": 0.005499890780915585, "actual_fraction": 0.07988587731811697, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.22734825211816304, "metric_std": 0.007159499128586012, "actual_fraction": 0.0898716119828816, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.23157273800881573, "metric_std": 0.004760740936534534, "actual_fraction": 0.09985734664764621, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.23388520173206576, "metric_std": 0.0048288825228009445, "actual_fraction": 0.10984308131241086, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.12, "metric": 0.23620822138221037, "metric_std": 0.005101558898065631, "actual_fraction": 0.11982881597717546, "actual_fraction_std": 0.0}, {"budget": 0.13, "metric": 0.23982664034442416, "metric_std": 0.00545041183615696, "actual_fraction": 0.12981455064194009, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.24281402973418026, "metric_std": 0.004552140945044228, "actual_fraction": 0.1398002853067047, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.24457629407306097, "metric_std": 0.004033261155582126, "actual_fraction": 0.14978601997146934, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.24551117361861827, "metric_std": 0.002622450366919109, "actual_fraction": 0.15977175463623394, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.24785713537489648, "metric_std": 0.002736525511381191, "actual_fraction": 0.16975748930099857, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.24946587285039534, "metric_std": 0.0022359682775620086, "actual_fraction": 0.1797432239657632, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.2525330834532621, "metric_std": 0.002213705374406783, "actual_fraction": 0.18972895863052786, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.2555289307302101, "metric_std": 0.003506905269376495, "actual_fraction": 0.19971469329529243, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.2577364003437821, "metric_std": 0.0024148782589033034, "actual_fraction": 0.20970042796005703, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.261291262181055, "metric_std": 0.002860060622316392, "actual_fraction": 0.2196861626248217, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.23, "metric": 0.2632267677055301, "metric_std": 0.0036929752353654934, "actual_fraction": 0.22967189728958629, "actual_fraction_std": 0.0}, {"budget": 0.24, "metric": 0.2653164842681211, "metric_std": 0.003767298202440387, "actual_fraction": 0.2396576319543509, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.2684294899124301, "metric_std": 0.0020142864810541424, "actual_fraction": 0.24964336661911551, "actual_fraction_std": 0.0}, {"budget": 0.26, "metric": 0.27180941670077197, "metric_std": 0.0018569269170122795, "actual_fraction": 0.25962910128388017, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.27555330404280853, "metric_std": 0.003928765001224238, "actual_fraction": 0.26961483594864477, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.27822329322415507, "metric_std": 0.004373627241845955, "actual_fraction": 0.2796005706134094, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.2807395724123204, "metric_std": 0.0032654138257328068, "actual_fraction": 0.289586305278174, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.2805421641986968, "metric_std": 0.00725976196960327, "actual_fraction": 0.2995720399429387, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.28197676911969777, "metric_std": 0.010532664796664835, "actual_fraction": 0.3095577746077033, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.2827207086355976, "metric_std": 0.010470890993027754, "actual_fraction": 0.3195435092724679, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.28556391244501894, "metric_std": 0.010164940456265075, "actual_fraction": 0.3295292439372325, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.2873989803600749, "metric_std": 0.007154982879561229, "actual_fraction": 0.33951497860199714, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.29199342706318426, "metric_std": 0.005236305222812428, "actual_fraction": 0.34950071326676174, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.2933470596766348, "metric_std": 0.003716390790881962, "actual_fraction": 0.3594864479315264, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.2943486589675873, "metric_std": 0.006113450797192407, "actual_fraction": 0.36947218259629094, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.29970175379366676, "metric_std": 0.006073771533841333, "actual_fraction": 0.3794579172610557, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.3015562322881789, "metric_std": 0.005660739181123813, "actual_fraction": 0.38944365192582026, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.3044210657318992, "metric_std": 0.003643751389633866, "actual_fraction": 0.39942938659058486, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.3091782687807298, "metric_std": 0.008667225167322422, "actual_fraction": 0.4094151212553495, "actual_fraction_std": 0.0}, {"budget": 0.42, "metric": 0.3111950457862032, "metric_std": 0.008845057496164892, "actual_fraction": 0.41940085592011406, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.31266854160330515, "metric_std": 0.008896638809820397, "actual_fraction": 0.4293865905848787, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.44, "metric": 0.316963700455159, "metric_std": 0.007414960631401855, "actual_fraction": 0.4393723252496434, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.45, "metric": 0.3272265236614811, "metric_std": 0.0016059901815064727, "actual_fraction": 0.44935805991440797, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.3268259389284465, "metric_std": 0.0010654494085346777, "actual_fraction": 0.45934379457917257, "actual_fraction_std": 0.0}, {"budget": 0.47, "metric": 0.3240801205513569, "metric_std": 0.001236912902158883, "actual_fraction": 0.4693295292439372, "actual_fraction_std": 0.0}, {"budget": 0.48, "metric": 0.3216957242533992, "metric_std": 0.0015374985093369608, "actual_fraction": 0.4793152639087018, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.3203446365414787, "metric_std": 0.0009912029398095538, "actual_fraction": 0.4893009985734665, "actual_fraction_std": 0.0}, {"budget": 0.5, "metric": 0.3195900445059678, "metric_std": 0.0009289610100700262, "actual_fraction": 0.49928673323823103, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.3210010530842741, "metric_std": 0.003845495557173994, "actual_fraction": 0.5092724679029957, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.3205734995812588, "metric_std": 0.004623211361410618, "actual_fraction": 0.5192582025677603, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.3207691468652326, "metric_std": 0.004774695597478126, "actual_fraction": 0.5292439372325249, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.3202245945990815, "metric_std": 0.004541258489114731, "actual_fraction": 0.5392296718972895, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.31827081273562563, "metric_std": 0.004709214736237644, "actual_fraction": 0.5492154065620543, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.31678166249788653, "metric_std": 0.0036688630971361234, "actual_fraction": 0.5592011412268189, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.317182130621585, "metric_std": 0.0038895723906385046, "actual_fraction": 0.5691868758915835, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.31820175390303373, "metric_std": 0.003109375254540461, "actual_fraction": 0.579172610556348, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.31778308175563813, "metric_std": 0.003471297808044635, "actual_fraction": 0.5891583452211127, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.3167601278942825, "metric_std": 0.004158368284930865, "actual_fraction": 0.5991440798858774, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.31597480880560613, "metric_std": 0.004154312612703999, "actual_fraction": 0.609129814550642, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.3168462075874646, "metric_std": 0.00399848534936111, "actual_fraction": 0.6191155492154066, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.3153511132906385, "metric_std": 0.0033586760501744255, "actual_fraction": 0.6291012838801712, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.31410662827817754, "metric_std": 0.003805044026632794, "actual_fraction": 0.6390870185449358, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.31485752730741223, "metric_std": 0.004781248155958444, "actual_fraction": 0.6490727532097005, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.3156734858514754, "metric_std": 0.005788520550007701, "actual_fraction": 0.659058487874465, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.31332319037886547, "metric_std": 0.005010026129235311, "actual_fraction": 0.6690442225392297, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.3114199562811094, "metric_std": 0.005807313679139454, "actual_fraction": 0.6790299572039943, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.3105955504840529, "metric_std": 0.005503451929403116, "actual_fraction": 0.6890156918687589, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.30958479095922137, "metric_std": 0.005703261994209684, "actual_fraction": 0.6990014265335235, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.30787310129652024, "metric_std": 0.005913148195785456, "actual_fraction": 0.7089871611982882, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.30577885413084294, "metric_std": 0.004509760833154281, "actual_fraction": 0.7189728958630528, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.30564148802557334, "metric_std": 0.004693536971433788, "actual_fraction": 0.7289586305278174, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.30544887017114675, "metric_std": 0.006682392500756683, "actual_fraction": 0.7389443651925819, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.3052059693837779, "metric_std": 0.005792739135018072, "actual_fraction": 0.7489300998573466, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.3039824930849123, "metric_std": 0.0031884895943319986, "actual_fraction": 0.7589158345221114, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.30387642326891173, "metric_std": 0.002934752514016686, "actual_fraction": 0.7689015691868759, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.30123693449834066, "metric_std": 0.004326589110271964, "actual_fraction": 0.7788873038516405, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.2987484036947057, "metric_std": 0.004754958169630352, "actual_fraction": 0.7888730385164051, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.2978234896702188, "metric_std": 0.00507227164197391, "actual_fraction": 0.7988587731811697, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.29513412504320724, "metric_std": 0.004088094214785229, "actual_fraction": 0.8088445078459344, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.29300825159100613, "metric_std": 0.004995181426432676, "actual_fraction": 0.818830242510699, "actual_fraction_std": 0.0}, {"budget": 0.83, "metric": 0.2916173634105682, "metric_std": 0.0046761139804475105, "actual_fraction": 0.8288159771754635, "actual_fraction_std": 0.0}, {"budget": 0.84, "metric": 0.28961064116948726, "metric_std": 0.004466123509172176, "actual_fraction": 0.8388017118402281, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.28776363861489723, "metric_std": 0.005206170448612012, "actual_fraction": 0.8487874465049929, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.86, "metric": 0.28440737568091884, "metric_std": 0.004946146781939293, "actual_fraction": 0.8587731811697574, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.87, "metric": 0.28141410719369586, "metric_std": 0.0039784289109150404, "actual_fraction": 0.8687589158345222, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.2786464654445557, "metric_std": 0.0037292318814998947, "actual_fraction": 0.8787446504992869, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.89, "metric": 0.27650192309938354, "metric_std": 0.0025425498377929464, "actual_fraction": 0.8887303851640513, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.27470154082203513, "metric_std": 0.0033459166107049562, "actual_fraction": 0.8987161198288159, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.27331491953443177, "metric_std": 0.0021767655985309754, "actual_fraction": 0.9087018544935808, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.26939511912274333, "metric_std": 0.0005826953837829392, "actual_fraction": 0.9186875891583451, "actual_fraction_std": 0.0}, {"budget": 0.93, "metric": 0.2688213730308589, "metric_std": 0.0009425171057370925, "actual_fraction": 0.9286733238231101, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.94, "metric": 0.2667797339602439, "metric_std": 0.0008079793763427523, "actual_fraction": 0.9386590584878745, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.26514170471064447, "metric_std": 0.0022899848561648804, "actual_fraction": 0.948644793152639, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.2642465041877276, "metric_std": 0.0025117530568304043, "actual_fraction": 0.9586305278174037, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.2609620676141799, "metric_std": 0.0017453928519938222, "actual_fraction": 0.9686162624821684, "actual_fraction_std": 0.0}, {"budget": 0.98, "metric": 0.25827861997038015, "metric_std": 0.0022900249159270723, "actual_fraction": 0.978601997146933, "actual_fraction_std": 0.0}, {"budget": 0.99, "metric": 0.2565890923758709, "metric_std": 0.0007272762053378024, "actual_fraction": 0.9885877318116976, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.25448626222520154, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.2979678330127048, "actual_fraction": 0.44766977363515315, "metric_std": 0.0031655067078900046, "actual_fraction_std": 0.003373535568617398, "per_seed": [{"seed": 2026, "budget": 0.45, "threshold": 0.015573756125582155, "operator": "ge", "alpha": 0.75, "epoch": 4, "metric": 0.2994456979113117, "actual_fraction": 0.4487350199733689, "rescue": 1234, "harm": 316, "missed_rescue": 48}, {"seed": 2027, "budget": 0.45, "threshold": 0.023982490021399305, "operator": "ge", "alpha": 0.5, "epoch": 3, "metric": 0.299648658140793, "actual_fraction": 0.44474034620505987, "rescue": 1229, "harm": 315, "missed_rescue": 53}, {"seed": 2028, "budget": 0.46, "threshold": -0.028658352427870838, "operator": "ge", "alpha": 0.25, "epoch": 4, "metric": 0.2923257040620524, "actual_fraction": 0.4530625832223702, "rescue": 1234, "harm": 316, "missed_rescue": 48}, {"seed": 2029, "budget": 0.45, "threshold": 0.046315565722561215, "operator": "ge", "alpha": 0.25, "epoch": 4, "metric": 0.29893105690852895, "actual_fraction": 0.44540612516644473, "rescue": 1231, "harm": 315, "missed_rescue": 51}, {"seed": 2030, "budget": 0.45, "threshold": -0.005500473325143033, "operator": "ge", "alpha": 0.75, "epoch": 4, "metric": 0.2994880480408382, "actual_fraction": 0.446404793608522, "rescue": 1230, "harm": 313, "missed_rescue": 52}], "rescued": 1231.6, "harmed": 315.0, "missed_rescue": 50.4, "avoided_harm": 462.0}, "test_curve": [{"budget": 0.0, "metric": 0.18129624718318374, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.18372855448704165, "metric_std": 0.0017904492102250018, "actual_fraction": 0.007789613848202396, "actual_fraction_std": 0.0029830315335845343}, {"budget": 0.02, "metric": 0.18685908224889786, "metric_std": 0.0017029005625388614, "actual_fraction": 0.018908122503328895, "actual_fraction_std": 0.007504424662154166}, {"budget": 0.03, "metric": 0.1884468119835047, "metric_std": 0.0025376052946853972, "actual_fraction": 0.029227696404793613, "actual_fraction_std": 0.006391998147945036}, {"budget": 0.04, "metric": 0.19016333859244455, "metric_std": 0.002821421049597687, "actual_fraction": 0.037150466045272966, "actual_fraction_std": 0.007276510232023008}, {"budget": 0.05, "metric": 0.19203936879648048, "metric_std": 0.00491480251401695, "actual_fraction": 0.047203728362183756, "actual_fraction_std": 0.011171405938275949}, {"budget": 0.06, "metric": 0.1973276972772499, "metric_std": 0.005390596624967683, "actual_fraction": 0.06245006657789613, "actual_fraction_std": 0.009298884964302086}, {"budget": 0.07, "metric": 0.19856778453770563, "metric_std": 0.002464801477637776, "actual_fraction": 0.07423435419440745, "actual_fraction_std": 0.008503355375795472}, {"budget": 0.08, "metric": 0.19964444666304987, "metric_std": 0.001150911291712205, "actual_fraction": 0.08342210386151798, "actual_fraction_std": 0.0072114956980305}, {"budget": 0.09, "metric": 0.2008447299915772, "metric_std": 0.0019195246602231296, "actual_fraction": 0.09094540612516644, "actual_fraction_std": 0.004922939098748167}, {"budget": 0.1, "metric": 0.2053760837500663, "metric_std": 0.003628384040168178, "actual_fraction": 0.10252996005326231, "actual_fraction_std": 0.0038462782983659663}, {"budget": 0.11, "metric": 0.2084699431938867, "metric_std": 0.0025360549300233984, "actual_fraction": 0.11284953395472704, "actual_fraction_std": 0.003846278298365958}, {"budget": 0.12, "metric": 0.21117730070554924, "metric_std": 0.003452093378428324, "actual_fraction": 0.12343541944074565, "actual_fraction_std": 0.0061354701155672325}, {"budget": 0.13, "metric": 0.21355089822347736, "metric_std": 0.0035324427684870607, "actual_fraction": 0.1318908122503329, "actual_fraction_std": 0.0076361683339453415}, {"budget": 0.14, "metric": 0.21604151501630087, "metric_std": 0.0031825313569350845, "actual_fraction": 0.14500665778961386, "actual_fraction_std": 0.009367157453166549}, {"budget": 0.15, "metric": 0.21788898058414213, "metric_std": 0.0025763261283533946, "actual_fraction": 0.1533288948069241, "actual_fraction_std": 0.008469404497012781}, {"budget": 0.16, "metric": 0.2206453715306817, "metric_std": 0.00230752851231373, "actual_fraction": 0.16624500665778957, "actual_fraction_std": 0.010814041720241828}, {"budget": 0.17, "metric": 0.22256047317491415, "metric_std": 0.003253213734436099, "actual_fraction": 0.17569906790945405, "actual_fraction_std": 0.011814997561837586}, {"budget": 0.18, "metric": 0.22392279063908965, "metric_std": 0.004172922803741693, "actual_fraction": 0.18202396804260984, "actual_fraction_std": 0.013230838878167932}, {"budget": 0.19, "metric": 0.22639256671969005, "metric_std": 0.002826168923464902, "actual_fraction": 0.1914780292942743, "actual_fraction_std": 0.012416376014038779}, {"budget": 0.2, "metric": 0.2303544739053073, "metric_std": 0.0013314062339103835, "actual_fraction": 0.2051264980026631, "actual_fraction_std": 0.009120807286155977}, {"budget": 0.21, "metric": 0.2328940374950383, "metric_std": 0.002501549212840741, "actual_fraction": 0.2138482023968043, "actual_fraction_std": 0.007261265064988609}, {"budget": 0.22, "metric": 0.23741897056590364, "metric_std": 0.0021505457461758913, "actual_fraction": 0.22656458055925435, "actual_fraction_std": 0.0026485118882711143}, {"budget": 0.23, "metric": 0.24026689104972293, "metric_std": 0.0017997740057847647, "actual_fraction": 0.23794940079893476, "actual_fraction_std": 0.006880431788422815}, {"budget": 0.24, "metric": 0.2431042580703525, "metric_std": 0.0030496147888626913, "actual_fraction": 0.2474034620505992, "actual_fraction_std": 0.008300237233435887}, {"budget": 0.25, "metric": 0.2446393126551028, "metric_std": 0.0032239176774468926, "actual_fraction": 0.2549267643142477, "actual_fraction_std": 0.007928082900987574}, {"budget": 0.26, "metric": 0.24855675020919854, "metric_std": 0.0019372091121957673, "actual_fraction": 0.26198402130492676, "actual_fraction_std": 0.00569328607739249}, {"budget": 0.27, "metric": 0.25234723459722813, "metric_std": 0.0032027241320088796, "actual_fraction": 0.2728362183754993, "actual_fraction_std": 0.010580964277471405}, {"budget": 0.28, "metric": 0.2555614615399192, "metric_std": 0.003577881240287513, "actual_fraction": 0.28169107856191744, "actual_fraction_std": 0.009108649446459371}, {"budget": 0.29, "metric": 0.25818509905301673, "metric_std": 0.004895295755060871, "actual_fraction": 0.2905459387483355, "actual_fraction_std": 0.007973380627202223}, {"budget": 0.3, "metric": 0.2606012166289425, "metric_std": 0.004750286015588925, "actual_fraction": 0.30099866844207723, "actual_fraction_std": 0.007704791573088038}, {"budget": 0.31, "metric": 0.2653611798288512, "metric_std": 0.00354978331151432, "actual_fraction": 0.31178428761651134, "actual_fraction_std": 0.0075859370232355325}, {"budget": 0.32, "metric": 0.26884911383364996, "metric_std": 0.0034970613210190482, "actual_fraction": 0.3237017310252996, "actual_fraction_std": 0.006122814056471162}, {"budget": 0.33, "metric": 0.2729730863348169, "metric_std": 0.002584021694793537, "actual_fraction": 0.336884154460719, "actual_fraction_std": 0.006793718918761165}, {"budget": 0.34, "metric": 0.27765278557626005, "metric_std": 0.005058246200798084, "actual_fraction": 0.34800266311584555, "actual_fraction_std": 0.008001821154731062}, {"budget": 0.35, "metric": 0.2796074774091736, "metric_std": 0.004826446189064472, "actual_fraction": 0.3575233022636485, "actual_fraction_std": 0.0047195229290139026}, {"budget": 0.36, "metric": 0.28204715333825625, "metric_std": 0.0056393456515187925, "actual_fraction": 0.3648468708388815, "actual_fraction_std": 0.002844208969812759}, {"budget": 0.37, "metric": 0.2865153302629788, "metric_std": 0.003553072156480893, "actual_fraction": 0.37490013315579224, "actual_fraction_std": 0.0034945468633850066}, {"budget": 0.38, "metric": 0.2895286620478625, "metric_std": 0.003056790725018284, "actual_fraction": 0.3859520639147802, "actual_fraction_std": 0.0023015161141735933}, {"budget": 0.39, "metric": 0.2924523439388038, "metric_std": 0.0021465490229648668, "actual_fraction": 0.39387483355525965, "actual_fraction_std": 0.004317308345372443}, {"budget": 0.4, "metric": 0.29493220967499056, "metric_std": 0.0016370746528689797, "actual_fraction": 0.403595206391478, "actual_fraction_std": 0.0010151764768293486}, {"budget": 0.41, "metric": 0.29679244853296216, "metric_std": 0.0013392596478636284, "actual_fraction": 0.4097869507323568, "actual_fraction_std": 0.0035150985489252186}, {"budget": 0.42, "metric": 0.29824032060818284, "metric_std": 0.002136275334471677, "actual_fraction": 0.4171105193075899, "actual_fraction_std": 0.003593048126651599}, {"budget": 0.43, "metric": 0.30060258544213647, "metric_std": 0.0029601659682642763, "actual_fraction": 0.42676431424766975, "actual_fraction_std": 0.004484751526400603}, {"budget": 0.44, "metric": 0.30092311829652235, "metric_std": 0.003192249791677643, "actual_fraction": 0.4334886817576565, "actual_fraction_std": 0.0028867487867748017}, {"budget": 0.45, "metric": 0.29908337970039717, "metric_std": 0.0007123967460060591, "actual_fraction": 0.44560585885486015, "actual_fraction_std": 0.0022031147946437674}, {"budget": 0.46, "metric": 0.2977899223131896, "metric_std": 0.003088363579488726, "actual_fraction": 0.4533288948069242, "actual_fraction_std": 0.0013603739508457177}, {"budget": 0.47, "metric": 0.2982768381293317, "metric_std": 0.001312700413574007, "actual_fraction": 0.45952063914780295, "actual_fraction_std": 0.0016068599715242488}, {"budget": 0.48, "metric": 0.29713930300206126, "metric_std": 0.0018380665606655927, "actual_fraction": 0.46677762982689747, "actual_fraction_std": 0.0025504635565083774}, {"budget": 0.49, "metric": 0.2981419006299502, "metric_std": 0.0005785680956027753, "actual_fraction": 0.4787616511318243, "actual_fraction_std": 0.006135470115567245}, {"budget": 0.5, "metric": 0.30134517763453006, "metric_std": 0.0025270927988784325, "actual_fraction": 0.4900133155792277, "actual_fraction_std": 0.0053779941482701415}, {"budget": 0.51, "metric": 0.3013386133731364, "metric_std": 0.003034739061112812, "actual_fraction": 0.5019973368841545, "actual_fraction_std": 0.0038462782983659668}, {"budget": 0.52, "metric": 0.3006032949818893, "metric_std": 0.0029262997361378445, "actual_fraction": 0.5176431424766977, "actual_fraction_std": 0.006752817168808528}, {"budget": 0.53, "metric": 0.29926650757486717, "metric_std": 0.0029996256801084597, "actual_fraction": 0.5346205059920106, "actual_fraction_std": 0.0056099532444582965}, {"budget": 0.54, "metric": 0.29911586063303675, "metric_std": 0.003192097202845822, "actual_fraction": 0.5474700399467377, "actual_fraction_std": 0.0070167859482687086}, {"budget": 0.55, "metric": 0.29813893282180814, "metric_std": 0.002046793703478832, "actual_fraction": 0.5575898801597869, "actual_fraction_std": 0.00704987303080802}, {"budget": 0.56, "metric": 0.2974509950363013, "metric_std": 0.001930815175165221, "actual_fraction": 0.5675765645805592, "actual_fraction_std": 0.008782227668623787}, {"budget": 0.57, "metric": 0.29724357969129245, "metric_std": 0.0017471524892829426, "actual_fraction": 0.5804260985352863, "actual_fraction_std": 0.00462824135590932}, {"budget": 0.58, "metric": 0.29665775151403084, "metric_std": 0.002678970681690905, "actual_fraction": 0.5938082556591212, "actual_fraction_std": 0.004336516396666341}, {"budget": 0.59, "metric": 0.2975661409457673, "metric_std": 0.0031563081106147483, "actual_fraction": 0.603595206391478, "actual_fraction_std": 0.004603032228347186}, {"budget": 0.6, "metric": 0.29631335243812956, "metric_std": 0.003380854299800894, "actual_fraction": 0.6135153129161118, "actual_fraction_std": 0.00662023362228741}, {"budget": 0.61, "metric": 0.2960348088956203, "metric_std": 0.003468590773594877, "actual_fraction": 0.6214380825565912, "actual_fraction_std": 0.009901433734710068}, {"budget": 0.62, "metric": 0.2951989613231927, "metric_std": 0.0030052924087018464, "actual_fraction": 0.6318242343541943, "actual_fraction_std": 0.0078034009119380575}, {"budget": 0.63, "metric": 0.29386485972911613, "metric_std": 0.0031357317886145796, "actual_fraction": 0.6431424766977363, "actual_fraction_std": 0.006874792419461787}, {"budget": 0.64, "metric": 0.292479005789999, "metric_std": 0.0023189407158123315, "actual_fraction": 0.6534620505992011, "actual_fraction_std": 0.009570210726674478}, {"budget": 0.65, "metric": 0.2926539927366717, "metric_std": 0.002905913533355878, "actual_fraction": 0.6655126498002664, "actual_fraction_std": 0.008439257174940455}, {"budget": 0.66, "metric": 0.29278795045342987, "metric_std": 0.0023318839191906276, "actual_fraction": 0.6742343541944075, "actual_fraction_std": 0.008122094028237156}, {"budget": 0.67, "metric": 0.29144368638516527, "metric_std": 0.0026754778469665614, "actual_fraction": 0.6844207723035952, "actual_fraction_std": 0.007620188510159544}, {"budget": 0.68, "metric": 0.2913839507485529, "metric_std": 0.0030040292848531286, "actual_fraction": 0.6950732356857523, "actual_fraction_std": 0.004748782495901212}, {"budget": 0.69, "metric": 0.2916685120367808, "metric_std": 0.002431135193703369, "actual_fraction": 0.70219707057257, "actual_fraction_std": 0.0023842882262942442}, {"budget": 0.7, "metric": 0.2908903102338221, "metric_std": 0.002494254362738777, "actual_fraction": 0.70745672436751, "actual_fraction_std": 0.0035961309655942824}, {"budget": 0.71, "metric": 0.29068952549932253, "metric_std": 0.003129248911442597, "actual_fraction": 0.7203728362183754, "actual_fraction_std": 0.007838822795816635}, {"budget": 0.72, "metric": 0.28982721462710204, "metric_std": 0.0030250065905502704, "actual_fraction": 0.7292942743009322, "actual_fraction_std": 0.00809065266429662}, {"budget": 0.73, "metric": 0.289887284031958, "metric_std": 0.0033878248668131957, "actual_fraction": 0.7372836218375498, "actual_fraction_std": 0.009971137476852876}, {"budget": 0.74, "metric": 0.2914457139433469, "metric_std": 0.0034097163613990237, "actual_fraction": 0.7460719041278296, "actual_fraction_std": 0.008179196832019252}, {"budget": 0.75, "metric": 0.29113442803741185, "metric_std": 0.0029040055885706955, "actual_fraction": 0.7552596537949402, "actual_fraction_std": 0.009890235547758376}, {"budget": 0.76, "metric": 0.29031315276962183, "metric_std": 0.002751633299065853, "actual_fraction": 0.7674434087882822, "actual_fraction_std": 0.006068274839852459}, {"budget": 0.77, "metric": 0.289990968297447, "metric_std": 0.0018300370656399846, "actual_fraction": 0.7755659121171771, "actual_fraction_std": 0.006012319481965097}, {"budget": 0.78, "metric": 0.2901920651624007, "metric_std": 0.0013892525983880458, "actual_fraction": 0.7833555259653795, "actual_fraction_std": 0.005374902458363435}, {"budget": 0.79, "metric": 0.2901181158873772, "metric_std": 0.0017101153608205279, "actual_fraction": 0.7904793608521972, "actual_fraction_std": 0.0034642904768968992}, {"budget": 0.8, "metric": 0.28973901353907683, "metric_std": 0.002081307988079742, "actual_fraction": 0.7987350199733687, "actual_fraction_std": 0.00468180385932069}, {"budget": 0.81, "metric": 0.28888237446044834, "metric_std": 0.004500484566500762, "actual_fraction": 0.8127829560585885, "actual_fraction_std": 0.008416904970544353}, {"budget": 0.82, "metric": 0.2890360783246395, "metric_std": 0.002720049483376199, "actual_fraction": 0.8225699067909453, "actual_fraction_std": 0.007053801631297311}, {"budget": 0.83, "metric": 0.28767900194586976, "metric_std": 0.0026591687076655247, "actual_fraction": 0.8346205059920105, "actual_fraction_std": 0.005582230044275503}, {"budget": 0.84, "metric": 0.28869706841582, "metric_std": 0.005192440401405421, "actual_fraction": 0.8463382157123837, "actual_fraction_std": 0.0071497655172646935}, {"budget": 0.85, "metric": 0.2875418385011496, "metric_std": 0.0048135308185107495, "actual_fraction": 0.8567243675099867, "actual_fraction_std": 0.007242163477037522}, {"budget": 0.86, "metric": 0.28776751513017684, "metric_std": 0.0037248140315773236, "actual_fraction": 0.86491344873502, "actual_fraction_std": 0.00588659303565445}, {"budget": 0.87, "metric": 0.2880879394318149, "metric_std": 0.0008339683301719694, "actual_fraction": 0.8767643142476697, "actual_fraction_std": 0.0064351936532719605}, {"budget": 0.88, "metric": 0.287273877029594, "metric_std": 0.0021429159302818607, "actual_fraction": 0.8856857523302264, "actual_fraction_std": 0.008270813160950418}, {"budget": 0.89, "metric": 0.28547632945289336, "metric_std": 0.0017580416297739154, "actual_fraction": 0.8951398135818909, "actual_fraction_std": 0.0072818385000756785}, {"budget": 0.9, "metric": 0.284269379519687, "metric_std": 0.0017668399073315878, "actual_fraction": 0.9029960053262316, "actual_fraction_std": 0.008176486693829456}, {"budget": 0.91, "metric": 0.28250852235623786, "metric_std": 0.0016521490606986185, "actual_fraction": 0.9131824234354193, "actual_fraction_std": 0.008843213910405587}, {"budget": 0.92, "metric": 0.2814829677111451, "metric_std": 0.0017145752569778805, "actual_fraction": 0.9243675099866845, "actual_fraction_std": 0.008426116000392664}, {"budget": 0.93, "metric": 0.2800886018148173, "metric_std": 0.001320354758268392, "actual_fraction": 0.9328229027962717, "actual_fraction_std": 0.008226479724730553}, {"budget": 0.94, "metric": 0.27854520316604037, "metric_std": 0.0009764546187026387, "actual_fraction": 0.9400798934753661, "actual_fraction_std": 0.009368340399114663}, {"budget": 0.95, "metric": 0.27746458817138653, "metric_std": 0.0014319531496362432, "actual_fraction": 0.9482689747003995, "actual_fraction_std": 0.007773521794254069}, {"budget": 0.96, "metric": 0.2760878894837333, "metric_std": 0.0021111589514397045, "actual_fraction": 0.9603861517976032, "actual_fraction_std": 0.004250048380427328}, {"budget": 0.97, "metric": 0.27468746115763226, "metric_std": 0.0022038971274238282, "actual_fraction": 0.9677762982689748, "actual_fraction_std": 0.001975019570464853}, {"budget": 0.98, "metric": 0.2732948963509506, "metric_std": 0.0013636196027434443, "actual_fraction": 0.9782290279627164, "actual_fraction_std": 0.002019407574980477}, {"budget": 0.99, "metric": 0.27348968306795485, "metric_std": 0.001043194051403076, "actual_fraction": 0.9849533954727031, "actual_fraction_std": 0.0022503919815120605}, {"budget": 1.0, "metric": 0.27001009067886245, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 2189, "rescuable": 307, "harmful": 220, "both_wrong": 88, "unit": "event", "total": 2804}, "test": {"both_correct": 9508, "rescuable": 1282, "harmful": 777, "both_wrong": 449, "unit": "event", "total": 12016}}, "small": {"validation": 0.18254072168965788, "test": 0.18129624718318377}, "large": {"validation": 0.25448626222520154, "test": 0.27001009067886245}, "learned_minus_confidence_test": 0.038218832536494274, "learned_minus_small_test": 0.11667158582952103, "learned_minus_large_test": 0.02795774233384235, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_yolo26x_Qwen3.8"}, {"key": ["construction", "rtdetr_x_fp32", "MiniCPM"], "expert_display": "RT-DETR-X", "run": "outputs/full_budget_20260913/runs/full_rtdetr_x_fp32_MiniCPM_s{2026..2030}", "validation_rows": 701, "test_rows": 3004, "confidence_rule": {"column": "max_score", "columns": ["detection_count", "has_detections", "max_score", "mean_score", "detection_count_valid", "has_detections_valid", "max_score_valid", "mean_score_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/rtdetr_x_fp32/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 100, "validation_area_0_25": 0.08994758708890332, "validation_point": {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.20666016863629594, "rescued": 786, "harmed": 475, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 1391, "selected_both_wrong": 152}, "test_point": {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.22361115727328065, "rescued": 3314, "harmed": 1945, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 6060, "selected_both_wrong": 697}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.07641657905799261, "rescued": 0, "harmed": 0, "missed_rescue": 786, "avoided_harm": 475, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.021043073385953903, "operator": "ge", "actual_fraction": 0.009985734664764621, "metric": 0.07628160544643427, "rescued": 0, "harmed": 5, "missed_rescue": 786, "avoided_harm": 470, "selected_both_correct": 23, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -0.022327054291963577, "operator": "ge", "actual_fraction": 0.019971469329529243, "metric": 0.07620479856453766, "rescued": 0, "harmed": 10, "missed_rescue": 786, "avoided_harm": 465, "selected_both_correct": 46, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -0.024909960106015205, "operator": "ge", "actual_fraction": 0.029957203994293864, "metric": 0.076179559423678, "rescued": 0, "harmed": 16, "missed_rescue": 786, "avoided_harm": 459, "selected_both_correct": 68, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": -0.027814723551273346, "operator": "ge", "actual_fraction": 0.039942938659058486, "metric": 0.07608036783512472, "rescued": 0, "harmed": 21, "missed_rescue": 786, "avoided_harm": 454, "selected_both_correct": 91, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": -0.028914429247379303, "operator": "ge", "actual_fraction": 0.04992867332382311, "metric": 0.09455032380508069, "rescued": 1, "harmed": 25, "missed_rescue": 785, "avoided_harm": 450, "selected_both_correct": 114, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": -0.030170045793056488, "operator": "ge", "actual_fraction": 0.05991440798858773, "metric": 0.09386491892417406, "rescued": 1, "harmed": 27, "missed_rescue": 785, "avoided_harm": 448, "selected_both_correct": 139, "selected_both_wrong": 1}, {"budget": 0.07, "threshold": -0.03123854100704193, "operator": "ge", "actual_fraction": 0.06990014265335236, "metric": 0.09204233166673514, "rescued": 1, "harmed": 34, "missed_rescue": 785, "avoided_harm": 441, "selected_both_correct": 160, "selected_both_wrong": 1}, {"budget": 0.08, "threshold": -0.03410686179995537, "operator": "ge", "actual_fraction": 0.07988587731811697, "metric": 0.091007586947048, "rescued": 1, "harmed": 38, "missed_rescue": 785, "avoided_harm": 437, "selected_both_correct": 184, "selected_both_wrong": 1}, {"budget": 0.09, "threshold": -0.03476816788315773, "operator": "ge", "actual_fraction": 0.0898716119828816, "metric": 0.08934652523756015, "rescued": 1, "harmed": 43, "missed_rescue": 785, "avoided_harm": 432, "selected_both_correct": 207, "selected_both_wrong": 1}, {"budget": 0.1, "threshold": -0.0365506075322628, "operator": "ge", "actual_fraction": 0.09985734664764621, "metric": 0.08860787026161702, "rescued": 1, "harmed": 47, "missed_rescue": 785, "avoided_harm": 428, "selected_both_correct": 231, "selected_both_wrong": 1}, {"budget": 0.11, "threshold": -0.03818900138139725, "operator": "ge", "actual_fraction": 0.10984308131241084, "metric": 0.0875161669606114, "rescued": 1, "harmed": 54, "missed_rescue": 785, "avoided_harm": 421, "selected_both_correct": 252, "selected_both_wrong": 1}, {"budget": 0.12, "threshold": -0.039673544466495514, "operator": "ge", "actual_fraction": 0.11982881597717546, "metric": 0.0869090348188258, "rescued": 1, "harmed": 63, "missed_rescue": 785, "avoided_harm": 412, "selected_both_correct": 271, "selected_both_wrong": 1}, {"budget": 0.13, "threshold": -0.04085410386323929, "operator": "ge", "actual_fraction": 0.12981455064194009, "metric": 0.08635025131899243, "rescued": 1, "harmed": 68, "missed_rescue": 785, "avoided_harm": 407, "selected_both_correct": 294, "selected_both_wrong": 1}, {"budget": 0.14, "threshold": -0.04204101860523224, "operator": "ge", "actual_fraction": 0.1398002853067047, "metric": 0.0857792865956536, "rescued": 1, "harmed": 75, "missed_rescue": 785, "avoided_harm": 400, "selected_both_correct": 315, "selected_both_wrong": 1}, {"budget": 0.15, "threshold": -0.04278091713786125, "operator": "ge", "actual_fraction": 0.14978601997146934, "metric": 0.09418918796431942, "rescued": 2, "harmed": 81, "missed_rescue": 784, "avoided_harm": 394, "selected_both_correct": 335, "selected_both_wrong": 2}, {"budget": 0.16, "threshold": -0.04322759434580803, "operator": "ge", "actual_fraction": 0.15977175463623394, "metric": 0.09315792624073146, "rescued": 2, "harmed": 85, "missed_rescue": 784, "avoided_harm": 390, "selected_both_correct": 359, "selected_both_wrong": 2}, {"budget": 0.17, "threshold": -0.04394488036632538, "operator": "ge", "actual_fraction": 0.16975748930099857, "metric": 0.09222958631972636, "rescued": 2, "harmed": 89, "missed_rescue": 784, "avoided_harm": 386, "selected_both_correct": 383, "selected_both_wrong": 2}, {"budget": 0.18, "threshold": -0.04530126228928566, "operator": "ge", "actual_fraction": 0.1797432239657632, "metric": 0.0917587567603033, "rescued": 2, "harmed": 95, "missed_rescue": 784, "avoided_harm": 380, "selected_both_correct": 405, "selected_both_wrong": 2}, {"budget": 0.19, "threshold": -0.04661082476377487, "operator": "ge", "actual_fraction": 0.18972895863052783, "metric": 0.09090632738556746, "rescued": 2, "harmed": 101, "missed_rescue": 784, "avoided_harm": 374, "selected_both_correct": 427, "selected_both_wrong": 2}, {"budget": 0.2, "threshold": -0.04730507358908653, "operator": "ge", "actual_fraction": 0.19971469329529243, "metric": 0.09027150780105647, "rescued": 2, "harmed": 106, "missed_rescue": 784, "avoided_harm": 369, "selected_both_correct": 450, "selected_both_wrong": 2}, {"budget": 0.21, "threshold": -0.048246145248413086, "operator": "ge", "actual_fraction": 0.20970042796005706, "metric": 0.08904603886890428, "rescued": 2, "harmed": 113, "missed_rescue": 784, "avoided_harm": 362, "selected_both_correct": 471, "selected_both_wrong": 2}, {"budget": 0.22, "threshold": -0.04900278151035309, "operator": "ge", "actual_fraction": 0.21968616262482168, "metric": 0.08819855956614042, "rescued": 2, "harmed": 121, "missed_rescue": 784, "avoided_harm": 354, "selected_both_correct": 491, "selected_both_wrong": 2}, {"budget": 0.23, "threshold": -0.050042688846588135, "operator": "ge", "actual_fraction": 0.2296718972895863, "metric": 0.08769891005218598, "rescued": 2, "harmed": 126, "missed_rescue": 784, "avoided_harm": 349, "selected_both_correct": 513, "selected_both_wrong": 3}, {"budget": 0.24, "threshold": -0.050901688635349274, "operator": "ge", "actual_fraction": 0.2396576319543509, "metric": 0.08812163994607147, "rescued": 8, "harmed": 130, "missed_rescue": 778, "avoided_harm": 345, "selected_both_correct": 530, "selected_both_wrong": 4}, {"budget": 0.25, "threshold": -0.05161922052502632, "operator": "ge", "actual_fraction": 0.24964336661911554, "metric": 0.08782151216689346, "rescued": 14, "harmed": 135, "missed_rescue": 772, "avoided_harm": 340, "selected_both_correct": 546, "selected_both_wrong": 5}, {"budget": 0.26, "threshold": -0.05245445668697357, "operator": "ge", "actual_fraction": 0.25962910128388017, "metric": 0.08802770491567173, "rescued": 21, "harmed": 139, "missed_rescue": 765, "avoided_harm": 336, "selected_both_correct": 563, "selected_both_wrong": 5}, {"budget": 0.27, "threshold": -0.053079016506671906, "operator": "ge", "actual_fraction": 0.26961483594864477, "metric": 0.08800890547752473, "rescued": 28, "harmed": 144, "missed_rescue": 758, "avoided_harm": 331, "selected_both_correct": 578, "selected_both_wrong": 6}, {"budget": 0.28, "threshold": -0.053604189306497574, "operator": "ge", "actual_fraction": 0.2796005706134094, "metric": 0.08849150328196631, "rescued": 36, "harmed": 148, "missed_rescue": 750, "avoided_harm": 327, "selected_both_correct": 591, "selected_both_wrong": 9}, {"budget": 0.29, "threshold": -0.05505518987774849, "operator": "ge", "actual_fraction": 0.289586305278174, "metric": 0.0887205564920429, "rescued": 46, "harmed": 153, "missed_rescue": 740, "avoided_harm": 322, "selected_both_correct": 604, "selected_both_wrong": 9}, {"budget": 0.3, "threshold": -0.05605518817901611, "operator": "ge", "actual_fraction": 0.2995720399429387, "metric": 0.08901323629975644, "rescued": 53, "harmed": 157, "missed_rescue": 733, "avoided_harm": 318, "selected_both_correct": 621, "selected_both_wrong": 9}, {"budget": 0.31, "threshold": -0.056986160576343536, "operator": "ge", "actual_fraction": 0.3095577746077033, "metric": 0.08941457984183532, "rescued": 61, "harmed": 162, "missed_rescue": 725, "avoided_harm": 313, "selected_both_correct": 635, "selected_both_wrong": 10}, {"budget": 0.32, "threshold": -0.0582931749522686, "operator": "ge", "actual_fraction": 0.3195435092724679, "metric": 0.08986547999267158, "rescued": 69, "harmed": 167, "missed_rescue": 717, "avoided_harm": 308, "selected_both_correct": 650, "selected_both_wrong": 10}, {"budget": 0.33, "threshold": -0.0596650205552578, "operator": "ge", "actual_fraction": 0.32952924393723254, "metric": 0.0900623716189768, "rescued": 76, "harmed": 173, "missed_rescue": 710, "avoided_harm": 302, "selected_both_correct": 664, "selected_both_wrong": 11}, {"budget": 0.34, "threshold": -0.060549296438694, "operator": "ge", "actual_fraction": 0.33951497860199714, "metric": 0.09018872466182963, "rescued": 87, "harmed": 180, "missed_rescue": 699, "avoided_harm": 295, "selected_both_correct": 673, "selected_both_wrong": 12}, {"budget": 0.35, "threshold": -0.06178095191717148, "operator": "ge", "actual_fraction": 0.34950071326676174, "metric": 0.09513465140887797, "rescued": 96, "harmed": 184, "missed_rescue": 690, "avoided_harm": 291, "selected_both_correct": 686, "selected_both_wrong": 14}, {"budget": 0.36, "threshold": -0.06328777223825455, "operator": "ge", "actual_fraction": 0.3594864479315264, "metric": 0.09560176179757868, "rescued": 108, "harmed": 190, "missed_rescue": 678, "avoided_harm": 285, "selected_both_correct": 696, "selected_both_wrong": 14}, {"budget": 0.37, "threshold": -0.0647548958659172, "operator": "ge", "actual_fraction": 0.369472182596291, "metric": 0.09606601222778545, "rescued": 118, "harmed": 196, "missed_rescue": 668, "avoided_harm": 279, "selected_both_correct": 707, "selected_both_wrong": 15}, {"budget": 0.38, "threshold": -0.06637481600046158, "operator": "ge", "actual_fraction": 0.37945791726105566, "metric": 0.09683684024140088, "rescued": 126, "harmed": 199, "missed_rescue": 660, "avoided_harm": 276, "selected_both_correct": 722, "selected_both_wrong": 17}, {"budget": 0.39, "threshold": -0.06774259358644485, "operator": "ge", "actual_fraction": 0.38944365192582026, "metric": 0.09748138328334532, "rescued": 135, "harmed": 201, "missed_rescue": 651, "avoided_harm": 274, "selected_both_correct": 736, "selected_both_wrong": 20}, {"budget": 0.4, "threshold": -0.06843380630016327, "operator": "ge", "actual_fraction": 0.39942938659058486, "metric": 0.09725002075008596, "rescued": 145, "harmed": 205, "missed_rescue": 641, "avoided_harm": 270, "selected_both_correct": 750, "selected_both_wrong": 20}, {"budget": 0.41, "threshold": -0.0694691464304924, "operator": "ge", "actual_fraction": 0.4094151212553495, "metric": 0.09794654894022267, "rescued": 155, "harmed": 210, "missed_rescue": 631, "avoided_harm": 265, "selected_both_correct": 763, "selected_both_wrong": 20}, {"budget": 0.42, "threshold": -0.07112707942724228, "operator": "ge", "actual_fraction": 0.4194008559201141, "metric": 0.09874354292060185, "rescued": 165, "harmed": 216, "missed_rescue": 621, "avoided_harm": 259, "selected_both_correct": 773, "selected_both_wrong": 22}, {"budget": 0.43, "threshold": -0.07192998379468918, "operator": "ge", "actual_fraction": 0.42938659058487877, "metric": 0.09944895007868021, "rescued": 174, "harmed": 222, "missed_rescue": 612, "avoided_harm": 253, "selected_both_correct": 785, "selected_both_wrong": 23}, {"budget": 0.44, "threshold": -0.07266208529472351, "operator": "ge", "actual_fraction": 0.43937232524964337, "metric": 0.10024323502584373, "rescued": 184, "harmed": 226, "missed_rescue": 602, "avoided_harm": 249, "selected_both_correct": 798, "selected_both_wrong": 24}, {"budget": 0.45, "threshold": -0.0744253620505333, "operator": "ge", "actual_fraction": 0.44935805991440797, "metric": 0.10107630515550795, "rescued": 194, "harmed": 230, "missed_rescue": 592, "avoided_harm": 245, "selected_both_correct": 810, "selected_both_wrong": 26}, {"budget": 0.46, "threshold": -0.07532940804958344, "operator": "ge", "actual_fraction": 0.4593437945791726, "metric": 0.10181572307751467, "rescued": 203, "harmed": 235, "missed_rescue": 583, "avoided_harm": 240, "selected_both_correct": 823, "selected_both_wrong": 27}, {"budget": 0.47, "threshold": -0.0770750343799591, "operator": "ge", "actual_fraction": 0.4693295292439372, "metric": 0.10291838837461804, "rescued": 218, "harmed": 240, "missed_rescue": 568, "avoided_harm": 235, "selected_both_correct": 830, "selected_both_wrong": 28}, {"budget": 0.48, "threshold": -0.07818037271499634, "operator": "ge", "actual_fraction": 0.4793152639087018, "metric": 0.10398791390534362, "rescued": 228, "harmed": 245, "missed_rescue": 558, "avoided_harm": 230, "selected_both_correct": 843, "selected_both_wrong": 28}, {"budget": 0.49, "threshold": -0.07998209446668625, "operator": "ge", "actual_fraction": 0.4893009985734665, "metric": 0.1035500155095013, "rescued": 235, "harmed": 252, "missed_rescue": 551, "avoided_harm": 223, "selected_both_correct": 857, "selected_both_wrong": 28}, {"budget": 0.5, "threshold": -0.08124949783086777, "operator": "ge", "actual_fraction": 0.4992867332382311, "metric": 0.10310317656730472, "rescued": 242, "harmed": 256, "missed_rescue": 544, "avoided_harm": 219, "selected_both_correct": 871, "selected_both_wrong": 31}, {"budget": 0.51, "threshold": -0.08398465067148209, "operator": "ge", "actual_fraction": 0.5092724679029957, "metric": 0.10437739691349379, "rescued": 254, "harmed": 261, "missed_rescue": 532, "avoided_harm": 214, "selected_both_correct": 881, "selected_both_wrong": 32}, {"budget": 0.52, "threshold": -0.08537434041500092, "operator": "ge", "actual_fraction": 0.5192582025677603, "metric": 0.10682502852650196, "rescued": 262, "harmed": 268, "missed_rescue": 524, "avoided_harm": 207, "selected_both_correct": 890, "selected_both_wrong": 36}, {"budget": 0.53, "threshold": -0.08664409816265106, "operator": "ge", "actual_fraction": 0.5292439372325249, "metric": 0.10806066671218575, "rescued": 273, "harmed": 275, "missed_rescue": 513, "avoided_harm": 200, "selected_both_correct": 899, "selected_both_wrong": 37}, {"budget": 0.54, "threshold": -0.08777308464050293, "operator": "ge", "actual_fraction": 0.5392296718972895, "metric": 0.10917273744598222, "rescued": 284, "harmed": 278, "missed_rescue": 502, "avoided_harm": 197, "selected_both_correct": 911, "selected_both_wrong": 39}, {"budget": 0.55, "threshold": -0.08980843424797058, "operator": "ge", "actual_fraction": 0.5492154065620543, "metric": 0.11036769675516778, "rescued": 296, "harmed": 284, "missed_rescue": 490, "avoided_harm": 191, "selected_both_correct": 920, "selected_both_wrong": 40}, {"budget": 0.56, "threshold": -0.0906188115477562, "operator": "ge", "actual_fraction": 0.5592011412268189, "metric": 0.11423875293101385, "rescued": 307, "harmed": 287, "missed_rescue": 479, "avoided_harm": 188, "selected_both_correct": 931, "selected_both_wrong": 43}, {"budget": 0.57, "threshold": -0.09160211682319641, "operator": "ge", "actual_fraction": 0.5691868758915835, "metric": 0.11535691235491538, "rescued": 317, "harmed": 294, "missed_rescue": 469, "avoided_harm": 181, "selected_both_correct": 940, "selected_both_wrong": 45}, {"budget": 0.58, "threshold": -0.09283093363046646, "operator": "ge", "actual_fraction": 0.579172610556348, "metric": 0.11669725808975426, "rescued": 328, "harmed": 298, "missed_rescue": 458, "avoided_harm": 177, "selected_both_correct": 952, "selected_both_wrong": 46}, {"budget": 0.59, "threshold": -0.0941123366355896, "operator": "ge", "actual_fraction": 0.5891583452211127, "metric": 0.1177679103161983, "rescued": 338, "harmed": 303, "missed_rescue": 448, "avoided_harm": 172, "selected_both_correct": 964, "selected_both_wrong": 47}, {"budget": 0.6, "threshold": -0.09614869952201843, "operator": "ge", "actual_fraction": 0.5991440798858774, "metric": 0.11921675545489711, "rescued": 351, "harmed": 309, "missed_rescue": 435, "avoided_harm": 166, "selected_both_correct": 972, "selected_both_wrong": 48}, {"budget": 0.61, "threshold": -0.09772664308547974, "operator": "ge", "actual_fraction": 0.609129814550642, "metric": 0.12100079401350544, "rescued": 364, "harmed": 313, "missed_rescue": 422, "avoided_harm": 162, "selected_both_correct": 981, "selected_both_wrong": 50}, {"budget": 0.62, "threshold": -0.09911532700061798, "operator": "ge", "actual_fraction": 0.6191155492154066, "metric": 0.12498749148623647, "rescued": 375, "harmed": 315, "missed_rescue": 411, "avoided_harm": 160, "selected_both_correct": 993, "selected_both_wrong": 53}, {"budget": 0.63, "threshold": -0.10007988661527634, "operator": "ge", "actual_fraction": 0.6291012838801712, "metric": 0.12628905170512741, "rescued": 387, "harmed": 319, "missed_rescue": 399, "avoided_harm": 156, "selected_both_correct": 1004, "selected_both_wrong": 54}, {"budget": 0.64, "threshold": -0.10177135467529297, "operator": "ge", "actual_fraction": 0.6390870185449358, "metric": 0.12942700411115388, "rescued": 403, "harmed": 324, "missed_rescue": 383, "avoided_harm": 151, "selected_both_correct": 1010, "selected_both_wrong": 55}, {"budget": 0.65, "threshold": -0.10395076125860214, "operator": "ge", "actual_fraction": 0.6490727532097005, "metric": 0.13068664709512867, "rescued": 410, "harmed": 328, "missed_rescue": 376, "avoided_harm": 147, "selected_both_correct": 1024, "selected_both_wrong": 58}, {"budget": 0.66, "threshold": -0.10524322092533112, "operator": "ge", "actual_fraction": 0.6590584878744651, "metric": 0.1324582559366836, "rescued": 422, "harmed": 332, "missed_rescue": 364, "avoided_harm": 143, "selected_both_correct": 1033, "selected_both_wrong": 61}, {"budget": 0.67, "threshold": -0.10689426958560944, "operator": "ge", "actual_fraction": 0.6690442225392297, "metric": 0.13402207594539173, "rescued": 435, "harmed": 338, "missed_rescue": 351, "avoided_harm": 137, "selected_both_correct": 1041, "selected_both_wrong": 62}, {"budget": 0.68, "threshold": -0.11014843732118607, "operator": "ge", "actual_fraction": 0.6790299572039943, "metric": 0.13570216904654514, "rescued": 447, "harmed": 341, "missed_rescue": 339, "avoided_harm": 134, "selected_both_correct": 1052, "selected_both_wrong": 64}, {"budget": 0.69, "threshold": -0.11117545515298843, "operator": "ge", "actual_fraction": 0.6890156918687589, "metric": 0.13971639727625254, "rescued": 458, "harmed": 343, "missed_rescue": 328, "avoided_harm": 132, "selected_both_correct": 1062, "selected_both_wrong": 69}, {"budget": 0.7, "threshold": -0.11230025440454483, "operator": "ge", "actual_fraction": 0.6990014265335235, "metric": 0.14039490967751408, "rescued": 469, "harmed": 346, "missed_rescue": 317, "avoided_harm": 129, "selected_both_correct": 1074, "selected_both_wrong": 71}, {"budget": 0.71, "threshold": -0.11423606425523758, "operator": "ge", "actual_fraction": 0.7089871611982882, "metric": 0.1389440908530641, "rescued": 476, "harmed": 351, "missed_rescue": 310, "avoided_harm": 124, "selected_both_correct": 1084, "selected_both_wrong": 77}, {"budget": 0.72, "threshold": -0.11647249013185501, "operator": "ge", "actual_fraction": 0.7189728958630528, "metric": 0.1412819215041655, "rescued": 492, "harmed": 358, "missed_rescue": 294, "avoided_harm": 117, "selected_both_correct": 1089, "selected_both_wrong": 77}, {"budget": 0.73, "threshold": -0.11962417513132095, "operator": "ge", "actual_fraction": 0.7289586305278174, "metric": 0.14288929438332787, "rescued": 503, "harmed": 361, "missed_rescue": 283, "avoided_harm": 114, "selected_both_correct": 1099, "selected_both_wrong": 81}, {"budget": 0.74, "threshold": -0.12126146256923676, "operator": "ge", "actual_fraction": 0.738944365192582, "metric": 0.14459529608777652, "rescued": 514, "harmed": 363, "missed_rescue": 272, "avoided_harm": 112, "selected_both_correct": 1111, "selected_both_wrong": 84}, {"budget": 0.75, "threshold": -0.1230982318520546, "operator": "ge", "actual_fraction": 0.7489300998573466, "metric": 0.14606069842667782, "rescued": 525, "harmed": 367, "missed_rescue": 261, "avoided_harm": 108, "selected_both_correct": 1121, "selected_both_wrong": 87}, {"budget": 0.76, "threshold": -0.12517565488815308, "operator": "ge", "actual_fraction": 0.7589158345221113, "metric": 0.14665677647800887, "rescued": 537, "harmed": 373, "missed_rescue": 249, "avoided_harm": 102, "selected_both_correct": 1130, "selected_both_wrong": 88}, {"budget": 0.77, "threshold": -0.12628257274627686, "operator": "ge", "actual_fraction": 0.7689015691868759, "metric": 0.1471300269894615, "rescued": 546, "harmed": 381, "missed_rescue": 240, "avoided_harm": 94, "selected_both_correct": 1139, "selected_both_wrong": 90}, {"budget": 0.78, "threshold": -0.12975578010082245, "operator": "ge", "actual_fraction": 0.7788873038516405, "metric": 0.1503016824168161, "rescued": 558, "harmed": 384, "missed_rescue": 228, "avoided_harm": 91, "selected_both_correct": 1151, "selected_both_wrong": 91}, {"budget": 0.79, "threshold": -0.13229140639305115, "operator": "ge", "actual_fraction": 0.7888730385164051, "metric": 0.15155183161566646, "rescued": 566, "harmed": 386, "missed_rescue": 220, "avoided_harm": 89, "selected_both_correct": 1163, "selected_both_wrong": 97}, {"budget": 0.8, "threshold": -0.1373685747385025, "operator": "ge", "actual_fraction": 0.7988587731811697, "metric": 0.15468771641968415, "rescued": 579, "harmed": 392, "missed_rescue": 207, "avoided_harm": 83, "selected_both_correct": 1170, "selected_both_wrong": 99}, {"budget": 0.81, "threshold": -0.1407174915075302, "operator": "ge", "actual_fraction": 0.8088445078459344, "metric": 0.16068290931933413, "rescued": 597, "harmed": 396, "missed_rescue": 189, "avoided_harm": 79, "selected_both_correct": 1175, "selected_both_wrong": 100}, {"budget": 0.82, "threshold": -0.1433146893978119, "operator": "ge", "actual_fraction": 0.818830242510699, "metric": 0.16451572912479911, "rescued": 612, "harmed": 400, "missed_rescue": 174, "avoided_harm": 75, "selected_both_correct": 1183, "selected_both_wrong": 101}, {"budget": 0.83, "threshold": -0.1457100659608841, "operator": "ge", "actual_fraction": 0.8288159771754636, "metric": 0.16497266103579622, "rescued": 619, "harmed": 406, "missed_rescue": 167, "avoided_harm": 69, "selected_both_correct": 1196, "selected_both_wrong": 103}, {"budget": 0.84, "threshold": -0.148710235953331, "operator": "ge", "actual_fraction": 0.8388017118402282, "metric": 0.1662001358662362, "rescued": 628, "harmed": 409, "missed_rescue": 158, "avoided_harm": 66, "selected_both_correct": 1209, "selected_both_wrong": 106}, {"budget": 0.85, "threshold": -0.15291842818260193, "operator": "ge", "actual_fraction": 0.8487874465049928, "metric": 0.1711356788768453, "rescued": 639, "harmed": 411, "missed_rescue": 147, "avoided_harm": 64, "selected_both_correct": 1218, "selected_both_wrong": 112}, {"budget": 0.86, "threshold": -0.15889839828014374, "operator": "ge", "actual_fraction": 0.8587731811697575, "metric": 0.17267738998377383, "rescued": 647, "harmed": 416, "missed_rescue": 139, "avoided_harm": 59, "selected_both_correct": 1230, "selected_both_wrong": 115}, {"budget": 0.87, "threshold": -0.16629797220230103, "operator": "ge", "actual_fraction": 0.8687589158345221, "metric": 0.17579084329823536, "rescued": 656, "harmed": 422, "missed_rescue": 130, "avoided_harm": 53, "selected_both_correct": 1241, "selected_both_wrong": 117}, {"budget": 0.88, "threshold": -0.17295491695404053, "operator": "ge", "actual_fraction": 0.8787446504992867, "metric": 0.17752359710692284, "rescued": 666, "harmed": 427, "missed_rescue": 120, "avoided_harm": 48, "selected_both_correct": 1253, "selected_both_wrong": 118}, {"budget": 0.89, "threshold": -0.18053314089775085, "operator": "ge", "actual_fraction": 0.8887303851640513, "metric": 0.18131944434726105, "rescued": 676, "harmed": 431, "missed_rescue": 110, "avoided_harm": 44, "selected_both_correct": 1266, "selected_both_wrong": 119}, {"budget": 0.9, "threshold": -0.19442276656627655, "operator": "ge", "actual_fraction": 0.8987161198288159, "metric": 0.18427430914771328, "rescued": 688, "harmed": 433, "missed_rescue": 98, "avoided_harm": 42, "selected_both_correct": 1277, "selected_both_wrong": 122}, {"budget": 0.91, "threshold": -0.20900504291057587, "operator": "ge", "actual_fraction": 0.9087018544935807, "metric": 0.18611646191425543, "rescued": 697, "harmed": 437, "missed_rescue": 89, "avoided_harm": 38, "selected_both_correct": 1289, "selected_both_wrong": 125}, {"budget": 0.92, "threshold": -0.22383525967597961, "operator": "ge", "actual_fraction": 0.9186875891583453, "metric": 0.1910397694023846, "rescued": 707, "harmed": 440, "missed_rescue": 79, "avoided_harm": 35, "selected_both_correct": 1299, "selected_both_wrong": 130}, {"budget": 0.93, "threshold": -0.25952714681625366, "operator": "ge", "actual_fraction": 0.9286733238231099, "metric": 0.19063222783395556, "rescued": 717, "harmed": 447, "missed_rescue": 69, "avoided_harm": 28, "selected_both_correct": 1308, "selected_both_wrong": 132}, {"budget": 0.94, "threshold": -0.2804492115974426, "operator": "ge", "actual_fraction": 0.9386590584878745, "metric": 0.19317109979249197, "rescued": 726, "harmed": 451, "missed_rescue": 60, "avoided_harm": 24, "selected_both_correct": 1318, "selected_both_wrong": 137}, {"budget": 0.95, "threshold": -0.2984963357448578, "operator": "ge", "actual_fraction": 0.948644793152639, "metric": 0.19747037448756763, "rescued": 738, "harmed": 453, "missed_rescue": 48, "avoided_harm": 22, "selected_both_correct": 1329, "selected_both_wrong": 140}, {"budget": 0.96, "threshold": -0.32223379611968994, "operator": "ge", "actual_fraction": 0.9586305278174037, "metric": 0.2013335009829101, "rescued": 748, "harmed": 455, "missed_rescue": 38, "avoided_harm": 20, "selected_both_correct": 1343, "selected_both_wrong": 142}, {"budget": 0.97, "threshold": -0.36503666639328003, "operator": "ge", "actual_fraction": 0.9686162624821684, "metric": 0.2032615598092342, "rescued": 758, "harmed": 460, "missed_rescue": 28, "avoided_harm": 15, "selected_both_correct": 1355, "selected_both_wrong": 143}, {"budget": 0.98, "threshold": -0.4221048653125763, "operator": "ge", "actual_fraction": 0.978601997146933, "metric": 0.20572745422879232, "rescued": 770, "harmed": 465, "missed_rescue": 16, "avoided_harm": 10, "selected_both_correct": 1366, "selected_both_wrong": 143}, {"budget": 0.99, "threshold": -0.44162338972091675, "operator": "ge", "actual_fraction": 0.9885877318116976, "metric": 0.20340063714272894, "rescued": 778, "harmed": 472, "missed_rescue": 8, "avoided_harm": 3, "selected_both_correct": 1374, "selected_both_wrong": 148}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.20666016863629594, "rescued": 786, "harmed": 475, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 1391, "selected_both_wrong": 152}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.07887397871043929, "rescued": 0, "harmed": 0, "missed_rescue": 3314, "avoided_harm": 1945, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.021043073385953903, "operator": "ge", "actual_fraction": 0.007989347536617843, "metric": 0.0788343605891772, "rescued": 0, "harmed": 21, "missed_rescue": 3314, "avoided_harm": 1924, "selected_both_correct": 75, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -0.022327054291963577, "operator": "ge", "actual_fraction": 0.016644474034620507, "metric": 0.07894526738759718, "rescued": 1, "harmed": 39, "missed_rescue": 3313, "avoided_harm": 1906, "selected_both_correct": 160, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -0.024909960106015205, "operator": "ge", "actual_fraction": 0.0296271637816245, "metric": 0.07886280908239668, "rescued": 1, "harmed": 69, "missed_rescue": 3313, "avoided_harm": 1876, "selected_both_correct": 286, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": -0.027814723551273346, "operator": "ge", "actual_fraction": 0.046937416777629824, "metric": 0.07876261254060994, "rescued": 1, "harmed": 102, "missed_rescue": 3313, "avoided_harm": 1843, "selected_both_correct": 461, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": -0.028914429247379303, "operator": "ge", "actual_fraction": 0.05326231691078562, "metric": 0.07922967968226176, "rescued": 2, "harmed": 110, "missed_rescue": 3312, "avoided_harm": 1835, "selected_both_correct": 528, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": -0.030170045793056488, "operator": "ge", "actual_fraction": 0.05992010652463382, "metric": 0.07917808767949343, "rescued": 2, "harmed": 124, "missed_rescue": 3312, "avoided_harm": 1821, "selected_both_correct": 594, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": -0.03123854100704193, "operator": "ge", "actual_fraction": 0.06724367509986684, "metric": 0.08652168148030992, "rescued": 5, "harmed": 139, "missed_rescue": 3309, "avoided_harm": 1806, "selected_both_correct": 664, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": -0.03410686179995537, "operator": "ge", "actual_fraction": 0.08721704394141146, "metric": 0.08573997960186801, "rescued": 6, "harmed": 182, "missed_rescue": 3308, "avoided_harm": 1763, "selected_both_correct": 860, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": -0.03476816788315773, "operator": "ge", "actual_fraction": 0.09287616511318242, "metric": 0.08533855094545621, "rescued": 6, "harmed": 200, "missed_rescue": 3308, "avoided_harm": 1745, "selected_both_correct": 910, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": -0.0365506075322628, "operator": "ge", "actual_fraction": 0.10785619174434088, "metric": 0.08462726172642225, "rescued": 6, "harmed": 235, "missed_rescue": 3308, "avoided_harm": 1710, "selected_both_correct": 1054, "selected_both_wrong": 1}, {"budget": 0.11, "threshold": -0.03818900138139725, "operator": "ge", "actual_fraction": 0.11784287616511319, "metric": 0.08462444866867365, "rescued": 7, "harmed": 256, "missed_rescue": 3307, "avoided_harm": 1689, "selected_both_correct": 1151, "selected_both_wrong": 2}, {"budget": 0.12, "threshold": -0.039673544466495514, "operator": "ge", "actual_fraction": 0.12716378162450068, "metric": 0.08431158675236691, "rescued": 7, "harmed": 279, "missed_rescue": 3307, "avoided_harm": 1666, "selected_both_correct": 1240, "selected_both_wrong": 2}, {"budget": 0.13, "threshold": -0.04085410386323929, "operator": "ge", "actual_fraction": 0.13648468708388814, "metric": 0.0840276750632089, "rescued": 7, "harmed": 300, "missed_rescue": 3307, "avoided_harm": 1645, "selected_both_correct": 1331, "selected_both_wrong": 2}, {"budget": 0.14, "threshold": -0.04204101860523224, "operator": "ge", "actual_fraction": 0.14613848202396804, "metric": 0.08385470737270147, "rescued": 7, "harmed": 319, "missed_rescue": 3307, "avoided_harm": 1626, "selected_both_correct": 1428, "selected_both_wrong": 2}, {"budget": 0.15, "threshold": -0.04278091713786125, "operator": "ge", "actual_fraction": 0.1514647137150466, "metric": 0.08370670942943897, "rescued": 7, "harmed": 332, "missed_rescue": 3307, "avoided_harm": 1613, "selected_both_correct": 1479, "selected_both_wrong": 2}, {"budget": 0.16, "threshold": -0.04322759434580803, "operator": "ge", "actual_fraction": 0.15579227696404793, "metric": 0.08361544458700533, "rescued": 7, "harmed": 342, "missed_rescue": 3307, "avoided_harm": 1603, "selected_both_correct": 1521, "selected_both_wrong": 2}, {"budget": 0.17, "threshold": -0.04394488036632538, "operator": "ge", "actual_fraction": 0.1617842876165113, "metric": 0.08351220434393924, "rescued": 7, "harmed": 352, "missed_rescue": 3307, "avoided_harm": 1593, "selected_both_correct": 1583, "selected_both_wrong": 2}, {"budget": 0.18, "threshold": -0.04530126228928566, "operator": "ge", "actual_fraction": 0.17177097203728361, "metric": 0.08333120709231198, "rescued": 7, "harmed": 379, "missed_rescue": 3307, "avoided_harm": 1566, "selected_both_correct": 1676, "selected_both_wrong": 2}, {"budget": 0.19, "threshold": -0.04661082476377487, "operator": "ge", "actual_fraction": 0.1810918774966711, "metric": 0.08341398307559608, "rescued": 8, "harmed": 398, "missed_rescue": 3306, "avoided_harm": 1547, "selected_both_correct": 1768, "selected_both_wrong": 2}, {"budget": 0.2, "threshold": -0.04730507358908653, "operator": "ge", "actual_fraction": 0.1870838881491345, "metric": 0.08516887352267764, "rescued": 9, "harmed": 415, "missed_rescue": 3305, "avoided_harm": 1530, "selected_both_correct": 1821, "selected_both_wrong": 3}, {"budget": 0.21, "threshold": -0.048246145248413086, "operator": "ge", "actual_fraction": 0.19640479360852198, "metric": 0.08675280750372345, "rescued": 10, "harmed": 435, "missed_rescue": 3304, "avoided_harm": 1510, "selected_both_correct": 1911, "selected_both_wrong": 4}, {"budget": 0.22, "threshold": -0.04900278151035309, "operator": "ge", "actual_fraction": 0.20838881491344874, "metric": 0.08619173713061849, "rescued": 10, "harmed": 468, "missed_rescue": 3304, "avoided_harm": 1477, "selected_both_correct": 2019, "selected_both_wrong": 7}, {"budget": 0.23, "threshold": -0.050042688846588135, "operator": "ge", "actual_fraction": 0.21471371504660453, "metric": 0.08594774173844119, "rescued": 10, "harmed": 483, "missed_rescue": 3304, "avoided_harm": 1462, "selected_both_correct": 2080, "selected_both_wrong": 7}, {"budget": 0.24, "threshold": -0.050901688635349274, "operator": "ge", "actual_fraction": 0.22536617842876164, "metric": 0.08804054363419953, "rescued": 41, "harmed": 502, "missed_rescue": 3273, "avoided_harm": 1443, "selected_both_correct": 2154, "selected_both_wrong": 11}, {"budget": 0.25, "threshold": -0.05161922052502632, "operator": "ge", "actual_fraction": 0.23135818908122505, "metric": 0.08818772258585045, "rescued": 60, "harmed": 517, "missed_rescue": 3254, "avoided_harm": 1428, "selected_both_correct": 2192, "selected_both_wrong": 11}, {"budget": 0.26, "threshold": -0.05245445668697357, "operator": "ge", "actual_fraction": 0.237017310252996, "metric": 0.08841839409893854, "rescued": 76, "harmed": 525, "missed_rescue": 3238, "avoided_harm": 1420, "selected_both_correct": 2235, "selected_both_wrong": 12}, {"budget": 0.27, "threshold": -0.053079016506671906, "operator": "ge", "actual_fraction": 0.24167776298268975, "metric": 0.08850628284208313, "rescued": 90, "harmed": 536, "missed_rescue": 3224, "avoided_harm": 1409, "selected_both_correct": 2265, "selected_both_wrong": 13}, {"budget": 0.28, "threshold": -0.053604189306497574, "operator": "ge", "actual_fraction": 0.2470039946737683, "metric": 0.0886323434450378, "rescued": 106, "harmed": 548, "missed_rescue": 3208, "avoided_harm": 1397, "selected_both_correct": 2301, "selected_both_wrong": 13}, {"budget": 0.29, "threshold": -0.05505518987774849, "operator": "ge", "actual_fraction": 0.2623169107856192, "metric": 0.09067349863003854, "rescued": 158, "harmed": 582, "missed_rescue": 3156, "avoided_harm": 1363, "selected_both_correct": 2396, "selected_both_wrong": 16}, {"budget": 0.3, "threshold": -0.05605518817901611, "operator": "ge", "actual_fraction": 0.27463382157123833, "metric": 0.09250161598017743, "rescued": 201, "harmed": 607, "missed_rescue": 3113, "avoided_harm": 1338, "selected_both_correct": 2469, "selected_both_wrong": 23}, {"budget": 0.31, "threshold": -0.056986160576343536, "operator": "ge", "actual_fraction": 0.28462050599201066, "metric": 0.09322240603889413, "rescued": 233, "harmed": 625, "missed_rescue": 3081, "avoided_harm": 1320, "selected_both_correct": 2537, "selected_both_wrong": 25}, {"budget": 0.32, "threshold": -0.0582931749522686, "operator": "ge", "actual_fraction": 0.29427430093209056, "metric": 0.09484741449397432, "rescued": 264, "harmed": 642, "missed_rescue": 3050, "avoided_harm": 1303, "selected_both_correct": 2598, "selected_both_wrong": 32}, {"budget": 0.33, "threshold": -0.0596650205552578, "operator": "ge", "actual_fraction": 0.3062583222370173, "metric": 0.09509370405935859, "rescued": 304, "harmed": 671, "missed_rescue": 3010, "avoided_harm": 1274, "selected_both_correct": 2671, "selected_both_wrong": 34}, {"budget": 0.34, "threshold": -0.060549296438694, "operator": "ge", "actual_fraction": 0.3132490013315579, "metric": 0.09537687922458282, "rescued": 331, "harmed": 687, "missed_rescue": 2983, "avoided_harm": 1258, "selected_both_correct": 2709, "selected_both_wrong": 37}, {"budget": 0.35, "threshold": -0.06178095191717148, "operator": "ge", "actual_fraction": 0.3189081225033289, "metric": 0.09518164881276557, "rescued": 349, "harmed": 703, "missed_rescue": 2965, "avoided_harm": 1242, "selected_both_correct": 2741, "selected_both_wrong": 39}, {"budget": 0.36, "threshold": -0.06328777223825455, "operator": "ge", "actual_fraction": 0.3318908122503329, "metric": 0.09579948445467232, "rescued": 389, "harmed": 731, "missed_rescue": 2925, "avoided_harm": 1214, "selected_both_correct": 2819, "selected_both_wrong": 49}, {"budget": 0.37, "threshold": -0.0647548958659172, "operator": "ge", "actual_fraction": 0.3418774966711052, "metric": 0.09604832793526258, "rescued": 424, "harmed": 753, "missed_rescue": 2890, "avoided_harm": 1192, "selected_both_correct": 2877, "selected_both_wrong": 54}, {"budget": 0.38, "threshold": -0.06637481600046158, "operator": "ge", "actual_fraction": 0.35352862849533956, "metric": 0.09866030704730355, "rescued": 472, "harmed": 774, "missed_rescue": 2842, "avoided_harm": 1171, "selected_both_correct": 2942, "selected_both_wrong": 60}, {"budget": 0.39, "threshold": -0.06774259358644485, "operator": "ge", "actual_fraction": 0.3641810918774967, "metric": 0.10022118439970583, "rescued": 514, "harmed": 797, "missed_rescue": 2800, "avoided_harm": 1148, "selected_both_correct": 3000, "selected_both_wrong": 65}, {"budget": 0.4, "threshold": -0.06843380630016327, "operator": "ge", "actual_fraction": 0.3698402130492676, "metric": 0.10099642512611362, "rescued": 537, "harmed": 810, "missed_rescue": 2777, "avoided_harm": 1135, "selected_both_correct": 3027, "selected_both_wrong": 70}, {"budget": 0.41, "threshold": -0.0694691464304924, "operator": "ge", "actual_fraction": 0.3801597869507324, "metric": 0.10286568189357695, "rescued": 582, "harmed": 836, "missed_rescue": 2732, "avoided_harm": 1109, "selected_both_correct": 3076, "selected_both_wrong": 74}, {"budget": 0.42, "threshold": -0.07112707942724228, "operator": "ge", "actual_fraction": 0.3938082556591212, "metric": 0.1036415927515812, "rescued": 630, "harmed": 864, "missed_rescue": 2684, "avoided_harm": 1081, "selected_both_correct": 3152, "selected_both_wrong": 86}, {"budget": 0.43, "threshold": -0.07192998379468918, "operator": "ge", "actual_fraction": 0.3988015978695073, "metric": 0.10383106693904906, "rescued": 650, "harmed": 874, "missed_rescue": 2664, "avoided_harm": 1071, "selected_both_correct": 3180, "selected_both_wrong": 88}, {"budget": 0.44, "threshold": -0.07266208529472351, "operator": "ge", "actual_fraction": 0.40279627163781623, "metric": 0.10497587071399195, "rescued": 665, "harmed": 882, "missed_rescue": 2649, "avoided_harm": 1063, "selected_both_correct": 3204, "selected_both_wrong": 89}, {"budget": 0.45, "threshold": -0.0744253620505333, "operator": "ge", "actual_fraction": 0.41711051930758986, "metric": 0.10625030009652583, "rescued": 721, "harmed": 910, "missed_rescue": 2593, "avoided_harm": 1035, "selected_both_correct": 3282, "selected_both_wrong": 99}, {"budget": 0.46, "threshold": -0.07532940804958344, "operator": "ge", "actual_fraction": 0.42243675099866845, "metric": 0.10676350886210961, "rescued": 745, "harmed": 919, "missed_rescue": 2569, "avoided_harm": 1026, "selected_both_correct": 3307, "selected_both_wrong": 105}, {"budget": 0.47, "threshold": -0.0770750343799591, "operator": "ge", "actual_fraction": 0.4380825565912117, "metric": 0.10893787008842995, "rescued": 817, "harmed": 955, "missed_rescue": 2497, "avoided_harm": 990, "selected_both_correct": 3377, "selected_both_wrong": 115}, {"budget": 0.48, "threshold": -0.07818037271499634, "operator": "ge", "actual_fraction": 0.44573901464713717, "metric": 0.10946867779982243, "rescued": 849, "harmed": 973, "missed_rescue": 2465, "avoided_harm": 972, "selected_both_correct": 3412, "selected_both_wrong": 122}, {"budget": 0.49, "threshold": -0.07998209446668625, "operator": "ge", "actual_fraction": 0.45905459387483355, "metric": 0.11125477030640087, "rescued": 900, "harmed": 995, "missed_rescue": 2414, "avoided_harm": 950, "selected_both_correct": 3484, "selected_both_wrong": 137}, {"budget": 0.5, "threshold": -0.08124949783086777, "operator": "ge", "actual_fraction": 0.46937416777629826, "metric": 0.1119628778237788, "rescued": 946, "harmed": 1016, "missed_rescue": 2368, "avoided_harm": 929, "selected_both_correct": 3537, "selected_both_wrong": 141}, {"budget": 0.51, "threshold": -0.08398465067148209, "operator": "ge", "actual_fraction": 0.49467376830892146, "metric": 0.11739090876226205, "rescued": 1064, "harmed": 1070, "missed_rescue": 2250, "avoided_harm": 875, "selected_both_correct": 3653, "selected_both_wrong": 157}, {"budget": 0.52, "threshold": -0.08537434041500092, "operator": "ge", "actual_fraction": 0.5056591211717709, "metric": 0.11981696243623577, "rescued": 1112, "harmed": 1093, "missed_rescue": 2202, "avoided_harm": 852, "selected_both_correct": 3704, "selected_both_wrong": 167}, {"budget": 0.53, "threshold": -0.08664409816265106, "operator": "ge", "actual_fraction": 0.5159786950732357, "metric": 0.12050661028396976, "rescued": 1152, "harmed": 1116, "missed_rescue": 2162, "avoided_harm": 829, "selected_both_correct": 3756, "selected_both_wrong": 176}, {"budget": 0.54, "threshold": -0.08777308464050293, "operator": "ge", "actual_fraction": 0.524300932090546, "metric": 0.12109159247774551, "rescued": 1190, "harmed": 1137, "missed_rescue": 2124, "avoided_harm": 808, "selected_both_correct": 3793, "selected_both_wrong": 180}, {"budget": 0.55, "threshold": -0.08980843424797058, "operator": "ge", "actual_fraction": 0.5386151797603196, "metric": 0.12283117300011699, "rescued": 1260, "harmed": 1164, "missed_rescue": 2054, "avoided_harm": 781, "selected_both_correct": 3860, "selected_both_wrong": 188}, {"budget": 0.56, "threshold": -0.0906188115477562, "operator": "ge", "actual_fraction": 0.546271637816245, "metric": 0.12427034914228642, "rescued": 1293, "harmed": 1180, "missed_rescue": 2021, "avoided_harm": 765, "selected_both_correct": 3897, "selected_both_wrong": 194}, {"budget": 0.57, "threshold": -0.09160211682319641, "operator": "ge", "actual_fraction": 0.5555925432756325, "metric": 0.12600739832532146, "rescued": 1339, "harmed": 1194, "missed_rescue": 1975, "avoided_harm": 751, "selected_both_correct": 3940, "selected_both_wrong": 203}, {"budget": 0.58, "threshold": -0.09283093363046646, "operator": "ge", "actual_fraction": 0.568242343541944, "metric": 0.12948657153370394, "rescued": 1401, "harmed": 1224, "missed_rescue": 1913, "avoided_harm": 721, "selected_both_correct": 3996, "selected_both_wrong": 207}, {"budget": 0.59, "threshold": -0.0941123366355896, "operator": "ge", "actual_fraction": 0.5778961384820239, "metric": 0.1321361114270833, "rescued": 1447, "harmed": 1245, "missed_rescue": 1867, "avoided_harm": 700, "selected_both_correct": 4036, "selected_both_wrong": 216}, {"budget": 0.6, "threshold": -0.09614869952201843, "operator": "ge", "actual_fraction": 0.5878828229027963, "metric": 0.13359562560258884, "rescued": 1495, "harmed": 1262, "missed_rescue": 1819, "avoided_harm": 683, "selected_both_correct": 4083, "selected_both_wrong": 224}, {"budget": 0.61, "threshold": -0.09772664308547974, "operator": "ge", "actual_fraction": 0.5975366178428761, "metric": 0.135418307343696, "rescued": 1537, "harmed": 1279, "missed_rescue": 1777, "avoided_harm": 666, "selected_both_correct": 4133, "selected_both_wrong": 231}, {"budget": 0.62, "threshold": -0.09911532700061798, "operator": "ge", "actual_fraction": 0.6081890812250332, "metric": 0.13720391892557468, "rescued": 1585, "harmed": 1294, "missed_rescue": 1729, "avoided_harm": 651, "selected_both_correct": 4189, "selected_both_wrong": 240}, {"budget": 0.63, "threshold": -0.10007988661527634, "operator": "ge", "actual_fraction": 0.6161784287616512, "metric": 0.13839536227953134, "rescued": 1625, "harmed": 1312, "missed_rescue": 1689, "avoided_harm": 633, "selected_both_correct": 4222, "selected_both_wrong": 245}, {"budget": 0.64, "threshold": -0.10177135467529297, "operator": "ge", "actual_fraction": 0.6278295605858855, "metric": 0.14186473384550358, "rescued": 1685, "harmed": 1327, "missed_rescue": 1629, "avoided_harm": 618, "selected_both_correct": 4280, "selected_both_wrong": 252}, {"budget": 0.65, "threshold": -0.10395076125860214, "operator": "ge", "actual_fraction": 0.6451398135818908, "metric": 0.14514480336324412, "rescued": 1771, "harmed": 1357, "missed_rescue": 1543, "avoided_harm": 588, "selected_both_correct": 4362, "selected_both_wrong": 262}, {"budget": 0.66, "threshold": -0.10524322092533112, "operator": "ge", "actual_fraction": 0.6491344873501997, "metric": 0.14705419026423194, "rescued": 1792, "harmed": 1362, "missed_rescue": 1522, "avoided_harm": 583, "selected_both_correct": 4381, "selected_both_wrong": 265}, {"budget": 0.67, "threshold": -0.10689426958560944, "operator": "ge", "actual_fraction": 0.6601198402130493, "metric": 0.14778857092985256, "rescued": 1842, "harmed": 1385, "missed_rescue": 1472, "avoided_harm": 560, "selected_both_correct": 4428, "selected_both_wrong": 277}, {"budget": 0.68, "threshold": -0.11014843732118607, "operator": "ge", "actual_fraction": 0.6754327563249002, "metric": 0.15144863687072044, "rescued": 1911, "harmed": 1408, "missed_rescue": 1403, "avoided_harm": 537, "selected_both_correct": 4509, "selected_both_wrong": 288}, {"budget": 0.69, "threshold": -0.11117545515298843, "operator": "ge", "actual_fraction": 0.6790945406125166, "metric": 0.1527001767891735, "rescued": 1931, "harmed": 1414, "missed_rescue": 1383, "avoided_harm": 531, "selected_both_correct": 4524, "selected_both_wrong": 291}, {"budget": 0.7, "threshold": -0.11230025440454483, "operator": "ge", "actual_fraction": 0.6870838881491345, "metric": 0.15446609724321056, "rescued": 1973, "harmed": 1424, "missed_rescue": 1341, "avoided_harm": 521, "selected_both_correct": 4561, "selected_both_wrong": 298}, {"budget": 0.71, "threshold": -0.11423606425523758, "operator": "ge", "actual_fraction": 0.6960719041278296, "metric": 0.1566074758661257, "rescued": 2016, "harmed": 1440, "missed_rescue": 1298, "avoided_harm": 505, "selected_both_correct": 4599, "selected_both_wrong": 309}, {"budget": 0.72, "threshold": -0.11647249013185501, "operator": "ge", "actual_fraction": 0.7053928095872171, "metric": 0.15830843131436304, "rescued": 2059, "harmed": 1456, "missed_rescue": 1255, "avoided_harm": 489, "selected_both_correct": 4643, "selected_both_wrong": 318}, {"budget": 0.73, "threshold": -0.11962417513132095, "operator": "ge", "actual_fraction": 0.7193741677762983, "metric": 0.16212144588492078, "rescued": 2121, "harmed": 1481, "missed_rescue": 1193, "avoided_harm": 464, "selected_both_correct": 4706, "selected_both_wrong": 336}, {"budget": 0.74, "threshold": -0.12126146256923676, "operator": "ge", "actual_fraction": 0.7250332889480693, "metric": 0.16312718485088668, "rescued": 2145, "harmed": 1491, "missed_rescue": 1169, "avoided_harm": 454, "selected_both_correct": 4732, "selected_both_wrong": 344}, {"budget": 0.75, "threshold": -0.1230982318520546, "operator": "ge", "actual_fraction": 0.7366844207723036, "metric": 0.1660266588911306, "rescued": 2198, "harmed": 1515, "missed_rescue": 1116, "avoided_harm": 430, "selected_both_correct": 4785, "selected_both_wrong": 354}, {"budget": 0.76, "threshold": -0.12517565488815308, "operator": "ge", "actual_fraction": 0.7446737683089214, "metric": 0.16822905417177203, "rescued": 2234, "harmed": 1526, "missed_rescue": 1080, "avoided_harm": 419, "selected_both_correct": 4828, "selected_both_wrong": 360}, {"budget": 0.77, "threshold": -0.12628257274627686, "operator": "ge", "actual_fraction": 0.7480026631158455, "metric": 0.1691185459978786, "rescued": 2251, "harmed": 1531, "missed_rescue": 1063, "avoided_harm": 414, "selected_both_correct": 4843, "selected_both_wrong": 363}, {"budget": 0.78, "threshold": -0.12975578010082245, "operator": "ge", "actual_fraction": 0.7603195739014648, "metric": 0.17190478913940857, "rescued": 2306, "harmed": 1552, "missed_rescue": 1008, "avoided_harm": 393, "selected_both_correct": 4897, "selected_both_wrong": 381}, {"budget": 0.79, "threshold": -0.13229140639305115, "operator": "ge", "actual_fraction": 0.7713049267643143, "metric": 0.17440054880534744, "rescued": 2355, "harmed": 1568, "missed_rescue": 959, "avoided_harm": 377, "selected_both_correct": 4952, "selected_both_wrong": 393}, {"budget": 0.8, "threshold": -0.1373685747385025, "operator": "ge", "actual_fraction": 0.7909454061251664, "metric": 0.17894927236127425, "rescued": 2455, "harmed": 1608, "missed_rescue": 859, "avoided_harm": 337, "selected_both_correct": 5030, "selected_both_wrong": 411}, {"budget": 0.81, "threshold": -0.1407174915075302, "operator": "ge", "actual_fraction": 0.8012649800266312, "metric": 0.18049284683090172, "rescued": 2502, "harmed": 1631, "missed_rescue": 812, "avoided_harm": 314, "selected_both_correct": 5069, "selected_both_wrong": 426}, {"budget": 0.82, "threshold": -0.1433146893978119, "operator": "ge", "actual_fraction": 0.8125832223701731, "metric": 0.18188035357936716, "rescued": 2556, "harmed": 1658, "missed_rescue": 758, "avoided_harm": 287, "selected_both_correct": 5115, "selected_both_wrong": 435}, {"budget": 0.83, "threshold": -0.1457100659608841, "operator": "ge", "actual_fraction": 0.8199067909454061, "metric": 0.1842965007470308, "rescued": 2593, "harmed": 1670, "missed_rescue": 721, "avoided_harm": 275, "selected_both_correct": 5145, "selected_both_wrong": 444}, {"budget": 0.84, "threshold": -0.148710235953331, "operator": "ge", "actual_fraction": 0.8242343541944075, "metric": 0.18526487033383585, "rescued": 2610, "harmed": 1675, "missed_rescue": 704, "avoided_harm": 270, "selected_both_correct": 5167, "selected_both_wrong": 452}, {"budget": 0.85, "threshold": -0.15291842818260193, "operator": "ge", "actual_fraction": 0.8338881491344874, "metric": 0.18825160118942358, "rescued": 2656, "harmed": 1694, "missed_rescue": 658, "avoided_harm": 251, "selected_both_correct": 5208, "selected_both_wrong": 462}, {"budget": 0.86, "threshold": -0.15889839828014374, "operator": "ge", "actual_fraction": 0.8458721704394141, "metric": 0.19240881033397328, "rescued": 2704, "harmed": 1712, "missed_rescue": 610, "avoided_harm": 233, "selected_both_correct": 5271, "selected_both_wrong": 477}, {"budget": 0.87, "threshold": -0.16629797220230103, "operator": "ge", "actual_fraction": 0.8585219707057257, "metric": 0.1975333545503005, "rescued": 2763, "harmed": 1734, "missed_rescue": 551, "avoided_harm": 211, "selected_both_correct": 5328, "selected_both_wrong": 491}, {"budget": 0.88, "threshold": -0.17295491695404053, "operator": "ge", "actual_fraction": 0.8655126498002663, "metric": 0.19989071661116012, "rescued": 2792, "harmed": 1743, "missed_rescue": 522, "avoided_harm": 202, "selected_both_correct": 5365, "selected_both_wrong": 500}, {"budget": 0.89, "threshold": -0.18053314089775085, "operator": "ge", "actual_fraction": 0.8738348868175766, "metric": 0.20467943147275502, "rescued": 2828, "harmed": 1754, "missed_rescue": 486, "avoided_harm": 191, "selected_both_correct": 5407, "selected_both_wrong": 511}, {"budget": 0.9, "threshold": -0.19442276656627655, "operator": "ge", "actual_fraction": 0.8878162450066578, "metric": 0.20833827941063784, "rescued": 2888, "harmed": 1779, "missed_rescue": 426, "avoided_harm": 166, "selected_both_correct": 5471, "selected_both_wrong": 530}, {"budget": 0.91, "threshold": -0.20900504291057587, "operator": "ge", "actual_fraction": 0.8961384820239681, "metric": 0.2085245066088965, "rescued": 2924, "harmed": 1790, "missed_rescue": 390, "avoided_harm": 155, "selected_both_correct": 5515, "selected_both_wrong": 539}, {"budget": 0.92, "threshold": -0.22383525967597961, "operator": "ge", "actual_fraction": 0.9027962716378163, "metric": 0.2108748825898504, "rescued": 2952, "harmed": 1799, "missed_rescue": 362, "avoided_harm": 146, "selected_both_correct": 5550, "selected_both_wrong": 547}, {"budget": 0.93, "threshold": -0.25952714681625366, "operator": "ge", "actual_fraction": 0.9184420772303595, "metric": 0.21438099190536286, "rescued": 3019, "harmed": 1827, "missed_rescue": 295, "avoided_harm": 118, "selected_both_correct": 5620, "selected_both_wrong": 570}, {"budget": 0.94, "threshold": -0.2804492115974426, "operator": "ge", "actual_fraction": 0.9257656458055925, "metric": 0.21506366828770151, "rescued": 3050, "harmed": 1842, "missed_rescue": 264, "avoided_harm": 103, "selected_both_correct": 5649, "selected_both_wrong": 583}, {"budget": 0.95, "threshold": -0.2984963357448578, "operator": "ge", "actual_fraction": 0.9320905459387483, "metric": 0.21567531720794794, "rescued": 3069, "harmed": 1849, "missed_rescue": 245, "avoided_harm": 96, "selected_both_correct": 5689, "selected_both_wrong": 593}, {"budget": 0.96, "threshold": -0.32223379611968994, "operator": "ge", "actual_fraction": 0.9410785619174434, "metric": 0.21264209690310557, "rescued": 3101, "harmed": 1865, "missed_rescue": 213, "avoided_harm": 80, "selected_both_correct": 5735, "selected_both_wrong": 607}, {"budget": 0.97, "threshold": -0.36503666639328003, "operator": "ge", "actual_fraction": 0.9573901464713716, "metric": 0.21331564253317523, "rescued": 3160, "harmed": 1887, "missed_rescue": 154, "avoided_harm": 58, "selected_both_correct": 5818, "selected_both_wrong": 639}, {"budget": 0.98, "threshold": -0.4221048653125763, "operator": "ge", "actual_fraction": 0.9790279627163782, "metric": 0.21609512872414097, "rescued": 3244, "harmed": 1926, "missed_rescue": 70, "avoided_harm": 19, "selected_both_correct": 5928, "selected_both_wrong": 666}, {"budget": 0.99, "threshold": -0.44162338972091675, "operator": "ge", "actual_fraction": 0.9860186418109188, "metric": 0.21817095656956434, "rescued": 3268, "harmed": 1932, "missed_rescue": 46, "avoided_harm": 13, "selected_both_correct": 5967, "selected_both_wrong": 681}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.22361115727328065, "rescued": 3314, "harmed": 1945, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 6060, "selected_both_wrong": 697}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.21991914076339864, "actual_fraction": 0.7728958630527817, "metric_std": 0.0008461234201884544, "actual_fraction_std": 0.008931512606275843, "per_seed": [{"seed": 2026, "budget": 0.76, "threshold": -0.01648801565170288, "operator": "ge", "alpha": 0.0, "epoch": 5, "metric": 0.22072697083038884, "actual_fraction": 0.7589158345221114, "rescue": 781, "harm": 340, "missed_rescue": 5}, {"seed": 2027, "budget": 0.77, "threshold": -0.008658885955810547, "operator": "ge", "alpha": 0.0, "epoch": 4, "metric": 0.2209509348306372, "actual_fraction": 0.7689015691868759, "rescue": 783, "harm": 346, "missed_rescue": 3}, {"seed": 2028, "budget": 0.78, "threshold": -0.2036621480054694, "operator": "ge", "alpha": 0.25, "epoch": 6, "metric": 0.2193613683209817, "actual_fraction": 0.7788873038516405, "rescue": 784, "harm": 350, "missed_rescue": 2}, {"seed": 2029, "budget": 0.78, "threshold": -0.07148108631372452, "operator": "ge", "alpha": 0.0, "epoch": 7, "metric": 0.2193613683209817, "actual_fraction": 0.7788873038516405, "rescue": 784, "harm": 350, "missed_rescue": 2}, {"seed": 2030, "budget": 0.78, "threshold": -0.0899617075920105, "operator": "ge", "alpha": 0.0, "epoch": 4, "metric": 0.21919506151400395, "actual_fraction": 0.7788873038516405, "rescue": 784, "harm": 352, "missed_rescue": 2}], "rescued": 783.2, "harmed": 347.6, "missed_rescue": 2.8, "avoided_harm": 127.39999999999998}, "validation_curve": [{"budget": 0.0, "metric": 0.07641657905799261, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.08338921568169272, "metric_std": 0.013156651655131938, "actual_fraction": 0.009985734664764621, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.08254534591731555, "metric_std": 0.010019925880860201, "actual_fraction": 0.019971469329529243, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.08282020120648026, "metric_std": 0.008020369609656816, "actual_fraction": 0.029957203994293864, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.08818906934958237, "metric_std": 0.00881753529671969, "actual_fraction": 0.039942938659058486, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.09042277132014556, "metric_std": 0.005999756216733253, "actual_fraction": 0.04992867332382311, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.09039670009236037, "metric_std": 0.0053571137917998056, "actual_fraction": 0.05991440798858773, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.09271507280614087, "metric_std": 0.0007898137627342854, "actual_fraction": 0.06990014265335236, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.09271783095170578, "metric_std": 0.0006326421189781108, "actual_fraction": 0.07988587731811697, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.09317027477256859, "metric_std": 0.0005388166570330193, "actual_fraction": 0.0898716119828816, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.09354854615350235, "metric_std": 0.000439484874397691, "actual_fraction": 0.09985734664764621, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.09433961732725846, "metric_std": 0.000428213266439539, "actual_fraction": 0.10984308131241086, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.12, "metric": 0.0948664413421527, "metric_std": 0.0004960104327867893, "actual_fraction": 0.11982881597717546, "actual_fraction_std": 0.0}, {"budget": 0.13, "metric": 0.09511476857465531, "metric_std": 0.0009711408803295944, "actual_fraction": 0.12981455064194009, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.0987246886691763, "metric_std": 0.00326314915931011, "actual_fraction": 0.1398002853067047, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.10079601555323392, "metric_std": 0.004285986719685645, "actual_fraction": 0.14978601997146934, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.10275291222609828, "metric_std": 0.0038214086755819594, "actual_fraction": 0.15977175463623394, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.10610249397231808, "metric_std": 0.005031801160628947, "actual_fraction": 0.16975748930099857, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.10935199905228668, "metric_std": 0.007601364046549993, "actual_fraction": 0.1797432239657632, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.11002057330430912, "metric_std": 0.007249833045099907, "actual_fraction": 0.18972895863052786, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.11686022407032977, "metric_std": 0.006718042282434146, "actual_fraction": 0.19971469329529243, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.11985228440223188, "metric_std": 0.005643787903056527, "actual_fraction": 0.20970042796005703, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.1208986907277397, "metric_std": 0.006236422436248517, "actual_fraction": 0.2196861626248217, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.23, "metric": 0.12135253348689475, "metric_std": 0.006606220033244298, "actual_fraction": 0.22967189728958629, "actual_fraction_std": 0.0}, {"budget": 0.24, "metric": 0.12270755995294544, "metric_std": 0.004241545449600179, "actual_fraction": 0.2396576319543509, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.12595477819082565, "metric_std": 0.001778599242917857, "actual_fraction": 0.24964336661911551, "actual_fraction_std": 0.0}, {"budget": 0.26, "metric": 0.12710727441636913, "metric_std": 0.0014622572773552364, "actual_fraction": 0.25962910128388017, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.12972723163545766, "metric_std": 0.002127427217442173, "actual_fraction": 0.26961483594864477, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.13049784124344618, "metric_std": 0.001932219168812355, "actual_fraction": 0.2796005706134094, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.1308655799238289, "metric_std": 0.0025196941429129105, "actual_fraction": 0.289586305278174, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.13346649058610513, "metric_std": 0.0015049755586024106, "actual_fraction": 0.2995720399429387, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.13484252715220696, "metric_std": 0.0023778680198214158, "actual_fraction": 0.3095577746077033, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.13653558679382605, "metric_std": 0.002399701595835136, "actual_fraction": 0.3195435092724679, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.13935013948314895, "metric_std": 0.003404276146181599, "actual_fraction": 0.3295292439372325, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.1431180649624519, "metric_std": 0.004359795857738276, "actual_fraction": 0.33951497860199714, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.14524193118056436, "metric_std": 0.005338481645551131, "actual_fraction": 0.34950071326676174, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.14623205257012234, "metric_std": 0.005273497731497387, "actual_fraction": 0.3594864479315264, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.1503247032474309, "metric_std": 0.004548283251468576, "actual_fraction": 0.36947218259629094, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.1522505147336096, "metric_std": 0.00468545212399622, "actual_fraction": 0.3794579172610557, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.15301079024726444, "metric_std": 0.004659522822729499, "actual_fraction": 0.38944365192582026, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.15462587511927947, "metric_std": 0.00307274750840958, "actual_fraction": 0.39942938659058486, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.15644011891309012, "metric_std": 0.00157762768227431, "actual_fraction": 0.4094151212553495, "actual_fraction_std": 0.0}, {"budget": 0.42, "metric": 0.15894255577154046, "metric_std": 0.0018080938859879173, "actual_fraction": 0.41940085592011406, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.1604505807469183, "metric_std": 0.002099724903304191, "actual_fraction": 0.4293865905848787, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.44, "metric": 0.16240600787723533, "metric_std": 0.0011811864077783986, "actual_fraction": 0.4393723252496434, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.45, "metric": 0.16377333071304206, "metric_std": 0.0009949254350633584, "actual_fraction": 0.44935805991440797, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.16518347915442702, "metric_std": 0.0009039716709736596, "actual_fraction": 0.45934379457917257, "actual_fraction_std": 0.0}, {"budget": 0.47, "metric": 0.1660647638121496, "metric_std": 0.0008541896205389739, "actual_fraction": 0.4693295292439372, "actual_fraction_std": 0.0}, {"budget": 0.48, "metric": 0.16733538583182175, "metric_std": 0.002142242884957183, "actual_fraction": 0.4793152639087018, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.16859699268637937, "metric_std": 0.002942526630871446, "actual_fraction": 0.4893009985734665, "actual_fraction_std": 0.0}, {"budget": 0.5, "metric": 0.16974581621500143, "metric_std": 0.004852022979462967, "actual_fraction": 0.49928673323823103, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.17132659032219583, "metric_std": 0.005670020600719076, "actual_fraction": 0.5092724679029957, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.17323058221080168, "metric_std": 0.005570480103030789, "actual_fraction": 0.5192582025677603, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.17547405286556159, "metric_std": 0.0058285888905602965, "actual_fraction": 0.5292439372325249, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.17687724094859658, "metric_std": 0.00555107854265531, "actual_fraction": 0.5392296718972895, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.17862330298383697, "metric_std": 0.00592765402501693, "actual_fraction": 0.5492154065620543, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.18030439854510352, "metric_std": 0.006005365160611073, "actual_fraction": 0.5592011412268189, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.180916077290024, "metric_std": 0.005404959854922733, "actual_fraction": 0.5691868758915835, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.18255357344835083, "metric_std": 0.006868431130244264, "actual_fraction": 0.579172610556348, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.18459502535354969, "metric_std": 0.006715634554541828, "actual_fraction": 0.5891583452211127, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.18660026564813442, "metric_std": 0.006468369206725944, "actual_fraction": 0.5991440798858774, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.18931848298586282, "metric_std": 0.0074698472068701356, "actual_fraction": 0.609129814550642, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.19148170671070106, "metric_std": 0.007303774493564469, "actual_fraction": 0.6191155492154066, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.1926282828812828, "metric_std": 0.00837721678443553, "actual_fraction": 0.6291012838801712, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.19521043702034185, "metric_std": 0.008691866910525418, "actual_fraction": 0.6390870185449358, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.19739642296672105, "metric_std": 0.007889866304611904, "actual_fraction": 0.6490727532097005, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.19829436289456465, "metric_std": 0.008232369518418195, "actual_fraction": 0.659058487874465, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.19977102823370071, "metric_std": 0.00718717270416535, "actual_fraction": 0.6690442225392297, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.20031665682963817, "metric_std": 0.007300666980029299, "actual_fraction": 0.6790299572039943, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.20284672457043582, "metric_std": 0.007206580138138918, "actual_fraction": 0.6890156918687589, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.2045647548877132, "metric_std": 0.00418912618891339, "actual_fraction": 0.6990014265335235, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.20613980339596877, "metric_std": 0.0020091723398601834, "actual_fraction": 0.7089871611982882, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.20787577330566903, "metric_std": 0.001989862107377955, "actual_fraction": 0.7189728958630528, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.21106299399477374, "metric_std": 0.002769429824700505, "actual_fraction": 0.7289586305278174, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.21448134065791163, "metric_std": 0.0023633378099220444, "actual_fraction": 0.7389443651925819, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.2150840403361903, "metric_std": 0.001426833386399145, "actual_fraction": 0.7489300998573466, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.2178182699059205, "metric_std": 0.0017044667008688033, "actual_fraction": 0.7589158345221114, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.219281538799108, "metric_std": 0.0009334633166739029, "actual_fraction": 0.7689015691868759, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.21930876832704543, "metric_std": 0.00010761989912760467, "actual_fraction": 0.7788873038516405, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.2185630487399564, "metric_std": 0.0009592197739102619, "actual_fraction": 0.7888730385164051, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.21787299284199832, "metric_std": 0.001105728521619101, "actual_fraction": 0.7988587731811697, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.21796229632224776, "metric_std": 0.0008830815947082544, "actual_fraction": 0.8088445078459344, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.2172580934526433, "metric_std": 0.0011584227499835539, "actual_fraction": 0.818830242510699, "actual_fraction_std": 0.0}, {"budget": 0.83, "metric": 0.21629293459906523, "metric_std": 0.00208671399420215, "actual_fraction": 0.8288159771754635, "actual_fraction_std": 0.0}, {"budget": 0.84, "metric": 0.21603045882109523, "metric_std": 0.0020732065480179953, "actual_fraction": 0.8388017118402281, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.21607361386705107, "metric_std": 0.0018120933723774527, "actual_fraction": 0.8487874465049929, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.86, "metric": 0.21552952500473258, "metric_std": 0.002083024640584593, "actual_fraction": 0.8587731811697574, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.87, "metric": 0.21482057111816447, "metric_std": 0.0019074390624013126, "actual_fraction": 0.8687589158345222, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.2141013923925708, "metric_std": 0.0015851851933416303, "actual_fraction": 0.8787446504992869, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.89, "metric": 0.21361256295696363, "metric_std": 0.0004583112581589424, "actual_fraction": 0.8887303851640513, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.2131762569115895, "metric_std": 0.00048469613659501883, "actual_fraction": 0.8987161198288159, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.2125101070399809, "metric_std": 0.00042628543935384355, "actual_fraction": 0.9087018544935808, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.21158502922748382, "metric_std": 0.0011019574252365614, "actual_fraction": 0.9186875891583451, "actual_fraction_std": 0.0}, {"budget": 0.93, "metric": 0.21076967063104698, "metric_std": 0.0006584903910486059, "actual_fraction": 0.9286733238231101, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.94, "metric": 0.20990521484416397, "metric_std": 0.0003546819155648376, "actual_fraction": 0.9386590584878745, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.20903247333719524, "metric_std": 0.0003762986158467286, "actual_fraction": 0.948644793152639, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.20812436933016215, "metric_std": 0.0005728802925297892, "actual_fraction": 0.9586305278174037, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.2076739415546467, "metric_std": 0.0012064101579333218, "actual_fraction": 0.9686162624821684, "actual_fraction_std": 0.0}, {"budget": 0.98, "metric": 0.20722367198907454, "metric_std": 0.0012361100526525903, "actual_fraction": 0.978601997146933, "actual_fraction_std": 0.0}, {"budget": 0.99, "metric": 0.2077801908595506, "metric_std": 0.0010326666151966649, "actual_fraction": 0.9885877318116976, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.20666016863629594, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.2279931433315267, "actual_fraction": 0.7843541944074567, "metric_std": 0.003705941741110254, "actual_fraction_std": 0.012042596161029985, "per_seed": [{"seed": 2026, "budget": 0.76, "threshold": -0.01648801565170288, "operator": "ge", "alpha": 0.0, "epoch": 5, "metric": 0.22151919702262796, "actual_fraction": 0.7683089214380826, "rescue": 3258, "harm": 1424, "missed_rescue": 56}, {"seed": 2027, "budget": 0.77, "threshold": -0.008658885955810547, "operator": "ge", "alpha": 0.0, "epoch": 4, "metric": 0.228248953805867, "actual_fraction": 0.7746338215712385, "rescue": 3278, "harm": 1437, "missed_rescue": 36}, {"seed": 2028, "budget": 0.78, "threshold": -0.2036621480054694, "operator": "ge", "alpha": 0.25, "epoch": 6, "metric": 0.22993133826371603, "actual_fraction": 0.7949400798934754, "rescue": 3304, "harm": 1472, "missed_rescue": 10}, {"seed": 2029, "budget": 0.78, "threshold": -0.07148108631372452, "operator": "ge", "alpha": 0.0, "epoch": 7, "metric": 0.23027982827701313, "actual_fraction": 0.7912782956058588, "rescue": 3302, "harm": 1461, "missed_rescue": 12}, {"seed": 2030, "budget": 0.78, "threshold": -0.0899617075920105, "operator": "ge", "alpha": 0.0, "epoch": 4, "metric": 0.22998639928840947, "actual_fraction": 0.7926098535286284, "rescue": 3301, "harm": 1468, "missed_rescue": 13}], "rescued": 3288.6, "harmed": 1452.4, "missed_rescue": 25.4, "avoided_harm": 492.5999999999999}, "test_curve": [{"budget": 0.0, "metric": 0.07887397871043929, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.07959720988408628, "metric_std": 0.0003319781554978558, "actual_fraction": 0.00625832223701731, "actual_fraction_std": 0.0028674907413259692}, {"budget": 0.02, "metric": 0.08488020809843055, "metric_std": 0.002638363936346364, "actual_fraction": 0.01251664447403462, "actual_fraction_std": 0.0035744952584676744}, {"budget": 0.03, "metric": 0.09077072889467792, "metric_std": 0.004620000224680201, "actual_fraction": 0.018442077230359517, "actual_fraction_std": 0.0033751775913812444}, {"budget": 0.04, "metric": 0.09213983246476096, "metric_std": 0.007015176963618525, "actual_fraction": 0.023302263648468706, "actual_fraction_std": 0.0024911167688241942}, {"budget": 0.05, "metric": 0.09561392562662235, "metric_std": 0.006051104030092947, "actual_fraction": 0.029693741677762987, "actual_fraction_std": 0.00449339148505911}, {"budget": 0.06, "metric": 0.09989832684336886, "metric_std": 0.008320865624363924, "actual_fraction": 0.03575233022636485, "actual_fraction_std": 0.00526660196054556}, {"budget": 0.07, "metric": 0.10475786074298052, "metric_std": 0.0074540245931587045, "actual_fraction": 0.045006657789613845, "actual_fraction_std": 0.005132331058178239}, {"budget": 0.08, "metric": 0.10930019666491764, "metric_std": 0.005065157420427044, "actual_fraction": 0.05599201065246338, "actual_fraction_std": 0.006667768819469658}, {"budget": 0.09, "metric": 0.11126341583291069, "metric_std": 0.003978992855433337, "actual_fraction": 0.06165113182423436, "actual_fraction_std": 0.005323109937352686}, {"budget": 0.1, "metric": 0.1128523868105757, "metric_std": 0.0016559187501769906, "actual_fraction": 0.0718375499334221, "actual_fraction_std": 0.004304455359585784}, {"budget": 0.11, "metric": 0.11339332889451888, "metric_std": 0.0007682706722803367, "actual_fraction": 0.07762982689747003, "actual_fraction_std": 0.0030455294304880683}, {"budget": 0.12, "metric": 0.1135484410657855, "metric_std": 0.0011375606700216572, "actual_fraction": 0.08528628495339549, "actual_fraction_std": 0.002528645674411843}, {"budget": 0.13, "metric": 0.11379701759324483, "metric_std": 0.0016616381669361307, "actual_fraction": 0.09207723035952065, "actual_fraction_std": 0.0036053636661950235}, {"budget": 0.14, "metric": 0.1149837940764945, "metric_std": 0.00339842303424254, "actual_fraction": 0.10186418109187752, "actual_fraction_std": 0.004131049815576182}, {"budget": 0.15, "metric": 0.11596470728157397, "metric_std": 0.003881107618740854, "actual_fraction": 0.1096537949400799, "actual_fraction_std": 0.005523357798451395}, {"budget": 0.16, "metric": 0.11685293516106614, "metric_std": 0.0038258195522895373, "actual_fraction": 0.12157123834886815, "actual_fraction_std": 0.00296252935569733}, {"budget": 0.17, "metric": 0.11788424364443661, "metric_std": 0.004494556676404492, "actual_fraction": 0.13475366178428763, "actual_fraction_std": 0.0013603739508457142}, {"budget": 0.18, "metric": 0.11797748383442805, "metric_std": 0.00476410891699676, "actual_fraction": 0.14760319573901465, "actual_fraction_std": 0.003180793391635483}, {"budget": 0.19, "metric": 0.11858495256998858, "metric_std": 0.0043062868453196115, "actual_fraction": 0.1565912117177097, "actual_fraction_std": 0.0050572913488236785}, {"budget": 0.2, "metric": 0.11848889816248478, "metric_std": 0.0038336654296440066, "actual_fraction": 0.1633155792276964, "actual_fraction_std": 0.006274768033813428}, {"budget": 0.21, "metric": 0.12018348227000972, "metric_std": 0.004241418418397687, "actual_fraction": 0.17483355525965377, "actual_fraction_std": 0.004239605985335023}, {"budget": 0.22, "metric": 0.12194787767274125, "metric_std": 0.0023407978168642255, "actual_fraction": 0.1880825565912117, "actual_fraction_std": 0.00422389398816561}, {"budget": 0.23, "metric": 0.1239668078114233, "metric_std": 0.0030985921424182424, "actual_fraction": 0.19680426098535286, "actual_fraction_std": 0.006117382015724302}, {"budget": 0.24, "metric": 0.124913968530368, "metric_std": 0.0027215553760034233, "actual_fraction": 0.21091877496671105, "actual_fraction_std": 0.008230519904915879}, {"budget": 0.25, "metric": 0.12579312450397037, "metric_std": 0.002539657687926565, "actual_fraction": 0.22203728362183756, "actual_fraction_std": 0.006540240580688565}, {"budget": 0.26, "metric": 0.1273366960151278, "metric_std": 0.0034192093686767665, "actual_fraction": 0.23308921438082558, "actual_fraction_std": 0.005970704834826171}, {"budget": 0.27, "metric": 0.12881642801982113, "metric_std": 0.0030954235004688846, "actual_fraction": 0.24420772303595206, "actual_fraction_std": 0.004717174323280534}, {"budget": 0.28, "metric": 0.13061125485265995, "metric_std": 0.003383430035845582, "actual_fraction": 0.25366178428761654, "actual_fraction_std": 0.006467829303503814}, {"budget": 0.29, "metric": 0.1318147674416982, "metric_std": 0.0033459955273728555, "actual_fraction": 0.2657789613848202, "actual_fraction_std": 0.00997169314223697}, {"budget": 0.3, "metric": 0.1330625379869194, "metric_std": 0.003813704334256455, "actual_fraction": 0.27756324900133156, "actual_fraction_std": 0.011484445857093399}, {"budget": 0.31, "metric": 0.13534847478282983, "metric_std": 0.003412473139102988, "actual_fraction": 0.2888814913448735, "actual_fraction_std": 0.011852455046827765}, {"budget": 0.32, "metric": 0.13621842535730794, "metric_std": 0.003972120297267552, "actual_fraction": 0.3021304926764314, "actual_fraction_std": 0.010622774039391552}, {"budget": 0.33, "metric": 0.1382691703885371, "metric_std": 0.0029323163178764547, "actual_fraction": 0.3123834886817577, "actual_fraction_std": 0.010106910585270319}, {"budget": 0.34, "metric": 0.13983302999929856, "metric_std": 0.0027021175496496803, "actual_fraction": 0.3235685752330227, "actual_fraction_std": 0.009394917304707132}, {"budget": 0.35, "metric": 0.14121896478856683, "metric_std": 0.0023205746560854583, "actual_fraction": 0.3355525965379494, "actual_fraction_std": 0.008981849243163777}, {"budget": 0.36, "metric": 0.14231842089695101, "metric_std": 0.0026840565747632142, "actual_fraction": 0.346404793608522, "actual_fraction_std": 0.011238656510641454}, {"budget": 0.37, "metric": 0.14408972784451518, "metric_std": 0.00259451028949215, "actual_fraction": 0.35639147802929433, "actual_fraction_std": 0.011184293929179079}, {"budget": 0.38, "metric": 0.14541769517975192, "metric_std": 0.002533402116043762, "actual_fraction": 0.36697736351531296, "actual_fraction_std": 0.01311559206768398}, {"budget": 0.39, "metric": 0.14714198648395724, "metric_std": 0.0027784375572308365, "actual_fraction": 0.3801597869507324, "actual_fraction_std": 0.007436187717606095}, {"budget": 0.4, "metric": 0.1488794416470895, "metric_std": 0.0021381258877393033, "actual_fraction": 0.3904793608521971, "actual_fraction_std": 0.0036844113883453427}, {"budget": 0.41, "metric": 0.15117840692627454, "metric_std": 0.0013728227014211654, "actual_fraction": 0.40346205059920115, "actual_fraction_std": 0.0033948199158407204}, {"budget": 0.42, "metric": 0.153877985658002, "metric_std": 0.002000259993103127, "actual_fraction": 0.4178428761651132, "actual_fraction_std": 0.003906313025753952}, {"budget": 0.43, "metric": 0.15512995832965268, "metric_std": 0.0016935452899005023, "actual_fraction": 0.4302929427430093, "actual_fraction_std": 0.0056767173761097705}, {"budget": 0.44, "metric": 0.15646110337614394, "metric_std": 0.0011557056207739007, "actual_fraction": 0.4428761651131824, "actual_fraction_std": 0.0023254659382920097}, {"budget": 0.45, "metric": 0.15729295405945726, "metric_std": 0.0008780811187484744, "actual_fraction": 0.4515978695073236, "actual_fraction_std": 0.0019065008073603392}, {"budget": 0.46, "metric": 0.15955690672157036, "metric_std": 0.0011787569543210735, "actual_fraction": 0.46631158455392807, "actual_fraction_std": 0.002943767106449304}, {"budget": 0.47, "metric": 0.16039779576485763, "metric_std": 0.0015584139214274318, "actual_fraction": 0.4766311584553928, "actual_fraction_std": 0.004797536857604224}, {"budget": 0.48, "metric": 0.16153867214796946, "metric_std": 0.003039907287930013, "actual_fraction": 0.48821571238348865, "actual_fraction_std": 0.008122094028237173}, {"budget": 0.49, "metric": 0.16264050110470998, "metric_std": 0.0051615786883870765, "actual_fraction": 0.4989347536617842, "actual_fraction_std": 0.0072068842753924495}, {"budget": 0.5, "metric": 0.16328694790454698, "metric_std": 0.0062377774105485265, "actual_fraction": 0.505659121171771, "actual_fraction_std": 0.008464169197797008}, {"budget": 0.51, "metric": 0.16490125294422991, "metric_std": 0.007744278062926317, "actual_fraction": 0.5180426098535286, "actual_fraction_std": 0.006721565381010893}, {"budget": 0.52, "metric": 0.16678307545611598, "metric_std": 0.007480003588801224, "actual_fraction": 0.5290279627163782, "actual_fraction_std": 0.005270808512302899}, {"budget": 0.53, "metric": 0.16812907427028442, "metric_std": 0.00724341676709438, "actual_fraction": 0.5372836218375499, "actual_fraction_std": 0.008935463070219387}, {"budget": 0.54, "metric": 0.16954518220520104, "metric_std": 0.007047202485096458, "actual_fraction": 0.5485352862849534, "actual_fraction_std": 0.010408853096952194}, {"budget": 0.55, "metric": 0.17186799137817907, "metric_std": 0.008221784319647207, "actual_fraction": 0.5575233022636484, "actual_fraction_std": 0.0065967575426014325}, {"budget": 0.56, "metric": 0.17481928787379591, "metric_std": 0.008993057733330026, "actual_fraction": 0.5690412782956059, "actual_fraction_std": 0.0049744427063963025}, {"budget": 0.57, "metric": 0.17762223050632292, "metric_std": 0.010004556952135672, "actual_fraction": 0.579693741677763, "actual_fraction_std": 0.006194786131979529}, {"budget": 0.58, "metric": 0.17836379455465967, "metric_std": 0.010060962369345084, "actual_fraction": 0.5886817576564581, "actual_fraction_std": 0.008502052079692209}, {"budget": 0.59, "metric": 0.18112191245860995, "metric_std": 0.010617677563317602, "actual_fraction": 0.5992010652463382, "actual_fraction_std": 0.008437943978453146}, {"budget": 0.6, "metric": 0.18250823413298106, "metric_std": 0.009560059809112292, "actual_fraction": 0.6081225033288947, "actual_fraction_std": 0.008880727712486474}, {"budget": 0.61, "metric": 0.18300142935388689, "metric_std": 0.01059965089981378, "actual_fraction": 0.6192410119840213, "actual_fraction_std": 0.006827887041671611}, {"budget": 0.62, "metric": 0.1834949711105765, "metric_std": 0.01146229149031108, "actual_fraction": 0.6271637816245007, "actual_fraction_std": 0.006645294533181473}, {"budget": 0.63, "metric": 0.1859873358646193, "metric_std": 0.010777763361899053, "actual_fraction": 0.6394141145139813, "actual_fraction_std": 0.008280186707571796}, {"budget": 0.64, "metric": 0.1876834569858839, "metric_std": 0.00990625208163946, "actual_fraction": 0.6501997336884153, "actual_fraction_std": 0.005291791113231514}, {"budget": 0.65, "metric": 0.19001208817025206, "metric_std": 0.009606679310869141, "actual_fraction": 0.6615845539280959, "actual_fraction_std": 0.006292403729170472}, {"budget": 0.66, "metric": 0.19459585177403277, "metric_std": 0.007465974927105992, "actual_fraction": 0.6748335552596538, "actual_fraction_std": 0.005724344124820708}, {"budget": 0.67, "metric": 0.19771095282623058, "metric_std": 0.007446279381589621, "actual_fraction": 0.6855525965379494, "actual_fraction_std": 0.007865636742716169}, {"budget": 0.68, "metric": 0.20016490491714514, "metric_std": 0.006750413273861206, "actual_fraction": 0.6946737683089215, "actual_fraction_std": 0.007055372458982144}, {"budget": 0.69, "metric": 0.20291165354164126, "metric_std": 0.006217115559441101, "actual_fraction": 0.7053262316910784, "actual_fraction_std": 0.005034231166080841}, {"budget": 0.7, "metric": 0.20866714989110857, "metric_std": 0.007467025405551712, "actual_fraction": 0.7180426098535286, "actual_fraction_std": 0.0057417404496206545}, {"budget": 0.71, "metric": 0.2107541131338393, "metric_std": 0.007169756695696881, "actual_fraction": 0.7247669773635154, "actual_fraction_std": 0.007347738834259072}, {"budget": 0.72, "metric": 0.21302553293414012, "metric_std": 0.006997892247103818, "actual_fraction": 0.7350199733688416, "actual_fraction_std": 0.005119359686827426}, {"budget": 0.73, "metric": 0.21565551772110764, "metric_std": 0.007039994121207512, "actual_fraction": 0.7426098535286284, "actual_fraction_std": 0.004285103596546181}, {"budget": 0.74, "metric": 0.21953761946600697, "metric_std": 0.003665036555025599, "actual_fraction": 0.7507323568575233, "actual_fraction_std": 0.002299107410097467}, {"budget": 0.75, "metric": 0.22260023150234573, "metric_std": 0.002016228035808884, "actual_fraction": 0.7600532623169108, "actual_fraction_std": 0.003962643446059382}, {"budget": 0.76, "metric": 0.22399178136553083, "metric_std": 0.0016809636454170915, "actual_fraction": 0.7687749667110519, "actual_fraction_std": 0.0032497243307231956}, {"budget": 0.77, "metric": 0.2288907735781051, "metric_std": 0.0006133462751710848, "actual_fraction": 0.7794940079893474, "actual_fraction_std": 0.002936228637975953}, {"budget": 0.78, "metric": 0.2302230431773146, "metric_std": 0.00025312937906506053, "actual_fraction": 0.7920772303595206, "actual_fraction_std": 0.0024977804903201215}, {"budget": 0.79, "metric": 0.23101275856257222, "metric_std": 0.0020847997087421635, "actual_fraction": 0.8043941411451397, "actual_fraction_std": 0.002606335436892587}, {"budget": 0.8, "metric": 0.23078757307729245, "metric_std": 0.002193810569480955, "actual_fraction": 0.8145805592543274, "actual_fraction_std": 0.005289696598558855}, {"budget": 0.81, "metric": 0.2305294242150834, "metric_std": 0.0017167350022139378, "actual_fraction": 0.8228362183754993, "actual_fraction_std": 0.0044500239455262285}, {"budget": 0.82, "metric": 0.23040338701440236, "metric_std": 0.001626774131819025, "actual_fraction": 0.8311584553928096, "actual_fraction_std": 0.004774382587032527}, {"budget": 0.83, "metric": 0.23031803942480264, "metric_std": 0.0016090421197902007, "actual_fraction": 0.8400133155792278, "actual_fraction_std": 0.004723043647839221}, {"budget": 0.84, "metric": 0.2291557682601603, "metric_std": 0.0014644314714603336, "actual_fraction": 0.8524633821571237, "actual_fraction_std": 0.0015327382733982934}, {"budget": 0.85, "metric": 0.2285375508395297, "metric_std": 0.0014371417138440538, "actual_fraction": 0.8599866844207723, "actual_fraction_std": 0.002595684246287368}, {"budget": 0.86, "metric": 0.2279300866828322, "metric_std": 0.001294647899099488, "actual_fraction": 0.8676431424766978, "actual_fraction_std": 0.0028790610266897013}, {"budget": 0.87, "metric": 0.22740109084727797, "metric_std": 0.0007355520370207059, "actual_fraction": 0.8729027962716378, "actual_fraction_std": 0.003323897714079001}, {"budget": 0.88, "metric": 0.22701716950085365, "metric_std": 0.0007804909945321954, "actual_fraction": 0.8795605858854861, "actual_fraction_std": 0.003580690228901393}, {"budget": 0.89, "metric": 0.22648218866173186, "metric_std": 0.0006096237923356348, "actual_fraction": 0.890612516644474, "actual_fraction_std": 0.0019065008073604066}, {"budget": 0.9, "metric": 0.22570081525201866, "metric_std": 0.0007796775154898412, "actual_fraction": 0.9012649800266311, "actual_fraction_std": 0.004394897647021059}, {"budget": 0.91, "metric": 0.22529674743385839, "metric_std": 0.0008165689681813244, "actual_fraction": 0.9108521970705725, "actual_fraction_std": 0.0038347365520198288}, {"budget": 0.92, "metric": 0.22511393509248548, "metric_std": 0.0004569859794330817, "actual_fraction": 0.9204394141145139, "actual_fraction_std": 0.002257766305967126}, {"budget": 0.93, "metric": 0.22471389665202426, "metric_std": 0.0005576280205415703, "actual_fraction": 0.933422103861518, "actual_fraction_std": 0.003987732540183395}, {"budget": 0.94, "metric": 0.22437990038501057, "metric_std": 0.0005608476145369537, "actual_fraction": 0.9438082556591212, "actual_fraction_std": 0.003791142071240467}, {"budget": 0.95, "metric": 0.22439118975755679, "metric_std": 0.0006733270811850856, "actual_fraction": 0.9547270306258323, "actual_fraction_std": 0.003824608952422057}, {"budget": 0.96, "metric": 0.22404258530862614, "metric_std": 0.0005786627820072384, "actual_fraction": 0.9623834886817576, "actual_fraction_std": 0.0032102698938059403}, {"budget": 0.97, "metric": 0.22417998022789315, "metric_std": 0.0006156366440683307, "actual_fraction": 0.9705725699067909, "actual_fraction_std": 0.002964399049386581}, {"budget": 0.98, "metric": 0.2239736232579181, "metric_std": 0.0005594089907606738, "actual_fraction": 0.9816910785619175, "actual_fraction_std": 0.0024911167688242233}, {"budget": 0.99, "metric": 0.22376589550334983, "metric_std": 0.000509807517637664, "actual_fraction": 0.9916111850865512, "actual_fraction_std": 0.0016544305778680825}, {"budget": 1.0, "metric": 0.22361115727328068, "metric_std": 3.1031676915590914e-17, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 1391, "rescuable": 786, "harmful": 475, "both_wrong": 152, "unit": "event", "total": 2804}, "test": {"both_correct": 6060, "rescuable": 3314, "harmful": 1945, "both_wrong": 697, "unit": "event", "total": 12016}}, "small": {"validation": 0.07641657905799261, "test": 0.07887397871043929}, "large": {"validation": 0.20666016863629594, "test": 0.22361115727328065}, "learned_minus_confidence_test": 0.004381986058246051, "learned_minus_small_test": 0.14911916462108743, "learned_minus_large_test": 0.004381986058246051, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_rtdetr_x_fp32_MiniCPM"}, {"key": ["construction", "rtdetr_x_fp32", "Qwen3.8"], "expert_display": "RT-DETR-X", "run": "outputs/full_budget_20260913/runs/full_rtdetr_x_fp32_Qwen3.8_s{2026..2030}", "validation_rows": 701, "test_rows": 3004, "confidence_rule": {"column": "max_score", "columns": ["detection_count", "has_detections", "max_score", "mean_score", "detection_count_valid", "has_detections_valid", "max_score_valid", "mean_score_valid"], "source": "/mnt/HithinkOmniSSD/user_workspace/caisihang/research/ICASSP_ModelCollaboration/outputs/router_exploration_25pct_20260910/datasets/rtdetr_x_fp32/completion.json", "missing": "routes first, score=1; valid score=-confidence"}, "methods": {"confidence": {"selected_budget_index": 100, "validation_area_0_25": 0.10053453664183931, "validation_point": {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.25448626222520154, "rescued": 815, "harmed": 185, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 1681, "selected_both_wrong": 123}, "test_point": {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.27001009067886245, "rescued": 3461, "harmed": 676, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 7329, "selected_both_wrong": 550}, "validation_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.07641657905799261, "rescued": 0, "harmed": 0, "missed_rescue": 815, "avoided_harm": 185, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.021043073385953903, "operator": "ge", "actual_fraction": 0.009985734664764621, "metric": 0.07641657905799261, "rescued": 0, "harmed": 1, "missed_rescue": 815, "avoided_harm": 184, "selected_both_correct": 27, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -0.022327054291963577, "operator": "ge", "actual_fraction": 0.019971469329529243, "metric": 0.07641657905799261, "rescued": 0, "harmed": 2, "missed_rescue": 815, "avoided_harm": 183, "selected_both_correct": 54, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -0.024909960106015205, "operator": "ge", "actual_fraction": 0.029957203994293864, "metric": 0.07641657905799261, "rescued": 0, "harmed": 5, "missed_rescue": 815, "avoided_harm": 180, "selected_both_correct": 79, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": -0.027814723551273346, "operator": "ge", "actual_fraction": 0.039942938659058486, "metric": 0.07641657905799261, "rescued": 0, "harmed": 5, "missed_rescue": 815, "avoided_harm": 180, "selected_both_correct": 107, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": -0.028914429247379303, "operator": "ge", "actual_fraction": 0.04992867332382311, "metric": 0.10273236853167682, "rescued": 1, "harmed": 5, "missed_rescue": 814, "avoided_harm": 180, "selected_both_correct": 134, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": -0.030170045793056488, "operator": "ge", "actual_fraction": 0.05991440798858773, "metric": 0.10349184930294494, "rescued": 2, "harmed": 6, "missed_rescue": 813, "avoided_harm": 179, "selected_both_correct": 160, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": -0.03123854100704193, "operator": "ge", "actual_fraction": 0.06990014265335236, "metric": 0.10193024779448791, "rescued": 2, "harmed": 10, "missed_rescue": 813, "avoided_harm": 175, "selected_both_correct": 184, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": -0.03410686179995537, "operator": "ge", "actual_fraction": 0.07988587731811697, "metric": 0.10070576146899149, "rescued": 2, "harmed": 12, "missed_rescue": 813, "avoided_harm": 173, "selected_both_correct": 210, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": -0.03476816788315773, "operator": "ge", "actual_fraction": 0.0898716119828816, "metric": 0.09863536809425029, "rescued": 2, "harmed": 14, "missed_rescue": 813, "avoided_harm": 171, "selected_both_correct": 236, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": -0.0365506075322628, "operator": "ge", "actual_fraction": 0.09985734664764621, "metric": 0.09772957099280101, "rescued": 2, "harmed": 15, "missed_rescue": 813, "avoided_harm": 170, "selected_both_correct": 263, "selected_both_wrong": 0}, {"budget": 0.11, "threshold": -0.03818900138139725, "operator": "ge", "actual_fraction": 0.10984308131241084, "metric": 0.09678681181213102, "rescued": 2, "harmed": 17, "missed_rescue": 813, "avoided_harm": 168, "selected_both_correct": 289, "selected_both_wrong": 0}, {"budget": 0.12, "threshold": -0.039673544466495514, "operator": "ge", "actual_fraction": 0.11982881597717546, "metric": 0.0951962932074597, "rescued": 2, "harmed": 20, "missed_rescue": 813, "avoided_harm": 165, "selected_both_correct": 314, "selected_both_wrong": 0}, {"budget": 0.13, "threshold": -0.04085410386323929, "operator": "ge", "actual_fraction": 0.12981455064194009, "metric": 0.09473838918283363, "rescued": 2, "harmed": 25, "missed_rescue": 813, "avoided_harm": 160, "selected_both_correct": 337, "selected_both_wrong": 0}, {"budget": 0.14, "threshold": -0.04204101860523224, "operator": "ge", "actual_fraction": 0.1398002853067047, "metric": 0.09404328215803626, "rescued": 2, "harmed": 27, "missed_rescue": 813, "avoided_harm": 158, "selected_both_correct": 363, "selected_both_wrong": 0}, {"budget": 0.15, "threshold": -0.04278091713786125, "operator": "ge", "actual_fraction": 0.14978601997146934, "metric": 0.10930560071560966, "rescued": 3, "harmed": 30, "missed_rescue": 812, "avoided_harm": 155, "selected_both_correct": 386, "selected_both_wrong": 1}, {"budget": 0.16, "threshold": -0.04322759434580803, "operator": "ge", "actual_fraction": 0.15977175463623394, "metric": 0.10819687589305238, "rescued": 3, "harmed": 32, "missed_rescue": 812, "avoided_harm": 153, "selected_both_correct": 412, "selected_both_wrong": 1}, {"budget": 0.17, "threshold": -0.04394488036632538, "operator": "ge", "actual_fraction": 0.16975748930099857, "metric": 0.10718881137692335, "rescued": 3, "harmed": 33, "missed_rescue": 812, "avoided_harm": 152, "selected_both_correct": 439, "selected_both_wrong": 1}, {"budget": 0.18, "threshold": -0.04530126228928566, "operator": "ge", "actual_fraction": 0.1797432239657632, "metric": 0.106004048568, "rescued": 3, "harmed": 37, "missed_rescue": 812, "avoided_harm": 148, "selected_both_correct": 463, "selected_both_wrong": 1}, {"budget": 0.19, "threshold": -0.04661082476377487, "operator": "ge", "actual_fraction": 0.18972895863052783, "metric": 0.106004048568, "rescued": 3, "harmed": 37, "missed_rescue": 812, "avoided_harm": 148, "selected_both_correct": 491, "selected_both_wrong": 1}, {"budget": 0.2, "threshold": -0.04730507358908653, "operator": "ge", "actual_fraction": 0.19971469329529243, "metric": 0.10416681691465626, "rescued": 3, "harmed": 40, "missed_rescue": 812, "avoided_harm": 145, "selected_both_correct": 516, "selected_both_wrong": 1}, {"budget": 0.21, "threshold": -0.048246145248413086, "operator": "ge", "actual_fraction": 0.20970042796005706, "metric": 0.10337316612100547, "rescued": 3, "harmed": 41, "missed_rescue": 812, "avoided_harm": 144, "selected_both_correct": 543, "selected_both_wrong": 1}, {"budget": 0.22, "threshold": -0.04900278151035309, "operator": "ge", "actual_fraction": 0.21968616262482168, "metric": 0.10251715413933904, "rescued": 3, "harmed": 43, "missed_rescue": 812, "avoided_harm": 142, "selected_both_correct": 569, "selected_both_wrong": 1}, {"budget": 0.23, "threshold": -0.050042688846588135, "operator": "ge", "actual_fraction": 0.2296718972895863, "metric": 0.10180591658599622, "rescued": 3, "harmed": 44, "missed_rescue": 812, "avoided_harm": 141, "selected_both_correct": 595, "selected_both_wrong": 2}, {"budget": 0.24, "threshold": -0.050901688635349274, "operator": "ge", "actual_fraction": 0.2396576319543509, "metric": 0.10179370893656609, "rescued": 9, "harmed": 48, "missed_rescue": 806, "avoided_harm": 137, "selected_both_correct": 612, "selected_both_wrong": 3}, {"budget": 0.25, "threshold": -0.05161922052502632, "operator": "ge", "actual_fraction": 0.24964336661911554, "metric": 0.10106007305091304, "rescued": 16, "harmed": 50, "missed_rescue": 799, "avoided_harm": 135, "selected_both_correct": 631, "selected_both_wrong": 3}, {"budget": 0.26, "threshold": -0.05245445668697357, "operator": "ge", "actual_fraction": 0.25962910128388017, "metric": 0.10034763622704679, "rescued": 22, "harmed": 52, "missed_rescue": 793, "avoided_harm": 133, "selected_both_correct": 650, "selected_both_wrong": 4}, {"budget": 0.27, "threshold": -0.053079016506671906, "operator": "ge", "actual_fraction": 0.26961483594864477, "metric": 0.10038369304556356, "rescued": 30, "harmed": 53, "missed_rescue": 785, "avoided_harm": 132, "selected_both_correct": 669, "selected_both_wrong": 4}, {"budget": 0.28, "threshold": -0.053604189306497574, "operator": "ge", "actual_fraction": 0.2796005706134094, "metric": 0.10087055535678022, "rescued": 38, "harmed": 53, "missed_rescue": 777, "avoided_harm": 132, "selected_both_correct": 686, "selected_both_wrong": 7}, {"budget": 0.29, "threshold": -0.05505518987774849, "operator": "ge", "actual_fraction": 0.289586305278174, "metric": 0.10109647230706402, "rescued": 48, "harmed": 54, "missed_rescue": 767, "avoided_harm": 131, "selected_both_correct": 703, "selected_both_wrong": 7}, {"budget": 0.3, "threshold": -0.05605518817901611, "operator": "ge", "actual_fraction": 0.2995720399429387, "metric": 0.10072202785599546, "rescued": 54, "harmed": 57, "missed_rescue": 761, "avoided_harm": 128, "selected_both_correct": 721, "selected_both_wrong": 8}, {"budget": 0.31, "threshold": -0.056986160576343536, "operator": "ge", "actual_fraction": 0.3095577746077033, "metric": 0.10144063504785386, "rescued": 62, "harmed": 60, "missed_rescue": 753, "avoided_harm": 125, "selected_both_correct": 737, "selected_both_wrong": 9}, {"budget": 0.32, "threshold": -0.0582931749522686, "operator": "ge", "actual_fraction": 0.3195435092724679, "metric": 0.1016168805568569, "rescued": 70, "harmed": 61, "missed_rescue": 745, "avoided_harm": 124, "selected_both_correct": 756, "selected_both_wrong": 9}, {"budget": 0.33, "threshold": -0.0596650205552578, "operator": "ge", "actual_fraction": 0.32952924393723254, "metric": 0.10207649449543332, "rescued": 76, "harmed": 61, "missed_rescue": 739, "avoided_harm": 124, "selected_both_correct": 776, "selected_both_wrong": 11}, {"budget": 0.34, "threshold": -0.060549296438694, "operator": "ge", "actual_fraction": 0.33951497860199714, "metric": 0.10147122890331242, "rescued": 87, "harmed": 65, "missed_rescue": 728, "avoided_harm": 120, "selected_both_correct": 788, "selected_both_wrong": 12}, {"budget": 0.35, "threshold": -0.06178095191717148, "operator": "ge", "actual_fraction": 0.34950071326676174, "metric": 0.11031878913558303, "rescued": 97, "harmed": 67, "missed_rescue": 718, "avoided_harm": 118, "selected_both_correct": 803, "selected_both_wrong": 13}, {"budget": 0.36, "threshold": -0.06328777223825455, "operator": "ge", "actual_fraction": 0.3594864479315264, "metric": 0.1108012937861455, "rescued": 109, "harmed": 68, "missed_rescue": 706, "avoided_harm": 117, "selected_both_correct": 818, "selected_both_wrong": 13}, {"budget": 0.37, "threshold": -0.0647548958659172, "operator": "ge", "actual_fraction": 0.369472182596291, "metric": 0.11116507183385692, "rescued": 119, "harmed": 69, "missed_rescue": 696, "avoided_harm": 116, "selected_both_correct": 834, "selected_both_wrong": 14}, {"budget": 0.38, "threshold": -0.06637481600046158, "operator": "ge", "actual_fraction": 0.37945791726105566, "metric": 0.11160958901594166, "rescued": 128, "harmed": 70, "missed_rescue": 687, "avoided_harm": 115, "selected_both_correct": 851, "selected_both_wrong": 15}, {"budget": 0.39, "threshold": -0.06774259358644485, "operator": "ge", "actual_fraction": 0.38944365192582026, "metric": 0.11234765239703698, "rescued": 137, "harmed": 70, "missed_rescue": 678, "avoided_harm": 115, "selected_both_correct": 867, "selected_both_wrong": 18}, {"budget": 0.4, "threshold": -0.06843380630016327, "operator": "ge", "actual_fraction": 0.39942938659058486, "metric": 0.11018425519727987, "rescued": 146, "harmed": 72, "missed_rescue": 669, "avoided_harm": 113, "selected_both_correct": 883, "selected_both_wrong": 19}, {"budget": 0.41, "threshold": -0.0694691464304924, "operator": "ge", "actual_fraction": 0.4094151212553495, "metric": 0.11067980295566503, "rescued": 154, "harmed": 76, "missed_rescue": 661, "avoided_harm": 109, "selected_both_correct": 897, "selected_both_wrong": 21}, {"budget": 0.42, "threshold": -0.07112707942724228, "operator": "ge", "actual_fraction": 0.4194008559201141, "metric": 0.11116873445502742, "rescued": 166, "harmed": 78, "missed_rescue": 649, "avoided_harm": 107, "selected_both_correct": 911, "selected_both_wrong": 21}, {"budget": 0.43, "threshold": -0.07192998379468918, "operator": "ge", "actual_fraction": 0.42938659058487877, "metric": 0.11166452557368921, "rescued": 175, "harmed": 82, "missed_rescue": 640, "avoided_harm": 103, "selected_both_correct": 925, "selected_both_wrong": 22}, {"budget": 0.44, "threshold": -0.07266208529472351, "operator": "ge", "actual_fraction": 0.43937232524964337, "metric": 0.11224389412088417, "rescued": 185, "harmed": 84, "missed_rescue": 630, "avoided_harm": 101, "selected_both_correct": 940, "selected_both_wrong": 23}, {"budget": 0.45, "threshold": -0.0744253620505333, "operator": "ge", "actual_fraction": 0.44935805991440797, "metric": 0.11275924414114559, "rescued": 194, "harmed": 85, "missed_rescue": 621, "avoided_harm": 100, "selected_both_correct": 955, "selected_both_wrong": 26}, {"budget": 0.46, "threshold": -0.07532940804958344, "operator": "ge", "actual_fraction": 0.4593437945791726, "metric": 0.11288099124724625, "rescued": 203, "harmed": 88, "missed_rescue": 612, "avoided_harm": 97, "selected_both_correct": 970, "selected_both_wrong": 27}, {"budget": 0.47, "threshold": -0.0770750343799591, "operator": "ge", "actual_fraction": 0.4693295292439372, "metric": 0.11312993997763046, "rescued": 218, "harmed": 91, "missed_rescue": 597, "avoided_harm": 94, "selected_both_correct": 979, "selected_both_wrong": 28}, {"budget": 0.48, "threshold": -0.07818037271499634, "operator": "ge", "actual_fraction": 0.4793152639087018, "metric": 0.11421587032830263, "rescued": 227, "harmed": 91, "missed_rescue": 588, "avoided_harm": 94, "selected_both_correct": 997, "selected_both_wrong": 29}, {"budget": 0.49, "threshold": -0.07998209446668625, "operator": "ge", "actual_fraction": 0.4893009985734665, "metric": 0.11231395556672395, "rescued": 233, "harmed": 95, "missed_rescue": 582, "avoided_harm": 90, "selected_both_correct": 1014, "selected_both_wrong": 30}, {"budget": 0.5, "threshold": -0.08124949783086777, "operator": "ge", "actual_fraction": 0.4992867332382311, "metric": 0.11317276185775925, "rescued": 242, "harmed": 95, "missed_rescue": 573, "avoided_harm": 90, "selected_both_correct": 1032, "selected_both_wrong": 31}, {"budget": 0.51, "threshold": -0.08398465067148209, "operator": "ge", "actual_fraction": 0.5092724679029957, "metric": 0.11397997643625071, "rescued": 255, "harmed": 98, "missed_rescue": 560, "avoided_harm": 87, "selected_both_correct": 1044, "selected_both_wrong": 31}, {"budget": 0.52, "threshold": -0.08537434041500092, "operator": "ge", "actual_fraction": 0.5192582025677603, "metric": 0.12321191882582982, "rescued": 266, "harmed": 101, "missed_rescue": 549, "avoided_harm": 84, "selected_both_correct": 1057, "selected_both_wrong": 32}, {"budget": 0.53, "threshold": -0.08664409816265106, "operator": "ge", "actual_fraction": 0.5292439372325249, "metric": 0.12377158600708722, "rescued": 276, "harmed": 103, "missed_rescue": 539, "avoided_harm": 82, "selected_both_correct": 1071, "selected_both_wrong": 34}, {"budget": 0.54, "threshold": -0.08777308464050293, "operator": "ge", "actual_fraction": 0.5392296718972895, "metric": 0.12471535828861256, "rescued": 288, "harmed": 105, "missed_rescue": 527, "avoided_harm": 80, "selected_both_correct": 1084, "selected_both_wrong": 35}, {"budget": 0.55, "threshold": -0.08980843424797058, "operator": "ge", "actual_fraction": 0.5492154065620543, "metric": 0.12627445993419065, "rescued": 301, "harmed": 106, "missed_rescue": 514, "avoided_harm": 79, "selected_both_correct": 1098, "selected_both_wrong": 35}, {"budget": 0.56, "threshold": -0.0906188115477562, "operator": "ge", "actual_fraction": 0.5592011412268189, "metric": 0.1336597715556373, "rescued": 314, "harmed": 107, "missed_rescue": 501, "avoided_harm": 78, "selected_both_correct": 1111, "selected_both_wrong": 36}, {"budget": 0.57, "threshold": -0.09160211682319641, "operator": "ge", "actual_fraction": 0.5691868758915835, "metric": 0.13507916719864918, "rescued": 325, "harmed": 108, "missed_rescue": 490, "avoided_harm": 77, "selected_both_correct": 1126, "selected_both_wrong": 37}, {"budget": 0.58, "threshold": -0.09283093363046646, "operator": "ge", "actual_fraction": 0.579172610556348, "metric": 0.13601889857898128, "rescued": 336, "harmed": 110, "missed_rescue": 479, "avoided_harm": 75, "selected_both_correct": 1140, "selected_both_wrong": 38}, {"budget": 0.59, "threshold": -0.0941123366355896, "operator": "ge", "actual_fraction": 0.5891583452211127, "metric": 0.13626938313219553, "rescued": 346, "harmed": 114, "missed_rescue": 469, "avoided_harm": 71, "selected_both_correct": 1153, "selected_both_wrong": 39}, {"budget": 0.6, "threshold": -0.09614869952201843, "operator": "ge", "actual_fraction": 0.5991440798858774, "metric": 0.13736913676837675, "rescued": 359, "harmed": 116, "missed_rescue": 456, "avoided_harm": 69, "selected_both_correct": 1165, "selected_both_wrong": 40}, {"budget": 0.61, "threshold": -0.09772664308547974, "operator": "ge", "actual_fraction": 0.609129814550642, "metric": 0.13874937058804088, "rescued": 371, "harmed": 117, "missed_rescue": 444, "avoided_harm": 68, "selected_both_correct": 1177, "selected_both_wrong": 43}, {"budget": 0.62, "threshold": -0.09911532700061798, "operator": "ge", "actual_fraction": 0.6191155492154066, "metric": 0.1401946427657629, "rescued": 383, "harmed": 118, "missed_rescue": 432, "avoided_harm": 67, "selected_both_correct": 1190, "selected_both_wrong": 45}, {"budget": 0.63, "threshold": -0.10007988661527634, "operator": "ge", "actual_fraction": 0.6291012838801712, "metric": 0.14163850172491793, "rescued": 395, "harmed": 119, "missed_rescue": 420, "avoided_harm": 66, "selected_both_correct": 1204, "selected_both_wrong": 46}, {"budget": 0.64, "threshold": -0.10177135467529297, "operator": "ge", "actual_fraction": 0.6390870185449358, "metric": 0.14631539139000474, "rescued": 409, "harmed": 124, "missed_rescue": 406, "avoided_harm": 61, "selected_both_correct": 1210, "selected_both_wrong": 49}, {"budget": 0.65, "threshold": -0.10395076125860214, "operator": "ge", "actual_fraction": 0.6490727532097005, "metric": 0.14751840006124584, "rescued": 418, "harmed": 126, "missed_rescue": 397, "avoided_harm": 59, "selected_both_correct": 1226, "selected_both_wrong": 50}, {"budget": 0.66, "threshold": -0.10524322092533112, "operator": "ge", "actual_fraction": 0.6590584878744651, "metric": 0.14849238248708915, "rescued": 430, "harmed": 129, "missed_rescue": 385, "avoided_harm": 56, "selected_both_correct": 1236, "selected_both_wrong": 53}, {"budget": 0.67, "threshold": -0.10689426958560944, "operator": "ge", "actual_fraction": 0.6690442225392297, "metric": 0.15045288951822525, "rescued": 444, "harmed": 130, "missed_rescue": 371, "avoided_harm": 55, "selected_both_correct": 1249, "selected_both_wrong": 53}, {"budget": 0.68, "threshold": -0.11014843732118607, "operator": "ge", "actual_fraction": 0.6790299572039943, "metric": 0.15143717933254464, "rescued": 455, "harmed": 132, "missed_rescue": 360, "avoided_harm": 53, "selected_both_correct": 1261, "selected_both_wrong": 56}, {"budget": 0.69, "threshold": -0.11117545515298843, "operator": "ge", "actual_fraction": 0.6890156918687589, "metric": 0.158399005658965, "rescued": 469, "harmed": 136, "missed_rescue": 346, "avoided_harm": 49, "selected_both_correct": 1269, "selected_both_wrong": 58}, {"budget": 0.7, "threshold": -0.11230025440454483, "operator": "ge", "actual_fraction": 0.6990014265335235, "metric": 0.15918455527939407, "rescued": 480, "harmed": 137, "missed_rescue": 335, "avoided_harm": 48, "selected_both_correct": 1283, "selected_both_wrong": 60}, {"budget": 0.71, "threshold": -0.11423606425523758, "operator": "ge", "actual_fraction": 0.7089871611982882, "metric": 0.16079826154850815, "rescued": 491, "harmed": 138, "missed_rescue": 324, "avoided_harm": 47, "selected_both_correct": 1297, "selected_both_wrong": 62}, {"budget": 0.72, "threshold": -0.11647249013185501, "operator": "ge", "actual_fraction": 0.7189728958630528, "metric": 0.16232381124917994, "rescued": 506, "harmed": 142, "missed_rescue": 309, "avoided_harm": 43, "selected_both_correct": 1305, "selected_both_wrong": 63}, {"budget": 0.73, "threshold": -0.11962417513132095, "operator": "ge", "actual_fraction": 0.7289586305278174, "metric": 0.16281139401042977, "rescued": 516, "harmed": 145, "missed_rescue": 299, "avoided_harm": 40, "selected_both_correct": 1315, "selected_both_wrong": 68}, {"budget": 0.74, "threshold": -0.12126146256923676, "operator": "ge", "actual_fraction": 0.738944365192582, "metric": 0.16441172693350847, "rescued": 527, "harmed": 146, "missed_rescue": 288, "avoided_harm": 39, "selected_both_correct": 1328, "selected_both_wrong": 71}, {"budget": 0.75, "threshold": -0.1230982318520546, "operator": "ge", "actual_fraction": 0.7489300998573466, "metric": 0.16630001788027923, "rescued": 538, "harmed": 146, "missed_rescue": 277, "avoided_harm": 39, "selected_both_correct": 1342, "selected_both_wrong": 74}, {"budget": 0.76, "threshold": -0.12517565488815308, "operator": "ge", "actual_fraction": 0.7589158345221113, "metric": 0.16644279851003654, "rescued": 549, "harmed": 149, "missed_rescue": 266, "avoided_harm": 36, "selected_both_correct": 1354, "selected_both_wrong": 76}, {"budget": 0.77, "threshold": -0.12628257274627686, "operator": "ge", "actual_fraction": 0.7689015691868759, "metric": 0.16746586450739254, "rescued": 556, "harmed": 155, "missed_rescue": 259, "avoided_harm": 30, "selected_both_correct": 1365, "selected_both_wrong": 80}, {"budget": 0.78, "threshold": -0.12975578010082245, "operator": "ge", "actual_fraction": 0.7788873038516405, "metric": 0.1703501656136907, "rescued": 568, "harmed": 156, "missed_rescue": 247, "avoided_harm": 29, "selected_both_correct": 1379, "selected_both_wrong": 81}, {"budget": 0.79, "threshold": -0.13229140639305115, "operator": "ge", "actual_fraction": 0.7888730385164051, "metric": 0.17215569220087368, "rescued": 577, "harmed": 157, "missed_rescue": 238, "avoided_harm": 28, "selected_both_correct": 1392, "selected_both_wrong": 86}, {"budget": 0.8, "threshold": -0.1373685747385025, "operator": "ge", "actual_fraction": 0.7988587731811697, "metric": 0.1747173129779689, "rescued": 591, "harmed": 160, "missed_rescue": 224, "avoided_harm": 25, "selected_both_correct": 1402, "selected_both_wrong": 87}, {"budget": 0.81, "threshold": -0.1407174915075302, "operator": "ge", "actual_fraction": 0.8088445078459344, "metric": 0.18313694998955, "rescued": 609, "harmed": 160, "missed_rescue": 206, "avoided_harm": 25, "selected_both_correct": 1411, "selected_both_wrong": 88}, {"budget": 0.82, "threshold": -0.1433146893978119, "operator": "ge", "actual_fraction": 0.818830242510699, "metric": 0.18621937310674674, "rescued": 623, "harmed": 162, "missed_rescue": 192, "avoided_harm": 23, "selected_both_correct": 1421, "selected_both_wrong": 90}, {"budget": 0.83, "threshold": -0.1457100659608841, "operator": "ge", "actual_fraction": 0.8288159771754636, "metric": 0.1886261591198746, "rescued": 632, "harmed": 163, "missed_rescue": 183, "avoided_harm": 22, "selected_both_correct": 1439, "selected_both_wrong": 90}, {"budget": 0.84, "threshold": -0.148710235953331, "operator": "ge", "actual_fraction": 0.8388017118402282, "metric": 0.19060328592765763, "rescued": 642, "harmed": 167, "missed_rescue": 173, "avoided_harm": 18, "selected_both_correct": 1451, "selected_both_wrong": 92}, {"budget": 0.85, "threshold": -0.15291842818260193, "operator": "ge", "actual_fraction": 0.8487874465049928, "metric": 0.1979618629476584, "rescued": 658, "harmed": 169, "missed_rescue": 157, "avoided_harm": 16, "selected_both_correct": 1460, "selected_both_wrong": 93}, {"budget": 0.86, "threshold": -0.15889839828014374, "operator": "ge", "actual_fraction": 0.8587731811697575, "metric": 0.2003153021893066, "rescued": 667, "harmed": 170, "missed_rescue": 148, "avoided_harm": 15, "selected_both_correct": 1476, "selected_both_wrong": 95}, {"budget": 0.87, "threshold": -0.16629797220230103, "operator": "ge", "actual_fraction": 0.8687589158345221, "metric": 0.20401478964488712, "rescued": 678, "harmed": 172, "missed_rescue": 137, "avoided_harm": 13, "selected_both_correct": 1491, "selected_both_wrong": 95}, {"budget": 0.88, "threshold": -0.17295491695404053, "operator": "ge", "actual_fraction": 0.8787446504992867, "metric": 0.20519541918721101, "rescued": 687, "harmed": 173, "missed_rescue": 128, "avoided_harm": 12, "selected_both_correct": 1507, "selected_both_wrong": 97}, {"budget": 0.89, "threshold": -0.18053314089775085, "operator": "ge", "actual_fraction": 0.8887303851640513, "metric": 0.20764101752605946, "rescued": 695, "harmed": 175, "missed_rescue": 120, "avoided_harm": 10, "selected_both_correct": 1522, "selected_both_wrong": 100}, {"budget": 0.9, "threshold": -0.19442276656627655, "operator": "ge", "actual_fraction": 0.8987161198288159, "metric": 0.2119684936431676, "rescued": 709, "harmed": 175, "missed_rescue": 106, "avoided_harm": 10, "selected_both_correct": 1535, "selected_both_wrong": 101}, {"budget": 0.91, "threshold": -0.20900504291057587, "operator": "ge", "actual_fraction": 0.9087018544935807, "metric": 0.21605348977753452, "rescued": 719, "harmed": 175, "missed_rescue": 96, "avoided_harm": 10, "selected_both_correct": 1551, "selected_both_wrong": 103}, {"budget": 0.92, "threshold": -0.22383525967597961, "operator": "ge", "actual_fraction": 0.9186875891583453, "metric": 0.2203498969356328, "rescued": 731, "harmed": 175, "missed_rescue": 84, "avoided_harm": 10, "selected_both_correct": 1564, "selected_both_wrong": 106}, {"budget": 0.93, "threshold": -0.25952714681625366, "operator": "ge", "actual_fraction": 0.9286733238231099, "metric": 0.22073473473473473, "rescued": 741, "harmed": 178, "missed_rescue": 74, "avoided_harm": 7, "selected_both_correct": 1577, "selected_both_wrong": 108}, {"budget": 0.94, "threshold": -0.2804492115974426, "operator": "ge", "actual_fraction": 0.9386590584878745, "metric": 0.21785099794688836, "rescued": 751, "harmed": 179, "missed_rescue": 64, "avoided_harm": 6, "selected_both_correct": 1590, "selected_both_wrong": 112}, {"budget": 0.95, "threshold": -0.2984963357448578, "operator": "ge", "actual_fraction": 0.948644793152639, "metric": 0.22594140701088306, "rescued": 764, "harmed": 179, "missed_rescue": 51, "avoided_harm": 6, "selected_both_correct": 1603, "selected_both_wrong": 114}, {"budget": 0.96, "threshold": -0.32223379611968994, "operator": "ge", "actual_fraction": 0.9586305278174037, "metric": 0.23296274186615876, "rescued": 774, "harmed": 179, "missed_rescue": 41, "avoided_harm": 6, "selected_both_correct": 1619, "selected_both_wrong": 116}, {"budget": 0.97, "threshold": -0.36503666639328003, "operator": "ge", "actual_fraction": 0.9686162624821684, "metric": 0.23681731712393103, "rescued": 784, "harmed": 180, "missed_rescue": 31, "avoided_harm": 5, "selected_both_correct": 1635, "selected_both_wrong": 117}, {"budget": 0.98, "threshold": -0.4221048653125763, "operator": "ge", "actual_fraction": 0.978601997146933, "metric": 0.24211204189132882, "rescued": 796, "harmed": 182, "missed_rescue": 19, "avoided_harm": 3, "selected_both_correct": 1649, "selected_both_wrong": 117}, {"budget": 0.99, "threshold": -0.44162338972091675, "operator": "ge", "actual_fraction": 0.9885877318116976, "metric": 0.24862274646934207, "rescued": 805, "harmed": 184, "missed_rescue": 10, "avoided_harm": 1, "selected_both_correct": 1662, "selected_both_wrong": 121}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.25448626222520154, "rescued": 815, "harmed": 185, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 1681, "selected_both_wrong": 123}], "test_curve": [{"budget": 0.0, "threshold": null, "operator": "never", "actual_fraction": 0.0, "metric": 0.07887397871043929, "rescued": 0, "harmed": 0, "missed_rescue": 3461, "avoided_harm": 676, "selected_both_correct": 0, "selected_both_wrong": 0}, {"budget": 0.01, "threshold": -0.021043073385953903, "operator": "ge", "actual_fraction": 0.007989347536617843, "metric": 0.07887397871043929, "rescued": 0, "harmed": 5, "missed_rescue": 3461, "avoided_harm": 671, "selected_both_correct": 91, "selected_both_wrong": 0}, {"budget": 0.02, "threshold": -0.022327054291963577, "operator": "ge", "actual_fraction": 0.016644474034620507, "metric": 0.07904050790892239, "rescued": 1, "harmed": 8, "missed_rescue": 3460, "avoided_harm": 668, "selected_both_correct": 191, "selected_both_wrong": 0}, {"budget": 0.03, "threshold": -0.024909960106015205, "operator": "ge", "actual_fraction": 0.0296271637816245, "metric": 0.07904050790892239, "rescued": 1, "harmed": 11, "missed_rescue": 3460, "avoided_harm": 665, "selected_both_correct": 344, "selected_both_wrong": 0}, {"budget": 0.04, "threshold": -0.027814723551273346, "operator": "ge", "actual_fraction": 0.046937416777629824, "metric": 0.07894575740459331, "rescued": 1, "harmed": 19, "missed_rescue": 3460, "avoided_harm": 657, "selected_both_correct": 544, "selected_both_wrong": 0}, {"budget": 0.05, "threshold": -0.028914429247379303, "operator": "ge", "actual_fraction": 0.05326231691078562, "metric": 0.07941709585555166, "rescued": 2, "harmed": 22, "missed_rescue": 3459, "avoided_harm": 654, "selected_both_correct": 616, "selected_both_wrong": 0}, {"budget": 0.06, "threshold": -0.030170045793056488, "operator": "ge", "actual_fraction": 0.05992010652463382, "metric": 0.07940750674336808, "rescued": 2, "harmed": 23, "missed_rescue": 3459, "avoided_harm": 653, "selected_both_correct": 695, "selected_both_wrong": 0}, {"budget": 0.07, "threshold": -0.03123854100704193, "operator": "ge", "actual_fraction": 0.06724367509986684, "metric": 0.09221541101182128, "rescued": 5, "harmed": 28, "missed_rescue": 3456, "avoided_harm": 648, "selected_both_correct": 775, "selected_both_wrong": 0}, {"budget": 0.08, "threshold": -0.03410686179995537, "operator": "ge", "actual_fraction": 0.08721704394141146, "metric": 0.09167716576535306, "rescued": 6, "harmed": 39, "missed_rescue": 3455, "avoided_harm": 637, "selected_both_correct": 1003, "selected_both_wrong": 0}, {"budget": 0.09, "threshold": -0.03476816788315773, "operator": "ge", "actual_fraction": 0.09287616511318242, "metric": 0.09085537954855229, "rescued": 6, "harmed": 46, "missed_rescue": 3455, "avoided_harm": 630, "selected_both_correct": 1064, "selected_both_wrong": 0}, {"budget": 0.1, "threshold": -0.0365506075322628, "operator": "ge", "actual_fraction": 0.10785619174434088, "metric": 0.09006366890142659, "rescued": 6, "harmed": 56, "missed_rescue": 3455, "avoided_harm": 620, "selected_both_correct": 1233, "selected_both_wrong": 1}, {"budget": 0.11, "threshold": -0.03818900138139725, "operator": "ge", "actual_fraction": 0.11784287616511319, "metric": 0.09042296916775287, "rescued": 8, "harmed": 58, "missed_rescue": 3453, "avoided_harm": 618, "selected_both_correct": 1349, "selected_both_wrong": 1}, {"budget": 0.12, "threshold": -0.039673544466495514, "operator": "ge", "actual_fraction": 0.12716378162450068, "metric": 0.08993827377942153, "rescued": 8, "harmed": 65, "missed_rescue": 3453, "avoided_harm": 611, "selected_both_correct": 1454, "selected_both_wrong": 1}, {"budget": 0.13, "threshold": -0.04085410386323929, "operator": "ge", "actual_fraction": 0.13648468708388814, "metric": 0.08955740758160691, "rescued": 8, "harmed": 73, "missed_rescue": 3453, "avoided_harm": 603, "selected_both_correct": 1558, "selected_both_wrong": 1}, {"budget": 0.14, "threshold": -0.04204101860523224, "operator": "ge", "actual_fraction": 0.14613848202396804, "metric": 0.0893664971618984, "rescued": 8, "harmed": 76, "missed_rescue": 3453, "avoided_harm": 600, "selected_both_correct": 1671, "selected_both_wrong": 1}, {"budget": 0.15, "threshold": -0.04278091713786125, "operator": "ge", "actual_fraction": 0.1514647137150466, "metric": 0.08915747670369788, "rescued": 8, "harmed": 82, "missed_rescue": 3453, "avoided_harm": 594, "selected_both_correct": 1729, "selected_both_wrong": 1}, {"budget": 0.16, "threshold": -0.04322759434580803, "operator": "ge", "actual_fraction": 0.15579227696404793, "metric": 0.08910079841343506, "rescued": 8, "harmed": 85, "missed_rescue": 3453, "avoided_harm": 591, "selected_both_correct": 1778, "selected_both_wrong": 1}, {"budget": 0.17, "threshold": -0.04394488036632538, "operator": "ge", "actual_fraction": 0.1617842876165113, "metric": 0.08889771271713626, "rescued": 8, "harmed": 89, "missed_rescue": 3453, "avoided_harm": 587, "selected_both_correct": 1846, "selected_both_wrong": 1}, {"budget": 0.18, "threshold": -0.04530126228928566, "operator": "ge", "actual_fraction": 0.17177097203728361, "metric": 0.08882240770994018, "rescued": 8, "harmed": 94, "missed_rescue": 3453, "avoided_harm": 582, "selected_both_correct": 1961, "selected_both_wrong": 1}, {"budget": 0.19, "threshold": -0.04661082476377487, "operator": "ge", "actual_fraction": 0.1810918774966711, "metric": 0.08900834067514127, "rescued": 9, "harmed": 101, "missed_rescue": 3452, "avoided_harm": 575, "selected_both_correct": 2065, "selected_both_wrong": 1}, {"budget": 0.2, "threshold": -0.04730507358908653, "operator": "ge", "actual_fraction": 0.1870838881491345, "metric": 0.09716527158339765, "rescued": 11, "harmed": 107, "missed_rescue": 3450, "avoided_harm": 569, "selected_both_correct": 2129, "selected_both_wrong": 1}, {"budget": 0.21, "threshold": -0.048246145248413086, "operator": "ge", "actual_fraction": 0.19640479360852198, "metric": 0.09733135745692614, "rescued": 12, "harmed": 112, "missed_rescue": 3449, "avoided_harm": 564, "selected_both_correct": 2234, "selected_both_wrong": 2}, {"budget": 0.22, "threshold": -0.04900278151035309, "operator": "ge", "actual_fraction": 0.20838881491344874, "metric": 0.09686185392269288, "rescued": 14, "harmed": 124, "missed_rescue": 3447, "avoided_harm": 552, "selected_both_correct": 2363, "selected_both_wrong": 3}, {"budget": 0.23, "threshold": -0.050042688846588135, "operator": "ge", "actual_fraction": 0.21471371504660453, "metric": 0.09673072357913137, "rescued": 14, "harmed": 125, "missed_rescue": 3447, "avoided_harm": 551, "selected_both_correct": 2438, "selected_both_wrong": 3}, {"budget": 0.24, "threshold": -0.050901688635349274, "operator": "ge", "actual_fraction": 0.22536617842876164, "metric": 0.10080716337703104, "rescued": 48, "harmed": 130, "missed_rescue": 3413, "avoided_harm": 546, "selected_both_correct": 2526, "selected_both_wrong": 4}, {"budget": 0.25, "threshold": -0.05161922052502632, "operator": "ge", "actual_fraction": 0.23135818908122505, "metric": 0.10058232286929361, "rescued": 65, "harmed": 134, "missed_rescue": 3396, "avoided_harm": 542, "selected_both_correct": 2575, "selected_both_wrong": 6}, {"budget": 0.26, "threshold": -0.05245445668697357, "operator": "ge", "actual_fraction": 0.237017310252996, "metric": 0.10078195333663684, "rescued": 81, "harmed": 136, "missed_rescue": 3380, "avoided_harm": 540, "selected_both_correct": 2624, "selected_both_wrong": 7}, {"budget": 0.27, "threshold": -0.053079016506671906, "operator": "ge", "actual_fraction": 0.24167776298268975, "metric": 0.10061353044193348, "rescued": 95, "harmed": 141, "missed_rescue": 3366, "avoided_harm": 535, "selected_both_correct": 2660, "selected_both_wrong": 8}, {"budget": 0.28, "threshold": -0.053604189306497574, "operator": "ge", "actual_fraction": 0.2470039946737683, "metric": 0.10058305058987213, "rescued": 111, "harmed": 144, "missed_rescue": 3350, "avoided_harm": 532, "selected_both_correct": 2705, "selected_both_wrong": 8}, {"budget": 0.29, "threshold": -0.05505518987774849, "operator": "ge", "actual_fraction": 0.2623169107856192, "metric": 0.10443538242821891, "rescued": 159, "harmed": 148, "missed_rescue": 3302, "avoided_harm": 528, "selected_both_correct": 2830, "selected_both_wrong": 15}, {"budget": 0.3, "threshold": -0.05605518817901611, "operator": "ge", "actual_fraction": 0.27463382157123833, "metric": 0.10759036763747168, "rescued": 204, "harmed": 158, "missed_rescue": 3257, "avoided_harm": 518, "selected_both_correct": 2918, "selected_both_wrong": 20}, {"budget": 0.31, "threshold": -0.056986160576343536, "operator": "ge", "actual_fraction": 0.28462050599201066, "metric": 0.10779091241357944, "rescued": 236, "harmed": 165, "missed_rescue": 3225, "avoided_harm": 511, "selected_both_correct": 2997, "selected_both_wrong": 22}, {"budget": 0.32, "threshold": -0.0582931749522686, "operator": "ge", "actual_fraction": 0.29427430093209056, "metric": 0.11066132714400925, "rescued": 268, "harmed": 172, "missed_rescue": 3193, "avoided_harm": 504, "selected_both_correct": 3068, "selected_both_wrong": 28}, {"budget": 0.33, "threshold": -0.0596650205552578, "operator": "ge", "actual_fraction": 0.3062583222370173, "metric": 0.11055610302472227, "rescued": 309, "harmed": 182, "missed_rescue": 3152, "avoided_harm": 494, "selected_both_correct": 3160, "selected_both_wrong": 29}, {"budget": 0.34, "threshold": -0.060549296438694, "operator": "ge", "actual_fraction": 0.3132490013315579, "metric": 0.1105675542193679, "rescued": 337, "harmed": 186, "missed_rescue": 3124, "avoided_harm": 490, "selected_both_correct": 3210, "selected_both_wrong": 31}, {"budget": 0.35, "threshold": -0.06178095191717148, "operator": "ge", "actual_fraction": 0.3189081225033289, "metric": 0.11000647533107741, "rescued": 355, "harmed": 194, "missed_rescue": 3106, "avoided_harm": 482, "selected_both_correct": 3250, "selected_both_wrong": 33}, {"budget": 0.36, "threshold": -0.06328777223825455, "operator": "ge", "actual_fraction": 0.3318908122503329, "metric": 0.11032470041071389, "rescued": 397, "harmed": 200, "missed_rescue": 3064, "avoided_harm": 476, "selected_both_correct": 3350, "selected_both_wrong": 41}, {"budget": 0.37, "threshold": -0.0647548958659172, "operator": "ge", "actual_fraction": 0.3418774966711052, "metric": 0.11048463654687253, "rescued": 437, "harmed": 206, "missed_rescue": 3024, "avoided_harm": 470, "selected_both_correct": 3424, "selected_both_wrong": 41}, {"budget": 0.38, "threshold": -0.06637481600046158, "operator": "ge", "actual_fraction": 0.35352862849533956, "metric": 0.11263903545823285, "rescued": 484, "harmed": 220, "missed_rescue": 2977, "avoided_harm": 456, "selected_both_correct": 3496, "selected_both_wrong": 48}, {"budget": 0.39, "threshold": -0.06774259358644485, "operator": "ge", "actual_fraction": 0.3641810918774967, "metric": 0.11519364722073223, "rescued": 527, "harmed": 227, "missed_rescue": 2934, "avoided_harm": 449, "selected_both_correct": 3570, "selected_both_wrong": 52}, {"budget": 0.4, "threshold": -0.06843380630016327, "operator": "ge", "actual_fraction": 0.3698402130492676, "metric": 0.11587206094412077, "rescued": 550, "harmed": 228, "missed_rescue": 2911, "avoided_harm": 448, "selected_both_correct": 3609, "selected_both_wrong": 57}, {"budget": 0.41, "threshold": -0.0694691464304924, "operator": "ge", "actual_fraction": 0.3801597869507324, "metric": 0.11650140520518415, "rescued": 593, "harmed": 240, "missed_rescue": 2868, "avoided_harm": 436, "selected_both_correct": 3672, "selected_both_wrong": 63}, {"budget": 0.42, "threshold": -0.07112707942724228, "operator": "ge", "actual_fraction": 0.3938082556591212, "metric": 0.11662726768380335, "rescued": 646, "harmed": 251, "missed_rescue": 2815, "avoided_harm": 425, "selected_both_correct": 3765, "selected_both_wrong": 70}, {"budget": 0.43, "threshold": -0.07192998379468918, "operator": "ge", "actual_fraction": 0.3988015978695073, "metric": 0.1170616446996618, "rescued": 667, "harmed": 252, "missed_rescue": 2794, "avoided_harm": 424, "selected_both_correct": 3802, "selected_both_wrong": 71}, {"budget": 0.44, "threshold": -0.07266208529472351, "operator": "ge", "actual_fraction": 0.40279627163781623, "metric": 0.1191343028439446, "rescued": 682, "harmed": 256, "missed_rescue": 2779, "avoided_harm": 420, "selected_both_correct": 3830, "selected_both_wrong": 72}, {"budget": 0.45, "threshold": -0.0744253620505333, "operator": "ge", "actual_fraction": 0.41711051930758986, "metric": 0.12016591353047154, "rescued": 738, "harmed": 265, "missed_rescue": 2723, "avoided_harm": 411, "selected_both_correct": 3927, "selected_both_wrong": 82}, {"budget": 0.46, "threshold": -0.07532940804958344, "operator": "ge", "actual_fraction": 0.42243675099866845, "metric": 0.12056162004139302, "rescued": 764, "harmed": 271, "missed_rescue": 2697, "avoided_harm": 405, "selected_both_correct": 3955, "selected_both_wrong": 86}, {"budget": 0.47, "threshold": -0.0770750343799591, "operator": "ge", "actual_fraction": 0.4380825565912117, "metric": 0.121709486901851, "rescued": 839, "harmed": 283, "missed_rescue": 2622, "avoided_harm": 393, "selected_both_correct": 4049, "selected_both_wrong": 93}, {"budget": 0.48, "threshold": -0.07818037271499634, "operator": "ge", "actual_fraction": 0.44573901464713717, "metric": 0.1221443001236399, "rescued": 873, "harmed": 288, "missed_rescue": 2588, "avoided_harm": 388, "selected_both_correct": 4097, "selected_both_wrong": 98}, {"budget": 0.49, "threshold": -0.07998209446668625, "operator": "ge", "actual_fraction": 0.45905459387483355, "metric": 0.1251607808901142, "rescued": 928, "harmed": 294, "missed_rescue": 2533, "avoided_harm": 382, "selected_both_correct": 4185, "selected_both_wrong": 109}, {"budget": 0.5, "threshold": -0.08124949783086777, "operator": "ge", "actual_fraction": 0.46937416777629826, "metric": 0.12551652022923962, "rescued": 974, "harmed": 309, "missed_rescue": 2487, "avoided_harm": 367, "selected_both_correct": 4244, "selected_both_wrong": 113}, {"budget": 0.51, "threshold": -0.08398465067148209, "operator": "ge", "actual_fraction": 0.49467376830892146, "metric": 0.12932775258830043, "rescued": 1093, "harmed": 334, "missed_rescue": 2368, "avoided_harm": 342, "selected_both_correct": 4389, "selected_both_wrong": 128}, {"budget": 0.52, "threshold": -0.08537434041500092, "operator": "ge", "actual_fraction": 0.5056591211717709, "metric": 0.13382418312702152, "rescued": 1145, "harmed": 340, "missed_rescue": 2316, "avoided_harm": 336, "selected_both_correct": 4457, "selected_both_wrong": 134}, {"budget": 0.53, "threshold": -0.08664409816265106, "operator": "ge", "actual_fraction": 0.5159786950732357, "metric": 0.1342986185420429, "rescued": 1190, "harmed": 350, "missed_rescue": 2271, "avoided_harm": 326, "selected_both_correct": 4522, "selected_both_wrong": 138}, {"budget": 0.54, "threshold": -0.08777308464050293, "operator": "ge", "actual_fraction": 0.524300932090546, "metric": 0.13511006926922334, "rescued": 1228, "harmed": 356, "missed_rescue": 2233, "avoided_harm": 320, "selected_both_correct": 4574, "selected_both_wrong": 142}, {"budget": 0.55, "threshold": -0.08980843424797058, "operator": "ge", "actual_fraction": 0.5386151797603196, "metric": 0.13883610114027645, "rescued": 1297, "harmed": 367, "missed_rescue": 2164, "avoided_harm": 309, "selected_both_correct": 4657, "selected_both_wrong": 151}, {"budget": 0.56, "threshold": -0.0906188115477562, "operator": "ge", "actual_fraction": 0.546271637816245, "metric": 0.13992298734058117, "rescued": 1332, "harmed": 375, "missed_rescue": 2129, "avoided_harm": 301, "selected_both_correct": 4702, "selected_both_wrong": 155}, {"budget": 0.57, "threshold": -0.09160211682319641, "operator": "ge", "actual_fraction": 0.5555925432756325, "metric": 0.14072696062294485, "rescued": 1379, "harmed": 383, "missed_rescue": 2082, "avoided_harm": 293, "selected_both_correct": 4751, "selected_both_wrong": 163}, {"budget": 0.58, "threshold": -0.09283093363046646, "operator": "ge", "actual_fraction": 0.568242343541944, "metric": 0.1472561682256289, "rescued": 1434, "harmed": 393, "missed_rescue": 2027, "avoided_harm": 283, "selected_both_correct": 4827, "selected_both_wrong": 174}, {"budget": 0.59, "threshold": -0.0941123366355896, "operator": "ge", "actual_fraction": 0.5778961384820239, "metric": 0.15078547139034337, "rescued": 1482, "harmed": 399, "missed_rescue": 1979, "avoided_harm": 277, "selected_both_correct": 4882, "selected_both_wrong": 181}, {"budget": 0.6, "threshold": -0.09614869952201843, "operator": "ge", "actual_fraction": 0.5878828229027963, "metric": 0.15176768468059784, "rescued": 1528, "harmed": 406, "missed_rescue": 1933, "avoided_harm": 270, "selected_both_correct": 4939, "selected_both_wrong": 191}, {"budget": 0.61, "threshold": -0.09772664308547974, "operator": "ge", "actual_fraction": 0.5975366178428761, "metric": 0.1532082936739425, "rescued": 1567, "harmed": 414, "missed_rescue": 1894, "avoided_harm": 262, "selected_both_correct": 4998, "selected_both_wrong": 201}, {"budget": 0.62, "threshold": -0.09911532700061798, "operator": "ge", "actual_fraction": 0.6081890812250332, "metric": 0.15424732264646662, "rescued": 1615, "harmed": 422, "missed_rescue": 1846, "avoided_harm": 254, "selected_both_correct": 5061, "selected_both_wrong": 210}, {"budget": 0.63, "threshold": -0.10007988661527634, "operator": "ge", "actual_fraction": 0.6161784287616512, "metric": 0.1548871102093026, "rescued": 1652, "harmed": 430, "missed_rescue": 1809, "avoided_harm": 246, "selected_both_correct": 5104, "selected_both_wrong": 218}, {"budget": 0.64, "threshold": -0.10177135467529297, "operator": "ge", "actual_fraction": 0.6278295605858855, "metric": 0.15779981350208042, "rescued": 1709, "harmed": 438, "missed_rescue": 1752, "avoided_harm": 238, "selected_both_correct": 5169, "selected_both_wrong": 228}, {"budget": 0.65, "threshold": -0.10395076125860214, "operator": "ge", "actual_fraction": 0.6451398135818908, "metric": 0.16144984235200888, "rescued": 1795, "harmed": 448, "missed_rescue": 1666, "avoided_harm": 228, "selected_both_correct": 5271, "selected_both_wrong": 238}, {"budget": 0.66, "threshold": -0.10524322092533112, "operator": "ge", "actual_fraction": 0.6491344873501997, "metric": 0.16469010323619201, "rescued": 1816, "harmed": 448, "missed_rescue": 1645, "avoided_harm": 228, "selected_both_correct": 5295, "selected_both_wrong": 241}, {"budget": 0.67, "threshold": -0.10689426958560944, "operator": "ge", "actual_fraction": 0.6601198402130493, "metric": 0.16600232483815708, "rescued": 1872, "harmed": 456, "missed_rescue": 1589, "avoided_harm": 220, "selected_both_correct": 5357, "selected_both_wrong": 247}, {"budget": 0.68, "threshold": -0.11014843732118607, "operator": "ge", "actual_fraction": 0.6754327563249002, "metric": 0.16941529572817854, "rescued": 1941, "harmed": 465, "missed_rescue": 1520, "avoided_harm": 211, "selected_both_correct": 5452, "selected_both_wrong": 258}, {"budget": 0.69, "threshold": -0.11117545515298843, "operator": "ge", "actual_fraction": 0.6790945406125166, "metric": 0.17007927366959905, "rescued": 1960, "harmed": 467, "missed_rescue": 1501, "avoided_harm": 209, "selected_both_correct": 5471, "selected_both_wrong": 262}, {"budget": 0.7, "threshold": -0.11230025440454483, "operator": "ge", "actual_fraction": 0.6870838881491345, "metric": 0.17172535263617786, "rescued": 2005, "harmed": 471, "missed_rescue": 1456, "avoided_harm": 205, "selected_both_correct": 5514, "selected_both_wrong": 266}, {"budget": 0.71, "threshold": -0.11423606425523758, "operator": "ge", "actual_fraction": 0.6960719041278296, "metric": 0.17427123861822869, "rescued": 2047, "harmed": 477, "missed_rescue": 1414, "avoided_harm": 199, "selected_both_correct": 5562, "selected_both_wrong": 278}, {"budget": 0.72, "threshold": -0.11647249013185501, "operator": "ge", "actual_fraction": 0.7053928095872171, "metric": 0.17661848328954455, "rescued": 2090, "harmed": 486, "missed_rescue": 1371, "avoided_harm": 190, "selected_both_correct": 5613, "selected_both_wrong": 287}, {"budget": 0.73, "threshold": -0.11962417513132095, "operator": "ge", "actual_fraction": 0.7193741677762983, "metric": 0.18093016254845928, "rescued": 2156, "harmed": 500, "missed_rescue": 1305, "avoided_harm": 176, "selected_both_correct": 5687, "selected_both_wrong": 301}, {"budget": 0.74, "threshold": -0.12126146256923676, "operator": "ge", "actual_fraction": 0.7250332889480693, "metric": 0.18214911344128, "rescued": 2186, "harmed": 504, "missed_rescue": 1275, "avoided_harm": 172, "selected_both_correct": 5719, "selected_both_wrong": 303}, {"budget": 0.75, "threshold": -0.1230982318520546, "operator": "ge", "actual_fraction": 0.7366844207723036, "metric": 0.18474146474609127, "rescued": 2239, "harmed": 510, "missed_rescue": 1222, "avoided_harm": 166, "selected_both_correct": 5790, "selected_both_wrong": 313}, {"budget": 0.76, "threshold": -0.12517565488815308, "operator": "ge", "actual_fraction": 0.7446737683089214, "metric": 0.1868238104649864, "rescued": 2278, "harmed": 513, "missed_rescue": 1183, "avoided_harm": 163, "selected_both_correct": 5841, "selected_both_wrong": 316}, {"budget": 0.77, "threshold": -0.12628257274627686, "operator": "ge", "actual_fraction": 0.7480026631158455, "metric": 0.18780967278670027, "rescued": 2296, "harmed": 513, "missed_rescue": 1165, "avoided_harm": 163, "selected_both_correct": 5861, "selected_both_wrong": 318}, {"budget": 0.78, "threshold": -0.12975578010082245, "operator": "ge", "actual_fraction": 0.7603195739014648, "metric": 0.19033940523013337, "rescued": 2354, "harmed": 518, "missed_rescue": 1107, "avoided_harm": 158, "selected_both_correct": 5931, "selected_both_wrong": 333}, {"budget": 0.79, "threshold": -0.13229140639305115, "operator": "ge", "actual_fraction": 0.7713049267643143, "metric": 0.19289795000814608, "rescued": 2407, "harmed": 524, "missed_rescue": 1054, "avoided_harm": 152, "selected_both_correct": 5996, "selected_both_wrong": 341}, {"budget": 0.8, "threshold": -0.1373685747385025, "operator": "ge", "actual_fraction": 0.7909454061251664, "metric": 0.1970331406058424, "rescued": 2510, "harmed": 548, "missed_rescue": 951, "avoided_harm": 128, "selected_both_correct": 6090, "selected_both_wrong": 356}, {"budget": 0.81, "threshold": -0.1407174915075302, "operator": "ge", "actual_fraction": 0.8012649800266312, "metric": 0.1991542632866029, "rescued": 2560, "harmed": 556, "missed_rescue": 901, "avoided_harm": 120, "selected_both_correct": 6144, "selected_both_wrong": 368}, {"budget": 0.82, "threshold": -0.1433146893978119, "operator": "ge", "actual_fraction": 0.8125832223701731, "metric": 0.2006018605686543, "rescued": 2613, "harmed": 569, "missed_rescue": 848, "avoided_harm": 107, "selected_both_correct": 6204, "selected_both_wrong": 378}, {"budget": 0.83, "threshold": -0.1457100659608841, "operator": "ge", "actual_fraction": 0.8199067909454061, "metric": 0.20192079535680077, "rescued": 2654, "harmed": 577, "missed_rescue": 807, "avoided_harm": 99, "selected_both_correct": 6238, "selected_both_wrong": 383}, {"budget": 0.84, "threshold": -0.148710235953331, "operator": "ge", "actual_fraction": 0.8242343541944075, "metric": 0.20317936971986447, "rescued": 2674, "harmed": 578, "missed_rescue": 787, "avoided_harm": 98, "selected_both_correct": 6264, "selected_both_wrong": 388}, {"budget": 0.85, "threshold": -0.15291842818260193, "operator": "ge", "actual_fraction": 0.8338881491344874, "metric": 0.2060155880710891, "rescued": 2727, "harmed": 591, "missed_rescue": 734, "avoided_harm": 85, "selected_both_correct": 6311, "selected_both_wrong": 391}, {"budget": 0.86, "threshold": -0.15889839828014374, "operator": "ge", "actual_fraction": 0.8458721704394141, "metric": 0.20894280280516728, "rescued": 2777, "harmed": 600, "missed_rescue": 684, "avoided_harm": 76, "selected_both_correct": 6383, "selected_both_wrong": 404}, {"budget": 0.87, "threshold": -0.16629797220230103, "operator": "ge", "actual_fraction": 0.8585219707057257, "metric": 0.2146238532926011, "rescued": 2839, "harmed": 603, "missed_rescue": 622, "avoided_harm": 73, "selected_both_correct": 6459, "selected_both_wrong": 415}, {"budget": 0.88, "threshold": -0.17295491695404053, "operator": "ge", "actual_fraction": 0.8655126498002663, "metric": 0.21521435730631172, "rescued": 2868, "harmed": 611, "missed_rescue": 593, "avoided_harm": 65, "selected_both_correct": 6497, "selected_both_wrong": 424}, {"budget": 0.89, "threshold": -0.18053314089775085, "operator": "ge", "actual_fraction": 0.8738348868175766, "metric": 0.2202221543306381, "rescued": 2899, "harmed": 617, "missed_rescue": 562, "avoided_harm": 59, "selected_both_correct": 6544, "selected_both_wrong": 440}, {"budget": 0.9, "threshold": -0.19442276656627655, "operator": "ge", "actual_fraction": 0.8878162450066578, "metric": 0.22393167777841913, "rescued": 2961, "harmed": 626, "missed_rescue": 500, "avoided_harm": 50, "selected_both_correct": 6624, "selected_both_wrong": 457}, {"budget": 0.91, "threshold": -0.20900504291057587, "operator": "ge", "actual_fraction": 0.8961384820239681, "metric": 0.22795997864890608, "rescued": 3002, "harmed": 629, "missed_rescue": 459, "avoided_harm": 47, "selected_both_correct": 6676, "selected_both_wrong": 461}, {"budget": 0.92, "threshold": -0.22383525967597961, "operator": "ge", "actual_fraction": 0.9027962716378163, "metric": 0.23118004657749075, "rescued": 3034, "harmed": 634, "missed_rescue": 427, "avoided_harm": 42, "selected_both_correct": 6715, "selected_both_wrong": 465}, {"budget": 0.93, "threshold": -0.25952714681625366, "operator": "ge", "actual_fraction": 0.9184420772303595, "metric": 0.23697523764758885, "rescued": 3106, "harmed": 646, "missed_rescue": 355, "avoided_harm": 30, "selected_both_correct": 6801, "selected_both_wrong": 483}, {"budget": 0.94, "threshold": -0.2804492115974426, "operator": "ge", "actual_fraction": 0.9257656458055925, "metric": 0.24091409959663834, "rescued": 3141, "harmed": 647, "missed_rescue": 320, "avoided_harm": 29, "selected_both_correct": 6844, "selected_both_wrong": 492}, {"budget": 0.95, "threshold": -0.2984963357448578, "operator": "ge", "actual_fraction": 0.9320905459387483, "metric": 0.2426031328046549, "rescued": 3163, "harmed": 648, "missed_rescue": 298, "avoided_harm": 28, "selected_both_correct": 6890, "selected_both_wrong": 499}, {"budget": 0.96, "threshold": -0.32223379611968994, "operator": "ge", "actual_fraction": 0.9410785619174434, "metric": 0.24537310927695588, "rescued": 3203, "harmed": 656, "missed_rescue": 258, "avoided_harm": 20, "selected_both_correct": 6944, "selected_both_wrong": 505}, {"budget": 0.97, "threshold": -0.36503666639328003, "operator": "ge", "actual_fraction": 0.9573901464713716, "metric": 0.25207284548974734, "rescued": 3277, "harmed": 663, "missed_rescue": 184, "avoided_harm": 13, "selected_both_correct": 7042, "selected_both_wrong": 522}, {"budget": 0.98, "threshold": -0.4221048653125763, "operator": "ge", "actual_fraction": 0.9790279627163782, "metric": 0.2631165680871859, "rescued": 3373, "harmed": 668, "missed_rescue": 88, "avoided_harm": 8, "selected_both_correct": 7186, "selected_both_wrong": 537}, {"budget": 0.99, "threshold": -0.44162338972091675, "operator": "ge", "actual_fraction": 0.9860186418109188, "metric": 0.26818431283513144, "rescued": 3404, "harmed": 671, "missed_rescue": 57, "avoided_harm": 5, "selected_both_correct": 7228, "selected_both_wrong": 545}, {"budget": 1.0, "threshold": null, "operator": "always", "actual_fraction": 1.0, "metric": 0.27001009067886245, "rescued": 3461, "harmed": 676, "missed_rescue": 0, "avoided_harm": 0, "selected_both_correct": 7329, "selected_both_wrong": 550}]}, "learned": {"selection": "full-budget normalized four-state LR; validation-only epoch, alpha and thresholds", "validation_point": {"metric": 0.2722897443613517, "actual_fraction": 0.8308131241084166, "metric_std": 0.0018330891277857937, "actual_fraction_std": 0.02164855687885186, "per_seed": [{"seed": 2026, "budget": 0.86, "threshold": -0.041384633630514145, "operator": "ge", "alpha": 0.0, "epoch": 2, "metric": 0.2713340601085888, "actual_fraction": 0.8587731811697575, "rescue": 815, "harm": 154, "missed_rescue": 0}, {"seed": 2027, "budget": 0.82, "threshold": -0.013656230013165556, "operator": "ge", "alpha": 0.25, "epoch": 1, "metric": 0.27453926046943244, "actual_fraction": 0.818830242510699, "rescue": 814, "harm": 146, "missed_rescue": 1}, {"seed": 2028, "budget": 0.85, "threshold": -0.075476986549853, "operator": "ge", "alpha": 0.25, "epoch": 2, "metric": 0.27194364490228756, "actual_fraction": 0.8487874465049928, "rescue": 815, "harm": 152, "missed_rescue": 0}, {"seed": 2029, "budget": 0.81, "threshold": -0.00946833472698927, "operator": "ge", "alpha": 0.0, "epoch": 4, "metric": 0.2699586861016554, "actual_fraction": 0.8088445078459344, "rescue": 813, "harm": 143, "missed_rescue": 2}, {"seed": 2030, "budget": 0.82, "threshold": -0.04945142256409497, "operator": "ge", "alpha": 0.5, "epoch": 1, "metric": 0.27367307022479437, "actual_fraction": 0.818830242510699, "rescue": 814, "harm": 147, "missed_rescue": 1}], "rescued": 814.2, "harmed": 148.4, "missed_rescue": 0.8, "avoided_harm": 36.599999999999994}, "validation_curve": [{"budget": 0.0, "metric": 0.07641657905799261, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.07760420414815937, "metric_std": 3.13804202221597e-05, "actual_fraction": 0.009985734664764621, "actual_fraction_std": 0.0}, {"budget": 0.02, "metric": 0.08364279768467295, "metric_std": 0.0120136750914809, "actual_fraction": 0.019971469329529243, "actual_fraction_std": 0.0}, {"budget": 0.03, "metric": 0.08288474056619287, "metric_std": 0.010138106389499088, "actual_fraction": 0.029957203994293864, "actual_fraction_std": 0.0}, {"budget": 0.04, "metric": 0.08718292794287197, "metric_std": 0.010864557006028392, "actual_fraction": 0.039942938659058486, "actual_fraction_std": 0.0}, {"budget": 0.05, "metric": 0.0903985335002441, "metric_std": 0.010436139909192978, "actual_fraction": 0.04992867332382311, "actual_fraction_std": 0.0}, {"budget": 0.06, "metric": 0.09006416913125892, "metric_std": 0.009594219453243156, "actual_fraction": 0.05991440798858773, "actual_fraction_std": 0.0}, {"budget": 0.07, "metric": 0.09104841252841733, "metric_std": 0.009286260765291068, "actual_fraction": 0.06990014265335236, "actual_fraction_std": 0.0}, {"budget": 0.08, "metric": 0.09064956489490203, "metric_std": 0.007787224836013084, "actual_fraction": 0.07988587731811697, "actual_fraction_std": 0.0}, {"budget": 0.09, "metric": 0.09730323775133182, "metric_std": 0.00019598055200555913, "actual_fraction": 0.0898716119828816, "actual_fraction_std": 0.0}, {"budget": 0.1, "metric": 0.09781901730119323, "metric_std": 0.0005729094233299627, "actual_fraction": 0.09985734664764621, "actual_fraction_std": 0.0}, {"budget": 0.11, "metric": 0.1011732794521178, "metric_std": 0.005572397832711977, "actual_fraction": 0.10984308131241086, "actual_fraction_std": 1.5515838457795457e-17}, {"budget": 0.12, "metric": 0.10428078521668382, "metric_std": 0.010942734498706216, "actual_fraction": 0.11982881597717546, "actual_fraction_std": 0.0}, {"budget": 0.13, "metric": 0.1067234605214755, "metric_std": 0.01139139810261127, "actual_fraction": 0.12981455064194009, "actual_fraction_std": 0.0}, {"budget": 0.14, "metric": 0.11588868166684385, "metric_std": 0.010420979445529782, "actual_fraction": 0.1398002853067047, "actual_fraction_std": 0.0}, {"budget": 0.15, "metric": 0.12288152236321456, "metric_std": 0.008304122614151607, "actual_fraction": 0.14978601997146934, "actual_fraction_std": 0.0}, {"budget": 0.16, "metric": 0.12873115760506323, "metric_std": 0.008659283636972777, "actual_fraction": 0.15977175463623394, "actual_fraction_std": 0.0}, {"budget": 0.17, "metric": 0.1303893634212186, "metric_std": 0.010477587753947118, "actual_fraction": 0.16975748930099857, "actual_fraction_std": 0.0}, {"budget": 0.18, "metric": 0.13285840350808348, "metric_std": 0.013216097122377415, "actual_fraction": 0.1797432239657632, "actual_fraction_std": 0.0}, {"budget": 0.19, "metric": 0.13313125346288085, "metric_std": 0.012551954502143654, "actual_fraction": 0.18972895863052786, "actual_fraction_std": 0.0}, {"budget": 0.2, "metric": 0.13557212834312704, "metric_std": 0.014274468779118627, "actual_fraction": 0.19971469329529243, "actual_fraction_std": 0.0}, {"budget": 0.21, "metric": 0.13626410496070496, "metric_std": 0.014275243539950404, "actual_fraction": 0.20970042796005703, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.22, "metric": 0.138808502026701, "metric_std": 0.012643766665603438, "actual_fraction": 0.2196861626248217, "actual_fraction_std": 3.1031676915590914e-17}, {"budget": 0.23, "metric": 0.1413677126199937, "metric_std": 0.014822243126161296, "actual_fraction": 0.22967189728958629, "actual_fraction_std": 0.0}, {"budget": 0.24, "metric": 0.14331854841583022, "metric_std": 0.01713253047941123, "actual_fraction": 0.2396576319543509, "actual_fraction_std": 0.0}, {"budget": 0.25, "metric": 0.14561281648295704, "metric_std": 0.01628463969631159, "actual_fraction": 0.24964336661911551, "actual_fraction_std": 0.0}, {"budget": 0.26, "metric": 0.14682389378439567, "metric_std": 0.016019827549994405, "actual_fraction": 0.25962910128388017, "actual_fraction_std": 0.0}, {"budget": 0.27, "metric": 0.14889542843308515, "metric_std": 0.016245402307732897, "actual_fraction": 0.26961483594864477, "actual_fraction_std": 0.0}, {"budget": 0.28, "metric": 0.1543360615584524, "metric_std": 0.015518384367427827, "actual_fraction": 0.2796005706134094, "actual_fraction_std": 0.0}, {"budget": 0.29, "metric": 0.1547515491677909, "metric_std": 0.0149568271455956, "actual_fraction": 0.289586305278174, "actual_fraction_std": 0.0}, {"budget": 0.3, "metric": 0.15581427908667372, "metric_std": 0.014389088209609251, "actual_fraction": 0.2995720399429387, "actual_fraction_std": 0.0}, {"budget": 0.31, "metric": 0.1613209923470381, "metric_std": 0.010517917367042592, "actual_fraction": 0.3095577746077033, "actual_fraction_std": 0.0}, {"budget": 0.32, "metric": 0.16453245539075154, "metric_std": 0.005100461734567098, "actual_fraction": 0.3195435092724679, "actual_fraction_std": 0.0}, {"budget": 0.33, "metric": 0.16554816678591117, "metric_std": 0.005408516701371362, "actual_fraction": 0.3295292439372325, "actual_fraction_std": 0.0}, {"budget": 0.34, "metric": 0.16860843594166003, "metric_std": 0.008004273096619349, "actual_fraction": 0.33951497860199714, "actual_fraction_std": 0.0}, {"budget": 0.35, "metric": 0.1707136761070432, "metric_std": 0.006749714544730724, "actual_fraction": 0.34950071326676174, "actual_fraction_std": 0.0}, {"budget": 0.36, "metric": 0.17183376729723546, "metric_std": 0.006426445183940542, "actual_fraction": 0.3594864479315264, "actual_fraction_std": 0.0}, {"budget": 0.37, "metric": 0.17361642587323528, "metric_std": 0.0065526794810730924, "actual_fraction": 0.36947218259629094, "actual_fraction_std": 0.0}, {"budget": 0.38, "metric": 0.17692503146213218, "metric_std": 0.00659297270407118, "actual_fraction": 0.3794579172610557, "actual_fraction_std": 0.0}, {"budget": 0.39, "metric": 0.18105455039929227, "metric_std": 0.0038203864643502213, "actual_fraction": 0.38944365192582026, "actual_fraction_std": 0.0}, {"budget": 0.4, "metric": 0.18372829721365588, "metric_std": 0.004546036622700707, "actual_fraction": 0.39942938659058486, "actual_fraction_std": 0.0}, {"budget": 0.41, "metric": 0.18647762337175228, "metric_std": 0.005679677754429664, "actual_fraction": 0.4094151212553495, "actual_fraction_std": 0.0}, {"budget": 0.42, "metric": 0.18857264195439002, "metric_std": 0.005631715058562497, "actual_fraction": 0.41940085592011406, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.43, "metric": 0.19263109157027464, "metric_std": 0.0024684454730610496, "actual_fraction": 0.4293865905848787, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.44, "metric": 0.19624461810507046, "metric_std": 0.0020478964789141774, "actual_fraction": 0.4393723252496434, "actual_fraction_std": 6.206335383118183e-17}, {"budget": 0.45, "metric": 0.19832903889376946, "metric_std": 0.0023885466186863245, "actual_fraction": 0.44935805991440797, "actual_fraction_std": 0.0}, {"budget": 0.46, "metric": 0.20070993717215538, "metric_std": 0.004395803914150421, "actual_fraction": 0.45934379457917257, "actual_fraction_std": 0.0}, {"budget": 0.47, "metric": 0.20423933780612485, "metric_std": 0.005308263991511532, "actual_fraction": 0.4693295292439372, "actual_fraction_std": 0.0}, {"budget": 0.48, "metric": 0.20555206747414548, "metric_std": 0.006353743726502917, "actual_fraction": 0.4793152639087018, "actual_fraction_std": 0.0}, {"budget": 0.49, "metric": 0.20668184940507608, "metric_std": 0.004273022628908141, "actual_fraction": 0.4893009985734665, "actual_fraction_std": 0.0}, {"budget": 0.5, "metric": 0.2099392356449362, "metric_std": 0.006088438644691793, "actual_fraction": 0.49928673323823103, "actual_fraction_std": 0.0}, {"budget": 0.51, "metric": 0.2113633505127349, "metric_std": 0.006240455934194265, "actual_fraction": 0.5092724679029957, "actual_fraction_std": 0.0}, {"budget": 0.52, "metric": 0.2124409832059179, "metric_std": 0.0062704633156593255, "actual_fraction": 0.5192582025677603, "actual_fraction_std": 0.0}, {"budget": 0.53, "metric": 0.2138282061123228, "metric_std": 0.005743829815116192, "actual_fraction": 0.5292439372325249, "actual_fraction_std": 0.0}, {"budget": 0.54, "metric": 0.21711481994528134, "metric_std": 0.004222621278723385, "actual_fraction": 0.5392296718972895, "actual_fraction_std": 0.0}, {"budget": 0.55, "metric": 0.21973415243360578, "metric_std": 0.0064741155086086985, "actual_fraction": 0.5492154065620543, "actual_fraction_std": 0.0}, {"budget": 0.56, "metric": 0.21912914180047882, "metric_std": 0.0032208913297636682, "actual_fraction": 0.5592011412268189, "actual_fraction_std": 0.0}, {"budget": 0.57, "metric": 0.22036415027282885, "metric_std": 0.002787870078945069, "actual_fraction": 0.5691868758915835, "actual_fraction_std": 0.0}, {"budget": 0.58, "metric": 0.22182835027791245, "metric_std": 0.0027709039169427933, "actual_fraction": 0.579172610556348, "actual_fraction_std": 0.0}, {"budget": 0.59, "metric": 0.2233196815170091, "metric_std": 0.0033350437682473817, "actual_fraction": 0.5891583452211127, "actual_fraction_std": 0.0}, {"budget": 0.6, "metric": 0.22290685391156712, "metric_std": 0.0033279283509387745, "actual_fraction": 0.5991440798858774, "actual_fraction_std": 0.0}, {"budget": 0.61, "metric": 0.2249418332654817, "metric_std": 0.0030718999357763916, "actual_fraction": 0.609129814550642, "actual_fraction_std": 0.0}, {"budget": 0.62, "metric": 0.22725520402547766, "metric_std": 0.00376964303262346, "actual_fraction": 0.6191155492154066, "actual_fraction_std": 0.0}, {"budget": 0.63, "metric": 0.22940539347315828, "metric_std": 0.0062768232829704075, "actual_fraction": 0.6291012838801712, "actual_fraction_std": 0.0}, {"budget": 0.64, "metric": 0.2323340388582536, "metric_std": 0.006333737823407219, "actual_fraction": 0.6390870185449358, "actual_fraction_std": 0.0}, {"budget": 0.65, "metric": 0.23468693132604224, "metric_std": 0.005943119457681367, "actual_fraction": 0.6490727532097005, "actual_fraction_std": 0.0}, {"budget": 0.66, "metric": 0.2369490066417553, "metric_std": 0.006016598479123984, "actual_fraction": 0.659058487874465, "actual_fraction_std": 0.0}, {"budget": 0.67, "metric": 0.23870364007721984, "metric_std": 0.005839011041878133, "actual_fraction": 0.6690442225392297, "actual_fraction_std": 0.0}, {"budget": 0.68, "metric": 0.241246158728642, "metric_std": 0.006008124892847718, "actual_fraction": 0.6790299572039943, "actual_fraction_std": 0.0}, {"budget": 0.69, "metric": 0.24324221656804407, "metric_std": 0.005223605488425164, "actual_fraction": 0.6890156918687589, "actual_fraction_std": 0.0}, {"budget": 0.7, "metric": 0.24487024135876262, "metric_std": 0.004628627863344786, "actual_fraction": 0.6990014265335235, "actual_fraction_std": 0.0}, {"budget": 0.71, "metric": 0.24730234614788485, "metric_std": 0.004504583057964129, "actual_fraction": 0.7089871611982882, "actual_fraction_std": 0.0}, {"budget": 0.72, "metric": 0.24907169554499972, "metric_std": 0.0041298435215759936, "actual_fraction": 0.7189728958630528, "actual_fraction_std": 0.0}, {"budget": 0.73, "metric": 0.2519551211147789, "metric_std": 0.003243339800532077, "actual_fraction": 0.7289586305278174, "actual_fraction_std": 0.0}, {"budget": 0.74, "metric": 0.25449255636644186, "metric_std": 0.0020262222679700475, "actual_fraction": 0.7389443651925819, "actual_fraction_std": 0.0}, {"budget": 0.75, "metric": 0.2581476354067899, "metric_std": 0.001877548979133935, "actual_fraction": 0.7489300998573466, "actual_fraction_std": 0.0}, {"budget": 0.76, "metric": 0.26197696663970216, "metric_std": 0.0025630578097892457, "actual_fraction": 0.7589158345221114, "actual_fraction_std": 0.0}, {"budget": 0.77, "metric": 0.2662698135247865, "metric_std": 0.002864967869652235, "actual_fraction": 0.7689015691868759, "actual_fraction_std": 0.0}, {"budget": 0.78, "metric": 0.26732267690931, "metric_std": 0.002426892878249774, "actual_fraction": 0.7788873038516405, "actual_fraction_std": 0.0}, {"budget": 0.79, "metric": 0.2687167183317788, "metric_std": 0.0016924707776094295, "actual_fraction": 0.7888730385164051, "actual_fraction_std": 0.0}, {"budget": 0.8, "metric": 0.27020170885260913, "metric_std": 0.001710535093326589, "actual_fraction": 0.7988587731811697, "actual_fraction_std": 0.0}, {"budget": 0.81, "metric": 0.27063984279539804, "metric_std": 0.0011381770968044783, "actual_fraction": 0.8088445078459344, "actual_fraction_std": 0.0}, {"budget": 0.82, "metric": 0.2708581148548966, "metric_std": 0.003514153318869339, "actual_fraction": 0.818830242510699, "actual_fraction_std": 0.0}, {"budget": 0.83, "metric": 0.26919460478258017, "metric_std": 0.00277248976630742, "actual_fraction": 0.8288159771754635, "actual_fraction_std": 0.0}, {"budget": 0.84, "metric": 0.2699364825484262, "metric_std": 0.0031191101821669114, "actual_fraction": 0.8388017118402281, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.85, "metric": 0.2689267822115659, "metric_std": 0.0033865225770732714, "actual_fraction": 0.8487874465049929, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.86, "metric": 0.2691823189464066, "metric_std": 0.002179491876404108, "actual_fraction": 0.8587731811697574, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.87, "metric": 0.26762550949099123, "metric_std": 0.0016542671461111116, "actual_fraction": 0.8687589158345222, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.88, "metric": 0.2676668928840348, "metric_std": 0.0012188229982531586, "actual_fraction": 0.8787446504992869, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.89, "metric": 0.2664595419004704, "metric_std": 0.0016845258864134956, "actual_fraction": 0.8887303851640513, "actual_fraction_std": 0.0}, {"budget": 0.9, "metric": 0.2655170551885693, "metric_std": 0.0022637095583849737, "actual_fraction": 0.8987161198288159, "actual_fraction_std": 0.0}, {"budget": 0.91, "metric": 0.26509909851288616, "metric_std": 0.0028300326381077968, "actual_fraction": 0.9087018544935808, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.92, "metric": 0.2643480513105606, "metric_std": 0.0025847303495537158, "actual_fraction": 0.9186875891583451, "actual_fraction_std": 0.0}, {"budget": 0.93, "metric": 0.26326203709919477, "metric_std": 0.0022138711549044457, "actual_fraction": 0.9286733238231101, "actual_fraction_std": 1.2412670766236366e-16}, {"budget": 0.94, "metric": 0.2619940380973683, "metric_std": 0.0020164403876586985, "actual_fraction": 0.9386590584878745, "actual_fraction_std": 0.0}, {"budget": 0.95, "metric": 0.2605444443835352, "metric_std": 0.0014538717258036754, "actual_fraction": 0.948644793152639, "actual_fraction_std": 0.0}, {"budget": 0.96, "metric": 0.25964423047609164, "metric_std": 0.0004945435552899227, "actual_fraction": 0.9586305278174037, "actual_fraction_std": 0.0}, {"budget": 0.97, "metric": 0.2592608203378428, "metric_std": 0.0002691919631488578, "actual_fraction": 0.9686162624821684, "actual_fraction_std": 0.0}, {"budget": 0.98, "metric": 0.2586068761049241, "metric_std": 0.0007322929718748753, "actual_fraction": 0.978601997146933, "actual_fraction_std": 0.0}, {"budget": 0.99, "metric": 0.25657989662979863, "metric_std": 0.0007430705019011831, "actual_fraction": 0.9885877318116976, "actual_fraction_std": 0.0}, {"budget": 1.0, "metric": 0.25448626222520154, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}], "test_point": {"metric": 0.26783631605767233, "actual_fraction": 0.8356857523302264, "metric_std": 0.0007555702359011525, "actual_fraction_std": 0.021729295510904342, "per_seed": [{"seed": 2026, "budget": 0.86, "threshold": -0.041384633630514145, "operator": "ge", "alpha": 0.0, "epoch": 2, "metric": 0.26843442031752185, "actual_fraction": 0.8638482023968043, "rescue": 3450, "harm": 582, "missed_rescue": 11}, {"seed": 2027, "budget": 0.82, "threshold": -0.013656230013165556, "operator": "ge", "alpha": 0.25, "epoch": 1, "metric": 0.26776779485961383, "actual_fraction": 0.8199067909454061, "rescue": 3448, "harm": 572, "missed_rescue": 13}, {"seed": 2028, "budget": 0.85, "threshold": -0.075476986549853, "operator": "ge", "alpha": 0.25, "epoch": 2, "metric": 0.26671311424348265, "actual_fraction": 0.8541944074567244, "rescue": 3448, "harm": 585, "missed_rescue": 13}, {"seed": 2029, "budget": 0.81, "threshold": -0.00946833472698927, "operator": "ge", "alpha": 0.0, "epoch": 4, "metric": 0.268624728799547, "actual_fraction": 0.8239014647137151, "rescue": 3447, "harm": 565, "missed_rescue": 14}, {"seed": 2030, "budget": 0.82, "threshold": -0.04945142256409497, "operator": "ge", "alpha": 0.5, "epoch": 1, "metric": 0.2676415220681962, "actual_fraction": 0.816577896138482, "rescue": 3447, "harm": 570, "missed_rescue": 14}], "rescued": 3448.0, "harmed": 574.8, "missed_rescue": 13.0, "avoided_harm": 101.20000000000005}, "test_curve": [{"budget": 0.0, "metric": 0.07887397871043929, "metric_std": 0.0, "actual_fraction": 0.0, "actual_fraction_std": 0.0}, {"budget": 0.01, "metric": 0.08757166559951829, "metric_std": 0.01077655250663246, "actual_fraction": 0.006924101198402132, "actual_fraction_std": 0.002517665832382029}, {"budget": 0.02, "metric": 0.10460339788877256, "metric_std": 0.007507239500355689, "actual_fraction": 0.016378162450066576, "actual_fraction_std": 0.0015145556812353293}, {"budget": 0.03, "metric": 0.10682788156566658, "metric_std": 0.007287110070573894, "actual_fraction": 0.02230359520639148, "actual_fraction_std": 0.003244605307859176}, {"budget": 0.04, "metric": 0.10835296714409866, "metric_std": 0.0066520281917125425, "actual_fraction": 0.03022636484687084, "actual_fraction_std": 0.0022870258371927663}, {"budget": 0.05, "metric": 0.11172057206769052, "metric_std": 0.00578014160302038, "actual_fraction": 0.03754993342210386, "actual_fraction_std": 0.0028481024815189635}, {"budget": 0.06, "metric": 0.11390212200530787, "metric_std": 0.0019932491797450065, "actual_fraction": 0.043941411451398134, "actual_fraction_std": 0.003959845956514753}, {"budget": 0.07, "metric": 0.113403656045915, "metric_std": 0.0020259677651590937, "actual_fraction": 0.04806924101198402, "actual_fraction_std": 0.003726281043663308}, {"budget": 0.08, "metric": 0.11350965461338114, "metric_std": 0.0027820588884723864, "actual_fraction": 0.0533288948069241, "actual_fraction_std": 0.0032903898250736517}, {"budget": 0.09, "metric": 0.1175214518842326, "metric_std": 0.0015273393216411791, "actual_fraction": 0.061784287616511314, "actual_fraction_std": 0.003755902185770526}, {"budget": 0.1, "metric": 0.11932558918999554, "metric_std": 0.0023008909891892326, "actual_fraction": 0.06770972037283621, "actual_fraction_std": 0.0037559021857705268}, {"budget": 0.11, "metric": 0.1212302275306284, "metric_std": 0.0011680167967856798, "actual_fraction": 0.07463382157123835, "actual_fraction_std": 0.005801261837756903}, {"budget": 0.12, "metric": 0.12204092472728105, "metric_std": 0.0016753594342781843, "actual_fraction": 0.0836884154460719, "actual_fraction_std": 0.00450693509882192}, {"budget": 0.13, "metric": 0.12377018974316686, "metric_std": 0.0012991515494432033, "actual_fraction": 0.09533954727030627, "actual_fraction_std": 0.008843840444389909}, {"budget": 0.14, "metric": 0.12395633512277186, "metric_std": 0.0013563081899442108, "actual_fraction": 0.1044607190412783, "actual_fraction_std": 0.00957715572811583}, {"budget": 0.15, "metric": 0.12419695003286606, "metric_std": 0.0017167969309496327, "actual_fraction": 0.1166444740346205, "actual_fraction_std": 0.009943871836018288}, {"budget": 0.16, "metric": 0.12350982567553151, "metric_std": 0.0014705288535510914, "actual_fraction": 0.12842876165113182, "actual_fraction_std": 0.004147113582529268}, {"budget": 0.17, "metric": 0.12382685589306941, "metric_std": 0.0019709362812068264, "actual_fraction": 0.13788282290279627, "actual_fraction_std": 0.0014395305133448521}, {"budget": 0.18, "metric": 0.1268684217609487, "metric_std": 0.0023585275898612363, "actual_fraction": 0.14846870838881493, "actual_fraction_std": 0.005668903583863646}, {"budget": 0.19, "metric": 0.12937271820709742, "metric_std": 0.002081191760438066, "actual_fraction": 0.15758988015978692, "actual_fraction_std": 0.00613637312096112}, {"budget": 0.2, "metric": 0.1324010329252992, "metric_std": 0.002502905842287525, "actual_fraction": 0.17057256990679093, "actual_fraction_std": 0.004240912690863138}, {"budget": 0.21, "metric": 0.13349543501216948, "metric_std": 0.0021769665674402985, "actual_fraction": 0.18115845539280956, "actual_fraction_std": 0.0060810444087249105}, {"budget": 0.22, "metric": 0.13401333697580414, "metric_std": 0.002571860475226747, "actual_fraction": 0.18954727030625831, "actual_fraction_std": 0.0065088193898563355}, {"budget": 0.23, "metric": 0.13397816368152862, "metric_std": 0.0022969538947532745, "actual_fraction": 0.19900133155792277, "actual_fraction_std": 0.009522035784442493}, {"budget": 0.24, "metric": 0.13491363667919004, "metric_std": 0.0015355026661156076, "actual_fraction": 0.20978695073235681, "actual_fraction_std": 0.010288385497779946}, {"budget": 0.25, "metric": 0.13487165208310056, "metric_std": 0.001863614107624181, "actual_fraction": 0.21884154460719044, "actual_fraction_std": 0.01375885019459759}, {"budget": 0.26, "metric": 0.13715357977903225, "metric_std": 0.0018536989751356588, "actual_fraction": 0.23462050599201065, "actual_fraction_std": 0.00969217312926752}, {"budget": 0.27, "metric": 0.13833008215415926, "metric_std": 0.003019432769504127, "actual_fraction": 0.24613848202396804, "actual_fraction_std": 0.00977528132078498}, {"budget": 0.28, "metric": 0.14051386551782027, "metric_std": 0.0021819288530588363, "actual_fraction": 0.25918774966711056, "actual_fraction_std": 0.005892237844081734}, {"budget": 0.29, "metric": 0.14248305092917973, "metric_std": 0.0024199218915006275, "actual_fraction": 0.26897470039946736, "actual_fraction_std": 0.008037747315345975}, {"budget": 0.3, "metric": 0.14567041765464278, "metric_std": 0.0019191892640399437, "actual_fraction": 0.2792276964047936, "actual_fraction_std": 0.011426404148648052}, {"budget": 0.31, "metric": 0.14852586615731456, "metric_std": 0.0011114326119782436, "actual_fraction": 0.29141145139813585, "actual_fraction_std": 0.008307576967943561}, {"budget": 0.32, "metric": 0.1498475097358298, "metric_std": 0.0014123073818496873, "actual_fraction": 0.30279627163781625, "actual_fraction_std": 0.011407477007798634}, {"budget": 0.33, "metric": 0.15253241248445146, "metric_std": 0.0023497246164877685, "actual_fraction": 0.3189081225033289, "actual_fraction_std": 0.007014416613084388}, {"budget": 0.34, "metric": 0.15395895968408063, "metric_std": 0.0024945367637092854, "actual_fraction": 0.33115845539280964, "actual_fraction_std": 0.012072465466464056}, {"budget": 0.35, "metric": 0.15488355038532386, "metric_std": 0.002826658906257943, "actual_fraction": 0.3399467376830892, "actual_fraction_std": 0.010488396154287177}, {"budget": 0.36, "metric": 0.15736839179719622, "metric_std": 0.004101182954750341, "actual_fraction": 0.3518641810918775, "actual_fraction_std": 0.011217437047599347}, {"budget": 0.37, "metric": 0.15865678537271144, "metric_std": 0.0033937227430927033, "actual_fraction": 0.3637150466045273, "actual_fraction_std": 0.010690366157180707}, {"budget": 0.38, "metric": 0.16139846548320752, "metric_std": 0.002127880199817294, "actual_fraction": 0.3762982689747004, "actual_fraction_std": 0.008862615868065908}, {"budget": 0.39, "metric": 0.16312082223148333, "metric_std": 0.0010511520108659185, "actual_fraction": 0.38315579227696406, "actual_fraction_std": 0.00874429130263895}, {"budget": 0.4, "metric": 0.16502401295677457, "metric_std": 0.0015356795262103102, "actual_fraction": 0.39707057256990674, "actual_fraction_std": 0.005858287407455267}, {"budget": 0.41, "metric": 0.16731051871184222, "metric_std": 0.0014827604289643563, "actual_fraction": 0.4093209054593875, "actual_fraction_std": 0.0038144544635502617}, {"budget": 0.42, "metric": 0.17075990416662384, "metric_std": 0.001873270045446746, "actual_fraction": 0.42416777629826896, "actual_fraction_std": 0.003791142071240479}, {"budget": 0.43, "metric": 0.17424409501390517, "metric_std": 0.0028303665007586866, "actual_fraction": 0.43495339547270306, "actual_fraction_std": 0.006309990134983539}, {"budget": 0.44, "metric": 0.17648253297807862, "metric_std": 0.002939856198778324, "actual_fraction": 0.4451398135818908, "actual_fraction_std": 0.00410008501669829}, {"budget": 0.45, "metric": 0.17969611645498293, "metric_std": 0.0018416776033861025, "actual_fraction": 0.45612516644474044, "actual_fraction_std": 0.0025935487590921986}, {"budget": 0.46, "metric": 0.18183045177824414, "metric_std": 0.002237145403827163, "actual_fraction": 0.4651131824234354, "actual_fraction_std": 0.0028578130534400578}, {"budget": 0.47, "metric": 0.18440659527724507, "metric_std": 0.003834255917796977, "actual_fraction": 0.4766977363515313, "actual_fraction_std": 0.003411102119160991}, {"budget": 0.48, "metric": 0.18685063507320088, "metric_std": 0.0032831708091817973, "actual_fraction": 0.48528628495339543, "actual_fraction_std": 0.00127631754411989}, {"budget": 0.49, "metric": 0.18884895692481324, "metric_std": 0.0034534086981776076, "actual_fraction": 0.49627163781624495, "actual_fraction_std": 0.0024394235633237514}, {"budget": 0.5, "metric": 0.18962185463397055, "metric_std": 0.0028122051127516014, "actual_fraction": 0.511318242343542, "actual_fraction_std": 0.004731248469890764}, {"budget": 0.51, "metric": 0.1900421766849928, "metric_std": 0.0032980072977146354, "actual_fraction": 0.5215712383488681, "actual_fraction_std": 0.008672393919419747}, {"budget": 0.52, "metric": 0.19219042191755156, "metric_std": 0.0030200162824359786, "actual_fraction": 0.5322902796271637, "actual_fraction_std": 0.011541235568148931}, {"budget": 0.53, "metric": 0.19438852630943243, "metric_std": 0.0031904995059163126, "actual_fraction": 0.5454061251664447, "actual_fraction_std": 0.011994650560627283}, {"budget": 0.54, "metric": 0.19696685707219339, "metric_std": 0.0037941722651010624, "actual_fraction": 0.5581890812250334, "actual_fraction_std": 0.008751891731107825}, {"budget": 0.55, "metric": 0.198102568677536, "metric_std": 0.004624761949029326, "actual_fraction": 0.5666444740346205, "actual_fraction_std": 0.008066647792735505}, {"budget": 0.56, "metric": 0.20189163754174927, "metric_std": 0.004948661416283359, "actual_fraction": 0.5778295605858854, "actual_fraction_std": 0.009208471585629872}, {"budget": 0.57, "metric": 0.20421715557926076, "metric_std": 0.0054832590378431625, "actual_fraction": 0.5875499334221039, "actual_fraction_std": 0.009222300384284343}, {"budget": 0.58, "metric": 0.206789404387869, "metric_std": 0.0066741435858427385, "actual_fraction": 0.5963382157123835, "actual_fraction_std": 0.009698459512452818}, {"budget": 0.59, "metric": 0.208580287903873, "metric_std": 0.006204550205743023, "actual_fraction": 0.6041944074567243, "actual_fraction_std": 0.007402581876809942}, {"budget": 0.6, "metric": 0.21121456286391815, "metric_std": 0.006493378761187845, "actual_fraction": 0.6163781624500666, "actual_fraction_std": 0.010191529834973855}, {"budget": 0.61, "metric": 0.21450604383742217, "metric_std": 0.005906440797941267, "actual_fraction": 0.6277629826897471, "actual_fraction_std": 0.009743488011225572}, {"budget": 0.62, "metric": 0.2190083544839201, "metric_std": 0.006721063590059664, "actual_fraction": 0.6376830892143809, "actual_fraction_std": 0.010798659736655458}, {"budget": 0.63, "metric": 0.22322950218882154, "metric_std": 0.0070467291665484945, "actual_fraction": 0.6495339547270307, "actual_fraction_std": 0.011091765600482929}, {"budget": 0.64, "metric": 0.2262927854760482, "metric_std": 0.007959296703925074, "actual_fraction": 0.6602529960053262, "actual_fraction_std": 0.013494552873817659}, {"budget": 0.65, "metric": 0.2286810489523116, "metric_std": 0.00569647196017078, "actual_fraction": 0.6683089214380826, "actual_fraction_std": 0.010207826732278992}, {"budget": 0.66, "metric": 0.23362476000600063, "metric_std": 0.006185961555455149, "actual_fraction": 0.6802929427430092, "actual_fraction_std": 0.01168579587585214}, {"budget": 0.67, "metric": 0.23666064727962421, "metric_std": 0.005908255486759012, "actual_fraction": 0.6908788282290279, "actual_fraction_std": 0.010614947252927708}, {"budget": 0.68, "metric": 0.23935957985366768, "metric_std": 0.006623559423878983, "actual_fraction": 0.7005992010652464, "actual_fraction_std": 0.011275571635702171}, {"budget": 0.69, "metric": 0.24232960618896623, "metric_std": 0.006323319596406592, "actual_fraction": 0.7097203728362184, "actual_fraction_std": 0.009889115031633096}, {"budget": 0.7, "metric": 0.24624215904029803, "metric_std": 0.0047070060931020275, "actual_fraction": 0.7197736351531291, "actual_fraction_std": 0.007650666483193523}, {"budget": 0.71, "metric": 0.24960201687378336, "metric_std": 0.005431878625390472, "actual_fraction": 0.7270972037283621, "actual_fraction_std": 0.008559856697536668}, {"budget": 0.72, "metric": 0.2538402232137829, "metric_std": 0.0038225840648052384, "actual_fraction": 0.7395472703062584, "actual_fraction_std": 0.0059250587113946936}, {"budget": 0.73, "metric": 0.2563238516220624, "metric_std": 0.0030947630608107396, "actual_fraction": 0.7467376830892143, "actual_fraction_std": 0.007290963614558185}, {"budget": 0.74, "metric": 0.2590672259765537, "metric_std": 0.0028865700652408164, "actual_fraction": 0.7554593874833555, "actual_fraction_std": 0.005835544051165676}, {"budget": 0.75, "metric": 0.2613953540437151, "metric_std": 0.002784472901477333, "actual_fraction": 0.7621837549933421, "actual_fraction_std": 0.00747927918507784}, {"budget": 0.76, "metric": 0.2651272390182007, "metric_std": 0.0012358775883734175, "actual_fraction": 0.7713715046604527, "actual_fraction_std": 0.003976601373906232}, {"budget": 0.77, "metric": 0.2681957046744783, "metric_std": 0.0014175968266461486, "actual_fraction": 0.7810252996005327, "actual_fraction_std": 0.0020574616698218577}, {"budget": 0.78, "metric": 0.26955292645029827, "metric_std": 0.0011723507997981049, "actual_fraction": 0.7876165113182424, "actual_fraction_std": 0.0019410625482174115}, {"budget": 0.79, "metric": 0.2699378171561613, "metric_std": 0.00026539110476943653, "actual_fraction": 0.7940745672436751, "actual_fraction_std": 0.003964041450493807}, {"budget": 0.8, "metric": 0.2694545155759426, "metric_std": 0.0003580169921237279, "actual_fraction": 0.8041944074567244, "actual_fraction_std": 0.0030726979385391016}, {"budget": 0.81, "metric": 0.26867211217327724, "metric_std": 0.0004246703352362427, "actual_fraction": 0.8141810918774967, "actual_fraction_std": 0.006039903291633412}, {"budget": 0.82, "metric": 0.2679937422857512, "metric_std": 0.0004784477513540633, "actual_fraction": 0.8218375499334222, "actual_fraction_std": 0.005410862256532306}, {"budget": 0.83, "metric": 0.26738688692079055, "metric_std": 0.0006000249669573502, "actual_fraction": 0.830226364846871, "actual_fraction_std": 0.004236991365298923}, {"budget": 0.84, "metric": 0.2675636044917876, "metric_std": 0.0013073025634650856, "actual_fraction": 0.8416777629826898, "actual_fraction_std": 0.001302104341917976}, {"budget": 0.85, "metric": 0.26719733853056876, "metric_std": 0.001423012675789301, "actual_fraction": 0.8531957390146472, "actual_fraction_std": 0.004453757709806798}, {"budget": 0.86, "metric": 0.2676983066027047, "metric_std": 0.0011745593816858688, "actual_fraction": 0.8653129161118509, "actual_fraction_std": 0.0028790610266897576}, {"budget": 0.87, "metric": 0.26746929511864453, "metric_std": 0.0011040115013683377, "actual_fraction": 0.8757656458055925, "actual_fraction_std": 0.0037617984251708866}, {"budget": 0.88, "metric": 0.2671674742299283, "metric_std": 0.0005447936835780084, "actual_fraction": 0.8824900133155793, "actual_fraction_std": 0.0033620189541817415}, {"budget": 0.89, "metric": 0.26725829225537984, "metric_std": 0.0010358249154295636, "actual_fraction": 0.8938082556591211, "actual_fraction_std": 0.004731248469890785}, {"budget": 0.9, "metric": 0.2682334895959223, "metric_std": 0.002631043022996776, "actual_fraction": 0.9042609853528628, "actual_fraction_std": 0.003857785514199235}, {"budget": 0.91, "metric": 0.2698175989198681, "metric_std": 0.0017323365673736254, "actual_fraction": 0.916711051930759, "actual_fraction_std": 0.002999701817861216}, {"budget": 0.92, "metric": 0.26991947716205444, "metric_std": 0.0018995080337003627, "actual_fraction": 0.9266977363515313, "actual_fraction_std": 0.006331029338017869}, {"budget": 0.93, "metric": 0.2701784098423855, "metric_std": 0.0015398432819181671, "actual_fraction": 0.9339547270306259, "actual_fraction_std": 0.00442004028720366}, {"budget": 0.94, "metric": 0.27012991155525545, "metric_std": 0.0016820874304928036, "actual_fraction": 0.9413448735019972, "actual_fraction_std": 0.004809072187691421}, {"budget": 0.95, "metric": 0.27131836752171967, "metric_std": 0.0010915488046070053, "actual_fraction": 0.9496671105193076, "actual_fraction_std": 0.004481043587340562}, {"budget": 0.96, "metric": 0.2718694990970568, "metric_std": 0.0012800102327303184, "actual_fraction": 0.9584553928095871, "actual_fraction_std": 0.004252654972748665}, {"budget": 0.97, "metric": 0.27149292269642605, "metric_std": 0.001275624005639333, "actual_fraction": 0.9678428761651132, "actual_fraction_std": 0.0024530137718961733}, {"budget": 0.98, "metric": 0.270512526775008, "metric_std": 0.0004631595694962044, "actual_fraction": 0.9800266311584555, "actual_fraction_std": 0.0038173585023900405}, {"budget": 0.99, "metric": 0.2690695021257777, "metric_std": 0.00022444753444754505, "actual_fraction": 0.9900133155792277, "actual_fraction_std": 0.0018233107773140838}, {"budget": 1.0, "metric": 0.27001009067886245, "metric_std": 0.0, "actual_fraction": 1.0, "actual_fraction_std": 0.0}]}}, "opportunity": {"val": {"both_correct": 1681, "rescuable": 815, "harmful": 185, "both_wrong": 123, "unit": "event", "total": 2804}, "test": {"both_correct": 7329, "rescuable": 3461, "harmful": 676, "both_wrong": 550, "unit": "event", "total": 12016}}, "small": {"validation": 0.07641657905799261, "test": 0.07887397871043929}, "large": {"validation": 0.25448626222520154, "test": 0.27001009067886245}, "learned_minus_confidence_test": -0.0021737746211901166, "learned_minus_small_test": 0.18896233734723306, "learned_minus_large_test": -0.0021737746211901166, "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "selected_method": "LR_4S_normalized_full_budget", "historical_run": "shared_12_rtdetr_x_fp32_Qwen3.8"}], "summary": {"learned_beats_confidence": 13, "learned_beats_both": 8, "learned_beats_large": 12}, "multiseed_protocol": {"selected_method": "four_difference", "selection": "task-balanced validation Score25 mean minus sample standard deviation over five router seeds", "repeat_seeds": [2026, 2027, 2028, 2029, 2030], "comparison": [{"method": "error", "validation_seed_scores": [0.652854712535587, 0.653201861091734, 0.653549705081995, 0.6529296233301471, 0.6533734477104217], "validation_mean": 0.653181869949977, "validation_std": 0.00029285862604605935, "selection_value": 0.6528890113239308}, {"method": "four_difference", "validation_seed_scores": [0.6659279432360811, 0.6666052399411169, 0.6664902552160231, 0.666460246201781, 0.6671766800294149], "validation_mean": 0.6665320729248834, "validation_std": 0.0004451831677229395, "selection_value": 0.6660868897571605}, {"method": "four_ratio", "validation_seed_scores": [0.6669800355113474, 0.6654694961331916, 0.6669276041336858, 0.6664081580190494, 0.6645222674050018], "validation_mean": 0.6660615122404552, "validation_std": 0.001052820164752406, "selection_value": 0.6650086920757028}, {"method": "gain", "validation_seed_scores": [0.6656365114137299, 0.6665939365548585, 0.6663926476912081, 0.6648075719596137, 0.6656891621835576], "validation_mean": 0.6658239659605936, "validation_std": 0.0007075700773489459, "selection_value": 0.6651163958832447}, {"method": "route", "validation_seed_scores": [0.6522091985724042, 0.6522614154026475, 0.6528619080142624, 0.6518457956309271, 0.6514639634291153], "validation_mean": 0.6521284562098713, "validation_std": 0.0005204797508024464, "selection_value": 0.6516079764590688}], "selector_reads_test_records": false, "historical_pilot_selected_method": "four_ratio", "rule_amendment": "User changed selection criterion after the seed-42 pilot and partial CUB test results had been reported. The final numerical criterion uses validation records only; this is not a prospectively untouched test set."}, "summary_comparison": "Five-seed mean metrics; differences within 1e-12 in raw fractions are treated as ties. CUB/ResNet/Q has exactly two rescues and two harms across five runs."} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/scaling_selected.json b/releases/20260913_lr/paper_data/scaling_selected.json new file mode 100644 index 0000000000000000000000000000000000000000..13b8eed7d75aa48de23cc24acd760a2611b272e4 --- /dev/null +++ b/releases/20260913_lr/paper_data/scaling_selected.json @@ -0,0 +1,1250 @@ +[ + { + "key": [ + "cub", + "cub", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0185", + "tag": "baseline_25pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7849499482222989, + "fixed25_actual_fraction": 0.2587159130134622, + "fixed25_rescue": 31, + "fixed25_harm": 201, + "rank25_metric": 0.7884017949603037, + "validation_selection": 0.9741666666666666, + "best_epoch": 12, + "seconds": 5.802937984466553 + }, + { + "id": "formal_0186", + "tag": "baseline_50pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.7904729030031067, + "fixed25_actual_fraction": 0.2549188816016569, + "fixed25_rescue": 38, + "fixed25_harm": 176, + "rank25_metric": 0.7930617880566103, + "validation_selection": 0.9731666666666667, + "best_epoch": 7, + "seconds": 7.1614015102386475 + }, + { + "id": "formal_0001", + "tag": "baseline_100pct", + "task": "cub", + "expert": "cub", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.787193648602002, + "fixed25_actual_fraction": 0.2557818432861581, + "fixed25_rescue": 33, + "fixed25_harm": 190, + "rank25_metric": 0.7913358646876079, + "validation_selection": 0.9744166666666666, + "best_epoch": 7, + "seconds": 2.9200518131256104 + } + ] + }, + { + "key": [ + "cub", + "cub", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0187", + "tag": "baseline_25pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6874352778736624, + "fixed25_actual_fraction": 0.6039005868139454, + "fixed25_rescue": 358, + "fixed25_harm": 1093, + "rank25_metric": 0.8060062133241284, + "validation_selection": 0.9316666666666666, + "best_epoch": 5, + "seconds": 4.623678922653198 + }, + { + "id": "formal_0188", + "tag": "baseline_50pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.694338971349672, + "fixed25_actual_fraction": 0.5997583707283397, + "fixed25_rescue": 357, + "fixed25_harm": 1052, + "rank25_metric": 0.8096306523990335, + "validation_selection": 0.9334166666666667, + "best_epoch": 3, + "seconds": 4.7386627197265625 + }, + { + "id": "formal_0009", + "tag": "baseline_100pct", + "task": "cub", + "expert": "cub", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "residual_blocks__layer_16", + "fixed25_metric": 0.6829478771142561, + "fixed25_actual_fraction": 0.6049361408353469, + "fixed25_rescue": 358, + "fixed25_harm": 1119, + "rank25_metric": 0.8091128753883328, + "validation_selection": 0.928, + "best_epoch": 1, + "seconds": 2.053396463394165 + } + ] + }, + { + "key": [ + "cub", + "glsim", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0189", + "tag": "baseline_25pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8674490852606145, + "fixed25_actual_fraction": 0.24818778046254747, + "fixed25_rescue": 11, + "fixed25_harm": 209, + "rank25_metric": 0.8674490852606145, + "validation_selection": 0.9786666666666667, + "best_epoch": 12, + "seconds": 6.889333963394165 + }, + { + "id": "formal_0190", + "tag": "baseline_50pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.870037970314118, + "fixed25_actual_fraction": 0.24991370383154987, + "fixed25_rescue": 13, + "fixed25_harm": 196, + "rank25_metric": 0.870037970314118, + "validation_selection": 0.9773333333333334, + "best_epoch": 6, + "seconds": 5.409059524536133 + }, + { + "id": "formal_0019", + "tag": "baseline_100pct", + "task": "cub", + "expert": "glsim", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "aggregator__layer_01", + "fixed25_metric": 0.8658957542285123, + "fixed25_actual_fraction": 0.25509147393855713, + "fixed25_rescue": 13, + "fixed25_harm": 220, + "rank25_metric": 0.8679668622713151, + "validation_selection": 0.9788333333333333, + "best_epoch": 9, + "seconds": 7.6912243366241455 + } + ] + }, + { + "key": [ + "cub", + "glsim", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0191", + "tag": "baseline_25pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8797031411805316, + "fixed25_actual_fraction": 0.23662409389023129, + "fixed25_rescue": 18, + "fixed25_harm": 145, + "rank25_metric": 0.8776320331377286, + "validation_selection": 0.98025, + "best_epoch": 18, + "seconds": 7.610384702682495 + }, + { + "id": "formal_0192", + "tag": "baseline_50pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.876423886779427, + "fixed25_actual_fraction": 0.24525371073524335, + "fixed25_rescue": 8, + "fixed25_harm": 154, + "rank25_metric": 0.8762512944425268, + "validation_selection": 0.981, + "best_epoch": 25, + "seconds": 10.510464906692505 + }, + { + "id": "formal_0027", + "tag": "baseline_100pct", + "task": "cub", + "expert": "glsim", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden", + "fusion": "concat", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "global_local_encoder__layer_12", + "fixed25_metric": 0.8684846392820159, + "fixed25_actual_fraction": 0.2507766655160511, + "fixed25_rescue": 17, + "fixed25_harm": 209, + "rank25_metric": 0.8684846392820159, + "validation_selection": 0.97525, + "best_epoch": 1, + "seconds": 3.935011863708496 + } + ] + }, + { + "key": [ + "grefcoco", + "groundingdino", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0193", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.38158976831239544, + "fixed25_actual_fraction": 0.24836230609987806, + "fixed25_rescue": 4083, + "fixed25_harm": 38, + "rank25_metric": 0.38201514335138814, + "validation_selection": 0.6237472766884531, + "best_epoch": 1, + "seconds": 28.353870153427124 + }, + { + "id": "formal_0194", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3658792502055979, + "fixed25_actual_fraction": 0.22034427019822478, + "fixed25_rescue": 3503, + "fixed25_harm": 12, + "rank25_metric": 0.3758330261180274, + "validation_selection": 0.6238526951999438, + "best_epoch": 2, + "seconds": 56.78019833564758 + }, + { + "id": "formal_0042", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.3693106088534725, + "fixed25_actual_fraction": 0.22853982928281769, + "fixed25_rescue": 3642, + "fixed25_harm": 30, + "rank25_metric": 0.3758330261180274, + "validation_selection": 0.6221730269168599, + "best_epoch": 9, + "seconds": 476.6986792087555 + } + ] + }, + { + "key": [ + "grefcoco", + "groundingdino", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0195", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.4562572668235828, + "fixed25_actual_fraction": 0.24856081445140799, + "fixed25_rescue": 6732, + "fixed25_harm": 54, + "rank25_metric": 0.45705130022970253, + "validation_selection": 0.6656265373532926, + "best_epoch": 1, + "seconds": 18.048375368118286 + }, + { + "id": "formal_0196", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.434648214842753, + "fixed25_actual_fraction": 0.22008904517482913, + "fixed25_rescue": 5930, + "fixed25_harm": 14, + "rank25_metric": 0.4536766582536937, + "validation_selection": 0.6660130718954248, + "best_epoch": 2, + "seconds": 33.53883171081543 + }, + { + "id": "formal_0057", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "groundingdino", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "error", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "text_encoder__layer_09", + "fixed25_metric": 0.43033774778096023, + "fixed25_actual_fraction": 0.2126591611604231, + "fixed25_rescue": 5784, + "fixed25_harm": 20, + "rank25_metric": 0.4544139749879477, + "validation_selection": 0.6650924168950734, + "best_epoch": 5, + "seconds": 281.1101198196411 + } + ] + }, + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0197", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6523268014632901, + "fixed25_actual_fraction": 0.3479284235601055, + "fixed25_rescue": 1394, + "fixed25_harm": 2000, + "rank25_metric": 0.6652582026486685, + "validation_selection": 0.753925082577834, + "best_epoch": 13, + "seconds": 28.585034132003784 + }, + { + "id": "formal_0198", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.6567223435328815, + "fixed25_actual_fraction": 0.3123670703003148, + "fixed25_rescue": 1366, + "fixed25_harm": 1817, + "rank25_metric": 0.662762669086578, + "validation_selection": 0.7576779815868999, + "best_epoch": 13, + "seconds": 44.9037389755249 + }, + { + "id": "formal_0072", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_generation__layer_01", + "fixed25_metric": 0.654765618353515, + "fixed25_actual_fraction": 0.33953435612398264, + "fixed25_rescue": 1510, + "fixed25_harm": 2030, + "rank25_metric": 0.6655985026798628, + "validation_selection": 0.7589219200224893, + "best_epoch": 3, + "seconds": 163.15981340408325 + } + ] + }, + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0199", + "tag": "baseline_25pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7098375067351048, + "fixed25_actual_fraction": 0.10637211808411083, + "fixed25_rescue": 1492, + "fixed25_harm": 70, + "rank25_metric": 0.7019538893457732, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 14.837297677993774 + }, + { + "id": "formal_0200", + "tag": "baseline_50pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7416555596517597, + "fixed25_actual_fraction": 0.4823752942177353, + "fixed25_rescue": 3418, + "fixed25_harm": 874, + "rank25_metric": 0.7207838244051838, + "validation_selection": 0.8144985592803429, + "best_epoch": 1, + "seconds": 21.554462432861328 + }, + { + "id": "formal_0083", + "tag": "baseline_100pct", + "task": "grefcoco", + "expert": "instancevg", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "lateral_features__layer_01", + "fixed25_metric": 0.7090151149930523, + "fixed25_actual_fraction": 0.12330204463602076, + "fixed25_rescue": 1514, + "fixed25_harm": 121, + "rank25_metric": 0.7021523976973031, + "validation_selection": 0.7565746011666316, + "best_epoch": 1, + "seconds": 121.36263680458069 + } + ] + }, + { + "key": [ + "nlvr2", + "nlvr2", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0201", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.863815742268781, + "fixed25_actual_fraction": 0.253928392049939, + "fixed25_rescue": 1100, + "fixed25_harm": 211, + "rank25_metric": 0.8630264762861448, + "validation_selection": 0.8331853337152678, + "best_epoch": 2, + "seconds": 11.065314054489136 + }, + { + "id": "formal_0202", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.868336083805697, + "fixed25_actual_fraction": 0.24388318863456984, + "fixed25_rescue": 1152, + "fixed25_harm": 200, + "rank25_metric": 0.8689818468823993, + "validation_selection": 0.8330850759094816, + "best_epoch": 3, + "seconds": 16.297266960144043 + }, + { + "id": "formal_0091", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_12", + "fixed25_metric": 0.8707756332065725, + "fixed25_actual_fraction": 0.26117528879959817, + "fixed25_rescue": 1197, + "fixed25_harm": 211, + "rank25_metric": 0.8689100954294324, + "validation_selection": 0.8335577198510455, + "best_epoch": 6, + "seconds": 82.51407408714294 + } + ] + }, + { + "key": [ + "nlvr2", + "nlvr2", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0203", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8717801535481093, + "fixed25_actual_fraction": 0.25299562316136903, + "fixed25_rescue": 1180, + "fixed25_harm": 180, + "rank25_metric": 0.8714213962832748, + "validation_selection": 0.8382268690919508, + "best_epoch": 3, + "seconds": 12.40796971321106 + }, + { + "id": "formal_0204", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.872067159359977, + "fixed25_actual_fraction": 0.25407189495587285, + "fixed25_rescue": 1178, + "fixed25_harm": 174, + "rank25_metric": 0.8703451244887709, + "validation_selection": 0.8385276425093097, + "best_epoch": 11, + "seconds": 27.56421136856079 + }, + { + "id": "formal_0104", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "nlvr2", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "multimodal_encoder__layer_10", + "fixed25_metric": 0.8709191361125063, + "fixed25_actual_fraction": 0.25400014350290595, + "fixed25_rescue": 1153, + "fixed25_harm": 165, + "rank25_metric": 0.8695558585061347, + "validation_selection": 0.8383414494414207, + "best_epoch": 7, + "seconds": 91.79227209091187 + } + ] + }, + { + "key": [ + "nlvr2", + "vilt", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0209", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8323886058692689, + "fixed25_actual_fraction": 0.24072612470402527, + "fixed25_rescue": 1189, + "fixed25_harm": 239, + "rank25_metric": 0.8343258950993757, + "validation_selection": 0.7950730449727872, + "best_epoch": 4, + "seconds": 18.022375106811523 + }, + { + "id": "formal_0210", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8337518834756403, + "fixed25_actual_fraction": 0.24704025256511444, + "fixed25_rescue": 1213, + "fixed25_harm": 244, + "rank25_metric": 0.8344693980053096, + "validation_selection": 0.7959610426811802, + "best_epoch": 8, + "seconds": 38.606887102127075 + }, + { + "id": "formal_0139", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "8M", + "branches": "image+text+hidden+confidence", + "fusion": "attention", + "target": "four_state", + "strategy": "natural", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_12", + "fixed25_metric": 0.8327473631341035, + "fixed25_actual_fraction": 0.24208940231039677, + "fixed25_rescue": 1196, + "fixed25_harm": 241, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7952162704096247, + "best_epoch": 3, + "seconds": 136.20692539215088 + } + ] + }, + { + "key": [ + "nlvr2", + "vilt", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0211", + "tag": "baseline_25pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8336083805697065, + "fixed25_actual_fraction": 0.24431369735237138, + "fixed25_rescue": 1234, + "fixed25_harm": 267, + "rank25_metric": 0.8346846523642104, + "validation_selection": 0.7945717559438557, + "best_epoch": 2, + "seconds": 12.189603567123413 + }, + { + "id": "formal_0212", + "tag": "baseline_50pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8308818253569635, + "fixed25_actual_fraction": 0.2361340317141422, + "fixed25_rescue": 1178, + "fixed25_harm": 249, + "rank25_metric": 0.834110640740475, + "validation_selection": 0.7955027212832999, + "best_epoch": 3, + "seconds": 18.873005151748657 + }, + { + "id": "formal_0154", + "tag": "baseline_100pct", + "task": "nlvr2", + "expert": "vilt", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "2M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "ordered_image_text__layer_10", + "fixed25_metric": 0.8329626174930043, + "fixed25_actual_fraction": 0.24094137906292604, + "fixed25_rescue": 1225, + "fixed25_harm": 267, + "rank25_metric": 0.8348281552701442, + "validation_selection": 0.7948438842738471, + "best_epoch": 2, + "seconds": 78.18392181396484 + } + ] + }, + { + "key": [ + "construction", + "yolo26x", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0213", + "tag": "baseline_25pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.245059255252362, + "best_epoch": 2, + "seconds": 2.557858467102051 + }, + { + "id": "formal_0214", + "tag": "baseline_50pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2225771535665814, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 608, + "fixed25_harm": 265, + "rank25_metric": 0.22250151381741792, + "validation_selection": 0.2411747600492345, + "best_epoch": 1, + "seconds": 3.123697280883789 + }, + { + "id": "formal_0169", + "tag": "baseline_100pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.22276943862299337, + "fixed25_actual_fraction": 0.18874833555259654, + "fixed25_rescue": 614, + "fixed25_harm": 271, + "rank25_metric": 0.22272342953859628, + "validation_selection": 0.23173775721145673, + "best_epoch": 4, + "seconds": 9.216264009475708 + } + ] + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0215", + "tag": "baseline_25pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 7, + "seconds": 3.354280948638916 + }, + { + "id": "formal_0216", + "tag": "baseline_50pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.2490976644605033, + "best_epoch": 14, + "seconds": 4.801910877227783 + }, + { + "id": "formal_0177", + "tag": "baseline_100pct", + "task": "construction", + "expert": "yolo26x", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "output", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_14", + "fixed25_metric": 0.2503793990148833, + "fixed25_actual_fraction": 0.18342210386151797, + "fixed25_rescue": 653, + "fixed25_harm": 116, + "rank25_metric": 0.26327645391521537, + "validation_selection": 0.24929743985848513, + "best_epoch": 13, + "seconds": 11.173577785491943 + } + ] + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "MiniCPM" + ], + "rows": [ + { + "id": "formal_0205", + "tag": "baseline_25pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 0.25, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.12161748372078302, + "fixed25_actual_fraction": 0.26897470039946736, + "fixed25_rescue": 1103, + "fixed25_harm": 518, + "rank25_metric": 0.12103479129839005, + "validation_selection": 0.13942018347651794, + "best_epoch": 1, + "seconds": 3.9248127937316895 + }, + { + "id": "formal_0206", + "tag": "baseline_50pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 0.5, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11138731664705415, + "fixed25_actual_fraction": 0.25798934753661784, + "fixed25_rescue": 1337, + "fixed25_harm": 464, + "rank25_metric": 0.11157434949576527, + "validation_selection": 0.10286119045745137, + "best_epoch": 3, + "seconds": 5.573187828063965 + }, + { + "id": "formal_0114", + "tag": "baseline_100pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "MiniCPM", + "fraction": 1.0, + "student": "4M", + "branches": "image+text+hidden+confidence", + "fusion": "concat", + "target": "gain", + "strategy": "weighted", + "representation_dim": 512, + "hidden_layer": "query_decoder__layer_04", + "fixed25_metric": 0.11409050364241882, + "fixed25_actual_fraction": 0.2333555259653795, + "fixed25_rescue": 1155, + "fixed25_harm": 428, + "rank25_metric": 0.1142018021781044, + "validation_selection": 0.09196748903785351, + "best_epoch": 1, + "seconds": 7.1226747035980225 + } + ] + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "Qwen3.8" + ], + "rows": [ + { + "id": "formal_0207", + "tag": "baseline_25pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 0.25, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.14471609214350373, + "fixed25_actual_fraction": 0.2892809587217044, + "fixed25_rescue": 1383, + "fixed25_harm": 258, + "rank25_metric": 0.14346432207238047, + "validation_selection": 0.16233096000534752, + "best_epoch": 17, + "seconds": 8.028136253356934 + }, + { + "id": "formal_0208", + "tag": "baseline_50pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 0.5, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1499641421353766, + "fixed25_actual_fraction": 0.26797603195739017, + "fixed25_rescue": 1348, + "fixed25_harm": 225, + "rank25_metric": 0.14991013725904712, + "validation_selection": 0.14960536227871302, + "best_epoch": 14, + "seconds": 7.761091709136963 + }, + { + "id": "formal_0129", + "tag": "baseline_100pct", + "task": "construction", + "expert": "rtdetr_x_fp32", + "endpoint": "Qwen3.8", + "fraction": 1.0, + "student": "4M", + "branches": "hidden", + "fusion": "concat", + "target": "error", + "strategy": "balanced", + "representation_dim": 512, + "hidden_layer": "feature_blocks__layer_11", + "fixed25_metric": 0.1498069310931111, + "fixed25_actual_fraction": 0.2539946737683089, + "fixed25_rescue": 1284, + "fixed25_harm": 203, + "rank25_metric": 0.15030144907600507, + "validation_selection": 0.14151521056562297, + "best_epoch": 16, + "seconds": 13.48578929901123 + } + ] + } +] \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/shared_search.json b/releases/20260913_lr/paper_data/shared_search.json new file mode 100644 index 0000000000000000000000000000000000000000..9392aa0d85468e9f65ad61fa23018b5535b43532 --- /dev/null +++ b/releases/20260913_lr/paper_data/shared_search.json @@ -0,0 +1,720 @@ +{ + "status": "search_complete", + "started_at": 1789041525.1799183, + "completed_stages": [ + "hidden_percent", + "student", + "representation_dim", + "lr", + "target", + "strategy" + ], + "selected": "shared_12", + "candidates": [ + { + "id": "shared_00", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_00_cub_MiniCPM", + "shared_00_cub_Qwen3.8", + "shared_00_glsim_MiniCPM", + "shared_00_glsim_Qwen3.8", + "shared_00_groundingdino_MiniCPM", + "shared_00_groundingdino_Qwen3.8", + "shared_00_instancevg_MiniCPM", + "shared_00_instancevg_Qwen3.8", + "shared_00_nlvr2_MiniCPM", + "shared_00_nlvr2_Qwen3.8", + "shared_00_vilt_MiniCPM", + "shared_00_vilt_Qwen3.8", + "shared_00_yolo26x_MiniCPM", + "shared_00_yolo26x_Qwen3.8", + "shared_00_rtdetr_x_fp32_MiniCPM", + "shared_00_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9498541666666667, + "grefcoco": 0.7090097687820648, + "nlvr2": 0.8164207963334288, + "construction": 0.175986962717605 + }, + "validation_mean": 0.6628179236249413, + "mean_active_parameters": 4881924.0, + "mean_fit_seconds": 37.63353890167855 + }, + { + "id": "shared_01", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 30 + }, + "runs": [ + "shared_01_cub_MiniCPM", + "shared_01_cub_Qwen3.8", + "shared_01_glsim_MiniCPM", + "shared_01_glsim_Qwen3.8", + "shared_01_groundingdino_MiniCPM", + "shared_01_groundingdino_Qwen3.8", + "shared_01_instancevg_MiniCPM", + "shared_01_instancevg_Qwen3.8", + "shared_01_nlvr2_MiniCPM", + "shared_01_nlvr2_Qwen3.8", + "shared_01_vilt_MiniCPM", + "shared_01_vilt_Qwen3.8", + "shared_01_yolo26x_MiniCPM", + "shared_01_yolo26x_Qwen3.8", + "shared_01_rtdetr_x_fp32_MiniCPM", + "shared_01_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9346041666666667, + "grefcoco": 0.7090431513107035, + "nlvr2": 0.8157691205958177, + "construction": 0.18298715773892013 + }, + "validation_mean": 0.660600899078027, + "mean_active_parameters": 4783620.0, + "mean_fit_seconds": 39.49221354418114 + }, + { + "id": "shared_02", + "settings": { + "student": "4M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 60 + }, + "runs": [ + "shared_02_cub_MiniCPM", + "shared_02_cub_Qwen3.8", + "shared_02_glsim_MiniCPM", + "shared_02_glsim_Qwen3.8", + "shared_02_groundingdino_MiniCPM", + "shared_02_groundingdino_Qwen3.8", + "shared_02_instancevg_MiniCPM", + "shared_02_instancevg_Qwen3.8", + "shared_02_nlvr2_MiniCPM", + "shared_02_nlvr2_Qwen3.8", + "shared_02_vilt_MiniCPM", + "shared_02_vilt_Qwen3.8", + "shared_02_yolo26x_MiniCPM", + "shared_02_yolo26x_Qwen3.8", + "shared_02_rtdetr_x_fp32_MiniCPM", + "shared_02_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9472291666666667, + "grefcoco": 0.708183990442055, + "nlvr2": 0.8155900887997709, + "construction": 0.1758169352446146 + }, + "validation_mean": 0.6617050452882768, + "mean_active_parameters": 4816388.0, + "mean_fit_seconds": 34.18548409075447 + }, + { + "id": "shared_03", + "settings": { + "student": "1M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_03_cub_MiniCPM", + "shared_03_cub_Qwen3.8", + "shared_03_glsim_MiniCPM", + "shared_03_glsim_Qwen3.8", + "shared_03_groundingdino_MiniCPM", + "shared_03_groundingdino_Qwen3.8", + "shared_03_instancevg_MiniCPM", + "shared_03_instancevg_Qwen3.8", + "shared_03_nlvr2_MiniCPM", + "shared_03_nlvr2_Qwen3.8", + "shared_03_vilt_MiniCPM", + "shared_03_vilt_Qwen3.8", + "shared_03_yolo26x_MiniCPM", + "shared_03_yolo26x_Qwen3.8", + "shared_03_rtdetr_x_fp32_MiniCPM", + "shared_03_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9559375, + "grefcoco": 0.7093348091924943, + "nlvr2": 0.8166929246634202, + "construction": 0.173092415345378 + }, + "validation_mean": 0.6637644123003231, + "mean_active_parameters": 2615822.0, + "mean_fit_seconds": 57.72181743498368 + }, + { + "id": "shared_04", + "settings": { + "student": "2M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_04_cub_MiniCPM", + "shared_04_cub_Qwen3.8", + "shared_04_glsim_MiniCPM", + "shared_04_glsim_Qwen3.8", + "shared_04_groundingdino_MiniCPM", + "shared_04_groundingdino_Qwen3.8", + "shared_04_instancevg_MiniCPM", + "shared_04_instancevg_Qwen3.8", + "shared_04_nlvr2_MiniCPM", + "shared_04_nlvr2_Qwen3.8", + "shared_04_vilt_MiniCPM", + "shared_04_vilt_Qwen3.8", + "shared_04_yolo26x_MiniCPM", + "shared_04_yolo26x_Qwen3.8", + "shared_04_rtdetr_x_fp32_MiniCPM", + "shared_04_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9448958333333333, + "grefcoco": 0.7087286527514232, + "nlvr2": 0.8167144084789458, + "construction": 0.1764152469711854 + }, + "validation_mean": 0.661688535383722, + "mean_active_parameters": 3402222.0, + "mean_fit_seconds": 57.64163886100141 + }, + { + "id": "shared_05", + "settings": { + "student": "8M", + "representation_dim": 512, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_05_cub_MiniCPM", + "shared_05_cub_Qwen3.8", + "shared_05_glsim_MiniCPM", + "shared_05_glsim_Qwen3.8", + "shared_05_groundingdino_MiniCPM", + "shared_05_groundingdino_Qwen3.8", + "shared_05_instancevg_MiniCPM", + "shared_05_instancevg_Qwen3.8", + "shared_05_nlvr2_MiniCPM", + "shared_05_nlvr2_Qwen3.8", + "shared_05_vilt_MiniCPM", + "shared_05_vilt_Qwen3.8", + "shared_05_yolo26x_MiniCPM", + "shared_05_yolo26x_Qwen3.8", + "shared_05_rtdetr_x_fp32_MiniCPM", + "shared_05_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9542916666666666, + "grefcoco": 0.7095351043643263, + "nlvr2": 0.8168110856488112, + "construction": 0.17955466204144832 + }, + "validation_mean": 0.6650481296803131, + "mean_active_parameters": 7994758.0, + "mean_fit_seconds": 46.836150376082514 + }, + { + "id": "shared_06", + "settings": { + "student": "8M", + "representation_dim": 32, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_06_cub_MiniCPM", + "shared_06_cub_Qwen3.8", + "shared_06_glsim_MiniCPM", + "shared_06_glsim_Qwen3.8", + "shared_06_groundingdino_MiniCPM", + "shared_06_groundingdino_Qwen3.8", + "shared_06_instancevg_MiniCPM", + "shared_06_instancevg_Qwen3.8", + "shared_06_nlvr2_MiniCPM", + "shared_06_nlvr2_Qwen3.8", + "shared_06_vilt_MiniCPM", + "shared_06_vilt_Qwen3.8", + "shared_06_yolo26x_MiniCPM", + "shared_06_yolo26x_Qwen3.8", + "shared_06_rtdetr_x_fp32_MiniCPM", + "shared_06_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.966125, + "grefcoco": 0.7070665542202544, + "nlvr2": 0.8169149240905185, + "construction": 0.17258963439949287 + }, + "validation_mean": 0.6656740281775665, + "mean_active_parameters": 6222718.0, + "mean_fit_seconds": 63.611915712936025 + }, + { + "id": "shared_07", + "settings": { + "student": "8M", + "representation_dim": 64, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_07_cub_MiniCPM", + "shared_07_cub_Qwen3.8", + "shared_07_glsim_MiniCPM", + "shared_07_glsim_Qwen3.8", + "shared_07_groundingdino_MiniCPM", + "shared_07_groundingdino_Qwen3.8", + "shared_07_instancevg_MiniCPM", + "shared_07_instancevg_Qwen3.8", + "shared_07_nlvr2_MiniCPM", + "shared_07_nlvr2_Qwen3.8", + "shared_07_vilt_MiniCPM", + "shared_07_vilt_Qwen3.8", + "shared_07_yolo26x_MiniCPM", + "shared_07_yolo26x_Qwen3.8", + "shared_07_rtdetr_x_fp32_MiniCPM", + "shared_07_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9693333333333334, + "grefcoco": 0.7083755007379295, + "nlvr2": 0.8167716986536809, + "construction": 0.17603867039292925 + }, + "validation_mean": 0.6676298007794683, + "mean_active_parameters": 6340854.0, + "mean_fit_seconds": 62.082761554156605 + }, + { + "id": "shared_08", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_08_cub_MiniCPM", + "shared_08_cub_Qwen3.8", + "shared_08_glsim_MiniCPM", + "shared_08_glsim_Qwen3.8", + "shared_08_groundingdino_MiniCPM", + "shared_08_groundingdino_Qwen3.8", + "shared_08_instancevg_MiniCPM", + "shared_08_instancevg_Qwen3.8", + "shared_08_nlvr2_MiniCPM", + "shared_08_nlvr2_Qwen3.8", + "shared_08_vilt_MiniCPM", + "shared_08_vilt_Qwen3.8", + "shared_08_yolo26x_MiniCPM", + "shared_08_yolo26x_Qwen3.8", + "shared_08_rtdetr_x_fp32_MiniCPM", + "shared_08_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9704166666666667, + "grefcoco": 0.7082156159955021, + "nlvr2": 0.8169006015468347, + "construction": 0.17553875298671762 + }, + "validation_mean": 0.6677679092989303, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 58.37819891100662 + }, + { + "id": "shared_09", + "settings": { + "student": "8M", + "representation_dim": 256, + "lr": 0.001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_09_cub_MiniCPM", + "shared_09_cub_Qwen3.8", + "shared_09_glsim_MiniCPM", + "shared_09_glsim_Qwen3.8", + "shared_09_groundingdino_MiniCPM", + "shared_09_groundingdino_Qwen3.8", + "shared_09_instancevg_MiniCPM", + "shared_09_instancevg_Qwen3.8", + "shared_09_nlvr2_MiniCPM", + "shared_09_nlvr2_Qwen3.8", + "shared_09_vilt_MiniCPM", + "shared_09_vilt_Qwen3.8", + "shared_09_yolo26x_MiniCPM", + "shared_09_yolo26x_Qwen3.8", + "shared_09_rtdetr_x_fp32_MiniCPM", + "shared_09_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9678541666666667, + "grefcoco": 0.7090659919881931, + "nlvr2": 0.8167609567459181, + "construction": 0.17265650809871438 + }, + "validation_mean": 0.6665844058748731, + "mean_active_parameters": 7049670.0, + "mean_fit_seconds": 50.47316628225235 + }, + { + "id": "shared_10", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.0001, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_10_cub_MiniCPM", + "shared_10_cub_Qwen3.8", + "shared_10_glsim_MiniCPM", + "shared_10_glsim_Qwen3.8", + "shared_10_groundingdino_MiniCPM", + "shared_10_groundingdino_Qwen3.8", + "shared_10_instancevg_MiniCPM", + "shared_10_instancevg_Qwen3.8", + "shared_10_nlvr2_MiniCPM", + "shared_10_nlvr2_Qwen3.8", + "shared_10_vilt_MiniCPM", + "shared_10_vilt_Qwen3.8", + "shared_10_yolo26x_MiniCPM", + "shared_10_yolo26x_Qwen3.8", + "shared_10_rtdetr_x_fp32_MiniCPM", + "shared_10_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9694166666666667, + "grefcoco": 0.7047965422728231, + "nlvr2": 0.8161844743626469, + "construction": 0.17403212405686308 + }, + "validation_mean": 0.66610745183975, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 91.61671701988234 + }, + { + "id": "shared_11", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.0003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_11_cub_MiniCPM", + "shared_11_cub_Qwen3.8", + "shared_11_glsim_MiniCPM", + "shared_11_glsim_Qwen3.8", + "shared_11_groundingdino_MiniCPM", + "shared_11_groundingdino_Qwen3.8", + "shared_11_instancevg_MiniCPM", + "shared_11_instancevg_Qwen3.8", + "shared_11_nlvr2_MiniCPM", + "shared_11_nlvr2_Qwen3.8", + "shared_11_vilt_MiniCPM", + "shared_11_vilt_Qwen3.8", + "shared_11_yolo26x_MiniCPM", + "shared_11_yolo26x_Qwen3.8", + "shared_11_rtdetr_x_fp32_MiniCPM", + "shared_11_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9692083333333333, + "grefcoco": 0.7071210204511913, + "nlvr2": 0.8166034087653967, + "construction": 0.17173346213113433 + }, + "validation_mean": 0.6661665561702639, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 75.3759171601705 + }, + { + "id": "shared_12", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_12_cub_MiniCPM", + "shared_12_cub_Qwen3.8", + "shared_12_glsim_MiniCPM", + "shared_12_glsim_Qwen3.8", + "shared_12_groundingdino_MiniCPM", + "shared_12_groundingdino_Qwen3.8", + "shared_12_instancevg_MiniCPM", + "shared_12_instancevg_Qwen3.8", + "shared_12_nlvr2_MiniCPM", + "shared_12_nlvr2_Qwen3.8", + "shared_12_vilt_MiniCPM", + "shared_12_vilt_Qwen3.8", + "shared_12_yolo26x_MiniCPM", + "shared_12_yolo26x_Qwen3.8", + "shared_12_rtdetr_x_fp32_MiniCPM", + "shared_12_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9698125, + "grefcoco": 0.7082964368543117, + "nlvr2": 0.8164959896877686, + "construction": 0.1768778191883302 + }, + "validation_mean": 0.6678706864326026, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 67.41994623254868 + }, + { + "id": "shared_13", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.01, + "target": "four_state", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_13_cub_MiniCPM", + "shared_13_cub_Qwen3.8", + "shared_13_glsim_MiniCPM", + "shared_13_glsim_Qwen3.8", + "shared_13_groundingdino_MiniCPM", + "shared_13_groundingdino_Qwen3.8", + "shared_13_instancevg_MiniCPM", + "shared_13_instancevg_Qwen3.8", + "shared_13_nlvr2_MiniCPM", + "shared_13_nlvr2_Qwen3.8", + "shared_13_vilt_MiniCPM", + "shared_13_vilt_Qwen3.8", + "shared_13_yolo26x_MiniCPM", + "shared_13_yolo26x_Qwen3.8", + "shared_13_rtdetr_x_fp32_MiniCPM", + "shared_13_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9509375, + "grefcoco": 0.7100024597652681, + "nlvr2": 0.8169936980807792, + "construction": 0.17572600052962756 + }, + "validation_mean": 0.6634149145939187, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 58.952753572441 + }, + { + "id": "shared_14", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "gain", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_14_cub_MiniCPM", + "shared_14_cub_Qwen3.8", + "shared_14_glsim_MiniCPM", + "shared_14_glsim_Qwen3.8", + "shared_14_groundingdino_MiniCPM", + "shared_14_groundingdino_Qwen3.8", + "shared_14_instancevg_MiniCPM", + "shared_14_instancevg_Qwen3.8", + "shared_14_nlvr2_MiniCPM", + "shared_14_nlvr2_Qwen3.8", + "shared_14_vilt_MiniCPM", + "shared_14_vilt_Qwen3.8", + "shared_14_yolo26x_MiniCPM", + "shared_14_yolo26x_Qwen3.8", + "shared_14_rtdetr_x_fp32_MiniCPM", + "shared_14_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.97975, + "grefcoco": 0.6943653805608265, + "nlvr2": 0.815819249498711, + "construction": 0.17119724948093312 + }, + "validation_mean": 0.6652829698851177, + "mean_active_parameters": 6576739.0, + "mean_fit_seconds": 46.42365771972982 + }, + { + "id": "shared_15", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "error", + "strategy": "natural", + "hidden_percent": 90 + }, + "runs": [ + "shared_15_cub_MiniCPM", + "shared_15_cub_Qwen3.8", + "shared_15_glsim_MiniCPM", + "shared_15_glsim_Qwen3.8", + "shared_15_groundingdino_MiniCPM", + "shared_15_groundingdino_Qwen3.8", + "shared_15_instancevg_MiniCPM", + "shared_15_instancevg_Qwen3.8", + "shared_15_nlvr2_MiniCPM", + "shared_15_nlvr2_Qwen3.8", + "shared_15_vilt_MiniCPM", + "shared_15_vilt_Qwen3.8", + "shared_15_yolo26x_MiniCPM", + "shared_15_yolo26x_Qwen3.8", + "shared_15_rtdetr_x_fp32_MiniCPM", + "shared_15_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9172083333333333, + "grefcoco": 0.707660411834985, + "nlvr2": 0.8152570896591235, + "construction": 0.17163397920287538 + }, + "validation_mean": 0.6529399535075793, + "mean_active_parameters": 6576739.0, + "mean_fit_seconds": 39.98559669695533 + }, + { + "id": "shared_16", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "weighted", + "hidden_percent": 90 + }, + "runs": [ + "shared_16_cub_MiniCPM", + "shared_16_cub_Qwen3.8", + "shared_16_glsim_MiniCPM", + "shared_16_glsim_Qwen3.8", + "shared_16_groundingdino_MiniCPM", + "shared_16_groundingdino_Qwen3.8", + "shared_16_instancevg_MiniCPM", + "shared_16_instancevg_Qwen3.8", + "shared_16_nlvr2_MiniCPM", + "shared_16_nlvr2_Qwen3.8", + "shared_16_vilt_MiniCPM", + "shared_16_vilt_Qwen3.8", + "shared_16_yolo26x_MiniCPM", + "shared_16_yolo26x_Qwen3.8", + "shared_16_rtdetr_x_fp32_MiniCPM", + "shared_16_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.958625, + "grefcoco": 0.7108879752617893, + "nlvr2": 0.8163420223431682, + "construction": 0.17381623998937223 + }, + "validation_mean": 0.6649178093985824, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 39.95715373130224 + }, + { + "id": "shared_17", + "settings": { + "student": "8M", + "representation_dim": 128, + "lr": 0.003, + "target": "four_state", + "strategy": "balanced", + "hidden_percent": 90 + }, + "runs": [ + "shared_17_cub_MiniCPM", + "shared_17_cub_Qwen3.8", + "shared_17_glsim_MiniCPM", + "shared_17_glsim_Qwen3.8", + "shared_17_groundingdino_MiniCPM", + "shared_17_groundingdino_Qwen3.8", + "shared_17_instancevg_MiniCPM", + "shared_17_instancevg_Qwen3.8", + "shared_17_nlvr2_MiniCPM", + "shared_17_nlvr2_Qwen3.8", + "shared_17_vilt_MiniCPM", + "shared_17_vilt_Qwen3.8", + "shared_17_yolo26x_MiniCPM", + "shared_17_yolo26x_Qwen3.8", + "shared_17_rtdetr_x_fp32_MiniCPM", + "shared_17_rtdetr_x_fp32_Qwen3.8" + ], + "status": "complete", + "task_means": { + "cub": 0.9554583333333334, + "grefcoco": 0.7062583456321596, + "nlvr2": 0.8156903466055572, + "construction": 0.17510649266382114 + }, + "validation_mean": 0.6631283795587178, + "mean_active_parameters": 6577126.0, + "mean_fit_seconds": 35.547329766683106 + } + ], + "failures": [], + "test_loaded": false, + "finished_at": 1789043234.371715 +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/table_audit.json b/releases/20260913_lr/paper_data/table_audit.json new file mode 100644 index 0000000000000000000000000000000000000000..c2ef8f466490ba6790a8c535fc8193e4333d087a --- /dev/null +++ b/releases/20260913_lr/paper_data/table_audit.json @@ -0,0 +1,443 @@ +{ + "counts": { + "better_metric": 11, + "fewer_calls": 9, + "both": 8, + "above_endpoints": 10, + "better_ranked_area": 16 + }, + "rows": [ + { + "key": [ + "cub", + "cub", + "MiniCPM" + ], + "validation": { + "spec": 98.58333333333333, + "vlm": 27.750000000000004, + "conf": 98.58333333333333, + "call_conf": 0.0, + "learned": 98.61666666666666, + "call_learned": 0.4, + "change_pp": 0.4, + "metric_change_pp": 0.03333333333332966 + }, + "test": { + "spec": 81.429064549534, + "vlm": 26.561960648947185, + "conf": 81.429064549534, + "call_conf": 0.0, + "learned": 81.42906454953402, + "call_learned": 0.5937176389368313, + "change_pp": 0.5937176389368313, + "metric_change_pp": 1.1102230246251565e-14 + } + }, + { + "key": [ + "cub", + "cub", + "Qwen3.8" + ], + "validation": { + "spec": 98.58333333333333, + "vlm": 55.08333333333333, + "conf": 98.58333333333333, + "call_conf": 0.0, + "learned": 98.58333333333334, + "call_learned": 0.0, + "change_pp": 0.0, + "metric_change_pp": 1.1102230246251565e-14 + }, + "test": { + "spec": 81.429064549534, + "vlm": 54.81532619951675, + "conf": 81.429064549534, + "call_conf": 0.0, + "learned": 81.42906454953402, + "call_learned": 0.0, + "change_pp": 0.0, + "metric_change_pp": 1.1102230246251565e-14 + } + }, + { + "key": [ + "cub", + "glsim", + "MiniCPM" + ], + "validation": { + "spec": 98.91666666666666, + "vlm": 27.750000000000004, + "conf": 98.91666666666666, + "call_conf": 0.0, + "learned": 98.94999999999999, + "call_learned": 0.4, + "change_pp": 0.4, + "metric_change_pp": 0.03333333333332966 + }, + "test": { + "spec": 90.16223679668623, + "vlm": 26.561960648947185, + "conf": 90.16223679668623, + "call_conf": 0.0, + "learned": 90.13117017604418, + "call_learned": 0.8491542975491888, + "change_pp": 0.8491542975491888, + "metric_change_pp": -0.031066620642039577 + } + }, + { + "key": [ + "cub", + "glsim", + "Qwen3.8" + ], + "validation": { + "spec": 98.91666666666666, + "vlm": 55.08333333333333, + "conf": 98.91666666666666, + "call_conf": 0.0, + "learned": 98.91666666666666, + "call_learned": 0.0, + "change_pp": 0.0, + "metric_change_pp": -1.1102230246251565e-14 + }, + "test": { + "spec": 90.16223679668623, + "vlm": 54.81532619951675, + "conf": 90.16223679668623, + "call_conf": 0.0, + "learned": 90.16223679668623, + "call_learned": 0.0, + "change_pp": 0.0, + "metric_change_pp": 1.1102230246251565e-14 + } + }, + { + "key": [ + "grefcoco", + "groundingdino", + "MiniCPM" + ], + "validation": { + "spec": 56.05453650994448, + "vlm": 64.12256658935975, + "conf": 64.12256658935975, + "call_conf": 100.0, + "learned": 65.9076533839342, + "call_learned": 45.79520697167756, + "change_pp": -54.20479302832244, + "metric_change_pp": 1.785086794574453 + }, + "test": { + "spec": 26.688029946402747, + "vlm": 50.00708958398321, + "conf": 50.00708958398321, + "call_conf": 100.0, + "learned": 50.22828460425942, + "call_learned": 58.18676800045374, + "change_pp": -41.81323199954626, + "metric_change_pp": 0.2211950202762103 + } + }, + { + "key": [ + "grefcoco", + "groundingdino", + "Qwen3.8" + ], + "validation": { + "spec": 56.05453650994448, + "vlm": 80.60299388572633, + "conf": 80.60299388572633, + "call_conf": 100.0, + "learned": 81.40698573336145, + "call_learned": 77.59786351816713, + "change_pp": -22.40213648183287, + "metric_change_pp": 0.8039918476351104 + }, + "test": { + "spec": 26.688029946402747, + "vlm": 74.47182599325072, + "conf": 74.47182599325072, + "call_conf": 100.0, + "learned": 75.12179905283158, + "call_learned": 88.46666477611093, + "change_pp": -11.533335223889074, + "metric_change_pp": 0.649973059580855 + } + }, + { + "key": [ + "grefcoco", + "instancevg", + "MiniCPM" + ], + "validation": { + "spec": 75.02986857825567, + "vlm": 64.12256658935975, + "conf": 77.16635041113219, + "call_conf": 64.99402628434886, + "learned": 75.66800196781222, + "call_learned": 14.395951929158763, + "change_pp": -50.59807435519011, + "metric_change_pp": -1.4983484433199745 + }, + "test": { + "spec": 66.95119530385956, + "vlm": 50.00708958398321, + "conf": 61.877889005473165, + "call_conf": 52.09426310864078, + "learned": 67.91367722542041, + "call_learned": 19.038085245157816, + "change_pp": -33.056177863482965, + "metric_change_pp": 6.035788219947246 + } + }, + { + "key": [ + "grefcoco", + "instancevg", + "Qwen3.8" + ], + "validation": { + "spec": 75.02986857825567, + "vlm": 80.60299388572633, + "conf": 82.01560193970062, + "call_conf": 70.99585353854803, + "learned": 83.38042026846581, + "call_learned": 45.59561458992199, + "change_pp": -25.40023894862604, + "metric_change_pp": 1.364818328765205 + }, + "test": { + "spec": 66.95119530385956, + "vlm": 74.47182599325072, + "conf": 72.43286163967898, + "call_conf": 68.38612710206165, + "learned": 76.24365482233502, + "call_learned": 62.00153135014037, + "change_pp": -6.384595751921285, + "metric_change_pp": 3.810793182656036 + } + }, + { + "key": [ + "nlvr2", + "nlvr2", + "MiniCPM" + ], + "validation": { + "spec": 79.81953594958465, + "vlm": 89.90260670295045, + "conf": 90.04583213978803, + "call_conf": 85.87797192781437, + "learned": 89.99140647378974, + "call_learned": 91.39215124606132, + "change_pp": 5.514179318246937, + "metric_change_pp": -0.05442566599829224 + }, + "test": { + "spec": 80.00287005811867, + "vlm": 90.7368874219703, + "conf": 90.77276314845376, + "call_conf": 85.80038745784601, + "learned": 90.80002870058121, + "call_learned": 91.10425486116092, + "change_pp": 5.303867403314911, + "metric_change_pp": 0.027265552127442305 + } + }, + { + "key": [ + "nlvr2", + "nlvr2", + "Qwen3.8" + ], + "validation": { + "spec": 79.81953594958465, + "vlm": 90.50415353766829, + "conf": 90.57576625608708, + "call_conf": 85.87797192781437, + "learned": 91.08851331996561, + "call_learned": 84.19077628186767, + "change_pp": -1.6871956459467152, + "metric_change_pp": 0.5127470638785381 + }, + "test": { + "spec": 80.00287005811867, + "vlm": 90.06242376408122, + "conf": 90.24180239649853, + "call_conf": 85.80038745784601, + "learned": 90.60055966133314, + "call_learned": 84.34096290449882, + "change_pp": -1.4594245533471994, + "metric_change_pp": 0.35875726483460824 + } + }, + { + "key": [ + "nlvr2", + "vilt", + "MiniCPM" + ], + "validation": { + "spec": 75.56574047550845, + "vlm": 89.90260670295045, + "conf": 89.97421942136924, + "call_conf": 95.9896877685477, + "learned": 89.93125179031794, + "call_learned": 98.19535949584647, + "change_pp": 2.205671727298786, + "metric_change_pp": -0.042967631051293864 + }, + "test": { + "spec": 76.42247255506925, + "vlm": 90.7368874219703, + "conf": 90.64361053311329, + "call_conf": 95.93169261677549, + "learned": 90.69383655019013, + "call_learned": 98.05984071177441, + "change_pp": 2.1281480949989273, + "metric_change_pp": 0.050226017076837604 + } + }, + { + "key": [ + "nlvr2", + "vilt", + "Qwen3.8" + ], + "validation": { + "spec": 75.56574047550845, + "vlm": 90.50415353766829, + "conf": 90.59008879977084, + "call_conf": 95.9896877685477, + "learned": 90.50415353766829, + "call_learned": 99.59896877685478, + "change_pp": 3.6092810083070748, + "metric_change_pp": -0.08593526210255442 + }, + "test": { + "spec": 76.42247255506925, + "vlm": 90.06242376408122, + "conf": 89.96197172992753, + "call_conf": 95.93169261677549, + "learned": 90.0652938221999, + "call_learned": 99.61684724115663, + "change_pp": 3.6851546243811484, + "metric_change_pp": 0.10332209227237321 + } + }, + { + "key": [ + "construction", + "yolo26x", + "MiniCPM" + ], + "validation": { + "spec": 18.254072168965788, + "vlm": 20.666016863629594, + "conf": 22.641590025257546, + "call_conf": 93.86590584878745, + "learned": 28.70655827112144, + "call_learned": 45.73466476462197, + "change_pp": -48.131241084165474, + "metric_change_pp": 6.064968245863894 + }, + "test": { + "spec": 18.129624718318375, + "vlm": 22.361115727328066, + "conf": 25.23047077304935, + "call_conf": 92.54327563249002, + "learned": 25.880436219165098, + "call_learned": 45.31957390146471, + "change_pp": -47.22370173102531, + "metric_change_pp": 0.6499654461157467 + } + }, + { + "key": [ + "construction", + "yolo26x", + "Qwen3.8" + ], + "validation": { + "spec": 18.254072168965788, + "vlm": 25.448626222520154, + "conf": 26.42878791936722, + "call_conf": 92.86733238231099, + "learned": 32.79802334681434, + "call_learned": 45.135520684736086, + "change_pp": -47.7318116975749, + "metric_change_pp": 6.3692354274471175 + }, + "test": { + "spec": 18.129624718318375, + "vlm": 27.001009067886244, + "conf": 25.974900047621052, + "call_conf": 92.21038615179761, + "learned": 29.79678330127048, + "call_learned": 44.766977363515316, + "change_pp": -47.44340878828229, + "metric_change_pp": 3.8218832536494274 + } + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "MiniCPM" + ], + "validation": { + "spec": 7.641657905799261, + "vlm": 20.666016863629594, + "conf": 20.666016863629594, + "call_conf": 100.0, + "learned": 21.991914076339864, + "call_learned": 77.28958630527816, + "change_pp": -22.710413694721833, + "metric_change_pp": 1.3258972127102697 + }, + "test": { + "spec": 7.8873978710439285, + "vlm": 22.361115727328066, + "conf": 22.361115727328066, + "call_conf": 100.0, + "learned": 22.79931433315267, + "call_learned": 78.43541944074566, + "change_pp": -21.56458055925433, + "metric_change_pp": 0.4381986058246051 + } + }, + { + "key": [ + "construction", + "rtdetr_x_fp32", + "Qwen3.8" + ], + "validation": { + "spec": 7.641657905799261, + "vlm": 25.448626222520154, + "conf": 25.448626222520154, + "call_conf": 100.0, + "learned": 27.22897443613517, + "call_learned": 83.08131241084166, + "change_pp": -16.918687589158342, + "metric_change_pp": 1.7803482136150184 + }, + "test": { + "spec": 7.8873978710439285, + "vlm": 27.001009067886244, + "conf": 27.001009067886244, + "call_conf": 100.0, + "learned": 26.783631605767233, + "call_learned": 83.56857523302264, + "change_pp": -16.431424766977365, + "metric_change_pp": -0.21737746211901166 + } + } + ] +} \ No newline at end of file diff --git a/releases/20260913_lr/paper_data/test_results.json b/releases/20260913_lr/paper_data/test_results.json new file mode 100644 index 0000000000000000000000000000000000000000..f99d28bd706dabbc69c8f0b3389422e4507e19c1 --- /dev/null +++ b/releases/20260913_lr/paper_data/test_results.json @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:22185d5d24bc4905d6431755b0e9c8de5c4e693d3732312253caae56889e6f40 +size 49988686 diff --git a/releases/20260913_lr/paper_data/validation_manifest.json b/releases/20260913_lr/paper_data/validation_manifest.json new file mode 100644 index 0000000000000000000000000000000000000000..113fad776dbebb32a83e53fb06815b21b6102f6d --- /dev/null +++ b/releases/20260913_lr/paper_data/validation_manifest.json @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8e9f51ffe2c639717e0313052c1146047db06d51be8768c9a0079511734228e5 +size 36532970 diff --git a/releases/20260913_lr/replay_five_seed_results.py b/releases/20260913_lr/replay_five_seed_results.py new file mode 100644 index 0000000000000000000000000000000000000000..83adba208db47bb87535814a1f3ea552dcfd969e --- /dev/null +++ b/releases/20260913_lr/replay_five_seed_results.py @@ -0,0 +1,434 @@ +"""Replay the revised paper from saved router outputs and endpoint outcomes. + +Requires Python 3.10+ and NumPy, without GPUs, model weights, or network access. +Run from a downloaded/extracted release: + python scripts/replay_five_seed_results.py --artifacts . --output-dir replayed + +All reported values are fractions (multiply by 100 for paper percentages). +Recipe selection reads validation scores only. Test outcomes are evaluated after +selection. This numerical separation does not erase the disclosed study history. +""" +import argparse +import csv +import json +from pathlib import Path + +import numpy as np + + +SEEDS = [2026, 2027, 2028, 2029, 2030] +TASKS = ["cub", "grefcoco", "nlvr2", "construction"] +METHODS = ["error", "four_difference", "four_ratio", "gain", "route"] +METRICS = ["validation_score25", "test_fixed_score25", "test_ranked_score25", + "validation_metric", "test_metric", "validation_call", "test_call"] +RANK_BUDGETS = [0, .05, .10, .15, .20, .25, .50, .75, 1.] +FIXED_BUDGETS = [i / 100 for i in range(101)] +TOL = 1e-12 + + +def read_json(path): + return json.loads(path.read_text(encoding="utf-8")) + + +def check_close(actual, expected, context, tolerance=TOL): + if not np.isfinite(actual) or abs(actual - expected) > tolerance: + raise ValueError(f"{context}: replayed={actual!r}, recorded={expected!r}") + + +def scores_from_logits(raw, target, ranking): + if target == "four_state": + if ranking == "log_ratio": + return raw[:, 1] - raw[:, 2] + if ranking == "probability_difference": + exp = np.exp(raw - raw.max(axis=1, keepdims=True)) + prob = exp / exp.sum(axis=1, keepdims=True) + return prob[:, 1] - prob[:, 2] + elif target in ("error", "route"): + # Stable sigmoid with the original float32 output precision. + x = raw[:, 0] + return np.where(x >= 0, 1 / (1 + np.exp(-np.abs(x))), + np.exp(-np.abs(x)) / (1 + np.exp(-np.abs(x)))) + elif target == "gain": + return raw[:, 0] + raise ValueError((target, ranking)) + + +def saved_scores(run, raw, target, ranking, split): + # Saved with the original PyTorch CPU scoring operations, preserving exact + # float32 threshold ties. NumPy independently checks the scoring formula. + with np.load(run / "replay_scores.npz") as data: + scores = data[split + "_" + ranking] + calculated = scores_from_logits(raw, target, ranking) + if scores.shape != calculated.shape or not np.allclose(scores, calculated, rtol=0, atol=8e-7): + raise ValueError(f"{run.name}/{split}/{ranking}: raw outputs do not reproduce saved scores") + return scores + + +def prepare_curve(scores, small, large, small_correct, large_correct): + """Sufficient statistics for routing the first k items in stable score order.""" + if len(scores) != len(small) or small.shape != large.shape: + raise ValueError("Router output and endpoint outcome dimensions disagree") + if not np.all(np.isfinite(scores)): + raise ValueError("Non-finite saved router output") + order = np.argsort(-scores, kind="stable") + if not (small.ndim == 1 or (small.ndim == 3 and small.shape[1:] == (4, 3))): + raise ValueError(f"Unexpected endpoint outcomes shape: {small.shape}") + # Correctness labels are separate from F1 sufficient statistics: a failed + # prediction can have an all-zero TP/FP/FN triple yet be marked incorrect. + s, l = small_correct, large_correct + if s.shape != l.shape or len(s) != len(scores) or s.ndim != 2: + raise ValueError("Endpoint correctness label dimensions disagree") + rescue, harm = (s == 0) & (l == 1), (s == 1) & (l == 0) + delta = large - small + cumulative = np.concatenate([np.zeros_like(delta[:1]), np.cumsum(delta[order], axis=0)]) + events = np.stack([rescue.sum(1), harm.sum(1), (s == l).sum(1)], axis=1) + event_cumulative = np.concatenate([np.zeros((1, 3), dtype=np.int64), + np.cumsum(events[order], axis=0)]) + return {"scores": scores[order], "base": small.sum(axis=0), + "delta": cumulative, "events": event_cumulative, + "total_rescue": int(rescue.sum()), "n": len(scores)} + + +def point_at_count(prepared, count, budget): + total = prepared["base"] + prepared["delta"][count] + if np.ndim(total) == 0: + metric = float(total / prepared["n"]) + else: + tp, fp, fn = total.T + den = 2 * tp + fp + fn + metric = float(np.divide(2 * tp, den, out=np.zeros_like(tp), where=den > 0).mean()) + rescue, harm, wasted = prepared["events"][count] + return {"budget": budget, "actual_fraction": count / prepared["n"], "metric": metric, + "rescue": int(rescue), "harm": int(harm), + "missed_rescue": prepared["total_rescue"] - int(rescue), "wasted_events": int(wasted)} + + +def ranked_curve(prepared): + return [point_at_count(prepared, int(np.floor(prepared["n"] * b)), b) for b in RANK_BUDGETS] + + +def score25(curve): + return float(sum((a["metric"] + b["metric"]) * .5 * (b["budget"] - a["budget"]) + for a, b in zip(curve[:5], curve[1:6])) / .25) + + +def validation_thresholds(prepared): + ordered, n = prepared["scores"], prepared["n"] + thresholds = [] + for budget in FIXED_BUDGETS: + count = int(np.floor(n * budget)) + if count == 0: + threshold, operator = None, "never" + elif count == n: + threshold, operator = None, "always" + else: + threshold = float(ordered[count - 1]) + count_ge = int(np.searchsorted(-ordered, -threshold, side="right")) + operator = "ge" if count_ge <= count else "gt" + thresholds.append({"budget": budget, "threshold": threshold, "operator": operator}) + return thresholds + + +def fixed_curve(prepared, thresholds): + curve = [] + for item in thresholds: + op = item["operator"] + if op == "never": + count = 0 + elif op == "always": + count = prepared["n"] + elif op in ("ge", "gt"): + count = int(np.searchsorted(-prepared["scores"], -item["threshold"], + side="right" if op == "ge" else "left")) + else: + raise ValueError(op) + point = point_at_count(prepared, count, item["budget"]) + point.pop("wasted_events") + curve.append(dict(point, **{"threshold": item["threshold"], "operator": op})) + return curve + + +def verify_curve(actual, recorded, context): + if len(actual) != len(recorded): + raise ValueError(f"{context}: curve length mismatch") + for i, (a, b) in enumerate(zip(actual, recorded)): + for field in ("budget", "metric", "actual_fraction", "rescue", "harm", "missed_rescue"): + check_close(a[field], b[field], f"{context}[{i}].{field}") + if "wasted_events" in b: + check_close(a["wasted_events"], b["wasted_events"], f"{context}[{i}].wasted_events") + if "operator" in b: + if a["operator"] != b["operator"]: + raise ValueError(f"{context}[{i}]: threshold tie operator changed") + if a["threshold"] is None or b["threshold"] is None: + if a["threshold"] != b["threshold"]: + raise ValueError(f"{context}[{i}]: threshold endpoint changed") + else: + check_close(a["threshold"], b["threshold"], f"{context}[{i}].threshold") + + +def selected_index(curve): + # Exactly the evaluator's rule: metric, fewer calls, then smaller budget. + # No approximate tie is inserted into experimental threshold selection. + return min(range(1, 100), key=lambda i: (-curve[i]["metric"], + curve[i]["actual_fraction"], curve[i]["budget"])) + + +def aggregate_objectives(primary, metrics): + output = [] + for method in METHODS: + row = {"method": method} + for metric in metrics: + values = [] + for seed in SEEDS: + task_means = [] + for task in TASKS: + group = [r[metric] for r in primary if r["method"] == method + and r["seed"] == seed and r["task"] == task] + if len(group) != 4: + raise ValueError((method, seed, task, "expected four model pairs", len(group))) + task_means.append(float(np.mean(group))) + values.append(float(np.mean(task_means))) + row[metric + "_mean"] = float(np.mean(values)) + row[metric + "_std"] = float(np.std(values, ddof=1)) + row["validation_mean_minus_std"] = row["validation_score25_mean"] - row["validation_score25_std"] + output.append(row) + return output + + +def write_csv(path, rows): + with path.open("w", newline="", encoding="utf-8") as stream: + writer = csv.DictWriter(stream, fieldnames=list(rows[0])) + writer.writeheader() + writer.writerows(rows) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--artifacts", type=Path, required=True) + parser.add_argument("--output-dir", type=Path, default=Path("replayed")) + args = parser.parse_args() + root = args.artifacts / "experiment_data" + paper = read_json(args.artifacts / "paper_results" / "data" / "results.json") + lock = read_json(root / "method_lock_five_seeds.json") + runs = sorted((root / "runs").glob("repeat_*")) + if len(runs) != 400 or lock["repeat_seeds"] != SEEDS: + raise ValueError("The complete release requires 400 runs and seeds 2026–2030") + outcomes, validation_rows, replay, configurations, confidence_scores = {}, [], {}, {}, {} + for run in runs: + cfg = read_json(run / "config.json") + configurations[run.name] = cfg + pair = f"shared_12_{cfg['expert']}_{cfg['endpoint']}" + if pair not in outcomes: + outcomes[pair] = {} + confidence_scores[pair] = {} + for split in ("val", "test"): + with np.load(root / "pairs" / pair / (split + ".npz")) as data: + outcomes[pair][split] = (data["small"], data["large"], + data["small_correct"], data["large_correct"]) + confidence_scores[pair][split] = np.where(data["confidence_valid"], -data["confidence"], 1.) + with np.load(run / "paired_logits.npz") as data: + raw = data["validation"] + rankings = ["log_ratio", "probability_difference"] if cfg["target"] == "four_state" else [cfg["ranking"]] + done = read_json(run / "completion.json") + if done["status"] != "validation_complete" or done["test_loaded"]: + raise ValueError(f"{run.name}: invalid training-completion evidence") + replay[run.name] = {} + for ranking in rankings: + scores = saved_scores(run, raw, cfg["target"], ranking, "validation") + prepared = prepare_curve(scores, *outcomes[pair]["val"]) + thresholds = validation_thresholds(prepared) + curve = fixed_curve(prepared, thresholds) + area = score25(ranked_curve(prepared)) + best = selected_index(curve) + replay[run.name][ranking] = {"validation_score25": area, "validation_curve": curve, + "validation_selected": curve[best], "best": best} + if ranking == cfg["ranking"]: + check_close(area, done["validation_selection_score"], f"{run.name}: training validation Score25") + validation_rows.append({"task": cfg["task"], "expert": cfg["expert"], + "endpoint": cfg["endpoint"], "method": cfg["method"], + "seed": cfg["seed"], "validation_score25": area}) + if len({(r["expert"], r["endpoint"], r["method"], r["seed"]) for r in validation_rows}) != 400: + raise ValueError("Duplicate or missing five-seed run") + validation_summary = aggregate_objectives(validation_rows, ["validation_score25"]) + selected = min(validation_summary, key=lambda r: (-r["validation_mean_minus_std"], + r["method"] != "four_ratio", r["method"]))["method"] + if selected != lock["selected_method"]: + raise ValueError(f"Validation-only recipe selection disagrees: {selected}") + for summary in validation_summary: + expected = next(r for r in lock["comparison"] if r["method"] == summary["method"]) + for field, recorded in [("validation_score25_mean", "validation_mean"), + ("validation_score25_std", "validation_std"), + ("validation_mean_minus_std", "selection_value")]: + check_close(summary[field], expected[recorded], f"method selection {summary['method']} {field}") + print(f"Validation-only recipe selection verified: {selected}", flush=True) + + per_seed, curve_points = [], 0 + for run_index, run in enumerate(runs): + cfg = configurations[run.name] + pair = f"shared_12_{cfg['expert']}_{cfg['endpoint']}" + ev = read_json(run / "test_evaluation.json") + if ev["config"] != cfg: + raise ValueError(f"{run.name}: evaluation configuration differs") + with np.load(run / "paired_logits.npz") as data: + raw = data["test"] + for ranking, result in replay[run.name].items(): + expected = ev["comparisons"][ranking] + scores = saved_scores(run, raw, cfg["target"], ranking, "test") + prepared = prepare_curve(scores, *outcomes[pair]["test"]) + curve = fixed_curve(prepared, result["validation_curve"]) + exact = ranked_curve(prepared) + result["test_curve"] = curve + result["test_selected"] = curve[result["best"]] + result["test_ranked_score25"] = score25(exact) + result["test_fixed_score25"] = sum((curve[i]["metric"] + curve[i + 5]["metric"]) * .5 * .05 + for i in range(0, 25, 5)) / .25 + for field, points in [("validation_curve", result["validation_curve"]), + ("test_curve", curve), ("test_exact_budget_diagnostic", exact)]: + verify_curve(points, expected[field], f"{run.name}/{ranking}/{field}") + curve_points += len(points) + for field in ("validation_selected", "test_selected"): + verify_curve([result[field]], [expected[field]], f"{run.name}/{ranking}/{field}") + for field in ("validation_score25", "test_fixed_score25", "test_ranked_score25"): + check_close(result[field], expected[field], f"{run.name}/{ranking}/{field}") + check_close(point_at_count(prepared, 0, 0)["metric"], ev["small_test"], run.name + "/specialist") + check_close(point_at_count(prepared, prepared["n"], 1)["metric"], ev["large_test"], run.name + "/VLM") + row = {k: cfg[k] for k in ("task", "expert", "endpoint", "method", "seed")} + row.update({"ranking": ranking, "same_checkpoint_control": ranking != cfg["ranking"]}) + for metric in METRICS[:3]: + row[metric] = result[metric] + for split in ("validation", "test"): + row[split + "_metric"] = result[split + "_selected"]["metric"] + row[split + "_call"] = result[split + "_selected"]["actual_fraction"] + row.update({key: result["validation_selected"][key] for key in ("threshold", "operator")}) + row["nominal_budget"] = result["validation_selected"]["budget"] + per_seed.append(row) + if (run_index + 1) % 50 == 0: + print(f"Verified {run_index + 1}/400 runs", flush=True) + + primary = [r for r in per_seed if not r["same_checkpoint_control"]] + objectives = aggregate_objectives(primary, METRICS) + with (root / "objective_mean_std.csv").open(encoding="utf-8", newline="") as stream: + recorded_objectives = {r["method"]: r for r in csv.DictReader(stream)} + for row in objectives: + for key, value in row.items(): + if key != "method": + check_close(value, float(recorded_objectives[row["method"]][key]), + f"published objective summary {row['method']}/{key}") + with (root / "per_seed.csv").open(encoding="utf-8", newline="") as stream: + recorded_runs = {(r["expert"], r["endpoint"], r["method"], int(r["seed"]), r["ranking"]): r + for r in csv.DictReader(stream)} + if len(recorded_runs) != len(per_seed): + raise ValueError("Published per-seed summary has a different number of comparisons") + for row in per_seed: + key = tuple(row[k] for k in ("expert", "endpoint", "method", "seed", "ranking")) + for field in METRICS + ["threshold", "nominal_budget"]: + check_close(row[field], float(recorded_runs[key][field]), f"published per-seed summary {key}/{field}") + if row["operator"] != recorded_runs[key]["operator"]: + raise ValueError(f"Published per-seed summary operator differs: {key}") + with (root / "pair_mean_std.csv").open(encoding="utf-8", newline="") as stream: + recorded_pairs = list(csv.DictReader(stream)) + if len(recorded_pairs) != 80: + raise ValueError("The full objective ablation requires 80 model-pair/objective rows") + for expected in recorded_pairs: + rr = [r for r in primary if all(r[k] == expected[k] for k in ("task", "expert", "endpoint", "method"))] + if sorted(r["seed"] for r in rr) != SEEDS: + raise ValueError(f"Incomplete pair/objective summary: {expected}") + for metric in METRICS: + values = [r[metric] for r in rr] + check_close(float(np.mean(values)), float(expected[metric + "_mean"]), + f"published full ablation {expected['expert']}/{expected['endpoint']}/{expected['method']}/{metric}/mean") + check_close(float(np.std(values, ddof=1)), float(expected[metric + "_std"]), + f"published full ablation {expected['expert']}/{expected['endpoint']}/{expected['method']}/{metric}/SD") + main_rows, confidence_curve_points = [], 0 + counts = {"higher_metric_than_confidence": 0, "fewer_calls_than_confidence": 0, + "higher_metric_and_fewer_calls": 0, "above_both_endpoints": 0} + for group in paper["groups"]: + task, expert, endpoint = group["key"] + rr = [r for r in primary if (r["task"], r["expert"], r["endpoint"], r["method"]) + == (task, expert, endpoint, selected)] + if sorted(r["seed"] for r in rr) != SEEDS: + raise ValueError(f"Missing paper model pair: {group['key']}") + row = {"task": task, "expert": expert, "endpoint": endpoint, "method": selected} + for metric in METRICS: + row[metric + "_mean"] = float(np.mean([r[metric] for r in rr])) + row[metric + "_std"] = float(np.std([r[metric] for r in rr], ddof=1)) + learned, confidence = group["methods"]["learned"], group["methods"]["confidence"] + pair = f"shared_12_{expert}_{endpoint}" + prepared_val = prepare_curve(confidence_scores[pair]["val"], *outcomes[pair]["val"]) + conf_thresholds = validation_thresholds(prepared_val) + conf_curves = {"validation": fixed_curve(prepared_val, conf_thresholds), + "test": fixed_curve(prepare_curve(confidence_scores[pair]["test"], + *outcomes[pair]["test"]), conf_thresholds)} + conf_best = selected_index(conf_curves["validation"]) + if conf_best != confidence["selected_budget_index"]: + raise ValueError(f"Confidence validation-selected threshold differs: {expert}/{endpoint}") + conf_area = score25([conf_curves["validation"][i] for i in (0, 5, 10, 15, 20, 25)]) + check_close(conf_area, confidence["validation_area_0_25"], f"confidence {expert}/{endpoint}/validation area") + for split in ("validation", "test"): + reference_curve = [dict(point, rescue=point["rescued"], harm=point["harmed"]) + for point in confidence[split + "_curve"]] + verify_curve(conf_curves[split], reference_curve, f"confidence {expert}/{endpoint}/{split}") + reference_point = confidence[split + "_point"] + verify_curve([conf_curves[split][conf_best]], + [dict(reference_point, rescue=reference_point["rescued"], harm=reference_point["harmed"])], + f"confidence {expert}/{endpoint}/{split}/selected") + confidence_curve_points += len(reference_curve) + for field, label in [("metric", "metric"), ("actual_fraction", "call")]: + check_close(row[split + "_" + label + "_mean"], learned[split + "_point"][field], + f"paper {expert}/{endpoint}/{split}/{field} mean") + check_close(row[split + "_" + label + "_std"], learned[split + "_point"][field + "_std"], + f"paper {expert}/{endpoint}/{split}/{field} SD") + row["confidence_" + split + "_" + label] = conf_curves[split][conf_best][field] + for i, point in enumerate(learned[split + "_curve"]): + for field in ("metric", "actual_fraction"): + values = [replay[f"repeat_{expert}_{endpoint}_{selected}_s{s}"] + [configurations[f"repeat_{expert}_{endpoint}_{selected}_s{s}"]["ranking"]] + [split + "_curve"][i][field] for s in SEEDS] + check_close(float(np.mean(values)), point[field], f"paper mean curve {expert}/{endpoint}/{split}/{i}/{field}") + check_close(float(np.std(values, ddof=1)), point[field + "_std"], + f"paper curve SD {expert}/{endpoint}/{split}/{i}/{field}") + row["specialist_test_metric"] = group["small"]["test"] + row["VLM_test_metric"] = group["large"]["test"] + better = row["test_metric_mean"] > row["confidence_test_metric"] + TOL + fewer = row["test_call_mean"] < row["confidence_test_call"] - TOL + counts["higher_metric_than_confidence"] += int(better) + counts["fewer_calls_than_confidence"] += int(fewer) + counts["higher_metric_and_fewer_calls"] += int(better and fewer) + counts["above_both_endpoints"] += int(row["test_metric_mean"] > max(row["specialist_test_metric"], row["VLM_test_metric"]) + TOL) + main_rows.append(row) + + controls = [] + for run in runs: + cfg = configurations[run.name] + if cfg["target"] != "four_state": + continue + row = {k: cfg[k] for k in ("task", "expert", "endpoint", "method", "seed")} + for metric in METRICS[:3]: + difference = replay[run.name]["probability_difference"][metric] + ratio = replay[run.name]["log_ratio"][metric] + row[metric + "_probability_difference"] = difference + row[metric + "_log_ratio"] = ratio + row[metric + "_difference_minus_ratio"] = difference - ratio + controls.append(row) + args.output_dir.mkdir(parents=True, exist_ok=True) + write_csv(args.output_dir / "replayed_main_five_seeds.csv", main_rows) + write_csv(args.output_dir / "replayed_objectives_five_seeds.csv", objectives) + write_csv(args.output_dir / "replayed_per_seed.csv", per_seed) + write_csv(args.output_dir / "replayed_same_checkpoint_controls.csv", controls) + verification = {"status": "passed", "selected_method": selected, "seeds": SEEDS, + "primary_runs": len(primary), "same_checkpoint_controls": len(controls), + "verified_curve_points": curve_points, "main_pairs": len(main_rows), + "verified_confidence_curve_points": confidence_curve_points, + "sample_standard_deviation_ddof": 1, "metric_and_call_tolerance": TOL, + "verified_pair_objective_summary_rows": len(recorded_pairs), + "raw_logit_score_check_absolute_tolerance": 8e-7, + "counts_use_strict_improvement_above_tolerance": counts, + "selection": "validation task-balanced Score25 mean minus sample SD, before test replay", + "scope": "Saved router outputs, frozen endpoint outcomes, explicit per-event correctness labels and specialist confidence/validity. Both learned and confidence curves and validation-selected points are independently replayed; no model retraining.", + "study_history": lock["rule_amendment"]} + (args.output_dir / "verification_five_seeds.json").write_text(json.dumps(verification, indent=2) + "\n", encoding="utf-8") + print(json.dumps(verification, indent=2), flush=True) + + +if __name__ == "__main__": + main() diff --git a/releases/20260913_lr/replay_latest_results.py b/releases/20260913_lr/replay_latest_results.py new file mode 100644 index 0000000000000000000000000000000000000000..3e220e43fb7ef8c301fbc96ca323569f6ba8c386 --- /dev/null +++ b/releases/20260913_lr/replay_latest_results.py @@ -0,0 +1,87 @@ +"""Recompute Tables 1 and 3 from released per-example outputs; no GPU or training.""" +import argparse,json +from pathlib import Path +from collections import defaultdict +import numpy as np +import torch +from replay_five_seed_results import prepare_curve,point_at_count,validation_thresholds,fixed_curve + +parser=argparse.ArgumentParser() +parser.add_argument('--artifacts',type=Path,required=True) +parser.add_argument('--paper-data',type=Path,default=Path(__file__).resolve().parents[1]/'paper/data') +parser.add_argument('--output',type=Path,default=Path('verification_latest.json')) +args=parser.parse_args();torch.set_num_threads(2) +root=args.artifacts;cache={};main=defaultdict(list);areas=defaultdict(list);curve_points=0 + +def outcomes(c,split): + key=(c['expert'],c['endpoint'],split) + if key not in cache: + with np.load(root/'pairs'/('shared_12_'+c['expert']+'_'+c['endpoint'])/(split+'.npz')) as d:cache[key]={k:d[k] for k in d.files} + return cache[key] + +def prepared(score,c,split): + d=outcomes(c,split) + return prepare_curve(score,d['small'],d['large'],d['small_correct'],d['large_correct']) + +def exact(score,c,split): + p=prepared(score,c,split);count=np.floor(p['n']*np.arange(101)/100).astype(int) + ys=np.array([point_at_count(p,int(k),k/p['n'])['metric'] for k in count])*100 + return float(np.trapezoid(ys,x=count/p['n'])) + +def normalized(raw,alpha): + p=torch.from_numpy(raw).softmax(1).numpy() + return (p[:,1]-p[:,2])/(p[:,1].astype(float)+p[:,2]+2*p[:,3]+1e-8)**alpha + +for kind in ['main','four_difference','gain','three','posthoc']: + runs=sorted((root/'paper_replay_runs'/kind).iterdir());assert len(runs)==80,(kind,len(runs)) + for run in runs: + c=json.loads((run/'config.json').read_text()) + with np.load(run/'paired_logits.npz') as data:raw={k:data[k] for k in data.files} + if kind=='main': + record=json.loads((run/'evaluation.json').read_text())['methods']['normalized'] + alpha=json.loads((run/'selection.json').read_text())['alpha'] + scores={s:normalized(raw[k],alpha) for s,k in [('val','validation'),('test','test')]} + threshold=validation_thresholds(prepared(scores['val'],c,'val')) + curves={s:fixed_curve(prepared(scores[s],c,s),threshold) for s in ['val','test']} + for split in curves: + for i,point in enumerate(curves[split]): + assert abs(point['metric']*100-record[split]['score'][i])<1e-9,(run.name,split,i,'score') + assert abs(point['actual_fraction']*100-record[split]['calls'][i])<1e-9,(run.name,split,i,'calls') + curve_points+=1 + selected=record['selected_budget'];best=max(p['metric'] for p in curves['val']) + assert abs(curves['val'][selected]['metric']-best)<1e-12 + main[(c['expert'],c['endpoint'])].append(curves['test'][selected]) + elif kind=='four_difference': + alpha=max([0,.25,.5,.75,1],key=lambda a:(exact(normalized(raw['validation'],a),c,'val'),-a)) + z=torch.from_numpy(raw['test']);p=z.softmax(1).numpy() + scores={'4S_difference':p[:,1]-p[:,2],'4S_RH':raw['test'][:,1]-raw['test'][:,2], + '4S_error_ratio':(torch.logsumexp(z[:,[1,3]],1)-torch.logsumexp(z[:,[2,3]],1)).numpy(), + '4S_normalized':normalized(raw['test'],alpha)} + for method,score in scores.items():areas[(method,c['task'])].append(exact(score,c,'test')) + else: + z=raw['test'] + if kind=='three': + p=torch.from_numpy(z).softmax(1).numpy();score=p[:,0]-p[:,1] + else:score=z[:,0] + areas[(kind,c['task'])].append(exact(score,c,'test')) + print('Replayed',kind,len(runs),flush=True) +expected=json.loads((args.paper_data/'results.json').read_text())['groups'] +for g in expected: + key=tuple(g['key'][1:]);rr=main[key];assert len(rr)==5 + for field in ['metric','actual_fraction']: + assert abs(np.mean([r[field] for r in rr])-g['methods']['learned']['test_point'][field])<1e-10,(key,field) + c={'expert':key[0],'endpoint':key[1]};scores={s:np.where(outcomes(c,s)['confidence_valid'],-outcomes(c,s)['confidence'],1.) for s in ['val','test']} + thresholds=validation_thresholds(prepared(scores['val'],c,'val')) + for split,tag in [('val','validation'),('test','test')]: + curve=fixed_curve(prepared(scores[split],c,split),thresholds) + for a,b in zip(curve,g['methods']['confidence'][tag+'_curve']): + assert abs(a['metric']-b['metric'])<1e-10,(key,tag,'confidence metric') + assert abs(a['actual_fraction']-b['actual_fraction'])<1e-10 + curve_points+=1 +summary=json.loads((args.paper_data/'lr_control_summary.json').read_text());table={} +for (method,task),values in areas.items(): + assert len(values)==20,(method,task,len(values)) + value=float(np.mean(values));assert abs(value-summary[method][task]['full'])<1e-8,(method,task,value,summary[method][task]['full']) + table.setdefault(method,{})[task]=value +report={'status':'passed','main_runs':80,'table3_trained_runs':320,'main_and_confidence_curve_points':curve_points,'table3':table,'main_pairs':16} +args.output.write_text(json.dumps(report,indent=2));print(json.dumps(report),flush=True) diff --git a/releases/20260913_lr/verification_latest.json b/releases/20260913_lr/verification_latest.json new file mode 100644 index 0000000000000000000000000000000000000000..2ec285c67c014a8cc86b0c041823686931e39e05 --- /dev/null +++ b/releases/20260913_lr/verification_latest.json @@ -0,0 +1,51 @@ +{ + "status": "passed", + "main_runs": 80, + "table3_trained_runs": 320, + "main_and_confidence_curve_points": 19392, + "table3": { + "4S_difference": { + "cub": 71.30477624717177, + "grefcoco": 60.10334108982569, + "nlvr2": 86.4268866182737, + "construction": 21.68664457361428 + }, + "4S_RH": { + "cub": 67.80147679642886, + "grefcoco": 59.83696029450736, + "nlvr2": 87.36051668387572, + "construction": 21.113192599681106 + }, + "4S_error_ratio": { + "cub": 67.63433680158339, + "grefcoco": 59.9030469001769, + "nlvr2": 87.36099315635701, + "construction": 21.29756392944788 + }, + "4S_normalized": { + "cub": 71.3715854777698, + "grefcoco": 60.65894058616112, + "nlvr2": 87.3608531491271, + "construction": 21.625047131135073 + }, + "gain": { + "cub": 71.01409198260043, + "grefcoco": 59.9669645631997, + "nlvr2": 86.29461067562258, + "construction": 21.60783215955454 + }, + "three": { + "cub": 71.27395612317935, + "grefcoco": 60.12755657790998, + "nlvr2": 86.340413568538, + "construction": 21.76578337259674 + }, + "posthoc": { + "cub": 65.85980220810953, + "grefcoco": 59.367755773800184, + "nlvr2": 87.32516457516525, + "construction": 21.23728513819176 + } + }, + "main_pairs": 16 +} \ No newline at end of file diff --git a/releases/20260913_lr/weight_reload_verification.json b/releases/20260913_lr/weight_reload_verification.json new file mode 100644 index 0000000000000000000000000000000000000000..82eeab87c3e0906cdee354f2d0bc9722b1ae7c40 --- /dev/null +++ b/releases/20260913_lr/weight_reload_verification.json @@ -0,0 +1,327 @@ +{ + "status": "passed", + "runs": 80, + "examples_per_run": 128, + "checks": [ + { + "run": "full_cub_MiniCPM_s2026", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_cub_MiniCPM_s2027", + "max_abs_error": 4.291534423828125e-06 + }, + { + "run": "full_cub_MiniCPM_s2028", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_cub_MiniCPM_s2029", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_cub_MiniCPM_s2030", + "max_abs_error": 3.337860107421875e-06 + }, + { + "run": "full_cub_Qwen3.8_s2026", + "max_abs_error": 4.291534423828125e-06 + }, + { + "run": "full_cub_Qwen3.8_s2027", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_cub_Qwen3.8_s2028", + "max_abs_error": 4.291534423828125e-06 + }, + { + "run": "full_cub_Qwen3.8_s2029", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_cub_Qwen3.8_s2030", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_glsim_MiniCPM_s2026", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_glsim_MiniCPM_s2027", + "max_abs_error": 3.337860107421875e-06 + }, + { + "run": "full_glsim_MiniCPM_s2028", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_glsim_MiniCPM_s2029", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_glsim_MiniCPM_s2030", + "max_abs_error": 4.291534423828125e-06 + }, + { + "run": "full_glsim_Qwen3.8_s2026", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_glsim_Qwen3.8_s2027", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_glsim_Qwen3.8_s2028", + "max_abs_error": 4.291534423828125e-06 + }, + { + "run": "full_glsim_Qwen3.8_s2029", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_glsim_Qwen3.8_s2030", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_groundingdino_MiniCPM_s2026", + "max_abs_error": 8.58306884765625e-06 + }, + { + "run": "full_groundingdino_MiniCPM_s2027", + "max_abs_error": 9.5367431640625e-06 + }, + { + "run": "full_groundingdino_MiniCPM_s2028", + "max_abs_error": 8.58306884765625e-06 + }, + { + "run": "full_groundingdino_MiniCPM_s2029", + "max_abs_error": 6.67572021484375e-06 + }, + { + "run": "full_groundingdino_MiniCPM_s2030", + "max_abs_error": 5.7220458984375e-06 + }, + { + "run": "full_groundingdino_Qwen3.8_s2026", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_groundingdino_Qwen3.8_s2027", + "max_abs_error": 6.67572021484375e-06 + }, + { + "run": "full_groundingdino_Qwen3.8_s2028", + "max_abs_error": 4.291534423828125e-06 + }, + { + "run": "full_groundingdino_Qwen3.8_s2029", + "max_abs_error": 8.106231689453125e-06 + }, + { + "run": "full_groundingdino_Qwen3.8_s2030", + "max_abs_error": 6.67572021484375e-06 + }, + { + "run": "full_instancevg_MiniCPM_s2026", + "max_abs_error": 5.7220458984375e-06 + }, + { + "run": "full_instancevg_MiniCPM_s2027", + "max_abs_error": 6.67572021484375e-06 + }, + { + "run": "full_instancevg_MiniCPM_s2028", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_instancevg_MiniCPM_s2029", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_instancevg_MiniCPM_s2030", + "max_abs_error": 9.5367431640625e-06 + }, + { + "run": "full_instancevg_Qwen3.8_s2026", + "max_abs_error": 6.67572021484375e-06 + }, + { + "run": "full_instancevg_Qwen3.8_s2027", + "max_abs_error": 6.67572021484375e-06 + }, + { + "run": "full_instancevg_Qwen3.8_s2028", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_instancevg_Qwen3.8_s2029", + "max_abs_error": 5.7220458984375e-06 + }, + { + "run": "full_instancevg_Qwen3.8_s2030", + "max_abs_error": 7.62939453125e-06 + }, + { + "run": "full_nlvr2_MiniCPM_s2026", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_nlvr2_MiniCPM_s2027", + "max_abs_error": 2.384185791015625e-06 + }, + { + "run": "full_nlvr2_MiniCPM_s2028", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_nlvr2_MiniCPM_s2029", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_nlvr2_MiniCPM_s2030", + "max_abs_error": 3.337860107421875e-06 + }, + { + "run": "full_nlvr2_Qwen3.8_s2026", + "max_abs_error": 2.384185791015625e-06 + }, + { + "run": "full_nlvr2_Qwen3.8_s2027", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_nlvr2_Qwen3.8_s2028", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_nlvr2_Qwen3.8_s2029", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_nlvr2_Qwen3.8_s2030", + "max_abs_error": 2.384185791015625e-06 + }, + { + "run": "full_rtdetr_x_fp32_MiniCPM_s2026", + "max_abs_error": 1.430511474609375e-06 + }, + { + "run": "full_rtdetr_x_fp32_MiniCPM_s2027", + "max_abs_error": 1.430511474609375e-06 + }, + { + "run": "full_rtdetr_x_fp32_MiniCPM_s2028", + "max_abs_error": 1.6689300537109375e-06 + }, + { + "run": "full_rtdetr_x_fp32_MiniCPM_s2029", + "max_abs_error": 1.6689300537109375e-06 + }, + { + "run": "full_rtdetr_x_fp32_MiniCPM_s2030", + "max_abs_error": 1.430511474609375e-06 + }, + { + "run": "full_rtdetr_x_fp32_Qwen3.8_s2026", + "max_abs_error": 2.1457672119140625e-06 + }, + { + "run": "full_rtdetr_x_fp32_Qwen3.8_s2027", + "max_abs_error": 1.430511474609375e-06 + }, + { + "run": "full_rtdetr_x_fp32_Qwen3.8_s2028", + "max_abs_error": 1.6689300537109375e-06 + }, + { + "run": "full_rtdetr_x_fp32_Qwen3.8_s2029", + "max_abs_error": 1.6689300537109375e-06 + }, + { + "run": "full_rtdetr_x_fp32_Qwen3.8_s2030", + "max_abs_error": 1.6689300537109375e-06 + }, + { + "run": "full_vilt_MiniCPM_s2026", + "max_abs_error": 3.337860107421875e-06 + }, + { + "run": "full_vilt_MiniCPM_s2027", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_vilt_MiniCPM_s2028", + "max_abs_error": 5.7220458984375e-06 + }, + { + "run": "full_vilt_MiniCPM_s2029", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_vilt_MiniCPM_s2030", + "max_abs_error": 2.1457672119140625e-06 + }, + { + "run": "full_vilt_Qwen3.8_s2026", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_vilt_Qwen3.8_s2027", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_vilt_Qwen3.8_s2028", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_vilt_Qwen3.8_s2029", + "max_abs_error": 2.384185791015625e-06 + }, + { + "run": "full_vilt_Qwen3.8_s2030", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_yolo26x_MiniCPM_s2026", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_yolo26x_MiniCPM_s2027", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_yolo26x_MiniCPM_s2028", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_yolo26x_MiniCPM_s2029", + "max_abs_error": 2.86102294921875e-06 + }, + { + "run": "full_yolo26x_MiniCPM_s2030", + "max_abs_error": 4.291534423828125e-06 + }, + { + "run": "full_yolo26x_Qwen3.8_s2026", + "max_abs_error": 4.76837158203125e-06 + }, + { + "run": "full_yolo26x_Qwen3.8_s2027", + "max_abs_error": 2.384185791015625e-06 + }, + { + "run": "full_yolo26x_Qwen3.8_s2028", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_yolo26x_Qwen3.8_s2029", + "max_abs_error": 3.814697265625e-06 + }, + { + "run": "full_yolo26x_Qwen3.8_s2030", + "max_abs_error": 5.7220458984375e-06 + } + ] +} \ No newline at end of file